2024-04-17T10:23:41.130 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-17T10:23:41.130 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/neesingh-2024-04-17_09:52:10-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7660174 --description fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/scrub} --name neesingh-2024-04-17_09:52:10-fs:functional-wip-neesingh-testing-240416-distro-default-smithi --owner scheduled_neesingh@teuthology -- /home/teuthworker/archive/neesingh-2024-04-17_09:52:10-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7660174/orig.config.yaml 2024-04-17T10:23:41.172 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-17T10:23:41.253 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2024-04-17_09:52:10-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7660174 branch: wip-neesingh-testing-240416 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/scrub} email: neesingh@redhat.com first_in_suite: false job_id: '7660174' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2024-04-17_09:52:10-fs:functional-wip-neesingh-testing-240416-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-neesingh-testing-240416 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds cache max size: 1000 mds debug frag: true mds debug scatterstat: true mds log max segments: 1 mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - Replacing daemon mds - Scrub error on inode - Behind on trimming - Metadata damage detected - bad backtrace on inode - overall HEALTH_ - \(MDS_TRIM\) sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-neesingh-testing-240416 sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec owner: scheduled_neesingh@teuthology priority: 10 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 558 sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-neesingh-testing-240416 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi083.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHr9+q9Te3W+pbDiE+Htwdkau6E+Sf3z9uafMwd2gkpuElBki7FIY7+wmf/mUPwhJNOmtGTHAuXdIn30VmzrvHo= smithi167.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBECZPeOI70yBWy+c5CdXEWaPmEAYKBn0CSit7jaHx6+l7Dc3NPw0uERZI1qUx7JfM4v9YWgerWoL4WcglL5TIyM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_scrub_checks - tasks.cephfs.test_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 6c637841c215537a4502385240412f1966e0faab timestamp: 2024-04-17_09:52:10 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-17T10:23:41.253 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/qa; will attempt to use it 2024-04-17T10:23:41.254 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/qa/tasks 2024-04-17T10:23:41.254 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-17T10:23:41.256 INFO:teuthology.task.internal:Checking packages... 2024-04-17T10:23:41.280 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec' 2024-04-17T10:23:41.280 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-17T10:23:41.280 INFO:teuthology.packaging:ref: None 2024-04-17T10:23:41.280 INFO:teuthology.packaging:tag: None 2024-04-17T10:23:41.280 INFO:teuthology.packaging:branch: wip-neesingh-testing-240416 2024-04-17T10:23:41.280 INFO:teuthology.packaging:sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-17T10:23:41.281 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-neesingh-testing-240416 2024-04-17T10:23:41.510 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1628-g97ae6cc9-1jammy 2024-04-17T10:23:41.512 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-17T10:23:41.518 INFO:teuthology.task.internal:no buildpackages task found 2024-04-17T10:23:41.519 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-17T10:23:41.554 INFO:teuthology.task.internal:Saving configuration 2024-04-17T10:23:41.571 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-17T10:23:41.578 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-17T10:23:41.599 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-17_09:52:10-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7660174', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-17 10:17:26.101311', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIwz3WayDx/fODpRRAqx4RxjM/0hsLVYPef3AXBfr27y'} 2024-04-17T10:23:41.617 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-17_09:52:10-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7660174', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-17 10:17:26.100029', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHkcGfT5twCwXyz0XHOFBJ1cOK3jgaoeFqtihgqlYCtr'} 2024-04-17T10:23:41.617 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-17T10:23:41.622 INFO:teuthology.task.internal:roles: ubuntu@smithi083.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-17T10:23:41.622 INFO:teuthology.task.internal:roles: ubuntu@smithi167.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-17T10:23:41.623 INFO:teuthology.run_tasks:Running task console_log... 2024-04-17T10:23:41.709 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fcb7eaaa4c0>, signals=[15]) 2024-04-17T10:23:41.710 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-17T10:23:41.716 INFO:teuthology.task.internal:Opening connections... 2024-04-17T10:23:41.716 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi083.front.sepia.ceph.com 2024-04-17T10:23:41.718 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:23:41.796 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi167.front.sepia.ceph.com 2024-04-17T10:23:41.798 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:23:41.875 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-17T10:23:41.881 DEBUG:teuthology.orchestra.run.smithi083:> uname -m 2024-04-17T10:23:41.886 INFO:teuthology.orchestra.run.smithi083.stdout:x86_64 2024-04-17T10:23:41.887 DEBUG:teuthology.orchestra.run.smithi083:> cat /etc/os-release 2024-04-17T10:23:41.934 INFO:teuthology.orchestra.run.smithi083.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-17T10:23:41.934 INFO:teuthology.orchestra.run.smithi083.stdout:NAME="Ubuntu" 2024-04-17T10:23:41.934 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_ID="22.04" 2024-04-17T10:23:41.934 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-17T10:23:41.934 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_CODENAME=jammy 2024-04-17T10:23:41.934 INFO:teuthology.orchestra.run.smithi083.stdout:ID=ubuntu 2024-04-17T10:23:41.934 INFO:teuthology.orchestra.run.smithi083.stdout:ID_LIKE=debian 2024-04-17T10:23:41.934 INFO:teuthology.orchestra.run.smithi083.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-17T10:23:41.934 INFO:teuthology.orchestra.run.smithi083.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-17T10:23:41.934 INFO:teuthology.orchestra.run.smithi083.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-17T10:23:41.935 INFO:teuthology.orchestra.run.smithi083.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-17T10:23:41.935 INFO:teuthology.orchestra.run.smithi083.stdout:UBUNTU_CODENAME=jammy 2024-04-17T10:23:41.935 INFO:teuthology.lock.ops:Updating smithi083.front.sepia.ceph.com on lock server 2024-04-17T10:23:41.960 DEBUG:teuthology.orchestra.run.smithi167:> uname -m 2024-04-17T10:23:41.965 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64 2024-04-17T10:23:41.965 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release 2024-04-17T10:23:42.010 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-17T10:23:42.010 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="Ubuntu" 2024-04-17T10:23:42.010 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="22.04" 2024-04-17T10:23:42.010 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-17T10:23:42.011 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_CODENAME=jammy 2024-04-17T10:23:42.011 INFO:teuthology.orchestra.run.smithi167.stdout:ID=ubuntu 2024-04-17T10:23:42.011 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE=debian 2024-04-17T10:23:42.011 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-17T10:23:42.011 INFO:teuthology.orchestra.run.smithi167.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-17T10:23:42.011 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-17T10:23:42.011 INFO:teuthology.orchestra.run.smithi167.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-17T10:23:42.011 INFO:teuthology.orchestra.run.smithi167.stdout:UBUNTU_CODENAME=jammy 2024-04-17T10:23:42.011 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2024-04-17T10:23:42.035 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-17T10:23:42.045 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-17T10:23:42.080 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-17T10:23:42.081 DEBUG:teuthology.orchestra.run.smithi083:> test '!' -e /home/ubuntu/cephtest 2024-04-17T10:23:42.083 DEBUG:teuthology.orchestra.run.smithi167:> test '!' -e /home/ubuntu/cephtest 2024-04-17T10:23:42.088 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-17T10:23:42.104 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-17T10:23:42.105 DEBUG:teuthology.orchestra.run.smithi083:> test -z $(ls -A /var/lib/ceph) 2024-04-17T10:23:42.132 DEBUG:teuthology.orchestra.run.smithi167:> test -z $(ls -A /var/lib/ceph) 2024-04-17T10:23:42.139 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-17T10:23:42.187 INFO:teuthology.run_tasks:Running task kernel... 2024-04-17T10:23:42.202 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-17T10:23:42.202 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-17T10:23:42.202 DEBUG:teuthology.orchestra.run.smithi083:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-17T10:23:42.203 DEBUG:teuthology.orchestra.run.smithi167:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-17T10:23:42.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:23:42.207 DEBUG:teuthology.orchestra.run.smithi083:> uname -r 2024-04-17T10:23:42.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:23:42.209 DEBUG:teuthology.orchestra.run.smithi167:> uname -r 2024-04-17T10:23:42.254 INFO:teuthology.orchestra.run.smithi083.stdout:5.15.0-56-generic 2024-04-17T10:23:42.255 INFO:teuthology.task.kernel:Running kernel on smithi083: 5.15.0-56-generic 2024-04-17T10:23:42.255 DEBUG:teuthology.orchestra.run.smithi083:> sudo apt-get clean 2024-04-17T10:23:42.256 INFO:teuthology.orchestra.run.smithi167.stdout:5.15.0-56-generic 2024-04-17T10:23:42.256 INFO:teuthology.task.kernel:Running kernel on smithi167: 5.15.0-56-generic 2024-04-17T10:23:42.256 DEBUG:teuthology.orchestra.run.smithi167:> sudo apt-get clean 2024-04-17T10:23:42.403 DEBUG:teuthology.orchestra.run.smithi083:> sudo apt-get update 2024-04-17T10:23:42.497 DEBUG:teuthology.orchestra.run.smithi167:> sudo apt-get update 2024-04-17T10:23:42.560 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-17T10:23:42.560 INFO:teuthology.orchestra.run.smithi083.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-17T10:23:42.563 INFO:teuthology.orchestra.run.smithi083.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-17T10:23:42.659 INFO:teuthology.orchestra.run.smithi083.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-17T10:23:42.775 INFO:teuthology.orchestra.run.smithi167.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-17T10:23:42.777 INFO:teuthology.orchestra.run.smithi167.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-17T10:23:42.796 INFO:teuthology.orchestra.run.smithi083.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,346 kB] 2024-04-17T10:23:42.825 INFO:teuthology.orchestra.run.smithi167.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-17T10:23:42.866 INFO:teuthology.orchestra.run.smithi083.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,562 kB] 2024-04-17T10:23:42.918 INFO:teuthology.orchestra.run.smithi083.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [445 kB] 2024-04-17T10:23:42.927 INFO:teuthology.orchestra.run.smithi083.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [237 kB] 2024-04-17T10:23:42.933 INFO:teuthology.orchestra.run.smithi083.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-17T10:23:42.933 INFO:teuthology.orchestra.run.smithi083.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [34.1 kB] 2024-04-17T10:23:42.934 INFO:teuthology.orchestra.run.smithi083.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,670 kB] 2024-04-17T10:23:42.974 INFO:teuthology.orchestra.run.smithi083.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [280 kB] 2024-04-17T10:23:42.983 INFO:teuthology.orchestra.run.smithi083.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-17T10:23:42.983 INFO:teuthology.orchestra.run.smithi083.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [600 kB] 2024-04-17T10:23:43.003 INFO:teuthology.orchestra.run.smithi083.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [852 kB] 2024-04-17T10:23:43.020 INFO:teuthology.orchestra.run.smithi083.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-17T10:23:43.022 INFO:teuthology.orchestra.run.smithi083.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-17T10:23:43.024 INFO:teuthology.orchestra.run.smithi083.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-17T10:23:43.037 INFO:teuthology.orchestra.run.smithi083.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-17T10:23:43.038 INFO:teuthology.orchestra.run.smithi083.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [611 kB] 2024-04-17T10:23:43.038 INFO:teuthology.orchestra.run.smithi083.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-17T10:23:43.045 INFO:teuthology.orchestra.run.smithi083.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [298 kB] 2024-04-17T10:23:43.051 INFO:teuthology.orchestra.run.smithi083.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-17T10:23:43.051 INFO:teuthology.orchestra.run.smithi083.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,702 kB] 2024-04-17T10:23:43.066 INFO:teuthology.orchestra.run.smithi083.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-17T10:23:43.105 INFO:teuthology.orchestra.run.smithi083.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.7 kB] 2024-04-17T10:23:43.106 INFO:teuthology.orchestra.run.smithi083.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [286 kB] 2024-04-17T10:23:43.109 INFO:teuthology.orchestra.run.smithi083.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-17T10:23:43.109 INFO:teuthology.orchestra.run.smithi083.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,075 kB] 2024-04-17T10:23:43.128 INFO:teuthology.orchestra.run.smithi083.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [699 kB] 2024-04-17T10:23:43.144 INFO:teuthology.orchestra.run.smithi083.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [245 kB] 2024-04-17T10:23:43.148 INFO:teuthology.orchestra.run.smithi083.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-17T10:23:43.149 INFO:teuthology.orchestra.run.smithi083.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [49.7 kB] 2024-04-17T10:23:43.150 INFO:teuthology.orchestra.run.smithi083.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [6,028 B] 2024-04-17T10:23:43.156 INFO:teuthology.orchestra.run.smithi083.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [12.1 kB] 2024-04-17T10:23:43.157 INFO:teuthology.orchestra.run.smithi083.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-17T10:23:43.157 INFO:teuthology.orchestra.run.smithi083.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-17T10:23:43.157 INFO:teuthology.orchestra.run.smithi083.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-04-17T10:23:43.175 INFO:teuthology.orchestra.run.smithi083.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-17T10:23:43.176 INFO:teuthology.orchestra.run.smithi083.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-17T10:23:43.176 INFO:teuthology.orchestra.run.smithi083.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-04-17T10:23:43.177 INFO:teuthology.orchestra.run.smithi083.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-04-17T10:23:43.177 INFO:teuthology.orchestra.run.smithi083.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-17T10:23:43.180 INFO:teuthology.orchestra.run.smithi167.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [445 kB] 2024-04-17T10:23:43.181 INFO:teuthology.orchestra.run.smithi083.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-17T10:23:43.187 INFO:teuthology.orchestra.run.smithi167.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-17T10:23:43.327 INFO:teuthology.orchestra.run.smithi167.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,562 kB] 2024-04-17T10:23:43.471 INFO:teuthology.orchestra.run.smithi167.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,346 kB] 2024-04-17T10:23:43.635 INFO:teuthology.orchestra.run.smithi167.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [611 kB] 2024-04-17T10:23:43.637 INFO:teuthology.orchestra.run.smithi167.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [237 kB] 2024-04-17T10:23:43.649 INFO:teuthology.orchestra.run.smithi167.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-17T10:23:43.650 INFO:teuthology.orchestra.run.smithi167.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [34.1 kB] 2024-04-17T10:23:43.652 INFO:teuthology.orchestra.run.smithi167.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,670 kB] 2024-04-17T10:23:43.673 INFO:teuthology.orchestra.run.smithi167.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [298 kB] 2024-04-17T10:23:43.688 INFO:teuthology.orchestra.run.smithi167.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-17T10:23:43.689 INFO:teuthology.orchestra.run.smithi167.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.7 kB] 2024-04-17T10:23:43.691 INFO:teuthology.orchestra.run.smithi167.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,702 kB] 2024-04-17T10:23:43.752 INFO:teuthology.orchestra.run.smithi167.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [280 kB] 2024-04-17T10:23:43.752 INFO:teuthology.orchestra.run.smithi167.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [286 kB] 2024-04-17T10:23:43.761 INFO:teuthology.orchestra.run.smithi167.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-17T10:23:43.761 INFO:teuthology.orchestra.run.smithi167.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-17T10:23:43.761 INFO:teuthology.orchestra.run.smithi167.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [600 kB] 2024-04-17T10:23:43.771 INFO:teuthology.orchestra.run.smithi167.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [852 kB] 2024-04-17T10:23:43.926 INFO:teuthology.orchestra.run.smithi167.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,075 kB] 2024-04-17T10:23:43.948 INFO:teuthology.orchestra.run.smithi167.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-17T10:23:44.358 INFO:teuthology.orchestra.run.smithi167.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-17T10:23:44.363 INFO:teuthology.orchestra.run.smithi167.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-17T10:23:44.364 INFO:teuthology.orchestra.run.smithi167.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-17T10:23:44.378 INFO:teuthology.orchestra.run.smithi167.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-17T10:23:44.381 INFO:teuthology.orchestra.run.smithi167.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-17T10:23:44.556 INFO:teuthology.orchestra.run.smithi167.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [699 kB] 2024-04-17T10:23:44.593 INFO:teuthology.orchestra.run.smithi167.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [245 kB] 2024-04-17T10:23:44.606 INFO:teuthology.orchestra.run.smithi167.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-17T10:23:44.607 INFO:teuthology.orchestra.run.smithi167.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [6,028 B] 2024-04-17T10:23:44.608 INFO:teuthology.orchestra.run.smithi167.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [49.7 kB] 2024-04-17T10:23:44.610 INFO:teuthology.orchestra.run.smithi167.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [12.1 kB] 2024-04-17T10:23:44.611 INFO:teuthology.orchestra.run.smithi167.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-17T10:23:44.611 INFO:teuthology.orchestra.run.smithi167.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-04-17T10:23:44.638 INFO:teuthology.orchestra.run.smithi167.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-17T10:23:44.641 INFO:teuthology.orchestra.run.smithi167.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-17T10:23:44.805 INFO:teuthology.orchestra.run.smithi167.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-17T10:23:44.888 INFO:teuthology.orchestra.run.smithi167.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-04-17T10:23:44.969 INFO:teuthology.orchestra.run.smithi167.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-04-17T10:23:45.050 INFO:teuthology.orchestra.run.smithi167.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-17T10:23:45.081 INFO:teuthology.orchestra.run.smithi167.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-17T10:23:48.875 INFO:teuthology.orchestra.run.smithi083.stdout:Fetched 12.9 MB in 2s (7,157 kB/s) 2024-04-17T10:23:49.406 INFO:teuthology.orchestra.run.smithi167.stdout:Fetched 12.9 MB in 3s (5,106 kB/s) 2024-04-17T10:23:50.268 INFO:teuthology.orchestra.run.smithi083.stdout:Reading package lists... 2024-04-17T10:23:50.286 DEBUG:teuthology.orchestra.run.smithi083:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-17T10:23:50.348 INFO:teuthology.orchestra.run.smithi083.stdout:Reading package lists... 2024-04-17T10:23:50.526 INFO:teuthology.orchestra.run.smithi083.stdout:Building dependency tree... 2024-04-17T10:23:50.527 INFO:teuthology.orchestra.run.smithi083.stdout:Reading state information... 2024-04-17T10:23:50.682 INFO:teuthology.orchestra.run.smithi083.stdout:The following packages were automatically installed and are no longer required: 2024-04-17T10:23:50.682 INFO:teuthology.orchestra.run.smithi083.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-17T10:23:50.682 INFO:teuthology.orchestra.run.smithi083.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-17T10:23:50.682 INFO:teuthology.orchestra.run.smithi083.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-17T10:23:50.682 INFO:teuthology.orchestra.run.smithi083.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-17T10:23:50.683 INFO:teuthology.orchestra.run.smithi083.stdout: python2.7-minimal 2024-04-17T10:23:50.683 INFO:teuthology.orchestra.run.smithi083.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-17T10:23:50.683 INFO:teuthology.orchestra.run.smithi083.stdout:The following additional packages will be installed: 2024-04-17T10:23:50.684 INFO:teuthology.orchestra.run.smithi083.stdout: linux-generic linux-headers-5.15.0-102 linux-headers-5.15.0-102-generic 2024-04-17T10:23:50.684 INFO:teuthology.orchestra.run.smithi083.stdout: linux-headers-generic linux-image-5.15.0-102-generic 2024-04-17T10:23:50.684 INFO:teuthology.orchestra.run.smithi083.stdout: linux-modules-5.15.0-102-generic linux-modules-extra-5.15.0-102-generic 2024-04-17T10:23:50.685 INFO:teuthology.orchestra.run.smithi083.stdout:Suggested packages: 2024-04-17T10:23:50.685 INFO:teuthology.orchestra.run.smithi083.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-17T10:23:50.685 INFO:teuthology.orchestra.run.smithi083.stdout:Recommended packages: 2024-04-17T10:23:50.685 INFO:teuthology.orchestra.run.smithi083.stdout: thermald 2024-04-17T10:23:50.721 INFO:teuthology.orchestra.run.smithi167.stdout:Reading package lists... 2024-04-17T10:23:50.739 DEBUG:teuthology.orchestra.run.smithi167:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-17T10:23:50.743 INFO:teuthology.orchestra.run.smithi083.stdout:The following NEW packages will be installed: 2024-04-17T10:23:50.743 INFO:teuthology.orchestra.run.smithi083.stdout: linux-headers-5.15.0-102 linux-headers-5.15.0-102-generic 2024-04-17T10:23:50.743 INFO:teuthology.orchestra.run.smithi083.stdout: linux-image-5.15.0-102-generic linux-modules-5.15.0-102-generic 2024-04-17T10:23:50.744 INFO:teuthology.orchestra.run.smithi083.stdout: linux-modules-extra-5.15.0-102-generic 2024-04-17T10:23:50.745 INFO:teuthology.orchestra.run.smithi083.stdout:The following packages will be upgraded: 2024-04-17T10:23:50.745 INFO:teuthology.orchestra.run.smithi083.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-17T10:23:50.792 INFO:teuthology.orchestra.run.smithi167.stdout:Reading package lists... 2024-04-17T10:23:50.926 INFO:teuthology.orchestra.run.smithi083.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-17T10:23:50.926 INFO:teuthology.orchestra.run.smithi083.stdout:Need to get 113 MB of archives. 2024-04-17T10:23:50.926 INFO:teuthology.orchestra.run.smithi083.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-17T10:23:50.926 INFO:teuthology.orchestra.run.smithi083.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-102-generic amd64 5.15.0-102.112 [22.7 MB] 2024-04-17T10:23:50.989 INFO:teuthology.orchestra.run.smithi167.stdout:Building dependency tree... 2024-04-17T10:23:50.989 INFO:teuthology.orchestra.run.smithi167.stdout:Reading state information... 2024-04-17T10:23:51.153 INFO:teuthology.orchestra.run.smithi167.stdout:The following packages were automatically installed and are no longer required: 2024-04-17T10:23:51.153 INFO:teuthology.orchestra.run.smithi167.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-17T10:23:51.153 INFO:teuthology.orchestra.run.smithi167.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-17T10:23:51.154 INFO:teuthology.orchestra.run.smithi167.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-17T10:23:51.154 INFO:teuthology.orchestra.run.smithi167.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-17T10:23:51.154 INFO:teuthology.orchestra.run.smithi167.stdout: python2.7-minimal 2024-04-17T10:23:51.154 INFO:teuthology.orchestra.run.smithi167.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-17T10:23:51.154 INFO:teuthology.orchestra.run.smithi167.stdout:The following additional packages will be installed: 2024-04-17T10:23:51.155 INFO:teuthology.orchestra.run.smithi167.stdout: linux-generic linux-headers-5.15.0-102 linux-headers-5.15.0-102-generic 2024-04-17T10:23:51.155 INFO:teuthology.orchestra.run.smithi167.stdout: linux-headers-generic linux-image-5.15.0-102-generic 2024-04-17T10:23:51.155 INFO:teuthology.orchestra.run.smithi167.stdout: linux-modules-5.15.0-102-generic linux-modules-extra-5.15.0-102-generic 2024-04-17T10:23:51.156 INFO:teuthology.orchestra.run.smithi167.stdout:Suggested packages: 2024-04-17T10:23:51.156 INFO:teuthology.orchestra.run.smithi167.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-17T10:23:51.156 INFO:teuthology.orchestra.run.smithi167.stdout:Recommended packages: 2024-04-17T10:23:51.156 INFO:teuthology.orchestra.run.smithi167.stdout: thermald 2024-04-17T10:23:51.210 INFO:teuthology.orchestra.run.smithi167.stdout:The following NEW packages will be installed: 2024-04-17T10:23:51.210 INFO:teuthology.orchestra.run.smithi167.stdout: linux-headers-5.15.0-102 linux-headers-5.15.0-102-generic 2024-04-17T10:23:51.210 INFO:teuthology.orchestra.run.smithi167.stdout: linux-image-5.15.0-102-generic linux-modules-5.15.0-102-generic 2024-04-17T10:23:51.211 INFO:teuthology.orchestra.run.smithi167.stdout: linux-modules-extra-5.15.0-102-generic 2024-04-17T10:23:51.211 INFO:teuthology.orchestra.run.smithi167.stdout:The following packages will be upgraded: 2024-04-17T10:23:51.212 INFO:teuthology.orchestra.run.smithi167.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-17T10:23:51.333 INFO:teuthology.orchestra.run.smithi167.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-17T10:23:51.333 INFO:teuthology.orchestra.run.smithi167.stdout:Need to get 113 MB of archives. 2024-04-17T10:23:51.333 INFO:teuthology.orchestra.run.smithi167.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-17T10:23:51.333 INFO:teuthology.orchestra.run.smithi167.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-102-generic amd64 5.15.0-102.112 [22.7 MB] 2024-04-17T10:23:52.333 INFO:teuthology.orchestra.run.smithi083.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-102-generic amd64 5.15.0-102.112 [11.5 MB] 2024-04-17T10:23:52.680 INFO:teuthology.orchestra.run.smithi083.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-102-generic amd64 5.15.0-102.112 [63.9 MB] 2024-04-17T10:23:52.752 INFO:teuthology.orchestra.run.smithi167.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-102-generic amd64 5.15.0-102.112 [11.5 MB] 2024-04-17T10:23:53.054 INFO:teuthology.orchestra.run.smithi167.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-102-generic amd64 5.15.0-102.112 [63.9 MB] 2024-04-17T10:23:54.608 INFO:teuthology.orchestra.run.smithi167.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.102.99 [1,700 B] 2024-04-17T10:23:54.608 INFO:teuthology.orchestra.run.smithi167.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.102.99 [2,512 B] 2024-04-17T10:23:54.608 INFO:teuthology.orchestra.run.smithi167.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-102 all 5.15.0-102.112 [12.4 MB] 2024-04-17T10:23:54.862 INFO:teuthology.orchestra.run.smithi083.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.102.99 [1,700 B] 2024-04-17T10:23:54.863 INFO:teuthology.orchestra.run.smithi083.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.102.99 [2,512 B] 2024-04-17T10:23:54.863 INFO:teuthology.orchestra.run.smithi083.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-102 all 5.15.0-102.112 [12.4 MB] 2024-04-17T10:23:54.905 INFO:teuthology.orchestra.run.smithi167.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-102-generic amd64 5.15.0-102.112 [2,877 kB] 2024-04-17T10:23:54.952 INFO:teuthology.orchestra.run.smithi167.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.102.99 [2,360 B] 2024-04-17T10:23:55.355 INFO:teuthology.orchestra.run.smithi167.stdout:Fetched 113 MB in 4s (30.4 MB/s) 2024-04-17T10:23:55.412 INFO:teuthology.orchestra.run.smithi083.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-102-generic amd64 5.15.0-102.112 [2,877 kB] 2024-04-17T10:23:55.469 INFO:teuthology.orchestra.run.smithi083.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.102.99 [2,360 B] 2024-04-17T10:23:55.524 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package linux-modules-5.15.0-102-generic. 2024-04-17T10:23:55.903 INFO:teuthology.orchestra.run.smithi083.stdout:Fetched 113 MB in 5s (24.0 MB/s) 2024-04-17T10:23:56.069 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package linux-modules-5.15.0-102-generic. 2024-04-17T10:23:57.573 INFO:teuthology.orchestra.run.smithi167.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-17T10:23:57.577 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../0-linux-modules-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-17T10:23:57.663 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking linux-modules-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T10:23:58.038 INFO:teuthology.orchestra.run.smithi083.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-17T10:23:58.042 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../0-linux-modules-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-17T10:23:58.108 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-modules-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T10:23:59.682 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package linux-image-5.15.0-102-generic. 2024-04-17T10:23:59.703 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../1-linux-image-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-17T10:23:59.772 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T10:24:00.076 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package linux-image-5.15.0-102-generic. 2024-04-17T10:24:00.098 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../1-linux-image-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-17T10:24:00.184 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T10:24:00.211 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-102-generic. 2024-04-17T10:24:00.231 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-17T10:24:00.269 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking linux-modules-extra-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T10:24:00.571 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-102-generic. 2024-04-17T10:24:00.591 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-17T10:24:00.687 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-modules-extra-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T10:24:06.446 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../3-linux-generic_5.15.0.102.99_amd64.deb ... 2024-04-17T10:24:06.549 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking linux-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-17T10:24:06.739 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../3-linux-generic_5.15.0.102.99_amd64.deb ... 2024-04-17T10:24:06.867 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-17T10:24:06.906 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.102.99_amd64.deb ... 2024-04-17T10:24:07.027 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking linux-image-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-17T10:24:07.183 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.102.99_amd64.deb ... 2024-04-17T10:24:07.304 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-image-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-17T10:24:07.305 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package linux-headers-5.15.0-102. 2024-04-17T10:24:07.327 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../5-linux-headers-5.15.0-102_5.15.0-102.112_all.deb ... 2024-04-17T10:24:07.371 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking linux-headers-5.15.0-102 (5.15.0-102.112) ... 2024-04-17T10:24:07.599 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package linux-headers-5.15.0-102. 2024-04-17T10:24:07.621 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../5-linux-headers-5.15.0-102_5.15.0-102.112_all.deb ... 2024-04-17T10:24:07.665 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-headers-5.15.0-102 (5.15.0-102.112) ... 2024-04-17T10:24:12.798 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package linux-headers-5.15.0-102-generic. 2024-04-17T10:24:12.834 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../6-linux-headers-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-17T10:24:12.889 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking linux-headers-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T10:24:13.710 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package linux-headers-5.15.0-102-generic. 2024-04-17T10:24:13.747 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../6-linux-headers-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-17T10:24:13.785 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-headers-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T10:24:15.398 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.102.99_amd64.deb ... 2024-04-17T10:24:15.528 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking linux-headers-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-17T10:24:15.853 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up linux-headers-5.15.0-102 (5.15.0-102.112) ... 2024-04-17T10:24:15.975 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up linux-headers-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T10:24:16.092 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up linux-headers-generic (5.15.0.102.99) ... 2024-04-17T10:24:16.221 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T10:24:16.276 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.102.99_amd64.deb ... 2024-04-17T10:24:16.399 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-headers-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-17T10:24:16.749 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-headers-5.15.0-102 (5.15.0-102.112) ... 2024-04-17T10:24:16.879 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-headers-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T10:24:17.005 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-headers-generic (5.15.0.102.99) ... 2024-04-17T10:24:17.133 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T10:24:17.533 INFO:teuthology.orchestra.run.smithi167.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-17T10:24:17.533 INFO:teuthology.orchestra.run.smithi167.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-17T10:24:17.533 INFO:teuthology.orchestra.run.smithi167.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-102-generic 2024-04-17T10:24:17.534 INFO:teuthology.orchestra.run.smithi167.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-102-generic 2024-04-17T10:24:17.682 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up linux-modules-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T10:24:18.611 INFO:teuthology.orchestra.run.smithi083.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-17T10:24:18.611 INFO:teuthology.orchestra.run.smithi083.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-17T10:24:18.611 INFO:teuthology.orchestra.run.smithi083.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-102-generic 2024-04-17T10:24:18.611 INFO:teuthology.orchestra.run.smithi083.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-102-generic 2024-04-17T10:24:18.811 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-modules-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T10:24:19.175 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up linux-modules-extra-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T10:24:20.212 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-modules-extra-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T10:24:20.619 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up linux-image-generic (5.15.0.102.99) ... 2024-04-17T10:24:20.744 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up linux-generic (5.15.0.102.99) ... 2024-04-17T10:24:20.871 INFO:teuthology.orchestra.run.smithi167.stdout:Processing triggers for linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T10:24:20.940 INFO:teuthology.orchestra.run.smithi167.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-17T10:24:20.941 INFO:teuthology.orchestra.run.smithi167.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-102-generic 2024-04-17T10:24:21.563 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-image-generic (5.15.0.102.99) ... 2024-04-17T10:24:21.689 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-generic (5.15.0.102.99) ... 2024-04-17T10:24:21.816 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T10:24:21.879 INFO:teuthology.orchestra.run.smithi083.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-17T10:24:21.879 INFO:teuthology.orchestra.run.smithi083.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-102-generic 2024-04-17T10:24:34.784 INFO:teuthology.orchestra.run.smithi167.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-17T10:24:34.785 INFO:teuthology.orchestra.run.smithi167.stdout:Sourcing file `/etc/default/grub' 2024-04-17T10:24:34.807 INFO:teuthology.orchestra.run.smithi167.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-17T10:24:34.830 INFO:teuthology.orchestra.run.smithi167.stdout:Generating grub configuration file ... 2024-04-17T10:24:35.157 INFO:teuthology.orchestra.run.smithi167.stdout:Found linux image: /boot/vmlinuz-5.15.0-102-generic 2024-04-17T10:24:35.187 INFO:teuthology.orchestra.run.smithi167.stdout:Found initrd image: /boot/initrd.img-5.15.0-102-generic 2024-04-17T10:24:35.560 INFO:teuthology.orchestra.run.smithi167.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-17T10:24:35.566 INFO:teuthology.orchestra.run.smithi167.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-17T10:24:35.636 INFO:teuthology.orchestra.run.smithi167.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-17T10:24:35.642 INFO:teuthology.orchestra.run.smithi167.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-17T10:24:35.793 INFO:teuthology.orchestra.run.smithi167.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-17T10:24:35.793 INFO:teuthology.orchestra.run.smithi167.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-17T10:24:35.793 INFO:teuthology.orchestra.run.smithi167.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-17T10:24:35.829 INFO:teuthology.orchestra.run.smithi167.stdout:done 2024-04-17T10:24:36.072 INFO:teuthology.orchestra.run.smithi083.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-17T10:24:36.072 INFO:teuthology.orchestra.run.smithi083.stdout:Sourcing file `/etc/default/grub' 2024-04-17T10:24:36.104 INFO:teuthology.orchestra.run.smithi083.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-17T10:24:36.126 INFO:teuthology.orchestra.run.smithi083.stdout:Generating grub configuration file ... 2024-04-17T10:24:36.300 DEBUG:teuthology.orchestra.run.smithi167:> dpkg -s linux-image-generic 2024-04-17T10:24:36.322 INFO:teuthology.orchestra.run.smithi167.stdout:Package: linux-image-generic 2024-04-17T10:24:36.322 INFO:teuthology.orchestra.run.smithi167.stdout:Status: install ok installed 2024-04-17T10:24:36.322 INFO:teuthology.orchestra.run.smithi167.stdout:Priority: optional 2024-04-17T10:24:36.322 INFO:teuthology.orchestra.run.smithi167.stdout:Section: kernel 2024-04-17T10:24:36.322 INFO:teuthology.orchestra.run.smithi167.stdout:Installed-Size: 21 2024-04-17T10:24:36.322 INFO:teuthology.orchestra.run.smithi167.stdout:Maintainer: Ubuntu Kernel Team 2024-04-17T10:24:36.322 INFO:teuthology.orchestra.run.smithi167.stdout:Architecture: amd64 2024-04-17T10:24:36.322 INFO:teuthology.orchestra.run.smithi167.stdout:Source: linux-meta 2024-04-17T10:24:36.322 INFO:teuthology.orchestra.run.smithi167.stdout:Version: 5.15.0.102.99 2024-04-17T10:24:36.323 INFO:teuthology.orchestra.run.smithi167.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-102), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-17T10:24:36.323 INFO:teuthology.orchestra.run.smithi167.stdout:Depends: linux-image-5.15.0-102-generic, linux-modules-extra-5.15.0-102-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-17T10:24:36.323 INFO:teuthology.orchestra.run.smithi167.stdout:Recommends: thermald 2024-04-17T10:24:36.323 INFO:teuthology.orchestra.run.smithi167.stdout:Description: Generic Linux kernel image 2024-04-17T10:24:36.323 INFO:teuthology.orchestra.run.smithi167.stdout: This package will always depend on the latest generic kernel image 2024-04-17T10:24:36.323 INFO:teuthology.orchestra.run.smithi167.stdout: available. 2024-04-17T10:24:36.323 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-102-generic 2024-04-17T10:24:36.323 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-17T10:24:36.323 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-17T10:24:36.324 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-04-17T10:24:36.324 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi167.front.sepia.ceph.com, path=None, version=distro) 2024-04-17T10:24:36.324 DEBUG:teuthology.orchestra.run.smithi167:> sudo apt-get clean 2024-04-17T10:24:36.383 INFO:teuthology.orchestra.run.smithi083.stdout:Found linux image: /boot/vmlinuz-5.15.0-102-generic 2024-04-17T10:24:36.408 INFO:teuthology.orchestra.run.smithi083.stdout:Found initrd image: /boot/initrd.img-5.15.0-102-generic 2024-04-17T10:24:36.502 DEBUG:teuthology.orchestra.run.smithi167:> sudo apt-get update 2024-04-17T10:24:36.652 INFO:teuthology.orchestra.run.smithi167.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-17T10:24:36.693 INFO:teuthology.orchestra.run.smithi083.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-17T10:24:36.698 INFO:teuthology.orchestra.run.smithi083.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-17T10:24:36.734 INFO:teuthology.orchestra.run.smithi167.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-17T10:24:36.767 INFO:teuthology.orchestra.run.smithi083.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-17T10:24:36.773 INFO:teuthology.orchestra.run.smithi083.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-17T10:24:36.827 INFO:teuthology.orchestra.run.smithi167.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-17T10:24:36.904 INFO:teuthology.orchestra.run.smithi167.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-17T10:24:36.946 INFO:teuthology.orchestra.run.smithi083.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-17T10:24:36.946 INFO:teuthology.orchestra.run.smithi083.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-17T10:24:36.946 INFO:teuthology.orchestra.run.smithi083.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-17T10:24:37.010 INFO:teuthology.orchestra.run.smithi083.stdout:done 2024-04-17T10:24:37.524 DEBUG:teuthology.orchestra.run.smithi083:> dpkg -s linux-image-generic 2024-04-17T10:24:37.582 INFO:teuthology.orchestra.run.smithi083.stdout:Package: linux-image-generic 2024-04-17T10:24:37.582 INFO:teuthology.orchestra.run.smithi083.stdout:Status: install ok installed 2024-04-17T10:24:37.582 INFO:teuthology.orchestra.run.smithi083.stdout:Priority: optional 2024-04-17T10:24:37.582 INFO:teuthology.orchestra.run.smithi083.stdout:Section: kernel 2024-04-17T10:24:37.582 INFO:teuthology.orchestra.run.smithi083.stdout:Installed-Size: 21 2024-04-17T10:24:37.582 INFO:teuthology.orchestra.run.smithi083.stdout:Maintainer: Ubuntu Kernel Team 2024-04-17T10:24:37.582 INFO:teuthology.orchestra.run.smithi083.stdout:Architecture: amd64 2024-04-17T10:24:37.583 INFO:teuthology.orchestra.run.smithi083.stdout:Source: linux-meta 2024-04-17T10:24:37.583 INFO:teuthology.orchestra.run.smithi083.stdout:Version: 5.15.0.102.99 2024-04-17T10:24:37.583 INFO:teuthology.orchestra.run.smithi083.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-102), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-17T10:24:37.583 INFO:teuthology.orchestra.run.smithi083.stdout:Depends: linux-image-5.15.0-102-generic, linux-modules-extra-5.15.0-102-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-17T10:24:37.583 INFO:teuthology.orchestra.run.smithi083.stdout:Recommends: thermald 2024-04-17T10:24:37.583 INFO:teuthology.orchestra.run.smithi083.stdout:Description: Generic Linux kernel image 2024-04-17T10:24:37.583 INFO:teuthology.orchestra.run.smithi083.stdout: This package will always depend on the latest generic kernel image 2024-04-17T10:24:37.583 INFO:teuthology.orchestra.run.smithi083.stdout: available. 2024-04-17T10:24:37.583 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-102-generic 2024-04-17T10:24:37.583 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-17T10:24:37.584 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-17T10:24:37.584 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-17T10:24:37.584 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi083.front.sepia.ceph.com, path=None, version=distro) 2024-04-17T10:24:37.584 DEBUG:teuthology.orchestra.run.smithi083:> sudo apt-get clean 2024-04-17T10:24:37.713 DEBUG:teuthology.orchestra.run.smithi083:> sudo apt-get update 2024-04-17T10:24:37.863 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-17T10:24:37.951 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-17T10:24:38.038 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-17T10:24:38.039 INFO:teuthology.orchestra.run.smithi167.stdout:Reading package lists... 2024-04-17T10:24:38.057 DEBUG:teuthology.orchestra.run.smithi167:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-17T10:24:38.114 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-17T10:24:38.116 INFO:teuthology.orchestra.run.smithi167.stdout:Reading package lists... 2024-04-17T10:24:38.315 INFO:teuthology.orchestra.run.smithi167.stdout:Building dependency tree... 2024-04-17T10:24:38.315 INFO:teuthology.orchestra.run.smithi167.stdout:Reading state information... 2024-04-17T10:24:38.488 INFO:teuthology.orchestra.run.smithi167.stdout:linux-image-generic is already the newest version (5.15.0.102.99). 2024-04-17T10:24:38.489 INFO:teuthology.orchestra.run.smithi167.stdout:The following packages were automatically installed and are no longer required: 2024-04-17T10:24:38.489 INFO:teuthology.orchestra.run.smithi167.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-17T10:24:38.489 INFO:teuthology.orchestra.run.smithi167.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-17T10:24:38.489 INFO:teuthology.orchestra.run.smithi167.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-17T10:24:38.489 INFO:teuthology.orchestra.run.smithi167.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-17T10:24:38.489 INFO:teuthology.orchestra.run.smithi167.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-17T10:24:38.490 INFO:teuthology.orchestra.run.smithi167.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-17T10:24:38.527 INFO:teuthology.orchestra.run.smithi167.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-17T10:24:38.529 DEBUG:teuthology.orchestra.run.smithi167:> dpkg -s linux-image-generic 2024-04-17T10:24:38.581 INFO:teuthology.orchestra.run.smithi167.stdout:Package: linux-image-generic 2024-04-17T10:24:38.581 INFO:teuthology.orchestra.run.smithi167.stdout:Status: install ok installed 2024-04-17T10:24:38.581 INFO:teuthology.orchestra.run.smithi167.stdout:Priority: optional 2024-04-17T10:24:38.581 INFO:teuthology.orchestra.run.smithi167.stdout:Section: kernel 2024-04-17T10:24:38.581 INFO:teuthology.orchestra.run.smithi167.stdout:Installed-Size: 21 2024-04-17T10:24:38.581 INFO:teuthology.orchestra.run.smithi167.stdout:Maintainer: Ubuntu Kernel Team 2024-04-17T10:24:38.581 INFO:teuthology.orchestra.run.smithi167.stdout:Architecture: amd64 2024-04-17T10:24:38.581 INFO:teuthology.orchestra.run.smithi167.stdout:Source: linux-meta 2024-04-17T10:24:38.581 INFO:teuthology.orchestra.run.smithi167.stdout:Version: 5.15.0.102.99 2024-04-17T10:24:38.581 INFO:teuthology.orchestra.run.smithi167.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-102), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-17T10:24:38.581 INFO:teuthology.orchestra.run.smithi167.stdout:Depends: linux-image-5.15.0-102-generic, linux-modules-extra-5.15.0-102-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-17T10:24:38.581 INFO:teuthology.orchestra.run.smithi167.stdout:Recommends: thermald 2024-04-17T10:24:38.582 INFO:teuthology.orchestra.run.smithi167.stdout:Description: Generic Linux kernel image 2024-04-17T10:24:38.582 INFO:teuthology.orchestra.run.smithi167.stdout: This package will always depend on the latest generic kernel image 2024-04-17T10:24:38.582 INFO:teuthology.orchestra.run.smithi167.stdout: available. 2024-04-17T10:24:38.582 DEBUG:teuthology.orchestra.run.smithi167:> mktemp 2024-04-17T10:24:38.630 INFO:teuthology.orchestra.run.smithi167.stdout:/tmp/tmp.ASFh8sT4GX 2024-04-17T10:24:38.630 DEBUG:teuthology.orchestra.run.smithi167:> sudo cp /boot/grub/grub.cfg /tmp/tmp.ASFh8sT4GX 2024-04-17T10:24:38.686 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0666 /tmp/tmp.ASFh8sT4GX 2024-04-17T10:24:38.810 DEBUG:teuthology.orchestra.remote:smithi167:/tmp/tmp.ASFh8sT4GX is 10KB 2024-04-17T10:24:38.823 DEBUG:teuthology.orchestra.run.smithi167:> rm -fr /tmp/tmp.ASFh8sT4GX 2024-04-17T10:24:38.828 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-17T10:24:38.881 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:24:38.881 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-17T10:24:38.881 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-17T10:24:38.945 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-102-generic 2024-04-17T10:24:38.945 DEBUG:teuthology.orchestra.run.smithi167:> sudo update-grub 2024-04-17T10:24:39.224 INFO:teuthology.orchestra.run.smithi083.stdout:Reading package lists... 2024-04-17T10:24:39.241 DEBUG:teuthology.orchestra.run.smithi083:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-17T10:24:39.291 INFO:teuthology.orchestra.run.smithi083.stdout:Reading package lists... 2024-04-17T10:24:39.482 INFO:teuthology.orchestra.run.smithi083.stdout:Building dependency tree... 2024-04-17T10:24:39.483 INFO:teuthology.orchestra.run.smithi083.stdout:Reading state information... 2024-04-17T10:24:39.638 INFO:teuthology.orchestra.run.smithi083.stdout:linux-image-generic is already the newest version (5.15.0.102.99). 2024-04-17T10:24:39.638 INFO:teuthology.orchestra.run.smithi083.stdout:The following packages were automatically installed and are no longer required: 2024-04-17T10:24:39.638 INFO:teuthology.orchestra.run.smithi083.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-17T10:24:39.638 INFO:teuthology.orchestra.run.smithi083.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-17T10:24:39.638 INFO:teuthology.orchestra.run.smithi083.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-17T10:24:39.638 INFO:teuthology.orchestra.run.smithi083.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-17T10:24:39.638 INFO:teuthology.orchestra.run.smithi083.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-17T10:24:39.638 INFO:teuthology.orchestra.run.smithi083.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-17T10:24:39.677 INFO:teuthology.orchestra.run.smithi083.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-17T10:24:39.679 DEBUG:teuthology.orchestra.run.smithi083:> dpkg -s linux-image-generic 2024-04-17T10:24:39.739 INFO:teuthology.orchestra.run.smithi083.stdout:Package: linux-image-generic 2024-04-17T10:24:39.739 INFO:teuthology.orchestra.run.smithi083.stdout:Status: install ok installed 2024-04-17T10:24:39.740 INFO:teuthology.orchestra.run.smithi083.stdout:Priority: optional 2024-04-17T10:24:39.740 INFO:teuthology.orchestra.run.smithi083.stdout:Section: kernel 2024-04-17T10:24:39.740 INFO:teuthology.orchestra.run.smithi083.stdout:Installed-Size: 21 2024-04-17T10:24:39.740 INFO:teuthology.orchestra.run.smithi083.stdout:Maintainer: Ubuntu Kernel Team 2024-04-17T10:24:39.740 INFO:teuthology.orchestra.run.smithi083.stdout:Architecture: amd64 2024-04-17T10:24:39.740 INFO:teuthology.orchestra.run.smithi083.stdout:Source: linux-meta 2024-04-17T10:24:39.740 INFO:teuthology.orchestra.run.smithi083.stdout:Version: 5.15.0.102.99 2024-04-17T10:24:39.740 INFO:teuthology.orchestra.run.smithi083.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-102), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-17T10:24:39.740 INFO:teuthology.orchestra.run.smithi083.stdout:Depends: linux-image-5.15.0-102-generic, linux-modules-extra-5.15.0-102-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-17T10:24:39.740 INFO:teuthology.orchestra.run.smithi083.stdout:Recommends: thermald 2024-04-17T10:24:39.740 INFO:teuthology.orchestra.run.smithi083.stdout:Description: Generic Linux kernel image 2024-04-17T10:24:39.740 INFO:teuthology.orchestra.run.smithi083.stdout: This package will always depend on the latest generic kernel image 2024-04-17T10:24:39.740 INFO:teuthology.orchestra.run.smithi083.stdout: available. 2024-04-17T10:24:39.741 DEBUG:teuthology.orchestra.run.smithi083:> mktemp 2024-04-17T10:24:39.785 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/tmp.UYMFsVTwmi 2024-04-17T10:24:39.785 DEBUG:teuthology.orchestra.run.smithi083:> sudo cp /boot/grub/grub.cfg /tmp/tmp.UYMFsVTwmi 2024-04-17T10:24:39.841 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0666 /tmp/tmp.UYMFsVTwmi 2024-04-17T10:24:39.957 DEBUG:teuthology.orchestra.remote:smithi083:/tmp/tmp.UYMFsVTwmi is 10KB 2024-04-17T10:24:39.970 DEBUG:teuthology.orchestra.run.smithi083:> rm -fr /tmp/tmp.UYMFsVTwmi 2024-04-17T10:24:39.975 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-17T10:24:40.029 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:24:40.029 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-17T10:24:40.029 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-17T10:24:40.095 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-102-generic 2024-04-17T10:24:40.095 DEBUG:teuthology.orchestra.run.smithi083:> sudo update-grub 2024-04-17T10:24:40.102 INFO:teuthology.orchestra.run.smithi167.stderr:Sourcing file `/etc/default/grub' 2024-04-17T10:24:40.104 INFO:teuthology.orchestra.run.smithi167.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-17T10:24:40.106 INFO:teuthology.orchestra.run.smithi167.stderr:Generating grub configuration file ... 2024-04-17T10:24:40.355 INFO:teuthology.orchestra.run.smithi167.stderr:Found linux image: /boot/vmlinuz-5.15.0-102-generic 2024-04-17T10:24:40.361 INFO:teuthology.orchestra.run.smithi167.stderr:Found initrd image: /boot/initrd.img-5.15.0-102-generic 2024-04-17T10:24:40.665 INFO:teuthology.orchestra.run.smithi167.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-17T10:24:40.671 INFO:teuthology.orchestra.run.smithi167.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-17T10:24:40.712 INFO:teuthology.orchestra.run.smithi167.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-17T10:24:40.717 INFO:teuthology.orchestra.run.smithi167.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-17T10:24:40.834 INFO:teuthology.orchestra.run.smithi167.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-17T10:24:40.835 INFO:teuthology.orchestra.run.smithi167.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-17T10:24:40.835 INFO:teuthology.orchestra.run.smithi167.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-17T10:24:40.851 INFO:teuthology.orchestra.run.smithi167.stderr:done 2024-04-17T10:24:40.866 DEBUG:teuthology.orchestra.run.smithi167:> sudo shutdown -r now 2024-04-17T10:24:41.295 INFO:teuthology.orchestra.run.smithi083.stderr:Sourcing file `/etc/default/grub' 2024-04-17T10:24:41.296 INFO:teuthology.orchestra.run.smithi083.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-17T10:24:41.298 INFO:teuthology.orchestra.run.smithi083.stderr:Generating grub configuration file ... 2024-04-17T10:24:41.488 INFO:teuthology.orchestra.run.smithi083.stderr:Found linux image: /boot/vmlinuz-5.15.0-102-generic 2024-04-17T10:24:41.493 INFO:teuthology.orchestra.run.smithi083.stderr:Found initrd image: /boot/initrd.img-5.15.0-102-generic 2024-04-17T10:24:41.739 INFO:teuthology.orchestra.run.smithi083.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-17T10:24:41.744 INFO:teuthology.orchestra.run.smithi083.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-17T10:24:41.784 INFO:teuthology.orchestra.run.smithi083.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-17T10:24:41.790 INFO:teuthology.orchestra.run.smithi083.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-17T10:24:41.912 INFO:teuthology.orchestra.run.smithi083.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-17T10:24:41.912 INFO:teuthology.orchestra.run.smithi083.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-17T10:24:41.912 INFO:teuthology.orchestra.run.smithi083.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-17T10:24:41.930 INFO:teuthology.orchestra.run.smithi083.stderr:done 2024-04-17T10:24:41.933 DEBUG:teuthology.orchestra.run.smithi083:> sudo shutdown -r now 2024-04-17T10:25:10.894 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-17T10:25:10.895 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-04-17T10:25:10.896 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:25:11.936 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-17T10:25:11.937 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-17T10:25:11.937 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:25:29.375 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-04-17T10:25:30.398 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-04-17T10:25:38.384 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-04-17T10:25:38.385 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:25:39.401 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-17T10:25:39.401 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:25:41.438 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-04-17T10:25:42.462 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-04-17T10:25:53.449 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-04-17T10:25:53.450 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:25:54.465 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-17T10:25:54.466 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:26:12.894 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-04-17T10:26:27.908 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-17T10:26:27.909 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:26:28.373 DEBUG:teuthology.orchestra.run.smithi083:> true 2024-04-17T10:26:29.599 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-17T10:26:29.599 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-102-generic"... 2024-04-17T10:26:29.600 DEBUG:teuthology.orchestra.run.smithi083:> uname -r 2024-04-17T10:26:29.605 INFO:teuthology.orchestra.run.smithi083.stdout:5.15.0-102-generic 2024-04-17T10:26:29.605 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-102-generic vs 5.15.0-102-generic 2024-04-17T10:26:29.605 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-17T10:26:29.605 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-17T10:26:30.606 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-17T10:26:30.607 DEBUG:teuthology.orchestra.run.smithi083:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-17T10:26:30.791 INFO:teuthology.orchestra.run.smithi083.stdout:ttyS1 2024-04-17T10:26:30.810 DEBUG:teuthology.parallel:result is None 2024-04-17T10:26:53.453 DEBUG:teuthology.orchestra.remote:timed out 2024-04-17T10:27:08.455 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-04-17T10:27:08.456 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:27:08.731 DEBUG:teuthology.orchestra.run.smithi167:> true 2024-04-17T10:27:09.246 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-04-17T10:27:09.246 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-102-generic"... 2024-04-17T10:27:09.246 DEBUG:teuthology.orchestra.run.smithi167:> uname -r 2024-04-17T10:27:09.294 INFO:teuthology.orchestra.run.smithi167.stdout:5.15.0-102-generic 2024-04-17T10:27:09.294 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-102-generic vs 5.15.0-102-generic 2024-04-17T10:27:09.294 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-17T10:27:09.295 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-17T10:27:10.296 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-17T10:27:10.296 DEBUG:teuthology.orchestra.run.smithi167:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-17T10:27:10.660 INFO:teuthology.orchestra.run.smithi167.stdout:ttyS1 2024-04-17T10:27:10.680 DEBUG:teuthology.parallel:result is None 2024-04-17T10:27:10.680 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-17T10:27:10.688 INFO:teuthology.task.internal:Creating test directory... 2024-04-17T10:27:10.688 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-17T10:27:10.691 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-17T10:27:10.697 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-17T10:27:10.733 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-17T10:27:10.758 INFO:teuthology.task.internal:Creating archive directory... 2024-04-17T10:27:10.758 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-17T10:27:10.761 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-17T10:27:10.853 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-17T10:27:10.860 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-17T10:27:10.860 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-17T10:27:10.863 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-17T10:27:10.879 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T10:27:10.882 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T10:27:10.887 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T10:27:10.890 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T10:27:10.892 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-17T10:27:10.899 INFO:teuthology.task.internal:Configuring sudo... 2024-04-17T10:27:10.899 DEBUG:teuthology.orchestra.run.smithi083:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-17T10:27:10.933 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-17T10:27:10.961 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-17T10:27:10.971 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-17T10:27:10.972 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-17T10:27:11.005 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-17T10:27:11.010 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-17T10:27:11.055 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-17T10:27:11.104 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:27:11.104 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-17T10:27:11.175 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-17T10:27:11.181 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-17T10:27:11.227 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:27:11.227 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-17T10:27:11.297 DEBUG:teuthology.orchestra.run.smithi083:> sudo service rsyslog restart 2024-04-17T10:27:11.300 DEBUG:teuthology.orchestra.run.smithi167:> sudo service rsyslog restart 2024-04-17T10:27:11.370 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-17T10:27:11.378 INFO:teuthology.task.internal:Starting timer... 2024-04-17T10:27:11.378 INFO:teuthology.run_tasks:Running task pcp... 2024-04-17T10:27:11.413 INFO:teuthology.run_tasks:Running task selinux... 2024-04-17T10:27:11.436 DEBUG:teuthology.task.selinux:Excluding smithi083: OS 'ubuntu' does not support SELinux 2024-04-17T10:27:11.436 DEBUG:teuthology.task.selinux:Excluding smithi167: OS 'ubuntu' does not support SELinux 2024-04-17T10:27:11.436 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-17T10:27:11.436 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-04-17T10:27:11.437 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-17T10:27:11.437 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-17T10:27:11.462 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-17T10:27:11.478 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-17T10:27:11.480 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi083.front.sepia.ceph.com,smithi167.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-17T10:34:19.679 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi083.front.sepia.ceph.com'), Remote(name='ubuntu@smithi167.front.sepia.ceph.com')] 2024-04-17T10:34:19.681 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-17T10:34:19.683 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:34:19.769 DEBUG:teuthology.orchestra.run.smithi083:> true 2024-04-17T10:34:19.846 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-17T10:34:19.847 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-04-17T10:34:19.847 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:34:19.927 DEBUG:teuthology.orchestra.run.smithi167:> true 2024-04-17T10:34:20.006 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-04-17T10:34:20.007 INFO:teuthology.run_tasks:Running task clock... 2024-04-17T10:34:20.018 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-17T10:34:20.019 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-17T10:34:20.019 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T10:34:20.022 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-17T10:34:20.022 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T10:34:20.051 INFO:teuthology.orchestra.run.smithi083.stdout:17 Apr 10:34:20 ntpd[17254]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-17T10:34:20.051 INFO:teuthology.orchestra.run.smithi083.stdout:17 Apr 10:34:20 ntpd[17254]: Command line: ntpd -gq 2024-04-17T10:34:20.051 INFO:teuthology.orchestra.run.smithi083.stdout:17 Apr 10:34:20 ntpd[17254]: ---------------------------------------------------- 2024-04-17T10:34:20.051 INFO:teuthology.orchestra.run.smithi083.stdout:17 Apr 10:34:20 ntpd[17254]: ntp-4 is maintained by Network Time Foundation, 2024-04-17T10:34:20.051 INFO:teuthology.orchestra.run.smithi083.stdout:17 Apr 10:34:20 ntpd[17254]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-17T10:34:20.051 INFO:teuthology.orchestra.run.smithi083.stdout:17 Apr 10:34:20 ntpd[17254]: corporation. Support and training for ntp-4 are 2024-04-17T10:34:20.051 INFO:teuthology.orchestra.run.smithi083.stdout:17 Apr 10:34:20 ntpd[17254]: available at https://www.nwtime.org/support 2024-04-17T10:34:20.051 INFO:teuthology.orchestra.run.smithi083.stdout:17 Apr 10:34:20 ntpd[17254]: ---------------------------------------------------- 2024-04-17T10:34:20.051 INFO:teuthology.orchestra.run.smithi083.stdout:17 Apr 10:34:20 ntpd[17254]: proto: precision = 0.061 usec (-24) 2024-04-17T10:34:20.051 INFO:teuthology.orchestra.run.smithi083.stdout:17 Apr 10:34:20 ntpd[17254]: basedate set to 2022-02-04 2024-04-17T10:34:20.052 INFO:teuthology.orchestra.run.smithi083.stdout:17 Apr 10:34:20 ntpd[17254]: gps base set to 2022-02-06 (week 2196) 2024-04-17T10:34:20.052 INFO:teuthology.orchestra.run.smithi083.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-17T10:34:20.052 INFO:teuthology.orchestra.run.smithi083.stdout:17 Apr 10:34:20 ntpd[17254]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-17T10:34:20.052 INFO:teuthology.orchestra.run.smithi083.stdout:17 Apr 10:34:20 ntpd[17254]: restrict ::: KOD does nothing without LIMITED. 2024-04-17T10:34:20.053 INFO:teuthology.orchestra.run.smithi083.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-17T10:34:20.053 INFO:teuthology.orchestra.run.smithi083.stdout:17 Apr 10:34:20 ntpd[17254]: Listen and drop on 0 v6wildcard [::]:123 2024-04-17T10:34:20.054 INFO:teuthology.orchestra.run.smithi083.stdout:17 Apr 10:34:20 ntpd[17254]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-17T10:34:20.054 INFO:teuthology.orchestra.run.smithi083.stdout:17 Apr 10:34:20 ntpd[17254]: Listen normally on 2 lo 127.0.0.1:123 2024-04-17T10:34:20.054 INFO:teuthology.orchestra.run.smithi083.stdout:17 Apr 10:34:20 ntpd[17254]: Listen normally on 3 enp3s0f1 172.21.15.83:123 2024-04-17T10:34:20.054 INFO:teuthology.orchestra.run.smithi083.stdout:17 Apr 10:34:20 ntpd[17254]: Listen normally on 4 lo [::1]:123 2024-04-17T10:34:20.054 INFO:teuthology.orchestra.run.smithi083.stdout:17 Apr 10:34:20 ntpd[17254]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:ce91%5]:123 2024-04-17T10:34:20.054 INFO:teuthology.orchestra.run.smithi083.stdout:17 Apr 10:34:20 ntpd[17254]: Listening on routing socket on fd #22 for interface updates 2024-04-17T10:34:20.080 INFO:teuthology.orchestra.run.smithi167.stdout:17 Apr 10:34:20 ntpd[17189]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-17T10:34:20.080 INFO:teuthology.orchestra.run.smithi167.stdout:17 Apr 10:34:20 ntpd[17189]: Command line: ntpd -gq 2024-04-17T10:34:20.080 INFO:teuthology.orchestra.run.smithi167.stdout:17 Apr 10:34:20 ntpd[17189]: ---------------------------------------------------- 2024-04-17T10:34:20.081 INFO:teuthology.orchestra.run.smithi167.stdout:17 Apr 10:34:20 ntpd[17189]: ntp-4 is maintained by Network Time Foundation, 2024-04-17T10:34:20.081 INFO:teuthology.orchestra.run.smithi167.stdout:17 Apr 10:34:20 ntpd[17189]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-17T10:34:20.081 INFO:teuthology.orchestra.run.smithi167.stdout:17 Apr 10:34:20 ntpd[17189]: corporation. Support and training for ntp-4 are 2024-04-17T10:34:20.081 INFO:teuthology.orchestra.run.smithi167.stdout:17 Apr 10:34:20 ntpd[17189]: available at https://www.nwtime.org/support 2024-04-17T10:34:20.081 INFO:teuthology.orchestra.run.smithi167.stdout:17 Apr 10:34:20 ntpd[17189]: ---------------------------------------------------- 2024-04-17T10:34:20.081 INFO:teuthology.orchestra.run.smithi167.stdout:17 Apr 10:34:20 ntpd[17189]: proto: precision = 0.056 usec (-24) 2024-04-17T10:34:20.082 INFO:teuthology.orchestra.run.smithi167.stdout:17 Apr 10:34:20 ntpd[17189]: basedate set to 2022-02-04 2024-04-17T10:34:20.082 INFO:teuthology.orchestra.run.smithi167.stdout:17 Apr 10:34:20 ntpd[17189]: gps base set to 2022-02-06 (week 2196) 2024-04-17T10:34:20.082 INFO:teuthology.orchestra.run.smithi167.stdout:17 Apr 10:34:20 ntpd[17189]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-17T10:34:20.082 INFO:teuthology.orchestra.run.smithi167.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-17T10:34:20.082 INFO:teuthology.orchestra.run.smithi167.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-17T10:34:20.083 INFO:teuthology.orchestra.run.smithi167.stdout:17 Apr 10:34:20 ntpd[17189]: restrict ::: KOD does nothing without LIMITED. 2024-04-17T10:34:20.083 INFO:teuthology.orchestra.run.smithi167.stdout:17 Apr 10:34:20 ntpd[17189]: Listen and drop on 0 v6wildcard [::]:123 2024-04-17T10:34:20.084 INFO:teuthology.orchestra.run.smithi167.stdout:17 Apr 10:34:20 ntpd[17189]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-17T10:34:20.084 INFO:teuthology.orchestra.run.smithi167.stdout:17 Apr 10:34:20 ntpd[17189]: Listen normally on 2 lo 127.0.0.1:123 2024-04-17T10:34:20.084 INFO:teuthology.orchestra.run.smithi167.stdout:17 Apr 10:34:20 ntpd[17189]: Listen normally on 3 enp3s0f1 172.21.15.167:123 2024-04-17T10:34:20.084 INFO:teuthology.orchestra.run.smithi167.stdout:17 Apr 10:34:20 ntpd[17189]: Listen normally on 4 lo [::1]:123 2024-04-17T10:34:20.084 INFO:teuthology.orchestra.run.smithi167.stdout:17 Apr 10:34:20 ntpd[17189]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7309%5]:123 2024-04-17T10:34:20.084 INFO:teuthology.orchestra.run.smithi167.stdout:17 Apr 10:34:20 ntpd[17189]: Listening on routing socket on fd #22 for interface updates 2024-04-17T10:34:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:21 ntpd[17254]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:21 ntpd[17254]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:21 ntpd[17254]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:21 ntpd[17254]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:21.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:21 ntpd[17254]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:21.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:21 ntpd[17254]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:21.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:21 ntpd[17254]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:21.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:21 ntpd[17254]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:21.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:21 ntpd[17254]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T10:34:21.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:21 ntpd[17254]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T10:34:21.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:21 ntpd[17254]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:21.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:21 ntpd[17254]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:21.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:21 ntpd[17254]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T10:34:21.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:21 ntpd[17254]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T10:34:21.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:21 ntpd[17189]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:21.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:21 ntpd[17189]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:21.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:21 ntpd[17189]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:21.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:21 ntpd[17189]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:21.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:21 ntpd[17189]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:21.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:21 ntpd[17189]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:21.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:21 ntpd[17189]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T10:34:21.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:21 ntpd[17189]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T10:34:23.051 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:23 ntpd[17254]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:23.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:23 ntpd[17254]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:23.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:23 ntpd[17254]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:23.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:23 ntpd[17254]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:23.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:23 ntpd[17254]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:23.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:23 ntpd[17254]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:23.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:23 ntpd[17254]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:23.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:23 ntpd[17254]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:23.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:23 ntpd[17254]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T10:34:23.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:23 ntpd[17254]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T10:34:23.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:23 ntpd[17254]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:23.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:23 ntpd[17254]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:23.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:23 ntpd[17254]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T10:34:23.053 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:23 ntpd[17254]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T10:34:23.053 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:23 ntpd[17254]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:23.053 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:23 ntpd[17254]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:23.053 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:23 ntpd[17254]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T10:34:23.053 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:23 ntpd[17254]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T10:34:23.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:23 ntpd[17189]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:23.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:23 ntpd[17189]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:23.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:23 ntpd[17189]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:23.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:23 ntpd[17189]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:23.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:23 ntpd[17189]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T10:34:23.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:23 ntpd[17189]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T10:34:24.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:24 ntpd[17189]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:24.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:24 ntpd[17189]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:24.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:24 ntpd[17189]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:24.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:24 ntpd[17189]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:24.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:24 ntpd[17189]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:24.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:24 ntpd[17189]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:24.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:24 ntpd[17189]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T10:34:24.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:24 ntpd[17189]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T10:34:24.082 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:24 ntpd[17189]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:24.082 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:24 ntpd[17189]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:24.082 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:24 ntpd[17189]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T10:34:24.082 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:24 ntpd[17189]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T10:34:25.051 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:25 ntpd[17254]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:25.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:25 ntpd[17254]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:25.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:25 ntpd[17254]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:25.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:25 ntpd[17254]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:25.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:25 ntpd[17254]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:25.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:25 ntpd[17254]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:25.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:25 ntpd[17254]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:25.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:25 ntpd[17254]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:25.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:25 ntpd[17254]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T10:34:25.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:25 ntpd[17254]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T10:34:25.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:25 ntpd[17254]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:25.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:25 ntpd[17254]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:25.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:25 ntpd[17254]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T10:34:25.053 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:25 ntpd[17254]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T10:34:25.053 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:25 ntpd[17254]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:25.053 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:25 ntpd[17254]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:25.053 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:25 ntpd[17254]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T10:34:25.053 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:25 ntpd[17254]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T10:34:25.080 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:25 ntpd[17189]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:25.080 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:25 ntpd[17189]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:25.080 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:25 ntpd[17189]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:25.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:25 ntpd[17189]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:25.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:25 ntpd[17189]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T10:34:25.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:25 ntpd[17189]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T10:34:26.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:26 ntpd[17189]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:26.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:26 ntpd[17189]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:26.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:26 ntpd[17189]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:26.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:26 ntpd[17189]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:26.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:26 ntpd[17189]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:26.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:26 ntpd[17189]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:26.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:26 ntpd[17189]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T10:34:26.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:26 ntpd[17189]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T10:34:26.082 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:26 ntpd[17189]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:26.082 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:26 ntpd[17189]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:26.082 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:26 ntpd[17189]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T10:34:26.082 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:26 ntpd[17189]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T10:34:27.051 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:27 ntpd[17254]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:27.051 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:27 ntpd[17254]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:27.051 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:27 ntpd[17254]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:27.051 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:27 ntpd[17254]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:27.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:27 ntpd[17254]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:27.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:27 ntpd[17254]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:27.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:27 ntpd[17254]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:27.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:27 ntpd[17254]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:27.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:27 ntpd[17254]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T10:34:27.052 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:27 ntpd[17254]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T10:34:27.052 INFO:teuthology.orchestra.run.smithi083.stdout:17 Apr 10:34:27 ntpd[17254]: ntpd: time slew -0.001935 s 2024-04-17T10:34:27.052 INFO:teuthology.orchestra.run.smithi083.stdout:ntpd: time slew -0.001935s 2024-04-17T10:34:27.053 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:27 ntpd[17254]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-17T10:34:27.053 INFO:teuthology.orchestra.run.smithi083.stderr:17 Apr 10:34:27 ntpd[17254]: can't open /var/log/ntpstats/loopstats.20240417: Permission denied 2024-04-17T10:34:27.080 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:27 ntpd[17189]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:27.080 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:27 ntpd[17189]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:27.080 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:27 ntpd[17189]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T10:34:27.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:27 ntpd[17189]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T10:34:27.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:27 ntpd[17189]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T10:34:27.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:27 ntpd[17189]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T10:34:27.081 INFO:teuthology.orchestra.run.smithi167.stdout:17 Apr 10:34:27 ntpd[17189]: ntpd: time slew -0.003517 s 2024-04-17T10:34:27.081 INFO:teuthology.orchestra.run.smithi167.stdout:ntpd: time slew -0.003517s 2024-04-17T10:34:27.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:27 ntpd[17189]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-17T10:34:27.081 INFO:teuthology.orchestra.run.smithi167.stderr:17 Apr 10:34:27 ntpd[17189]: can't open /var/log/ntpstats/loopstats.20240417: Permission denied 2024-04-17T10:34:27.105 INFO:teuthology.orchestra.run.smithi083.stdout: remote refid st t when poll reach delay offset jitter 2024-04-17T10:34:27.105 INFO:teuthology.orchestra.run.smithi083.stdout:============================================================================== 2024-04-17T10:34:27.105 INFO:teuthology.orchestra.run.smithi083.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-17T10:34:27.106 INFO:teuthology.orchestra.run.smithi083.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-17T10:34:27.106 INFO:teuthology.orchestra.run.smithi083.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-17T10:34:27.106 INFO:teuthology.orchestra.run.smithi083.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-17T10:34:27.130 INFO:teuthology.orchestra.run.smithi167.stdout: remote refid st t when poll reach delay offset jitter 2024-04-17T10:34:27.130 INFO:teuthology.orchestra.run.smithi167.stdout:============================================================================== 2024-04-17T10:34:27.130 INFO:teuthology.orchestra.run.smithi167.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-17T10:34:27.130 INFO:teuthology.orchestra.run.smithi167.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-17T10:34:27.130 INFO:teuthology.orchestra.run.smithi167.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-17T10:34:27.130 INFO:teuthology.orchestra.run.smithi167.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-17T10:34:27.131 INFO:teuthology.run_tasks:Running task install... 2024-04-17T10:34:27.141 DEBUG:teuthology.task.install:project ceph 2024-04-17T10:34:27.141 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}} 2024-04-17T10:34:27.141 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'} 2024-04-17T10:34:27.141 INFO:teuthology.task.install:Using flavor: default 2024-04-17T10:34:27.147 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-17T10:34:27.147 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-17T10:34:27.148 DEBUG:teuthology.orchestra.run.smithi083:> sudo apt-key list | grep Ceph 2024-04-17T10:34:27.148 DEBUG:teuthology.orchestra.run.smithi167:> sudo apt-key list | grep Ceph 2024-04-17T10:34:27.212 INFO:teuthology.orchestra.run.smithi083.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-17T10:34:27.232 INFO:teuthology.orchestra.run.smithi167.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-17T10:34:27.239 INFO:teuthology.orchestra.run.smithi083.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-17T10:34:27.239 INFO:teuthology.orchestra.run.smithi083.stdout:uid [ unknown] Ceph.com (release key) 2024-04-17T10:34:27.240 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-17T10:34:27.240 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-17T10:34:27.240 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-17T10:34:27.260 INFO:teuthology.orchestra.run.smithi167.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-17T10:34:27.260 INFO:teuthology.orchestra.run.smithi167.stdout:uid [ unknown] Ceph.com (release key) 2024-04-17T10:34:27.261 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-17T10:34:27.261 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-17T10:34:27.261 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-17T10:34:27.380 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default/ 2024-04-17T10:34:27.380 INFO:teuthology.task.install.deb:Package version is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-17T10:34:27.404 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default/ 2024-04-17T10:34:27.404 INFO:teuthology.task.install.deb:Package version is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-17T10:34:27.551 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:34:27.551 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-17T10:34:27.565 DEBUG:teuthology.orchestra.run.smithi083:> sudo apt-get update 2024-04-17T10:34:27.571 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:34:27.571 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-17T10:34:27.585 DEBUG:teuthology.orchestra.run.smithi167:> sudo apt-get update 2024-04-17T10:34:27.757 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-17T10:34:27.803 INFO:teuthology.orchestra.run.smithi167.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy InRelease 2024-04-17T10:34:27.824 INFO:teuthology.orchestra.run.smithi167.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-17T10:34:27.843 INFO:teuthology.orchestra.run.smithi083.stdout:Ign:2 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy InRelease 2024-04-17T10:34:27.847 INFO:teuthology.orchestra.run.smithi167.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-17T10:34:27.848 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-17T10:34:27.879 INFO:teuthology.orchestra.run.smithi167.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-17T10:34:27.879 INFO:teuthology.orchestra.run.smithi167.stdout:Hit:5 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-17T10:34:27.879 INFO:teuthology.orchestra.run.smithi167.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,398 B] 2024-04-17T10:34:27.880 INFO:teuthology.orchestra.run.smithi083.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-17T10:34:27.909 INFO:teuthology.orchestra.run.smithi167.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-04-17T10:34:27.917 INFO:teuthology.orchestra.run.smithi083.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-17T10:34:27.930 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-17T10:34:27.950 INFO:teuthology.orchestra.run.smithi167.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-17T10:34:27.953 INFO:teuthology.orchestra.run.smithi083.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,398 B] 2024-04-17T10:34:27.997 INFO:teuthology.orchestra.run.smithi083.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-04-17T10:34:28.012 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-17T10:34:28.034 INFO:teuthology.orchestra.run.smithi167.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-17T10:34:28.153 INFO:teuthology.orchestra.run.smithi083.stdout:Fetched 28.9 kB in 0s (69.7 kB/s) 2024-04-17T10:34:28.248 INFO:teuthology.orchestra.run.smithi167.stdout:Fetched 28.9 kB in 0s (60.1 kB/s) 2024-04-17T10:34:29.103 INFO:teuthology.orchestra.run.smithi083.stdout:Reading package lists... 2024-04-17T10:34:29.119 DEBUG:teuthology.orchestra.run.smithi083:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1628-g97ae6cc9-1jammy cephadm=19.0.0-1628-g97ae6cc9-1jammy ceph-mds=19.0.0-1628-g97ae6cc9-1jammy ceph-mgr=19.0.0-1628-g97ae6cc9-1jammy ceph-common=19.0.0-1628-g97ae6cc9-1jammy ceph-fuse=19.0.0-1628-g97ae6cc9-1jammy ceph-test=19.0.0-1628-g97ae6cc9-1jammy ceph-volume=19.0.0-1628-g97ae6cc9-1jammy radosgw=19.0.0-1628-g97ae6cc9-1jammy python3-rados=19.0.0-1628-g97ae6cc9-1jammy python3-rgw=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy python3-rbd=19.0.0-1628-g97ae6cc9-1jammy libcephfs2=19.0.0-1628-g97ae6cc9-1jammy libcephfs-dev=19.0.0-1628-g97ae6cc9-1jammy librados2=19.0.0-1628-g97ae6cc9-1jammy librbd1=19.0.0-1628-g97ae6cc9-1jammy rbd-fuse=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy cephfs-shell=19.0.0-1628-g97ae6cc9-1jammy cephfs-top=19.0.0-1628-g97ae6cc9-1jammy cephfs-mirror=19.0.0-1628-g97ae6cc9-1jammy 2024-04-17T10:34:29.180 INFO:teuthology.orchestra.run.smithi083.stdout:Reading package lists... 2024-04-17T10:34:29.196 INFO:teuthology.orchestra.run.smithi167.stdout:Reading package lists... 2024-04-17T10:34:29.212 DEBUG:teuthology.orchestra.run.smithi167:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1628-g97ae6cc9-1jammy cephadm=19.0.0-1628-g97ae6cc9-1jammy ceph-mds=19.0.0-1628-g97ae6cc9-1jammy ceph-mgr=19.0.0-1628-g97ae6cc9-1jammy ceph-common=19.0.0-1628-g97ae6cc9-1jammy ceph-fuse=19.0.0-1628-g97ae6cc9-1jammy ceph-test=19.0.0-1628-g97ae6cc9-1jammy ceph-volume=19.0.0-1628-g97ae6cc9-1jammy radosgw=19.0.0-1628-g97ae6cc9-1jammy python3-rados=19.0.0-1628-g97ae6cc9-1jammy python3-rgw=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy python3-rbd=19.0.0-1628-g97ae6cc9-1jammy libcephfs2=19.0.0-1628-g97ae6cc9-1jammy libcephfs-dev=19.0.0-1628-g97ae6cc9-1jammy librados2=19.0.0-1628-g97ae6cc9-1jammy librbd1=19.0.0-1628-g97ae6cc9-1jammy rbd-fuse=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy cephfs-shell=19.0.0-1628-g97ae6cc9-1jammy cephfs-top=19.0.0-1628-g97ae6cc9-1jammy cephfs-mirror=19.0.0-1628-g97ae6cc9-1jammy 2024-04-17T10:34:29.263 INFO:teuthology.orchestra.run.smithi167.stdout:Reading package lists... 2024-04-17T10:34:29.374 INFO:teuthology.orchestra.run.smithi083.stdout:Building dependency tree... 2024-04-17T10:34:29.375 INFO:teuthology.orchestra.run.smithi083.stdout:Reading state information... 2024-04-17T10:34:29.444 INFO:teuthology.orchestra.run.smithi167.stdout:Building dependency tree... 2024-04-17T10:34:29.445 INFO:teuthology.orchestra.run.smithi167.stdout:Reading state information... 2024-04-17T10:34:29.568 INFO:teuthology.orchestra.run.smithi083.stdout:The following packages were automatically installed and are no longer required: 2024-04-17T10:34:29.568 INFO:teuthology.orchestra.run.smithi083.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-17T10:34:29.568 INFO:teuthology.orchestra.run.smithi083.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-17T10:34:29.568 INFO:teuthology.orchestra.run.smithi083.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-17T10:34:29.569 INFO:teuthology.orchestra.run.smithi083.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-17T10:34:29.569 INFO:teuthology.orchestra.run.smithi083.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-17T10:34:29.569 INFO:teuthology.orchestra.run.smithi083.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-17T10:34:29.570 INFO:teuthology.orchestra.run.smithi083.stdout:The following additional packages will be installed: 2024-04-17T10:34:29.571 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-17T10:34:29.571 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-17T10:34:29.571 INFO:teuthology.orchestra.run.smithi083.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-17T10:34:29.571 INFO:teuthology.orchestra.run.smithi083.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-17T10:34:29.571 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-17T10:34:29.571 INFO:teuthology.orchestra.run.smithi083.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-17T10:34:29.571 INFO:teuthology.orchestra.run.smithi083.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-17T10:34:29.571 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-17T10:34:29.571 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-17T10:34:29.571 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-17T10:34:29.571 INFO:teuthology.orchestra.run.smithi083.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-17T10:34:29.571 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-17T10:34:29.572 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-17T10:34:29.572 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-17T10:34:29.572 INFO:teuthology.orchestra.run.smithi083.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-17T10:34:29.572 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-17T10:34:29.572 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-17T10:34:29.572 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-17T10:34:29.572 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-17T10:34:29.572 INFO:teuthology.orchestra.run.smithi083.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-17T10:34:29.572 INFO:teuthology.orchestra.run.smithi083.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-17T10:34:29.572 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-17T10:34:29.572 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-17T10:34:29.572 INFO:teuthology.orchestra.run.smithi083.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-17T10:34:29.573 INFO:teuthology.orchestra.run.smithi083.stdout:Suggested packages: 2024-04-17T10:34:29.573 INFO:teuthology.orchestra.run.smithi083.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-17T10:34:29.573 INFO:teuthology.orchestra.run.smithi083.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-17T10:34:29.573 INFO:teuthology.orchestra.run.smithi083.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-17T10:34:29.573 INFO:teuthology.orchestra.run.smithi083.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-17T10:34:29.573 INFO:teuthology.orchestra.run.smithi083.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-17T10:34:29.573 INFO:teuthology.orchestra.run.smithi083.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-17T10:34:29.573 INFO:teuthology.orchestra.run.smithi083.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-17T10:34:29.573 INFO:teuthology.orchestra.run.smithi083.stdout:Recommended packages: 2024-04-17T10:34:29.573 INFO:teuthology.orchestra.run.smithi083.stdout: btrfs-tools 2024-04-17T10:34:29.616 INFO:teuthology.orchestra.run.smithi167.stdout:The following packages were automatically installed and are no longer required: 2024-04-17T10:34:29.616 INFO:teuthology.orchestra.run.smithi167.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-17T10:34:29.616 INFO:teuthology.orchestra.run.smithi167.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-17T10:34:29.616 INFO:teuthology.orchestra.run.smithi167.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-17T10:34:29.617 INFO:teuthology.orchestra.run.smithi167.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-17T10:34:29.617 INFO:teuthology.orchestra.run.smithi167.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-17T10:34:29.617 INFO:teuthology.orchestra.run.smithi167.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-17T10:34:29.618 INFO:teuthology.orchestra.run.smithi167.stdout:The following additional packages will be installed: 2024-04-17T10:34:29.619 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-17T10:34:29.619 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-17T10:34:29.619 INFO:teuthology.orchestra.run.smithi167.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-17T10:34:29.619 INFO:teuthology.orchestra.run.smithi167.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-17T10:34:29.619 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-17T10:34:29.619 INFO:teuthology.orchestra.run.smithi167.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-17T10:34:29.619 INFO:teuthology.orchestra.run.smithi167.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-17T10:34:29.619 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-17T10:34:29.619 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-17T10:34:29.619 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-17T10:34:29.619 INFO:teuthology.orchestra.run.smithi167.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-17T10:34:29.619 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-17T10:34:29.619 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-17T10:34:29.620 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-17T10:34:29.620 INFO:teuthology.orchestra.run.smithi167.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-17T10:34:29.620 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-17T10:34:29.620 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-17T10:34:29.620 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-17T10:34:29.620 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-17T10:34:29.620 INFO:teuthology.orchestra.run.smithi167.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-17T10:34:29.620 INFO:teuthology.orchestra.run.smithi167.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-17T10:34:29.620 INFO:teuthology.orchestra.run.smithi167.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-17T10:34:29.620 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-17T10:34:29.620 INFO:teuthology.orchestra.run.smithi167.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-17T10:34:29.621 INFO:teuthology.orchestra.run.smithi167.stdout:Suggested packages: 2024-04-17T10:34:29.621 INFO:teuthology.orchestra.run.smithi167.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-17T10:34:29.621 INFO:teuthology.orchestra.run.smithi167.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-17T10:34:29.621 INFO:teuthology.orchestra.run.smithi167.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-17T10:34:29.621 INFO:teuthology.orchestra.run.smithi167.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-17T10:34:29.621 INFO:teuthology.orchestra.run.smithi167.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-17T10:34:29.621 INFO:teuthology.orchestra.run.smithi167.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-17T10:34:29.621 INFO:teuthology.orchestra.run.smithi167.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-17T10:34:29.621 INFO:teuthology.orchestra.run.smithi167.stdout:Recommended packages: 2024-04-17T10:34:29.621 INFO:teuthology.orchestra.run.smithi167.stdout: btrfs-tools 2024-04-17T10:34:29.678 INFO:teuthology.orchestra.run.smithi083.stdout:The following NEW packages will be installed: 2024-04-17T10:34:29.678 INFO:teuthology.orchestra.run.smithi083.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-17T10:34:29.678 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-17T10:34:29.678 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-17T10:34:29.678 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-17T10:34:29.680 INFO:teuthology.orchestra.run.smithi083.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-17T10:34:29.680 INFO:teuthology.orchestra.run.smithi083.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-17T10:34:29.680 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-17T10:34:29.680 INFO:teuthology.orchestra.run.smithi083.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-17T10:34:29.680 INFO:teuthology.orchestra.run.smithi083.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-17T10:34:29.680 INFO:teuthology.orchestra.run.smithi083.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-17T10:34:29.680 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-17T10:34:29.680 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-17T10:34:29.680 INFO:teuthology.orchestra.run.smithi083.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-17T10:34:29.681 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-17T10:34:29.681 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-17T10:34:29.681 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-17T10:34:29.681 INFO:teuthology.orchestra.run.smithi083.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-17T10:34:29.681 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-17T10:34:29.681 INFO:teuthology.orchestra.run.smithi083.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-17T10:34:29.681 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-17T10:34:29.681 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-17T10:34:29.681 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-17T10:34:29.681 INFO:teuthology.orchestra.run.smithi083.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-17T10:34:29.681 INFO:teuthology.orchestra.run.smithi083.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-17T10:34:29.682 INFO:teuthology.orchestra.run.smithi083.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-17T10:34:29.682 INFO:teuthology.orchestra.run.smithi083.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-17T10:34:29.682 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-17T10:34:29.682 INFO:teuthology.orchestra.run.smithi083.stdout: socat unzip xmlstarlet zip 2024-04-17T10:34:29.724 INFO:teuthology.orchestra.run.smithi167.stdout:The following NEW packages will be installed: 2024-04-17T10:34:29.724 INFO:teuthology.orchestra.run.smithi167.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-17T10:34:29.724 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-17T10:34:29.725 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-17T10:34:29.725 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-17T10:34:29.725 INFO:teuthology.orchestra.run.smithi167.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-17T10:34:29.725 INFO:teuthology.orchestra.run.smithi167.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-17T10:34:29.725 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-17T10:34:29.725 INFO:teuthology.orchestra.run.smithi167.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-17T10:34:29.726 INFO:teuthology.orchestra.run.smithi167.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-17T10:34:29.726 INFO:teuthology.orchestra.run.smithi167.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-17T10:34:29.726 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-17T10:34:29.726 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-17T10:34:29.726 INFO:teuthology.orchestra.run.smithi167.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-17T10:34:29.727 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-17T10:34:29.727 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-17T10:34:29.727 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-17T10:34:29.727 INFO:teuthology.orchestra.run.smithi167.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-17T10:34:29.727 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-17T10:34:29.727 INFO:teuthology.orchestra.run.smithi167.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-17T10:34:29.727 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-17T10:34:29.727 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-17T10:34:29.727 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-17T10:34:29.727 INFO:teuthology.orchestra.run.smithi167.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-17T10:34:29.727 INFO:teuthology.orchestra.run.smithi167.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-17T10:34:29.727 INFO:teuthology.orchestra.run.smithi167.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-17T10:34:29.727 INFO:teuthology.orchestra.run.smithi167.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-17T10:34:29.727 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-17T10:34:29.728 INFO:teuthology.orchestra.run.smithi167.stdout: socat unzip xmlstarlet zip 2024-04-17T10:34:29.748 INFO:teuthology.orchestra.run.smithi083.stdout:0 upgraded, 124 newly installed, 0 to remove and 302 not upgraded. 2024-04-17T10:34:29.748 INFO:teuthology.orchestra.run.smithi083.stdout:Need to get 176 MB of archives. 2024-04-17T10:34:29.749 INFO:teuthology.orchestra.run.smithi083.stdout:After this operation, 732 MB of additional disk space will be used. 2024-04-17T10:34:29.749 INFO:teuthology.orchestra.run.smithi083.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-17T10:34:29.770 INFO:teuthology.orchestra.run.smithi083.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-17T10:34:29.782 INFO:teuthology.orchestra.run.smithi083.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-17T10:34:29.792 INFO:teuthology.orchestra.run.smithi167.stdout:0 upgraded, 124 newly installed, 0 to remove and 302 not upgraded. 2024-04-17T10:34:29.792 INFO:teuthology.orchestra.run.smithi167.stdout:Need to get 176 MB of archives. 2024-04-17T10:34:29.792 INFO:teuthology.orchestra.run.smithi167.stdout:After this operation, 732 MB of additional disk space will be used. 2024-04-17T10:34:29.792 INFO:teuthology.orchestra.run.smithi167.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-17T10:34:29.813 INFO:teuthology.orchestra.run.smithi167.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-17T10:34:29.825 INFO:teuthology.orchestra.run.smithi167.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-17T10:34:29.831 INFO:teuthology.orchestra.run.smithi083.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-17T10:34:29.857 INFO:teuthology.orchestra.run.smithi083.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-17T10:34:29.860 INFO:teuthology.orchestra.run.smithi083.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-17T10:34:29.869 INFO:teuthology.orchestra.run.smithi167.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,533 kB] 2024-04-17T10:34:29.874 INFO:teuthology.orchestra.run.smithi083.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-17T10:34:29.874 INFO:teuthology.orchestra.run.smithi167.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-17T10:34:29.897 INFO:teuthology.orchestra.run.smithi083.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,533 kB] 2024-04-17T10:34:29.900 INFO:teuthology.orchestra.run.smithi167.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-17T10:34:29.903 INFO:teuthology.orchestra.run.smithi167.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-17T10:34:29.917 INFO:teuthology.orchestra.run.smithi167.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-17T10:34:29.926 INFO:teuthology.orchestra.run.smithi083.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-17T10:34:29.930 INFO:teuthology.orchestra.run.smithi083.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-17T10:34:29.942 INFO:teuthology.orchestra.run.smithi083.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-17T10:34:29.946 INFO:teuthology.orchestra.run.smithi083.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-17T10:34:29.946 INFO:teuthology.orchestra.run.smithi083.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-17T10:34:29.954 INFO:teuthology.orchestra.run.smithi083.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-17T10:34:29.956 INFO:teuthology.orchestra.run.smithi083.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-17T10:34:29.959 INFO:teuthology.orchestra.run.smithi083.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-17T10:34:29.960 INFO:teuthology.orchestra.run.smithi083.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-17T10:34:29.960 INFO:teuthology.orchestra.run.smithi083.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-17T10:34:29.972 INFO:teuthology.orchestra.run.smithi083.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-17T10:34:29.975 INFO:teuthology.orchestra.run.smithi083.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-17T10:34:29.975 INFO:teuthology.orchestra.run.smithi167.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-17T10:34:29.976 INFO:teuthology.orchestra.run.smithi083.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-17T10:34:29.977 INFO:teuthology.orchestra.run.smithi083.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-17T10:34:29.979 INFO:teuthology.orchestra.run.smithi167.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-17T10:34:29.991 INFO:teuthology.orchestra.run.smithi167.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-17T10:34:29.993 INFO:teuthology.orchestra.run.smithi083.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-17T10:34:29.994 INFO:teuthology.orchestra.run.smithi083.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-17T10:34:29.994 INFO:teuthology.orchestra.run.smithi083.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-17T10:34:29.994 INFO:teuthology.orchestra.run.smithi083.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-17T10:34:29.995 INFO:teuthology.orchestra.run.smithi083.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-17T10:34:29.995 INFO:teuthology.orchestra.run.smithi083.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-17T10:34:29.996 INFO:teuthology.orchestra.run.smithi167.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-17T10:34:29.997 INFO:teuthology.orchestra.run.smithi167.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-17T10:34:30.007 INFO:teuthology.orchestra.run.smithi167.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-17T10:34:30.009 INFO:teuthology.orchestra.run.smithi167.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-17T10:34:30.011 INFO:teuthology.orchestra.run.smithi167.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-17T10:34:30.013 INFO:teuthology.orchestra.run.smithi167.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-17T10:34:30.013 INFO:teuthology.orchestra.run.smithi167.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-17T10:34:30.013 INFO:teuthology.orchestra.run.smithi083.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-17T10:34:30.013 INFO:teuthology.orchestra.run.smithi083.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-17T10:34:30.016 INFO:teuthology.orchestra.run.smithi083.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-17T10:34:30.016 INFO:teuthology.orchestra.run.smithi083.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-17T10:34:30.020 INFO:teuthology.orchestra.run.smithi167.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-17T10:34:30.022 INFO:teuthology.orchestra.run.smithi167.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-17T10:34:30.034 INFO:teuthology.orchestra.run.smithi083.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-17T10:34:30.034 INFO:teuthology.orchestra.run.smithi083.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-17T10:34:30.035 INFO:teuthology.orchestra.run.smithi083.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-17T10:34:30.035 INFO:teuthology.orchestra.run.smithi083.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-17T10:34:30.036 INFO:teuthology.orchestra.run.smithi167.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-17T10:34:30.036 INFO:teuthology.orchestra.run.smithi083.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-17T10:34:30.037 INFO:teuthology.orchestra.run.smithi167.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-17T10:34:30.037 INFO:teuthology.orchestra.run.smithi167.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-17T10:34:30.037 INFO:teuthology.orchestra.run.smithi083.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-17T10:34:30.038 INFO:teuthology.orchestra.run.smithi167.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-17T10:34:30.038 INFO:teuthology.orchestra.run.smithi167.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-17T10:34:30.039 INFO:teuthology.orchestra.run.smithi167.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-17T10:34:30.039 INFO:teuthology.orchestra.run.smithi167.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-17T10:34:30.041 INFO:teuthology.orchestra.run.smithi167.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-17T10:34:30.054 INFO:teuthology.orchestra.run.smithi083.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-17T10:34:30.055 INFO:teuthology.orchestra.run.smithi083.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-17T10:34:30.060 INFO:teuthology.orchestra.run.smithi083.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-17T10:34:30.060 INFO:teuthology.orchestra.run.smithi083.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-17T10:34:30.061 INFO:teuthology.orchestra.run.smithi167.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-17T10:34:30.061 INFO:teuthology.orchestra.run.smithi167.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-17T10:34:30.063 INFO:teuthology.orchestra.run.smithi167.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-17T10:34:30.064 INFO:teuthology.orchestra.run.smithi167.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-17T10:34:30.064 INFO:teuthology.orchestra.run.smithi167.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-17T10:34:30.065 INFO:teuthology.orchestra.run.smithi167.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-17T10:34:30.066 INFO:teuthology.orchestra.run.smithi167.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-17T10:34:30.066 INFO:teuthology.orchestra.run.smithi167.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-17T10:34:30.066 INFO:teuthology.orchestra.run.smithi167.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-17T10:34:30.075 INFO:teuthology.orchestra.run.smithi083.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-17T10:34:30.076 INFO:teuthology.orchestra.run.smithi083.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-17T10:34:30.077 INFO:teuthology.orchestra.run.smithi083.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-17T10:34:30.079 INFO:teuthology.orchestra.run.smithi083.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-17T10:34:30.079 INFO:teuthology.orchestra.run.smithi083.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-17T10:34:30.081 INFO:teuthology.orchestra.run.smithi083.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-17T10:34:30.086 INFO:teuthology.orchestra.run.smithi167.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-17T10:34:30.086 INFO:teuthology.orchestra.run.smithi167.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-17T10:34:30.100 INFO:teuthology.orchestra.run.smithi083.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-17T10:34:30.108 INFO:teuthology.orchestra.run.smithi167.stdout:Get:40 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,217 kB] 2024-04-17T10:34:30.112 INFO:teuthology.orchestra.run.smithi167.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-17T10:34:30.116 INFO:teuthology.orchestra.run.smithi083.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-17T10:34:30.117 INFO:teuthology.orchestra.run.smithi167.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-17T10:34:30.117 INFO:teuthology.orchestra.run.smithi167.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-17T10:34:30.118 INFO:teuthology.orchestra.run.smithi083.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-17T10:34:30.118 INFO:teuthology.orchestra.run.smithi167.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-17T10:34:30.118 INFO:teuthology.orchestra.run.smithi083.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-17T10:34:30.118 INFO:teuthology.orchestra.run.smithi167.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-17T10:34:30.119 INFO:teuthology.orchestra.run.smithi167.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-17T10:34:30.121 INFO:teuthology.orchestra.run.smithi167.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-17T10:34:30.122 INFO:teuthology.orchestra.run.smithi167.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-17T10:34:30.123 INFO:teuthology.orchestra.run.smithi083.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-17T10:34:30.148 INFO:teuthology.orchestra.run.smithi167.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-17T10:34:30.149 INFO:teuthology.orchestra.run.smithi167.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-17T10:34:30.175 INFO:teuthology.orchestra.run.smithi083.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-17T10:34:30.176 INFO:teuthology.orchestra.run.smithi083.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-17T10:34:30.177 INFO:teuthology.orchestra.run.smithi083.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-17T10:34:30.177 INFO:teuthology.orchestra.run.smithi083.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-17T10:34:30.178 INFO:teuthology.orchestra.run.smithi083.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-17T10:34:30.179 INFO:teuthology.orchestra.run.smithi167.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-17T10:34:30.180 INFO:teuthology.orchestra.run.smithi167.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-17T10:34:30.180 INFO:teuthology.orchestra.run.smithi167.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-17T10:34:30.183 INFO:teuthology.orchestra.run.smithi167.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-17T10:34:30.222 INFO:teuthology.orchestra.run.smithi083.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-17T10:34:30.224 INFO:teuthology.orchestra.run.smithi083.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-17T10:34:30.225 INFO:teuthology.orchestra.run.smithi083.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-17T10:34:30.247 INFO:teuthology.orchestra.run.smithi167.stdout:Get:55 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1628-g97ae6cc9-1jammy [709 kB] 2024-04-17T10:34:30.261 INFO:teuthology.orchestra.run.smithi083.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-17T10:34:30.262 INFO:teuthology.orchestra.run.smithi083.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-17T10:34:30.268 INFO:teuthology.orchestra.run.smithi167.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-17T10:34:30.270 INFO:teuthology.orchestra.run.smithi167.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-17T10:34:30.272 INFO:teuthology.orchestra.run.smithi167.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-17T10:34:30.272 INFO:teuthology.orchestra.run.smithi167.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-17T10:34:30.274 INFO:teuthology.orchestra.run.smithi167.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-17T10:34:30.275 INFO:teuthology.orchestra.run.smithi083.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-17T10:34:30.275 INFO:teuthology.orchestra.run.smithi083.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-17T10:34:30.276 INFO:teuthology.orchestra.run.smithi083.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-17T10:34:30.276 INFO:teuthology.orchestra.run.smithi083.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-17T10:34:30.276 INFO:teuthology.orchestra.run.smithi083.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-17T10:34:30.277 INFO:teuthology.orchestra.run.smithi083.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-17T10:34:30.278 INFO:teuthology.orchestra.run.smithi167.stdout:Get:61 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1628-g97ae6cc9-1jammy [356 kB] 2024-04-17T10:34:30.281 INFO:teuthology.orchestra.run.smithi083.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-17T10:34:30.296 INFO:teuthology.orchestra.run.smithi167.stdout:Get:62 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1628-g97ae6cc9-1jammy [32.8 kB] 2024-04-17T10:34:30.296 INFO:teuthology.orchestra.run.smithi167.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-17T10:34:30.297 INFO:teuthology.orchestra.run.smithi167.stdout:Get:64 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1628-g97ae6cc9-1jammy [180 kB] 2024-04-17T10:34:30.298 INFO:teuthology.orchestra.run.smithi167.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-17T10:34:30.298 INFO:teuthology.orchestra.run.smithi167.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-17T10:34:30.304 INFO:teuthology.orchestra.run.smithi083.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-17T10:34:30.305 INFO:teuthology.orchestra.run.smithi167.stdout:Get:67 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1628-g97ae6cc9-1jammy [64.8 kB] 2024-04-17T10:34:30.306 INFO:teuthology.orchestra.run.smithi083.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-17T10:34:30.307 INFO:teuthology.orchestra.run.smithi167.stdout:Get:68 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1628-g97ae6cc9-1jammy [330 kB] 2024-04-17T10:34:30.307 INFO:teuthology.orchestra.run.smithi083.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-17T10:34:30.318 INFO:teuthology.orchestra.run.smithi167.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-17T10:34:30.318 INFO:teuthology.orchestra.run.smithi167.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-17T10:34:30.319 INFO:teuthology.orchestra.run.smithi167.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-17T10:34:30.319 INFO:teuthology.orchestra.run.smithi167.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-17T10:34:30.320 INFO:teuthology.orchestra.run.smithi167.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-17T10:34:30.323 INFO:teuthology.orchestra.run.smithi167.stdout:Get:74 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1628-g97ae6cc9-1jammy [6,575 kB] 2024-04-17T10:34:30.326 INFO:teuthology.orchestra.run.smithi083.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-17T10:34:30.329 INFO:teuthology.orchestra.run.smithi083.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-17T10:34:30.330 INFO:teuthology.orchestra.run.smithi083.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-17T10:34:30.331 INFO:teuthology.orchestra.run.smithi083.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-17T10:34:30.331 INFO:teuthology.orchestra.run.smithi083.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-17T10:34:30.337 INFO:teuthology.orchestra.run.smithi083.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-17T10:34:30.338 INFO:teuthology.orchestra.run.smithi167.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-17T10:34:30.338 INFO:teuthology.orchestra.run.smithi083.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-17T10:34:30.339 INFO:teuthology.orchestra.run.smithi167.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-17T10:34:30.339 INFO:teuthology.orchestra.run.smithi167.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-17T10:34:30.342 INFO:teuthology.orchestra.run.smithi167.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-17T10:34:30.347 INFO:teuthology.orchestra.run.smithi083.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-17T10:34:30.347 INFO:teuthology.orchestra.run.smithi083.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-17T10:34:30.348 INFO:teuthology.orchestra.run.smithi083.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-17T10:34:30.359 INFO:teuthology.orchestra.run.smithi167.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-17T10:34:30.360 INFO:teuthology.orchestra.run.smithi167.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-17T10:34:30.361 INFO:teuthology.orchestra.run.smithi167.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-17T10:34:30.364 INFO:teuthology.orchestra.run.smithi167.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-17T10:34:30.367 INFO:teuthology.orchestra.run.smithi167.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-17T10:34:30.367 INFO:teuthology.orchestra.run.smithi167.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-17T10:34:30.369 INFO:teuthology.orchestra.run.smithi083.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-17T10:34:30.372 INFO:teuthology.orchestra.run.smithi083.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-17T10:34:30.373 INFO:teuthology.orchestra.run.smithi083.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-17T10:34:30.381 INFO:teuthology.orchestra.run.smithi167.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-17T10:34:30.382 INFO:teuthology.orchestra.run.smithi167.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-17T10:34:30.384 INFO:teuthology.orchestra.run.smithi083.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-17T10:34:30.385 INFO:teuthology.orchestra.run.smithi083.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-17T10:34:30.385 INFO:teuthology.orchestra.run.smithi083.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-17T10:34:30.387 INFO:teuthology.orchestra.run.smithi167.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-17T10:34:30.387 INFO:teuthology.orchestra.run.smithi167.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-17T10:34:30.389 INFO:teuthology.orchestra.run.smithi083.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-17T10:34:30.390 INFO:teuthology.orchestra.run.smithi083.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-17T10:34:30.401 INFO:teuthology.orchestra.run.smithi083.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,217 kB] 2024-04-17T10:34:30.417 INFO:teuthology.orchestra.run.smithi167.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-17T10:34:30.417 INFO:teuthology.orchestra.run.smithi167.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-17T10:34:30.418 INFO:teuthology.orchestra.run.smithi167.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-17T10:34:30.418 INFO:teuthology.orchestra.run.smithi167.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-17T10:34:30.420 INFO:teuthology.orchestra.run.smithi167.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-17T10:34:30.421 INFO:teuthology.orchestra.run.smithi167.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-17T10:34:30.441 INFO:teuthology.orchestra.run.smithi167.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-17T10:34:30.442 INFO:teuthology.orchestra.run.smithi167.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-17T10:34:30.442 INFO:teuthology.orchestra.run.smithi167.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-17T10:34:30.445 INFO:teuthology.orchestra.run.smithi167.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-17T10:34:30.464 INFO:teuthology.orchestra.run.smithi167.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-17T10:34:30.639 INFO:teuthology.orchestra.run.smithi167.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1628-g97ae6cc9-1jammy [112 kB] 2024-04-17T10:34:30.642 INFO:teuthology.orchestra.run.smithi167.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1628-g97ae6cc9-1jammy [467 kB] 2024-04-17T10:34:30.664 INFO:teuthology.orchestra.run.smithi167.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1628-g97ae6cc9-1jammy [24.1 MB] 2024-04-17T10:34:30.681 INFO:teuthology.orchestra.run.smithi083.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1628-g97ae6cc9-1jammy [709 kB] 2024-04-17T10:34:30.743 INFO:teuthology.orchestra.run.smithi083.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1628-g97ae6cc9-1jammy [356 kB] 2024-04-17T10:34:30.774 INFO:teuthology.orchestra.run.smithi083.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1628-g97ae6cc9-1jammy [32.8 kB] 2024-04-17T10:34:30.776 INFO:teuthology.orchestra.run.smithi083.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1628-g97ae6cc9-1jammy [180 kB] 2024-04-17T10:34:30.791 INFO:teuthology.orchestra.run.smithi083.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1628-g97ae6cc9-1jammy [64.8 kB] 2024-04-17T10:34:30.797 INFO:teuthology.orchestra.run.smithi083.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1628-g97ae6cc9-1jammy [330 kB] 2024-04-17T10:34:30.835 INFO:teuthology.orchestra.run.smithi083.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1628-g97ae6cc9-1jammy [6,575 kB] 2024-04-17T10:34:31.387 INFO:teuthology.orchestra.run.smithi083.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1628-g97ae6cc9-1jammy [112 kB] 2024-04-17T10:34:31.395 INFO:teuthology.orchestra.run.smithi083.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1628-g97ae6cc9-1jammy [467 kB] 2024-04-17T10:34:31.430 INFO:teuthology.orchestra.run.smithi083.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1628-g97ae6cc9-1jammy [24.1 MB] 2024-04-17T10:34:32.129 INFO:teuthology.orchestra.run.smithi167.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1628-g97ae6cc9-1jammy [5,076 kB] 2024-04-17T10:34:32.305 INFO:teuthology.orchestra.run.smithi167.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1628-g97ae6cc9-1jammy [243 kB] 2024-04-17T10:34:32.322 INFO:teuthology.orchestra.run.smithi167.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [124 kB] 2024-04-17T10:34:32.330 INFO:teuthology.orchestra.run.smithi167.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1628-g97ae6cc9-1jammy [1,520 kB] 2024-04-17T10:34:32.417 INFO:teuthology.orchestra.run.smithi167.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1628-g97ae6cc9-1jammy [6,237 kB] 2024-04-17T10:34:32.787 INFO:teuthology.orchestra.run.smithi167.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1628-g97ae6cc9-1jammy [22.7 MB] 2024-04-17T10:34:33.279 INFO:teuthology.orchestra.run.smithi083.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1628-g97ae6cc9-1jammy [5,076 kB] 2024-04-17T10:34:33.591 INFO:teuthology.orchestra.run.smithi083.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1628-g97ae6cc9-1jammy [243 kB] 2024-04-17T10:34:33.609 INFO:teuthology.orchestra.run.smithi083.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [124 kB] 2024-04-17T10:34:33.619 INFO:teuthology.orchestra.run.smithi083.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1628-g97ae6cc9-1jammy [1,520 kB] 2024-04-17T10:34:33.770 INFO:teuthology.orchestra.run.smithi083.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1628-g97ae6cc9-1jammy [6,237 kB] 2024-04-17T10:34:34.153 INFO:teuthology.orchestra.run.smithi167.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [14.0 kB] 2024-04-17T10:34:34.153 INFO:teuthology.orchestra.run.smithi167.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [866 kB] 2024-04-17T10:34:34.171 INFO:teuthology.orchestra.run.smithi167.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1628-g97ae6cc9-1jammy [2,191 kB] 2024-04-17T10:34:34.230 INFO:teuthology.orchestra.run.smithi167.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1628-g97ae6cc9-1jammy [776 kB] 2024-04-17T10:34:34.272 INFO:teuthology.orchestra.run.smithi167.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1628-g97ae6cc9-1jammy [144 kB] 2024-04-17T10:34:34.279 INFO:teuthology.orchestra.run.smithi167.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1628-g97ae6cc9-1jammy [1,921 kB] 2024-04-17T10:34:34.312 INFO:teuthology.orchestra.run.smithi083.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1628-g97ae6cc9-1jammy [22.7 MB] 2024-04-17T10:34:34.378 INFO:teuthology.orchestra.run.smithi167.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1628-g97ae6cc9-1jammy [8,625 kB] 2024-04-17T10:34:34.887 INFO:teuthology.orchestra.run.smithi167.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1628-g97ae6cc9-1jammy [14.2 kB] 2024-04-17T10:34:34.888 INFO:teuthology.orchestra.run.smithi167.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1628-g97ae6cc9-1jammy [49.9 MB] 2024-04-17T10:34:36.045 INFO:teuthology.orchestra.run.smithi083.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [14.0 kB] 2024-04-17T10:34:36.045 INFO:teuthology.orchestra.run.smithi083.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [866 kB] 2024-04-17T10:34:36.062 INFO:teuthology.orchestra.run.smithi083.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1628-g97ae6cc9-1jammy [2,191 kB] 2024-04-17T10:34:36.172 INFO:teuthology.orchestra.run.smithi083.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1628-g97ae6cc9-1jammy [776 kB] 2024-04-17T10:34:36.217 INFO:teuthology.orchestra.run.smithi083.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1628-g97ae6cc9-1jammy [144 kB] 2024-04-17T10:34:36.229 INFO:teuthology.orchestra.run.smithi083.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1628-g97ae6cc9-1jammy [1,921 kB] 2024-04-17T10:34:36.363 INFO:teuthology.orchestra.run.smithi083.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1628-g97ae6cc9-1jammy [8,625 kB] 2024-04-17T10:34:36.978 INFO:teuthology.orchestra.run.smithi083.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1628-g97ae6cc9-1jammy [14.2 kB] 2024-04-17T10:34:36.979 INFO:teuthology.orchestra.run.smithi083.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1628-g97ae6cc9-1jammy [49.9 MB] 2024-04-17T10:34:38.193 INFO:teuthology.orchestra.run.smithi167.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1628-g97ae6cc9-1jammy [121 kB] 2024-04-17T10:34:38.196 INFO:teuthology.orchestra.run.smithi167.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-1628-g97ae6cc9-1jammy [228 kB] 2024-04-17T10:34:38.200 INFO:teuthology.orchestra.run.smithi167.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-1628-g97ae6cc9-1jammy [34.0 kB] 2024-04-17T10:34:38.201 INFO:teuthology.orchestra.run.smithi167.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-1628-g97ae6cc9-1jammy [23.4 kB] 2024-04-17T10:34:38.202 INFO:teuthology.orchestra.run.smithi167.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1628-g97ae6cc9-1jammy [38.4 kB] 2024-04-17T10:34:38.203 INFO:teuthology.orchestra.run.smithi167.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1628-g97ae6cc9-1jammy [13.0 MB] 2024-04-17T10:34:38.879 INFO:teuthology.orchestra.run.smithi167.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [91.5 kB] 2024-04-17T10:34:39.496 INFO:teuthology.orchestra.run.smithi167.stdout:Fetched 176 MB in 9s (19.3 MB/s) 2024-04-17T10:34:39.632 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-17T10:34:39.718 INFO:teuthology.orchestra.run.smithi167.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-17T10:34:39.724 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-17T10:34:39.780 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-17T10:34:39.848 INFO:teuthology.orchestra.run.smithi083.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1628-g97ae6cc9-1jammy [121 kB] 2024-04-17T10:34:39.850 INFO:teuthology.orchestra.run.smithi083.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-1628-g97ae6cc9-1jammy [228 kB] 2024-04-17T10:34:39.855 INFO:teuthology.orchestra.run.smithi083.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-1628-g97ae6cc9-1jammy [34.0 kB] 2024-04-17T10:34:39.856 INFO:teuthology.orchestra.run.smithi083.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-1628-g97ae6cc9-1jammy [23.4 kB] 2024-04-17T10:34:39.857 INFO:teuthology.orchestra.run.smithi083.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1628-g97ae6cc9-1jammy [38.4 kB] 2024-04-17T10:34:39.858 INFO:teuthology.orchestra.run.smithi083.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1628-g97ae6cc9-1jammy [13.0 MB] 2024-04-17T10:34:40.033 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-17T10:34:40.048 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-17T10:34:40.083 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-17T10:34:40.198 INFO:teuthology.orchestra.run.smithi083.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [91.5 kB] 2024-04-17T10:34:40.411 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-cmd2. 2024-04-17T10:34:40.437 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-17T10:34:40.477 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-17T10:34:40.805 INFO:teuthology.orchestra.run.smithi083.stdout:Fetched 176 MB in 11s (16.8 MB/s) 2024-04-17T10:34:40.830 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-17T10:34:40.857 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-17T10:34:40.939 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-17T10:34:40.943 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-17T10:34:41.030 INFO:teuthology.orchestra.run.smithi083.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-17T10:34:41.035 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-17T10:34:41.075 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-17T10:34:41.234 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-17T10:34:41.261 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-17T10:34:41.292 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-17T10:34:41.370 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-17T10:34:41.396 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-17T10:34:41.436 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-17T10:34:41.629 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-17T10:34:41.655 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-17T10:34:41.696 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-17T10:34:41.731 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-cmd2. 2024-04-17T10:34:41.757 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-17T10:34:41.797 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-17T10:34:42.151 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-17T10:34:42.177 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-17T10:34:42.259 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-17T10:34:42.341 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-17T10:34:42.367 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-17T10:34:42.416 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T10:34:42.612 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-17T10:34:42.638 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-17T10:34:42.679 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-17T10:34:42.744 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-17T10:34:42.770 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-17T10:34:42.802 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T10:34:42.982 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-17T10:34:43.008 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-17T10:34:43.040 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-17T10:34:43.156 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-17T10:34:43.182 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-17T10:34:43.222 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T10:34:43.575 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-17T10:34:43.601 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-17T10:34:43.658 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-17T10:34:43.677 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-17T10:34:43.703 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-17T10:34:43.752 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T10:34:43.945 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package librados2. 2024-04-17T10:34:43.971 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../010-librados2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:34:44.003 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:34:44.130 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-17T10:34:44.156 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-17T10:34:44.190 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T10:34:44.448 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package librbd1. 2024-04-17T10:34:44.474 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../011-librbd1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:34:44.514 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:34:44.525 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-17T10:34:44.551 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-17T10:34:44.625 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T10:34:44.968 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libcephfs2. 2024-04-17T10:34:44.969 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-17T10:34:44.994 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:34:44.996 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-17T10:34:45.034 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:34:45.036 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-17T10:34:45.323 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package librados2. 2024-04-17T10:34:45.328 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-rados. 2024-04-17T10:34:45.349 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../010-librados2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:34:45.355 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../013-python3-rados_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:34:45.380 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:34:45.395 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:34:45.681 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-17T10:34:45.708 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-17T10:34:45.739 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:34:45.800 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package librbd1. 2024-04-17T10:34:45.826 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../011-librbd1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:34:45.867 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:34:45.975 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-cephfs. 2024-04-17T10:34:45.990 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:34:46.033 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:34:46.328 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libcephfs2. 2024-04-17T10:34:46.344 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-17T10:34:46.354 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:34:46.371 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-17T10:34:46.394 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:34:46.411 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:34:46.681 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-rados. 2024-04-17T10:34:46.707 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../013-python3-rados_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:34:46.708 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-prettytable. 2024-04-17T10:34:46.734 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-17T10:34:46.747 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:34:46.774 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-17T10:34:47.027 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-rbd. 2024-04-17T10:34:47.041 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-17T10:34:47.053 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:34:47.067 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-17T10:34:47.093 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:34:47.108 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:34:47.360 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-cephfs. 2024-04-17T10:34:47.387 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:34:47.427 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:34:47.430 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-17T10:34:47.458 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-17T10:34:47.505 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-17T10:34:47.738 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-17T10:34:47.765 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-17T10:34:47.774 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-17T10:34:47.790 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-17T10:34:47.805 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:34:47.833 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-17T10:34:48.124 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-prettytable. 2024-04-17T10:34:48.127 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-17T10:34:48.142 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-17T10:34:48.151 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-17T10:34:48.185 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-17T10:34:48.191 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-17T10:34:48.427 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-rbd. 2024-04-17T10:34:48.442 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:34:48.472 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package lua5.1. 2024-04-17T10:34:48.477 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:34:48.487 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-17T10:34:48.522 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-17T10:34:48.826 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package lua-any. 2024-04-17T10:34:48.841 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-17T10:34:48.851 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-17T10:34:48.867 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-17T10:34:48.891 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-17T10:34:48.907 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-17T10:34:49.136 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package zip. 2024-04-17T10:34:49.151 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-17T10:34:49.185 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-17T10:34:49.194 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking zip (3.0-12build2) ... 2024-04-17T10:34:49.208 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-17T10:34:49.251 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-17T10:34:49.506 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package unzip. 2024-04-17T10:34:49.534 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-17T10:34:49.538 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-17T10:34:49.564 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-17T10:34:49.572 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-17T10:34:49.604 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-17T10:34:49.935 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package luarocks. 2024-04-17T10:34:49.961 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-17T10:34:49.966 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package lua5.1. 2024-04-17T10:34:49.993 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-17T10:34:50.001 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-17T10:34:50.033 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-17T10:34:50.342 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package librgw2. 2024-04-17T10:34:50.344 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package lua-any. 2024-04-17T10:34:50.369 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../027-librgw2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:34:50.370 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-17T10:34:50.400 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:34:50.411 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-17T10:34:50.638 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package zip. 2024-04-17T10:34:50.653 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-17T10:34:50.697 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking zip (3.0-12build2) ... 2024-04-17T10:34:50.944 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-rgw. 2024-04-17T10:34:50.959 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:34:51.003 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:34:51.025 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package unzip. 2024-04-17T10:34:51.052 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-17T10:34:51.092 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-17T10:34:51.339 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-17T10:34:51.365 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-17T10:34:51.406 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-17T10:34:51.487 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package luarocks. 2024-04-17T10:34:51.514 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-17T10:34:51.554 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-17T10:34:51.692 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libradosstriper1. 2024-04-17T10:34:51.718 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:34:51.758 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:34:51.937 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package librgw2. 2024-04-17T10:34:51.964 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../027-librgw2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:34:51.995 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:34:52.145 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-common. 2024-04-17T10:34:52.171 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../031-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:34:52.212 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:34:52.531 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-rgw. 2024-04-17T10:34:52.558 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:34:52.597 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:34:52.943 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-17T10:34:52.969 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-17T10:34:53.009 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-17T10:34:53.312 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libradosstriper1. 2024-04-17T10:34:53.338 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:34:53.379 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:34:53.600 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-base. 2024-04-17T10:34:53.627 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../032-ceph-base_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:34:53.693 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:34:53.922 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-common. 2024-04-17T10:34:53.948 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../031-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:34:53.990 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:34:54.229 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-17T10:34:54.256 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-17T10:34:54.296 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-17T10:34:54.574 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-cheroot. 2024-04-17T10:34:54.600 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-17T10:34:54.632 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-17T10:34:54.902 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-17T10:34:54.925 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-17T10:34:54.960 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-17T10:34:55.237 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-base. 2024-04-17T10:34:55.264 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../032-ceph-base_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:34:55.280 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-17T10:34:55.306 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-17T10:34:55.313 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:34:55.346 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-17T10:34:55.658 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-17T10:34:55.684 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-17T10:34:55.724 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-17T10:34:55.867 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-17T10:34:55.893 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-17T10:34:55.933 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-17T10:34:55.985 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-tempora. 2024-04-17T10:34:56.000 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-17T10:34:56.035 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-17T10:34:56.203 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-cheroot. 2024-04-17T10:34:56.229 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-17T10:34:56.261 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-17T10:34:56.321 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-portend. 2024-04-17T10:34:56.348 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-17T10:34:56.388 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-17T10:34:56.548 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-17T10:34:56.574 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-17T10:34:56.614 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-17T10:34:56.691 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-17T10:34:56.717 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-17T10:34:56.757 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-17T10:34:56.951 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-17T10:34:56.977 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-17T10:34:57.017 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-17T10:34:57.077 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-17T10:34:57.103 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-17T10:34:57.143 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-17T10:34:57.320 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-17T10:34:57.346 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-17T10:34:57.387 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-17T10:34:57.489 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-natsort. 2024-04-17T10:34:57.515 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-17T10:34:57.555 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-17T10:34:57.656 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-tempora. 2024-04-17T10:34:57.683 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-17T10:34:57.723 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-17T10:34:57.876 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-logutils. 2024-04-17T10:34:57.901 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-17T10:34:57.942 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-17T10:34:58.018 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-portend. 2024-04-17T10:34:58.044 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-17T10:34:58.084 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-17T10:34:58.220 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-17T10:34:58.246 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-17T10:34:58.278 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-17T10:34:58.404 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-17T10:34:58.430 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-17T10:34:58.471 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-17T10:34:58.564 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-mako. 2024-04-17T10:34:58.590 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-17T10:34:58.630 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-17T10:34:58.740 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-17T10:34:58.755 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-17T10:34:58.790 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-17T10:34:58.959 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-17T10:34:58.985 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-17T10:34:59.025 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-17T10:34:59.127 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-natsort. 2024-04-17T10:34:59.158 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-17T10:34:59.311 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-17T10:34:59.328 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-17T10:34:59.355 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-17T10:34:59.395 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-17T10:34:59.606 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-logutils. 2024-04-17T10:34:59.632 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-17T10:34:59.672 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-17T10:34:59.689 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-webob. 2024-04-17T10:34:59.716 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-17T10:34:59.747 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-17T10:34:59.950 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-17T10:34:59.977 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-17T10:35:00.017 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-17T10:35:00.042 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-waitress. 2024-04-17T10:35:00.069 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-17T10:35:00.112 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-17T10:35:00.320 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-mako. 2024-04-17T10:35:00.346 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-17T10:35:00.386 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-17T10:35:00.420 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-tempita. 2024-04-17T10:35:00.447 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-17T10:35:00.487 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-17T10:35:00.748 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-17T10:35:00.775 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-17T10:35:00.790 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-paste. 2024-04-17T10:35:00.814 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-17T10:35:00.816 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-17T10:35:00.873 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-17T10:35:01.126 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-17T10:35:01.152 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-17T10:35:01.209 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-17T10:35:01.218 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-17T10:35:01.234 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-17T10:35:01.276 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-17T10:35:01.488 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-webob. 2024-04-17T10:35:01.514 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-17T10:35:01.554 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-17T10:35:01.579 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-17T10:35:01.605 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-17T10:35:01.645 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-17T10:35:01.891 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-waitress. 2024-04-17T10:35:01.917 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-17T10:35:01.949 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-webtest. 2024-04-17T10:35:01.960 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-17T10:35:01.975 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-17T10:35:02.015 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-17T10:35:02.269 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-tempita. 2024-04-17T10:35:02.293 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-pecan. 2024-04-17T10:35:02.295 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-17T10:35:02.320 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-17T10:35:02.335 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-17T10:35:02.351 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-17T10:35:02.638 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-paste. 2024-04-17T10:35:02.665 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-17T10:35:02.673 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-certifi. 2024-04-17T10:35:02.700 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-17T10:35:02.704 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-17T10:35:02.731 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-17T10:35:03.051 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-idna. 2024-04-17T10:35:03.077 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-17T10:35:03.267 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-17T10:35:03.293 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-17T10:35:03.378 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-17T10:35:03.692 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-17T10:35:04.095 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-17T10:35:04.121 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-17T10:35:04.161 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-17T10:35:04.387 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-urllib3. 2024-04-17T10:35:04.414 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-17T10:35:04.453 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-17T10:35:04.465 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-webtest. 2024-04-17T10:35:04.491 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-17T10:35:04.531 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-17T10:35:04.765 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-requests. 2024-04-17T10:35:04.791 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-17T10:35:04.826 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pecan. 2024-04-17T10:35:04.833 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-17T10:35:04.853 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-17T10:35:04.892 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-17T10:35:05.143 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-17T10:35:05.169 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-17T10:35:05.209 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-17T10:35:05.231 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-certifi. 2024-04-17T10:35:05.257 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-17T10:35:05.297 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-17T10:35:05.496 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-17T10:35:05.522 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-17T10:35:05.562 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:05.592 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-idna. 2024-04-17T10:35:05.618 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-17T10:35:05.667 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-17T10:35:05.909 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-17T10:35:05.924 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:35:05.961 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-urllib3. 2024-04-17T10:35:05.967 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:05.988 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-17T10:35:06.028 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-17T10:35:06.278 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-17T10:35:06.305 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-17T10:35:06.345 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-17T10:35:06.423 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-requests. 2024-04-17T10:35:06.450 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-17T10:35:06.490 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-17T10:35:06.706 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-openssl. 2024-04-17T10:35:06.733 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-17T10:35:06.772 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-17T10:35:06.801 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-17T10:35:06.828 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-17T10:35:06.868 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-17T10:35:07.126 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-mgr. 2024-04-17T10:35:07.152 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:35:07.162 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-17T10:35:07.178 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-17T10:35:07.192 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:07.220 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:07.595 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-mon. 2024-04-17T10:35:07.600 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-17T10:35:07.622 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:35:07.626 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:35:07.662 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:07.667 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:07.979 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-17T10:35:08.005 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-17T10:35:08.045 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-17T10:35:08.190 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-osd. 2024-04-17T10:35:08.217 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:35:08.256 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:08.348 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-openssl. 2024-04-17T10:35:08.375 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-17T10:35:08.415 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-17T10:35:08.726 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mgr. 2024-04-17T10:35:08.753 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:35:08.793 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:09.180 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mon. 2024-04-17T10:35:09.206 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:35:09.246 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:09.310 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph. 2024-04-17T10:35:09.325 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../068-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:35:09.368 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:09.687 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-fuse. 2024-04-17T10:35:09.714 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:35:09.754 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:09.850 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-osd. 2024-04-17T10:35:09.876 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:35:09.916 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:10.090 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-mds. 2024-04-17T10:35:10.117 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:35:10.149 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:10.535 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package cephadm. 2024-04-17T10:35:10.562 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../071-cephadm_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:35:10.601 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:10.894 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph. 2024-04-17T10:35:10.910 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../068-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:35:10.912 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-17T10:35:10.939 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-17T10:35:10.952 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:10.979 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-17T10:35:11.290 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-17T10:35:11.305 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-17T10:35:11.339 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-17T10:35:11.389 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-fuse. 2024-04-17T10:35:11.415 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:35:11.455 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:11.817 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mds. 2024-04-17T10:35:11.843 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:35:11.883 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:12.253 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package cephadm. 2024-04-17T10:35:12.280 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../071-cephadm_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:35:12.311 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:12.320 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-babel. 2024-04-17T10:35:12.347 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-17T10:35:12.389 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-17T10:35:12.597 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-17T10:35:12.624 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-17T10:35:12.664 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-17T10:35:12.706 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-jinja2. 2024-04-17T10:35:12.733 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-17T10:35:12.772 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-17T10:35:12.958 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-17T10:35:12.974 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-17T10:35:13.008 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-17T10:35:13.109 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-17T10:35:13.136 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-17T10:35:13.175 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:13.520 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-17T10:35:13.547 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-17T10:35:13.587 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-17T10:35:14.039 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-babel. 2024-04-17T10:35:14.066 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-17T10:35:14.073 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-routes. 2024-04-17T10:35:14.100 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-17T10:35:14.105 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-17T10:35:14.139 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-17T10:35:14.433 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-jinja2. 2024-04-17T10:35:14.460 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-17T10:35:14.460 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-17T10:35:14.486 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-17T10:35:14.526 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:14.546 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-17T10:35:14.929 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-17T10:35:14.945 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-17T10:35:14.987 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-17T10:35:15.061 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-17T10:35:15.088 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-17T10:35:15.128 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:15.466 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-joblib. 2024-04-17T10:35:15.490 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-17T10:35:15.492 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-17T10:35:15.516 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-17T10:35:15.532 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-17T10:35:15.556 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-17T10:35:15.859 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-routes. 2024-04-17T10:35:15.861 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-17T10:35:15.875 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-17T10:35:15.888 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-17T10:35:15.917 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-17T10:35:16.004 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-17T10:35:16.246 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-17T10:35:16.272 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-17T10:35:16.307 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-sklearn. 2024-04-17T10:35:16.312 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:16.334 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-17T10:35:16.374 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-17T10:35:16.740 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-17T10:35:16.756 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-17T10:35:16.798 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-17T10:35:17.091 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-17T10:35:17.119 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-17T10:35:17.158 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:17.336 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-joblib. 2024-04-17T10:35:17.363 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-17T10:35:17.402 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-17T10:35:17.773 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-17T10:35:17.800 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-17T10:35:17.925 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-17T10:35:18.103 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-cachetools. 2024-04-17T10:35:18.130 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-17T10:35:18.169 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-17T10:35:18.228 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-sklearn. 2024-04-17T10:35:18.254 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-17T10:35:18.294 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-17T10:35:18.430 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-17T10:35:18.457 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-17T10:35:18.488 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-17T10:35:18.767 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-17T10:35:18.794 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-17T10:35:18.833 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-17T10:35:19.037 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-17T10:35:19.065 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-17T10:35:19.103 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:19.178 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-rsa. 2024-04-17T10:35:19.205 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-17T10:35:19.261 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-17T10:35:19.541 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-google-auth. 2024-04-17T10:35:19.567 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-17T10:35:19.598 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-17T10:35:19.893 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-17T10:35:19.920 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-17T10:35:19.960 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-17T10:35:20.091 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-cachetools. 2024-04-17T10:35:20.118 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-17T10:35:20.157 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-17T10:35:20.288 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-websocket. 2024-04-17T10:35:20.315 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-17T10:35:20.354 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-17T10:35:20.469 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-17T10:35:20.496 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-17T10:35:20.535 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-17T10:35:20.666 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-17T10:35:20.693 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-17T10:35:20.732 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-17T10:35:20.847 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-17T10:35:20.874 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-17T10:35:20.905 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-17T10:35:21.200 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-rsa. 2024-04-17T10:35:21.228 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-17T10:35:21.266 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-17T10:35:21.363 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-17T10:35:21.391 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-17T10:35:21.429 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:21.595 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-google-auth. 2024-04-17T10:35:21.623 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-17T10:35:21.662 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-17T10:35:21.724 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-17T10:35:21.752 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-17T10:35:21.790 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-17T10:35:22.007 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-17T10:35:22.034 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-17T10:35:22.073 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-17T10:35:22.102 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-17T10:35:22.130 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-17T10:35:22.169 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-17T10:35:22.360 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-websocket. 2024-04-17T10:35:22.387 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-17T10:35:22.413 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package jq. 2024-04-17T10:35:22.426 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-17T10:35:22.441 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-17T10:35:22.479 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-17T10:35:22.722 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-17T10:35:22.732 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package socat. 2024-04-17T10:35:22.749 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-17T10:35:22.759 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-17T10:35:22.788 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-17T10:35:22.798 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-17T10:35:23.127 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package xmlstarlet. 2024-04-17T10:35:23.143 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-17T10:35:23.185 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-17T10:35:23.444 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-17T10:35:23.472 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-17T10:35:23.510 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:23.539 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-test. 2024-04-17T10:35:23.567 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../099-ceph-test_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:35:23.631 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:23.841 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-17T10:35:23.866 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-17T10:35:23.905 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-17T10:35:24.208 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-17T10:35:24.236 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-17T10:35:24.274 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-17T10:35:24.686 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package jq. 2024-04-17T10:35:24.714 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-17T10:35:24.893 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-17T10:35:25.230 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package socat. 2024-04-17T10:35:25.257 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-17T10:35:25.296 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-17T10:35:25.625 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package xmlstarlet. 2024-04-17T10:35:25.652 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-17T10:35:25.691 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-17T10:35:26.039 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-test. 2024-04-17T10:35:26.055 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../099-ceph-test_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:35:26.062 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-volume. 2024-04-17T10:35:26.089 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-17T10:35:26.097 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:26.128 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:26.508 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-17T10:35:26.536 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:35:26.574 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:26.885 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-colorama. 2024-04-17T10:35:26.913 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-17T10:35:26.969 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-17T10:35:27.221 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package cephfs-shell. 2024-04-17T10:35:27.237 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-17T10:35:27.271 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:27.532 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package cephfs-top. 2024-04-17T10:35:27.548 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-17T10:35:27.591 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:27.843 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-17T10:35:27.871 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:35:27.910 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:28.238 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-17T10:35:28.265 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-17T10:35:28.302 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-volume. 2024-04-17T10:35:28.304 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-17T10:35:28.329 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-17T10:35:28.368 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:28.609 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-17T10:35:28.625 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-17T10:35:28.659 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-17T10:35:28.715 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-17T10:35:28.742 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:35:28.781 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:29.013 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package nvme-cli. 2024-04-17T10:35:29.040 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-17T10:35:29.079 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-17T10:35:29.109 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-colorama. 2024-04-17T10:35:29.137 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-17T10:35:29.175 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-17T10:35:29.411 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package cephfs-shell. 2024-04-17T10:35:29.428 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-17T10:35:29.461 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:29.510 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package pkg-config. 2024-04-17T10:35:29.537 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-17T10:35:29.576 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-17T10:35:29.714 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package cephfs-top. 2024-04-17T10:35:29.742 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-17T10:35:29.781 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:29.787 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-17T10:35:29.804 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-17T10:35:29.846 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-17T10:35:30.033 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-17T10:35:30.049 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:35:30.092 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:30.291 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-17T10:35:30.319 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-17T10:35:30.349 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-17T10:35:30.411 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-17T10:35:30.439 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-17T10:35:30.477 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-17T10:35:30.627 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-pastescript. 2024-04-17T10:35:30.654 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-17T10:35:30.693 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-17T10:35:30.774 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-17T10:35:30.802 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-17T10:35:30.832 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-17T10:35:30.997 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-pluggy. 2024-04-17T10:35:31.024 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-17T10:35:31.063 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-17T10:35:31.186 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package nvme-cli. 2024-04-17T10:35:31.214 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-17T10:35:31.253 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-17T10:35:31.375 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-psutil. 2024-04-17T10:35:31.402 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-17T10:35:31.441 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-17T10:35:31.667 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package pkg-config. 2024-04-17T10:35:31.695 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-17T10:35:31.733 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-17T10:35:31.786 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-py. 2024-04-17T10:35:31.813 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-17T10:35:31.852 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-17T10:35:32.011 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-17T10:35:32.027 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-17T10:35:32.069 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-17T10:35:32.198 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-pygments. 2024-04-17T10:35:32.226 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-17T10:35:32.264 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-17T10:35:32.481 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-17T10:35:32.509 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-17T10:35:32.547 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-17T10:35:32.744 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-17T10:35:32.771 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-17T10:35:32.810 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-17T10:35:32.875 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pastescript. 2024-04-17T10:35:32.903 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-17T10:35:32.942 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-17T10:35:33.088 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-toml. 2024-04-17T10:35:33.104 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-17T10:35:33.146 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-17T10:35:33.287 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pluggy. 2024-04-17T10:35:33.315 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-17T10:35:33.353 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-17T10:35:33.450 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-pytest. 2024-04-17T10:35:33.476 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-17T10:35:33.515 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-17T10:35:33.632 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-psutil. 2024-04-17T10:35:33.660 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-17T10:35:33.690 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-17T10:35:33.853 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-simplejson. 2024-04-17T10:35:33.880 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-17T10:35:33.910 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-17T10:35:34.026 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-py. 2024-04-17T10:35:34.042 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-17T10:35:34.076 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-17T10:35:34.163 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-17T10:35:34.179 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-17T10:35:34.222 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-17T10:35:34.388 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pygments. 2024-04-17T10:35:34.416 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-17T10:35:34.446 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-17T10:35:34.886 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package radosgw. 2024-04-17T10:35:34.892 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-17T10:35:34.912 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../122-radosgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:35:34.920 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-17T10:35:34.950 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:34.958 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-17T10:35:35.261 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-toml. 2024-04-17T10:35:35.289 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-17T10:35:35.328 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-17T10:35:35.639 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pytest. 2024-04-17T10:35:35.668 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-17T10:35:35.706 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-17T10:35:35.762 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package rbd-fuse. 2024-04-17T10:35:35.790 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:35:35.829 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:36.043 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-simplejson. 2024-04-17T10:35:36.070 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-17T10:35:36.109 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-17T10:35:36.151 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-17T10:35:36.384 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-17T10:35:36.403 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-17T10:35:36.420 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-17T10:35:36.462 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-17T10:35:36.510 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-17T10:35:36.767 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-17T10:35:37.085 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-17T10:35:37.125 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package radosgw. 2024-04-17T10:35:37.152 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../122-radosgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:35:37.182 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:37.503 INFO:teuthology.orchestra.run.smithi167.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-17T10:35:37.579 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:37.962 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package rbd-fuse. 2024-04-17T10:35:37.988 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-17T10:35:38.027 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:38.045 INFO:teuthology.orchestra.run.smithi167.stdout:Adding system user cephadm....done 2024-04-17T10:35:38.194 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-17T10:35:38.349 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-17T10:35:38.479 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-17T10:35:38.591 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-17T10:35:38.700 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-17T10:35:38.739 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-17T10:35:38.865 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-17T10:35:38.965 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-17T10:35:39.124 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-17T10:35:39.306 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-17T10:35:39.392 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-17T10:35:39.509 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-17T10:35:39.750 INFO:teuthology.orchestra.run.smithi083.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-17T10:35:40.193 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-17T10:35:40.193 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:40.214 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-17T10:35:40.369 INFO:teuthology.orchestra.run.smithi083.stdout:Adding system user cephadm....done 2024-04-17T10:35:40.481 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-17T10:35:40.483 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-17T10:35:40.724 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-17T10:35:40.760 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-17T10:35:40.975 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-17T10:35:41.019 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-17T10:35:41.146 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-17T10:35:41.244 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-17T10:35:41.405 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-17T10:35:41.503 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-17T10:35:41.656 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-17T10:35:41.771 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-17T10:35:41.774 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-17T10:35:42.027 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:42.068 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-17T10:35:42.340 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-17T10:35:42.428 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-17T10:35:42.466 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-17T10:35:42.592 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-17T10:35:42.688 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-17T10:35:42.852 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-17T10:35:42.947 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-17T10:35:42.978 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-17T10:35:43.232 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-17T10:35:43.237 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-17T10:35:43.542 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-17T10:35:43.643 INFO:teuthology.orchestra.run.smithi167.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-17T10:35:43.686 INFO:teuthology.orchestra.run.smithi167.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-17T10:35:43.749 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-17T10:35:43.793 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-17T10:35:43.875 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-17T10:35:44.052 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-17T10:35:44.235 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-17T10:35:44.303 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:44.361 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-17T10:35:44.588 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-17T10:35:44.620 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-17T10:35:44.706 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-17T10:35:44.807 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-17T10:35:44.863 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-17T10:35:45.050 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-17T10:35:45.156 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up zip (3.0-12build2) ... 2024-04-17T10:35:45.167 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-17T10:35:45.283 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-17T10:35:45.440 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-17T10:35:45.918 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-17T10:35:45.953 INFO:teuthology.orchestra.run.smithi083.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-17T10:35:45.976 INFO:teuthology.orchestra.run.smithi083.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-17T10:35:46.038 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-17T10:35:46.156 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-17T10:35:46.169 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-17T10:35:46.295 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-17T10:35:46.421 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-17T10:35:46.499 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-17T10:35:46.608 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-17T10:35:46.697 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-17T10:35:46.876 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-17T10:35:47.048 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-17T10:35:47.144 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-17T10:35:47.299 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-17T10:35:47.437 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up zip (3.0-12build2) ... 2024-04-17T10:35:47.572 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-17T10:35:47.676 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-17T10:35:47.969 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-17T10:35:48.257 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-17T10:35:48.295 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-17T10:35:48.508 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-17T10:35:48.622 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-17T10:35:48.634 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-17T10:35:48.760 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-17T10:35:48.856 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-17T10:35:48.982 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:49.053 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-17T10:35:49.267 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-idna (3.3-1) ... 2024-04-17T10:35:49.438 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-17T10:35:49.576 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-17T10:35:49.697 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-17T10:35:50.065 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-17T10:35:50.342 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-17T10:35:50.643 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-17T10:35:50.716 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-17T10:35:50.976 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T10:35:50.994 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-17T10:35:51.110 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-17T10:35:51.254 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-17T10:35:51.379 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-17T10:35:51.389 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:35:51.488 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-17T10:35:51.597 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-17T10:35:51.682 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-idna (3.3-1) ... 2024-04-17T10:35:51.882 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-17T10:35:51.991 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-17T10:35:52.133 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-17T10:35:52.384 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T10:35:52.510 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-17T10:35:52.804 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-17T10:35:53.038 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-17T10:35:53.164 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-17T10:35:53.281 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-17T10:35:53.399 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T10:35:53.516 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-17T10:35:53.549 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-17T10:35:53.776 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-17T10:35:53.800 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-17T10:35:53.902 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-17T10:35:54.028 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-17T10:35:54.067 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-17T10:35:54.193 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-17T10:35:54.322 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-17T10:35:54.478 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-17T10:35:54.573 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-17T10:35:54.596 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-17T10:35:54.807 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T10:35:54.872 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-17T10:35:54.916 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-17T10:35:55.173 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-17T10:35:55.218 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-17T10:35:55.425 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-17T10:35:55.469 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-17T10:35:55.553 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-17T10:35:55.676 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-17T10:35:55.766 INFO:teuthology.orchestra.run.smithi167.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-17T10:35:55.854 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-17T10:35:55.995 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-17T10:35:56.096 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-17T10:35:56.438 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-17T10:35:56.464 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-17T10:35:56.690 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-17T10:35:56.715 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-17T10:35:56.841 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-17T10:35:56.932 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-17T10:35:57.042 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-17T10:35:57.126 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T10:35:57.252 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-17T10:35:57.318 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-17T10:35:57.444 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-17T10:35:57.570 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-17T10:35:57.737 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-17T10:35:57.854 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-17T10:35:58.047 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-17T10:35:58.114 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-17T10:35:58.289 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-17T10:35:58.325 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-17T10:35:58.390 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-17T10:35:58.500 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-17T10:35:58.594 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-17T10:35:58.605 INFO:teuthology.orchestra.run.smithi083.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-17T10:35:58.835 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-17T10:35:58.861 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-17T10:35:58.961 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-17T10:35:59.113 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-17T10:35:59.363 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-17T10:35:59.473 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-17T10:35:59.634 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-17T10:35:59.757 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-17T10:35:59.766 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-17T10:35:59.999 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T10:36:00.110 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-17T10:36:00.125 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-17T10:36:00.443 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-17T10:36:00.544 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:00.744 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-17T10:36:00.780 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:00.906 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-17T10:36:01.021 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-17T10:36:01.192 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:01.224 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-17T10:36:01.318 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:01.444 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-17T10:36:01.492 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-17T10:36:01.721 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:01.768 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-17T10:36:01.830 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:01.956 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:02.028 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-17T10:36:02.171 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-17T10:36:02.171 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-17T10:36:02.405 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-17T10:36:02.641 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:02.690 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-17T10:36:02.767 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:02.910 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:03.009 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-17T10:36:03.044 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:03.135 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:03.170 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-17T10:36:03.261 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:03.387 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-17T10:36:03.707 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:03.825 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:03.951 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-17T10:36:04.227 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:04.354 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:04.357 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:04.458 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:04.479 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:04.576 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:04.702 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:04.709 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-17T10:36:04.709 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-17T10:36:04.828 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:04.954 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:05.214 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:05.324 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:05.433 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:05.497 INFO:teuthology.orchestra.run.smithi167.stdout:Adding group ceph....done 2024-04-17T10:36:05.559 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:05.685 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-17T10:36:05.849 INFO:teuthology.orchestra.run.smithi167.stdout:Adding system user ceph....done 2024-04-17T10:36:05.959 INFO:teuthology.orchestra.run.smithi167.stdout:Setting system user ceph properties....done 2024-04-17T10:36:05.971 INFO:teuthology.orchestra.run.smithi167.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-17T10:36:06.112 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-17T10:36:06.415 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-17T10:36:06.889 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:06.898 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:07.007 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:07.101 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-17T10:36:07.102 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-17T10:36:07.116 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:07.242 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:07.368 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:07.494 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:07.557 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:07.692 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:08.038 INFO:teuthology.orchestra.run.smithi083.stdout:Adding group ceph....done 2024-04-17T10:36:08.110 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-17T10:36:08.110 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-17T10:36:08.422 INFO:teuthology.orchestra.run.smithi083.stdout:Adding system user ceph....done 2024-04-17T10:36:08.560 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:08.617 INFO:teuthology.orchestra.run.smithi083.stdout:Setting system user ceph properties....done 2024-04-17T10:36:08.634 INFO:teuthology.orchestra.run.smithi083.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-17T10:36:08.769 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-17T10:36:08.902 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-17T10:36:09.074 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-17T10:36:09.346 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:09.575 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-17T10:36:09.575 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-17T10:36:09.613 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:09.824 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-17T10:36:09.824 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-17T10:36:09.997 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:10.209 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-17T10:36:10.209 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-17T10:36:10.281 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:10.390 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:10.674 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:10.845 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-17T10:36:10.845 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-17T10:36:10.938 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-17T10:36:10.947 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-17T10:36:11.318 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:11.393 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:11.502 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:11.654 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:11.660 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-17T10:36:11.861 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-17T10:36:11.861 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-17T10:36:12.153 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:12.338 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:12.373 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-17T10:36:12.373 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-17T10:36:12.481 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:12.607 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:12.825 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:12.872 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:13.098 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-17T10:36:13.098 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-17T10:36:13.211 INFO:teuthology.orchestra.run.smithi167.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-17T10:36:13.304 INFO:teuthology.orchestra.run.smithi167.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-17T10:36:13.458 INFO:teuthology.orchestra.run.smithi167.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-17T10:36:13.565 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:13.812 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-17T10:36:13.813 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-17T10:36:14.283 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:14.426 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:14.535 INFO:teuthology.orchestra.run.smithi167.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-17T10:36:14.577 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:14.788 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-17T10:36:14.788 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-17T10:36:15.160 INFO:teuthology.orchestra.run.smithi167.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-17T10:36:15.164 DEBUG:teuthology.orchestra.run.smithi167:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-17T10:36:15.261 INFO:teuthology.orchestra.run.smithi167.stdout:Reading package lists... 2024-04-17T10:36:15.286 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:15.438 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:15.459 INFO:teuthology.orchestra.run.smithi167.stdout:Building dependency tree... 2024-04-17T10:36:15.460 INFO:teuthology.orchestra.run.smithi167.stdout:Reading state information... 2024-04-17T10:36:15.556 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:15.651 INFO:teuthology.orchestra.run.smithi167.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-17T10:36:15.651 INFO:teuthology.orchestra.run.smithi167.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-17T10:36:15.651 INFO:teuthology.orchestra.run.smithi167.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-17T10:36:15.651 INFO:teuthology.orchestra.run.smithi167.stdout:The following packages were automatically installed and are no longer required: 2024-04-17T10:36:15.651 INFO:teuthology.orchestra.run.smithi167.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-17T10:36:15.651 INFO:teuthology.orchestra.run.smithi167.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-17T10:36:15.651 INFO:teuthology.orchestra.run.smithi167.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-17T10:36:15.652 INFO:teuthology.orchestra.run.smithi167.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-17T10:36:15.652 INFO:teuthology.orchestra.run.smithi167.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-17T10:36:15.652 INFO:teuthology.orchestra.run.smithi167.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-17T10:36:15.653 INFO:teuthology.orchestra.run.smithi167.stdout:The following additional packages will be installed: 2024-04-17T10:36:15.653 INFO:teuthology.orchestra.run.smithi167.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-17T10:36:15.653 INFO:teuthology.orchestra.run.smithi167.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-17T10:36:15.653 INFO:teuthology.orchestra.run.smithi167.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-17T10:36:15.654 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-17T10:36:15.654 INFO:teuthology.orchestra.run.smithi167.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-17T10:36:15.655 INFO:teuthology.orchestra.run.smithi167.stdout:Suggested packages: 2024-04-17T10:36:15.655 INFO:teuthology.orchestra.run.smithi167.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-17T10:36:15.655 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-17T10:36:15.655 INFO:teuthology.orchestra.run.smithi167.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-17T10:36:15.695 INFO:teuthology.orchestra.run.smithi167.stdout:The following NEW packages will be installed: 2024-04-17T10:36:15.696 INFO:teuthology.orchestra.run.smithi167.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-17T10:36:15.696 INFO:teuthology.orchestra.run.smithi167.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-17T10:36:15.696 INFO:teuthology.orchestra.run.smithi167.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-17T10:36:15.696 INFO:teuthology.orchestra.run.smithi167.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-17T10:36:15.696 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-17T10:36:15.696 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-17T10:36:15.697 INFO:teuthology.orchestra.run.smithi167.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-17T10:36:15.697 INFO:teuthology.orchestra.run.smithi167.stdout:The following packages will be upgraded: 2024-04-17T10:36:15.698 INFO:teuthology.orchestra.run.smithi167.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-17T10:36:15.702 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-17T10:36:15.878 INFO:teuthology.orchestra.run.smithi167.stdout:4 upgraded, 39 newly installed, 0 to remove and 298 not upgraded. 2024-04-17T10:36:15.878 INFO:teuthology.orchestra.run.smithi167.stdout:Need to get 55.7 MB of archives. 2024-04-17T10:36:15.878 INFO:teuthology.orchestra.run.smithi167.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-17T10:36:15.878 INFO:teuthology.orchestra.run.smithi167.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-17T10:36:16.035 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-17T10:36:16.144 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-17T10:36:16.297 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-17T10:36:16.505 INFO:teuthology.orchestra.run.smithi167.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-17T10:36:16.522 INFO:teuthology.orchestra.run.smithi167.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-17T10:36:16.646 INFO:teuthology.orchestra.run.smithi167.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-17T10:36:16.647 INFO:teuthology.orchestra.run.smithi167.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-17T10:36:16.649 INFO:teuthology.orchestra.run.smithi167.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-17T10:36:16.677 INFO:teuthology.orchestra.run.smithi167.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-17T10:36:16.678 INFO:teuthology.orchestra.run.smithi167.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-17T10:36:16.682 INFO:teuthology.orchestra.run.smithi167.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-17T10:36:16.684 INFO:teuthology.orchestra.run.smithi167.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-17T10:36:16.685 INFO:teuthology.orchestra.run.smithi167.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-17T10:36:16.851 INFO:teuthology.orchestra.run.smithi167.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-17T10:36:17.144 INFO:teuthology.orchestra.run.smithi167.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-17T10:36:17.154 INFO:teuthology.orchestra.run.smithi167.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-17T10:36:17.159 INFO:teuthology.orchestra.run.smithi167.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-17T10:36:17.244 INFO:teuthology.orchestra.run.smithi167.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-17T10:36:17.442 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-17T10:36:18.090 INFO:teuthology.orchestra.run.smithi083.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-17T10:36:18.093 DEBUG:teuthology.orchestra.run.smithi083:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-17T10:36:18.146 INFO:teuthology.orchestra.run.smithi083.stdout:Reading package lists... 2024-04-17T10:36:18.329 INFO:teuthology.orchestra.run.smithi167.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-17T10:36:18.333 INFO:teuthology.orchestra.run.smithi167.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-17T10:36:18.333 INFO:teuthology.orchestra.run.smithi167.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-17T10:36:18.334 INFO:teuthology.orchestra.run.smithi083.stdout:Building dependency tree... 2024-04-17T10:36:18.335 INFO:teuthology.orchestra.run.smithi083.stdout:Reading state information... 2024-04-17T10:36:18.339 INFO:teuthology.orchestra.run.smithi167.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-17T10:36:18.385 INFO:teuthology.orchestra.run.smithi167.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-17T10:36:18.396 INFO:teuthology.orchestra.run.smithi167.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-17T10:36:18.502 INFO:teuthology.orchestra.run.smithi083.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-17T10:36:18.502 INFO:teuthology.orchestra.run.smithi083.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-17T10:36:18.502 INFO:teuthology.orchestra.run.smithi083.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-17T10:36:18.502 INFO:teuthology.orchestra.run.smithi083.stdout:The following packages were automatically installed and are no longer required: 2024-04-17T10:36:18.502 INFO:teuthology.orchestra.run.smithi083.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-17T10:36:18.502 INFO:teuthology.orchestra.run.smithi083.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-17T10:36:18.503 INFO:teuthology.orchestra.run.smithi083.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-17T10:36:18.503 INFO:teuthology.orchestra.run.smithi083.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-17T10:36:18.503 INFO:teuthology.orchestra.run.smithi083.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-17T10:36:18.503 INFO:teuthology.orchestra.run.smithi083.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-17T10:36:18.504 INFO:teuthology.orchestra.run.smithi083.stdout:The following additional packages will be installed: 2024-04-17T10:36:18.504 INFO:teuthology.orchestra.run.smithi083.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-17T10:36:18.504 INFO:teuthology.orchestra.run.smithi083.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-17T10:36:18.504 INFO:teuthology.orchestra.run.smithi083.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-17T10:36:18.505 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-17T10:36:18.505 INFO:teuthology.orchestra.run.smithi083.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-17T10:36:18.505 INFO:teuthology.orchestra.run.smithi083.stdout:Suggested packages: 2024-04-17T10:36:18.506 INFO:teuthology.orchestra.run.smithi083.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-17T10:36:18.506 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-17T10:36:18.506 INFO:teuthology.orchestra.run.smithi083.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-17T10:36:18.545 INFO:teuthology.orchestra.run.smithi083.stdout:The following NEW packages will be installed: 2024-04-17T10:36:18.546 INFO:teuthology.orchestra.run.smithi083.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-17T10:36:18.546 INFO:teuthology.orchestra.run.smithi083.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-17T10:36:18.547 INFO:teuthology.orchestra.run.smithi083.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-17T10:36:18.547 INFO:teuthology.orchestra.run.smithi083.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-17T10:36:18.547 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-17T10:36:18.547 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-17T10:36:18.547 INFO:teuthology.orchestra.run.smithi083.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-17T10:36:18.548 INFO:teuthology.orchestra.run.smithi083.stdout:The following packages will be upgraded: 2024-04-17T10:36:18.548 INFO:teuthology.orchestra.run.smithi083.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-17T10:36:18.563 INFO:teuthology.orchestra.run.smithi167.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-17T10:36:18.632 INFO:teuthology.orchestra.run.smithi083.stdout:4 upgraded, 39 newly installed, 0 to remove and 298 not upgraded. 2024-04-17T10:36:18.633 INFO:teuthology.orchestra.run.smithi083.stdout:Need to get 55.7 MB of archives. 2024-04-17T10:36:18.633 INFO:teuthology.orchestra.run.smithi083.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-17T10:36:18.633 INFO:teuthology.orchestra.run.smithi083.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-17T10:36:18.837 INFO:teuthology.orchestra.run.smithi083.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-17T10:36:18.844 INFO:teuthology.orchestra.run.smithi083.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-17T10:36:18.884 INFO:teuthology.orchestra.run.smithi083.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-17T10:36:18.885 INFO:teuthology.orchestra.run.smithi083.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-17T10:36:18.885 INFO:teuthology.orchestra.run.smithi083.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-17T10:36:18.896 INFO:teuthology.orchestra.run.smithi083.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-17T10:36:18.897 INFO:teuthology.orchestra.run.smithi083.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-17T10:36:18.899 INFO:teuthology.orchestra.run.smithi083.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-17T10:36:18.901 INFO:teuthology.orchestra.run.smithi083.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-17T10:36:18.901 INFO:teuthology.orchestra.run.smithi083.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-17T10:36:18.903 INFO:teuthology.orchestra.run.smithi083.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-17T10:36:18.904 INFO:teuthology.orchestra.run.smithi083.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-17T10:36:18.924 INFO:teuthology.orchestra.run.smithi083.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-17T10:36:18.924 INFO:teuthology.orchestra.run.smithi083.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-17T10:36:18.925 INFO:teuthology.orchestra.run.smithi083.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-17T10:36:19.158 INFO:teuthology.orchestra.run.smithi167.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-17T10:36:19.159 INFO:teuthology.orchestra.run.smithi167.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-17T10:36:19.312 INFO:teuthology.orchestra.run.smithi083.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-17T10:36:19.314 INFO:teuthology.orchestra.run.smithi083.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-17T10:36:19.314 INFO:teuthology.orchestra.run.smithi083.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-17T10:36:19.317 INFO:teuthology.orchestra.run.smithi083.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-17T10:36:19.318 INFO:teuthology.orchestra.run.smithi083.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-17T10:36:19.323 INFO:teuthology.orchestra.run.smithi083.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-17T10:36:19.325 INFO:teuthology.orchestra.run.smithi083.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-17T10:36:19.333 INFO:teuthology.orchestra.run.smithi083.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-17T10:36:19.334 INFO:teuthology.orchestra.run.smithi083.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-17T10:36:19.337 INFO:teuthology.orchestra.run.smithi167.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-17T10:36:19.337 INFO:teuthology.orchestra.run.smithi167.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-17T10:36:19.360 INFO:teuthology.orchestra.run.smithi083.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-17T10:36:19.360 INFO:teuthology.orchestra.run.smithi083.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-17T10:36:19.371 INFO:teuthology.orchestra.run.smithi083.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-17T10:36:19.376 INFO:teuthology.orchestra.run.smithi167.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-17T10:36:19.392 INFO:teuthology.orchestra.run.smithi083.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-17T10:36:19.395 INFO:teuthology.orchestra.run.smithi083.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-17T10:36:19.395 INFO:teuthology.orchestra.run.smithi083.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-17T10:36:19.396 INFO:teuthology.orchestra.run.smithi083.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-17T10:36:19.396 INFO:teuthology.orchestra.run.smithi083.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-17T10:36:19.407 INFO:teuthology.orchestra.run.smithi083.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-17T10:36:19.409 INFO:teuthology.orchestra.run.smithi083.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-17T10:36:19.439 INFO:teuthology.orchestra.run.smithi167.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-17T10:36:19.446 INFO:teuthology.orchestra.run.smithi167.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-17T10:36:19.447 INFO:teuthology.orchestra.run.smithi167.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-17T10:36:19.448 INFO:teuthology.orchestra.run.smithi167.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-17T10:36:19.449 INFO:teuthology.orchestra.run.smithi167.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-17T10:36:19.640 INFO:teuthology.orchestra.run.smithi083.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-17T10:36:19.640 INFO:teuthology.orchestra.run.smithi083.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-17T10:36:19.640 INFO:teuthology.orchestra.run.smithi083.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-17T10:36:19.640 INFO:teuthology.orchestra.run.smithi083.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-17T10:36:19.640 INFO:teuthology.orchestra.run.smithi083.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-17T10:36:19.640 INFO:teuthology.orchestra.run.smithi083.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-17T10:36:19.652 INFO:teuthology.orchestra.run.smithi083.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-17T10:36:19.652 INFO:teuthology.orchestra.run.smithi083.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-17T10:36:19.679 INFO:teuthology.orchestra.run.smithi167.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-17T10:36:20.056 INFO:teuthology.orchestra.run.smithi083.stdout:Preconfiguring packages ... 2024-04-17T10:36:20.082 INFO:teuthology.orchestra.run.smithi167.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-17T10:36:20.199 INFO:teuthology.orchestra.run.smithi083.stdout:Fetched 55.7 MB in 1s (50.8 MB/s) 2024-04-17T10:36:20.520 INFO:teuthology.orchestra.run.smithi083.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173655 files and directories currently installed.) 2024-04-17T10:36:20.526 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-17T10:36:20.665 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-17T10:36:20.883 INFO:teuthology.orchestra.run.smithi167.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-17T10:36:20.883 INFO:teuthology.orchestra.run.smithi167.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-17T10:36:20.883 INFO:teuthology.orchestra.run.smithi167.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-17T10:36:20.927 INFO:teuthology.orchestra.run.smithi167.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-17T10:36:20.928 INFO:teuthology.orchestra.run.smithi167.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-17T10:36:20.928 INFO:teuthology.orchestra.run.smithi167.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-17T10:36:20.960 INFO:teuthology.orchestra.run.smithi167.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-17T10:36:20.961 INFO:teuthology.orchestra.run.smithi167.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-17T10:36:21.311 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-17T10:36:21.370 INFO:teuthology.orchestra.run.smithi167.stdout:Preconfiguring packages ... 2024-04-17T10:36:21.518 INFO:teuthology.orchestra.run.smithi167.stdout:Fetched 55.7 MB in 5s (10.6 MB/s) 2024-04-17T10:36:21.784 INFO:teuthology.orchestra.run.smithi167.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173655 files and directories currently installed.) 2024-04-17T10:36:21.788 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-17T10:36:21.922 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-17T10:36:22.252 INFO:teuthology.orchestra.run.smithi083.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-17T10:36:22.491 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-17T10:36:22.512 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package flex. 2024-04-17T10:36:22.612 INFO:teuthology.orchestra.run.smithi083.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173655 files and directories currently installed.) 2024-04-17T10:36:22.618 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-17T10:36:22.660 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-17T10:36:23.152 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-17T10:36:23.316 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-17T10:36:23.404 INFO:teuthology.orchestra.run.smithi167.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-17T10:36:23.683 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package flex. 2024-04-17T10:36:23.718 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-17T10:36:23.784 INFO:teuthology.orchestra.run.smithi167.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173655 files and directories currently installed.) 2024-04-17T10:36:23.790 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-17T10:36:23.839 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-17T10:36:24.229 INFO:teuthology.orchestra.run.smithi083.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173728 files and directories currently installed.) 2024-04-17T10:36:24.234 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-17T10:36:24.338 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-17T10:36:24.365 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-17T10:36:24.487 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-17T10:36:24.774 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-17T10:36:24.873 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-17T10:36:24.905 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-17T10:36:25.232 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-17T10:36:25.375 INFO:teuthology.orchestra.run.smithi167.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173728 files and directories currently installed.) 2024-04-17T10:36:25.381 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-17T10:36:25.512 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-17T10:36:25.598 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package bison. 2024-04-17T10:36:25.702 INFO:teuthology.orchestra.run.smithi083.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173728 files and directories currently installed.) 2024-04-17T10:36:25.708 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-17T10:36:25.750 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-17T10:36:25.887 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-17T10:36:26.002 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-17T10:36:26.159 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-17T10:36:26.187 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-17T10:36:26.226 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-17T10:36:26.330 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-17T10:36:26.570 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package dump. 2024-04-17T10:36:26.598 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-17T10:36:26.637 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking dump (0.4b47-3) ... 2024-04-17T10:36:26.712 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package bison. 2024-04-17T10:36:26.817 INFO:teuthology.orchestra.run.smithi167.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173728 files and directories currently installed.) 2024-04-17T10:36:26.823 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-17T10:36:26.881 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-17T10:36:26.907 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package indent. 2024-04-17T10:36:26.923 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-17T10:36:27.129 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-17T10:36:27.290 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-17T10:36:27.318 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-17T10:36:27.357 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-17T10:36:27.635 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package dump. 2024-04-17T10:36:27.663 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-17T10:36:27.693 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking dump (0.4b47-3) ... 2024-04-17T10:36:27.745 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-17T10:36:27.772 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-17T10:36:27.811 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-17T10:36:27.964 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package indent. 2024-04-17T10:36:27.992 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-17T10:36:28.031 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-17T10:36:28.140 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-17T10:36:28.167 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-17T10:36:28.198 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-17T10:36:28.400 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-17T10:36:28.428 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-17T10:36:28.451 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-17T10:36:28.467 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-17T10:36:28.467 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-17T10:36:28.509 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-17T10:36:28.729 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-17T10:36:28.757 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-17T10:36:28.788 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-17T10:36:28.846 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-17T10:36:28.874 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-17T10:36:28.913 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-17T10:36:29.049 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-17T10:36:29.077 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-17T10:36:29.116 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-17T10:36:29.165 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-17T10:36:29.193 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-17T10:36:29.232 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-17T10:36:29.453 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-17T10:36:29.460 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libjson-perl. 2024-04-17T10:36:29.476 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-17T10:36:29.480 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-17T10:36:29.518 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-17T10:36:29.519 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-17T10:36:29.747 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-17T10:36:29.763 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-17T10:36:29.797 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-17T10:36:29.830 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-17T10:36:29.857 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-17T10:36:29.896 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-17T10:36:30.033 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libjson-perl. 2024-04-17T10:36:30.049 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-17T10:36:30.091 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-17T10:36:30.437 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-17T10:36:30.464 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-17T10:36:30.503 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-17T10:36:31.549 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-17T10:36:31.577 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-17T10:36:31.616 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-17T10:36:31.852 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-17T10:36:31.868 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-17T10:36:31.902 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-17T10:36:32.213 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-17T10:36:32.241 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-17T10:36:32.280 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-17T10:36:32.599 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-17T10:36:32.627 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-17T10:36:32.657 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-17T10:36:32.684 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-17T10:36:32.712 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-17T10:36:32.750 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-17T10:36:32.927 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-17T10:36:32.943 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-17T10:36:32.985 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-17T10:36:33.037 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-17T10:36:33.064 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-17T10:36:33.103 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-17T10:36:33.296 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-17T10:36:33.324 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-17T10:36:33.363 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-17T10:36:33.456 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-17T10:36:33.484 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-17T10:36:33.523 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-17T10:36:33.691 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-17T10:36:33.719 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-17T10:36:33.749 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-17T10:36:33.892 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-17T10:36:33.920 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-17T10:36:33.959 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-17T10:36:34.069 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-17T10:36:34.097 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-17T10:36:34.136 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-17T10:36:34.245 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-17T10:36:34.273 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-17T10:36:34.303 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-17T10:36:34.497 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-17T10:36:34.514 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-17T10:36:34.531 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-17T10:36:34.555 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-17T10:36:34.558 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-17T10:36:34.597 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-17T10:36:34.859 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-17T10:36:34.876 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-17T10:36:34.909 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-17T10:36:35.068 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-17T10:36:35.095 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-17T10:36:35.145 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-17T10:36:35.198 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-17T10:36:35.211 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-17T10:36:35.253 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-17T10:36:35.489 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-17T10:36:35.505 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-17T10:36:35.539 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-17T10:36:35.621 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package modemmanager. 2024-04-17T10:36:35.648 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-17T10:36:35.687 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-17T10:36:36.026 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-17T10:36:36.054 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-17T10:36:36.093 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-17T10:36:36.303 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package network-manager. 2024-04-17T10:36:36.330 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-17T10:36:36.378 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-17T10:36:36.446 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package modemmanager. 2024-04-17T10:36:36.474 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-17T10:36:36.513 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-17T10:36:36.977 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ppp. 2024-04-17T10:36:37.004 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-17T10:36:37.220 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package network-manager. 2024-04-17T10:36:37.248 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-17T10:36:37.276 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-17T10:36:37.294 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-17T10:36:37.606 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package pptp-linux. 2024-04-17T10:36:37.635 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-17T10:36:37.673 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-17T10:36:37.913 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ppp. 2024-04-17T10:36:37.940 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-17T10:36:38.069 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-17T10:36:38.097 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-17T10:36:38.135 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-17T10:36:38.193 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-17T10:36:38.430 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-17T10:36:38.458 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-17T10:36:38.496 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking postgresql-client-common (238) ... 2024-04-17T10:36:38.509 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package pptp-linux. 2024-04-17T10:36:38.537 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-17T10:36:38.576 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-17T10:36:38.783 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-17T10:36:38.804 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-17T10:36:38.812 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-17T10:36:38.832 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-17T10:36:38.850 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-17T10:36:38.870 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-17T10:36:39.174 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-17T10:36:39.202 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-17T10:36:39.249 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking postgresql-client-common (238) ... 2024-04-17T10:36:39.422 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package postgresql-common. 2024-04-17T10:36:39.449 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-17T10:36:39.506 INFO:teuthology.orchestra.run.smithi083.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-17T10:36:39.528 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-17T10:36:39.536 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking postgresql-common (238) ... 2024-04-17T10:36:39.556 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-17T10:36:39.594 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-17T10:36:39.996 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package postgresql-14. 2024-04-17T10:36:40.023 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-17T10:36:40.066 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-17T10:36:40.158 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package postgresql-common. 2024-04-17T10:36:40.186 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-17T10:36:40.242 INFO:teuthology.orchestra.run.smithi167.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-17T10:36:40.272 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking postgresql-common (238) ... 2024-04-17T10:36:40.757 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package postgresql-14. 2024-04-17T10:36:40.785 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-17T10:36:40.827 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-17T10:36:41.382 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package postgresql. 2024-04-17T10:36:41.399 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-17T10:36:41.440 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking postgresql (14+238) ... 2024-04-17T10:36:41.667 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package postgresql-client. 2024-04-17T10:36:41.684 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-17T10:36:41.734 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking postgresql-client (14+238) ... 2024-04-17T10:36:41.978 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-17T10:36:41.995 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-17T10:36:42.036 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-17T10:36:42.211 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package postgresql. 2024-04-17T10:36:42.232 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-17T10:36:42.269 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking postgresql (14+238) ... 2024-04-17T10:36:42.337 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package tcl8.6. 2024-04-17T10:36:42.359 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-17T10:36:42.397 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-17T10:36:42.521 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package postgresql-client. 2024-04-17T10:36:42.551 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-17T10:36:42.588 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking postgresql-client (14+238) ... 2024-04-17T10:36:42.650 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package tcl. 2024-04-17T10:36:42.678 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-17T10:36:42.721 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-17T10:36:42.848 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-17T10:36:42.877 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-17T10:36:42.915 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-17T10:36:43.078 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package wpasupplicant. 2024-04-17T10:36:43.106 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-17T10:36:43.144 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-17T10:36:43.184 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package tcl8.6. 2024-04-17T10:36:43.201 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-17T10:36:43.234 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-17T10:36:43.498 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-17T10:36:43.514 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-17T10:36:43.537 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package tcl. 2024-04-17T10:36:43.548 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-17T10:36:43.565 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-17T10:36:43.609 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-17T10:36:43.918 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-17T10:36:43.946 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-17T10:36:43.957 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package wpasupplicant. 2024-04-17T10:36:43.985 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-17T10:36:43.994 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-17T10:36:44.023 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-17T10:36:44.289 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up postgresql-client-common (238) ... 2024-04-17T10:36:44.361 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-17T10:36:44.377 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-17T10:36:44.419 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-17T10:36:44.524 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-17T10:36:44.650 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-17T10:36:44.776 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-17T10:36:44.781 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-17T10:36:44.809 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-17T10:36:44.856 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-17T10:36:45.052 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-17T10:36:45.143 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up postgresql-client-common (238) ... 2024-04-17T10:36:45.162 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-17T10:36:45.279 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-17T10:36:45.354 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-17T10:36:45.480 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-17T10:36:45.606 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-17T10:36:45.707 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-17T10:36:45.778 INFO:teuthology.orchestra.run.smithi083.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-17T10:36:45.816 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-17T10:36:45.868 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-17T10:36:45.943 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-17T10:36:46.257 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-17T10:36:46.257 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-17T10:36:46.410 INFO:teuthology.orchestra.run.smithi167.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-17T10:36:46.490 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-17T10:36:46.780 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-17T10:36:46.839 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-17T10:36:46.839 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-17T10:36:46.906 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-17T10:36:47.032 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-17T10:36:47.158 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-17T10:36:47.284 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-17T10:36:47.310 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-17T10:36:47.410 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-17T10:36:47.436 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-17T10:36:47.536 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-17T10:36:47.562 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-17T10:36:47.662 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-17T10:36:47.688 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-17T10:36:47.789 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-17T10:36:48.274 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-17T10:36:48.317 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-17T10:36:48.400 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-17T10:36:48.443 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up dump (0.4b47-3) ... 2024-04-17T10:36:48.526 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-17T10:36:48.537 INFO:teuthology.orchestra.run.smithi083.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-17T10:36:48.603 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-17T10:36:48.713 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-17T10:36:48.838 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-17T10:36:48.964 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-17T10:36:49.090 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-17T10:36:49.148 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-17T10:36:49.181 INFO:teuthology.orchestra.run.smithi083.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-17T10:36:49.249 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up dump (0.4b47-3) ... 2024-04-17T10:36:49.250 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-17T10:36:49.343 INFO:teuthology.orchestra.run.smithi167.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-17T10:36:49.376 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-17T10:36:49.409 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-17T10:36:49.535 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-17T10:36:49.553 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-17T10:36:49.661 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-17T10:36:49.662 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-17T10:36:49.771 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-17T10:36:49.804 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-17T10:36:49.930 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-17T10:36:49.939 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up postgresql-client (14+238) ... 2024-04-17T10:36:50.011 INFO:teuthology.orchestra.run.smithi167.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-17T10:36:50.065 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-17T10:36:50.073 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-17T10:36:50.199 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-17T10:36:50.376 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-17T10:36:50.502 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-17T10:36:50.503 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:36:50.503 INFO:teuthology.orchestra.run.smithi083.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-17T10:36:50.504 INFO:teuthology.orchestra.run.smithi083.stdout:which means they are currently configured by ifupdown: 2024-04-17T10:36:50.504 INFO:teuthology.orchestra.run.smithi083.stdout:- enp3s0f1 2024-04-17T10:36:50.504 INFO:teuthology.orchestra.run.smithi083.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-17T10:36:50.504 INFO:teuthology.orchestra.run.smithi083.stdout:remove their configuration from /etc/network/interfaces. 2024-04-17T10:36:50.504 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:36:50.628 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-17T10:36:50.637 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-17T10:36:50.796 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up postgresql-client (14+238) ... 2024-04-17T10:36:50.922 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-17T10:36:50.961 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-17T10:36:51.286 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-17T10:36:51.353 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:36:51.354 INFO:teuthology.orchestra.run.smithi167.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-17T10:36:51.354 INFO:teuthology.orchestra.run.smithi167.stdout:which means they are currently configured by ifupdown: 2024-04-17T10:36:51.354 INFO:teuthology.orchestra.run.smithi167.stdout:- enp3s0f1 2024-04-17T10:36:51.354 INFO:teuthology.orchestra.run.smithi167.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-17T10:36:51.354 INFO:teuthology.orchestra.run.smithi167.stdout:remove their configuration from /etc/network/interfaces. 2024-04-17T10:36:51.354 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:36:51.486 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-17T10:36:51.743 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-17T10:36:51.811 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-17T10:36:51.903 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-17T10:36:52.020 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-17T10:36:52.121 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up postgresql-common (238) ... 2024-04-17T10:36:52.128 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-17T10:36:52.660 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-17T10:36:52.898 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-17T10:36:53.021 INFO:teuthology.orchestra.run.smithi083.stdout:Adding user postgres to group ssl-cert 2024-04-17T10:36:53.119 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-17T10:36:53.247 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:36:53.247 INFO:teuthology.orchestra.run.smithi083.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-17T10:36:53.336 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up postgresql-common (238) ... 2024-04-17T10:36:53.440 INFO:teuthology.orchestra.run.smithi083.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-17T10:36:53.440 INFO:teuthology.orchestra.run.smithi083.stdout:Removing obsolete dictionary files: 2024-04-17T10:36:54.055 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-17T10:36:54.424 INFO:teuthology.orchestra.run.smithi167.stdout:Adding user postgres to group ssl-cert 2024-04-17T10:36:54.431 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-17T10:36:54.655 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:36:54.655 INFO:teuthology.orchestra.run.smithi167.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-17T10:36:54.759 INFO:teuthology.orchestra.run.smithi083.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-17T10:36:54.780 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-17T10:36:54.819 INFO:teuthology.orchestra.run.smithi083.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-17T10:36:54.819 INFO:teuthology.orchestra.run.smithi083.stdout:This user must also own the server process. 2024-04-17T10:36:54.819 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:36:54.820 INFO:teuthology.orchestra.run.smithi083.stdout:The database cluster will be initialized with locale "en_US". 2024-04-17T10:36:54.820 INFO:teuthology.orchestra.run.smithi083.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-17T10:36:54.820 INFO:teuthology.orchestra.run.smithi083.stdout:The default text search configuration will be set to "english". 2024-04-17T10:36:54.820 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:36:54.820 INFO:teuthology.orchestra.run.smithi083.stdout:Data page checksums are disabled. 2024-04-17T10:36:54.820 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:36:54.820 INFO:teuthology.orchestra.run.smithi083.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-17T10:36:54.820 INFO:teuthology.orchestra.run.smithi083.stdout:creating subdirectories ... ok 2024-04-17T10:36:54.820 INFO:teuthology.orchestra.run.smithi083.stdout:selecting dynamic shared memory implementation ... posix 2024-04-17T10:36:54.835 INFO:teuthology.orchestra.run.smithi083.stdout:selecting default max_connections ... 100 2024-04-17T10:36:54.844 INFO:teuthology.orchestra.run.smithi167.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-17T10:36:54.844 INFO:teuthology.orchestra.run.smithi167.stdout:Removing obsolete dictionary files: 2024-04-17T10:36:54.855 INFO:teuthology.orchestra.run.smithi083.stdout:selecting default shared_buffers ... 128MB 2024-04-17T10:36:54.864 INFO:teuthology.orchestra.run.smithi083.stdout:selecting default time zone ... Etc/UTC 2024-04-17T10:36:54.865 INFO:teuthology.orchestra.run.smithi083.stdout:creating configuration files ... ok 2024-04-17T10:36:54.980 INFO:teuthology.orchestra.run.smithi083.stdout:running bootstrap script ... ok 2024-04-17T10:36:55.369 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-17T10:36:55.635 INFO:teuthology.orchestra.run.smithi083.stdout:performing post-bootstrap initialization ... ok 2024-04-17T10:36:55.725 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-17T10:36:56.045 INFO:teuthology.orchestra.run.smithi167.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-17T10:36:56.067 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-17T10:36:56.115 INFO:teuthology.orchestra.run.smithi167.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-17T10:36:56.115 INFO:teuthology.orchestra.run.smithi167.stdout:This user must also own the server process. 2024-04-17T10:36:56.115 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:36:56.115 INFO:teuthology.orchestra.run.smithi167.stdout:The database cluster will be initialized with locale "en_US". 2024-04-17T10:36:56.115 INFO:teuthology.orchestra.run.smithi167.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-17T10:36:56.115 INFO:teuthology.orchestra.run.smithi167.stdout:The default text search configuration will be set to "english". 2024-04-17T10:36:56.115 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:36:56.115 INFO:teuthology.orchestra.run.smithi167.stdout:Data page checksums are disabled. 2024-04-17T10:36:56.115 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:36:56.115 INFO:teuthology.orchestra.run.smithi167.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-17T10:36:56.116 INFO:teuthology.orchestra.run.smithi167.stdout:creating subdirectories ... ok 2024-04-17T10:36:56.116 INFO:teuthology.orchestra.run.smithi167.stdout:selecting dynamic shared memory implementation ... posix 2024-04-17T10:36:56.130 INFO:teuthology.orchestra.run.smithi167.stdout:selecting default max_connections ... 100 2024-04-17T10:36:56.151 INFO:teuthology.orchestra.run.smithi167.stdout:selecting default shared_buffers ... 128MB 2024-04-17T10:36:56.161 INFO:teuthology.orchestra.run.smithi167.stdout:selecting default time zone ... Etc/UTC 2024-04-17T10:36:56.161 INFO:teuthology.orchestra.run.smithi167.stdout:creating configuration files ... ok 2024-04-17T10:36:56.261 INFO:teuthology.orchestra.run.smithi083.stdout:syncing data to disk ... ok 2024-04-17T10:36:56.278 INFO:teuthology.orchestra.run.smithi167.stdout:running bootstrap script ... ok 2024-04-17T10:36:56.639 INFO:teuthology.orchestra.run.smithi083.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-17T10:36:56.900 INFO:teuthology.orchestra.run.smithi167.stdout:performing post-bootstrap initialization ... ok 2024-04-17T10:36:57.838 INFO:teuthology.orchestra.run.smithi167.stdout:syncing data to disk ... ok 2024-04-17T10:36:58.209 INFO:teuthology.orchestra.run.smithi167.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-17T10:36:59.390 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-17T10:36:59.516 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-17T10:36:59.642 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up postgresql (14+238) ... 2024-04-17T10:36:59.768 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-17T10:36:59.894 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-17T10:37:00.148 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-17T10:37:00.148 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-17T10:37:00.688 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-17T10:37:00.772 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-17T10:37:00.964 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-17T10:37:01.065 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-17T10:37:01.091 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-17T10:37:01.183 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up postgresql (14+238) ... 2024-04-17T10:37:01.260 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-17T10:37:01.309 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-17T10:37:01.452 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-17T10:37:01.710 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-17T10:37:01.710 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-17T10:37:01.994 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-17T10:37:02.102 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-17T10:37:02.263 INFO:teuthology.orchestra.run.smithi167.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-17T10:37:02.355 INFO:teuthology.orchestra.run.smithi167.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-17T10:37:02.604 INFO:teuthology.orchestra.run.smithi083.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-17T10:37:02.610 DEBUG:teuthology.parallel:result is None 2024-04-17T10:37:02.651 INFO:teuthology.orchestra.run.smithi167.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-17T10:37:02.792 INFO:teuthology.orchestra.run.smithi167.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-17T10:37:03.760 INFO:teuthology.orchestra.run.smithi167.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-17T10:37:03.860 INFO:teuthology.orchestra.run.smithi167.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-17T10:37:04.359 INFO:teuthology.orchestra.run.smithi167.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-17T10:37:04.364 DEBUG:teuthology.parallel:result is None 2024-04-17T10:37:04.365 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-17T10:37:04.512 DEBUG:teuthology.orchestra.run.smithi083:> dpkg-query -W -f '${Version}' ceph 2024-04-17T10:37:04.532 INFO:teuthology.orchestra.run.smithi083.stdout:19.0.0-1628-g97ae6cc9-1jammy 2024-04-17T10:37:04.533 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-17T10:37:04.533 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628-g97ae6cc9-1jammy is installed. 2024-04-17T10:37:04.535 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-17T10:37:04.756 DEBUG:teuthology.orchestra.run.smithi167:> dpkg-query -W -f '${Version}' ceph 2024-04-17T10:37:04.776 INFO:teuthology.orchestra.run.smithi167.stdout:19.0.0-1628-g97ae6cc9-1jammy 2024-04-17T10:37:04.776 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-17T10:37:04.776 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628-g97ae6cc9-1jammy is installed. 2024-04-17T10:37:04.778 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-17T10:37:04.778 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:37:04.779 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-17T10:37:04.794 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:37:04.794 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-17T10:37:04.833 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-17T10:37:04.833 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:37:04.833 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/daemon-helper 2024-04-17T10:37:04.850 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-17T10:37:04.905 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:37:04.905 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/daemon-helper 2024-04-17T10:37:04.918 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-17T10:37:04.971 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-17T10:37:04.972 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:37:04.972 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-17T10:37:04.984 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-17T10:37:05.037 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:37:05.037 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-17T10:37:05.050 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-17T10:37:05.102 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-17T10:37:05.102 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:37:05.103 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/stdin-killer 2024-04-17T10:37:05.115 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-17T10:37:05.170 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:37:05.170 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/stdin-killer 2024-04-17T10:37:05.183 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-17T10:37:05.240 INFO:teuthology.run_tasks:Running task ceph... 2024-04-17T10:37:05.332 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-17T10:37:05.333 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 777 /var/log/ceph 2024-04-17T10:37:05.335 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 777 /var/log/ceph 2024-04-17T10:37:05.348 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-17T10:37:05.348 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-17T10:37:05.391 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-17T10:37:05.403 INFO:tasks.ceph:Creating extra log directories... 2024-04-17T10:37:05.403 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-17T10:37:05.447 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-17T10:37:05.466 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-17T10:37:05.466 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-17T10:37:05.466 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-17T10:37:05.467 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:37:05.467 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-17T10:37:05.506 DEBUG:teuthology.orchestra.run.smithi083:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-17T10:37:05.579 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:37:05.579 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-17T10:37:05.584 DEBUG:teuthology.orchestra.run.smithi167:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-17T10:37:05.667 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-17T10:37:05.667 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-17T10:37:05.668 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2024-04-17_09:52:10-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7660174', 'branch': 'wip-neesingh-testing-240416', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/scrub}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7660174', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2024-04-17_09:52:10-fs:functional-wip-neesingh-testing-240416-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing-240416'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-neesingh-testing-240416', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 10, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 558, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-neesingh-testing-240416', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi083.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHr9+q9Te3W+pbDiE+Htwdkau6E+Sf3z9uafMwd2gkpuElBki7FIY7+wmf/mUPwhJNOmtGTHAuXdIn30VmzrvHo=', 'smithi167.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBECZPeOI70yBWy+c5CdXEWaPmEAYKBn0CSit7jaHx6+l7Dc3NPw0uERZI1qUx7JfM4v9YWgerWoL4WcglL5TIyM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_scrub_checks', 'tasks.cephfs.test_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '6c637841c215537a4502385240412f1966e0faab', 'timestamp': '2024-04-17_09:52:10', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-17T10:37:05.668 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-17T10:37:05.671 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-17T10:37:05.709 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m0777 -- /var/run/ceph 2024-04-17T10:37:05.719 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m0777 -- /var/run/ceph 2024-04-17T10:37:05.764 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:37:05.764 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/scratch_devs of=/dev/stdout 2024-04-17T10:37:05.778 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-17T10:37:05.778 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_1 2024-04-17T10:37:05.827 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-17T10:37:05.827 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T10:37:05.827 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 883 Links: 1 2024-04-17T10:37:05.827 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T10:37:05.827 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-17 10:31:05.463128355 +0000 2024-04-17T10:37:05.827 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-17 10:31:05.207130644 +0000 2024-04-17T10:37:05.827 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-17 10:31:05.207130644 +0000 2024-04-17T10:37:05.827 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: - 2024-04-17T10:37:05.828 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-17T10:37:05.881 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:37:05.881 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:37:05.881 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000439877 s, 1.2 MB/s 2024-04-17T10:37:05.882 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-17T10:37:05.931 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_2 2024-04-17T10:37:05.978 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-17T10:37:05.978 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T10:37:05.978 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 905 Links: 1 2024-04-17T10:37:05.978 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T10:37:05.979 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-17 10:31:05.691126318 +0000 2024-04-17T10:37:05.979 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-17 10:31:05.687126354 +0000 2024-04-17T10:37:05.979 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-17 10:31:05.687126354 +0000 2024-04-17T10:37:05.979 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: - 2024-04-17T10:37:05.979 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-17T10:37:06.032 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:37:06.033 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:37:06.033 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000476866 s, 1.1 MB/s 2024-04-17T10:37:06.034 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-17T10:37:06.084 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_3 2024-04-17T10:37:06.130 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-17T10:37:06.130 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T10:37:06.130 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 923 Links: 1 2024-04-17T10:37:06.130 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T10:37:06.130 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-17 10:31:06.451119526 +0000 2024-04-17T10:37:06.131 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-17 10:31:06.207121706 +0000 2024-04-17T10:37:06.131 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-17 10:31:06.207121706 +0000 2024-04-17T10:37:06.131 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: - 2024-04-17T10:37:06.131 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-17T10:37:06.184 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:37:06.184 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:37:06.184 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000375526 s, 1.4 MB/s 2024-04-17T10:37:06.185 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-17T10:37:06.235 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_4 2024-04-17T10:37:06.282 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-17T10:37:06.282 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T10:37:06.282 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 933 Links: 1 2024-04-17T10:37:06.283 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T10:37:06.283 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-17 10:31:06.951115056 +0000 2024-04-17T10:37:06.283 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-17 10:31:06.683117452 +0000 2024-04-17T10:37:06.283 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-17 10:31:06.683117452 +0000 2024-04-17T10:37:06.283 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: - 2024-04-17T10:37:06.283 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-17T10:37:06.336 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:37:06.336 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:37:06.336 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000378172 s, 1.4 MB/s 2024-04-17T10:37:06.337 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-17T10:37:06.387 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-17T10:37:06.387 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:37:06.387 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/scratch_devs of=/dev/stdout 2024-04-17T10:37:06.393 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-17T10:37:06.394 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_1 2024-04-17T10:37:06.440 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-17T10:37:06.440 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T10:37:06.441 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 876 Links: 1 2024-04-17T10:37:06.441 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T10:37:06.441 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-04-17 10:30:56.700934364 +0000 2024-04-17T10:37:06.441 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-04-17 10:30:56.428936793 +0000 2024-04-17T10:37:06.441 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-04-17 10:30:56.428936793 +0000 2024-04-17T10:37:06.441 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2024-04-17T10:37:06.441 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-17T10:37:06.495 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-04-17T10:37:06.495 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-04-17T10:37:06.495 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000414616 s, 1.2 MB/s 2024-04-17T10:37:06.496 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-17T10:37:06.546 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_2 2024-04-17T10:37:06.592 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-17T10:37:06.592 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T10:37:06.592 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 897 Links: 1 2024-04-17T10:37:06.593 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T10:37:06.593 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-04-17 10:30:57.216929755 +0000 2024-04-17T10:37:06.593 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-04-17 10:30:56.932932292 +0000 2024-04-17T10:37:06.593 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-04-17 10:30:56.932932292 +0000 2024-04-17T10:37:06.593 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2024-04-17T10:37:06.593 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-17T10:37:06.646 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-04-17T10:37:06.647 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-04-17T10:37:06.647 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000444493 s, 1.2 MB/s 2024-04-17T10:37:06.648 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-17T10:37:06.698 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_3 2024-04-17T10:37:06.744 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-17T10:37:06.744 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T10:37:06.744 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 915 Links: 1 2024-04-17T10:37:06.744 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T10:37:06.745 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-04-17 10:30:57.920923466 +0000 2024-04-17T10:37:06.745 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-04-17 10:30:57.456927612 +0000 2024-04-17T10:37:06.745 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-04-17 10:30:57.456927612 +0000 2024-04-17T10:37:06.745 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2024-04-17T10:37:06.745 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-17T10:37:06.799 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-04-17T10:37:06.799 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-04-17T10:37:06.799 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.00041492 s, 1.2 MB/s 2024-04-17T10:37:06.800 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-17T10:37:06.854 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_4 2024-04-17T10:37:06.900 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-17T10:37:06.900 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T10:37:06.900 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 931 Links: 1 2024-04-17T10:37:06.900 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T10:37:06.900 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-04-17 10:30:58.148921430 +0000 2024-04-17T10:37:06.901 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-04-17 10:30:58.144921465 +0000 2024-04-17T10:37:06.901 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-04-17 10:30:58.144921465 +0000 2024-04-17T10:37:06.901 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2024-04-17T10:37:06.901 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-17T10:37:06.954 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-04-17T10:37:06.954 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-04-17T10:37:06.954 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000341906 s, 1.5 MB/s 2024-04-17T10:37:06.956 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-17T10:37:07.005 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-17T10:37:07.005 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi083.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi167.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-17T10:37:07.006 INFO:tasks.ceph:Generating config... 2024-04-17T10:37:07.007 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-17T10:37:07.007 INFO:tasks.ceph:[client] debug client = 20 2024-04-17T10:37:07.007 INFO:tasks.ceph:[client] debug ms = 1 2024-04-17T10:37:07.007 INFO:tasks.ceph:[client] pid file = 2024-04-17T10:37:07.007 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-17T10:37:07.007 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-17T10:37:07.008 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-17T10:37:07.008 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-17T10:37:07.008 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-17T10:37:07.008 INFO:tasks.ceph:[mds] mds cache max size = 1000 2024-04-17T10:37:07.008 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-17T10:37:07.008 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-17T10:37:07.008 INFO:tasks.ceph:[mds] mds log max segments = 1 2024-04-17T10:37:07.008 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-17T10:37:07.008 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-17T10:37:07.008 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-17T10:37:07.008 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-17T10:37:07.008 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-17T10:37:07.009 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-17T10:37:07.009 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-17T10:37:07.009 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-17T10:37:07.009 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-17T10:37:07.009 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-17T10:37:07.009 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-17T10:37:07.009 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-17T10:37:07.009 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-17T10:37:07.009 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-17T10:37:07.009 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-17T10:37:07.009 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-17T10:37:07.009 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-17T10:37:07.010 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-17T10:37:07.010 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-17T10:37:07.010 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-17T10:37:07.010 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-17T10:37:07.010 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-17T10:37:07.010 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-17T10:37:07.010 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-17T10:37:07.010 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-17T10:37:07.010 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-17T10:37:07.010 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-17T10:37:07.011 INFO:tasks.ceph:Setting up mon.a... 2024-04-17T10:37:07.011 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-17T10:37:07.045 INFO:teuthology.orchestra.run.smithi083.stdout:creating /etc/ceph/ceph.keyring 2024-04-17T10:37:07.061 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-17T10:37:07.135 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-17T10:37:07.188 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.83'), ('mon.b', '172.21.15.167'), ('mon.c', '[v2:172.21.15.167:3301,v1:172.21.15.167:6790]')] 2024-04-17T10:37:07.189 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.83,172.21.15.167,[v2:172.21.15.167:3301,v1:172.21.15.167:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-17T10:37:07.190 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:37:07.190 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-17T10:37:07.234 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.83 --add b 172.21.15.167 --addv c '[v2:172.21.15.167:3301,v1:172.21.15.167:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-17T10:37:07.320 INFO:teuthology.orchestra.run.smithi083.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-17T10:37:07.320 INFO:teuthology.orchestra.run.smithi083.stdout:monmaptool: generated fsid 3de9b7be-af7f-488a-969e-2b6b8db63bcd 2024-04-17T10:37:07.321 INFO:teuthology.orchestra.run.smithi083.stdout:setting min_mon_release = quincy 2024-04-17T10:37:07.321 INFO:teuthology.orchestra.run.smithi083.stdout:epoch 0 2024-04-17T10:37:07.321 INFO:teuthology.orchestra.run.smithi083.stdout:fsid 3de9b7be-af7f-488a-969e-2b6b8db63bcd 2024-04-17T10:37:07.321 INFO:teuthology.orchestra.run.smithi083.stdout:last_changed 2024-04-17T10:37:07.328300+0000 2024-04-17T10:37:07.321 INFO:teuthology.orchestra.run.smithi083.stdout:created 2024-04-17T10:37:07.328300+0000 2024-04-17T10:37:07.321 INFO:teuthology.orchestra.run.smithi083.stdout:min_mon_release 17 (quincy) 2024-04-17T10:37:07.321 INFO:teuthology.orchestra.run.smithi083.stdout:election_strategy: 1 2024-04-17T10:37:07.321 INFO:teuthology.orchestra.run.smithi083.stdout:0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-04-17T10:37:07.321 INFO:teuthology.orchestra.run.smithi083.stdout:1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.b 2024-04-17T10:37:07.321 INFO:teuthology.orchestra.run.smithi083.stdout:2: [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] mon.c 2024-04-17T10:37:07.322 INFO:teuthology.orchestra.run.smithi083.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-17T10:37:07.323 DEBUG:teuthology.orchestra.run.smithi083:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-17T10:37:07.370 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 3de9b7be-af7f-488a-969e-2b6b8db63bcd... 2024-04-17T10:37:07.371 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-17T10:37:07.415 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-17T10:37:07.442 INFO:teuthology.orchestra.run.smithi083.stdout:[global] 2024-04-17T10:37:07.443 INFO:teuthology.orchestra.run.smithi083.stdout: chdir = "" 2024-04-17T10:37:07.443 INFO:teuthology.orchestra.run.smithi083.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-17T10:37:07.443 INFO:teuthology.orchestra.run.smithi083.stdout: auth supported = cephx 2024-04-17T10:37:07.443 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:37:07.443 INFO:teuthology.orchestra.run.smithi083.stdout: filestore xattr use omap = true 2024-04-17T10:37:07.443 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:37:07.443 INFO:teuthology.orchestra.run.smithi083.stdout: mon clock drift allowed = 1.000 2024-04-17T10:37:07.443 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:37:07.443 INFO:teuthology.orchestra.run.smithi083.stdout: osd crush chooseleaf type = 0 2024-04-17T10:37:07.444 INFO:teuthology.orchestra.run.smithi083.stdout: auth debug = true 2024-04-17T10:37:07.444 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:37:07.444 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on old message = true 2024-04-17T10:37:07.444 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on bug = true 2024-04-17T10:37:07.444 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:37:07.444 INFO:teuthology.orchestra.run.smithi083.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-17T10:37:07.444 INFO:teuthology.orchestra.run.smithi083.stdout: mon pg warn max object skew = 0 2024-04-17T10:37:07.444 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:37:07.444 INFO:teuthology.orchestra.run.smithi083.stdout: # disable pg_autoscaler by default for new pools 2024-04-17T10:37:07.444 INFO:teuthology.orchestra.run.smithi083.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-17T10:37:07.444 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:37:07.444 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default size = 2 2024-04-17T10:37:07.445 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:37:07.445 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow primary affinity = true 2024-04-17T10:37:07.445 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow pg remap = true 2024-04-17T10:37:07.445 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on legacy crush tunables = false 2024-04-17T10:37:07.445 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on crush straw calc version zero = false 2024-04-17T10:37:07.445 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on no sortbitwise = false 2024-04-17T10:37:07.445 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on osd down out interval zero = false 2024-04-17T10:37:07.445 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on too few osds = false 2024-04-17T10:37:07.445 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-17T10:37:07.445 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-17T10:37:07.445 INFO:teuthology.orchestra.run.smithi083.stdout: mon_allow_pool_size_one = true 2024-04-17T10:37:07.445 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:37:07.445 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-17T10:37:07.446 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:37:07.446 INFO:teuthology.orchestra.run.smithi083.stdout: osd default data pool replay window = 5 2024-04-17T10:37:07.446 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:37:07.446 INFO:teuthology.orchestra.run.smithi083.stdout: mon allow pool delete = true 2024-04-17T10:37:07.446 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:37:07.446 INFO:teuthology.orchestra.run.smithi083.stdout: mon cluster log file level = debug 2024-04-17T10:37:07.446 INFO:teuthology.orchestra.run.smithi083.stdout: debug asserts on shutdown = true 2024-04-17T10:37:07.446 INFO:teuthology.orchestra.run.smithi083.stdout: mon health detail to clog = false 2024-04-17T10:37:07.446 INFO:teuthology.orchestra.run.smithi083.stdout: mon host = "172.21.15.83,172.21.15.167,[v2:172.21.15.167:3301,v1:172.21.15.167:6790]" 2024-04-17T10:37:07.446 INFO:teuthology.orchestra.run.smithi083.stdout: fsid = 3de9b7be-af7f-488a-969e-2b6b8db63bcd 2024-04-17T10:37:07.446 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:37:07.446 INFO:teuthology.orchestra.run.smithi083.stdout:[osd] 2024-04-17T10:37:07.446 INFO:teuthology.orchestra.run.smithi083.stdout: osd journal size = 100 2024-04-17T10:37:07.446 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:37:07.447 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub load threshold = 5.0 2024-04-17T10:37:07.447 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub max interval = 600 2024-04-17T10:37:07.447 INFO:teuthology.orchestra.run.smithi083.stdout: osd mclock profile = high_recovery_ops 2024-04-17T10:37:07.447 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:37:07.447 INFO:teuthology.orchestra.run.smithi083.stdout: osd recover clone overlap = true 2024-04-17T10:37:07.447 INFO:teuthology.orchestra.run.smithi083.stdout: osd recovery max chunk = 1048576 2024-04-17T10:37:07.447 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:37:07.447 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug shutdown = true 2024-04-17T10:37:07.447 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug op order = true 2024-04-17T10:37:07.447 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug verify stray on activate = true 2024-04-17T10:37:07.447 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug trim objects = true 2024-04-17T10:37:07.447 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:37:07.447 INFO:teuthology.orchestra.run.smithi083.stdout: osd open classes on start = true 2024-04-17T10:37:07.447 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug pg log writeout = true 2024-04-17T10:37:07.447 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:37:07.448 INFO:teuthology.orchestra.run.smithi083.stdout: osd deep scrub update digest min age = 30 2024-04-17T10:37:07.448 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:37:07.448 INFO:teuthology.orchestra.run.smithi083.stdout: osd map max advance = 10 2024-04-17T10:37:07.448 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:37:07.448 INFO:teuthology.orchestra.run.smithi083.stdout: journal zero on create = true 2024-04-17T10:37:07.448 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:37:07.448 INFO:teuthology.orchestra.run.smithi083.stdout: filestore ondisk finisher threads = 3 2024-04-17T10:37:07.448 INFO:teuthology.orchestra.run.smithi083.stdout: filestore apply finisher threads = 3 2024-04-17T10:37:07.448 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:37:07.448 INFO:teuthology.orchestra.run.smithi083.stdout: bdev debug aio = true 2024-04-17T10:37:07.448 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug misdirected ops = true 2024-04-17T10:37:07.448 INFO:teuthology.orchestra.run.smithi083.stdout: bdev async discard = True 2024-04-17T10:37:07.448 INFO:teuthology.orchestra.run.smithi083.stdout: bdev enable discard = True 2024-04-17T10:37:07.448 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore allocator = bitmap 2024-04-17T10:37:07.448 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore block size = 96636764160 2024-04-17T10:37:07.449 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore fsck on mount = True 2024-04-17T10:37:07.449 INFO:teuthology.orchestra.run.smithi083.stdout: debug bluefs = 1/20 2024-04-17T10:37:07.449 INFO:teuthology.orchestra.run.smithi083.stdout: debug bluestore = 1/20 2024-04-17T10:37:07.449 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-04-17T10:37:07.449 INFO:teuthology.orchestra.run.smithi083.stdout: debug osd = 20 2024-04-17T10:37:07.449 INFO:teuthology.orchestra.run.smithi083.stdout: debug rocksdb = 4/10 2024-04-17T10:37:07.449 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd backfillfull_ratio = 0.85 2024-04-17T10:37:07.449 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd full ratio = 0.9 2024-04-17T10:37:07.449 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd nearfull ratio = 0.8 2024-04-17T10:37:07.449 INFO:teuthology.orchestra.run.smithi083.stdout: osd failsafe full ratio = 0.95 2024-04-17T10:37:07.449 INFO:teuthology.orchestra.run.smithi083.stdout: osd objectstore = bluestore 2024-04-17T10:37:07.449 INFO:teuthology.orchestra.run.smithi083.stdout: osd op complaint time = 180 2024-04-17T10:37:07.449 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:37:07.450 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr] 2024-04-17T10:37:07.450 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-04-17T10:37:07.450 INFO:teuthology.orchestra.run.smithi083.stdout: debug mgr = 20 2024-04-17T10:37:07.450 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2024-04-17T10:37:07.450 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2024-04-17T10:37:07.450 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2024-04-17T10:37:07.450 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2024-04-17T10:37:07.450 INFO:teuthology.orchestra.run.smithi083.stdout: mgr/telemetry/nag = false 2024-04-17T10:37:07.450 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:37:07.450 INFO:teuthology.orchestra.run.smithi083.stdout:[mon] 2024-04-17T10:37:07.450 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-04-17T10:37:07.450 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2024-04-17T10:37:07.450 INFO:teuthology.orchestra.run.smithi083.stdout: debug paxos = 20 2024-04-17T10:37:07.450 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2024-04-17T10:37:07.450 INFO:teuthology.orchestra.run.smithi083.stdout: mon data avail warn = 5 2024-04-17T10:37:07.451 INFO:teuthology.orchestra.run.smithi083.stdout: mon mgr mkfs grace = 240 2024-04-17T10:37:07.451 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2024-04-17T10:37:07.451 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd reporter subtree level = osd 2024-04-17T10:37:07.451 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd prime pg temp = true 2024-04-17T10:37:07.451 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2024-04-17T10:37:07.451 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:37:07.451 INFO:teuthology.orchestra.run.smithi083.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-17T10:37:07.451 INFO:teuthology.orchestra.run.smithi083.stdout: auth mon ticket ttl = 660 # 11m 2024-04-17T10:37:07.451 INFO:teuthology.orchestra.run.smithi083.stdout: auth service ticket ttl = 240 # 4m 2024-04-17T10:37:07.451 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:37:07.451 INFO:teuthology.orchestra.run.smithi083.stdout: # don't complain about insecure global_id in the test suite 2024-04-17T10:37:07.451 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-17T10:37:07.451 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-17T10:37:07.451 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:37:07.451 INFO:teuthology.orchestra.run.smithi083.stdout: # 1m isn't quite enough 2024-04-17T10:37:07.452 INFO:teuthology.orchestra.run.smithi083.stdout: mon_down_mkfs_grace = 2m 2024-04-17T10:37:07.452 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:37:07.452 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_filestore_osds = false 2024-04-17T10:37:07.452 INFO:teuthology.orchestra.run.smithi083.stdout: mon op complaint time = 120 2024-04-17T10:37:07.452 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:37:07.452 INFO:teuthology.orchestra.run.smithi083.stdout:[client] 2024-04-17T10:37:07.452 INFO:teuthology.orchestra.run.smithi083.stdout: rgw cache enabled = true 2024-04-17T10:37:07.452 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable ops log = true 2024-04-17T10:37:07.452 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable usage log = true 2024-04-17T10:37:07.452 INFO:teuthology.orchestra.run.smithi083.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-17T10:37:07.452 INFO:teuthology.orchestra.run.smithi083.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-17T10:37:07.452 INFO:teuthology.orchestra.run.smithi083.stdout: client mount timeout = 600 2024-04-17T10:37:07.452 INFO:teuthology.orchestra.run.smithi083.stdout: debug client = 20 2024-04-17T10:37:07.452 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-04-17T10:37:07.453 INFO:teuthology.orchestra.run.smithi083.stdout: pid file = "" 2024-04-17T10:37:07.453 INFO:teuthology.orchestra.run.smithi083.stdout: rados mon op timeout = 900 2024-04-17T10:37:07.453 INFO:teuthology.orchestra.run.smithi083.stdout: rados osd op timeout = 900 2024-04-17T10:37:07.453 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.a] 2024-04-17T10:37:07.453 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.b] 2024-04-17T10:37:07.453 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.c] 2024-04-17T10:37:07.453 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.a] 2024-04-17T10:37:07.453 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.c] 2024-04-17T10:37:07.453 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.b] 2024-04-17T10:37:07.453 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.d] 2024-04-17T10:37:07.453 INFO:teuthology.orchestra.run.smithi083.stdout:[mds] 2024-04-17T10:37:07.453 INFO:teuthology.orchestra.run.smithi083.stdout: debug mds = 20 2024-04-17T10:37:07.453 INFO:teuthology.orchestra.run.smithi083.stdout: debug mds balancer = 20 2024-04-17T10:37:07.454 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-04-17T10:37:07.454 INFO:teuthology.orchestra.run.smithi083.stdout: mds cache max size = 1000 2024-04-17T10:37:07.454 INFO:teuthology.orchestra.run.smithi083.stdout: mds debug frag = True 2024-04-17T10:37:07.454 INFO:teuthology.orchestra.run.smithi083.stdout: mds debug scatterstat = True 2024-04-17T10:37:07.454 INFO:teuthology.orchestra.run.smithi083.stdout: mds log max segments = 1 2024-04-17T10:37:07.454 INFO:teuthology.orchestra.run.smithi083.stdout: mds op complaint time = 180 2024-04-17T10:37:07.454 INFO:teuthology.orchestra.run.smithi083.stdout: mds verify scatter = True 2024-04-17T10:37:07.454 INFO:teuthology.orchestra.run.smithi083.stdout: osd op complaint time = 180 2024-04-17T10:37:07.454 INFO:teuthology.orchestra.run.smithi083.stdout: rados mon op timeout = 900 2024-04-17T10:37:07.454 INFO:teuthology.orchestra.run.smithi083.stdout: rados osd op timeout = 900 2024-04-17T10:37:07.456 INFO:teuthology.orchestra.run.smithi167.stdout:[global] 2024-04-17T10:37:07.456 INFO:teuthology.orchestra.run.smithi167.stdout: chdir = "" 2024-04-17T10:37:07.456 INFO:teuthology.orchestra.run.smithi167.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-17T10:37:07.456 INFO:teuthology.orchestra.run.smithi167.stdout: auth supported = cephx 2024-04-17T10:37:07.456 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:37:07.456 INFO:teuthology.orchestra.run.smithi167.stdout: filestore xattr use omap = true 2024-04-17T10:37:07.457 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:37:07.457 INFO:teuthology.orchestra.run.smithi167.stdout: mon clock drift allowed = 1.000 2024-04-17T10:37:07.457 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:37:07.457 INFO:teuthology.orchestra.run.smithi167.stdout: osd crush chooseleaf type = 0 2024-04-17T10:37:07.457 INFO:teuthology.orchestra.run.smithi167.stdout: auth debug = true 2024-04-17T10:37:07.457 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:37:07.457 INFO:teuthology.orchestra.run.smithi167.stdout: ms die on old message = true 2024-04-17T10:37:07.457 INFO:teuthology.orchestra.run.smithi167.stdout: ms die on bug = true 2024-04-17T10:37:07.457 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:37:07.457 INFO:teuthology.orchestra.run.smithi167.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-17T10:37:07.457 INFO:teuthology.orchestra.run.smithi167.stdout: mon pg warn max object skew = 0 2024-04-17T10:37:07.458 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:37:07.458 INFO:teuthology.orchestra.run.smithi167.stdout: # disable pg_autoscaler by default for new pools 2024-04-17T10:37:07.458 INFO:teuthology.orchestra.run.smithi167.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-17T10:37:07.458 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:37:07.458 INFO:teuthology.orchestra.run.smithi167.stdout: osd pool default size = 2 2024-04-17T10:37:07.458 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:37:07.458 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd allow primary affinity = true 2024-04-17T10:37:07.458 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd allow pg remap = true 2024-04-17T10:37:07.458 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on legacy crush tunables = false 2024-04-17T10:37:07.458 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on crush straw calc version zero = false 2024-04-17T10:37:07.458 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on no sortbitwise = false 2024-04-17T10:37:07.458 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on osd down out interval zero = false 2024-04-17T10:37:07.459 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on too few osds = false 2024-04-17T10:37:07.459 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-17T10:37:07.459 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-17T10:37:07.459 INFO:teuthology.orchestra.run.smithi167.stdout: mon_allow_pool_size_one = true 2024-04-17T10:37:07.459 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:37:07.459 INFO:teuthology.orchestra.run.smithi167.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-17T10:37:07.459 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:37:07.459 INFO:teuthology.orchestra.run.smithi167.stdout: osd default data pool replay window = 5 2024-04-17T10:37:07.459 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:37:07.459 INFO:teuthology.orchestra.run.smithi167.stdout: mon allow pool delete = true 2024-04-17T10:37:07.459 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:37:07.460 INFO:teuthology.orchestra.run.smithi167.stdout: mon cluster log file level = debug 2024-04-17T10:37:07.460 INFO:teuthology.orchestra.run.smithi167.stdout: debug asserts on shutdown = true 2024-04-17T10:37:07.460 INFO:teuthology.orchestra.run.smithi167.stdout: mon health detail to clog = false 2024-04-17T10:37:07.460 INFO:teuthology.orchestra.run.smithi167.stdout: mon host = "172.21.15.83,172.21.15.167,[v2:172.21.15.167:3301,v1:172.21.15.167:6790]" 2024-04-17T10:37:07.460 INFO:teuthology.orchestra.run.smithi167.stdout: fsid = 3de9b7be-af7f-488a-969e-2b6b8db63bcd 2024-04-17T10:37:07.460 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:37:07.460 INFO:teuthology.orchestra.run.smithi167.stdout:[osd] 2024-04-17T10:37:07.460 INFO:teuthology.orchestra.run.smithi167.stdout: osd journal size = 100 2024-04-17T10:37:07.460 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:37:07.460 INFO:teuthology.orchestra.run.smithi167.stdout: osd scrub load threshold = 5.0 2024-04-17T10:37:07.460 INFO:teuthology.orchestra.run.smithi167.stdout: osd scrub max interval = 600 2024-04-17T10:37:07.461 INFO:teuthology.orchestra.run.smithi167.stdout: osd mclock profile = high_recovery_ops 2024-04-17T10:37:07.461 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:37:07.461 INFO:teuthology.orchestra.run.smithi167.stdout: osd recover clone overlap = true 2024-04-17T10:37:07.461 INFO:teuthology.orchestra.run.smithi167.stdout: osd recovery max chunk = 1048576 2024-04-17T10:37:07.461 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:37:07.461 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug shutdown = true 2024-04-17T10:37:07.461 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug op order = true 2024-04-17T10:37:07.461 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug verify stray on activate = true 2024-04-17T10:37:07.461 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug trim objects = true 2024-04-17T10:37:07.461 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:37:07.461 INFO:teuthology.orchestra.run.smithi167.stdout: osd open classes on start = true 2024-04-17T10:37:07.462 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug pg log writeout = true 2024-04-17T10:37:07.462 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:37:07.462 INFO:teuthology.orchestra.run.smithi167.stdout: osd deep scrub update digest min age = 30 2024-04-17T10:37:07.462 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:37:07.462 INFO:teuthology.orchestra.run.smithi167.stdout: osd map max advance = 10 2024-04-17T10:37:07.462 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:37:07.462 INFO:teuthology.orchestra.run.smithi167.stdout: journal zero on create = true 2024-04-17T10:37:07.462 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:37:07.462 INFO:teuthology.orchestra.run.smithi167.stdout: filestore ondisk finisher threads = 3 2024-04-17T10:37:07.462 INFO:teuthology.orchestra.run.smithi167.stdout: filestore apply finisher threads = 3 2024-04-17T10:37:07.462 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:37:07.463 INFO:teuthology.orchestra.run.smithi167.stdout: bdev debug aio = true 2024-04-17T10:37:07.463 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug misdirected ops = true 2024-04-17T10:37:07.463 INFO:teuthology.orchestra.run.smithi167.stdout: bdev async discard = True 2024-04-17T10:37:07.463 INFO:teuthology.orchestra.run.smithi167.stdout: bdev enable discard = True 2024-04-17T10:37:07.463 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore allocator = bitmap 2024-04-17T10:37:07.463 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore block size = 96636764160 2024-04-17T10:37:07.463 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore fsck on mount = True 2024-04-17T10:37:07.463 INFO:teuthology.orchestra.run.smithi167.stdout: debug bluefs = 1/20 2024-04-17T10:37:07.463 INFO:teuthology.orchestra.run.smithi167.stdout: debug bluestore = 1/20 2024-04-17T10:37:07.463 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2024-04-17T10:37:07.463 INFO:teuthology.orchestra.run.smithi167.stdout: debug osd = 20 2024-04-17T10:37:07.463 INFO:teuthology.orchestra.run.smithi167.stdout: debug rocksdb = 4/10 2024-04-17T10:37:07.464 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd backfillfull_ratio = 0.85 2024-04-17T10:37:07.464 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd full ratio = 0.9 2024-04-17T10:37:07.464 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd nearfull ratio = 0.8 2024-04-17T10:37:07.464 INFO:teuthology.orchestra.run.smithi167.stdout: osd failsafe full ratio = 0.95 2024-04-17T10:37:07.464 INFO:teuthology.orchestra.run.smithi167.stdout: osd objectstore = bluestore 2024-04-17T10:37:07.464 INFO:teuthology.orchestra.run.smithi167.stdout: osd op complaint time = 180 2024-04-17T10:37:07.464 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:37:07.464 INFO:teuthology.orchestra.run.smithi167.stdout:[mgr] 2024-04-17T10:37:07.464 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2024-04-17T10:37:07.464 INFO:teuthology.orchestra.run.smithi167.stdout: debug mgr = 20 2024-04-17T10:37:07.464 INFO:teuthology.orchestra.run.smithi167.stdout: debug mon = 20 2024-04-17T10:37:07.465 INFO:teuthology.orchestra.run.smithi167.stdout: debug auth = 20 2024-04-17T10:37:07.465 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min pgs per osd = 4 2024-04-17T10:37:07.465 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min bytes per osd = 10 2024-04-17T10:37:07.465 INFO:teuthology.orchestra.run.smithi167.stdout: mgr/telemetry/nag = false 2024-04-17T10:37:07.465 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:37:07.465 INFO:teuthology.orchestra.run.smithi167.stdout:[mon] 2024-04-17T10:37:07.465 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2024-04-17T10:37:07.465 INFO:teuthology.orchestra.run.smithi167.stdout: debug mon = 20 2024-04-17T10:37:07.465 INFO:teuthology.orchestra.run.smithi167.stdout: debug paxos = 20 2024-04-17T10:37:07.465 INFO:teuthology.orchestra.run.smithi167.stdout: debug auth = 20 2024-04-17T10:37:07.465 INFO:teuthology.orchestra.run.smithi167.stdout: mon data avail warn = 5 2024-04-17T10:37:07.465 INFO:teuthology.orchestra.run.smithi167.stdout: mon mgr mkfs grace = 240 2024-04-17T10:37:07.466 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min pgs per osd = 4 2024-04-17T10:37:07.466 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd reporter subtree level = osd 2024-04-17T10:37:07.466 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd prime pg temp = true 2024-04-17T10:37:07.466 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min bytes per osd = 10 2024-04-17T10:37:07.466 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:37:07.466 INFO:teuthology.orchestra.run.smithi167.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-17T10:37:07.466 INFO:teuthology.orchestra.run.smithi167.stdout: auth mon ticket ttl = 660 # 11m 2024-04-17T10:37:07.466 INFO:teuthology.orchestra.run.smithi167.stdout: auth service ticket ttl = 240 # 4m 2024-04-17T10:37:07.466 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:37:07.466 INFO:teuthology.orchestra.run.smithi167.stdout: # don't complain about insecure global_id in the test suite 2024-04-17T10:37:07.466 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-17T10:37:07.466 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-17T10:37:07.467 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:37:07.467 INFO:teuthology.orchestra.run.smithi167.stdout: # 1m isn't quite enough 2024-04-17T10:37:07.470 INFO:teuthology.orchestra.run.smithi167.stdout: mon_down_mkfs_grace = 2m 2024-04-17T10:37:07.470 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:37:07.470 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_filestore_osds = false 2024-04-17T10:37:07.470 INFO:teuthology.orchestra.run.smithi167.stdout: mon op complaint time = 120 2024-04-17T10:37:07.470 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-17T10:37:07.470 INFO:teuthology.orchestra.run.smithi167.stdout:[client] 2024-04-17T10:37:07.470 INFO:teuthology.orchestra.run.smithi167.stdout: rgw cache enabled = true 2024-04-17T10:37:07.471 INFO:teuthology.orchestra.run.smithi167.stdout: rgw enable ops log = true 2024-04-17T10:37:07.471 INFO:teuthology.orchestra.run.smithi167.stdout: rgw enable usage log = true 2024-04-17T10:37:07.471 INFO:teuthology.orchestra.run.smithi167.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-17T10:37:07.471 INFO:teuthology.orchestra.run.smithi167.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-17T10:37:07.471 INFO:teuthology.orchestra.run.smithi167.stdout: client mount timeout = 600 2024-04-17T10:37:07.471 INFO:teuthology.orchestra.run.smithi167.stdout: debug client = 20 2024-04-17T10:37:07.471 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2024-04-17T10:37:07.471 INFO:teuthology.orchestra.run.smithi167.stdout: pid file = "" 2024-04-17T10:37:07.471 INFO:teuthology.orchestra.run.smithi167.stdout: rados mon op timeout = 900 2024-04-17T10:37:07.471 INFO:teuthology.orchestra.run.smithi167.stdout: rados osd op timeout = 900 2024-04-17T10:37:07.471 INFO:teuthology.orchestra.run.smithi167.stdout:[mon.a] 2024-04-17T10:37:07.472 INFO:teuthology.orchestra.run.smithi167.stdout:[mon.b] 2024-04-17T10:37:07.472 INFO:teuthology.orchestra.run.smithi167.stdout:[mon.c] 2024-04-17T10:37:07.472 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.a] 2024-04-17T10:37:07.472 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.c] 2024-04-17T10:37:07.472 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.b] 2024-04-17T10:37:07.472 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.d] 2024-04-17T10:37:07.472 INFO:teuthology.orchestra.run.smithi167.stdout:[mds] 2024-04-17T10:37:07.472 INFO:teuthology.orchestra.run.smithi167.stdout: debug mds = 20 2024-04-17T10:37:07.472 INFO:teuthology.orchestra.run.smithi167.stdout: debug mds balancer = 20 2024-04-17T10:37:07.472 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2024-04-17T10:37:07.472 INFO:teuthology.orchestra.run.smithi167.stdout: mds cache max size = 1000 2024-04-17T10:37:07.472 INFO:teuthology.orchestra.run.smithi167.stdout: mds debug frag = True 2024-04-17T10:37:07.473 INFO:teuthology.orchestra.run.smithi167.stdout: mds debug scatterstat = True 2024-04-17T10:37:07.473 INFO:teuthology.orchestra.run.smithi167.stdout: mds log max segments = 1 2024-04-17T10:37:07.473 INFO:teuthology.orchestra.run.smithi167.stdout: mds op complaint time = 180 2024-04-17T10:37:07.473 INFO:teuthology.orchestra.run.smithi167.stdout: mds verify scatter = True 2024-04-17T10:37:07.473 INFO:teuthology.orchestra.run.smithi167.stdout: osd op complaint time = 180 2024-04-17T10:37:07.473 INFO:teuthology.orchestra.run.smithi167.stdout: rados mon op timeout = 900 2024-04-17T10:37:07.473 INFO:teuthology.orchestra.run.smithi167.stdout: rados osd op timeout = 900 2024-04-17T10:37:07.473 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-17T10:37:07.473 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-17T10:37:07.544 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-17T10:37:07.544 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:37:07.545 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-17T10:37:07.550 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:37:07.550 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-17T10:37:07.598 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi083.front.sepia.ceph.com 2024-04-17T10:37:07.598 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:37:07.598 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-17T10:37:07.598 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-17T10:37:07.662 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:37:07.662 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-17T10:37:07.710 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi167.front.sepia.ceph.com 2024-04-17T10:37:07.711 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:37:07.711 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-17T10:37:07.711 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-17T10:37:07.735 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:37:07.735 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-17T10:37:07.784 INFO:tasks.ceph:Setting up mon nodes... 2024-04-17T10:37:07.785 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-17T10:37:07.785 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-17T10:37:07.828 INFO:teuthology.orchestra.run.smithi083.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-17T10:37:07.831 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-17T10:37:07.874 INFO:teuthology.orchestra.run.smithi167.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-17T10:37:07.880 INFO:tasks.ceph:Setting up mds nodes... 2024-04-17T10:37:07.880 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-17T10:37:07.922 INFO:teuthology.orchestra.run.smithi083.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-17T10:37:07.924 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-17T10:37:07.978 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-17T10:37:08.062 INFO:teuthology.orchestra.run.smithi083.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-17T10:37:08.065 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-17T10:37:08.122 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-17T10:37:08.166 INFO:teuthology.orchestra.run.smithi167.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-17T10:37:08.169 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-17T10:37:08.224 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-17T10:37:08.308 INFO:teuthology.orchestra.run.smithi167.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-17T10:37:08.311 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-17T10:37:08.368 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-17T10:37:08.368 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-17T10:37:08.404 INFO:teuthology.orchestra.run.smithi083.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-17T10:37:08.416 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-17T10:37:08.491 INFO:teuthology.orchestra.run.smithi083.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-17T10:37:08.498 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-17T10:37:08.533 INFO:teuthology.orchestra.run.smithi167.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-17T10:37:08.545 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-17T10:37:08.618 INFO:teuthology.orchestra.run.smithi167.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-17T10:37:08.628 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-17T10:37:08.628 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi083.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi167.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-17T10:37:08.628 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-17T10:37:08.642 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-17T10:37:08.642 INFO:tasks.ceph:role: osd.0 2024-04-17T10:37:08.642 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi083.front.sepia.ceph.com 2024-04-17T10:37:08.642 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-17T10:37:08.705 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T10:37:08.705 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T10:37:08.705 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T10:37:08.705 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-17T10:37:08.705 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T10:37:08.705 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2024-04-17T10:37:08.705 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T10:37:08.706 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-17T10:37:08.706 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T10:37:08.706 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T10:37:09.162 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2024-04-17T10:37:09.163 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2024-04-17T10:37:09.164 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-17T10:37:09.265 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-17T10:37:09.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:37:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: /sbin/restorecon: command not found 2024-04-17T10:37:09.277 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-17T10:37:09.333 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-17T10:37:09.334 INFO:tasks.ceph:role: osd.1 2024-04-17T10:37:09.334 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi083.front.sepia.ceph.com 2024-04-17T10:37:09.334 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-17T10:37:09.398 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T10:37:09.398 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T10:37:09.398 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T10:37:09.398 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-17T10:37:09.398 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T10:37:09.398 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2024-04-17T10:37:09.399 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T10:37:09.399 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-17T10:37:09.399 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T10:37:09.399 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T10:37:09.880 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2024-04-17T10:37:09.882 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2024-04-17T10:37:09.883 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-17T10:37:10.044 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-17T10:37:10.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:37:10.056 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: /sbin/restorecon: command not found 2024-04-17T10:37:10.056 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-17T10:37:10.109 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-17T10:37:10.109 INFO:tasks.ceph:role: osd.2 2024-04-17T10:37:10.109 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi083.front.sepia.ceph.com 2024-04-17T10:37:10.109 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-17T10:37:10.172 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T10:37:10.172 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T10:37:10.172 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T10:37:10.172 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-17T10:37:10.173 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T10:37:10.173 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2024-04-17T10:37:10.173 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T10:37:10.173 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-17T10:37:10.173 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T10:37:10.173 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T10:37:10.618 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2024-04-17T10:37:10.620 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2024-04-17T10:37:10.620 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-17T10:37:10.783 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-17T10:37:10.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:37:10.796 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: /sbin/restorecon: command not found 2024-04-17T10:37:10.796 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-17T10:37:10.849 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-17T10:37:10.849 INFO:tasks.ceph:role: osd.3 2024-04-17T10:37:10.850 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi083.front.sepia.ceph.com 2024-04-17T10:37:10.850 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-17T10:37:10.913 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T10:37:10.913 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T10:37:10.913 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T10:37:10.913 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-17T10:37:10.913 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T10:37:10.913 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2024-04-17T10:37:10.913 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T10:37:10.913 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-17T10:37:10.914 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T10:37:10.914 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T10:37:11.341 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2024-04-17T10:37:11.343 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2024-04-17T10:37:11.343 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-17T10:37:11.443 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-17T10:37:11.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:37:11.452 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: /sbin/restorecon: command not found 2024-04-17T10:37:11.453 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T10:37:11.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:11.559+0000 7f8e47f037c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-17T10:37:11.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:11.563+0000 7f8e47f037c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-17T10:37:11.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:11.563+0000 7f8e47f037c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-17T10:37:11.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:11.563+0000 7f8e47f037c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-17T10:37:11.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:11.563+0000 7f8e47f037c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-17T10:37:11.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:11.563+0000 7f8e47f037c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-17T10:37:14.530 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-17T10:37:14.544 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T10:37:14.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:14.627+0000 7f79f8c847c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-17T10:37:14.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:14.627+0000 7f79f8c847c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-17T10:37:14.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:14.627+0000 7f79f8c847c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-17T10:37:14.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:14.627+0000 7f79f8c847c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-17T10:37:14.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:14.627+0000 7f79f8c847c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-17T10:37:14.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:14.631+0000 7f79f8c847c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-17T10:37:17.611 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-17T10:37:17.671 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T10:37:17.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:17.755+0000 7fb60b8487c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-17T10:37:17.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:17.755+0000 7fb60b8487c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-17T10:37:17.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:17.755+0000 7fb60b8487c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-17T10:37:17.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:17.755+0000 7fb60b8487c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-17T10:37:17.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:17.755+0000 7fb60b8487c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-17T10:37:17.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:17.755+0000 7fb60b8487c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-17T10:37:20.742 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-17T10:37:20.802 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T10:37:20.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:20.887+0000 7f2f134407c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-17T10:37:20.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:20.887+0000 7f2f134407c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-17T10:37:20.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:20.887+0000 7f2f134407c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-17T10:37:20.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:20.887+0000 7f2f134407c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-17T10:37:20.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:20.887+0000 7f2f134407c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-17T10:37:20.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:20.891+0000 7f2f134407c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-17T10:37:23.853 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-17T10:37:23.911 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-17T10:37:23.925 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-17T10:37:23.925 INFO:tasks.ceph:role: osd.4 2024-04-17T10:37:23.925 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi167.front.sepia.ceph.com 2024-04-17T10:37:23.926 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-17T10:37:23.988 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T10:37:23.988 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T10:37:23.988 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T10:37:23.988 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-17T10:37:23.988 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T10:37:23.988 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2024-04-17T10:37:23.989 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T10:37:23.989 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-17T10:37:23.989 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T10:37:23.989 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T10:37:24.438 INFO:teuthology.orchestra.run.smithi167.stdout:Discarding blocks...Done. 2024-04-17T10:37:24.440 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2024-04-17T10:37:24.440 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-17T10:37:24.537 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-17T10:37:24.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:37:24.548 INFO:teuthology.orchestra.run.smithi167.stderr:sudo: /sbin/restorecon: command not found 2024-04-17T10:37:24.548 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-17T10:37:24.603 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-17T10:37:24.603 INFO:tasks.ceph:role: osd.5 2024-04-17T10:37:24.603 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi167.front.sepia.ceph.com 2024-04-17T10:37:24.603 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-17T10:37:24.668 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T10:37:24.668 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T10:37:24.668 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T10:37:24.668 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-17T10:37:24.668 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T10:37:24.669 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2024-04-17T10:37:24.669 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T10:37:24.669 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-17T10:37:24.669 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T10:37:24.669 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T10:37:25.134 INFO:teuthology.orchestra.run.smithi167.stdout:Discarding blocks...Done. 2024-04-17T10:37:25.136 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2024-04-17T10:37:25.136 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-17T10:37:25.275 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-17T10:37:25.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:37:25.287 INFO:teuthology.orchestra.run.smithi167.stderr:sudo: /sbin/restorecon: command not found 2024-04-17T10:37:25.288 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-17T10:37:25.343 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-17T10:37:25.343 INFO:tasks.ceph:role: osd.6 2024-04-17T10:37:25.343 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi167.front.sepia.ceph.com 2024-04-17T10:37:25.344 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-17T10:37:25.407 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T10:37:25.408 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T10:37:25.408 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T10:37:25.408 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-17T10:37:25.408 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T10:37:25.408 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2024-04-17T10:37:25.408 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T10:37:25.408 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-17T10:37:25.408 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T10:37:25.408 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T10:37:25.892 INFO:teuthology.orchestra.run.smithi167.stdout:Discarding blocks...Done. 2024-04-17T10:37:25.894 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2024-04-17T10:37:25.895 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-17T10:37:26.048 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-17T10:37:26.060 INFO:teuthology.orchestra.run.smithi167.stderr:sudo: /sbin/restorecon: command not found 2024-04-17T10:37:26.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:37:26.061 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-17T10:37:26.115 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-17T10:37:26.116 INFO:tasks.ceph:role: osd.7 2024-04-17T10:37:26.116 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi167.front.sepia.ceph.com 2024-04-17T10:37:26.116 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-17T10:37:26.180 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T10:37:26.180 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T10:37:26.180 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T10:37:26.180 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-17T10:37:26.180 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T10:37:26.180 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2024-04-17T10:37:26.180 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T10:37:26.180 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-17T10:37:26.180 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T10:37:26.180 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T10:37:26.611 INFO:teuthology.orchestra.run.smithi167.stdout:Discarding blocks...Done. 2024-04-17T10:37:26.614 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2024-04-17T10:37:26.614 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-17T10:37:26.717 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-17T10:37:26.727 INFO:teuthology.orchestra.run.smithi167.stderr:sudo: /sbin/restorecon: command not found 2024-04-17T10:37:26.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:37:26.728 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T10:37:26.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:26.837+0000 7faf70df57c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-17T10:37:26.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:26.837+0000 7faf70df57c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-17T10:37:26.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:26.837+0000 7faf70df57c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-17T10:37:26.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:26.837+0000 7faf70df57c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-17T10:37:26.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:26.837+0000 7faf70df57c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-17T10:37:26.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:26.841+0000 7faf70df57c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-17T10:37:29.819 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-17T10:37:29.833 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T10:37:29.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:29.917+0000 7f6ba95357c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-17T10:37:29.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:29.917+0000 7f6ba95357c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-17T10:37:29.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:29.917+0000 7f6ba95357c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-17T10:37:29.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:29.917+0000 7f6ba95357c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-17T10:37:29.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:29.917+0000 7f6ba95357c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-17T10:37:29.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:29.917+0000 7f6ba95357c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-17T10:37:32.913 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-17T10:37:32.968 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T10:37:33.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:33.049+0000 7ff70e95f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-17T10:37:33.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:33.049+0000 7ff70e95f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-17T10:37:33.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:33.049+0000 7ff70e95f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-17T10:37:33.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:33.049+0000 7ff70e95f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-17T10:37:33.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:33.049+0000 7ff70e95f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-17T10:37:33.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:33.049+0000 7ff70e95f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-17T10:37:35.670 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:37:35.672 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:37:36.021 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-17T10:37:36.035 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T10:37:36.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:36.117+0000 7f02645227c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-17T10:37:36.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:36.117+0000 7f02645227c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-17T10:37:36.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:36.117+0000 7f02645227c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-17T10:37:36.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:36.117+0000 7f02645227c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-17T10:37:36.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:36.117+0000 7f02645227c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-17T10:37:36.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:36.121+0000 7f02645227c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-17T10:37:39.104 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-17T10:37:39.160 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-17T10:37:39.160 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:37:39.161 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-17T10:37:39.174 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:37:39.174 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-17T10:37:39.229 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:37:39.229 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-17T10:37:39.284 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:37:39.285 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-17T10:37:39.340 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:37:39.340 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-17T10:37:39.392 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:37:39.392 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-17T10:37:39.444 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:37:39.444 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-17T10:37:39.496 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:37:39.496 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-17T10:37:39.510 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:37:39.510 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-17T10:37:39.563 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:37:39.563 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-17T10:37:39.615 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:37:39.615 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-17T10:37:39.671 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:37:39.671 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-17T10:37:39.726 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:37:39.726 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-17T10:37:39.779 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:37:39.779 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-17T10:37:39.835 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:37:39.835 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-17T10:37:39.841 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:37:39.841 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-17T10:37:39.889 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:37:39.890 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-17T10:37:39.895 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:37:39.895 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-17T10:37:39.944 INFO:tasks.ceph:Adding keys to all mons... 2024-04-17T10:37:39.944 DEBUG:teuthology.orchestra.run.smithi083:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-17T10:37:39.947 DEBUG:teuthology.orchestra.run.smithi167:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-17T10:37:39.991 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr.y] 2024-04-17T10:37:39.991 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBTph9mjFbRMRAAjY2d8POBzp8hEp9GSO0ylQ== 2024-04-17T10:37:39.991 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.a] 2024-04-17T10:37:39.991 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBTph9mfpxsNxAANpL7rcWevXwd2KMftSQnEQ== 2024-04-17T10:37:39.991 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.c] 2024-04-17T10:37:39.992 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBUph9mupo3BBAAyKH801vpFLXYs9mfzSlT0w== 2024-04-17T10:37:39.992 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.0] 2024-04-17T10:37:39.992 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBXph9m1HWhIRAAOq+9nGq7OGonUCE2aeiAzQ== 2024-04-17T10:37:39.992 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.1] 2024-04-17T10:37:39.992 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBaph9mlMmtJRAAXHLMjXfm/+WjIFURLkbAfA== 2024-04-17T10:37:39.992 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.2] 2024-04-17T10:37:39.993 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBdph9mBsNPLRAAtR5hwGdDkjLDLS19QUDDYQ== 2024-04-17T10:37:39.993 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.3] 2024-04-17T10:37:39.993 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBgph9mZ4EwNRAAIyL7e0EK1523dPXmbOOkzw== 2024-04-17T10:37:39.993 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr.x] 2024-04-17T10:37:39.993 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBTph9mMFptNBAA/G0zEfMZTCy9Iox+MwHG3Q== 2024-04-17T10:37:39.993 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.b] 2024-04-17T10:37:39.993 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBUph9mXlI9ChAAYdcrGVmtYdtWWgCLVmpTTg== 2024-04-17T10:37:39.994 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.d] 2024-04-17T10:37:39.994 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBUph9mbm++EhAAIBMOtQoD+kPeMqR6gK5ctw== 2024-04-17T10:37:39.994 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.4] 2024-04-17T10:37:39.994 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBmph9mphAsMhAAtF4uV28Nb66PNjuugB12Hw== 2024-04-17T10:37:39.994 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.5] 2024-04-17T10:37:39.994 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBpph9mHJQNNxAA72eOFYFbhakQac1JKRNjzw== 2024-04-17T10:37:39.995 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.6] 2024-04-17T10:37:39.995 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBtph9mb6RKAxAAoVX6Cp7l7j3Lm1ES2Sq+fQ== 2024-04-17T10:37:39.995 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.7] 2024-04-17T10:37:39.995 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBwph9mdrdrBxAApDwjml2ahsLbRol4uL/BxA== 2024-04-17T10:37:39.995 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-04-17T10:37:39.995 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBUph9mHGmXGBAAZF+JiAG9YrFtbW+dCy2IEQ== 2024-04-17T10:37:39.995 INFO:teuthology.orchestra.run.smithi083.stdout:[client.2] 2024-04-17T10:37:39.995 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBUph9m+wKgHRAAe8IUFx34ItLpsaDqSYyEgA== 2024-04-17T10:37:39.996 INFO:teuthology.orchestra.run.smithi083.stdout:[client.1] 2024-04-17T10:37:39.996 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBUph9mB7wgIBAABdavAo/uzVuSjfkkcYtqJg== 2024-04-17T10:37:39.996 INFO:teuthology.orchestra.run.smithi083.stdout:[client.3] 2024-04-17T10:37:39.996 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBUph9mfJ0yJRAAHH7rRh/BUIujTT04AF2l0w== 2024-04-17T10:37:39.998 INFO:teuthology.orchestra.run.smithi167.stdout:[mgr.y] 2024-04-17T10:37:39.998 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQBTph9mjFbRMRAAjY2d8POBzp8hEp9GSO0ylQ== 2024-04-17T10:37:39.998 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.a] 2024-04-17T10:37:39.998 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQBTph9mfpxsNxAANpL7rcWevXwd2KMftSQnEQ== 2024-04-17T10:37:39.998 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.c] 2024-04-17T10:37:39.998 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQBUph9mupo3BBAAyKH801vpFLXYs9mfzSlT0w== 2024-04-17T10:37:39.998 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.0] 2024-04-17T10:37:39.999 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQBXph9m1HWhIRAAOq+9nGq7OGonUCE2aeiAzQ== 2024-04-17T10:37:39.999 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.1] 2024-04-17T10:37:39.999 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQBaph9mlMmtJRAAXHLMjXfm/+WjIFURLkbAfA== 2024-04-17T10:37:39.999 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.2] 2024-04-17T10:37:39.999 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQBdph9mBsNPLRAAtR5hwGdDkjLDLS19QUDDYQ== 2024-04-17T10:37:39.999 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.3] 2024-04-17T10:37:39.999 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQBgph9mZ4EwNRAAIyL7e0EK1523dPXmbOOkzw== 2024-04-17T10:37:39.999 INFO:teuthology.orchestra.run.smithi167.stdout:[mgr.x] 2024-04-17T10:37:39.999 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQBTph9mMFptNBAA/G0zEfMZTCy9Iox+MwHG3Q== 2024-04-17T10:37:39.999 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.b] 2024-04-17T10:37:40.000 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQBUph9mXlI9ChAAYdcrGVmtYdtWWgCLVmpTTg== 2024-04-17T10:37:40.000 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.d] 2024-04-17T10:37:40.000 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQBUph9mbm++EhAAIBMOtQoD+kPeMqR6gK5ctw== 2024-04-17T10:37:40.000 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.4] 2024-04-17T10:37:40.000 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQBmph9mphAsMhAAtF4uV28Nb66PNjuugB12Hw== 2024-04-17T10:37:40.000 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.5] 2024-04-17T10:37:40.000 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQBpph9mHJQNNxAA72eOFYFbhakQac1JKRNjzw== 2024-04-17T10:37:40.000 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.6] 2024-04-17T10:37:40.000 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQBtph9mb6RKAxAAoVX6Cp7l7j3Lm1ES2Sq+fQ== 2024-04-17T10:37:40.000 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.7] 2024-04-17T10:37:40.000 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQBwph9mdrdrBxAApDwjml2ahsLbRol4uL/BxA== 2024-04-17T10:37:40.000 INFO:teuthology.orchestra.run.smithi167.stdout:[client.0] 2024-04-17T10:37:40.000 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQBUph9mHGmXGBAAZF+JiAG9YrFtbW+dCy2IEQ== 2024-04-17T10:37:40.001 INFO:teuthology.orchestra.run.smithi167.stdout:[client.2] 2024-04-17T10:37:40.001 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQBUph9m+wKgHRAAe8IUFx34ItLpsaDqSYyEgA== 2024-04-17T10:37:40.001 INFO:teuthology.orchestra.run.smithi167.stdout:[client.1] 2024-04-17T10:37:40.001 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQBUph9mB7wgIBAABdavAo/uzVuSjfkkcYtqJg== 2024-04-17T10:37:40.001 INFO:teuthology.orchestra.run.smithi167.stdout:[client.3] 2024-04-17T10:37:40.001 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQBUph9mfJ0yJRAAHH7rRh/BUIujTT04AF2l0w== 2024-04-17T10:37:40.002 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-17T10:37:40.043 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-17T10:37:40.089 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-17T10:37:40.091 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-17T10:37:40.135 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-17T10:37:40.182 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-17T10:37:40.225 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T10:37:40.267 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T10:37:40.312 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T10:37:40.346 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T10:37:40.397 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T10:37:40.427 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T10:37:40.479 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T10:37:40.506 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T10:37:40.563 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-17T10:37:40.590 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-17T10:37:40.647 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-17T10:37:40.675 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-17T10:37:40.733 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-17T10:37:40.762 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-17T10:37:40.823 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T10:37:40.850 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T10:37:40.892 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T10:37:40.934 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T10:37:40.973 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T10:37:41.019 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T10:37:41.059 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T10:37:41.103 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T10:37:41.146 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-17T10:37:41.148 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-17T10:37:41.229 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-17T10:37:41.234 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-17T10:37:41.278 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-17T10:37:41.280 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-17T10:37:41.356 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-17T10:37:41.362 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-17T10:37:41.439 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-17T10:37:41.440 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-17T10:37:41.456 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-17T10:37:41.543 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-17T10:37:41.553 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-17T10:37:41.568 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-17T10:37:41.660 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-17T10:37:41.671 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-17T10:37:41.725 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-17T10:37:41.812 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-17T10:37:41.866 DEBUG:teuthology.orchestra.run.smithi083:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-17T10:37:41.868 DEBUG:teuthology.orchestra.run.smithi167:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-17T10:37:41.911 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-17T10:37:41.912 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-17T10:37:41.912 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-17T10:37:41.914 INFO:tasks.ceph.mon.a:Started 2024-04-17T10:37:41.915 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-17T10:37:41.915 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-17T10:37:41.957 INFO:tasks.ceph.mon.b:Started 2024-04-17T10:37:41.957 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-17T10:37:41.957 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-17T10:37:41.959 INFO:tasks.ceph.mon.c:Started 2024-04-17T10:37:41.960 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-17T10:37:41.960 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-17T10:37:41.960 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-17T10:37:41.963 INFO:tasks.ceph.mgr.y:Started 2024-04-17T10:37:41.963 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-17T10:37:41.963 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-17T10:37:41.965 INFO:tasks.ceph.mgr.x:Started 2024-04-17T10:37:41.965 INFO:tasks.ceph:Setting crush tunables to default 2024-04-17T10:37:41.966 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph osd crush tunables default 2024-04-17T10:37:42.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:42.087+0000 7f88a041f640 1 Processor -- start 2024-04-17T10:37:42.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:42.087+0000 7f88a041f640 1 -- start start 2024-04-17T10:37:42.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:42.087+0000 7f88a041f640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f889806f150 0x7f88980ca520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:42.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:42.087+0000 7f88a041f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88980cae70 con 0x7f889815dbe0 2024-04-17T10:37:42.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:42.087+0000 7f88a041f640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88980c6850 con 0x7f88980caa60 2024-04-17T10:37:42.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:42.087+0000 7f88a041f640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88980c5fb0 con 0x7f889806f150 2024-04-17T10:37:42.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:42.087+0000 7f889e1bb640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f889815dbe0 0x7f88980c5780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54074/0 (socket says 172.21.15.83:54074) 2024-04-17T10:37:42.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:42.087+0000 7f889e1bb640 1 -- 172.21.15.83:0/1322837979 learned_addr learned my addr 172.21.15.83:0/1322837979 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:37:42.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:42.087+0000 7f889d9ba640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.167:3300/0 conn(0x7f889806f150 0x7f88980ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:42.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:42.087+0000 7f889d9ba640 1 -- 172.21.15.83:0/1322837979 >> v2:172.21.15.167:3300/0 conn(0x7f889806f150 msgr2=0x7f88980ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-17T10:37:42.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:42.087+0000 7f889d9ba640 1 -- 172.21.15.83:0/1322837979 >> v2:172.21.15.167:3300/0 conn(0x7f889806f150 msgr2=0x7f88980ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T10:37:42.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:42.087+0000 7f889d9ba640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.167:3300/0 conn(0x7f889806f150 0x7f88980ca520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-17T10:37:42.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:42.087+0000 7f889d9ba640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.167:3300/0 conn(0x7f889806f150 0x7f88980ca520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-17T10:37:42.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:42.291+0000 7f889d9ba640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.167:3300/0 conn(0x7f889806f150 0x7f88980ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:42.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:42.291+0000 7f889d9ba640 1 -- 172.21.15.83:0/1322837979 >> v2:172.21.15.167:3300/0 conn(0x7f889806f150 msgr2=0x7f88980ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-17T10:37:42.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:42.291+0000 7f889d9ba640 1 -- 172.21.15.83:0/1322837979 >> v2:172.21.15.167:3300/0 conn(0x7f889806f150 msgr2=0x7f88980ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T10:37:42.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:42.291+0000 7f889d9ba640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.167:3300/0 conn(0x7f889806f150 0x7f88980ca520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-17T10:37:42.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:42.291+0000 7f889d9ba640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.167:3300/0 conn(0x7f889806f150 0x7f88980ca520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-17T10:37:42.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:42.695+0000 7f889d9ba640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.167:3300/0 conn(0x7f889806f150 0x7f88980ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:42.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:42.695+0000 7f889d9ba640 1 -- 172.21.15.83:0/1322837979 >> v2:172.21.15.167:3300/0 conn(0x7f889806f150 msgr2=0x7f88980ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-17T10:37:42.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:42.695+0000 7f889d9ba640 1 -- 172.21.15.83:0/1322837979 >> v2:172.21.15.167:3300/0 conn(0x7f889806f150 msgr2=0x7f88980ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T10:37:42.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:42.695+0000 7f889d9ba640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.167:3300/0 conn(0x7f889806f150 0x7f88980ca520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-17T10:37:42.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:42.695+0000 7f889d9ba640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.167:3300/0 conn(0x7f889806f150 0x7f88980ca520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-17T10:37:43.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:43.499+0000 7f889d9ba640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.167:3300/0 conn(0x7f889806f150 0x7f88980ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:43.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:43.499+0000 7f889d9ba640 1 -- 172.21.15.83:0/1322837979 >> v2:172.21.15.167:3300/0 conn(0x7f889806f150 msgr2=0x7f88980ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-17T10:37:43.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:43.499+0000 7f889d9ba640 1 -- 172.21.15.83:0/1322837979 >> v2:172.21.15.167:3300/0 conn(0x7f889806f150 msgr2=0x7f88980ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T10:37:43.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:43.499+0000 7f889d9ba640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.167:3300/0 conn(0x7f889806f150 0x7f88980ca520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-17T10:37:43.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:43.499+0000 7f889d9ba640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.167:3300/0 conn(0x7f889806f150 0x7f88980ca520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-17T10:37:45.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.087+0000 7f888ffff640 1 -- 172.21.15.83:0/1322837979 >> v2:172.21.15.167:3300/0 conn(0x7f889806f150 msgr2=0x7f88980ca520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T10:37:45.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.087+0000 7f888ffff640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.167:3300/0 conn(0x7f889806f150 0x7f88980ca520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:45.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.087+0000 7f888ffff640 1 -- 172.21.15.83:0/1322837979 >> v1:172.21.15.167:6789/0 conn(0x7f88980caa60 legacy=0x7f88980ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:45.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.087+0000 7f888ffff640 1 -- 172.21.15.83:0/1322837979 >> v1:172.21.15.83:6789/0 conn(0x7f889815dbe0 legacy=0x7f88980c5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:45.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.087+0000 7f888ffff640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.83:3300/0 conn(0x7f887c0036f0 0x7f887c005b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:45.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.087+0000 7f888ffff640 1 --2- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c006190 0x7f887c008650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:45.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.087+0000 7f888ffff640 1 -- 172.21.15.83:0/1322837979 --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88980c5fb0 con 0x7f887c0036f0 2024-04-17T10:37:45.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.087+0000 7f888ffff640 1 -- 172.21.15.83:0/1322837979 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887c008d70 con 0x7f887c006190 2024-04-17T10:37:45.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.087+0000 7f889e1bb640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.83:3300/0 conn(0x7f887c0036f0 0x7f887c005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:45.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.087+0000 7f889d9ba640 1 --2- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c006190 0x7f887c008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:45.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.087+0000 7f889e1bb640 1 -- 172.21.15.83:0/1322837979 >> v2:172.21.15.83:3300/0 conn(0x7f887c0036f0 msgr2=0x7f887c005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-17T10:37:45.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.087+0000 7f889e1bb640 1 -- 172.21.15.83:0/1322837979 >> v2:172.21.15.83:3300/0 conn(0x7f887c0036f0 msgr2=0x7f887c005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T10:37:45.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.087+0000 7f889e1bb640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.83:3300/0 conn(0x7f887c0036f0 0x7f887c005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-17T10:37:45.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.087+0000 7f889d9ba640 1 -- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c006190 msgr2=0x7f887c008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-17T10:37:45.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.087+0000 7f889d9ba640 1 -- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c006190 msgr2=0x7f887c008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T10:37:45.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.087+0000 7f889d9ba640 1 --2- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c006190 0x7f887c008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-17T10:37:45.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.087+0000 7f889d9ba640 1 --2- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c006190 0x7f887c008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-17T10:37:45.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.087+0000 7f889e1bb640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.83:3300/0 conn(0x7f887c0036f0 0x7f887c005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-17T10:37:45.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.291+0000 7f889e1bb640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.83:3300/0 conn(0x7f887c0036f0 0x7f887c005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:45.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.291+0000 7f889d9ba640 1 --2- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c006190 0x7f887c008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:45.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.291+0000 7f889e1bb640 1 -- 172.21.15.83:0/1322837979 >> v2:172.21.15.83:3300/0 conn(0x7f887c0036f0 msgr2=0x7f887c005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-17T10:37:45.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.291+0000 7f889e1bb640 1 -- 172.21.15.83:0/1322837979 >> v2:172.21.15.83:3300/0 conn(0x7f887c0036f0 msgr2=0x7f887c005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T10:37:45.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.291+0000 7f889e1bb640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.83:3300/0 conn(0x7f887c0036f0 0x7f887c005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-17T10:37:45.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.291+0000 7f889e1bb640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.83:3300/0 conn(0x7f887c0036f0 0x7f887c005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-17T10:37:45.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.291+0000 7f889d9ba640 1 -- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c006190 msgr2=0x7f887c008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-17T10:37:45.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.291+0000 7f889d9ba640 1 -- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c006190 msgr2=0x7f887c008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T10:37:45.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.291+0000 7f889d9ba640 1 --2- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c006190 0x7f887c008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-17T10:37:45.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.291+0000 7f889d9ba640 1 --2- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c006190 0x7f887c008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-17T10:37:45.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.695+0000 7f889e1bb640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.83:3300/0 conn(0x7f887c0036f0 0x7f887c005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:45.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.695+0000 7f889d9ba640 1 --2- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c006190 0x7f887c008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:45.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.695+0000 7f889e1bb640 1 -- 172.21.15.83:0/1322837979 >> v2:172.21.15.83:3300/0 conn(0x7f887c0036f0 msgr2=0x7f887c005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-17T10:37:45.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.695+0000 7f889e1bb640 1 -- 172.21.15.83:0/1322837979 >> v2:172.21.15.83:3300/0 conn(0x7f887c0036f0 msgr2=0x7f887c005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T10:37:45.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.695+0000 7f889e1bb640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.83:3300/0 conn(0x7f887c0036f0 0x7f887c005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-17T10:37:45.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.695+0000 7f889e1bb640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.83:3300/0 conn(0x7f887c0036f0 0x7f887c005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-17T10:37:45.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.695+0000 7f889d9ba640 1 -- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c006190 msgr2=0x7f887c008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-17T10:37:45.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.695+0000 7f889d9ba640 1 -- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c006190 msgr2=0x7f887c008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T10:37:45.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.695+0000 7f889d9ba640 1 --2- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c006190 0x7f887c008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-17T10:37:45.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:45.695+0000 7f889d9ba640 1 --2- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c006190 0x7f887c008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-17T10:37:46.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:46.499+0000 7f889e1bb640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.83:3300/0 conn(0x7f887c0036f0 0x7f887c005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:46.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:46.499+0000 7f889d9ba640 1 --2- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c006190 0x7f887c008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:46.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:46.499+0000 7f889e1bb640 1 -- 172.21.15.83:0/1322837979 >> v2:172.21.15.83:3300/0 conn(0x7f887c0036f0 msgr2=0x7f887c005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-17T10:37:46.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:46.499+0000 7f889e1bb640 1 -- 172.21.15.83:0/1322837979 >> v2:172.21.15.83:3300/0 conn(0x7f887c0036f0 msgr2=0x7f887c005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T10:37:46.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:46.499+0000 7f889e1bb640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.83:3300/0 conn(0x7f887c0036f0 0x7f887c005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-17T10:37:46.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:46.499+0000 7f889e1bb640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.83:3300/0 conn(0x7f887c0036f0 0x7f887c005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-17T10:37:46.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:46.499+0000 7f889d9ba640 1 -- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c006190 msgr2=0x7f887c008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-17T10:37:46.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:46.499+0000 7f889d9ba640 1 -- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c006190 msgr2=0x7f887c008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T10:37:46.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:46.499+0000 7f889d9ba640 1 --2- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c006190 0x7f887c008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-17T10:37:46.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:46.499+0000 7f889d9ba640 1 --2- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c006190 0x7f887c008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-17T10:37:47.610 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-17T10:37:47.615+0000 7f4d3e02c180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-17T10:37:48.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.086+0000 7f888ffff640 1 -- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c006190 msgr2=0x7f887c008650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T10:37:48.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.086+0000 7f888ffff640 1 --2- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c006190 0x7f887c008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:48.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.086+0000 7f888ffff640 1 -- 172.21.15.83:0/1322837979 >> v2:172.21.15.83:3300/0 conn(0x7f887c0036f0 msgr2=0x7f887c005b70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T10:37:48.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.086+0000 7f888ffff640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.83:3300/0 conn(0x7f887c0036f0 0x7f887c005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:48.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.086+0000 7f889e9bc640 1 -- 172.21.15.83:0/1322837979 reap_dead start 2024-04-17T10:37:48.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.086+0000 7f888ffff640 1 --2- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c003b60 0x7f887c006640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:48.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.086+0000 7f888ffff640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.167:3300/0 conn(0x7f887c006b80 0x7f887c006f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:48.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.086+0000 7f888ffff640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.83:3300/0 conn(0x7f887c0074a0 0x7f887c007950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:48.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.086+0000 7f888ffff640 1 -- 172.21.15.83:0/1322837979 --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88980c5fb0 con 0x7f887c0074a0 2024-04-17T10:37:48.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.086+0000 7f888ffff640 1 -- 172.21.15.83:0/1322837979 --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f887c008d70 con 0x7f887c006b80 2024-04-17T10:37:48.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.086+0000 7f888ffff640 1 -- 172.21.15.83:0/1322837979 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887c008130 con 0x7f887c003b60 2024-04-17T10:37:48.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.086+0000 7f889e9bc640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.83:3300/0 conn(0x7f887c0074a0 0x7f887c007950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:48.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.086+0000 7f889e1bb640 1 --2- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c003b60 0x7f887c006640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:48.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.086+0000 7f889d9ba640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.167:3300/0 conn(0x7f887c006b80 0x7f887c006f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:48.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.086+0000 7f889d9ba640 1 -- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c003b60 msgr2=0x7f887c006640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:37:48.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.086+0000 7f889d9ba640 1 --2- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c003b60 0x7f887c006640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:48.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.086+0000 7f889d9ba640 1 -- 172.21.15.83:0/1322837979 >> v2:172.21.15.83:3300/0 conn(0x7f887c0074a0 msgr2=0x7f887c007950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:37:48.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.086+0000 7f889d9ba640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.83:3300/0 conn(0x7f887c0074a0 0x7f887c007950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:48.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.086+0000 7f889d9ba640 1 -- 172.21.15.83:0/1322837979 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88980c6290 con 0x7f887c006b80 2024-04-17T10:37:48.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.086+0000 7f889e1bb640 1 --2- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c003b60 0x7f887c006640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:37:48.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.086+0000 7f889e9bc640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.83:3300/0 conn(0x7f887c0074a0 0x7f887c007950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:37:48.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.094+0000 7f889d9ba640 1 --2- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f887c006b80 0x7f887c006f60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8898068460 tx=0x7f888801d9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa646275bf6aab16 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:37:48.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.094+0000 7f889d1b9640 1 -- 172.21.15.83:0/1322837979 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88880b3070 con 0x7f887c006b80 2024-04-17T10:37:48.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.094+0000 7f889d1b9640 1 -- 172.21.15.83:0/1322837979 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88880a6940 con 0x7f887c006b80 2024-04-17T10:37:48.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.094+0000 7f889d1b9640 1 -- 172.21.15.83:0/1322837979 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888800f040 con 0x7f887c006b80 2024-04-17T10:37:48.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.094+0000 7f88a041f640 1 -- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f887c006b80 msgr2=0x7f887c006f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:48.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.094+0000 7f88a041f640 1 --2- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f887c006b80 0x7f887c006f60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8898068460 tx=0x7f888801d9a0 comp rx=0 tx=0).stop 2024-04-17T10:37:48.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.094+0000 7f88a041f640 1 -- 172.21.15.83:0/1322837979 shutdown_connections 2024-04-17T10:37:48.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.094+0000 7f88a041f640 1 --2- 172.21.15.83:0/1322837979 >> v2:172.21.15.83:3300/0 conn(0x7f887c0074a0 0x7f887c007950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:48.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.094+0000 7f88a041f640 1 --2- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f887c006b80 0x7f887c006f60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:48.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.094+0000 7f88a041f640 1 --2- 172.21.15.83:0/1322837979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c003b60 0x7f887c006640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:48.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.094+0000 7f88a041f640 1 -- 172.21.15.83:0/1322837979 >> 172.21.15.83:0/1322837979 conn(0x7f889806abe0 msgr2=0x7f889806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:37:48.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.094+0000 7f88a041f640 1 -- 172.21.15.83:0/1322837979 shutdown_connections 2024-04-17T10:37:48.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.094+0000 7f88a041f640 1 -- 172.21.15.83:0/1322837979 wait complete. 2024-04-17T10:37:48.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.098+0000 7f88a041f640 1 Processor -- start 2024-04-17T10:37:48.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.098+0000 7f88a041f640 1 -- start start 2024-04-17T10:37:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.098+0000 7f88a041f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f887c003b60 0x7f8898150510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.098+0000 7f88a041f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f887c006b80 0x7f8898152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.098+0000 7f88a041f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c0074a0 0x7f88981533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.098+0000 7f88a041f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889815dfc0 con 0x7f887c006b80 2024-04-17T10:37:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.098+0000 7f88a041f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88980cdb40 con 0x7f887c003b60 2024-04-17T10:37:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.098+0000 7f88a041f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889815e140 con 0x7f887c0074a0 2024-04-17T10:37:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.098+0000 7f889d9ba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f887c006b80 0x7f8898152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.098+0000 7f889d9ba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f887c006b80 0x7f8898152e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35472/0 (socket says 172.21.15.83:35472) 2024-04-17T10:37:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.098+0000 7f889d9ba640 1 -- 172.21.15.83:0/1481233377 learned_addr learned my addr 172.21.15.83:0/1481233377 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:37:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.098+0000 7f889e9bc640 1 --2- 172.21.15.83:0/1481233377 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c0074a0 0x7f88981533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.098+0000 7f889e1bb640 1 --2- 172.21.15.83:0/1481233377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f887c003b60 0x7f8898150510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.098+0000 7f889d9ba640 1 -- 172.21.15.83:0/1481233377 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c0074a0 msgr2=0x7f88981533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.098+0000 7f889d9ba640 1 --2- 172.21.15.83:0/1481233377 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c0074a0 0x7f88981533a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:48.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.098+0000 7f889d9ba640 1 -- 172.21.15.83:0/1481233377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f887c003b60 msgr2=0x7f8898150510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:48.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.098+0000 7f889d9ba640 1 --2- 172.21.15.83:0/1481233377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f887c003b60 0x7f8898150510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:48.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.102+0000 7f889d9ba640 1 -- 172.21.15.83:0/1481233377 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f889815d170 con 0x7f887c006b80 2024-04-17T10:37:48.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.102+0000 7f889d9ba640 1 --2- 172.21.15.83:0/1481233377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f887c006b80 0x7f8898152e60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8888007f50 tx=0x7f888801b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:37:48.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.102+0000 7f888effd640 1 -- 172.21.15.83:0/1481233377 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88880b3070 con 0x7f887c006b80 2024-04-17T10:37:48.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.102+0000 7f888effd640 1 -- 172.21.15.83:0/1481233377 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88880b4040 con 0x7f887c006b80 2024-04-17T10:37:48.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.102+0000 7f888effd640 1 -- 172.21.15.83:0/1481233377 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88880ba690 con 0x7f887c006b80 2024-04-17T10:37:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.102+0000 7f88a041f640 1 -- 172.21.15.83:0/1481233377 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8898153a00 con 0x7f887c006b80 2024-04-17T10:37:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.102+0000 7f88a041f640 1 -- 172.21.15.83:0/1481233377 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f889809c160 con 0x7f887c006b80 2024-04-17T10:37:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.102+0000 7f888effd640 1 -- 172.21.15.83:0/1481233377 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f88880c2020 con 0x7f887c006b80 2024-04-17T10:37:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.102+0000 7f888effd640 1 -- 172.21.15.83:0/1481233377 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f88880ae7a0 con 0x7f887c006b80 2024-04-17T10:37:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.102+0000 7f88a041f640 1 -- 172.21.15.83:0/1481233377 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8898074f90 con 0x7f887c006b80 2024-04-17T10:37:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.102+0000 7f888effd640 1 -- 172.21.15.83:0/1481233377 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f88880084d0 con 0x7f887c006b80 2024-04-17T10:37:48.121 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-17T10:37:48.126+0000 7f4d3e02c180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-17T10:37:48.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.166+0000 7f88a041f640 1 -- 172.21.15.83:0/1481233377 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f8898066bc0 con 0x7f887c006b80 2024-04-17T10:37:48.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.170+0000 7f888effd640 1 -- 172.21.15.83:0/1481233377 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f88880bad50 con 0x7f887c006b80 2024-04-17T10:37:48.165 INFO:teuthology.orchestra.run.smithi083.stderr:adjusted tunables profile to default 2024-04-17T10:37:48.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.170+0000 7f88a041f640 1 -- 172.21.15.83:0/1481233377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f887c006b80 msgr2=0x7f8898152e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:48.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.170+0000 7f88a041f640 1 --2- 172.21.15.83:0/1481233377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f887c006b80 0x7f8898152e60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8888007f50 tx=0x7f888801b600 comp rx=0 tx=0).stop 2024-04-17T10:37:48.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.170+0000 7f88a041f640 1 -- 172.21.15.83:0/1481233377 shutdown_connections 2024-04-17T10:37:48.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.170+0000 7f88a041f640 1 --2- 172.21.15.83:0/1481233377 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f887c0074a0 0x7f88981533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:48.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.170+0000 7f88a041f640 1 --2- 172.21.15.83:0/1481233377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f887c006b80 0x7f8898152e60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:48.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.170+0000 7f88a041f640 1 --2- 172.21.15.83:0/1481233377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f887c003b60 0x7f8898150510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:48.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.170+0000 7f88a041f640 1 -- 172.21.15.83:0/1481233377 >> 172.21.15.83:0/1481233377 conn(0x7f889806abe0 msgr2=0x7f88980b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:37:48.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.170+0000 7f88a041f640 1 -- 172.21.15.83:0/1481233377 shutdown_connections 2024-04-17T10:37:48.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:48.170+0000 7f88a041f640 1 -- 172.21.15.83:0/1481233377 wait complete. 2024-04-17T10:37:48.185 INFO:tasks.ceph:check_enable_crimson: False 2024-04-17T10:37:48.185 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-17T10:37:48.185 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:37:48.185 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-17T10:37:48.197 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:37:48.197 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-17T10:37:48.253 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:37:48.253 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-17T10:37:48.264 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-04-17T10:37:48.265+0000 7f1e3a550180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-17T10:37:48.310 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:37:48.310 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-17T10:37:48.331 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-17T10:37:48.334+0000 7f4d3e02c180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-17T10:37:48.342 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:37:48.343 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-17T10:37:48.356 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:37:48.356 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-17T10:37:48.411 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:37:48.411 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-17T10:37:48.467 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:37:48.467 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-17T10:37:48.523 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 5641e679-fdd4-491b-9c50-3c7909264821 0 2024-04-17T10:37:48.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.685+0000 7f9e029c3640 1 Processor -- start 2024-04-17T10:37:48.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.685+0000 7f9e029c3640 1 -- start start 2024-04-17T10:37:48.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.689+0000 7f9e029c3640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9dfc15aa10 0x7f9dfc0729d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:48.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.689+0000 7f9e029c3640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dfc0cbca0 0x7f9dfc06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:48.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.689+0000 7f9e029c3640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9dfc06c130 0x7f9dfc06db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:48.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.689+0000 7f9e029c3640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dfc072f10 con 0x7f9dfc15aa10 2024-04-17T10:37:48.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.689+0000 7f9e029c3640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dfc0cc510 con 0x7f9dfc06c130 2024-04-17T10:37:48.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.689+0000 7f9e029c3640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc15e740 con 0x7f9dfc0cbca0 2024-04-17T10:37:48.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.689+0000 7f9dfb7fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dfc0cbca0 0x7f9dfc06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:48.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.689+0000 7f9e00f60640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9dfc06c130 0x7f9dfc06db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:48.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.689+0000 7f9dfb7fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dfc0cbca0 0x7f9dfc06bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:47944/0 (socket says 172.21.15.167:47944) 2024-04-17T10:37:48.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.689+0000 7f9dfb7fe640 1 -- 172.21.15.167:0/2786201131 learned_addr learned my addr 172.21.15.167:0/2786201131 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-04-17T10:37:48.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.689+0000 7f9dfbfff640 1 --2- 172.21.15.167:0/2786201131 >> v2:172.21.15.83:3300/0 conn(0x7f9dfc15aa10 0x7f9dfc0729d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:48.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.689+0000 7f9e00f60640 1 -- 172.21.15.167:0/2786201131 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dfc0cbca0 msgr2=0x7f9dfc06bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:37:48.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.689+0000 7f9e00f60640 1 --2- 172.21.15.167:0/2786201131 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dfc0cbca0 0x7f9dfc06bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:48.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.689+0000 7f9e00f60640 1 -- 172.21.15.167:0/2786201131 >> v2:172.21.15.83:3300/0 conn(0x7f9dfc15aa10 msgr2=0x7f9dfc0729d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:37:48.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.689+0000 7f9e00f60640 1 --2- 172.21.15.167:0/2786201131 >> v2:172.21.15.83:3300/0 conn(0x7f9dfc15aa10 0x7f9dfc0729d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:48.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.689+0000 7f9e00f60640 1 -- 172.21.15.167:0/2786201131 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dfc15ea20 con 0x7f9dfc06c130 2024-04-17T10:37:48.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.689+0000 7f9dfb7fe640 1 --2- 172.21.15.167:0/2786201131 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dfc0cbca0 0x7f9dfc06bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:37:48.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.689+0000 7f9dfbfff640 1 --2- 172.21.15.167:0/2786201131 >> v2:172.21.15.83:3300/0 conn(0x7f9dfc15aa10 0x7f9dfc0729d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:37:48.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.689+0000 7f9e00f60640 1 --2- 172.21.15.167:0/2786201131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dfc06c130 0x7f9dfc06db70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc070b80 tx=0x7f9dec01c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6508e74dc2f134c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:37:48.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.689+0000 7f9dfaffd640 1 -- 172.21.15.167:0/2786201131 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec0b0070 con 0x7f9dfc06c130 2024-04-17T10:37:48.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.689+0000 7f9dfaffd640 1 -- 172.21.15.167:0/2786201131 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dec0ab600 con 0x7f9dfc06c130 2024-04-17T10:37:48.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.689+0000 7f9dfaffd640 1 -- 172.21.15.167:0/2786201131 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec0ab9d0 con 0x7f9dfc06c130 2024-04-17T10:37:48.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.689+0000 7f9e029c3640 1 -- 172.21.15.167:0/2786201131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dfc06c130 msgr2=0x7f9dfc06db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:48.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.689+0000 7f9e029c3640 1 --2- 172.21.15.167:0/2786201131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dfc06c130 0x7f9dfc06db70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc070b80 tx=0x7f9dec01c3f0 comp rx=0 tx=0).stop 2024-04-17T10:37:48.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.689+0000 7f9e029c3640 1 -- 172.21.15.167:0/2786201131 shutdown_connections 2024-04-17T10:37:48.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.689+0000 7f9e029c3640 1 --2- 172.21.15.167:0/2786201131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dfc06c130 0x7f9dfc06db70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:48.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.689+0000 7f9e029c3640 1 --2- 172.21.15.167:0/2786201131 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dfc0cbca0 0x7f9dfc06bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:48.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.689+0000 7f9e029c3640 1 --2- 172.21.15.167:0/2786201131 >> v2:172.21.15.83:3300/0 conn(0x7f9dfc15aa10 0x7f9dfc0729d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:48.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.689+0000 7f9e029c3640 1 -- 172.21.15.167:0/2786201131 >> 172.21.15.167:0/2786201131 conn(0x7f9dfc0cc990 msgr2=0x7f9dfc0ccda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:37:48.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.689+0000 7f9e029c3640 1 -- 172.21.15.167:0/2786201131 shutdown_connections 2024-04-17T10:37:48.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.689+0000 7f9e029c3640 1 -- 172.21.15.167:0/2786201131 wait complete. 2024-04-17T10:37:48.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.693+0000 7f9e029c3640 1 Processor -- start 2024-04-17T10:37:48.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.693+0000 7f9e029c3640 1 -- start start 2024-04-17T10:37:48.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.693+0000 7f9e029c3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dfc06c130 0x7f9dfc0b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:48.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.693+0000 7f9e029c3640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dfc0cbca0 0x7f9dfc1f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:48.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.693+0000 7f9e029c3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dfc15aa10 0x7f9dfc1f1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:48.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.693+0000 7f9e029c3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc15af70 con 0x7f9dfc15aa10 2024-04-17T10:37:48.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.693+0000 7f9e029c3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc15adf0 con 0x7f9dfc06c130 2024-04-17T10:37:48.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.693+0000 7f9e029c3640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc0ca660 con 0x7f9dfc0cbca0 2024-04-17T10:37:48.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.693+0000 7f9dfbfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dfc06c130 0x7f9dfc0b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:48.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.693+0000 7f9dfb7fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dfc0cbca0 0x7f9dfc1f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:48.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.693+0000 7f9dfbfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dfc06c130 0x7f9dfc0b04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:36614/0 (socket says 172.21.15.167:36614) 2024-04-17T10:37:48.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.693+0000 7f9dfbfff640 1 -- 172.21.15.167:0/3674229580 learned_addr learned my addr 172.21.15.167:0/3674229580 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-04-17T10:37:48.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.693+0000 7f9dfbfff640 1 -- 172.21.15.167:0/3674229580 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dfc0cbca0 msgr2=0x7f9dfc1f5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:48.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.693+0000 7f9dfbfff640 1 --2- 172.21.15.167:0/3674229580 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dfc0cbca0 0x7f9dfc1f5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:48.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.693+0000 7f9dfbfff640 1 -- 172.21.15.167:0/3674229580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dfc15aa10 msgr2=0x7f9dfc1f1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:48.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.693+0000 7f9dfbfff640 1 --2- 172.21.15.167:0/3674229580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dfc15aa10 0x7f9dfc1f1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:48.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.693+0000 7f9dfbfff640 1 -- 172.21.15.167:0/3674229580 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dfc1f6240 con 0x7f9dfc06c130 2024-04-17T10:37:48.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.693+0000 7f9dfb7fe640 1 --2- 172.21.15.167:0/3674229580 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dfc0cbca0 0x7f9dfc1f5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:37:48.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.693+0000 7f9dfbfff640 1 --2- 172.21.15.167:0/3674229580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dfc06c130 0x7f9dfc0b04f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9df400ce00 tx=0x7f9df400a650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:37:48.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.693+0000 7f9df8ff9640 1 -- 172.21.15.167:0/3674229580 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df400f890 con 0x7f9dfc06c130 2024-04-17T10:37:48.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.693+0000 7f9df8ff9640 1 -- 172.21.15.167:0/3674229580 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df400fa30 con 0x7f9dfc06c130 2024-04-17T10:37:48.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.693+0000 7f9e029c3640 1 -- 172.21.15.167:0/3674229580 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dfc1f6520 con 0x7f9dfc06c130 2024-04-17T10:37:48.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.693+0000 7f9df8ff9640 1 -- 172.21.15.167:0/3674229580 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df400fe40 con 0x7f9dfc06c130 2024-04-17T10:37:48.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.693+0000 7f9e029c3640 1 -- 172.21.15.167:0/3674229580 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dfc14d8f0 con 0x7f9dfc06c130 2024-04-17T10:37:48.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.693+0000 7f9df8ff9640 1 -- 172.21.15.167:0/3674229580 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f9df4018400 con 0x7f9dfc06c130 2024-04-17T10:37:48.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.693+0000 7f9df8ff9640 1 -- 172.21.15.167:0/3674229580 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9df40218f0 con 0x7f9dfc06c130 2024-04-17T10:37:48.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.693+0000 7f9e029c3640 1 -- 172.21.15.167:0/3674229580 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dfc074f90 con 0x7f9dfc06c130 2024-04-17T10:37:48.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.705+0000 7f9df8ff9640 1 -- 172.21.15.167:0/3674229580 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9df4026020 con 0x7f9dfc06c130 2024-04-17T10:37:48.709 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-04-17T10:37:48.709+0000 7f1e3a550180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-17T10:37:48.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.773+0000 7f9e029c3640 1 -- 172.21.15.167:0/3674229580 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5641e679-fdd4-491b-9c50-3c7909264821", "id": 0} v 0) v1 -- 0x7f9dfc066ae0 con 0x7f9dfc06c130 2024-04-17T10:37:48.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.785+0000 7f9df8ff9640 1 -- 172.21.15.167:0/3674229580 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5641e679-fdd4-491b-9c50-3c7909264821", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9df4014030 con 0x7f9dfc06c130 2024-04-17T10:37:48.785 INFO:teuthology.orchestra.run.smithi167.stdout:0 2024-04-17T10:37:48.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.785+0000 7f9e029c3640 1 -- 172.21.15.167:0/3674229580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dfc06c130 msgr2=0x7f9dfc0b04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:48.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.785+0000 7f9e029c3640 1 --2- 172.21.15.167:0/3674229580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dfc06c130 0x7f9dfc0b04f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9df400ce00 tx=0x7f9df400a650 comp rx=0 tx=0).stop 2024-04-17T10:37:48.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.785+0000 7f9e029c3640 1 -- 172.21.15.167:0/3674229580 shutdown_connections 2024-04-17T10:37:48.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.785+0000 7f9e029c3640 1 --2- 172.21.15.167:0/3674229580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dfc15aa10 0x7f9dfc1f1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:48.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.785+0000 7f9e029c3640 1 --2- 172.21.15.167:0/3674229580 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dfc0cbca0 0x7f9dfc1f5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:48.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.785+0000 7f9e029c3640 1 --2- 172.21.15.167:0/3674229580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dfc06c130 0x7f9dfc0b04f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:48.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.785+0000 7f9e029c3640 1 -- 172.21.15.167:0/3674229580 >> 172.21.15.167:0/3674229580 conn(0x7f9dfc0cc990 msgr2=0x7f9dfc0b0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:37:48.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.785+0000 7f9e029c3640 1 -- 172.21.15.167:0/3674229580 shutdown_connections 2024-04-17T10:37:48.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.785+0000 7f9e029c3640 1 -- 172.21.15.167:0/3674229580 wait complete. 2024-04-17T10:37:48.801 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 37721235-ee08-4822-9539-9f65a28dc6a3 1 2024-04-17T10:37:48.894 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-17T10:37:48.898+0000 7f4d3e02c180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-17T10:37:48.906 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-04-17T10:37:48.905+0000 7f1e3a550180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-17T10:37:48.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.913+0000 7ff2ad874640 1 Processor -- start 2024-04-17T10:37:48.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.913+0000 7ff2ad874640 1 -- start start 2024-04-17T10:37:48.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.913+0000 7ff2ad874640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2a80afde0 0x7ff2a815ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:48.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.913+0000 7ff2ad874640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff2a815ae20 0x7ff2a815b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:48.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.913+0000 7ff2ad874640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a815e590 con 0x7ff2a80b4200 2024-04-17T10:37:48.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.913+0000 7ff2ad874640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a80b01c0 con 0x7ff2a815ae20 2024-04-17T10:37:48.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.913+0000 7ff2ad874640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a815b8e0 con 0x7ff2a80afde0 2024-04-17T10:37:48.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.913+0000 7ff29ffff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff2a815ae20 0x7ff2a815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:48.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.913+0000 7ff2a6ffd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2a80afde0 0x7ff2a815ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:48.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.917+0000 7ff29ffff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff2a815ae20 0x7ff2a815b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:36620/0 (socket says 172.21.15.167:36620) 2024-04-17T10:37:48.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.917+0000 7ff29ffff640 1 -- 172.21.15.167:0/2924454640 learned_addr learned my addr 172.21.15.167:0/2924454640 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-04-17T10:37:48.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.917+0000 7ff29ffff640 1 -- 172.21.15.167:0/2924454640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2a80afde0 msgr2=0x7ff2a815ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:37:48.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.917+0000 7ff29ffff640 1 --2- 172.21.15.167:0/2924454640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2a80afde0 0x7ff2a815ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:48.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.917+0000 7ff29ffff640 1 -- 172.21.15.167:0/2924454640 >> v1:172.21.15.83:6789/0 conn(0x7ff2a80b4200 legacy=0x7ff2a815bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:37:48.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.917+0000 7ff29ffff640 1 -- 172.21.15.167:0/2924454640 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a815bbc0 con 0x7ff2a815ae20 2024-04-17T10:37:48.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.917+0000 7ff29ffff640 1 --2- 172.21.15.167:0/2924454640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2a815ae20 0x7ff2a815b280 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff290009870 tx=0x7ff29001c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6420dae7c1b0815 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:37:48.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.917+0000 7ff2a67fc640 1 -- 172.21.15.167:0/2924454640 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2900aa660 con 0x7ff2a815ae20 2024-04-17T10:37:48.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.917+0000 7ff2a67fc640 1 -- 172.21.15.167:0/2924454640 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2900af070 con 0x7ff2a815ae20 2024-04-17T10:37:48.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.917+0000 7ff2a67fc640 1 -- 172.21.15.167:0/2924454640 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2900aaa20 con 0x7ff2a815ae20 2024-04-17T10:37:48.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.917+0000 7ff2ad874640 1 -- 172.21.15.167:0/2924454640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2a815ae20 msgr2=0x7ff2a815b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:48.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.917+0000 7ff2ad874640 1 --2- 172.21.15.167:0/2924454640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2a815ae20 0x7ff2a815b280 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff290009870 tx=0x7ff29001c520 comp rx=0 tx=0).stop 2024-04-17T10:37:48.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.917+0000 7ff2ad874640 1 -- 172.21.15.167:0/2924454640 shutdown_connections 2024-04-17T10:37:48.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.917+0000 7ff2ad874640 1 --2- 172.21.15.167:0/2924454640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2a815ae20 0x7ff2a815b280 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:48.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.917+0000 7ff2ad874640 1 --2- 172.21.15.167:0/2924454640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2a80afde0 0x7ff2a815ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:48.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.917+0000 7ff2ad874640 1 -- 172.21.15.167:0/2924454640 >> 172.21.15.167:0/2924454640 conn(0x7ff2a806bcf0 msgr2=0x7ff2a806c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:37:48.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.917+0000 7ff2ad874640 1 -- 172.21.15.167:0/2924454640 shutdown_connections 2024-04-17T10:37:48.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.917+0000 7ff2ad874640 1 -- 172.21.15.167:0/2924454640 wait complete. 2024-04-17T10:37:48.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.921+0000 7ff2ad874640 1 Processor -- start 2024-04-17T10:37:48.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.921+0000 7ff2ad874640 1 -- start start 2024-04-17T10:37:48.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.921+0000 7ff2ad874640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2a80afde0 0x7ff2a8152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:48.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.921+0000 7ff2ad874640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2a80b4200 0x7ff2a81f6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:48.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.921+0000 7ff2ad874640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2a815ae20 0x7ff2a806abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:48.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.921+0000 7ff2ad874640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a8071660 con 0x7ff2a80b4200 2024-04-17T10:37:48.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.921+0000 7ff2ad874640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a815f2b0 con 0x7ff2a80afde0 2024-04-17T10:37:48.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.921+0000 7ff2ad874640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a80717e0 con 0x7ff2a815ae20 2024-04-17T10:37:48.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.921+0000 7ff2a77fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2a815ae20 0x7ff2a806abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:48.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.921+0000 7ff2a6ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2a80afde0 0x7ff2a8152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:48.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.921+0000 7ff2a77fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2a815ae20 0x7ff2a806abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:47974/0 (socket says 172.21.15.167:47974) 2024-04-17T10:37:48.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.921+0000 7ff2a6ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2a80afde0 0x7ff2a8152f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:36628/0 (socket says 172.21.15.167:36628) 2024-04-17T10:37:48.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.921+0000 7ff2a77fe640 1 -- 172.21.15.167:0/1029801344 learned_addr learned my addr 172.21.15.167:0/1029801344 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-04-17T10:37:48.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.921+0000 7ff29ffff640 1 --2- 172.21.15.167:0/1029801344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2a80b4200 0x7ff2a81f6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:48.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.921+0000 7ff2a77fe640 1 -- 172.21.15.167:0/1029801344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2a80afde0 msgr2=0x7ff2a8152f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:48.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.921+0000 7ff2a77fe640 1 --2- 172.21.15.167:0/1029801344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2a80afde0 0x7ff2a8152f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:48.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.921+0000 7ff2a77fe640 1 -- 172.21.15.167:0/1029801344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2a80b4200 msgr2=0x7ff2a81f6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:48.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.921+0000 7ff2a77fe640 1 --2- 172.21.15.167:0/1029801344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2a80b4200 0x7ff2a81f6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:48.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.921+0000 7ff2a77fe640 1 -- 172.21.15.167:0/1029801344 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a80ae6c0 con 0x7ff2a815ae20 2024-04-17T10:37:48.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.921+0000 7ff2a6ffd640 1 --2- 172.21.15.167:0/1029801344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2a80afde0 0x7ff2a8152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:37:48.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.921+0000 7ff2a77fe640 1 --2- 172.21.15.167:0/1029801344 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2a815ae20 0x7ff2a806abe0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000b9a0 tx=0x7ff2a000be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:37:48.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.921+0000 7ff2a4ff9640 1 -- 172.21.15.167:0/1029801344 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0019070 con 0x7ff2a815ae20 2024-04-17T10:37:48.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.921+0000 7ff2a4ff9640 1 -- 172.21.15.167:0/1029801344 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a000cba0 con 0x7ff2a815ae20 2024-04-17T10:37:48.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.921+0000 7ff2ad874640 1 -- 172.21.15.167:0/1029801344 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a80ae9a0 con 0x7ff2a815ae20 2024-04-17T10:37:48.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.921+0000 7ff2a4ff9640 1 -- 172.21.15.167:0/1029801344 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0014440 con 0x7ff2a815ae20 2024-04-17T10:37:48.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.921+0000 7ff2ad874640 1 -- 172.21.15.167:0/1029801344 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a80ced20 con 0x7ff2a815ae20 2024-04-17T10:37:48.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.921+0000 7ff2a4ff9640 1 -- 172.21.15.167:0/1029801344 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7ff2a001c070 con 0x7ff2a815ae20 2024-04-17T10:37:48.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.921+0000 7ff2a4ff9640 1 -- 172.21.15.167:0/1029801344 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7ff2a00149b0 con 0x7ff2a815ae20 2024-04-17T10:37:48.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.921+0000 7ff2ad874640 1 -- 172.21.15.167:0/1029801344 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a8074f90 con 0x7ff2a815ae20 2024-04-17T10:37:48.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.925+0000 7ff2a4ff9640 1 -- 172.21.15.167:0/1029801344 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7ff2a0003ac0 con 0x7ff2a815ae20 2024-04-17T10:37:48.948 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-17T10:37:48.950+0000 7f4d3e02c180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-17T10:37:48.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.989+0000 7ff2ad874640 1 -- 172.21.15.167:0/1029801344 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd new", "uuid": "37721235-ee08-4822-9539-9f65a28dc6a3", "id": 1} v 0) v1 -- 0x7ff2a80b19f0 con 0x7ff2a815ae20 2024-04-17T10:37:48.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:48.997+0000 7ff2a4ff9640 1 -- 172.21.15.167:0/1029801344 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "37721235-ee08-4822-9539-9f65a28dc6a3", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff2a0003d70 con 0x7ff2a815ae20 2024-04-17T10:37:48.999 INFO:teuthology.orchestra.run.smithi167.stdout:1 2024-04-17T10:37:49.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.001+0000 7ff2ad874640 1 -- 172.21.15.167:0/1029801344 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2a815ae20 msgr2=0x7ff2a806abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:49.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.001+0000 7ff2ad874640 1 --2- 172.21.15.167:0/1029801344 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2a815ae20 0x7ff2a806abe0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000b9a0 tx=0x7ff2a000be60 comp rx=0 tx=0).stop 2024-04-17T10:37:49.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.001+0000 7ff2ad874640 1 -- 172.21.15.167:0/1029801344 shutdown_connections 2024-04-17T10:37:49.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.001+0000 7ff2ad874640 1 --2- 172.21.15.167:0/1029801344 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2a815ae20 0x7ff2a806abe0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.001+0000 7ff2ad874640 1 --2- 172.21.15.167:0/1029801344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2a80b4200 0x7ff2a81f6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.001+0000 7ff2ad874640 1 --2- 172.21.15.167:0/1029801344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2a80afde0 0x7ff2a8152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.001+0000 7ff2ad874640 1 -- 172.21.15.167:0/1029801344 >> 172.21.15.167:0/1029801344 conn(0x7ff2a806bcf0 msgr2=0x7ff2a806c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:37:49.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.001+0000 7ff2ad874640 1 -- 172.21.15.167:0/1029801344 shutdown_connections 2024-04-17T10:37:49.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.001+0000 7ff2ad874640 1 -- 172.21.15.167:0/1029801344 wait complete. 2024-04-17T10:37:49.002 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-17T10:37:49.002+0000 7f4d3e02c180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-17T10:37:49.021 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new d4c58363-8146-4b9d-8331-feb0dd150dd7 2 2024-04-17T10:37:49.077 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-17T10:37:49.078+0000 7f4d3e02c180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-17T10:37:49.128 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-17T10:37:49.130+0000 7f4d3e02c180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-17T10:37:49.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.137+0000 7fd539262640 1 Processor -- start 2024-04-17T10:37:49.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.137+0000 7fd539262640 1 -- start start 2024-04-17T10:37:49.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.137+0000 7fd539262640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd53415ae20 0x7fd53415b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:49.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.137+0000 7fd539262640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd5340b4200 0x7fd5340b4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:49.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.137+0000 7fd539262640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd53415e590 con 0x7fd5340afde0 2024-04-17T10:37:49.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.137+0000 7fd539262640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5340b01c0 con 0x7fd5340b4200 2024-04-17T10:37:49.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.137+0000 7fd539262640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd53415b8e0 con 0x7fd53415ae20 2024-04-17T10:37:49.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.137+0000 7fd5337fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd5340b4200 0x7fd5340b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:49.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.137+0000 7fd52bfff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd53415ae20 0x7fd53415b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:49.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.137+0000 7fd5337fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd5340b4200 0x7fd5340b4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:36640/0 (socket says 172.21.15.167:36640) 2024-04-17T10:37:49.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.137+0000 7fd5337fe640 1 -- 172.21.15.167:0/3904031864 learned_addr learned my addr 172.21.15.167:0/3904031864 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-04-17T10:37:49.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.137+0000 7fd52bfff640 1 -- 172.21.15.167:0/3904031864 >> v2:172.21.15.167:3300/0 conn(0x7fd5340b4200 msgr2=0x7fd5340b4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:37:49.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.137+0000 7fd52bfff640 1 --2- 172.21.15.167:0/3904031864 >> v2:172.21.15.167:3300/0 conn(0x7fd5340b4200 0x7fd5340b4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.137+0000 7fd52bfff640 1 -- 172.21.15.167:0/3904031864 >> v1:172.21.15.83:6789/0 conn(0x7fd5340afde0 legacy=0x7fd53415ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:49.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.137+0000 7fd52bfff640 1 -- 172.21.15.167:0/3904031864 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53415bbc0 con 0x7fd53415ae20 2024-04-17T10:37:49.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.137+0000 7fd5337fe640 1 --2- 172.21.15.167:0/3904031864 >> v2:172.21.15.167:3300/0 conn(0x7fd5340b4200 0x7fd5340b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:37:49.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.141+0000 7fd52bfff640 1 --2- 172.21.15.167:0/3904031864 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd53415ae20 0x7fd53415b280 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd51c0097c0 tx=0x7fd51c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff26635265efc534 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:37:49.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.141+0000 7fd5327fc640 1 -- 172.21.15.167:0/3904031864 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c0b1070 con 0x7fd53415ae20 2024-04-17T10:37:49.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.141+0000 7fd5327fc640 1 -- 172.21.15.167:0/3904031864 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51c0ac410 con 0x7fd53415ae20 2024-04-17T10:37:49.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.141+0000 7fd5327fc640 1 -- 172.21.15.167:0/3904031864 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c0ac7e0 con 0x7fd53415ae20 2024-04-17T10:37:49.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.141+0000 7fd539262640 1 -- 172.21.15.167:0/3904031864 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd53415ae20 msgr2=0x7fd53415b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:49.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.141+0000 7fd539262640 1 --2- 172.21.15.167:0/3904031864 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd53415ae20 0x7fd53415b280 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd51c0097c0 tx=0x7fd51c01c4a0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.141+0000 7fd539262640 1 -- 172.21.15.167:0/3904031864 shutdown_connections 2024-04-17T10:37:49.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.141+0000 7fd539262640 1 --2- 172.21.15.167:0/3904031864 >> v2:172.21.15.167:3300/0 conn(0x7fd5340b4200 0x7fd5340b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.141+0000 7fd539262640 1 --2- 172.21.15.167:0/3904031864 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd53415ae20 0x7fd53415b280 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.141+0000 7fd539262640 1 -- 172.21.15.167:0/3904031864 >> 172.21.15.167:0/3904031864 conn(0x7fd53406bcf0 msgr2=0x7fd53406c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:37:49.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.141+0000 7fd539262640 1 -- 172.21.15.167:0/3904031864 shutdown_connections 2024-04-17T10:37:49.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.141+0000 7fd539262640 1 -- 172.21.15.167:0/3904031864 wait complete. 2024-04-17T10:37:49.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.141+0000 7fd539262640 1 Processor -- start 2024-04-17T10:37:49.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.141+0000 7fd539262640 1 -- start start 2024-04-17T10:37:49.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.145+0000 7fd539262640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5340afde0 0x7fd5341450c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:49.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.145+0000 7fd539262640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5340b4200 0x7fd5340ab280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:49.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.145+0000 7fd539262640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd53415ae20 0x7fd5340ab7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:49.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.145+0000 7fd539262640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53415ca00 con 0x7fd5340afde0 2024-04-17T10:37:49.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.145+0000 7fd539262640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5341677b0 con 0x7fd5340b4200 2024-04-17T10:37:49.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.145+0000 7fd539262640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd534145600 con 0x7fd53415ae20 2024-04-17T10:37:49.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.145+0000 7fd52bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5340b4200 0x7fd5340ab280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:49.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.145+0000 7fd5337fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd53415ae20 0x7fd5340ab7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:49.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.145+0000 7fd52bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5340b4200 0x7fd5340ab280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:36646/0 (socket says 172.21.15.167:36646) 2024-04-17T10:37:49.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.145+0000 7fd5337fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd53415ae20 0x7fd5340ab7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:47996/0 (socket says 172.21.15.167:47996) 2024-04-17T10:37:49.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.145+0000 7fd52bfff640 1 -- 172.21.15.167:0/2937136992 learned_addr learned my addr 172.21.15.167:0/2937136992 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-04-17T10:37:49.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.145+0000 7fd532ffd640 1 --2- 172.21.15.167:0/2937136992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5340afde0 0x7fd5341450c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:49.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.145+0000 7fd52bfff640 1 -- 172.21.15.167:0/2937136992 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd53415ae20 msgr2=0x7fd5340ab7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:49.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.145+0000 7fd52bfff640 1 --2- 172.21.15.167:0/2937136992 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd53415ae20 0x7fd5340ab7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.145+0000 7fd52bfff640 1 -- 172.21.15.167:0/2937136992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5340afde0 msgr2=0x7fd5341450c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:49.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.145+0000 7fd52bfff640 1 --2- 172.21.15.167:0/2937136992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5340afde0 0x7fd5341450c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.145+0000 7fd52bfff640 1 -- 172.21.15.167:0/2937136992 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5340a97f0 con 0x7fd5340b4200 2024-04-17T10:37:49.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.145+0000 7fd532ffd640 1 --2- 172.21.15.167:0/2937136992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5340afde0 0x7fd5341450c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:37:49.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.145+0000 7fd5337fe640 1 --2- 172.21.15.167:0/2937136992 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd53415ae20 0x7fd5340ab7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:37:49.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.145+0000 7fd52bfff640 1 --2- 172.21.15.167:0/2937136992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5340b4200 0x7fd5340ab280 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd51c0af040 tx=0x7fd51c019240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:37:49.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.145+0000 7fd52b7fe640 1 -- 172.21.15.167:0/2937136992 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c0b1040 con 0x7fd5340b4200 2024-04-17T10:37:49.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.145+0000 7fd52b7fe640 1 -- 172.21.15.167:0/2937136992 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51c0bc070 con 0x7fd5340b4200 2024-04-17T10:37:49.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.145+0000 7fd539262640 1 -- 172.21.15.167:0/2937136992 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5340a9ad0 con 0x7fd5340b4200 2024-04-17T10:37:49.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.145+0000 7fd52b7fe640 1 -- 172.21.15.167:0/2937136992 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c0b5410 con 0x7fd5340b4200 2024-04-17T10:37:49.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.145+0000 7fd539262640 1 -- 172.21.15.167:0/2937136992 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5340aa090 con 0x7fd5340b4200 2024-04-17T10:37:49.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.145+0000 7fd52b7fe640 1 -- 172.21.15.167:0/2937136992 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fd51c0b55b0 con 0x7fd5340b4200 2024-04-17T10:37:49.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.145+0000 7fd52b7fe640 1 -- 172.21.15.167:0/2937136992 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fd51c0c1970 con 0x7fd5340b4200 2024-04-17T10:37:49.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.145+0000 7fd539262640 1 -- 172.21.15.167:0/2937136992 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5340acf80 con 0x7fd5340b4200 2024-04-17T10:37:49.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.149+0000 7fd52b7fe640 1 -- 172.21.15.167:0/2937136992 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fd5340acf80 con 0x7fd5340b4200 2024-04-17T10:37:49.204 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-17T10:37:49.206+0000 7f4d3e02c180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-17T10:37:49.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.213+0000 7fd539262640 1 -- 172.21.15.167:0/2937136992 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d4c58363-8146-4b9d-8331-feb0dd150dd7", "id": 2} v 0) v1 -- 0x7fd534074f90 con 0x7fd5340b4200 2024-04-17T10:37:49.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.225+0000 7fd52b7fe640 1 -- 172.21.15.167:0/2937136992 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d4c58363-8146-4b9d-8331-feb0dd150dd7", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd51c016090 con 0x7fd5340b4200 2024-04-17T10:37:49.225 INFO:teuthology.orchestra.run.smithi167.stdout:2 2024-04-17T10:37:49.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.225+0000 7fd539262640 1 -- 172.21.15.167:0/2937136992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5340b4200 msgr2=0x7fd5340ab280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:49.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.225+0000 7fd539262640 1 --2- 172.21.15.167:0/2937136992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5340b4200 0x7fd5340ab280 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd51c0af040 tx=0x7fd51c019240 comp rx=0 tx=0).stop 2024-04-17T10:37:49.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.225+0000 7fd539262640 1 -- 172.21.15.167:0/2937136992 shutdown_connections 2024-04-17T10:37:49.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.225+0000 7fd539262640 1 --2- 172.21.15.167:0/2937136992 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd53415ae20 0x7fd5340ab7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.225+0000 7fd539262640 1 --2- 172.21.15.167:0/2937136992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5340b4200 0x7fd5340ab280 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.225+0000 7fd539262640 1 --2- 172.21.15.167:0/2937136992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5340afde0 0x7fd5341450c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.225+0000 7fd539262640 1 -- 172.21.15.167:0/2937136992 >> 172.21.15.167:0/2937136992 conn(0x7fd53406bcf0 msgr2=0x7fd5340b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:37:49.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.225+0000 7fd539262640 1 -- 172.21.15.167:0/2937136992 shutdown_connections 2024-04-17T10:37:49.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.225+0000 7fd539262640 1 -- 172.21.15.167:0/2937136992 wait complete. 2024-04-17T10:37:49.243 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 74b46730-8ac3-4eb1-b573-45dd02be747d 3 2024-04-17T10:37:49.256 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-17T10:37:49.258+0000 7f4d3e02c180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-17T10:37:49.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.357+0000 7f7392cbf640 1 Processor -- start 2024-04-17T10:37:49.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.357+0000 7f7392cbf640 1 -- start start 2024-04-17T10:37:49.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.357+0000 7f7392cbf640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f738c15be00 0x7f738c0d0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:49.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.357+0000 7f7392cbf640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f738c0d05b0 0x7f738c15d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:49.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.357+0000 7f7392cbf640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f738c0b18a0 con 0x7f738c0d19c0 2024-04-17T10:37:49.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.357+0000 7f7392cbf640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f738c0b1b80 con 0x7f738c15be00 2024-04-17T10:37:49.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.357+0000 7f7392cbf640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f738c0b1e60 con 0x7f738c0d05b0 2024-04-17T10:37:49.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.357+0000 7f738bfff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f738c15be00 0x7f738c0d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:49.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.357+0000 7f739125c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f738c0d05b0 0x7f738c15d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:49.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.357+0000 7f7390a5b640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f738c0d19c0 0x7f738c0d1da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:36048/0 (socket says 172.21.15.167:36048) 2024-04-17T10:37:49.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.357+0000 7f7390a5b640 1 -- 172.21.15.167:0/283532808 learned_addr learned my addr 172.21.15.167:0/283532808 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-04-17T10:37:49.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.361+0000 7f739125c640 1 -- 172.21.15.167:0/283532808 >> v2:172.21.15.167:3300/0 conn(0x7f738c15be00 msgr2=0x7f738c0d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:37:49.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.361+0000 7f739125c640 1 --2- 172.21.15.167:0/283532808 >> v2:172.21.15.167:3300/0 conn(0x7f738c15be00 0x7f738c0d0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.361+0000 7f739125c640 1 -- 172.21.15.167:0/283532808 >> v1:172.21.15.167:6789/0 conn(0x7f738c0d19c0 legacy=0x7f738c0d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:49.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.361+0000 7f738b7fe640 1 -- 172.21.15.167:0/283532808 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4231097356 0 0) 0x7f738c0b18a0 con 0x7f738c0d19c0 2024-04-17T10:37:49.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.361+0000 7f739125c640 1 -- 172.21.15.167:0/283532808 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f738c0b2140 con 0x7f738c0d05b0 2024-04-17T10:37:49.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.361+0000 7f738bfff640 1 --2- 172.21.15.167:0/283532808 >> v2:172.21.15.167:3300/0 conn(0x7f738c15be00 0x7f738c0d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:37:49.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.361+0000 7f739125c640 1 --2- 172.21.15.167:0/283532808 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f738c0d05b0 0x7f738c15d4f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7384009d90 tx=0x7f738401c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e0aa077ff3227ee server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:37:49.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.361+0000 7f738b7fe640 1 -- 172.21.15.167:0/283532808 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73840b2070 con 0x7f738c0d05b0 2024-04-17T10:37:49.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.361+0000 7f738b7fe640 1 -- 172.21.15.167:0/283532808 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73840a58a0 con 0x7f738c0d05b0 2024-04-17T10:37:49.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.361+0000 7f738b7fe640 1 -- 172.21.15.167:0/283532808 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73840a5c70 con 0x7f738c0d05b0 2024-04-17T10:37:49.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.365+0000 7f7392cbf640 1 -- 172.21.15.167:0/283532808 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f738c0d05b0 msgr2=0x7f738c15d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:49.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.365+0000 7f7392cbf640 1 --2- 172.21.15.167:0/283532808 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f738c0d05b0 0x7f738c15d4f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7384009d90 tx=0x7f738401c8b0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.365+0000 7f7392cbf640 1 -- 172.21.15.167:0/283532808 shutdown_connections 2024-04-17T10:37:49.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.365+0000 7f7392cbf640 1 --2- 172.21.15.167:0/283532808 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f738c0d05b0 0x7f738c15d4f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.365+0000 7f7392cbf640 1 --2- 172.21.15.167:0/283532808 >> v2:172.21.15.167:3300/0 conn(0x7f738c15be00 0x7f738c0d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.365+0000 7f7392cbf640 1 -- 172.21.15.167:0/283532808 >> 172.21.15.167:0/283532808 conn(0x7f738c09a9d0 msgr2=0x7f738c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:37:49.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.365+0000 7f7392cbf640 1 -- 172.21.15.167:0/283532808 shutdown_connections 2024-04-17T10:37:49.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.365+0000 7f7392cbf640 1 -- 172.21.15.167:0/283532808 wait complete. 2024-04-17T10:37:49.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.365+0000 7f7392cbf640 1 Processor -- start 2024-04-17T10:37:49.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.365+0000 7f7392cbf640 1 -- start start 2024-04-17T10:37:49.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.365+0000 7f7392cbf640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f738c0d05b0 0x7f738c148fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:49.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.365+0000 7f7392cbf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c0d19c0 0x7f738c155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:49.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.365+0000 7f7392cbf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f738c1564b0 0x7f738c14d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:49.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.365+0000 7f7392cbf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738c168dc0 con 0x7f738c0d19c0 2024-04-17T10:37:49.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.365+0000 7f7392cbf640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738c168c40 con 0x7f738c1564b0 2024-04-17T10:37:49.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.365+0000 7f7392cbf640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f738c168f40 con 0x7f738c0d05b0 2024-04-17T10:37:49.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.365+0000 7f7390a5b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f738c0d05b0 0x7f738c148fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:49.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.365+0000 7f7390a5b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f738c0d05b0 0x7f738c148fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:48020/0 (socket says 172.21.15.167:48020) 2024-04-17T10:37:49.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.365+0000 7f7390a5b640 1 -- 172.21.15.167:0/1649998698 learned_addr learned my addr 172.21.15.167:0/1649998698 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-04-17T10:37:49.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.365+0000 7f739125c640 1 --2- 172.21.15.167:0/1649998698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f738c1564b0 0x7f738c14d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:49.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.365+0000 7f738bfff640 1 --2- 172.21.15.167:0/1649998698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c0d19c0 0x7f738c155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:49.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.365+0000 7f7390a5b640 1 -- 172.21.15.167:0/1649998698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f738c1564b0 msgr2=0x7f738c14d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:49.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.365+0000 7f7390a5b640 1 --2- 172.21.15.167:0/1649998698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f738c1564b0 0x7f738c14d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.365+0000 7f7390a5b640 1 -- 172.21.15.167:0/1649998698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c0d19c0 msgr2=0x7f738c155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:49.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.365+0000 7f7390a5b640 1 --2- 172.21.15.167:0/1649998698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c0d19c0 0x7f738c155f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.365+0000 7f7390a5b640 1 -- 172.21.15.167:0/1649998698 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f738c14e3a0 con 0x7f738c0d05b0 2024-04-17T10:37:49.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.365+0000 7f7390a5b640 1 --2- 172.21.15.167:0/1649998698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f738c0d05b0 0x7f738c148fb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f73800011f0 tx=0x7f738000c9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:37:49.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.365+0000 7f73897fa640 1 -- 172.21.15.167:0/1649998698 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7380018070 con 0x7f738c0d05b0 2024-04-17T10:37:49.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.365+0000 7f73897fa640 1 -- 172.21.15.167:0/1649998698 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73800074e0 con 0x7f738c0d05b0 2024-04-17T10:37:49.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.365+0000 7f7392cbf640 1 -- 172.21.15.167:0/1649998698 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f738c14e680 con 0x7f738c0d05b0 2024-04-17T10:37:49.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.365+0000 7f73897fa640 1 -- 172.21.15.167:0/1649998698 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7380004a10 con 0x7f738c0d05b0 2024-04-17T10:37:49.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.365+0000 7f7392cbf640 1 -- 172.21.15.167:0/1649998698 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f738c14ec40 con 0x7f738c0d05b0 2024-04-17T10:37:49.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.369+0000 7f73897fa640 1 -- 172.21.15.167:0/1649998698 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f738001b070 con 0x7f738c0d05b0 2024-04-17T10:37:49.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.369+0000 7f73897fa640 1 -- 172.21.15.167:0/1649998698 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f7380012890 con 0x7f738c0d05b0 2024-04-17T10:37:49.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.369+0000 7f7392cbf640 1 -- 172.21.15.167:0/1649998698 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f738c1568c0 con 0x7f738c0d05b0 2024-04-17T10:37:49.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.369+0000 7f73897fa640 1 -- 172.21.15.167:0/1649998698 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f738c1568c0 con 0x7f738c0d05b0 2024-04-17T10:37:49.406 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-17T10:37:49.410+0000 7f4d3e02c180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-17T10:37:49.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.437+0000 7f7392cbf640 1 -- 172.21.15.167:0/1649998698 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd new", "uuid": "74b46730-8ac3-4eb1-b573-45dd02be747d", "id": 3} v 0) v1 -- 0x7f738c074f90 con 0x7f738c0d05b0 2024-04-17T10:37:49.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.445+0000 7f73897fa640 1 -- 172.21.15.167:0/1649998698 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "74b46730-8ac3-4eb1-b573-45dd02be747d", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7380012dc0 con 0x7f738c0d05b0 2024-04-17T10:37:49.445 INFO:teuthology.orchestra.run.smithi167.stdout:3 2024-04-17T10:37:49.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.445+0000 7f7392cbf640 1 -- 172.21.15.167:0/1649998698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f738c0d05b0 msgr2=0x7f738c148fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:49.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.445+0000 7f7392cbf640 1 --2- 172.21.15.167:0/1649998698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f738c0d05b0 0x7f738c148fb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f73800011f0 tx=0x7f738000c9b0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.445+0000 7f7392cbf640 1 -- 172.21.15.167:0/1649998698 shutdown_connections 2024-04-17T10:37:49.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.445+0000 7f7392cbf640 1 --2- 172.21.15.167:0/1649998698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f738c1564b0 0x7f738c14d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.445+0000 7f7392cbf640 1 --2- 172.21.15.167:0/1649998698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c0d19c0 0x7f738c155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.445+0000 7f7392cbf640 1 --2- 172.21.15.167:0/1649998698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f738c0d05b0 0x7f738c148fb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.445+0000 7f7392cbf640 1 -- 172.21.15.167:0/1649998698 >> 172.21.15.167:0/1649998698 conn(0x7f738c09a9d0 msgr2=0x7f738c0b3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:37:49.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.445+0000 7f7392cbf640 1 -- 172.21.15.167:0/1649998698 shutdown_connections 2024-04-17T10:37:49.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.445+0000 7f7392cbf640 1 -- 172.21.15.167:0/1649998698 wait complete. 2024-04-17T10:37:49.461 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new d28dd508-25ed-4dc0-bff9-bcd2480906ff 4 2024-04-17T10:37:49.468 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-17T10:37:49.470+0000 7f4d3e02c180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-17T10:37:49.482 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-04-17T10:37:49.481+0000 7f1e3a550180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-17T10:37:49.538 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-04-17T10:37:49.537+0000 7f1e3a550180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-17T10:37:49.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.573+0000 7f9caad12640 1 Processor -- start 2024-04-17T10:37:49.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.573+0000 7f9caad12640 1 -- start start 2024-04-17T10:37:49.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.577+0000 7f9caad12640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9ca415dd20 0x7f9ca40b1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:49.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.577+0000 7f9caad12640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9ca406c2c0 0x7f9ca40ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:49.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.577+0000 7f9caad12640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ca40d1850 0x7f9ca40d1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:49.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.577+0000 7f9caad12640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ca40cec10 con 0x7f9ca406c2c0 2024-04-17T10:37:49.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.577+0000 7f9caad12640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ca40ceef0 con 0x7f9ca415dd20 2024-04-17T10:37:49.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.577+0000 7f9caad12640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca40cf1d0 con 0x7f9ca40d1850 2024-04-17T10:37:49.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.577+0000 7f9ca8aae640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9ca415dd20 0x7f9ca40b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:49.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.577+0000 7f9ca92af640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ca40d1850 0x7f9ca40d1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:49.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.577+0000 7f9ca8aae640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9ca415dd20 0x7f9ca40b1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:36668/0 (socket says 172.21.15.167:36668) 2024-04-17T10:37:49.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.577+0000 7f9ca8aae640 1 -- 172.21.15.167:0/4150100719 learned_addr learned my addr 172.21.15.167:0/4150100719 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-04-17T10:37:49.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.577+0000 7f9ca3fff640 1 --2- 172.21.15.167:0/4150100719 >> v2:172.21.15.83:3300/0 conn(0x7f9ca406c2c0 0x7f9ca40ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:49.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.577+0000 7f9ca8aae640 1 -- 172.21.15.167:0/4150100719 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ca40d1850 msgr2=0x7f9ca40d1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:37:49.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.577+0000 7f9ca8aae640 1 --2- 172.21.15.167:0/4150100719 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ca40d1850 0x7f9ca40d1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.577+0000 7f9ca8aae640 1 -- 172.21.15.167:0/4150100719 >> v2:172.21.15.83:3300/0 conn(0x7f9ca406c2c0 msgr2=0x7f9ca40ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:37:49.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.577+0000 7f9ca8aae640 1 --2- 172.21.15.167:0/4150100719 >> v2:172.21.15.83:3300/0 conn(0x7f9ca406c2c0 0x7f9ca40ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.577+0000 7f9ca8aae640 1 -- 172.21.15.167:0/4150100719 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca40cfe70 con 0x7f9ca415dd20 2024-04-17T10:37:49.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.577+0000 7f9ca8aae640 1 --2- 172.21.15.167:0/4150100719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ca415dd20 0x7f9ca40b1700 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40cf610 tx=0x7f9c9801c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d11e9356183effda server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:37:49.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.577+0000 7f9ca37fe640 1 -- 172.21.15.167:0/4150100719 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c980ab5f0 con 0x7f9ca415dd20 2024-04-17T10:37:49.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.577+0000 7f9ca37fe640 1 -- 172.21.15.167:0/4150100719 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c980b0070 con 0x7f9ca415dd20 2024-04-17T10:37:49.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.577+0000 7f9ca37fe640 1 -- 172.21.15.167:0/4150100719 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c980ab9b0 con 0x7f9ca415dd20 2024-04-17T10:37:49.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.577+0000 7f9caad12640 1 -- 172.21.15.167:0/4150100719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ca415dd20 msgr2=0x7f9ca40b1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:49.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.577+0000 7f9caad12640 1 --2- 172.21.15.167:0/4150100719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ca415dd20 0x7f9ca40b1700 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40cf610 tx=0x7f9c9801c520 comp rx=0 tx=0).stop 2024-04-17T10:37:49.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.577+0000 7f9caad12640 1 -- 172.21.15.167:0/4150100719 shutdown_connections 2024-04-17T10:37:49.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.577+0000 7f9caad12640 1 --2- 172.21.15.167:0/4150100719 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ca40d1850 0x7f9ca40d1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.577+0000 7f9caad12640 1 --2- 172.21.15.167:0/4150100719 >> v2:172.21.15.83:3300/0 conn(0x7f9ca406c2c0 0x7f9ca40ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.577+0000 7f9caad12640 1 --2- 172.21.15.167:0/4150100719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ca415dd20 0x7f9ca40b1700 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.577+0000 7f9caad12640 1 -- 172.21.15.167:0/4150100719 >> 172.21.15.167:0/4150100719 conn(0x7f9ca409a9d0 msgr2=0x7f9ca409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:37:49.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.577+0000 7f9caad12640 1 -- 172.21.15.167:0/4150100719 shutdown_connections 2024-04-17T10:37:49.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.581+0000 7f9caad12640 1 -- 172.21.15.167:0/4150100719 wait complete. 2024-04-17T10:37:49.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.581+0000 7f9caad12640 1 Processor -- start 2024-04-17T10:37:49.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.581+0000 7f9caad12640 1 -- start start 2024-04-17T10:37:49.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.581+0000 7f9caad12640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ca406c2c0 0x7f9ca415e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:49.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.581+0000 7f9caad12640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ca40d1850 0x7f9ca40b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:49.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.581+0000 7f9caad12640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ca415dd20 0x7f9ca41568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:49.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.581+0000 7f9caad12640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca415e280 con 0x7f9ca415dd20 2024-04-17T10:37:49.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.581+0000 7f9caad12640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca415e100 con 0x7f9ca40d1850 2024-04-17T10:37:49.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.581+0000 7f9caad12640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca40d0ba0 con 0x7f9ca406c2c0 2024-04-17T10:37:49.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.581+0000 7f9ca8aae640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ca406c2c0 0x7f9ca415e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:49.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.581+0000 7f9ca8aae640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ca406c2c0 0x7f9ca415e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:48040/0 (socket says 172.21.15.167:48040) 2024-04-17T10:37:49.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.581+0000 7f9ca8aae640 1 -- 172.21.15.167:0/1406640643 learned_addr learned my addr 172.21.15.167:0/1406640643 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-04-17T10:37:49.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.581+0000 7f9ca3fff640 1 --2- 172.21.15.167:0/1406640643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ca40d1850 0x7f9ca40b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:49.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.581+0000 7f9ca92af640 1 --2- 172.21.15.167:0/1406640643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ca415dd20 0x7f9ca41568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:49.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.581+0000 7f9ca8aae640 1 -- 172.21.15.167:0/1406640643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ca40d1850 msgr2=0x7f9ca40b51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:49.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.581+0000 7f9ca8aae640 1 --2- 172.21.15.167:0/1406640643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ca40d1850 0x7f9ca40b51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.581+0000 7f9ca8aae640 1 -- 172.21.15.167:0/1406640643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ca415dd20 msgr2=0x7f9ca41568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:49.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.581+0000 7f9ca8aae640 1 --2- 172.21.15.167:0/1406640643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ca415dd20 0x7f9ca41568d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.581+0000 7f9ca8aae640 1 -- 172.21.15.167:0/1406640643 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca40cd050 con 0x7f9ca406c2c0 2024-04-17T10:37:49.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.581+0000 7f9ca92af640 1 --2- 172.21.15.167:0/1406640643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ca415dd20 0x7f9ca41568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:37:49.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.581+0000 7f9ca8aae640 1 --2- 172.21.15.167:0/1406640643 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ca406c2c0 0x7f9ca415e490 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9c980ad040 tx=0x7f9c9801ad60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:37:49.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.581+0000 7f9ca17fa640 1 -- 172.21.15.167:0/1406640643 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c980be3f0 con 0x7f9ca406c2c0 2024-04-17T10:37:49.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.581+0000 7f9ca17fa640 1 -- 172.21.15.167:0/1406640643 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c980be590 con 0x7f9ca406c2c0 2024-04-17T10:37:49.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.581+0000 7f9caad12640 1 -- 172.21.15.167:0/1406640643 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca415ad20 con 0x7f9ca406c2c0 2024-04-17T10:37:49.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.581+0000 7f9ca17fa640 1 -- 172.21.15.167:0/1406640643 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c980be960 con 0x7f9ca406c2c0 2024-04-17T10:37:49.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.581+0000 7f9caad12640 1 -- 172.21.15.167:0/1406640643 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca415b2e0 con 0x7f9ca406c2c0 2024-04-17T10:37:49.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.581+0000 7f9ca17fa640 1 -- 172.21.15.167:0/1406640643 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f9c980b43f0 con 0x7f9ca406c2c0 2024-04-17T10:37:49.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.581+0000 7f9ca17fa640 1 -- 172.21.15.167:0/1406640643 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f9c980b3980 con 0x7f9ca406c2c0 2024-04-17T10:37:49.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.581+0000 7f9caad12640 1 -- 172.21.15.167:0/1406640643 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca4074f90 con 0x7f9ca406c2c0 2024-04-17T10:37:49.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.585+0000 7f9ca17fa640 1 -- 172.21.15.167:0/1406640643 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9c980c4060 con 0x7f9ca406c2c0 2024-04-17T10:37:49.593 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-04-17T10:37:49.593+0000 7f1e3a550180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-17T10:37:49.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.653+0000 7f9caad12640 1 -- 172.21.15.167:0/1406640643 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d28dd508-25ed-4dc0-bff9-bcd2480906ff", "id": 4} v 0) v1 -- 0x7f9ca406c0a0 con 0x7f9ca406c2c0 2024-04-17T10:37:49.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.657+0000 7f9ca17fa640 1 -- 172.21.15.167:0/1406640643 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d28dd508-25ed-4dc0-bff9-bcd2480906ff", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9c98016110 con 0x7f9ca406c2c0 2024-04-17T10:37:49.658 INFO:teuthology.orchestra.run.smithi167.stdout:4 2024-04-17T10:37:49.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.657+0000 7f9caad12640 1 -- 172.21.15.167:0/1406640643 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ca406c2c0 msgr2=0x7f9ca415e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:49.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.657+0000 7f9caad12640 1 --2- 172.21.15.167:0/1406640643 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ca406c2c0 0x7f9ca415e490 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9c980ad040 tx=0x7f9c9801ad60 comp rx=0 tx=0).stop 2024-04-17T10:37:49.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.657+0000 7f9caad12640 1 -- 172.21.15.167:0/1406640643 shutdown_connections 2024-04-17T10:37:49.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.657+0000 7f9caad12640 1 --2- 172.21.15.167:0/1406640643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ca415dd20 0x7f9ca41568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.657+0000 7f9caad12640 1 --2- 172.21.15.167:0/1406640643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ca40d1850 0x7f9ca40b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.657+0000 7f9caad12640 1 --2- 172.21.15.167:0/1406640643 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ca406c2c0 0x7f9ca415e490 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.657+0000 7f9caad12640 1 -- 172.21.15.167:0/1406640643 >> 172.21.15.167:0/1406640643 conn(0x7f9ca409a9d0 msgr2=0x7f9ca4159ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:37:49.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.657+0000 7f9caad12640 1 -- 172.21.15.167:0/1406640643 shutdown_connections 2024-04-17T10:37:49.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.657+0000 7f9caad12640 1 -- 172.21.15.167:0/1406640643 wait complete. 2024-04-17T10:37:49.673 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-04-17T10:37:49.673+0000 7f1e3a550180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-17T10:37:49.674 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 200a346c-44d2-4cf2-bc5a-66d9cfa79c93 5 2024-04-17T10:37:49.726 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-04-17T10:37:49.725+0000 7f1e3a550180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-17T10:37:49.805 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-04-17T10:37:49.805+0000 7f1e3a550180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-17T10:37:49.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.829+0000 7fcbc74e7640 1 Processor -- start 2024-04-17T10:37:49.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.829+0000 7fcbc74e7640 1 -- start start 2024-04-17T10:37:49.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.829+0000 7fcbc74e7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbc00afde0 0x7fcbc00b5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:49.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.829+0000 7fcbc74e7640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcbc006b2f0 0x7fcbc00b3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:49.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.829+0000 7fcbc74e7640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbc00b5cd0 con 0x7fcbc015cbd0 2024-04-17T10:37:49.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.829+0000 7fcbc74e7640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc00d1550 con 0x7fcbc006b2f0 2024-04-17T10:37:49.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.829+0000 7fcbc74e7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc00b01c0 con 0x7fcbc00afde0 2024-04-17T10:37:49.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.829+0000 7fcbc5a84640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcbc006b2f0 0x7fcbc00b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:49.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.829+0000 7fcbc5283640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbc00afde0 0x7fcbc00b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:49.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.829+0000 7fcbc5a84640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcbc006b2f0 0x7fcbc00b3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:36686/0 (socket says 172.21.15.167:36686) 2024-04-17T10:37:49.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.829+0000 7fcbc5a84640 1 -- 172.21.15.167:0/2675863629 learned_addr learned my addr 172.21.15.167:0/2675863629 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-04-17T10:37:49.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.829+0000 7fcbc5283640 1 -- 172.21.15.167:0/2675863629 >> v2:172.21.15.167:3300/0 conn(0x7fcbc006b2f0 msgr2=0x7fcbc00b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:37:49.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.829+0000 7fcbc5283640 1 --2- 172.21.15.167:0/2675863629 >> v2:172.21.15.167:3300/0 conn(0x7fcbc006b2f0 0x7fcbc00b3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.829+0000 7fcbc5283640 1 -- 172.21.15.167:0/2675863629 >> v1:172.21.15.83:6789/0 conn(0x7fcbc015cbd0 legacy=0x7fcbc006abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:37:49.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.829+0000 7fcbc5283640 1 -- 172.21.15.167:0/2675863629 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc00d07b0 con 0x7fcbc00afde0 2024-04-17T10:37:49.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.829+0000 7fcbc5a84640 1 --2- 172.21.15.167:0/2675863629 >> v2:172.21.15.167:3300/0 conn(0x7fcbc006b2f0 0x7fcbc00b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:37:49.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.833+0000 7fcbc5283640 1 --2- 172.21.15.167:0/2675863629 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbc00afde0 0x7fcbc00b5380 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8009d90 tx=0x7fcbb801c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae3b692ac3ffe5b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:37:49.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.833+0000 7fcbb7fff640 1 -- 172.21.15.167:0/2675863629 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb80b2070 con 0x7fcbc00afde0 2024-04-17T10:37:49.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.833+0000 7fcbb7fff640 1 -- 172.21.15.167:0/2675863629 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb80a5980 con 0x7fcbc00afde0 2024-04-17T10:37:49.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.833+0000 7fcbb7fff640 1 -- 172.21.15.167:0/2675863629 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb80a5d50 con 0x7fcbc00afde0 2024-04-17T10:37:49.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.833+0000 7fcbc74e7640 1 -- 172.21.15.167:0/2675863629 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbc00afde0 msgr2=0x7fcbc00b5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:49.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.833+0000 7fcbc74e7640 1 --2- 172.21.15.167:0/2675863629 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbc00afde0 0x7fcbc00b5380 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8009d90 tx=0x7fcbb801c900 comp rx=0 tx=0).stop 2024-04-17T10:37:49.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.833+0000 7fcbc74e7640 1 -- 172.21.15.167:0/2675863629 shutdown_connections 2024-04-17T10:37:49.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.833+0000 7fcbc74e7640 1 --2- 172.21.15.167:0/2675863629 >> v2:172.21.15.167:3300/0 conn(0x7fcbc006b2f0 0x7fcbc00b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.833+0000 7fcbc74e7640 1 --2- 172.21.15.167:0/2675863629 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbc00afde0 0x7fcbc00b5380 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.833+0000 7fcbc74e7640 1 -- 172.21.15.167:0/2675863629 >> 172.21.15.167:0/2675863629 conn(0x7fcbc006bcf0 msgr2=0x7fcbc006c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:37:49.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.833+0000 7fcbc74e7640 1 -- 172.21.15.167:0/2675863629 shutdown_connections 2024-04-17T10:37:49.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.833+0000 7fcbc74e7640 1 -- 172.21.15.167:0/2675863629 wait complete. 2024-04-17T10:37:49.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.833+0000 7fcbc74e7640 1 Processor -- start 2024-04-17T10:37:49.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.833+0000 7fcbc74e7640 1 -- start start 2024-04-17T10:37:49.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.837+0000 7fcbc74e7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbc006b2f0 0x7fcbc0149190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:49.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.837+0000 7fcbc74e7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbc00afde0 0x7fcbc0151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:49.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.837+0000 7fcbc74e7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbc015cbd0 0x7fcbc01580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:49.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.837+0000 7fcbc74e7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc00b5a40 con 0x7fcbc00afde0 2024-04-17T10:37:49.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.837+0000 7fcbc74e7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc00b58c0 con 0x7fcbc006b2f0 2024-04-17T10:37:49.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.837+0000 7fcbc74e7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc00b44d0 con 0x7fcbc015cbd0 2024-04-17T10:37:49.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.837+0000 7fcbc5283640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbc006b2f0 0x7fcbc0149190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:49.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.837+0000 7fcbc5283640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbc006b2f0 0x7fcbc0149190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:36690/0 (socket says 172.21.15.167:36690) 2024-04-17T10:37:49.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.837+0000 7fcbc5283640 1 -- 172.21.15.167:0/14783128 learned_addr learned my addr 172.21.15.167:0/14783128 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-04-17T10:37:49.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.837+0000 7fcbc5a84640 1 --2- 172.21.15.167:0/14783128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbc015cbd0 0x7fcbc01580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:49.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.837+0000 7fcbc4a82640 1 --2- 172.21.15.167:0/14783128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbc00afde0 0x7fcbc0151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:49.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.837+0000 7fcbc5283640 1 -- 172.21.15.167:0/14783128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbc015cbd0 msgr2=0x7fcbc01580d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:49.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.837+0000 7fcbc5283640 1 --2- 172.21.15.167:0/14783128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbc015cbd0 0x7fcbc01580d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.837+0000 7fcbc5283640 1 -- 172.21.15.167:0/14783128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbc00afde0 msgr2=0x7fcbc0151c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:49.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.837+0000 7fcbc5283640 1 --2- 172.21.15.167:0/14783128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbc00afde0 0x7fcbc0151c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.837+0000 7fcbc5283640 1 -- 172.21.15.167:0/14783128 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc0150930 con 0x7fcbc006b2f0 2024-04-17T10:37:49.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.837+0000 7fcbc5283640 1 --2- 172.21.15.167:0/14783128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbc006b2f0 0x7fcbc0149190 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcbb80b0040 tx=0x7fcbb80196a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:37:49.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.837+0000 7fcbb5ffb640 1 -- 172.21.15.167:0/14783128 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb80b2050 con 0x7fcbc006b2f0 2024-04-17T10:37:49.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.837+0000 7fcbb5ffb640 1 -- 172.21.15.167:0/14783128 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb80be070 con 0x7fcbc006b2f0 2024-04-17T10:37:49.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.837+0000 7fcbc74e7640 1 -- 172.21.15.167:0/14783128 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbc0150c10 con 0x7fcbc006b2f0 2024-04-17T10:37:49.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.837+0000 7fcbb5ffb640 1 -- 172.21.15.167:0/14783128 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb80ada90 con 0x7fcbc006b2f0 2024-04-17T10:37:49.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.837+0000 7fcbc74e7640 1 -- 172.21.15.167:0/14783128 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc01511d0 con 0x7fcbc006b2f0 2024-04-17T10:37:49.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.837+0000 7fcbb5ffb640 1 -- 172.21.15.167:0/14783128 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fcbb80adc30 con 0x7fcbc006b2f0 2024-04-17T10:37:49.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.837+0000 7fcbb5ffb640 1 -- 172.21.15.167:0/14783128 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fcbb80c3d70 con 0x7fcbc006b2f0 2024-04-17T10:37:49.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.837+0000 7fcbc74e7640 1 -- 172.21.15.167:0/14783128 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbc0074f90 con 0x7fcbc006b2f0 2024-04-17T10:37:49.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.841+0000 7fcbb5ffb640 1 -- 172.21.15.167:0/14783128 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fcbb8016520 con 0x7fcbc006b2f0 2024-04-17T10:37:49.861 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-04-17T10:37:49.861+0000 7f1e3a550180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-17T10:37:49.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.901+0000 7fcbc74e7640 1 -- 172.21.15.167:0/14783128 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd new", "uuid": "200a346c-44d2-4cf2-bc5a-66d9cfa79c93", "id": 5} v 0) v1 -- 0x7fcbc0153200 con 0x7fcbc006b2f0 2024-04-17T10:37:49.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.909+0000 7fcbb5ffb640 1 -- 172.21.15.167:0/14783128 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "200a346c-44d2-4cf2-bc5a-66d9cfa79c93", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcbb80b9450 con 0x7fcbc006b2f0 2024-04-17T10:37:49.909 INFO:teuthology.orchestra.run.smithi167.stdout:5 2024-04-17T10:37:49.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.909+0000 7fcbc74e7640 1 -- 172.21.15.167:0/14783128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbc006b2f0 msgr2=0x7fcbc0149190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:49.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.909+0000 7fcbc74e7640 1 --2- 172.21.15.167:0/14783128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbc006b2f0 0x7fcbc0149190 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcbb80b0040 tx=0x7fcbb80196a0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.909+0000 7fcbc74e7640 1 -- 172.21.15.167:0/14783128 shutdown_connections 2024-04-17T10:37:49.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.909+0000 7fcbc74e7640 1 --2- 172.21.15.167:0/14783128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbc015cbd0 0x7fcbc01580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.909+0000 7fcbc74e7640 1 --2- 172.21.15.167:0/14783128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbc00afde0 0x7fcbc0151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.909+0000 7fcbc74e7640 1 --2- 172.21.15.167:0/14783128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbc006b2f0 0x7fcbc0149190 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:49.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.909+0000 7fcbc74e7640 1 -- 172.21.15.167:0/14783128 >> 172.21.15.167:0/14783128 conn(0x7fcbc006bcf0 msgr2=0x7fcbc0158fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:37:49.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.909+0000 7fcbc74e7640 1 -- 172.21.15.167:0/14783128 shutdown_connections 2024-04-17T10:37:49.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:49.909+0000 7fcbc74e7640 1 -- 172.21.15.167:0/14783128 wait complete. 2024-04-17T10:37:49.925 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 30bd412b-9333-4b1a-bc0a-96bea209dae7 6 2024-04-17T10:37:50.013 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-04-17T10:37:50.013+0000 7f1e3a550180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-17T10:37:50.078 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-04-17T10:37:50.077+0000 7f1e3a550180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-17T10:37:50.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3d3620640 1 Processor -- start 2024-04-17T10:37:50.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3d3620640 1 -- start start 2024-04-17T10:37:50.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3d3620640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd3cc0cf320 0x7fd3cc066ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:50.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3d3620640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3cc072350 con 0x7fd3cc0ca560 2024-04-17T10:37:50.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3d3620640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3cc072630 con 0x7fd3cc06bbf0 2024-04-17T10:37:50.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3d3620640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3cc067000 con 0x7fd3cc0cf320 2024-04-17T10:37:50.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3d13bc640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fd3cc06bbf0 0x7fd3cc06bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:36062/0 (socket says 172.21.15.167:36062) 2024-04-17T10:37:50.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3d13bc640 1 -- 172.21.15.167:0/703061036 learned_addr learned my addr 172.21.15.167:0/703061036 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-04-17T10:37:50.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3c3fff640 1 -- 172.21.15.167:0/703061036 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4265336637 0 0) 0x7fd3cc072630 con 0x7fd3cc06bbf0 2024-04-17T10:37:50.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3c3fff640 1 -- 172.21.15.167:0/703061036 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3b8003530 con 0x7fd3cc06bbf0 2024-04-17T10:37:50.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3d0bbb640 1 --2- 172.21.15.167:0/703061036 >> v2:172.21.15.83:3300/0 conn(0x7fd3cc0cf320 0x7fd3cc066ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:50.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3c3fff640 1 -- 172.21.15.167:0/703061036 <== mon.1 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 880425351 0 0) 0x7fd3b8003530 con 0x7fd3cc06bbf0 2024-04-17T10:37:50.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3c3fff640 1 -- 172.21.15.167:0/703061036 >> v2:172.21.15.83:3300/0 conn(0x7fd3cc0cf320 msgr2=0x7fd3cc066ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:37:50.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3c3fff640 1 --2- 172.21.15.167:0/703061036 >> v2:172.21.15.83:3300/0 conn(0x7fd3cc0cf320 0x7fd3cc066ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:50.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3c3fff640 1 -- 172.21.15.167:0/703061036 >> v1:172.21.15.83:6789/0 conn(0x7fd3cc0ca560 legacy=0x7fd3cc0ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:37:50.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3c3fff640 1 -- 172.21.15.167:0/703061036 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3cc072910 con 0x7fd3cc06bbf0 2024-04-17T10:37:50.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3c3fff640 1 -- 172.21.15.167:0/703061036 <== mon.1 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fd3c8001680 con 0x7fd3cc06bbf0 2024-04-17T10:37:50.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3c3fff640 1 -- 172.21.15.167:0/703061036 >> v1:172.21.15.167:6789/0 conn(0x7fd3cc06bbf0 legacy=0x7fd3cc06bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:50.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3c3fff640 1 --2- 172.21.15.167:0/703061036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3b8003a30 0x7fd3b8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:50.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3c3fff640 1 --2- 172.21.15.167:0/703061036 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3b8004810 0x7fd3b8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:50.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3c3fff640 1 --2- 172.21.15.167:0/703061036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3b80052f0 0x7fd3b800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:50.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3c3fff640 1 -- 172.21.15.167:0/703061036 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b800c1e0 con 0x7fd3b8003a30 2024-04-17T10:37:50.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3c3fff640 1 -- 172.21.15.167:0/703061036 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b800c390 con 0x7fd3b80052f0 2024-04-17T10:37:50.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3c3fff640 1 -- 172.21.15.167:0/703061036 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b800c590 con 0x7fd3b8004810 2024-04-17T10:37:50.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3d1bbd640 1 --2- 172.21.15.167:0/703061036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3b80052f0 0x7fd3b800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:50.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3d0bbb640 1 --2- 172.21.15.167:0/703061036 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3b8004810 0x7fd3b8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:50.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3d13bc640 1 --2- 172.21.15.167:0/703061036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3b8003a30 0x7fd3b8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:50.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3d0bbb640 1 -- 172.21.15.167:0/703061036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3b80052f0 msgr2=0x7fd3b800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:37:50.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3d0bbb640 1 --2- 172.21.15.167:0/703061036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3b80052f0 0x7fd3b800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:50.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3d0bbb640 1 -- 172.21.15.167:0/703061036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3b8003a30 msgr2=0x7fd3b8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:37:50.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3d0bbb640 1 --2- 172.21.15.167:0/703061036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3b8003a30 0x7fd3b8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:50.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3d0bbb640 1 -- 172.21.15.167:0/703061036 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b8003530 con 0x7fd3b8004810 2024-04-17T10:37:50.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3d1bbd640 1 --2- 172.21.15.167:0/703061036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3b80052f0 0x7fd3b800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:37:50.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.101+0000 7fd3d1bbd640 1 -- 172.21.15.167:0/703061036 reap_dead start 2024-04-17T10:37:50.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.105+0000 7fd3d0bbb640 1 --2- 172.21.15.167:0/703061036 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3b8004810 0x7fd3b8004c90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00ba30 tx=0x7fd3bc01c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=969dbaf2c044c948 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:37:50.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.105+0000 7fd3c3fff640 1 -- 172.21.15.167:0/703061036 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc0b2070 con 0x7fd3b8004810 2024-04-17T10:37:50.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.105+0000 7fd3c3fff640 1 -- 172.21.15.167:0/703061036 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3bc0ad410 con 0x7fd3b8004810 2024-04-17T10:37:50.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.105+0000 7fd3c3fff640 1 -- 172.21.15.167:0/703061036 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc00b600 con 0x7fd3b8004810 2024-04-17T10:37:50.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.105+0000 7fd3d3620640 1 -- 172.21.15.167:0/703061036 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3b8004810 msgr2=0x7fd3b8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:50.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.105+0000 7fd3d3620640 1 --2- 172.21.15.167:0/703061036 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3b8004810 0x7fd3b8004c90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00ba30 tx=0x7fd3bc01c680 comp rx=0 tx=0).stop 2024-04-17T10:37:50.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.105+0000 7fd3d3620640 1 -- 172.21.15.167:0/703061036 shutdown_connections 2024-04-17T10:37:50.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.105+0000 7fd3d3620640 1 --2- 172.21.15.167:0/703061036 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3b8004810 0x7fd3b8004c90 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:50.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.105+0000 7fd3d3620640 1 -- 172.21.15.167:0/703061036 >> 172.21.15.167:0/703061036 conn(0x7fd3cc0cc990 msgr2=0x7fd3cc0ccda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:37:50.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.105+0000 7fd3d3620640 1 -- 172.21.15.167:0/703061036 shutdown_connections 2024-04-17T10:37:50.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.105+0000 7fd3d3620640 1 -- 172.21.15.167:0/703061036 wait complete. 2024-04-17T10:37:50.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.105+0000 7fd3d3620640 1 Processor -- start 2024-04-17T10:37:50.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.105+0000 7fd3d3620640 1 -- start start 2024-04-17T10:37:50.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.109+0000 7fd3d3620640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3b8004810 0x7fd3cc15a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:50.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.109+0000 7fd3d3620640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3cc157950 0x7fd3cc09c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:50.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.109+0000 7fd3d3620640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3cc09cc10 0x7fd3cc06d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:50.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.109+0000 7fd3d3620640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc0cee90 con 0x7fd3cc09cc10 2024-04-17T10:37:50.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.109+0000 7fd3d3620640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc0ced10 con 0x7fd3cc157950 2024-04-17T10:37:50.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.109+0000 7fd3d3620640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc0cf010 con 0x7fd3b8004810 2024-04-17T10:37:50.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.109+0000 7fd3d13bc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3b8004810 0x7fd3cc15a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:50.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.109+0000 7fd3d0bbb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3cc157950 0x7fd3cc09c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:50.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.109+0000 7fd3d13bc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3b8004810 0x7fd3cc15a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:48062/0 (socket says 172.21.15.167:48062) 2024-04-17T10:37:50.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.109+0000 7fd3d13bc640 1 -- 172.21.15.167:0/157841413 learned_addr learned my addr 172.21.15.167:0/157841413 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-04-17T10:37:50.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.109+0000 7fd3d1bbd640 1 --2- 172.21.15.167:0/157841413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3cc09cc10 0x7fd3cc06d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:50.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.109+0000 7fd3d13bc640 1 -- 172.21.15.167:0/157841413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3cc157950 msgr2=0x7fd3cc09c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:50.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.109+0000 7fd3d13bc640 1 --2- 172.21.15.167:0/157841413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3cc157950 0x7fd3cc09c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:50.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.109+0000 7fd3d13bc640 1 -- 172.21.15.167:0/157841413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3cc09cc10 msgr2=0x7fd3cc06d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:50.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.109+0000 7fd3d13bc640 1 --2- 172.21.15.167:0/157841413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3cc09cc10 0x7fd3cc06d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:50.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.109+0000 7fd3d13bc640 1 -- 172.21.15.167:0/157841413 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3cc15a6b0 con 0x7fd3b8004810 2024-04-17T10:37:50.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.109+0000 7fd3d0bbb640 1 --2- 172.21.15.167:0/157841413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3cc157950 0x7fd3cc09c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:37:50.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.109+0000 7fd3d1bbd640 1 --2- 172.21.15.167:0/157841413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3cc09cc10 0x7fd3cc06d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:37:50.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.109+0000 7fd3d13bc640 1 --2- 172.21.15.167:0/157841413 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3b8004810 0x7fd3cc15a170 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8007290 tx=0x7fd3c8007750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:37:50.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.109+0000 7fd3c1ffb640 1 -- 172.21.15.167:0/157841413 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c800f940 con 0x7fd3b8004810 2024-04-17T10:37:50.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.109+0000 7fd3c1ffb640 1 -- 172.21.15.167:0/157841413 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c8018070 con 0x7fd3b8004810 2024-04-17T10:37:50.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.109+0000 7fd3c1ffb640 1 -- 172.21.15.167:0/157841413 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c800fd00 con 0x7fd3b8004810 2024-04-17T10:37:50.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.109+0000 7fd3d3620640 1 -- 172.21.15.167:0/157841413 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3cc1593b0 con 0x7fd3b8004810 2024-04-17T10:37:50.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.109+0000 7fd3d3620640 1 -- 172.21.15.167:0/157841413 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3cc159970 con 0x7fd3b8004810 2024-04-17T10:37:50.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.109+0000 7fd3c1ffb640 1 -- 172.21.15.167:0/157841413 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fd3c801d460 con 0x7fd3b8004810 2024-04-17T10:37:50.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.109+0000 7fd3c1ffb640 1 -- 172.21.15.167:0/157841413 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fd3c8013c10 con 0x7fd3b8004810 2024-04-17T10:37:50.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.109+0000 7fd3d3620640 1 -- 172.21.15.167:0/157841413 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3cc074f90 con 0x7fd3b8004810 2024-04-17T10:37:50.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.113+0000 7fd3c1ffb640 1 -- 172.21.15.167:0/157841413 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fd3c8013e10 con 0x7fd3b8004810 2024-04-17T10:37:50.164 INFO:tasks.ceph.mgr.y.smithi083.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-17T10:37:50.164 INFO:tasks.ceph.mgr.y.smithi083.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-17T10:37:50.164 INFO:tasks.ceph.mgr.y.smithi083.stderr: from numpy import show_config as show_numpy_config 2024-04-17T10:37:50.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.181+0000 7fd3d3620640 1 -- 172.21.15.167:0/157841413 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd new", "uuid": "30bd412b-9333-4b1a-bc0a-96bea209dae7", "id": 6} v 0) v1 -- 0x7fd3cc0b5e90 con 0x7fd3b8004810 2024-04-17T10:37:50.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.185+0000 7fd3c1ffb640 1 -- 172.21.15.167:0/157841413 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "30bd412b-9333-4b1a-bc0a-96bea209dae7", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd3c8013620 con 0x7fd3b8004810 2024-04-17T10:37:50.186 INFO:teuthology.orchestra.run.smithi167.stdout:6 2024-04-17T10:37:50.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.189+0000 7fd3d3620640 1 -- 172.21.15.167:0/157841413 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3b8004810 msgr2=0x7fd3cc15a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:50.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.189+0000 7fd3d3620640 1 --2- 172.21.15.167:0/157841413 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3b8004810 0x7fd3cc15a170 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8007290 tx=0x7fd3c8007750 comp rx=0 tx=0).stop 2024-04-17T10:37:50.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.189+0000 7fd3d3620640 1 -- 172.21.15.167:0/157841413 shutdown_connections 2024-04-17T10:37:50.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.189+0000 7fd3d3620640 1 --2- 172.21.15.167:0/157841413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3cc09cc10 0x7fd3cc06d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:50.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.189+0000 7fd3d3620640 1 --2- 172.21.15.167:0/157841413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3cc157950 0x7fd3cc09c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:50.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.189+0000 7fd3d3620640 1 --2- 172.21.15.167:0/157841413 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3b8004810 0x7fd3cc15a170 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:50.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.189+0000 7fd3d3620640 1 -- 172.21.15.167:0/157841413 >> 172.21.15.167:0/157841413 conn(0x7fd3cc0cc990 msgr2=0x7fd3cc1587f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:37:50.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.189+0000 7fd3d3620640 1 -- 172.21.15.167:0/157841413 shutdown_connections 2024-04-17T10:37:50.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.189+0000 7fd3d3620640 1 -- 172.21.15.167:0/157841413 wait complete. 2024-04-17T10:37:50.203 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 23b53196-ca17-4727-b79c-1c43b245c21a 7 2024-04-17T10:37:50.224 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-17T10:37:50.226+0000 7f4d3e02c180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-17T10:37:50.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.317+0000 7f504364f640 1 Processor -- start 2024-04-17T10:37:50.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.317+0000 7f504364f640 1 -- start start 2024-04-17T10:37:50.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.317+0000 7f504364f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f503c0afde0 0x7f503c0b5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:50.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.317+0000 7f504364f640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f503c15cbd0 0x7f503c06abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:50.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.317+0000 7f504364f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f503c0b5cd0 con 0x7f503c06b120 2024-04-17T10:37:50.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.317+0000 7f504364f640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f503c06b590 con 0x7f503c0afde0 2024-04-17T10:37:50.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.317+0000 7f504364f640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f503c0d1550 con 0x7f503c15cbd0 2024-04-17T10:37:50.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.317+0000 7f5040bea640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f503c15cbd0 0x7f503c06abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:50.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.317+0000 7f5040bea640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f503c15cbd0 0x7f503c06abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:36712/0 (socket says 172.21.15.167:36712) 2024-04-17T10:37:50.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.317+0000 7f5040bea640 1 -- 172.21.15.167:0/499280461 learned_addr learned my addr 172.21.15.167:0/499280461 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-04-17T10:37:50.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.317+0000 7f50413eb640 1 --2- 172.21.15.167:0/499280461 >> v2:172.21.15.83:3300/0 conn(0x7f503c0afde0 0x7f503c0b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:50.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.317+0000 7f5040bea640 1 -- 172.21.15.167:0/499280461 >> v2:172.21.15.83:3300/0 conn(0x7f503c0afde0 msgr2=0x7f503c0b5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:37:50.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.317+0000 7f5040bea640 1 --2- 172.21.15.167:0/499280461 >> v2:172.21.15.83:3300/0 conn(0x7f503c0afde0 0x7f503c0b5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:50.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.317+0000 7f5040bea640 1 -- 172.21.15.167:0/499280461 >> v1:172.21.15.83:6789/0 conn(0x7f503c06b120 legacy=0x7f503c15f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:37:50.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.317+0000 7f5040bea640 1 -- 172.21.15.167:0/499280461 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f503c0b01c0 con 0x7f503c15cbd0 2024-04-17T10:37:50.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.317+0000 7f50413eb640 1 --2- 172.21.15.167:0/499280461 >> v2:172.21.15.83:3300/0 conn(0x7f503c0afde0 0x7f503c0b5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:37:50.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.321+0000 7f5040bea640 1 --2- 172.21.15.167:0/499280461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f503c15cbd0 0x7f503c06abe0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f503c071260 tx=0x7f502c01c4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caad261b30903530 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:37:50.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.321+0000 7f5033fff640 1 -- 172.21.15.167:0/499280461 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c0b1070 con 0x7f503c15cbd0 2024-04-17T10:37:50.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.321+0000 7f5033fff640 1 -- 172.21.15.167:0/499280461 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502c0ac630 con 0x7f503c15cbd0 2024-04-17T10:37:50.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.321+0000 7f5033fff640 1 -- 172.21.15.167:0/499280461 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c0a8050 con 0x7f503c15cbd0 2024-04-17T10:37:50.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.321+0000 7f504364f640 1 -- 172.21.15.167:0/499280461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f503c15cbd0 msgr2=0x7f503c06abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:50.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.321+0000 7f504364f640 1 --2- 172.21.15.167:0/499280461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f503c15cbd0 0x7f503c06abe0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f503c071260 tx=0x7f502c01c4b0 comp rx=0 tx=0).stop 2024-04-17T10:37:50.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.321+0000 7f504364f640 1 -- 172.21.15.167:0/499280461 shutdown_connections 2024-04-17T10:37:50.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.321+0000 7f504364f640 1 --2- 172.21.15.167:0/499280461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f503c15cbd0 0x7f503c06abe0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:50.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.321+0000 7f504364f640 1 --2- 172.21.15.167:0/499280461 >> v2:172.21.15.83:3300/0 conn(0x7f503c0afde0 0x7f503c0b5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:50.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.321+0000 7f504364f640 1 -- 172.21.15.167:0/499280461 >> 172.21.15.167:0/499280461 conn(0x7f503c06bcf0 msgr2=0x7f503c06c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:37:50.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.321+0000 7f504364f640 1 -- 172.21.15.167:0/499280461 shutdown_connections 2024-04-17T10:37:50.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.321+0000 7f504364f640 1 -- 172.21.15.167:0/499280461 wait complete. 2024-04-17T10:37:50.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.325+0000 7f504364f640 1 Processor -- start 2024-04-17T10:37:50.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.325+0000 7f504364f640 1 -- start start 2024-04-17T10:37:50.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.325+0000 7f504364f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f503c06b120 0x7f503c0ac810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:50.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.325+0000 7f504364f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f503c0afde0 0x7f503c1555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:50.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.325+0000 7f504364f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f503c15cbd0 0x7f503c0ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:50.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.325+0000 7f504364f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503c0b4790 con 0x7f503c06b120 2024-04-17T10:37:50.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.325+0000 7f504364f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503c0d1050 con 0x7f503c0afde0 2024-04-17T10:37:50.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.325+0000 7f504364f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f503c0b58c0 con 0x7f503c15cbd0 2024-04-17T10:37:50.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.325+0000 7f5040bea640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f503c0afde0 0x7f503c1555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:50.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.325+0000 7f5040bea640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f503c0afde0 0x7f503c1555d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:36718/0 (socket says 172.21.15.167:36718) 2024-04-17T10:37:50.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.325+0000 7f5040bea640 1 -- 172.21.15.167:0/1812442706 learned_addr learned my addr 172.21.15.167:0/1812442706 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-04-17T10:37:50.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.325+0000 7f5041bec640 1 --2- 172.21.15.167:0/1812442706 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f503c15cbd0 0x7f503c0ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:50.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.325+0000 7f50413eb640 1 --2- 172.21.15.167:0/1812442706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f503c06b120 0x7f503c0ac810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:50.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.325+0000 7f5040bea640 1 -- 172.21.15.167:0/1812442706 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f503c15cbd0 msgr2=0x7f503c0ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:50.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.325+0000 7f5040bea640 1 --2- 172.21.15.167:0/1812442706 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f503c15cbd0 0x7f503c0ae890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:50.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.325+0000 7f5040bea640 1 -- 172.21.15.167:0/1812442706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f503c06b120 msgr2=0x7f503c0ac810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:50.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.325+0000 7f5040bea640 1 --2- 172.21.15.167:0/1812442706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f503c06b120 0x7f503c0ac810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:50.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.325+0000 7f5040bea640 1 -- 172.21.15.167:0/1812442706 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f503c0ad0d0 con 0x7f503c0afde0 2024-04-17T10:37:50.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.325+0000 7f50413eb640 1 --2- 172.21.15.167:0/1812442706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f503c06b120 0x7f503c0ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:37:50.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.325+0000 7f5040bea640 1 --2- 172.21.15.167:0/1812442706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f503c0afde0 0x7f503c1555d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f502c0af040 tx=0x7f502c019250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:37:50.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.325+0000 7f5031ffb640 1 -- 172.21.15.167:0/1812442706 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c0b1040 con 0x7f503c0afde0 2024-04-17T10:37:50.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.325+0000 7f5031ffb640 1 -- 172.21.15.167:0/1812442706 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502c0b7450 con 0x7f503c0afde0 2024-04-17T10:37:50.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.325+0000 7f504364f640 1 -- 172.21.15.167:0/1812442706 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f503c0acdf0 con 0x7f503c0afde0 2024-04-17T10:37:50.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.325+0000 7f5031ffb640 1 -- 172.21.15.167:0/1812442706 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c0a8040 con 0x7f503c0afde0 2024-04-17T10:37:50.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.325+0000 7f504364f640 1 -- 172.21.15.167:0/1812442706 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f503c154dd0 con 0x7f503c0afde0 2024-04-17T10:37:50.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.325+0000 7f5031ffb640 1 -- 172.21.15.167:0/1812442706 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f502c0be020 con 0x7f503c0afde0 2024-04-17T10:37:50.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.325+0000 7f5031ffb640 1 -- 172.21.15.167:0/1812442706 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f502c0c0aa0 con 0x7f503c0afde0 2024-04-17T10:37:50.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.325+0000 7f504364f640 1 -- 172.21.15.167:0/1812442706 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f503c074f90 con 0x7f503c0afde0 2024-04-17T10:37:50.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.329+0000 7f5031ffb640 1 -- 172.21.15.167:0/1812442706 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f502c0c5020 con 0x7f503c0afde0 2024-04-17T10:37:50.405 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-17T10:37:50.406+0000 7f4d3e02c180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-17T10:37:50.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.425+0000 7f504364f640 1 -- 172.21.15.167:0/1812442706 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd new", "uuid": "23b53196-ca17-4727-b79c-1c43b245c21a", "id": 7} v 0) v1 -- 0x7f503c071d60 con 0x7f503c0afde0 2024-04-17T10:37:50.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.433+0000 7f5031ffb640 1 -- 172.21.15.167:0/1812442706 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "23b53196-ca17-4727-b79c-1c43b245c21a", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f503c0ad0d0 con 0x7f503c0afde0 2024-04-17T10:37:50.435 INFO:teuthology.orchestra.run.smithi167.stdout:7 2024-04-17T10:37:50.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.449+0000 7f504364f640 1 -- 172.21.15.167:0/1812442706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f503c0afde0 msgr2=0x7f503c1555d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:50.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.449+0000 7f504364f640 1 --2- 172.21.15.167:0/1812442706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f503c0afde0 0x7f503c1555d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f502c0af040 tx=0x7f502c019250 comp rx=0 tx=0).stop 2024-04-17T10:37:50.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.449+0000 7f504364f640 1 -- 172.21.15.167:0/1812442706 shutdown_connections 2024-04-17T10:37:50.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.449+0000 7f504364f640 1 --2- 172.21.15.167:0/1812442706 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f503c15cbd0 0x7f503c0ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:50.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.449+0000 7f504364f640 1 --2- 172.21.15.167:0/1812442706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f503c0afde0 0x7f503c1555d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:50.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.449+0000 7f504364f640 1 --2- 172.21.15.167:0/1812442706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f503c06b120 0x7f503c0ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:50.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.449+0000 7f504364f640 1 -- 172.21.15.167:0/1812442706 >> 172.21.15.167:0/1812442706 conn(0x7f503c06bcf0 msgr2=0x7f503c06c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:37:50.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.449+0000 7f504364f640 1 -- 172.21.15.167:0/1812442706 shutdown_connections 2024-04-17T10:37:50.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:37:50.449+0000 7f504364f640 1 -- 172.21.15.167:0/1812442706 wait complete. 2024-04-17T10:37:50.465 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-17T10:37:50.465 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-17T10:37:50.467 INFO:tasks.ceph.osd.0:Started 2024-04-17T10:37:50.468 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-17T10:37:50.468 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-17T10:37:50.470 INFO:tasks.ceph.osd.1:Started 2024-04-17T10:37:50.470 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-17T10:37:50.470 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-17T10:37:50.473 INFO:tasks.ceph.osd.2:Started 2024-04-17T10:37:50.473 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-17T10:37:50.473 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-17T10:37:50.475 INFO:tasks.ceph.osd.3:Started 2024-04-17T10:37:50.475 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-17T10:37:50.476 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-17T10:37:50.478 INFO:tasks.ceph.osd.4:Started 2024-04-17T10:37:50.478 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-17T10:37:50.478 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-17T10:37:50.480 INFO:tasks.ceph.osd.5:Started 2024-04-17T10:37:50.480 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-17T10:37:50.480 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-17T10:37:50.482 INFO:tasks.ceph.osd.6:Started 2024-04-17T10:37:50.482 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-17T10:37:50.482 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-17T10:37:50.484 INFO:tasks.ceph.osd.7:Started 2024-04-17T10:37:50.484 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:37:50.593 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-17T10:37:50.594+0000 7f4d3e02c180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-17T10:37:50.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.606+0000 7f50b1029640 1 Processor -- start 2024-04-17T10:37:50.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.606+0000 7f50b1029640 1 -- start start 2024-04-17T10:37:50.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.610+0000 7f50b1029640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f50ac06c300 0x7f50ac072ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:50.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.610+0000 7f50b1029640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50ac06cbf0 0x7f50ac0cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:50.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.610+0000 7f50b1029640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50ac06d080 con 0x7f50ac15a900 2024-04-17T10:37:50.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.610+0000 7f50b1029640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50ac071000 con 0x7f50ac06c300 2024-04-17T10:37:50.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.610+0000 7f50b1029640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac06ef90 con 0x7f50ac06cbf0 2024-04-17T10:37:50.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.610+0000 7f50ab59e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50ac06cbf0 0x7f50ac0cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:50.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.610+0000 7f50aa59c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f50ac06c300 0x7f50ac072ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:50.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.610+0000 7f50aad9d640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f50ac15a900 0x7f50ac06bbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:36306/0 (socket says 172.21.15.83:36306) 2024-04-17T10:37:50.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.610+0000 7f50aad9d640 1 -- 172.21.15.83:0/3984657664 learned_addr learned my addr 172.21.15.83:0/3984657664 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:37:50.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.610+0000 7f50aa59c640 1 -- 172.21.15.83:0/3984657664 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50ac06cbf0 msgr2=0x7f50ac0cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:37:50.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.610+0000 7f50aa59c640 1 --2- 172.21.15.83:0/3984657664 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50ac06cbf0 0x7f50ac0cc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:50.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.610+0000 7f50aa59c640 1 -- 172.21.15.83:0/3984657664 >> v1:172.21.15.167:6789/0 conn(0x7f50ac15a900 legacy=0x7f50ac06bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:50.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.610+0000 7f50aa59c640 1 -- 172.21.15.83:0/3984657664 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ac06f270 con 0x7f50ac06c300 2024-04-17T10:37:50.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.610+0000 7f50aa59c640 1 --2- 172.21.15.83:0/3984657664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50ac06c300 0x7f50ac072ab0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f509c0097c0 tx=0x7f509c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e85506f32b261bb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:37:50.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.610+0000 7f50a9d9b640 1 -- 172.21.15.83:0/3984657664 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c0b2070 con 0x7f50ac06c300 2024-04-17T10:37:50.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.610+0000 7f50a9d9b640 1 -- 172.21.15.83:0/3984657664 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509c0ad640 con 0x7f50ac06c300 2024-04-17T10:37:50.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.610+0000 7f50a9d9b640 1 -- 172.21.15.83:0/3984657664 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c0ada10 con 0x7f50ac06c300 2024-04-17T10:37:50.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50b1029640 1 -- 172.21.15.83:0/3984657664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50ac06c300 msgr2=0x7f50ac072ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:50.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50b1029640 1 --2- 172.21.15.83:0/3984657664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50ac06c300 0x7f50ac072ab0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f509c0097c0 tx=0x7f509c01c4a0 comp rx=0 tx=0).stop 2024-04-17T10:37:50.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50b1029640 1 -- 172.21.15.83:0/3984657664 shutdown_connections 2024-04-17T10:37:50.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50b1029640 1 --2- 172.21.15.83:0/3984657664 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50ac06cbf0 0x7f50ac0cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:50.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50b1029640 1 --2- 172.21.15.83:0/3984657664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50ac06c300 0x7f50ac072ab0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:50.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50b1029640 1 -- 172.21.15.83:0/3984657664 >> 172.21.15.83:0/3984657664 conn(0x7f50ac0cc910 msgr2=0x7f50ac0ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:37:50.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50b1029640 1 -- 172.21.15.83:0/3984657664 shutdown_connections 2024-04-17T10:37:50.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50b1029640 1 -- 172.21.15.83:0/3984657664 wait complete. 2024-04-17T10:37:50.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50b1029640 1 Processor -- start 2024-04-17T10:37:50.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50b1029640 1 -- start start 2024-04-17T10:37:50.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50b1029640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50ac06c300 0x7f50ac155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:50.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50b1029640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50ac06cbf0 0x7f50ac154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:50.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50b1029640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50ac15a900 0x7f50ac09ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:50.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50b1029640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac0cb880 con 0x7f50ac06cbf0 2024-04-17T10:37:50.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50b1029640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac0cb700 con 0x7f50ac06c300 2024-04-17T10:37:50.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50b1029640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac0cba00 con 0x7f50ac15a900 2024-04-17T10:37:50.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50aa59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50ac06cbf0 0x7f50ac154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:50.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50aa59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50ac06cbf0 0x7f50ac154870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35516/0 (socket says 172.21.15.83:35516) 2024-04-17T10:37:50.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50aa59c640 1 -- 172.21.15.83:0/1668192123 learned_addr learned my addr 172.21.15.83:0/1668192123 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:37:50.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50aad9d640 1 --2- 172.21.15.83:0/1668192123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50ac06c300 0x7f50ac155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:50.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50ab59e640 1 --2- 172.21.15.83:0/1668192123 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50ac15a900 0x7f50ac09ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:50.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50aa59c640 1 -- 172.21.15.83:0/1668192123 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50ac15a900 msgr2=0x7f50ac09ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:50.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50aa59c640 1 --2- 172.21.15.83:0/1668192123 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50ac15a900 0x7f50ac09ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:50.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50aa59c640 1 -- 172.21.15.83:0/1668192123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50ac06c300 msgr2=0x7f50ac155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:50.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50aa59c640 1 --2- 172.21.15.83:0/1668192123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50ac06c300 0x7f50ac155290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:50.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50aa59c640 1 -- 172.21.15.83:0/1668192123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ac0a0b10 con 0x7f50ac06cbf0 2024-04-17T10:37:50.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50aa59c640 1 --2- 172.21.15.83:0/1668192123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50ac06cbf0 0x7f50ac154870 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f509c01a0d0 tx=0x7f509c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:37:50.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50937fe640 1 -- 172.21.15.83:0/1668192123 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c0b2040 con 0x7f50ac06cbf0 2024-04-17T10:37:50.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50937fe640 1 -- 172.21.15.83:0/1668192123 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509c0bd070 con 0x7f50ac06cbf0 2024-04-17T10:37:50.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50b1029640 1 -- 172.21.15.83:0/1668192123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50ac0a0d90 con 0x7f50ac06cbf0 2024-04-17T10:37:50.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50937fe640 1 -- 172.21.15.83:0/1668192123 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c0b6630 con 0x7f50ac06cbf0 2024-04-17T10:37:50.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50b1029640 1 -- 172.21.15.83:0/1668192123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50ac0a1050 con 0x7f50ac06cbf0 2024-04-17T10:37:50.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.614+0000 7f50b1029640 1 -- 172.21.15.83:0/1668192123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50ac074f90 con 0x7f50ac06cbf0 2024-04-17T10:37:50.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.618+0000 7f50937fe640 1 -- 172.21.15.83:0/1668192123 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f509c0b67d0 con 0x7f50ac06cbf0 2024-04-17T10:37:50.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.618+0000 7f50937fe640 1 -- 172.21.15.83:0/1668192123 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f509c0cc430 con 0x7f50ac06cbf0 2024-04-17T10:37:50.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.618+0000 7f50937fe640 1 -- 172.21.15.83:0/1668192123 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f50ac0a0b10 con 0x7f50ac06cbf0 2024-04-17T10:37:50.665 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-17T10:37:50.666+0000 7f4d3e02c180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-17T10:37:50.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.674+0000 7f50b1029640 1 -- 172.21.15.83:0/1668192123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50ac068150 con 0x7f50ac06cbf0 2024-04-17T10:37:50.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.674+0000 7f50937fe640 1 -- 172.21.15.83:0/1668192123 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f509c016090 con 0x7f50ac06cbf0 2024-04-17T10:37:50.672 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:37:50.672 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":10,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:37:50.436350+0000","last_up_change":"0.000000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:37:50.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.674+0000 7f50b1029640 1 -- 172.21.15.83:0/1668192123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50ac06cbf0 msgr2=0x7f50ac154870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:50.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.674+0000 7f50b1029640 1 --2- 172.21.15.83:0/1668192123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50ac06cbf0 0x7f50ac154870 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f509c01a0d0 tx=0x7f509c019240 comp rx=0 tx=0).stop 2024-04-17T10:37:50.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.674+0000 7f50b1029640 1 -- 172.21.15.83:0/1668192123 shutdown_connections 2024-04-17T10:37:50.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.674+0000 7f50b1029640 1 --2- 172.21.15.83:0/1668192123 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50ac15a900 0x7f50ac09ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:50.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.674+0000 7f50b1029640 1 --2- 172.21.15.83:0/1668192123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50ac06cbf0 0x7f50ac154870 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:50.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.674+0000 7f50b1029640 1 --2- 172.21.15.83:0/1668192123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50ac06c300 0x7f50ac155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:50.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.674+0000 7f50b1029640 1 -- 172.21.15.83:0/1668192123 >> 172.21.15.83:0/1668192123 conn(0x7f50ac0cc910 msgr2=0x7f50ac06b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:37:50.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.674+0000 7f50b1029640 1 -- 172.21.15.83:0/1668192123 shutdown_connections 2024-04-17T10:37:50.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:37:50.674+0000 7f50b1029640 1 -- 172.21.15.83:0/1668192123 wait complete. 2024-04-17T10:37:50.688 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-17T10:37:50.688 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-17T10:37:50.714 INFO:tasks.ceph.mgr.x.smithi167.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-17T10:37:50.714 INFO:tasks.ceph.mgr.x.smithi167.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-17T10:37:50.715 INFO:tasks.ceph.mgr.x.smithi167.stderr: from numpy import show_config as show_numpy_config 2024-04-17T10:37:50.779 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-04-17T10:37:50.777+0000 7f1e3a550180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-17T10:37:50.809 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-17T10:37:50.814+0000 7f4d3e02c180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-17T10:37:50.834 INFO:tasks.ceph.osd.5.smithi167.stderr:2024-04-17T10:37:50.833+0000 7f4e444b07c0 -1 Falling back to public interface 2024-04-17T10:37:50.953 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-04-17T10:37:50.953+0000 7f1e3a550180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-17T10:37:51.088 INFO:tasks.ceph.osd.7.smithi167.stderr:2024-04-17T10:37:51.089+0000 7fe072cff7c0 -1 Falling back to public interface 2024-04-17T10:37:51.090 INFO:tasks.ceph.osd.4.smithi167.stderr:2024-04-17T10:37:51.089+0000 7ff143ffe7c0 -1 Falling back to public interface 2024-04-17T10:37:51.090 INFO:tasks.ceph.osd.6.smithi167.stderr:2024-04-17T10:37:51.089+0000 7fd05ba0d7c0 -1 Falling back to public interface 2024-04-17T10:37:51.099 INFO:tasks.ceph.osd.3.smithi083.stderr:2024-04-17T10:37:51.102+0000 7f0c38cbf7c0 -1 Falling back to public interface 2024-04-17T10:37:51.110 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-04-17T10:37:51.114+0000 7f7e379dc7c0 -1 Falling back to public interface 2024-04-17T10:37:51.111 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-04-17T10:37:51.113+0000 7f1e3a550180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-17T10:37:51.114 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-04-17T10:37:51.118+0000 7f35e8dcd7c0 -1 Falling back to public interface 2024-04-17T10:37:51.114 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-04-17T10:37:51.118+0000 7f79362757c0 -1 Falling back to public interface 2024-04-17T10:37:51.175 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-04-17T10:37:51.173+0000 7f1e3a550180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-17T10:37:51.307 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-04-17T10:37:51.309+0000 7f1e3a550180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-17T10:37:51.447 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-17T10:37:51.450+0000 7f4d3e02c180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-17T10:37:51.493 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-17T10:37:51.522 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-17T10:37:51.526+0000 7f4d3e02c180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-17T10:37:51.601 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.606+0000 7fd4619ee640 1 Processor -- start 2024-04-17T10:37:51.602 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.606+0000 7fd4619ee640 1 -- start start 2024-04-17T10:37:51.603 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.606+0000 7fd4619ee640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd45c070a50 0x7fd45c15e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:51.604 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.606+0000 7fd4619ee640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd45c0cdca0 0x7fd45c0ad930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:51.604 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.606+0000 7fd4619ee640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd45c0ce210 con 0x7fd45c158350 2024-04-17T10:37:51.604 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.606+0000 7fd4619ee640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd45c0ce4f0 con 0x7fd45c070a50 2024-04-17T10:37:51.604 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.606+0000 7fd4619ee640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c0ce7d0 con 0x7fd45c0cdca0 2024-04-17T10:37:51.604 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.606+0000 7fd45affd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd45c070a50 0x7fd45c15e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:51.604 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.606+0000 7fd45affd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd45c070a50 0x7fd45c15e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35520/0 (socket says 172.21.15.83:35520) 2024-04-17T10:37:51.604 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.606+0000 7fd45affd640 1 -- 172.21.15.83:0/2983962984 learned_addr learned my addr 172.21.15.83:0/2983962984 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:37:51.604 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.606+0000 7fd45affd640 1 -- 172.21.15.83:0/2983962984 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd45c0cdca0 msgr2=0x7fd45c0ad930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:37:51.604 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.606+0000 7fd45affd640 1 --2- 172.21.15.83:0/2983962984 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd45c0cdca0 0x7fd45c0ad930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:51.604 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.606+0000 7fd45affd640 1 -- 172.21.15.83:0/2983962984 >> v1:172.21.15.167:6789/0 conn(0x7fd45c158350 legacy=0x7fd45c070e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:37:51.604 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.606+0000 7fd45affd640 1 -- 172.21.15.83:0/2983962984 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd45c0ceab0 con 0x7fd45c070a50 2024-04-17T10:37:51.606 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd45affd640 1 --2- 172.21.15.83:0/2983962984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd45c070a50 0x7fd45c15e080 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd44c009870 tx=0x7fd44c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d21dcf1dfda658c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:37:51.606 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd45a7fc640 1 -- 172.21.15.83:0/2983962984 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c0aa430 con 0x7fd45c070a50 2024-04-17T10:37:51.606 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd45a7fc640 1 -- 172.21.15.83:0/2983962984 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44c0af070 con 0x7fd45c070a50 2024-04-17T10:37:51.606 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd45a7fc640 1 -- 172.21.15.83:0/2983962984 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c0aa7f0 con 0x7fd45c070a50 2024-04-17T10:37:51.607 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd4619ee640 1 -- 172.21.15.83:0/2983962984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd45c070a50 msgr2=0x7fd45c15e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:51.607 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd4619ee640 1 --2- 172.21.15.83:0/2983962984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd45c070a50 0x7fd45c15e080 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd44c009870 tx=0x7fd44c01c4f0 comp rx=0 tx=0).stop 2024-04-17T10:37:51.607 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd4619ee640 1 -- 172.21.15.83:0/2983962984 shutdown_connections 2024-04-17T10:37:51.607 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd4619ee640 1 --2- 172.21.15.83:0/2983962984 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd45c0cdca0 0x7fd45c0ad930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:51.607 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd4619ee640 1 --2- 172.21.15.83:0/2983962984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd45c070a50 0x7fd45c15e080 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:51.607 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd4619ee640 1 -- 172.21.15.83:0/2983962984 >> 172.21.15.83:0/2983962984 conn(0x7fd45c06abf0 msgr2=0x7fd45c06b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:37:51.607 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd4619ee640 1 -- 172.21.15.83:0/2983962984 shutdown_connections 2024-04-17T10:37:51.607 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd4619ee640 1 -- 172.21.15.83:0/2983962984 wait complete. 2024-04-17T10:37:51.607 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd4619ee640 1 Processor -- start 2024-04-17T10:37:51.608 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd4619ee640 1 -- start start 2024-04-17T10:37:51.610 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd4619ee640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd45c070a50 0x7fd45c156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:51.610 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd4619ee640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd45c0cdca0 0x7fd45c14ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:51.610 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd4619ee640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd45c150500 0x7fd45c14e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:51.610 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd4619ee640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c0aea50 con 0x7fd45c150500 2024-04-17T10:37:51.610 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd4619ee640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c0ae8d0 con 0x7fd45c0cdca0 2024-04-17T10:37:51.610 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd4619ee640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c0aebd0 con 0x7fd45c070a50 2024-04-17T10:37:51.610 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd45b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd45c150500 0x7fd45c14e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:51.610 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd45b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd45c150500 0x7fd45c14e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35534/0 (socket says 172.21.15.83:35534) 2024-04-17T10:37:51.610 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd45b7fe640 1 -- 172.21.15.83:0/1833280225 learned_addr learned my addr 172.21.15.83:0/1833280225 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:37:51.611 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd453fff640 1 --2- 172.21.15.83:0/1833280225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd45c0cdca0 0x7fd45c14ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:51.611 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd45affd640 1 --2- 172.21.15.83:0/1833280225 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd45c070a50 0x7fd45c156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:51.611 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd45b7fe640 1 -- 172.21.15.83:0/1833280225 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd45c070a50 msgr2=0x7fd45c156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:51.611 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd45b7fe640 1 --2- 172.21.15.83:0/1833280225 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd45c070a50 0x7fd45c156ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:51.611 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd45b7fe640 1 -- 172.21.15.83:0/1833280225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd45c0cdca0 msgr2=0x7fd45c14ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:51.611 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd45b7fe640 1 --2- 172.21.15.83:0/1833280225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd45c0cdca0 0x7fd45c14ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:51.611 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd45b7fe640 1 -- 172.21.15.83:0/1833280225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd45c06cd50 con 0x7fd45c150500 2024-04-17T10:37:51.611 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd45affd640 1 --2- 172.21.15.83:0/1833280225 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd45c070a50 0x7fd45c156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:37:51.611 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd453fff640 1 --2- 172.21.15.83:0/1833280225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd45c0cdca0 0x7fd45c14ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:37:51.611 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd45b7fe640 1 --2- 172.21.15.83:0/1833280225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd45c150500 0x7fd45c14e980 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd45400b970 tx=0x7fd45400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:37:51.611 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd458ff9640 1 -- 172.21.15.83:0/1833280225 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd454019070 con 0x7fd45c150500 2024-04-17T10:37:51.611 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd458ff9640 1 -- 172.21.15.83:0/1833280225 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd45400cb60 con 0x7fd45c150500 2024-04-17T10:37:51.611 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd458ff9640 1 -- 172.21.15.83:0/1833280225 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd454014440 con 0x7fd45c150500 2024-04-17T10:37:51.611 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.610+0000 7fd4619ee640 1 -- 172.21.15.83:0/1833280225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd45c06d030 con 0x7fd45c150500 2024-04-17T10:37:51.611 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.614+0000 7fd4619ee640 1 -- 172.21.15.83:0/1833280225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd45c06d5f0 con 0x7fd45c150500 2024-04-17T10:37:51.612 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.614+0000 7fd458ff9640 1 -- 172.21.15.83:0/1833280225 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fd45401c070 con 0x7fd45c150500 2024-04-17T10:37:51.612 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.614+0000 7fd458ff9640 1 -- 172.21.15.83:0/1833280225 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fd454014cb0 con 0x7fd45c150500 2024-04-17T10:37:51.612 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.614+0000 7fd4619ee640 1 -- 172.21.15.83:0/1833280225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd45c06d7f0 con 0x7fd45c150500 2024-04-17T10:37:51.614 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.618+0000 7fd458ff9640 1 -- 172.21.15.83:0/1833280225 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fd45c06d7f0 con 0x7fd45c150500 2024-04-17T10:37:51.667 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.670+0000 7fd4619ee640 1 -- 172.21.15.83:0/1833280225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd45c04ef30 con 0x7fd45c150500 2024-04-17T10:37:51.668 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.670+0000 7fd458ff9640 1 -- 172.21.15.83:0/1833280225 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fd454023070 con 0x7fd45c150500 2024-04-17T10:37:51.668 INFO:teuthology.misc.health.smithi083.stdout: 2024-04-17T10:37:51.668 INFO:teuthology.misc.health.smithi083.stdout:{"epoch":10,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:37:50.436350+0000","last_up_change":"0.000000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:37:51.669 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.670+0000 7fd4619ee640 1 -- 172.21.15.83:0/1833280225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd45c150500 msgr2=0x7fd45c14e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:51.669 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.670+0000 7fd4619ee640 1 --2- 172.21.15.83:0/1833280225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd45c150500 0x7fd45c14e980 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd45400b970 tx=0x7fd45400be30 comp rx=0 tx=0).stop 2024-04-17T10:37:51.669 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.670+0000 7fd4619ee640 1 -- 172.21.15.83:0/1833280225 shutdown_connections 2024-04-17T10:37:51.669 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.670+0000 7fd4619ee640 1 --2- 172.21.15.83:0/1833280225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd45c150500 0x7fd45c14e980 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:51.669 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.670+0000 7fd4619ee640 1 --2- 172.21.15.83:0/1833280225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd45c0cdca0 0x7fd45c14ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:51.669 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.670+0000 7fd4619ee640 1 --2- 172.21.15.83:0/1833280225 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd45c070a50 0x7fd45c156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:51.669 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.670+0000 7fd4619ee640 1 -- 172.21.15.83:0/1833280225 >> 172.21.15.83:0/1833280225 conn(0x7fd45c06abf0 msgr2=0x7fd45c151cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:37:51.669 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.670+0000 7fd4619ee640 1 -- 172.21.15.83:0/1833280225 shutdown_connections 2024-04-17T10:37:51.670 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:51.670+0000 7fd4619ee640 1 -- 172.21.15.83:0/1833280225 wait complete. 2024-04-17T10:37:51.683 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-17T10:37:51.710 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-17T10:37:51.714+0000 7f4d3e02c180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-17T10:37:51.784 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-17T10:37:51.786+0000 7f4d3e02c180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-17T10:37:51.935 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-04-17T10:37:51.937+0000 7f1e3a550180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-17T10:37:52.010 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-04-17T10:37:52.009+0000 7f1e3a550180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-17T10:37:52.074 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-17T10:37:52.078+0000 7f4d3e02c180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-17T10:37:52.136 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-17T10:37:52.138+0000 7f4d3e02c180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-17T10:37:52.192 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-04-17T10:37:52.193+0000 7f1e3a550180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-17T10:37:52.270 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-04-17T10:37:52.269+0000 7f1e3a550180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-17T10:37:52.561 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-04-17T10:37:52.561+0000 7f1e3a550180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-17T10:37:52.623 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-04-17T10:37:52.621+0000 7f1e3a550180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-17T10:37:53.177 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-17T10:37:53.178+0000 7f4ce5479640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-17T10:37:53.178 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-04-17T10:37:53.182+0000 7f4ce5479640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-17T10:37:54.755 INFO:tasks.ceph.osd.5.smithi167.stderr:2024-04-17T10:37:54.757+0000 7f4e444b07c0 -1 osd.5 0 log_to_monitors true 2024-04-17T10:37:54.882 INFO:tasks.ceph.osd.3.smithi083.stderr:2024-04-17T10:37:54.886+0000 7f0c38cbf7c0 -1 osd.3 0 log_to_monitors true 2024-04-17T10:37:54.941 INFO:tasks.ceph.osd.4.smithi167.stderr:2024-04-17T10:37:54.941+0000 7ff143ffe7c0 -1 osd.4 0 log_to_monitors true 2024-04-17T10:37:54.987 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-04-17T10:37:54.990+0000 7f7e379dc7c0 -1 osd.1 0 log_to_monitors true 2024-04-17T10:37:54.988 INFO:tasks.ceph.osd.7.smithi167.stderr:2024-04-17T10:37:54.989+0000 7fe072cff7c0 -1 osd.7 0 log_to_monitors true 2024-04-17T10:37:54.988 INFO:tasks.ceph.osd.6.smithi167.stderr:2024-04-17T10:37:54.989+0000 7fd05ba0d7c0 -1 osd.6 0 log_to_monitors true 2024-04-17T10:37:55.055 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-04-17T10:37:55.058+0000 7f35e8dcd7c0 -1 osd.2 0 log_to_monitors true 2024-04-17T10:37:55.067 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-04-17T10:37:55.070+0000 7f79362757c0 -1 osd.0 0 log_to_monitors true 2024-04-17T10:37:58.314 INFO:tasks.ceph.osd.3.smithi083.stderr:2024-04-17T10:37:58.314+0000 7f0c35c7b640 -1 osd.3 0 waiting for initial osdmap 2024-04-17T10:37:58.350 INFO:tasks.ceph.osd.3.smithi083.stderr:2024-04-17T10:37:58.354+0000 7f0c2ca8c640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T10:37:58.369 INFO:tasks.ceph.osd.7.smithi167.stderr:2024-04-17T10:37:58.369+0000 7fe06fcbb640 -1 osd.7 0 waiting for initial osdmap 2024-04-17T10:37:58.400 INFO:tasks.ceph.osd.7.smithi167.stderr:2024-04-17T10:37:58.401+0000 7fe066acc640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T10:37:58.412 INFO:tasks.ceph.osd.6.smithi167.stderr:2024-04-17T10:37:58.413+0000 7fd057193640 -1 osd.6 0 waiting for initial osdmap 2024-04-17T10:37:58.440 INFO:tasks.ceph.osd.6.smithi167.stderr:2024-04-17T10:37:58.441+0000 7fd04f7da640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T10:37:58.489 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-17T10:37:58.496 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-04-17T10:37:58.498+0000 7f79319fb640 -1 osd.0 0 waiting for initial osdmap 2024-04-17T10:37:58.499 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-04-17T10:37:58.502+0000 7f7e33162640 -1 osd.1 0 waiting for initial osdmap 2024-04-17T10:37:58.500 INFO:tasks.ceph.osd.5.smithi167.stderr:2024-04-17T10:37:58.501+0000 7f4e3fc36640 -1 osd.5 0 waiting for initial osdmap 2024-04-17T10:37:58.515 INFO:tasks.ceph.osd.4.smithi167.stderr:2024-04-17T10:37:58.517+0000 7ff13f784640 -1 osd.4 0 waiting for initial osdmap 2024-04-17T10:37:58.524 INFO:tasks.ceph.osd.5.smithi167.stderr:2024-04-17T10:37:58.525+0000 7f4e3827d640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T10:37:58.526 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-04-17T10:37:58.530+0000 7f792a042640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T10:37:58.527 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-04-17T10:37:58.530+0000 7f7e2b7a9640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T10:37:58.536 INFO:tasks.ceph.osd.4.smithi167.stderr:2024-04-17T10:37:58.537+0000 7ff137dcb640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T10:37:58.537 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-04-17T10:37:58.538+0000 7f35e4553640 -1 osd.2 0 waiting for initial osdmap 2024-04-17T10:37:58.570 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-04-17T10:37:58.574+0000 7f35dcb9a640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T10:37:58.606 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.610+0000 7f90ea5b9640 1 Processor -- start 2024-04-17T10:37:58.606 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.610+0000 7f90ea5b9640 1 -- start start 2024-04-17T10:37:58.608 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.610+0000 7f90ea5b9640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90e40cbe80 0x7f90e406cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:58.608 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.610+0000 7f90ea5b9640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f90e40b25c0 0x7f90e40b4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:58.608 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.610+0000 7f90ea5b9640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f90e40b2a20 0x7f90e40b4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:58.608 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.610+0000 7f90ea5b9640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e40b34f0 con 0x7f90e40b25c0 2024-04-17T10:37:58.608 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.610+0000 7f90ea5b9640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e40b2f30 con 0x7f90e40b2a20 2024-04-17T10:37:58.608 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.610+0000 7f90ea5b9640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e40b3210 con 0x7f90e40cbe80 2024-04-17T10:37:58.608 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.610+0000 7f90dbfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f90e40b25c0 0x7f90e40b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:58.608 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.610+0000 7f90dbfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f90e40b25c0 0x7f90e40b4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33906/0 (socket says 172.21.15.83:33906) 2024-04-17T10:37:58.608 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.610+0000 7f90dbfff640 1 -- 172.21.15.83:0/4242132801 learned_addr learned my addr 172.21.15.83:0/4242132801 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:37:58.608 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.610+0000 7f90e8b56640 1 --2- 172.21.15.83:0/4242132801 >> v2:172.21.15.167:3300/0 conn(0x7f90e40b2a20 0x7f90e40b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:58.609 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.610+0000 7f90e3fff640 1 --2- 172.21.15.83:0/4242132801 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90e40cbe80 0x7f90e406cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:58.609 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.610+0000 7f90dbfff640 1 -- 172.21.15.83:0/4242132801 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90e40cbe80 msgr2=0x7f90e406cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:37:58.609 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.610+0000 7f90dbfff640 1 --2- 172.21.15.83:0/4242132801 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90e40cbe80 0x7f90e406cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:58.609 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.610+0000 7f90dbfff640 1 -- 172.21.15.83:0/4242132801 >> v2:172.21.15.167:3300/0 conn(0x7f90e40b2a20 msgr2=0x7f90e40b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:37:58.609 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.610+0000 7f90dbfff640 1 --2- 172.21.15.83:0/4242132801 >> v2:172.21.15.167:3300/0 conn(0x7f90e40b2a20 0x7f90e40b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:58.609 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.610+0000 7f90dbfff640 1 -- 172.21.15.83:0/4242132801 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e406c360 con 0x7f90e40b25c0 2024-04-17T10:37:58.610 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.610+0000 7f90dbfff640 1 --2- 172.21.15.83:0/4242132801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90e40b25c0 0x7f90e40b4050 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f90e4159a30 tx=0x7f90cc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f977c7446b2f7a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:37:58.610 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.610+0000 7f90e37fe640 1 -- 172.21.15.83:0/4242132801 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc0ab640 con 0x7f90e40b25c0 2024-04-17T10:37:58.610 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.610+0000 7f90e37fe640 1 -- 172.21.15.83:0/4242132801 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90cc0b0070 con 0x7f90e40b25c0 2024-04-17T10:37:58.610 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.610+0000 7f90e37fe640 1 -- 172.21.15.83:0/4242132801 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc0aba00 con 0x7f90e40b25c0 2024-04-17T10:37:58.610 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90ea5b9640 1 -- 172.21.15.83:0/4242132801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90e40b25c0 msgr2=0x7f90e40b4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:58.610 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90ea5b9640 1 --2- 172.21.15.83:0/4242132801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90e40b25c0 0x7f90e40b4050 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f90e4159a30 tx=0x7f90cc01c460 comp rx=0 tx=0).stop 2024-04-17T10:37:58.611 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90ea5b9640 1 -- 172.21.15.83:0/4242132801 shutdown_connections 2024-04-17T10:37:58.611 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90ea5b9640 1 --2- 172.21.15.83:0/4242132801 >> v2:172.21.15.167:3300/0 conn(0x7f90e40b2a20 0x7f90e40b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:58.611 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90ea5b9640 1 --2- 172.21.15.83:0/4242132801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90e40b25c0 0x7f90e40b4050 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:58.611 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90ea5b9640 1 --2- 172.21.15.83:0/4242132801 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90e40cbe80 0x7f90e406cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:58.611 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90ea5b9640 1 -- 172.21.15.83:0/4242132801 >> 172.21.15.83:0/4242132801 conn(0x7f90e406abf0 msgr2=0x7f90e406b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:37:58.611 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90ea5b9640 1 -- 172.21.15.83:0/4242132801 shutdown_connections 2024-04-17T10:37:58.611 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90ea5b9640 1 -- 172.21.15.83:0/4242132801 wait complete. 2024-04-17T10:37:58.613 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90ea5b9640 1 Processor -- start 2024-04-17T10:37:58.614 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90ea5b9640 1 -- start start 2024-04-17T10:37:58.614 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90ea5b9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90e40b25c0 0x7f90e4150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:58.614 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90ea5b9640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90e40b2a20 0x7f90e415b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:58.614 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90ea5b9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90e40cbe80 0x7f90e4159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:58.614 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90ea5b9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e415c230 con 0x7f90e40b25c0 2024-04-17T10:37:58.614 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90ea5b9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e415c0b0 con 0x7f90e40cbe80 2024-04-17T10:37:58.614 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90ea5b9640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e415d810 con 0x7f90e40b2a20 2024-04-17T10:37:58.614 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90e3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90e40b25c0 0x7f90e4150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:58.614 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90e3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90e40b25c0 0x7f90e4150c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33922/0 (socket says 172.21.15.83:33922) 2024-04-17T10:37:58.614 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90e3fff640 1 -- 172.21.15.83:0/2714156910 learned_addr learned my addr 172.21.15.83:0/2714156910 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:37:58.614 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90e8b56640 1 --2- 172.21.15.83:0/2714156910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90e40cbe80 0x7f90e4159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:58.614 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90dbfff640 1 --2- 172.21.15.83:0/2714156910 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90e40b2a20 0x7f90e415b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:58.615 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90e3fff640 1 -- 172.21.15.83:0/2714156910 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90e40b2a20 msgr2=0x7f90e415b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:58.615 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90e3fff640 1 --2- 172.21.15.83:0/2714156910 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90e40b2a20 0x7f90e415b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:58.615 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90e3fff640 1 -- 172.21.15.83:0/2714156910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90e40cbe80 msgr2=0x7f90e4159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:58.615 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90e3fff640 1 --2- 172.21.15.83:0/2714156910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90e40cbe80 0x7f90e4159720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:58.615 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90e3fff640 1 -- 172.21.15.83:0/2714156910 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e415aa90 con 0x7f90e40b25c0 2024-04-17T10:37:58.615 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90e8b56640 1 --2- 172.21.15.83:0/2714156910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90e40cbe80 0x7f90e4159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:37:58.615 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90e3fff640 1 --2- 172.21.15.83:0/2714156910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90e40b25c0 0x7f90e4150c10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f90d400c7f0 tx=0x7f90d400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:37:58.615 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90e17fa640 1 -- 172.21.15.83:0/2714156910 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d4015070 con 0x7f90e40b25c0 2024-04-17T10:37:58.615 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90e17fa640 1 -- 172.21.15.83:0/2714156910 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d4004220 con 0x7f90e40b25c0 2024-04-17T10:37:58.615 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90ea5b9640 1 -- 172.21.15.83:0/2714156910 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e4159c60 con 0x7f90e40b25c0 2024-04-17T10:37:58.615 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90e17fa640 1 -- 172.21.15.83:0/2714156910 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d4003620 con 0x7f90e40b25c0 2024-04-17T10:37:58.615 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.614+0000 7f90ea5b9640 1 -- 172.21.15.83:0/2714156910 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e4161410 con 0x7f90e40b25c0 2024-04-17T10:37:58.615 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.618+0000 7f90ea5b9640 1 -- 172.21.15.83:0/2714156910 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e404ef30 con 0x7f90e40b25c0 2024-04-17T10:37:58.616 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.618+0000 7f90e17fa640 1 -- 172.21.15.83:0/2714156910 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 4) v1 ==== 86671+0+0 (secure 0 0 0) 0x7f90d40039d0 con 0x7f90e40b25c0 2024-04-17T10:37:58.616 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.618+0000 7f90e17fa640 1 --2- 172.21.15.83:0/2714156910 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f90c4066ed0 0x7f90c4069390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:37:58.616 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.618+0000 7f90e17fa640 1 -- 172.21.15.83:0/2714156910 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f90d40840b0 con 0x7f90e40b25c0 2024-04-17T10:37:58.616 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.618+0000 7f90dbfff640 1 --2- 172.21.15.83:0/2714156910 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f90c4066ed0 0x7f90c4069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:37:58.616 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.618+0000 7f90dbfff640 1 --2- 172.21.15.83:0/2714156910 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f90c4066ed0 0x7f90c4069390 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f90cc0af000 tx=0x7f90cc0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:37:58.619 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.622+0000 7f90e17fa640 1 -- 172.21.15.83:0/2714156910 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f90e415aa90 con 0x7f90e40b25c0 2024-04-17T10:37:58.776 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.778+0000 7f90ea5b9640 1 -- 172.21.15.83:0/2714156910 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90e4159e90 con 0x7f90e40b25c0 2024-04-17T10:37:58.777 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.778+0000 7f90e17fa640 1 -- 172.21.15.83:0/2714156910 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f90d4055f30 con 0x7f90e40b25c0 2024-04-17T10:37:58.778 INFO:teuthology.misc.health.smithi083.stdout: 2024-04-17T10:37:58.778 INFO:teuthology.misc.health.smithi083.stdout:{"epoch":12,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:37:56.341466+0000","last_up_change":"0.000000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:37:58.780 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.782+0000 7f90ea5b9640 1 -- 172.21.15.83:0/2714156910 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f90c4066ed0 msgr2=0x7f90c4069390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:58.780 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.782+0000 7f90ea5b9640 1 --2- 172.21.15.83:0/2714156910 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f90c4066ed0 0x7f90c4069390 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f90cc0af000 tx=0x7f90cc0a7000 comp rx=0 tx=0).stop 2024-04-17T10:37:58.780 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.782+0000 7f90ea5b9640 1 -- 172.21.15.83:0/2714156910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90e40b25c0 msgr2=0x7f90e4150c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:37:58.780 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.782+0000 7f90ea5b9640 1 --2- 172.21.15.83:0/2714156910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90e40b25c0 0x7f90e4150c10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f90d400c7f0 tx=0x7f90d400ccb0 comp rx=0 tx=0).stop 2024-04-17T10:37:58.780 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.782+0000 7f90ea5b9640 1 -- 172.21.15.83:0/2714156910 shutdown_connections 2024-04-17T10:37:58.780 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.782+0000 7f90ea5b9640 1 --2- 172.21.15.83:0/2714156910 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f90c4066ed0 0x7f90c4069390 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:58.780 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.782+0000 7f90ea5b9640 1 --2- 172.21.15.83:0/2714156910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90e40cbe80 0x7f90e4159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:58.780 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.782+0000 7f90ea5b9640 1 --2- 172.21.15.83:0/2714156910 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90e40b2a20 0x7f90e415b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:58.780 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.782+0000 7f90ea5b9640 1 --2- 172.21.15.83:0/2714156910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90e40b25c0 0x7f90e4150c10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:37:58.780 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.782+0000 7f90ea5b9640 1 -- 172.21.15.83:0/2714156910 >> 172.21.15.83:0/2714156910 conn(0x7f90e406abf0 msgr2=0x7f90e40cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:37:58.780 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.782+0000 7f90ea5b9640 1 -- 172.21.15.83:0/2714156910 shutdown_connections 2024-04-17T10:37:58.781 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:37:58.782+0000 7f90ea5b9640 1 -- 172.21.15.83:0/2714156910 wait complete. 2024-04-17T10:37:58.795 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-17T10:38:05.603 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-17T10:38:05.726 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.729+0000 7f313f51b640 1 Processor -- start 2024-04-17T10:38:05.726 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.729+0000 7f313f51b640 1 -- start start 2024-04-17T10:38:05.727 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.733+0000 7f313f51b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f313815dcc0 0x7f31380d2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:05.728 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.733+0000 7f313f51b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31380cea20 con 0x7f313806b6d0 2024-04-17T10:38:05.728 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.733+0000 7f313f51b640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31380ce6e0 con 0x7f31380ce220 2024-04-17T10:38:05.728 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.733+0000 7f313f51b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31380cf280 con 0x7f313815dcc0 2024-04-17T10:38:05.728 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.733+0000 7f313d2b7640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f313815dcc0 0x7f31380d2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:05.728 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.733+0000 7f313dab8640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f313806b6d0 0x7f31380b7160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38880/0 (socket says 172.21.15.83:38880) 2024-04-17T10:38:05.728 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.733+0000 7f313dab8640 1 -- 172.21.15.83:0/109946162 learned_addr learned my addr 172.21.15.83:0/109946162 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:05.728 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.733+0000 7f313d2b7640 1 -- 172.21.15.83:0/109946162 >> v1:172.21.15.167:6789/0 conn(0x7f31380ce220 legacy=0x7f313806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:05.728 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.733+0000 7f313d2b7640 1 -- 172.21.15.83:0/109946162 >> v1:172.21.15.83:6789/0 conn(0x7f313806b6d0 legacy=0x7f31380b7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:05.728 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.733+0000 7f312bfff640 1 -- 172.21.15.83:0/109946162 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2554415474 0 0) 0x7f31380cea20 con 0x7f313806b6d0 2024-04-17T10:38:05.728 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.733+0000 7f313d2b7640 1 -- 172.21.15.83:0/109946162 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31380cecc0 con 0x7f313815dcc0 2024-04-17T10:38:05.730 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.733+0000 7f313d2b7640 1 --2- 172.21.15.83:0/109946162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f313815dcc0 0x7f31380d2fb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3130009d90 tx=0x7f313001f170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af681a30f9ae06 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:05.730 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.733+0000 7f312bfff640 1 -- 172.21.15.83:0/109946162 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31300b4070 con 0x7f313815dcc0 2024-04-17T10:38:05.730 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.733+0000 7f312bfff640 1 -- 172.21.15.83:0/109946162 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31300af600 con 0x7f313815dcc0 2024-04-17T10:38:05.730 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.733+0000 7f312bfff640 1 -- 172.21.15.83:0/109946162 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31300af9d0 con 0x7f313815dcc0 2024-04-17T10:38:05.730 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:38:05.732 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f313f51b640 1 -- 172.21.15.83:0/109946162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f313815dcc0 msgr2=0x7f31380d2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:05.732 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f313f51b640 1 --2- 172.21.15.83:0/109946162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f313815dcc0 0x7f31380d2fb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3130009d90 tx=0x7f313001f170 comp rx=0 tx=0).stop 2024-04-17T10:38:05.732 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f313f51b640 1 -- 172.21.15.83:0/109946162 shutdown_connections 2024-04-17T10:38:05.732 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f313f51b640 1 --2- 172.21.15.83:0/109946162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f313815dcc0 0x7f31380d2fb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:05.732 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f313f51b640 1 -- 172.21.15.83:0/109946162 >> 172.21.15.83:0/109946162 conn(0x7f31380d39c0 msgr2=0x7f31380d3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:05.732 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f313f51b640 1 -- 172.21.15.83:0/109946162 shutdown_connections 2024-04-17T10:38:05.732 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f313f51b640 1 -- 172.21.15.83:0/109946162 wait complete. 2024-04-17T10:38:05.733 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f313f51b640 1 Processor -- start 2024-04-17T10:38:05.733 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f313f51b640 1 -- start start 2024-04-17T10:38:05.733 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:38:05.735 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f313f51b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f313806b6d0 0x7f31380c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:05.735 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f313f51b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31380ce220 0x7f31381f3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:05.735 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f313f51b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f313815dcc0 0x7f31380d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:05.735 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f313f51b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313815e0a0 con 0x7f313815dcc0 2024-04-17T10:38:05.735 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f313f51b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31380d4820 con 0x7f313806b6d0 2024-04-17T10:38:05.736 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f313f51b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31380b6bd0 con 0x7f31380ce220 2024-04-17T10:38:05.736 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f313dab8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f313815dcc0 0x7f31380d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:05.736 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f313dab8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f313815dcc0 0x7f31380d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33976/0 (socket says 172.21.15.83:33976) 2024-04-17T10:38:05.736 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f313dab8640 1 -- 172.21.15.83:0/728529814 learned_addr learned my addr 172.21.15.83:0/728529814 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:05.736 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f313cab6640 1 --2- 172.21.15.83:0/728529814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31380ce220 0x7f31381f3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:05.736 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f313d2b7640 1 --2- 172.21.15.83:0/728529814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f313806b6d0 0x7f31380c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:05.736 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f313dab8640 1 -- 172.21.15.83:0/728529814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31380ce220 msgr2=0x7f31381f3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:05.736 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f313dab8640 1 --2- 172.21.15.83:0/728529814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31380ce220 0x7f31381f3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:05.736 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f313dab8640 1 -- 172.21.15.83:0/728529814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f313806b6d0 msgr2=0x7f31380c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:05.736 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f313dab8640 1 --2- 172.21.15.83:0/728529814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f313806b6d0 0x7f31380c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:05.736 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f313dab8640 1 -- 172.21.15.83:0/728529814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31380c7590 con 0x7f313815dcc0 2024-04-17T10:38:05.736 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f313cab6640 1 --2- 172.21.15.83:0/728529814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31380ce220 0x7f31381f3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:38:05.736 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f313d2b7640 1 --2- 172.21.15.83:0/728529814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f313806b6d0 0x7f31380c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:38:05.737 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f313dab8640 1 --2- 172.21.15.83:0/728529814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f313815dcc0 0x7f31380d14d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f313400c440 tx=0x7f313400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:05.737 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f3129ffb640 1 -- 172.21.15.83:0/728529814 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31340074e0 con 0x7f313815dcc0 2024-04-17T10:38:05.737 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f3129ffb640 1 -- 172.21.15.83:0/728529814 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31340047f0 con 0x7f313815dcc0 2024-04-17T10:38:05.737 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f313f51b640 1 -- 172.21.15.83:0/728529814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31380c7870 con 0x7f313815dcc0 2024-04-17T10:38:05.737 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f3129ffb640 1 -- 172.21.15.83:0/728529814 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3134007680 con 0x7f313815dcc0 2024-04-17T10:38:05.737 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.737+0000 7f313f51b640 1 -- 172.21.15.83:0/728529814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31380c7e30 con 0x7f313815dcc0 2024-04-17T10:38:05.737 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.741+0000 7f313f51b640 1 -- 172.21.15.83:0/728529814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f313804ef30 con 0x7f313815dcc0 2024-04-17T10:38:05.737 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.741+0000 7f3129ffb640 1 -- 172.21.15.83:0/728529814 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3134011070 con 0x7f313815dcc0 2024-04-17T10:38:05.738 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.741+0000 7f3129ffb640 1 --2- 172.21.15.83:0/728529814 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f311c066f60 0x7f311c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:05.738 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.741+0000 7f313d2b7640 1 --2- 172.21.15.83:0/728529814 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f311c066f60 0x7f311c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:05.738 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.741+0000 7f3129ffb640 1 -- 172.21.15.83:0/728529814 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3134084730 con 0x7f313815dcc0 2024-04-17T10:38:05.738 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.741+0000 7f313d2b7640 1 --2- 172.21.15.83:0/728529814 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f311c066f60 0x7f311c069420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f31300b1000 tx=0x7f31300aa000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:05.749 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.753+0000 7f3129ffb640 1 -- 172.21.15.83:0/728529814 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3134055b20 con 0x7f313815dcc0 2024-04-17T10:38:05.906 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.913+0000 7f313f51b640 1 -- 172.21.15.83:0/728529814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31380b8450 con 0x7f313815dcc0 2024-04-17T10:38:05.908 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.913+0000 7f3129ffb640 1 -- 172.21.15.83:0/728529814 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12812 (secure 0 0 0) 0x7f3134055420 con 0x7f313815dcc0 2024-04-17T10:38:05.909 INFO:teuthology.misc.health.smithi083.stdout: 2024-04-17T10:38:05.909 INFO:teuthology.misc.health.smithi083.stdout:{"epoch":16,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:38:02.578015+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:38:05.911 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.917+0000 7f313f51b640 1 -- 172.21.15.83:0/728529814 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f311c066f60 msgr2=0x7f311c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:05.911 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.917+0000 7f313f51b640 1 --2- 172.21.15.83:0/728529814 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f311c066f60 0x7f311c069420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f31300b1000 tx=0x7f31300aa000 comp rx=0 tx=0).stop 2024-04-17T10:38:05.911 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.917+0000 7f313f51b640 1 -- 172.21.15.83:0/728529814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f313815dcc0 msgr2=0x7f31380d14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:05.911 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.917+0000 7f313f51b640 1 --2- 172.21.15.83:0/728529814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f313815dcc0 0x7f31380d14d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f313400c440 tx=0x7f313400c900 comp rx=0 tx=0).stop 2024-04-17T10:38:05.912 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.917+0000 7f313f51b640 1 -- 172.21.15.83:0/728529814 shutdown_connections 2024-04-17T10:38:05.912 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.917+0000 7f313f51b640 1 --2- 172.21.15.83:0/728529814 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f311c066f60 0x7f311c069420 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:05.912 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.917+0000 7f313f51b640 1 --2- 172.21.15.83:0/728529814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f313815dcc0 0x7f31380d14d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:05.912 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.917+0000 7f313f51b640 1 --2- 172.21.15.83:0/728529814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31380ce220 0x7f31381f3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:05.912 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.917+0000 7f313f51b640 1 --2- 172.21.15.83:0/728529814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f313806b6d0 0x7f31380c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:05.912 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.917+0000 7f313f51b640 1 -- 172.21.15.83:0/728529814 >> 172.21.15.83:0/728529814 conn(0x7f31380d39c0 msgr2=0x7f313815eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:05.912 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.917+0000 7f313f51b640 1 -- 172.21.15.83:0/728529814 shutdown_connections 2024-04-17T10:38:05.912 INFO:teuthology.misc.health.smithi083.stderr:2024-04-17T10:38:05.917+0000 7f313f51b640 1 -- 172.21.15.83:0/728529814 wait complete. 2024-04-17T10:38:05.927 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-17T10:38:05.927 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-17T10:38:05.927 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-17T10:38:05.928 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-17T10:38:05.930 INFO:tasks.ceph.mds.a:Started 2024-04-17T10:38:05.930 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-17T10:38:05.930 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-17T10:38:05.932 INFO:tasks.ceph.mds.c:Started 2024-04-17T10:38:05.932 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-17T10:38:05.933 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-17T10:38:05.935 INFO:tasks.ceph.mds.b:Started 2024-04-17T10:38:05.935 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-17T10:38:05.935 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-17T10:38:05.938 INFO:tasks.ceph.mds.d:Started 2024-04-17T10:38:05.938 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-17T10:38:05.941 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:38:06.000 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-04-17T10:38:06.000 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-04-17T10:38:06.020 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2024-04-17T10:38:06.026 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2024-04-17T10:38:06.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.053+0000 7f889bab1640 1 Processor -- start 2024-04-17T10:38:06.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.053+0000 7f889bab1640 1 -- start start 2024-04-17T10:38:06.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.053+0000 7f889bab1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f88940ceca0 0x7f88940ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:06.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.053+0000 7f889bab1640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f88940ce620 0x7f88940cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:06.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.053+0000 7f889bab1640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f889415f6c0 con 0x7f889415de90 2024-04-17T10:38:06.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.053+0000 7f889bab1640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f889406bd90 con 0x7f88940ceca0 2024-04-17T10:38:06.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.053+0000 7f889bab1640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f889406c030 con 0x7f88940ce620 2024-04-17T10:38:06.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.053+0000 7f889904c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f88940ceca0 0x7f88940ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:06.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.053+0000 7f889904c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f88940ceca0 0x7f88940ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34002/0 (socket says 172.21.15.83:34002) 2024-04-17T10:38:06.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.053+0000 7f889904c640 1 -- 172.21.15.83:0/2024120193 learned_addr learned my addr 172.21.15.83:0/2024120193 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:06.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.053+0000 7f889a04e640 1 --2- 172.21.15.83:0/2024120193 >> v2:172.21.15.167:3300/0 conn(0x7f88940ce620 0x7f88940cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:06.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.053+0000 7f889904c640 1 -- 172.21.15.83:0/2024120193 >> v2:172.21.15.167:3300/0 conn(0x7f88940ce620 msgr2=0x7f88940cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:06.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.053+0000 7f889904c640 1 --2- 172.21.15.83:0/2024120193 >> v2:172.21.15.167:3300/0 conn(0x7f88940ce620 0x7f88940cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:06.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.053+0000 7f889904c640 1 -- 172.21.15.83:0/2024120193 >> v1:172.21.15.167:6789/0 conn(0x7f889415de90 legacy=0x7f889415efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:06.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.053+0000 7f889904c640 1 -- 172.21.15.83:0/2024120193 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f889406c310 con 0x7f88940ceca0 2024-04-17T10:38:06.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.053+0000 7f889a04e640 1 --2- 172.21.15.83:0/2024120193 >> v2:172.21.15.167:3300/0 conn(0x7f88940ce620 0x7f88940cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:06.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.057+0000 7f889904c640 1 --2- 172.21.15.83:0/2024120193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88940ceca0 0x7f88940ce0e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8894072220 tx=0x7f889001c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f1bb3d9eb0f27d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:06.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.057+0000 7f889884b640 1 -- 172.21.15.83:0/2024120193 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88900b0070 con 0x7f88940ceca0 2024-04-17T10:38:06.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.057+0000 7f889884b640 1 -- 172.21.15.83:0/2024120193 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88900ab600 con 0x7f88940ceca0 2024-04-17T10:38:06.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.057+0000 7f889884b640 1 -- 172.21.15.83:0/2024120193 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88900ab9d0 con 0x7f88940ceca0 2024-04-17T10:38:06.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.057+0000 7f889bab1640 1 -- 172.21.15.83:0/2024120193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88940ceca0 msgr2=0x7f88940ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:06.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.057+0000 7f889bab1640 1 --2- 172.21.15.83:0/2024120193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88940ceca0 0x7f88940ce0e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8894072220 tx=0x7f889001c300 comp rx=0 tx=0).stop 2024-04-17T10:38:06.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.057+0000 7f889bab1640 1 -- 172.21.15.83:0/2024120193 shutdown_connections 2024-04-17T10:38:06.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.057+0000 7f889bab1640 1 --2- 172.21.15.83:0/2024120193 >> v2:172.21.15.167:3300/0 conn(0x7f88940ce620 0x7f88940cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:06.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.057+0000 7f889bab1640 1 --2- 172.21.15.83:0/2024120193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88940ceca0 0x7f88940ce0e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:06.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.057+0000 7f889bab1640 1 -- 172.21.15.83:0/2024120193 >> 172.21.15.83:0/2024120193 conn(0x7f88940d17b0 msgr2=0x7f88940d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:06.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.057+0000 7f889bab1640 1 -- 172.21.15.83:0/2024120193 shutdown_connections 2024-04-17T10:38:06.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.057+0000 7f889bab1640 1 -- 172.21.15.83:0/2024120193 wait complete. 2024-04-17T10:38:06.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.057+0000 7f889bab1640 1 Processor -- start 2024-04-17T10:38:06.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.057+0000 7f889bab1640 1 -- start start 2024-04-17T10:38:06.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.061+0000 7f889bab1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88940ce620 0x7f88940c52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:06.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.061+0000 7f889bab1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88940ceca0 0x7f88940c7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:06.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.061+0000 7f889bab1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f889415de90 0x7f889414d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:06.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.061+0000 7f889bab1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889415e3f0 con 0x7f889415de90 2024-04-17T10:38:06.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.061+0000 7f889bab1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889415e270 con 0x7f88940ce620 2024-04-17T10:38:06.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.061+0000 7f889bab1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88940d0620 con 0x7f88940ceca0 2024-04-17T10:38:06.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.061+0000 7f889a04e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f889415de90 0x7f889414d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:06.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.061+0000 7f889a04e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f889415de90 0x7f889414d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34006/0 (socket says 172.21.15.83:34006) 2024-04-17T10:38:06.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.061+0000 7f889a04e640 1 -- 172.21.15.83:0/176152063 learned_addr learned my addr 172.21.15.83:0/176152063 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:06.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.061+0000 7f889904c640 1 --2- 172.21.15.83:0/176152063 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88940ceca0 0x7f88940c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:06.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.061+0000 7f889984d640 1 --2- 172.21.15.83:0/176152063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88940ce620 0x7f88940c52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:06.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.061+0000 7f889a04e640 1 -- 172.21.15.83:0/176152063 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88940ceca0 msgr2=0x7f88940c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:06.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.061+0000 7f889a04e640 1 --2- 172.21.15.83:0/176152063 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88940ceca0 0x7f88940c7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:06.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.061+0000 7f889a04e640 1 -- 172.21.15.83:0/176152063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88940ce620 msgr2=0x7f88940c52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:06.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.061+0000 7f889a04e640 1 --2- 172.21.15.83:0/176152063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88940ce620 0x7f88940c52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:06.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.061+0000 7f889a04e640 1 -- 172.21.15.83:0/176152063 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88940c59a0 con 0x7f889415de90 2024-04-17T10:38:06.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.061+0000 7f889a04e640 1 --2- 172.21.15.83:0/176152063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f889415de90 0x7f889414d840 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f888c00cef0 tx=0x7f888c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:06.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.061+0000 7f888a7fc640 1 -- 172.21.15.83:0/176152063 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c01a070 con 0x7f889415de90 2024-04-17T10:38:06.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.061+0000 7f888a7fc640 1 -- 172.21.15.83:0/176152063 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f888c00d860 con 0x7f889415de90 2024-04-17T10:38:06.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.061+0000 7f889bab1640 1 -- 172.21.15.83:0/176152063 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88940c5c80 con 0x7f889415de90 2024-04-17T10:38:06.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.061+0000 7f888a7fc640 1 -- 172.21.15.83:0/176152063 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c00dc50 con 0x7f889415de90 2024-04-17T10:38:06.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.061+0000 7f889bab1640 1 -- 172.21.15.83:0/176152063 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88940c6240 con 0x7f889415de90 2024-04-17T10:38:06.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.061+0000 7f889bab1640 1 -- 172.21.15.83:0/176152063 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8894074f90 con 0x7f889415de90 2024-04-17T10:38:06.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.061+0000 7f888a7fc640 1 -- 172.21.15.83:0/176152063 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f888c01d070 con 0x7f889415de90 2024-04-17T10:38:06.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.061+0000 7f888a7fc640 1 --2- 172.21.15.83:0/176152063 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8878066f60 0x7f8878069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:06.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.061+0000 7f889984d640 1 --2- 172.21.15.83:0/176152063 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8878066f60 0x7f8878069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:06.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.061+0000 7f888a7fc640 1 -- 172.21.15.83:0/176152063 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f888c0854e0 con 0x7f889415de90 2024-04-17T10:38:06.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.061+0000 7f889984d640 1 --2- 172.21.15.83:0/176152063 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8878066f60 0x7f8878069420 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f88840098e0 tx=0x7f8884009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:06.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.065+0000 7f888a7fc640 1 -- 172.21.15.83:0/176152063 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f888c0562b0 con 0x7f889415de90 2024-04-17T10:38:06.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.225+0000 7f889bab1640 1 -- 172.21.15.83:0/176152063 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88940cba40 con 0x7f889415de90 2024-04-17T10:38:06.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.225+0000 7f888a7fc640 1 -- 172.21.15.83:0/176152063 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12812 (secure 0 0 0) 0x7f888c05a350 con 0x7f889415de90 2024-04-17T10:38:06.221 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:06.221 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":16,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:38:02.578015+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:38:06.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.229+0000 7f889bab1640 1 -- 172.21.15.83:0/176152063 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8878066f60 msgr2=0x7f8878069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:06.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.229+0000 7f889bab1640 1 --2- 172.21.15.83:0/176152063 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8878066f60 0x7f8878069420 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f88840098e0 tx=0x7f8884009060 comp rx=0 tx=0).stop 2024-04-17T10:38:06.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.229+0000 7f889bab1640 1 -- 172.21.15.83:0/176152063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f889415de90 msgr2=0x7f889414d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:06.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.229+0000 7f889bab1640 1 --2- 172.21.15.83:0/176152063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f889415de90 0x7f889414d840 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f888c00cef0 tx=0x7f888c00a710 comp rx=0 tx=0).stop 2024-04-17T10:38:06.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.229+0000 7f889bab1640 1 -- 172.21.15.83:0/176152063 shutdown_connections 2024-04-17T10:38:06.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.229+0000 7f889bab1640 1 --2- 172.21.15.83:0/176152063 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8878066f60 0x7f8878069420 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:06.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.229+0000 7f889bab1640 1 --2- 172.21.15.83:0/176152063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f889415de90 0x7f889414d840 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:06.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.229+0000 7f889bab1640 1 --2- 172.21.15.83:0/176152063 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88940ceca0 0x7f88940c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:06.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.229+0000 7f889bab1640 1 --2- 172.21.15.83:0/176152063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88940ce620 0x7f88940c52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:06.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.229+0000 7f889bab1640 1 -- 172.21.15.83:0/176152063 >> 172.21.15.83:0/176152063 conn(0x7f88940d17b0 msgr2=0x7f88940d1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:06.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.229+0000 7f889bab1640 1 -- 172.21.15.83:0/176152063 shutdown_connections 2024-04-17T10:38:06.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.229+0000 7f889bab1640 1 -- 172.21.15.83:0/176152063 wait complete. 2024-04-17T10:38:06.239 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:38:06.239 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:38:06.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.365+0000 7fa186f80640 1 Processor -- start 2024-04-17T10:38:06.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.369+0000 7fa186f80640 1 -- start start 2024-04-17T10:38:06.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.369+0000 7fa186f80640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa18006ae10 0x7fa18006b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:06.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.369+0000 7fa186f80640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1801625d0 con 0x7fa180070aa0 2024-04-17T10:38:06.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.369+0000 7fa186f80640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa18006b730 con 0x7fa18015e330 2024-04-17T10:38:06.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.369+0000 7fa186f80640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa18015eeb0 con 0x7fa18006ae10 2024-04-17T10:38:06.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.369+0000 7fa18551d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa180070aa0 0x7fa180070f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38882/0 (socket says 172.21.15.83:38882) 2024-04-17T10:38:06.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.369+0000 7fa18551d640 1 -- 172.21.15.83:0/2472836897 learned_addr learned my addr 172.21.15.83:0/2472836897 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:06.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.369+0000 7fa184d1c640 1 --2- 172.21.15.83:0/2472836897 >> v2:172.21.15.167:3300/0 conn(0x7fa18006ae10 0x7fa18006b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:06.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.369+0000 7fa17f7fe640 1 -- 172.21.15.83:0/2472836897 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2456995063 0 0) 0x7fa1801625d0 con 0x7fa180070aa0 2024-04-17T10:38:06.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.369+0000 7fa17f7fe640 1 -- 172.21.15.83:0/2472836897 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa168003530 con 0x7fa180070aa0 2024-04-17T10:38:06.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.369+0000 7fa17f7fe640 1 -- 172.21.15.83:0/2472836897 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4246136301 0 0) 0x7fa168003530 con 0x7fa180070aa0 2024-04-17T10:38:06.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.369+0000 7fa17f7fe640 1 -- 172.21.15.83:0/2472836897 >> v2:172.21.15.167:3300/0 conn(0x7fa18006ae10 msgr2=0x7fa18006b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:06.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.369+0000 7fa17f7fe640 1 --2- 172.21.15.83:0/2472836897 >> v2:172.21.15.167:3300/0 conn(0x7fa18006ae10 0x7fa18006b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:06.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.369+0000 7fa17f7fe640 1 -- 172.21.15.83:0/2472836897 >> v1:172.21.15.167:6789/0 conn(0x7fa18015e330 legacy=0x7fa18015e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:06.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.369+0000 7fa17f7fe640 1 -- 172.21.15.83:0/2472836897 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa180161810 con 0x7fa180070aa0 2024-04-17T10:38:06.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.369+0000 7fa17f7fe640 1 -- 172.21.15.83:0/2472836897 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fa178001a00 con 0x7fa180070aa0 2024-04-17T10:38:06.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.369+0000 7fa17f7fe640 1 -- 172.21.15.83:0/2472836897 >> v1:172.21.15.83:6789/0 conn(0x7fa180070aa0 legacy=0x7fa180070f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:06.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.369+0000 7fa17f7fe640 1 --2- 172.21.15.83:0/2472836897 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa168003a30 0x7fa168003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:06.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.369+0000 7fa17f7fe640 1 --2- 172.21.15.83:0/2472836897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa168004810 0x7fa168004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:06.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.369+0000 7fa17f7fe640 1 --2- 172.21.15.83:0/2472836897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa168005240 0x7fa16800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:06.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.369+0000 7fa17f7fe640 1 -- 172.21.15.83:0/2472836897 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16800c170 con 0x7fa168004810 2024-04-17T10:38:06.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.369+0000 7fa17f7fe640 1 -- 172.21.15.83:0/2472836897 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16800c320 con 0x7fa168005240 2024-04-17T10:38:06.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.369+0000 7fa17f7fe640 1 -- 172.21.15.83:0/2472836897 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16800c520 con 0x7fa168003a30 2024-04-17T10:38:06.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.369+0000 7fa17ffff640 1 --2- 172.21.15.83:0/2472836897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa168004810 0x7fa168004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:06.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.369+0000 7fa184d1c640 1 --2- 172.21.15.83:0/2472836897 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa168003a30 0x7fa168003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:06.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.369+0000 7fa18551d640 1 --2- 172.21.15.83:0/2472836897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa168005240 0x7fa16800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:06.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.369+0000 7fa17ffff640 1 -- 172.21.15.83:0/2472836897 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa168003a30 msgr2=0x7fa168003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:06.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.369+0000 7fa17ffff640 1 --2- 172.21.15.83:0/2472836897 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa168003a30 0x7fa168003e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:06.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.369+0000 7fa17ffff640 1 -- 172.21.15.83:0/2472836897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa168005240 msgr2=0x7fa16800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:06.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.369+0000 7fa17ffff640 1 --2- 172.21.15.83:0/2472836897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa168005240 0x7fa16800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:06.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.369+0000 7fa18551d640 1 -- 172.21.15.83:0/2472836897 reap_dead start 2024-04-17T10:38:06.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.369+0000 7fa17ffff640 1 -- 172.21.15.83:0/2472836897 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa168003530 con 0x7fa168004810 2024-04-17T10:38:06.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa17ffff640 1 --2- 172.21.15.83:0/2472836897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa168004810 0x7fa168004c90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa174001780 tx=0x7fa17401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b996b441c4f712 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:06.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa17f7fe640 1 -- 172.21.15.83:0/2472836897 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1740013a0 con 0x7fa168004810 2024-04-17T10:38:06.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa17f7fe640 1 -- 172.21.15.83:0/2472836897 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1740052f0 con 0x7fa168004810 2024-04-17T10:38:06.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa17f7fe640 1 -- 172.21.15.83:0/2472836897 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174003840 con 0x7fa168004810 2024-04-17T10:38:06.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa186f80640 1 -- 172.21.15.83:0/2472836897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa168004810 msgr2=0x7fa168004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:06.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa186f80640 1 --2- 172.21.15.83:0/2472836897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa168004810 0x7fa168004c90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa174001780 tx=0x7fa17401c400 comp rx=0 tx=0).stop 2024-04-17T10:38:06.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa186f80640 1 -- 172.21.15.83:0/2472836897 shutdown_connections 2024-04-17T10:38:06.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa186f80640 1 --2- 172.21.15.83:0/2472836897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa168004810 0x7fa168004c90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:06.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa186f80640 1 -- 172.21.15.83:0/2472836897 >> 172.21.15.83:0/2472836897 conn(0x7fa1800b15b0 msgr2=0x7fa1800b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:06.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa186f80640 1 -- 172.21.15.83:0/2472836897 shutdown_connections 2024-04-17T10:38:06.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa186f80640 1 -- 172.21.15.83:0/2472836897 wait complete. 2024-04-17T10:38:06.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa186f80640 1 Processor -- start 2024-04-17T10:38:06.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa186f80640 1 -- start start 2024-04-17T10:38:06.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa186f80640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa168004810 0x7fa180160c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:06.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa186f80640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa180152080 0x7fa180154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:06.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa186f80640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa180152550 0x7fa18014e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:06.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa186f80640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1800cf470 con 0x7fa180152080 2024-04-17T10:38:06.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa186f80640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1800cf2f0 con 0x7fa168004810 2024-04-17T10:38:06.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa186f80640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa18015eb50 con 0x7fa180152550 2024-04-17T10:38:06.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa17ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa180152080 0x7fa180154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:06.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa17ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa180152080 0x7fa180154ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34016/0 (socket says 172.21.15.83:34016) 2024-04-17T10:38:06.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa17ffff640 1 -- 172.21.15.83:0/1808419738 learned_addr learned my addr 172.21.15.83:0/1808419738 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:06.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa18551d640 1 --2- 172.21.15.83:0/1808419738 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa180152550 0x7fa18014e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:06.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa184d1c640 1 --2- 172.21.15.83:0/1808419738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa168004810 0x7fa180160c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:06.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa17ffff640 1 -- 172.21.15.83:0/1808419738 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa180152550 msgr2=0x7fa18014e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:06.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa17ffff640 1 --2- 172.21.15.83:0/1808419738 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa180152550 0x7fa18014e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:06.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa17ffff640 1 -- 172.21.15.83:0/1808419738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa168004810 msgr2=0x7fa180160c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:06.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa17ffff640 1 --2- 172.21.15.83:0/1808419738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa168004810 0x7fa180160c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:06.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa17ffff640 1 -- 172.21.15.83:0/1808419738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18015fbc0 con 0x7fa180152080 2024-04-17T10:38:06.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa18551d640 1 --2- 172.21.15.83:0/1808419738 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa180152550 0x7fa18014e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:06.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa184d1c640 1 --2- 172.21.15.83:0/1808419738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa168004810 0x7fa180160c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:06.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa17ffff640 1 --2- 172.21.15.83:0/1808419738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa180152080 0x7fa180154ec0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa174005040 tx=0x7fa168005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:06.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa17d7fa640 1 -- 172.21.15.83:0/1808419738 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174007a40 con 0x7fa180152080 2024-04-17T10:38:06.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa17d7fa640 1 -- 172.21.15.83:0/1808419738 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1740ba070 con 0x7fa180152080 2024-04-17T10:38:06.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa186f80640 1 -- 172.21.15.83:0/1808419738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa18015fea0 con 0x7fa180152080 2024-04-17T10:38:06.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa17d7fa640 1 -- 172.21.15.83:0/1808419738 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1740054a0 con 0x7fa180152080 2024-04-17T10:38:06.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.373+0000 7fa186f80640 1 -- 172.21.15.83:0/1808419738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa180160460 con 0x7fa180152080 2024-04-17T10:38:06.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.377+0000 7fa186f80640 1 -- 172.21.15.83:0/1808419738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa180074f90 con 0x7fa180152080 2024-04-17T10:38:06.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.377+0000 7fa17d7fa640 1 -- 172.21.15.83:0/1808419738 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa174007030 con 0x7fa180152080 2024-04-17T10:38:06.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.377+0000 7fa17d7fa640 1 --2- 172.21.15.83:0/1808419738 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa16c066f60 0x7fa16c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:06.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.377+0000 7fa17d7fa640 1 -- 172.21.15.83:0/1808419738 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa17411f7e0 con 0x7fa180152080 2024-04-17T10:38:06.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.377+0000 7fa184d1c640 1 --2- 172.21.15.83:0/1808419738 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa16c066f60 0x7fa16c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:06.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.377+0000 7fa184d1c640 1 --2- 172.21.15.83:0/1808419738 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa16c066f60 0x7fa16c069420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa170009a80 tx=0x7fa170008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:06.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.381+0000 7fa17d7fa640 1 -- 172.21.15.83:0/1808419738 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa174016110 con 0x7fa180152080 2024-04-17T10:38:06.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.525+0000 7fa186f80640 1 -- 172.21.15.83:0/1808419738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa180072740 con 0x7fa180152080 2024-04-17T10:38:06.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.529+0000 7fa17d7fa640 1 -- 172.21.15.83:0/1808419738 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa1740f0dd0 con 0x7fa180152080 2024-04-17T10:38:06.523 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:38:06.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.529+0000 7fa186f80640 1 -- 172.21.15.83:0/1808419738 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa16c066f60 msgr2=0x7fa16c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:06.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.529+0000 7fa186f80640 1 --2- 172.21.15.83:0/1808419738 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa16c066f60 0x7fa16c069420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa170009a80 tx=0x7fa170008040 comp rx=0 tx=0).stop 2024-04-17T10:38:06.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.529+0000 7fa186f80640 1 -- 172.21.15.83:0/1808419738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa180152080 msgr2=0x7fa180154ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:06.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.529+0000 7fa186f80640 1 --2- 172.21.15.83:0/1808419738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa180152080 0x7fa180154ec0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa174005040 tx=0x7fa168005840 comp rx=0 tx=0).stop 2024-04-17T10:38:06.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.529+0000 7fa186f80640 1 -- 172.21.15.83:0/1808419738 shutdown_connections 2024-04-17T10:38:06.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.529+0000 7fa186f80640 1 --2- 172.21.15.83:0/1808419738 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa16c066f60 0x7fa16c069420 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:06.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.529+0000 7fa186f80640 1 --2- 172.21.15.83:0/1808419738 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa180152550 0x7fa18014e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:06.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.529+0000 7fa186f80640 1 --2- 172.21.15.83:0/1808419738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa180152080 0x7fa180154ec0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:06.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.529+0000 7fa186f80640 1 --2- 172.21.15.83:0/1808419738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa168004810 0x7fa180160c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:06.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.529+0000 7fa186f80640 1 -- 172.21.15.83:0/1808419738 >> 172.21.15.83:0/1808419738 conn(0x7fa1800b15b0 msgr2=0x7fa1800716e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:06.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.529+0000 7fa186f80640 1 -- 172.21.15.83:0/1808419738 shutdown_connections 2024-04-17T10:38:06.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.529+0000 7fa186f80640 1 -- 172.21.15.83:0/1808419738 wait complete. 2024-04-17T10:38:06.540 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:38:06.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.649+0000 7fa177885640 1 Processor -- start 2024-04-17T10:38:06.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.649+0000 7fa177885640 1 -- start start 2024-04-17T10:38:06.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.649+0000 7fa177885640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa1700b4270 0x7fa1700b4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:06.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.649+0000 7fa177885640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa17015e480 0x7fa17015e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:06.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.649+0000 7fa177885640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa17015f1b0 con 0x7fa1700afd00 2024-04-17T10:38:06.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.649+0000 7fa177885640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa17006dce0 con 0x7fa1700b4270 2024-04-17T10:38:06.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.649+0000 7fa177885640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa17006dfc0 con 0x7fa17015e480 2024-04-17T10:38:06.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.649+0000 7fa175621640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa1700afd00 0x7fa17006e7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38896/0 (socket says 172.21.15.83:38896) 2024-04-17T10:38:06.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.649+0000 7fa175621640 1 -- 172.21.15.83:0/2366731698 learned_addr learned my addr 172.21.15.83:0/2366731698 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:06.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.649+0000 7fa167fff640 1 -- 172.21.15.83:0/2366731698 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2092159378 0 0) 0x7fa17015f1b0 con 0x7fa1700afd00 2024-04-17T10:38:06.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.649+0000 7fa167fff640 1 -- 172.21.15.83:0/2366731698 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa15c003530 con 0x7fa1700afd00 2024-04-17T10:38:06.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.649+0000 7fa174e20640 1 --2- 172.21.15.83:0/2366731698 >> v2:172.21.15.167:3300/0 conn(0x7fa1700b4270 0x7fa1700b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:06.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.649+0000 7fa175e22640 1 --2- 172.21.15.83:0/2366731698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa17015e480 0x7fa17015e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:06.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.653+0000 7fa167fff640 1 -- 172.21.15.83:0/2366731698 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1774371738 0 0) 0x7fa15c003530 con 0x7fa1700afd00 2024-04-17T10:38:06.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.653+0000 7fa167fff640 1 -- 172.21.15.83:0/2366731698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa17015e480 msgr2=0x7fa17015e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:06.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.653+0000 7fa167fff640 1 --2- 172.21.15.83:0/2366731698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa17015e480 0x7fa17015e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:06.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.653+0000 7fa167fff640 1 -- 172.21.15.83:0/2366731698 >> v2:172.21.15.167:3300/0 conn(0x7fa1700b4270 msgr2=0x7fa1700b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:06.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.653+0000 7fa167fff640 1 --2- 172.21.15.83:0/2366731698 >> v2:172.21.15.167:3300/0 conn(0x7fa1700b4270 0x7fa1700b4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:06.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.653+0000 7fa167fff640 1 -- 172.21.15.83:0/2366731698 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17006e2a0 con 0x7fa1700afd00 2024-04-17T10:38:06.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.653+0000 7fa167fff640 1 -- 172.21.15.83:0/2366731698 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fa168001680 con 0x7fa1700afd00 2024-04-17T10:38:06.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.653+0000 7fa167fff640 1 -- 172.21.15.83:0/2366731698 >> v1:172.21.15.83:6789/0 conn(0x7fa1700afd00 legacy=0x7fa17006e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:06.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.653+0000 7fa167fff640 1 --2- 172.21.15.83:0/2366731698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa17015e480 0x7fa15c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:06.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.653+0000 7fa167fff640 1 --2- 172.21.15.83:0/2366731698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa15c003ff0 0x7fa15c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:06.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.653+0000 7fa167fff640 1 --2- 172.21.15.83:0/2366731698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa15c004d50 0x7fa15c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:06.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.653+0000 7fa167fff640 1 -- 172.21.15.83:0/2366731698 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17006dce0 con 0x7fa15c004d50 2024-04-17T10:38:06.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.653+0000 7fa167fff640 1 -- 172.21.15.83:0/2366731698 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17006dfc0 con 0x7fa15c003ff0 2024-04-17T10:38:06.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.653+0000 7fa167fff640 1 -- 172.21.15.83:0/2366731698 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c00bad0 con 0x7fa17015e480 2024-04-17T10:38:06.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.653+0000 7fa175e22640 1 --2- 172.21.15.83:0/2366731698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa15c004d50 0x7fa15c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:06.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.653+0000 7fa174e20640 1 --2- 172.21.15.83:0/2366731698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa15c003ff0 0x7fa15c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:06.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.653+0000 7fa175621640 1 --2- 172.21.15.83:0/2366731698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa17015e480 0x7fa15c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:06.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.653+0000 7fa175e22640 1 -- 172.21.15.83:0/2366731698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa17015e480 msgr2=0x7fa15c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:06.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.653+0000 7fa175e22640 1 --2- 172.21.15.83:0/2366731698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa17015e480 0x7fa15c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:06.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.653+0000 7fa175e22640 1 -- 172.21.15.83:0/2366731698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa15c003ff0 msgr2=0x7fa15c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:06.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.653+0000 7fa175e22640 1 --2- 172.21.15.83:0/2366731698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa15c003ff0 0x7fa15c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:06.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.653+0000 7fa175e22640 1 -- 172.21.15.83:0/2366731698 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15c003530 con 0x7fa15c004d50 2024-04-17T10:38:06.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.653+0000 7fa175621640 1 --2- 172.21.15.83:0/2366731698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa17015e480 0x7fa15c0039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:38:06.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.653+0000 7fa175e22640 1 --2- 172.21.15.83:0/2366731698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa15c004d50 0x7fa15c0051d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa16c007360 tx=0x7fa16c01ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4bcfc8e54054ea server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:06.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.653+0000 7fa167fff640 1 -- 172.21.15.83:0/2366731698 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c0b2070 con 0x7fa15c004d50 2024-04-17T10:38:06.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.653+0000 7fa167fff640 1 -- 172.21.15.83:0/2366731698 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa16c0a5d40 con 0x7fa15c004d50 2024-04-17T10:38:06.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.653+0000 7fa167fff640 1 -- 172.21.15.83:0/2366731698 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c009be0 con 0x7fa15c004d50 2024-04-17T10:38:06.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.657+0000 7fa177885640 1 -- 172.21.15.83:0/2366731698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa15c004d50 msgr2=0x7fa15c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:06.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.657+0000 7fa177885640 1 --2- 172.21.15.83:0/2366731698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa15c004d50 0x7fa15c0051d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa16c007360 tx=0x7fa16c01ce40 comp rx=0 tx=0).stop 2024-04-17T10:38:06.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.657+0000 7fa175e22640 1 -- 172.21.15.83:0/2366731698 reap_dead start 2024-04-17T10:38:06.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.657+0000 7fa177885640 1 -- 172.21.15.83:0/2366731698 shutdown_connections 2024-04-17T10:38:06.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.657+0000 7fa177885640 1 -- 172.21.15.83:0/2366731698 >> 172.21.15.83:0/2366731698 conn(0x7fa17006bcf0 msgr2=0x7fa17006c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:06.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.657+0000 7fa177885640 1 -- 172.21.15.83:0/2366731698 shutdown_connections 2024-04-17T10:38:06.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.657+0000 7fa177885640 1 -- 172.21.15.83:0/2366731698 wait complete. 2024-04-17T10:38:06.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.657+0000 7fa177885640 1 Processor -- start 2024-04-17T10:38:06.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.657+0000 7fa177885640 1 -- start start 2024-04-17T10:38:06.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.657+0000 7fa177885640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa15c004d50 0x7fa1700a9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:06.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.657+0000 7fa177885640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa170149040 0x7fa1701f1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:06.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.657+0000 7fa177885640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1701494c0 0x7fa1701f1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:06.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.657+0000 7fa177885640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17015ea40 con 0x7fa1701494c0 2024-04-17T10:38:06.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.657+0000 7fa177885640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17015e8c0 con 0x7fa15c004d50 2024-04-17T10:38:06.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.657+0000 7fa177885640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa17015ebc0 con 0x7fa170149040 2024-04-17T10:38:06.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.657+0000 7fa175e22640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1701494c0 0x7fa1701f1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:06.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.657+0000 7fa175e22640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1701494c0 0x7fa1701f1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34040/0 (socket says 172.21.15.83:34040) 2024-04-17T10:38:06.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.657+0000 7fa175e22640 1 -- 172.21.15.83:0/4186888977 learned_addr learned my addr 172.21.15.83:0/4186888977 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:06.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.657+0000 7fa175e22640 1 -- 172.21.15.83:0/4186888977 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa170149040 msgr2=0x7fa1701f1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:06.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.657+0000 7fa175e22640 1 --2- 172.21.15.83:0/4186888977 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa170149040 0x7fa1701f1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:06.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.657+0000 7fa175e22640 1 -- 172.21.15.83:0/4186888977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa15c004d50 msgr2=0x7fa1700a9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:06.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.657+0000 7fa175e22640 1 --2- 172.21.15.83:0/4186888977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa15c004d50 0x7fa1700a9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:06.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.657+0000 7fa175e22640 1 -- 172.21.15.83:0/4186888977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa170155350 con 0x7fa1701494c0 2024-04-17T10:38:06.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.657+0000 7fa175e22640 1 --2- 172.21.15.83:0/4186888977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1701494c0 0x7fa1701f1a70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0b0040 tx=0x7fa15c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:06.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.657+0000 7fa165ffb640 1 -- 172.21.15.83:0/4186888977 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c0b2040 con 0x7fa1701494c0 2024-04-17T10:38:06.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.657+0000 7fa165ffb640 1 -- 172.21.15.83:0/4186888977 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa16c0a5b40 con 0x7fa1701494c0 2024-04-17T10:38:06.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.657+0000 7fa177885640 1 -- 172.21.15.83:0/4186888977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1701542b0 con 0x7fa1701494c0 2024-04-17T10:38:06.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.657+0000 7fa165ffb640 1 -- 172.21.15.83:0/4186888977 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c009be0 con 0x7fa1701494c0 2024-04-17T10:38:06.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.657+0000 7fa177885640 1 -- 172.21.15.83:0/4186888977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa170154870 con 0x7fa1701494c0 2024-04-17T10:38:06.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.657+0000 7fa177885640 1 -- 172.21.15.83:0/4186888977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa170074f90 con 0x7fa1701494c0 2024-04-17T10:38:06.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.661+0000 7fa165ffb640 1 -- 172.21.15.83:0/4186888977 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa16c0bf060 con 0x7fa1701494c0 2024-04-17T10:38:06.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.661+0000 7fa165ffb640 1 --2- 172.21.15.83:0/4186888977 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa154066f60 0x7fa154069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:06.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.661+0000 7fa165ffb640 1 -- 172.21.15.83:0/4186888977 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa16c11ece0 con 0x7fa1701494c0 2024-04-17T10:38:06.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.661+0000 7fa175621640 1 --2- 172.21.15.83:0/4186888977 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa154066f60 0x7fa154069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:06.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.661+0000 7fa175621640 1 --2- 172.21.15.83:0/4186888977 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa154066f60 0x7fa154069420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa168001ec0 tx=0x7fa168001e50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:06.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.665+0000 7fa165ffb640 1 -- 172.21.15.83:0/4186888977 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa16c016b50 con 0x7fa1701494c0 2024-04-17T10:38:06.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.841+0000 7fa177885640 1 -- 172.21.15.83:0/4186888977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa17006d5e0 con 0x7fa1701494c0 2024-04-17T10:38:06.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.841+0000 7fa165ffb640 1 -- 172.21.15.83:0/4186888977 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7fa16c0effb0 con 0x7fa1701494c0 2024-04-17T10:38:06.837 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:06.837 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1635176834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1635176834},{"type":"v1","addr":"172.21.15.83:6837","nonce":1635176834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-04-17T10:38:06.837 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 2 2024-04-17T10:38:06.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.841+0000 7fa177885640 1 -- 172.21.15.83:0/4186888977 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa154066f60 msgr2=0x7fa154069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:06.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.841+0000 7fa177885640 1 --2- 172.21.15.83:0/4186888977 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa154066f60 0x7fa154069420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa168001ec0 tx=0x7fa168001e50 comp rx=0 tx=0).stop 2024-04-17T10:38:06.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.841+0000 7fa177885640 1 -- 172.21.15.83:0/4186888977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1701494c0 msgr2=0x7fa1701f1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:06.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.841+0000 7fa177885640 1 --2- 172.21.15.83:0/4186888977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1701494c0 0x7fa1701f1a70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0b0040 tx=0x7fa15c005840 comp rx=0 tx=0).stop 2024-04-17T10:38:06.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.845+0000 7fa177885640 1 -- 172.21.15.83:0/4186888977 shutdown_connections 2024-04-17T10:38:06.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.845+0000 7fa177885640 1 --2- 172.21.15.83:0/4186888977 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa154066f60 0x7fa154069420 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:06.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.845+0000 7fa177885640 1 --2- 172.21.15.83:0/4186888977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1701494c0 0x7fa1701f1a70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:06.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.845+0000 7fa177885640 1 --2- 172.21.15.83:0/4186888977 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa170149040 0x7fa1701f1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:06.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.845+0000 7fa177885640 1 --2- 172.21.15.83:0/4186888977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa15c004d50 0x7fa1700a9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:06.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.845+0000 7fa177885640 1 -- 172.21.15.83:0/4186888977 >> 172.21.15.83:0/4186888977 conn(0x7fa17006bcf0 msgr2=0x7fa1700b4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:06.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.845+0000 7fa177885640 1 -- 172.21.15.83:0/4186888977 shutdown_connections 2024-04-17T10:38:06.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:06.845+0000 7fa177885640 1 -- 172.21.15.83:0/4186888977 wait complete. 2024-04-17T10:38:08.854 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:38:08.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.989+0000 7f5d3bea8640 1 Processor -- start 2024-04-17T10:38:08.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.989+0000 7f5d3bea8640 1 -- start start 2024-04-17T10:38:08.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d3bea8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d3415db50 0x7f5d34070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:08.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d3bea8640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5d340aedd0 0x7f5d3406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:08.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d3bea8640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d340709b0 con 0x7f5d3406dae0 2024-04-17T10:38:08.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d3bea8640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d3415e440 con 0x7f5d340aedd0 2024-04-17T10:38:08.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d3bea8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3415e720 con 0x7f5d3415db50 2024-04-17T10:38:08.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d3a445640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5d3406dae0 0x7f5d340cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42224/0 (socket says 172.21.15.83:42224) 2024-04-17T10:38:08.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d3a445640 1 -- 172.21.15.83:0/4033460454 learned_addr learned my addr 172.21.15.83:0/4033460454 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d39c44640 1 --2- 172.21.15.83:0/4033460454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d3415db50 0x7f5d34070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d39443640 1 --2- 172.21.15.83:0/4033460454 >> v2:172.21.15.167:3300/0 conn(0x7f5d340aedd0 0x7f5d3406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d38c42640 1 -- 172.21.15.83:0/4033460454 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2804114981 0 0) 0x7f5d2c0014a0 con 0x7f5d3406dae0 2024-04-17T10:38:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d38c42640 1 -- 172.21.15.83:0/4033460454 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d20003530 con 0x7f5d3406dae0 2024-04-17T10:38:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d38c42640 1 -- 172.21.15.83:0/4033460454 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2080977354 0 0) 0x7f5d20003530 con 0x7f5d3406dae0 2024-04-17T10:38:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d38c42640 1 -- 172.21.15.83:0/4033460454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d3415db50 msgr2=0x7f5d34070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d38c42640 1 --2- 172.21.15.83:0/4033460454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d3415db50 0x7f5d34070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d38c42640 1 -- 172.21.15.83:0/4033460454 >> v2:172.21.15.167:3300/0 conn(0x7f5d340aedd0 msgr2=0x7f5d3406d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d38c42640 1 --2- 172.21.15.83:0/4033460454 >> v2:172.21.15.167:3300/0 conn(0x7f5d340aedd0 0x7f5d3406d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d38c42640 1 -- 172.21.15.83:0/4033460454 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3415ea00 con 0x7f5d3406dae0 2024-04-17T10:38:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d38c42640 1 -- 172.21.15.83:0/4033460454 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f5d2c001bd0 con 0x7f5d3406dae0 2024-04-17T10:38:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d38c42640 1 -- 172.21.15.83:0/4033460454 >> v1:172.21.15.83:6789/0 conn(0x7f5d3406dae0 legacy=0x7f5d340cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d38c42640 1 --2- 172.21.15.83:0/4033460454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d200039c0 0x7f5d20003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d38c42640 1 --2- 172.21.15.83:0/4033460454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d3415db50 0x7f5d20004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d38c42640 1 --2- 172.21.15.83:0/4033460454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d20004d50 0x7f5d200051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d38c42640 1 -- 172.21.15.83:0/4033460454 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3415e440 con 0x7f5d200039c0 2024-04-17T10:38:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d38c42640 1 -- 172.21.15.83:0/4033460454 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3415e720 con 0x7f5d20004d50 2024-04-17T10:38:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d38c42640 1 -- 172.21.15.83:0/4033460454 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2000b9d0 con 0x7f5d3415db50 2024-04-17T10:38:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d39c44640 1 --2- 172.21.15.83:0/4033460454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d200039c0 0x7f5d20003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d3a445640 1 --2- 172.21.15.83:0/4033460454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d20004d50 0x7f5d200051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d39443640 1 --2- 172.21.15.83:0/4033460454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d3415db50 0x7f5d20004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d39c44640 1 -- 172.21.15.83:0/4033460454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d3415db50 msgr2=0x7f5d20004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d39c44640 1 --2- 172.21.15.83:0/4033460454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d3415db50 0x7f5d20004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d39c44640 1 -- 172.21.15.83:0/4033460454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d20004d50 msgr2=0x7f5d200051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d39c44640 1 --2- 172.21.15.83:0/4033460454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d20004d50 0x7f5d200051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:08.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.993+0000 7f5d39c44640 1 -- 172.21.15.83:0/4033460454 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d20003530 con 0x7f5d200039c0 2024-04-17T10:38:08.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.997+0000 7f5d39c44640 1 --2- 172.21.15.83:0/4033460454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d200039c0 0x7f5d20003de0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5d30009ae0 tx=0x7f5d3001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a5d1e5b058befd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:08.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.997+0000 7f5d38c42640 1 -- 172.21.15.83:0/4033460454 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d300b2070 con 0x7f5d200039c0 2024-04-17T10:38:08.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.997+0000 7f5d38c42640 1 -- 172.21.15.83:0/4033460454 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d300a5530 con 0x7f5d200039c0 2024-04-17T10:38:08.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.997+0000 7f5d38c42640 1 -- 172.21.15.83:0/4033460454 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d300096b0 con 0x7f5d200039c0 2024-04-17T10:38:08.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.997+0000 7f5d3bea8640 1 -- 172.21.15.83:0/4033460454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d200039c0 msgr2=0x7f5d20003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:08.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.997+0000 7f5d3bea8640 1 --2- 172.21.15.83:0/4033460454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d200039c0 0x7f5d20003de0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5d30009ae0 tx=0x7f5d3001ca70 comp rx=0 tx=0).stop 2024-04-17T10:38:08.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.997+0000 7f5d3a445640 1 -- 172.21.15.83:0/4033460454 reap_dead start 2024-04-17T10:38:08.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.997+0000 7f5d3bea8640 1 -- 172.21.15.83:0/4033460454 shutdown_connections 2024-04-17T10:38:08.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.997+0000 7f5d3bea8640 1 -- 172.21.15.83:0/4033460454 >> 172.21.15.83:0/4033460454 conn(0x7f5d3406abe0 msgr2=0x7f5d3406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:08.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.997+0000 7f5d3bea8640 1 -- 172.21.15.83:0/4033460454 shutdown_connections 2024-04-17T10:38:08.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.997+0000 7f5d3bea8640 1 -- 172.21.15.83:0/4033460454 wait complete. 2024-04-17T10:38:08.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.997+0000 7f5d3bea8640 1 Processor -- start 2024-04-17T10:38:08.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.997+0000 7f5d3bea8640 1 -- start start 2024-04-17T10:38:08.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.997+0000 7f5d3bea8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d200039c0 0x7f5d341523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:08.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.997+0000 7f5d3bea8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d340cd9a0 0x7f5d340cde00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:08.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.997+0000 7f5d3bea8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d34157bd0 0x7f5d34158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:08.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.997+0000 7f5d3bea8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d341657d0 con 0x7f5d340cd9a0 2024-04-17T10:38:08.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.997+0000 7f5d3bea8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d34165650 con 0x7f5d200039c0 2024-04-17T10:38:08.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.997+0000 7f5d3bea8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d34165950 con 0x7f5d34157bd0 2024-04-17T10:38:08.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.997+0000 7f5d39443640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d340cd9a0 0x7f5d340cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:08.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.997+0000 7f5d39443640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d340cd9a0 0x7f5d340cde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55726/0 (socket says 172.21.15.83:55726) 2024-04-17T10:38:08.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.997+0000 7f5d39443640 1 -- 172.21.15.83:0/3422952482 learned_addr learned my addr 172.21.15.83:0/3422952482 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:08.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.997+0000 7f5d3a445640 1 --2- 172.21.15.83:0/3422952482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d34157bd0 0x7f5d34158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:08.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.997+0000 7f5d39c44640 1 --2- 172.21.15.83:0/3422952482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d200039c0 0x7f5d341523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:08.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.997+0000 7f5d39443640 1 -- 172.21.15.83:0/3422952482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d34157bd0 msgr2=0x7f5d34158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:08.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.997+0000 7f5d39443640 1 --2- 172.21.15.83:0/3422952482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d34157bd0 0x7f5d34158050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:08.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.997+0000 7f5d39443640 1 -- 172.21.15.83:0/3422952482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d200039c0 msgr2=0x7f5d341523c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:08.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.997+0000 7f5d39443640 1 --2- 172.21.15.83:0/3422952482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d200039c0 0x7f5d341523c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:08.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.997+0000 7f5d39443640 1 -- 172.21.15.83:0/3422952482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d34152900 con 0x7f5d340cd9a0 2024-04-17T10:38:08.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.997+0000 7f5d39c44640 1 --2- 172.21.15.83:0/3422952482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d200039c0 0x7f5d341523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:08.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:08.997+0000 7f5d3a445640 1 --2- 172.21.15.83:0/3422952482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d34157bd0 0x7f5d34158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:08.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.001+0000 7f5d39443640 1 --2- 172.21.15.83:0/3422952482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d340cd9a0 0x7f5d340cde00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5d24010960 tx=0x7f5d24010e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:08.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.001+0000 7f5d2a7fc640 1 -- 172.21.15.83:0/3422952482 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2400f040 con 0x7f5d340cd9a0 2024-04-17T10:38:08.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.001+0000 7f5d2a7fc640 1 -- 172.21.15.83:0/3422952482 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d2400ec20 con 0x7f5d340cd9a0 2024-04-17T10:38:08.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.001+0000 7f5d3bea8640 1 -- 172.21.15.83:0/3422952482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d34159cb0 con 0x7f5d340cd9a0 2024-04-17T10:38:08.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.001+0000 7f5d2a7fc640 1 -- 172.21.15.83:0/3422952482 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2401c400 con 0x7f5d340cd9a0 2024-04-17T10:38:08.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.001+0000 7f5d3bea8640 1 -- 172.21.15.83:0/3422952482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d3415a270 con 0x7f5d340cd9a0 2024-04-17T10:38:08.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.001+0000 7f5d3bea8640 1 -- 172.21.15.83:0/3422952482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d34074f90 con 0x7f5d340cd9a0 2024-04-17T10:38:08.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.001+0000 7f5d2a7fc640 1 -- 172.21.15.83:0/3422952482 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5d2401c5a0 con 0x7f5d340cd9a0 2024-04-17T10:38:08.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.001+0000 7f5d2a7fc640 1 --2- 172.21.15.83:0/3422952482 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5d18066f60 0x7f5d18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:08.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.001+0000 7f5d2a7fc640 1 -- 172.21.15.83:0/3422952482 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5d24085470 con 0x7f5d340cd9a0 2024-04-17T10:38:08.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.001+0000 7f5d39c44640 1 --2- 172.21.15.83:0/3422952482 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5d18066f60 0x7f5d18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:08.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.001+0000 7f5d39c44640 1 --2- 172.21.15.83:0/3422952482 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5d18066f60 0x7f5d18069420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5d300af000 tx=0x7f5d300046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:09.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.005+0000 7f5d2a7fc640 1 -- 172.21.15.83:0/3422952482 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5d24018030 con 0x7f5d340cd9a0 2024-04-17T10:38:09.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.201+0000 7f5d3bea8640 1 -- 172.21.15.83:0/3422952482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d340b2af0 con 0x7f5d340cd9a0 2024-04-17T10:38:09.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.201+0000 7f5d2a7fc640 1 -- 172.21.15.83:0/3422952482 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3700 (secure 0 0 0) 0x7f5d24056740 con 0x7f5d340cd9a0 2024-04-17T10:38:09.195 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:09.195 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2884469968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2884469968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2884469968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2919571009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2919571009},{"type":"v1","addr":"172.21.15.83:6836","nonce":2919571009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1635176834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1635176834},{"type":"v1","addr":"172.21.15.83:6837","nonce":1635176834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3871588351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3871588351},{"type":"v1","addr":"172.21.15.167:6835","nonce":3871588351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-04-17T10:38:09.196 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 3 2024-04-17T10:38:09.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.201+0000 7f5d3bea8640 1 -- 172.21.15.83:0/3422952482 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5d18066f60 msgr2=0x7f5d18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:09.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.201+0000 7f5d3bea8640 1 --2- 172.21.15.83:0/3422952482 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5d18066f60 0x7f5d18069420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5d300af000 tx=0x7f5d300046d0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.201+0000 7f5d3bea8640 1 -- 172.21.15.83:0/3422952482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d340cd9a0 msgr2=0x7f5d340cde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:09.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.201+0000 7f5d3bea8640 1 --2- 172.21.15.83:0/3422952482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d340cd9a0 0x7f5d340cde00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5d24010960 tx=0x7f5d24010e20 comp rx=0 tx=0).stop 2024-04-17T10:38:09.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.201+0000 7f5d3bea8640 1 -- 172.21.15.83:0/3422952482 shutdown_connections 2024-04-17T10:38:09.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.201+0000 7f5d3bea8640 1 --2- 172.21.15.83:0/3422952482 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5d18066f60 0x7f5d18069420 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.201+0000 7f5d3bea8640 1 --2- 172.21.15.83:0/3422952482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d34157bd0 0x7f5d34158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.201+0000 7f5d3bea8640 1 --2- 172.21.15.83:0/3422952482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d340cd9a0 0x7f5d340cde00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.201+0000 7f5d3bea8640 1 --2- 172.21.15.83:0/3422952482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d200039c0 0x7f5d341523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.201+0000 7f5d3bea8640 1 -- 172.21.15.83:0/3422952482 >> 172.21.15.83:0/3422952482 conn(0x7f5d3406abe0 msgr2=0x7f5d340ccdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:09.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.201+0000 7f5d3bea8640 1 -- 172.21.15.83:0/3422952482 shutdown_connections 2024-04-17T10:38:09.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.201+0000 7f5d3bea8640 1 -- 172.21.15.83:0/3422952482 wait complete. 2024-04-17T10:38:09.214 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:38:09.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.337+0000 7ff5454ba640 1 Processor -- start 2024-04-17T10:38:09.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.337+0000 7ff5454ba640 1 -- start start 2024-04-17T10:38:09.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.341+0000 7ff5454ba640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff54015ba30 0x7ff540072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:09.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.341+0000 7ff5454ba640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff54015c960 0x7ff5400cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:09.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.341+0000 7ff5454ba640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff5400cfa10 0x7ff54006c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:09.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.341+0000 7ff5454ba640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff540071ea0 con 0x7ff5400cfa10 2024-04-17T10:38:09.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.341+0000 7ff5454ba640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5400cfea0 con 0x7ff54015c960 2024-04-17T10:38:09.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.341+0000 7ff5454ba640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff54006e320 con 0x7ff54015ba30 2024-04-17T10:38:09.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.341+0000 7ff53f7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff5400cfa10 0x7ff54006c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:09.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.341+0000 7ff53f7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff5400cfa10 0x7ff54006c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55728/0 (socket says 172.21.15.83:55728) 2024-04-17T10:38:09.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.341+0000 7ff53f7fe640 1 -- 172.21.15.83:0/2029273982 learned_addr learned my addr 172.21.15.83:0/2029273982 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:09.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.341+0000 7ff53effd640 1 --2- 172.21.15.83:0/2029273982 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff54015ba30 0x7ff540072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:09.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.341+0000 7ff53e7fc640 1 --2- 172.21.15.83:0/2029273982 >> v2:172.21.15.167:3300/0 conn(0x7ff54015c960 0x7ff5400cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:09.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.341+0000 7ff53f7fe640 1 -- 172.21.15.83:0/2029273982 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff54015ba30 msgr2=0x7ff540072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:09.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.341+0000 7ff53f7fe640 1 --2- 172.21.15.83:0/2029273982 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff54015ba30 0x7ff540072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.341+0000 7ff53f7fe640 1 -- 172.21.15.83:0/2029273982 >> v2:172.21.15.167:3300/0 conn(0x7ff54015c960 msgr2=0x7ff5400cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:09.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.341+0000 7ff53f7fe640 1 --2- 172.21.15.83:0/2029273982 >> v2:172.21.15.167:3300/0 conn(0x7ff54015c960 0x7ff5400cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.341+0000 7ff53f7fe640 1 -- 172.21.15.83:0/2029273982 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54006e040 con 0x7ff5400cfa10 2024-04-17T10:38:09.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.341+0000 7ff53effd640 1 --2- 172.21.15.83:0/2029273982 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff54015ba30 0x7ff540072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:38:09.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.341+0000 7ff53f7fe640 1 --2- 172.21.15.83:0/2029273982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5400cfa10 0x7ff54006c490 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff540070060 tx=0x7ff53801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344355d19061bbf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:09.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.341+0000 7ff53dffb640 1 -- 172.21.15.83:0/2029273982 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5380a56d0 con 0x7ff5400cfa10 2024-04-17T10:38:09.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.341+0000 7ff53dffb640 1 -- 172.21.15.83:0/2029273982 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5380a5870 con 0x7ff5400cfa10 2024-04-17T10:38:09.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.341+0000 7ff53dffb640 1 -- 172.21.15.83:0/2029273982 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5380a5c40 con 0x7ff5400cfa10 2024-04-17T10:38:09.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.341+0000 7ff5454ba640 1 -- 172.21.15.83:0/2029273982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5400cfa10 msgr2=0x7ff54006c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:09.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.341+0000 7ff5454ba640 1 --2- 172.21.15.83:0/2029273982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5400cfa10 0x7ff54006c490 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff540070060 tx=0x7ff53801c960 comp rx=0 tx=0).stop 2024-04-17T10:38:09.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.341+0000 7ff5454ba640 1 -- 172.21.15.83:0/2029273982 shutdown_connections 2024-04-17T10:38:09.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.341+0000 7ff5454ba640 1 --2- 172.21.15.83:0/2029273982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5400cfa10 0x7ff54006c490 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.341+0000 7ff5454ba640 1 --2- 172.21.15.83:0/2029273982 >> v2:172.21.15.167:3300/0 conn(0x7ff54015c960 0x7ff5400cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.341+0000 7ff5454ba640 1 --2- 172.21.15.83:0/2029273982 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff54015ba30 0x7ff540072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.341+0000 7ff5454ba640 1 -- 172.21.15.83:0/2029273982 >> 172.21.15.83:0/2029273982 conn(0x7ff54006abe0 msgr2=0x7ff54006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:09.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.341+0000 7ff5454ba640 1 -- 172.21.15.83:0/2029273982 shutdown_connections 2024-04-17T10:38:09.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.341+0000 7ff5454ba640 1 -- 172.21.15.83:0/2029273982 wait complete. 2024-04-17T10:38:09.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.345+0000 7ff5454ba640 1 Processor -- start 2024-04-17T10:38:09.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.345+0000 7ff5454ba640 1 -- start start 2024-04-17T10:38:09.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.345+0000 7ff5454ba640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5400cfa10 0x7ff540153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:09.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.345+0000 7ff5454ba640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff54015ba30 0x7ff5401485d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:09.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.345+0000 7ff5454ba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54015c960 0x7ff540155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:09.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.345+0000 7ff5454ba640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54006de20 con 0x7ff54015c960 2024-04-17T10:38:09.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.345+0000 7ff5454ba640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54006dca0 con 0x7ff54015ba30 2024-04-17T10:38:09.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.345+0000 7ff5454ba640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff54015cd40 con 0x7ff5400cfa10 2024-04-17T10:38:09.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.345+0000 7ff53f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54015c960 0x7ff540155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:09.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.345+0000 7ff53f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54015c960 0x7ff540155550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55736/0 (socket says 172.21.15.83:55736) 2024-04-17T10:38:09.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.345+0000 7ff53f7fe640 1 -- 172.21.15.83:0/1030571861 learned_addr learned my addr 172.21.15.83:0/1030571861 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:09.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.345+0000 7ff53effd640 1 --2- 172.21.15.83:0/1030571861 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5400cfa10 0x7ff540153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:09.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.345+0000 7ff53e7fc640 1 --2- 172.21.15.83:0/1030571861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff54015ba30 0x7ff5401485d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:09.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.345+0000 7ff53f7fe640 1 -- 172.21.15.83:0/1030571861 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5400cfa10 msgr2=0x7ff540153d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:09.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.345+0000 7ff53f7fe640 1 --2- 172.21.15.83:0/1030571861 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5400cfa10 0x7ff540153d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.345+0000 7ff53f7fe640 1 -- 172.21.15.83:0/1030571861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff54015ba30 msgr2=0x7ff5401485d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:09.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.345+0000 7ff53f7fe640 1 --2- 172.21.15.83:0/1030571861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff54015ba30 0x7ff5401485d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.345+0000 7ff53f7fe640 1 -- 172.21.15.83:0/1030571861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff540154290 con 0x7ff54015c960 2024-04-17T10:38:09.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.345+0000 7ff53f7fe640 1 --2- 172.21.15.83:0/1030571861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54015c960 0x7ff540155550 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff53801b010 tx=0x7ff538019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:09.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.345+0000 7ff5277fe640 1 -- 172.21.15.83:0/1030571861 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff538009510 con 0x7ff54015c960 2024-04-17T10:38:09.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.345+0000 7ff5277fe640 1 -- 172.21.15.83:0/1030571861 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5380be070 con 0x7ff54015c960 2024-04-17T10:38:09.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.345+0000 7ff5454ba640 1 -- 172.21.15.83:0/1030571861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff540154530 con 0x7ff54015c960 2024-04-17T10:38:09.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.345+0000 7ff5277fe640 1 -- 172.21.15.83:0/1030571861 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5380b88b0 con 0x7ff54015c960 2024-04-17T10:38:09.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.345+0000 7ff5454ba640 1 -- 172.21.15.83:0/1030571861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54014b930 con 0x7ff54015c960 2024-04-17T10:38:09.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.349+0000 7ff5454ba640 1 -- 172.21.15.83:0/1030571861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff540074f90 con 0x7ff54015c960 2024-04-17T10:38:09.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.349+0000 7ff5277fe640 1 -- 172.21.15.83:0/1030571861 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff5380b8a50 con 0x7ff54015c960 2024-04-17T10:38:09.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.349+0000 7ff5277fe640 1 --2- 172.21.15.83:0/1030571861 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff52c066f60 0x7ff52c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:09.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.349+0000 7ff5277fe640 1 -- 172.21.15.83:0/1030571861 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff53811ff30 con 0x7ff54015c960 2024-04-17T10:38:09.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.349+0000 7ff53effd640 1 --2- 172.21.15.83:0/1030571861 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff52c066f60 0x7ff52c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:09.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.349+0000 7ff53effd640 1 --2- 172.21.15.83:0/1030571861 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff52c066f60 0x7ff52c069420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff5340045f0 tx=0x7ff534008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:09.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.353+0000 7ff5277fe640 1 -- 172.21.15.83:0/1030571861 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff5380165e0 con 0x7ff54015c960 2024-04-17T10:38:09.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.509+0000 7ff5454ba640 1 -- 172.21.15.83:0/1030571861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5400b24f0 con 0x7ff54015c960 2024-04-17T10:38:09.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.513+0000 7ff5277fe640 1 -- 172.21.15.83:0/1030571861 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12812 (secure 0 0 0) 0x7ff5380f0cd0 con 0x7ff54015c960 2024-04-17T10:38:09.507 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:09.507 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":16,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:38:02.578015+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:38:09.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.513+0000 7ff5454ba640 1 -- 172.21.15.83:0/1030571861 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff52c066f60 msgr2=0x7ff52c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.513+0000 7ff5454ba640 1 --2- 172.21.15.83:0/1030571861 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff52c066f60 0x7ff52c069420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff5340045f0 tx=0x7ff534008040 comp rx=0 tx=0).stop 2024-04-17T10:38:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.513+0000 7ff5454ba640 1 -- 172.21.15.83:0/1030571861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54015c960 msgr2=0x7ff540155550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.513+0000 7ff5454ba640 1 --2- 172.21.15.83:0/1030571861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54015c960 0x7ff540155550 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff53801b010 tx=0x7ff538019700 comp rx=0 tx=0).stop 2024-04-17T10:38:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.513+0000 7ff5454ba640 1 -- 172.21.15.83:0/1030571861 shutdown_connections 2024-04-17T10:38:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.513+0000 7ff5454ba640 1 --2- 172.21.15.83:0/1030571861 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff52c066f60 0x7ff52c069420 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.513+0000 7ff5454ba640 1 --2- 172.21.15.83:0/1030571861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54015c960 0x7ff540155550 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.513+0000 7ff5454ba640 1 --2- 172.21.15.83:0/1030571861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff54015ba30 0x7ff5401485d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.513+0000 7ff5454ba640 1 --2- 172.21.15.83:0/1030571861 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5400cfa10 0x7ff540153d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.513+0000 7ff5454ba640 1 -- 172.21.15.83:0/1030571861 >> 172.21.15.83:0/1030571861 conn(0x7ff54006abe0 msgr2=0x7ff540073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.513+0000 7ff5454ba640 1 -- 172.21.15.83:0/1030571861 shutdown_connections 2024-04-17T10:38:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.513+0000 7ff5454ba640 1 -- 172.21.15.83:0/1030571861 wait complete. 2024-04-17T10:38:09.525 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:38:09.526 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:38:09.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.641+0000 7f87be66b640 1 Processor -- start 2024-04-17T10:38:09.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.641+0000 7f87be66b640 1 -- start start 2024-04-17T10:38:09.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.641+0000 7f87be66b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f87b815b6e0 0x7f87b8071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:09.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.641+0000 7f87be66b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b815bac0 0x7f87b80cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:09.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.641+0000 7f87be66b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f87b8070470 0x7f87b806ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:09.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.641+0000 7f87be66b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b806c1f0 con 0x7f87b815b6e0 2024-04-17T10:38:09.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.641+0000 7f87be66b640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b80cdd30 con 0x7f87b8070470 2024-04-17T10:38:09.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.641+0000 7f87be66b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b8070a10 con 0x7f87b815bac0 2024-04-17T10:38:09.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.645+0000 7f87b7fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f87b815b6e0 0x7f87b8071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:09.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.645+0000 7f87b7fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f87b815b6e0 0x7f87b8071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55738/0 (socket says 172.21.15.83:55738) 2024-04-17T10:38:09.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.645+0000 7f87b7fff640 1 -- 172.21.15.83:0/3674357531 learned_addr learned my addr 172.21.15.83:0/3674357531 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:09.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.645+0000 7f87b77fe640 1 --2- 172.21.15.83:0/3674357531 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b815bac0 0x7f87b80cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:09.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.645+0000 7f87bcc08640 1 --2- 172.21.15.83:0/3674357531 >> v2:172.21.15.167:3300/0 conn(0x7f87b8070470 0x7f87b806ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:09.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.645+0000 7f87b7fff640 1 -- 172.21.15.83:0/3674357531 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b815bac0 msgr2=0x7f87b80cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:09.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.645+0000 7f87b7fff640 1 --2- 172.21.15.83:0/3674357531 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b815bac0 0x7f87b80cd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.645+0000 7f87b7fff640 1 -- 172.21.15.83:0/3674357531 >> v2:172.21.15.167:3300/0 conn(0x7f87b8070470 msgr2=0x7f87b806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:09.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.645+0000 7f87b7fff640 1 --2- 172.21.15.83:0/3674357531 >> v2:172.21.15.167:3300/0 conn(0x7f87b8070470 0x7f87b806ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.645+0000 7f87b7fff640 1 -- 172.21.15.83:0/3674357531 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b8070cf0 con 0x7f87b815b6e0 2024-04-17T10:38:09.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.645+0000 7f87b7fff640 1 --2- 172.21.15.83:0/3674357531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87b815b6e0 0x7f87b8071cc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f87b80cd050 tx=0x7f87a801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27aa478999d4b63b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:09.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.645+0000 7f87b6ffd640 1 -- 172.21.15.83:0/3674357531 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a80b0070 con 0x7f87b815b6e0 2024-04-17T10:38:09.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.645+0000 7f87b6ffd640 1 -- 172.21.15.83:0/3674357531 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a80ab600 con 0x7f87b815b6e0 2024-04-17T10:38:09.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.645+0000 7f87b6ffd640 1 -- 172.21.15.83:0/3674357531 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a80ab9d0 con 0x7f87b815b6e0 2024-04-17T10:38:09.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.645+0000 7f87be66b640 1 -- 172.21.15.83:0/3674357531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87b815b6e0 msgr2=0x7f87b8071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:09.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.645+0000 7f87be66b640 1 --2- 172.21.15.83:0/3674357531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87b815b6e0 0x7f87b8071cc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f87b80cd050 tx=0x7f87a801c520 comp rx=0 tx=0).stop 2024-04-17T10:38:09.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.645+0000 7f87be66b640 1 -- 172.21.15.83:0/3674357531 shutdown_connections 2024-04-17T10:38:09.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.645+0000 7f87be66b640 1 --2- 172.21.15.83:0/3674357531 >> v2:172.21.15.167:3300/0 conn(0x7f87b8070470 0x7f87b806ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.645+0000 7f87be66b640 1 --2- 172.21.15.83:0/3674357531 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b815bac0 0x7f87b80cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.645+0000 7f87be66b640 1 --2- 172.21.15.83:0/3674357531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87b815b6e0 0x7f87b8071cc0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.645+0000 7f87be66b640 1 -- 172.21.15.83:0/3674357531 >> 172.21.15.83:0/3674357531 conn(0x7f87b80b15b0 msgr2=0x7f87b80b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:09.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.645+0000 7f87be66b640 1 -- 172.21.15.83:0/3674357531 shutdown_connections 2024-04-17T10:38:09.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.645+0000 7f87be66b640 1 -- 172.21.15.83:0/3674357531 wait complete. 2024-04-17T10:38:09.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.649+0000 7f87be66b640 1 Processor -- start 2024-04-17T10:38:09.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.649+0000 7f87be66b640 1 -- start start 2024-04-17T10:38:09.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.649+0000 7f87be66b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b8070470 0x7f87b80abae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:09.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.649+0000 7f87be66b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b815b6e0 0x7f87b815ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:09.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.649+0000 7f87be66b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87b815bac0 0x7f87b815eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:09.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.649+0000 7f87be66b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b806d270 con 0x7f87b815bac0 2024-04-17T10:38:09.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.649+0000 7f87be66b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b806d0f0 con 0x7f87b815b6e0 2024-04-17T10:38:09.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.649+0000 7f87be66b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b806b760 con 0x7f87b8070470 2024-04-17T10:38:09.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.649+0000 7f87bcc08640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87b815bac0 0x7f87b815eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:09.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.649+0000 7f87bcc08640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87b815bac0 0x7f87b815eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55748/0 (socket says 172.21.15.83:55748) 2024-04-17T10:38:09.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.649+0000 7f87bcc08640 1 -- 172.21.15.83:0/3778160855 learned_addr learned my addr 172.21.15.83:0/3778160855 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:09.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.649+0000 7f87b77fe640 1 --2- 172.21.15.83:0/3778160855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b815b6e0 0x7f87b815ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:09.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.649+0000 7f87b7fff640 1 --2- 172.21.15.83:0/3778160855 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b8070470 0x7f87b80abae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:09.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.649+0000 7f87bcc08640 1 -- 172.21.15.83:0/3778160855 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b8070470 msgr2=0x7f87b80abae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:09.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.649+0000 7f87bcc08640 1 --2- 172.21.15.83:0/3778160855 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b8070470 0x7f87b80abae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.649+0000 7f87bcc08640 1 -- 172.21.15.83:0/3778160855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b815b6e0 msgr2=0x7f87b815ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:09.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.649+0000 7f87bcc08640 1 --2- 172.21.15.83:0/3778160855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b815b6e0 0x7f87b815ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.649+0000 7f87bcc08640 1 -- 172.21.15.83:0/3778160855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b80ac020 con 0x7f87b815bac0 2024-04-17T10:38:09.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.649+0000 7f87b77fe640 1 --2- 172.21.15.83:0/3778160855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b815b6e0 0x7f87b815ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:09.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.649+0000 7f87bcc08640 1 --2- 172.21.15.83:0/3778160855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87b815bac0 0x7f87b815eca0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f87b000cc30 tx=0x7f87b000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:09.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.649+0000 7f87b4ff9640 1 -- 172.21.15.83:0/3778160855 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b000ce40 con 0x7f87b815bac0 2024-04-17T10:38:09.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.649+0000 7f87b4ff9640 1 -- 172.21.15.83:0/3778160855 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b00039a0 con 0x7f87b815bac0 2024-04-17T10:38:09.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.649+0000 7f87be66b640 1 -- 172.21.15.83:0/3778160855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b815d2d0 con 0x7f87b815bac0 2024-04-17T10:38:09.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.649+0000 7f87b4ff9640 1 -- 172.21.15.83:0/3778160855 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b0003d50 con 0x7f87b815bac0 2024-04-17T10:38:09.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.649+0000 7f87be66b640 1 -- 172.21.15.83:0/3778160855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b815d890 con 0x7f87b815bac0 2024-04-17T10:38:09.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.649+0000 7f87be66b640 1 -- 172.21.15.83:0/3778160855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b8074f90 con 0x7f87b815bac0 2024-04-17T10:38:09.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.649+0000 7f87b4ff9640 1 -- 172.21.15.83:0/3778160855 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f87b0003f30 con 0x7f87b815bac0 2024-04-17T10:38:09.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.653+0000 7f87b4ff9640 1 --2- 172.21.15.83:0/3778160855 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f87a4066f90 0x7f87a4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:09.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.653+0000 7f87b4ff9640 1 -- 172.21.15.83:0/3778160855 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f87b00851a0 con 0x7f87b815bac0 2024-04-17T10:38:09.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.653+0000 7f87b7fff640 1 --2- 172.21.15.83:0/3778160855 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f87a4066f90 0x7f87a4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:09.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.653+0000 7f87b7fff640 1 --2- 172.21.15.83:0/3778160855 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f87a4066f90 0x7f87a4069450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f87a80ad000 tx=0x7f87a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:09.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.657+0000 7f87b4ff9640 1 -- 172.21.15.83:0/3778160855 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f87b0014030 con 0x7f87b815bac0 2024-04-17T10:38:09.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.805+0000 7f87be66b640 1 -- 172.21.15.83:0/3778160855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f87b806ae80 con 0x7f87b815bac0 2024-04-17T10:38:09.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.805+0000 7f87b4ff9640 1 -- 172.21.15.83:0/3778160855 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f87b0056460 con 0x7f87b815bac0 2024-04-17T10:38:09.802 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:38:09.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.809+0000 7f87be66b640 1 -- 172.21.15.83:0/3778160855 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f87a4066f90 msgr2=0x7f87a4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:09.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.809+0000 7f87be66b640 1 --2- 172.21.15.83:0/3778160855 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f87a4066f90 0x7f87a4069450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f87a80ad000 tx=0x7f87a8008040 comp rx=0 tx=0).stop 2024-04-17T10:38:09.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.809+0000 7f87be66b640 1 -- 172.21.15.83:0/3778160855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87b815bac0 msgr2=0x7f87b815eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:09.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.809+0000 7f87be66b640 1 --2- 172.21.15.83:0/3778160855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87b815bac0 0x7f87b815eca0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f87b000cc30 tx=0x7f87b000a5c0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.809+0000 7f87be66b640 1 -- 172.21.15.83:0/3778160855 shutdown_connections 2024-04-17T10:38:09.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.809+0000 7f87be66b640 1 --2- 172.21.15.83:0/3778160855 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f87a4066f90 0x7f87a4069450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.809+0000 7f87be66b640 1 --2- 172.21.15.83:0/3778160855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87b815bac0 0x7f87b815eca0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.809+0000 7f87be66b640 1 --2- 172.21.15.83:0/3778160855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b815b6e0 0x7f87b815ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.809+0000 7f87be66b640 1 --2- 172.21.15.83:0/3778160855 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b8070470 0x7f87b80abae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.809+0000 7f87be66b640 1 -- 172.21.15.83:0/3778160855 >> 172.21.15.83:0/3778160855 conn(0x7f87b80b15b0 msgr2=0x7f87b80b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:09.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.809+0000 7f87be66b640 1 -- 172.21.15.83:0/3778160855 shutdown_connections 2024-04-17T10:38:09.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.809+0000 7f87be66b640 1 -- 172.21.15.83:0/3778160855 wait complete. 2024-04-17T10:38:09.819 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-17T10:38:09.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.941+0000 7f3fb5811640 1 Processor -- start 2024-04-17T10:38:09.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.941+0000 7f3fb5811640 1 -- start start 2024-04-17T10:38:09.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.941+0000 7f3fb5811640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3fb00aecd0 0x7f3fb00cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:09.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.941+0000 7f3fb5811640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3fb0166550 0x7f3fb0071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:09.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.941+0000 7f3fb5811640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fb00b3f80 0x7f3fb00b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:09.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.941+0000 7f3fb5811640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fb00cf810 con 0x7f3fb00aecd0 2024-04-17T10:38:09.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.941+0000 7f3fb5811640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fb00b4950 con 0x7f3fb0166550 2024-04-17T10:38:09.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.941+0000 7f3fb5811640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb01669b0 con 0x7f3fb00b3f80 2024-04-17T10:38:09.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.941+0000 7f3faeffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3fb00aecd0 0x7f3fb00cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:09.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.941+0000 7f3faeffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3fb00aecd0 0x7f3fb00cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55758/0 (socket says 172.21.15.83:55758) 2024-04-17T10:38:09.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.941+0000 7f3faeffd640 1 -- 172.21.15.83:0/2633411856 learned_addr learned my addr 172.21.15.83:0/2633411856 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:09.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.941+0000 7f3faf7fe640 1 --2- 172.21.15.83:0/2633411856 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fb00b3f80 0x7f3fb00b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:09.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.941+0000 7f3fa7fff640 1 --2- 172.21.15.83:0/2633411856 >> v2:172.21.15.167:3300/0 conn(0x7f3fb0166550 0x7f3fb0071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:09.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.941+0000 7f3faeffd640 1 -- 172.21.15.83:0/2633411856 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fb00b3f80 msgr2=0x7f3fb00b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:09.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.941+0000 7f3faeffd640 1 --2- 172.21.15.83:0/2633411856 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fb00b3f80 0x7f3fb00b4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.941+0000 7f3faeffd640 1 -- 172.21.15.83:0/2633411856 >> v2:172.21.15.167:3300/0 conn(0x7f3fb0166550 msgr2=0x7f3fb0071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:09.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.941+0000 7f3faeffd640 1 --2- 172.21.15.83:0/2633411856 >> v2:172.21.15.167:3300/0 conn(0x7f3fb0166550 0x7f3fb0071be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.941+0000 7f3faeffd640 1 -- 172.21.15.83:0/2633411856 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb0072760 con 0x7f3fb00aecd0 2024-04-17T10:38:09.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.941+0000 7f3fa7fff640 1 --2- 172.21.15.83:0/2633411856 >> v2:172.21.15.167:3300/0 conn(0x7f3fb0166550 0x7f3fb0071be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:09.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3faeffd640 1 --2- 172.21.15.83:0/2633411856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb00aecd0 0x7f3fb00cfca0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3fb006e500 tx=0x7f3fa001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d567858da9476eca server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:09.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3fae7fc640 1 -- 172.21.15.83:0/2633411856 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa00b0070 con 0x7f3fb00aecd0 2024-04-17T10:38:09.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3fae7fc640 1 -- 172.21.15.83:0/2633411856 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa00ab600 con 0x7f3fb00aecd0 2024-04-17T10:38:09.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3fae7fc640 1 -- 172.21.15.83:0/2633411856 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa00ab9d0 con 0x7f3fb00aecd0 2024-04-17T10:38:09.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3fb5811640 1 -- 172.21.15.83:0/2633411856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb00aecd0 msgr2=0x7f3fb00cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:09.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3fb5811640 1 --2- 172.21.15.83:0/2633411856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb00aecd0 0x7f3fb00cfca0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3fb006e500 tx=0x7f3fa001c490 comp rx=0 tx=0).stop 2024-04-17T10:38:09.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3fb5811640 1 -- 172.21.15.83:0/2633411856 shutdown_connections 2024-04-17T10:38:09.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3fb5811640 1 --2- 172.21.15.83:0/2633411856 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fb00b3f80 0x7f3fb00b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3fb5811640 1 --2- 172.21.15.83:0/2633411856 >> v2:172.21.15.167:3300/0 conn(0x7f3fb0166550 0x7f3fb0071be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3fb5811640 1 --2- 172.21.15.83:0/2633411856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb00aecd0 0x7f3fb00cfca0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3fb5811640 1 -- 172.21.15.83:0/2633411856 >> 172.21.15.83:0/2633411856 conn(0x7f3fb006abe0 msgr2=0x7f3fb006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:09.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3fb5811640 1 -- 172.21.15.83:0/2633411856 shutdown_connections 2024-04-17T10:38:09.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3fb5811640 1 -- 172.21.15.83:0/2633411856 wait complete. 2024-04-17T10:38:09.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3fb5811640 1 Processor -- start 2024-04-17T10:38:09.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3fb5811640 1 -- start start 2024-04-17T10:38:09.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3fb5811640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fb00aecd0 0x7f3fb00ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:09.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3fb5811640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb00b3f80 0x7f3fb00aa470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:09.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3fb5811640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fb0166550 0x7f3fb0154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:09.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3fb5811640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb00b2e80 con 0x7f3fb00b3f80 2024-04-17T10:38:09.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3fb5811640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb0072120 con 0x7f3fb0166550 2024-04-17T10:38:09.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3fb5811640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb00b3000 con 0x7f3fb00aecd0 2024-04-17T10:38:09.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3fa7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb00b3f80 0x7f3fb00aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:09.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3fa7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb00b3f80 0x7f3fb00aa470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55762/0 (socket says 172.21.15.83:55762) 2024-04-17T10:38:09.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3fa7fff640 1 -- 172.21.15.83:0/1413655165 learned_addr learned my addr 172.21.15.83:0/1413655165 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:09.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3faeffd640 1 --2- 172.21.15.83:0/1413655165 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fb00aecd0 0x7f3fb00ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:09.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3faf7fe640 1 --2- 172.21.15.83:0/1413655165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fb0166550 0x7f3fb0154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:09.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3fa7fff640 1 -- 172.21.15.83:0/1413655165 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fb00aecd0 msgr2=0x7f3fb00ade90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:09.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3fa7fff640 1 --2- 172.21.15.83:0/1413655165 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fb00aecd0 0x7f3fb00ade90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3fa7fff640 1 -- 172.21.15.83:0/1413655165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fb0166550 msgr2=0x7f3fb0154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:09.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3fa7fff640 1 --2- 172.21.15.83:0/1413655165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fb0166550 0x7f3fb0154fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:09.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3fa7fff640 1 -- 172.21.15.83:0/1413655165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb015d5d0 con 0x7f3fb00b3f80 2024-04-17T10:38:09.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3fa7fff640 1 --2- 172.21.15.83:0/1413655165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb00b3f80 0x7f3fb00aa470 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3f9800c7f0 tx=0x7f3f9800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:09.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3facff9640 1 -- 172.21.15.83:0/1413655165 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f98015070 con 0x7f3fb00b3f80 2024-04-17T10:38:09.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3facff9640 1 -- 172.21.15.83:0/1413655165 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f98007820 con 0x7f3fb00b3f80 2024-04-17T10:38:09.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3facff9640 1 -- 172.21.15.83:0/1413655165 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f98007c10 con 0x7f3fb00b3f80 2024-04-17T10:38:09.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.945+0000 7f3fb5811640 1 -- 172.21.15.83:0/1413655165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fb015d8b0 con 0x7f3fb00b3f80 2024-04-17T10:38:09.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.949+0000 7f3fb5811640 1 -- 172.21.15.83:0/1413655165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fb015de70 con 0x7f3fb00b3f80 2024-04-17T10:38:09.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.949+0000 7f3fb5811640 1 -- 172.21.15.83:0/1413655165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fb0074f90 con 0x7f3fb00b3f80 2024-04-17T10:38:09.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.949+0000 7f3facff9640 1 -- 172.21.15.83:0/1413655165 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3f98018070 con 0x7f3fb00b3f80 2024-04-17T10:38:09.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.949+0000 7f3facff9640 1 --2- 172.21.15.83:0/1413655165 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3f90066f60 0x7f3f90069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:09.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.949+0000 7f3faeffd640 1 --2- 172.21.15.83:0/1413655165 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3f90066f60 0x7f3f90069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:09.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.949+0000 7f3facff9640 1 -- 172.21.15.83:0/1413655165 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3f98084cf0 con 0x7f3fb00b3f80 2024-04-17T10:38:09.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.949+0000 7f3faeffd640 1 --2- 172.21.15.83:0/1413655165 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3f90066f60 0x7f3f90069420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3fb006ffe0 tx=0x7f3fa0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:09.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:09.953+0000 7f3facff9640 1 -- 172.21.15.83:0/1413655165 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3f98089070 con 0x7f3fb00b3f80 2024-04-17T10:38:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.101+0000 7f3fb5811640 1 -- 172.21.15.83:0/1413655165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3fb006b300 con 0x7f3fb00b3f80 2024-04-17T10:38:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.101+0000 7f3facff9640 1 -- 172.21.15.83:0/1413655165 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3f980560e0 con 0x7f3fb00b3f80 2024-04-17T10:38:10.097 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:10.097 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-04-17T10:38:10.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:38:10.098 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2024-04-17T10:38:10.098 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2024-04-17T10:38:10.098 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:38:10.098 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-04-17T10:38:10.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.105+0000 7f3fb5811640 1 -- 172.21.15.83:0/1413655165 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3f90066f60 msgr2=0x7f3f90069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:10.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.105+0000 7f3fb5811640 1 --2- 172.21.15.83:0/1413655165 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3f90066f60 0x7f3f90069420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3fb006ffe0 tx=0x7f3fa0008040 comp rx=0 tx=0).stop 2024-04-17T10:38:10.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.105+0000 7f3fb5811640 1 -- 172.21.15.83:0/1413655165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb00b3f80 msgr2=0x7f3fb00aa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:10.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.105+0000 7f3fb5811640 1 --2- 172.21.15.83:0/1413655165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb00b3f80 0x7f3fb00aa470 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3f9800c7f0 tx=0x7f3f9800ccb0 comp rx=0 tx=0).stop 2024-04-17T10:38:10.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.105+0000 7f3fb5811640 1 -- 172.21.15.83:0/1413655165 shutdown_connections 2024-04-17T10:38:10.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.105+0000 7f3fb5811640 1 --2- 172.21.15.83:0/1413655165 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3f90066f60 0x7f3f90069420 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:10.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.105+0000 7f3fb5811640 1 --2- 172.21.15.83:0/1413655165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fb0166550 0x7f3fb0154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:10.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.105+0000 7f3fb5811640 1 --2- 172.21.15.83:0/1413655165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb00b3f80 0x7f3fb00aa470 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:10.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.105+0000 7f3fb5811640 1 --2- 172.21.15.83:0/1413655165 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fb00aecd0 0x7f3fb00ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:10.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.105+0000 7f3fb5811640 1 -- 172.21.15.83:0/1413655165 >> 172.21.15.83:0/1413655165 conn(0x7f3fb006abe0 msgr2=0x7f3fb00af560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:10.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.105+0000 7f3fb5811640 1 -- 172.21.15.83:0/1413655165 shutdown_connections 2024-04-17T10:38:10.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.105+0000 7f3fb5811640 1 -- 172.21.15.83:0/1413655165 wait complete. 2024-04-17T10:38:10.114 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-17T10:38:10.115 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-17T10:38:10.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.249+0000 7f63292c3640 1 Processor -- start 2024-04-17T10:38:10.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.249+0000 7f63292c3640 1 -- start start 2024-04-17T10:38:10.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.253+0000 7f63292c3640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f632415cb40 0x7f632406abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:10.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.253+0000 7f63292c3640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f632406b120 0x7f6324071720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:10.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.253+0000 7f63292c3640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63240b39d0 con 0x7f63240aedf0 2024-04-17T10:38:10.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.253+0000 7f63292c3640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6324071290 con 0x7f632406b120 2024-04-17T10:38:10.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.253+0000 7f63292c3640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6324070c40 con 0x7f632415cb40 2024-04-17T10:38:10.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.253+0000 7f63227fc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f632415cb40 0x7f632406abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:10.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.253+0000 7f63237fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f632406b120 0x7f6324071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:10.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.253+0000 7f6322ffd640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f63240aedf0 0x7f63240b30f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:38150/0 (socket says 172.21.15.83:38150) 2024-04-17T10:38:10.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.253+0000 7f6322ffd640 1 -- 172.21.15.83:0/3019446798 learned_addr learned my addr 172.21.15.83:0/3019446798 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:10.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.253+0000 7f63237fe640 1 -- 172.21.15.83:0/3019446798 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f632415cb40 msgr2=0x7f632406abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:10.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.253+0000 7f63237fe640 1 --2- 172.21.15.83:0/3019446798 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f632415cb40 0x7f632406abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:10.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.253+0000 7f63237fe640 1 -- 172.21.15.83:0/3019446798 >> v1:172.21.15.167:6789/0 conn(0x7f63240aedf0 legacy=0x7f63240b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:10.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.253+0000 7f63237fe640 1 -- 172.21.15.83:0/3019446798 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6324070f20 con 0x7f632406b120 2024-04-17T10:38:10.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.253+0000 7f63237fe640 1 --2- 172.21.15.83:0/3019446798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f632406b120 0x7f6324071720 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f631c009d90 tx=0x7f631c01c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc050d9bef7097bd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:10.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.253+0000 7f6321ffb640 1 -- 172.21.15.83:0/3019446798 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c0b2070 con 0x7f632406b120 2024-04-17T10:38:10.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f6321ffb640 1 -- 172.21.15.83:0/3019446798 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631c0a57a0 con 0x7f632406b120 2024-04-17T10:38:10.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f6321ffb640 1 -- 172.21.15.83:0/3019446798 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c0a5b70 con 0x7f632406b120 2024-04-17T10:38:10.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f63292c3640 1 -- 172.21.15.83:0/3019446798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f632406b120 msgr2=0x7f6324071720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:10.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f63292c3640 1 --2- 172.21.15.83:0/3019446798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f632406b120 0x7f6324071720 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f631c009d90 tx=0x7f631c01c8e0 comp rx=0 tx=0).stop 2024-04-17T10:38:10.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f63292c3640 1 -- 172.21.15.83:0/3019446798 shutdown_connections 2024-04-17T10:38:10.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f63292c3640 1 --2- 172.21.15.83:0/3019446798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f632406b120 0x7f6324071720 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:10.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f63292c3640 1 --2- 172.21.15.83:0/3019446798 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f632415cb40 0x7f632406abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:10.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f63292c3640 1 -- 172.21.15.83:0/3019446798 >> 172.21.15.83:0/3019446798 conn(0x7f63240cc050 msgr2=0x7f63240cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:10.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f63292c3640 1 -- 172.21.15.83:0/3019446798 shutdown_connections 2024-04-17T10:38:10.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f63292c3640 1 -- 172.21.15.83:0/3019446798 wait complete. 2024-04-17T10:38:10.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f63292c3640 1 Processor -- start 2024-04-17T10:38:10.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f63292c3640 1 -- start start 2024-04-17T10:38:10.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f63292c3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f632406b120 0x7f6324152f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:10.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f63292c3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63240aedf0 0x7f632415a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:10.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f63292c3640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f632415cb40 0x7f632414a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:10.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f63292c3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6324168890 con 0x7f632406b120 2024-04-17T10:38:10.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f63292c3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6324072680 con 0x7f63240aedf0 2024-04-17T10:38:10.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f63292c3640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6324168a10 con 0x7f632415cb40 2024-04-17T10:38:10.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f6322ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f632406b120 0x7f6324152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:10.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f6322ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f632406b120 0x7f6324152f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55774/0 (socket says 172.21.15.83:55774) 2024-04-17T10:38:10.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f6322ffd640 1 -- 172.21.15.83:0/686059995 learned_addr learned my addr 172.21.15.83:0/686059995 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:10.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f63227fc640 1 --2- 172.21.15.83:0/686059995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63240aedf0 0x7f632415a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:10.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f63237fe640 1 --2- 172.21.15.83:0/686059995 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f632415cb40 0x7f632414a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:10.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f6322ffd640 1 -- 172.21.15.83:0/686059995 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f632415cb40 msgr2=0x7f632414a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:10.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f6322ffd640 1 --2- 172.21.15.83:0/686059995 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f632415cb40 0x7f632414a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:10.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f6322ffd640 1 -- 172.21.15.83:0/686059995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63240aedf0 msgr2=0x7f632415a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:10.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f6322ffd640 1 --2- 172.21.15.83:0/686059995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63240aedf0 0x7f632415a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:10.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f6322ffd640 1 -- 172.21.15.83:0/686059995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632415ae10 con 0x7f632406b120 2024-04-17T10:38:10.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f6322ffd640 1 --2- 172.21.15.83:0/686059995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f632406b120 0x7f6324152f30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f631400c500 tx=0x7f631400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:10.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f630b7fe640 1 -- 172.21.15.83:0/686059995 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6314017070 con 0x7f632406b120 2024-04-17T10:38:10.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f630b7fe640 1 -- 172.21.15.83:0/686059995 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6314005cf0 con 0x7f632406b120 2024-04-17T10:38:10.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f63292c3640 1 -- 172.21.15.83:0/686059995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6324157200 con 0x7f632406b120 2024-04-17T10:38:10.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f630b7fe640 1 -- 172.21.15.83:0/686059995 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63140072f0 con 0x7f632406b120 2024-04-17T10:38:10.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.257+0000 7f63292c3640 1 -- 172.21.15.83:0/686059995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63241577c0 con 0x7f632406b120 2024-04-17T10:38:10.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.261+0000 7f63292c3640 1 -- 172.21.15.83:0/686059995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6324075000 con 0x7f632406b120 2024-04-17T10:38:10.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.261+0000 7f630b7fe640 1 -- 172.21.15.83:0/686059995 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f631401a070 con 0x7f632406b120 2024-04-17T10:38:10.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.261+0000 7f630b7fe640 1 --2- 172.21.15.83:0/686059995 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6310066f60 0x7f6310069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:10.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.261+0000 7f630b7fe640 1 -- 172.21.15.83:0/686059995 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6314083f60 con 0x7f632406b120 2024-04-17T10:38:10.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.261+0000 7f63227fc640 1 --2- 172.21.15.83:0/686059995 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6310066f60 0x7f6310069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:10.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.261+0000 7f63227fc640 1 --2- 172.21.15.83:0/686059995 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6310066f60 0x7f6310069420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6324156370 tx=0x7f6318008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:10.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.265+0000 7f630b7fe640 1 -- 172.21.15.83:0/686059995 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6314054d00 con 0x7f632406b120 2024-04-17T10:38:10.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.425+0000 7f63292c3640 1 -- 172.21.15.83:0/686059995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f63240ae970 con 0x7f632406b120 2024-04-17T10:38:10.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.665+0000 7f630b7fe640 1 -- 172.21.15.83:0/686059995 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6314058da0 con 0x7f632406b120 2024-04-17T10:38:10.662 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2024-04-17T10:38:10.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.669+0000 7f63292c3640 1 -- 172.21.15.83:0/686059995 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6310066f60 msgr2=0x7f6310069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:10.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.669+0000 7f63292c3640 1 --2- 172.21.15.83:0/686059995 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6310066f60 0x7f6310069420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6324156370 tx=0x7f6318008040 comp rx=0 tx=0).stop 2024-04-17T10:38:10.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.669+0000 7f63292c3640 1 -- 172.21.15.83:0/686059995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f632406b120 msgr2=0x7f6324152f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:10.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.669+0000 7f63292c3640 1 --2- 172.21.15.83:0/686059995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f632406b120 0x7f6324152f30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f631400c500 tx=0x7f631400c9c0 comp rx=0 tx=0).stop 2024-04-17T10:38:10.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.669+0000 7f63292c3640 1 -- 172.21.15.83:0/686059995 shutdown_connections 2024-04-17T10:38:10.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.669+0000 7f63292c3640 1 --2- 172.21.15.83:0/686059995 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6310066f60 0x7f6310069420 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:10.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.669+0000 7f63292c3640 1 --2- 172.21.15.83:0/686059995 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f632415cb40 0x7f632414a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:10.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.669+0000 7f63292c3640 1 --2- 172.21.15.83:0/686059995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63240aedf0 0x7f632415a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:10.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.669+0000 7f63292c3640 1 --2- 172.21.15.83:0/686059995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f632406b120 0x7f6324152f30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:10.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.669+0000 7f63292c3640 1 -- 172.21.15.83:0/686059995 >> 172.21.15.83:0/686059995 conn(0x7f63240cc050 msgr2=0x7f632415bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:10.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.669+0000 7f63292c3640 1 -- 172.21.15.83:0/686059995 shutdown_connections 2024-04-17T10:38:10.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.669+0000 7f63292c3640 1 -- 172.21.15.83:0/686059995 wait complete. 2024-04-17T10:38:10.690 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-17T10:38:10.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.829+0000 7fde96136640 1 Processor -- start 2024-04-17T10:38:10.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.829+0000 7fde96136640 1 -- start start 2024-04-17T10:38:10.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.829+0000 7fde96136640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fde90154670 0x7fde900aea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:10.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.829+0000 7fde96136640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fde9006cc80 0x7fde90070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:10.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.829+0000 7fde96136640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde900b2ff0 con 0x7fde900b4090 2024-04-17T10:38:10.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.829+0000 7fde96136640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde90154a50 con 0x7fde9006cc80 2024-04-17T10:38:10.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.829+0000 7fde96136640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde900b32d0 con 0x7fde90154670 2024-04-17T10:38:10.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.829+0000 7fde87fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fde900b4090 0x7fde9006c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42234/0 (socket says 172.21.15.83:42234) 2024-04-17T10:38:10.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.829+0000 7fde87fff640 1 -- 172.21.15.83:0/2052049183 learned_addr learned my addr 172.21.15.83:0/2052049183 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:10.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.833+0000 7fde8ffff640 1 --2- 172.21.15.83:0/2052049183 >> v2:172.21.15.83:3300/0 conn(0x7fde9006cc80 0x7fde90070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:10.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.833+0000 7fde8f7fe640 1 --2- 172.21.15.83:0/2052049183 >> v2:172.21.15.167:3300/0 conn(0x7fde90154670 0x7fde900aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:10.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.833+0000 7fde8effd640 1 -- 172.21.15.83:0/2052049183 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 392066704 0 0) 0x7fde900b2ff0 con 0x7fde900b4090 2024-04-17T10:38:10.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.833+0000 7fde8effd640 1 -- 172.21.15.83:0/2052049183 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde7c003530 con 0x7fde900b4090 2024-04-17T10:38:10.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.833+0000 7fde8ffff640 1 -- 172.21.15.83:0/2052049183 >> v2:172.21.15.167:3300/0 conn(0x7fde90154670 msgr2=0x7fde900aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:10.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.833+0000 7fde8ffff640 1 --2- 172.21.15.83:0/2052049183 >> v2:172.21.15.167:3300/0 conn(0x7fde90154670 0x7fde900aea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:10.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.833+0000 7fde8ffff640 1 -- 172.21.15.83:0/2052049183 >> v1:172.21.15.83:6789/0 conn(0x7fde900b4090 legacy=0x7fde9006c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:10.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.833+0000 7fde8ffff640 1 -- 172.21.15.83:0/2052049183 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde900b35b0 con 0x7fde9006cc80 2024-04-17T10:38:10.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.833+0000 7fde8ffff640 1 --2- 172.21.15.83:0/2052049183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde9006cc80 0x7fde90070470 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fde9006bd70 tx=0x7fde8801c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1752eadfac28de54 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:10.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.833+0000 7fde8effd640 1 -- 172.21.15.83:0/2052049183 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde880b2070 con 0x7fde9006cc80 2024-04-17T10:38:10.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.833+0000 7fde8effd640 1 -- 172.21.15.83:0/2052049183 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde880a5930 con 0x7fde9006cc80 2024-04-17T10:38:10.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.833+0000 7fde8effd640 1 -- 172.21.15.83:0/2052049183 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde880a5ce0 con 0x7fde9006cc80 2024-04-17T10:38:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.833+0000 7fde96136640 1 -- 172.21.15.83:0/2052049183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde9006cc80 msgr2=0x7fde90070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.833+0000 7fde96136640 1 --2- 172.21.15.83:0/2052049183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde9006cc80 0x7fde90070470 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fde9006bd70 tx=0x7fde8801c870 comp rx=0 tx=0).stop 2024-04-17T10:38:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.833+0000 7fde96136640 1 -- 172.21.15.83:0/2052049183 shutdown_connections 2024-04-17T10:38:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.833+0000 7fde96136640 1 --2- 172.21.15.83:0/2052049183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde9006cc80 0x7fde90070470 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.833+0000 7fde96136640 1 --2- 172.21.15.83:0/2052049183 >> v2:172.21.15.167:3300/0 conn(0x7fde90154670 0x7fde900aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.833+0000 7fde96136640 1 -- 172.21.15.83:0/2052049183 >> 172.21.15.83:0/2052049183 conn(0x7fde900b15b0 msgr2=0x7fde900b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.833+0000 7fde96136640 1 -- 172.21.15.83:0/2052049183 shutdown_connections 2024-04-17T10:38:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.833+0000 7fde96136640 1 -- 172.21.15.83:0/2052049183 wait complete. 2024-04-17T10:38:10.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.837+0000 7fde96136640 1 Processor -- start 2024-04-17T10:38:10.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.837+0000 7fde96136640 1 -- start start 2024-04-17T10:38:10.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.837+0000 7fde96136640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde9006cc80 0x7fde90146f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:10.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.837+0000 7fde96136640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde900b4090 0x7fde9014d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:10.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.837+0000 7fde96136640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde90154670 0x7fde901476d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:10.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.837+0000 7fde96136640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde900aef90 con 0x7fde9006cc80 2024-04-17T10:38:10.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.837+0000 7fde96136640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9015b3c0 con 0x7fde900b4090 2024-04-17T10:38:10.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.837+0000 7fde96136640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde900af110 con 0x7fde90154670 2024-04-17T10:38:10.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.837+0000 7fde8f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde9006cc80 0x7fde90146f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:10.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.837+0000 7fde8f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde9006cc80 0x7fde90146f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55796/0 (socket says 172.21.15.83:55796) 2024-04-17T10:38:10.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.837+0000 7fde8f7fe640 1 -- 172.21.15.83:0/1699802985 learned_addr learned my addr 172.21.15.83:0/1699802985 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:10.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.837+0000 7fde8ffff640 1 --2- 172.21.15.83:0/1699802985 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde90154670 0x7fde901476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:10.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.837+0000 7fde87fff640 1 --2- 172.21.15.83:0/1699802985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde900b4090 0x7fde9014d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:10.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.837+0000 7fde8f7fe640 1 -- 172.21.15.83:0/1699802985 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde90154670 msgr2=0x7fde901476d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:10.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.837+0000 7fde8f7fe640 1 --2- 172.21.15.83:0/1699802985 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde90154670 0x7fde901476d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:10.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.837+0000 7fde8f7fe640 1 -- 172.21.15.83:0/1699802985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde900b4090 msgr2=0x7fde9014d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:10.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.837+0000 7fde8f7fe640 1 --2- 172.21.15.83:0/1699802985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde900b4090 0x7fde9014d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:10.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.837+0000 7fde8f7fe640 1 -- 172.21.15.83:0/1699802985 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde901474b0 con 0x7fde9006cc80 2024-04-17T10:38:10.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.837+0000 7fde8ffff640 1 --2- 172.21.15.83:0/1699802985 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde90154670 0x7fde901476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:38:10.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.837+0000 7fde87fff640 1 --2- 172.21.15.83:0/1699802985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde900b4090 0x7fde9014d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:38:10.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.837+0000 7fde8f7fe640 1 --2- 172.21.15.83:0/1699802985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde9006cc80 0x7fde90146f70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fde8000e870 tx=0x7fde8000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:10.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.837+0000 7fde8cff9640 1 -- 172.21.15.83:0/1699802985 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde80017070 con 0x7fde9006cc80 2024-04-17T10:38:10.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.837+0000 7fde8cff9640 1 -- 172.21.15.83:0/1699802985 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8000c940 con 0x7fde9006cc80 2024-04-17T10:38:10.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.837+0000 7fde8cff9640 1 -- 172.21.15.83:0/1699802985 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8000ccf0 con 0x7fde9006cc80 2024-04-17T10:38:10.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.837+0000 7fde96136640 1 -- 172.21.15.83:0/1699802985 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde900adc80 con 0x7fde9006cc80 2024-04-17T10:38:10.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.837+0000 7fde96136640 1 -- 172.21.15.83:0/1699802985 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde900ae240 con 0x7fde9006cc80 2024-04-17T10:38:10.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.841+0000 7fde96136640 1 -- 172.21.15.83:0/1699802985 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde90074f90 con 0x7fde9006cc80 2024-04-17T10:38:10.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.841+0000 7fde8cff9640 1 -- 172.21.15.83:0/1699802985 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fde8001a070 con 0x7fde9006cc80 2024-04-17T10:38:10.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.841+0000 7fde8cff9640 1 --2- 172.21.15.83:0/1699802985 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fde70066f60 0x7fde70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:10.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.841+0000 7fde87fff640 1 --2- 172.21.15.83:0/1699802985 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fde70066f60 0x7fde70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:10.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.841+0000 7fde8cff9640 1 -- 172.21.15.83:0/1699802985 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fde80084d40 con 0x7fde9006cc80 2024-04-17T10:38:10.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.841+0000 7fde87fff640 1 --2- 172.21.15.83:0/1699802985 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fde70066f60 0x7fde70069420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fde9015ab70 tx=0x7fde78009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:10.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:10.845+0000 7fde8cff9640 1 -- 172.21.15.83:0/1699802985 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fde80055f20 con 0x7fde9006cc80 2024-04-17T10:38:10.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.001+0000 7fde96136640 1 -- 172.21.15.83:0/1699802985 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fde900b2a00 con 0x7fde9006cc80 2024-04-17T10:38:11.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.669+0000 7fde8cff9640 1 -- 172.21.15.83:0/1699802985 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7fde80055820 con 0x7fde9006cc80 2024-04-17T10:38:11.666 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2024-04-17T10:38:11.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.673+0000 7fde96136640 1 -- 172.21.15.83:0/1699802985 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fde70066f60 msgr2=0x7fde70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:11.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.673+0000 7fde96136640 1 --2- 172.21.15.83:0/1699802985 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fde70066f60 0x7fde70069420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fde9015ab70 tx=0x7fde78009040 comp rx=0 tx=0).stop 2024-04-17T10:38:11.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.673+0000 7fde96136640 1 -- 172.21.15.83:0/1699802985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde9006cc80 msgr2=0x7fde90146f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:11.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.673+0000 7fde96136640 1 --2- 172.21.15.83:0/1699802985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde9006cc80 0x7fde90146f70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fde8000e870 tx=0x7fde8000ec70 comp rx=0 tx=0).stop 2024-04-17T10:38:11.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.677+0000 7fde96136640 1 -- 172.21.15.83:0/1699802985 shutdown_connections 2024-04-17T10:38:11.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.677+0000 7fde96136640 1 --2- 172.21.15.83:0/1699802985 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fde70066f60 0x7fde70069420 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:11.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.677+0000 7fde96136640 1 --2- 172.21.15.83:0/1699802985 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde90154670 0x7fde901476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:11.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.677+0000 7fde96136640 1 --2- 172.21.15.83:0/1699802985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde900b4090 0x7fde9014d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:11.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.677+0000 7fde96136640 1 --2- 172.21.15.83:0/1699802985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde9006cc80 0x7fde90146f70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:11.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.677+0000 7fde96136640 1 -- 172.21.15.83:0/1699802985 >> 172.21.15.83:0/1699802985 conn(0x7fde900b15b0 msgr2=0x7fde9015abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:11.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.677+0000 7fde96136640 1 -- 172.21.15.83:0/1699802985 shutdown_connections 2024-04-17T10:38:11.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.677+0000 7fde96136640 1 -- 172.21.15.83:0/1699802985 wait complete. 2024-04-17T10:38:11.706 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-17T10:38:11.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.837+0000 7f42fe017640 1 Processor -- start 2024-04-17T10:38:11.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.837+0000 7f42fe017640 1 -- start start 2024-04-17T10:38:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.837+0000 7f42fe017640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f8154de0 0x7f42f806ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.837+0000 7f42fe017640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f42f806b350 0x7f42f80b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.837+0000 7f42fe017640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42f815bd70 con 0x7f42f80b4040 2024-04-17T10:38:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.837+0000 7f42fe017640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42f80b4870 con 0x7f42f806b350 2024-04-17T10:38:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.837+0000 7f42fe017640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f815bf60 con 0x7f42f8154de0 2024-04-17T10:38:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.837+0000 7f42f6ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f42f806b350 0x7f42f80b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.837+0000 7f42f6ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f42f806b350 0x7f42f80b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55810/0 (socket says 172.21.15.83:55810) 2024-04-17T10:38:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.837+0000 7f42f6ffd640 1 -- 172.21.15.83:0/155801103 learned_addr learned my addr 172.21.15.83:0/155801103 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.837+0000 7f42f77fe640 1 --2- 172.21.15.83:0/155801103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f8154de0 0x7f42f806ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:11.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.837+0000 7f42f6ffd640 1 -- 172.21.15.83:0/155801103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f8154de0 msgr2=0x7f42f806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:11.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.837+0000 7f42f6ffd640 1 --2- 172.21.15.83:0/155801103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f8154de0 0x7f42f806ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:11.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.837+0000 7f42f6ffd640 1 -- 172.21.15.83:0/155801103 >> v1:172.21.15.83:6789/0 conn(0x7f42f80b4040 legacy=0x7f42f8071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:11.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.837+0000 7f42f6ffd640 1 -- 172.21.15.83:0/155801103 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f815afb0 con 0x7f42f806b350 2024-04-17T10:38:11.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.837+0000 7f42f77fe640 1 --2- 172.21.15.83:0/155801103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f8154de0 0x7f42f806ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:38:11.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.841+0000 7f42f6ffd640 1 --2- 172.21.15.83:0/155801103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42f806b350 0x7f42f80b3b00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f42ec009870 tx=0x7f42ec01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a78b36aeab3fabc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:11.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.841+0000 7f42f67fc640 1 -- 172.21.15.83:0/155801103 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec0b0070 con 0x7f42f806b350 2024-04-17T10:38:11.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.841+0000 7f42f67fc640 1 -- 172.21.15.83:0/155801103 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ec0ab600 con 0x7f42f806b350 2024-04-17T10:38:11.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.841+0000 7f42f67fc640 1 -- 172.21.15.83:0/155801103 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec0ab9d0 con 0x7f42f806b350 2024-04-17T10:38:11.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.841+0000 7f42fe017640 1 -- 172.21.15.83:0/155801103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42f806b350 msgr2=0x7f42f80b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:11.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.841+0000 7f42fe017640 1 --2- 172.21.15.83:0/155801103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42f806b350 0x7f42f80b3b00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f42ec009870 tx=0x7f42ec01c560 comp rx=0 tx=0).stop 2024-04-17T10:38:11.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.841+0000 7f42fe017640 1 -- 172.21.15.83:0/155801103 shutdown_connections 2024-04-17T10:38:11.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.841+0000 7f42fe017640 1 --2- 172.21.15.83:0/155801103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42f806b350 0x7f42f80b3b00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:11.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.841+0000 7f42fe017640 1 --2- 172.21.15.83:0/155801103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f8154de0 0x7f42f806ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:11.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.841+0000 7f42fe017640 1 -- 172.21.15.83:0/155801103 >> 172.21.15.83:0/155801103 conn(0x7f42f80b15b0 msgr2=0x7f42f80b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:11.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.841+0000 7f42fe017640 1 -- 172.21.15.83:0/155801103 shutdown_connections 2024-04-17T10:38:11.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.841+0000 7f42fe017640 1 -- 172.21.15.83:0/155801103 wait complete. 2024-04-17T10:38:11.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.841+0000 7f42fe017640 1 Processor -- start 2024-04-17T10:38:11.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.841+0000 7f42fe017640 1 -- start start 2024-04-17T10:38:11.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.841+0000 7f42fe017640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f806b350 0x7f42f80cd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:11.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.841+0000 7f42fe017640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42f80b4040 0x7f42f8153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:11.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.841+0000 7f42fe017640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42f8154de0 0x7f42f81541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:11.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.841+0000 7f42fe017640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f80b4450 con 0x7f42f8154de0 2024-04-17T10:38:11.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.841+0000 7f42fe017640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f81551c0 con 0x7f42f80b4040 2024-04-17T10:38:11.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.841+0000 7f42fe017640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f80b45d0 con 0x7f42f806b350 2024-04-17T10:38:11.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.841+0000 7f42f7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42f8154de0 0x7f42f81541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:11.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.841+0000 7f42f7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42f8154de0 0x7f42f81541b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55814/0 (socket says 172.21.15.83:55814) 2024-04-17T10:38:11.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.841+0000 7f42f7fff640 1 -- 172.21.15.83:0/4026641874 learned_addr learned my addr 172.21.15.83:0/4026641874 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:11.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.841+0000 7f42f77fe640 1 --2- 172.21.15.83:0/4026641874 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f806b350 0x7f42f80cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:11.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.841+0000 7f42f6ffd640 1 --2- 172.21.15.83:0/4026641874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42f80b4040 0x7f42f8153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:11.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.841+0000 7f42f7fff640 1 -- 172.21.15.83:0/4026641874 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f806b350 msgr2=0x7f42f80cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:11.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.841+0000 7f42f7fff640 1 --2- 172.21.15.83:0/4026641874 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f806b350 0x7f42f80cd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:11.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.841+0000 7f42f7fff640 1 -- 172.21.15.83:0/4026641874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42f80b4040 msgr2=0x7f42f8153c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:11.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.841+0000 7f42f7fff640 1 --2- 172.21.15.83:0/4026641874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42f80b4040 0x7f42f8153c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:11.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.841+0000 7f42f7fff640 1 -- 172.21.15.83:0/4026641874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f814d210 con 0x7f42f8154de0 2024-04-17T10:38:11.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.841+0000 7f42f7fff640 1 --2- 172.21.15.83:0/4026641874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42f8154de0 0x7f42f81541b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f42f000c730 tx=0x7f42f000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:11.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.845+0000 7f42dffff640 1 -- 172.21.15.83:0/4026641874 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f0016020 con 0x7f42f8154de0 2024-04-17T10:38:11.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.845+0000 7f42dffff640 1 -- 172.21.15.83:0/4026641874 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f0006d60 con 0x7f42f8154de0 2024-04-17T10:38:11.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.845+0000 7f42fe017640 1 -- 172.21.15.83:0/4026641874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f815a2a0 con 0x7f42f8154de0 2024-04-17T10:38:11.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.845+0000 7f42dffff640 1 -- 172.21.15.83:0/4026641874 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f0003bc0 con 0x7f42f8154de0 2024-04-17T10:38:11.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.845+0000 7f42fe017640 1 -- 172.21.15.83:0/4026641874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f815a860 con 0x7f42f8154de0 2024-04-17T10:38:11.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.845+0000 7f42fe017640 1 -- 172.21.15.83:0/4026641874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42f814d3e0 con 0x7f42f8154de0 2024-04-17T10:38:11.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.845+0000 7f42dffff640 1 -- 172.21.15.83:0/4026641874 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f42f00050d0 con 0x7f42f8154de0 2024-04-17T10:38:11.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.845+0000 7f42dffff640 1 --2- 172.21.15.83:0/4026641874 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f42e4066f60 0x7f42e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:11.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.845+0000 7f42dffff640 1 -- 172.21.15.83:0/4026641874 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f42f0084b60 con 0x7f42f8154de0 2024-04-17T10:38:11.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.845+0000 7f42f77fe640 1 --2- 172.21.15.83:0/4026641874 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f42e4066f60 0x7f42e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:11.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.845+0000 7f42f77fe640 1 --2- 172.21.15.83:0/4026641874 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f42e4066f60 0x7f42e4069420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f42e80098b0 tx=0x7f42e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:11.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:11.849+0000 7f42dffff640 1 -- 172.21.15.83:0/4026641874 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f42f0020020 con 0x7f42f8154de0 2024-04-17T10:38:12.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.029+0000 7f42fe017640 1 -- 172.21.15.83:0/4026641874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f42f8071680 con 0x7f42f8154de0 2024-04-17T10:38:12.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.689+0000 7f42dffff640 1 -- 172.21.15.83:0/4026641874 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f42f00079d0 con 0x7f42f8154de0 2024-04-17T10:38:12.684 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 2 and data pool 3 2024-04-17T10:38:12.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.693+0000 7f42fe017640 1 -- 172.21.15.83:0/4026641874 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f42e4066f60 msgr2=0x7f42e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:12.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.693+0000 7f42fe017640 1 --2- 172.21.15.83:0/4026641874 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f42e4066f60 0x7f42e4069420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f42e80098b0 tx=0x7f42e8008040 comp rx=0 tx=0).stop 2024-04-17T10:38:12.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.693+0000 7f42fe017640 1 -- 172.21.15.83:0/4026641874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42f8154de0 msgr2=0x7f42f81541b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:12.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.693+0000 7f42fe017640 1 --2- 172.21.15.83:0/4026641874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42f8154de0 0x7f42f81541b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f42f000c730 tx=0x7f42f000cb30 comp rx=0 tx=0).stop 2024-04-17T10:38:12.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.693+0000 7f42fe017640 1 -- 172.21.15.83:0/4026641874 shutdown_connections 2024-04-17T10:38:12.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.693+0000 7f42fe017640 1 --2- 172.21.15.83:0/4026641874 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f42e4066f60 0x7f42e4069420 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:12.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.693+0000 7f42fe017640 1 --2- 172.21.15.83:0/4026641874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42f8154de0 0x7f42f81541b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:12.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.693+0000 7f42fe017640 1 --2- 172.21.15.83:0/4026641874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42f80b4040 0x7f42f8153c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:12.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.693+0000 7f42fe017640 1 --2- 172.21.15.83:0/4026641874 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f806b350 0x7f42f80cd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:12.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.693+0000 7f42fe017640 1 -- 172.21.15.83:0/4026641874 >> 172.21.15.83:0/4026641874 conn(0x7f42f80b15b0 msgr2=0x7f42f806d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:12.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.693+0000 7f42fe017640 1 -- 172.21.15.83:0/4026641874 shutdown_connections 2024-04-17T10:38:12.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.693+0000 7f42fe017640 1 -- 172.21.15.83:0/4026641874 wait complete. 2024-04-17T10:38:12.727 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:38:12.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6b2d07640 1 Processor -- start 2024-04-17T10:38:12.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6b2d07640 1 -- start start 2024-04-17T10:38:12.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6b2d07640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa6ac0ce980 0x7fa6ac0b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:12.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6b2d07640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6ac0cee00 0x7fa6ac0b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:12.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6b2d07640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6ac162a00 con 0x7fa6ac15dfa0 2024-04-17T10:38:12.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6b2d07640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6ac161e80 con 0x7fa6ac0ce980 2024-04-17T10:38:12.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6b2d07640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac162160 con 0x7fa6ac0cee00 2024-04-17T10:38:12.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6b0aa3640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa6ac15dfa0 0x7fa6ac0b6c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42246/0 (socket says 172.21.15.83:42246) 2024-04-17T10:38:12.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6b0aa3640 1 -- 172.21.15.83:0/3441736367 learned_addr learned my addr 172.21.15.83:0/3441736367 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:12.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6b12a4640 1 --2- 172.21.15.83:0/3441736367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6ac0cee00 0x7fa6ac0b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:12.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6ab7fe640 1 -- 172.21.15.83:0/3441736367 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1105637886 0 0) 0x7fa6ac162a00 con 0x7fa6ac15dfa0 2024-04-17T10:38:12.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6abfff640 1 --2- 172.21.15.83:0/3441736367 >> v2:172.21.15.167:3300/0 conn(0x7fa6ac0ce980 0x7fa6ac0b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:12.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6ab7fe640 1 -- 172.21.15.83:0/3441736367 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa694003530 con 0x7fa6ac15dfa0 2024-04-17T10:38:12.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6ab7fe640 1 -- 172.21.15.83:0/3441736367 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 729466033 0 0) 0x7fa694003530 con 0x7fa6ac15dfa0 2024-04-17T10:38:12.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6ab7fe640 1 -- 172.21.15.83:0/3441736367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6ac0cee00 msgr2=0x7fa6ac0b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:12.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6ab7fe640 1 --2- 172.21.15.83:0/3441736367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6ac0cee00 0x7fa6ac0b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:12.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6ab7fe640 1 -- 172.21.15.83:0/3441736367 >> v2:172.21.15.167:3300/0 conn(0x7fa6ac0ce980 msgr2=0x7fa6ac0b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:12.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6ab7fe640 1 --2- 172.21.15.83:0/3441736367 >> v2:172.21.15.167:3300/0 conn(0x7fa6ac0ce980 0x7fa6ac0b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:12.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6ab7fe640 1 -- 172.21.15.83:0/3441736367 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ac162440 con 0x7fa6ac15dfa0 2024-04-17T10:38:12.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6ab7fe640 1 -- 172.21.15.83:0/3441736367 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fa69c001680 con 0x7fa6ac15dfa0 2024-04-17T10:38:12.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6abfff640 1 --2- 172.21.15.83:0/3441736367 >> v2:172.21.15.167:3300/0 conn(0x7fa6ac0ce980 0x7fa6ac0b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:38:12.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6ab7fe640 1 -- 172.21.15.83:0/3441736367 >> v1:172.21.15.83:6789/0 conn(0x7fa6ac15dfa0 legacy=0x7fa6ac0b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:12.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6ab7fe640 1 --2- 172.21.15.83:0/3441736367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6ac0cee00 0x7fa6940039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:12.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6ab7fe640 1 --2- 172.21.15.83:0/3441736367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa694003ff0 0x7fa694004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:12.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6ab7fe640 1 --2- 172.21.15.83:0/3441736367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa694004d50 0x7fa6940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:12.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6ab7fe640 1 -- 172.21.15.83:0/3441736367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac161e80 con 0x7fa694004d50 2024-04-17T10:38:12.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6ab7fe640 1 -- 172.21.15.83:0/3441736367 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac162160 con 0x7fa694003ff0 2024-04-17T10:38:12.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6ab7fe640 1 -- 172.21.15.83:0/3441736367 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa69400bb90 con 0x7fa6ac0cee00 2024-04-17T10:38:12.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6b12a4640 1 --2- 172.21.15.83:0/3441736367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa694004d50 0x7fa6940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:12.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6abfff640 1 --2- 172.21.15.83:0/3441736367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa694003ff0 0x7fa694004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:12.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6b0aa3640 1 --2- 172.21.15.83:0/3441736367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6ac0cee00 0x7fa6940039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:12.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6b12a4640 1 -- 172.21.15.83:0/3441736367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6ac0cee00 msgr2=0x7fa6940039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:12.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6b12a4640 1 --2- 172.21.15.83:0/3441736367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6ac0cee00 0x7fa6940039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:12.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6b12a4640 1 -- 172.21.15.83:0/3441736367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa694003ff0 msgr2=0x7fa694004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:12.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6b12a4640 1 --2- 172.21.15.83:0/3441736367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa694003ff0 0x7fa694004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:12.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6b12a4640 1 -- 172.21.15.83:0/3441736367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa694003530 con 0x7fa694004d50 2024-04-17T10:38:12.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.849+0000 7fa6abfff640 1 --2- 172.21.15.83:0/3441736367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa694003ff0 0x7fa694004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:12.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.853+0000 7fa6b12a4640 1 --2- 172.21.15.83:0/3441736367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa694004d50 0x7fa6940051d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4007360 tx=0x7fa6a401ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8153a6d73206b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:12.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.853+0000 7fa6ab7fe640 1 -- 172.21.15.83:0/3441736367 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a40b2070 con 0x7fa694004d50 2024-04-17T10:38:12.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.853+0000 7fa6ab7fe640 1 -- 172.21.15.83:0/3441736367 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69c001d40 con 0x7fa694004d50 2024-04-17T10:38:12.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.853+0000 7fa6ab7fe640 1 -- 172.21.15.83:0/3441736367 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4009be0 con 0x7fa694004d50 2024-04-17T10:38:12.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.853+0000 7fa6b2d07640 1 -- 172.21.15.83:0/3441736367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa694004d50 msgr2=0x7fa6940051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:12.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.853+0000 7fa6b2d07640 1 --2- 172.21.15.83:0/3441736367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa694004d50 0x7fa6940051d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4007360 tx=0x7fa6a401ce40 comp rx=0 tx=0).stop 2024-04-17T10:38:12.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.853+0000 7fa6b12a4640 1 -- 172.21.15.83:0/3441736367 reap_dead start 2024-04-17T10:38:12.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.853+0000 7fa6b2d07640 1 -- 172.21.15.83:0/3441736367 shutdown_connections 2024-04-17T10:38:12.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.853+0000 7fa6b2d07640 1 -- 172.21.15.83:0/3441736367 >> 172.21.15.83:0/3441736367 conn(0x7fa6ac0d0600 msgr2=0x7fa6ac0d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:12.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.853+0000 7fa6b2d07640 1 -- 172.21.15.83:0/3441736367 shutdown_connections 2024-04-17T10:38:12.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.853+0000 7fa6b2d07640 1 -- 172.21.15.83:0/3441736367 wait complete. 2024-04-17T10:38:12.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.853+0000 7fa6b2d07640 1 Processor -- start 2024-04-17T10:38:12.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.853+0000 7fa6b2d07640 1 -- start start 2024-04-17T10:38:12.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.853+0000 7fa6b2d07640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa694004d50 0x7fa6ac0c6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:12.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.853+0000 7fa6b2d07640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6ac156c30 0x7fa6ac0c63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:12.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.853+0000 7fa6b2d07640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6ac1554b0 0x7fa6ac0ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:12.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.853+0000 7fa6b2d07640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac161b10 con 0x7fa694004d50 2024-04-17T10:38:12.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.853+0000 7fa6b2d07640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac15e380 con 0x7fa6ac1554b0 2024-04-17T10:38:12.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.853+0000 7fa6b2d07640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac0b5970 con 0x7fa6ac156c30 2024-04-17T10:38:12.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.853+0000 7fa6b0aa3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa694004d50 0x7fa6ac0c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:12.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.857+0000 7fa6b0aa3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa694004d50 0x7fa6ac0c6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55838/0 (socket says 172.21.15.83:55838) 2024-04-17T10:38:12.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.857+0000 7fa6b0aa3640 1 -- 172.21.15.83:0/3045593769 learned_addr learned my addr 172.21.15.83:0/3045593769 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:12.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.857+0000 7fa6abfff640 1 --2- 172.21.15.83:0/3045593769 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6ac156c30 0x7fa6ac0c63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:12.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.857+0000 7fa6b12a4640 1 --2- 172.21.15.83:0/3045593769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6ac1554b0 0x7fa6ac0ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:12.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.857+0000 7fa6b0aa3640 1 -- 172.21.15.83:0/3045593769 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6ac156c30 msgr2=0x7fa6ac0c63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:12.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.857+0000 7fa6b0aa3640 1 --2- 172.21.15.83:0/3045593769 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6ac156c30 0x7fa6ac0c63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:12.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.857+0000 7fa6b0aa3640 1 -- 172.21.15.83:0/3045593769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6ac1554b0 msgr2=0x7fa6ac0ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:12.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.857+0000 7fa6b0aa3640 1 --2- 172.21.15.83:0/3045593769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6ac1554b0 0x7fa6ac0ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:12.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.857+0000 7fa6b0aa3640 1 -- 172.21.15.83:0/3045593769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ac155b90 con 0x7fa694004d50 2024-04-17T10:38:12.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.857+0000 7fa6b12a4640 1 --2- 172.21.15.83:0/3045593769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6ac1554b0 0x7fa6ac0ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:38:12.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.857+0000 7fa6b0aa3640 1 --2- 172.21.15.83:0/3045593769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa694004d50 0x7fa6ac0c6960 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa69c010be0 tx=0x7fa69c00e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:12.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.857+0000 7fa6a97fa640 1 -- 172.21.15.83:0/3045593769 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c01b070 con 0x7fa694004d50 2024-04-17T10:38:12.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.857+0000 7fa6a97fa640 1 -- 172.21.15.83:0/3045593769 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69c011040 con 0x7fa694004d50 2024-04-17T10:38:12.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.857+0000 7fa6b2d07640 1 -- 172.21.15.83:0/3045593769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6ac155e70 con 0x7fa694004d50 2024-04-17T10:38:12.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.857+0000 7fa6a97fa640 1 -- 172.21.15.83:0/3045593769 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c0143a0 con 0x7fa694004d50 2024-04-17T10:38:12.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.857+0000 7fa6b2d07640 1 -- 172.21.15.83:0/3045593769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6ac156430 con 0x7fa694004d50 2024-04-17T10:38:12.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.857+0000 7fa6b2d07640 1 -- 172.21.15.83:0/3045593769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6ac0681a0 con 0x7fa694004d50 2024-04-17T10:38:12.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.857+0000 7fa6a97fa640 1 -- 172.21.15.83:0/3045593769 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa69c014540 con 0x7fa694004d50 2024-04-17T10:38:12.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.857+0000 7fa6a97fa640 1 --2- 172.21.15.83:0/3045593769 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa698066f90 0x7fa698069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:12.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.857+0000 7fa6abfff640 1 --2- 172.21.15.83:0/3045593769 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa698066f90 0x7fa698069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:12.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.857+0000 7fa6a97fa640 1 -- 172.21.15.83:0/3045593769 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa69c0879c0 con 0x7fa694004d50 2024-04-17T10:38:12.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.857+0000 7fa6abfff640 1 --2- 172.21.15.83:0/3045593769 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa698066f90 0x7fa698069450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac1502c0 tx=0x7fa6a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:12.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:12.861+0000 7fa6a97fa640 1 -- 172.21.15.83:0/3045593769 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa69c089e20 con 0x7fa694004d50 2024-04-17T10:38:13.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.021+0000 7fa6b2d07640 1 -- 172.21.15.83:0/3045593769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6ac074f90 con 0x7fa694004d50 2024-04-17T10:38:13.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.025+0000 7fa6a97fa640 1 -- 172.21.15.83:0/3045593769 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16546 (secure 0 0 0) 0x7fa69c058380 con 0x7fa694004d50 2024-04-17T10:38:13.019 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:13.020 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":19,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:38:12.032255+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:38:10.464490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T10:38:11.043134+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:38:13.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.025+0000 7fa6b2d07640 1 -- 172.21.15.83:0/3045593769 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa698066f90 msgr2=0x7fa698069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:13.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.025+0000 7fa6b2d07640 1 --2- 172.21.15.83:0/3045593769 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa698066f90 0x7fa698069450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac1502c0 tx=0x7fa6a0008040 comp rx=0 tx=0).stop 2024-04-17T10:38:13.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.025+0000 7fa6b2d07640 1 -- 172.21.15.83:0/3045593769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa694004d50 msgr2=0x7fa6ac0c6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:13.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.025+0000 7fa6b2d07640 1 --2- 172.21.15.83:0/3045593769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa694004d50 0x7fa6ac0c6960 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa69c010be0 tx=0x7fa69c00e4d0 comp rx=0 tx=0).stop 2024-04-17T10:38:13.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.025+0000 7fa6b2d07640 1 -- 172.21.15.83:0/3045593769 shutdown_connections 2024-04-17T10:38:13.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.025+0000 7fa6b2d07640 1 --2- 172.21.15.83:0/3045593769 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa698066f90 0x7fa698069450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:13.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.025+0000 7fa6b2d07640 1 --2- 172.21.15.83:0/3045593769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6ac1554b0 0x7fa6ac0ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:13.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.025+0000 7fa6b2d07640 1 --2- 172.21.15.83:0/3045593769 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6ac156c30 0x7fa6ac0c63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:13.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.025+0000 7fa6b2d07640 1 --2- 172.21.15.83:0/3045593769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa694004d50 0x7fa6ac0c6960 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:13.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.025+0000 7fa6b2d07640 1 -- 172.21.15.83:0/3045593769 >> 172.21.15.83:0/3045593769 conn(0x7fa6ac0d0600 msgr2=0x7fa6ac0d2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:13.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.025+0000 7fa6b2d07640 1 -- 172.21.15.83:0/3045593769 shutdown_connections 2024-04-17T10:38:13.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.025+0000 7fa6b2d07640 1 -- 172.21.15.83:0/3045593769 wait complete. 2024-04-17T10:38:13.039 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:38:13.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.161+0000 7f656743b640 1 Processor -- start 2024-04-17T10:38:13.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.161+0000 7f656743b640 1 -- start start 2024-04-17T10:38:13.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.161+0000 7f656743b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f656006bbf0 0x7f656006c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:13.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.161+0000 7f656743b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6560071750 0x7f6560071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:13.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.161+0000 7f656743b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6560161900 con 0x7f6560161300 2024-04-17T10:38:13.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.161+0000 7f656743b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6560161ba0 con 0x7f656006bbf0 2024-04-17T10:38:13.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.161+0000 7f656743b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6560161e80 con 0x7f6560071750 2024-04-17T10:38:13.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.161+0000 7f65649d6640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f656006bbf0 0x7f656006c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:13.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.161+0000 7f65651d7640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6560161300 0x7f656006ee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42256/0 (socket says 172.21.15.83:42256) 2024-04-17T10:38:13.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.161+0000 7f65651d7640 1 -- 172.21.15.83:0/234043069 learned_addr learned my addr 172.21.15.83:0/234043069 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:13.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.161+0000 7f65659d8640 1 --2- 172.21.15.83:0/234043069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6560071750 0x7f6560071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:13.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.161+0000 7f65649d6640 1 -- 172.21.15.83:0/234043069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6560071750 msgr2=0x7f6560071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:13.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.161+0000 7f65649d6640 1 --2- 172.21.15.83:0/234043069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6560071750 0x7f6560071be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:13.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.161+0000 7f65649d6640 1 -- 172.21.15.83:0/234043069 >> v1:172.21.15.83:6789/0 conn(0x7f6560161300 legacy=0x7f656006ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:13.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.161+0000 7f65649d6640 1 -- 172.21.15.83:0/234043069 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6560162160 con 0x7f656006bbf0 2024-04-17T10:38:13.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.165+0000 7f65649d6640 1 --2- 172.21.15.83:0/234043069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f656006bbf0 0x7f656006c070 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f65500097c0 tx=0x7f655001c2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d84fc8235bc363 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:13.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.165+0000 7f6557fff640 1 -- 172.21.15.83:0/234043069 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65500b0070 con 0x7f656006bbf0 2024-04-17T10:38:13.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.165+0000 7f6557fff640 1 -- 172.21.15.83:0/234043069 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65500ab600 con 0x7f656006bbf0 2024-04-17T10:38:13.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.165+0000 7f6557fff640 1 -- 172.21.15.83:0/234043069 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65500ab9d0 con 0x7f656006bbf0 2024-04-17T10:38:13.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.165+0000 7f656743b640 1 -- 172.21.15.83:0/234043069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f656006bbf0 msgr2=0x7f656006c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:13.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.165+0000 7f656743b640 1 --2- 172.21.15.83:0/234043069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f656006bbf0 0x7f656006c070 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f65500097c0 tx=0x7f655001c2a0 comp rx=0 tx=0).stop 2024-04-17T10:38:13.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.165+0000 7f656743b640 1 -- 172.21.15.83:0/234043069 shutdown_connections 2024-04-17T10:38:13.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.165+0000 7f656743b640 1 --2- 172.21.15.83:0/234043069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6560071750 0x7f6560071be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:13.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.165+0000 7f656743b640 1 --2- 172.21.15.83:0/234043069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f656006bbf0 0x7f656006c070 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:13.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.165+0000 7f656743b640 1 -- 172.21.15.83:0/234043069 >> 172.21.15.83:0/234043069 conn(0x7f65600cc910 msgr2=0x7f65600ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:13.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.165+0000 7f656743b640 1 -- 172.21.15.83:0/234043069 shutdown_connections 2024-04-17T10:38:13.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.165+0000 7f656743b640 1 -- 172.21.15.83:0/234043069 wait complete. 2024-04-17T10:38:13.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.165+0000 7f656743b640 1 Processor -- start 2024-04-17T10:38:13.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.165+0000 7f656743b640 1 -- start start 2024-04-17T10:38:13.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.165+0000 7f656743b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f656006bbf0 0x7f6560155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:13.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.165+0000 7f656743b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6560071750 0x7f65600a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:13.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.165+0000 7f656743b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6560161300 0x7f656009b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:13.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.165+0000 7f656743b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656016af30 con 0x7f6560071750 2024-04-17T10:38:13.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.165+0000 7f656743b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656016adb0 con 0x7f656006bbf0 2024-04-17T10:38:13.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.165+0000 7f656743b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f656016b0b0 con 0x7f6560161300 2024-04-17T10:38:13.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.165+0000 7f65649d6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6560071750 0x7f65600a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:13.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.165+0000 7f65649d6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6560071750 0x7f65600a09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55856/0 (socket says 172.21.15.83:55856) 2024-04-17T10:38:13.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.165+0000 7f65649d6640 1 -- 172.21.15.83:0/1725833982 learned_addr learned my addr 172.21.15.83:0/1725833982 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:13.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.169+0000 7f65651d7640 1 --2- 172.21.15.83:0/1725833982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f656006bbf0 0x7f6560155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:13.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.169+0000 7f65649d6640 1 -- 172.21.15.83:0/1725833982 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6560161300 msgr2=0x7f656009b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:13.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.169+0000 7f65659d8640 1 --2- 172.21.15.83:0/1725833982 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6560161300 0x7f656009b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:13.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.169+0000 7f65649d6640 1 --2- 172.21.15.83:0/1725833982 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6560161300 0x7f656009b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:13.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.169+0000 7f65649d6640 1 -- 172.21.15.83:0/1725833982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f656006bbf0 msgr2=0x7f6560155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:13.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.169+0000 7f65649d6640 1 --2- 172.21.15.83:0/1725833982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f656006bbf0 0x7f6560155680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:13.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.169+0000 7f65649d6640 1 -- 172.21.15.83:0/1725833982 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656009f950 con 0x7f6560071750 2024-04-17T10:38:13.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.169+0000 7f65651d7640 1 --2- 172.21.15.83:0/1725833982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f656006bbf0 0x7f6560155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:13.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.169+0000 7f65649d6640 1 --2- 172.21.15.83:0/1725833982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6560071750 0x7f65600a09f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f65500ae040 tx=0x7f655001aae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:13.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.169+0000 7f6555ffb640 1 -- 172.21.15.83:0/1725833982 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65500b0050 con 0x7f6560071750 2024-04-17T10:38:13.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.169+0000 7f6555ffb640 1 -- 172.21.15.83:0/1725833982 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65500bc070 con 0x7f6560071750 2024-04-17T10:38:13.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.169+0000 7f656743b640 1 -- 172.21.15.83:0/1725833982 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f656009fc30 con 0x7f6560071750 2024-04-17T10:38:13.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.169+0000 7f6555ffb640 1 -- 172.21.15.83:0/1725833982 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65500b5850 con 0x7f6560071750 2024-04-17T10:38:13.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.169+0000 7f656743b640 1 -- 172.21.15.83:0/1725833982 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65600a01f0 con 0x7f6560071750 2024-04-17T10:38:13.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.169+0000 7f656743b640 1 -- 172.21.15.83:0/1725833982 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6560074f90 con 0x7f6560071750 2024-04-17T10:38:13.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.169+0000 7f6555ffb640 1 -- 172.21.15.83:0/1725833982 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f65500b59f0 con 0x7f6560071750 2024-04-17T10:38:13.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.169+0000 7f6555ffb640 1 --2- 172.21.15.83:0/1725833982 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6544066f90 0x7f6544069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:13.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.169+0000 7f6555ffb640 1 -- 172.21.15.83:0/1725833982 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f655011f8b0 con 0x7f6560071750 2024-04-17T10:38:13.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.169+0000 7f65651d7640 1 --2- 172.21.15.83:0/1725833982 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6544066f90 0x7f6544069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:13.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.169+0000 7f65651d7640 1 --2- 172.21.15.83:0/1725833982 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6544066f90 0x7f6544069450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f655c000f70 tx=0x7f655c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:13.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.173+0000 7f6555ffb640 1 -- 172.21.15.83:0/1725833982 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6550015fb0 con 0x7f6560071750 2024-04-17T10:38:13.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.325+0000 7f656743b640 1 -- 172.21.15.83:0/1725833982 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6560068150 con 0x7f6560071750 2024-04-17T10:38:13.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.329+0000 7f6555ffb640 1 -- 172.21.15.83:0/1725833982 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16546 (secure 0 0 0) 0x7f65500f08c0 con 0x7f6560071750 2024-04-17T10:38:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:13.323 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":19,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:38:12.032255+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:38:10.464490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T10:38:11.043134+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:38:13.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.329+0000 7f656743b640 1 -- 172.21.15.83:0/1725833982 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6544066f90 msgr2=0x7f6544069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:13.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.329+0000 7f656743b640 1 --2- 172.21.15.83:0/1725833982 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6544066f90 0x7f6544069450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f655c000f70 tx=0x7f655c009040 comp rx=0 tx=0).stop 2024-04-17T10:38:13.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.329+0000 7f656743b640 1 -- 172.21.15.83:0/1725833982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6560071750 msgr2=0x7f65600a09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:13.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.329+0000 7f656743b640 1 --2- 172.21.15.83:0/1725833982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6560071750 0x7f65600a09f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f65500ae040 tx=0x7f655001aae0 comp rx=0 tx=0).stop 2024-04-17T10:38:13.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.329+0000 7f656743b640 1 -- 172.21.15.83:0/1725833982 shutdown_connections 2024-04-17T10:38:13.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.329+0000 7f656743b640 1 --2- 172.21.15.83:0/1725833982 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6544066f90 0x7f6544069450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:13.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.329+0000 7f656743b640 1 --2- 172.21.15.83:0/1725833982 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6560161300 0x7f656009b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:13.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.329+0000 7f656743b640 1 --2- 172.21.15.83:0/1725833982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6560071750 0x7f65600a09f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:13.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.329+0000 7f656743b640 1 --2- 172.21.15.83:0/1725833982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f656006bbf0 0x7f6560155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:13.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.329+0000 7f656743b640 1 -- 172.21.15.83:0/1725833982 >> 172.21.15.83:0/1725833982 conn(0x7f65600cc910 msgr2=0x7f656015fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:13.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.329+0000 7f656743b640 1 -- 172.21.15.83:0/1725833982 shutdown_connections 2024-04-17T10:38:13.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.329+0000 7f656743b640 1 -- 172.21.15.83:0/1725833982 wait complete. 2024-04-17T10:38:13.340 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-17T10:38:13.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.469+0000 7f0aba7b7640 1 Processor -- start 2024-04-17T10:38:13.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.469+0000 7f0aba7b7640 1 -- start start 2024-04-17T10:38:13.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.469+0000 7f0aba7b7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ab406b520 0x7f0ab4071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:13.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.469+0000 7f0aba7b7640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab406c1f0 con 0x7f0ab415b6e0 2024-04-17T10:38:13.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.469+0000 7f0aba7b7640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab4070a10 con 0x7f0ab415bac0 2024-04-17T10:38:13.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.469+0000 7f0aba7b7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab4070cf0 con 0x7f0ab406b520 2024-04-17T10:38:13.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.469+0000 7f0ab3fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0ab415b6e0 0x7f0ab40cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42262/0 (socket says 172.21.15.83:42262) 2024-04-17T10:38:13.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.469+0000 7f0ab3fff640 1 -- 172.21.15.83:0/636376764 learned_addr learned my addr 172.21.15.83:0/636376764 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:13.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.469+0000 7f0ab8d54640 1 --2- 172.21.15.83:0/636376764 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ab406b520 0x7f0ab4071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:13.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.469+0000 7f0ab2ffd640 1 -- 172.21.15.83:0/636376764 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2265843276 0 0) 0x7f0ab406c1f0 con 0x7f0ab415b6e0 2024-04-17T10:38:13.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.469+0000 7f0ab2ffd640 1 -- 172.21.15.83:0/636376764 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aa0003530 con 0x7f0ab415b6e0 2024-04-17T10:38:13.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.469+0000 7f0ab2ffd640 1 -- 172.21.15.83:0/636376764 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4016622521 0 0) 0x7f0aa0003530 con 0x7f0ab415b6e0 2024-04-17T10:38:13.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.469+0000 7f0ab2ffd640 1 -- 172.21.15.83:0/636376764 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ab406b520 msgr2=0x7f0ab4071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:13.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.469+0000 7f0ab2ffd640 1 --2- 172.21.15.83:0/636376764 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ab406b520 0x7f0ab4071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:13.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.469+0000 7f0ab2ffd640 1 -- 172.21.15.83:0/636376764 >> v1:172.21.15.167:6789/0 conn(0x7f0ab415bac0 legacy=0x7f0ab406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:13.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.469+0000 7f0ab2ffd640 1 -- 172.21.15.83:0/636376764 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab4070fd0 con 0x7f0ab415b6e0 2024-04-17T10:38:13.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.469+0000 7f0ab2ffd640 1 -- 172.21.15.83:0/636376764 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f0aa4001a00 con 0x7f0ab415b6e0 2024-04-17T10:38:13.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.469+0000 7f0ab2ffd640 1 -- 172.21.15.83:0/636376764 >> v1:172.21.15.83:6789/0 conn(0x7f0ab415b6e0 legacy=0x7f0ab40cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:13.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.469+0000 7f0ab2ffd640 1 --2- 172.21.15.83:0/636376764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0aa0003a60 0x7f0aa0003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:13.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.469+0000 7f0ab2ffd640 1 --2- 172.21.15.83:0/636376764 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ab406b520 0x7f0aa0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:13.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.469+0000 7f0ab2ffd640 1 --2- 172.21.15.83:0/636376764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0aa0004d50 0x7f0aa00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:13.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.469+0000 7f0ab2ffd640 1 -- 172.21.15.83:0/636376764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab4070cf0 con 0x7f0aa0003a60 2024-04-17T10:38:13.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.469+0000 7f0ab2ffd640 1 -- 172.21.15.83:0/636376764 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa000bb10 con 0x7f0aa0004d50 2024-04-17T10:38:13.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.469+0000 7f0ab2ffd640 1 -- 172.21.15.83:0/636376764 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa000bd10 con 0x7f0ab406b520 2024-04-17T10:38:13.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.469+0000 7f0ab3fff640 1 --2- 172.21.15.83:0/636376764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0aa0003a60 0x7f0aa0003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:13.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.469+0000 7f0ab37fe640 1 --2- 172.21.15.83:0/636376764 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ab406b520 0x7f0aa0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:13.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.469+0000 7f0ab8d54640 1 --2- 172.21.15.83:0/636376764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0aa0004d50 0x7f0aa00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:13.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.469+0000 7f0ab3fff640 1 -- 172.21.15.83:0/636376764 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ab406b520 msgr2=0x7f0aa0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:13.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.469+0000 7f0ab3fff640 1 --2- 172.21.15.83:0/636376764 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ab406b520 0x7f0aa0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:13.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.469+0000 7f0ab3fff640 1 -- 172.21.15.83:0/636376764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0aa0004d50 msgr2=0x7f0aa00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:13.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.469+0000 7f0ab3fff640 1 --2- 172.21.15.83:0/636376764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0aa0004d50 0x7f0aa00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:13.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.469+0000 7f0ab3fff640 1 -- 172.21.15.83:0/636376764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa0003530 con 0x7f0aa0003a60 2024-04-17T10:38:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.473+0000 7f0ab3fff640 1 --2- 172.21.15.83:0/636376764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0aa0003a60 0x7f0aa0003e80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0aa400a830 tx=0x7f0aa401d2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8fc8fc6b698360f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.473+0000 7f0ab2ffd640 1 -- 172.21.15.83:0/636376764 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa400a480 con 0x7f0aa0003a60 2024-04-17T10:38:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.473+0000 7f0ab2ffd640 1 -- 172.21.15.83:0/636376764 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa40ad630 con 0x7f0aa0003a60 2024-04-17T10:38:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.473+0000 7f0ab2ffd640 1 -- 172.21.15.83:0/636376764 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa40ada60 con 0x7f0aa0003a60 2024-04-17T10:38:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.473+0000 7f0aba7b7640 1 -- 172.21.15.83:0/636376764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0aa0003a60 msgr2=0x7f0aa0003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.473+0000 7f0aba7b7640 1 --2- 172.21.15.83:0/636376764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0aa0003a60 0x7f0aa0003e80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0aa400a830 tx=0x7f0aa401d2a0 comp rx=0 tx=0).stop 2024-04-17T10:38:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.473+0000 7f0ab8d54640 1 -- 172.21.15.83:0/636376764 reap_dead start 2024-04-17T10:38:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.473+0000 7f0aba7b7640 1 -- 172.21.15.83:0/636376764 shutdown_connections 2024-04-17T10:38:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.473+0000 7f0aba7b7640 1 -- 172.21.15.83:0/636376764 >> 172.21.15.83:0/636376764 conn(0x7f0ab40b15b0 msgr2=0x7f0ab40b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:13.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.473+0000 7f0aba7b7640 1 -- 172.21.15.83:0/636376764 shutdown_connections 2024-04-17T10:38:13.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.473+0000 7f0aba7b7640 1 -- 172.21.15.83:0/636376764 wait complete. 2024-04-17T10:38:13.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.473+0000 7f0aba7b7640 1 Processor -- start 2024-04-17T10:38:13.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.473+0000 7f0aba7b7640 1 -- start start 2024-04-17T10:38:13.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.477+0000 7f0aba7b7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0aa0003a60 0x7f0ab41f6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:13.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.477+0000 7f0aba7b7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ab414cdf0 0x7f0ab41571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:13.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.477+0000 7f0aba7b7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ab4150ed0 0x7f0ab40ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:13.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.477+0000 7f0aba7b7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab406d2b0 con 0x7f0aa0003a60 2024-04-17T10:38:13.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.477+0000 7f0aba7b7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab406d130 con 0x7f0ab414cdf0 2024-04-17T10:38:13.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.477+0000 7f0aba7b7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab4167600 con 0x7f0ab4150ed0 2024-04-17T10:38:13.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.477+0000 7f0ab3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0aa0003a60 0x7f0ab41f6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:13.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.477+0000 7f0ab3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0aa0003a60 0x7f0ab41f6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55880/0 (socket says 172.21.15.83:55880) 2024-04-17T10:38:13.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.477+0000 7f0ab3fff640 1 -- 172.21.15.83:0/3147293608 learned_addr learned my addr 172.21.15.83:0/3147293608 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:13.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.477+0000 7f0ab37fe640 1 --2- 172.21.15.83:0/3147293608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ab414cdf0 0x7f0ab41571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:13.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.477+0000 7f0ab8d54640 1 --2- 172.21.15.83:0/3147293608 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ab4150ed0 0x7f0ab40ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:13.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.477+0000 7f0ab3fff640 1 -- 172.21.15.83:0/3147293608 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ab4150ed0 msgr2=0x7f0ab40ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:13.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.477+0000 7f0ab3fff640 1 --2- 172.21.15.83:0/3147293608 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ab4150ed0 0x7f0ab40ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:13.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.477+0000 7f0ab3fff640 1 -- 172.21.15.83:0/3147293608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ab414cdf0 msgr2=0x7f0ab41571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:13.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.477+0000 7f0ab3fff640 1 --2- 172.21.15.83:0/3147293608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ab414cdf0 0x7f0ab41571c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:13.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.477+0000 7f0ab3fff640 1 -- 172.21.15.83:0/3147293608 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab41f7540 con 0x7f0aa0003a60 2024-04-17T10:38:13.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.477+0000 7f0ab3fff640 1 --2- 172.21.15.83:0/3147293608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0aa0003a60 0x7f0ab41f6ea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0aa401d0d0 tx=0x7f0aa0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:13.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.477+0000 7f0ab0ff9640 1 -- 172.21.15.83:0/3147293608 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa40ad430 con 0x7f0aa0003a60 2024-04-17T10:38:13.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.477+0000 7f0ab0ff9640 1 -- 172.21.15.83:0/3147293608 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa40bd070 con 0x7f0aa0003a60 2024-04-17T10:38:13.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.477+0000 7f0aba7b7640 1 -- 172.21.15.83:0/3147293608 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab41f7820 con 0x7f0aa0003a60 2024-04-17T10:38:13.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.477+0000 7f0ab0ff9640 1 -- 172.21.15.83:0/3147293608 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa40017c0 con 0x7f0aa0003a60 2024-04-17T10:38:13.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.477+0000 7f0aba7b7640 1 -- 172.21.15.83:0/3147293608 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab41f7de0 con 0x7f0aa0003a60 2024-04-17T10:38:13.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.477+0000 7f0aba7b7640 1 -- 172.21.15.83:0/3147293608 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab4074f90 con 0x7f0aa0003a60 2024-04-17T10:38:13.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.477+0000 7f0ab0ff9640 1 -- 172.21.15.83:0/3147293608 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0aa40b5070 con 0x7f0aa0003a60 2024-04-17T10:38:13.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.477+0000 7f0ab0ff9640 1 --2- 172.21.15.83:0/3147293608 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0a94066f90 0x7f0a94069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:13.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.477+0000 7f0ab37fe640 1 --2- 172.21.15.83:0/3147293608 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0a94066f90 0x7f0a94069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:13.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.477+0000 7f0ab0ff9640 1 -- 172.21.15.83:0/3147293608 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0aa411f860 con 0x7f0aa0003a60 2024-04-17T10:38:13.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.477+0000 7f0ab37fe640 1 --2- 172.21.15.83:0/3147293608 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0a94066f90 0x7f0a94069450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4159ce0 tx=0x7f0a9c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:13.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.481+0000 7f0ab0ff9640 1 -- 172.21.15.83:0/3147293608 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0aa4016fb0 con 0x7f0aa0003a60 2024-04-17T10:38:13.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.673+0000 7f0aba7b7640 1 -- 172.21.15.83:0/3147293608 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0ab415e940 con 0x7f0aa0003a60 2024-04-17T10:38:13.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.729+0000 7f0ab0ff9640 1 -- 172.21.15.83:0/3147293608 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0aa411f030 con 0x7f0aa0003a60 2024-04-17T10:38:13.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.733+0000 7f0aba7b7640 1 -- 172.21.15.83:0/3147293608 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0a94066f90 msgr2=0x7f0a94069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:13.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.733+0000 7f0aba7b7640 1 --2- 172.21.15.83:0/3147293608 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0a94066f90 0x7f0a94069450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4159ce0 tx=0x7f0a9c009040 comp rx=0 tx=0).stop 2024-04-17T10:38:13.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.733+0000 7f0aba7b7640 1 -- 172.21.15.83:0/3147293608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0aa0003a60 msgr2=0x7f0ab41f6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:13.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.733+0000 7f0aba7b7640 1 --2- 172.21.15.83:0/3147293608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0aa0003a60 0x7f0ab41f6ea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0aa401d0d0 tx=0x7f0aa0005840 comp rx=0 tx=0).stop 2024-04-17T10:38:13.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.733+0000 7f0aba7b7640 1 -- 172.21.15.83:0/3147293608 shutdown_connections 2024-04-17T10:38:13.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.733+0000 7f0aba7b7640 1 --2- 172.21.15.83:0/3147293608 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0a94066f90 0x7f0a94069450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:13.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.733+0000 7f0aba7b7640 1 --2- 172.21.15.83:0/3147293608 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ab4150ed0 0x7f0ab40ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:13.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.733+0000 7f0aba7b7640 1 --2- 172.21.15.83:0/3147293608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ab414cdf0 0x7f0ab41571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:13.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.733+0000 7f0aba7b7640 1 --2- 172.21.15.83:0/3147293608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0aa0003a60 0x7f0ab41f6ea0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:13.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.733+0000 7f0aba7b7640 1 -- 172.21.15.83:0/3147293608 >> 172.21.15.83:0/3147293608 conn(0x7f0ab40b15b0 msgr2=0x7f0ab406ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:13.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.733+0000 7f0aba7b7640 1 -- 172.21.15.83:0/3147293608 shutdown_connections 2024-04-17T10:38:13.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.733+0000 7f0aba7b7640 1 -- 172.21.15.83:0/3147293608 wait complete. 2024-04-17T10:38:13.745 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-17T10:38:13.745 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-17T10:38:13.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.869+0000 7f6c03d3a640 1 Processor -- start 2024-04-17T10:38:13.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.869+0000 7f6c03d3a640 1 -- start start 2024-04-17T10:38:13.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.869+0000 7f6c03d3a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6bfc154de0 0x7f6bfc06ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:13.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.869+0000 7f6c03d3a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bfc15bd70 con 0x7f6bfc06b350 2024-04-17T10:38:13.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.869+0000 7f6c03d3a640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bfc15bf60 con 0x7f6bfc0b4210 2024-04-17T10:38:13.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.869+0000 7f6c03d3a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc0b4870 con 0x7f6bfc154de0 2024-04-17T10:38:13.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.869+0000 7f6c012d5640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6bfc06b350 0x7f6bfc0b3b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42266/0 (socket says 172.21.15.83:42266) 2024-04-17T10:38:13.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.869+0000 7f6c012d5640 1 -- 172.21.15.83:0/789440393 learned_addr learned my addr 172.21.15.83:0/789440393 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:13.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.869+0000 7f6c00ad4640 1 -- 172.21.15.83:0/789440393 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 612090051 0 0) 0x7f6bfc15bd70 con 0x7f6bfc06b350 2024-04-17T10:38:13.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.869+0000 7f6c00ad4640 1 -- 172.21.15.83:0/789440393 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6be8005740 con 0x7f6bfc06b350 2024-04-17T10:38:13.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.869+0000 7f6c01ad6640 1 --2- 172.21.15.83:0/789440393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6bfc154de0 0x7f6bfc06ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:13.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.869+0000 7f6c00ad4640 1 -- 172.21.15.83:0/789440393 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1195848410 0 0) 0x7f6be8005740 con 0x7f6bfc06b350 2024-04-17T10:38:13.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.869+0000 7f6c00ad4640 1 -- 172.21.15.83:0/789440393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6bfc154de0 msgr2=0x7f6bfc06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:13.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.869+0000 7f6c00ad4640 1 --2- 172.21.15.83:0/789440393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6bfc154de0 0x7f6bfc06ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:13.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.869+0000 7f6c00ad4640 1 -- 172.21.15.83:0/789440393 >> v1:172.21.15.167:6789/0 conn(0x7f6bfc0b4210 legacy=0x7f6bfc06cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:13.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.869+0000 7f6c00ad4640 1 -- 172.21.15.83:0/789440393 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bfc15afb0 con 0x7f6bfc06b350 2024-04-17T10:38:13.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.869+0000 7f6c00ad4640 1 -- 172.21.15.83:0/789440393 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f6bec001680 con 0x7f6bfc06b350 2024-04-17T10:38:13.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.869+0000 7f6c00ad4640 1 -- 172.21.15.83:0/789440393 >> v1:172.21.15.83:6789/0 conn(0x7f6bfc06b350 legacy=0x7f6bfc0b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:13.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.869+0000 7f6c00ad4640 1 --2- 172.21.15.83:0/789440393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6bfc154de0 0x7f6be8005c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:13.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.869+0000 7f6c00ad4640 1 --2- 172.21.15.83:0/789440393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6be80062a0 0x7f6be8006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:13.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.869+0000 7f6c00ad4640 1 --2- 172.21.15.83:0/789440393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6be8006fb0 0x7f6be8007430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:13.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.869+0000 7f6c00ad4640 1 -- 172.21.15.83:0/789440393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be800ddc0 con 0x7f6be80062a0 2024-04-17T10:38:13.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.869+0000 7f6c00ad4640 1 -- 172.21.15.83:0/789440393 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be800df70 con 0x7f6be8006fb0 2024-04-17T10:38:13.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.869+0000 7f6c00ad4640 1 -- 172.21.15.83:0/789440393 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be800e170 con 0x7f6bfc154de0 2024-04-17T10:38:13.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.869+0000 7f6c012d5640 1 --2- 172.21.15.83:0/789440393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6be80062a0 0x7f6be8006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:13.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.869+0000 7f6c01ad6640 1 --2- 172.21.15.83:0/789440393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6bfc154de0 0x7f6be8005c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:13.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.869+0000 7f6c022d7640 1 --2- 172.21.15.83:0/789440393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6be8006fb0 0x7f6be8007430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:13.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.869+0000 7f6c012d5640 1 -- 172.21.15.83:0/789440393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6bfc154de0 msgr2=0x7f6be8005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:13.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.869+0000 7f6c012d5640 1 --2- 172.21.15.83:0/789440393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6bfc154de0 0x7f6be8005c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:13.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.869+0000 7f6c012d5640 1 -- 172.21.15.83:0/789440393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6be8006fb0 msgr2=0x7f6be8007430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:13.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.869+0000 7f6c012d5640 1 --2- 172.21.15.83:0/789440393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6be8006fb0 0x7f6be8007430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:13.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.869+0000 7f6c012d5640 1 -- 172.21.15.83:0/789440393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be8005740 con 0x7f6be80062a0 2024-04-17T10:38:13.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.873+0000 7f6c012d5640 1 --2- 172.21.15.83:0/789440393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6be80062a0 0x7f6be8006a20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6bec00c210 tx=0x7f6bec01f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d88c818282ad2b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:13.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.873+0000 7f6c00ad4640 1 -- 172.21.15.83:0/789440393 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bec007d40 con 0x7f6be80062a0 2024-04-17T10:38:13.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.873+0000 7f6c00ad4640 1 -- 172.21.15.83:0/789440393 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bec005b00 con 0x7f6be80062a0 2024-04-17T10:38:13.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.873+0000 7f6c00ad4640 1 -- 172.21.15.83:0/789440393 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bec007070 con 0x7f6be80062a0 2024-04-17T10:38:13.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.873+0000 7f6c03d3a640 1 -- 172.21.15.83:0/789440393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6be80062a0 msgr2=0x7f6be8006a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:13.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.873+0000 7f6c03d3a640 1 --2- 172.21.15.83:0/789440393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6be80062a0 0x7f6be8006a20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6bec00c210 tx=0x7f6bec01f3f0 comp rx=0 tx=0).stop 2024-04-17T10:38:13.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.873+0000 7f6c022d7640 1 -- 172.21.15.83:0/789440393 reap_dead start 2024-04-17T10:38:13.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.873+0000 7f6c03d3a640 1 -- 172.21.15.83:0/789440393 shutdown_connections 2024-04-17T10:38:13.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.873+0000 7f6c03d3a640 1 -- 172.21.15.83:0/789440393 >> 172.21.15.83:0/789440393 conn(0x7f6bfc0b15b0 msgr2=0x7f6bfc0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:13.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.873+0000 7f6c03d3a640 1 -- 172.21.15.83:0/789440393 shutdown_connections 2024-04-17T10:38:13.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.873+0000 7f6c03d3a640 1 -- 172.21.15.83:0/789440393 wait complete. 2024-04-17T10:38:13.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.873+0000 7f6c03d3a640 1 Processor -- start 2024-04-17T10:38:13.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.873+0000 7f6c03d3a640 1 -- start start 2024-04-17T10:38:13.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.873+0000 7f6c03d3a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6be80062a0 0x7f6bfc159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:13.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.873+0000 7f6c03d3a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6bfc156470 0x7f6bfc148f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:13.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.873+0000 7f6c03d3a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bfc0aff30 0x7f6bfc0b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:13.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.873+0000 7f6c03d3a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc15b850 con 0x7f6bfc0aff30 2024-04-17T10:38:13.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.873+0000 7f6c03d3a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc15b570 con 0x7f6be80062a0 2024-04-17T10:38:13.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.873+0000 7f6c03d3a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc1551c0 con 0x7f6bfc156470 2024-04-17T10:38:13.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.873+0000 7f6c022d7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bfc0aff30 0x7f6bfc0b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:13.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.873+0000 7f6c022d7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bfc0aff30 0x7f6bfc0b03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55904/0 (socket says 172.21.15.83:55904) 2024-04-17T10:38:13.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.873+0000 7f6c022d7640 1 -- 172.21.15.83:0/514800551 learned_addr learned my addr 172.21.15.83:0/514800551 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:13.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.877+0000 7f6c01ad6640 1 --2- 172.21.15.83:0/514800551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6be80062a0 0x7f6bfc159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:13.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.877+0000 7f6c012d5640 1 --2- 172.21.15.83:0/514800551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6bfc156470 0x7f6bfc148f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:13.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.877+0000 7f6c022d7640 1 -- 172.21.15.83:0/514800551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6bfc156470 msgr2=0x7f6bfc148f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:13.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.877+0000 7f6c022d7640 1 --2- 172.21.15.83:0/514800551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6bfc156470 0x7f6bfc148f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:13.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.877+0000 7f6c022d7640 1 -- 172.21.15.83:0/514800551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6be80062a0 msgr2=0x7f6bfc159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:13.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.877+0000 7f6c022d7640 1 --2- 172.21.15.83:0/514800551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6be80062a0 0x7f6bfc159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:13.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.877+0000 7f6c022d7640 1 -- 172.21.15.83:0/514800551 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bfc156970 con 0x7f6bfc0aff30 2024-04-17T10:38:13.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.877+0000 7f6c022d7640 1 --2- 172.21.15.83:0/514800551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bfc0aff30 0x7f6bfc0b03c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc15e2f0 tx=0x7f6bf400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:13.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.877+0000 7f6bf27fc640 1 -- 172.21.15.83:0/514800551 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4018070 con 0x7f6bfc0aff30 2024-04-17T10:38:13.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.877+0000 7f6bf27fc640 1 -- 172.21.15.83:0/514800551 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf4003b80 con 0x7f6bfc0aff30 2024-04-17T10:38:13.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.877+0000 7f6c03d3a640 1 -- 172.21.15.83:0/514800551 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bfc156c50 con 0x7f6bfc0aff30 2024-04-17T10:38:13.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.877+0000 7f6bf27fc640 1 -- 172.21.15.83:0/514800551 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4005070 con 0x7f6bfc0aff30 2024-04-17T10:38:13.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.877+0000 7f6c03d3a640 1 -- 172.21.15.83:0/514800551 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bfc157210 con 0x7f6bfc0aff30 2024-04-17T10:38:13.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.877+0000 7f6c03d3a640 1 -- 172.21.15.83:0/514800551 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bfc074f90 con 0x7f6bfc0aff30 2024-04-17T10:38:13.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.877+0000 7f6bf27fc640 1 -- 172.21.15.83:0/514800551 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6bf401b070 con 0x7f6bfc0aff30 2024-04-17T10:38:13.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.877+0000 7f6bf27fc640 1 --2- 172.21.15.83:0/514800551 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6be0066f60 0x7f6be0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:13.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.877+0000 7f6bf27fc640 1 -- 172.21.15.83:0/514800551 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6bf4084840 con 0x7f6bfc0aff30 2024-04-17T10:38:13.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.877+0000 7f6c01ad6640 1 --2- 172.21.15.83:0/514800551 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6be0066f60 0x7f6be0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:13.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.877+0000 7f6c01ad6640 1 --2- 172.21.15.83:0/514800551 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6be0066f60 0x7f6be0069420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc0b1040 tx=0x7f6bf8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:13.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:13.881+0000 7f6bf27fc640 1 -- 172.21.15.83:0/514800551 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6bf4055850 con 0x7f6bfc0aff30 2024-04-17T10:38:14.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.061+0000 7f6c03d3a640 1 -- 172.21.15.83:0/514800551 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6bfc0b3280 con 0x7f6bfc0aff30 2024-04-17T10:38:14.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.745+0000 7f6bf27fc640 1 -- 172.21.15.83:0/514800551 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6bf4055150 con 0x7f6bfc0aff30 2024-04-17T10:38:14.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.749+0000 7f6c03d3a640 1 -- 172.21.15.83:0/514800551 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6be0066f60 msgr2=0x7f6be0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:14.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.749+0000 7f6c03d3a640 1 --2- 172.21.15.83:0/514800551 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6be0066f60 0x7f6be0069420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc0b1040 tx=0x7f6bf8008040 comp rx=0 tx=0).stop 2024-04-17T10:38:14.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.749+0000 7f6c03d3a640 1 -- 172.21.15.83:0/514800551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bfc0aff30 msgr2=0x7f6bfc0b03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:14.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.749+0000 7f6c03d3a640 1 --2- 172.21.15.83:0/514800551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bfc0aff30 0x7f6bfc0b03c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc15e2f0 tx=0x7f6bf400dc00 comp rx=0 tx=0).stop 2024-04-17T10:38:14.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.749+0000 7f6c03d3a640 1 -- 172.21.15.83:0/514800551 shutdown_connections 2024-04-17T10:38:14.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.749+0000 7f6c03d3a640 1 --2- 172.21.15.83:0/514800551 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6be0066f60 0x7f6be0069420 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:14.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.749+0000 7f6c03d3a640 1 --2- 172.21.15.83:0/514800551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bfc0aff30 0x7f6bfc0b03c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:14.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.749+0000 7f6c03d3a640 1 --2- 172.21.15.83:0/514800551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6bfc156470 0x7f6bfc148f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:14.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.749+0000 7f6c03d3a640 1 --2- 172.21.15.83:0/514800551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6be80062a0 0x7f6bfc159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:14.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.749+0000 7f6c03d3a640 1 -- 172.21.15.83:0/514800551 >> 172.21.15.83:0/514800551 conn(0x7f6bfc0b15b0 msgr2=0x7f6bfc071470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:14.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.749+0000 7f6c03d3a640 1 -- 172.21.15.83:0/514800551 shutdown_connections 2024-04-17T10:38:14.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.749+0000 7f6c03d3a640 1 -- 172.21.15.83:0/514800551 wait complete. 2024-04-17T10:38:14.766 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-17T10:38:14.766 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:38:14.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.893+0000 7fbeedd91640 1 Processor -- start 2024-04-17T10:38:14.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.893+0000 7fbeedd91640 1 -- start start 2024-04-17T10:38:14.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.897+0000 7fbeedd91640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbee8072330 0x7fbee80cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:14.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.897+0000 7fbeedd91640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbee80cfa10 0x7fbee806c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:14.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.897+0000 7fbeedd91640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbee80cfea0 con 0x7fbee815ba30 2024-04-17T10:38:14.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.897+0000 7fbeedd91640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbee8071ea0 con 0x7fbee8072330 2024-04-17T10:38:14.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.897+0000 7fbeedd91640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee806e320 con 0x7fbee80cfa10 2024-04-17T10:38:14.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.897+0000 7fbee77fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbee815ba30 0x7fbee815c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42276/0 (socket says 172.21.15.83:42276) 2024-04-17T10:38:14.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.897+0000 7fbee77fe640 1 -- 172.21.15.83:0/1310465430 learned_addr learned my addr 172.21.15.83:0/1310465430 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:14.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.897+0000 7fbee6ffd640 1 --2- 172.21.15.83:0/1310465430 >> v2:172.21.15.83:3300/0 conn(0x7fbee8072330 0x7fbee80cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:14.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.897+0000 7fbee67fc640 1 -- 172.21.15.83:0/1310465430 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1145841080 0 0) 0x7fbee80cfea0 con 0x7fbee815ba30 2024-04-17T10:38:14.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.897+0000 7fbee67fc640 1 -- 172.21.15.83:0/1310465430 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbed0003530 con 0x7fbee815ba30 2024-04-17T10:38:14.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.897+0000 7fbee7fff640 1 --2- 172.21.15.83:0/1310465430 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbee80cfa10 0x7fbee806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:14.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.897+0000 7fbee6ffd640 1 -- 172.21.15.83:0/1310465430 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbee80cfa10 msgr2=0x7fbee806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:14.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.897+0000 7fbee6ffd640 1 --2- 172.21.15.83:0/1310465430 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbee80cfa10 0x7fbee806c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:14.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.897+0000 7fbee6ffd640 1 -- 172.21.15.83:0/1310465430 >> v1:172.21.15.83:6789/0 conn(0x7fbee815ba30 legacy=0x7fbee815c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:14.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.897+0000 7fbee6ffd640 1 -- 172.21.15.83:0/1310465430 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee806e040 con 0x7fbee8072330 2024-04-17T10:38:14.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.897+0000 7fbee6ffd640 1 --2- 172.21.15.83:0/1310465430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbee8072330 0x7fbee80cf4d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbee8072020 tx=0x7fbedc01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a23605f88c1a55 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:14.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.897+0000 7fbee67fc640 1 -- 172.21.15.83:0/1310465430 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc0b0070 con 0x7fbee8072330 2024-04-17T10:38:14.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.897+0000 7fbee67fc640 1 -- 172.21.15.83:0/1310465430 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbedc0ab600 con 0x7fbee8072330 2024-04-17T10:38:14.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.897+0000 7fbee67fc640 1 -- 172.21.15.83:0/1310465430 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc0ab9d0 con 0x7fbee8072330 2024-04-17T10:38:14.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.901+0000 7fbeedd91640 1 -- 172.21.15.83:0/1310465430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbee8072330 msgr2=0x7fbee80cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:14.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.901+0000 7fbeedd91640 1 --2- 172.21.15.83:0/1310465430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbee8072330 0x7fbee80cf4d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbee8072020 tx=0x7fbedc01c410 comp rx=0 tx=0).stop 2024-04-17T10:38:14.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.901+0000 7fbeedd91640 1 -- 172.21.15.83:0/1310465430 shutdown_connections 2024-04-17T10:38:14.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.901+0000 7fbeedd91640 1 --2- 172.21.15.83:0/1310465430 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbee80cfa10 0x7fbee806c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:14.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.901+0000 7fbeedd91640 1 --2- 172.21.15.83:0/1310465430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbee8072330 0x7fbee80cf4d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:14.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.901+0000 7fbeedd91640 1 -- 172.21.15.83:0/1310465430 >> 172.21.15.83:0/1310465430 conn(0x7fbee806abe0 msgr2=0x7fbee806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:14.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.901+0000 7fbeedd91640 1 -- 172.21.15.83:0/1310465430 shutdown_connections 2024-04-17T10:38:14.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.901+0000 7fbeedd91640 1 -- 172.21.15.83:0/1310465430 wait complete. 2024-04-17T10:38:14.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.901+0000 7fbeedd91640 1 Processor -- start 2024-04-17T10:38:14.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.901+0000 7fbeedd91640 1 -- start start 2024-04-17T10:38:14.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.901+0000 7fbeedd91640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbee8072330 0x7fbee80ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:14.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.901+0000 7fbeedd91640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbee80cfa10 0x7fbee80add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:14.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.901+0000 7fbeedd91640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbee815ba30 0x7fbee814fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:14.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.901+0000 7fbeedd91640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee806de20 con 0x7fbee815ba30 2024-04-17T10:38:14.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.901+0000 7fbeedd91640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee806dca0 con 0x7fbee8072330 2024-04-17T10:38:14.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.901+0000 7fbeedd91640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee8168520 con 0x7fbee80cfa10 2024-04-17T10:38:14.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.901+0000 7fbee7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbee815ba30 0x7fbee814fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:14.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.901+0000 7fbee7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbee815ba30 0x7fbee814fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55920/0 (socket says 172.21.15.83:55920) 2024-04-17T10:38:14.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.901+0000 7fbee7fff640 1 -- 172.21.15.83:0/4089989006 learned_addr learned my addr 172.21.15.83:0/4089989006 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:14.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.901+0000 7fbee6ffd640 1 --2- 172.21.15.83:0/4089989006 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbee80cfa10 0x7fbee80add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:14.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.901+0000 7fbee77fe640 1 --2- 172.21.15.83:0/4089989006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbee8072330 0x7fbee80ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:14.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.901+0000 7fbee7fff640 1 -- 172.21.15.83:0/4089989006 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbee80cfa10 msgr2=0x7fbee80add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:14.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.901+0000 7fbee7fff640 1 --2- 172.21.15.83:0/4089989006 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbee80cfa10 0x7fbee80add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:14.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.901+0000 7fbee7fff640 1 -- 172.21.15.83:0/4089989006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbee8072330 msgr2=0x7fbee80ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:14.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.901+0000 7fbee7fff640 1 --2- 172.21.15.83:0/4089989006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbee8072330 0x7fbee80ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:14.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.901+0000 7fbee7fff640 1 -- 172.21.15.83:0/4089989006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee8150dd0 con 0x7fbee815ba30 2024-04-17T10:38:14.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.901+0000 7fbee7fff640 1 --2- 172.21.15.83:0/4089989006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbee815ba30 0x7fbee814fae0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbed8007e40 tx=0x7fbed800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:14.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.901+0000 7fbecffff640 1 -- 172.21.15.83:0/4089989006 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed8014620 con 0x7fbee815ba30 2024-04-17T10:38:14.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.901+0000 7fbecffff640 1 -- 172.21.15.83:0/4089989006 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed8019070 con 0x7fbee815ba30 2024-04-17T10:38:14.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.901+0000 7fbeedd91640 1 -- 172.21.15.83:0/4089989006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee81510b0 con 0x7fbee815ba30 2024-04-17T10:38:14.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.901+0000 7fbecffff640 1 -- 172.21.15.83:0/4089989006 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed8014a20 con 0x7fbee815ba30 2024-04-17T10:38:14.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.901+0000 7fbeedd91640 1 -- 172.21.15.83:0/4089989006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee815e510 con 0x7fbee815ba30 2024-04-17T10:38:14.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.905+0000 7fbeedd91640 1 -- 172.21.15.83:0/4089989006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbee8074f90 con 0x7fbee815ba30 2024-04-17T10:38:14.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.905+0000 7fbecffff640 1 -- 172.21.15.83:0/4089989006 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbed8014bc0 con 0x7fbee815ba30 2024-04-17T10:38:14.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.905+0000 7fbecffff640 1 --2- 172.21.15.83:0/4089989006 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbed4066f60 0x7fbed4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:14.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.905+0000 7fbecffff640 1 -- 172.21.15.83:0/4089989006 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbed8085940 con 0x7fbee815ba30 2024-04-17T10:38:14.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.905+0000 7fbee77fe640 1 --2- 172.21.15.83:0/4089989006 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbed4066f60 0x7fbed4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:14.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.905+0000 7fbee77fe640 1 --2- 172.21.15.83:0/4089989006 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbed4066f60 0x7fbed4069420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbee0005890 tx=0x7fbee0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:14.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:14.909+0000 7fbecffff640 1 -- 172.21.15.83:0/4089989006 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbed8056950 con 0x7fbee815ba30 2024-04-17T10:38:15.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.085+0000 7fbeedd91640 1 -- 172.21.15.83:0/4089989006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbee8074480 con 0x7fbee815ba30 2024-04-17T10:38:15.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.085+0000 7fbecffff640 1 -- 172.21.15.83:0/4089989006 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4938 (secure 0 0 0) 0x7fbed8056250 con 0x7fbee815ba30 2024-04-17T10:38:15.082 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:15.082 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2884469968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2884469968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2884469968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2919571009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2919571009},{"type":"v1","addr":"172.21.15.83:6836","nonce":2919571009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1635176834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1635176834},{"type":"v1","addr":"172.21.15.83:6837","nonce":1635176834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:38:12.032695+0000","modified":"2024-04-17T10:38:14.064978+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6835/3871588351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3871588351},{"type":"v1","addr":"172.21.15.167:6835","nonce":3871588351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-17T10:38:15.083 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 6 2024-04-17T10:38:15.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.089+0000 7fbeedd91640 1 -- 172.21.15.83:0/4089989006 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbed4066f60 msgr2=0x7fbed4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:15.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.089+0000 7fbeedd91640 1 --2- 172.21.15.83:0/4089989006 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbed4066f60 0x7fbed4069420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbee0005890 tx=0x7fbee0009040 comp rx=0 tx=0).stop 2024-04-17T10:38:15.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.089+0000 7fbeedd91640 1 -- 172.21.15.83:0/4089989006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbee815ba30 msgr2=0x7fbee814fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:15.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.089+0000 7fbeedd91640 1 --2- 172.21.15.83:0/4089989006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbee815ba30 0x7fbee814fae0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbed8007e40 tx=0x7fbed800e4b0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.089+0000 7fbeedd91640 1 -- 172.21.15.83:0/4089989006 shutdown_connections 2024-04-17T10:38:15.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.089+0000 7fbeedd91640 1 --2- 172.21.15.83:0/4089989006 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbed4066f60 0x7fbed4069420 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.089+0000 7fbeedd91640 1 --2- 172.21.15.83:0/4089989006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbee815ba30 0x7fbee814fae0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.089+0000 7fbeedd91640 1 --2- 172.21.15.83:0/4089989006 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbee80cfa10 0x7fbee80add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.089+0000 7fbeedd91640 1 --2- 172.21.15.83:0/4089989006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbee8072330 0x7fbee80ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.089+0000 7fbeedd91640 1 -- 172.21.15.83:0/4089989006 >> 172.21.15.83:0/4089989006 conn(0x7fbee806abe0 msgr2=0x7fbee80cefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:15.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.089+0000 7fbeedd91640 1 -- 172.21.15.83:0/4089989006 shutdown_connections 2024-04-17T10:38:15.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.089+0000 7fbeedd91640 1 -- 172.21.15.83:0/4089989006 wait complete. 2024-04-17T10:38:15.100 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:38:15.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.229+0000 7fb6cfd68640 1 Processor -- start 2024-04-17T10:38:15.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.229+0000 7fb6cfd68640 1 -- start start 2024-04-17T10:38:15.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.229+0000 7fb6cfd68640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb6c815db50 0x7fb6c8070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:15.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.229+0000 7fb6cfd68640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6c806dcb0 0x7fb6c80b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:15.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.229+0000 7fb6cfd68640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6c80709b0 con 0x7fb6c80aedd0 2024-04-17T10:38:15.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.229+0000 7fb6cfd68640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6c815e440 con 0x7fb6c815db50 2024-04-17T10:38:15.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.229+0000 7fb6cfd68640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c815e720 con 0x7fb6c806dcb0 2024-04-17T10:38:15.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.229+0000 7fb6cdb04640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb6c815db50 0x7fb6c8070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:15.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.229+0000 7fb6ce305640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6c806dcb0 0x7fb6c80b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:15.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.229+0000 7fb6cdb04640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb6c815db50 0x7fb6c8070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:43154/0 (socket says 172.21.15.83:43154) 2024-04-17T10:38:15.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.229+0000 7fb6cdb04640 1 -- 172.21.15.83:0/1848472846 learned_addr learned my addr 172.21.15.83:0/1848472846 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:15.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.229+0000 7fb6cd303640 1 --1- 172.21.15.83:0/1848472846 >> v1:172.21.15.167:6789/0 conn(0x7fb6c80aedd0 0x7fb6c806d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:38182/0 (socket says 172.21.15.83:38182) 2024-04-17T10:38:15.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.229+0000 7fb6cdb04640 1 -- 172.21.15.83:0/1848472846 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6c806dcb0 msgr2=0x7fb6c80b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:15.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.229+0000 7fb6cdb04640 1 --2- 172.21.15.83:0/1848472846 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6c806dcb0 0x7fb6c80b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.229+0000 7fb6cdb04640 1 -- 172.21.15.83:0/1848472846 >> v1:172.21.15.167:6789/0 conn(0x7fb6c80aedd0 legacy=0x7fb6c806d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:15.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.229+0000 7fb6cdb04640 1 -- 172.21.15.83:0/1848472846 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c815ea00 con 0x7fb6c815db50 2024-04-17T10:38:15.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.233+0000 7fb6cdb04640 1 --2- 172.21.15.83:0/1848472846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6c815db50 0x7fb6c8070470 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb6c400bb10 tx=0x7fb6c401ec00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51d471265d50c32a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:15.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.233+0000 7fb6ccb02640 1 -- 172.21.15.83:0/1848472846 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c40b4070 con 0x7fb6c815db50 2024-04-17T10:38:15.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.233+0000 7fb6ccb02640 1 -- 172.21.15.83:0/1848472846 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c40a7cd0 con 0x7fb6c815db50 2024-04-17T10:38:15.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.233+0000 7fb6ccb02640 1 -- 172.21.15.83:0/1848472846 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c40af660 con 0x7fb6c815db50 2024-04-17T10:38:15.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.233+0000 7fb6cfd68640 1 -- 172.21.15.83:0/1848472846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6c815db50 msgr2=0x7fb6c8070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:15.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.233+0000 7fb6cfd68640 1 --2- 172.21.15.83:0/1848472846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6c815db50 0x7fb6c8070470 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb6c400bb10 tx=0x7fb6c401ec00 comp rx=0 tx=0).stop 2024-04-17T10:38:15.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.233+0000 7fb6cfd68640 1 -- 172.21.15.83:0/1848472846 shutdown_connections 2024-04-17T10:38:15.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.233+0000 7fb6cfd68640 1 --2- 172.21.15.83:0/1848472846 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6c806dcb0 0x7fb6c80b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.233+0000 7fb6cfd68640 1 --2- 172.21.15.83:0/1848472846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6c815db50 0x7fb6c8070470 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.233+0000 7fb6cfd68640 1 -- 172.21.15.83:0/1848472846 >> 172.21.15.83:0/1848472846 conn(0x7fb6c806abe0 msgr2=0x7fb6c806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:15.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.233+0000 7fb6cfd68640 1 -- 172.21.15.83:0/1848472846 shutdown_connections 2024-04-17T10:38:15.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.233+0000 7fb6cfd68640 1 -- 172.21.15.83:0/1848472846 wait complete. 2024-04-17T10:38:15.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.233+0000 7fb6cfd68640 1 Processor -- start 2024-04-17T10:38:15.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.233+0000 7fb6cfd68640 1 -- start start 2024-04-17T10:38:15.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.237+0000 7fb6cfd68640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6c806dcb0 0x7fb6c815c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:15.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.237+0000 7fb6cfd68640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6c80aedd0 0x7fb6c814a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:15.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.237+0000 7fb6cfd68640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6c815db50 0x7fb6c80ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:15.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.237+0000 7fb6cfd68640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c80b3ce0 con 0x7fb6c806dcb0 2024-04-17T10:38:15.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.237+0000 7fb6cfd68640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c80b3b60 con 0x7fb6c80aedd0 2024-04-17T10:38:15.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.237+0000 7fb6cfd68640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c80b3e60 con 0x7fb6c815db50 2024-04-17T10:38:15.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.237+0000 7fb6cdb04640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6c806dcb0 0x7fb6c815c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:15.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.237+0000 7fb6cdb04640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6c806dcb0 0x7fb6c815c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55934/0 (socket says 172.21.15.83:55934) 2024-04-17T10:38:15.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.237+0000 7fb6cdb04640 1 -- 172.21.15.83:0/1845524676 learned_addr learned my addr 172.21.15.83:0/1845524676 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:15.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.237+0000 7fb6cd303640 1 --2- 172.21.15.83:0/1845524676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6c80aedd0 0x7fb6c814a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:15.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.237+0000 7fb6ce305640 1 --2- 172.21.15.83:0/1845524676 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6c815db50 0x7fb6c80ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:15.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.237+0000 7fb6cdb04640 1 -- 172.21.15.83:0/1845524676 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6c815db50 msgr2=0x7fb6c80ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:15.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.237+0000 7fb6cdb04640 1 --2- 172.21.15.83:0/1845524676 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6c815db50 0x7fb6c80ccc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.237+0000 7fb6cdb04640 1 -- 172.21.15.83:0/1845524676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6c80aedd0 msgr2=0x7fb6c814a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:15.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.237+0000 7fb6cdb04640 1 --2- 172.21.15.83:0/1845524676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6c80aedd0 0x7fb6c814a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.237+0000 7fb6cdb04640 1 -- 172.21.15.83:0/1845524676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c815b550 con 0x7fb6c806dcb0 2024-04-17T10:38:15.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.237+0000 7fb6cdb04640 1 --2- 172.21.15.83:0/1845524676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6c806dcb0 0x7fb6c815c000 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb6c400b2c0 tx=0x7fb6c401b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:15.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.237+0000 7fb6be7fc640 1 -- 172.21.15.83:0/1845524676 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c40be070 con 0x7fb6c806dcb0 2024-04-17T10:38:15.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.237+0000 7fb6be7fc640 1 -- 172.21.15.83:0/1845524676 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c40afc30 con 0x7fb6c806dcb0 2024-04-17T10:38:15.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.237+0000 7fb6cfd68640 1 -- 172.21.15.83:0/1845524676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6c815d040 con 0x7fb6c806dcb0 2024-04-17T10:38:15.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.237+0000 7fb6be7fc640 1 -- 172.21.15.83:0/1845524676 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c40b4040 con 0x7fb6c806dcb0 2024-04-17T10:38:15.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.237+0000 7fb6cfd68640 1 -- 172.21.15.83:0/1845524676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6c815c540 con 0x7fb6c806dcb0 2024-04-17T10:38:15.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.237+0000 7fb6cfd68640 1 -- 172.21.15.83:0/1845524676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6c815bc30 con 0x7fb6c806dcb0 2024-04-17T10:38:15.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.237+0000 7fb6be7fc640 1 -- 172.21.15.83:0/1845524676 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb6c40c1070 con 0x7fb6c806dcb0 2024-04-17T10:38:15.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.237+0000 7fb6be7fc640 1 --2- 172.21.15.83:0/1845524676 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb6ac066f60 0x7fb6ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:15.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.237+0000 7fb6be7fc640 1 -- 172.21.15.83:0/1845524676 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6c4122180 con 0x7fb6c806dcb0 2024-04-17T10:38:15.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.237+0000 7fb6cd303640 1 --2- 172.21.15.83:0/1845524676 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb6ac066f60 0x7fb6ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:15.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.241+0000 7fb6cd303640 1 --2- 172.21.15.83:0/1845524676 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb6ac066f60 0x7fb6ac069420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb6c80ae810 tx=0x7fb6b8007cd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:15.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.245+0000 7fb6be7fc640 1 -- 172.21.15.83:0/1845524676 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb6c4018820 con 0x7fb6c806dcb0 2024-04-17T10:38:15.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.392+0000 7fb6cfd68640 1 -- 172.21.15.83:0/1845524676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6c8074f90 con 0x7fb6c806dcb0 2024-04-17T10:38:15.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.392+0000 7fb6be7fc640 1 -- 172.21.15.83:0/1845524676 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16533 (secure 0 0 0) 0x7fb6c40f3340 con 0x7fb6c806dcb0 2024-04-17T10:38:15.388 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:15.389 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":20,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:38:13.690876+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:38:10.464490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T10:38:11.043134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:38:15.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.392+0000 7fb6cfd68640 1 -- 172.21.15.83:0/1845524676 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb6ac066f60 msgr2=0x7fb6ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:15.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.392+0000 7fb6cfd68640 1 --2- 172.21.15.83:0/1845524676 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb6ac066f60 0x7fb6ac069420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb6c80ae810 tx=0x7fb6b8007cd0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.392+0000 7fb6cfd68640 1 -- 172.21.15.83:0/1845524676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6c806dcb0 msgr2=0x7fb6c815c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:15.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.392+0000 7fb6cfd68640 1 --2- 172.21.15.83:0/1845524676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6c806dcb0 0x7fb6c815c000 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb6c400b2c0 tx=0x7fb6c401b9a0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.392+0000 7fb6cfd68640 1 -- 172.21.15.83:0/1845524676 shutdown_connections 2024-04-17T10:38:15.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.392+0000 7fb6cfd68640 1 --2- 172.21.15.83:0/1845524676 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb6ac066f60 0x7fb6ac069420 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.392+0000 7fb6cfd68640 1 --2- 172.21.15.83:0/1845524676 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6c815db50 0x7fb6c80ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.392+0000 7fb6cfd68640 1 --2- 172.21.15.83:0/1845524676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6c80aedd0 0x7fb6c814a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.392+0000 7fb6cfd68640 1 --2- 172.21.15.83:0/1845524676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6c806dcb0 0x7fb6c815c000 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.392+0000 7fb6cfd68640 1 -- 172.21.15.83:0/1845524676 >> 172.21.15.83:0/1845524676 conn(0x7fb6c806abe0 msgr2=0x7fb6c806c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:15.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.396+0000 7fb6cfd68640 1 -- 172.21.15.83:0/1845524676 shutdown_connections 2024-04-17T10:38:15.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.396+0000 7fb6cfd68640 1 -- 172.21.15.83:0/1845524676 wait complete. 2024-04-17T10:38:15.405 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-17T10:38:15.406 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:38:15.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f682b8db640 1 Processor -- start 2024-04-17T10:38:15.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f682b8db640 1 -- start start 2024-04-17T10:38:15.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f682b8db640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f682415ba30 0x7f6824072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:15.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f682b8db640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f682406e320 con 0x7f68240cfbe0 2024-04-17T10:38:15.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f682b8db640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f682406e040 con 0x7f682415c960 2024-04-17T10:38:15.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f682b8db640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6824071ea0 con 0x7f682415ba30 2024-04-17T10:38:15.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f6829e78640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f68240cfbe0 0x7f682406c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42288/0 (socket says 172.21.15.83:42288) 2024-04-17T10:38:15.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f6829e78640 1 -- 172.21.15.83:0/1507070525 learned_addr learned my addr 172.21.15.83:0/1507070525 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:15.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f6829677640 1 --2- 172.21.15.83:0/1507070525 >> v2:172.21.15.167:3300/0 conn(0x7f682415ba30 0x7f6824072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:15.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f681bfff640 1 -- 172.21.15.83:0/1507070525 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2368810517 0 0) 0x7f682406e320 con 0x7f68240cfbe0 2024-04-17T10:38:15.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f681bfff640 1 -- 172.21.15.83:0/1507070525 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6810003530 con 0x7f68240cfbe0 2024-04-17T10:38:15.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f681bfff640 1 -- 172.21.15.83:0/1507070525 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3883624119 0 0) 0x7f6810003530 con 0x7f68240cfbe0 2024-04-17T10:38:15.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f681bfff640 1 -- 172.21.15.83:0/1507070525 >> v2:172.21.15.167:3300/0 conn(0x7f682415ba30 msgr2=0x7f6824072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:15.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f681bfff640 1 --2- 172.21.15.83:0/1507070525 >> v2:172.21.15.167:3300/0 conn(0x7f682415ba30 0x7f6824072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f681bfff640 1 -- 172.21.15.83:0/1507070525 >> v1:172.21.15.167:6789/0 conn(0x7f682415c960 legacy=0x7f68240cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:15.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f681bfff640 1 -- 172.21.15.83:0/1507070525 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6824071850 con 0x7f68240cfbe0 2024-04-17T10:38:15.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f681bfff640 1 -- 172.21.15.83:0/1507070525 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f681c001a20 con 0x7f68240cfbe0 2024-04-17T10:38:15.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f681bfff640 1 -- 172.21.15.83:0/1507070525 >> v1:172.21.15.83:6789/0 conn(0x7f68240cfbe0 legacy=0x7f682406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:15.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f681bfff640 1 --2- 172.21.15.83:0/1507070525 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68100039c0 0x7f6810003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:15.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f681bfff640 1 --2- 172.21.15.83:0/1507070525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6810004810 0x7f6810004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:15.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f681bfff640 1 --2- 172.21.15.83:0/1507070525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68100051d0 0x7f681000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:15.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f681bfff640 1 -- 172.21.15.83:0/1507070525 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6824071ea0 con 0x7f68100051d0 2024-04-17T10:38:15.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f681bfff640 1 -- 172.21.15.83:0/1507070525 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681000c170 con 0x7f6810004810 2024-04-17T10:38:15.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f681bfff640 1 -- 172.21.15.83:0/1507070525 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f681000c370 con 0x7f68100039c0 2024-04-17T10:38:15.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f6829e78640 1 --2- 172.21.15.83:0/1507070525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68100051d0 0x7f681000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:15.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f6828e76640 1 --2- 172.21.15.83:0/1507070525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6810004810 0x7f6810004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:15.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f6829677640 1 --2- 172.21.15.83:0/1507070525 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68100039c0 0x7f6810003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:15.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f6829e78640 1 -- 172.21.15.83:0/1507070525 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68100039c0 msgr2=0x7f6810003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:15.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f6829e78640 1 --2- 172.21.15.83:0/1507070525 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68100039c0 0x7f6810003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f6829e78640 1 -- 172.21.15.83:0/1507070525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6810004810 msgr2=0x7f6810004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:15.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f6829e78640 1 --2- 172.21.15.83:0/1507070525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6810004810 0x7f6810004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f6829e78640 1 -- 172.21.15.83:0/1507070525 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6810003530 con 0x7f68100051d0 2024-04-17T10:38:15.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.516+0000 7f6828e76640 1 --2- 172.21.15.83:0/1507070525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6810004810 0x7f6810004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:38:15.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.520+0000 7f6829e78640 1 -- 172.21.15.83:0/1507070525 reap_dead start 2024-04-17T10:38:15.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.520+0000 7f6829e78640 1 --2- 172.21.15.83:0/1507070525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68100051d0 0x7f681000b9d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f681c00a850 tx=0x7f681c01fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b0011e7376bf8b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:15.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.520+0000 7f681bfff640 1 -- 172.21.15.83:0/1507070525 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c00a4a0 con 0x7f68100051d0 2024-04-17T10:38:15.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.520+0000 7f681bfff640 1 -- 172.21.15.83:0/1507070525 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681c0aa4f0 con 0x7f68100051d0 2024-04-17T10:38:15.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.520+0000 7f681bfff640 1 -- 172.21.15.83:0/1507070525 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c0aa8c0 con 0x7f68100051d0 2024-04-17T10:38:15.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.520+0000 7f682b8db640 1 -- 172.21.15.83:0/1507070525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68100051d0 msgr2=0x7f681000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:15.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.520+0000 7f682b8db640 1 --2- 172.21.15.83:0/1507070525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68100051d0 0x7f681000b9d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f681c00a850 tx=0x7f681c01fac0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.520+0000 7f682b8db640 1 -- 172.21.15.83:0/1507070525 shutdown_connections 2024-04-17T10:38:15.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.520+0000 7f682b8db640 1 --2- 172.21.15.83:0/1507070525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68100051d0 0x7f681000b9d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.520+0000 7f682b8db640 1 -- 172.21.15.83:0/1507070525 >> 172.21.15.83:0/1507070525 conn(0x7f682406abe0 msgr2=0x7f682406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:15.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.520+0000 7f682b8db640 1 -- 172.21.15.83:0/1507070525 shutdown_connections 2024-04-17T10:38:15.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.520+0000 7f682b8db640 1 -- 172.21.15.83:0/1507070525 wait complete. 2024-04-17T10:38:15.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.520+0000 7f682b8db640 1 Processor -- start 2024-04-17T10:38:15.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.520+0000 7f682b8db640 1 -- start start 2024-04-17T10:38:15.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.524+0000 7f682b8db640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68100051d0 0x7f682415eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:15.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.524+0000 7f682b8db640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68240b2770 0x7f682414b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:15.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.524+0000 7f682b8db640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6824154600 0x7f68240ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:15.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.524+0000 7f682b8db640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68240aeae0 con 0x7f68240b2770 2024-04-17T10:38:15.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.524+0000 7f682b8db640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682415cde0 con 0x7f6824154600 2024-04-17T10:38:15.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.524+0000 7f682b8db640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f682406dca0 con 0x7f68100051d0 2024-04-17T10:38:15.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.524+0000 7f6828e76640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68240b2770 0x7f682414b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:15.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.524+0000 7f6828e76640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68240b2770 0x7f682414b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55956/0 (socket says 172.21.15.83:55956) 2024-04-17T10:38:15.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.524+0000 7f6828e76640 1 -- 172.21.15.83:0/2747024681 learned_addr learned my addr 172.21.15.83:0/2747024681 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:15.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.524+0000 7f6829677640 1 --2- 172.21.15.83:0/2747024681 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68100051d0 0x7f682415eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:15.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.524+0000 7f6829e78640 1 --2- 172.21.15.83:0/2747024681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6824154600 0x7f68240ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:15.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.524+0000 7f6828e76640 1 -- 172.21.15.83:0/2747024681 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68100051d0 msgr2=0x7f682415eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:15.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.524+0000 7f6828e76640 1 --2- 172.21.15.83:0/2747024681 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68100051d0 0x7f682415eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.524+0000 7f6828e76640 1 -- 172.21.15.83:0/2747024681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6824154600 msgr2=0x7f68240ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:15.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.524+0000 7f6828e76640 1 --2- 172.21.15.83:0/2747024681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6824154600 0x7f68240ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.524+0000 7f6828e76640 1 -- 172.21.15.83:0/2747024681 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682415efe0 con 0x7f68240b2770 2024-04-17T10:38:15.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.524+0000 7f6829e78640 1 --2- 172.21.15.83:0/2747024681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6824154600 0x7f68240ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:15.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.524+0000 7f6828e76640 1 --2- 172.21.15.83:0/2747024681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68240b2770 0x7f682414b370 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f681400e480 tx=0x7f681400e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:15.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.524+0000 7f6819ffb640 1 -- 172.21.15.83:0/2747024681 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68140052e0 con 0x7f68240b2770 2024-04-17T10:38:15.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.524+0000 7f6819ffb640 1 -- 172.21.15.83:0/2747024681 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6814018070 con 0x7f68240b2770 2024-04-17T10:38:15.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.524+0000 7f682b8db640 1 -- 172.21.15.83:0/2747024681 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68240b2d10 con 0x7f68240b2770 2024-04-17T10:38:15.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.524+0000 7f6819ffb640 1 -- 172.21.15.83:0/2747024681 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6814005480 con 0x7f68240b2770 2024-04-17T10:38:15.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.524+0000 7f682b8db640 1 -- 172.21.15.83:0/2747024681 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68240b32d0 con 0x7f68240b2770 2024-04-17T10:38:15.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.524+0000 7f682b8db640 1 -- 172.21.15.83:0/2747024681 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6824074f90 con 0x7f68240b2770 2024-04-17T10:38:15.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.524+0000 7f6819ffb640 1 -- 172.21.15.83:0/2747024681 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6814003820 con 0x7f68240b2770 2024-04-17T10:38:15.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.524+0000 7f6819ffb640 1 --2- 172.21.15.83:0/2747024681 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6808066f60 0x7f6808069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:15.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.524+0000 7f6829677640 1 --2- 172.21.15.83:0/2747024681 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6808066f60 0x7f6808069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:15.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.524+0000 7f6819ffb640 1 -- 172.21.15.83:0/2747024681 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6814084c60 con 0x7f68240b2770 2024-04-17T10:38:15.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.524+0000 7f6829677640 1 --2- 172.21.15.83:0/2747024681 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6808066f60 0x7f6808069420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f68240b4910 tx=0x7f6820008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:15.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.528+0000 7f6819ffb640 1 -- 172.21.15.83:0/2747024681 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f682415efe0 con 0x7f68240b2770 2024-04-17T10:38:15.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.672+0000 7f682b8db640 1 -- 172.21.15.83:0/2747024681 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f682415add0 con 0x7f6808066f60 2024-04-17T10:38:15.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.688+0000 7f6819ffb640 1 -- 172.21.15.83:0/2747024681 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+237143 (secure 0 0 0) 0x7f682415add0 con 0x7f6808066f60 2024-04-17T10:38:15.687 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:15.689 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:38:15.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.692+0000 7f682b8db640 1 -- 172.21.15.83:0/2747024681 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6808066f60 msgr2=0x7f6808069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:15.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.692+0000 7f682b8db640 1 --2- 172.21.15.83:0/2747024681 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6808066f60 0x7f6808069420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f68240b4910 tx=0x7f6820008040 comp rx=0 tx=0).stop 2024-04-17T10:38:15.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.692+0000 7f682b8db640 1 -- 172.21.15.83:0/2747024681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68240b2770 msgr2=0x7f682414b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:15.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.692+0000 7f682b8db640 1 --2- 172.21.15.83:0/2747024681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68240b2770 0x7f682414b370 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f681400e480 tx=0x7f681400e940 comp rx=0 tx=0).stop 2024-04-17T10:38:15.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.692+0000 7f682b8db640 1 -- 172.21.15.83:0/2747024681 shutdown_connections 2024-04-17T10:38:15.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.692+0000 7f682b8db640 1 --2- 172.21.15.83:0/2747024681 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6808066f60 0x7f6808069420 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.692+0000 7f682b8db640 1 --2- 172.21.15.83:0/2747024681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6824154600 0x7f68240ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.692+0000 7f682b8db640 1 --2- 172.21.15.83:0/2747024681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68240b2770 0x7f682414b370 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.692+0000 7f682b8db640 1 --2- 172.21.15.83:0/2747024681 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68100051d0 0x7f682415eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.692+0000 7f682b8db640 1 -- 172.21.15.83:0/2747024681 >> 172.21.15.83:0/2747024681 conn(0x7f682406abe0 msgr2=0x7f682415be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:15.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.692+0000 7f682b8db640 1 -- 172.21.15.83:0/2747024681 shutdown_connections 2024-04-17T10:38:15.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.692+0000 7f682b8db640 1 -- 172.21.15.83:0/2747024681 wait complete. 2024-04-17T10:38:15.703 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-17T10:38:14.180339+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":218800,"kb_used_data":2376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755920,"statfs":{"total":773094113280,"available":772870062080,"internally_reserved":0,"allocated":2433024,"data_stored":1546872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001028"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-17T10:38:03.660615+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:38:03.660615+0000","last_peered":"2024-04-17T10:38:03.660615+0000","last_clean":"2024-04-17T10:38:03.660615+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:38:03.660615+0000","last_undegraded":"2024-04-17T10:38:03.660615+0000","last_fullsized":"2024-04-17T10:38:03.660615+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:38:15.707 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:38:15.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.868+0000 7f8c0285a640 1 Processor -- start 2024-04-17T10:38:15.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.868+0000 7f8c0285a640 1 -- start start 2024-04-17T10:38:15.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.868+0000 7f8c0285a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8bfc15db50 0x7f8bfc070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:15.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.868+0000 7f8c0285a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8bfc0aedd0 0x7f8bfc06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:15.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.868+0000 7f8c0285a640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8bfc06dae0 0x7f8bfc0cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:15.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.868+0000 7f8c0285a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bfc0709b0 con 0x7f8bfc15db50 2024-04-17T10:38:15.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.868+0000 7f8c0285a640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bfc15e440 con 0x7f8bfc06dae0 2024-04-17T10:38:15.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.868+0000 7f8c0285a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc15e720 con 0x7f8bfc0aedd0 2024-04-17T10:38:15.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.868+0000 7f8bfbfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8bfc15db50 0x7f8bfc070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:15.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.868+0000 7f8bfbfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8bfc15db50 0x7f8bfc070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55968/0 (socket says 172.21.15.83:55968) 2024-04-17T10:38:15.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.868+0000 7f8bfbfff640 1 -- 172.21.15.83:0/3189203374 learned_addr learned my addr 172.21.15.83:0/3189203374 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:15.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.868+0000 7f8bfb7fe640 1 --2- 172.21.15.83:0/3189203374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8bfc0aedd0 0x7f8bfc06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:15.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.868+0000 7f8c00df7640 1 --2- 172.21.15.83:0/3189203374 >> v2:172.21.15.167:3300/0 conn(0x7f8bfc06dae0 0x7f8bfc0cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:15.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.868+0000 7f8bfbfff640 1 -- 172.21.15.83:0/3189203374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8bfc0aedd0 msgr2=0x7f8bfc06d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:15.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.868+0000 7f8bfbfff640 1 --2- 172.21.15.83:0/3189203374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8bfc0aedd0 0x7f8bfc06d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.868+0000 7f8bfbfff640 1 -- 172.21.15.83:0/3189203374 >> v2:172.21.15.167:3300/0 conn(0x7f8bfc06dae0 msgr2=0x7f8bfc0cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:15.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.868+0000 7f8bfbfff640 1 --2- 172.21.15.83:0/3189203374 >> v2:172.21.15.167:3300/0 conn(0x7f8bfc06dae0 0x7f8bfc0cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.868+0000 7f8bfbfff640 1 -- 172.21.15.83:0/3189203374 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bfc15ea00 con 0x7f8bfc15db50 2024-04-17T10:38:15.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.872+0000 7f8bfbfff640 1 --2- 172.21.15.83:0/3189203374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc15db50 0x7f8bfc070470 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc0b2ac0 tx=0x7f8bec01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7021189f592c77de server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:15.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.872+0000 7f8bfaffd640 1 -- 172.21.15.83:0/3189203374 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec0b0070 con 0x7f8bfc15db50 2024-04-17T10:38:15.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.872+0000 7f8bfaffd640 1 -- 172.21.15.83:0/3189203374 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bec0ab600 con 0x7f8bfc15db50 2024-04-17T10:38:15.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.872+0000 7f8bfaffd640 1 -- 172.21.15.83:0/3189203374 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec0ab9d0 con 0x7f8bfc15db50 2024-04-17T10:38:15.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.872+0000 7f8c0285a640 1 -- 172.21.15.83:0/3189203374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc15db50 msgr2=0x7f8bfc070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:15.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.872+0000 7f8c0285a640 1 --2- 172.21.15.83:0/3189203374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc15db50 0x7f8bfc070470 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc0b2ac0 tx=0x7f8bec01c260 comp rx=0 tx=0).stop 2024-04-17T10:38:15.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.872+0000 7f8c0285a640 1 -- 172.21.15.83:0/3189203374 shutdown_connections 2024-04-17T10:38:15.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.872+0000 7f8c0285a640 1 --2- 172.21.15.83:0/3189203374 >> v2:172.21.15.167:3300/0 conn(0x7f8bfc06dae0 0x7f8bfc0cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.872+0000 7f8c0285a640 1 --2- 172.21.15.83:0/3189203374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8bfc0aedd0 0x7f8bfc06d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.872+0000 7f8c0285a640 1 --2- 172.21.15.83:0/3189203374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc15db50 0x7f8bfc070470 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.872+0000 7f8c0285a640 1 -- 172.21.15.83:0/3189203374 >> 172.21.15.83:0/3189203374 conn(0x7f8bfc06abe0 msgr2=0x7f8bfc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:15.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.872+0000 7f8c0285a640 1 -- 172.21.15.83:0/3189203374 shutdown_connections 2024-04-17T10:38:15.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.872+0000 7f8c0285a640 1 -- 172.21.15.83:0/3189203374 wait complete. 2024-04-17T10:38:15.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.872+0000 7f8c0285a640 1 Processor -- start 2024-04-17T10:38:15.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.872+0000 7f8c0285a640 1 -- start start 2024-04-17T10:38:15.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.872+0000 7f8c0285a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bfc06dae0 0x7f8bfc13a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:15.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.872+0000 7f8c0285a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8bfc0aedd0 0x7f8bfc14ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:15.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.872+0000 7f8c0285a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc15db50 0x7f8bfc151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:15.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.872+0000 7f8c0285a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc0b2f90 con 0x7f8bfc15db50 2024-04-17T10:38:15.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.872+0000 7f8c0285a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc0b2e10 con 0x7f8bfc06dae0 2024-04-17T10:38:15.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.872+0000 7f8c0285a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc0b3110 con 0x7f8bfc0aedd0 2024-04-17T10:38:15.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.872+0000 7f8c00df7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc15db50 0x7f8bfc151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:15.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.872+0000 7f8c00df7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc15db50 0x7f8bfc151f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55976/0 (socket says 172.21.15.83:55976) 2024-04-17T10:38:15.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.872+0000 7f8c00df7640 1 -- 172.21.15.83:0/3007988582 learned_addr learned my addr 172.21.15.83:0/3007988582 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:15.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.872+0000 7f8bfb7fe640 1 --2- 172.21.15.83:0/3007988582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8bfc0aedd0 0x7f8bfc14ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:15.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.872+0000 7f8bfbfff640 1 --2- 172.21.15.83:0/3007988582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bfc06dae0 0x7f8bfc13a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:15.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.872+0000 7f8c00df7640 1 -- 172.21.15.83:0/3007988582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8bfc0aedd0 msgr2=0x7f8bfc14ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:15.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.872+0000 7f8c00df7640 1 --2- 172.21.15.83:0/3007988582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8bfc0aedd0 0x7f8bfc14ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.872+0000 7f8c00df7640 1 -- 172.21.15.83:0/3007988582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bfc06dae0 msgr2=0x7f8bfc13a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:15.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.872+0000 7f8c00df7640 1 --2- 172.21.15.83:0/3007988582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bfc06dae0 0x7f8bfc13a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:15.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.872+0000 7f8c00df7640 1 -- 172.21.15.83:0/3007988582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bfc14f7d0 con 0x7f8bfc15db50 2024-04-17T10:38:15.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.872+0000 7f8c00df7640 1 --2- 172.21.15.83:0/3007988582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc15db50 0x7f8bfc151f00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4007e90 tx=0x7f8bf400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:15.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.876+0000 7f8bf8ff9640 1 -- 172.21.15.83:0/3007988582 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4014420 con 0x7f8bfc15db50 2024-04-17T10:38:15.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.876+0000 7f8bf8ff9640 1 -- 172.21.15.83:0/3007988582 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf4019070 con 0x7f8bfc15db50 2024-04-17T10:38:15.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.876+0000 7f8c0285a640 1 -- 172.21.15.83:0/3007988582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bfc14fab0 con 0x7f8bfc15db50 2024-04-17T10:38:15.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.876+0000 7f8bf8ff9640 1 -- 172.21.15.83:0/3007988582 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4014820 con 0x7f8bfc15db50 2024-04-17T10:38:15.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.876+0000 7f8c0285a640 1 -- 172.21.15.83:0/3007988582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bfc150070 con 0x7f8bfc15db50 2024-04-17T10:38:15.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.876+0000 7f8c0285a640 1 -- 172.21.15.83:0/3007988582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bfc074f90 con 0x7f8bfc15db50 2024-04-17T10:38:15.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.876+0000 7f8bf8ff9640 1 -- 172.21.15.83:0/3007988582 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8bf40149c0 con 0x7f8bfc15db50 2024-04-17T10:38:15.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.876+0000 7f8bf8ff9640 1 --2- 172.21.15.83:0/3007988582 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8be8066f60 0x7f8be8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:15.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.876+0000 7f8bf8ff9640 1 -- 172.21.15.83:0/3007988582 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8bf4085870 con 0x7f8bfc15db50 2024-04-17T10:38:15.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.876+0000 7f8bfbfff640 1 --2- 172.21.15.83:0/3007988582 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8be8066f60 0x7f8be8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:15.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.876+0000 7f8bfbfff640 1 --2- 172.21.15.83:0/3007988582 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8be8066f60 0x7f8be8069420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8bec0ad000 tx=0x7f8bec0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:15.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:15.880+0000 7f8bf8ff9640 1 -- 172.21.15.83:0/3007988582 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8bf4056910 con 0x7f8bfc15db50 2024-04-17T10:38:16.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.036+0000 7f8c0285a640 1 -- 172.21.15.83:0/3007988582 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8bfc0b2af0 con 0x7f8be8066f60 2024-04-17T10:38:16.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.044+0000 7f8bf8ff9640 1 -- 172.21.15.83:0/3007988582 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+237143 (secure 0 0 0) 0x7f8bfc0b2af0 con 0x7f8be8066f60 2024-04-17T10:38:16.043 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:16.045 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:38:16.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.048+0000 7f8c0285a640 1 -- 172.21.15.83:0/3007988582 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8be8066f60 msgr2=0x7f8be8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:16.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.048+0000 7f8c0285a640 1 --2- 172.21.15.83:0/3007988582 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8be8066f60 0x7f8be8069420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8bec0ad000 tx=0x7f8bec0a6000 comp rx=0 tx=0).stop 2024-04-17T10:38:16.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.048+0000 7f8c0285a640 1 -- 172.21.15.83:0/3007988582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc15db50 msgr2=0x7f8bfc151f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:16.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.048+0000 7f8c0285a640 1 --2- 172.21.15.83:0/3007988582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc15db50 0x7f8bfc151f00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4007e90 tx=0x7f8bf400e4b0 comp rx=0 tx=0).stop 2024-04-17T10:38:16.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.048+0000 7f8c0285a640 1 -- 172.21.15.83:0/3007988582 shutdown_connections 2024-04-17T10:38:16.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.048+0000 7f8c0285a640 1 --2- 172.21.15.83:0/3007988582 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8be8066f60 0x7f8be8069420 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:16.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.048+0000 7f8c0285a640 1 --2- 172.21.15.83:0/3007988582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc15db50 0x7f8bfc151f00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:16.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.048+0000 7f8c0285a640 1 --2- 172.21.15.83:0/3007988582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8bfc0aedd0 0x7f8bfc14ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:16.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.048+0000 7f8c0285a640 1 --2- 172.21.15.83:0/3007988582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bfc06dae0 0x7f8bfc13a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:16.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.048+0000 7f8c0285a640 1 -- 172.21.15.83:0/3007988582 >> 172.21.15.83:0/3007988582 conn(0x7f8bfc06abe0 msgr2=0x7f8bfc0af1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:16.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.048+0000 7f8c0285a640 1 -- 172.21.15.83:0/3007988582 shutdown_connections 2024-04-17T10:38:16.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.048+0000 7f8c0285a640 1 -- 172.21.15.83:0/3007988582 wait complete. 2024-04-17T10:38:16.061 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-17T10:38:14.180339+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":218800,"kb_used_data":2376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755920,"statfs":{"total":773094113280,"available":772870062080,"internally_reserved":0,"allocated":2433024,"data_stored":1546872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001028"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-17T10:38:03.660615+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:38:03.660615+0000","last_peered":"2024-04-17T10:38:03.660615+0000","last_clean":"2024-04-17T10:38:03.660615+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:38:03.660615+0000","last_undegraded":"2024-04-17T10:38:03.660615+0000","last_fullsized":"2024-04-17T10:38:03.660615+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:11.668582+0000","last_change":"2024-04-17T10:38:11.668582+0000","last_active":"2024-04-17T10:38:11.668582+0000","last_peered":"2024-04-17T10:38:11.668582+0000","last_clean":"2024-04-17T10:38:11.668582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:11.668582+0000","last_undegraded":"2024-04-17T10:38:11.668582+0000","last_fullsized":"2024-04-17T10:38:11.668582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:38:10.660092+0000","last_change":"2024-04-17T10:38:10.660092+0000","last_active":"2024-04-17T10:38:10.660092+0000","last_peered":"2024-04-17T10:38:10.660092+0000","last_clean":"2024-04-17T10:38:10.660092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:38:10.660092+0000","last_undegraded":"2024-04-17T10:38:10.660092+0000","last_fullsized":"2024-04-17T10:38:10.660092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:38:16.065 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:38:16.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.228+0000 7fdeede6e640 1 Processor -- start 2024-04-17T10:38:16.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.228+0000 7fdeede6e640 1 -- start start 2024-04-17T10:38:16.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.232+0000 7fdeede6e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdee815ddd0 0x7fdee80d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:16.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.232+0000 7fdeede6e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee80d18c0 con 0x7fdee80cf5f0 2024-04-17T10:38:16.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.232+0000 7fdeede6e640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee80d1b60 con 0x7fdee80b2f70 2024-04-17T10:38:16.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.232+0000 7fdeede6e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee80d1e40 con 0x7fdee815ddd0 2024-04-17T10:38:16.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.232+0000 7fdee7fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fdee80cf5f0 0x7fdee806be50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42294/0 (socket says 172.21.15.83:42294) 2024-04-17T10:38:16.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.232+0000 7fdee7fff640 1 -- 172.21.15.83:0/2961000829 learned_addr learned my addr 172.21.15.83:0/2961000829 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.232+0000 7fdee77fe640 1 --2- 172.21.15.83:0/2961000829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdee815ddd0 0x7fdee80d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.232+0000 7fdee67fc640 1 -- 172.21.15.83:0/2961000829 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1389669630 0 0) 0x7fdee80d18c0 con 0x7fdee80cf5f0 2024-04-17T10:38:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.232+0000 7fdee67fc640 1 -- 172.21.15.83:0/2961000829 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fded0003530 con 0x7fdee80cf5f0 2024-04-17T10:38:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.232+0000 7fdee67fc640 1 -- 172.21.15.83:0/2961000829 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3801810010 0 0) 0x7fded0003530 con 0x7fdee80cf5f0 2024-04-17T10:38:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.232+0000 7fdee67fc640 1 -- 172.21.15.83:0/2961000829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdee815ddd0 msgr2=0x7fdee80d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.232+0000 7fdee67fc640 1 --2- 172.21.15.83:0/2961000829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdee815ddd0 0x7fdee80d2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.232+0000 7fdee67fc640 1 -- 172.21.15.83:0/2961000829 >> v1:172.21.15.167:6789/0 conn(0x7fdee80b2f70 legacy=0x7fdee806b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.232+0000 7fdee67fc640 1 -- 172.21.15.83:0/2961000829 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee806b220 con 0x7fdee80cf5f0 2024-04-17T10:38:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.232+0000 7fdee67fc640 1 -- 172.21.15.83:0/2961000829 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fdee0001a00 con 0x7fdee80cf5f0 2024-04-17T10:38:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.232+0000 7fdee67fc640 1 -- 172.21.15.83:0/2961000829 >> v1:172.21.15.83:6789/0 conn(0x7fdee80cf5f0 legacy=0x7fdee806be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.232+0000 7fdee67fc640 1 --2- 172.21.15.83:0/2961000829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fded00039f0 0x7fded0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.232+0000 7fdee67fc640 1 --2- 172.21.15.83:0/2961000829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fded0004810 0x7fded0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.232+0000 7fdee67fc640 1 --2- 172.21.15.83:0/2961000829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdee815ddd0 0x7fded00052a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.232+0000 7fdee67fc640 1 -- 172.21.15.83:0/2961000829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee80d1e40 con 0x7fded00039f0 2024-04-17T10:38:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.232+0000 7fdee67fc640 1 -- 172.21.15.83:0/2961000829 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded000bb90 con 0x7fded0004810 2024-04-17T10:38:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.232+0000 7fdee67fc640 1 -- 172.21.15.83:0/2961000829 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded000bd90 con 0x7fdee815ddd0 2024-04-17T10:38:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.232+0000 7fdee77fe640 1 --2- 172.21.15.83:0/2961000829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fded00039f0 0x7fded0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.232+0000 7fdee6ffd640 1 --2- 172.21.15.83:0/2961000829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fded0004810 0x7fded0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.232+0000 7fdee7fff640 1 --2- 172.21.15.83:0/2961000829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdee815ddd0 0x7fded00052a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.232+0000 7fdee77fe640 1 -- 172.21.15.83:0/2961000829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdee815ddd0 msgr2=0x7fded00052a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.232+0000 7fdee77fe640 1 --2- 172.21.15.83:0/2961000829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdee815ddd0 0x7fded00052a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.232+0000 7fdee77fe640 1 -- 172.21.15.83:0/2961000829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fded0004810 msgr2=0x7fded0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.232+0000 7fdee77fe640 1 --2- 172.21.15.83:0/2961000829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fded0004810 0x7fded0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.232+0000 7fdee77fe640 1 -- 172.21.15.83:0/2961000829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded0003530 con 0x7fded00039f0 2024-04-17T10:38:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.232+0000 7fdee6ffd640 1 --2- 172.21.15.83:0/2961000829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fded0004810 0x7fded0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.232+0000 7fdee7fff640 1 --2- 172.21.15.83:0/2961000829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdee815ddd0 0x7fded00052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:16.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdee77fe640 1 --2- 172.21.15.83:0/2961000829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fded00039f0 0x7fded0003e10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fded8009ae0 tx=0x7fded801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b55f58a2380a04b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:16.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdee67fc640 1 -- 172.21.15.83:0/2961000829 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded80096b0 con 0x7fded00039f0 2024-04-17T10:38:16.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdee67fc640 1 -- 172.21.15.83:0/2961000829 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded80a6450 con 0x7fded00039f0 2024-04-17T10:38:16.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdee67fc640 1 -- 172.21.15.83:0/2961000829 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded80a6840 con 0x7fded00039f0 2024-04-17T10:38:16.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdeede6e640 1 -- 172.21.15.83:0/2961000829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fded00039f0 msgr2=0x7fded0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:16.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdeede6e640 1 --2- 172.21.15.83:0/2961000829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fded00039f0 0x7fded0003e10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fded8009ae0 tx=0x7fded801ca70 comp rx=0 tx=0).stop 2024-04-17T10:38:16.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdee7fff640 1 -- 172.21.15.83:0/2961000829 reap_dead start 2024-04-17T10:38:16.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdeede6e640 1 -- 172.21.15.83:0/2961000829 shutdown_connections 2024-04-17T10:38:16.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdeede6e640 1 -- 172.21.15.83:0/2961000829 >> 172.21.15.83:0/2961000829 conn(0x7fdee80d3a40 msgr2=0x7fdee80d3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:16.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdeede6e640 1 -- 172.21.15.83:0/2961000829 shutdown_connections 2024-04-17T10:38:16.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdeede6e640 1 -- 172.21.15.83:0/2961000829 wait complete. 2024-04-17T10:38:16.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdeede6e640 1 Processor -- start 2024-04-17T10:38:16.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdeede6e640 1 -- start start 2024-04-17T10:38:16.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdeede6e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fded00039f0 0x7fdee8156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:16.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdeede6e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdee80b4d60 0x7fdee80c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:16.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdeede6e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdee80c7ab0 0x7fdee8144a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:16.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdeede6e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee815e330 con 0x7fdee80b4d60 2024-04-17T10:38:16.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdeede6e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee815e1b0 con 0x7fded00039f0 2024-04-17T10:38:16.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdeede6e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee8169a50 con 0x7fdee80c7ab0 2024-04-17T10:38:16.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdee6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdee80b4d60 0x7fdee80c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:16.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdee6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdee80b4d60 0x7fdee80c7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55990/0 (socket says 172.21.15.83:55990) 2024-04-17T10:38:16.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdee6ffd640 1 -- 172.21.15.83:0/2372211244 learned_addr learned my addr 172.21.15.83:0/2372211244 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:16.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdee6ffd640 1 -- 172.21.15.83:0/2372211244 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdee80c7ab0 msgr2=0x7fdee8144a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:16.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdee6ffd640 1 --2- 172.21.15.83:0/2372211244 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdee80c7ab0 0x7fdee8144a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:16.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdee6ffd640 1 -- 172.21.15.83:0/2372211244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fded00039f0 msgr2=0x7fdee8156890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:16.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdee77fe640 1 --2- 172.21.15.83:0/2372211244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fded00039f0 0x7fdee8156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:16.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdee6ffd640 1 --2- 172.21.15.83:0/2372211244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fded00039f0 0x7fdee8156890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:16.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdee6ffd640 1 -- 172.21.15.83:0/2372211244 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee80b5230 con 0x7fdee80b4d60 2024-04-17T10:38:16.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdee6ffd640 1 --2- 172.21.15.83:0/2372211244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdee80b4d60 0x7fdee80c7570 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdedc000c10 tx=0x7fdedc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:16.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdecffff640 1 -- 172.21.15.83:0/2372211244 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc019070 con 0x7fdee80b4d60 2024-04-17T10:38:16.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdecffff640 1 -- 172.21.15.83:0/2372211244 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdedc005270 con 0x7fdee80b4d60 2024-04-17T10:38:16.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdecffff640 1 -- 172.21.15.83:0/2372211244 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc007030 con 0x7fdee80b4d60 2024-04-17T10:38:16.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.236+0000 7fdeede6e640 1 -- 172.21.15.83:0/2372211244 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee80b5510 con 0x7fdee80b4d60 2024-04-17T10:38:16.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.240+0000 7fdeede6e640 1 -- 172.21.15.83:0/2372211244 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee80b5ad0 con 0x7fdee80b4d60 2024-04-17T10:38:16.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.240+0000 7fdeede6e640 1 -- 172.21.15.83:0/2372211244 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee80b5e10 con 0x7fdee80b4d60 2024-04-17T10:38:16.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.240+0000 7fdecffff640 1 -- 172.21.15.83:0/2372211244 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdedc01c070 con 0x7fdee80b4d60 2024-04-17T10:38:16.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.240+0000 7fdecffff640 1 --2- 172.21.15.83:0/2372211244 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fded4066f90 0x7fded4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:16.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.240+0000 7fdecffff640 1 -- 172.21.15.83:0/2372211244 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdedc084960 con 0x7fdee80b4d60 2024-04-17T10:38:16.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.240+0000 7fdee77fe640 1 --2- 172.21.15.83:0/2372211244 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fded4066f90 0x7fded4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:16.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.240+0000 7fdee77fe640 1 --2- 172.21.15.83:0/2372211244 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fded4066f90 0x7fded4069450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fded80ad000 tx=0x7fded80046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:16.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.244+0000 7fdecffff640 1 -- 172.21.15.83:0/2372211244 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdee80b5e10 con 0x7fdee80b4d60 2024-04-17T10:38:16.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.388+0000 7fdeede6e640 1 -- 172.21.15.83:0/2372211244 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdee815d270 con 0x7fded4066f90 2024-04-17T10:38:16.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.408+0000 7fdecffff640 1 -- 172.21.15.83:0/2372211244 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251946 (secure 0 0 0) 0x7fdee815d270 con 0x7fded4066f90 2024-04-17T10:38:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:16.407 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:38:16.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.412+0000 7fdeede6e640 1 -- 172.21.15.83:0/2372211244 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fded4066f90 msgr2=0x7fded4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:16.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.412+0000 7fdeede6e640 1 --2- 172.21.15.83:0/2372211244 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fded4066f90 0x7fded4069450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fded80ad000 tx=0x7fded80046d0 comp rx=0 tx=0).stop 2024-04-17T10:38:16.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.412+0000 7fdeede6e640 1 -- 172.21.15.83:0/2372211244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdee80b4d60 msgr2=0x7fdee80c7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:16.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.412+0000 7fdeede6e640 1 --2- 172.21.15.83:0/2372211244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdee80b4d60 0x7fdee80c7570 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdedc000c10 tx=0x7fdedc00dc00 comp rx=0 tx=0).stop 2024-04-17T10:38:16.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.412+0000 7fdeede6e640 1 -- 172.21.15.83:0/2372211244 shutdown_connections 2024-04-17T10:38:16.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.412+0000 7fdeede6e640 1 --2- 172.21.15.83:0/2372211244 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fded4066f90 0x7fded4069450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:16.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.412+0000 7fdeede6e640 1 --2- 172.21.15.83:0/2372211244 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdee80c7ab0 0x7fdee8144a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:16.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.412+0000 7fdeede6e640 1 --2- 172.21.15.83:0/2372211244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdee80b4d60 0x7fdee80c7570 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:16.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.412+0000 7fdeede6e640 1 --2- 172.21.15.83:0/2372211244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fded00039f0 0x7fdee8156890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:16.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.412+0000 7fdeede6e640 1 -- 172.21.15.83:0/2372211244 >> 172.21.15.83:0/2372211244 conn(0x7fdee80d3a40 msgr2=0x7fdee8168d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:16.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.412+0000 7fdeede6e640 1 -- 172.21.15.83:0/2372211244 shutdown_connections 2024-04-17T10:38:16.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.412+0000 7fdeede6e640 1 -- 172.21.15.83:0/2372211244 wait complete. 2024-04-17T10:38:16.423 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-04-17T10:38:16.180839+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218964,"kb_used_data":2776,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755756,"statfs":{"total":773094113280,"available":772869894144,"internally_reserved":0,"allocated":2842624,"data_stored":1727828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.510640"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702753+0000","last_change":"2024-04-17T10:38:12.718400+0000","last_active":"2024-04-17T10:38:13.702753+0000","last_peered":"2024-04-17T10:38:13.702753+0000","last_clean":"2024-04-17T10:38:13.702753+0000","last_became_active":"2024-04-17T10:38:12.718121+0000","last_became_peered":"2024-04-17T10:38:12.718121+0000","last_unstale":"2024-04-17T10:38:13.702753+0000","last_undegraded":"2024-04-17T10:38:13.702753+0000","last_fullsized":"2024-04-17T10:38:13.702753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:37.203406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695895+0000","last_change":"2024-04-17T10:38:12.710479+0000","last_active":"2024-04-17T10:38:13.695895+0000","last_peered":"2024-04-17T10:38:13.695895+0000","last_clean":"2024-04-17T10:38:13.695895+0000","last_became_active":"2024-04-17T10:38:12.710092+0000","last_became_peered":"2024-04-17T10:38:12.710092+0000","last_unstale":"2024-04-17T10:38:13.695895+0000","last_undegraded":"2024-04-17T10:38:13.695895+0000","last_fullsized":"2024-04-17T10:38:13.695895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:43.917742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305973+0000","last_change":"2024-04-17T10:38:12.722110+0000","last_active":"2024-04-17T10:38:14.305973+0000","last_peered":"2024-04-17T10:38:14.305973+0000","last_clean":"2024-04-17T10:38:14.305973+0000","last_became_active":"2024-04-17T10:38:12.721720+0000","last_became_peered":"2024-04-17T10:38:12.721720+0000","last_unstale":"2024-04-17T10:38:14.305973+0000","last_undegraded":"2024-04-17T10:38:14.305973+0000","last_fullsized":"2024-04-17T10:38:14.305973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:43.917742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970398+0000","last_change":"2024-04-17T10:38:12.727777+0000","last_active":"2024-04-17T10:38:13.970398+0000","last_peered":"2024-04-17T10:38:13.970398+0000","last_clean":"2024-04-17T10:38:13.970398+0000","last_became_active":"2024-04-17T10:38:12.726952+0000","last_became_peered":"2024-04-17T10:38:12.726952+0000","last_unstale":"2024-04-17T10:38:13.970398+0000","last_undegraded":"2024-04-17T10:38:13.970398+0000","last_fullsized":"2024-04-17T10:38:13.970398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:06.508830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701762+0000","last_change":"2024-04-17T10:38:12.721627+0000","last_active":"2024-04-17T10:38:13.701762+0000","last_peered":"2024-04-17T10:38:13.701762+0000","last_clean":"2024-04-17T10:38:13.701762+0000","last_became_active":"2024-04-17T10:38:12.718951+0000","last_became_peered":"2024-04-17T10:38:12.718951+0000","last_unstale":"2024-04-17T10:38:13.701762+0000","last_undegraded":"2024-04-17T10:38:13.701762+0000","last_fullsized":"2024-04-17T10:38:13.701762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:02.184227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970163+0000","last_change":"2024-04-17T10:38:12.729603+0000","last_active":"2024-04-17T10:38:13.970163+0000","last_peered":"2024-04-17T10:38:13.970163+0000","last_clean":"2024-04-17T10:38:13.970163+0000","last_became_active":"2024-04-17T10:38:12.728738+0000","last_became_peered":"2024-04-17T10:38:12.728738+0000","last_unstale":"2024-04-17T10:38:13.970163+0000","last_undegraded":"2024-04-17T10:38:13.970163+0000","last_fullsized":"2024-04-17T10:38:13.970163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:54:26.054113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701635+0000","last_change":"2024-04-17T10:38:12.708732+0000","last_active":"2024-04-17T10:38:13.701635+0000","last_peered":"2024-04-17T10:38:13.701635+0000","last_clean":"2024-04-17T10:38:13.701635+0000","last_became_active":"2024-04-17T10:38:12.708291+0000","last_became_peered":"2024-04-17T10:38:12.708291+0000","last_unstale":"2024-04-17T10:38:13.701635+0000","last_undegraded":"2024-04-17T10:38:13.701635+0000","last_fullsized":"2024-04-17T10:38:13.701635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:54:26.054113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297197+0000","last_change":"2024-04-17T10:38:12.726672+0000","last_active":"2024-04-17T10:38:14.297197+0000","last_peered":"2024-04-17T10:38:14.297197+0000","last_clean":"2024-04-17T10:38:14.297197+0000","last_became_active":"2024-04-17T10:38:12.726395+0000","last_became_peered":"2024-04-17T10:38:12.726395+0000","last_unstale":"2024-04-17T10:38:14.297197+0000","last_undegraded":"2024-04-17T10:38:14.297197+0000","last_fullsized":"2024-04-17T10:38:14.297197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:31:48.663733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702702+0000","last_change":"2024-04-17T10:38:12.727392+0000","last_active":"2024-04-17T10:38:13.702702+0000","last_peered":"2024-04-17T10:38:13.702702+0000","last_clean":"2024-04-17T10:38:13.702702+0000","last_became_active":"2024-04-17T10:38:12.727171+0000","last_became_peered":"2024-04-17T10:38:12.727171+0000","last_unstale":"2024-04-17T10:38:13.702702+0000","last_undegraded":"2024-04-17T10:38:13.702702+0000","last_fullsized":"2024-04-17T10:38:13.702702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:11:49.847070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803824+0000","last_change":"2024-04-17T10:38:12.726158+0000","last_active":"2024-04-17T10:38:13.803824+0000","last_peered":"2024-04-17T10:38:13.803824+0000","last_clean":"2024-04-17T10:38:13.803824+0000","last_became_active":"2024-04-17T10:38:12.724920+0000","last_became_peered":"2024-04-17T10:38:12.724920+0000","last_unstale":"2024-04-17T10:38:13.803824+0000","last_undegraded":"2024-04-17T10:38:13.803824+0000","last_fullsized":"2024-04-17T10:38:13.803824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:43.917742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803453+0000","last_change":"2024-04-17T10:38:12.725420+0000","last_active":"2024-04-17T10:38:13.803453+0000","last_peered":"2024-04-17T10:38:13.803453+0000","last_clean":"2024-04-17T10:38:13.803453+0000","last_became_active":"2024-04-17T10:38:12.724959+0000","last_became_peered":"2024-04-17T10:38:12.724959+0000","last_unstale":"2024-04-17T10:38:13.803453+0000","last_undegraded":"2024-04-17T10:38:13.803453+0000","last_fullsized":"2024-04-17T10:38:13.803453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:39.640730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.699955+0000","last_change":"2024-04-17T10:38:12.712887+0000","last_active":"2024-04-17T10:38:13.699955+0000","last_peered":"2024-04-17T10:38:13.699955+0000","last_clean":"2024-04-17T10:38:13.699955+0000","last_became_active":"2024-04-17T10:38:12.712582+0000","last_became_peered":"2024-04-17T10:38:12.712582+0000","last_unstale":"2024-04-17T10:38:13.699955+0000","last_undegraded":"2024-04-17T10:38:13.699955+0000","last_fullsized":"2024-04-17T10:38:13.699955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:23:30.964531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298210+0000","last_change":"2024-04-17T10:38:12.722191+0000","last_active":"2024-04-17T10:38:14.298210+0000","last_peered":"2024-04-17T10:38:14.298210+0000","last_clean":"2024-04-17T10:38:14.298210+0000","last_became_active":"2024-04-17T10:38:12.721912+0000","last_became_peered":"2024-04-17T10:38:12.721912+0000","last_unstale":"2024-04-17T10:38:14.298210+0000","last_undegraded":"2024-04-17T10:38:14.298210+0000","last_fullsized":"2024-04-17T10:38:14.298210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:09:07.090758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.304662+0000","last_change":"2024-04-17T10:38:12.710211+0000","last_active":"2024-04-17T10:38:14.304662+0000","last_peered":"2024-04-17T10:38:14.304662+0000","last_clean":"2024-04-17T10:38:14.304662+0000","last_became_active":"2024-04-17T10:38:12.709818+0000","last_became_peered":"2024-04-17T10:38:12.709818+0000","last_unstale":"2024-04-17T10:38:14.304662+0000","last_undegraded":"2024-04-17T10:38:14.304662+0000","last_fullsized":"2024-04-17T10:38:14.304662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:23:26.747986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695484+0000","last_change":"2024-04-17T10:38:12.722762+0000","last_active":"2024-04-17T10:38:13.695484+0000","last_peered":"2024-04-17T10:38:13.695484+0000","last_clean":"2024-04-17T10:38:13.695484+0000","last_became_active":"2024-04-17T10:38:12.718394+0000","last_became_peered":"2024-04-17T10:38:12.718394+0000","last_unstale":"2024-04-17T10:38:13.695484+0000","last_undegraded":"2024-04-17T10:38:13.695484+0000","last_fullsized":"2024-04-17T10:38:13.695484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:08.872431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.969994+0000","last_change":"2024-04-17T10:38:12.729270+0000","last_active":"2024-04-17T10:38:13.969994+0000","last_peered":"2024-04-17T10:38:13.969994+0000","last_clean":"2024-04-17T10:38:13.969994+0000","last_became_active":"2024-04-17T10:38:12.728798+0000","last_became_peered":"2024-04-17T10:38:12.728798+0000","last_unstale":"2024-04-17T10:38:13.969994+0000","last_undegraded":"2024-04-17T10:38:13.969994+0000","last_fullsized":"2024-04-17T10:38:13.969994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:23.465112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.971229+0000","last_change":"2024-04-17T10:38:12.731612+0000","last_active":"2024-04-17T10:38:13.971229+0000","last_peered":"2024-04-17T10:38:13.971229+0000","last_clean":"2024-04-17T10:38:13.971229+0000","last_became_active":"2024-04-17T10:38:12.731071+0000","last_became_peered":"2024-04-17T10:38:12.731071+0000","last_unstale":"2024-04-17T10:38:13.971229+0000","last_undegraded":"2024-04-17T10:38:13.971229+0000","last_fullsized":"2024-04-17T10:38:13.971229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:37.203406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.303651+0000","last_change":"2024-04-17T10:38:12.720913+0000","last_active":"2024-04-17T10:38:14.303651+0000","last_peered":"2024-04-17T10:38:14.303651+0000","last_clean":"2024-04-17T10:38:14.303651+0000","last_became_active":"2024-04-17T10:38:12.719779+0000","last_became_peered":"2024-04-17T10:38:12.719779+0000","last_unstale":"2024-04-17T10:38:14.303651+0000","last_undegraded":"2024-04-17T10:38:14.303651+0000","last_fullsized":"2024-04-17T10:38:14.303651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:23:26.747986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702068+0000","last_change":"2024-04-17T10:38:12.719229+0000","last_active":"2024-04-17T10:38:13.702068+0000","last_peered":"2024-04-17T10:38:13.702068+0000","last_clean":"2024-04-17T10:38:13.702068+0000","last_became_active":"2024-04-17T10:38:12.718934+0000","last_became_peered":"2024-04-17T10:38:12.718934+0000","last_unstale":"2024-04-17T10:38:13.702068+0000","last_undegraded":"2024-04-17T10:38:13.702068+0000","last_fullsized":"2024-04-17T10:38:13.702068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:38:39.116589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803564+0000","last_change":"2024-04-17T10:38:12.717999+0000","last_active":"2024-04-17T10:38:13.803564+0000","last_peered":"2024-04-17T10:38:13.803564+0000","last_clean":"2024-04-17T10:38:13.803564+0000","last_became_active":"2024-04-17T10:38:12.716817+0000","last_became_peered":"2024-04-17T10:38:12.716817+0000","last_unstale":"2024-04-17T10:38:13.803564+0000","last_undegraded":"2024-04-17T10:38:13.803564+0000","last_fullsized":"2024-04-17T10:38:13.803564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:42.256740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.304981+0000","last_change":"2024-04-17T10:38:12.710385+0000","last_active":"2024-04-17T10:38:14.304981+0000","last_peered":"2024-04-17T10:38:14.304981+0000","last_clean":"2024-04-17T10:38:14.304981+0000","last_became_active":"2024-04-17T10:38:12.707950+0000","last_became_peered":"2024-04-17T10:38:12.707950+0000","last_unstale":"2024-04-17T10:38:14.304981+0000","last_undegraded":"2024-04-17T10:38:14.304981+0000","last_fullsized":"2024-04-17T10:38:14.304981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:44:43.972627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970232+0000","last_change":"2024-04-17T10:38:12.731401+0000","last_active":"2024-04-17T10:38:13.970232+0000","last_peered":"2024-04-17T10:38:13.970232+0000","last_clean":"2024-04-17T10:38:13.970232+0000","last_became_active":"2024-04-17T10:38:12.729441+0000","last_became_peered":"2024-04-17T10:38:12.729441+0000","last_unstale":"2024-04-17T10:38:13.970232+0000","last_undegraded":"2024-04-17T10:38:13.970232+0000","last_fullsized":"2024-04-17T10:38:13.970232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:02.184227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297301+0000","last_change":"2024-04-17T10:38:12.725791+0000","last_active":"2024-04-17T10:38:14.297301+0000","last_peered":"2024-04-17T10:38:14.297301+0000","last_clean":"2024-04-17T10:38:14.297301+0000","last_became_active":"2024-04-17T10:38:12.723417+0000","last_became_peered":"2024-04-17T10:38:12.723417+0000","last_unstale":"2024-04-17T10:38:14.297301+0000","last_undegraded":"2024-04-17T10:38:14.297301+0000","last_fullsized":"2024-04-17T10:38:14.297301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:26:02.126931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970408+0000","last_change":"2024-04-17T10:38:12.727592+0000","last_active":"2024-04-17T10:38:13.970408+0000","last_peered":"2024-04-17T10:38:13.970408+0000","last_clean":"2024-04-17T10:38:13.970408+0000","last_became_active":"2024-04-17T10:38:12.726482+0000","last_became_peered":"2024-04-17T10:38:12.726482+0000","last_unstale":"2024-04-17T10:38:13.970408+0000","last_undegraded":"2024-04-17T10:38:13.970408+0000","last_fullsized":"2024-04-17T10:38:13.970408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:09:07.090758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297926+0000","last_change":"2024-04-17T10:38:12.716910+0000","last_active":"2024-04-17T10:38:14.297926+0000","last_peered":"2024-04-17T10:38:14.297926+0000","last_clean":"2024-04-17T10:38:14.297926+0000","last_became_active":"2024-04-17T10:38:12.716527+0000","last_became_peered":"2024-04-17T10:38:12.716527+0000","last_unstale":"2024-04-17T10:38:14.297926+0000","last_undegraded":"2024-04-17T10:38:14.297926+0000","last_fullsized":"2024-04-17T10:38:14.297926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:43.917742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803906+0000","last_change":"2024-04-17T10:38:12.728066+0000","last_active":"2024-04-17T10:38:13.803906+0000","last_peered":"2024-04-17T10:38:13.803906+0000","last_clean":"2024-04-17T10:38:13.803906+0000","last_became_active":"2024-04-17T10:38:12.727694+0000","last_became_peered":"2024-04-17T10:38:12.727694+0000","last_unstale":"2024-04-17T10:38:13.803906+0000","last_undegraded":"2024-04-17T10:38:13.803906+0000","last_fullsized":"2024-04-17T10:38:13.803906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:05:21.619291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695757+0000","last_change":"2024-04-17T10:38:12.714887+0000","last_active":"2024-04-17T10:38:13.695757+0000","last_peered":"2024-04-17T10:38:13.695757+0000","last_clean":"2024-04-17T10:38:13.695757+0000","last_became_active":"2024-04-17T10:38:12.714599+0000","last_became_peered":"2024-04-17T10:38:12.714599+0000","last_unstale":"2024-04-17T10:38:13.695757+0000","last_undegraded":"2024-04-17T10:38:13.695757+0000","last_fullsized":"2024-04-17T10:38:13.695757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:42.256740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297989+0000","last_change":"2024-04-17T10:38:12.725678+0000","last_active":"2024-04-17T10:38:14.297989+0000","last_peered":"2024-04-17T10:38:14.297989+0000","last_clean":"2024-04-17T10:38:14.297989+0000","last_became_active":"2024-04-17T10:38:12.723311+0000","last_became_peered":"2024-04-17T10:38:12.723311+0000","last_unstale":"2024-04-17T10:38:14.297989+0000","last_undegraded":"2024-04-17T10:38:14.297989+0000","last_fullsized":"2024-04-17T10:38:14.297989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:05:21.619291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305173+0000","last_change":"2024-04-17T10:38:12.720525+0000","last_active":"2024-04-17T10:38:14.305173+0000","last_peered":"2024-04-17T10:38:14.305173+0000","last_clean":"2024-04-17T10:38:14.305173+0000","last_became_active":"2024-04-17T10:38:12.720134+0000","last_became_peered":"2024-04-17T10:38:12.720134+0000","last_unstale":"2024-04-17T10:38:14.305173+0000","last_undegraded":"2024-04-17T10:38:14.305173+0000","last_fullsized":"2024-04-17T10:38:14.305173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:08.872431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970581+0000","last_change":"2024-04-17T10:38:12.727969+0000","last_active":"2024-04-17T10:38:13.970581+0000","last_peered":"2024-04-17T10:38:13.970581+0000","last_clean":"2024-04-17T10:38:13.970581+0000","last_became_active":"2024-04-17T10:38:12.726330+0000","last_became_peered":"2024-04-17T10:38:12.726330+0000","last_unstale":"2024-04-17T10:38:13.970581+0000","last_undegraded":"2024-04-17T10:38:13.970581+0000","last_fullsized":"2024-04-17T10:38:13.970581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:33:29.583300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.303139+0000","last_change":"2024-04-17T10:38:12.732783+0000","last_active":"2024-04-17T10:38:14.303139+0000","last_peered":"2024-04-17T10:38:14.303139+0000","last_clean":"2024-04-17T10:38:14.303139+0000","last_became_active":"2024-04-17T10:38:12.732525+0000","last_became_peered":"2024-04-17T10:38:12.732525+0000","last_unstale":"2024-04-17T10:38:14.303139+0000","last_undegraded":"2024-04-17T10:38:14.303139+0000","last_fullsized":"2024-04-17T10:38:14.303139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:21:11.071817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803187+0000","last_change":"2024-04-17T10:38:12.720029+0000","last_active":"2024-04-17T10:38:13.803187+0000","last_peered":"2024-04-17T10:38:13.803187+0000","last_clean":"2024-04-17T10:38:13.803187+0000","last_became_active":"2024-04-17T10:38:12.719579+0000","last_became_peered":"2024-04-17T10:38:12.719579+0000","last_unstale":"2024-04-17T10:38:13.803187+0000","last_undegraded":"2024-04-17T10:38:13.803187+0000","last_fullsized":"2024-04-17T10:38:13.803187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:31:48.663733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.699682+0000","last_change":"2024-04-17T10:38:12.709872+0000","last_active":"2024-04-17T10:38:13.699682+0000","last_peered":"2024-04-17T10:38:13.699682+0000","last_clean":"2024-04-17T10:38:13.699682+0000","last_became_active":"2024-04-17T10:38:12.709614+0000","last_became_peered":"2024-04-17T10:38:12.709614+0000","last_unstale":"2024-04-17T10:38:13.699682+0000","last_undegraded":"2024-04-17T10:38:13.699682+0000","last_fullsized":"2024-04-17T10:38:13.699682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:46.375063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298173+0000","last_change":"2024-04-17T10:38:11.718367+0000","last_active":"2024-04-17T10:38:14.298173+0000","last_peered":"2024-04-17T10:38:14.298173+0000","last_clean":"2024-04-17T10:38:14.298173+0000","last_became_active":"2024-04-17T10:38:11.718004+0000","last_became_peered":"2024-04-17T10:38:11.718004+0000","last_unstale":"2024-04-17T10:38:14.298173+0000","last_undegraded":"2024-04-17T10:38:14.298173+0000","last_fullsized":"2024-04-17T10:38:14.298173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:39.210602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305196+0000","last_change":"2024-04-17T10:38:12.710623+0000","last_active":"2024-04-17T10:38:14.305196+0000","last_peered":"2024-04-17T10:38:14.305196+0000","last_clean":"2024-04-17T10:38:14.305196+0000","last_became_active":"2024-04-17T10:38:12.708883+0000","last_became_peered":"2024-04-17T10:38:12.708883+0000","last_unstale":"2024-04-17T10:38:14.305196+0000","last_undegraded":"2024-04-17T10:38:14.305196+0000","last_fullsized":"2024-04-17T10:38:14.305196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:26:02.126931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803042+0000","last_change":"2024-04-17T10:38:11.718224+0000","last_active":"2024-04-17T10:38:13.803042+0000","last_peered":"2024-04-17T10:38:13.803042+0000","last_clean":"2024-04-17T10:38:13.803042+0000","last_became_active":"2024-04-17T10:38:11.717433+0000","last_became_peered":"2024-04-17T10:38:11.717433+0000","last_unstale":"2024-04-17T10:38:13.803042+0000","last_undegraded":"2024-04-17T10:38:13.803042+0000","last_fullsized":"2024-04-17T10:38:13.803042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:39.210602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298350+0000","last_change":"2024-04-17T10:38:11.719972+0000","last_active":"2024-04-17T10:38:14.298350+0000","last_peered":"2024-04-17T10:38:14.298350+0000","last_clean":"2024-04-17T10:38:14.298350+0000","last_became_active":"2024-04-17T10:38:11.719469+0000","last_became_peered":"2024-04-17T10:38:11.719469+0000","last_unstale":"2024-04-17T10:38:14.298350+0000","last_undegraded":"2024-04-17T10:38:14.298350+0000","last_fullsized":"2024-04-17T10:38:14.298350+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:59:51.693435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297286+0000","last_change":"2024-04-17T10:38:12.722655+0000","last_active":"2024-04-17T10:38:14.297286+0000","last_peered":"2024-04-17T10:38:14.297286+0000","last_clean":"2024-04-17T10:38:14.297286+0000","last_became_active":"2024-04-17T10:38:12.722213+0000","last_became_peered":"2024-04-17T10:38:12.722213+0000","last_unstale":"2024-04-17T10:38:14.297286+0000","last_undegraded":"2024-04-17T10:38:14.297286+0000","last_fullsized":"2024-04-17T10:38:14.297286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:08.872431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970337+0000","last_change":"2024-04-17T10:38:11.704898+0000","last_active":"2024-04-17T10:38:13.970337+0000","last_peered":"2024-04-17T10:38:13.970337+0000","last_clean":"2024-04-17T10:38:13.970337+0000","last_became_active":"2024-04-17T10:38:11.704184+0000","last_became_peered":"2024-04-17T10:38:11.704184+0000","last_unstale":"2024-04-17T10:38:13.970337+0000","last_undegraded":"2024-04-17T10:38:13.970337+0000","last_fullsized":"2024-04-17T10:38:13.970337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:45.366572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695486+0000","last_change":"2024-04-17T10:38:12.714844+0000","last_active":"2024-04-17T10:38:13.695486+0000","last_peered":"2024-04-17T10:38:13.695486+0000","last_clean":"2024-04-17T10:38:13.695486+0000","last_became_active":"2024-04-17T10:38:12.714565+0000","last_became_peered":"2024-04-17T10:38:12.714565+0000","last_unstale":"2024-04-17T10:38:13.695486+0000","last_undegraded":"2024-04-17T10:38:13.695486+0000","last_fullsized":"2024-04-17T10:38:13.695486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:26:02.126931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803577+0000","last_change":"2024-04-17T10:38:11.725276+0000","last_active":"2024-04-17T10:38:13.803577+0000","last_peered":"2024-04-17T10:38:13.803577+0000","last_clean":"2024-04-17T10:38:13.803577+0000","last_became_active":"2024-04-17T10:38:11.724660+0000","last_became_peered":"2024-04-17T10:38:11.724660+0000","last_unstale":"2024-04-17T10:38:13.803577+0000","last_undegraded":"2024-04-17T10:38:13.803577+0000","last_fullsized":"2024-04-17T10:38:13.803577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:21:00.724523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970977+0000","last_change":"2024-04-17T10:38:12.731384+0000","last_active":"2024-04-17T10:38:13.970977+0000","last_peered":"2024-04-17T10:38:13.970977+0000","last_clean":"2024-04-17T10:38:13.970977+0000","last_became_active":"2024-04-17T10:38:12.730178+0000","last_became_peered":"2024-04-17T10:38:12.730178+0000","last_unstale":"2024-04-17T10:38:13.970977+0000","last_undegraded":"2024-04-17T10:38:13.970977+0000","last_fullsized":"2024-04-17T10:38:13.970977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:11:49.847070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970956+0000","last_change":"2024-04-17T10:38:11.727591+0000","last_active":"2024-04-17T10:38:13.970956+0000","last_peered":"2024-04-17T10:38:13.970956+0000","last_clean":"2024-04-17T10:38:13.970956+0000","last_became_active":"2024-04-17T10:38:11.726319+0000","last_became_peered":"2024-04-17T10:38:11.726319+0000","last_unstale":"2024-04-17T10:38:13.970956+0000","last_undegraded":"2024-04-17T10:38:13.970956+0000","last_fullsized":"2024-04-17T10:38:13.970956+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:50:59.373356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305290+0000","last_change":"2024-04-17T10:38:12.727239+0000","last_active":"2024-04-17T10:38:14.305290+0000","last_peered":"2024-04-17T10:38:14.305290+0000","last_clean":"2024-04-17T10:38:14.305290+0000","last_became_active":"2024-04-17T10:38:12.726192+0000","last_became_peered":"2024-04-17T10:38:12.726192+0000","last_unstale":"2024-04-17T10:38:14.305290+0000","last_undegraded":"2024-04-17T10:38:14.305290+0000","last_fullsized":"2024-04-17T10:38:14.305290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:39.640730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695810+0000","last_change":"2024-04-17T10:38:11.710534+0000","last_active":"2024-04-17T10:38:13.695810+0000","last_peered":"2024-04-17T10:38:13.695810+0000","last_clean":"2024-04-17T10:38:13.695810+0000","last_became_active":"2024-04-17T10:38:11.709944+0000","last_became_peered":"2024-04-17T10:38:11.709944+0000","last_unstale":"2024-04-17T10:38:13.695810+0000","last_undegraded":"2024-04-17T10:38:13.695810+0000","last_fullsized":"2024-04-17T10:38:13.695810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:45.366572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.700329+0000","last_change":"2024-04-17T10:38:12.709577+0000","last_active":"2024-04-17T10:38:13.700329+0000","last_peered":"2024-04-17T10:38:13.700329+0000","last_clean":"2024-04-17T10:38:13.700329+0000","last_became_active":"2024-04-17T10:38:12.709186+0000","last_became_peered":"2024-04-17T10:38:12.709186+0000","last_unstale":"2024-04-17T10:38:13.700329+0000","last_undegraded":"2024-04-17T10:38:13.700329+0000","last_fullsized":"2024-04-17T10:38:13.700329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:20:45.810408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803828+0000","last_change":"2024-04-17T10:38:11.715117+0000","last_active":"2024-04-17T10:38:13.803828+0000","last_peered":"2024-04-17T10:38:13.803828+0000","last_clean":"2024-04-17T10:38:13.803828+0000","last_became_active":"2024-04-17T10:38:11.713981+0000","last_became_peered":"2024-04-17T10:38:11.713981+0000","last_unstale":"2024-04-17T10:38:13.803828+0000","last_undegraded":"2024-04-17T10:38:13.803828+0000","last_fullsized":"2024-04-17T10:38:13.803828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:27.569244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701871+0000","last_change":"2024-04-17T10:38:12.708377+0000","last_active":"2024-04-17T10:38:13.701871+0000","last_peered":"2024-04-17T10:38:13.701871+0000","last_clean":"2024-04-17T10:38:13.701871+0000","last_became_active":"2024-04-17T10:38:12.707183+0000","last_became_peered":"2024-04-17T10:38:12.707183+0000","last_unstale":"2024-04-17T10:38:13.701871+0000","last_undegraded":"2024-04-17T10:38:13.701871+0000","last_fullsized":"2024-04-17T10:38:13.701871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:06.508830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305423+0000","last_change":"2024-04-17T10:38:11.702326+0000","last_active":"2024-04-17T10:38:14.305423+0000","last_peered":"2024-04-17T10:38:14.305423+0000","last_clean":"2024-04-17T10:38:14.305423+0000","last_became_active":"2024-04-17T10:38:11.701735+0000","last_became_peered":"2024-04-17T10:38:11.701735+0000","last_unstale":"2024-04-17T10:38:14.305423+0000","last_undegraded":"2024-04-17T10:38:14.305423+0000","last_fullsized":"2024-04-17T10:38:14.305423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:11:43.267871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.304822+0000","last_change":"2024-04-17T10:38:12.705702+0000","last_active":"2024-04-17T10:38:14.304822+0000","last_peered":"2024-04-17T10:38:14.304822+0000","last_clean":"2024-04-17T10:38:14.304822+0000","last_became_active":"2024-04-17T10:38:12.705297+0000","last_became_peered":"2024-04-17T10:38:12.705297+0000","last_unstale":"2024-04-17T10:38:14.304822+0000","last_undegraded":"2024-04-17T10:38:14.304822+0000","last_fullsized":"2024-04-17T10:38:14.304822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:15:31.987120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970479+0000","last_change":"2024-04-17T10:38:11.715689+0000","last_active":"2024-04-17T10:38:13.970479+0000","last_peered":"2024-04-17T10:38:13.970479+0000","last_clean":"2024-04-17T10:38:13.970479+0000","last_became_active":"2024-04-17T10:38:11.714329+0000","last_became_peered":"2024-04-17T10:38:11.714329+0000","last_unstale":"2024-04-17T10:38:13.970479+0000","last_undegraded":"2024-04-17T10:38:13.970479+0000","last_fullsized":"2024-04-17T10:38:13.970479+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:27.569244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702011+0000","last_change":"2024-04-17T10:38:12.708536+0000","last_active":"2024-04-17T10:38:13.702011+0000","last_peered":"2024-04-17T10:38:13.702011+0000","last_clean":"2024-04-17T10:38:13.702011+0000","last_became_active":"2024-04-17T10:38:12.707643+0000","last_became_peered":"2024-04-17T10:38:12.707643+0000","last_unstale":"2024-04-17T10:38:13.702011+0000","last_undegraded":"2024-04-17T10:38:13.702011+0000","last_fullsized":"2024-04-17T10:38:13.702011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:22:10.364740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803484+0000","last_change":"2024-04-17T10:38:11.717992+0000","last_active":"2024-04-17T10:38:13.803484+0000","last_peered":"2024-04-17T10:38:13.803484+0000","last_clean":"2024-04-17T10:38:13.803484+0000","last_became_active":"2024-04-17T10:38:11.717124+0000","last_became_peered":"2024-04-17T10:38:11.717124+0000","last_unstale":"2024-04-17T10:38:13.803484+0000","last_undegraded":"2024-04-17T10:38:13.803484+0000","last_fullsized":"2024-04-17T10:38:13.803484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:24:23.818791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803260+0000","last_change":"2024-04-17T10:38:12.720282+0000","last_active":"2024-04-17T10:38:13.803260+0000","last_peered":"2024-04-17T10:38:13.803260+0000","last_clean":"2024-04-17T10:38:13.803260+0000","last_became_active":"2024-04-17T10:38:12.719197+0000","last_became_peered":"2024-04-17T10:38:12.719197+0000","last_unstale":"2024-04-17T10:38:13.803260+0000","last_undegraded":"2024-04-17T10:38:13.803260+0000","last_fullsized":"2024-04-17T10:38:13.803260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:08.872431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.303153+0000","last_change":"2024-04-17T10:38:11.689943+0000","last_active":"2024-04-17T10:38:14.303153+0000","last_peered":"2024-04-17T10:38:14.303153+0000","last_clean":"2024-04-17T10:38:14.303153+0000","last_became_active":"2024-04-17T10:38:11.689627+0000","last_became_peered":"2024-04-17T10:38:11.689627+0000","last_unstale":"2024-04-17T10:38:14.303153+0000","last_undegraded":"2024-04-17T10:38:14.303153+0000","last_fullsized":"2024-04-17T10:38:14.303153+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:39.210602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701741+0000","last_change":"2024-04-17T10:38:12.710693+0000","last_active":"2024-04-17T10:38:13.701741+0000","last_peered":"2024-04-17T10:38:13.701741+0000","last_clean":"2024-04-17T10:38:13.701741+0000","last_became_active":"2024-04-17T10:38:12.710263+0000","last_became_peered":"2024-04-17T10:38:12.710263+0000","last_unstale":"2024-04-17T10:38:13.701741+0000","last_undegraded":"2024-04-17T10:38:13.701741+0000","last_fullsized":"2024-04-17T10:38:13.701741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:09:07.090758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.700517+0000","last_change":"2024-04-17T10:38:11.701185+0000","last_active":"2024-04-17T10:38:13.700517+0000","last_peered":"2024-04-17T10:38:13.700517+0000","last_clean":"2024-04-17T10:38:13.700517+0000","last_became_active":"2024-04-17T10:38:11.700292+0000","last_became_peered":"2024-04-17T10:38:11.700292+0000","last_unstale":"2024-04-17T10:38:13.700517+0000","last_undegraded":"2024-04-17T10:38:13.700517+0000","last_fullsized":"2024-04-17T10:38:13.700517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:59:51.693435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702731+0000","last_change":"2024-04-17T10:38:12.704267+0000","last_active":"2024-04-17T10:38:13.702731+0000","last_peered":"2024-04-17T10:38:13.702731+0000","last_clean":"2024-04-17T10:38:13.702731+0000","last_became_active":"2024-04-17T10:38:12.703820+0000","last_became_peered":"2024-04-17T10:38:12.703820+0000","last_unstale":"2024-04-17T10:38:13.702731+0000","last_undegraded":"2024-04-17T10:38:13.702731+0000","last_fullsized":"2024-04-17T10:38:13.702731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:43:51.398103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.303738+0000","last_change":"2024-04-17T10:38:11.699100+0000","last_active":"2024-04-17T10:38:14.303738+0000","last_peered":"2024-04-17T10:38:14.303738+0000","last_clean":"2024-04-17T10:38:14.303738+0000","last_became_active":"2024-04-17T10:38:11.698786+0000","last_became_peered":"2024-04-17T10:38:11.698786+0000","last_unstale":"2024-04-17T10:38:14.303738+0000","last_undegraded":"2024-04-17T10:38:14.303738+0000","last_fullsized":"2024-04-17T10:38:14.303738+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:21:00.724523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702808+0000","last_change":"2024-04-17T10:38:12.705166+0000","last_active":"2024-04-17T10:38:13.702808+0000","last_peered":"2024-04-17T10:38:13.702808+0000","last_clean":"2024-04-17T10:38:13.702808+0000","last_became_active":"2024-04-17T10:38:12.704805+0000","last_became_peered":"2024-04-17T10:38:12.704805+0000","last_unstale":"2024-04-17T10:38:13.702808+0000","last_undegraded":"2024-04-17T10:38:13.702808+0000","last_fullsized":"2024-04-17T10:38:13.702808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:49:18.397560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.696028+0000","last_change":"2024-04-17T10:38:11.692209+0000","last_active":"2024-04-17T10:38:13.696028+0000","last_peered":"2024-04-17T10:38:13.696028+0000","last_clean":"2024-04-17T10:38:13.696028+0000","last_became_active":"2024-04-17T10:38:11.691905+0000","last_became_peered":"2024-04-17T10:38:11.691905+0000","last_unstale":"2024-04-17T10:38:13.696028+0000","last_undegraded":"2024-04-17T10:38:13.696028+0000","last_fullsized":"2024-04-17T10:38:13.696028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:50:59.373356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970996+0000","last_change":"2024-04-17T10:38:12.727145+0000","last_active":"2024-04-17T10:38:13.970996+0000","last_peered":"2024-04-17T10:38:13.970996+0000","last_clean":"2024-04-17T10:38:13.970996+0000","last_became_active":"2024-04-17T10:38:12.726797+0000","last_became_peered":"2024-04-17T10:38:12.726797+0000","last_unstale":"2024-04-17T10:38:13.970996+0000","last_undegraded":"2024-04-17T10:38:13.970996+0000","last_fullsized":"2024-04-17T10:38:13.970996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:43:51.398103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.303519+0000","last_change":"2024-04-17T10:38:11.705218+0000","last_active":"2024-04-17T10:38:14.303519+0000","last_peered":"2024-04-17T10:38:14.303519+0000","last_clean":"2024-04-17T10:38:14.303519+0000","last_became_active":"2024-04-17T10:38:11.704928+0000","last_became_peered":"2024-04-17T10:38:11.704928+0000","last_unstale":"2024-04-17T10:38:14.303519+0000","last_undegraded":"2024-04-17T10:38:14.303519+0000","last_fullsized":"2024-04-17T10:38:14.303519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:27.569244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701937+0000","last_change":"2024-04-17T10:38:12.722659+0000","last_active":"2024-04-17T10:38:13.701937+0000","last_peered":"2024-04-17T10:38:13.701937+0000","last_clean":"2024-04-17T10:38:13.701937+0000","last_became_active":"2024-04-17T10:38:12.722265+0000","last_became_peered":"2024-04-17T10:38:12.722265+0000","last_unstale":"2024-04-17T10:38:13.701937+0000","last_undegraded":"2024-04-17T10:38:13.701937+0000","last_fullsized":"2024-04-17T10:38:13.701937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:33:29.583300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701282+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:38:13.701282+0000","last_peered":"2024-04-17T10:38:13.701282+0000","last_clean":"2024-04-17T10:38:13.701282+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:38:13.701282+0000","last_undegraded":"2024-04-17T10:38:13.701282+0000","last_fullsized":"2024-04-17T10:38:13.701282+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970082+0000","last_change":"2024-04-17T10:38:11.702024+0000","last_active":"2024-04-17T10:38:13.970082+0000","last_peered":"2024-04-17T10:38:13.970082+0000","last_clean":"2024-04-17T10:38:13.970082+0000","last_became_active":"2024-04-17T10:38:11.701157+0000","last_became_peered":"2024-04-17T10:38:11.701157+0000","last_unstale":"2024-04-17T10:38:13.970082+0000","last_undegraded":"2024-04-17T10:38:13.970082+0000","last_fullsized":"2024-04-17T10:38:13.970082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:24:23.818791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297692+0000","last_change":"2024-04-17T10:38:12.723529+0000","last_active":"2024-04-17T10:38:14.297692+0000","last_peered":"2024-04-17T10:38:14.297692+0000","last_clean":"2024-04-17T10:38:14.297692+0000","last_became_active":"2024-04-17T10:38:12.723103+0000","last_became_peered":"2024-04-17T10:38:12.723103+0000","last_unstale":"2024-04-17T10:38:14.297692+0000","last_undegraded":"2024-04-17T10:38:14.297692+0000","last_fullsized":"2024-04-17T10:38:14.297692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:42.256740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701197+0000","last_change":"2024-04-17T10:38:11.716453+0000","last_active":"2024-04-17T10:38:13.701197+0000","last_peered":"2024-04-17T10:38:13.701197+0000","last_clean":"2024-04-17T10:38:13.701197+0000","last_became_active":"2024-04-17T10:38:11.715859+0000","last_became_peered":"2024-04-17T10:38:11.715859+0000","last_unstale":"2024-04-17T10:38:13.701197+0000","last_undegraded":"2024-04-17T10:38:13.701197+0000","last_fullsized":"2024-04-17T10:38:13.701197+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:50:59.373356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.304804+0000","last_change":"2024-04-17T10:38:12.707314+0000","last_active":"2024-04-17T10:38:14.304804+0000","last_peered":"2024-04-17T10:38:14.304804+0000","last_clean":"2024-04-17T10:38:14.304804+0000","last_became_active":"2024-04-17T10:38:12.706874+0000","last_became_peered":"2024-04-17T10:38:12.706874+0000","last_unstale":"2024-04-17T10:38:14.304804+0000","last_undegraded":"2024-04-17T10:38:14.304804+0000","last_fullsized":"2024-04-17T10:38:14.304804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:07:07.037555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.303300+0000","last_change":"2024-04-17T10:38:11.699752+0000","last_active":"2024-04-17T10:38:14.303300+0000","last_peered":"2024-04-17T10:38:14.303300+0000","last_clean":"2024-04-17T10:38:14.303300+0000","last_became_active":"2024-04-17T10:38:11.699467+0000","last_became_peered":"2024-04-17T10:38:11.699467+0000","last_unstale":"2024-04-17T10:38:14.303300+0000","last_undegraded":"2024-04-17T10:38:14.303300+0000","last_fullsized":"2024-04-17T10:38:14.303300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:24:23.818791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695234+0000","last_change":"2024-04-17T10:38:12.723809+0000","last_active":"2024-04-17T10:38:13.695234+0000","last_peered":"2024-04-17T10:38:13.695234+0000","last_clean":"2024-04-17T10:38:13.695234+0000","last_became_active":"2024-04-17T10:38:12.723282+0000","last_became_peered":"2024-04-17T10:38:12.723282+0000","last_unstale":"2024-04-17T10:38:13.695234+0000","last_undegraded":"2024-04-17T10:38:13.695234+0000","last_fullsized":"2024-04-17T10:38:13.695234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:31:48.663733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.971078+0000","last_change":"2024-04-17T10:38:11.715164+0000","last_active":"2024-04-17T10:38:13.971078+0000","last_peered":"2024-04-17T10:38:13.971078+0000","last_clean":"2024-04-17T10:38:13.971078+0000","last_became_active":"2024-04-17T10:38:11.712376+0000","last_became_peered":"2024-04-17T10:38:11.712376+0000","last_unstale":"2024-04-17T10:38:13.971078+0000","last_undegraded":"2024-04-17T10:38:13.971078+0000","last_fullsized":"2024-04-17T10:38:13.971078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:21:00.724523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.699765+0000","last_change":"2024-04-17T10:38:12.720339+0000","last_active":"2024-04-17T10:38:13.699765+0000","last_peered":"2024-04-17T10:38:13.699765+0000","last_clean":"2024-04-17T10:38:13.699765+0000","last_became_active":"2024-04-17T10:38:12.720051+0000","last_became_peered":"2024-04-17T10:38:12.720051+0000","last_unstale":"2024-04-17T10:38:13.699765+0000","last_undegraded":"2024-04-17T10:38:13.699765+0000","last_fullsized":"2024-04-17T10:38:13.699765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:51:00.381846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695713+0000","last_change":"2024-04-17T10:38:11.693095+0000","last_active":"2024-04-17T10:38:13.695713+0000","last_peered":"2024-04-17T10:38:13.695713+0000","last_clean":"2024-04-17T10:38:13.695713+0000","last_became_active":"2024-04-17T10:38:11.692723+0000","last_became_peered":"2024-04-17T10:38:11.692723+0000","last_unstale":"2024-04-17T10:38:13.695713+0000","last_undegraded":"2024-04-17T10:38:13.695713+0000","last_fullsized":"2024-04-17T10:38:13.695713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:27.569244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970920+0000","last_change":"2024-04-17T10:38:12.728559+0000","last_active":"2024-04-17T10:38:13.970920+0000","last_peered":"2024-04-17T10:38:13.970920+0000","last_clean":"2024-04-17T10:38:13.970920+0000","last_became_active":"2024-04-17T10:38:12.728285+0000","last_became_peered":"2024-04-17T10:38:12.728285+0000","last_unstale":"2024-04-17T10:38:13.970920+0000","last_undegraded":"2024-04-17T10:38:13.970920+0000","last_fullsized":"2024-04-17T10:38:13.970920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:22:10.364740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702148+0000","last_change":"2024-04-17T10:38:11.717791+0000","last_active":"2024-04-17T10:38:13.702148+0000","last_peered":"2024-04-17T10:38:13.702148+0000","last_clean":"2024-04-17T10:38:13.702148+0000","last_became_active":"2024-04-17T10:38:11.717130+0000","last_became_peered":"2024-04-17T10:38:11.717130+0000","last_unstale":"2024-04-17T10:38:13.702148+0000","last_undegraded":"2024-04-17T10:38:13.702148+0000","last_fullsized":"2024-04-17T10:38:13.702148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:21:10.063327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695581+0000","last_change":"2024-04-17T10:38:12.723436+0000","last_active":"2024-04-17T10:38:13.695581+0000","last_peered":"2024-04-17T10:38:13.695581+0000","last_clean":"2024-04-17T10:38:13.695581+0000","last_became_active":"2024-04-17T10:38:12.723138+0000","last_became_peered":"2024-04-17T10:38:12.723138+0000","last_unstale":"2024-04-17T10:38:13.695581+0000","last_undegraded":"2024-04-17T10:38:13.695581+0000","last_fullsized":"2024-04-17T10:38:13.695581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:39.640730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.696103+0000","last_change":"2024-04-17T10:38:11.730180+0000","last_active":"2024-04-17T10:38:13.696103+0000","last_peered":"2024-04-17T10:38:13.696103+0000","last_clean":"2024-04-17T10:38:13.696103+0000","last_became_active":"2024-04-17T10:38:11.729843+0000","last_became_peered":"2024-04-17T10:38:11.729843+0000","last_unstale":"2024-04-17T10:38:13.696103+0000","last_undegraded":"2024-04-17T10:38:13.696103+0000","last_fullsized":"2024-04-17T10:38:13.696103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:23:29.956041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298370+0000","last_change":"2024-04-17T10:38:12.732643+0000","last_active":"2024-04-17T10:38:14.298370+0000","last_peered":"2024-04-17T10:38:14.298370+0000","last_clean":"2024-04-17T10:38:14.298370+0000","last_became_active":"2024-04-17T10:38:12.732259+0000","last_became_peered":"2024-04-17T10:38:12.732259+0000","last_unstale":"2024-04-17T10:38:14.298370+0000","last_undegraded":"2024-04-17T10:38:14.298370+0000","last_fullsized":"2024-04-17T10:38:14.298370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:02.184227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803741+0000","last_change":"2024-04-17T10:38:11.725444+0000","last_active":"2024-04-17T10:38:13.803741+0000","last_peered":"2024-04-17T10:38:13.803741+0000","last_clean":"2024-04-17T10:38:13.803741+0000","last_became_active":"2024-04-17T10:38:11.724780+0000","last_became_peered":"2024-04-17T10:38:11.724780+0000","last_unstale":"2024-04-17T10:38:13.803741+0000","last_undegraded":"2024-04-17T10:38:13.803741+0000","last_fullsized":"2024-04-17T10:38:13.803741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:45.366572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305401+0000","last_change":"2024-04-17T10:38:12.705770+0000","last_active":"2024-04-17T10:38:14.305401+0000","last_peered":"2024-04-17T10:38:14.305401+0000","last_clean":"2024-04-17T10:38:14.305401+0000","last_became_active":"2024-04-17T10:38:12.705400+0000","last_became_peered":"2024-04-17T10:38:12.705400+0000","last_unstale":"2024-04-17T10:38:14.305401+0000","last_undegraded":"2024-04-17T10:38:14.305401+0000","last_fullsized":"2024-04-17T10:38:14.305401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:42.256740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701197+0000","last_change":"2024-04-17T10:38:11.716433+0000","last_active":"2024-04-17T10:38:13.701197+0000","last_peered":"2024-04-17T10:38:13.701197+0000","last_clean":"2024-04-17T10:38:13.701197+0000","last_became_active":"2024-04-17T10:38:11.715803+0000","last_became_peered":"2024-04-17T10:38:11.715803+0000","last_unstale":"2024-04-17T10:38:13.701197+0000","last_undegraded":"2024-04-17T10:38:13.701197+0000","last_fullsized":"2024-04-17T10:38:13.701197+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:45.366572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.700697+0000","last_change":"2024-04-17T10:38:12.717613+0000","last_active":"2024-04-17T10:38:13.700697+0000","last_peered":"2024-04-17T10:38:13.700697+0000","last_clean":"2024-04-17T10:38:13.700697+0000","last_became_active":"2024-04-17T10:38:12.717308+0000","last_became_peered":"2024-04-17T10:38:12.717308+0000","last_unstale":"2024-04-17T10:38:13.700697+0000","last_undegraded":"2024-04-17T10:38:13.700697+0000","last_fullsized":"2024-04-17T10:38:13.700697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:21:11.071817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695770+0000","last_change":"2024-04-17T10:38:11.716778+0000","last_active":"2024-04-17T10:38:13.695770+0000","last_peered":"2024-04-17T10:38:13.695770+0000","last_clean":"2024-04-17T10:38:13.695770+0000","last_became_active":"2024-04-17T10:38:11.716526+0000","last_became_peered":"2024-04-17T10:38:11.716526+0000","last_unstale":"2024-04-17T10:38:13.695770+0000","last_undegraded":"2024-04-17T10:38:13.695770+0000","last_fullsized":"2024-04-17T10:38:13.695770+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:24:23.818791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297622+0000","last_change":"2024-04-17T10:38:12.717941+0000","last_active":"2024-04-17T10:38:14.297622+0000","last_peered":"2024-04-17T10:38:14.297622+0000","last_clean":"2024-04-17T10:38:14.297622+0000","last_became_active":"2024-04-17T10:38:12.717350+0000","last_became_peered":"2024-04-17T10:38:12.717350+0000","last_unstale":"2024-04-17T10:38:14.297622+0000","last_undegraded":"2024-04-17T10:38:14.297622+0000","last_fullsized":"2024-04-17T10:38:14.297622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:39.640730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695661+0000","last_change":"2024-04-17T10:38:11.699325+0000","last_active":"2024-04-17T10:38:13.695661+0000","last_peered":"2024-04-17T10:38:13.695661+0000","last_clean":"2024-04-17T10:38:13.695661+0000","last_became_active":"2024-04-17T10:38:11.698981+0000","last_became_peered":"2024-04-17T10:38:11.698981+0000","last_unstale":"2024-04-17T10:38:13.695661+0000","last_undegraded":"2024-04-17T10:38:13.695661+0000","last_fullsized":"2024-04-17T10:38:13.695661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:21:00.724523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298006+0000","last_change":"2024-04-17T10:38:12.731391+0000","last_active":"2024-04-17T10:38:14.298006+0000","last_peered":"2024-04-17T10:38:14.298006+0000","last_clean":"2024-04-17T10:38:14.298006+0000","last_became_active":"2024-04-17T10:38:12.727638+0000","last_became_peered":"2024-04-17T10:38:12.727638+0000","last_unstale":"2024-04-17T10:38:14.298006+0000","last_undegraded":"2024-04-17T10:38:14.298006+0000","last_fullsized":"2024-04-17T10:38:14.298006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:23.465112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701909+0000","last_change":"2024-04-17T10:38:11.716353+0000","last_active":"2024-04-17T10:38:13.701909+0000","last_peered":"2024-04-17T10:38:13.701909+0000","last_clean":"2024-04-17T10:38:13.701909+0000","last_became_active":"2024-04-17T10:38:11.715722+0000","last_became_peered":"2024-04-17T10:38:11.715722+0000","last_unstale":"2024-04-17T10:38:13.701909+0000","last_undegraded":"2024-04-17T10:38:13.701909+0000","last_fullsized":"2024-04-17T10:38:13.701909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:20:44.801917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803339+0000","last_change":"2024-04-17T10:38:12.715919+0000","last_active":"2024-04-17T10:38:13.803339+0000","last_peered":"2024-04-17T10:38:13.803339+0000","last_clean":"2024-04-17T10:38:13.803339+0000","last_became_active":"2024-04-17T10:38:12.715568+0000","last_became_peered":"2024-04-17T10:38:12.715568+0000","last_unstale":"2024-04-17T10:38:13.803339+0000","last_undegraded":"2024-04-17T10:38:13.803339+0000","last_fullsized":"2024-04-17T10:38:13.803339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:26:02.126931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970940+0000","last_change":"2024-04-17T10:38:12.724977+0000","last_active":"2024-04-17T10:38:13.970940+0000","last_peered":"2024-04-17T10:38:13.970940+0000","last_clean":"2024-04-17T10:38:13.970940+0000","last_became_active":"2024-04-17T10:38:12.724700+0000","last_became_peered":"2024-04-17T10:38:12.724700+0000","last_unstale":"2024-04-17T10:38:13.970940+0000","last_undegraded":"2024-04-17T10:38:13.970940+0000","last_fullsized":"2024-04-17T10:38:13.970940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:38:39.116589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702197+0000","last_change":"2024-04-17T10:38:11.719799+0000","last_active":"2024-04-17T10:38:13.702197+0000","last_peered":"2024-04-17T10:38:13.702197+0000","last_clean":"2024-04-17T10:38:13.702197+0000","last_became_active":"2024-04-17T10:38:11.719147+0000","last_became_peered":"2024-04-17T10:38:11.719147+0000","last_unstale":"2024-04-17T10:38:13.702197+0000","last_undegraded":"2024-04-17T10:38:13.702197+0000","last_fullsized":"2024-04-17T10:38:13.702197+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:23:29.956041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.971331+0000","last_change":"2024-04-17T10:38:12.725123+0000","last_active":"2024-04-17T10:38:13.971331+0000","last_peered":"2024-04-17T10:38:13.971331+0000","last_clean":"2024-04-17T10:38:13.971331+0000","last_became_active":"2024-04-17T10:38:12.724760+0000","last_became_peered":"2024-04-17T10:38:12.724760+0000","last_unstale":"2024-04-17T10:38:13.971331+0000","last_undegraded":"2024-04-17T10:38:13.971331+0000","last_fullsized":"2024-04-17T10:38:13.971331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:49:18.397560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702167+0000","last_change":"2024-04-17T10:38:11.727050+0000","last_active":"2024-04-17T10:38:13.702167+0000","last_peered":"2024-04-17T10:38:13.702167+0000","last_clean":"2024-04-17T10:38:13.702167+0000","last_became_active":"2024-04-17T10:38:11.726393+0000","last_became_peered":"2024-04-17T10:38:11.726393+0000","last_unstale":"2024-04-17T10:38:13.702167+0000","last_undegraded":"2024-04-17T10:38:13.702167+0000","last_fullsized":"2024-04-17T10:38:13.702167+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:23:25.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305077+0000","last_change":"2024-04-17T10:38:12.726864+0000","last_active":"2024-04-17T10:38:14.305077+0000","last_peered":"2024-04-17T10:38:14.305077+0000","last_clean":"2024-04-17T10:38:14.305077+0000","last_became_active":"2024-04-17T10:38:12.726581+0000","last_became_peered":"2024-04-17T10:38:12.726581+0000","last_unstale":"2024-04-17T10:38:14.305077+0000","last_undegraded":"2024-04-17T10:38:14.305077+0000","last_fullsized":"2024-04-17T10:38:14.305077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:31:48.663733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298342+0000","last_change":"2024-04-17T10:38:11.700825+0000","last_active":"2024-04-17T10:38:14.298342+0000","last_peered":"2024-04-17T10:38:14.298342+0000","last_clean":"2024-04-17T10:38:14.298342+0000","last_became_active":"2024-04-17T10:38:11.700354+0000","last_became_peered":"2024-04-17T10:38:11.700354+0000","last_unstale":"2024-04-17T10:38:14.298342+0000","last_undegraded":"2024-04-17T10:38:14.298342+0000","last_fullsized":"2024-04-17T10:38:14.298342+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:22:08.746889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298080+0000","last_change":"2024-04-17T10:38:12.719771+0000","last_active":"2024-04-17T10:38:14.298080+0000","last_peered":"2024-04-17T10:38:14.298080+0000","last_clean":"2024-04-17T10:38:14.298080+0000","last_became_active":"2024-04-17T10:38:12.716013+0000","last_became_peered":"2024-04-17T10:38:12.716013+0000","last_unstale":"2024-04-17T10:38:14.298080+0000","last_undegraded":"2024-04-17T10:38:14.298080+0000","last_fullsized":"2024-04-17T10:38:14.298080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:54:26.054113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305389+0000","last_change":"2024-04-17T10:38:11.710717+0000","last_active":"2024-04-17T10:38:14.305389+0000","last_peered":"2024-04-17T10:38:14.305389+0000","last_clean":"2024-04-17T10:38:14.305389+0000","last_became_active":"2024-04-17T10:38:11.710342+0000","last_became_peered":"2024-04-17T10:38:11.710342+0000","last_unstale":"2024-04-17T10:38:14.305389+0000","last_undegraded":"2024-04-17T10:38:14.305389+0000","last_fullsized":"2024-04-17T10:38:14.305389+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:16:30.923882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":196608,"data_stored":64724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":798720,"data_stored":655640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:38:16.427 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-17T10:38:16.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.588+0000 7fa66ec03640 1 Processor -- start 2024-04-17T10:38:16.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.588+0000 7fa66ec03640 1 -- start start 2024-04-17T10:38:16.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.588+0000 7fa66ec03640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa6680b2d80 0x7fa668146e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:16.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.588+0000 7fa66ec03640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa66806d690 0x7fa6681620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:16.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.588+0000 7fa66ec03640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa668147570 con 0x7fa668146a00 2024-04-17T10:38:16.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.588+0000 7fa66ec03640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa668162610 con 0x7fa6680b2d80 2024-04-17T10:38:16.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.588+0000 7fa66ec03640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa66806c960 con 0x7fa66806d690 2024-04-17T10:38:16.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.592+0000 7fa667fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa6680b2d80 0x7fa668146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:16.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.592+0000 7fa667fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa6680b2d80 0x7fa668146e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56000/0 (socket says 172.21.15.83:56000) 2024-04-17T10:38:16.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.592+0000 7fa667fff640 1 -- 172.21.15.83:0/3956800565 learned_addr learned my addr 172.21.15.83:0/3956800565 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:16.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.592+0000 7fa66d1a0640 1 --2- 172.21.15.83:0/3956800565 >> v2:172.21.15.167:3300/0 conn(0x7fa66806d690 0x7fa6681620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:16.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.592+0000 7fa667fff640 1 -- 172.21.15.83:0/3956800565 >> v2:172.21.15.167:3300/0 conn(0x7fa66806d690 msgr2=0x7fa6681620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:16.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.592+0000 7fa667fff640 1 --2- 172.21.15.83:0/3956800565 >> v2:172.21.15.167:3300/0 conn(0x7fa66806d690 0x7fa6681620d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:16.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.592+0000 7fa667fff640 1 -- 172.21.15.83:0/3956800565 >> v1:172.21.15.167:6789/0 conn(0x7fa668146a00 legacy=0x7fa6680b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:16.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.592+0000 7fa667fff640 1 -- 172.21.15.83:0/3956800565 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa66806cbb0 con 0x7fa6680b2d80 2024-04-17T10:38:16.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.592+0000 7fa667fff640 1 --2- 172.21.15.83:0/3956800565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6680b2d80 0x7fa668146e70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa66806e560 tx=0x7fa65c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61162e246a076e3c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:16.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.592+0000 7fa6677fe640 1 -- 172.21.15.83:0/3956800565 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65c0ab430 con 0x7fa6680b2d80 2024-04-17T10:38:16.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.592+0000 7fa6677fe640 1 -- 172.21.15.83:0/3956800565 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65c0b0070 con 0x7fa6680b2d80 2024-04-17T10:38:16.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.592+0000 7fa6677fe640 1 -- 172.21.15.83:0/3956800565 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65c0ab7f0 con 0x7fa6680b2d80 2024-04-17T10:38:16.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.592+0000 7fa66ec03640 1 -- 172.21.15.83:0/3956800565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6680b2d80 msgr2=0x7fa668146e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:16.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.592+0000 7fa66ec03640 1 --2- 172.21.15.83:0/3956800565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6680b2d80 0x7fa668146e70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa66806e560 tx=0x7fa65c01c520 comp rx=0 tx=0).stop 2024-04-17T10:38:16.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.592+0000 7fa66ec03640 1 -- 172.21.15.83:0/3956800565 shutdown_connections 2024-04-17T10:38:16.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.592+0000 7fa66ec03640 1 --2- 172.21.15.83:0/3956800565 >> v2:172.21.15.167:3300/0 conn(0x7fa66806d690 0x7fa6681620d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:16.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.592+0000 7fa66ec03640 1 --2- 172.21.15.83:0/3956800565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6680b2d80 0x7fa668146e70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:16.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.592+0000 7fa66ec03640 1 -- 172.21.15.83:0/3956800565 >> 172.21.15.83:0/3956800565 conn(0x7fa66806abe0 msgr2=0x7fa66806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:16.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.592+0000 7fa66ec03640 1 -- 172.21.15.83:0/3956800565 shutdown_connections 2024-04-17T10:38:16.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.592+0000 7fa66ec03640 1 -- 172.21.15.83:0/3956800565 wait complete. 2024-04-17T10:38:16.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.596+0000 7fa66ec03640 1 Processor -- start 2024-04-17T10:38:16.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.596+0000 7fa66ec03640 1 -- start start 2024-04-17T10:38:16.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.596+0000 7fa66ec03640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa66806d690 0x7fa66813d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:16.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.596+0000 7fa66ec03640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6680b2d80 0x7fa66815d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:16.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.596+0000 7fa66ec03640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa668146a00 0x7fa66813b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:16.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.596+0000 7fa66ec03640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa668071010 con 0x7fa66806d690 2024-04-17T10:38:16.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.596+0000 7fa66ec03640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa668070e90 con 0x7fa668146a00 2024-04-17T10:38:16.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.596+0000 7fa66ec03640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa66813def0 con 0x7fa6680b2d80 2024-04-17T10:38:16.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.596+0000 7fa66c99f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa66806d690 0x7fa66813d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:16.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.596+0000 7fa66c99f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa66806d690 0x7fa66813d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56008/0 (socket says 172.21.15.83:56008) 2024-04-17T10:38:16.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.596+0000 7fa66c99f640 1 -- 172.21.15.83:0/973135384 learned_addr learned my addr 172.21.15.83:0/973135384 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:16.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.596+0000 7fa66d1a0640 1 --2- 172.21.15.83:0/973135384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa668146a00 0x7fa66813b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:16.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.596+0000 7fa667fff640 1 --2- 172.21.15.83:0/973135384 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6680b2d80 0x7fa66815d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:16.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.596+0000 7fa66c99f640 1 -- 172.21.15.83:0/973135384 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6680b2d80 msgr2=0x7fa66815d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:16.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.596+0000 7fa66c99f640 1 --2- 172.21.15.83:0/973135384 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6680b2d80 0x7fa66815d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:16.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.596+0000 7fa66c99f640 1 -- 172.21.15.83:0/973135384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa668146a00 msgr2=0x7fa66813b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:16.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.596+0000 7fa66c99f640 1 --2- 172.21.15.83:0/973135384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa668146a00 0x7fa66813b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:16.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.596+0000 7fa66c99f640 1 -- 172.21.15.83:0/973135384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa66813e1d0 con 0x7fa66806d690 2024-04-17T10:38:16.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.596+0000 7fa66d1a0640 1 --2- 172.21.15.83:0/973135384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa668146a00 0x7fa66813b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:38:16.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.596+0000 7fa66c99f640 1 --2- 172.21.15.83:0/973135384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa66806d690 0x7fa66813d910 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa65800c730 tx=0x7fa65800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:16.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.596+0000 7fa6657fa640 1 -- 172.21.15.83:0/973135384 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa658016020 con 0x7fa66806d690 2024-04-17T10:38:16.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.596+0000 7fa6657fa640 1 -- 172.21.15.83:0/973135384 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa658006d60 con 0x7fa66806d690 2024-04-17T10:38:16.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.596+0000 7fa66ec03640 1 -- 172.21.15.83:0/973135384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa66815e100 con 0x7fa66806d690 2024-04-17T10:38:16.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.596+0000 7fa6657fa640 1 -- 172.21.15.83:0/973135384 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa658003bc0 con 0x7fa66806d690 2024-04-17T10:38:16.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.596+0000 7fa66ec03640 1 -- 172.21.15.83:0/973135384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa66815e6c0 con 0x7fa66806d690 2024-04-17T10:38:16.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.596+0000 7fa66ec03640 1 -- 172.21.15.83:0/973135384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa668074f90 con 0x7fa66806d690 2024-04-17T10:38:16.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.600+0000 7fa6657fa640 1 -- 172.21.15.83:0/973135384 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa65800a850 con 0x7fa66806d690 2024-04-17T10:38:16.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.600+0000 7fa6657fa640 1 --2- 172.21.15.83:0/973135384 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa654066f90 0x7fa654069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:16.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.600+0000 7fa6657fa640 1 -- 172.21.15.83:0/973135384 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa658084c10 con 0x7fa66806d690 2024-04-17T10:38:16.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.600+0000 7fa667fff640 1 --2- 172.21.15.83:0/973135384 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa654066f90 0x7fa654069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:16.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.600+0000 7fa667fff640 1 --2- 172.21.15.83:0/973135384 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa654066f90 0x7fa654069450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa65c0af000 tx=0x7fa65c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.604+0000 7fa6657fa640 1 -- 172.21.15.83:0/973135384 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa658020020 con 0x7fa66806d690 2024-04-17T10:38:16.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.812+0000 7fa66ec03640 1 -- 172.21.15.83:0/973135384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa668146ef0 con 0x7fa66806d690 2024-04-17T10:38:16.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.812+0000 7fa6657fa640 1 -- 172.21.15.83:0/973135384 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1011 (secure 0 0 0) 0x7fa658055cb0 con 0x7fa66806d690 2024-04-17T10:38:16.809 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:16.809 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":29,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":20,"num_osds":8,"num_up_osds":8,"osd_up_since":1713350279,"num_in_osds":8,"osd_in_since":1713350270,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592778,"bytes_used":224219136,"bytes_avail":772869894144,"bytes_total":773094113280,"write_bytes_sec":2601,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":6,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"d","status":"up:active","gid":4273}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-04-17T10:37:47.302694+0000","services":{}},"progress_events":{}} 2024-04-17T10:38:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.816+0000 7fa66ec03640 1 -- 172.21.15.83:0/973135384 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa654066f90 msgr2=0x7fa654069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.816+0000 7fa66ec03640 1 --2- 172.21.15.83:0/973135384 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa654066f90 0x7fa654069450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa65c0af000 tx=0x7fa65c008040 comp rx=0 tx=0).stop 2024-04-17T10:38:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.816+0000 7fa66ec03640 1 -- 172.21.15.83:0/973135384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa66806d690 msgr2=0x7fa66813d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.816+0000 7fa66ec03640 1 --2- 172.21.15.83:0/973135384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa66806d690 0x7fa66813d910 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa65800c730 tx=0x7fa65800cbf0 comp rx=0 tx=0).stop 2024-04-17T10:38:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.816+0000 7fa66ec03640 1 -- 172.21.15.83:0/973135384 shutdown_connections 2024-04-17T10:38:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.816+0000 7fa66ec03640 1 --2- 172.21.15.83:0/973135384 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa654066f90 0x7fa654069450 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.816+0000 7fa66ec03640 1 --2- 172.21.15.83:0/973135384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa668146a00 0x7fa66813b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.816+0000 7fa66ec03640 1 --2- 172.21.15.83:0/973135384 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6680b2d80 0x7fa66815d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.816+0000 7fa66ec03640 1 --2- 172.21.15.83:0/973135384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa66806d690 0x7fa66813d910 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.816+0000 7fa66ec03640 1 -- 172.21.15.83:0/973135384 >> 172.21.15.83:0/973135384 conn(0x7fa66806abe0 msgr2=0x7fa6680ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.816+0000 7fa66ec03640 1 -- 172.21.15.83:0/973135384 shutdown_connections 2024-04-17T10:38:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.816+0000 7fa66ec03640 1 -- 172.21.15.83:0/973135384 wait complete. 2024-04-17T10:38:16.827 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-17T10:38:16.827 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:38:16.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.940+0000 7f37ec40b640 1 Processor -- start 2024-04-17T10:38:16.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.940+0000 7f37ec40b640 1 -- start start 2024-04-17T10:38:16.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.940+0000 7f37ec40b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f37e40d2730 0x7f37e406b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:16.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.940+0000 7f37ec40b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f37e415e1b0 0x7f37e406b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:16.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.940+0000 7f37ec40b640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37e40d1940 con 0x7f37e415ddd0 2024-04-17T10:38:16.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.940+0000 7f37ec40b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37e40cfc70 con 0x7f37e40d2730 2024-04-17T10:38:16.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.940+0000 7f37ec40b640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37e40d1c20 con 0x7f37e415e1b0 2024-04-17T10:38:16.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.940+0000 7f37e99a6640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f37e40d2730 0x7f37e406b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:16.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.940+0000 7f37e99a6640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f37e40d2730 0x7f37e406b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47870/0 (socket says 172.21.15.83:47870) 2024-04-17T10:38:16.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.940+0000 7f37e99a6640 1 -- 172.21.15.83:0/2456603604 learned_addr learned my addr 172.21.15.83:0/2456603604 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:16.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.940+0000 7f37e99a6640 1 -- 172.21.15.83:0/2456603604 >> v2:172.21.15.167:3300/0 conn(0x7f37e415e1b0 msgr2=0x7f37e406b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:16.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.940+0000 7f37e99a6640 1 --2- 172.21.15.83:0/2456603604 >> v2:172.21.15.167:3300/0 conn(0x7f37e415e1b0 0x7f37e406b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:16.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.940+0000 7f37e99a6640 1 -- 172.21.15.83:0/2456603604 >> v1:172.21.15.167:6789/0 conn(0x7f37e415ddd0 legacy=0x7f37e40d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:16.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.940+0000 7f37e99a6640 1 -- 172.21.15.83:0/2456603604 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e40cf900 con 0x7f37e40d2730 2024-04-17T10:38:16.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.944+0000 7f37e99a6640 1 --2- 172.21.15.83:0/2456603604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37e40d2730 0x7f37e406b3d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f37e408e140 tx=0x7f37d401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22666274d5fab9c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:16.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.944+0000 7f37e91a5640 1 -- 172.21.15.83:0/2456603604 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d40a7070 con 0x7f37e40d2730 2024-04-17T10:38:16.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.944+0000 7f37e91a5640 1 -- 172.21.15.83:0/2456603604 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d40b0070 con 0x7f37e40d2730 2024-04-17T10:38:16.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.944+0000 7f37e91a5640 1 -- 172.21.15.83:0/2456603604 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d40ab890 con 0x7f37e40d2730 2024-04-17T10:38:16.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.944+0000 7f37ec40b640 1 -- 172.21.15.83:0/2456603604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37e40d2730 msgr2=0x7f37e406b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:16.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.944+0000 7f37ec40b640 1 --2- 172.21.15.83:0/2456603604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37e40d2730 0x7f37e406b3d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f37e408e140 tx=0x7f37d401c450 comp rx=0 tx=0).stop 2024-04-17T10:38:16.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.944+0000 7f37ec40b640 1 -- 172.21.15.83:0/2456603604 shutdown_connections 2024-04-17T10:38:16.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.944+0000 7f37ec40b640 1 --2- 172.21.15.83:0/2456603604 >> v2:172.21.15.167:3300/0 conn(0x7f37e415e1b0 0x7f37e406b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:16.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.944+0000 7f37ec40b640 1 --2- 172.21.15.83:0/2456603604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37e40d2730 0x7f37e406b3d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:16.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.944+0000 7f37ec40b640 1 -- 172.21.15.83:0/2456603604 >> 172.21.15.83:0/2456603604 conn(0x7f37e40d3a40 msgr2=0x7f37e40d3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:16.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.944+0000 7f37ec40b640 1 -- 172.21.15.83:0/2456603604 shutdown_connections 2024-04-17T10:38:16.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.944+0000 7f37ec40b640 1 -- 172.21.15.83:0/2456603604 wait complete. 2024-04-17T10:38:16.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.944+0000 7f37ec40b640 1 Processor -- start 2024-04-17T10:38:16.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.944+0000 7f37ec40b640 1 -- start start 2024-04-17T10:38:16.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.948+0000 7f37ec40b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37e40d2730 0x7f37e414bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:16.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.948+0000 7f37ec40b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37e415ddd0 0x7f37e40b1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:16.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.948+0000 7f37ec40b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37e415e1b0 0x7f37e40b1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:16.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.948+0000 7f37ec40b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e416af50 con 0x7f37e415e1b0 2024-04-17T10:38:16.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.948+0000 7f37ec40b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e416add0 con 0x7f37e415ddd0 2024-04-17T10:38:16.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.948+0000 7f37ec40b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e416b0d0 con 0x7f37e40d2730 2024-04-17T10:38:16.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.948+0000 7f37ea9a8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37e415e1b0 0x7f37e40b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:16.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.948+0000 7f37ea9a8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37e415e1b0 0x7f37e40b1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47874/0 (socket says 172.21.15.83:47874) 2024-04-17T10:38:16.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.948+0000 7f37ea9a8640 1 -- 172.21.15.83:0/3090409049 learned_addr learned my addr 172.21.15.83:0/3090409049 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:16.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.948+0000 7f37e99a6640 1 --2- 172.21.15.83:0/3090409049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37e415ddd0 0x7f37e40b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:16.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.948+0000 7f37ea9a8640 1 -- 172.21.15.83:0/3090409049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37e40d2730 msgr2=0x7f37e414bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:16.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.948+0000 7f37ea9a8640 1 --2- 172.21.15.83:0/3090409049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37e40d2730 0x7f37e414bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:16.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.948+0000 7f37ea9a8640 1 -- 172.21.15.83:0/3090409049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37e415ddd0 msgr2=0x7f37e40b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:16.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.948+0000 7f37ea9a8640 1 --2- 172.21.15.83:0/3090409049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37e415ddd0 0x7f37e40b1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:16.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.948+0000 7f37ea9a8640 1 -- 172.21.15.83:0/3090409049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e408dd40 con 0x7f37e415e1b0 2024-04-17T10:38:16.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.948+0000 7f37ea9a8640 1 --2- 172.21.15.83:0/3090409049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37e415e1b0 0x7f37e40b1940 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00b9a0 tx=0x7f37dc00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:16.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.948+0000 7f37daffd640 1 -- 172.21.15.83:0/3090409049 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc019070 con 0x7f37e415e1b0 2024-04-17T10:38:16.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.948+0000 7f37daffd640 1 -- 172.21.15.83:0/3090409049 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37dc00cbc0 con 0x7f37e415e1b0 2024-04-17T10:38:16.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.948+0000 7f37daffd640 1 -- 172.21.15.83:0/3090409049 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc014440 con 0x7f37e415e1b0 2024-04-17T10:38:16.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.948+0000 7f37ec40b640 1 -- 172.21.15.83:0/3090409049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37e40c9600 con 0x7f37e415e1b0 2024-04-17T10:38:16.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.948+0000 7f37ec40b640 1 -- 172.21.15.83:0/3090409049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37e40c9bc0 con 0x7f37e415e1b0 2024-04-17T10:38:16.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.948+0000 7f37ec40b640 1 -- 172.21.15.83:0/3090409049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37e40681a0 con 0x7f37e415e1b0 2024-04-17T10:38:16.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.948+0000 7f37daffd640 1 -- 172.21.15.83:0/3090409049 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f37dc01c070 con 0x7f37e415e1b0 2024-04-17T10:38:16.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.948+0000 7f37daffd640 1 --2- 172.21.15.83:0/3090409049 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f37c8066f60 0x7f37c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:16.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.948+0000 7f37daffd640 1 -- 172.21.15.83:0/3090409049 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f37dc084d90 con 0x7f37e415e1b0 2024-04-17T10:38:16.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.948+0000 7f37ea1a7640 1 --2- 172.21.15.83:0/3090409049 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f37c8066f60 0x7f37c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:16.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.948+0000 7f37ea1a7640 1 --2- 172.21.15.83:0/3090409049 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f37c8066f60 0x7f37c8069420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f37e0003cf0 tx=0x7f37e00079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:16.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:16.952+0000 7f37daffd640 1 -- 172.21.15.83:0/3090409049 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f37dc055d80 con 0x7f37e415e1b0 2024-04-17T10:38:17.029 INFO:tasks.ceph.mds.d.smithi167.stderr:2024-04-17T10:38:17.033+0000 7f2b77b6d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T10:38:17.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:17.108+0000 7f37ec40b640 1 -- 172.21.15.83:0/3090409049 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f37e4074f90 con 0x7f37c8066f60 2024-04-17T10:38:17.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:17.116+0000 7f37daffd640 1 -- 172.21.15.83:0/3090409049 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251946 (secure 0 0 0) 0x7f37e4074f90 con 0x7f37c8066f60 2024-04-17T10:38:17.116 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:17.118 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:38:17.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:17.124+0000 7f37ec40b640 1 -- 172.21.15.83:0/3090409049 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f37c8066f60 msgr2=0x7f37c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:17.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:17.124+0000 7f37ec40b640 1 --2- 172.21.15.83:0/3090409049 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f37c8066f60 0x7f37c8069420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f37e0003cf0 tx=0x7f37e00079c0 comp rx=0 tx=0).stop 2024-04-17T10:38:17.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:17.124+0000 7f37ec40b640 1 -- 172.21.15.83:0/3090409049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37e415e1b0 msgr2=0x7f37e40b1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:17.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:17.124+0000 7f37ec40b640 1 --2- 172.21.15.83:0/3090409049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37e415e1b0 0x7f37e40b1940 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00b9a0 tx=0x7f37dc00be60 comp rx=0 tx=0).stop 2024-04-17T10:38:17.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:17.124+0000 7f37ec40b640 1 -- 172.21.15.83:0/3090409049 shutdown_connections 2024-04-17T10:38:17.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:17.124+0000 7f37ec40b640 1 --2- 172.21.15.83:0/3090409049 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f37c8066f60 0x7f37c8069420 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:17.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:17.124+0000 7f37ec40b640 1 --2- 172.21.15.83:0/3090409049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37e415e1b0 0x7f37e40b1940 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:17.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:17.124+0000 7f37ec40b640 1 --2- 172.21.15.83:0/3090409049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37e415ddd0 0x7f37e40b1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:17.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:17.124+0000 7f37ec40b640 1 --2- 172.21.15.83:0/3090409049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37e40d2730 0x7f37e414bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:17.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:17.124+0000 7f37ec40b640 1 -- 172.21.15.83:0/3090409049 >> 172.21.15.83:0/3090409049 conn(0x7f37e40d3a40 msgr2=0x7f37e40b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:17.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:17.124+0000 7f37ec40b640 1 -- 172.21.15.83:0/3090409049 shutdown_connections 2024-04-17T10:38:17.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:17.124+0000 7f37ec40b640 1 -- 172.21.15.83:0/3090409049 wait complete. 2024-04-17T10:38:17.135 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-04-17T10:38:16.180839+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218964,"kb_used_data":2776,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755756,"statfs":{"total":773094113280,"available":772869894144,"internally_reserved":0,"allocated":2842624,"data_stored":1727828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.510640"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702753+0000","last_change":"2024-04-17T10:38:12.718400+0000","last_active":"2024-04-17T10:38:13.702753+0000","last_peered":"2024-04-17T10:38:13.702753+0000","last_clean":"2024-04-17T10:38:13.702753+0000","last_became_active":"2024-04-17T10:38:12.718121+0000","last_became_peered":"2024-04-17T10:38:12.718121+0000","last_unstale":"2024-04-17T10:38:13.702753+0000","last_undegraded":"2024-04-17T10:38:13.702753+0000","last_fullsized":"2024-04-17T10:38:13.702753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:37.203406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695895+0000","last_change":"2024-04-17T10:38:12.710479+0000","last_active":"2024-04-17T10:38:13.695895+0000","last_peered":"2024-04-17T10:38:13.695895+0000","last_clean":"2024-04-17T10:38:13.695895+0000","last_became_active":"2024-04-17T10:38:12.710092+0000","last_became_peered":"2024-04-17T10:38:12.710092+0000","last_unstale":"2024-04-17T10:38:13.695895+0000","last_undegraded":"2024-04-17T10:38:13.695895+0000","last_fullsized":"2024-04-17T10:38:13.695895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:43.917742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305973+0000","last_change":"2024-04-17T10:38:12.722110+0000","last_active":"2024-04-17T10:38:14.305973+0000","last_peered":"2024-04-17T10:38:14.305973+0000","last_clean":"2024-04-17T10:38:14.305973+0000","last_became_active":"2024-04-17T10:38:12.721720+0000","last_became_peered":"2024-04-17T10:38:12.721720+0000","last_unstale":"2024-04-17T10:38:14.305973+0000","last_undegraded":"2024-04-17T10:38:14.305973+0000","last_fullsized":"2024-04-17T10:38:14.305973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:43.917742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970398+0000","last_change":"2024-04-17T10:38:12.727777+0000","last_active":"2024-04-17T10:38:13.970398+0000","last_peered":"2024-04-17T10:38:13.970398+0000","last_clean":"2024-04-17T10:38:13.970398+0000","last_became_active":"2024-04-17T10:38:12.726952+0000","last_became_peered":"2024-04-17T10:38:12.726952+0000","last_unstale":"2024-04-17T10:38:13.970398+0000","last_undegraded":"2024-04-17T10:38:13.970398+0000","last_fullsized":"2024-04-17T10:38:13.970398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:06.508830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701762+0000","last_change":"2024-04-17T10:38:12.721627+0000","last_active":"2024-04-17T10:38:13.701762+0000","last_peered":"2024-04-17T10:38:13.701762+0000","last_clean":"2024-04-17T10:38:13.701762+0000","last_became_active":"2024-04-17T10:38:12.718951+0000","last_became_peered":"2024-04-17T10:38:12.718951+0000","last_unstale":"2024-04-17T10:38:13.701762+0000","last_undegraded":"2024-04-17T10:38:13.701762+0000","last_fullsized":"2024-04-17T10:38:13.701762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:02.184227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970163+0000","last_change":"2024-04-17T10:38:12.729603+0000","last_active":"2024-04-17T10:38:13.970163+0000","last_peered":"2024-04-17T10:38:13.970163+0000","last_clean":"2024-04-17T10:38:13.970163+0000","last_became_active":"2024-04-17T10:38:12.728738+0000","last_became_peered":"2024-04-17T10:38:12.728738+0000","last_unstale":"2024-04-17T10:38:13.970163+0000","last_undegraded":"2024-04-17T10:38:13.970163+0000","last_fullsized":"2024-04-17T10:38:13.970163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:54:26.054113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701635+0000","last_change":"2024-04-17T10:38:12.708732+0000","last_active":"2024-04-17T10:38:13.701635+0000","last_peered":"2024-04-17T10:38:13.701635+0000","last_clean":"2024-04-17T10:38:13.701635+0000","last_became_active":"2024-04-17T10:38:12.708291+0000","last_became_peered":"2024-04-17T10:38:12.708291+0000","last_unstale":"2024-04-17T10:38:13.701635+0000","last_undegraded":"2024-04-17T10:38:13.701635+0000","last_fullsized":"2024-04-17T10:38:13.701635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:54:26.054113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297197+0000","last_change":"2024-04-17T10:38:12.726672+0000","last_active":"2024-04-17T10:38:14.297197+0000","last_peered":"2024-04-17T10:38:14.297197+0000","last_clean":"2024-04-17T10:38:14.297197+0000","last_became_active":"2024-04-17T10:38:12.726395+0000","last_became_peered":"2024-04-17T10:38:12.726395+0000","last_unstale":"2024-04-17T10:38:14.297197+0000","last_undegraded":"2024-04-17T10:38:14.297197+0000","last_fullsized":"2024-04-17T10:38:14.297197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:31:48.663733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702702+0000","last_change":"2024-04-17T10:38:12.727392+0000","last_active":"2024-04-17T10:38:13.702702+0000","last_peered":"2024-04-17T10:38:13.702702+0000","last_clean":"2024-04-17T10:38:13.702702+0000","last_became_active":"2024-04-17T10:38:12.727171+0000","last_became_peered":"2024-04-17T10:38:12.727171+0000","last_unstale":"2024-04-17T10:38:13.702702+0000","last_undegraded":"2024-04-17T10:38:13.702702+0000","last_fullsized":"2024-04-17T10:38:13.702702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:11:49.847070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803824+0000","last_change":"2024-04-17T10:38:12.726158+0000","last_active":"2024-04-17T10:38:13.803824+0000","last_peered":"2024-04-17T10:38:13.803824+0000","last_clean":"2024-04-17T10:38:13.803824+0000","last_became_active":"2024-04-17T10:38:12.724920+0000","last_became_peered":"2024-04-17T10:38:12.724920+0000","last_unstale":"2024-04-17T10:38:13.803824+0000","last_undegraded":"2024-04-17T10:38:13.803824+0000","last_fullsized":"2024-04-17T10:38:13.803824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:43.917742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803453+0000","last_change":"2024-04-17T10:38:12.725420+0000","last_active":"2024-04-17T10:38:13.803453+0000","last_peered":"2024-04-17T10:38:13.803453+0000","last_clean":"2024-04-17T10:38:13.803453+0000","last_became_active":"2024-04-17T10:38:12.724959+0000","last_became_peered":"2024-04-17T10:38:12.724959+0000","last_unstale":"2024-04-17T10:38:13.803453+0000","last_undegraded":"2024-04-17T10:38:13.803453+0000","last_fullsized":"2024-04-17T10:38:13.803453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:39.640730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.699955+0000","last_change":"2024-04-17T10:38:12.712887+0000","last_active":"2024-04-17T10:38:13.699955+0000","last_peered":"2024-04-17T10:38:13.699955+0000","last_clean":"2024-04-17T10:38:13.699955+0000","last_became_active":"2024-04-17T10:38:12.712582+0000","last_became_peered":"2024-04-17T10:38:12.712582+0000","last_unstale":"2024-04-17T10:38:13.699955+0000","last_undegraded":"2024-04-17T10:38:13.699955+0000","last_fullsized":"2024-04-17T10:38:13.699955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:23:30.964531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298210+0000","last_change":"2024-04-17T10:38:12.722191+0000","last_active":"2024-04-17T10:38:14.298210+0000","last_peered":"2024-04-17T10:38:14.298210+0000","last_clean":"2024-04-17T10:38:14.298210+0000","last_became_active":"2024-04-17T10:38:12.721912+0000","last_became_peered":"2024-04-17T10:38:12.721912+0000","last_unstale":"2024-04-17T10:38:14.298210+0000","last_undegraded":"2024-04-17T10:38:14.298210+0000","last_fullsized":"2024-04-17T10:38:14.298210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:09:07.090758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.304662+0000","last_change":"2024-04-17T10:38:12.710211+0000","last_active":"2024-04-17T10:38:14.304662+0000","last_peered":"2024-04-17T10:38:14.304662+0000","last_clean":"2024-04-17T10:38:14.304662+0000","last_became_active":"2024-04-17T10:38:12.709818+0000","last_became_peered":"2024-04-17T10:38:12.709818+0000","last_unstale":"2024-04-17T10:38:14.304662+0000","last_undegraded":"2024-04-17T10:38:14.304662+0000","last_fullsized":"2024-04-17T10:38:14.304662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:23:26.747986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695484+0000","last_change":"2024-04-17T10:38:12.722762+0000","last_active":"2024-04-17T10:38:13.695484+0000","last_peered":"2024-04-17T10:38:13.695484+0000","last_clean":"2024-04-17T10:38:13.695484+0000","last_became_active":"2024-04-17T10:38:12.718394+0000","last_became_peered":"2024-04-17T10:38:12.718394+0000","last_unstale":"2024-04-17T10:38:13.695484+0000","last_undegraded":"2024-04-17T10:38:13.695484+0000","last_fullsized":"2024-04-17T10:38:13.695484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:08.872431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.969994+0000","last_change":"2024-04-17T10:38:12.729270+0000","last_active":"2024-04-17T10:38:13.969994+0000","last_peered":"2024-04-17T10:38:13.969994+0000","last_clean":"2024-04-17T10:38:13.969994+0000","last_became_active":"2024-04-17T10:38:12.728798+0000","last_became_peered":"2024-04-17T10:38:12.728798+0000","last_unstale":"2024-04-17T10:38:13.969994+0000","last_undegraded":"2024-04-17T10:38:13.969994+0000","last_fullsized":"2024-04-17T10:38:13.969994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:23.465112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.971229+0000","last_change":"2024-04-17T10:38:12.731612+0000","last_active":"2024-04-17T10:38:13.971229+0000","last_peered":"2024-04-17T10:38:13.971229+0000","last_clean":"2024-04-17T10:38:13.971229+0000","last_became_active":"2024-04-17T10:38:12.731071+0000","last_became_peered":"2024-04-17T10:38:12.731071+0000","last_unstale":"2024-04-17T10:38:13.971229+0000","last_undegraded":"2024-04-17T10:38:13.971229+0000","last_fullsized":"2024-04-17T10:38:13.971229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:37.203406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.303651+0000","last_change":"2024-04-17T10:38:12.720913+0000","last_active":"2024-04-17T10:38:14.303651+0000","last_peered":"2024-04-17T10:38:14.303651+0000","last_clean":"2024-04-17T10:38:14.303651+0000","last_became_active":"2024-04-17T10:38:12.719779+0000","last_became_peered":"2024-04-17T10:38:12.719779+0000","last_unstale":"2024-04-17T10:38:14.303651+0000","last_undegraded":"2024-04-17T10:38:14.303651+0000","last_fullsized":"2024-04-17T10:38:14.303651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:23:26.747986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702068+0000","last_change":"2024-04-17T10:38:12.719229+0000","last_active":"2024-04-17T10:38:13.702068+0000","last_peered":"2024-04-17T10:38:13.702068+0000","last_clean":"2024-04-17T10:38:13.702068+0000","last_became_active":"2024-04-17T10:38:12.718934+0000","last_became_peered":"2024-04-17T10:38:12.718934+0000","last_unstale":"2024-04-17T10:38:13.702068+0000","last_undegraded":"2024-04-17T10:38:13.702068+0000","last_fullsized":"2024-04-17T10:38:13.702068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:38:39.116589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803564+0000","last_change":"2024-04-17T10:38:12.717999+0000","last_active":"2024-04-17T10:38:13.803564+0000","last_peered":"2024-04-17T10:38:13.803564+0000","last_clean":"2024-04-17T10:38:13.803564+0000","last_became_active":"2024-04-17T10:38:12.716817+0000","last_became_peered":"2024-04-17T10:38:12.716817+0000","last_unstale":"2024-04-17T10:38:13.803564+0000","last_undegraded":"2024-04-17T10:38:13.803564+0000","last_fullsized":"2024-04-17T10:38:13.803564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:42.256740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.304981+0000","last_change":"2024-04-17T10:38:12.710385+0000","last_active":"2024-04-17T10:38:14.304981+0000","last_peered":"2024-04-17T10:38:14.304981+0000","last_clean":"2024-04-17T10:38:14.304981+0000","last_became_active":"2024-04-17T10:38:12.707950+0000","last_became_peered":"2024-04-17T10:38:12.707950+0000","last_unstale":"2024-04-17T10:38:14.304981+0000","last_undegraded":"2024-04-17T10:38:14.304981+0000","last_fullsized":"2024-04-17T10:38:14.304981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:44:43.972627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970232+0000","last_change":"2024-04-17T10:38:12.731401+0000","last_active":"2024-04-17T10:38:13.970232+0000","last_peered":"2024-04-17T10:38:13.970232+0000","last_clean":"2024-04-17T10:38:13.970232+0000","last_became_active":"2024-04-17T10:38:12.729441+0000","last_became_peered":"2024-04-17T10:38:12.729441+0000","last_unstale":"2024-04-17T10:38:13.970232+0000","last_undegraded":"2024-04-17T10:38:13.970232+0000","last_fullsized":"2024-04-17T10:38:13.970232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:02.184227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297301+0000","last_change":"2024-04-17T10:38:12.725791+0000","last_active":"2024-04-17T10:38:14.297301+0000","last_peered":"2024-04-17T10:38:14.297301+0000","last_clean":"2024-04-17T10:38:14.297301+0000","last_became_active":"2024-04-17T10:38:12.723417+0000","last_became_peered":"2024-04-17T10:38:12.723417+0000","last_unstale":"2024-04-17T10:38:14.297301+0000","last_undegraded":"2024-04-17T10:38:14.297301+0000","last_fullsized":"2024-04-17T10:38:14.297301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:26:02.126931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970408+0000","last_change":"2024-04-17T10:38:12.727592+0000","last_active":"2024-04-17T10:38:13.970408+0000","last_peered":"2024-04-17T10:38:13.970408+0000","last_clean":"2024-04-17T10:38:13.970408+0000","last_became_active":"2024-04-17T10:38:12.726482+0000","last_became_peered":"2024-04-17T10:38:12.726482+0000","last_unstale":"2024-04-17T10:38:13.970408+0000","last_undegraded":"2024-04-17T10:38:13.970408+0000","last_fullsized":"2024-04-17T10:38:13.970408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:09:07.090758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297926+0000","last_change":"2024-04-17T10:38:12.716910+0000","last_active":"2024-04-17T10:38:14.297926+0000","last_peered":"2024-04-17T10:38:14.297926+0000","last_clean":"2024-04-17T10:38:14.297926+0000","last_became_active":"2024-04-17T10:38:12.716527+0000","last_became_peered":"2024-04-17T10:38:12.716527+0000","last_unstale":"2024-04-17T10:38:14.297926+0000","last_undegraded":"2024-04-17T10:38:14.297926+0000","last_fullsized":"2024-04-17T10:38:14.297926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:43.917742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803906+0000","last_change":"2024-04-17T10:38:12.728066+0000","last_active":"2024-04-17T10:38:13.803906+0000","last_peered":"2024-04-17T10:38:13.803906+0000","last_clean":"2024-04-17T10:38:13.803906+0000","last_became_active":"2024-04-17T10:38:12.727694+0000","last_became_peered":"2024-04-17T10:38:12.727694+0000","last_unstale":"2024-04-17T10:38:13.803906+0000","last_undegraded":"2024-04-17T10:38:13.803906+0000","last_fullsized":"2024-04-17T10:38:13.803906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:05:21.619291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695757+0000","last_change":"2024-04-17T10:38:12.714887+0000","last_active":"2024-04-17T10:38:13.695757+0000","last_peered":"2024-04-17T10:38:13.695757+0000","last_clean":"2024-04-17T10:38:13.695757+0000","last_became_active":"2024-04-17T10:38:12.714599+0000","last_became_peered":"2024-04-17T10:38:12.714599+0000","last_unstale":"2024-04-17T10:38:13.695757+0000","last_undegraded":"2024-04-17T10:38:13.695757+0000","last_fullsized":"2024-04-17T10:38:13.695757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:42.256740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297989+0000","last_change":"2024-04-17T10:38:12.725678+0000","last_active":"2024-04-17T10:38:14.297989+0000","last_peered":"2024-04-17T10:38:14.297989+0000","last_clean":"2024-04-17T10:38:14.297989+0000","last_became_active":"2024-04-17T10:38:12.723311+0000","last_became_peered":"2024-04-17T10:38:12.723311+0000","last_unstale":"2024-04-17T10:38:14.297989+0000","last_undegraded":"2024-04-17T10:38:14.297989+0000","last_fullsized":"2024-04-17T10:38:14.297989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:05:21.619291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305173+0000","last_change":"2024-04-17T10:38:12.720525+0000","last_active":"2024-04-17T10:38:14.305173+0000","last_peered":"2024-04-17T10:38:14.305173+0000","last_clean":"2024-04-17T10:38:14.305173+0000","last_became_active":"2024-04-17T10:38:12.720134+0000","last_became_peered":"2024-04-17T10:38:12.720134+0000","last_unstale":"2024-04-17T10:38:14.305173+0000","last_undegraded":"2024-04-17T10:38:14.305173+0000","last_fullsized":"2024-04-17T10:38:14.305173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:08.872431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970581+0000","last_change":"2024-04-17T10:38:12.727969+0000","last_active":"2024-04-17T10:38:13.970581+0000","last_peered":"2024-04-17T10:38:13.970581+0000","last_clean":"2024-04-17T10:38:13.970581+0000","last_became_active":"2024-04-17T10:38:12.726330+0000","last_became_peered":"2024-04-17T10:38:12.726330+0000","last_unstale":"2024-04-17T10:38:13.970581+0000","last_undegraded":"2024-04-17T10:38:13.970581+0000","last_fullsized":"2024-04-17T10:38:13.970581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:33:29.583300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.303139+0000","last_change":"2024-04-17T10:38:12.732783+0000","last_active":"2024-04-17T10:38:14.303139+0000","last_peered":"2024-04-17T10:38:14.303139+0000","last_clean":"2024-04-17T10:38:14.303139+0000","last_became_active":"2024-04-17T10:38:12.732525+0000","last_became_peered":"2024-04-17T10:38:12.732525+0000","last_unstale":"2024-04-17T10:38:14.303139+0000","last_undegraded":"2024-04-17T10:38:14.303139+0000","last_fullsized":"2024-04-17T10:38:14.303139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:21:11.071817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803187+0000","last_change":"2024-04-17T10:38:12.720029+0000","last_active":"2024-04-17T10:38:13.803187+0000","last_peered":"2024-04-17T10:38:13.803187+0000","last_clean":"2024-04-17T10:38:13.803187+0000","last_became_active":"2024-04-17T10:38:12.719579+0000","last_became_peered":"2024-04-17T10:38:12.719579+0000","last_unstale":"2024-04-17T10:38:13.803187+0000","last_undegraded":"2024-04-17T10:38:13.803187+0000","last_fullsized":"2024-04-17T10:38:13.803187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:31:48.663733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.699682+0000","last_change":"2024-04-17T10:38:12.709872+0000","last_active":"2024-04-17T10:38:13.699682+0000","last_peered":"2024-04-17T10:38:13.699682+0000","last_clean":"2024-04-17T10:38:13.699682+0000","last_became_active":"2024-04-17T10:38:12.709614+0000","last_became_peered":"2024-04-17T10:38:12.709614+0000","last_unstale":"2024-04-17T10:38:13.699682+0000","last_undegraded":"2024-04-17T10:38:13.699682+0000","last_fullsized":"2024-04-17T10:38:13.699682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:46.375063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298173+0000","last_change":"2024-04-17T10:38:11.718367+0000","last_active":"2024-04-17T10:38:14.298173+0000","last_peered":"2024-04-17T10:38:14.298173+0000","last_clean":"2024-04-17T10:38:14.298173+0000","last_became_active":"2024-04-17T10:38:11.718004+0000","last_became_peered":"2024-04-17T10:38:11.718004+0000","last_unstale":"2024-04-17T10:38:14.298173+0000","last_undegraded":"2024-04-17T10:38:14.298173+0000","last_fullsized":"2024-04-17T10:38:14.298173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:39.210602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305196+0000","last_change":"2024-04-17T10:38:12.710623+0000","last_active":"2024-04-17T10:38:14.305196+0000","last_peered":"2024-04-17T10:38:14.305196+0000","last_clean":"2024-04-17T10:38:14.305196+0000","last_became_active":"2024-04-17T10:38:12.708883+0000","last_became_peered":"2024-04-17T10:38:12.708883+0000","last_unstale":"2024-04-17T10:38:14.305196+0000","last_undegraded":"2024-04-17T10:38:14.305196+0000","last_fullsized":"2024-04-17T10:38:14.305196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:26:02.126931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803042+0000","last_change":"2024-04-17T10:38:11.718224+0000","last_active":"2024-04-17T10:38:13.803042+0000","last_peered":"2024-04-17T10:38:13.803042+0000","last_clean":"2024-04-17T10:38:13.803042+0000","last_became_active":"2024-04-17T10:38:11.717433+0000","last_became_peered":"2024-04-17T10:38:11.717433+0000","last_unstale":"2024-04-17T10:38:13.803042+0000","last_undegraded":"2024-04-17T10:38:13.803042+0000","last_fullsized":"2024-04-17T10:38:13.803042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:39.210602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298350+0000","last_change":"2024-04-17T10:38:11.719972+0000","last_active":"2024-04-17T10:38:14.298350+0000","last_peered":"2024-04-17T10:38:14.298350+0000","last_clean":"2024-04-17T10:38:14.298350+0000","last_became_active":"2024-04-17T10:38:11.719469+0000","last_became_peered":"2024-04-17T10:38:11.719469+0000","last_unstale":"2024-04-17T10:38:14.298350+0000","last_undegraded":"2024-04-17T10:38:14.298350+0000","last_fullsized":"2024-04-17T10:38:14.298350+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:59:51.693435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297286+0000","last_change":"2024-04-17T10:38:12.722655+0000","last_active":"2024-04-17T10:38:14.297286+0000","last_peered":"2024-04-17T10:38:14.297286+0000","last_clean":"2024-04-17T10:38:14.297286+0000","last_became_active":"2024-04-17T10:38:12.722213+0000","last_became_peered":"2024-04-17T10:38:12.722213+0000","last_unstale":"2024-04-17T10:38:14.297286+0000","last_undegraded":"2024-04-17T10:38:14.297286+0000","last_fullsized":"2024-04-17T10:38:14.297286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:08.872431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970337+0000","last_change":"2024-04-17T10:38:11.704898+0000","last_active":"2024-04-17T10:38:13.970337+0000","last_peered":"2024-04-17T10:38:13.970337+0000","last_clean":"2024-04-17T10:38:13.970337+0000","last_became_active":"2024-04-17T10:38:11.704184+0000","last_became_peered":"2024-04-17T10:38:11.704184+0000","last_unstale":"2024-04-17T10:38:13.970337+0000","last_undegraded":"2024-04-17T10:38:13.970337+0000","last_fullsized":"2024-04-17T10:38:13.970337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:45.366572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695486+0000","last_change":"2024-04-17T10:38:12.714844+0000","last_active":"2024-04-17T10:38:13.695486+0000","last_peered":"2024-04-17T10:38:13.695486+0000","last_clean":"2024-04-17T10:38:13.695486+0000","last_became_active":"2024-04-17T10:38:12.714565+0000","last_became_peered":"2024-04-17T10:38:12.714565+0000","last_unstale":"2024-04-17T10:38:13.695486+0000","last_undegraded":"2024-04-17T10:38:13.695486+0000","last_fullsized":"2024-04-17T10:38:13.695486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:26:02.126931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803577+0000","last_change":"2024-04-17T10:38:11.725276+0000","last_active":"2024-04-17T10:38:13.803577+0000","last_peered":"2024-04-17T10:38:13.803577+0000","last_clean":"2024-04-17T10:38:13.803577+0000","last_became_active":"2024-04-17T10:38:11.724660+0000","last_became_peered":"2024-04-17T10:38:11.724660+0000","last_unstale":"2024-04-17T10:38:13.803577+0000","last_undegraded":"2024-04-17T10:38:13.803577+0000","last_fullsized":"2024-04-17T10:38:13.803577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:21:00.724523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970977+0000","last_change":"2024-04-17T10:38:12.731384+0000","last_active":"2024-04-17T10:38:13.970977+0000","last_peered":"2024-04-17T10:38:13.970977+0000","last_clean":"2024-04-17T10:38:13.970977+0000","last_became_active":"2024-04-17T10:38:12.730178+0000","last_became_peered":"2024-04-17T10:38:12.730178+0000","last_unstale":"2024-04-17T10:38:13.970977+0000","last_undegraded":"2024-04-17T10:38:13.970977+0000","last_fullsized":"2024-04-17T10:38:13.970977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:11:49.847070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970956+0000","last_change":"2024-04-17T10:38:11.727591+0000","last_active":"2024-04-17T10:38:13.970956+0000","last_peered":"2024-04-17T10:38:13.970956+0000","last_clean":"2024-04-17T10:38:13.970956+0000","last_became_active":"2024-04-17T10:38:11.726319+0000","last_became_peered":"2024-04-17T10:38:11.726319+0000","last_unstale":"2024-04-17T10:38:13.970956+0000","last_undegraded":"2024-04-17T10:38:13.970956+0000","last_fullsized":"2024-04-17T10:38:13.970956+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:50:59.373356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305290+0000","last_change":"2024-04-17T10:38:12.727239+0000","last_active":"2024-04-17T10:38:14.305290+0000","last_peered":"2024-04-17T10:38:14.305290+0000","last_clean":"2024-04-17T10:38:14.305290+0000","last_became_active":"2024-04-17T10:38:12.726192+0000","last_became_peered":"2024-04-17T10:38:12.726192+0000","last_unstale":"2024-04-17T10:38:14.305290+0000","last_undegraded":"2024-04-17T10:38:14.305290+0000","last_fullsized":"2024-04-17T10:38:14.305290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:39.640730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695810+0000","last_change":"2024-04-17T10:38:11.710534+0000","last_active":"2024-04-17T10:38:13.695810+0000","last_peered":"2024-04-17T10:38:13.695810+0000","last_clean":"2024-04-17T10:38:13.695810+0000","last_became_active":"2024-04-17T10:38:11.709944+0000","last_became_peered":"2024-04-17T10:38:11.709944+0000","last_unstale":"2024-04-17T10:38:13.695810+0000","last_undegraded":"2024-04-17T10:38:13.695810+0000","last_fullsized":"2024-04-17T10:38:13.695810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:45.366572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.700329+0000","last_change":"2024-04-17T10:38:12.709577+0000","last_active":"2024-04-17T10:38:13.700329+0000","last_peered":"2024-04-17T10:38:13.700329+0000","last_clean":"2024-04-17T10:38:13.700329+0000","last_became_active":"2024-04-17T10:38:12.709186+0000","last_became_peered":"2024-04-17T10:38:12.709186+0000","last_unstale":"2024-04-17T10:38:13.700329+0000","last_undegraded":"2024-04-17T10:38:13.700329+0000","last_fullsized":"2024-04-17T10:38:13.700329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:20:45.810408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803828+0000","last_change":"2024-04-17T10:38:11.715117+0000","last_active":"2024-04-17T10:38:13.803828+0000","last_peered":"2024-04-17T10:38:13.803828+0000","last_clean":"2024-04-17T10:38:13.803828+0000","last_became_active":"2024-04-17T10:38:11.713981+0000","last_became_peered":"2024-04-17T10:38:11.713981+0000","last_unstale":"2024-04-17T10:38:13.803828+0000","last_undegraded":"2024-04-17T10:38:13.803828+0000","last_fullsized":"2024-04-17T10:38:13.803828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:27.569244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701871+0000","last_change":"2024-04-17T10:38:12.708377+0000","last_active":"2024-04-17T10:38:13.701871+0000","last_peered":"2024-04-17T10:38:13.701871+0000","last_clean":"2024-04-17T10:38:13.701871+0000","last_became_active":"2024-04-17T10:38:12.707183+0000","last_became_peered":"2024-04-17T10:38:12.707183+0000","last_unstale":"2024-04-17T10:38:13.701871+0000","last_undegraded":"2024-04-17T10:38:13.701871+0000","last_fullsized":"2024-04-17T10:38:13.701871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:06.508830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305423+0000","last_change":"2024-04-17T10:38:11.702326+0000","last_active":"2024-04-17T10:38:14.305423+0000","last_peered":"2024-04-17T10:38:14.305423+0000","last_clean":"2024-04-17T10:38:14.305423+0000","last_became_active":"2024-04-17T10:38:11.701735+0000","last_became_peered":"2024-04-17T10:38:11.701735+0000","last_unstale":"2024-04-17T10:38:14.305423+0000","last_undegraded":"2024-04-17T10:38:14.305423+0000","last_fullsized":"2024-04-17T10:38:14.305423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:11:43.267871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.304822+0000","last_change":"2024-04-17T10:38:12.705702+0000","last_active":"2024-04-17T10:38:14.304822+0000","last_peered":"2024-04-17T10:38:14.304822+0000","last_clean":"2024-04-17T10:38:14.304822+0000","last_became_active":"2024-04-17T10:38:12.705297+0000","last_became_peered":"2024-04-17T10:38:12.705297+0000","last_unstale":"2024-04-17T10:38:14.304822+0000","last_undegraded":"2024-04-17T10:38:14.304822+0000","last_fullsized":"2024-04-17T10:38:14.304822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:15:31.987120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970479+0000","last_change":"2024-04-17T10:38:11.715689+0000","last_active":"2024-04-17T10:38:13.970479+0000","last_peered":"2024-04-17T10:38:13.970479+0000","last_clean":"2024-04-17T10:38:13.970479+0000","last_became_active":"2024-04-17T10:38:11.714329+0000","last_became_peered":"2024-04-17T10:38:11.714329+0000","last_unstale":"2024-04-17T10:38:13.970479+0000","last_undegraded":"2024-04-17T10:38:13.970479+0000","last_fullsized":"2024-04-17T10:38:13.970479+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:27.569244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702011+0000","last_change":"2024-04-17T10:38:12.708536+0000","last_active":"2024-04-17T10:38:13.702011+0000","last_peered":"2024-04-17T10:38:13.702011+0000","last_clean":"2024-04-17T10:38:13.702011+0000","last_became_active":"2024-04-17T10:38:12.707643+0000","last_became_peered":"2024-04-17T10:38:12.707643+0000","last_unstale":"2024-04-17T10:38:13.702011+0000","last_undegraded":"2024-04-17T10:38:13.702011+0000","last_fullsized":"2024-04-17T10:38:13.702011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:22:10.364740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803484+0000","last_change":"2024-04-17T10:38:11.717992+0000","last_active":"2024-04-17T10:38:13.803484+0000","last_peered":"2024-04-17T10:38:13.803484+0000","last_clean":"2024-04-17T10:38:13.803484+0000","last_became_active":"2024-04-17T10:38:11.717124+0000","last_became_peered":"2024-04-17T10:38:11.717124+0000","last_unstale":"2024-04-17T10:38:13.803484+0000","last_undegraded":"2024-04-17T10:38:13.803484+0000","last_fullsized":"2024-04-17T10:38:13.803484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:24:23.818791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803260+0000","last_change":"2024-04-17T10:38:12.720282+0000","last_active":"2024-04-17T10:38:13.803260+0000","last_peered":"2024-04-17T10:38:13.803260+0000","last_clean":"2024-04-17T10:38:13.803260+0000","last_became_active":"2024-04-17T10:38:12.719197+0000","last_became_peered":"2024-04-17T10:38:12.719197+0000","last_unstale":"2024-04-17T10:38:13.803260+0000","last_undegraded":"2024-04-17T10:38:13.803260+0000","last_fullsized":"2024-04-17T10:38:13.803260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:08.872431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.303153+0000","last_change":"2024-04-17T10:38:11.689943+0000","last_active":"2024-04-17T10:38:14.303153+0000","last_peered":"2024-04-17T10:38:14.303153+0000","last_clean":"2024-04-17T10:38:14.303153+0000","last_became_active":"2024-04-17T10:38:11.689627+0000","last_became_peered":"2024-04-17T10:38:11.689627+0000","last_unstale":"2024-04-17T10:38:14.303153+0000","last_undegraded":"2024-04-17T10:38:14.303153+0000","last_fullsized":"2024-04-17T10:38:14.303153+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:39.210602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701741+0000","last_change":"2024-04-17T10:38:12.710693+0000","last_active":"2024-04-17T10:38:13.701741+0000","last_peered":"2024-04-17T10:38:13.701741+0000","last_clean":"2024-04-17T10:38:13.701741+0000","last_became_active":"2024-04-17T10:38:12.710263+0000","last_became_peered":"2024-04-17T10:38:12.710263+0000","last_unstale":"2024-04-17T10:38:13.701741+0000","last_undegraded":"2024-04-17T10:38:13.701741+0000","last_fullsized":"2024-04-17T10:38:13.701741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:09:07.090758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.700517+0000","last_change":"2024-04-17T10:38:11.701185+0000","last_active":"2024-04-17T10:38:13.700517+0000","last_peered":"2024-04-17T10:38:13.700517+0000","last_clean":"2024-04-17T10:38:13.700517+0000","last_became_active":"2024-04-17T10:38:11.700292+0000","last_became_peered":"2024-04-17T10:38:11.700292+0000","last_unstale":"2024-04-17T10:38:13.700517+0000","last_undegraded":"2024-04-17T10:38:13.700517+0000","last_fullsized":"2024-04-17T10:38:13.700517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:59:51.693435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702731+0000","last_change":"2024-04-17T10:38:12.704267+0000","last_active":"2024-04-17T10:38:13.702731+0000","last_peered":"2024-04-17T10:38:13.702731+0000","last_clean":"2024-04-17T10:38:13.702731+0000","last_became_active":"2024-04-17T10:38:12.703820+0000","last_became_peered":"2024-04-17T10:38:12.703820+0000","last_unstale":"2024-04-17T10:38:13.702731+0000","last_undegraded":"2024-04-17T10:38:13.702731+0000","last_fullsized":"2024-04-17T10:38:13.702731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:43:51.398103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.303738+0000","last_change":"2024-04-17T10:38:11.699100+0000","last_active":"2024-04-17T10:38:14.303738+0000","last_peered":"2024-04-17T10:38:14.303738+0000","last_clean":"2024-04-17T10:38:14.303738+0000","last_became_active":"2024-04-17T10:38:11.698786+0000","last_became_peered":"2024-04-17T10:38:11.698786+0000","last_unstale":"2024-04-17T10:38:14.303738+0000","last_undegraded":"2024-04-17T10:38:14.303738+0000","last_fullsized":"2024-04-17T10:38:14.303738+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:21:00.724523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702808+0000","last_change":"2024-04-17T10:38:12.705166+0000","last_active":"2024-04-17T10:38:13.702808+0000","last_peered":"2024-04-17T10:38:13.702808+0000","last_clean":"2024-04-17T10:38:13.702808+0000","last_became_active":"2024-04-17T10:38:12.704805+0000","last_became_peered":"2024-04-17T10:38:12.704805+0000","last_unstale":"2024-04-17T10:38:13.702808+0000","last_undegraded":"2024-04-17T10:38:13.702808+0000","last_fullsized":"2024-04-17T10:38:13.702808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:49:18.397560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.696028+0000","last_change":"2024-04-17T10:38:11.692209+0000","last_active":"2024-04-17T10:38:13.696028+0000","last_peered":"2024-04-17T10:38:13.696028+0000","last_clean":"2024-04-17T10:38:13.696028+0000","last_became_active":"2024-04-17T10:38:11.691905+0000","last_became_peered":"2024-04-17T10:38:11.691905+0000","last_unstale":"2024-04-17T10:38:13.696028+0000","last_undegraded":"2024-04-17T10:38:13.696028+0000","last_fullsized":"2024-04-17T10:38:13.696028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:50:59.373356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970996+0000","last_change":"2024-04-17T10:38:12.727145+0000","last_active":"2024-04-17T10:38:13.970996+0000","last_peered":"2024-04-17T10:38:13.970996+0000","last_clean":"2024-04-17T10:38:13.970996+0000","last_became_active":"2024-04-17T10:38:12.726797+0000","last_became_peered":"2024-04-17T10:38:12.726797+0000","last_unstale":"2024-04-17T10:38:13.970996+0000","last_undegraded":"2024-04-17T10:38:13.970996+0000","last_fullsized":"2024-04-17T10:38:13.970996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:43:51.398103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.303519+0000","last_change":"2024-04-17T10:38:11.705218+0000","last_active":"2024-04-17T10:38:14.303519+0000","last_peered":"2024-04-17T10:38:14.303519+0000","last_clean":"2024-04-17T10:38:14.303519+0000","last_became_active":"2024-04-17T10:38:11.704928+0000","last_became_peered":"2024-04-17T10:38:11.704928+0000","last_unstale":"2024-04-17T10:38:14.303519+0000","last_undegraded":"2024-04-17T10:38:14.303519+0000","last_fullsized":"2024-04-17T10:38:14.303519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:27.569244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701937+0000","last_change":"2024-04-17T10:38:12.722659+0000","last_active":"2024-04-17T10:38:13.701937+0000","last_peered":"2024-04-17T10:38:13.701937+0000","last_clean":"2024-04-17T10:38:13.701937+0000","last_became_active":"2024-04-17T10:38:12.722265+0000","last_became_peered":"2024-04-17T10:38:12.722265+0000","last_unstale":"2024-04-17T10:38:13.701937+0000","last_undegraded":"2024-04-17T10:38:13.701937+0000","last_fullsized":"2024-04-17T10:38:13.701937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:33:29.583300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701282+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:38:13.701282+0000","last_peered":"2024-04-17T10:38:13.701282+0000","last_clean":"2024-04-17T10:38:13.701282+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:38:13.701282+0000","last_undegraded":"2024-04-17T10:38:13.701282+0000","last_fullsized":"2024-04-17T10:38:13.701282+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970082+0000","last_change":"2024-04-17T10:38:11.702024+0000","last_active":"2024-04-17T10:38:13.970082+0000","last_peered":"2024-04-17T10:38:13.970082+0000","last_clean":"2024-04-17T10:38:13.970082+0000","last_became_active":"2024-04-17T10:38:11.701157+0000","last_became_peered":"2024-04-17T10:38:11.701157+0000","last_unstale":"2024-04-17T10:38:13.970082+0000","last_undegraded":"2024-04-17T10:38:13.970082+0000","last_fullsized":"2024-04-17T10:38:13.970082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:24:23.818791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297692+0000","last_change":"2024-04-17T10:38:12.723529+0000","last_active":"2024-04-17T10:38:14.297692+0000","last_peered":"2024-04-17T10:38:14.297692+0000","last_clean":"2024-04-17T10:38:14.297692+0000","last_became_active":"2024-04-17T10:38:12.723103+0000","last_became_peered":"2024-04-17T10:38:12.723103+0000","last_unstale":"2024-04-17T10:38:14.297692+0000","last_undegraded":"2024-04-17T10:38:14.297692+0000","last_fullsized":"2024-04-17T10:38:14.297692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:42.256740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701197+0000","last_change":"2024-04-17T10:38:11.716453+0000","last_active":"2024-04-17T10:38:13.701197+0000","last_peered":"2024-04-17T10:38:13.701197+0000","last_clean":"2024-04-17T10:38:13.701197+0000","last_became_active":"2024-04-17T10:38:11.715859+0000","last_became_peered":"2024-04-17T10:38:11.715859+0000","last_unstale":"2024-04-17T10:38:13.701197+0000","last_undegraded":"2024-04-17T10:38:13.701197+0000","last_fullsized":"2024-04-17T10:38:13.701197+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:50:59.373356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.304804+0000","last_change":"2024-04-17T10:38:12.707314+0000","last_active":"2024-04-17T10:38:14.304804+0000","last_peered":"2024-04-17T10:38:14.304804+0000","last_clean":"2024-04-17T10:38:14.304804+0000","last_became_active":"2024-04-17T10:38:12.706874+0000","last_became_peered":"2024-04-17T10:38:12.706874+0000","last_unstale":"2024-04-17T10:38:14.304804+0000","last_undegraded":"2024-04-17T10:38:14.304804+0000","last_fullsized":"2024-04-17T10:38:14.304804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:07:07.037555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.303300+0000","last_change":"2024-04-17T10:38:11.699752+0000","last_active":"2024-04-17T10:38:14.303300+0000","last_peered":"2024-04-17T10:38:14.303300+0000","last_clean":"2024-04-17T10:38:14.303300+0000","last_became_active":"2024-04-17T10:38:11.699467+0000","last_became_peered":"2024-04-17T10:38:11.699467+0000","last_unstale":"2024-04-17T10:38:14.303300+0000","last_undegraded":"2024-04-17T10:38:14.303300+0000","last_fullsized":"2024-04-17T10:38:14.303300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:24:23.818791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695234+0000","last_change":"2024-04-17T10:38:12.723809+0000","last_active":"2024-04-17T10:38:13.695234+0000","last_peered":"2024-04-17T10:38:13.695234+0000","last_clean":"2024-04-17T10:38:13.695234+0000","last_became_active":"2024-04-17T10:38:12.723282+0000","last_became_peered":"2024-04-17T10:38:12.723282+0000","last_unstale":"2024-04-17T10:38:13.695234+0000","last_undegraded":"2024-04-17T10:38:13.695234+0000","last_fullsized":"2024-04-17T10:38:13.695234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:31:48.663733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.971078+0000","last_change":"2024-04-17T10:38:11.715164+0000","last_active":"2024-04-17T10:38:13.971078+0000","last_peered":"2024-04-17T10:38:13.971078+0000","last_clean":"2024-04-17T10:38:13.971078+0000","last_became_active":"2024-04-17T10:38:11.712376+0000","last_became_peered":"2024-04-17T10:38:11.712376+0000","last_unstale":"2024-04-17T10:38:13.971078+0000","last_undegraded":"2024-04-17T10:38:13.971078+0000","last_fullsized":"2024-04-17T10:38:13.971078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:21:00.724523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.699765+0000","last_change":"2024-04-17T10:38:12.720339+0000","last_active":"2024-04-17T10:38:13.699765+0000","last_peered":"2024-04-17T10:38:13.699765+0000","last_clean":"2024-04-17T10:38:13.699765+0000","last_became_active":"2024-04-17T10:38:12.720051+0000","last_became_peered":"2024-04-17T10:38:12.720051+0000","last_unstale":"2024-04-17T10:38:13.699765+0000","last_undegraded":"2024-04-17T10:38:13.699765+0000","last_fullsized":"2024-04-17T10:38:13.699765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:51:00.381846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695713+0000","last_change":"2024-04-17T10:38:11.693095+0000","last_active":"2024-04-17T10:38:13.695713+0000","last_peered":"2024-04-17T10:38:13.695713+0000","last_clean":"2024-04-17T10:38:13.695713+0000","last_became_active":"2024-04-17T10:38:11.692723+0000","last_became_peered":"2024-04-17T10:38:11.692723+0000","last_unstale":"2024-04-17T10:38:13.695713+0000","last_undegraded":"2024-04-17T10:38:13.695713+0000","last_fullsized":"2024-04-17T10:38:13.695713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:27.569244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970920+0000","last_change":"2024-04-17T10:38:12.728559+0000","last_active":"2024-04-17T10:38:13.970920+0000","last_peered":"2024-04-17T10:38:13.970920+0000","last_clean":"2024-04-17T10:38:13.970920+0000","last_became_active":"2024-04-17T10:38:12.728285+0000","last_became_peered":"2024-04-17T10:38:12.728285+0000","last_unstale":"2024-04-17T10:38:13.970920+0000","last_undegraded":"2024-04-17T10:38:13.970920+0000","last_fullsized":"2024-04-17T10:38:13.970920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:22:10.364740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702148+0000","last_change":"2024-04-17T10:38:11.717791+0000","last_active":"2024-04-17T10:38:13.702148+0000","last_peered":"2024-04-17T10:38:13.702148+0000","last_clean":"2024-04-17T10:38:13.702148+0000","last_became_active":"2024-04-17T10:38:11.717130+0000","last_became_peered":"2024-04-17T10:38:11.717130+0000","last_unstale":"2024-04-17T10:38:13.702148+0000","last_undegraded":"2024-04-17T10:38:13.702148+0000","last_fullsized":"2024-04-17T10:38:13.702148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:21:10.063327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695581+0000","last_change":"2024-04-17T10:38:12.723436+0000","last_active":"2024-04-17T10:38:13.695581+0000","last_peered":"2024-04-17T10:38:13.695581+0000","last_clean":"2024-04-17T10:38:13.695581+0000","last_became_active":"2024-04-17T10:38:12.723138+0000","last_became_peered":"2024-04-17T10:38:12.723138+0000","last_unstale":"2024-04-17T10:38:13.695581+0000","last_undegraded":"2024-04-17T10:38:13.695581+0000","last_fullsized":"2024-04-17T10:38:13.695581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:39.640730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.696103+0000","last_change":"2024-04-17T10:38:11.730180+0000","last_active":"2024-04-17T10:38:13.696103+0000","last_peered":"2024-04-17T10:38:13.696103+0000","last_clean":"2024-04-17T10:38:13.696103+0000","last_became_active":"2024-04-17T10:38:11.729843+0000","last_became_peered":"2024-04-17T10:38:11.729843+0000","last_unstale":"2024-04-17T10:38:13.696103+0000","last_undegraded":"2024-04-17T10:38:13.696103+0000","last_fullsized":"2024-04-17T10:38:13.696103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:23:29.956041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298370+0000","last_change":"2024-04-17T10:38:12.732643+0000","last_active":"2024-04-17T10:38:14.298370+0000","last_peered":"2024-04-17T10:38:14.298370+0000","last_clean":"2024-04-17T10:38:14.298370+0000","last_became_active":"2024-04-17T10:38:12.732259+0000","last_became_peered":"2024-04-17T10:38:12.732259+0000","last_unstale":"2024-04-17T10:38:14.298370+0000","last_undegraded":"2024-04-17T10:38:14.298370+0000","last_fullsized":"2024-04-17T10:38:14.298370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:02.184227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803741+0000","last_change":"2024-04-17T10:38:11.725444+0000","last_active":"2024-04-17T10:38:13.803741+0000","last_peered":"2024-04-17T10:38:13.803741+0000","last_clean":"2024-04-17T10:38:13.803741+0000","last_became_active":"2024-04-17T10:38:11.724780+0000","last_became_peered":"2024-04-17T10:38:11.724780+0000","last_unstale":"2024-04-17T10:38:13.803741+0000","last_undegraded":"2024-04-17T10:38:13.803741+0000","last_fullsized":"2024-04-17T10:38:13.803741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:45.366572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305401+0000","last_change":"2024-04-17T10:38:12.705770+0000","last_active":"2024-04-17T10:38:14.305401+0000","last_peered":"2024-04-17T10:38:14.305401+0000","last_clean":"2024-04-17T10:38:14.305401+0000","last_became_active":"2024-04-17T10:38:12.705400+0000","last_became_peered":"2024-04-17T10:38:12.705400+0000","last_unstale":"2024-04-17T10:38:14.305401+0000","last_undegraded":"2024-04-17T10:38:14.305401+0000","last_fullsized":"2024-04-17T10:38:14.305401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:42.256740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701197+0000","last_change":"2024-04-17T10:38:11.716433+0000","last_active":"2024-04-17T10:38:13.701197+0000","last_peered":"2024-04-17T10:38:13.701197+0000","last_clean":"2024-04-17T10:38:13.701197+0000","last_became_active":"2024-04-17T10:38:11.715803+0000","last_became_peered":"2024-04-17T10:38:11.715803+0000","last_unstale":"2024-04-17T10:38:13.701197+0000","last_undegraded":"2024-04-17T10:38:13.701197+0000","last_fullsized":"2024-04-17T10:38:13.701197+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:45.366572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.700697+0000","last_change":"2024-04-17T10:38:12.717613+0000","last_active":"2024-04-17T10:38:13.700697+0000","last_peered":"2024-04-17T10:38:13.700697+0000","last_clean":"2024-04-17T10:38:13.700697+0000","last_became_active":"2024-04-17T10:38:12.717308+0000","last_became_peered":"2024-04-17T10:38:12.717308+0000","last_unstale":"2024-04-17T10:38:13.700697+0000","last_undegraded":"2024-04-17T10:38:13.700697+0000","last_fullsized":"2024-04-17T10:38:13.700697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:21:11.071817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695770+0000","last_change":"2024-04-17T10:38:11.716778+0000","last_active":"2024-04-17T10:38:13.695770+0000","last_peered":"2024-04-17T10:38:13.695770+0000","last_clean":"2024-04-17T10:38:13.695770+0000","last_became_active":"2024-04-17T10:38:11.716526+0000","last_became_peered":"2024-04-17T10:38:11.716526+0000","last_unstale":"2024-04-17T10:38:13.695770+0000","last_undegraded":"2024-04-17T10:38:13.695770+0000","last_fullsized":"2024-04-17T10:38:13.695770+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:24:23.818791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297622+0000","last_change":"2024-04-17T10:38:12.717941+0000","last_active":"2024-04-17T10:38:14.297622+0000","last_peered":"2024-04-17T10:38:14.297622+0000","last_clean":"2024-04-17T10:38:14.297622+0000","last_became_active":"2024-04-17T10:38:12.717350+0000","last_became_peered":"2024-04-17T10:38:12.717350+0000","last_unstale":"2024-04-17T10:38:14.297622+0000","last_undegraded":"2024-04-17T10:38:14.297622+0000","last_fullsized":"2024-04-17T10:38:14.297622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:39.640730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695661+0000","last_change":"2024-04-17T10:38:11.699325+0000","last_active":"2024-04-17T10:38:13.695661+0000","last_peered":"2024-04-17T10:38:13.695661+0000","last_clean":"2024-04-17T10:38:13.695661+0000","last_became_active":"2024-04-17T10:38:11.698981+0000","last_became_peered":"2024-04-17T10:38:11.698981+0000","last_unstale":"2024-04-17T10:38:13.695661+0000","last_undegraded":"2024-04-17T10:38:13.695661+0000","last_fullsized":"2024-04-17T10:38:13.695661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:21:00.724523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298006+0000","last_change":"2024-04-17T10:38:12.731391+0000","last_active":"2024-04-17T10:38:14.298006+0000","last_peered":"2024-04-17T10:38:14.298006+0000","last_clean":"2024-04-17T10:38:14.298006+0000","last_became_active":"2024-04-17T10:38:12.727638+0000","last_became_peered":"2024-04-17T10:38:12.727638+0000","last_unstale":"2024-04-17T10:38:14.298006+0000","last_undegraded":"2024-04-17T10:38:14.298006+0000","last_fullsized":"2024-04-17T10:38:14.298006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:23.465112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701909+0000","last_change":"2024-04-17T10:38:11.716353+0000","last_active":"2024-04-17T10:38:13.701909+0000","last_peered":"2024-04-17T10:38:13.701909+0000","last_clean":"2024-04-17T10:38:13.701909+0000","last_became_active":"2024-04-17T10:38:11.715722+0000","last_became_peered":"2024-04-17T10:38:11.715722+0000","last_unstale":"2024-04-17T10:38:13.701909+0000","last_undegraded":"2024-04-17T10:38:13.701909+0000","last_fullsized":"2024-04-17T10:38:13.701909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:20:44.801917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803339+0000","last_change":"2024-04-17T10:38:12.715919+0000","last_active":"2024-04-17T10:38:13.803339+0000","last_peered":"2024-04-17T10:38:13.803339+0000","last_clean":"2024-04-17T10:38:13.803339+0000","last_became_active":"2024-04-17T10:38:12.715568+0000","last_became_peered":"2024-04-17T10:38:12.715568+0000","last_unstale":"2024-04-17T10:38:13.803339+0000","last_undegraded":"2024-04-17T10:38:13.803339+0000","last_fullsized":"2024-04-17T10:38:13.803339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:26:02.126931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970940+0000","last_change":"2024-04-17T10:38:12.724977+0000","last_active":"2024-04-17T10:38:13.970940+0000","last_peered":"2024-04-17T10:38:13.970940+0000","last_clean":"2024-04-17T10:38:13.970940+0000","last_became_active":"2024-04-17T10:38:12.724700+0000","last_became_peered":"2024-04-17T10:38:12.724700+0000","last_unstale":"2024-04-17T10:38:13.970940+0000","last_undegraded":"2024-04-17T10:38:13.970940+0000","last_fullsized":"2024-04-17T10:38:13.970940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:38:39.116589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702197+0000","last_change":"2024-04-17T10:38:11.719799+0000","last_active":"2024-04-17T10:38:13.702197+0000","last_peered":"2024-04-17T10:38:13.702197+0000","last_clean":"2024-04-17T10:38:13.702197+0000","last_became_active":"2024-04-17T10:38:11.719147+0000","last_became_peered":"2024-04-17T10:38:11.719147+0000","last_unstale":"2024-04-17T10:38:13.702197+0000","last_undegraded":"2024-04-17T10:38:13.702197+0000","last_fullsized":"2024-04-17T10:38:13.702197+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:23:29.956041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.971331+0000","last_change":"2024-04-17T10:38:12.725123+0000","last_active":"2024-04-17T10:38:13.971331+0000","last_peered":"2024-04-17T10:38:13.971331+0000","last_clean":"2024-04-17T10:38:13.971331+0000","last_became_active":"2024-04-17T10:38:12.724760+0000","last_became_peered":"2024-04-17T10:38:12.724760+0000","last_unstale":"2024-04-17T10:38:13.971331+0000","last_undegraded":"2024-04-17T10:38:13.971331+0000","last_fullsized":"2024-04-17T10:38:13.971331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:49:18.397560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702167+0000","last_change":"2024-04-17T10:38:11.727050+0000","last_active":"2024-04-17T10:38:13.702167+0000","last_peered":"2024-04-17T10:38:13.702167+0000","last_clean":"2024-04-17T10:38:13.702167+0000","last_became_active":"2024-04-17T10:38:11.726393+0000","last_became_peered":"2024-04-17T10:38:11.726393+0000","last_unstale":"2024-04-17T10:38:13.702167+0000","last_undegraded":"2024-04-17T10:38:13.702167+0000","last_fullsized":"2024-04-17T10:38:13.702167+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:23:25.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305077+0000","last_change":"2024-04-17T10:38:12.726864+0000","last_active":"2024-04-17T10:38:14.305077+0000","last_peered":"2024-04-17T10:38:14.305077+0000","last_clean":"2024-04-17T10:38:14.305077+0000","last_became_active":"2024-04-17T10:38:12.726581+0000","last_became_peered":"2024-04-17T10:38:12.726581+0000","last_unstale":"2024-04-17T10:38:14.305077+0000","last_undegraded":"2024-04-17T10:38:14.305077+0000","last_fullsized":"2024-04-17T10:38:14.305077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:31:48.663733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298342+0000","last_change":"2024-04-17T10:38:11.700825+0000","last_active":"2024-04-17T10:38:14.298342+0000","last_peered":"2024-04-17T10:38:14.298342+0000","last_clean":"2024-04-17T10:38:14.298342+0000","last_became_active":"2024-04-17T10:38:11.700354+0000","last_became_peered":"2024-04-17T10:38:11.700354+0000","last_unstale":"2024-04-17T10:38:14.298342+0000","last_undegraded":"2024-04-17T10:38:14.298342+0000","last_fullsized":"2024-04-17T10:38:14.298342+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:22:08.746889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298080+0000","last_change":"2024-04-17T10:38:12.719771+0000","last_active":"2024-04-17T10:38:14.298080+0000","last_peered":"2024-04-17T10:38:14.298080+0000","last_clean":"2024-04-17T10:38:14.298080+0000","last_became_active":"2024-04-17T10:38:12.716013+0000","last_became_peered":"2024-04-17T10:38:12.716013+0000","last_unstale":"2024-04-17T10:38:14.298080+0000","last_undegraded":"2024-04-17T10:38:14.298080+0000","last_fullsized":"2024-04-17T10:38:14.298080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:54:26.054113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305389+0000","last_change":"2024-04-17T10:38:11.710717+0000","last_active":"2024-04-17T10:38:14.305389+0000","last_peered":"2024-04-17T10:38:14.305389+0000","last_clean":"2024-04-17T10:38:14.305389+0000","last_became_active":"2024-04-17T10:38:11.710342+0000","last_became_peered":"2024-04-17T10:38:11.710342+0000","last_unstale":"2024-04-17T10:38:14.305389+0000","last_undegraded":"2024-04-17T10:38:14.305389+0000","last_fullsized":"2024-04-17T10:38:14.305389+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:16:30.923882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":196608,"data_stored":64724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":798720,"data_stored":655640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:38:20.140 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:38:20.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.276+0000 7f453b131640 1 Processor -- start 2024-04-17T10:38:20.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.276+0000 7f453b131640 1 -- start start 2024-04-17T10:38:20.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.276+0000 7f453b131640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4534072330 0x7f45340cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:20.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.276+0000 7f453b131640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f45340cfa10 0x7f453406c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:20.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.276+0000 7f453b131640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45340cfea0 con 0x7f453415ba30 2024-04-17T10:38:20.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.276+0000 7f453b131640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4534071ea0 con 0x7f4534072330 2024-04-17T10:38:20.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.276+0000 7f453b131640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f453406e320 con 0x7f45340cfa10 2024-04-17T10:38:20.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.276+0000 7f4533fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4534072330 0x7f45340cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:20.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.276+0000 7f4533fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4534072330 0x7f45340cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47890/0 (socket says 172.21.15.83:47890) 2024-04-17T10:38:20.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.276+0000 7f4533fff640 1 -- 172.21.15.83:0/3839267692 learned_addr learned my addr 172.21.15.83:0/3839267692 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:20.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.276+0000 7f45396ce640 1 --2- 172.21.15.83:0/3839267692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f45340cfa10 0x7f453406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:20.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.276+0000 7f4533fff640 1 -- 172.21.15.83:0/3839267692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f45340cfa10 msgr2=0x7f453406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:20.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.276+0000 7f4533fff640 1 --2- 172.21.15.83:0/3839267692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f45340cfa10 0x7f453406c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:20.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.276+0000 7f4533fff640 1 -- 172.21.15.83:0/3839267692 >> v1:172.21.15.167:6789/0 conn(0x7f453415ba30 legacy=0x7f453415c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:20.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.276+0000 7f4533fff640 1 -- 172.21.15.83:0/3839267692 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453406e040 con 0x7f4534072330 2024-04-17T10:38:20.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.280+0000 7f4533fff640 1 --2- 172.21.15.83:0/3839267692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4534072330 0x7f45340cf4d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4534072020 tx=0x7f452001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b536388720bf96a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:20.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.280+0000 7f45337fe640 1 -- 172.21.15.83:0/3839267692 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45200b0070 con 0x7f4534072330 2024-04-17T10:38:20.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.280+0000 7f45337fe640 1 -- 172.21.15.83:0/3839267692 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45200ab600 con 0x7f4534072330 2024-04-17T10:38:20.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.280+0000 7f45337fe640 1 -- 172.21.15.83:0/3839267692 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45200ab9d0 con 0x7f4534072330 2024-04-17T10:38:20.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.280+0000 7f453b131640 1 -- 172.21.15.83:0/3839267692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4534072330 msgr2=0x7f45340cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:20.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.280+0000 7f453b131640 1 --2- 172.21.15.83:0/3839267692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4534072330 0x7f45340cf4d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4534072020 tx=0x7f452001c470 comp rx=0 tx=0).stop 2024-04-17T10:38:20.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.280+0000 7f453b131640 1 -- 172.21.15.83:0/3839267692 shutdown_connections 2024-04-17T10:38:20.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.280+0000 7f453b131640 1 --2- 172.21.15.83:0/3839267692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f45340cfa10 0x7f453406c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:20.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.280+0000 7f453b131640 1 --2- 172.21.15.83:0/3839267692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4534072330 0x7f45340cf4d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:20.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.280+0000 7f453b131640 1 -- 172.21.15.83:0/3839267692 >> 172.21.15.83:0/3839267692 conn(0x7f453406abe0 msgr2=0x7f453406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:20.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.280+0000 7f453b131640 1 -- 172.21.15.83:0/3839267692 shutdown_connections 2024-04-17T10:38:20.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.280+0000 7f453b131640 1 -- 172.21.15.83:0/3839267692 wait complete. 2024-04-17T10:38:20.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.280+0000 7f453b131640 1 Processor -- start 2024-04-17T10:38:20.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.280+0000 7f453b131640 1 -- start start 2024-04-17T10:38:20.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.280+0000 7f453b131640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4534072330 0x7f4534149c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:20.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.280+0000 7f453b131640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45340cfa10 0x7f453415d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:20.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.280+0000 7f453b131640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f453415ba30 0x7f45340ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:20.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.280+0000 7f453b131640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453406de20 con 0x7f4534072330 2024-04-17T10:38:20.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.280+0000 7f453b131640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453406dca0 con 0x7f45340cfa10 2024-04-17T10:38:20.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.280+0000 7f453b131640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4534168520 con 0x7f453415ba30 2024-04-17T10:38:20.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.280+0000 7f4538ecd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4534072330 0x7f4534149c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:20.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.280+0000 7f4538ecd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4534072330 0x7f4534149c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47904/0 (socket says 172.21.15.83:47904) 2024-04-17T10:38:20.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.280+0000 7f4538ecd640 1 -- 172.21.15.83:0/2447756898 learned_addr learned my addr 172.21.15.83:0/2447756898 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:20.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.280+0000 7f4533fff640 1 --2- 172.21.15.83:0/2447756898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45340cfa10 0x7f453415d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:20.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.280+0000 7f45396ce640 1 --2- 172.21.15.83:0/2447756898 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f453415ba30 0x7f45340ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:20.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.284+0000 7f4538ecd640 1 -- 172.21.15.83:0/2447756898 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f453415ba30 msgr2=0x7f45340ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:20.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.284+0000 7f4538ecd640 1 --2- 172.21.15.83:0/2447756898 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f453415ba30 0x7f45340ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:20.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.284+0000 7f4538ecd640 1 -- 172.21.15.83:0/2447756898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45340cfa10 msgr2=0x7f453415d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:20.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.284+0000 7f4538ecd640 1 --2- 172.21.15.83:0/2447756898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45340cfa10 0x7f453415d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:20.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.284+0000 7f4538ecd640 1 -- 172.21.15.83:0/2447756898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45340b3cc0 con 0x7f4534072330 2024-04-17T10:38:20.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.284+0000 7f4533fff640 1 --2- 172.21.15.83:0/2447756898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45340cfa10 0x7f453415d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:20.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.284+0000 7f4538ecd640 1 --2- 172.21.15.83:0/2447756898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4534072330 0x7f4534149c50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f452800c450 tx=0x7f452800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:20.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.284+0000 7f45317fa640 1 -- 172.21.15.83:0/2447756898 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4528012830 con 0x7f4534072330 2024-04-17T10:38:20.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.284+0000 7f45317fa640 1 -- 172.21.15.83:0/2447756898 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4528017070 con 0x7f4534072330 2024-04-17T10:38:20.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.284+0000 7f453b131640 1 -- 172.21.15.83:0/2447756898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45340b3fa0 con 0x7f4534072330 2024-04-17T10:38:20.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.284+0000 7f45317fa640 1 -- 172.21.15.83:0/2447756898 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4528012c30 con 0x7f4534072330 2024-04-17T10:38:20.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.284+0000 7f453b131640 1 -- 172.21.15.83:0/2447756898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4534156c30 con 0x7f4534072330 2024-04-17T10:38:20.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.284+0000 7f453b131640 1 -- 172.21.15.83:0/2447756898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45341570a0 con 0x7f4534072330 2024-04-17T10:38:20.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.284+0000 7f45317fa640 1 -- 172.21.15.83:0/2447756898 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f452801c460 con 0x7f4534072330 2024-04-17T10:38:20.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.284+0000 7f45317fa640 1 --2- 172.21.15.83:0/2447756898 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4518066f60 0x7f4518069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:20.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.284+0000 7f45317fa640 1 -- 172.21.15.83:0/2447756898 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4528084e70 con 0x7f4534072330 2024-04-17T10:38:20.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.284+0000 7f4533fff640 1 --2- 172.21.15.83:0/2447756898 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4518066f60 0x7f4518069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:20.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.284+0000 7f4533fff640 1 --2- 172.21.15.83:0/2447756898 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4518066f60 0x7f4518069420 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f45340b2410 tx=0x7f45200a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:20.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.288+0000 7f45317fa640 1 -- 172.21.15.83:0/2447756898 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4528055e80 con 0x7f4534072330 2024-04-17T10:38:20.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.444+0000 7f453b131640 1 -- 172.21.15.83:0/2447756898 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4534074f90 con 0x7f4518066f60 2024-04-17T10:38:20.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.460+0000 7f45317fa640 1 -- 172.21.15.83:0/2447756898 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251860 (secure 0 0 0) 0x7f4534074f90 con 0x7f4518066f60 2024-04-17T10:38:20.459 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:20.460 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:38:20.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.464+0000 7f453b131640 1 -- 172.21.15.83:0/2447756898 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4518066f60 msgr2=0x7f4518069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:20.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.464+0000 7f453b131640 1 --2- 172.21.15.83:0/2447756898 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4518066f60 0x7f4518069420 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f45340b2410 tx=0x7f45200a6000 comp rx=0 tx=0).stop 2024-04-17T10:38:20.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.464+0000 7f453b131640 1 -- 172.21.15.83:0/2447756898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4534072330 msgr2=0x7f4534149c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:20.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.464+0000 7f453b131640 1 --2- 172.21.15.83:0/2447756898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4534072330 0x7f4534149c50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f452800c450 tx=0x7f452800c910 comp rx=0 tx=0).stop 2024-04-17T10:38:20.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.464+0000 7f453b131640 1 -- 172.21.15.83:0/2447756898 shutdown_connections 2024-04-17T10:38:20.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.464+0000 7f453b131640 1 --2- 172.21.15.83:0/2447756898 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4518066f60 0x7f4518069420 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:20.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.464+0000 7f453b131640 1 --2- 172.21.15.83:0/2447756898 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f453415ba30 0x7f45340ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:20.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.464+0000 7f453b131640 1 --2- 172.21.15.83:0/2447756898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45340cfa10 0x7f453415d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:20.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.464+0000 7f453b131640 1 --2- 172.21.15.83:0/2447756898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4534072330 0x7f4534149c50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:20.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.464+0000 7f453b131640 1 -- 172.21.15.83:0/2447756898 >> 172.21.15.83:0/2447756898 conn(0x7f453406abe0 msgr2=0x7f453414ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:20.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.464+0000 7f453b131640 1 -- 172.21.15.83:0/2447756898 shutdown_connections 2024-04-17T10:38:20.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.464+0000 7f453b131640 1 -- 172.21.15.83:0/2447756898 wait complete. 2024-04-17T10:38:20.477 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-17T10:38:20.182054+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219004,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755716,"statfs":{"total":773094113280,"available":772869853184,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002298"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702753+0000","last_change":"2024-04-17T10:38:12.718400+0000","last_active":"2024-04-17T10:38:13.702753+0000","last_peered":"2024-04-17T10:38:13.702753+0000","last_clean":"2024-04-17T10:38:13.702753+0000","last_became_active":"2024-04-17T10:38:12.718121+0000","last_became_peered":"2024-04-17T10:38:12.718121+0000","last_unstale":"2024-04-17T10:38:13.702753+0000","last_undegraded":"2024-04-17T10:38:13.702753+0000","last_fullsized":"2024-04-17T10:38:13.702753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:37.203406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695895+0000","last_change":"2024-04-17T10:38:12.710479+0000","last_active":"2024-04-17T10:38:13.695895+0000","last_peered":"2024-04-17T10:38:13.695895+0000","last_clean":"2024-04-17T10:38:13.695895+0000","last_became_active":"2024-04-17T10:38:12.710092+0000","last_became_peered":"2024-04-17T10:38:12.710092+0000","last_unstale":"2024-04-17T10:38:13.695895+0000","last_undegraded":"2024-04-17T10:38:13.695895+0000","last_fullsized":"2024-04-17T10:38:13.695895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:43.917742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305973+0000","last_change":"2024-04-17T10:38:12.722110+0000","last_active":"2024-04-17T10:38:14.305973+0000","last_peered":"2024-04-17T10:38:14.305973+0000","last_clean":"2024-04-17T10:38:14.305973+0000","last_became_active":"2024-04-17T10:38:12.721720+0000","last_became_peered":"2024-04-17T10:38:12.721720+0000","last_unstale":"2024-04-17T10:38:14.305973+0000","last_undegraded":"2024-04-17T10:38:14.305973+0000","last_fullsized":"2024-04-17T10:38:14.305973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:43.917742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970398+0000","last_change":"2024-04-17T10:38:12.727777+0000","last_active":"2024-04-17T10:38:13.970398+0000","last_peered":"2024-04-17T10:38:13.970398+0000","last_clean":"2024-04-17T10:38:13.970398+0000","last_became_active":"2024-04-17T10:38:12.726952+0000","last_became_peered":"2024-04-17T10:38:12.726952+0000","last_unstale":"2024-04-17T10:38:13.970398+0000","last_undegraded":"2024-04-17T10:38:13.970398+0000","last_fullsized":"2024-04-17T10:38:13.970398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:06.508830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701762+0000","last_change":"2024-04-17T10:38:12.721627+0000","last_active":"2024-04-17T10:38:13.701762+0000","last_peered":"2024-04-17T10:38:13.701762+0000","last_clean":"2024-04-17T10:38:13.701762+0000","last_became_active":"2024-04-17T10:38:12.718951+0000","last_became_peered":"2024-04-17T10:38:12.718951+0000","last_unstale":"2024-04-17T10:38:13.701762+0000","last_undegraded":"2024-04-17T10:38:13.701762+0000","last_fullsized":"2024-04-17T10:38:13.701762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:02.184227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970163+0000","last_change":"2024-04-17T10:38:12.729603+0000","last_active":"2024-04-17T10:38:13.970163+0000","last_peered":"2024-04-17T10:38:13.970163+0000","last_clean":"2024-04-17T10:38:13.970163+0000","last_became_active":"2024-04-17T10:38:12.728738+0000","last_became_peered":"2024-04-17T10:38:12.728738+0000","last_unstale":"2024-04-17T10:38:13.970163+0000","last_undegraded":"2024-04-17T10:38:13.970163+0000","last_fullsized":"2024-04-17T10:38:13.970163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:54:26.054113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701635+0000","last_change":"2024-04-17T10:38:12.708732+0000","last_active":"2024-04-17T10:38:13.701635+0000","last_peered":"2024-04-17T10:38:13.701635+0000","last_clean":"2024-04-17T10:38:13.701635+0000","last_became_active":"2024-04-17T10:38:12.708291+0000","last_became_peered":"2024-04-17T10:38:12.708291+0000","last_unstale":"2024-04-17T10:38:13.701635+0000","last_undegraded":"2024-04-17T10:38:13.701635+0000","last_fullsized":"2024-04-17T10:38:13.701635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:54:26.054113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297197+0000","last_change":"2024-04-17T10:38:12.726672+0000","last_active":"2024-04-17T10:38:14.297197+0000","last_peered":"2024-04-17T10:38:14.297197+0000","last_clean":"2024-04-17T10:38:14.297197+0000","last_became_active":"2024-04-17T10:38:12.726395+0000","last_became_peered":"2024-04-17T10:38:12.726395+0000","last_unstale":"2024-04-17T10:38:14.297197+0000","last_undegraded":"2024-04-17T10:38:14.297197+0000","last_fullsized":"2024-04-17T10:38:14.297197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:31:48.663733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702702+0000","last_change":"2024-04-17T10:38:12.727392+0000","last_active":"2024-04-17T10:38:13.702702+0000","last_peered":"2024-04-17T10:38:13.702702+0000","last_clean":"2024-04-17T10:38:13.702702+0000","last_became_active":"2024-04-17T10:38:12.727171+0000","last_became_peered":"2024-04-17T10:38:12.727171+0000","last_unstale":"2024-04-17T10:38:13.702702+0000","last_undegraded":"2024-04-17T10:38:13.702702+0000","last_fullsized":"2024-04-17T10:38:13.702702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:11:49.847070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803824+0000","last_change":"2024-04-17T10:38:12.726158+0000","last_active":"2024-04-17T10:38:13.803824+0000","last_peered":"2024-04-17T10:38:13.803824+0000","last_clean":"2024-04-17T10:38:13.803824+0000","last_became_active":"2024-04-17T10:38:12.724920+0000","last_became_peered":"2024-04-17T10:38:12.724920+0000","last_unstale":"2024-04-17T10:38:13.803824+0000","last_undegraded":"2024-04-17T10:38:13.803824+0000","last_fullsized":"2024-04-17T10:38:13.803824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:43.917742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803453+0000","last_change":"2024-04-17T10:38:12.725420+0000","last_active":"2024-04-17T10:38:13.803453+0000","last_peered":"2024-04-17T10:38:13.803453+0000","last_clean":"2024-04-17T10:38:13.803453+0000","last_became_active":"2024-04-17T10:38:12.724959+0000","last_became_peered":"2024-04-17T10:38:12.724959+0000","last_unstale":"2024-04-17T10:38:13.803453+0000","last_undegraded":"2024-04-17T10:38:13.803453+0000","last_fullsized":"2024-04-17T10:38:13.803453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:39.640730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.699955+0000","last_change":"2024-04-17T10:38:12.712887+0000","last_active":"2024-04-17T10:38:13.699955+0000","last_peered":"2024-04-17T10:38:13.699955+0000","last_clean":"2024-04-17T10:38:13.699955+0000","last_became_active":"2024-04-17T10:38:12.712582+0000","last_became_peered":"2024-04-17T10:38:12.712582+0000","last_unstale":"2024-04-17T10:38:13.699955+0000","last_undegraded":"2024-04-17T10:38:13.699955+0000","last_fullsized":"2024-04-17T10:38:13.699955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:23:30.964531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298210+0000","last_change":"2024-04-17T10:38:12.722191+0000","last_active":"2024-04-17T10:38:14.298210+0000","last_peered":"2024-04-17T10:38:14.298210+0000","last_clean":"2024-04-17T10:38:14.298210+0000","last_became_active":"2024-04-17T10:38:12.721912+0000","last_became_peered":"2024-04-17T10:38:12.721912+0000","last_unstale":"2024-04-17T10:38:14.298210+0000","last_undegraded":"2024-04-17T10:38:14.298210+0000","last_fullsized":"2024-04-17T10:38:14.298210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:09:07.090758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.304662+0000","last_change":"2024-04-17T10:38:12.710211+0000","last_active":"2024-04-17T10:38:14.304662+0000","last_peered":"2024-04-17T10:38:14.304662+0000","last_clean":"2024-04-17T10:38:14.304662+0000","last_became_active":"2024-04-17T10:38:12.709818+0000","last_became_peered":"2024-04-17T10:38:12.709818+0000","last_unstale":"2024-04-17T10:38:14.304662+0000","last_undegraded":"2024-04-17T10:38:14.304662+0000","last_fullsized":"2024-04-17T10:38:14.304662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:23:26.747986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695484+0000","last_change":"2024-04-17T10:38:12.722762+0000","last_active":"2024-04-17T10:38:13.695484+0000","last_peered":"2024-04-17T10:38:13.695484+0000","last_clean":"2024-04-17T10:38:13.695484+0000","last_became_active":"2024-04-17T10:38:12.718394+0000","last_became_peered":"2024-04-17T10:38:12.718394+0000","last_unstale":"2024-04-17T10:38:13.695484+0000","last_undegraded":"2024-04-17T10:38:13.695484+0000","last_fullsized":"2024-04-17T10:38:13.695484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:08.872431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.969994+0000","last_change":"2024-04-17T10:38:12.729270+0000","last_active":"2024-04-17T10:38:13.969994+0000","last_peered":"2024-04-17T10:38:13.969994+0000","last_clean":"2024-04-17T10:38:13.969994+0000","last_became_active":"2024-04-17T10:38:12.728798+0000","last_became_peered":"2024-04-17T10:38:12.728798+0000","last_unstale":"2024-04-17T10:38:13.969994+0000","last_undegraded":"2024-04-17T10:38:13.969994+0000","last_fullsized":"2024-04-17T10:38:13.969994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:23.465112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.971229+0000","last_change":"2024-04-17T10:38:12.731612+0000","last_active":"2024-04-17T10:38:13.971229+0000","last_peered":"2024-04-17T10:38:13.971229+0000","last_clean":"2024-04-17T10:38:13.971229+0000","last_became_active":"2024-04-17T10:38:12.731071+0000","last_became_peered":"2024-04-17T10:38:12.731071+0000","last_unstale":"2024-04-17T10:38:13.971229+0000","last_undegraded":"2024-04-17T10:38:13.971229+0000","last_fullsized":"2024-04-17T10:38:13.971229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:37.203406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.303651+0000","last_change":"2024-04-17T10:38:12.720913+0000","last_active":"2024-04-17T10:38:14.303651+0000","last_peered":"2024-04-17T10:38:14.303651+0000","last_clean":"2024-04-17T10:38:14.303651+0000","last_became_active":"2024-04-17T10:38:12.719779+0000","last_became_peered":"2024-04-17T10:38:12.719779+0000","last_unstale":"2024-04-17T10:38:14.303651+0000","last_undegraded":"2024-04-17T10:38:14.303651+0000","last_fullsized":"2024-04-17T10:38:14.303651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:23:26.747986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702068+0000","last_change":"2024-04-17T10:38:12.719229+0000","last_active":"2024-04-17T10:38:13.702068+0000","last_peered":"2024-04-17T10:38:13.702068+0000","last_clean":"2024-04-17T10:38:13.702068+0000","last_became_active":"2024-04-17T10:38:12.718934+0000","last_became_peered":"2024-04-17T10:38:12.718934+0000","last_unstale":"2024-04-17T10:38:13.702068+0000","last_undegraded":"2024-04-17T10:38:13.702068+0000","last_fullsized":"2024-04-17T10:38:13.702068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:38:39.116589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803564+0000","last_change":"2024-04-17T10:38:12.717999+0000","last_active":"2024-04-17T10:38:13.803564+0000","last_peered":"2024-04-17T10:38:13.803564+0000","last_clean":"2024-04-17T10:38:13.803564+0000","last_became_active":"2024-04-17T10:38:12.716817+0000","last_became_peered":"2024-04-17T10:38:12.716817+0000","last_unstale":"2024-04-17T10:38:13.803564+0000","last_undegraded":"2024-04-17T10:38:13.803564+0000","last_fullsized":"2024-04-17T10:38:13.803564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:42.256740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.304981+0000","last_change":"2024-04-17T10:38:12.710385+0000","last_active":"2024-04-17T10:38:14.304981+0000","last_peered":"2024-04-17T10:38:14.304981+0000","last_clean":"2024-04-17T10:38:14.304981+0000","last_became_active":"2024-04-17T10:38:12.707950+0000","last_became_peered":"2024-04-17T10:38:12.707950+0000","last_unstale":"2024-04-17T10:38:14.304981+0000","last_undegraded":"2024-04-17T10:38:14.304981+0000","last_fullsized":"2024-04-17T10:38:14.304981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:44:43.972627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970232+0000","last_change":"2024-04-17T10:38:12.731401+0000","last_active":"2024-04-17T10:38:13.970232+0000","last_peered":"2024-04-17T10:38:13.970232+0000","last_clean":"2024-04-17T10:38:13.970232+0000","last_became_active":"2024-04-17T10:38:12.729441+0000","last_became_peered":"2024-04-17T10:38:12.729441+0000","last_unstale":"2024-04-17T10:38:13.970232+0000","last_undegraded":"2024-04-17T10:38:13.970232+0000","last_fullsized":"2024-04-17T10:38:13.970232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:02.184227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297301+0000","last_change":"2024-04-17T10:38:12.725791+0000","last_active":"2024-04-17T10:38:14.297301+0000","last_peered":"2024-04-17T10:38:14.297301+0000","last_clean":"2024-04-17T10:38:14.297301+0000","last_became_active":"2024-04-17T10:38:12.723417+0000","last_became_peered":"2024-04-17T10:38:12.723417+0000","last_unstale":"2024-04-17T10:38:14.297301+0000","last_undegraded":"2024-04-17T10:38:14.297301+0000","last_fullsized":"2024-04-17T10:38:14.297301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:26:02.126931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970408+0000","last_change":"2024-04-17T10:38:12.727592+0000","last_active":"2024-04-17T10:38:13.970408+0000","last_peered":"2024-04-17T10:38:13.970408+0000","last_clean":"2024-04-17T10:38:13.970408+0000","last_became_active":"2024-04-17T10:38:12.726482+0000","last_became_peered":"2024-04-17T10:38:12.726482+0000","last_unstale":"2024-04-17T10:38:13.970408+0000","last_undegraded":"2024-04-17T10:38:13.970408+0000","last_fullsized":"2024-04-17T10:38:13.970408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:09:07.090758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297926+0000","last_change":"2024-04-17T10:38:12.716910+0000","last_active":"2024-04-17T10:38:14.297926+0000","last_peered":"2024-04-17T10:38:14.297926+0000","last_clean":"2024-04-17T10:38:14.297926+0000","last_became_active":"2024-04-17T10:38:12.716527+0000","last_became_peered":"2024-04-17T10:38:12.716527+0000","last_unstale":"2024-04-17T10:38:14.297926+0000","last_undegraded":"2024-04-17T10:38:14.297926+0000","last_fullsized":"2024-04-17T10:38:14.297926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:43.917742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803906+0000","last_change":"2024-04-17T10:38:12.728066+0000","last_active":"2024-04-17T10:38:13.803906+0000","last_peered":"2024-04-17T10:38:13.803906+0000","last_clean":"2024-04-17T10:38:13.803906+0000","last_became_active":"2024-04-17T10:38:12.727694+0000","last_became_peered":"2024-04-17T10:38:12.727694+0000","last_unstale":"2024-04-17T10:38:13.803906+0000","last_undegraded":"2024-04-17T10:38:13.803906+0000","last_fullsized":"2024-04-17T10:38:13.803906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:05:21.619291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695757+0000","last_change":"2024-04-17T10:38:12.714887+0000","last_active":"2024-04-17T10:38:13.695757+0000","last_peered":"2024-04-17T10:38:13.695757+0000","last_clean":"2024-04-17T10:38:13.695757+0000","last_became_active":"2024-04-17T10:38:12.714599+0000","last_became_peered":"2024-04-17T10:38:12.714599+0000","last_unstale":"2024-04-17T10:38:13.695757+0000","last_undegraded":"2024-04-17T10:38:13.695757+0000","last_fullsized":"2024-04-17T10:38:13.695757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:42.256740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297989+0000","last_change":"2024-04-17T10:38:12.725678+0000","last_active":"2024-04-17T10:38:14.297989+0000","last_peered":"2024-04-17T10:38:14.297989+0000","last_clean":"2024-04-17T10:38:14.297989+0000","last_became_active":"2024-04-17T10:38:12.723311+0000","last_became_peered":"2024-04-17T10:38:12.723311+0000","last_unstale":"2024-04-17T10:38:14.297989+0000","last_undegraded":"2024-04-17T10:38:14.297989+0000","last_fullsized":"2024-04-17T10:38:14.297989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:05:21.619291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305173+0000","last_change":"2024-04-17T10:38:12.720525+0000","last_active":"2024-04-17T10:38:14.305173+0000","last_peered":"2024-04-17T10:38:14.305173+0000","last_clean":"2024-04-17T10:38:14.305173+0000","last_became_active":"2024-04-17T10:38:12.720134+0000","last_became_peered":"2024-04-17T10:38:12.720134+0000","last_unstale":"2024-04-17T10:38:14.305173+0000","last_undegraded":"2024-04-17T10:38:14.305173+0000","last_fullsized":"2024-04-17T10:38:14.305173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:08.872431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970581+0000","last_change":"2024-04-17T10:38:12.727969+0000","last_active":"2024-04-17T10:38:13.970581+0000","last_peered":"2024-04-17T10:38:13.970581+0000","last_clean":"2024-04-17T10:38:13.970581+0000","last_became_active":"2024-04-17T10:38:12.726330+0000","last_became_peered":"2024-04-17T10:38:12.726330+0000","last_unstale":"2024-04-17T10:38:13.970581+0000","last_undegraded":"2024-04-17T10:38:13.970581+0000","last_fullsized":"2024-04-17T10:38:13.970581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:33:29.583300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.303139+0000","last_change":"2024-04-17T10:38:12.732783+0000","last_active":"2024-04-17T10:38:14.303139+0000","last_peered":"2024-04-17T10:38:14.303139+0000","last_clean":"2024-04-17T10:38:14.303139+0000","last_became_active":"2024-04-17T10:38:12.732525+0000","last_became_peered":"2024-04-17T10:38:12.732525+0000","last_unstale":"2024-04-17T10:38:14.303139+0000","last_undegraded":"2024-04-17T10:38:14.303139+0000","last_fullsized":"2024-04-17T10:38:14.303139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:21:11.071817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803187+0000","last_change":"2024-04-17T10:38:12.720029+0000","last_active":"2024-04-17T10:38:13.803187+0000","last_peered":"2024-04-17T10:38:13.803187+0000","last_clean":"2024-04-17T10:38:13.803187+0000","last_became_active":"2024-04-17T10:38:12.719579+0000","last_became_peered":"2024-04-17T10:38:12.719579+0000","last_unstale":"2024-04-17T10:38:13.803187+0000","last_undegraded":"2024-04-17T10:38:13.803187+0000","last_fullsized":"2024-04-17T10:38:13.803187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:31:48.663733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.699682+0000","last_change":"2024-04-17T10:38:12.709872+0000","last_active":"2024-04-17T10:38:13.699682+0000","last_peered":"2024-04-17T10:38:13.699682+0000","last_clean":"2024-04-17T10:38:13.699682+0000","last_became_active":"2024-04-17T10:38:12.709614+0000","last_became_peered":"2024-04-17T10:38:12.709614+0000","last_unstale":"2024-04-17T10:38:13.699682+0000","last_undegraded":"2024-04-17T10:38:13.699682+0000","last_fullsized":"2024-04-17T10:38:13.699682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:46.375063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298173+0000","last_change":"2024-04-17T10:38:11.718367+0000","last_active":"2024-04-17T10:38:14.298173+0000","last_peered":"2024-04-17T10:38:14.298173+0000","last_clean":"2024-04-17T10:38:14.298173+0000","last_became_active":"2024-04-17T10:38:11.718004+0000","last_became_peered":"2024-04-17T10:38:11.718004+0000","last_unstale":"2024-04-17T10:38:14.298173+0000","last_undegraded":"2024-04-17T10:38:14.298173+0000","last_fullsized":"2024-04-17T10:38:14.298173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:39.210602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305196+0000","last_change":"2024-04-17T10:38:12.710623+0000","last_active":"2024-04-17T10:38:14.305196+0000","last_peered":"2024-04-17T10:38:14.305196+0000","last_clean":"2024-04-17T10:38:14.305196+0000","last_became_active":"2024-04-17T10:38:12.708883+0000","last_became_peered":"2024-04-17T10:38:12.708883+0000","last_unstale":"2024-04-17T10:38:14.305196+0000","last_undegraded":"2024-04-17T10:38:14.305196+0000","last_fullsized":"2024-04-17T10:38:14.305196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:26:02.126931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803042+0000","last_change":"2024-04-17T10:38:11.718224+0000","last_active":"2024-04-17T10:38:13.803042+0000","last_peered":"2024-04-17T10:38:13.803042+0000","last_clean":"2024-04-17T10:38:13.803042+0000","last_became_active":"2024-04-17T10:38:11.717433+0000","last_became_peered":"2024-04-17T10:38:11.717433+0000","last_unstale":"2024-04-17T10:38:13.803042+0000","last_undegraded":"2024-04-17T10:38:13.803042+0000","last_fullsized":"2024-04-17T10:38:13.803042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:39.210602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298350+0000","last_change":"2024-04-17T10:38:11.719972+0000","last_active":"2024-04-17T10:38:14.298350+0000","last_peered":"2024-04-17T10:38:14.298350+0000","last_clean":"2024-04-17T10:38:14.298350+0000","last_became_active":"2024-04-17T10:38:11.719469+0000","last_became_peered":"2024-04-17T10:38:11.719469+0000","last_unstale":"2024-04-17T10:38:14.298350+0000","last_undegraded":"2024-04-17T10:38:14.298350+0000","last_fullsized":"2024-04-17T10:38:14.298350+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:59:51.693435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297286+0000","last_change":"2024-04-17T10:38:12.722655+0000","last_active":"2024-04-17T10:38:14.297286+0000","last_peered":"2024-04-17T10:38:14.297286+0000","last_clean":"2024-04-17T10:38:14.297286+0000","last_became_active":"2024-04-17T10:38:12.722213+0000","last_became_peered":"2024-04-17T10:38:12.722213+0000","last_unstale":"2024-04-17T10:38:14.297286+0000","last_undegraded":"2024-04-17T10:38:14.297286+0000","last_fullsized":"2024-04-17T10:38:14.297286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:08.872431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970337+0000","last_change":"2024-04-17T10:38:11.704898+0000","last_active":"2024-04-17T10:38:13.970337+0000","last_peered":"2024-04-17T10:38:13.970337+0000","last_clean":"2024-04-17T10:38:13.970337+0000","last_became_active":"2024-04-17T10:38:11.704184+0000","last_became_peered":"2024-04-17T10:38:11.704184+0000","last_unstale":"2024-04-17T10:38:13.970337+0000","last_undegraded":"2024-04-17T10:38:13.970337+0000","last_fullsized":"2024-04-17T10:38:13.970337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:45.366572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695486+0000","last_change":"2024-04-17T10:38:12.714844+0000","last_active":"2024-04-17T10:38:13.695486+0000","last_peered":"2024-04-17T10:38:13.695486+0000","last_clean":"2024-04-17T10:38:13.695486+0000","last_became_active":"2024-04-17T10:38:12.714565+0000","last_became_peered":"2024-04-17T10:38:12.714565+0000","last_unstale":"2024-04-17T10:38:13.695486+0000","last_undegraded":"2024-04-17T10:38:13.695486+0000","last_fullsized":"2024-04-17T10:38:13.695486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:26:02.126931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803577+0000","last_change":"2024-04-17T10:38:11.725276+0000","last_active":"2024-04-17T10:38:13.803577+0000","last_peered":"2024-04-17T10:38:13.803577+0000","last_clean":"2024-04-17T10:38:13.803577+0000","last_became_active":"2024-04-17T10:38:11.724660+0000","last_became_peered":"2024-04-17T10:38:11.724660+0000","last_unstale":"2024-04-17T10:38:13.803577+0000","last_undegraded":"2024-04-17T10:38:13.803577+0000","last_fullsized":"2024-04-17T10:38:13.803577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:21:00.724523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970977+0000","last_change":"2024-04-17T10:38:12.731384+0000","last_active":"2024-04-17T10:38:13.970977+0000","last_peered":"2024-04-17T10:38:13.970977+0000","last_clean":"2024-04-17T10:38:13.970977+0000","last_became_active":"2024-04-17T10:38:12.730178+0000","last_became_peered":"2024-04-17T10:38:12.730178+0000","last_unstale":"2024-04-17T10:38:13.970977+0000","last_undegraded":"2024-04-17T10:38:13.970977+0000","last_fullsized":"2024-04-17T10:38:13.970977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:11:49.847070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970956+0000","last_change":"2024-04-17T10:38:11.727591+0000","last_active":"2024-04-17T10:38:13.970956+0000","last_peered":"2024-04-17T10:38:13.970956+0000","last_clean":"2024-04-17T10:38:13.970956+0000","last_became_active":"2024-04-17T10:38:11.726319+0000","last_became_peered":"2024-04-17T10:38:11.726319+0000","last_unstale":"2024-04-17T10:38:13.970956+0000","last_undegraded":"2024-04-17T10:38:13.970956+0000","last_fullsized":"2024-04-17T10:38:13.970956+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:50:59.373356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305290+0000","last_change":"2024-04-17T10:38:12.727239+0000","last_active":"2024-04-17T10:38:14.305290+0000","last_peered":"2024-04-17T10:38:14.305290+0000","last_clean":"2024-04-17T10:38:14.305290+0000","last_became_active":"2024-04-17T10:38:12.726192+0000","last_became_peered":"2024-04-17T10:38:12.726192+0000","last_unstale":"2024-04-17T10:38:14.305290+0000","last_undegraded":"2024-04-17T10:38:14.305290+0000","last_fullsized":"2024-04-17T10:38:14.305290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:39.640730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695810+0000","last_change":"2024-04-17T10:38:11.710534+0000","last_active":"2024-04-17T10:38:13.695810+0000","last_peered":"2024-04-17T10:38:13.695810+0000","last_clean":"2024-04-17T10:38:13.695810+0000","last_became_active":"2024-04-17T10:38:11.709944+0000","last_became_peered":"2024-04-17T10:38:11.709944+0000","last_unstale":"2024-04-17T10:38:13.695810+0000","last_undegraded":"2024-04-17T10:38:13.695810+0000","last_fullsized":"2024-04-17T10:38:13.695810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:45.366572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.700329+0000","last_change":"2024-04-17T10:38:12.709577+0000","last_active":"2024-04-17T10:38:13.700329+0000","last_peered":"2024-04-17T10:38:13.700329+0000","last_clean":"2024-04-17T10:38:13.700329+0000","last_became_active":"2024-04-17T10:38:12.709186+0000","last_became_peered":"2024-04-17T10:38:12.709186+0000","last_unstale":"2024-04-17T10:38:13.700329+0000","last_undegraded":"2024-04-17T10:38:13.700329+0000","last_fullsized":"2024-04-17T10:38:13.700329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:20:45.810408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803828+0000","last_change":"2024-04-17T10:38:11.715117+0000","last_active":"2024-04-17T10:38:13.803828+0000","last_peered":"2024-04-17T10:38:13.803828+0000","last_clean":"2024-04-17T10:38:13.803828+0000","last_became_active":"2024-04-17T10:38:11.713981+0000","last_became_peered":"2024-04-17T10:38:11.713981+0000","last_unstale":"2024-04-17T10:38:13.803828+0000","last_undegraded":"2024-04-17T10:38:13.803828+0000","last_fullsized":"2024-04-17T10:38:13.803828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:27.569244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701871+0000","last_change":"2024-04-17T10:38:12.708377+0000","last_active":"2024-04-17T10:38:13.701871+0000","last_peered":"2024-04-17T10:38:13.701871+0000","last_clean":"2024-04-17T10:38:13.701871+0000","last_became_active":"2024-04-17T10:38:12.707183+0000","last_became_peered":"2024-04-17T10:38:12.707183+0000","last_unstale":"2024-04-17T10:38:13.701871+0000","last_undegraded":"2024-04-17T10:38:13.701871+0000","last_fullsized":"2024-04-17T10:38:13.701871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:06.508830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305423+0000","last_change":"2024-04-17T10:38:11.702326+0000","last_active":"2024-04-17T10:38:14.305423+0000","last_peered":"2024-04-17T10:38:14.305423+0000","last_clean":"2024-04-17T10:38:14.305423+0000","last_became_active":"2024-04-17T10:38:11.701735+0000","last_became_peered":"2024-04-17T10:38:11.701735+0000","last_unstale":"2024-04-17T10:38:14.305423+0000","last_undegraded":"2024-04-17T10:38:14.305423+0000","last_fullsized":"2024-04-17T10:38:14.305423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:11:43.267871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.304822+0000","last_change":"2024-04-17T10:38:12.705702+0000","last_active":"2024-04-17T10:38:14.304822+0000","last_peered":"2024-04-17T10:38:14.304822+0000","last_clean":"2024-04-17T10:38:14.304822+0000","last_became_active":"2024-04-17T10:38:12.705297+0000","last_became_peered":"2024-04-17T10:38:12.705297+0000","last_unstale":"2024-04-17T10:38:14.304822+0000","last_undegraded":"2024-04-17T10:38:14.304822+0000","last_fullsized":"2024-04-17T10:38:14.304822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:15:31.987120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970479+0000","last_change":"2024-04-17T10:38:11.715689+0000","last_active":"2024-04-17T10:38:13.970479+0000","last_peered":"2024-04-17T10:38:13.970479+0000","last_clean":"2024-04-17T10:38:13.970479+0000","last_became_active":"2024-04-17T10:38:11.714329+0000","last_became_peered":"2024-04-17T10:38:11.714329+0000","last_unstale":"2024-04-17T10:38:13.970479+0000","last_undegraded":"2024-04-17T10:38:13.970479+0000","last_fullsized":"2024-04-17T10:38:13.970479+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:27.569244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702011+0000","last_change":"2024-04-17T10:38:12.708536+0000","last_active":"2024-04-17T10:38:13.702011+0000","last_peered":"2024-04-17T10:38:13.702011+0000","last_clean":"2024-04-17T10:38:13.702011+0000","last_became_active":"2024-04-17T10:38:12.707643+0000","last_became_peered":"2024-04-17T10:38:12.707643+0000","last_unstale":"2024-04-17T10:38:13.702011+0000","last_undegraded":"2024-04-17T10:38:13.702011+0000","last_fullsized":"2024-04-17T10:38:13.702011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:22:10.364740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803484+0000","last_change":"2024-04-17T10:38:11.717992+0000","last_active":"2024-04-17T10:38:13.803484+0000","last_peered":"2024-04-17T10:38:13.803484+0000","last_clean":"2024-04-17T10:38:13.803484+0000","last_became_active":"2024-04-17T10:38:11.717124+0000","last_became_peered":"2024-04-17T10:38:11.717124+0000","last_unstale":"2024-04-17T10:38:13.803484+0000","last_undegraded":"2024-04-17T10:38:13.803484+0000","last_fullsized":"2024-04-17T10:38:13.803484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:24:23.818791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803260+0000","last_change":"2024-04-17T10:38:12.720282+0000","last_active":"2024-04-17T10:38:13.803260+0000","last_peered":"2024-04-17T10:38:13.803260+0000","last_clean":"2024-04-17T10:38:13.803260+0000","last_became_active":"2024-04-17T10:38:12.719197+0000","last_became_peered":"2024-04-17T10:38:12.719197+0000","last_unstale":"2024-04-17T10:38:13.803260+0000","last_undegraded":"2024-04-17T10:38:13.803260+0000","last_fullsized":"2024-04-17T10:38:13.803260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:08.872431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.303153+0000","last_change":"2024-04-17T10:38:11.689943+0000","last_active":"2024-04-17T10:38:14.303153+0000","last_peered":"2024-04-17T10:38:14.303153+0000","last_clean":"2024-04-17T10:38:14.303153+0000","last_became_active":"2024-04-17T10:38:11.689627+0000","last_became_peered":"2024-04-17T10:38:11.689627+0000","last_unstale":"2024-04-17T10:38:14.303153+0000","last_undegraded":"2024-04-17T10:38:14.303153+0000","last_fullsized":"2024-04-17T10:38:14.303153+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:39.210602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701741+0000","last_change":"2024-04-17T10:38:12.710693+0000","last_active":"2024-04-17T10:38:13.701741+0000","last_peered":"2024-04-17T10:38:13.701741+0000","last_clean":"2024-04-17T10:38:13.701741+0000","last_became_active":"2024-04-17T10:38:12.710263+0000","last_became_peered":"2024-04-17T10:38:12.710263+0000","last_unstale":"2024-04-17T10:38:13.701741+0000","last_undegraded":"2024-04-17T10:38:13.701741+0000","last_fullsized":"2024-04-17T10:38:13.701741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:09:07.090758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.700517+0000","last_change":"2024-04-17T10:38:11.701185+0000","last_active":"2024-04-17T10:38:13.700517+0000","last_peered":"2024-04-17T10:38:13.700517+0000","last_clean":"2024-04-17T10:38:13.700517+0000","last_became_active":"2024-04-17T10:38:11.700292+0000","last_became_peered":"2024-04-17T10:38:11.700292+0000","last_unstale":"2024-04-17T10:38:13.700517+0000","last_undegraded":"2024-04-17T10:38:13.700517+0000","last_fullsized":"2024-04-17T10:38:13.700517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:59:51.693435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702731+0000","last_change":"2024-04-17T10:38:12.704267+0000","last_active":"2024-04-17T10:38:13.702731+0000","last_peered":"2024-04-17T10:38:13.702731+0000","last_clean":"2024-04-17T10:38:13.702731+0000","last_became_active":"2024-04-17T10:38:12.703820+0000","last_became_peered":"2024-04-17T10:38:12.703820+0000","last_unstale":"2024-04-17T10:38:13.702731+0000","last_undegraded":"2024-04-17T10:38:13.702731+0000","last_fullsized":"2024-04-17T10:38:13.702731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:43:51.398103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.303738+0000","last_change":"2024-04-17T10:38:11.699100+0000","last_active":"2024-04-17T10:38:14.303738+0000","last_peered":"2024-04-17T10:38:14.303738+0000","last_clean":"2024-04-17T10:38:14.303738+0000","last_became_active":"2024-04-17T10:38:11.698786+0000","last_became_peered":"2024-04-17T10:38:11.698786+0000","last_unstale":"2024-04-17T10:38:14.303738+0000","last_undegraded":"2024-04-17T10:38:14.303738+0000","last_fullsized":"2024-04-17T10:38:14.303738+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:21:00.724523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702808+0000","last_change":"2024-04-17T10:38:12.705166+0000","last_active":"2024-04-17T10:38:13.702808+0000","last_peered":"2024-04-17T10:38:13.702808+0000","last_clean":"2024-04-17T10:38:13.702808+0000","last_became_active":"2024-04-17T10:38:12.704805+0000","last_became_peered":"2024-04-17T10:38:12.704805+0000","last_unstale":"2024-04-17T10:38:13.702808+0000","last_undegraded":"2024-04-17T10:38:13.702808+0000","last_fullsized":"2024-04-17T10:38:13.702808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:49:18.397560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.696028+0000","last_change":"2024-04-17T10:38:11.692209+0000","last_active":"2024-04-17T10:38:13.696028+0000","last_peered":"2024-04-17T10:38:13.696028+0000","last_clean":"2024-04-17T10:38:13.696028+0000","last_became_active":"2024-04-17T10:38:11.691905+0000","last_became_peered":"2024-04-17T10:38:11.691905+0000","last_unstale":"2024-04-17T10:38:13.696028+0000","last_undegraded":"2024-04-17T10:38:13.696028+0000","last_fullsized":"2024-04-17T10:38:13.696028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:50:59.373356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970996+0000","last_change":"2024-04-17T10:38:12.727145+0000","last_active":"2024-04-17T10:38:13.970996+0000","last_peered":"2024-04-17T10:38:13.970996+0000","last_clean":"2024-04-17T10:38:13.970996+0000","last_became_active":"2024-04-17T10:38:12.726797+0000","last_became_peered":"2024-04-17T10:38:12.726797+0000","last_unstale":"2024-04-17T10:38:13.970996+0000","last_undegraded":"2024-04-17T10:38:13.970996+0000","last_fullsized":"2024-04-17T10:38:13.970996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:43:51.398103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.303519+0000","last_change":"2024-04-17T10:38:11.705218+0000","last_active":"2024-04-17T10:38:14.303519+0000","last_peered":"2024-04-17T10:38:14.303519+0000","last_clean":"2024-04-17T10:38:14.303519+0000","last_became_active":"2024-04-17T10:38:11.704928+0000","last_became_peered":"2024-04-17T10:38:11.704928+0000","last_unstale":"2024-04-17T10:38:14.303519+0000","last_undegraded":"2024-04-17T10:38:14.303519+0000","last_fullsized":"2024-04-17T10:38:14.303519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:27.569244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701937+0000","last_change":"2024-04-17T10:38:12.722659+0000","last_active":"2024-04-17T10:38:13.701937+0000","last_peered":"2024-04-17T10:38:13.701937+0000","last_clean":"2024-04-17T10:38:13.701937+0000","last_became_active":"2024-04-17T10:38:12.722265+0000","last_became_peered":"2024-04-17T10:38:12.722265+0000","last_unstale":"2024-04-17T10:38:13.701937+0000","last_undegraded":"2024-04-17T10:38:13.701937+0000","last_fullsized":"2024-04-17T10:38:13.701937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:33:29.583300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701282+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:38:13.701282+0000","last_peered":"2024-04-17T10:38:13.701282+0000","last_clean":"2024-04-17T10:38:13.701282+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:38:13.701282+0000","last_undegraded":"2024-04-17T10:38:13.701282+0000","last_fullsized":"2024-04-17T10:38:13.701282+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970082+0000","last_change":"2024-04-17T10:38:11.702024+0000","last_active":"2024-04-17T10:38:13.970082+0000","last_peered":"2024-04-17T10:38:13.970082+0000","last_clean":"2024-04-17T10:38:13.970082+0000","last_became_active":"2024-04-17T10:38:11.701157+0000","last_became_peered":"2024-04-17T10:38:11.701157+0000","last_unstale":"2024-04-17T10:38:13.970082+0000","last_undegraded":"2024-04-17T10:38:13.970082+0000","last_fullsized":"2024-04-17T10:38:13.970082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:24:23.818791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297692+0000","last_change":"2024-04-17T10:38:12.723529+0000","last_active":"2024-04-17T10:38:14.297692+0000","last_peered":"2024-04-17T10:38:14.297692+0000","last_clean":"2024-04-17T10:38:14.297692+0000","last_became_active":"2024-04-17T10:38:12.723103+0000","last_became_peered":"2024-04-17T10:38:12.723103+0000","last_unstale":"2024-04-17T10:38:14.297692+0000","last_undegraded":"2024-04-17T10:38:14.297692+0000","last_fullsized":"2024-04-17T10:38:14.297692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:42.256740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701197+0000","last_change":"2024-04-17T10:38:11.716453+0000","last_active":"2024-04-17T10:38:13.701197+0000","last_peered":"2024-04-17T10:38:13.701197+0000","last_clean":"2024-04-17T10:38:13.701197+0000","last_became_active":"2024-04-17T10:38:11.715859+0000","last_became_peered":"2024-04-17T10:38:11.715859+0000","last_unstale":"2024-04-17T10:38:13.701197+0000","last_undegraded":"2024-04-17T10:38:13.701197+0000","last_fullsized":"2024-04-17T10:38:13.701197+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:50:59.373356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.304804+0000","last_change":"2024-04-17T10:38:12.707314+0000","last_active":"2024-04-17T10:38:14.304804+0000","last_peered":"2024-04-17T10:38:14.304804+0000","last_clean":"2024-04-17T10:38:14.304804+0000","last_became_active":"2024-04-17T10:38:12.706874+0000","last_became_peered":"2024-04-17T10:38:12.706874+0000","last_unstale":"2024-04-17T10:38:14.304804+0000","last_undegraded":"2024-04-17T10:38:14.304804+0000","last_fullsized":"2024-04-17T10:38:14.304804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:07:07.037555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.303300+0000","last_change":"2024-04-17T10:38:11.699752+0000","last_active":"2024-04-17T10:38:14.303300+0000","last_peered":"2024-04-17T10:38:14.303300+0000","last_clean":"2024-04-17T10:38:14.303300+0000","last_became_active":"2024-04-17T10:38:11.699467+0000","last_became_peered":"2024-04-17T10:38:11.699467+0000","last_unstale":"2024-04-17T10:38:14.303300+0000","last_undegraded":"2024-04-17T10:38:14.303300+0000","last_fullsized":"2024-04-17T10:38:14.303300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:24:23.818791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695234+0000","last_change":"2024-04-17T10:38:12.723809+0000","last_active":"2024-04-17T10:38:13.695234+0000","last_peered":"2024-04-17T10:38:13.695234+0000","last_clean":"2024-04-17T10:38:13.695234+0000","last_became_active":"2024-04-17T10:38:12.723282+0000","last_became_peered":"2024-04-17T10:38:12.723282+0000","last_unstale":"2024-04-17T10:38:13.695234+0000","last_undegraded":"2024-04-17T10:38:13.695234+0000","last_fullsized":"2024-04-17T10:38:13.695234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:31:48.663733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.971078+0000","last_change":"2024-04-17T10:38:11.715164+0000","last_active":"2024-04-17T10:38:13.971078+0000","last_peered":"2024-04-17T10:38:13.971078+0000","last_clean":"2024-04-17T10:38:13.971078+0000","last_became_active":"2024-04-17T10:38:11.712376+0000","last_became_peered":"2024-04-17T10:38:11.712376+0000","last_unstale":"2024-04-17T10:38:13.971078+0000","last_undegraded":"2024-04-17T10:38:13.971078+0000","last_fullsized":"2024-04-17T10:38:13.971078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:21:00.724523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.699765+0000","last_change":"2024-04-17T10:38:12.720339+0000","last_active":"2024-04-17T10:38:13.699765+0000","last_peered":"2024-04-17T10:38:13.699765+0000","last_clean":"2024-04-17T10:38:13.699765+0000","last_became_active":"2024-04-17T10:38:12.720051+0000","last_became_peered":"2024-04-17T10:38:12.720051+0000","last_unstale":"2024-04-17T10:38:13.699765+0000","last_undegraded":"2024-04-17T10:38:13.699765+0000","last_fullsized":"2024-04-17T10:38:13.699765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:51:00.381846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695713+0000","last_change":"2024-04-17T10:38:11.693095+0000","last_active":"2024-04-17T10:38:13.695713+0000","last_peered":"2024-04-17T10:38:13.695713+0000","last_clean":"2024-04-17T10:38:13.695713+0000","last_became_active":"2024-04-17T10:38:11.692723+0000","last_became_peered":"2024-04-17T10:38:11.692723+0000","last_unstale":"2024-04-17T10:38:13.695713+0000","last_undegraded":"2024-04-17T10:38:13.695713+0000","last_fullsized":"2024-04-17T10:38:13.695713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:27.569244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970920+0000","last_change":"2024-04-17T10:38:12.728559+0000","last_active":"2024-04-17T10:38:13.970920+0000","last_peered":"2024-04-17T10:38:13.970920+0000","last_clean":"2024-04-17T10:38:13.970920+0000","last_became_active":"2024-04-17T10:38:12.728285+0000","last_became_peered":"2024-04-17T10:38:12.728285+0000","last_unstale":"2024-04-17T10:38:13.970920+0000","last_undegraded":"2024-04-17T10:38:13.970920+0000","last_fullsized":"2024-04-17T10:38:13.970920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:22:10.364740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702148+0000","last_change":"2024-04-17T10:38:11.717791+0000","last_active":"2024-04-17T10:38:13.702148+0000","last_peered":"2024-04-17T10:38:13.702148+0000","last_clean":"2024-04-17T10:38:13.702148+0000","last_became_active":"2024-04-17T10:38:11.717130+0000","last_became_peered":"2024-04-17T10:38:11.717130+0000","last_unstale":"2024-04-17T10:38:13.702148+0000","last_undegraded":"2024-04-17T10:38:13.702148+0000","last_fullsized":"2024-04-17T10:38:13.702148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:21:10.063327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695581+0000","last_change":"2024-04-17T10:38:12.723436+0000","last_active":"2024-04-17T10:38:13.695581+0000","last_peered":"2024-04-17T10:38:13.695581+0000","last_clean":"2024-04-17T10:38:13.695581+0000","last_became_active":"2024-04-17T10:38:12.723138+0000","last_became_peered":"2024-04-17T10:38:12.723138+0000","last_unstale":"2024-04-17T10:38:13.695581+0000","last_undegraded":"2024-04-17T10:38:13.695581+0000","last_fullsized":"2024-04-17T10:38:13.695581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:39.640730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.365894+0000","last_change":"2024-04-17T10:38:11.730180+0000","last_active":"2024-04-17T10:38:14.365894+0000","last_peered":"2024-04-17T10:38:14.365894+0000","last_clean":"2024-04-17T10:38:14.365894+0000","last_became_active":"2024-04-17T10:38:11.729843+0000","last_became_peered":"2024-04-17T10:38:11.729843+0000","last_unstale":"2024-04-17T10:38:14.365894+0000","last_undegraded":"2024-04-17T10:38:14.365894+0000","last_fullsized":"2024-04-17T10:38:14.365894+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:23:29.956041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298370+0000","last_change":"2024-04-17T10:38:12.732643+0000","last_active":"2024-04-17T10:38:14.298370+0000","last_peered":"2024-04-17T10:38:14.298370+0000","last_clean":"2024-04-17T10:38:14.298370+0000","last_became_active":"2024-04-17T10:38:12.732259+0000","last_became_peered":"2024-04-17T10:38:12.732259+0000","last_unstale":"2024-04-17T10:38:14.298370+0000","last_undegraded":"2024-04-17T10:38:14.298370+0000","last_fullsized":"2024-04-17T10:38:14.298370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:02.184227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803741+0000","last_change":"2024-04-17T10:38:11.725444+0000","last_active":"2024-04-17T10:38:13.803741+0000","last_peered":"2024-04-17T10:38:13.803741+0000","last_clean":"2024-04-17T10:38:13.803741+0000","last_became_active":"2024-04-17T10:38:11.724780+0000","last_became_peered":"2024-04-17T10:38:11.724780+0000","last_unstale":"2024-04-17T10:38:13.803741+0000","last_undegraded":"2024-04-17T10:38:13.803741+0000","last_fullsized":"2024-04-17T10:38:13.803741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:45.366572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305401+0000","last_change":"2024-04-17T10:38:12.705770+0000","last_active":"2024-04-17T10:38:14.305401+0000","last_peered":"2024-04-17T10:38:14.305401+0000","last_clean":"2024-04-17T10:38:14.305401+0000","last_became_active":"2024-04-17T10:38:12.705400+0000","last_became_peered":"2024-04-17T10:38:12.705400+0000","last_unstale":"2024-04-17T10:38:14.305401+0000","last_undegraded":"2024-04-17T10:38:14.305401+0000","last_fullsized":"2024-04-17T10:38:14.305401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:42.256740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701197+0000","last_change":"2024-04-17T10:38:11.716433+0000","last_active":"2024-04-17T10:38:13.701197+0000","last_peered":"2024-04-17T10:38:13.701197+0000","last_clean":"2024-04-17T10:38:13.701197+0000","last_became_active":"2024-04-17T10:38:11.715803+0000","last_became_peered":"2024-04-17T10:38:11.715803+0000","last_unstale":"2024-04-17T10:38:13.701197+0000","last_undegraded":"2024-04-17T10:38:13.701197+0000","last_fullsized":"2024-04-17T10:38:13.701197+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:45.366572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.700697+0000","last_change":"2024-04-17T10:38:12.717613+0000","last_active":"2024-04-17T10:38:13.700697+0000","last_peered":"2024-04-17T10:38:13.700697+0000","last_clean":"2024-04-17T10:38:13.700697+0000","last_became_active":"2024-04-17T10:38:12.717308+0000","last_became_peered":"2024-04-17T10:38:12.717308+0000","last_unstale":"2024-04-17T10:38:13.700697+0000","last_undegraded":"2024-04-17T10:38:13.700697+0000","last_fullsized":"2024-04-17T10:38:13.700697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:21:11.071817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695770+0000","last_change":"2024-04-17T10:38:11.716778+0000","last_active":"2024-04-17T10:38:13.695770+0000","last_peered":"2024-04-17T10:38:13.695770+0000","last_clean":"2024-04-17T10:38:13.695770+0000","last_became_active":"2024-04-17T10:38:11.716526+0000","last_became_peered":"2024-04-17T10:38:11.716526+0000","last_unstale":"2024-04-17T10:38:13.695770+0000","last_undegraded":"2024-04-17T10:38:13.695770+0000","last_fullsized":"2024-04-17T10:38:13.695770+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:24:23.818791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297622+0000","last_change":"2024-04-17T10:38:12.717941+0000","last_active":"2024-04-17T10:38:14.297622+0000","last_peered":"2024-04-17T10:38:14.297622+0000","last_clean":"2024-04-17T10:38:14.297622+0000","last_became_active":"2024-04-17T10:38:12.717350+0000","last_became_peered":"2024-04-17T10:38:12.717350+0000","last_unstale":"2024-04-17T10:38:14.297622+0000","last_undegraded":"2024-04-17T10:38:14.297622+0000","last_fullsized":"2024-04-17T10:38:14.297622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:39.640730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695661+0000","last_change":"2024-04-17T10:38:11.699325+0000","last_active":"2024-04-17T10:38:13.695661+0000","last_peered":"2024-04-17T10:38:13.695661+0000","last_clean":"2024-04-17T10:38:13.695661+0000","last_became_active":"2024-04-17T10:38:11.698981+0000","last_became_peered":"2024-04-17T10:38:11.698981+0000","last_unstale":"2024-04-17T10:38:13.695661+0000","last_undegraded":"2024-04-17T10:38:13.695661+0000","last_fullsized":"2024-04-17T10:38:13.695661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:21:00.724523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298006+0000","last_change":"2024-04-17T10:38:12.731391+0000","last_active":"2024-04-17T10:38:14.298006+0000","last_peered":"2024-04-17T10:38:14.298006+0000","last_clean":"2024-04-17T10:38:14.298006+0000","last_became_active":"2024-04-17T10:38:12.727638+0000","last_became_peered":"2024-04-17T10:38:12.727638+0000","last_unstale":"2024-04-17T10:38:14.298006+0000","last_undegraded":"2024-04-17T10:38:14.298006+0000","last_fullsized":"2024-04-17T10:38:14.298006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:23.465112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701909+0000","last_change":"2024-04-17T10:38:11.716353+0000","last_active":"2024-04-17T10:38:13.701909+0000","last_peered":"2024-04-17T10:38:13.701909+0000","last_clean":"2024-04-17T10:38:13.701909+0000","last_became_active":"2024-04-17T10:38:11.715722+0000","last_became_peered":"2024-04-17T10:38:11.715722+0000","last_unstale":"2024-04-17T10:38:13.701909+0000","last_undegraded":"2024-04-17T10:38:13.701909+0000","last_fullsized":"2024-04-17T10:38:13.701909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:20:44.801917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803339+0000","last_change":"2024-04-17T10:38:12.715919+0000","last_active":"2024-04-17T10:38:13.803339+0000","last_peered":"2024-04-17T10:38:13.803339+0000","last_clean":"2024-04-17T10:38:13.803339+0000","last_became_active":"2024-04-17T10:38:12.715568+0000","last_became_peered":"2024-04-17T10:38:12.715568+0000","last_unstale":"2024-04-17T10:38:13.803339+0000","last_undegraded":"2024-04-17T10:38:13.803339+0000","last_fullsized":"2024-04-17T10:38:13.803339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:26:02.126931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970940+0000","last_change":"2024-04-17T10:38:12.724977+0000","last_active":"2024-04-17T10:38:13.970940+0000","last_peered":"2024-04-17T10:38:13.970940+0000","last_clean":"2024-04-17T10:38:13.970940+0000","last_became_active":"2024-04-17T10:38:12.724700+0000","last_became_peered":"2024-04-17T10:38:12.724700+0000","last_unstale":"2024-04-17T10:38:13.970940+0000","last_undegraded":"2024-04-17T10:38:13.970940+0000","last_fullsized":"2024-04-17T10:38:13.970940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:38:39.116589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702197+0000","last_change":"2024-04-17T10:38:11.719799+0000","last_active":"2024-04-17T10:38:13.702197+0000","last_peered":"2024-04-17T10:38:13.702197+0000","last_clean":"2024-04-17T10:38:13.702197+0000","last_became_active":"2024-04-17T10:38:11.719147+0000","last_became_peered":"2024-04-17T10:38:11.719147+0000","last_unstale":"2024-04-17T10:38:13.702197+0000","last_undegraded":"2024-04-17T10:38:13.702197+0000","last_fullsized":"2024-04-17T10:38:13.702197+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:23:29.956041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.971331+0000","last_change":"2024-04-17T10:38:12.725123+0000","last_active":"2024-04-17T10:38:13.971331+0000","last_peered":"2024-04-17T10:38:13.971331+0000","last_clean":"2024-04-17T10:38:13.971331+0000","last_became_active":"2024-04-17T10:38:12.724760+0000","last_became_peered":"2024-04-17T10:38:12.724760+0000","last_unstale":"2024-04-17T10:38:13.971331+0000","last_undegraded":"2024-04-17T10:38:13.971331+0000","last_fullsized":"2024-04-17T10:38:13.971331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:49:18.397560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702167+0000","last_change":"2024-04-17T10:38:11.727050+0000","last_active":"2024-04-17T10:38:13.702167+0000","last_peered":"2024-04-17T10:38:13.702167+0000","last_clean":"2024-04-17T10:38:13.702167+0000","last_became_active":"2024-04-17T10:38:11.726393+0000","last_became_peered":"2024-04-17T10:38:11.726393+0000","last_unstale":"2024-04-17T10:38:13.702167+0000","last_undegraded":"2024-04-17T10:38:13.702167+0000","last_fullsized":"2024-04-17T10:38:13.702167+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:23:25.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305077+0000","last_change":"2024-04-17T10:38:12.726864+0000","last_active":"2024-04-17T10:38:14.305077+0000","last_peered":"2024-04-17T10:38:14.305077+0000","last_clean":"2024-04-17T10:38:14.305077+0000","last_became_active":"2024-04-17T10:38:12.726581+0000","last_became_peered":"2024-04-17T10:38:12.726581+0000","last_unstale":"2024-04-17T10:38:14.305077+0000","last_undegraded":"2024-04-17T10:38:14.305077+0000","last_fullsized":"2024-04-17T10:38:14.305077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:31:48.663733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298342+0000","last_change":"2024-04-17T10:38:11.700825+0000","last_active":"2024-04-17T10:38:14.298342+0000","last_peered":"2024-04-17T10:38:14.298342+0000","last_clean":"2024-04-17T10:38:14.298342+0000","last_became_active":"2024-04-17T10:38:11.700354+0000","last_became_peered":"2024-04-17T10:38:11.700354+0000","last_unstale":"2024-04-17T10:38:14.298342+0000","last_undegraded":"2024-04-17T10:38:14.298342+0000","last_fullsized":"2024-04-17T10:38:14.298342+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:22:08.746889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298080+0000","last_change":"2024-04-17T10:38:12.719771+0000","last_active":"2024-04-17T10:38:14.298080+0000","last_peered":"2024-04-17T10:38:14.298080+0000","last_clean":"2024-04-17T10:38:14.298080+0000","last_became_active":"2024-04-17T10:38:12.716013+0000","last_became_peered":"2024-04-17T10:38:12.716013+0000","last_unstale":"2024-04-17T10:38:14.298080+0000","last_undegraded":"2024-04-17T10:38:14.298080+0000","last_fullsized":"2024-04-17T10:38:14.298080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:54:26.054113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305389+0000","last_change":"2024-04-17T10:38:11.710717+0000","last_active":"2024-04-17T10:38:14.305389+0000","last_peered":"2024-04-17T10:38:14.305389+0000","last_clean":"2024-04-17T10:38:14.305389+0000","last_became_active":"2024-04-17T10:38:11.710342+0000","last_became_peered":"2024-04-17T10:38:11.710342+0000","last_unstale":"2024-04-17T10:38:14.305389+0000","last_undegraded":"2024-04-17T10:38:14.305389+0000","last_fullsized":"2024-04-17T10:38:14.305389+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:16:30.923882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:38:20.481 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-17T10:38:20.482 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-17T10:38:20.482 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-17T10:38:20.482 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-17T10:38:20.482 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-17T10:38:20.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.644+0000 7f18a25b6640 1 Processor -- start 2024-04-17T10:38:20.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.644+0000 7f18a25b6640 1 -- start start 2024-04-17T10:38:20.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.644+0000 7f18a25b6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f189c15ba30 0x7f189c072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:20.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.644+0000 7f18a25b6640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f189c06e320 con 0x7f189c15c960 2024-04-17T10:38:20.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.644+0000 7f18a25b6640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f189c06e040 con 0x7f189c0cfbe0 2024-04-17T10:38:20.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.644+0000 7f18a25b6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f189c071ea0 con 0x7f189c15ba30 2024-04-17T10:38:20.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.644+0000 7f189b7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f189c15c960 0x7f189c0cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39836/0 (socket says 172.21.15.83:39836) 2024-04-17T10:38:20.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.644+0000 7f189b7fe640 1 -- 172.21.15.83:0/717092710 learned_addr learned my addr 172.21.15.83:0/717092710 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:20.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.644+0000 7f189bfff640 1 --2- 172.21.15.83:0/717092710 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f189c15ba30 0x7f189c072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:20.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.644+0000 7f189affd640 1 -- 172.21.15.83:0/717092710 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1792237028 0 0) 0x7f189c06e320 con 0x7f189c15c960 2024-04-17T10:38:20.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.644+0000 7f189affd640 1 -- 172.21.15.83:0/717092710 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1884003530 con 0x7f189c15c960 2024-04-17T10:38:20.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.644+0000 7f189affd640 1 -- 172.21.15.83:0/717092710 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 650694628 0 0) 0x7f1884003530 con 0x7f189c15c960 2024-04-17T10:38:20.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.644+0000 7f189affd640 1 -- 172.21.15.83:0/717092710 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f189c15ba30 msgr2=0x7f189c072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:20.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.644+0000 7f189affd640 1 --2- 172.21.15.83:0/717092710 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f189c15ba30 0x7f189c072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:20.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.644+0000 7f189affd640 1 -- 172.21.15.83:0/717092710 >> v1:172.21.15.167:6789/0 conn(0x7f189c0cfbe0 legacy=0x7f189c06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:20.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.644+0000 7f189affd640 1 -- 172.21.15.83:0/717092710 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f189c071850 con 0x7f189c15c960 2024-04-17T10:38:20.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.644+0000 7f189affd640 1 -- 172.21.15.83:0/717092710 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f18900016a0 con 0x7f189c15c960 2024-04-17T10:38:20.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.644+0000 7f189affd640 1 -- 172.21.15.83:0/717092710 >> v1:172.21.15.83:6789/0 conn(0x7f189c15c960 legacy=0x7f189c0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:20.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.644+0000 7f189affd640 1 --2- 172.21.15.83:0/717092710 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f189c15ba30 0x7f18840039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:20.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.644+0000 7f189affd640 1 --2- 172.21.15.83:0/717092710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1884003fc0 0x7f1884004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:20.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.644+0000 7f189affd640 1 --2- 172.21.15.83:0/717092710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1884004d50 0x7f18840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:20.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.644+0000 7f189affd640 1 -- 172.21.15.83:0/717092710 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189c071ea0 con 0x7f1884004d50 2024-04-17T10:38:20.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.644+0000 7f189affd640 1 -- 172.21.15.83:0/717092710 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188400bbc0 con 0x7f1884003fc0 2024-04-17T10:38:20.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.644+0000 7f189affd640 1 -- 172.21.15.83:0/717092710 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f188400bdc0 con 0x7f189c15ba30 2024-04-17T10:38:20.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.644+0000 7f18a0b53640 1 --2- 172.21.15.83:0/717092710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1884004d50 0x7f18840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:20.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.644+0000 7f189b7fe640 1 --2- 172.21.15.83:0/717092710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1884003fc0 0x7f1884004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:20.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.644+0000 7f18a0b53640 1 -- 172.21.15.83:0/717092710 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f189c15ba30 msgr2=0x7f18840039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:20.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.644+0000 7f18a0b53640 1 --2- 172.21.15.83:0/717092710 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f189c15ba30 0x7f18840039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:20.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.644+0000 7f18a0b53640 1 -- 172.21.15.83:0/717092710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1884003fc0 msgr2=0x7f1884004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:20.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.644+0000 7f18a0b53640 1 --2- 172.21.15.83:0/717092710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1884003fc0 0x7f1884004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:20.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.644+0000 7f18a0b53640 1 -- 172.21.15.83:0/717092710 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1884003530 con 0x7f1884004d50 2024-04-17T10:38:20.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.648+0000 7f18a0b53640 1 --2- 172.21.15.83:0/717092710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1884004d50 0x7f18840051d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1894001e70 tx=0x7f189401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9279a0551436395c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:20.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.648+0000 7f189affd640 1 -- 172.21.15.83:0/717092710 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1894001a90 con 0x7f1884004d50 2024-04-17T10:38:20.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.648+0000 7f189affd640 1 -- 172.21.15.83:0/717092710 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1894003b80 con 0x7f1884004d50 2024-04-17T10:38:20.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.648+0000 7f189affd640 1 -- 172.21.15.83:0/717092710 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1894005080 con 0x7f1884004d50 2024-04-17T10:38:20.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.648+0000 7f18a25b6640 1 -- 172.21.15.83:0/717092710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1884004d50 msgr2=0x7f18840051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:20.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.648+0000 7f18a25b6640 1 --2- 172.21.15.83:0/717092710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1884004d50 0x7f18840051d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1894001e70 tx=0x7f189401c780 comp rx=0 tx=0).stop 2024-04-17T10:38:20.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.648+0000 7f18a0b53640 1 -- 172.21.15.83:0/717092710 reap_dead start 2024-04-17T10:38:20.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.648+0000 7f18a25b6640 1 -- 172.21.15.83:0/717092710 shutdown_connections 2024-04-17T10:38:20.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.648+0000 7f18a25b6640 1 -- 172.21.15.83:0/717092710 >> 172.21.15.83:0/717092710 conn(0x7f189c06abe0 msgr2=0x7f189c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.648+0000 7f18a25b6640 1 -- 172.21.15.83:0/717092710 shutdown_connections 2024-04-17T10:38:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.648+0000 7f18a25b6640 1 -- 172.21.15.83:0/717092710 wait complete. 2024-04-17T10:38:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.648+0000 7f18a25b6640 1 Processor -- start 2024-04-17T10:38:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.648+0000 7f18a25b6640 1 -- start start 2024-04-17T10:38:20.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.648+0000 7f18a25b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f189c14e280 0x7f189c1f3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:20.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.648+0000 7f18a25b6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f189c14e660 0x7f189c0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:20.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.648+0000 7f18a25b6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f189c0add80 0x7f189c0b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:20.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.648+0000 7f18a25b6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189c0aeae0 con 0x7f189c14e280 2024-04-17T10:38:20.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.648+0000 7f18a25b6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189c15cde0 con 0x7f189c14e660 2024-04-17T10:38:20.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.648+0000 7f18a25b6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f189c06dca0 con 0x7f189c0add80 2024-04-17T10:38:20.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.648+0000 7f189bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f189c14e280 0x7f189c1f3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:20.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.648+0000 7f189bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f189c14e280 0x7f189c1f3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47912/0 (socket says 172.21.15.83:47912) 2024-04-17T10:38:20.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.648+0000 7f189bfff640 1 -- 172.21.15.83:0/3017714707 learned_addr learned my addr 172.21.15.83:0/3017714707 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:20.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.648+0000 7f189b7fe640 1 --2- 172.21.15.83:0/3017714707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f189c14e660 0x7f189c0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:20.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.648+0000 7f18a0b53640 1 --2- 172.21.15.83:0/3017714707 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f189c0add80 0x7f189c0b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:20.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.648+0000 7f189bfff640 1 -- 172.21.15.83:0/3017714707 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f189c0add80 msgr2=0x7f189c0b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:20.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.648+0000 7f189bfff640 1 --2- 172.21.15.83:0/3017714707 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f189c0add80 0x7f189c0b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:20.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.648+0000 7f189bfff640 1 -- 172.21.15.83:0/3017714707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f189c14e660 msgr2=0x7f189c0ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:20.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.648+0000 7f189bfff640 1 --2- 172.21.15.83:0/3017714707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f189c14e660 0x7f189c0ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:20.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.648+0000 7f189bfff640 1 -- 172.21.15.83:0/3017714707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f189c154a10 con 0x7f189c14e280 2024-04-17T10:38:20.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.648+0000 7f18a0b53640 1 --2- 172.21.15.83:0/3017714707 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f189c0add80 0x7f189c0b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:20.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.652+0000 7f189bfff640 1 --2- 172.21.15.83:0/3017714707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f189c14e280 0x7f189c1f3460 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f188c00e7c0 tx=0x7f188c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:20.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.652+0000 7f1898ff9640 1 -- 172.21.15.83:0/3017714707 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c017070 con 0x7f189c14e280 2024-04-17T10:38:20.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.652+0000 7f1898ff9640 1 -- 172.21.15.83:0/3017714707 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188c00c8c0 con 0x7f189c14e280 2024-04-17T10:38:20.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.652+0000 7f18a25b6640 1 -- 172.21.15.83:0/3017714707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f189c154cf0 con 0x7f189c14e280 2024-04-17T10:38:20.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.652+0000 7f1898ff9640 1 -- 172.21.15.83:0/3017714707 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c00cc90 con 0x7f189c14e280 2024-04-17T10:38:20.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.652+0000 7f18a25b6640 1 -- 172.21.15.83:0/3017714707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f189c1552b0 con 0x7f189c14e280 2024-04-17T10:38:20.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.652+0000 7f18a25b6640 1 -- 172.21.15.83:0/3017714707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f189c0ae190 con 0x7f189c14e280 2024-04-17T10:38:20.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.652+0000 7f1898ff9640 1 -- 172.21.15.83:0/3017714707 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f188c01a070 con 0x7f189c14e280 2024-04-17T10:38:20.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.652+0000 7f1898ff9640 1 --2- 172.21.15.83:0/3017714707 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1888066f60 0x7f1888069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:20.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.652+0000 7f1898ff9640 1 -- 172.21.15.83:0/3017714707 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f188c0851c0 con 0x7f189c14e280 2024-04-17T10:38:20.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.652+0000 7f189b7fe640 1 --2- 172.21.15.83:0/3017714707 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1888066f60 0x7f1888069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:20.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.652+0000 7f189b7fe640 1 --2- 172.21.15.83:0/3017714707 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1888066f60 0x7f1888069420 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f189c14eaa0 tx=0x7f1890009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.656+0000 7f1898ff9640 1 -- 172.21.15.83:0/3017714707 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f189c0ae190 con 0x7f189c14e280 2024-04-17T10:38:20.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.860+0000 7f18a25b6640 1 -- 172.21.15.83:0/3017714707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f189c0cf9e0 con 0x7f189c14e280 2024-04-17T10:38:20.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.864+0000 7f1898ff9640 1 -- 172.21.15.83:0/3017714707 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+165733 (secure 0 0 0) 0x7f188c056260 con 0x7f189c14e280 2024-04-17T10:38:20.862 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:20.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.864+0000 7f18a25b6640 1 -- 172.21.15.83:0/3017714707 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1888066f60 msgr2=0x7f1888069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:20.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.868+0000 7f18a25b6640 1 --2- 172.21.15.83:0/3017714707 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1888066f60 0x7f1888069420 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f189c14eaa0 tx=0x7f1890009040 comp rx=0 tx=0).stop 2024-04-17T10:38:20.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.868+0000 7f18a25b6640 1 -- 172.21.15.83:0/3017714707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f189c14e280 msgr2=0x7f189c1f3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:20.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.868+0000 7f18a25b6640 1 --2- 172.21.15.83:0/3017714707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f189c14e280 0x7f189c1f3460 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f188c00e7c0 tx=0x7f188c00ec80 comp rx=0 tx=0).stop 2024-04-17T10:38:20.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.868+0000 7f18a25b6640 1 -- 172.21.15.83:0/3017714707 shutdown_connections 2024-04-17T10:38:20.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.868+0000 7f18a25b6640 1 --2- 172.21.15.83:0/3017714707 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1888066f60 0x7f1888069420 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:20.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.868+0000 7f18a25b6640 1 --2- 172.21.15.83:0/3017714707 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f189c0add80 0x7f189c0b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:20.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.868+0000 7f18a25b6640 1 --2- 172.21.15.83:0/3017714707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f189c14e660 0x7f189c0ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:20.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.868+0000 7f18a25b6640 1 --2- 172.21.15.83:0/3017714707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f189c14e280 0x7f189c1f3460 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:20.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.868+0000 7f18a25b6640 1 -- 172.21.15.83:0/3017714707 >> 172.21.15.83:0/3017714707 conn(0x7f189c06abe0 msgr2=0x7f189c159080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:20.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.868+0000 7f18a25b6640 1 -- 172.21.15.83:0/3017714707 shutdown_connections 2024-04-17T10:38:20.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:20.868+0000 7f18a25b6640 1 -- 172.21.15.83:0/3017714707 wait complete. 2024-04-17T10:38:20.879 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2577814962},{"type":"v1","addr":"172.21.15.83:6833","nonce":2577814962}]},"active_addr":"172.21.15.83:6833/2577814962","active_change":"2024-04-17T10:37:52.163183+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4102,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":3097849083}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":4207601105}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":3686077534}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":1793621929}]}]} 2024-04-17T10:38:20.882 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-17T10:38:20.882 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-17T10:38:20.882 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:38:21.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.004+0000 7f832e67b640 1 Processor -- start 2024-04-17T10:38:21.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.004+0000 7f832e67b640 1 -- start start 2024-04-17T10:38:21.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.008+0000 7f832e67b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f832815bb70 0x7f83280ced00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.008+0000 7f832e67b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83280cce20 0x7f83280cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.008+0000 7f832e67b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f83280cd280 0x7f83280d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.008+0000 7f832e67b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83280cdf40 con 0x7f832815bb70 2024-04-17T10:38:21.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.008+0000 7f832e67b640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83280ce220 con 0x7f83280cd280 2024-04-17T10:38:21.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.008+0000 7f832e67b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83280ce500 con 0x7f83280cce20 2024-04-17T10:38:21.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.008+0000 7f8327fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f832815bb70 0x7f83280ced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.008+0000 7f8327fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f832815bb70 0x7f83280ced00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47916/0 (socket says 172.21.15.83:47916) 2024-04-17T10:38:21.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.008+0000 7f8327fff640 1 -- 172.21.15.83:0/3140581149 learned_addr learned my addr 172.21.15.83:0/3140581149 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:21.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.008+0000 7f832cc18640 1 --2- 172.21.15.83:0/3140581149 >> v2:172.21.15.167:3300/0 conn(0x7f83280cd280 0x7f83280d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.008+0000 7f83277fe640 1 --2- 172.21.15.83:0/3140581149 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83280cce20 0x7f83280cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.008+0000 7f8327fff640 1 -- 172.21.15.83:0/3140581149 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83280cce20 msgr2=0x7f83280cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:21.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.008+0000 7f8327fff640 1 --2- 172.21.15.83:0/3140581149 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83280cce20 0x7f83280cfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.008+0000 7f8327fff640 1 -- 172.21.15.83:0/3140581149 >> v2:172.21.15.167:3300/0 conn(0x7f83280cd280 msgr2=0x7f83280d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:21.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.008+0000 7f8327fff640 1 --2- 172.21.15.83:0/3140581149 >> v2:172.21.15.167:3300/0 conn(0x7f83280cd280 0x7f83280d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.008+0000 7f8327fff640 1 -- 172.21.15.83:0/3140581149 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832815bf50 con 0x7f832815bb70 2024-04-17T10:38:21.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.008+0000 7f832cc18640 1 --2- 172.21.15.83:0/3140581149 >> v2:172.21.15.167:3300/0 conn(0x7f83280cd280 0x7f83280d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:21.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.008+0000 7f8327fff640 1 --2- 172.21.15.83:0/3140581149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f832815bb70 0x7f83280ced00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f83280cda40 tx=0x7f832001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3509d54adcfdc262 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.008+0000 7f8326ffd640 1 -- 172.21.15.83:0/3140581149 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83200a5670 con 0x7f832815bb70 2024-04-17T10:38:21.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.008+0000 7f8326ffd640 1 -- 172.21.15.83:0/3140581149 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83200a5810 con 0x7f832815bb70 2024-04-17T10:38:21.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.008+0000 7f8326ffd640 1 -- 172.21.15.83:0/3140581149 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83200a5be0 con 0x7f832815bb70 2024-04-17T10:38:21.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.008+0000 7f832e67b640 1 -- 172.21.15.83:0/3140581149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f832815bb70 msgr2=0x7f83280ced00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.008+0000 7f832e67b640 1 --2- 172.21.15.83:0/3140581149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f832815bb70 0x7f83280ced00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f83280cda40 tx=0x7f832001c9f0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.008+0000 7f832e67b640 1 -- 172.21.15.83:0/3140581149 shutdown_connections 2024-04-17T10:38:21.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.008+0000 7f832e67b640 1 --2- 172.21.15.83:0/3140581149 >> v2:172.21.15.167:3300/0 conn(0x7f83280cd280 0x7f83280d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.008+0000 7f832e67b640 1 --2- 172.21.15.83:0/3140581149 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83280cce20 0x7f83280cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.008+0000 7f832e67b640 1 --2- 172.21.15.83:0/3140581149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f832815bb70 0x7f83280ced00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.008+0000 7f832e67b640 1 -- 172.21.15.83:0/3140581149 >> 172.21.15.83:0/3140581149 conn(0x7f83280d1770 msgr2=0x7f83280d1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:21.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.012+0000 7f832e67b640 1 -- 172.21.15.83:0/3140581149 shutdown_connections 2024-04-17T10:38:21.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.012+0000 7f832e67b640 1 -- 172.21.15.83:0/3140581149 wait complete. 2024-04-17T10:38:21.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.012+0000 7f832e67b640 1 Processor -- start 2024-04-17T10:38:21.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.012+0000 7f832e67b640 1 -- start start 2024-04-17T10:38:21.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.012+0000 7f832e67b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83280cce20 0x7f83280b6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.012+0000 7f832e67b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83280cd280 0x7f83280caf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.012+0000 7f832e67b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f832815bb70 0x7f8328154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.012+0000 7f832e67b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832806f1f0 con 0x7f83280cd280 2024-04-17T10:38:21.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.012+0000 7f832e67b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832806f070 con 0x7f832815bb70 2024-04-17T10:38:21.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.012+0000 7f832e67b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8328155c90 con 0x7f83280cce20 2024-04-17T10:38:21.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.012+0000 7f83277fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83280cd280 0x7f83280caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.012+0000 7f83277fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83280cd280 0x7f83280caf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47928/0 (socket says 172.21.15.83:47928) 2024-04-17T10:38:21.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.012+0000 7f83277fe640 1 -- 172.21.15.83:0/4037500114 learned_addr learned my addr 172.21.15.83:0/4037500114 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:21.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.012+0000 7f832cc18640 1 --2- 172.21.15.83:0/4037500114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f832815bb70 0x7f8328154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.012+0000 7f83277fe640 1 -- 172.21.15.83:0/4037500114 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83280cce20 msgr2=0x7f83280b6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.012+0000 7f83277fe640 1 --2- 172.21.15.83:0/4037500114 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83280cce20 0x7f83280b6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.012+0000 7f83277fe640 1 -- 172.21.15.83:0/4037500114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f832815bb70 msgr2=0x7f8328154b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.012+0000 7f83277fe640 1 --2- 172.21.15.83:0/4037500114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f832815bb70 0x7f8328154b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.012+0000 7f83277fe640 1 -- 172.21.15.83:0/4037500114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8328155f70 con 0x7f83280cd280 2024-04-17T10:38:21.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.012+0000 7f83277fe640 1 --2- 172.21.15.83:0/4037500114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83280cd280 0x7f83280caf20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f831000c570 tx=0x7f831000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.012+0000 7f8324ff9640 1 -- 172.21.15.83:0/4037500114 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8310015070 con 0x7f83280cd280 2024-04-17T10:38:21.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.012+0000 7f8324ff9640 1 -- 172.21.15.83:0/4037500114 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8310011800 con 0x7f83280cd280 2024-04-17T10:38:21.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.012+0000 7f8324ff9640 1 -- 172.21.15.83:0/4037500114 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8310011bf0 con 0x7f83280cd280 2024-04-17T10:38:21.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.012+0000 7f832e67b640 1 -- 172.21.15.83:0/4037500114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8328156250 con 0x7f83280cd280 2024-04-17T10:38:21.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.012+0000 7f832e67b640 1 -- 172.21.15.83:0/4037500114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8328156810 con 0x7f83280cd280 2024-04-17T10:38:21.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.012+0000 7f832e67b640 1 -- 172.21.15.83:0/4037500114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8328156a50 con 0x7f83280cd280 2024-04-17T10:38:21.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.016+0000 7f8324ff9640 1 -- 172.21.15.83:0/4037500114 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8310018070 con 0x7f83280cd280 2024-04-17T10:38:21.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.016+0000 7f8324ff9640 1 --2- 172.21.15.83:0/4037500114 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8308066f90 0x7f8308069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.016+0000 7f8324ff9640 1 -- 172.21.15.83:0/4037500114 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8310084e00 con 0x7f83280cd280 2024-04-17T10:38:21.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.016+0000 7f8327fff640 1 --2- 172.21.15.83:0/4037500114 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8308066f90 0x7f8308069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.016+0000 7f8327fff640 1 --2- 172.21.15.83:0/4037500114 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8308066f90 0x7f8308069450 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f83200ae000 tx=0x7f8320004ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.020+0000 7f8324ff9640 1 -- 172.21.15.83:0/4037500114 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8328156a50 con 0x7f83280cd280 2024-04-17T10:38:21.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.168+0000 7f832e67b640 1 -- 172.21.15.83:0/4037500114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8328074f90 con 0x7f83280cd280 2024-04-17T10:38:21.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.168+0000 7f8324ff9640 1 -- 172.21.15.83:0/4037500114 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16533 (secure 0 0 0) 0x7f8310055e10 con 0x7f83280cd280 2024-04-17T10:38:21.166 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:21.167 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":20,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:38:13.690876+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:38:10.464490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T10:38:11.043134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:38:21.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.172+0000 7f832e67b640 1 -- 172.21.15.83:0/4037500114 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8308066f90 msgr2=0x7f8308069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.172+0000 7f832e67b640 1 --2- 172.21.15.83:0/4037500114 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8308066f90 0x7f8308069450 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f83200ae000 tx=0x7f8320004ac0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.172+0000 7f832e67b640 1 -- 172.21.15.83:0/4037500114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83280cd280 msgr2=0x7f83280caf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.172+0000 7f832e67b640 1 --2- 172.21.15.83:0/4037500114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83280cd280 0x7f83280caf20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f831000c570 tx=0x7f831000ca30 comp rx=0 tx=0).stop 2024-04-17T10:38:21.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.172+0000 7f832e67b640 1 -- 172.21.15.83:0/4037500114 shutdown_connections 2024-04-17T10:38:21.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.172+0000 7f832e67b640 1 --2- 172.21.15.83:0/4037500114 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8308066f90 0x7f8308069450 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.172+0000 7f832e67b640 1 --2- 172.21.15.83:0/4037500114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f832815bb70 0x7f8328154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.172+0000 7f832e67b640 1 --2- 172.21.15.83:0/4037500114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83280cd280 0x7f83280caf20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.172+0000 7f832e67b640 1 --2- 172.21.15.83:0/4037500114 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83280cce20 0x7f83280b6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.172+0000 7f832e67b640 1 -- 172.21.15.83:0/4037500114 >> 172.21.15.83:0/4037500114 conn(0x7f83280d1770 msgr2=0x7f8328157e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:21.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.172+0000 7f832e67b640 1 -- 172.21.15.83:0/4037500114 shutdown_connections 2024-04-17T10:38:21.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.172+0000 7f832e67b640 1 -- 172.21.15.83:0/4037500114 wait complete. 2024-04-17T10:38:21.183 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-17T10:38:21.184 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:38:21.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.304+0000 7f8c039c4640 1 Processor -- start 2024-04-17T10:38:21.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.304+0000 7f8c039c4640 1 -- start start 2024-04-17T10:38:21.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.304+0000 7f8c039c4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8bfc06c300 0x7f8bfc166a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.304+0000 7f8c039c4640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bfc06e980 con 0x7f8bfc15eeb0 2024-04-17T10:38:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.304+0000 7f8c039c4640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bfc06e0e0 con 0x7f8bfc0cf780 2024-04-17T10:38:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.304+0000 7f8c039c4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bfc0cf030 con 0x7f8bfc06c300 2024-04-17T10:38:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.304+0000 7f8c01760640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8bfc15eeb0 0x7f8bfc06d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39838/0 (socket says 172.21.15.83:39838) 2024-04-17T10:38:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.304+0000 7f8c01760640 1 -- 172.21.15.83:0/2609092977 learned_addr learned my addr 172.21.15.83:0/2609092977 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.304+0000 7f8c01f61640 1 --2- 172.21.15.83:0/2609092977 >> v2:172.21.15.83:3300/0 conn(0x7f8bfc06c300 0x7f8bfc166a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.304+0000 7f8bf3fff640 1 -- 172.21.15.83:0/2609092977 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3202053629 0 0) 0x7f8bfc06e980 con 0x7f8bfc15eeb0 2024-04-17T10:38:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.304+0000 7f8bf3fff640 1 -- 172.21.15.83:0/2609092977 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8be8003530 con 0x7f8bfc15eeb0 2024-04-17T10:38:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.304+0000 7f8c01f61640 1 -- 172.21.15.83:0/2609092977 >> v1:172.21.15.167:6789/0 conn(0x7f8bfc0cf780 legacy=0x7f8bfc06bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.304+0000 7f8c01f61640 1 -- 172.21.15.83:0/2609092977 >> v1:172.21.15.83:6789/0 conn(0x7f8bfc15eeb0 legacy=0x7f8bfc06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.304+0000 7f8c01f61640 1 -- 172.21.15.83:0/2609092977 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bfc06e3c0 con 0x7f8bfc06c300 2024-04-17T10:38:21.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.308+0000 7f8c01f61640 1 --2- 172.21.15.83:0/2609092977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc06c300 0x7f8bfc166a70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009d90 tx=0x7f8bf401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0ce25bdd8fd83a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.308+0000 7f8bf3fff640 1 -- 172.21.15.83:0/2609092977 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf40b2070 con 0x7f8bfc06c300 2024-04-17T10:38:21.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.308+0000 7f8bf3fff640 1 -- 172.21.15.83:0/2609092977 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf40a5940 con 0x7f8bfc06c300 2024-04-17T10:38:21.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.308+0000 7f8bf3fff640 1 -- 172.21.15.83:0/2609092977 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf40a5d10 con 0x7f8bfc06c300 2024-04-17T10:38:21.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.308+0000 7f8c039c4640 1 -- 172.21.15.83:0/2609092977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc06c300 msgr2=0x7f8bfc166a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.308+0000 7f8c039c4640 1 --2- 172.21.15.83:0/2609092977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc06c300 0x7f8bfc166a70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009d90 tx=0x7f8bf401c970 comp rx=0 tx=0).stop 2024-04-17T10:38:21.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.308+0000 7f8c039c4640 1 -- 172.21.15.83:0/2609092977 shutdown_connections 2024-04-17T10:38:21.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.308+0000 7f8c039c4640 1 --2- 172.21.15.83:0/2609092977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc06c300 0x7f8bfc166a70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.308+0000 7f8c039c4640 1 -- 172.21.15.83:0/2609092977 >> 172.21.15.83:0/2609092977 conn(0x7f8bfc0cc910 msgr2=0x7f8bfc0ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:21.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.308+0000 7f8c039c4640 1 -- 172.21.15.83:0/2609092977 shutdown_connections 2024-04-17T10:38:21.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.308+0000 7f8c039c4640 1 -- 172.21.15.83:0/2609092977 wait complete. 2024-04-17T10:38:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.308+0000 7f8c039c4640 1 Processor -- start 2024-04-17T10:38:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.308+0000 7f8c039c4640 1 -- start start 2024-04-17T10:38:21.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.308+0000 7f8c039c4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc06c300 0x7f8bfc0b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.308+0000 7f8c039c4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bfc0cf780 0x7f8bfc15bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.308+0000 7f8c039c4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8bfc15eeb0 0x7f8bfc15c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.308+0000 7f8c039c4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc169950 con 0x7f8bfc06c300 2024-04-17T10:38:21.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.308+0000 7f8c039c4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc1697d0 con 0x7f8bfc0cf780 2024-04-17T10:38:21.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.308+0000 7f8c039c4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc169ad0 con 0x7f8bfc15eeb0 2024-04-17T10:38:21.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.308+0000 7f8c01760640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc06c300 0x7f8bfc0b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.308+0000 7f8c01760640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc06c300 0x7f8bfc0b11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47946/0 (socket says 172.21.15.83:47946) 2024-04-17T10:38:21.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.308+0000 7f8c01760640 1 -- 172.21.15.83:0/453601964 learned_addr learned my addr 172.21.15.83:0/453601964 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:21.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.308+0000 7f8c00f5f640 1 --2- 172.21.15.83:0/453601964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bfc0cf780 0x7f8bfc15bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.308+0000 7f8c01f61640 1 --2- 172.21.15.83:0/453601964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8bfc15eeb0 0x7f8bfc15c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.308+0000 7f8c01760640 1 -- 172.21.15.83:0/453601964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8bfc15eeb0 msgr2=0x7f8bfc15c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.308+0000 7f8c01760640 1 --2- 172.21.15.83:0/453601964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8bfc15eeb0 0x7f8bfc15c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.308+0000 7f8c01760640 1 -- 172.21.15.83:0/453601964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bfc0cf780 msgr2=0x7f8bfc15bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.308+0000 7f8c01760640 1 --2- 172.21.15.83:0/453601964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bfc0cf780 0x7f8bfc15bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.308+0000 7f8c01760640 1 -- 172.21.15.83:0/453601964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bfc09f450 con 0x7f8bfc06c300 2024-04-17T10:38:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.308+0000 7f8c00f5f640 1 --2- 172.21.15.83:0/453601964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bfc0cf780 0x7f8bfc15bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.308+0000 7f8c01760640 1 --2- 172.21.15.83:0/453601964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc06c300 0x7f8bfc0b11c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800c420 tx=0x7f8bf800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.312+0000 7f8bf1ffb640 1 -- 172.21.15.83:0/453601964 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8017070 con 0x7f8bfc06c300 2024-04-17T10:38:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.312+0000 7f8bf1ffb640 1 -- 172.21.15.83:0/453601964 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf8005cf0 con 0x7f8bfc06c300 2024-04-17T10:38:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.312+0000 7f8c039c4640 1 -- 172.21.15.83:0/453601964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bfc157e80 con 0x7f8bfc06c300 2024-04-17T10:38:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.312+0000 7f8bf1ffb640 1 -- 172.21.15.83:0/453601964 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf80072d0 con 0x7f8bfc06c300 2024-04-17T10:38:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.312+0000 7f8c039c4640 1 -- 172.21.15.83:0/453601964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bfc0a0980 con 0x7f8bfc06c300 2024-04-17T10:38:21.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.312+0000 7f8c039c4640 1 -- 172.21.15.83:0/453601964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bfc074f90 con 0x7f8bfc06c300 2024-04-17T10:38:21.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.312+0000 7f8bf1ffb640 1 -- 172.21.15.83:0/453601964 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8bf801a070 con 0x7f8bfc06c300 2024-04-17T10:38:21.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.312+0000 7f8bf1ffb640 1 --2- 172.21.15.83:0/453601964 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8be0066f60 0x7f8be0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.312+0000 7f8c00f5f640 1 --2- 172.21.15.83:0/453601964 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8be0066f60 0x7f8be0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.312+0000 7f8bf1ffb640 1 -- 172.21.15.83:0/453601964 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8bf8084290 con 0x7f8bfc06c300 2024-04-17T10:38:21.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.312+0000 7f8c00f5f640 1 --2- 172.21.15.83:0/453601964 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8be0066f60 0x7f8be0069420 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc06eda0 tx=0x7f8bec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.316+0000 7f8bf1ffb640 1 -- 172.21.15.83:0/453601964 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8bfc09f450 con 0x7f8bfc06c300 2024-04-17T10:38:21.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.472+0000 7f8c039c4640 1 -- 172.21.15.83:0/453601964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8bfc068150 con 0x7f8bfc06c300 2024-04-17T10:38:21.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.472+0000 7f8bf1ffb640 1 -- 172.21.15.83:0/453601964 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16533 (secure 0 0 0) 0x7f8bf80810b0 con 0x7f8bfc06c300 2024-04-17T10:38:21.471 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:21.471 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":20,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:38:13.690876+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:38:10.464490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T10:38:11.043134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:38:21.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.476+0000 7f8c039c4640 1 -- 172.21.15.83:0/453601964 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8be0066f60 msgr2=0x7f8be0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.476+0000 7f8c039c4640 1 --2- 172.21.15.83:0/453601964 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8be0066f60 0x7f8be0069420 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc06eda0 tx=0x7f8bec009040 comp rx=0 tx=0).stop 2024-04-17T10:38:21.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.476+0000 7f8c039c4640 1 -- 172.21.15.83:0/453601964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc06c300 msgr2=0x7f8bfc0b11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.476+0000 7f8c039c4640 1 --2- 172.21.15.83:0/453601964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc06c300 0x7f8bfc0b11c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800c420 tx=0x7f8bf800c8e0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.476+0000 7f8c039c4640 1 -- 172.21.15.83:0/453601964 shutdown_connections 2024-04-17T10:38:21.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.476+0000 7f8c039c4640 1 --2- 172.21.15.83:0/453601964 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8be0066f60 0x7f8be0069420 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.476+0000 7f8c039c4640 1 --2- 172.21.15.83:0/453601964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8bfc15eeb0 0x7f8bfc15c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.476+0000 7f8c039c4640 1 --2- 172.21.15.83:0/453601964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bfc0cf780 0x7f8bfc15bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.476+0000 7f8c039c4640 1 --2- 172.21.15.83:0/453601964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc06c300 0x7f8bfc0b11c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.476+0000 7f8c039c4640 1 -- 172.21.15.83:0/453601964 >> 172.21.15.83:0/453601964 conn(0x7f8bfc0cc910 msgr2=0x7f8bfc09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:21.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.476+0000 7f8c039c4640 1 -- 172.21.15.83:0/453601964 shutdown_connections 2024-04-17T10:38:21.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.476+0000 7f8c039c4640 1 -- 172.21.15.83:0/453601964 wait complete. 2024-04-17T10:38:21.488 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-17T10:38:21.489 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-17T10:38:21.489 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-17T10:38:21.489 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-17T10:38:21.490 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-17T10:38:21.490 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-17T10:38:21.491 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-17T10:38:21.491 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-17T10:38:21.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.668+0000 7f08e7fff640 1 Processor -- start 2024-04-17T10:38:21.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.668+0000 7f08e7fff640 1 -- start start 2024-04-17T10:38:21.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.668+0000 7f08e7fff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f08e8150760 0x7f08e815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.668+0000 7f08e7fff640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08e80b2910 con 0x7f08e8150bc0 2024-04-17T10:38:21.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.668+0000 7f08e7fff640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08e80b2bf0 con 0x7f08e806b700 2024-04-17T10:38:21.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.668+0000 7f08e7fff640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08e80b3520 con 0x7f08e8150760 2024-04-17T10:38:21.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f08e77fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f08e8150bc0 0x7f08e8071a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39842/0 (socket says 172.21.15.83:39842) 2024-04-17T10:38:21.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f08e77fe640 1 -- 172.21.15.83:0/2294871384 learned_addr learned my addr 172.21.15.83:0/2294871384 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:21.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f08e67fc640 1 --2- 172.21.15.83:0/2294871384 >> v2:172.21.15.167:3300/0 conn(0x7f08e8150760 0x7f08e815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f08e5ffb640 1 -- 172.21.15.83:0/2294871384 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1687224006 0 0) 0x7f08e80b2bf0 con 0x7f08e806b700 2024-04-17T10:38:21.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f08e5ffb640 1 -- 172.21.15.83:0/2294871384 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08d0003530 con 0x7f08e806b700 2024-04-17T10:38:21.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f08e5ffb640 1 -- 172.21.15.83:0/2294871384 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1224471285 0 0) 0x7f08e80b2910 con 0x7f08e8150bc0 2024-04-17T10:38:21.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f08e5ffb640 1 -- 172.21.15.83:0/2294871384 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08e80b2bf0 con 0x7f08e8150bc0 2024-04-17T10:38:21.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f08e67fc640 1 -- 172.21.15.83:0/2294871384 >> v1:172.21.15.167:6789/0 conn(0x7f08e806b700 legacy=0x7f08e80b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f08e67fc640 1 -- 172.21.15.83:0/2294871384 >> v1:172.21.15.83:6789/0 conn(0x7f08e8150bc0 legacy=0x7f08e8071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f08e67fc640 1 -- 172.21.15.83:0/2294871384 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e8161850 con 0x7f08e8150760 2024-04-17T10:38:21.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f4e88fda640 1 Processor -- start 2024-04-17T10:38:21.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f4e88fda640 1 -- start start 2024-04-17T10:38:21.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f7592ddf640 1 Processor -- start 2024-04-17T10:38:21.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f082359e640 1 Processor -- start 2024-04-17T10:38:21.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f7592ddf640 1 -- start start 2024-04-17T10:38:21.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f082359e640 1 -- start start 2024-04-17T10:38:21.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f4e88fda640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4e84143c10 0x7f4e84070230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f7592ddf640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f75840df770 0x7f7584006730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f7592ddf640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7584053620 0x7f7584006c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f7592ddf640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7584053cc0 0x7f7584054150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f7592ddf640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7584050b10 con 0x7f7584053cc0 2024-04-17T10:38:21.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f7592ddf640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7584052610 con 0x7f75840df770 2024-04-17T10:38:21.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f7592ddf640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75840dfb50 con 0x7f7584053620 2024-04-17T10:38:21.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f75915dc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7584053620 0x7f7584006c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f75915dc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7584053620 0x7f7584006c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:59534/0 (socket says 172.21.15.83:59534) 2024-04-17T10:38:21.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f75915dc640 1 -- 172.21.15.83:0/3200188210 learned_addr learned my addr 172.21.15.83:0/3200188210 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:21.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7fecaebef640 1 Processor -- start 2024-04-17T10:38:21.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7fecaebef640 1 -- start start 2024-04-17T10:38:21.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f4e88fda640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e84143ff0 0x7f4e840ad840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f4e88fda640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e8406daf0 con 0x7f4e840add80 2024-04-17T10:38:21.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f4e88fda640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e840b49f0 con 0x7f4e84143c10 2024-04-17T10:38:21.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f4e88fda640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8406c480 con 0x7f4e84143ff0 2024-04-17T10:38:21.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f4e837fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4e84143c10 0x7f4e84070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f4e837fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4e84143c10 0x7f4e84070230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47960/0 (socket says 172.21.15.83:47960) 2024-04-17T10:38:21.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f4e837fe640 1 -- 172.21.15.83:0/3481815980 learned_addr learned my addr 172.21.15.83:0/3481815980 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:21.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f5b04f53640 1 Processor -- start 2024-04-17T10:38:21.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f082359e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f081c00aae0 0x7f081c0df050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f082359e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f081c0501b0 0x7f081c006540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f082359e640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081c054400 con 0x7f081c033d00 2024-04-17T10:38:21.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f082359e640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f081c0546e0 con 0x7f081c0501b0 2024-04-17T10:38:21.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f082359e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f081c006020 con 0x7f081c00aae0 2024-04-17T10:38:21.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f7591ddd640 1 --2- 172.21.15.83:0/3200188210 >> v2:172.21.15.167:3300/0 conn(0x7f75840df770 0x7f7584006730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f75925de640 1 --2- 172.21.15.83:0/3200188210 >> v2:172.21.15.83:3300/0 conn(0x7f7584053cc0 0x7f7584054150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f4e82ffd640 1 --2- 172.21.15.83:0/3481815980 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e84143ff0 0x7f4e840ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f5b04f53640 1 -- start start 2024-04-17T10:38:21.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f0821d9b640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f081c033d00 0x7f081c04faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:52150/0 (socket says 172.21.15.83:52150) 2024-04-17T10:38:21.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f0821d9b640 1 -- 172.21.15.83:0/1434895792 learned_addr learned my addr 172.21.15.83:0/1434895792 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:21.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f082259c640 1 --2- 172.21.15.83:0/1434895792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f081c00aae0 0x7f081c0df050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f0822d9d640 1 --2- 172.21.15.83:0/1434895792 >> v2:172.21.15.167:3300/0 conn(0x7f081c0501b0 0x7f081c006540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f4563743640 1 Processor -- start 2024-04-17T10:38:21.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f4563743640 1 -- start start 2024-04-17T10:38:21.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.672+0000 7f75925de640 1 -- 172.21.15.83:0/3200188210 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7584053620 msgr2=0x7f7584006c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:21.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f75925de640 1 --2- 172.21.15.83:0/3200188210 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7584053620 0x7f7584006c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f75925de640 1 -- 172.21.15.83:0/3200188210 >> v2:172.21.15.167:3300/0 conn(0x7f75840df770 msgr2=0x7f7584006730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:21.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f75925de640 1 --2- 172.21.15.83:0/3200188210 >> v2:172.21.15.167:3300/0 conn(0x7f75840df770 0x7f7584006730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f75925de640 1 -- 172.21.15.83:0/3200188210 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7584052860 con 0x7f7584053cc0 2024-04-17T10:38:21.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f24c351c640 1 Processor -- start 2024-04-17T10:38:21.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7fecaebef640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7feca815bc10 0x7feca80d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7fecaebef640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feca806e500 0x7feca806e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7fecaebef640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feca80ced00 con 0x7feca80d0490 2024-04-17T10:38:21.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7fecaebef640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feca80cefe0 con 0x7feca806e500 2024-04-17T10:38:21.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7fecaebef640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feca80cc690 con 0x7feca815bc10 2024-04-17T10:38:21.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f082159a640 1 -- 172.21.15.83:0/1434895792 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2216055968 0 0) 0x7f081c054400 con 0x7f081c033d00 2024-04-17T10:38:21.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f082159a640 1 -- 172.21.15.83:0/1434895792 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f080c003530 con 0x7f081c033d00 2024-04-17T10:38:21.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f0822d9d640 1 -- 172.21.15.83:0/1434895792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f081c00aae0 msgr2=0x7f081c0df050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:21.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f0822d9d640 1 --2- 172.21.15.83:0/1434895792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f081c00aae0 0x7f081c0df050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f0822d9d640 1 -- 172.21.15.83:0/1434895792 >> v1:172.21.15.167:6789/0 conn(0x7f081c033d00 legacy=0x7f081c04faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f0822d9d640 1 -- 172.21.15.83:0/1434895792 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081c0342e0 con 0x7f081c0501b0 2024-04-17T10:38:21.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f24c351c640 1 -- start start 2024-04-17T10:38:21.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f08e67fc640 1 --2- 172.21.15.83:0/2294871384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08e8150760 0x7f08e815d4c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f08e0009d90 tx=0x7f08e001c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f83b9ec4c37e84b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7fecac98b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7feca815bc10 0x7feca80d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7fecac98b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7feca815bc10 0x7feca80d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:36138/0 (socket says 172.21.15.83:36138) 2024-04-17T10:38:21.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7fecac98b640 1 -- 172.21.15.83:0/3743687822 learned_addr learned my addr 172.21.15.83:0/3743687822 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:21.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7feca7fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7feca80d0490 0x7feca80cce90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39858/0 (socket says 172.21.15.83:39858) 2024-04-17T10:38:21.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f5b04f53640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5b000b3900 0x7f5b0015efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f5b04f53640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b000af140 con 0x7f5b000aed60 2024-04-17T10:38:21.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f5b04f53640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b00071300 con 0x7f5b00072460 2024-04-17T10:38:21.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f5b04f53640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b000715e0 con 0x7f5b000b3900 2024-04-17T10:38:21.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f5afe59c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5b000aed60 0x7f5b000b31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39870/0 (socket says 172.21.15.83:39870) 2024-04-17T10:38:21.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f5afe59c640 1 -- 172.21.15.83:0/1623006238 learned_addr learned my addr 172.21.15.83:0/1623006238 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:21.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f08e5ffb640 1 -- 172.21.15.83:0/2294871384 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e00b2070 con 0x7f08e8150760 2024-04-17T10:38:21.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f08e5ffb640 1 -- 172.21.15.83:0/2294871384 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e00a5900 con 0x7f08e8150760 2024-04-17T10:38:21.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f08e5ffb640 1 -- 172.21.15.83:0/2294871384 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e00a5cd0 con 0x7f08e8150760 2024-04-17T10:38:21.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f4563743640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f455c155e60 0x7f455c070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f4563743640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f455c15ff20 0x7f455c0cde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f4563743640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f455c071e70 con 0x7f455c15f3b0 2024-04-17T10:38:21.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f4563743640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f455c0cd710 con 0x7f455c155e60 2024-04-17T10:38:21.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f4563743640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455c156240 con 0x7f455c15ff20 2024-04-17T10:38:21.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f5afdd9b640 1 --2- 172.21.15.83:0/1623006238 >> v2:172.21.15.167:3300/0 conn(0x7f5b000b3900 0x7f5b0015efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f4560cde640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f455c15f3b0 0x7f455c15f810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39882/0 (socket says 172.21.15.83:39882) 2024-04-17T10:38:21.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f4560cde640 1 -- 172.21.15.83:0/3973456889 learned_addr learned my addr 172.21.15.83:0/3973456889 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:21.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f4561ce0640 1 --2- 172.21.15.83:0/3973456889 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f455c15ff20 0x7f455c0cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f5afdd9b640 1 -- 172.21.15.83:0/1623006238 >> v1:172.21.15.167:6789/0 conn(0x7f5b00072460 legacy=0x7f5b0015f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f5afdd9b640 1 -- 172.21.15.83:0/1623006238 >> v1:172.21.15.83:6789/0 conn(0x7f5b000aed60 legacy=0x7f5b000b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f5afdd9b640 1 -- 172.21.15.83:0/1623006238 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b000718c0 con 0x7f5b000b3900 2024-04-17T10:38:21.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f24c351c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f24b4047c20 0x7f24b40eb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f24c351c640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24b4034220 con 0x7f24b4033e40 2024-04-17T10:38:21.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f24c351c640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24b404fa30 con 0x7f24b4005ea0 2024-04-17T10:38:21.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f24c351c640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24b4050210 con 0x7f24b4047c20 2024-04-17T10:38:21.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7fecac98b640 1 -- 172.21.15.83:0/3743687822 >> v2:172.21.15.83:3300/0 conn(0x7feca806e500 msgr2=0x7feca806e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:21.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7fecac98b640 1 --2- 172.21.15.83:0/3743687822 >> v2:172.21.15.83:3300/0 conn(0x7feca806e500 0x7feca806e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7fecac98b640 1 -- 172.21.15.83:0/3743687822 >> v1:172.21.15.83:6789/0 conn(0x7feca80d0490 legacy=0x7feca80cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7fecac98b640 1 -- 172.21.15.83:0/3743687822 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca80cc970 con 0x7feca815bc10 2024-04-17T10:38:21.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f4561ce0640 1 -- 172.21.15.83:0/3973456889 >> v2:172.21.15.83:3300/0 conn(0x7f455c155e60 msgr2=0x7f455c070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:21.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f4561ce0640 1 --2- 172.21.15.83:0/3973456889 >> v2:172.21.15.83:3300/0 conn(0x7f455c155e60 0x7f455c070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f4561ce0640 1 -- 172.21.15.83:0/3973456889 >> v1:172.21.15.83:6789/0 conn(0x7f455c15f3b0 legacy=0x7f455c15f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f4561ce0640 1 -- 172.21.15.83:0/3973456889 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455c0715d0 con 0x7f455c15ff20 2024-04-17T10:38:21.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f24c251a640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f24b4033e40 0x7f24b40eb180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39890/0 (socket says 172.21.15.83:39890) 2024-04-17T10:38:21.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f24c251a640 1 -- 172.21.15.83:0/799217591 learned_addr learned my addr 172.21.15.83:0/799217591 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:21.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f24c1518640 1 -- 172.21.15.83:0/799217591 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3945355026 0 0) 0x7f24b4034220 con 0x7f24b4033e40 2024-04-17T10:38:21.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f24c1518640 1 -- 172.21.15.83:0/799217591 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24a8003530 con 0x7f24b4033e40 2024-04-17T10:38:21.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f082259c640 1 --2- 172.21.15.83:0/1434895792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f081c00aae0 0x7f081c0df050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:38:21.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f4e837fe640 1 -- 172.21.15.83:0/3481815980 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e84143ff0 msgr2=0x7f4e840ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:21.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f4e837fe640 1 --2- 172.21.15.83:0/3481815980 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e84143ff0 0x7f4e840ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f4e837fe640 1 -- 172.21.15.83:0/3481815980 >> v1:172.21.15.83:6789/0 conn(0x7f4e840add80 legacy=0x7f4e840b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f4e837fe640 1 -- 172.21.15.83:0/3481815980 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8406c150 con 0x7f4e84143c10 2024-04-17T10:38:21.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f4e827fc640 1 -- 172.21.15.83:0/3481815980 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3705236286 0 0) 0x7f4e8406daf0 con 0x7f4e840add80 2024-04-17T10:38:21.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f4e837fe640 1 --2- 172.21.15.83:0/3481815980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e84143c10 0x7f4e84070230 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4e8406c600 tx=0x7f4e7c024c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc94c6d5a4c3ca7d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f75925de640 1 --2- 172.21.15.83:0/3200188210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7584053cc0 0x7f7584054150 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7584052540 tx=0x7f758801ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6bfa53ae50bca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f24c1518640 1 -- 172.21.15.83:0/799217591 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2870658960 0 0) 0x7f24a8003530 con 0x7f24b4033e40 2024-04-17T10:38:21.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f24c1518640 1 -- 172.21.15.83:0/799217591 >> v2:172.21.15.167:3300/0 conn(0x7f24b4047c20 msgr2=0x7f24b40eb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:21.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f24c1518640 1 --2- 172.21.15.83:0/799217591 >> v2:172.21.15.167:3300/0 conn(0x7f24b4047c20 0x7f24b40eb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f4e827fc640 1 -- 172.21.15.83:0/3481815980 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c0adc10 con 0x7f4e84143c10 2024-04-17T10:38:21.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f4e827fc640 1 -- 172.21.15.83:0/3481815980 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7c0addb0 con 0x7f4e84143c10 2024-04-17T10:38:21.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f4e827fc640 1 -- 172.21.15.83:0/3481815980 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c0112f0 con 0x7f4e84143c10 2024-04-17T10:38:21.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f08e7fff640 1 -- 172.21.15.83:0/2294871384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08e8150760 msgr2=0x7f08e815d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f08e7fff640 1 --2- 172.21.15.83:0/2294871384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08e8150760 0x7f08e815d4c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f08e0009d90 tx=0x7f08e001c930 comp rx=0 tx=0).stop 2024-04-17T10:38:21.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.676+0000 7f0822d9d640 1 --2- 172.21.15.83:0/1434895792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f081c0501b0 0x7f081c006540 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f08140097c0 tx=0x7f081401c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a5b38064c0ae8a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f24c1518640 1 -- 172.21.15.83:0/799217591 >> v1:172.21.15.167:6789/0 conn(0x7f24b4005ea0 legacy=0x7f24b40062b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f24c1518640 1 -- 172.21.15.83:0/799217591 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b404f2e0 con 0x7f24b4033e40 2024-04-17T10:38:21.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f08e7fff640 1 -- 172.21.15.83:0/2294871384 shutdown_connections 2024-04-17T10:38:21.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f08e7fff640 1 --2- 172.21.15.83:0/2294871384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08e8150760 0x7f08e815d4c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f08e7fff640 1 -- 172.21.15.83:0/2294871384 >> 172.21.15.83:0/2294871384 conn(0x7f08e806abe0 msgr2=0x7f08e806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:21.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f24c1518640 1 -- 172.21.15.83:0/799217591 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f24b8001680 con 0x7f24b4033e40 2024-04-17T10:38:21.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f08e7fff640 1 -- 172.21.15.83:0/2294871384 shutdown_connections 2024-04-17T10:38:21.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f24c1518640 1 -- 172.21.15.83:0/799217591 >> v1:172.21.15.83:6789/0 conn(0x7f24b4033e40 legacy=0x7f24b40eb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f24c1518640 1 --2- 172.21.15.83:0/799217591 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a80039c0 0x7f24a8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f7590ddb640 1 -- 172.21.15.83:0/3200188210 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75880b5070 con 0x7f7584053cc0 2024-04-17T10:38:21.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f24c1518640 1 --2- 172.21.15.83:0/799217591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24a8004810 0x7f24a8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f7590ddb640 1 -- 172.21.15.83:0/3200188210 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75880a8d00 con 0x7f7584053cc0 2024-04-17T10:38:21.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f7590ddb640 1 -- 172.21.15.83:0/3200188210 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758800b040 con 0x7f7584053cc0 2024-04-17T10:38:21.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f24c1518640 1 --2- 172.21.15.83:0/799217591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24a80051d0 0x7f24a800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f24c1518640 1 -- 172.21.15.83:0/799217591 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b4050210 con 0x7f24a80051d0 2024-04-17T10:38:21.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f24c1518640 1 -- 172.21.15.83:0/799217591 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a800c080 con 0x7f24a8004810 2024-04-17T10:38:21.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f24c1518640 1 -- 172.21.15.83:0/799217591 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a800c280 con 0x7f24a80039c0 2024-04-17T10:38:21.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7fecac98b640 1 --2- 172.21.15.83:0/3743687822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feca815bc10 0x7feca80d2300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7feca80678a0 tx=0x7fec9c01c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e5f20a95539d36 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7feca77fe640 1 -- 172.21.15.83:0/3743687822 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9c0b0070 con 0x7feca815bc10 2024-04-17T10:38:21.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7feca77fe640 1 -- 172.21.15.83:0/3743687822 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec9c0ab600 con 0x7feca815bc10 2024-04-17T10:38:21.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7feca77fe640 1 -- 172.21.15.83:0/3743687822 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9c0ab9d0 con 0x7feca815bc10 2024-04-17T10:38:21.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f5afdd9b640 1 --2- 172.21.15.83:0/1623006238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b000b3900 0x7f5b0015efb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5af00097c0 tx=0x7f5af001c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca4b485bffcd5e96 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f4e88fda640 1 -- 172.21.15.83:0/3481815980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e84143c10 msgr2=0x7f4e84070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f4e88fda640 1 --2- 172.21.15.83:0/3481815980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e84143c10 0x7f4e84070230 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4e8406c600 tx=0x7f4e7c024c50 comp rx=0 tx=0).stop 2024-04-17T10:38:21.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f4e88fda640 1 -- 172.21.15.83:0/3481815980 shutdown_connections 2024-04-17T10:38:21.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f4e88fda640 1 --2- 172.21.15.83:0/3481815980 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e84143ff0 0x7f4e840ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f4e88fda640 1 --2- 172.21.15.83:0/3481815980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e84143c10 0x7f4e84070230 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f4e88fda640 1 -- 172.21.15.83:0/3481815980 >> 172.21.15.83:0/3481815980 conn(0x7f4e8406abe0 msgr2=0x7f4e8406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:21.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f24c251a640 1 --2- 172.21.15.83:0/799217591 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a80039c0 0x7f24a8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f4561ce0640 1 --2- 172.21.15.83:0/3973456889 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f455c15ff20 0x7f455c0cde60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4554009d90 tx=0x7f455401c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f49172b69fc25941 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f7592ddf640 1 -- 172.21.15.83:0/3200188210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7584053cc0 msgr2=0x7f7584054150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f7592ddf640 1 --2- 172.21.15.83:0/3200188210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7584053cc0 0x7f7584054150 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7584052540 tx=0x7f758801ef30 comp rx=0 tx=0).stop 2024-04-17T10:38:21.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f24c2d1b640 1 --2- 172.21.15.83:0/799217591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24a80051d0 0x7f24a800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f5afd59a640 1 -- 172.21.15.83:0/1623006238 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af00a7070 con 0x7f5b000b3900 2024-04-17T10:38:21.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f5afd59a640 1 -- 172.21.15.83:0/1623006238 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af00b0070 con 0x7f5b000b3900 2024-04-17T10:38:21.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f7592ddf640 1 -- 172.21.15.83:0/3200188210 shutdown_connections 2024-04-17T10:38:21.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f7592ddf640 1 --2- 172.21.15.83:0/3200188210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7584053cc0 0x7f7584054150 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f7592ddf640 1 --2- 172.21.15.83:0/3200188210 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7584053620 0x7f7584006c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f5afd59a640 1 -- 172.21.15.83:0/1623006238 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af00ab890 con 0x7f5b000b3900 2024-04-17T10:38:21.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f7592ddf640 1 --2- 172.21.15.83:0/3200188210 >> v2:172.21.15.167:3300/0 conn(0x7f75840df770 0x7f7584006730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f7592ddf640 1 -- 172.21.15.83:0/3200188210 >> 172.21.15.83:0/3200188210 conn(0x7f7584009fc0 msgr2=0x7f758400a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:21.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f4553fff640 1 -- 172.21.15.83:0/3973456889 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45540b2070 con 0x7f455c15ff20 2024-04-17T10:38:21.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f08e7fff640 1 -- 172.21.15.83:0/2294871384 wait complete. 2024-04-17T10:38:21.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f08e7fff640 1 Processor -- start 2024-04-17T10:38:21.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f7592ddf640 1 -- 172.21.15.83:0/3200188210 shutdown_connections 2024-04-17T10:38:21.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7fecaebef640 1 -- 172.21.15.83:0/3743687822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feca815bc10 msgr2=0x7feca80d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7fecaebef640 1 --2- 172.21.15.83:0/3743687822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feca815bc10 0x7feca80d2300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7feca80678a0 tx=0x7fec9c01c4d0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7fecaebef640 1 -- 172.21.15.83:0/3743687822 shutdown_connections 2024-04-17T10:38:21.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7fecaebef640 1 --2- 172.21.15.83:0/3743687822 >> v2:172.21.15.83:3300/0 conn(0x7feca806e500 0x7feca806e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7fecaebef640 1 --2- 172.21.15.83:0/3743687822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feca815bc10 0x7feca80d2300 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7fecaebef640 1 -- 172.21.15.83:0/3743687822 >> 172.21.15.83:0/3743687822 conn(0x7feca80d17e0 msgr2=0x7feca80d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:21.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f5b04f53640 1 -- 172.21.15.83:0/1623006238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b000b3900 msgr2=0x7f5b0015efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f5b04f53640 1 --2- 172.21.15.83:0/1623006238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b000b3900 0x7f5b0015efb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5af00097c0 tx=0x7f5af001c490 comp rx=0 tx=0).stop 2024-04-17T10:38:21.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f082159a640 1 -- 172.21.15.83:0/1434895792 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08140b0070 con 0x7f081c0501b0 2024-04-17T10:38:21.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f082159a640 1 -- 172.21.15.83:0/1434895792 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08140ab400 con 0x7f081c0501b0 2024-04-17T10:38:21.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f082159a640 1 -- 172.21.15.83:0/1434895792 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08140ab7d0 con 0x7f081c0501b0 2024-04-17T10:38:21.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f082359e640 1 -- 172.21.15.83:0/1434895792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f081c0501b0 msgr2=0x7f081c006540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f082359e640 1 --2- 172.21.15.83:0/1434895792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f081c0501b0 0x7f081c006540 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f08140097c0 tx=0x7f081401c430 comp rx=0 tx=0).stop 2024-04-17T10:38:21.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f4e88fda640 1 -- 172.21.15.83:0/3481815980 shutdown_connections 2024-04-17T10:38:21.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f4553fff640 1 -- 172.21.15.83:0/3973456889 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45540a5740 con 0x7f455c15ff20 2024-04-17T10:38:21.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f4553fff640 1 -- 172.21.15.83:0/3973456889 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45540a5b10 con 0x7f455c15ff20 2024-04-17T10:38:21.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f4563743640 1 -- 172.21.15.83:0/3973456889 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f455c15ff20 msgr2=0x7f455c0cde60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f4563743640 1 --2- 172.21.15.83:0/3973456889 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f455c15ff20 0x7f455c0cde60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4554009d90 tx=0x7f455401c970 comp rx=0 tx=0).stop 2024-04-17T10:38:21.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f4563743640 1 -- 172.21.15.83:0/3973456889 shutdown_connections 2024-04-17T10:38:21.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f4563743640 1 --2- 172.21.15.83:0/3973456889 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f455c15ff20 0x7f455c0cde60 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f4563743640 1 --2- 172.21.15.83:0/3973456889 >> v2:172.21.15.83:3300/0 conn(0x7f455c155e60 0x7f455c070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f4563743640 1 -- 172.21.15.83:0/3973456889 >> 172.21.15.83:0/3973456889 conn(0x7f455c0cf630 msgr2=0x7f455c0cfa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:21.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f082359e640 1 -- 172.21.15.83:0/1434895792 shutdown_connections 2024-04-17T10:38:21.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f082359e640 1 --2- 172.21.15.83:0/1434895792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f081c0501b0 0x7f081c006540 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f082359e640 1 --2- 172.21.15.83:0/1434895792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f081c00aae0 0x7f081c0df050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f082359e640 1 -- 172.21.15.83:0/1434895792 >> 172.21.15.83:0/1434895792 conn(0x7f081c009fc0 msgr2=0x7f081c00a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:21.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7fecaebef640 1 -- 172.21.15.83:0/3743687822 shutdown_connections 2024-04-17T10:38:21.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7fecaebef640 1 -- 172.21.15.83:0/3743687822 wait complete. 2024-04-17T10:38:21.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f082359e640 1 -- 172.21.15.83:0/1434895792 shutdown_connections 2024-04-17T10:38:21.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f4e88fda640 1 -- 172.21.15.83:0/3481815980 wait complete. 2024-04-17T10:38:21.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f08e7fff640 1 -- start start 2024-04-17T10:38:21.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f08e7fff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08e806b700 0x7f08e80b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f08e7fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08e8150760 0x7f08e8146560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f08e7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08e8150bc0 0x7f08e80af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f08e7fff640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e816be80 con 0x7f08e8150bc0 2024-04-17T10:38:21.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f08e7fff640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e815de10 con 0x7f08e8150760 2024-04-17T10:38:21.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f08e7fff640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e816c000 con 0x7f08e806b700 2024-04-17T10:38:21.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f08e77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08e8150bc0 0x7f08e80af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f08e6ffd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08e806b700 0x7f08e80b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f08e77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08e8150bc0 0x7f08e80af9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47988/0 (socket says 172.21.15.83:47988) 2024-04-17T10:38:21.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f08e77fe640 1 -- 172.21.15.83:0/2511271651 learned_addr learned my addr 172.21.15.83:0/2511271651 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:21.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f5b04f53640 1 -- 172.21.15.83:0/1623006238 shutdown_connections 2024-04-17T10:38:21.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f5b04f53640 1 --2- 172.21.15.83:0/1623006238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b000b3900 0x7f5b0015efb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f5b04f53640 1 -- 172.21.15.83:0/1623006238 >> 172.21.15.83:0/1623006238 conn(0x7f5b0006abe0 msgr2=0x7f5b0006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:21.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f7592ddf640 1 -- 172.21.15.83:0/3200188210 wait complete. 2024-04-17T10:38:21.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f7592ddf640 1 Processor -- start 2024-04-17T10:38:21.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f08e77fe640 1 -- 172.21.15.83:0/2511271651 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08e806b700 msgr2=0x7f08e80b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f08e77fe640 1 --2- 172.21.15.83:0/2511271651 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08e806b700 0x7f08e80b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f08e77fe640 1 -- 172.21.15.83:0/2511271651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08e8150760 msgr2=0x7f08e8146560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f08e77fe640 1 --2- 172.21.15.83:0/2511271651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08e8150760 0x7f08e8146560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f08e77fe640 1 -- 172.21.15.83:0/2511271651 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e80b0120 con 0x7f08e8150bc0 2024-04-17T10:38:21.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f24c251a640 1 -- 172.21.15.83:0/799217591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24a8004810 msgr2=0x7f24a8004c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T10:38:21.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f24c251a640 1 --2- 172.21.15.83:0/799217591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24a8004810 0x7f24a8004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f24c251a640 1 -- 172.21.15.83:0/799217591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24a80051d0 msgr2=0x7f24a800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:21.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f24c251a640 1 --2- 172.21.15.83:0/799217591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24a80051d0 0x7f24a800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f24c251a640 1 -- 172.21.15.83:0/799217591 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b404f2e0 con 0x7f24a80039c0 2024-04-17T10:38:21.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f24c2d1b640 1 -- 172.21.15.83:0/799217591 reap_dead start 2024-04-17T10:38:21.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f24c251a640 1 --2- 172.21.15.83:0/799217591 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a80039c0 0x7f24a8003da0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f24b800c290 tx=0x7f24b801cd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e0fb95efc67f56b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f24c1518640 1 -- 172.21.15.83:0/799217591 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b8007d80 con 0x7f24a80039c0 2024-04-17T10:38:21.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f24c1518640 1 -- 172.21.15.83:0/799217591 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b80a9070 con 0x7f24a80039c0 2024-04-17T10:38:21.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f24c1518640 1 -- 172.21.15.83:0/799217591 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b80058e0 con 0x7f24a80039c0 2024-04-17T10:38:21.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f24c351c640 1 -- 172.21.15.83:0/799217591 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a80039c0 msgr2=0x7f24a8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f24c351c640 1 --2- 172.21.15.83:0/799217591 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a80039c0 0x7f24a8003da0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f24b800c290 tx=0x7f24b801cd60 comp rx=0 tx=0).stop 2024-04-17T10:38:21.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f24c351c640 1 -- 172.21.15.83:0/799217591 shutdown_connections 2024-04-17T10:38:21.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f24c351c640 1 --2- 172.21.15.83:0/799217591 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a80039c0 0x7f24a8003da0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f24c351c640 1 -- 172.21.15.83:0/799217591 >> 172.21.15.83:0/799217591 conn(0x7f24b4009fc0 msgr2=0x7f24b400a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:21.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.680+0000 7f7592ddf640 1 -- start start 2024-04-17T10:38:21.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f7592ddf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7584053620 0x7f75840d9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f7591ddd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7584053620 0x7f75840d9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f7591ddd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7584053620 0x7f75840d9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47996/0 (socket says 172.21.15.83:47996) 2024-04-17T10:38:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f7592ddf640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7584053cc0 0x7f7584043dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f7592ddf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75840df770 0x7f7584038320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f7592ddf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7584053100 con 0x7f7584053620 2024-04-17T10:38:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f7592ddf640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7584052e20 con 0x7f75840df770 2024-04-17T10:38:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f7591ddd640 1 -- 172.21.15.83:0/2603951484 learned_addr learned my addr 172.21.15.83:0/2603951484 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f7592ddf640 1 -- 172.21.15.83:0/2603951484 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7584054690 con 0x7f7584053cc0 2024-04-17T10:38:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f75915dc640 1 --2- 172.21.15.83:0/2603951484 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7584053cc0 0x7f7584043dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f7591ddd640 1 -- 172.21.15.83:0/2603951484 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7584053cc0 msgr2=0x7f7584043dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f7591ddd640 1 --2- 172.21.15.83:0/2603951484 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7584053cc0 0x7f7584043dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f7591ddd640 1 -- 172.21.15.83:0/2603951484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75840df770 msgr2=0x7f7584038320 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T10:38:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f7591ddd640 1 --2- 172.21.15.83:0/2603951484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75840df770 0x7f7584038320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f7591ddd640 1 -- 172.21.15.83:0/2603951484 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75840dbac0 con 0x7f7584053620 2024-04-17T10:38:21.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f7591ddd640 1 --2- 172.21.15.83:0/2603951484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7584053620 0x7f75840d9cf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f758c07ad60 tx=0x7f758c069310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f75827fc640 1 -- 172.21.15.83:0/2603951484 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c098070 con 0x7f7584053620 2024-04-17T10:38:21.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f75827fc640 1 -- 172.21.15.83:0/2603951484 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f758c0687e0 con 0x7f7584053620 2024-04-17T10:38:21.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f75827fc640 1 -- 172.21.15.83:0/2603951484 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c093630 con 0x7f7584053620 2024-04-17T10:38:21.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f7592ddf640 1 -- 172.21.15.83:0/2603951484 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75840dbda0 con 0x7f7584053620 2024-04-17T10:38:21.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f7592ddf640 1 -- 172.21.15.83:0/2603951484 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75840dc360 con 0x7f7584053620 2024-04-17T10:38:21.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7fecaebef640 1 Processor -- start 2024-04-17T10:38:21.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7fecaebef640 1 -- start start 2024-04-17T10:38:21.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7fecaebef640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feca806e500 0x7feca80c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7fecaebef640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feca80d0490 0x7feca80c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7fecaebef640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feca815bc10 0x7feca814ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7fecaebef640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca8168b10 con 0x7feca815bc10 2024-04-17T10:38:21.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7fecaebef640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca8168990 con 0x7feca80d0490 2024-04-17T10:38:21.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7fecaebef640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feca8168c90 con 0x7feca806e500 2024-04-17T10:38:21.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7fecac98b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feca806e500 0x7feca80c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7feca7fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feca80d0490 0x7feca80c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7fecad18c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feca815bc10 0x7feca814ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7fecad18c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feca815bc10 0x7feca814ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48000/0 (socket says 172.21.15.83:48000) 2024-04-17T10:38:21.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7fecad18c640 1 -- 172.21.15.83:0/1442468215 learned_addr learned my addr 172.21.15.83:0/1442468215 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:21.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7fecac98b640 1 -- 172.21.15.83:0/1442468215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feca80d0490 msgr2=0x7feca80c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7fecac98b640 1 --2- 172.21.15.83:0/1442468215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feca80d0490 0x7feca80c5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7fecac98b640 1 -- 172.21.15.83:0/1442468215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feca815bc10 msgr2=0x7feca814ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7fecac98b640 1 --2- 172.21.15.83:0/1442468215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feca815bc10 0x7feca814ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7fecac98b640 1 -- 172.21.15.83:0/1442468215 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca80c62c0 con 0x7feca806e500 2024-04-17T10:38:21.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7fecac98b640 1 --2- 172.21.15.83:0/1442468215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feca806e500 0x7feca80c88b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0ab000 tx=0x7fec9c01ad10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f4e88fda640 1 Processor -- start 2024-04-17T10:38:21.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f4e88fda640 1 -- start start 2024-04-17T10:38:21.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f4e88fda640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e840add80 0x7f4e84160c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f4e88fda640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e84143c10 0x7f4e84130a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f4e88fda640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e84143ff0 0x7f4e8415fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f4e88fda640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e840ab440 con 0x7f4e84143c10 2024-04-17T10:38:21.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f4e88fda640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e84070980 con 0x7f4e840add80 2024-04-17T10:38:21.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f4e82ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e84143c10 0x7f4e84130a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f4e88fda640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e840ab5c0 con 0x7f4e84143ff0 2024-04-17T10:38:21.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f4e82ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e84143c10 0x7f4e84130a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48006/0 (socket says 172.21.15.83:48006) 2024-04-17T10:38:21.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f4e82ffd640 1 -- 172.21.15.83:0/2367553727 learned_addr learned my addr 172.21.15.83:0/2367553727 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:21.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f4e82ffd640 1 -- 172.21.15.83:0/2367553727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e84143ff0 msgr2=0x7f4e8415fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f4e82ffd640 1 --2- 172.21.15.83:0/2367553727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e84143ff0 0x7f4e8415fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f4e82ffd640 1 -- 172.21.15.83:0/2367553727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e840add80 msgr2=0x7f4e84160c10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-17T10:38:21.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f4e82ffd640 1 --2- 172.21.15.83:0/2367553727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e840add80 0x7f4e84160c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f4e82ffd640 1 -- 172.21.15.83:0/2367553727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e84158bb0 con 0x7f4e84143c10 2024-04-17T10:38:21.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f08e77fe640 1 --2- 172.21.15.83:0/2511271651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08e8150bc0 0x7f08e80af9d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00c490 tx=0x7f08dc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f08cffff640 1 -- 172.21.15.83:0/2511271651 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc018070 con 0x7f08e8150bc0 2024-04-17T10:38:21.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f08e7fff640 1 -- 172.21.15.83:0/2511271651 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e80b0400 con 0x7f08e8150bc0 2024-04-17T10:38:21.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f08e7fff640 1 -- 172.21.15.83:0/2511271651 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e80b09c0 con 0x7f08e8150bc0 2024-04-17T10:38:21.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f08cffff640 1 -- 172.21.15.83:0/2511271651 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08dc0074e0 con 0x7f08e8150bc0 2024-04-17T10:38:21.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f08cffff640 1 -- 172.21.15.83:0/2511271651 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc0049f0 con 0x7f08e8150bc0 2024-04-17T10:38:21.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f08e7fff640 1 -- 172.21.15.83:0/2511271651 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f08e804efa0 con 0x7f08e8150bc0 2024-04-17T10:38:21.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f5b04f53640 1 -- 172.21.15.83:0/1623006238 shutdown_connections 2024-04-17T10:38:21.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f5b04f53640 1 -- 172.21.15.83:0/1623006238 wait complete. 2024-04-17T10:38:21.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f5b04f53640 1 Processor -- start 2024-04-17T10:38:21.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f5b04f53640 1 -- start start 2024-04-17T10:38:21.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f5b04f53640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b00072460 0x7f5b00142650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f5b04f53640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b000aed60 0x7f5b0015b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f5b04f53640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b000b3900 0x7f5b0015b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f5b04f53640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b00071010 con 0x7f5b00072460 2024-04-17T10:38:21.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f5b04f53640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b00070e90 con 0x7f5b000aed60 2024-04-17T10:38:21.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f5b04f53640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b00170500 con 0x7f5b000b3900 2024-04-17T10:38:21.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f4563743640 1 -- 172.21.15.83:0/3973456889 shutdown_connections 2024-04-17T10:38:21.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f4563743640 1 -- 172.21.15.83:0/3973456889 wait complete. 2024-04-17T10:38:21.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f4563743640 1 Processor -- start 2024-04-17T10:38:21.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f082359e640 1 -- 172.21.15.83:0/1434895792 wait complete. 2024-04-17T10:38:21.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f082359e640 1 Processor -- start 2024-04-17T10:38:21.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f082359e640 1 -- start start 2024-04-17T10:38:21.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f082359e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f081c00aae0 0x7f081c04db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f082359e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f081c033d00 0x7f081c04e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f082359e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f081c0501b0 0x7f081c04bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f082359e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081c007410 con 0x7f081c0501b0 2024-04-17T10:38:21.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f082359e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081c007290 con 0x7f081c033d00 2024-04-17T10:38:21.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.684+0000 7f082359e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f081c007590 con 0x7f081c00aae0 2024-04-17T10:38:21.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7feca57fa640 1 -- 172.21.15.83:0/1442468215 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9c0be070 con 0x7feca806e500 2024-04-17T10:38:21.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f5afed9d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b000b3900 0x7f5b0015b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f5afdd9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b000aed60 0x7f5b0015b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f5afed9d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b000b3900 0x7f5b0015b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:59594/0 (socket says 172.21.15.83:59594) 2024-04-17T10:38:21.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f5afed9d640 1 -- 172.21.15.83:0/2408881409 learned_addr learned my addr 172.21.15.83:0/2408881409 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:21.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7feca57fa640 1 -- 172.21.15.83:0/1442468215 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec9c0b8e20 con 0x7feca806e500 2024-04-17T10:38:21.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7feca57fa640 1 -- 172.21.15.83:0/1442468215 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9c0b0040 con 0x7feca806e500 2024-04-17T10:38:21.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f7592ddf640 1 -- 172.21.15.83:0/2603951484 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f75840053a0 con 0x7f7584053620 2024-04-17T10:38:21.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f082259c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f081c00aae0 0x7f081c04db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f082259c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f081c00aae0 0x7f081c04db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:59596/0 (socket says 172.21.15.83:59596) 2024-04-17T10:38:21.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f082259c640 1 -- 172.21.15.83:0/1544570610 learned_addr learned my addr 172.21.15.83:0/1544570610 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:21.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f0821d9b640 1 --2- 172.21.15.83:0/1544570610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f081c033d00 0x7f081c04e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f08cffff640 1 -- 172.21.15.83:0/2511271651 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f08dc01b070 con 0x7f08e8150bc0 2024-04-17T10:38:21.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7fecaebef640 1 -- 172.21.15.83:0/1442468215 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feca80c65a0 con 0x7feca806e500 2024-04-17T10:38:21.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7fecaebef640 1 -- 172.21.15.83:0/1442468215 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feca80c6b60 con 0x7feca806e500 2024-04-17T10:38:21.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f5afdd9b640 1 -- 172.21.15.83:0/2408881409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b000b3900 msgr2=0x7f5b0015b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f5afdd9b640 1 --2- 172.21.15.83:0/2408881409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b000b3900 0x7f5b0015b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f5afdd9b640 1 -- 172.21.15.83:0/2408881409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b00072460 msgr2=0x7f5b00142650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f5afdd9b640 1 --2- 172.21.15.83:0/2408881409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b00072460 0x7f5b00142650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f5afdd9b640 1 -- 172.21.15.83:0/2408881409 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b00153360 con 0x7f5b000aed60 2024-04-17T10:38:21.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f08cffff640 1 --2- 172.21.15.83:0/2511271651 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f08d4066fe0 0x7f08d40694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f24c351c640 1 -- 172.21.15.83:0/799217591 shutdown_connections 2024-04-17T10:38:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f08cffff640 1 -- 172.21.15.83:0/2511271651 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08dc084260 con 0x7f08e8150bc0 2024-04-17T10:38:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f08e6ffd640 1 --2- 172.21.15.83:0/2511271651 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f08d4066fe0 0x7f08d40694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f082259c640 1 -- 172.21.15.83:0/1544570610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f081c033d00 msgr2=0x7f081c04e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f082259c640 1 --2- 172.21.15.83:0/1544570610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f081c033d00 0x7f081c04e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f082259c640 1 -- 172.21.15.83:0/1544570610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f081c0501b0 msgr2=0x7f081c04bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f082259c640 1 --2- 172.21.15.83:0/1544570610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f081c0501b0 0x7f081c04bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f082259c640 1 -- 172.21.15.83:0/1544570610 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081c0df920 con 0x7f081c00aae0 2024-04-17T10:38:21.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f0821d9b640 1 --2- 172.21.15.83:0/1544570610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f081c033d00 0x7f081c04e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:38:21.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f08cffff640 1 --2- 172.21.15.83:0/2511271651 >> [v2:172.21.15.167:6813/77470876,v1:172.21.15.167:6817/77470876] conn(0x7f08d406f1e0 0x7f08d4071600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f08cffff640 1 -- 172.21.15.83:0/2511271651 --> [v2:172.21.15.167:6813/77470876,v1:172.21.15.167:6817/77470876] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f08d4071cb0 con 0x7f08d406f1e0 2024-04-17T10:38:21.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f5afdd9b640 1 --2- 172.21.15.83:0/2408881409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b000aed60 0x7f5b0015b430 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5b0006e710 tx=0x7f5af001acd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7fecaebef640 1 -- 172.21.15.83:0/1442468215 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7feca8074f90 con 0x7feca806e500 2024-04-17T10:38:21.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f4563743640 1 -- start start 2024-04-17T10:38:21.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f4563743640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f455c155e60 0x7f455c06e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f4563743640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f455c15f3b0 0x7f455c1431d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f4563743640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f455c15ff20 0x7f455c143710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f4563743640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455c0ced30 con 0x7f455c15ff20 2024-04-17T10:38:21.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f4563743640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455c0cebb0 con 0x7f455c15f3b0 2024-04-17T10:38:21.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f4563743640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455c0ceeb0 con 0x7f455c155e60 2024-04-17T10:38:21.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f082259c640 1 --2- 172.21.15.83:0/1544570610 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f081c00aae0 0x7f081c04db30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0824068370 tx=0x7f082404f630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f08e67fc640 1 --2- 172.21.15.83:0/2511271651 >> [v2:172.21.15.167:6813/77470876,v1:172.21.15.167:6817/77470876] conn(0x7f08d406f1e0 0x7f08d4071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f5aeeffd640 1 -- 172.21.15.83:0/2408881409 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af00b5610 con 0x7f5b000aed60 2024-04-17T10:38:21.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f5aeeffd640 1 -- 172.21.15.83:0/2408881409 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af00b57f0 con 0x7f5b000aed60 2024-04-17T10:38:21.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f45614df640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f455c155e60 0x7f455c06e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f5aeeffd640 1 -- 172.21.15.83:0/2408881409 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af00b5be0 con 0x7f5b000aed60 2024-04-17T10:38:21.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f4560cde640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f455c15f3b0 0x7f455c1431d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f45614df640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f455c155e60 0x7f455c06e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:59610/0 (socket says 172.21.15.83:59610) 2024-04-17T10:38:21.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f45614df640 1 -- 172.21.15.83:0/1583498103 learned_addr learned my addr 172.21.15.83:0/1583498103 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f08e6ffd640 1 --2- 172.21.15.83:0/2511271651 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f08d4066fe0 0x7f08d40694a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f08d80011f0 tx=0x7f08d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f4560cde640 1 -- 172.21.15.83:0/1583498103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f455c155e60 msgr2=0x7f455c06e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f4560cde640 1 --2- 172.21.15.83:0/1583498103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f455c155e60 0x7f455c06e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f4560cde640 1 -- 172.21.15.83:0/1583498103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f455c15ff20 msgr2=0x7f455c143710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f4560cde640 1 --2- 172.21.15.83:0/1583498103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f455c15ff20 0x7f455c143710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f4560cde640 1 -- 172.21.15.83:0/1583498103 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455c168060 con 0x7f455c15f3b0 2024-04-17T10:38:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f45614df640 1 --2- 172.21.15.83:0/1583498103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f455c155e60 0x7f455c06e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:38:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f24c351c640 1 -- 172.21.15.83:0/799217591 wait complete. 2024-04-17T10:38:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f4e82ffd640 1 --2- 172.21.15.83:0/2367553727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e84143c10 0x7f4e84130a80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4e7400efc0 tx=0x7f4e7400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f5b04f53640 1 -- 172.21.15.83:0/2408881409 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b00153640 con 0x7f5b000aed60 2024-04-17T10:38:21.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f5b04f53640 1 -- 172.21.15.83:0/2408881409 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b00153c00 con 0x7f5b000aed60 2024-04-17T10:38:21.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f08e67fc640 1 --2- 172.21.15.83:0/2511271651 >> [v2:172.21.15.167:6813/77470876,v1:172.21.15.167:6817/77470876] conn(0x7f08d406f1e0 0x7f08d4071600 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f4560cde640 1 --2- 172.21.15.83:0/1583498103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f455c15f3b0 0x7f455c1431d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f454c00c460 tx=0x7f454c00c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f0812ffd640 1 -- 172.21.15.83:0/1544570610 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0824098070 con 0x7f081c00aae0 2024-04-17T10:38:21.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f0812ffd640 1 -- 172.21.15.83:0/1544570610 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0824093430 con 0x7f081c00aae0 2024-04-17T10:38:21.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f0812ffd640 1 -- 172.21.15.83:0/1544570610 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08240937e0 con 0x7f081c00aae0 2024-04-17T10:38:21.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f5b04f53640 1 -- 172.21.15.83:0/2408881409 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5b00153110 con 0x7f5b000aed60 2024-04-17T10:38:21.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f082359e640 1 -- 172.21.15.83:0/1544570610 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f081c0dfc00 con 0x7f081c00aae0 2024-04-17T10:38:21.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f082359e640 1 -- 172.21.15.83:0/1544570610 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f081c0e01c0 con 0x7f081c00aae0 2024-04-17T10:38:21.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7feca57fa640 1 -- 172.21.15.83:0/1442468215 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fec9c0c9060 con 0x7feca806e500 2024-04-17T10:38:21.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f4551ffb640 1 -- 172.21.15.83:0/1583498103 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454c017070 con 0x7f455c15f3b0 2024-04-17T10:38:21.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.688+0000 7f24c351c640 1 Processor -- start 2024-04-17T10:38:21.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f4551ffb640 1 -- 172.21.15.83:0/1583498103 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f454c005cf0 con 0x7f455c15f3b0 2024-04-17T10:38:21.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f082359e640 1 -- 172.21.15.83:0/1544570610 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0808001040 con 0x7f081c00aae0 2024-04-17T10:38:21.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7feca57fa640 1 --2- 172.21.15.83:0/1442468215 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fec94066f90 0x7fec94069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7feca57fa640 1 -- 172.21.15.83:0/1442468215 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec9c1272c0 con 0x7feca806e500 2024-04-17T10:38:21.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7feca7fff640 1 --2- 172.21.15.83:0/1442468215 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fec94066f90 0x7fec94069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f75827fc640 1 -- 172.21.15.83:0/2603951484 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f758c0937d0 con 0x7f7584053620 2024-04-17T10:38:21.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7feca57fa640 1 --2- 172.21.15.83:0/1442468215 >> [v2:172.21.15.83:6816/3642321714,v1:172.21.15.83:6818/3642321714] conn(0x7fec9406f130 0x7fec94071550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f4e6bfff640 1 -- 172.21.15.83:0/2367553727 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e74018020 con 0x7f4e84143c10 2024-04-17T10:38:21.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f4e6bfff640 1 -- 172.21.15.83:0/2367553727 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e74014480 con 0x7f4e84143c10 2024-04-17T10:38:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f08cffff640 1 -- 172.21.15.83:0/2511271651 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08dc08b050 con 0x7f08e8150bc0 2024-04-17T10:38:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f4e88fda640 1 -- 172.21.15.83:0/2367553727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e8415f1d0 con 0x7f4e84143c10 2024-04-17T10:38:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7feca57fa640 1 -- 172.21.15.83:0/1442468215 --> [v2:172.21.15.83:6816/3642321714,v1:172.21.15.83:6818/3642321714] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fec94071c00 con 0x7fec9406f130 2024-04-17T10:38:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7feca57fa640 1 -- 172.21.15.83:0/1442468215 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec9c127750 con 0x7feca806e500 2024-04-17T10:38:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f4e88fda640 1 -- 172.21.15.83:0/2367553727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e8413c030 con 0x7f4e84143c10 2024-04-17T10:38:21.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f24c351c640 1 -- start start 2024-04-17T10:38:21.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f4e6bfff640 1 -- 172.21.15.83:0/2367553727 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e74014850 con 0x7f4e84143c10 2024-04-17T10:38:21.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f75827fc640 1 --2- 172.21.15.83:0/2603951484 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7570066f60 0x7f7570069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f75827fc640 1 -- 172.21.15.83:0/2603951484 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f758c1003a0 con 0x7f7584053620 2024-04-17T10:38:21.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f75827fc640 1 --2- 172.21.15.83:0/2603951484 >> [v2:172.21.15.83:6808/2278974484,v1:172.21.15.83:6809/2278974484] conn(0x7f757006f100 0x7f7570071520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f75827fc640 1 -- 172.21.15.83:0/2603951484 --> [v2:172.21.15.83:6808/2278974484,v1:172.21.15.83:6809/2278974484] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f758c097ec0 con 0x7f757006f100 2024-04-17T10:38:21.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f75827fc640 1 -- 172.21.15.83:0/2603951484 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f758c100750 con 0x7f7584053620 2024-04-17T10:38:21.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f4563743640 1 -- 172.21.15.83:0/1583498103 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f455c168340 con 0x7f455c15f3b0 2024-04-17T10:38:21.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f4563743640 1 -- 172.21.15.83:0/1583498103 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f455c168900 con 0x7f455c15f3b0 2024-04-17T10:38:21.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f4551ffb640 1 -- 172.21.15.83:0/1583498103 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454c007270 con 0x7f455c15f3b0 2024-04-17T10:38:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f24c351c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24b41889e0 0x7f24b40e71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f24c351c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24b4188dc0 0x7f24b4189240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f24c351c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24b4036b30 0x7f24b4189780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f24c351c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b404fd80 con 0x7f24b4036b30 2024-04-17T10:38:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f24c351c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b404f090 con 0x7f24b41889e0 2024-04-17T10:38:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f24c351c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b40482c0 con 0x7f24b4188dc0 2024-04-17T10:38:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f4e88fda640 1 -- 172.21.15.83:0/2367553727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4e64001040 con 0x7f4e84143c10 2024-04-17T10:38:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f08cffff640 1 -- 172.21.15.83:0/2511271651 <== osd.4 v2:172.21.15.167:6813/77470876 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f08d4071cb0 con 0x7f08d406f1e0 2024-04-17T10:38:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f24c1d19640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24b4188dc0 0x7f24b4189240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f24c2d1b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24b4036b30 0x7f24b4189780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f24c1d19640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24b4188dc0 0x7f24b4189240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:59626/0 (socket says 172.21.15.83:59626) 2024-04-17T10:38:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f24c1d19640 1 -- 172.21.15.83:0/3951976890 learned_addr learned my addr 172.21.15.83:0/3951976890 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f24c2d1b640 1 --2- 172.21.15.83:0/3951976890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24b4036b30 0x7f24b4189780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48044/0 (socket says 172.21.15.83:48044) 2024-04-17T10:38:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f5aeeffd640 1 -- 172.21.15.83:0/2408881409 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5af00bd020 con 0x7f5b000aed60 2024-04-17T10:38:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f24c2d1b640 1 -- 172.21.15.83:0/3951976890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24b4188dc0 msgr2=0x7f24b4189240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f24c2d1b640 1 --2- 172.21.15.83:0/3951976890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24b4188dc0 0x7f24b4189240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f24c2d1b640 1 -- 172.21.15.83:0/3951976890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24b41889e0 msgr2=0x7f24b40e71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f24c2d1b640 1 --2- 172.21.15.83:0/3951976890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24b41889e0 0x7f24b40e71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f24c2d1b640 1 -- 172.21.15.83:0/3951976890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b40e9fe0 con 0x7f24b4036b30 2024-04-17T10:38:21.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f24c2d1b640 1 --2- 172.21.15.83:0/3951976890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24b4036b30 0x7f24b4189780 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f24bc06a390 tx=0x7f24bc068a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f5aeeffd640 1 --2- 172.21.15.83:0/2408881409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5ae0066f60 0x7f5ae0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f5aeeffd640 1 -- 172.21.15.83:0/2408881409 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5af011fa70 con 0x7f5b000aed60 2024-04-17T10:38:21.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f5aeeffd640 1 --2- 172.21.15.83:0/2408881409 >> [v2:172.21.15.167:6800/1776227116,v1:172.21.15.167:6801/1776227116] conn(0x7f5ae006f100 0x7f5ae0071520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f5aeeffd640 1 -- 172.21.15.83:0/2408881409 --> [v2:172.21.15.167:6800/1776227116,v1:172.21.15.167:6801/1776227116] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5af00b3e90 con 0x7f5ae006f100 2024-04-17T10:38:21.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f5aeeffd640 1 -- 172.21.15.83:0/2408881409 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5af011fe60 con 0x7f5b000aed60 2024-04-17T10:38:21.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f24b2ffd640 1 -- 172.21.15.83:0/3951976890 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24bc098070 con 0x7f24b4036b30 2024-04-17T10:38:21.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f75925de640 1 --2- 172.21.15.83:0/2603951484 >> [v2:172.21.15.83:6808/2278974484,v1:172.21.15.83:6809/2278974484] conn(0x7f757006f100 0x7f7570071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f75925de640 1 --2- 172.21.15.83:0/2603951484 >> [v2:172.21.15.83:6808/2278974484,v1:172.21.15.83:6809/2278974484] conn(0x7f757006f100 0x7f7570071520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f0812ffd640 1 -- 172.21.15.83:0/1544570610 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f082409b070 con 0x7f081c00aae0 2024-04-17T10:38:21.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f5afe59c640 1 --2- 172.21.15.83:0/2408881409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5ae0066f60 0x7f5ae0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7fecad18c640 1 --2- 172.21.15.83:0/1442468215 >> [v2:172.21.15.83:6816/3642321714,v1:172.21.15.83:6818/3642321714] conn(0x7fec9406f130 0x7fec94071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f0812ffd640 1 --2- 172.21.15.83:0/1544570610 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0804067060 0x7f0804069520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f0812ffd640 1 -- 172.21.15.83:0/1544570610 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08240ff9d0 con 0x7f081c00aae0 2024-04-17T10:38:21.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f0812ffd640 1 --2- 172.21.15.83:0/1544570610 >> [v2:172.21.15.167:6808/285324536,v1:172.21.15.167:6809/285324536] conn(0x7f080406f280 0x7f08040716a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f0812ffd640 1 -- 172.21.15.83:0/1544570610 --> [v2:172.21.15.167:6808/285324536,v1:172.21.15.167:6809/285324536] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0804071d50 con 0x7f080406f280 2024-04-17T10:38:21.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f0812ffd640 1 -- 172.21.15.83:0/1544570610 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08240ffdc0 con 0x7f081c00aae0 2024-04-17T10:38:21.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f24c351c640 1 -- 172.21.15.83:0/3951976890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b4037030 con 0x7f24b4036b30 2024-04-17T10:38:21.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f24c351c640 1 -- 172.21.15.83:0/3951976890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b40dfa60 con 0x7f24b4036b30 2024-04-17T10:38:21.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f75915dc640 1 --2- 172.21.15.83:0/2603951484 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7570066f60 0x7f7570069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7fecad18c640 1 --2- 172.21.15.83:0/1442468215 >> [v2:172.21.15.83:6816/3642321714,v1:172.21.15.83:6818/3642321714] conn(0x7fec9406f130 0x7fec94071550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.692+0000 7f4e6bfff640 1 -- 172.21.15.83:0/2367553727 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4e7401b020 con 0x7f4e84143c10 2024-04-17T10:38:21.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f4e6bfff640 1 --2- 172.21.15.83:0/2367553727 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4e70066f60 0x7f4e70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f4e6bfff640 1 -- 172.21.15.83:0/2367553727 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e74085af0 con 0x7f4e84143c10 2024-04-17T10:38:21.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f4e6bfff640 1 --2- 172.21.15.83:0/2367553727 >> [v2:172.21.15.83:6800/367778918,v1:172.21.15.83:6801/367778918] conn(0x7f4e7006f100 0x7f4e70071520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f4e6bfff640 1 -- 172.21.15.83:0/2367553727 --> [v2:172.21.15.83:6800/367778918,v1:172.21.15.83:6801/367778918] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4e7401bec0 con 0x7f4e7006f100 2024-04-17T10:38:21.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f4e837fe640 1 --2- 172.21.15.83:0/2367553727 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4e70066f60 0x7f4e70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f4e83fff640 1 --2- 172.21.15.83:0/2367553727 >> [v2:172.21.15.83:6800/367778918,v1:172.21.15.83:6801/367778918] conn(0x7f4e7006f100 0x7f4e70071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f5afed9d640 1 --2- 172.21.15.83:0/2408881409 >> [v2:172.21.15.167:6800/1776227116,v1:172.21.15.167:6801/1776227116] conn(0x7f5ae006f100 0x7f5ae0071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f5afe59c640 1 --2- 172.21.15.83:0/2408881409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5ae0066f60 0x7f5ae0069420 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5af4001600 tx=0x7f5af4007a50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f4551ffb640 1 -- 172.21.15.83:0/1583498103 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f454c01a070 con 0x7f455c15f3b0 2024-04-17T10:38:21.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f4563743640 1 -- 172.21.15.83:0/1583498103 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f455c166800 con 0x7f455c15f3b0 2024-04-17T10:38:21.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f0822d9d640 1 --2- 172.21.15.83:0/1544570610 >> [v2:172.21.15.167:6808/285324536,v1:172.21.15.167:6809/285324536] conn(0x7f080406f280 0x7f08040716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f0821d9b640 1 --2- 172.21.15.83:0/1544570610 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0804067060 0x7f0804069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f75915dc640 1 --2- 172.21.15.83:0/2603951484 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7570066f60 0x7f7570069420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f75841808c0 tx=0x7f757c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f0822d9d640 1 --2- 172.21.15.83:0/1544570610 >> [v2:172.21.15.167:6808/285324536,v1:172.21.15.167:6809/285324536] conn(0x7f080406f280 0x7f08040716a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f4e83fff640 1 --2- 172.21.15.83:0/2367553727 >> [v2:172.21.15.83:6800/367778918,v1:172.21.15.83:6801/367778918] conn(0x7f4e7006f100 0x7f4e70071520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f4551ffb640 1 --2- 172.21.15.83:0/1583498103 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4540066f60 0x7f4540069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f0821d9b640 1 --2- 172.21.15.83:0/1544570610 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0804067060 0x7f0804069520 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f081c04b6e0 tx=0x7f0818009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f4551ffb640 1 -- 172.21.15.83:0/1583498103 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f454c084330 con 0x7f455c15f3b0 2024-04-17T10:38:21.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f24b2ffd640 1 -- 172.21.15.83:0/3951976890 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24bc09d440 con 0x7f24b4036b30 2024-04-17T10:38:21.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f24b2ffd640 1 -- 172.21.15.83:0/3951976890 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24bc0a2060 con 0x7f24b4036b30 2024-04-17T10:38:21.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f4551ffb640 1 --2- 172.21.15.83:0/1583498103 >> [v2:172.21.15.167:6812/1109787903,v1:172.21.15.167:6815/1109787903] conn(0x7f454006f160 0x7f4540071580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7feca7fff640 1 --2- 172.21.15.83:0/1442468215 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fec94066f90 0x7fec94069450 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7feca81587b0 tx=0x7feca0001700 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f5afed9d640 1 --2- 172.21.15.83:0/2408881409 >> [v2:172.21.15.167:6800/1776227116,v1:172.21.15.167:6801/1776227116] conn(0x7f5ae006f100 0x7f5ae0071520 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f4551ffb640 1 -- 172.21.15.83:0/1583498103 --> [v2:172.21.15.167:6812/1109787903,v1:172.21.15.167:6815/1109787903] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f454c005cf0 con 0x7f454006f160 2024-04-17T10:38:21.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f75827fc640 1 -- 172.21.15.83:0/2603951484 <== osd.1 v2:172.21.15.83:6808/2278974484 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f758c097ec0 con 0x7f757006f100 2024-04-17T10:38:21.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7feca57fa640 1 -- 172.21.15.83:0/1442468215 <== osd.2 v2:172.21.15.83:6816/3642321714 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fec94071c00 con 0x7fec9406f130 2024-04-17T10:38:21.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f24c351c640 1 -- 172.21.15.83:0/3951976890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f24b4037310 con 0x7f24b4036b30 2024-04-17T10:38:21.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f4e6bfff640 1 -- 172.21.15.83:0/2367553727 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e74052600 con 0x7f4e84143c10 2024-04-17T10:38:21.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f4551ffb640 1 -- 172.21.15.83:0/1583498103 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f454c050e70 con 0x7f455c15f3b0 2024-04-17T10:38:21.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f4e6bfff640 1 -- 172.21.15.83:0/2367553727 <== osd.3 v2:172.21.15.83:6800/367778918 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4e7401bec0 con 0x7f4e7006f100 2024-04-17T10:38:21.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f4e837fe640 1 --2- 172.21.15.83:0/2367553727 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4e70066f60 0x7f4e70069420 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0b6000 tx=0x7f4e7c0af070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f45614df640 1 --2- 172.21.15.83:0/1583498103 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4540066f60 0x7f4540069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f45614df640 1 --2- 172.21.15.83:0/1583498103 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4540066f60 0x7f4540069420 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4558009e20 tx=0x7f4558009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.696+0000 7f24b2ffd640 1 -- 172.21.15.83:0/3951976890 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f24bc09b070 con 0x7f24b4036b30 2024-04-17T10:38:21.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.700+0000 7f4561ce0640 1 --2- 172.21.15.83:0/1583498103 >> [v2:172.21.15.167:6812/1109787903,v1:172.21.15.167:6815/1109787903] conn(0x7f454006f160 0x7f4540071580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.700+0000 7f24b2ffd640 1 --2- 172.21.15.83:0/3951976890 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f24a0066f60 0x7f24a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.700+0000 7f24b2ffd640 1 -- 172.21.15.83:0/3951976890 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24bc1003d0 con 0x7f24b4036b30 2024-04-17T10:38:21.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.700+0000 7f24b2ffd640 1 --2- 172.21.15.83:0/3951976890 >> [v2:172.21.15.83:6817/2375732,v1:172.21.15.83:6819/2375732] conn(0x7f24a006f160 0x7f24a0071580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.700+0000 7f24b2ffd640 1 -- 172.21.15.83:0/3951976890 --> [v2:172.21.15.83:6817/2375732,v1:172.21.15.83:6819/2375732] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f24a0071c30 con 0x7f24a006f160 2024-04-17T10:38:21.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.700+0000 7f24b2ffd640 1 -- 172.21.15.83:0/3951976890 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24bc100860 con 0x7f24b4036b30 2024-04-17T10:38:21.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.700+0000 7f4561ce0640 1 --2- 172.21.15.83:0/1583498103 >> [v2:172.21.15.167:6812/1109787903,v1:172.21.15.167:6815/1109787903] conn(0x7f454006f160 0x7f4540071580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.700+0000 7f24c251a640 1 --2- 172.21.15.83:0/3951976890 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f24a0066f60 0x7f24a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.700+0000 7f24c251a640 1 --2- 172.21.15.83:0/3951976890 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f24a0066f60 0x7f24a0069420 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f24b80ab000 tx=0x7f24b8001980 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.700+0000 7f5aeeffd640 1 -- 172.21.15.83:0/2408881409 <== osd.5 v2:172.21.15.167:6800/1776227116 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5af00b3e90 con 0x7f5ae006f100 2024-04-17T10:38:21.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.704+0000 7f24c1d19640 1 --2- 172.21.15.83:0/3951976890 >> [v2:172.21.15.83:6817/2375732,v1:172.21.15.83:6819/2375732] conn(0x7f24a006f160 0x7f24a0071580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.704+0000 7f24c1d19640 1 --2- 172.21.15.83:0/3951976890 >> [v2:172.21.15.83:6817/2375732,v1:172.21.15.83:6819/2375732] conn(0x7f24a006f160 0x7f24a0071580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:21.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.704+0000 7f24b2ffd640 1 -- 172.21.15.83:0/3951976890 <== osd.0 v2:172.21.15.83:6817/2375732 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f24a0071c30 con 0x7f24a006f160 2024-04-17T10:38:21.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.708+0000 7f0812ffd640 1 -- 172.21.15.83:0/1544570610 <== osd.7 v2:172.21.15.167:6808/285324536 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0804071d50 con 0x7f080406f280 2024-04-17T10:38:21.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.708+0000 7f4551ffb640 1 -- 172.21.15.83:0/1583498103 <== osd.6 v2:172.21.15.167:6812/1109787903 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f454c005cf0 con 0x7f454006f160 2024-04-17T10:38:21.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.724+0000 7f08e7fff640 1 -- 172.21.15.83:0/2511271651 --> [v2:172.21.15.167:6813/77470876,v1:172.21.15.167:6817/77470876] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f08e80b0e50 con 0x7f08d406f1e0 2024-04-17T10:38:21.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.724+0000 7fecaebef640 1 -- 172.21.15.83:0/1442468215 --> [v2:172.21.15.83:6816/3642321714,v1:172.21.15.83:6818/3642321714] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7feca8074f90 con 0x7fec9406f130 2024-04-17T10:38:21.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.724+0000 7f08cffff640 1 -- 172.21.15.83:0/2511271651 <== osd.4 v2:172.21.15.167:6813/77470876 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f08e80b0e50 con 0x7f08d406f1e0 2024-04-17T10:38:21.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.724+0000 7f7592ddf640 1 -- 172.21.15.83:0/2603951484 --> [v2:172.21.15.83:6808/2278974484,v1:172.21.15.83:6809/2278974484] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f758404ecf0 con 0x7f757006f100 2024-04-17T10:38:21.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.724+0000 7feca57fa640 1 -- 172.21.15.83:0/1442468215 <== osd.2 v2:172.21.15.83:6816/3642321714 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7feca8074f90 con 0x7fec9406f130 2024-04-17T10:38:21.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.724+0000 7f75827fc640 1 -- 172.21.15.83:0/2603951484 <== osd.1 v2:172.21.15.83:6808/2278974484 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f758404ecf0 con 0x7f757006f100 2024-04-17T10:38:21.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.724+0000 7f08cdffb640 1 -- 172.21.15.83:0/2511271651 >> [v2:172.21.15.167:6813/77470876,v1:172.21.15.167:6817/77470876] conn(0x7f08d406f1e0 msgr2=0x7f08d4071600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.724+0000 7f08cdffb640 1 --2- 172.21.15.83:0/2511271651 >> [v2:172.21.15.167:6813/77470876,v1:172.21.15.167:6817/77470876] conn(0x7f08d406f1e0 0x7f08d4071600 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.724+0000 7f08cdffb640 1 -- 172.21.15.83:0/2511271651 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f08d4066fe0 msgr2=0x7f08d40694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.724+0000 7f08cdffb640 1 --2- 172.21.15.83:0/2511271651 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f08d4066fe0 0x7f08d40694a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f08d80011f0 tx=0x7f08d8009040 comp rx=0 tx=0).stop 2024-04-17T10:38:21.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.724+0000 7f08cdffb640 1 -- 172.21.15.83:0/2511271651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08e8150bc0 msgr2=0x7f08e80af9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.724+0000 7f08cdffb640 1 --2- 172.21.15.83:0/2511271651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08e8150bc0 0x7f08e80af9d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00c490 tx=0x7f08dc00c950 comp rx=0 tx=0).stop 2024-04-17T10:38:21.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.724+0000 7f08e77fe640 1 -- 172.21.15.83:0/2511271651 reap_dead start 2024-04-17T10:38:21.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.724+0000 7f756ffff640 1 -- 172.21.15.83:0/2603951484 >> [v2:172.21.15.83:6808/2278974484,v1:172.21.15.83:6809/2278974484] conn(0x7f757006f100 msgr2=0x7f7570071520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.724+0000 7f756ffff640 1 --2- 172.21.15.83:0/2603951484 >> [v2:172.21.15.83:6808/2278974484,v1:172.21.15.83:6809/2278974484] conn(0x7f757006f100 0x7f7570071520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.724+0000 7f08cdffb640 1 -- 172.21.15.83:0/2511271651 shutdown_connections 2024-04-17T10:38:21.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.724+0000 7f08cdffb640 1 -- 172.21.15.83:0/2511271651 >> 172.21.15.83:0/2511271651 conn(0x7f08e806abe0 msgr2=0x7f08e8167300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:21.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.724+0000 7f756ffff640 1 -- 172.21.15.83:0/2603951484 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7570066f60 msgr2=0x7f7570069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.724+0000 7f756ffff640 1 --2- 172.21.15.83:0/2603951484 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7570066f60 0x7f7570069420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f75841808c0 tx=0x7f757c008040 comp rx=0 tx=0).stop 2024-04-17T10:38:21.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.724+0000 7f4563743640 1 -- 172.21.15.83:0/1583498103 --> [v2:172.21.15.167:6812/1109787903,v1:172.21.15.167:6815/1109787903] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f455c074f90 con 0x7f454006f160 2024-04-17T10:38:21.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.724+0000 7f756ffff640 1 -- 172.21.15.83:0/2603951484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7584053620 msgr2=0x7f75840d9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.724+0000 7f756ffff640 1 --2- 172.21.15.83:0/2603951484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7584053620 0x7f75840d9cf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f758c07ad60 tx=0x7f758c069310 comp rx=0 tx=0).stop 2024-04-17T10:38:21.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.724+0000 7f08cdffb640 1 -- 172.21.15.83:0/2511271651 shutdown_connections 2024-04-17T10:38:21.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.724+0000 7f75925de640 1 -- 172.21.15.83:0/2603951484 reap_dead start 2024-04-17T10:38:21.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.724+0000 7f08cdffb640 1 -- 172.21.15.83:0/2511271651 wait complete. 2024-04-17T10:38:21.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.724+0000 7f756ffff640 1 -- 172.21.15.83:0/2603951484 shutdown_connections 2024-04-17T10:38:21.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.724+0000 7f756ffff640 1 -- 172.21.15.83:0/2603951484 >> 172.21.15.83:0/2603951484 conn(0x7f7584009fc0 msgr2=0x7f75840e08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:21.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.724+0000 7f756ffff640 1 -- 172.21.15.83:0/2603951484 shutdown_connections 2024-04-17T10:38:21.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.724+0000 7f756ffff640 1 -- 172.21.15.83:0/2603951484 wait complete. 2024-04-17T10:38:21.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7fecaebef640 1 -- 172.21.15.83:0/1442468215 >> [v2:172.21.15.83:6816/3642321714,v1:172.21.15.83:6818/3642321714] conn(0x7fec9406f130 msgr2=0x7fec94071550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7fecaebef640 1 --2- 172.21.15.83:0/1442468215 >> [v2:172.21.15.83:6816/3642321714,v1:172.21.15.83:6818/3642321714] conn(0x7fec9406f130 0x7fec94071550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7f4551ffb640 1 -- 172.21.15.83:0/1583498103 <== osd.6 v2:172.21.15.167:6812/1109787903 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f455c074f90 con 0x7f454006f160 2024-04-17T10:38:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7fecaebef640 1 -- 172.21.15.83:0/1442468215 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fec94066f90 msgr2=0x7fec94069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7fecaebef640 1 --2- 172.21.15.83:0/1442468215 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fec94066f90 0x7fec94069450 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7feca81587b0 tx=0x7feca0001700 comp rx=0 tx=0).stop 2024-04-17T10:38:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7f4e88fda640 1 -- 172.21.15.83:0/2367553727 --> [v2:172.21.15.83:6800/367778918,v1:172.21.15.83:6801/367778918] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4e64002de0 con 0x7f4e7006f100 2024-04-17T10:38:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7fecaebef640 1 -- 172.21.15.83:0/1442468215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feca806e500 msgr2=0x7feca80c88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7fecaebef640 1 --2- 172.21.15.83:0/1442468215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feca806e500 0x7feca80c88b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0ab000 tx=0x7fec9c01ad10 comp rx=0 tx=0).stop 2024-04-17T10:38:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7f4563743640 1 -- 172.21.15.83:0/1583498103 >> [v2:172.21.15.167:6812/1109787903,v1:172.21.15.167:6815/1109787903] conn(0x7f454006f160 msgr2=0x7f4540071580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7f4563743640 1 --2- 172.21.15.83:0/1583498103 >> [v2:172.21.15.167:6812/1109787903,v1:172.21.15.167:6815/1109787903] conn(0x7f454006f160 0x7f4540071580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7f4563743640 1 -- 172.21.15.83:0/1583498103 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4540066f60 msgr2=0x7f4540069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7f4563743640 1 --2- 172.21.15.83:0/1583498103 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4540066f60 0x7f4540069420 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4558009e20 tx=0x7f4558009210 comp rx=0 tx=0).stop 2024-04-17T10:38:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7fecad18c640 1 -- 172.21.15.83:0/1442468215 reap_dead start 2024-04-17T10:38:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7fecaebef640 1 -- 172.21.15.83:0/1442468215 shutdown_connections 2024-04-17T10:38:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7f4563743640 1 -- 172.21.15.83:0/1583498103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f455c15f3b0 msgr2=0x7f455c1431d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7f4e6bfff640 1 -- 172.21.15.83:0/2367553727 <== osd.3 v2:172.21.15.83:6800/367778918 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4e64002de0 con 0x7f4e7006f100 2024-04-17T10:38:21.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7f4563743640 1 --2- 172.21.15.83:0/1583498103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f455c15f3b0 0x7f455c1431d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f454c00c460 tx=0x7f454c00c920 comp rx=0 tx=0).stop 2024-04-17T10:38:21.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7fecaebef640 1 -- 172.21.15.83:0/1442468215 >> 172.21.15.83:0/1442468215 conn(0x7feca80d17e0 msgr2=0x7feca80b0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:21.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7f4561ce0640 1 -- 172.21.15.83:0/1583498103 reap_dead start 2024-04-17T10:38:21.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7f4563743640 1 -- 172.21.15.83:0/1583498103 shutdown_connections 2024-04-17T10:38:21.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7fecaebef640 1 -- 172.21.15.83:0/1442468215 shutdown_connections 2024-04-17T10:38:21.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7f4563743640 1 -- 172.21.15.83:0/1583498103 >> 172.21.15.83:0/1583498103 conn(0x7f455c0cf630 msgr2=0x7f455c15da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7fecaebef640 1 -- 172.21.15.83:0/1442468215 wait complete. 2024-04-17T10:38:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7f4e88fda640 1 -- 172.21.15.83:0/2367553727 >> [v2:172.21.15.83:6800/367778918,v1:172.21.15.83:6801/367778918] conn(0x7f4e7006f100 msgr2=0x7f4e70071520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7f4563743640 1 -- 172.21.15.83:0/1583498103 shutdown_connections 2024-04-17T10:38:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7f4e88fda640 1 --2- 172.21.15.83:0/2367553727 >> [v2:172.21.15.83:6800/367778918,v1:172.21.15.83:6801/367778918] conn(0x7f4e7006f100 0x7f4e70071520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7f4563743640 1 -- 172.21.15.83:0/1583498103 wait complete. 2024-04-17T10:38:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7f4e88fda640 1 -- 172.21.15.83:0/2367553727 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4e70066f60 msgr2=0x7f4e70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7f4e88fda640 1 --2- 172.21.15.83:0/2367553727 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4e70066f60 0x7f4e70069420 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0b6000 tx=0x7f4e7c0af070 comp rx=0 tx=0).stop 2024-04-17T10:38:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7f4e88fda640 1 -- 172.21.15.83:0/2367553727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e84143c10 msgr2=0x7f4e84130a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7f4e88fda640 1 --2- 172.21.15.83:0/2367553727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e84143c10 0x7f4e84130a80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4e7400efc0 tx=0x7f4e7400c490 comp rx=0 tx=0).stop 2024-04-17T10:38:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7f4e83fff640 1 -- 172.21.15.83:0/2367553727 reap_dead start 2024-04-17T10:38:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7f4e88fda640 1 -- 172.21.15.83:0/2367553727 shutdown_connections 2024-04-17T10:38:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7f4e88fda640 1 -- 172.21.15.83:0/2367553727 >> 172.21.15.83:0/2367553727 conn(0x7f4e8406abe0 msgr2=0x7f4e840b0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7f4e88fda640 1 -- 172.21.15.83:0/2367553727 shutdown_connections 2024-04-17T10:38:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7f4e88fda640 1 -- 172.21.15.83:0/2367553727 wait complete. 2024-04-17T10:38:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7f24c351c640 1 -- 172.21.15.83:0/3951976890 --> [v2:172.21.15.83:6817/2375732,v1:172.21.15.83:6819/2375732] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f24b40053a0 con 0x7f24a006f160 2024-04-17T10:38:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.728+0000 7f24b2ffd640 1 -- 172.21.15.83:0/3951976890 <== osd.0 v2:172.21.15.83:6817/2375732 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f24b40053a0 con 0x7f24a006f160 2024-04-17T10:38:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.736+0000 7f5b04f53640 1 -- 172.21.15.83:0/2408881409 --> [v2:172.21.15.167:6800/1776227116,v1:172.21.15.167:6801/1776227116] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5b00074f90 con 0x7f5ae006f100 2024-04-17T10:38:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.736+0000 7f24c351c640 1 -- 172.21.15.83:0/3951976890 >> [v2:172.21.15.83:6817/2375732,v1:172.21.15.83:6819/2375732] conn(0x7f24a006f160 msgr2=0x7f24a0071580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.736+0000 7f24c351c640 1 --2- 172.21.15.83:0/3951976890 >> [v2:172.21.15.83:6817/2375732,v1:172.21.15.83:6819/2375732] conn(0x7f24a006f160 0x7f24a0071580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.736+0000 7f24c351c640 1 -- 172.21.15.83:0/3951976890 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f24a0066f60 msgr2=0x7f24a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.736+0000 7f24c351c640 1 --2- 172.21.15.83:0/3951976890 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f24a0066f60 0x7f24a0069420 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f24b80ab000 tx=0x7f24b8001980 comp rx=0 tx=0).stop 2024-04-17T10:38:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.736+0000 7f5aeeffd640 1 -- 172.21.15.83:0/2408881409 <== osd.5 v2:172.21.15.167:6800/1776227116 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5b00074f90 con 0x7f5ae006f100 2024-04-17T10:38:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.736+0000 7f24c351c640 1 -- 172.21.15.83:0/3951976890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24b4036b30 msgr2=0x7f24b4189780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.736+0000 7f24c351c640 1 --2- 172.21.15.83:0/3951976890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24b4036b30 0x7f24b4189780 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f24bc06a390 tx=0x7f24bc068a40 comp rx=0 tx=0).stop 2024-04-17T10:38:21.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.736+0000 7f24c2d1b640 1 -- 172.21.15.83:0/3951976890 reap_dead start 2024-04-17T10:38:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.736+0000 7f24c351c640 1 -- 172.21.15.83:0/3951976890 shutdown_connections 2024-04-17T10:38:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.736+0000 7f24c351c640 1 -- 172.21.15.83:0/3951976890 >> 172.21.15.83:0/3951976890 conn(0x7f24b4009fc0 msgr2=0x7f24b40471d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.736+0000 7f24c351c640 1 -- 172.21.15.83:0/3951976890 shutdown_connections 2024-04-17T10:38:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.736+0000 7f24c351c640 1 -- 172.21.15.83:0/3951976890 wait complete. 2024-04-17T10:38:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.740+0000 7f082359e640 1 -- 172.21.15.83:0/1544570610 --> [v2:172.21.15.167:6808/285324536,v1:172.21.15.167:6809/285324536] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0808002d60 con 0x7f080406f280 2024-04-17T10:38:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.740+0000 7f0812ffd640 1 -- 172.21.15.83:0/1544570610 <== osd.7 v2:172.21.15.167:6808/285324536 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0808002d60 con 0x7f080406f280 2024-04-17T10:38:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.740+0000 7f082359e640 1 -- 172.21.15.83:0/1544570610 >> [v2:172.21.15.167:6808/285324536,v1:172.21.15.167:6809/285324536] conn(0x7f080406f280 msgr2=0x7f08040716a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.740+0000 7f082359e640 1 --2- 172.21.15.83:0/1544570610 >> [v2:172.21.15.167:6808/285324536,v1:172.21.15.167:6809/285324536] conn(0x7f080406f280 0x7f08040716a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.740+0000 7f082359e640 1 -- 172.21.15.83:0/1544570610 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0804067060 msgr2=0x7f0804069520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.740+0000 7f082359e640 1 --2- 172.21.15.83:0/1544570610 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0804067060 0x7f0804069520 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f081c04b6e0 tx=0x7f0818009040 comp rx=0 tx=0).stop 2024-04-17T10:38:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.740+0000 7f082359e640 1 -- 172.21.15.83:0/1544570610 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f081c00aae0 msgr2=0x7f081c04db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.740+0000 7f082359e640 1 --2- 172.21.15.83:0/1544570610 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f081c00aae0 0x7f081c04db30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0824068370 tx=0x7f082404f630 comp rx=0 tx=0).stop 2024-04-17T10:38:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.740+0000 7f0822d9d640 1 -- 172.21.15.83:0/1544570610 reap_dead start 2024-04-17T10:38:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.740+0000 7f082359e640 1 -- 172.21.15.83:0/1544570610 shutdown_connections 2024-04-17T10:38:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.740+0000 7f082359e640 1 -- 172.21.15.83:0/1544570610 >> 172.21.15.83:0/1544570610 conn(0x7f081c009fc0 msgr2=0x7f081c034d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.740+0000 7f082359e640 1 -- 172.21.15.83:0/1544570610 shutdown_connections 2024-04-17T10:38:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.740+0000 7f082359e640 1 -- 172.21.15.83:0/1544570610 wait complete. 2024-04-17T10:38:21.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.752+0000 7f5b04f53640 1 -- 172.21.15.83:0/2408881409 >> [v2:172.21.15.167:6800/1776227116,v1:172.21.15.167:6801/1776227116] conn(0x7f5ae006f100 msgr2=0x7f5ae0071520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.752+0000 7f5b04f53640 1 --2- 172.21.15.83:0/2408881409 >> [v2:172.21.15.167:6800/1776227116,v1:172.21.15.167:6801/1776227116] conn(0x7f5ae006f100 0x7f5ae0071520 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.752+0000 7f5b04f53640 1 -- 172.21.15.83:0/2408881409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5ae0066f60 msgr2=0x7f5ae0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.752+0000 7f5b04f53640 1 --2- 172.21.15.83:0/2408881409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5ae0066f60 0x7f5ae0069420 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5af4001600 tx=0x7f5af4007a50 comp rx=0 tx=0).stop 2024-04-17T10:38:21.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.752+0000 7f5b04f53640 1 -- 172.21.15.83:0/2408881409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b000aed60 msgr2=0x7f5b0015b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:21.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.752+0000 7f5b04f53640 1 --2- 172.21.15.83:0/2408881409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b000aed60 0x7f5b0015b430 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5b0006e710 tx=0x7f5af001acd0 comp rx=0 tx=0).stop 2024-04-17T10:38:21.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.752+0000 7f5afed9d640 1 -- 172.21.15.83:0/2408881409 reap_dead start 2024-04-17T10:38:21.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.756+0000 7f5b04f53640 1 -- 172.21.15.83:0/2408881409 shutdown_connections 2024-04-17T10:38:21.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.756+0000 7f5b04f53640 1 -- 172.21.15.83:0/2408881409 >> 172.21.15.83:0/2408881409 conn(0x7f5b0006abe0 msgr2=0x7f5b0006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:21.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.756+0000 7f5b04f53640 1 -- 172.21.15.83:0/2408881409 shutdown_connections 2024-04-17T10:38:21.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.756+0000 7f5b04f53640 1 -- 172.21.15.83:0/2408881409 wait complete. 2024-04-17T10:38:21.776 INFO:teuthology.orchestra.run.smithi083.stdout:55834574855 2024-04-17T10:38:21.776 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-17T10:38:21.777 INFO:teuthology.orchestra.run.smithi083.stdout:55834574855 2024-04-17T10:38:21.777 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-17T10:38:21.778 INFO:teuthology.orchestra.run.smithi083.stdout:55834574855 2024-04-17T10:38:21.778 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-17T10:38:21.779 INFO:teuthology.orchestra.run.smithi083.stdout:55834574857 2024-04-17T10:38:21.779 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-17T10:38:21.780 INFO:teuthology.orchestra.run.smithi083.stdout:55834574855 2024-04-17T10:38:21.780 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-17T10:38:21.781 INFO:teuthology.orchestra.run.smithi083.stdout:55834574855 2024-04-17T10:38:21.781 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-17T10:38:21.782 INFO:teuthology.orchestra.run.smithi083.stdout:55834574856 2024-04-17T10:38:21.782 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-17T10:38:21.783 INFO:teuthology.orchestra.run.smithi083.stdout:55834574856 2024-04-17T10:38:21.784 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-17T10:38:21.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.992+0000 7f4623548640 1 Processor -- start 2024-04-17T10:38:21.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.992+0000 7f4623548640 1 -- start start 2024-04-17T10:38:21.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.992+0000 7fedefbb9640 1 Processor -- start 2024-04-17T10:38:21.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.992+0000 7f3dcdea0640 1 Processor -- start 2024-04-17T10:38:21.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.992+0000 7fedefbb9640 1 -- start start 2024-04-17T10:38:21.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.992+0000 7f3dcdea0640 1 -- start start 2024-04-17T10:38:21.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.992+0000 7f4ca30d6640 1 Processor -- start 2024-04-17T10:38:21.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.992+0000 7f4ca30d6640 1 -- start start 2024-04-17T10:38:21.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7ff080c54640 1 Processor -- start 2024-04-17T10:38:21.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7fc82f26c640 1 Processor -- start 2024-04-17T10:38:21.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7fc82f26c640 1 -- start start 2024-04-17T10:38:21.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f400c8ef640 1 Processor -- start 2024-04-17T10:38:21.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7ff080c54640 1 -- start start 2024-04-17T10:38:21.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f4623548640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f461c15db60 0x7f461c0b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f4622546640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f461c071550 0x7f461c15d450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39900/0 (socket says 172.21.15.83:39900) 2024-04-17T10:38:21.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f3dcdea0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3dc806c970 0x7f3dc80b7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7fedefbb9640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fede0008330 0x7fede00379c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7fedefbb9640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fede0007440 con 0x7fede00e30a0 2024-04-17T10:38:21.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7fedefbb9640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fede0056440 con 0x7fede00e2cc0 2024-04-17T10:38:21.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7fedefbb9640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fede0037530 con 0x7fede0008330 2024-04-17T10:38:21.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f4622546640 1 -- 172.21.15.83:0/2700362413 learned_addr learned my addr 172.21.15.83:0/2700362413 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:21.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f4623548640 1 --2- 172.21.15.83:0/2700362413 >> v2:172.21.15.83:3300/0 conn(0x7f461c071930 0x7f461c071d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f4623548640 1 -- 172.21.15.83:0/2700362413 --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f461c0b27e0 con 0x7f461c071550 2024-04-17T10:38:21.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f3dcdea0640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3dc806cd50 0x7f3dc80b2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f3dcdea0640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dc81627a0 con 0x7f3dc80b2cc0 2024-04-17T10:38:21.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f3dcdea0640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dc81633b0 con 0x7f3dc806c970 2024-04-17T10:38:21.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f3dcdea0640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dc80b7e50 con 0x7f3dc806cd50 2024-04-17T10:38:21.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f400c8ef640 1 -- start start 2024-04-17T10:38:21.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7feded154640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fede00e30a0 0x7fede0007c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39916/0 (socket says 172.21.15.83:39916) 2024-04-17T10:38:21.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7feded154640 1 -- 172.21.15.83:0/3364253324 learned_addr learned my addr 172.21.15.83:0/3364253324 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:21.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f4623548640 1 -- 172.21.15.83:0/2700362413 --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f461c0b2ac0 con 0x7f461c071930 2024-04-17T10:38:21.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f4623548640 1 -- 172.21.15.83:0/2700362413 --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f461c0b2da0 con 0x7f461c15db60 2024-04-17T10:38:21.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f4621d45640 1 --2- 172.21.15.83:0/2700362413 >> v2:172.21.15.167:3300/0 conn(0x7f461c15db60 0x7f461c0b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f3dcd69f640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f3dc80b2cc0 0x7f3dc8165d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:52194/0 (socket says 172.21.15.83:52194) 2024-04-17T10:38:21.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f3dcd69f640 1 -- 172.21.15.83:0/3092611283 learned_addr learned my addr 172.21.15.83:0/3092611283 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:21.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7fedee957640 1 --2- 172.21.15.83:0/3364253324 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fede0008330 0x7fede00379c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7fedec953640 1 -- 172.21.15.83:0/3364253324 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 138900962 0 0) 0x7fede0007440 con 0x7fede00e30a0 2024-04-17T10:38:21.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7fedec953640 1 -- 172.21.15.83:0/3364253324 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedd4003530 con 0x7fede00e30a0 2024-04-17T10:38:21.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7fc82f26c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc8281669f0 0x7fc828168de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:21.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7fc82f26c640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8281625d0 con 0x7fc82806ae10 2024-04-17T10:38:21.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7fc82f26c640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc828161810 con 0x7fc82815e330 2024-04-17T10:38:21.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7fc82f26c640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc82815eeb0 con 0x7fc8281669f0 2024-04-17T10:38:21.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f4622d47640 1 --2- 172.21.15.83:0/2700362413 >> v2:172.21.15.83:3300/0 conn(0x7f461c071930 0x7f461c071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:21.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f3dcce9e640 1 --2- 172.21.15.83:0/3092611283 >> v2:172.21.15.83:3300/0 conn(0x7f3dc806c970 0x7f3dc80b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f3dc7fff640 1 --2- 172.21.15.83:0/3092611283 >> v2:172.21.15.167:3300/0 conn(0x7f3dc806cd50 0x7f3dc80b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f4ca30d6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c9c0d0010 0x7f4c9c1624c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f4ca30d6640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4c9c06b2f0 0x7f4c9c0b5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f4ca30d6640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c9c0d03f0 con 0x7f4c9c0b54c0 2024-04-17T10:38:22.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f4ca30d6640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c9c0d1570 con 0x7f4c9c06b2f0 2024-04-17T10:38:22.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f4ca30d6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c0d09f0 con 0x7f4c9c0d0010 2024-04-17T10:38:22.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f4ca0e72640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c9c0d0010 0x7f4c9c1624c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f4ca0e72640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c9c0d0010 0x7f4c9c1624c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:59646/0 (socket says 172.21.15.83:59646) 2024-04-17T10:38:22.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f4ca0e72640 1 -- 172.21.15.83:0/3711461858 learned_addr learned my addr 172.21.15.83:0/3711461858 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:22.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7fedec953640 1 -- 172.21.15.83:0/3364253324 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1193489677 0 0) 0x7fedd4003530 con 0x7fede00e30a0 2024-04-17T10:38:22.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7fedec953640 1 -- 172.21.15.83:0/3364253324 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fede0008330 msgr2=0x7fede00379c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:22.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7fedec953640 1 --2- 172.21.15.83:0/3364253324 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fede0008330 0x7fede00379c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7fedec953640 1 -- 172.21.15.83:0/3364253324 >> v1:172.21.15.167:6789/0 conn(0x7fede00e2cc0 legacy=0x7fede0006d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f400c8ef640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4008165d50 0x7f40080b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f400c8ef640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4008166cb0 con 0x7f40080ce5b0 2024-04-17T10:38:22.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f400c8ef640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40081666f0 con 0x7f40081661d0 2024-04-17T10:38:22.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f400c8ef640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4008162c50 con 0x7f4008165d50 2024-04-17T10:38:22.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f4621d45640 1 -- 172.21.15.83:0/2700362413 >> v2:172.21.15.83:3300/0 conn(0x7f461c071930 msgr2=0x7f461c071d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:22.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f4621d45640 1 --2- 172.21.15.83:0/2700362413 >> v2:172.21.15.83:3300/0 conn(0x7f461c071930 0x7f461c071d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f4621d45640 1 -- 172.21.15.83:0/2700362413 >> v1:172.21.15.83:6789/0 conn(0x7f461c071550 legacy=0x7f461c15d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f4621d45640 1 -- 172.21.15.83:0/2700362413 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461c0b3080 con 0x7f461c15db60 2024-04-17T10:38:22.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f3dc77fe640 1 -- 172.21.15.83:0/3092611283 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3139237608 0 0) 0x7f3dc81627a0 con 0x7f3dc80b2cc0 2024-04-17T10:38:22.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f3dc77fe640 1 -- 172.21.15.83:0/3092611283 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3db0003530 con 0x7f3dc80b2cc0 2024-04-17T10:38:22.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f0916724640 1 Processor -- start 2024-04-17T10:38:22.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f0916724640 1 -- start start 2024-04-17T10:38:22.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7ff080c54640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff07c15db60 0x7ff07c0b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7ff080c54640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff07c071930 0x7ff07c071d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7ff080c54640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff07c0b27e0 con 0x7ff07c071550 2024-04-17T10:38:22.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f4ca1673640 1 --2- 172.21.15.83:0/3711461858 >> v2:172.21.15.167:3300/0 conn(0x7f4c9c06b2f0 0x7f4c9c0b5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7ff080c54640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff07c0b2ac0 con 0x7ff07c15db60 2024-04-17T10:38:22.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7ff080c54640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c0b2da0 con 0x7ff07c071930 2024-04-17T10:38:22.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7fc82d809640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc8281669f0 0x7fc828168de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7fc82d809640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc8281669f0 0x7fc828168de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48076/0 (socket says 172.21.15.83:48076) 2024-04-17T10:38:22.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7fc82d809640 1 -- 172.21.15.83:0/1895076002 learned_addr learned my addr 172.21.15.83:0/1895076002 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:22.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f3dc7fff640 1 -- 172.21.15.83:0/3092611283 >> v2:172.21.15.83:3300/0 conn(0x7f3dc806c970 msgr2=0x7f3dc80b7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:22.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f3dc7fff640 1 --2- 172.21.15.83:0/3092611283 >> v2:172.21.15.83:3300/0 conn(0x7f3dc806c970 0x7f3dc80b7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f3dc7fff640 1 -- 172.21.15.83:0/3092611283 >> v1:172.21.15.167:6789/0 conn(0x7f3dc80b2cc0 legacy=0x7f3dc8165d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f3dc7fff640 1 -- 172.21.15.83:0/3092611283 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc8162a80 con 0x7f3dc806cd50 2024-04-17T10:38:22.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f3dcce9e640 1 --2- 172.21.15.83:0/3092611283 >> v2:172.21.15.83:3300/0 conn(0x7f3dc806c970 0x7f3dc80b7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:38:22.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7ff07affd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff07c15db60 0x7ff07c0b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7ff07bfff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff07c071930 0x7ff07c071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7ff07bfff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff07c071930 0x7ff07c071d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:59672/0 (socket says 172.21.15.83:59672) 2024-04-17T10:38:22.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7ff07bfff640 1 -- 172.21.15.83:0/577386122 learned_addr learned my addr 172.21.15.83:0/577386122 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:22.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7ff07affd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff07c15db60 0x7ff07c0b3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48078/0 (socket says 172.21.15.83:48078) 2024-04-17T10:38:22.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7fedec953640 1 -- 172.21.15.83:0/3364253324 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede0055e80 con 0x7fede00e30a0 2024-04-17T10:38:22.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7fedec953640 1 -- 172.21.15.83:0/3364253324 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fede804f450 con 0x7fede00e30a0 2024-04-17T10:38:22.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7fedec953640 1 -- 172.21.15.83:0/3364253324 >> v1:172.21.15.83:6789/0 conn(0x7fede00e30a0 legacy=0x7fede0007c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7fedec953640 1 --2- 172.21.15.83:0/3364253324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd40039f0 0x7fedd4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7fedec953640 1 --2- 172.21.15.83:0/3364253324 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fede0008330 0x7fedd4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7fedec953640 1 --2- 172.21.15.83:0/3364253324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedd4004d50 0x7fedd40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7fedec953640 1 -- 172.21.15.83:0/3364253324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede0037530 con 0x7fedd40039f0 2024-04-17T10:38:22.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7fedec953640 1 -- 172.21.15.83:0/3364253324 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd400b9d0 con 0x7fedd4004d50 2024-04-17T10:38:22.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7fedec953640 1 -- 172.21.15.83:0/3364253324 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd400bb50 con 0x7fede0008330 2024-04-17T10:38:22.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7feded955640 1 --2- 172.21.15.83:0/3364253324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd40039f0 0x7fedd4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7fc82d809640 1 -- 172.21.15.83:0/1895076002 >> v1:172.21.15.167:6789/0 conn(0x7fc82815e330 legacy=0x7fc82815e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7fc82d809640 1 -- 172.21.15.83:0/1895076002 >> v1:172.21.15.83:6789/0 conn(0x7fc82806ae10 legacy=0x7fc82806b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7fc82d809640 1 -- 172.21.15.83:0/1895076002 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc828161af0 con 0x7fc8281669f0 2024-04-17T10:38:22.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7ff07a7fc640 1 -- 172.21.15.83:0/577386122 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1617753062 0 0) 0x7ff07c0b27e0 con 0x7ff07c071550 2024-04-17T10:38:22.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7ff07a7fc640 1 -- 172.21.15.83:0/577386122 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff064003530 con 0x7ff07c071550 2024-04-17T10:38:22.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7feded154640 1 --2- 172.21.15.83:0/3364253324 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fede0008330 0x7fedd4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7ff07a7fc640 1 -- 172.21.15.83:0/577386122 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2878292685 0 0) 0x7ff064003530 con 0x7ff07c071550 2024-04-17T10:38:22.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7ff07a7fc640 1 -- 172.21.15.83:0/577386122 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff07c071930 msgr2=0x7ff07c071d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:22.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7ff07a7fc640 1 --2- 172.21.15.83:0/577386122 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff07c071930 0x7ff07c071d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7ff07a7fc640 1 -- 172.21.15.83:0/577386122 >> v2:172.21.15.83:3300/0 conn(0x7ff07c15db60 msgr2=0x7ff07c0b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:22.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7ff07a7fc640 1 --2- 172.21.15.83:0/577386122 >> v2:172.21.15.83:3300/0 conn(0x7ff07c15db60 0x7ff07c0b3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7ff07a7fc640 1 -- 172.21.15.83:0/577386122 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff07c0b3080 con 0x7ff07c071550 2024-04-17T10:38:22.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7ff07a7fc640 1 -- 172.21.15.83:0/577386122 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7ff06c001680 con 0x7ff07c071550 2024-04-17T10:38:22.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7ff07a7fc640 1 -- 172.21.15.83:0/577386122 >> v1:172.21.15.83:6789/0 conn(0x7ff07c071550 legacy=0x7ff07c15d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7ff07a7fc640 1 --2- 172.21.15.83:0/577386122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0640039c0 0x7ff064003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7ff07a7fc640 1 --2- 172.21.15.83:0/577386122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff064004810 0x7ff064004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7ff07a7fc640 1 --2- 172.21.15.83:0/577386122 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0640051d0 0x7ff06400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7ff07a7fc640 1 -- 172.21.15.83:0/577386122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c0b2ac0 con 0x7ff0640039c0 2024-04-17T10:38:22.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7ff07a7fc640 1 -- 172.21.15.83:0/577386122 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c0b2da0 con 0x7ff064004810 2024-04-17T10:38:22.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7ff07a7fc640 1 -- 172.21.15.83:0/577386122 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff06400bf10 con 0x7ff0640051d0 2024-04-17T10:38:22.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7ff07b7fe640 1 --2- 172.21.15.83:0/577386122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0640039c0 0x7ff064003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7feded154640 1 -- 172.21.15.83:0/3364253324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedd4004d50 msgr2=0x7fedd40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:22.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7feded154640 1 --2- 172.21.15.83:0/3364253324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedd4004d50 0x7fedd40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7feded154640 1 -- 172.21.15.83:0/3364253324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd40039f0 msgr2=0x7fedd4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:22.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7feded154640 1 --2- 172.21.15.83:0/3364253324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd40039f0 0x7fedd4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7feded154640 1 -- 172.21.15.83:0/3364253324 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede0055e80 con 0x7fede0008330 2024-04-17T10:38:22.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f4005d9b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4008165d50 0x7f40080b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f400659c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f40080ce5b0 0x7f40081630e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39946/0 (socket says 172.21.15.83:39946) 2024-04-17T10:38:22.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f400659c640 1 -- 172.21.15.83:0/377232751 learned_addr learned my addr 172.21.15.83:0/377232751 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:22.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f4005d9b640 1 -- 172.21.15.83:0/377232751 >> v1:172.21.15.167:6789/0 conn(0x7f40081661d0 legacy=0x7f40080d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f4005d9b640 1 -- 172.21.15.83:0/377232751 >> v1:172.21.15.83:6789/0 conn(0x7f40080ce5b0 legacy=0x7f40081630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f4005d9b640 1 -- 172.21.15.83:0/377232751 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40081669d0 con 0x7f4008165d50 2024-04-17T10:38:22.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f4ca0e72640 1 -- 172.21.15.83:0/3711461858 >> v2:172.21.15.167:3300/0 conn(0x7f4c9c06b2f0 msgr2=0x7f4c9c0b5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:22.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f4ca0e72640 1 --2- 172.21.15.83:0/3711461858 >> v2:172.21.15.167:3300/0 conn(0x7f4c9c06b2f0 0x7f4c9c0b5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f4ca0e72640 1 -- 172.21.15.83:0/3711461858 >> v1:172.21.15.167:6789/0 conn(0x7f4c9c0b54c0 legacy=0x7f4c9c06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:22.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:21.996+0000 7f4ca0e72640 1 -- 172.21.15.83:0/3711461858 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9c0d0cd0 con 0x7f4c9c0d0010 2024-04-17T10:38:22.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f0916724640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0910159b40 0x7f09100b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f0916724640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f09100b3650 0x7f091006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f0916724640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09100b2bb0 con 0x7f09100b3ae0 2024-04-17T10:38:22.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f0916724640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f091015e550 con 0x7f09100b3650 2024-04-17T10:38:22.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f0916724640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09100b31c0 con 0x7f0910159b40 2024-04-17T10:38:22.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7ff07b7fe640 1 -- 172.21.15.83:0/577386122 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0640051d0 msgr2=0x7ff06400b9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T10:38:22.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7ff07b7fe640 1 --2- 172.21.15.83:0/577386122 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0640051d0 0x7ff06400b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7ff07b7fe640 1 -- 172.21.15.83:0/577386122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff064004810 msgr2=0x7ff064004c90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T10:38:22.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7ff07b7fe640 1 --2- 172.21.15.83:0/577386122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff064004810 0x7ff064004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7ff07b7fe640 1 -- 172.21.15.83:0/577386122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff064003530 con 0x7ff0640039c0 2024-04-17T10:38:22.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f4621d45640 1 --2- 172.21.15.83:0/2700362413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f461c15db60 0x7f461c0b3b60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f461c0af0e0 tx=0x7f461801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41d1dc776a7be982 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:22.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f4ca0e72640 1 --2- 172.21.15.83:0/3711461858 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c9c0d0010 0x7f4c9c1624c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009870 tx=0x7f4c8c01c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de5178b5f4557b24 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:22.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f0914cc1640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f09100b3ae0 0x7f09100ccee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39960/0 (socket says 172.21.15.83:39960) 2024-04-17T10:38:22.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f0914cc1640 1 -- 172.21.15.83:0/923025404 learned_addr learned my addr 172.21.15.83:0/923025404 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:22.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f090f7fe640 1 --2- 172.21.15.83:0/923025404 >> v2:172.21.15.83:3300/0 conn(0x7f09100b3650 0x7f091006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f090ffff640 1 --2- 172.21.15.83:0/923025404 >> v2:172.21.15.167:3300/0 conn(0x7f0910159b40 0x7f09100b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f3dc7fff640 1 --2- 172.21.15.83:0/3092611283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3dc806cd50 0x7f3dc80b2780 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3dc808c2e0 tx=0x7f3dc001c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92d175419f8ecb93 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:22.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f4c9b7fe640 1 -- 172.21.15.83:0/3711461858 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c0b1070 con 0x7f4c9c0d0010 2024-04-17T10:38:22.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7fc82d809640 1 --2- 172.21.15.83:0/1895076002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8281669f0 0x7fc828168de0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc820009e50 tx=0x7fc82001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d89afabded4fb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:22.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f4621544640 1 -- 172.21.15.83:0/2700362413 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46180ab650 con 0x7f461c15db60 2024-04-17T10:38:22.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f4621544640 1 -- 172.21.15.83:0/2700362413 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46180b0070 con 0x7f461c15db60 2024-04-17T10:38:22.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f4621544640 1 -- 172.21.15.83:0/2700362413 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46180aba10 con 0x7f461c15db60 2024-04-17T10:38:22.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f090f7fe640 1 -- 172.21.15.83:0/923025404 >> v2:172.21.15.167:3300/0 conn(0x7f0910159b40 msgr2=0x7f09100b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:22.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f090f7fe640 1 --2- 172.21.15.83:0/923025404 >> v2:172.21.15.167:3300/0 conn(0x7f0910159b40 0x7f09100b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f090f7fe640 1 -- 172.21.15.83:0/923025404 >> v1:172.21.15.83:6789/0 conn(0x7f09100b3ae0 legacy=0x7f09100ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f090f7fe640 1 -- 172.21.15.83:0/923025404 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091015e270 con 0x7f09100b3650 2024-04-17T10:38:22.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f090ffff640 1 --2- 172.21.15.83:0/923025404 >> v2:172.21.15.167:3300/0 conn(0x7f0910159b40 0x7f09100b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:38:22.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f4005d9b640 1 --2- 172.21.15.83:0/377232751 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4008165d50 0x7f40080b5450 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc0097f0 tx=0x7f3ffc01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=185527a09be4f081 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:22.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f400559a640 1 -- 172.21.15.83:0/377232751 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc0b0070 con 0x7f4008165d50 2024-04-17T10:38:22.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f400559a640 1 -- 172.21.15.83:0/377232751 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ffc0ab600 con 0x7f4008165d50 2024-04-17T10:38:22.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f400559a640 1 -- 172.21.15.83:0/377232751 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc0ab9d0 con 0x7f4008165d50 2024-04-17T10:38:22.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f4c9b7fe640 1 -- 172.21.15.83:0/3711461858 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8c0ac410 con 0x7f4c9c0d0010 2024-04-17T10:38:22.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f4c9b7fe640 1 -- 172.21.15.83:0/3711461858 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c0ac820 con 0x7f4c9c0d0010 2024-04-17T10:38:22.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f4ca30d6640 1 -- 172.21.15.83:0/3711461858 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c9c0d0010 msgr2=0x7f4c9c1624c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f4ca30d6640 1 --2- 172.21.15.83:0/3711461858 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c9c0d0010 0x7f4c9c1624c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009870 tx=0x7f4c8c01c3f0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f3dc77fe640 1 -- 172.21.15.83:0/3092611283 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc00b2070 con 0x7f3dc806cd50 2024-04-17T10:38:22.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f3dc77fe640 1 -- 172.21.15.83:0/3092611283 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc00a58e0 con 0x7f3dc806cd50 2024-04-17T10:38:22.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f3dc77fe640 1 -- 172.21.15.83:0/3092611283 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc00a5cb0 con 0x7f3dc806cd50 2024-04-17T10:38:22.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f4ca30d6640 1 -- 172.21.15.83:0/3711461858 shutdown_connections 2024-04-17T10:38:22.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f4ca30d6640 1 --2- 172.21.15.83:0/3711461858 >> v2:172.21.15.167:3300/0 conn(0x7f4c9c06b2f0 0x7f4c9c0b5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f4ca30d6640 1 --2- 172.21.15.83:0/3711461858 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c9c0d0010 0x7f4c9c1624c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f4ca30d6640 1 -- 172.21.15.83:0/3711461858 >> 172.21.15.83:0/3711461858 conn(0x7f4c9c06bcf0 msgr2=0x7f4c9c06c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:22.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f4ca30d6640 1 -- 172.21.15.83:0/3711461858 shutdown_connections 2024-04-17T10:38:22.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.000+0000 7f4ca30d6640 1 -- 172.21.15.83:0/3711461858 wait complete. 2024-04-17T10:38:22.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f400c8ef640 1 -- 172.21.15.83:0/377232751 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4008165d50 msgr2=0x7f40080b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f400c8ef640 1 --2- 172.21.15.83:0/377232751 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4008165d50 0x7f40080b5450 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc0097f0 tx=0x7f3ffc01c450 comp rx=0 tx=0).stop 2024-04-17T10:38:22.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7ff07b7fe640 1 --2- 172.21.15.83:0/577386122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0640039c0 0x7ff064003de0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff06c00c0e0 tx=0x7ff06c01ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=696116aca4ac87b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:22.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f400c8ef640 1 -- 172.21.15.83:0/377232751 shutdown_connections 2024-04-17T10:38:22.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f400c8ef640 1 --2- 172.21.15.83:0/377232751 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4008165d50 0x7f40080b5450 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f400c8ef640 1 -- 172.21.15.83:0/377232751 >> 172.21.15.83:0/377232751 conn(0x7f40080d0450 msgr2=0x7f40080d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:22.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f3dcdea0640 1 -- 172.21.15.83:0/3092611283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3dc806cd50 msgr2=0x7f3dc80b2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f3dcdea0640 1 --2- 172.21.15.83:0/3092611283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3dc806cd50 0x7f3dc80b2780 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3dc808c2e0 tx=0x7f3dc001c930 comp rx=0 tx=0).stop 2024-04-17T10:38:22.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f4623548640 1 -- 172.21.15.83:0/2700362413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f461c15db60 msgr2=0x7f461c0b3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f4623548640 1 --2- 172.21.15.83:0/2700362413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f461c15db60 0x7f461c0b3b60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f461c0af0e0 tx=0x7f461801c450 comp rx=0 tx=0).stop 2024-04-17T10:38:22.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f3dcdea0640 1 -- 172.21.15.83:0/3092611283 shutdown_connections 2024-04-17T10:38:22.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f3dcdea0640 1 --2- 172.21.15.83:0/3092611283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3dc806cd50 0x7f3dc80b2780 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3dc808c2e0 tx=0x7f3dc001c930 comp rx=0 tx=0).stop 2024-04-17T10:38:22.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f3dcdea0640 1 --2- 172.21.15.83:0/3092611283 >> v2:172.21.15.83:3300/0 conn(0x7f3dc806c970 0x7f3dc80b7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f3dcdea0640 1 -- 172.21.15.83:0/3092611283 >> 172.21.15.83:0/3092611283 conn(0x7f3dc80d3880 msgr2=0x7f3dc80d3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:22.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7fc81ffff640 1 -- 172.21.15.83:0/1895076002 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8200a5750 con 0x7fc8281669f0 2024-04-17T10:38:22.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f400c8ef640 1 -- 172.21.15.83:0/377232751 shutdown_connections 2024-04-17T10:38:22.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f400c8ef640 1 -- 172.21.15.83:0/377232751 wait complete. 2024-04-17T10:38:22.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f3dcdea0640 1 -- 172.21.15.83:0/3092611283 shutdown_connections 2024-04-17T10:38:22.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f3dcdea0640 1 -- 172.21.15.83:0/3092611283 wait complete. 2024-04-17T10:38:22.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7ff07a7fc640 1 -- 172.21.15.83:0/577386122 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c0b2070 con 0x7ff0640039c0 2024-04-17T10:38:22.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7ff07a7fc640 1 -- 172.21.15.83:0/577386122 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06c0a5920 con 0x7ff0640039c0 2024-04-17T10:38:22.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7ff07a7fc640 1 -- 172.21.15.83:0/577386122 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c007c00 con 0x7ff0640039c0 2024-04-17T10:38:22.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f090f7fe640 1 --2- 172.21.15.83:0/923025404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09100b3650 0x7f091006d6f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0910050040 tx=0x7f090401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff1a51d03594c74 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:22.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f090effd640 1 -- 172.21.15.83:0/923025404 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09040ab640 con 0x7f09100b3650 2024-04-17T10:38:22.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f090effd640 1 -- 172.21.15.83:0/923025404 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09040b0070 con 0x7f09100b3650 2024-04-17T10:38:22.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f090effd640 1 -- 172.21.15.83:0/923025404 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09040aba00 con 0x7f09100b3650 2024-04-17T10:38:22.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7fc81ffff640 1 -- 172.21.15.83:0/1895076002 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8200a58f0 con 0x7fc8281669f0 2024-04-17T10:38:22.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7fc81ffff640 1 -- 172.21.15.83:0/1895076002 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8200a5cc0 con 0x7fc8281669f0 2024-04-17T10:38:22.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7fc82f26c640 1 -- 172.21.15.83:0/1895076002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8281669f0 msgr2=0x7fc828168de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7fc82f26c640 1 --2- 172.21.15.83:0/1895076002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8281669f0 0x7fc828168de0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc820009e50 tx=0x7fc82001c9a0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7ff080c54640 1 -- 172.21.15.83:0/577386122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0640039c0 msgr2=0x7ff064003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7ff080c54640 1 --2- 172.21.15.83:0/577386122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0640039c0 0x7ff064003de0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff06c00c0e0 tx=0x7ff06c01ca90 comp rx=0 tx=0).stop 2024-04-17T10:38:22.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7feded154640 1 --2- 172.21.15.83:0/3364253324 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fede0008330 0x7fedd4004810 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fede807ad60 tx=0x7fede8098020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47c6052993e30ab7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:22.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7fc82f26c640 1 -- 172.21.15.83:0/1895076002 shutdown_connections 2024-04-17T10:38:22.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7fc82f26c640 1 --2- 172.21.15.83:0/1895076002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8281669f0 0x7fc828168de0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7fc82f26c640 1 -- 172.21.15.83:0/1895076002 >> 172.21.15.83:0/1895076002 conn(0x7fc8280b15b0 msgr2=0x7fc8280b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:22.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f4ca30d6640 1 Processor -- start 2024-04-17T10:38:22.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f4ca30d6640 1 -- start start 2024-04-17T10:38:22.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f400c8ef640 1 Processor -- start 2024-04-17T10:38:22.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f4623548640 1 -- 172.21.15.83:0/2700362413 shutdown_connections 2024-04-17T10:38:22.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f4623548640 1 --2- 172.21.15.83:0/2700362413 >> v2:172.21.15.83:3300/0 conn(0x7f461c071930 0x7f461c071d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f4623548640 1 --2- 172.21.15.83:0/2700362413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f461c15db60 0x7f461c0b3b60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f4623548640 1 -- 172.21.15.83:0/2700362413 >> 172.21.15.83:0/2700362413 conn(0x7f461c06abe0 msgr2=0x7f461c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:22.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f3dcdea0640 1 Processor -- start 2024-04-17T10:38:22.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7ff07bfff640 1 -- 172.21.15.83:0/577386122 reap_dead start 2024-04-17T10:38:22.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f400c8ef640 1 -- start start 2024-04-17T10:38:22.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f4623548640 1 -- 172.21.15.83:0/2700362413 shutdown_connections 2024-04-17T10:38:22.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7ff080c54640 1 -- 172.21.15.83:0/577386122 shutdown_connections 2024-04-17T10:38:22.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7ff080c54640 1 -- 172.21.15.83:0/577386122 >> 172.21.15.83:0/577386122 conn(0x7ff07c06abe0 msgr2=0x7ff07c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:22.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f0916724640 1 -- 172.21.15.83:0/923025404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09100b3650 msgr2=0x7f091006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f0916724640 1 --2- 172.21.15.83:0/923025404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09100b3650 0x7f091006d6f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0910050040 tx=0x7f090401c480 comp rx=0 tx=0).stop 2024-04-17T10:38:22.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7ff080c54640 1 -- 172.21.15.83:0/577386122 shutdown_connections 2024-04-17T10:38:22.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7ff080c54640 1 -- 172.21.15.83:0/577386122 wait complete. 2024-04-17T10:38:22.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f4ca30d6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c9c06b2f0 0x7f4c9c158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f4ca30d6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c9c0b54c0 0x7f4c9c15cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f4ca30d6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c9c0d0010 0x7f4c9c206eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f4ca30d6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c162b80 con 0x7f4c9c0d0010 2024-04-17T10:38:22.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f4ca30d6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c162a00 con 0x7f4c9c0b54c0 2024-04-17T10:38:22.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f4ca30d6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c166610 con 0x7f4c9c06b2f0 2024-04-17T10:38:22.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f4ca1673640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c9c0d0010 0x7f4c9c206eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f4ca1673640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c9c0d0010 0x7f4c9c206eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48108/0 (socket says 172.21.15.83:48108) 2024-04-17T10:38:22.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f4ca1673640 1 -- 172.21.15.83:0/1636061907 learned_addr learned my addr 172.21.15.83:0/1636061907 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:22.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f3dcdea0640 1 -- start start 2024-04-17T10:38:22.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f3dcdea0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3dc806c970 0x7f3dc8156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f3dcdea0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3dc80b2cc0 0x7f3dc815e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f3dcdea0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dc80d1780 0x7f3dc815e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f3dcdea0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc806d770 con 0x7f3dc80d1780 2024-04-17T10:38:22.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f3dcdea0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc815fd60 con 0x7f3dc80b2cc0 2024-04-17T10:38:22.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f3dcdea0640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc8156dc0 con 0x7f3dc806c970 2024-04-17T10:38:22.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f4ca0e72640 1 --2- 172.21.15.83:0/1636061907 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c9c06b2f0 0x7f4c9c158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7fedec953640 1 -- 172.21.15.83:0/3364253324 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede8075050 con 0x7fede0008330 2024-04-17T10:38:22.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7fedec953640 1 -- 172.21.15.83:0/3364253324 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede8120d90 con 0x7fede0008330 2024-04-17T10:38:22.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7fedec953640 1 -- 172.21.15.83:0/3364253324 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede8128680 con 0x7fede0008330 2024-04-17T10:38:22.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f400c8ef640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40080ce5b0 0x7f40080c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f400c8ef640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4008165d50 0x7f40080cced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f400c8ef640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40081661d0 0x7f40081547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f400c8ef640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4008170910 con 0x7f4008165d50 2024-04-17T10:38:22.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f400c8ef640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40080b5da0 con 0x7f40081661d0 2024-04-17T10:38:22.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f400c8ef640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4008170a90 con 0x7f40080ce5b0 2024-04-17T10:38:22.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4006d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40081661d0 0x7f40081547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4006d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40081661d0 0x7f40081547a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:36298/0 (socket says 172.21.15.83:36298) 2024-04-17T10:38:22.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4006d9d640 1 -- 172.21.15.83:0/1730486398 learned_addr learned my addr 172.21.15.83:0/1730486398 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:22.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4005d9b640 1 --2- 172.21.15.83:0/1730486398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4008165d50 0x7f40080cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f4623548640 1 -- 172.21.15.83:0/2700362413 wait complete. 2024-04-17T10:38:22.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4623548640 1 Processor -- start 2024-04-17T10:38:22.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f3dcce9e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3dc806c970 0x7f3dc8156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f3dcd69f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dc80d1780 0x7f3dc815e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f3dcce9e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3dc806c970 0x7f3dc8156880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:59698/0 (socket says 172.21.15.83:59698) 2024-04-17T10:38:22.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f3dc7fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3dc80b2cc0 0x7f3dc815e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f3dcce9e640 1 -- 172.21.15.83:0/1247967723 learned_addr learned my addr 172.21.15.83:0/1247967723 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:22.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f3dcd69f640 1 -- 172.21.15.83:0/1247967723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3dc806c970 msgr2=0x7f3dc8156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f3dcd69f640 1 --2- 172.21.15.83:0/1247967723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3dc806c970 0x7f3dc8156880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f3dcd69f640 1 -- 172.21.15.83:0/1247967723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3dc80b2cc0 msgr2=0x7f3dc815e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f3dcd69f640 1 --2- 172.21.15.83:0/1247967723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3dc80b2cc0 0x7f3dc815e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f3dcd69f640 1 -- 172.21.15.83:0/1247967723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc815b090 con 0x7f3dc80d1780 2024-04-17T10:38:22.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f3dcd69f640 1 --2- 172.21.15.83:0/1247967723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dc80d1780 0x7f3dc815e620 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc00c390 tx=0x7f3dbc001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:22.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f4c9bfff640 1 --2- 172.21.15.83:0/1636061907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c9c0b54c0 0x7f4c9c15cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f0916724640 1 -- 172.21.15.83:0/923025404 shutdown_connections 2024-04-17T10:38:22.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f0916724640 1 --2- 172.21.15.83:0/923025404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09100b3650 0x7f091006d6f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f0916724640 1 --2- 172.21.15.83:0/923025404 >> v2:172.21.15.167:3300/0 conn(0x7f0910159b40 0x7f09100b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f0916724640 1 -- 172.21.15.83:0/923025404 >> 172.21.15.83:0/923025404 conn(0x7f09100b1710 msgr2=0x7f09100b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:22.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f0916724640 1 -- 172.21.15.83:0/923025404 shutdown_connections 2024-04-17T10:38:22.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f0916724640 1 -- 172.21.15.83:0/923025404 wait complete. 2024-04-17T10:38:22.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4623548640 1 -- start start 2024-04-17T10:38:22.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f4ca0e72640 1 -- 172.21.15.83:0/1636061907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c9c0b54c0 msgr2=0x7f4c9c15cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f4ca0e72640 1 --2- 172.21.15.83:0/1636061907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c9c0b54c0 0x7f4c9c15cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f4ca0e72640 1 -- 172.21.15.83:0/1636061907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c9c0d0010 msgr2=0x7f4c9c206eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f4ca0e72640 1 --2- 172.21.15.83:0/1636061907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c9c0d0010 0x7f4c9c206eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.004+0000 7f4ca0e72640 1 -- 172.21.15.83:0/1636061907 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9c15b780 con 0x7f4c9c06b2f0 2024-04-17T10:38:22.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4ca0e72640 1 --2- 172.21.15.83:0/1636061907 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c9c06b2f0 0x7f4c9c158cb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0ac000 tx=0x7f4c8c019190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:22.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4c997fa640 1 -- 172.21.15.83:0/1636061907 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c0b1040 con 0x7f4c9c06b2f0 2024-04-17T10:38:22.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4005d9b640 1 -- 172.21.15.83:0/1730486398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40080ce5b0 msgr2=0x7f40080c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4005d9b640 1 --2- 172.21.15.83:0/1730486398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40080ce5b0 0x7f40080c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4005d9b640 1 -- 172.21.15.83:0/1730486398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40081661d0 msgr2=0x7f40081547a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4005d9b640 1 --2- 172.21.15.83:0/1730486398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40081661d0 0x7f40081547a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4005d9b640 1 -- 172.21.15.83:0/1730486398 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f400806ded0 con 0x7f4008165d50 2024-04-17T10:38:22.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4ca30d6640 1 -- 172.21.15.83:0/1636061907 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c9c15ba60 con 0x7f4c9c06b2f0 2024-04-17T10:38:22.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4ca30d6640 1 -- 172.21.15.83:0/1636061907 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c9c15c020 con 0x7f4c9c06b2f0 2024-04-17T10:38:22.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7fedefbb9640 1 -- 172.21.15.83:0/3364253324 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fede0008330 msgr2=0x7fedd4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7fedefbb9640 1 --2- 172.21.15.83:0/3364253324 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fede0008330 0x7fedd4004810 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fede807ad60 tx=0x7fede8098020 comp rx=0 tx=0).stop 2024-04-17T10:38:22.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4005d9b640 1 --2- 172.21.15.83:0/1730486398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4008165d50 0x7f40080cced0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc0ab000 tx=0x7f3ffc0191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:22.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7fedee957640 1 -- 172.21.15.83:0/3364253324 reap_dead start 2024-04-17T10:38:22.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f3dc57fa640 1 -- 172.21.15.83:0/1247967723 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc017070 con 0x7f3dc80d1780 2024-04-17T10:38:22.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f3ff6ffd640 1 -- 172.21.15.83:0/1730486398 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc0b8020 con 0x7f4008165d50 2024-04-17T10:38:22.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7fedefbb9640 1 -- 172.21.15.83:0/3364253324 shutdown_connections 2024-04-17T10:38:22.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7fedefbb9640 1 -- 172.21.15.83:0/3364253324 >> 172.21.15.83:0/3364253324 conn(0x7fede001e8a0 msgr2=0x7fede00555d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:22.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f3dcdea0640 1 -- 172.21.15.83:0/1247967723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc815b370 con 0x7f3dc80d1780 2024-04-17T10:38:22.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f3dcdea0640 1 -- 172.21.15.83:0/1247967723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc815b930 con 0x7f3dc80d1780 2024-04-17T10:38:22.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7fedefbb9640 1 -- 172.21.15.83:0/3364253324 shutdown_connections 2024-04-17T10:38:22.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4c997fa640 1 -- 172.21.15.83:0/1636061907 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8c009470 con 0x7f4c9c06b2f0 2024-04-17T10:38:22.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4c997fa640 1 -- 172.21.15.83:0/1636061907 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c0b7450 con 0x7f4c9c06b2f0 2024-04-17T10:38:22.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7fc82f26c640 1 -- 172.21.15.83:0/1895076002 shutdown_connections 2024-04-17T10:38:22.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4623548640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f461c071550 0x7f461c158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4623548640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f461c071930 0x7f461c0ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4623548640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f461c15db60 0x7f461c1f9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4623548640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c06cb50 con 0x7f461c071930 2024-04-17T10:38:22.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4623548640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c06c9d0 con 0x7f461c071550 2024-04-17T10:38:22.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4623548640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f461c06ccd0 con 0x7f461c15db60 2024-04-17T10:38:22.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f400c8ef640 1 -- 172.21.15.83:0/1730486398 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40081570b0 con 0x7f4008165d50 2024-04-17T10:38:22.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f400c8ef640 1 -- 172.21.15.83:0/1730486398 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f400806e170 con 0x7f4008165d50 2024-04-17T10:38:22.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f3ff6ffd640 1 -- 172.21.15.83:0/1730486398 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ffc0b4610 con 0x7f4008165d50 2024-04-17T10:38:22.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f3ff6ffd640 1 -- 172.21.15.83:0/1730486398 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc0b0040 con 0x7f4008165d50 2024-04-17T10:38:22.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7fedefbb9640 1 -- 172.21.15.83:0/3364253324 wait complete. 2024-04-17T10:38:22.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7fedefbb9640 1 Processor -- start 2024-04-17T10:38:22.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7fedefbb9640 1 -- start start 2024-04-17T10:38:22.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7fc82f26c640 1 -- 172.21.15.83:0/1895076002 wait complete. 2024-04-17T10:38:22.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7fc82f26c640 1 Processor -- start 2024-04-17T10:38:22.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4622546640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f461c071550 0x7f461c158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4621d45640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f461c071930 0x7f461c0ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4621d45640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f461c071930 0x7f461c0ab7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48132/0 (socket says 172.21.15.83:48132) 2024-04-17T10:38:22.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4621d45640 1 -- 172.21.15.83:0/3412492672 learned_addr learned my addr 172.21.15.83:0/3412492672 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:22.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4621d45640 1 -- 172.21.15.83:0/3412492672 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f461c15db60 msgr2=0x7f461c1f9190 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-17T10:38:22.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f3dcdea0640 1 -- 172.21.15.83:0/1247967723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dc804efa0 con 0x7f3dc80d1780 2024-04-17T10:38:22.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f3dc57fa640 1 -- 172.21.15.83:0/1247967723 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dbc005cf0 con 0x7f3dc80d1780 2024-04-17T10:38:22.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f3dc57fa640 1 -- 172.21.15.83:0/1247967723 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc021060 con 0x7f3dc80d1780 2024-04-17T10:38:22.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f0916724640 1 Processor -- start 2024-04-17T10:38:22.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4621d45640 1 --2- 172.21.15.83:0/3412492672 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f461c15db60 0x7f461c1f9190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4621d45640 1 -- 172.21.15.83:0/3412492672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f461c071550 msgr2=0x7f461c158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4621d45640 1 --2- 172.21.15.83:0/3412492672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f461c071550 0x7f461c158e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7f4621d45640 1 -- 172.21.15.83:0/3412492672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461c0ac250 con 0x7f461c071930 2024-04-17T10:38:22.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f4621d45640 1 --2- 172.21.15.83:0/3412492672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f461c071930 0x7f461c0ab7b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f46180ad040 tx=0x7f461801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:22.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7fc82f26c640 1 -- start start 2024-04-17T10:38:22.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7fc82f26c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc82806ae10 0x7fc828159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7fc82f26c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc82815e330 0x7fc82815a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7fc82f26c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8281669f0 0x7fc828165ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7fc82f26c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8280cf470 con 0x7fc82815e330 2024-04-17T10:38:22.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7fc82f26c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8280cf2f0 con 0x7fc82806ae10 2024-04-17T10:38:22.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7fc82f26c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc82816bb40 con 0x7fc8281669f0 2024-04-17T10:38:22.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f3dc57fa640 1 -- 172.21.15.83:0/1247967723 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3dbc01a070 con 0x7f3dc80d1780 2024-04-17T10:38:22.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f3dc57fa640 1 --2- 172.21.15.83:0/1247967723 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3db4066f90 0x7f3db4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f3dc57fa640 1 -- 172.21.15.83:0/1247967723 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3dbc084130 con 0x7f3dc80d1780 2024-04-17T10:38:22.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f4c997fa640 1 -- 172.21.15.83:0/1636061907 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4c8c0be030 con 0x7f4c9c06b2f0 2024-04-17T10:38:22.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f4ca30d6640 1 -- 172.21.15.83:0/1636061907 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c7c0052a0 con 0x7f4c9c06b2f0 2024-04-17T10:38:22.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f4c997fa640 1 --2- 172.21.15.83:0/1636061907 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4c88066f60 0x7f4c88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f4c997fa640 1 -- 172.21.15.83:0/1636061907 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c8c11e940 con 0x7f4c9c06b2f0 2024-04-17T10:38:22.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f4c9bfff640 1 --2- 172.21.15.83:0/1636061907 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4c88066f60 0x7f4c88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7ff080c54640 1 Processor -- start 2024-04-17T10:38:22.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.008+0000 7ff080c54640 1 -- start start 2024-04-17T10:38:22.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7ff080c54640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff07c1fe320 0x7ff07c0cd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7ff080c54640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff07c1fe700 0x7ff07c0cd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7ff080c54640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff07c153240 0x7ff07c1536c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7ff080c54640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c06c340 con 0x7ff07c1fe700 2024-04-17T10:38:22.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7ff080c54640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c06c1c0 con 0x7ff07c153240 2024-04-17T10:38:22.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7ff080c54640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c06c4c0 con 0x7ff07c1fe320 2024-04-17T10:38:22.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7ff07bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff07c153240 0x7ff07c1536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7ff07bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff07c153240 0x7ff07c1536c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:36314/0 (socket says 172.21.15.83:36314) 2024-04-17T10:38:22.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7ff07bfff640 1 -- 172.21.15.83:0/3125691458 learned_addr learned my addr 172.21.15.83:0/3125691458 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:22.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7ff07bfff640 1 -- 172.21.15.83:0/3125691458 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff07c1fe320 msgr2=0x7ff07c0cd450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T10:38:22.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7ff07bfff640 1 --2- 172.21.15.83:0/3125691458 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff07c1fe320 0x7ff07c0cd450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7ff07bfff640 1 -- 172.21.15.83:0/3125691458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff07c1fe700 msgr2=0x7ff07c0cd990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T10:38:22.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7ff07bfff640 1 --2- 172.21.15.83:0/3125691458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff07c1fe700 0x7ff07c0cd990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7ff07bfff640 1 -- 172.21.15.83:0/3125691458 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff07c153f70 con 0x7ff07c153240 2024-04-17T10:38:22.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f0916724640 1 -- start start 2024-04-17T10:38:22.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f0916724640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09100b3650 0x7f09101616b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f0916724640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09100b3ae0 0x7f0910163200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f0916724640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0910159b40 0x7f0910163740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f0916724640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09100cd980 con 0x7f09100b3650 2024-04-17T10:38:22.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f0916724640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09100cd800 con 0x7f09100b3ae0 2024-04-17T10:38:22.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f0916724640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091006dc30 con 0x7f0910159b40 2024-04-17T10:38:22.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7fc82d008640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc82806ae10 0x7fc828159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7fc82d809640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8281669f0 0x7fc828165ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7fc82d008640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc82806ae10 0x7fc828159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:36334/0 (socket says 172.21.15.83:36334) 2024-04-17T10:38:22.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7fc82d008640 1 -- 172.21.15.83:0/1115042690 learned_addr learned my addr 172.21.15.83:0/1115042690 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:22.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f3dcce9e640 1 --2- 172.21.15.83:0/1247967723 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3db4066f90 0x7f3db4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f4c9bfff640 1 --2- 172.21.15.83:0/1636061907 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4c88066f60 0x7f4c88069420 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c1591f0 tx=0x7f4c90009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:22.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7ff07bfff640 1 --2- 172.21.15.83:0/3125691458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff07c153240 0x7ff07c1536c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff07400bfd0 tx=0x7ff07400e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:22.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7fedefbb9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fede0008330 0x7fede00d5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7fedefbb9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fede00db460 0x7fede00d7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7feded955640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fede0008330 0x7fede00d5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7feded955640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fede0008330 0x7fede00d5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:36322/0 (socket says 172.21.15.83:36322) 2024-04-17T10:38:22.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7feded955640 1 -- 172.21.15.83:0/1120444975 learned_addr learned my addr 172.21.15.83:0/1120444975 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:22.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7fedefbb9640 1 --2- 172.21.15.83:0/1120444975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fede00d47e0 0x7fede00d7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7fedefbb9640 1 -- 172.21.15.83:0/1120444975 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede0053110 con 0x7fede00db460 2024-04-17T10:38:22.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7fedefbb9640 1 -- 172.21.15.83:0/1120444975 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede0007820 con 0x7fede0008330 2024-04-17T10:38:22.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7fedefbb9640 1 -- 172.21.15.83:0/1120444975 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fede0053290 con 0x7fede00d47e0 2024-04-17T10:38:22.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f3ff6ffd640 1 -- 172.21.15.83:0/1730486398 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3ffc0bb020 con 0x7f4008165d50 2024-04-17T10:38:22.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f3ff6ffd640 1 --2- 172.21.15.83:0/1730486398 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3fe8066f60 0x7f3fe8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f3ff6ffd640 1 -- 172.21.15.83:0/1730486398 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ffc11ebc0 con 0x7f4008165d50 2024-04-17T10:38:22.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f400659c640 1 --2- 172.21.15.83:0/1730486398 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3fe8066f60 0x7f3fe8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f4612ffd640 1 -- 172.21.15.83:0/3412492672 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46180bf610 con 0x7f461c071930 2024-04-17T10:38:22.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f4623548640 1 -- 172.21.15.83:0/3412492672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f461c0ac530 con 0x7f461c071930 2024-04-17T10:38:22.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f4623548640 1 -- 172.21.15.83:0/3412492672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f461c0ad5e0 con 0x7f461c071930 2024-04-17T10:38:22.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7ff063fff640 1 -- 172.21.15.83:0/3125691458 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07401a070 con 0x7ff07c153240 2024-04-17T10:38:22.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7feded955640 1 -- 172.21.15.83:0/1120444975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fede00d47e0 msgr2=0x7fede00d7890 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-17T10:38:22.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7feded955640 1 --2- 172.21.15.83:0/1120444975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fede00d47e0 0x7fede00d7890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7feded955640 1 -- 172.21.15.83:0/1120444975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fede00db460 msgr2=0x7fede00d7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7feded955640 1 --2- 172.21.15.83:0/1120444975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fede00db460 0x7fede00d7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7feded955640 1 -- 172.21.15.83:0/1120444975 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede0183950 con 0x7fede0008330 2024-04-17T10:38:22.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7fc82d809640 1 -- 172.21.15.83:0/1115042690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc82806ae10 msgr2=0x7fc828159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7fc82d809640 1 --2- 172.21.15.83:0/1115042690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc82806ae10 0x7fc828159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7fc82d809640 1 -- 172.21.15.83:0/1115042690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc82815e330 msgr2=0x7fc82815a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7fc82d809640 1 --2- 172.21.15.83:0/1115042690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc82815e330 0x7fc82815a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7fc82d809640 1 -- 172.21.15.83:0/1115042690 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc828203c90 con 0x7fc8281669f0 2024-04-17T10:38:22.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f3dcce9e640 1 --2- 172.21.15.83:0/1247967723 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3db4066f90 0x7f3db4069450 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009580 tx=0x7f3db8009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:22.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7ff080c54640 1 -- 172.21.15.83:0/3125691458 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff07c154250 con 0x7ff07c153240 2024-04-17T10:38:22.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7ff080c54640 1 -- 172.21.15.83:0/3125691458 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff07c154810 con 0x7ff07c153240 2024-04-17T10:38:22.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7ff063fff640 1 -- 172.21.15.83:0/3125691458 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07400b2d0 con 0x7ff07c153240 2024-04-17T10:38:22.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7ff063fff640 1 -- 172.21.15.83:0/3125691458 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07400b480 con 0x7ff07c153240 2024-04-17T10:38:22.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7feded955640 1 --2- 172.21.15.83:0/1120444975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fede0008330 0x7fede00d5860 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fede400e990 tx=0x7fede400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:22.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f4623548640 1 -- 172.21.15.83:0/3412492672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f461c04efa0 con 0x7f461c071930 2024-04-17T10:38:22.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f0914cc1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0910159b40 0x7f0910163740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f0914cc1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0910159b40 0x7f0910163740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:59740/0 (socket says 172.21.15.83:59740) 2024-04-17T10:38:22.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f0914cc1640 1 -- 172.21.15.83:0/1109829749 learned_addr learned my addr 172.21.15.83:0/1109829749 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:22.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f090f7fe640 1 --2- 172.21.15.83:0/1109829749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09100b3ae0 0x7f0910163200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f0914cc1640 1 -- 172.21.15.83:0/1109829749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09100b3ae0 msgr2=0x7f0910163200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f0914cc1640 1 --2- 172.21.15.83:0/1109829749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09100b3ae0 0x7f0910163200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f0914cc1640 1 -- 172.21.15.83:0/1109829749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09100b3650 msgr2=0x7f09101616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f0914cc1640 1 --2- 172.21.15.83:0/1109829749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09100b3650 0x7f09101616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f0914cc1640 1 -- 172.21.15.83:0/1109829749 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091015ed20 con 0x7f0910159b40 2024-04-17T10:38:22.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f090f7fe640 1 --2- 172.21.15.83:0/1109829749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09100b3ae0 0x7f0910163200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:22.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f0914cc1640 1 --2- 172.21.15.83:0/1109829749 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0910159b40 0x7f0910163740 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f090800caa0 tx=0x7f090800cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:22.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7f400c8ef640 1 -- 172.21.15.83:0/1730486398 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fec0052a0 con 0x7f4008165d50 2024-04-17T10:38:22.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7ff080c54640 1 -- 172.21.15.83:0/3125691458 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff05c0052a0 con 0x7ff07c153240 2024-04-17T10:38:22.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7fedde7fc640 1 -- 172.21.15.83:0/1120444975 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede4015020 con 0x7fede0008330 2024-04-17T10:38:22.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7fedde7fc640 1 -- 172.21.15.83:0/1120444975 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede400c760 con 0x7fede0008330 2024-04-17T10:38:22.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.012+0000 7fc82d809640 1 --2- 172.21.15.83:0/1115042690 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8281669f0 0x7fc828165ba0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc82806bc90 tx=0x7fc820019740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:22.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.016+0000 7fedde7fc640 1 -- 172.21.15.83:0/1120444975 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede400cb30 con 0x7fede0008330 2024-04-17T10:38:22.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.016+0000 7fedefbb9640 1 -- 172.21.15.83:0/1120444975 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede0183c30 con 0x7fede0008330 2024-04-17T10:38:22.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.016+0000 7fedefbb9640 1 -- 172.21.15.83:0/1120444975 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede01841f0 con 0x7fede0008330 2024-04-17T10:38:22.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.016+0000 7f400659c640 1 --2- 172.21.15.83:0/1730486398 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3fe8066f60 0x7f3fe8069420 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8005890 tx=0x7f3ff8007a30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:22.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.016+0000 7fedefbb9640 1 -- 172.21.15.83:0/1120444975 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fede00059c0 con 0x7fede0008330 2024-04-17T10:38:22.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.016+0000 7ff063fff640 1 -- 172.21.15.83:0/3125691458 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff07401d070 con 0x7ff07c153240 2024-04-17T10:38:22.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.016+0000 7ff063fff640 1 --2- 172.21.15.83:0/3125691458 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff068066f60 0x7ff068069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.016+0000 7ff063fff640 1 -- 172.21.15.83:0/3125691458 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff074085630 con 0x7ff07c153240 2024-04-17T10:38:22.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.016+0000 7ff07b7fe640 1 --2- 172.21.15.83:0/3125691458 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff068066f60 0x7ff068069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.016+0000 7fedde7fc640 1 -- 172.21.15.83:0/1120444975 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fede4018020 con 0x7fede0008330 2024-04-17T10:38:22.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.016+0000 7ff07b7fe640 1 --2- 172.21.15.83:0/3125691458 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff068066f60 0x7ff068069420 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff06c0af000 tx=0x7ff07c0b29b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:22.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.016+0000 7fedde7fc640 1 --2- 172.21.15.83:0/1120444975 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fedcc066f30 0x7fedcc0693f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.016+0000 7fedde7fc640 1 -- 172.21.15.83:0/1120444975 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fede40858f0 con 0x7fede0008330 2024-04-17T10:38:22.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.016+0000 7feded154640 1 --2- 172.21.15.83:0/1120444975 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fedcc066f30 0x7fedcc0693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.016+0000 7feded154640 1 --2- 172.21.15.83:0/1120444975 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fedcc066f30 0x7fedcc0693f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fedd400bd20 tx=0x7fede80968b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:22.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.016+0000 7f3dc57fa640 1 -- 172.21.15.83:0/1247967723 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3dbc054a90 con 0x7f3dc80d1780 2024-04-17T10:38:22.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.016+0000 7f4612ffd640 1 -- 172.21.15.83:0/3412492672 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46180b0030 con 0x7f461c071930 2024-04-17T10:38:22.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.016+0000 7f4612ffd640 1 -- 172.21.15.83:0/3412492672 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4618005ba0 con 0x7f461c071930 2024-04-17T10:38:22.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.024+0000 7f3ff6ffd640 1 -- 172.21.15.83:0/1730486398 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3ffc0c2030 con 0x7f4008165d50 2024-04-17T10:38:22.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.024+0000 7f4612ffd640 1 -- 172.21.15.83:0/3412492672 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f46180bd020 con 0x7f461c071930 2024-04-17T10:38:22.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.024+0000 7f4612ffd640 1 --2- 172.21.15.83:0/3412492672 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4600066f90 0x7f4600069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.024+0000 7f4612ffd640 1 -- 172.21.15.83:0/3412492672 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f461811fe20 con 0x7f461c071930 2024-04-17T10:38:22.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.024+0000 7f4622546640 1 --2- 172.21.15.83:0/3412492672 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4600066f90 0x7f4600069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.024+0000 7f4622546640 1 --2- 172.21.15.83:0/3412492672 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4600066f90 0x7f4600069450 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f460c005890 tx=0x7f460c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:22.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.028+0000 7fc81dffb640 1 -- 172.21.15.83:0/1115042690 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8200bc070 con 0x7fc8281669f0 2024-04-17T10:38:22.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.028+0000 7fc81dffb640 1 -- 172.21.15.83:0/1115042690 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8200b8640 con 0x7fc8281669f0 2024-04-17T10:38:22.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.028+0000 7fc81dffb640 1 -- 172.21.15.83:0/1115042690 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8200b89f0 con 0x7fc8281669f0 2024-04-17T10:38:22.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.028+0000 7fc82f26c640 1 -- 172.21.15.83:0/1115042690 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc828203f70 con 0x7fc8281669f0 2024-04-17T10:38:22.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.028+0000 7fc82f26c640 1 -- 172.21.15.83:0/1115042690 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc828204530 con 0x7fc8281669f0 2024-04-17T10:38:22.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.028+0000 7f090cff9640 1 -- 172.21.15.83:0/1109829749 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0908017070 con 0x7f0910159b40 2024-04-17T10:38:22.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.028+0000 7f090cff9640 1 -- 172.21.15.83:0/1109829749 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f090800ac00 con 0x7f0910159b40 2024-04-17T10:38:22.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.028+0000 7f090cff9640 1 -- 172.21.15.83:0/1109829749 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0908012620 con 0x7f0910159b40 2024-04-17T10:38:22.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.028+0000 7f0916724640 1 -- 172.21.15.83:0/1109829749 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f091015f000 con 0x7f0910159b40 2024-04-17T10:38:22.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.028+0000 7f0916724640 1 -- 172.21.15.83:0/1109829749 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091015f5c0 con 0x7f0910159b40 2024-04-17T10:38:22.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.028+0000 7f4c997fa640 1 -- 172.21.15.83:0/1636061907 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4c8c0c5020 con 0x7f4c9c06b2f0 2024-04-17T10:38:22.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.028+0000 7fc82f26c640 1 -- 172.21.15.83:0/1115042690 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc828074f90 con 0x7fc8281669f0 2024-04-17T10:38:22.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.028+0000 7f4612ffd640 1 -- 172.21.15.83:0/3412492672 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f46180c4060 con 0x7f461c071930 2024-04-17T10:38:22.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.028+0000 7f0916724640 1 -- 172.21.15.83:0/1109829749 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f00052a0 con 0x7f0910159b40 2024-04-17T10:38:22.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.028+0000 7ff063fff640 1 -- 172.21.15.83:0/3125691458 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff0740560d0 con 0x7ff07c153240 2024-04-17T10:38:22.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.028+0000 7fc81dffb640 1 -- 172.21.15.83:0/1115042690 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc8200b8b90 con 0x7fc8281669f0 2024-04-17T10:38:22.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.032+0000 7fc81dffb640 1 --2- 172.21.15.83:0/1115042690 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc80c066f60 0x7fc80c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.032+0000 7fc81dffb640 1 -- 172.21.15.83:0/1115042690 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8201201d0 con 0x7fc8281669f0 2024-04-17T10:38:22.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.032+0000 7fc82d008640 1 --2- 172.21.15.83:0/1115042690 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc80c066f60 0x7fc80c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.032+0000 7fc82d008640 1 --2- 172.21.15.83:0/1115042690 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc80c066f60 0x7fc80c069420 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc8280ad810 tx=0x7fc824009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:22.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.032+0000 7f090cff9640 1 -- 172.21.15.83:0/1109829749 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f090801a070 con 0x7f0910159b40 2024-04-17T10:38:22.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.032+0000 7f090cff9640 1 --2- 172.21.15.83:0/1109829749 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f08fc067030 0x7f08fc0694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.032+0000 7f090cff9640 1 -- 172.21.15.83:0/1109829749 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0908085190 con 0x7f0910159b40 2024-04-17T10:38:22.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.032+0000 7f090ffff640 1 --2- 172.21.15.83:0/1109829749 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f08fc067030 0x7f08fc0694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.032+0000 7f090ffff640 1 --2- 172.21.15.83:0/1109829749 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f08fc067030 0x7f08fc0694f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f090000afd0 tx=0x7f090000c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:22.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.052+0000 7fedde7fc640 1 -- 172.21.15.83:0/1120444975 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fede401f020 con 0x7fede0008330 2024-04-17T10:38:22.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.056+0000 7fc81dffb640 1 -- 172.21.15.83:0/1115042690 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc820016620 con 0x7fc8281669f0 2024-04-17T10:38:22.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.068+0000 7f090cff9640 1 -- 172.21.15.83:0/1109829749 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0908021020 con 0x7f0910159b40 2024-04-17T10:38:22.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.296+0000 7f400c8ef640 1 -- 172.21.15.83:0/1730486398 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f3fec005590 con 0x7f4008165d50 2024-04-17T10:38:22.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.296+0000 7f3ff6ffd640 1 -- 172.21.15.83:0/1730486398 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3ffc0160d0 con 0x7f4008165d50 2024-04-17T10:38:22.295 INFO:teuthology.orchestra.run.smithi083.stdout:55834574855 2024-04-17T10:38:22.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.300+0000 7f3ff4ff9640 1 -- 172.21.15.83:0/1730486398 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3fe8066f60 msgr2=0x7f3fe8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.300+0000 7f3ff4ff9640 1 --2- 172.21.15.83:0/1730486398 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3fe8066f60 0x7f3fe8069420 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8005890 tx=0x7f3ff8007a30 comp rx=0 tx=0).stop 2024-04-17T10:38:22.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.300+0000 7f3ff4ff9640 1 -- 172.21.15.83:0/1730486398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4008165d50 msgr2=0x7f40080cced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.300+0000 7f3ff4ff9640 1 --2- 172.21.15.83:0/1730486398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4008165d50 0x7f40080cced0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc0ab000 tx=0x7f3ffc0191f0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.300+0000 7f3ff4ff9640 1 -- 172.21.15.83:0/1730486398 shutdown_connections 2024-04-17T10:38:22.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.300+0000 7f3ff4ff9640 1 --2- 172.21.15.83:0/1730486398 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3fe8066f60 0x7f3fe8069420 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.300+0000 7f3ff4ff9640 1 --2- 172.21.15.83:0/1730486398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40081661d0 0x7f40081547a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.300+0000 7f3ff4ff9640 1 --2- 172.21.15.83:0/1730486398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4008165d50 0x7f40080cced0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.300+0000 7f3ff4ff9640 1 --2- 172.21.15.83:0/1730486398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40080ce5b0 0x7f40080c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.300+0000 7f3ff4ff9640 1 -- 172.21.15.83:0/1730486398 >> 172.21.15.83:0/1730486398 conn(0x7f40080d0450 msgr2=0x7f40080b2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:22.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.300+0000 7f3ff4ff9640 1 -- 172.21.15.83:0/1730486398 shutdown_connections 2024-04-17T10:38:22.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.300+0000 7f3ff4ff9640 1 -- 172.21.15.83:0/1730486398 wait complete. 2024-04-17T10:38:22.317 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2024-04-17T10:38:22.317 DEBUG:teuthology.parallel:result is None 2024-04-17T10:38:22.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.328+0000 7f3dcdea0640 1 -- 172.21.15.83:0/1247967723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f3dc816bdd0 con 0x7f3dc80d1780 2024-04-17T10:38:22.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.328+0000 7f3dc57fa640 1 -- 172.21.15.83:0/1247967723 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3dbc058b30 con 0x7f3dc80d1780 2024-04-17T10:38:22.325 INFO:teuthology.orchestra.run.smithi083.stdout:55834574855 2024-04-17T10:38:22.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.328+0000 7f4ca30d6640 1 -- 172.21.15.83:0/1636061907 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4c7c005590 con 0x7f4c9c06b2f0 2024-04-17T10:38:22.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.332+0000 7f4623548640 1 -- 172.21.15.83:0/3412492672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f461c070cf0 con 0x7f461c071930 2024-04-17T10:38:22.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.332+0000 7f4c997fa640 1 -- 172.21.15.83:0/1636061907 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4c8c016130 con 0x7f4c9c06b2f0 2024-04-17T10:38:22.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.332+0000 7f4612ffd640 1 -- 172.21.15.83:0/3412492672 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4618016070 con 0x7f461c071930 2024-04-17T10:38:22.328 INFO:teuthology.orchestra.run.smithi083.stdout:55834574857 2024-04-17T10:38:22.328 INFO:teuthology.orchestra.run.smithi083.stdout:55834574855 2024-04-17T10:38:22.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.332+0000 7f3dcdea0640 1 -- 172.21.15.83:0/1247967723 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3db4066f90 msgr2=0x7f3db4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.332+0000 7f3dcdea0640 1 --2- 172.21.15.83:0/1247967723 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3db4066f90 0x7f3db4069450 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009580 tx=0x7f3db8009190 comp rx=0 tx=0).stop 2024-04-17T10:38:22.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.332+0000 7f3dcdea0640 1 -- 172.21.15.83:0/1247967723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dc80d1780 msgr2=0x7f3dc815e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.332+0000 7f3dcdea0640 1 --2- 172.21.15.83:0/1247967723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dc80d1780 0x7f3dc815e620 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc00c390 tx=0x7f3dbc001e80 comp rx=0 tx=0).stop 2024-04-17T10:38:22.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.332+0000 7f3dcdea0640 1 -- 172.21.15.83:0/1247967723 shutdown_connections 2024-04-17T10:38:22.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.332+0000 7f3dcdea0640 1 --2- 172.21.15.83:0/1247967723 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3db4066f90 0x7f3db4069450 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.332+0000 7f3dcdea0640 1 --2- 172.21.15.83:0/1247967723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dc80d1780 0x7f3dc815e620 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.332+0000 7f3dcdea0640 1 --2- 172.21.15.83:0/1247967723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3dc80b2cc0 0x7f3dc815e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.332+0000 7f3dcdea0640 1 --2- 172.21.15.83:0/1247967723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3dc806c970 0x7f3dc8156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.332+0000 7f3dcdea0640 1 -- 172.21.15.83:0/1247967723 >> 172.21.15.83:0/1247967723 conn(0x7f3dc80d3880 msgr2=0x7f3dc8161650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:22.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.332+0000 7f3dcdea0640 1 -- 172.21.15.83:0/1247967723 shutdown_connections 2024-04-17T10:38:22.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.332+0000 7f3dcdea0640 1 -- 172.21.15.83:0/1247967723 wait complete. 2024-04-17T10:38:22.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.336+0000 7f4623548640 1 -- 172.21.15.83:0/3412492672 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4600066f90 msgr2=0x7f4600069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.336+0000 7f4623548640 1 --2- 172.21.15.83:0/3412492672 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4600066f90 0x7f4600069450 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f460c005890 tx=0x7f460c009040 comp rx=0 tx=0).stop 2024-04-17T10:38:22.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.336+0000 7f4623548640 1 -- 172.21.15.83:0/3412492672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f461c071930 msgr2=0x7f461c0ab7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.336+0000 7f4623548640 1 --2- 172.21.15.83:0/3412492672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f461c071930 0x7f461c0ab7b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f46180ad040 tx=0x7f461801ac90 comp rx=0 tx=0).stop 2024-04-17T10:38:22.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.336+0000 7f4623548640 1 -- 172.21.15.83:0/3412492672 shutdown_connections 2024-04-17T10:38:22.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.336+0000 7f4ca30d6640 1 -- 172.21.15.83:0/1636061907 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4c88066f60 msgr2=0x7f4c88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.336+0000 7f4ca30d6640 1 --2- 172.21.15.83:0/1636061907 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4c88066f60 0x7f4c88069420 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c1591f0 tx=0x7f4c90009040 comp rx=0 tx=0).stop 2024-04-17T10:38:22.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.336+0000 7f4623548640 1 --2- 172.21.15.83:0/3412492672 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4600066f90 0x7f4600069450 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.336+0000 7f4ca30d6640 1 -- 172.21.15.83:0/1636061907 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c9c06b2f0 msgr2=0x7f4c9c158cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.336+0000 7f4623548640 1 --2- 172.21.15.83:0/3412492672 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f461c15db60 0x7f461c1f9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.336+0000 7f4623548640 1 --2- 172.21.15.83:0/3412492672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f461c071930 0x7f461c0ab7b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.336+0000 7f4623548640 1 --2- 172.21.15.83:0/3412492672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f461c071550 0x7f461c158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.336+0000 7f4623548640 1 -- 172.21.15.83:0/3412492672 >> 172.21.15.83:0/3412492672 conn(0x7f461c06abe0 msgr2=0x7f461c0b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:22.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.336+0000 7f4623548640 1 -- 172.21.15.83:0/3412492672 shutdown_connections 2024-04-17T10:38:22.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.336+0000 7f4ca30d6640 1 --2- 172.21.15.83:0/1636061907 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c9c06b2f0 0x7f4c9c158cb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0ac000 tx=0x7f4c8c019190 comp rx=0 tx=0).stop 2024-04-17T10:38:22.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.336+0000 7f4623548640 1 -- 172.21.15.83:0/3412492672 wait complete. 2024-04-17T10:38:22.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.336+0000 7f4ca30d6640 1 -- 172.21.15.83:0/1636061907 shutdown_connections 2024-04-17T10:38:22.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.336+0000 7f4ca30d6640 1 --2- 172.21.15.83:0/1636061907 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4c88066f60 0x7f4c88069420 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.336+0000 7f4ca30d6640 1 --2- 172.21.15.83:0/1636061907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c9c0d0010 0x7f4c9c206eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.336+0000 7f4ca30d6640 1 --2- 172.21.15.83:0/1636061907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c9c0b54c0 0x7f4c9c15cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.336+0000 7f4ca30d6640 1 --2- 172.21.15.83:0/1636061907 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c9c06b2f0 0x7f4c9c158cb0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.336+0000 7f4ca30d6640 1 -- 172.21.15.83:0/1636061907 >> 172.21.15.83:0/1636061907 conn(0x7f4c9c06bcf0 msgr2=0x7f4c9c164860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:22.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.336+0000 7f4ca30d6640 1 -- 172.21.15.83:0/1636061907 shutdown_connections 2024-04-17T10:38:22.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.336+0000 7f4ca30d6640 1 -- 172.21.15.83:0/1636061907 wait complete. 2024-04-17T10:38:22.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.348+0000 7ff080c54640 1 -- 172.21.15.83:0/3125691458 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7ff05c005860 con 0x7ff07c153240 2024-04-17T10:38:22.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.352+0000 7ff063fff640 1 -- 172.21.15.83:0/3125691458 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff07405a170 con 0x7ff07c153240 2024-04-17T10:38:22.347 INFO:teuthology.orchestra.run.smithi083.stdout:55834574856 2024-04-17T10:38:22.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.352+0000 7ff080c54640 1 -- 172.21.15.83:0/3125691458 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff068066f60 msgr2=0x7ff068069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.352+0000 7ff080c54640 1 --2- 172.21.15.83:0/3125691458 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff068066f60 0x7ff068069420 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff06c0af000 tx=0x7ff07c0b29b0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.352+0000 7ff080c54640 1 -- 172.21.15.83:0/3125691458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff07c153240 msgr2=0x7ff07c1536c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.352+0000 7ff080c54640 1 --2- 172.21.15.83:0/3125691458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff07c153240 0x7ff07c1536c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff07400bfd0 tx=0x7ff07400e6c0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.352+0000 7ff080c54640 1 -- 172.21.15.83:0/3125691458 shutdown_connections 2024-04-17T10:38:22.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.352+0000 7ff080c54640 1 --2- 172.21.15.83:0/3125691458 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff068066f60 0x7ff068069420 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.352+0000 7ff080c54640 1 --2- 172.21.15.83:0/3125691458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff07c153240 0x7ff07c1536c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.352+0000 7ff080c54640 1 --2- 172.21.15.83:0/3125691458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff07c1fe700 0x7ff07c0cd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.352+0000 7ff080c54640 1 --2- 172.21.15.83:0/3125691458 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff07c1fe320 0x7ff07c0cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.352+0000 7ff080c54640 1 -- 172.21.15.83:0/3125691458 >> 172.21.15.83:0/3125691458 conn(0x7ff07c06abe0 msgr2=0x7ff07c151c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:22.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.352+0000 7ff080c54640 1 -- 172.21.15.83:0/3125691458 shutdown_connections 2024-04-17T10:38:22.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.352+0000 7ff080c54640 1 -- 172.21.15.83:0/3125691458 wait complete. 2024-04-17T10:38:22.356 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2024-04-17T10:38:22.356 DEBUG:teuthology.parallel:result is None 2024-04-17T10:38:22.361 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2024-04-17T10:38:22.361 DEBUG:teuthology.parallel:result is None 2024-04-17T10:38:22.363 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.3 2024-04-17T10:38:22.363 DEBUG:teuthology.parallel:result is None 2024-04-17T10:38:22.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.368+0000 7fc82f26c640 1 -- 172.21.15.83:0/1115042690 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc828072740 con 0x7fc8281669f0 2024-04-17T10:38:22.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.368+0000 7fedefbb9640 1 -- 172.21.15.83:0/1120444975 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fede00df860 con 0x7fede0008330 2024-04-17T10:38:22.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.368+0000 7fc81dffb640 1 -- 172.21.15.83:0/1115042690 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc8200f0d20 con 0x7fc8281669f0 2024-04-17T10:38:22.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.368+0000 7fedde7fc640 1 -- 172.21.15.83:0/1120444975 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fede4056a70 con 0x7fede0008330 2024-04-17T10:38:22.364 INFO:teuthology.orchestra.run.smithi083.stdout:55834574855 2024-04-17T10:38:22.365 INFO:teuthology.orchestra.run.smithi083.stdout:55834574855 2024-04-17T10:38:22.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.368+0000 7f0916724640 1 -- 172.21.15.83:0/1109829749 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f08f0005860 con 0x7f0910159b40 2024-04-17T10:38:22.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.368+0000 7fedefbb9640 1 -- 172.21.15.83:0/1120444975 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fedcc066f30 msgr2=0x7fedcc0693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.368+0000 7fedefbb9640 1 --2- 172.21.15.83:0/1120444975 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fedcc066f30 0x7fedcc0693f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fedd400bd20 tx=0x7fede80968b0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.368+0000 7fedefbb9640 1 -- 172.21.15.83:0/1120444975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fede0008330 msgr2=0x7fede00d5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.368+0000 7fedefbb9640 1 --2- 172.21.15.83:0/1120444975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fede0008330 0x7fede00d5860 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fede400e990 tx=0x7fede400ee50 comp rx=0 tx=0).stop 2024-04-17T10:38:22.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.368+0000 7fc82f26c640 1 -- 172.21.15.83:0/1115042690 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc80c066f60 msgr2=0x7fc80c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.368+0000 7fc82f26c640 1 --2- 172.21.15.83:0/1115042690 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc80c066f60 0x7fc80c069420 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc8280ad810 tx=0x7fc824009040 comp rx=0 tx=0).stop 2024-04-17T10:38:22.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.368+0000 7fc82f26c640 1 -- 172.21.15.83:0/1115042690 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8281669f0 msgr2=0x7fc828165ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.368+0000 7fc82f26c640 1 --2- 172.21.15.83:0/1115042690 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8281669f0 0x7fc828165ba0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc82806bc90 tx=0x7fc820019740 comp rx=0 tx=0).stop 2024-04-17T10:38:22.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.368+0000 7fc82f26c640 1 -- 172.21.15.83:0/1115042690 shutdown_connections 2024-04-17T10:38:22.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.368+0000 7fc82f26c640 1 --2- 172.21.15.83:0/1115042690 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc80c066f60 0x7fc80c069420 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.368+0000 7fc82f26c640 1 --2- 172.21.15.83:0/1115042690 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8281669f0 0x7fc828165ba0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.368+0000 7fc82f26c640 1 --2- 172.21.15.83:0/1115042690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc82815e330 0x7fc82815a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.368+0000 7fc82f26c640 1 --2- 172.21.15.83:0/1115042690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc82806ae10 0x7fc828159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.368+0000 7fc82f26c640 1 -- 172.21.15.83:0/1115042690 >> 172.21.15.83:0/1115042690 conn(0x7fc8280b15b0 msgr2=0x7fc82815e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:22.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.368+0000 7fc82f26c640 1 -- 172.21.15.83:0/1115042690 shutdown_connections 2024-04-17T10:38:22.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.368+0000 7fc82f26c640 1 -- 172.21.15.83:0/1115042690 wait complete. 2024-04-17T10:38:22.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.368+0000 7fedefbb9640 1 -- 172.21.15.83:0/1120444975 shutdown_connections 2024-04-17T10:38:22.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.368+0000 7fedefbb9640 1 --2- 172.21.15.83:0/1120444975 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fedcc066f30 0x7fedcc0693f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.368+0000 7fedefbb9640 1 --2- 172.21.15.83:0/1120444975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fede00d47e0 0x7fede00d7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.368+0000 7fedefbb9640 1 --2- 172.21.15.83:0/1120444975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fede00db460 0x7fede00d7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.368+0000 7fedefbb9640 1 --2- 172.21.15.83:0/1120444975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fede0008330 0x7fede00d5860 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.372+0000 7fedefbb9640 1 -- 172.21.15.83:0/1120444975 >> 172.21.15.83:0/1120444975 conn(0x7fede001e8a0 msgr2=0x7fede0050a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:22.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.372+0000 7fedefbb9640 1 -- 172.21.15.83:0/1120444975 shutdown_connections 2024-04-17T10:38:22.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.372+0000 7fedefbb9640 1 -- 172.21.15.83:0/1120444975 wait complete. 2024-04-17T10:38:22.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.372+0000 7f090cff9640 1 -- 172.21.15.83:0/1109829749 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0908056210 con 0x7f0910159b40 2024-04-17T10:38:22.370 INFO:teuthology.orchestra.run.smithi083.stdout:55834574856 2024-04-17T10:38:22.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.372+0000 7f0916724640 1 -- 172.21.15.83:0/1109829749 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f08fc067030 msgr2=0x7f08fc0694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.372+0000 7f0916724640 1 --2- 172.21.15.83:0/1109829749 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f08fc067030 0x7f08fc0694f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f090000afd0 tx=0x7f090000c040 comp rx=0 tx=0).stop 2024-04-17T10:38:22.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.372+0000 7f0916724640 1 -- 172.21.15.83:0/1109829749 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0910159b40 msgr2=0x7f0910163740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.372+0000 7f0916724640 1 --2- 172.21.15.83:0/1109829749 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0910159b40 0x7f0910163740 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f090800caa0 tx=0x7f090800cf60 comp rx=0 tx=0).stop 2024-04-17T10:38:22.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.372+0000 7f0916724640 1 -- 172.21.15.83:0/1109829749 shutdown_connections 2024-04-17T10:38:22.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.372+0000 7f0916724640 1 --2- 172.21.15.83:0/1109829749 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f08fc067030 0x7f08fc0694f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.372+0000 7f0916724640 1 --2- 172.21.15.83:0/1109829749 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0910159b40 0x7f0910163740 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.372+0000 7f0916724640 1 --2- 172.21.15.83:0/1109829749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09100b3ae0 0x7f0910163200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.372+0000 7f0916724640 1 --2- 172.21.15.83:0/1109829749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09100b3650 0x7f09101616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.372+0000 7f0916724640 1 -- 172.21.15.83:0/1109829749 >> 172.21.15.83:0/1109829749 conn(0x7f09100b1710 msgr2=0x7f09100af680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:22.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.372+0000 7f0916724640 1 -- 172.21.15.83:0/1109829749 shutdown_connections 2024-04-17T10:38:22.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.372+0000 7f0916724640 1 -- 172.21.15.83:0/1109829749 wait complete. 2024-04-17T10:38:22.374 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2024-04-17T10:38:22.375 DEBUG:teuthology.parallel:result is None 2024-04-17T10:38:22.388 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.7 2024-04-17T10:38:22.388 DEBUG:teuthology.parallel:result is None 2024-04-17T10:38:22.395 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2024-04-17T10:38:22.395 DEBUG:teuthology.parallel:result is None 2024-04-17T10:38:22.395 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-04-17T10:38:22.395 DEBUG:teuthology.parallel:result is None 2024-04-17T10:38:22.396 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-17T10:38:22.396 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:38:22.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.548+0000 7f8aca3c2640 1 Processor -- start 2024-04-17T10:38:22.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.548+0000 7f8aca3c2640 1 -- start start 2024-04-17T10:38:22.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.548+0000 7f8aca3c2640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8ac415f090 0x7f8ac40b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.548+0000 7f8aca3c2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8ac40b24e0 0x7f8ac40b1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.548+0000 7f8aca3c2640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac40b1e40 con 0x7f8ac40b17c0 2024-04-17T10:38:22.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.548+0000 7f8aca3c2640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac40b2140 con 0x7f8ac40b24e0 2024-04-17T10:38:22.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.548+0000 7f8aca3c2640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac40b3e70 con 0x7f8ac415f090 2024-04-17T10:38:22.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.548+0000 7f8ac37fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8ac40b24e0 0x7f8ac40b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.548+0000 7f8ac37fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8ac40b24e0 0x7f8ac40b1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48164/0 (socket says 172.21.15.83:48164) 2024-04-17T10:38:22.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.548+0000 7f8ac37fe640 1 -- 172.21.15.83:0/909745097 learned_addr learned my addr 172.21.15.83:0/909745097 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:22.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.548+0000 7f8ac37fe640 1 -- 172.21.15.83:0/909745097 >> v2:172.21.15.167:3300/0 conn(0x7f8ac415f090 msgr2=0x7f8ac40b0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:22.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.548+0000 7f8ac37fe640 1 --2- 172.21.15.83:0/909745097 >> v2:172.21.15.167:3300/0 conn(0x7f8ac415f090 0x7f8ac40b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.548+0000 7f8ac37fe640 1 -- 172.21.15.83:0/909745097 >> v1:172.21.15.167:6789/0 conn(0x7f8ac40b17c0 legacy=0x7f8ac415f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:22.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.548+0000 7f8ac37fe640 1 -- 172.21.15.83:0/909745097 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac40b4110 con 0x7f8ac40b24e0 2024-04-17T10:38:22.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.552+0000 7f8ac37fe640 1 --2- 172.21.15.83:0/909745097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ac40b24e0 0x7f8ac40b1280 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4071ab0 tx=0x7f8ab801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e9bfcbbcba41e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:22.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.552+0000 7f8ac2ffd640 1 -- 172.21.15.83:0/909745097 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab80b0070 con 0x7f8ac40b24e0 2024-04-17T10:38:22.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.552+0000 7f8ac2ffd640 1 -- 172.21.15.83:0/909745097 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab80ab600 con 0x7f8ac40b24e0 2024-04-17T10:38:22.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.552+0000 7f8ac2ffd640 1 -- 172.21.15.83:0/909745097 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac41607b0 con 0x7f8ac40b24e0 2024-04-17T10:38:22.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.552+0000 7f8aca3c2640 1 -- 172.21.15.83:0/909745097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ac40b24e0 msgr2=0x7f8ac40b1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.552+0000 7f8aca3c2640 1 --2- 172.21.15.83:0/909745097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ac40b24e0 0x7f8ac40b1280 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4071ab0 tx=0x7f8ab801c4e0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.552+0000 7f8aca3c2640 1 -- 172.21.15.83:0/909745097 shutdown_connections 2024-04-17T10:38:22.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.552+0000 7f8aca3c2640 1 --2- 172.21.15.83:0/909745097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ac40b24e0 0x7f8ac40b1280 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.552+0000 7f8aca3c2640 1 --2- 172.21.15.83:0/909745097 >> v2:172.21.15.167:3300/0 conn(0x7f8ac415f090 0x7f8ac40b0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.552+0000 7f8aca3c2640 1 -- 172.21.15.83:0/909745097 >> 172.21.15.83:0/909745097 conn(0x7f8ac40b32b0 msgr2=0x7f8ac40b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:22.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.552+0000 7f8aca3c2640 1 -- 172.21.15.83:0/909745097 shutdown_connections 2024-04-17T10:38:22.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.552+0000 7f8aca3c2640 1 -- 172.21.15.83:0/909745097 wait complete. 2024-04-17T10:38:22.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.556+0000 7f8aca3c2640 1 Processor -- start 2024-04-17T10:38:22.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.556+0000 7f8aca3c2640 1 -- start start 2024-04-17T10:38:22.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.556+0000 7f8aca3c2640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ac40b17c0 0x7f8ac414b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.556+0000 7f8aca3c2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ac40b24e0 0x7f8ac415e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.556+0000 7f8aca3c2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ac415f090 0x7f8ac40ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.556+0000 7f8aca3c2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac4168d50 con 0x7f8ac40b24e0 2024-04-17T10:38:22.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.556+0000 7f8aca3c2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac4168bd0 con 0x7f8ac415f090 2024-04-17T10:38:22.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.556+0000 7f8aca3c2640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac4168ed0 con 0x7f8ac40b17c0 2024-04-17T10:38:22.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.556+0000 7f8ac37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ac40b24e0 0x7f8ac415e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.556+0000 7f8ac37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ac40b24e0 0x7f8ac415e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48178/0 (socket says 172.21.15.83:48178) 2024-04-17T10:38:22.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.556+0000 7f8ac37fe640 1 -- 172.21.15.83:0/1862342467 learned_addr learned my addr 172.21.15.83:0/1862342467 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:22.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.556+0000 7f8ac37fe640 1 -- 172.21.15.83:0/1862342467 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ac40b17c0 msgr2=0x7f8ac414b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.556+0000 7f8ac37fe640 1 --2- 172.21.15.83:0/1862342467 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ac40b17c0 0x7f8ac414b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.556+0000 7f8ac37fe640 1 -- 172.21.15.83:0/1862342467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ac415f090 msgr2=0x7f8ac40ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.556+0000 7f8ac37fe640 1 --2- 172.21.15.83:0/1862342467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ac415f090 0x7f8ac40ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.556+0000 7f8ac37fe640 1 -- 172.21.15.83:0/1862342467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac414bb50 con 0x7f8ac40b24e0 2024-04-17T10:38:22.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.556+0000 7f8ac37fe640 1 --2- 172.21.15.83:0/1862342467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ac40b24e0 0x7f8ac415e160 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8ab80ae040 tx=0x7f8ab8019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:22.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.556+0000 7f8ac0ff9640 1 -- 172.21.15.83:0/1862342467 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab80b0040 con 0x7f8ac40b24e0 2024-04-17T10:38:22.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.556+0000 7f8ac0ff9640 1 -- 172.21.15.83:0/1862342467 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab80b9070 con 0x7f8ac40b24e0 2024-04-17T10:38:22.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.556+0000 7f8aca3c2640 1 -- 172.21.15.83:0/1862342467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac40cb210 con 0x7f8ac40b24e0 2024-04-17T10:38:22.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.556+0000 7f8ac0ff9640 1 -- 172.21.15.83:0/1862342467 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab80b48a0 con 0x7f8ac40b24e0 2024-04-17T10:38:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.556+0000 7f8aca3c2640 1 -- 172.21.15.83:0/1862342467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac40b48f0 con 0x7f8ac40b24e0 2024-04-17T10:38:22.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.556+0000 7f8aca3c2640 1 -- 172.21.15.83:0/1862342467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac4074f90 con 0x7f8ac40b24e0 2024-04-17T10:38:22.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.556+0000 7f8ac0ff9640 1 -- 172.21.15.83:0/1862342467 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8ab80ab4c0 con 0x7f8ac40b24e0 2024-04-17T10:38:22.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.556+0000 7f8ac0ff9640 1 --2- 172.21.15.83:0/1862342467 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8ab0066f60 0x7f8ab0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.556+0000 7f8ac0ff9640 1 -- 172.21.15.83:0/1862342467 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ab811ecc0 con 0x7f8ac40b24e0 2024-04-17T10:38:22.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.556+0000 7f8ac3fff640 1 --2- 172.21.15.83:0/1862342467 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8ab0066f60 0x7f8ab0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.560+0000 7f8ac3fff640 1 --2- 172.21.15.83:0/1862342467 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8ab0066f60 0x7f8ab0069420 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009730 tx=0x7f8ab4006ca0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:22.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.564+0000 7f8ac0ff9640 1 -- 172.21.15.83:0/1862342467 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8ab80160d0 con 0x7f8ac40b24e0 2024-04-17T10:38:22.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.704+0000 7f8aca3c2640 1 -- 172.21.15.83:0/1862342467 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ac40c5060 con 0x7f8ab0066f60 2024-04-17T10:38:22.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.720+0000 7f8ac0ff9640 1 -- 172.21.15.83:0/1862342467 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251836 (secure 0 0 0) 0x7f8ac40c5060 con 0x7f8ab0066f60 2024-04-17T10:38:22.717 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:22.721 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:38:22.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.724+0000 7f8aca3c2640 1 -- 172.21.15.83:0/1862342467 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8ab0066f60 msgr2=0x7f8ab0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.724+0000 7f8aca3c2640 1 --2- 172.21.15.83:0/1862342467 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8ab0066f60 0x7f8ab0069420 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009730 tx=0x7f8ab4006ca0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.724+0000 7f8aca3c2640 1 -- 172.21.15.83:0/1862342467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ac40b24e0 msgr2=0x7f8ac415e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.724+0000 7f8aca3c2640 1 --2- 172.21.15.83:0/1862342467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ac40b24e0 0x7f8ac415e160 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8ab80ae040 tx=0x7f8ab8019280 comp rx=0 tx=0).stop 2024-04-17T10:38:22.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.724+0000 7f8aca3c2640 1 -- 172.21.15.83:0/1862342467 shutdown_connections 2024-04-17T10:38:22.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.724+0000 7f8aca3c2640 1 --2- 172.21.15.83:0/1862342467 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8ab0066f60 0x7f8ab0069420 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.724+0000 7f8aca3c2640 1 --2- 172.21.15.83:0/1862342467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ac415f090 0x7f8ac40ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.724+0000 7f8aca3c2640 1 --2- 172.21.15.83:0/1862342467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ac40b24e0 0x7f8ac415e160 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.724+0000 7f8aca3c2640 1 --2- 172.21.15.83:0/1862342467 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ac40b17c0 0x7f8ac414b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.724+0000 7f8aca3c2640 1 -- 172.21.15.83:0/1862342467 >> 172.21.15.83:0/1862342467 conn(0x7f8ac40b32b0 msgr2=0x7f8ac406eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:22.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.724+0000 7f8aca3c2640 1 -- 172.21.15.83:0/1862342467 shutdown_connections 2024-04-17T10:38:22.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.724+0000 7f8aca3c2640 1 -- 172.21.15.83:0/1862342467 wait complete. 2024-04-17T10:38:22.736 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-04-17T10:38:22.182830+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219004,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755716,"statfs":{"total":773094113280,"available":772869853184,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.486193"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702753+0000","last_change":"2024-04-17T10:38:12.718400+0000","last_active":"2024-04-17T10:38:13.702753+0000","last_peered":"2024-04-17T10:38:13.702753+0000","last_clean":"2024-04-17T10:38:13.702753+0000","last_became_active":"2024-04-17T10:38:12.718121+0000","last_became_peered":"2024-04-17T10:38:12.718121+0000","last_unstale":"2024-04-17T10:38:13.702753+0000","last_undegraded":"2024-04-17T10:38:13.702753+0000","last_fullsized":"2024-04-17T10:38:13.702753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:37.203406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695895+0000","last_change":"2024-04-17T10:38:12.710479+0000","last_active":"2024-04-17T10:38:13.695895+0000","last_peered":"2024-04-17T10:38:13.695895+0000","last_clean":"2024-04-17T10:38:13.695895+0000","last_became_active":"2024-04-17T10:38:12.710092+0000","last_became_peered":"2024-04-17T10:38:12.710092+0000","last_unstale":"2024-04-17T10:38:13.695895+0000","last_undegraded":"2024-04-17T10:38:13.695895+0000","last_fullsized":"2024-04-17T10:38:13.695895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:43.917742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305973+0000","last_change":"2024-04-17T10:38:12.722110+0000","last_active":"2024-04-17T10:38:14.305973+0000","last_peered":"2024-04-17T10:38:14.305973+0000","last_clean":"2024-04-17T10:38:14.305973+0000","last_became_active":"2024-04-17T10:38:12.721720+0000","last_became_peered":"2024-04-17T10:38:12.721720+0000","last_unstale":"2024-04-17T10:38:14.305973+0000","last_undegraded":"2024-04-17T10:38:14.305973+0000","last_fullsized":"2024-04-17T10:38:14.305973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:43.917742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970398+0000","last_change":"2024-04-17T10:38:12.727777+0000","last_active":"2024-04-17T10:38:13.970398+0000","last_peered":"2024-04-17T10:38:13.970398+0000","last_clean":"2024-04-17T10:38:13.970398+0000","last_became_active":"2024-04-17T10:38:12.726952+0000","last_became_peered":"2024-04-17T10:38:12.726952+0000","last_unstale":"2024-04-17T10:38:13.970398+0000","last_undegraded":"2024-04-17T10:38:13.970398+0000","last_fullsized":"2024-04-17T10:38:13.970398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:06.508830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701762+0000","last_change":"2024-04-17T10:38:12.721627+0000","last_active":"2024-04-17T10:38:13.701762+0000","last_peered":"2024-04-17T10:38:13.701762+0000","last_clean":"2024-04-17T10:38:13.701762+0000","last_became_active":"2024-04-17T10:38:12.718951+0000","last_became_peered":"2024-04-17T10:38:12.718951+0000","last_unstale":"2024-04-17T10:38:13.701762+0000","last_undegraded":"2024-04-17T10:38:13.701762+0000","last_fullsized":"2024-04-17T10:38:13.701762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:02.184227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970163+0000","last_change":"2024-04-17T10:38:12.729603+0000","last_active":"2024-04-17T10:38:13.970163+0000","last_peered":"2024-04-17T10:38:13.970163+0000","last_clean":"2024-04-17T10:38:13.970163+0000","last_became_active":"2024-04-17T10:38:12.728738+0000","last_became_peered":"2024-04-17T10:38:12.728738+0000","last_unstale":"2024-04-17T10:38:13.970163+0000","last_undegraded":"2024-04-17T10:38:13.970163+0000","last_fullsized":"2024-04-17T10:38:13.970163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:54:26.054113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701635+0000","last_change":"2024-04-17T10:38:12.708732+0000","last_active":"2024-04-17T10:38:13.701635+0000","last_peered":"2024-04-17T10:38:13.701635+0000","last_clean":"2024-04-17T10:38:13.701635+0000","last_became_active":"2024-04-17T10:38:12.708291+0000","last_became_peered":"2024-04-17T10:38:12.708291+0000","last_unstale":"2024-04-17T10:38:13.701635+0000","last_undegraded":"2024-04-17T10:38:13.701635+0000","last_fullsized":"2024-04-17T10:38:13.701635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:54:26.054113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297197+0000","last_change":"2024-04-17T10:38:12.726672+0000","last_active":"2024-04-17T10:38:14.297197+0000","last_peered":"2024-04-17T10:38:14.297197+0000","last_clean":"2024-04-17T10:38:14.297197+0000","last_became_active":"2024-04-17T10:38:12.726395+0000","last_became_peered":"2024-04-17T10:38:12.726395+0000","last_unstale":"2024-04-17T10:38:14.297197+0000","last_undegraded":"2024-04-17T10:38:14.297197+0000","last_fullsized":"2024-04-17T10:38:14.297197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:31:48.663733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702702+0000","last_change":"2024-04-17T10:38:12.727392+0000","last_active":"2024-04-17T10:38:13.702702+0000","last_peered":"2024-04-17T10:38:13.702702+0000","last_clean":"2024-04-17T10:38:13.702702+0000","last_became_active":"2024-04-17T10:38:12.727171+0000","last_became_peered":"2024-04-17T10:38:12.727171+0000","last_unstale":"2024-04-17T10:38:13.702702+0000","last_undegraded":"2024-04-17T10:38:13.702702+0000","last_fullsized":"2024-04-17T10:38:13.702702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:11:49.847070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803824+0000","last_change":"2024-04-17T10:38:12.726158+0000","last_active":"2024-04-17T10:38:13.803824+0000","last_peered":"2024-04-17T10:38:13.803824+0000","last_clean":"2024-04-17T10:38:13.803824+0000","last_became_active":"2024-04-17T10:38:12.724920+0000","last_became_peered":"2024-04-17T10:38:12.724920+0000","last_unstale":"2024-04-17T10:38:13.803824+0000","last_undegraded":"2024-04-17T10:38:13.803824+0000","last_fullsized":"2024-04-17T10:38:13.803824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:43.917742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803453+0000","last_change":"2024-04-17T10:38:12.725420+0000","last_active":"2024-04-17T10:38:13.803453+0000","last_peered":"2024-04-17T10:38:13.803453+0000","last_clean":"2024-04-17T10:38:13.803453+0000","last_became_active":"2024-04-17T10:38:12.724959+0000","last_became_peered":"2024-04-17T10:38:12.724959+0000","last_unstale":"2024-04-17T10:38:13.803453+0000","last_undegraded":"2024-04-17T10:38:13.803453+0000","last_fullsized":"2024-04-17T10:38:13.803453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:39.640730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.699955+0000","last_change":"2024-04-17T10:38:12.712887+0000","last_active":"2024-04-17T10:38:13.699955+0000","last_peered":"2024-04-17T10:38:13.699955+0000","last_clean":"2024-04-17T10:38:13.699955+0000","last_became_active":"2024-04-17T10:38:12.712582+0000","last_became_peered":"2024-04-17T10:38:12.712582+0000","last_unstale":"2024-04-17T10:38:13.699955+0000","last_undegraded":"2024-04-17T10:38:13.699955+0000","last_fullsized":"2024-04-17T10:38:13.699955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:23:30.964531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298210+0000","last_change":"2024-04-17T10:38:12.722191+0000","last_active":"2024-04-17T10:38:14.298210+0000","last_peered":"2024-04-17T10:38:14.298210+0000","last_clean":"2024-04-17T10:38:14.298210+0000","last_became_active":"2024-04-17T10:38:12.721912+0000","last_became_peered":"2024-04-17T10:38:12.721912+0000","last_unstale":"2024-04-17T10:38:14.298210+0000","last_undegraded":"2024-04-17T10:38:14.298210+0000","last_fullsized":"2024-04-17T10:38:14.298210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:09:07.090758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.304662+0000","last_change":"2024-04-17T10:38:12.710211+0000","last_active":"2024-04-17T10:38:14.304662+0000","last_peered":"2024-04-17T10:38:14.304662+0000","last_clean":"2024-04-17T10:38:14.304662+0000","last_became_active":"2024-04-17T10:38:12.709818+0000","last_became_peered":"2024-04-17T10:38:12.709818+0000","last_unstale":"2024-04-17T10:38:14.304662+0000","last_undegraded":"2024-04-17T10:38:14.304662+0000","last_fullsized":"2024-04-17T10:38:14.304662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:23:26.747986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695484+0000","last_change":"2024-04-17T10:38:12.722762+0000","last_active":"2024-04-17T10:38:13.695484+0000","last_peered":"2024-04-17T10:38:13.695484+0000","last_clean":"2024-04-17T10:38:13.695484+0000","last_became_active":"2024-04-17T10:38:12.718394+0000","last_became_peered":"2024-04-17T10:38:12.718394+0000","last_unstale":"2024-04-17T10:38:13.695484+0000","last_undegraded":"2024-04-17T10:38:13.695484+0000","last_fullsized":"2024-04-17T10:38:13.695484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:08.872431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.969994+0000","last_change":"2024-04-17T10:38:12.729270+0000","last_active":"2024-04-17T10:38:13.969994+0000","last_peered":"2024-04-17T10:38:13.969994+0000","last_clean":"2024-04-17T10:38:13.969994+0000","last_became_active":"2024-04-17T10:38:12.728798+0000","last_became_peered":"2024-04-17T10:38:12.728798+0000","last_unstale":"2024-04-17T10:38:13.969994+0000","last_undegraded":"2024-04-17T10:38:13.969994+0000","last_fullsized":"2024-04-17T10:38:13.969994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:23.465112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.971229+0000","last_change":"2024-04-17T10:38:12.731612+0000","last_active":"2024-04-17T10:38:13.971229+0000","last_peered":"2024-04-17T10:38:13.971229+0000","last_clean":"2024-04-17T10:38:13.971229+0000","last_became_active":"2024-04-17T10:38:12.731071+0000","last_became_peered":"2024-04-17T10:38:12.731071+0000","last_unstale":"2024-04-17T10:38:13.971229+0000","last_undegraded":"2024-04-17T10:38:13.971229+0000","last_fullsized":"2024-04-17T10:38:13.971229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:37.203406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.303651+0000","last_change":"2024-04-17T10:38:12.720913+0000","last_active":"2024-04-17T10:38:14.303651+0000","last_peered":"2024-04-17T10:38:14.303651+0000","last_clean":"2024-04-17T10:38:14.303651+0000","last_became_active":"2024-04-17T10:38:12.719779+0000","last_became_peered":"2024-04-17T10:38:12.719779+0000","last_unstale":"2024-04-17T10:38:14.303651+0000","last_undegraded":"2024-04-17T10:38:14.303651+0000","last_fullsized":"2024-04-17T10:38:14.303651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:23:26.747986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702068+0000","last_change":"2024-04-17T10:38:12.719229+0000","last_active":"2024-04-17T10:38:13.702068+0000","last_peered":"2024-04-17T10:38:13.702068+0000","last_clean":"2024-04-17T10:38:13.702068+0000","last_became_active":"2024-04-17T10:38:12.718934+0000","last_became_peered":"2024-04-17T10:38:12.718934+0000","last_unstale":"2024-04-17T10:38:13.702068+0000","last_undegraded":"2024-04-17T10:38:13.702068+0000","last_fullsized":"2024-04-17T10:38:13.702068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:38:39.116589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803564+0000","last_change":"2024-04-17T10:38:12.717999+0000","last_active":"2024-04-17T10:38:13.803564+0000","last_peered":"2024-04-17T10:38:13.803564+0000","last_clean":"2024-04-17T10:38:13.803564+0000","last_became_active":"2024-04-17T10:38:12.716817+0000","last_became_peered":"2024-04-17T10:38:12.716817+0000","last_unstale":"2024-04-17T10:38:13.803564+0000","last_undegraded":"2024-04-17T10:38:13.803564+0000","last_fullsized":"2024-04-17T10:38:13.803564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:42.256740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.304981+0000","last_change":"2024-04-17T10:38:12.710385+0000","last_active":"2024-04-17T10:38:14.304981+0000","last_peered":"2024-04-17T10:38:14.304981+0000","last_clean":"2024-04-17T10:38:14.304981+0000","last_became_active":"2024-04-17T10:38:12.707950+0000","last_became_peered":"2024-04-17T10:38:12.707950+0000","last_unstale":"2024-04-17T10:38:14.304981+0000","last_undegraded":"2024-04-17T10:38:14.304981+0000","last_fullsized":"2024-04-17T10:38:14.304981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:44:43.972627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970232+0000","last_change":"2024-04-17T10:38:12.731401+0000","last_active":"2024-04-17T10:38:13.970232+0000","last_peered":"2024-04-17T10:38:13.970232+0000","last_clean":"2024-04-17T10:38:13.970232+0000","last_became_active":"2024-04-17T10:38:12.729441+0000","last_became_peered":"2024-04-17T10:38:12.729441+0000","last_unstale":"2024-04-17T10:38:13.970232+0000","last_undegraded":"2024-04-17T10:38:13.970232+0000","last_fullsized":"2024-04-17T10:38:13.970232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:02.184227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297301+0000","last_change":"2024-04-17T10:38:12.725791+0000","last_active":"2024-04-17T10:38:14.297301+0000","last_peered":"2024-04-17T10:38:14.297301+0000","last_clean":"2024-04-17T10:38:14.297301+0000","last_became_active":"2024-04-17T10:38:12.723417+0000","last_became_peered":"2024-04-17T10:38:12.723417+0000","last_unstale":"2024-04-17T10:38:14.297301+0000","last_undegraded":"2024-04-17T10:38:14.297301+0000","last_fullsized":"2024-04-17T10:38:14.297301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:26:02.126931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970408+0000","last_change":"2024-04-17T10:38:12.727592+0000","last_active":"2024-04-17T10:38:13.970408+0000","last_peered":"2024-04-17T10:38:13.970408+0000","last_clean":"2024-04-17T10:38:13.970408+0000","last_became_active":"2024-04-17T10:38:12.726482+0000","last_became_peered":"2024-04-17T10:38:12.726482+0000","last_unstale":"2024-04-17T10:38:13.970408+0000","last_undegraded":"2024-04-17T10:38:13.970408+0000","last_fullsized":"2024-04-17T10:38:13.970408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:09:07.090758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297926+0000","last_change":"2024-04-17T10:38:12.716910+0000","last_active":"2024-04-17T10:38:14.297926+0000","last_peered":"2024-04-17T10:38:14.297926+0000","last_clean":"2024-04-17T10:38:14.297926+0000","last_became_active":"2024-04-17T10:38:12.716527+0000","last_became_peered":"2024-04-17T10:38:12.716527+0000","last_unstale":"2024-04-17T10:38:14.297926+0000","last_undegraded":"2024-04-17T10:38:14.297926+0000","last_fullsized":"2024-04-17T10:38:14.297926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:43.917742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803906+0000","last_change":"2024-04-17T10:38:12.728066+0000","last_active":"2024-04-17T10:38:13.803906+0000","last_peered":"2024-04-17T10:38:13.803906+0000","last_clean":"2024-04-17T10:38:13.803906+0000","last_became_active":"2024-04-17T10:38:12.727694+0000","last_became_peered":"2024-04-17T10:38:12.727694+0000","last_unstale":"2024-04-17T10:38:13.803906+0000","last_undegraded":"2024-04-17T10:38:13.803906+0000","last_fullsized":"2024-04-17T10:38:13.803906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:05:21.619291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695757+0000","last_change":"2024-04-17T10:38:12.714887+0000","last_active":"2024-04-17T10:38:13.695757+0000","last_peered":"2024-04-17T10:38:13.695757+0000","last_clean":"2024-04-17T10:38:13.695757+0000","last_became_active":"2024-04-17T10:38:12.714599+0000","last_became_peered":"2024-04-17T10:38:12.714599+0000","last_unstale":"2024-04-17T10:38:13.695757+0000","last_undegraded":"2024-04-17T10:38:13.695757+0000","last_fullsized":"2024-04-17T10:38:13.695757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:42.256740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297989+0000","last_change":"2024-04-17T10:38:12.725678+0000","last_active":"2024-04-17T10:38:14.297989+0000","last_peered":"2024-04-17T10:38:14.297989+0000","last_clean":"2024-04-17T10:38:14.297989+0000","last_became_active":"2024-04-17T10:38:12.723311+0000","last_became_peered":"2024-04-17T10:38:12.723311+0000","last_unstale":"2024-04-17T10:38:14.297989+0000","last_undegraded":"2024-04-17T10:38:14.297989+0000","last_fullsized":"2024-04-17T10:38:14.297989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:05:21.619291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305173+0000","last_change":"2024-04-17T10:38:12.720525+0000","last_active":"2024-04-17T10:38:14.305173+0000","last_peered":"2024-04-17T10:38:14.305173+0000","last_clean":"2024-04-17T10:38:14.305173+0000","last_became_active":"2024-04-17T10:38:12.720134+0000","last_became_peered":"2024-04-17T10:38:12.720134+0000","last_unstale":"2024-04-17T10:38:14.305173+0000","last_undegraded":"2024-04-17T10:38:14.305173+0000","last_fullsized":"2024-04-17T10:38:14.305173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:08.872431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970581+0000","last_change":"2024-04-17T10:38:12.727969+0000","last_active":"2024-04-17T10:38:13.970581+0000","last_peered":"2024-04-17T10:38:13.970581+0000","last_clean":"2024-04-17T10:38:13.970581+0000","last_became_active":"2024-04-17T10:38:12.726330+0000","last_became_peered":"2024-04-17T10:38:12.726330+0000","last_unstale":"2024-04-17T10:38:13.970581+0000","last_undegraded":"2024-04-17T10:38:13.970581+0000","last_fullsized":"2024-04-17T10:38:13.970581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:33:29.583300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.303139+0000","last_change":"2024-04-17T10:38:12.732783+0000","last_active":"2024-04-17T10:38:14.303139+0000","last_peered":"2024-04-17T10:38:14.303139+0000","last_clean":"2024-04-17T10:38:14.303139+0000","last_became_active":"2024-04-17T10:38:12.732525+0000","last_became_peered":"2024-04-17T10:38:12.732525+0000","last_unstale":"2024-04-17T10:38:14.303139+0000","last_undegraded":"2024-04-17T10:38:14.303139+0000","last_fullsized":"2024-04-17T10:38:14.303139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:21:11.071817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803187+0000","last_change":"2024-04-17T10:38:12.720029+0000","last_active":"2024-04-17T10:38:13.803187+0000","last_peered":"2024-04-17T10:38:13.803187+0000","last_clean":"2024-04-17T10:38:13.803187+0000","last_became_active":"2024-04-17T10:38:12.719579+0000","last_became_peered":"2024-04-17T10:38:12.719579+0000","last_unstale":"2024-04-17T10:38:13.803187+0000","last_undegraded":"2024-04-17T10:38:13.803187+0000","last_fullsized":"2024-04-17T10:38:13.803187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:31:48.663733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.699682+0000","last_change":"2024-04-17T10:38:12.709872+0000","last_active":"2024-04-17T10:38:13.699682+0000","last_peered":"2024-04-17T10:38:13.699682+0000","last_clean":"2024-04-17T10:38:13.699682+0000","last_became_active":"2024-04-17T10:38:12.709614+0000","last_became_peered":"2024-04-17T10:38:12.709614+0000","last_unstale":"2024-04-17T10:38:13.699682+0000","last_undegraded":"2024-04-17T10:38:13.699682+0000","last_fullsized":"2024-04-17T10:38:13.699682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:46.375063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298173+0000","last_change":"2024-04-17T10:38:11.718367+0000","last_active":"2024-04-17T10:38:14.298173+0000","last_peered":"2024-04-17T10:38:14.298173+0000","last_clean":"2024-04-17T10:38:14.298173+0000","last_became_active":"2024-04-17T10:38:11.718004+0000","last_became_peered":"2024-04-17T10:38:11.718004+0000","last_unstale":"2024-04-17T10:38:14.298173+0000","last_undegraded":"2024-04-17T10:38:14.298173+0000","last_fullsized":"2024-04-17T10:38:14.298173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:39.210602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305196+0000","last_change":"2024-04-17T10:38:12.710623+0000","last_active":"2024-04-17T10:38:14.305196+0000","last_peered":"2024-04-17T10:38:14.305196+0000","last_clean":"2024-04-17T10:38:14.305196+0000","last_became_active":"2024-04-17T10:38:12.708883+0000","last_became_peered":"2024-04-17T10:38:12.708883+0000","last_unstale":"2024-04-17T10:38:14.305196+0000","last_undegraded":"2024-04-17T10:38:14.305196+0000","last_fullsized":"2024-04-17T10:38:14.305196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:26:02.126931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803042+0000","last_change":"2024-04-17T10:38:11.718224+0000","last_active":"2024-04-17T10:38:13.803042+0000","last_peered":"2024-04-17T10:38:13.803042+0000","last_clean":"2024-04-17T10:38:13.803042+0000","last_became_active":"2024-04-17T10:38:11.717433+0000","last_became_peered":"2024-04-17T10:38:11.717433+0000","last_unstale":"2024-04-17T10:38:13.803042+0000","last_undegraded":"2024-04-17T10:38:13.803042+0000","last_fullsized":"2024-04-17T10:38:13.803042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:39.210602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298350+0000","last_change":"2024-04-17T10:38:11.719972+0000","last_active":"2024-04-17T10:38:14.298350+0000","last_peered":"2024-04-17T10:38:14.298350+0000","last_clean":"2024-04-17T10:38:14.298350+0000","last_became_active":"2024-04-17T10:38:11.719469+0000","last_became_peered":"2024-04-17T10:38:11.719469+0000","last_unstale":"2024-04-17T10:38:14.298350+0000","last_undegraded":"2024-04-17T10:38:14.298350+0000","last_fullsized":"2024-04-17T10:38:14.298350+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:59:51.693435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297286+0000","last_change":"2024-04-17T10:38:12.722655+0000","last_active":"2024-04-17T10:38:14.297286+0000","last_peered":"2024-04-17T10:38:14.297286+0000","last_clean":"2024-04-17T10:38:14.297286+0000","last_became_active":"2024-04-17T10:38:12.722213+0000","last_became_peered":"2024-04-17T10:38:12.722213+0000","last_unstale":"2024-04-17T10:38:14.297286+0000","last_undegraded":"2024-04-17T10:38:14.297286+0000","last_fullsized":"2024-04-17T10:38:14.297286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:08.872431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970337+0000","last_change":"2024-04-17T10:38:11.704898+0000","last_active":"2024-04-17T10:38:13.970337+0000","last_peered":"2024-04-17T10:38:13.970337+0000","last_clean":"2024-04-17T10:38:13.970337+0000","last_became_active":"2024-04-17T10:38:11.704184+0000","last_became_peered":"2024-04-17T10:38:11.704184+0000","last_unstale":"2024-04-17T10:38:13.970337+0000","last_undegraded":"2024-04-17T10:38:13.970337+0000","last_fullsized":"2024-04-17T10:38:13.970337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:45.366572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695486+0000","last_change":"2024-04-17T10:38:12.714844+0000","last_active":"2024-04-17T10:38:13.695486+0000","last_peered":"2024-04-17T10:38:13.695486+0000","last_clean":"2024-04-17T10:38:13.695486+0000","last_became_active":"2024-04-17T10:38:12.714565+0000","last_became_peered":"2024-04-17T10:38:12.714565+0000","last_unstale":"2024-04-17T10:38:13.695486+0000","last_undegraded":"2024-04-17T10:38:13.695486+0000","last_fullsized":"2024-04-17T10:38:13.695486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:26:02.126931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803577+0000","last_change":"2024-04-17T10:38:11.725276+0000","last_active":"2024-04-17T10:38:13.803577+0000","last_peered":"2024-04-17T10:38:13.803577+0000","last_clean":"2024-04-17T10:38:13.803577+0000","last_became_active":"2024-04-17T10:38:11.724660+0000","last_became_peered":"2024-04-17T10:38:11.724660+0000","last_unstale":"2024-04-17T10:38:13.803577+0000","last_undegraded":"2024-04-17T10:38:13.803577+0000","last_fullsized":"2024-04-17T10:38:13.803577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:21:00.724523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970977+0000","last_change":"2024-04-17T10:38:12.731384+0000","last_active":"2024-04-17T10:38:13.970977+0000","last_peered":"2024-04-17T10:38:13.970977+0000","last_clean":"2024-04-17T10:38:13.970977+0000","last_became_active":"2024-04-17T10:38:12.730178+0000","last_became_peered":"2024-04-17T10:38:12.730178+0000","last_unstale":"2024-04-17T10:38:13.970977+0000","last_undegraded":"2024-04-17T10:38:13.970977+0000","last_fullsized":"2024-04-17T10:38:13.970977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:11:49.847070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970956+0000","last_change":"2024-04-17T10:38:11.727591+0000","last_active":"2024-04-17T10:38:13.970956+0000","last_peered":"2024-04-17T10:38:13.970956+0000","last_clean":"2024-04-17T10:38:13.970956+0000","last_became_active":"2024-04-17T10:38:11.726319+0000","last_became_peered":"2024-04-17T10:38:11.726319+0000","last_unstale":"2024-04-17T10:38:13.970956+0000","last_undegraded":"2024-04-17T10:38:13.970956+0000","last_fullsized":"2024-04-17T10:38:13.970956+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:50:59.373356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305290+0000","last_change":"2024-04-17T10:38:12.727239+0000","last_active":"2024-04-17T10:38:14.305290+0000","last_peered":"2024-04-17T10:38:14.305290+0000","last_clean":"2024-04-17T10:38:14.305290+0000","last_became_active":"2024-04-17T10:38:12.726192+0000","last_became_peered":"2024-04-17T10:38:12.726192+0000","last_unstale":"2024-04-17T10:38:14.305290+0000","last_undegraded":"2024-04-17T10:38:14.305290+0000","last_fullsized":"2024-04-17T10:38:14.305290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:39.640730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695810+0000","last_change":"2024-04-17T10:38:11.710534+0000","last_active":"2024-04-17T10:38:13.695810+0000","last_peered":"2024-04-17T10:38:13.695810+0000","last_clean":"2024-04-17T10:38:13.695810+0000","last_became_active":"2024-04-17T10:38:11.709944+0000","last_became_peered":"2024-04-17T10:38:11.709944+0000","last_unstale":"2024-04-17T10:38:13.695810+0000","last_undegraded":"2024-04-17T10:38:13.695810+0000","last_fullsized":"2024-04-17T10:38:13.695810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:45.366572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.700329+0000","last_change":"2024-04-17T10:38:12.709577+0000","last_active":"2024-04-17T10:38:13.700329+0000","last_peered":"2024-04-17T10:38:13.700329+0000","last_clean":"2024-04-17T10:38:13.700329+0000","last_became_active":"2024-04-17T10:38:12.709186+0000","last_became_peered":"2024-04-17T10:38:12.709186+0000","last_unstale":"2024-04-17T10:38:13.700329+0000","last_undegraded":"2024-04-17T10:38:13.700329+0000","last_fullsized":"2024-04-17T10:38:13.700329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:20:45.810408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803828+0000","last_change":"2024-04-17T10:38:11.715117+0000","last_active":"2024-04-17T10:38:13.803828+0000","last_peered":"2024-04-17T10:38:13.803828+0000","last_clean":"2024-04-17T10:38:13.803828+0000","last_became_active":"2024-04-17T10:38:11.713981+0000","last_became_peered":"2024-04-17T10:38:11.713981+0000","last_unstale":"2024-04-17T10:38:13.803828+0000","last_undegraded":"2024-04-17T10:38:13.803828+0000","last_fullsized":"2024-04-17T10:38:13.803828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:27.569244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701871+0000","last_change":"2024-04-17T10:38:12.708377+0000","last_active":"2024-04-17T10:38:13.701871+0000","last_peered":"2024-04-17T10:38:13.701871+0000","last_clean":"2024-04-17T10:38:13.701871+0000","last_became_active":"2024-04-17T10:38:12.707183+0000","last_became_peered":"2024-04-17T10:38:12.707183+0000","last_unstale":"2024-04-17T10:38:13.701871+0000","last_undegraded":"2024-04-17T10:38:13.701871+0000","last_fullsized":"2024-04-17T10:38:13.701871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:06.508830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305423+0000","last_change":"2024-04-17T10:38:11.702326+0000","last_active":"2024-04-17T10:38:14.305423+0000","last_peered":"2024-04-17T10:38:14.305423+0000","last_clean":"2024-04-17T10:38:14.305423+0000","last_became_active":"2024-04-17T10:38:11.701735+0000","last_became_peered":"2024-04-17T10:38:11.701735+0000","last_unstale":"2024-04-17T10:38:14.305423+0000","last_undegraded":"2024-04-17T10:38:14.305423+0000","last_fullsized":"2024-04-17T10:38:14.305423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:11:43.267871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.304822+0000","last_change":"2024-04-17T10:38:12.705702+0000","last_active":"2024-04-17T10:38:14.304822+0000","last_peered":"2024-04-17T10:38:14.304822+0000","last_clean":"2024-04-17T10:38:14.304822+0000","last_became_active":"2024-04-17T10:38:12.705297+0000","last_became_peered":"2024-04-17T10:38:12.705297+0000","last_unstale":"2024-04-17T10:38:14.304822+0000","last_undegraded":"2024-04-17T10:38:14.304822+0000","last_fullsized":"2024-04-17T10:38:14.304822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:15:31.987120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970479+0000","last_change":"2024-04-17T10:38:11.715689+0000","last_active":"2024-04-17T10:38:13.970479+0000","last_peered":"2024-04-17T10:38:13.970479+0000","last_clean":"2024-04-17T10:38:13.970479+0000","last_became_active":"2024-04-17T10:38:11.714329+0000","last_became_peered":"2024-04-17T10:38:11.714329+0000","last_unstale":"2024-04-17T10:38:13.970479+0000","last_undegraded":"2024-04-17T10:38:13.970479+0000","last_fullsized":"2024-04-17T10:38:13.970479+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:27.569244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702011+0000","last_change":"2024-04-17T10:38:12.708536+0000","last_active":"2024-04-17T10:38:13.702011+0000","last_peered":"2024-04-17T10:38:13.702011+0000","last_clean":"2024-04-17T10:38:13.702011+0000","last_became_active":"2024-04-17T10:38:12.707643+0000","last_became_peered":"2024-04-17T10:38:12.707643+0000","last_unstale":"2024-04-17T10:38:13.702011+0000","last_undegraded":"2024-04-17T10:38:13.702011+0000","last_fullsized":"2024-04-17T10:38:13.702011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:22:10.364740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803484+0000","last_change":"2024-04-17T10:38:11.717992+0000","last_active":"2024-04-17T10:38:13.803484+0000","last_peered":"2024-04-17T10:38:13.803484+0000","last_clean":"2024-04-17T10:38:13.803484+0000","last_became_active":"2024-04-17T10:38:11.717124+0000","last_became_peered":"2024-04-17T10:38:11.717124+0000","last_unstale":"2024-04-17T10:38:13.803484+0000","last_undegraded":"2024-04-17T10:38:13.803484+0000","last_fullsized":"2024-04-17T10:38:13.803484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:24:23.818791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803260+0000","last_change":"2024-04-17T10:38:12.720282+0000","last_active":"2024-04-17T10:38:13.803260+0000","last_peered":"2024-04-17T10:38:13.803260+0000","last_clean":"2024-04-17T10:38:13.803260+0000","last_became_active":"2024-04-17T10:38:12.719197+0000","last_became_peered":"2024-04-17T10:38:12.719197+0000","last_unstale":"2024-04-17T10:38:13.803260+0000","last_undegraded":"2024-04-17T10:38:13.803260+0000","last_fullsized":"2024-04-17T10:38:13.803260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:08.872431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.303153+0000","last_change":"2024-04-17T10:38:11.689943+0000","last_active":"2024-04-17T10:38:14.303153+0000","last_peered":"2024-04-17T10:38:14.303153+0000","last_clean":"2024-04-17T10:38:14.303153+0000","last_became_active":"2024-04-17T10:38:11.689627+0000","last_became_peered":"2024-04-17T10:38:11.689627+0000","last_unstale":"2024-04-17T10:38:14.303153+0000","last_undegraded":"2024-04-17T10:38:14.303153+0000","last_fullsized":"2024-04-17T10:38:14.303153+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:39.210602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701741+0000","last_change":"2024-04-17T10:38:12.710693+0000","last_active":"2024-04-17T10:38:13.701741+0000","last_peered":"2024-04-17T10:38:13.701741+0000","last_clean":"2024-04-17T10:38:13.701741+0000","last_became_active":"2024-04-17T10:38:12.710263+0000","last_became_peered":"2024-04-17T10:38:12.710263+0000","last_unstale":"2024-04-17T10:38:13.701741+0000","last_undegraded":"2024-04-17T10:38:13.701741+0000","last_fullsized":"2024-04-17T10:38:13.701741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:09:07.090758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.700517+0000","last_change":"2024-04-17T10:38:11.701185+0000","last_active":"2024-04-17T10:38:13.700517+0000","last_peered":"2024-04-17T10:38:13.700517+0000","last_clean":"2024-04-17T10:38:13.700517+0000","last_became_active":"2024-04-17T10:38:11.700292+0000","last_became_peered":"2024-04-17T10:38:11.700292+0000","last_unstale":"2024-04-17T10:38:13.700517+0000","last_undegraded":"2024-04-17T10:38:13.700517+0000","last_fullsized":"2024-04-17T10:38:13.700517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:59:51.693435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702731+0000","last_change":"2024-04-17T10:38:12.704267+0000","last_active":"2024-04-17T10:38:13.702731+0000","last_peered":"2024-04-17T10:38:13.702731+0000","last_clean":"2024-04-17T10:38:13.702731+0000","last_became_active":"2024-04-17T10:38:12.703820+0000","last_became_peered":"2024-04-17T10:38:12.703820+0000","last_unstale":"2024-04-17T10:38:13.702731+0000","last_undegraded":"2024-04-17T10:38:13.702731+0000","last_fullsized":"2024-04-17T10:38:13.702731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:43:51.398103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.303738+0000","last_change":"2024-04-17T10:38:11.699100+0000","last_active":"2024-04-17T10:38:14.303738+0000","last_peered":"2024-04-17T10:38:14.303738+0000","last_clean":"2024-04-17T10:38:14.303738+0000","last_became_active":"2024-04-17T10:38:11.698786+0000","last_became_peered":"2024-04-17T10:38:11.698786+0000","last_unstale":"2024-04-17T10:38:14.303738+0000","last_undegraded":"2024-04-17T10:38:14.303738+0000","last_fullsized":"2024-04-17T10:38:14.303738+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:21:00.724523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702808+0000","last_change":"2024-04-17T10:38:12.705166+0000","last_active":"2024-04-17T10:38:13.702808+0000","last_peered":"2024-04-17T10:38:13.702808+0000","last_clean":"2024-04-17T10:38:13.702808+0000","last_became_active":"2024-04-17T10:38:12.704805+0000","last_became_peered":"2024-04-17T10:38:12.704805+0000","last_unstale":"2024-04-17T10:38:13.702808+0000","last_undegraded":"2024-04-17T10:38:13.702808+0000","last_fullsized":"2024-04-17T10:38:13.702808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:49:18.397560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.696028+0000","last_change":"2024-04-17T10:38:11.692209+0000","last_active":"2024-04-17T10:38:13.696028+0000","last_peered":"2024-04-17T10:38:13.696028+0000","last_clean":"2024-04-17T10:38:13.696028+0000","last_became_active":"2024-04-17T10:38:11.691905+0000","last_became_peered":"2024-04-17T10:38:11.691905+0000","last_unstale":"2024-04-17T10:38:13.696028+0000","last_undegraded":"2024-04-17T10:38:13.696028+0000","last_fullsized":"2024-04-17T10:38:13.696028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:50:59.373356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970996+0000","last_change":"2024-04-17T10:38:12.727145+0000","last_active":"2024-04-17T10:38:13.970996+0000","last_peered":"2024-04-17T10:38:13.970996+0000","last_clean":"2024-04-17T10:38:13.970996+0000","last_became_active":"2024-04-17T10:38:12.726797+0000","last_became_peered":"2024-04-17T10:38:12.726797+0000","last_unstale":"2024-04-17T10:38:13.970996+0000","last_undegraded":"2024-04-17T10:38:13.970996+0000","last_fullsized":"2024-04-17T10:38:13.970996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:43:51.398103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.303519+0000","last_change":"2024-04-17T10:38:11.705218+0000","last_active":"2024-04-17T10:38:14.303519+0000","last_peered":"2024-04-17T10:38:14.303519+0000","last_clean":"2024-04-17T10:38:14.303519+0000","last_became_active":"2024-04-17T10:38:11.704928+0000","last_became_peered":"2024-04-17T10:38:11.704928+0000","last_unstale":"2024-04-17T10:38:14.303519+0000","last_undegraded":"2024-04-17T10:38:14.303519+0000","last_fullsized":"2024-04-17T10:38:14.303519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:27.569244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701937+0000","last_change":"2024-04-17T10:38:12.722659+0000","last_active":"2024-04-17T10:38:13.701937+0000","last_peered":"2024-04-17T10:38:13.701937+0000","last_clean":"2024-04-17T10:38:13.701937+0000","last_became_active":"2024-04-17T10:38:12.722265+0000","last_became_peered":"2024-04-17T10:38:12.722265+0000","last_unstale":"2024-04-17T10:38:13.701937+0000","last_undegraded":"2024-04-17T10:38:13.701937+0000","last_fullsized":"2024-04-17T10:38:13.701937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:33:29.583300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701282+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:38:13.701282+0000","last_peered":"2024-04-17T10:38:13.701282+0000","last_clean":"2024-04-17T10:38:13.701282+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:38:13.701282+0000","last_undegraded":"2024-04-17T10:38:13.701282+0000","last_fullsized":"2024-04-17T10:38:13.701282+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970082+0000","last_change":"2024-04-17T10:38:11.702024+0000","last_active":"2024-04-17T10:38:13.970082+0000","last_peered":"2024-04-17T10:38:13.970082+0000","last_clean":"2024-04-17T10:38:13.970082+0000","last_became_active":"2024-04-17T10:38:11.701157+0000","last_became_peered":"2024-04-17T10:38:11.701157+0000","last_unstale":"2024-04-17T10:38:13.970082+0000","last_undegraded":"2024-04-17T10:38:13.970082+0000","last_fullsized":"2024-04-17T10:38:13.970082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:24:23.818791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297692+0000","last_change":"2024-04-17T10:38:12.723529+0000","last_active":"2024-04-17T10:38:14.297692+0000","last_peered":"2024-04-17T10:38:14.297692+0000","last_clean":"2024-04-17T10:38:14.297692+0000","last_became_active":"2024-04-17T10:38:12.723103+0000","last_became_peered":"2024-04-17T10:38:12.723103+0000","last_unstale":"2024-04-17T10:38:14.297692+0000","last_undegraded":"2024-04-17T10:38:14.297692+0000","last_fullsized":"2024-04-17T10:38:14.297692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:42.256740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701197+0000","last_change":"2024-04-17T10:38:11.716453+0000","last_active":"2024-04-17T10:38:13.701197+0000","last_peered":"2024-04-17T10:38:13.701197+0000","last_clean":"2024-04-17T10:38:13.701197+0000","last_became_active":"2024-04-17T10:38:11.715859+0000","last_became_peered":"2024-04-17T10:38:11.715859+0000","last_unstale":"2024-04-17T10:38:13.701197+0000","last_undegraded":"2024-04-17T10:38:13.701197+0000","last_fullsized":"2024-04-17T10:38:13.701197+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:50:59.373356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.304804+0000","last_change":"2024-04-17T10:38:12.707314+0000","last_active":"2024-04-17T10:38:14.304804+0000","last_peered":"2024-04-17T10:38:14.304804+0000","last_clean":"2024-04-17T10:38:14.304804+0000","last_became_active":"2024-04-17T10:38:12.706874+0000","last_became_peered":"2024-04-17T10:38:12.706874+0000","last_unstale":"2024-04-17T10:38:14.304804+0000","last_undegraded":"2024-04-17T10:38:14.304804+0000","last_fullsized":"2024-04-17T10:38:14.304804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:07:07.037555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.303300+0000","last_change":"2024-04-17T10:38:11.699752+0000","last_active":"2024-04-17T10:38:14.303300+0000","last_peered":"2024-04-17T10:38:14.303300+0000","last_clean":"2024-04-17T10:38:14.303300+0000","last_became_active":"2024-04-17T10:38:11.699467+0000","last_became_peered":"2024-04-17T10:38:11.699467+0000","last_unstale":"2024-04-17T10:38:14.303300+0000","last_undegraded":"2024-04-17T10:38:14.303300+0000","last_fullsized":"2024-04-17T10:38:14.303300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:24:23.818791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695234+0000","last_change":"2024-04-17T10:38:12.723809+0000","last_active":"2024-04-17T10:38:13.695234+0000","last_peered":"2024-04-17T10:38:13.695234+0000","last_clean":"2024-04-17T10:38:13.695234+0000","last_became_active":"2024-04-17T10:38:12.723282+0000","last_became_peered":"2024-04-17T10:38:12.723282+0000","last_unstale":"2024-04-17T10:38:13.695234+0000","last_undegraded":"2024-04-17T10:38:13.695234+0000","last_fullsized":"2024-04-17T10:38:13.695234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:31:48.663733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.971078+0000","last_change":"2024-04-17T10:38:11.715164+0000","last_active":"2024-04-17T10:38:13.971078+0000","last_peered":"2024-04-17T10:38:13.971078+0000","last_clean":"2024-04-17T10:38:13.971078+0000","last_became_active":"2024-04-17T10:38:11.712376+0000","last_became_peered":"2024-04-17T10:38:11.712376+0000","last_unstale":"2024-04-17T10:38:13.971078+0000","last_undegraded":"2024-04-17T10:38:13.971078+0000","last_fullsized":"2024-04-17T10:38:13.971078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:21:00.724523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.699765+0000","last_change":"2024-04-17T10:38:12.720339+0000","last_active":"2024-04-17T10:38:13.699765+0000","last_peered":"2024-04-17T10:38:13.699765+0000","last_clean":"2024-04-17T10:38:13.699765+0000","last_became_active":"2024-04-17T10:38:12.720051+0000","last_became_peered":"2024-04-17T10:38:12.720051+0000","last_unstale":"2024-04-17T10:38:13.699765+0000","last_undegraded":"2024-04-17T10:38:13.699765+0000","last_fullsized":"2024-04-17T10:38:13.699765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:51:00.381846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695713+0000","last_change":"2024-04-17T10:38:11.693095+0000","last_active":"2024-04-17T10:38:13.695713+0000","last_peered":"2024-04-17T10:38:13.695713+0000","last_clean":"2024-04-17T10:38:13.695713+0000","last_became_active":"2024-04-17T10:38:11.692723+0000","last_became_peered":"2024-04-17T10:38:11.692723+0000","last_unstale":"2024-04-17T10:38:13.695713+0000","last_undegraded":"2024-04-17T10:38:13.695713+0000","last_fullsized":"2024-04-17T10:38:13.695713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:27.569244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970920+0000","last_change":"2024-04-17T10:38:12.728559+0000","last_active":"2024-04-17T10:38:13.970920+0000","last_peered":"2024-04-17T10:38:13.970920+0000","last_clean":"2024-04-17T10:38:13.970920+0000","last_became_active":"2024-04-17T10:38:12.728285+0000","last_became_peered":"2024-04-17T10:38:12.728285+0000","last_unstale":"2024-04-17T10:38:13.970920+0000","last_undegraded":"2024-04-17T10:38:13.970920+0000","last_fullsized":"2024-04-17T10:38:13.970920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:22:10.364740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702148+0000","last_change":"2024-04-17T10:38:11.717791+0000","last_active":"2024-04-17T10:38:13.702148+0000","last_peered":"2024-04-17T10:38:13.702148+0000","last_clean":"2024-04-17T10:38:13.702148+0000","last_became_active":"2024-04-17T10:38:11.717130+0000","last_became_peered":"2024-04-17T10:38:11.717130+0000","last_unstale":"2024-04-17T10:38:13.702148+0000","last_undegraded":"2024-04-17T10:38:13.702148+0000","last_fullsized":"2024-04-17T10:38:13.702148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:21:10.063327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695581+0000","last_change":"2024-04-17T10:38:12.723436+0000","last_active":"2024-04-17T10:38:13.695581+0000","last_peered":"2024-04-17T10:38:13.695581+0000","last_clean":"2024-04-17T10:38:13.695581+0000","last_became_active":"2024-04-17T10:38:12.723138+0000","last_became_peered":"2024-04-17T10:38:12.723138+0000","last_unstale":"2024-04-17T10:38:13.695581+0000","last_undegraded":"2024-04-17T10:38:13.695581+0000","last_fullsized":"2024-04-17T10:38:13.695581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:39.640730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.365894+0000","last_change":"2024-04-17T10:38:11.730180+0000","last_active":"2024-04-17T10:38:14.365894+0000","last_peered":"2024-04-17T10:38:14.365894+0000","last_clean":"2024-04-17T10:38:14.365894+0000","last_became_active":"2024-04-17T10:38:11.729843+0000","last_became_peered":"2024-04-17T10:38:11.729843+0000","last_unstale":"2024-04-17T10:38:14.365894+0000","last_undegraded":"2024-04-17T10:38:14.365894+0000","last_fullsized":"2024-04-17T10:38:14.365894+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:23:29.956041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298370+0000","last_change":"2024-04-17T10:38:12.732643+0000","last_active":"2024-04-17T10:38:14.298370+0000","last_peered":"2024-04-17T10:38:14.298370+0000","last_clean":"2024-04-17T10:38:14.298370+0000","last_became_active":"2024-04-17T10:38:12.732259+0000","last_became_peered":"2024-04-17T10:38:12.732259+0000","last_unstale":"2024-04-17T10:38:14.298370+0000","last_undegraded":"2024-04-17T10:38:14.298370+0000","last_fullsized":"2024-04-17T10:38:14.298370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:02.184227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803741+0000","last_change":"2024-04-17T10:38:11.725444+0000","last_active":"2024-04-17T10:38:13.803741+0000","last_peered":"2024-04-17T10:38:13.803741+0000","last_clean":"2024-04-17T10:38:13.803741+0000","last_became_active":"2024-04-17T10:38:11.724780+0000","last_became_peered":"2024-04-17T10:38:11.724780+0000","last_unstale":"2024-04-17T10:38:13.803741+0000","last_undegraded":"2024-04-17T10:38:13.803741+0000","last_fullsized":"2024-04-17T10:38:13.803741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:45.366572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305401+0000","last_change":"2024-04-17T10:38:12.705770+0000","last_active":"2024-04-17T10:38:14.305401+0000","last_peered":"2024-04-17T10:38:14.305401+0000","last_clean":"2024-04-17T10:38:14.305401+0000","last_became_active":"2024-04-17T10:38:12.705400+0000","last_became_peered":"2024-04-17T10:38:12.705400+0000","last_unstale":"2024-04-17T10:38:14.305401+0000","last_undegraded":"2024-04-17T10:38:14.305401+0000","last_fullsized":"2024-04-17T10:38:14.305401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:42.256740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701197+0000","last_change":"2024-04-17T10:38:11.716433+0000","last_active":"2024-04-17T10:38:13.701197+0000","last_peered":"2024-04-17T10:38:13.701197+0000","last_clean":"2024-04-17T10:38:13.701197+0000","last_became_active":"2024-04-17T10:38:11.715803+0000","last_became_peered":"2024-04-17T10:38:11.715803+0000","last_unstale":"2024-04-17T10:38:13.701197+0000","last_undegraded":"2024-04-17T10:38:13.701197+0000","last_fullsized":"2024-04-17T10:38:13.701197+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:45.366572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.700697+0000","last_change":"2024-04-17T10:38:12.717613+0000","last_active":"2024-04-17T10:38:13.700697+0000","last_peered":"2024-04-17T10:38:13.700697+0000","last_clean":"2024-04-17T10:38:13.700697+0000","last_became_active":"2024-04-17T10:38:12.717308+0000","last_became_peered":"2024-04-17T10:38:12.717308+0000","last_unstale":"2024-04-17T10:38:13.700697+0000","last_undegraded":"2024-04-17T10:38:13.700697+0000","last_fullsized":"2024-04-17T10:38:13.700697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:21:11.071817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695770+0000","last_change":"2024-04-17T10:38:11.716778+0000","last_active":"2024-04-17T10:38:13.695770+0000","last_peered":"2024-04-17T10:38:13.695770+0000","last_clean":"2024-04-17T10:38:13.695770+0000","last_became_active":"2024-04-17T10:38:11.716526+0000","last_became_peered":"2024-04-17T10:38:11.716526+0000","last_unstale":"2024-04-17T10:38:13.695770+0000","last_undegraded":"2024-04-17T10:38:13.695770+0000","last_fullsized":"2024-04-17T10:38:13.695770+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:24:23.818791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297622+0000","last_change":"2024-04-17T10:38:12.717941+0000","last_active":"2024-04-17T10:38:14.297622+0000","last_peered":"2024-04-17T10:38:14.297622+0000","last_clean":"2024-04-17T10:38:14.297622+0000","last_became_active":"2024-04-17T10:38:12.717350+0000","last_became_peered":"2024-04-17T10:38:12.717350+0000","last_unstale":"2024-04-17T10:38:14.297622+0000","last_undegraded":"2024-04-17T10:38:14.297622+0000","last_fullsized":"2024-04-17T10:38:14.297622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:39.640730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695661+0000","last_change":"2024-04-17T10:38:11.699325+0000","last_active":"2024-04-17T10:38:13.695661+0000","last_peered":"2024-04-17T10:38:13.695661+0000","last_clean":"2024-04-17T10:38:13.695661+0000","last_became_active":"2024-04-17T10:38:11.698981+0000","last_became_peered":"2024-04-17T10:38:11.698981+0000","last_unstale":"2024-04-17T10:38:13.695661+0000","last_undegraded":"2024-04-17T10:38:13.695661+0000","last_fullsized":"2024-04-17T10:38:13.695661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:21:00.724523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298006+0000","last_change":"2024-04-17T10:38:12.731391+0000","last_active":"2024-04-17T10:38:14.298006+0000","last_peered":"2024-04-17T10:38:14.298006+0000","last_clean":"2024-04-17T10:38:14.298006+0000","last_became_active":"2024-04-17T10:38:12.727638+0000","last_became_peered":"2024-04-17T10:38:12.727638+0000","last_unstale":"2024-04-17T10:38:14.298006+0000","last_undegraded":"2024-04-17T10:38:14.298006+0000","last_fullsized":"2024-04-17T10:38:14.298006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:23.465112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701909+0000","last_change":"2024-04-17T10:38:11.716353+0000","last_active":"2024-04-17T10:38:13.701909+0000","last_peered":"2024-04-17T10:38:13.701909+0000","last_clean":"2024-04-17T10:38:13.701909+0000","last_became_active":"2024-04-17T10:38:11.715722+0000","last_became_peered":"2024-04-17T10:38:11.715722+0000","last_unstale":"2024-04-17T10:38:13.701909+0000","last_undegraded":"2024-04-17T10:38:13.701909+0000","last_fullsized":"2024-04-17T10:38:13.701909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:20:44.801917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803339+0000","last_change":"2024-04-17T10:38:12.715919+0000","last_active":"2024-04-17T10:38:13.803339+0000","last_peered":"2024-04-17T10:38:13.803339+0000","last_clean":"2024-04-17T10:38:13.803339+0000","last_became_active":"2024-04-17T10:38:12.715568+0000","last_became_peered":"2024-04-17T10:38:12.715568+0000","last_unstale":"2024-04-17T10:38:13.803339+0000","last_undegraded":"2024-04-17T10:38:13.803339+0000","last_fullsized":"2024-04-17T10:38:13.803339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:26:02.126931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970940+0000","last_change":"2024-04-17T10:38:12.724977+0000","last_active":"2024-04-17T10:38:13.970940+0000","last_peered":"2024-04-17T10:38:13.970940+0000","last_clean":"2024-04-17T10:38:13.970940+0000","last_became_active":"2024-04-17T10:38:12.724700+0000","last_became_peered":"2024-04-17T10:38:12.724700+0000","last_unstale":"2024-04-17T10:38:13.970940+0000","last_undegraded":"2024-04-17T10:38:13.970940+0000","last_fullsized":"2024-04-17T10:38:13.970940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:38:39.116589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702197+0000","last_change":"2024-04-17T10:38:11.719799+0000","last_active":"2024-04-17T10:38:13.702197+0000","last_peered":"2024-04-17T10:38:13.702197+0000","last_clean":"2024-04-17T10:38:13.702197+0000","last_became_active":"2024-04-17T10:38:11.719147+0000","last_became_peered":"2024-04-17T10:38:11.719147+0000","last_unstale":"2024-04-17T10:38:13.702197+0000","last_undegraded":"2024-04-17T10:38:13.702197+0000","last_fullsized":"2024-04-17T10:38:13.702197+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:23:29.956041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.971331+0000","last_change":"2024-04-17T10:38:12.725123+0000","last_active":"2024-04-17T10:38:13.971331+0000","last_peered":"2024-04-17T10:38:13.971331+0000","last_clean":"2024-04-17T10:38:13.971331+0000","last_became_active":"2024-04-17T10:38:12.724760+0000","last_became_peered":"2024-04-17T10:38:12.724760+0000","last_unstale":"2024-04-17T10:38:13.971331+0000","last_undegraded":"2024-04-17T10:38:13.971331+0000","last_fullsized":"2024-04-17T10:38:13.971331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:49:18.397560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702167+0000","last_change":"2024-04-17T10:38:11.727050+0000","last_active":"2024-04-17T10:38:13.702167+0000","last_peered":"2024-04-17T10:38:13.702167+0000","last_clean":"2024-04-17T10:38:13.702167+0000","last_became_active":"2024-04-17T10:38:11.726393+0000","last_became_peered":"2024-04-17T10:38:11.726393+0000","last_unstale":"2024-04-17T10:38:13.702167+0000","last_undegraded":"2024-04-17T10:38:13.702167+0000","last_fullsized":"2024-04-17T10:38:13.702167+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:23:25.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305077+0000","last_change":"2024-04-17T10:38:12.726864+0000","last_active":"2024-04-17T10:38:14.305077+0000","last_peered":"2024-04-17T10:38:14.305077+0000","last_clean":"2024-04-17T10:38:14.305077+0000","last_became_active":"2024-04-17T10:38:12.726581+0000","last_became_peered":"2024-04-17T10:38:12.726581+0000","last_unstale":"2024-04-17T10:38:14.305077+0000","last_undegraded":"2024-04-17T10:38:14.305077+0000","last_fullsized":"2024-04-17T10:38:14.305077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:31:48.663733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298342+0000","last_change":"2024-04-17T10:38:11.700825+0000","last_active":"2024-04-17T10:38:14.298342+0000","last_peered":"2024-04-17T10:38:14.298342+0000","last_clean":"2024-04-17T10:38:14.298342+0000","last_became_active":"2024-04-17T10:38:11.700354+0000","last_became_peered":"2024-04-17T10:38:11.700354+0000","last_unstale":"2024-04-17T10:38:14.298342+0000","last_undegraded":"2024-04-17T10:38:14.298342+0000","last_fullsized":"2024-04-17T10:38:14.298342+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:22:08.746889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298080+0000","last_change":"2024-04-17T10:38:12.719771+0000","last_active":"2024-04-17T10:38:14.298080+0000","last_peered":"2024-04-17T10:38:14.298080+0000","last_clean":"2024-04-17T10:38:14.298080+0000","last_became_active":"2024-04-17T10:38:12.716013+0000","last_became_peered":"2024-04-17T10:38:12.716013+0000","last_unstale":"2024-04-17T10:38:14.298080+0000","last_undegraded":"2024-04-17T10:38:14.298080+0000","last_fullsized":"2024-04-17T10:38:14.298080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:54:26.054113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305389+0000","last_change":"2024-04-17T10:38:11.710717+0000","last_active":"2024-04-17T10:38:14.305389+0000","last_peered":"2024-04-17T10:38:14.305389+0000","last_clean":"2024-04-17T10:38:14.305389+0000","last_became_active":"2024-04-17T10:38:11.710342+0000","last_became_peered":"2024-04-17T10:38:11.710342+0000","last_unstale":"2024-04-17T10:38:14.305389+0000","last_undegraded":"2024-04-17T10:38:14.305389+0000","last_fullsized":"2024-04-17T10:38:14.305389+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:16:30.923882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:38:22.740 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:38:22.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.900+0000 7fbf94df7640 1 Processor -- start 2024-04-17T10:38:22.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.900+0000 7fbf94df7640 1 -- start start 2024-04-17T10:38:22.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.904+0000 7fbf94df7640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbf9015ba30 0x7fbf90072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.904+0000 7fbf94df7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf9015c960 0x7fbf900cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.904+0000 7fbf94df7640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbf900cfa10 0x7fbf9006c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.904+0000 7fbf94df7640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf90071ea0 con 0x7fbf900cfa10 2024-04-17T10:38:22.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.904+0000 7fbf94df7640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf900cfea0 con 0x7fbf9015ba30 2024-04-17T10:38:22.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.904+0000 7fbf94df7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9006e320 con 0x7fbf9015c960 2024-04-17T10:38:22.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.904+0000 7fbf8ed9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbf900cfa10 0x7fbf9006c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.904+0000 7fbf8ed9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbf900cfa10 0x7fbf9006c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48180/0 (socket says 172.21.15.83:48180) 2024-04-17T10:38:22.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.904+0000 7fbf8ed9d640 1 -- 172.21.15.83:0/2078886721 learned_addr learned my addr 172.21.15.83:0/2078886721 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:22.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.904+0000 7fbf8e59c640 1 --2- 172.21.15.83:0/2078886721 >> v2:172.21.15.167:3300/0 conn(0x7fbf9015ba30 0x7fbf90072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.904+0000 7fbf8dd9b640 1 --2- 172.21.15.83:0/2078886721 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf9015c960 0x7fbf900cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.904+0000 7fbf8ed9d640 1 -- 172.21.15.83:0/2078886721 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf9015c960 msgr2=0x7fbf900cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:22.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.904+0000 7fbf8ed9d640 1 --2- 172.21.15.83:0/2078886721 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf9015c960 0x7fbf900cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.904+0000 7fbf8ed9d640 1 -- 172.21.15.83:0/2078886721 >> v2:172.21.15.167:3300/0 conn(0x7fbf9015ba30 msgr2=0x7fbf90072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:22.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.904+0000 7fbf8ed9d640 1 --2- 172.21.15.83:0/2078886721 >> v2:172.21.15.167:3300/0 conn(0x7fbf9015ba30 0x7fbf90072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.904+0000 7fbf8ed9d640 1 -- 172.21.15.83:0/2078886721 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9006e040 con 0x7fbf900cfa10 2024-04-17T10:38:22.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.904+0000 7fbf8dd9b640 1 --2- 172.21.15.83:0/2078886721 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf9015c960 0x7fbf900cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:22.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.904+0000 7fbf8ed9d640 1 --2- 172.21.15.83:0/2078886721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf900cfa10 0x7fbf9006c490 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbf900b2410 tx=0x7fbf8801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9addb7ce395d8e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:22.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.904+0000 7fbf8d59a640 1 -- 172.21.15.83:0/2078886721 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf880a56d0 con 0x7fbf900cfa10 2024-04-17T10:38:22.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.904+0000 7fbf8d59a640 1 -- 172.21.15.83:0/2078886721 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf880a5870 con 0x7fbf900cfa10 2024-04-17T10:38:22.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.904+0000 7fbf8d59a640 1 -- 172.21.15.83:0/2078886721 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf880a5c40 con 0x7fbf900cfa10 2024-04-17T10:38:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf94df7640 1 -- 172.21.15.83:0/2078886721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf900cfa10 msgr2=0x7fbf9006c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf94df7640 1 --2- 172.21.15.83:0/2078886721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf900cfa10 0x7fbf9006c490 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbf900b2410 tx=0x7fbf8801c960 comp rx=0 tx=0).stop 2024-04-17T10:38:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf94df7640 1 -- 172.21.15.83:0/2078886721 shutdown_connections 2024-04-17T10:38:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf94df7640 1 --2- 172.21.15.83:0/2078886721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf900cfa10 0x7fbf9006c490 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf94df7640 1 --2- 172.21.15.83:0/2078886721 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf9015c960 0x7fbf900cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf94df7640 1 --2- 172.21.15.83:0/2078886721 >> v2:172.21.15.167:3300/0 conn(0x7fbf9015ba30 0x7fbf90072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf94df7640 1 -- 172.21.15.83:0/2078886721 >> 172.21.15.83:0/2078886721 conn(0x7fbf9006abe0 msgr2=0x7fbf9006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:22.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf94df7640 1 -- 172.21.15.83:0/2078886721 shutdown_connections 2024-04-17T10:38:22.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf94df7640 1 -- 172.21.15.83:0/2078886721 wait complete. 2024-04-17T10:38:22.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf94df7640 1 Processor -- start 2024-04-17T10:38:22.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf94df7640 1 -- start start 2024-04-17T10:38:22.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf94df7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf900cfa10 0x7fbf90151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf94df7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf9015ba30 0x7fbf900ce540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf94df7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf9015c960 0x7fbf900adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf94df7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9006de20 con 0x7fbf900cfa10 2024-04-17T10:38:22.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf94df7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9006dca0 con 0x7fbf9015c960 2024-04-17T10:38:22.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf94df7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9015cd40 con 0x7fbf9015ba30 2024-04-17T10:38:22.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf8e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf900cfa10 0x7fbf90151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf8e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf900cfa10 0x7fbf90151170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48188/0 (socket says 172.21.15.83:48188) 2024-04-17T10:38:22.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf8e59c640 1 -- 172.21.15.83:0/4106720225 learned_addr learned my addr 172.21.15.83:0/4106720225 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:22.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf8e59c640 1 -- 172.21.15.83:0/4106720225 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf9015ba30 msgr2=0x7fbf900ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf8ed9d640 1 --2- 172.21.15.83:0/4106720225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf9015c960 0x7fbf900adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf8dd9b640 1 --2- 172.21.15.83:0/4106720225 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf9015ba30 0x7fbf900ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf8e59c640 1 --2- 172.21.15.83:0/4106720225 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf9015ba30 0x7fbf900ce540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf8e59c640 1 -- 172.21.15.83:0/4106720225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf9015c960 msgr2=0x7fbf900adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:22.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf8e59c640 1 --2- 172.21.15.83:0/4106720225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf9015c960 0x7fbf900adaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:22.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf8e59c640 1 -- 172.21.15.83:0/4106720225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf90151800 con 0x7fbf900cfa10 2024-04-17T10:38:22.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf8e59c640 1 --2- 172.21.15.83:0/4106720225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf900cfa10 0x7fbf90151170 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbf8000d800 tx=0x7fbf8000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:22.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf7effd640 1 -- 172.21.15.83:0/4106720225 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf80018030 con 0x7fbf900cfa10 2024-04-17T10:38:22.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf7effd640 1 -- 172.21.15.83:0/4106720225 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf800096a0 con 0x7fbf900cfa10 2024-04-17T10:38:22.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf7effd640 1 -- 172.21.15.83:0/4106720225 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf80009870 con 0x7fbf900cfa10 2024-04-17T10:38:22.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf94df7640 1 -- 172.21.15.83:0/4106720225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf90151ae0 con 0x7fbf900cfa10 2024-04-17T10:38:22.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.908+0000 7fbf94df7640 1 -- 172.21.15.83:0/4106720225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf900b2c00 con 0x7fbf900cfa10 2024-04-17T10:38:22.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.912+0000 7fbf94df7640 1 -- 172.21.15.83:0/4106720225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf90152000 con 0x7fbf900cfa10 2024-04-17T10:38:22.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.912+0000 7fbf7effd640 1 -- 172.21.15.83:0/4106720225 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbf8001b070 con 0x7fbf900cfa10 2024-04-17T10:38:22.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.912+0000 7fbf7effd640 1 --2- 172.21.15.83:0/4106720225 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbf70066f60 0x7fbf70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:22.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.912+0000 7fbf7effd640 1 -- 172.21.15.83:0/4106720225 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf80085310 con 0x7fbf900cfa10 2024-04-17T10:38:22.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.912+0000 7fbf8dd9b640 1 --2- 172.21.15.83:0/4106720225 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbf70066f60 0x7fbf70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:22.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.912+0000 7fbf8dd9b640 1 --2- 172.21.15.83:0/4106720225 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbf70066f60 0x7fbf70069420 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbf84006fd0 tx=0x7fbf84008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:22.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:22.916+0000 7fbf7effd640 1 -- 172.21.15.83:0/4106720225 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbf90152000 con 0x7fbf900cfa10 2024-04-17T10:38:23.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.056+0000 7fbf94df7640 1 -- 172.21.15.83:0/4106720225 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbf90074f90 con 0x7fbf70066f60 2024-04-17T10:38:23.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.064+0000 7fbf7effd640 1 -- 172.21.15.83:0/4106720225 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251836 (secure 0 0 0) 0x7fbf90074f90 con 0x7fbf70066f60 2024-04-17T10:38:23.064 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:23.066 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:38:23.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.068+0000 7fbf94df7640 1 -- 172.21.15.83:0/4106720225 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbf70066f60 msgr2=0x7fbf70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:23.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.068+0000 7fbf94df7640 1 --2- 172.21.15.83:0/4106720225 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbf70066f60 0x7fbf70069420 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbf84006fd0 tx=0x7fbf84008040 comp rx=0 tx=0).stop 2024-04-17T10:38:23.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.068+0000 7fbf94df7640 1 -- 172.21.15.83:0/4106720225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf900cfa10 msgr2=0x7fbf90151170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:23.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.068+0000 7fbf94df7640 1 --2- 172.21.15.83:0/4106720225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf900cfa10 0x7fbf90151170 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbf8000d800 tx=0x7fbf8000dcc0 comp rx=0 tx=0).stop 2024-04-17T10:38:23.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.068+0000 7fbf94df7640 1 -- 172.21.15.83:0/4106720225 shutdown_connections 2024-04-17T10:38:23.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.068+0000 7fbf94df7640 1 --2- 172.21.15.83:0/4106720225 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbf70066f60 0x7fbf70069420 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:23.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.068+0000 7fbf94df7640 1 --2- 172.21.15.83:0/4106720225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf9015c960 0x7fbf900adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:23.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.068+0000 7fbf94df7640 1 --2- 172.21.15.83:0/4106720225 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf9015ba30 0x7fbf900ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:23.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.068+0000 7fbf94df7640 1 --2- 172.21.15.83:0/4106720225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf900cfa10 0x7fbf90151170 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:23.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.068+0000 7fbf94df7640 1 -- 172.21.15.83:0/4106720225 >> 172.21.15.83:0/4106720225 conn(0x7fbf9006abe0 msgr2=0x7fbf9015e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:23.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.068+0000 7fbf94df7640 1 -- 172.21.15.83:0/4106720225 shutdown_connections 2024-04-17T10:38:23.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.068+0000 7fbf94df7640 1 -- 172.21.15.83:0/4106720225 wait complete. 2024-04-17T10:38:23.081 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-04-17T10:38:22.182830+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219004,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755716,"statfs":{"total":773094113280,"available":772869853184,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.486193"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702753+0000","last_change":"2024-04-17T10:38:12.718400+0000","last_active":"2024-04-17T10:38:13.702753+0000","last_peered":"2024-04-17T10:38:13.702753+0000","last_clean":"2024-04-17T10:38:13.702753+0000","last_became_active":"2024-04-17T10:38:12.718121+0000","last_became_peered":"2024-04-17T10:38:12.718121+0000","last_unstale":"2024-04-17T10:38:13.702753+0000","last_undegraded":"2024-04-17T10:38:13.702753+0000","last_fullsized":"2024-04-17T10:38:13.702753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:37.203406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695895+0000","last_change":"2024-04-17T10:38:12.710479+0000","last_active":"2024-04-17T10:38:13.695895+0000","last_peered":"2024-04-17T10:38:13.695895+0000","last_clean":"2024-04-17T10:38:13.695895+0000","last_became_active":"2024-04-17T10:38:12.710092+0000","last_became_peered":"2024-04-17T10:38:12.710092+0000","last_unstale":"2024-04-17T10:38:13.695895+0000","last_undegraded":"2024-04-17T10:38:13.695895+0000","last_fullsized":"2024-04-17T10:38:13.695895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:43.917742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305973+0000","last_change":"2024-04-17T10:38:12.722110+0000","last_active":"2024-04-17T10:38:14.305973+0000","last_peered":"2024-04-17T10:38:14.305973+0000","last_clean":"2024-04-17T10:38:14.305973+0000","last_became_active":"2024-04-17T10:38:12.721720+0000","last_became_peered":"2024-04-17T10:38:12.721720+0000","last_unstale":"2024-04-17T10:38:14.305973+0000","last_undegraded":"2024-04-17T10:38:14.305973+0000","last_fullsized":"2024-04-17T10:38:14.305973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:43.917742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970398+0000","last_change":"2024-04-17T10:38:12.727777+0000","last_active":"2024-04-17T10:38:13.970398+0000","last_peered":"2024-04-17T10:38:13.970398+0000","last_clean":"2024-04-17T10:38:13.970398+0000","last_became_active":"2024-04-17T10:38:12.726952+0000","last_became_peered":"2024-04-17T10:38:12.726952+0000","last_unstale":"2024-04-17T10:38:13.970398+0000","last_undegraded":"2024-04-17T10:38:13.970398+0000","last_fullsized":"2024-04-17T10:38:13.970398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:06.508830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701762+0000","last_change":"2024-04-17T10:38:12.721627+0000","last_active":"2024-04-17T10:38:13.701762+0000","last_peered":"2024-04-17T10:38:13.701762+0000","last_clean":"2024-04-17T10:38:13.701762+0000","last_became_active":"2024-04-17T10:38:12.718951+0000","last_became_peered":"2024-04-17T10:38:12.718951+0000","last_unstale":"2024-04-17T10:38:13.701762+0000","last_undegraded":"2024-04-17T10:38:13.701762+0000","last_fullsized":"2024-04-17T10:38:13.701762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:02.184227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970163+0000","last_change":"2024-04-17T10:38:12.729603+0000","last_active":"2024-04-17T10:38:13.970163+0000","last_peered":"2024-04-17T10:38:13.970163+0000","last_clean":"2024-04-17T10:38:13.970163+0000","last_became_active":"2024-04-17T10:38:12.728738+0000","last_became_peered":"2024-04-17T10:38:12.728738+0000","last_unstale":"2024-04-17T10:38:13.970163+0000","last_undegraded":"2024-04-17T10:38:13.970163+0000","last_fullsized":"2024-04-17T10:38:13.970163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:54:26.054113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701635+0000","last_change":"2024-04-17T10:38:12.708732+0000","last_active":"2024-04-17T10:38:13.701635+0000","last_peered":"2024-04-17T10:38:13.701635+0000","last_clean":"2024-04-17T10:38:13.701635+0000","last_became_active":"2024-04-17T10:38:12.708291+0000","last_became_peered":"2024-04-17T10:38:12.708291+0000","last_unstale":"2024-04-17T10:38:13.701635+0000","last_undegraded":"2024-04-17T10:38:13.701635+0000","last_fullsized":"2024-04-17T10:38:13.701635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:54:26.054113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297197+0000","last_change":"2024-04-17T10:38:12.726672+0000","last_active":"2024-04-17T10:38:14.297197+0000","last_peered":"2024-04-17T10:38:14.297197+0000","last_clean":"2024-04-17T10:38:14.297197+0000","last_became_active":"2024-04-17T10:38:12.726395+0000","last_became_peered":"2024-04-17T10:38:12.726395+0000","last_unstale":"2024-04-17T10:38:14.297197+0000","last_undegraded":"2024-04-17T10:38:14.297197+0000","last_fullsized":"2024-04-17T10:38:14.297197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:31:48.663733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702702+0000","last_change":"2024-04-17T10:38:12.727392+0000","last_active":"2024-04-17T10:38:13.702702+0000","last_peered":"2024-04-17T10:38:13.702702+0000","last_clean":"2024-04-17T10:38:13.702702+0000","last_became_active":"2024-04-17T10:38:12.727171+0000","last_became_peered":"2024-04-17T10:38:12.727171+0000","last_unstale":"2024-04-17T10:38:13.702702+0000","last_undegraded":"2024-04-17T10:38:13.702702+0000","last_fullsized":"2024-04-17T10:38:13.702702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:11:49.847070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803824+0000","last_change":"2024-04-17T10:38:12.726158+0000","last_active":"2024-04-17T10:38:13.803824+0000","last_peered":"2024-04-17T10:38:13.803824+0000","last_clean":"2024-04-17T10:38:13.803824+0000","last_became_active":"2024-04-17T10:38:12.724920+0000","last_became_peered":"2024-04-17T10:38:12.724920+0000","last_unstale":"2024-04-17T10:38:13.803824+0000","last_undegraded":"2024-04-17T10:38:13.803824+0000","last_fullsized":"2024-04-17T10:38:13.803824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:43.917742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803453+0000","last_change":"2024-04-17T10:38:12.725420+0000","last_active":"2024-04-17T10:38:13.803453+0000","last_peered":"2024-04-17T10:38:13.803453+0000","last_clean":"2024-04-17T10:38:13.803453+0000","last_became_active":"2024-04-17T10:38:12.724959+0000","last_became_peered":"2024-04-17T10:38:12.724959+0000","last_unstale":"2024-04-17T10:38:13.803453+0000","last_undegraded":"2024-04-17T10:38:13.803453+0000","last_fullsized":"2024-04-17T10:38:13.803453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:39.640730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.699955+0000","last_change":"2024-04-17T10:38:12.712887+0000","last_active":"2024-04-17T10:38:13.699955+0000","last_peered":"2024-04-17T10:38:13.699955+0000","last_clean":"2024-04-17T10:38:13.699955+0000","last_became_active":"2024-04-17T10:38:12.712582+0000","last_became_peered":"2024-04-17T10:38:12.712582+0000","last_unstale":"2024-04-17T10:38:13.699955+0000","last_undegraded":"2024-04-17T10:38:13.699955+0000","last_fullsized":"2024-04-17T10:38:13.699955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:23:30.964531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298210+0000","last_change":"2024-04-17T10:38:12.722191+0000","last_active":"2024-04-17T10:38:14.298210+0000","last_peered":"2024-04-17T10:38:14.298210+0000","last_clean":"2024-04-17T10:38:14.298210+0000","last_became_active":"2024-04-17T10:38:12.721912+0000","last_became_peered":"2024-04-17T10:38:12.721912+0000","last_unstale":"2024-04-17T10:38:14.298210+0000","last_undegraded":"2024-04-17T10:38:14.298210+0000","last_fullsized":"2024-04-17T10:38:14.298210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:09:07.090758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.304662+0000","last_change":"2024-04-17T10:38:12.710211+0000","last_active":"2024-04-17T10:38:14.304662+0000","last_peered":"2024-04-17T10:38:14.304662+0000","last_clean":"2024-04-17T10:38:14.304662+0000","last_became_active":"2024-04-17T10:38:12.709818+0000","last_became_peered":"2024-04-17T10:38:12.709818+0000","last_unstale":"2024-04-17T10:38:14.304662+0000","last_undegraded":"2024-04-17T10:38:14.304662+0000","last_fullsized":"2024-04-17T10:38:14.304662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:23:26.747986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695484+0000","last_change":"2024-04-17T10:38:12.722762+0000","last_active":"2024-04-17T10:38:13.695484+0000","last_peered":"2024-04-17T10:38:13.695484+0000","last_clean":"2024-04-17T10:38:13.695484+0000","last_became_active":"2024-04-17T10:38:12.718394+0000","last_became_peered":"2024-04-17T10:38:12.718394+0000","last_unstale":"2024-04-17T10:38:13.695484+0000","last_undegraded":"2024-04-17T10:38:13.695484+0000","last_fullsized":"2024-04-17T10:38:13.695484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:08.872431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.969994+0000","last_change":"2024-04-17T10:38:12.729270+0000","last_active":"2024-04-17T10:38:13.969994+0000","last_peered":"2024-04-17T10:38:13.969994+0000","last_clean":"2024-04-17T10:38:13.969994+0000","last_became_active":"2024-04-17T10:38:12.728798+0000","last_became_peered":"2024-04-17T10:38:12.728798+0000","last_unstale":"2024-04-17T10:38:13.969994+0000","last_undegraded":"2024-04-17T10:38:13.969994+0000","last_fullsized":"2024-04-17T10:38:13.969994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:23.465112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.971229+0000","last_change":"2024-04-17T10:38:12.731612+0000","last_active":"2024-04-17T10:38:13.971229+0000","last_peered":"2024-04-17T10:38:13.971229+0000","last_clean":"2024-04-17T10:38:13.971229+0000","last_became_active":"2024-04-17T10:38:12.731071+0000","last_became_peered":"2024-04-17T10:38:12.731071+0000","last_unstale":"2024-04-17T10:38:13.971229+0000","last_undegraded":"2024-04-17T10:38:13.971229+0000","last_fullsized":"2024-04-17T10:38:13.971229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:37.203406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.303651+0000","last_change":"2024-04-17T10:38:12.720913+0000","last_active":"2024-04-17T10:38:14.303651+0000","last_peered":"2024-04-17T10:38:14.303651+0000","last_clean":"2024-04-17T10:38:14.303651+0000","last_became_active":"2024-04-17T10:38:12.719779+0000","last_became_peered":"2024-04-17T10:38:12.719779+0000","last_unstale":"2024-04-17T10:38:14.303651+0000","last_undegraded":"2024-04-17T10:38:14.303651+0000","last_fullsized":"2024-04-17T10:38:14.303651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:23:26.747986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702068+0000","last_change":"2024-04-17T10:38:12.719229+0000","last_active":"2024-04-17T10:38:13.702068+0000","last_peered":"2024-04-17T10:38:13.702068+0000","last_clean":"2024-04-17T10:38:13.702068+0000","last_became_active":"2024-04-17T10:38:12.718934+0000","last_became_peered":"2024-04-17T10:38:12.718934+0000","last_unstale":"2024-04-17T10:38:13.702068+0000","last_undegraded":"2024-04-17T10:38:13.702068+0000","last_fullsized":"2024-04-17T10:38:13.702068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:38:39.116589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803564+0000","last_change":"2024-04-17T10:38:12.717999+0000","last_active":"2024-04-17T10:38:13.803564+0000","last_peered":"2024-04-17T10:38:13.803564+0000","last_clean":"2024-04-17T10:38:13.803564+0000","last_became_active":"2024-04-17T10:38:12.716817+0000","last_became_peered":"2024-04-17T10:38:12.716817+0000","last_unstale":"2024-04-17T10:38:13.803564+0000","last_undegraded":"2024-04-17T10:38:13.803564+0000","last_fullsized":"2024-04-17T10:38:13.803564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:42.256740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.304981+0000","last_change":"2024-04-17T10:38:12.710385+0000","last_active":"2024-04-17T10:38:14.304981+0000","last_peered":"2024-04-17T10:38:14.304981+0000","last_clean":"2024-04-17T10:38:14.304981+0000","last_became_active":"2024-04-17T10:38:12.707950+0000","last_became_peered":"2024-04-17T10:38:12.707950+0000","last_unstale":"2024-04-17T10:38:14.304981+0000","last_undegraded":"2024-04-17T10:38:14.304981+0000","last_fullsized":"2024-04-17T10:38:14.304981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:44:43.972627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970232+0000","last_change":"2024-04-17T10:38:12.731401+0000","last_active":"2024-04-17T10:38:13.970232+0000","last_peered":"2024-04-17T10:38:13.970232+0000","last_clean":"2024-04-17T10:38:13.970232+0000","last_became_active":"2024-04-17T10:38:12.729441+0000","last_became_peered":"2024-04-17T10:38:12.729441+0000","last_unstale":"2024-04-17T10:38:13.970232+0000","last_undegraded":"2024-04-17T10:38:13.970232+0000","last_fullsized":"2024-04-17T10:38:13.970232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:02.184227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297301+0000","last_change":"2024-04-17T10:38:12.725791+0000","last_active":"2024-04-17T10:38:14.297301+0000","last_peered":"2024-04-17T10:38:14.297301+0000","last_clean":"2024-04-17T10:38:14.297301+0000","last_became_active":"2024-04-17T10:38:12.723417+0000","last_became_peered":"2024-04-17T10:38:12.723417+0000","last_unstale":"2024-04-17T10:38:14.297301+0000","last_undegraded":"2024-04-17T10:38:14.297301+0000","last_fullsized":"2024-04-17T10:38:14.297301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:26:02.126931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970408+0000","last_change":"2024-04-17T10:38:12.727592+0000","last_active":"2024-04-17T10:38:13.970408+0000","last_peered":"2024-04-17T10:38:13.970408+0000","last_clean":"2024-04-17T10:38:13.970408+0000","last_became_active":"2024-04-17T10:38:12.726482+0000","last_became_peered":"2024-04-17T10:38:12.726482+0000","last_unstale":"2024-04-17T10:38:13.970408+0000","last_undegraded":"2024-04-17T10:38:13.970408+0000","last_fullsized":"2024-04-17T10:38:13.970408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:09:07.090758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297926+0000","last_change":"2024-04-17T10:38:12.716910+0000","last_active":"2024-04-17T10:38:14.297926+0000","last_peered":"2024-04-17T10:38:14.297926+0000","last_clean":"2024-04-17T10:38:14.297926+0000","last_became_active":"2024-04-17T10:38:12.716527+0000","last_became_peered":"2024-04-17T10:38:12.716527+0000","last_unstale":"2024-04-17T10:38:14.297926+0000","last_undegraded":"2024-04-17T10:38:14.297926+0000","last_fullsized":"2024-04-17T10:38:14.297926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:43.917742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803906+0000","last_change":"2024-04-17T10:38:12.728066+0000","last_active":"2024-04-17T10:38:13.803906+0000","last_peered":"2024-04-17T10:38:13.803906+0000","last_clean":"2024-04-17T10:38:13.803906+0000","last_became_active":"2024-04-17T10:38:12.727694+0000","last_became_peered":"2024-04-17T10:38:12.727694+0000","last_unstale":"2024-04-17T10:38:13.803906+0000","last_undegraded":"2024-04-17T10:38:13.803906+0000","last_fullsized":"2024-04-17T10:38:13.803906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:05:21.619291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695757+0000","last_change":"2024-04-17T10:38:12.714887+0000","last_active":"2024-04-17T10:38:13.695757+0000","last_peered":"2024-04-17T10:38:13.695757+0000","last_clean":"2024-04-17T10:38:13.695757+0000","last_became_active":"2024-04-17T10:38:12.714599+0000","last_became_peered":"2024-04-17T10:38:12.714599+0000","last_unstale":"2024-04-17T10:38:13.695757+0000","last_undegraded":"2024-04-17T10:38:13.695757+0000","last_fullsized":"2024-04-17T10:38:13.695757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:42.256740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297989+0000","last_change":"2024-04-17T10:38:12.725678+0000","last_active":"2024-04-17T10:38:14.297989+0000","last_peered":"2024-04-17T10:38:14.297989+0000","last_clean":"2024-04-17T10:38:14.297989+0000","last_became_active":"2024-04-17T10:38:12.723311+0000","last_became_peered":"2024-04-17T10:38:12.723311+0000","last_unstale":"2024-04-17T10:38:14.297989+0000","last_undegraded":"2024-04-17T10:38:14.297989+0000","last_fullsized":"2024-04-17T10:38:14.297989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:05:21.619291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305173+0000","last_change":"2024-04-17T10:38:12.720525+0000","last_active":"2024-04-17T10:38:14.305173+0000","last_peered":"2024-04-17T10:38:14.305173+0000","last_clean":"2024-04-17T10:38:14.305173+0000","last_became_active":"2024-04-17T10:38:12.720134+0000","last_became_peered":"2024-04-17T10:38:12.720134+0000","last_unstale":"2024-04-17T10:38:14.305173+0000","last_undegraded":"2024-04-17T10:38:14.305173+0000","last_fullsized":"2024-04-17T10:38:14.305173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:08.872431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970581+0000","last_change":"2024-04-17T10:38:12.727969+0000","last_active":"2024-04-17T10:38:13.970581+0000","last_peered":"2024-04-17T10:38:13.970581+0000","last_clean":"2024-04-17T10:38:13.970581+0000","last_became_active":"2024-04-17T10:38:12.726330+0000","last_became_peered":"2024-04-17T10:38:12.726330+0000","last_unstale":"2024-04-17T10:38:13.970581+0000","last_undegraded":"2024-04-17T10:38:13.970581+0000","last_fullsized":"2024-04-17T10:38:13.970581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:33:29.583300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.303139+0000","last_change":"2024-04-17T10:38:12.732783+0000","last_active":"2024-04-17T10:38:14.303139+0000","last_peered":"2024-04-17T10:38:14.303139+0000","last_clean":"2024-04-17T10:38:14.303139+0000","last_became_active":"2024-04-17T10:38:12.732525+0000","last_became_peered":"2024-04-17T10:38:12.732525+0000","last_unstale":"2024-04-17T10:38:14.303139+0000","last_undegraded":"2024-04-17T10:38:14.303139+0000","last_fullsized":"2024-04-17T10:38:14.303139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:21:11.071817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803187+0000","last_change":"2024-04-17T10:38:12.720029+0000","last_active":"2024-04-17T10:38:13.803187+0000","last_peered":"2024-04-17T10:38:13.803187+0000","last_clean":"2024-04-17T10:38:13.803187+0000","last_became_active":"2024-04-17T10:38:12.719579+0000","last_became_peered":"2024-04-17T10:38:12.719579+0000","last_unstale":"2024-04-17T10:38:13.803187+0000","last_undegraded":"2024-04-17T10:38:13.803187+0000","last_fullsized":"2024-04-17T10:38:13.803187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:31:48.663733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.699682+0000","last_change":"2024-04-17T10:38:12.709872+0000","last_active":"2024-04-17T10:38:13.699682+0000","last_peered":"2024-04-17T10:38:13.699682+0000","last_clean":"2024-04-17T10:38:13.699682+0000","last_became_active":"2024-04-17T10:38:12.709614+0000","last_became_peered":"2024-04-17T10:38:12.709614+0000","last_unstale":"2024-04-17T10:38:13.699682+0000","last_undegraded":"2024-04-17T10:38:13.699682+0000","last_fullsized":"2024-04-17T10:38:13.699682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:46.375063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298173+0000","last_change":"2024-04-17T10:38:11.718367+0000","last_active":"2024-04-17T10:38:14.298173+0000","last_peered":"2024-04-17T10:38:14.298173+0000","last_clean":"2024-04-17T10:38:14.298173+0000","last_became_active":"2024-04-17T10:38:11.718004+0000","last_became_peered":"2024-04-17T10:38:11.718004+0000","last_unstale":"2024-04-17T10:38:14.298173+0000","last_undegraded":"2024-04-17T10:38:14.298173+0000","last_fullsized":"2024-04-17T10:38:14.298173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:39.210602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305196+0000","last_change":"2024-04-17T10:38:12.710623+0000","last_active":"2024-04-17T10:38:14.305196+0000","last_peered":"2024-04-17T10:38:14.305196+0000","last_clean":"2024-04-17T10:38:14.305196+0000","last_became_active":"2024-04-17T10:38:12.708883+0000","last_became_peered":"2024-04-17T10:38:12.708883+0000","last_unstale":"2024-04-17T10:38:14.305196+0000","last_undegraded":"2024-04-17T10:38:14.305196+0000","last_fullsized":"2024-04-17T10:38:14.305196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:26:02.126931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803042+0000","last_change":"2024-04-17T10:38:11.718224+0000","last_active":"2024-04-17T10:38:13.803042+0000","last_peered":"2024-04-17T10:38:13.803042+0000","last_clean":"2024-04-17T10:38:13.803042+0000","last_became_active":"2024-04-17T10:38:11.717433+0000","last_became_peered":"2024-04-17T10:38:11.717433+0000","last_unstale":"2024-04-17T10:38:13.803042+0000","last_undegraded":"2024-04-17T10:38:13.803042+0000","last_fullsized":"2024-04-17T10:38:13.803042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:39.210602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298350+0000","last_change":"2024-04-17T10:38:11.719972+0000","last_active":"2024-04-17T10:38:14.298350+0000","last_peered":"2024-04-17T10:38:14.298350+0000","last_clean":"2024-04-17T10:38:14.298350+0000","last_became_active":"2024-04-17T10:38:11.719469+0000","last_became_peered":"2024-04-17T10:38:11.719469+0000","last_unstale":"2024-04-17T10:38:14.298350+0000","last_undegraded":"2024-04-17T10:38:14.298350+0000","last_fullsized":"2024-04-17T10:38:14.298350+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:59:51.693435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297286+0000","last_change":"2024-04-17T10:38:12.722655+0000","last_active":"2024-04-17T10:38:14.297286+0000","last_peered":"2024-04-17T10:38:14.297286+0000","last_clean":"2024-04-17T10:38:14.297286+0000","last_became_active":"2024-04-17T10:38:12.722213+0000","last_became_peered":"2024-04-17T10:38:12.722213+0000","last_unstale":"2024-04-17T10:38:14.297286+0000","last_undegraded":"2024-04-17T10:38:14.297286+0000","last_fullsized":"2024-04-17T10:38:14.297286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:08.872431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970337+0000","last_change":"2024-04-17T10:38:11.704898+0000","last_active":"2024-04-17T10:38:13.970337+0000","last_peered":"2024-04-17T10:38:13.970337+0000","last_clean":"2024-04-17T10:38:13.970337+0000","last_became_active":"2024-04-17T10:38:11.704184+0000","last_became_peered":"2024-04-17T10:38:11.704184+0000","last_unstale":"2024-04-17T10:38:13.970337+0000","last_undegraded":"2024-04-17T10:38:13.970337+0000","last_fullsized":"2024-04-17T10:38:13.970337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:45.366572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695486+0000","last_change":"2024-04-17T10:38:12.714844+0000","last_active":"2024-04-17T10:38:13.695486+0000","last_peered":"2024-04-17T10:38:13.695486+0000","last_clean":"2024-04-17T10:38:13.695486+0000","last_became_active":"2024-04-17T10:38:12.714565+0000","last_became_peered":"2024-04-17T10:38:12.714565+0000","last_unstale":"2024-04-17T10:38:13.695486+0000","last_undegraded":"2024-04-17T10:38:13.695486+0000","last_fullsized":"2024-04-17T10:38:13.695486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:26:02.126931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803577+0000","last_change":"2024-04-17T10:38:11.725276+0000","last_active":"2024-04-17T10:38:13.803577+0000","last_peered":"2024-04-17T10:38:13.803577+0000","last_clean":"2024-04-17T10:38:13.803577+0000","last_became_active":"2024-04-17T10:38:11.724660+0000","last_became_peered":"2024-04-17T10:38:11.724660+0000","last_unstale":"2024-04-17T10:38:13.803577+0000","last_undegraded":"2024-04-17T10:38:13.803577+0000","last_fullsized":"2024-04-17T10:38:13.803577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:21:00.724523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970977+0000","last_change":"2024-04-17T10:38:12.731384+0000","last_active":"2024-04-17T10:38:13.970977+0000","last_peered":"2024-04-17T10:38:13.970977+0000","last_clean":"2024-04-17T10:38:13.970977+0000","last_became_active":"2024-04-17T10:38:12.730178+0000","last_became_peered":"2024-04-17T10:38:12.730178+0000","last_unstale":"2024-04-17T10:38:13.970977+0000","last_undegraded":"2024-04-17T10:38:13.970977+0000","last_fullsized":"2024-04-17T10:38:13.970977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:11:49.847070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970956+0000","last_change":"2024-04-17T10:38:11.727591+0000","last_active":"2024-04-17T10:38:13.970956+0000","last_peered":"2024-04-17T10:38:13.970956+0000","last_clean":"2024-04-17T10:38:13.970956+0000","last_became_active":"2024-04-17T10:38:11.726319+0000","last_became_peered":"2024-04-17T10:38:11.726319+0000","last_unstale":"2024-04-17T10:38:13.970956+0000","last_undegraded":"2024-04-17T10:38:13.970956+0000","last_fullsized":"2024-04-17T10:38:13.970956+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:50:59.373356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305290+0000","last_change":"2024-04-17T10:38:12.727239+0000","last_active":"2024-04-17T10:38:14.305290+0000","last_peered":"2024-04-17T10:38:14.305290+0000","last_clean":"2024-04-17T10:38:14.305290+0000","last_became_active":"2024-04-17T10:38:12.726192+0000","last_became_peered":"2024-04-17T10:38:12.726192+0000","last_unstale":"2024-04-17T10:38:14.305290+0000","last_undegraded":"2024-04-17T10:38:14.305290+0000","last_fullsized":"2024-04-17T10:38:14.305290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:39.640730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695810+0000","last_change":"2024-04-17T10:38:11.710534+0000","last_active":"2024-04-17T10:38:13.695810+0000","last_peered":"2024-04-17T10:38:13.695810+0000","last_clean":"2024-04-17T10:38:13.695810+0000","last_became_active":"2024-04-17T10:38:11.709944+0000","last_became_peered":"2024-04-17T10:38:11.709944+0000","last_unstale":"2024-04-17T10:38:13.695810+0000","last_undegraded":"2024-04-17T10:38:13.695810+0000","last_fullsized":"2024-04-17T10:38:13.695810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:45.366572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.700329+0000","last_change":"2024-04-17T10:38:12.709577+0000","last_active":"2024-04-17T10:38:13.700329+0000","last_peered":"2024-04-17T10:38:13.700329+0000","last_clean":"2024-04-17T10:38:13.700329+0000","last_became_active":"2024-04-17T10:38:12.709186+0000","last_became_peered":"2024-04-17T10:38:12.709186+0000","last_unstale":"2024-04-17T10:38:13.700329+0000","last_undegraded":"2024-04-17T10:38:13.700329+0000","last_fullsized":"2024-04-17T10:38:13.700329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:20:45.810408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803828+0000","last_change":"2024-04-17T10:38:11.715117+0000","last_active":"2024-04-17T10:38:13.803828+0000","last_peered":"2024-04-17T10:38:13.803828+0000","last_clean":"2024-04-17T10:38:13.803828+0000","last_became_active":"2024-04-17T10:38:11.713981+0000","last_became_peered":"2024-04-17T10:38:11.713981+0000","last_unstale":"2024-04-17T10:38:13.803828+0000","last_undegraded":"2024-04-17T10:38:13.803828+0000","last_fullsized":"2024-04-17T10:38:13.803828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:27.569244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701871+0000","last_change":"2024-04-17T10:38:12.708377+0000","last_active":"2024-04-17T10:38:13.701871+0000","last_peered":"2024-04-17T10:38:13.701871+0000","last_clean":"2024-04-17T10:38:13.701871+0000","last_became_active":"2024-04-17T10:38:12.707183+0000","last_became_peered":"2024-04-17T10:38:12.707183+0000","last_unstale":"2024-04-17T10:38:13.701871+0000","last_undegraded":"2024-04-17T10:38:13.701871+0000","last_fullsized":"2024-04-17T10:38:13.701871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:06.508830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305423+0000","last_change":"2024-04-17T10:38:11.702326+0000","last_active":"2024-04-17T10:38:14.305423+0000","last_peered":"2024-04-17T10:38:14.305423+0000","last_clean":"2024-04-17T10:38:14.305423+0000","last_became_active":"2024-04-17T10:38:11.701735+0000","last_became_peered":"2024-04-17T10:38:11.701735+0000","last_unstale":"2024-04-17T10:38:14.305423+0000","last_undegraded":"2024-04-17T10:38:14.305423+0000","last_fullsized":"2024-04-17T10:38:14.305423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:11:43.267871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.304822+0000","last_change":"2024-04-17T10:38:12.705702+0000","last_active":"2024-04-17T10:38:14.304822+0000","last_peered":"2024-04-17T10:38:14.304822+0000","last_clean":"2024-04-17T10:38:14.304822+0000","last_became_active":"2024-04-17T10:38:12.705297+0000","last_became_peered":"2024-04-17T10:38:12.705297+0000","last_unstale":"2024-04-17T10:38:14.304822+0000","last_undegraded":"2024-04-17T10:38:14.304822+0000","last_fullsized":"2024-04-17T10:38:14.304822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:15:31.987120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970479+0000","last_change":"2024-04-17T10:38:11.715689+0000","last_active":"2024-04-17T10:38:13.970479+0000","last_peered":"2024-04-17T10:38:13.970479+0000","last_clean":"2024-04-17T10:38:13.970479+0000","last_became_active":"2024-04-17T10:38:11.714329+0000","last_became_peered":"2024-04-17T10:38:11.714329+0000","last_unstale":"2024-04-17T10:38:13.970479+0000","last_undegraded":"2024-04-17T10:38:13.970479+0000","last_fullsized":"2024-04-17T10:38:13.970479+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:27.569244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702011+0000","last_change":"2024-04-17T10:38:12.708536+0000","last_active":"2024-04-17T10:38:13.702011+0000","last_peered":"2024-04-17T10:38:13.702011+0000","last_clean":"2024-04-17T10:38:13.702011+0000","last_became_active":"2024-04-17T10:38:12.707643+0000","last_became_peered":"2024-04-17T10:38:12.707643+0000","last_unstale":"2024-04-17T10:38:13.702011+0000","last_undegraded":"2024-04-17T10:38:13.702011+0000","last_fullsized":"2024-04-17T10:38:13.702011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:22:10.364740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803484+0000","last_change":"2024-04-17T10:38:11.717992+0000","last_active":"2024-04-17T10:38:13.803484+0000","last_peered":"2024-04-17T10:38:13.803484+0000","last_clean":"2024-04-17T10:38:13.803484+0000","last_became_active":"2024-04-17T10:38:11.717124+0000","last_became_peered":"2024-04-17T10:38:11.717124+0000","last_unstale":"2024-04-17T10:38:13.803484+0000","last_undegraded":"2024-04-17T10:38:13.803484+0000","last_fullsized":"2024-04-17T10:38:13.803484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:24:23.818791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803260+0000","last_change":"2024-04-17T10:38:12.720282+0000","last_active":"2024-04-17T10:38:13.803260+0000","last_peered":"2024-04-17T10:38:13.803260+0000","last_clean":"2024-04-17T10:38:13.803260+0000","last_became_active":"2024-04-17T10:38:12.719197+0000","last_became_peered":"2024-04-17T10:38:12.719197+0000","last_unstale":"2024-04-17T10:38:13.803260+0000","last_undegraded":"2024-04-17T10:38:13.803260+0000","last_fullsized":"2024-04-17T10:38:13.803260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:08.872431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.303153+0000","last_change":"2024-04-17T10:38:11.689943+0000","last_active":"2024-04-17T10:38:14.303153+0000","last_peered":"2024-04-17T10:38:14.303153+0000","last_clean":"2024-04-17T10:38:14.303153+0000","last_became_active":"2024-04-17T10:38:11.689627+0000","last_became_peered":"2024-04-17T10:38:11.689627+0000","last_unstale":"2024-04-17T10:38:14.303153+0000","last_undegraded":"2024-04-17T10:38:14.303153+0000","last_fullsized":"2024-04-17T10:38:14.303153+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:39.210602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701741+0000","last_change":"2024-04-17T10:38:12.710693+0000","last_active":"2024-04-17T10:38:13.701741+0000","last_peered":"2024-04-17T10:38:13.701741+0000","last_clean":"2024-04-17T10:38:13.701741+0000","last_became_active":"2024-04-17T10:38:12.710263+0000","last_became_peered":"2024-04-17T10:38:12.710263+0000","last_unstale":"2024-04-17T10:38:13.701741+0000","last_undegraded":"2024-04-17T10:38:13.701741+0000","last_fullsized":"2024-04-17T10:38:13.701741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:09:07.090758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.700517+0000","last_change":"2024-04-17T10:38:11.701185+0000","last_active":"2024-04-17T10:38:13.700517+0000","last_peered":"2024-04-17T10:38:13.700517+0000","last_clean":"2024-04-17T10:38:13.700517+0000","last_became_active":"2024-04-17T10:38:11.700292+0000","last_became_peered":"2024-04-17T10:38:11.700292+0000","last_unstale":"2024-04-17T10:38:13.700517+0000","last_undegraded":"2024-04-17T10:38:13.700517+0000","last_fullsized":"2024-04-17T10:38:13.700517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:59:51.693435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702731+0000","last_change":"2024-04-17T10:38:12.704267+0000","last_active":"2024-04-17T10:38:13.702731+0000","last_peered":"2024-04-17T10:38:13.702731+0000","last_clean":"2024-04-17T10:38:13.702731+0000","last_became_active":"2024-04-17T10:38:12.703820+0000","last_became_peered":"2024-04-17T10:38:12.703820+0000","last_unstale":"2024-04-17T10:38:13.702731+0000","last_undegraded":"2024-04-17T10:38:13.702731+0000","last_fullsized":"2024-04-17T10:38:13.702731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:43:51.398103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.303738+0000","last_change":"2024-04-17T10:38:11.699100+0000","last_active":"2024-04-17T10:38:14.303738+0000","last_peered":"2024-04-17T10:38:14.303738+0000","last_clean":"2024-04-17T10:38:14.303738+0000","last_became_active":"2024-04-17T10:38:11.698786+0000","last_became_peered":"2024-04-17T10:38:11.698786+0000","last_unstale":"2024-04-17T10:38:14.303738+0000","last_undegraded":"2024-04-17T10:38:14.303738+0000","last_fullsized":"2024-04-17T10:38:14.303738+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:21:00.724523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702808+0000","last_change":"2024-04-17T10:38:12.705166+0000","last_active":"2024-04-17T10:38:13.702808+0000","last_peered":"2024-04-17T10:38:13.702808+0000","last_clean":"2024-04-17T10:38:13.702808+0000","last_became_active":"2024-04-17T10:38:12.704805+0000","last_became_peered":"2024-04-17T10:38:12.704805+0000","last_unstale":"2024-04-17T10:38:13.702808+0000","last_undegraded":"2024-04-17T10:38:13.702808+0000","last_fullsized":"2024-04-17T10:38:13.702808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:49:18.397560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.696028+0000","last_change":"2024-04-17T10:38:11.692209+0000","last_active":"2024-04-17T10:38:13.696028+0000","last_peered":"2024-04-17T10:38:13.696028+0000","last_clean":"2024-04-17T10:38:13.696028+0000","last_became_active":"2024-04-17T10:38:11.691905+0000","last_became_peered":"2024-04-17T10:38:11.691905+0000","last_unstale":"2024-04-17T10:38:13.696028+0000","last_undegraded":"2024-04-17T10:38:13.696028+0000","last_fullsized":"2024-04-17T10:38:13.696028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:50:59.373356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970996+0000","last_change":"2024-04-17T10:38:12.727145+0000","last_active":"2024-04-17T10:38:13.970996+0000","last_peered":"2024-04-17T10:38:13.970996+0000","last_clean":"2024-04-17T10:38:13.970996+0000","last_became_active":"2024-04-17T10:38:12.726797+0000","last_became_peered":"2024-04-17T10:38:12.726797+0000","last_unstale":"2024-04-17T10:38:13.970996+0000","last_undegraded":"2024-04-17T10:38:13.970996+0000","last_fullsized":"2024-04-17T10:38:13.970996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:43:51.398103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.303519+0000","last_change":"2024-04-17T10:38:11.705218+0000","last_active":"2024-04-17T10:38:14.303519+0000","last_peered":"2024-04-17T10:38:14.303519+0000","last_clean":"2024-04-17T10:38:14.303519+0000","last_became_active":"2024-04-17T10:38:11.704928+0000","last_became_peered":"2024-04-17T10:38:11.704928+0000","last_unstale":"2024-04-17T10:38:14.303519+0000","last_undegraded":"2024-04-17T10:38:14.303519+0000","last_fullsized":"2024-04-17T10:38:14.303519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:27.569244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701937+0000","last_change":"2024-04-17T10:38:12.722659+0000","last_active":"2024-04-17T10:38:13.701937+0000","last_peered":"2024-04-17T10:38:13.701937+0000","last_clean":"2024-04-17T10:38:13.701937+0000","last_became_active":"2024-04-17T10:38:12.722265+0000","last_became_peered":"2024-04-17T10:38:12.722265+0000","last_unstale":"2024-04-17T10:38:13.701937+0000","last_undegraded":"2024-04-17T10:38:13.701937+0000","last_fullsized":"2024-04-17T10:38:13.701937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:33:29.583300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701282+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:38:13.701282+0000","last_peered":"2024-04-17T10:38:13.701282+0000","last_clean":"2024-04-17T10:38:13.701282+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:38:13.701282+0000","last_undegraded":"2024-04-17T10:38:13.701282+0000","last_fullsized":"2024-04-17T10:38:13.701282+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970082+0000","last_change":"2024-04-17T10:38:11.702024+0000","last_active":"2024-04-17T10:38:13.970082+0000","last_peered":"2024-04-17T10:38:13.970082+0000","last_clean":"2024-04-17T10:38:13.970082+0000","last_became_active":"2024-04-17T10:38:11.701157+0000","last_became_peered":"2024-04-17T10:38:11.701157+0000","last_unstale":"2024-04-17T10:38:13.970082+0000","last_undegraded":"2024-04-17T10:38:13.970082+0000","last_fullsized":"2024-04-17T10:38:13.970082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:24:23.818791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297692+0000","last_change":"2024-04-17T10:38:12.723529+0000","last_active":"2024-04-17T10:38:14.297692+0000","last_peered":"2024-04-17T10:38:14.297692+0000","last_clean":"2024-04-17T10:38:14.297692+0000","last_became_active":"2024-04-17T10:38:12.723103+0000","last_became_peered":"2024-04-17T10:38:12.723103+0000","last_unstale":"2024-04-17T10:38:14.297692+0000","last_undegraded":"2024-04-17T10:38:14.297692+0000","last_fullsized":"2024-04-17T10:38:14.297692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:42.256740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701197+0000","last_change":"2024-04-17T10:38:11.716453+0000","last_active":"2024-04-17T10:38:13.701197+0000","last_peered":"2024-04-17T10:38:13.701197+0000","last_clean":"2024-04-17T10:38:13.701197+0000","last_became_active":"2024-04-17T10:38:11.715859+0000","last_became_peered":"2024-04-17T10:38:11.715859+0000","last_unstale":"2024-04-17T10:38:13.701197+0000","last_undegraded":"2024-04-17T10:38:13.701197+0000","last_fullsized":"2024-04-17T10:38:13.701197+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:50:59.373356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.304804+0000","last_change":"2024-04-17T10:38:12.707314+0000","last_active":"2024-04-17T10:38:14.304804+0000","last_peered":"2024-04-17T10:38:14.304804+0000","last_clean":"2024-04-17T10:38:14.304804+0000","last_became_active":"2024-04-17T10:38:12.706874+0000","last_became_peered":"2024-04-17T10:38:12.706874+0000","last_unstale":"2024-04-17T10:38:14.304804+0000","last_undegraded":"2024-04-17T10:38:14.304804+0000","last_fullsized":"2024-04-17T10:38:14.304804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:07:07.037555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.303300+0000","last_change":"2024-04-17T10:38:11.699752+0000","last_active":"2024-04-17T10:38:14.303300+0000","last_peered":"2024-04-17T10:38:14.303300+0000","last_clean":"2024-04-17T10:38:14.303300+0000","last_became_active":"2024-04-17T10:38:11.699467+0000","last_became_peered":"2024-04-17T10:38:11.699467+0000","last_unstale":"2024-04-17T10:38:14.303300+0000","last_undegraded":"2024-04-17T10:38:14.303300+0000","last_fullsized":"2024-04-17T10:38:14.303300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:24:23.818791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695234+0000","last_change":"2024-04-17T10:38:12.723809+0000","last_active":"2024-04-17T10:38:13.695234+0000","last_peered":"2024-04-17T10:38:13.695234+0000","last_clean":"2024-04-17T10:38:13.695234+0000","last_became_active":"2024-04-17T10:38:12.723282+0000","last_became_peered":"2024-04-17T10:38:12.723282+0000","last_unstale":"2024-04-17T10:38:13.695234+0000","last_undegraded":"2024-04-17T10:38:13.695234+0000","last_fullsized":"2024-04-17T10:38:13.695234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:31:48.663733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.971078+0000","last_change":"2024-04-17T10:38:11.715164+0000","last_active":"2024-04-17T10:38:13.971078+0000","last_peered":"2024-04-17T10:38:13.971078+0000","last_clean":"2024-04-17T10:38:13.971078+0000","last_became_active":"2024-04-17T10:38:11.712376+0000","last_became_peered":"2024-04-17T10:38:11.712376+0000","last_unstale":"2024-04-17T10:38:13.971078+0000","last_undegraded":"2024-04-17T10:38:13.971078+0000","last_fullsized":"2024-04-17T10:38:13.971078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:21:00.724523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.699765+0000","last_change":"2024-04-17T10:38:12.720339+0000","last_active":"2024-04-17T10:38:13.699765+0000","last_peered":"2024-04-17T10:38:13.699765+0000","last_clean":"2024-04-17T10:38:13.699765+0000","last_became_active":"2024-04-17T10:38:12.720051+0000","last_became_peered":"2024-04-17T10:38:12.720051+0000","last_unstale":"2024-04-17T10:38:13.699765+0000","last_undegraded":"2024-04-17T10:38:13.699765+0000","last_fullsized":"2024-04-17T10:38:13.699765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:51:00.381846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695713+0000","last_change":"2024-04-17T10:38:11.693095+0000","last_active":"2024-04-17T10:38:13.695713+0000","last_peered":"2024-04-17T10:38:13.695713+0000","last_clean":"2024-04-17T10:38:13.695713+0000","last_became_active":"2024-04-17T10:38:11.692723+0000","last_became_peered":"2024-04-17T10:38:11.692723+0000","last_unstale":"2024-04-17T10:38:13.695713+0000","last_undegraded":"2024-04-17T10:38:13.695713+0000","last_fullsized":"2024-04-17T10:38:13.695713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:27.569244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970920+0000","last_change":"2024-04-17T10:38:12.728559+0000","last_active":"2024-04-17T10:38:13.970920+0000","last_peered":"2024-04-17T10:38:13.970920+0000","last_clean":"2024-04-17T10:38:13.970920+0000","last_became_active":"2024-04-17T10:38:12.728285+0000","last_became_peered":"2024-04-17T10:38:12.728285+0000","last_unstale":"2024-04-17T10:38:13.970920+0000","last_undegraded":"2024-04-17T10:38:13.970920+0000","last_fullsized":"2024-04-17T10:38:13.970920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:22:10.364740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702148+0000","last_change":"2024-04-17T10:38:11.717791+0000","last_active":"2024-04-17T10:38:13.702148+0000","last_peered":"2024-04-17T10:38:13.702148+0000","last_clean":"2024-04-17T10:38:13.702148+0000","last_became_active":"2024-04-17T10:38:11.717130+0000","last_became_peered":"2024-04-17T10:38:11.717130+0000","last_unstale":"2024-04-17T10:38:13.702148+0000","last_undegraded":"2024-04-17T10:38:13.702148+0000","last_fullsized":"2024-04-17T10:38:13.702148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:21:10.063327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695581+0000","last_change":"2024-04-17T10:38:12.723436+0000","last_active":"2024-04-17T10:38:13.695581+0000","last_peered":"2024-04-17T10:38:13.695581+0000","last_clean":"2024-04-17T10:38:13.695581+0000","last_became_active":"2024-04-17T10:38:12.723138+0000","last_became_peered":"2024-04-17T10:38:12.723138+0000","last_unstale":"2024-04-17T10:38:13.695581+0000","last_undegraded":"2024-04-17T10:38:13.695581+0000","last_fullsized":"2024-04-17T10:38:13.695581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:39.640730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.365894+0000","last_change":"2024-04-17T10:38:11.730180+0000","last_active":"2024-04-17T10:38:14.365894+0000","last_peered":"2024-04-17T10:38:14.365894+0000","last_clean":"2024-04-17T10:38:14.365894+0000","last_became_active":"2024-04-17T10:38:11.729843+0000","last_became_peered":"2024-04-17T10:38:11.729843+0000","last_unstale":"2024-04-17T10:38:14.365894+0000","last_undegraded":"2024-04-17T10:38:14.365894+0000","last_fullsized":"2024-04-17T10:38:14.365894+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:23:29.956041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298370+0000","last_change":"2024-04-17T10:38:12.732643+0000","last_active":"2024-04-17T10:38:14.298370+0000","last_peered":"2024-04-17T10:38:14.298370+0000","last_clean":"2024-04-17T10:38:14.298370+0000","last_became_active":"2024-04-17T10:38:12.732259+0000","last_became_peered":"2024-04-17T10:38:12.732259+0000","last_unstale":"2024-04-17T10:38:14.298370+0000","last_undegraded":"2024-04-17T10:38:14.298370+0000","last_fullsized":"2024-04-17T10:38:14.298370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:02.184227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803741+0000","last_change":"2024-04-17T10:38:11.725444+0000","last_active":"2024-04-17T10:38:13.803741+0000","last_peered":"2024-04-17T10:38:13.803741+0000","last_clean":"2024-04-17T10:38:13.803741+0000","last_became_active":"2024-04-17T10:38:11.724780+0000","last_became_peered":"2024-04-17T10:38:11.724780+0000","last_unstale":"2024-04-17T10:38:13.803741+0000","last_undegraded":"2024-04-17T10:38:13.803741+0000","last_fullsized":"2024-04-17T10:38:13.803741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:45.366572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305401+0000","last_change":"2024-04-17T10:38:12.705770+0000","last_active":"2024-04-17T10:38:14.305401+0000","last_peered":"2024-04-17T10:38:14.305401+0000","last_clean":"2024-04-17T10:38:14.305401+0000","last_became_active":"2024-04-17T10:38:12.705400+0000","last_became_peered":"2024-04-17T10:38:12.705400+0000","last_unstale":"2024-04-17T10:38:14.305401+0000","last_undegraded":"2024-04-17T10:38:14.305401+0000","last_fullsized":"2024-04-17T10:38:14.305401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:42.256740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701197+0000","last_change":"2024-04-17T10:38:11.716433+0000","last_active":"2024-04-17T10:38:13.701197+0000","last_peered":"2024-04-17T10:38:13.701197+0000","last_clean":"2024-04-17T10:38:13.701197+0000","last_became_active":"2024-04-17T10:38:11.715803+0000","last_became_peered":"2024-04-17T10:38:11.715803+0000","last_unstale":"2024-04-17T10:38:13.701197+0000","last_undegraded":"2024-04-17T10:38:13.701197+0000","last_fullsized":"2024-04-17T10:38:13.701197+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:45.366572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.700697+0000","last_change":"2024-04-17T10:38:12.717613+0000","last_active":"2024-04-17T10:38:13.700697+0000","last_peered":"2024-04-17T10:38:13.700697+0000","last_clean":"2024-04-17T10:38:13.700697+0000","last_became_active":"2024-04-17T10:38:12.717308+0000","last_became_peered":"2024-04-17T10:38:12.717308+0000","last_unstale":"2024-04-17T10:38:13.700697+0000","last_undegraded":"2024-04-17T10:38:13.700697+0000","last_fullsized":"2024-04-17T10:38:13.700697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:21:11.071817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695770+0000","last_change":"2024-04-17T10:38:11.716778+0000","last_active":"2024-04-17T10:38:13.695770+0000","last_peered":"2024-04-17T10:38:13.695770+0000","last_clean":"2024-04-17T10:38:13.695770+0000","last_became_active":"2024-04-17T10:38:11.716526+0000","last_became_peered":"2024-04-17T10:38:11.716526+0000","last_unstale":"2024-04-17T10:38:13.695770+0000","last_undegraded":"2024-04-17T10:38:13.695770+0000","last_fullsized":"2024-04-17T10:38:13.695770+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:24:23.818791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.297622+0000","last_change":"2024-04-17T10:38:12.717941+0000","last_active":"2024-04-17T10:38:14.297622+0000","last_peered":"2024-04-17T10:38:14.297622+0000","last_clean":"2024-04-17T10:38:14.297622+0000","last_became_active":"2024-04-17T10:38:12.717350+0000","last_became_peered":"2024-04-17T10:38:12.717350+0000","last_unstale":"2024-04-17T10:38:14.297622+0000","last_undegraded":"2024-04-17T10:38:14.297622+0000","last_fullsized":"2024-04-17T10:38:14.297622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:39.640730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.695661+0000","last_change":"2024-04-17T10:38:11.699325+0000","last_active":"2024-04-17T10:38:13.695661+0000","last_peered":"2024-04-17T10:38:13.695661+0000","last_clean":"2024-04-17T10:38:13.695661+0000","last_became_active":"2024-04-17T10:38:11.698981+0000","last_became_peered":"2024-04-17T10:38:11.698981+0000","last_unstale":"2024-04-17T10:38:13.695661+0000","last_undegraded":"2024-04-17T10:38:13.695661+0000","last_fullsized":"2024-04-17T10:38:13.695661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:21:00.724523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298006+0000","last_change":"2024-04-17T10:38:12.731391+0000","last_active":"2024-04-17T10:38:14.298006+0000","last_peered":"2024-04-17T10:38:14.298006+0000","last_clean":"2024-04-17T10:38:14.298006+0000","last_became_active":"2024-04-17T10:38:12.727638+0000","last_became_peered":"2024-04-17T10:38:12.727638+0000","last_unstale":"2024-04-17T10:38:14.298006+0000","last_undegraded":"2024-04-17T10:38:14.298006+0000","last_fullsized":"2024-04-17T10:38:14.298006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:23.465112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.701909+0000","last_change":"2024-04-17T10:38:11.716353+0000","last_active":"2024-04-17T10:38:13.701909+0000","last_peered":"2024-04-17T10:38:13.701909+0000","last_clean":"2024-04-17T10:38:13.701909+0000","last_became_active":"2024-04-17T10:38:11.715722+0000","last_became_peered":"2024-04-17T10:38:11.715722+0000","last_unstale":"2024-04-17T10:38:13.701909+0000","last_undegraded":"2024-04-17T10:38:13.701909+0000","last_fullsized":"2024-04-17T10:38:13.701909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:20:44.801917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.803339+0000","last_change":"2024-04-17T10:38:12.715919+0000","last_active":"2024-04-17T10:38:13.803339+0000","last_peered":"2024-04-17T10:38:13.803339+0000","last_clean":"2024-04-17T10:38:13.803339+0000","last_became_active":"2024-04-17T10:38:12.715568+0000","last_became_peered":"2024-04-17T10:38:12.715568+0000","last_unstale":"2024-04-17T10:38:13.803339+0000","last_undegraded":"2024-04-17T10:38:13.803339+0000","last_fullsized":"2024-04-17T10:38:13.803339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:26:02.126931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.970940+0000","last_change":"2024-04-17T10:38:12.724977+0000","last_active":"2024-04-17T10:38:13.970940+0000","last_peered":"2024-04-17T10:38:13.970940+0000","last_clean":"2024-04-17T10:38:13.970940+0000","last_became_active":"2024-04-17T10:38:12.724700+0000","last_became_peered":"2024-04-17T10:38:12.724700+0000","last_unstale":"2024-04-17T10:38:13.970940+0000","last_undegraded":"2024-04-17T10:38:13.970940+0000","last_fullsized":"2024-04-17T10:38:13.970940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:38:39.116589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702197+0000","last_change":"2024-04-17T10:38:11.719799+0000","last_active":"2024-04-17T10:38:13.702197+0000","last_peered":"2024-04-17T10:38:13.702197+0000","last_clean":"2024-04-17T10:38:13.702197+0000","last_became_active":"2024-04-17T10:38:11.719147+0000","last_became_peered":"2024-04-17T10:38:11.719147+0000","last_unstale":"2024-04-17T10:38:13.702197+0000","last_undegraded":"2024-04-17T10:38:13.702197+0000","last_fullsized":"2024-04-17T10:38:13.702197+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:23:29.956041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.971331+0000","last_change":"2024-04-17T10:38:12.725123+0000","last_active":"2024-04-17T10:38:13.971331+0000","last_peered":"2024-04-17T10:38:13.971331+0000","last_clean":"2024-04-17T10:38:13.971331+0000","last_became_active":"2024-04-17T10:38:12.724760+0000","last_became_peered":"2024-04-17T10:38:12.724760+0000","last_unstale":"2024-04-17T10:38:13.971331+0000","last_undegraded":"2024-04-17T10:38:13.971331+0000","last_fullsized":"2024-04-17T10:38:13.971331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:49:18.397560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:13.702167+0000","last_change":"2024-04-17T10:38:11.727050+0000","last_active":"2024-04-17T10:38:13.702167+0000","last_peered":"2024-04-17T10:38:13.702167+0000","last_clean":"2024-04-17T10:38:13.702167+0000","last_became_active":"2024-04-17T10:38:11.726393+0000","last_became_peered":"2024-04-17T10:38:11.726393+0000","last_unstale":"2024-04-17T10:38:13.702167+0000","last_undegraded":"2024-04-17T10:38:13.702167+0000","last_fullsized":"2024-04-17T10:38:13.702167+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:23:25.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305077+0000","last_change":"2024-04-17T10:38:12.726864+0000","last_active":"2024-04-17T10:38:14.305077+0000","last_peered":"2024-04-17T10:38:14.305077+0000","last_clean":"2024-04-17T10:38:14.305077+0000","last_became_active":"2024-04-17T10:38:12.726581+0000","last_became_peered":"2024-04-17T10:38:12.726581+0000","last_unstale":"2024-04-17T10:38:14.305077+0000","last_undegraded":"2024-04-17T10:38:14.305077+0000","last_fullsized":"2024-04-17T10:38:14.305077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:31:48.663733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298342+0000","last_change":"2024-04-17T10:38:11.700825+0000","last_active":"2024-04-17T10:38:14.298342+0000","last_peered":"2024-04-17T10:38:14.298342+0000","last_clean":"2024-04-17T10:38:14.298342+0000","last_became_active":"2024-04-17T10:38:11.700354+0000","last_became_peered":"2024-04-17T10:38:11.700354+0000","last_unstale":"2024-04-17T10:38:14.298342+0000","last_undegraded":"2024-04-17T10:38:14.298342+0000","last_fullsized":"2024-04-17T10:38:14.298342+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:22:08.746889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.298080+0000","last_change":"2024-04-17T10:38:12.719771+0000","last_active":"2024-04-17T10:38:14.298080+0000","last_peered":"2024-04-17T10:38:14.298080+0000","last_clean":"2024-04-17T10:38:14.298080+0000","last_became_active":"2024-04-17T10:38:12.716013+0000","last_became_peered":"2024-04-17T10:38:12.716013+0000","last_unstale":"2024-04-17T10:38:14.298080+0000","last_undegraded":"2024-04-17T10:38:14.298080+0000","last_fullsized":"2024-04-17T10:38:14.298080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:11.668582+0000","last_clean_scrub_stamp":"2024-04-17T10:38:11.668582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:54:26.054113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T10:38:14.305389+0000","last_change":"2024-04-17T10:38:11.710717+0000","last_active":"2024-04-17T10:38:14.305389+0000","last_peered":"2024-04-17T10:38:14.305389+0000","last_clean":"2024-04-17T10:38:14.305389+0000","last_became_active":"2024-04-17T10:38:11.710342+0000","last_became_peered":"2024-04-17T10:38:11.710342+0000","last_unstale":"2024-04-17T10:38:14.305389+0000","last_undegraded":"2024-04-17T10:38:14.305389+0000","last_fullsized":"2024-04-17T10:38:14.305389+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:10.660092+0000","last_clean_scrub_stamp":"2024-04-17T10:38:10.660092+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:16:30.923882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:38:23.085 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-17T10:38:23.085 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-17T10:38:23.085 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-17T10:38:23.085 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-17T10:38:23.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.236+0000 7f540d799640 1 Processor -- start 2024-04-17T10:38:23.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.236+0000 7f540d799640 1 -- start start 2024-04-17T10:38:23.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.236+0000 7f540d799640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f540815ba30 0x7f5408072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:23.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.240+0000 7f540d799640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f540806e320 con 0x7f54080cfbe0 2024-04-17T10:38:23.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.240+0000 7f540d799640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f540806e040 con 0x7f540815c960 2024-04-17T10:38:23.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.240+0000 7f540d799640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5408071ea0 con 0x7f540815ba30 2024-04-17T10:38:23.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.240+0000 7f54077fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f54080cfbe0 0x7f540806c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39972/0 (socket says 172.21.15.83:39972) 2024-04-17T10:38:23.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.240+0000 7f54077fe640 1 -- 172.21.15.83:0/1456353491 learned_addr learned my addr 172.21.15.83:0/1456353491 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:23.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.240+0000 7f5406ffd640 1 --2- 172.21.15.83:0/1456353491 >> v2:172.21.15.167:3300/0 conn(0x7f540815ba30 0x7f5408072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:23.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.240+0000 7f54067fc640 1 -- 172.21.15.83:0/1456353491 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 226104716 0 0) 0x7f540806e320 con 0x7f54080cfbe0 2024-04-17T10:38:23.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.240+0000 7f54067fc640 1 -- 172.21.15.83:0/1456353491 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53f4003530 con 0x7f54080cfbe0 2024-04-17T10:38:23.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.240+0000 7f54067fc640 1 -- 172.21.15.83:0/1456353491 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 635539167 0 0) 0x7f53f4003530 con 0x7f54080cfbe0 2024-04-17T10:38:23.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.240+0000 7f54067fc640 1 -- 172.21.15.83:0/1456353491 >> v2:172.21.15.167:3300/0 conn(0x7f540815ba30 msgr2=0x7f5408072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:23.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.240+0000 7f54067fc640 1 --2- 172.21.15.83:0/1456353491 >> v2:172.21.15.167:3300/0 conn(0x7f540815ba30 0x7f5408072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:23.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.240+0000 7f54067fc640 1 -- 172.21.15.83:0/1456353491 >> v1:172.21.15.167:6789/0 conn(0x7f540815c960 legacy=0x7f54080cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:23.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.240+0000 7f54067fc640 1 -- 172.21.15.83:0/1456353491 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5408071850 con 0x7f54080cfbe0 2024-04-17T10:38:23.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.240+0000 7f54067fc640 1 -- 172.21.15.83:0/1456353491 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f5400001a20 con 0x7f54080cfbe0 2024-04-17T10:38:23.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.240+0000 7f54067fc640 1 -- 172.21.15.83:0/1456353491 >> v1:172.21.15.83:6789/0 conn(0x7f54080cfbe0 legacy=0x7f540806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:23.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.240+0000 7f54067fc640 1 --2- 172.21.15.83:0/1456353491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53f40039c0 0x7f53f4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:23.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.240+0000 7f54067fc640 1 --2- 172.21.15.83:0/1456353491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53f4004810 0x7f53f4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:23.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.240+0000 7f54067fc640 1 --2- 172.21.15.83:0/1456353491 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53f40051d0 0x7f53f400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:23.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.240+0000 7f54067fc640 1 -- 172.21.15.83:0/1456353491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5408071ea0 con 0x7f53f4004810 2024-04-17T10:38:23.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.240+0000 7f54067fc640 1 -- 172.21.15.83:0/1456353491 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f400c170 con 0x7f53f40039c0 2024-04-17T10:38:23.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.240+0000 7f54067fc640 1 -- 172.21.15.83:0/1456353491 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f400c370 con 0x7f53f40051d0 2024-04-17T10:38:23.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.240+0000 7f53fffff640 1 --2- 172.21.15.83:0/1456353491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53f4004810 0x7f53f4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:23.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.240+0000 7f5406ffd640 1 --2- 172.21.15.83:0/1456353491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53f40039c0 0x7f53f4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:23.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.240+0000 7f53fffff640 1 -- 172.21.15.83:0/1456353491 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53f40051d0 msgr2=0x7f53f400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:23.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.240+0000 7f53fffff640 1 --2- 172.21.15.83:0/1456353491 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53f40051d0 0x7f53f400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:23.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.240+0000 7f53fffff640 1 -- 172.21.15.83:0/1456353491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53f40039c0 msgr2=0x7f53f4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:23.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.240+0000 7f53fffff640 1 --2- 172.21.15.83:0/1456353491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53f40039c0 0x7f53f4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:23.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.240+0000 7f54077fe640 1 -- 172.21.15.83:0/1456353491 reap_dead start 2024-04-17T10:38:23.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.240+0000 7f53fffff640 1 -- 172.21.15.83:0/1456353491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f4003530 con 0x7f53f4004810 2024-04-17T10:38:23.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f53fffff640 1 --2- 172.21.15.83:0/1456353491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53f4004810 0x7f53f4004c90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f53f0001780 tx=0x7f53f001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f87f6839073e4b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:23.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f54067fc640 1 -- 172.21.15.83:0/1456353491 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f00013a0 con 0x7f53f4004810 2024-04-17T10:38:23.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f54067fc640 1 -- 172.21.15.83:0/1456353491 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f00052f0 con 0x7f53f4004810 2024-04-17T10:38:23.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f54067fc640 1 -- 172.21.15.83:0/1456353491 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f0003840 con 0x7f53f4004810 2024-04-17T10:38:23.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f540d799640 1 -- 172.21.15.83:0/1456353491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53f4004810 msgr2=0x7f53f4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:23.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f540d799640 1 --2- 172.21.15.83:0/1456353491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53f4004810 0x7f53f4004c90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f53f0001780 tx=0x7f53f001c420 comp rx=0 tx=0).stop 2024-04-17T10:38:23.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f540d799640 1 -- 172.21.15.83:0/1456353491 shutdown_connections 2024-04-17T10:38:23.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f540d799640 1 --2- 172.21.15.83:0/1456353491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53f4004810 0x7f53f4004c90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:23.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f540d799640 1 -- 172.21.15.83:0/1456353491 >> 172.21.15.83:0/1456353491 conn(0x7f540806abe0 msgr2=0x7f540806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:23.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f540d799640 1 -- 172.21.15.83:0/1456353491 shutdown_connections 2024-04-17T10:38:23.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f540d799640 1 -- 172.21.15.83:0/1456353491 wait complete. 2024-04-17T10:38:23.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f540d799640 1 Processor -- start 2024-04-17T10:38:23.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f540d799640 1 -- start start 2024-04-17T10:38:23.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f540d799640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53f4004810 0x7f5408145210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:23.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f540d799640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5408153040 0x7f54081534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:23.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f540d799640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54080ad840 0x7f54080adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:23.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f540d799640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54080cfa60 con 0x7f54080ad840 2024-04-17T10:38:23.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f540d799640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54080cf8e0 con 0x7f5408153040 2024-04-17T10:38:23.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f540d799640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f540815cd60 con 0x7f53f4004810 2024-04-17T10:38:23.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f54077fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54080ad840 0x7f54080adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:23.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f54077fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54080ad840 0x7f54080adcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48198/0 (socket says 172.21.15.83:48198) 2024-04-17T10:38:23.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f54077fe640 1 -- 172.21.15.83:0/1042083374 learned_addr learned my addr 172.21.15.83:0/1042083374 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:23.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f53fffff640 1 --2- 172.21.15.83:0/1042083374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5408153040 0x7f54081534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:23.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f5406ffd640 1 --2- 172.21.15.83:0/1042083374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53f4004810 0x7f5408145210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:23.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f54077fe640 1 -- 172.21.15.83:0/1042083374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53f4004810 msgr2=0x7f5408145210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:23.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f54077fe640 1 --2- 172.21.15.83:0/1042083374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53f4004810 0x7f5408145210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:23.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f54077fe640 1 -- 172.21.15.83:0/1042083374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5408153040 msgr2=0x7f54081534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:23.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f54077fe640 1 --2- 172.21.15.83:0/1042083374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5408153040 0x7f54081534c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:23.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f54077fe640 1 -- 172.21.15.83:0/1042083374 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54081f7410 con 0x7f54080ad840 2024-04-17T10:38:23.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f5406ffd640 1 --2- 172.21.15.83:0/1042083374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53f4004810 0x7f5408145210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:23.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f54077fe640 1 --2- 172.21.15.83:0/1042083374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54080ad840 0x7f54080adcf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f54080ae840 tx=0x7f540000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:23.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f53ff7fe640 1 -- 172.21.15.83:0/1042083374 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540000ce50 con 0x7f54080ad840 2024-04-17T10:38:23.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f53ff7fe640 1 -- 172.21.15.83:0/1042083374 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f540000a5b0 con 0x7f54080ad840 2024-04-17T10:38:23.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f540d799640 1 -- 172.21.15.83:0/1042083374 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54081f76f0 con 0x7f54080ad840 2024-04-17T10:38:23.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f53ff7fe640 1 -- 172.21.15.83:0/1042083374 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540000a9a0 con 0x7f54080ad840 2024-04-17T10:38:23.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.244+0000 7f540d799640 1 -- 172.21.15.83:0/1042083374 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54081f7cb0 con 0x7f54080ad840 2024-04-17T10:38:23.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.248+0000 7f540d799640 1 -- 172.21.15.83:0/1042083374 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5408074f90 con 0x7f54080ad840 2024-04-17T10:38:23.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.248+0000 7f53ff7fe640 1 -- 172.21.15.83:0/1042083374 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f540000ab40 con 0x7f54080ad840 2024-04-17T10:38:23.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.248+0000 7f53ff7fe640 1 --2- 172.21.15.83:0/1042083374 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f53e8066f90 0x7f53e8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:23.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.248+0000 7f53ff7fe640 1 -- 172.21.15.83:0/1042083374 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5400084830 con 0x7f54080ad840 2024-04-17T10:38:23.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.248+0000 7f5406ffd640 1 --2- 172.21.15.83:0/1042083374 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f53e8066f90 0x7f53e8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:23.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.248+0000 7f5406ffd640 1 --2- 172.21.15.83:0/1042083374 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f53e8066f90 0x7f53e8069450 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5408152ac0 tx=0x7f53f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:23.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.252+0000 7f53ff7fe640 1 -- 172.21.15.83:0/1042083374 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5400055220 con 0x7f54080ad840 2024-04-17T10:38:23.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.428+0000 7f540d799640 1 -- 172.21.15.83:0/1042083374 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f54081f7e80 con 0x7f54080ad840 2024-04-17T10:38:23.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.428+0000 7f53ff7fe640 1 -- 172.21.15.83:0/1042083374 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f54000592c0 con 0x7f54080ad840 2024-04-17T10:38:23.426 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:23.427 INFO:teuthology.orchestra.run.smithi083.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-17T10:38:23.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.432+0000 7f540d799640 1 -- 172.21.15.83:0/1042083374 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f53e8066f90 msgr2=0x7f53e8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:23.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.432+0000 7f540d799640 1 --2- 172.21.15.83:0/1042083374 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f53e8066f90 0x7f53e8069450 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5408152ac0 tx=0x7f53f8008040 comp rx=0 tx=0).stop 2024-04-17T10:38:23.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.432+0000 7f540d799640 1 -- 172.21.15.83:0/1042083374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54080ad840 msgr2=0x7f54080adcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:23.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.432+0000 7f540d799640 1 --2- 172.21.15.83:0/1042083374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54080ad840 0x7f54080adcf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f54080ae840 tx=0x7f540000cc00 comp rx=0 tx=0).stop 2024-04-17T10:38:23.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.432+0000 7f540d799640 1 -- 172.21.15.83:0/1042083374 shutdown_connections 2024-04-17T10:38:23.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.432+0000 7f540d799640 1 --2- 172.21.15.83:0/1042083374 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f53e8066f90 0x7f53e8069450 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:23.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.432+0000 7f540d799640 1 --2- 172.21.15.83:0/1042083374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54080ad840 0x7f54080adcf0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:23.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.432+0000 7f540d799640 1 --2- 172.21.15.83:0/1042083374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5408153040 0x7f54081534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:23.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.432+0000 7f540d799640 1 --2- 172.21.15.83:0/1042083374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53f4004810 0x7f5408145210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:23.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.432+0000 7f540d799640 1 -- 172.21.15.83:0/1042083374 >> 172.21.15.83:0/1042083374 conn(0x7f540806abe0 msgr2=0x7f540815f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:23.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.432+0000 7f540d799640 1 -- 172.21.15.83:0/1042083374 shutdown_connections 2024-04-17T10:38:23.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.432+0000 7f540d799640 1 -- 172.21.15.83:0/1042083374 wait complete. 2024-04-17T10:38:23.443 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-17T10:38:23.444 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:38:23.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.556+0000 7f8c6e673640 1 Processor -- start 2024-04-17T10:38:23.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.556+0000 7f8c6e673640 1 -- start start 2024-04-17T10:38:23.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.556+0000 7f8c6e673640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c6806dcb0 0x7f8c68165650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:23.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.556+0000 7f8c6e673640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c6815e440 con 0x7f8c680aedd0 2024-04-17T10:38:23.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.556+0000 7f8c6e673640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c6815e720 con 0x7f8c6815db50 2024-04-17T10:38:23.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.556+0000 7f8c6e673640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6815ea00 con 0x7f8c6806dcb0 2024-04-17T10:38:23.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.556+0000 7f8c677fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8c680aedd0 0x7f8c6806d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39974/0 (socket says 172.21.15.83:39974) 2024-04-17T10:38:23.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.556+0000 7f8c677fe640 1 -- 172.21.15.83:0/3833553065 learned_addr learned my addr 172.21.15.83:0/3833553065 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:23.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.556+0000 7f8c6cc10640 1 --2- 172.21.15.83:0/3833553065 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c6806dcb0 0x7f8c68165650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:23.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.556+0000 7f8c66ffd640 1 -- 172.21.15.83:0/3833553065 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3801870793 0 0) 0x7f8c6815e440 con 0x7f8c680aedd0 2024-04-17T10:38:23.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.556+0000 7f8c66ffd640 1 -- 172.21.15.83:0/3833553065 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c50003530 con 0x7f8c680aedd0 2024-04-17T10:38:23.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.556+0000 7f8c66ffd640 1 -- 172.21.15.83:0/3833553065 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3426043816 0 0) 0x7f8c50003530 con 0x7f8c680aedd0 2024-04-17T10:38:23.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.556+0000 7f8c66ffd640 1 -- 172.21.15.83:0/3833553065 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c6806dcb0 msgr2=0x7f8c68165650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:23.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.556+0000 7f8c66ffd640 1 --2- 172.21.15.83:0/3833553065 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c6806dcb0 0x7f8c68165650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:23.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.556+0000 7f8c66ffd640 1 -- 172.21.15.83:0/3833553065 >> v1:172.21.15.167:6789/0 conn(0x7f8c6815db50 legacy=0x7f8c68070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:23.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.556+0000 7f8c66ffd640 1 -- 172.21.15.83:0/3833553065 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c6815ece0 con 0x7f8c680aedd0 2024-04-17T10:38:23.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.556+0000 7f8c66ffd640 1 -- 172.21.15.83:0/3833553065 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f8c5c001680 con 0x7f8c680aedd0 2024-04-17T10:38:23.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.556+0000 7f8c66ffd640 1 -- 172.21.15.83:0/3833553065 >> v1:172.21.15.83:6789/0 conn(0x7f8c680aedd0 legacy=0x7f8c6806d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:23.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.556+0000 7f8c66ffd640 1 --2- 172.21.15.83:0/3833553065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c500039f0 0x7f8c50003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:23.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.556+0000 7f8c66ffd640 1 --2- 172.21.15.83:0/3833553065 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c6806dcb0 0x7f8c50004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:23.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.556+0000 7f8c66ffd640 1 --2- 172.21.15.83:0/3833553065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c50004d50 0x7f8c500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:23.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.556+0000 7f8c66ffd640 1 -- 172.21.15.83:0/3833553065 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6815ea00 con 0x7f8c500039f0 2024-04-17T10:38:23.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.556+0000 7f8c66ffd640 1 -- 172.21.15.83:0/3833553065 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5000bab0 con 0x7f8c50004d50 2024-04-17T10:38:23.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.556+0000 7f8c66ffd640 1 -- 172.21.15.83:0/3833553065 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5000bcb0 con 0x7f8c6806dcb0 2024-04-17T10:38:23.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.556+0000 7f8c67fff640 1 --2- 172.21.15.83:0/3833553065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c500039f0 0x7f8c50003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:23.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.556+0000 7f8c6cc10640 1 --2- 172.21.15.83:0/3833553065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c50004d50 0x7f8c500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:23.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.556+0000 7f8c67fff640 1 -- 172.21.15.83:0/3833553065 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c6806dcb0 msgr2=0x7f8c50004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:23.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.556+0000 7f8c67fff640 1 --2- 172.21.15.83:0/3833553065 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c6806dcb0 0x7f8c50004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:23.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.556+0000 7f8c67fff640 1 -- 172.21.15.83:0/3833553065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c50004d50 msgr2=0x7f8c500051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:23.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.556+0000 7f8c67fff640 1 --2- 172.21.15.83:0/3833553065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c50004d50 0x7f8c500051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:23.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.556+0000 7f8c67fff640 1 -- 172.21.15.83:0/3833553065 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c50003530 con 0x7f8c500039f0 2024-04-17T10:38:23.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c67fff640 1 --2- 172.21.15.83:0/3833553065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c500039f0 0x7f8c50003e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8c58001750 tx=0x7f8c5801c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3cf0ba62f12c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:23.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c66ffd640 1 -- 172.21.15.83:0/3833553065 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c580013a0 con 0x7f8c500039f0 2024-04-17T10:38:23.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c66ffd640 1 -- 172.21.15.83:0/3833553065 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c580052f0 con 0x7f8c500039f0 2024-04-17T10:38:23.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c66ffd640 1 -- 172.21.15.83:0/3833553065 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c58003840 con 0x7f8c500039f0 2024-04-17T10:38:23.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c6e673640 1 -- 172.21.15.83:0/3833553065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c500039f0 msgr2=0x7f8c50003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:23.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c6e673640 1 --2- 172.21.15.83:0/3833553065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c500039f0 0x7f8c50003e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8c58001750 tx=0x7f8c5801c3b0 comp rx=0 tx=0).stop 2024-04-17T10:38:23.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c6cc10640 1 -- 172.21.15.83:0/3833553065 reap_dead start 2024-04-17T10:38:23.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c6e673640 1 -- 172.21.15.83:0/3833553065 shutdown_connections 2024-04-17T10:38:23.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c6e673640 1 -- 172.21.15.83:0/3833553065 >> 172.21.15.83:0/3833553065 conn(0x7f8c6806abe0 msgr2=0x7f8c6806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:23.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c6e673640 1 -- 172.21.15.83:0/3833553065 shutdown_connections 2024-04-17T10:38:23.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c6e673640 1 -- 172.21.15.83:0/3833553065 wait complete. 2024-04-17T10:38:23.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c6e673640 1 Processor -- start 2024-04-17T10:38:23.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c6e673640 1 -- start start 2024-04-17T10:38:23.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c6e673640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c500039f0 0x7f8c680b0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:23.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c6e673640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c681facc0 0x7f8c681f7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:23.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c6e673640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c68149c50 0x7f8c680ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:23.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c6e673640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c681661f0 con 0x7f8c681facc0 2024-04-17T10:38:23.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c6e673640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c68166070 con 0x7f8c68149c50 2024-04-17T10:38:23.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c6e673640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c68166370 con 0x7f8c500039f0 2024-04-17T10:38:23.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c677fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c681facc0 0x7f8c681f7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:23.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c677fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c681facc0 0x7f8c681f7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48230/0 (socket says 172.21.15.83:48230) 2024-04-17T10:38:23.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c677fe640 1 -- 172.21.15.83:0/1211752429 learned_addr learned my addr 172.21.15.83:0/1211752429 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:23.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c67fff640 1 --2- 172.21.15.83:0/1211752429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c500039f0 0x7f8c680b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:23.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c6cc10640 1 --2- 172.21.15.83:0/1211752429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c68149c50 0x7f8c680ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:23.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c677fe640 1 -- 172.21.15.83:0/1211752429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c500039f0 msgr2=0x7f8c680b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:23.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c677fe640 1 --2- 172.21.15.83:0/1211752429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c500039f0 0x7f8c680b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:23.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c677fe640 1 -- 172.21.15.83:0/1211752429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c68149c50 msgr2=0x7f8c680ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:23.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c677fe640 1 --2- 172.21.15.83:0/1211752429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c68149c50 0x7f8c680ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:23.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c677fe640 1 -- 172.21.15.83:0/1211752429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c680b13d0 con 0x7f8c681facc0 2024-04-17T10:38:23.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c677fe640 1 --2- 172.21.15.83:0/1211752429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c681facc0 0x7f8c681f7580 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c00c700 tx=0x7f8c5c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:23.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c64ff9640 1 -- 172.21.15.83:0/1211752429 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c018070 con 0x7f8c681facc0 2024-04-17T10:38:23.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c64ff9640 1 -- 172.21.15.83:0/1211752429 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c5c0074e0 con 0x7f8c681facc0 2024-04-17T10:38:23.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c6e673640 1 -- 172.21.15.83:0/1211752429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c680b10f0 con 0x7f8c681facc0 2024-04-17T10:38:23.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c64ff9640 1 -- 172.21.15.83:0/1211752429 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c004a30 con 0x7f8c681facc0 2024-04-17T10:38:23.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.560+0000 7f8c6e673640 1 -- 172.21.15.83:0/1211752429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c681fa4c0 con 0x7f8c681facc0 2024-04-17T10:38:23.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.564+0000 7f8c6e673640 1 -- 172.21.15.83:0/1211752429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c68074f90 con 0x7f8c681facc0 2024-04-17T10:38:23.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.564+0000 7f8c64ff9640 1 -- 172.21.15.83:0/1211752429 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8c5c01b070 con 0x7f8c681facc0 2024-04-17T10:38:23.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.564+0000 7f8c64ff9640 1 --2- 172.21.15.83:0/1211752429 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8c54066f90 0x7f8c54069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:23.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.564+0000 7f8c64ff9640 1 -- 172.21.15.83:0/1211752429 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c5c084450 con 0x7f8c681facc0 2024-04-17T10:38:23.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.564+0000 7f8c67fff640 1 --2- 172.21.15.83:0/1211752429 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8c54066f90 0x7f8c54069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:23.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.564+0000 7f8c67fff640 1 --2- 172.21.15.83:0/1211752429 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8c54066f90 0x7f8c54069450 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8c500057f0 tx=0x7f8c580011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:23.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.568+0000 7f8c64ff9640 1 -- 172.21.15.83:0/1211752429 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8c5c0554f0 con 0x7f8c681facc0 2024-04-17T10:38:23.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.716+0000 7f8c6e673640 1 -- 172.21.15.83:0/1211752429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c680b2af0 con 0x7f8c681facc0 2024-04-17T10:38:23.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.716+0000 7f8c64ff9640 1 -- 172.21.15.83:0/1211752429 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16533 (secure 0 0 0) 0x7f8c5c054df0 con 0x7f8c681facc0 2024-04-17T10:38:23.714 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:23.714 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":20,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:38:13.690876+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:38:10.464490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T10:38:11.043134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:38:23.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.720+0000 7f8c6e673640 1 -- 172.21.15.83:0/1211752429 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8c54066f90 msgr2=0x7f8c54069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:23.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.720+0000 7f8c6e673640 1 --2- 172.21.15.83:0/1211752429 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8c54066f90 0x7f8c54069450 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8c500057f0 tx=0x7f8c580011a0 comp rx=0 tx=0).stop 2024-04-17T10:38:23.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.720+0000 7f8c6e673640 1 -- 172.21.15.83:0/1211752429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c681facc0 msgr2=0x7f8c681f7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:23.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.720+0000 7f8c6e673640 1 --2- 172.21.15.83:0/1211752429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c681facc0 0x7f8c681f7580 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c00c700 tx=0x7f8c5c00cbc0 comp rx=0 tx=0).stop 2024-04-17T10:38:23.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.720+0000 7f8c6e673640 1 -- 172.21.15.83:0/1211752429 shutdown_connections 2024-04-17T10:38:23.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.720+0000 7f8c6e673640 1 --2- 172.21.15.83:0/1211752429 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8c54066f90 0x7f8c54069450 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:23.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.720+0000 7f8c6e673640 1 --2- 172.21.15.83:0/1211752429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c68149c50 0x7f8c680ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:23.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.720+0000 7f8c6e673640 1 --2- 172.21.15.83:0/1211752429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c681facc0 0x7f8c681f7580 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:23.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.720+0000 7f8c6e673640 1 --2- 172.21.15.83:0/1211752429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c500039f0 0x7f8c680b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:23.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.720+0000 7f8c6e673640 1 -- 172.21.15.83:0/1211752429 >> 172.21.15.83:0/1211752429 conn(0x7f8c6806abe0 msgr2=0x7f8c68070470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:23.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.720+0000 7f8c6e673640 1 -- 172.21.15.83:0/1211752429 shutdown_connections 2024-04-17T10:38:23.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.720+0000 7f8c6e673640 1 -- 172.21.15.83:0/1211752429 wait complete. 2024-04-17T10:38:23.731 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:38:10.464490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:38:11.043134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:38:23.732 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:38:23.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.848+0000 7f59a10b2640 1 Processor -- start 2024-04-17T10:38:23.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.848+0000 7f59a10b2640 1 -- start start 2024-04-17T10:38:23.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.848+0000 7f59a10b2640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f599c15e0f0 0x7f599c0b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:23.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.848+0000 7f59a10b2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f599c0ce400 0x7f599c161c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:23.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.848+0000 7f59a10b2640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f599c15e4d0 con 0x7f599c0b5450 2024-04-17T10:38:23.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.848+0000 7f59a10b2640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f599c162b40 con 0x7f599c0ce400 2024-04-17T10:38:23.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.848+0000 7f59a10b2640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f599c162580 con 0x7f599c15e0f0 2024-04-17T10:38:23.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.848+0000 7f5993fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f599c0ce400 0x7f599c161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:23.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.848+0000 7f5993fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f599c0ce400 0x7f599c161c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48246/0 (socket says 172.21.15.83:48246) 2024-04-17T10:38:23.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.848+0000 7f5993fff640 1 -- 172.21.15.83:0/1349488566 learned_addr learned my addr 172.21.15.83:0/1349488566 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:23.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.848+0000 7f599ad9d640 1 --2- 172.21.15.83:0/1349488566 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f599c15e0f0 0x7f599c0b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:23.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.848+0000 7f5993fff640 1 -- 172.21.15.83:0/1349488566 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f599c15e0f0 msgr2=0x7f599c0b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:23.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.848+0000 7f5993fff640 1 --2- 172.21.15.83:0/1349488566 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f599c15e0f0 0x7f599c0b6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:23.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.848+0000 7f5993fff640 1 -- 172.21.15.83:0/1349488566 >> v1:172.21.15.167:6789/0 conn(0x7f599c0b5450 legacy=0x7f599c0d2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:23.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.848+0000 7f5993fff640 1 -- 172.21.15.83:0/1349488566 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f599c162860 con 0x7f599c0ce400 2024-04-17T10:38:23.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.848+0000 7f599ad9d640 1 --2- 172.21.15.83:0/1349488566 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f599c15e0f0 0x7f599c0b6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:23.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.852+0000 7f5993fff640 1 --2- 172.21.15.83:0/1349488566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f599c0ce400 0x7f599c161c10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f598c009870 tx=0x7f598c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c2de57221c7ac6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:23.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.852+0000 7f599a59c640 1 -- 172.21.15.83:0/1349488566 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c0b1070 con 0x7f599c0ce400 2024-04-17T10:38:23.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.852+0000 7f599a59c640 1 -- 172.21.15.83:0/1349488566 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f598c0ac640 con 0x7f599c0ce400 2024-04-17T10:38:23.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.852+0000 7f599a59c640 1 -- 172.21.15.83:0/1349488566 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c0aca10 con 0x7f599c0ce400 2024-04-17T10:38:23.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.852+0000 7f59a10b2640 1 -- 172.21.15.83:0/1349488566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f599c0ce400 msgr2=0x7f599c161c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:23.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.852+0000 7f59a10b2640 1 --2- 172.21.15.83:0/1349488566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f599c0ce400 0x7f599c161c10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f598c009870 tx=0x7f598c01c490 comp rx=0 tx=0).stop 2024-04-17T10:38:23.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.852+0000 7f59a10b2640 1 -- 172.21.15.83:0/1349488566 shutdown_connections 2024-04-17T10:38:23.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.852+0000 7f59a10b2640 1 --2- 172.21.15.83:0/1349488566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f599c0ce400 0x7f599c161c10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:23.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.852+0000 7f59a10b2640 1 --2- 172.21.15.83:0/1349488566 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f599c15e0f0 0x7f599c0b6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:23.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.852+0000 7f59a10b2640 1 -- 172.21.15.83:0/1349488566 >> 172.21.15.83:0/1349488566 conn(0x7f599c0d0450 msgr2=0x7f599c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:23.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.852+0000 7f59a10b2640 1 -- 172.21.15.83:0/1349488566 shutdown_connections 2024-04-17T10:38:23.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.852+0000 7f59a10b2640 1 -- 172.21.15.83:0/1349488566 wait complete. 2024-04-17T10:38:23.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.852+0000 7f59a10b2640 1 Processor -- start 2024-04-17T10:38:23.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.852+0000 7f59a10b2640 1 -- start start 2024-04-17T10:38:23.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.856+0000 7f59a10b2640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f599c0b5450 0x7f599c0c9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:23.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.856+0000 7f59a10b2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f599c0ce400 0x7f599c153c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:23.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.856+0000 7f59a10b2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f599c15e0f0 0x7f599c154190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:23.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.856+0000 7f59a10b2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599c1622d0 con 0x7f599c0ce400 2024-04-17T10:38:23.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.856+0000 7f59a10b2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599c162150 con 0x7f599c15e0f0 2024-04-17T10:38:23.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.856+0000 7f59a10b2640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f599c16af50 con 0x7f599c0b5450 2024-04-17T10:38:23.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.856+0000 7f5993fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f599c0ce400 0x7f599c153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:23.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.856+0000 7f5993fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f599c0ce400 0x7f599c153c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48252/0 (socket says 172.21.15.83:48252) 2024-04-17T10:38:23.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.856+0000 7f5993fff640 1 -- 172.21.15.83:0/284474706 learned_addr learned my addr 172.21.15.83:0/284474706 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:23.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.856+0000 7f599ad9d640 1 --2- 172.21.15.83:0/284474706 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f599c0b5450 0x7f599c0c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:23.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.856+0000 7f5993fff640 1 -- 172.21.15.83:0/284474706 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f599c0b5450 msgr2=0x7f599c0c9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:23.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.856+0000 7f5993fff640 1 --2- 172.21.15.83:0/284474706 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f599c0b5450 0x7f599c0c9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:23.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.856+0000 7f5993fff640 1 -- 172.21.15.83:0/284474706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f599c15e0f0 msgr2=0x7f599c154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:23.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.856+0000 7f599b59e640 1 --2- 172.21.15.83:0/284474706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f599c15e0f0 0x7f599c154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:23.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.856+0000 7f5993fff640 1 --2- 172.21.15.83:0/284474706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f599c15e0f0 0x7f599c154190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:23.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.856+0000 7f5993fff640 1 -- 172.21.15.83:0/284474706 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f599c156ff0 con 0x7f599c0ce400 2024-04-17T10:38:23.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.856+0000 7f5993fff640 1 --2- 172.21.15.83:0/284474706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f599c0ce400 0x7f599c153c50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f598c01ab40 tx=0x7f598c019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:23.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.856+0000 7f59937fe640 1 -- 172.21.15.83:0/284474706 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c0b1040 con 0x7f599c0ce400 2024-04-17T10:38:23.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.856+0000 7f59937fe640 1 -- 172.21.15.83:0/284474706 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f598c0ba070 con 0x7f599c0ce400 2024-04-17T10:38:23.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.856+0000 7f59937fe640 1 -- 172.21.15.83:0/284474706 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c0b5880 con 0x7f599c0ce400 2024-04-17T10:38:23.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.856+0000 7f59a10b2640 1 -- 172.21.15.83:0/284474706 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f599c1572d0 con 0x7f599c0ce400 2024-04-17T10:38:23.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.856+0000 7f59a10b2640 1 -- 172.21.15.83:0/284474706 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f599c157890 con 0x7f599c0ce400 2024-04-17T10:38:23.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.856+0000 7f59a10b2640 1 -- 172.21.15.83:0/284474706 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f599c0681a0 con 0x7f599c0ce400 2024-04-17T10:38:23.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.856+0000 7f59937fe640 1 -- 172.21.15.83:0/284474706 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f598c0b5a20 con 0x7f599c0ce400 2024-04-17T10:38:23.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.856+0000 7f59937fe640 1 --2- 172.21.15.83:0/284474706 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5980066f60 0x7f5980069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:23.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.856+0000 7f599ad9d640 1 --2- 172.21.15.83:0/284474706 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5980066f60 0x7f5980069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:23.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.856+0000 7f59937fe640 1 -- 172.21.15.83:0/284474706 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f598c11ef80 con 0x7f599c0ce400 2024-04-17T10:38:23.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.860+0000 7f599ad9d640 1 --2- 172.21.15.83:0/284474706 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5980066f60 0x7f5980069420 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5984006fd0 tx=0x7f5984008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:23.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:23.860+0000 7f59937fe640 1 -- 172.21.15.83:0/284474706 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f598c016110 con 0x7f599c0ce400 2024-04-17T10:38:24.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.008+0000 7f59a10b2640 1 -- 172.21.15.83:0/284474706 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f599c074f90 con 0x7f599c0ce400 2024-04-17T10:38:24.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.008+0000 7f59937fe640 1 -- 172.21.15.83:0/284474706 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f598c0f00e0 con 0x7f599c0ce400 2024-04-17T10:38:24.005 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:38:24.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.012+0000 7f59a10b2640 1 -- 172.21.15.83:0/284474706 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5980066f60 msgr2=0x7f5980069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:24.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.012+0000 7f59a10b2640 1 --2- 172.21.15.83:0/284474706 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5980066f60 0x7f5980069420 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5984006fd0 tx=0x7f5984008040 comp rx=0 tx=0).stop 2024-04-17T10:38:24.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.012+0000 7f59a10b2640 1 -- 172.21.15.83:0/284474706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f599c0ce400 msgr2=0x7f599c153c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:24.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.012+0000 7f59a10b2640 1 --2- 172.21.15.83:0/284474706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f599c0ce400 0x7f599c153c50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f598c01ab40 tx=0x7f598c019230 comp rx=0 tx=0).stop 2024-04-17T10:38:24.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.012+0000 7f59a10b2640 1 -- 172.21.15.83:0/284474706 shutdown_connections 2024-04-17T10:38:24.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.012+0000 7f59a10b2640 1 --2- 172.21.15.83:0/284474706 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5980066f60 0x7f5980069420 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.012+0000 7f59a10b2640 1 --2- 172.21.15.83:0/284474706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f599c15e0f0 0x7f599c154190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.012+0000 7f59a10b2640 1 --2- 172.21.15.83:0/284474706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f599c0ce400 0x7f599c153c50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.012+0000 7f59a10b2640 1 --2- 172.21.15.83:0/284474706 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f599c0b5450 0x7f599c0c9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.012+0000 7f59a10b2640 1 -- 172.21.15.83:0/284474706 >> 172.21.15.83:0/284474706 conn(0x7f599c0d0450 msgr2=0x7f599c15c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:24.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.012+0000 7f59a10b2640 1 -- 172.21.15.83:0/284474706 shutdown_connections 2024-04-17T10:38:24.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.012+0000 7f59a10b2640 1 -- 172.21.15.83:0/284474706 wait complete. 2024-04-17T10:38:24.022 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:38:24.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.140+0000 7f9b2e6ed640 1 Processor -- start 2024-04-17T10:38:24.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.140+0000 7f9b2e6ed640 1 -- start start 2024-04-17T10:38:24.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.144+0000 7f9b2e6ed640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b280b4090 0x7f9b2806c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:24.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.144+0000 7f9b2e6ed640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9b280b44f0 0x7f9b2806cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:24.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.144+0000 7f9b2e6ed640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b280af160 con 0x7f9b28154670 2024-04-17T10:38:24.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.144+0000 7f9b2e6ed640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b28154a50 con 0x7f9b280b44f0 2024-04-17T10:38:24.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.144+0000 7f9b2e6ed640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b280b2ff0 con 0x7f9b280b4090 2024-04-17T10:38:24.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.144+0000 7f9b2cc8a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9b280b44f0 0x7f9b2806cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:24.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.144+0000 7f9b27fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9b28154670 0x7f9b280aea50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39980/0 (socket says 172.21.15.83:39980) 2024-04-17T10:38:24.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.144+0000 7f9b27fff640 1 -- 172.21.15.83:0/2573879134 learned_addr learned my addr 172.21.15.83:0/2573879134 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:24.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.144+0000 7f9b277fe640 1 --2- 172.21.15.83:0/2573879134 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b280b4090 0x7f9b2806c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:24.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.144+0000 7f9b2cc8a640 1 -- 172.21.15.83:0/2573879134 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b280b4090 msgr2=0x7f9b2806c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:24.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.144+0000 7f9b2cc8a640 1 --2- 172.21.15.83:0/2573879134 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b280b4090 0x7f9b2806c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.144+0000 7f9b2cc8a640 1 -- 172.21.15.83:0/2573879134 >> v1:172.21.15.83:6789/0 conn(0x7f9b28154670 legacy=0x7f9b280aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:24.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.144+0000 7f9b2cc8a640 1 -- 172.21.15.83:0/2573879134 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b280b32d0 con 0x7f9b280b44f0 2024-04-17T10:38:24.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.144+0000 7f9b26ffd640 1 -- 172.21.15.83:0/2573879134 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1449348998 0 0) 0x7f9b280af160 con 0x7f9b28154670 2024-04-17T10:38:24.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.144+0000 7f9b277fe640 1 --2- 172.21.15.83:0/2573879134 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b280b4090 0x7f9b2806c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:24.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.144+0000 7f9b2cc8a640 1 --2- 172.21.15.83:0/2573879134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b280b44f0 0x7f9b2806cab0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000aea0 tx=0x7f9b2001da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7d4d431ac3a7f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:24.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.144+0000 7f9b26ffd640 1 -- 172.21.15.83:0/2573879134 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b200a6880 con 0x7f9b280b44f0 2024-04-17T10:38:24.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.144+0000 7f9b26ffd640 1 -- 172.21.15.83:0/2573879134 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b200a6a20 con 0x7f9b280b44f0 2024-04-17T10:38:24.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.144+0000 7f9b26ffd640 1 -- 172.21.15.83:0/2573879134 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b200a6df0 con 0x7f9b280b44f0 2024-04-17T10:38:24.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.144+0000 7f9b2e6ed640 1 -- 172.21.15.83:0/2573879134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b280b44f0 msgr2=0x7f9b2806cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:24.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.144+0000 7f9b2e6ed640 1 --2- 172.21.15.83:0/2573879134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b280b44f0 0x7f9b2806cab0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000aea0 tx=0x7f9b2001da40 comp rx=0 tx=0).stop 2024-04-17T10:38:24.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.144+0000 7f9b2e6ed640 1 -- 172.21.15.83:0/2573879134 shutdown_connections 2024-04-17T10:38:24.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.144+0000 7f9b2e6ed640 1 --2- 172.21.15.83:0/2573879134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b280b44f0 0x7f9b2806cab0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.144+0000 7f9b2e6ed640 1 --2- 172.21.15.83:0/2573879134 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b280b4090 0x7f9b2806c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.144+0000 7f9b2e6ed640 1 -- 172.21.15.83:0/2573879134 >> 172.21.15.83:0/2573879134 conn(0x7f9b280b15b0 msgr2=0x7f9b280b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:24.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.144+0000 7f9b2e6ed640 1 -- 172.21.15.83:0/2573879134 shutdown_connections 2024-04-17T10:38:24.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.144+0000 7f9b2e6ed640 1 -- 172.21.15.83:0/2573879134 wait complete. 2024-04-17T10:38:24.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.148+0000 7f9b2e6ed640 1 Processor -- start 2024-04-17T10:38:24.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.148+0000 7f9b2e6ed640 1 -- start start 2024-04-17T10:38:24.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.148+0000 7f9b2e6ed640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b280b4090 0x7f9b28143d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:24.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.148+0000 7f9b2e6ed640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b280b44f0 0x7f9b2815cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:24.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.148+0000 7f9b2e6ed640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b28154670 0x7f9b2814c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:24.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.148+0000 7f9b2e6ed640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b28070470 con 0x7f9b28154670 2024-04-17T10:38:24.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.148+0000 7f9b2e6ed640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2815b3c0 con 0x7f9b280b44f0 2024-04-17T10:38:24.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.148+0000 7f9b2e6ed640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b280705f0 con 0x7f9b280b4090 2024-04-17T10:38:24.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.148+0000 7f9b2cc8a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b28154670 0x7f9b2814c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:24.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.148+0000 7f9b2cc8a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b28154670 0x7f9b2814c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48268/0 (socket says 172.21.15.83:48268) 2024-04-17T10:38:24.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.148+0000 7f9b2cc8a640 1 -- 172.21.15.83:0/3411217735 learned_addr learned my addr 172.21.15.83:0/3411217735 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:24.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.148+0000 7f9b27fff640 1 --2- 172.21.15.83:0/3411217735 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b280b4090 0x7f9b28143d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:24.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.148+0000 7f9b2cc8a640 1 -- 172.21.15.83:0/3411217735 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b280b4090 msgr2=0x7f9b28143d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:24.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.148+0000 7f9b2cc8a640 1 --2- 172.21.15.83:0/3411217735 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b280b4090 0x7f9b28143d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.148+0000 7f9b2cc8a640 1 -- 172.21.15.83:0/3411217735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b280b44f0 msgr2=0x7f9b2815cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:24.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.148+0000 7f9b2cc8a640 1 --2- 172.21.15.83:0/3411217735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b280b44f0 0x7f9b2815cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.148+0000 7f9b2cc8a640 1 -- 172.21.15.83:0/3411217735 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b2814f3a0 con 0x7f9b28154670 2024-04-17T10:38:24.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.148+0000 7f9b2cc8a640 1 --2- 172.21.15.83:0/3411217735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b28154670 0x7f9b2814c7b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9b2001c0f0 tx=0x7f9b2001a7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:24.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.148+0000 7f9b24ff9640 1 -- 172.21.15.83:0/3411217735 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b200bd020 con 0x7f9b28154670 2024-04-17T10:38:24.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.148+0000 7f9b24ff9640 1 -- 172.21.15.83:0/3411217735 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b200b9880 con 0x7f9b28154670 2024-04-17T10:38:24.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.148+0000 7f9b24ff9640 1 -- 172.21.15.83:0/3411217735 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b200b9c70 con 0x7f9b28154670 2024-04-17T10:38:24.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.148+0000 7f9b2e6ed640 1 -- 172.21.15.83:0/3411217735 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b28144420 con 0x7f9b28154670 2024-04-17T10:38:24.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.148+0000 7f9b2e6ed640 1 -- 172.21.15.83:0/3411217735 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b281449e0 con 0x7f9b28154670 2024-04-17T10:38:24.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.148+0000 7f9b2e6ed640 1 -- 172.21.15.83:0/3411217735 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b28074f90 con 0x7f9b28154670 2024-04-17T10:38:24.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.148+0000 7f9b24ff9640 1 -- 172.21.15.83:0/3411217735 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9b200c0020 con 0x7f9b28154670 2024-04-17T10:38:24.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.152+0000 7f9b24ff9640 1 --2- 172.21.15.83:0/3411217735 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9b14066f60 0x7f9b14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:24.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.152+0000 7f9b24ff9640 1 -- 172.21.15.83:0/3411217735 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b201213a0 con 0x7f9b28154670 2024-04-17T10:38:24.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.152+0000 7f9b27fff640 1 --2- 172.21.15.83:0/3411217735 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9b14066f60 0x7f9b14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:24.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.152+0000 7f9b27fff640 1 --2- 172.21.15.83:0/3411217735 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9b14066f60 0x7f9b14069420 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9b18000f70 tx=0x7f9b18009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:24.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.156+0000 7f9b24ff9640 1 -- 172.21.15.83:0/3411217735 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9b200c7030 con 0x7f9b28154670 2024-04-17T10:38:24.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.300+0000 7f9b2e6ed640 1 -- 172.21.15.83:0/3411217735 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9b280b03a0 con 0x7f9b28154670 2024-04-17T10:38:24.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.304+0000 7f9b24ff9640 1 -- 172.21.15.83:0/3411217735 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9b20017660 con 0x7f9b28154670 2024-04-17T10:38:24.299 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:38:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.304+0000 7f9b2e6ed640 1 -- 172.21.15.83:0/3411217735 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9b14066f60 msgr2=0x7f9b14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.304+0000 7f9b2e6ed640 1 --2- 172.21.15.83:0/3411217735 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9b14066f60 0x7f9b14069420 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9b18000f70 tx=0x7f9b18009040 comp rx=0 tx=0).stop 2024-04-17T10:38:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.304+0000 7f9b2e6ed640 1 -- 172.21.15.83:0/3411217735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b28154670 msgr2=0x7f9b2814c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.304+0000 7f9b2e6ed640 1 --2- 172.21.15.83:0/3411217735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b28154670 0x7f9b2814c7b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9b2001c0f0 tx=0x7f9b2001a7e0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.304+0000 7f9b2e6ed640 1 -- 172.21.15.83:0/3411217735 shutdown_connections 2024-04-17T10:38:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.304+0000 7f9b2e6ed640 1 --2- 172.21.15.83:0/3411217735 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9b14066f60 0x7f9b14069420 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.304+0000 7f9b2e6ed640 1 --2- 172.21.15.83:0/3411217735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b28154670 0x7f9b2814c7b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.304+0000 7f9b2e6ed640 1 --2- 172.21.15.83:0/3411217735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b280b44f0 0x7f9b2815cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.304+0000 7f9b2e6ed640 1 --2- 172.21.15.83:0/3411217735 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b280b4090 0x7f9b28143d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.304+0000 7f9b2e6ed640 1 -- 172.21.15.83:0/3411217735 >> 172.21.15.83:0/3411217735 conn(0x7f9b280b15b0 msgr2=0x7f9b280b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.304+0000 7f9b2e6ed640 1 -- 172.21.15.83:0/3411217735 shutdown_connections 2024-04-17T10:38:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.304+0000 7f9b2e6ed640 1 -- 172.21.15.83:0/3411217735 wait complete. 2024-04-17T10:38:24.316 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:38:24.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f7da1d640 1 Processor -- start 2024-04-17T10:38:24.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f7da1d640 1 -- start start 2024-04-17T10:38:24.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f7da1d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f7815bac0 0x7f6f78071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:24.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f7da1d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f7806c1f0 con 0x7f6f78070470 2024-04-17T10:38:24.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f7da1d640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f78070a10 con 0x7f6f7815b6e0 2024-04-17T10:38:24.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f7da1d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f78070cf0 con 0x7f6f7815bac0 2024-04-17T10:38:24.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f777fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6f78070470 0x7f6f7806ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39988/0 (socket says 172.21.15.83:39988) 2024-04-17T10:38:24.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f777fe640 1 -- 172.21.15.83:0/2468790429 learned_addr learned my addr 172.21.15.83:0/2468790429 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:24.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f6ffff640 1 --2- 172.21.15.83:0/2468790429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f7815bac0 0x7f6f78071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:24.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f767fc640 1 -- 172.21.15.83:0/2468790429 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 771243122 0 0) 0x7f6f7806c1f0 con 0x7f6f78070470 2024-04-17T10:38:24.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f767fc640 1 -- 172.21.15.83:0/2468790429 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f64003530 con 0x7f6f78070470 2024-04-17T10:38:24.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f767fc640 1 -- 172.21.15.83:0/2468790429 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3220279591 0 0) 0x7f6f64003530 con 0x7f6f78070470 2024-04-17T10:38:24.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f767fc640 1 -- 172.21.15.83:0/2468790429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f7815bac0 msgr2=0x7f6f78071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:24.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f767fc640 1 --2- 172.21.15.83:0/2468790429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f7815bac0 0x7f6f78071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f767fc640 1 -- 172.21.15.83:0/2468790429 >> v1:172.21.15.167:6789/0 conn(0x7f6f7815b6e0 legacy=0x7f6f780cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:24.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f767fc640 1 -- 172.21.15.83:0/2468790429 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f78070fd0 con 0x7f6f78070470 2024-04-17T10:38:24.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f767fc640 1 -- 172.21.15.83:0/2468790429 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f6f70001a00 con 0x7f6f78070470 2024-04-17T10:38:24.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f767fc640 1 -- 172.21.15.83:0/2468790429 >> v1:172.21.15.83:6789/0 conn(0x7f6f78070470 legacy=0x7f6f7806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:24.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f767fc640 1 --2- 172.21.15.83:0/2468790429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f64003a60 0x7f6f64003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:24.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f767fc640 1 --2- 172.21.15.83:0/2468790429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f7815bac0 0x7f6f64004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:24.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f767fc640 1 --2- 172.21.15.83:0/2468790429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f64004d50 0x7f6f640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:24.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f767fc640 1 -- 172.21.15.83:0/2468790429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f78070cf0 con 0x7f6f64004d50 2024-04-17T10:38:24.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f767fc640 1 -- 172.21.15.83:0/2468790429 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6400bae0 con 0x7f6f64003a60 2024-04-17T10:38:24.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f767fc640 1 -- 172.21.15.83:0/2468790429 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6400bce0 con 0x7f6f7815bac0 2024-04-17T10:38:24.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f777fe640 1 --2- 172.21.15.83:0/2468790429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f64004d50 0x7f6f640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:24.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f76ffd640 1 --2- 172.21.15.83:0/2468790429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f64003a60 0x7f6f64003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:24.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f6ffff640 1 --2- 172.21.15.83:0/2468790429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f7815bac0 0x7f6f64004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:24.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f777fe640 1 -- 172.21.15.83:0/2468790429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f7815bac0 msgr2=0x7f6f64004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:24.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f777fe640 1 --2- 172.21.15.83:0/2468790429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f7815bac0 0x7f6f64004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f777fe640 1 -- 172.21.15.83:0/2468790429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f64003a60 msgr2=0x7f6f64003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:24.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f777fe640 1 --2- 172.21.15.83:0/2468790429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f64003a60 0x7f6f64003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f777fe640 1 -- 172.21.15.83:0/2468790429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f64003530 con 0x7f6f64004d50 2024-04-17T10:38:24.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.440+0000 7f6f76ffd640 1 --2- 172.21.15.83:0/2468790429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f64003a60 0x7f6f64003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:24.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.444+0000 7f6f777fe640 1 --2- 172.21.15.83:0/2468790429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f64004d50 0x7f6f640051d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6f7000a830 tx=0x7f6f7001d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6948ca4cb00e66 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:24.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.444+0000 7f6f767fc640 1 -- 172.21.15.83:0/2468790429 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7000a480 con 0x7f6f64004d50 2024-04-17T10:38:24.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.444+0000 7f6f767fc640 1 -- 172.21.15.83:0/2468790429 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f700ad630 con 0x7f6f64004d50 2024-04-17T10:38:24.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.444+0000 7f6f767fc640 1 -- 172.21.15.83:0/2468790429 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f700ada60 con 0x7f6f64004d50 2024-04-17T10:38:24.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.444+0000 7f6f7da1d640 1 -- 172.21.15.83:0/2468790429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f64004d50 msgr2=0x7f6f640051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:24.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.444+0000 7f6f7da1d640 1 --2- 172.21.15.83:0/2468790429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f64004d50 0x7f6f640051d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6f7000a830 tx=0x7f6f7001d360 comp rx=0 tx=0).stop 2024-04-17T10:38:24.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.444+0000 7f6f777fe640 1 -- 172.21.15.83:0/2468790429 reap_dead start 2024-04-17T10:38:24.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.444+0000 7f6f7da1d640 1 -- 172.21.15.83:0/2468790429 shutdown_connections 2024-04-17T10:38:24.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.444+0000 7f6f7da1d640 1 -- 172.21.15.83:0/2468790429 >> 172.21.15.83:0/2468790429 conn(0x7f6f780b15b0 msgr2=0x7f6f780b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:24.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.444+0000 7f6f7da1d640 1 -- 172.21.15.83:0/2468790429 shutdown_connections 2024-04-17T10:38:24.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.444+0000 7f6f7da1d640 1 -- 172.21.15.83:0/2468790429 wait complete. 2024-04-17T10:38:24.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.444+0000 7f6f7da1d640 1 Processor -- start 2024-04-17T10:38:24.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.444+0000 7f6f7da1d640 1 -- start start 2024-04-17T10:38:24.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.444+0000 7f6f7da1d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f781f1ba0 0x7f6f7815a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:24.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.444+0000 7f6f7da1d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f78144c80 0x7f6f78145100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:24.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.444+0000 7f6f7da1d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f781f4ea0 0x7f6f781f5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:24.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.444+0000 7f6f7da1d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7806b6a0 con 0x7f6f781f4ea0 2024-04-17T10:38:24.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.444+0000 7f6f7da1d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7806b520 con 0x7f6f78144c80 2024-04-17T10:38:24.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.444+0000 7f6f7da1d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f78071cc0 con 0x7f6f781f1ba0 2024-04-17T10:38:24.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.444+0000 7f6f777fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f781f4ea0 0x7f6f781f5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:24.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.444+0000 7f6f777fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f781f4ea0 0x7f6f781f5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48278/0 (socket says 172.21.15.83:48278) 2024-04-17T10:38:24.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.444+0000 7f6f777fe640 1 -- 172.21.15.83:0/579269409 learned_addr learned my addr 172.21.15.83:0/579269409 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:24.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.444+0000 7f6f76ffd640 1 --2- 172.21.15.83:0/579269409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f781f1ba0 0x7f6f7815a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:24.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.444+0000 7f6f6ffff640 1 --2- 172.21.15.83:0/579269409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f78144c80 0x7f6f78145100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:24.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.444+0000 7f6f777fe640 1 -- 172.21.15.83:0/579269409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f781f1ba0 msgr2=0x7f6f7815a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:24.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.444+0000 7f6f777fe640 1 --2- 172.21.15.83:0/579269409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f781f1ba0 0x7f6f7815a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.444+0000 7f6f777fe640 1 -- 172.21.15.83:0/579269409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f78144c80 msgr2=0x7f6f78145100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:24.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.444+0000 7f6f777fe640 1 --2- 172.21.15.83:0/579269409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f78144c80 0x7f6f78145100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.444+0000 7f6f777fe640 1 -- 172.21.15.83:0/579269409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7815a880 con 0x7f6f781f4ea0 2024-04-17T10:38:24.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.444+0000 7f6f6ffff640 1 --2- 172.21.15.83:0/579269409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f78144c80 0x7f6f78145100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:24.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.448+0000 7f6f777fe640 1 --2- 172.21.15.83:0/579269409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f781f4ea0 0x7f6f781f5320 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6f7001d190 tx=0x7f6f64005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:24.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.448+0000 7f6f74ff9640 1 -- 172.21.15.83:0/579269409 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7000a480 con 0x7f6f781f4ea0 2024-04-17T10:38:24.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.448+0000 7f6f74ff9640 1 -- 172.21.15.83:0/579269409 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f700019d0 con 0x7f6f781f4ea0 2024-04-17T10:38:24.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.448+0000 7f6f74ff9640 1 -- 172.21.15.83:0/579269409 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f700ad430 con 0x7f6f781f4ea0 2024-04-17T10:38:24.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.448+0000 7f6f7da1d640 1 -- 172.21.15.83:0/579269409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f7815ddc0 con 0x7f6f781f4ea0 2024-04-17T10:38:24.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.448+0000 7f6f7da1d640 1 -- 172.21.15.83:0/579269409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f7815e380 con 0x7f6f781f4ea0 2024-04-17T10:38:24.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.448+0000 7f6f7da1d640 1 -- 172.21.15.83:0/579269409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f78074f90 con 0x7f6f781f4ea0 2024-04-17T10:38:24.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.448+0000 7f6f74ff9640 1 -- 172.21.15.83:0/579269409 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6f700bd030 con 0x7f6f781f4ea0 2024-04-17T10:38:24.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.448+0000 7f6f74ff9640 1 --2- 172.21.15.83:0/579269409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6f58066f60 0x7f6f58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:24.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.448+0000 7f6f74ff9640 1 -- 172.21.15.83:0/579269409 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f7011eab0 con 0x7f6f781f4ea0 2024-04-17T10:38:24.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.448+0000 7f6f76ffd640 1 --2- 172.21.15.83:0/579269409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6f58066f60 0x7f6f58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:24.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.448+0000 7f6f76ffd640 1 --2- 172.21.15.83:0/579269409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6f58066f60 0x7f6f58069420 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6f68001760 tx=0x7f6f68009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:24.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.452+0000 7f6f74ff9640 1 -- 172.21.15.83:0/579269409 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6f700c4060 con 0x7f6f781f4ea0 2024-04-17T10:38:24.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.596+0000 7f6f7da1d640 1 -- 172.21.15.83:0/579269409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6f7815e550 con 0x7f6f781f4ea0 2024-04-17T10:38:24.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.596+0000 7f6f74ff9640 1 -- 172.21.15.83:0/579269409 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6f70017070 con 0x7f6f781f4ea0 2024-04-17T10:38:24.594 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:38:24.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.600+0000 7f6f7da1d640 1 -- 172.21.15.83:0/579269409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6f58066f60 msgr2=0x7f6f58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:24.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.600+0000 7f6f7da1d640 1 --2- 172.21.15.83:0/579269409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6f58066f60 0x7f6f58069420 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6f68001760 tx=0x7f6f68009040 comp rx=0 tx=0).stop 2024-04-17T10:38:24.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.600+0000 7f6f7da1d640 1 -- 172.21.15.83:0/579269409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f781f4ea0 msgr2=0x7f6f781f5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:24.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.600+0000 7f6f7da1d640 1 --2- 172.21.15.83:0/579269409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f781f4ea0 0x7f6f781f5320 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6f7001d190 tx=0x7f6f64005840 comp rx=0 tx=0).stop 2024-04-17T10:38:24.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.600+0000 7f6f7da1d640 1 -- 172.21.15.83:0/579269409 shutdown_connections 2024-04-17T10:38:24.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.600+0000 7f6f7da1d640 1 --2- 172.21.15.83:0/579269409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6f58066f60 0x7f6f58069420 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.600+0000 7f6f7da1d640 1 --2- 172.21.15.83:0/579269409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f781f4ea0 0x7f6f781f5320 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.600+0000 7f6f7da1d640 1 --2- 172.21.15.83:0/579269409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f78144c80 0x7f6f78145100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.600+0000 7f6f7da1d640 1 --2- 172.21.15.83:0/579269409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f781f1ba0 0x7f6f7815a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.600+0000 7f6f7da1d640 1 -- 172.21.15.83:0/579269409 >> 172.21.15.83:0/579269409 conn(0x7f6f780b15b0 msgr2=0x7f6f7806c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:24.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.600+0000 7f6f7da1d640 1 -- 172.21.15.83:0/579269409 shutdown_connections 2024-04-17T10:38:24.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.600+0000 7f6f7da1d640 1 -- 172.21.15.83:0/579269409 wait complete. 2024-04-17T10:38:24.612 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:38:24.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.720+0000 7fd3594f5640 1 Processor -- start 2024-04-17T10:38:24.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.720+0000 7fd3594f5640 1 -- start start 2024-04-17T10:38:24.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.724+0000 7fd3594f5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd35415ba30 0x7fd354072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:24.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.724+0000 7fd3594f5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd3540cfbe0 0x7fd35406c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:24.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.724+0000 7fd3594f5640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd35406e320 con 0x7fd35415c960 2024-04-17T10:38:24.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.724+0000 7fd3594f5640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd354071ea0 con 0x7fd3540cfbe0 2024-04-17T10:38:24.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.724+0000 7fd3594f5640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd35406e040 con 0x7fd35415ba30 2024-04-17T10:38:24.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.724+0000 7fd34a7fc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd35415c960 0x7fd3540cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40002/0 (socket says 172.21.15.83:40002) 2024-04-17T10:38:24.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.724+0000 7fd34a7fc640 1 -- 172.21.15.83:0/1733488130 learned_addr learned my addr 172.21.15.83:0/1733488130 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:24.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.724+0000 7fd3537fe640 1 --2- 172.21.15.83:0/1733488130 >> v2:172.21.15.83:3300/0 conn(0x7fd3540cfbe0 0x7fd35406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:24.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.724+0000 7fd352ffd640 1 --2- 172.21.15.83:0/1733488130 >> v2:172.21.15.167:3300/0 conn(0x7fd35415ba30 0x7fd354072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:24.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.724+0000 7fd3527fc640 1 -- 172.21.15.83:0/1733488130 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 791476177 0 0) 0x7fd35406e320 con 0x7fd35415c960 2024-04-17T10:38:24.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.724+0000 7fd3527fc640 1 -- 172.21.15.83:0/1733488130 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd340003530 con 0x7fd35415c960 2024-04-17T10:38:24.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.724+0000 7fd3537fe640 1 -- 172.21.15.83:0/1733488130 >> v2:172.21.15.167:3300/0 conn(0x7fd35415ba30 msgr2=0x7fd354072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:24.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.724+0000 7fd3537fe640 1 --2- 172.21.15.83:0/1733488130 >> v2:172.21.15.167:3300/0 conn(0x7fd35415ba30 0x7fd354072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.724+0000 7fd3537fe640 1 -- 172.21.15.83:0/1733488130 >> v1:172.21.15.83:6789/0 conn(0x7fd35415c960 legacy=0x7fd3540cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:24.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.724+0000 7fd3527fc640 1 -- 172.21.15.83:0/1733488130 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3638900998 0 0) 0x7fd340003530 con 0x7fd35415c960 2024-04-17T10:38:24.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.724+0000 7fd3537fe640 1 -- 172.21.15.83:0/1733488130 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd354071850 con 0x7fd3540cfbe0 2024-04-17T10:38:24.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.724+0000 7fd352ffd640 1 --2- 172.21.15.83:0/1733488130 >> v2:172.21.15.167:3300/0 conn(0x7fd35415ba30 0x7fd354072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:38:24.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.724+0000 7fd3537fe640 1 --2- 172.21.15.83:0/1733488130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3540cfbe0 0x7fd35406c490 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd3540b2410 tx=0x7fd34c01c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e472d8efce390032 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:24.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.724+0000 7fd3527fc640 1 -- 172.21.15.83:0/1733488130 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c0ac070 con 0x7fd3540cfbe0 2024-04-17T10:38:24.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.724+0000 7fd3527fc640 1 -- 172.21.15.83:0/1733488130 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34c0a6490 con 0x7fd3540cfbe0 2024-04-17T10:38:24.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.724+0000 7fd3527fc640 1 -- 172.21.15.83:0/1733488130 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c0016a0 con 0x7fd3540cfbe0 2024-04-17T10:38:24.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.728+0000 7fd3594f5640 1 -- 172.21.15.83:0/1733488130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3540cfbe0 msgr2=0x7fd35406c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:24.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.728+0000 7fd3594f5640 1 --2- 172.21.15.83:0/1733488130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3540cfbe0 0x7fd35406c490 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd3540b2410 tx=0x7fd34c01c7c0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.728+0000 7fd3594f5640 1 -- 172.21.15.83:0/1733488130 shutdown_connections 2024-04-17T10:38:24.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.728+0000 7fd3594f5640 1 --2- 172.21.15.83:0/1733488130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3540cfbe0 0x7fd35406c490 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.728+0000 7fd3594f5640 1 --2- 172.21.15.83:0/1733488130 >> v2:172.21.15.167:3300/0 conn(0x7fd35415ba30 0x7fd354072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.728+0000 7fd3594f5640 1 -- 172.21.15.83:0/1733488130 >> 172.21.15.83:0/1733488130 conn(0x7fd35406abe0 msgr2=0x7fd35406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:24.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.728+0000 7fd3594f5640 1 -- 172.21.15.83:0/1733488130 shutdown_connections 2024-04-17T10:38:24.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.728+0000 7fd3594f5640 1 -- 172.21.15.83:0/1733488130 wait complete. 2024-04-17T10:38:24.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.728+0000 7fd3594f5640 1 Processor -- start 2024-04-17T10:38:24.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.728+0000 7fd3594f5640 1 -- start start 2024-04-17T10:38:24.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.728+0000 7fd3594f5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3540cfbe0 0x7fd35414d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:24.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.728+0000 7fd3594f5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd35415ba30 0x7fd354150270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:24.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.728+0000 7fd3594f5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd35415c960 0x7fd354154620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:24.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.728+0000 7fd3594f5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35406de20 con 0x7fd35415ba30 2024-04-17T10:38:24.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.728+0000 7fd3594f5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35406dca0 con 0x7fd3540cfbe0 2024-04-17T10:38:24.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.728+0000 7fd3594f5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd354168520 con 0x7fd35415c960 2024-04-17T10:38:24.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.728+0000 7fd34a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd35415ba30 0x7fd354150270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:24.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.728+0000 7fd34a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd35415ba30 0x7fd354150270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48288/0 (socket says 172.21.15.83:48288) 2024-04-17T10:38:24.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.728+0000 7fd34a7fc640 1 -- 172.21.15.83:0/3239662812 learned_addr learned my addr 172.21.15.83:0/3239662812 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:24.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.728+0000 7fd34a7fc640 1 -- 172.21.15.83:0/3239662812 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd35415c960 msgr2=0x7fd354154620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:24.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.728+0000 7fd34a7fc640 1 --2- 172.21.15.83:0/3239662812 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd35415c960 0x7fd354154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.728+0000 7fd34a7fc640 1 -- 172.21.15.83:0/3239662812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3540cfbe0 msgr2=0x7fd35414d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:24.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.728+0000 7fd34a7fc640 1 --2- 172.21.15.83:0/3239662812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3540cfbe0 0x7fd35414d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.728+0000 7fd34a7fc640 1 -- 172.21.15.83:0/3239662812 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35414c2f0 con 0x7fd35415ba30 2024-04-17T10:38:24.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.728+0000 7fd34a7fc640 1 --2- 172.21.15.83:0/3239662812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd35415ba30 0x7fd354150270 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd33c001ec0 tx=0x7fd33c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:24.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.728+0000 7fd34bfff640 1 -- 172.21.15.83:0/3239662812 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c018070 con 0x7fd35415ba30 2024-04-17T10:38:24.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.728+0000 7fd34bfff640 1 -- 172.21.15.83:0/3239662812 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33c0074e0 con 0x7fd35415ba30 2024-04-17T10:38:24.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.728+0000 7fd34bfff640 1 -- 172.21.15.83:0/3239662812 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c004a30 con 0x7fd35415ba30 2024-04-17T10:38:24.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.728+0000 7fd3594f5640 1 -- 172.21.15.83:0/3239662812 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd35414c5d0 con 0x7fd35415ba30 2024-04-17T10:38:24.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.728+0000 7fd3594f5640 1 -- 172.21.15.83:0/3239662812 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd35414ce70 con 0x7fd35415ba30 2024-04-17T10:38:24.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.732+0000 7fd3594f5640 1 -- 172.21.15.83:0/3239662812 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd354074f90 con 0x7fd35415ba30 2024-04-17T10:38:24.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.732+0000 7fd34bfff640 1 -- 172.21.15.83:0/3239662812 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd33c01b070 con 0x7fd35415ba30 2024-04-17T10:38:24.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.732+0000 7fd34bfff640 1 --2- 172.21.15.83:0/3239662812 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd334066f60 0x7fd334069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:24.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.732+0000 7fd34bfff640 1 -- 172.21.15.83:0/3239662812 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd33c0843b0 con 0x7fd35415ba30 2024-04-17T10:38:24.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.732+0000 7fd352ffd640 1 --2- 172.21.15.83:0/3239662812 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd334066f60 0x7fd334069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:24.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.732+0000 7fd352ffd640 1 --2- 172.21.15.83:0/3239662812 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd334066f60 0x7fd334069420 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd344006f10 tx=0x7fd344008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:24.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.736+0000 7fd34bfff640 1 -- 172.21.15.83:0/3239662812 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd33c055450 con 0x7fd35415ba30 2024-04-17T10:38:24.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.920+0000 7fd3594f5640 1 -- 172.21.15.83:0/3239662812 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd35406b3e0 con 0x7fd35415ba30 2024-04-17T10:38:24.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.920+0000 7fd34bfff640 1 -- 172.21.15.83:0/3239662812 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4938 (secure 0 0 0) 0x7fd33c054d50 con 0x7fd35415ba30 2024-04-17T10:38:24.917 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:24.917 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2884469968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2884469968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2884469968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2919571009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2919571009},{"type":"v1","addr":"172.21.15.83:6836","nonce":2919571009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1635176834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1635176834},{"type":"v1","addr":"172.21.15.83:6837","nonce":1635176834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:38:12.032695+0000","modified":"2024-04-17T10:38:14.064978+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6835/3871588351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3871588351},{"type":"v1","addr":"172.21.15.167:6835","nonce":3871588351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-17T10:38:24.917 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 6 2024-04-17T10:38:24.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.924+0000 7fd3594f5640 1 -- 172.21.15.83:0/3239662812 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd334066f60 msgr2=0x7fd334069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:24.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.924+0000 7fd3594f5640 1 --2- 172.21.15.83:0/3239662812 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd334066f60 0x7fd334069420 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd344006f10 tx=0x7fd344008040 comp rx=0 tx=0).stop 2024-04-17T10:38:24.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.924+0000 7fd3594f5640 1 -- 172.21.15.83:0/3239662812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd35415ba30 msgr2=0x7fd354150270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:24.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.924+0000 7fd3594f5640 1 --2- 172.21.15.83:0/3239662812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd35415ba30 0x7fd354150270 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd33c001ec0 tx=0x7fd33c00c9b0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.924+0000 7fd3594f5640 1 -- 172.21.15.83:0/3239662812 shutdown_connections 2024-04-17T10:38:24.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.924+0000 7fd3594f5640 1 --2- 172.21.15.83:0/3239662812 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd334066f60 0x7fd334069420 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.924+0000 7fd3594f5640 1 --2- 172.21.15.83:0/3239662812 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd35415c960 0x7fd354154620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.924+0000 7fd3594f5640 1 --2- 172.21.15.83:0/3239662812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd35415ba30 0x7fd354150270 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.924+0000 7fd3594f5640 1 --2- 172.21.15.83:0/3239662812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3540cfbe0 0x7fd35414d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:24.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.924+0000 7fd3594f5640 1 -- 172.21.15.83:0/3239662812 >> 172.21.15.83:0/3239662812 conn(0x7fd35406abe0 msgr2=0x7fd35415d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:24.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.924+0000 7fd3594f5640 1 -- 172.21.15.83:0/3239662812 shutdown_connections 2024-04-17T10:38:24.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:24.924+0000 7fd3594f5640 1 -- 172.21.15.83:0/3239662812 wait complete. 2024-04-17T10:38:24.935 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:38:25.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.052+0000 7fa44da3f640 1 Processor -- start 2024-04-17T10:38:25.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.052+0000 7fa44da3f640 1 -- start start 2024-04-17T10:38:25.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.052+0000 7fa44da3f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa4480cfa20 0x7fa4480cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:25.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.052+0000 7fa44da3f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4480cf3f0 0x7fa4480b1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:25.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.052+0000 7fa44da3f640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4480cc330 con 0x7fa4480d2340 2024-04-17T10:38:25.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.052+0000 7fa44da3f640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4480cc940 con 0x7fa4480cfa20 2024-04-17T10:38:25.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.052+0000 7fa44da3f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4480d0340 con 0x7fa4480cf3f0 2024-04-17T10:38:25.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.052+0000 7fa446ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa4480cfa20 0x7fa4480cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:25.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.052+0000 7fa446ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa4480cfa20 0x7fa4480cfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48290/0 (socket says 172.21.15.83:48290) 2024-04-17T10:38:25.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.052+0000 7fa446ffd640 1 -- 172.21.15.83:0/2073924437 learned_addr learned my addr 172.21.15.83:0/2073924437 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:25.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.056+0000 7fa446ffd640 1 -- 172.21.15.83:0/2073924437 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4480cf3f0 msgr2=0x7fa4480b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:25.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.056+0000 7fa4467fc640 1 --2- 172.21.15.83:0/2073924437 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4480cf3f0 0x7fa4480b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:25.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.056+0000 7fa446ffd640 1 --2- 172.21.15.83:0/2073924437 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4480cf3f0 0x7fa4480b1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.056+0000 7fa446ffd640 1 -- 172.21.15.83:0/2073924437 >> v1:172.21.15.167:6789/0 conn(0x7fa4480d2340 legacy=0x7fa4480ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:25.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.056+0000 7fa446ffd640 1 -- 172.21.15.83:0/2073924437 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4480cec30 con 0x7fa4480cfa20 2024-04-17T10:38:25.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.056+0000 7fa4467fc640 1 --2- 172.21.15.83:0/2073924437 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4480cf3f0 0x7fa4480b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:25.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.056+0000 7fa446ffd640 1 --2- 172.21.15.83:0/2073924437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4480cfa20 0x7fa4480cfe00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa438009870 tx=0x7fa43801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=700d7906c8d54c35 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:25.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.056+0000 7fa445ffb640 1 -- 172.21.15.83:0/2073924437 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4380b0070 con 0x7fa4480cfa20 2024-04-17T10:38:25.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.056+0000 7fa445ffb640 1 -- 172.21.15.83:0/2073924437 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4380ab600 con 0x7fa4480cfa20 2024-04-17T10:38:25.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.056+0000 7fa445ffb640 1 -- 172.21.15.83:0/2073924437 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4380ab9d0 con 0x7fa4480cfa20 2024-04-17T10:38:25.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.056+0000 7fa44da3f640 1 -- 172.21.15.83:0/2073924437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4480cfa20 msgr2=0x7fa4480cfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:25.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.056+0000 7fa44da3f640 1 --2- 172.21.15.83:0/2073924437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4480cfa20 0x7fa4480cfe00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa438009870 tx=0x7fa43801c490 comp rx=0 tx=0).stop 2024-04-17T10:38:25.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.056+0000 7fa44da3f640 1 -- 172.21.15.83:0/2073924437 shutdown_connections 2024-04-17T10:38:25.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.056+0000 7fa44da3f640 1 --2- 172.21.15.83:0/2073924437 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4480cf3f0 0x7fa4480b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.056+0000 7fa44da3f640 1 --2- 172.21.15.83:0/2073924437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4480cfa20 0x7fa4480cfe00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.056+0000 7fa44da3f640 1 -- 172.21.15.83:0/2073924437 >> 172.21.15.83:0/2073924437 conn(0x7fa4480d1820 msgr2=0x7fa4480d1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:25.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.056+0000 7fa44da3f640 1 -- 172.21.15.83:0/2073924437 shutdown_connections 2024-04-17T10:38:25.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.056+0000 7fa44da3f640 1 -- 172.21.15.83:0/2073924437 wait complete. 2024-04-17T10:38:25.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.060+0000 7fa44da3f640 1 Processor -- start 2024-04-17T10:38:25.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.060+0000 7fa44da3f640 1 -- start start 2024-04-17T10:38:25.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.060+0000 7fa44da3f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4480cf3f0 0x7fa44814a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:25.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.060+0000 7fa44da3f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4480cfa20 0x7fa44815a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:25.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.060+0000 7fa44da3f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4480d2340 0x7fa4480b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:25.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.060+0000 7fa44da3f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4480b0f10 con 0x7fa4480d2340 2024-04-17T10:38:25.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.060+0000 7fa44da3f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4480b0d90 con 0x7fa4480cf3f0 2024-04-17T10:38:25.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.060+0000 7fa44da3f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa44815be50 con 0x7fa4480cfa20 2024-04-17T10:38:25.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.060+0000 7fa4477fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4480d2340 0x7fa4480b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:25.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.060+0000 7fa4477fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4480d2340 0x7fa4480b18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48304/0 (socket says 172.21.15.83:48304) 2024-04-17T10:38:25.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.060+0000 7fa4477fe640 1 -- 172.21.15.83:0/1919648434 learned_addr learned my addr 172.21.15.83:0/1919648434 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:25.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.060+0000 7fa4467fc640 1 --2- 172.21.15.83:0/1919648434 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4480cfa20 0x7fa44815a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:25.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.060+0000 7fa446ffd640 1 --2- 172.21.15.83:0/1919648434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4480cf3f0 0x7fa44814a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:25.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.060+0000 7fa4477fe640 1 -- 172.21.15.83:0/1919648434 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4480cfa20 msgr2=0x7fa44815a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:25.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.060+0000 7fa4477fe640 1 --2- 172.21.15.83:0/1919648434 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4480cfa20 0x7fa44815a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.060+0000 7fa4477fe640 1 -- 172.21.15.83:0/1919648434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4480cf3f0 msgr2=0x7fa44814a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:25.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.060+0000 7fa4477fe640 1 --2- 172.21.15.83:0/1919648434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4480cf3f0 0x7fa44814a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.060+0000 7fa4477fe640 1 -- 172.21.15.83:0/1919648434 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44815e3a0 con 0x7fa4480d2340 2024-04-17T10:38:25.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.060+0000 7fa446ffd640 1 --2- 172.21.15.83:0/1919648434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4480cf3f0 0x7fa44814a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:25.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.060+0000 7fa4477fe640 1 --2- 172.21.15.83:0/1919648434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4480d2340 0x7fa4480b18d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa44000c7b0 tx=0x7fa44000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:25.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.060+0000 7fa42ffff640 1 -- 172.21.15.83:0/1919648434 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa440017070 con 0x7fa4480d2340 2024-04-17T10:38:25.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.060+0000 7fa42ffff640 1 -- 172.21.15.83:0/1919648434 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa440003db0 con 0x7fa4480d2340 2024-04-17T10:38:25.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.060+0000 7fa44da3f640 1 -- 172.21.15.83:0/1919648434 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa44815e680 con 0x7fa4480d2340 2024-04-17T10:38:25.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.060+0000 7fa42ffff640 1 -- 172.21.15.83:0/1919648434 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4400052b0 con 0x7fa4480d2340 2024-04-17T10:38:25.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.060+0000 7fa44da3f640 1 -- 172.21.15.83:0/1919648434 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa448155f40 con 0x7fa4480d2340 2024-04-17T10:38:25.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.060+0000 7fa44da3f640 1 -- 172.21.15.83:0/1919648434 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa448074f90 con 0x7fa4480d2340 2024-04-17T10:38:25.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.060+0000 7fa42ffff640 1 -- 172.21.15.83:0/1919648434 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa44001a070 con 0x7fa4480d2340 2024-04-17T10:38:25.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.060+0000 7fa42ffff640 1 --2- 172.21.15.83:0/1919648434 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa434066f90 0x7fa434069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:25.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.060+0000 7fa42ffff640 1 -- 172.21.15.83:0/1919648434 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa440084650 con 0x7fa4480d2340 2024-04-17T10:38:25.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.060+0000 7fa446ffd640 1 --2- 172.21.15.83:0/1919648434 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa434066f90 0x7fa434069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:25.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.060+0000 7fa446ffd640 1 --2- 172.21.15.83:0/1919648434 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa434066f90 0x7fa434069450 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa4380ad000 tx=0x7fa438008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:25.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.064+0000 7fa42ffff640 1 -- 172.21.15.83:0/1919648434 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa4400556f0 con 0x7fa4480d2340 2024-04-17T10:38:25.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.224+0000 7fa44da3f640 1 -- 172.21.15.83:0/1919648434 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa448156110 con 0x7fa4480d2340 2024-04-17T10:38:25.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.224+0000 7fa42ffff640 1 -- 172.21.15.83:0/1919648434 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16533 (secure 0 0 0) 0x7fa440054ff0 con 0x7fa4480d2340 2024-04-17T10:38:25.221 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:25.222 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":20,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:38:13.690876+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:38:10.464490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T10:38:11.043134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:38:25.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.228+0000 7fa44da3f640 1 -- 172.21.15.83:0/1919648434 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa434066f90 msgr2=0x7fa434069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:25.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.228+0000 7fa44da3f640 1 --2- 172.21.15.83:0/1919648434 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa434066f90 0x7fa434069450 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa4380ad000 tx=0x7fa438008040 comp rx=0 tx=0).stop 2024-04-17T10:38:25.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.228+0000 7fa44da3f640 1 -- 172.21.15.83:0/1919648434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4480d2340 msgr2=0x7fa4480b18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:25.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.228+0000 7fa44da3f640 1 --2- 172.21.15.83:0/1919648434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4480d2340 0x7fa4480b18d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa44000c7b0 tx=0x7fa44000cc70 comp rx=0 tx=0).stop 2024-04-17T10:38:25.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.228+0000 7fa44da3f640 1 -- 172.21.15.83:0/1919648434 shutdown_connections 2024-04-17T10:38:25.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.228+0000 7fa44da3f640 1 --2- 172.21.15.83:0/1919648434 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa434066f90 0x7fa434069450 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.228+0000 7fa44da3f640 1 --2- 172.21.15.83:0/1919648434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4480d2340 0x7fa4480b18d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.228+0000 7fa44da3f640 1 --2- 172.21.15.83:0/1919648434 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4480cfa20 0x7fa44815a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.228+0000 7fa44da3f640 1 --2- 172.21.15.83:0/1919648434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4480cf3f0 0x7fa44814a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.228+0000 7fa44da3f640 1 -- 172.21.15.83:0/1919648434 >> 172.21.15.83:0/1919648434 conn(0x7fa4480d1820 msgr2=0x7fa4480b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:25.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.228+0000 7fa44da3f640 1 -- 172.21.15.83:0/1919648434 shutdown_connections 2024-04-17T10:38:25.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.228+0000 7fa44da3f640 1 -- 172.21.15.83:0/1919648434 wait complete. 2024-04-17T10:38:25.241 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:38:10.464490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:38:11.043134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:38:25.241 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:38:25.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.364+0000 7ff3f42df640 1 Processor -- start 2024-04-17T10:38:25.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.368+0000 7ff3f42df640 1 -- start start 2024-04-17T10:38:25.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.368+0000 7ff3f42df640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff3ec15b6e0 0x7ff3ec071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:25.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.368+0000 7ff3f42df640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff3ec15bac0 0x7ff3ec0cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:25.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.368+0000 7ff3f42df640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3ec06c1f0 con 0x7ff3ec070470 2024-04-17T10:38:25.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.368+0000 7ff3f42df640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3ec0cdd30 con 0x7ff3ec15b6e0 2024-04-17T10:38:25.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.368+0000 7ff3f42df640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3ec070a10 con 0x7ff3ec15bac0 2024-04-17T10:38:25.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.368+0000 7ff3f207b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff3ec15b6e0 0x7ff3ec071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:25.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.368+0000 7ff3f287c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff3ec070470 0x7ff3ec06ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40018/0 (socket says 172.21.15.83:40018) 2024-04-17T10:38:25.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.368+0000 7ff3f207b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff3ec15b6e0 0x7ff3ec071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48306/0 (socket says 172.21.15.83:48306) 2024-04-17T10:38:25.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.368+0000 7ff3f287c640 1 -- 172.21.15.83:0/849105313 learned_addr learned my addr 172.21.15.83:0/849105313 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:25.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.368+0000 7ff3f187a640 1 --2- 172.21.15.83:0/849105313 >> v2:172.21.15.167:3300/0 conn(0x7ff3ec15bac0 0x7ff3ec0cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:25.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.368+0000 7ff3f207b640 1 -- 172.21.15.83:0/849105313 >> v2:172.21.15.167:3300/0 conn(0x7ff3ec15bac0 msgr2=0x7ff3ec0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:25.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.368+0000 7ff3f207b640 1 --2- 172.21.15.83:0/849105313 >> v2:172.21.15.167:3300/0 conn(0x7ff3ec15bac0 0x7ff3ec0cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.368+0000 7ff3f207b640 1 -- 172.21.15.83:0/849105313 >> v1:172.21.15.83:6789/0 conn(0x7ff3ec070470 legacy=0x7ff3ec06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:25.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.368+0000 7ff3f1079640 1 -- 172.21.15.83:0/849105313 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1060692124 0 0) 0x7ff3ec06c1f0 con 0x7ff3ec070470 2024-04-17T10:38:25.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.368+0000 7ff3f207b640 1 -- 172.21.15.83:0/849105313 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec070cf0 con 0x7ff3ec15b6e0 2024-04-17T10:38:25.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.368+0000 7ff3f187a640 1 --2- 172.21.15.83:0/849105313 >> v2:172.21.15.167:3300/0 conn(0x7ff3ec15bac0 0x7ff3ec0cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:25.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.368+0000 7ff3f207b640 1 --2- 172.21.15.83:0/849105313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3ec15b6e0 0x7ff3ec071cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec071c50 tx=0x7ff3e001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42221d9112ff0959 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:25.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f1079640 1 -- 172.21.15.83:0/849105313 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e00b0070 con 0x7ff3ec15b6e0 2024-04-17T10:38:25.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f1079640 1 -- 172.21.15.83:0/849105313 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e00ab600 con 0x7ff3ec15b6e0 2024-04-17T10:38:25.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f1079640 1 -- 172.21.15.83:0/849105313 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e00ab9d0 con 0x7ff3ec15b6e0 2024-04-17T10:38:25.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f42df640 1 -- 172.21.15.83:0/849105313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3ec15b6e0 msgr2=0x7ff3ec071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:25.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f42df640 1 --2- 172.21.15.83:0/849105313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3ec15b6e0 0x7ff3ec071cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec071c50 tx=0x7ff3e001c480 comp rx=0 tx=0).stop 2024-04-17T10:38:25.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f42df640 1 -- 172.21.15.83:0/849105313 shutdown_connections 2024-04-17T10:38:25.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f42df640 1 --2- 172.21.15.83:0/849105313 >> v2:172.21.15.167:3300/0 conn(0x7ff3ec15bac0 0x7ff3ec0cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f42df640 1 --2- 172.21.15.83:0/849105313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3ec15b6e0 0x7ff3ec071cc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f42df640 1 -- 172.21.15.83:0/849105313 >> 172.21.15.83:0/849105313 conn(0x7ff3ec0b15b0 msgr2=0x7ff3ec0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:25.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f42df640 1 -- 172.21.15.83:0/849105313 shutdown_connections 2024-04-17T10:38:25.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f42df640 1 -- 172.21.15.83:0/849105313 wait complete. 2024-04-17T10:38:25.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f42df640 1 Processor -- start 2024-04-17T10:38:25.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f42df640 1 -- start start 2024-04-17T10:38:25.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f42df640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3ec070470 0x7ff3ec137e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:25.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f42df640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3ec15b6e0 0x7ff3ec14a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:25.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f42df640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3ec15bac0 0x7ff3ec154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f42df640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec06cee0 con 0x7ff3ec15b6e0 2024-04-17T10:38:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f42df640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec06b730 con 0x7ff3ec15bac0 2024-04-17T10:38:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f42df640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec06d060 con 0x7ff3ec070470 2024-04-17T10:38:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f187a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3ec15b6e0 0x7ff3ec14a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f187a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3ec15b6e0 0x7ff3ec14a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48318/0 (socket says 172.21.15.83:48318) 2024-04-17T10:38:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f187a640 1 -- 172.21.15.83:0/1746915214 learned_addr learned my addr 172.21.15.83:0/1746915214 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f207b640 1 --2- 172.21.15.83:0/1746915214 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3ec070470 0x7ff3ec137e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f287c640 1 --2- 172.21.15.83:0/1746915214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3ec15bac0 0x7ff3ec154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f187a640 1 -- 172.21.15.83:0/1746915214 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3ec070470 msgr2=0x7ff3ec137e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f187a640 1 --2- 172.21.15.83:0/1746915214 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3ec070470 0x7ff3ec137e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f187a640 1 -- 172.21.15.83:0/1746915214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3ec15bac0 msgr2=0x7ff3ec154360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:25.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f187a640 1 --2- 172.21.15.83:0/1746915214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3ec15bac0 0x7ff3ec154360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f187a640 1 -- 172.21.15.83:0/1746915214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec0b0cd0 con 0x7ff3ec15b6e0 2024-04-17T10:38:25.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f287c640 1 --2- 172.21.15.83:0/1746915214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3ec15bac0 0x7ff3ec154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:25.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f187a640 1 --2- 172.21.15.83:0/1746915214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3ec15b6e0 0x7ff3ec14a830 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800c7f0 tx=0x7ff3e800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:25.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3deffd640 1 -- 172.21.15.83:0/1746915214 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8015070 con 0x7ff3ec15b6e0 2024-04-17T10:38:25.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3deffd640 1 -- 172.21.15.83:0/1746915214 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e8007820 con 0x7ff3ec15b6e0 2024-04-17T10:38:25.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f42df640 1 -- 172.21.15.83:0/1746915214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ec14b5c0 con 0x7ff3ec15b6e0 2024-04-17T10:38:25.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3deffd640 1 -- 172.21.15.83:0/1746915214 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8007c10 con 0x7ff3ec15b6e0 2024-04-17T10:38:25.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.372+0000 7ff3f42df640 1 -- 172.21.15.83:0/1746915214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ec14be60 con 0x7ff3ec15b6e0 2024-04-17T10:38:25.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.376+0000 7ff3f42df640 1 -- 172.21.15.83:0/1746915214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3ec074f90 con 0x7ff3ec15b6e0 2024-04-17T10:38:25.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.376+0000 7ff3deffd640 1 -- 172.21.15.83:0/1746915214 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff3e8018070 con 0x7ff3ec15b6e0 2024-04-17T10:38:25.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.376+0000 7ff3deffd640 1 --2- 172.21.15.83:0/1746915214 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff3d0066f60 0x7ff3d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:25.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.376+0000 7ff3deffd640 1 -- 172.21.15.83:0/1746915214 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff3e8085240 con 0x7ff3ec15b6e0 2024-04-17T10:38:25.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.376+0000 7ff3f207b640 1 --2- 172.21.15.83:0/1746915214 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff3d0066f60 0x7ff3d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:25.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.376+0000 7ff3f207b640 1 --2- 172.21.15.83:0/1746915214 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff3d0066f60 0x7ff3d0069420 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff3e00ad000 tx=0x7ff3e00a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:25.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.380+0000 7ff3deffd640 1 -- 172.21.15.83:0/1746915214 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff3e80561b0 con 0x7ff3ec15b6e0 2024-04-17T10:38:25.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.536+0000 7ff3f42df640 1 -- 172.21.15.83:0/1746915214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff3ec0b1c70 con 0x7ff3ec15b6e0 2024-04-17T10:38:25.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.536+0000 7ff3deffd640 1 -- 172.21.15.83:0/1746915214 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff3e8055ab0 con 0x7ff3ec15b6e0 2024-04-17T10:38:25.532 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:38:25.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.536+0000 7ff3f42df640 1 -- 172.21.15.83:0/1746915214 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff3d0066f60 msgr2=0x7ff3d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:25.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.536+0000 7ff3f42df640 1 --2- 172.21.15.83:0/1746915214 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff3d0066f60 0x7ff3d0069420 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff3e00ad000 tx=0x7ff3e00a6000 comp rx=0 tx=0).stop 2024-04-17T10:38:25.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.536+0000 7ff3f42df640 1 -- 172.21.15.83:0/1746915214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3ec15b6e0 msgr2=0x7ff3ec14a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:25.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.536+0000 7ff3f42df640 1 --2- 172.21.15.83:0/1746915214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3ec15b6e0 0x7ff3ec14a830 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800c7f0 tx=0x7ff3e800ccb0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.536+0000 7ff3f42df640 1 -- 172.21.15.83:0/1746915214 shutdown_connections 2024-04-17T10:38:25.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.536+0000 7ff3f42df640 1 --2- 172.21.15.83:0/1746915214 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff3d0066f60 0x7ff3d0069420 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.536+0000 7ff3f42df640 1 --2- 172.21.15.83:0/1746915214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3ec15bac0 0x7ff3ec154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.536+0000 7ff3f42df640 1 --2- 172.21.15.83:0/1746915214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3ec15b6e0 0x7ff3ec14a830 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.536+0000 7ff3f42df640 1 --2- 172.21.15.83:0/1746915214 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3ec070470 0x7ff3ec137e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.536+0000 7ff3f42df640 1 -- 172.21.15.83:0/1746915214 >> 172.21.15.83:0/1746915214 conn(0x7ff3ec0b15b0 msgr2=0x7ff3ec0b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:25.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.536+0000 7ff3f42df640 1 -- 172.21.15.83:0/1746915214 shutdown_connections 2024-04-17T10:38:25.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.536+0000 7ff3f42df640 1 -- 172.21.15.83:0/1746915214 wait complete. 2024-04-17T10:38:25.550 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:38:25.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.676+0000 7f0519b8a640 1 Processor -- start 2024-04-17T10:38:25.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.676+0000 7f0519b8a640 1 -- start start 2024-04-17T10:38:25.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.676+0000 7f0519b8a640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0514154670 0x7f05140aea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:25.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.676+0000 7f0519b8a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f051406cc80 0x7f0514070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:25.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.676+0000 7f0519b8a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05140b2ff0 con 0x7f05140b4090 2024-04-17T10:38:25.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.676+0000 7f0519b8a640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0514154a50 con 0x7f0514154670 2024-04-17T10:38:25.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.676+0000 7f0519b8a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05140b32d0 con 0x7f051406cc80 2024-04-17T10:38:25.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f050affd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f05140b4090 0x7f051406c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40026/0 (socket says 172.21.15.83:40026) 2024-04-17T10:38:25.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f050affd640 1 -- 172.21.15.83:0/304876353 learned_addr learned my addr 172.21.15.83:0/304876353 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:25.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f0513fff640 1 --2- 172.21.15.83:0/304876353 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f051406cc80 0x7f0514070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:25.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f05137fe640 1 --2- 172.21.15.83:0/304876353 >> v2:172.21.15.167:3300/0 conn(0x7f0514154670 0x7f05140aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:25.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f0512ffd640 1 -- 172.21.15.83:0/304876353 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3137016792 0 0) 0x7f05140b2ff0 con 0x7f05140b4090 2024-04-17T10:38:25.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f0512ffd640 1 -- 172.21.15.83:0/304876353 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0500004640 con 0x7f05140b4090 2024-04-17T10:38:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f0512ffd640 1 -- 172.21.15.83:0/304876353 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 648812120 0 0) 0x7f0500004640 con 0x7f05140b4090 2024-04-17T10:38:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f0512ffd640 1 -- 172.21.15.83:0/304876353 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f051406cc80 msgr2=0x7f0514070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f0512ffd640 1 --2- 172.21.15.83:0/304876353 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f051406cc80 0x7f0514070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f0512ffd640 1 -- 172.21.15.83:0/304876353 >> v2:172.21.15.167:3300/0 conn(0x7f0514154670 msgr2=0x7f05140aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f0512ffd640 1 --2- 172.21.15.83:0/304876353 >> v2:172.21.15.167:3300/0 conn(0x7f0514154670 0x7f05140aea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f0512ffd640 1 -- 172.21.15.83:0/304876353 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05140b35b0 con 0x7f05140b4090 2024-04-17T10:38:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f0512ffd640 1 -- 172.21.15.83:0/304876353 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f04fc001680 con 0x7f05140b4090 2024-04-17T10:38:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f0512ffd640 1 -- 172.21.15.83:0/304876353 >> v1:172.21.15.83:6789/0 conn(0x7f05140b4090 legacy=0x7f051406c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f0512ffd640 1 --2- 172.21.15.83:0/304876353 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0500004b40 0x7f0500004f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f0512ffd640 1 --2- 172.21.15.83:0/304876353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0500005920 0x7f0500005da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f0512ffd640 1 --2- 172.21.15.83:0/304876353 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f051406cc80 0x7f05000062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f0512ffd640 1 -- 172.21.15.83:0/304876353 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0514154a50 con 0x7f0500005920 2024-04-17T10:38:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f0512ffd640 1 -- 172.21.15.83:0/304876353 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05140b32d0 con 0x7f0500004b40 2024-04-17T10:38:25.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f0512ffd640 1 -- 172.21.15.83:0/304876353 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f050000cae0 con 0x7f051406cc80 2024-04-17T10:38:25.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f050affd640 1 --2- 172.21.15.83:0/304876353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0500005920 0x7f0500005da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:25.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f0513fff640 1 --2- 172.21.15.83:0/304876353 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f051406cc80 0x7f05000062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:25.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f05137fe640 1 --2- 172.21.15.83:0/304876353 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0500004b40 0x7f0500004f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:25.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f050affd640 1 -- 172.21.15.83:0/304876353 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f051406cc80 msgr2=0x7f05000062e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:25.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f050affd640 1 --2- 172.21.15.83:0/304876353 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f051406cc80 0x7f05000062e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f050affd640 1 -- 172.21.15.83:0/304876353 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0500004b40 msgr2=0x7f0500004f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:25.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f050affd640 1 --2- 172.21.15.83:0/304876353 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0500004b40 0x7f0500004f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f050affd640 1 -- 172.21.15.83:0/304876353 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0500004640 con 0x7f0500005920 2024-04-17T10:38:25.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f05137fe640 1 --2- 172.21.15.83:0/304876353 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0500004b40 0x7f0500004f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:25.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f050affd640 1 --2- 172.21.15.83:0/304876353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0500005920 0x7f0500005da0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f04fc00c2f0 tx=0x7f04fc01f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e9bfc91cb69a70 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:25.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f0512ffd640 1 -- 172.21.15.83:0/304876353 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04fc007d60 con 0x7f0500005920 2024-04-17T10:38:25.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f0512ffd640 1 -- 172.21.15.83:0/304876353 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04fc005b00 con 0x7f0500005920 2024-04-17T10:38:25.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f0512ffd640 1 -- 172.21.15.83:0/304876353 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04fc007070 con 0x7f0500005920 2024-04-17T10:38:25.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f0519b8a640 1 -- 172.21.15.83:0/304876353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0500005920 msgr2=0x7f0500005da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:25.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f0519b8a640 1 --2- 172.21.15.83:0/304876353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0500005920 0x7f0500005da0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f04fc00c2f0 tx=0x7f04fc01f490 comp rx=0 tx=0).stop 2024-04-17T10:38:25.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f0513fff640 1 -- 172.21.15.83:0/304876353 reap_dead start 2024-04-17T10:38:25.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f0519b8a640 1 -- 172.21.15.83:0/304876353 shutdown_connections 2024-04-17T10:38:25.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.680+0000 7f0519b8a640 1 -- 172.21.15.83:0/304876353 >> 172.21.15.83:0/304876353 conn(0x7f05140b15b0 msgr2=0x7f05140b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:25.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.684+0000 7f0519b8a640 1 -- 172.21.15.83:0/304876353 shutdown_connections 2024-04-17T10:38:25.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.684+0000 7f0519b8a640 1 -- 172.21.15.83:0/304876353 wait complete. 2024-04-17T10:38:25.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.684+0000 7f0519b8a640 1 Processor -- start 2024-04-17T10:38:25.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.684+0000 7f0519b8a640 1 -- start start 2024-04-17T10:38:25.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.684+0000 7f0519b8a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0500005920 0x7f051414f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:25.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.684+0000 7f0519b8a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05141e8560 0x7f05141e5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:25.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.684+0000 7f0519b8a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f051414aa80 0x7f05140af570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:25.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.684+0000 7f0519b8a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051415b3c0 con 0x7f051414aa80 2024-04-17T10:38:25.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.684+0000 7f0519b8a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05140b3b70 con 0x7f05141e8560 2024-04-17T10:38:25.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.684+0000 7f0519b8a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05140aef90 con 0x7f0500005920 2024-04-17T10:38:25.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.684+0000 7f0513fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f051414aa80 0x7f05140af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:25.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.684+0000 7f0513fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f051414aa80 0x7f05140af570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48342/0 (socket says 172.21.15.83:48342) 2024-04-17T10:38:25.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.684+0000 7f0513fff640 1 -- 172.21.15.83:0/3016312054 learned_addr learned my addr 172.21.15.83:0/3016312054 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:25.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.684+0000 7f05137fe640 1 --2- 172.21.15.83:0/3016312054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0500005920 0x7f051414f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:25.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.684+0000 7f0513fff640 1 -- 172.21.15.83:0/3016312054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0500005920 msgr2=0x7f051414f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:25.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.684+0000 7f0513fff640 1 --2- 172.21.15.83:0/3016312054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0500005920 0x7f051414f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.684+0000 7f050affd640 1 --2- 172.21.15.83:0/3016312054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05141e8560 0x7f05141e5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:25.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.684+0000 7f0513fff640 1 -- 172.21.15.83:0/3016312054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05141e8560 msgr2=0x7f05141e5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:25.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.684+0000 7f0513fff640 1 --2- 172.21.15.83:0/3016312054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05141e8560 0x7f05141e5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.684+0000 7f0513fff640 1 -- 172.21.15.83:0/3016312054 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05141e5680 con 0x7f051414aa80 2024-04-17T10:38:25.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.684+0000 7f05137fe640 1 --2- 172.21.15.83:0/3016312054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0500005920 0x7f051414f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:38:25.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.684+0000 7f0513fff640 1 --2- 172.21.15.83:0/3016312054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f051414aa80 0x7f05140af570 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f05140b4dd0 tx=0x7f050c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:25.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.684+0000 7f0510ff9640 1 -- 172.21.15.83:0/3016312054 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c011e40 con 0x7f051414aa80 2024-04-17T10:38:25.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.684+0000 7f0510ff9640 1 -- 172.21.15.83:0/3016312054 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f050c01b070 con 0x7f051414aa80 2024-04-17T10:38:25.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.684+0000 7f0519b8a640 1 -- 172.21.15.83:0/3016312054 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f051414f950 con 0x7f051414aa80 2024-04-17T10:38:25.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.684+0000 7f0510ff9640 1 -- 172.21.15.83:0/3016312054 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c0039a0 con 0x7f051414aa80 2024-04-17T10:38:25.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.684+0000 7f0519b8a640 1 -- 172.21.15.83:0/3016312054 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05141e5d50 con 0x7f051414aa80 2024-04-17T10:38:25.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.684+0000 7f0519b8a640 1 -- 172.21.15.83:0/3016312054 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0514074f90 con 0x7f051414aa80 2024-04-17T10:38:25.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.688+0000 7f0510ff9640 1 -- 172.21.15.83:0/3016312054 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f050c003b40 con 0x7f051414aa80 2024-04-17T10:38:25.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.688+0000 7f0510ff9640 1 --2- 172.21.15.83:0/3016312054 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f04f4066f60 0x7f04f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:25.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.688+0000 7f0510ff9640 1 -- 172.21.15.83:0/3016312054 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f050c086480 con 0x7f051414aa80 2024-04-17T10:38:25.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.688+0000 7f05137fe640 1 --2- 172.21.15.83:0/3016312054 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f04f4066f60 0x7f04f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:25.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.688+0000 7f05137fe640 1 --2- 172.21.15.83:0/3016312054 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f04f4066f60 0x7f04f4069420 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0504009be0 tx=0x7f0504008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:25.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.692+0000 7f0510ff9640 1 -- 172.21.15.83:0/3016312054 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f050c057570 con 0x7f051414aa80 2024-04-17T10:38:25.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.832+0000 7f0519b8a640 1 -- 172.21.15.83:0/3016312054 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f05140b2a00 con 0x7f051414aa80 2024-04-17T10:38:25.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.836+0000 7f0510ff9640 1 -- 172.21.15.83:0/3016312054 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f050c056e70 con 0x7f051414aa80 2024-04-17T10:38:25.831 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:38:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.836+0000 7f0519b8a640 1 -- 172.21.15.83:0/3016312054 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f04f4066f60 msgr2=0x7f04f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.836+0000 7f0519b8a640 1 --2- 172.21.15.83:0/3016312054 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f04f4066f60 0x7f04f4069420 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0504009be0 tx=0x7f0504008040 comp rx=0 tx=0).stop 2024-04-17T10:38:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.836+0000 7f0519b8a640 1 -- 172.21.15.83:0/3016312054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f051414aa80 msgr2=0x7f05140af570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.836+0000 7f0519b8a640 1 --2- 172.21.15.83:0/3016312054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f051414aa80 0x7f05140af570 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f05140b4dd0 tx=0x7f050c0115a0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.836+0000 7f0519b8a640 1 -- 172.21.15.83:0/3016312054 shutdown_connections 2024-04-17T10:38:25.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.836+0000 7f0519b8a640 1 --2- 172.21.15.83:0/3016312054 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f04f4066f60 0x7f04f4069420 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.836+0000 7f0519b8a640 1 --2- 172.21.15.83:0/3016312054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f051414aa80 0x7f05140af570 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.836+0000 7f0519b8a640 1 --2- 172.21.15.83:0/3016312054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05141e8560 0x7f05141e5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.836+0000 7f0519b8a640 1 --2- 172.21.15.83:0/3016312054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0500005920 0x7f051414f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.836+0000 7f0519b8a640 1 -- 172.21.15.83:0/3016312054 >> 172.21.15.83:0/3016312054 conn(0x7f05140b15b0 msgr2=0x7f051415c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:25.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.836+0000 7f0519b8a640 1 -- 172.21.15.83:0/3016312054 shutdown_connections 2024-04-17T10:38:25.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.836+0000 7f0519b8a640 1 -- 172.21.15.83:0/3016312054 wait complete. 2024-04-17T10:38:25.848 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:38:25.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.968+0000 7f2361257640 1 Processor -- start 2024-04-17T10:38:25.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.972+0000 7f2361257640 1 -- start start 2024-04-17T10:38:25.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.972+0000 7f2361257640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f235c15b6e0 0x7f235c071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:25.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.972+0000 7f2361257640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f235c15bac0 0x7f235c0cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:25.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.972+0000 7f2361257640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f235c06c1f0 con 0x7f235c070470 2024-04-17T10:38:25.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.972+0000 7f2361257640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f235c0cdd30 con 0x7f235c15bac0 2024-04-17T10:38:25.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.972+0000 7f2361257640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f235c070a10 con 0x7f235c15b6e0 2024-04-17T10:38:25.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.972+0000 7f235259c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f235c15bac0 0x7f235c0cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:25.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.972+0000 7f235259c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f235c15bac0 0x7f235c0cd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48348/0 (socket says 172.21.15.83:48348) 2024-04-17T10:38:25.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.972+0000 7f235259c640 1 -- 172.21.15.83:0/3163782363 learned_addr learned my addr 172.21.15.83:0/3163782363 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:25.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.972+0000 7f235ad9d640 1 --2- 172.21.15.83:0/3163782363 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f235c15b6e0 0x7f235c071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:25.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.972+0000 7f235259c640 1 -- 172.21.15.83:0/3163782363 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f235c15b6e0 msgr2=0x7f235c071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:25.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.972+0000 7f235259c640 1 --2- 172.21.15.83:0/3163782363 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f235c15b6e0 0x7f235c071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.972+0000 7f235259c640 1 -- 172.21.15.83:0/3163782363 >> v1:172.21.15.167:6789/0 conn(0x7f235c070470 legacy=0x7f235c06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:25.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.972+0000 7f235259c640 1 -- 172.21.15.83:0/3163782363 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235c070cf0 con 0x7f235c15bac0 2024-04-17T10:38:25.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.972+0000 7f235ad9d640 1 --2- 172.21.15.83:0/3163782363 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f235c15b6e0 0x7f235c071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:25.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.972+0000 7f235259c640 1 --2- 172.21.15.83:0/3163782363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f235c15bac0 0x7f235c0cd7f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f235c0b49f0 tx=0x7f234401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd3114494dc70f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:25.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f235a59c640 1 -- 172.21.15.83:0/3163782363 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23440b0070 con 0x7f235c15bac0 2024-04-17T10:38:25.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f235a59c640 1 -- 172.21.15.83:0/3163782363 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23440ab600 con 0x7f235c15bac0 2024-04-17T10:38:25.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f235a59c640 1 -- 172.21.15.83:0/3163782363 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23440ab9d0 con 0x7f235c15bac0 2024-04-17T10:38:25.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f2361257640 1 -- 172.21.15.83:0/3163782363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f235c15bac0 msgr2=0x7f235c0cd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:25.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f2361257640 1 --2- 172.21.15.83:0/3163782363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f235c15bac0 0x7f235c0cd7f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f235c0b49f0 tx=0x7f234401c4e0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f2361257640 1 -- 172.21.15.83:0/3163782363 shutdown_connections 2024-04-17T10:38:25.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f2361257640 1 --2- 172.21.15.83:0/3163782363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f235c15bac0 0x7f235c0cd7f0 secure :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f235c0b49f0 tx=0x7f234401c4e0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f2361257640 1 --2- 172.21.15.83:0/3163782363 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f235c15b6e0 0x7f235c071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f2361257640 1 -- 172.21.15.83:0/3163782363 >> 172.21.15.83:0/3163782363 conn(0x7f235c0b15b0 msgr2=0x7f235c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:25.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f2361257640 1 -- 172.21.15.83:0/3163782363 shutdown_connections 2024-04-17T10:38:25.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f2361257640 1 -- 172.21.15.83:0/3163782363 wait complete. 2024-04-17T10:38:25.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f2361257640 1 Processor -- start 2024-04-17T10:38:25.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f2361257640 1 -- start start 2024-04-17T10:38:25.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f2361257640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f235c070470 0x7f235c1529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:25.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f2361257640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f235c15b6e0 0x7f235c137710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:25.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f2361257640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f235c14ad20 0x7f235c137c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:25.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f2361257640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c06cee0 con 0x7f235c070470 2024-04-17T10:38:25.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f2361257640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c06b730 con 0x7f235c14ad20 2024-04-17T10:38:25.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f2361257640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f235c06d060 con 0x7f235c15b6e0 2024-04-17T10:38:25.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f235ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f235c070470 0x7f235c1529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:25.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f235ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f235c070470 0x7f235c1529c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48362/0 (socket says 172.21.15.83:48362) 2024-04-17T10:38:25.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f235ad9d640 1 -- 172.21.15.83:0/4113822527 learned_addr learned my addr 172.21.15.83:0/4113822527 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:25.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f235259c640 1 --2- 172.21.15.83:0/4113822527 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f235c15b6e0 0x7f235c137710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:25.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f235ad9d640 1 -- 172.21.15.83:0/4113822527 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f235c15b6e0 msgr2=0x7f235c137710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:25.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f235b59e640 1 --2- 172.21.15.83:0/4113822527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f235c14ad20 0x7f235c137c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:25.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f235ad9d640 1 --2- 172.21.15.83:0/4113822527 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f235c15b6e0 0x7f235c137710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f235ad9d640 1 -- 172.21.15.83:0/4113822527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f235c14ad20 msgr2=0x7f235c137c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:25.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f235ad9d640 1 --2- 172.21.15.83:0/4113822527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f235c14ad20 0x7f235c137c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:25.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f235ad9d640 1 -- 172.21.15.83:0/4113822527 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235c135030 con 0x7f235c070470 2024-04-17T10:38:25.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f235259c640 1 --2- 172.21.15.83:0/4113822527 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f235c15b6e0 0x7f235c137710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:25.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f235ad9d640 1 --2- 172.21.15.83:0/4113822527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f235c070470 0x7f235c1529c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f234c00c7f0 tx=0x7f234c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:25.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f2353fff640 1 -- 172.21.15.83:0/4113822527 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c015070 con 0x7f235c070470 2024-04-17T10:38:25.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f2353fff640 1 -- 172.21.15.83:0/4113822527 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234c007820 con 0x7f235c070470 2024-04-17T10:38:25.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f2361257640 1 -- 172.21.15.83:0/4113822527 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f235c14a240 con 0x7f235c070470 2024-04-17T10:38:25.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f2353fff640 1 -- 172.21.15.83:0/4113822527 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c007c10 con 0x7f235c070470 2024-04-17T10:38:25.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.976+0000 7f2361257640 1 -- 172.21.15.83:0/4113822527 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f235c14a800 con 0x7f235c070470 2024-04-17T10:38:25.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.980+0000 7f2361257640 1 -- 172.21.15.83:0/4113822527 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f235c074f90 con 0x7f235c070470 2024-04-17T10:38:25.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.980+0000 7f2353fff640 1 -- 172.21.15.83:0/4113822527 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f234c018070 con 0x7f235c070470 2024-04-17T10:38:25.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.980+0000 7f2353fff640 1 --2- 172.21.15.83:0/4113822527 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f233c066f60 0x7f233c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:25.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.980+0000 7f2353fff640 1 -- 172.21.15.83:0/4113822527 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f234c085140 con 0x7f235c070470 2024-04-17T10:38:25.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.980+0000 7f235259c640 1 --2- 172.21.15.83:0/4113822527 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f233c066f60 0x7f233c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:25.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.980+0000 7f235259c640 1 --2- 172.21.15.83:0/4113822527 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f233c066f60 0x7f233c069420 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f23440ad000 tx=0x7f2344008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:25.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:25.984+0000 7f2353fff640 1 -- 172.21.15.83:0/4113822527 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f234c056150 con 0x7f235c070470 2024-04-17T10:38:26.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.132+0000 7f2361257640 1 -- 172.21.15.83:0/4113822527 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f235c154c40 con 0x7f235c070470 2024-04-17T10:38:26.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.132+0000 7f2353fff640 1 -- 172.21.15.83:0/4113822527 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f234c055a50 con 0x7f235c070470 2024-04-17T10:38:26.128 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:38:26.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.132+0000 7f2361257640 1 -- 172.21.15.83:0/4113822527 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f233c066f60 msgr2=0x7f233c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:26.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.132+0000 7f2361257640 1 --2- 172.21.15.83:0/4113822527 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f233c066f60 0x7f233c069420 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f23440ad000 tx=0x7f2344008040 comp rx=0 tx=0).stop 2024-04-17T10:38:26.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.132+0000 7f2361257640 1 -- 172.21.15.83:0/4113822527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f235c070470 msgr2=0x7f235c1529c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:26.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.132+0000 7f2361257640 1 --2- 172.21.15.83:0/4113822527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f235c070470 0x7f235c1529c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f234c00c7f0 tx=0x7f234c00ccb0 comp rx=0 tx=0).stop 2024-04-17T10:38:26.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.132+0000 7f2361257640 1 -- 172.21.15.83:0/4113822527 shutdown_connections 2024-04-17T10:38:26.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.132+0000 7f2361257640 1 --2- 172.21.15.83:0/4113822527 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f233c066f60 0x7f233c069420 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:26.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.132+0000 7f2361257640 1 --2- 172.21.15.83:0/4113822527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f235c14ad20 0x7f235c137c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:26.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.132+0000 7f2361257640 1 --2- 172.21.15.83:0/4113822527 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f235c15b6e0 0x7f235c137710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:26.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.132+0000 7f2361257640 1 --2- 172.21.15.83:0/4113822527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f235c070470 0x7f235c1529c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:26.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.132+0000 7f2361257640 1 -- 172.21.15.83:0/4113822527 >> 172.21.15.83:0/4113822527 conn(0x7f235c0b15b0 msgr2=0x7f235c0cfd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:26.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.132+0000 7f2361257640 1 -- 172.21.15.83:0/4113822527 shutdown_connections 2024-04-17T10:38:26.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.132+0000 7f2361257640 1 -- 172.21.15.83:0/4113822527 wait complete. 2024-04-17T10:38:26.146 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:38:26.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.268+0000 7fc3f861e640 1 Processor -- start 2024-04-17T10:38:26.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.268+0000 7fc3f861e640 1 -- start start 2024-04-17T10:38:26.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.268+0000 7fc3f861e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc3f00cfbe0 0x7fc3f006c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:26.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.268+0000 7fc3f861e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3f006e320 con 0x7fc3f0072330 2024-04-17T10:38:26.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.268+0000 7fc3f861e640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3f006e040 con 0x7fc3f015ba30 2024-04-17T10:38:26.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.268+0000 7fc3f861e640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3f0071ea0 con 0x7fc3f00cfbe0 2024-04-17T10:38:26.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.268+0000 7fc3f5bb9640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc3f0072330 0x7fc3f00cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40034/0 (socket says 172.21.15.83:40034) 2024-04-17T10:38:26.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.268+0000 7fc3f5bb9640 1 -- 172.21.15.83:0/829105227 learned_addr learned my addr 172.21.15.83:0/829105227 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:26.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.268+0000 7fc3f6bbb640 1 --2- 172.21.15.83:0/829105227 >> v2:172.21.15.167:3300/0 conn(0x7fc3f00cfbe0 0x7fc3f006c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:26.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.268+0000 7fc3f53b8640 1 -- 172.21.15.83:0/829105227 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1099596470 0 0) 0x7fc3f006e320 con 0x7fc3f0072330 2024-04-17T10:38:26.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.268+0000 7fc3f53b8640 1 -- 172.21.15.83:0/829105227 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3dc003530 con 0x7fc3f0072330 2024-04-17T10:38:26.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.268+0000 7fc3f53b8640 1 -- 172.21.15.83:0/829105227 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 119666015 0 0) 0x7fc3dc003530 con 0x7fc3f0072330 2024-04-17T10:38:26.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.268+0000 7fc3f53b8640 1 -- 172.21.15.83:0/829105227 >> v2:172.21.15.167:3300/0 conn(0x7fc3f00cfbe0 msgr2=0x7fc3f006c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:26.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.268+0000 7fc3f53b8640 1 --2- 172.21.15.83:0/829105227 >> v2:172.21.15.167:3300/0 conn(0x7fc3f00cfbe0 0x7fc3f006c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:26.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.268+0000 7fc3f53b8640 1 -- 172.21.15.83:0/829105227 >> v1:172.21.15.167:6789/0 conn(0x7fc3f015ba30 legacy=0x7fc3f015c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:26.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.268+0000 7fc3f53b8640 1 -- 172.21.15.83:0/829105227 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f0071850 con 0x7fc3f0072330 2024-04-17T10:38:26.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.268+0000 7fc3f53b8640 1 -- 172.21.15.83:0/829105227 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fc3e00016a0 con 0x7fc3f0072330 2024-04-17T10:38:26.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.268+0000 7fc3f53b8640 1 -- 172.21.15.83:0/829105227 >> v1:172.21.15.83:6789/0 conn(0x7fc3f0072330 legacy=0x7fc3f00cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:26.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.268+0000 7fc3f53b8640 1 --2- 172.21.15.83:0/829105227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3dc003a10 0x7fc3dc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:26.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.268+0000 7fc3f53b8640 1 --2- 172.21.15.83:0/829105227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3dc004810 0x7fc3dc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:26.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.268+0000 7fc3f53b8640 1 --2- 172.21.15.83:0/829105227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3dc0052f0 0x7fc3dc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:26.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.268+0000 7fc3f53b8640 1 -- 172.21.15.83:0/829105227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f0071ea0 con 0x7fc3dc003a10 2024-04-17T10:38:26.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.268+0000 7fc3f53b8640 1 -- 172.21.15.83:0/829105227 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3dc00c210 con 0x7fc3dc0052f0 2024-04-17T10:38:26.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.268+0000 7fc3f53b8640 1 -- 172.21.15.83:0/829105227 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3dc00c410 con 0x7fc3dc004810 2024-04-17T10:38:26.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.268+0000 7fc3f63ba640 1 --2- 172.21.15.83:0/829105227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3dc003a10 0x7fc3dc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:26.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.268+0000 7fc3f5bb9640 1 --2- 172.21.15.83:0/829105227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3dc004810 0x7fc3dc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:26.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.268+0000 7fc3f63ba640 1 -- 172.21.15.83:0/829105227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3dc004810 msgr2=0x7fc3dc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:26.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.268+0000 7fc3f63ba640 1 --2- 172.21.15.83:0/829105227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3dc004810 0x7fc3dc004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:26.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.268+0000 7fc3f63ba640 1 -- 172.21.15.83:0/829105227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3dc0052f0 msgr2=0x7fc3dc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:26.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.268+0000 7fc3f63ba640 1 --2- 172.21.15.83:0/829105227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3dc0052f0 0x7fc3dc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:26.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.268+0000 7fc3f63ba640 1 -- 172.21.15.83:0/829105227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3dc003530 con 0x7fc3dc003a10 2024-04-17T10:38:26.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.268+0000 7fc3f6bbb640 1 -- 172.21.15.83:0/829105227 reap_dead start 2024-04-17T10:38:26.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.272+0000 7fc3f63ba640 1 --2- 172.21.15.83:0/829105227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3dc003a10 0x7fc3dc003e30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec001780 tx=0x7fc3ec01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1409562c920a138 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:26.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.272+0000 7fc3f53b8640 1 -- 172.21.15.83:0/829105227 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec0013a0 con 0x7fc3dc003a10 2024-04-17T10:38:26.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.272+0000 7fc3f53b8640 1 -- 172.21.15.83:0/829105227 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ec0052f0 con 0x7fc3dc003a10 2024-04-17T10:38:26.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.272+0000 7fc3f53b8640 1 -- 172.21.15.83:0/829105227 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec003840 con 0x7fc3dc003a10 2024-04-17T10:38:26.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.272+0000 7fc3f861e640 1 -- 172.21.15.83:0/829105227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3dc003a10 msgr2=0x7fc3dc003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:26.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.272+0000 7fc3f861e640 1 --2- 172.21.15.83:0/829105227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3dc003a10 0x7fc3dc003e30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec001780 tx=0x7fc3ec01c420 comp rx=0 tx=0).stop 2024-04-17T10:38:26.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.272+0000 7fc3f861e640 1 -- 172.21.15.83:0/829105227 shutdown_connections 2024-04-17T10:38:26.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.272+0000 7fc3f861e640 1 --2- 172.21.15.83:0/829105227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3dc003a10 0x7fc3dc003e30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:26.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.272+0000 7fc3f861e640 1 -- 172.21.15.83:0/829105227 >> 172.21.15.83:0/829105227 conn(0x7fc3f006abe0 msgr2=0x7fc3f006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:26.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.272+0000 7fc3f861e640 1 -- 172.21.15.83:0/829105227 shutdown_connections 2024-04-17T10:38:26.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.272+0000 7fc3f861e640 1 -- 172.21.15.83:0/829105227 wait complete. 2024-04-17T10:38:26.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.272+0000 7fc3f861e640 1 Processor -- start 2024-04-17T10:38:26.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.272+0000 7fc3f861e640 1 -- start start 2024-04-17T10:38:26.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.272+0000 7fc3f861e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3dc003a10 0x7fc3f01f7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:26.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.272+0000 7fc3f861e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3f00cee20 0x7fc3f0159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:26.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.272+0000 7fc3f861e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3f01f6d50 0x7fc3f00ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:26.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.272+0000 7fc3f861e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f00cfa60 con 0x7fc3dc003a10 2024-04-17T10:38:26.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.272+0000 7fc3f861e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f00cf8e0 con 0x7fc3f01f6d50 2024-04-17T10:38:26.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.272+0000 7fc3f861e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f015cd70 con 0x7fc3f00cee20 2024-04-17T10:38:26.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.276+0000 7fc3f63ba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3dc003a10 0x7fc3f01f7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:26.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.276+0000 7fc3f63ba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3dc003a10 0x7fc3f01f7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48386/0 (socket says 172.21.15.83:48386) 2024-04-17T10:38:26.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.276+0000 7fc3f63ba640 1 -- 172.21.15.83:0/3585676043 learned_addr learned my addr 172.21.15.83:0/3585676043 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:26.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.276+0000 7fc3f5bb9640 1 --2- 172.21.15.83:0/3585676043 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3f00cee20 0x7fc3f0159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:26.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.276+0000 7fc3f6bbb640 1 --2- 172.21.15.83:0/3585676043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3f01f6d50 0x7fc3f00ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:26.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.276+0000 7fc3f63ba640 1 -- 172.21.15.83:0/3585676043 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3f00cee20 msgr2=0x7fc3f0159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:26.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.276+0000 7fc3f63ba640 1 --2- 172.21.15.83:0/3585676043 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3f00cee20 0x7fc3f0159780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:26.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.276+0000 7fc3f63ba640 1 -- 172.21.15.83:0/3585676043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3f01f6d50 msgr2=0x7fc3f00ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:26.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.276+0000 7fc3f63ba640 1 --2- 172.21.15.83:0/3585676043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3f01f6d50 0x7fc3f00ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:26.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.276+0000 7fc3f63ba640 1 -- 172.21.15.83:0/3585676043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f01f75b0 con 0x7fc3dc003a10 2024-04-17T10:38:26.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.276+0000 7fc3f63ba640 1 --2- 172.21.15.83:0/3585676043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3dc003a10 0x7fc3f01f7ef0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec005040 tx=0x7fc3dc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:26.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.276+0000 7fc3e6ffd640 1 -- 172.21.15.83:0/3585676043 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec007a40 con 0x7fc3dc003a10 2024-04-17T10:38:26.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.276+0000 7fc3e6ffd640 1 -- 172.21.15.83:0/3585676043 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ec0ba070 con 0x7fc3dc003a10 2024-04-17T10:38:26.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.276+0000 7fc3e6ffd640 1 -- 172.21.15.83:0/3585676043 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec0054a0 con 0x7fc3dc003a10 2024-04-17T10:38:26.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.276+0000 7fc3f861e640 1 -- 172.21.15.83:0/3585676043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f01f7890 con 0x7fc3dc003a10 2024-04-17T10:38:26.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.276+0000 7fc3f861e640 1 -- 172.21.15.83:0/3585676043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f01f71d0 con 0x7fc3dc003a10 2024-04-17T10:38:26.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.276+0000 7fc3f861e640 1 -- 172.21.15.83:0/3585676043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f0074f90 con 0x7fc3dc003a10 2024-04-17T10:38:26.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.276+0000 7fc3e6ffd640 1 -- 172.21.15.83:0/3585676043 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc3ec007030 con 0x7fc3dc003a10 2024-04-17T10:38:26.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.276+0000 7fc3e6ffd640 1 --2- 172.21.15.83:0/3585676043 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc3d4066f90 0x7fc3d4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:26.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.276+0000 7fc3e6ffd640 1 -- 172.21.15.83:0/3585676043 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc3ec11ecc0 con 0x7fc3dc003a10 2024-04-17T10:38:26.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.276+0000 7fc3f5bb9640 1 --2- 172.21.15.83:0/3585676043 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc3d4066f90 0x7fc3d4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:26.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.280+0000 7fc3f5bb9640 1 --2- 172.21.15.83:0/3585676043 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc3d4066f90 0x7fc3d4069450 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc3f006b920 tx=0x7fc3e0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:26.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.284+0000 7fc3e6ffd640 1 -- 172.21.15.83:0/3585676043 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc3ec016130 con 0x7fc3dc003a10 2024-04-17T10:38:26.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.476+0000 7fc3f861e640 1 -- 172.21.15.83:0/3585676043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3f015c550 con 0x7fc3dc003a10 2024-04-17T10:38:26.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.476+0000 7fc3e6ffd640 1 -- 172.21.15.83:0/3585676043 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4938 (secure 0 0 0) 0x7fc3ec0efdf0 con 0x7fc3dc003a10 2024-04-17T10:38:26.472 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:26.472 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2884469968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2884469968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2884469968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2919571009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2919571009},{"type":"v1","addr":"172.21.15.83:6836","nonce":2919571009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1635176834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1635176834},{"type":"v1","addr":"172.21.15.83:6837","nonce":1635176834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:38:12.032695+0000","modified":"2024-04-17T10:38:14.064978+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6835/3871588351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3871588351},{"type":"v1","addr":"172.21.15.167:6835","nonce":3871588351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-17T10:38:26.472 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 6 2024-04-17T10:38:26.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.476+0000 7fc3f861e640 1 -- 172.21.15.83:0/3585676043 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc3d4066f90 msgr2=0x7fc3d4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:26.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.476+0000 7fc3f861e640 1 --2- 172.21.15.83:0/3585676043 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc3d4066f90 0x7fc3d4069450 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc3f006b920 tx=0x7fc3e0009040 comp rx=0 tx=0).stop 2024-04-17T10:38:26.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.476+0000 7fc3f861e640 1 -- 172.21.15.83:0/3585676043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3dc003a10 msgr2=0x7fc3f01f7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:26.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.476+0000 7fc3f861e640 1 --2- 172.21.15.83:0/3585676043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3dc003a10 0x7fc3f01f7ef0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec005040 tx=0x7fc3dc005840 comp rx=0 tx=0).stop 2024-04-17T10:38:26.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.476+0000 7fc3f861e640 1 -- 172.21.15.83:0/3585676043 shutdown_connections 2024-04-17T10:38:26.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.476+0000 7fc3f861e640 1 --2- 172.21.15.83:0/3585676043 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc3d4066f90 0x7fc3d4069450 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:26.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.476+0000 7fc3f861e640 1 --2- 172.21.15.83:0/3585676043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3f01f6d50 0x7fc3f00ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:26.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.476+0000 7fc3f861e640 1 --2- 172.21.15.83:0/3585676043 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3f00cee20 0x7fc3f0159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:26.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.476+0000 7fc3f861e640 1 --2- 172.21.15.83:0/3585676043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3dc003a10 0x7fc3f01f7ef0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:26.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.476+0000 7fc3f861e640 1 -- 172.21.15.83:0/3585676043 >> 172.21.15.83:0/3585676043 conn(0x7fc3f006abe0 msgr2=0x7fc3f0070470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:26.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.476+0000 7fc3f861e640 1 -- 172.21.15.83:0/3585676043 shutdown_connections 2024-04-17T10:38:26.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.476+0000 7fc3f861e640 1 -- 172.21.15.83:0/3585676043 wait complete. 2024-04-17T10:38:26.489 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:38:26.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.596+0000 7f5149cb6640 1 Processor -- start 2024-04-17T10:38:26.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.596+0000 7f5149cb6640 1 -- start start 2024-04-17T10:38:26.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.596+0000 7f5149cb6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51440ce980 0x7f51440b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:26.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f5149cb6640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5144162a00 con 0x7f514415dfa0 2024-04-17T10:38:26.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f5149cb6640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5144161e80 con 0x7f51440cee00 2024-04-17T10:38:26.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f5149cb6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5144162160 con 0x7f51440ce980 2024-04-17T10:38:26.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f51437fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f514415dfa0 0x7f51440b6c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40042/0 (socket says 172.21.15.83:40042) 2024-04-17T10:38:26.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f51437fe640 1 -- 172.21.15.83:0/2880228823 learned_addr learned my addr 172.21.15.83:0/2880228823 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:26.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f51427fc640 1 -- 172.21.15.83:0/2880228823 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3859688263 0 0) 0x7f5144162a00 con 0x7f514415dfa0 2024-04-17T10:38:26.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f51427fc640 1 -- 172.21.15.83:0/2880228823 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f512c003530 con 0x7f514415dfa0 2024-04-17T10:38:26.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f5142ffd640 1 --2- 172.21.15.83:0/2880228823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51440ce980 0x7f51440b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:26.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f51427fc640 1 -- 172.21.15.83:0/2880228823 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3741236 0 0) 0x7f512c003530 con 0x7f514415dfa0 2024-04-17T10:38:26.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f51427fc640 1 -- 172.21.15.83:0/2880228823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51440ce980 msgr2=0x7f51440b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:26.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f51427fc640 1 --2- 172.21.15.83:0/2880228823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51440ce980 0x7f51440b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:26.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f51427fc640 1 -- 172.21.15.83:0/2880228823 >> v1:172.21.15.167:6789/0 conn(0x7f51440cee00 legacy=0x7f51440d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:26.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f51427fc640 1 -- 172.21.15.83:0/2880228823 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5144162440 con 0x7f514415dfa0 2024-04-17T10:38:26.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f51427fc640 1 -- 172.21.15.83:0/2880228823 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f5134001680 con 0x7f514415dfa0 2024-04-17T10:38:26.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f5142ffd640 1 --2- 172.21.15.83:0/2880228823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51440ce980 0x7f51440b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:38:26.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f51427fc640 1 -- 172.21.15.83:0/2880228823 >> v1:172.21.15.83:6789/0 conn(0x7f514415dfa0 legacy=0x7f51440b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:26.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f51427fc640 1 --2- 172.21.15.83:0/2880228823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f512c0039f0 0x7f512c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:26.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f51427fc640 1 --2- 172.21.15.83:0/2880228823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51440ce980 0x7f512c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:26.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f51427fc640 1 --2- 172.21.15.83:0/2880228823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f512c004d50 0x7f512c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:26.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f51427fc640 1 -- 172.21.15.83:0/2880228823 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5144162160 con 0x7f512c004d50 2024-04-17T10:38:26.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f51427fc640 1 -- 172.21.15.83:0/2880228823 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c00bb60 con 0x7f512c0039f0 2024-04-17T10:38:26.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f51427fc640 1 -- 172.21.15.83:0/2880228823 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512c00bd60 con 0x7f51440ce980 2024-04-17T10:38:26.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f5143fff640 1 --2- 172.21.15.83:0/2880228823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f512c004d50 0x7f512c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:26.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f51437fe640 1 --2- 172.21.15.83:0/2880228823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f512c0039f0 0x7f512c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:26.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f5142ffd640 1 --2- 172.21.15.83:0/2880228823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51440ce980 0x7f512c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:26.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f5143fff640 1 -- 172.21.15.83:0/2880228823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51440ce980 msgr2=0x7f512c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:26.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f5143fff640 1 --2- 172.21.15.83:0/2880228823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51440ce980 0x7f512c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:26.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f5143fff640 1 -- 172.21.15.83:0/2880228823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f512c0039f0 msgr2=0x7f512c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:26.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f5143fff640 1 --2- 172.21.15.83:0/2880228823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f512c0039f0 0x7f512c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:26.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f5143fff640 1 -- 172.21.15.83:0/2880228823 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512c003530 con 0x7f512c004d50 2024-04-17T10:38:26.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f5143fff640 1 --2- 172.21.15.83:0/2880228823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f512c004d50 0x7f512c0051d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f513c001e70 tx=0x7f513c01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d59cab7ced7463 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:26.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f51427fc640 1 -- 172.21.15.83:0/2880228823 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c001a90 con 0x7f512c004d50 2024-04-17T10:38:26.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f51427fc640 1 -- 172.21.15.83:0/2880228823 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f513c003b80 con 0x7f512c004d50 2024-04-17T10:38:26.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f51427fc640 1 -- 172.21.15.83:0/2880228823 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c005080 con 0x7f512c004d50 2024-04-17T10:38:26.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f5149cb6640 1 -- 172.21.15.83:0/2880228823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f512c004d50 msgr2=0x7f512c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:26.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f5149cb6640 1 --2- 172.21.15.83:0/2880228823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f512c004d50 0x7f512c0051d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f513c001e70 tx=0x7f513c01c780 comp rx=0 tx=0).stop 2024-04-17T10:38:26.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f5143fff640 1 -- 172.21.15.83:0/2880228823 reap_dead start 2024-04-17T10:38:26.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f5149cb6640 1 -- 172.21.15.83:0/2880228823 shutdown_connections 2024-04-17T10:38:26.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.600+0000 7f5149cb6640 1 -- 172.21.15.83:0/2880228823 >> 172.21.15.83:0/2880228823 conn(0x7f51440d0600 msgr2=0x7f51440d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:26.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.604+0000 7f5149cb6640 1 -- 172.21.15.83:0/2880228823 shutdown_connections 2024-04-17T10:38:26.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.604+0000 7f5149cb6640 1 -- 172.21.15.83:0/2880228823 wait complete. 2024-04-17T10:38:26.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.604+0000 7f5149cb6640 1 Processor -- start 2024-04-17T10:38:26.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.604+0000 7f5149cb6640 1 -- start start 2024-04-17T10:38:26.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.604+0000 7f5149cb6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f512c004d50 0x7f5144145260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:26.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.604+0000 7f5149cb6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51440c8fa0 0x7f51440c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:26.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.604+0000 7f5149cb6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51440cc480 0x7f51440cc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:26.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.604+0000 7f5149cb6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5144161b10 con 0x7f512c004d50 2024-04-17T10:38:26.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.604+0000 7f5149cb6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514415e380 con 0x7f51440cc480 2024-04-17T10:38:26.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.604+0000 7f5149cb6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51440d2440 con 0x7f51440c8fa0 2024-04-17T10:38:26.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.604+0000 7f51437fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f512c004d50 0x7f5144145260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:26.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.604+0000 7f51437fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f512c004d50 0x7f5144145260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48402/0 (socket says 172.21.15.83:48402) 2024-04-17T10:38:26.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.604+0000 7f51437fe640 1 -- 172.21.15.83:0/328790840 learned_addr learned my addr 172.21.15.83:0/328790840 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:26.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.604+0000 7f5142ffd640 1 --2- 172.21.15.83:0/328790840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51440c8fa0 0x7f51440c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:26.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.604+0000 7f5143fff640 1 --2- 172.21.15.83:0/328790840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51440cc480 0x7f51440cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:26.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.604+0000 7f51437fe640 1 -- 172.21.15.83:0/328790840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51440c8fa0 msgr2=0x7f51440c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:26.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.604+0000 7f51437fe640 1 --2- 172.21.15.83:0/328790840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51440c8fa0 0x7f51440c9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:26.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.604+0000 7f51437fe640 1 -- 172.21.15.83:0/328790840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51440cc480 msgr2=0x7f51440cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:26.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.604+0000 7f51437fe640 1 --2- 172.21.15.83:0/328790840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51440cc480 0x7f51440cc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:26.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.604+0000 7f51437fe640 1 -- 172.21.15.83:0/328790840 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514415e5a0 con 0x7f512c004d50 2024-04-17T10:38:26.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.604+0000 7f5143fff640 1 --2- 172.21.15.83:0/328790840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51440cc480 0x7f51440cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:26.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.604+0000 7f51437fe640 1 --2- 172.21.15.83:0/328790840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f512c004d50 0x7f5144145260 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5134001ec0 tx=0x7f5134011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:26.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.604+0000 7f512bfff640 1 -- 172.21.15.83:0/328790840 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5134010040 con 0x7f512c004d50 2024-04-17T10:38:26.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.604+0000 7f512bfff640 1 -- 172.21.15.83:0/328790840 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f513401a070 con 0x7f512c004d50 2024-04-17T10:38:26.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.604+0000 7f5149cb6640 1 -- 172.21.15.83:0/328790840 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f514415e880 con 0x7f512c004d50 2024-04-17T10:38:26.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.604+0000 7f512bfff640 1 -- 172.21.15.83:0/328790840 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513400fdf0 con 0x7f512c004d50 2024-04-17T10:38:26.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.604+0000 7f5149cb6640 1 -- 172.21.15.83:0/328790840 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f514415ee40 con 0x7f512c004d50 2024-04-17T10:38:26.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.604+0000 7f5149cb6640 1 -- 172.21.15.83:0/328790840 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51440681a0 con 0x7f512c004d50 2024-04-17T10:38:26.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.608+0000 7f512bfff640 1 -- 172.21.15.83:0/328790840 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f513401f460 con 0x7f512c004d50 2024-04-17T10:38:26.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.608+0000 7f512bfff640 1 --2- 172.21.15.83:0/328790840 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5130066f60 0x7f5130069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:26.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.608+0000 7f512bfff640 1 -- 172.21.15.83:0/328790840 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5134088490 con 0x7f512c004d50 2024-04-17T10:38:26.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.608+0000 7f5142ffd640 1 --2- 172.21.15.83:0/328790840 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5130066f60 0x7f5130069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:26.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.608+0000 7f5142ffd640 1 --2- 172.21.15.83:0/328790840 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5130066f60 0x7f5130069420 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f51440ca4e0 tx=0x7f5138008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:26.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.612+0000 7f512bfff640 1 -- 172.21.15.83:0/328790840 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5134059480 con 0x7f512c004d50 2024-04-17T10:38:26.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.768+0000 7f5149cb6640 1 -- 172.21.15.83:0/328790840 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5144074f90 con 0x7f512c004d50 2024-04-17T10:38:26.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.768+0000 7f512bfff640 1 -- 172.21.15.83:0/328790840 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16533 (secure 0 0 0) 0x7f5134058d80 con 0x7f512c004d50 2024-04-17T10:38:26.766 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:26.766 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":20,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:38:13.690876+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:38:10.464490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T10:38:11.043134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:38:26.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.772+0000 7f5149cb6640 1 -- 172.21.15.83:0/328790840 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5130066f60 msgr2=0x7f5130069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:26.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.772+0000 7f5149cb6640 1 --2- 172.21.15.83:0/328790840 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5130066f60 0x7f5130069420 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f51440ca4e0 tx=0x7f5138008040 comp rx=0 tx=0).stop 2024-04-17T10:38:26.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.772+0000 7f5149cb6640 1 -- 172.21.15.83:0/328790840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f512c004d50 msgr2=0x7f5144145260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:26.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.772+0000 7f5149cb6640 1 --2- 172.21.15.83:0/328790840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f512c004d50 0x7f5144145260 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5134001ec0 tx=0x7f5134011de0 comp rx=0 tx=0).stop 2024-04-17T10:38:26.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.772+0000 7f5149cb6640 1 -- 172.21.15.83:0/328790840 shutdown_connections 2024-04-17T10:38:26.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.772+0000 7f5149cb6640 1 --2- 172.21.15.83:0/328790840 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5130066f60 0x7f5130069420 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:26.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.772+0000 7f5149cb6640 1 --2- 172.21.15.83:0/328790840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51440cc480 0x7f51440cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:26.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.772+0000 7f5149cb6640 1 --2- 172.21.15.83:0/328790840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51440c8fa0 0x7f51440c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:26.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.772+0000 7f5149cb6640 1 --2- 172.21.15.83:0/328790840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f512c004d50 0x7f5144145260 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:26.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.772+0000 7f5149cb6640 1 -- 172.21.15.83:0/328790840 >> 172.21.15.83:0/328790840 conn(0x7f51440d0600 msgr2=0x7f51440d2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:26.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.772+0000 7f5149cb6640 1 -- 172.21.15.83:0/328790840 shutdown_connections 2024-04-17T10:38:26.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.772+0000 7f5149cb6640 1 -- 172.21.15.83:0/328790840 wait complete. 2024-04-17T10:38:26.784 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:38:26.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.908+0000 7f7d37918640 1 Processor -- start 2024-04-17T10:38:26.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.908+0000 7f7d37918640 1 -- start start 2024-04-17T10:38:26.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.912+0000 7f7d37918640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d300afdd0 0x7f7d30071580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:26.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.912+0000 7f7d37918640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7d3006ecb0 0x7f7d300ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:26.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.912+0000 7f7d37918640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d300b01b0 con 0x7f7d30071ac0 2024-04-17T10:38:26.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.912+0000 7f7d37918640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d30071f20 con 0x7f7d3006ecb0 2024-04-17T10:38:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.912+0000 7f7d37918640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3006f2d0 con 0x7f7d300afdd0 2024-04-17T10:38:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.912+0000 7f7d35eb5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7d3006ecb0 0x7f7d300ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.912+0000 7f7d34eb3640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7d30071ac0 0x7f7d3006e5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:43782/0 (socket says 172.21.15.83:43782) 2024-04-17T10:38:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.912+0000 7f7d35eb5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7d3006ecb0 0x7f7d300ce5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38284/0 (socket says 172.21.15.83:38284) 2024-04-17T10:38:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.912+0000 7f7d34eb3640 1 -- 172.21.15.83:0/3813769736 learned_addr learned my addr 172.21.15.83:0/3813769736 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.912+0000 7f7d356b4640 1 --2- 172.21.15.83:0/3813769736 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d300afdd0 0x7f7d30071580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.912+0000 7f7d35eb5640 1 -- 172.21.15.83:0/3813769736 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d300afdd0 msgr2=0x7f7d30071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.912+0000 7f7d35eb5640 1 --2- 172.21.15.83:0/3813769736 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d300afdd0 0x7f7d30071580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.912+0000 7f7d35eb5640 1 -- 172.21.15.83:0/3813769736 >> v1:172.21.15.83:6789/0 conn(0x7f7d30071ac0 legacy=0x7f7d3006e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.912+0000 7f7d35eb5640 1 -- 172.21.15.83:0/3813769736 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d3006f5b0 con 0x7f7d3006ecb0 2024-04-17T10:38:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.912+0000 7f7d356b4640 1 --2- 172.21.15.83:0/3813769736 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d300afdd0 0x7f7d30071580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:26.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.912+0000 7f7d35eb5640 1 --2- 172.21.15.83:0/3813769736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d3006ecb0 0x7f7d300ce5c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7d28009d90 tx=0x7f7d2801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13086cc3d01811d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:26.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.912+0000 7f7d27fff640 1 -- 172.21.15.83:0/3813769736 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d280b2070 con 0x7f7d3006ecb0 2024-04-17T10:38:26.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.912+0000 7f7d27fff640 1 -- 172.21.15.83:0/3813769736 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d280a5a00 con 0x7f7d3006ecb0 2024-04-17T10:38:26.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.912+0000 7f7d27fff640 1 -- 172.21.15.83:0/3813769736 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d280a5dd0 con 0x7f7d3006ecb0 2024-04-17T10:38:26.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.912+0000 7f7d37918640 1 -- 172.21.15.83:0/3813769736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d3006ecb0 msgr2=0x7f7d300ce5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:26.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.912+0000 7f7d37918640 1 --2- 172.21.15.83:0/3813769736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d3006ecb0 0x7f7d300ce5c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7d28009d90 tx=0x7f7d2801c940 comp rx=0 tx=0).stop 2024-04-17T10:38:26.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.912+0000 7f7d37918640 1 -- 172.21.15.83:0/3813769736 shutdown_connections 2024-04-17T10:38:26.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.912+0000 7f7d37918640 1 --2- 172.21.15.83:0/3813769736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d3006ecb0 0x7f7d300ce5c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:26.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.912+0000 7f7d37918640 1 --2- 172.21.15.83:0/3813769736 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d300afdd0 0x7f7d30071580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:26.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.912+0000 7f7d37918640 1 -- 172.21.15.83:0/3813769736 >> 172.21.15.83:0/3813769736 conn(0x7f7d300d07b0 msgr2=0x7f7d300d0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:26.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.912+0000 7f7d37918640 1 -- 172.21.15.83:0/3813769736 shutdown_connections 2024-04-17T10:38:26.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.912+0000 7f7d37918640 1 -- 172.21.15.83:0/3813769736 wait complete. 2024-04-17T10:38:26.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.916+0000 7f7d37918640 1 Processor -- start 2024-04-17T10:38:26.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.916+0000 7f7d37918640 1 -- start start 2024-04-17T10:38:26.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.916+0000 7f7d37918640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d3006ecb0 0x7f7d301573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:26.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.916+0000 7f7d37918640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d30071ac0 0x7f7d300b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:26.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.916+0000 7f7d37918640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d300afdd0 0x7f7d30167810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:26.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.916+0000 7f7d37918640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d300cf490 con 0x7f7d3006ecb0 2024-04-17T10:38:26.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.916+0000 7f7d37918640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d300cf310 con 0x7f7d300afdd0 2024-04-17T10:38:26.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.916+0000 7f7d37918640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d300cf610 con 0x7f7d30071ac0 2024-04-17T10:38:26.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.916+0000 7f7d356b4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d3006ecb0 0x7f7d301573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:26.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.916+0000 7f7d356b4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d3006ecb0 0x7f7d301573c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38296/0 (socket says 172.21.15.83:38296) 2024-04-17T10:38:26.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.916+0000 7f7d356b4640 1 -- 172.21.15.83:0/3418565466 learned_addr learned my addr 172.21.15.83:0/3418565466 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:26.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.916+0000 7f7d356b4640 1 -- 172.21.15.83:0/3418565466 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d30071ac0 msgr2=0x7f7d300b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:26.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.916+0000 7f7d356b4640 1 --2- 172.21.15.83:0/3418565466 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d30071ac0 0x7f7d300b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:26.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.916+0000 7f7d356b4640 1 -- 172.21.15.83:0/3418565466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d300afdd0 msgr2=0x7f7d30167810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:26.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.916+0000 7f7d356b4640 1 --2- 172.21.15.83:0/3418565466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d300afdd0 0x7f7d30167810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:26.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.916+0000 7f7d356b4640 1 -- 172.21.15.83:0/3418565466 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d30157900 con 0x7f7d3006ecb0 2024-04-17T10:38:26.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.916+0000 7f7d356b4640 1 --2- 172.21.15.83:0/3418565466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d3006ecb0 0x7f7d301573c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000c7f0 tx=0x7f7d2000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:26.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.916+0000 7f7d25ffb640 1 -- 172.21.15.83:0/3418565466 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d20015070 con 0x7f7d3006ecb0 2024-04-17T10:38:26.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.916+0000 7f7d25ffb640 1 -- 172.21.15.83:0/3418565466 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d20007820 con 0x7f7d3006ecb0 2024-04-17T10:38:26.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.916+0000 7f7d37918640 1 -- 172.21.15.83:0/3418565466 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d300b3e50 con 0x7f7d3006ecb0 2024-04-17T10:38:26.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.916+0000 7f7d25ffb640 1 -- 172.21.15.83:0/3418565466 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d20007c10 con 0x7f7d3006ecb0 2024-04-17T10:38:26.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.916+0000 7f7d37918640 1 -- 172.21.15.83:0/3418565466 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d300b4410 con 0x7f7d3006ecb0 2024-04-17T10:38:26.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.916+0000 7f7d37918640 1 -- 172.21.15.83:0/3418565466 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d300b4ad0 con 0x7f7d3006ecb0 2024-04-17T10:38:26.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.916+0000 7f7d25ffb640 1 -- 172.21.15.83:0/3418565466 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7d20018070 con 0x7f7d3006ecb0 2024-04-17T10:38:26.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.920+0000 7f7d25ffb640 1 --2- 172.21.15.83:0/3418565466 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7d14066f60 0x7f7d14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:26.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.920+0000 7f7d25ffb640 1 -- 172.21.15.83:0/3418565466 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d200850f0 con 0x7f7d3006ecb0 2024-04-17T10:38:26.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.920+0000 7f7d34eb3640 1 --2- 172.21.15.83:0/3418565466 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7d14066f60 0x7f7d14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:26.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.920+0000 7f7d34eb3640 1 --2- 172.21.15.83:0/3418565466 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7d14066f60 0x7f7d14069420 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c000f70 tx=0x7f7d2c007a90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:26.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:26.924+0000 7f7d25ffb640 1 -- 172.21.15.83:0/3418565466 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7d300b4ad0 con 0x7f7d3006ecb0 2024-04-17T10:38:27.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.120+0000 7f7d37918640 1 -- 172.21.15.83:0/3418565466 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d30074f90 con 0x7f7d3006ecb0 2024-04-17T10:38:27.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.120+0000 7f7d25ffb640 1 -- 172.21.15.83:0/3418565466 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4938 (secure 0 0 0) 0x7f7d20056100 con 0x7f7d3006ecb0 2024-04-17T10:38:27.116 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:27.116 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2884469968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2884469968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2884469968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2919571009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2919571009},{"type":"v1","addr":"172.21.15.83:6836","nonce":2919571009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1635176834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1635176834},{"type":"v1","addr":"172.21.15.83:6837","nonce":1635176834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:38:12.032695+0000","modified":"2024-04-17T10:38:14.064978+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6835/3871588351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3871588351},{"type":"v1","addr":"172.21.15.167:6835","nonce":3871588351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-17T10:38:27.116 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 6 2024-04-17T10:38:27.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.120+0000 7f7d37918640 1 -- 172.21.15.83:0/3418565466 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7d14066f60 msgr2=0x7f7d14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:27.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.120+0000 7f7d37918640 1 --2- 172.21.15.83:0/3418565466 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7d14066f60 0x7f7d14069420 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c000f70 tx=0x7f7d2c007a90 comp rx=0 tx=0).stop 2024-04-17T10:38:27.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.120+0000 7f7d37918640 1 -- 172.21.15.83:0/3418565466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d3006ecb0 msgr2=0x7f7d301573c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:27.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.120+0000 7f7d37918640 1 --2- 172.21.15.83:0/3418565466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d3006ecb0 0x7f7d301573c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000c7f0 tx=0x7f7d2000ccb0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.120+0000 7f7d37918640 1 -- 172.21.15.83:0/3418565466 shutdown_connections 2024-04-17T10:38:27.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.120+0000 7f7d37918640 1 --2- 172.21.15.83:0/3418565466 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7d14066f60 0x7f7d14069420 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.120+0000 7f7d37918640 1 --2- 172.21.15.83:0/3418565466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d300afdd0 0x7f7d30167810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.120+0000 7f7d37918640 1 --2- 172.21.15.83:0/3418565466 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d30071ac0 0x7f7d300b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.120+0000 7f7d37918640 1 --2- 172.21.15.83:0/3418565466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d3006ecb0 0x7f7d301573c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.120+0000 7f7d37918640 1 -- 172.21.15.83:0/3418565466 >> 172.21.15.83:0/3418565466 conn(0x7f7d300d07b0 msgr2=0x7f7d300b0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:27.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.120+0000 7f7d37918640 1 -- 172.21.15.83:0/3418565466 shutdown_connections 2024-04-17T10:38:27.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.120+0000 7f7d37918640 1 -- 172.21.15.83:0/3418565466 wait complete. 2024-04-17T10:38:27.536 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T10:38:12.032695+0000', 'modified': '2024-04-17T10:38:14.064978+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4273}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4273': {'gid': 4273, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.167:6835/3871588351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6834', 'nonce': 3871588351}, {'type': 'v1', 'addr': '172.21.15.167:6835', 'nonce': 3871588351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:38:27.536 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T10:38:27.537 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-17T10:38:27.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.672+0000 7fd5f29ba640 1 Processor -- start 2024-04-17T10:38:27.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.672+0000 7fd5f29ba640 1 -- start start 2024-04-17T10:38:27.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.672+0000 7fd5f29ba640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5ec154670 0x7fd5ec0aea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:27.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.672+0000 7fd5f29ba640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd5ec06cc80 0x7fd5ec070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:27.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.672+0000 7fd5f29ba640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5ec0b2ff0 con 0x7fd5ec0b4090 2024-04-17T10:38:27.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.672+0000 7fd5f29ba640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5ec154a50 con 0x7fd5ec06cc80 2024-04-17T10:38:27.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.672+0000 7fd5f29ba640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec0b32d0 con 0x7fd5ec154670 2024-04-17T10:38:27.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.672+0000 7fd5f0f57640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd5ec06cc80 0x7fd5ec070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:27.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.672+0000 7fd5f0f57640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd5ec06cc80 0x7fd5ec070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38300/0 (socket says 172.21.15.83:38300) 2024-04-17T10:38:27.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.672+0000 7fd5f0f57640 1 -- 172.21.15.83:0/1923088219 learned_addr learned my addr 172.21.15.83:0/1923088219 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:27.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.672+0000 7fd5ebfff640 1 --2- 172.21.15.83:0/1923088219 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5ec154670 0x7fd5ec0aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:27.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.672+0000 7fd5f0f57640 1 -- 172.21.15.83:0/1923088219 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5ec154670 msgr2=0x7fd5ec0aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:27.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.672+0000 7fd5f0f57640 1 --2- 172.21.15.83:0/1923088219 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5ec154670 0x7fd5ec0aea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.672+0000 7fd5f0f57640 1 -- 172.21.15.83:0/1923088219 >> v1:172.21.15.167:6789/0 conn(0x7fd5ec0b4090 legacy=0x7fd5ec06c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:27.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.672+0000 7fd5f0f57640 1 -- 172.21.15.83:0/1923088219 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ec0b35b0 con 0x7fd5ec06cc80 2024-04-17T10:38:27.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.676+0000 7fd5f0f57640 1 --2- 172.21.15.83:0/1923088219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5ec06cc80 0x7fd5ec070470 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd5e400d150 tx=0x7fd5e401eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02cf6d6ef5327a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:27.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.676+0000 7fd5eb7fe640 1 -- 172.21.15.83:0/1923088219 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e400b2d0 con 0x7fd5ec06cc80 2024-04-17T10:38:27.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.676+0000 7fd5eb7fe640 1 -- 172.21.15.83:0/1923088219 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e400b470 con 0x7fd5ec06cc80 2024-04-17T10:38:27.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.676+0000 7fd5eb7fe640 1 -- 172.21.15.83:0/1923088219 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e40a7b70 con 0x7fd5ec06cc80 2024-04-17T10:38:27.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.676+0000 7fd5f29ba640 1 -- 172.21.15.83:0/1923088219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5ec06cc80 msgr2=0x7fd5ec070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:27.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.676+0000 7fd5f29ba640 1 --2- 172.21.15.83:0/1923088219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5ec06cc80 0x7fd5ec070470 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd5e400d150 tx=0x7fd5e401eb70 comp rx=0 tx=0).stop 2024-04-17T10:38:27.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.676+0000 7fd5f29ba640 1 -- 172.21.15.83:0/1923088219 shutdown_connections 2024-04-17T10:38:27.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.676+0000 7fd5f29ba640 1 --2- 172.21.15.83:0/1923088219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5ec06cc80 0x7fd5ec070470 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.676+0000 7fd5f29ba640 1 --2- 172.21.15.83:0/1923088219 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5ec154670 0x7fd5ec0aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.676+0000 7fd5f29ba640 1 -- 172.21.15.83:0/1923088219 >> 172.21.15.83:0/1923088219 conn(0x7fd5ec0b15b0 msgr2=0x7fd5ec0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:27.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.676+0000 7fd5f29ba640 1 -- 172.21.15.83:0/1923088219 shutdown_connections 2024-04-17T10:38:27.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.676+0000 7fd5f29ba640 1 -- 172.21.15.83:0/1923088219 wait complete. 2024-04-17T10:38:27.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.680+0000 7fd5f29ba640 1 Processor -- start 2024-04-17T10:38:27.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.680+0000 7fd5f29ba640 1 -- start start 2024-04-17T10:38:27.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.680+0000 7fd5f29ba640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5ec06cc80 0x7fd5ec0adee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:27.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.680+0000 7fd5f29ba640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5ec0b4090 0x7fd5ec149b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:27.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.680+0000 7fd5f29ba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5ec154670 0x7fd5ec145ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:27.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.680+0000 7fd5f29ba640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec0aef90 con 0x7fd5ec154670 2024-04-17T10:38:27.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.680+0000 7fd5f29ba640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec15b3c0 con 0x7fd5ec06cc80 2024-04-17T10:38:27.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.680+0000 7fd5f29ba640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec0af110 con 0x7fd5ec0b4090 2024-04-17T10:38:27.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.680+0000 7fd5f0f57640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5ec154670 0x7fd5ec145ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:27.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.680+0000 7fd5f0f57640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5ec154670 0x7fd5ec145ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38306/0 (socket says 172.21.15.83:38306) 2024-04-17T10:38:27.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.680+0000 7fd5f0f57640 1 -- 172.21.15.83:0/3899308427 learned_addr learned my addr 172.21.15.83:0/3899308427 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:27.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.680+0000 7fd5e37fe640 1 --2- 172.21.15.83:0/3899308427 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5ec0b4090 0x7fd5ec149b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:27.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.680+0000 7fd5ebfff640 1 --2- 172.21.15.83:0/3899308427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5ec06cc80 0x7fd5ec0adee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:27.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.680+0000 7fd5f0f57640 1 -- 172.21.15.83:0/3899308427 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5ec0b4090 msgr2=0x7fd5ec149b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:27.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.680+0000 7fd5f0f57640 1 --2- 172.21.15.83:0/3899308427 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5ec0b4090 0x7fd5ec149b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.680+0000 7fd5f0f57640 1 -- 172.21.15.83:0/3899308427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5ec06cc80 msgr2=0x7fd5ec0adee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:27.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.680+0000 7fd5f0f57640 1 --2- 172.21.15.83:0/3899308427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5ec06cc80 0x7fd5ec0adee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.680+0000 7fd5f0f57640 1 -- 172.21.15.83:0/3899308427 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ec150110 con 0x7fd5ec154670 2024-04-17T10:38:27.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.680+0000 7fd5ebfff640 1 --2- 172.21.15.83:0/3899308427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5ec06cc80 0x7fd5ec0adee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:27.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.680+0000 7fd5f0f57640 1 --2- 172.21.15.83:0/3899308427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5ec154670 0x7fd5ec145ce0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd5e401c7a0 tx=0x7fd5e401b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:27.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.680+0000 7fd5e97fa640 1 -- 172.21.15.83:0/3899308427 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e40be020 con 0x7fd5ec154670 2024-04-17T10:38:27.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.680+0000 7fd5e97fa640 1 -- 172.21.15.83:0/3899308427 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e4009800 con 0x7fd5ec154670 2024-04-17T10:38:27.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.680+0000 7fd5f29ba640 1 -- 172.21.15.83:0/3899308427 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5ec14a060 con 0x7fd5ec154670 2024-04-17T10:38:27.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.680+0000 7fd5e97fa640 1 -- 172.21.15.83:0/3899308427 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4009bd0 con 0x7fd5ec154670 2024-04-17T10:38:27.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.680+0000 7fd5f29ba640 1 -- 172.21.15.83:0/3899308427 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5ec150690 con 0x7fd5ec154670 2024-04-17T10:38:27.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.680+0000 7fd5e97fa640 1 -- 172.21.15.83:0/3899308427 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd5e40c1020 con 0x7fd5ec154670 2024-04-17T10:38:27.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5e97fa640 1 --2- 172.21.15.83:0/3899308427 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd5cc066f60 0x7fd5cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:27.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5e97fa640 1 -- 172.21.15.83:0/3899308427 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd5e4122240 con 0x7fd5ec154670 2024-04-17T10:38:27.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5ebfff640 1 --2- 172.21.15.83:0/3899308427 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd5cc066f60 0x7fd5cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:27.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5ebfff640 1 --2- 172.21.15.83:0/3899308427 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd5cc066f60 0x7fd5cc069420 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc0097a0 tx=0x7fd5dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:27.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5f5b27000 1 Processor -- start 2024-04-17T10:38:27.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5f5b27000 1 -- start start 2024-04-17T10:38:27.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5f5b27000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0560b920 0x55ba0560bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:27.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5f5b27000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ba0560c240 0x55ba055c19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:27.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5f5b27000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba055c1f30 0x55ba055c2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:27.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5f5b27000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba054a95f0 con 0x55ba055c1f30 2024-04-17T10:38:27.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5f5b27000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba05410da0 con 0x55ba0560b920 2024-04-17T10:38:27.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5f5b27000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba05399150 con 0x55ba0560c240 2024-04-17T10:38:27.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5f0f57640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba055c1f30 0x55ba055c2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:27.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5f0f57640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba055c1f30 0x55ba055c2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38318/0 (socket says 172.21.15.83:38318) 2024-04-17T10:38:27.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5f0f57640 1 -- 172.21.15.83:0/3195264237 learned_addr learned my addr 172.21.15.83:0/3195264237 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:27.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5e37fe640 1 --2- 172.21.15.83:0/3195264237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ba0560c240 0x55ba055c19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:27.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5ebfff640 1 --2- 172.21.15.83:0/3195264237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0560b920 0x55ba0560bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:27.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5f0f57640 1 -- 172.21.15.83:0/3195264237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ba0560c240 msgr2=0x55ba055c19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:27.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5f0f57640 1 --2- 172.21.15.83:0/3195264237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ba0560c240 0x55ba055c19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5f0f57640 1 -- 172.21.15.83:0/3195264237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0560b920 msgr2=0x55ba0560bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:27.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5f0f57640 1 --2- 172.21.15.83:0/3195264237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0560b920 0x55ba0560bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5f0f57640 1 -- 172.21.15.83:0/3195264237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba053b2ca0 con 0x55ba055c1f30 2024-04-17T10:38:27.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5f0f57640 1 --2- 172.21.15.83:0/3195264237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba055c1f30 0x55ba055c2310 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40fafa0 tx=0x7fd5e400ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68b1b16d316169e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:27.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5e1ffb640 1 -- 172.21.15.83:0/3195264237 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e40f6f10 con 0x55ba055c1f30 2024-04-17T10:38:27.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5e1ffb640 1 -- 172.21.15.83:0/3195264237 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e40fb4d0 con 0x55ba055c1f30 2024-04-17T10:38:27.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5e1ffb640 1 -- 172.21.15.83:0/3195264237 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e40b7e80 con 0x55ba055c1f30 2024-04-17T10:38:27.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5f5b27000 1 -- 172.21.15.83:0/3195264237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba055c1f30 msgr2=0x55ba055c2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:27.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5f5b27000 1 --2- 172.21.15.83:0/3195264237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba055c1f30 0x55ba055c2310 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40fafa0 tx=0x7fd5e400ba20 comp rx=0 tx=0).stop 2024-04-17T10:38:27.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5f5b27000 1 -- 172.21.15.83:0/3195264237 shutdown_connections 2024-04-17T10:38:27.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5f5b27000 1 --2- 172.21.15.83:0/3195264237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba055c1f30 0x55ba055c2310 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5f5b27000 1 --2- 172.21.15.83:0/3195264237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ba0560c240 0x55ba055c19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5f5b27000 1 --2- 172.21.15.83:0/3195264237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0560b920 0x55ba0560bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5f5b27000 1 -- 172.21.15.83:0/3195264237 >> 172.21.15.83:0/3195264237 conn(0x55ba055c82a0 msgr2=0x55ba055c02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:27.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5f5b27000 1 -- 172.21.15.83:0/3195264237 shutdown_connections 2024-04-17T10:38:27.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5f5b27000 1 -- 172.21.15.83:0/3195264237 wait complete. 2024-04-17T10:38:27.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5f5b27000 1 -- start start 2024-04-17T10:38:27.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.684+0000 7fd5f5b27000 1 Processor -- start 2024-04-17T10:38:27.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5f5b27000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ba055c1f30 0x55ba05612c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:27.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5f5b27000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba0560b920 0x55ba056131c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:27.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5f5b27000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0560c240 0x55ba055c5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:27.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5f5b27000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba053cb990 con 0x55ba0560b920 2024-04-17T10:38:27.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5f5b27000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba0540ce40 con 0x55ba0560c240 2024-04-17T10:38:27.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5f5b27000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba0539c900 con 0x55ba055c1f30 2024-04-17T10:38:27.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5f5b27000 10 client.0 _ll_register_callbacks cb 0x55ba054c41c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:38:27.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5e37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba0560b920 0x55ba056131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:27.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5e37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba0560b920 0x55ba056131c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38332/0 (socket says 172.21.15.83:38332) 2024-04-17T10:38:27.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5e37fe640 1 -- 172.21.15.83:0/3737907445 learned_addr learned my addr 172.21.15.83:0/3737907445 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:27.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5ebfff640 1 --2- 172.21.15.83:0/3737907445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ba055c1f30 0x55ba05612c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:27.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5f0f57640 1 --2- 172.21.15.83:0/3737907445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0560c240 0x55ba055c5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:27.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5e37fe640 1 -- 172.21.15.83:0/3737907445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ba055c1f30 msgr2=0x55ba05612c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:27.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5e37fe640 1 --2- 172.21.15.83:0/3737907445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ba055c1f30 0x55ba05612c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5e37fe640 1 -- 172.21.15.83:0/3737907445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0560c240 msgr2=0x55ba055c5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:27.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5e37fe640 1 --2- 172.21.15.83:0/3737907445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0560c240 0x55ba055c5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5e37fe640 1 -- 172.21.15.83:0/3737907445 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba053c9630 con 0x55ba0560b920 2024-04-17T10:38:27.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5f0f57640 1 --2- 172.21.15.83:0/3737907445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0560c240 0x55ba055c5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:27.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5e37fe640 1 --2- 172.21.15.83:0/3737907445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba0560b920 0x55ba056131c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4001d00 tx=0x7fd5d400f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b4bcae40cc4159 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:27.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5caffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:38:27.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5caffd640 1 -- 172.21.15.83:0/3737907445 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d40050d0 con 0x55ba0560b920 2024-04-17T10:38:27.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5f5b27000 1 -- 172.21.15.83:0/3737907445 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5e40fb4d0 con 0x55ba0560b920 2024-04-17T10:38:27.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5caffd640 1 -- 172.21.15.83:0/3737907445 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd5d4005270 con 0x55ba0560b920 2024-04-17T10:38:27.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5caffd640 1 -- 172.21.15.83:0/3737907445 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5d4020050 con 0x55ba0560b920 2024-04-17T10:38:27.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5f5b27000 10 client.4602 fetch_fsmap learned FSMap version 6 2024-04-17T10:38:27.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5f5b27000 1 -- 172.21.15.83:0/3737907445 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55ba053993a0 con 0x55ba0560b920 2024-04-17T10:38:27.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5caffd640 1 -- 172.21.15.83:0/3737907445 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e40fb4d0 con 0x55ba0560b920 2024-04-17T10:38:27.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5caffd640 1 -- 172.21.15.83:0/3737907445 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 6) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd5d4023020 con 0x55ba0560b920 2024-04-17T10:38:27.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5f5b27000 10 client.4602 fetch_fsmap finished waiting for FSMap version 6 2024-04-17T10:38:27.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5f5b27000 10 client.4602 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-17T10:38:27.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5f5b27000 20 client.4602 populate_metadata read hostname 'smithi083' 2024-04-17T10:38:27.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5f5b27000 1 --2- 172.21.15.83:0/3737907445 >> [v2:172.21.15.167:6834/3871588351,v1:172.21.15.167:6835/3871588351] conn(0x55ba055c64b0 0x55ba055c6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:27.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5f5b27000 4 client.4602 mds_command: new command op to 4273 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:38:27.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5f5b27000 1 -- 172.21.15.83:0/3737907445 --> [v2:172.21.15.167:6834/3871588351,v1:172.21.15.167:6835/3871588351] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba053aca50 con 0x55ba055c64b0 2024-04-17T10:38:27.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5ebfff640 1 --2- 172.21.15.83:0/3737907445 >> [v2:172.21.15.167:6834/3871588351,v1:172.21.15.167:6835/3871588351] conn(0x55ba055c64b0 0x55ba055c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:27.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5caffd640 10 client.4602 ms_handle_connect on v2:172.21.15.167:6834/3871588351 2024-04-17T10:38:27.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.688+0000 7fd5ebfff640 1 --2- 172.21.15.83:0/3737907445 >> [v2:172.21.15.167:6834/3871588351,v1:172.21.15.167:6835/3871588351] conn(0x55ba055c64b0 0x55ba055c6890 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b98c0ede4e1af355 server_cookie=fef0fc0f82a8a29f in_seq=0 out_seq=0 2024-04-17T10:38:27.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.692+0000 7fd5caffd640 1 -- 172.21.15.83:0/3737907445 <== mds.0 v2:172.21.15.167:6834/3871588351 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ba053aca50 con 0x55ba055c64b0 2024-04-17T10:38:27.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.692+0000 7fd5caffd640 10 client.4602 handle_command_reply: tid=0 2024-04-17T10:38:27.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.692+0000 7fd5f5b27000 1 client.4602 shutdown 2024-04-17T10:38:27.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.692+0000 7fd5f5b27000 1 -- 172.21.15.83:0/3737907445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba0560b920 msgr2=0x55ba056131c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:27.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.692+0000 7fd5f5b27000 1 --2- 172.21.15.83:0/3737907445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba0560b920 0x55ba056131c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4001d00 tx=0x7fd5d400f5a0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.692+0000 7fd5f5b27000 1 -- 172.21.15.83:0/3737907445 shutdown_connections 2024-04-17T10:38:27.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.692+0000 7fd5f5b27000 1 --2- 172.21.15.83:0/3737907445 >> [v2:172.21.15.167:6834/3871588351,v1:172.21.15.167:6835/3871588351] conn(0x55ba055c64b0 0x55ba055c6890 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.692+0000 7fd5f5b27000 1 --2- 172.21.15.83:0/3737907445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0560c240 0x55ba055c5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.692+0000 7fd5f5b27000 1 --2- 172.21.15.83:0/3737907445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba0560b920 0x55ba056131c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.692+0000 7fd5f5b27000 1 --2- 172.21.15.83:0/3737907445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ba055c1f30 0x55ba05612c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.692+0000 7fd5caffd640 0 client.4602 ms_handle_reset on v2:172.21.15.167:6834/3871588351 2024-04-17T10:38:27.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.692+0000 7fd5f5b27000 1 -- 172.21.15.83:0/3737907445 >> 172.21.15.83:0/3737907445 conn(0x55ba055c82a0 msgr2=0x55ba055c09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:27.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.692+0000 7fd5f5b27000 1 -- 172.21.15.83:0/3737907445 shutdown_connections 2024-04-17T10:38:27.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.692+0000 7fd5f5b27000 1 -- 172.21.15.83:0/3737907445 wait complete. 2024-04-17T10:38:27.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.692+0000 7fd5f5b27000 20 client.4602 trim_cache size 0 max 16384 2024-04-17T10:38:27.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.708+0000 7fd5f5b27000 1 Processor -- start 2024-04-17T10:38:27.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.708+0000 7fd5f5b27000 1 -- start start 2024-04-17T10:38:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.708+0000 7fd5f5b27000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba055c1f30 0x55ba055c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.708+0000 7fd5f5b27000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ba0560b920 0x55ba055c5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.708+0000 7fd5f5b27000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0560c240 0x55ba056158b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.708+0000 7fd5f5b27000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec14eb40 con 0x55ba055c1f30 2024-04-17T10:38:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.708+0000 7fd5f5b27000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec14df40 con 0x55ba0560c240 2024-04-17T10:38:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.708+0000 7fd5f5b27000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec14e540 con 0x55ba0560b920 2024-04-17T10:38:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.708+0000 7fd5ebfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba055c1f30 0x55ba055c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.708+0000 7fd5ebfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba055c1f30 0x55ba055c5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38342/0 (socket says 172.21.15.83:38342) 2024-04-17T10:38:27.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.708+0000 7fd5ebfff640 1 -- 172.21.15.83:0/1354546814 learned_addr learned my addr 172.21.15.83:0/1354546814 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:27.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.708+0000 7fd5ebfff640 1 -- 172.21.15.83:0/1354546814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ba0560b920 msgr2=0x55ba055c5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:27.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.708+0000 7fd5ebfff640 1 --2- 172.21.15.83:0/1354546814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ba0560b920 0x55ba055c5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.708+0000 7fd5ebfff640 1 -- 172.21.15.83:0/1354546814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0560c240 msgr2=0x55ba056158b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:27.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.708+0000 7fd5ebfff640 1 --2- 172.21.15.83:0/1354546814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0560c240 0x55ba056158b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.708+0000 7fd5ebfff640 1 -- 172.21.15.83:0/1354546814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba05492600 con 0x55ba055c1f30 2024-04-17T10:38:27.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.708+0000 7fd5ebfff640 1 --2- 172.21.15.83:0/1354546814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba055c1f30 0x55ba055c5330 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55ba0540d750 tx=0x7fd5dc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0f5845cc9dd13e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:27.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.708+0000 7fd5e17fa640 1 -- 172.21.15.83:0/1354546814 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc00f040 con 0x55ba055c1f30 2024-04-17T10:38:27.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.708+0000 7fd5e17fa640 1 -- 172.21.15.83:0/1354546814 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5dc025620 con 0x55ba055c1f30 2024-04-17T10:38:27.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.708+0000 7fd5e17fa640 1 -- 172.21.15.83:0/1354546814 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc0059a0 con 0x55ba055c1f30 2024-04-17T10:38:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.708+0000 7fd5f5b27000 1 -- 172.21.15.83:0/1354546814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba055c1f30 msgr2=0x55ba055c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.708+0000 7fd5f5b27000 1 --2- 172.21.15.83:0/1354546814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba055c1f30 0x55ba055c5330 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55ba0540d750 tx=0x7fd5dc014000 comp rx=0 tx=0).stop 2024-04-17T10:38:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.708+0000 7fd5f5b27000 1 -- 172.21.15.83:0/1354546814 shutdown_connections 2024-04-17T10:38:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.708+0000 7fd5f5b27000 1 --2- 172.21.15.83:0/1354546814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0560c240 0x55ba056158b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.708+0000 7fd5f5b27000 1 --2- 172.21.15.83:0/1354546814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ba0560b920 0x55ba055c5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.708+0000 7fd5f5b27000 1 --2- 172.21.15.83:0/1354546814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba055c1f30 0x55ba055c5330 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.708+0000 7fd5f5b27000 1 -- 172.21.15.83:0/1354546814 >> 172.21.15.83:0/1354546814 conn(0x55ba055c82a0 msgr2=0x55ba055f7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.708+0000 7fd5f5b27000 1 -- 172.21.15.83:0/1354546814 shutdown_connections 2024-04-17T10:38:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.708+0000 7fd5f5b27000 1 -- 172.21.15.83:0/1354546814 wait complete. 2024-04-17T10:38:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.708+0000 7fd5f5b27000 1 -- start start 2024-04-17T10:38:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.708+0000 7fd5f5b27000 1 Processor -- start 2024-04-17T10:38:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5f5b27000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba055c1f30 0x55ba056187e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5f5b27000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0560b920 0x55ba05652a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5f5b27000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ba0560c240 0x55ba05652f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5f5b27000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba054153d0 con 0x55ba055c1f30 2024-04-17T10:38:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5f5b27000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba054ccd50 con 0x55ba0560b920 2024-04-17T10:38:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5f5b27000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba053ab980 con 0x55ba0560c240 2024-04-17T10:38:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5f5b27000 10 client.0 _ll_register_callbacks cb 0x7fd5e400bdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:38:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5ebfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba055c1f30 0x55ba056187e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5ebfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba055c1f30 0x55ba056187e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38350/0 (socket says 172.21.15.83:38350) 2024-04-17T10:38:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5ebfff640 1 -- 172.21.15.83:0/664678622 learned_addr learned my addr 172.21.15.83:0/664678622 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5f0f57640 1 --2- 172.21.15.83:0/664678622 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ba0560c240 0x55ba05652f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:27.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5e37fe640 1 --2- 172.21.15.83:0/664678622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0560b920 0x55ba05652a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:27.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5ebfff640 1 -- 172.21.15.83:0/664678622 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ba0560c240 msgr2=0x55ba05652f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:27.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5ebfff640 1 --2- 172.21.15.83:0/664678622 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ba0560c240 0x55ba05652f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5ebfff640 1 -- 172.21.15.83:0/664678622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0560b920 msgr2=0x55ba05652a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:27.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5ebfff640 1 --2- 172.21.15.83:0/664678622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0560b920 0x55ba05652a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5ebfff640 1 -- 172.21.15.83:0/664678622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba053a59b0 con 0x55ba055c1f30 2024-04-17T10:38:27.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5ebfff640 1 --2- 172.21.15.83:0/664678622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba055c1f30 0x55ba056187e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x55ba050d58f0 tx=0x7fd5dc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6234148a89927c83 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:27.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5caffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:38:27.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5caffd640 1 -- 172.21.15.83:0/664678622 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc005bb0 con 0x55ba055c1f30 2024-04-17T10:38:27.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5caffd640 1 -- 172.21.15.83:0/664678622 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd5dc01f070 con 0x55ba055c1f30 2024-04-17T10:38:27.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5f5b27000 1 -- 172.21.15.83:0/664678622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5dc025620 con 0x55ba055c1f30 2024-04-17T10:38:27.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5caffd640 1 -- 172.21.15.83:0/664678622 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5dc03d050 con 0x55ba055c1f30 2024-04-17T10:38:27.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5f5b27000 10 client.4608 fetch_fsmap learned FSMap version 6 2024-04-17T10:38:27.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5f5b27000 1 -- 172.21.15.83:0/664678622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55ba053b80b0 con 0x55ba055c1f30 2024-04-17T10:38:27.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5caffd640 1 -- 172.21.15.83:0/664678622 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5dc025620 con 0x55ba055c1f30 2024-04-17T10:38:27.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5caffd640 1 -- 172.21.15.83:0/664678622 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 6) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd5dc040020 con 0x55ba055c1f30 2024-04-17T10:38:27.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5f5b27000 10 client.4608 fetch_fsmap finished waiting for FSMap version 6 2024-04-17T10:38:27.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5f5b27000 10 client.4608 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-17T10:38:27.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5f5b27000 20 client.4608 populate_metadata read hostname 'smithi083' 2024-04-17T10:38:27.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5f5b27000 1 --2- 172.21.15.83:0/664678622 >> [v2:172.21.15.167:6834/3871588351,v1:172.21.15.167:6835/3871588351] conn(0x55ba05668de0 0x55ba056691c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:27.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5f5b27000 4 client.4608 mds_command: new command op to 4273 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T10:38:27.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5f5b27000 1 -- 172.21.15.83:0/664678622 --> [v2:172.21.15.167:6834/3871588351,v1:172.21.15.167:6835/3871588351] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ba0541e2e0 con 0x55ba05668de0 2024-04-17T10:38:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5e37fe640 1 --2- 172.21.15.83:0/664678622 >> [v2:172.21.15.167:6834/3871588351,v1:172.21.15.167:6835/3871588351] conn(0x55ba05668de0 0x55ba056691c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5caffd640 10 client.4608 ms_handle_connect on v2:172.21.15.167:6834/3871588351 2024-04-17T10:38:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5e37fe640 1 --2- 172.21.15.83:0/664678622 >> [v2:172.21.15.167:6834/3871588351,v1:172.21.15.167:6835/3871588351] conn(0x55ba05668de0 0x55ba056691c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd7bda66669557b server_cookie=17495ba9c51cd6ae in_seq=0 out_seq=0 2024-04-17T10:38:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5caffd640 1 -- 172.21.15.83:0/664678622 <== mds.0 v2:172.21.15.167:6834/3871588351 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55ba0541e2e0 con 0x55ba05668de0 2024-04-17T10:38:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5caffd640 10 client.4608 handle_command_reply: tid=0 2024-04-17T10:38:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5f5b27000 1 client.4608 shutdown 2024-04-17T10:38:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5f5b27000 1 -- 172.21.15.83:0/664678622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba055c1f30 msgr2=0x55ba056187e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5f5b27000 1 --2- 172.21.15.83:0/664678622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba055c1f30 0x55ba056187e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x55ba050d58f0 tx=0x7fd5dc004740 comp rx=0 tx=0).stop 2024-04-17T10:38:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5f5b27000 1 -- 172.21.15.83:0/664678622 shutdown_connections 2024-04-17T10:38:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.712+0000 7fd5f5b27000 1 --2- 172.21.15.83:0/664678622 >> [v2:172.21.15.167:6834/3871588351,v1:172.21.15.167:6835/3871588351] conn(0x55ba05668de0 0x55ba056691c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.716+0000 7fd5f5b27000 1 --2- 172.21.15.83:0/664678622 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ba0560c240 0x55ba05652f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.716+0000 7fd5f5b27000 1 --2- 172.21.15.83:0/664678622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0560b920 0x55ba05652a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.716+0000 7fd5caffd640 0 client.4608 ms_handle_reset on v2:172.21.15.167:6834/3871588351 2024-04-17T10:38:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.716+0000 7fd5f5b27000 1 --2- 172.21.15.83:0/664678622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba055c1f30 0x55ba056187e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.716+0000 7fd5f5b27000 1 -- 172.21.15.83:0/664678622 >> 172.21.15.83:0/664678622 conn(0x55ba055c82a0 msgr2=0x55ba055c4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.716+0000 7fd5f5b27000 1 -- 172.21.15.83:0/664678622 shutdown_connections 2024-04-17T10:38:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.716+0000 7fd5f5b27000 1 -- 172.21.15.83:0/664678622 wait complete. 2024-04-17T10:38:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.716+0000 7fd5f5b27000 20 client.4608 trim_cache size 0 max 16384 2024-04-17T10:38:27.716 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:38:27.716 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "3de9b7be-af7f-488a-969e-2b6b8db63bcd", 2024-04-17T10:38:27.716 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-04-17T10:38:27.717 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 4273, 2024-04-17T10:38:27.717 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-17T10:38:27.717 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-17T10:38:27.717 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-17T10:38:27.717 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 15.024404321, 2024-04-17T10:38:27.717 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 6, 2024-04-17T10:38:27.717 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 20, 2024-04-17T10:38:27.717 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 20, 2024-04-17T10:38:27.717 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 21.682697045000001 2024-04-17T10:38:27.717 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:38:27.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.716+0000 7fd5f29ba640 1 -- 172.21.15.83:0/3899308427 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd5cc066f60 msgr2=0x7fd5cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:27.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.716+0000 7fd5f29ba640 1 --2- 172.21.15.83:0/3899308427 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd5cc066f60 0x7fd5cc069420 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc0097a0 tx=0x7fd5dc008040 comp rx=0 tx=0).stop 2024-04-17T10:38:27.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.716+0000 7fd5f29ba640 1 -- 172.21.15.83:0/3899308427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5ec154670 msgr2=0x7fd5ec145ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:27.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.716+0000 7fd5f29ba640 1 --2- 172.21.15.83:0/3899308427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5ec154670 0x7fd5ec145ce0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd5e401c7a0 tx=0x7fd5e401b910 comp rx=0 tx=0).stop 2024-04-17T10:38:27.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.716+0000 7fd5f29ba640 1 -- 172.21.15.83:0/3899308427 shutdown_connections 2024-04-17T10:38:27.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.716+0000 7fd5f29ba640 1 --2- 172.21.15.83:0/3899308427 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd5cc066f60 0x7fd5cc069420 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.716+0000 7fd5f29ba640 1 --2- 172.21.15.83:0/3899308427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5ec154670 0x7fd5ec145ce0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.716+0000 7fd5f29ba640 1 --2- 172.21.15.83:0/3899308427 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5ec0b4090 0x7fd5ec149b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.716+0000 7fd5f29ba640 1 --2- 172.21.15.83:0/3899308427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5ec06cc80 0x7fd5ec0adee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:27.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.716+0000 7fd5f29ba640 1 -- 172.21.15.83:0/3899308427 >> 172.21.15.83:0/3899308427 conn(0x7fd5ec0b15b0 msgr2=0x7fd5ec0af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:27.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.716+0000 7fd5f29ba640 1 -- 172.21.15.83:0/3899308427 shutdown_connections 2024-04-17T10:38:27.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:27.716+0000 7fd5f29ba640 1 -- 172.21.15.83:0/3899308427 wait complete. 2024-04-17T10:38:27.727 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-17T10:38:27.752 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-17T10:38:27.752 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-17T10:38:27.752 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-17T10:38:27.752 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-17T10:38:27.752 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T10:38:27.752 INFO:tasks.ceph_fuse:client.2 config is {} 2024-04-17T10:38:27.752 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-17T10:38:27.752 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-17T10:38:27.752 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-17T10:38:27.753 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-17T10:38:27.753 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-17T10:38:27.753 INFO:tasks.ceph_fuse:client.3 config is {} 2024-04-17T10:38:27.753 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-17T10:38:27.753 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-17T10:38:27.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:27.753 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-17T10:38:27.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:27.759 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2024-04-17T10:38:27.815 INFO:teuthology.orchestra.run.smithi083.stderr:Cannot find device "ceph-brx" 2024-04-17T10:38:27.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:38:27.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:27.817 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-04-17T10:38:27.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:27.824 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link delete ceph-brx 2024-04-17T10:38:27.882 INFO:teuthology.orchestra.run.smithi167.stderr:Cannot find device "ceph-brx" 2024-04-17T10:38:27.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:38:27.883 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-17T10:38:27.884 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T10:38:28.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.012+0000 7f3d23c64640 1 Processor -- start 2024-04-17T10:38:28.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.012+0000 7f3d23c64640 1 -- start start 2024-04-17T10:38:28.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.012+0000 7f3d23c64640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3d1c0b3990 0x7f3d1c070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:28.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.012+0000 7f3d23c64640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3d1c0709b0 0x7f3d1c06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:28.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.012+0000 7f3d23c64640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d1c070dc0 con 0x7f3d1c0cf060 2024-04-17T10:38:28.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.012+0000 7f3d23c64640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d1c0b2980 con 0x7f3d1c0b3990 2024-04-17T10:38:28.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.012+0000 7f3d23c64640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d1c0cf440 con 0x7f3d1c0709b0 2024-04-17T10:38:28.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.012+0000 7f3d21a00640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3d1c0cf060 0x7f3d1c0b4060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:43794/0 (socket says 172.21.15.83:43794) 2024-04-17T10:38:28.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.012+0000 7f3d21a00640 1 -- 172.21.15.83:0/3091726231 learned_addr learned my addr 172.21.15.83:0/3091726231 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:28.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.012+0000 7f3d211ff640 1 --2- 172.21.15.83:0/3091726231 >> v2:172.21.15.83:3300/0 conn(0x7f3d1c0b3990 0x7f3d1c070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:28.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.012+0000 7f3d22201640 1 --2- 172.21.15.83:0/3091726231 >> v2:172.21.15.167:3300/0 conn(0x7f3d1c0709b0 0x7f3d1c06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:28.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.012+0000 7f3d209fe640 1 -- 172.21.15.83:0/3091726231 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1220654832 0 0) 0x7f3d18001120 con 0x7f3d1c0cf060 2024-04-17T10:38:28.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.012+0000 7f3d209fe640 1 -- 172.21.15.83:0/3091726231 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d08004640 con 0x7f3d1c0cf060 2024-04-17T10:38:28.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.012+0000 7f3d209fe640 1 -- 172.21.15.83:0/3091726231 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4147499279 0 0) 0x7f3d08004640 con 0x7f3d1c0cf060 2024-04-17T10:38:28.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.012+0000 7f3d211ff640 1 -- 172.21.15.83:0/3091726231 >> v2:172.21.15.167:3300/0 conn(0x7f3d1c0709b0 msgr2=0x7f3d1c06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:28.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.012+0000 7f3d211ff640 1 --2- 172.21.15.83:0/3091726231 >> v2:172.21.15.167:3300/0 conn(0x7f3d1c0709b0 0x7f3d1c06c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:28.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.012+0000 7f3d211ff640 1 -- 172.21.15.83:0/3091726231 >> v1:172.21.15.83:6789/0 conn(0x7f3d1c0cf060 legacy=0x7f3d1c0b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:28.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.012+0000 7f3d211ff640 1 -- 172.21.15.83:0/3091726231 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d1c0b2bd0 con 0x7f3d1c0b3990 2024-04-17T10:38:28.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.016+0000 7f3d211ff640 1 --2- 172.21.15.83:0/3091726231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d1c0b3990 0x7f3d1c070470 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c0aebd0 tx=0x7f3d1001c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb852e9bdb780e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:28.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.016+0000 7f3d209fe640 1 -- 172.21.15.83:0/3091726231 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d100b0070 con 0x7f3d1c0b3990 2024-04-17T10:38:28.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.016+0000 7f3d209fe640 1 -- 172.21.15.83:0/3091726231 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d100ab400 con 0x7f3d1c0b3990 2024-04-17T10:38:28.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.016+0000 7f3d209fe640 1 -- 172.21.15.83:0/3091726231 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d18001850 con 0x7f3d1c0b3990 2024-04-17T10:38:28.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.016+0000 7f3d23c64640 1 -- 172.21.15.83:0/3091726231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d1c0b3990 msgr2=0x7f3d1c070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:28.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.016+0000 7f3d23c64640 1 --2- 172.21.15.83:0/3091726231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d1c0b3990 0x7f3d1c070470 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c0aebd0 tx=0x7f3d1001c2b0 comp rx=0 tx=0).stop 2024-04-17T10:38:28.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.016+0000 7f3d23c64640 1 -- 172.21.15.83:0/3091726231 shutdown_connections 2024-04-17T10:38:28.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.016+0000 7f3d23c64640 1 --2- 172.21.15.83:0/3091726231 >> v2:172.21.15.167:3300/0 conn(0x7f3d1c0709b0 0x7f3d1c06c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:28.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.016+0000 7f3d23c64640 1 --2- 172.21.15.83:0/3091726231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d1c0b3990 0x7f3d1c070470 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:28.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.016+0000 7f3d23c64640 1 -- 172.21.15.83:0/3091726231 >> 172.21.15.83:0/3091726231 conn(0x7f3d1c06abe0 msgr2=0x7f3d1c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:28.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.016+0000 7f3d23c64640 1 -- 172.21.15.83:0/3091726231 shutdown_connections 2024-04-17T10:38:28.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.016+0000 7f3d23c64640 1 -- 172.21.15.83:0/3091726231 wait complete. 2024-04-17T10:38:28.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.016+0000 7f3d23c64640 1 Processor -- start 2024-04-17T10:38:28.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.016+0000 7f3d23c64640 1 -- start start 2024-04-17T10:38:28.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.016+0000 7f3d23c64640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d1c0709b0 0x7f3d1c146cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:28.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.016+0000 7f3d23c64640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d1c0b3990 0x7f3d1c0b0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:28.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.016+0000 7f3d23c64640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d1c0cf060 0x7f3d1c1517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:28.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.016+0000 7f3d23c64640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c0b4770 con 0x7f3d1c0cf060 2024-04-17T10:38:28.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.016+0000 7f3d23c64640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c15b310 con 0x7f3d1c0b3990 2024-04-17T10:38:28.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.016+0000 7f3d23c64640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c06c9d0 con 0x7f3d1c0709b0 2024-04-17T10:38:28.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.020+0000 7f3d22201640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d1c0cf060 0x7f3d1c1517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:28.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.020+0000 7f3d22201640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d1c0cf060 0x7f3d1c1517a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38374/0 (socket says 172.21.15.83:38374) 2024-04-17T10:38:28.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.020+0000 7f3d22201640 1 -- 172.21.15.83:0/1244490173 learned_addr learned my addr 172.21.15.83:0/1244490173 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:28.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.020+0000 7f3d22201640 1 -- 172.21.15.83:0/1244490173 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d1c0709b0 msgr2=0x7f3d1c146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:28.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.020+0000 7f3d22201640 1 --2- 172.21.15.83:0/1244490173 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d1c0709b0 0x7f3d1c146cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:28.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.020+0000 7f3d22201640 1 -- 172.21.15.83:0/1244490173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d1c0b3990 msgr2=0x7f3d1c0b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:28.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.020+0000 7f3d211ff640 1 --2- 172.21.15.83:0/1244490173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d1c0b3990 0x7f3d1c0b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:28.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.020+0000 7f3d22201640 1 --2- 172.21.15.83:0/1244490173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d1c0b3990 0x7f3d1c0b0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:28.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.020+0000 7f3d22201640 1 -- 172.21.15.83:0/1244490173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d1c0aec20 con 0x7f3d1c0cf060 2024-04-17T10:38:28.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.020+0000 7f3d22201640 1 --2- 172.21.15.83:0/1244490173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d1c0cf060 0x7f3d1c1517a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3d1400efc0 tx=0x7f3d1400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:28.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.020+0000 7f3d0e7fc640 1 -- 172.21.15.83:0/1244490173 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d14018020 con 0x7f3d1c0cf060 2024-04-17T10:38:28.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.020+0000 7f3d0e7fc640 1 -- 172.21.15.83:0/1244490173 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d14014480 con 0x7f3d1c0cf060 2024-04-17T10:38:28.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.020+0000 7f3d23c64640 1 -- 172.21.15.83:0/1244490173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d1c0aef00 con 0x7f3d1c0cf060 2024-04-17T10:38:28.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.020+0000 7f3d0e7fc640 1 -- 172.21.15.83:0/1244490173 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d14014870 con 0x7f3d1c0cf060 2024-04-17T10:38:28.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.020+0000 7f3d23c64640 1 -- 172.21.15.83:0/1244490173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d1c0af4c0 con 0x7f3d1c0cf060 2024-04-17T10:38:28.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.020+0000 7f3d23c64640 1 -- 172.21.15.83:0/1244490173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d1c074f90 con 0x7f3d1c0cf060 2024-04-17T10:38:28.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.020+0000 7f3d0e7fc640 1 -- 172.21.15.83:0/1244490173 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3d1401b020 con 0x7f3d1c0cf060 2024-04-17T10:38:28.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.020+0000 7f3d0e7fc640 1 --2- 172.21.15.83:0/1244490173 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3d00066f90 0x7f3d00069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:28.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.020+0000 7f3d0e7fc640 1 -- 172.21.15.83:0/1244490173 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3d14085b50 con 0x7f3d1c0cf060 2024-04-17T10:38:28.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.020+0000 7f3d21a00640 1 --2- 172.21.15.83:0/1244490173 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3d00066f90 0x7f3d00069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:28.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.020+0000 7f3d21a00640 1 --2- 172.21.15.83:0/1244490173 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3d00066f90 0x7f3d00069450 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3d18005890 tx=0x7f3d180016b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:28.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.024+0000 7f3d0e7fc640 1 -- 172.21.15.83:0/1244490173 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3d14022070 con 0x7f3d1c0cf060 2024-04-17T10:38:28.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.204+0000 7f3d23c64640 1 -- 172.21.15.83:0/1244490173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3d1c153fd0 con 0x7f3d1c0cf060 2024-04-17T10:38:28.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.204+0000 7f3d0e7fc640 1 -- 172.21.15.83:0/1244490173 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3d14010040 con 0x7f3d1c0cf060 2024-04-17T10:38:28.201 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T10:38:28.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.204+0000 7f3d23c64640 1 -- 172.21.15.83:0/1244490173 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3d00066f90 msgr2=0x7f3d00069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:28.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.204+0000 7f3d23c64640 1 --2- 172.21.15.83:0/1244490173 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3d00066f90 0x7f3d00069450 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3d18005890 tx=0x7f3d180016b0 comp rx=0 tx=0).stop 2024-04-17T10:38:28.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.208+0000 7f3d23c64640 1 -- 172.21.15.83:0/1244490173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d1c0cf060 msgr2=0x7f3d1c1517a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:28.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.208+0000 7f3d23c64640 1 --2- 172.21.15.83:0/1244490173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d1c0cf060 0x7f3d1c1517a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3d1400efc0 tx=0x7f3d1400c490 comp rx=0 tx=0).stop 2024-04-17T10:38:28.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.208+0000 7f3d23c64640 1 -- 172.21.15.83:0/1244490173 shutdown_connections 2024-04-17T10:38:28.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.208+0000 7f3d23c64640 1 --2- 172.21.15.83:0/1244490173 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3d00066f90 0x7f3d00069450 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:28.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.208+0000 7f3d23c64640 1 --2- 172.21.15.83:0/1244490173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d1c0cf060 0x7f3d1c1517a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:28.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.208+0000 7f3d23c64640 1 --2- 172.21.15.83:0/1244490173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d1c0b3990 0x7f3d1c0b0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:28.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.208+0000 7f3d23c64640 1 --2- 172.21.15.83:0/1244490173 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d1c0709b0 0x7f3d1c146cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:28.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.208+0000 7f3d23c64640 1 -- 172.21.15.83:0/1244490173 >> 172.21.15.83:0/1244490173 conn(0x7f3d1c06abe0 msgr2=0x7f3d1c1593f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:28.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.208+0000 7f3d23c64640 1 -- 172.21.15.83:0/1244490173 shutdown_connections 2024-04-17T10:38:28.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:28.208+0000 7f3d23c64640 1 -- 172.21.15.83:0/1244490173 wait complete. 2024-04-17T10:38:28.218 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T10:38:28.218 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T10:38:28.218 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-04-17T10:38:28.218 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T10:38:28.218 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T10:38:28.218 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T10:38:28.218 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T10:38:28.218 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T10:38:28.218 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:38:28.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:28.219 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-04-17T10:38:28.225 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T10:38:28.225 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T10:38:28.225 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T10:38:28.225 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:28.225 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-17T10:38:28.225 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:28.225 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:38:28.225 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-04-17T10:38:28.225 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-04-17T10:38:28.225 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:38:28.226 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-04-17T10:38:28.226 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-04-17T10:38:28.226 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:38:28.226 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:38:28.226 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T10:38:28.226 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:38:28.226 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-17T10:38:28.226 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42476sec preferred_lft 42476sec 2024-04-17T10:38:28.226 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-04-17T10:38:28.226 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:28.226 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T10:38:28.226 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:37:0e:23:b2 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:38:28.226 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T10:38:28.227 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:28.227 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-17T10:38:28.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:28.227 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2024-04-17T10:38:28.227 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:38:28.227 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add name ceph-brx type bridge 2024-04-17T10:38:28.227 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip addr flush dev ceph-brx 2024-04-17T10:38:28.227 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set ceph-brx up 2024-04-17T10:38:28.228 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-17T10:38:28.228 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:38:28.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:28.304 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T10:38:28.355 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-04-17T10:38:28.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:28.357 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-04-17T10:38:28.426 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-04-17T10:38:28.426 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T10:38:28.426 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-17T10:38:28.426 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-17T10:38:28.426 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-17T10:38:28.426 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T10:38:28.426 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-17T10:38:28.427 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:38:28.427 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:38:28.427 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:38:28.427 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T10:38:28.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:28.427 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2024-04-17T10:38:28.427 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:38:28.428 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-17T10:38:28.428 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-17T10:38:28.428 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-17T10:38:28.428 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:38:28.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:28.501 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-17T10:38:28.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:28.547 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list-id 2024-04-17T10:38:28.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:28.595 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2024-04-17T10:38:28.595 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:38:28.595 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:38:28.595 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-17T10:38:28.596 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:38:28.665 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-17T10:38:28.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:28.666 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2024-04-17T10:38:28.666 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:38:28.666 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-17T10:38:28.666 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-17T10:38:28.666 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-17T10:38:28.666 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-17T10:38:28.666 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-17T10:38:28.666 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:38:28.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:28.769 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2024-04-17T10:38:28.769 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:38:28.769 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-04-17T10:38:28.769 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set dev brx.0 master ceph-brx 2024-04-17T10:38:28.769 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:38:28.787 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-17T10:38:28.788 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:38:28.788 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T10:38:28.834 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:38:28.834 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:38:28.835 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:38:28.882 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:38:28.936 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:38:28.984 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:38:28.984 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:38:28.984 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:38:28.984 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:38:28.984 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:38:28.984 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:38:28.984 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:38:28.984 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:38:28.984 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:38:28.985 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:38:28.985 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:38:28.985 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:38:28.985 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:38:28.985 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:38:28.985 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:38:28.985 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:38:28.985 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:38:28.985 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:38:28.985 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:38:28.985 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:38:28.985 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:38:28.985 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:38:28.986 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:38:28.986 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:38:28.986 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:38:28.986 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:38:28.986 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:38:28.986 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:38:28.986 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:38:28.986 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:38:28.986 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:38:28.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:28.987 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:38:29.035 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T10:38:29.035 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-17T10:38:29.080 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:38:29.093 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:38:29.139 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:38:29.139 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:38:29.140 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:38:29.140 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:38:29.140 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:38:29.140 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:38:29.140 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:38:29.140 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:38:29.140 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:38:29.140 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:38:29.140 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:38:29.140 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:38:29.140 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:38:29.140 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:38:29.140 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:38:29.140 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:38:29.141 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:38:29.141 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:38:29.141 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:38:29.141 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:38:29.141 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:38:29.141 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:38:29.141 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:38:29.141 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:38:29.141 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:38:29.141 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:38:29.141 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:38:29.141 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:38:29.141 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:38:29.141 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:38:29.142 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:38:29.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:29.143 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:38:29.157 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:2024-04-17T10:38:29.160+0000 7f7fb85d0580 -1 init, newargv = 0x55e8055d02d0 newargc=15 2024-04-17T10:38:29.157 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[34847]: starting ceph client 2024-04-17T10:38:29.165 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[34847]: starting fuse 2024-04-17T10:38:30.262 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:38:30.277 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:38:30.326 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:38:30.326 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:38:30.326 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:38:30.326 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:38:30.326 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:38:30.326 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:38:30.326 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:38:30.327 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:38:30.327 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:38:30.327 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:38:30.327 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:38:30.327 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:38:30.327 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:38:30.327 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:38:30.327 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:38:30.327 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:38:30.327 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:38:30.327 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:38:30.327 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:38:30.328 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:38:30.328 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:38:30.328 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:38:30.328 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:38:30.328 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:38:30.328 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:38:30.328 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:38:30.328 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:38:30.328 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:38:30.328 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:38:30.328 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:38:30.328 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:38:30.328 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-17T10:38:30.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:30.329 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:38:30.379 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-04-17T10:38:30.379 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-17T10:38:30.379 DEBUG:teuthology.orchestra.run.smithi083:> sudo stdin-killer -- python3 -c ' 2024-04-17T10:38:30.379 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2024-04-17T10:38:30.380 DEBUG:teuthology.orchestra.run.smithi083:> import re 2024-04-17T10:38:30.380 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-04-17T10:38:30.380 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2024-04-17T10:38:30.380 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:38:30.380 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2024-04-17T10:38:30.380 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-17T10:38:30.380 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2024-04-17T10:38:30.380 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-17T10:38:30.380 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:38:30.380 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2024-04-17T10:38:30.380 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2024-04-17T10:38:30.380 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2024-04-17T10:38:30.380 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2024-04-17T10:38:30.380 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:38:30.380 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2024-04-17T10:38:30.381 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T10:38:30.381 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T10:38:30.381 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T10:38:30.381 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2024-04-17T10:38:30.381 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2024-04-17T10:38:30.381 DEBUG:teuthology.orchestra.run.smithi083:> return f 2024-04-17T10:38:30.381 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T10:38:30.381 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:38:30.381 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.0")) 2024-04-17T10:38:30.381 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-04-17T10:38:30.502 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.0.34847.asok 2024-04-17T10:38:30.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:38:30.514 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.34847.asok 2024-04-17T10:38:30.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:30.514 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.34847.asok status 2024-04-17T10:38:30.672 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:38:30.673 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2024-04-17T10:38:30.673 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-17T10:38:30.673 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-17T10:38:30.673 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "0", 2024-04-17T10:38:30.673 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-04-17T10:38:30.673 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-17T10:38:30.673 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "34847", 2024-04-17T10:38:30.673 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-04-17T10:38:30.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:38:30.673 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2024-04-17T10:38:30.673 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2024-04-17T10:38:30.674 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 4623, 2024-04-17T10:38:30.674 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2024-04-17T10:38:30.674 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-04-17T10:38:30.674 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-04-17T10:38:30.674 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 4623 2024-04-17T10:38:30.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:38:30.674 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-04-17T10:38:30.674 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-17T10:38:30.674 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:38:30.675 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2137746812 2024-04-17T10:38:30.675 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:38:30.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:38:30.675 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-04-17T10:38:30.675 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-17T10:38:30.675 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:38:30.675 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2137746812 2024-04-17T10:38:30.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:38:30.675 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.4623 192.168.0.1:0/2137746812", 2024-04-17T10:38:30.675 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.1:0/2137746812", 2024-04-17T10:38:30.675 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 1, 2024-04-17T10:38:30.675 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 6, 2024-04-17T10:38:30.675 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 20, 2024-04-17T10:38:30.675 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2024-04-17T10:38:30.676 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2024-04-17T10:38:30.676 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2024-04-17T10:38:30.676 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:38:30.690 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T10:38:30.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f6248f1c640 1 Processor -- start 2024-04-17T10:38:30.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f6248f1c640 1 -- start start 2024-04-17T10:38:30.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f6248f1c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62440cfbe0 0x7f624406c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:30.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f6248f1c640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f624406e320 con 0x7f6244072330 2024-04-17T10:38:30.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f6248f1c640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f624406e040 con 0x7f624415ba30 2024-04-17T10:38:30.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f6248f1c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6244071ea0 con 0x7f62440cfbe0 2024-04-17T10:38:30.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f6241d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6244072330 0x7f62440cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:43800/0 (socket says 172.21.15.83:43800) 2024-04-17T10:38:30.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f6241d9b640 1 -- 172.21.15.83:0/3635688329 learned_addr learned my addr 172.21.15.83:0/3635688329 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:30.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f6242d9d640 1 --2- 172.21.15.83:0/3635688329 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62440cfbe0 0x7f624406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:30.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f624159a640 1 -- 172.21.15.83:0/3635688329 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 217678112 0 0) 0x7f624406e320 con 0x7f6244072330 2024-04-17T10:38:30.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f624159a640 1 -- 172.21.15.83:0/3635688329 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f622c003530 con 0x7f6244072330 2024-04-17T10:38:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f624159a640 1 -- 172.21.15.83:0/3635688329 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 818220218 0 0) 0x7f622c003530 con 0x7f6244072330 2024-04-17T10:38:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f624159a640 1 -- 172.21.15.83:0/3635688329 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62440cfbe0 msgr2=0x7f624406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f624159a640 1 --2- 172.21.15.83:0/3635688329 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62440cfbe0 0x7f624406c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f624159a640 1 -- 172.21.15.83:0/3635688329 >> v1:172.21.15.167:6789/0 conn(0x7f624415ba30 legacy=0x7f624415c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f624159a640 1 -- 172.21.15.83:0/3635688329 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6244071850 con 0x7f6244072330 2024-04-17T10:38:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f624159a640 1 -- 172.21.15.83:0/3635688329 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f6238001680 con 0x7f6244072330 2024-04-17T10:38:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f624159a640 1 -- 172.21.15.83:0/3635688329 >> v1:172.21.15.83:6789/0 conn(0x7f6244072330 legacy=0x7f62440cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f624159a640 1 --2- 172.21.15.83:0/3635688329 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62440cfbe0 0x7f622c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f624159a640 1 --2- 172.21.15.83:0/3635688329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f622c003f70 0x7f622c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f624159a640 1 --2- 172.21.15.83:0/3635688329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f622c004d50 0x7f622c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f624159a640 1 -- 172.21.15.83:0/3635688329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6244071ea0 con 0x7f622c003f70 2024-04-17T10:38:30.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f624159a640 1 -- 172.21.15.83:0/3635688329 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c00bb60 con 0x7f622c004d50 2024-04-17T10:38:30.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f624159a640 1 -- 172.21.15.83:0/3635688329 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f622c00bd60 con 0x7f62440cfbe0 2024-04-17T10:38:30.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f6241d9b640 1 --2- 172.21.15.83:0/3635688329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f622c003f70 0x7f622c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:30.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f624259c640 1 --2- 172.21.15.83:0/3635688329 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62440cfbe0 0x7f622c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:30.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f6242d9d640 1 --2- 172.21.15.83:0/3635688329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f622c004d50 0x7f622c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:30.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f6241d9b640 1 -- 172.21.15.83:0/3635688329 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62440cfbe0 msgr2=0x7f622c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:30.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f6241d9b640 1 --2- 172.21.15.83:0/3635688329 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62440cfbe0 0x7f622c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:30.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f6241d9b640 1 -- 172.21.15.83:0/3635688329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f622c004d50 msgr2=0x7f622c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:30.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f6241d9b640 1 --2- 172.21.15.83:0/3635688329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f622c004d50 0x7f622c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:30.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f6241d9b640 1 -- 172.21.15.83:0/3635688329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f622c003530 con 0x7f622c003f70 2024-04-17T10:38:30.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.820+0000 7f6242d9d640 1 --2- 172.21.15.83:0/3635688329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f622c004d50 0x7f622c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:30.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.824+0000 7f6241d9b640 1 --2- 172.21.15.83:0/3635688329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f622c003f70 0x7f622c004810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f623800c240 tx=0x7f623801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b981a6b5f7cc1e15 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:30.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.824+0000 7f624159a640 1 -- 172.21.15.83:0/3635688329 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6238007d20 con 0x7f622c003f70 2024-04-17T10:38:30.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.824+0000 7f624159a640 1 -- 172.21.15.83:0/3635688329 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62380a9070 con 0x7f622c003f70 2024-04-17T10:38:30.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.824+0000 7f624159a640 1 -- 172.21.15.83:0/3635688329 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6238005d60 con 0x7f622c003f70 2024-04-17T10:38:30.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.824+0000 7f6248f1c640 1 -- 172.21.15.83:0/3635688329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f622c003f70 msgr2=0x7f622c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:30.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.824+0000 7f6248f1c640 1 --2- 172.21.15.83:0/3635688329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f622c003f70 0x7f622c004810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f623800c240 tx=0x7f623801cc80 comp rx=0 tx=0).stop 2024-04-17T10:38:30.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.824+0000 7f6242d9d640 1 -- 172.21.15.83:0/3635688329 reap_dead start 2024-04-17T10:38:30.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.824+0000 7f6248f1c640 1 -- 172.21.15.83:0/3635688329 shutdown_connections 2024-04-17T10:38:30.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.824+0000 7f6248f1c640 1 -- 172.21.15.83:0/3635688329 >> 172.21.15.83:0/3635688329 conn(0x7f624406abe0 msgr2=0x7f624406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:30.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.824+0000 7f6248f1c640 1 -- 172.21.15.83:0/3635688329 shutdown_connections 2024-04-17T10:38:30.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.824+0000 7f6248f1c640 1 -- 172.21.15.83:0/3635688329 wait complete. 2024-04-17T10:38:30.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.824+0000 7f6248f1c640 1 Processor -- start 2024-04-17T10:38:30.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.824+0000 7f6248f1c640 1 -- start start 2024-04-17T10:38:30.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.824+0000 7f6248f1c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f622c003f70 0x7f6244147e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:30.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.824+0000 7f6248f1c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6244148470 0x7f62440cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:30.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.824+0000 7f6248f1c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62440ad840 0x7f62440adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:30.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.824+0000 7f6248f1c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624406ce20 con 0x7f6244148470 2024-04-17T10:38:30.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.824+0000 7f6248f1c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624406cca0 con 0x7f622c003f70 2024-04-17T10:38:30.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.824+0000 7f6248f1c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f624406cfa0 con 0x7f62440ad840 2024-04-17T10:38:30.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.828+0000 7f6241d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6244148470 0x7f62440cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:30.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.828+0000 7f6241d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6244148470 0x7f62440cecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38390/0 (socket says 172.21.15.83:38390) 2024-04-17T10:38:30.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.828+0000 7f6241d9b640 1 -- 172.21.15.83:0/3653159667 learned_addr learned my addr 172.21.15.83:0/3653159667 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:30.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.828+0000 7f624259c640 1 --2- 172.21.15.83:0/3653159667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f622c003f70 0x7f6244147e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:30.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.828+0000 7f6242d9d640 1 --2- 172.21.15.83:0/3653159667 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62440ad840 0x7f62440adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:30.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.828+0000 7f6241d9b640 1 -- 172.21.15.83:0/3653159667 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62440ad840 msgr2=0x7f62440adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:30.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.828+0000 7f6241d9b640 1 --2- 172.21.15.83:0/3653159667 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62440ad840 0x7f62440adcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:30.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.828+0000 7f6241d9b640 1 -- 172.21.15.83:0/3653159667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f622c003f70 msgr2=0x7f6244147e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:30.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.828+0000 7f6241d9b640 1 --2- 172.21.15.83:0/3653159667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f622c003f70 0x7f6244147e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:30.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.828+0000 7f6241d9b640 1 -- 172.21.15.83:0/3653159667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6244150b70 con 0x7f6244148470 2024-04-17T10:38:30.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.828+0000 7f624259c640 1 --2- 172.21.15.83:0/3653159667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f622c003f70 0x7f6244147e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:38:30.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.828+0000 7f6241d9b640 1 --2- 172.21.15.83:0/3653159667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6244148470 0x7f62440cecf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6238098590 tx=0x7f622c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:30.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.828+0000 7f6232ffd640 1 -- 172.21.15.83:0/3653159667 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6238007d20 con 0x7f6244148470 2024-04-17T10:38:30.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.828+0000 7f6232ffd640 1 -- 172.21.15.83:0/3653159667 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62380a5870 con 0x7f6244148470 2024-04-17T10:38:30.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.828+0000 7f6248f1c640 1 -- 172.21.15.83:0/3653159667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6244150e10 con 0x7f6244148470 2024-04-17T10:38:30.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.828+0000 7f6232ffd640 1 -- 172.21.15.83:0/3653159667 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62380a5c60 con 0x7f6244148470 2024-04-17T10:38:30.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.828+0000 7f6248f1c640 1 -- 172.21.15.83:0/3653159667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62441513d0 con 0x7f6244148470 2024-04-17T10:38:30.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.828+0000 7f6248f1c640 1 -- 172.21.15.83:0/3653159667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6244074f90 con 0x7f6244148470 2024-04-17T10:38:30.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.828+0000 7f6232ffd640 1 -- 172.21.15.83:0/3653159667 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f62380b7030 con 0x7f6244148470 2024-04-17T10:38:30.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.828+0000 7f6232ffd640 1 --2- 172.21.15.83:0/3653159667 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6224066f60 0x7f6224069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:30.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.828+0000 7f6232ffd640 1 -- 172.21.15.83:0/3653159667 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f623811e1c0 con 0x7f6244148470 2024-04-17T10:38:30.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.828+0000 7f624259c640 1 --2- 172.21.15.83:0/3653159667 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6224066f60 0x7f6224069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:30.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.828+0000 7f624259c640 1 --2- 172.21.15.83:0/3653159667 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6224066f60 0x7f6224069420 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6234000c10 tx=0x7f6234009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:30.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:30.832+0000 7f6232ffd640 1 -- 172.21.15.83:0/3653159667 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6238016990 con 0x7f6244148470 2024-04-17T10:38:31.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:31.028+0000 7f6248f1c640 1 -- 172.21.15.83:0/3653159667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6244158410 con 0x7f6244148470 2024-04-17T10:38:31.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:31.028+0000 7f6232ffd640 1 -- 172.21.15.83:0/3653159667 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f62380ef1d0 con 0x7f6244148470 2024-04-17T10:38:31.025 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T10:38:31.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:31.028+0000 7f6248f1c640 1 -- 172.21.15.83:0/3653159667 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6224066f60 msgr2=0x7f6224069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:31.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:31.028+0000 7f6248f1c640 1 --2- 172.21.15.83:0/3653159667 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6224066f60 0x7f6224069420 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6234000c10 tx=0x7f6234009040 comp rx=0 tx=0).stop 2024-04-17T10:38:31.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:31.028+0000 7f6248f1c640 1 -- 172.21.15.83:0/3653159667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6244148470 msgr2=0x7f62440cecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:31.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:31.028+0000 7f6248f1c640 1 --2- 172.21.15.83:0/3653159667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6244148470 0x7f62440cecf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6238098590 tx=0x7f622c005840 comp rx=0 tx=0).stop 2024-04-17T10:38:31.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:31.032+0000 7f6248f1c640 1 -- 172.21.15.83:0/3653159667 shutdown_connections 2024-04-17T10:38:31.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:31.032+0000 7f6248f1c640 1 --2- 172.21.15.83:0/3653159667 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6224066f60 0x7f6224069420 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:31.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:31.032+0000 7f6248f1c640 1 --2- 172.21.15.83:0/3653159667 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62440ad840 0x7f62440adcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:31.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:31.032+0000 7f6248f1c640 1 --2- 172.21.15.83:0/3653159667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6244148470 0x7f62440cecf0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:31.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:31.032+0000 7f6248f1c640 1 --2- 172.21.15.83:0/3653159667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f622c003f70 0x7f6244147e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:31.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:31.032+0000 7f6248f1c640 1 -- 172.21.15.83:0/3653159667 >> 172.21.15.83:0/3653159667 conn(0x7f624406abe0 msgr2=0x7f624415a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:31.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:31.032+0000 7f6248f1c640 1 -- 172.21.15.83:0/3653159667 shutdown_connections 2024-04-17T10:38:31.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:31.032+0000 7f6248f1c640 1 -- 172.21.15.83:0/3653159667 wait complete. 2024-04-17T10:38:31.043 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T10:38:31.043 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T10:38:31.043 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-04-17T10:38:31.043 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-17T10:38:31.043 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-17T10:38:31.043 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T10:38:31.043 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T10:38:31.043 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T10:38:31.043 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-17T10:38:31.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:31.044 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-04-17T10:38:31.050 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T10:38:31.050 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T10:38:31.050 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T10:38:31.050 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:31.050 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-17T10:38:31.050 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:31.050 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:38:31.050 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-04-17T10:38:31.051 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-04-17T10:38:31.051 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:38:31.051 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-04-17T10:38:31.051 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-04-17T10:38:31.051 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:38:31.051 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:38:31.051 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T10:38:31.051 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:38:31.051 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-17T10:38:31.051 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42473sec preferred_lft 42473sec 2024-04-17T10:38:31.051 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-04-17T10:38:31.051 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:31.051 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T10:38:31.051 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:37:0e:23:b2 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:38:31.052 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T10:38:31.052 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:31.052 INFO:teuthology.orchestra.run.smithi083.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T10:38:31.052 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-17T10:38:31.052 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T10:38:31.052 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:31.052 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-17T10:38:31.052 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:31.052 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:38:31.052 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:38:31.052 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-17T10:38:31.053 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:31.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:31.053 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T10:38:31.109 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-04-17T10:38:31.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:31.111 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-04-17T10:38:31.167 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-04-17T10:38:31.168 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T10:38:31.168 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-17T10:38:31.168 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-17T10:38:31.168 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-17T10:38:31.168 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T10:38:31.168 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-17T10:38:31.168 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:38:31.168 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:38:31.168 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:38:31.168 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T10:38:31.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:31.169 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2024-04-17T10:38:31.169 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:38:31.169 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-17T10:38:31.169 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-17T10:38:31.169 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-17T10:38:31.169 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:38:31.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:31.246 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-17T10:38:31.299 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T10:38:31.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:31.300 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list-id 2024-04-17T10:38:31.349 INFO:teuthology.orchestra.run.smithi083.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-17T10:38:31.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:31.350 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2024-04-17T10:38:31.350 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:38:31.350 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:38:31.350 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-17T10:38:31.350 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:38:31.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:31.424 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-17T10:38:31.480 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T10:38:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T10:38:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T10:38:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-17T10:38:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:31.480 INFO:teuthology.orchestra.run.smithi083.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T10:38:31.480 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 46:99:4a:e7:6b:b3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-17T10:38:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-17T10:38:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::4499:4aff:fee7:6bb3/64 scope link 2024-04-17T10:38:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:31.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:31.482 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-17T10:38:31.545 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T10:38:31.545 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T10:38:31.545 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T10:38:31.545 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:31.545 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-17T10:38:31.546 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:31.546 INFO:teuthology.orchestra.run.smithi083.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T10:38:31.546 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 46:99:4a:e7:6b:b3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-17T10:38:31.546 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-17T10:38:31.546 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:31.546 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::4499:4aff:fee7:6bb3/64 scope link 2024-04-17T10:38:31.546 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:31.548 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-17T10:38:31.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:31.548 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2024-04-17T10:38:31.548 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:38:31.548 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-17T10:38:31.549 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-17T10:38:31.549 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-17T10:38:31.549 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-17T10:38:31.549 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-17T10:38:31.549 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:38:31.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:31.645 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2024-04-17T10:38:31.645 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:38:31.645 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.1 up 2024-04-17T10:38:31.645 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set dev brx.1 master ceph-brx 2024-04-17T10:38:31.645 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:38:31.664 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-17T10:38:31.665 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:38:31.665 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-17T10:38:31.710 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-17T10:38:31.711 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:38:31.711 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-17T10:38:31.758 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:38:31.815 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:38:31.864 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:38:31.864 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:38:31.864 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:38:31.864 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:38:31.864 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:38:31.864 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:38:31.864 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:38:31.864 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:38:31.864 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:38:31.864 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:38:31.865 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:38:31.865 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:38:31.865 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:38:31.865 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:38:31.865 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:38:31.865 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:38:31.865 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:38:31.865 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:38:31.865 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:38:31.865 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:38:31.865 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:38:31.865 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:38:31.865 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:38:31.866 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:38:31.866 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:38:31.866 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:38:31.866 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:38:31.866 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:38:31.866 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:38:31.866 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:38:31.866 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:38:31.866 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-17T10:38:31.866 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:38:31.866 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:38:31.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:31.867 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:38:31.915 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-04-17T10:38:31.915 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-04-17T10:38:31.916 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-04-17T10:38:31.960 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:38:31.974 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:38:32.023 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:38:32.023 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:38:32.023 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:38:32.023 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:38:32.023 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:38:32.023 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:38:32.024 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:38:32.024 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:38:32.024 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:38:32.024 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:38:32.024 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:38:32.024 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:38:32.024 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:38:32.024 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:38:32.024 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:38:32.024 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:38:32.024 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:38:32.024 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:38:32.025 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:38:32.025 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:38:32.025 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:38:32.025 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:38:32.025 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:38:32.025 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:38:32.025 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:38:32.025 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:38:32.025 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:38:32.025 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:38:32.025 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:38:32.025 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:38:32.025 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:38:32.025 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-17T10:38:32.026 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:38:32.026 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:38:32.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:32.027 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:38:32.036 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi083.stderr:2024-04-17T10:38:32.040+0000 7faa448c7580 -1 init, newargv = 0x563a7a24c170 newargc=15 2024-04-17T10:38:32.036 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi083.stderr:ceph-fuse[34988]: starting ceph client 2024-04-17T10:38:32.040 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-04-17T10:38:32.043 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi083.stderr:ceph-fuse[34988]: starting fuse 2024-04-17T10:38:33.142 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:38:33.157 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:38:33.206 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:38:33.206 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:38:33.206 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:38:33.206 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:38:33.206 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:38:33.206 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:38:33.206 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:38:33.206 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:38:33.207 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:38:33.207 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:38:33.207 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:38:33.207 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:38:33.207 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:38:33.207 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:38:33.207 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:38:33.207 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:38:33.207 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:38:33.207 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:38:33.207 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:38:33.207 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:38:33.207 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:38:33.207 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:38:33.208 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:38:33.208 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:38:33.208 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:38:33.208 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:38:33.208 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:38:33.208 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:38:33.208 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:38:33.208 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:38:33.208 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:38:33.208 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-17T10:38:33.208 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:38:33.208 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:38:33.208 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.2 2024-04-17T10:38:33.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:33.209 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:38:33.255 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-04-17T10:38:33.256 INFO:teuthology.orchestra.run.smithi083.stdout:46 2024-04-17T10:38:33.256 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-04-17T10:38:33.256 DEBUG:teuthology.orchestra.run.smithi083:> sudo stdin-killer -- python3 -c ' 2024-04-17T10:38:33.256 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2024-04-17T10:38:33.256 DEBUG:teuthology.orchestra.run.smithi083:> import re 2024-04-17T10:38:33.256 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-04-17T10:38:33.256 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2024-04-17T10:38:33.256 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:38:33.256 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2024-04-17T10:38:33.256 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-17T10:38:33.256 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2024-04-17T10:38:33.257 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-17T10:38:33.257 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:38:33.257 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2024-04-17T10:38:33.257 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2024-04-17T10:38:33.257 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2024-04-17T10:38:33.257 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2024-04-17T10:38:33.257 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:38:33.257 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2024-04-17T10:38:33.257 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T10:38:33.257 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T10:38:33.257 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T10:38:33.257 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2024-04-17T10:38:33.257 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2024-04-17T10:38:33.257 DEBUG:teuthology.orchestra.run.smithi083:> return f 2024-04-17T10:38:33.258 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T10:38:33.258 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:38:33.258 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.2")) 2024-04-17T10:38:33.258 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-04-17T10:38:33.378 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.2.34988.asok 2024-04-17T10:38:33.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:38:33.389 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.34988.asok 2024-04-17T10:38:33.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:33.389 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.34988.asok status 2024-04-17T10:38:33.547 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:38:33.547 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2024-04-17T10:38:33.547 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-17T10:38:33.547 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-17T10:38:33.547 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "2", 2024-04-17T10:38:33.547 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-04-17T10:38:33.547 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-17T10:38:33.547 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "34988", 2024-04-17T10:38:33.547 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-04-17T10:38:33.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:38:33.548 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2024-04-17T10:38:33.548 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2024-04-17T10:38:33.548 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 4638, 2024-04-17T10:38:33.548 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2024-04-17T10:38:33.548 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-04-17T10:38:33.548 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-04-17T10:38:33.548 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 4638 2024-04-17T10:38:33.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:38:33.548 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-04-17T10:38:33.548 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-17T10:38:33.548 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.2:0", 2024-04-17T10:38:33.548 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2970944230 2024-04-17T10:38:33.549 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:38:33.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:38:33.549 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-04-17T10:38:33.549 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-17T10:38:33.549 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.2:0", 2024-04-17T10:38:33.549 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2970944230 2024-04-17T10:38:33.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:38:33.549 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.4638 192.168.0.2:0/2970944230", 2024-04-17T10:38:33.549 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.2:0/2970944230", 2024-04-17T10:38:33.549 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 1, 2024-04-17T10:38:33.549 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 6, 2024-04-17T10:38:33.549 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 20, 2024-04-17T10:38:33.549 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2024-04-17T10:38:33.549 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2024-04-17T10:38:33.549 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2024-04-17T10:38:33.550 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:38:33.560 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T10:38:33.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.688+0000 7f421df17640 1 Processor -- start 2024-04-17T10:38:33.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.688+0000 7f421df17640 1 -- start start 2024-04-17T10:38:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.688+0000 7f421df17640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f42180cf060 0x7f42180b4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.688+0000 7f421df17640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f42180b3990 0x7f4218070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.688+0000 7f421df17640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42180b2bd0 con 0x7f42180b45a0 2024-04-17T10:38:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.688+0000 7f421df17640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42180b2980 con 0x7f42180b3990 2024-04-17T10:38:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.688+0000 7f421df17640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42180b2eb0 con 0x7f42180cf060 2024-04-17T10:38:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.688+0000 7f4216ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f42180b3990 0x7f4218070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.688+0000 7f4217fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f42180b45a0 0x7f421815d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:43812/0 (socket says 172.21.15.83:43812) 2024-04-17T10:38:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.688+0000 7f4217fff640 1 -- 172.21.15.83:0/1279527851 learned_addr learned my addr 172.21.15.83:0/1279527851 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.688+0000 7f42177fe640 1 --2- 172.21.15.83:0/1279527851 >> v2:172.21.15.167:3300/0 conn(0x7f42180cf060 0x7f42180b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.688+0000 7f4216ffd640 1 -- 172.21.15.83:0/1279527851 >> v2:172.21.15.167:3300/0 conn(0x7f42180cf060 msgr2=0x7f42180b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.688+0000 7f4216ffd640 1 --2- 172.21.15.83:0/1279527851 >> v2:172.21.15.167:3300/0 conn(0x7f42180cf060 0x7f42180b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.688+0000 7f4216ffd640 1 -- 172.21.15.83:0/1279527851 >> v1:172.21.15.83:6789/0 conn(0x7f42180b45a0 legacy=0x7f421815d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.688+0000 7f4216ffd640 1 -- 172.21.15.83:0/1279527851 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42180b3190 con 0x7f42180b3990 2024-04-17T10:38:33.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f4216ffd640 1 --2- 172.21.15.83:0/1279527851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42180b3990 0x7f4218070470 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f421806e6c0 tx=0x7f420001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9251da0a2e3e8685 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:33.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f42167fc640 1 -- 172.21.15.83:0/1279527851 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42000b2070 con 0x7f42180b3990 2024-04-17T10:38:33.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f42167fc640 1 -- 172.21.15.83:0/1279527851 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42000a9070 con 0x7f42180b3990 2024-04-17T10:38:33.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f42167fc640 1 -- 172.21.15.83:0/1279527851 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42000ad880 con 0x7f42180b3990 2024-04-17T10:38:33.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f421df17640 1 -- 172.21.15.83:0/1279527851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42180b3990 msgr2=0x7f4218070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:33.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f421df17640 1 --2- 172.21.15.83:0/1279527851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42180b3990 0x7f4218070470 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f421806e6c0 tx=0x7f420001c450 comp rx=0 tx=0).stop 2024-04-17T10:38:33.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f421df17640 1 -- 172.21.15.83:0/1279527851 shutdown_connections 2024-04-17T10:38:33.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f421df17640 1 --2- 172.21.15.83:0/1279527851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42180b3990 0x7f4218070470 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:33.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f421df17640 1 --2- 172.21.15.83:0/1279527851 >> v2:172.21.15.167:3300/0 conn(0x7f42180cf060 0x7f42180b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:33.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f421df17640 1 -- 172.21.15.83:0/1279527851 >> 172.21.15.83:0/1279527851 conn(0x7f421806abe0 msgr2=0x7f421806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:33.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f421df17640 1 -- 172.21.15.83:0/1279527851 shutdown_connections 2024-04-17T10:38:33.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f421df17640 1 -- 172.21.15.83:0/1279527851 wait complete. 2024-04-17T10:38:33.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f421df17640 1 Processor -- start 2024-04-17T10:38:33.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f421df17640 1 -- start start 2024-04-17T10:38:33.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f421df17640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42180b3990 0x7f4218150410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:33.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f421df17640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42180b45a0 0x7f42181572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:33.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f421df17640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42180cf060 0x7f421814f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:33.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f421df17640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42180cf810 con 0x7f42180cf060 2024-04-17T10:38:33.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f421df17640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42180cf690 con 0x7f42180b3990 2024-04-17T10:38:33.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f421df17640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42180cf990 con 0x7f42180b45a0 2024-04-17T10:38:33.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f4217fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42180cf060 0x7f421814f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:33.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f4217fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42180cf060 0x7f421814f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38404/0 (socket says 172.21.15.83:38404) 2024-04-17T10:38:33.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f4217fff640 1 -- 172.21.15.83:0/1719522207 learned_addr learned my addr 172.21.15.83:0/1719522207 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:33.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f42177fe640 1 --2- 172.21.15.83:0/1719522207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42180b3990 0x7f4218150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:33.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f4216ffd640 1 --2- 172.21.15.83:0/1719522207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42180b45a0 0x7f42181572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:33.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f4217fff640 1 -- 172.21.15.83:0/1719522207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42180b45a0 msgr2=0x7f42181572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:33.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f4217fff640 1 --2- 172.21.15.83:0/1719522207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42180b45a0 0x7f42181572a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:33.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f4217fff640 1 -- 172.21.15.83:0/1719522207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42180b3990 msgr2=0x7f4218150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:33.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f4217fff640 1 --2- 172.21.15.83:0/1719522207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42180b3990 0x7f4218150410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:33.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f4217fff640 1 -- 172.21.15.83:0/1719522207 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42181577e0 con 0x7f42180cf060 2024-04-17T10:38:33.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f42177fe640 1 --2- 172.21.15.83:0/1719522207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42180b3990 0x7f4218150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:33.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f4217fff640 1 --2- 172.21.15.83:0/1719522207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42180cf060 0x7f421814f040 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f421000cbf0 tx=0x7f421000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:33.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f420ffff640 1 -- 172.21.15.83:0/1719522207 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4210017070 con 0x7f42180cf060 2024-04-17T10:38:33.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f420ffff640 1 -- 172.21.15.83:0/1719522207 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421000ad40 con 0x7f42180cf060 2024-04-17T10:38:33.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f421df17640 1 -- 172.21.15.83:0/1719522207 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4218151240 con 0x7f42180cf060 2024-04-17T10:38:33.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f420ffff640 1 -- 172.21.15.83:0/1719522207 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4210012600 con 0x7f42180cf060 2024-04-17T10:38:33.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.692+0000 7f421df17640 1 -- 172.21.15.83:0/1719522207 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4218156aa0 con 0x7f42180cf060 2024-04-17T10:38:33.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.696+0000 7f421df17640 1 -- 172.21.15.83:0/1719522207 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4218074f90 con 0x7f42180cf060 2024-04-17T10:38:33.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.696+0000 7f420ffff640 1 -- 172.21.15.83:0/1719522207 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f421001a070 con 0x7f42180cf060 2024-04-17T10:38:33.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.696+0000 7f420ffff640 1 --2- 172.21.15.83:0/1719522207 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f41f8066f60 0x7f41f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:33.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.696+0000 7f420ffff640 1 -- 172.21.15.83:0/1719522207 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4210084be0 con 0x7f42180cf060 2024-04-17T10:38:33.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.696+0000 7f42177fe640 1 --2- 172.21.15.83:0/1719522207 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f41f8066f60 0x7f41f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:33.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.696+0000 7f42177fe640 1 --2- 172.21.15.83:0/1719522207 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f41f8066f60 0x7f41f8069420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f42080097a0 tx=0x7f4208008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:33.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.700+0000 7f420ffff640 1 -- 172.21.15.83:0/1719522207 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4210055630 con 0x7f42180cf060 2024-04-17T10:38:33.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.888+0000 7f421df17640 1 -- 172.21.15.83:0/1719522207 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4218166c80 con 0x7f42180cf060 2024-04-17T10:38:33.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.888+0000 7f420ffff640 1 -- 172.21.15.83:0/1719522207 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f42100596d0 con 0x7f42180cf060 2024-04-17T10:38:33.886 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T10:38:33.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.892+0000 7f421df17640 1 -- 172.21.15.83:0/1719522207 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f41f8066f60 msgr2=0x7f41f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:33.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.892+0000 7f421df17640 1 --2- 172.21.15.83:0/1719522207 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f41f8066f60 0x7f41f8069420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f42080097a0 tx=0x7f4208008040 comp rx=0 tx=0).stop 2024-04-17T10:38:33.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.892+0000 7f421df17640 1 -- 172.21.15.83:0/1719522207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42180cf060 msgr2=0x7f421814f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:33.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.892+0000 7f421df17640 1 --2- 172.21.15.83:0/1719522207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42180cf060 0x7f421814f040 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f421000cbf0 tx=0x7f421000a430 comp rx=0 tx=0).stop 2024-04-17T10:38:33.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.892+0000 7f421df17640 1 -- 172.21.15.83:0/1719522207 shutdown_connections 2024-04-17T10:38:33.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.892+0000 7f421df17640 1 --2- 172.21.15.83:0/1719522207 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f41f8066f60 0x7f41f8069420 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:33.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.892+0000 7f421df17640 1 --2- 172.21.15.83:0/1719522207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42180cf060 0x7f421814f040 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:33.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.892+0000 7f421df17640 1 --2- 172.21.15.83:0/1719522207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42180b45a0 0x7f42181572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:33.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.892+0000 7f421df17640 1 --2- 172.21.15.83:0/1719522207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42180b3990 0x7f4218150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:33.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.892+0000 7f421df17640 1 -- 172.21.15.83:0/1719522207 >> 172.21.15.83:0/1719522207 conn(0x7f421806abe0 msgr2=0x7f4218158120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:33.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.892+0000 7f421df17640 1 -- 172.21.15.83:0/1719522207 shutdown_connections 2024-04-17T10:38:33.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:33.892+0000 7f421df17640 1 -- 172.21.15.83:0/1719522207 wait complete. 2024-04-17T10:38:33.903 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T10:38:33.903 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T10:38:33.903 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2024-04-17T10:38:33.903 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-17T10:38:33.903 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-17T10:38:33.903 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T10:38:33.903 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T10:38:33.903 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T10:38:33.904 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-17T10:38:33.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:33.904 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2024-04-17T10:38:33.911 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T10:38:33.911 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T10:38:33.911 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T10:38:33.911 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:33.911 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2024-04-17T10:38:33.911 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:33.911 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:38:33.911 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:38:33.911 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2024-04-17T10:38:33.912 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:38:33.912 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:38:33.912 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2024-04-17T10:38:33.912 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:38:33.912 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:38:33.912 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T10:38:33.912 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:38:33.912 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-17T10:38:33.912 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 42476sec preferred_lft 42476sec 2024-04-17T10:38:33.912 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2024-04-17T10:38:33.912 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:33.912 INFO:teuthology.orchestra.run.smithi167.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T10:38:33.912 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 02:42:8b:0a:99:24 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:38:33.912 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T10:38:33.913 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:33.913 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-17T10:38:33.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:33.913 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2024-04-17T10:38:33.913 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-04-17T10:38:33.913 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link add name ceph-brx type bridge 2024-04-17T10:38:33.913 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip addr flush dev ceph-brx 2024-04-17T10:38:33.913 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set ceph-brx up 2024-04-17T10:38:33.913 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-17T10:38:33.913 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-04-17T10:38:33.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:33.998 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T10:38:34.048 INFO:teuthology.orchestra.run.smithi167.stdout:1 2024-04-17T10:38:34.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:34.050 DEBUG:teuthology.orchestra.run.smithi167:> route 2024-04-17T10:38:34.133 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2024-04-17T10:38:34.133 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T10:38:34.133 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-17T10:38:34.133 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-17T10:38:34.133 INFO:teuthology.orchestra.run.smithi167.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-17T10:38:34.133 INFO:teuthology.orchestra.run.smithi167.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T10:38:34.133 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-17T10:38:34.133 INFO:teuthology.orchestra.run.smithi167.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:38:34.133 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:38:34.133 INFO:teuthology.orchestra.run.smithi167.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:38:34.133 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T10:38:34.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:34.134 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2024-04-17T10:38:34.134 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-04-17T10:38:34.134 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-17T10:38:34.134 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-17T10:38:34.134 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-17T10:38:34.134 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-04-17T10:38:34.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:34.210 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-04-17T10:38:34.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:34.260 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list-id 2024-04-17T10:38:34.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:34.308 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2024-04-17T10:38:34.308 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-04-17T10:38:34.308 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-17T10:38:34.308 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-17T10:38:34.309 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-04-17T10:38:34.378 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-17T10:38:34.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:34.378 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2024-04-17T10:38:34.378 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-04-17T10:38:34.378 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-17T10:38:34.378 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-17T10:38:34.378 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-17T10:38:34.378 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-17T10:38:34.379 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-17T10:38:34.379 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-04-17T10:38:34.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:34.479 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2024-04-17T10:38:34.479 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-04-17T10:38:34.479 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2024-04-17T10:38:34.479 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set dev brx.0 master ceph-brx 2024-04-17T10:38:34.479 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-04-17T10:38:34.498 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-17T10:38:34.498 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:38:34.498 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-17T10:38:34.543 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-17T10:38:34.543 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:38:34.544 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-17T10:38:34.591 DEBUG:teuthology.orchestra.run.smithi167:> sudo modprobe fuse 2024-04-17T10:38:34.649 DEBUG:teuthology.orchestra.run.smithi167:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:38:34.700 INFO:teuthology.orchestra.run.smithi167.stdout:/sys 2024-04-17T10:38:34.701 INFO:teuthology.orchestra.run.smithi167.stdout:/proc 2024-04-17T10:38:34.701 INFO:teuthology.orchestra.run.smithi167.stdout:/dev 2024-04-17T10:38:34.701 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/pts 2024-04-17T10:38:34.701 INFO:teuthology.orchestra.run.smithi167.stdout:/run 2024-04-17T10:38:34.701 INFO:teuthology.orchestra.run.smithi167.stdout:/ 2024-04-17T10:38:34.701 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/security 2024-04-17T10:38:34.701 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/shm 2024-04-17T10:38:34.701 INFO:teuthology.orchestra.run.smithi167.stdout:/run/lock 2024-04-17T10:38:34.701 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/cgroup 2024-04-17T10:38:34.701 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/pstore 2024-04-17T10:38:34.702 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/bpf 2024-04-17T10:38:34.702 INFO:teuthology.orchestra.run.smithi167.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:38:34.702 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/hugepages 2024-04-17T10:38:34.702 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/mqueue 2024-04-17T10:38:34.702 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/debug 2024-04-17T10:38:34.702 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/tracing 2024-04-17T10:38:34.702 INFO:teuthology.orchestra.run.smithi167.stdout:/proc/fs/nfsd 2024-04-17T10:38:34.702 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/fuse/connections 2024-04-17T10:38:34.702 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/config 2024-04-17T10:38:34.702 INFO:teuthology.orchestra.run.smithi167.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:38:34.702 INFO:teuthology.orchestra.run.smithi167.stdout:/run/rpc_pipefs 2024-04-17T10:38:34.703 INFO:teuthology.orchestra.run.smithi167.stdout:/run/user/1000 2024-04-17T10:38:34.703 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph 2024-04-17T10:38:34.703 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-4 2024-04-17T10:38:34.703 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-5 2024-04-17T10:38:34.703 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-6 2024-04-17T10:38:34.703 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-7 2024-04-17T10:38:34.703 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns 2024-04-17T10:38:34.703 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-17T10:38:34.703 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-17T10:38:34.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:34.704 DEBUG:teuthology.orchestra.run.smithi167:> ls /sys/fs/fuse/connections 2024-04-17T10:38:34.751 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T10:38:34.751 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-17T10:38:34.797 DEBUG:teuthology.orchestra.run.smithi167:> sudo modprobe fuse 2024-04-17T10:38:34.812 DEBUG:teuthology.orchestra.run.smithi167:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:38:34.860 INFO:teuthology.orchestra.run.smithi167.stdout:/sys 2024-04-17T10:38:34.860 INFO:teuthology.orchestra.run.smithi167.stdout:/proc 2024-04-17T10:38:34.860 INFO:teuthology.orchestra.run.smithi167.stdout:/dev 2024-04-17T10:38:34.860 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/pts 2024-04-17T10:38:34.860 INFO:teuthology.orchestra.run.smithi167.stdout:/run 2024-04-17T10:38:34.860 INFO:teuthology.orchestra.run.smithi167.stdout:/ 2024-04-17T10:38:34.861 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/security 2024-04-17T10:38:34.861 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/shm 2024-04-17T10:38:34.861 INFO:teuthology.orchestra.run.smithi167.stdout:/run/lock 2024-04-17T10:38:34.861 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/cgroup 2024-04-17T10:38:34.861 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/pstore 2024-04-17T10:38:34.861 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/bpf 2024-04-17T10:38:34.861 INFO:teuthology.orchestra.run.smithi167.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:38:34.861 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/hugepages 2024-04-17T10:38:34.861 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/mqueue 2024-04-17T10:38:34.861 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/debug 2024-04-17T10:38:34.862 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/tracing 2024-04-17T10:38:34.862 INFO:teuthology.orchestra.run.smithi167.stdout:/proc/fs/nfsd 2024-04-17T10:38:34.862 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/fuse/connections 2024-04-17T10:38:34.862 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/config 2024-04-17T10:38:34.862 INFO:teuthology.orchestra.run.smithi167.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:38:34.862 INFO:teuthology.orchestra.run.smithi167.stdout:/run/rpc_pipefs 2024-04-17T10:38:34.862 INFO:teuthology.orchestra.run.smithi167.stdout:/run/user/1000 2024-04-17T10:38:34.862 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph 2024-04-17T10:38:34.862 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-4 2024-04-17T10:38:34.862 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-5 2024-04-17T10:38:34.863 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-6 2024-04-17T10:38:34.863 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-7 2024-04-17T10:38:34.863 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns 2024-04-17T10:38:34.863 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-17T10:38:34.863 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-17T10:38:34.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:34.864 DEBUG:teuthology.orchestra.run.smithi167:> ls /sys/fs/fuse/connections 2024-04-17T10:38:34.876 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi167.stderr:2024-04-17T10:38:34.881+0000 7f83db1c2580 -1 init, newargv = 0x55641da4f200 newargc=15 2024-04-17T10:38:34.876 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi167.stderr:ceph-fuse[33582]: starting ceph client 2024-04-17T10:38:34.882 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi167.stderr:ceph-fuse[33582]: starting fuse 2024-04-17T10:38:35.878 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:38:35.898 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:38:35.982 DEBUG:teuthology.orchestra.run.smithi167:> sudo modprobe fuse 2024-04-17T10:38:35.997 DEBUG:teuthology.orchestra.run.smithi167:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:38:36.055 INFO:teuthology.orchestra.run.smithi167.stdout:/sys 2024-04-17T10:38:36.056 INFO:teuthology.orchestra.run.smithi167.stdout:/proc 2024-04-17T10:38:36.056 INFO:teuthology.orchestra.run.smithi167.stdout:/dev 2024-04-17T10:38:36.056 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/pts 2024-04-17T10:38:36.056 INFO:teuthology.orchestra.run.smithi167.stdout:/run 2024-04-17T10:38:36.056 INFO:teuthology.orchestra.run.smithi167.stdout:/ 2024-04-17T10:38:36.056 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/security 2024-04-17T10:38:36.056 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/shm 2024-04-17T10:38:36.056 INFO:teuthology.orchestra.run.smithi167.stdout:/run/lock 2024-04-17T10:38:36.056 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/cgroup 2024-04-17T10:38:36.056 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/pstore 2024-04-17T10:38:36.056 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/bpf 2024-04-17T10:38:36.056 INFO:teuthology.orchestra.run.smithi167.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:38:36.057 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/hugepages 2024-04-17T10:38:36.057 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/mqueue 2024-04-17T10:38:36.057 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/debug 2024-04-17T10:38:36.057 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/tracing 2024-04-17T10:38:36.057 INFO:teuthology.orchestra.run.smithi167.stdout:/proc/fs/nfsd 2024-04-17T10:38:36.057 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/fuse/connections 2024-04-17T10:38:36.057 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/config 2024-04-17T10:38:36.057 INFO:teuthology.orchestra.run.smithi167.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:38:36.057 INFO:teuthology.orchestra.run.smithi167.stdout:/run/rpc_pipefs 2024-04-17T10:38:36.057 INFO:teuthology.orchestra.run.smithi167.stdout:/run/user/1000 2024-04-17T10:38:36.057 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph 2024-04-17T10:38:36.057 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-4 2024-04-17T10:38:36.057 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-5 2024-04-17T10:38:36.057 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-6 2024-04-17T10:38:36.058 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-7 2024-04-17T10:38:36.058 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns 2024-04-17T10:38:36.058 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-17T10:38:36.058 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-17T10:38:36.058 INFO:teuthology.orchestra.run.smithi167.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-17T10:38:36.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:36.059 DEBUG:teuthology.orchestra.run.smithi167:> ls /sys/fs/fuse/connections 2024-04-17T10:38:36.108 INFO:teuthology.orchestra.run.smithi167.stdout:45 2024-04-17T10:38:36.108 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-17T10:38:36.108 DEBUG:teuthology.orchestra.run.smithi167:> sudo stdin-killer -- python3 -c ' 2024-04-17T10:38:36.108 DEBUG:teuthology.orchestra.run.smithi167:> import glob 2024-04-17T10:38:36.108 DEBUG:teuthology.orchestra.run.smithi167:> import re 2024-04-17T10:38:36.108 DEBUG:teuthology.orchestra.run.smithi167:> import os 2024-04-17T10:38:36.108 DEBUG:teuthology.orchestra.run.smithi167:> import subprocess 2024-04-17T10:38:36.109 DEBUG:teuthology.orchestra.run.smithi167:> 2024-04-17T10:38:36.109 DEBUG:teuthology.orchestra.run.smithi167:> def _find_admin_socket(client_name): 2024-04-17T10:38:36.109 DEBUG:teuthology.orchestra.run.smithi167:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-17T10:38:36.109 DEBUG:teuthology.orchestra.run.smithi167:> files = glob.glob(asok_path) 2024-04-17T10:38:36.109 DEBUG:teuthology.orchestra.run.smithi167:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-17T10:38:36.109 DEBUG:teuthology.orchestra.run.smithi167:> 2024-04-17T10:38:36.109 DEBUG:teuthology.orchestra.run.smithi167:> # Given a non-glob path, it better be there 2024-04-17T10:38:36.109 DEBUG:teuthology.orchestra.run.smithi167:> if "*" not in asok_path: 2024-04-17T10:38:36.109 DEBUG:teuthology.orchestra.run.smithi167:> assert(len(files) == 1) 2024-04-17T10:38:36.109 DEBUG:teuthology.orchestra.run.smithi167:> return files[0] 2024-04-17T10:38:36.109 DEBUG:teuthology.orchestra.run.smithi167:> 2024-04-17T10:38:36.109 DEBUG:teuthology.orchestra.run.smithi167:> for f in files: 2024-04-17T10:38:36.109 DEBUG:teuthology.orchestra.run.smithi167:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T10:38:36.109 DEBUG:teuthology.orchestra.run.smithi167:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T10:38:36.109 DEBUG:teuthology.orchestra.run.smithi167:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T10:38:36.110 DEBUG:teuthology.orchestra.run.smithi167:> contents = proc_f.read() 2024-04-17T10:38:36.110 DEBUG:teuthology.orchestra.run.smithi167:> if mountpoint in contents: 2024-04-17T10:38:36.110 DEBUG:teuthology.orchestra.run.smithi167:> return f 2024-04-17T10:38:36.110 DEBUG:teuthology.orchestra.run.smithi167:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T10:38:36.110 DEBUG:teuthology.orchestra.run.smithi167:> 2024-04-17T10:38:36.110 DEBUG:teuthology.orchestra.run.smithi167:> print(_find_admin_socket("client.1")) 2024-04-17T10:38:36.110 DEBUG:teuthology.orchestra.run.smithi167:> ' 2024-04-17T10:38:36.230 INFO:teuthology.orchestra.run.smithi167.stdout:/var/run/ceph/ceph-client.1.33582.asok 2024-04-17T10:38:36.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:38:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:38:36.241 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33582.asok 2024-04-17T10:38:36.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:36.242 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33582.asok status 2024-04-17T10:38:36.394 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-04-17T10:38:36.395 INFO:teuthology.orchestra.run.smithi167.stdout: "metadata": { 2024-04-17T10:38:36.395 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-17T10:38:36.395 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-17T10:38:36.395 INFO:teuthology.orchestra.run.smithi167.stdout: "entity_id": "1", 2024-04-17T10:38:36.395 INFO:teuthology.orchestra.run.smithi167.stdout: "hostname": "smithi167", 2024-04-17T10:38:36.395 INFO:teuthology.orchestra.run.smithi167.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-17T10:38:36.395 INFO:teuthology.orchestra.run.smithi167.stdout: "pid": "33582", 2024-04-17T10:38:36.395 INFO:teuthology.orchestra.run.smithi167.stdout: "root": "/" 2024-04-17T10:38:36.395 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-04-17T10:38:36.395 INFO:teuthology.orchestra.run.smithi167.stdout: "dentry_count": 0, 2024-04-17T10:38:36.395 INFO:teuthology.orchestra.run.smithi167.stdout: "dentry_pinned_count": 0, 2024-04-17T10:38:36.396 INFO:teuthology.orchestra.run.smithi167.stdout: "id": 4501, 2024-04-17T10:38:36.396 INFO:teuthology.orchestra.run.smithi167.stdout: "inst": { 2024-04-17T10:38:36.396 INFO:teuthology.orchestra.run.smithi167.stdout: "name": { 2024-04-17T10:38:36.396 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "client", 2024-04-17T10:38:36.396 INFO:teuthology.orchestra.run.smithi167.stdout: "num": 4501 2024-04-17T10:38:36.396 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-04-17T10:38:36.396 INFO:teuthology.orchestra.run.smithi167.stdout: "addr": { 2024-04-17T10:38:36.396 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "v1", 2024-04-17T10:38:36.396 INFO:teuthology.orchestra.run.smithi167.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:38:36.396 INFO:teuthology.orchestra.run.smithi167.stdout: "nonce": 2636880051 2024-04-17T10:38:36.396 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-04-17T10:38:36.396 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-04-17T10:38:36.396 INFO:teuthology.orchestra.run.smithi167.stdout: "addr": { 2024-04-17T10:38:36.396 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "v1", 2024-04-17T10:38:36.397 INFO:teuthology.orchestra.run.smithi167.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:38:36.397 INFO:teuthology.orchestra.run.smithi167.stdout: "nonce": 2636880051 2024-04-17T10:38:36.397 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-04-17T10:38:36.397 INFO:teuthology.orchestra.run.smithi167.stdout: "inst_str": "client.4501 192.168.0.1:0/2636880051", 2024-04-17T10:38:36.397 INFO:teuthology.orchestra.run.smithi167.stdout: "addr_str": "192.168.0.1:0/2636880051", 2024-04-17T10:38:36.397 INFO:teuthology.orchestra.run.smithi167.stdout: "inode_count": 1, 2024-04-17T10:38:36.397 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_epoch": 6, 2024-04-17T10:38:36.397 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_epoch": 20, 2024-04-17T10:38:36.397 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_epoch_barrier": 0, 2024-04-17T10:38:36.397 INFO:teuthology.orchestra.run.smithi167.stdout: "blocklisted": false, 2024-04-17T10:38:36.397 INFO:teuthology.orchestra.run.smithi167.stdout: "fs_name": "cephfs" 2024-04-17T10:38:36.397 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-04-17T10:38:36.409 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T10:38:36.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf66fe6640 1 Processor -- start 2024-04-17T10:38:36.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf66fe6640 1 -- start start 2024-04-17T10:38:36.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf66fe6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf6015db50 0x7fdf60070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:36.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf66fe6640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdf600aedd0 0x7fdf6006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:36.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf66fe6640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf600709b0 con 0x7fdf6006dae0 2024-04-17T10:38:36.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf66fe6640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf6015e440 con 0x7fdf600aedd0 2024-04-17T10:38:36.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf66fe6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6015e720 con 0x7fdf6015db50 2024-04-17T10:38:36.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf65583640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fdf6006dae0 0x7fdf600cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:43818/0 (socket says 172.21.15.83:43818) 2024-04-17T10:38:36.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf65583640 1 -- 172.21.15.83:0/2828061499 learned_addr learned my addr 172.21.15.83:0/2828061499 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:36.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf64d82640 1 --2- 172.21.15.83:0/2828061499 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf6015db50 0x7fdf60070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:36.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf5ffff640 1 --2- 172.21.15.83:0/2828061499 >> v2:172.21.15.167:3300/0 conn(0x7fdf600aedd0 0x7fdf6006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:36.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf5f7fe640 1 -- 172.21.15.83:0/2828061499 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3991506208 0 0) 0x7fdf580014a0 con 0x7fdf6006dae0 2024-04-17T10:38:36.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf5f7fe640 1 -- 172.21.15.83:0/2828061499 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf50003530 con 0x7fdf6006dae0 2024-04-17T10:38:36.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf5f7fe640 1 -- 172.21.15.83:0/2828061499 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 928582811 0 0) 0x7fdf50003530 con 0x7fdf6006dae0 2024-04-17T10:38:36.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf5f7fe640 1 -- 172.21.15.83:0/2828061499 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf6015db50 msgr2=0x7fdf60070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:36.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf5f7fe640 1 --2- 172.21.15.83:0/2828061499 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf6015db50 0x7fdf60070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:36.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf5f7fe640 1 -- 172.21.15.83:0/2828061499 >> v2:172.21.15.167:3300/0 conn(0x7fdf600aedd0 msgr2=0x7fdf6006d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:36.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf5f7fe640 1 --2- 172.21.15.83:0/2828061499 >> v2:172.21.15.167:3300/0 conn(0x7fdf600aedd0 0x7fdf6006d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:36.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf5f7fe640 1 -- 172.21.15.83:0/2828061499 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf6015ea00 con 0x7fdf6006dae0 2024-04-17T10:38:36.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf5f7fe640 1 -- 172.21.15.83:0/2828061499 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fdf58001bd0 con 0x7fdf6006dae0 2024-04-17T10:38:36.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf5f7fe640 1 -- 172.21.15.83:0/2828061499 >> v1:172.21.15.83:6789/0 conn(0x7fdf6006dae0 legacy=0x7fdf600cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:36.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf5f7fe640 1 --2- 172.21.15.83:0/2828061499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf500039c0 0x7fdf50003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:36.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf5f7fe640 1 --2- 172.21.15.83:0/2828061499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf50004810 0x7fdf50004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:36.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf5f7fe640 1 --2- 172.21.15.83:0/2828061499 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf6015db50 0x7fdf500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:36.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf5f7fe640 1 -- 172.21.15.83:0/2828061499 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6015e440 con 0x7fdf50004810 2024-04-17T10:38:36.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf5f7fe640 1 -- 172.21.15.83:0/2828061499 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6015e720 con 0x7fdf500039c0 2024-04-17T10:38:36.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf5f7fe640 1 -- 172.21.15.83:0/2828061499 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5000b9d0 con 0x7fdf6015db50 2024-04-17T10:38:36.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf5ffff640 1 --2- 172.21.15.83:0/2828061499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf50004810 0x7fdf50004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:36.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf65583640 1 --2- 172.21.15.83:0/2828061499 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf6015db50 0x7fdf500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:36.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf64d82640 1 --2- 172.21.15.83:0/2828061499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf500039c0 0x7fdf50003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:36.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf5ffff640 1 -- 172.21.15.83:0/2828061499 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf6015db50 msgr2=0x7fdf500051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:36.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf5ffff640 1 --2- 172.21.15.83:0/2828061499 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf6015db50 0x7fdf500051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:36.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf5ffff640 1 -- 172.21.15.83:0/2828061499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf500039c0 msgr2=0x7fdf50003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:36.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf5ffff640 1 --2- 172.21.15.83:0/2828061499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf500039c0 0x7fdf50003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:36.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.544+0000 7fdf5ffff640 1 -- 172.21.15.83:0/2828061499 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf50003530 con 0x7fdf50004810 2024-04-17T10:38:36.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.548+0000 7fdf5ffff640 1 --2- 172.21.15.83:0/2828061499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf50004810 0x7fdf50004c90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c009ae0 tx=0x7fdf4c01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1edc31949905f561 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:36.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.548+0000 7fdf5f7fe640 1 -- 172.21.15.83:0/2828061499 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c0b2070 con 0x7fdf50004810 2024-04-17T10:38:36.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.548+0000 7fdf5f7fe640 1 -- 172.21.15.83:0/2828061499 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4c0a55c0 con 0x7fdf50004810 2024-04-17T10:38:36.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.548+0000 7fdf5f7fe640 1 -- 172.21.15.83:0/2828061499 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c0096b0 con 0x7fdf50004810 2024-04-17T10:38:36.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.548+0000 7fdf66fe6640 1 -- 172.21.15.83:0/2828061499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf50004810 msgr2=0x7fdf50004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:36.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.548+0000 7fdf66fe6640 1 --2- 172.21.15.83:0/2828061499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf50004810 0x7fdf50004c90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c009ae0 tx=0x7fdf4c01cab0 comp rx=0 tx=0).stop 2024-04-17T10:38:36.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.548+0000 7fdf65583640 1 -- 172.21.15.83:0/2828061499 reap_dead start 2024-04-17T10:38:36.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.548+0000 7fdf66fe6640 1 -- 172.21.15.83:0/2828061499 shutdown_connections 2024-04-17T10:38:36.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.548+0000 7fdf66fe6640 1 -- 172.21.15.83:0/2828061499 >> 172.21.15.83:0/2828061499 conn(0x7fdf6006abe0 msgr2=0x7fdf6006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:36.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.548+0000 7fdf66fe6640 1 -- 172.21.15.83:0/2828061499 shutdown_connections 2024-04-17T10:38:36.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.548+0000 7fdf66fe6640 1 -- 172.21.15.83:0/2828061499 wait complete. 2024-04-17T10:38:36.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.548+0000 7fdf66fe6640 1 Processor -- start 2024-04-17T10:38:36.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.548+0000 7fdf66fe6640 1 -- start start 2024-04-17T10:38:36.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.548+0000 7fdf66fe6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf50004810 0x7fdf600add20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:36.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.548+0000 7fdf66fe6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf60147f50 0x7fdf600cd630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:36.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.548+0000 7fdf66fe6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf600cdb70 0x7fdf600abd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:36.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.548+0000 7fdf66fe6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6015df30 con 0x7fdf50004810 2024-04-17T10:38:36.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.552+0000 7fdf66fe6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6015efc0 con 0x7fdf60147f50 2024-04-17T10:38:36.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.552+0000 7fdf66fe6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6015e0b0 con 0x7fdf600cdb70 2024-04-17T10:38:36.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.552+0000 7fdf64d82640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf50004810 0x7fdf600add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:36.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.552+0000 7fdf64d82640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf50004810 0x7fdf600add20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38418/0 (socket says 172.21.15.83:38418) 2024-04-17T10:38:36.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.552+0000 7fdf64d82640 1 -- 172.21.15.83:0/202767694 learned_addr learned my addr 172.21.15.83:0/202767694 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:36.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.552+0000 7fdf65583640 1 --2- 172.21.15.83:0/202767694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf600cdb70 0x7fdf600abd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:36.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.552+0000 7fdf5ffff640 1 --2- 172.21.15.83:0/202767694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf60147f50 0x7fdf600cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:36.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.552+0000 7fdf64d82640 1 -- 172.21.15.83:0/202767694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf600cdb70 msgr2=0x7fdf600abd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:36.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.552+0000 7fdf64d82640 1 --2- 172.21.15.83:0/202767694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf600cdb70 0x7fdf600abd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:36.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.552+0000 7fdf64d82640 1 -- 172.21.15.83:0/202767694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf60147f50 msgr2=0x7fdf600cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:36.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.552+0000 7fdf64d82640 1 --2- 172.21.15.83:0/202767694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf60147f50 0x7fdf600cd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:36.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.552+0000 7fdf64d82640 1 -- 172.21.15.83:0/202767694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf60148500 con 0x7fdf50004810 2024-04-17T10:38:36.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.552+0000 7fdf5ffff640 1 --2- 172.21.15.83:0/202767694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf60147f50 0x7fdf600cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:36.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.552+0000 7fdf64d82640 1 --2- 172.21.15.83:0/202767694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf50004810 0x7fdf600add20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdf54010890 tx=0x7fdf54010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:36.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.552+0000 7fdf5d7fa640 1 -- 172.21.15.83:0/202767694 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5400f040 con 0x7fdf50004810 2024-04-17T10:38:36.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.552+0000 7fdf5d7fa640 1 -- 172.21.15.83:0/202767694 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5400eb60 con 0x7fdf50004810 2024-04-17T10:38:36.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.552+0000 7fdf5d7fa640 1 -- 172.21.15.83:0/202767694 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5401c400 con 0x7fdf50004810 2024-04-17T10:38:36.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.552+0000 7fdf66fe6640 1 -- 172.21.15.83:0/202767694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf601487e0 con 0x7fdf50004810 2024-04-17T10:38:36.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.552+0000 7fdf66fe6640 1 -- 172.21.15.83:0/202767694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf60148da0 con 0x7fdf50004810 2024-04-17T10:38:36.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.552+0000 7fdf66fe6640 1 -- 172.21.15.83:0/202767694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf60074f90 con 0x7fdf50004810 2024-04-17T10:38:36.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.552+0000 7fdf5d7fa640 1 -- 172.21.15.83:0/202767694 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdf5401c5a0 con 0x7fdf50004810 2024-04-17T10:38:36.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.552+0000 7fdf5d7fa640 1 --2- 172.21.15.83:0/202767694 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdf44066f60 0x7fdf44069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:36.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.552+0000 7fdf5d7fa640 1 -- 172.21.15.83:0/202767694 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf54085560 con 0x7fdf50004810 2024-04-17T10:38:36.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.552+0000 7fdf5ffff640 1 --2- 172.21.15.83:0/202767694 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdf44066f60 0x7fdf44069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:36.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.552+0000 7fdf5ffff640 1 --2- 172.21.15.83:0/202767694 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdf44066f60 0x7fdf44069420 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c0af000 tx=0x7fdf4c0046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:36.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.556+0000 7fdf5d7fa640 1 -- 172.21.15.83:0/202767694 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdf54018030 con 0x7fdf50004810 2024-04-17T10:38:36.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.752+0000 7fdf66fe6640 1 -- 172.21.15.83:0/202767694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdf60146000 con 0x7fdf50004810 2024-04-17T10:38:36.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.752+0000 7fdf5d7fa640 1 -- 172.21.15.83:0/202767694 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdf54055fb0 con 0x7fdf50004810 2024-04-17T10:38:36.751 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T10:38:36.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.756+0000 7fdf66fe6640 1 -- 172.21.15.83:0/202767694 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdf44066f60 msgr2=0x7fdf44069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:36.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.756+0000 7fdf66fe6640 1 --2- 172.21.15.83:0/202767694 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdf44066f60 0x7fdf44069420 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c0af000 tx=0x7fdf4c0046d0 comp rx=0 tx=0).stop 2024-04-17T10:38:36.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.756+0000 7fdf66fe6640 1 -- 172.21.15.83:0/202767694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf50004810 msgr2=0x7fdf600add20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:36.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.756+0000 7fdf66fe6640 1 --2- 172.21.15.83:0/202767694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf50004810 0x7fdf600add20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdf54010890 tx=0x7fdf54010d50 comp rx=0 tx=0).stop 2024-04-17T10:38:36.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.756+0000 7fdf66fe6640 1 -- 172.21.15.83:0/202767694 shutdown_connections 2024-04-17T10:38:36.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.756+0000 7fdf66fe6640 1 --2- 172.21.15.83:0/202767694 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdf44066f60 0x7fdf44069420 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:36.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.756+0000 7fdf66fe6640 1 --2- 172.21.15.83:0/202767694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf600cdb70 0x7fdf600abd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:36.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.756+0000 7fdf66fe6640 1 --2- 172.21.15.83:0/202767694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf60147f50 0x7fdf600cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:36.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.756+0000 7fdf66fe6640 1 --2- 172.21.15.83:0/202767694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf50004810 0x7fdf600add20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:36.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.756+0000 7fdf66fe6640 1 -- 172.21.15.83:0/202767694 >> 172.21.15.83:0/202767694 conn(0x7fdf6006abe0 msgr2=0x7fdf60070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:36.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.756+0000 7fdf66fe6640 1 -- 172.21.15.83:0/202767694 shutdown_connections 2024-04-17T10:38:36.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:36.756+0000 7fdf66fe6640 1 -- 172.21.15.83:0/202767694 wait complete. 2024-04-17T10:38:36.767 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T10:38:36.767 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T10:38:36.768 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2024-04-17T10:38:36.768 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-17T10:38:36.768 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-17T10:38:36.768 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T10:38:36.768 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T10:38:36.768 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T10:38:36.768 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-17T10:38:36.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:36.769 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2024-04-17T10:38:36.776 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T10:38:36.776 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T10:38:36.776 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T10:38:36.776 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:36.776 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2024-04-17T10:38:36.776 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:36.776 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:38:36.776 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:38:36.777 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2024-04-17T10:38:36.777 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:38:36.777 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:38:36.777 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2024-04-17T10:38:36.777 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:38:36.777 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:38:36.777 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T10:38:36.777 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:38:36.777 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-17T10:38:36.777 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 42473sec preferred_lft 42473sec 2024-04-17T10:38:36.777 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2024-04-17T10:38:36.777 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:36.777 INFO:teuthology.orchestra.run.smithi167.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T10:38:36.777 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 02:42:8b:0a:99:24 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:38:36.778 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T10:38:36.778 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:36.778 INFO:teuthology.orchestra.run.smithi167.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T10:38:36.778 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-17T10:38:36.778 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T10:38:36.778 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:36.778 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-17T10:38:36.778 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:36.778 INFO:teuthology.orchestra.run.smithi167.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:38:36.778 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-17T10:38:36.778 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-17T10:38:36.778 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:36.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:36.779 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T10:38:36.831 INFO:teuthology.orchestra.run.smithi167.stdout:1 2024-04-17T10:38:36.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:36.833 DEBUG:teuthology.orchestra.run.smithi167:> route 2024-04-17T10:38:36.885 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2024-04-17T10:38:36.885 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T10:38:36.885 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-17T10:38:36.885 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-17T10:38:36.885 INFO:teuthology.orchestra.run.smithi167.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-17T10:38:36.885 INFO:teuthology.orchestra.run.smithi167.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T10:38:36.885 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-17T10:38:36.885 INFO:teuthology.orchestra.run.smithi167.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:38:36.885 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:38:36.886 INFO:teuthology.orchestra.run.smithi167.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:38:36.886 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T10:38:36.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:36.886 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2024-04-17T10:38:36.886 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-04-17T10:38:36.886 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-17T10:38:36.886 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-17T10:38:36.886 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-17T10:38:36.886 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-04-17T10:38:36.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:36.963 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-04-17T10:38:37.011 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-17T10:38:37.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:37.012 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list-id 2024-04-17T10:38:37.059 INFO:teuthology.orchestra.run.smithi167.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-17T10:38:37.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:37.060 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2024-04-17T10:38:37.060 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-04-17T10:38:37.060 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-17T10:38:37.060 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-17T10:38:37.060 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-04-17T10:38:37.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:37.128 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-17T10:38:37.181 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T10:38:37.181 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T10:38:37.181 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T10:38:37.181 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:37.181 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2024-04-17T10:38:37.182 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:37.182 INFO:teuthology.orchestra.run.smithi167.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T10:38:37.182 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 3e:4c:41:01:a4:38 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-17T10:38:37.182 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-17T10:38:37.182 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:37.182 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::3c4c:41ff:fe01:a438/64 scope link 2024-04-17T10:38:37.182 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:37.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:37.183 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-17T10:38:37.238 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T10:38:37.238 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T10:38:37.238 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T10:38:37.238 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:37.239 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2024-04-17T10:38:37.239 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:37.239 INFO:teuthology.orchestra.run.smithi167.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T10:38:37.239 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 3e:4c:41:01:a4:38 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-17T10:38:37.239 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-17T10:38:37.239 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:37.239 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::3c4c:41ff:fe01:a438/64 scope link 2024-04-17T10:38:37.239 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:38:37.240 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-17T10:38:37.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:37.240 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2024-04-17T10:38:37.241 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-04-17T10:38:37.241 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-17T10:38:37.241 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-17T10:38:37.241 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-17T10:38:37.241 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-17T10:38:37.241 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-17T10:38:37.241 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-04-17T10:38:37.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:37.334 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2024-04-17T10:38:37.334 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-04-17T10:38:37.334 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.1 up 2024-04-17T10:38:37.334 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set dev brx.1 master ceph-brx 2024-04-17T10:38:37.334 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-04-17T10:38:37.353 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-17T10:38:37.353 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:38:37.353 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-17T10:38:37.399 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-17T10:38:37.399 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:38:37.399 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-17T10:38:37.447 DEBUG:teuthology.orchestra.run.smithi167:> sudo modprobe fuse 2024-04-17T10:38:37.505 DEBUG:teuthology.orchestra.run.smithi167:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:38:37.553 INFO:teuthology.orchestra.run.smithi167.stdout:/sys 2024-04-17T10:38:37.553 INFO:teuthology.orchestra.run.smithi167.stdout:/proc 2024-04-17T10:38:37.553 INFO:teuthology.orchestra.run.smithi167.stdout:/dev 2024-04-17T10:38:37.553 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/pts 2024-04-17T10:38:37.553 INFO:teuthology.orchestra.run.smithi167.stdout:/run 2024-04-17T10:38:37.553 INFO:teuthology.orchestra.run.smithi167.stdout:/ 2024-04-17T10:38:37.553 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/security 2024-04-17T10:38:37.553 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/shm 2024-04-17T10:38:37.553 INFO:teuthology.orchestra.run.smithi167.stdout:/run/lock 2024-04-17T10:38:37.554 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/cgroup 2024-04-17T10:38:37.554 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/pstore 2024-04-17T10:38:37.554 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/bpf 2024-04-17T10:38:37.554 INFO:teuthology.orchestra.run.smithi167.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:38:37.554 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/hugepages 2024-04-17T10:38:37.554 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/mqueue 2024-04-17T10:38:37.554 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/debug 2024-04-17T10:38:37.554 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/tracing 2024-04-17T10:38:37.554 INFO:teuthology.orchestra.run.smithi167.stdout:/proc/fs/nfsd 2024-04-17T10:38:37.554 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/fuse/connections 2024-04-17T10:38:37.554 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/config 2024-04-17T10:38:37.554 INFO:teuthology.orchestra.run.smithi167.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:38:37.555 INFO:teuthology.orchestra.run.smithi167.stdout:/run/rpc_pipefs 2024-04-17T10:38:37.555 INFO:teuthology.orchestra.run.smithi167.stdout:/run/user/1000 2024-04-17T10:38:37.555 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph 2024-04-17T10:38:37.555 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-4 2024-04-17T10:38:37.555 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-5 2024-04-17T10:38:37.555 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-6 2024-04-17T10:38:37.555 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-7 2024-04-17T10:38:37.555 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns 2024-04-17T10:38:37.555 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-17T10:38:37.555 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-17T10:38:37.555 INFO:teuthology.orchestra.run.smithi167.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-17T10:38:37.555 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-17T10:38:37.555 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-17T10:38:37.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:37.556 DEBUG:teuthology.orchestra.run.smithi167:> ls /sys/fs/fuse/connections 2024-04-17T10:38:37.599 INFO:teuthology.orchestra.run.smithi167.stdout:45 2024-04-17T10:38:37.599 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-04-17T10:38:37.599 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-04-17T10:38:37.644 DEBUG:teuthology.orchestra.run.smithi167:> sudo modprobe fuse 2024-04-17T10:38:37.658 DEBUG:teuthology.orchestra.run.smithi167:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:38:37.708 INFO:teuthology.orchestra.run.smithi167.stdout:/sys 2024-04-17T10:38:37.708 INFO:teuthology.orchestra.run.smithi167.stdout:/proc 2024-04-17T10:38:37.708 INFO:teuthology.orchestra.run.smithi167.stdout:/dev 2024-04-17T10:38:37.708 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/pts 2024-04-17T10:38:37.708 INFO:teuthology.orchestra.run.smithi167.stdout:/run 2024-04-17T10:38:37.708 INFO:teuthology.orchestra.run.smithi167.stdout:/ 2024-04-17T10:38:37.708 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/security 2024-04-17T10:38:37.708 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/shm 2024-04-17T10:38:37.708 INFO:teuthology.orchestra.run.smithi167.stdout:/run/lock 2024-04-17T10:38:37.708 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/cgroup 2024-04-17T10:38:37.708 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/pstore 2024-04-17T10:38:37.708 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/bpf 2024-04-17T10:38:37.709 INFO:teuthology.orchestra.run.smithi167.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:38:37.709 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/hugepages 2024-04-17T10:38:37.709 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/mqueue 2024-04-17T10:38:37.709 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/debug 2024-04-17T10:38:37.709 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/tracing 2024-04-17T10:38:37.709 INFO:teuthology.orchestra.run.smithi167.stdout:/proc/fs/nfsd 2024-04-17T10:38:37.709 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/fuse/connections 2024-04-17T10:38:37.709 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/config 2024-04-17T10:38:37.709 INFO:teuthology.orchestra.run.smithi167.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:38:37.709 INFO:teuthology.orchestra.run.smithi167.stdout:/run/rpc_pipefs 2024-04-17T10:38:37.709 INFO:teuthology.orchestra.run.smithi167.stdout:/run/user/1000 2024-04-17T10:38:37.709 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph 2024-04-17T10:38:37.709 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-4 2024-04-17T10:38:37.709 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-5 2024-04-17T10:38:37.710 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-6 2024-04-17T10:38:37.710 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-7 2024-04-17T10:38:37.710 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns 2024-04-17T10:38:37.710 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-17T10:38:37.710 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-17T10:38:37.710 INFO:teuthology.orchestra.run.smithi167.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-17T10:38:37.710 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-17T10:38:37.710 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-17T10:38:37.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:37.710 DEBUG:teuthology.orchestra.run.smithi167:> ls /sys/fs/fuse/connections 2024-04-17T10:38:37.722 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi167.stderr:2024-04-17T10:38:37.725+0000 7fb52f118580 -1 init, newargv = 0x558ef8d25ee0 newargc=15 2024-04-17T10:38:37.722 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi167.stderr:ceph-fuse[33699]: starting ceph client 2024-04-17T10:38:37.726 INFO:teuthology.orchestra.run.smithi167.stdout:45 2024-04-17T10:38:37.728 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi167.stderr:ceph-fuse[33699]: starting fuse 2024-04-17T10:38:38.828 DEBUG:teuthology.orchestra.run.smithi167:> sudo modprobe fuse 2024-04-17T10:38:38.845 DEBUG:teuthology.orchestra.run.smithi167:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:38:38.892 INFO:teuthology.orchestra.run.smithi167.stdout:/sys 2024-04-17T10:38:38.893 INFO:teuthology.orchestra.run.smithi167.stdout:/proc 2024-04-17T10:38:38.893 INFO:teuthology.orchestra.run.smithi167.stdout:/dev 2024-04-17T10:38:38.893 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/pts 2024-04-17T10:38:38.893 INFO:teuthology.orchestra.run.smithi167.stdout:/run 2024-04-17T10:38:38.893 INFO:teuthology.orchestra.run.smithi167.stdout:/ 2024-04-17T10:38:38.893 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/security 2024-04-17T10:38:38.894 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/shm 2024-04-17T10:38:38.894 INFO:teuthology.orchestra.run.smithi167.stdout:/run/lock 2024-04-17T10:38:38.894 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/cgroup 2024-04-17T10:38:38.894 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/pstore 2024-04-17T10:38:38.894 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/bpf 2024-04-17T10:38:38.894 INFO:teuthology.orchestra.run.smithi167.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:38:38.894 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/hugepages 2024-04-17T10:38:38.894 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/mqueue 2024-04-17T10:38:38.894 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/debug 2024-04-17T10:38:38.894 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/tracing 2024-04-17T10:38:38.894 INFO:teuthology.orchestra.run.smithi167.stdout:/proc/fs/nfsd 2024-04-17T10:38:38.894 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/fuse/connections 2024-04-17T10:38:38.895 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/config 2024-04-17T10:38:38.895 INFO:teuthology.orchestra.run.smithi167.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:38:38.895 INFO:teuthology.orchestra.run.smithi167.stdout:/run/rpc_pipefs 2024-04-17T10:38:38.895 INFO:teuthology.orchestra.run.smithi167.stdout:/run/user/1000 2024-04-17T10:38:38.895 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph 2024-04-17T10:38:38.895 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-4 2024-04-17T10:38:38.895 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-5 2024-04-17T10:38:38.895 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-6 2024-04-17T10:38:38.895 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-7 2024-04-17T10:38:38.895 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns 2024-04-17T10:38:38.895 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-17T10:38:38.895 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-17T10:38:38.895 INFO:teuthology.orchestra.run.smithi167.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-17T10:38:38.895 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-17T10:38:38.896 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-17T10:38:38.896 INFO:teuthology.orchestra.run.smithi167.stdout:/home/ubuntu/cephtest/mnt.3 2024-04-17T10:38:38.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:38.897 DEBUG:teuthology.orchestra.run.smithi167:> ls /sys/fs/fuse/connections 2024-04-17T10:38:38.944 INFO:teuthology.orchestra.run.smithi167.stdout:45 2024-04-17T10:38:38.944 INFO:teuthology.orchestra.run.smithi167.stdout:46 2024-04-17T10:38:38.944 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-04-17T10:38:38.944 DEBUG:teuthology.orchestra.run.smithi167:> sudo stdin-killer -- python3 -c ' 2024-04-17T10:38:38.944 DEBUG:teuthology.orchestra.run.smithi167:> import glob 2024-04-17T10:38:38.944 DEBUG:teuthology.orchestra.run.smithi167:> import re 2024-04-17T10:38:38.944 DEBUG:teuthology.orchestra.run.smithi167:> import os 2024-04-17T10:38:38.944 DEBUG:teuthology.orchestra.run.smithi167:> import subprocess 2024-04-17T10:38:38.945 DEBUG:teuthology.orchestra.run.smithi167:> 2024-04-17T10:38:38.945 DEBUG:teuthology.orchestra.run.smithi167:> def _find_admin_socket(client_name): 2024-04-17T10:38:38.945 DEBUG:teuthology.orchestra.run.smithi167:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-17T10:38:38.945 DEBUG:teuthology.orchestra.run.smithi167:> files = glob.glob(asok_path) 2024-04-17T10:38:38.945 DEBUG:teuthology.orchestra.run.smithi167:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-17T10:38:38.945 DEBUG:teuthology.orchestra.run.smithi167:> 2024-04-17T10:38:38.945 DEBUG:teuthology.orchestra.run.smithi167:> # Given a non-glob path, it better be there 2024-04-17T10:38:38.945 DEBUG:teuthology.orchestra.run.smithi167:> if "*" not in asok_path: 2024-04-17T10:38:38.945 DEBUG:teuthology.orchestra.run.smithi167:> assert(len(files) == 1) 2024-04-17T10:38:38.945 DEBUG:teuthology.orchestra.run.smithi167:> return files[0] 2024-04-17T10:38:38.945 DEBUG:teuthology.orchestra.run.smithi167:> 2024-04-17T10:38:38.945 DEBUG:teuthology.orchestra.run.smithi167:> for f in files: 2024-04-17T10:38:38.945 DEBUG:teuthology.orchestra.run.smithi167:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T10:38:38.945 DEBUG:teuthology.orchestra.run.smithi167:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T10:38:38.945 DEBUG:teuthology.orchestra.run.smithi167:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T10:38:38.946 DEBUG:teuthology.orchestra.run.smithi167:> contents = proc_f.read() 2024-04-17T10:38:38.946 DEBUG:teuthology.orchestra.run.smithi167:> if mountpoint in contents: 2024-04-17T10:38:38.946 DEBUG:teuthology.orchestra.run.smithi167:> return f 2024-04-17T10:38:38.946 DEBUG:teuthology.orchestra.run.smithi167:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T10:38:38.946 DEBUG:teuthology.orchestra.run.smithi167:> 2024-04-17T10:38:38.946 DEBUG:teuthology.orchestra.run.smithi167:> print(_find_admin_socket("client.3")) 2024-04-17T10:38:38.946 DEBUG:teuthology.orchestra.run.smithi167:> ' 2024-04-17T10:38:39.066 INFO:teuthology.orchestra.run.smithi167.stdout:/var/run/ceph/ceph-client.3.33699.asok 2024-04-17T10:38:39.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-17T10:38:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:38:39.077 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.33699.asok 2024-04-17T10:38:39.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:39.078 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.33699.asok status 2024-04-17T10:38:39.233 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-04-17T10:38:39.234 INFO:teuthology.orchestra.run.smithi167.stdout: "metadata": { 2024-04-17T10:38:39.234 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-17T10:38:39.234 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-17T10:38:39.234 INFO:teuthology.orchestra.run.smithi167.stdout: "entity_id": "3", 2024-04-17T10:38:39.234 INFO:teuthology.orchestra.run.smithi167.stdout: "hostname": "smithi167", 2024-04-17T10:38:39.234 INFO:teuthology.orchestra.run.smithi167.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-17T10:38:39.234 INFO:teuthology.orchestra.run.smithi167.stdout: "pid": "33699", 2024-04-17T10:38:39.234 INFO:teuthology.orchestra.run.smithi167.stdout: "root": "/" 2024-04-17T10:38:39.234 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-04-17T10:38:39.234 INFO:teuthology.orchestra.run.smithi167.stdout: "dentry_count": 0, 2024-04-17T10:38:39.235 INFO:teuthology.orchestra.run.smithi167.stdout: "dentry_pinned_count": 0, 2024-04-17T10:38:39.235 INFO:teuthology.orchestra.run.smithi167.stdout: "id": 4513, 2024-04-17T10:38:39.235 INFO:teuthology.orchestra.run.smithi167.stdout: "inst": { 2024-04-17T10:38:39.235 INFO:teuthology.orchestra.run.smithi167.stdout: "name": { 2024-04-17T10:38:39.235 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "client", 2024-04-17T10:38:39.235 INFO:teuthology.orchestra.run.smithi167.stdout: "num": 4513 2024-04-17T10:38:39.235 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-04-17T10:38:39.235 INFO:teuthology.orchestra.run.smithi167.stdout: "addr": { 2024-04-17T10:38:39.235 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "v1", 2024-04-17T10:38:39.235 INFO:teuthology.orchestra.run.smithi167.stdout: "addr": "192.168.0.2:0", 2024-04-17T10:38:39.235 INFO:teuthology.orchestra.run.smithi167.stdout: "nonce": 352080441 2024-04-17T10:38:39.235 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-04-17T10:38:39.235 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-04-17T10:38:39.235 INFO:teuthology.orchestra.run.smithi167.stdout: "addr": { 2024-04-17T10:38:39.236 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "v1", 2024-04-17T10:38:39.236 INFO:teuthology.orchestra.run.smithi167.stdout: "addr": "192.168.0.2:0", 2024-04-17T10:38:39.236 INFO:teuthology.orchestra.run.smithi167.stdout: "nonce": 352080441 2024-04-17T10:38:39.236 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-04-17T10:38:39.236 INFO:teuthology.orchestra.run.smithi167.stdout: "inst_str": "client.4513 192.168.0.2:0/352080441", 2024-04-17T10:38:39.236 INFO:teuthology.orchestra.run.smithi167.stdout: "addr_str": "192.168.0.2:0/352080441", 2024-04-17T10:38:39.236 INFO:teuthology.orchestra.run.smithi167.stdout: "inode_count": 1, 2024-04-17T10:38:39.236 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_epoch": 6, 2024-04-17T10:38:39.236 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_epoch": 20, 2024-04-17T10:38:39.236 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_epoch_barrier": 0, 2024-04-17T10:38:39.236 INFO:teuthology.orchestra.run.smithi167.stdout: "blocklisted": false, 2024-04-17T10:38:39.236 INFO:teuthology.orchestra.run.smithi167.stdout: "fs_name": "cephfs" 2024-04-17T10:38:39.236 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-04-17T10:38:39.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:39.248 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2024-04-17T10:38:39.248 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-17T10:38:39.255 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2024-04-17T10:38:39.255 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-17T10:38:39.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:39.256 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:38:39.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:39.315 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2024-04-17T10:38:39.315 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.2 2024-04-17T10:38:39.368 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2024-04-17T10:38:39.368 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-04-17T10:38:39.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:39.369 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-17T10:38:39.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:39.429 DEBUG:teuthology.orchestra.run.smithi167:> stat --file-system '--printf=%T 2024-04-17T10:38:39.429 DEBUG:teuthology.orchestra.run.smithi167:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-17T10:38:39.436 INFO:teuthology.orchestra.run.smithi167.stdout:fuseblk 2024-04-17T10:38:39.436 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-17T10:38:39.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:39.436 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-17T10:38:39.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:39.492 DEBUG:teuthology.orchestra.run.smithi167:> stat --file-system '--printf=%T 2024-04-17T10:38:39.492 DEBUG:teuthology.orchestra.run.smithi167:> ' -- /home/ubuntu/cephtest/mnt.3 2024-04-17T10:38:39.541 INFO:teuthology.orchestra.run.smithi167.stdout:fuseblk 2024-04-17T10:38:39.541 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-04-17T10:38:39.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:39.541 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-17T10:38:39.596 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-17T10:38:39.611 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:38:39.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61ecfe3640 1 Processor -- start 2024-04-17T10:38:39.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61ecfe3640 1 -- start start 2024-04-17T10:38:39.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61ecfe3640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f61e815ba30 0x7f61e8072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:39.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61ecfe3640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e806e320 con 0x7f61e80cfbe0 2024-04-17T10:38:39.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61ecfe3640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e806e040 con 0x7f61e815c960 2024-04-17T10:38:39.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61ecfe3640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61e8071ea0 con 0x7f61e815ba30 2024-04-17T10:38:39.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61e6d9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f61e80cfbe0 0x7f61e806c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47222/0 (socket says 172.21.15.83:47222) 2024-04-17T10:38:39.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61e6d9d640 1 -- 172.21.15.83:0/1281800578 learned_addr learned my addr 172.21.15.83:0/1281800578 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:39.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61e559a640 1 -- 172.21.15.83:0/1281800578 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 224146575 0 0) 0x7f61e806e320 con 0x7f61e80cfbe0 2024-04-17T10:38:39.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61e659c640 1 --2- 172.21.15.83:0/1281800578 >> v2:172.21.15.167:3300/0 conn(0x7f61e815ba30 0x7f61e8072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:39.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61e559a640 1 -- 172.21.15.83:0/1281800578 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61d0003530 con 0x7f61e80cfbe0 2024-04-17T10:38:39.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61e559a640 1 -- 172.21.15.83:0/1281800578 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1890237046 0 0) 0x7f61d0003530 con 0x7f61e80cfbe0 2024-04-17T10:38:39.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61e559a640 1 -- 172.21.15.83:0/1281800578 >> v2:172.21.15.167:3300/0 conn(0x7f61e815ba30 msgr2=0x7f61e8072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:39.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61e559a640 1 --2- 172.21.15.83:0/1281800578 >> v2:172.21.15.167:3300/0 conn(0x7f61e815ba30 0x7f61e8072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:39.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61e559a640 1 -- 172.21.15.83:0/1281800578 >> v1:172.21.15.167:6789/0 conn(0x7f61e815c960 legacy=0x7f61e80cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:39.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61e559a640 1 -- 172.21.15.83:0/1281800578 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e8071850 con 0x7f61e80cfbe0 2024-04-17T10:38:39.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61e559a640 1 -- 172.21.15.83:0/1281800578 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f61e0001a20 con 0x7f61e80cfbe0 2024-04-17T10:38:39.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61e559a640 1 -- 172.21.15.83:0/1281800578 >> v1:172.21.15.83:6789/0 conn(0x7f61e80cfbe0 legacy=0x7f61e806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:39.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61e559a640 1 --2- 172.21.15.83:0/1281800578 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61d00039c0 0x7f61d0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:39.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61e559a640 1 --2- 172.21.15.83:0/1281800578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61d0004810 0x7f61d0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:39.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61e559a640 1 --2- 172.21.15.83:0/1281800578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61d00051d0 0x7f61d000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:39.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61e559a640 1 -- 172.21.15.83:0/1281800578 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e8071ea0 con 0x7f61d0004810 2024-04-17T10:38:39.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61e559a640 1 -- 172.21.15.83:0/1281800578 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d000c170 con 0x7f61d00051d0 2024-04-17T10:38:39.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61e559a640 1 -- 172.21.15.83:0/1281800578 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d000c370 con 0x7f61d00039c0 2024-04-17T10:38:39.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61e5d9b640 1 --2- 172.21.15.83:0/1281800578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61d0004810 0x7f61d0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:39.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61e659c640 1 --2- 172.21.15.83:0/1281800578 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61d00039c0 0x7f61d0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:39.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61e6d9d640 1 --2- 172.21.15.83:0/1281800578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61d00051d0 0x7f61d000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:39.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61e5d9b640 1 -- 172.21.15.83:0/1281800578 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61d00039c0 msgr2=0x7f61d0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:39.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61e5d9b640 1 --2- 172.21.15.83:0/1281800578 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61d00039c0 0x7f61d0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:39.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61e5d9b640 1 -- 172.21.15.83:0/1281800578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61d00051d0 msgr2=0x7f61d000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:39.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61e5d9b640 1 --2- 172.21.15.83:0/1281800578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61d00051d0 0x7f61d000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:39.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61e5d9b640 1 -- 172.21.15.83:0/1281800578 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d0003530 con 0x7f61d0004810 2024-04-17T10:38:39.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61e6d9d640 1 --2- 172.21.15.83:0/1281800578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61d00051d0 0x7f61d000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:39.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.744+0000 7f61e6d9d640 1 -- 172.21.15.83:0/1281800578 reap_dead start 2024-04-17T10:38:39.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.748+0000 7f61e5d9b640 1 --2- 172.21.15.83:0/1281800578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61d0004810 0x7f61d0004c90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f61dc001780 tx=0x7f61dc01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c71e053deb82fe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:39.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.748+0000 7f61e559a640 1 -- 172.21.15.83:0/1281800578 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc0013a0 con 0x7f61d0004810 2024-04-17T10:38:39.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.748+0000 7f61e559a640 1 -- 172.21.15.83:0/1281800578 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61dc0052f0 con 0x7f61d0004810 2024-04-17T10:38:39.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.748+0000 7f61e559a640 1 -- 172.21.15.83:0/1281800578 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc003840 con 0x7f61d0004810 2024-04-17T10:38:39.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.748+0000 7f61ecfe3640 1 -- 172.21.15.83:0/1281800578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61d0004810 msgr2=0x7f61d0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:39.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.748+0000 7f61ecfe3640 1 --2- 172.21.15.83:0/1281800578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61d0004810 0x7f61d0004c90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f61dc001780 tx=0x7f61dc01c400 comp rx=0 tx=0).stop 2024-04-17T10:38:39.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.748+0000 7f61ecfe3640 1 -- 172.21.15.83:0/1281800578 shutdown_connections 2024-04-17T10:38:39.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.748+0000 7f61ecfe3640 1 --2- 172.21.15.83:0/1281800578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61d0004810 0x7f61d0004c90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:39.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.748+0000 7f61ecfe3640 1 -- 172.21.15.83:0/1281800578 >> 172.21.15.83:0/1281800578 conn(0x7f61e806abe0 msgr2=0x7f61e806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:39.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.748+0000 7f61ecfe3640 1 -- 172.21.15.83:0/1281800578 shutdown_connections 2024-04-17T10:38:39.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.748+0000 7f61ecfe3640 1 -- 172.21.15.83:0/1281800578 wait complete. 2024-04-17T10:38:39.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.748+0000 7f61ecfe3640 1 Processor -- start 2024-04-17T10:38:39.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.748+0000 7f61ecfe3640 1 -- start start 2024-04-17T10:38:39.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.752+0000 7f61ecfe3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61d0004810 0x7f61e815a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:39.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.752+0000 7f61ecfe3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61e815a5d0 0x7f61e8158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:39.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.752+0000 7f61ecfe3640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61e8159320 0x7f61e81597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:39.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.752+0000 7f61ecfe3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e80cfa60 con 0x7f61d0004810 2024-04-17T10:38:39.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.752+0000 7f61ecfe3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e80cf8e0 con 0x7f61e815a5d0 2024-04-17T10:38:39.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.752+0000 7f61ecfe3640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e815cd60 con 0x7f61e8159320 2024-04-17T10:38:39.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.752+0000 7f61e659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61d0004810 0x7f61e815a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:39.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.752+0000 7f61e659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61d0004810 0x7f61e815a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52546/0 (socket says 172.21.15.83:52546) 2024-04-17T10:38:39.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.752+0000 7f61e659c640 1 -- 172.21.15.83:0/420558664 learned_addr learned my addr 172.21.15.83:0/420558664 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:39.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.752+0000 7f61e6d9d640 1 --2- 172.21.15.83:0/420558664 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61e8159320 0x7f61e81597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:39.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.752+0000 7f61e5d9b640 1 --2- 172.21.15.83:0/420558664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61e815a5d0 0x7f61e8158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:39.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.752+0000 7f61e659c640 1 -- 172.21.15.83:0/420558664 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61e8159320 msgr2=0x7f61e81597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:39.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.752+0000 7f61e659c640 1 --2- 172.21.15.83:0/420558664 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61e8159320 0x7f61e81597a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:39.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.752+0000 7f61e659c640 1 -- 172.21.15.83:0/420558664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61e815a5d0 msgr2=0x7f61e8158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:39.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.752+0000 7f61e659c640 1 --2- 172.21.15.83:0/420558664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61e815a5d0 0x7f61e8158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:39.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.752+0000 7f61e659c640 1 -- 172.21.15.83:0/420558664 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e8153090 con 0x7f61d0004810 2024-04-17T10:38:39.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.752+0000 7f61e5d9b640 1 --2- 172.21.15.83:0/420558664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61e815a5d0 0x7f61e8158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:39.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.752+0000 7f61e659c640 1 --2- 172.21.15.83:0/420558664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61d0004810 0x7f61e815a090 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f61d8011690 tx=0x7f61d8011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:39.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.752+0000 7f61d6ffd640 1 -- 172.21.15.83:0/420558664 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d800f9c0 con 0x7f61d0004810 2024-04-17T10:38:39.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.752+0000 7f61d6ffd640 1 -- 172.21.15.83:0/420558664 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d800fb60 con 0x7f61d0004810 2024-04-17T10:38:39.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.752+0000 7f61ecfe3640 1 -- 172.21.15.83:0/420558664 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61e8153370 con 0x7f61d0004810 2024-04-17T10:38:39.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.752+0000 7f61d6ffd640 1 -- 172.21.15.83:0/420558664 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d8015460 con 0x7f61d0004810 2024-04-17T10:38:39.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.752+0000 7f61ecfe3640 1 -- 172.21.15.83:0/420558664 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e8153930 con 0x7f61d0004810 2024-04-17T10:38:39.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.752+0000 7f61ecfe3640 1 -- 172.21.15.83:0/420558664 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61e8074f90 con 0x7f61d0004810 2024-04-17T10:38:39.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.752+0000 7f61d6ffd640 1 -- 172.21.15.83:0/420558664 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f61d8015600 con 0x7f61d0004810 2024-04-17T10:38:39.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.752+0000 7f61d6ffd640 1 --2- 172.21.15.83:0/420558664 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f61c8066f60 0x7f61c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:39.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.752+0000 7f61d6ffd640 1 -- 172.21.15.83:0/420558664 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61d8085ab0 con 0x7f61d0004810 2024-04-17T10:38:39.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.752+0000 7f61e5d9b640 1 --2- 172.21.15.83:0/420558664 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f61c8066f60 0x7f61c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:39.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.752+0000 7f61e5d9b640 1 --2- 172.21.15.83:0/420558664 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f61c8066f60 0x7f61c8069420 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f61dc001750 tx=0x7f61dc0011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:39.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.756+0000 7f61d6ffd640 1 -- 172.21.15.83:0/420558664 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f61d8019030 con 0x7f61d0004810 2024-04-17T10:38:39.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.916+0000 7f61ecfe3640 1 -- 172.21.15.83:0/420558664 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61e815a9b0 con 0x7f61d0004810 2024-04-17T10:38:39.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.920+0000 7f61d6ffd640 1 -- 172.21.15.83:0/420558664 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16533 (secure 0 0 0) 0x7f61d8056ac0 con 0x7f61d0004810 2024-04-17T10:38:39.915 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:39.915 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":20,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:38:13.690876+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:38:10.464490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T10:38:11.043134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:38:39.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.920+0000 7f61ecfe3640 1 -- 172.21.15.83:0/420558664 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f61c8066f60 msgr2=0x7f61c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:39.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.920+0000 7f61ecfe3640 1 --2- 172.21.15.83:0/420558664 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f61c8066f60 0x7f61c8069420 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f61dc001750 tx=0x7f61dc0011a0 comp rx=0 tx=0).stop 2024-04-17T10:38:39.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.920+0000 7f61ecfe3640 1 -- 172.21.15.83:0/420558664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61d0004810 msgr2=0x7f61e815a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:39.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.920+0000 7f61ecfe3640 1 --2- 172.21.15.83:0/420558664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61d0004810 0x7f61e815a090 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f61d8011690 tx=0x7f61d8011b50 comp rx=0 tx=0).stop 2024-04-17T10:38:39.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.920+0000 7f61ecfe3640 1 -- 172.21.15.83:0/420558664 shutdown_connections 2024-04-17T10:38:39.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.920+0000 7f61ecfe3640 1 --2- 172.21.15.83:0/420558664 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f61c8066f60 0x7f61c8069420 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:39.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.920+0000 7f61ecfe3640 1 --2- 172.21.15.83:0/420558664 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61e8159320 0x7f61e81597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:39.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.920+0000 7f61ecfe3640 1 --2- 172.21.15.83:0/420558664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61e815a5d0 0x7f61e8158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:39.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.920+0000 7f61ecfe3640 1 --2- 172.21.15.83:0/420558664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61d0004810 0x7f61e815a090 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:39.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.920+0000 7f61ecfe3640 1 -- 172.21.15.83:0/420558664 >> 172.21.15.83:0/420558664 conn(0x7f61e806abe0 msgr2=0x7f61e80cea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:39.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.920+0000 7f61ecfe3640 1 -- 172.21.15.83:0/420558664 shutdown_connections 2024-04-17T10:38:39.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:39.920+0000 7f61ecfe3640 1 -- 172.21.15.83:0/420558664 wait complete. 2024-04-17T10:38:39.934 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:38:10.464490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:38:11.043134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:38:39.935 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:38:40.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.060+0000 7faf0c803640 1 Processor -- start 2024-04-17T10:38:40.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.060+0000 7faf0c803640 1 -- start start 2024-04-17T10:38:40.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.060+0000 7faf0c803640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf0815bab0 0x7faf0815be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:40.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.060+0000 7faf0c803640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faf0806c260 0x7faf080acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:40.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.060+0000 7faf0c803640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faf080ad470 0x7faf08072060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:40.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.060+0000 7faf0c803640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf080ad8a0 con 0x7faf080ad470 2024-04-17T10:38:40.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.060+0000 7faf0c803640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf0806aef0 con 0x7faf0806c260 2024-04-17T10:38:40.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.060+0000 7faf0c803640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf0806b1d0 con 0x7faf0815bab0 2024-04-17T10:38:40.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.060+0000 7faf06d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faf080ad470 0x7faf08072060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:40.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.060+0000 7faf06d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faf080ad470 0x7faf08072060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52560/0 (socket says 172.21.15.83:52560) 2024-04-17T10:38:40.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.060+0000 7faf06d9d640 1 -- 172.21.15.83:0/758008295 learned_addr learned my addr 172.21.15.83:0/758008295 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:40.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.060+0000 7faf0659c640 1 --2- 172.21.15.83:0/758008295 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf0815bab0 0x7faf0815be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:40.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.060+0000 7faf05d9b640 1 --2- 172.21.15.83:0/758008295 >> v2:172.21.15.167:3300/0 conn(0x7faf0806c260 0x7faf080acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:40.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.060+0000 7faf06d9d640 1 -- 172.21.15.83:0/758008295 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf0815bab0 msgr2=0x7faf0815be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:40.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.060+0000 7faf06d9d640 1 --2- 172.21.15.83:0/758008295 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf0815bab0 0x7faf0815be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.060+0000 7faf06d9d640 1 -- 172.21.15.83:0/758008295 >> v2:172.21.15.167:3300/0 conn(0x7faf0806c260 msgr2=0x7faf080acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:40.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.060+0000 7faf06d9d640 1 --2- 172.21.15.83:0/758008295 >> v2:172.21.15.167:3300/0 conn(0x7faf0806c260 0x7faf080acf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.060+0000 7faf06d9d640 1 -- 172.21.15.83:0/758008295 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf0806b4b0 con 0x7faf080ad470 2024-04-17T10:38:40.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.064+0000 7faf06d9d640 1 --2- 172.21.15.83:0/758008295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf080ad470 0x7faf08072060 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faf08067200 tx=0x7faf0001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e8875e987c9e926 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:40.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.064+0000 7faf0559a640 1 -- 172.21.15.83:0/758008295 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf000a5840 con 0x7faf080ad470 2024-04-17T10:38:40.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.064+0000 7faf0559a640 1 -- 172.21.15.83:0/758008295 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf000a59e0 con 0x7faf080ad470 2024-04-17T10:38:40.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.064+0000 7faf0559a640 1 -- 172.21.15.83:0/758008295 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf000a5db0 con 0x7faf080ad470 2024-04-17T10:38:40.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.064+0000 7faf0c803640 1 -- 172.21.15.83:0/758008295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf080ad470 msgr2=0x7faf08072060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:40.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.064+0000 7faf0c803640 1 --2- 172.21.15.83:0/758008295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf080ad470 0x7faf08072060 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faf08067200 tx=0x7faf0001c9c0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.064+0000 7faf0c803640 1 -- 172.21.15.83:0/758008295 shutdown_connections 2024-04-17T10:38:40.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.064+0000 7faf0c803640 1 --2- 172.21.15.83:0/758008295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf080ad470 0x7faf08072060 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.064+0000 7faf0c803640 1 --2- 172.21.15.83:0/758008295 >> v2:172.21.15.167:3300/0 conn(0x7faf0806c260 0x7faf080acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.064+0000 7faf0c803640 1 --2- 172.21.15.83:0/758008295 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf0815bab0 0x7faf0815be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.064+0000 7faf0c803640 1 -- 172.21.15.83:0/758008295 >> 172.21.15.83:0/758008295 conn(0x7faf080b1e20 msgr2=0x7faf080b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:40.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.064+0000 7faf0c803640 1 -- 172.21.15.83:0/758008295 shutdown_connections 2024-04-17T10:38:40.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.064+0000 7faf0c803640 1 -- 172.21.15.83:0/758008295 wait complete. 2024-04-17T10:38:40.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.064+0000 7faf0c803640 1 Processor -- start 2024-04-17T10:38:40.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.064+0000 7faf0c803640 1 -- start start 2024-04-17T10:38:40.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.068+0000 7faf0c803640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf0806c260 0x7faf0815ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:40.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.068+0000 7faf0c803640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf080ad470 0x7faf0806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:40.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.068+0000 7faf0c803640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf0815bab0 0x7faf080c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:40.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.068+0000 7faf0c803640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf080ca8c0 con 0x7faf0815bab0 2024-04-17T10:38:40.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.068+0000 7faf0c803640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf080ca740 con 0x7faf0806c260 2024-04-17T10:38:40.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.068+0000 7faf0c803640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf080caa40 con 0x7faf080ad470 2024-04-17T10:38:40.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.068+0000 7faf06d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf0815bab0 0x7faf080c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:40.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.068+0000 7faf06d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf0815bab0 0x7faf080c92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52562/0 (socket says 172.21.15.83:52562) 2024-04-17T10:38:40.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.068+0000 7faf06d9d640 1 -- 172.21.15.83:0/315924063 learned_addr learned my addr 172.21.15.83:0/315924063 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:40.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.068+0000 7faf05d9b640 1 --2- 172.21.15.83:0/315924063 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf080ad470 0x7faf0806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:40.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.068+0000 7faf0659c640 1 --2- 172.21.15.83:0/315924063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf0806c260 0x7faf0815ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:40.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.068+0000 7faf06d9d640 1 -- 172.21.15.83:0/315924063 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf080ad470 msgr2=0x7faf0806d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:40.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.068+0000 7faf06d9d640 1 --2- 172.21.15.83:0/315924063 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf080ad470 0x7faf0806d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.068+0000 7faf06d9d640 1 -- 172.21.15.83:0/315924063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf0806c260 msgr2=0x7faf0815ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:40.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.068+0000 7faf06d9d640 1 --2- 172.21.15.83:0/315924063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf0806c260 0x7faf0815ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.068+0000 7faf06d9d640 1 -- 172.21.15.83:0/315924063 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf0806ebd0 con 0x7faf0815bab0 2024-04-17T10:38:40.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.068+0000 7faf06d9d640 1 --2- 172.21.15.83:0/315924063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf0815bab0 0x7faf080c92d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7faf000af040 tx=0x7faf00019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:40.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.068+0000 7faef6ffd640 1 -- 172.21.15.83:0/315924063 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf000b8670 con 0x7faf0815bab0 2024-04-17T10:38:40.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.068+0000 7faef6ffd640 1 -- 172.21.15.83:0/315924063 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf000be070 con 0x7faf0815bab0 2024-04-17T10:38:40.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.068+0000 7faf0c803640 1 -- 172.21.15.83:0/315924063 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf0815b2a0 con 0x7faf0815bab0 2024-04-17T10:38:40.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.068+0000 7faef6ffd640 1 -- 172.21.15.83:0/315924063 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf000b8a50 con 0x7faf0815bab0 2024-04-17T10:38:40.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.068+0000 7faf0c803640 1 -- 172.21.15.83:0/315924063 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf0806e330 con 0x7faf0815bab0 2024-04-17T10:38:40.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.068+0000 7faf0c803640 1 -- 172.21.15.83:0/315924063 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf08074f90 con 0x7faf0815bab0 2024-04-17T10:38:40.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.068+0000 7faef6ffd640 1 -- 172.21.15.83:0/315924063 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faf000b8bf0 con 0x7faf0815bab0 2024-04-17T10:38:40.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.068+0000 7faef6ffd640 1 --2- 172.21.15.83:0/315924063 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faee8066f60 0x7faee8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:40.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.068+0000 7faef6ffd640 1 -- 172.21.15.83:0/315924063 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf00120240 con 0x7faf0815bab0 2024-04-17T10:38:40.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.068+0000 7faf0659c640 1 --2- 172.21.15.83:0/315924063 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faee8066f60 0x7faee8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:40.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.068+0000 7faf0659c640 1 --2- 172.21.15.83:0/315924063 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faee8066f60 0x7faee8069420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7faefc009a50 tx=0x7faefc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:40.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.072+0000 7faef6ffd640 1 -- 172.21.15.83:0/315924063 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faf000165e0 con 0x7faf0815bab0 2024-04-17T10:38:40.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.232+0000 7faf0c803640 1 -- 172.21.15.83:0/315924063 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faf080b2200 con 0x7faf0815bab0 2024-04-17T10:38:40.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.232+0000 7faef6ffd640 1 -- 172.21.15.83:0/315924063 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7faf000f0cc0 con 0x7faf0815bab0 2024-04-17T10:38:40.230 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:38:40.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.236+0000 7faf0c803640 1 -- 172.21.15.83:0/315924063 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faee8066f60 msgr2=0x7faee8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:40.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.236+0000 7faf0c803640 1 --2- 172.21.15.83:0/315924063 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faee8066f60 0x7faee8069420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7faefc009a50 tx=0x7faefc008040 comp rx=0 tx=0).stop 2024-04-17T10:38:40.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.236+0000 7faf0c803640 1 -- 172.21.15.83:0/315924063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf0815bab0 msgr2=0x7faf080c92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:40.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.236+0000 7faf0c803640 1 --2- 172.21.15.83:0/315924063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf0815bab0 0x7faf080c92d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7faf000af040 tx=0x7faf00019760 comp rx=0 tx=0).stop 2024-04-17T10:38:40.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.236+0000 7faf0c803640 1 -- 172.21.15.83:0/315924063 shutdown_connections 2024-04-17T10:38:40.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.236+0000 7faf0c803640 1 --2- 172.21.15.83:0/315924063 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faee8066f60 0x7faee8069420 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.236+0000 7faf0c803640 1 --2- 172.21.15.83:0/315924063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf0815bab0 0x7faf080c92d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.236+0000 7faf0c803640 1 --2- 172.21.15.83:0/315924063 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf080ad470 0x7faf0806d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.236+0000 7faf0c803640 1 --2- 172.21.15.83:0/315924063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf0806c260 0x7faf0815ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.236+0000 7faf0c803640 1 -- 172.21.15.83:0/315924063 >> 172.21.15.83:0/315924063 conn(0x7faf080b1e20 msgr2=0x7faf080b0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:40.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.236+0000 7faf0c803640 1 -- 172.21.15.83:0/315924063 shutdown_connections 2024-04-17T10:38:40.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.236+0000 7faf0c803640 1 -- 172.21.15.83:0/315924063 wait complete. 2024-04-17T10:38:40.249 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:38:40.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.376+0000 7fe431165640 1 Processor -- start 2024-04-17T10:38:40.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.376+0000 7fe431165640 1 -- start start 2024-04-17T10:38:40.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.376+0000 7fe431165640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe42c15e0f0 0x7fe42c0b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:40.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.376+0000 7fe431165640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe42c0b5450 0x7fe42c0b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:40.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.376+0000 7fe431165640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe42c15e4d0 con 0x7fe42c0ce400 2024-04-17T10:38:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.376+0000 7fe431165640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe42c162b40 con 0x7fe42c15e0f0 2024-04-17T10:38:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.376+0000 7fe431165640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe42c162320 con 0x7fe42c0b5450 2024-04-17T10:38:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.376+0000 7fe42ad9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe42c15e0f0 0x7fe42c0b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.376+0000 7fe42a59c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe42c0ce400 0x7fe42c161c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47234/0 (socket says 172.21.15.83:47234) 2024-04-17T10:38:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.376+0000 7fe42a59c640 1 -- 172.21.15.83:0/1778347010 learned_addr learned my addr 172.21.15.83:0/1778347010 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.376+0000 7fe42b59e640 1 --2- 172.21.15.83:0/1778347010 >> v2:172.21.15.167:3300/0 conn(0x7fe42c0b5450 0x7fe42c0b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.376+0000 7fe42ad9d640 1 -- 172.21.15.83:0/1778347010 >> v2:172.21.15.167:3300/0 conn(0x7fe42c0b5450 msgr2=0x7fe42c0b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.376+0000 7fe42ad9d640 1 --2- 172.21.15.83:0/1778347010 >> v2:172.21.15.167:3300/0 conn(0x7fe42c0b5450 0x7fe42c0b58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.376+0000 7fe42ad9d640 1 -- 172.21.15.83:0/1778347010 >> v1:172.21.15.83:6789/0 conn(0x7fe42c0ce400 legacy=0x7fe42c161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.376+0000 7fe42ad9d640 1 -- 172.21.15.83:0/1778347010 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe42c162580 con 0x7fe42c15e0f0 2024-04-17T10:38:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.376+0000 7fe429d9b640 1 -- 172.21.15.83:0/1778347010 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3237276916 0 0) 0x7fe42c15e4d0 con 0x7fe42c0ce400 2024-04-17T10:38:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.376+0000 7fe42b59e640 1 --2- 172.21.15.83:0/1778347010 >> v2:172.21.15.167:3300/0 conn(0x7fe42c0b5450 0x7fe42c0b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:40.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.380+0000 7fe42ad9d640 1 --2- 172.21.15.83:0/1778347010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe42c15e0f0 0x7fe42c0b6d00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe42c0d11a0 tx=0x7fe42001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a352fb8e2cb915c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:40.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.380+0000 7fe429d9b640 1 -- 172.21.15.83:0/1778347010 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4200b0070 con 0x7fe42c15e0f0 2024-04-17T10:38:40.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.380+0000 7fe429d9b640 1 -- 172.21.15.83:0/1778347010 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4200ab600 con 0x7fe42c15e0f0 2024-04-17T10:38:40.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.380+0000 7fe429d9b640 1 -- 172.21.15.83:0/1778347010 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4200a4e60 con 0x7fe42c15e0f0 2024-04-17T10:38:40.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.380+0000 7fe431165640 1 -- 172.21.15.83:0/1778347010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe42c15e0f0 msgr2=0x7fe42c0b6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:40.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.380+0000 7fe431165640 1 --2- 172.21.15.83:0/1778347010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe42c15e0f0 0x7fe42c0b6d00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe42c0d11a0 tx=0x7fe42001c3d0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.380+0000 7fe431165640 1 -- 172.21.15.83:0/1778347010 shutdown_connections 2024-04-17T10:38:40.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.380+0000 7fe431165640 1 --2- 172.21.15.83:0/1778347010 >> v2:172.21.15.167:3300/0 conn(0x7fe42c0b5450 0x7fe42c0b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.380+0000 7fe431165640 1 --2- 172.21.15.83:0/1778347010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe42c15e0f0 0x7fe42c0b6d00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.380+0000 7fe431165640 1 -- 172.21.15.83:0/1778347010 >> 172.21.15.83:0/1778347010 conn(0x7fe42c0d0450 msgr2=0x7fe42c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:40.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.380+0000 7fe431165640 1 -- 172.21.15.83:0/1778347010 shutdown_connections 2024-04-17T10:38:40.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.380+0000 7fe431165640 1 -- 172.21.15.83:0/1778347010 wait complete. 2024-04-17T10:38:40.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.380+0000 7fe431165640 1 Processor -- start 2024-04-17T10:38:40.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.384+0000 7fe431165640 1 -- start start 2024-04-17T10:38:40.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.384+0000 7fe431165640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe42c0b5450 0x7fe42c0cce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:40.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.384+0000 7fe431165640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe42c0ce400 0x7fe42c151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:40.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.384+0000 7fe431165640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe42c15e0f0 0x7fe42c0c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:40.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.384+0000 7fe431165640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c0d2ed0 con 0x7fe42c0b5450 2024-04-17T10:38:40.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.384+0000 7fe431165640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c0d2d50 con 0x7fe42c15e0f0 2024-04-17T10:38:40.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.384+0000 7fe431165640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c0d3050 con 0x7fe42c0ce400 2024-04-17T10:38:40.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.384+0000 7fe42ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe42c0b5450 0x7fe42c0cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:40.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.384+0000 7fe42ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe42c0b5450 0x7fe42c0cce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52568/0 (socket says 172.21.15.83:52568) 2024-04-17T10:38:40.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.384+0000 7fe42ad9d640 1 -- 172.21.15.83:0/4052622625 learned_addr learned my addr 172.21.15.83:0/4052622625 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:40.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.384+0000 7fe42a59c640 1 --2- 172.21.15.83:0/4052622625 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe42c0ce400 0x7fe42c151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:40.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.384+0000 7fe42ad9d640 1 -- 172.21.15.83:0/4052622625 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe42c0ce400 msgr2=0x7fe42c151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:40.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.384+0000 7fe42ad9d640 1 --2- 172.21.15.83:0/4052622625 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe42c0ce400 0x7fe42c151730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.384+0000 7fe42b59e640 1 --2- 172.21.15.83:0/4052622625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe42c15e0f0 0x7fe42c0c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:40.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.384+0000 7fe42ad9d640 1 -- 172.21.15.83:0/4052622625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe42c15e0f0 msgr2=0x7fe42c0c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:40.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.384+0000 7fe42ad9d640 1 --2- 172.21.15.83:0/4052622625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe42c15e0f0 0x7fe42c0c9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.384+0000 7fe42ad9d640 1 -- 172.21.15.83:0/4052622625 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe42c151c70 con 0x7fe42c0b5450 2024-04-17T10:38:40.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.384+0000 7fe42ad9d640 1 --2- 172.21.15.83:0/4052622625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe42c0b5450 0x7fe42c0cce90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe4200ae040 tx=0x7fe42001ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:40.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.384+0000 7fe4137fe640 1 -- 172.21.15.83:0/4052622625 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4200b0050 con 0x7fe42c0b5450 2024-04-17T10:38:40.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.384+0000 7fe4137fe640 1 -- 172.21.15.83:0/4052622625 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4200bc070 con 0x7fe42c0b5450 2024-04-17T10:38:40.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.384+0000 7fe431165640 1 -- 172.21.15.83:0/4052622625 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe42c0cd4f0 con 0x7fe42c0b5450 2024-04-17T10:38:40.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.384+0000 7fe4137fe640 1 -- 172.21.15.83:0/4052622625 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4200b5870 con 0x7fe42c0b5450 2024-04-17T10:38:40.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.384+0000 7fe431165640 1 -- 172.21.15.83:0/4052622625 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe42c14bbb0 con 0x7fe42c0b5450 2024-04-17T10:38:40.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.384+0000 7fe431165640 1 -- 172.21.15.83:0/4052622625 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe42c0681a0 con 0x7fe42c0b5450 2024-04-17T10:38:40.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.384+0000 7fe4137fe640 1 -- 172.21.15.83:0/4052622625 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe4200b5a10 con 0x7fe42c0b5450 2024-04-17T10:38:40.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.384+0000 7fe4137fe640 1 --2- 172.21.15.83:0/4052622625 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe418066f60 0x7fe418069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:40.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.384+0000 7fe4137fe640 1 -- 172.21.15.83:0/4052622625 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe42011f730 con 0x7fe42c0b5450 2024-04-17T10:38:40.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.384+0000 7fe42a59c640 1 --2- 172.21.15.83:0/4052622625 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe418066f60 0x7fe418069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:40.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.388+0000 7fe42a59c640 1 --2- 172.21.15.83:0/4052622625 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe418066f60 0x7fe418069420 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe42c1515f0 tx=0x7fe41c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:40.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.392+0000 7fe4137fe640 1 -- 172.21.15.83:0/4052622625 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe420016050 con 0x7fe42c0b5450 2024-04-17T10:38:40.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.548+0000 7fe431165640 1 -- 172.21.15.83:0/4052622625 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe42c074f90 con 0x7fe42c0b5450 2024-04-17T10:38:40.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.548+0000 7fe4137fe640 1 -- 172.21.15.83:0/4052622625 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe4200f08f0 con 0x7fe42c0b5450 2024-04-17T10:38:40.545 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:38:40.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.552+0000 7fe431165640 1 -- 172.21.15.83:0/4052622625 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe418066f60 msgr2=0x7fe418069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:40.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.552+0000 7fe431165640 1 --2- 172.21.15.83:0/4052622625 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe418066f60 0x7fe418069420 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe42c1515f0 tx=0x7fe41c009040 comp rx=0 tx=0).stop 2024-04-17T10:38:40.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.552+0000 7fe431165640 1 -- 172.21.15.83:0/4052622625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe42c0b5450 msgr2=0x7fe42c0cce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:40.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.552+0000 7fe431165640 1 --2- 172.21.15.83:0/4052622625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe42c0b5450 0x7fe42c0cce90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe4200ae040 tx=0x7fe42001ac10 comp rx=0 tx=0).stop 2024-04-17T10:38:40.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.552+0000 7fe431165640 1 -- 172.21.15.83:0/4052622625 shutdown_connections 2024-04-17T10:38:40.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.552+0000 7fe431165640 1 --2- 172.21.15.83:0/4052622625 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe418066f60 0x7fe418069420 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.552+0000 7fe431165640 1 --2- 172.21.15.83:0/4052622625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe42c15e0f0 0x7fe42c0c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.552+0000 7fe431165640 1 --2- 172.21.15.83:0/4052622625 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe42c0ce400 0x7fe42c151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.552+0000 7fe431165640 1 --2- 172.21.15.83:0/4052622625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe42c0b5450 0x7fe42c0cce90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.552+0000 7fe431165640 1 -- 172.21.15.83:0/4052622625 >> 172.21.15.83:0/4052622625 conn(0x7fe42c0d0450 msgr2=0x7fe42c14d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:40.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.552+0000 7fe431165640 1 -- 172.21.15.83:0/4052622625 shutdown_connections 2024-04-17T10:38:40.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.552+0000 7fe431165640 1 -- 172.21.15.83:0/4052622625 wait complete. 2024-04-17T10:38:40.564 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:38:40.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febbe892640 1 Processor -- start 2024-04-17T10:38:40.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febbe892640 1 -- start start 2024-04-17T10:38:40.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febbe892640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febb806ae10 0x7febb806b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:40.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febbe892640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7febb815e330 0x7febb815e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:40.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febbe892640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febb81625d0 con 0x7febb80ad840 2024-04-17T10:38:40.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febbe892640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febb806b730 con 0x7febb815e330 2024-04-17T10:38:40.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febbe892640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febb8161810 con 0x7febb806ae10 2024-04-17T10:38:40.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febbce2f640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7febb80ad840 0x7febb80adcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47238/0 (socket says 172.21.15.83:47238) 2024-04-17T10:38:40.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febbce2f640 1 -- 172.21.15.83:0/3978652206 learned_addr learned my addr 172.21.15.83:0/3978652206 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:40.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febb7fff640 1 --2- 172.21.15.83:0/3978652206 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febb806ae10 0x7febb806b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:40.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febb77fe640 1 --2- 172.21.15.83:0/3978652206 >> v2:172.21.15.167:3300/0 conn(0x7febb815e330 0x7febb815e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:40.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febb6ffd640 1 -- 172.21.15.83:0/3978652206 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1225723204 0 0) 0x7febb81625d0 con 0x7febb80ad840 2024-04-17T10:38:40.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febb6ffd640 1 -- 172.21.15.83:0/3978652206 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feba0003530 con 0x7febb80ad840 2024-04-17T10:38:40.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febb6ffd640 1 -- 172.21.15.83:0/3978652206 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1817010363 0 0) 0x7feba0003530 con 0x7febb80ad840 2024-04-17T10:38:40.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febb6ffd640 1 -- 172.21.15.83:0/3978652206 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febb806ae10 msgr2=0x7febb806b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:40.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febb6ffd640 1 --2- 172.21.15.83:0/3978652206 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febb806ae10 0x7febb806b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febb6ffd640 1 -- 172.21.15.83:0/3978652206 >> v2:172.21.15.167:3300/0 conn(0x7febb815e330 msgr2=0x7febb815e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:40.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febb6ffd640 1 --2- 172.21.15.83:0/3978652206 >> v2:172.21.15.167:3300/0 conn(0x7febb815e330 0x7febb815e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febb6ffd640 1 -- 172.21.15.83:0/3978652206 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb8161af0 con 0x7febb80ad840 2024-04-17T10:38:40.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febb6ffd640 1 -- 172.21.15.83:0/3978652206 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7febb0001a00 con 0x7febb80ad840 2024-04-17T10:38:40.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febb6ffd640 1 -- 172.21.15.83:0/3978652206 >> v1:172.21.15.83:6789/0 conn(0x7febb80ad840 legacy=0x7febb80adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:40.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febb6ffd640 1 --2- 172.21.15.83:0/3978652206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feba0003a30 0x7feba0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:40.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febb6ffd640 1 --2- 172.21.15.83:0/3978652206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feba0004810 0x7feba0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:40.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febb6ffd640 1 --2- 172.21.15.83:0/3978652206 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febb806ae10 0x7feba00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:40.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febb6ffd640 1 -- 172.21.15.83:0/3978652206 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb806b730 con 0x7feba0004810 2024-04-17T10:38:40.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febb6ffd640 1 -- 172.21.15.83:0/3978652206 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb8161810 con 0x7feba0003a30 2024-04-17T10:38:40.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febb6ffd640 1 -- 172.21.15.83:0/3978652206 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba000ba50 con 0x7febb806ae10 2024-04-17T10:38:40.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febb77fe640 1 --2- 172.21.15.83:0/3978652206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feba0004810 0x7feba0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:40.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febb7fff640 1 --2- 172.21.15.83:0/3978652206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feba0003a30 0x7feba0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:40.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febbce2f640 1 --2- 172.21.15.83:0/3978652206 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febb806ae10 0x7feba00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:40.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febb77fe640 1 -- 172.21.15.83:0/3978652206 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febb806ae10 msgr2=0x7feba00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:40.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febb77fe640 1 --2- 172.21.15.83:0/3978652206 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febb806ae10 0x7feba00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febb77fe640 1 -- 172.21.15.83:0/3978652206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feba0003a30 msgr2=0x7feba0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:40.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febb77fe640 1 --2- 172.21.15.83:0/3978652206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feba0003a30 0x7feba0003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.688+0000 7febb77fe640 1 -- 172.21.15.83:0/3978652206 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba0003530 con 0x7feba0004810 2024-04-17T10:38:40.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.692+0000 7febb77fe640 1 --2- 172.21.15.83:0/3978652206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feba0004810 0x7feba0004c90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7febac009ae0 tx=0x7febac01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3426a0a2d78798b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:40.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.692+0000 7febb6ffd640 1 -- 172.21.15.83:0/3978652206 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac0096b0 con 0x7feba0004810 2024-04-17T10:38:40.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.692+0000 7febb6ffd640 1 -- 172.21.15.83:0/3978652206 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febac0a64a0 con 0x7feba0004810 2024-04-17T10:38:40.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.692+0000 7febb6ffd640 1 -- 172.21.15.83:0/3978652206 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac0a6890 con 0x7feba0004810 2024-04-17T10:38:40.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.692+0000 7febbe892640 1 -- 172.21.15.83:0/3978652206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feba0004810 msgr2=0x7feba0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:40.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.692+0000 7febbe892640 1 --2- 172.21.15.83:0/3978652206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feba0004810 0x7feba0004c90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7febac009ae0 tx=0x7febac01cab0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.692+0000 7febbce2f640 1 -- 172.21.15.83:0/3978652206 reap_dead start 2024-04-17T10:38:40.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.692+0000 7febbe892640 1 -- 172.21.15.83:0/3978652206 shutdown_connections 2024-04-17T10:38:40.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.692+0000 7febbe892640 1 -- 172.21.15.83:0/3978652206 >> 172.21.15.83:0/3978652206 conn(0x7febb80b15b0 msgr2=0x7febb80b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:40.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.692+0000 7febbe892640 1 -- 172.21.15.83:0/3978652206 shutdown_connections 2024-04-17T10:38:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.692+0000 7febbe892640 1 -- 172.21.15.83:0/3978652206 wait complete. 2024-04-17T10:38:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.692+0000 7febbe892640 1 Processor -- start 2024-04-17T10:38:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.692+0000 7febbe892640 1 -- start start 2024-04-17T10:38:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.696+0000 7febbe892640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feba0004810 0x7febb8143430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.696+0000 7febbe892640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febb8166610 0x7febb8166a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.696+0000 7febbe892640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febb8143d90 0x7febb8144210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.696+0000 7febbe892640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb80cf470 con 0x7feba0004810 2024-04-17T10:38:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.696+0000 7febbe892640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb80cf2f0 con 0x7febb8166610 2024-04-17T10:38:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.696+0000 7febbe892640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febb815ecb0 con 0x7febb8143d90 2024-04-17T10:38:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.696+0000 7febb7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feba0004810 0x7febb8143430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.696+0000 7febb7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feba0004810 0x7febb8143430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52598/0 (socket says 172.21.15.83:52598) 2024-04-17T10:38:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.696+0000 7febb7fff640 1 -- 172.21.15.83:0/1189734266 learned_addr learned my addr 172.21.15.83:0/1189734266 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.696+0000 7febbce2f640 1 --2- 172.21.15.83:0/1189734266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febb8143d90 0x7febb8144210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.696+0000 7febb77fe640 1 --2- 172.21.15.83:0/1189734266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febb8166610 0x7febb8166a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:40.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.696+0000 7febb7fff640 1 -- 172.21.15.83:0/1189734266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febb8143d90 msgr2=0x7febb8144210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:40.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.696+0000 7febb7fff640 1 --2- 172.21.15.83:0/1189734266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febb8143d90 0x7febb8144210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.696+0000 7febb7fff640 1 -- 172.21.15.83:0/1189734266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febb8166610 msgr2=0x7febb8166a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:40.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.696+0000 7febb7fff640 1 --2- 172.21.15.83:0/1189734266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febb8166610 0x7febb8166a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.696+0000 7febb7fff640 1 -- 172.21.15.83:0/1189734266 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb8165570 con 0x7feba0004810 2024-04-17T10:38:40.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.696+0000 7febb77fe640 1 --2- 172.21.15.83:0/1189734266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febb8166610 0x7febb8166a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:40.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.696+0000 7febb7fff640 1 --2- 172.21.15.83:0/1189734266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feba0004810 0x7febb8143430 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7feba8011610 tx=0x7feba8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:40.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.696+0000 7febb4ff9640 1 -- 172.21.15.83:0/1189734266 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba800f8f0 con 0x7feba0004810 2024-04-17T10:38:40.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.696+0000 7febb4ff9640 1 -- 172.21.15.83:0/1189734266 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba800fa90 con 0x7feba0004810 2024-04-17T10:38:40.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.696+0000 7febbe892640 1 -- 172.21.15.83:0/1189734266 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febb8165850 con 0x7feba0004810 2024-04-17T10:38:40.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.696+0000 7febb4ff9640 1 -- 172.21.15.83:0/1189734266 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba800fe40 con 0x7feba0004810 2024-04-17T10:38:40.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.696+0000 7febbe892640 1 -- 172.21.15.83:0/1189734266 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb8165e10 con 0x7feba0004810 2024-04-17T10:38:40.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.696+0000 7febbe892640 1 -- 172.21.15.83:0/1189734266 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febb8074f90 con 0x7feba0004810 2024-04-17T10:38:40.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.696+0000 7febb4ff9640 1 -- 172.21.15.83:0/1189734266 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feba8015460 con 0x7feba0004810 2024-04-17T10:38:40.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.696+0000 7febb4ff9640 1 --2- 172.21.15.83:0/1189734266 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feba4066f60 0x7feba4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:40.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.696+0000 7febb4ff9640 1 -- 172.21.15.83:0/1189734266 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feba80858e0 con 0x7feba0004810 2024-04-17T10:38:40.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.696+0000 7febb77fe640 1 --2- 172.21.15.83:0/1189734266 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feba4066f60 0x7feba4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:40.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.696+0000 7febb77fe640 1 --2- 172.21.15.83:0/1189734266 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feba4066f60 0x7feba4069420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7febac0ad000 tx=0x7febac0046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:40.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.700+0000 7febb4ff9640 1 -- 172.21.15.83:0/1189734266 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feba8019030 con 0x7feba0004810 2024-04-17T10:38:40.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.856+0000 7febbe892640 1 -- 172.21.15.83:0/1189734266 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7febb8072740 con 0x7feba0004810 2024-04-17T10:38:40.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.856+0000 7febb4ff9640 1 -- 172.21.15.83:0/1189734266 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7feba80568f0 con 0x7feba0004810 2024-04-17T10:38:40.854 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:38:40.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.860+0000 7febbe892640 1 -- 172.21.15.83:0/1189734266 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feba4066f60 msgr2=0x7feba4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:40.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.860+0000 7febbe892640 1 --2- 172.21.15.83:0/1189734266 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feba4066f60 0x7feba4069420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7febac0ad000 tx=0x7febac0046d0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.860+0000 7febbe892640 1 -- 172.21.15.83:0/1189734266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feba0004810 msgr2=0x7febb8143430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:40.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.860+0000 7febbe892640 1 --2- 172.21.15.83:0/1189734266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feba0004810 0x7febb8143430 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7feba8011610 tx=0x7feba8011ad0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.860+0000 7febbe892640 1 -- 172.21.15.83:0/1189734266 shutdown_connections 2024-04-17T10:38:40.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.860+0000 7febbe892640 1 --2- 172.21.15.83:0/1189734266 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feba4066f60 0x7feba4069420 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.860+0000 7febbe892640 1 --2- 172.21.15.83:0/1189734266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febb8143d90 0x7febb8144210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.860+0000 7febbe892640 1 --2- 172.21.15.83:0/1189734266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febb8166610 0x7febb8166a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.860+0000 7febbe892640 1 --2- 172.21.15.83:0/1189734266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feba0004810 0x7febb8143430 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.860+0000 7febbe892640 1 -- 172.21.15.83:0/1189734266 >> 172.21.15.83:0/1189734266 conn(0x7febb80b15b0 msgr2=0x7febb80aff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:40.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.860+0000 7febbe892640 1 -- 172.21.15.83:0/1189734266 shutdown_connections 2024-04-17T10:38:40.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.860+0000 7febbe892640 1 -- 172.21.15.83:0/1189734266 wait complete. 2024-04-17T10:38:40.872 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:38:40.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.984+0000 7f4858860640 1 Processor -- start 2024-04-17T10:38:40.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.984+0000 7f4858860640 1 -- start start 2024-04-17T10:38:40.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.984+0000 7f4858860640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f485415dfa0 0x7f48540b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:40.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.984+0000 7f4858860640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48540ce980 0x7f48540b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:40.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.984+0000 7f4858860640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4854162a00 con 0x7f48540cee00 2024-04-17T10:38:40.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.984+0000 7f4858860640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4854161e80 con 0x7f485415dfa0 2024-04-17T10:38:40.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.984+0000 7f4858860640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4854162160 con 0x7f48540ce980 2024-04-17T10:38:40.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.984+0000 7f485259c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f485415dfa0 0x7f48540b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:40.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.984+0000 7f484bfff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48540ce980 0x7f48540b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:40.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.984+0000 7f484bfff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48540ce980 0x7f48540b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:48774/0 (socket says 172.21.15.83:48774) 2024-04-17T10:38:40.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.984+0000 7f484bfff640 1 -- 172.21.15.83:0/2845250991 learned_addr learned my addr 172.21.15.83:0/2845250991 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:40.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.984+0000 7f485259c640 1 -- 172.21.15.83:0/2845250991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48540ce980 msgr2=0x7f48540b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:40.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.984+0000 7f485259c640 1 --2- 172.21.15.83:0/2845250991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48540ce980 0x7f48540b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.984+0000 7f485259c640 1 -- 172.21.15.83:0/2845250991 >> v1:172.21.15.167:6789/0 conn(0x7f48540cee00 legacy=0x7f48540d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:40.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.984+0000 7f485259c640 1 -- 172.21.15.83:0/2845250991 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4854162440 con 0x7f485415dfa0 2024-04-17T10:38:40.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.984+0000 7f484bfff640 1 --2- 172.21.15.83:0/2845250991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48540ce980 0x7f48540b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:38:40.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.988+0000 7f485259c640 1 --2- 172.21.15.83:0/2845250991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f485415dfa0 0x7f48540b6c20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f48440097c0 tx=0x7f484401c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a738c23be1cf506a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:40.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.988+0000 7f4851d9b640 1 -- 172.21.15.83:0/2845250991 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48440b0070 con 0x7f485415dfa0 2024-04-17T10:38:40.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.988+0000 7f4851d9b640 1 -- 172.21.15.83:0/2845250991 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48440ab600 con 0x7f485415dfa0 2024-04-17T10:38:40.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.988+0000 7f4851d9b640 1 -- 172.21.15.83:0/2845250991 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48440ab9d0 con 0x7f485415dfa0 2024-04-17T10:38:40.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.988+0000 7f4858860640 1 -- 172.21.15.83:0/2845250991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f485415dfa0 msgr2=0x7f48540b6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:40.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.988+0000 7f4858860640 1 --2- 172.21.15.83:0/2845250991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f485415dfa0 0x7f48540b6c20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f48440097c0 tx=0x7f484401c3e0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.988+0000 7f4858860640 1 -- 172.21.15.83:0/2845250991 shutdown_connections 2024-04-17T10:38:40.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.988+0000 7f4858860640 1 --2- 172.21.15.83:0/2845250991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48540ce980 0x7f48540b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.988+0000 7f4858860640 1 --2- 172.21.15.83:0/2845250991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f485415dfa0 0x7f48540b6c20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.988+0000 7f4858860640 1 -- 172.21.15.83:0/2845250991 >> 172.21.15.83:0/2845250991 conn(0x7f48540d0600 msgr2=0x7f48540d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:40.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.988+0000 7f4858860640 1 -- 172.21.15.83:0/2845250991 shutdown_connections 2024-04-17T10:38:40.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.988+0000 7f4858860640 1 -- 172.21.15.83:0/2845250991 wait complete. 2024-04-17T10:38:40.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.988+0000 7f4858860640 1 Processor -- start 2024-04-17T10:38:40.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.988+0000 7f4858860640 1 -- start start 2024-04-17T10:38:40.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.988+0000 7f4858860640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48540ce980 0x7f4854154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:40.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.988+0000 7f4858860640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48540cee00 0x7f485414fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:40.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.988+0000 7f4858860640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f485415dfa0 0x7f4854150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:40.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.988+0000 7f4858860640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4854161b10 con 0x7f485415dfa0 2024-04-17T10:38:40.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.988+0000 7f4858860640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48540d2850 con 0x7f48540cee00 2024-04-17T10:38:40.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.988+0000 7f4858860640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f485416b660 con 0x7f48540ce980 2024-04-17T10:38:40.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.988+0000 7f4852d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f485415dfa0 0x7f4854150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:40.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.988+0000 7f4852d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f485415dfa0 0x7f4854150530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52610/0 (socket says 172.21.15.83:52610) 2024-04-17T10:38:40.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.988+0000 7f4852d9d640 1 -- 172.21.15.83:0/1006383941 learned_addr learned my addr 172.21.15.83:0/1006383941 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:40.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.992+0000 7f484bfff640 1 --2- 172.21.15.83:0/1006383941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48540cee00 0x7f485414fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:40.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.992+0000 7f485259c640 1 --2- 172.21.15.83:0/1006383941 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48540ce980 0x7f4854154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:40.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.992+0000 7f4852d9d640 1 -- 172.21.15.83:0/1006383941 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48540ce980 msgr2=0x7f4854154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:40.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.992+0000 7f4852d9d640 1 --2- 172.21.15.83:0/1006383941 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48540ce980 0x7f4854154180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.992+0000 7f4852d9d640 1 -- 172.21.15.83:0/1006383941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48540cee00 msgr2=0x7f485414fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:40.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.992+0000 7f4852d9d640 1 --2- 172.21.15.83:0/1006383941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48540cee00 0x7f485414fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:40.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.992+0000 7f4852d9d640 1 -- 172.21.15.83:0/1006383941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f485414ca70 con 0x7f485415dfa0 2024-04-17T10:38:40.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.992+0000 7f4852d9d640 1 --2- 172.21.15.83:0/1006383941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f485415dfa0 0x7f4854150530 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f484c00c760 tx=0x7f484c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:40.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.992+0000 7f484affd640 1 -- 172.21.15.83:0/1006383941 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484c003b80 con 0x7f485415dfa0 2024-04-17T10:38:40.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.992+0000 7f484affd640 1 -- 172.21.15.83:0/1006383941 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f484c003d20 con 0x7f485415dfa0 2024-04-17T10:38:40.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.992+0000 7f4858860640 1 -- 172.21.15.83:0/1006383941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f485414cd50 con 0x7f485415dfa0 2024-04-17T10:38:40.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.992+0000 7f484affd640 1 -- 172.21.15.83:0/1006383941 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484c00aa30 con 0x7f485415dfa0 2024-04-17T10:38:40.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.992+0000 7f4858860640 1 -- 172.21.15.83:0/1006383941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f485414d310 con 0x7f485415dfa0 2024-04-17T10:38:40.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.992+0000 7f4858860640 1 -- 172.21.15.83:0/1006383941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4854074f90 con 0x7f485415dfa0 2024-04-17T10:38:40.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.992+0000 7f484affd640 1 -- 172.21.15.83:0/1006383941 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f484c00abd0 con 0x7f485415dfa0 2024-04-17T10:38:40.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.992+0000 7f484affd640 1 --2- 172.21.15.83:0/1006383941 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4834066f60 0x7f4834069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:40.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.992+0000 7f484affd640 1 -- 172.21.15.83:0/1006383941 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f484c084f30 con 0x7f485415dfa0 2024-04-17T10:38:40.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.992+0000 7f485259c640 1 --2- 172.21.15.83:0/1006383941 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4834066f60 0x7f4834069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:40.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.992+0000 7f485259c640 1 --2- 172.21.15.83:0/1006383941 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4834066f60 0x7f4834069420 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f48440ad000 tx=0x7f48440a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:40.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:40.996+0000 7f484affd640 1 -- 172.21.15.83:0/1006383941 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f484c055e90 con 0x7f485415dfa0 2024-04-17T10:38:41.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.144+0000 7f4858860640 1 -- 172.21.15.83:0/1006383941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48540d17a0 con 0x7f485415dfa0 2024-04-17T10:38:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.148+0000 7f484affd640 1 -- 172.21.15.83:0/1006383941 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16533 (secure 0 0 0) 0x7f484c055790 con 0x7f485415dfa0 2024-04-17T10:38:41.144 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:41.144 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":20,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:38:13.690876+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:38:10.464490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T10:38:11.043134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:38:41.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.148+0000 7f4858860640 1 -- 172.21.15.83:0/1006383941 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4834066f60 msgr2=0x7f4834069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:41.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.148+0000 7f4858860640 1 --2- 172.21.15.83:0/1006383941 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4834066f60 0x7f4834069420 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f48440ad000 tx=0x7f48440a6000 comp rx=0 tx=0).stop 2024-04-17T10:38:41.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.148+0000 7f4858860640 1 -- 172.21.15.83:0/1006383941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f485415dfa0 msgr2=0x7f4854150530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:41.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.148+0000 7f4858860640 1 --2- 172.21.15.83:0/1006383941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f485415dfa0 0x7f4854150530 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f484c00c760 tx=0x7f484c00cc20 comp rx=0 tx=0).stop 2024-04-17T10:38:41.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.148+0000 7f4858860640 1 -- 172.21.15.83:0/1006383941 shutdown_connections 2024-04-17T10:38:41.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.148+0000 7f4858860640 1 --2- 172.21.15.83:0/1006383941 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4834066f60 0x7f4834069420 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:41.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.148+0000 7f4858860640 1 --2- 172.21.15.83:0/1006383941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f485415dfa0 0x7f4854150530 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:41.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.148+0000 7f4858860640 1 --2- 172.21.15.83:0/1006383941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48540cee00 0x7f485414fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:41.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.148+0000 7f4858860640 1 --2- 172.21.15.83:0/1006383941 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48540ce980 0x7f4854154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:41.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.148+0000 7f4858860640 1 -- 172.21.15.83:0/1006383941 >> 172.21.15.83:0/1006383941 conn(0x7f48540d0600 msgr2=0x7f48541609e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:41.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.148+0000 7f4858860640 1 -- 172.21.15.83:0/1006383941 shutdown_connections 2024-04-17T10:38:41.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.148+0000 7f4858860640 1 -- 172.21.15.83:0/1006383941 wait complete. 2024-04-17T10:38:41.162 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:38:10.464490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:38:11.043134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:38:41.162 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:38:41.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca9f09640 1 Processor -- start 2024-04-17T10:38:41.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca9f09640 1 -- start start 2024-04-17T10:38:41.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca9f09640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6ca415bc10 0x7f6ca40d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:41.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca9f09640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca40ced00 con 0x7f6ca406e500 2024-04-17T10:38:41.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca9f09640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca40cefe0 con 0x7f6ca40d0490 2024-04-17T10:38:41.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca9f09640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ca40cc690 con 0x7f6ca415bc10 2024-04-17T10:38:41.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca3fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6ca406e500 0x7f6ca40cf860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47246/0 (socket says 172.21.15.83:47246) 2024-04-17T10:38:41.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca3fff640 1 -- 172.21.15.83:0/3498702145 learned_addr learned my addr 172.21.15.83:0/3498702145 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:41.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca37fe640 1 --2- 172.21.15.83:0/3498702145 >> v2:172.21.15.167:3300/0 conn(0x7f6ca415bc10 0x7f6ca40d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:41.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca27fc640 1 -- 172.21.15.83:0/3498702145 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1816405319 0 0) 0x7f6ca40ced00 con 0x7f6ca406e500 2024-04-17T10:38:41.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca27fc640 1 -- 172.21.15.83:0/3498702145 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c90003530 con 0x7f6ca406e500 2024-04-17T10:38:41.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca27fc640 1 -- 172.21.15.83:0/3498702145 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 904779511 0 0) 0x7f6c90003530 con 0x7f6ca406e500 2024-04-17T10:38:41.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca27fc640 1 -- 172.21.15.83:0/3498702145 >> v2:172.21.15.167:3300/0 conn(0x7f6ca415bc10 msgr2=0x7f6ca40d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:41.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca27fc640 1 --2- 172.21.15.83:0/3498702145 >> v2:172.21.15.167:3300/0 conn(0x7f6ca415bc10 0x7f6ca40d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:41.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca27fc640 1 -- 172.21.15.83:0/3498702145 >> v1:172.21.15.167:6789/0 conn(0x7f6ca40d0490 legacy=0x7f6ca40cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:41.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca27fc640 1 -- 172.21.15.83:0/3498702145 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca40cc970 con 0x7f6ca406e500 2024-04-17T10:38:41.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca27fc640 1 -- 172.21.15.83:0/3498702145 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f6c9c001680 con 0x7f6ca406e500 2024-04-17T10:38:41.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca27fc640 1 -- 172.21.15.83:0/3498702145 >> v1:172.21.15.83:6789/0 conn(0x7f6ca406e500 legacy=0x7f6ca40cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:41.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca27fc640 1 --2- 172.21.15.83:0/3498702145 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c90003a30 0x7f6c90003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:41.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca27fc640 1 --2- 172.21.15.83:0/3498702145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c90004810 0x7f6c90004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:41.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca27fc640 1 --2- 172.21.15.83:0/3498702145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c90005240 0x7f6c9000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:41.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca27fc640 1 -- 172.21.15.83:0/3498702145 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca40cc690 con 0x7f6c90004810 2024-04-17T10:38:41.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca27fc640 1 -- 172.21.15.83:0/3498702145 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9000c1a0 con 0x7f6c90005240 2024-04-17T10:38:41.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca27fc640 1 -- 172.21.15.83:0/3498702145 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9000c3a0 con 0x7f6c90003a30 2024-04-17T10:38:41.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca2ffd640 1 --2- 172.21.15.83:0/3498702145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c90004810 0x7f6c90004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:41.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca37fe640 1 --2- 172.21.15.83:0/3498702145 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c90003a30 0x7f6c90003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:41.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca2ffd640 1 -- 172.21.15.83:0/3498702145 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c90003a30 msgr2=0x7f6c90003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:41.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca2ffd640 1 --2- 172.21.15.83:0/3498702145 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c90003a30 0x7f6c90003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:41.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca3fff640 1 --2- 172.21.15.83:0/3498702145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c90005240 0x7f6c9000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:41.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca2ffd640 1 -- 172.21.15.83:0/3498702145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c90005240 msgr2=0x7f6c9000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:41.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca2ffd640 1 --2- 172.21.15.83:0/3498702145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c90005240 0x7f6c9000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:41.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca2ffd640 1 -- 172.21.15.83:0/3498702145 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c90003530 con 0x7f6c90004810 2024-04-17T10:38:41.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca37fe640 1 --2- 172.21.15.83:0/3498702145 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c90003a30 0x7f6c90003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:41.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca3fff640 1 --2- 172.21.15.83:0/3498702145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c90005240 0x7f6c9000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:41.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.292+0000 7f6ca3fff640 1 -- 172.21.15.83:0/3498702145 reap_dead start 2024-04-17T10:38:41.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.296+0000 7f6ca2ffd640 1 --2- 172.21.15.83:0/3498702145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c90004810 0x7f6c90004c90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c001e70 tx=0x7f6c8c01c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=812d3eacfe28f50e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:41.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.296+0000 7f6ca27fc640 1 -- 172.21.15.83:0/3498702145 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c0b0070 con 0x7f6c90004810 2024-04-17T10:38:41.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.296+0000 7f6ca27fc640 1 -- 172.21.15.83:0/3498702145 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c8c0a7070 con 0x7f6c90004810 2024-04-17T10:38:41.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.296+0000 7f6ca27fc640 1 -- 172.21.15.83:0/3498702145 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c001a90 con 0x7f6c90004810 2024-04-17T10:38:41.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.296+0000 7f6ca9f09640 1 -- 172.21.15.83:0/3498702145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c90004810 msgr2=0x7f6c90004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:41.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.296+0000 7f6ca9f09640 1 --2- 172.21.15.83:0/3498702145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c90004810 0x7f6c90004c90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c001e70 tx=0x7f6c8c01c760 comp rx=0 tx=0).stop 2024-04-17T10:38:41.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.296+0000 7f6ca9f09640 1 -- 172.21.15.83:0/3498702145 shutdown_connections 2024-04-17T10:38:41.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.296+0000 7f6ca9f09640 1 --2- 172.21.15.83:0/3498702145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c90004810 0x7f6c90004c90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:41.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.296+0000 7f6ca9f09640 1 -- 172.21.15.83:0/3498702145 >> 172.21.15.83:0/3498702145 conn(0x7f6ca40d17e0 msgr2=0x7f6ca40d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:41.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.296+0000 7f6ca9f09640 1 -- 172.21.15.83:0/3498702145 shutdown_connections 2024-04-17T10:38:41.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.296+0000 7f6ca9f09640 1 -- 172.21.15.83:0/3498702145 wait complete. 2024-04-17T10:38:41.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.296+0000 7f6ca9f09640 1 Processor -- start 2024-04-17T10:38:41.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.296+0000 7f6ca9f09640 1 -- start start 2024-04-17T10:38:41.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.300+0000 7f6ca9f09640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c90004810 0x7f6ca40c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:41.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.300+0000 7f6ca9f09640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ca415b020 0x7f6ca41f89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:41.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.300+0000 7f6ca9f09640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca41f5c70 0x7f6ca40c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:41.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.300+0000 7f6ca9f09640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca40cf860 con 0x7f6ca41f5c70 2024-04-17T10:38:41.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.300+0000 7f6ca9f09640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca40cd6a0 con 0x7f6c90004810 2024-04-17T10:38:41.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.300+0000 7f6ca9f09640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca40cf9e0 con 0x7f6ca415b020 2024-04-17T10:38:41.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.300+0000 7f6ca3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca41f5c70 0x7f6ca40c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:41.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.300+0000 7f6ca3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca41f5c70 0x7f6ca40c9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52624/0 (socket says 172.21.15.83:52624) 2024-04-17T10:38:41.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.300+0000 7f6ca3fff640 1 -- 172.21.15.83:0/2058801924 learned_addr learned my addr 172.21.15.83:0/2058801924 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:41.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.300+0000 7f6ca37fe640 1 --2- 172.21.15.83:0/2058801924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c90004810 0x7f6ca40c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:41.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.300+0000 7f6ca2ffd640 1 --2- 172.21.15.83:0/2058801924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ca415b020 0x7f6ca41f89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:41.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.300+0000 7f6ca3fff640 1 -- 172.21.15.83:0/2058801924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ca415b020 msgr2=0x7f6ca41f89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:41.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.300+0000 7f6ca3fff640 1 --2- 172.21.15.83:0/2058801924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ca415b020 0x7f6ca41f89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:41.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.300+0000 7f6ca3fff640 1 -- 172.21.15.83:0/2058801924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c90004810 msgr2=0x7f6ca40c62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:41.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.300+0000 7f6ca3fff640 1 --2- 172.21.15.83:0/2058801924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c90004810 0x7f6ca40c62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:41.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.300+0000 7f6ca3fff640 1 -- 172.21.15.83:0/2058801924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca41f9220 con 0x7f6ca41f5c70 2024-04-17T10:38:41.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.300+0000 7f6ca3fff640 1 --2- 172.21.15.83:0/2058801924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca41f5c70 0x7f6ca40c9e80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40cf2c0 tx=0x7f6c9c00d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:41.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.300+0000 7f6c9bfff640 1 -- 172.21.15.83:0/2058801924 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c0074e0 con 0x7f6ca41f5c70 2024-04-17T10:38:41.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.300+0000 7f6c9bfff640 1 -- 172.21.15.83:0/2058801924 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9c0047f0 con 0x7f6ca41f5c70 2024-04-17T10:38:41.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.300+0000 7f6ca9f09640 1 -- 172.21.15.83:0/2058801924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca41f9500 con 0x7f6ca41f5c70 2024-04-17T10:38:41.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.300+0000 7f6c9bfff640 1 -- 172.21.15.83:0/2058801924 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c007680 con 0x7f6ca41f5c70 2024-04-17T10:38:41.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.300+0000 7f6ca9f09640 1 -- 172.21.15.83:0/2058801924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca41f5750 con 0x7f6ca41f5c70 2024-04-17T10:38:41.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.300+0000 7f6ca9f09640 1 -- 172.21.15.83:0/2058801924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca41f5950 con 0x7f6ca41f5c70 2024-04-17T10:38:41.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.300+0000 7f6c9bfff640 1 -- 172.21.15.83:0/2058801924 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6c9c01b020 con 0x7f6ca41f5c70 2024-04-17T10:38:41.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.300+0000 7f6c9bfff640 1 --2- 172.21.15.83:0/2058801924 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6c84066f90 0x7f6c84069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:41.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.300+0000 7f6c9bfff640 1 -- 172.21.15.83:0/2058801924 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c9c084a00 con 0x7f6ca41f5c70 2024-04-17T10:38:41.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.300+0000 7f6ca37fe640 1 --2- 172.21.15.83:0/2058801924 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6c84066f90 0x7f6c84069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:41.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.300+0000 7f6ca37fe640 1 --2- 172.21.15.83:0/2058801924 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6c84066f90 0x7f6c84069450 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6ca406a390 tx=0x7f6c94008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:41.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.304+0000 7f6c9bfff640 1 -- 172.21.15.83:0/2058801924 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6ca41f5950 con 0x7f6ca41f5c70 2024-04-17T10:38:41.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.468+0000 7f6ca9f09640 1 -- 172.21.15.83:0/2058801924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6ca40cb970 con 0x7f6ca41f5c70 2024-04-17T10:38:41.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.468+0000 7f6c9bfff640 1 -- 172.21.15.83:0/2058801924 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6c9c00f2f0 con 0x7f6ca41f5c70 2024-04-17T10:38:41.463 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:38:41.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.468+0000 7f6ca9f09640 1 -- 172.21.15.83:0/2058801924 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6c84066f90 msgr2=0x7f6c84069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:41.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.468+0000 7f6ca9f09640 1 --2- 172.21.15.83:0/2058801924 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6c84066f90 0x7f6c84069450 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6ca406a390 tx=0x7f6c94008040 comp rx=0 tx=0).stop 2024-04-17T10:38:41.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.468+0000 7f6ca9f09640 1 -- 172.21.15.83:0/2058801924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca41f5c70 msgr2=0x7f6ca40c9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:41.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.468+0000 7f6ca9f09640 1 --2- 172.21.15.83:0/2058801924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca41f5c70 0x7f6ca40c9e80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40cf2c0 tx=0x7f6c9c00d9d0 comp rx=0 tx=0).stop 2024-04-17T10:38:41.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.468+0000 7f6ca9f09640 1 -- 172.21.15.83:0/2058801924 shutdown_connections 2024-04-17T10:38:41.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.468+0000 7f6ca9f09640 1 --2- 172.21.15.83:0/2058801924 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6c84066f90 0x7f6c84069450 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:41.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.468+0000 7f6ca9f09640 1 --2- 172.21.15.83:0/2058801924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca41f5c70 0x7f6ca40c9e80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:41.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.468+0000 7f6ca9f09640 1 --2- 172.21.15.83:0/2058801924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ca415b020 0x7f6ca41f89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:41.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.468+0000 7f6ca9f09640 1 --2- 172.21.15.83:0/2058801924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c90004810 0x7f6ca40c62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:41.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.468+0000 7f6ca9f09640 1 -- 172.21.15.83:0/2058801924 >> 172.21.15.83:0/2058801924 conn(0x7f6ca40d17e0 msgr2=0x7f6ca40b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:41.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.468+0000 7f6ca9f09640 1 -- 172.21.15.83:0/2058801924 shutdown_connections 2024-04-17T10:38:41.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.468+0000 7f6ca9f09640 1 -- 172.21.15.83:0/2058801924 wait complete. 2024-04-17T10:38:41.482 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:38:41.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.600+0000 7fc19ccb6640 1 Processor -- start 2024-04-17T10:38:41.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.600+0000 7fc19ccb6640 1 -- start start 2024-04-17T10:38:41.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.604+0000 7fc19ccb6640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc19806cc80 0x7fc198070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:41.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.604+0000 7fc19ccb6640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1980af160 con 0x7fc198154670 2024-04-17T10:38:41.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.604+0000 7fc19ccb6640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1980b2ff0 con 0x7fc1980b4090 2024-04-17T10:38:41.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.604+0000 7fc19ccb6640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc198154a50 con 0x7fc19806cc80 2024-04-17T10:38:41.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.604+0000 7fc19659c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc198154670 0x7fc1980aea50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47260/0 (socket says 172.21.15.83:47260) 2024-04-17T10:38:41.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.604+0000 7fc19659c640 1 -- 172.21.15.83:0/3057201537 learned_addr learned my addr 172.21.15.83:0/3057201537 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:41.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.604+0000 7fc196d9d640 1 --2- 172.21.15.83:0/3057201537 >> v2:172.21.15.83:3300/0 conn(0x7fc19806cc80 0x7fc198070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:41.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.604+0000 7fc19559a640 1 -- 172.21.15.83:0/3057201537 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2979189717 0 0) 0x7fc1980af160 con 0x7fc198154670 2024-04-17T10:38:41.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.604+0000 7fc19559a640 1 -- 172.21.15.83:0/3057201537 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc180003530 con 0x7fc198154670 2024-04-17T10:38:41.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.604+0000 7fc196d9d640 1 -- 172.21.15.83:0/3057201537 >> v1:172.21.15.167:6789/0 conn(0x7fc1980b4090 legacy=0x7fc19806c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:41.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.604+0000 7fc196d9d640 1 -- 172.21.15.83:0/3057201537 >> v1:172.21.15.83:6789/0 conn(0x7fc198154670 legacy=0x7fc1980aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:41.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.604+0000 7fc196d9d640 1 -- 172.21.15.83:0/3057201537 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1980b32d0 con 0x7fc19806cc80 2024-04-17T10:38:41.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.604+0000 7fc196d9d640 1 --2- 172.21.15.83:0/3057201537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc19806cc80 0x7fc198070470 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc19000aea0 tx=0x7fc19001d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ea1b35779bfae9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:41.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.604+0000 7fc19559a640 1 -- 172.21.15.83:0/3057201537 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1900a6830 con 0x7fc19806cc80 2024-04-17T10:38:41.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.604+0000 7fc19559a640 1 -- 172.21.15.83:0/3057201537 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1900a69d0 con 0x7fc19806cc80 2024-04-17T10:38:41.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.604+0000 7fc19559a640 1 -- 172.21.15.83:0/3057201537 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1900a6da0 con 0x7fc19806cc80 2024-04-17T10:38:41.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc19ccb6640 1 -- 172.21.15.83:0/3057201537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc19806cc80 msgr2=0x7fc198070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:41.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc19ccb6640 1 --2- 172.21.15.83:0/3057201537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc19806cc80 0x7fc198070470 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc19000aea0 tx=0x7fc19001d970 comp rx=0 tx=0).stop 2024-04-17T10:38:41.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc19ccb6640 1 -- 172.21.15.83:0/3057201537 shutdown_connections 2024-04-17T10:38:41.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc19ccb6640 1 --2- 172.21.15.83:0/3057201537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc19806cc80 0x7fc198070470 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:41.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc19ccb6640 1 -- 172.21.15.83:0/3057201537 >> 172.21.15.83:0/3057201537 conn(0x7fc1980b15b0 msgr2=0x7fc1980b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:41.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc19ccb6640 1 -- 172.21.15.83:0/3057201537 shutdown_connections 2024-04-17T10:38:41.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc19ccb6640 1 -- 172.21.15.83:0/3057201537 wait complete. 2024-04-17T10:38:41.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc19ccb6640 1 Processor -- start 2024-04-17T10:38:41.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc19ccb6640 1 -- start start 2024-04-17T10:38:41.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc19ccb6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc19806cc80 0x7fc198143890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:41.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc19ccb6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1980b4090 0x7fc198149870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:41.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc19ccb6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc198154670 0x7fc198144450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:41.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc19ccb6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19815b3c0 con 0x7fc19806cc80 2024-04-17T10:38:41.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc19ccb6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc198070dc0 con 0x7fc1980b4090 2024-04-17T10:38:41.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc19ccb6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc198167eb0 con 0x7fc198154670 2024-04-17T10:38:41.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc19659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc19806cc80 0x7fc198143890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:41.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc19659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc19806cc80 0x7fc198143890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52634/0 (socket says 172.21.15.83:52634) 2024-04-17T10:38:41.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc19659c640 1 -- 172.21.15.83:0/2867305604 learned_addr learned my addr 172.21.15.83:0/2867305604 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:41.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc196d9d640 1 --2- 172.21.15.83:0/2867305604 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc198154670 0x7fc198144450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:41.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc195d9b640 1 --2- 172.21.15.83:0/2867305604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1980b4090 0x7fc198149870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:41.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc19659c640 1 -- 172.21.15.83:0/2867305604 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc198154670 msgr2=0x7fc198144450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:41.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc19659c640 1 --2- 172.21.15.83:0/2867305604 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc198154670 0x7fc198144450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:41.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc19659c640 1 -- 172.21.15.83:0/2867305604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1980b4090 msgr2=0x7fc198149870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:41.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc19659c640 1 --2- 172.21.15.83:0/2867305604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1980b4090 0x7fc198149870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:41.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc19659c640 1 -- 172.21.15.83:0/2867305604 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc198144990 con 0x7fc19806cc80 2024-04-17T10:38:41.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc196d9d640 1 --2- 172.21.15.83:0/2867305604 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc198154670 0x7fc198144450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:41.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc19659c640 1 --2- 172.21.15.83:0/2867305604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc19806cc80 0x7fc198143890 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc18800c3c0 tx=0x7fc18800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:41.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc186ffd640 1 -- 172.21.15.83:0/2867305604 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc188017070 con 0x7fc19806cc80 2024-04-17T10:38:41.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc186ffd640 1 -- 172.21.15.83:0/2867305604 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc188005cf0 con 0x7fc19806cc80 2024-04-17T10:38:41.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc19ccb6640 1 -- 172.21.15.83:0/2867305604 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc198142ad0 con 0x7fc19806cc80 2024-04-17T10:38:41.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc186ffd640 1 -- 172.21.15.83:0/2867305604 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1880072f0 con 0x7fc19806cc80 2024-04-17T10:38:41.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc19ccb6640 1 -- 172.21.15.83:0/2867305604 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc198143090 con 0x7fc19806cc80 2024-04-17T10:38:41.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.608+0000 7fc19ccb6640 1 -- 172.21.15.83:0/2867305604 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc198074f90 con 0x7fc19806cc80 2024-04-17T10:38:41.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.612+0000 7fc186ffd640 1 -- 172.21.15.83:0/2867305604 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc18801a070 con 0x7fc19806cc80 2024-04-17T10:38:41.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.612+0000 7fc186ffd640 1 --2- 172.21.15.83:0/2867305604 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc178066f60 0x7fc178069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:41.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.612+0000 7fc186ffd640 1 -- 172.21.15.83:0/2867305604 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc188084440 con 0x7fc19806cc80 2024-04-17T10:38:41.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.612+0000 7fc195d9b640 1 --2- 172.21.15.83:0/2867305604 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc178066f60 0x7fc178069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:41.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.612+0000 7fc195d9b640 1 --2- 172.21.15.83:0/2867305604 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc178066f60 0x7fc178069420 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc19815d220 tx=0x7fc18c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:41.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.616+0000 7fc186ffd640 1 -- 172.21.15.83:0/2867305604 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc188055440 con 0x7fc19806cc80 2024-04-17T10:38:41.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.760+0000 7fc19ccb6640 1 -- 172.21.15.83:0/2867305604 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc1980ced20 con 0x7fc19806cc80 2024-04-17T10:38:41.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.760+0000 7fc186ffd640 1 -- 172.21.15.83:0/2867305604 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc188054d40 con 0x7fc19806cc80 2024-04-17T10:38:41.756 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:38:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.760+0000 7fc19ccb6640 1 -- 172.21.15.83:0/2867305604 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc178066f60 msgr2=0x7fc178069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.760+0000 7fc19ccb6640 1 --2- 172.21.15.83:0/2867305604 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc178066f60 0x7fc178069420 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc19815d220 tx=0x7fc18c009040 comp rx=0 tx=0).stop 2024-04-17T10:38:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.760+0000 7fc19ccb6640 1 -- 172.21.15.83:0/2867305604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc19806cc80 msgr2=0x7fc198143890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.760+0000 7fc19ccb6640 1 --2- 172.21.15.83:0/2867305604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc19806cc80 0x7fc198143890 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc18800c3c0 tx=0x7fc18800c880 comp rx=0 tx=0).stop 2024-04-17T10:38:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.764+0000 7fc19ccb6640 1 -- 172.21.15.83:0/2867305604 shutdown_connections 2024-04-17T10:38:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.764+0000 7fc19ccb6640 1 --2- 172.21.15.83:0/2867305604 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc178066f60 0x7fc178069420 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.764+0000 7fc19ccb6640 1 --2- 172.21.15.83:0/2867305604 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc198154670 0x7fc198144450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.764+0000 7fc19ccb6640 1 --2- 172.21.15.83:0/2867305604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1980b4090 0x7fc198149870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.764+0000 7fc19ccb6640 1 --2- 172.21.15.83:0/2867305604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc19806cc80 0x7fc198143890 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.764+0000 7fc19ccb6640 1 -- 172.21.15.83:0/2867305604 >> 172.21.15.83:0/2867305604 conn(0x7fc1980b15b0 msgr2=0x7fc19806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.764+0000 7fc19ccb6640 1 -- 172.21.15.83:0/2867305604 shutdown_connections 2024-04-17T10:38:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.764+0000 7fc19ccb6640 1 -- 172.21.15.83:0/2867305604 wait complete. 2024-04-17T10:38:41.774 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:38:41.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.888+0000 7fd21aeac640 1 Processor -- start 2024-04-17T10:38:41.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.888+0000 7fd21aeac640 1 -- start start 2024-04-17T10:38:41.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.888+0000 7fd21aeac640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd21415da50 0x7fd2140ae950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:41.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.888+0000 7fd21aeac640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2140afe20 0x7fd21406abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:41.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.888+0000 7fd21aeac640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2140af9e0 con 0x7fd21406b120 2024-04-17T10:38:41.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.888+0000 7fd21aeac640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2140af3a0 con 0x7fd21415da50 2024-04-17T10:38:41.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.888+0000 7fd21aeac640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2140aee90 con 0x7fd2140afe20 2024-04-17T10:38:41.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.888+0000 7fd20bfff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2140afe20 0x7fd21406abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:41.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.888+0000 7fd218c48640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd21415da50 0x7fd2140ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:41.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.888+0000 7fd20bfff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2140afe20 0x7fd21406abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:48802/0 (socket says 172.21.15.83:48802) 2024-04-17T10:38:41.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.888+0000 7fd20bfff640 1 -- 172.21.15.83:0/348457784 learned_addr learned my addr 172.21.15.83:0/348457784 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:41.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.888+0000 7fd218c48640 1 -- 172.21.15.83:0/348457784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2140afe20 msgr2=0x7fd21406abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:41.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.888+0000 7fd218c48640 1 --2- 172.21.15.83:0/348457784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2140afe20 0x7fd21406abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:41.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.888+0000 7fd218c48640 1 -- 172.21.15.83:0/348457784 >> v1:172.21.15.167:6789/0 conn(0x7fd21406b120 legacy=0x7fd214071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:41.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.888+0000 7fd218c48640 1 -- 172.21.15.83:0/348457784 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2140b4e80 con 0x7fd21415da50 2024-04-17T10:38:41.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.892+0000 7fd20bfff640 1 --2- 172.21.15.83:0/348457784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2140afe20 0x7fd21406abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:38:41.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.892+0000 7fd218c48640 1 --2- 172.21.15.83:0/348457784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd21415da50 0x7fd2140ae950 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd2100097c0 tx=0x7fd21001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbe69d30f31ce4ce server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:41.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.892+0000 7fd20b7fe640 1 -- 172.21.15.83:0/348457784 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2100b0070 con 0x7fd21415da50 2024-04-17T10:38:41.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.892+0000 7fd20b7fe640 1 -- 172.21.15.83:0/348457784 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2100a4e50 con 0x7fd21415da50 2024-04-17T10:38:41.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.892+0000 7fd20b7fe640 1 -- 172.21.15.83:0/348457784 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2100ab820 con 0x7fd21415da50 2024-04-17T10:38:41.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.892+0000 7fd21aeac640 1 -- 172.21.15.83:0/348457784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd21415da50 msgr2=0x7fd2140ae950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:41.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.892+0000 7fd21aeac640 1 --2- 172.21.15.83:0/348457784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd21415da50 0x7fd2140ae950 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd2100097c0 tx=0x7fd21001c4a0 comp rx=0 tx=0).stop 2024-04-17T10:38:41.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.892+0000 7fd21aeac640 1 -- 172.21.15.83:0/348457784 shutdown_connections 2024-04-17T10:38:41.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.892+0000 7fd21aeac640 1 --2- 172.21.15.83:0/348457784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2140afe20 0x7fd21406abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:41.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.892+0000 7fd21aeac640 1 --2- 172.21.15.83:0/348457784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd21415da50 0x7fd2140ae950 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:41.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.892+0000 7fd21aeac640 1 -- 172.21.15.83:0/348457784 >> 172.21.15.83:0/348457784 conn(0x7fd2140b24f0 msgr2=0x7fd2140b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:41.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.892+0000 7fd21aeac640 1 -- 172.21.15.83:0/348457784 shutdown_connections 2024-04-17T10:38:41.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.892+0000 7fd21aeac640 1 -- 172.21.15.83:0/348457784 wait complete. 2024-04-17T10:38:41.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.896+0000 7fd21aeac640 1 Processor -- start 2024-04-17T10:38:41.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.896+0000 7fd21aeac640 1 -- start start 2024-04-17T10:38:41.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.896+0000 7fd21aeac640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd21406b120 0x7fd21415fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:41.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.896+0000 7fd21aeac640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2140afe20 0x7fd21414f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:41.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.896+0000 7fd21aeac640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd21415da50 0x7fd2141591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:41.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.896+0000 7fd21aeac640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21406ebb0 con 0x7fd2140afe20 2024-04-17T10:38:41.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.896+0000 7fd21aeac640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2140b4680 con 0x7fd21406b120 2024-04-17T10:38:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.896+0000 7fd21aeac640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd21406ed30 con 0x7fd21415da50 2024-04-17T10:38:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.896+0000 7fd20bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2140afe20 0x7fd21414f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.896+0000 7fd20bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2140afe20 0x7fd21414f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52640/0 (socket says 172.21.15.83:52640) 2024-04-17T10:38:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.896+0000 7fd20bfff640 1 -- 172.21.15.83:0/3472599960 learned_addr learned my addr 172.21.15.83:0/3472599960 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.896+0000 7fd219449640 1 --2- 172.21.15.83:0/3472599960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd21415da50 0x7fd2141591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.896+0000 7fd218c48640 1 --2- 172.21.15.83:0/3472599960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd21406b120 0x7fd21415fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.896+0000 7fd20bfff640 1 -- 172.21.15.83:0/3472599960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd21415da50 msgr2=0x7fd2141591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.896+0000 7fd20bfff640 1 --2- 172.21.15.83:0/3472599960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd21415da50 0x7fd2141591b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.896+0000 7fd20bfff640 1 -- 172.21.15.83:0/3472599960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd21406b120 msgr2=0x7fd21415fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.896+0000 7fd20bfff640 1 --2- 172.21.15.83:0/3472599960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd21406b120 0x7fd21415fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.896+0000 7fd20bfff640 1 -- 172.21.15.83:0/3472599960 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2141596f0 con 0x7fd2140afe20 2024-04-17T10:38:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.896+0000 7fd20bfff640 1 --2- 172.21.15.83:0/3472599960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2140afe20 0x7fd21414f270 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd20400e9d0 tx=0x7fd20400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:41.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.896+0000 7fd2097fa640 1 -- 172.21.15.83:0/3472599960 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204017070 con 0x7fd2140afe20 2024-04-17T10:38:41.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.896+0000 7fd2097fa640 1 -- 172.21.15.83:0/3472599960 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd20400ca60 con 0x7fd2140afe20 2024-04-17T10:38:41.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.896+0000 7fd21aeac640 1 -- 172.21.15.83:0/3472599960 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd214159990 con 0x7fd2140afe20 2024-04-17T10:38:41.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.896+0000 7fd2097fa640 1 -- 172.21.15.83:0/3472599960 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20400ce10 con 0x7fd2140afe20 2024-04-17T10:38:41.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.896+0000 7fd21aeac640 1 -- 172.21.15.83:0/3472599960 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd214159f50 con 0x7fd2140afe20 2024-04-17T10:38:41.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.896+0000 7fd21aeac640 1 -- 172.21.15.83:0/3472599960 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2140683d0 con 0x7fd2140afe20 2024-04-17T10:38:41.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.896+0000 7fd2097fa640 1 -- 172.21.15.83:0/3472599960 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd20401a070 con 0x7fd2140afe20 2024-04-17T10:38:41.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.896+0000 7fd2097fa640 1 --2- 172.21.15.83:0/3472599960 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd200066f60 0x7fd200069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:41.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.900+0000 7fd218c48640 1 --2- 172.21.15.83:0/3472599960 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd200066f60 0x7fd200069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:41.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.900+0000 7fd2097fa640 1 -- 172.21.15.83:0/3472599960 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd204084f20 con 0x7fd2140afe20 2024-04-17T10:38:41.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.900+0000 7fd218c48640 1 --2- 172.21.15.83:0/3472599960 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd200066f60 0x7fd200069420 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd2100ad000 tx=0x7fd2100a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:41.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:41.904+0000 7fd2097fa640 1 -- 172.21.15.83:0/3472599960 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd204055fc0 con 0x7fd2140afe20 2024-04-17T10:38:42.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.048+0000 7fd21aeac640 1 -- 172.21.15.83:0/3472599960 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd214071970 con 0x7fd2140afe20 2024-04-17T10:38:42.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.048+0000 7fd2097fa640 1 -- 172.21.15.83:0/3472599960 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd2040558c0 con 0x7fd2140afe20 2024-04-17T10:38:42.046 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:38:42.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.052+0000 7fd21aeac640 1 -- 172.21.15.83:0/3472599960 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd200066f60 msgr2=0x7fd200069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:42.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.052+0000 7fd21aeac640 1 --2- 172.21.15.83:0/3472599960 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd200066f60 0x7fd200069420 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd2100ad000 tx=0x7fd2100a6000 comp rx=0 tx=0).stop 2024-04-17T10:38:42.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.052+0000 7fd21aeac640 1 -- 172.21.15.83:0/3472599960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2140afe20 msgr2=0x7fd21414f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:42.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.052+0000 7fd21aeac640 1 --2- 172.21.15.83:0/3472599960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2140afe20 0x7fd21414f270 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd20400e9d0 tx=0x7fd20400ee90 comp rx=0 tx=0).stop 2024-04-17T10:38:42.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.052+0000 7fd21aeac640 1 -- 172.21.15.83:0/3472599960 shutdown_connections 2024-04-17T10:38:42.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.052+0000 7fd21aeac640 1 --2- 172.21.15.83:0/3472599960 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd200066f60 0x7fd200069420 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.052+0000 7fd21aeac640 1 --2- 172.21.15.83:0/3472599960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd21415da50 0x7fd2141591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.052+0000 7fd21aeac640 1 --2- 172.21.15.83:0/3472599960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2140afe20 0x7fd21414f270 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.052+0000 7fd21aeac640 1 --2- 172.21.15.83:0/3472599960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd21406b120 0x7fd21415fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.052+0000 7fd21aeac640 1 -- 172.21.15.83:0/3472599960 >> 172.21.15.83:0/3472599960 conn(0x7fd2140b24f0 msgr2=0x7fd2140b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:42.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.052+0000 7fd21aeac640 1 -- 172.21.15.83:0/3472599960 shutdown_connections 2024-04-17T10:38:42.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.052+0000 7fd21aeac640 1 -- 172.21.15.83:0/3472599960 wait complete. 2024-04-17T10:38:42.063 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:38:42.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.176+0000 7f2e31d1e640 1 Processor -- start 2024-04-17T10:38:42.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.176+0000 7f2e31d1e640 1 -- start start 2024-04-17T10:38:42.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e31d1e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2e2c072330 0x7f2e2c0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:42.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e31d1e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e2c0cfa10 0x7f2e2c06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:42.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e31d1e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e2c0cfea0 con 0x7f2e2c15ba30 2024-04-17T10:38:42.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e31d1e640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e2c071ea0 con 0x7f2e2c072330 2024-04-17T10:38:42.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e31d1e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c06e320 con 0x7f2e2c0cfa10 2024-04-17T10:38:42.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e2b7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2e2c15ba30 0x7f2e2c15c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47268/0 (socket says 172.21.15.83:47268) 2024-04-17T10:38:42.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e2b7fe640 1 -- 172.21.15.83:0/1726464379 learned_addr learned my addr 172.21.15.83:0/1726464379 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:42.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e2bfff640 1 --2- 172.21.15.83:0/1726464379 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e2c0cfa10 0x7f2e2c06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:42.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e2affd640 1 --2- 172.21.15.83:0/1726464379 >> v2:172.21.15.167:3300/0 conn(0x7f2e2c072330 0x7f2e2c0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:42.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e2a7fc640 1 -- 172.21.15.83:0/1726464379 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4032894995 0 0) 0x7f2e2c0cfea0 con 0x7f2e2c15ba30 2024-04-17T10:38:42.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e2a7fc640 1 -- 172.21.15.83:0/1726464379 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e14003530 con 0x7f2e2c15ba30 2024-04-17T10:38:42.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e2a7fc640 1 -- 172.21.15.83:0/1726464379 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 622013092 0 0) 0x7f2e14003530 con 0x7f2e2c15ba30 2024-04-17T10:38:42.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e2a7fc640 1 -- 172.21.15.83:0/1726464379 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e2c0cfa10 msgr2=0x7f2e2c06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:42.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e2a7fc640 1 --2- 172.21.15.83:0/1726464379 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e2c0cfa10 0x7f2e2c06c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e2a7fc640 1 -- 172.21.15.83:0/1726464379 >> v2:172.21.15.167:3300/0 conn(0x7f2e2c072330 msgr2=0x7f2e2c0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:42.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e2a7fc640 1 --2- 172.21.15.83:0/1726464379 >> v2:172.21.15.167:3300/0 conn(0x7f2e2c072330 0x7f2e2c0cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e2a7fc640 1 -- 172.21.15.83:0/1726464379 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2c06e040 con 0x7f2e2c15ba30 2024-04-17T10:38:42.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e2a7fc640 1 -- 172.21.15.83:0/1726464379 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f2e1c001680 con 0x7f2e2c15ba30 2024-04-17T10:38:42.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e2affd640 1 --2- 172.21.15.83:0/1726464379 >> v2:172.21.15.167:3300/0 conn(0x7f2e2c072330 0x7f2e2c0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:38:42.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e2a7fc640 1 -- 172.21.15.83:0/1726464379 >> v1:172.21.15.83:6789/0 conn(0x7f2e2c15ba30 legacy=0x7f2e2c15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:42.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e2a7fc640 1 --2- 172.21.15.83:0/1726464379 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e2c0cfa10 0x7f2e140039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:42.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e2a7fc640 1 --2- 172.21.15.83:0/1726464379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e14004060 0x7f2e14004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:42.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e2a7fc640 1 --2- 172.21.15.83:0/1726464379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e14004d50 0x7f2e140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:42.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e2a7fc640 1 -- 172.21.15.83:0/1726464379 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c071ea0 con 0x7f2e14004060 2024-04-17T10:38:42.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e2a7fc640 1 -- 172.21.15.83:0/1726464379 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c06e320 con 0x7f2e14004d50 2024-04-17T10:38:42.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e2a7fc640 1 -- 172.21.15.83:0/1726464379 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1400bc10 con 0x7f2e2c0cfa10 2024-04-17T10:38:42.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e2affd640 1 --2- 172.21.15.83:0/1726464379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e14004060 0x7f2e14004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:42.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e2b7fe640 1 --2- 172.21.15.83:0/1726464379 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e2c0cfa10 0x7f2e140039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:42.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e2bfff640 1 --2- 172.21.15.83:0/1726464379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e14004d50 0x7f2e140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:42.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e2affd640 1 -- 172.21.15.83:0/1726464379 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e2c0cfa10 msgr2=0x7f2e140039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:42.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e2affd640 1 --2- 172.21.15.83:0/1726464379 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e2c0cfa10 0x7f2e140039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e2affd640 1 -- 172.21.15.83:0/1726464379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e14004d50 msgr2=0x7f2e140051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:42.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e2affd640 1 --2- 172.21.15.83:0/1726464379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e14004d50 0x7f2e140051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e2affd640 1 -- 172.21.15.83:0/1726464379 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e14003530 con 0x7f2e14004060 2024-04-17T10:38:42.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.180+0000 7f2e2affd640 1 --2- 172.21.15.83:0/1726464379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e14004060 0x7f2e14004810 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2e200098d0 tx=0x7f2e2001c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd2fa63d51f5637 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:42.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e2a7fc640 1 -- 172.21.15.83:0/1726464379 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2000f040 con 0x7f2e14004060 2024-04-17T10:38:42.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e2a7fc640 1 -- 172.21.15.83:0/1726464379 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e200a9650 con 0x7f2e14004060 2024-04-17T10:38:42.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e2a7fc640 1 -- 172.21.15.83:0/1726464379 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e200a9a00 con 0x7f2e14004060 2024-04-17T10:38:42.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e31d1e640 1 -- 172.21.15.83:0/1726464379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e14004060 msgr2=0x7f2e14004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:42.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e31d1e640 1 --2- 172.21.15.83:0/1726464379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e14004060 0x7f2e14004810 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2e200098d0 tx=0x7f2e2001c650 comp rx=0 tx=0).stop 2024-04-17T10:38:42.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e2bfff640 1 -- 172.21.15.83:0/1726464379 reap_dead start 2024-04-17T10:38:42.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e31d1e640 1 -- 172.21.15.83:0/1726464379 shutdown_connections 2024-04-17T10:38:42.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e31d1e640 1 -- 172.21.15.83:0/1726464379 >> 172.21.15.83:0/1726464379 conn(0x7f2e2c06abe0 msgr2=0x7f2e2c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:42.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e31d1e640 1 -- 172.21.15.83:0/1726464379 shutdown_connections 2024-04-17T10:38:42.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e31d1e640 1 -- 172.21.15.83:0/1726464379 wait complete. 2024-04-17T10:38:42.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e31d1e640 1 Processor -- start 2024-04-17T10:38:42.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e31d1e640 1 -- start start 2024-04-17T10:38:42.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e31d1e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e14004060 0x7f2e2c156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:42.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e31d1e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e2c143e50 0x7f2e2c1442d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:42.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e31d1e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e2c15e170 0x7f2e2c15e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:42.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e31d1e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c06de20 con 0x7f2e2c15e170 2024-04-17T10:38:42.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e31d1e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c06dca0 con 0x7f2e2c143e50 2024-04-17T10:38:42.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e31d1e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c06cca0 con 0x7f2e14004060 2024-04-17T10:38:42.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e2bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e2c15e170 0x7f2e2c15e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:42.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e2bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e2c15e170 0x7f2e2c15e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52662/0 (socket says 172.21.15.83:52662) 2024-04-17T10:38:42.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e2bfff640 1 -- 172.21.15.83:0/1233764473 learned_addr learned my addr 172.21.15.83:0/1233764473 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:42.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e2b7fe640 1 --2- 172.21.15.83:0/1233764473 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e14004060 0x7f2e2c156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:42.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e2affd640 1 --2- 172.21.15.83:0/1233764473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e2c143e50 0x7f2e2c1442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:42.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e2bfff640 1 -- 172.21.15.83:0/1233764473 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e14004060 msgr2=0x7f2e2c156410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:42.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e2bfff640 1 --2- 172.21.15.83:0/1233764473 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e14004060 0x7f2e2c156410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e2bfff640 1 -- 172.21.15.83:0/1233764473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e2c143e50 msgr2=0x7f2e2c1442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:42.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e2bfff640 1 --2- 172.21.15.83:0/1233764473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e2c143e50 0x7f2e2c1442d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e2bfff640 1 -- 172.21.15.83:0/1233764473 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2c0aed90 con 0x7f2e2c15e170 2024-04-17T10:38:42.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e2bfff640 1 --2- 172.21.15.83:0/1233764473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e2c15e170 0x7f2e2c15e620 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c072c70 tx=0x7f2e24009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:42.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e13fff640 1 -- 172.21.15.83:0/1233764473 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e24007c70 con 0x7f2e2c15e170 2024-04-17T10:38:42.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e13fff640 1 -- 172.21.15.83:0/1233764473 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e24019070 con 0x7f2e2c15e170 2024-04-17T10:38:42.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e13fff640 1 -- 172.21.15.83:0/1233764473 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e24014650 con 0x7f2e2c15e170 2024-04-17T10:38:42.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e31d1e640 1 -- 172.21.15.83:0/1233764473 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e2c0af070 con 0x7f2e2c15e170 2024-04-17T10:38:42.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.184+0000 7f2e31d1e640 1 -- 172.21.15.83:0/1233764473 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e2c0af630 con 0x7f2e2c15e170 2024-04-17T10:38:42.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.188+0000 7f2e31d1e640 1 -- 172.21.15.83:0/1233764473 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e2c074f90 con 0x7f2e2c15e170 2024-04-17T10:38:42.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.188+0000 7f2e13fff640 1 -- 172.21.15.83:0/1233764473 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2e240147f0 con 0x7f2e2c15e170 2024-04-17T10:38:42.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.188+0000 7f2e13fff640 1 --2- 172.21.15.83:0/1233764473 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2e18066f60 0x7f2e18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:42.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.188+0000 7f2e13fff640 1 -- 172.21.15.83:0/1233764473 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e24085450 con 0x7f2e2c15e170 2024-04-17T10:38:42.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.188+0000 7f2e2b7fe640 1 --2- 172.21.15.83:0/1233764473 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2e18066f60 0x7f2e18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:42.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.188+0000 7f2e2b7fe640 1 --2- 172.21.15.83:0/1233764473 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2e18066f60 0x7f2e18069420 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c001ec0 tx=0x7f2e1c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:42.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.192+0000 7f2e13fff640 1 -- 172.21.15.83:0/1233764473 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2e24056460 con 0x7f2e2c15e170 2024-04-17T10:38:42.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.340+0000 7f2e31d1e640 1 -- 172.21.15.83:0/1233764473 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e2c154990 con 0x7f2e2c15e170 2024-04-17T10:38:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.344+0000 7f2e13fff640 1 -- 172.21.15.83:0/1233764473 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16533 (secure 0 0 0) 0x7f2e24055d60 con 0x7f2e2c15e170 2024-04-17T10:38:42.340 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:42.340 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":20,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:38:13.690876+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:38:10.464490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T10:38:11.043134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:38:42.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.344+0000 7f2e31d1e640 1 -- 172.21.15.83:0/1233764473 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2e18066f60 msgr2=0x7f2e18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:42.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.344+0000 7f2e31d1e640 1 --2- 172.21.15.83:0/1233764473 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2e18066f60 0x7f2e18069420 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c001ec0 tx=0x7f2e1c009040 comp rx=0 tx=0).stop 2024-04-17T10:38:42.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.344+0000 7f2e31d1e640 1 -- 172.21.15.83:0/1233764473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e2c15e170 msgr2=0x7f2e2c15e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:42.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.344+0000 7f2e31d1e640 1 --2- 172.21.15.83:0/1233764473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e2c15e170 0x7f2e2c15e620 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c072c70 tx=0x7f2e24009d40 comp rx=0 tx=0).stop 2024-04-17T10:38:42.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.344+0000 7f2e31d1e640 1 -- 172.21.15.83:0/1233764473 shutdown_connections 2024-04-17T10:38:42.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.344+0000 7f2e31d1e640 1 --2- 172.21.15.83:0/1233764473 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2e18066f60 0x7f2e18069420 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.344+0000 7f2e31d1e640 1 --2- 172.21.15.83:0/1233764473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e2c15e170 0x7f2e2c15e620 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.344+0000 7f2e31d1e640 1 --2- 172.21.15.83:0/1233764473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e2c143e50 0x7f2e2c1442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.344+0000 7f2e31d1e640 1 --2- 172.21.15.83:0/1233764473 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e14004060 0x7f2e2c156410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.344+0000 7f2e31d1e640 1 -- 172.21.15.83:0/1233764473 >> 172.21.15.83:0/1233764473 conn(0x7f2e2c06abe0 msgr2=0x7f2e2c0cd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:42.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.344+0000 7f2e31d1e640 1 -- 172.21.15.83:0/1233764473 shutdown_connections 2024-04-17T10:38:42.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.344+0000 7f2e31d1e640 1 -- 172.21.15.83:0/1233764473 wait complete. 2024-04-17T10:38:42.357 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:38:10.464490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:38:11.043134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:38:42.357 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:38:42.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.476+0000 7fd9b8d9a640 1 Processor -- start 2024-04-17T10:38:42.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.476+0000 7fd9b8d9a640 1 -- start start 2024-04-17T10:38:42.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.476+0000 7fd9b8d9a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9b415bc10 0x7fd9b40d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:42.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.476+0000 7fd9b8d9a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9b40ced00 con 0x7fd9b40d0490 2024-04-17T10:38:42.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.476+0000 7fd9b8d9a640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9b40cefe0 con 0x7fd9b406e500 2024-04-17T10:38:42.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.476+0000 7fd9b8d9a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b40cc690 con 0x7fd9b415bc10 2024-04-17T10:38:42.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.476+0000 7fd9b1d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd9b40d0490 0x7fd9b40cce90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47280/0 (socket says 172.21.15.83:47280) 2024-04-17T10:38:42.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.476+0000 7fd9b1d9b640 1 -- 172.21.15.83:0/3012959115 learned_addr learned my addr 172.21.15.83:0/3012959115 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:42.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b259c640 1 --2- 172.21.15.83:0/3012959115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9b415bc10 0x7fd9b40d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:42.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b159a640 1 -- 172.21.15.83:0/3012959115 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3244715658 0 0) 0x7fd9b40ced00 con 0x7fd9b40d0490 2024-04-17T10:38:42.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b159a640 1 -- 172.21.15.83:0/3012959115 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd99c003530 con 0x7fd9b40d0490 2024-04-17T10:38:42.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b159a640 1 -- 172.21.15.83:0/3012959115 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2394362449 0 0) 0x7fd99c003530 con 0x7fd9b40d0490 2024-04-17T10:38:42.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b159a640 1 -- 172.21.15.83:0/3012959115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9b415bc10 msgr2=0x7fd9b40d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:42.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b159a640 1 --2- 172.21.15.83:0/3012959115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9b415bc10 0x7fd9b40d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b159a640 1 -- 172.21.15.83:0/3012959115 >> v1:172.21.15.167:6789/0 conn(0x7fd9b406e500 legacy=0x7fd9b40cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:42.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b159a640 1 -- 172.21.15.83:0/3012959115 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b40cc970 con 0x7fd9b40d0490 2024-04-17T10:38:42.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b159a640 1 -- 172.21.15.83:0/3012959115 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fd9a8001680 con 0x7fd9b40d0490 2024-04-17T10:38:42.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b159a640 1 -- 172.21.15.83:0/3012959115 >> v1:172.21.15.83:6789/0 conn(0x7fd9b40d0490 legacy=0x7fd9b40cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:42.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b159a640 1 --2- 172.21.15.83:0/3012959115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd99c003a60 0x7fd99c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:42.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b159a640 1 --2- 172.21.15.83:0/3012959115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd99c004810 0x7fd99c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:42.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b159a640 1 --2- 172.21.15.83:0/3012959115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9b415bc10 0x7fd99c005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:42.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b159a640 1 -- 172.21.15.83:0/3012959115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b40cc690 con 0x7fd99c004810 2024-04-17T10:38:42.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b159a640 1 -- 172.21.15.83:0/3012959115 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c00bb90 con 0x7fd99c003a60 2024-04-17T10:38:42.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b159a640 1 -- 172.21.15.83:0/3012959115 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c00bd90 con 0x7fd9b415bc10 2024-04-17T10:38:42.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b1d9b640 1 --2- 172.21.15.83:0/3012959115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd99c004810 0x7fd99c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:42.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b259c640 1 --2- 172.21.15.83:0/3012959115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd99c003a60 0x7fd99c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:42.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b1d9b640 1 -- 172.21.15.83:0/3012959115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9b415bc10 msgr2=0x7fd99c005280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:42.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b1d9b640 1 --2- 172.21.15.83:0/3012959115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9b415bc10 0x7fd99c005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b1d9b640 1 -- 172.21.15.83:0/3012959115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd99c003a60 msgr2=0x7fd99c003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:42.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b1d9b640 1 --2- 172.21.15.83:0/3012959115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd99c003a60 0x7fd99c003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b1d9b640 1 -- 172.21.15.83:0/3012959115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99c003530 con 0x7fd99c004810 2024-04-17T10:38:42.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b1d9b640 1 --2- 172.21.15.83:0/3012959115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd99c004810 0x7fd99c004c90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd9b40d0890 tx=0x7fd9a801cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65aed17bd9f9fc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:42.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b159a640 1 -- 172.21.15.83:0/3012959115 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8007d00 con 0x7fd99c004810 2024-04-17T10:38:42.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b159a640 1 -- 172.21.15.83:0/3012959115 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a80a9020 con 0x7fd99c004810 2024-04-17T10:38:42.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b159a640 1 -- 172.21.15.83:0/3012959115 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8005ae0 con 0x7fd99c004810 2024-04-17T10:38:42.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b8d9a640 1 -- 172.21.15.83:0/3012959115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd99c004810 msgr2=0x7fd99c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:42.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b8d9a640 1 --2- 172.21.15.83:0/3012959115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd99c004810 0x7fd99c004c90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd9b40d0890 tx=0x7fd9a801cc00 comp rx=0 tx=0).stop 2024-04-17T10:38:42.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b2d9d640 1 -- 172.21.15.83:0/3012959115 reap_dead start 2024-04-17T10:38:42.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b8d9a640 1 -- 172.21.15.83:0/3012959115 shutdown_connections 2024-04-17T10:38:42.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b8d9a640 1 -- 172.21.15.83:0/3012959115 >> 172.21.15.83:0/3012959115 conn(0x7fd9b40d17e0 msgr2=0x7fd9b40d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:42.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b8d9a640 1 -- 172.21.15.83:0/3012959115 shutdown_connections 2024-04-17T10:38:42.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.480+0000 7fd9b8d9a640 1 -- 172.21.15.83:0/3012959115 wait complete. 2024-04-17T10:38:42.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.484+0000 7fd9b8d9a640 1 Processor -- start 2024-04-17T10:38:42.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.484+0000 7fd9b8d9a640 1 -- start start 2024-04-17T10:38:42.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.484+0000 7fd9b8d9a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd99c004810 0x7fd9b4152fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:42.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.484+0000 7fd9b8d9a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9b40b4de0 0x7fd9b40b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:42.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.484+0000 7fd9b8d9a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9b40b2bf0 0x7fd9b40c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:42.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.484+0000 7fd9b8d9a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b40d00f0 con 0x7fd99c004810 2024-04-17T10:38:42.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.484+0000 7fd9b8d9a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b40cff70 con 0x7fd9b40b4de0 2024-04-17T10:38:42.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.484+0000 7fd9b8d9a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b40d2300 con 0x7fd9b40b2bf0 2024-04-17T10:38:42.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.484+0000 7fd9b259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd99c004810 0x7fd9b4152fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:42.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.484+0000 7fd9b259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd99c004810 0x7fd9b4152fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52684/0 (socket says 172.21.15.83:52684) 2024-04-17T10:38:42.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.484+0000 7fd9b259c640 1 -- 172.21.15.83:0/2875108080 learned_addr learned my addr 172.21.15.83:0/2875108080 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:42.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.484+0000 7fd9b1d9b640 1 --2- 172.21.15.83:0/2875108080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9b40b4de0 0x7fd9b40b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:42.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.484+0000 7fd9b259c640 1 -- 172.21.15.83:0/2875108080 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9b40b2bf0 msgr2=0x7fd9b40c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:42.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.484+0000 7fd9b259c640 1 --2- 172.21.15.83:0/2875108080 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9b40b2bf0 0x7fd9b40c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.484+0000 7fd9b259c640 1 -- 172.21.15.83:0/2875108080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9b40b4de0 msgr2=0x7fd9b40b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:42.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.484+0000 7fd9b259c640 1 --2- 172.21.15.83:0/2875108080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9b40b4de0 0x7fd9b40b26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.484+0000 7fd9b259c640 1 -- 172.21.15.83:0/2875108080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b40b5f60 con 0x7fd99c004810 2024-04-17T10:38:42.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.484+0000 7fd9b1d9b640 1 --2- 172.21.15.83:0/2875108080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9b40b4de0 0x7fd9b40b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:42.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.484+0000 7fd9b259c640 1 --2- 172.21.15.83:0/2875108080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd99c004810 0x7fd9b4152fa0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4011660 tx=0x7fd9a4011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:42.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.484+0000 7fd9a2ffd640 1 -- 172.21.15.83:0/2875108080 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4004220 con 0x7fd99c004810 2024-04-17T10:38:42.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.484+0000 7fd9a2ffd640 1 -- 172.21.15.83:0/2875108080 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a40043c0 con 0x7fd99c004810 2024-04-17T10:38:42.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.484+0000 7fd9b8d9a640 1 -- 172.21.15.83:0/2875108080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b40b5350 con 0x7fd99c004810 2024-04-17T10:38:42.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.484+0000 7fd9a2ffd640 1 -- 172.21.15.83:0/2875108080 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a400f9c0 con 0x7fd99c004810 2024-04-17T10:38:42.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.484+0000 7fd9b8d9a640 1 -- 172.21.15.83:0/2875108080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b40b5910 con 0x7fd99c004810 2024-04-17T10:38:42.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.484+0000 7fd9b8d9a640 1 -- 172.21.15.83:0/2875108080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b40b3010 con 0x7fd99c004810 2024-04-17T10:38:42.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.488+0000 7fd9a2ffd640 1 -- 172.21.15.83:0/2875108080 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd9a400fb60 con 0x7fd99c004810 2024-04-17T10:38:42.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.488+0000 7fd9a2ffd640 1 --2- 172.21.15.83:0/2875108080 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd994066f60 0x7fd994069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:42.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.488+0000 7fd9a2ffd640 1 -- 172.21.15.83:0/2875108080 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd9a40853f0 con 0x7fd99c004810 2024-04-17T10:38:42.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.488+0000 7fd9b1d9b640 1 --2- 172.21.15.83:0/2875108080 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd994066f60 0x7fd994069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:42.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.488+0000 7fd9b1d9b640 1 --2- 172.21.15.83:0/2875108080 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd994066f60 0x7fd994069420 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd9a80aa000 tx=0x7fd9b40ceed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:42.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.492+0000 7fd9a2ffd640 1 -- 172.21.15.83:0/2875108080 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd9b40b5f60 con 0x7fd99c004810 2024-04-17T10:38:42.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.636+0000 7fd9b8d9a640 1 -- 172.21.15.83:0/2875108080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd9b4074f90 con 0x7fd99c004810 2024-04-17T10:38:42.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.640+0000 7fd9a2ffd640 1 -- 172.21.15.83:0/2875108080 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd9a4019030 con 0x7fd99c004810 2024-04-17T10:38:42.635 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:38:42.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.640+0000 7fd9b8d9a640 1 -- 172.21.15.83:0/2875108080 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd994066f60 msgr2=0x7fd994069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:42.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.640+0000 7fd9b8d9a640 1 --2- 172.21.15.83:0/2875108080 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd994066f60 0x7fd994069420 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd9a80aa000 tx=0x7fd9b40ceed0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.640+0000 7fd9b8d9a640 1 -- 172.21.15.83:0/2875108080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd99c004810 msgr2=0x7fd9b4152fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:42.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.640+0000 7fd9b8d9a640 1 --2- 172.21.15.83:0/2875108080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd99c004810 0x7fd9b4152fa0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4011660 tx=0x7fd9a4011b20 comp rx=0 tx=0).stop 2024-04-17T10:38:42.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.640+0000 7fd9b8d9a640 1 -- 172.21.15.83:0/2875108080 shutdown_connections 2024-04-17T10:38:42.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.640+0000 7fd9b8d9a640 1 --2- 172.21.15.83:0/2875108080 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd994066f60 0x7fd994069420 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.640+0000 7fd9b8d9a640 1 --2- 172.21.15.83:0/2875108080 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9b40b2bf0 0x7fd9b40c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.640+0000 7fd9b8d9a640 1 --2- 172.21.15.83:0/2875108080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9b40b4de0 0x7fd9b40b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.640+0000 7fd9b8d9a640 1 --2- 172.21.15.83:0/2875108080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd99c004810 0x7fd9b4152fa0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.640+0000 7fd9b8d9a640 1 -- 172.21.15.83:0/2875108080 >> 172.21.15.83:0/2875108080 conn(0x7fd9b40d17e0 msgr2=0x7fd9b40b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:42.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.640+0000 7fd9b8d9a640 1 -- 172.21.15.83:0/2875108080 shutdown_connections 2024-04-17T10:38:42.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.640+0000 7fd9b8d9a640 1 -- 172.21.15.83:0/2875108080 wait complete. 2024-04-17T10:38:42.653 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:38:42.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.776+0000 7fb77774f640 1 Processor -- start 2024-04-17T10:38:42.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.776+0000 7fb77774f640 1 -- start start 2024-04-17T10:38:42.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.776+0000 7fb77774f640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb77006b520 0x7fb770071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:42.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.776+0000 7fb77774f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb77006c1f0 con 0x7fb77015b6e0 2024-04-17T10:38:42.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.776+0000 7fb77774f640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb770070a10 con 0x7fb77015bac0 2024-04-17T10:38:42.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.776+0000 7fb77774f640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb770070cf0 con 0x7fb77006b520 2024-04-17T10:38:42.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.776+0000 7fb7754eb640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb77015b6e0 0x7fb7700cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47286/0 (socket says 172.21.15.83:47286) 2024-04-17T10:38:42.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.776+0000 7fb7754eb640 1 -- 172.21.15.83:0/2312313076 learned_addr learned my addr 172.21.15.83:0/2312313076 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:42.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb775cec640 1 --2- 172.21.15.83:0/2312313076 >> v2:172.21.15.167:3300/0 conn(0x7fb77006b520 0x7fb770071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:42.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb767fff640 1 -- 172.21.15.83:0/2312313076 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4255592636 0 0) 0x7fb77006c1f0 con 0x7fb77015b6e0 2024-04-17T10:38:42.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb767fff640 1 -- 172.21.15.83:0/2312313076 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb75c003530 con 0x7fb77015b6e0 2024-04-17T10:38:42.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb767fff640 1 -- 172.21.15.83:0/2312313076 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1262729061 0 0) 0x7fb75c003530 con 0x7fb77015b6e0 2024-04-17T10:38:42.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb767fff640 1 -- 172.21.15.83:0/2312313076 >> v2:172.21.15.167:3300/0 conn(0x7fb77006b520 msgr2=0x7fb770071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:42.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb767fff640 1 --2- 172.21.15.83:0/2312313076 >> v2:172.21.15.167:3300/0 conn(0x7fb77006b520 0x7fb770071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb767fff640 1 -- 172.21.15.83:0/2312313076 >> v1:172.21.15.167:6789/0 conn(0x7fb77015bac0 legacy=0x7fb77006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:42.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb767fff640 1 -- 172.21.15.83:0/2312313076 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb770070fd0 con 0x7fb77015b6e0 2024-04-17T10:38:42.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb767fff640 1 -- 172.21.15.83:0/2312313076 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fb76c001680 con 0x7fb77015b6e0 2024-04-17T10:38:42.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb775cec640 1 --2- 172.21.15.83:0/2312313076 >> v2:172.21.15.167:3300/0 conn(0x7fb77006b520 0x7fb770071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:38:42.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb767fff640 1 -- 172.21.15.83:0/2312313076 >> v1:172.21.15.83:6789/0 conn(0x7fb77015b6e0 legacy=0x7fb7700cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:42.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb767fff640 1 --2- 172.21.15.83:0/2312313076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb75c003a30 0x7fb75c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:42.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb767fff640 1 --2- 172.21.15.83:0/2312313076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb75c004810 0x7fb75c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:42.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb767fff640 1 --2- 172.21.15.83:0/2312313076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb75c005240 0x7fb75c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:42.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb767fff640 1 -- 172.21.15.83:0/2312313076 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb770070cf0 con 0x7fb75c005240 2024-04-17T10:38:42.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb767fff640 1 -- 172.21.15.83:0/2312313076 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c00c1a0 con 0x7fb75c004810 2024-04-17T10:38:42.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb767fff640 1 -- 172.21.15.83:0/2312313076 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c00c3a0 con 0x7fb75c003a30 2024-04-17T10:38:42.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb775cec640 1 --2- 172.21.15.83:0/2312313076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb75c005240 0x7fb75c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:42.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb774cea640 1 --2- 172.21.15.83:0/2312313076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb75c004810 0x7fb75c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:42.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb7754eb640 1 --2- 172.21.15.83:0/2312313076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb75c003a30 0x7fb75c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:42.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb775cec640 1 -- 172.21.15.83:0/2312313076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb75c003a30 msgr2=0x7fb75c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:42.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb775cec640 1 --2- 172.21.15.83:0/2312313076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb75c003a30 0x7fb75c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb775cec640 1 -- 172.21.15.83:0/2312313076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb75c004810 msgr2=0x7fb75c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:42.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb775cec640 1 --2- 172.21.15.83:0/2312313076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb75c004810 0x7fb75c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb775cec640 1 -- 172.21.15.83:0/2312313076 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb75c003530 con 0x7fb75c005240 2024-04-17T10:38:42.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb7754eb640 1 --2- 172.21.15.83:0/2312313076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb75c003a30 0x7fb75c003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:42.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb775cec640 1 -- 172.21.15.83:0/2312313076 reap_dead start 2024-04-17T10:38:42.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb775cec640 1 --2- 172.21.15.83:0/2312313076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb75c005240 0x7fb75c00b9d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb7680073e0 tx=0x7fb76801cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66731d0305209d33 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:42.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb767fff640 1 -- 172.21.15.83:0/2312313076 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb768009c80 con 0x7fb75c005240 2024-04-17T10:38:42.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb767fff640 1 -- 172.21.15.83:0/2312313076 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7680ad440 con 0x7fb75c005240 2024-04-17T10:38:42.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb767fff640 1 -- 172.21.15.83:0/2312313076 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7680ad830 con 0x7fb75c005240 2024-04-17T10:38:42.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb77774f640 1 -- 172.21.15.83:0/2312313076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb75c005240 msgr2=0x7fb75c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:42.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb77774f640 1 --2- 172.21.15.83:0/2312313076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb75c005240 0x7fb75c00b9d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb7680073e0 tx=0x7fb76801cf40 comp rx=0 tx=0).stop 2024-04-17T10:38:42.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb77774f640 1 -- 172.21.15.83:0/2312313076 shutdown_connections 2024-04-17T10:38:42.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb77774f640 1 --2- 172.21.15.83:0/2312313076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb75c005240 0x7fb75c00b9d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb77774f640 1 -- 172.21.15.83:0/2312313076 >> 172.21.15.83:0/2312313076 conn(0x7fb7700b15b0 msgr2=0x7fb7700b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:42.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb77774f640 1 -- 172.21.15.83:0/2312313076 shutdown_connections 2024-04-17T10:38:42.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.780+0000 7fb77774f640 1 -- 172.21.15.83:0/2312313076 wait complete. 2024-04-17T10:38:42.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.784+0000 7fb77774f640 1 Processor -- start 2024-04-17T10:38:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.784+0000 7fb77774f640 1 -- start start 2024-04-17T10:38:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.784+0000 7fb77774f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb75c005240 0x7fb7700ad910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.784+0000 7fb77774f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb7700a9430 0x7fb7701564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.784+0000 7fb77774f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7700ab740 0x7fb7700abbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.784+0000 7fb77774f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7700705f0 con 0x7fb75c005240 2024-04-17T10:38:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.784+0000 7fb77774f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb770070470 con 0x7fb7700ab740 2024-04-17T10:38:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.784+0000 7fb77774f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb770070770 con 0x7fb7700a9430 2024-04-17T10:38:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.784+0000 7fb7754eb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb75c005240 0x7fb7700ad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.784+0000 7fb7754eb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb75c005240 0x7fb7700ad910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52698/0 (socket says 172.21.15.83:52698) 2024-04-17T10:38:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.784+0000 7fb7754eb640 1 -- 172.21.15.83:0/586555997 learned_addr learned my addr 172.21.15.83:0/586555997 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.784+0000 7fb774cea640 1 --2- 172.21.15.83:0/586555997 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb7700a9430 0x7fb7701564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.784+0000 7fb775cec640 1 --2- 172.21.15.83:0/586555997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7700ab740 0x7fb7700abbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.784+0000 7fb7754eb640 1 -- 172.21.15.83:0/586555997 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb7700a9430 msgr2=0x7fb7701564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:42.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.784+0000 7fb7754eb640 1 --2- 172.21.15.83:0/586555997 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb7700a9430 0x7fb7701564d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.784+0000 7fb7754eb640 1 -- 172.21.15.83:0/586555997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7700ab740 msgr2=0x7fb7700abbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:42.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.784+0000 7fb7754eb640 1 --2- 172.21.15.83:0/586555997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7700ab740 0x7fb7700abbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.784+0000 7fb7754eb640 1 -- 172.21.15.83:0/586555997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7700a98b0 con 0x7fb75c005240 2024-04-17T10:38:42.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.784+0000 7fb775cec640 1 --2- 172.21.15.83:0/586555997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7700ab740 0x7fb7700abbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:42.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.784+0000 7fb7754eb640 1 --2- 172.21.15.83:0/586555997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb75c005240 0x7fb7700ad910 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb76c001210 tx=0x7fb76c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:42.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.784+0000 7fb765ffb640 1 -- 172.21.15.83:0/586555997 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c019070 con 0x7fb75c005240 2024-04-17T10:38:42.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.784+0000 7fb765ffb640 1 -- 172.21.15.83:0/586555997 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76c0074e0 con 0x7fb75c005240 2024-04-17T10:38:42.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.784+0000 7fb765ffb640 1 -- 172.21.15.83:0/586555997 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c0049f0 con 0x7fb75c005240 2024-04-17T10:38:42.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.784+0000 7fb77774f640 1 -- 172.21.15.83:0/586555997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7700a9b30 con 0x7fb75c005240 2024-04-17T10:38:42.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.784+0000 7fb77774f640 1 -- 172.21.15.83:0/586555997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7700aa0f0 con 0x7fb75c005240 2024-04-17T10:38:42.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.784+0000 7fb77774f640 1 -- 172.21.15.83:0/586555997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb770074f90 con 0x7fb75c005240 2024-04-17T10:38:42.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.788+0000 7fb765ffb640 1 -- 172.21.15.83:0/586555997 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb76c004b90 con 0x7fb75c005240 2024-04-17T10:38:42.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.788+0000 7fb765ffb640 1 --2- 172.21.15.83:0/586555997 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb754066f90 0x7fb754069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:42.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.788+0000 7fb765ffb640 1 -- 172.21.15.83:0/586555997 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb76c084a30 con 0x7fb75c005240 2024-04-17T10:38:42.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.788+0000 7fb774cea640 1 --2- 172.21.15.83:0/586555997 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb754066f90 0x7fb754069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:42.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.788+0000 7fb774cea640 1 --2- 172.21.15.83:0/586555997 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb754066f90 0x7fb754069450 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb760000c10 tx=0x7fb760009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:42.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.792+0000 7fb765ffb640 1 -- 172.21.15.83:0/586555997 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb7700a98b0 con 0x7fb75c005240 2024-04-17T10:38:42.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.944+0000 7fb77774f640 1 -- 172.21.15.83:0/586555997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb770071770 con 0x7fb75c005240 2024-04-17T10:38:42.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.944+0000 7fb765ffb640 1 -- 172.21.15.83:0/586555997 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb76c001d10 con 0x7fb75c005240 2024-04-17T10:38:42.941 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:38:42.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.944+0000 7fb77774f640 1 -- 172.21.15.83:0/586555997 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb754066f90 msgr2=0x7fb754069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:42.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.944+0000 7fb77774f640 1 --2- 172.21.15.83:0/586555997 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb754066f90 0x7fb754069450 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb760000c10 tx=0x7fb760009040 comp rx=0 tx=0).stop 2024-04-17T10:38:42.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.944+0000 7fb77774f640 1 -- 172.21.15.83:0/586555997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb75c005240 msgr2=0x7fb7700ad910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:42.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.944+0000 7fb77774f640 1 --2- 172.21.15.83:0/586555997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb75c005240 0x7fb7700ad910 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb76c001210 tx=0x7fb76c00dd40 comp rx=0 tx=0).stop 2024-04-17T10:38:42.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.948+0000 7fb77774f640 1 -- 172.21.15.83:0/586555997 shutdown_connections 2024-04-17T10:38:42.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.948+0000 7fb77774f640 1 --2- 172.21.15.83:0/586555997 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb754066f90 0x7fb754069450 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.948+0000 7fb77774f640 1 --2- 172.21.15.83:0/586555997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7700ab740 0x7fb7700abbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.948+0000 7fb77774f640 1 --2- 172.21.15.83:0/586555997 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb7700a9430 0x7fb7701564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.948+0000 7fb77774f640 1 --2- 172.21.15.83:0/586555997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb75c005240 0x7fb7700ad910 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:42.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.948+0000 7fb77774f640 1 -- 172.21.15.83:0/586555997 >> 172.21.15.83:0/586555997 conn(0x7fb7700b15b0 msgr2=0x7fb77006ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:42.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.948+0000 7fb77774f640 1 -- 172.21.15.83:0/586555997 shutdown_connections 2024-04-17T10:38:42.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:42.948+0000 7fb77774f640 1 -- 172.21.15.83:0/586555997 wait complete. 2024-04-17T10:38:42.958 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:38:43.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.076+0000 7f032ada9640 1 Processor -- start 2024-04-17T10:38:43.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.076+0000 7f032ada9640 1 -- start start 2024-04-17T10:38:43.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.076+0000 7f032ada9640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f032406e800 0x7f032406bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:43.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.080+0000 7f032ada9640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f032406ec90 con 0x7f032415a8a0 2024-04-17T10:38:43.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.080+0000 7f032ada9640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03240afbb0 con 0x7f032406c230 2024-04-17T10:38:43.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.080+0000 7f032ada9640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f032406d760 con 0x7f032406e800 2024-04-17T10:38:43.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.080+0000 7f031bfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f032406e800 0x7f032406bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:43.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.080+0000 7f0328b45640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f032415a8a0 0x7f03240b36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47290/0 (socket says 172.21.15.83:47290) 2024-04-17T10:38:43.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.080+0000 7f0328b45640 1 -- 172.21.15.83:0/1551642530 learned_addr learned my addr 172.21.15.83:0/1551642530 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:43.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.080+0000 7f031bfff640 1 -- 172.21.15.83:0/1551642530 >> v1:172.21.15.167:6789/0 conn(0x7f032406c230 legacy=0x7f03240cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:43.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.080+0000 7f031bfff640 1 -- 172.21.15.83:0/1551642530 >> v1:172.21.15.83:6789/0 conn(0x7f032415a8a0 legacy=0x7f03240b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:43.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.080+0000 7f031bfff640 1 -- 172.21.15.83:0/1551642530 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032406da40 con 0x7f032406e800 2024-04-17T10:38:43.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.080+0000 7f031bfff640 1 --2- 172.21.15.83:0/1551642530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f032406e800 0x7f032406bcf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f03140097c0 tx=0x7f031401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6949a971213613e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:43.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.080+0000 7f031b7fe640 1 -- 172.21.15.83:0/1551642530 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03140b0070 con 0x7f032406e800 2024-04-17T10:38:43.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.080+0000 7f031b7fe640 1 -- 172.21.15.83:0/1551642530 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03140ab600 con 0x7f032406e800 2024-04-17T10:38:43.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.080+0000 7f031b7fe640 1 -- 172.21.15.83:0/1551642530 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03140a4e60 con 0x7f032406e800 2024-04-17T10:38:43.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.080+0000 7f032ada9640 1 -- 172.21.15.83:0/1551642530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f032406e800 msgr2=0x7f032406bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:43.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.080+0000 7f032ada9640 1 --2- 172.21.15.83:0/1551642530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f032406e800 0x7f032406bcf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f03140097c0 tx=0x7f031401c3d0 comp rx=0 tx=0).stop 2024-04-17T10:38:43.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.080+0000 7f032ada9640 1 -- 172.21.15.83:0/1551642530 shutdown_connections 2024-04-17T10:38:43.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.080+0000 7f032ada9640 1 --2- 172.21.15.83:0/1551642530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f032406e800 0x7f032406bcf0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:43.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.080+0000 7f032ada9640 1 -- 172.21.15.83:0/1551642530 >> 172.21.15.83:0/1551642530 conn(0x7f03240b23f0 msgr2=0x7f03240b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:43.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.080+0000 7f032ada9640 1 -- 172.21.15.83:0/1551642530 shutdown_connections 2024-04-17T10:38:43.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.080+0000 7f032ada9640 1 -- 172.21.15.83:0/1551642530 wait complete. 2024-04-17T10:38:43.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.084+0000 7f032ada9640 1 Processor -- start 2024-04-17T10:38:43.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.084+0000 7f032ada9640 1 -- start start 2024-04-17T10:38:43.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.084+0000 7f032ada9640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f032406c230 0x7f032414c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:43.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.084+0000 7f032ada9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f032406e800 0x7f0324145240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:43.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.084+0000 7f032ada9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f032415a8a0 0x7f032414dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:43.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.084+0000 7f032ada9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03241683b0 con 0x7f032406e800 2024-04-17T10:38:43.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.084+0000 7f032ada9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032406d3f0 con 0x7f032415a8a0 2024-04-17T10:38:43.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.084+0000 7f032ada9640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0324168530 con 0x7f032406c230 2024-04-17T10:38:43.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.084+0000 7f031bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f032406e800 0x7f0324145240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:43.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.084+0000 7f031bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f032406e800 0x7f0324145240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52712/0 (socket says 172.21.15.83:52712) 2024-04-17T10:38:43.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.084+0000 7f031bfff640 1 -- 172.21.15.83:0/2124499830 learned_addr learned my addr 172.21.15.83:0/2124499830 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:43.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.084+0000 7f0328b45640 1 --2- 172.21.15.83:0/2124499830 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f032406c230 0x7f032414c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:43.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.084+0000 7f0329346640 1 --2- 172.21.15.83:0/2124499830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f032415a8a0 0x7f032414dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:43.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.084+0000 7f031bfff640 1 -- 172.21.15.83:0/2124499830 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f032406c230 msgr2=0x7f032414c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:43.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.084+0000 7f031bfff640 1 --2- 172.21.15.83:0/2124499830 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f032406c230 0x7f032414c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:43.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.084+0000 7f031bfff640 1 -- 172.21.15.83:0/2124499830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f032415a8a0 msgr2=0x7f032414dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:43.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.084+0000 7f031bfff640 1 --2- 172.21.15.83:0/2124499830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f032415a8a0 0x7f032414dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:43.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.084+0000 7f031bfff640 1 -- 172.21.15.83:0/2124499830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032414e200 con 0x7f032406e800 2024-04-17T10:38:43.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.084+0000 7f0329346640 1 --2- 172.21.15.83:0/2124499830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f032415a8a0 0x7f032414dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:38:43.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.084+0000 7f031bfff640 1 --2- 172.21.15.83:0/2124499830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f032406e800 0x7f0324145240 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f031401a000 tx=0x7f031401ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:43.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.084+0000 7f03197fa640 1 -- 172.21.15.83:0/2124499830 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03140b0050 con 0x7f032406e800 2024-04-17T10:38:43.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.084+0000 7f03197fa640 1 -- 172.21.15.83:0/2124499830 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03140bc070 con 0x7f032406e800 2024-04-17T10:38:43.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.084+0000 7f032ada9640 1 -- 172.21.15.83:0/2124499830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f032414e4e0 con 0x7f032406e800 2024-04-17T10:38:43.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.084+0000 7f03197fa640 1 -- 172.21.15.83:0/2124499830 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03140b5830 con 0x7f032406e800 2024-04-17T10:38:43.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.084+0000 7f032ada9640 1 -- 172.21.15.83:0/2124499830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f032415afe0 con 0x7f032406e800 2024-04-17T10:38:43.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.084+0000 7f032ada9640 1 -- 172.21.15.83:0/2124499830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f032414cec0 con 0x7f032406e800 2024-04-17T10:38:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.088+0000 7f03197fa640 1 -- 172.21.15.83:0/2124499830 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f03140b59d0 con 0x7f032406e800 2024-04-17T10:38:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.088+0000 7f03197fa640 1 --2- 172.21.15.83:0/2124499830 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0310066f60 0x7f0310069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.088+0000 7f03197fa640 1 -- 172.21.15.83:0/2124499830 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f031411f800 con 0x7f032406e800 2024-04-17T10:38:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.088+0000 7f0328b45640 1 --2- 172.21.15.83:0/2124499830 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0310066f60 0x7f0310069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.088+0000 7f0328b45640 1 --2- 172.21.15.83:0/2124499830 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0310066f60 0x7f0310069420 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0320000f70 tx=0x7f0320007e90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:43.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.092+0000 7f03197fa640 1 -- 172.21.15.83:0/2124499830 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f032414cec0 con 0x7f032406e800 2024-04-17T10:38:43.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.244+0000 7f032ada9640 1 -- 172.21.15.83:0/2124499830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0324074f90 con 0x7f032406e800 2024-04-17T10:38:43.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.244+0000 7f03197fa640 1 -- 172.21.15.83:0/2124499830 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0314016050 con 0x7f032406e800 2024-04-17T10:38:43.241 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:38:43.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.248+0000 7f032ada9640 1 -- 172.21.15.83:0/2124499830 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0310066f60 msgr2=0x7f0310069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:43.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.248+0000 7f032ada9640 1 --2- 172.21.15.83:0/2124499830 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0310066f60 0x7f0310069420 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0320000f70 tx=0x7f0320007e90 comp rx=0 tx=0).stop 2024-04-17T10:38:43.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.248+0000 7f032ada9640 1 -- 172.21.15.83:0/2124499830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f032406e800 msgr2=0x7f0324145240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:43.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.248+0000 7f032ada9640 1 --2- 172.21.15.83:0/2124499830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f032406e800 0x7f0324145240 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f031401a000 tx=0x7f031401ac10 comp rx=0 tx=0).stop 2024-04-17T10:38:43.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.248+0000 7f032ada9640 1 -- 172.21.15.83:0/2124499830 shutdown_connections 2024-04-17T10:38:43.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.248+0000 7f032ada9640 1 --2- 172.21.15.83:0/2124499830 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0310066f60 0x7f0310069420 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:43.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.248+0000 7f032ada9640 1 --2- 172.21.15.83:0/2124499830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f032415a8a0 0x7f032414dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:43.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.248+0000 7f032ada9640 1 --2- 172.21.15.83:0/2124499830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f032406e800 0x7f0324145240 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:43.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.248+0000 7f032ada9640 1 --2- 172.21.15.83:0/2124499830 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f032406c230 0x7f032414c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:43.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.248+0000 7f032ada9640 1 -- 172.21.15.83:0/2124499830 >> 172.21.15.83:0/2124499830 conn(0x7f03240b23f0 msgr2=0x7f03240b0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:43.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.248+0000 7f032ada9640 1 -- 172.21.15.83:0/2124499830 shutdown_connections 2024-04-17T10:38:43.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.248+0000 7f032ada9640 1 -- 172.21.15.83:0/2124499830 wait complete. 2024-04-17T10:38:43.259 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:38:43.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.380+0000 7f5e66f9a640 1 Processor -- start 2024-04-17T10:38:43.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.380+0000 7f5e66f9a640 1 -- start start 2024-04-17T10:38:43.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.384+0000 7f5e66f9a640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5e6015dfa0 0x7f5e600b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:43.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.384+0000 7f5e66f9a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5e600ce980 0x7f5e600b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:43.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.384+0000 7f5e66f9a640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e60162a00 con 0x7f5e600cee00 2024-04-17T10:38:43.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.384+0000 7f5e66f9a640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e60161e80 con 0x7f5e6015dfa0 2024-04-17T10:38:43.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.384+0000 7f5e66f9a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e60162160 con 0x7f5e600ce980 2024-04-17T10:38:43.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.384+0000 7f5e5ffff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5e600ce980 0x7f5e600b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:43.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.384+0000 7f5e64d36640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5e6015dfa0 0x7f5e600b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:43.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.384+0000 7f5e5ffff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5e600ce980 0x7f5e600b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:48898/0 (socket says 172.21.15.83:48898) 2024-04-17T10:38:43.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.384+0000 7f5e5ffff640 1 -- 172.21.15.83:0/2973932361 learned_addr learned my addr 172.21.15.83:0/2973932361 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:43.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.384+0000 7f5e64d36640 1 -- 172.21.15.83:0/2973932361 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5e600ce980 msgr2=0x7f5e600b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:43.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.384+0000 7f5e64d36640 1 --2- 172.21.15.83:0/2973932361 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5e600ce980 0x7f5e600b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:43.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.384+0000 7f5e64d36640 1 -- 172.21.15.83:0/2973932361 >> v1:172.21.15.167:6789/0 conn(0x7f5e600cee00 legacy=0x7f5e600d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:43.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.384+0000 7f5e64d36640 1 -- 172.21.15.83:0/2973932361 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e60162440 con 0x7f5e6015dfa0 2024-04-17T10:38:43.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.384+0000 7f5e5ffff640 1 --2- 172.21.15.83:0/2973932361 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5e600ce980 0x7f5e600b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:38:43.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.388+0000 7f5e64d36640 1 --2- 172.21.15.83:0/2973932361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e6015dfa0 0x7f5e600b6c20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5e540097c0 tx=0x7f5e5401c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59657b804589d6f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:43.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.388+0000 7f5e5f7fe640 1 -- 172.21.15.83:0/2973932361 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e540b0070 con 0x7f5e6015dfa0 2024-04-17T10:38:43.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.388+0000 7f5e5f7fe640 1 -- 172.21.15.83:0/2973932361 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e540ab600 con 0x7f5e6015dfa0 2024-04-17T10:38:43.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.388+0000 7f5e5f7fe640 1 -- 172.21.15.83:0/2973932361 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e540ab9d0 con 0x7f5e6015dfa0 2024-04-17T10:38:43.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.388+0000 7f5e66f9a640 1 -- 172.21.15.83:0/2973932361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e6015dfa0 msgr2=0x7f5e600b6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:43.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.388+0000 7f5e66f9a640 1 --2- 172.21.15.83:0/2973932361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e6015dfa0 0x7f5e600b6c20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5e540097c0 tx=0x7f5e5401c3e0 comp rx=0 tx=0).stop 2024-04-17T10:38:43.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.388+0000 7f5e66f9a640 1 -- 172.21.15.83:0/2973932361 shutdown_connections 2024-04-17T10:38:43.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.388+0000 7f5e66f9a640 1 --2- 172.21.15.83:0/2973932361 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5e600ce980 0x7f5e600b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:43.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.388+0000 7f5e66f9a640 1 --2- 172.21.15.83:0/2973932361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e6015dfa0 0x7f5e600b6c20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:43.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.388+0000 7f5e66f9a640 1 -- 172.21.15.83:0/2973932361 >> 172.21.15.83:0/2973932361 conn(0x7f5e600d0600 msgr2=0x7f5e600d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:43.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.388+0000 7f5e66f9a640 1 -- 172.21.15.83:0/2973932361 shutdown_connections 2024-04-17T10:38:43.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.388+0000 7f5e66f9a640 1 -- 172.21.15.83:0/2973932361 wait complete. 2024-04-17T10:38:43.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.388+0000 7f5e66f9a640 1 Processor -- start 2024-04-17T10:38:43.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.388+0000 7f5e66f9a640 1 -- start start 2024-04-17T10:38:43.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.388+0000 7f5e66f9a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e600ce980 0x7f5e60154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:43.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.388+0000 7f5e66f9a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5e600cee00 0x7f5e6014fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:43.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.388+0000 7f5e66f9a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e6015dfa0 0x7f5e60150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:43.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.388+0000 7f5e66f9a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e60161b10 con 0x7f5e600ce980 2024-04-17T10:38:43.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.388+0000 7f5e66f9a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e600d2850 con 0x7f5e6015dfa0 2024-04-17T10:38:43.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.388+0000 7f5e66f9a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6016b660 con 0x7f5e600cee00 2024-04-17T10:38:43.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.388+0000 7f5e64d36640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e600ce980 0x7f5e60154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:43.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.388+0000 7f5e64d36640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e600ce980 0x7f5e60154180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52726/0 (socket says 172.21.15.83:52726) 2024-04-17T10:38:43.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.388+0000 7f5e64d36640 1 -- 172.21.15.83:0/2809832511 learned_addr learned my addr 172.21.15.83:0/2809832511 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:43.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.388+0000 7f5e5ffff640 1 --2- 172.21.15.83:0/2809832511 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5e600cee00 0x7f5e6014fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:43.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.388+0000 7f5e65537640 1 --2- 172.21.15.83:0/2809832511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e6015dfa0 0x7f5e60150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:43.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.388+0000 7f5e64d36640 1 -- 172.21.15.83:0/2809832511 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5e600cee00 msgr2=0x7f5e6014fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:43.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.388+0000 7f5e64d36640 1 --2- 172.21.15.83:0/2809832511 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5e600cee00 0x7f5e6014fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:43.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.388+0000 7f5e64d36640 1 -- 172.21.15.83:0/2809832511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e6015dfa0 msgr2=0x7f5e60150530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:43.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.388+0000 7f5e64d36640 1 --2- 172.21.15.83:0/2809832511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e6015dfa0 0x7f5e60150530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:43.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.388+0000 7f5e64d36640 1 -- 172.21.15.83:0/2809832511 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e6014ca70 con 0x7f5e600ce980 2024-04-17T10:38:43.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.388+0000 7f5e65537640 1 --2- 172.21.15.83:0/2809832511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e6015dfa0 0x7f5e60150530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:38:43.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.392+0000 7f5e64d36640 1 --2- 172.21.15.83:0/2809832511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e600ce980 0x7f5e60154180 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5e540ae040 tx=0x7f5e5401ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:43.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.392+0000 7f5e5d7fa640 1 -- 172.21.15.83:0/2809832511 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e540b0050 con 0x7f5e600ce980 2024-04-17T10:38:43.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.392+0000 7f5e5d7fa640 1 -- 172.21.15.83:0/2809832511 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e540b1040 con 0x7f5e600ce980 2024-04-17T10:38:43.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.392+0000 7f5e66f9a640 1 -- 172.21.15.83:0/2809832511 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e6014cd50 con 0x7f5e600ce980 2024-04-17T10:38:43.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.392+0000 7f5e5d7fa640 1 -- 172.21.15.83:0/2809832511 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e54006e20 con 0x7f5e600ce980 2024-04-17T10:38:43.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.392+0000 7f5e66f9a640 1 -- 172.21.15.83:0/2809832511 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e6014d310 con 0x7f5e600ce980 2024-04-17T10:38:43.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.392+0000 7f5e66f9a640 1 -- 172.21.15.83:0/2809832511 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e60074f90 con 0x7f5e600ce980 2024-04-17T10:38:43.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.392+0000 7f5e5d7fa640 1 -- 172.21.15.83:0/2809832511 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5e540be030 con 0x7f5e600ce980 2024-04-17T10:38:43.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.392+0000 7f5e5d7fa640 1 --2- 172.21.15.83:0/2809832511 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5e44066f60 0x7f5e44069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:43.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.392+0000 7f5e5d7fa640 1 -- 172.21.15.83:0/2809832511 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5e5411eda0 con 0x7f5e600ce980 2024-04-17T10:38:43.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.392+0000 7f5e5ffff640 1 --2- 172.21.15.83:0/2809832511 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5e44066f60 0x7f5e44069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:43.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.392+0000 7f5e5ffff640 1 --2- 172.21.15.83:0/2809832511 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5e44066f60 0x7f5e44069420 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5e600c70b0 tx=0x7f5e4c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:43.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.396+0000 7f5e5d7fa640 1 -- 172.21.15.83:0/2809832511 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5e540c5060 con 0x7f5e600ce980 2024-04-17T10:38:43.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.556+0000 7f5e66f9a640 1 -- 172.21.15.83:0/2809832511 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e6014e570 con 0x7f5e600ce980 2024-04-17T10:38:43.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.560+0000 7f5e5d7fa640 1 -- 172.21.15.83:0/2809832511 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16533 (secure 0 0 0) 0x7f5e54016090 con 0x7f5e600ce980 2024-04-17T10:38:43.555 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:43.555 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":20,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:38:13.690876+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:38:10.464490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T10:38:11.043134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:38:43.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.560+0000 7f5e66f9a640 1 -- 172.21.15.83:0/2809832511 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5e44066f60 msgr2=0x7f5e44069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:43.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.560+0000 7f5e66f9a640 1 --2- 172.21.15.83:0/2809832511 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5e44066f60 0x7f5e44069420 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5e600c70b0 tx=0x7f5e4c008040 comp rx=0 tx=0).stop 2024-04-17T10:38:43.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.560+0000 7f5e66f9a640 1 -- 172.21.15.83:0/2809832511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e600ce980 msgr2=0x7f5e60154180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:43.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.560+0000 7f5e66f9a640 1 --2- 172.21.15.83:0/2809832511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e600ce980 0x7f5e60154180 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5e540ae040 tx=0x7f5e5401ac20 comp rx=0 tx=0).stop 2024-04-17T10:38:43.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.560+0000 7f5e66f9a640 1 -- 172.21.15.83:0/2809832511 shutdown_connections 2024-04-17T10:38:43.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.560+0000 7f5e66f9a640 1 --2- 172.21.15.83:0/2809832511 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5e44066f60 0x7f5e44069420 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:43.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.560+0000 7f5e66f9a640 1 --2- 172.21.15.83:0/2809832511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e6015dfa0 0x7f5e60150530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:43.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.560+0000 7f5e66f9a640 1 --2- 172.21.15.83:0/2809832511 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5e600cee00 0x7f5e6014fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:43.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.560+0000 7f5e66f9a640 1 --2- 172.21.15.83:0/2809832511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e600ce980 0x7f5e60154180 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:43.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.560+0000 7f5e66f9a640 1 -- 172.21.15.83:0/2809832511 >> 172.21.15.83:0/2809832511 conn(0x7f5e600d0600 msgr2=0x7f5e601609e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:43.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.560+0000 7f5e66f9a640 1 -- 172.21.15.83:0/2809832511 shutdown_connections 2024-04-17T10:38:43.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.560+0000 7f5e66f9a640 1 -- 172.21.15.83:0/2809832511 wait complete. 2024-04-17T10:38:43.574 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:38:10.464490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:38:11.043134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:38:43.575 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:38:43.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.696+0000 7fe8fed74640 1 Processor -- start 2024-04-17T10:38:43.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.700+0000 7fe8fed74640 1 -- start start 2024-04-17T10:38:43.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.700+0000 7fe8fed74640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe8f815aa10 0x7fe8f80729d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:43.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.700+0000 7fe8fed74640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe8f80cbca0 0x7fe8f806bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:43.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.700+0000 7fe8fed74640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f80cc510 con 0x7fe8f806c130 2024-04-17T10:38:43.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.700+0000 7fe8fed74640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8f8072f10 con 0x7fe8f815aa10 2024-04-17T10:38:43.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.700+0000 7fe8fed74640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8f806c560 con 0x7fe8f80cbca0 2024-04-17T10:38:43.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.700+0000 7fe8fcb10640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe8f815aa10 0x7fe8f80729d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:43.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.700+0000 7fe8fd311640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe8f806c130 0x7fe8f806db70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47294/0 (socket says 172.21.15.83:47294) 2024-04-17T10:38:43.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.700+0000 7fe8fd311640 1 -- 172.21.15.83:0/3947020297 learned_addr learned my addr 172.21.15.83:0/3947020297 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:43.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.700+0000 7fe8f7fff640 1 --2- 172.21.15.83:0/3947020297 >> v2:172.21.15.167:3300/0 conn(0x7fe8f80cbca0 0x7fe8f806bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:43.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.700+0000 7fe8f77fe640 1 -- 172.21.15.83:0/3947020297 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 383147465 0 0) 0x7fe8f80cc510 con 0x7fe8f806c130 2024-04-17T10:38:43.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.700+0000 7fe8f77fe640 1 -- 172.21.15.83:0/3947020297 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8e8005740 con 0x7fe8f806c130 2024-04-17T10:38:43.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.700+0000 7fe8fcb10640 1 -- 172.21.15.83:0/3947020297 >> v2:172.21.15.167:3300/0 conn(0x7fe8f80cbca0 msgr2=0x7fe8f806bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:43.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.700+0000 7fe8fcb10640 1 --2- 172.21.15.83:0/3947020297 >> v2:172.21.15.167:3300/0 conn(0x7fe8f80cbca0 0x7fe8f806bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:43.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.700+0000 7fe8fcb10640 1 -- 172.21.15.83:0/3947020297 >> v1:172.21.15.83:6789/0 conn(0x7fe8f806c130 legacy=0x7fe8f806db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:43.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.700+0000 7fe8fcb10640 1 -- 172.21.15.83:0/3947020297 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f815e740 con 0x7fe8f815aa10 2024-04-17T10:38:43.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8fcb10640 1 --2- 172.21.15.83:0/3947020297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8f815aa10 0x7fe8f80729d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8072180 tx=0x7fe8ec01c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4187143390637b33 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:43.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8f77fe640 1 -- 172.21.15.83:0/3947020297 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec0b0070 con 0x7fe8f815aa10 2024-04-17T10:38:43.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8f77fe640 1 -- 172.21.15.83:0/3947020297 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8ec0ab600 con 0x7fe8f815aa10 2024-04-17T10:38:43.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8f77fe640 1 -- 172.21.15.83:0/3947020297 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec0ab9b0 con 0x7fe8f815aa10 2024-04-17T10:38:43.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8fed74640 1 -- 172.21.15.83:0/3947020297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8f815aa10 msgr2=0x7fe8f80729d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:43.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8fed74640 1 --2- 172.21.15.83:0/3947020297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8f815aa10 0x7fe8f80729d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8072180 tx=0x7fe8ec01c3a0 comp rx=0 tx=0).stop 2024-04-17T10:38:43.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8fed74640 1 -- 172.21.15.83:0/3947020297 shutdown_connections 2024-04-17T10:38:43.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8fed74640 1 --2- 172.21.15.83:0/3947020297 >> v2:172.21.15.167:3300/0 conn(0x7fe8f80cbca0 0x7fe8f806bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:43.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8fed74640 1 --2- 172.21.15.83:0/3947020297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8f815aa10 0x7fe8f80729d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:43.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8fed74640 1 -- 172.21.15.83:0/3947020297 >> 172.21.15.83:0/3947020297 conn(0x7fe8f80cc990 msgr2=0x7fe8f80ccda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:43.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8fed74640 1 -- 172.21.15.83:0/3947020297 shutdown_connections 2024-04-17T10:38:43.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8fed74640 1 -- 172.21.15.83:0/3947020297 wait complete. 2024-04-17T10:38:43.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8fed74640 1 Processor -- start 2024-04-17T10:38:43.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8fed74640 1 -- start start 2024-04-17T10:38:43.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8fed74640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8f806c130 0x7fe8f809fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:43.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8fed74640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8f80cbca0 0x7fe8f80c2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:43.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8fed74640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8f815aa10 0x7fe8f80ce280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:43.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8fed74640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f815adf0 con 0x7fe8f80cbca0 2024-04-17T10:38:43.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8fed74640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f815f2c0 con 0x7fe8f806c130 2024-04-17T10:38:43.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8fed74640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f815af70 con 0x7fe8f815aa10 2024-04-17T10:38:43.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8f7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8f80cbca0 0x7fe8f80c2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:43.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8f7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8f80cbca0 0x7fe8f80c2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52752/0 (socket says 172.21.15.83:52752) 2024-04-17T10:38:43.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8f7fff640 1 -- 172.21.15.83:0/2057729092 learned_addr learned my addr 172.21.15.83:0/2057729092 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:43.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8fd311640 1 --2- 172.21.15.83:0/2057729092 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8f815aa10 0x7fe8f80ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:43.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8f7fff640 1 -- 172.21.15.83:0/2057729092 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8f815aa10 msgr2=0x7fe8f80ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:43.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8f7fff640 1 --2- 172.21.15.83:0/2057729092 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8f815aa10 0x7fe8f80ce280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:43.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8f7fff640 1 -- 172.21.15.83:0/2057729092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8f806c130 msgr2=0x7fe8f809fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:43.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8fcb10640 1 --2- 172.21.15.83:0/2057729092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8f806c130 0x7fe8f809fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:43.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8f7fff640 1 --2- 172.21.15.83:0/2057729092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8f806c130 0x7fe8f809fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:43.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8f7fff640 1 -- 172.21.15.83:0/2057729092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f806da10 con 0x7fe8f80cbca0 2024-04-17T10:38:43.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8f7fff640 1 --2- 172.21.15.83:0/2057729092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8f80cbca0 0x7fe8f80c2ce0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe8e400d830 tx=0x7fe8e400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:43.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8f57fa640 1 -- 172.21.15.83:0/2057729092 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e400b6c0 con 0x7fe8f80cbca0 2024-04-17T10:38:43.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8f57fa640 1 -- 172.21.15.83:0/2057729092 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e4019070 con 0x7fe8f80cbca0 2024-04-17T10:38:43.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8fed74640 1 -- 172.21.15.83:0/2057729092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f809f340 con 0x7fe8f80cbca0 2024-04-17T10:38:43.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8f57fa640 1 -- 172.21.15.83:0/2057729092 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e400ba60 con 0x7fe8f80cbca0 2024-04-17T10:38:43.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.704+0000 7fe8fed74640 1 -- 172.21.15.83:0/2057729092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f809f5e0 con 0x7fe8f80cbca0 2024-04-17T10:38:43.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.708+0000 7fe8fed74640 1 -- 172.21.15.83:0/2057729092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8f8074f90 con 0x7fe8f80cbca0 2024-04-17T10:38:43.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.708+0000 7fe8f57fa640 1 -- 172.21.15.83:0/2057729092 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe8e400bc00 con 0x7fe8f80cbca0 2024-04-17T10:38:43.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.708+0000 7fe8f57fa640 1 --2- 172.21.15.83:0/2057729092 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe8dc066f60 0x7fe8dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:43.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.708+0000 7fe8f57fa640 1 -- 172.21.15.83:0/2057729092 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8e4085400 con 0x7fe8f80cbca0 2024-04-17T10:38:43.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.708+0000 7fe8fcb10640 1 --2- 172.21.15.83:0/2057729092 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe8dc066f60 0x7fe8dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:43.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.708+0000 7fe8fcb10640 1 --2- 172.21.15.83:0/2057729092 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe8dc066f60 0x7fe8dc069420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec0ad000 tx=0x7fe8ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:43.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.712+0000 7fe8f57fa640 1 -- 172.21.15.83:0/2057729092 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe8e40564a0 con 0x7fe8f80cbca0 2024-04-17T10:38:43.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.860+0000 7fe8fed74640 1 -- 172.21.15.83:0/2057729092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe8f8066ae0 con 0x7fe8f80cbca0 2024-04-17T10:38:43.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.860+0000 7fe8f57fa640 1 -- 172.21.15.83:0/2057729092 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe8e4055da0 con 0x7fe8f80cbca0 2024-04-17T10:38:43.856 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:38:43.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.860+0000 7fe8fed74640 1 -- 172.21.15.83:0/2057729092 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe8dc066f60 msgr2=0x7fe8dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:43.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.860+0000 7fe8fed74640 1 --2- 172.21.15.83:0/2057729092 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe8dc066f60 0x7fe8dc069420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec0ad000 tx=0x7fe8ec008040 comp rx=0 tx=0).stop 2024-04-17T10:38:43.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.860+0000 7fe8fed74640 1 -- 172.21.15.83:0/2057729092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8f80cbca0 msgr2=0x7fe8f80c2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:43.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.860+0000 7fe8fed74640 1 --2- 172.21.15.83:0/2057729092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8f80cbca0 0x7fe8f80c2ce0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe8e400d830 tx=0x7fe8e400dcf0 comp rx=0 tx=0).stop 2024-04-17T10:38:43.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.860+0000 7fe8fed74640 1 -- 172.21.15.83:0/2057729092 shutdown_connections 2024-04-17T10:38:43.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.860+0000 7fe8fed74640 1 --2- 172.21.15.83:0/2057729092 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe8dc066f60 0x7fe8dc069420 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:43.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.860+0000 7fe8fed74640 1 --2- 172.21.15.83:0/2057729092 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8f815aa10 0x7fe8f80ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:43.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.860+0000 7fe8fed74640 1 --2- 172.21.15.83:0/2057729092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8f80cbca0 0x7fe8f80c2ce0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:43.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.860+0000 7fe8fed74640 1 --2- 172.21.15.83:0/2057729092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8f806c130 0x7fe8f809fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:43.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.860+0000 7fe8fed74640 1 -- 172.21.15.83:0/2057729092 >> 172.21.15.83:0/2057729092 conn(0x7fe8f80cc990 msgr2=0x7fe8f806ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:43.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.860+0000 7fe8fed74640 1 -- 172.21.15.83:0/2057729092 shutdown_connections 2024-04-17T10:38:43.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:43.860+0000 7fe8fed74640 1 -- 172.21.15.83:0/2057729092 wait complete. 2024-04-17T10:38:43.873 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:38:43.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.003+0000 7f3c39dd3640 1 Processor -- start 2024-04-17T10:38:43.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.003+0000 7f3c39dd3640 1 -- start start 2024-04-17T10:38:44.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.003+0000 7f3c39dd3640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3c3415e330 0x7f3c3415e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.003+0000 7f3c39dd3640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3c34070aa0 0x7f3c34070f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.003+0000 7f3c39dd3640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c341625d0 con 0x7f3c3406ae10 2024-04-17T10:38:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.003+0000 7f3c39dd3640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c34161810 con 0x7f3c34070aa0 2024-04-17T10:38:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.003+0000 7f3c39dd3640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c34161af0 con 0x7f3c3415e330 2024-04-17T10:38:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.003+0000 7f3c33fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3c34070aa0 0x7f3c34070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.003+0000 7f3c33fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3c34070aa0 0x7f3c34070f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52764/0 (socket says 172.21.15.83:52764) 2024-04-17T10:38:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.003+0000 7f3c33fff640 1 -- 172.21.15.83:0/2153550075 learned_addr learned my addr 172.21.15.83:0/2153550075 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.003+0000 7f3c32ffd640 1 --2- 172.21.15.83:0/2153550075 >> v2:172.21.15.167:3300/0 conn(0x7f3c3415e330 0x7f3c3415e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.003+0000 7f3c33fff640 1 -- 172.21.15.83:0/2153550075 >> v2:172.21.15.167:3300/0 conn(0x7f3c3415e330 msgr2=0x7f3c3415e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:44.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.003+0000 7f3c33fff640 1 --2- 172.21.15.83:0/2153550075 >> v2:172.21.15.167:3300/0 conn(0x7f3c3415e330 0x7f3c3415e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.003+0000 7f3c33fff640 1 -- 172.21.15.83:0/2153550075 >> v1:172.21.15.167:6789/0 conn(0x7f3c3406ae10 legacy=0x7f3c3406b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:44.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.003+0000 7f3c33fff640 1 -- 172.21.15.83:0/2153550075 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c34161dd0 con 0x7f3c34070aa0 2024-04-17T10:38:44.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c33fff640 1 --2- 172.21.15.83:0/2153550075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c34070aa0 0x7f3c34070f30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3c340b2480 tx=0x7f3c2c01c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64168ec279d2899 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:44.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c327fc640 1 -- 172.21.15.83:0/2153550075 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c0a5630 con 0x7f3c34070aa0 2024-04-17T10:38:44.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c327fc640 1 -- 172.21.15.83:0/2153550075 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2c0a57d0 con 0x7f3c34070aa0 2024-04-17T10:38:44.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c327fc640 1 -- 172.21.15.83:0/2153550075 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c0a5ba0 con 0x7f3c34070aa0 2024-04-17T10:38:44.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c39dd3640 1 -- 172.21.15.83:0/2153550075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c34070aa0 msgr2=0x7f3c34070f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:44.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c39dd3640 1 --2- 172.21.15.83:0/2153550075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c34070aa0 0x7f3c34070f30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3c340b2480 tx=0x7f3c2c01c860 comp rx=0 tx=0).stop 2024-04-17T10:38:44.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c39dd3640 1 -- 172.21.15.83:0/2153550075 shutdown_connections 2024-04-17T10:38:44.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c39dd3640 1 --2- 172.21.15.83:0/2153550075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c34070aa0 0x7f3c34070f30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c39dd3640 1 --2- 172.21.15.83:0/2153550075 >> v2:172.21.15.167:3300/0 conn(0x7f3c3415e330 0x7f3c3415e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c39dd3640 1 -- 172.21.15.83:0/2153550075 >> 172.21.15.83:0/2153550075 conn(0x7f3c340b15b0 msgr2=0x7f3c340b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:44.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c39dd3640 1 -- 172.21.15.83:0/2153550075 shutdown_connections 2024-04-17T10:38:44.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c39dd3640 1 -- 172.21.15.83:0/2153550075 wait complete. 2024-04-17T10:38:44.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c39dd3640 1 Processor -- start 2024-04-17T10:38:44.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c39dd3640 1 -- start start 2024-04-17T10:38:44.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c39dd3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c3406ae10 0x7f3c3406df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:44.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c39dd3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c34070aa0 0x7f3c341416b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:44.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c39dd3640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c3415e330 0x7f3c34141ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:44.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c39dd3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3416ab90 con 0x7f3c34070aa0 2024-04-17T10:38:44.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c39dd3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3416aa10 con 0x7f3c3406ae10 2024-04-17T10:38:44.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c39dd3640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3416ad10 con 0x7f3c3415e330 2024-04-17T10:38:44.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c32ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c34070aa0 0x7f3c341416b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:44.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c32ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c34070aa0 0x7f3c341416b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52774/0 (socket says 172.21.15.83:52774) 2024-04-17T10:38:44.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c32ffd640 1 -- 172.21.15.83:0/2135190855 learned_addr learned my addr 172.21.15.83:0/2135190855 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:44.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c33fff640 1 --2- 172.21.15.83:0/2135190855 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c3415e330 0x7f3c34141ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:44.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c337fe640 1 --2- 172.21.15.83:0/2135190855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c3406ae10 0x7f3c3406df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:44.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c32ffd640 1 -- 172.21.15.83:0/2135190855 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c3415e330 msgr2=0x7f3c34141ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:44.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c32ffd640 1 --2- 172.21.15.83:0/2135190855 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c3415e330 0x7f3c34141ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c32ffd640 1 -- 172.21.15.83:0/2135190855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c3406ae10 msgr2=0x7f3c3406df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:44.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c32ffd640 1 --2- 172.21.15.83:0/2135190855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c3406ae10 0x7f3c3406df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c32ffd640 1 -- 172.21.15.83:0/2135190855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c3415cd20 con 0x7f3c34070aa0 2024-04-17T10:38:44.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c33fff640 1 --2- 172.21.15.83:0/2135190855 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c3415e330 0x7f3c34141ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:44.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c32ffd640 1 --2- 172.21.15.83:0/2135190855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c34070aa0 0x7f3c341416b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3c2800d830 tx=0x7f3c2800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:44.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.007+0000 7f3c1bfff640 1 -- 172.21.15.83:0/2135190855 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2800b6c0 con 0x7f3c34070aa0 2024-04-17T10:38:44.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.011+0000 7f3c1bfff640 1 -- 172.21.15.83:0/2135190855 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c28019070 con 0x7f3c34070aa0 2024-04-17T10:38:44.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.011+0000 7f3c39dd3640 1 -- 172.21.15.83:0/2135190855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c3415d580 con 0x7f3c34070aa0 2024-04-17T10:38:44.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.011+0000 7f3c1bfff640 1 -- 172.21.15.83:0/2135190855 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2800ba60 con 0x7f3c34070aa0 2024-04-17T10:38:44.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.011+0000 7f3c39dd3640 1 -- 172.21.15.83:0/2135190855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c3415d2a0 con 0x7f3c34070aa0 2024-04-17T10:38:44.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.011+0000 7f3c39dd3640 1 -- 172.21.15.83:0/2135190855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c34074f90 con 0x7f3c34070aa0 2024-04-17T10:38:44.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.011+0000 7f3c1bfff640 1 -- 172.21.15.83:0/2135190855 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3c2801e460 con 0x7f3c34070aa0 2024-04-17T10:38:44.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.011+0000 7f3c1bfff640 1 --2- 172.21.15.83:0/2135190855 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3c20066f60 0x7f3c20069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:44.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.011+0000 7f3c337fe640 1 --2- 172.21.15.83:0/2135190855 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3c20066f60 0x7f3c20069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:44.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.011+0000 7f3c1bfff640 1 -- 172.21.15.83:0/2135190855 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c28085620 con 0x7f3c34070aa0 2024-04-17T10:38:44.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.011+0000 7f3c337fe640 1 --2- 172.21.15.83:0/2135190855 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3c20066f60 0x7f3c20069420 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3c2400a750 tx=0x7f3c24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:44.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.015+0000 7f3c1bfff640 1 -- 172.21.15.83:0/2135190855 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3c280566c0 con 0x7f3c34070aa0 2024-04-17T10:38:44.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.167+0000 7f3c39dd3640 1 -- 172.21.15.83:0/2135190855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3c340b0520 con 0x7f3c34070aa0 2024-04-17T10:38:44.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.167+0000 7f3c1bfff640 1 -- 172.21.15.83:0/2135190855 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3c28055fc0 con 0x7f3c34070aa0 2024-04-17T10:38:44.165 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:38:44.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.171+0000 7f3c39dd3640 1 -- 172.21.15.83:0/2135190855 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3c20066f60 msgr2=0x7f3c20069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:44.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.171+0000 7f3c39dd3640 1 --2- 172.21.15.83:0/2135190855 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3c20066f60 0x7f3c20069420 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3c2400a750 tx=0x7f3c24008040 comp rx=0 tx=0).stop 2024-04-17T10:38:44.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.171+0000 7f3c39dd3640 1 -- 172.21.15.83:0/2135190855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c34070aa0 msgr2=0x7f3c341416b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:44.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.171+0000 7f3c39dd3640 1 --2- 172.21.15.83:0/2135190855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c34070aa0 0x7f3c341416b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3c2800d830 tx=0x7f3c2800dcf0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.171+0000 7f3c39dd3640 1 -- 172.21.15.83:0/2135190855 shutdown_connections 2024-04-17T10:38:44.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.171+0000 7f3c39dd3640 1 --2- 172.21.15.83:0/2135190855 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3c20066f60 0x7f3c20069420 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.171+0000 7f3c39dd3640 1 --2- 172.21.15.83:0/2135190855 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c3415e330 0x7f3c34141ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.171+0000 7f3c39dd3640 1 --2- 172.21.15.83:0/2135190855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c34070aa0 0x7f3c341416b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.171+0000 7f3c39dd3640 1 --2- 172.21.15.83:0/2135190855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c3406ae10 0x7f3c3406df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.171+0000 7f3c39dd3640 1 -- 172.21.15.83:0/2135190855 >> 172.21.15.83:0/2135190855 conn(0x7f3c340b15b0 msgr2=0x7f3c34144470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:44.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.171+0000 7f3c39dd3640 1 -- 172.21.15.83:0/2135190855 shutdown_connections 2024-04-17T10:38:44.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.171+0000 7f3c39dd3640 1 -- 172.21.15.83:0/2135190855 wait complete. 2024-04-17T10:38:44.182 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:38:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.295+0000 7f01faed0640 1 Processor -- start 2024-04-17T10:38:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.295+0000 7f01faed0640 1 -- start start 2024-04-17T10:38:44.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.299+0000 7f01faed0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01f406b520 0x7f01f4071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:44.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.299+0000 7f01faed0640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f406c1f0 con 0x7f01f415b6e0 2024-04-17T10:38:44.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.299+0000 7f01faed0640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f4070a10 con 0x7f01f415bac0 2024-04-17T10:38:44.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.299+0000 7f01faed0640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f4070cf0 con 0x7f01f406b520 2024-04-17T10:38:44.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.299+0000 7f01f8c6c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f01f415b6e0 0x7f01f40cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47306/0 (socket says 172.21.15.83:47306) 2024-04-17T10:38:44.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.299+0000 7f01f8c6c640 1 -- 172.21.15.83:0/1729781252 learned_addr learned my addr 172.21.15.83:0/1729781252 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:44.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.299+0000 7f01f37fe640 1 -- 172.21.15.83:0/1729781252 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3655978527 0 0) 0x7f01f406c1f0 con 0x7f01f415b6e0 2024-04-17T10:38:44.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.299+0000 7f01f37fe640 1 -- 172.21.15.83:0/1729781252 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01dc003530 con 0x7f01f415b6e0 2024-04-17T10:38:44.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.299+0000 7f01f946d640 1 --2- 172.21.15.83:0/1729781252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01f406b520 0x7f01f4071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:44.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.299+0000 7f01f37fe640 1 -- 172.21.15.83:0/1729781252 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3431487630 0 0) 0x7f01dc003530 con 0x7f01f415b6e0 2024-04-17T10:38:44.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.299+0000 7f01f37fe640 1 -- 172.21.15.83:0/1729781252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01f406b520 msgr2=0x7f01f4071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:44.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.299+0000 7f01f37fe640 1 --2- 172.21.15.83:0/1729781252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01f406b520 0x7f01f4071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.299+0000 7f01f37fe640 1 -- 172.21.15.83:0/1729781252 >> v1:172.21.15.167:6789/0 conn(0x7f01f415bac0 legacy=0x7f01f406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:44.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.299+0000 7f01f37fe640 1 -- 172.21.15.83:0/1729781252 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f4070fd0 con 0x7f01f415b6e0 2024-04-17T10:38:44.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.299+0000 7f01f37fe640 1 -- 172.21.15.83:0/1729781252 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f01e4001680 con 0x7f01f415b6e0 2024-04-17T10:38:44.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.299+0000 7f01f37fe640 1 -- 172.21.15.83:0/1729781252 >> v1:172.21.15.83:6789/0 conn(0x7f01f415b6e0 legacy=0x7f01f40cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:44.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.299+0000 7f01f37fe640 1 --2- 172.21.15.83:0/1729781252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01dc003a60 0x7f01dc003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:44.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.299+0000 7f01f37fe640 1 --2- 172.21.15.83:0/1729781252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01f406b520 0x7f01dc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:44.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.299+0000 7f01f37fe640 1 --2- 172.21.15.83:0/1729781252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01dc004da0 0x7f01dc005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:44.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.299+0000 7f01f37fe640 1 -- 172.21.15.83:0/1729781252 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f4070cf0 con 0x7f01dc003a60 2024-04-17T10:38:44.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.299+0000 7f01f37fe640 1 -- 172.21.15.83:0/1729781252 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01dc00bbc0 con 0x7f01dc004da0 2024-04-17T10:38:44.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.299+0000 7f01f37fe640 1 -- 172.21.15.83:0/1729781252 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01dc00bdc0 con 0x7f01f406b520 2024-04-17T10:38:44.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.299+0000 7f01f8c6c640 1 --2- 172.21.15.83:0/1729781252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01dc003a60 0x7f01dc003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:44.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.299+0000 7f01f3fff640 1 --2- 172.21.15.83:0/1729781252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01f406b520 0x7f01dc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:44.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.299+0000 7f01f946d640 1 --2- 172.21.15.83:0/1729781252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01dc004da0 0x7f01dc005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:44.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.299+0000 7f01f8c6c640 1 -- 172.21.15.83:0/1729781252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01f406b520 msgr2=0x7f01dc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:44.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.299+0000 7f01f8c6c640 1 --2- 172.21.15.83:0/1729781252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01f406b520 0x7f01dc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.299+0000 7f01f8c6c640 1 -- 172.21.15.83:0/1729781252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01dc004da0 msgr2=0x7f01dc005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:44.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.299+0000 7f01f8c6c640 1 --2- 172.21.15.83:0/1729781252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01dc004da0 0x7f01dc005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.299+0000 7f01f8c6c640 1 -- 172.21.15.83:0/1729781252 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01dc003530 con 0x7f01dc003a60 2024-04-17T10:38:44.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.299+0000 7f01f946d640 1 --2- 172.21.15.83:0/1729781252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01dc004da0 0x7f01dc005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:38:44.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.303+0000 7f01f8c6c640 1 --2- 172.21.15.83:0/1729781252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01dc003a60 0x7f01dc003e80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f01e400c240 tx=0x7f01e401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a175c581701ecd18 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:44.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.303+0000 7f01f37fe640 1 -- 172.21.15.83:0/1729781252 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e4007d20 con 0x7f01dc003a60 2024-04-17T10:38:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.303+0000 7f01f37fe640 1 -- 172.21.15.83:0/1729781252 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e40a9070 con 0x7f01dc003a60 2024-04-17T10:38:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.303+0000 7f01f37fe640 1 -- 172.21.15.83:0/1729781252 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e4005d60 con 0x7f01dc003a60 2024-04-17T10:38:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.303+0000 7f01faed0640 1 -- 172.21.15.83:0/1729781252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01dc003a60 msgr2=0x7f01dc003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:44.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.303+0000 7f01faed0640 1 --2- 172.21.15.83:0/1729781252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01dc003a60 0x7f01dc003e80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f01e400c240 tx=0x7f01e401cc80 comp rx=0 tx=0).stop 2024-04-17T10:38:44.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.303+0000 7f01f946d640 1 -- 172.21.15.83:0/1729781252 reap_dead start 2024-04-17T10:38:44.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.303+0000 7f01faed0640 1 -- 172.21.15.83:0/1729781252 shutdown_connections 2024-04-17T10:38:44.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.303+0000 7f01faed0640 1 -- 172.21.15.83:0/1729781252 >> 172.21.15.83:0/1729781252 conn(0x7f01f40b15b0 msgr2=0x7f01f40b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:44.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.303+0000 7f01faed0640 1 -- 172.21.15.83:0/1729781252 shutdown_connections 2024-04-17T10:38:44.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.303+0000 7f01faed0640 1 -- 172.21.15.83:0/1729781252 wait complete. 2024-04-17T10:38:44.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.303+0000 7f01faed0640 1 Processor -- start 2024-04-17T10:38:44.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.303+0000 7f01faed0640 1 -- start start 2024-04-17T10:38:44.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.307+0000 7f01faed0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01dc003a60 0x7f01f4151060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:44.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.307+0000 7f01faed0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01f415cf20 0x7f01f413f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:44.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.307+0000 7f01faed0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01f4151d30 0x7f01f41521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:44.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.307+0000 7f01faed0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f406d2b0 con 0x7f01f4151d30 2024-04-17T10:38:44.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.307+0000 7f01faed0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f406d130 con 0x7f01f415cf20 2024-04-17T10:38:44.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.307+0000 7f01faed0640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f4167600 con 0x7f01dc003a60 2024-04-17T10:38:44.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.307+0000 7f01f946d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01f4151d30 0x7f01f41521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:44.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.307+0000 7f01f946d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01f4151d30 0x7f01f41521b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52794/0 (socket says 172.21.15.83:52794) 2024-04-17T10:38:44.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.307+0000 7f01f946d640 1 -- 172.21.15.83:0/1954871839 learned_addr learned my addr 172.21.15.83:0/1954871839 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:44.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.307+0000 7f01f8c6c640 1 --2- 172.21.15.83:0/1954871839 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01dc003a60 0x7f01f4151060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:44.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.307+0000 7f01f946d640 1 -- 172.21.15.83:0/1954871839 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01dc003a60 msgr2=0x7f01f4151060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:44.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.307+0000 7f01f946d640 1 --2- 172.21.15.83:0/1954871839 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01dc003a60 0x7f01f4151060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.307+0000 7f01f946d640 1 -- 172.21.15.83:0/1954871839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01f415cf20 msgr2=0x7f01f413f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:44.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.307+0000 7f01f3fff640 1 --2- 172.21.15.83:0/1954871839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01f415cf20 0x7f01f413f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:44.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.307+0000 7f01f946d640 1 --2- 172.21.15.83:0/1954871839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01f415cf20 0x7f01f413f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.307+0000 7f01f946d640 1 -- 172.21.15.83:0/1954871839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f4155660 con 0x7f01f4151d30 2024-04-17T10:38:44.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.307+0000 7f01f946d640 1 --2- 172.21.15.83:0/1954871839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01f4151d30 0x7f01f41521b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f01f40b2790 tx=0x7f01ec004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:44.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.307+0000 7f01f17fa640 1 -- 172.21.15.83:0/1954871839 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec007d60 con 0x7f01f4151d30 2024-04-17T10:38:44.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.307+0000 7f01f17fa640 1 -- 172.21.15.83:0/1954871839 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ec019070 con 0x7f01f4151d30 2024-04-17T10:38:44.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.307+0000 7f01faed0640 1 -- 172.21.15.83:0/1954871839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f4155940 con 0x7f01f4151d30 2024-04-17T10:38:44.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.307+0000 7f01f17fa640 1 -- 172.21.15.83:0/1954871839 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec014650 con 0x7f01f4151d30 2024-04-17T10:38:44.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.307+0000 7f01faed0640 1 -- 172.21.15.83:0/1954871839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f4155f00 con 0x7f01f4151d30 2024-04-17T10:38:44.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.307+0000 7f01faed0640 1 -- 172.21.15.83:0/1954871839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f4074f90 con 0x7f01f4151d30 2024-04-17T10:38:44.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.307+0000 7f01f17fa640 1 -- 172.21.15.83:0/1954871839 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f01ec0147f0 con 0x7f01f4151d30 2024-04-17T10:38:44.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.307+0000 7f01f17fa640 1 --2- 172.21.15.83:0/1954871839 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f01e0066f60 0x7f01e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:44.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.307+0000 7f01f17fa640 1 -- 172.21.15.83:0/1954871839 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01ec085580 con 0x7f01f4151d30 2024-04-17T10:38:44.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.307+0000 7f01f8c6c640 1 --2- 172.21.15.83:0/1954871839 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f01e0066f60 0x7f01e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:44.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.307+0000 7f01f8c6c640 1 --2- 172.21.15.83:0/1954871839 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f01e0066f60 0x7f01e0069420 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f01e40ab000 tx=0x7f01e40a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:44.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.311+0000 7f01f17fa640 1 -- 172.21.15.83:0/1954871839 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f01ec056590 con 0x7f01f4151d30 2024-04-17T10:38:44.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.467+0000 7f01faed0640 1 -- 172.21.15.83:0/1954871839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f01f406e390 con 0x7f01f4151d30 2024-04-17T10:38:44.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.467+0000 7f01f17fa640 1 -- 172.21.15.83:0/1954871839 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f01ec055e90 con 0x7f01f4151d30 2024-04-17T10:38:44.464 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:38:44.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.467+0000 7f01faed0640 1 -- 172.21.15.83:0/1954871839 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f01e0066f60 msgr2=0x7f01e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:44.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.467+0000 7f01faed0640 1 --2- 172.21.15.83:0/1954871839 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f01e0066f60 0x7f01e0069420 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f01e40ab000 tx=0x7f01e40a7070 comp rx=0 tx=0).stop 2024-04-17T10:38:44.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.467+0000 7f01faed0640 1 -- 172.21.15.83:0/1954871839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01f4151d30 msgr2=0x7f01f41521b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:44.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.467+0000 7f01faed0640 1 --2- 172.21.15.83:0/1954871839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01f4151d30 0x7f01f41521b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f01f40b2790 tx=0x7f01ec004840 comp rx=0 tx=0).stop 2024-04-17T10:38:44.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.467+0000 7f01faed0640 1 -- 172.21.15.83:0/1954871839 shutdown_connections 2024-04-17T10:38:44.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.467+0000 7f01faed0640 1 --2- 172.21.15.83:0/1954871839 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f01e0066f60 0x7f01e0069420 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.467+0000 7f01faed0640 1 --2- 172.21.15.83:0/1954871839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01f4151d30 0x7f01f41521b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.467+0000 7f01faed0640 1 --2- 172.21.15.83:0/1954871839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01f415cf20 0x7f01f413f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.467+0000 7f01faed0640 1 --2- 172.21.15.83:0/1954871839 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01dc003a60 0x7f01f4151060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.467+0000 7f01faed0640 1 -- 172.21.15.83:0/1954871839 >> 172.21.15.83:0/1954871839 conn(0x7f01f40b15b0 msgr2=0x7f01f406ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:44.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.467+0000 7f01faed0640 1 -- 172.21.15.83:0/1954871839 shutdown_connections 2024-04-17T10:38:44.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.467+0000 7f01faed0640 1 -- 172.21.15.83:0/1954871839 wait complete. 2024-04-17T10:38:44.492 INFO:tasks.cephfs_test_runner:Starting test: test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) test flag scrub_mdsdir 2024-04-17T10:38:44.492 INFO:tasks.cephfs_test_runner:test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-17T10:38:44.492 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:38:44.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.607+0000 7fe4469ce640 1 Processor -- start 2024-04-17T10:38:44.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.607+0000 7fe4469ce640 1 -- start start 2024-04-17T10:38:44.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.611+0000 7fe4469ce640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe44015ba30 0x7fe440072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:44.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.611+0000 7fe4469ce640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe44015c960 0x7fe4400cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:44.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.611+0000 7fe4469ce640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4400cfea0 con 0x7fe4400cfa10 2024-04-17T10:38:44.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.611+0000 7fe4469ce640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe440071ea0 con 0x7fe44015ba30 2024-04-17T10:38:44.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.611+0000 7fe4469ce640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe44006e320 con 0x7fe44015c960 2024-04-17T10:38:44.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.611+0000 7fe43ffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe44015ba30 0x7fe440072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:44.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.611+0000 7fe43ffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe44015ba30 0x7fe440072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52798/0 (socket says 172.21.15.83:52798) 2024-04-17T10:38:44.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.611+0000 7fe43ffff640 1 -- 172.21.15.83:0/3808233497 learned_addr learned my addr 172.21.15.83:0/3808233497 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:44.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.611+0000 7fe43f7fe640 1 --2- 172.21.15.83:0/3808233497 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe44015c960 0x7fe4400cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:44.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.611+0000 7fe43ffff640 1 -- 172.21.15.83:0/3808233497 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe44015c960 msgr2=0x7fe4400cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:44.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.611+0000 7fe43ffff640 1 --2- 172.21.15.83:0/3808233497 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe44015c960 0x7fe4400cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.611+0000 7fe43ffff640 1 -- 172.21.15.83:0/3808233497 >> v1:172.21.15.167:6789/0 conn(0x7fe4400cfa10 legacy=0x7fe44006c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:44.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.611+0000 7fe43ffff640 1 -- 172.21.15.83:0/3808233497 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe44006e040 con 0x7fe44015ba30 2024-04-17T10:38:44.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.611+0000 7fe43f7fe640 1 --2- 172.21.15.83:0/3808233497 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe44015c960 0x7fe4400cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:44.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.611+0000 7fe43ffff640 1 --2- 172.21.15.83:0/3808233497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe44015ba30 0x7fe440072330 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe430009e50 tx=0x7fe43001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7966541a749fa3ee server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:44.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.611+0000 7fe43effd640 1 -- 172.21.15.83:0/3808233497 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4300a56d0 con 0x7fe44015ba30 2024-04-17T10:38:44.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.611+0000 7fe43effd640 1 -- 172.21.15.83:0/3808233497 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4300a5870 con 0x7fe44015ba30 2024-04-17T10:38:44.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.611+0000 7fe43effd640 1 -- 172.21.15.83:0/3808233497 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4300a5c40 con 0x7fe44015ba30 2024-04-17T10:38:44.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe4469ce640 1 -- 172.21.15.83:0/3808233497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe44015ba30 msgr2=0x7fe440072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:44.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe4469ce640 1 --2- 172.21.15.83:0/3808233497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe44015ba30 0x7fe440072330 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe430009e50 tx=0x7fe43001c960 comp rx=0 tx=0).stop 2024-04-17T10:38:44.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe4469ce640 1 -- 172.21.15.83:0/3808233497 shutdown_connections 2024-04-17T10:38:44.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe4469ce640 1 --2- 172.21.15.83:0/3808233497 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe44015c960 0x7fe4400cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe4469ce640 1 --2- 172.21.15.83:0/3808233497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe44015ba30 0x7fe440072330 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe4469ce640 1 -- 172.21.15.83:0/3808233497 >> 172.21.15.83:0/3808233497 conn(0x7fe44006abe0 msgr2=0x7fe44006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe4469ce640 1 -- 172.21.15.83:0/3808233497 shutdown_connections 2024-04-17T10:38:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe4469ce640 1 -- 172.21.15.83:0/3808233497 wait complete. 2024-04-17T10:38:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe4469ce640 1 Processor -- start 2024-04-17T10:38:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe4469ce640 1 -- start start 2024-04-17T10:38:44.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe4469ce640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4400cfa10 0x7fe44014e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:44.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe4469ce640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe44015ba30 0x7fe4400ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:44.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe4469ce640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe44015c960 0x7fe4400add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:44.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe4469ce640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44006de20 con 0x7fe44015ba30 2024-04-17T10:38:44.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe4469ce640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44006dca0 con 0x7fe4400cfa10 2024-04-17T10:38:44.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe4469ce640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe44015cd40 con 0x7fe44015c960 2024-04-17T10:38:44.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe43f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe44015ba30 0x7fe4400ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:44.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe43f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe44015ba30 0x7fe4400ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52812/0 (socket says 172.21.15.83:52812) 2024-04-17T10:38:44.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe43f7fe640 1 -- 172.21.15.83:0/3161486387 learned_addr learned my addr 172.21.15.83:0/3161486387 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:44.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe444f6b640 1 --2- 172.21.15.83:0/3161486387 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe44015c960 0x7fe4400add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:44.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe43f7fe640 1 -- 172.21.15.83:0/3161486387 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe44015c960 msgr2=0x7fe4400add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:44.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe43f7fe640 1 --2- 172.21.15.83:0/3161486387 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe44015c960 0x7fe4400add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe43f7fe640 1 -- 172.21.15.83:0/3161486387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4400cfa10 msgr2=0x7fe44014e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:44.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe43ffff640 1 --2- 172.21.15.83:0/3161486387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4400cfa10 0x7fe44014e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:44.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe43f7fe640 1 --2- 172.21.15.83:0/3161486387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4400cfa10 0x7fe44014e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe43f7fe640 1 -- 172.21.15.83:0/3161486387 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4400b3410 con 0x7fe44015ba30 2024-04-17T10:38:44.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe43f7fe640 1 --2- 172.21.15.83:0/3161486387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe44015ba30 0x7fe4400ad840 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe42800c7c0 tx=0x7fe42800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:44.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe43cff9640 1 -- 172.21.15.83:0/3161486387 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe428007a30 con 0x7fe44015ba30 2024-04-17T10:38:44.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe43cff9640 1 -- 172.21.15.83:0/3161486387 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe428007bd0 con 0x7fe44015ba30 2024-04-17T10:38:44.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe4469ce640 1 -- 172.21.15.83:0/3161486387 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4400b36f0 con 0x7fe44015ba30 2024-04-17T10:38:44.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe43cff9640 1 -- 172.21.15.83:0/3161486387 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4280113f0 con 0x7fe44015ba30 2024-04-17T10:38:44.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe4469ce640 1 -- 172.21.15.83:0/3161486387 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4400b3cb0 con 0x7fe44015ba30 2024-04-17T10:38:44.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.615+0000 7fe4469ce640 1 -- 172.21.15.83:0/3161486387 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe440074f90 con 0x7fe44015ba30 2024-04-17T10:38:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.619+0000 7fe43cff9640 1 -- 172.21.15.83:0/3161486387 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe428011590 con 0x7fe44015ba30 2024-04-17T10:38:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.619+0000 7fe43cff9640 1 --2- 172.21.15.83:0/3161486387 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe420066f60 0x7fe420069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.619+0000 7fe43cff9640 1 -- 172.21.15.83:0/3161486387 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4280851e0 con 0x7fe44015ba30 2024-04-17T10:38:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.619+0000 7fe43ffff640 1 --2- 172.21.15.83:0/3161486387 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe420066f60 0x7fe420069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.619+0000 7fe43ffff640 1 --2- 172.21.15.83:0/3161486387 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe420066f60 0x7fe420069420 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe4300ae000 tx=0x7fe430004ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:44.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.623+0000 7fe43cff9640 1 -- 172.21.15.83:0/3161486387 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe428015020 con 0x7fe44015ba30 2024-04-17T10:38:44.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.775+0000 7fe4469ce640 1 -- 172.21.15.83:0/3161486387 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe440074480 con 0x7fe44015ba30 2024-04-17T10:38:44.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.775+0000 7fe43cff9640 1 -- 172.21.15.83:0/3161486387 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16533 (secure 0 0 0) 0x7fe428056280 con 0x7fe44015ba30 2024-04-17T10:38:44.774 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:44.774 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":20,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:38:13.690876+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:38:10.464490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T10:38:11.043134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:38:44.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.779+0000 7fe4469ce640 1 -- 172.21.15.83:0/3161486387 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe420066f60 msgr2=0x7fe420069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:44.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.779+0000 7fe4469ce640 1 --2- 172.21.15.83:0/3161486387 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe420066f60 0x7fe420069420 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe4300ae000 tx=0x7fe430004ac0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.779+0000 7fe4469ce640 1 -- 172.21.15.83:0/3161486387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe44015ba30 msgr2=0x7fe4400ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:44.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.779+0000 7fe4469ce640 1 --2- 172.21.15.83:0/3161486387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe44015ba30 0x7fe4400ad840 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe42800c7c0 tx=0x7fe42800cc80 comp rx=0 tx=0).stop 2024-04-17T10:38:44.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.779+0000 7fe4469ce640 1 -- 172.21.15.83:0/3161486387 shutdown_connections 2024-04-17T10:38:44.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.779+0000 7fe4469ce640 1 --2- 172.21.15.83:0/3161486387 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe420066f60 0x7fe420069420 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.779+0000 7fe4469ce640 1 --2- 172.21.15.83:0/3161486387 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe44015c960 0x7fe4400add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.779+0000 7fe4469ce640 1 --2- 172.21.15.83:0/3161486387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe44015ba30 0x7fe4400ad840 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.779+0000 7fe4469ce640 1 --2- 172.21.15.83:0/3161486387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4400cfa10 0x7fe44014e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.779+0000 7fe4469ce640 1 -- 172.21.15.83:0/3161486387 >> 172.21.15.83:0/3161486387 conn(0x7fe44006abe0 msgr2=0x7fe44015d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:44.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.779+0000 7fe4469ce640 1 -- 172.21.15.83:0/3161486387 shutdown_connections 2024-04-17T10:38:44.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.779+0000 7fe4469ce640 1 -- 172.21.15.83:0/3161486387 wait complete. 2024-04-17T10:38:44.792 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:38:10.464490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:38:11.043134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:38:44.793 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:38:44.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.923+0000 7ff848818640 1 Processor -- start 2024-04-17T10:38:44.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.923+0000 7ff848818640 1 -- start start 2024-04-17T10:38:44.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.923+0000 7ff848818640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff84415b6e0 0x7ff844071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:44.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.923+0000 7ff848818640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff84415bac0 0x7ff8440cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:44.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.923+0000 7ff848818640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff84406c1f0 con 0x7ff844070470 2024-04-17T10:38:44.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.923+0000 7ff848818640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8440cdd30 con 0x7ff84415b6e0 2024-04-17T10:38:44.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.923+0000 7ff848818640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff844070a10 con 0x7ff84415bac0 2024-04-17T10:38:44.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.923+0000 7ff84259c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff84415b6e0 0x7ff844071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:44.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.923+0000 7ff84259c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff84415b6e0 0x7ff844071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52824/0 (socket says 172.21.15.83:52824) 2024-04-17T10:38:44.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.923+0000 7ff842d9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff844070470 0x7ff84406ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47318/0 (socket says 172.21.15.83:47318) 2024-04-17T10:38:44.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.923+0000 7ff84259c640 1 -- 172.21.15.83:0/3777878402 learned_addr learned my addr 172.21.15.83:0/3777878402 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:44.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.923+0000 7ff841d9b640 1 --2- 172.21.15.83:0/3777878402 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff84415bac0 0x7ff8440cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:44.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.923+0000 7ff84259c640 1 -- 172.21.15.83:0/3777878402 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff84415bac0 msgr2=0x7ff8440cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:44.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.923+0000 7ff84259c640 1 --2- 172.21.15.83:0/3777878402 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff84415bac0 0x7ff8440cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.923+0000 7ff84259c640 1 -- 172.21.15.83:0/3777878402 >> v1:172.21.15.83:6789/0 conn(0x7ff844070470 legacy=0x7ff84406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:44.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.923+0000 7ff84259c640 1 -- 172.21.15.83:0/3777878402 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff844070cf0 con 0x7ff84415b6e0 2024-04-17T10:38:44.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.923+0000 7ff84159a640 1 -- 172.21.15.83:0/3777878402 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 640860536 0 0) 0x7ff84406c1f0 con 0x7ff844070470 2024-04-17T10:38:44.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff84259c640 1 --2- 172.21.15.83:0/3777878402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff84415b6e0 0x7ff844071cc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff834009870 tx=0x7ff83401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7754fbaafd9ed3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:44.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff84159a640 1 -- 172.21.15.83:0/3777878402 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8340b2070 con 0x7ff84415b6e0 2024-04-17T10:38:44.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff84159a640 1 -- 172.21.15.83:0/3777878402 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8340a9070 con 0x7ff84415b6e0 2024-04-17T10:38:44.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff84159a640 1 -- 172.21.15.83:0/3777878402 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8340ad880 con 0x7ff84415b6e0 2024-04-17T10:38:44.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff848818640 1 -- 172.21.15.83:0/3777878402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff84415b6e0 msgr2=0x7ff844071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:44.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff848818640 1 --2- 172.21.15.83:0/3777878402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff84415b6e0 0x7ff844071cc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff834009870 tx=0x7ff83401c560 comp rx=0 tx=0).stop 2024-04-17T10:38:44.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff848818640 1 -- 172.21.15.83:0/3777878402 shutdown_connections 2024-04-17T10:38:44.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff848818640 1 --2- 172.21.15.83:0/3777878402 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff84415bac0 0x7ff8440cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff848818640 1 --2- 172.21.15.83:0/3777878402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff84415b6e0 0x7ff844071cc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff848818640 1 -- 172.21.15.83:0/3777878402 >> 172.21.15.83:0/3777878402 conn(0x7ff8440b15b0 msgr2=0x7ff8440b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:44.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff848818640 1 -- 172.21.15.83:0/3777878402 shutdown_connections 2024-04-17T10:38:44.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff848818640 1 -- 172.21.15.83:0/3777878402 wait complete. 2024-04-17T10:38:44.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff848818640 1 Processor -- start 2024-04-17T10:38:44.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff848818640 1 -- start start 2024-04-17T10:38:44.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff848818640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff844070470 0x7ff844147d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:44.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff848818640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff84415b6e0 0x7ff8440cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:44.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff848818640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff84415bac0 0x7ff8440ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:44.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff848818640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84406cee0 con 0x7ff84415bac0 2024-04-17T10:38:44.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff848818640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84406b730 con 0x7ff84415b6e0 2024-04-17T10:38:44.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff848818640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff84406d060 con 0x7ff844070470 2024-04-17T10:38:44.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff842d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff84415bac0 0x7ff8440ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:44.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff842d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff84415bac0 0x7ff8440ce3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52826/0 (socket says 172.21.15.83:52826) 2024-04-17T10:38:44.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff842d9d640 1 -- 172.21.15.83:0/3485871336 learned_addr learned my addr 172.21.15.83:0/3485871336 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:44.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff84259c640 1 --2- 172.21.15.83:0/3485871336 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff844070470 0x7ff844147d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:44.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff842d9d640 1 -- 172.21.15.83:0/3485871336 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff844070470 msgr2=0x7ff844147d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:44.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff842d9d640 1 --2- 172.21.15.83:0/3485871336 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff844070470 0x7ff844147d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff842d9d640 1 -- 172.21.15.83:0/3485871336 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff84415b6e0 msgr2=0x7ff8440cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:44.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff841d9b640 1 --2- 172.21.15.83:0/3485871336 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff84415b6e0 0x7ff8440cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:44.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff842d9d640 1 --2- 172.21.15.83:0/3485871336 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff84415b6e0 0x7ff8440cdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:44.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff842d9d640 1 -- 172.21.15.83:0/3485871336 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff84414a010 con 0x7ff84415bac0 2024-04-17T10:38:44.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff842d9d640 1 --2- 172.21.15.83:0/3485871336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff84415bac0 0x7ff8440ce3f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00c7e0 tx=0x7ff83c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:44.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff832ffd640 1 -- 172.21.15.83:0/3485871336 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83c017070 con 0x7ff84415bac0 2024-04-17T10:38:44.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff832ffd640 1 -- 172.21.15.83:0/3485871336 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff83c00a640 con 0x7ff84415bac0 2024-04-17T10:38:44.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.927+0000 7ff832ffd640 1 -- 172.21.15.83:0/3485871336 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83c00aa10 con 0x7ff84415bac0 2024-04-17T10:38:44.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.931+0000 7ff848818640 1 -- 172.21.15.83:0/3485871336 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff84414a2f0 con 0x7ff84415bac0 2024-04-17T10:38:44.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.931+0000 7ff848818640 1 -- 172.21.15.83:0/3485871336 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff84414a8b0 con 0x7ff84415bac0 2024-04-17T10:38:44.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.931+0000 7ff848818640 1 -- 172.21.15.83:0/3485871336 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff844074f90 con 0x7ff84415bac0 2024-04-17T10:38:44.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.931+0000 7ff832ffd640 1 -- 172.21.15.83:0/3485871336 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff83c01a070 con 0x7ff84415bac0 2024-04-17T10:38:44.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.931+0000 7ff832ffd640 1 --2- 172.21.15.83:0/3485871336 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff824066f60 0x7ff824069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:44.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.931+0000 7ff832ffd640 1 -- 172.21.15.83:0/3485871336 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff83c0846c0 con 0x7ff84415bac0 2024-04-17T10:38:44.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.931+0000 7ff84259c640 1 --2- 172.21.15.83:0/3485871336 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff824066f60 0x7ff824069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:44.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.931+0000 7ff84259c640 1 --2- 172.21.15.83:0/3485871336 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff824066f60 0x7ff824069420 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff8340af000 tx=0x7ff834008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:44.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:44.935+0000 7ff832ffd640 1 -- 172.21.15.83:0/3485871336 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff83c0556d0 con 0x7ff84415bac0 2024-04-17T10:38:45.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.091+0000 7ff848818640 1 -- 172.21.15.83:0/3485871336 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff8440b0d20 con 0x7ff84415bac0 2024-04-17T10:38:45.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.095+0000 7ff832ffd640 1 -- 172.21.15.83:0/3485871336 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff83c054fd0 con 0x7ff84415bac0 2024-04-17T10:38:45.091 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:38:45.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.095+0000 7ff848818640 1 -- 172.21.15.83:0/3485871336 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff824066f60 msgr2=0x7ff824069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:45.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.095+0000 7ff848818640 1 --2- 172.21.15.83:0/3485871336 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff824066f60 0x7ff824069420 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff8340af000 tx=0x7ff834008040 comp rx=0 tx=0).stop 2024-04-17T10:38:45.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.095+0000 7ff848818640 1 -- 172.21.15.83:0/3485871336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff84415bac0 msgr2=0x7ff8440ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:45.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.095+0000 7ff848818640 1 --2- 172.21.15.83:0/3485871336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff84415bac0 0x7ff8440ce3f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00c7e0 tx=0x7ff83c00cca0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.095+0000 7ff848818640 1 -- 172.21.15.83:0/3485871336 shutdown_connections 2024-04-17T10:38:45.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.095+0000 7ff848818640 1 --2- 172.21.15.83:0/3485871336 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff824066f60 0x7ff824069420 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.095+0000 7ff848818640 1 --2- 172.21.15.83:0/3485871336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff84415bac0 0x7ff8440ce3f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.095+0000 7ff848818640 1 --2- 172.21.15.83:0/3485871336 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff84415b6e0 0x7ff8440cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.095+0000 7ff848818640 1 --2- 172.21.15.83:0/3485871336 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff844070470 0x7ff844147d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.095+0000 7ff848818640 1 -- 172.21.15.83:0/3485871336 >> 172.21.15.83:0/3485871336 conn(0x7ff8440b15b0 msgr2=0x7ff84406ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:45.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.095+0000 7ff848818640 1 -- 172.21.15.83:0/3485871336 shutdown_connections 2024-04-17T10:38:45.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.095+0000 7ff848818640 1 -- 172.21.15.83:0/3485871336 wait complete. 2024-04-17T10:38:45.110 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:38:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.235+0000 7f73a56c5640 1 Processor -- start 2024-04-17T10:38:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.235+0000 7f73a56c5640 1 -- start start 2024-04-17T10:38:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f73a56c5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f73a015b6e0 0x7f73a0071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f73a56c5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73a015bac0 0x7f73a00cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f73a56c5640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73a006c1f0 con 0x7f73a0070470 2024-04-17T10:38:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f73a56c5640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73a00cdd30 con 0x7f73a015b6e0 2024-04-17T10:38:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f73a56c5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73a0070a10 con 0x7f73a015bac0 2024-04-17T10:38:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f739f7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f73a0070470 0x7f73a006ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47322/0 (socket says 172.21.15.83:47322) 2024-04-17T10:38:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f739f7fe640 1 -- 172.21.15.83:0/2111883562 learned_addr learned my addr 172.21.15.83:0/2111883562 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f739e7fc640 1 --2- 172.21.15.83:0/2111883562 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73a015bac0 0x7f73a00cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f739dffb640 1 -- 172.21.15.83:0/2111883562 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2095143455 0 0) 0x7f73a006c1f0 con 0x7f73a0070470 2024-04-17T10:38:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f739dffb640 1 -- 172.21.15.83:0/2111883562 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f738c003530 con 0x7f73a0070470 2024-04-17T10:38:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f739effd640 1 --2- 172.21.15.83:0/2111883562 >> v2:172.21.15.167:3300/0 conn(0x7f73a015b6e0 0x7f73a0071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f739dffb640 1 -- 172.21.15.83:0/2111883562 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3805707797 0 0) 0x7f738c003530 con 0x7f73a0070470 2024-04-17T10:38:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f739dffb640 1 -- 172.21.15.83:0/2111883562 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73a015bac0 msgr2=0x7f73a00cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f739dffb640 1 --2- 172.21.15.83:0/2111883562 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73a015bac0 0x7f73a00cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f739dffb640 1 -- 172.21.15.83:0/2111883562 >> v2:172.21.15.167:3300/0 conn(0x7f73a015b6e0 msgr2=0x7f73a0071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f739dffb640 1 --2- 172.21.15.83:0/2111883562 >> v2:172.21.15.167:3300/0 conn(0x7f73a015b6e0 0x7f73a0071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f739dffb640 1 -- 172.21.15.83:0/2111883562 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a0070cf0 con 0x7f73a0070470 2024-04-17T10:38:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f739dffb640 1 -- 172.21.15.83:0/2111883562 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f7398001a20 con 0x7f73a0070470 2024-04-17T10:38:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f739dffb640 1 -- 172.21.15.83:0/2111883562 >> v1:172.21.15.83:6789/0 conn(0x7f73a0070470 legacy=0x7f73a006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f739dffb640 1 --2- 172.21.15.83:0/2111883562 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73a015bac0 0x7f738c003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f739dffb640 1 --2- 172.21.15.83:0/2111883562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f738c004060 0x7f738c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f739dffb640 1 --2- 172.21.15.83:0/2111883562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c004d50 0x7f738c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f739dffb640 1 -- 172.21.15.83:0/2111883562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a00cdd30 con 0x7f738c004d50 2024-04-17T10:38:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f739dffb640 1 -- 172.21.15.83:0/2111883562 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a0070a10 con 0x7f738c004060 2024-04-17T10:38:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f739dffb640 1 -- 172.21.15.83:0/2111883562 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f738c00bb60 con 0x7f73a015bac0 2024-04-17T10:38:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f739f7fe640 1 --2- 172.21.15.83:0/2111883562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c004d50 0x7f738c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f739effd640 1 --2- 172.21.15.83:0/2111883562 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73a015bac0 0x7f738c003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f739e7fc640 1 --2- 172.21.15.83:0/2111883562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f738c004060 0x7f738c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f739f7fe640 1 -- 172.21.15.83:0/2111883562 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73a015bac0 msgr2=0x7f738c003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f739f7fe640 1 --2- 172.21.15.83:0/2111883562 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73a015bac0 0x7f738c003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f739f7fe640 1 -- 172.21.15.83:0/2111883562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f738c004060 msgr2=0x7f738c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f739f7fe640 1 --2- 172.21.15.83:0/2111883562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f738c004060 0x7f738c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.239+0000 7f739f7fe640 1 -- 172.21.15.83:0/2111883562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f738c003530 con 0x7f738c004d50 2024-04-17T10:38:45.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.243+0000 7f739f7fe640 1 --2- 172.21.15.83:0/2111883562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c004d50 0x7f738c0051d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f73a006c3c0 tx=0x7f739801d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb94d3e2019bbdf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:45.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.243+0000 7f739dffb640 1 -- 172.21.15.83:0/2111883562 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739800a4e0 con 0x7f738c004d50 2024-04-17T10:38:45.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.243+0000 7f739dffb640 1 -- 172.21.15.83:0/2111883562 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73980072f0 con 0x7f738c004d50 2024-04-17T10:38:45.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.243+0000 7f739dffb640 1 -- 172.21.15.83:0/2111883562 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73980076e0 con 0x7f738c004d50 2024-04-17T10:38:45.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.243+0000 7f73a56c5640 1 -- 172.21.15.83:0/2111883562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c004d50 msgr2=0x7f738c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:45.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.243+0000 7f73a56c5640 1 --2- 172.21.15.83:0/2111883562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c004d50 0x7f738c0051d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f73a006c3c0 tx=0x7f739801d380 comp rx=0 tx=0).stop 2024-04-17T10:38:45.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.243+0000 7f739f7fe640 1 -- 172.21.15.83:0/2111883562 reap_dead start 2024-04-17T10:38:45.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.243+0000 7f73a56c5640 1 -- 172.21.15.83:0/2111883562 shutdown_connections 2024-04-17T10:38:45.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.243+0000 7f73a56c5640 1 -- 172.21.15.83:0/2111883562 >> 172.21.15.83:0/2111883562 conn(0x7f73a00b15b0 msgr2=0x7f73a00b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:45.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.243+0000 7f73a56c5640 1 -- 172.21.15.83:0/2111883562 shutdown_connections 2024-04-17T10:38:45.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.243+0000 7f73a56c5640 1 -- 172.21.15.83:0/2111883562 wait complete. 2024-04-17T10:38:45.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.243+0000 7f73a56c5640 1 Processor -- start 2024-04-17T10:38:45.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.243+0000 7f73a56c5640 1 -- start start 2024-04-17T10:38:45.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.243+0000 7f73a56c5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c004d50 0x7f73a01f0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:45.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.243+0000 7f73a56c5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73a00aa230 0x7f73a00aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:45.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.243+0000 7f73a56c5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73a015e4f0 0x7f73a015e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:45.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.243+0000 7f73a56c5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a006b520 con 0x7f738c004d50 2024-04-17T10:38:45.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.243+0000 7f73a56c5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a00712b0 con 0x7f73a015e4f0 2024-04-17T10:38:45.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.243+0000 7f73a56c5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73a006b6a0 con 0x7f73a00aa230 2024-04-17T10:38:45.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.243+0000 7f739effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c004d50 0x7f73a01f0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:45.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.243+0000 7f739effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c004d50 0x7f73a01f0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52836/0 (socket says 172.21.15.83:52836) 2024-04-17T10:38:45.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.243+0000 7f739effd640 1 -- 172.21.15.83:0/1556775162 learned_addr learned my addr 172.21.15.83:0/1556775162 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:45.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.243+0000 7f739e7fc640 1 --2- 172.21.15.83:0/1556775162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73a00aa230 0x7f73a00aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:45.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.243+0000 7f739f7fe640 1 --2- 172.21.15.83:0/1556775162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73a015e4f0 0x7f73a015e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:45.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.243+0000 7f739effd640 1 -- 172.21.15.83:0/1556775162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73a00aa230 msgr2=0x7f73a00aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:45.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.243+0000 7f739effd640 1 --2- 172.21.15.83:0/1556775162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73a00aa230 0x7f73a00aa6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.243+0000 7f739effd640 1 -- 172.21.15.83:0/1556775162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73a015e4f0 msgr2=0x7f73a015e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:45.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.243+0000 7f739effd640 1 --2- 172.21.15.83:0/1556775162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73a015e4f0 0x7f73a015e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.243+0000 7f739effd640 1 -- 172.21.15.83:0/1556775162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a015eee0 con 0x7f738c004d50 2024-04-17T10:38:45.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.243+0000 7f739effd640 1 --2- 172.21.15.83:0/1556775162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c004d50 0x7f73a01f0aa0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f739000f600 tx=0x7f739000fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:45.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.247+0000 7f73977fe640 1 -- 172.21.15.83:0/1556775162 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7390019020 con 0x7f738c004d50 2024-04-17T10:38:45.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.247+0000 7f73977fe640 1 -- 172.21.15.83:0/1556775162 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7390014870 con 0x7f738c004d50 2024-04-17T10:38:45.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.247+0000 7f73a56c5640 1 -- 172.21.15.83:0/1556775162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a015f170 con 0x7f738c004d50 2024-04-17T10:38:45.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.247+0000 7f73977fe640 1 -- 172.21.15.83:0/1556775162 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7390014c40 con 0x7f738c004d50 2024-04-17T10:38:45.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.247+0000 7f73a56c5640 1 -- 172.21.15.83:0/1556775162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a015f730 con 0x7f738c004d50 2024-04-17T10:38:45.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.247+0000 7f73a56c5640 1 -- 172.21.15.83:0/1556775162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73a0074f90 con 0x7f738c004d50 2024-04-17T10:38:45.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.247+0000 7f73977fe640 1 -- 172.21.15.83:0/1556775162 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f739001c020 con 0x7f738c004d50 2024-04-17T10:38:45.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.247+0000 7f73977fe640 1 --2- 172.21.15.83:0/1556775162 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7380066f90 0x7f7380069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:45.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.247+0000 7f73977fe640 1 -- 172.21.15.83:0/1556775162 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f73900854e0 con 0x7f738c004d50 2024-04-17T10:38:45.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.247+0000 7f739e7fc640 1 --2- 172.21.15.83:0/1556775162 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7380066f90 0x7f7380069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:45.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.247+0000 7f739e7fc640 1 --2- 172.21.15.83:0/1556775162 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7380066f90 0x7f7380069450 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f73a015b100 tx=0x7f7388008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:45.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.251+0000 7f73977fe640 1 -- 172.21.15.83:0/1556775162 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f73900564f0 con 0x7f738c004d50 2024-04-17T10:38:45.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.407+0000 7f73a56c5640 1 -- 172.21.15.83:0/1556775162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f73a0071700 con 0x7f738c004d50 2024-04-17T10:38:45.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.411+0000 7f73977fe640 1 -- 172.21.15.83:0/1556775162 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7390055df0 con 0x7f738c004d50 2024-04-17T10:38:45.407 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:38:45.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.411+0000 7f73a56c5640 1 -- 172.21.15.83:0/1556775162 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7380066f90 msgr2=0x7f7380069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:45.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.411+0000 7f73a56c5640 1 --2- 172.21.15.83:0/1556775162 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7380066f90 0x7f7380069450 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f73a015b100 tx=0x7f7388008040 comp rx=0 tx=0).stop 2024-04-17T10:38:45.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.411+0000 7f73a56c5640 1 -- 172.21.15.83:0/1556775162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c004d50 msgr2=0x7f73a01f0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:45.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.411+0000 7f73a56c5640 1 --2- 172.21.15.83:0/1556775162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c004d50 0x7f73a01f0aa0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f739000f600 tx=0x7f739000fac0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.411+0000 7f73a56c5640 1 -- 172.21.15.83:0/1556775162 shutdown_connections 2024-04-17T10:38:45.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.411+0000 7f73a56c5640 1 --2- 172.21.15.83:0/1556775162 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7380066f90 0x7f7380069450 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.411+0000 7f73a56c5640 1 --2- 172.21.15.83:0/1556775162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73a015e4f0 0x7f73a015e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.411+0000 7f73a56c5640 1 --2- 172.21.15.83:0/1556775162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73a00aa230 0x7f73a00aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.411+0000 7f73a56c5640 1 --2- 172.21.15.83:0/1556775162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c004d50 0x7f73a01f0aa0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.411+0000 7f73a56c5640 1 -- 172.21.15.83:0/1556775162 >> 172.21.15.83:0/1556775162 conn(0x7f73a00b15b0 msgr2=0x7f73a00cfd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:45.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.411+0000 7f73a56c5640 1 -- 172.21.15.83:0/1556775162 shutdown_connections 2024-04-17T10:38:45.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.411+0000 7f73a56c5640 1 -- 172.21.15.83:0/1556775162 wait complete. 2024-04-17T10:38:45.427 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:38:45.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.551+0000 7fa296002640 1 Processor -- start 2024-04-17T10:38:45.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.551+0000 7fa296002640 1 -- start start 2024-04-17T10:38:45.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.551+0000 7fa296002640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa29015dab0 0x7fa290161840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:45.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.551+0000 7fa296002640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa29015de90 0x7fa290161d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:45.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.551+0000 7fa296002640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2900ce4e0 con 0x7fa2901622c0 2024-04-17T10:38:45.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.551+0000 7fa296002640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2900cf370 con 0x7fa29015de90 2024-04-17T10:38:45.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.551+0000 7fa296002640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2900d4910 con 0x7fa29015dab0 2024-04-17T10:38:45.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.551+0000 7fa286ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa29015de90 0x7fa290161d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:45.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.551+0000 7fa28ffff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa2901622c0 0x7fa2900d15a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47330/0 (socket says 172.21.15.83:47330) 2024-04-17T10:38:45.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.551+0000 7fa28ffff640 1 -- 172.21.15.83:0/2128666339 learned_addr learned my addr 172.21.15.83:0/2128666339 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:45.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.551+0000 7fa28f7fe640 1 --2- 172.21.15.83:0/2128666339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa29015dab0 0x7fa290161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:45.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.551+0000 7fa286ffd640 1 -- 172.21.15.83:0/2128666339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa29015dab0 msgr2=0x7fa290161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:45.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.551+0000 7fa286ffd640 1 --2- 172.21.15.83:0/2128666339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa29015dab0 0x7fa290161840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.551+0000 7fa286ffd640 1 -- 172.21.15.83:0/2128666339 >> v1:172.21.15.83:6789/0 conn(0x7fa2901622c0 legacy=0x7fa2900d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:45.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.551+0000 7fa286ffd640 1 -- 172.21.15.83:0/2128666339 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2900ce760 con 0x7fa29015de90 2024-04-17T10:38:45.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.551+0000 7fa28effd640 1 -- 172.21.15.83:0/2128666339 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3908915258 0 0) 0x7fa2880014a0 con 0x7fa2901622c0 2024-04-17T10:38:45.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa286ffd640 1 --2- 172.21.15.83:0/2128666339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa29015de90 0x7fa290161d80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa2780097f0 tx=0x7fa27801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17652d3c896562a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:45.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa28effd640 1 -- 172.21.15.83:0/2128666339 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2780b0070 con 0x7fa29015de90 2024-04-17T10:38:45.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa28effd640 1 -- 172.21.15.83:0/2128666339 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2780ab600 con 0x7fa29015de90 2024-04-17T10:38:45.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa28effd640 1 -- 172.21.15.83:0/2128666339 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2780ab9d0 con 0x7fa29015de90 2024-04-17T10:38:45.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa296002640 1 -- 172.21.15.83:0/2128666339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa29015de90 msgr2=0x7fa290161d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:45.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa296002640 1 --2- 172.21.15.83:0/2128666339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa29015de90 0x7fa290161d80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa2780097f0 tx=0x7fa27801c4a0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa296002640 1 -- 172.21.15.83:0/2128666339 shutdown_connections 2024-04-17T10:38:45.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa296002640 1 --2- 172.21.15.83:0/2128666339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa29015de90 0x7fa290161d80 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa296002640 1 --2- 172.21.15.83:0/2128666339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa29015dab0 0x7fa290161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa296002640 1 -- 172.21.15.83:0/2128666339 >> 172.21.15.83:0/2128666339 conn(0x7fa29009cd70 msgr2=0x7fa2900d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:45.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa296002640 1 -- 172.21.15.83:0/2128666339 shutdown_connections 2024-04-17T10:38:45.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa296002640 1 -- 172.21.15.83:0/2128666339 wait complete. 2024-04-17T10:38:45.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa296002640 1 Processor -- start 2024-04-17T10:38:45.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa296002640 1 -- start start 2024-04-17T10:38:45.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa296002640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa29015dab0 0x7fa2900caf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:45.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa296002640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa29015de90 0x7fa29015b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:45.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa296002640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2901622c0 0x7fa2900cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:45.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa296002640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2900cf960 con 0x7fa29015de90 2024-04-17T10:38:45.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa296002640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2900cf7e0 con 0x7fa2901622c0 2024-04-17T10:38:45.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa296002640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2900d24c0 con 0x7fa29015dab0 2024-04-17T10:38:45.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa286ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa29015de90 0x7fa29015b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:45.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa286ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa29015de90 0x7fa29015b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52850/0 (socket says 172.21.15.83:52850) 2024-04-17T10:38:45.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa286ffd640 1 -- 172.21.15.83:0/1367044227 learned_addr learned my addr 172.21.15.83:0/1367044227 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:45.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa28f7fe640 1 --2- 172.21.15.83:0/1367044227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa29015dab0 0x7fa2900caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:45.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa286ffd640 1 -- 172.21.15.83:0/1367044227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa29015dab0 msgr2=0x7fa2900caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:45.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa286ffd640 1 --2- 172.21.15.83:0/1367044227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa29015dab0 0x7fa2900caf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa286ffd640 1 -- 172.21.15.83:0/1367044227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2901622c0 msgr2=0x7fa2900cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:45.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa28ffff640 1 --2- 172.21.15.83:0/1367044227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2901622c0 0x7fa2900cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:45.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa286ffd640 1 --2- 172.21.15.83:0/1367044227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2901622c0 0x7fa2900cc830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa286ffd640 1 -- 172.21.15.83:0/1367044227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa29015be70 con 0x7fa29015de90 2024-04-17T10:38:45.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa286ffd640 1 --2- 172.21.15.83:0/1367044227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa29015de90 0x7fa29015b930 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa278009740 tx=0x7fa27801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:45.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa28cff9640 1 -- 172.21.15.83:0/1367044227 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2780b0040 con 0x7fa29015de90 2024-04-17T10:38:45.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa28cff9640 1 -- 172.21.15.83:0/1367044227 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa278004030 con 0x7fa29015de90 2024-04-17T10:38:45.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa28cff9640 1 -- 172.21.15.83:0/1367044227 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa278004420 con 0x7fa29015de90 2024-04-17T10:38:45.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa296002640 1 -- 172.21.15.83:0/1367044227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa29015c110 con 0x7fa29015de90 2024-04-17T10:38:45.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.555+0000 7fa296002640 1 -- 172.21.15.83:0/1367044227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa29015c6d0 con 0x7fa29015de90 2024-04-17T10:38:45.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.559+0000 7fa296002640 1 -- 172.21.15.83:0/1367044227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2900681a0 con 0x7fa29015de90 2024-04-17T10:38:45.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.559+0000 7fa28cff9640 1 -- 172.21.15.83:0/1367044227 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa278003620 con 0x7fa29015de90 2024-04-17T10:38:45.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.559+0000 7fa28cff9640 1 --2- 172.21.15.83:0/1367044227 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa270066f90 0x7fa270069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:45.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.559+0000 7fa28cff9640 1 -- 172.21.15.83:0/1367044227 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa27811fa20 con 0x7fa29015de90 2024-04-17T10:38:45.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.559+0000 7fa28f7fe640 1 --2- 172.21.15.83:0/1367044227 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa270066f90 0x7fa270069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:45.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.559+0000 7fa28f7fe640 1 --2- 172.21.15.83:0/1367044227 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa270066f90 0x7fa270069450 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa2900ca7c0 tx=0x7fa280008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:45.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.563+0000 7fa28cff9640 1 -- 172.21.15.83:0/1367044227 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa278016090 con 0x7fa29015de90 2024-04-17T10:38:45.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.719+0000 7fa296002640 1 -- 172.21.15.83:0/1367044227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa290074f90 con 0x7fa29015de90 2024-04-17T10:38:45.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.719+0000 7fa28cff9640 1 -- 172.21.15.83:0/1367044227 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa2780f0be0 con 0x7fa29015de90 2024-04-17T10:38:45.718 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:38:45.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.723+0000 7fa296002640 1 -- 172.21.15.83:0/1367044227 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa270066f90 msgr2=0x7fa270069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:45.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.723+0000 7fa296002640 1 --2- 172.21.15.83:0/1367044227 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa270066f90 0x7fa270069450 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa2900ca7c0 tx=0x7fa280008040 comp rx=0 tx=0).stop 2024-04-17T10:38:45.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.723+0000 7fa296002640 1 -- 172.21.15.83:0/1367044227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa29015de90 msgr2=0x7fa29015b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:45.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.723+0000 7fa296002640 1 --2- 172.21.15.83:0/1367044227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa29015de90 0x7fa29015b930 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa278009740 tx=0x7fa27801ace0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.723+0000 7fa296002640 1 -- 172.21.15.83:0/1367044227 shutdown_connections 2024-04-17T10:38:45.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.723+0000 7fa296002640 1 --2- 172.21.15.83:0/1367044227 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa270066f90 0x7fa270069450 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.723+0000 7fa296002640 1 --2- 172.21.15.83:0/1367044227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2901622c0 0x7fa2900cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.723+0000 7fa296002640 1 --2- 172.21.15.83:0/1367044227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa29015de90 0x7fa29015b930 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.723+0000 7fa296002640 1 --2- 172.21.15.83:0/1367044227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa29015dab0 0x7fa2900caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.723+0000 7fa296002640 1 -- 172.21.15.83:0/1367044227 >> 172.21.15.83:0/1367044227 conn(0x7fa29009cd70 msgr2=0x7fa29015e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:45.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.723+0000 7fa296002640 1 -- 172.21.15.83:0/1367044227 shutdown_connections 2024-04-17T10:38:45.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.723+0000 7fa296002640 1 -- 172.21.15.83:0/1367044227 wait complete. 2024-04-17T10:38:45.737 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2024-04-17T10:38:45.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9ba70aa640 1 Processor -- start 2024-04-17T10:38:45.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9ba70aa640 1 -- start start 2024-04-17T10:38:45.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9ba70aa640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9ba015db50 0x7f9ba0070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:45.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9ba70aa640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ba00709b0 con 0x7f9ba00aedd0 2024-04-17T10:38:45.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9ba70aa640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ba015e440 con 0x7f9ba006dcb0 2024-04-17T10:38:45.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9ba70aa640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ba015e720 con 0x7f9ba015db50 2024-04-17T10:38:45.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9b9ffff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9ba00aedd0 0x7f9ba006d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47334/0 (socket says 172.21.15.83:47334) 2024-04-17T10:38:45.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9b9ffff640 1 -- 172.21.15.83:0/1454414139 learned_addr learned my addr 172.21.15.83:0/1454414139 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:45.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9ba4e46640 1 --2- 172.21.15.83:0/1454414139 >> v2:172.21.15.167:3300/0 conn(0x7f9ba015db50 0x7f9ba0070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:45.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9b9f7fe640 1 -- 172.21.15.83:0/1454414139 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 161248717 0 0) 0x7f9b8c001120 con 0x7f9ba00aedd0 2024-04-17T10:38:45.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9b9f7fe640 1 -- 172.21.15.83:0/1454414139 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b90003530 con 0x7f9ba00aedd0 2024-04-17T10:38:45.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9b9f7fe640 1 -- 172.21.15.83:0/1454414139 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2394397513 0 0) 0x7f9b90003530 con 0x7f9ba00aedd0 2024-04-17T10:38:45.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9b9f7fe640 1 -- 172.21.15.83:0/1454414139 >> v2:172.21.15.167:3300/0 conn(0x7f9ba015db50 msgr2=0x7f9ba0070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:45.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9b9f7fe640 1 --2- 172.21.15.83:0/1454414139 >> v2:172.21.15.167:3300/0 conn(0x7f9ba015db50 0x7f9ba0070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9b9f7fe640 1 -- 172.21.15.83:0/1454414139 >> v1:172.21.15.167:6789/0 conn(0x7f9ba006dcb0 legacy=0x7f9ba00b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:45.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9b9f7fe640 1 -- 172.21.15.83:0/1454414139 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba015ea00 con 0x7f9ba00aedd0 2024-04-17T10:38:45.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9b9f7fe640 1 -- 172.21.15.83:0/1454414139 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f9b8c001850 con 0x7f9ba00aedd0 2024-04-17T10:38:45.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9b9f7fe640 1 -- 172.21.15.83:0/1454414139 >> v1:172.21.15.83:6789/0 conn(0x7f9ba00aedd0 legacy=0x7f9ba006d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:45.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9b9f7fe640 1 --2- 172.21.15.83:0/1454414139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b900039c0 0x7f9b90003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:45.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9b9f7fe640 1 --2- 172.21.15.83:0/1454414139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b90004810 0x7f9b90004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:45.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9b9f7fe640 1 --2- 172.21.15.83:0/1454414139 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b900051d0 0x7f9b9000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:45.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9b9f7fe640 1 -- 172.21.15.83:0/1454414139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba015e720 con 0x7f9b90004810 2024-04-17T10:38:45.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9b9f7fe640 1 -- 172.21.15.83:0/1454414139 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9000c070 con 0x7f9b900039c0 2024-04-17T10:38:45.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9b9f7fe640 1 -- 172.21.15.83:0/1454414139 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9000c270 con 0x7f9b900051d0 2024-04-17T10:38:45.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9b9ffff640 1 --2- 172.21.15.83:0/1454414139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b90004810 0x7f9b90004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:45.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9ba5647640 1 --2- 172.21.15.83:0/1454414139 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b900051d0 0x7f9b9000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:45.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9b9ffff640 1 -- 172.21.15.83:0/1454414139 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b900051d0 msgr2=0x7f9b9000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:45.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9b9ffff640 1 --2- 172.21.15.83:0/1454414139 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b900051d0 0x7f9b9000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9ba4e46640 1 --2- 172.21.15.83:0/1454414139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b900039c0 0x7f9b90003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:45.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9b9ffff640 1 -- 172.21.15.83:0/1454414139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b900039c0 msgr2=0x7f9b90003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:45.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9b9ffff640 1 --2- 172.21.15.83:0/1454414139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b900039c0 0x7f9b90003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9ba5647640 1 -- 172.21.15.83:0/1454414139 reap_dead start 2024-04-17T10:38:45.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.855+0000 7f9b9ffff640 1 -- 172.21.15.83:0/1454414139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b90003530 con 0x7f9b90004810 2024-04-17T10:38:45.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.859+0000 7f9b9ffff640 1 --2- 172.21.15.83:0/1454414139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b90004810 0x7f9b90004c90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00c490 tx=0x7f9b8c01f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8011f12db39c4e90 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:45.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.859+0000 7f9b9f7fe640 1 -- 172.21.15.83:0/1454414139 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c00c0e0 con 0x7f9b90004810 2024-04-17T10:38:45.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.859+0000 7f9b9f7fe640 1 -- 172.21.15.83:0/1454414139 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8c005b00 con 0x7f9b90004810 2024-04-17T10:38:45.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.859+0000 7f9b9f7fe640 1 -- 172.21.15.83:0/1454414139 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c007070 con 0x7f9b90004810 2024-04-17T10:38:45.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.859+0000 7f9ba70aa640 1 -- 172.21.15.83:0/1454414139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b90004810 msgr2=0x7f9b90004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:45.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.859+0000 7f9ba70aa640 1 --2- 172.21.15.83:0/1454414139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b90004810 0x7f9b90004c90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00c490 tx=0x7f9b8c01f3c0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.859+0000 7f9ba70aa640 1 -- 172.21.15.83:0/1454414139 shutdown_connections 2024-04-17T10:38:45.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.859+0000 7f9ba70aa640 1 --2- 172.21.15.83:0/1454414139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b90004810 0x7f9b90004c90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.859+0000 7f9ba70aa640 1 -- 172.21.15.83:0/1454414139 >> 172.21.15.83:0/1454414139 conn(0x7f9ba006abe0 msgr2=0x7f9ba006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:45.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.859+0000 7f9ba70aa640 1 -- 172.21.15.83:0/1454414139 shutdown_connections 2024-04-17T10:38:45.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.859+0000 7f9ba70aa640 1 -- 172.21.15.83:0/1454414139 wait complete. 2024-04-17T10:38:45.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.859+0000 7f9ba70aa640 1 Processor -- start 2024-04-17T10:38:45.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.859+0000 7f9ba70aa640 1 -- start start 2024-04-17T10:38:45.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.859+0000 7f9ba70aa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b90004810 0x7f9ba015b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:45.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.859+0000 7f9ba70aa640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ba015c6d0 0x7f9ba01f35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:45.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.859+0000 7f9ba70aa640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ba015cb50 0x7f9ba01f3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:45.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.859+0000 7f9ba70aa640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba015df30 con 0x7f9b90004810 2024-04-17T10:38:45.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.859+0000 7f9ba70aa640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba015efc0 con 0x7f9ba015c6d0 2024-04-17T10:38:45.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.859+0000 7f9ba70aa640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba015e0b0 con 0x7f9ba015cb50 2024-04-17T10:38:45.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.859+0000 7f9ba4e46640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b90004810 0x7f9ba015b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:45.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.859+0000 7f9ba4e46640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b90004810 0x7f9ba015b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52868/0 (socket says 172.21.15.83:52868) 2024-04-17T10:38:45.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.859+0000 7f9ba4e46640 1 -- 172.21.15.83:0/3420718537 learned_addr learned my addr 172.21.15.83:0/3420718537 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:45.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.863+0000 7f9b9ffff640 1 --2- 172.21.15.83:0/3420718537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ba015c6d0 0x7f9ba01f35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:45.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.863+0000 7f9ba5647640 1 --2- 172.21.15.83:0/3420718537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ba015cb50 0x7f9ba01f3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:45.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.863+0000 7f9ba4e46640 1 -- 172.21.15.83:0/3420718537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ba015cb50 msgr2=0x7f9ba01f3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:45.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.863+0000 7f9ba4e46640 1 --2- 172.21.15.83:0/3420718537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ba015cb50 0x7f9ba01f3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.863+0000 7f9ba4e46640 1 -- 172.21.15.83:0/3420718537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ba015c6d0 msgr2=0x7f9ba01f35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:45.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.863+0000 7f9ba4e46640 1 --2- 172.21.15.83:0/3420718537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ba015c6d0 0x7f9ba01f35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:45.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.863+0000 7f9ba4e46640 1 -- 172.21.15.83:0/3420718537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba015bab0 con 0x7f9b90004810 2024-04-17T10:38:45.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.863+0000 7f9ba4e46640 1 --2- 172.21.15.83:0/3420718537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b90004810 0x7f9ba015b570 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9b940129d0 tx=0x7f9b94012e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:45.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.863+0000 7f9b9d7fa640 1 -- 172.21.15.83:0/3420718537 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9401f070 con 0x7f9b90004810 2024-04-17T10:38:45.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.863+0000 7f9b9d7fa640 1 -- 172.21.15.83:0/3420718537 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b94010c30 con 0x7f9b90004810 2024-04-17T10:38:45.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.863+0000 7f9ba70aa640 1 -- 172.21.15.83:0/3420718537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ba0155620 con 0x7f9b90004810 2024-04-17T10:38:45.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.863+0000 7f9b9d7fa640 1 -- 172.21.15.83:0/3420718537 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9401a410 con 0x7f9b90004810 2024-04-17T10:38:45.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.863+0000 7f9ba70aa640 1 -- 172.21.15.83:0/3420718537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ba0155be0 con 0x7f9b90004810 2024-04-17T10:38:45.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.863+0000 7f9ba70aa640 1 -- 172.21.15.83:0/3420718537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ba0074f90 con 0x7f9b90004810 2024-04-17T10:38:45.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.863+0000 7f9b9d7fa640 1 -- 172.21.15.83:0/3420718537 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9b94022070 con 0x7f9b90004810 2024-04-17T10:38:45.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.863+0000 7f9b9d7fa640 1 --2- 172.21.15.83:0/3420718537 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9b84066f90 0x7f9b84069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:45.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.863+0000 7f9b9d7fa640 1 -- 172.21.15.83:0/3420718537 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b94087470 con 0x7f9b90004810 2024-04-17T10:38:45.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.863+0000 7f9b9ffff640 1 --2- 172.21.15.83:0/3420718537 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9b84066f90 0x7f9b84069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:45.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.863+0000 7f9b9ffff640 1 --2- 172.21.15.83:0/3420718537 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9b84066f90 0x7f9b84069450 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c0ae000 tx=0x7f9b8c007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:45.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:45.867+0000 7f9b9d7fa640 1 -- 172.21.15.83:0/3420718537 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9b94058480 con 0x7f9b90004810 2024-04-17T10:38:46.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.079+0000 7f9ba70aa640 1 -- 172.21.15.83:0/3420718537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) v1 -- 0x7f9ba0159a60 con 0x7f9b90004810 2024-04-17T10:38:46.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.347+0000 7f9b9d7fa640 1 -- 172.21.15.83:0/3420718537 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v54) v1 ==== 151+0+0 (secure 0 0 0) 0x7f9b94057d80 con 0x7f9b90004810 2024-04-17T10:38:46.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.351+0000 7f9ba70aa640 1 -- 172.21.15.83:0/3420718537 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9b84066f90 msgr2=0x7f9b84069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:46.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.351+0000 7f9ba70aa640 1 --2- 172.21.15.83:0/3420718537 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9b84066f90 0x7f9b84069450 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c0ae000 tx=0x7f9b8c007040 comp rx=0 tx=0).stop 2024-04-17T10:38:46.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.351+0000 7f9ba70aa640 1 -- 172.21.15.83:0/3420718537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b90004810 msgr2=0x7f9ba015b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:46.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.351+0000 7f9ba70aa640 1 --2- 172.21.15.83:0/3420718537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b90004810 0x7f9ba015b570 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9b940129d0 tx=0x7f9b94012e90 comp rx=0 tx=0).stop 2024-04-17T10:38:46.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.351+0000 7f9ba70aa640 1 -- 172.21.15.83:0/3420718537 shutdown_connections 2024-04-17T10:38:46.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.351+0000 7f9ba70aa640 1 --2- 172.21.15.83:0/3420718537 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9b84066f90 0x7f9b84069450 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:46.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.351+0000 7f9ba70aa640 1 --2- 172.21.15.83:0/3420718537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ba015cb50 0x7f9ba01f3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:46.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.351+0000 7f9ba70aa640 1 --2- 172.21.15.83:0/3420718537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ba015c6d0 0x7f9ba01f35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:46.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.351+0000 7f9ba70aa640 1 --2- 172.21.15.83:0/3420718537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b90004810 0x7f9ba015b570 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:46.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.351+0000 7f9ba70aa640 1 -- 172.21.15.83:0/3420718537 >> 172.21.15.83:0/3420718537 conn(0x7f9ba006abe0 msgr2=0x7f9ba00b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:46.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.351+0000 7f9ba70aa640 1 -- 172.21.15.83:0/3420718537 shutdown_connections 2024-04-17T10:38:46.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.351+0000 7f9ba70aa640 1 -- 172.21.15.83:0/3420718537 wait complete. 2024-04-17T10:38:46.369 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-17T10:38:46.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.499+0000 7f58f046c640 1 Processor -- start 2024-04-17T10:38:46.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.499+0000 7f58f046c640 1 -- start start 2024-04-17T10:38:46.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.499+0000 7f58f046c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f58e815bac0 0x7f58e8071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:46.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.499+0000 7f58f046c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f58e8070470 0x7f58e806ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:46.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.499+0000 7f58f046c640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58e806c1f0 con 0x7f58e815b6e0 2024-04-17T10:38:46.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.499+0000 7f58f046c640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58e8070a10 con 0x7f58e8070470 2024-04-17T10:38:46.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.499+0000 7f58f046c640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58e8070cf0 con 0x7f58e815bac0 2024-04-17T10:38:46.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.499+0000 7f58eea09640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f58e8070470 0x7f58e806ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:46.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.499+0000 7f58ee208640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f58e815b6e0 0x7f58e80cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47344/0 (socket says 172.21.15.83:47344) 2024-04-17T10:38:46.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.499+0000 7f58ee208640 1 -- 172.21.15.83:0/3584133024 learned_addr learned my addr 172.21.15.83:0/3584133024 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:46.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.499+0000 7f58eea09640 1 -- 172.21.15.83:0/3584133024 >> v2:172.21.15.167:3300/0 conn(0x7f58e815bac0 msgr2=0x7f58e8071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:46.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.499+0000 7f58eea09640 1 --2- 172.21.15.83:0/3584133024 >> v2:172.21.15.167:3300/0 conn(0x7f58e815bac0 0x7f58e8071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:46.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.499+0000 7f58eea09640 1 -- 172.21.15.83:0/3584133024 >> v1:172.21.15.83:6789/0 conn(0x7f58e815b6e0 legacy=0x7f58e80cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:46.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.499+0000 7f58ed206640 1 -- 172.21.15.83:0/3584133024 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2713584164 0 0) 0x7f58e806c1f0 con 0x7f58e815b6e0 2024-04-17T10:38:46.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.499+0000 7f58eea09640 1 -- 172.21.15.83:0/3584133024 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e8070fd0 con 0x7f58e8070470 2024-04-17T10:38:46.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.503+0000 7f58eea09640 1 --2- 172.21.15.83:0/3584133024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e8070470 0x7f58e806ae10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f58e8071c50 tx=0x7f58e001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f9cd8bfe64356c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:46.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.503+0000 7f58ed206640 1 -- 172.21.15.83:0/3584133024 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e00b2070 con 0x7f58e8070470 2024-04-17T10:38:46.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.503+0000 7f58ed206640 1 -- 172.21.15.83:0/3584133024 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e00a57f0 con 0x7f58e8070470 2024-04-17T10:38:46.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.503+0000 7f58ed206640 1 -- 172.21.15.83:0/3584133024 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e00a5bc0 con 0x7f58e8070470 2024-04-17T10:38:46.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.503+0000 7f58f046c640 1 -- 172.21.15.83:0/3584133024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e8070470 msgr2=0x7f58e806ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:46.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.503+0000 7f58f046c640 1 --2- 172.21.15.83:0/3584133024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e8070470 0x7f58e806ae10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f58e8071c50 tx=0x7f58e001c930 comp rx=0 tx=0).stop 2024-04-17T10:38:46.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.503+0000 7f58f046c640 1 -- 172.21.15.83:0/3584133024 shutdown_connections 2024-04-17T10:38:46.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.503+0000 7f58f046c640 1 --2- 172.21.15.83:0/3584133024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e8070470 0x7f58e806ae10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:46.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.503+0000 7f58f046c640 1 --2- 172.21.15.83:0/3584133024 >> v2:172.21.15.167:3300/0 conn(0x7f58e815bac0 0x7f58e8071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:46.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.503+0000 7f58f046c640 1 -- 172.21.15.83:0/3584133024 >> 172.21.15.83:0/3584133024 conn(0x7f58e80b15b0 msgr2=0x7f58e80b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:46.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.503+0000 7f58f046c640 1 -- 172.21.15.83:0/3584133024 shutdown_connections 2024-04-17T10:38:46.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.503+0000 7f58f046c640 1 -- 172.21.15.83:0/3584133024 wait complete. 2024-04-17T10:38:46.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.503+0000 7f58f046c640 1 Processor -- start 2024-04-17T10:38:46.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.503+0000 7f58f046c640 1 -- start start 2024-04-17T10:38:46.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.503+0000 7f58f046c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58e8070470 0x7f58e814f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:46.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.503+0000 7f58f046c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e815b6e0 0x7f58e813da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:46.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.503+0000 7f58f046c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58e815bac0 0x7f58e813a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:46.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.503+0000 7f58f046c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e806d2f0 con 0x7f58e815b6e0 2024-04-17T10:38:46.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.503+0000 7f58f046c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e806d170 con 0x7f58e815bac0 2024-04-17T10:38:46.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.503+0000 7f58f046c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e806b760 con 0x7f58e8070470 2024-04-17T10:38:46.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.503+0000 7f58eda07640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e815b6e0 0x7f58e813da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:46.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.503+0000 7f58eda07640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e815b6e0 0x7f58e813da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52882/0 (socket says 172.21.15.83:52882) 2024-04-17T10:38:46.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.503+0000 7f58eda07640 1 -- 172.21.15.83:0/2314386538 learned_addr learned my addr 172.21.15.83:0/2314386538 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:46.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.503+0000 7f58ee208640 1 --2- 172.21.15.83:0/2314386538 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58e8070470 0x7f58e814f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:46.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.503+0000 7f58eea09640 1 --2- 172.21.15.83:0/2314386538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58e815bac0 0x7f58e813a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:46.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.503+0000 7f58eda07640 1 -- 172.21.15.83:0/2314386538 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58e8070470 msgr2=0x7f58e814f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:46.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.503+0000 7f58eda07640 1 --2- 172.21.15.83:0/2314386538 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58e8070470 0x7f58e814f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:46.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.503+0000 7f58eda07640 1 -- 172.21.15.83:0/2314386538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58e815bac0 msgr2=0x7f58e813a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:46.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.503+0000 7f58eda07640 1 --2- 172.21.15.83:0/2314386538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58e815bac0 0x7f58e813a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:46.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.503+0000 7f58eda07640 1 -- 172.21.15.83:0/2314386538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e813df50 con 0x7f58e815b6e0 2024-04-17T10:38:46.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.503+0000 7f58eea09640 1 --2- 172.21.15.83:0/2314386538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58e815bac0 0x7f58e813a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:46.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.507+0000 7f58eda07640 1 --2- 172.21.15.83:0/2314386538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e815b6e0 0x7f58e813da10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f58e400b440 tx=0x7f58e400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:46.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.507+0000 7f58deffd640 1 -- 172.21.15.83:0/2314386538 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e4015020 con 0x7f58e815b6e0 2024-04-17T10:38:46.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.507+0000 7f58deffd640 1 -- 172.21.15.83:0/2314386538 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e40118b0 con 0x7f58e815b6e0 2024-04-17T10:38:46.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.507+0000 7f58deffd640 1 -- 172.21.15.83:0/2314386538 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e4011ca0 con 0x7f58e815b6e0 2024-04-17T10:38:46.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.507+0000 7f58f046c640 1 -- 172.21.15.83:0/2314386538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e814f6f0 con 0x7f58e815b6e0 2024-04-17T10:38:46.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.507+0000 7f58f046c640 1 -- 172.21.15.83:0/2314386538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e814fcb0 con 0x7f58e815b6e0 2024-04-17T10:38:46.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.507+0000 7f58f046c640 1 -- 172.21.15.83:0/2314386538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58e8074f90 con 0x7f58e815b6e0 2024-04-17T10:38:46.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.507+0000 7f58deffd640 1 -- 172.21.15.83:0/2314386538 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f58e4018020 con 0x7f58e815b6e0 2024-04-17T10:38:46.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.507+0000 7f58deffd640 1 --2- 172.21.15.83:0/2314386538 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f58cc066f60 0x7f58cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:46.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.507+0000 7f58deffd640 1 -- 172.21.15.83:0/2314386538 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58e40850a0 con 0x7f58e815b6e0 2024-04-17T10:38:46.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.507+0000 7f58ee208640 1 --2- 172.21.15.83:0/2314386538 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f58cc066f60 0x7f58cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:46.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.507+0000 7f58ee208640 1 --2- 172.21.15.83:0/2314386538 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f58cc066f60 0x7f58cc069420 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f58d8000f70 tx=0x7f58d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:46.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.511+0000 7f58deffd640 1 -- 172.21.15.83:0/2314386538 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f58e401f020 con 0x7f58e815b6e0 2024-04-17T10:38:46.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.667+0000 7f58f046c640 1 -- 172.21.15.83:0/2314386538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f58e8071df0 con 0x7f58e815b6e0 2024-04-17T10:38:46.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.675+0000 7f58deffd640 1 -- 172.21.15.83:0/2314386538 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f58e400d040 con 0x7f58e815b6e0 2024-04-17T10:38:46.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.675+0000 7f58f046c640 1 -- 172.21.15.83:0/2314386538 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f58cc066f60 msgr2=0x7f58cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:46.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.675+0000 7f58f046c640 1 --2- 172.21.15.83:0/2314386538 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f58cc066f60 0x7f58cc069420 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f58d8000f70 tx=0x7f58d8009040 comp rx=0 tx=0).stop 2024-04-17T10:38:46.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.675+0000 7f58f046c640 1 -- 172.21.15.83:0/2314386538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e815b6e0 msgr2=0x7f58e813da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:46.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.675+0000 7f58f046c640 1 --2- 172.21.15.83:0/2314386538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e815b6e0 0x7f58e813da10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f58e400b440 tx=0x7f58e400b900 comp rx=0 tx=0).stop 2024-04-17T10:38:46.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.675+0000 7f58f046c640 1 -- 172.21.15.83:0/2314386538 shutdown_connections 2024-04-17T10:38:46.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.675+0000 7f58f046c640 1 --2- 172.21.15.83:0/2314386538 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f58cc066f60 0x7f58cc069420 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:46.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.675+0000 7f58f046c640 1 --2- 172.21.15.83:0/2314386538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58e815bac0 0x7f58e813a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:46.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.675+0000 7f58f046c640 1 --2- 172.21.15.83:0/2314386538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e815b6e0 0x7f58e813da10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:46.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.675+0000 7f58f046c640 1 --2- 172.21.15.83:0/2314386538 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58e8070470 0x7f58e814f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:46.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.675+0000 7f58f046c640 1 -- 172.21.15.83:0/2314386538 >> 172.21.15.83:0/2314386538 conn(0x7f58e80b15b0 msgr2=0x7f58e80cd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:46.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.675+0000 7f58f046c640 1 -- 172.21.15.83:0/2314386538 shutdown_connections 2024-04-17T10:38:46.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.675+0000 7f58f046c640 1 -- 172.21.15.83:0/2314386538 wait complete. 2024-04-17T10:38:46.690 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T10:38:46.691 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:38:46.736 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:38:46.773 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T10:38:46.773 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:38:46.773 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:38:46.778 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:38:46.779 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:38:46.828 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:38:46.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.991+0000 7ff5f0093640 1 Processor -- start 2024-04-17T10:38:46.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.991+0000 7ff5f0093640 1 -- start start 2024-04-17T10:38:46.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.995+0000 7ff5f0093640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff5e80cf060 0x7ff5e80b4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:46.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.995+0000 7ff5f0093640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5e8070b80 0x7ff5e806c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:46.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.995+0000 7ff5f0093640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e80b2bd0 con 0x7ff5e80b3990 2024-04-17T10:38:46.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.995+0000 7ff5f0093640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5e80b2980 con 0x7ff5e80cf060 2024-04-17T10:38:46.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.995+0000 7ff5f0093640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e80cf440 con 0x7ff5e8070b80 2024-04-17T10:38:46.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.995+0000 7ff5ede2f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff5e80cf060 0x7ff5e80b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:46.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.995+0000 7ff5ed62e640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff5e80b3990 0x7ff5e8070470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40004/0 (socket says 172.21.15.83:40004) 2024-04-17T10:38:46.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.995+0000 7ff5ed62e640 1 -- 172.21.15.83:0/2141850957 learned_addr learned my addr 172.21.15.83:0/2141850957 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:46.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.995+0000 7ff5ee630640 1 --2- 172.21.15.83:0/2141850957 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5e8070b80 0x7ff5e806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:46.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.995+0000 7ff5ede2f640 1 -- 172.21.15.83:0/2141850957 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5e8070b80 msgr2=0x7ff5e806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:46.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.995+0000 7ff5ede2f640 1 --2- 172.21.15.83:0/2141850957 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5e8070b80 0x7ff5e806c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:46.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.995+0000 7ff5ede2f640 1 -- 172.21.15.83:0/2141850957 >> v1:172.21.15.83:6789/0 conn(0x7ff5e80b3990 legacy=0x7ff5e8070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:46.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.995+0000 7ff5ece2d640 1 -- 172.21.15.83:0/2141850957 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3668966618 0 0) 0x7ff5e80b2bd0 con 0x7ff5e80b3990 2024-04-17T10:38:46.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.995+0000 7ff5ede2f640 1 -- 172.21.15.83:0/2141850957 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e80b2eb0 con 0x7ff5e80cf060 2024-04-17T10:38:46.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.995+0000 7ff5ee630640 1 --2- 172.21.15.83:0/2141850957 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5e8070b80 0x7ff5e806c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:46.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.995+0000 7ff5ede2f640 1 --2- 172.21.15.83:0/2141850957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5e80cf060 0x7ff5e80b4060 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80b2b00 tx=0x7ff5e401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ebcd00c012867e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:46.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.995+0000 7ff5ece2d640 1 -- 172.21.15.83:0/2141850957 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e40b1070 con 0x7ff5e80cf060 2024-04-17T10:38:46.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.995+0000 7ff5ece2d640 1 -- 172.21.15.83:0/2141850957 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e40ac410 con 0x7ff5e80cf060 2024-04-17T10:38:46.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.995+0000 7ff5ece2d640 1 -- 172.21.15.83:0/2141850957 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e40ac7e0 con 0x7ff5e80cf060 2024-04-17T10:38:46.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.995+0000 7ff5f0093640 1 -- 172.21.15.83:0/2141850957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5e80cf060 msgr2=0x7ff5e80b4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:46.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.995+0000 7ff5f0093640 1 --2- 172.21.15.83:0/2141850957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5e80cf060 0x7ff5e80b4060 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80b2b00 tx=0x7ff5e401c470 comp rx=0 tx=0).stop 2024-04-17T10:38:46.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.995+0000 7ff5f0093640 1 -- 172.21.15.83:0/2141850957 shutdown_connections 2024-04-17T10:38:46.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.995+0000 7ff5f0093640 1 --2- 172.21.15.83:0/2141850957 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5e8070b80 0x7ff5e806c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:46.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.995+0000 7ff5f0093640 1 --2- 172.21.15.83:0/2141850957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5e80cf060 0x7ff5e80b4060 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:46.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.995+0000 7ff5f0093640 1 -- 172.21.15.83:0/2141850957 >> 172.21.15.83:0/2141850957 conn(0x7ff5e806abe0 msgr2=0x7ff5e806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:46.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.995+0000 7ff5f0093640 1 -- 172.21.15.83:0/2141850957 shutdown_connections 2024-04-17T10:38:46.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.995+0000 7ff5f0093640 1 -- 172.21.15.83:0/2141850957 wait complete. 2024-04-17T10:38:46.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.999+0000 7ff5f0093640 1 Processor -- start 2024-04-17T10:38:46.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.999+0000 7ff5f0093640 1 -- start start 2024-04-17T10:38:46.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.999+0000 7ff5f0093640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5e8070b80 0x7ff5e815d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:46.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.999+0000 7ff5f0093640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5e80b3990 0x7ff5e815b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:46.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.999+0000 7ff5f0093640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5e80cf060 0x7ff5e81f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:46.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.999+0000 7ff5f0093640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e806cb50 con 0x7ff5e8070b80 2024-04-17T10:38:46.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.999+0000 7ff5f0093640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e806c9d0 con 0x7ff5e80b3990 2024-04-17T10:38:46.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.999+0000 7ff5f0093640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e806ccd0 con 0x7ff5e80cf060 2024-04-17T10:38:46.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.999+0000 7ff5ede2f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5e8070b80 0x7ff5e815d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:46.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.999+0000 7ff5ede2f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5e8070b80 0x7ff5e815d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51358/0 (socket says 172.21.15.83:51358) 2024-04-17T10:38:46.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.999+0000 7ff5ede2f640 1 -- 172.21.15.83:0/313675883 learned_addr learned my addr 172.21.15.83:0/313675883 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:46.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.999+0000 7ff5ede2f640 1 -- 172.21.15.83:0/313675883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5e80cf060 msgr2=0x7ff5e81f6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:46.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.999+0000 7ff5ede2f640 1 --2- 172.21.15.83:0/313675883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5e80cf060 0x7ff5e81f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:46.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.999+0000 7ff5ede2f640 1 -- 172.21.15.83:0/313675883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5e80b3990 msgr2=0x7ff5e815b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:46.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.999+0000 7ff5ede2f640 1 --2- 172.21.15.83:0/313675883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5e80b3990 0x7ff5e815b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:46.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.999+0000 7ff5ede2f640 1 -- 172.21.15.83:0/313675883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e815db20 con 0x7ff5e8070b80 2024-04-17T10:38:46.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.999+0000 7ff5ede2f640 1 --2- 172.21.15.83:0/313675883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5e8070b80 0x7ff5e815d4d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff5e401a0a0 tx=0x7ff5e4019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:46.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.999+0000 7ff5de7fc640 1 -- 172.21.15.83:0/313675883 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e40b1040 con 0x7ff5e8070b80 2024-04-17T10:38:46.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.999+0000 7ff5de7fc640 1 -- 172.21.15.83:0/313675883 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e40bb070 con 0x7ff5e8070b80 2024-04-17T10:38:46.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.999+0000 7ff5f0093640 1 -- 172.21.15.83:0/313675883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e815de00 con 0x7ff5e8070b80 2024-04-17T10:38:46.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.999+0000 7ff5de7fc640 1 -- 172.21.15.83:0/313675883 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e40b5630 con 0x7ff5e8070b80 2024-04-17T10:38:46.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.999+0000 7ff5f0093640 1 -- 172.21.15.83:0/313675883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e80af230 con 0x7ff5e8070b80 2024-04-17T10:38:47.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:46.999+0000 7ff5f0093640 1 -- 172.21.15.83:0/313675883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e815e320 con 0x7ff5e8070b80 2024-04-17T10:38:47.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.003+0000 7ff5de7fc640 1 -- 172.21.15.83:0/313675883 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff5e40b57d0 con 0x7ff5e8070b80 2024-04-17T10:38:47.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.003+0000 7ff5de7fc640 1 --2- 172.21.15.83:0/313675883 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff5cc066f60 0x7ff5cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:47.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.003+0000 7ff5de7fc640 1 -- 172.21.15.83:0/313675883 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff5e411ec20 con 0x7ff5e8070b80 2024-04-17T10:38:47.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.003+0000 7ff5ed62e640 1 --2- 172.21.15.83:0/313675883 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff5cc066f60 0x7ff5cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:47.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.003+0000 7ff5ed62e640 1 --2- 172.21.15.83:0/313675883 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff5cc066f60 0x7ff5cc069420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80011f0 tx=0x7ff5d8007930 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:47.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.007+0000 7ff5de7fc640 1 -- 172.21.15.83:0/313675883 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff5e4016090 con 0x7ff5e8070b80 2024-04-17T10:38:47.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.151+0000 7ff5f0093640 1 -- 172.21.15.83:0/313675883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5e8074f90 con 0x7ff5e8070b80 2024-04-17T10:38:47.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.155+0000 7ff5de7fc640 1 -- 172.21.15.83:0/313675883 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16533 (secure 0 0 0) 0x7ff5e40ace20 con 0x7ff5e8070b80 2024-04-17T10:38:47.153 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:47.153 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":20,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:38:13.690876+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:38:10.464490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T10:38:11.043134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:38:47.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.155+0000 7ff5f0093640 1 -- 172.21.15.83:0/313675883 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff5cc066f60 msgr2=0x7ff5cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:47.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.155+0000 7ff5f0093640 1 --2- 172.21.15.83:0/313675883 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff5cc066f60 0x7ff5cc069420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80011f0 tx=0x7ff5d8007930 comp rx=0 tx=0).stop 2024-04-17T10:38:47.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.155+0000 7ff5f0093640 1 -- 172.21.15.83:0/313675883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5e8070b80 msgr2=0x7ff5e815d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:47.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.155+0000 7ff5f0093640 1 --2- 172.21.15.83:0/313675883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5e8070b80 0x7ff5e815d4d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff5e401a0a0 tx=0x7ff5e4019210 comp rx=0 tx=0).stop 2024-04-17T10:38:47.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.155+0000 7ff5f0093640 1 -- 172.21.15.83:0/313675883 shutdown_connections 2024-04-17T10:38:47.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.155+0000 7ff5f0093640 1 --2- 172.21.15.83:0/313675883 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff5cc066f60 0x7ff5cc069420 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.155+0000 7ff5f0093640 1 --2- 172.21.15.83:0/313675883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5e80cf060 0x7ff5e81f6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.155+0000 7ff5f0093640 1 --2- 172.21.15.83:0/313675883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5e80b3990 0x7ff5e815b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.155+0000 7ff5f0093640 1 --2- 172.21.15.83:0/313675883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5e8070b80 0x7ff5e815d4d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.155+0000 7ff5f0093640 1 -- 172.21.15.83:0/313675883 >> 172.21.15.83:0/313675883 conn(0x7ff5e806abe0 msgr2=0x7ff5e8154ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:47.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.159+0000 7ff5f0093640 1 -- 172.21.15.83:0/313675883 shutdown_connections 2024-04-17T10:38:47.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.159+0000 7ff5f0093640 1 -- 172.21.15.83:0/313675883 wait complete. 2024-04-17T10:38:47.170 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:38:10.464490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:38:11.043134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:38:47.171 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:38:47.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.287+0000 7fc54ba9b640 1 Processor -- start 2024-04-17T10:38:47.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.287+0000 7fc54ba9b640 1 -- start start 2024-04-17T10:38:47.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.287+0000 7fc54ba9b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc54415e0f0 0x7fc5440b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:47.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.287+0000 7fc54ba9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc5440ce400 0x7fc544161c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:47.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.287+0000 7fc54ba9b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc5440b5450 0x7fc5440b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:47.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.287+0000 7fc54ba9b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5440b5df0 con 0x7fc5440ce400 2024-04-17T10:38:47.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.287+0000 7fc54ba9b640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc544162b40 con 0x7fc5440b5450 2024-04-17T10:38:47.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.287+0000 7fc54ba9b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54415e4d0 con 0x7fc54415e0f0 2024-04-17T10:38:47.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.287+0000 7fc549036640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc5440ce400 0x7fc544161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:47.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.287+0000 7fc549036640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc5440ce400 0x7fc544161c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51370/0 (socket says 172.21.15.83:51370) 2024-04-17T10:38:47.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.287+0000 7fc549036640 1 -- 172.21.15.83:0/2681013121 learned_addr learned my addr 172.21.15.83:0/2681013121 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:47.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.287+0000 7fc549036640 1 -- 172.21.15.83:0/2681013121 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc54415e0f0 msgr2=0x7fc5440b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:47.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.287+0000 7fc549837640 1 --2- 172.21.15.83:0/2681013121 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc54415e0f0 0x7fc5440b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:47.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.287+0000 7fc54a038640 1 --2- 172.21.15.83:0/2681013121 >> v2:172.21.15.167:3300/0 conn(0x7fc5440b5450 0x7fc5440b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:47.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.287+0000 7fc549036640 1 --2- 172.21.15.83:0/2681013121 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc54415e0f0 0x7fc5440b6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.287+0000 7fc549036640 1 -- 172.21.15.83:0/2681013121 >> v2:172.21.15.167:3300/0 conn(0x7fc5440b5450 msgr2=0x7fc5440b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:47.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.287+0000 7fc549036640 1 --2- 172.21.15.83:0/2681013121 >> v2:172.21.15.167:3300/0 conn(0x7fc5440b5450 0x7fc5440b58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.287+0000 7fc549036640 1 -- 172.21.15.83:0/2681013121 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc544162580 con 0x7fc5440ce400 2024-04-17T10:38:47.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.291+0000 7fc549036640 1 --2- 172.21.15.83:0/2681013121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5440ce400 0x7fc544161c10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc5440b94e0 tx=0x7fc53401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c116e91b89ebcb00 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:47.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.291+0000 7fc548835640 1 -- 172.21.15.83:0/2681013121 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5340b0070 con 0x7fc5440ce400 2024-04-17T10:38:47.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.291+0000 7fc548835640 1 -- 172.21.15.83:0/2681013121 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5340ab600 con 0x7fc5440ce400 2024-04-17T10:38:47.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.291+0000 7fc548835640 1 -- 172.21.15.83:0/2681013121 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5340ab9d0 con 0x7fc5440ce400 2024-04-17T10:38:47.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.291+0000 7fc54ba9b640 1 -- 172.21.15.83:0/2681013121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5440ce400 msgr2=0x7fc544161c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:47.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.291+0000 7fc54ba9b640 1 --2- 172.21.15.83:0/2681013121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5440ce400 0x7fc544161c10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc5440b94e0 tx=0x7fc53401c490 comp rx=0 tx=0).stop 2024-04-17T10:38:47.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.291+0000 7fc54ba9b640 1 -- 172.21.15.83:0/2681013121 shutdown_connections 2024-04-17T10:38:47.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.291+0000 7fc54ba9b640 1 --2- 172.21.15.83:0/2681013121 >> v2:172.21.15.167:3300/0 conn(0x7fc5440b5450 0x7fc5440b58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.291+0000 7fc54ba9b640 1 --2- 172.21.15.83:0/2681013121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5440ce400 0x7fc544161c10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.291+0000 7fc54ba9b640 1 --2- 172.21.15.83:0/2681013121 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc54415e0f0 0x7fc5440b6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.291+0000 7fc54ba9b640 1 -- 172.21.15.83:0/2681013121 >> 172.21.15.83:0/2681013121 conn(0x7fc5440d0450 msgr2=0x7fc5440d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:47.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.291+0000 7fc54ba9b640 1 -- 172.21.15.83:0/2681013121 shutdown_connections 2024-04-17T10:38:47.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.291+0000 7fc54ba9b640 1 -- 172.21.15.83:0/2681013121 wait complete. 2024-04-17T10:38:47.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.291+0000 7fc54ba9b640 1 Processor -- start 2024-04-17T10:38:47.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.291+0000 7fc54ba9b640 1 -- start start 2024-04-17T10:38:47.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.291+0000 7fc54ba9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5440b5450 0x7fc5440c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:47.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.291+0000 7fc54ba9b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5440ce400 0x7fc544160c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:47.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.291+0000 7fc54ba9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc54415e0f0 0x7fc5440c8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:47.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.291+0000 7fc54ba9b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5441622d0 con 0x7fc5440b5450 2024-04-17T10:38:47.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.291+0000 7fc54ba9b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc544162150 con 0x7fc54415e0f0 2024-04-17T10:38:47.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.291+0000 7fc54ba9b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5440d2d50 con 0x7fc5440ce400 2024-04-17T10:38:47.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.291+0000 7fc549837640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5440b5450 0x7fc5440c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:47.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.291+0000 7fc549837640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5440b5450 0x7fc5440c8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51382/0 (socket says 172.21.15.83:51382) 2024-04-17T10:38:47.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.291+0000 7fc549837640 1 -- 172.21.15.83:0/1794223961 learned_addr learned my addr 172.21.15.83:0/1794223961 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:47.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.291+0000 7fc549837640 1 -- 172.21.15.83:0/1794223961 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5440ce400 msgr2=0x7fc544160c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:47.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.291+0000 7fc549837640 1 --2- 172.21.15.83:0/1794223961 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5440ce400 0x7fc544160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.291+0000 7fc549837640 1 -- 172.21.15.83:0/1794223961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc54415e0f0 msgr2=0x7fc5440c8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:47.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.291+0000 7fc549837640 1 --2- 172.21.15.83:0/1794223961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc54415e0f0 0x7fc5440c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.291+0000 7fc549837640 1 -- 172.21.15.83:0/1794223961 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54414b580 con 0x7fc5440b5450 2024-04-17T10:38:47.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.295+0000 7fc549837640 1 --2- 172.21.15.83:0/1794223961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5440b5450 0x7fc5440c8200 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc54000c7f0 tx=0x7fc54000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:47.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.295+0000 7fc53a7fc640 1 -- 172.21.15.83:0/1794223961 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc540015070 con 0x7fc5440b5450 2024-04-17T10:38:47.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.295+0000 7fc53a7fc640 1 -- 172.21.15.83:0/1794223961 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc540007820 con 0x7fc5440b5450 2024-04-17T10:38:47.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.295+0000 7fc53a7fc640 1 -- 172.21.15.83:0/1794223961 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc540007c10 con 0x7fc5440b5450 2024-04-17T10:38:47.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.295+0000 7fc54ba9b640 1 -- 172.21.15.83:0/1794223961 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5441532e0 con 0x7fc5440b5450 2024-04-17T10:38:47.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.295+0000 7fc54ba9b640 1 -- 172.21.15.83:0/1794223961 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5441538a0 con 0x7fc5440b5450 2024-04-17T10:38:47.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.295+0000 7fc54ba9b640 1 -- 172.21.15.83:0/1794223961 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5440681a0 con 0x7fc5440b5450 2024-04-17T10:38:47.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.295+0000 7fc53a7fc640 1 -- 172.21.15.83:0/1794223961 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc540018070 con 0x7fc5440b5450 2024-04-17T10:38:47.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.295+0000 7fc53a7fc640 1 --2- 172.21.15.83:0/1794223961 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc528066f60 0x7fc528069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:47.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.295+0000 7fc53a7fc640 1 -- 172.21.15.83:0/1794223961 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5400850c0 con 0x7fc5440b5450 2024-04-17T10:38:47.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.295+0000 7fc549036640 1 --2- 172.21.15.83:0/1794223961 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc528066f60 0x7fc528069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:47.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.295+0000 7fc549036640 1 --2- 172.21.15.83:0/1794223961 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc528066f60 0x7fc528069420 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc5440d4a30 tx=0x7fc534004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:47.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.299+0000 7fc53a7fc640 1 -- 172.21.15.83:0/1794223961 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc5400560d0 con 0x7fc5440b5450 2024-04-17T10:38:47.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.459+0000 7fc54ba9b640 1 -- 172.21.15.83:0/1794223961 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc54414b750 con 0x7fc5440b5450 2024-04-17T10:38:47.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.459+0000 7fc53a7fc640 1 -- 172.21.15.83:0/1794223961 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc5400559d0 con 0x7fc5440b5450 2024-04-17T10:38:47.456 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:38:47.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.459+0000 7fc54ba9b640 1 -- 172.21.15.83:0/1794223961 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc528066f60 msgr2=0x7fc528069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:47.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.459+0000 7fc54ba9b640 1 --2- 172.21.15.83:0/1794223961 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc528066f60 0x7fc528069420 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc5440d4a30 tx=0x7fc534004740 comp rx=0 tx=0).stop 2024-04-17T10:38:47.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.459+0000 7fc54ba9b640 1 -- 172.21.15.83:0/1794223961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5440b5450 msgr2=0x7fc5440c8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:47.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.459+0000 7fc54ba9b640 1 --2- 172.21.15.83:0/1794223961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5440b5450 0x7fc5440c8200 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc54000c7f0 tx=0x7fc54000ccb0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.459+0000 7fc54ba9b640 1 -- 172.21.15.83:0/1794223961 shutdown_connections 2024-04-17T10:38:47.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.459+0000 7fc54ba9b640 1 --2- 172.21.15.83:0/1794223961 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc528066f60 0x7fc528069420 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.459+0000 7fc54ba9b640 1 --2- 172.21.15.83:0/1794223961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc54415e0f0 0x7fc5440c8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.459+0000 7fc54ba9b640 1 --2- 172.21.15.83:0/1794223961 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5440ce400 0x7fc544160c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.459+0000 7fc54ba9b640 1 --2- 172.21.15.83:0/1794223961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5440b5450 0x7fc5440c8200 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.459+0000 7fc54ba9b640 1 -- 172.21.15.83:0/1794223961 >> 172.21.15.83:0/1794223961 conn(0x7fc5440d0450 msgr2=0x7fc544150ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:47.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.459+0000 7fc54ba9b640 1 -- 172.21.15.83:0/1794223961 shutdown_connections 2024-04-17T10:38:47.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.459+0000 7fc54ba9b640 1 -- 172.21.15.83:0/1794223961 wait complete. 2024-04-17T10:38:47.474 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:38:47.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.603+0000 7f1e75535640 1 Processor -- start 2024-04-17T10:38:47.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.603+0000 7f1e75535640 1 -- start start 2024-04-17T10:38:47.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.607+0000 7f1e75535640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1e7015b6e0 0x7f1e70071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:47.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.607+0000 7f1e75535640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e7015bac0 0x7f1e700cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:47.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.607+0000 7f1e75535640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1e70070470 0x7f1e7006ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:47.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.607+0000 7f1e75535640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e7006c1f0 con 0x7f1e7015b6e0 2024-04-17T10:38:47.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.607+0000 7f1e75535640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e700cdd30 con 0x7f1e70070470 2024-04-17T10:38:47.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.607+0000 7f1e75535640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e70070a10 con 0x7f1e7015bac0 2024-04-17T10:38:47.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.607+0000 7f1e6effd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1e7015b6e0 0x7f1e70071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:47.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.607+0000 7f1e6effd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1e7015b6e0 0x7f1e70071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51384/0 (socket says 172.21.15.83:51384) 2024-04-17T10:38:47.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.607+0000 7f1e6effd640 1 -- 172.21.15.83:0/3522053357 learned_addr learned my addr 172.21.15.83:0/3522053357 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:47.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.607+0000 7f1e6e7fc640 1 --2- 172.21.15.83:0/3522053357 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e7015bac0 0x7f1e700cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:47.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.607+0000 7f1e6effd640 1 -- 172.21.15.83:0/3522053357 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e7015bac0 msgr2=0x7f1e700cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:47.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.607+0000 7f1e6effd640 1 --2- 172.21.15.83:0/3522053357 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e7015bac0 0x7f1e700cd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.607+0000 7f1e6effd640 1 -- 172.21.15.83:0/3522053357 >> v2:172.21.15.167:3300/0 conn(0x7f1e70070470 msgr2=0x7f1e7006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:47.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.607+0000 7f1e6f7fe640 1 --2- 172.21.15.83:0/3522053357 >> v2:172.21.15.167:3300/0 conn(0x7f1e70070470 0x7f1e7006ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:47.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.607+0000 7f1e6effd640 1 --2- 172.21.15.83:0/3522053357 >> v2:172.21.15.167:3300/0 conn(0x7f1e70070470 0x7f1e7006ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.607+0000 7f1e6effd640 1 -- 172.21.15.83:0/3522053357 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e70070cf0 con 0x7f1e7015b6e0 2024-04-17T10:38:47.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.607+0000 7f1e6effd640 1 --2- 172.21.15.83:0/3522053357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e7015b6e0 0x7f1e70071cc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1e700cd050 tx=0x7f1e6001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f89688d0e40420 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:47.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.607+0000 7f1e6dffb640 1 -- 172.21.15.83:0/3522053357 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e600b0070 con 0x7f1e7015b6e0 2024-04-17T10:38:47.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.607+0000 7f1e6dffb640 1 -- 172.21.15.83:0/3522053357 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e600ab600 con 0x7f1e7015b6e0 2024-04-17T10:38:47.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.607+0000 7f1e6dffb640 1 -- 172.21.15.83:0/3522053357 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e600ab9d0 con 0x7f1e7015b6e0 2024-04-17T10:38:47.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e75535640 1 -- 172.21.15.83:0/3522053357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e7015b6e0 msgr2=0x7f1e70071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:47.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e75535640 1 --2- 172.21.15.83:0/3522053357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e7015b6e0 0x7f1e70071cc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1e700cd050 tx=0x7f1e6001c520 comp rx=0 tx=0).stop 2024-04-17T10:38:47.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e75535640 1 -- 172.21.15.83:0/3522053357 shutdown_connections 2024-04-17T10:38:47.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e75535640 1 --2- 172.21.15.83:0/3522053357 >> v2:172.21.15.167:3300/0 conn(0x7f1e70070470 0x7f1e7006ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e75535640 1 --2- 172.21.15.83:0/3522053357 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e7015bac0 0x7f1e700cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e75535640 1 --2- 172.21.15.83:0/3522053357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e7015b6e0 0x7f1e70071cc0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e75535640 1 -- 172.21.15.83:0/3522053357 >> 172.21.15.83:0/3522053357 conn(0x7f1e700b15b0 msgr2=0x7f1e700b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:47.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e75535640 1 -- 172.21.15.83:0/3522053357 shutdown_connections 2024-04-17T10:38:47.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e75535640 1 -- 172.21.15.83:0/3522053357 wait complete. 2024-04-17T10:38:47.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e75535640 1 Processor -- start 2024-04-17T10:38:47.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e75535640 1 -- start start 2024-04-17T10:38:47.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e75535640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e70070470 0x7f1e70164c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:47.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e75535640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e7015b6e0 0x7f1e700ace70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:47.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e75535640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e7015bac0 0x7f1e7014ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:47.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e75535640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7006d270 con 0x7f1e7015b6e0 2024-04-17T10:38:47.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e75535640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7006d0f0 con 0x7f1e70070470 2024-04-17T10:38:47.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e75535640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7006b760 con 0x7f1e7015bac0 2024-04-17T10:38:47.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e6e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e7015b6e0 0x7f1e700ace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:47.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e6e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e7015b6e0 0x7f1e700ace70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51390/0 (socket says 172.21.15.83:51390) 2024-04-17T10:38:47.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e6e7fc640 1 -- 172.21.15.83:0/2871759918 learned_addr learned my addr 172.21.15.83:0/2871759918 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:47.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e6f7fe640 1 --2- 172.21.15.83:0/2871759918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e7015bac0 0x7f1e7014ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:47.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e6effd640 1 --2- 172.21.15.83:0/2871759918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e70070470 0x7f1e70164c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:47.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e6e7fc640 1 -- 172.21.15.83:0/2871759918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e7015bac0 msgr2=0x7f1e7014ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:47.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e6e7fc640 1 --2- 172.21.15.83:0/2871759918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e7015bac0 0x7f1e7014ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e6e7fc640 1 -- 172.21.15.83:0/2871759918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e70070470 msgr2=0x7f1e70164c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:47.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e6e7fc640 1 --2- 172.21.15.83:0/2871759918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e70070470 0x7f1e70164c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e6e7fc640 1 -- 172.21.15.83:0/2871759918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e700ad920 con 0x7f1e7015b6e0 2024-04-17T10:38:47.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e6e7fc640 1 --2- 172.21.15.83:0/2871759918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e7015b6e0 0x7f1e700ace70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1e6400c7f0 tx=0x7f1e6400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:47.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e577fe640 1 -- 172.21.15.83:0/2871759918 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e64004230 con 0x7f1e7015b6e0 2024-04-17T10:38:47.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e577fe640 1 -- 172.21.15.83:0/2871759918 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e640043d0 con 0x7f1e7015b6e0 2024-04-17T10:38:47.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e577fe640 1 -- 172.21.15.83:0/2871759918 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e64003860 con 0x7f1e7015b6e0 2024-04-17T10:38:47.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e75535640 1 -- 172.21.15.83:0/2871759918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e700adc00 con 0x7f1e7015b6e0 2024-04-17T10:38:47.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.611+0000 7f1e75535640 1 -- 172.21.15.83:0/2871759918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e700b2cf0 con 0x7f1e7015b6e0 2024-04-17T10:38:47.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.615+0000 7f1e75535640 1 -- 172.21.15.83:0/2871759918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e70074f90 con 0x7f1e7015b6e0 2024-04-17T10:38:47.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.615+0000 7f1e577fe640 1 -- 172.21.15.83:0/2871759918 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1e64003c10 con 0x7f1e7015b6e0 2024-04-17T10:38:47.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.615+0000 7f1e577fe640 1 --2- 172.21.15.83:0/2871759918 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1e5c066f60 0x7f1e5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:47.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.615+0000 7f1e577fe640 1 -- 172.21.15.83:0/2871759918 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1e64085210 con 0x7f1e7015b6e0 2024-04-17T10:38:47.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.615+0000 7f1e6effd640 1 --2- 172.21.15.83:0/2871759918 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1e5c066f60 0x7f1e5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:47.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.615+0000 7f1e6effd640 1 --2- 172.21.15.83:0/2871759918 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1e5c066f60 0x7f1e5c069420 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1e600ad000 tx=0x7f1e60008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:47.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.619+0000 7f1e577fe640 1 -- 172.21.15.83:0/2871759918 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1e64015020 con 0x7f1e7015b6e0 2024-04-17T10:38:47.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.775+0000 7f1e75535640 1 -- 172.21.15.83:0/2871759918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1e700b27e0 con 0x7f1e7015b6e0 2024-04-17T10:38:47.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.775+0000 7f1e577fe640 1 -- 172.21.15.83:0/2871759918 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1e64056220 con 0x7f1e7015b6e0 2024-04-17T10:38:47.774 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:38:47.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.779+0000 7f1e75535640 1 -- 172.21.15.83:0/2871759918 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1e5c066f60 msgr2=0x7f1e5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:47.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.779+0000 7f1e75535640 1 --2- 172.21.15.83:0/2871759918 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1e5c066f60 0x7f1e5c069420 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1e600ad000 tx=0x7f1e60008040 comp rx=0 tx=0).stop 2024-04-17T10:38:47.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.779+0000 7f1e75535640 1 -- 172.21.15.83:0/2871759918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e7015b6e0 msgr2=0x7f1e700ace70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:47.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.779+0000 7f1e75535640 1 --2- 172.21.15.83:0/2871759918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e7015b6e0 0x7f1e700ace70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1e6400c7f0 tx=0x7f1e6400ccb0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.779+0000 7f1e75535640 1 -- 172.21.15.83:0/2871759918 shutdown_connections 2024-04-17T10:38:47.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.779+0000 7f1e75535640 1 --2- 172.21.15.83:0/2871759918 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1e5c066f60 0x7f1e5c069420 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.779+0000 7f1e75535640 1 --2- 172.21.15.83:0/2871759918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e7015bac0 0x7f1e7014ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.779+0000 7f1e75535640 1 --2- 172.21.15.83:0/2871759918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e7015b6e0 0x7f1e700ace70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.779+0000 7f1e75535640 1 --2- 172.21.15.83:0/2871759918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e70070470 0x7f1e70164c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.779+0000 7f1e75535640 1 -- 172.21.15.83:0/2871759918 >> 172.21.15.83:0/2871759918 conn(0x7f1e700b15b0 msgr2=0x7f1e700b00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:47.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.779+0000 7f1e75535640 1 -- 172.21.15.83:0/2871759918 shutdown_connections 2024-04-17T10:38:47.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.779+0000 7f1e75535640 1 -- 172.21.15.83:0/2871759918 wait complete. 2024-04-17T10:38:47.793 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:38:47.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f6a47e640 1 Processor -- start 2024-04-17T10:38:47.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f6a47e640 1 -- start start 2024-04-17T10:38:47.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f6a47e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f6406b520 0x7f7f64071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f6a47e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f6406c1f0 con 0x7f7f6415b6e0 2024-04-17T10:38:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f6a47e640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f64070a10 con 0x7f7f6415bac0 2024-04-17T10:38:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f6a47e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f64070cf0 con 0x7f7f6406b520 2024-04-17T10:38:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f63fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7f6415b6e0 0x7f7f640cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40006/0 (socket says 172.21.15.83:40006) 2024-04-17T10:38:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f63fff640 1 -- 172.21.15.83:0/1997018779 learned_addr learned my addr 172.21.15.83:0/1997018779 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f68a1b640 1 --2- 172.21.15.83:0/1997018779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f6406b520 0x7f7f64071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f62ffd640 1 -- 172.21.15.83:0/1997018779 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3869272783 0 0) 0x7f7f6406c1f0 con 0x7f7f6415b6e0 2024-04-17T10:38:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f62ffd640 1 -- 172.21.15.83:0/1997018779 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f4c003530 con 0x7f7f6415b6e0 2024-04-17T10:38:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f62ffd640 1 -- 172.21.15.83:0/1997018779 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 46929347 0 0) 0x7f7f4c003530 con 0x7f7f6415b6e0 2024-04-17T10:38:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f62ffd640 1 -- 172.21.15.83:0/1997018779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f6406b520 msgr2=0x7f7f64071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f62ffd640 1 --2- 172.21.15.83:0/1997018779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f6406b520 0x7f7f64071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f62ffd640 1 -- 172.21.15.83:0/1997018779 >> v1:172.21.15.167:6789/0 conn(0x7f7f6415bac0 legacy=0x7f7f6406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:47.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f62ffd640 1 -- 172.21.15.83:0/1997018779 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f64070fd0 con 0x7f7f6415b6e0 2024-04-17T10:38:47.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f62ffd640 1 -- 172.21.15.83:0/1997018779 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f7f54001680 con 0x7f7f6415b6e0 2024-04-17T10:38:47.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f62ffd640 1 -- 172.21.15.83:0/1997018779 >> v1:172.21.15.83:6789/0 conn(0x7f7f6415b6e0 legacy=0x7f7f640cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:47.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f62ffd640 1 --2- 172.21.15.83:0/1997018779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f6406b520 0x7f7f4c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:47.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f62ffd640 1 --2- 172.21.15.83:0/1997018779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f4c004070 0x7f7f4c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:47.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f62ffd640 1 --2- 172.21.15.83:0/1997018779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f4c004d50 0x7f7f4c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:47.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f62ffd640 1 -- 172.21.15.83:0/1997018779 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f64070cf0 con 0x7f7f4c004d50 2024-04-17T10:38:47.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f62ffd640 1 -- 172.21.15.83:0/1997018779 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c00bc10 con 0x7f7f4c004070 2024-04-17T10:38:47.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f62ffd640 1 -- 172.21.15.83:0/1997018779 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c00be10 con 0x7f7f6406b520 2024-04-17T10:38:47.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f68a1b640 1 --2- 172.21.15.83:0/1997018779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f4c004d50 0x7f7f4c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:47.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f63fff640 1 --2- 172.21.15.83:0/1997018779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f6406b520 0x7f7f4c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:47.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f637fe640 1 --2- 172.21.15.83:0/1997018779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f4c004070 0x7f7f4c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:47.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f68a1b640 1 -- 172.21.15.83:0/1997018779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f6406b520 msgr2=0x7f7f4c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:47.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f68a1b640 1 --2- 172.21.15.83:0/1997018779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f6406b520 0x7f7f4c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f68a1b640 1 -- 172.21.15.83:0/1997018779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f4c004070 msgr2=0x7f7f4c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:47.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f68a1b640 1 --2- 172.21.15.83:0/1997018779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f4c004070 0x7f7f4c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f68a1b640 1 -- 172.21.15.83:0/1997018779 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f4c003530 con 0x7f7f4c004d50 2024-04-17T10:38:47.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.915+0000 7f7f637fe640 1 --2- 172.21.15.83:0/1997018779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f4c004070 0x7f7f4c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:47.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.919+0000 7f7f68a1b640 1 --2- 172.21.15.83:0/1997018779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f4c004d50 0x7f7f4c0051d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c007360 tx=0x7f7f5c01ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229434f2d5f00c5d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:47.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.919+0000 7f7f62ffd640 1 -- 172.21.15.83:0/1997018779 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c0b2070 con 0x7f7f4c004d50 2024-04-17T10:38:47.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.919+0000 7f7f62ffd640 1 -- 172.21.15.83:0/1997018779 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5c0a5d40 con 0x7f7f4c004d50 2024-04-17T10:38:47.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.919+0000 7f7f62ffd640 1 -- 172.21.15.83:0/1997018779 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c009be0 con 0x7f7f4c004d50 2024-04-17T10:38:47.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.919+0000 7f7f6a47e640 1 -- 172.21.15.83:0/1997018779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f4c004d50 msgr2=0x7f7f4c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:47.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.919+0000 7f7f6a47e640 1 --2- 172.21.15.83:0/1997018779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f4c004d50 0x7f7f4c0051d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c007360 tx=0x7f7f5c01ce40 comp rx=0 tx=0).stop 2024-04-17T10:38:47.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.919+0000 7f7f68a1b640 1 -- 172.21.15.83:0/1997018779 reap_dead start 2024-04-17T10:38:47.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.919+0000 7f7f6a47e640 1 -- 172.21.15.83:0/1997018779 shutdown_connections 2024-04-17T10:38:47.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.919+0000 7f7f6a47e640 1 -- 172.21.15.83:0/1997018779 >> 172.21.15.83:0/1997018779 conn(0x7f7f640b15b0 msgr2=0x7f7f640b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:47.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.919+0000 7f7f6a47e640 1 -- 172.21.15.83:0/1997018779 shutdown_connections 2024-04-17T10:38:47.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.919+0000 7f7f6a47e640 1 -- 172.21.15.83:0/1997018779 wait complete. 2024-04-17T10:38:47.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.919+0000 7f7f6a47e640 1 Processor -- start 2024-04-17T10:38:47.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.919+0000 7f7f6a47e640 1 -- start start 2024-04-17T10:38:47.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.923+0000 7f7f6a47e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f4c004d50 0x7f7f6415d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:47.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.923+0000 7f7f6a47e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f6415bf50 0x7f7f6415fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:47.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.923+0000 7f7f6a47e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f640a9430 0x7f7f640a98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:47.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.923+0000 7f7f6a47e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f640705f0 con 0x7f7f4c004d50 2024-04-17T10:38:47.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.923+0000 7f7f6a47e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f64070470 con 0x7f7f640a9430 2024-04-17T10:38:47.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.923+0000 7f7f6a47e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f64070770 con 0x7f7f6415bf50 2024-04-17T10:38:47.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.923+0000 7f7f63fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f4c004d50 0x7f7f6415d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:47.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.923+0000 7f7f63fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f4c004d50 0x7f7f6415d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51404/0 (socket says 172.21.15.83:51404) 2024-04-17T10:38:47.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.923+0000 7f7f63fff640 1 -- 172.21.15.83:0/2914891978 learned_addr learned my addr 172.21.15.83:0/2914891978 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:47.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.923+0000 7f7f637fe640 1 --2- 172.21.15.83:0/2914891978 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f6415bf50 0x7f7f6415fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:47.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.923+0000 7f7f68a1b640 1 --2- 172.21.15.83:0/2914891978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f640a9430 0x7f7f640a98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:47.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.923+0000 7f7f63fff640 1 -- 172.21.15.83:0/2914891978 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f6415bf50 msgr2=0x7f7f6415fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:47.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.923+0000 7f7f63fff640 1 --2- 172.21.15.83:0/2914891978 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f6415bf50 0x7f7f6415fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.923+0000 7f7f63fff640 1 -- 172.21.15.83:0/2914891978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f640a9430 msgr2=0x7f7f640a98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:47.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.923+0000 7f7f63fff640 1 --2- 172.21.15.83:0/2914891978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f640a9430 0x7f7f640a98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:47.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.923+0000 7f7f63fff640 1 -- 172.21.15.83:0/2914891978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f640aa440 con 0x7f7f4c004d50 2024-04-17T10:38:47.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.923+0000 7f7f637fe640 1 --2- 172.21.15.83:0/2914891978 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f6415bf50 0x7f7f6415fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:47.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.923+0000 7f7f68a1b640 1 --2- 172.21.15.83:0/2914891978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f640a9430 0x7f7f640a98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:47.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.923+0000 7f7f63fff640 1 --2- 172.21.15.83:0/2914891978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f4c004d50 0x7f7f6415d490 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400e850 tx=0x7f7f5400ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:47.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.923+0000 7f7f60ff9640 1 -- 172.21.15.83:0/2914891978 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5400c870 con 0x7f7f4c004d50 2024-04-17T10:38:47.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.923+0000 7f7f60ff9640 1 -- 172.21.15.83:0/2914891978 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f54018070 con 0x7f7f4c004d50 2024-04-17T10:38:47.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.923+0000 7f7f6a47e640 1 -- 172.21.15.83:0/2914891978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f6415c3f0 con 0x7f7f4c004d50 2024-04-17T10:38:47.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.923+0000 7f7f60ff9640 1 -- 172.21.15.83:0/2914891978 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5400cc50 con 0x7f7f4c004d50 2024-04-17T10:38:47.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.923+0000 7f7f6a47e640 1 -- 172.21.15.83:0/2914891978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f6415c9b0 con 0x7f7f4c004d50 2024-04-17T10:38:47.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.923+0000 7f7f6a47e640 1 -- 172.21.15.83:0/2914891978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f64074f90 con 0x7f7f4c004d50 2024-04-17T10:38:47.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.923+0000 7f7f60ff9640 1 -- 172.21.15.83:0/2914891978 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7f5401d460 con 0x7f7f4c004d50 2024-04-17T10:38:47.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.923+0000 7f7f60ff9640 1 --2- 172.21.15.83:0/2914891978 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7f50066f60 0x7f7f50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:47.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.923+0000 7f7f637fe640 1 --2- 172.21.15.83:0/2914891978 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7f50066f60 0x7f7f50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:47.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.923+0000 7f7f60ff9640 1 -- 172.21.15.83:0/2914891978 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f54084f10 con 0x7f7f4c004d50 2024-04-17T10:38:47.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.923+0000 7f7f637fe640 1 --2- 172.21.15.83:0/2914891978 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7f50066f60 0x7f7f50069420 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7f641f4560 tx=0x7f7f58009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:47.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:47.927+0000 7f7f60ff9640 1 -- 172.21.15.83:0/2914891978 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7f54055f20 con 0x7f7f4c004d50 2024-04-17T10:38:48.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.087+0000 7f7f6a47e640 1 -- 172.21.15.83:0/2914891978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7f640b1c60 con 0x7f7f4c004d50 2024-04-17T10:38:48.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.087+0000 7f7f60ff9640 1 -- 172.21.15.83:0/2914891978 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7f54055820 con 0x7f7f4c004d50 2024-04-17T10:38:48.085 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:38:48.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.087+0000 7f7f6a47e640 1 -- 172.21.15.83:0/2914891978 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7f50066f60 msgr2=0x7f7f50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:48.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.087+0000 7f7f6a47e640 1 --2- 172.21.15.83:0/2914891978 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7f50066f60 0x7f7f50069420 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7f641f4560 tx=0x7f7f58009040 comp rx=0 tx=0).stop 2024-04-17T10:38:48.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.087+0000 7f7f6a47e640 1 -- 172.21.15.83:0/2914891978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f4c004d50 msgr2=0x7f7f6415d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:48.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.087+0000 7f7f6a47e640 1 --2- 172.21.15.83:0/2914891978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f4c004d50 0x7f7f6415d490 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400e850 tx=0x7f7f5400ed10 comp rx=0 tx=0).stop 2024-04-17T10:38:48.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.087+0000 7f7f6a47e640 1 -- 172.21.15.83:0/2914891978 shutdown_connections 2024-04-17T10:38:48.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.087+0000 7f7f6a47e640 1 --2- 172.21.15.83:0/2914891978 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7f50066f60 0x7f7f50069420 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:48.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.087+0000 7f7f6a47e640 1 --2- 172.21.15.83:0/2914891978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f640a9430 0x7f7f640a98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:48.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.087+0000 7f7f6a47e640 1 --2- 172.21.15.83:0/2914891978 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f6415bf50 0x7f7f6415fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:48.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.087+0000 7f7f6a47e640 1 --2- 172.21.15.83:0/2914891978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f4c004d50 0x7f7f6415d490 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:48.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.091+0000 7f7f6a47e640 1 -- 172.21.15.83:0/2914891978 >> 172.21.15.83:0/2914891978 conn(0x7f7f640b15b0 msgr2=0x7f7f640ce790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:48.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.091+0000 7f7f6a47e640 1 -- 172.21.15.83:0/2914891978 shutdown_connections 2024-04-17T10:38:48.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.091+0000 7f7f6a47e640 1 -- 172.21.15.83:0/2914891978 wait complete. 2024-04-17T10:38:48.104 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:38:48.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.219+0000 7f9dbc1f5640 1 Processor -- start 2024-04-17T10:38:48.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.219+0000 7f9dbc1f5640 1 -- start start 2024-04-17T10:38:48.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.219+0000 7f9dbc1f5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9db40b4090 0x7f9db406c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:48.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.219+0000 7f9dbc1f5640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9db40af160 con 0x7f9db40b4520 2024-04-17T10:38:48.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.219+0000 7f9dbc1f5640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9db40b2ff0 con 0x7f9db4154670 2024-04-17T10:38:48.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.219+0000 7f9dbc1f5640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9db4154a50 con 0x7f9db40b4090 2024-04-17T10:38:48.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.219+0000 7f9dba792640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9db40b4520 0x7f9db406ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40022/0 (socket says 172.21.15.83:40022) 2024-04-17T10:38:48.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.219+0000 7f9dba792640 1 -- 172.21.15.83:0/1046906642 learned_addr learned my addr 172.21.15.83:0/1046906642 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:48.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.219+0000 7f9db9790640 1 --2- 172.21.15.83:0/1046906642 >> v2:172.21.15.83:3300/0 conn(0x7f9db40b4090 0x7f9db406c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:48.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.219+0000 7f9db8f8f640 1 -- 172.21.15.83:0/1046906642 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1297815656 0 0) 0x7f9db40af160 con 0x7f9db40b4520 2024-04-17T10:38:48.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.219+0000 7f9db8f8f640 1 -- 172.21.15.83:0/1046906642 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9da0003530 con 0x7f9db40b4520 2024-04-17T10:38:48.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.219+0000 7f9db9790640 1 -- 172.21.15.83:0/1046906642 >> v1:172.21.15.167:6789/0 conn(0x7f9db4154670 legacy=0x7f9db40aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:48.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.219+0000 7f9db9790640 1 -- 172.21.15.83:0/1046906642 >> v1:172.21.15.83:6789/0 conn(0x7f9db40b4520 legacy=0x7f9db406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:48.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.219+0000 7f9db9790640 1 -- 172.21.15.83:0/1046906642 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db40b32d0 con 0x7f9db40b4090 2024-04-17T10:38:48.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.223+0000 7f9db9790640 1 --2- 172.21.15.83:0/1046906642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9db40b4090 0x7f9db406c570 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9db00097c0 tx=0x7f9db001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be282dfd8728d405 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:48.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.223+0000 7f9db8f8f640 1 -- 172.21.15.83:0/1046906642 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db00b2070 con 0x7f9db40b4090 2024-04-17T10:38:48.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.223+0000 7f9db8f8f640 1 -- 172.21.15.83:0/1046906642 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db00ad680 con 0x7f9db40b4090 2024-04-17T10:38:48.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.223+0000 7f9db8f8f640 1 -- 172.21.15.83:0/1046906642 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db00ada50 con 0x7f9db40b4090 2024-04-17T10:38:48.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.223+0000 7f9dbc1f5640 1 -- 172.21.15.83:0/1046906642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9db40b4090 msgr2=0x7f9db406c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:48.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.223+0000 7f9dbc1f5640 1 --2- 172.21.15.83:0/1046906642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9db40b4090 0x7f9db406c570 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9db00097c0 tx=0x7f9db001c470 comp rx=0 tx=0).stop 2024-04-17T10:38:48.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.223+0000 7f9dbc1f5640 1 -- 172.21.15.83:0/1046906642 shutdown_connections 2024-04-17T10:38:48.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.223+0000 7f9dbc1f5640 1 --2- 172.21.15.83:0/1046906642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9db40b4090 0x7f9db406c570 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:48.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.223+0000 7f9dbc1f5640 1 -- 172.21.15.83:0/1046906642 >> 172.21.15.83:0/1046906642 conn(0x7f9db40b15b0 msgr2=0x7f9db40b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:48.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.223+0000 7f9dbc1f5640 1 -- 172.21.15.83:0/1046906642 shutdown_connections 2024-04-17T10:38:48.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.223+0000 7f9dbc1f5640 1 -- 172.21.15.83:0/1046906642 wait complete. 2024-04-17T10:38:48.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.223+0000 7f9dbc1f5640 1 Processor -- start 2024-04-17T10:38:48.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.223+0000 7f9dbc1f5640 1 -- start start 2024-04-17T10:38:48.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.223+0000 7f9dbc1f5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9db40b4090 0x7f9db4148a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:48.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.223+0000 7f9dbc1f5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9db40b4520 0x7f9db40acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:48.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.223+0000 7f9dbc1f5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9db4154670 0x7f9db40aab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:48.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.223+0000 7f9dbc1f5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db415b3c0 con 0x7f9db4154670 2024-04-17T10:38:48.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.223+0000 7f9dbc1f5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db406b520 con 0x7f9db40b4520 2024-04-17T10:38:48.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.223+0000 7f9dbc1f5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db4168fc0 con 0x7f9db40b4090 2024-04-17T10:38:48.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.223+0000 7f9dba792640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9db4154670 0x7f9db40aab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:48.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.223+0000 7f9dba792640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9db4154670 0x7f9db40aab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51424/0 (socket says 172.21.15.83:51424) 2024-04-17T10:38:48.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.223+0000 7f9dba792640 1 -- 172.21.15.83:0/4253042152 learned_addr learned my addr 172.21.15.83:0/4253042152 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:48.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.223+0000 7f9db9f91640 1 --2- 172.21.15.83:0/4253042152 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9db40b4090 0x7f9db4148a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:48.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.223+0000 7f9db9790640 1 --2- 172.21.15.83:0/4253042152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9db40b4520 0x7f9db40acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:48.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.223+0000 7f9dba792640 1 -- 172.21.15.83:0/4253042152 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9db40b4090 msgr2=0x7f9db4148a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:48.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.223+0000 7f9dba792640 1 --2- 172.21.15.83:0/4253042152 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9db40b4090 0x7f9db4148a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:48.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.223+0000 7f9dba792640 1 -- 172.21.15.83:0/4253042152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9db40b4520 msgr2=0x7f9db40acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:48.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.223+0000 7f9dba792640 1 --2- 172.21.15.83:0/4253042152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9db40b4520 0x7f9db40acdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:48.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.223+0000 7f9dba792640 1 -- 172.21.15.83:0/4253042152 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db40cf190 con 0x7f9db4154670 2024-04-17T10:38:48.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.227+0000 7f9dba792640 1 --2- 172.21.15.83:0/4253042152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9db4154670 0x7f9db40aab30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9dac00cb40 tx=0x7f9dac00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:48.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.227+0000 7f9daa7fc640 1 -- 172.21.15.83:0/4253042152 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac017070 con 0x7f9db4154670 2024-04-17T10:38:48.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.227+0000 7f9daa7fc640 1 -- 172.21.15.83:0/4253042152 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dac00aca0 con 0x7f9db4154670 2024-04-17T10:38:48.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.227+0000 7f9dbc1f5640 1 -- 172.21.15.83:0/4253042152 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db40ae280 con 0x7f9db4154670 2024-04-17T10:38:48.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.227+0000 7f9daa7fc640 1 -- 172.21.15.83:0/4253042152 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac012620 con 0x7f9db4154670 2024-04-17T10:38:48.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.227+0000 7f9dbc1f5640 1 -- 172.21.15.83:0/4253042152 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db40adcc0 con 0x7f9db4154670 2024-04-17T10:38:48.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.227+0000 7f9dbc1f5640 1 -- 172.21.15.83:0/4253042152 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9db4074f90 con 0x7f9db4154670 2024-04-17T10:38:48.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.227+0000 7f9daa7fc640 1 -- 172.21.15.83:0/4253042152 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9dac01a070 con 0x7f9db4154670 2024-04-17T10:38:48.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.227+0000 7f9daa7fc640 1 --2- 172.21.15.83:0/4253042152 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9d98066f90 0x7f9d98069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:48.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.227+0000 7f9daa7fc640 1 -- 172.21.15.83:0/4253042152 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9dac084e00 con 0x7f9db4154670 2024-04-17T10:38:48.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.227+0000 7f9db9f91640 1 --2- 172.21.15.83:0/4253042152 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9d98066f90 0x7f9d98069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:48.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.227+0000 7f9db9f91640 1 --2- 172.21.15.83:0/4253042152 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9d98066f90 0x7f9d98069450 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9da4000f70 tx=0x7f9da4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:48.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.231+0000 7f9daa7fc640 1 -- 172.21.15.83:0/4253042152 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9dac055e00 con 0x7f9db4154670 2024-04-17T10:38:48.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.403+0000 7f9dbc1f5640 1 -- 172.21.15.83:0/4253042152 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9db40b2a00 con 0x7f9db4154670 2024-04-17T10:38:48.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.403+0000 7f9daa7fc640 1 -- 172.21.15.83:0/4253042152 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9dac055700 con 0x7f9db4154670 2024-04-17T10:38:48.399 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-17T10:38:48.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.403+0000 7f9dbc1f5640 1 -- 172.21.15.83:0/4253042152 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9d98066f90 msgr2=0x7f9d98069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:48.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.403+0000 7f9dbc1f5640 1 --2- 172.21.15.83:0/4253042152 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9d98066f90 0x7f9d98069450 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9da4000f70 tx=0x7f9da4009040 comp rx=0 tx=0).stop 2024-04-17T10:38:48.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.403+0000 7f9dbc1f5640 1 -- 172.21.15.83:0/4253042152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9db4154670 msgr2=0x7f9db40aab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:48.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.403+0000 7f9dbc1f5640 1 --2- 172.21.15.83:0/4253042152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9db4154670 0x7f9db40aab30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9dac00cb40 tx=0x7f9dac00a060 comp rx=0 tx=0).stop 2024-04-17T10:38:48.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.403+0000 7f9dbc1f5640 1 -- 172.21.15.83:0/4253042152 shutdown_connections 2024-04-17T10:38:48.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.403+0000 7f9dbc1f5640 1 --2- 172.21.15.83:0/4253042152 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9d98066f90 0x7f9d98069450 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:48.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.403+0000 7f9dbc1f5640 1 --2- 172.21.15.83:0/4253042152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9db4154670 0x7f9db40aab30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:48.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.403+0000 7f9dbc1f5640 1 --2- 172.21.15.83:0/4253042152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9db40b4520 0x7f9db40acdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:48.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.403+0000 7f9dbc1f5640 1 --2- 172.21.15.83:0/4253042152 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9db40b4090 0x7f9db4148a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:48.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.403+0000 7f9dbc1f5640 1 -- 172.21.15.83:0/4253042152 >> 172.21.15.83:0/4253042152 conn(0x7f9db40b15b0 msgr2=0x7f9db40af350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:48.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.403+0000 7f9dbc1f5640 1 -- 172.21.15.83:0/4253042152 shutdown_connections 2024-04-17T10:38:48.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.403+0000 7f9dbc1f5640 1 -- 172.21.15.83:0/4253042152 wait complete. 2024-04-17T10:38:48.417 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:38:48.418 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:38:48.423 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:38:48.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.575+0000 7fdeac205640 1 Processor -- start 2024-04-17T10:38:48.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.575+0000 7fdeac205640 1 -- start start 2024-04-17T10:38:48.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.575+0000 7fdeac205640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdea415ba30 0x7fdea4072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:48.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.575+0000 7fdeac205640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdea40cfbe0 0x7fdea406c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:48.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.575+0000 7fdeac205640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdea406e320 con 0x7fdea415c960 2024-04-17T10:38:48.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.575+0000 7fdeac205640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea4071ea0 con 0x7fdea40cfbe0 2024-04-17T10:38:48.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.575+0000 7fdeac205640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea406e040 con 0x7fdea415ba30 2024-04-17T10:38:48.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.575+0000 7fdeaa7a2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdea40cfbe0 0x7fdea406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:48.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.575+0000 7fdea97a0640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fdea415c960 0x7fdea40cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40024/0 (socket says 172.21.15.83:40024) 2024-04-17T10:38:48.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.575+0000 7fdea97a0640 1 -- 172.21.15.83:0/1565690899 learned_addr learned my addr 172.21.15.83:0/1565690899 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:48.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.575+0000 7fdea9fa1640 1 --2- 172.21.15.83:0/1565690899 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdea415ba30 0x7fdea4072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:48.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.575+0000 7fdeaa7a2640 1 -- 172.21.15.83:0/1565690899 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdea415ba30 msgr2=0x7fdea4072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:48.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.575+0000 7fdeaa7a2640 1 --2- 172.21.15.83:0/1565690899 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdea415ba30 0x7fdea4072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:48.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.575+0000 7fdeaa7a2640 1 -- 172.21.15.83:0/1565690899 >> v1:172.21.15.83:6789/0 conn(0x7fdea415c960 legacy=0x7fdea40cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:48.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.575+0000 7fdeaa7a2640 1 -- 172.21.15.83:0/1565690899 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea4071850 con 0x7fdea40cfbe0 2024-04-17T10:38:48.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.575+0000 7fdea9fa1640 1 --2- 172.21.15.83:0/1565690899 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdea415ba30 0x7fdea4072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:48.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fdeaa7a2640 1 --2- 172.21.15.83:0/1565690899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdea40cfbe0 0x7fdea406c490 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdea0009d90 tx=0x7fdea001c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce8863237e3b5f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:48.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fdea8f9f640 1 -- 172.21.15.83:0/1565690899 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea00a8630 con 0x7fdea40cfbe0 2024-04-17T10:38:48.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fdea8f9f640 1 -- 172.21.15.83:0/1565690899 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea00a87d0 con 0x7fdea40cfbe0 2024-04-17T10:38:48.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fdea8f9f640 1 -- 172.21.15.83:0/1565690899 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea00a8ba0 con 0x7fdea40cfbe0 2024-04-17T10:38:48.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fdeac205640 1 -- 172.21.15.83:0/1565690899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdea40cfbe0 msgr2=0x7fdea406c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:48.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fdeac205640 1 --2- 172.21.15.83:0/1565690899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdea40cfbe0 0x7fdea406c490 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdea0009d90 tx=0x7fdea001c770 comp rx=0 tx=0).stop 2024-04-17T10:38:48.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fdeac205640 1 -- 172.21.15.83:0/1565690899 shutdown_connections 2024-04-17T10:38:48.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fdeac205640 1 --2- 172.21.15.83:0/1565690899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdea40cfbe0 0x7fdea406c490 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:48.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fdeac205640 1 --2- 172.21.15.83:0/1565690899 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdea415ba30 0x7fdea4072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:48.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fdeac205640 1 -- 172.21.15.83:0/1565690899 >> 172.21.15.83:0/1565690899 conn(0x7fdea406abe0 msgr2=0x7fdea406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:48.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fdeac205640 1 -- 172.21.15.83:0/1565690899 shutdown_connections 2024-04-17T10:38:48.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fdeac205640 1 -- 172.21.15.83:0/1565690899 wait complete. 2024-04-17T10:38:48.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fdeac205640 1 Processor -- start 2024-04-17T10:38:48.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fdeac205640 1 -- start start 2024-04-17T10:38:48.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fdeac205640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdea40cfbe0 0x7fdea414cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:48.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fdeac205640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdea415ba30 0x7fdea4151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:48.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fdeac205640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdea415c960 0x7fdea414b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:48.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fdeac205640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea406de20 con 0x7fdea40cfbe0 2024-04-17T10:38:48.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fdeac205640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea406dca0 con 0x7fdea415c960 2024-04-17T10:38:48.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fdeac205640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea4168520 con 0x7fdea415ba30 2024-04-17T10:38:48.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fdea9fa1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdea40cfbe0 0x7fdea414cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:48.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fdea9fa1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdea40cfbe0 0x7fdea414cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51430/0 (socket says 172.21.15.83:51430) 2024-04-17T10:38:48.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fdea9fa1640 1 -- 172.21.15.83:0/3571467801 learned_addr learned my addr 172.21.15.83:0/3571467801 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:48.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fdea97a0640 1 --2- 172.21.15.83:0/3571467801 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdea415ba30 0x7fdea4151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:48.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fdea9fa1640 1 -- 172.21.15.83:0/3571467801 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdea415ba30 msgr2=0x7fdea4151130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:48.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fdea9fa1640 1 --2- 172.21.15.83:0/3571467801 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdea415ba30 0x7fdea4151130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:48.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fdea9fa1640 1 -- 172.21.15.83:0/3571467801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdea415c960 msgr2=0x7fdea414b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:48.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fdea9fa1640 1 --2- 172.21.15.83:0/3571467801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdea415c960 0x7fdea414b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:48.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fdea9fa1640 1 -- 172.21.15.83:0/3571467801 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea4151de0 con 0x7fdea40cfbe0 2024-04-17T10:38:48.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fdea9fa1640 1 --2- 172.21.15.83:0/3571467801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdea40cfbe0 0x7fdea414cfc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fde9c00c7f0 tx=0x7fde9c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:48.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fde9a7fc640 1 -- 172.21.15.83:0/3571467801 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9c0079d0 con 0x7fdea40cfbe0 2024-04-17T10:38:48.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fde9a7fc640 1 -- 172.21.15.83:0/3571467801 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9c007b70 con 0x7fdea40cfbe0 2024-04-17T10:38:48.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fdeac205640 1 -- 172.21.15.83:0/3571467801 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea4151b00 con 0x7fdea40cfbe0 2024-04-17T10:38:48.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fde9a7fc640 1 -- 172.21.15.83:0/3571467801 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9c0113f0 con 0x7fdea40cfbe0 2024-04-17T10:38:48.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.579+0000 7fdeac205640 1 -- 172.21.15.83:0/3571467801 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea40cd820 con 0x7fdea40cfbe0 2024-04-17T10:38:48.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.583+0000 7fdeac205640 1 -- 172.21.15.83:0/3571467801 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdea40cda20 con 0x7fdea40cfbe0 2024-04-17T10:38:48.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.583+0000 7fde9a7fc640 1 -- 172.21.15.83:0/3571467801 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fde9c011590 con 0x7fdea40cfbe0 2024-04-17T10:38:48.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.583+0000 7fde9a7fc640 1 --2- 172.21.15.83:0/3571467801 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fde88066f60 0x7fde88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:48.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.583+0000 7fdea97a0640 1 --2- 172.21.15.83:0/3571467801 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fde88066f60 0x7fde88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:48.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.583+0000 7fde9a7fc640 1 -- 172.21.15.83:0/3571467801 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde9c0851c0 con 0x7fdea40cfbe0 2024-04-17T10:38:48.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.583+0000 7fdea97a0640 1 --2- 172.21.15.83:0/3571467801 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fde88066f60 0x7fde88069420 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdea4152d80 tx=0x7fde94009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:48.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.587+0000 7fde9a7fc640 1 -- 172.21.15.83:0/3571467801 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdea40cda20 con 0x7fdea40cfbe0 2024-04-17T10:38:48.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.735+0000 7fdeac205640 1 -- 172.21.15.83:0/3571467801 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdea414d6a0 con 0x7fdea40cfbe0 2024-04-17T10:38:48.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.739+0000 7fde9a7fc640 1 -- 172.21.15.83:0/3571467801 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16533 (secure 0 0 0) 0x7fde9c015020 con 0x7fdea40cfbe0 2024-04-17T10:38:48.735 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:48.736 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":20,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:38:13.690876+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:38:10.464490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T10:38:11.043134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:38:48.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.739+0000 7fdeac205640 1 -- 172.21.15.83:0/3571467801 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fde88066f60 msgr2=0x7fde88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:48.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.739+0000 7fdeac205640 1 --2- 172.21.15.83:0/3571467801 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fde88066f60 0x7fde88069420 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdea4152d80 tx=0x7fde94009040 comp rx=0 tx=0).stop 2024-04-17T10:38:48.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.739+0000 7fdeac205640 1 -- 172.21.15.83:0/3571467801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdea40cfbe0 msgr2=0x7fdea414cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:48.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.739+0000 7fdeac205640 1 --2- 172.21.15.83:0/3571467801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdea40cfbe0 0x7fdea414cfc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fde9c00c7f0 tx=0x7fde9c00ccb0 comp rx=0 tx=0).stop 2024-04-17T10:38:48.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.739+0000 7fdeac205640 1 -- 172.21.15.83:0/3571467801 shutdown_connections 2024-04-17T10:38:48.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.739+0000 7fdeac205640 1 --2- 172.21.15.83:0/3571467801 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fde88066f60 0x7fde88069420 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:48.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.739+0000 7fdeac205640 1 --2- 172.21.15.83:0/3571467801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdea415c960 0x7fdea414b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:48.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.739+0000 7fdeac205640 1 --2- 172.21.15.83:0/3571467801 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdea415ba30 0x7fdea4151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:48.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.739+0000 7fdeac205640 1 --2- 172.21.15.83:0/3571467801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdea40cfbe0 0x7fdea414cfc0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:48.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.739+0000 7fdeac205640 1 -- 172.21.15.83:0/3571467801 >> 172.21.15.83:0/3571467801 conn(0x7fdea406abe0 msgr2=0x7fdea41525f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:48.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.739+0000 7fdeac205640 1 -- 172.21.15.83:0/3571467801 shutdown_connections 2024-04-17T10:38:48.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.739+0000 7fdeac205640 1 -- 172.21.15.83:0/3571467801 wait complete. 2024-04-17T10:38:48.754 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:38:10.464490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:38:11.043134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:38:48.755 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:38:48.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.867+0000 7f749d1a0640 1 Processor -- start 2024-04-17T10:38:48.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.867+0000 7f749d1a0640 1 -- start start 2024-04-17T10:38:48.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.867+0000 7f749d1a0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f749815e0f0 0x7f74980b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:48.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.867+0000 7f749d1a0640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f74980b5450 0x7f74980b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:48.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.867+0000 7f749d1a0640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f749815e4d0 con 0x7f74980ce400 2024-04-17T10:38:48.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.867+0000 7f749d1a0640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7498162b40 con 0x7f749815e0f0 2024-04-17T10:38:48.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.867+0000 7f749d1a0640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7498162320 con 0x7f74980b5450 2024-04-17T10:38:48.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.867+0000 7f7496d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f749815e0f0 0x7f74980b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:48.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.867+0000 7f7496d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f749815e0f0 0x7f74980b6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51442/0 (socket says 172.21.15.83:51442) 2024-04-17T10:38:48.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.867+0000 7f7496d9d640 1 -- 172.21.15.83:0/1994172307 learned_addr learned my addr 172.21.15.83:0/1994172307 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:48.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.867+0000 7f7496d9d640 1 -- 172.21.15.83:0/1994172307 >> v2:172.21.15.167:3300/0 conn(0x7f74980b5450 msgr2=0x7f74980b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:48.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.867+0000 7f7496d9d640 1 --2- 172.21.15.83:0/1994172307 >> v2:172.21.15.167:3300/0 conn(0x7f74980b5450 0x7f74980b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:48.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.867+0000 7f7496d9d640 1 -- 172.21.15.83:0/1994172307 >> v1:172.21.15.167:6789/0 conn(0x7f74980ce400 legacy=0x7f7498161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:48.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.867+0000 7f7496d9d640 1 -- 172.21.15.83:0/1994172307 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7498162580 con 0x7f749815e0f0 2024-04-17T10:38:48.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f7496d9d640 1 --2- 172.21.15.83:0/1994172307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f749815e0f0 0x7f74980b6d00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f74980d11a0 tx=0x7f748801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb88a2ebc0f644b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:48.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f7495d9b640 1 -- 172.21.15.83:0/1994172307 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74880b0070 con 0x7f749815e0f0 2024-04-17T10:38:48.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f7495d9b640 1 -- 172.21.15.83:0/1994172307 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74880ab600 con 0x7f749815e0f0 2024-04-17T10:38:48.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f7495d9b640 1 -- 172.21.15.83:0/1994172307 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74880ab9d0 con 0x7f749815e0f0 2024-04-17T10:38:48.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f749d1a0640 1 -- 172.21.15.83:0/1994172307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f749815e0f0 msgr2=0x7f74980b6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:48.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f749d1a0640 1 --2- 172.21.15.83:0/1994172307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f749815e0f0 0x7f74980b6d00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f74980d11a0 tx=0x7f748801c450 comp rx=0 tx=0).stop 2024-04-17T10:38:48.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f749d1a0640 1 -- 172.21.15.83:0/1994172307 shutdown_connections 2024-04-17T10:38:48.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f749d1a0640 1 --2- 172.21.15.83:0/1994172307 >> v2:172.21.15.167:3300/0 conn(0x7f74980b5450 0x7f74980b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:48.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f749d1a0640 1 --2- 172.21.15.83:0/1994172307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f749815e0f0 0x7f74980b6d00 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:48.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f749d1a0640 1 -- 172.21.15.83:0/1994172307 >> 172.21.15.83:0/1994172307 conn(0x7f74980d0450 msgr2=0x7f74980d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:48.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f749d1a0640 1 -- 172.21.15.83:0/1994172307 shutdown_connections 2024-04-17T10:38:48.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f749d1a0640 1 -- 172.21.15.83:0/1994172307 wait complete. 2024-04-17T10:38:48.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f749d1a0640 1 Processor -- start 2024-04-17T10:38:48.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f749d1a0640 1 -- start start 2024-04-17T10:38:48.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f749d1a0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74980b5450 0x7f749814d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:48.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f749d1a0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74980ce400 0x7f74980c6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:48.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f749d1a0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f749815e0f0 0x7f74980c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:48.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f749d1a0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74980d2ed0 con 0x7f74980b5450 2024-04-17T10:38:48.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f749d1a0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74980d2d50 con 0x7f74980ce400 2024-04-17T10:38:48.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f749d1a0640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74980d3050 con 0x7f749815e0f0 2024-04-17T10:38:48.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f7496d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74980b5450 0x7f749814d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:48.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f7496d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74980b5450 0x7f749814d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51456/0 (socket says 172.21.15.83:51456) 2024-04-17T10:38:48.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f7496d9d640 1 -- 172.21.15.83:0/699951892 learned_addr learned my addr 172.21.15.83:0/699951892 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:48.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f7496d9d640 1 -- 172.21.15.83:0/699951892 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f749815e0f0 msgr2=0x7f74980c68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:48.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f7496d9d640 1 --2- 172.21.15.83:0/699951892 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f749815e0f0 0x7f74980c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:48.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f7496d9d640 1 -- 172.21.15.83:0/699951892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74980ce400 msgr2=0x7f74980c6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:48.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f7496d9d640 1 --2- 172.21.15.83:0/699951892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74980ce400 0x7f74980c6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:48.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f7496d9d640 1 -- 172.21.15.83:0/699951892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749814ce80 con 0x7f74980b5450 2024-04-17T10:38:48.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f7496d9d640 1 --2- 172.21.15.83:0/699951892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74980b5450 0x7f749814d160 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f74880ae040 tx=0x7f748801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:48.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f747f7fe640 1 -- 172.21.15.83:0/699951892 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74880ba020 con 0x7f74980b5450 2024-04-17T10:38:48.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f747f7fe640 1 -- 172.21.15.83:0/699951892 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74880b6620 con 0x7f74980b5450 2024-04-17T10:38:48.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f749d1a0640 1 -- 172.21.15.83:0/699951892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74980cbaf0 con 0x7f74980b5450 2024-04-17T10:38:48.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f747f7fe640 1 -- 172.21.15.83:0/699951892 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74880b0040 con 0x7f74980b5450 2024-04-17T10:38:48.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.871+0000 7f749d1a0640 1 -- 172.21.15.83:0/699951892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74980cc0b0 con 0x7f74980b5450 2024-04-17T10:38:48.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.875+0000 7f749d1a0640 1 -- 172.21.15.83:0/699951892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74980681a0 con 0x7f74980b5450 2024-04-17T10:38:48.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.875+0000 7f747f7fe640 1 -- 172.21.15.83:0/699951892 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f74880bd020 con 0x7f74980b5450 2024-04-17T10:38:48.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.875+0000 7f747f7fe640 1 --2- 172.21.15.83:0/699951892 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7484066f60 0x7f7484069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:48.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.875+0000 7f747f7fe640 1 -- 172.21.15.83:0/699951892 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f748811f1d0 con 0x7f74980b5450 2024-04-17T10:38:48.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.875+0000 7f749659c640 1 --2- 172.21.15.83:0/699951892 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7484066f60 0x7f7484069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:48.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.875+0000 7f749659c640 1 --2- 172.21.15.83:0/699951892 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7484066f60 0x7f7484069420 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f748c007fd0 tx=0x7f748c0079a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:48.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:48.879+0000 7f747f7fe640 1 -- 172.21.15.83:0/699951892 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f74880c4030 con 0x7f74980b5450 2024-04-17T10:38:49.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.023+0000 7f749d1a0640 1 -- 172.21.15.83:0/699951892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7498074f90 con 0x7f74980b5450 2024-04-17T10:38:49.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.023+0000 7f747f7fe640 1 -- 172.21.15.83:0/699951892 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f74880160d0 con 0x7f74980b5450 2024-04-17T10:38:49.022 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:38:49.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.027+0000 7f749d1a0640 1 -- 172.21.15.83:0/699951892 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7484066f60 msgr2=0x7f7484069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:49.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.027+0000 7f749d1a0640 1 --2- 172.21.15.83:0/699951892 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7484066f60 0x7f7484069420 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f748c007fd0 tx=0x7f748c0079a0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.027+0000 7f749d1a0640 1 -- 172.21.15.83:0/699951892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74980b5450 msgr2=0x7f749814d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:49.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.027+0000 7f749d1a0640 1 --2- 172.21.15.83:0/699951892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74980b5450 0x7f749814d160 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f74880ae040 tx=0x7f748801ac90 comp rx=0 tx=0).stop 2024-04-17T10:38:49.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.027+0000 7f749d1a0640 1 -- 172.21.15.83:0/699951892 shutdown_connections 2024-04-17T10:38:49.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.027+0000 7f749d1a0640 1 --2- 172.21.15.83:0/699951892 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7484066f60 0x7f7484069420 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.027+0000 7f749d1a0640 1 --2- 172.21.15.83:0/699951892 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f749815e0f0 0x7f74980c68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.027+0000 7f749d1a0640 1 --2- 172.21.15.83:0/699951892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74980ce400 0x7f74980c6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.027+0000 7f749d1a0640 1 --2- 172.21.15.83:0/699951892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74980b5450 0x7f749814d160 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.027+0000 7f749d1a0640 1 -- 172.21.15.83:0/699951892 >> 172.21.15.83:0/699951892 conn(0x7f74980d0450 msgr2=0x7f74980b7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:49.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.027+0000 7f749d1a0640 1 -- 172.21.15.83:0/699951892 shutdown_connections 2024-04-17T10:38:49.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.027+0000 7f749d1a0640 1 -- 172.21.15.83:0/699951892 wait complete. 2024-04-17T10:38:49.041 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:38:49.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.159+0000 7f5d912a4640 1 Processor -- start 2024-04-17T10:38:49.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.159+0000 7f5d912a4640 1 -- start start 2024-04-17T10:38:49.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.163+0000 7f5d912a4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5d8c154de0 0x7f5d8c06ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:49.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.163+0000 7f5d912a4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d8c06b350 0x7f5d8c0b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:49.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.163+0000 7f5d912a4640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d8c15bd70 con 0x7f5d8c0b4040 2024-04-17T10:38:49.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.163+0000 7f5d912a4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d8c0b4870 con 0x7f5d8c154de0 2024-04-17T10:38:49.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.163+0000 7f5d912a4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c15bf60 con 0x7f5d8c06b350 2024-04-17T10:38:49.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.163+0000 7f5d8affd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5d8c154de0 0x7f5d8c06ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:49.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.163+0000 7f5d8b7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5d8c0b4040 0x7f5d8c071580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40038/0 (socket says 172.21.15.83:40038) 2024-04-17T10:38:49.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.163+0000 7f5d8b7fe640 1 -- 172.21.15.83:0/200628972 learned_addr learned my addr 172.21.15.83:0/200628972 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:49.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.163+0000 7f5d8a7fc640 1 --2- 172.21.15.83:0/200628972 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d8c06b350 0x7f5d8c0b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:49.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.163+0000 7f5d8affd640 1 -- 172.21.15.83:0/200628972 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d8c06b350 msgr2=0x7f5d8c0b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:49.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.163+0000 7f5d8affd640 1 --2- 172.21.15.83:0/200628972 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d8c06b350 0x7f5d8c0b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.163+0000 7f5d8affd640 1 -- 172.21.15.83:0/200628972 >> v1:172.21.15.83:6789/0 conn(0x7f5d8c0b4040 legacy=0x7f5d8c071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:49.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.163+0000 7f5d8affd640 1 -- 172.21.15.83:0/200628972 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8c15afb0 con 0x7f5d8c154de0 2024-04-17T10:38:49.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.163+0000 7f5d89ffb640 1 -- 172.21.15.83:0/200628972 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 658132281 0 0) 0x7f5d8c15bd70 con 0x7f5d8c0b4040 2024-04-17T10:38:49.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.163+0000 7f5d8affd640 1 --2- 172.21.15.83:0/200628972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d8c154de0 0x7f5d8c06ae10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0097c0 tx=0x7f5d7c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3234863be3e03824 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:49.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.163+0000 7f5d89ffb640 1 -- 172.21.15.83:0/200628972 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c0b0070 con 0x7f5d8c154de0 2024-04-17T10:38:49.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.163+0000 7f5d89ffb640 1 -- 172.21.15.83:0/200628972 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7c0ab600 con 0x7f5d8c154de0 2024-04-17T10:38:49.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.163+0000 7f5d89ffb640 1 -- 172.21.15.83:0/200628972 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c0ab9d0 con 0x7f5d8c154de0 2024-04-17T10:38:49.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.163+0000 7f5d912a4640 1 -- 172.21.15.83:0/200628972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d8c154de0 msgr2=0x7f5d8c06ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:49.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.163+0000 7f5d912a4640 1 --2- 172.21.15.83:0/200628972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d8c154de0 0x7f5d8c06ae10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0097c0 tx=0x7f5d7c01c4a0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.163+0000 7f5d912a4640 1 -- 172.21.15.83:0/200628972 shutdown_connections 2024-04-17T10:38:49.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.163+0000 7f5d912a4640 1 --2- 172.21.15.83:0/200628972 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d8c06b350 0x7f5d8c0b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.163+0000 7f5d912a4640 1 --2- 172.21.15.83:0/200628972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d8c154de0 0x7f5d8c06ae10 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.163+0000 7f5d912a4640 1 -- 172.21.15.83:0/200628972 >> 172.21.15.83:0/200628972 conn(0x7f5d8c0b15b0 msgr2=0x7f5d8c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:49.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.163+0000 7f5d912a4640 1 -- 172.21.15.83:0/200628972 shutdown_connections 2024-04-17T10:38:49.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.163+0000 7f5d912a4640 1 -- 172.21.15.83:0/200628972 wait complete. 2024-04-17T10:38:49.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.167+0000 7f5d912a4640 1 Processor -- start 2024-04-17T10:38:49.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.167+0000 7f5d912a4640 1 -- start start 2024-04-17T10:38:49.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.167+0000 7f5d912a4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d8c06b350 0x7f5d8c135620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:49.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.167+0000 7f5d912a4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d8c0b4040 0x7f5d8c14f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:49.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.167+0000 7f5d912a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d8c154de0 0x7f5d8c146350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:49.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.167+0000 7f5d912a4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c1551c0 con 0x7f5d8c154de0 2024-04-17T10:38:49.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.167+0000 7f5d912a4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c15b850 con 0x7f5d8c0b4040 2024-04-17T10:38:49.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.167+0000 7f5d912a4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c0b4490 con 0x7f5d8c06b350 2024-04-17T10:38:49.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.167+0000 7f5d8b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d8c154de0 0x7f5d8c146350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:49.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.167+0000 7f5d8b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d8c154de0 0x7f5d8c146350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51470/0 (socket says 172.21.15.83:51470) 2024-04-17T10:38:49.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.167+0000 7f5d8b7fe640 1 -- 172.21.15.83:0/2925685166 learned_addr learned my addr 172.21.15.83:0/2925685166 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:49.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.167+0000 7f5d8affd640 1 --2- 172.21.15.83:0/2925685166 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d8c06b350 0x7f5d8c135620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:49.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.167+0000 7f5d8a7fc640 1 --2- 172.21.15.83:0/2925685166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d8c0b4040 0x7f5d8c14f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:49.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.167+0000 7f5d8b7fe640 1 -- 172.21.15.83:0/2925685166 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d8c06b350 msgr2=0x7f5d8c135620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:49.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.167+0000 7f5d8b7fe640 1 --2- 172.21.15.83:0/2925685166 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d8c06b350 0x7f5d8c135620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.167+0000 7f5d8b7fe640 1 -- 172.21.15.83:0/2925685166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d8c0b4040 msgr2=0x7f5d8c14f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:49.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.167+0000 7f5d8b7fe640 1 --2- 172.21.15.83:0/2925685166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d8c0b4040 0x7f5d8c14f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.167+0000 7f5d8b7fe640 1 -- 172.21.15.83:0/2925685166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8c14f8d0 con 0x7f5d8c154de0 2024-04-17T10:38:49.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.167+0000 7f5d8a7fc640 1 --2- 172.21.15.83:0/2925685166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d8c0b4040 0x7f5d8c14f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:49.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.167+0000 7f5d8b7fe640 1 --2- 172.21.15.83:0/2925685166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d8c154de0 0x7f5d8c146350 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400c730 tx=0x7f5d8400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:49.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.167+0000 7f5d737fe640 1 -- 172.21.15.83:0/2925685166 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84018070 con 0x7f5d8c154de0 2024-04-17T10:38:49.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.167+0000 7f5d737fe640 1 -- 172.21.15.83:0/2925685166 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8400ce40 con 0x7f5d8c154de0 2024-04-17T10:38:49.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.167+0000 7f5d912a4640 1 -- 172.21.15.83:0/2925685166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d8c14fbb0 con 0x7f5d8c154de0 2024-04-17T10:38:49.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.167+0000 7f5d737fe640 1 -- 172.21.15.83:0/2925685166 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8400a860 con 0x7f5d8c154de0 2024-04-17T10:38:49.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.167+0000 7f5d912a4640 1 -- 172.21.15.83:0/2925685166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d8c150170 con 0x7f5d8c154de0 2024-04-17T10:38:49.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.167+0000 7f5d912a4640 1 -- 172.21.15.83:0/2925685166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d8c074f90 con 0x7f5d8c154de0 2024-04-17T10:38:49.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.171+0000 7f5d737fe640 1 -- 172.21.15.83:0/2925685166 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5d8401b070 con 0x7f5d8c154de0 2024-04-17T10:38:49.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.171+0000 7f5d737fe640 1 --2- 172.21.15.83:0/2925685166 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5d78066f60 0x7f5d78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:49.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.171+0000 7f5d737fe640 1 -- 172.21.15.83:0/2925685166 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d84084540 con 0x7f5d8c154de0 2024-04-17T10:38:49.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.171+0000 7f5d8affd640 1 --2- 172.21.15.83:0/2925685166 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5d78066f60 0x7f5d78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:49.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.171+0000 7f5d8affd640 1 --2- 172.21.15.83:0/2925685166 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5d78066f60 0x7f5d78069420 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0ad000 tx=0x7f5d7c0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:49.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.175+0000 7f5d737fe640 1 -- 172.21.15.83:0/2925685166 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5d840555e0 con 0x7f5d8c154de0 2024-04-17T10:38:49.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.323+0000 7f5d912a4640 1 -- 172.21.15.83:0/2925685166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5d8c0b3280 con 0x7f5d8c154de0 2024-04-17T10:38:49.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.323+0000 7f5d737fe640 1 -- 172.21.15.83:0/2925685166 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5d84054ee0 con 0x7f5d8c154de0 2024-04-17T10:38:49.321 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:38:49.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.323+0000 7f5d912a4640 1 -- 172.21.15.83:0/2925685166 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5d78066f60 msgr2=0x7f5d78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:49.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.323+0000 7f5d912a4640 1 --2- 172.21.15.83:0/2925685166 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5d78066f60 0x7f5d78069420 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0ad000 tx=0x7f5d7c0a6000 comp rx=0 tx=0).stop 2024-04-17T10:38:49.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.323+0000 7f5d912a4640 1 -- 172.21.15.83:0/2925685166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d8c154de0 msgr2=0x7f5d8c146350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:49.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.323+0000 7f5d912a4640 1 --2- 172.21.15.83:0/2925685166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d8c154de0 0x7f5d8c146350 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400c730 tx=0x7f5d8400cbf0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.323+0000 7f5d912a4640 1 -- 172.21.15.83:0/2925685166 shutdown_connections 2024-04-17T10:38:49.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.323+0000 7f5d912a4640 1 --2- 172.21.15.83:0/2925685166 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5d78066f60 0x7f5d78069420 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.323+0000 7f5d912a4640 1 --2- 172.21.15.83:0/2925685166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d8c154de0 0x7f5d8c146350 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.327+0000 7f5d912a4640 1 --2- 172.21.15.83:0/2925685166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d8c0b4040 0x7f5d8c14f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.327+0000 7f5d912a4640 1 --2- 172.21.15.83:0/2925685166 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d8c06b350 0x7f5d8c135620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.327+0000 7f5d912a4640 1 -- 172.21.15.83:0/2925685166 >> 172.21.15.83:0/2925685166 conn(0x7f5d8c0b15b0 msgr2=0x7f5d8c06d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:49.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.327+0000 7f5d912a4640 1 -- 172.21.15.83:0/2925685166 shutdown_connections 2024-04-17T10:38:49.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.327+0000 7f5d912a4640 1 -- 172.21.15.83:0/2925685166 wait complete. 2024-04-17T10:38:49.339 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:38:49.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.459+0000 7f6cca7e5640 1 Processor -- start 2024-04-17T10:38:49.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.459+0000 7f6cca7e5640 1 -- start start 2024-04-17T10:38:49.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cca7e5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc406b520 0x7f6cc4071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:49.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cca7e5640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cc406c1f0 con 0x7f6cc415bac0 2024-04-17T10:38:49.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cca7e5640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cc4070a10 con 0x7f6cc415b6e0 2024-04-17T10:38:49.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cca7e5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc4070cf0 con 0x7f6cc406b520 2024-04-17T10:38:49.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cc37fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6cc415bac0 0x7f6cc406ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40042/0 (socket says 172.21.15.83:40042) 2024-04-17T10:38:49.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cc37fe640 1 -- 172.21.15.83:0/2076327312 learned_addr learned my addr 172.21.15.83:0/2076327312 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:49.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cc8d82640 1 --2- 172.21.15.83:0/2076327312 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc406b520 0x7f6cc4071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:49.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cc2ffd640 1 -- 172.21.15.83:0/2076327312 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3592229024 0 0) 0x7f6cc406c1f0 con 0x7f6cc415bac0 2024-04-17T10:38:49.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cc2ffd640 1 -- 172.21.15.83:0/2076327312 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cac003530 con 0x7f6cc415bac0 2024-04-17T10:38:49.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cc2ffd640 1 -- 172.21.15.83:0/2076327312 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3982430647 0 0) 0x7f6cac003530 con 0x7f6cc415bac0 2024-04-17T10:38:49.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cc2ffd640 1 -- 172.21.15.83:0/2076327312 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc406b520 msgr2=0x7f6cc4071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:49.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cc2ffd640 1 --2- 172.21.15.83:0/2076327312 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc406b520 0x7f6cc4071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cc2ffd640 1 -- 172.21.15.83:0/2076327312 >> v1:172.21.15.167:6789/0 conn(0x7f6cc415b6e0 legacy=0x7f6cc40cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:49.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cc2ffd640 1 -- 172.21.15.83:0/2076327312 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc4070fd0 con 0x7f6cc415bac0 2024-04-17T10:38:49.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cc2ffd640 1 -- 172.21.15.83:0/2076327312 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f6cb8001680 con 0x7f6cc415bac0 2024-04-17T10:38:49.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cc2ffd640 1 -- 172.21.15.83:0/2076327312 >> v1:172.21.15.83:6789/0 conn(0x7f6cc415bac0 legacy=0x7f6cc406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:49.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cc2ffd640 1 --2- 172.21.15.83:0/2076327312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cac003a60 0x7f6cac003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:49.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cc2ffd640 1 --2- 172.21.15.83:0/2076327312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cac004810 0x7f6cac004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:49.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cc2ffd640 1 --2- 172.21.15.83:0/2076327312 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc406b520 0x7f6cac005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:49.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cc2ffd640 1 -- 172.21.15.83:0/2076327312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc4070cf0 con 0x7f6cac004810 2024-04-17T10:38:49.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cc2ffd640 1 -- 172.21.15.83:0/2076327312 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac00bbc0 con 0x7f6cac003a60 2024-04-17T10:38:49.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cc2ffd640 1 -- 172.21.15.83:0/2076327312 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac00bdc0 con 0x7f6cc406b520 2024-04-17T10:38:49.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cc37fe640 1 --2- 172.21.15.83:0/2076327312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cac004810 0x7f6cac004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:49.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cc3fff640 1 --2- 172.21.15.83:0/2076327312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cac003a60 0x7f6cac003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:49.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cc37fe640 1 -- 172.21.15.83:0/2076327312 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc406b520 msgr2=0x7f6cac005280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:49.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cc8d82640 1 --2- 172.21.15.83:0/2076327312 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc406b520 0x7f6cac005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:49.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cc37fe640 1 --2- 172.21.15.83:0/2076327312 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc406b520 0x7f6cac005280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cc37fe640 1 -- 172.21.15.83:0/2076327312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cac003a60 msgr2=0x7f6cac003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:49.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cc37fe640 1 --2- 172.21.15.83:0/2076327312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cac003a60 0x7f6cac003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cc37fe640 1 -- 172.21.15.83:0/2076327312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cac003530 con 0x7f6cac004810 2024-04-17T10:38:49.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cc3fff640 1 --2- 172.21.15.83:0/2076327312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cac003a60 0x7f6cac003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:49.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cc37fe640 1 --2- 172.21.15.83:0/2076327312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cac004810 0x7f6cac004c90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6cb800c240 tx=0x7f6cb801f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d000d8200eda3f19 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:49.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cc2ffd640 1 -- 172.21.15.83:0/2076327312 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8007d20 con 0x7f6cac004810 2024-04-17T10:38:49.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cc2ffd640 1 -- 172.21.15.83:0/2076327312 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb8005ae0 con 0x7f6cac004810 2024-04-17T10:38:49.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.463+0000 7f6cc2ffd640 1 -- 172.21.15.83:0/2076327312 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8007070 con 0x7f6cac004810 2024-04-17T10:38:49.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cca7e5640 1 -- 172.21.15.83:0/2076327312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cac004810 msgr2=0x7f6cac004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:49.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cca7e5640 1 --2- 172.21.15.83:0/2076327312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cac004810 0x7f6cac004c90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6cb800c240 tx=0x7f6cb801f3f0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cc8d82640 1 -- 172.21.15.83:0/2076327312 reap_dead start 2024-04-17T10:38:49.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cca7e5640 1 -- 172.21.15.83:0/2076327312 shutdown_connections 2024-04-17T10:38:49.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cca7e5640 1 -- 172.21.15.83:0/2076327312 >> 172.21.15.83:0/2076327312 conn(0x7f6cc40b15b0 msgr2=0x7f6cc40b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:49.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cca7e5640 1 -- 172.21.15.83:0/2076327312 shutdown_connections 2024-04-17T10:38:49.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cca7e5640 1 -- 172.21.15.83:0/2076327312 wait complete. 2024-04-17T10:38:49.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cca7e5640 1 Processor -- start 2024-04-17T10:38:49.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cca7e5640 1 -- start start 2024-04-17T10:38:49.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cca7e5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cac004810 0x7f6cc40ac040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:49.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cca7e5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc40b27e0 0x7f6cc4148f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:49.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cca7e5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cc40b2c40 0x7f6cc41494a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:49.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cca7e5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc40705f0 con 0x7f6cac004810 2024-04-17T10:38:49.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cca7e5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc4070470 con 0x7f6cc40b2c40 2024-04-17T10:38:49.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cca7e5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc4070770 con 0x7f6cc40b27e0 2024-04-17T10:38:49.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cc3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cac004810 0x7f6cc40ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:49.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cc3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cac004810 0x7f6cc40ac040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51488/0 (socket says 172.21.15.83:51488) 2024-04-17T10:38:49.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cc3fff640 1 -- 172.21.15.83:0/69740291 learned_addr learned my addr 172.21.15.83:0/69740291 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:49.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cc37fe640 1 --2- 172.21.15.83:0/69740291 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc40b27e0 0x7f6cc4148f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:49.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cc3fff640 1 -- 172.21.15.83:0/69740291 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc40b27e0 msgr2=0x7f6cc4148f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:49.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cc8d82640 1 --2- 172.21.15.83:0/69740291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cc40b2c40 0x7f6cc41494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:49.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cc3fff640 1 --2- 172.21.15.83:0/69740291 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc40b27e0 0x7f6cc4148f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cc3fff640 1 -- 172.21.15.83:0/69740291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cc40b2c40 msgr2=0x7f6cc41494a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:49.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cc3fff640 1 --2- 172.21.15.83:0/69740291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cc40b2c40 0x7f6cc41494a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cc3fff640 1 -- 172.21.15.83:0/69740291 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc40b31c0 con 0x7f6cac004810 2024-04-17T10:38:49.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cc37fe640 1 --2- 172.21.15.83:0/69740291 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc40b27e0 0x7f6cc4148f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:49.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cc3fff640 1 --2- 172.21.15.83:0/69740291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cac004810 0x7f6cc40ac040 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400d770 tx=0x7f6cb400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:49.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cc0ff9640 1 -- 172.21.15.83:0/69740291 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4005270 con 0x7f6cac004810 2024-04-17T10:38:49.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cc0ff9640 1 -- 172.21.15.83:0/69740291 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb4017070 con 0x7f6cac004810 2024-04-17T10:38:49.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cca7e5640 1 -- 172.21.15.83:0/69740291 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cc40b34a0 con 0x7f6cac004810 2024-04-17T10:38:49.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cc0ff9640 1 -- 172.21.15.83:0/69740291 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb400b6b0 con 0x7f6cac004810 2024-04-17T10:38:49.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cca7e5640 1 -- 172.21.15.83:0/69740291 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc40b3a60 con 0x7f6cac004810 2024-04-17T10:38:49.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.467+0000 7f6cca7e5640 1 -- 172.21.15.83:0/69740291 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cc4074f90 con 0x7f6cac004810 2024-04-17T10:38:49.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.471+0000 7f6cc0ff9640 1 -- 172.21.15.83:0/69740291 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6cb400b850 con 0x7f6cac004810 2024-04-17T10:38:49.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.471+0000 7f6cc0ff9640 1 --2- 172.21.15.83:0/69740291 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6cb0066f60 0x7f6cb0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:49.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.471+0000 7f6cc37fe640 1 --2- 172.21.15.83:0/69740291 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6cb0066f60 0x7f6cb0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:49.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.471+0000 7f6cc0ff9640 1 -- 172.21.15.83:0/69740291 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6cb4084f80 con 0x7f6cac004810 2024-04-17T10:38:49.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.471+0000 7f6cc37fe640 1 --2- 172.21.15.83:0/69740291 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6cb0066f60 0x7f6cb0069420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80af000 tx=0x7f6cb80aa000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:49.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.475+0000 7f6cc0ff9640 1 -- 172.21.15.83:0/69740291 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6cb4019310 con 0x7f6cac004810 2024-04-17T10:38:49.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.627+0000 7f6cca7e5640 1 -- 172.21.15.83:0/69740291 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6cc406b030 con 0x7f6cac004810 2024-04-17T10:38:49.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.627+0000 7f6cc0ff9640 1 -- 172.21.15.83:0/69740291 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6cb4055f90 con 0x7f6cac004810 2024-04-17T10:38:49.626 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:38:49.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.631+0000 7f6cca7e5640 1 -- 172.21.15.83:0/69740291 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6cb0066f60 msgr2=0x7f6cb0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:49.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.631+0000 7f6cca7e5640 1 --2- 172.21.15.83:0/69740291 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6cb0066f60 0x7f6cb0069420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80af000 tx=0x7f6cb80aa000 comp rx=0 tx=0).stop 2024-04-17T10:38:49.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.631+0000 7f6cca7e5640 1 -- 172.21.15.83:0/69740291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cac004810 msgr2=0x7f6cc40ac040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:49.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.631+0000 7f6cca7e5640 1 --2- 172.21.15.83:0/69740291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cac004810 0x7f6cc40ac040 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400d770 tx=0x7f6cb400dc30 comp rx=0 tx=0).stop 2024-04-17T10:38:49.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.631+0000 7f6cca7e5640 1 -- 172.21.15.83:0/69740291 shutdown_connections 2024-04-17T10:38:49.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.631+0000 7f6cca7e5640 1 --2- 172.21.15.83:0/69740291 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6cb0066f60 0x7f6cb0069420 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.631+0000 7f6cca7e5640 1 --2- 172.21.15.83:0/69740291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cc40b2c40 0x7f6cc41494a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.631+0000 7f6cca7e5640 1 --2- 172.21.15.83:0/69740291 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc40b27e0 0x7f6cc4148f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.631+0000 7f6cca7e5640 1 --2- 172.21.15.83:0/69740291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cac004810 0x7f6cc40ac040 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.631+0000 7f6cca7e5640 1 -- 172.21.15.83:0/69740291 >> 172.21.15.83:0/69740291 conn(0x7f6cc40b15b0 msgr2=0x7f6cc4155a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:49.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.631+0000 7f6cca7e5640 1 -- 172.21.15.83:0/69740291 shutdown_connections 2024-04-17T10:38:49.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.631+0000 7f6cca7e5640 1 -- 172.21.15.83:0/69740291 wait complete. 2024-04-17T10:38:49.645 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:38:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.755+0000 7fcd42139640 1 Processor -- start 2024-04-17T10:38:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.755+0000 7fcd42139640 1 -- start start 2024-04-17T10:38:49.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.755+0000 7fcd42139640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcd3c06ae10 0x7fcd3c06b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:49.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.755+0000 7fcd42139640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd3c15e330 0x7fcd3c15e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:49.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.755+0000 7fcd42139640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcd3c0ad840 0x7fcd3c0adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:49.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.755+0000 7fcd42139640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd3c1625d0 con 0x7fcd3c06ae10 2024-04-17T10:38:49.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.755+0000 7fcd42139640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd3c06b730 con 0x7fcd3c0ad840 2024-04-17T10:38:49.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.755+0000 7fcd42139640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c161810 con 0x7fcd3c15e330 2024-04-17T10:38:49.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.755+0000 7fcd3b7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcd3c06ae10 0x7fcd3c06b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:49.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.755+0000 7fcd3b7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcd3c06ae10 0x7fcd3c06b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51500/0 (socket says 172.21.15.83:51500) 2024-04-17T10:38:49.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.755+0000 7fcd3b7fe640 1 -- 172.21.15.83:0/3289555781 learned_addr learned my addr 172.21.15.83:0/3289555781 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:49.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.755+0000 7fcd3bfff640 1 --2- 172.21.15.83:0/3289555781 >> v2:172.21.15.167:3300/0 conn(0x7fcd3c0ad840 0x7fcd3c0adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:49.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.755+0000 7fcd33fff640 1 --2- 172.21.15.83:0/3289555781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd3c15e330 0x7fcd3c15e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:49.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.755+0000 7fcd3b7fe640 1 -- 172.21.15.83:0/3289555781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd3c15e330 msgr2=0x7fcd3c15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:49.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.755+0000 7fcd3b7fe640 1 --2- 172.21.15.83:0/3289555781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd3c15e330 0x7fcd3c15e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.755+0000 7fcd3b7fe640 1 -- 172.21.15.83:0/3289555781 >> v2:172.21.15.167:3300/0 conn(0x7fcd3c0ad840 msgr2=0x7fcd3c0adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:49.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.755+0000 7fcd3b7fe640 1 --2- 172.21.15.83:0/3289555781 >> v2:172.21.15.167:3300/0 conn(0x7fcd3c0ad840 0x7fcd3c0adcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.755+0000 7fcd3b7fe640 1 -- 172.21.15.83:0/3289555781 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3c161af0 con 0x7fcd3c06ae10 2024-04-17T10:38:49.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.755+0000 7fcd3bfff640 1 --2- 172.21.15.83:0/3289555781 >> v2:172.21.15.167:3300/0 conn(0x7fcd3c0ad840 0x7fcd3c0adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:49.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.755+0000 7fcd3b7fe640 1 --2- 172.21.15.83:0/3289555781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd3c06ae10 0x7fcd3c06b1f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c15bc40 tx=0x7fcd2401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18726c569e8430e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:49.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd3affd640 1 -- 172.21.15.83:0/3289555781 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd240b0070 con 0x7fcd3c06ae10 2024-04-17T10:38:49.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd3affd640 1 -- 172.21.15.83:0/3289555781 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd240ab600 con 0x7fcd3c06ae10 2024-04-17T10:38:49.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd3affd640 1 -- 172.21.15.83:0/3289555781 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd240ab9d0 con 0x7fcd3c06ae10 2024-04-17T10:38:49.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd42139640 1 -- 172.21.15.83:0/3289555781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd3c06ae10 msgr2=0x7fcd3c06b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:49.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd42139640 1 --2- 172.21.15.83:0/3289555781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd3c06ae10 0x7fcd3c06b1f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c15bc40 tx=0x7fcd2401c490 comp rx=0 tx=0).stop 2024-04-17T10:38:49.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd42139640 1 -- 172.21.15.83:0/3289555781 shutdown_connections 2024-04-17T10:38:49.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd42139640 1 --2- 172.21.15.83:0/3289555781 >> v2:172.21.15.167:3300/0 conn(0x7fcd3c0ad840 0x7fcd3c0adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd42139640 1 --2- 172.21.15.83:0/3289555781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd3c15e330 0x7fcd3c15e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd42139640 1 --2- 172.21.15.83:0/3289555781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd3c06ae10 0x7fcd3c06b1f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd42139640 1 -- 172.21.15.83:0/3289555781 >> 172.21.15.83:0/3289555781 conn(0x7fcd3c0b15b0 msgr2=0x7fcd3c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:49.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd42139640 1 -- 172.21.15.83:0/3289555781 shutdown_connections 2024-04-17T10:38:49.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd42139640 1 -- 172.21.15.83:0/3289555781 wait complete. 2024-04-17T10:38:49.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd42139640 1 Processor -- start 2024-04-17T10:38:49.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd42139640 1 -- start start 2024-04-17T10:38:49.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd42139640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd3c06ae10 0x7fcd3c137d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:49.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd42139640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd3c0ad840 0x7fcd3c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:49.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd42139640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd3c15e330 0x7fcd3c0afac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:49.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd42139640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c0cf470 con 0x7fcd3c0ad840 2024-04-17T10:38:49.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd42139640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c0cf2f0 con 0x7fcd3c15e330 2024-04-17T10:38:49.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd42139640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c15ecb0 con 0x7fcd3c06ae10 2024-04-17T10:38:49.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd33fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd3c0ad840 0x7fcd3c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:49.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd33fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd3c0ad840 0x7fcd3c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51514/0 (socket says 172.21.15.83:51514) 2024-04-17T10:38:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd33fff640 1 -- 172.21.15.83:0/2726935991 learned_addr learned my addr 172.21.15.83:0/2726935991 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd3bfff640 1 --2- 172.21.15.83:0/2726935991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd3c15e330 0x7fcd3c0afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd3b7fe640 1 --2- 172.21.15.83:0/2726935991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd3c06ae10 0x7fcd3c137d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd33fff640 1 -- 172.21.15.83:0/2726935991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd3c06ae10 msgr2=0x7fcd3c137d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd33fff640 1 --2- 172.21.15.83:0/2726935991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd3c06ae10 0x7fcd3c137d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd33fff640 1 -- 172.21.15.83:0/2726935991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd3c15e330 msgr2=0x7fcd3c0afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd33fff640 1 --2- 172.21.15.83:0/2726935991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd3c15e330 0x7fcd3c0afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd33fff640 1 -- 172.21.15.83:0/2726935991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3c145900 con 0x7fcd3c0ad840 2024-04-17T10:38:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd33fff640 1 --2- 172.21.15.83:0/2726935991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd3c0ad840 0x7fcd3c15d5d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00c7f0 tx=0x7fcd2c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd38ff9640 1 -- 172.21.15.83:0/2726935991 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c015070 con 0x7fcd3c0ad840 2024-04-17T10:38:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd38ff9640 1 -- 172.21.15.83:0/2726935991 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2c007820 con 0x7fcd3c0ad840 2024-04-17T10:38:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd42139640 1 -- 172.21.15.83:0/2726935991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd3c145be0 con 0x7fcd3c0ad840 2024-04-17T10:38:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd38ff9640 1 -- 172.21.15.83:0/2726935991 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c007c50 con 0x7fcd3c0ad840 2024-04-17T10:38:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.759+0000 7fcd42139640 1 -- 172.21.15.83:0/2726935991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd3c1461a0 con 0x7fcd3c0ad840 2024-04-17T10:38:49.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.763+0000 7fcd42139640 1 -- 172.21.15.83:0/2726935991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd3c074f90 con 0x7fcd3c0ad840 2024-04-17T10:38:49.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.763+0000 7fcd38ff9640 1 -- 172.21.15.83:0/2726935991 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcd2c018070 con 0x7fcd3c0ad840 2024-04-17T10:38:49.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.763+0000 7fcd38ff9640 1 --2- 172.21.15.83:0/2726935991 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd1c066f60 0x7fcd1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:49.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.763+0000 7fcd3b7fe640 1 --2- 172.21.15.83:0/2726935991 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd1c066f60 0x7fcd1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:49.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.763+0000 7fcd38ff9640 1 -- 172.21.15.83:0/2726935991 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd2c085210 con 0x7fcd3c0ad840 2024-04-17T10:38:49.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.763+0000 7fcd3b7fe640 1 --2- 172.21.15.83:0/2726935991 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd1c066f60 0x7fcd1c069420 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c0aece0 tx=0x7fcd24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:49.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.767+0000 7fcd38ff9640 1 -- 172.21.15.83:0/2726935991 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcd2c056180 con 0x7fcd3c0ad840 2024-04-17T10:38:49.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.923+0000 7fcd42139640 1 -- 172.21.15.83:0/2726935991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcd3c072740 con 0x7fcd3c0ad840 2024-04-17T10:38:49.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.923+0000 7fcd38ff9640 1 -- 172.21.15.83:0/2726935991 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcd2c055a80 con 0x7fcd3c0ad840 2024-04-17T10:38:49.921 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-17T10:38:49.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.923+0000 7fcd42139640 1 -- 172.21.15.83:0/2726935991 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd1c066f60 msgr2=0x7fcd1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:49.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.923+0000 7fcd42139640 1 --2- 172.21.15.83:0/2726935991 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd1c066f60 0x7fcd1c069420 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c0aece0 tx=0x7fcd24008040 comp rx=0 tx=0).stop 2024-04-17T10:38:49.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.923+0000 7fcd42139640 1 -- 172.21.15.83:0/2726935991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd3c0ad840 msgr2=0x7fcd3c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:49.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.923+0000 7fcd42139640 1 --2- 172.21.15.83:0/2726935991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd3c0ad840 0x7fcd3c15d5d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00c7f0 tx=0x7fcd2c00ccb0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.927+0000 7fcd42139640 1 -- 172.21.15.83:0/2726935991 shutdown_connections 2024-04-17T10:38:49.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.927+0000 7fcd42139640 1 --2- 172.21.15.83:0/2726935991 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd1c066f60 0x7fcd1c069420 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.927+0000 7fcd42139640 1 --2- 172.21.15.83:0/2726935991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd3c15e330 0x7fcd3c0afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.927+0000 7fcd42139640 1 --2- 172.21.15.83:0/2726935991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd3c0ad840 0x7fcd3c15d5d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.927+0000 7fcd42139640 1 --2- 172.21.15.83:0/2726935991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd3c06ae10 0x7fcd3c137d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:49.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.927+0000 7fcd42139640 1 -- 172.21.15.83:0/2726935991 >> 172.21.15.83:0/2726935991 conn(0x7fcd3c0b15b0 msgr2=0x7fcd3c0b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:49.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.927+0000 7fcd42139640 1 -- 172.21.15.83:0/2726935991 shutdown_connections 2024-04-17T10:38:49.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:49.927+0000 7fcd42139640 1 -- 172.21.15.83:0/2726935991 wait complete. 2024-04-17T10:38:49.938 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2024-04-17T10:38:49.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:49.939 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-17T10:38:49.956 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:38:49.956 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-17T10:38:49.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:49.957 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T10:38:50.010 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:38:50.010 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:38:50.016 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:38:50.016 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:38:50.064 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:38:50.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.191+0000 7f5a28e8b640 1 Processor -- start 2024-04-17T10:38:50.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.191+0000 7f5a28e8b640 1 -- start start 2024-04-17T10:38:50.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.191+0000 7f5a28e8b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5a24161300 0x7f5a2406ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:50.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.191+0000 7f5a28e8b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a2406f360 0x7f5a2406bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:50.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.191+0000 7f5a28e8b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5a2406c130 0x7f5a240ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:50.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.191+0000 7f5a28e8b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a24161900 con 0x7f5a2406c130 2024-04-17T10:38:50.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.191+0000 7f5a28e8b640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a24161ba0 con 0x7f5a24161300 2024-04-17T10:38:50.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.191+0000 7f5a28e8b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a24161e80 con 0x7f5a2406f360 2024-04-17T10:38:50.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.191+0000 7f5a22d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5a2406c130 0x7f5a240ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:50.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.191+0000 7f5a22d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5a2406c130 0x7f5a240ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51522/0 (socket says 172.21.15.83:51522) 2024-04-17T10:38:50.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.191+0000 7f5a22d9d640 1 -- 172.21.15.83:0/522225883 learned_addr learned my addr 172.21.15.83:0/522225883 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:50.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.191+0000 7f5a2259c640 1 --2- 172.21.15.83:0/522225883 >> v2:172.21.15.167:3300/0 conn(0x7f5a24161300 0x7f5a2406ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:50.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.191+0000 7f5a19d9b640 1 --2- 172.21.15.83:0/522225883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a2406f360 0x7f5a2406bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:50.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.191+0000 7f5a22d9d640 1 -- 172.21.15.83:0/522225883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a2406f360 msgr2=0x7f5a2406bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:50.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.191+0000 7f5a22d9d640 1 --2- 172.21.15.83:0/522225883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a2406f360 0x7f5a2406bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.191+0000 7f5a22d9d640 1 -- 172.21.15.83:0/522225883 >> v2:172.21.15.167:3300/0 conn(0x7f5a24161300 msgr2=0x7f5a2406ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:50.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.191+0000 7f5a22d9d640 1 --2- 172.21.15.83:0/522225883 >> v2:172.21.15.167:3300/0 conn(0x7f5a24161300 0x7f5a2406ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.191+0000 7f5a22d9d640 1 -- 172.21.15.83:0/522225883 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a24162160 con 0x7f5a2406c130 2024-04-17T10:38:50.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.191+0000 7f5a19d9b640 1 --2- 172.21.15.83:0/522225883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a2406f360 0x7f5a2406bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:50.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.191+0000 7f5a2259c640 1 --2- 172.21.15.83:0/522225883 >> v2:172.21.15.167:3300/0 conn(0x7f5a24161300 0x7f5a2406ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:38:50.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a22d9d640 1 --2- 172.21.15.83:0/522225883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a2406c130 0x7f5a240ce620 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5a24067200 tx=0x7f5a1c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=610d63cd67f26c0d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:50.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a21d9b640 1 -- 172.21.15.83:0/522225883 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c0a55e0 con 0x7f5a2406c130 2024-04-17T10:38:50.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a21d9b640 1 -- 172.21.15.83:0/522225883 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a1c0a5780 con 0x7f5a2406c130 2024-04-17T10:38:50.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a21d9b640 1 -- 172.21.15.83:0/522225883 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c0a5b50 con 0x7f5a2406c130 2024-04-17T10:38:50.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a28e8b640 1 -- 172.21.15.83:0/522225883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a2406c130 msgr2=0x7f5a240ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:50.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a28e8b640 1 --2- 172.21.15.83:0/522225883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a2406c130 0x7f5a240ce620 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5a24067200 tx=0x7f5a1c01c960 comp rx=0 tx=0).stop 2024-04-17T10:38:50.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a28e8b640 1 -- 172.21.15.83:0/522225883 shutdown_connections 2024-04-17T10:38:50.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a28e8b640 1 --2- 172.21.15.83:0/522225883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a2406c130 0x7f5a240ce620 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a28e8b640 1 --2- 172.21.15.83:0/522225883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a2406f360 0x7f5a2406bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a28e8b640 1 --2- 172.21.15.83:0/522225883 >> v2:172.21.15.167:3300/0 conn(0x7f5a24161300 0x7f5a2406ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a28e8b640 1 -- 172.21.15.83:0/522225883 >> 172.21.15.83:0/522225883 conn(0x7f5a240cc910 msgr2=0x7f5a240ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:50.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a28e8b640 1 -- 172.21.15.83:0/522225883 shutdown_connections 2024-04-17T10:38:50.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a28e8b640 1 -- 172.21.15.83:0/522225883 wait complete. 2024-04-17T10:38:50.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a28e8b640 1 Processor -- start 2024-04-17T10:38:50.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a28e8b640 1 -- start start 2024-04-17T10:38:50.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a28e8b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a2406c130 0x7f5a2409a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:50.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a28e8b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a2406f360 0x7f5a2415d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:50.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a28e8b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a24161300 0x7f5a2415da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:50.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a28e8b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a24072330 con 0x7f5a2406c130 2024-04-17T10:38:50.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a28e8b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a240721b0 con 0x7f5a2406f360 2024-04-17T10:38:50.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a28e8b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a240724b0 con 0x7f5a24161300 2024-04-17T10:38:50.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a2259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a2406c130 0x7f5a2409a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:50.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a2259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a2406c130 0x7f5a2409a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51526/0 (socket says 172.21.15.83:51526) 2024-04-17T10:38:50.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a2259c640 1 -- 172.21.15.83:0/2826055756 learned_addr learned my addr 172.21.15.83:0/2826055756 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:50.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a19d9b640 1 --2- 172.21.15.83:0/2826055756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a2406f360 0x7f5a2415d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:50.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a22d9d640 1 --2- 172.21.15.83:0/2826055756 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a24161300 0x7f5a2415da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:50.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a2259c640 1 -- 172.21.15.83:0/2826055756 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a24161300 msgr2=0x7f5a2415da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:50.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a2259c640 1 --2- 172.21.15.83:0/2826055756 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a24161300 0x7f5a2415da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a2259c640 1 -- 172.21.15.83:0/2826055756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a2406f360 msgr2=0x7f5a2415d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:50.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a2259c640 1 --2- 172.21.15.83:0/2826055756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a2406f360 0x7f5a2415d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a2259c640 1 -- 172.21.15.83:0/2826055756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a2409d7f0 con 0x7f5a2406c130 2024-04-17T10:38:50.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a2259c640 1 --2- 172.21.15.83:0/2826055756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a2406c130 0x7f5a2409a9d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5a1400d830 tx=0x7f5a1400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:50.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a1b7fe640 1 -- 172.21.15.83:0/2826055756 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a14018020 con 0x7f5a2406c130 2024-04-17T10:38:50.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a1b7fe640 1 -- 172.21.15.83:0/2826055756 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a140096c0 con 0x7f5a2406c130 2024-04-17T10:38:50.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a28e8b640 1 -- 172.21.15.83:0/2826055756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a2415c440 con 0x7f5a2406c130 2024-04-17T10:38:50.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a1b7fe640 1 -- 172.21.15.83:0/2826055756 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a14009ab0 con 0x7f5a2406c130 2024-04-17T10:38:50.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.195+0000 7f5a28e8b640 1 -- 172.21.15.83:0/2826055756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a2415ca00 con 0x7f5a2406c130 2024-04-17T10:38:50.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.199+0000 7f5a28e8b640 1 -- 172.21.15.83:0/2826055756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a24074f90 con 0x7f5a2406c130 2024-04-17T10:38:50.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.199+0000 7f5a1b7fe640 1 -- 172.21.15.83:0/2826055756 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5a1401b020 con 0x7f5a2406c130 2024-04-17T10:38:50.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.199+0000 7f5a1b7fe640 1 --2- 172.21.15.83:0/2826055756 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5a04066f60 0x7f5a04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:50.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.199+0000 7f5a1b7fe640 1 -- 172.21.15.83:0/2826055756 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a140855b0 con 0x7f5a2406c130 2024-04-17T10:38:50.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.199+0000 7f5a19d9b640 1 --2- 172.21.15.83:0/2826055756 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5a04066f60 0x7f5a04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:50.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.199+0000 7f5a19d9b640 1 --2- 172.21.15.83:0/2826055756 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5a04066f60 0x7f5a04069420 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5a241fa640 tx=0x7f5a0c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:50.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.203+0000 7f5a1b7fe640 1 -- 172.21.15.83:0/2826055756 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5a14022020 con 0x7f5a2406c130 2024-04-17T10:38:50.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.351+0000 7f5a28e8b640 1 -- 172.21.15.83:0/2826055756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a24066d20 con 0x7f5a2406c130 2024-04-17T10:38:50.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.351+0000 7f5a1b7fe640 1 -- 172.21.15.83:0/2826055756 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16533 (secure 0 0 0) 0x7f5a14056540 con 0x7f5a2406c130 2024-04-17T10:38:50.350 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:50.350 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":20,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:38:13.690876+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:38:10.464490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T10:38:11.043134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:38:50.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.355+0000 7f5a28e8b640 1 -- 172.21.15.83:0/2826055756 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5a04066f60 msgr2=0x7f5a04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:50.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.355+0000 7f5a28e8b640 1 --2- 172.21.15.83:0/2826055756 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5a04066f60 0x7f5a04069420 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5a241fa640 tx=0x7f5a0c008040 comp rx=0 tx=0).stop 2024-04-17T10:38:50.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.355+0000 7f5a28e8b640 1 -- 172.21.15.83:0/2826055756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a2406c130 msgr2=0x7f5a2409a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:50.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.355+0000 7f5a28e8b640 1 --2- 172.21.15.83:0/2826055756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a2406c130 0x7f5a2409a9d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5a1400d830 tx=0x7f5a1400dcf0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.355+0000 7f5a28e8b640 1 -- 172.21.15.83:0/2826055756 shutdown_connections 2024-04-17T10:38:50.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.355+0000 7f5a28e8b640 1 --2- 172.21.15.83:0/2826055756 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5a04066f60 0x7f5a04069420 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.355+0000 7f5a28e8b640 1 --2- 172.21.15.83:0/2826055756 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a24161300 0x7f5a2415da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.355+0000 7f5a28e8b640 1 --2- 172.21.15.83:0/2826055756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a2406f360 0x7f5a2415d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.355+0000 7f5a28e8b640 1 --2- 172.21.15.83:0/2826055756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a2406c130 0x7f5a2409a9d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.355+0000 7f5a28e8b640 1 -- 172.21.15.83:0/2826055756 >> 172.21.15.83:0/2826055756 conn(0x7f5a240cc910 msgr2=0x7f5a2415f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:50.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.355+0000 7f5a28e8b640 1 -- 172.21.15.83:0/2826055756 shutdown_connections 2024-04-17T10:38:50.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.355+0000 7f5a28e8b640 1 -- 172.21.15.83:0/2826055756 wait complete. 2024-04-17T10:38:50.369 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:38:10.464490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:38:11.043134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:38:50.370 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:38:50.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.483+0000 7faabf747640 1 Processor -- start 2024-04-17T10:38:50.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.483+0000 7faabf747640 1 -- start start 2024-04-17T10:38:50.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.483+0000 7faabf747640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faab8154de0 0x7faab806ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:50.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.483+0000 7faabf747640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faab815bd70 con 0x7faab80b4210 2024-04-17T10:38:50.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.483+0000 7faabf747640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faab815bf60 con 0x7faab806b350 2024-04-17T10:38:50.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.483+0000 7faabf747640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab80b4870 con 0x7faab8154de0 2024-04-17T10:38:50.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.483+0000 7faabdce4640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7faab80b4210 0x7faab806cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40046/0 (socket says 172.21.15.83:40046) 2024-04-17T10:38:50.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.483+0000 7faabdce4640 1 -- 172.21.15.83:0/758804389 learned_addr learned my addr 172.21.15.83:0/758804389 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:50.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.483+0000 7faabd4e3640 1 --2- 172.21.15.83:0/758804389 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faab8154de0 0x7faab806ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:50.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.483+0000 7faaaffff640 1 -- 172.21.15.83:0/758804389 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1018544058 0 0) 0x7faab815bd70 con 0x7faab80b4210 2024-04-17T10:38:50.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.483+0000 7faaaffff640 1 -- 172.21.15.83:0/758804389 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faaa4003530 con 0x7faab80b4210 2024-04-17T10:38:50.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.483+0000 7faaaffff640 1 -- 172.21.15.83:0/758804389 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 307717302 0 0) 0x7faab815bf60 con 0x7faab806b350 2024-04-17T10:38:50.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faaaffff640 1 -- 172.21.15.83:0/758804389 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faab815bd70 con 0x7faab806b350 2024-04-17T10:38:50.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faaaffff640 1 -- 172.21.15.83:0/758804389 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3961275618 0 0) 0x7faaa4003530 con 0x7faab80b4210 2024-04-17T10:38:50.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faaaffff640 1 -- 172.21.15.83:0/758804389 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faab8154de0 msgr2=0x7faab806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:50.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faaaffff640 1 --2- 172.21.15.83:0/758804389 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faab8154de0 0x7faab806ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faaaffff640 1 -- 172.21.15.83:0/758804389 >> v1:172.21.15.167:6789/0 conn(0x7faab806b350 legacy=0x7faab80b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:50.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faaaffff640 1 -- 172.21.15.83:0/758804389 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab815afb0 con 0x7faab80b4210 2024-04-17T10:38:50.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faaaffff640 1 -- 172.21.15.83:0/758804389 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7faab0001a00 con 0x7faab80b4210 2024-04-17T10:38:50.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faabd4e3640 1 --2- 172.21.15.83:0/758804389 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faab8154de0 0x7faab806ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:38:50.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faaaffff640 1 -- 172.21.15.83:0/758804389 >> v1:172.21.15.83:6789/0 conn(0x7faab80b4210 legacy=0x7faab806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:50.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faaaffff640 1 --2- 172.21.15.83:0/758804389 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faab8154de0 0x7faaa4003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:50.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faaaffff640 1 --2- 172.21.15.83:0/758804389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faaa4004180 0x7faaa4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:50.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faaaffff640 1 --2- 172.21.15.83:0/758804389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaa4004d50 0x7faaa40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:50.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faaaffff640 1 -- 172.21.15.83:0/758804389 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa400bb60 con 0x7faaa4004d50 2024-04-17T10:38:50.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faaaffff640 1 -- 172.21.15.83:0/758804389 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa400bce0 con 0x7faaa4004180 2024-04-17T10:38:50.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faaaffff640 1 -- 172.21.15.83:0/758804389 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa400bee0 con 0x7faab8154de0 2024-04-17T10:38:50.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faabdce4640 1 --2- 172.21.15.83:0/758804389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaa4004d50 0x7faaa40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:50.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faabd4e3640 1 --2- 172.21.15.83:0/758804389 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faab8154de0 0x7faaa4003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:50.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faabcce2640 1 --2- 172.21.15.83:0/758804389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faaa4004180 0x7faaa4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:50.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faabdce4640 1 -- 172.21.15.83:0/758804389 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faab8154de0 msgr2=0x7faaa4003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:50.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faabdce4640 1 --2- 172.21.15.83:0/758804389 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faab8154de0 0x7faaa4003bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faabdce4640 1 -- 172.21.15.83:0/758804389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faaa4004180 msgr2=0x7faaa4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:50.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faabdce4640 1 --2- 172.21.15.83:0/758804389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faaa4004180 0x7faaa4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faabdce4640 1 -- 172.21.15.83:0/758804389 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa4003530 con 0x7faaa4004d50 2024-04-17T10:38:50.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faabcce2640 1 --2- 172.21.15.83:0/758804389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faaa4004180 0x7faaa4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:38:50.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faabdce4640 1 --2- 172.21.15.83:0/758804389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaa4004d50 0x7faaa40051d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7faab000a760 tx=0x7faab001f9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad609b100d48c1a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:50.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faaaffff640 1 -- 172.21.15.83:0/758804389 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab000a3b0 con 0x7faaa4004d50 2024-04-17T10:38:50.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faaaffff640 1 -- 172.21.15.83:0/758804389 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab00b1070 con 0x7faaa4004d50 2024-04-17T10:38:50.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faaaffff640 1 -- 172.21.15.83:0/758804389 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab00ac8d0 con 0x7faaa4004d50 2024-04-17T10:38:50.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faabf747640 1 -- 172.21.15.83:0/758804389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaa4004d50 msgr2=0x7faaa40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:50.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faabf747640 1 --2- 172.21.15.83:0/758804389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaa4004d50 0x7faaa40051d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7faab000a760 tx=0x7faab001f9a0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faabdce4640 1 -- 172.21.15.83:0/758804389 reap_dead start 2024-04-17T10:38:50.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faabf747640 1 -- 172.21.15.83:0/758804389 shutdown_connections 2024-04-17T10:38:50.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faabf747640 1 -- 172.21.15.83:0/758804389 >> 172.21.15.83:0/758804389 conn(0x7faab80b15b0 msgr2=0x7faab80b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:50.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faabf747640 1 -- 172.21.15.83:0/758804389 shutdown_connections 2024-04-17T10:38:50.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.487+0000 7faabf747640 1 -- 172.21.15.83:0/758804389 wait complete. 2024-04-17T10:38:50.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.491+0000 7faabf747640 1 Processor -- start 2024-04-17T10:38:50.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.491+0000 7faabf747640 1 -- start start 2024-04-17T10:38:50.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.491+0000 7faabf747640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faab813e760 0x7faab8157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:50.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.491+0000 7faabf747640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faab813eb40 0x7faab80ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:50.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.491+0000 7faabf747640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faab80add80 0x7faab8141190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:50.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.491+0000 7faabf747640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab815b850 con 0x7faab80add80 2024-04-17T10:38:50.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.491+0000 7faabf747640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab815b570 con 0x7faab813eb40 2024-04-17T10:38:50.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.491+0000 7faabf747640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab81551c0 con 0x7faab813e760 2024-04-17T10:38:50.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.491+0000 7faabdce4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faab80add80 0x7faab8141190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:50.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.491+0000 7faabdce4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faab80add80 0x7faab8141190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51554/0 (socket says 172.21.15.83:51554) 2024-04-17T10:38:50.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.491+0000 7faabdce4640 1 -- 172.21.15.83:0/2742974656 learned_addr learned my addr 172.21.15.83:0/2742974656 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:50.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.491+0000 7faabd4e3640 1 --2- 172.21.15.83:0/2742974656 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faab813e760 0x7faab8157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:50.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.491+0000 7faabcce2640 1 --2- 172.21.15.83:0/2742974656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faab813eb40 0x7faab80ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:50.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.491+0000 7faabdce4640 1 -- 172.21.15.83:0/2742974656 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faab813e760 msgr2=0x7faab8157e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:50.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.491+0000 7faabdce4640 1 --2- 172.21.15.83:0/2742974656 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faab813e760 0x7faab8157e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.491+0000 7faabdce4640 1 -- 172.21.15.83:0/2742974656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faab813eb40 msgr2=0x7faab80ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:50.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.491+0000 7faabdce4640 1 --2- 172.21.15.83:0/2742974656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faab813eb40 0x7faab80ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.491+0000 7faabdce4640 1 -- 172.21.15.83:0/2742974656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab80b4bf0 con 0x7faab80add80 2024-04-17T10:38:50.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.491+0000 7faabdce4640 1 --2- 172.21.15.83:0/2742974656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faab80add80 0x7faab8141190 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7faab0020240 tx=0x7faaa4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:50.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.491+0000 7faaadffb640 1 -- 172.21.15.83:0/2742974656 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab000a3b0 con 0x7faab80add80 2024-04-17T10:38:50.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.491+0000 7faaadffb640 1 -- 172.21.15.83:0/2742974656 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab00ba070 con 0x7faab80add80 2024-04-17T10:38:50.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.491+0000 7faabf747640 1 -- 172.21.15.83:0/2742974656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faab81570a0 con 0x7faab80add80 2024-04-17T10:38:50.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.491+0000 7faaadffb640 1 -- 172.21.15.83:0/2742974656 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab0001ee0 con 0x7faab80add80 2024-04-17T10:38:50.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.491+0000 7faabf747640 1 -- 172.21.15.83:0/2742974656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faab8157320 con 0x7faab80add80 2024-04-17T10:38:50.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.491+0000 7faabf747640 1 -- 172.21.15.83:0/2742974656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faab8074f90 con 0x7faab80add80 2024-04-17T10:38:50.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.491+0000 7faaadffb640 1 -- 172.21.15.83:0/2742974656 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faab00bc030 con 0x7faab80add80 2024-04-17T10:38:50.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.491+0000 7faaadffb640 1 --2- 172.21.15.83:0/2742974656 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faa9c066f90 0x7faa9c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:50.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.491+0000 7faaadffb640 1 -- 172.21.15.83:0/2742974656 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faab0122b70 con 0x7faab80add80 2024-04-17T10:38:50.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.491+0000 7faabd4e3640 1 --2- 172.21.15.83:0/2742974656 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faa9c066f90 0x7faa9c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:50.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.495+0000 7faabd4e3640 1 --2- 172.21.15.83:0/2742974656 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faa9c066f90 0x7faa9c069450 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7faab4009580 tx=0x7faab4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:50.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.495+0000 7faaadffb640 1 -- 172.21.15.83:0/2742974656 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faab81570a0 con 0x7faab80add80 2024-04-17T10:38:50.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.655+0000 7faabf747640 1 -- 172.21.15.83:0/2742974656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faab80b3280 con 0x7faab80add80 2024-04-17T10:38:50.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.655+0000 7faaadffb640 1 -- 172.21.15.83:0/2742974656 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7faab00196b0 con 0x7faab80add80 2024-04-17T10:38:50.652 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:38:50.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.655+0000 7faabf747640 1 -- 172.21.15.83:0/2742974656 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faa9c066f90 msgr2=0x7faa9c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:50.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.655+0000 7faabf747640 1 --2- 172.21.15.83:0/2742974656 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faa9c066f90 0x7faa9c069450 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7faab4009580 tx=0x7faab4008040 comp rx=0 tx=0).stop 2024-04-17T10:38:50.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.655+0000 7faabf747640 1 -- 172.21.15.83:0/2742974656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faab80add80 msgr2=0x7faab8141190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:50.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.655+0000 7faabf747640 1 --2- 172.21.15.83:0/2742974656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faab80add80 0x7faab8141190 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7faab0020240 tx=0x7faaa4005840 comp rx=0 tx=0).stop 2024-04-17T10:38:50.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.655+0000 7faabf747640 1 -- 172.21.15.83:0/2742974656 shutdown_connections 2024-04-17T10:38:50.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.655+0000 7faabf747640 1 --2- 172.21.15.83:0/2742974656 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faa9c066f90 0x7faa9c069450 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.655+0000 7faabf747640 1 --2- 172.21.15.83:0/2742974656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faab80add80 0x7faab8141190 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.655+0000 7faabf747640 1 --2- 172.21.15.83:0/2742974656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faab813eb40 0x7faab80ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.655+0000 7faabf747640 1 --2- 172.21.15.83:0/2742974656 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faab813e760 0x7faab8157e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.655+0000 7faabf747640 1 -- 172.21.15.83:0/2742974656 >> 172.21.15.83:0/2742974656 conn(0x7faab80b15b0 msgr2=0x7faab8071e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:50.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.655+0000 7faabf747640 1 -- 172.21.15.83:0/2742974656 shutdown_connections 2024-04-17T10:38:50.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.655+0000 7faabf747640 1 -- 172.21.15.83:0/2742974656 wait complete. 2024-04-17T10:38:50.671 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:38:50.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.791+0000 7ffa2328d640 1 Processor -- start 2024-04-17T10:38:50.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.791+0000 7ffa2328d640 1 -- start start 2024-04-17T10:38:50.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.791+0000 7ffa2328d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ffa1c0b4210 0x7ffa1c15e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:50.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.791+0000 7ffa2328d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa1c15bd70 con 0x7ffa1c06b520 2024-04-17T10:38:50.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.791+0000 7ffa2328d640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa1c15bf60 con 0x7ffa1c154de0 2024-04-17T10:38:50.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.791+0000 7ffa2328d640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa1c0b4870 con 0x7ffa1c0b4210 2024-04-17T10:38:50.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.791+0000 7ffa20828640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ffa1c06b520 0x7ffa1c0b3b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40052/0 (socket says 172.21.15.83:40052) 2024-04-17T10:38:50.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.791+0000 7ffa20828640 1 -- 172.21.15.83:0/2416181307 learned_addr learned my addr 172.21.15.83:0/2416181307 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:50.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.791+0000 7ffa2182a640 1 --2- 172.21.15.83:0/2416181307 >> v2:172.21.15.167:3300/0 conn(0x7ffa1c0b4210 0x7ffa1c15e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:50.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.791+0000 7ffa13fff640 1 -- 172.21.15.83:0/2416181307 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4221972865 0 0) 0x7ffa1c15bd70 con 0x7ffa1c06b520 2024-04-17T10:38:50.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.791+0000 7ffa13fff640 1 -- 172.21.15.83:0/2416181307 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa08003530 con 0x7ffa1c06b520 2024-04-17T10:38:50.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.791+0000 7ffa13fff640 1 -- 172.21.15.83:0/2416181307 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3005625334 0 0) 0x7ffa08003530 con 0x7ffa1c06b520 2024-04-17T10:38:50.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.791+0000 7ffa13fff640 1 -- 172.21.15.83:0/2416181307 >> v2:172.21.15.167:3300/0 conn(0x7ffa1c0b4210 msgr2=0x7ffa1c15e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:50.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.791+0000 7ffa13fff640 1 --2- 172.21.15.83:0/2416181307 >> v2:172.21.15.167:3300/0 conn(0x7ffa1c0b4210 0x7ffa1c15e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.791+0000 7ffa13fff640 1 -- 172.21.15.83:0/2416181307 >> v1:172.21.15.167:6789/0 conn(0x7ffa1c154de0 legacy=0x7ffa1c06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:50.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.791+0000 7ffa13fff640 1 -- 172.21.15.83:0/2416181307 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa1c15afb0 con 0x7ffa1c06b520 2024-04-17T10:38:50.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.791+0000 7ffa13fff640 1 -- 172.21.15.83:0/2416181307 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7ffa0c001680 con 0x7ffa1c06b520 2024-04-17T10:38:50.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.791+0000 7ffa13fff640 1 -- 172.21.15.83:0/2416181307 >> v1:172.21.15.83:6789/0 conn(0x7ffa1c06b520 legacy=0x7ffa1c0b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:50.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.791+0000 7ffa13fff640 1 --2- 172.21.15.83:0/2416181307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa08003a30 0x7ffa08003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:50.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.791+0000 7ffa13fff640 1 --2- 172.21.15.83:0/2416181307 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa08004810 0x7ffa08004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:50.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.795+0000 7ffa13fff640 1 --2- 172.21.15.83:0/2416181307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa080051d0 0x7ffa0800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:50.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.795+0000 7ffa13fff640 1 -- 172.21.15.83:0/2416181307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0800c0d0 con 0x7ffa08003a30 2024-04-17T10:38:50.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.795+0000 7ffa13fff640 1 -- 172.21.15.83:0/2416181307 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0800c280 con 0x7ffa080051d0 2024-04-17T10:38:50.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.795+0000 7ffa13fff640 1 -- 172.21.15.83:0/2416181307 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0800c480 con 0x7ffa08004810 2024-04-17T10:38:50.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.795+0000 7ffa21029640 1 --2- 172.21.15.83:0/2416181307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa08003a30 0x7ffa08003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:50.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.795+0000 7ffa20828640 1 --2- 172.21.15.83:0/2416181307 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa08004810 0x7ffa08004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:50.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.795+0000 7ffa2182a640 1 --2- 172.21.15.83:0/2416181307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa080051d0 0x7ffa0800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:50.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.795+0000 7ffa21029640 1 -- 172.21.15.83:0/2416181307 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa08004810 msgr2=0x7ffa08004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:50.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.795+0000 7ffa21029640 1 --2- 172.21.15.83:0/2416181307 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa08004810 0x7ffa08004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.795+0000 7ffa21029640 1 -- 172.21.15.83:0/2416181307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa080051d0 msgr2=0x7ffa0800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:50.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.795+0000 7ffa21029640 1 --2- 172.21.15.83:0/2416181307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa080051d0 0x7ffa0800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.795+0000 7ffa21029640 1 -- 172.21.15.83:0/2416181307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa08003530 con 0x7ffa08003a30 2024-04-17T10:38:50.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.795+0000 7ffa2182a640 1 --2- 172.21.15.83:0/2416181307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa080051d0 0x7ffa0800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:50.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.795+0000 7ffa2182a640 1 -- 172.21.15.83:0/2416181307 reap_dead start 2024-04-17T10:38:50.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.795+0000 7ffa21029640 1 --2- 172.21.15.83:0/2416181307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa08003a30 0x7ffa08003e50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ffa18001780 tx=0x7ffa1801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20717eb5f4a6258a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:50.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.795+0000 7ffa13fff640 1 -- 172.21.15.83:0/2416181307 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa180013a0 con 0x7ffa08003a30 2024-04-17T10:38:50.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.795+0000 7ffa13fff640 1 -- 172.21.15.83:0/2416181307 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa180052f0 con 0x7ffa08003a30 2024-04-17T10:38:50.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.795+0000 7ffa13fff640 1 -- 172.21.15.83:0/2416181307 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa18003840 con 0x7ffa08003a30 2024-04-17T10:38:50.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.795+0000 7ffa2328d640 1 -- 172.21.15.83:0/2416181307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa08003a30 msgr2=0x7ffa08003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:50.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.795+0000 7ffa2328d640 1 --2- 172.21.15.83:0/2416181307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa08003a30 0x7ffa08003e50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ffa18001780 tx=0x7ffa1801c400 comp rx=0 tx=0).stop 2024-04-17T10:38:50.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.795+0000 7ffa2328d640 1 -- 172.21.15.83:0/2416181307 shutdown_connections 2024-04-17T10:38:50.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.795+0000 7ffa2328d640 1 --2- 172.21.15.83:0/2416181307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa08003a30 0x7ffa08003e50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.795+0000 7ffa2328d640 1 -- 172.21.15.83:0/2416181307 >> 172.21.15.83:0/2416181307 conn(0x7ffa1c0b15b0 msgr2=0x7ffa1c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:50.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.795+0000 7ffa2328d640 1 -- 172.21.15.83:0/2416181307 shutdown_connections 2024-04-17T10:38:50.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.795+0000 7ffa2328d640 1 -- 172.21.15.83:0/2416181307 wait complete. 2024-04-17T10:38:50.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.799+0000 7ffa2328d640 1 Processor -- start 2024-04-17T10:38:50.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.799+0000 7ffa2328d640 1 -- start start 2024-04-17T10:38:50.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.799+0000 7ffa2328d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa08003a30 0x7ffa1c15a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:50.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.799+0000 7ffa2328d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa1c14ed40 0x7ffa1c0cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:50.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.799+0000 7ffa2328d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa1c155eb0 0x7ffa1c0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:50.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.799+0000 7ffa2328d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c0b3f10 con 0x7ffa1c14ed40 2024-04-17T10:38:50.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.799+0000 7ffa2328d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c1551c0 con 0x7ffa1c155eb0 2024-04-17T10:38:50.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.799+0000 7ffa2328d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c0b4090 con 0x7ffa08003a30 2024-04-17T10:38:50.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.799+0000 7ffa20828640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa1c14ed40 0x7ffa1c0cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:50.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.799+0000 7ffa20828640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa1c14ed40 0x7ffa1c0cdfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51582/0 (socket says 172.21.15.83:51582) 2024-04-17T10:38:50.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.799+0000 7ffa20828640 1 -- 172.21.15.83:0/1386615059 learned_addr learned my addr 172.21.15.83:0/1386615059 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:50.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.799+0000 7ffa2182a640 1 --2- 172.21.15.83:0/1386615059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa1c155eb0 0x7ffa1c0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:50.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.799+0000 7ffa21029640 1 --2- 172.21.15.83:0/1386615059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa08003a30 0x7ffa1c15a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:50.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.799+0000 7ffa20828640 1 -- 172.21.15.83:0/1386615059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa08003a30 msgr2=0x7ffa1c15a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:50.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.799+0000 7ffa20828640 1 --2- 172.21.15.83:0/1386615059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa08003a30 0x7ffa1c15a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.799+0000 7ffa20828640 1 -- 172.21.15.83:0/1386615059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa1c155eb0 msgr2=0x7ffa1c0ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:50.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.799+0000 7ffa20828640 1 --2- 172.21.15.83:0/1386615059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa1c155eb0 0x7ffa1c0ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.799+0000 7ffa20828640 1 -- 172.21.15.83:0/1386615059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa1c152bf0 con 0x7ffa1c14ed40 2024-04-17T10:38:50.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.799+0000 7ffa2182a640 1 --2- 172.21.15.83:0/1386615059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa1c155eb0 0x7ffa1c0ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:38:50.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.799+0000 7ffa20828640 1 --2- 172.21.15.83:0/1386615059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa1c14ed40 0x7ffa1c0cdfd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c007fd0 tx=0x7ffa0c00db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:50.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.799+0000 7ffa11ffb640 1 -- 172.21.15.83:0/1386615059 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c019070 con 0x7ffa1c14ed40 2024-04-17T10:38:50.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.799+0000 7ffa11ffb640 1 -- 172.21.15.83:0/1386615059 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa0c0072d0 con 0x7ffa1c14ed40 2024-04-17T10:38:50.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.799+0000 7ffa2328d640 1 -- 172.21.15.83:0/1386615059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa1c152ed0 con 0x7ffa1c14ed40 2024-04-17T10:38:50.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.799+0000 7ffa11ffb640 1 -- 172.21.15.83:0/1386615059 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c0047f0 con 0x7ffa1c14ed40 2024-04-17T10:38:50.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.799+0000 7ffa2328d640 1 -- 172.21.15.83:0/1386615059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa1c0add80 con 0x7ffa1c14ed40 2024-04-17T10:38:50.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.799+0000 7ffa2328d640 1 -- 172.21.15.83:0/1386615059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa1c074f90 con 0x7ffa1c14ed40 2024-04-17T10:38:50.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.803+0000 7ffa11ffb640 1 -- 172.21.15.83:0/1386615059 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffa0c01c070 con 0x7ffa1c14ed40 2024-04-17T10:38:50.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.803+0000 7ffa11ffb640 1 --2- 172.21.15.83:0/1386615059 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffa00066f60 0x7ffa00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:50.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.803+0000 7ffa11ffb640 1 -- 172.21.15.83:0/1386615059 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa0c0847c0 con 0x7ffa1c14ed40 2024-04-17T10:38:50.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.803+0000 7ffa21029640 1 --2- 172.21.15.83:0/1386615059 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffa00066f60 0x7ffa00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:50.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.803+0000 7ffa21029640 1 --2- 172.21.15.83:0/1386615059 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffa00066f60 0x7ffa00069420 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ffa18001750 tx=0x7ffa180011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:50.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.807+0000 7ffa11ffb640 1 -- 172.21.15.83:0/1386615059 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffa0c055180 con 0x7ffa1c14ed40 2024-04-17T10:38:50.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.963+0000 7ffa2328d640 1 -- 172.21.15.83:0/1386615059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffa1c0b3280 con 0x7ffa1c14ed40 2024-04-17T10:38:50.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.963+0000 7ffa11ffb640 1 -- 172.21.15.83:0/1386615059 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffa0c059220 con 0x7ffa1c14ed40 2024-04-17T10:38:50.960 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:38:50.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.963+0000 7ffa2328d640 1 -- 172.21.15.83:0/1386615059 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffa00066f60 msgr2=0x7ffa00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:50.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.963+0000 7ffa2328d640 1 --2- 172.21.15.83:0/1386615059 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffa00066f60 0x7ffa00069420 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ffa18001750 tx=0x7ffa180011a0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.963+0000 7ffa2328d640 1 -- 172.21.15.83:0/1386615059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa1c14ed40 msgr2=0x7ffa1c0cdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:50.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.963+0000 7ffa2328d640 1 --2- 172.21.15.83:0/1386615059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa1c14ed40 0x7ffa1c0cdfd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c007fd0 tx=0x7ffa0c00db50 comp rx=0 tx=0).stop 2024-04-17T10:38:50.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.963+0000 7ffa2328d640 1 -- 172.21.15.83:0/1386615059 shutdown_connections 2024-04-17T10:38:50.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.963+0000 7ffa2328d640 1 --2- 172.21.15.83:0/1386615059 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffa00066f60 0x7ffa00069420 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.963+0000 7ffa2328d640 1 --2- 172.21.15.83:0/1386615059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa1c155eb0 0x7ffa1c0ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.963+0000 7ffa2328d640 1 --2- 172.21.15.83:0/1386615059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa1c14ed40 0x7ffa1c0cdfd0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.963+0000 7ffa2328d640 1 --2- 172.21.15.83:0/1386615059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa08003a30 0x7ffa1c15a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:50.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.963+0000 7ffa2328d640 1 -- 172.21.15.83:0/1386615059 >> 172.21.15.83:0/1386615059 conn(0x7ffa1c0b15b0 msgr2=0x7ffa1c0b0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:50.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.963+0000 7ffa2328d640 1 -- 172.21.15.83:0/1386615059 shutdown_connections 2024-04-17T10:38:50.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:50.963+0000 7ffa2328d640 1 -- 172.21.15.83:0/1386615059 wait complete. 2024-04-17T10:38:50.978 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:38:51.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.103+0000 7fc09e5ff640 1 Processor -- start 2024-04-17T10:38:51.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.103+0000 7fc09e5ff640 1 -- start start 2024-04-17T10:38:51.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.107+0000 7fc09e5ff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc09815b6e0 0x7fc098071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:51.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.107+0000 7fc09e5ff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc098070470 0x7fc09806ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:51.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.107+0000 7fc09e5ff640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc09806c1f0 con 0x7fc09815bac0 2024-04-17T10:38:51.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.107+0000 7fc09e5ff640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc098070a10 con 0x7fc09815b6e0 2024-04-17T10:38:51.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.107+0000 7fc09e5ff640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc098070cf0 con 0x7fc098070470 2024-04-17T10:38:51.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.107+0000 7fc097fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc09815b6e0 0x7fc098071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:51.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.107+0000 7fc097fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc09815b6e0 0x7fc098071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51588/0 (socket says 172.21.15.83:51588) 2024-04-17T10:38:51.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.107+0000 7fc097fff640 1 -- 172.21.15.83:0/3540193552 learned_addr learned my addr 172.21.15.83:0/3540193552 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:51.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.107+0000 7fc09cb9c640 1 --2- 172.21.15.83:0/3540193552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc098070470 0x7fc09806ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:51.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.107+0000 7fc097fff640 1 -- 172.21.15.83:0/3540193552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc098070470 msgr2=0x7fc09806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:51.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.107+0000 7fc097fff640 1 --2- 172.21.15.83:0/3540193552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc098070470 0x7fc09806ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:51.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.107+0000 7fc097fff640 1 -- 172.21.15.83:0/3540193552 >> v1:172.21.15.167:6789/0 conn(0x7fc09815bac0 legacy=0x7fc0980cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:51.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.107+0000 7fc097fff640 1 -- 172.21.15.83:0/3540193552 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc098070fd0 con 0x7fc09815b6e0 2024-04-17T10:38:51.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.107+0000 7fc097fff640 1 --2- 172.21.15.83:0/3540193552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc09815b6e0 0x7fc098071cc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc0980728a0 tx=0x7fc08801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2acc51a56cd7f98a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:51.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.107+0000 7fc096ffd640 1 -- 172.21.15.83:0/3540193552 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0880b0070 con 0x7fc09815b6e0 2024-04-17T10:38:51.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.107+0000 7fc096ffd640 1 -- 172.21.15.83:0/3540193552 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0880ab600 con 0x7fc09815b6e0 2024-04-17T10:38:51.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.107+0000 7fc096ffd640 1 -- 172.21.15.83:0/3540193552 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0880ab9d0 con 0x7fc09815b6e0 2024-04-17T10:38:51.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.107+0000 7fc09e5ff640 1 -- 172.21.15.83:0/3540193552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc09815b6e0 msgr2=0x7fc098071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:51.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.107+0000 7fc09e5ff640 1 --2- 172.21.15.83:0/3540193552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc09815b6e0 0x7fc098071cc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc0980728a0 tx=0x7fc08801c450 comp rx=0 tx=0).stop 2024-04-17T10:38:51.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.107+0000 7fc09e5ff640 1 -- 172.21.15.83:0/3540193552 shutdown_connections 2024-04-17T10:38:51.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.107+0000 7fc09e5ff640 1 --2- 172.21.15.83:0/3540193552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc098070470 0x7fc09806ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:51.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.107+0000 7fc09e5ff640 1 --2- 172.21.15.83:0/3540193552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc09815b6e0 0x7fc098071cc0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:51.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.107+0000 7fc09e5ff640 1 -- 172.21.15.83:0/3540193552 >> 172.21.15.83:0/3540193552 conn(0x7fc0980b15b0 msgr2=0x7fc0980b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:51.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.107+0000 7fc09e5ff640 1 -- 172.21.15.83:0/3540193552 shutdown_connections 2024-04-17T10:38:51.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.107+0000 7fc09e5ff640 1 -- 172.21.15.83:0/3540193552 wait complete. 2024-04-17T10:38:51.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.111+0000 7fc09e5ff640 1 Processor -- start 2024-04-17T10:38:51.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.111+0000 7fc09e5ff640 1 -- start start 2024-04-17T10:38:51.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.111+0000 7fc09e5ff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc098070470 0x7fc0980aa180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:51.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.111+0000 7fc09e5ff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc09815b6e0 0x7fc0980acb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:51.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.111+0000 7fc09e5ff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc09815bac0 0x7fc0980ad0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:51.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.111+0000 7fc09e5ff640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09806d2f0 con 0x7fc098070470 2024-04-17T10:38:51.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.111+0000 7fc09e5ff640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09806d170 con 0x7fc09815bac0 2024-04-17T10:38:51.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.111+0000 7fc09e5ff640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc09806b760 con 0x7fc09815b6e0 2024-04-17T10:38:51.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.111+0000 7fc097fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc098070470 0x7fc0980aa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:51.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.111+0000 7fc097fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc098070470 0x7fc0980aa180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51596/0 (socket says 172.21.15.83:51596) 2024-04-17T10:38:51.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.111+0000 7fc097fff640 1 -- 172.21.15.83:0/2534203744 learned_addr learned my addr 172.21.15.83:0/2534203744 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:51.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.111+0000 7fc09cb9c640 1 --2- 172.21.15.83:0/2534203744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc09815bac0 0x7fc0980ad0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:51.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.111+0000 7fc0977fe640 1 --2- 172.21.15.83:0/2534203744 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc09815b6e0 0x7fc0980acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:51.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.111+0000 7fc097fff640 1 -- 172.21.15.83:0/2534203744 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc09815b6e0 msgr2=0x7fc0980acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:51.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.111+0000 7fc097fff640 1 --2- 172.21.15.83:0/2534203744 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc09815b6e0 0x7fc0980acb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:51.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.111+0000 7fc097fff640 1 -- 172.21.15.83:0/2534203744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc09815bac0 msgr2=0x7fc0980ad0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:51.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.111+0000 7fc097fff640 1 --2- 172.21.15.83:0/2534203744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc09815bac0 0x7fc0980ad0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:51.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.111+0000 7fc097fff640 1 -- 172.21.15.83:0/2534203744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0980726d0 con 0x7fc098070470 2024-04-17T10:38:51.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.111+0000 7fc097fff640 1 --2- 172.21.15.83:0/2534203744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc098070470 0x7fc0980aa180 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc0880ae040 tx=0x7fc08801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.111+0000 7fc094ff9640 1 -- 172.21.15.83:0/2534203744 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0880ba020 con 0x7fc098070470 2024-04-17T10:38:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.111+0000 7fc094ff9640 1 -- 172.21.15.83:0/2534203744 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0880b6640 con 0x7fc098070470 2024-04-17T10:38:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.111+0000 7fc09e5ff640 1 -- 172.21.15.83:0/2534203744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0980aedf0 con 0x7fc098070470 2024-04-17T10:38:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.111+0000 7fc094ff9640 1 -- 172.21.15.83:0/2534203744 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0880b0040 con 0x7fc098070470 2024-04-17T10:38:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.111+0000 7fc09e5ff640 1 -- 172.21.15.83:0/2534203744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0980aaa40 con 0x7fc098070470 2024-04-17T10:38:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.111+0000 7fc09e5ff640 1 -- 172.21.15.83:0/2534203744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc098074f90 con 0x7fc098070470 2024-04-17T10:38:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.115+0000 7fc094ff9640 1 -- 172.21.15.83:0/2534203744 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc0880bd020 con 0x7fc098070470 2024-04-17T10:38:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.115+0000 7fc094ff9640 1 --2- 172.21.15.83:0/2534203744 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc084066f60 0x7fc084069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.115+0000 7fc094ff9640 1 -- 172.21.15.83:0/2534203744 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc08811f590 con 0x7fc098070470 2024-04-17T10:38:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.115+0000 7fc0977fe640 1 --2- 172.21.15.83:0/2534203744 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc084066f60 0x7fc084069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.115+0000 7fc0977fe640 1 --2- 172.21.15.83:0/2534203744 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc084066f60 0x7fc084069420 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc098155f10 tx=0x7fc08c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:51.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.119+0000 7fc094ff9640 1 -- 172.21.15.83:0/2534203744 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc0880c4030 con 0x7fc098070470 2024-04-17T10:38:51.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.271+0000 7fc09e5ff640 1 -- 172.21.15.83:0/2534203744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc0980b1df0 con 0x7fc098070470 2024-04-17T10:38:51.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.271+0000 7fc094ff9640 1 -- 172.21.15.83:0/2534203744 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc0880160d0 con 0x7fc098070470 2024-04-17T10:38:51.270 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:38:51.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.275+0000 7fc09e5ff640 1 -- 172.21.15.83:0/2534203744 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc084066f60 msgr2=0x7fc084069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:51.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.275+0000 7fc09e5ff640 1 --2- 172.21.15.83:0/2534203744 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc084066f60 0x7fc084069420 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc098155f10 tx=0x7fc08c009040 comp rx=0 tx=0).stop 2024-04-17T10:38:51.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.275+0000 7fc09e5ff640 1 -- 172.21.15.83:0/2534203744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc098070470 msgr2=0x7fc0980aa180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:51.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.275+0000 7fc09e5ff640 1 --2- 172.21.15.83:0/2534203744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc098070470 0x7fc0980aa180 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc0880ae040 tx=0x7fc08801ac90 comp rx=0 tx=0).stop 2024-04-17T10:38:51.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.275+0000 7fc09e5ff640 1 -- 172.21.15.83:0/2534203744 shutdown_connections 2024-04-17T10:38:51.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.275+0000 7fc09e5ff640 1 --2- 172.21.15.83:0/2534203744 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc084066f60 0x7fc084069420 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:51.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.275+0000 7fc09e5ff640 1 --2- 172.21.15.83:0/2534203744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc09815bac0 0x7fc0980ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:51.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.275+0000 7fc09e5ff640 1 --2- 172.21.15.83:0/2534203744 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc09815b6e0 0x7fc0980acb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:51.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.275+0000 7fc09e5ff640 1 --2- 172.21.15.83:0/2534203744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc098070470 0x7fc0980aa180 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:51.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.275+0000 7fc09e5ff640 1 -- 172.21.15.83:0/2534203744 >> 172.21.15.83:0/2534203744 conn(0x7fc0980b15b0 msgr2=0x7fc09815a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:51.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.275+0000 7fc09e5ff640 1 -- 172.21.15.83:0/2534203744 shutdown_connections 2024-04-17T10:38:51.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.275+0000 7fc09e5ff640 1 -- 172.21.15.83:0/2534203744 wait complete. 2024-04-17T10:38:51.287 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:38:51.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.399+0000 7f868b5a1640 1 Processor -- start 2024-04-17T10:38:51.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.399+0000 7f868b5a1640 1 -- start start 2024-04-17T10:38:51.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.399+0000 7f868b5a1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f868415b6e0 0x7f8684071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:51.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.399+0000 7f868b5a1640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f868415bac0 0x7f86840cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:51.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.399+0000 7f868b5a1640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f868406c1f0 con 0x7f8684070470 2024-04-17T10:38:51.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.399+0000 7f868b5a1640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86840cdd30 con 0x7f868415b6e0 2024-04-17T10:38:51.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.399+0000 7f868b5a1640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8684070a10 con 0x7f868415bac0 2024-04-17T10:38:51.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.399+0000 7f868933d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f868415b6e0 0x7f8684071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:51.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.399+0000 7f868933d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f868415b6e0 0x7f8684071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51598/0 (socket says 172.21.15.83:51598) 2024-04-17T10:38:51.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.399+0000 7f868933d640 1 -- 172.21.15.83:0/3658018606 learned_addr learned my addr 172.21.15.83:0/3658018606 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:51.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.399+0000 7f8688b3c640 1 --2- 172.21.15.83:0/3658018606 >> v2:172.21.15.167:3300/0 conn(0x7f868415bac0 0x7f86840cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:51.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.403+0000 7f868933d640 1 -- 172.21.15.83:0/3658018606 >> v2:172.21.15.167:3300/0 conn(0x7f868415bac0 msgr2=0x7f86840cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:51.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.403+0000 7f868933d640 1 --2- 172.21.15.83:0/3658018606 >> v2:172.21.15.167:3300/0 conn(0x7f868415bac0 0x7f86840cd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:51.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.403+0000 7f868933d640 1 -- 172.21.15.83:0/3658018606 >> v1:172.21.15.167:6789/0 conn(0x7f8684070470 legacy=0x7f868406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:51.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.403+0000 7f868933d640 1 -- 172.21.15.83:0/3658018606 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8684070cf0 con 0x7f868415b6e0 2024-04-17T10:38:51.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.403+0000 7f868933d640 1 --2- 172.21.15.83:0/3658018606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f868415b6e0 0x7f8684071cc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8684071c50 tx=0x7f867c01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c961758c1555fd86 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:51.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.403+0000 7f8683fff640 1 -- 172.21.15.83:0/3658018606 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c0a5670 con 0x7f868415b6e0 2024-04-17T10:38:51.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.403+0000 7f8683fff640 1 -- 172.21.15.83:0/3658018606 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867c0a5810 con 0x7f868415b6e0 2024-04-17T10:38:51.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.403+0000 7f8683fff640 1 -- 172.21.15.83:0/3658018606 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c0a5be0 con 0x7f868415b6e0 2024-04-17T10:38:51.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.403+0000 7f868b5a1640 1 -- 172.21.15.83:0/3658018606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f868415b6e0 msgr2=0x7f8684071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:51.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.403+0000 7f868b5a1640 1 --2- 172.21.15.83:0/3658018606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f868415b6e0 0x7f8684071cc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8684071c50 tx=0x7f867c01c9b0 comp rx=0 tx=0).stop 2024-04-17T10:38:51.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.403+0000 7f868b5a1640 1 -- 172.21.15.83:0/3658018606 shutdown_connections 2024-04-17T10:38:51.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.403+0000 7f868b5a1640 1 --2- 172.21.15.83:0/3658018606 >> v2:172.21.15.167:3300/0 conn(0x7f868415bac0 0x7f86840cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:51.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.403+0000 7f868b5a1640 1 --2- 172.21.15.83:0/3658018606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f868415b6e0 0x7f8684071cc0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:51.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.403+0000 7f868b5a1640 1 -- 172.21.15.83:0/3658018606 >> 172.21.15.83:0/3658018606 conn(0x7f86840b15b0 msgr2=0x7f86840b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:51.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.403+0000 7f868b5a1640 1 -- 172.21.15.83:0/3658018606 shutdown_connections 2024-04-17T10:38:51.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.403+0000 7f868b5a1640 1 -- 172.21.15.83:0/3658018606 wait complete. 2024-04-17T10:38:51.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.403+0000 7f868b5a1640 1 Processor -- start 2024-04-17T10:38:51.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.403+0000 7f868b5a1640 1 -- start start 2024-04-17T10:38:51.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.407+0000 7f868b5a1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8684070470 0x7f868414c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:51.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.407+0000 7f868b5a1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f868415b6e0 0x7f868415efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:51.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.407+0000 7f868b5a1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f868415bac0 0x7f868415f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:51.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.407+0000 7f868b5a1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868406cee0 con 0x7f868415b6e0 2024-04-17T10:38:51.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.407+0000 7f868b5a1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868406b770 con 0x7f868415bac0 2024-04-17T10:38:51.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.407+0000 7f868b5a1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f868406d060 con 0x7f8684070470 2024-04-17T10:38:51.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.407+0000 7f8688b3c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f868415b6e0 0x7f868415efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:51.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.407+0000 7f8688b3c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f868415b6e0 0x7f868415efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51600/0 (socket says 172.21.15.83:51600) 2024-04-17T10:38:51.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.407+0000 7f8688b3c640 1 -- 172.21.15.83:0/2394960402 learned_addr learned my addr 172.21.15.83:0/2394960402 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:51.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.407+0000 7f8689b3e640 1 --2- 172.21.15.83:0/2394960402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f868415bac0 0x7f868415f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:51.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.407+0000 7f868933d640 1 --2- 172.21.15.83:0/2394960402 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8684070470 0x7f868414c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:51.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.407+0000 7f8688b3c640 1 -- 172.21.15.83:0/2394960402 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8684070470 msgr2=0x7f868414c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:51.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.407+0000 7f8688b3c640 1 --2- 172.21.15.83:0/2394960402 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8684070470 0x7f868414c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:51.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.407+0000 7f8688b3c640 1 -- 172.21.15.83:0/2394960402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f868415bac0 msgr2=0x7f868415f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:51.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.407+0000 7f8688b3c640 1 --2- 172.21.15.83:0/2394960402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f868415bac0 0x7f868415f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:51.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.407+0000 7f8688b3c640 1 -- 172.21.15.83:0/2394960402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f868414c8c0 con 0x7f868415b6e0 2024-04-17T10:38:51.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.407+0000 7f8689b3e640 1 --2- 172.21.15.83:0/2394960402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f868415bac0 0x7f868415f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:51.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.407+0000 7f868933d640 1 --2- 172.21.15.83:0/2394960402 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8684070470 0x7f868414c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:51.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.407+0000 7f8688b3c640 1 --2- 172.21.15.83:0/2394960402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f868415b6e0 0x7f868415efc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f867800c7f0 tx=0x7f867800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:51.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.407+0000 7f8681ffb640 1 -- 172.21.15.83:0/2394960402 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8678015070 con 0x7f868415b6e0 2024-04-17T10:38:51.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.407+0000 7f8681ffb640 1 -- 172.21.15.83:0/2394960402 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8678007820 con 0x7f868415b6e0 2024-04-17T10:38:51.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.407+0000 7f8681ffb640 1 -- 172.21.15.83:0/2394960402 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8678007c10 con 0x7f868415b6e0 2024-04-17T10:38:51.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.407+0000 7f868b5a1640 1 -- 172.21.15.83:0/2394960402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86840cebe0 con 0x7f868415b6e0 2024-04-17T10:38:51.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.407+0000 7f868b5a1640 1 -- 172.21.15.83:0/2394960402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8684155e40 con 0x7f868415b6e0 2024-04-17T10:38:51.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.407+0000 7f868b5a1640 1 -- 172.21.15.83:0/2394960402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8684074f90 con 0x7f868415b6e0 2024-04-17T10:38:51.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.407+0000 7f8681ffb640 1 -- 172.21.15.83:0/2394960402 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8678018070 con 0x7f868415b6e0 2024-04-17T10:38:51.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.407+0000 7f8681ffb640 1 --2- 172.21.15.83:0/2394960402 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8670066f60 0x7f8670069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:51.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.407+0000 7f8681ffb640 1 -- 172.21.15.83:0/2394960402 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8678085070 con 0x7f868415b6e0 2024-04-17T10:38:51.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.407+0000 7f868933d640 1 --2- 172.21.15.83:0/2394960402 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8670066f60 0x7f8670069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:51.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.407+0000 7f868933d640 1 --2- 172.21.15.83:0/2394960402 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8670066f60 0x7f8670069420 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f867c0ae000 tx=0x7f867c0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:51.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.411+0000 7f8681ffb640 1 -- 172.21.15.83:0/2394960402 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8678056080 con 0x7f868415b6e0 2024-04-17T10:38:51.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.579+0000 7f868b5a1640 1 -- 172.21.15.83:0/2394960402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8684074480 con 0x7f868415b6e0 2024-04-17T10:38:51.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.579+0000 7f8681ffb640 1 -- 172.21.15.83:0/2394960402 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8678055980 con 0x7f868415b6e0 2024-04-17T10:38:51.579 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-17T10:38:51.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.583+0000 7f868b5a1640 1 -- 172.21.15.83:0/2394960402 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8670066f60 msgr2=0x7f8670069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:51.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.583+0000 7f868b5a1640 1 --2- 172.21.15.83:0/2394960402 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8670066f60 0x7f8670069420 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f867c0ae000 tx=0x7f867c0a7070 comp rx=0 tx=0).stop 2024-04-17T10:38:51.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.583+0000 7f868b5a1640 1 -- 172.21.15.83:0/2394960402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f868415b6e0 msgr2=0x7f868415efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:51.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.583+0000 7f868b5a1640 1 --2- 172.21.15.83:0/2394960402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f868415b6e0 0x7f868415efc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f867800c7f0 tx=0x7f867800ccb0 comp rx=0 tx=0).stop 2024-04-17T10:38:51.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.583+0000 7f868b5a1640 1 -- 172.21.15.83:0/2394960402 shutdown_connections 2024-04-17T10:38:51.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.583+0000 7f868b5a1640 1 --2- 172.21.15.83:0/2394960402 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8670066f60 0x7f8670069420 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:51.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.583+0000 7f868b5a1640 1 --2- 172.21.15.83:0/2394960402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f868415bac0 0x7f868415f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:51.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.583+0000 7f868b5a1640 1 --2- 172.21.15.83:0/2394960402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f868415b6e0 0x7f868415efc0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:51.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.583+0000 7f868b5a1640 1 --2- 172.21.15.83:0/2394960402 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8684070470 0x7f868414c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:51.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.583+0000 7f868b5a1640 1 -- 172.21.15.83:0/2394960402 >> 172.21.15.83:0/2394960402 conn(0x7f86840b15b0 msgr2=0x7f868406ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:51.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.583+0000 7f868b5a1640 1 -- 172.21.15.83:0/2394960402 shutdown_connections 2024-04-17T10:38:51.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.583+0000 7f868b5a1640 1 -- 172.21.15.83:0/2394960402 wait complete. 2024-04-17T10:38:51.596 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:38:51.596 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:38:51.604 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:38:51.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.727+0000 7fdac71ec640 1 Processor -- start 2024-04-17T10:38:51.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.727+0000 7fdac71ec640 1 -- start start 2024-04-17T10:38:51.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.727+0000 7fdac71ec640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdac00cf960 0x7fdac00d1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:51.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.727+0000 7fdac71ec640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdac00d1ca0 0x7fdac0167790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:51.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.727+0000 7fdac71ec640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdac015e060 con 0x7fdac015dc80 2024-04-17T10:38:51.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.727+0000 7fdac71ec640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdac006ca10 con 0x7fdac00cf960 2024-04-17T10:38:51.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.727+0000 7fdac71ec640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdac006cc60 con 0x7fdac00d1ca0 2024-04-17T10:38:51.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.727+0000 7fdabffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdac00cf960 0x7fdac00d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:51.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.727+0000 7fdac4f88640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fdac015dc80 0x7fdac00cffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40064/0 (socket says 172.21.15.83:40064) 2024-04-17T10:38:51.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.727+0000 7fdabffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdac00cf960 0x7fdac00d1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51602/0 (socket says 172.21.15.83:51602) 2024-04-17T10:38:51.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.727+0000 7fdac4f88640 1 -- 172.21.15.83:0/2847262296 learned_addr learned my addr 172.21.15.83:0/2847262296 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:51.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.727+0000 7fdac5789640 1 --2- 172.21.15.83:0/2847262296 >> v2:172.21.15.167:3300/0 conn(0x7fdac00d1ca0 0x7fdac0167790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:51.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.727+0000 7fdabffff640 1 -- 172.21.15.83:0/2847262296 >> v2:172.21.15.167:3300/0 conn(0x7fdac00d1ca0 msgr2=0x7fdac0167790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:51.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.727+0000 7fdabffff640 1 --2- 172.21.15.83:0/2847262296 >> v2:172.21.15.167:3300/0 conn(0x7fdac00d1ca0 0x7fdac0167790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:51.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.727+0000 7fdabf7fe640 1 -- 172.21.15.83:0/2847262296 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1606764054 0 0) 0x7fdac015e060 con 0x7fdac015dc80 2024-04-17T10:38:51.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.727+0000 7fdabffff640 1 -- 172.21.15.83:0/2847262296 >> v1:172.21.15.83:6789/0 conn(0x7fdac015dc80 legacy=0x7fdac00cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:51.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.727+0000 7fdabffff640 1 -- 172.21.15.83:0/2847262296 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac006cf40 con 0x7fdac00cf960 2024-04-17T10:38:51.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.727+0000 7fdac5789640 1 --2- 172.21.15.83:0/2847262296 >> v2:172.21.15.167:3300/0 conn(0x7fdac00d1ca0 0x7fdac0167790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:51.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.731+0000 7fdabffff640 1 --2- 172.21.15.83:0/2847262296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdac00cf960 0x7fdac00d1760 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdac00689c0 tx=0x7fdaa801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c69a0b063c47bf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:51.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.731+0000 7fdabf7fe640 1 -- 172.21.15.83:0/2847262296 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa80b0070 con 0x7fdac00cf960 2024-04-17T10:38:51.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.731+0000 7fdabf7fe640 1 -- 172.21.15.83:0/2847262296 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa80ab600 con 0x7fdac00cf960 2024-04-17T10:38:51.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.731+0000 7fdabf7fe640 1 -- 172.21.15.83:0/2847262296 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa80ab9d0 con 0x7fdac00cf960 2024-04-17T10:38:51.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.731+0000 7fdac71ec640 1 -- 172.21.15.83:0/2847262296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdac00cf960 msgr2=0x7fdac00d1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:51.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.731+0000 7fdac71ec640 1 --2- 172.21.15.83:0/2847262296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdac00cf960 0x7fdac00d1760 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdac00689c0 tx=0x7fdaa801c480 comp rx=0 tx=0).stop 2024-04-17T10:38:51.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.731+0000 7fdac71ec640 1 -- 172.21.15.83:0/2847262296 shutdown_connections 2024-04-17T10:38:51.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.731+0000 7fdac71ec640 1 --2- 172.21.15.83:0/2847262296 >> v2:172.21.15.167:3300/0 conn(0x7fdac00d1ca0 0x7fdac0167790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:51.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.731+0000 7fdac71ec640 1 --2- 172.21.15.83:0/2847262296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdac00cf960 0x7fdac00d1760 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:51.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.731+0000 7fdac71ec640 1 -- 172.21.15.83:0/2847262296 >> 172.21.15.83:0/2847262296 conn(0x7fdac009a9d0 msgr2=0x7fdac009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:51.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.731+0000 7fdac71ec640 1 -- 172.21.15.83:0/2847262296 shutdown_connections 2024-04-17T10:38:51.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.731+0000 7fdac71ec640 1 -- 172.21.15.83:0/2847262296 wait complete. 2024-04-17T10:38:51.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.731+0000 7fdac71ec640 1 Processor -- start 2024-04-17T10:38:51.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.731+0000 7fdac71ec640 1 -- start start 2024-04-17T10:38:51.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.731+0000 7fdac71ec640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdac00cf960 0x7fdac015bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:51.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.731+0000 7fdac71ec640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdac00d1ca0 0x7fdac015b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:51.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.731+0000 7fdac71ec640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdac015dc80 0x7fdac00b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:51.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.731+0000 7fdac71ec640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac006dba0 con 0x7fdac00d1ca0 2024-04-17T10:38:51.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.731+0000 7fdac71ec640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac006da20 con 0x7fdac00cf960 2024-04-17T10:38:51.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.731+0000 7fdac71ec640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac016a4f0 con 0x7fdac015dc80 2024-04-17T10:38:51.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.731+0000 7fdabffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdac00d1ca0 0x7fdac015b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:51.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.731+0000 7fdabffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdac00d1ca0 0x7fdac015b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51610/0 (socket says 172.21.15.83:51610) 2024-04-17T10:38:51.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.731+0000 7fdabffff640 1 -- 172.21.15.83:0/67945156 learned_addr learned my addr 172.21.15.83:0/67945156 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:51.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.731+0000 7fdac4f88640 1 --2- 172.21.15.83:0/67945156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdac00cf960 0x7fdac015bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:51.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.731+0000 7fdabffff640 1 -- 172.21.15.83:0/67945156 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdac015dc80 msgr2=0x7fdac00b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:51.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.731+0000 7fdac5789640 1 --2- 172.21.15.83:0/67945156 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdac015dc80 0x7fdac00b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:51.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.731+0000 7fdabffff640 1 --2- 172.21.15.83:0/67945156 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdac015dc80 0x7fdac00b14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:51.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.731+0000 7fdabffff640 1 -- 172.21.15.83:0/67945156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdac00cf960 msgr2=0x7fdac015bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:51.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.731+0000 7fdabffff640 1 --2- 172.21.15.83:0/67945156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdac00cf960 0x7fdac015bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:51.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.731+0000 7fdabffff640 1 -- 172.21.15.83:0/67945156 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac015c3a0 con 0x7fdac00d1ca0 2024-04-17T10:38:51.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.731+0000 7fdac4f88640 1 --2- 172.21.15.83:0/67945156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdac00cf960 0x7fdac015bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:51.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.731+0000 7fdabffff640 1 --2- 172.21.15.83:0/67945156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdac00d1ca0 0x7fdac015b3a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80ae040 tx=0x7fdaa8019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:51.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.735+0000 7fdabd7fa640 1 -- 172.21.15.83:0/67945156 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa80b0040 con 0x7fdac00d1ca0 2024-04-17T10:38:51.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.735+0000 7fdabd7fa640 1 -- 172.21.15.83:0/67945156 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa80ba070 con 0x7fdac00d1ca0 2024-04-17T10:38:51.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.735+0000 7fdac71ec640 1 -- 172.21.15.83:0/67945156 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac015c680 con 0x7fdac00d1ca0 2024-04-17T10:38:51.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.735+0000 7fdabd7fa640 1 -- 172.21.15.83:0/67945156 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa80b4870 con 0x7fdac00d1ca0 2024-04-17T10:38:51.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.735+0000 7fdac71ec640 1 -- 172.21.15.83:0/67945156 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac015cc40 con 0x7fdac00d1ca0 2024-04-17T10:38:51.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.735+0000 7fdac71ec640 1 -- 172.21.15.83:0/67945156 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac0074f90 con 0x7fdac00d1ca0 2024-04-17T10:38:51.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.735+0000 7fdabd7fa640 1 -- 172.21.15.83:0/67945156 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdaa80b4a10 con 0x7fdac00d1ca0 2024-04-17T10:38:51.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.735+0000 7fdabd7fa640 1 --2- 172.21.15.83:0/67945156 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdaa0066f90 0x7fdaa0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:51.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.735+0000 7fdabd7fa640 1 -- 172.21.15.83:0/67945156 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdaa811ee70 con 0x7fdac00d1ca0 2024-04-17T10:38:51.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.735+0000 7fdac4f88640 1 --2- 172.21.15.83:0/67945156 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdaa0066f90 0x7fdaa0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:51.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.735+0000 7fdac4f88640 1 --2- 172.21.15.83:0/67945156 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdaa0066f90 0x7fdaa0069450 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdab80012f0 tx=0x7fdab8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:51.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.739+0000 7fdabd7fa640 1 -- 172.21.15.83:0/67945156 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdaa8016070 con 0x7fdac00d1ca0 2024-04-17T10:38:51.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.899+0000 7fdac71ec640 1 -- 172.21.15.83:0/67945156 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdac00d24c0 con 0x7fdac00d1ca0 2024-04-17T10:38:51.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.903+0000 7fdabd7fa640 1 -- 172.21.15.83:0/67945156 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16533 (secure 0 0 0) 0x7fdaa80eff10 con 0x7fdac00d1ca0 2024-04-17T10:38:51.900 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:51.900 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":20,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:38:13.690876+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:38:10.464490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T10:38:11.043134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:38:51.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.903+0000 7fdac71ec640 1 -- 172.21.15.83:0/67945156 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdaa0066f90 msgr2=0x7fdaa0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:51.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.903+0000 7fdac71ec640 1 --2- 172.21.15.83:0/67945156 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdaa0066f90 0x7fdaa0069450 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdab80012f0 tx=0x7fdab8009040 comp rx=0 tx=0).stop 2024-04-17T10:38:51.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.903+0000 7fdac71ec640 1 -- 172.21.15.83:0/67945156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdac00d1ca0 msgr2=0x7fdac015b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:51.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.903+0000 7fdac71ec640 1 --2- 172.21.15.83:0/67945156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdac00d1ca0 0x7fdac015b3a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80ae040 tx=0x7fdaa8019220 comp rx=0 tx=0).stop 2024-04-17T10:38:51.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.903+0000 7fdac71ec640 1 -- 172.21.15.83:0/67945156 shutdown_connections 2024-04-17T10:38:51.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.903+0000 7fdac71ec640 1 --2- 172.21.15.83:0/67945156 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdaa0066f90 0x7fdaa0069450 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:51.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.903+0000 7fdac71ec640 1 --2- 172.21.15.83:0/67945156 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdac015dc80 0x7fdac00b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:51.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.903+0000 7fdac71ec640 1 --2- 172.21.15.83:0/67945156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdac00d1ca0 0x7fdac015b3a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:51.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.903+0000 7fdac71ec640 1 --2- 172.21.15.83:0/67945156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdac00cf960 0x7fdac015bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:51.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.903+0000 7fdac71ec640 1 -- 172.21.15.83:0/67945156 >> 172.21.15.83:0/67945156 conn(0x7fdac009a9d0 msgr2=0x7fdac006f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:51.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.903+0000 7fdac71ec640 1 -- 172.21.15.83:0/67945156 shutdown_connections 2024-04-17T10:38:51.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:51.903+0000 7fdac71ec640 1 -- 172.21.15.83:0/67945156 wait complete. 2024-04-17T10:38:51.919 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:38:10.464490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:38:11.043134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:38:51.919 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:38:52.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.047+0000 7f5d320bd640 1 Processor -- start 2024-04-17T10:38:52.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.047+0000 7f5d320bd640 1 -- start start 2024-04-17T10:38:52.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.047+0000 7f5d320bd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5d2c0b01b0 0x7f5d2c0ae590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:52.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.047+0000 7f5d320bd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5d2c14b7e0 0x7f5d2c14bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:52.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.047+0000 7f5d320bd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d2c0c47b0 0x7f5d2c0c4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:52.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.047+0000 7f5d320bd640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d2c0c5160 con 0x7f5d2c0b01b0 2024-04-17T10:38:52.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.047+0000 7f5d320bd640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d2c0aead0 con 0x7f5d2c14b7e0 2024-04-17T10:38:52.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.047+0000 7f5d320bd640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c06dd40 con 0x7f5d2c0c47b0 2024-04-17T10:38:52.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.047+0000 7f5d2b7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5d2c0b01b0 0x7f5d2c0ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:52.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.047+0000 7f5d2b7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5d2c0b01b0 0x7f5d2c0ae590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51616/0 (socket says 172.21.15.83:51616) 2024-04-17T10:38:52.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.047+0000 7f5d2b7fe640 1 -- 172.21.15.83:0/2128986574 learned_addr learned my addr 172.21.15.83:0/2128986574 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:52.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.047+0000 7f5d2bfff640 1 --2- 172.21.15.83:0/2128986574 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d2c0c47b0 0x7f5d2c0c4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:52.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.047+0000 7f5d2affd640 1 --2- 172.21.15.83:0/2128986574 >> v2:172.21.15.167:3300/0 conn(0x7f5d2c14b7e0 0x7f5d2c14bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:52.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.047+0000 7f5d2b7fe640 1 -- 172.21.15.83:0/2128986574 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d2c0c47b0 msgr2=0x7f5d2c0c4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:52.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.047+0000 7f5d2b7fe640 1 --2- 172.21.15.83:0/2128986574 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d2c0c47b0 0x7f5d2c0c4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.047+0000 7f5d2b7fe640 1 -- 172.21.15.83:0/2128986574 >> v2:172.21.15.167:3300/0 conn(0x7f5d2c14b7e0 msgr2=0x7f5d2c14bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:52.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.047+0000 7f5d2b7fe640 1 --2- 172.21.15.83:0/2128986574 >> v2:172.21.15.167:3300/0 conn(0x7f5d2c14b7e0 0x7f5d2c14bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.047+0000 7f5d2b7fe640 1 -- 172.21.15.83:0/2128986574 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2c06e6a0 con 0x7f5d2c0b01b0 2024-04-17T10:38:52.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.047+0000 7f5d2affd640 1 --2- 172.21.15.83:0/2128986574 >> v2:172.21.15.167:3300/0 conn(0x7f5d2c14b7e0 0x7f5d2c14bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:52.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.051+0000 7f5d2b7fe640 1 --2- 172.21.15.83:0/2128986574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d2c0b01b0 0x7f5d2c0ae590 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c0718b0 tx=0x7f5d1c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e671969e3cec222f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:52.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.051+0000 7f5d2a7fc640 1 -- 172.21.15.83:0/2128986574 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c0aa610 con 0x7f5d2c0b01b0 2024-04-17T10:38:52.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.051+0000 7f5d2a7fc640 1 -- 172.21.15.83:0/2128986574 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1c0af070 con 0x7f5d2c0b01b0 2024-04-17T10:38:52.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.051+0000 7f5d2a7fc640 1 -- 172.21.15.83:0/2128986574 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c0aa9d0 con 0x7f5d2c0b01b0 2024-04-17T10:38:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.051+0000 7f5d320bd640 1 -- 172.21.15.83:0/2128986574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d2c0b01b0 msgr2=0x7f5d2c0ae590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.051+0000 7f5d320bd640 1 --2- 172.21.15.83:0/2128986574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d2c0b01b0 0x7f5d2c0ae590 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c0718b0 tx=0x7f5d1c01c4f0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.051+0000 7f5d320bd640 1 -- 172.21.15.83:0/2128986574 shutdown_connections 2024-04-17T10:38:52.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.051+0000 7f5d320bd640 1 --2- 172.21.15.83:0/2128986574 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d2c0c47b0 0x7f5d2c0c4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.051+0000 7f5d320bd640 1 --2- 172.21.15.83:0/2128986574 >> v2:172.21.15.167:3300/0 conn(0x7f5d2c14b7e0 0x7f5d2c14bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.051+0000 7f5d320bd640 1 --2- 172.21.15.83:0/2128986574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d2c0b01b0 0x7f5d2c0ae590 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.051+0000 7f5d320bd640 1 -- 172.21.15.83:0/2128986574 >> 172.21.15.83:0/2128986574 conn(0x7f5d2c0d17e0 msgr2=0x7f5d2c0d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:52.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.051+0000 7f5d320bd640 1 -- 172.21.15.83:0/2128986574 shutdown_connections 2024-04-17T10:38:52.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.051+0000 7f5d320bd640 1 -- 172.21.15.83:0/2128986574 wait complete. 2024-04-17T10:38:52.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.051+0000 7f5d320bd640 1 Processor -- start 2024-04-17T10:38:52.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.051+0000 7f5d320bd640 1 -- start start 2024-04-17T10:38:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.055+0000 7f5d320bd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d2c0b01b0 0x7f5d2c14a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.055+0000 7f5d320bd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d2c0c47b0 0x7f5d2c14ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.055+0000 7f5d320bd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d2c14b7e0 0x7f5d2c15d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:52.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.055+0000 7f5d320bd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c0b0590 con 0x7f5d2c0b01b0 2024-04-17T10:38:52.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.055+0000 7f5d320bd640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c06f220 con 0x7f5d2c0c47b0 2024-04-17T10:38:52.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.055+0000 7f5d320bd640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c0b0710 con 0x7f5d2c14b7e0 2024-04-17T10:38:52.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.055+0000 7f5d2b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d2c0b01b0 0x7f5d2c14a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:52.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.055+0000 7f5d2b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d2c0b01b0 0x7f5d2c14a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51632/0 (socket says 172.21.15.83:51632) 2024-04-17T10:38:52.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.055+0000 7f5d2b7fe640 1 -- 172.21.15.83:0/1326490869 learned_addr learned my addr 172.21.15.83:0/1326490869 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:52.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.055+0000 7f5d2affd640 1 --2- 172.21.15.83:0/1326490869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d2c0c47b0 0x7f5d2c14ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:52.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.055+0000 7f5d2bfff640 1 --2- 172.21.15.83:0/1326490869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d2c14b7e0 0x7f5d2c15d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:52.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.055+0000 7f5d2b7fe640 1 -- 172.21.15.83:0/1326490869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d2c14b7e0 msgr2=0x7f5d2c15d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:52.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.055+0000 7f5d2b7fe640 1 --2- 172.21.15.83:0/1326490869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d2c14b7e0 0x7f5d2c15d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.055+0000 7f5d2b7fe640 1 -- 172.21.15.83:0/1326490869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d2c0c47b0 msgr2=0x7f5d2c14ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:52.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.055+0000 7f5d2b7fe640 1 --2- 172.21.15.83:0/1326490869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d2c0c47b0 0x7f5d2c14ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.055+0000 7f5d2b7fe640 1 -- 172.21.15.83:0/1326490869 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2c15c4c0 con 0x7f5d2c0b01b0 2024-04-17T10:38:52.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.055+0000 7f5d2bfff640 1 --2- 172.21.15.83:0/1326490869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d2c14b7e0 0x7f5d2c15d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:52.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.055+0000 7f5d2b7fe640 1 --2- 172.21.15.83:0/1326490869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d2c0b01b0 0x7f5d2c14a320 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c0ac040 tx=0x7f5d1c01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:52.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.055+0000 7f5d13fff640 1 -- 172.21.15.83:0/1326490869 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c0bd5f0 con 0x7f5d2c0b01b0 2024-04-17T10:38:52.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.055+0000 7f5d13fff640 1 -- 172.21.15.83:0/1326490869 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1c0bd790 con 0x7f5d2c0b01b0 2024-04-17T10:38:52.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.055+0000 7f5d320bd640 1 -- 172.21.15.83:0/1326490869 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d2c15c7a0 con 0x7f5d2c0b01b0 2024-04-17T10:38:52.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.055+0000 7f5d13fff640 1 -- 172.21.15.83:0/1326490869 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c0bdb80 con 0x7f5d2c0b01b0 2024-04-17T10:38:52.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.055+0000 7f5d320bd640 1 -- 172.21.15.83:0/1326490869 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d2c15cd60 con 0x7f5d2c0b01b0 2024-04-17T10:38:52.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.055+0000 7f5d320bd640 1 -- 172.21.15.83:0/1326490869 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d2c074f90 con 0x7f5d2c0b01b0 2024-04-17T10:38:52.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.055+0000 7f5d13fff640 1 -- 172.21.15.83:0/1326490869 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5d1c0b33f0 con 0x7f5d2c0b01b0 2024-04-17T10:38:52.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.055+0000 7f5d13fff640 1 --2- 172.21.15.83:0/1326490869 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5d18066f90 0x7f5d18069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:52.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.055+0000 7f5d13fff640 1 -- 172.21.15.83:0/1326490869 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d1c11fae0 con 0x7f5d2c0b01b0 2024-04-17T10:38:52.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.055+0000 7f5d2affd640 1 --2- 172.21.15.83:0/1326490869 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5d18066f90 0x7f5d18069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:52.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.059+0000 7f5d2affd640 1 --2- 172.21.15.83:0/1326490869 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5d18066f90 0x7f5d18069450 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5d20006fd0 tx=0x7f5d20008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:52.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.063+0000 7f5d13fff640 1 -- 172.21.15.83:0/1326490869 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5d1c0c3060 con 0x7f5d2c0b01b0 2024-04-17T10:38:52.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.207+0000 7f5d320bd640 1 -- 172.21.15.83:0/1326490869 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5d2c0ca340 con 0x7f5d2c0b01b0 2024-04-17T10:38:52.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.207+0000 7f5d13fff640 1 -- 172.21.15.83:0/1326490869 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5d1c0a7070 con 0x7f5d2c0b01b0 2024-04-17T10:38:52.206 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:38:52.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.211+0000 7f5d320bd640 1 -- 172.21.15.83:0/1326490869 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5d18066f90 msgr2=0x7f5d18069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:52.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.211+0000 7f5d320bd640 1 --2- 172.21.15.83:0/1326490869 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5d18066f90 0x7f5d18069450 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5d20006fd0 tx=0x7f5d20008040 comp rx=0 tx=0).stop 2024-04-17T10:38:52.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.211+0000 7f5d320bd640 1 -- 172.21.15.83:0/1326490869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d2c0b01b0 msgr2=0x7f5d2c14a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:52.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.211+0000 7f5d320bd640 1 --2- 172.21.15.83:0/1326490869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d2c0b01b0 0x7f5d2c14a320 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c0ac040 tx=0x7f5d1c01ad30 comp rx=0 tx=0).stop 2024-04-17T10:38:52.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.211+0000 7f5d320bd640 1 -- 172.21.15.83:0/1326490869 shutdown_connections 2024-04-17T10:38:52.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.211+0000 7f5d320bd640 1 --2- 172.21.15.83:0/1326490869 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5d18066f90 0x7f5d18069450 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.211+0000 7f5d320bd640 1 --2- 172.21.15.83:0/1326490869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d2c14b7e0 0x7f5d2c15d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.211+0000 7f5d320bd640 1 --2- 172.21.15.83:0/1326490869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d2c0c47b0 0x7f5d2c14ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.211+0000 7f5d320bd640 1 --2- 172.21.15.83:0/1326490869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d2c0b01b0 0x7f5d2c14a320 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.211+0000 7f5d320bd640 1 -- 172.21.15.83:0/1326490869 >> 172.21.15.83:0/1326490869 conn(0x7f5d2c0d17e0 msgr2=0x7f5d2c0d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:52.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.211+0000 7f5d320bd640 1 -- 172.21.15.83:0/1326490869 shutdown_connections 2024-04-17T10:38:52.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.211+0000 7f5d320bd640 1 -- 172.21.15.83:0/1326490869 wait complete. 2024-04-17T10:38:52.224 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:38:52.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.347+0000 7fea68aff640 1 Processor -- start 2024-04-17T10:38:52.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.347+0000 7fea68aff640 1 -- start start 2024-04-17T10:38:52.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.351+0000 7fea68aff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fea6415b6e0 0x7fea64071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:52.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.351+0000 7fea68aff640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea6406c1f0 con 0x7fea64070470 2024-04-17T10:38:52.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.351+0000 7fea68aff640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea64070a10 con 0x7fea6415bac0 2024-04-17T10:38:52.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.351+0000 7fea68aff640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea64070cf0 con 0x7fea6415b6e0 2024-04-17T10:38:52.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.351+0000 7fea6259c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fea6415b6e0 0x7fea64071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:52.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.351+0000 7fea6259c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fea6415b6e0 0x7fea64071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51644/0 (socket says 172.21.15.83:51644) 2024-04-17T10:38:52.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.351+0000 7fea6259c640 1 -- 172.21.15.83:0/4250716260 learned_addr learned my addr 172.21.15.83:0/4250716260 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:52.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.351+0000 7fea6259c640 1 -- 172.21.15.83:0/4250716260 >> v1:172.21.15.167:6789/0 conn(0x7fea6415bac0 legacy=0x7fea640cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:52.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.351+0000 7fea6259c640 1 -- 172.21.15.83:0/4250716260 >> v1:172.21.15.83:6789/0 conn(0x7fea64070470 legacy=0x7fea6406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:52.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.351+0000 7fea6259c640 1 -- 172.21.15.83:0/4250716260 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea64070fd0 con 0x7fea6415b6e0 2024-04-17T10:38:52.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.351+0000 7fea6259c640 1 --2- 172.21.15.83:0/4250716260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea6415b6e0 0x7fea64071cc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fea54009870 tx=0x7fea5401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c357f9be7c50bdd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:52.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.351+0000 7fea61d9b640 1 -- 172.21.15.83:0/4250716260 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea540b0070 con 0x7fea6415b6e0 2024-04-17T10:38:52.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.351+0000 7fea61d9b640 1 -- 172.21.15.83:0/4250716260 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea540ab600 con 0x7fea6415b6e0 2024-04-17T10:38:52.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.351+0000 7fea61d9b640 1 -- 172.21.15.83:0/4250716260 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea540ab9d0 con 0x7fea6415b6e0 2024-04-17T10:38:52.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.355+0000 7fea68aff640 1 -- 172.21.15.83:0/4250716260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea6415b6e0 msgr2=0x7fea64071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:52.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.355+0000 7fea68aff640 1 --2- 172.21.15.83:0/4250716260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea6415b6e0 0x7fea64071cc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fea54009870 tx=0x7fea5401c520 comp rx=0 tx=0).stop 2024-04-17T10:38:52.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.355+0000 7fea68aff640 1 -- 172.21.15.83:0/4250716260 shutdown_connections 2024-04-17T10:38:52.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.355+0000 7fea68aff640 1 --2- 172.21.15.83:0/4250716260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea6415b6e0 0x7fea64071cc0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.355+0000 7fea68aff640 1 -- 172.21.15.83:0/4250716260 >> 172.21.15.83:0/4250716260 conn(0x7fea640b15b0 msgr2=0x7fea640b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:52.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.355+0000 7fea68aff640 1 -- 172.21.15.83:0/4250716260 shutdown_connections 2024-04-17T10:38:52.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.355+0000 7fea68aff640 1 -- 172.21.15.83:0/4250716260 wait complete. 2024-04-17T10:38:52.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.355+0000 7fea68aff640 1 Processor -- start 2024-04-17T10:38:52.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.355+0000 7fea68aff640 1 -- start start 2024-04-17T10:38:52.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.355+0000 7fea68aff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea64070470 0x7fea64151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:52.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.355+0000 7fea68aff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea6415b6e0 0x7fea641525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:52.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.355+0000 7fea68aff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea6415bac0 0x7fea6413bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:52.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.355+0000 7fea68aff640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6406cee0 con 0x7fea6415b6e0 2024-04-17T10:38:52.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.355+0000 7fea68aff640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6406d060 con 0x7fea64070470 2024-04-17T10:38:52.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.355+0000 7fea68aff640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea6406d1e0 con 0x7fea6415bac0 2024-04-17T10:38:52.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.355+0000 7fea59d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea6415b6e0 0x7fea641525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:52.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.355+0000 7fea59d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea6415b6e0 0x7fea641525c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51652/0 (socket says 172.21.15.83:51652) 2024-04-17T10:38:52.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.355+0000 7fea59d9b640 1 -- 172.21.15.83:0/2434854267 learned_addr learned my addr 172.21.15.83:0/2434854267 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:52.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.355+0000 7fea6259c640 1 --2- 172.21.15.83:0/2434854267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea64070470 0x7fea64151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:52.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.355+0000 7fea62d9d640 1 --2- 172.21.15.83:0/2434854267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea6415bac0 0x7fea6413bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:52.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.355+0000 7fea59d9b640 1 -- 172.21.15.83:0/2434854267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea6415bac0 msgr2=0x7fea6413bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:52.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.355+0000 7fea59d9b640 1 --2- 172.21.15.83:0/2434854267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea6415bac0 0x7fea6413bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.355+0000 7fea59d9b640 1 -- 172.21.15.83:0/2434854267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea64070470 msgr2=0x7fea64151ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:52.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.355+0000 7fea59d9b640 1 --2- 172.21.15.83:0/2434854267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea64070470 0x7fea64151ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.355+0000 7fea59d9b640 1 -- 172.21.15.83:0/2434854267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea64152d30 con 0x7fea6415b6e0 2024-04-17T10:38:52.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.355+0000 7fea59d9b640 1 --2- 172.21.15.83:0/2434854267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea6415b6e0 0x7fea641525c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fea4c00c490 tx=0x7fea4c001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:52.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.355+0000 7fea5b7fe640 1 -- 172.21.15.83:0/2434854267 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4c017070 con 0x7fea6415b6e0 2024-04-17T10:38:52.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.355+0000 7fea5b7fe640 1 -- 172.21.15.83:0/2434854267 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea4c003600 con 0x7fea6415b6e0 2024-04-17T10:38:52.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.355+0000 7fea68aff640 1 -- 172.21.15.83:0/2434854267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea64153010 con 0x7fea6415b6e0 2024-04-17T10:38:52.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.359+0000 7fea5b7fe640 1 -- 172.21.15.83:0/2434854267 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4c003a30 con 0x7fea6415b6e0 2024-04-17T10:38:52.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.359+0000 7fea68aff640 1 -- 172.21.15.83:0/2434854267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea641535d0 con 0x7fea6415b6e0 2024-04-17T10:38:52.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.359+0000 7fea68aff640 1 -- 172.21.15.83:0/2434854267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea64074f90 con 0x7fea6415b6e0 2024-04-17T10:38:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.359+0000 7fea5b7fe640 1 -- 172.21.15.83:0/2434854267 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fea4c01a070 con 0x7fea6415b6e0 2024-04-17T10:38:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.359+0000 7fea5b7fe640 1 --2- 172.21.15.83:0/2434854267 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fea44066f60 0x7fea44069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.359+0000 7fea5b7fe640 1 -- 172.21.15.83:0/2434854267 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea4c084400 con 0x7fea6415b6e0 2024-04-17T10:38:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.359+0000 7fea6259c640 1 --2- 172.21.15.83:0/2434854267 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fea44066f60 0x7fea44069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.359+0000 7fea6259c640 1 --2- 172.21.15.83:0/2434854267 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fea44066f60 0x7fea44069420 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fea540ad000 tx=0x7fea540a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:52.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.363+0000 7fea5b7fe640 1 -- 172.21.15.83:0/2434854267 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fea4c0554a0 con 0x7fea6415b6e0 2024-04-17T10:38:52.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.515+0000 7fea68aff640 1 -- 172.21.15.83:0/2434854267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fea64150750 con 0x7fea6415b6e0 2024-04-17T10:38:52.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.519+0000 7fea5b7fe640 1 -- 172.21.15.83:0/2434854267 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fea4c054da0 con 0x7fea6415b6e0 2024-04-17T10:38:52.515 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:38:52.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.519+0000 7fea68aff640 1 -- 172.21.15.83:0/2434854267 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fea44066f60 msgr2=0x7fea44069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:52.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.519+0000 7fea68aff640 1 --2- 172.21.15.83:0/2434854267 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fea44066f60 0x7fea44069420 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fea540ad000 tx=0x7fea540a6000 comp rx=0 tx=0).stop 2024-04-17T10:38:52.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.519+0000 7fea68aff640 1 -- 172.21.15.83:0/2434854267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea6415b6e0 msgr2=0x7fea641525c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:52.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.519+0000 7fea68aff640 1 --2- 172.21.15.83:0/2434854267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea6415b6e0 0x7fea641525c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fea4c00c490 tx=0x7fea4c001e80 comp rx=0 tx=0).stop 2024-04-17T10:38:52.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.519+0000 7fea68aff640 1 -- 172.21.15.83:0/2434854267 shutdown_connections 2024-04-17T10:38:52.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.519+0000 7fea68aff640 1 --2- 172.21.15.83:0/2434854267 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fea44066f60 0x7fea44069420 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.519+0000 7fea68aff640 1 --2- 172.21.15.83:0/2434854267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea6415bac0 0x7fea6413bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.519+0000 7fea68aff640 1 --2- 172.21.15.83:0/2434854267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea6415b6e0 0x7fea641525c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.519+0000 7fea68aff640 1 --2- 172.21.15.83:0/2434854267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea64070470 0x7fea64151ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.519+0000 7fea68aff640 1 -- 172.21.15.83:0/2434854267 >> 172.21.15.83:0/2434854267 conn(0x7fea640b15b0 msgr2=0x7fea6406ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:52.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.519+0000 7fea68aff640 1 -- 172.21.15.83:0/2434854267 shutdown_connections 2024-04-17T10:38:52.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.519+0000 7fea68aff640 1 -- 172.21.15.83:0/2434854267 wait complete. 2024-04-17T10:38:52.532 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:38:52.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f2037bb1640 1 Processor -- start 2024-04-17T10:38:52.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f2037bb1640 1 -- start start 2024-04-17T10:38:52.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f2037bb1640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f203006b520 0x7f20300b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:52.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f2037bb1640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f203015bd70 con 0x7f2030154de0 2024-04-17T10:38:52.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f2037bb1640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f203015bf60 con 0x7f20300b4040 2024-04-17T10:38:52.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f2037bb1640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20300b4870 con 0x7f203006b520 2024-04-17T10:38:52.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f203594d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2030154de0 0x7f203006ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40084/0 (socket says 172.21.15.83:40084) 2024-04-17T10:38:52.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f203594d640 1 -- 172.21.15.83:0/2827309112 learned_addr learned my addr 172.21.15.83:0/2827309112 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:52.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f203494b640 1 -- 172.21.15.83:0/2827309112 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3963763876 0 0) 0x7f203015bd70 con 0x7f2030154de0 2024-04-17T10:38:52.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f203494b640 1 -- 172.21.15.83:0/2827309112 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f201c003530 con 0x7f2030154de0 2024-04-17T10:38:52.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f203514c640 1 --2- 172.21.15.83:0/2827309112 >> v2:172.21.15.167:3300/0 conn(0x7f203006b520 0x7f20300b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:52.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f203494b640 1 -- 172.21.15.83:0/2827309112 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2964402347 0 0) 0x7f201c003530 con 0x7f2030154de0 2024-04-17T10:38:52.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f203494b640 1 -- 172.21.15.83:0/2827309112 >> v2:172.21.15.167:3300/0 conn(0x7f203006b520 msgr2=0x7f20300b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:52.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f203494b640 1 --2- 172.21.15.83:0/2827309112 >> v2:172.21.15.167:3300/0 conn(0x7f203006b520 0x7f20300b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f203494b640 1 -- 172.21.15.83:0/2827309112 >> v1:172.21.15.167:6789/0 conn(0x7f20300b4040 legacy=0x7f203006cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:52.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f203494b640 1 -- 172.21.15.83:0/2827309112 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203015afb0 con 0x7f2030154de0 2024-04-17T10:38:52.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f203494b640 1 -- 172.21.15.83:0/2827309112 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f202c001680 con 0x7f2030154de0 2024-04-17T10:38:52.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f203494b640 1 -- 172.21.15.83:0/2827309112 >> v1:172.21.15.83:6789/0 conn(0x7f2030154de0 legacy=0x7f203006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:52.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f203494b640 1 --2- 172.21.15.83:0/2827309112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f201c003a30 0x7f201c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:52.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f203494b640 1 --2- 172.21.15.83:0/2827309112 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f201c004810 0x7f201c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:52.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f203494b640 1 --2- 172.21.15.83:0/2827309112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f201c0051d0 0x7f201c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:52.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f203494b640 1 -- 172.21.15.83:0/2827309112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201c00c0d0 con 0x7f201c003a30 2024-04-17T10:38:52.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f203494b640 1 -- 172.21.15.83:0/2827309112 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201c00c280 con 0x7f201c0051d0 2024-04-17T10:38:52.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f203494b640 1 -- 172.21.15.83:0/2827309112 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f201c00c480 con 0x7f201c004810 2024-04-17T10:38:52.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f203594d640 1 --2- 172.21.15.83:0/2827309112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f201c003a30 0x7f201c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:52.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f203514c640 1 --2- 172.21.15.83:0/2827309112 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f201c004810 0x7f201c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:52.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f203614e640 1 --2- 172.21.15.83:0/2827309112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f201c0051d0 0x7f201c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:52.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f203594d640 1 -- 172.21.15.83:0/2827309112 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f201c004810 msgr2=0x7f201c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:52.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f203594d640 1 --2- 172.21.15.83:0/2827309112 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f201c004810 0x7f201c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f203594d640 1 -- 172.21.15.83:0/2827309112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f201c0051d0 msgr2=0x7f201c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:52.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f203594d640 1 --2- 172.21.15.83:0/2827309112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f201c0051d0 0x7f201c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f203614e640 1 -- 172.21.15.83:0/2827309112 reap_dead start 2024-04-17T10:38:52.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.647+0000 7f203594d640 1 -- 172.21.15.83:0/2827309112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f201c003530 con 0x7f201c003a30 2024-04-17T10:38:52.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.651+0000 7f203594d640 1 --2- 172.21.15.83:0/2827309112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f201c003a30 0x7f201c003e50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f202c00c240 tx=0x7f202c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a679f2bc89c45 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:52.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.651+0000 7f203494b640 1 -- 172.21.15.83:0/2827309112 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c007d20 con 0x7f201c003a30 2024-04-17T10:38:52.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.651+0000 7f203494b640 1 -- 172.21.15.83:0/2827309112 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202c0a9070 con 0x7f201c003a30 2024-04-17T10:38:52.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.651+0000 7f203494b640 1 -- 172.21.15.83:0/2827309112 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c005da0 con 0x7f201c003a30 2024-04-17T10:38:52.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.651+0000 7f2037bb1640 1 -- 172.21.15.83:0/2827309112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f201c003a30 msgr2=0x7f201c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:52.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.651+0000 7f2037bb1640 1 --2- 172.21.15.83:0/2827309112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f201c003a30 0x7f201c003e50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f202c00c240 tx=0x7f202c01cc80 comp rx=0 tx=0).stop 2024-04-17T10:38:52.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.651+0000 7f2037bb1640 1 -- 172.21.15.83:0/2827309112 shutdown_connections 2024-04-17T10:38:52.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.651+0000 7f2037bb1640 1 --2- 172.21.15.83:0/2827309112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f201c003a30 0x7f201c003e50 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.651+0000 7f2037bb1640 1 -- 172.21.15.83:0/2827309112 >> 172.21.15.83:0/2827309112 conn(0x7f20300b15b0 msgr2=0x7f20300b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:52.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.651+0000 7f2037bb1640 1 -- 172.21.15.83:0/2827309112 shutdown_connections 2024-04-17T10:38:52.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.651+0000 7f2037bb1640 1 -- 172.21.15.83:0/2827309112 wait complete. 2024-04-17T10:38:52.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.651+0000 7f2037bb1640 1 Processor -- start 2024-04-17T10:38:52.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.651+0000 7f2037bb1640 1 -- start start 2024-04-17T10:38:52.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.651+0000 7f2037bb1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f201c003a30 0x7f203013c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:52.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.651+0000 7f2037bb1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20300cd760 0x7f20300cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:52.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.651+0000 7f2037bb1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20301553d0 0x7f2030155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:52.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.651+0000 7f2037bb1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203006b220 con 0x7f201c003a30 2024-04-17T10:38:52.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.651+0000 7f2037bb1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20301551c0 con 0x7f20301553d0 2024-04-17T10:38:52.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.651+0000 7f2037bb1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203006b3a0 con 0x7f20300cd760 2024-04-17T10:38:52.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.651+0000 7f203594d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f201c003a30 0x7f203013c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:52.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.651+0000 7f203594d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f201c003a30 0x7f203013c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51666/0 (socket says 172.21.15.83:51666) 2024-04-17T10:38:52.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.651+0000 7f203594d640 1 -- 172.21.15.83:0/13964315 learned_addr learned my addr 172.21.15.83:0/13964315 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:52.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.655+0000 7f203514c640 1 --2- 172.21.15.83:0/13964315 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20300cd760 0x7f20300cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:52.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.655+0000 7f203614e640 1 --2- 172.21.15.83:0/13964315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20301553d0 0x7f2030155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:52.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.655+0000 7f203594d640 1 -- 172.21.15.83:0/13964315 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20300cd760 msgr2=0x7f20300cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:52.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.655+0000 7f203594d640 1 --2- 172.21.15.83:0/13964315 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20300cd760 0x7f20300cdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.655+0000 7f203594d640 1 -- 172.21.15.83:0/13964315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20301553d0 msgr2=0x7f2030155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:52.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.655+0000 7f203594d640 1 --2- 172.21.15.83:0/13964315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20301553d0 0x7f2030155880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.655+0000 7f203594d640 1 -- 172.21.15.83:0/13964315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203013f170 con 0x7f201c003a30 2024-04-17T10:38:52.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.655+0000 7f203594d640 1 --2- 172.21.15.83:0/13964315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f201c003a30 0x7f203013c740 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f202c00c240 tx=0x7f201c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:52.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.655+0000 7f20267fc640 1 -- 172.21.15.83:0/13964315 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c007d20 con 0x7f201c003a30 2024-04-17T10:38:52.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.655+0000 7f20267fc640 1 -- 172.21.15.83:0/13964315 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202c000ee0 con 0x7f201c003a30 2024-04-17T10:38:52.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.655+0000 7f20267fc640 1 -- 172.21.15.83:0/13964315 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c001480 con 0x7f201c003a30 2024-04-17T10:38:52.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.655+0000 7f2037bb1640 1 -- 172.21.15.83:0/13964315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20301560e0 con 0x7f201c003a30 2024-04-17T10:38:52.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.655+0000 7f2037bb1640 1 -- 172.21.15.83:0/13964315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f203013f710 con 0x7f201c003a30 2024-04-17T10:38:52.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.655+0000 7f2037bb1640 1 -- 172.21.15.83:0/13964315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20300ce100 con 0x7f201c003a30 2024-04-17T10:38:52.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.655+0000 7f20267fc640 1 -- 172.21.15.83:0/13964315 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f202c007180 con 0x7f201c003a30 2024-04-17T10:38:52.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.655+0000 7f20267fc640 1 --2- 172.21.15.83:0/13964315 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2014066f90 0x7f2014069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:52.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.655+0000 7f20267fc640 1 -- 172.21.15.83:0/13964315 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f202c11e7a0 con 0x7f201c003a30 2024-04-17T10:38:52.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.655+0000 7f203514c640 1 --2- 172.21.15.83:0/13964315 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2014066f90 0x7f2014069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:52.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.655+0000 7f203514c640 1 --2- 172.21.15.83:0/13964315 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2014066f90 0x7f2014069450 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f20300b3250 tx=0x7f2020008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:52.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.659+0000 7f20267fc640 1 -- 172.21.15.83:0/13964315 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f202c016990 con 0x7f201c003a30 2024-04-17T10:38:52.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.819+0000 7f2037bb1640 1 -- 172.21.15.83:0/13964315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f20300b3280 con 0x7f201c003a30 2024-04-17T10:38:52.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.819+0000 7f20267fc640 1 -- 172.21.15.83:0/13964315 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f202c0ef6a0 con 0x7f201c003a30 2024-04-17T10:38:52.817 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:38:52.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.823+0000 7f2037bb1640 1 -- 172.21.15.83:0/13964315 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2014066f90 msgr2=0x7f2014069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:52.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.823+0000 7f2037bb1640 1 --2- 172.21.15.83:0/13964315 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2014066f90 0x7f2014069450 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f20300b3250 tx=0x7f2020008040 comp rx=0 tx=0).stop 2024-04-17T10:38:52.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.823+0000 7f2037bb1640 1 -- 172.21.15.83:0/13964315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f201c003a30 msgr2=0x7f203013c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:52.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.823+0000 7f2037bb1640 1 --2- 172.21.15.83:0/13964315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f201c003a30 0x7f203013c740 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f202c00c240 tx=0x7f201c005840 comp rx=0 tx=0).stop 2024-04-17T10:38:52.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.823+0000 7f2037bb1640 1 -- 172.21.15.83:0/13964315 shutdown_connections 2024-04-17T10:38:52.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.823+0000 7f2037bb1640 1 --2- 172.21.15.83:0/13964315 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2014066f90 0x7f2014069450 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.823+0000 7f2037bb1640 1 --2- 172.21.15.83:0/13964315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20301553d0 0x7f2030155880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.823+0000 7f2037bb1640 1 --2- 172.21.15.83:0/13964315 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20300cd760 0x7f20300cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.823+0000 7f2037bb1640 1 --2- 172.21.15.83:0/13964315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f201c003a30 0x7f203013c740 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.823+0000 7f2037bb1640 1 -- 172.21.15.83:0/13964315 >> 172.21.15.83:0/13964315 conn(0x7f20300b15b0 msgr2=0x7f2030158850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:52.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.823+0000 7f2037bb1640 1 -- 172.21.15.83:0/13964315 shutdown_connections 2024-04-17T10:38:52.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.823+0000 7f2037bb1640 1 -- 172.21.15.83:0/13964315 wait complete. 2024-04-17T10:38:52.836 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:38:52.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.955+0000 7fa3ee0f7640 1 Processor -- start 2024-04-17T10:38:52.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.955+0000 7fa3ee0f7640 1 -- start start 2024-04-17T10:38:52.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.955+0000 7fa3ee0f7640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa3e8161940 0x7fa3e806bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:52.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.955+0000 7fa3ee0f7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3e815ed70 0x7fa3e806c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:52.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.955+0000 7fa3ee0f7640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3e80d2430 con 0x7fa3e80cfb80 2024-04-17T10:38:52.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.955+0000 7fa3ee0f7640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3e80d2710 con 0x7fa3e8161940 2024-04-17T10:38:52.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.955+0000 7fa3ee0f7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e80ced00 con 0x7fa3e815ed70 2024-04-17T10:38:52.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.955+0000 7fa3e7fff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3e815ed70 0x7fa3e806c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:52.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.955+0000 7fa3e77fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa3e8161940 0x7fa3e806bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:52.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.955+0000 7fa3e7fff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3e815ed70 0x7fa3e806c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:49284/0 (socket says 172.21.15.83:49284) 2024-04-17T10:38:52.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.955+0000 7fa3e7fff640 1 -- 172.21.15.83:0/2378109209 learned_addr learned my addr 172.21.15.83:0/2378109209 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:52.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.955+0000 7fa3e6ffd640 1 --1- 172.21.15.83:0/2378109209 >> v1:172.21.15.167:6789/0 conn(0x7fa3e80cfb80 0x7fa3e8161d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:32842/0 (socket says 172.21.15.83:32842) 2024-04-17T10:38:52.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.955+0000 7fa3e77fe640 1 -- 172.21.15.83:0/2378109209 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3e815ed70 msgr2=0x7fa3e806c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:52.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.955+0000 7fa3e77fe640 1 --2- 172.21.15.83:0/2378109209 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3e815ed70 0x7fa3e806c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.955+0000 7fa3e77fe640 1 -- 172.21.15.83:0/2378109209 >> v1:172.21.15.167:6789/0 conn(0x7fa3e80cfb80 legacy=0x7fa3e8161d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:52.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.955+0000 7fa3e77fe640 1 -- 172.21.15.83:0/2378109209 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e80cefe0 con 0x7fa3e8161940 2024-04-17T10:38:52.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.959+0000 7fa3e77fe640 1 --2- 172.21.15.83:0/2378109209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3e8161940 0x7fa3e806bb80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa3d80097c0 tx=0x7fa3d801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6829a3c06ad68a3c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:52.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.959+0000 7fa3e67fc640 1 -- 172.21.15.83:0/2378109209 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d80b0070 con 0x7fa3e8161940 2024-04-17T10:38:52.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.959+0000 7fa3e67fc640 1 -- 172.21.15.83:0/2378109209 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d80ab600 con 0x7fa3e8161940 2024-04-17T10:38:52.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.959+0000 7fa3e67fc640 1 -- 172.21.15.83:0/2378109209 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d80ab9d0 con 0x7fa3e8161940 2024-04-17T10:38:52.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.959+0000 7fa3ee0f7640 1 -- 172.21.15.83:0/2378109209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3e8161940 msgr2=0x7fa3e806bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:52.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.959+0000 7fa3ee0f7640 1 --2- 172.21.15.83:0/2378109209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3e8161940 0x7fa3e806bb80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa3d80097c0 tx=0x7fa3d801c410 comp rx=0 tx=0).stop 2024-04-17T10:38:52.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.959+0000 7fa3ee0f7640 1 -- 172.21.15.83:0/2378109209 shutdown_connections 2024-04-17T10:38:52.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.959+0000 7fa3ee0f7640 1 --2- 172.21.15.83:0/2378109209 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3e815ed70 0x7fa3e806c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.959+0000 7fa3ee0f7640 1 --2- 172.21.15.83:0/2378109209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3e8161940 0x7fa3e806bb80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.959+0000 7fa3ee0f7640 1 -- 172.21.15.83:0/2378109209 >> 172.21.15.83:0/2378109209 conn(0x7fa3e80d17e0 msgr2=0x7fa3e80d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:52.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.959+0000 7fa3ee0f7640 1 -- 172.21.15.83:0/2378109209 shutdown_connections 2024-04-17T10:38:52.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.959+0000 7fa3ee0f7640 1 -- 172.21.15.83:0/2378109209 wait complete. 2024-04-17T10:38:52.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.963+0000 7fa3ee0f7640 1 Processor -- start 2024-04-17T10:38:52.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.963+0000 7fa3ee0f7640 1 -- start start 2024-04-17T10:38:52.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.963+0000 7fa3ee0f7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3e80cfb80 0x7fa3e80b3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:52.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.963+0000 7fa3ee0f7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e815ed70 0x7fa3e80b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:52.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.963+0000 7fa3ee0f7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3e8161940 0x7fa3e81522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:52.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.963+0000 7fa3ee0f7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e816ac30 con 0x7fa3e815ed70 2024-04-17T10:38:52.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.963+0000 7fa3ee0f7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e80d01a0 con 0x7fa3e80cfb80 2024-04-17T10:38:52.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.963+0000 7fa3ee0f7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e816adb0 con 0x7fa3e8161940 2024-04-17T10:38:52.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.963+0000 7fa3e6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e815ed70 0x7fa3e80b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:52.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.963+0000 7fa3e6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e815ed70 0x7fa3e80b1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51670/0 (socket says 172.21.15.83:51670) 2024-04-17T10:38:52.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.963+0000 7fa3e6ffd640 1 -- 172.21.15.83:0/3848684461 learned_addr learned my addr 172.21.15.83:0/3848684461 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:52.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.963+0000 7fa3e7fff640 1 --2- 172.21.15.83:0/3848684461 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3e8161940 0x7fa3e81522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:52.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.963+0000 7fa3e77fe640 1 --2- 172.21.15.83:0/3848684461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3e80cfb80 0x7fa3e80b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:52.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.963+0000 7fa3e6ffd640 1 -- 172.21.15.83:0/3848684461 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3e8161940 msgr2=0x7fa3e81522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:52.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.963+0000 7fa3e6ffd640 1 --2- 172.21.15.83:0/3848684461 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3e8161940 0x7fa3e81522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.963+0000 7fa3e6ffd640 1 -- 172.21.15.83:0/3848684461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3e80cfb80 msgr2=0x7fa3e80b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:52.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.963+0000 7fa3e6ffd640 1 --2- 172.21.15.83:0/3848684461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3e80cfb80 0x7fa3e80b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:52.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.963+0000 7fa3e6ffd640 1 -- 172.21.15.83:0/3848684461 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e80b23d0 con 0x7fa3e815ed70 2024-04-17T10:38:52.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.963+0000 7fa3e6ffd640 1 --2- 172.21.15.83:0/3848684461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e815ed70 0x7fa3e80b1b50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc00c450 tx=0x7fa3dc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:52.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.963+0000 7fa3cffff640 1 -- 172.21.15.83:0/3848684461 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc018070 con 0x7fa3e815ed70 2024-04-17T10:38:52.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.963+0000 7fa3cffff640 1 -- 172.21.15.83:0/3848684461 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3dc007480 con 0x7fa3e815ed70 2024-04-17T10:38:52.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.963+0000 7fa3ee0f7640 1 -- 172.21.15.83:0/3848684461 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3e8157680 con 0x7fa3e815ed70 2024-04-17T10:38:52.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.963+0000 7fa3cffff640 1 -- 172.21.15.83:0/3848684461 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc0047f0 con 0x7fa3e815ed70 2024-04-17T10:38:52.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.963+0000 7fa3ee0f7640 1 -- 172.21.15.83:0/3848684461 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3e80b2950 con 0x7fa3e815ed70 2024-04-17T10:38:52.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.963+0000 7fa3ee0f7640 1 -- 172.21.15.83:0/3848684461 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3e8074f90 con 0x7fa3e815ed70 2024-04-17T10:38:52.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.963+0000 7fa3cffff640 1 -- 172.21.15.83:0/3848684461 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa3dc01b070 con 0x7fa3e815ed70 2024-04-17T10:38:52.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.967+0000 7fa3cffff640 1 --2- 172.21.15.83:0/3848684461 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa3d4066f60 0x7fa3d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:52.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.967+0000 7fa3cffff640 1 -- 172.21.15.83:0/3848684461 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa3dc084200 con 0x7fa3e815ed70 2024-04-17T10:38:52.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.967+0000 7fa3e77fe640 1 --2- 172.21.15.83:0/3848684461 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa3d4066f60 0x7fa3d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:52.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.967+0000 7fa3e77fe640 1 --2- 172.21.15.83:0/3848684461 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa3d4066f60 0x7fa3d4069420 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa3d80ad000 tx=0x7fa3d80a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:52.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:52.971+0000 7fa3cffff640 1 -- 172.21.15.83:0/3848684461 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa3dc054b40 con 0x7fa3e815ed70 2024-04-17T10:38:53.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.139+0000 7fa3ee0f7640 1 -- 172.21.15.83:0/3848684461 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa3e8072740 con 0x7fa3e815ed70 2024-04-17T10:38:53.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.139+0000 7fa3cffff640 1 -- 172.21.15.83:0/3848684461 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa3dc058be0 con 0x7fa3e815ed70 2024-04-17T10:38:53.137 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-17T10:38:53.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.143+0000 7fa3ee0f7640 1 -- 172.21.15.83:0/3848684461 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa3d4066f60 msgr2=0x7fa3d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:53.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.143+0000 7fa3ee0f7640 1 --2- 172.21.15.83:0/3848684461 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa3d4066f60 0x7fa3d4069420 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa3d80ad000 tx=0x7fa3d80a6000 comp rx=0 tx=0).stop 2024-04-17T10:38:53.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.143+0000 7fa3ee0f7640 1 -- 172.21.15.83:0/3848684461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e815ed70 msgr2=0x7fa3e80b1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:53.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.143+0000 7fa3ee0f7640 1 --2- 172.21.15.83:0/3848684461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e815ed70 0x7fa3e80b1b50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc00c450 tx=0x7fa3dc00c910 comp rx=0 tx=0).stop 2024-04-17T10:38:53.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.143+0000 7fa3ee0f7640 1 -- 172.21.15.83:0/3848684461 shutdown_connections 2024-04-17T10:38:53.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.143+0000 7fa3ee0f7640 1 --2- 172.21.15.83:0/3848684461 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa3d4066f60 0x7fa3d4069420 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:53.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.143+0000 7fa3ee0f7640 1 --2- 172.21.15.83:0/3848684461 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3e8161940 0x7fa3e81522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:53.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.143+0000 7fa3ee0f7640 1 --2- 172.21.15.83:0/3848684461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e815ed70 0x7fa3e80b1b50 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:53.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.143+0000 7fa3ee0f7640 1 --2- 172.21.15.83:0/3848684461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3e80cfb80 0x7fa3e80b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:53.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.143+0000 7fa3ee0f7640 1 -- 172.21.15.83:0/3848684461 >> 172.21.15.83:0/3848684461 conn(0x7fa3e80d17e0 msgr2=0x7fa3e80d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:53.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.143+0000 7fa3ee0f7640 1 -- 172.21.15.83:0/3848684461 shutdown_connections 2024-04-17T10:38:53.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.143+0000 7fa3ee0f7640 1 -- 172.21.15.83:0/3848684461 wait complete. 2024-04-17T10:38:53.156 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi167.front.sepia.ceph.com... 2024-04-17T10:38:53.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:53.156 DEBUG:teuthology.orchestra.run.smithi167:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-17T10:38:53.189 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:38:53.189 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-04-17T10:38:53.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:53.189 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-17T10:38:53.244 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:38:53.244 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:38:53.250 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:38:53.250 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:38:53.300 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:38:53.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.471+0000 7f18b3ca5640 1 Processor -- start 2024-04-17T10:38:53.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.471+0000 7f18b3ca5640 1 -- start start 2024-04-17T10:38:53.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.475+0000 7f18b3ca5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18ac0b3990 0x7f18ac070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:53.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.475+0000 7f18b3ca5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f18ac0709b0 0x7f18ac06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:53.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.475+0000 7f18b3ca5640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18ac0b2bd0 con 0x7f18ac0cf060 2024-04-17T10:38:53.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.475+0000 7f18b3ca5640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18ac070e20 con 0x7f18ac0709b0 2024-04-17T10:38:53.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.475+0000 7f18b3ca5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac0b2980 con 0x7f18ac0b3990 2024-04-17T10:38:53.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.475+0000 7f18b1240640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18ac0b3990 0x7f18ac070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:53.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.475+0000 7f18b2242640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f18ac0709b0 0x7f18ac06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:53.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.475+0000 7f18b1240640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18ac0b3990 0x7f18ac070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:49296/0 (socket says 172.21.15.83:49296) 2024-04-17T10:38:53.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.475+0000 7f18b2242640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f18ac0709b0 0x7f18ac06c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:58978/0 (socket says 172.21.15.83:58978) 2024-04-17T10:38:53.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.475+0000 7f18b1240640 1 -- 172.21.15.83:0/1851075245 learned_addr learned my addr 172.21.15.83:0/1851075245 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:53.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.475+0000 7f18b2242640 1 -- 172.21.15.83:0/1851075245 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18ac0b3990 msgr2=0x7f18ac070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:53.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.475+0000 7f18b2242640 1 --2- 172.21.15.83:0/1851075245 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18ac0b3990 0x7f18ac070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:53.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.475+0000 7f18b2242640 1 -- 172.21.15.83:0/1851075245 >> v1:172.21.15.167:6789/0 conn(0x7f18ac0cf060 legacy=0x7f18ac0b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:53.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.475+0000 7f18b2242640 1 -- 172.21.15.83:0/1851075245 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ac0b2eb0 con 0x7f18ac0709b0 2024-04-17T10:38:53.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.479+0000 7f18b2242640 1 --2- 172.21.15.83:0/1851075245 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18ac0709b0 0x7f18ac06c490 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f18a40097c0 tx=0x7f18a401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73d5f34c775a38ad server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:53.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.479+0000 7f18b0a3f640 1 -- 172.21.15.83:0/1851075245 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a40a7070 con 0x7f18ac0709b0 2024-04-17T10:38:53.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.479+0000 7f18b0a3f640 1 -- 172.21.15.83:0/1851075245 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a40b0070 con 0x7f18ac0709b0 2024-04-17T10:38:53.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.479+0000 7f18b0a3f640 1 -- 172.21.15.83:0/1851075245 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a40ab890 con 0x7f18ac0709b0 2024-04-17T10:38:53.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.479+0000 7f18b3ca5640 1 -- 172.21.15.83:0/1851075245 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18ac0709b0 msgr2=0x7f18ac06c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:53.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.479+0000 7f18b3ca5640 1 --2- 172.21.15.83:0/1851075245 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18ac0709b0 0x7f18ac06c490 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f18a40097c0 tx=0x7f18a401c470 comp rx=0 tx=0).stop 2024-04-17T10:38:53.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.479+0000 7f18b3ca5640 1 -- 172.21.15.83:0/1851075245 shutdown_connections 2024-04-17T10:38:53.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.479+0000 7f18b3ca5640 1 --2- 172.21.15.83:0/1851075245 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18ac0709b0 0x7f18ac06c490 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:53.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.479+0000 7f18b3ca5640 1 --2- 172.21.15.83:0/1851075245 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18ac0b3990 0x7f18ac070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:53.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.479+0000 7f18b3ca5640 1 -- 172.21.15.83:0/1851075245 >> 172.21.15.83:0/1851075245 conn(0x7f18ac06abe0 msgr2=0x7f18ac06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:53.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.479+0000 7f18b3ca5640 1 -- 172.21.15.83:0/1851075245 shutdown_connections 2024-04-17T10:38:53.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.479+0000 7f18b3ca5640 1 -- 172.21.15.83:0/1851075245 wait complete. 2024-04-17T10:38:53.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.479+0000 7f18b3ca5640 1 Processor -- start 2024-04-17T10:38:53.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.479+0000 7f18b3ca5640 1 -- start start 2024-04-17T10:38:53.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.479+0000 7f18b3ca5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18ac0709b0 0x7f18ac1574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:53.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.479+0000 7f18b3ca5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18ac0b3990 0x7f18ac143130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:53.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.479+0000 7f18b3ca5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18ac0cf060 0x7f18ac15d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:53.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.479+0000 7f18b3ca5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac06c9d0 con 0x7f18ac0cf060 2024-04-17T10:38:53.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.479+0000 7f18b3ca5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac0b4770 con 0x7f18ac0709b0 2024-04-17T10:38:53.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.479+0000 7f18b3ca5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac06cb50 con 0x7f18ac0b3990 2024-04-17T10:38:53.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.479+0000 7f18b2242640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18ac0cf060 0x7f18ac15d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:53.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.479+0000 7f18b2242640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18ac0cf060 0x7f18ac15d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51684/0 (socket says 172.21.15.83:51684) 2024-04-17T10:38:53.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.479+0000 7f18b2242640 1 -- 172.21.15.83:0/3518993632 learned_addr learned my addr 172.21.15.83:0/3518993632 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:53.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.483+0000 7f18b1240640 1 --2- 172.21.15.83:0/3518993632 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18ac0b3990 0x7f18ac143130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:53.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.483+0000 7f18b1a41640 1 --2- 172.21.15.83:0/3518993632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18ac0709b0 0x7f18ac1574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:53.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.483+0000 7f18b2242640 1 -- 172.21.15.83:0/3518993632 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18ac0b3990 msgr2=0x7f18ac143130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:53.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.483+0000 7f18b2242640 1 --2- 172.21.15.83:0/3518993632 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18ac0b3990 0x7f18ac143130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:53.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.483+0000 7f18b2242640 1 -- 172.21.15.83:0/3518993632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18ac0709b0 msgr2=0x7f18ac1574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:53.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.483+0000 7f18b2242640 1 --2- 172.21.15.83:0/3518993632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18ac0709b0 0x7f18ac1574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:53.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.483+0000 7f18b2242640 1 -- 172.21.15.83:0/3518993632 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ac1444b0 con 0x7f18ac0cf060 2024-04-17T10:38:53.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.483+0000 7f18b2242640 1 --2- 172.21.15.83:0/3518993632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18ac0cf060 0x7f18ac15d7d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f18a40ad040 tx=0x7f18a401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:53.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.483+0000 7f18aa7fc640 1 -- 172.21.15.83:0/3518993632 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a40abe60 con 0x7f18ac0cf060 2024-04-17T10:38:53.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.483+0000 7f18aa7fc640 1 -- 172.21.15.83:0/3518993632 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a40bf610 con 0x7f18ac0cf060 2024-04-17T10:38:53.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.483+0000 7f18aa7fc640 1 -- 172.21.15.83:0/3518993632 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a40bf9e0 con 0x7f18ac0cf060 2024-04-17T10:38:53.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.483+0000 7f18b3ca5640 1 -- 172.21.15.83:0/3518993632 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18ac156800 con 0x7f18ac0cf060 2024-04-17T10:38:53.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.483+0000 7f18b3ca5640 1 -- 172.21.15.83:0/3518993632 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18ac156ae0 con 0x7f18ac0cf060 2024-04-17T10:38:53.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.483+0000 7f18b3ca5640 1 -- 172.21.15.83:0/3518993632 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18ac074f90 con 0x7f18ac0cf060 2024-04-17T10:38:53.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.483+0000 7f18aa7fc640 1 -- 172.21.15.83:0/3518993632 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f18a40bd020 con 0x7f18ac0cf060 2024-04-17T10:38:53.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.483+0000 7f18aa7fc640 1 --2- 172.21.15.83:0/3518993632 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1898066f60 0x7f1898069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:53.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.483+0000 7f18aa7fc640 1 -- 172.21.15.83:0/3518993632 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f18a411f310 con 0x7f18ac0cf060 2024-04-17T10:38:53.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.483+0000 7f18b1a41640 1 --2- 172.21.15.83:0/3518993632 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1898066f60 0x7f1898069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:53.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.483+0000 7f18b1a41640 1 --2- 172.21.15.83:0/3518993632 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1898066f60 0x7f1898069420 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f18ac06c980 tx=0x7f189c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:53.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.487+0000 7f18aa7fc640 1 -- 172.21.15.83:0/3518993632 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f18a40c6030 con 0x7f18ac0cf060 2024-04-17T10:38:53.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.643+0000 7f18b3ca5640 1 -- 172.21.15.83:0/3518993632 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18ac159f90 con 0x7f18ac0cf060 2024-04-17T10:38:53.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.647+0000 7f18aa7fc640 1 -- 172.21.15.83:0/3518993632 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16782 (secure 0 0 0) 0x7f18a4016090 con 0x7f18ac0cf060 2024-04-17T10:38:53.645 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:53.646 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":22,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:38:53.385002+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:38:10.464490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T10:38:11.043134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:38:53.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.651+0000 7f18b3ca5640 1 -- 172.21.15.83:0/3518993632 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1898066f60 msgr2=0x7f1898069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:53.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.651+0000 7f18b3ca5640 1 --2- 172.21.15.83:0/3518993632 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1898066f60 0x7f1898069420 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f18ac06c980 tx=0x7f189c009040 comp rx=0 tx=0).stop 2024-04-17T10:38:53.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.651+0000 7f18b3ca5640 1 -- 172.21.15.83:0/3518993632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18ac0cf060 msgr2=0x7f18ac15d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:53.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.651+0000 7f18b3ca5640 1 --2- 172.21.15.83:0/3518993632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18ac0cf060 0x7f18ac15d7d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f18a40ad040 tx=0x7f18a401acb0 comp rx=0 tx=0).stop 2024-04-17T10:38:53.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.651+0000 7f18b3ca5640 1 -- 172.21.15.83:0/3518993632 shutdown_connections 2024-04-17T10:38:53.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.651+0000 7f18b3ca5640 1 --2- 172.21.15.83:0/3518993632 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1898066f60 0x7f1898069420 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:53.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.651+0000 7f18b3ca5640 1 --2- 172.21.15.83:0/3518993632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18ac0cf060 0x7f18ac15d7d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:53.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.651+0000 7f18b3ca5640 1 --2- 172.21.15.83:0/3518993632 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18ac0b3990 0x7f18ac143130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:53.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.651+0000 7f18b3ca5640 1 --2- 172.21.15.83:0/3518993632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18ac0709b0 0x7f18ac1574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:53.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.651+0000 7f18b3ca5640 1 -- 172.21.15.83:0/3518993632 >> 172.21.15.83:0/3518993632 conn(0x7f18ac06abe0 msgr2=0x7f18ac15b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:53.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.651+0000 7f18b3ca5640 1 -- 172.21.15.83:0/3518993632 shutdown_connections 2024-04-17T10:38:53.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.651+0000 7f18b3ca5640 1 -- 172.21.15.83:0/3518993632 wait complete. 2024-04-17T10:38:53.663 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:38:10.464490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:38:11.043134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:38:53.664 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:38:53.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.787+0000 7f4f6ffee640 1 Processor -- start 2024-04-17T10:38:53.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.791+0000 7f4f6ffee640 1 -- start start 2024-04-17T10:38:53.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.791+0000 7f4f6ffee640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4f68154de0 0x7f4f6806ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:53.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.791+0000 7f4f6ffee640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4f6806b350 0x7f4f680b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:53.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.791+0000 7f4f6ffee640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4f680b4040 0x7f4f68071580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:53.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.791+0000 7f4f6ffee640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f680b4870 con 0x7f4f6806b350 2024-04-17T10:38:53.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.791+0000 7f4f6ffee640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f6815bd70 con 0x7f4f680b4040 2024-04-17T10:38:53.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.791+0000 7f4f6ffee640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6815bf60 con 0x7f4f68154de0 2024-04-17T10:38:53.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.791+0000 7f4f6d589640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4f6806b350 0x7f4f680b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:53.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.791+0000 7f4f6d589640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4f6806b350 0x7f4f680b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51696/0 (socket says 172.21.15.83:51696) 2024-04-17T10:38:53.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.791+0000 7f4f6d589640 1 -- 172.21.15.83:0/2284048962 learned_addr learned my addr 172.21.15.83:0/2284048962 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:53.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.791+0000 7f4f6dd8a640 1 --2- 172.21.15.83:0/2284048962 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4f68154de0 0x7f4f6806ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:53.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.791+0000 7f4f6e58b640 1 --2- 172.21.15.83:0/2284048962 >> v2:172.21.15.167:3300/0 conn(0x7f4f680b4040 0x7f4f68071580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:53.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.791+0000 7f4f6d589640 1 -- 172.21.15.83:0/2284048962 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4f68154de0 msgr2=0x7f4f6806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:53.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.791+0000 7f4f6d589640 1 --2- 172.21.15.83:0/2284048962 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4f68154de0 0x7f4f6806ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:53.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.791+0000 7f4f6d589640 1 -- 172.21.15.83:0/2284048962 >> v2:172.21.15.167:3300/0 conn(0x7f4f680b4040 msgr2=0x7f4f68071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:53.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.791+0000 7f4f6d589640 1 --2- 172.21.15.83:0/2284048962 >> v2:172.21.15.167:3300/0 conn(0x7f4f680b4040 0x7f4f68071580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:53.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.791+0000 7f4f6d589640 1 -- 172.21.15.83:0/2284048962 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6815afb0 con 0x7f4f6806b350 2024-04-17T10:38:53.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.795+0000 7f4f6d589640 1 --2- 172.21.15.83:0/2284048962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f6806b350 0x7f4f680b3b00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4f6806e900 tx=0x7f4f5c01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5ad14f555e8c05 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:53.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.795+0000 7f4f6cd88640 1 -- 172.21.15.83:0/2284048962 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c0b0070 con 0x7f4f6806b350 2024-04-17T10:38:53.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.795+0000 7f4f6cd88640 1 -- 172.21.15.83:0/2284048962 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5c0ab600 con 0x7f4f6806b350 2024-04-17T10:38:53.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.795+0000 7f4f6cd88640 1 -- 172.21.15.83:0/2284048962 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c0ab9d0 con 0x7f4f6806b350 2024-04-17T10:38:53.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.795+0000 7f4f6ffee640 1 -- 172.21.15.83:0/2284048962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f6806b350 msgr2=0x7f4f680b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:53.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.795+0000 7f4f6ffee640 1 --2- 172.21.15.83:0/2284048962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f6806b350 0x7f4f680b3b00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4f6806e900 tx=0x7f4f5c01c260 comp rx=0 tx=0).stop 2024-04-17T10:38:53.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.795+0000 7f4f6ffee640 1 -- 172.21.15.83:0/2284048962 shutdown_connections 2024-04-17T10:38:53.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.795+0000 7f4f6ffee640 1 --2- 172.21.15.83:0/2284048962 >> v2:172.21.15.167:3300/0 conn(0x7f4f680b4040 0x7f4f68071580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:53.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.795+0000 7f4f6ffee640 1 --2- 172.21.15.83:0/2284048962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f6806b350 0x7f4f680b3b00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:53.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.795+0000 7f4f6ffee640 1 --2- 172.21.15.83:0/2284048962 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4f68154de0 0x7f4f6806ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:53.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.795+0000 7f4f6ffee640 1 -- 172.21.15.83:0/2284048962 >> 172.21.15.83:0/2284048962 conn(0x7f4f680b15b0 msgr2=0x7f4f680b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:53.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.795+0000 7f4f6ffee640 1 -- 172.21.15.83:0/2284048962 shutdown_connections 2024-04-17T10:38:53.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.795+0000 7f4f6ffee640 1 -- 172.21.15.83:0/2284048962 wait complete. 2024-04-17T10:38:53.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.795+0000 7f4f6ffee640 1 Processor -- start 2024-04-17T10:38:53.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.795+0000 7f4f6ffee640 1 -- start start 2024-04-17T10:38:53.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.795+0000 7f4f6ffee640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4f6806b350 0x7f4f6814d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:53.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.795+0000 7f4f6ffee640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f680b4040 0x7f4f68151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:53.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.795+0000 7f4f6ffee640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f68154de0 0x7f4f6814e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:53.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.795+0000 7f4f6ffee640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f680b4450 con 0x7f4f680b4040 2024-04-17T10:38:53.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.795+0000 7f4f6ffee640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f681551c0 con 0x7f4f68154de0 2024-04-17T10:38:53.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.795+0000 7f4f6ffee640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f680b45d0 con 0x7f4f6806b350 2024-04-17T10:38:53.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.795+0000 7f4f6d589640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f680b4040 0x7f4f68151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:53.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.795+0000 7f4f6d589640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f680b4040 0x7f4f68151bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51710/0 (socket says 172.21.15.83:51710) 2024-04-17T10:38:53.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.795+0000 7f4f6d589640 1 -- 172.21.15.83:0/1233568805 learned_addr learned my addr 172.21.15.83:0/1233568805 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:53.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.795+0000 7f4f6dd8a640 1 --2- 172.21.15.83:0/1233568805 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4f6806b350 0x7f4f6814d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:53.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.795+0000 7f4f6d589640 1 -- 172.21.15.83:0/1233568805 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4f6806b350 msgr2=0x7f4f6814d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:53.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.795+0000 7f4f6e58b640 1 --2- 172.21.15.83:0/1233568805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f68154de0 0x7f4f6814e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:53.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.795+0000 7f4f6d589640 1 --2- 172.21.15.83:0/1233568805 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4f6806b350 0x7f4f6814d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:53.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.795+0000 7f4f6d589640 1 -- 172.21.15.83:0/1233568805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f68154de0 msgr2=0x7f4f6814e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:53.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.795+0000 7f4f6d589640 1 --2- 172.21.15.83:0/1233568805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f68154de0 0x7f4f6814e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:53.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.795+0000 7f4f6d589640 1 -- 172.21.15.83:0/1233568805 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f68152500 con 0x7f4f680b4040 2024-04-17T10:38:53.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.799+0000 7f4f6d589640 1 --2- 172.21.15.83:0/1233568805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f680b4040 0x7f4f68151bb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c01a910 tx=0x7f4f5c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:53.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.799+0000 7f4f667fc640 1 -- 172.21.15.83:0/1233568805 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c0b0040 con 0x7f4f680b4040 2024-04-17T10:38:53.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.799+0000 7f4f667fc640 1 -- 172.21.15.83:0/1233568805 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5c0b4620 con 0x7f4f680b4040 2024-04-17T10:38:53.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.799+0000 7f4f6ffee640 1 -- 172.21.15.83:0/1233568805 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f681527e0 con 0x7f4f680b4040 2024-04-17T10:38:53.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.799+0000 7f4f667fc640 1 -- 172.21.15.83:0/1233568805 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c0b4a10 con 0x7f4f680b4040 2024-04-17T10:38:53.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.799+0000 7f4f6ffee640 1 -- 172.21.15.83:0/1233568805 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f68152da0 con 0x7f4f680b4040 2024-04-17T10:38:53.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.799+0000 7f4f6ffee640 1 -- 172.21.15.83:0/1233568805 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f68074f90 con 0x7f4f680b4040 2024-04-17T10:38:53.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.799+0000 7f4f667fc640 1 -- 172.21.15.83:0/1233568805 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4f5c0ab4c0 con 0x7f4f680b4040 2024-04-17T10:38:53.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.799+0000 7f4f667fc640 1 --2- 172.21.15.83:0/1233568805 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4f54066f90 0x7f4f54069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:53.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.799+0000 7f4f667fc640 1 -- 172.21.15.83:0/1233568805 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f4f5c11ec60 con 0x7f4f680b4040 2024-04-17T10:38:53.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.799+0000 7f4f6dd8a640 1 --2- 172.21.15.83:0/1233568805 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4f54066f90 0x7f4f54069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:53.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.799+0000 7f4f6dd8a640 1 --2- 172.21.15.83:0/1233568805 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4f54066f90 0x7f4f54069450 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4f60009710 tx=0x7f4f60008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:53.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.803+0000 7f4f667fc640 1 -- 172.21.15.83:0/1233568805 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4f5c016110 con 0x7f4f680b4040 2024-04-17T10:38:53.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.963+0000 7f4f6ffee640 1 -- 172.21.15.83:0/1233568805 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4f680b3280 con 0x7f4f680b4040 2024-04-17T10:38:53.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.963+0000 7f4f667fc640 1 -- 172.21.15.83:0/1233568805 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4f5c0efc60 con 0x7f4f680b4040 2024-04-17T10:38:53.961 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:38:53.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.967+0000 7f4f6ffee640 1 -- 172.21.15.83:0/1233568805 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4f54066f90 msgr2=0x7f4f54069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:53.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.967+0000 7f4f6ffee640 1 --2- 172.21.15.83:0/1233568805 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4f54066f90 0x7f4f54069450 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4f60009710 tx=0x7f4f60008040 comp rx=0 tx=0).stop 2024-04-17T10:38:53.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.967+0000 7f4f6ffee640 1 -- 172.21.15.83:0/1233568805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f680b4040 msgr2=0x7f4f68151bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:53.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.967+0000 7f4f6ffee640 1 --2- 172.21.15.83:0/1233568805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f680b4040 0x7f4f68151bb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c01a910 tx=0x7f4f5c019000 comp rx=0 tx=0).stop 2024-04-17T10:38:53.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.967+0000 7f4f6ffee640 1 -- 172.21.15.83:0/1233568805 shutdown_connections 2024-04-17T10:38:53.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.967+0000 7f4f6ffee640 1 --2- 172.21.15.83:0/1233568805 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4f54066f90 0x7f4f54069450 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:53.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.967+0000 7f4f6ffee640 1 --2- 172.21.15.83:0/1233568805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f68154de0 0x7f4f6814e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:53.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.967+0000 7f4f6ffee640 1 --2- 172.21.15.83:0/1233568805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f680b4040 0x7f4f68151bb0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:53.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.967+0000 7f4f6ffee640 1 --2- 172.21.15.83:0/1233568805 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4f6806b350 0x7f4f6814d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:53.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.967+0000 7f4f6ffee640 1 -- 172.21.15.83:0/1233568805 >> 172.21.15.83:0/1233568805 conn(0x7f4f680b15b0 msgr2=0x7f4f680cfd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:53.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.967+0000 7f4f6ffee640 1 -- 172.21.15.83:0/1233568805 shutdown_connections 2024-04-17T10:38:53.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:53.967+0000 7f4f6ffee640 1 -- 172.21.15.83:0/1233568805 wait complete. 2024-04-17T10:38:53.980 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:38:54.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.099+0000 7fc1d808a640 1 Processor -- start 2024-04-17T10:38:54.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.099+0000 7fc1d808a640 1 -- start start 2024-04-17T10:38:54.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.103+0000 7fc1d808a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1d015bac0 0x7fc1d0071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:54.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.103+0000 7fc1d808a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc1d0070470 0x7fc1d006ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:54.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.103+0000 7fc1d808a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1d006c1f0 con 0x7fc1d015b6e0 2024-04-17T10:38:54.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.103+0000 7fc1d808a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1d0070a10 con 0x7fc1d0070470 2024-04-17T10:38:54.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.103+0000 7fc1d808a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d0070cf0 con 0x7fc1d015bac0 2024-04-17T10:38:54.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.103+0000 7fc1d6627640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc1d0070470 0x7fc1d006ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:54.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.103+0000 7fc1d5e26640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc1d015b6e0 0x7fc1d00cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40094/0 (socket says 172.21.15.83:40094) 2024-04-17T10:38:54.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.103+0000 7fc1d5e26640 1 -- 172.21.15.83:0/1332558249 learned_addr learned my addr 172.21.15.83:0/1332558249 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:54.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.103+0000 7fc1d5625640 1 --2- 172.21.15.83:0/1332558249 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1d015bac0 0x7fc1d0071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:54.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.103+0000 7fc1d6627640 1 -- 172.21.15.83:0/1332558249 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1d015bac0 msgr2=0x7fc1d0071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:54.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.103+0000 7fc1d6627640 1 --2- 172.21.15.83:0/1332558249 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1d015bac0 0x7fc1d0071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:54.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.103+0000 7fc1d6627640 1 -- 172.21.15.83:0/1332558249 >> v1:172.21.15.83:6789/0 conn(0x7fc1d015b6e0 legacy=0x7fc1d00cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:54.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.103+0000 7fc1d6627640 1 -- 172.21.15.83:0/1332558249 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d0070fd0 con 0x7fc1d0070470 2024-04-17T10:38:54.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.103+0000 7fc1d4e24640 1 -- 172.21.15.83:0/1332558249 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1608574254 0 0) 0x7fc1d006c1f0 con 0x7fc1d015b6e0 2024-04-17T10:38:54.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.103+0000 7fc1d6627640 1 --2- 172.21.15.83:0/1332558249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1d0070470 0x7fc1d006ae10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8009d90 tx=0x7fc1c801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0d4b8a07838d4b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:54.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.103+0000 7fc1d4e24640 1 -- 172.21.15.83:0/1332558249 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c80b2070 con 0x7fc1d0070470 2024-04-17T10:38:54.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.103+0000 7fc1d4e24640 1 -- 172.21.15.83:0/1332558249 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1c80a5830 con 0x7fc1d0070470 2024-04-17T10:38:54.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.103+0000 7fc1d4e24640 1 -- 172.21.15.83:0/1332558249 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c80a5c00 con 0x7fc1d0070470 2024-04-17T10:38:54.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1d808a640 1 -- 172.21.15.83:0/1332558249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1d0070470 msgr2=0x7fc1d006ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:54.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1d808a640 1 --2- 172.21.15.83:0/1332558249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1d0070470 0x7fc1d006ae10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8009d90 tx=0x7fc1c801c970 comp rx=0 tx=0).stop 2024-04-17T10:38:54.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1d808a640 1 -- 172.21.15.83:0/1332558249 shutdown_connections 2024-04-17T10:38:54.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1d808a640 1 --2- 172.21.15.83:0/1332558249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1d0070470 0x7fc1d006ae10 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:54.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1d808a640 1 --2- 172.21.15.83:0/1332558249 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1d015bac0 0x7fc1d0071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:54.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1d808a640 1 -- 172.21.15.83:0/1332558249 >> 172.21.15.83:0/1332558249 conn(0x7fc1d00b15b0 msgr2=0x7fc1d00b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:54.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1d808a640 1 -- 172.21.15.83:0/1332558249 shutdown_connections 2024-04-17T10:38:54.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1d808a640 1 -- 172.21.15.83:0/1332558249 wait complete. 2024-04-17T10:38:54.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1d808a640 1 Processor -- start 2024-04-17T10:38:54.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1d808a640 1 -- start start 2024-04-17T10:38:54.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1d808a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1d0070470 0x7fc1d00adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:54.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1d808a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1d015b6e0 0x7fc1d015d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:54.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1d808a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1d015bac0 0x7fc1d014fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:54.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1d808a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d006d270 con 0x7fc1d015b6e0 2024-04-17T10:38:54.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1d808a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d006d0f0 con 0x7fc1d0070470 2024-04-17T10:38:54.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1d808a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d006b760 con 0x7fc1d015bac0 2024-04-17T10:38:54.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1d5625640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1d015b6e0 0x7fc1d015d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:54.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1d5625640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1d015b6e0 0x7fc1d015d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51728/0 (socket says 172.21.15.83:51728) 2024-04-17T10:38:54.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1d5625640 1 -- 172.21.15.83:0/269262117 learned_addr learned my addr 172.21.15.83:0/269262117 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:54.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1d5e26640 1 --2- 172.21.15.83:0/269262117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1d0070470 0x7fc1d00adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:54.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1d6627640 1 --2- 172.21.15.83:0/269262117 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1d015bac0 0x7fc1d014fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:54.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1d5625640 1 -- 172.21.15.83:0/269262117 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1d015bac0 msgr2=0x7fc1d014fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:54.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1d5625640 1 --2- 172.21.15.83:0/269262117 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1d015bac0 0x7fc1d014fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:54.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1d5625640 1 -- 172.21.15.83:0/269262117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1d0070470 msgr2=0x7fc1d00adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:54.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1d5625640 1 --2- 172.21.15.83:0/269262117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1d0070470 0x7fc1d00adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:54.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1d5625640 1 -- 172.21.15.83:0/269262117 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d015c3a0 con 0x7fc1d015b6e0 2024-04-17T10:38:54.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1d5e26640 1 --2- 172.21.15.83:0/269262117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1d0070470 0x7fc1d00adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:54.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1d5625640 1 --2- 172.21.15.83:0/269262117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1d015b6e0 0x7fc1d015d3b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc1c000d5c0 tx=0x7fc1c000da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:54.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1c67fc640 1 -- 172.21.15.83:0/269262117 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c00125f0 con 0x7fc1d015b6e0 2024-04-17T10:38:54.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1c67fc640 1 -- 172.21.15.83:0/269262117 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1c0019070 con 0x7fc1d015b6e0 2024-04-17T10:38:54.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1c67fc640 1 -- 172.21.15.83:0/269262117 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c00129d0 con 0x7fc1d015b6e0 2024-04-17T10:38:54.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1d808a640 1 -- 172.21.15.83:0/269262117 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1d015c680 con 0x7fc1d015b6e0 2024-04-17T10:38:54.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.107+0000 7fc1d808a640 1 -- 172.21.15.83:0/269262117 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1d015cc40 con 0x7fc1d015b6e0 2024-04-17T10:38:54.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.111+0000 7fc1d808a640 1 -- 172.21.15.83:0/269262117 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1d0074f90 con 0x7fc1d015b6e0 2024-04-17T10:38:54.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.111+0000 7fc1c67fc640 1 -- 172.21.15.83:0/269262117 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc1c0012b70 con 0x7fc1d015b6e0 2024-04-17T10:38:54.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.111+0000 7fc1c67fc640 1 --2- 172.21.15.83:0/269262117 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc1b4066f60 0x7fc1b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:54.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.111+0000 7fc1c67fc640 1 -- 172.21.15.83:0/269262117 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fc1c0085220 con 0x7fc1d015b6e0 2024-04-17T10:38:54.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.111+0000 7fc1d5e26640 1 --2- 172.21.15.83:0/269262117 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc1b4066f60 0x7fc1b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:54.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.111+0000 7fc1d5e26640 1 --2- 172.21.15.83:0/269262117 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc1b4066f60 0x7fc1b4069420 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc000f70 tx=0x7fc1cc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:54.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.115+0000 7fc1c67fc640 1 -- 172.21.15.83:0/269262117 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc1c0023020 con 0x7fc1d015b6e0 2024-04-17T10:38:54.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.263+0000 7fc1d808a640 1 -- 172.21.15.83:0/269262117 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc1d0071690 con 0x7fc1d015b6e0 2024-04-17T10:38:54.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.267+0000 7fc1c67fc640 1 -- 172.21.15.83:0/269262117 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc1c0056240 con 0x7fc1d015b6e0 2024-04-17T10:38:54.263 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:38:54.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.267+0000 7fc1d808a640 1 -- 172.21.15.83:0/269262117 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc1b4066f60 msgr2=0x7fc1b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:54.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.267+0000 7fc1d808a640 1 --2- 172.21.15.83:0/269262117 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc1b4066f60 0x7fc1b4069420 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc000f70 tx=0x7fc1cc009040 comp rx=0 tx=0).stop 2024-04-17T10:38:54.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.267+0000 7fc1d808a640 1 -- 172.21.15.83:0/269262117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1d015b6e0 msgr2=0x7fc1d015d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:54.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.267+0000 7fc1d808a640 1 --2- 172.21.15.83:0/269262117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1d015b6e0 0x7fc1d015d3b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc1c000d5c0 tx=0x7fc1c000da80 comp rx=0 tx=0).stop 2024-04-17T10:38:54.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.267+0000 7fc1d808a640 1 -- 172.21.15.83:0/269262117 shutdown_connections 2024-04-17T10:38:54.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.267+0000 7fc1d808a640 1 --2- 172.21.15.83:0/269262117 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc1b4066f60 0x7fc1b4069420 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:54.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.267+0000 7fc1d808a640 1 --2- 172.21.15.83:0/269262117 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1d015bac0 0x7fc1d014fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:54.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.267+0000 7fc1d808a640 1 --2- 172.21.15.83:0/269262117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1d015b6e0 0x7fc1d015d3b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:54.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.267+0000 7fc1d808a640 1 --2- 172.21.15.83:0/269262117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1d0070470 0x7fc1d00adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:54.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.267+0000 7fc1d808a640 1 -- 172.21.15.83:0/269262117 >> 172.21.15.83:0/269262117 conn(0x7fc1d00b15b0 msgr2=0x7fc1d00cdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:54.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.267+0000 7fc1d808a640 1 -- 172.21.15.83:0/269262117 shutdown_connections 2024-04-17T10:38:54.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.267+0000 7fc1d808a640 1 -- 172.21.15.83:0/269262117 wait complete. 2024-04-17T10:38:54.281 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:38:54.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.395+0000 7ff42a9fd640 1 Processor -- start 2024-04-17T10:38:54.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.395+0000 7ff42a9fd640 1 -- start start 2024-04-17T10:38:54.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.395+0000 7ff42a9fd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff4240d0970 0x7ff4240d17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:54.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.395+0000 7ff42a9fd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff4240d26c0 0x7ff42415d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:54.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.395+0000 7ff42a9fd640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4240ce940 con 0x7ff4240ce4e0 2024-04-17T10:38:54.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.395+0000 7ff42a9fd640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4240cf7d0 con 0x7ff4240d26c0 2024-04-17T10:38:54.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.395+0000 7ff42a9fd640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4240d4890 con 0x7ff4240d0970 2024-04-17T10:38:54.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.395+0000 7ff428f9a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff4240d26c0 0x7ff42415d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:54.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.395+0000 7ff428f9a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff4240d26c0 0x7ff42415d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51736/0 (socket says 172.21.15.83:51736) 2024-04-17T10:38:54.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.395+0000 7ff428f9a640 1 -- 172.21.15.83:0/4150302862 learned_addr learned my addr 172.21.15.83:0/4150302862 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:54.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.395+0000 7ff423fff640 1 --2- 172.21.15.83:0/4150302862 >> v2:172.21.15.167:3300/0 conn(0x7ff4240d0970 0x7ff4240d17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:54.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.395+0000 7ff428f9a640 1 -- 172.21.15.83:0/4150302862 >> v2:172.21.15.167:3300/0 conn(0x7ff4240d0970 msgr2=0x7ff4240d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:54.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.395+0000 7ff428f9a640 1 --2- 172.21.15.83:0/4150302862 >> v2:172.21.15.167:3300/0 conn(0x7ff4240d0970 0x7ff4240d17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:54.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.395+0000 7ff428f9a640 1 -- 172.21.15.83:0/4150302862 >> v1:172.21.15.167:6789/0 conn(0x7ff4240ce4e0 legacy=0x7ff4240d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:54.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.395+0000 7ff428f9a640 1 -- 172.21.15.83:0/4150302862 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4240cebc0 con 0x7ff4240d26c0 2024-04-17T10:38:54.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.399+0000 7ff428f9a640 1 --2- 172.21.15.83:0/4150302862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4240d26c0 0x7ff42415d620 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff42406af50 tx=0x7ff41c01c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bdf1f7c5b9d8662 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:54.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.399+0000 7ff422ffd640 1 -- 172.21.15.83:0/4150302862 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41c0a5660 con 0x7ff4240d26c0 2024-04-17T10:38:54.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.399+0000 7ff422ffd640 1 -- 172.21.15.83:0/4150302862 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff41c0a5800 con 0x7ff4240d26c0 2024-04-17T10:38:54.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.399+0000 7ff422ffd640 1 -- 172.21.15.83:0/4150302862 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41c0a5bd0 con 0x7ff4240d26c0 2024-04-17T10:38:54.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.399+0000 7ff42a9fd640 1 -- 172.21.15.83:0/4150302862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4240d26c0 msgr2=0x7ff42415d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:54.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.399+0000 7ff42a9fd640 1 --2- 172.21.15.83:0/4150302862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4240d26c0 0x7ff42415d620 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff42406af50 tx=0x7ff41c01c7a0 comp rx=0 tx=0).stop 2024-04-17T10:38:54.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.399+0000 7ff42a9fd640 1 -- 172.21.15.83:0/4150302862 shutdown_connections 2024-04-17T10:38:54.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.399+0000 7ff42a9fd640 1 --2- 172.21.15.83:0/4150302862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4240d26c0 0x7ff42415d620 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:54.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.399+0000 7ff42a9fd640 1 --2- 172.21.15.83:0/4150302862 >> v2:172.21.15.167:3300/0 conn(0x7ff4240d0970 0x7ff4240d17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:54.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.399+0000 7ff42a9fd640 1 -- 172.21.15.83:0/4150302862 >> 172.21.15.83:0/4150302862 conn(0x7ff42409cd50 msgr2=0x7ff4240d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:54.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.399+0000 7ff42a9fd640 1 -- 172.21.15.83:0/4150302862 shutdown_connections 2024-04-17T10:38:54.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.399+0000 7ff42a9fd640 1 -- 172.21.15.83:0/4150302862 wait complete. 2024-04-17T10:38:54.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.399+0000 7ff42a9fd640 1 Processor -- start 2024-04-17T10:38:54.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.399+0000 7ff42a9fd640 1 -- start start 2024-04-17T10:38:54.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.403+0000 7ff42a9fd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4240ce4e0 0x7ff4240b65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:54.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.403+0000 7ff42a9fd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4240d0970 0x7ff42414fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:54.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.403+0000 7ff42a9fd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4240d26c0 0x7ff4240cce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:54.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.403+0000 7ff42a9fd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42406b280 con 0x7ff4240d0970 2024-04-17T10:38:54.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.403+0000 7ff42a9fd640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4240d0d50 con 0x7ff4240d26c0 2024-04-17T10:38:54.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.403+0000 7ff42a9fd640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff42406b400 con 0x7ff4240ce4e0 2024-04-17T10:38:54.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.403+0000 7ff4237fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4240d0970 0x7ff42414fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:54.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.403+0000 7ff4237fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4240d0970 0x7ff42414fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51748/0 (socket says 172.21.15.83:51748) 2024-04-17T10:38:54.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.403+0000 7ff4237fe640 1 -- 172.21.15.83:0/341006180 learned_addr learned my addr 172.21.15.83:0/341006180 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:54.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.403+0000 7ff423fff640 1 --2- 172.21.15.83:0/341006180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4240ce4e0 0x7ff4240b65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:54.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.403+0000 7ff428f9a640 1 --2- 172.21.15.83:0/341006180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4240d26c0 0x7ff4240cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:54.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.403+0000 7ff4237fe640 1 -- 172.21.15.83:0/341006180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4240ce4e0 msgr2=0x7ff4240b65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:54.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.403+0000 7ff4237fe640 1 --2- 172.21.15.83:0/341006180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4240ce4e0 0x7ff4240b65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:54.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.403+0000 7ff4237fe640 1 -- 172.21.15.83:0/341006180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4240d26c0 msgr2=0x7ff4240cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:54.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.403+0000 7ff4237fe640 1 --2- 172.21.15.83:0/341006180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4240d26c0 0x7ff4240cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:54.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.403+0000 7ff4237fe640 1 -- 172.21.15.83:0/341006180 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4240cb5e0 con 0x7ff4240d0970 2024-04-17T10:38:54.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.403+0000 7ff4237fe640 1 --2- 172.21.15.83:0/341006180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4240d0970 0x7ff42414fe90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff41400c480 tx=0x7ff41400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:54.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.403+0000 7ff420ff9640 1 -- 172.21.15.83:0/341006180 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414012830 con 0x7ff4240d0970 2024-04-17T10:38:54.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.403+0000 7ff420ff9640 1 -- 172.21.15.83:0/341006180 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff414017070 con 0x7ff4240d0970 2024-04-17T10:38:54.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.403+0000 7ff420ff9640 1 -- 172.21.15.83:0/341006180 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414012c30 con 0x7ff4240d0970 2024-04-17T10:38:54.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.403+0000 7ff42a9fd640 1 -- 172.21.15.83:0/341006180 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4240cb8c0 con 0x7ff4240d0970 2024-04-17T10:38:54.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.403+0000 7ff42a9fd640 1 -- 172.21.15.83:0/341006180 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4240cbe80 con 0x7ff4240d0970 2024-04-17T10:38:54.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.403+0000 7ff42a9fd640 1 -- 172.21.15.83:0/341006180 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4240681a0 con 0x7ff4240d0970 2024-04-17T10:38:54.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.403+0000 7ff420ff9640 1 -- 172.21.15.83:0/341006180 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff41401c460 con 0x7ff4240d0970 2024-04-17T10:38:54.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.403+0000 7ff420ff9640 1 --2- 172.21.15.83:0/341006180 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff410066f90 0x7ff410069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:54.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.403+0000 7ff420ff9640 1 -- 172.21.15.83:0/341006180 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7ff414084e60 con 0x7ff4240d0970 2024-04-17T10:38:54.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.403+0000 7ff423fff640 1 --2- 172.21.15.83:0/341006180 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff410066f90 0x7ff410069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:54.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.407+0000 7ff423fff640 1 --2- 172.21.15.83:0/341006180 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff410066f90 0x7ff410069450 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff418009790 tx=0x7ff418008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:54.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.407+0000 7ff420ff9640 1 -- 172.21.15.83:0/341006180 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff414055e80 con 0x7ff4240d0970 2024-04-17T10:38:54.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.551+0000 7ff42a9fd640 1 -- 172.21.15.83:0/341006180 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff424074f90 con 0x7ff4240d0970 2024-04-17T10:38:54.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.555+0000 7ff420ff9640 1 -- 172.21.15.83:0/341006180 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff414055780 con 0x7ff4240d0970 2024-04-17T10:38:54.551 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:38:54.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.555+0000 7ff42a9fd640 1 -- 172.21.15.83:0/341006180 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff410066f90 msgr2=0x7ff410069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:54.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.555+0000 7ff42a9fd640 1 --2- 172.21.15.83:0/341006180 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff410066f90 0x7ff410069450 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff418009790 tx=0x7ff418008040 comp rx=0 tx=0).stop 2024-04-17T10:38:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.555+0000 7ff42a9fd640 1 -- 172.21.15.83:0/341006180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4240d0970 msgr2=0x7ff42414fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.555+0000 7ff42a9fd640 1 --2- 172.21.15.83:0/341006180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4240d0970 0x7ff42414fe90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff41400c480 tx=0x7ff41400c940 comp rx=0 tx=0).stop 2024-04-17T10:38:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.555+0000 7ff42a9fd640 1 -- 172.21.15.83:0/341006180 shutdown_connections 2024-04-17T10:38:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.555+0000 7ff42a9fd640 1 --2- 172.21.15.83:0/341006180 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff410066f90 0x7ff410069450 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.555+0000 7ff42a9fd640 1 --2- 172.21.15.83:0/341006180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4240d26c0 0x7ff4240cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.555+0000 7ff42a9fd640 1 --2- 172.21.15.83:0/341006180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4240d0970 0x7ff42414fe90 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.555+0000 7ff42a9fd640 1 --2- 172.21.15.83:0/341006180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4240ce4e0 0x7ff4240b65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.555+0000 7ff42a9fd640 1 -- 172.21.15.83:0/341006180 >> 172.21.15.83:0/341006180 conn(0x7ff42409cd50 msgr2=0x7ff4240b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.555+0000 7ff42a9fd640 1 -- 172.21.15.83:0/341006180 shutdown_connections 2024-04-17T10:38:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.555+0000 7ff42a9fd640 1 -- 172.21.15.83:0/341006180 wait complete. 2024-04-17T10:38:54.569 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:38:54.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.679+0000 7f2e18b38640 1 Processor -- start 2024-04-17T10:38:54.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.679+0000 7f2e18b38640 1 -- start start 2024-04-17T10:38:54.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.679+0000 7f2e18b38640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2e140b4090 0x7f2e1406c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:54.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.679+0000 7f2e18b38640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e140af160 con 0x7f2e140b4520 2024-04-17T10:38:54.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.679+0000 7f2e18b38640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e140b2ff0 con 0x7f2e14154670 2024-04-17T10:38:54.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.679+0000 7f2e18b38640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e14154a50 con 0x7f2e140b4090 2024-04-17T10:38:54.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.679+0000 7f2e12d9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2e140b4520 0x7f2e14070470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40106/0 (socket says 172.21.15.83:40106) 2024-04-17T10:38:54.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.679+0000 7f2e12d9d640 1 -- 172.21.15.83:0/2214300839 learned_addr learned my addr 172.21.15.83:0/2214300839 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:54.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.683+0000 7f2e11d9b640 1 --2- 172.21.15.83:0/2214300839 >> v2:172.21.15.167:3300/0 conn(0x7f2e140b4090 0x7f2e1406c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:54.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.683+0000 7f2e1159a640 1 -- 172.21.15.83:0/2214300839 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4131670675 0 0) 0x7f2e140af160 con 0x7f2e140b4520 2024-04-17T10:38:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.683+0000 7f2e1159a640 1 -- 172.21.15.83:0/2214300839 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2dfc003450 con 0x7f2e140b4520 2024-04-17T10:38:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.683+0000 7f2e1159a640 1 -- 172.21.15.83:0/2214300839 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1552599427 0 0) 0x7f2dfc003450 con 0x7f2e140b4520 2024-04-17T10:38:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.683+0000 7f2e1159a640 1 -- 172.21.15.83:0/2214300839 >> v2:172.21.15.167:3300/0 conn(0x7f2e140b4090 msgr2=0x7f2e1406c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.683+0000 7f2e1159a640 1 --2- 172.21.15.83:0/2214300839 >> v2:172.21.15.167:3300/0 conn(0x7f2e140b4090 0x7f2e1406c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.683+0000 7f2e1159a640 1 -- 172.21.15.83:0/2214300839 >> v1:172.21.15.167:6789/0 conn(0x7f2e14154670 legacy=0x7f2e140aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.683+0000 7f2e1159a640 1 -- 172.21.15.83:0/2214300839 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e140b32d0 con 0x7f2e140b4520 2024-04-17T10:38:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.683+0000 7f2e1159a640 1 -- 172.21.15.83:0/2214300839 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f2e0c001a00 con 0x7f2e140b4520 2024-04-17T10:38:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.683+0000 7f2e11d9b640 1 --2- 172.21.15.83:0/2214300839 >> v2:172.21.15.167:3300/0 conn(0x7f2e140b4090 0x7f2e1406c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:38:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.683+0000 7f2e1159a640 1 -- 172.21.15.83:0/2214300839 >> v1:172.21.15.83:6789/0 conn(0x7f2e140b4520 legacy=0x7f2e14070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.683+0000 7f2e1159a640 1 --2- 172.21.15.83:0/2214300839 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2dfc003b20 0x7f2dfc003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.683+0000 7f2e1159a640 1 --2- 172.21.15.83:0/2214300839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2dfc008960 0x7f2dfc00adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.683+0000 7f2e1159a640 1 --2- 172.21.15.83:0/2214300839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dfc00b5f0 0x7f2dfc00dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.683+0000 7f2e1159a640 1 -- 172.21.15.83:0/2214300839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e14154a50 con 0x7f2dfc00b5f0 2024-04-17T10:38:54.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.683+0000 7f2e1159a640 1 -- 172.21.15.83:0/2214300839 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dfc00e390 con 0x7f2dfc008960 2024-04-17T10:38:54.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.683+0000 7f2e1159a640 1 -- 172.21.15.83:0/2214300839 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dfc00e590 con 0x7f2dfc003b20 2024-04-17T10:38:54.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.683+0000 7f2e12d9d640 1 --2- 172.21.15.83:0/2214300839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dfc00b5f0 0x7f2dfc00dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:54.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.683+0000 7f2e11d9b640 1 --2- 172.21.15.83:0/2214300839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2dfc008960 0x7f2dfc00adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:54.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.683+0000 7f2e1259c640 1 --2- 172.21.15.83:0/2214300839 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2dfc003b20 0x7f2dfc003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:54.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.683+0000 7f2e12d9d640 1 -- 172.21.15.83:0/2214300839 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2dfc003b20 msgr2=0x7f2dfc003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:54.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.683+0000 7f2e12d9d640 1 --2- 172.21.15.83:0/2214300839 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2dfc003b20 0x7f2dfc003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:54.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.683+0000 7f2e12d9d640 1 -- 172.21.15.83:0/2214300839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2dfc008960 msgr2=0x7f2dfc00adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:54.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.683+0000 7f2e12d9d640 1 --2- 172.21.15.83:0/2214300839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2dfc008960 0x7f2dfc00adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:54.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.683+0000 7f2e12d9d640 1 -- 172.21.15.83:0/2214300839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dfc003450 con 0x7f2dfc00b5f0 2024-04-17T10:38:54.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.683+0000 7f2e1259c640 1 --2- 172.21.15.83:0/2214300839 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2dfc003b20 0x7f2dfc003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:54.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.683+0000 7f2e12d9d640 1 -- 172.21.15.83:0/2214300839 reap_dead start 2024-04-17T10:38:54.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.683+0000 7f2e12d9d640 1 --2- 172.21.15.83:0/2214300839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dfc00b5f0 0x7f2dfc00dab0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c00a860 tx=0x7f2e0c01d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385891849c119ae5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:54.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.683+0000 7f2e1159a640 1 -- 172.21.15.83:0/2214300839 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c00a480 con 0x7f2dfc00b5f0 2024-04-17T10:38:54.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.683+0000 7f2e1159a640 1 -- 172.21.15.83:0/2214300839 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e0c0ad630 con 0x7f2dfc00b5f0 2024-04-17T10:38:54.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.683+0000 7f2e1159a640 1 -- 172.21.15.83:0/2214300839 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c0ada00 con 0x7f2dfc00b5f0 2024-04-17T10:38:54.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.687+0000 7f2e18b38640 1 -- 172.21.15.83:0/2214300839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dfc00b5f0 msgr2=0x7f2dfc00dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:54.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.687+0000 7f2e18b38640 1 --2- 172.21.15.83:0/2214300839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dfc00b5f0 0x7f2dfc00dab0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c00a860 tx=0x7f2e0c01d380 comp rx=0 tx=0).stop 2024-04-17T10:38:54.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.687+0000 7f2e18b38640 1 -- 172.21.15.83:0/2214300839 shutdown_connections 2024-04-17T10:38:54.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.687+0000 7f2e18b38640 1 --2- 172.21.15.83:0/2214300839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dfc00b5f0 0x7f2dfc00dab0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:54.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.687+0000 7f2e18b38640 1 -- 172.21.15.83:0/2214300839 >> 172.21.15.83:0/2214300839 conn(0x7f2e140b15b0 msgr2=0x7f2e140b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:54.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.687+0000 7f2e18b38640 1 -- 172.21.15.83:0/2214300839 shutdown_connections 2024-04-17T10:38:54.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.687+0000 7f2e18b38640 1 -- 172.21.15.83:0/2214300839 wait complete. 2024-04-17T10:38:54.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.687+0000 7f2e18b38640 1 Processor -- start 2024-04-17T10:38:54.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.687+0000 7f2e18b38640 1 -- start start 2024-04-17T10:38:54.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.687+0000 7f2e18b38640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2dfc00b5f0 0x7f2e140ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:54.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.687+0000 7f2e18b38640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e140adbd0 0x7f2e1415ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:54.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.687+0000 7f2e18b38640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e14142080 0x7f2e14142530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:54.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.687+0000 7f2e18b38640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e140b3890 con 0x7f2e140adbd0 2024-04-17T10:38:54.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.687+0000 7f2e18b38640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e140b2e50 con 0x7f2dfc00b5f0 2024-04-17T10:38:54.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.687+0000 7f2e18b38640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e140b3b70 con 0x7f2e14142080 2024-04-17T10:38:54.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.687+0000 7f2e11d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e140adbd0 0x7f2e1415ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:54.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.687+0000 7f2e11d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e140adbd0 0x7f2e1415ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51768/0 (socket says 172.21.15.83:51768) 2024-04-17T10:38:54.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.687+0000 7f2e11d9b640 1 -- 172.21.15.83:0/2959483195 learned_addr learned my addr 172.21.15.83:0/2959483195 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:54.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.687+0000 7f2e12d9d640 1 --2- 172.21.15.83:0/2959483195 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e14142080 0x7f2e14142530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:54.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.687+0000 7f2e1259c640 1 --2- 172.21.15.83:0/2959483195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2dfc00b5f0 0x7f2e140ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:54.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.687+0000 7f2e11d9b640 1 -- 172.21.15.83:0/2959483195 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e14142080 msgr2=0x7f2e14142530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:54.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.687+0000 7f2e11d9b640 1 --2- 172.21.15.83:0/2959483195 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e14142080 0x7f2e14142530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:54.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.687+0000 7f2e11d9b640 1 -- 172.21.15.83:0/2959483195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2dfc00b5f0 msgr2=0x7f2e140ab7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:54.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.687+0000 7f2e11d9b640 1 --2- 172.21.15.83:0/2959483195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2dfc00b5f0 0x7f2e140ab7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:54.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.687+0000 7f2e11d9b640 1 -- 172.21.15.83:0/2959483195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e140ad150 con 0x7f2e140adbd0 2024-04-17T10:38:54.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.687+0000 7f2e11d9b640 1 --2- 172.21.15.83:0/2959483195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e140adbd0 0x7f2e1415ca90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2e0800f510 tx=0x7f2e0800f9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:54.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.687+0000 7f2e02ffd640 1 -- 172.21.15.83:0/2959483195 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e08018020 con 0x7f2e140adbd0 2024-04-17T10:38:54.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.687+0000 7f2e02ffd640 1 -- 172.21.15.83:0/2959483195 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e08014890 con 0x7f2e140adbd0 2024-04-17T10:38:54.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.687+0000 7f2e18b38640 1 -- 172.21.15.83:0/2959483195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e140ac0b0 con 0x7f2e140adbd0 2024-04-17T10:38:54.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.687+0000 7f2e02ffd640 1 -- 172.21.15.83:0/2959483195 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e08014c80 con 0x7f2e140adbd0 2024-04-17T10:38:54.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.687+0000 7f2e18b38640 1 -- 172.21.15.83:0/2959483195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e140ac670 con 0x7f2e140adbd0 2024-04-17T10:38:54.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.687+0000 7f2e18b38640 1 -- 172.21.15.83:0/2959483195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e14074f90 con 0x7f2e140adbd0 2024-04-17T10:38:54.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.691+0000 7f2e02ffd640 1 -- 172.21.15.83:0/2959483195 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2e0801b020 con 0x7f2e140adbd0 2024-04-17T10:38:54.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.691+0000 7f2e02ffd640 1 --2- 172.21.15.83:0/2959483195 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2df4066f90 0x7f2df4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:54.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.691+0000 7f2e1259c640 1 --2- 172.21.15.83:0/2959483195 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2df4066f90 0x7f2df4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:54.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.691+0000 7f2e02ffd640 1 -- 172.21.15.83:0/2959483195 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f2e080852e0 con 0x7f2e140adbd0 2024-04-17T10:38:54.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.691+0000 7f2e1259c640 1 --2- 172.21.15.83:0/2959483195 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2df4066f90 0x7f2df4069450 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2e04007fd0 tx=0x7f2e04009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:54.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.695+0000 7f2e02ffd640 1 -- 172.21.15.83:0/2959483195 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2e08022020 con 0x7f2e140adbd0 2024-04-17T10:38:54.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.851+0000 7f2e18b38640 1 -- 172.21.15.83:0/2959483195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2e140cf240 con 0x7f2e140adbd0 2024-04-17T10:38:54.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.851+0000 7f2e02ffd640 1 -- 172.21.15.83:0/2959483195 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2e08014e20 con 0x7f2e140adbd0 2024-04-17T10:38:54.849 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-17T10:38:54.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.855+0000 7f2e18b38640 1 -- 172.21.15.83:0/2959483195 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2df4066f90 msgr2=0x7f2df4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:54.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.855+0000 7f2e18b38640 1 --2- 172.21.15.83:0/2959483195 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2df4066f90 0x7f2df4069450 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2e04007fd0 tx=0x7f2e04009040 comp rx=0 tx=0).stop 2024-04-17T10:38:54.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.855+0000 7f2e18b38640 1 -- 172.21.15.83:0/2959483195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e140adbd0 msgr2=0x7f2e1415ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:54.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.855+0000 7f2e18b38640 1 --2- 172.21.15.83:0/2959483195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e140adbd0 0x7f2e1415ca90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2e0800f510 tx=0x7f2e0800f9d0 comp rx=0 tx=0).stop 2024-04-17T10:38:54.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.855+0000 7f2e18b38640 1 -- 172.21.15.83:0/2959483195 shutdown_connections 2024-04-17T10:38:54.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.855+0000 7f2e18b38640 1 --2- 172.21.15.83:0/2959483195 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2df4066f90 0x7f2df4069450 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:54.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.855+0000 7f2e18b38640 1 --2- 172.21.15.83:0/2959483195 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e14142080 0x7f2e14142530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:54.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.855+0000 7f2e18b38640 1 --2- 172.21.15.83:0/2959483195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e140adbd0 0x7f2e1415ca90 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:54.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.855+0000 7f2e18b38640 1 --2- 172.21.15.83:0/2959483195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2dfc00b5f0 0x7f2e140ab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:54.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.855+0000 7f2e18b38640 1 -- 172.21.15.83:0/2959483195 >> 172.21.15.83:0/2959483195 conn(0x7f2e140b15b0 msgr2=0x7f2e140714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:54.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.855+0000 7f2e18b38640 1 -- 172.21.15.83:0/2959483195 shutdown_connections 2024-04-17T10:38:54.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:54.855+0000 7f2e18b38640 1 -- 172.21.15.83:0/2959483195 wait complete. 2024-04-17T10:38:54.867 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:38:54.867 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:38:54.873 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:38:55.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.035+0000 7f33569bf640 1 Processor -- start 2024-04-17T10:38:55.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.035+0000 7f33569bf640 1 -- start start 2024-04-17T10:38:55.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.035+0000 7f33569bf640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f335015ba30 0x7f3350072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:55.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.035+0000 7f33569bf640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f335015c960 0x7f33500cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:55.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.035+0000 7f33569bf640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f33500cfa10 0x7f335006c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:55.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.035+0000 7f33569bf640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3350071ea0 con 0x7f335015c960 2024-04-17T10:38:55.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.035+0000 7f33569bf640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33500cfea0 con 0x7f33500cfa10 2024-04-17T10:38:55.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.035+0000 7f33569bf640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f335006e320 con 0x7f335015ba30 2024-04-17T10:38:55.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.035+0000 7f334f7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f335015c960 0x7f33500cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:55.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.035+0000 7f334f7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f335015c960 0x7f33500cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51784/0 (socket says 172.21.15.83:51784) 2024-04-17T10:38:55.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.035+0000 7f334f7fe640 1 -- 172.21.15.83:0/550746662 learned_addr learned my addr 172.21.15.83:0/550746662 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:55.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.035+0000 7f334ffff640 1 --2- 172.21.15.83:0/550746662 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f335015ba30 0x7f3350072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:55.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.035+0000 7f334f7fe640 1 -- 172.21.15.83:0/550746662 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f335015ba30 msgr2=0x7f3350072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:55.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.035+0000 7f3354f5c640 1 --2- 172.21.15.83:0/550746662 >> v2:172.21.15.167:3300/0 conn(0x7f33500cfa10 0x7f335006c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:55.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.035+0000 7f334f7fe640 1 --2- 172.21.15.83:0/550746662 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f335015ba30 0x7f3350072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.035+0000 7f334f7fe640 1 -- 172.21.15.83:0/550746662 >> v2:172.21.15.167:3300/0 conn(0x7f33500cfa10 msgr2=0x7f335006c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:55.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.035+0000 7f334f7fe640 1 --2- 172.21.15.83:0/550746662 >> v2:172.21.15.167:3300/0 conn(0x7f33500cfa10 0x7f335006c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.035+0000 7f334f7fe640 1 -- 172.21.15.83:0/550746662 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f335006e040 con 0x7f335015c960 2024-04-17T10:38:55.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f334f7fe640 1 --2- 172.21.15.83:0/550746662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f335015c960 0x7f33500cf4d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f335006d810 tx=0x7f334001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99030d22ca9d1d9d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:55.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f334effd640 1 -- 172.21.15.83:0/550746662 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33400b0070 con 0x7f335015c960 2024-04-17T10:38:55.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f334effd640 1 -- 172.21.15.83:0/550746662 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33400ab600 con 0x7f335015c960 2024-04-17T10:38:55.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f334effd640 1 -- 172.21.15.83:0/550746662 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33400ab9d0 con 0x7f335015c960 2024-04-17T10:38:55.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f33569bf640 1 -- 172.21.15.83:0/550746662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f335015c960 msgr2=0x7f33500cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:55.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f33569bf640 1 --2- 172.21.15.83:0/550746662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f335015c960 0x7f33500cf4d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f335006d810 tx=0x7f334001c490 comp rx=0 tx=0).stop 2024-04-17T10:38:55.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f33569bf640 1 -- 172.21.15.83:0/550746662 shutdown_connections 2024-04-17T10:38:55.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f33569bf640 1 --2- 172.21.15.83:0/550746662 >> v2:172.21.15.167:3300/0 conn(0x7f33500cfa10 0x7f335006c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f33569bf640 1 --2- 172.21.15.83:0/550746662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f335015c960 0x7f33500cf4d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f33569bf640 1 --2- 172.21.15.83:0/550746662 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f335015ba30 0x7f3350072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f33569bf640 1 -- 172.21.15.83:0/550746662 >> 172.21.15.83:0/550746662 conn(0x7f335006abe0 msgr2=0x7f335006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:55.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f33569bf640 1 -- 172.21.15.83:0/550746662 shutdown_connections 2024-04-17T10:38:55.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f33569bf640 1 -- 172.21.15.83:0/550746662 wait complete. 2024-04-17T10:38:55.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f33569bf640 1 Processor -- start 2024-04-17T10:38:55.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f33569bf640 1 -- start start 2024-04-17T10:38:55.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f33569bf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33500cfa10 0x7f33501506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:55.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f33569bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f335015ba30 0x7f33500af500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:55.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f33569bf640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f335015c960 0x7f33500afa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:55.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f33569bf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335006dca0 con 0x7f335015ba30 2024-04-17T10:38:55.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f33569bf640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33500aeae0 con 0x7f33500cfa10 2024-04-17T10:38:55.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f33569bf640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f335006de20 con 0x7f335015c960 2024-04-17T10:38:55.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f334f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f335015ba30 0x7f33500af500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:55.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f334f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f335015ba30 0x7f33500af500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51788/0 (socket says 172.21.15.83:51788) 2024-04-17T10:38:55.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f334f7fe640 1 -- 172.21.15.83:0/976497614 learned_addr learned my addr 172.21.15.83:0/976497614 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:55.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f3354f5c640 1 --2- 172.21.15.83:0/976497614 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f335015c960 0x7f33500afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:55.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f334ffff640 1 --2- 172.21.15.83:0/976497614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33500cfa10 0x7f33501506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:55.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f334f7fe640 1 -- 172.21.15.83:0/976497614 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f335015c960 msgr2=0x7f33500afa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:55.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f334f7fe640 1 --2- 172.21.15.83:0/976497614 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f335015c960 0x7f33500afa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f334f7fe640 1 -- 172.21.15.83:0/976497614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33500cfa10 msgr2=0x7f33501506d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:55.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f334f7fe640 1 --2- 172.21.15.83:0/976497614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33500cfa10 0x7f33501506d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f334f7fe640 1 -- 172.21.15.83:0/976497614 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3350156ac0 con 0x7f335015ba30 2024-04-17T10:38:55.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f334f7fe640 1 --2- 172.21.15.83:0/976497614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f335015ba30 0x7f33500af500 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f334001cd30 tx=0x7f3340019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:55.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f334cff9640 1 -- 172.21.15.83:0/976497614 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33400b9070 con 0x7f335015ba30 2024-04-17T10:38:55.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f334cff9640 1 -- 172.21.15.83:0/976497614 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33400b4640 con 0x7f335015ba30 2024-04-17T10:38:55.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f33569bf640 1 -- 172.21.15.83:0/976497614 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3350156da0 con 0x7f335015ba30 2024-04-17T10:38:55.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.039+0000 7f334cff9640 1 -- 172.21.15.83:0/976497614 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33400b0040 con 0x7f335015ba30 2024-04-17T10:38:55.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.043+0000 7f33569bf640 1 -- 172.21.15.83:0/976497614 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3350156750 con 0x7f335015ba30 2024-04-17T10:38:55.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.043+0000 7f33569bf640 1 -- 172.21.15.83:0/976497614 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3350074f90 con 0x7f335015ba30 2024-04-17T10:38:55.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.043+0000 7f334cff9640 1 -- 172.21.15.83:0/976497614 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f33400bc070 con 0x7f335015ba30 2024-04-17T10:38:55.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.043+0000 7f334cff9640 1 --2- 172.21.15.83:0/976497614 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f333c066f90 0x7f333c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:55.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.043+0000 7f334cff9640 1 -- 172.21.15.83:0/976497614 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f334011e310 con 0x7f335015ba30 2024-04-17T10:38:55.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.043+0000 7f334ffff640 1 --2- 172.21.15.83:0/976497614 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f333c066f90 0x7f333c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:55.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.043+0000 7f334ffff640 1 --2- 172.21.15.83:0/976497614 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f333c066f90 0x7f333c069450 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f335006dc20 tx=0x7f3344008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:55.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.047+0000 7f334cff9640 1 -- 172.21.15.83:0/976497614 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3340016110 con 0x7f335015ba30 2024-04-17T10:38:55.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.191+0000 7f33569bf640 1 -- 172.21.15.83:0/976497614 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33500b2530 con 0x7f335015ba30 2024-04-17T10:38:55.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.195+0000 7f334cff9640 1 -- 172.21.15.83:0/976497614 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16782 (secure 0 0 0) 0x7f334011b1a0 con 0x7f335015ba30 2024-04-17T10:38:55.193 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:55.193 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":22,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:38:53.385002+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:38:10.464490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T10:38:11.043134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:38:55.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.199+0000 7f33569bf640 1 -- 172.21.15.83:0/976497614 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f333c066f90 msgr2=0x7f333c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:55.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.199+0000 7f33569bf640 1 --2- 172.21.15.83:0/976497614 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f333c066f90 0x7f333c069450 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f335006dc20 tx=0x7f3344008040 comp rx=0 tx=0).stop 2024-04-17T10:38:55.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.199+0000 7f33569bf640 1 -- 172.21.15.83:0/976497614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f335015ba30 msgr2=0x7f33500af500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:55.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.199+0000 7f33569bf640 1 --2- 172.21.15.83:0/976497614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f335015ba30 0x7f33500af500 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f334001cd30 tx=0x7f3340019230 comp rx=0 tx=0).stop 2024-04-17T10:38:55.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.199+0000 7f33569bf640 1 -- 172.21.15.83:0/976497614 shutdown_connections 2024-04-17T10:38:55.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.199+0000 7f33569bf640 1 --2- 172.21.15.83:0/976497614 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f333c066f90 0x7f333c069450 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.199+0000 7f33569bf640 1 --2- 172.21.15.83:0/976497614 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f335015c960 0x7f33500afa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.199+0000 7f33569bf640 1 --2- 172.21.15.83:0/976497614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f335015ba30 0x7f33500af500 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.199+0000 7f33569bf640 1 --2- 172.21.15.83:0/976497614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33500cfa10 0x7f33501506d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.199+0000 7f33569bf640 1 -- 172.21.15.83:0/976497614 >> 172.21.15.83:0/976497614 conn(0x7f335006abe0 msgr2=0x7f335015e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:55.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.199+0000 7f33569bf640 1 -- 172.21.15.83:0/976497614 shutdown_connections 2024-04-17T10:38:55.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.199+0000 7f33569bf640 1 -- 172.21.15.83:0/976497614 wait complete. 2024-04-17T10:38:55.210 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:38:10.464490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:38:11.043134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:38:55.211 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:38:55.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.335+0000 7fde23a8e640 1 Processor -- start 2024-04-17T10:38:55.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.335+0000 7fde23a8e640 1 -- start start 2024-04-17T10:38:55.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.335+0000 7fde23a8e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fde1c15a8a0 0x7fde1c06e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:55.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.335+0000 7fde23a8e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde1c0b36f0 0x7fde1c06bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:55.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.335+0000 7fde23a8e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde1c0afbb0 con 0x7fde1c06c230 2024-04-17T10:38:55.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.335+0000 7fde23a8e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde1c06d760 con 0x7fde1c15a8a0 2024-04-17T10:38:55.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.335+0000 7fde23a8e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c06da40 con 0x7fde1c0b36f0 2024-04-17T10:38:55.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.335+0000 7fde2182a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fde1c15a8a0 0x7fde1c06e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:55.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.335+0000 7fde2202b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fde1c06c230 0x7fde1c0cd720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40114/0 (socket says 172.21.15.83:40114) 2024-04-17T10:38:55.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.335+0000 7fde2202b640 1 -- 172.21.15.83:0/3326233354 learned_addr learned my addr 172.21.15.83:0/3326233354 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:55.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.335+0000 7fde20828640 1 -- 172.21.15.83:0/3326233354 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3264799845 0 0) 0x7fde1c0afbb0 con 0x7fde1c06c230 2024-04-17T10:38:55.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.335+0000 7fde20828640 1 -- 172.21.15.83:0/3326233354 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde08005740 con 0x7fde1c06c230 2024-04-17T10:38:55.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.335+0000 7fde21029640 1 --2- 172.21.15.83:0/3326233354 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde1c0b36f0 0x7fde1c06bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:55.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.335+0000 7fde2182a640 1 -- 172.21.15.83:0/3326233354 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde1c0b36f0 msgr2=0x7fde1c06bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:55.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.335+0000 7fde2182a640 1 --2- 172.21.15.83:0/3326233354 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde1c0b36f0 0x7fde1c06bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.335+0000 7fde20828640 1 -- 172.21.15.83:0/3326233354 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 698973213 0 0) 0x7fde08005740 con 0x7fde1c06c230 2024-04-17T10:38:55.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.335+0000 7fde2182a640 1 -- 172.21.15.83:0/3326233354 >> v1:172.21.15.83:6789/0 conn(0x7fde1c06c230 legacy=0x7fde1c0cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:55.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.335+0000 7fde2182a640 1 -- 172.21.15.83:0/3326233354 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1c06dd20 con 0x7fde1c15a8a0 2024-04-17T10:38:55.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde2182a640 1 --2- 172.21.15.83:0/3326233354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde1c15a8a0 0x7fde1c06e800 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fde180097c0 tx=0x7fde1801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2ba1e3b877461d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:55.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde20828640 1 -- 172.21.15.83:0/3326233354 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde180b0070 con 0x7fde1c15a8a0 2024-04-17T10:38:55.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde20828640 1 -- 172.21.15.83:0/3326233354 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde180ab400 con 0x7fde1c15a8a0 2024-04-17T10:38:55.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde20828640 1 -- 172.21.15.83:0/3326233354 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde14001a00 con 0x7fde1c15a8a0 2024-04-17T10:38:55.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde23a8e640 1 -- 172.21.15.83:0/3326233354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde1c15a8a0 msgr2=0x7fde1c06e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:55.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde23a8e640 1 --2- 172.21.15.83:0/3326233354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde1c15a8a0 0x7fde1c06e800 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fde180097c0 tx=0x7fde1801c3c0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde23a8e640 1 -- 172.21.15.83:0/3326233354 shutdown_connections 2024-04-17T10:38:55.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde23a8e640 1 --2- 172.21.15.83:0/3326233354 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde1c0b36f0 0x7fde1c06bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde23a8e640 1 --2- 172.21.15.83:0/3326233354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde1c15a8a0 0x7fde1c06e800 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde23a8e640 1 -- 172.21.15.83:0/3326233354 >> 172.21.15.83:0/3326233354 conn(0x7fde1c0b23f0 msgr2=0x7fde1c0b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:55.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde23a8e640 1 -- 172.21.15.83:0/3326233354 shutdown_connections 2024-04-17T10:38:55.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde23a8e640 1 -- 172.21.15.83:0/3326233354 wait complete. 2024-04-17T10:38:55.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde23a8e640 1 Processor -- start 2024-04-17T10:38:55.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde23a8e640 1 -- start start 2024-04-17T10:38:55.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde23a8e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde1c06c230 0x7fde1c1417e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:55.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde23a8e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde1c0b36f0 0x7fde1c0b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:55.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde23a8e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde1c15a8a0 0x7fde1c0b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:55.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde23a8e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c0b3c50 con 0x7fde1c15a8a0 2024-04-17T10:38:55.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde23a8e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c0b3ad0 con 0x7fde1c0b36f0 2024-04-17T10:38:55.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde23a8e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c06abe0 con 0x7fde1c06c230 2024-04-17T10:38:55.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde2202b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde1c15a8a0 0x7fde1c0b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:55.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde2202b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde1c15a8a0 0x7fde1c0b1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51820/0 (socket says 172.21.15.83:51820) 2024-04-17T10:38:55.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde2202b640 1 -- 172.21.15.83:0/263313122 learned_addr learned my addr 172.21.15.83:0/263313122 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:55.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde2202b640 1 -- 172.21.15.83:0/263313122 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde1c06c230 msgr2=0x7fde1c1417e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:55.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde2202b640 1 --2- 172.21.15.83:0/263313122 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde1c06c230 0x7fde1c1417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde2202b640 1 -- 172.21.15.83:0/263313122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde1c0b36f0 msgr2=0x7fde1c0b14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:55.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde2202b640 1 --2- 172.21.15.83:0/263313122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde1c0b36f0 0x7fde1c0b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde2202b640 1 -- 172.21.15.83:0/263313122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1c1447d0 con 0x7fde1c15a8a0 2024-04-17T10:38:55.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde2202b640 1 --2- 172.21.15.83:0/263313122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde1c15a8a0 0x7fde1c0b1a20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fde140018a0 tx=0x7fde1400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:55.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde127fc640 1 -- 172.21.15.83:0/263313122 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde14012400 con 0x7fde1c15a8a0 2024-04-17T10:38:55.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde127fc640 1 -- 172.21.15.83:0/263313122 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde140125c0 con 0x7fde1c15a8a0 2024-04-17T10:38:55.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde23a8e640 1 -- 172.21.15.83:0/263313122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde1c141d20 con 0x7fde1c15a8a0 2024-04-17T10:38:55.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde127fc640 1 -- 172.21.15.83:0/263313122 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde140129b0 con 0x7fde1c15a8a0 2024-04-17T10:38:55.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.339+0000 7fde23a8e640 1 -- 172.21.15.83:0/263313122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde1c0cfc80 con 0x7fde1c15a8a0 2024-04-17T10:38:55.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.343+0000 7fde23a8e640 1 -- 172.21.15.83:0/263313122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde1c074f90 con 0x7fde1c15a8a0 2024-04-17T10:38:55.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.343+0000 7fde127fc640 1 -- 172.21.15.83:0/263313122 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fde14012b50 con 0x7fde1c15a8a0 2024-04-17T10:38:55.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.343+0000 7fde127fc640 1 --2- 172.21.15.83:0/263313122 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fde00066f60 0x7fde00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:55.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.343+0000 7fde127fc640 1 -- 172.21.15.83:0/263313122 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fde14085980 con 0x7fde1c15a8a0 2024-04-17T10:38:55.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.343+0000 7fde2182a640 1 --2- 172.21.15.83:0/263313122 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fde00066f60 0x7fde00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:55.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.343+0000 7fde2182a640 1 --2- 172.21.15.83:0/263313122 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fde00066f60 0x7fde00069420 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fde180ad000 tx=0x7fde18008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:55.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.347+0000 7fde127fc640 1 -- 172.21.15.83:0/263313122 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fde1c1447d0 con 0x7fde1c15a8a0 2024-04-17T10:38:55.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.503+0000 7fde23a8e640 1 -- 172.21.15.83:0/263313122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fde1c06f340 con 0x7fde1c15a8a0 2024-04-17T10:38:55.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.503+0000 7fde127fc640 1 -- 172.21.15.83:0/263313122 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fde14056910 con 0x7fde1c15a8a0 2024-04-17T10:38:55.499 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:38:55.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.503+0000 7fde23a8e640 1 -- 172.21.15.83:0/263313122 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fde00066f60 msgr2=0x7fde00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:55.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.503+0000 7fde23a8e640 1 --2- 172.21.15.83:0/263313122 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fde00066f60 0x7fde00069420 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fde180ad000 tx=0x7fde18008040 comp rx=0 tx=0).stop 2024-04-17T10:38:55.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.503+0000 7fde23a8e640 1 -- 172.21.15.83:0/263313122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde1c15a8a0 msgr2=0x7fde1c0b1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:55.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.503+0000 7fde23a8e640 1 --2- 172.21.15.83:0/263313122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde1c15a8a0 0x7fde1c0b1a20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fde140018a0 tx=0x7fde1400a430 comp rx=0 tx=0).stop 2024-04-17T10:38:55.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.503+0000 7fde23a8e640 1 -- 172.21.15.83:0/263313122 shutdown_connections 2024-04-17T10:38:55.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.503+0000 7fde23a8e640 1 --2- 172.21.15.83:0/263313122 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fde00066f60 0x7fde00069420 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.503+0000 7fde23a8e640 1 --2- 172.21.15.83:0/263313122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde1c15a8a0 0x7fde1c0b1a20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.503+0000 7fde23a8e640 1 --2- 172.21.15.83:0/263313122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde1c0b36f0 0x7fde1c0b14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.503+0000 7fde23a8e640 1 --2- 172.21.15.83:0/263313122 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde1c06c230 0x7fde1c1417e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.503+0000 7fde23a8e640 1 -- 172.21.15.83:0/263313122 >> 172.21.15.83:0/263313122 conn(0x7fde1c0b23f0 msgr2=0x7fde1c14af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:55.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.503+0000 7fde23a8e640 1 -- 172.21.15.83:0/263313122 shutdown_connections 2024-04-17T10:38:55.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.503+0000 7fde23a8e640 1 -- 172.21.15.83:0/263313122 wait complete. 2024-04-17T10:38:55.517 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:38:55.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f669c727640 1 Processor -- start 2024-04-17T10:38:55.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f669c727640 1 -- start start 2024-04-17T10:38:55.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f669c727640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f669415bac0 0x7f6694071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:55.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f669c727640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6694070470 0x7f669406ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:55.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f669c727640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f669406c1f0 con 0x7f669415b6e0 2024-04-17T10:38:55.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f669c727640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6694070a10 con 0x7f6694070470 2024-04-17T10:38:55.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f669c727640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6694070cf0 con 0x7f669415bac0 2024-04-17T10:38:55.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f669a4c3640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f669415b6e0 0x7f66940cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40130/0 (socket says 172.21.15.83:40130) 2024-04-17T10:38:55.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f669a4c3640 1 -- 172.21.15.83:0/101188187 learned_addr learned my addr 172.21.15.83:0/101188187 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:55.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f669acc4640 1 --2- 172.21.15.83:0/101188187 >> v2:172.21.15.167:3300/0 conn(0x7f6694070470 0x7f669406ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:55.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f6699cc2640 1 --2- 172.21.15.83:0/101188187 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f669415bac0 0x7f6694071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:55.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f66994c1640 1 -- 172.21.15.83:0/101188187 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1858419406 0 0) 0x7f669406c1f0 con 0x7f669415b6e0 2024-04-17T10:38:55.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f66994c1640 1 -- 172.21.15.83:0/101188187 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6680003530 con 0x7f669415b6e0 2024-04-17T10:38:55.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f66994c1640 1 -- 172.21.15.83:0/101188187 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3847168373 0 0) 0x7f6680003530 con 0x7f669415b6e0 2024-04-17T10:38:55.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f66994c1640 1 -- 172.21.15.83:0/101188187 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f669415bac0 msgr2=0x7f6694071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:55.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f66994c1640 1 --2- 172.21.15.83:0/101188187 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f669415bac0 0x7f6694071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f66994c1640 1 -- 172.21.15.83:0/101188187 >> v2:172.21.15.167:3300/0 conn(0x7f6694070470 msgr2=0x7f669406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:55.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f66994c1640 1 --2- 172.21.15.83:0/101188187 >> v2:172.21.15.167:3300/0 conn(0x7f6694070470 0x7f669406ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f66994c1640 1 -- 172.21.15.83:0/101188187 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6694070fd0 con 0x7f669415b6e0 2024-04-17T10:38:55.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f66994c1640 1 -- 172.21.15.83:0/101188187 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f6690001680 con 0x7f669415b6e0 2024-04-17T10:38:55.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f66994c1640 1 -- 172.21.15.83:0/101188187 >> v1:172.21.15.83:6789/0 conn(0x7f669415b6e0 legacy=0x7f66940cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:55.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f66994c1640 1 --2- 172.21.15.83:0/101188187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6680003a30 0x7f6680003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:55.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f66994c1640 1 --2- 172.21.15.83:0/101188187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6680004810 0x7f6680004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:55.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f66994c1640 1 --2- 172.21.15.83:0/101188187 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f669415bac0 0x7f66800051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:55.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f66994c1640 1 -- 172.21.15.83:0/101188187 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6694070a10 con 0x7f6680003a30 2024-04-17T10:38:55.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f66994c1640 1 -- 172.21.15.83:0/101188187 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6694070cf0 con 0x7f6680004810 2024-04-17T10:38:55.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f66994c1640 1 -- 172.21.15.83:0/101188187 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f668000b9d0 con 0x7f669415bac0 2024-04-17T10:38:55.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f669a4c3640 1 --2- 172.21.15.83:0/101188187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6680003a30 0x7f6680003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:55.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f6699cc2640 1 --2- 172.21.15.83:0/101188187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6680004810 0x7f6680004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:55.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f669a4c3640 1 -- 172.21.15.83:0/101188187 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f669415bac0 msgr2=0x7f66800051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:55.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f669a4c3640 1 --2- 172.21.15.83:0/101188187 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f669415bac0 0x7f66800051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f669a4c3640 1 -- 172.21.15.83:0/101188187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6680004810 msgr2=0x7f6680004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:55.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f669a4c3640 1 --2- 172.21.15.83:0/101188187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6680004810 0x7f6680004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.647+0000 7f669a4c3640 1 -- 172.21.15.83:0/101188187 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6680003530 con 0x7f6680003a30 2024-04-17T10:38:55.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.651+0000 7f669a4c3640 1 --2- 172.21.15.83:0/101188187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6680003a30 0x7f6680003e50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f669000c240 tx=0x7f669001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27a3e67654868ce server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:55.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.651+0000 7f66994c1640 1 -- 172.21.15.83:0/101188187 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6690007d60 con 0x7f6680003a30 2024-04-17T10:38:55.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.651+0000 7f66994c1640 1 -- 172.21.15.83:0/101188187 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66900a9020 con 0x7f6680003a30 2024-04-17T10:38:55.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.651+0000 7f66994c1640 1 -- 172.21.15.83:0/101188187 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6690005b20 con 0x7f6680003a30 2024-04-17T10:38:55.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.651+0000 7f669c727640 1 -- 172.21.15.83:0/101188187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6680003a30 msgr2=0x7f6680003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:55.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.651+0000 7f669c727640 1 --2- 172.21.15.83:0/101188187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6680003a30 0x7f6680003e50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f669000c240 tx=0x7f669001cc80 comp rx=0 tx=0).stop 2024-04-17T10:38:55.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.651+0000 7f669acc4640 1 -- 172.21.15.83:0/101188187 reap_dead start 2024-04-17T10:38:55.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.651+0000 7f669c727640 1 -- 172.21.15.83:0/101188187 shutdown_connections 2024-04-17T10:38:55.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.651+0000 7f669c727640 1 -- 172.21.15.83:0/101188187 >> 172.21.15.83:0/101188187 conn(0x7f66940b15b0 msgr2=0x7f66940b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:55.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.651+0000 7f669c727640 1 -- 172.21.15.83:0/101188187 shutdown_connections 2024-04-17T10:38:55.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.651+0000 7f669c727640 1 -- 172.21.15.83:0/101188187 wait complete. 2024-04-17T10:38:55.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.651+0000 7f669c727640 1 Processor -- start 2024-04-17T10:38:55.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.651+0000 7f669c727640 1 -- start start 2024-04-17T10:38:55.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.651+0000 7f669c727640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6680003a30 0x7f66940b42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:55.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.651+0000 7f669c727640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f669414a0f0 0x7f6694155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:55.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.651+0000 7f669c727640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66940ad870 0x7f66940adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:55.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.651+0000 7f669c727640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669406d270 con 0x7f6680003a30 2024-04-17T10:38:55.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.651+0000 7f669c727640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669406d0f0 con 0x7f669414a0f0 2024-04-17T10:38:55.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.651+0000 7f669c727640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66941665f0 con 0x7f66940ad870 2024-04-17T10:38:55.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.655+0000 7f669a4c3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6680003a30 0x7f66940b42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:55.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.655+0000 7f669a4c3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6680003a30 0x7f66940b42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51824/0 (socket says 172.21.15.83:51824) 2024-04-17T10:38:55.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.655+0000 7f669a4c3640 1 -- 172.21.15.83:0/2376908659 learned_addr learned my addr 172.21.15.83:0/2376908659 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:55.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.655+0000 7f669acc4640 1 --2- 172.21.15.83:0/2376908659 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66940ad870 0x7f66940adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:55.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.655+0000 7f6699cc2640 1 --2- 172.21.15.83:0/2376908659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f669414a0f0 0x7f6694155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:55.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.655+0000 7f669a4c3640 1 -- 172.21.15.83:0/2376908659 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66940ad870 msgr2=0x7f66940adcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:55.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.655+0000 7f669a4c3640 1 --2- 172.21.15.83:0/2376908659 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66940ad870 0x7f66940adcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.655+0000 7f669a4c3640 1 -- 172.21.15.83:0/2376908659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f669414a0f0 msgr2=0x7f6694155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:55.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.655+0000 7f669a4c3640 1 --2- 172.21.15.83:0/2376908659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f669414a0f0 0x7f6694155b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.655+0000 7f669a4c3640 1 -- 172.21.15.83:0/2376908659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6694155580 con 0x7f6680003a30 2024-04-17T10:38:55.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.655+0000 7f6699cc2640 1 --2- 172.21.15.83:0/2376908659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f669414a0f0 0x7f6694155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:55.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.655+0000 7f669a4c3640 1 --2- 172.21.15.83:0/2376908659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6680003a30 0x7f66940b42e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f669000c240 tx=0x7f6680005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:55.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.655+0000 7f668affd640 1 -- 172.21.15.83:0/2376908659 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66900047f0 con 0x7f6680003a30 2024-04-17T10:38:55.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.655+0000 7f668affd640 1 -- 172.21.15.83:0/2376908659 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6690004990 con 0x7f6680003a30 2024-04-17T10:38:55.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.655+0000 7f669c727640 1 -- 172.21.15.83:0/2376908659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66940b2f60 con 0x7f6680003a30 2024-04-17T10:38:55.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.655+0000 7f668affd640 1 -- 172.21.15.83:0/2376908659 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6690004d60 con 0x7f6680003a30 2024-04-17T10:38:55.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.655+0000 7f669c727640 1 -- 172.21.15.83:0/2376908659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66940b3520 con 0x7f6680003a30 2024-04-17T10:38:55.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.655+0000 7f669c727640 1 -- 172.21.15.83:0/2376908659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6694074f90 con 0x7f6680003a30 2024-04-17T10:38:55.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.655+0000 7f668affd640 1 -- 172.21.15.83:0/2376908659 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f66900070d0 con 0x7f6680003a30 2024-04-17T10:38:55.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.655+0000 7f668affd640 1 --2- 172.21.15.83:0/2376908659 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6678066f60 0x7f6678069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:55.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.655+0000 7f668affd640 1 -- 172.21.15.83:0/2376908659 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f669011e6b0 con 0x7f6680003a30 2024-04-17T10:38:55.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.655+0000 7f6699cc2640 1 --2- 172.21.15.83:0/2376908659 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6678066f60 0x7f6678069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:55.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.655+0000 7f6699cc2640 1 --2- 172.21.15.83:0/2376908659 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6678066f60 0x7f6678069420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f669414a910 tx=0x7f6684008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:55.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.659+0000 7f668affd640 1 -- 172.21.15.83:0/2376908659 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6690016990 con 0x7f6680003a30 2024-04-17T10:38:55.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.819+0000 7f669c727640 1 -- 172.21.15.83:0/2376908659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f66940ccd30 con 0x7f6680003a30 2024-04-17T10:38:55.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.819+0000 7f668affd640 1 -- 172.21.15.83:0/2376908659 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f66900b73a0 con 0x7f6680003a30 2024-04-17T10:38:55.817 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:38:55.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.823+0000 7f669c727640 1 -- 172.21.15.83:0/2376908659 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6678066f60 msgr2=0x7f6678069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:55.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.823+0000 7f669c727640 1 --2- 172.21.15.83:0/2376908659 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6678066f60 0x7f6678069420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f669414a910 tx=0x7f6684008040 comp rx=0 tx=0).stop 2024-04-17T10:38:55.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.823+0000 7f669c727640 1 -- 172.21.15.83:0/2376908659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6680003a30 msgr2=0x7f66940b42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:55.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.823+0000 7f669c727640 1 --2- 172.21.15.83:0/2376908659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6680003a30 0x7f66940b42e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f669000c240 tx=0x7f6680005840 comp rx=0 tx=0).stop 2024-04-17T10:38:55.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.823+0000 7f669c727640 1 -- 172.21.15.83:0/2376908659 shutdown_connections 2024-04-17T10:38:55.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.823+0000 7f669c727640 1 --2- 172.21.15.83:0/2376908659 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6678066f60 0x7f6678069420 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.823+0000 7f669c727640 1 --2- 172.21.15.83:0/2376908659 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66940ad870 0x7f66940adcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.823+0000 7f669c727640 1 --2- 172.21.15.83:0/2376908659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f669414a0f0 0x7f6694155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.823+0000 7f669c727640 1 --2- 172.21.15.83:0/2376908659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6680003a30 0x7f66940b42e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.823+0000 7f669c727640 1 -- 172.21.15.83:0/2376908659 >> 172.21.15.83:0/2376908659 conn(0x7f66940b15b0 msgr2=0x7f66940cffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:55.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.823+0000 7f669c727640 1 -- 172.21.15.83:0/2376908659 shutdown_connections 2024-04-17T10:38:55.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.823+0000 7f669c727640 1 -- 172.21.15.83:0/2376908659 wait complete. 2024-04-17T10:38:55.835 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:38:55.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a15cc1640 1 Processor -- start 2024-04-17T10:38:55.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a15cc1640 1 -- start start 2024-04-17T10:38:55.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a15cc1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a1015b6e0 0x7f4a10071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:55.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a15cc1640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4a10070470 0x7f4a1006ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:55.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a15cc1640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a1006c1f0 con 0x7f4a1015bac0 2024-04-17T10:38:55.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a15cc1640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a10070a10 con 0x7f4a10070470 2024-04-17T10:38:55.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a15cc1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a10070cf0 con 0x7f4a1015b6e0 2024-04-17T10:38:55.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a0effd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f4a1015bac0 0x7f4a100cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40142/0 (socket says 172.21.15.83:40142) 2024-04-17T10:38:55.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a0effd640 1 -- 172.21.15.83:0/1853037827 learned_addr learned my addr 172.21.15.83:0/1853037827 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:55.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a0e7fc640 1 -- 172.21.15.83:0/1853037827 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 206093603 0 0) 0x7f4a1006c1f0 con 0x7f4a1015bac0 2024-04-17T10:38:55.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a0ffff640 1 --2- 172.21.15.83:0/1853037827 >> v2:172.21.15.167:3300/0 conn(0x7f4a10070470 0x7f4a1006ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:55.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a0e7fc640 1 -- 172.21.15.83:0/1853037827 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49f8003530 con 0x7f4a1015bac0 2024-04-17T10:38:55.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a0f7fe640 1 --2- 172.21.15.83:0/1853037827 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a1015b6e0 0x7f4a10071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:55.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a0e7fc640 1 -- 172.21.15.83:0/1853037827 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 244853547 0 0) 0x7f49f8003530 con 0x7f4a1015bac0 2024-04-17T10:38:55.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a0e7fc640 1 -- 172.21.15.83:0/1853037827 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a1015b6e0 msgr2=0x7f4a10071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:55.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a0e7fc640 1 --2- 172.21.15.83:0/1853037827 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a1015b6e0 0x7f4a10071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a0e7fc640 1 -- 172.21.15.83:0/1853037827 >> v2:172.21.15.167:3300/0 conn(0x7f4a10070470 msgr2=0x7f4a1006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:55.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a0e7fc640 1 --2- 172.21.15.83:0/1853037827 >> v2:172.21.15.167:3300/0 conn(0x7f4a10070470 0x7f4a1006ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a0e7fc640 1 -- 172.21.15.83:0/1853037827 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a10070fd0 con 0x7f4a1015bac0 2024-04-17T10:38:55.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a0e7fc640 1 -- 172.21.15.83:0/1853037827 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f4a04001680 con 0x7f4a1015bac0 2024-04-17T10:38:55.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a0e7fc640 1 -- 172.21.15.83:0/1853037827 >> v1:172.21.15.83:6789/0 conn(0x7f4a1015bac0 legacy=0x7f4a100cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:55.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a0e7fc640 1 --2- 172.21.15.83:0/1853037827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49f8003a30 0x7f49f8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:55.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a0e7fc640 1 --2- 172.21.15.83:0/1853037827 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a1015b6e0 0x7f49f8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:55.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a0e7fc640 1 --2- 172.21.15.83:0/1853037827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49f8004d50 0x7f49f80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:55.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a0e7fc640 1 -- 172.21.15.83:0/1853037827 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a10070a10 con 0x7f49f8003a30 2024-04-17T10:38:55.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a0e7fc640 1 -- 172.21.15.83:0/1853037827 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a10070cf0 con 0x7f49f8004d50 2024-04-17T10:38:55.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a0e7fc640 1 -- 172.21.15.83:0/1853037827 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49f800bb00 con 0x7f4a1015b6e0 2024-04-17T10:38:55.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a0ffff640 1 --2- 172.21.15.83:0/1853037827 >> v2:172.21.15.167:3300/0 conn(0x7f4a10070470 0x7f4a1006ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:38:55.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a0f7fe640 1 --2- 172.21.15.83:0/1853037827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49f8003a30 0x7f49f8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:55.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a0effd640 1 --2- 172.21.15.83:0/1853037827 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a1015b6e0 0x7f49f8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:55.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a0ffff640 1 --2- 172.21.15.83:0/1853037827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49f8004d50 0x7f49f80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:55.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a0f7fe640 1 -- 172.21.15.83:0/1853037827 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a1015b6e0 msgr2=0x7f49f8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:55.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a0f7fe640 1 --2- 172.21.15.83:0/1853037827 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a1015b6e0 0x7f49f8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a0f7fe640 1 -- 172.21.15.83:0/1853037827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49f8004d50 msgr2=0x7f49f80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:55.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a0f7fe640 1 --2- 172.21.15.83:0/1853037827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49f8004d50 0x7f49f80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a0f7fe640 1 -- 172.21.15.83:0/1853037827 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f8003530 con 0x7f49f8003a30 2024-04-17T10:38:55.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.955+0000 7f4a0ffff640 1 --2- 172.21.15.83:0/1853037827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49f8004d50 0x7f49f80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:38:55.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.959+0000 7f4a0f7fe640 1 --2- 172.21.15.83:0/1853037827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49f8003a30 0x7f49f8003e50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4a0000ba00 tx=0x7f4a0001c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f772e1175115c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:55.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.959+0000 7f4a0e7fc640 1 -- 172.21.15.83:0/1853037827 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a000b3070 con 0x7f49f8003a30 2024-04-17T10:38:55.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.959+0000 7f4a0e7fc640 1 -- 172.21.15.83:0/1853037827 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a000a9070 con 0x7f49f8003a30 2024-04-17T10:38:55.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.959+0000 7f4a0e7fc640 1 -- 172.21.15.83:0/1853037827 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0000b600 con 0x7f49f8003a30 2024-04-17T10:38:55.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.959+0000 7f4a15cc1640 1 -- 172.21.15.83:0/1853037827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49f8003a30 msgr2=0x7f49f8003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:55.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.959+0000 7f4a15cc1640 1 --2- 172.21.15.83:0/1853037827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49f8003a30 0x7f49f8003e50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4a0000ba00 tx=0x7f4a0001c660 comp rx=0 tx=0).stop 2024-04-17T10:38:55.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.959+0000 7f4a0ffff640 1 -- 172.21.15.83:0/1853037827 reap_dead start 2024-04-17T10:38:55.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.959+0000 7f4a15cc1640 1 -- 172.21.15.83:0/1853037827 shutdown_connections 2024-04-17T10:38:55.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.959+0000 7f4a15cc1640 1 -- 172.21.15.83:0/1853037827 >> 172.21.15.83:0/1853037827 conn(0x7f4a100b15b0 msgr2=0x7f4a100b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:55.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.959+0000 7f4a15cc1640 1 -- 172.21.15.83:0/1853037827 shutdown_connections 2024-04-17T10:38:55.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.959+0000 7f4a15cc1640 1 -- 172.21.15.83:0/1853037827 wait complete. 2024-04-17T10:38:55.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.963+0000 7f4a15cc1640 1 Processor -- start 2024-04-17T10:38:55.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.963+0000 7f4a15cc1640 1 -- start start 2024-04-17T10:38:55.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.963+0000 7f4a15cc1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49f8003a30 0x7f4a10158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:55.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.963+0000 7f4a15cc1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a101ec0c0 0x7f4a101504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:55.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.963+0000 7f4a15cc1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a1015c5c0 0x7f4a1015ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:55.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.963+0000 7f4a15cc1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1006d270 con 0x7f49f8003a30 2024-04-17T10:38:55.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.963+0000 7f4a15cc1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1006d0f0 con 0x7f4a1015c5c0 2024-04-17T10:38:55.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.963+0000 7f4a15cc1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a101665f0 con 0x7f4a101ec0c0 2024-04-17T10:38:55.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.963+0000 7f4a0f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49f8003a30 0x7f4a10158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:55.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.963+0000 7f4a0f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49f8003a30 0x7f4a10158ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51840/0 (socket says 172.21.15.83:51840) 2024-04-17T10:38:55.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.963+0000 7f4a0f7fe640 1 -- 172.21.15.83:0/1715213657 learned_addr learned my addr 172.21.15.83:0/1715213657 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:55.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.963+0000 7f4a0effd640 1 --2- 172.21.15.83:0/1715213657 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a101ec0c0 0x7f4a101504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:55.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.963+0000 7f4a0f7fe640 1 -- 172.21.15.83:0/1715213657 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a101ec0c0 msgr2=0x7f4a101504e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:55.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.963+0000 7f4a0f7fe640 1 --2- 172.21.15.83:0/1715213657 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a101ec0c0 0x7f4a101504e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.963+0000 7f4a0f7fe640 1 -- 172.21.15.83:0/1715213657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a1015c5c0 msgr2=0x7f4a1015ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:55.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.963+0000 7f4a0ffff640 1 --2- 172.21.15.83:0/1715213657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a1015c5c0 0x7f4a1015ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:55.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.963+0000 7f4a0f7fe640 1 --2- 172.21.15.83:0/1715213657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a1015c5c0 0x7f4a1015ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:55.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.963+0000 7f4a0f7fe640 1 -- 172.21.15.83:0/1715213657 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a10159350 con 0x7f49f8003a30 2024-04-17T10:38:55.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.963+0000 7f4a0f7fe640 1 --2- 172.21.15.83:0/1715213657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49f8003a30 0x7f4a10158ce0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4a0001c6a0 tx=0x7f49f8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:55.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.963+0000 7f49f7fff640 1 -- 172.21.15.83:0/1715213657 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a000bd020 con 0x7f49f8003a30 2024-04-17T10:38:55.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.963+0000 7f49f7fff640 1 -- 172.21.15.83:0/1715213657 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a000aec50 con 0x7f49f8003a30 2024-04-17T10:38:55.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.963+0000 7f4a15cc1640 1 -- 172.21.15.83:0/1715213657 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a10159630 con 0x7f49f8003a30 2024-04-17T10:38:55.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.963+0000 7f49f7fff640 1 -- 172.21.15.83:0/1715213657 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a000b3040 con 0x7f49f8003a30 2024-04-17T10:38:55.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.963+0000 7f4a15cc1640 1 -- 172.21.15.83:0/1715213657 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a10150a20 con 0x7f49f8003a30 2024-04-17T10:38:55.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.963+0000 7f4a15cc1640 1 -- 172.21.15.83:0/1715213657 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a10074f90 con 0x7f49f8003a30 2024-04-17T10:38:55.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.963+0000 7f49f7fff640 1 -- 172.21.15.83:0/1715213657 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4a00004020 con 0x7f49f8003a30 2024-04-17T10:38:55.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.967+0000 7f49f7fff640 1 --2- 172.21.15.83:0/1715213657 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f49fc066f90 0x7f49fc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:55.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.967+0000 7f49f7fff640 1 -- 172.21.15.83:0/1715213657 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f4a0011f0c0 con 0x7f49f8003a30 2024-04-17T10:38:55.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.967+0000 7f4a0effd640 1 --2- 172.21.15.83:0/1715213657 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f49fc066f90 0x7f49fc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:55.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.967+0000 7f4a0effd640 1 --2- 172.21.15.83:0/1715213657 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f49fc066f90 0x7f49fc069450 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4a04001ec0 tx=0x7f4a04001db0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:55.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:55.971+0000 7f49f7fff640 1 -- 172.21.15.83:0/1715213657 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4a000c7020 con 0x7f49f8003a30 2024-04-17T10:38:56.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.127+0000 7f4a15cc1640 1 -- 172.21.15.83:0/1715213657 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4a10158880 con 0x7f49f8003a30 2024-04-17T10:38:56.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.127+0000 7f49f7fff640 1 -- 172.21.15.83:0/1715213657 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4a000ab070 con 0x7f49f8003a30 2024-04-17T10:38:56.124 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:38:56.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.127+0000 7f4a15cc1640 1 -- 172.21.15.83:0/1715213657 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f49fc066f90 msgr2=0x7f49fc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:56.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.127+0000 7f4a15cc1640 1 --2- 172.21.15.83:0/1715213657 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f49fc066f90 0x7f49fc069450 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4a04001ec0 tx=0x7f4a04001db0 comp rx=0 tx=0).stop 2024-04-17T10:38:56.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.127+0000 7f4a15cc1640 1 -- 172.21.15.83:0/1715213657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49f8003a30 msgr2=0x7f4a10158ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:56.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.127+0000 7f4a15cc1640 1 --2- 172.21.15.83:0/1715213657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49f8003a30 0x7f4a10158ce0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4a0001c6a0 tx=0x7f49f8005840 comp rx=0 tx=0).stop 2024-04-17T10:38:56.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.127+0000 7f4a15cc1640 1 -- 172.21.15.83:0/1715213657 shutdown_connections 2024-04-17T10:38:56.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.127+0000 7f4a15cc1640 1 --2- 172.21.15.83:0/1715213657 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f49fc066f90 0x7f49fc069450 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:56.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.127+0000 7f4a15cc1640 1 --2- 172.21.15.83:0/1715213657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a1015c5c0 0x7f4a1015ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:56.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.127+0000 7f4a15cc1640 1 --2- 172.21.15.83:0/1715213657 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a101ec0c0 0x7f4a101504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:56.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.127+0000 7f4a15cc1640 1 --2- 172.21.15.83:0/1715213657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49f8003a30 0x7f4a10158ce0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:56.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.127+0000 7f4a15cc1640 1 -- 172.21.15.83:0/1715213657 >> 172.21.15.83:0/1715213657 conn(0x7f4a100b15b0 msgr2=0x7f4a100ae170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:56.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.131+0000 7f4a15cc1640 1 -- 172.21.15.83:0/1715213657 shutdown_connections 2024-04-17T10:38:56.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.131+0000 7f4a15cc1640 1 -- 172.21.15.83:0/1715213657 wait complete. 2024-04-17T10:38:56.143 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:38:56.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.255+0000 7f6910328640 1 Processor -- start 2024-04-17T10:38:56.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.255+0000 7f6910328640 1 -- start start 2024-04-17T10:38:56.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.255+0000 7f6910328640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6908154de0 0x7f690806ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:56.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.255+0000 7f6910328640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f690806b350 0x7f69080b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:56.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.255+0000 7f6910328640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f690815bd70 con 0x7f69080b4040 2024-04-17T10:38:56.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.255+0000 7f6910328640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69080b4870 con 0x7f6908154de0 2024-04-17T10:38:56.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.255+0000 7f6910328640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f690815bf60 con 0x7f690806b350 2024-04-17T10:38:56.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.255+0000 7f690e0c4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6908154de0 0x7f690806ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:56.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.255+0000 7f690e0c4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6908154de0 0x7f690806ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51856/0 (socket says 172.21.15.83:51856) 2024-04-17T10:38:56.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.255+0000 7f690e0c4640 1 -- 172.21.15.83:0/4057883037 learned_addr learned my addr 172.21.15.83:0/4057883037 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:56.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.255+0000 7f690d8c3640 1 --2- 172.21.15.83:0/4057883037 >> v2:172.21.15.167:3300/0 conn(0x7f690806b350 0x7f69080b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:56.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.255+0000 7f690e0c4640 1 -- 172.21.15.83:0/4057883037 >> v2:172.21.15.167:3300/0 conn(0x7f690806b350 msgr2=0x7f69080b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:56.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.255+0000 7f690e0c4640 1 --2- 172.21.15.83:0/4057883037 >> v2:172.21.15.167:3300/0 conn(0x7f690806b350 0x7f69080b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:56.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.255+0000 7f690e0c4640 1 -- 172.21.15.83:0/4057883037 >> v1:172.21.15.167:6789/0 conn(0x7f69080b4040 legacy=0x7f6908071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:56.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.255+0000 7f690e0c4640 1 -- 172.21.15.83:0/4057883037 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690815afb0 con 0x7f6908154de0 2024-04-17T10:38:56.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.259+0000 7f690e0c4640 1 --2- 172.21.15.83:0/4057883037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6908154de0 0x7f690806ae10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f69080b2480 tx=0x7f690401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97376e9b13defa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:56.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.259+0000 7f690d0c2640 1 -- 172.21.15.83:0/4057883037 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69040b0070 con 0x7f6908154de0 2024-04-17T10:38:56.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.259+0000 7f690d0c2640 1 -- 172.21.15.83:0/4057883037 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69040ab600 con 0x7f6908154de0 2024-04-17T10:38:56.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.259+0000 7f690d0c2640 1 -- 172.21.15.83:0/4057883037 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69040ab9d0 con 0x7f6908154de0 2024-04-17T10:38:56.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.259+0000 7f6910328640 1 -- 172.21.15.83:0/4057883037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6908154de0 msgr2=0x7f690806ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:56.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.259+0000 7f6910328640 1 --2- 172.21.15.83:0/4057883037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6908154de0 0x7f690806ae10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f69080b2480 tx=0x7f690401c4e0 comp rx=0 tx=0).stop 2024-04-17T10:38:56.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.259+0000 7f6910328640 1 -- 172.21.15.83:0/4057883037 shutdown_connections 2024-04-17T10:38:56.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.259+0000 7f6910328640 1 --2- 172.21.15.83:0/4057883037 >> v2:172.21.15.167:3300/0 conn(0x7f690806b350 0x7f69080b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:56.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.259+0000 7f6910328640 1 --2- 172.21.15.83:0/4057883037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6908154de0 0x7f690806ae10 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:56.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.259+0000 7f6910328640 1 -- 172.21.15.83:0/4057883037 >> 172.21.15.83:0/4057883037 conn(0x7f69080b15b0 msgr2=0x7f69080b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:56.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.259+0000 7f6910328640 1 -- 172.21.15.83:0/4057883037 shutdown_connections 2024-04-17T10:38:56.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.259+0000 7f6910328640 1 -- 172.21.15.83:0/4057883037 wait complete. 2024-04-17T10:38:56.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.263+0000 7f6910328640 1 Processor -- start 2024-04-17T10:38:56.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.263+0000 7f6910328640 1 -- start start 2024-04-17T10:38:56.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.263+0000 7f6910328640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f690806b350 0x7f690814fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:56.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.263+0000 7f6910328640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69080b4040 0x7f690814d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:56.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.263+0000 7f6910328640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6908154de0 0x7f690814de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:56.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.263+0000 7f6910328640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69080b44d0 con 0x7f690806b350 2024-04-17T10:38:56.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.263+0000 7f6910328640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69081551c0 con 0x7f69080b4040 2024-04-17T10:38:56.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.263+0000 7f6910328640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69080b4650 con 0x7f6908154de0 2024-04-17T10:38:56.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.263+0000 7f690e0c4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f690806b350 0x7f690814fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:56.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.263+0000 7f690e0c4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f690806b350 0x7f690814fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51862/0 (socket says 172.21.15.83:51862) 2024-04-17T10:38:56.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.263+0000 7f690e0c4640 1 -- 172.21.15.83:0/671964134 learned_addr learned my addr 172.21.15.83:0/671964134 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:56.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.263+0000 7f690e0c4640 1 -- 172.21.15.83:0/671964134 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6908154de0 msgr2=0x7f690814de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:56.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.263+0000 7f690e0c4640 1 --2- 172.21.15.83:0/671964134 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6908154de0 0x7f690814de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:56.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.263+0000 7f690e0c4640 1 -- 172.21.15.83:0/671964134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69080b4040 msgr2=0x7f690814d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:56.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.263+0000 7f690e0c4640 1 --2- 172.21.15.83:0/671964134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69080b4040 0x7f690814d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:56.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.263+0000 7f690e0c4640 1 -- 172.21.15.83:0/671964134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690815a5b0 con 0x7f690806b350 2024-04-17T10:38:56.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.263+0000 7f690e0c4640 1 --2- 172.21.15.83:0/671964134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f690806b350 0x7f690814fb70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f69040ae040 tx=0x7f690401ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:56.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.263+0000 7f68feffd640 1 -- 172.21.15.83:0/671964134 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69040ba020 con 0x7f690806b350 2024-04-17T10:38:56.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.263+0000 7f68feffd640 1 -- 172.21.15.83:0/671964134 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69040b6620 con 0x7f690806b350 2024-04-17T10:38:56.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.263+0000 7f6910328640 1 -- 172.21.15.83:0/671964134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6908159750 con 0x7f690806b350 2024-04-17T10:38:56.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.263+0000 7f68feffd640 1 -- 172.21.15.83:0/671964134 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69040b0040 con 0x7f690806b350 2024-04-17T10:38:56.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.263+0000 7f6910328640 1 -- 172.21.15.83:0/671964134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6908159d10 con 0x7f690806b350 2024-04-17T10:38:56.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.263+0000 7f6910328640 1 -- 172.21.15.83:0/671964134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6908074f90 con 0x7f690806b350 2024-04-17T10:38:56.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.263+0000 7f68feffd640 1 -- 172.21.15.83:0/671964134 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f69040bd020 con 0x7f690806b350 2024-04-17T10:38:56.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.263+0000 7f68feffd640 1 --2- 172.21.15.83:0/671964134 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f68ec066f60 0x7f68ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:56.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.263+0000 7f68feffd640 1 -- 172.21.15.83:0/671964134 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f690411f360 con 0x7f690806b350 2024-04-17T10:38:56.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.263+0000 7f690d8c3640 1 --2- 172.21.15.83:0/671964134 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f68ec066f60 0x7f68ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:56.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.267+0000 7f690d8c3640 1 --2- 172.21.15.83:0/671964134 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f68ec066f60 0x7f68ec069420 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6908158cf0 tx=0x7f68f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:56.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.267+0000 7f68feffd640 1 -- 172.21.15.83:0/671964134 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f69040c4030 con 0x7f690806b350 2024-04-17T10:38:56.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.427+0000 7f6910328640 1 -- 172.21.15.83:0/671964134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f69080ae840 con 0x7f690806b350 2024-04-17T10:38:56.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.427+0000 7f68feffd640 1 -- 172.21.15.83:0/671964134 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f69040160d0 con 0x7f690806b350 2024-04-17T10:38:56.425 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-17T10:38:56.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.431+0000 7f6910328640 1 -- 172.21.15.83:0/671964134 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f68ec066f60 msgr2=0x7f68ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:56.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.431+0000 7f6910328640 1 --2- 172.21.15.83:0/671964134 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f68ec066f60 0x7f68ec069420 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6908158cf0 tx=0x7f68f8008040 comp rx=0 tx=0).stop 2024-04-17T10:38:56.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.431+0000 7f6910328640 1 -- 172.21.15.83:0/671964134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f690806b350 msgr2=0x7f690814fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:56.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.431+0000 7f6910328640 1 --2- 172.21.15.83:0/671964134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f690806b350 0x7f690814fb70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f69040ae040 tx=0x7f690401ad20 comp rx=0 tx=0).stop 2024-04-17T10:38:56.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.431+0000 7f6910328640 1 -- 172.21.15.83:0/671964134 shutdown_connections 2024-04-17T10:38:56.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.431+0000 7f6910328640 1 --2- 172.21.15.83:0/671964134 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f68ec066f60 0x7f68ec069420 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:56.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.431+0000 7f6910328640 1 --2- 172.21.15.83:0/671964134 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6908154de0 0x7f690814de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:56.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.431+0000 7f6910328640 1 --2- 172.21.15.83:0/671964134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69080b4040 0x7f690814d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:56.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.431+0000 7f6910328640 1 --2- 172.21.15.83:0/671964134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f690806b350 0x7f690814fb70 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:56.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.431+0000 7f6910328640 1 -- 172.21.15.83:0/671964134 >> 172.21.15.83:0/671964134 conn(0x7f69080b15b0 msgr2=0x7f6908153a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:56.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.431+0000 7f6910328640 1 -- 172.21.15.83:0/671964134 shutdown_connections 2024-04-17T10:38:56.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.431+0000 7f6910328640 1 -- 172.21.15.83:0/671964134 wait complete. 2024-04-17T10:38:56.442 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2024-04-17T10:38:56.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:56.443 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-17T10:38:56.454 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:38:56.454 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-17T10:38:56.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:56.454 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-17T10:38:56.499 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:38:56.499 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:38:56.506 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:38:56.506 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:38:56.552 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:38:56.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.679+0000 7f14ff9d9640 1 Processor -- start 2024-04-17T10:38:56.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.679+0000 7f14ff9d9640 1 -- start start 2024-04-17T10:38:56.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.679+0000 7f14ff9d9640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f14f80d2730 0x7f14f80ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:56.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.679+0000 7f14ff9d9640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14f80d18c0 con 0x7f14f806c750 2024-04-17T10:38:56.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.679+0000 7f14ff9d9640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14f80d1b60 con 0x7f14f815ddd0 2024-04-17T10:38:56.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.679+0000 7f14ff9d9640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14f80d1e40 con 0x7f14f80d2730 2024-04-17T10:38:56.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.679+0000 7f14fdf76640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f14f806c750 0x7f14f80ceb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40146/0 (socket says 172.21.15.83:40146) 2024-04-17T10:38:56.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.679+0000 7f14fdf76640 1 -- 172.21.15.83:0/2558854003 learned_addr learned my addr 172.21.15.83:0/2558854003 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:56.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.679+0000 7f14fcf74640 1 --2- 172.21.15.83:0/2558854003 >> v2:172.21.15.83:3300/0 conn(0x7f14f80d2730 0x7f14f80ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:56.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.679+0000 7f14f7fff640 1 -- 172.21.15.83:0/2558854003 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1940948816 0 0) 0x7f14f80d18c0 con 0x7f14f806c750 2024-04-17T10:38:56.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.679+0000 7f14f7fff640 1 -- 172.21.15.83:0/2558854003 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14e0003530 con 0x7f14f806c750 2024-04-17T10:38:56.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.679+0000 7f14fcf74640 1 -- 172.21.15.83:0/2558854003 >> v1:172.21.15.167:6789/0 conn(0x7f14f815ddd0 legacy=0x7f14f80d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:56.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.679+0000 7f14fcf74640 1 -- 172.21.15.83:0/2558854003 >> v1:172.21.15.83:6789/0 conn(0x7f14f806c750 legacy=0x7f14f80ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:56.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.679+0000 7f14fcf74640 1 -- 172.21.15.83:0/2558854003 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f806b220 con 0x7f14f80d2730 2024-04-17T10:38:56.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.683+0000 7f14fcf74640 1 --2- 172.21.15.83:0/2558854003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14f80d2730 0x7f14f80ce5e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0097c0 tx=0x7f14ec01c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33976e6ee749714e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:56.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.683+0000 7f14f7fff640 1 -- 172.21.15.83:0/2558854003 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec0b0070 con 0x7f14f80d2730 2024-04-17T10:38:56.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.683+0000 7f14f7fff640 1 -- 172.21.15.83:0/2558854003 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ec0ab600 con 0x7f14f80d2730 2024-04-17T10:38:56.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.683+0000 7f14f7fff640 1 -- 172.21.15.83:0/2558854003 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec0ab9d0 con 0x7f14f80d2730 2024-04-17T10:38:56.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.683+0000 7f14ff9d9640 1 -- 172.21.15.83:0/2558854003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14f80d2730 msgr2=0x7f14f80ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:56.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.683+0000 7f14ff9d9640 1 --2- 172.21.15.83:0/2558854003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14f80d2730 0x7f14f80ce5e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0097c0 tx=0x7f14ec01c320 comp rx=0 tx=0).stop 2024-04-17T10:38:56.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.683+0000 7f14ff9d9640 1 -- 172.21.15.83:0/2558854003 shutdown_connections 2024-04-17T10:38:56.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.683+0000 7f14ff9d9640 1 --2- 172.21.15.83:0/2558854003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14f80d2730 0x7f14f80ce5e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:56.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.683+0000 7f14ff9d9640 1 -- 172.21.15.83:0/2558854003 >> 172.21.15.83:0/2558854003 conn(0x7f14f80d3a40 msgr2=0x7f14f80d3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:56.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.683+0000 7f14ff9d9640 1 -- 172.21.15.83:0/2558854003 shutdown_connections 2024-04-17T10:38:56.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.683+0000 7f14ff9d9640 1 -- 172.21.15.83:0/2558854003 wait complete. 2024-04-17T10:38:56.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.683+0000 7f14ff9d9640 1 Processor -- start 2024-04-17T10:38:56.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.687+0000 7f14ff9d9640 1 -- start start 2024-04-17T10:38:56.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.687+0000 7f14ff9d9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14f806c750 0x7f14f80b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:56.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.687+0000 7f14ff9d9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14f80d2730 0x7f14f814c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:56.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.687+0000 7f14ff9d9640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14f815ddd0 0x7f14f815e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:56.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.687+0000 7f14ff9d9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f816bbe0 con 0x7f14f806c750 2024-04-17T10:38:56.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.687+0000 7f14ff9d9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f816ba60 con 0x7f14f80d2730 2024-04-17T10:38:56.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.687+0000 7f14ff9d9640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f816bd60 con 0x7f14f815ddd0 2024-04-17T10:38:56.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.687+0000 7f14fd775640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14f806c750 0x7f14f80b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:56.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.687+0000 7f14fd775640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14f806c750 0x7f14f80b5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51886/0 (socket says 172.21.15.83:51886) 2024-04-17T10:38:56.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.687+0000 7f14fd775640 1 -- 172.21.15.83:0/3896324081 learned_addr learned my addr 172.21.15.83:0/3896324081 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:56.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.687+0000 7f14fd775640 1 -- 172.21.15.83:0/3896324081 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14f815ddd0 msgr2=0x7f14f815e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:56.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.687+0000 7f14fd775640 1 --2- 172.21.15.83:0/3896324081 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14f815ddd0 0x7f14f815e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:56.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.687+0000 7f14fd775640 1 -- 172.21.15.83:0/3896324081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14f80d2730 msgr2=0x7f14f814c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:56.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.687+0000 7f14fd775640 1 --2- 172.21.15.83:0/3896324081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14f80d2730 0x7f14f814c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:56.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.687+0000 7f14fd775640 1 -- 172.21.15.83:0/3896324081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f80b6380 con 0x7f14f806c750 2024-04-17T10:38:56.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.687+0000 7f14fd775640 1 --2- 172.21.15.83:0/3896324081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14f806c750 0x7f14f80b5e40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f14f000c790 tx=0x7f14f000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:56.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.687+0000 7f14f5ffb640 1 -- 172.21.15.83:0/3896324081 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f0016020 con 0x7f14f806c750 2024-04-17T10:38:56.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.687+0000 7f14f5ffb640 1 -- 172.21.15.83:0/3896324081 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f0006d60 con 0x7f14f806c750 2024-04-17T10:38:56.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.687+0000 7f14ff9d9640 1 -- 172.21.15.83:0/3896324081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14f80b8ed0 con 0x7f14f806c750 2024-04-17T10:38:56.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.687+0000 7f14f5ffb640 1 -- 172.21.15.83:0/3896324081 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f0003bc0 con 0x7f14f806c750 2024-04-17T10:38:56.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.687+0000 7f14ff9d9640 1 -- 172.21.15.83:0/3896324081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14f80b9490 con 0x7f14f806c750 2024-04-17T10:38:56.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.687+0000 7f14ff9d9640 1 -- 172.21.15.83:0/3896324081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14f8074f90 con 0x7f14f806c750 2024-04-17T10:38:56.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.687+0000 7f14f5ffb640 1 -- 172.21.15.83:0/3896324081 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f14f000a850 con 0x7f14f806c750 2024-04-17T10:38:56.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.687+0000 7f14f5ffb640 1 --2- 172.21.15.83:0/3896324081 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f14e4066f60 0x7f14e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:56.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.687+0000 7f14f5ffb640 1 -- 172.21.15.83:0/3896324081 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f14f0084da0 con 0x7f14f806c750 2024-04-17T10:38:56.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.687+0000 7f14fcf74640 1 --2- 172.21.15.83:0/3896324081 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f14e4066f60 0x7f14e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:56.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.687+0000 7f14fcf74640 1 --2- 172.21.15.83:0/3896324081 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f14e4066f60 0x7f14e4069420 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f14f808def0 tx=0x7f14ec004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:56.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.691+0000 7f14f5ffb640 1 -- 172.21.15.83:0/3896324081 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f14f0020020 con 0x7f14f806c750 2024-04-17T10:38:56.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.839+0000 7f14ff9d9640 1 -- 172.21.15.83:0/3896324081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14f80d2cb0 con 0x7f14f806c750 2024-04-17T10:38:56.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.843+0000 7f14f5ffb640 1 -- 172.21.15.83:0/3896324081 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16782 (secure 0 0 0) 0x7f14f0055dc0 con 0x7f14f806c750 2024-04-17T10:38:56.840 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:56.840 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":22,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:38:53.385002+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:38:10.464490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T10:38:11.043134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:38:56.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.843+0000 7f14ff9d9640 1 -- 172.21.15.83:0/3896324081 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f14e4066f60 msgr2=0x7f14e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:56.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.843+0000 7f14ff9d9640 1 --2- 172.21.15.83:0/3896324081 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f14e4066f60 0x7f14e4069420 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f14f808def0 tx=0x7f14ec004740 comp rx=0 tx=0).stop 2024-04-17T10:38:56.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.843+0000 7f14ff9d9640 1 -- 172.21.15.83:0/3896324081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14f806c750 msgr2=0x7f14f80b5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:56.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.843+0000 7f14ff9d9640 1 --2- 172.21.15.83:0/3896324081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14f806c750 0x7f14f80b5e40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f14f000c790 tx=0x7f14f000cc50 comp rx=0 tx=0).stop 2024-04-17T10:38:56.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.843+0000 7f14ff9d9640 1 -- 172.21.15.83:0/3896324081 shutdown_connections 2024-04-17T10:38:56.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.843+0000 7f14ff9d9640 1 --2- 172.21.15.83:0/3896324081 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f14e4066f60 0x7f14e4069420 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:56.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.843+0000 7f14ff9d9640 1 --2- 172.21.15.83:0/3896324081 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14f815ddd0 0x7f14f815e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:56.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.843+0000 7f14ff9d9640 1 --2- 172.21.15.83:0/3896324081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14f80d2730 0x7f14f814c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:56.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.843+0000 7f14ff9d9640 1 --2- 172.21.15.83:0/3896324081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14f806c750 0x7f14f80b5e40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:56.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.843+0000 7f14ff9d9640 1 -- 172.21.15.83:0/3896324081 >> 172.21.15.83:0/3896324081 conn(0x7f14f80d3a40 msgr2=0x7f14f815fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:56.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.843+0000 7f14ff9d9640 1 -- 172.21.15.83:0/3896324081 shutdown_connections 2024-04-17T10:38:56.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.843+0000 7f14ff9d9640 1 -- 172.21.15.83:0/3896324081 wait complete. 2024-04-17T10:38:56.857 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:38:10.464490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:38:11.043134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:38:56.858 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:38:56.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.987+0000 7f1293125640 1 Processor -- start 2024-04-17T10:38:56.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.987+0000 7f1293125640 1 -- start start 2024-04-17T10:38:56.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.987+0000 7f1293125640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f128c071bb0 0x7f128c0cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:56.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.987+0000 7f1293125640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f128c0aff10 0x7f128c0cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:56.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.987+0000 7f1293125640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f128c0b3a70 con 0x7f128c15b760 2024-04-17T10:38:56.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.987+0000 7f1293125640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f128c06abe0 con 0x7f128c0aff10 2024-04-17T10:38:56.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.987+0000 7f1293125640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f128c0b3d50 con 0x7f128c071bb0 2024-04-17T10:38:56.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.987+0000 7f12916c2640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f128c0aff10 0x7f128c0cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:56.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.987+0000 7f128bfff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f128c071bb0 0x7f128c0cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:56.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.987+0000 7f1290ec1640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f128c15b760 0x7f128c0714a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:55654/0 (socket says 172.21.15.83:55654) 2024-04-17T10:38:56.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.987+0000 7f12916c2640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f128c0aff10 0x7f128c0cefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:44322/0 (socket says 172.21.15.83:44322) 2024-04-17T10:38:56.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.987+0000 7f1290ec1640 1 -- 172.21.15.83:0/1473073159 learned_addr learned my addr 172.21.15.83:0/1473073159 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:56.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.987+0000 7f12916c2640 1 -- 172.21.15.83:0/1473073159 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f128c071bb0 msgr2=0x7f128c0cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:56.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.987+0000 7f12916c2640 1 --2- 172.21.15.83:0/1473073159 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f128c071bb0 0x7f128c0cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:56.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.987+0000 7f12916c2640 1 -- 172.21.15.83:0/1473073159 >> v1:172.21.15.167:6789/0 conn(0x7f128c15b760 legacy=0x7f128c0714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:56.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.987+0000 7f12916c2640 1 -- 172.21.15.83:0/1473073159 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f128c0b4030 con 0x7f128c0aff10 2024-04-17T10:38:56.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.991+0000 7f12916c2640 1 --2- 172.21.15.83:0/1473073159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f128c0aff10 0x7f128c0cefd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1284009d90 tx=0x7f128401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39286b9434bcd45b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:56.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.991+0000 7f128b7fe640 1 -- 172.21.15.83:0/1473073159 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12840b2070 con 0x7f128c0aff10 2024-04-17T10:38:56.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.991+0000 7f128b7fe640 1 -- 172.21.15.83:0/1473073159 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12840a5890 con 0x7f128c0aff10 2024-04-17T10:38:56.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.991+0000 7f128b7fe640 1 -- 172.21.15.83:0/1473073159 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12840a5c60 con 0x7f128c0aff10 2024-04-17T10:38:56.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.991+0000 7f1293125640 1 -- 172.21.15.83:0/1473073159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f128c0aff10 msgr2=0x7f128c0cefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:56.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.991+0000 7f1293125640 1 --2- 172.21.15.83:0/1473073159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f128c0aff10 0x7f128c0cefd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1284009d90 tx=0x7f128401c970 comp rx=0 tx=0).stop 2024-04-17T10:38:56.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.991+0000 7f1293125640 1 -- 172.21.15.83:0/1473073159 shutdown_connections 2024-04-17T10:38:56.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.991+0000 7f1293125640 1 --2- 172.21.15.83:0/1473073159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f128c0aff10 0x7f128c0cefd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:56.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.991+0000 7f1293125640 1 --2- 172.21.15.83:0/1473073159 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f128c071bb0 0x7f128c0cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:56.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.991+0000 7f1293125640 1 -- 172.21.15.83:0/1473073159 >> 172.21.15.83:0/1473073159 conn(0x7f128c0b23f0 msgr2=0x7f128c0b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:56.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.991+0000 7f1293125640 1 -- 172.21.15.83:0/1473073159 shutdown_connections 2024-04-17T10:38:56.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.991+0000 7f1293125640 1 -- 172.21.15.83:0/1473073159 wait complete. 2024-04-17T10:38:56.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.995+0000 7f1293125640 1 Processor -- start 2024-04-17T10:38:56.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.995+0000 7f1293125640 1 -- start start 2024-04-17T10:38:56.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.995+0000 7f1293125640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f128c071bb0 0x7f128c1433b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:56.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.995+0000 7f1293125640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f128c0aff10 0x7f128c1532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:56.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.995+0000 7f1293125640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f128c15b760 0x7f128c14f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:56.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.995+0000 7f1293125640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c15bcc0 con 0x7f128c0aff10 2024-04-17T10:38:56.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.995+0000 7f1293125640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c15bb40 con 0x7f128c071bb0 2024-04-17T10:38:56.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.995+0000 7f1293125640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f128c1687c0 con 0x7f128c15b760 2024-04-17T10:38:56.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.995+0000 7f128bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f128c0aff10 0x7f128c1532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:56.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.995+0000 7f128bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f128c0aff10 0x7f128c1532e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50030/0 (socket says 172.21.15.83:50030) 2024-04-17T10:38:56.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.995+0000 7f128bfff640 1 -- 172.21.15.83:0/2041332538 learned_addr learned my addr 172.21.15.83:0/2041332538 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:56.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.995+0000 7f12916c2640 1 --2- 172.21.15.83:0/2041332538 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f128c15b760 0x7f128c14f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:56.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.995+0000 7f1290ec1640 1 --2- 172.21.15.83:0/2041332538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f128c071bb0 0x7f128c1433b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:56.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.995+0000 7f128bfff640 1 -- 172.21.15.83:0/2041332538 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f128c15b760 msgr2=0x7f128c14f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:56.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.995+0000 7f128bfff640 1 --2- 172.21.15.83:0/2041332538 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f128c15b760 0x7f128c14f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:56.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.995+0000 7f128bfff640 1 -- 172.21.15.83:0/2041332538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f128c071bb0 msgr2=0x7f128c1433b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:56.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.995+0000 7f128bfff640 1 --2- 172.21.15.83:0/2041332538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f128c071bb0 0x7f128c1433b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:56.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.995+0000 7f128bfff640 1 -- 172.21.15.83:0/2041332538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f128c152240 con 0x7f128c0aff10 2024-04-17T10:38:56.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.995+0000 7f1290ec1640 1 --2- 172.21.15.83:0/2041332538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f128c071bb0 0x7f128c1433b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:56.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.995+0000 7f128bfff640 1 --2- 172.21.15.83:0/2041332538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f128c0aff10 0x7f128c1532e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f127800e960 tx=0x7f127800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:56.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.995+0000 7f12897fa640 1 -- 172.21.15.83:0/2041332538 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1278017070 con 0x7f128c0aff10 2024-04-17T10:38:56.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.995+0000 7f12897fa640 1 -- 172.21.15.83:0/2041332538 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f127800ca00 con 0x7f128c0aff10 2024-04-17T10:38:56.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.995+0000 7f1293125640 1 -- 172.21.15.83:0/2041332538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f128c152520 con 0x7f128c0aff10 2024-04-17T10:38:56.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.995+0000 7f12897fa640 1 -- 172.21.15.83:0/2041332538 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127800cdb0 con 0x7f128c0aff10 2024-04-17T10:38:56.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.995+0000 7f1293125640 1 -- 172.21.15.83:0/2041332538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f128c152ae0 con 0x7f128c0aff10 2024-04-17T10:38:56.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.995+0000 7f1293125640 1 -- 172.21.15.83:0/2041332538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f128c074f90 con 0x7f128c0aff10 2024-04-17T10:38:56.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.995+0000 7f12897fa640 1 -- 172.21.15.83:0/2041332538 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f127801a070 con 0x7f128c0aff10 2024-04-17T10:38:56.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.999+0000 7f12897fa640 1 --2- 172.21.15.83:0/2041332538 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1270066f60 0x7f1270069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:56.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.999+0000 7f12897fa640 1 -- 172.21.15.83:0/2041332538 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f1278085430 con 0x7f128c0aff10 2024-04-17T10:38:56.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.999+0000 7f1290ec1640 1 --2- 172.21.15.83:0/2041332538 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1270066f60 0x7f1270069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:56.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:56.999+0000 7f1290ec1640 1 --2- 172.21.15.83:0/2041332538 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1270066f60 0x7f1270069420 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f128c14ea00 tx=0x7f1280009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:56.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.003+0000 7f12897fa640 1 -- 172.21.15.83:0/2041332538 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1278056450 con 0x7f128c0aff10 2024-04-17T10:38:57.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.159+0000 7f1293125640 1 -- 172.21.15.83:0/2041332538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f128c15f280 con 0x7f128c0aff10 2024-04-17T10:38:57.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.159+0000 7f12897fa640 1 -- 172.21.15.83:0/2041332538 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1278055d50 con 0x7f128c0aff10 2024-04-17T10:38:57.159 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:38:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.163+0000 7f1293125640 1 -- 172.21.15.83:0/2041332538 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1270066f60 msgr2=0x7f1270069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.163+0000 7f1293125640 1 --2- 172.21.15.83:0/2041332538 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1270066f60 0x7f1270069420 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f128c14ea00 tx=0x7f1280009040 comp rx=0 tx=0).stop 2024-04-17T10:38:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.163+0000 7f1293125640 1 -- 172.21.15.83:0/2041332538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f128c0aff10 msgr2=0x7f128c1532e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.163+0000 7f1293125640 1 --2- 172.21.15.83:0/2041332538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f128c0aff10 0x7f128c1532e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f127800e960 tx=0x7f127800ee20 comp rx=0 tx=0).stop 2024-04-17T10:38:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.163+0000 7f1293125640 1 -- 172.21.15.83:0/2041332538 shutdown_connections 2024-04-17T10:38:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.163+0000 7f1293125640 1 --2- 172.21.15.83:0/2041332538 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1270066f60 0x7f1270069420 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.163+0000 7f1293125640 1 --2- 172.21.15.83:0/2041332538 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f128c15b760 0x7f128c14f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.163+0000 7f1293125640 1 --2- 172.21.15.83:0/2041332538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f128c0aff10 0x7f128c1532e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.163+0000 7f1293125640 1 --2- 172.21.15.83:0/2041332538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f128c071bb0 0x7f128c1433b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.163+0000 7f1293125640 1 -- 172.21.15.83:0/2041332538 >> 172.21.15.83:0/2041332538 conn(0x7f128c0b23f0 msgr2=0x7f128c06aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.163+0000 7f1293125640 1 -- 172.21.15.83:0/2041332538 shutdown_connections 2024-04-17T10:38:57.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.163+0000 7f1293125640 1 -- 172.21.15.83:0/2041332538 wait complete. 2024-04-17T10:38:57.178 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:38:57.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.303+0000 7fecbd635640 1 Processor -- start 2024-04-17T10:38:57.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.307+0000 7fecbd635640 1 -- start start 2024-04-17T10:38:57.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.307+0000 7fecbd635640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fecb8154de0 0x7fecb806ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:57.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.307+0000 7fecbd635640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fecb806b350 0x7fecb80b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:57.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.307+0000 7fecbd635640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb815bd70 con 0x7fecb80b4040 2024-04-17T10:38:57.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.307+0000 7fecbd635640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecb80b4870 con 0x7fecb8154de0 2024-04-17T10:38:57.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.307+0000 7fecbd635640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecb815bf60 con 0x7fecb806b350 2024-04-17T10:38:57.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.307+0000 7fecb6ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fecb8154de0 0x7fecb806ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:57.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.307+0000 7fecb6ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fecb8154de0 0x7fecb806ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50034/0 (socket says 172.21.15.83:50034) 2024-04-17T10:38:57.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.307+0000 7fecb6ffd640 1 -- 172.21.15.83:0/950950580 learned_addr learned my addr 172.21.15.83:0/950950580 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:57.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.307+0000 7fecaffff640 1 --2- 172.21.15.83:0/950950580 >> v2:172.21.15.167:3300/0 conn(0x7fecb806b350 0x7fecb80b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:57.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.307+0000 7fecb6ffd640 1 -- 172.21.15.83:0/950950580 >> v2:172.21.15.167:3300/0 conn(0x7fecb806b350 msgr2=0x7fecb80b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:57.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.307+0000 7fecb6ffd640 1 --2- 172.21.15.83:0/950950580 >> v2:172.21.15.167:3300/0 conn(0x7fecb806b350 0x7fecb80b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:57.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.307+0000 7fecb6ffd640 1 -- 172.21.15.83:0/950950580 >> v1:172.21.15.83:6789/0 conn(0x7fecb80b4040 legacy=0x7fecb8071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:57.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.307+0000 7fecb67fc640 1 -- 172.21.15.83:0/950950580 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302860063 0 0) 0x7fecb815bd70 con 0x7fecb80b4040 2024-04-17T10:38:57.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.307+0000 7fecb6ffd640 1 -- 172.21.15.83:0/950950580 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb815afb0 con 0x7fecb8154de0 2024-04-17T10:38:57.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.311+0000 7fecb6ffd640 1 --2- 172.21.15.83:0/950950580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecb8154de0 0x7fecb806ae10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fecb80b2480 tx=0x7feca801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e43f52f861d9585 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:57.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.311+0000 7fecb67fc640 1 -- 172.21.15.83:0/950950580 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca80b0070 con 0x7fecb8154de0 2024-04-17T10:38:57.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.311+0000 7fecb67fc640 1 -- 172.21.15.83:0/950950580 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feca80ab600 con 0x7fecb8154de0 2024-04-17T10:38:57.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.311+0000 7fecb67fc640 1 -- 172.21.15.83:0/950950580 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca80ab9d0 con 0x7fecb8154de0 2024-04-17T10:38:57.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.311+0000 7fecbd635640 1 -- 172.21.15.83:0/950950580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecb8154de0 msgr2=0x7fecb806ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:57.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.311+0000 7fecbd635640 1 --2- 172.21.15.83:0/950950580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecb8154de0 0x7fecb806ae10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fecb80b2480 tx=0x7feca801c520 comp rx=0 tx=0).stop 2024-04-17T10:38:57.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.311+0000 7fecbd635640 1 -- 172.21.15.83:0/950950580 shutdown_connections 2024-04-17T10:38:57.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.311+0000 7fecbd635640 1 --2- 172.21.15.83:0/950950580 >> v2:172.21.15.167:3300/0 conn(0x7fecb806b350 0x7fecb80b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:57.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.311+0000 7fecbd635640 1 --2- 172.21.15.83:0/950950580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecb8154de0 0x7fecb806ae10 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:57.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.311+0000 7fecbd635640 1 -- 172.21.15.83:0/950950580 >> 172.21.15.83:0/950950580 conn(0x7fecb80b15b0 msgr2=0x7fecb80b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:57.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.311+0000 7fecbd635640 1 -- 172.21.15.83:0/950950580 shutdown_connections 2024-04-17T10:38:57.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.311+0000 7fecbd635640 1 -- 172.21.15.83:0/950950580 wait complete. 2024-04-17T10:38:57.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.311+0000 7fecbd635640 1 Processor -- start 2024-04-17T10:38:57.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.311+0000 7fecbd635640 1 -- start start 2024-04-17T10:38:57.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.311+0000 7fecbd635640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecb806b350 0x7fecb814edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:57.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.311+0000 7fecbd635640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecb80b4040 0x7fecb815ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:57.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.311+0000 7fecbd635640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecb8154de0 0x7fecb814d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:57.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.311+0000 7fecbd635640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb80b4490 con 0x7fecb80b4040 2024-04-17T10:38:57.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.311+0000 7fecbd635640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb81551c0 con 0x7fecb806b350 2024-04-17T10:38:57.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.311+0000 7fecbd635640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb80b4610 con 0x7fecb8154de0 2024-04-17T10:38:57.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.311+0000 7fecaffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecb80b4040 0x7fecb815ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:57.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.311+0000 7fecaffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecb80b4040 0x7fecb815ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50038/0 (socket says 172.21.15.83:50038) 2024-04-17T10:38:57.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.311+0000 7fecaffff640 1 -- 172.21.15.83:0/3943640044 learned_addr learned my addr 172.21.15.83:0/3943640044 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:57.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.311+0000 7fecb6ffd640 1 --2- 172.21.15.83:0/3943640044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecb806b350 0x7fecb814edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:57.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.311+0000 7fecaffff640 1 -- 172.21.15.83:0/3943640044 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecb8154de0 msgr2=0x7fecb814d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:57.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.311+0000 7fecaffff640 1 --2- 172.21.15.83:0/3943640044 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecb8154de0 0x7fecb814d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:57.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.311+0000 7fecaffff640 1 -- 172.21.15.83:0/3943640044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecb806b350 msgr2=0x7fecb814edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:57.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.311+0000 7fecaffff640 1 --2- 172.21.15.83:0/3943640044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecb806b350 0x7fecb814edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:57.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.311+0000 7fecaffff640 1 -- 172.21.15.83:0/3943640044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb81553d0 con 0x7fecb80b4040 2024-04-17T10:38:57.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.311+0000 7fecb6ffd640 1 --2- 172.21.15.83:0/3943640044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecb806b350 0x7fecb814edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:57.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.311+0000 7fecaffff640 1 --2- 172.21.15.83:0/3943640044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecb80b4040 0x7fecb815ca10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7feca000d5f0 tx=0x7feca000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:57.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.315+0000 7fecaf7fe640 1 -- 172.21.15.83:0/3943640044 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca00125f0 con 0x7fecb80b4040 2024-04-17T10:38:57.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.315+0000 7fecaf7fe640 1 -- 172.21.15.83:0/3943640044 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feca0019070 con 0x7fecb80b4040 2024-04-17T10:38:57.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.315+0000 7fecaf7fe640 1 -- 172.21.15.83:0/3943640044 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca00129b0 con 0x7fecb80b4040 2024-04-17T10:38:57.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.315+0000 7fecbd635640 1 -- 172.21.15.83:0/3943640044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb8155690 con 0x7fecb80b4040 2024-04-17T10:38:57.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.315+0000 7fecbd635640 1 -- 172.21.15.83:0/3943640044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb8155c50 con 0x7fecb80b4040 2024-04-17T10:38:57.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.315+0000 7fecbd635640 1 -- 172.21.15.83:0/3943640044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecb8074f90 con 0x7fecb80b4040 2024-04-17T10:38:57.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.315+0000 7fecaf7fe640 1 -- 172.21.15.83:0/3943640044 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feca0012b50 con 0x7fecb80b4040 2024-04-17T10:38:57.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.315+0000 7fecaf7fe640 1 --2- 172.21.15.83:0/3943640044 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fec98066f60 0x7fec98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:57.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.315+0000 7fecaf7fe640 1 -- 172.21.15.83:0/3943640044 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7feca0085460 con 0x7fecb80b4040 2024-04-17T10:38:57.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.315+0000 7fecb6ffd640 1 --2- 172.21.15.83:0/3943640044 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fec98066f60 0x7fec98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:57.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.315+0000 7fecb6ffd640 1 --2- 172.21.15.83:0/3943640044 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fec98066f60 0x7fec98069420 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7feca80ad000 tx=0x7feca80a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:57.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.319+0000 7fecaf7fe640 1 -- 172.21.15.83:0/3943640044 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feca0056480 con 0x7fecb80b4040 2024-04-17T10:38:57.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.475+0000 7fecbd635640 1 -- 172.21.15.83:0/3943640044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fecb814fc50 con 0x7fecb80b4040 2024-04-17T10:38:57.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.475+0000 7fecaf7fe640 1 -- 172.21.15.83:0/3943640044 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7feca0055d80 con 0x7fecb80b4040 2024-04-17T10:38:57.471 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:38:57.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.475+0000 7fecbd635640 1 -- 172.21.15.83:0/3943640044 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fec98066f60 msgr2=0x7fec98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:57.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.475+0000 7fecbd635640 1 --2- 172.21.15.83:0/3943640044 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fec98066f60 0x7fec98069420 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7feca80ad000 tx=0x7feca80a6000 comp rx=0 tx=0).stop 2024-04-17T10:38:57.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.475+0000 7fecbd635640 1 -- 172.21.15.83:0/3943640044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecb80b4040 msgr2=0x7fecb815ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:57.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.475+0000 7fecbd635640 1 --2- 172.21.15.83:0/3943640044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecb80b4040 0x7fecb815ca10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7feca000d5f0 tx=0x7feca000dab0 comp rx=0 tx=0).stop 2024-04-17T10:38:57.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.475+0000 7fecbd635640 1 -- 172.21.15.83:0/3943640044 shutdown_connections 2024-04-17T10:38:57.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.475+0000 7fecbd635640 1 --2- 172.21.15.83:0/3943640044 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fec98066f60 0x7fec98069420 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:57.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.475+0000 7fecbd635640 1 --2- 172.21.15.83:0/3943640044 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecb8154de0 0x7fecb814d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:57.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.475+0000 7fecbd635640 1 --2- 172.21.15.83:0/3943640044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecb80b4040 0x7fecb815ca10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:57.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.475+0000 7fecbd635640 1 --2- 172.21.15.83:0/3943640044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecb806b350 0x7fecb814edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:57.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.475+0000 7fecbd635640 1 -- 172.21.15.83:0/3943640044 >> 172.21.15.83:0/3943640044 conn(0x7fecb80b15b0 msgr2=0x7fecb80cee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:57.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.475+0000 7fecbd635640 1 -- 172.21.15.83:0/3943640044 shutdown_connections 2024-04-17T10:38:57.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.475+0000 7fecbd635640 1 -- 172.21.15.83:0/3943640044 wait complete. 2024-04-17T10:38:57.489 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:38:57.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.603+0000 7fdd345d7640 1 Processor -- start 2024-04-17T10:38:57.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.603+0000 7fdd345d7640 1 -- start start 2024-04-17T10:38:57.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.603+0000 7fdd345d7640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdd2c15b6e0 0x7fdd2c071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:57.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.603+0000 7fdd345d7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd2c15bac0 0x7fdd2c0cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:57.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.603+0000 7fdd345d7640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdd2c070470 0x7fdd2c06ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:57.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.603+0000 7fdd345d7640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd2c06c1f0 con 0x7fdd2c15b6e0 2024-04-17T10:38:57.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.603+0000 7fdd345d7640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd2c0cdd30 con 0x7fdd2c070470 2024-04-17T10:38:57.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.603+0000 7fdd345d7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c070a10 con 0x7fdd2c15bac0 2024-04-17T10:38:57.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.603+0000 7fdd32373640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdd2c15b6e0 0x7fdd2c071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:57.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.603+0000 7fdd32373640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdd2c15b6e0 0x7fdd2c071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50052/0 (socket says 172.21.15.83:50052) 2024-04-17T10:38:57.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.603+0000 7fdd32373640 1 -- 172.21.15.83:0/801319737 learned_addr learned my addr 172.21.15.83:0/801319737 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:57.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.607+0000 7fdd32373640 1 -- 172.21.15.83:0/801319737 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd2c15bac0 msgr2=0x7fdd2c0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:57.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.607+0000 7fdd32373640 1 --2- 172.21.15.83:0/801319737 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd2c15bac0 0x7fdd2c0cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:57.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.607+0000 7fdd32373640 1 -- 172.21.15.83:0/801319737 >> v2:172.21.15.167:3300/0 conn(0x7fdd2c070470 msgr2=0x7fdd2c06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:57.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.607+0000 7fdd32373640 1 --2- 172.21.15.83:0/801319737 >> v2:172.21.15.167:3300/0 conn(0x7fdd2c070470 0x7fdd2c06ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:57.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.607+0000 7fdd32373640 1 -- 172.21.15.83:0/801319737 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd2c070cf0 con 0x7fdd2c15b6e0 2024-04-17T10:38:57.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.607+0000 7fdd32373640 1 --2- 172.21.15.83:0/801319737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd2c15b6e0 0x7fdd2c071cc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c0cd050 tx=0x7fdd2801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd606cbe1587ba03 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:57.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.607+0000 7fdd31371640 1 -- 172.21.15.83:0/801319737 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd280b0070 con 0x7fdd2c15b6e0 2024-04-17T10:38:57.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.607+0000 7fdd31371640 1 -- 172.21.15.83:0/801319737 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd280ab600 con 0x7fdd2c15b6e0 2024-04-17T10:38:57.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.607+0000 7fdd31371640 1 -- 172.21.15.83:0/801319737 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd280ab9d0 con 0x7fdd2c15b6e0 2024-04-17T10:38:57.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.607+0000 7fdd345d7640 1 -- 172.21.15.83:0/801319737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd2c15b6e0 msgr2=0x7fdd2c071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:57.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.607+0000 7fdd345d7640 1 --2- 172.21.15.83:0/801319737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd2c15b6e0 0x7fdd2c071cc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c0cd050 tx=0x7fdd2801c520 comp rx=0 tx=0).stop 2024-04-17T10:38:57.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.607+0000 7fdd345d7640 1 -- 172.21.15.83:0/801319737 shutdown_connections 2024-04-17T10:38:57.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.607+0000 7fdd345d7640 1 --2- 172.21.15.83:0/801319737 >> v2:172.21.15.167:3300/0 conn(0x7fdd2c070470 0x7fdd2c06ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:57.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.607+0000 7fdd345d7640 1 --2- 172.21.15.83:0/801319737 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd2c15bac0 0x7fdd2c0cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:57.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.607+0000 7fdd345d7640 1 --2- 172.21.15.83:0/801319737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd2c15b6e0 0x7fdd2c071cc0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:57.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.607+0000 7fdd345d7640 1 -- 172.21.15.83:0/801319737 >> 172.21.15.83:0/801319737 conn(0x7fdd2c0b15b0 msgr2=0x7fdd2c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:57.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.607+0000 7fdd345d7640 1 -- 172.21.15.83:0/801319737 shutdown_connections 2024-04-17T10:38:57.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.607+0000 7fdd345d7640 1 -- 172.21.15.83:0/801319737 wait complete. 2024-04-17T10:38:57.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.611+0000 7fdd345d7640 1 Processor -- start 2024-04-17T10:38:57.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.611+0000 7fdd345d7640 1 -- start start 2024-04-17T10:38:57.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.611+0000 7fdd345d7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd2c070470 0x7fdd2c145ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.611+0000 7fdd345d7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd2c15b6e0 0x7fdd2c0aac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.611+0000 7fdd345d7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd2c15bac0 0x7fdd2c0ab170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.611+0000 7fdd345d7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c06d270 con 0x7fdd2c15b6e0 2024-04-17T10:38:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.611+0000 7fdd345d7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c06d0f0 con 0x7fdd2c15bac0 2024-04-17T10:38:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.611+0000 7fdd345d7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c06b760 con 0x7fdd2c070470 2024-04-17T10:38:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.611+0000 7fdd31b72640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd2c15b6e0 0x7fdd2c0aac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.611+0000 7fdd31b72640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd2c15b6e0 0x7fdd2c0aac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50062/0 (socket says 172.21.15.83:50062) 2024-04-17T10:38:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.611+0000 7fdd31b72640 1 -- 172.21.15.83:0/3223525584 learned_addr learned my addr 172.21.15.83:0/3223525584 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.611+0000 7fdd32373640 1 --2- 172.21.15.83:0/3223525584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd2c070470 0x7fdd2c145ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.611+0000 7fdd32b74640 1 --2- 172.21.15.83:0/3223525584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd2c15bac0 0x7fdd2c0ab170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.611+0000 7fdd31b72640 1 -- 172.21.15.83:0/3223525584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd2c070470 msgr2=0x7fdd2c145ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.611+0000 7fdd31b72640 1 --2- 172.21.15.83:0/3223525584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd2c070470 0x7fdd2c145ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.611+0000 7fdd31b72640 1 -- 172.21.15.83:0/3223525584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd2c15bac0 msgr2=0x7fdd2c0ab170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:57.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.611+0000 7fdd31b72640 1 --2- 172.21.15.83:0/3223525584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd2c15bac0 0x7fdd2c0ab170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:57.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.611+0000 7fdd31b72640 1 -- 172.21.15.83:0/3223525584 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd2c0a9430 con 0x7fdd2c15b6e0 2024-04-17T10:38:57.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.611+0000 7fdd31b72640 1 --2- 172.21.15.83:0/3223525584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd2c15b6e0 0x7fdd2c0aac30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c00b3f0 tx=0x7fdd1c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:57.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.611+0000 7fdd22ffd640 1 -- 172.21.15.83:0/3223525584 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c015020 con 0x7fdd2c15b6e0 2024-04-17T10:38:57.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.611+0000 7fdd22ffd640 1 -- 172.21.15.83:0/3223525584 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1c011850 con 0x7fdd2c15b6e0 2024-04-17T10:38:57.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.611+0000 7fdd345d7640 1 -- 172.21.15.83:0/3223525584 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd2c0a9710 con 0x7fdd2c15b6e0 2024-04-17T10:38:57.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.611+0000 7fdd22ffd640 1 -- 172.21.15.83:0/3223525584 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c011c40 con 0x7fdd2c15b6e0 2024-04-17T10:38:57.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.611+0000 7fdd345d7640 1 -- 172.21.15.83:0/3223525584 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd2c0a9cd0 con 0x7fdd2c15b6e0 2024-04-17T10:38:57.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.611+0000 7fdd345d7640 1 -- 172.21.15.83:0/3223525584 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd2c074f90 con 0x7fdd2c15b6e0 2024-04-17T10:38:57.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.615+0000 7fdd22ffd640 1 -- 172.21.15.83:0/3223525584 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdd1c018020 con 0x7fdd2c15b6e0 2024-04-17T10:38:57.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.615+0000 7fdd22ffd640 1 --2- 172.21.15.83:0/3223525584 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdd10066f60 0x7fdd10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:57.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.615+0000 7fdd22ffd640 1 -- 172.21.15.83:0/3223525584 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fdd1c085010 con 0x7fdd2c15b6e0 2024-04-17T10:38:57.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.615+0000 7fdd32373640 1 --2- 172.21.15.83:0/3223525584 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdd10066f60 0x7fdd10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:57.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.615+0000 7fdd32373640 1 --2- 172.21.15.83:0/3223525584 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdd10066f60 0x7fdd10069420 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdd280ad000 tx=0x7fdd28008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:57.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.619+0000 7fdd22ffd640 1 -- 172.21.15.83:0/3223525584 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdd1c01f020 con 0x7fdd2c15b6e0 2024-04-17T10:38:57.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.763+0000 7fdd345d7640 1 -- 172.21.15.83:0/3223525584 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdd2c074480 con 0x7fdd2c15b6e0 2024-04-17T10:38:57.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.767+0000 7fdd22ffd640 1 -- 172.21.15.83:0/3223525584 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdd1c0172c0 con 0x7fdd2c15b6e0 2024-04-17T10:38:57.763 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:38:57.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.767+0000 7fdd345d7640 1 -- 172.21.15.83:0/3223525584 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdd10066f60 msgr2=0x7fdd10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:57.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.767+0000 7fdd345d7640 1 --2- 172.21.15.83:0/3223525584 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdd10066f60 0x7fdd10069420 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdd280ad000 tx=0x7fdd28008040 comp rx=0 tx=0).stop 2024-04-17T10:38:57.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.767+0000 7fdd345d7640 1 -- 172.21.15.83:0/3223525584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd2c15b6e0 msgr2=0x7fdd2c0aac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:57.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.767+0000 7fdd345d7640 1 --2- 172.21.15.83:0/3223525584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd2c15b6e0 0x7fdd2c0aac30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c00b3f0 tx=0x7fdd1c00b8b0 comp rx=0 tx=0).stop 2024-04-17T10:38:57.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.767+0000 7fdd345d7640 1 -- 172.21.15.83:0/3223525584 shutdown_connections 2024-04-17T10:38:57.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.767+0000 7fdd345d7640 1 --2- 172.21.15.83:0/3223525584 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdd10066f60 0x7fdd10069420 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:57.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.767+0000 7fdd345d7640 1 --2- 172.21.15.83:0/3223525584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd2c15bac0 0x7fdd2c0ab170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:57.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.767+0000 7fdd345d7640 1 --2- 172.21.15.83:0/3223525584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd2c15b6e0 0x7fdd2c0aac30 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:57.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.767+0000 7fdd345d7640 1 --2- 172.21.15.83:0/3223525584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd2c070470 0x7fdd2c145ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:57.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.767+0000 7fdd345d7640 1 -- 172.21.15.83:0/3223525584 >> 172.21.15.83:0/3223525584 conn(0x7fdd2c0b15b0 msgr2=0x7fdd2c0b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:57.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.767+0000 7fdd345d7640 1 -- 172.21.15.83:0/3223525584 shutdown_connections 2024-04-17T10:38:57.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.767+0000 7fdd345d7640 1 -- 172.21.15.83:0/3223525584 wait complete. 2024-04-17T10:38:57.780 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:38:57.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.903+0000 7ff90e510640 1 Processor -- start 2024-04-17T10:38:57.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.903+0000 7ff90e510640 1 -- start start 2024-04-17T10:38:57.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.903+0000 7ff90e510640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff90806b520 0x7ff9080b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:57.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.903+0000 7ff90e510640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9080b4040 0x7ff90806cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:57.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.903+0000 7ff90e510640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff90815bd70 con 0x7ff908154de0 2024-04-17T10:38:57.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.903+0000 7ff90e510640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9080b4870 con 0x7ff90806b520 2024-04-17T10:38:57.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.903+0000 7ff90e510640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff90815bf60 con 0x7ff9080b4040 2024-04-17T10:38:57.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.903+0000 7ff8ff7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff90806b520 0x7ff9080b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:57.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.903+0000 7ff8ff7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff90806b520 0x7ff9080b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50068/0 (socket says 172.21.15.83:50068) 2024-04-17T10:38:57.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.903+0000 7ff8ff7fe640 1 -- 172.21.15.83:0/2084845275 learned_addr learned my addr 172.21.15.83:0/2084845275 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:57.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.903+0000 7ff907fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff908154de0 0x7ff90806ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54682/0 (socket says 172.21.15.83:54682) 2024-04-17T10:38:57.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.903+0000 7ff90caad640 1 --2- 172.21.15.83:0/2084845275 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9080b4040 0x7ff90806cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:57.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.903+0000 7ff8ff7fe640 1 -- 172.21.15.83:0/2084845275 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9080b4040 msgr2=0x7ff90806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:57.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.903+0000 7ff8ff7fe640 1 --2- 172.21.15.83:0/2084845275 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9080b4040 0x7ff90806cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:57.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.903+0000 7ff8ff7fe640 1 -- 172.21.15.83:0/2084845275 >> v1:172.21.15.83:6789/0 conn(0x7ff908154de0 legacy=0x7ff90806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:57.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.903+0000 7ff9077fe640 1 -- 172.21.15.83:0/2084845275 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 756346458 0 0) 0x7ff90815bd70 con 0x7ff908154de0 2024-04-17T10:38:57.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.903+0000 7ff8ff7fe640 1 -- 172.21.15.83:0/2084845275 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff90815afb0 con 0x7ff90806b520 2024-04-17T10:38:57.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.907+0000 7ff8ff7fe640 1 --2- 172.21.15.83:0/2084845275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff90806b520 0x7ff9080b3b00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff8f000bb10 tx=0x7ff8f001ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ee473bd954a58b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:57.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.907+0000 7ff9077fe640 1 -- 172.21.15.83:0/2084845275 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f00b4070 con 0x7ff90806b520 2024-04-17T10:38:57.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.907+0000 7ff9077fe640 1 -- 172.21.15.83:0/2084845275 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f00a7bf0 con 0x7ff90806b520 2024-04-17T10:38:57.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.907+0000 7ff9077fe640 1 -- 172.21.15.83:0/2084845275 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f00af440 con 0x7ff90806b520 2024-04-17T10:38:57.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.907+0000 7ff90e510640 1 -- 172.21.15.83:0/2084845275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff90806b520 msgr2=0x7ff9080b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:57.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.907+0000 7ff90e510640 1 --2- 172.21.15.83:0/2084845275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff90806b520 0x7ff9080b3b00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff8f000bb10 tx=0x7ff8f001ec30 comp rx=0 tx=0).stop 2024-04-17T10:38:57.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.907+0000 7ff90e510640 1 -- 172.21.15.83:0/2084845275 shutdown_connections 2024-04-17T10:38:57.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.907+0000 7ff90e510640 1 --2- 172.21.15.83:0/2084845275 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9080b4040 0x7ff90806cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:57.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.907+0000 7ff90e510640 1 --2- 172.21.15.83:0/2084845275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff90806b520 0x7ff9080b3b00 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:57.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.907+0000 7ff90e510640 1 -- 172.21.15.83:0/2084845275 >> 172.21.15.83:0/2084845275 conn(0x7ff9080b15b0 msgr2=0x7ff9080b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:57.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.907+0000 7ff90e510640 1 -- 172.21.15.83:0/2084845275 shutdown_connections 2024-04-17T10:38:57.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.907+0000 7ff90e510640 1 -- 172.21.15.83:0/2084845275 wait complete. 2024-04-17T10:38:57.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.911+0000 7ff90e510640 1 Processor -- start 2024-04-17T10:38:57.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.911+0000 7ff90e510640 1 -- start start 2024-04-17T10:38:57.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.911+0000 7ff90e510640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff90806b520 0x7ff90814c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:57.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.911+0000 7ff90e510640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9080b4040 0x7ff90813cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:57.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.911+0000 7ff90e510640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff908154de0 0x7ff9080ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:57.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.911+0000 7ff90e510640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9080b4470 con 0x7ff908154de0 2024-04-17T10:38:57.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.911+0000 7ff90e510640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9081551c0 con 0x7ff9080b4040 2024-04-17T10:38:57.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.911+0000 7ff90e510640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9080b45f0 con 0x7ff90806b520 2024-04-17T10:38:57.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.911+0000 7ff90caad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff908154de0 0x7ff9080ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:57.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.911+0000 7ff90caad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff908154de0 0x7ff9080ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50076/0 (socket says 172.21.15.83:50076) 2024-04-17T10:38:57.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.911+0000 7ff90caad640 1 -- 172.21.15.83:0/878509411 learned_addr learned my addr 172.21.15.83:0/878509411 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:57.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.911+0000 7ff907fff640 1 --2- 172.21.15.83:0/878509411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff90806b520 0x7ff90814c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:57.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.911+0000 7ff8ff7fe640 1 --2- 172.21.15.83:0/878509411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9080b4040 0x7ff90813cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:57.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.911+0000 7ff90caad640 1 -- 172.21.15.83:0/878509411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff90806b520 msgr2=0x7ff90814c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:57.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.911+0000 7ff90caad640 1 --2- 172.21.15.83:0/878509411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff90806b520 0x7ff90814c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:57.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.911+0000 7ff90caad640 1 -- 172.21.15.83:0/878509411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9080b4040 msgr2=0x7ff90813cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:57.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.911+0000 7ff90caad640 1 --2- 172.21.15.83:0/878509411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9080b4040 0x7ff90813cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:57.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.911+0000 7ff90caad640 1 -- 172.21.15.83:0/878509411 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff90814c630 con 0x7ff908154de0 2024-04-17T10:38:57.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.911+0000 7ff90caad640 1 --2- 172.21.15.83:0/878509411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff908154de0 0x7ff9080ad840 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff900007e40 tx=0x7ff90000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:57.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.911+0000 7ff9057fa640 1 -- 172.21.15.83:0/878509411 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff900014620 con 0x7ff908154de0 2024-04-17T10:38:57.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.911+0000 7ff9057fa640 1 -- 172.21.15.83:0/878509411 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff900019070 con 0x7ff908154de0 2024-04-17T10:38:57.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.911+0000 7ff90e510640 1 -- 172.21.15.83:0/878509411 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9080cd650 con 0x7ff908154de0 2024-04-17T10:38:57.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.911+0000 7ff9057fa640 1 -- 172.21.15.83:0/878509411 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff900014a20 con 0x7ff908154de0 2024-04-17T10:38:57.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.911+0000 7ff90e510640 1 -- 172.21.15.83:0/878509411 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9080cdc10 con 0x7ff908154de0 2024-04-17T10:38:57.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.911+0000 7ff90e510640 1 -- 172.21.15.83:0/878509411 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff908074f90 con 0x7ff908154de0 2024-04-17T10:38:57.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.915+0000 7ff9057fa640 1 -- 172.21.15.83:0/878509411 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff900014bc0 con 0x7ff908154de0 2024-04-17T10:38:57.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.915+0000 7ff9057fa640 1 --2- 172.21.15.83:0/878509411 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff8e8066f60 0x7ff8e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:57.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.915+0000 7ff9057fa640 1 -- 172.21.15.83:0/878509411 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7ff900085940 con 0x7ff908154de0 2024-04-17T10:38:57.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.915+0000 7ff907fff640 1 --2- 172.21.15.83:0/878509411 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff8e8066f60 0x7ff8e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:57.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.915+0000 7ff907fff640 1 --2- 172.21.15.83:0/878509411 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff8e8066f60 0x7ff8e8069420 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8000f70 tx=0x7ff8f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:57.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:57.919+0000 7ff9057fa640 1 -- 172.21.15.83:0/878509411 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff90814c630 con 0x7ff908154de0 2024-04-17T10:38:58.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.087+0000 7ff90e510640 1 -- 172.21.15.83:0/878509411 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff908070a40 con 0x7ff908154de0 2024-04-17T10:38:58.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.091+0000 7ff9057fa640 1 -- 172.21.15.83:0/878509411 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff9000568d0 con 0x7ff908154de0 2024-04-17T10:38:58.087 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-17T10:38:58.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.091+0000 7ff90e510640 1 -- 172.21.15.83:0/878509411 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff8e8066f60 msgr2=0x7ff8e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:58.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.091+0000 7ff90e510640 1 --2- 172.21.15.83:0/878509411 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff8e8066f60 0x7ff8e8069420 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8000f70 tx=0x7ff8f8009040 comp rx=0 tx=0).stop 2024-04-17T10:38:58.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.091+0000 7ff90e510640 1 -- 172.21.15.83:0/878509411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff908154de0 msgr2=0x7ff9080ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:58.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.091+0000 7ff90e510640 1 --2- 172.21.15.83:0/878509411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff908154de0 0x7ff9080ad840 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff900007e40 tx=0x7ff90000e4b0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.091+0000 7ff90e510640 1 -- 172.21.15.83:0/878509411 shutdown_connections 2024-04-17T10:38:58.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.091+0000 7ff90e510640 1 --2- 172.21.15.83:0/878509411 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff8e8066f60 0x7ff8e8069420 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.091+0000 7ff90e510640 1 --2- 172.21.15.83:0/878509411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff908154de0 0x7ff9080ad840 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.091+0000 7ff90e510640 1 --2- 172.21.15.83:0/878509411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9080b4040 0x7ff90813cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.091+0000 7ff90e510640 1 --2- 172.21.15.83:0/878509411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff90806b520 0x7ff90814c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.091+0000 7ff90e510640 1 -- 172.21.15.83:0/878509411 >> 172.21.15.83:0/878509411 conn(0x7ff9080b15b0 msgr2=0x7ff90806ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:58.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.091+0000 7ff90e510640 1 -- 172.21.15.83:0/878509411 shutdown_connections 2024-04-17T10:38:58.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.091+0000 7ff90e510640 1 -- 172.21.15.83:0/878509411 wait complete. 2024-04-17T10:38:58.105 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:38:58.106 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:38:58.112 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:38:58.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c9355a640 1 Processor -- start 2024-04-17T10:38:58.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c9355a640 1 -- start start 2024-04-17T10:38:58.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c9355a640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4c8c15f1d0 0x7f4c8c15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:58.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c9355a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c8c15f5b0 con 0x7f4c8c0b2de0 2024-04-17T10:38:58.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c9355a640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c8c0b2620 con 0x7f4c8c0d22d0 2024-04-17T10:38:58.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c9355a640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c8c0b2340 con 0x7f4c8c15f1d0 2024-04-17T10:38:58.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c90af5640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f4c8c0b2de0 0x7f4c8c15fd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54696/0 (socket says 172.21.15.83:54696) 2024-04-17T10:38:58.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c90af5640 1 -- 172.21.15.83:0/2538783024 learned_addr learned my addr 172.21.15.83:0/2538783024 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:58.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c912f6640 1 --2- 172.21.15.83:0/2538783024 >> v2:172.21.15.167:3300/0 conn(0x7f4c8c15f1d0 0x7f4c8c15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:58.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c83fff640 1 -- 172.21.15.83:0/2538783024 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1272715337 0 0) 0x7f4c8c15f5b0 con 0x7f4c8c0b2de0 2024-04-17T10:38:58.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c83fff640 1 -- 172.21.15.83:0/2538783024 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c78003530 con 0x7f4c8c0b2de0 2024-04-17T10:38:58.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c83fff640 1 -- 172.21.15.83:0/2538783024 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 273281662 0 0) 0x7f4c78003530 con 0x7f4c8c0b2de0 2024-04-17T10:38:58.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c83fff640 1 -- 172.21.15.83:0/2538783024 >> v2:172.21.15.167:3300/0 conn(0x7f4c8c15f1d0 msgr2=0x7f4c8c15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:58.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c83fff640 1 --2- 172.21.15.83:0/2538783024 >> v2:172.21.15.167:3300/0 conn(0x7f4c8c15f1d0 0x7f4c8c15f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c83fff640 1 -- 172.21.15.83:0/2538783024 >> v1:172.21.15.167:6789/0 conn(0x7f4c8c0d22d0 legacy=0x7f4c8c0ae520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:58.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c83fff640 1 -- 172.21.15.83:0/2538783024 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c8c0b1760 con 0x7f4c8c0b2de0 2024-04-17T10:38:58.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c83fff640 1 -- 172.21.15.83:0/2538783024 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f4c7c001680 con 0x7f4c8c0b2de0 2024-04-17T10:38:58.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c83fff640 1 -- 172.21.15.83:0/2538783024 >> v1:172.21.15.83:6789/0 conn(0x7f4c8c0b2de0 legacy=0x7f4c8c15fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:58.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c912f6640 1 --2- 172.21.15.83:0/2538783024 >> v2:172.21.15.167:3300/0 conn(0x7f4c8c15f1d0 0x7f4c8c15f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:38:58.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c83fff640 1 --2- 172.21.15.83:0/2538783024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c780039c0 0x7f4c78003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:58.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c83fff640 1 --2- 172.21.15.83:0/2538783024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c78004810 0x7f4c78004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:58.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c83fff640 1 --2- 172.21.15.83:0/2538783024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c780051d0 0x7f4c7800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:58.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c83fff640 1 -- 172.21.15.83:0/2538783024 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c0b2340 con 0x7f4c780051d0 2024-04-17T10:38:58.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c83fff640 1 -- 172.21.15.83:0/2538783024 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7800c120 con 0x7f4c780039c0 2024-04-17T10:38:58.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c83fff640 1 -- 172.21.15.83:0/2538783024 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7800c320 con 0x7f4c78004810 2024-04-17T10:38:58.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c91af7640 1 --2- 172.21.15.83:0/2538783024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c780051d0 0x7f4c7800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:58.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c912f6640 1 --2- 172.21.15.83:0/2538783024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c780039c0 0x7f4c78003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:58.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c90af5640 1 --2- 172.21.15.83:0/2538783024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c78004810 0x7f4c78004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:58.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c91af7640 1 -- 172.21.15.83:0/2538783024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c78004810 msgr2=0x7f4c78004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:58.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c91af7640 1 --2- 172.21.15.83:0/2538783024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c78004810 0x7f4c78004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c91af7640 1 -- 172.21.15.83:0/2538783024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c780039c0 msgr2=0x7f4c78003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:58.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c91af7640 1 --2- 172.21.15.83:0/2538783024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c780039c0 0x7f4c78003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c91af7640 1 -- 172.21.15.83:0/2538783024 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c78003530 con 0x7f4c780051d0 2024-04-17T10:38:58.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.231+0000 7f4c90af5640 1 --2- 172.21.15.83:0/2538783024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c78004810 0x7f4c78004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:58.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.235+0000 7f4c91af7640 1 -- 172.21.15.83:0/2538783024 reap_dead start 2024-04-17T10:38:58.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.235+0000 7f4c91af7640 1 --2- 172.21.15.83:0/2538783024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c780051d0 0x7f4c7800b9d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4c84001e70 tx=0x7f4c8401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c46e271f25e299c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:58.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.235+0000 7f4c83fff640 1 -- 172.21.15.83:0/2538783024 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c84001a90 con 0x7f4c780051d0 2024-04-17T10:38:58.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.235+0000 7f4c83fff640 1 -- 172.21.15.83:0/2538783024 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c84003bc0 con 0x7f4c780051d0 2024-04-17T10:38:58.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.235+0000 7f4c83fff640 1 -- 172.21.15.83:0/2538783024 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c84005020 con 0x7f4c780051d0 2024-04-17T10:38:58.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.235+0000 7f4c9355a640 1 -- 172.21.15.83:0/2538783024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c780051d0 msgr2=0x7f4c7800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:58.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.235+0000 7f4c9355a640 1 --2- 172.21.15.83:0/2538783024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c780051d0 0x7f4c7800b9d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4c84001e70 tx=0x7f4c8401c780 comp rx=0 tx=0).stop 2024-04-17T10:38:58.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.235+0000 7f4c9355a640 1 -- 172.21.15.83:0/2538783024 shutdown_connections 2024-04-17T10:38:58.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.235+0000 7f4c9355a640 1 --2- 172.21.15.83:0/2538783024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c780051d0 0x7f4c7800b9d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.235+0000 7f4c9355a640 1 -- 172.21.15.83:0/2538783024 >> 172.21.15.83:0/2538783024 conn(0x7f4c8c0d17b0 msgr2=0x7f4c8c0d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:58.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.235+0000 7f4c9355a640 1 -- 172.21.15.83:0/2538783024 shutdown_connections 2024-04-17T10:38:58.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.235+0000 7f4c9355a640 1 -- 172.21.15.83:0/2538783024 wait complete. 2024-04-17T10:38:58.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.235+0000 7f4c9355a640 1 Processor -- start 2024-04-17T10:38:58.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.235+0000 7f4c9355a640 1 -- start start 2024-04-17T10:38:58.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.235+0000 7f4c9355a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c780051d0 0x7f4c8c0b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:58.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.235+0000 7f4c9355a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c8c14ce80 0x7f4c8c0c8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:58.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.235+0000 7f4c9355a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c8c14e520 0x7f4c8c14e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:58.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.235+0000 7f4c9355a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c15f830 con 0x7f4c780051d0 2024-04-17T10:38:58.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.235+0000 7f4c9355a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c0b1d20 con 0x7f4c8c14ce80 2024-04-17T10:38:58.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.235+0000 7f4c9355a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c15f9b0 con 0x7f4c8c14e520 2024-04-17T10:38:58.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.235+0000 7f4c912f6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c780051d0 0x7f4c8c0b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:58.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.235+0000 7f4c912f6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c780051d0 0x7f4c8c0b6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50098/0 (socket says 172.21.15.83:50098) 2024-04-17T10:38:58.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.235+0000 7f4c912f6640 1 -- 172.21.15.83:0/1215443121 learned_addr learned my addr 172.21.15.83:0/1215443121 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:58.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.235+0000 7f4c91af7640 1 --2- 172.21.15.83:0/1215443121 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c8c14e520 0x7f4c8c14e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:58.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.235+0000 7f4c90af5640 1 --2- 172.21.15.83:0/1215443121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c8c14ce80 0x7f4c8c0c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:58.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.235+0000 7f4c912f6640 1 -- 172.21.15.83:0/1215443121 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c8c14e520 msgr2=0x7f4c8c14e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:58.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.235+0000 7f4c912f6640 1 --2- 172.21.15.83:0/1215443121 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c8c14e520 0x7f4c8c14e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.235+0000 7f4c912f6640 1 -- 172.21.15.83:0/1215443121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c8c14ce80 msgr2=0x7f4c8c0c8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:58.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.235+0000 7f4c912f6640 1 --2- 172.21.15.83:0/1215443121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c8c14ce80 0x7f4c8c0c8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.235+0000 7f4c912f6640 1 -- 172.21.15.83:0/1215443121 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c8c0c8dc0 con 0x7f4c780051d0 2024-04-17T10:38:58.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.239+0000 7f4c912f6640 1 --2- 172.21.15.83:0/1215443121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c780051d0 0x7f4c8c0b6ef0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4c880108f0 tx=0x7f4c88010db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:58.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.239+0000 7f4c81ffb640 1 -- 172.21.15.83:0/1215443121 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8801a070 con 0x7f4c780051d0 2024-04-17T10:38:58.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.239+0000 7f4c81ffb640 1 -- 172.21.15.83:0/1215443121 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8800cb40 con 0x7f4c780051d0 2024-04-17T10:38:58.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.239+0000 7f4c9355a640 1 -- 172.21.15.83:0/1215443121 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c8c0c9040 con 0x7f4c780051d0 2024-04-17T10:38:58.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.239+0000 7f4c81ffb640 1 -- 172.21.15.83:0/1215443121 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8800f040 con 0x7f4c780051d0 2024-04-17T10:38:58.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.239+0000 7f4c9355a640 1 -- 172.21.15.83:0/1215443121 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c8c0c9600 con 0x7f4c780051d0 2024-04-17T10:38:58.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.239+0000 7f4c9355a640 1 -- 172.21.15.83:0/1215443121 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c8c074f90 con 0x7f4c780051d0 2024-04-17T10:38:58.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.239+0000 7f4c81ffb640 1 -- 172.21.15.83:0/1215443121 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4c8801d070 con 0x7f4c780051d0 2024-04-17T10:38:58.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.239+0000 7f4c81ffb640 1 --2- 172.21.15.83:0/1215443121 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4c70066f60 0x7f4c70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:58.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.239+0000 7f4c90af5640 1 --2- 172.21.15.83:0/1215443121 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4c70066f60 0x7f4c70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:58.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.239+0000 7f4c81ffb640 1 -- 172.21.15.83:0/1215443121 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f4c88085540 con 0x7f4c780051d0 2024-04-17T10:38:58.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.239+0000 7f4c90af5640 1 --2- 172.21.15.83:0/1215443121 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4c70066f60 0x7f4c70069420 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c001ec0 tx=0x7f4c7c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:58.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.243+0000 7f4c81ffb640 1 -- 172.21.15.83:0/1215443121 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4c8c0c8dc0 con 0x7f4c780051d0 2024-04-17T10:38:58.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.403+0000 7f4c9355a640 1 -- 172.21.15.83:0/1215443121 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c8c072690 con 0x7f4c780051d0 2024-04-17T10:38:58.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.407+0000 7f4c81ffb640 1 -- 172.21.15.83:0/1215443121 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16782 (secure 0 0 0) 0x7f4c8800ee60 con 0x7f4c780051d0 2024-04-17T10:38:58.404 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:38:58.404 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":22,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:38:53.385002+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:38:10.464490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T10:38:11.043134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:38:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.407+0000 7f4c9355a640 1 -- 172.21.15.83:0/1215443121 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4c70066f60 msgr2=0x7f4c70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.407+0000 7f4c9355a640 1 --2- 172.21.15.83:0/1215443121 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4c70066f60 0x7f4c70069420 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c001ec0 tx=0x7f4c7c009040 comp rx=0 tx=0).stop 2024-04-17T10:38:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.407+0000 7f4c9355a640 1 -- 172.21.15.83:0/1215443121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c780051d0 msgr2=0x7f4c8c0b6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.407+0000 7f4c9355a640 1 --2- 172.21.15.83:0/1215443121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c780051d0 0x7f4c8c0b6ef0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4c880108f0 tx=0x7f4c88010db0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.407+0000 7f4c9355a640 1 -- 172.21.15.83:0/1215443121 shutdown_connections 2024-04-17T10:38:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.407+0000 7f4c9355a640 1 --2- 172.21.15.83:0/1215443121 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4c70066f60 0x7f4c70069420 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.407+0000 7f4c9355a640 1 --2- 172.21.15.83:0/1215443121 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c8c14e520 0x7f4c8c14e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.407+0000 7f4c9355a640 1 --2- 172.21.15.83:0/1215443121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c8c14ce80 0x7f4c8c0c8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.407+0000 7f4c9355a640 1 --2- 172.21.15.83:0/1215443121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c780051d0 0x7f4c8c0b6ef0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.407+0000 7f4c9355a640 1 -- 172.21.15.83:0/1215443121 >> 172.21.15.83:0/1215443121 conn(0x7f4c8c0d17b0 msgr2=0x7f4c8c0afce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.407+0000 7f4c9355a640 1 -- 172.21.15.83:0/1215443121 shutdown_connections 2024-04-17T10:38:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.407+0000 7f4c9355a640 1 -- 172.21.15.83:0/1215443121 wait complete. 2024-04-17T10:38:58.425 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:38:10.464490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:38:11.043134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:38:58.425 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:38:58.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.555+0000 7f89eb8c1640 1 Processor -- start 2024-04-17T10:38:58.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.555+0000 7f89eb8c1640 1 -- start start 2024-04-17T10:38:58.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.555+0000 7f89eb8c1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f89e415b6e0 0x7f89e4071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:58.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.555+0000 7f89eb8c1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89e415bac0 0x7f89e40cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:58.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.555+0000 7f89eb8c1640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e406c1f0 con 0x7f89e4070470 2024-04-17T10:38:58.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.555+0000 7f89eb8c1640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e40cdd30 con 0x7f89e415b6e0 2024-04-17T10:38:58.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.555+0000 7f89eb8c1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e4070a10 con 0x7f89e415bac0 2024-04-17T10:38:58.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.555+0000 7f89e965d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f89e415b6e0 0x7f89e4071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:58.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.555+0000 7f89e9e5e640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f89e4070470 0x7f89e406ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54700/0 (socket says 172.21.15.83:54700) 2024-04-17T10:38:58.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.555+0000 7f89e9e5e640 1 -- 172.21.15.83:0/97433266 learned_addr learned my addr 172.21.15.83:0/97433266 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:58.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.555+0000 7f89e8e5c640 1 --2- 172.21.15.83:0/97433266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89e415bac0 0x7f89e40cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:58.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.555+0000 7f89e965d640 1 -- 172.21.15.83:0/97433266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89e415bac0 msgr2=0x7f89e40cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:58.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.555+0000 7f89e965d640 1 --2- 172.21.15.83:0/97433266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89e415bac0 0x7f89e40cd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.555+0000 7f89e965d640 1 -- 172.21.15.83:0/97433266 >> v1:172.21.15.83:6789/0 conn(0x7f89e4070470 legacy=0x7f89e406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:58.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.555+0000 7f89e965d640 1 -- 172.21.15.83:0/97433266 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e4070cf0 con 0x7f89e415b6e0 2024-04-17T10:38:58.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.555+0000 7f89dbfff640 1 -- 172.21.15.83:0/97433266 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3018853483 0 0) 0x7f89e406c1f0 con 0x7f89e4070470 2024-04-17T10:38:58.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89e965d640 1 --2- 172.21.15.83:0/97433266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89e415b6e0 0x7f89e4071cc0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f89e00097c0 tx=0x7f89e001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b81a89cacf0c5a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:58.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89dbfff640 1 -- 172.21.15.83:0/97433266 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e00b0070 con 0x7f89e415b6e0 2024-04-17T10:38:58.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89dbfff640 1 -- 172.21.15.83:0/97433266 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e00ab600 con 0x7f89e415b6e0 2024-04-17T10:38:58.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89dbfff640 1 -- 172.21.15.83:0/97433266 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e00ab9d0 con 0x7f89e415b6e0 2024-04-17T10:38:58.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89eb8c1640 1 -- 172.21.15.83:0/97433266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89e415b6e0 msgr2=0x7f89e4071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:58.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89eb8c1640 1 --2- 172.21.15.83:0/97433266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89e415b6e0 0x7f89e4071cc0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f89e00097c0 tx=0x7f89e001c4a0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89eb8c1640 1 -- 172.21.15.83:0/97433266 shutdown_connections 2024-04-17T10:38:58.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89eb8c1640 1 --2- 172.21.15.83:0/97433266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89e415bac0 0x7f89e40cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89eb8c1640 1 --2- 172.21.15.83:0/97433266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89e415b6e0 0x7f89e4071cc0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89eb8c1640 1 -- 172.21.15.83:0/97433266 >> 172.21.15.83:0/97433266 conn(0x7f89e40b15b0 msgr2=0x7f89e40b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:58.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89eb8c1640 1 -- 172.21.15.83:0/97433266 shutdown_connections 2024-04-17T10:38:58.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89eb8c1640 1 -- 172.21.15.83:0/97433266 wait complete. 2024-04-17T10:38:58.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89eb8c1640 1 Processor -- start 2024-04-17T10:38:58.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89eb8c1640 1 -- start start 2024-04-17T10:38:58.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89eb8c1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89e4070470 0x7f89e414f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:58.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89eb8c1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89e415b6e0 0x7f89e415dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:58.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89eb8c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89e415bac0 0x7f89e414e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:58.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89eb8c1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e406cee0 con 0x7f89e415bac0 2024-04-17T10:38:58.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89eb8c1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e406b730 con 0x7f89e4070470 2024-04-17T10:38:58.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89eb8c1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e406d060 con 0x7f89e415b6e0 2024-04-17T10:38:58.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89e9e5e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89e415bac0 0x7f89e414e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:58.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89e9e5e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89e415bac0 0x7f89e414e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50114/0 (socket says 172.21.15.83:50114) 2024-04-17T10:38:58.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89e9e5e640 1 -- 172.21.15.83:0/1653497700 learned_addr learned my addr 172.21.15.83:0/1653497700 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:58.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89e8e5c640 1 --2- 172.21.15.83:0/1653497700 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89e415b6e0 0x7f89e415dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:58.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89e965d640 1 --2- 172.21.15.83:0/1653497700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89e4070470 0x7f89e414f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:58.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89e9e5e640 1 -- 172.21.15.83:0/1653497700 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89e415b6e0 msgr2=0x7f89e415dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:58.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89e9e5e640 1 --2- 172.21.15.83:0/1653497700 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89e415b6e0 0x7f89e415dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89e9e5e640 1 -- 172.21.15.83:0/1653497700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89e4070470 msgr2=0x7f89e414f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:58.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89e9e5e640 1 --2- 172.21.15.83:0/1653497700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89e4070470 0x7f89e414f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89e9e5e640 1 -- 172.21.15.83:0/1653497700 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e414fa50 con 0x7f89e415bac0 2024-04-17T10:38:58.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89e9e5e640 1 --2- 172.21.15.83:0/1653497700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89e415bac0 0x7f89e414e230 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00caf0 tx=0x7f89dc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:58.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89d9ffb640 1 -- 172.21.15.83:0/1653497700 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc00aca0 con 0x7f89e415bac0 2024-04-17T10:38:58.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89d9ffb640 1 -- 172.21.15.83:0/1653497700 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89dc00ae40 con 0x7f89e415bac0 2024-04-17T10:38:58.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89d9ffb640 1 -- 172.21.15.83:0/1653497700 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc012640 con 0x7f89e415bac0 2024-04-17T10:38:58.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.559+0000 7f89eb8c1640 1 -- 172.21.15.83:0/1653497700 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e40a94d0 con 0x7f89e415bac0 2024-04-17T10:38:58.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.563+0000 7f89eb8c1640 1 -- 172.21.15.83:0/1653497700 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e414fd30 con 0x7f89e415bac0 2024-04-17T10:38:58.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.563+0000 7f89eb8c1640 1 -- 172.21.15.83:0/1653497700 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e4074f90 con 0x7f89e415bac0 2024-04-17T10:38:58.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.563+0000 7f89d9ffb640 1 -- 172.21.15.83:0/1653497700 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f89dc0128a0 con 0x7f89e415bac0 2024-04-17T10:38:58.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.563+0000 7f89d9ffb640 1 --2- 172.21.15.83:0/1653497700 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f89c8066f60 0x7f89c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:58.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.563+0000 7f89e965d640 1 --2- 172.21.15.83:0/1653497700 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f89c8066f60 0x7f89c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:58.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.563+0000 7f89d9ffb640 1 -- 172.21.15.83:0/1653497700 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f89dc0857b0 con 0x7f89e415bac0 2024-04-17T10:38:58.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.563+0000 7f89e965d640 1 --2- 172.21.15.83:0/1653497700 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f89c8066f60 0x7f89c8069420 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f89e00ad000 tx=0x7f89e00a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:58.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.567+0000 7f89d9ffb640 1 -- 172.21.15.83:0/1653497700 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f89dc056740 con 0x7f89e415bac0 2024-04-17T10:38:58.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.719+0000 7f89eb8c1640 1 -- 172.21.15.83:0/1653497700 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f89e40b24a0 con 0x7f89e415bac0 2024-04-17T10:38:58.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.723+0000 7f89d9ffb640 1 -- 172.21.15.83:0/1653497700 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f89dc056040 con 0x7f89e415bac0 2024-04-17T10:38:58.719 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:38:58.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.723+0000 7f89eb8c1640 1 -- 172.21.15.83:0/1653497700 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f89c8066f60 msgr2=0x7f89c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:58.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.723+0000 7f89eb8c1640 1 --2- 172.21.15.83:0/1653497700 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f89c8066f60 0x7f89c8069420 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f89e00ad000 tx=0x7f89e00a6000 comp rx=0 tx=0).stop 2024-04-17T10:38:58.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.723+0000 7f89eb8c1640 1 -- 172.21.15.83:0/1653497700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89e415bac0 msgr2=0x7f89e414e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:58.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.723+0000 7f89eb8c1640 1 --2- 172.21.15.83:0/1653497700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89e415bac0 0x7f89e414e230 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00caf0 tx=0x7f89dc00a060 comp rx=0 tx=0).stop 2024-04-17T10:38:58.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.723+0000 7f89eb8c1640 1 -- 172.21.15.83:0/1653497700 shutdown_connections 2024-04-17T10:38:58.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.723+0000 7f89eb8c1640 1 --2- 172.21.15.83:0/1653497700 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f89c8066f60 0x7f89c8069420 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.723+0000 7f89eb8c1640 1 --2- 172.21.15.83:0/1653497700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89e415bac0 0x7f89e414e230 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.723+0000 7f89eb8c1640 1 --2- 172.21.15.83:0/1653497700 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89e415b6e0 0x7f89e415dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.723+0000 7f89eb8c1640 1 --2- 172.21.15.83:0/1653497700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89e4070470 0x7f89e414f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.723+0000 7f89eb8c1640 1 -- 172.21.15.83:0/1653497700 >> 172.21.15.83:0/1653497700 conn(0x7f89e40b15b0 msgr2=0x7f89e406ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:58.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.723+0000 7f89eb8c1640 1 -- 172.21.15.83:0/1653497700 shutdown_connections 2024-04-17T10:38:58.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.723+0000 7f89eb8c1640 1 -- 172.21.15.83:0/1653497700 wait complete. 2024-04-17T10:38:58.736 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:38:58.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3ad0ff2640 1 Processor -- start 2024-04-17T10:38:58.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3ad0ff2640 1 -- start start 2024-04-17T10:38:58.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3ad0ff2640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3acc15b760 0x7f3acc0714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:58.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3ad0ff2640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3acc0af9c0 0x7f3acc0719e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:58.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3ad0ff2640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3acc0b3a70 con 0x7f3acc0aff10 2024-04-17T10:38:58.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3ad0ff2640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3acc06abe0 con 0x7f3acc15b760 2024-04-17T10:38:58.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3ad0ff2640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc0b3d50 con 0x7f3acc0af9c0 2024-04-17T10:38:58.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3ac9d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3acc0aff10 0x7f3acc15f420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54716/0 (socket says 172.21.15.83:54716) 2024-04-17T10:38:58.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3ac9d9b640 1 -- 172.21.15.83:0/1011950626 learned_addr learned my addr 172.21.15.83:0/1011950626 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:58.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3ac959a640 1 -- 172.21.15.83:0/1011950626 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2698767013 0 0) 0x7f3acc0b3a70 con 0x7f3acc0aff10 2024-04-17T10:38:58.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3ac959a640 1 -- 172.21.15.83:0/1011950626 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ab4003530 con 0x7f3acc0aff10 2024-04-17T10:38:58.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3acad9d640 1 --2- 172.21.15.83:0/1011950626 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3acc0af9c0 0x7f3acc0719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:58.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3aca59c640 1 --2- 172.21.15.83:0/1011950626 >> v2:172.21.15.167:3300/0 conn(0x7f3acc15b760 0x7f3acc0714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:58.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3ac959a640 1 -- 172.21.15.83:0/1011950626 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1658152566 0 0) 0x7f3ab4003530 con 0x7f3acc0aff10 2024-04-17T10:38:58.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3ac959a640 1 -- 172.21.15.83:0/1011950626 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3acc0af9c0 msgr2=0x7f3acc0719e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:58.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3ac959a640 1 --2- 172.21.15.83:0/1011950626 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3acc0af9c0 0x7f3acc0719e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3ac959a640 1 -- 172.21.15.83:0/1011950626 >> v2:172.21.15.167:3300/0 conn(0x7f3acc15b760 msgr2=0x7f3acc0714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:58.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3ac959a640 1 --2- 172.21.15.83:0/1011950626 >> v2:172.21.15.167:3300/0 conn(0x7f3acc15b760 0x7f3acc0714a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3ac959a640 1 -- 172.21.15.83:0/1011950626 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3acc0b4030 con 0x7f3acc0aff10 2024-04-17T10:38:58.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3ac959a640 1 -- 172.21.15.83:0/1011950626 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f3ac0001680 con 0x7f3acc0aff10 2024-04-17T10:38:58.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3ac959a640 1 -- 172.21.15.83:0/1011950626 >> v1:172.21.15.83:6789/0 conn(0x7f3acc0aff10 legacy=0x7f3acc15f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:58.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3ac959a640 1 --2- 172.21.15.83:0/1011950626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ab4003a30 0x7f3ab4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:58.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3ac959a640 1 --2- 172.21.15.83:0/1011950626 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3acc0af9c0 0x7f3ab4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:58.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3ac959a640 1 --2- 172.21.15.83:0/1011950626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ab4004d50 0x7f3ab40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:58.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3ac959a640 1 -- 172.21.15.83:0/1011950626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc0b3d50 con 0x7f3ab4003a30 2024-04-17T10:38:58.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3ac959a640 1 -- 172.21.15.83:0/1011950626 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab400ba00 con 0x7f3ab4004d50 2024-04-17T10:38:58.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3ac959a640 1 -- 172.21.15.83:0/1011950626 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab400bc00 con 0x7f3acc0af9c0 2024-04-17T10:38:58.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3aca59c640 1 --2- 172.21.15.83:0/1011950626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ab4003a30 0x7f3ab4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:58.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3ac9d9b640 1 --2- 172.21.15.83:0/1011950626 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3acc0af9c0 0x7f3ab4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:58.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3acad9d640 1 --2- 172.21.15.83:0/1011950626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ab4004d50 0x7f3ab40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:58.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3aca59c640 1 -- 172.21.15.83:0/1011950626 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3acc0af9c0 msgr2=0x7f3ab4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:58.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3aca59c640 1 --2- 172.21.15.83:0/1011950626 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3acc0af9c0 0x7f3ab4004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3aca59c640 1 -- 172.21.15.83:0/1011950626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ab4004d50 msgr2=0x7f3ab40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:58.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3aca59c640 1 --2- 172.21.15.83:0/1011950626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ab4004d50 0x7f3ab40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3aca59c640 1 -- 172.21.15.83:0/1011950626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab4003530 con 0x7f3ab4003a30 2024-04-17T10:38:58.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.851+0000 7f3acad9d640 1 --2- 172.21.15.83:0/1011950626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ab4004d50 0x7f3ab40051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:58.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.855+0000 7f3aca59c640 1 --2- 172.21.15.83:0/1011950626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ab4003a30 0x7f3ab4003e50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009ae0 tx=0x7f3abc01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5566bfd8440a690 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:58.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.855+0000 7f3ac959a640 1 -- 172.21.15.83:0/1011950626 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc0096b0 con 0x7f3ab4003a30 2024-04-17T10:38:58.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.855+0000 7f3ac959a640 1 -- 172.21.15.83:0/1011950626 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3abc0a6460 con 0x7f3ab4003a30 2024-04-17T10:38:58.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.855+0000 7f3ac959a640 1 -- 172.21.15.83:0/1011950626 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc0a6850 con 0x7f3ab4003a30 2024-04-17T10:38:58.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.855+0000 7f3ad0ff2640 1 -- 172.21.15.83:0/1011950626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ab4003a30 msgr2=0x7f3ab4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:58.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.855+0000 7f3ad0ff2640 1 --2- 172.21.15.83:0/1011950626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ab4003a30 0x7f3ab4003e50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009ae0 tx=0x7f3abc01cab0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.855+0000 7f3acad9d640 1 -- 172.21.15.83:0/1011950626 reap_dead start 2024-04-17T10:38:58.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.855+0000 7f3ad0ff2640 1 -- 172.21.15.83:0/1011950626 shutdown_connections 2024-04-17T10:38:58.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.855+0000 7f3ad0ff2640 1 -- 172.21.15.83:0/1011950626 >> 172.21.15.83:0/1011950626 conn(0x7f3acc0b23f0 msgr2=0x7f3acc0b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:58.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.855+0000 7f3ad0ff2640 1 -- 172.21.15.83:0/1011950626 shutdown_connections 2024-04-17T10:38:58.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.855+0000 7f3ad0ff2640 1 -- 172.21.15.83:0/1011950626 wait complete. 2024-04-17T10:38:58.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.855+0000 7f3ad0ff2640 1 Processor -- start 2024-04-17T10:38:58.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.855+0000 7f3ad0ff2640 1 -- start start 2024-04-17T10:38:58.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.855+0000 7f3ad0ff2640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ab4003a30 0x7f3acc15e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:58.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.855+0000 7f3ad0ff2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3acc159770 0x7f3acc142f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:58.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.855+0000 7f3ad0ff2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3acc15c330 0x7f3acc15c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:58.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.855+0000 7f3ad0ff2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc0cfaa0 con 0x7f3acc159770 2024-04-17T10:38:58.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.855+0000 7f3ad0ff2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc071df0 con 0x7f3acc15c330 2024-04-17T10:38:58.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.855+0000 7f3ad0ff2640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc159fa0 con 0x7f3ab4003a30 2024-04-17T10:38:58.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.855+0000 7f3ac9d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3acc159770 0x7f3acc142f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:58.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.859+0000 7f3ac9d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3acc159770 0x7f3acc142f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50134/0 (socket says 172.21.15.83:50134) 2024-04-17T10:38:58.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.859+0000 7f3ac9d9b640 1 -- 172.21.15.83:0/104346772 learned_addr learned my addr 172.21.15.83:0/104346772 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:58.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.859+0000 7f3acad9d640 1 --2- 172.21.15.83:0/104346772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3acc15c330 0x7f3acc15c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:58.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.859+0000 7f3aca59c640 1 --2- 172.21.15.83:0/104346772 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ab4003a30 0x7f3acc15e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:58.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.859+0000 7f3ac9d9b640 1 -- 172.21.15.83:0/104346772 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ab4003a30 msgr2=0x7f3acc15e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:58.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.859+0000 7f3ac9d9b640 1 --2- 172.21.15.83:0/104346772 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ab4003a30 0x7f3acc15e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.859+0000 7f3ac9d9b640 1 -- 172.21.15.83:0/104346772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3acc15c330 msgr2=0x7f3acc15c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:58.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.859+0000 7f3ac9d9b640 1 --2- 172.21.15.83:0/104346772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3acc15c330 0x7f3acc15c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:58.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.859+0000 7f3ac9d9b640 1 -- 172.21.15.83:0/104346772 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3acc15a280 con 0x7f3acc159770 2024-04-17T10:38:58.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.859+0000 7f3acad9d640 1 --2- 172.21.15.83:0/104346772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3acc15c330 0x7f3acc15c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:58.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.859+0000 7f3ac9d9b640 1 --2- 172.21.15.83:0/104346772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3acc159770 0x7f3acc142f60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0007fd0 tx=0x7f3ac000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:58.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.859+0000 7f3abaffd640 1 -- 172.21.15.83:0/104346772 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0019070 con 0x7f3acc159770 2024-04-17T10:38:58.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.859+0000 7f3abaffd640 1 -- 172.21.15.83:0/104346772 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac00072d0 con 0x7f3acc159770 2024-04-17T10:38:58.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.859+0000 7f3abaffd640 1 -- 172.21.15.83:0/104346772 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac00047f0 con 0x7f3acc159770 2024-04-17T10:38:58.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.859+0000 7f3ad0ff2640 1 -- 172.21.15.83:0/104346772 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3acc15a560 con 0x7f3acc159770 2024-04-17T10:38:58.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.859+0000 7f3ad0ff2640 1 -- 172.21.15.83:0/104346772 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3acc1434a0 con 0x7f3acc159770 2024-04-17T10:38:58.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.859+0000 7f3ad0ff2640 1 -- 172.21.15.83:0/104346772 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3acc15a730 con 0x7f3acc159770 2024-04-17T10:38:58.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.859+0000 7f3abaffd640 1 -- 172.21.15.83:0/104346772 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3ac0004ba0 con 0x7f3acc159770 2024-04-17T10:38:58.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.859+0000 7f3abaffd640 1 --2- 172.21.15.83:0/104346772 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3aac066f90 0x7f3aac069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:58.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.859+0000 7f3abaffd640 1 -- 172.21.15.83:0/104346772 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f3ac0084c20 con 0x7f3acc159770 2024-04-17T10:38:58.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.859+0000 7f3aca59c640 1 --2- 172.21.15.83:0/104346772 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3aac066f90 0x7f3aac069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:58.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.859+0000 7f3aca59c640 1 --2- 172.21.15.83:0/104346772 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3aac066f90 0x7f3aac069450 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3abc0ad000 tx=0x7f3abc0046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:58.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:58.863+0000 7f3abaffd640 1 -- 172.21.15.83:0/104346772 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3acc15a730 con 0x7f3acc159770 2024-04-17T10:38:59.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.019+0000 7f3ad0ff2640 1 -- 172.21.15.83:0/104346772 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3acc150cd0 con 0x7f3acc159770 2024-04-17T10:38:59.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.019+0000 7f3abaffd640 1 -- 172.21.15.83:0/104346772 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3ac0055bb0 con 0x7f3acc159770 2024-04-17T10:38:59.016 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:38:59.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.023+0000 7f3ad0ff2640 1 -- 172.21.15.83:0/104346772 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3aac066f90 msgr2=0x7f3aac069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:59.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.023+0000 7f3ad0ff2640 1 --2- 172.21.15.83:0/104346772 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3aac066f90 0x7f3aac069450 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3abc0ad000 tx=0x7f3abc0046d0 comp rx=0 tx=0).stop 2024-04-17T10:38:59.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.023+0000 7f3ad0ff2640 1 -- 172.21.15.83:0/104346772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3acc159770 msgr2=0x7f3acc142f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:59.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.023+0000 7f3ad0ff2640 1 --2- 172.21.15.83:0/104346772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3acc159770 0x7f3acc142f60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0007fd0 tx=0x7f3ac000db90 comp rx=0 tx=0).stop 2024-04-17T10:38:59.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.023+0000 7f3ad0ff2640 1 -- 172.21.15.83:0/104346772 shutdown_connections 2024-04-17T10:38:59.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.023+0000 7f3ad0ff2640 1 --2- 172.21.15.83:0/104346772 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3aac066f90 0x7f3aac069450 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:59.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.023+0000 7f3ad0ff2640 1 --2- 172.21.15.83:0/104346772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3acc15c330 0x7f3acc15c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:59.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.023+0000 7f3ad0ff2640 1 --2- 172.21.15.83:0/104346772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3acc159770 0x7f3acc142f60 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:59.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.023+0000 7f3ad0ff2640 1 --2- 172.21.15.83:0/104346772 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ab4003a30 0x7f3acc15e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:59.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.023+0000 7f3ad0ff2640 1 -- 172.21.15.83:0/104346772 >> 172.21.15.83:0/104346772 conn(0x7f3acc0b23f0 msgr2=0x7f3acc153810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:59.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.023+0000 7f3ad0ff2640 1 -- 172.21.15.83:0/104346772 shutdown_connections 2024-04-17T10:38:59.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.023+0000 7f3ad0ff2640 1 -- 172.21.15.83:0/104346772 wait complete. 2024-04-17T10:38:59.034 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:38:59.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.159+0000 7f507e34d640 1 Processor -- start 2024-04-17T10:38:59.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.159+0000 7f507e34d640 1 -- start start 2024-04-17T10:38:59.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.159+0000 7f507e34d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f50780d26c0 0x7f50780d17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:59.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.159+0000 7f507e34d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50780ce4e0 0x7f507815d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:59.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.159+0000 7f507e34d640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50780ce910 con 0x7f50780d0970 2024-04-17T10:38:59.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.159+0000 7f507e34d640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50780cf7d0 con 0x7f50780d26c0 2024-04-17T10:38:59.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.159+0000 7f507e34d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50780d4890 con 0x7f50780ce4e0 2024-04-17T10:38:59.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.159+0000 7f507c8ea640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50780ce4e0 0x7f507815d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:59.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.159+0000 7f506f7fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f50780d26c0 0x7f50780d17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:59.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.159+0000 7f5077fff640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f50780d0970 0x7f50780d1fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:55682/0 (socket says 172.21.15.83:55682) 2024-04-17T10:38:59.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.159+0000 7f5077fff640 1 -- 172.21.15.83:0/2050293653 learned_addr learned my addr 172.21.15.83:0/2050293653 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:38:59.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.159+0000 7f506f7fe640 1 -- 172.21.15.83:0/2050293653 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50780ce4e0 msgr2=0x7f507815d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:59.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.159+0000 7f506f7fe640 1 --2- 172.21.15.83:0/2050293653 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50780ce4e0 0x7f507815d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:59.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.159+0000 7f506f7fe640 1 -- 172.21.15.83:0/2050293653 >> v1:172.21.15.167:6789/0 conn(0x7f50780d0970 legacy=0x7f50780d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:59.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.159+0000 7f506f7fe640 1 -- 172.21.15.83:0/2050293653 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50780cebc0 con 0x7f50780d26c0 2024-04-17T10:38:59.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.159+0000 7f507c8ea640 1 --2- 172.21.15.83:0/2050293653 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50780ce4e0 0x7f507815d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:38:59.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.159+0000 7f50777fe640 1 -- 172.21.15.83:0/2050293653 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1571493070 0 0) 0x7f50780ce910 con 0x7f50780d0970 2024-04-17T10:38:59.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.163+0000 7f506f7fe640 1 --2- 172.21.15.83:0/2050293653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50780d26c0 0x7f50780d17b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f50600097c0 tx=0x7f506001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd93438648227a17 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:59.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.163+0000 7f50777fe640 1 -- 172.21.15.83:0/2050293653 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50600a7070 con 0x7f50780d26c0 2024-04-17T10:38:59.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.163+0000 7f50777fe640 1 -- 172.21.15.83:0/2050293653 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50600b0070 con 0x7f50780d26c0 2024-04-17T10:38:59.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.163+0000 7f50777fe640 1 -- 172.21.15.83:0/2050293653 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50600ab890 con 0x7f50780d26c0 2024-04-17T10:38:59.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.163+0000 7f507e34d640 1 -- 172.21.15.83:0/2050293653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50780d26c0 msgr2=0x7f50780d17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:59.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.163+0000 7f507e34d640 1 --2- 172.21.15.83:0/2050293653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50780d26c0 0x7f50780d17b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f50600097c0 tx=0x7f506001c470 comp rx=0 tx=0).stop 2024-04-17T10:38:59.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.163+0000 7f507e34d640 1 -- 172.21.15.83:0/2050293653 shutdown_connections 2024-04-17T10:38:59.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.163+0000 7f507e34d640 1 --2- 172.21.15.83:0/2050293653 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50780ce4e0 0x7f507815d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:59.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.163+0000 7f507e34d640 1 --2- 172.21.15.83:0/2050293653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50780d26c0 0x7f50780d17b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:59.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.163+0000 7f507e34d640 1 -- 172.21.15.83:0/2050293653 >> 172.21.15.83:0/2050293653 conn(0x7f507809cd50 msgr2=0x7f50780d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:59.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.163+0000 7f507e34d640 1 -- 172.21.15.83:0/2050293653 shutdown_connections 2024-04-17T10:38:59.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.163+0000 7f507e34d640 1 -- 172.21.15.83:0/2050293653 wait complete. 2024-04-17T10:38:59.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.163+0000 7f507e34d640 1 Processor -- start 2024-04-17T10:38:59.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.163+0000 7f507e34d640 1 -- start start 2024-04-17T10:38:59.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.163+0000 7f507e34d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50780ce4e0 0x7f50780b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:59.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.163+0000 7f507e34d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50780d0970 0x7f50781554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.163+0000 7f507e34d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50780d26c0 0x7f5078160620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.163+0000 7f507e34d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507806b2a0 con 0x7f50780d26c0 2024-04-17T10:38:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.163+0000 7f507e34d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50780d0d50 con 0x7f50780d0970 2024-04-17T10:38:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.167+0000 7f507e34d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f507806b420 con 0x7f50780ce4e0 2024-04-17T10:38:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.167+0000 7f507c8ea640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50780d26c0 0x7f5078160620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.167+0000 7f507c8ea640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50780d26c0 0x7f5078160620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50140/0 (socket says 172.21.15.83:50140) 2024-04-17T10:38:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.167+0000 7f507c8ea640 1 -- 172.21.15.83:0/3631442402 learned_addr learned my addr 172.21.15.83:0/3631442402 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.167+0000 7f506f7fe640 1 --2- 172.21.15.83:0/3631442402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50780d0970 0x7f50781554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.167+0000 7f5077fff640 1 --2- 172.21.15.83:0/3631442402 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50780ce4e0 0x7f50780b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.167+0000 7f507c8ea640 1 -- 172.21.15.83:0/3631442402 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50780ce4e0 msgr2=0x7f50780b8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.167+0000 7f507c8ea640 1 --2- 172.21.15.83:0/3631442402 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50780ce4e0 0x7f50780b8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:59.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.167+0000 7f507c8ea640 1 -- 172.21.15.83:0/3631442402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50780d0970 msgr2=0x7f50781554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:59.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.167+0000 7f507c8ea640 1 --2- 172.21.15.83:0/3631442402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50780d0970 0x7f50781554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:59.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.167+0000 7f507c8ea640 1 -- 172.21.15.83:0/3631442402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50780c8fc0 con 0x7f50780d26c0 2024-04-17T10:38:59.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.167+0000 7f507c8ea640 1 --2- 172.21.15.83:0/3631442402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50780d26c0 0x7f5078160620 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f507000efc0 tx=0x7f5070009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:59.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.167+0000 7f50757fa640 1 -- 172.21.15.83:0/3631442402 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5070018030 con 0x7f50780d26c0 2024-04-17T10:38:59.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.167+0000 7f50757fa640 1 -- 172.21.15.83:0/3631442402 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507000cde0 con 0x7f50780d26c0 2024-04-17T10:38:59.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.167+0000 7f507e34d640 1 -- 172.21.15.83:0/3631442402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50780c9260 con 0x7f50780d26c0 2024-04-17T10:38:59.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.167+0000 7f50757fa640 1 -- 172.21.15.83:0/3631442402 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5070003bc0 con 0x7f50780d26c0 2024-04-17T10:38:59.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.167+0000 7f507e34d640 1 -- 172.21.15.83:0/3631442402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50780b9220 con 0x7f50780d26c0 2024-04-17T10:38:59.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.167+0000 7f507e34d640 1 -- 172.21.15.83:0/3631442402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50780681a0 con 0x7f50780d26c0 2024-04-17T10:38:59.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.167+0000 7f50757fa640 1 -- 172.21.15.83:0/3631442402 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f507001b070 con 0x7f50780d26c0 2024-04-17T10:38:59.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.167+0000 7f50757fa640 1 --2- 172.21.15.83:0/3631442402 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5058066f60 0x7f5058069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:59.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.167+0000 7f5077fff640 1 --2- 172.21.15.83:0/3631442402 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5058066f60 0x7f5058069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:59.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.167+0000 7f50757fa640 1 -- 172.21.15.83:0/3631442402 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f5070085a10 con 0x7f50780d26c0 2024-04-17T10:38:59.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.167+0000 7f5077fff640 1 --2- 172.21.15.83:0/3631442402 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5058066f60 0x7f5058069420 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5068000f70 tx=0x7f5068007b80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:59.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.171+0000 7f50757fa640 1 -- 172.21.15.83:0/3631442402 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5070010040 con 0x7f50780d26c0 2024-04-17T10:38:59.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.331+0000 7f507e34d640 1 -- 172.21.15.83:0/3631442402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5078074f90 con 0x7f50780d26c0 2024-04-17T10:38:59.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.331+0000 7f50757fa640 1 -- 172.21.15.83:0/3631442402 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5070056a30 con 0x7f50780d26c0 2024-04-17T10:38:59.328 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:38:59.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.331+0000 7f507e34d640 1 -- 172.21.15.83:0/3631442402 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5058066f60 msgr2=0x7f5058069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:59.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.331+0000 7f507e34d640 1 --2- 172.21.15.83:0/3631442402 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5058066f60 0x7f5058069420 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5068000f70 tx=0x7f5068007b80 comp rx=0 tx=0).stop 2024-04-17T10:38:59.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.331+0000 7f507e34d640 1 -- 172.21.15.83:0/3631442402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50780d26c0 msgr2=0x7f5078160620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:59.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.331+0000 7f507e34d640 1 --2- 172.21.15.83:0/3631442402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50780d26c0 0x7f5078160620 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f507000efc0 tx=0x7f5070009f90 comp rx=0 tx=0).stop 2024-04-17T10:38:59.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.331+0000 7f507e34d640 1 -- 172.21.15.83:0/3631442402 shutdown_connections 2024-04-17T10:38:59.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.331+0000 7f507e34d640 1 --2- 172.21.15.83:0/3631442402 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5058066f60 0x7f5058069420 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:59.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.331+0000 7f507e34d640 1 --2- 172.21.15.83:0/3631442402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50780d26c0 0x7f5078160620 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:59.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.331+0000 7f507e34d640 1 --2- 172.21.15.83:0/3631442402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50780d0970 0x7f50781554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:59.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.331+0000 7f507e34d640 1 --2- 172.21.15.83:0/3631442402 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50780ce4e0 0x7f50780b8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:59.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.331+0000 7f507e34d640 1 -- 172.21.15.83:0/3631442402 >> 172.21.15.83:0/3631442402 conn(0x7f507809cd50 msgr2=0x7f50780b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:59.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.331+0000 7f507e34d640 1 -- 172.21.15.83:0/3631442402 shutdown_connections 2024-04-17T10:38:59.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.331+0000 7f507e34d640 1 -- 172.21.15.83:0/3631442402 wait complete. 2024-04-17T10:38:59.347 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:38:59.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.471+0000 7fa04507a640 1 Processor -- start 2024-04-17T10:38:59.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.471+0000 7fa04507a640 1 -- start start 2024-04-17T10:38:59.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.471+0000 7fa04507a640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa04015bac0 0x7fa040071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:59.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.471+0000 7fa04507a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa040070470 0x7fa04006ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:59.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.471+0000 7fa04507a640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa04006c1f0 con 0x7fa04015b6e0 2024-04-17T10:38:59.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.471+0000 7fa04507a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa040070a10 con 0x7fa040070470 2024-04-17T10:38:59.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.471+0000 7fa04507a640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa040070cf0 con 0x7fa04015bac0 2024-04-17T10:38:59.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.471+0000 7fa03f59e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa040070470 0x7fa04006ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:59.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.471+0000 7fa03f59e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa040070470 0x7fa04006ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50144/0 (socket says 172.21.15.83:50144) 2024-04-17T10:38:59.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.471+0000 7fa03f59e640 1 -- 172.21.15.83:0/3982433829 learned_addr learned my addr 172.21.15.83:0/3982433829 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:59.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.471+0000 7fa037fff640 1 --2- 172.21.15.83:0/3982433829 >> v2:172.21.15.167:3300/0 conn(0x7fa04015bac0 0x7fa040071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:59.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.471+0000 7fa03f59e640 1 -- 172.21.15.83:0/3982433829 >> v2:172.21.15.167:3300/0 conn(0x7fa04015bac0 msgr2=0x7fa040071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:59.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.471+0000 7fa03f59e640 1 --2- 172.21.15.83:0/3982433829 >> v2:172.21.15.167:3300/0 conn(0x7fa04015bac0 0x7fa040071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:59.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.471+0000 7fa03f59e640 1 -- 172.21.15.83:0/3982433829 >> v1:172.21.15.167:6789/0 conn(0x7fa04015b6e0 legacy=0x7fa0400cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:59.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.471+0000 7fa03f59e640 1 -- 172.21.15.83:0/3982433829 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa040070fd0 con 0x7fa040070470 2024-04-17T10:38:59.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.475+0000 7fa03f59e640 1 --2- 172.21.15.83:0/3982433829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa040070470 0x7fa04006ae10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa0400cd5e0 tx=0x7fa03801c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70ad44913c614c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:59.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.475+0000 7fa03e59c640 1 -- 172.21.15.83:0/3982433829 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0380a5650 con 0x7fa040070470 2024-04-17T10:38:59.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.475+0000 7fa03e59c640 1 -- 172.21.15.83:0/3982433829 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0380a57f0 con 0x7fa040070470 2024-04-17T10:38:59.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.475+0000 7fa03e59c640 1 -- 172.21.15.83:0/3982433829 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0380a5bc0 con 0x7fa040070470 2024-04-17T10:38:59.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.475+0000 7fa04507a640 1 -- 172.21.15.83:0/3982433829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa040070470 msgr2=0x7fa04006ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:59.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.475+0000 7fa04507a640 1 --2- 172.21.15.83:0/3982433829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa040070470 0x7fa04006ae10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa0400cd5e0 tx=0x7fa03801c990 comp rx=0 tx=0).stop 2024-04-17T10:38:59.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.475+0000 7fa04507a640 1 -- 172.21.15.83:0/3982433829 shutdown_connections 2024-04-17T10:38:59.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.475+0000 7fa04507a640 1 --2- 172.21.15.83:0/3982433829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa040070470 0x7fa04006ae10 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:59.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.475+0000 7fa04507a640 1 --2- 172.21.15.83:0/3982433829 >> v2:172.21.15.167:3300/0 conn(0x7fa04015bac0 0x7fa040071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:59.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.475+0000 7fa04507a640 1 -- 172.21.15.83:0/3982433829 >> 172.21.15.83:0/3982433829 conn(0x7fa0400b15b0 msgr2=0x7fa0400b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:59.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.475+0000 7fa04507a640 1 -- 172.21.15.83:0/3982433829 shutdown_connections 2024-04-17T10:38:59.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.475+0000 7fa04507a640 1 -- 172.21.15.83:0/3982433829 wait complete. 2024-04-17T10:38:59.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.475+0000 7fa04507a640 1 Processor -- start 2024-04-17T10:38:59.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.475+0000 7fa04507a640 1 -- start start 2024-04-17T10:38:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.479+0000 7fa04507a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa040070470 0x7fa04014a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.479+0000 7fa04507a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa04015b6e0 0x7fa04015d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.479+0000 7fa04507a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa04015bac0 0x7fa04014d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.479+0000 7fa04507a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04006d2b0 con 0x7fa040070470 2024-04-17T10:38:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.479+0000 7fa04507a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04006d130 con 0x7fa04015bac0 2024-04-17T10:38:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.479+0000 7fa04507a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa04006b760 con 0x7fa04015b6e0 2024-04-17T10:38:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.479+0000 7fa03ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa040070470 0x7fa04014a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.479+0000 7fa03ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa040070470 0x7fa04014a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50148/0 (socket says 172.21.15.83:50148) 2024-04-17T10:38:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.479+0000 7fa03ed9d640 1 -- 172.21.15.83:0/890520667 learned_addr learned my addr 172.21.15.83:0/890520667 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.479+0000 7fa03ed9d640 1 -- 172.21.15.83:0/890520667 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa04015b6e0 msgr2=0x7fa04015d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.479+0000 7fa037fff640 1 --2- 172.21.15.83:0/890520667 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa04015b6e0 0x7fa04015d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.479+0000 7fa03f59e640 1 --2- 172.21.15.83:0/890520667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa04015bac0 0x7fa04014d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.479+0000 7fa03ed9d640 1 --2- 172.21.15.83:0/890520667 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa04015b6e0 0x7fa04015d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.479+0000 7fa03ed9d640 1 -- 172.21.15.83:0/890520667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa04015bac0 msgr2=0x7fa04014d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.479+0000 7fa03ed9d640 1 --2- 172.21.15.83:0/890520667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa04015bac0 0x7fa04014d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.479+0000 7fa03ed9d640 1 -- 172.21.15.83:0/890520667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa04014b010 con 0x7fa040070470 2024-04-17T10:38:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.479+0000 7fa03ed9d640 1 --2- 172.21.15.83:0/890520667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa040070470 0x7fa04014a510 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa03000c450 tx=0x7fa03000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:59.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.479+0000 7fa0377fe640 1 -- 172.21.15.83:0/890520667 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa030012830 con 0x7fa040070470 2024-04-17T10:38:59.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.479+0000 7fa0377fe640 1 -- 172.21.15.83:0/890520667 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa030017070 con 0x7fa040070470 2024-04-17T10:38:59.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.479+0000 7fa04507a640 1 -- 172.21.15.83:0/890520667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa04015c050 con 0x7fa040070470 2024-04-17T10:38:59.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.479+0000 7fa0377fe640 1 -- 172.21.15.83:0/890520667 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa030012c30 con 0x7fa040070470 2024-04-17T10:38:59.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.479+0000 7fa04507a640 1 -- 172.21.15.83:0/890520667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa04015c610 con 0x7fa040070470 2024-04-17T10:38:59.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.479+0000 7fa04507a640 1 -- 172.21.15.83:0/890520667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa040074f90 con 0x7fa040070470 2024-04-17T10:38:59.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.479+0000 7fa0377fe640 1 -- 172.21.15.83:0/890520667 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa03001c460 con 0x7fa040070470 2024-04-17T10:38:59.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.479+0000 7fa0377fe640 1 --2- 172.21.15.83:0/890520667 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa020066f60 0x7fa020069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:59.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.479+0000 7fa037fff640 1 --2- 172.21.15.83:0/890520667 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa020066f60 0x7fa020069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:59.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.479+0000 7fa0377fe640 1 -- 172.21.15.83:0/890520667 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa030084e20 con 0x7fa040070470 2024-04-17T10:38:59.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.479+0000 7fa037fff640 1 --2- 172.21.15.83:0/890520667 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa020066f60 0x7fa020069420 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa0401587d0 tx=0x7fa028008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.483+0000 7fa0377fe640 1 -- 172.21.15.83:0/890520667 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa04014b010 con 0x7fa040070470 2024-04-17T10:38:59.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.651+0000 7fa04507a640 1 -- 172.21.15.83:0/890520667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa04014aa50 con 0x7fa040070470 2024-04-17T10:38:59.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.651+0000 7fa0377fe640 1 -- 172.21.15.83:0/890520667 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa030055db0 con 0x7fa040070470 2024-04-17T10:38:59.649 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-17T10:38:59.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.655+0000 7fa04507a640 1 -- 172.21.15.83:0/890520667 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa020066f60 msgr2=0x7fa020069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:59.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.655+0000 7fa04507a640 1 --2- 172.21.15.83:0/890520667 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa020066f60 0x7fa020069420 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa0401587d0 tx=0x7fa028008040 comp rx=0 tx=0).stop 2024-04-17T10:38:59.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.655+0000 7fa04507a640 1 -- 172.21.15.83:0/890520667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa040070470 msgr2=0x7fa04014a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:59.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.655+0000 7fa04507a640 1 --2- 172.21.15.83:0/890520667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa040070470 0x7fa04014a510 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa03000c450 tx=0x7fa03000c910 comp rx=0 tx=0).stop 2024-04-17T10:38:59.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.655+0000 7fa04507a640 1 -- 172.21.15.83:0/890520667 shutdown_connections 2024-04-17T10:38:59.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.655+0000 7fa04507a640 1 --2- 172.21.15.83:0/890520667 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa020066f60 0x7fa020069420 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:59.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.655+0000 7fa04507a640 1 --2- 172.21.15.83:0/890520667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa04015bac0 0x7fa04014d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:59.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.655+0000 7fa04507a640 1 --2- 172.21.15.83:0/890520667 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa04015b6e0 0x7fa04015d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:59.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.655+0000 7fa04507a640 1 --2- 172.21.15.83:0/890520667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa040070470 0x7fa04014a510 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:59.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.655+0000 7fa04507a640 1 -- 172.21.15.83:0/890520667 >> 172.21.15.83:0/890520667 conn(0x7fa0400b15b0 msgr2=0x7fa0400cfd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:59.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.655+0000 7fa04507a640 1 -- 172.21.15.83:0/890520667 shutdown_connections 2024-04-17T10:38:59.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.655+0000 7fa04507a640 1 -- 172.21.15.83:0/890520667 wait complete. 2024-04-17T10:38:59.666 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi167.front.sepia.ceph.com... 2024-04-17T10:38:59.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:59.667 DEBUG:teuthology.orchestra.run.smithi167:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-17T10:38:59.682 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:38:59.682 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-04-17T10:38:59.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:38:59.682 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-17T10:38:59.728 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-17T10:38:59.728 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:38:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.847+0000 7f3772f04640 1 Processor -- start 2024-04-17T10:38:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.847+0000 7f3772f04640 1 -- start start 2024-04-17T10:38:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.851+0000 7f3772f04640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f376c0b2d80 0x7f376c146e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.851+0000 7f3772f04640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f376c06d690 0x7f376c1620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.851+0000 7f3772f04640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f376c147570 con 0x7f376c146a00 2024-04-17T10:38:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.851+0000 7f3772f04640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f376c162610 con 0x7f376c0b2d80 2024-04-17T10:38:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.851+0000 7f3772f04640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f376c06c960 con 0x7f376c06d690 2024-04-17T10:38:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.851+0000 7f376bfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f376c0b2d80 0x7f376c146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.851+0000 7f376bfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f376c0b2d80 0x7f376c146e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50156/0 (socket says 172.21.15.83:50156) 2024-04-17T10:38:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.851+0000 7f376bfff640 1 -- 172.21.15.83:0/1727090847 learned_addr learned my addr 172.21.15.83:0/1727090847 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.851+0000 7f376bfff640 1 -- 172.21.15.83:0/1727090847 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f376c06d690 msgr2=0x7f376c1620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.851+0000 7f376bfff640 1 --2- 172.21.15.83:0/1727090847 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f376c06d690 0x7f376c1620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.851+0000 7f376bfff640 1 -- 172.21.15.83:0/1727090847 >> v1:172.21.15.167:6789/0 conn(0x7f376c146a00 legacy=0x7f376c0b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:38:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.851+0000 7f376bfff640 1 -- 172.21.15.83:0/1727090847 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f376c06cbb0 con 0x7f376c0b2d80 2024-04-17T10:38:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.851+0000 7f376bfff640 1 --2- 172.21.15.83:0/1727090847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f376c0b2d80 0x7f376c146e70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3760009870 tx=0x7f376001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e331f96d3f189bb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.851+0000 7f376b7fe640 1 -- 172.21.15.83:0/1727090847 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37600ab5f0 con 0x7f376c0b2d80 2024-04-17T10:38:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.851+0000 7f376b7fe640 1 -- 172.21.15.83:0/1727090847 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37600b0070 con 0x7f376c0b2d80 2024-04-17T10:38:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.851+0000 7f376b7fe640 1 -- 172.21.15.83:0/1727090847 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37600ab9b0 con 0x7f376c0b2d80 2024-04-17T10:38:59.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.851+0000 7f3772f04640 1 -- 172.21.15.83:0/1727090847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f376c0b2d80 msgr2=0x7f376c146e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:59.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.851+0000 7f3772f04640 1 --2- 172.21.15.83:0/1727090847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f376c0b2d80 0x7f376c146e70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3760009870 tx=0x7f376001c520 comp rx=0 tx=0).stop 2024-04-17T10:38:59.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.851+0000 7f3772f04640 1 -- 172.21.15.83:0/1727090847 shutdown_connections 2024-04-17T10:38:59.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.851+0000 7f3772f04640 1 --2- 172.21.15.83:0/1727090847 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f376c06d690 0x7f376c1620d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:59.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.851+0000 7f3772f04640 1 --2- 172.21.15.83:0/1727090847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f376c0b2d80 0x7f376c146e70 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:59.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.851+0000 7f3772f04640 1 -- 172.21.15.83:0/1727090847 >> 172.21.15.83:0/1727090847 conn(0x7f376c06abe0 msgr2=0x7f376c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:38:59.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.851+0000 7f3772f04640 1 -- 172.21.15.83:0/1727090847 shutdown_connections 2024-04-17T10:38:59.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.851+0000 7f3772f04640 1 -- 172.21.15.83:0/1727090847 wait complete. 2024-04-17T10:38:59.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.855+0000 7f3772f04640 1 Processor -- start 2024-04-17T10:38:59.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.855+0000 7f3772f04640 1 -- start start 2024-04-17T10:38:59.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.855+0000 7f3772f04640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f376c06d690 0x7f376c15b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:59.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.855+0000 7f3772f04640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f376c0b2d80 0x7f376c159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:59.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.855+0000 7f3772f04640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f376c146a00 0x7f376c15bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:59.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.855+0000 7f3772f04640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376c071010 con 0x7f376c146a00 2024-04-17T10:38:59.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.855+0000 7f3772f04640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376c070e90 con 0x7f376c0b2d80 2024-04-17T10:38:59.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.855+0000 7f3772f04640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f376c159860 con 0x7f376c06d690 2024-04-17T10:38:59.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.855+0000 7f37714a1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f376c146a00 0x7f376c15bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.855+0000 7f37714a1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f376c146a00 0x7f376c15bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50164/0 (socket says 172.21.15.83:50164) 2024-04-17T10:38:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.855+0000 7f37714a1640 1 -- 172.21.15.83:0/3475252560 learned_addr learned my addr 172.21.15.83:0/3475252560 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:38:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.855+0000 7f376bfff640 1 --2- 172.21.15.83:0/3475252560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f376c0b2d80 0x7f376c159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.855+0000 7f3770ca0640 1 --2- 172.21.15.83:0/3475252560 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f376c06d690 0x7f376c15b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.855+0000 7f37714a1640 1 -- 172.21.15.83:0/3475252560 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f376c06d690 msgr2=0x7f376c15b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.855+0000 7f37714a1640 1 --2- 172.21.15.83:0/3475252560 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f376c06d690 0x7f376c15b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.855+0000 7f37714a1640 1 -- 172.21.15.83:0/3475252560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f376c0b2d80 msgr2=0x7f376c159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:38:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.855+0000 7f37714a1640 1 --2- 172.21.15.83:0/3475252560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f376c0b2d80 0x7f376c159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:38:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.855+0000 7f37714a1640 1 -- 172.21.15.83:0/3475252560 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f376c15eac0 con 0x7f376c146a00 2024-04-17T10:38:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.855+0000 7f376bfff640 1 --2- 172.21.15.83:0/3475252560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f376c0b2d80 0x7f376c159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:38:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.855+0000 7f37714a1640 1 --2- 172.21.15.83:0/3475252560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f376c146a00 0x7f376c15bf10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f376400b430 tx=0x7f376400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.855+0000 7f37697fa640 1 -- 172.21.15.83:0/3475252560 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3764015020 con 0x7f376c146a00 2024-04-17T10:38:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.855+0000 7f37697fa640 1 -- 172.21.15.83:0/3475252560 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3764011830 con 0x7f376c146a00 2024-04-17T10:38:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.855+0000 7f3772f04640 1 -- 172.21.15.83:0/3475252560 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f376c15eda0 con 0x7f376c146a00 2024-04-17T10:38:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.855+0000 7f37697fa640 1 -- 172.21.15.83:0/3475252560 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3764011c20 con 0x7f376c146a00 2024-04-17T10:38:59.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.855+0000 7f3772f04640 1 -- 172.21.15.83:0/3475252560 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f376c159b40 con 0x7f376c146a00 2024-04-17T10:38:59.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.855+0000 7f3772f04640 1 -- 172.21.15.83:0/3475252560 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f376c074f90 con 0x7f376c146a00 2024-04-17T10:38:59.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.859+0000 7f37697fa640 1 -- 172.21.15.83:0/3475252560 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3764018020 con 0x7f376c146a00 2024-04-17T10:38:59.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.859+0000 7f37697fa640 1 --2- 172.21.15.83:0/3475252560 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3758066f60 0x7f3758069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:38:59.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.859+0000 7f37697fa640 1 -- 172.21.15.83:0/3475252560 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f3764085060 con 0x7f376c146a00 2024-04-17T10:38:59.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.859+0000 7f3770ca0640 1 --2- 172.21.15.83:0/3475252560 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3758066f60 0x7f3758069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:38:59.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.859+0000 7f3770ca0640 1 --2- 172.21.15.83:0/3475252560 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3758066f60 0x7f3758069420 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f375c007bf0 tx=0x7f375c007240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:38:59.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:38:59.863+0000 7f37697fa640 1 -- 172.21.15.83:0/3475252560 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f376401f020 con 0x7f376c146a00 2024-04-17T10:39:00.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.079+0000 7f3772f04640 1 -- 172.21.15.83:0/3475252560 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f376c074480 con 0x7f376c146a00 2024-04-17T10:39:00.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.079+0000 7f37697fa640 1 -- 172.21.15.83:0/3475252560 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4938 (secure 0 0 0) 0x7f37640172c0 con 0x7f376c146a00 2024-04-17T10:39:00.078 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:39:00.078 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2884469968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2884469968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2884469968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2919571009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2919571009},{"type":"v1","addr":"172.21.15.83:6836","nonce":2919571009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1635176834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1635176834},{"type":"v1","addr":"172.21.15.83:6837","nonce":1635176834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:38:12.032695+0000","modified":"2024-04-17T10:38:14.064978+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6835/3871588351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3871588351},{"type":"v1","addr":"172.21.15.167:6835","nonce":3871588351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-17T10:39:00.078 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 6 2024-04-17T10:39:00.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.083+0000 7f3772f04640 1 -- 172.21.15.83:0/3475252560 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3758066f60 msgr2=0x7f3758069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:00.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.083+0000 7f3772f04640 1 --2- 172.21.15.83:0/3475252560 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3758066f60 0x7f3758069420 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f375c007bf0 tx=0x7f375c007240 comp rx=0 tx=0).stop 2024-04-17T10:39:00.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.083+0000 7f3772f04640 1 -- 172.21.15.83:0/3475252560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f376c146a00 msgr2=0x7f376c15bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:00.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.083+0000 7f3772f04640 1 --2- 172.21.15.83:0/3475252560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f376c146a00 0x7f376c15bf10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f376400b430 tx=0x7f376400b8f0 comp rx=0 tx=0).stop 2024-04-17T10:39:00.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.083+0000 7f3772f04640 1 -- 172.21.15.83:0/3475252560 shutdown_connections 2024-04-17T10:39:00.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.083+0000 7f3772f04640 1 --2- 172.21.15.83:0/3475252560 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3758066f60 0x7f3758069420 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:00.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.083+0000 7f3772f04640 1 --2- 172.21.15.83:0/3475252560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f376c146a00 0x7f376c15bf10 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:00.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.083+0000 7f3772f04640 1 --2- 172.21.15.83:0/3475252560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f376c0b2d80 0x7f376c159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:00.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.083+0000 7f3772f04640 1 --2- 172.21.15.83:0/3475252560 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f376c06d690 0x7f376c15b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:00.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.083+0000 7f3772f04640 1 -- 172.21.15.83:0/3475252560 >> 172.21.15.83:0/3475252560 conn(0x7f376c06abe0 msgr2=0x7f376c13fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:00.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.083+0000 7f3772f04640 1 -- 172.21.15.83:0/3475252560 shutdown_connections 2024-04-17T10:39:00.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.083+0000 7f3772f04640 1 -- 172.21.15.83:0/3475252560 wait complete. 2024-04-17T10:39:00.096 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:39:00.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.211+0000 7f43898ce640 1 Processor -- start 2024-04-17T10:39:00.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.211+0000 7f43898ce640 1 -- start start 2024-04-17T10:39:00.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.211+0000 7f43898ce640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4384161300 0x7f438406ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:00.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.211+0000 7f43898ce640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f438406c300 0x7f4384071750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:00.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.211+0000 7f43898ce640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4384161900 con 0x7f438406f360 2024-04-17T10:39:00.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.211+0000 7f43898ce640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4384161ba0 con 0x7f4384161300 2024-04-17T10:39:00.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.211+0000 7f43898ce640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4384161e80 con 0x7f438406c300 2024-04-17T10:39:00.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.211+0000 7f4382ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4384161300 0x7f438406ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:00.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.211+0000 7f4382ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4384161300 0x7f438406ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50180/0 (socket says 172.21.15.83:50180) 2024-04-17T10:39:00.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.211+0000 7f4382ffd640 1 -- 172.21.15.83:0/2515984760 learned_addr learned my addr 172.21.15.83:0/2515984760 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:00.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.211+0000 7f43837fe640 1 --2- 172.21.15.83:0/2515984760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f438406c300 0x7f4384071750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:00.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.211+0000 7f4382ffd640 1 -- 172.21.15.83:0/2515984760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f438406c300 msgr2=0x7f4384071750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:00.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.211+0000 7f4382ffd640 1 --2- 172.21.15.83:0/2515984760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f438406c300 0x7f4384071750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:00.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.211+0000 7f4382ffd640 1 -- 172.21.15.83:0/2515984760 >> v1:172.21.15.83:6789/0 conn(0x7f438406f360 legacy=0x7f438406bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:00.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.211+0000 7f4382ffd640 1 -- 172.21.15.83:0/2515984760 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4384162160 con 0x7f4384161300 2024-04-17T10:39:00.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.211+0000 7f4382ffd640 1 --2- 172.21.15.83:0/2515984760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4384161300 0x7f438406ee20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4374009870 tx=0x7f437401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12708205e70efc4d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:00.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f4381ffb640 1 -- 172.21.15.83:0/2515984760 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43740b0070 con 0x7f4384161300 2024-04-17T10:39:00.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f4381ffb640 1 -- 172.21.15.83:0/2515984760 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43740ab600 con 0x7f4384161300 2024-04-17T10:39:00.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f4381ffb640 1 -- 172.21.15.83:0/2515984760 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43740ab9d0 con 0x7f4384161300 2024-04-17T10:39:00.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f43898ce640 1 -- 172.21.15.83:0/2515984760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4384161300 msgr2=0x7f438406ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:00.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f43898ce640 1 --2- 172.21.15.83:0/2515984760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4384161300 0x7f438406ee20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4374009870 tx=0x7f437401c4d0 comp rx=0 tx=0).stop 2024-04-17T10:39:00.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f43898ce640 1 -- 172.21.15.83:0/2515984760 shutdown_connections 2024-04-17T10:39:00.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f43898ce640 1 --2- 172.21.15.83:0/2515984760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f438406c300 0x7f4384071750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:00.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f43898ce640 1 --2- 172.21.15.83:0/2515984760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4384161300 0x7f438406ee20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:00.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f43898ce640 1 -- 172.21.15.83:0/2515984760 >> 172.21.15.83:0/2515984760 conn(0x7f43840cc910 msgr2=0x7f43840ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:00.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f43898ce640 1 -- 172.21.15.83:0/2515984760 shutdown_connections 2024-04-17T10:39:00.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f43898ce640 1 -- 172.21.15.83:0/2515984760 wait complete. 2024-04-17T10:39:00.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f43898ce640 1 Processor -- start 2024-04-17T10:39:00.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f43898ce640 1 -- start start 2024-04-17T10:39:00.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f43898ce640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f438406c300 0x7f4384154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:00.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f43898ce640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f438406f360 0x7f4384158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:00.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f43898ce640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4384161300 0x7f4384156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:00.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f43898ce640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4384072870 con 0x7f438406f360 2024-04-17T10:39:00.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f43898ce640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43840726f0 con 0x7f4384161300 2024-04-17T10:39:00.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f43898ce640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43840729f0 con 0x7f438406c300 2024-04-17T10:39:00.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f43827fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f438406f360 0x7f4384158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:00.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f43827fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f438406f360 0x7f4384158240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50192/0 (socket says 172.21.15.83:50192) 2024-04-17T10:39:00.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f43827fc640 1 -- 172.21.15.83:0/1372497265 learned_addr learned my addr 172.21.15.83:0/1372497265 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:00.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f43837fe640 1 --2- 172.21.15.83:0/1372497265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4384161300 0x7f4384156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:00.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f4382ffd640 1 --2- 172.21.15.83:0/1372497265 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f438406c300 0x7f4384154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:00.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f43827fc640 1 -- 172.21.15.83:0/1372497265 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f438406c300 msgr2=0x7f4384154f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:00.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f43827fc640 1 --2- 172.21.15.83:0/1372497265 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f438406c300 0x7f4384154f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:00.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f43827fc640 1 -- 172.21.15.83:0/1372497265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4384161300 msgr2=0x7f4384156730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:00.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f43827fc640 1 --2- 172.21.15.83:0/1372497265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4384161300 0x7f4384156730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:00.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f43827fc640 1 -- 172.21.15.83:0/1372497265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43840c6910 con 0x7f438406f360 2024-04-17T10:39:00.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f43827fc640 1 --2- 172.21.15.83:0/1372497265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f438406f360 0x7f4384158240 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f437800c4d0 tx=0x7f437800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:00.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f436bfff640 1 -- 172.21.15.83:0/1372497265 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43780052d0 con 0x7f438406f360 2024-04-17T10:39:00.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f436bfff640 1 -- 172.21.15.83:0/1372497265 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4378005470 con 0x7f438406f360 2024-04-17T10:39:00.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f43898ce640 1 -- 172.21.15.83:0/1372497265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f438409f380 con 0x7f438406f360 2024-04-17T10:39:00.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f436bfff640 1 -- 172.21.15.83:0/1372497265 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4378012830 con 0x7f438406f360 2024-04-17T10:39:00.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.215+0000 7f43898ce640 1 -- 172.21.15.83:0/1372497265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f438409f940 con 0x7f438406f360 2024-04-17T10:39:00.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.219+0000 7f43898ce640 1 -- 172.21.15.83:0/1372497265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4384074f90 con 0x7f438406f360 2024-04-17T10:39:00.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.219+0000 7f436bfff640 1 -- 172.21.15.83:0/1372497265 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f43780129d0 con 0x7f438406f360 2024-04-17T10:39:00.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.219+0000 7f436bfff640 1 --2- 172.21.15.83:0/1372497265 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4370066f60 0x7f4370069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:00.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.219+0000 7f436bfff640 1 -- 172.21.15.83:0/1372497265 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f4378084ca0 con 0x7f438406f360 2024-04-17T10:39:00.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.219+0000 7f4382ffd640 1 --2- 172.21.15.83:0/1372497265 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4370066f60 0x7f4370069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:00.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.219+0000 7f4382ffd640 1 --2- 172.21.15.83:0/1372497265 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4370066f60 0x7f4370069420 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f43740ad000 tx=0x7f4374008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:00.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.223+0000 7f436bfff640 1 -- 172.21.15.83:0/1372497265 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4378055c30 con 0x7f438406f360 2024-04-17T10:39:00.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.371+0000 7f43898ce640 1 -- 172.21.15.83:0/1372497265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4384155450 con 0x7f438406f360 2024-04-17T10:39:00.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.371+0000 7f436bfff640 1 -- 172.21.15.83:0/1372497265 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16782 (secure 0 0 0) 0x7f4378055530 con 0x7f438406f360 2024-04-17T10:39:00.369 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:39:00.369 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":22,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:38:53.385002+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:38:10.464490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T10:38:11.043134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:39:00.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.375+0000 7f43898ce640 1 -- 172.21.15.83:0/1372497265 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4370066f60 msgr2=0x7f4370069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:00.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.375+0000 7f43898ce640 1 --2- 172.21.15.83:0/1372497265 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4370066f60 0x7f4370069420 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f43740ad000 tx=0x7f4374008040 comp rx=0 tx=0).stop 2024-04-17T10:39:00.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.375+0000 7f43898ce640 1 -- 172.21.15.83:0/1372497265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f438406f360 msgr2=0x7f4384158240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:00.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.375+0000 7f43898ce640 1 --2- 172.21.15.83:0/1372497265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f438406f360 0x7f4384158240 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f437800c4d0 tx=0x7f437800c990 comp rx=0 tx=0).stop 2024-04-17T10:39:00.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.375+0000 7f43898ce640 1 -- 172.21.15.83:0/1372497265 shutdown_connections 2024-04-17T10:39:00.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.375+0000 7f43898ce640 1 --2- 172.21.15.83:0/1372497265 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4370066f60 0x7f4370069420 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:00.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.375+0000 7f43898ce640 1 --2- 172.21.15.83:0/1372497265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4384161300 0x7f4384156730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:00.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.375+0000 7f43898ce640 1 --2- 172.21.15.83:0/1372497265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f438406f360 0x7f4384158240 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:00.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.375+0000 7f43898ce640 1 --2- 172.21.15.83:0/1372497265 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f438406c300 0x7f4384154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:00.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.375+0000 7f43898ce640 1 -- 172.21.15.83:0/1372497265 >> 172.21.15.83:0/1372497265 conn(0x7f43840cc910 msgr2=0x7f438406ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:00.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.375+0000 7f43898ce640 1 -- 172.21.15.83:0/1372497265 shutdown_connections 2024-04-17T10:39:00.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.375+0000 7f43898ce640 1 -- 172.21.15.83:0/1372497265 wait complete. 2024-04-17T10:39:00.386 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:38:10.464490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:38:11.043134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:39:00.387 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:39:00.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.507+0000 7fcca184d640 1 Processor -- start 2024-04-17T10:39:00.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.507+0000 7fcca184d640 1 -- start start 2024-04-17T10:39:00.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.507+0000 7fcca184d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc9c15e0f0 0x7fcc9c0b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:00.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.507+0000 7fcca184d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc9c15e4d0 con 0x7fcc9c0b5450 2024-04-17T10:39:00.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.507+0000 7fcca184d640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc9c162580 con 0x7fcc9c0ce400 2024-04-17T10:39:00.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.507+0000 7fcca184d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9c162b40 con 0x7fcc9c15e0f0 2024-04-17T10:39:00.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.507+0000 7fcc9b7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcc9c0b5450 0x7fcc9c0d2540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54730/0 (socket says 172.21.15.83:54730) 2024-04-17T10:39:00.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.507+0000 7fcc9b7fe640 1 -- 172.21.15.83:0/772227096 learned_addr learned my addr 172.21.15.83:0/772227096 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:39:00.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.507+0000 7fcc9affd640 1 --2- 172.21.15.83:0/772227096 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc9c15e0f0 0x7fcc9c0b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:00.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.507+0000 7fcc9a7fc640 1 -- 172.21.15.83:0/772227096 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 118411136 0 0) 0x7fcc9c15e4d0 con 0x7fcc9c0b5450 2024-04-17T10:39:00.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.507+0000 7fcc9a7fc640 1 -- 172.21.15.83:0/772227096 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc88003530 con 0x7fcc9c0b5450 2024-04-17T10:39:00.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.507+0000 7fcc9a7fc640 1 -- 172.21.15.83:0/772227096 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 638876258 0 0) 0x7fcc88003530 con 0x7fcc9c0b5450 2024-04-17T10:39:00.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.507+0000 7fcc9a7fc640 1 -- 172.21.15.83:0/772227096 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc9c15e0f0 msgr2=0x7fcc9c0b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:00.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.507+0000 7fcc9a7fc640 1 --2- 172.21.15.83:0/772227096 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc9c15e0f0 0x7fcc9c0b6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:00.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.507+0000 7fcc9a7fc640 1 -- 172.21.15.83:0/772227096 >> v1:172.21.15.167:6789/0 conn(0x7fcc9c0ce400 legacy=0x7fcc9c161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:00.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.507+0000 7fcc9a7fc640 1 -- 172.21.15.83:0/772227096 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc9c162860 con 0x7fcc9c0b5450 2024-04-17T10:39:00.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.507+0000 7fcc9a7fc640 1 -- 172.21.15.83:0/772227096 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fcc94001a00 con 0x7fcc9c0b5450 2024-04-17T10:39:00.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.507+0000 7fcc9a7fc640 1 -- 172.21.15.83:0/772227096 >> v1:172.21.15.83:6789/0 conn(0x7fcc9c0b5450 legacy=0x7fcc9c0d2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:00.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.507+0000 7fcc9a7fc640 1 --2- 172.21.15.83:0/772227096 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc9c15e0f0 0x7fcc88003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:00.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.507+0000 7fcc9a7fc640 1 --2- 172.21.15.83:0/772227096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc88003fa0 0x7fcc88004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:00.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.507+0000 7fcc9a7fc640 1 --2- 172.21.15.83:0/772227096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc88004d50 0x7fcc880051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:00.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.507+0000 7fcc9a7fc640 1 -- 172.21.15.83:0/772227096 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9c162b40 con 0x7fcc88004d50 2024-04-17T10:39:00.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.507+0000 7fcc9a7fc640 1 -- 172.21.15.83:0/772227096 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8800bb60 con 0x7fcc88003fa0 2024-04-17T10:39:00.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.507+0000 7fcc9a7fc640 1 -- 172.21.15.83:0/772227096 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8800bd30 con 0x7fcc9c15e0f0 2024-04-17T10:39:00.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.507+0000 7fcc9b7fe640 1 --2- 172.21.15.83:0/772227096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc88004d50 0x7fcc880051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:00.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.507+0000 7fcc93fff640 1 --2- 172.21.15.83:0/772227096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc88003fa0 0x7fcc88004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:00.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.507+0000 7fcc9b7fe640 1 -- 172.21.15.83:0/772227096 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc9c15e0f0 msgr2=0x7fcc88003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:00.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.507+0000 7fcc9b7fe640 1 --2- 172.21.15.83:0/772227096 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc9c15e0f0 0x7fcc88003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:00.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.507+0000 7fcc9b7fe640 1 -- 172.21.15.83:0/772227096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc88003fa0 msgr2=0x7fcc88004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:00.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.507+0000 7fcc9b7fe640 1 --2- 172.21.15.83:0/772227096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc88003fa0 0x7fcc88004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:00.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.507+0000 7fcc9b7fe640 1 -- 172.21.15.83:0/772227096 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc88003530 con 0x7fcc88004d50 2024-04-17T10:39:00.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.511+0000 7fcc9b7fe640 1 --2- 172.21.15.83:0/772227096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc88004d50 0x7fcc880051d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400a920 tx=0x7fcc9401d420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677f7d0fa7dac763 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:00.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.511+0000 7fcc9a7fc640 1 -- 172.21.15.83:0/772227096 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9400a4c0 con 0x7fcc88004d50 2024-04-17T10:39:00.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.511+0000 7fcc9a7fc640 1 -- 172.21.15.83:0/772227096 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc940ad650 con 0x7fcc88004d50 2024-04-17T10:39:00.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.511+0000 7fcc9a7fc640 1 -- 172.21.15.83:0/772227096 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc940ada80 con 0x7fcc88004d50 2024-04-17T10:39:00.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.511+0000 7fcca184d640 1 -- 172.21.15.83:0/772227096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc88004d50 msgr2=0x7fcc880051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:00.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.511+0000 7fcca184d640 1 --2- 172.21.15.83:0/772227096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc88004d50 0x7fcc880051d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400a920 tx=0x7fcc9401d420 comp rx=0 tx=0).stop 2024-04-17T10:39:00.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.511+0000 7fcc9b7fe640 1 -- 172.21.15.83:0/772227096 reap_dead start 2024-04-17T10:39:00.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.511+0000 7fcca184d640 1 -- 172.21.15.83:0/772227096 shutdown_connections 2024-04-17T10:39:00.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.511+0000 7fcca184d640 1 -- 172.21.15.83:0/772227096 >> 172.21.15.83:0/772227096 conn(0x7fcc9c0d0450 msgr2=0x7fcc9c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:00.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.511+0000 7fcca184d640 1 -- 172.21.15.83:0/772227096 shutdown_connections 2024-04-17T10:39:00.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.511+0000 7fcca184d640 1 -- 172.21.15.83:0/772227096 wait complete. 2024-04-17T10:39:00.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.511+0000 7fcca184d640 1 Processor -- start 2024-04-17T10:39:00.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.511+0000 7fcca184d640 1 -- start start 2024-04-17T10:39:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.515+0000 7fcca184d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc88004d50 0x7fcc9c156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.515+0000 7fcca184d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc9c165930 0x7fcc9c166f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.515+0000 7fcca184d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc9c1569c0 0x7fcc9c167480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.515+0000 7fcca184d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9c0b5e00 con 0x7fcc88004d50 2024-04-17T10:39:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.515+0000 7fcca184d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9c0b5c80 con 0x7fcc9c1569c0 2024-04-17T10:39:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.515+0000 7fcca184d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9c169f50 con 0x7fcc9c165930 2024-04-17T10:39:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.515+0000 7fcc9affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc88004d50 0x7fcc9c156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.515+0000 7fcc9affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc88004d50 0x7fcc9c156200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50212/0 (socket says 172.21.15.83:50212) 2024-04-17T10:39:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.515+0000 7fcc9affd640 1 -- 172.21.15.83:0/2511705918 learned_addr learned my addr 172.21.15.83:0/2511705918 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.515+0000 7fcc9b7fe640 1 --2- 172.21.15.83:0/2511705918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc9c1569c0 0x7fcc9c167480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.515+0000 7fcc93fff640 1 --2- 172.21.15.83:0/2511705918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc9c165930 0x7fcc9c166f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.515+0000 7fcc9affd640 1 -- 172.21.15.83:0/2511705918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc9c165930 msgr2=0x7fcc9c166f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.515+0000 7fcc9affd640 1 --2- 172.21.15.83:0/2511705918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc9c165930 0x7fcc9c166f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.515+0000 7fcc9affd640 1 -- 172.21.15.83:0/2511705918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc9c1569c0 msgr2=0x7fcc9c167480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.515+0000 7fcc9affd640 1 --2- 172.21.15.83:0/2511705918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc9c1569c0 0x7fcc9c167480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.515+0000 7fcc9affd640 1 -- 172.21.15.83:0/2511705918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc9c165ea0 con 0x7fcc88004d50 2024-04-17T10:39:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.515+0000 7fcc9affd640 1 --2- 172.21.15.83:0/2511705918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc88004d50 0x7fcc9c156200 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00e7f0 tx=0x7fcc8c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.515+0000 7fcc98ff9640 1 -- 172.21.15.83:0/2511705918 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c017070 con 0x7fcc88004d50 2024-04-17T10:39:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.515+0000 7fcc98ff9640 1 -- 172.21.15.83:0/2511705918 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8c00c8c0 con 0x7fcc88004d50 2024-04-17T10:39:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.515+0000 7fcca184d640 1 -- 172.21.15.83:0/2511705918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc9c166180 con 0x7fcc88004d50 2024-04-17T10:39:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.515+0000 7fcc98ff9640 1 -- 172.21.15.83:0/2511705918 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c00cc90 con 0x7fcc88004d50 2024-04-17T10:39:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.515+0000 7fcca184d640 1 -- 172.21.15.83:0/2511705918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc9c166740 con 0x7fcc88004d50 2024-04-17T10:39:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.515+0000 7fcca184d640 1 -- 172.21.15.83:0/2511705918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc9c0681a0 con 0x7fcc88004d50 2024-04-17T10:39:00.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.515+0000 7fcc98ff9640 1 -- 172.21.15.83:0/2511705918 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcc8c01a070 con 0x7fcc88004d50 2024-04-17T10:39:00.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.515+0000 7fcc98ff9640 1 --2- 172.21.15.83:0/2511705918 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcc7c066f90 0x7fcc7c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:00.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.515+0000 7fcc98ff9640 1 -- 172.21.15.83:0/2511705918 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fcc8c084f00 con 0x7fcc88004d50 2024-04-17T10:39:00.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.515+0000 7fcc93fff640 1 --2- 172.21.15.83:0/2511705918 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcc7c066f90 0x7fcc7c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:00.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.515+0000 7fcc93fff640 1 --2- 172.21.15.83:0/2511705918 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcc7c066f90 0x7fcc7c069450 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c0b94f0 tx=0x7fcc84007b80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:00.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.519+0000 7fcc98ff9640 1 -- 172.21.15.83:0/2511705918 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcc8c055e90 con 0x7fcc88004d50 2024-04-17T10:39:00.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.675+0000 7fcca184d640 1 -- 172.21.15.83:0/2511705918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcc9c074f90 con 0x7fcc88004d50 2024-04-17T10:39:00.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.675+0000 7fcc98ff9640 1 -- 172.21.15.83:0/2511705918 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcc8c055790 con 0x7fcc88004d50 2024-04-17T10:39:00.674 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:39:00.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.679+0000 7fcca184d640 1 -- 172.21.15.83:0/2511705918 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcc7c066f90 msgr2=0x7fcc7c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:00.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.679+0000 7fcca184d640 1 --2- 172.21.15.83:0/2511705918 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcc7c066f90 0x7fcc7c069450 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c0b94f0 tx=0x7fcc84007b80 comp rx=0 tx=0).stop 2024-04-17T10:39:00.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.679+0000 7fcca184d640 1 -- 172.21.15.83:0/2511705918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc88004d50 msgr2=0x7fcc9c156200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:00.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.679+0000 7fcca184d640 1 --2- 172.21.15.83:0/2511705918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc88004d50 0x7fcc9c156200 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00e7f0 tx=0x7fcc8c00ecb0 comp rx=0 tx=0).stop 2024-04-17T10:39:00.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.679+0000 7fcca184d640 1 -- 172.21.15.83:0/2511705918 shutdown_connections 2024-04-17T10:39:00.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.679+0000 7fcca184d640 1 --2- 172.21.15.83:0/2511705918 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcc7c066f90 0x7fcc7c069450 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:00.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.679+0000 7fcca184d640 1 --2- 172.21.15.83:0/2511705918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc9c1569c0 0x7fcc9c167480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:00.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.679+0000 7fcca184d640 1 --2- 172.21.15.83:0/2511705918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc9c165930 0x7fcc9c166f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:00.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.679+0000 7fcca184d640 1 --2- 172.21.15.83:0/2511705918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc88004d50 0x7fcc9c156200 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:00.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.679+0000 7fcca184d640 1 -- 172.21.15.83:0/2511705918 >> 172.21.15.83:0/2511705918 conn(0x7fcc9c0d0450 msgr2=0x7fcc9c15e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:00.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.679+0000 7fcca184d640 1 -- 172.21.15.83:0/2511705918 shutdown_connections 2024-04-17T10:39:00.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.679+0000 7fcca184d640 1 -- 172.21.15.83:0/2511705918 wait complete. 2024-04-17T10:39:00.692 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:39:00.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.815+0000 7ffab1bd3640 1 Processor -- start 2024-04-17T10:39:00.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.815+0000 7ffab1bd3640 1 -- start start 2024-04-17T10:39:00.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.815+0000 7ffab1bd3640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffaac06b2f0 0x7ffaac06d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:00.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.815+0000 7ffab1bd3640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaac06ee40 con 0x7ffaac0aff10 2024-04-17T10:39:00.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.815+0000 7ffab1bd3640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaac06f120 con 0x7ffaac0d0db0 2024-04-17T10:39:00.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.815+0000 7ffab1bd3640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaac06ebf0 con 0x7ffaac06b2f0 2024-04-17T10:39:00.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.815+0000 7ffaab7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ffaac0aff10 0x7ffaac0b3780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54740/0 (socket says 172.21.15.83:54740) 2024-04-17T10:39:00.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.815+0000 7ffaab7fe640 1 -- 172.21.15.83:0/55916808 learned_addr learned my addr 172.21.15.83:0/55916808 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:39:00.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.815+0000 7ffaabfff640 1 --2- 172.21.15.83:0/55916808 >> v2:172.21.15.83:3300/0 conn(0x7ffaac06b2f0 0x7ffaac06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:00.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.815+0000 7ffaaa7fc640 1 -- 172.21.15.83:0/55916808 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 671944306 0 0) 0x7ffaac06ee40 con 0x7ffaac0aff10 2024-04-17T10:39:00.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.815+0000 7ffaaa7fc640 1 -- 172.21.15.83:0/55916808 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa94003530 con 0x7ffaac0aff10 2024-04-17T10:39:00.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.815+0000 7ffaabfff640 1 -- 172.21.15.83:0/55916808 >> v1:172.21.15.167:6789/0 conn(0x7ffaac0d0db0 legacy=0x7ffaac06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:00.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.815+0000 7ffaabfff640 1 -- 172.21.15.83:0/55916808 >> v1:172.21.15.83:6789/0 conn(0x7ffaac0aff10 legacy=0x7ffaac0b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:00.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.815+0000 7ffaabfff640 1 -- 172.21.15.83:0/55916808 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaac0d05b0 con 0x7ffaac06b2f0 2024-04-17T10:39:00.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.819+0000 7ffaabfff640 1 --2- 172.21.15.83:0/55916808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaac06b2f0 0x7ffaac06d220 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ffaac06ed70 tx=0x7ffa9c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200c314480bcef79 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:00.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.819+0000 7ffaaa7fc640 1 -- 172.21.15.83:0/55916808 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c0b0070 con 0x7ffaac06b2f0 2024-04-17T10:39:00.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.819+0000 7ffaaa7fc640 1 -- 172.21.15.83:0/55916808 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9c0ab600 con 0x7ffaac06b2f0 2024-04-17T10:39:00.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.819+0000 7ffaaa7fc640 1 -- 172.21.15.83:0/55916808 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c0ab9d0 con 0x7ffaac06b2f0 2024-04-17T10:39:00.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.819+0000 7ffab1bd3640 1 -- 172.21.15.83:0/55916808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaac06b2f0 msgr2=0x7ffaac06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:00.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.819+0000 7ffab1bd3640 1 --2- 172.21.15.83:0/55916808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaac06b2f0 0x7ffaac06d220 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ffaac06ed70 tx=0x7ffa9c01c460 comp rx=0 tx=0).stop 2024-04-17T10:39:00.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.819+0000 7ffab1bd3640 1 -- 172.21.15.83:0/55916808 shutdown_connections 2024-04-17T10:39:00.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.819+0000 7ffab1bd3640 1 --2- 172.21.15.83:0/55916808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaac06b2f0 0x7ffaac06d220 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:00.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.819+0000 7ffab1bd3640 1 -- 172.21.15.83:0/55916808 >> 172.21.15.83:0/55916808 conn(0x7ffaac06bcf0 msgr2=0x7ffaac06c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:00.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.819+0000 7ffab1bd3640 1 -- 172.21.15.83:0/55916808 shutdown_connections 2024-04-17T10:39:00.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.819+0000 7ffab1bd3640 1 -- 172.21.15.83:0/55916808 wait complete. 2024-04-17T10:39:00.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.819+0000 7ffab1bd3640 1 Processor -- start 2024-04-17T10:39:00.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.819+0000 7ffab1bd3640 1 -- start start 2024-04-17T10:39:00.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.819+0000 7ffab1bd3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaac06b2f0 0x7ffaac159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:00.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.819+0000 7ffab1bd3640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffaac0aff10 0x7ffaac160350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:00.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.819+0000 7ffab1bd3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffaac0d0db0 0x7ffaac160890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:00.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.819+0000 7ffab1bd3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac16c590 con 0x7ffaac06b2f0 2024-04-17T10:39:00.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.819+0000 7ffab1bd3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac16c410 con 0x7ffaac0d0db0 2024-04-17T10:39:00.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.819+0000 7ffab1bd3640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac16c710 con 0x7ffaac0aff10 2024-04-17T10:39:00.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.819+0000 7ffaab7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaac06b2f0 0x7ffaac159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:00.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.819+0000 7ffaab7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaac06b2f0 0x7ffaac159e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50228/0 (socket says 172.21.15.83:50228) 2024-04-17T10:39:00.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.819+0000 7ffaab7fe640 1 -- 172.21.15.83:0/2033519274 learned_addr learned my addr 172.21.15.83:0/2033519274 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:00.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.819+0000 7ffaabfff640 1 --2- 172.21.15.83:0/2033519274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffaac0d0db0 0x7ffaac160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:00.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.819+0000 7ffaaaffd640 1 --2- 172.21.15.83:0/2033519274 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffaac0aff10 0x7ffaac160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:00.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.819+0000 7ffaab7fe640 1 -- 172.21.15.83:0/2033519274 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffaac0aff10 msgr2=0x7ffaac160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:00.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.819+0000 7ffaab7fe640 1 --2- 172.21.15.83:0/2033519274 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffaac0aff10 0x7ffaac160350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:00.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.819+0000 7ffaab7fe640 1 -- 172.21.15.83:0/2033519274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffaac0d0db0 msgr2=0x7ffaac160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:00.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.819+0000 7ffaab7fe640 1 --2- 172.21.15.83:0/2033519274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffaac0d0db0 0x7ffaac160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:00.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.819+0000 7ffaab7fe640 1 -- 172.21.15.83:0/2033519274 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaac15edb0 con 0x7ffaac06b2f0 2024-04-17T10:39:00.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.819+0000 7ffaabfff640 1 --2- 172.21.15.83:0/2033519274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffaac0d0db0 0x7ffaac160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:00.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.823+0000 7ffaab7fe640 1 --2- 172.21.15.83:0/2033519274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaac06b2f0 0x7ffaac159e50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ffaa400c450 tx=0x7ffaa400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:00.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.823+0000 7ffa93fff640 1 -- 172.21.15.83:0/2033519274 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4017070 con 0x7ffaac06b2f0 2024-04-17T10:39:00.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.823+0000 7ffa93fff640 1 -- 172.21.15.83:0/2033519274 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa4005cf0 con 0x7ffaac06b2f0 2024-04-17T10:39:00.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.823+0000 7ffab1bd3640 1 -- 172.21.15.83:0/2033519274 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaac13edd0 con 0x7ffaac06b2f0 2024-04-17T10:39:00.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.823+0000 7ffa93fff640 1 -- 172.21.15.83:0/2033519274 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4007290 con 0x7ffaac06b2f0 2024-04-17T10:39:00.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.823+0000 7ffab1bd3640 1 -- 172.21.15.83:0/2033519274 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaac15e5b0 con 0x7ffaac06b2f0 2024-04-17T10:39:00.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.823+0000 7ffab1bd3640 1 -- 172.21.15.83:0/2033519274 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaac074f90 con 0x7ffaac06b2f0 2024-04-17T10:39:00.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.823+0000 7ffa93fff640 1 -- 172.21.15.83:0/2033519274 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffaa401a070 con 0x7ffaac06b2f0 2024-04-17T10:39:00.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.823+0000 7ffa93fff640 1 --2- 172.21.15.83:0/2033519274 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffa98066f60 0x7ffa98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:00.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.823+0000 7ffa93fff640 1 -- 172.21.15.83:0/2033519274 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7ffaa40843c0 con 0x7ffaac06b2f0 2024-04-17T10:39:00.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.823+0000 7ffaaaffd640 1 --2- 172.21.15.83:0/2033519274 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffa98066f60 0x7ffa98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:00.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.823+0000 7ffaaaffd640 1 --2- 172.21.15.83:0/2033519274 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffa98066f60 0x7ffa98069420 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ffaac0d0580 tx=0x7ffaa0007240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:00.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.827+0000 7ffa93fff640 1 -- 172.21.15.83:0/2033519274 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffaa40553e0 con 0x7ffaac06b2f0 2024-04-17T10:39:00.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.983+0000 7ffab1bd3640 1 -- 172.21.15.83:0/2033519274 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffaac0afa90 con 0x7ffaac06b2f0 2024-04-17T10:39:00.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.983+0000 7ffa93fff640 1 -- 172.21.15.83:0/2033519274 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffaa4054ce0 con 0x7ffaac06b2f0 2024-04-17T10:39:00.982 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:39:00.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.987+0000 7ffab1bd3640 1 -- 172.21.15.83:0/2033519274 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffa98066f60 msgr2=0x7ffa98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:00.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.987+0000 7ffab1bd3640 1 --2- 172.21.15.83:0/2033519274 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffa98066f60 0x7ffa98069420 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ffaac0d0580 tx=0x7ffaa0007240 comp rx=0 tx=0).stop 2024-04-17T10:39:00.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.987+0000 7ffab1bd3640 1 -- 172.21.15.83:0/2033519274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaac06b2f0 msgr2=0x7ffaac159e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:00.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.987+0000 7ffab1bd3640 1 --2- 172.21.15.83:0/2033519274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaac06b2f0 0x7ffaac159e50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ffaa400c450 tx=0x7ffaa400c910 comp rx=0 tx=0).stop 2024-04-17T10:39:00.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.987+0000 7ffab1bd3640 1 -- 172.21.15.83:0/2033519274 shutdown_connections 2024-04-17T10:39:00.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.987+0000 7ffab1bd3640 1 --2- 172.21.15.83:0/2033519274 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffa98066f60 0x7ffa98069420 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:00.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.987+0000 7ffab1bd3640 1 --2- 172.21.15.83:0/2033519274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffaac0d0db0 0x7ffaac160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:00.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.987+0000 7ffab1bd3640 1 --2- 172.21.15.83:0/2033519274 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffaac0aff10 0x7ffaac160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:00.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.987+0000 7ffab1bd3640 1 --2- 172.21.15.83:0/2033519274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaac06b2f0 0x7ffaac159e50 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:00.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.987+0000 7ffab1bd3640 1 -- 172.21.15.83:0/2033519274 >> 172.21.15.83:0/2033519274 conn(0x7ffaac06bcf0 msgr2=0x7ffaac0b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:00.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.987+0000 7ffab1bd3640 1 -- 172.21.15.83:0/2033519274 shutdown_connections 2024-04-17T10:39:00.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:00.987+0000 7ffab1bd3640 1 -- 172.21.15.83:0/2033519274 wait complete. 2024-04-17T10:39:01.001 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:39:01.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa6590b6640 1 Processor -- start 2024-04-17T10:39:01.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa6590b6640 1 -- start start 2024-04-17T10:39:01.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa6590b6640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa6540b5b60 0x7fa6540b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:01.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa6590b6640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa65415e4d0 con 0x7fa65415e0f0 2024-04-17T10:39:01.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa6590b6640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa654162580 con 0x7fa6540ce400 2024-04-17T10:39:01.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa6590b6640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa654162b40 con 0x7fa6540b5b60 2024-04-17T10:39:01.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa652d9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa65415e0f0 0x7fa654161c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54744/0 (socket says 172.21.15.83:54744) 2024-04-17T10:39:01.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa652d9d640 1 -- 172.21.15.83:0/3019657214 learned_addr learned my addr 172.21.15.83:0/3019657214 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:39:01.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa651d9b640 1 -- 172.21.15.83:0/3019657214 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2684637255 0 0) 0x7fa65415e4d0 con 0x7fa65415e0f0 2024-04-17T10:39:01.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa65359e640 1 --2- 172.21.15.83:0/3019657214 >> v2:172.21.15.167:3300/0 conn(0x7fa6540b5b60 0x7fa6540b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:01.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa651d9b640 1 -- 172.21.15.83:0/3019657214 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa63c003530 con 0x7fa65415e0f0 2024-04-17T10:39:01.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa651d9b640 1 -- 172.21.15.83:0/3019657214 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2446017223 0 0) 0x7fa63c003530 con 0x7fa65415e0f0 2024-04-17T10:39:01.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa651d9b640 1 -- 172.21.15.83:0/3019657214 >> v2:172.21.15.167:3300/0 conn(0x7fa6540b5b60 msgr2=0x7fa6540b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:01.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa651d9b640 1 --2- 172.21.15.83:0/3019657214 >> v2:172.21.15.167:3300/0 conn(0x7fa6540b5b60 0x7fa6540b6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:01.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa651d9b640 1 -- 172.21.15.83:0/3019657214 >> v1:172.21.15.167:6789/0 conn(0x7fa6540ce400 legacy=0x7fa6540b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:01.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa651d9b640 1 -- 172.21.15.83:0/3019657214 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa654162860 con 0x7fa65415e0f0 2024-04-17T10:39:01.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa651d9b640 1 -- 172.21.15.83:0/3019657214 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fa644001680 con 0x7fa65415e0f0 2024-04-17T10:39:01.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa651d9b640 1 -- 172.21.15.83:0/3019657214 >> v1:172.21.15.83:6789/0 conn(0x7fa65415e0f0 legacy=0x7fa654161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:01.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa651d9b640 1 --2- 172.21.15.83:0/3019657214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa63c003a30 0x7fa63c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:01.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa651d9b640 1 --2- 172.21.15.83:0/3019657214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa63c004810 0x7fa63c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:01.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa651d9b640 1 --2- 172.21.15.83:0/3019657214 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa63c005240 0x7fa63c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:01.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa651d9b640 1 -- 172.21.15.83:0/3019657214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa654162b40 con 0x7fa63c004810 2024-04-17T10:39:01.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa651d9b640 1 -- 172.21.15.83:0/3019657214 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c00c1a0 con 0x7fa63c003a30 2024-04-17T10:39:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa651d9b640 1 -- 172.21.15.83:0/3019657214 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c00c3a0 con 0x7fa63c005240 2024-04-17T10:39:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa65259c640 1 --2- 172.21.15.83:0/3019657214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa63c004810 0x7fa63c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa652d9d640 1 --2- 172.21.15.83:0/3019657214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa63c003a30 0x7fa63c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa65359e640 1 --2- 172.21.15.83:0/3019657214 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa63c005240 0x7fa63c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa65259c640 1 -- 172.21.15.83:0/3019657214 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa63c005240 msgr2=0x7fa63c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa65259c640 1 --2- 172.21.15.83:0/3019657214 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa63c005240 0x7fa63c00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa65259c640 1 -- 172.21.15.83:0/3019657214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa63c003a30 msgr2=0x7fa63c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa65259c640 1 --2- 172.21.15.83:0/3019657214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa63c003a30 0x7fa63c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa65259c640 1 -- 172.21.15.83:0/3019657214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63c003530 con 0x7fa63c004810 2024-04-17T10:39:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa652d9d640 1 --2- 172.21.15.83:0/3019657214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa63c003a30 0x7fa63c003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.127+0000 7fa65359e640 1 -- 172.21.15.83:0/3019657214 reap_dead start 2024-04-17T10:39:01.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.131+0000 7fa65259c640 1 --2- 172.21.15.83:0/3019657214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa63c004810 0x7fa63c004c90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa648001750 tx=0x7fa64801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d05c6e758aa2f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:01.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.131+0000 7fa651d9b640 1 -- 172.21.15.83:0/3019657214 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6480013a0 con 0x7fa63c004810 2024-04-17T10:39:01.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.131+0000 7fa651d9b640 1 -- 172.21.15.83:0/3019657214 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6480052f0 con 0x7fa63c004810 2024-04-17T10:39:01.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.131+0000 7fa651d9b640 1 -- 172.21.15.83:0/3019657214 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa648003840 con 0x7fa63c004810 2024-04-17T10:39:01.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.131+0000 7fa6590b6640 1 -- 172.21.15.83:0/3019657214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa63c004810 msgr2=0x7fa63c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:01.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.131+0000 7fa6590b6640 1 --2- 172.21.15.83:0/3019657214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa63c004810 0x7fa63c004c90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa648001750 tx=0x7fa64801c390 comp rx=0 tx=0).stop 2024-04-17T10:39:01.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.131+0000 7fa6590b6640 1 -- 172.21.15.83:0/3019657214 shutdown_connections 2024-04-17T10:39:01.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.131+0000 7fa6590b6640 1 --2- 172.21.15.83:0/3019657214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa63c004810 0x7fa63c004c90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:01.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.131+0000 7fa6590b6640 1 -- 172.21.15.83:0/3019657214 >> 172.21.15.83:0/3019657214 conn(0x7fa6540d0450 msgr2=0x7fa6540d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:01.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.131+0000 7fa6590b6640 1 -- 172.21.15.83:0/3019657214 shutdown_connections 2024-04-17T10:39:01.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.131+0000 7fa6590b6640 1 -- 172.21.15.83:0/3019657214 wait complete. 2024-04-17T10:39:01.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.131+0000 7fa6590b6640 1 Processor -- start 2024-04-17T10:39:01.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.131+0000 7fa6590b6640 1 -- start start 2024-04-17T10:39:01.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.131+0000 7fa6590b6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa63c004810 0x7fa654154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:01.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.131+0000 7fa6590b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6540ca100 0x7fa6541608c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:01.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.131+0000 7fa6590b6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6540cc9d0 0x7fa654160e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:01.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.131+0000 7fa6590b6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6540b59e0 con 0x7fa6540ca100 2024-04-17T10:39:01.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.131+0000 7fa6590b6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6540b5860 con 0x7fa63c004810 2024-04-17T10:39:01.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.131+0000 7fa6590b6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6540d2540 con 0x7fa6540cc9d0 2024-04-17T10:39:01.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.131+0000 7fa65259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6540ca100 0x7fa6541608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:01.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.131+0000 7fa65259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6540ca100 0x7fa6541608c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50250/0 (socket says 172.21.15.83:50250) 2024-04-17T10:39:01.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.131+0000 7fa65259c640 1 -- 172.21.15.83:0/1651300183 learned_addr learned my addr 172.21.15.83:0/1651300183 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:01.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.135+0000 7fa65359e640 1 --2- 172.21.15.83:0/1651300183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6540cc9d0 0x7fa654160e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:01.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.135+0000 7fa652d9d640 1 --2- 172.21.15.83:0/1651300183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa63c004810 0x7fa654154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:01.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.135+0000 7fa65259c640 1 -- 172.21.15.83:0/1651300183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6540cc9d0 msgr2=0x7fa654160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:01.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.135+0000 7fa65259c640 1 --2- 172.21.15.83:0/1651300183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6540cc9d0 0x7fa654160e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:01.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.135+0000 7fa65259c640 1 -- 172.21.15.83:0/1651300183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa63c004810 msgr2=0x7fa654154bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:01.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.135+0000 7fa65259c640 1 --2- 172.21.15.83:0/1651300183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa63c004810 0x7fa654154bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:01.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.135+0000 7fa65259c640 1 -- 172.21.15.83:0/1651300183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6541f7380 con 0x7fa6540ca100 2024-04-17T10:39:01.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.135+0000 7fa65259c640 1 --2- 172.21.15.83:0/1651300183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6540ca100 0x7fa6541608c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa64801c1c0 tx=0x7fa63c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:01.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.135+0000 7fa63b7fe640 1 -- 172.21.15.83:0/1651300183 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6480b8020 con 0x7fa6540ca100 2024-04-17T10:39:01.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.135+0000 7fa63b7fe640 1 -- 172.21.15.83:0/1651300183 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6480054a0 con 0x7fa6540ca100 2024-04-17T10:39:01.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.135+0000 7fa6590b6640 1 -- 172.21.15.83:0/1651300183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6541552b0 con 0x7fa6540ca100 2024-04-17T10:39:01.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.135+0000 7fa63b7fe640 1 -- 172.21.15.83:0/1651300183 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa648007a40 con 0x7fa6540ca100 2024-04-17T10:39:01.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.135+0000 7fa6590b6640 1 -- 172.21.15.83:0/1651300183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa654155870 con 0x7fa6540ca100 2024-04-17T10:39:01.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.135+0000 7fa6590b6640 1 -- 172.21.15.83:0/1651300183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6540681a0 con 0x7fa6540ca100 2024-04-17T10:39:01.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.135+0000 7fa63b7fe640 1 -- 172.21.15.83:0/1651300183 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa6480bb020 con 0x7fa6540ca100 2024-04-17T10:39:01.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.135+0000 7fa63b7fe640 1 --2- 172.21.15.83:0/1651300183 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa640066f60 0x7fa640069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:01.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.135+0000 7fa63b7fe640 1 -- 172.21.15.83:0/1651300183 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa64811ee20 con 0x7fa6540ca100 2024-04-17T10:39:01.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.135+0000 7fa652d9d640 1 --2- 172.21.15.83:0/1651300183 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa640066f60 0x7fa640069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:01.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.135+0000 7fa652d9d640 1 --2- 172.21.15.83:0/1651300183 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa640066f60 0x7fa640069420 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa644001930 tx=0x7fa644009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:01.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.139+0000 7fa63b7fe640 1 -- 172.21.15.83:0/1651300183 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa6480c2030 con 0x7fa6540ca100 2024-04-17T10:39:01.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.295+0000 7fa6590b6640 1 -- 172.21.15.83:0/1651300183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa654074f90 con 0x7fa6540ca100 2024-04-17T10:39:01.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.299+0000 7fa63b7fe640 1 -- 172.21.15.83:0/1651300183 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa6480160d0 con 0x7fa6540ca100 2024-04-17T10:39:01.295 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:39:01.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.299+0000 7fa6590b6640 1 -- 172.21.15.83:0/1651300183 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa640066f60 msgr2=0x7fa640069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:01.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.299+0000 7fa6590b6640 1 --2- 172.21.15.83:0/1651300183 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa640066f60 0x7fa640069420 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa644001930 tx=0x7fa644009040 comp rx=0 tx=0).stop 2024-04-17T10:39:01.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.299+0000 7fa6590b6640 1 -- 172.21.15.83:0/1651300183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6540ca100 msgr2=0x7fa6541608c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:01.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.299+0000 7fa6590b6640 1 --2- 172.21.15.83:0/1651300183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6540ca100 0x7fa6541608c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa64801c1c0 tx=0x7fa63c005840 comp rx=0 tx=0).stop 2024-04-17T10:39:01.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.299+0000 7fa6590b6640 1 -- 172.21.15.83:0/1651300183 shutdown_connections 2024-04-17T10:39:01.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.299+0000 7fa6590b6640 1 --2- 172.21.15.83:0/1651300183 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa640066f60 0x7fa640069420 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:01.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.299+0000 7fa6590b6640 1 --2- 172.21.15.83:0/1651300183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6540cc9d0 0x7fa654160e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:01.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.299+0000 7fa6590b6640 1 --2- 172.21.15.83:0/1651300183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6540ca100 0x7fa6541608c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:01.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.299+0000 7fa6590b6640 1 --2- 172.21.15.83:0/1651300183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa63c004810 0x7fa654154bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:01.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.299+0000 7fa6590b6640 1 -- 172.21.15.83:0/1651300183 >> 172.21.15.83:0/1651300183 conn(0x7fa6540d0450 msgr2=0x7fa65415d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:01.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.299+0000 7fa6590b6640 1 -- 172.21.15.83:0/1651300183 shutdown_connections 2024-04-17T10:39:01.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.299+0000 7fa6590b6640 1 -- 172.21.15.83:0/1651300183 wait complete. 2024-04-17T10:39:01.315 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:39:01.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.439+0000 7f6191dc3640 1 Processor -- start 2024-04-17T10:39:01.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.439+0000 7f6191dc3640 1 -- start start 2024-04-17T10:39:01.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.439+0000 7f6191dc3640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f618c072330 0x7f618c0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:01.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.439+0000 7f6191dc3640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f618c0cfa10 0x7f618c06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:01.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.439+0000 7f6191dc3640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f618c0cfea0 con 0x7f618c15ba30 2024-04-17T10:39:01.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.439+0000 7f6191dc3640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f618c071ea0 con 0x7f618c0cfa10 2024-04-17T10:39:01.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.439+0000 7f6191dc3640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618c06e320 con 0x7f618c072330 2024-04-17T10:39:01.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.439+0000 7f618bfff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f618c0cfa10 0x7f618c06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:01.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.439+0000 7f6183fff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f618c072330 0x7f618c0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:01.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.439+0000 7f618bfff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f618c0cfa10 0x7f618c06c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:44528/0 (socket says 172.21.15.83:44528) 2024-04-17T10:39:01.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.439+0000 7f618bfff640 1 -- 172.21.15.83:0/2872677700 learned_addr learned my addr 172.21.15.83:0/2872677700 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:01.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.439+0000 7f618bfff640 1 -- 172.21.15.83:0/2872677700 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f618c072330 msgr2=0x7f618c0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:01.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.439+0000 7f618bfff640 1 --2- 172.21.15.83:0/2872677700 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f618c072330 0x7f618c0cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:01.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.439+0000 7f618bfff640 1 -- 172.21.15.83:0/2872677700 >> v1:172.21.15.167:6789/0 conn(0x7f618c15ba30 legacy=0x7f618c15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:01.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.439+0000 7f618bfff640 1 -- 172.21.15.83:0/2872677700 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618c06e040 con 0x7f618c0cfa10 2024-04-17T10:39:01.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.443+0000 7f618bfff640 1 --2- 172.21.15.83:0/2872677700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f618c0cfa10 0x7f618c06c490 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6184009e50 tx=0x7f618401c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec1dc7827db7378e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:01.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.443+0000 7f618affd640 1 -- 172.21.15.83:0/2872677700 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61840a5770 con 0x7f618c0cfa10 2024-04-17T10:39:01.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.443+0000 7f618affd640 1 -- 172.21.15.83:0/2872677700 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61840a5910 con 0x7f618c0cfa10 2024-04-17T10:39:01.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.443+0000 7f618affd640 1 -- 172.21.15.83:0/2872677700 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61840a5ce0 con 0x7f618c0cfa10 2024-04-17T10:39:01.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.443+0000 7f6191dc3640 1 -- 172.21.15.83:0/2872677700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f618c0cfa10 msgr2=0x7f618c06c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:01.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.443+0000 7f6191dc3640 1 --2- 172.21.15.83:0/2872677700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f618c0cfa10 0x7f618c06c490 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6184009e50 tx=0x7f618401c9a0 comp rx=0 tx=0).stop 2024-04-17T10:39:01.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.443+0000 7f6191dc3640 1 -- 172.21.15.83:0/2872677700 shutdown_connections 2024-04-17T10:39:01.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.443+0000 7f6191dc3640 1 --2- 172.21.15.83:0/2872677700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f618c0cfa10 0x7f618c06c490 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:01.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.443+0000 7f6191dc3640 1 --2- 172.21.15.83:0/2872677700 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f618c072330 0x7f618c0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:01.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.443+0000 7f6191dc3640 1 -- 172.21.15.83:0/2872677700 >> 172.21.15.83:0/2872677700 conn(0x7f618c06abe0 msgr2=0x7f618c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:01.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.443+0000 7f6191dc3640 1 -- 172.21.15.83:0/2872677700 shutdown_connections 2024-04-17T10:39:01.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.443+0000 7f6191dc3640 1 -- 172.21.15.83:0/2872677700 wait complete. 2024-04-17T10:39:01.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.443+0000 7f6191dc3640 1 Processor -- start 2024-04-17T10:39:01.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.443+0000 7f6191dc3640 1 -- start start 2024-04-17T10:39:01.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.443+0000 7f6191dc3640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f618c072330 0x7f618c14ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:01.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.443+0000 7f6191dc3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f618c0cfa10 0x7f618c0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:01.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.443+0000 7f6191dc3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f618c15ba30 0x7f618c0add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:01.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.447+0000 7f6191dc3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618c06de20 con 0x7f618c0cfa10 2024-04-17T10:39:01.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.447+0000 7f6191dc3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618c06dca0 con 0x7f618c15ba30 2024-04-17T10:39:01.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.447+0000 7f6191dc3640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618c168520 con 0x7f618c072330 2024-04-17T10:39:01.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.447+0000 7f6183fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f618c0cfa10 0x7f618c0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:01.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.447+0000 7f6183fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f618c0cfa10 0x7f618c0ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50262/0 (socket says 172.21.15.83:50262) 2024-04-17T10:39:01.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.447+0000 7f6183fff640 1 -- 172.21.15.83:0/1630645449 learned_addr learned my addr 172.21.15.83:0/1630645449 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:01.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.447+0000 7f618bfff640 1 --2- 172.21.15.83:0/1630645449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f618c15ba30 0x7f618c0add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:01.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.447+0000 7f618b7fe640 1 --2- 172.21.15.83:0/1630645449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f618c072330 0x7f618c14ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:01.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.447+0000 7f6183fff640 1 -- 172.21.15.83:0/1630645449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f618c072330 msgr2=0x7f618c14ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:01.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.447+0000 7f6183fff640 1 --2- 172.21.15.83:0/1630645449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f618c072330 0x7f618c14ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:01.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.447+0000 7f6183fff640 1 -- 172.21.15.83:0/1630645449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f618c15ba30 msgr2=0x7f618c0add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:01.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.447+0000 7f6183fff640 1 --2- 172.21.15.83:0/1630645449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f618c15ba30 0x7f618c0add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:01.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.447+0000 7f6183fff640 1 -- 172.21.15.83:0/1630645449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618c159430 con 0x7f618c0cfa10 2024-04-17T10:39:01.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.447+0000 7f618b7fe640 1 --2- 172.21.15.83:0/1630645449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f618c072330 0x7f618c14ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:01.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.447+0000 7f6183fff640 1 --2- 172.21.15.83:0/1630645449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f618c0cfa10 0x7f618c0ad840 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f617400e870 tx=0x7f617400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:01.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.447+0000 7f6188ff9640 1 -- 172.21.15.83:0/1630645449 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6174017070 con 0x7f618c0cfa10 2024-04-17T10:39:01.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.447+0000 7f6188ff9640 1 -- 172.21.15.83:0/1630645449 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f617400c940 con 0x7f618c0cfa10 2024-04-17T10:39:01.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.447+0000 7f6191dc3640 1 -- 172.21.15.83:0/1630645449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f618c0ce6e0 con 0x7f618c0cfa10 2024-04-17T10:39:01.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.447+0000 7f6188ff9640 1 -- 172.21.15.83:0/1630645449 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617400ccf0 con 0x7f618c0cfa10 2024-04-17T10:39:01.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.447+0000 7f6191dc3640 1 -- 172.21.15.83:0/1630645449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f618c14f380 con 0x7f618c0cfa10 2024-04-17T10:39:01.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.447+0000 7f6191dc3640 1 -- 172.21.15.83:0/1630645449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f618c0b2770 con 0x7f618c0cfa10 2024-04-17T10:39:01.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.447+0000 7f6188ff9640 1 -- 172.21.15.83:0/1630645449 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f617401a070 con 0x7f618c0cfa10 2024-04-17T10:39:01.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.447+0000 7f6188ff9640 1 --2- 172.21.15.83:0/1630645449 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f616c066f60 0x7f616c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:01.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.447+0000 7f618b7fe640 1 --2- 172.21.15.83:0/1630645449 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f616c066f60 0x7f616c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:01.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.447+0000 7f6188ff9640 1 -- 172.21.15.83:0/1630645449 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f61740852d0 con 0x7f618c0cfa10 2024-04-17T10:39:01.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.447+0000 7f618b7fe640 1 --2- 172.21.15.83:0/1630645449 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f616c066f60 0x7f616c069420 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f618c158ba0 tx=0x7f617c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:01.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.451+0000 7f6188ff9640 1 -- 172.21.15.83:0/1630645449 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f618c0b2770 con 0x7f618c0cfa10 2024-04-17T10:39:01.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.627+0000 7f6191dc3640 1 -- 172.21.15.83:0/1630645449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f618c155cd0 con 0x7f618c0cfa10 2024-04-17T10:39:01.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.631+0000 7f6188ff9640 1 -- 172.21.15.83:0/1630645449 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4938 (secure 0 0 0) 0x7f618c159430 con 0x7f618c0cfa10 2024-04-17T10:39:01.627 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:39:01.627 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2884469968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2884469968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2884469968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2919571009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2919571009},{"type":"v1","addr":"172.21.15.83:6836","nonce":2919571009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1635176834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1635176834},{"type":"v1","addr":"172.21.15.83:6837","nonce":1635176834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:38:12.032695+0000","modified":"2024-04-17T10:38:14.064978+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6835/3871588351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3871588351},{"type":"v1","addr":"172.21.15.167:6835","nonce":3871588351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-17T10:39:01.627 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 6 2024-04-17T10:39:01.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.631+0000 7f6191dc3640 1 -- 172.21.15.83:0/1630645449 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f616c066f60 msgr2=0x7f616c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:01.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.631+0000 7f6191dc3640 1 --2- 172.21.15.83:0/1630645449 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f616c066f60 0x7f616c069420 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f618c158ba0 tx=0x7f617c009040 comp rx=0 tx=0).stop 2024-04-17T10:39:01.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.631+0000 7f6191dc3640 1 -- 172.21.15.83:0/1630645449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f618c0cfa10 msgr2=0x7f618c0ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:01.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.631+0000 7f6191dc3640 1 --2- 172.21.15.83:0/1630645449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f618c0cfa10 0x7f618c0ad840 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f617400e870 tx=0x7f617400ec70 comp rx=0 tx=0).stop 2024-04-17T10:39:01.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.631+0000 7f6191dc3640 1 -- 172.21.15.83:0/1630645449 shutdown_connections 2024-04-17T10:39:01.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.631+0000 7f6191dc3640 1 --2- 172.21.15.83:0/1630645449 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f616c066f60 0x7f616c069420 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:01.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.631+0000 7f6191dc3640 1 --2- 172.21.15.83:0/1630645449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f618c15ba30 0x7f618c0add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:01.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.631+0000 7f6191dc3640 1 --2- 172.21.15.83:0/1630645449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f618c0cfa10 0x7f618c0ad840 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:01.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.631+0000 7f6191dc3640 1 --2- 172.21.15.83:0/1630645449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f618c072330 0x7f618c14ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:01.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.631+0000 7f6191dc3640 1 -- 172.21.15.83:0/1630645449 >> 172.21.15.83:0/1630645449 conn(0x7f618c06abe0 msgr2=0x7f618c070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:01.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.631+0000 7f6191dc3640 1 -- 172.21.15.83:0/1630645449 shutdown_connections 2024-04-17T10:39:01.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.631+0000 7f6191dc3640 1 -- 172.21.15.83:0/1630645449 wait complete. 2024-04-17T10:39:01.644 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:39:01.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.755+0000 7f2fb2608640 1 Processor -- start 2024-04-17T10:39:01.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.755+0000 7f2fb2608640 1 -- start start 2024-04-17T10:39:01.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.755+0000 7f2fb2608640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2fac06dcb0 0x7f2fac165650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:01.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.755+0000 7f2fb2608640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fac15e440 con 0x7f2fac15db50 2024-04-17T10:39:01.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.755+0000 7f2fb2608640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fac15e720 con 0x7f2fac0aedd0 2024-04-17T10:39:01.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.755+0000 7f2fb2608640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fac15ea00 con 0x7f2fac06dcb0 2024-04-17T10:39:01.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.755+0000 7f2fabfff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2fac15db50 0x7f2fac070470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54752/0 (socket says 172.21.15.83:54752) 2024-04-17T10:39:01.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.755+0000 7f2fabfff640 1 -- 172.21.15.83:0/1959015901 learned_addr learned my addr 172.21.15.83:0/1959015901 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:39:01.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2faaffd640 1 -- 172.21.15.83:0/1959015901 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 680923391 0 0) 0x7f2fac15e440 con 0x7f2fac15db50 2024-04-17T10:39:01.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2faaffd640 1 -- 172.21.15.83:0/1959015901 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f94003530 con 0x7f2fac15db50 2024-04-17T10:39:01.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2fb0ba5640 1 --2- 172.21.15.83:0/1959015901 >> v2:172.21.15.167:3300/0 conn(0x7f2fac06dcb0 0x7f2fac165650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:01.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2faaffd640 1 -- 172.21.15.83:0/1959015901 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3508727330 0 0) 0x7f2f94003530 con 0x7f2fac15db50 2024-04-17T10:39:01.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2faaffd640 1 -- 172.21.15.83:0/1959015901 >> v2:172.21.15.167:3300/0 conn(0x7f2fac06dcb0 msgr2=0x7f2fac165650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:01.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2faaffd640 1 --2- 172.21.15.83:0/1959015901 >> v2:172.21.15.167:3300/0 conn(0x7f2fac06dcb0 0x7f2fac165650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:01.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2faaffd640 1 -- 172.21.15.83:0/1959015901 >> v1:172.21.15.167:6789/0 conn(0x7f2fac0aedd0 legacy=0x7f2fac06d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:01.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2faaffd640 1 -- 172.21.15.83:0/1959015901 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fac15ece0 con 0x7f2fac15db50 2024-04-17T10:39:01.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2faaffd640 1 -- 172.21.15.83:0/1959015901 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f2f9c001680 con 0x7f2fac15db50 2024-04-17T10:39:01.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2faaffd640 1 -- 172.21.15.83:0/1959015901 >> v1:172.21.15.83:6789/0 conn(0x7f2fac15db50 legacy=0x7f2fac070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:01.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2faaffd640 1 --2- 172.21.15.83:0/1959015901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f940039f0 0x7f2f94003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:01.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2faaffd640 1 --2- 172.21.15.83:0/1959015901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f94004810 0x7f2f94004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:01.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2faaffd640 1 --2- 172.21.15.83:0/1959015901 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f94005220 0x7f2f9400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:01.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2faaffd640 1 -- 172.21.15.83:0/1959015901 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac15ea00 con 0x7f2f94004810 2024-04-17T10:39:01.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2faaffd640 1 -- 172.21.15.83:0/1959015901 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9400c150 con 0x7f2f940039f0 2024-04-17T10:39:01.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2faaffd640 1 -- 172.21.15.83:0/1959015901 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9400c350 con 0x7f2f94005220 2024-04-17T10:39:01.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2fab7fe640 1 --2- 172.21.15.83:0/1959015901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f94004810 0x7f2f94004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:01.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2fabfff640 1 --2- 172.21.15.83:0/1959015901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f940039f0 0x7f2f94003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:01.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2fab7fe640 1 -- 172.21.15.83:0/1959015901 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f94005220 msgr2=0x7f2f9400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:01.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2fab7fe640 1 --2- 172.21.15.83:0/1959015901 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f94005220 0x7f2f9400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:01.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2fab7fe640 1 -- 172.21.15.83:0/1959015901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f940039f0 msgr2=0x7f2f94003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:01.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2fab7fe640 1 --2- 172.21.15.83:0/1959015901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f940039f0 0x7f2f94003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:01.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2fb0ba5640 1 -- 172.21.15.83:0/1959015901 reap_dead start 2024-04-17T10:39:01.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2fab7fe640 1 -- 172.21.15.83:0/1959015901 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f94003530 con 0x7f2f94004810 2024-04-17T10:39:01.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2fabfff640 1 --2- 172.21.15.83:0/1959015901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f940039f0 0x7f2f94003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:01.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2fab7fe640 1 --2- 172.21.15.83:0/1959015901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f94004810 0x7f2f94004c90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0003db0 tx=0x7f2fa001c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db8a51793dd6e44 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:01.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2faaffd640 1 -- 172.21.15.83:0/1959015901 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa00b0070 con 0x7f2f94004810 2024-04-17T10:39:01.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2faaffd640 1 -- 172.21.15.83:0/1959015901 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa00ab620 con 0x7f2f94004810 2024-04-17T10:39:01.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2faaffd640 1 -- 172.21.15.83:0/1959015901 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa00039b0 con 0x7f2f94004810 2024-04-17T10:39:01.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2fb2608640 1 -- 172.21.15.83:0/1959015901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f94004810 msgr2=0x7f2f94004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:01.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2fb2608640 1 --2- 172.21.15.83:0/1959015901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f94004810 0x7f2f94004c90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0003db0 tx=0x7f2fa001c300 comp rx=0 tx=0).stop 2024-04-17T10:39:01.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2fb2608640 1 -- 172.21.15.83:0/1959015901 shutdown_connections 2024-04-17T10:39:01.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2fb2608640 1 --2- 172.21.15.83:0/1959015901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f94004810 0x7f2f94004c90 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:01.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2fb2608640 1 -- 172.21.15.83:0/1959015901 >> 172.21.15.83:0/1959015901 conn(0x7f2fac06abe0 msgr2=0x7f2fac06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:01.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2fb2608640 1 -- 172.21.15.83:0/1959015901 shutdown_connections 2024-04-17T10:39:01.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.759+0000 7f2fb2608640 1 -- 172.21.15.83:0/1959015901 wait complete. 2024-04-17T10:39:01.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.763+0000 7f2fb2608640 1 Processor -- start 2024-04-17T10:39:01.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.763+0000 7f2fb2608640 1 -- start start 2024-04-17T10:39:01.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.763+0000 7f2fb2608640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f94004810 0x7f2fac145f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:01.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.763+0000 7f2fb2608640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fac0ac3f0 0x7f2fac0ac870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:01.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.763+0000 7f2fb2608640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fac0cce20 0x7f2fac0cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:01.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.763+0000 7f2fb2608640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac15e0b0 con 0x7f2fac0ac3f0 2024-04-17T10:39:01.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.763+0000 7f2fb2608640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac15df30 con 0x7f2fac0cce20 2024-04-17T10:39:01.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.763+0000 7f2fb2608640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac070470 con 0x7f2f94004810 2024-04-17T10:39:01.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.763+0000 7f2fab7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fac0ac3f0 0x7f2fac0ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:01.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.763+0000 7f2fab7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fac0ac3f0 0x7f2fac0ac870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50288/0 (socket says 172.21.15.83:50288) 2024-04-17T10:39:01.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.763+0000 7f2fab7fe640 1 -- 172.21.15.83:0/1655841023 learned_addr learned my addr 172.21.15.83:0/1655841023 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:01.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.763+0000 7f2fab7fe640 1 -- 172.21.15.83:0/1655841023 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f94004810 msgr2=0x7f2fac145f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:01.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.763+0000 7f2fab7fe640 1 --2- 172.21.15.83:0/1655841023 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f94004810 0x7f2fac145f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:01.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.763+0000 7f2fab7fe640 1 -- 172.21.15.83:0/1655841023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fac0cce20 msgr2=0x7f2fac0cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:01.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.763+0000 7f2fab7fe640 1 --2- 172.21.15.83:0/1655841023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fac0cce20 0x7f2fac0cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:01.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.763+0000 7f2fab7fe640 1 -- 172.21.15.83:0/1655841023 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fac0acdb0 con 0x7f2fac0ac3f0 2024-04-17T10:39:01.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.763+0000 7f2fab7fe640 1 --2- 172.21.15.83:0/1655841023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fac0ac3f0 0x7f2fac0ac870 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2fa001c130 tx=0x7f2f94005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:01.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.763+0000 7f2fa8ff9640 1 -- 172.21.15.83:0/1655841023 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa00b9020 con 0x7f2fac0ac3f0 2024-04-17T10:39:01.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.763+0000 7f2fa8ff9640 1 -- 172.21.15.83:0/1655841023 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa0003990 con 0x7f2fac0ac3f0 2024-04-17T10:39:01.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.763+0000 7f2fb2608640 1 -- 172.21.15.83:0/1655841023 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fac0cdb10 con 0x7f2fac0ac3f0 2024-04-17T10:39:01.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.763+0000 7f2fa8ff9640 1 -- 172.21.15.83:0/1655841023 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa00b0040 con 0x7f2fac0ac3f0 2024-04-17T10:39:01.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.763+0000 7f2fb2608640 1 -- 172.21.15.83:0/1655841023 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fac146c90 con 0x7f2fac0ac3f0 2024-04-17T10:39:01.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.763+0000 7f2fb2608640 1 -- 172.21.15.83:0/1655841023 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fac074f90 con 0x7f2fac0ac3f0 2024-04-17T10:39:01.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.763+0000 7f2fa8ff9640 1 -- 172.21.15.83:0/1655841023 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2fa00bc020 con 0x7f2fac0ac3f0 2024-04-17T10:39:01.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.767+0000 7f2fa8ff9640 1 --2- 172.21.15.83:0/1655841023 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2f98066fb0 0x7f2f98069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:01.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.767+0000 7f2fabfff640 1 --2- 172.21.15.83:0/1655841023 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2f98066fb0 0x7f2f98069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:01.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.767+0000 7f2fa8ff9640 1 -- 172.21.15.83:0/1655841023 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f2fa011ed20 con 0x7f2fac0ac3f0 2024-04-17T10:39:01.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.767+0000 7f2fabfff640 1 --2- 172.21.15.83:0/1655841023 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2f98066fb0 0x7f2f98069470 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c0017f0 tx=0x7f2f9c000ee0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:01.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.771+0000 7f2fa8ff9640 1 -- 172.21.15.83:0/1655841023 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2fa00c3030 con 0x7f2fac0ac3f0 2024-04-17T10:39:01.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.927+0000 7f2fb2608640 1 -- 172.21.15.83:0/1655841023 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2fac0b2af0 con 0x7f2fac0ac3f0 2024-04-17T10:39:01.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.927+0000 7f2fa8ff9640 1 -- 172.21.15.83:0/1655841023 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16782 (secure 0 0 0) 0x7f2fa0016010 con 0x7f2fac0ac3f0 2024-04-17T10:39:01.926 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:39:01.927 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":22,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:38:53.385002+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:38:10.464490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T10:38:11.043134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:39:01.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.931+0000 7f2fb2608640 1 -- 172.21.15.83:0/1655841023 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2f98066fb0 msgr2=0x7f2f98069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:01.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.931+0000 7f2fb2608640 1 --2- 172.21.15.83:0/1655841023 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2f98066fb0 0x7f2f98069470 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c0017f0 tx=0x7f2f9c000ee0 comp rx=0 tx=0).stop 2024-04-17T10:39:01.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.931+0000 7f2fb2608640 1 -- 172.21.15.83:0/1655841023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fac0ac3f0 msgr2=0x7f2fac0ac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:01.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.931+0000 7f2fb2608640 1 --2- 172.21.15.83:0/1655841023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fac0ac3f0 0x7f2fac0ac870 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2fa001c130 tx=0x7f2f94005840 comp rx=0 tx=0).stop 2024-04-17T10:39:01.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.931+0000 7f2fb2608640 1 -- 172.21.15.83:0/1655841023 shutdown_connections 2024-04-17T10:39:01.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.931+0000 7f2fb2608640 1 --2- 172.21.15.83:0/1655841023 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2f98066fb0 0x7f2f98069470 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:01.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.931+0000 7f2fb2608640 1 --2- 172.21.15.83:0/1655841023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fac0cce20 0x7f2fac0cd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:01.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.931+0000 7f2fb2608640 1 --2- 172.21.15.83:0/1655841023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fac0ac3f0 0x7f2fac0ac870 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:01.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.931+0000 7f2fb2608640 1 --2- 172.21.15.83:0/1655841023 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f94004810 0x7f2fac145f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:01.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.931+0000 7f2fb2608640 1 -- 172.21.15.83:0/1655841023 >> 172.21.15.83:0/1655841023 conn(0x7f2fac06abe0 msgr2=0x7f2fac0b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:01.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.931+0000 7f2fb2608640 1 -- 172.21.15.83:0/1655841023 shutdown_connections 2024-04-17T10:39:01.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:01.931+0000 7f2fb2608640 1 -- 172.21.15.83:0/1655841023 wait complete. 2024-04-17T10:39:01.945 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-17T10:39:01.945 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:39:02.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.055+0000 7f8680b68640 1 Processor -- start 2024-04-17T10:39:02.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.055+0000 7f8680b68640 1 -- start start 2024-04-17T10:39:02.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f8680b68640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f867c15db50 0x7f867c070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:02.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f8680b68640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f867c06dcb0 0x7f867c0b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:02.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f8680b68640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f867c0709b0 con 0x7f867c0aedd0 2024-04-17T10:39:02.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f8680b68640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f867c15e440 con 0x7f867c15db50 2024-04-17T10:39:02.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f8680b68640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867c15e720 con 0x7f867c06dcb0 2024-04-17T10:39:02.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f8679d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f867c0aedd0 0x7f867c06d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54762/0 (socket says 172.21.15.83:54762) 2024-04-17T10:39:02.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f8679d9b640 1 -- 172.21.15.83:0/3191087063 learned_addr learned my addr 172.21.15.83:0/3191087063 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:39:02.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f867ad9d640 1 --2- 172.21.15.83:0/3191087063 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f867c06dcb0 0x7f867c0b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:02.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f867959a640 1 -- 172.21.15.83:0/3191087063 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2152159089 0 0) 0x7f8670001120 con 0x7f867c0aedd0 2024-04-17T10:39:02.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f867959a640 1 -- 172.21.15.83:0/3191087063 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8664003530 con 0x7f867c0aedd0 2024-04-17T10:39:02.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f867a59c640 1 --2- 172.21.15.83:0/3191087063 >> v2:172.21.15.167:3300/0 conn(0x7f867c15db50 0x7f867c070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:02.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f867959a640 1 -- 172.21.15.83:0/3191087063 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3144176537 0 0) 0x7f8664003530 con 0x7f867c0aedd0 2024-04-17T10:39:02.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f867959a640 1 -- 172.21.15.83:0/3191087063 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f867c06dcb0 msgr2=0x7f867c0b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:02.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f867959a640 1 --2- 172.21.15.83:0/3191087063 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f867c06dcb0 0x7f867c0b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f867959a640 1 -- 172.21.15.83:0/3191087063 >> v2:172.21.15.167:3300/0 conn(0x7f867c15db50 msgr2=0x7f867c070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:02.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f867959a640 1 --2- 172.21.15.83:0/3191087063 >> v2:172.21.15.167:3300/0 conn(0x7f867c15db50 0x7f867c070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f867959a640 1 -- 172.21.15.83:0/3191087063 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c15ea00 con 0x7f867c0aedd0 2024-04-17T10:39:02.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f867959a640 1 -- 172.21.15.83:0/3191087063 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f8670001850 con 0x7f867c0aedd0 2024-04-17T10:39:02.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f867959a640 1 -- 172.21.15.83:0/3191087063 >> v1:172.21.15.83:6789/0 conn(0x7f867c0aedd0 legacy=0x7f867c06d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:02.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f867959a640 1 --2- 172.21.15.83:0/3191087063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86640039c0 0x7f8664003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:02.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f867959a640 1 --2- 172.21.15.83:0/3191087063 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f867c06dcb0 0x7f8664004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:02.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f867959a640 1 --2- 172.21.15.83:0/3191087063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8664004d50 0x7f86640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:02.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f867959a640 1 -- 172.21.15.83:0/3191087063 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867c15e440 con 0x7f8664004d50 2024-04-17T10:39:02.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f867959a640 1 -- 172.21.15.83:0/3191087063 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867c15e720 con 0x7f86640039c0 2024-04-17T10:39:02.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f867959a640 1 -- 172.21.15.83:0/3191087063 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f866400ba00 con 0x7f867c06dcb0 2024-04-17T10:39:02.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f867ad9d640 1 --2- 172.21.15.83:0/3191087063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8664004d50 0x7f86640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:02.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f867a59c640 1 --2- 172.21.15.83:0/3191087063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86640039c0 0x7f8664003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:02.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f867ad9d640 1 -- 172.21.15.83:0/3191087063 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f867c06dcb0 msgr2=0x7f8664004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:02.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f867ad9d640 1 --2- 172.21.15.83:0/3191087063 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f867c06dcb0 0x7f8664004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f867ad9d640 1 -- 172.21.15.83:0/3191087063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86640039c0 msgr2=0x7f8664003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:02.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f867ad9d640 1 --2- 172.21.15.83:0/3191087063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86640039c0 0x7f8664003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f867ad9d640 1 -- 172.21.15.83:0/3191087063 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8664003530 con 0x7f8664004d50 2024-04-17T10:39:02.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f867a59c640 1 --2- 172.21.15.83:0/3191087063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86640039c0 0x7f8664003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:02.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f867ad9d640 1 --2- 172.21.15.83:0/3191087063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8664004d50 0x7f86640051d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f867c06e5a0 tx=0x7f867401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8b0867838077bb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:02.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f867959a640 1 -- 172.21.15.83:0/3191087063 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86740b2070 con 0x7f8664004d50 2024-04-17T10:39:02.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f867959a640 1 -- 172.21.15.83:0/3191087063 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86740a5c70 con 0x7f8664004d50 2024-04-17T10:39:02.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.059+0000 7f867959a640 1 -- 172.21.15.83:0/3191087063 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8674009c10 con 0x7f8664004d50 2024-04-17T10:39:02.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.063+0000 7f8680b68640 1 -- 172.21.15.83:0/3191087063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8664004d50 msgr2=0x7f86640051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:02.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.063+0000 7f8680b68640 1 --2- 172.21.15.83:0/3191087063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8664004d50 0x7f86640051d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f867c06e5a0 tx=0x7f867401cc60 comp rx=0 tx=0).stop 2024-04-17T10:39:02.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.063+0000 7f867ad9d640 1 -- 172.21.15.83:0/3191087063 reap_dead start 2024-04-17T10:39:02.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.063+0000 7f8680b68640 1 -- 172.21.15.83:0/3191087063 shutdown_connections 2024-04-17T10:39:02.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.063+0000 7f8680b68640 1 -- 172.21.15.83:0/3191087063 >> 172.21.15.83:0/3191087063 conn(0x7f867c06abe0 msgr2=0x7f867c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:02.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.063+0000 7f8680b68640 1 -- 172.21.15.83:0/3191087063 shutdown_connections 2024-04-17T10:39:02.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.063+0000 7f8680b68640 1 -- 172.21.15.83:0/3191087063 wait complete. 2024-04-17T10:39:02.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.063+0000 7f8680b68640 1 Processor -- start 2024-04-17T10:39:02.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.063+0000 7f8680b68640 1 -- start start 2024-04-17T10:39:02.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.063+0000 7f8680b68640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8664004d50 0x7f867c145310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:02.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.063+0000 7f8680b68640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f867c1466a0 0x7f867c147d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:02.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.063+0000 7f8680b68640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f867c146b20 0x7f867c148260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:02.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.063+0000 7f8680b68640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867c15df30 con 0x7f8664004d50 2024-04-17T10:39:02.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.063+0000 7f8680b68640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867c15efc0 con 0x7f867c1466a0 2024-04-17T10:39:02.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.063+0000 7f8680b68640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867c15e0b0 con 0x7f867c146b20 2024-04-17T10:39:02.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.063+0000 7f867a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8664004d50 0x7f867c145310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:02.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.063+0000 7f867a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8664004d50 0x7f867c145310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50298/0 (socket says 172.21.15.83:50298) 2024-04-17T10:39:02.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.063+0000 7f867a59c640 1 -- 172.21.15.83:0/1840147315 learned_addr learned my addr 172.21.15.83:0/1840147315 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:02.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.063+0000 7f8679d9b640 1 --2- 172.21.15.83:0/1840147315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f867c1466a0 0x7f867c147d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:02.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.063+0000 7f867a59c640 1 -- 172.21.15.83:0/1840147315 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f867c146b20 msgr2=0x7f867c148260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:02.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.063+0000 7f867a59c640 1 --2- 172.21.15.83:0/1840147315 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f867c146b20 0x7f867c148260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.063+0000 7f867a59c640 1 -- 172.21.15.83:0/1840147315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f867c1466a0 msgr2=0x7f867c147d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:02.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.063+0000 7f867a59c640 1 --2- 172.21.15.83:0/1840147315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f867c1466a0 0x7f867c147d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.063+0000 7f867a59c640 1 -- 172.21.15.83:0/1840147315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c147080 con 0x7f8664004d50 2024-04-17T10:39:02.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.063+0000 7f8679d9b640 1 --2- 172.21.15.83:0/1840147315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f867c1466a0 0x7f867c147d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:02.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.063+0000 7f867a59c640 1 --2- 172.21.15.83:0/1840147315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8664004d50 0x7f867c145310 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f866c00e7f0 tx=0x7f866c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:02.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.063+0000 7f866affd640 1 -- 172.21.15.83:0/1840147315 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c017070 con 0x7f8664004d50 2024-04-17T10:39:02.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.063+0000 7f866affd640 1 -- 172.21.15.83:0/1840147315 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866c00c8d0 con 0x7f8664004d50 2024-04-17T10:39:02.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.063+0000 7f8680b68640 1 -- 172.21.15.83:0/1840147315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f867c147360 con 0x7f8664004d50 2024-04-17T10:39:02.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.063+0000 7f866affd640 1 -- 172.21.15.83:0/1840147315 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c00f040 con 0x7f8664004d50 2024-04-17T10:39:02.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.063+0000 7f8680b68640 1 -- 172.21.15.83:0/1840147315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f867c147920 con 0x7f8664004d50 2024-04-17T10:39:02.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.067+0000 7f8680b68640 1 -- 172.21.15.83:0/1840147315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f867c074f90 con 0x7f8664004d50 2024-04-17T10:39:02.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.067+0000 7f866affd640 1 -- 172.21.15.83:0/1840147315 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f866c01a070 con 0x7f8664004d50 2024-04-17T10:39:02.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.067+0000 7f866affd640 1 --2- 172.21.15.83:0/1840147315 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f865c066ea0 0x7f865c069360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:02.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.067+0000 7f866affd640 1 -- 172.21.15.83:0/1840147315 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f866c084d40 con 0x7f8664004d50 2024-04-17T10:39:02.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.067+0000 7f8679d9b640 1 --2- 172.21.15.83:0/1840147315 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f865c066ea0 0x7f865c069360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:02.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.067+0000 7f8679d9b640 1 --2- 172.21.15.83:0/1840147315 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f865c066ea0 0x7f865c069360 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f86700019c0 tx=0x7f8670009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:02.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.071+0000 7f866affd640 1 -- 172.21.15.83:0/1840147315 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f866c055c50 con 0x7f8664004d50 2024-04-17T10:39:02.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.259+0000 7f8680b68640 1 -- 172.21.15.83:0/1840147315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f867c0b2af0 con 0x7f8664004d50 2024-04-17T10:39:02.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.259+0000 7f866affd640 1 -- 172.21.15.83:0/1840147315 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4938 (secure 0 0 0) 0x7f866c055550 con 0x7f8664004d50 2024-04-17T10:39:02.259 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:39:02.259 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2884469968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2884469968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2884469968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2919571009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2919571009},{"type":"v1","addr":"172.21.15.83:6836","nonce":2919571009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1635176834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1635176834},{"type":"v1","addr":"172.21.15.83:6837","nonce":1635176834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:38:12.032695+0000","modified":"2024-04-17T10:38:14.064978+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6835/3871588351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3871588351},{"type":"v1","addr":"172.21.15.167:6835","nonce":3871588351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-17T10:39:02.259 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 6 2024-04-17T10:39:02.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.263+0000 7f8680b68640 1 -- 172.21.15.83:0/1840147315 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f865c066ea0 msgr2=0x7f865c069360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:02.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.263+0000 7f8680b68640 1 --2- 172.21.15.83:0/1840147315 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f865c066ea0 0x7f865c069360 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f86700019c0 tx=0x7f8670009040 comp rx=0 tx=0).stop 2024-04-17T10:39:02.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.263+0000 7f8680b68640 1 -- 172.21.15.83:0/1840147315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8664004d50 msgr2=0x7f867c145310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:02.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.263+0000 7f8680b68640 1 --2- 172.21.15.83:0/1840147315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8664004d50 0x7f867c145310 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f866c00e7f0 tx=0x7f866c00ecb0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.263+0000 7f8680b68640 1 -- 172.21.15.83:0/1840147315 shutdown_connections 2024-04-17T10:39:02.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.263+0000 7f8680b68640 1 --2- 172.21.15.83:0/1840147315 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f865c066ea0 0x7f865c069360 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.263+0000 7f8680b68640 1 --2- 172.21.15.83:0/1840147315 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f867c146b20 0x7f867c148260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.263+0000 7f8680b68640 1 --2- 172.21.15.83:0/1840147315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f867c1466a0 0x7f867c147d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.263+0000 7f8680b68640 1 --2- 172.21.15.83:0/1840147315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8664004d50 0x7f867c145310 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.263+0000 7f8680b68640 1 -- 172.21.15.83:0/1840147315 >> 172.21.15.83:0/1840147315 conn(0x7f867c06abe0 msgr2=0x7f867c0ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:02.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.263+0000 7f8680b68640 1 -- 172.21.15.83:0/1840147315 shutdown_connections 2024-04-17T10:39:02.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.263+0000 7f8680b68640 1 -- 172.21.15.83:0/1840147315 wait complete. 2024-04-17T10:39:02.276 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:39:02.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.383+0000 7fb4c964d640 1 Processor -- start 2024-04-17T10:39:02.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.383+0000 7fb4c964d640 1 -- start start 2024-04-17T10:39:02.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c964d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c4070b80 0x7fb4c4164d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:02.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c964d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4c40b2bd0 con 0x7fb4c40cf060 2024-04-17T10:39:02.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c964d640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4c40b2eb0 con 0x7fb4c40b3990 2024-04-17T10:39:02.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c964d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c40b2980 con 0x7fb4c4070b80 2024-04-17T10:39:02.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c2ffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb4c40cf060 0x7fb4c40b4060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54768/0 (socket says 172.21.15.83:54768) 2024-04-17T10:39:02.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c2ffd640 1 -- 172.21.15.83:0/2204259660 learned_addr learned my addr 172.21.15.83:0/2204259660 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:39:02.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c37fe640 1 --2- 172.21.15.83:0/2204259660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c4070b80 0x7fb4c4164d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:02.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c1ffb640 1 -- 172.21.15.83:0/2204259660 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1577467008 0 0) 0x7fb4c40b2bd0 con 0x7fb4c40cf060 2024-04-17T10:39:02.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c1ffb640 1 -- 172.21.15.83:0/2204259660 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4ac003530 con 0x7fb4c40cf060 2024-04-17T10:39:02.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c1ffb640 1 -- 172.21.15.83:0/2204259660 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2551479157 0 0) 0x7fb4ac003530 con 0x7fb4c40cf060 2024-04-17T10:39:02.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c1ffb640 1 -- 172.21.15.83:0/2204259660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c4070b80 msgr2=0x7fb4c4164d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:02.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c1ffb640 1 --2- 172.21.15.83:0/2204259660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c4070b80 0x7fb4c4164d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c1ffb640 1 -- 172.21.15.83:0/2204259660 >> v1:172.21.15.167:6789/0 conn(0x7fb4c40b3990 legacy=0x7fb4c4070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:02.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c1ffb640 1 -- 172.21.15.83:0/2204259660 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c40b3190 con 0x7fb4c40cf060 2024-04-17T10:39:02.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c1ffb640 1 -- 172.21.15.83:0/2204259660 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fb4b4001680 con 0x7fb4c40cf060 2024-04-17T10:39:02.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c1ffb640 1 -- 172.21.15.83:0/2204259660 >> v1:172.21.15.83:6789/0 conn(0x7fb4c40cf060 legacy=0x7fb4c40b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:02.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c1ffb640 1 --2- 172.21.15.83:0/2204259660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4ac0039f0 0x7fb4ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:02.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c1ffb640 1 --2- 172.21.15.83:0/2204259660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4ac0043a0 0x7fb4ac004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:02.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c1ffb640 1 --2- 172.21.15.83:0/2204259660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c4070b80 0x7fb4ac0050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:02.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c1ffb640 1 -- 172.21.15.83:0/2204259660 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c40b2980 con 0x7fb4ac0043a0 2024-04-17T10:39:02.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c1ffb640 1 -- 172.21.15.83:0/2204259660 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ac00ba00 con 0x7fb4ac0039f0 2024-04-17T10:39:02.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c1ffb640 1 -- 172.21.15.83:0/2204259660 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ac00bc00 con 0x7fb4c4070b80 2024-04-17T10:39:02.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c27fc640 1 --2- 172.21.15.83:0/2204259660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4ac0043a0 0x7fb4ac004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:02.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c2ffd640 1 --2- 172.21.15.83:0/2204259660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4ac0039f0 0x7fb4ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:02.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c37fe640 1 --2- 172.21.15.83:0/2204259660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c4070b80 0x7fb4ac0050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:02.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c27fc640 1 -- 172.21.15.83:0/2204259660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c4070b80 msgr2=0x7fb4ac0050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:02.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c27fc640 1 --2- 172.21.15.83:0/2204259660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c4070b80 0x7fb4ac0050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c27fc640 1 -- 172.21.15.83:0/2204259660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4ac0039f0 msgr2=0x7fb4ac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:02.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c27fc640 1 --2- 172.21.15.83:0/2204259660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4ac0039f0 0x7fb4ac003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c27fc640 1 -- 172.21.15.83:0/2204259660 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4ac003530 con 0x7fb4ac0043a0 2024-04-17T10:39:02.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c37fe640 1 --2- 172.21.15.83:0/2204259660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c4070b80 0x7fb4ac0050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:02.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c27fc640 1 --2- 172.21.15.83:0/2204259660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4ac0043a0 0x7fb4ac004870 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8001860 tx=0x7fb4b801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4672438855976c2b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:02.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c1ffb640 1 -- 172.21.15.83:0/2204259660 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b80b5070 con 0x7fb4ac0043a0 2024-04-17T10:39:02.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c1ffb640 1 -- 172.21.15.83:0/2204259660 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b80a8890 con 0x7fb4ac0043a0 2024-04-17T10:39:02.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.387+0000 7fb4c1ffb640 1 -- 172.21.15.83:0/2204259660 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8001460 con 0x7fb4ac0043a0 2024-04-17T10:39:02.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.391+0000 7fb4c964d640 1 -- 172.21.15.83:0/2204259660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4ac0043a0 msgr2=0x7fb4ac004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:02.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.391+0000 7fb4c964d640 1 --2- 172.21.15.83:0/2204259660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4ac0043a0 0x7fb4ac004870 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8001860 tx=0x7fb4b801ec70 comp rx=0 tx=0).stop 2024-04-17T10:39:02.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.391+0000 7fb4c37fe640 1 -- 172.21.15.83:0/2204259660 reap_dead start 2024-04-17T10:39:02.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.391+0000 7fb4c964d640 1 -- 172.21.15.83:0/2204259660 shutdown_connections 2024-04-17T10:39:02.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.391+0000 7fb4c964d640 1 -- 172.21.15.83:0/2204259660 >> 172.21.15.83:0/2204259660 conn(0x7fb4c406abe0 msgr2=0x7fb4c406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:02.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.391+0000 7fb4c964d640 1 -- 172.21.15.83:0/2204259660 shutdown_connections 2024-04-17T10:39:02.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.391+0000 7fb4c964d640 1 -- 172.21.15.83:0/2204259660 wait complete. 2024-04-17T10:39:02.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.391+0000 7fb4c964d640 1 Processor -- start 2024-04-17T10:39:02.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.391+0000 7fb4c964d640 1 -- start start 2024-04-17T10:39:02.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.391+0000 7fb4c964d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4ac0043a0 0x7fb4c41f1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:02.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.391+0000 7fb4c964d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c414dbf0 0x7fb4c414a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:02.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.391+0000 7fb4c964d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4c414ab20 0x7fb4c41502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:02.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.391+0000 7fb4c964d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c40b4770 con 0x7fb4ac0043a0 2024-04-17T10:39:02.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.391+0000 7fb4c964d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c415b310 con 0x7fb4c414ab20 2024-04-17T10:39:02.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.391+0000 7fb4c964d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c40cf440 con 0x7fb4c414dbf0 2024-04-17T10:39:02.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.391+0000 7fb4c2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4ac0043a0 0x7fb4c41f1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:02.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.391+0000 7fb4c2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4ac0043a0 0x7fb4c41f1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50328/0 (socket says 172.21.15.83:50328) 2024-04-17T10:39:02.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.391+0000 7fb4c2ffd640 1 -- 172.21.15.83:0/4292789092 learned_addr learned my addr 172.21.15.83:0/4292789092 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:02.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.391+0000 7fb4c37fe640 1 --2- 172.21.15.83:0/4292789092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4c414ab20 0x7fb4c41502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:02.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.391+0000 7fb4c2ffd640 1 -- 172.21.15.83:0/4292789092 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c414dbf0 msgr2=0x7fb4c414a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:02.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.391+0000 7fb4c2ffd640 1 --2- 172.21.15.83:0/4292789092 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c414dbf0 0x7fb4c414a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.391+0000 7fb4c2ffd640 1 -- 172.21.15.83:0/4292789092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4c414ab20 msgr2=0x7fb4c41502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:02.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.391+0000 7fb4c2ffd640 1 --2- 172.21.15.83:0/4292789092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4c414ab20 0x7fb4c41502b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.391+0000 7fb4c2ffd640 1 -- 172.21.15.83:0/4292789092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c414b400 con 0x7fb4ac0043a0 2024-04-17T10:39:02.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.391+0000 7fb4c2ffd640 1 --2- 172.21.15.83:0/4292789092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4ac0043a0 0x7fb4c41f1660 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400e7e0 tx=0x7fb4b400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:02.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.391+0000 7fb4ab7fe640 1 -- 172.21.15.83:0/4292789092 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b400ee90 con 0x7fb4ac0043a0 2024-04-17T10:39:02.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.391+0000 7fb4ab7fe640 1 -- 172.21.15.83:0/4292789092 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b4018070 con 0x7fb4ac0043a0 2024-04-17T10:39:02.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.391+0000 7fb4ab7fe640 1 -- 172.21.15.83:0/4292789092 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b400cb10 con 0x7fb4ac0043a0 2024-04-17T10:39:02.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.391+0000 7fb4c964d640 1 -- 172.21.15.83:0/4292789092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4c414b6e0 con 0x7fb4ac0043a0 2024-04-17T10:39:02.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.391+0000 7fb4c964d640 1 -- 172.21.15.83:0/4292789092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c414bca0 con 0x7fb4ac0043a0 2024-04-17T10:39:02.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.391+0000 7fb4c964d640 1 -- 172.21.15.83:0/4292789092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4c4074f90 con 0x7fb4ac0043a0 2024-04-17T10:39:02.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.395+0000 7fb4ab7fe640 1 -- 172.21.15.83:0/4292789092 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb4b400ccb0 con 0x7fb4ac0043a0 2024-04-17T10:39:02.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.395+0000 7fb4ab7fe640 1 --2- 172.21.15.83:0/4292789092 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb4b0067130 0x7fb4b00695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:02.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.395+0000 7fb4c27fc640 1 --2- 172.21.15.83:0/4292789092 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb4b0067130 0x7fb4b00695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:02.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.395+0000 7fb4ab7fe640 1 -- 172.21.15.83:0/4292789092 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fb4b4084bb0 con 0x7fb4ac0043a0 2024-04-17T10:39:02.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.395+0000 7fb4c27fc640 1 --2- 172.21.15.83:0/4292789092 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb4b0067130 0x7fb4b00695f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80b2000 tx=0x7fb4b8001240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:02.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.399+0000 7fb4ab7fe640 1 -- 172.21.15.83:0/4292789092 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb4b4055b40 con 0x7fb4ac0043a0 2024-04-17T10:39:02.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.579+0000 7fb4c964d640 1 -- 172.21.15.83:0/4292789092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4c406d390 con 0x7fb4ac0043a0 2024-04-17T10:39:02.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.579+0000 7fb4ab7fe640 1 -- 172.21.15.83:0/4292789092 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4938 (secure 0 0 0) 0x7fb4b4055440 con 0x7fb4ac0043a0 2024-04-17T10:39:02.576 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:39:02.577 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2884469968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2884469968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2884469968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2919571009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2919571009},{"type":"v1","addr":"172.21.15.83:6836","nonce":2919571009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1635176834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1635176834},{"type":"v1","addr":"172.21.15.83:6837","nonce":1635176834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:38:12.032695+0000","modified":"2024-04-17T10:38:14.064978+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6835/3871588351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3871588351},{"type":"v1","addr":"172.21.15.167:6835","nonce":3871588351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-17T10:39:02.577 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 6 2024-04-17T10:39:02.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.579+0000 7fb4c964d640 1 -- 172.21.15.83:0/4292789092 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb4b0067130 msgr2=0x7fb4b00695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:02.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.579+0000 7fb4c964d640 1 --2- 172.21.15.83:0/4292789092 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb4b0067130 0x7fb4b00695f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80b2000 tx=0x7fb4b8001240 comp rx=0 tx=0).stop 2024-04-17T10:39:02.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.579+0000 7fb4c964d640 1 -- 172.21.15.83:0/4292789092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4ac0043a0 msgr2=0x7fb4c41f1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:02.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.579+0000 7fb4c964d640 1 --2- 172.21.15.83:0/4292789092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4ac0043a0 0x7fb4c41f1660 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400e7e0 tx=0x7fb4b400eca0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.579+0000 7fb4c964d640 1 -- 172.21.15.83:0/4292789092 shutdown_connections 2024-04-17T10:39:02.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.579+0000 7fb4c964d640 1 --2- 172.21.15.83:0/4292789092 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb4b0067130 0x7fb4b00695f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.579+0000 7fb4c964d640 1 --2- 172.21.15.83:0/4292789092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4c414ab20 0x7fb4c41502b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.579+0000 7fb4c964d640 1 --2- 172.21.15.83:0/4292789092 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c414dbf0 0x7fb4c414a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.579+0000 7fb4c964d640 1 --2- 172.21.15.83:0/4292789092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4ac0043a0 0x7fb4c41f1660 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.579+0000 7fb4c964d640 1 -- 172.21.15.83:0/4292789092 >> 172.21.15.83:0/4292789092 conn(0x7fb4c406abe0 msgr2=0x7fb4c41540e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:02.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.583+0000 7fb4c964d640 1 -- 172.21.15.83:0/4292789092 shutdown_connections 2024-04-17T10:39:02.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.583+0000 7fb4c964d640 1 -- 172.21.15.83:0/4292789092 wait complete. 2024-04-17T10:39:02.594 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:39:02.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.699+0000 7f3d076dd640 1 Processor -- start 2024-04-17T10:39:02.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.699+0000 7f3d076dd640 1 -- start start 2024-04-17T10:39:02.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.703+0000 7f3d076dd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3d0015dfa0 0x7f3d000b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:02.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.703+0000 7f3d076dd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d000ce980 0x7f3d000b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:02.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.703+0000 7f3d076dd640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d00162a00 con 0x7f3d000cee00 2024-04-17T10:39:02.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.703+0000 7f3d076dd640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d00161e80 con 0x7f3d0015dfa0 2024-04-17T10:39:02.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.703+0000 7f3d076dd640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d00162160 con 0x7f3d000ce980 2024-04-17T10:39:02.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.703+0000 7f3d05479640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3d0015dfa0 0x7f3d000b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:02.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.703+0000 7f3d05c7a640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3d000cee00 0x7f3d000d1d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54772/0 (socket says 172.21.15.83:54772) 2024-04-17T10:39:02.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.703+0000 7f3d05c7a640 1 -- 172.21.15.83:0/2393458336 learned_addr learned my addr 172.21.15.83:0/2393458336 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:39:02.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.703+0000 7f3d05479640 1 -- 172.21.15.83:0/2393458336 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d000ce980 msgr2=0x7f3d000b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:02.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.703+0000 7f3d04c78640 1 --2- 172.21.15.83:0/2393458336 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d000ce980 0x7f3d000b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:02.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.703+0000 7f3cf7fff640 1 -- 172.21.15.83:0/2393458336 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2088320183 0 0) 0x7f3d00162a00 con 0x7f3d000cee00 2024-04-17T10:39:02.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.703+0000 7f3d05479640 1 --2- 172.21.15.83:0/2393458336 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d000ce980 0x7f3d000b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.703+0000 7f3d05479640 1 -- 172.21.15.83:0/2393458336 >> v1:172.21.15.83:6789/0 conn(0x7f3d000cee00 legacy=0x7f3d000d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:02.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.703+0000 7f3d05479640 1 -- 172.21.15.83:0/2393458336 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d00162440 con 0x7f3d0015dfa0 2024-04-17T10:39:02.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.703+0000 7f3d04c78640 1 --2- 172.21.15.83:0/2393458336 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d000ce980 0x7f3d000b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:02.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.703+0000 7f3d05479640 1 --2- 172.21.15.83:0/2393458336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d0015dfa0 0x7f3d000b6c20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc0097c0 tx=0x7f3cfc01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b8dda694dc6e5fa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:02.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.703+0000 7f3cf7fff640 1 -- 172.21.15.83:0/2393458336 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc0b0070 con 0x7f3d0015dfa0 2024-04-17T10:39:02.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.703+0000 7f3cf7fff640 1 -- 172.21.15.83:0/2393458336 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cfc0ab600 con 0x7f3d0015dfa0 2024-04-17T10:39:02.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.703+0000 7f3cf7fff640 1 -- 172.21.15.83:0/2393458336 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc0ab9d0 con 0x7f3d0015dfa0 2024-04-17T10:39:02.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.707+0000 7f3d076dd640 1 -- 172.21.15.83:0/2393458336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d0015dfa0 msgr2=0x7f3d000b6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:02.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.707+0000 7f3d076dd640 1 --2- 172.21.15.83:0/2393458336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d0015dfa0 0x7f3d000b6c20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc0097c0 tx=0x7f3cfc01c3e0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.707+0000 7f3d076dd640 1 -- 172.21.15.83:0/2393458336 shutdown_connections 2024-04-17T10:39:02.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.707+0000 7f3d076dd640 1 --2- 172.21.15.83:0/2393458336 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d000ce980 0x7f3d000b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.707+0000 7f3d076dd640 1 --2- 172.21.15.83:0/2393458336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d0015dfa0 0x7f3d000b6c20 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.707+0000 7f3d076dd640 1 -- 172.21.15.83:0/2393458336 >> 172.21.15.83:0/2393458336 conn(0x7f3d000d0600 msgr2=0x7f3d000d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:02.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.707+0000 7f3d076dd640 1 -- 172.21.15.83:0/2393458336 shutdown_connections 2024-04-17T10:39:02.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.707+0000 7f3d076dd640 1 -- 172.21.15.83:0/2393458336 wait complete. 2024-04-17T10:39:02.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.707+0000 7f3d076dd640 1 Processor -- start 2024-04-17T10:39:02.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.707+0000 7f3d076dd640 1 -- start start 2024-04-17T10:39:02.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.707+0000 7f3d076dd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d000ce980 0x7f3d0015cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:02.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.707+0000 7f3d076dd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d000cee00 0x7f3d0015b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:02.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.707+0000 7f3d076dd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d0015dfa0 0x7f3d0014f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:02.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.707+0000 7f3d076dd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d00161b10 con 0x7f3d000ce980 2024-04-17T10:39:02.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.707+0000 7f3d076dd640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d000d2850 con 0x7f3d0015dfa0 2024-04-17T10:39:02.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.707+0000 7f3d076dd640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0016b660 con 0x7f3d000cee00 2024-04-17T10:39:02.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.707+0000 7f3d05479640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d000ce980 0x7f3d0015cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:02.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.707+0000 7f3d05479640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d000ce980 0x7f3d0015cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50346/0 (socket says 172.21.15.83:50346) 2024-04-17T10:39:02.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.707+0000 7f3d05479640 1 -- 172.21.15.83:0/3484200854 learned_addr learned my addr 172.21.15.83:0/3484200854 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:02.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.707+0000 7f3d05479640 1 -- 172.21.15.83:0/3484200854 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d000cee00 msgr2=0x7f3d0015b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:02.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.707+0000 7f3d04c78640 1 --2- 172.21.15.83:0/3484200854 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d000cee00 0x7f3d0015b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:02.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.707+0000 7f3d05479640 1 --2- 172.21.15.83:0/3484200854 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d000cee00 0x7f3d0015b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.707+0000 7f3d05479640 1 -- 172.21.15.83:0/3484200854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d0015dfa0 msgr2=0x7f3d0014f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:02.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.707+0000 7f3d05479640 1 --2- 172.21.15.83:0/3484200854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d0015dfa0 0x7f3d0014f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.707+0000 7f3d05479640 1 -- 172.21.15.83:0/3484200854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d0014f7f0 con 0x7f3d000ce980 2024-04-17T10:39:02.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.707+0000 7f3d05479640 1 --2- 172.21.15.83:0/3484200854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d000ce980 0x7f3d0015cf00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc0097c0 tx=0x7f3cfc01ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:02.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.707+0000 7f3cf5ffb640 1 -- 172.21.15.83:0/3484200854 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc0b0040 con 0x7f3d000ce980 2024-04-17T10:39:02.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.707+0000 7f3cf5ffb640 1 -- 172.21.15.83:0/3484200854 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cfc004030 con 0x7f3d000ce980 2024-04-17T10:39:02.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.707+0000 7f3d076dd640 1 -- 172.21.15.83:0/3484200854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d000c6bc0 con 0x7f3d000ce980 2024-04-17T10:39:02.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.707+0000 7f3cf5ffb640 1 -- 172.21.15.83:0/3484200854 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc004400 con 0x7f3d000ce980 2024-04-17T10:39:02.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.707+0000 7f3d076dd640 1 -- 172.21.15.83:0/3484200854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d0015ada0 con 0x7f3d000ce980 2024-04-17T10:39:02.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.707+0000 7f3d076dd640 1 -- 172.21.15.83:0/3484200854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d000c70e0 con 0x7f3d000ce980 2024-04-17T10:39:02.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.711+0000 7f3cf5ffb640 1 -- 172.21.15.83:0/3484200854 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3cfc003620 con 0x7f3d000ce980 2024-04-17T10:39:02.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.711+0000 7f3cf5ffb640 1 --2- 172.21.15.83:0/3484200854 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3ce4066f60 0x7f3ce4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:02.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.711+0000 7f3cf5ffb640 1 -- 172.21.15.83:0/3484200854 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f3cfc11fcf0 con 0x7f3d000ce980 2024-04-17T10:39:02.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.711+0000 7f3d04c78640 1 --2- 172.21.15.83:0/3484200854 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3ce4066f60 0x7f3ce4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:02.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.711+0000 7f3d04c78640 1 --2- 172.21.15.83:0/3484200854 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3ce4066f60 0x7f3ce4069420 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0006fd0 tx=0x7f3cf0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:02.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.715+0000 7f3cf5ffb640 1 -- 172.21.15.83:0/3484200854 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3d000c70e0 con 0x7f3d000ce980 2024-04-17T10:39:02.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.859+0000 7f3d076dd640 1 -- 172.21.15.83:0/3484200854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d00074f90 con 0x7f3d000ce980 2024-04-17T10:39:02.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.863+0000 7f3cf5ffb640 1 -- 172.21.15.83:0/3484200854 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16782 (secure 0 0 0) 0x7f3cfc124070 con 0x7f3d000ce980 2024-04-17T10:39:02.859 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:39:02.859 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":22,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:38:53.385002+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:38:10.464490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T10:38:11.043134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:39:02.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.863+0000 7f3d076dd640 1 -- 172.21.15.83:0/3484200854 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3ce4066f60 msgr2=0x7f3ce4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:02.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.863+0000 7f3d076dd640 1 --2- 172.21.15.83:0/3484200854 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3ce4066f60 0x7f3ce4069420 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0006fd0 tx=0x7f3cf0008040 comp rx=0 tx=0).stop 2024-04-17T10:39:02.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.863+0000 7f3d076dd640 1 -- 172.21.15.83:0/3484200854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d000ce980 msgr2=0x7f3d0015cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:02.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.863+0000 7f3d076dd640 1 --2- 172.21.15.83:0/3484200854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d000ce980 0x7f3d0015cf00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc0097c0 tx=0x7f3cfc01ac20 comp rx=0 tx=0).stop 2024-04-17T10:39:02.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.863+0000 7f3d076dd640 1 -- 172.21.15.83:0/3484200854 shutdown_connections 2024-04-17T10:39:02.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.863+0000 7f3d076dd640 1 --2- 172.21.15.83:0/3484200854 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3ce4066f60 0x7f3ce4069420 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.863+0000 7f3d076dd640 1 --2- 172.21.15.83:0/3484200854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d0015dfa0 0x7f3d0014f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.863+0000 7f3d076dd640 1 --2- 172.21.15.83:0/3484200854 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d000cee00 0x7f3d0015b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.863+0000 7f3d076dd640 1 --2- 172.21.15.83:0/3484200854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d000ce980 0x7f3d0015cf00 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.863+0000 7f3d076dd640 1 -- 172.21.15.83:0/3484200854 >> 172.21.15.83:0/3484200854 conn(0x7f3d000d0600 msgr2=0x7f3d00154210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:02.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.863+0000 7f3d076dd640 1 -- 172.21.15.83:0/3484200854 shutdown_connections 2024-04-17T10:39:02.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.863+0000 7f3d076dd640 1 -- 172.21.15.83:0/3484200854 wait complete. 2024-04-17T10:39:02.876 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-17T10:39:02.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.987+0000 7fcfe2827640 1 Processor -- start 2024-04-17T10:39:02.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.987+0000 7fcfe2827640 1 -- start start 2024-04-17T10:39:02.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.987+0000 7fcfe2827640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcfdc06b520 0x7fcfdc071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:02.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.987+0000 7fcfe2827640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfdc06c1f0 con 0x7fcfdc15bac0 2024-04-17T10:39:02.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.987+0000 7fcfe2827640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfdc070a10 con 0x7fcfdc15b6e0 2024-04-17T10:39:02.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.987+0000 7fcfe2827640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfdc070cf0 con 0x7fcfdc06b520 2024-04-17T10:39:02.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.987+0000 7fcfdb7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcfdc15bac0 0x7fcfdc06ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54786/0 (socket says 172.21.15.83:54786) 2024-04-17T10:39:02.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.987+0000 7fcfdb7fe640 1 -- 172.21.15.83:0/1863185464 learned_addr learned my addr 172.21.15.83:0/1863185464 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:39:02.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.987+0000 7fcfdaffd640 1 -- 172.21.15.83:0/1863185464 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 611681714 0 0) 0x7fcfdc06c1f0 con 0x7fcfdc15bac0 2024-04-17T10:39:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.987+0000 7fcfdaffd640 1 -- 172.21.15.83:0/1863185464 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfc8003530 con 0x7fcfdc15bac0 2024-04-17T10:39:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.987+0000 7fcfe0dc4640 1 --2- 172.21.15.83:0/1863185464 >> v2:172.21.15.167:3300/0 conn(0x7fcfdc06b520 0x7fcfdc071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.987+0000 7fcfdaffd640 1 -- 172.21.15.83:0/1863185464 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2735002835 0 0) 0x7fcfc8003530 con 0x7fcfdc15bac0 2024-04-17T10:39:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.987+0000 7fcfdaffd640 1 -- 172.21.15.83:0/1863185464 >> v2:172.21.15.167:3300/0 conn(0x7fcfdc06b520 msgr2=0x7fcfdc071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.987+0000 7fcfdaffd640 1 --2- 172.21.15.83:0/1863185464 >> v2:172.21.15.167:3300/0 conn(0x7fcfdc06b520 0x7fcfdc071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.987+0000 7fcfdaffd640 1 -- 172.21.15.83:0/1863185464 >> v1:172.21.15.167:6789/0 conn(0x7fcfdc15b6e0 legacy=0x7fcfdc0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.987+0000 7fcfdaffd640 1 -- 172.21.15.83:0/1863185464 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfdc070fd0 con 0x7fcfdc15bac0 2024-04-17T10:39:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.987+0000 7fcfdaffd640 1 -- 172.21.15.83:0/1863185464 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fcfc4001680 con 0x7fcfdc15bac0 2024-04-17T10:39:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.987+0000 7fcfe0dc4640 1 --2- 172.21.15.83:0/1863185464 >> v2:172.21.15.167:3300/0 conn(0x7fcfdc06b520 0x7fcfdc071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:39:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.987+0000 7fcfdaffd640 1 -- 172.21.15.83:0/1863185464 >> v1:172.21.15.83:6789/0 conn(0x7fcfdc15bac0 legacy=0x7fcfdc06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.987+0000 7fcfdaffd640 1 --2- 172.21.15.83:0/1863185464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfc8003a30 0x7fcfc8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.987+0000 7fcfdaffd640 1 --2- 172.21.15.83:0/1863185464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc8004810 0x7fcfc8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.987+0000 7fcfdaffd640 1 --2- 172.21.15.83:0/1863185464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfc8005240 0x7fcfc800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:02.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.987+0000 7fcfdaffd640 1 -- 172.21.15.83:0/1863185464 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc070cf0 con 0x7fcfc8004810 2024-04-17T10:39:02.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.987+0000 7fcfdaffd640 1 -- 172.21.15.83:0/1863185464 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc800c1a0 con 0x7fcfc8005240 2024-04-17T10:39:02.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.987+0000 7fcfdaffd640 1 -- 172.21.15.83:0/1863185464 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc800c3a0 con 0x7fcfc8003a30 2024-04-17T10:39:02.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.987+0000 7fcfdb7fe640 1 --2- 172.21.15.83:0/1863185464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc8004810 0x7fcfc8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:02.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.987+0000 7fcfdbfff640 1 --2- 172.21.15.83:0/1863185464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfc8003a30 0x7fcfc8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:02.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.987+0000 7fcfe0dc4640 1 --2- 172.21.15.83:0/1863185464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfc8005240 0x7fcfc800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:02.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.991+0000 7fcfdb7fe640 1 -- 172.21.15.83:0/1863185464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfc8003a30 msgr2=0x7fcfc8003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:02.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.991+0000 7fcfdb7fe640 1 --2- 172.21.15.83:0/1863185464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfc8003a30 0x7fcfc8003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.991+0000 7fcfdb7fe640 1 -- 172.21.15.83:0/1863185464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfc8005240 msgr2=0x7fcfc800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:02.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.991+0000 7fcfdb7fe640 1 --2- 172.21.15.83:0/1863185464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfc8005240 0x7fcfc800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.991+0000 7fcfdb7fe640 1 -- 172.21.15.83:0/1863185464 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc8003530 con 0x7fcfc8004810 2024-04-17T10:39:02.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.991+0000 7fcfe0dc4640 1 --2- 172.21.15.83:0/1863185464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfc8005240 0x7fcfc800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:39:02.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.991+0000 7fcfe0dc4640 1 -- 172.21.15.83:0/1863185464 reap_dead start 2024-04-17T10:39:02.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.991+0000 7fcfdb7fe640 1 --2- 172.21.15.83:0/1863185464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc8004810 0x7fcfc8004c90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcfc400c240 tx=0x7fcfc401f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4a21b790affc72 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:02.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.991+0000 7fcfdaffd640 1 -- 172.21.15.83:0/1863185464 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4007d20 con 0x7fcfc8004810 2024-04-17T10:39:02.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.991+0000 7fcfdaffd640 1 -- 172.21.15.83:0/1863185464 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc4005ae0 con 0x7fcfc8004810 2024-04-17T10:39:02.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.991+0000 7fcfdaffd640 1 -- 172.21.15.83:0/1863185464 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4007070 con 0x7fcfc8004810 2024-04-17T10:39:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfe2827640 1 -- 172.21.15.83:0/1863185464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc8004810 msgr2=0x7fcfc8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfe2827640 1 --2- 172.21.15.83:0/1863185464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc8004810 0x7fcfc8004c90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcfc400c240 tx=0x7fcfc401f3f0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfe2827640 1 -- 172.21.15.83:0/1863185464 shutdown_connections 2024-04-17T10:39:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfe2827640 1 --2- 172.21.15.83:0/1863185464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc8004810 0x7fcfc8004c90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfe2827640 1 -- 172.21.15.83:0/1863185464 >> 172.21.15.83:0/1863185464 conn(0x7fcfdc0b15b0 msgr2=0x7fcfdc0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:02.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfe2827640 1 -- 172.21.15.83:0/1863185464 shutdown_connections 2024-04-17T10:39:02.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfe2827640 1 -- 172.21.15.83:0/1863185464 wait complete. 2024-04-17T10:39:02.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfe2827640 1 Processor -- start 2024-04-17T10:39:02.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfe2827640 1 -- start start 2024-04-17T10:39:02.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfe2827640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfc8004810 0x7fcfdc0aaea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:02.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfe2827640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfdc0adc10 0x7fcfdc159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:02.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfe2827640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfdc0ae070 0x7fcfdc147400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:02.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfe2827640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc0705f0 con 0x7fcfdc0adc10 2024-04-17T10:39:02.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfe2827640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc070470 con 0x7fcfc8004810 2024-04-17T10:39:02.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfe2827640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc070770 con 0x7fcfdc0ae070 2024-04-17T10:39:02.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfdb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfdc0adc10 0x7fcfdc159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:02.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfdb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfdc0adc10 0x7fcfdc159ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50362/0 (socket says 172.21.15.83:50362) 2024-04-17T10:39:02.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfdb7fe640 1 -- 172.21.15.83:0/1271462538 learned_addr learned my addr 172.21.15.83:0/1271462538 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:02.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfe0dc4640 1 --2- 172.21.15.83:0/1271462538 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfdc0ae070 0x7fcfdc147400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:02.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfdbfff640 1 --2- 172.21.15.83:0/1271462538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfc8004810 0x7fcfdc0aaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:02.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfdb7fe640 1 -- 172.21.15.83:0/1271462538 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfdc0ae070 msgr2=0x7fcfdc147400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:02.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfdb7fe640 1 --2- 172.21.15.83:0/1271462538 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfdc0ae070 0x7fcfdc147400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfdb7fe640 1 -- 172.21.15.83:0/1271462538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfc8004810 msgr2=0x7fcfdc0aaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:02.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfdb7fe640 1 --2- 172.21.15.83:0/1271462538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfc8004810 0x7fcfdc0aaea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:02.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfdb7fe640 1 -- 172.21.15.83:0/1271462538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfdc148090 con 0x7fcfdc0adc10 2024-04-17T10:39:02.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfdbfff640 1 --2- 172.21.15.83:0/1271462538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfc8004810 0x7fcfdc0aaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:02.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfdb7fe640 1 --2- 172.21.15.83:0/1271462538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfdc0adc10 0x7fcfdc159ed0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fcfc400c240 tx=0x7fcfc8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:02.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfd8ff9640 1 -- 172.21.15.83:0/1271462538 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4007d20 con 0x7fcfdc0adc10 2024-04-17T10:39:02.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfd8ff9640 1 -- 172.21.15.83:0/1271462538 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc40b6070 con 0x7fcfdc0adc10 2024-04-17T10:39:02.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfe2827640 1 -- 172.21.15.83:0/1271462538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfdc148370 con 0x7fcfdc0adc10 2024-04-17T10:39:02.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfd8ff9640 1 -- 172.21.15.83:0/1271462538 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4001680 con 0x7fcfdc0adc10 2024-04-17T10:39:02.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfe2827640 1 -- 172.21.15.83:0/1271462538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfdc148930 con 0x7fcfdc0adc10 2024-04-17T10:39:02.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.995+0000 7fcfe2827640 1 -- 172.21.15.83:0/1271462538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfdc074f90 con 0x7fcfdc0adc10 2024-04-17T10:39:02.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.999+0000 7fcfd8ff9640 1 -- 172.21.15.83:0/1271462538 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcfc40b8020 con 0x7fcfdc0adc10 2024-04-17T10:39:02.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.999+0000 7fcfd8ff9640 1 --2- 172.21.15.83:0/1271462538 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcfbc066f90 0x7fcfbc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:02.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.999+0000 7fcfd8ff9640 1 -- 172.21.15.83:0/1271462538 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fcfc4122a00 con 0x7fcfdc0adc10 2024-04-17T10:39:02.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.999+0000 7fcfdbfff640 1 --2- 172.21.15.83:0/1271462538 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcfbc066f90 0x7fcfbc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:02.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:02.999+0000 7fcfdbfff640 1 --2- 172.21.15.83:0/1271462538 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcfbc066f90 0x7fcfbc069450 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc000c10 tx=0x7fcfcc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:03.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.003+0000 7fcfd8ff9640 1 -- 172.21.15.83:0/1271462538 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcfc40c9020 con 0x7fcfdc0adc10 2024-04-17T10:39:03.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.199+0000 7fcfe2827640 1 -- 172.21.15.83:0/1271462538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fcfdc0b07a0 con 0x7fcfdc0adc10 2024-04-17T10:39:03.197 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-04-17T10:39:03.199+0000 7f35c3b81640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T10:39:03.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.203+0000 7fcfd8ff9640 1 -- 172.21.15.83:0/1271462538 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) v1 ==== 161+0+0 (secure 0 0 0) 0x7fcfc4019100 con 0x7fcfdc0adc10 2024-04-17T10:39:03.200 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-17T10:39:03.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.207+0000 7fcfe2827640 1 -- 172.21.15.83:0/1271462538 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcfbc066f90 msgr2=0x7fcfbc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:03.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.207+0000 7fcfe2827640 1 --2- 172.21.15.83:0/1271462538 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcfbc066f90 0x7fcfbc069450 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc000c10 tx=0x7fcfcc009040 comp rx=0 tx=0).stop 2024-04-17T10:39:03.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.207+0000 7fcfe2827640 1 -- 172.21.15.83:0/1271462538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfdc0adc10 msgr2=0x7fcfdc159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:03.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.207+0000 7fcfe2827640 1 --2- 172.21.15.83:0/1271462538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfdc0adc10 0x7fcfdc159ed0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fcfc400c240 tx=0x7fcfc8005840 comp rx=0 tx=0).stop 2024-04-17T10:39:03.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.207+0000 7fcfe2827640 1 -- 172.21.15.83:0/1271462538 shutdown_connections 2024-04-17T10:39:03.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.207+0000 7fcfe2827640 1 --2- 172.21.15.83:0/1271462538 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcfbc066f90 0x7fcfbc069450 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:03.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.207+0000 7fcfe2827640 1 --2- 172.21.15.83:0/1271462538 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfdc0ae070 0x7fcfdc147400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:03.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.207+0000 7fcfe2827640 1 --2- 172.21.15.83:0/1271462538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfdc0adc10 0x7fcfdc159ed0 secure :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fcfc400c240 tx=0x7fcfc8005840 comp rx=0 tx=0).stop 2024-04-17T10:39:03.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.207+0000 7fcfe2827640 1 --2- 172.21.15.83:0/1271462538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfc8004810 0x7fcfdc0aaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:03.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.207+0000 7fcfe2827640 1 -- 172.21.15.83:0/1271462538 >> 172.21.15.83:0/1271462538 conn(0x7fcfdc0b15b0 msgr2=0x7fcfdc0b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:03.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.207+0000 7fcfe2827640 1 -- 172.21.15.83:0/1271462538 shutdown_connections 2024-04-17T10:39:03.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.207+0000 7fcfe2827640 1 -- 172.21.15.83:0/1271462538 wait complete. 2024-04-17T10:39:03.216 INFO:tasks.ceph.mds.d.smithi167.stderr: -3179> 2024-04-17T10:38:17.033+0000 7f2b77b6d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T10:39:03.221 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-17T10:39:03.269 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2024-04-17T10:39:03.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.331+0000 7f8cac904640 1 Processor -- start 2024-04-17T10:39:03.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.331+0000 7f8cac904640 1 -- start start 2024-04-17T10:39:03.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.335+0000 7f8cac904640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8ca815a9c0 0x7f8ca80cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:03.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.335+0000 7f8cac904640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ca806dc00 0x7f8ca80ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:03.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.335+0000 7f8cac904640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ca806e170 con 0x7f8ca80d0710 2024-04-17T10:39:03.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.335+0000 7f8cac904640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ca806ed80 con 0x7f8ca815a9c0 2024-04-17T10:39:03.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.335+0000 7f8cac904640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca806e450 con 0x7f8ca806dc00 2024-04-17T10:39:03.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.335+0000 7f8ca5d9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8ca815a9c0 0x7f8ca80cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:03.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.335+0000 7f8ca5d9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8ca815a9c0 0x7f8ca80cdb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50378/0 (socket says 172.21.15.83:50378) 2024-04-17T10:39:03.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.335+0000 7f8ca5d9b640 1 -- 172.21.15.83:0/2302245239 learned_addr learned my addr 172.21.15.83:0/2302245239 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:03.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.335+0000 7f8ca5d9b640 1 -- 172.21.15.83:0/2302245239 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ca806dc00 msgr2=0x7f8ca80ce0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:03.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.335+0000 7f8ca6d9d640 1 --2- 172.21.15.83:0/2302245239 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ca806dc00 0x7f8ca80ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:03.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.335+0000 7f8ca5d9b640 1 --2- 172.21.15.83:0/2302245239 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ca806dc00 0x7f8ca80ce0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:03.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.335+0000 7f8ca5d9b640 1 -- 172.21.15.83:0/2302245239 >> v1:172.21.15.167:6789/0 conn(0x7f8ca80d0710 legacy=0x7f8ca80d0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:03.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.335+0000 7f8ca5d9b640 1 -- 172.21.15.83:0/2302245239 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca806e730 con 0x7f8ca815a9c0 2024-04-17T10:39:03.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.335+0000 7f8ca6d9d640 1 --2- 172.21.15.83:0/2302245239 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ca806dc00 0x7f8ca80ce0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:03.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.335+0000 7f8ca5d9b640 1 --2- 172.21.15.83:0/2302245239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ca815a9c0 0x7f8ca80cdb70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8ca806ef00 tx=0x7f8c9c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd709a2aa1aa629 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:03.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.335+0000 7f8ca559a640 1 -- 172.21.15.83:0/2302245239 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c0b1070 con 0x7f8ca815a9c0 2024-04-17T10:39:03.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.335+0000 7f8ca559a640 1 -- 172.21.15.83:0/2302245239 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9c0ac600 con 0x7f8ca815a9c0 2024-04-17T10:39:03.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.335+0000 7f8ca559a640 1 -- 172.21.15.83:0/2302245239 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c0ac9d0 con 0x7f8ca815a9c0 2024-04-17T10:39:03.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8cac904640 1 -- 172.21.15.83:0/2302245239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ca815a9c0 msgr2=0x7f8ca80cdb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:03.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8cac904640 1 --2- 172.21.15.83:0/2302245239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ca815a9c0 0x7f8ca80cdb70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8ca806ef00 tx=0x7f8c9c01c4d0 comp rx=0 tx=0).stop 2024-04-17T10:39:03.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8cac904640 1 -- 172.21.15.83:0/2302245239 shutdown_connections 2024-04-17T10:39:03.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8cac904640 1 --2- 172.21.15.83:0/2302245239 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ca806dc00 0x7f8ca80ce0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:03.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8cac904640 1 --2- 172.21.15.83:0/2302245239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ca815a9c0 0x7f8ca80cdb70 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:03.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8cac904640 1 -- 172.21.15.83:0/2302245239 >> 172.21.15.83:0/2302245239 conn(0x7f8ca806bcf0 msgr2=0x7f8ca806c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:03.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8cac904640 1 -- 172.21.15.83:0/2302245239 shutdown_connections 2024-04-17T10:39:03.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8cac904640 1 -- 172.21.15.83:0/2302245239 wait complete. 2024-04-17T10:39:03.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8cac904640 1 Processor -- start 2024-04-17T10:39:03.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8cac904640 1 -- start start 2024-04-17T10:39:03.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8cac904640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ca806dc00 0x7f8ca80b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:03.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8cac904640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ca80d0710 0x7f8ca815b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:03.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8cac904640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ca815a9c0 0x7f8ca815cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:03.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8cac904640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca8165790 con 0x7f8ca815a9c0 2024-04-17T10:39:03.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8cac904640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca8165610 con 0x7f8ca80d0710 2024-04-17T10:39:03.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8cac904640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca8165910 con 0x7f8ca806dc00 2024-04-17T10:39:03.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8ca6d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ca815a9c0 0x7f8ca815cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:03.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8ca6d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ca815a9c0 0x7f8ca815cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50388/0 (socket says 172.21.15.83:50388) 2024-04-17T10:39:03.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8ca6d9d640 1 -- 172.21.15.83:0/2834163699 learned_addr learned my addr 172.21.15.83:0/2834163699 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:03.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8ca659c640 1 --2- 172.21.15.83:0/2834163699 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ca806dc00 0x7f8ca80b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:03.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8ca5d9b640 1 --2- 172.21.15.83:0/2834163699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ca80d0710 0x7f8ca815b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:03.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8ca6d9d640 1 -- 172.21.15.83:0/2834163699 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ca806dc00 msgr2=0x7f8ca80b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:03.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8ca6d9d640 1 --2- 172.21.15.83:0/2834163699 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ca806dc00 0x7f8ca80b12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:03.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8ca6d9d640 1 -- 172.21.15.83:0/2834163699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ca80d0710 msgr2=0x7f8ca815b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:03.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8ca6d9d640 1 --2- 172.21.15.83:0/2834163699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ca80d0710 0x7f8ca815b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:03.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8ca6d9d640 1 -- 172.21.15.83:0/2834163699 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca815bf00 con 0x7f8ca815a9c0 2024-04-17T10:39:03.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8ca659c640 1 --2- 172.21.15.83:0/2834163699 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ca806dc00 0x7f8ca80b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:39:03.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8ca5d9b640 1 --2- 172.21.15.83:0/2834163699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ca80d0710 0x7f8ca815b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:03.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8ca6d9d640 1 --2- 172.21.15.83:0/2834163699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ca815a9c0 0x7f8ca815cfa0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000cef0 tx=0x7f8ca000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:03.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8c96ffd640 1 -- 172.21.15.83:0/2834163699 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca001a070 con 0x7f8ca815a9c0 2024-04-17T10:39:03.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8c96ffd640 1 -- 172.21.15.83:0/2834163699 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca000d860 con 0x7f8ca815a9c0 2024-04-17T10:39:03.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8cac904640 1 -- 172.21.15.83:0/2834163699 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca815c1e0 con 0x7f8ca815a9c0 2024-04-17T10:39:03.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8c96ffd640 1 -- 172.21.15.83:0/2834163699 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca000dc50 con 0x7f8ca815a9c0 2024-04-17T10:39:03.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8cac904640 1 -- 172.21.15.83:0/2834163699 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca815c7a0 con 0x7f8ca815a9c0 2024-04-17T10:39:03.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.339+0000 7f8cac904640 1 -- 172.21.15.83:0/2834163699 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ca8074f90 con 0x7f8ca815a9c0 2024-04-17T10:39:03.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.343+0000 7f8c96ffd640 1 -- 172.21.15.83:0/2834163699 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8ca001d070 con 0x7f8ca815a9c0 2024-04-17T10:39:03.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.343+0000 7f8c96ffd640 1 --2- 172.21.15.83:0/2834163699 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8c88066f60 0x7f8c88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:03.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.343+0000 7f8c96ffd640 1 -- 172.21.15.83:0/2834163699 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f8ca0085940 con 0x7f8ca815a9c0 2024-04-17T10:39:03.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.343+0000 7f8ca659c640 1 --2- 172.21.15.83:0/2834163699 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8c88066f60 0x7f8c88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:03.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.343+0000 7f8ca659c640 1 --2- 172.21.15.83:0/2834163699 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8c88066f60 0x7f8c88069420 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8ca815b720 tx=0x7f8c98009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:03.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.347+0000 7f8c96ffd640 1 -- 172.21.15.83:0/2834163699 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8ca00569d0 con 0x7f8ca815a9c0 2024-04-17T10:39:03.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:03.523+0000 7f8cac904640 1 -- 172.21.15.83:0/2834163699 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8ca8166030 con 0x7f8ca815a9c0 2024-04-17T10:39:04.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.207+0000 7f8c96ffd640 1 -- 172.21.15.83:0/2834163699 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8ca00562d0 con 0x7f8ca815a9c0 2024-04-17T10:39:04.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.207+0000 7f8cac904640 1 -- 172.21.15.83:0/2834163699 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8c88066f60 msgr2=0x7f8c88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:04.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.207+0000 7f8cac904640 1 --2- 172.21.15.83:0/2834163699 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8c88066f60 0x7f8c88069420 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8ca815b720 tx=0x7f8c98009040 comp rx=0 tx=0).stop 2024-04-17T10:39:04.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.207+0000 7f8cac904640 1 -- 172.21.15.83:0/2834163699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ca815a9c0 msgr2=0x7f8ca815cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:04.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.207+0000 7f8cac904640 1 --2- 172.21.15.83:0/2834163699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ca815a9c0 0x7f8ca815cfa0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000cef0 tx=0x7f8ca000a710 comp rx=0 tx=0).stop 2024-04-17T10:39:04.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.211+0000 7f8cac904640 1 -- 172.21.15.83:0/2834163699 shutdown_connections 2024-04-17T10:39:04.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.211+0000 7f8cac904640 1 --2- 172.21.15.83:0/2834163699 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8c88066f60 0x7f8c88069420 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:04.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.211+0000 7f8cac904640 1 --2- 172.21.15.83:0/2834163699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ca815a9c0 0x7f8ca815cfa0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:04.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.211+0000 7f8cac904640 1 --2- 172.21.15.83:0/2834163699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ca80d0710 0x7f8ca815b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:04.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.211+0000 7f8cac904640 1 --2- 172.21.15.83:0/2834163699 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ca806dc00 0x7f8ca80b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:04.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.211+0000 7f8cac904640 1 -- 172.21.15.83:0/2834163699 >> 172.21.15.83:0/2834163699 conn(0x7f8ca806bcf0 msgr2=0x7f8ca8071d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:04.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.211+0000 7f8cac904640 1 -- 172.21.15.83:0/2834163699 shutdown_connections 2024-04-17T10:39:04.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.211+0000 7f8cac904640 1 -- 172.21.15.83:0/2834163699 wait complete. 2024-04-17T10:39:04.229 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-17T10:39:04.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.367+0000 7f268cba1640 1 Processor -- start 2024-04-17T10:39:04.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.367+0000 7f268cba1640 1 -- start start 2024-04-17T10:39:04.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.367+0000 7f268cba1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f26880cf790 0x7f26880ce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:04.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.367+0000 7f268cba1640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26880b4220 con 0x7f26880cca70 2024-04-17T10:39:04.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.367+0000 7f268cba1640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26880d0930 con 0x7f268815dd80 2024-04-17T10:39:04.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.367+0000 7f268cba1640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f268806bcf0 con 0x7f26880cf790 2024-04-17T10:39:04.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.367+0000 7f2686d9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f26880cca70 0x7f26880ccf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54802/0 (socket says 172.21.15.83:54802) 2024-04-17T10:39:04.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.367+0000 7f2686d9d640 1 -- 172.21.15.83:0/1480955177 learned_addr learned my addr 172.21.15.83:0/1480955177 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:39:04.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.367+0000 7f2685d9b640 1 --2- 172.21.15.83:0/1480955177 >> v2:172.21.15.83:3300/0 conn(0x7f26880cf790 0x7f26880ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:04.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.367+0000 7f268559a640 1 -- 172.21.15.83:0/1480955177 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3083226412 0 0) 0x7f26880b4220 con 0x7f26880cca70 2024-04-17T10:39:04.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.367+0000 7f2685d9b640 1 -- 172.21.15.83:0/1480955177 >> v1:172.21.15.167:6789/0 conn(0x7f268815dd80 legacy=0x7f26880ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:04.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.367+0000 7f2685d9b640 1 -- 172.21.15.83:0/1480955177 >> v1:172.21.15.83:6789/0 conn(0x7f26880cca70 legacy=0x7f26880ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:04.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.367+0000 7f2685d9b640 1 -- 172.21.15.83:0/1480955177 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26880cbf90 con 0x7f26880cf790 2024-04-17T10:39:04.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.371+0000 7f2685d9b640 1 --2- 172.21.15.83:0/1480955177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26880cf790 0x7f26880ce730 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f267c0097c0 tx=0x7f267c01ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0337d0ba7e2bfef server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:04.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.371+0000 7f268559a640 1 -- 172.21.15.83:0/1480955177 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c0b5070 con 0x7f26880cf790 2024-04-17T10:39:04.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.371+0000 7f268559a640 1 -- 172.21.15.83:0/1480955177 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267c0a8990 con 0x7f26880cf790 2024-04-17T10:39:04.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.371+0000 7f268559a640 1 -- 172.21.15.83:0/1480955177 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c0a8d60 con 0x7f26880cf790 2024-04-17T10:39:04.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.371+0000 7f268cba1640 1 -- 172.21.15.83:0/1480955177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26880cf790 msgr2=0x7f26880ce730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:04.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.371+0000 7f268cba1640 1 --2- 172.21.15.83:0/1480955177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26880cf790 0x7f26880ce730 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f267c0097c0 tx=0x7f267c01ec90 comp rx=0 tx=0).stop 2024-04-17T10:39:04.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.371+0000 7f268cba1640 1 -- 172.21.15.83:0/1480955177 shutdown_connections 2024-04-17T10:39:04.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.371+0000 7f268cba1640 1 --2- 172.21.15.83:0/1480955177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26880cf790 0x7f26880ce730 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:04.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.371+0000 7f268cba1640 1 -- 172.21.15.83:0/1480955177 >> 172.21.15.83:0/1480955177 conn(0x7f26880b3330 msgr2=0x7f26880b3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:04.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.371+0000 7f268cba1640 1 -- 172.21.15.83:0/1480955177 shutdown_connections 2024-04-17T10:39:04.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.371+0000 7f268cba1640 1 -- 172.21.15.83:0/1480955177 wait complete. 2024-04-17T10:39:04.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.371+0000 7f268cba1640 1 Processor -- start 2024-04-17T10:39:04.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.371+0000 7f268cba1640 1 -- start start 2024-04-17T10:39:04.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.375+0000 7f268cba1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26880cca70 0x7f26880b5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:04.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.375+0000 7f268cba1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26880cf790 0x7f26881509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:04.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.375+0000 7f268cba1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f268815dd80 0x7f26880caea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:04.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.375+0000 7f268cba1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268815e160 con 0x7f26880cca70 2024-04-17T10:39:04.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.375+0000 7f268cba1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26880cc550 con 0x7f268815dd80 2024-04-17T10:39:04.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.375+0000 7f268cba1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268815e2e0 con 0x7f26880cf790 2024-04-17T10:39:04.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.375+0000 7f268659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26880cca70 0x7f26880b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:04.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.375+0000 7f268659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26880cca70 0x7f26880b5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50418/0 (socket says 172.21.15.83:50418) 2024-04-17T10:39:04.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.375+0000 7f268659c640 1 -- 172.21.15.83:0/1874838189 learned_addr learned my addr 172.21.15.83:0/1874838189 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:04.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.375+0000 7f268659c640 1 -- 172.21.15.83:0/1874838189 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26880cf790 msgr2=0x7f26881509f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:04.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.375+0000 7f268659c640 1 --2- 172.21.15.83:0/1874838189 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26880cf790 0x7f26881509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:04.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.375+0000 7f268659c640 1 -- 172.21.15.83:0/1874838189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f268815dd80 msgr2=0x7f26880caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:04.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.375+0000 7f268659c640 1 --2- 172.21.15.83:0/1874838189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f268815dd80 0x7f26880caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:04.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.375+0000 7f268659c640 1 -- 172.21.15.83:0/1874838189 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26880c9b20 con 0x7f26880cca70 2024-04-17T10:39:04.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.375+0000 7f268659c640 1 --2- 172.21.15.83:0/1874838189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26880cca70 0x7f26880b5860 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f267800c3f0 tx=0x7f267800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:04.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.375+0000 7f2676ffd640 1 -- 172.21.15.83:0/1874838189 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2678017070 con 0x7f26880cca70 2024-04-17T10:39:04.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.375+0000 7f2676ffd640 1 -- 172.21.15.83:0/1874838189 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26780052d0 con 0x7f26880cca70 2024-04-17T10:39:04.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.375+0000 7f2676ffd640 1 -- 172.21.15.83:0/1874838189 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2678003840 con 0x7f26880cca70 2024-04-17T10:39:04.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.375+0000 7f268cba1640 1 -- 172.21.15.83:0/1874838189 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26880c9e00 con 0x7f26880cca70 2024-04-17T10:39:04.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.375+0000 7f268cba1640 1 -- 172.21.15.83:0/1874838189 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26880ca3c0 con 0x7f26880cca70 2024-04-17T10:39:04.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.375+0000 7f268cba1640 1 -- 172.21.15.83:0/1874838189 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2688075000 con 0x7f26880cca70 2024-04-17T10:39:04.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.375+0000 7f2676ffd640 1 -- 172.21.15.83:0/1874838189 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f267801a070 con 0x7f26880cca70 2024-04-17T10:39:04.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.375+0000 7f2676ffd640 1 --2- 172.21.15.83:0/1874838189 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2668066f60 0x7f2668069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:04.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.375+0000 7f2685d9b640 1 --2- 172.21.15.83:0/1874838189 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2668066f60 0x7f2668069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:04.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.375+0000 7f2676ffd640 1 -- 172.21.15.83:0/1874838189 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f26780844a0 con 0x7f26880cca70 2024-04-17T10:39:04.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.375+0000 7f2685d9b640 1 --2- 172.21.15.83:0/1874838189 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2668066f60 0x7f2668069420 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f267c0b2000 tx=0x7f267c004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:04.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.379+0000 7f2676ffd640 1 -- 172.21.15.83:0/1874838189 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2678005480 con 0x7f26880cca70 2024-04-17T10:39:04.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:04.531+0000 7f268cba1640 1 -- 172.21.15.83:0/1874838189 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2688074480 con 0x7f26880cca70 2024-04-17T10:39:05.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.463+0000 7f2676ffd640 1 -- 172.21.15.83:0/1874838189 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7f26780553e0 con 0x7f26880cca70 2024-04-17T10:39:05.462 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2024-04-17T10:39:05.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.467+0000 7f268cba1640 1 -- 172.21.15.83:0/1874838189 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2668066f60 msgr2=0x7f2668069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:05.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.467+0000 7f268cba1640 1 --2- 172.21.15.83:0/1874838189 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2668066f60 0x7f2668069420 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f267c0b2000 tx=0x7f267c004740 comp rx=0 tx=0).stop 2024-04-17T10:39:05.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.467+0000 7f268cba1640 1 -- 172.21.15.83:0/1874838189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26880cca70 msgr2=0x7f26880b5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:05.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.467+0000 7f268cba1640 1 --2- 172.21.15.83:0/1874838189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26880cca70 0x7f26880b5860 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f267800c3f0 tx=0x7f267800c8b0 comp rx=0 tx=0).stop 2024-04-17T10:39:05.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.471+0000 7f268cba1640 1 -- 172.21.15.83:0/1874838189 shutdown_connections 2024-04-17T10:39:05.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.471+0000 7f268cba1640 1 --2- 172.21.15.83:0/1874838189 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2668066f60 0x7f2668069420 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:05.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.471+0000 7f268cba1640 1 --2- 172.21.15.83:0/1874838189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f268815dd80 0x7f26880caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:05.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.471+0000 7f268cba1640 1 --2- 172.21.15.83:0/1874838189 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26880cf790 0x7f26881509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:05.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.471+0000 7f268cba1640 1 --2- 172.21.15.83:0/1874838189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26880cca70 0x7f26880b5860 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:05.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.471+0000 7f268cba1640 1 -- 172.21.15.83:0/1874838189 >> 172.21.15.83:0/1874838189 conn(0x7f26880b3330 msgr2=0x7f26880b3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:05.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.471+0000 7f268cba1640 1 -- 172.21.15.83:0/1874838189 shutdown_connections 2024-04-17T10:39:05.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.471+0000 7f268cba1640 1 -- 172.21.15.83:0/1874838189 wait complete. 2024-04-17T10:39:05.488 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-17T10:39:05.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.611+0000 7fbd87cd6640 1 Processor -- start 2024-04-17T10:39:05.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.611+0000 7fbd87cd6640 1 -- start start 2024-04-17T10:39:05.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.615+0000 7fbd87cd6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd8015f120 0x7fbd8006e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:05.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.615+0000 7fbd87cd6640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbd800702c0 0x7fbd8006ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:05.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.615+0000 7fbd87cd6640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd800d04f0 con 0x7fbd800cef20 2024-04-17T10:39:05.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.615+0000 7fbd87cd6640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd800cfd80 con 0x7fbd800702c0 2024-04-17T10:39:05.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.615+0000 7fbd87cd6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd800d0060 con 0x7fbd8015f120 2024-04-17T10:39:05.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.615+0000 7fbd85a72640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd8015f120 0x7fbd8006e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:05.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.615+0000 7fbd86273640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbd800702c0 0x7fbd8006ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:05.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.615+0000 7fbd85271640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fbd800cef20 0x7fbd800cdea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:55790/0 (socket says 172.21.15.83:55790) 2024-04-17T10:39:05.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.615+0000 7fbd85271640 1 -- 172.21.15.83:0/3203084906 learned_addr learned my addr 172.21.15.83:0/3203084906 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:39:05.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.615+0000 7fbd86273640 1 -- 172.21.15.83:0/3203084906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd8015f120 msgr2=0x7fbd8006e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:05.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.615+0000 7fbd86273640 1 --2- 172.21.15.83:0/3203084906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd8015f120 0x7fbd8006e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:05.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.615+0000 7fbd86273640 1 -- 172.21.15.83:0/3203084906 >> v1:172.21.15.167:6789/0 conn(0x7fbd800cef20 legacy=0x7fbd800cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:05.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.615+0000 7fbd86273640 1 -- 172.21.15.83:0/3203084906 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd800707f0 con 0x7fbd800702c0 2024-04-17T10:39:05.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.615+0000 7fbd86273640 1 --2- 172.21.15.83:0/3203084906 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd800702c0 0x7fbd8006ec80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbd78009d90 tx=0x7fbd7801c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c60c1a497118296 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:05.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd84a70640 1 -- 172.21.15.83:0/3203084906 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd780b2070 con 0x7fbd800702c0 2024-04-17T10:39:05.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd84a70640 1 -- 172.21.15.83:0/3203084906 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd780a5820 con 0x7fbd800702c0 2024-04-17T10:39:05.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd84a70640 1 -- 172.21.15.83:0/3203084906 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd780a5bf0 con 0x7fbd800702c0 2024-04-17T10:39:05.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd87cd6640 1 -- 172.21.15.83:0/3203084906 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd800702c0 msgr2=0x7fbd8006ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:05.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd87cd6640 1 --2- 172.21.15.83:0/3203084906 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd800702c0 0x7fbd8006ec80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbd78009d90 tx=0x7fbd7801c800 comp rx=0 tx=0).stop 2024-04-17T10:39:05.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd87cd6640 1 -- 172.21.15.83:0/3203084906 shutdown_connections 2024-04-17T10:39:05.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd87cd6640 1 --2- 172.21.15.83:0/3203084906 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd800702c0 0x7fbd8006ec80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:05.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd87cd6640 1 --2- 172.21.15.83:0/3203084906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd8015f120 0x7fbd8006e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:05.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd87cd6640 1 -- 172.21.15.83:0/3203084906 >> 172.21.15.83:0/3203084906 conn(0x7fbd800b3330 msgr2=0x7fbd800b3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:05.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd87cd6640 1 -- 172.21.15.83:0/3203084906 shutdown_connections 2024-04-17T10:39:05.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd87cd6640 1 -- 172.21.15.83:0/3203084906 wait complete. 2024-04-17T10:39:05.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd87cd6640 1 Processor -- start 2024-04-17T10:39:05.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd87cd6640 1 -- start start 2024-04-17T10:39:05.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd87cd6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd800702c0 0x7fbd800c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:05.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd87cd6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd800cef20 0x7fbd800b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:05.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd87cd6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd8015f120 0x7fbd8006c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:05.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd87cd6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8015f500 con 0x7fbd800702c0 2024-04-17T10:39:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd87cd6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8006f660 con 0x7fbd8015f120 2024-04-17T10:39:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd87cd6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8015f680 con 0x7fbd800cef20 2024-04-17T10:39:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd85a72640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd800702c0 0x7fbd800c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd85a72640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd800702c0 0x7fbd800c6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50434/0 (socket says 172.21.15.83:50434) 2024-04-17T10:39:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd85a72640 1 -- 172.21.15.83:0/320751924 learned_addr learned my addr 172.21.15.83:0/320751924 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd85271640 1 --2- 172.21.15.83:0/320751924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd800cef20 0x7fbd800b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd86273640 1 --2- 172.21.15.83:0/320751924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd8015f120 0x7fbd8006c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd85a72640 1 -- 172.21.15.83:0/320751924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd800cef20 msgr2=0x7fbd800b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd85a72640 1 --2- 172.21.15.83:0/320751924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd800cef20 0x7fbd800b20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd85a72640 1 -- 172.21.15.83:0/320751924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd8015f120 msgr2=0x7fbd8006c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd85a72640 1 --2- 172.21.15.83:0/320751924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd8015f120 0x7fbd8006c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd85a72640 1 -- 172.21.15.83:0/320751924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8006c940 con 0x7fbd800702c0 2024-04-17T10:39:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd85271640 1 --2- 172.21.15.83:0/320751924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd800cef20 0x7fbd800b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:05.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd85a72640 1 --2- 172.21.15.83:0/320751924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd800702c0 0x7fbd800c6780 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c00e870 tx=0x7fbd7c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:05.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd767fc640 1 -- 172.21.15.83:0/320751924 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c017070 con 0x7fbd800702c0 2024-04-17T10:39:05.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd767fc640 1 -- 172.21.15.83:0/320751924 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd7c00c940 con 0x7fbd800702c0 2024-04-17T10:39:05.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd87cd6640 1 -- 172.21.15.83:0/320751924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd8006dc80 con 0x7fbd800702c0 2024-04-17T10:39:05.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd767fc640 1 -- 172.21.15.83:0/320751924 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c00ccf0 con 0x7fbd800702c0 2024-04-17T10:39:05.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.619+0000 7fbd87cd6640 1 -- 172.21.15.83:0/320751924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd8006cec0 con 0x7fbd800702c0 2024-04-17T10:39:05.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.623+0000 7fbd87cd6640 1 -- 172.21.15.83:0/320751924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd80075000 con 0x7fbd800702c0 2024-04-17T10:39:05.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.623+0000 7fbd767fc640 1 -- 172.21.15.83:0/320751924 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbd7c01a070 con 0x7fbd800702c0 2024-04-17T10:39:05.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.623+0000 7fbd767fc640 1 --2- 172.21.15.83:0/320751924 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbd64066f60 0x7fbd64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:05.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.623+0000 7fbd767fc640 1 -- 172.21.15.83:0/320751924 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5473+0+0 (secure 0 0 0) 0x7fbd7c084e30 con 0x7fbd800702c0 2024-04-17T10:39:05.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.623+0000 7fbd85271640 1 --2- 172.21.15.83:0/320751924 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbd64066f60 0x7fbd64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:05.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.623+0000 7fbd85271640 1 --2- 172.21.15.83:0/320751924 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbd64066f60 0x7fbd64069420 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbd8015d4d0 tx=0x7fbd7000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:05.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.627+0000 7fbd767fc640 1 -- 172.21.15.83:0/320751924 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbd7c055f20 con 0x7fbd800702c0 2024-04-17T10:39:05.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:05.783+0000 7fbd87cd6640 1 -- 172.21.15.83:0/320751924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbd8015c7e0 con 0x7fbd800702c0 2024-04-17T10:39:06.009 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:39:06.012 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:39:06.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.475+0000 7fbd767fc640 1 -- 172.21.15.83:0/320751924 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbd7c055820 con 0x7fbd800702c0 2024-04-17T10:39:06.471 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2024-04-17T10:39:06.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.479+0000 7fbd87cd6640 1 -- 172.21.15.83:0/320751924 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbd64066f60 msgr2=0x7fbd64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:06.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.479+0000 7fbd87cd6640 1 --2- 172.21.15.83:0/320751924 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbd64066f60 0x7fbd64069420 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbd8015d4d0 tx=0x7fbd7000b040 comp rx=0 tx=0).stop 2024-04-17T10:39:06.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.479+0000 7fbd87cd6640 1 -- 172.21.15.83:0/320751924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd800702c0 msgr2=0x7fbd800c6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:06.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.479+0000 7fbd87cd6640 1 --2- 172.21.15.83:0/320751924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd800702c0 0x7fbd800c6780 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c00e870 tx=0x7fbd7c00ec70 comp rx=0 tx=0).stop 2024-04-17T10:39:06.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.479+0000 7fbd87cd6640 1 -- 172.21.15.83:0/320751924 shutdown_connections 2024-04-17T10:39:06.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.479+0000 7fbd87cd6640 1 --2- 172.21.15.83:0/320751924 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbd64066f60 0x7fbd64069420 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:06.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.479+0000 7fbd87cd6640 1 --2- 172.21.15.83:0/320751924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd8015f120 0x7fbd8006c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:06.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.479+0000 7fbd87cd6640 1 --2- 172.21.15.83:0/320751924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd800cef20 0x7fbd800b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:06.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.479+0000 7fbd87cd6640 1 --2- 172.21.15.83:0/320751924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd800702c0 0x7fbd800c6780 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:06.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.479+0000 7fbd87cd6640 1 -- 172.21.15.83:0/320751924 >> 172.21.15.83:0/320751924 conn(0x7fbd800b3330 msgr2=0x7fbd800cd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:06.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.479+0000 7fbd87cd6640 1 -- 172.21.15.83:0/320751924 shutdown_connections 2024-04-17T10:39:06.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.479+0000 7fbd87cd6640 1 -- 172.21.15.83:0/320751924 wait complete. 2024-04-17T10:39:06.501 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-17T10:39:06.502 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-17T10:39:06.502 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-17T10:39:06.502 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:39:06.502 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-17T10:39:06.502 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-17T10:39:06.502 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-17T10:39:06.502 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:39:06.503 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-17T10:39:06.503 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-17T10:39:06.503 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-17T10:39:06.503 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:39:06.503 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-17T10:39:06.503 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-17T10:39:06.504 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-17T10:39:06.504 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:39:06.512 INFO:tasks.ceph.mds.a:Stopped 2024-04-17T10:39:06.512 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-17T10:39:06.514 INFO:tasks.ceph.mds.c:Stopped 2024-04-17T10:39:06.514 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-17T10:39:06.520 INFO:tasks.ceph.mds.b:Stopped 2024-04-17T10:39:06.520 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-17T10:39:06.520 INFO:tasks.ceph.mds.d:Stopped 2024-04-17T10:39:06.521 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-17T10:39:06.556 INFO:tasks.ceph.mds.a:Started 2024-04-17T10:39:06.556 DEBUG:teuthology.parallel:result is None 2024-04-17T10:39:06.557 INFO:tasks.ceph.mds.c:Started 2024-04-17T10:39:06.557 DEBUG:teuthology.parallel:result is None 2024-04-17T10:39:06.564 INFO:tasks.ceph.mds.b:Started 2024-04-17T10:39:06.564 DEBUG:teuthology.parallel:result is None 2024-04-17T10:39:06.565 INFO:tasks.ceph.mds.d:Started 2024-04-17T10:39:06.565 DEBUG:teuthology.parallel:result is None 2024-04-17T10:39:06.565 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-17T10:39:06.621 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-04-17T10:39:06.622 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-04-17T10:39:06.628 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2024-04-17T10:39:06.629 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2024-04-17T10:39:06.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.687+0000 7ff3a583b640 1 Processor -- start 2024-04-17T10:39:06.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.687+0000 7ff3a583b640 1 -- start start 2024-04-17T10:39:06.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.691+0000 7ff3a583b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff3a015e0f0 0x7ff3a00b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:06.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.691+0000 7ff3a583b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff3a00ce400 0x7ff3a0161c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:06.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.691+0000 7ff3a583b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3a00b5450 0x7ff3a00b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:06.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.691+0000 7ff3a583b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a0162b40 con 0x7ff3a015e0f0 2024-04-17T10:39:06.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.691+0000 7ff3a583b640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a015e4d0 con 0x7ff3a00ce400 2024-04-17T10:39:06.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.691+0000 7ff3a583b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a0162580 con 0x7ff3a00b5450 2024-04-17T10:39:06.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.691+0000 7ff39effd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff3a015e0f0 0x7ff3a00b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:06.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.691+0000 7ff39effd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff3a015e0f0 0x7ff3a00b6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50486/0 (socket says 172.21.15.83:50486) 2024-04-17T10:39:06.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.691+0000 7ff39effd640 1 -- 172.21.15.83:0/1736782703 learned_addr learned my addr 172.21.15.83:0/1736782703 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:06.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.691+0000 7ff39e7fc640 1 --2- 172.21.15.83:0/1736782703 >> v2:172.21.15.167:3300/0 conn(0x7ff3a00ce400 0x7ff3a0161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:06.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.691+0000 7ff39f7fe640 1 --2- 172.21.15.83:0/1736782703 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3a00b5450 0x7ff3a00b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:06.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.691+0000 7ff39effd640 1 -- 172.21.15.83:0/1736782703 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3a00b5450 msgr2=0x7ff3a00b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:06.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.691+0000 7ff39effd640 1 --2- 172.21.15.83:0/1736782703 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3a00b5450 0x7ff3a00b58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:06.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.691+0000 7ff39effd640 1 -- 172.21.15.83:0/1736782703 >> v2:172.21.15.167:3300/0 conn(0x7ff3a00ce400 msgr2=0x7ff3a0161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:06.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.691+0000 7ff39effd640 1 --2- 172.21.15.83:0/1736782703 >> v2:172.21.15.167:3300/0 conn(0x7ff3a00ce400 0x7ff3a0161c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:06.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.691+0000 7ff39effd640 1 -- 172.21.15.83:0/1736782703 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a0162860 con 0x7ff3a015e0f0 2024-04-17T10:39:06.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.691+0000 7ff39f7fe640 1 --2- 172.21.15.83:0/1736782703 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3a00b5450 0x7ff3a00b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:39:06.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.691+0000 7ff39effd640 1 --2- 172.21.15.83:0/1736782703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3a015e0f0 0x7ff3a00b6d00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0068170 tx=0x7ff39401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d251f510be71f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:06.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.691+0000 7ff39dffb640 1 -- 172.21.15.83:0/1736782703 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3940b0070 con 0x7ff3a015e0f0 2024-04-17T10:39:06.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.691+0000 7ff39dffb640 1 -- 172.21.15.83:0/1736782703 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3940ab600 con 0x7ff3a015e0f0 2024-04-17T10:39:06.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.691+0000 7ff39dffb640 1 -- 172.21.15.83:0/1736782703 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3940ab9d0 con 0x7ff3a015e0f0 2024-04-17T10:39:06.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.695+0000 7ff3a583b640 1 -- 172.21.15.83:0/1736782703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3a015e0f0 msgr2=0x7ff3a00b6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:06.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.695+0000 7ff3a583b640 1 --2- 172.21.15.83:0/1736782703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3a015e0f0 0x7ff3a00b6d00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0068170 tx=0x7ff39401c490 comp rx=0 tx=0).stop 2024-04-17T10:39:06.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.695+0000 7ff3a583b640 1 -- 172.21.15.83:0/1736782703 shutdown_connections 2024-04-17T10:39:06.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.695+0000 7ff3a583b640 1 --2- 172.21.15.83:0/1736782703 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3a00b5450 0x7ff3a00b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:06.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.695+0000 7ff3a583b640 1 --2- 172.21.15.83:0/1736782703 >> v2:172.21.15.167:3300/0 conn(0x7ff3a00ce400 0x7ff3a0161c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:06.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.695+0000 7ff3a583b640 1 --2- 172.21.15.83:0/1736782703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3a015e0f0 0x7ff3a00b6d00 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:06.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.695+0000 7ff3a583b640 1 -- 172.21.15.83:0/1736782703 >> 172.21.15.83:0/1736782703 conn(0x7ff3a00d0450 msgr2=0x7ff3a00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:06.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.695+0000 7ff3a583b640 1 -- 172.21.15.83:0/1736782703 shutdown_connections 2024-04-17T10:39:06.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.695+0000 7ff3a583b640 1 -- 172.21.15.83:0/1736782703 wait complete. 2024-04-17T10:39:06.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.695+0000 7ff3a583b640 1 Processor -- start 2024-04-17T10:39:06.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.695+0000 7ff3a583b640 1 -- start start 2024-04-17T10:39:06.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.695+0000 7ff3a583b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3a00b5450 0x7ff3a014cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:06.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.695+0000 7ff3a583b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3a00ce400 0x7ff3a0153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:06.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.695+0000 7ff3a583b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3a015e0f0 0x7ff3a014eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:06.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.695+0000 7ff3a583b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a00d2d50 con 0x7ff3a00b5450 2024-04-17T10:39:06.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.695+0000 7ff3a583b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a0162360 con 0x7ff3a015e0f0 2024-04-17T10:39:06.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.695+0000 7ff3a583b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a00d2ed0 con 0x7ff3a00ce400 2024-04-17T10:39:06.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.695+0000 7ff39effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3a00b5450 0x7ff3a014cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:06.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.695+0000 7ff39effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3a00b5450 0x7ff3a014cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50490/0 (socket says 172.21.15.83:50490) 2024-04-17T10:39:06.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.695+0000 7ff39effd640 1 -- 172.21.15.83:0/3326443304 learned_addr learned my addr 172.21.15.83:0/3326443304 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:06.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.695+0000 7ff39effd640 1 -- 172.21.15.83:0/3326443304 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3a00ce400 msgr2=0x7ff3a0153750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:06.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.695+0000 7ff39effd640 1 --2- 172.21.15.83:0/3326443304 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3a00ce400 0x7ff3a0153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:06.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.695+0000 7ff39effd640 1 -- 172.21.15.83:0/3326443304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3a015e0f0 msgr2=0x7ff3a014eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:06.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.695+0000 7ff39effd640 1 --2- 172.21.15.83:0/3326443304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3a015e0f0 0x7ff3a014eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:06.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.695+0000 7ff39effd640 1 -- 172.21.15.83:0/3326443304 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a014bcf0 con 0x7ff3a00b5450 2024-04-17T10:39:06.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.695+0000 7ff39effd640 1 --2- 172.21.15.83:0/3326443304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3a00b5450 0x7ff3a014cd90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff3940ae040 tx=0x7ff394019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:06.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.695+0000 7ff387fff640 1 -- 172.21.15.83:0/3326443304 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3940b0040 con 0x7ff3a00b5450 2024-04-17T10:39:06.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.695+0000 7ff387fff640 1 -- 172.21.15.83:0/3326443304 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3940b9070 con 0x7ff3a00b5450 2024-04-17T10:39:06.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.695+0000 7ff3a583b640 1 -- 172.21.15.83:0/3326443304 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a014bfd0 con 0x7ff3a00b5450 2024-04-17T10:39:06.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.695+0000 7ff387fff640 1 -- 172.21.15.83:0/3326443304 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3940b46a0 con 0x7ff3a00b5450 2024-04-17T10:39:06.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.695+0000 7ff3a583b640 1 -- 172.21.15.83:0/3326443304 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a014c590 con 0x7ff3a00b5450 2024-04-17T10:39:06.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.699+0000 7ff3a583b640 1 -- 172.21.15.83:0/3326443304 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3a00681a0 con 0x7ff3a00b5450 2024-04-17T10:39:06.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.699+0000 7ff387fff640 1 -- 172.21.15.83:0/3326443304 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff3940b4840 con 0x7ff3a00b5450 2024-04-17T10:39:06.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.699+0000 7ff387fff640 1 --2- 172.21.15.83:0/3326443304 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff38c066f60 0x7ff38c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:06.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.699+0000 7ff387fff640 1 -- 172.21.15.83:0/3326443304 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff39411e950 con 0x7ff3a00b5450 2024-04-17T10:39:06.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.699+0000 7ff39e7fc640 1 --2- 172.21.15.83:0/3326443304 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff38c066f60 0x7ff38c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:06.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.699+0000 7ff39e7fc640 1 --2- 172.21.15.83:0/3326443304 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff38c066f60 0x7ff38c069420 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff3a014d370 tx=0x7ff390008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:06.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.703+0000 7ff387fff640 1 -- 172.21.15.83:0/3326443304 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff394016110 con 0x7ff3a00b5450 2024-04-17T10:39:06.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:06.871+0000 7ff3a583b640 1 -- 172.21.15.83:0/3326443304 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff3a0074f90 con 0x7ff3a00b5450 2024-04-17T10:39:07.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.495+0000 7ff387fff640 1 -- 172.21.15.83:0/3326443304 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff3940efd80 con 0x7ff3a00b5450 2024-04-17T10:39:07.495 INFO:teuthology.orchestra.run.smithi083.stderr: removed all blocklist entries 2024-04-17T10:39:07.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.499+0000 7ff3a583b640 1 -- 172.21.15.83:0/3326443304 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff38c066f60 msgr2=0x7ff38c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:07.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.499+0000 7ff3a583b640 1 --2- 172.21.15.83:0/3326443304 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff38c066f60 0x7ff38c069420 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff3a014d370 tx=0x7ff390008040 comp rx=0 tx=0).stop 2024-04-17T10:39:07.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.499+0000 7ff3a583b640 1 -- 172.21.15.83:0/3326443304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3a00b5450 msgr2=0x7ff3a014cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:07.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.499+0000 7ff3a583b640 1 --2- 172.21.15.83:0/3326443304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3a00b5450 0x7ff3a014cd90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff3940ae040 tx=0x7ff394019230 comp rx=0 tx=0).stop 2024-04-17T10:39:07.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.499+0000 7ff3a583b640 1 -- 172.21.15.83:0/3326443304 shutdown_connections 2024-04-17T10:39:07.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.499+0000 7ff3a583b640 1 --2- 172.21.15.83:0/3326443304 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff38c066f60 0x7ff38c069420 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:07.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.499+0000 7ff3a583b640 1 --2- 172.21.15.83:0/3326443304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3a015e0f0 0x7ff3a014eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:07.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.499+0000 7ff3a583b640 1 --2- 172.21.15.83:0/3326443304 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3a00ce400 0x7ff3a0153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:07.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.499+0000 7ff3a583b640 1 --2- 172.21.15.83:0/3326443304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3a00b5450 0x7ff3a014cd90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:07.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.499+0000 7ff3a583b640 1 -- 172.21.15.83:0/3326443304 >> 172.21.15.83:0/3326443304 conn(0x7ff3a00d0450 msgr2=0x7ff3a015e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:07.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.499+0000 7ff3a583b640 1 -- 172.21.15.83:0/3326443304 shutdown_connections 2024-04-17T10:39:07.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.499+0000 7ff3a583b640 1 -- 172.21.15.83:0/3326443304 wait complete. 2024-04-17T10:39:07.521 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-17T10:39:07.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.647+0000 7f90efa13640 1 Processor -- start 2024-04-17T10:39:07.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.647+0000 7f90efa13640 1 -- start start 2024-04-17T10:39:07.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.651+0000 7f90efa13640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f90e815f1d0 0x7f90e815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:07.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.651+0000 7f90efa13640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90e80d22d0 0x7f90e80ae520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:07.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.651+0000 7f90efa13640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e815f5b0 con 0x7f90e80b2de0 2024-04-17T10:39:07.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.651+0000 7f90efa13640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e80b2620 con 0x7f90e815f1d0 2024-04-17T10:39:07.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.651+0000 7f90efa13640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e80b2340 con 0x7f90e80d22d0 2024-04-17T10:39:07.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.651+0000 7f90ed7af640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f90e815f1d0 0x7f90e815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:07.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.651+0000 7f90ed7af640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f90e815f1d0 0x7f90e815f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36902/0 (socket says 172.21.15.83:36902) 2024-04-17T10:39:07.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.651+0000 7f90ed7af640 1 -- 172.21.15.83:0/2085339455 learned_addr learned my addr 172.21.15.83:0/2085339455 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:07.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.651+0000 7f90edfb0640 1 --2- 172.21.15.83:0/2085339455 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90e80d22d0 0x7f90e80ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:07.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.651+0000 7f90ed7af640 1 -- 172.21.15.83:0/2085339455 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90e80d22d0 msgr2=0x7f90e80ae520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:07.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.651+0000 7f90ed7af640 1 --2- 172.21.15.83:0/2085339455 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90e80d22d0 0x7f90e80ae520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:07.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.651+0000 7f90ed7af640 1 -- 172.21.15.83:0/2085339455 >> v1:172.21.15.167:6789/0 conn(0x7f90e80b2de0 legacy=0x7f90e815fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:07.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.651+0000 7f90ed7af640 1 -- 172.21.15.83:0/2085339455 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e80b1760 con 0x7f90e815f1d0 2024-04-17T10:39:07.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.651+0000 7f90edfb0640 1 --2- 172.21.15.83:0/2085339455 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90e80d22d0 0x7f90e80ae520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.651+0000 7f90ed7af640 1 --2- 172.21.15.83:0/2085339455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90e815f1d0 0x7f90e815f830 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f90d8009870 tx=0x7f90d801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f9e28f9f2b22520 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.651+0000 7f90dffff640 1 -- 172.21.15.83:0/2085339455 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d80aa610 con 0x7f90e815f1d0 2024-04-17T10:39:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.651+0000 7f90dffff640 1 -- 172.21.15.83:0/2085339455 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d80af070 con 0x7f90e815f1d0 2024-04-17T10:39:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.651+0000 7f90dffff640 1 -- 172.21.15.83:0/2085339455 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d80aa9d0 con 0x7f90e815f1d0 2024-04-17T10:39:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.651+0000 7f90efa13640 1 -- 172.21.15.83:0/2085339455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90e815f1d0 msgr2=0x7f90e815f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.651+0000 7f90efa13640 1 --2- 172.21.15.83:0/2085339455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90e815f1d0 0x7f90e815f830 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f90d8009870 tx=0x7f90d801c4f0 comp rx=0 tx=0).stop 2024-04-17T10:39:07.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.651+0000 7f90efa13640 1 -- 172.21.15.83:0/2085339455 shutdown_connections 2024-04-17T10:39:07.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.651+0000 7f90efa13640 1 --2- 172.21.15.83:0/2085339455 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90e80d22d0 0x7f90e80ae520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:07.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.651+0000 7f90efa13640 1 --2- 172.21.15.83:0/2085339455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90e815f1d0 0x7f90e815f830 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:07.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.651+0000 7f90efa13640 1 -- 172.21.15.83:0/2085339455 >> 172.21.15.83:0/2085339455 conn(0x7f90e80d17b0 msgr2=0x7f90e80d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:07.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.651+0000 7f90efa13640 1 -- 172.21.15.83:0/2085339455 shutdown_connections 2024-04-17T10:39:07.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.651+0000 7f90efa13640 1 -- 172.21.15.83:0/2085339455 wait complete. 2024-04-17T10:39:07.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.655+0000 7f90efa13640 1 Processor -- start 2024-04-17T10:39:07.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.655+0000 7f90efa13640 1 -- start start 2024-04-17T10:39:07.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.655+0000 7f90efa13640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90e80d22d0 0x7f90e81498d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:07.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.655+0000 7f90efa13640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90e815f1d0 0x7f90e8149e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:07.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.655+0000 7f90efa13640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90e806d8e0 0x7f90e815cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:07.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.655+0000 7f90efa13640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e80af270 con 0x7f90e815f1d0 2024-04-17T10:39:07.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.655+0000 7f90efa13640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e8160690 con 0x7f90e806d8e0 2024-04-17T10:39:07.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.655+0000 7f90efa13640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e80af3f0 con 0x7f90e80d22d0 2024-04-17T10:39:07.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.655+0000 7f90ecfae640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90e815f1d0 0x7f90e8149e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:07.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.655+0000 7f90ecfae640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90e815f1d0 0x7f90e8149e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36916/0 (socket says 172.21.15.83:36916) 2024-04-17T10:39:07.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.655+0000 7f90ecfae640 1 -- 172.21.15.83:0/579990992 learned_addr learned my addr 172.21.15.83:0/579990992 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:07.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.655+0000 7f90edfb0640 1 --2- 172.21.15.83:0/579990992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90e806d8e0 0x7f90e815cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.655+0000 7f90ed7af640 1 --2- 172.21.15.83:0/579990992 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90e80d22d0 0x7f90e81498d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.655+0000 7f90ecfae640 1 -- 172.21.15.83:0/579990992 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90e80d22d0 msgr2=0x7f90e81498d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.655+0000 7f90ecfae640 1 --2- 172.21.15.83:0/579990992 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90e80d22d0 0x7f90e81498d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.655+0000 7f90ecfae640 1 -- 172.21.15.83:0/579990992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90e806d8e0 msgr2=0x7f90e815cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.655+0000 7f90ecfae640 1 --2- 172.21.15.83:0/579990992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90e806d8e0 0x7f90e815cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.655+0000 7f90ecfae640 1 -- 172.21.15.83:0/579990992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e80c9410 con 0x7f90e815f1d0 2024-04-17T10:39:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.655+0000 7f90ed7af640 1 --2- 172.21.15.83:0/579990992 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90e80d22d0 0x7f90e81498d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:07.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.655+0000 7f90ecfae640 1 --2- 172.21.15.83:0/579990992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90e815f1d0 0x7f90e8149e10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f90e400c480 tx=0x7f90e400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:07.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.655+0000 7f90ddffb640 1 -- 172.21.15.83:0/579990992 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e4012830 con 0x7f90e815f1d0 2024-04-17T10:39:07.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.655+0000 7f90ddffb640 1 -- 172.21.15.83:0/579990992 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e4017070 con 0x7f90e815f1d0 2024-04-17T10:39:07.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.655+0000 7f90efa13640 1 -- 172.21.15.83:0/579990992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e80c96f0 con 0x7f90e815f1d0 2024-04-17T10:39:07.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.655+0000 7f90ddffb640 1 -- 172.21.15.83:0/579990992 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e4012c30 con 0x7f90e815f1d0 2024-04-17T10:39:07.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.655+0000 7f90efa13640 1 -- 172.21.15.83:0/579990992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e815e620 con 0x7f90e815f1d0 2024-04-17T10:39:07.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.655+0000 7f90efa13640 1 -- 172.21.15.83:0/579990992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e8074f90 con 0x7f90e815f1d0 2024-04-17T10:39:07.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.659+0000 7f90ddffb640 1 -- 172.21.15.83:0/579990992 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f90e401c460 con 0x7f90e815f1d0 2024-04-17T10:39:07.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.659+0000 7f90ddffb640 1 --2- 172.21.15.83:0/579990992 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f90cc066f60 0x7f90cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:07.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.659+0000 7f90ddffb640 1 -- 172.21.15.83:0/579990992 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f90e4084a90 con 0x7f90e815f1d0 2024-04-17T10:39:07.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.659+0000 7f90ed7af640 1 --2- 172.21.15.83:0/579990992 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f90cc066f60 0x7f90cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:07.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.659+0000 7f90ed7af640 1 --2- 172.21.15.83:0/579990992 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f90cc066f60 0x7f90cc069420 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f90d80ae000 tx=0x7f90d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:07.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.663+0000 7f90ddffb640 1 -- 172.21.15.83:0/579990992 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f90e4055e80 con 0x7f90e815f1d0 2024-04-17T10:39:07.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.879+0000 7f90efa13640 1 -- 172.21.15.83:0/579990992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f90e8072690 con 0x7f90e815f1d0 2024-04-17T10:39:07.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.879+0000 7f90ddffb640 1 -- 172.21.15.83:0/579990992 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f90e4055780 con 0x7f90e815f1d0 2024-04-17T10:39:07.876 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:39:07.876 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:39:07.876 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_dump": [ 2024-04-17T10:39:07.876 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:39:07.876 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.a", 2024-04-17T10:39:07.876 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mfpxsNxAANpL7rcWevXwd2KMftSQnEQ==", 2024-04-17T10:39:07.876 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:39:07.877 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:39:07.877 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:39:07.877 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:39:07.877 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:39:07.877 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:39:07.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:39:07.877 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:39:07.877 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.b", 2024-04-17T10:39:07.877 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mXlI9ChAAYdcrGVmtYdtWWgCLVmpTTg==", 2024-04-17T10:39:07.877 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:39:07.877 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:39:07.878 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:39:07.878 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:39:07.878 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:39:07.878 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:39:07.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:39:07.878 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:39:07.878 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.c", 2024-04-17T10:39:07.878 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mupo3BBAAyKH801vpFLXYs9mfzSlT0w==", 2024-04-17T10:39:07.878 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:39:07.878 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:39:07.878 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:39:07.878 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:39:07.879 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:39:07.879 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:39:07.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:39:07.879 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:39:07.879 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.d", 2024-04-17T10:39:07.879 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mbm++EhAAIBMOtQoD+kPeMqR6gK5ctw==", 2024-04-17T10:39:07.879 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:39:07.879 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:39:07.879 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:39:07.879 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:39:07.879 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:39:07.879 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:39:07.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:39:07.880 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:39:07.880 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.0", 2024-04-17T10:39:07.880 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBXph9m1HWhIRAAOq+9nGq7OGonUCE2aeiAzQ==", 2024-04-17T10:39:07.880 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:39:07.880 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:39:07.880 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:39:07.880 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:39:07.880 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:39:07.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:39:07.880 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:39:07.880 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.1", 2024-04-17T10:39:07.880 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBaph9mlMmtJRAAXHLMjXfm/+WjIFURLkbAfA==", 2024-04-17T10:39:07.880 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:39:07.880 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:39:07.880 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:39:07.881 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:39:07.881 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:39:07.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:39:07.881 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:39:07.881 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.2", 2024-04-17T10:39:07.881 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBdph9mBsNPLRAAtR5hwGdDkjLDLS19QUDDYQ==", 2024-04-17T10:39:07.881 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:39:07.881 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:39:07.881 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:39:07.881 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:39:07.881 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:39:07.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:39:07.881 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:39:07.881 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.3", 2024-04-17T10:39:07.881 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBgph9mZ4EwNRAAIyL7e0EK1523dPXmbOOkzw==", 2024-04-17T10:39:07.881 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:39:07.882 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:39:07.882 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:39:07.882 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:39:07.882 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:39:07.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:39:07.882 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:39:07.882 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.4", 2024-04-17T10:39:07.882 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBmph9mphAsMhAAtF4uV28Nb66PNjuugB12Hw==", 2024-04-17T10:39:07.882 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:39:07.882 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:39:07.882 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:39:07.882 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:39:07.882 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:39:07.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:39:07.882 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:39:07.883 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.5", 2024-04-17T10:39:07.883 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBpph9mHJQNNxAA72eOFYFbhakQac1JKRNjzw==", 2024-04-17T10:39:07.883 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:39:07.883 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:39:07.883 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:39:07.883 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:39:07.883 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:39:07.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:39:07.883 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:39:07.883 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.6", 2024-04-17T10:39:07.883 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBtph9mb6RKAxAAoVX6Cp7l7j3Lm1ES2Sq+fQ==", 2024-04-17T10:39:07.883 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:39:07.883 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:39:07.883 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:39:07.884 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:39:07.884 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:39:07.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:39:07.884 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:39:07.884 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.7", 2024-04-17T10:39:07.884 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBwph9mdrdrBxAApDwjml2ahsLbRol4uL/BxA==", 2024-04-17T10:39:07.884 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:39:07.884 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:39:07.884 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:39:07.884 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:39:07.884 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:39:07.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:39:07.884 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:39:07.885 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.0", 2024-04-17T10:39:07.885 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mHGmXGBAAZF+JiAG9YrFtbW+dCy2IEQ==", 2024-04-17T10:39:07.885 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:39:07.885 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:39:07.885 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-17T10:39:07.885 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-17T10:39:07.885 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-17T10:39:07.885 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:39:07.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:39:07.885 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:39:07.885 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.1", 2024-04-17T10:39:07.886 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mB7wgIBAABdavAo/uzVuSjfkkcYtqJg==", 2024-04-17T10:39:07.886 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:39:07.886 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:39:07.886 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-17T10:39:07.886 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-17T10:39:07.886 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-17T10:39:07.886 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:39:07.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:39:07.886 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:39:07.886 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.2", 2024-04-17T10:39:07.887 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9m+wKgHRAAe8IUFx34ItLpsaDqSYyEgA==", 2024-04-17T10:39:07.887 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:39:07.887 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:39:07.887 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-17T10:39:07.887 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-17T10:39:07.887 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-17T10:39:07.887 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:39:07.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:39:07.887 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:39:07.887 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.3", 2024-04-17T10:39:07.887 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mfJ0yJRAAHH7rRh/BUIujTT04AF2l0w==", 2024-04-17T10:39:07.887 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:39:07.887 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:39:07.888 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-17T10:39:07.888 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-17T10:39:07.888 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-17T10:39:07.888 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:39:07.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:39:07.888 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:39:07.888 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.admin", 2024-04-17T10:39:07.888 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mjJ3GIBAA/2qaCbRiLJ5X1cv3ZGdfoA==", 2024-04-17T10:39:07.888 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:39:07.888 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-17T10:39:07.888 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:39:07.888 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:39:07.889 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:39:07.889 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:39:07.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:39:07.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:39:07.889 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.bootstrap-mds", 2024-04-17T10:39:07.889 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB7ph9m2ObvERAAq1TMzDJz4P/lASOO7G5Jdg==", 2024-04-17T10:39:07.889 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:39:07.889 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile bootstrap-mds" 2024-04-17T10:39:07.889 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:39:07.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:39:07.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:39:07.890 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.bootstrap-mgr", 2024-04-17T10:39:07.890 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB7ph9mr//vERAACEuOVq2V42fRn2MePCcfVA==", 2024-04-17T10:39:07.890 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:39:07.890 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-17T10:39:07.890 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:39:07.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:39:07.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:39:07.890 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.bootstrap-osd", 2024-04-17T10:39:07.891 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB7ph9mkhXwERAA2nITdRQs+2xWLo4PTKMP/A==", 2024-04-17T10:39:07.891 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:39:07.891 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile bootstrap-osd" 2024-04-17T10:39:07.891 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:39:07.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:39:07.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:39:07.891 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.bootstrap-rbd", 2024-04-17T10:39:07.891 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB7ph9mtirwERAADYJ0o6b/HawLh/HURn2fQQ==", 2024-04-17T10:39:07.891 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:39:07.891 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-17T10:39:07.891 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:39:07.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:39:07.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:39:07.892 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-17T10:39:07.892 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB7ph9mhUDwERAAQhFSI5KFtrH/fAcRxZ7jpg==", 2024-04-17T10:39:07.892 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:39:07.892 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-17T10:39:07.892 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:39:07.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:39:07.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:39:07.892 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.bootstrap-rgw", 2024-04-17T10:39:07.892 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB7ph9mJljwERAA/YzRWzKLcAmTzNM1Im0fFQ==", 2024-04-17T10:39:07.892 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:39:07.893 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-17T10:39:07.893 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:39:07.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:39:07.893 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:39:07.893 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.x", 2024-04-17T10:39:07.893 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mMFptNBAA/G0zEfMZTCy9Iox+MwHG3Q==", 2024-04-17T10:39:07.893 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:39:07.893 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-17T10:39:07.893 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-04-17T10:39:07.893 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:39:07.893 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:39:07.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:39:07.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:39:07.894 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.y", 2024-04-17T10:39:07.894 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mjFbRMRAAjY2d8POBzp8hEp9GSO0ylQ==", 2024-04-17T10:39:07.894 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:39:07.894 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-17T10:39:07.894 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-04-17T10:39:07.894 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:39:07.894 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:39:07.894 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:39:07.895 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-17T10:39:07.895 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:39:07.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.879+0000 7f90efa13640 1 -- 172.21.15.83:0/579990992 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f90cc066f60 msgr2=0x7f90cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:07.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.879+0000 7f90efa13640 1 --2- 172.21.15.83:0/579990992 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f90cc066f60 0x7f90cc069420 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f90d80ae000 tx=0x7f90d8008040 comp rx=0 tx=0).stop 2024-04-17T10:39:07.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.879+0000 7f90efa13640 1 -- 172.21.15.83:0/579990992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90e815f1d0 msgr2=0x7f90e8149e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:07.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.879+0000 7f90efa13640 1 --2- 172.21.15.83:0/579990992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90e815f1d0 0x7f90e8149e10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f90e400c480 tx=0x7f90e400c940 comp rx=0 tx=0).stop 2024-04-17T10:39:07.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.879+0000 7f90efa13640 1 -- 172.21.15.83:0/579990992 shutdown_connections 2024-04-17T10:39:07.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.879+0000 7f90efa13640 1 --2- 172.21.15.83:0/579990992 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f90cc066f60 0x7f90cc069420 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:07.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.879+0000 7f90efa13640 1 --2- 172.21.15.83:0/579990992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90e806d8e0 0x7f90e815cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:07.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.879+0000 7f90efa13640 1 --2- 172.21.15.83:0/579990992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90e815f1d0 0x7f90e8149e10 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:07.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.879+0000 7f90efa13640 1 --2- 172.21.15.83:0/579990992 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90e80d22d0 0x7f90e81498d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:07.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.879+0000 7f90efa13640 1 -- 172.21.15.83:0/579990992 >> 172.21.15.83:0/579990992 conn(0x7f90e80d17b0 msgr2=0x7f90e806f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:07.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.879+0000 7f90efa13640 1 -- 172.21.15.83:0/579990992 shutdown_connections 2024-04-17T10:39:07.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:07.879+0000 7f90efa13640 1 -- 172.21.15.83:0/579990992 wait complete. 2024-04-17T10:39:07.898 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-17T10:39:08.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.015+0000 7f8bcadab640 1 Processor -- start 2024-04-17T10:39:08.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.015+0000 7f8bcadab640 1 -- start start 2024-04-17T10:39:08.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.015+0000 7f8bcadab640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8bc4154de0 0x7f8bc406ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:08.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.015+0000 7f8bcadab640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8bc40b4210 0x7f8bc406cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:08.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.015+0000 7f8bcadab640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bc415bd70 con 0x7f8bc406b350 2024-04-17T10:39:08.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.015+0000 7f8bcadab640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc40b4870 con 0x7f8bc40b4210 2024-04-17T10:39:08.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.015+0000 7f8bcadab640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc415bf60 con 0x7f8bc4154de0 2024-04-17T10:39:08.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.015+0000 7f8bc9348640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8bc40b4210 0x7f8bc406cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:08.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.015+0000 7f8bc9348640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8bc40b4210 0x7f8bc406cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36918/0 (socket says 172.21.15.83:36918) 2024-04-17T10:39:08.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.015+0000 7f8bc9348640 1 -- 172.21.15.83:0/2602408843 learned_addr learned my addr 172.21.15.83:0/2602408843 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:08.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.015+0000 7f8bc8b47640 1 --2- 172.21.15.83:0/2602408843 >> v2:172.21.15.167:3300/0 conn(0x7f8bc4154de0 0x7f8bc406ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:08.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.015+0000 7f8bc9348640 1 -- 172.21.15.83:0/2602408843 >> v2:172.21.15.167:3300/0 conn(0x7f8bc4154de0 msgr2=0x7f8bc406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:08.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.015+0000 7f8bc9348640 1 --2- 172.21.15.83:0/2602408843 >> v2:172.21.15.167:3300/0 conn(0x7f8bc4154de0 0x7f8bc406ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:08.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.015+0000 7f8bc9348640 1 -- 172.21.15.83:0/2602408843 >> v1:172.21.15.167:6789/0 conn(0x7f8bc406b350 legacy=0x7f8bc40b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:08.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.015+0000 7f8bc9348640 1 -- 172.21.15.83:0/2602408843 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc415afb0 con 0x7f8bc40b4210 2024-04-17T10:39:08.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.015+0000 7f8bc8b47640 1 --2- 172.21.15.83:0/2602408843 >> v2:172.21.15.167:3300/0 conn(0x7f8bc4154de0 0x7f8bc406ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:08.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.019+0000 7f8bc9348640 1 --2- 172.21.15.83:0/2602408843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bc40b4210 0x7f8bc406cbe0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40b2de0 tx=0x7f8bbc01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9884b70480069cca server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:08.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.019+0000 7f8bc37fe640 1 -- 172.21.15.83:0/2602408843 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc0a5670 con 0x7f8bc40b4210 2024-04-17T10:39:08.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.019+0000 7f8bc37fe640 1 -- 172.21.15.83:0/2602408843 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bbc0a5810 con 0x7f8bc40b4210 2024-04-17T10:39:08.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.019+0000 7f8bc37fe640 1 -- 172.21.15.83:0/2602408843 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc0a5be0 con 0x7f8bc40b4210 2024-04-17T10:39:08.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.019+0000 7f8bcadab640 1 -- 172.21.15.83:0/2602408843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bc40b4210 msgr2=0x7f8bc406cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:08.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.019+0000 7f8bcadab640 1 --2- 172.21.15.83:0/2602408843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bc40b4210 0x7f8bc406cbe0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40b2de0 tx=0x7f8bbc01c9b0 comp rx=0 tx=0).stop 2024-04-17T10:39:08.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.019+0000 7f8bcadab640 1 -- 172.21.15.83:0/2602408843 shutdown_connections 2024-04-17T10:39:08.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.019+0000 7f8bcadab640 1 --2- 172.21.15.83:0/2602408843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bc40b4210 0x7f8bc406cbe0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:08.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.019+0000 7f8bcadab640 1 --2- 172.21.15.83:0/2602408843 >> v2:172.21.15.167:3300/0 conn(0x7f8bc4154de0 0x7f8bc406ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:08.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.019+0000 7f8bcadab640 1 -- 172.21.15.83:0/2602408843 >> 172.21.15.83:0/2602408843 conn(0x7f8bc40b15b0 msgr2=0x7f8bc40b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:08.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.019+0000 7f8bcadab640 1 -- 172.21.15.83:0/2602408843 shutdown_connections 2024-04-17T10:39:08.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.019+0000 7f8bcadab640 1 -- 172.21.15.83:0/2602408843 wait complete. 2024-04-17T10:39:08.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.019+0000 7f8bcadab640 1 Processor -- start 2024-04-17T10:39:08.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.019+0000 7f8bcadab640 1 -- start start 2024-04-17T10:39:08.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.023+0000 7f8bcadab640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8bc406b350 0x7f8bc414b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:08.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.023+0000 7f8bcadab640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bc40b4210 0x7f8bc4159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:08.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.023+0000 7f8bcadab640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bc4154de0 0x7f8bc40ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:08.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.023+0000 7f8bcadab640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc415e390 con 0x7f8bc4154de0 2024-04-17T10:39:08.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.023+0000 7f8bcadab640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc41551c0 con 0x7f8bc40b4210 2024-04-17T10:39:08.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.023+0000 7f8bcadab640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc415e510 con 0x7f8bc406b350 2024-04-17T10:39:08.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.023+0000 7f8bc9348640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bc4154de0 0x7f8bc40ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:08.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.023+0000 7f8bc9348640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bc4154de0 0x7f8bc40ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36924/0 (socket says 172.21.15.83:36924) 2024-04-17T10:39:08.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.023+0000 7f8bc9348640 1 -- 172.21.15.83:0/2214920388 learned_addr learned my addr 172.21.15.83:0/2214920388 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:08.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.023+0000 7f8bc8b47640 1 --2- 172.21.15.83:0/2214920388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8bc406b350 0x7f8bc414b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:08.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.023+0000 7f8bc3fff640 1 --2- 172.21.15.83:0/2214920388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bc40b4210 0x7f8bc4159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:08.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.023+0000 7f8bc9348640 1 -- 172.21.15.83:0/2214920388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8bc406b350 msgr2=0x7f8bc414b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:08.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.023+0000 7f8bc9348640 1 --2- 172.21.15.83:0/2214920388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8bc406b350 0x7f8bc414b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:08.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.023+0000 7f8bc9348640 1 -- 172.21.15.83:0/2214920388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bc40b4210 msgr2=0x7f8bc4159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:08.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.023+0000 7f8bc9348640 1 --2- 172.21.15.83:0/2214920388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bc40b4210 0x7f8bc4159130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:08.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.023+0000 7f8bc9348640 1 -- 172.21.15.83:0/2214920388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc4159bf0 con 0x7f8bc4154de0 2024-04-17T10:39:08.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.023+0000 7f8bc3fff640 1 --2- 172.21.15.83:0/2214920388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bc40b4210 0x7f8bc4159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:08.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.023+0000 7f8bc8b47640 1 --2- 172.21.15.83:0/2214920388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8bc406b350 0x7f8bc414b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:08.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.023+0000 7f8bc9348640 1 --2- 172.21.15.83:0/2214920388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bc4154de0 0x7f8bc40ad840 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0af040 tx=0x7f8bbc019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:08.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.023+0000 7f8bc17fa640 1 -- 172.21.15.83:0/2214920388 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc0b8670 con 0x7f8bc4154de0 2024-04-17T10:39:08.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.023+0000 7f8bc17fa640 1 -- 172.21.15.83:0/2214920388 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bbc0be070 con 0x7f8bc4154de0 2024-04-17T10:39:08.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.023+0000 7f8bcadab640 1 -- 172.21.15.83:0/2214920388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc4159ed0 con 0x7f8bc4154de0 2024-04-17T10:39:08.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.023+0000 7f8bc17fa640 1 -- 172.21.15.83:0/2214920388 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc0b8a50 con 0x7f8bc4154de0 2024-04-17T10:39:08.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.023+0000 7f8bcadab640 1 -- 172.21.15.83:0/2214920388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc415a490 con 0x7f8bc4154de0 2024-04-17T10:39:08.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.023+0000 7f8bcadab640 1 -- 172.21.15.83:0/2214920388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc4074f90 con 0x7f8bc4154de0 2024-04-17T10:39:08.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.023+0000 7f8bc17fa640 1 -- 172.21.15.83:0/2214920388 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8bbc0b8bf0 con 0x7f8bc4154de0 2024-04-17T10:39:08.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.023+0000 7f8bc17fa640 1 --2- 172.21.15.83:0/2214920388 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8bb0066f60 0x7f8bb0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:08.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.023+0000 7f8bc17fa640 1 -- 172.21.15.83:0/2214920388 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8bbc11ff90 con 0x7f8bc4154de0 2024-04-17T10:39:08.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.023+0000 7f8bc8b47640 1 --2- 172.21.15.83:0/2214920388 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8bb0066f60 0x7f8bb0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:08.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.027+0000 7f8bc8b47640 1 --2- 172.21.15.83:0/2214920388 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8bb0066f60 0x7f8bb0069420 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4008890 tx=0x7f8bb4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:08.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.031+0000 7f8bc17fa640 1 -- 172.21.15.83:0/2214920388 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8bbc0165a0 con 0x7f8bc4154de0 2024-04-17T10:39:08.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.247+0000 7f8bcadab640 1 -- 172.21.15.83:0/2214920388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f8bc40b3ea0 con 0x7f8bc4154de0 2024-04-17T10:39:08.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.251+0000 7f8bc17fa640 1 -- 172.21.15.83:0/2214920388 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8bbc0f0c20 con 0x7f8bc4154de0 2024-04-17T10:39:08.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.251+0000 7f8bcadab640 1 -- 172.21.15.83:0/2214920388 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8bb0066f60 msgr2=0x7f8bb0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:08.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.251+0000 7f8bcadab640 1 --2- 172.21.15.83:0/2214920388 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8bb0066f60 0x7f8bb0069420 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4008890 tx=0x7f8bb4008040 comp rx=0 tx=0).stop 2024-04-17T10:39:08.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.251+0000 7f8bcadab640 1 -- 172.21.15.83:0/2214920388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bc4154de0 msgr2=0x7f8bc40ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:08.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.251+0000 7f8bcadab640 1 --2- 172.21.15.83:0/2214920388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bc4154de0 0x7f8bc40ad840 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0af040 tx=0x7f8bbc019750 comp rx=0 tx=0).stop 2024-04-17T10:39:08.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.251+0000 7f8bcadab640 1 -- 172.21.15.83:0/2214920388 shutdown_connections 2024-04-17T10:39:08.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.251+0000 7f8bcadab640 1 --2- 172.21.15.83:0/2214920388 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8bb0066f60 0x7f8bb0069420 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:08.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.251+0000 7f8bcadab640 1 --2- 172.21.15.83:0/2214920388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bc4154de0 0x7f8bc40ad840 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:08.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.251+0000 7f8bcadab640 1 --2- 172.21.15.83:0/2214920388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bc40b4210 0x7f8bc4159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:08.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.251+0000 7f8bcadab640 1 --2- 172.21.15.83:0/2214920388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8bc406b350 0x7f8bc414b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:08.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.251+0000 7f8bcadab640 1 -- 172.21.15.83:0/2214920388 >> 172.21.15.83:0/2214920388 conn(0x7f8bc40b15b0 msgr2=0x7f8bc4073c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:08.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.255+0000 7f8bcadab640 1 -- 172.21.15.83:0/2214920388 shutdown_connections 2024-04-17T10:39:08.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.255+0000 7f8bcadab640 1 -- 172.21.15.83:0/2214920388 wait complete. 2024-04-17T10:39:08.268 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-17T10:39:08.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d8addc640 1 Processor -- start 2024-04-17T10:39:08.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d8addc640 1 -- start start 2024-04-17T10:39:08.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d8addc640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3d840ce400 0x7f3d840b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:08.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d8addc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d840b5450 0x7f3d840b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:08.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d8addc640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d8415e4d0 con 0x7f3d8415e0f0 2024-04-17T10:39:08.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d8addc640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d84162b40 con 0x7f3d840ce400 2024-04-17T10:39:08.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d8addc640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d84162320 con 0x7f3d840b5450 2024-04-17T10:39:08.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d88b78640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3d8415e0f0 0x7f3d84161c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48598/0 (socket says 172.21.15.83:48598) 2024-04-17T10:39:08.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d88b78640 1 -- 172.21.15.83:0/1552682108 learned_addr learned my addr 172.21.15.83:0/1552682108 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:39:08.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d83fff640 1 --2- 172.21.15.83:0/1552682108 >> v2:172.21.15.167:3300/0 conn(0x7f3d840ce400 0x7f3d840b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:08.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d89379640 1 --2- 172.21.15.83:0/1552682108 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d840b5450 0x7f3d840b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:08.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d837fe640 1 -- 172.21.15.83:0/1552682108 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2421598576 0 0) 0x7f3d8415e4d0 con 0x7f3d8415e0f0 2024-04-17T10:39:08.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d837fe640 1 -- 172.21.15.83:0/1552682108 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d6c003530 con 0x7f3d8415e0f0 2024-04-17T10:39:08.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d837fe640 1 -- 172.21.15.83:0/1552682108 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1410536092 0 0) 0x7f3d6c003530 con 0x7f3d8415e0f0 2024-04-17T10:39:08.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d837fe640 1 -- 172.21.15.83:0/1552682108 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d840b5450 msgr2=0x7f3d840b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:08.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d837fe640 1 --2- 172.21.15.83:0/1552682108 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d840b5450 0x7f3d840b58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:08.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d837fe640 1 -- 172.21.15.83:0/1552682108 >> v2:172.21.15.167:3300/0 conn(0x7f3d840ce400 msgr2=0x7f3d840b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:08.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d837fe640 1 --2- 172.21.15.83:0/1552682108 >> v2:172.21.15.167:3300/0 conn(0x7f3d840ce400 0x7f3d840b6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:08.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d837fe640 1 -- 172.21.15.83:0/1552682108 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d84162580 con 0x7f3d8415e0f0 2024-04-17T10:39:08.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d837fe640 1 -- 172.21.15.83:0/1552682108 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f3d74001680 con 0x7f3d8415e0f0 2024-04-17T10:39:08.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d837fe640 1 -- 172.21.15.83:0/1552682108 >> v1:172.21.15.83:6789/0 conn(0x7f3d8415e0f0 legacy=0x7f3d84161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:08.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d837fe640 1 --2- 172.21.15.83:0/1552682108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d6c003a60 0x7f3d6c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:08.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d837fe640 1 --2- 172.21.15.83:0/1552682108 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d840b5450 0x7f3d6c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:08.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d837fe640 1 --2- 172.21.15.83:0/1552682108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d6c004d50 0x7f3d6c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:08.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d837fe640 1 -- 172.21.15.83:0/1552682108 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d84162b40 con 0x7f3d6c004d50 2024-04-17T10:39:08.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d837fe640 1 -- 172.21.15.83:0/1552682108 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d84162320 con 0x7f3d6c003a60 2024-04-17T10:39:08.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d837fe640 1 -- 172.21.15.83:0/1552682108 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c00ba80 con 0x7f3d840b5450 2024-04-17T10:39:08.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d89379640 1 --2- 172.21.15.83:0/1552682108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d6c004d50 0x7f3d6c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:08.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d88b78640 1 --2- 172.21.15.83:0/1552682108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d6c003a60 0x7f3d6c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:08.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d83fff640 1 --2- 172.21.15.83:0/1552682108 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d840b5450 0x7f3d6c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:08.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d89379640 1 -- 172.21.15.83:0/1552682108 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d840b5450 msgr2=0x7f3d6c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:08.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d89379640 1 --2- 172.21.15.83:0/1552682108 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d840b5450 0x7f3d6c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:08.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d89379640 1 -- 172.21.15.83:0/1552682108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d6c003a60 msgr2=0x7f3d6c003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:08.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d89379640 1 --2- 172.21.15.83:0/1552682108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d6c003a60 0x7f3d6c003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:08.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.391+0000 7f3d89379640 1 -- 172.21.15.83:0/1552682108 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d6c003530 con 0x7f3d6c004d50 2024-04-17T10:39:08.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.395+0000 7f3d89379640 1 --2- 172.21.15.83:0/1552682108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d6c004d50 0x7f3d6c0051d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c009fa0 tx=0x7f3d7c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24303ed426194db3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:08.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.395+0000 7f3d837fe640 1 -- 172.21.15.83:0/1552682108 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c0b2070 con 0x7f3d6c004d50 2024-04-17T10:39:08.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.395+0000 7f3d837fe640 1 -- 172.21.15.83:0/1552682108 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7c0a5c00 con 0x7f3d6c004d50 2024-04-17T10:39:08.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.395+0000 7f3d837fe640 1 -- 172.21.15.83:0/1552682108 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c009bf0 con 0x7f3d6c004d50 2024-04-17T10:39:08.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.395+0000 7f3d8addc640 1 -- 172.21.15.83:0/1552682108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d6c004d50 msgr2=0x7f3d6c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:08.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.395+0000 7f3d8addc640 1 --2- 172.21.15.83:0/1552682108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d6c004d50 0x7f3d6c0051d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c009fa0 tx=0x7f3d7c01cc40 comp rx=0 tx=0).stop 2024-04-17T10:39:08.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.395+0000 7f3d89379640 1 -- 172.21.15.83:0/1552682108 reap_dead start 2024-04-17T10:39:08.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.395+0000 7f3d8addc640 1 -- 172.21.15.83:0/1552682108 shutdown_connections 2024-04-17T10:39:08.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.395+0000 7f3d8addc640 1 -- 172.21.15.83:0/1552682108 >> 172.21.15.83:0/1552682108 conn(0x7f3d840d0450 msgr2=0x7f3d840d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:08.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.395+0000 7f3d8addc640 1 -- 172.21.15.83:0/1552682108 shutdown_connections 2024-04-17T10:39:08.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.395+0000 7f3d8addc640 1 -- 172.21.15.83:0/1552682108 wait complete. 2024-04-17T10:39:08.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.395+0000 7f3d8addc640 1 Processor -- start 2024-04-17T10:39:08.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.399+0000 7f3d8addc640 1 -- start start 2024-04-17T10:39:08.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.399+0000 7f3d8addc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d840c9520 0x7f3d84149df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:08.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.399+0000 7f3d8addc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d8415f000 0x7f3d84160a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:08.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.399+0000 7f3d8addc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d84160fa0 0x7f3d841f7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:08.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.399+0000 7f3d8addc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d840d2020 con 0x7f3d840c9520 2024-04-17T10:39:08.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.399+0000 7f3d8addc640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d840d1d40 con 0x7f3d8415f000 2024-04-17T10:39:08.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.399+0000 7f3d8addc640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d840b58e0 con 0x7f3d84160fa0 2024-04-17T10:39:08.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.399+0000 7f3d88b78640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d840c9520 0x7f3d84149df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:08.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.399+0000 7f3d88b78640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d840c9520 0x7f3d84149df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36934/0 (socket says 172.21.15.83:36934) 2024-04-17T10:39:08.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.399+0000 7f3d88b78640 1 -- 172.21.15.83:0/1939883986 learned_addr learned my addr 172.21.15.83:0/1939883986 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:08.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.399+0000 7f3d89379640 1 --2- 172.21.15.83:0/1939883986 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d84160fa0 0x7f3d841f7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:08.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.399+0000 7f3d83fff640 1 --2- 172.21.15.83:0/1939883986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d8415f000 0x7f3d84160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:08.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.399+0000 7f3d88b78640 1 -- 172.21.15.83:0/1939883986 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d84160fa0 msgr2=0x7f3d841f7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:08.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.399+0000 7f3d88b78640 1 --2- 172.21.15.83:0/1939883986 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d84160fa0 0x7f3d841f7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:08.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.399+0000 7f3d88b78640 1 -- 172.21.15.83:0/1939883986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d8415f000 msgr2=0x7f3d84160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:08.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.399+0000 7f3d88b78640 1 --2- 172.21.15.83:0/1939883986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d8415f000 0x7f3d84160a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:08.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.399+0000 7f3d88b78640 1 -- 172.21.15.83:0/1939883986 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d8414a480 con 0x7f3d840c9520 2024-04-17T10:39:08.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.399+0000 7f3d83fff640 1 --2- 172.21.15.83:0/1939883986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d8415f000 0x7f3d84160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:08.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.399+0000 7f3d88b78640 1 --2- 172.21.15.83:0/1939883986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d840c9520 0x7f3d84149df0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3d74012490 tx=0x7f3d74012950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:08.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.399+0000 7f3d817fa640 1 -- 172.21.15.83:0/1939883986 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7401c070 con 0x7f3d840c9520 2024-04-17T10:39:08.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.399+0000 7f3d817fa640 1 -- 172.21.15.83:0/1939883986 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d74014810 con 0x7f3d840c9520 2024-04-17T10:39:08.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.399+0000 7f3d8addc640 1 -- 172.21.15.83:0/1939883986 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d8414a760 con 0x7f3d840c9520 2024-04-17T10:39:08.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.399+0000 7f3d817fa640 1 -- 172.21.15.83:0/1939883986 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d74014be0 con 0x7f3d840c9520 2024-04-17T10:39:08.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.399+0000 7f3d8addc640 1 -- 172.21.15.83:0/1939883986 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d8414ad20 con 0x7f3d840c9520 2024-04-17T10:39:08.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.399+0000 7f3d8addc640 1 -- 172.21.15.83:0/1939883986 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d84074f90 con 0x7f3d840c9520 2024-04-17T10:39:08.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.399+0000 7f3d817fa640 1 -- 172.21.15.83:0/1939883986 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3d7401f070 con 0x7f3d840c9520 2024-04-17T10:39:08.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.399+0000 7f3d817fa640 1 --2- 172.21.15.83:0/1939883986 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3d70066f90 0x7f3d70069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:08.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.399+0000 7f3d817fa640 1 -- 172.21.15.83:0/1939883986 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3d740875e0 con 0x7f3d840c9520 2024-04-17T10:39:08.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.399+0000 7f3d83fff640 1 --2- 172.21.15.83:0/1939883986 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3d70066f90 0x7f3d70069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:08.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.403+0000 7f3d83fff640 1 --2- 172.21.15.83:0/1939883986 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3d70066f90 0x7f3d70069450 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3d841f71f0 tx=0x7f3d78008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:08.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.403+0000 7f3d817fa640 1 -- 172.21.15.83:0/1939883986 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3d740589d0 con 0x7f3d840c9520 2024-04-17T10:39:08.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.627+0000 7f3d8addc640 1 -- 172.21.15.83:0/1939883986 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f3d840d17c0 con 0x7f3d840c9520 2024-04-17T10:39:08.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.631+0000 7f3d817fa640 1 -- 172.21.15.83:0/1939883986 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7f3d740582d0 con 0x7f3d840c9520 2024-04-17T10:39:08.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.631+0000 7f3d8addc640 1 -- 172.21.15.83:0/1939883986 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3d70066f90 msgr2=0x7f3d70069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:08.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.631+0000 7f3d8addc640 1 --2- 172.21.15.83:0/1939883986 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3d70066f90 0x7f3d70069450 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3d841f71f0 tx=0x7f3d78008040 comp rx=0 tx=0).stop 2024-04-17T10:39:08.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.631+0000 7f3d8addc640 1 -- 172.21.15.83:0/1939883986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d840c9520 msgr2=0x7f3d84149df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:08.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.631+0000 7f3d8addc640 1 --2- 172.21.15.83:0/1939883986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d840c9520 0x7f3d84149df0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3d74012490 tx=0x7f3d74012950 comp rx=0 tx=0).stop 2024-04-17T10:39:08.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.631+0000 7f3d8addc640 1 -- 172.21.15.83:0/1939883986 shutdown_connections 2024-04-17T10:39:08.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.631+0000 7f3d8addc640 1 --2- 172.21.15.83:0/1939883986 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3d70066f90 0x7f3d70069450 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:08.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.631+0000 7f3d8addc640 1 --2- 172.21.15.83:0/1939883986 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d84160fa0 0x7f3d841f7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:08.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.631+0000 7f3d8addc640 1 --2- 172.21.15.83:0/1939883986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d8415f000 0x7f3d84160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:08.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.631+0000 7f3d8addc640 1 --2- 172.21.15.83:0/1939883986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d840c9520 0x7f3d84149df0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:08.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.631+0000 7f3d8addc640 1 -- 172.21.15.83:0/1939883986 >> 172.21.15.83:0/1939883986 conn(0x7f3d840d0450 msgr2=0x7f3d8415bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:08.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.635+0000 7f3d8addc640 1 -- 172.21.15.83:0/1939883986 shutdown_connections 2024-04-17T10:39:08.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.635+0000 7f3d8addc640 1 -- 172.21.15.83:0/1939883986 wait complete. 2024-04-17T10:39:08.648 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-17T10:39:08.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.771+0000 7fcea844f640 1 Processor -- start 2024-04-17T10:39:08.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.771+0000 7fcea844f640 1 -- start start 2024-04-17T10:39:08.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.771+0000 7fcea844f640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcea006b2f0 0x7fcea00ae950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:08.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.771+0000 7fcea844f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcea00afe20 0x7fcea006dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:08.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.771+0000 7fcea844f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea00af9e0 con 0x7fcea015da50 2024-04-17T10:39:08.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.771+0000 7fcea844f640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcea00af3a0 con 0x7fcea00afe20 2024-04-17T10:39:08.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.771+0000 7fcea844f640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcea00aee90 con 0x7fcea006b2f0 2024-04-17T10:39:08.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.771+0000 7fcea69ec640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcea00afe20 0x7fcea006dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:08.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.771+0000 7fcea61eb640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcea015da50 0x7fcea006abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48602/0 (socket says 172.21.15.83:48602) 2024-04-17T10:39:08.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.771+0000 7fcea61eb640 1 -- 172.21.15.83:0/467172672 learned_addr learned my addr 172.21.15.83:0/467172672 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:39:08.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.771+0000 7fcea59ea640 1 --2- 172.21.15.83:0/467172672 >> v2:172.21.15.167:3300/0 conn(0x7fcea006b2f0 0x7fcea00ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:08.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.771+0000 7fcea51e9640 1 -- 172.21.15.83:0/467172672 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2045914821 0 0) 0x7fcea00af9e0 con 0x7fcea015da50 2024-04-17T10:39:08.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.771+0000 7fcea51e9640 1 -- 172.21.15.83:0/467172672 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce8c003530 con 0x7fcea015da50 2024-04-17T10:39:08.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.771+0000 7fcea69ec640 1 -- 172.21.15.83:0/467172672 >> v2:172.21.15.167:3300/0 conn(0x7fcea006b2f0 msgr2=0x7fcea00ae950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:08.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.771+0000 7fcea69ec640 1 --2- 172.21.15.83:0/467172672 >> v2:172.21.15.167:3300/0 conn(0x7fcea006b2f0 0x7fcea00ae950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:08.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.771+0000 7fcea69ec640 1 -- 172.21.15.83:0/467172672 >> v1:172.21.15.83:6789/0 conn(0x7fcea015da50 legacy=0x7fcea006abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:08.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.771+0000 7fcea69ec640 1 -- 172.21.15.83:0/467172672 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea00b4e80 con 0x7fcea00afe20 2024-04-17T10:39:08.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.775+0000 7fcea69ec640 1 --2- 172.21.15.83:0/467172672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcea00afe20 0x7fcea006dba0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcea0161550 tx=0x7fce9801c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b4a17c60d95f159 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:08.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.775+0000 7fcea51e9640 1 -- 172.21.15.83:0/467172672 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce980b2070 con 0x7fcea00afe20 2024-04-17T10:39:08.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.775+0000 7fcea51e9640 1 -- 172.21.15.83:0/467172672 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce980a5520 con 0x7fcea00afe20 2024-04-17T10:39:08.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.775+0000 7fcea51e9640 1 -- 172.21.15.83:0/467172672 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c001680 con 0x7fcea00afe20 2024-04-17T10:39:08.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.775+0000 7fcea844f640 1 -- 172.21.15.83:0/467172672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcea00afe20 msgr2=0x7fcea006dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:08.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.775+0000 7fcea844f640 1 --2- 172.21.15.83:0/467172672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcea00afe20 0x7fcea006dba0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcea0161550 tx=0x7fce9801c7b0 comp rx=0 tx=0).stop 2024-04-17T10:39:08.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.775+0000 7fcea844f640 1 -- 172.21.15.83:0/467172672 shutdown_connections 2024-04-17T10:39:08.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.775+0000 7fcea844f640 1 --2- 172.21.15.83:0/467172672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcea00afe20 0x7fcea006dba0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:08.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.775+0000 7fcea844f640 1 --2- 172.21.15.83:0/467172672 >> v2:172.21.15.167:3300/0 conn(0x7fcea006b2f0 0x7fcea00ae950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:08.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.775+0000 7fcea844f640 1 -- 172.21.15.83:0/467172672 >> 172.21.15.83:0/467172672 conn(0x7fcea00b24f0 msgr2=0x7fcea00b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:08.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.775+0000 7fcea844f640 1 -- 172.21.15.83:0/467172672 shutdown_connections 2024-04-17T10:39:08.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.775+0000 7fcea844f640 1 -- 172.21.15.83:0/467172672 wait complete. 2024-04-17T10:39:08.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.775+0000 7fcea844f640 1 Processor -- start 2024-04-17T10:39:08.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.775+0000 7fcea844f640 1 -- start start 2024-04-17T10:39:08.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.779+0000 7fcea844f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcea006b2f0 0x7fcea0152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:08.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.779+0000 7fcea844f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcea00afe20 0x7fcea014f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:08.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.779+0000 7fcea844f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcea015da50 0x7fcea01601b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:08.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.779+0000 7fcea844f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea006e8f0 con 0x7fcea015da50 2024-04-17T10:39:08.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.779+0000 7fcea844f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea00b4680 con 0x7fcea006b2f0 2024-04-17T10:39:08.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.779+0000 7fcea844f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea006ea70 con 0x7fcea00afe20 2024-04-17T10:39:08.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.779+0000 7fcea69ec640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcea015da50 0x7fcea01601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:08.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.779+0000 7fcea69ec640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcea015da50 0x7fcea01601b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36956/0 (socket says 172.21.15.83:36956) 2024-04-17T10:39:08.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.779+0000 7fcea69ec640 1 -- 172.21.15.83:0/1174301355 learned_addr learned my addr 172.21.15.83:0/1174301355 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:08.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.779+0000 7fcea61eb640 1 --2- 172.21.15.83:0/1174301355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcea006b2f0 0x7fcea0152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:08.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.779+0000 7fcea59ea640 1 --2- 172.21.15.83:0/1174301355 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcea00afe20 0x7fcea014f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:08.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.779+0000 7fcea69ec640 1 -- 172.21.15.83:0/1174301355 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcea00afe20 msgr2=0x7fcea014f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:08.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.779+0000 7fcea69ec640 1 --2- 172.21.15.83:0/1174301355 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcea00afe20 0x7fcea014f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:08.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.779+0000 7fcea69ec640 1 -- 172.21.15.83:0/1174301355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcea006b2f0 msgr2=0x7fcea0152c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:08.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.779+0000 7fcea69ec640 1 --2- 172.21.15.83:0/1174301355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcea006b2f0 0x7fcea0152c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:08.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.779+0000 7fcea69ec640 1 -- 172.21.15.83:0/1174301355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea00aeb40 con 0x7fcea015da50 2024-04-17T10:39:08.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.779+0000 7fcea61eb640 1 --2- 172.21.15.83:0/1174301355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcea006b2f0 0x7fcea0152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:08.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.779+0000 7fcea69ec640 1 --2- 172.21.15.83:0/1174301355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcea015da50 0x7fcea01601b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fce980b0040 tx=0x7fce98019550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:08.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.779+0000 7fce96ffd640 1 -- 172.21.15.83:0/1174301355 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce980bd020 con 0x7fcea015da50 2024-04-17T10:39:08.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.779+0000 7fce96ffd640 1 -- 172.21.15.83:0/1174301355 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce980b8670 con 0x7fcea015da50 2024-04-17T10:39:08.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.779+0000 7fce96ffd640 1 -- 172.21.15.83:0/1174301355 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce980b2040 con 0x7fcea015da50 2024-04-17T10:39:08.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.779+0000 7fcea844f640 1 -- 172.21.15.83:0/1174301355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea0151bc0 con 0x7fcea015da50 2024-04-17T10:39:08.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.779+0000 7fcea844f640 1 -- 172.21.15.83:0/1174301355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea0152180 con 0x7fcea015da50 2024-04-17T10:39:08.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.779+0000 7fcea844f640 1 -- 172.21.15.83:0/1174301355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea00683d0 con 0x7fcea015da50 2024-04-17T10:39:08.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.779+0000 7fce96ffd640 1 -- 172.21.15.83:0/1174301355 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fce980c0020 con 0x7fcea015da50 2024-04-17T10:39:08.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.779+0000 7fce96ffd640 1 --2- 172.21.15.83:0/1174301355 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fce84066f60 0x7fce84069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:08.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.779+0000 7fcea61eb640 1 --2- 172.21.15.83:0/1174301355 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fce84066f60 0x7fce84069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:08.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.779+0000 7fce96ffd640 1 -- 172.21.15.83:0/1174301355 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fce9811fa60 con 0x7fcea015da50 2024-04-17T10:39:08.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.779+0000 7fcea61eb640 1 --2- 172.21.15.83:0/1174301355 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fce84066f60 0x7fce84069420 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fce9c000c10 tx=0x7fce9c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:08.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:08.783+0000 7fce96ffd640 1 -- 172.21.15.83:0/1174301355 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fce980c7030 con 0x7fcea015da50 2024-04-17T10:39:08.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.003+0000 7fcea844f640 1 -- 172.21.15.83:0/1174301355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fcea00af640 con 0x7fcea015da50 2024-04-17T10:39:09.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.003+0000 7fce96ffd640 1 -- 172.21.15.83:0/1174301355 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fce98016460 con 0x7fcea015da50 2024-04-17T10:39:09.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.007+0000 7fcea844f640 1 -- 172.21.15.83:0/1174301355 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fce84066f60 msgr2=0x7fce84069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:09.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.007+0000 7fcea844f640 1 --2- 172.21.15.83:0/1174301355 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fce84066f60 0x7fce84069420 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fce9c000c10 tx=0x7fce9c009040 comp rx=0 tx=0).stop 2024-04-17T10:39:09.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.007+0000 7fcea844f640 1 -- 172.21.15.83:0/1174301355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcea015da50 msgr2=0x7fcea01601b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:09.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.007+0000 7fcea844f640 1 --2- 172.21.15.83:0/1174301355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcea015da50 0x7fcea01601b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fce980b0040 tx=0x7fce98019550 comp rx=0 tx=0).stop 2024-04-17T10:39:09.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.007+0000 7fcea844f640 1 -- 172.21.15.83:0/1174301355 shutdown_connections 2024-04-17T10:39:09.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.007+0000 7fcea844f640 1 --2- 172.21.15.83:0/1174301355 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fce84066f60 0x7fce84069420 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:09.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.007+0000 7fcea844f640 1 --2- 172.21.15.83:0/1174301355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcea015da50 0x7fcea01601b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:09.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.007+0000 7fcea844f640 1 --2- 172.21.15.83:0/1174301355 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcea00afe20 0x7fcea014f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:09.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.007+0000 7fcea844f640 1 --2- 172.21.15.83:0/1174301355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcea006b2f0 0x7fcea0152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:09.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.007+0000 7fcea844f640 1 -- 172.21.15.83:0/1174301355 >> 172.21.15.83:0/1174301355 conn(0x7fcea00b24f0 msgr2=0x7fcea00b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:09.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.007+0000 7fcea844f640 1 -- 172.21.15.83:0/1174301355 shutdown_connections 2024-04-17T10:39:09.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.007+0000 7fcea844f640 1 -- 172.21.15.83:0/1174301355 wait complete. 2024-04-17T10:39:09.022 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-17T10:39:09.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.135+0000 7ffb36377640 1 Processor -- start 2024-04-17T10:39:09.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.135+0000 7ffb36377640 1 -- start start 2024-04-17T10:39:09.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.135+0000 7ffb36377640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ffb300d0970 0x7ffb300d17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:09.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.135+0000 7ffb36377640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb300d26c0 0x7ffb3015d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:09.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.135+0000 7ffb36377640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb300ce940 con 0x7ffb300ce4e0 2024-04-17T10:39:09.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.135+0000 7ffb36377640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb300cf7d0 con 0x7ffb300d0970 2024-04-17T10:39:09.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.135+0000 7ffb36377640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb300d4890 con 0x7ffb300d26c0 2024-04-17T10:39:09.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.135+0000 7ffb2f7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ffb300ce4e0 0x7ffb300d1fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48610/0 (socket says 172.21.15.83:48610) 2024-04-17T10:39:09.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.135+0000 7ffb2f7fe640 1 -- 172.21.15.83:0/4256311529 learned_addr learned my addr 172.21.15.83:0/4256311529 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:39:09.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.135+0000 7ffb34914640 1 --2- 172.21.15.83:0/4256311529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb300d26c0 0x7ffb3015d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:09.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.135+0000 7ffb2ffff640 1 --2- 172.21.15.83:0/4256311529 >> v2:172.21.15.167:3300/0 conn(0x7ffb300d0970 0x7ffb300d17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:09.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.135+0000 7ffb2effd640 1 -- 172.21.15.83:0/4256311529 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2278977550 0 0) 0x7ffb18001120 con 0x7ffb300ce4e0 2024-04-17T10:39:09.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.135+0000 7ffb2effd640 1 -- 172.21.15.83:0/4256311529 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb1c003530 con 0x7ffb300ce4e0 2024-04-17T10:39:09.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.135+0000 7ffb34914640 1 -- 172.21.15.83:0/4256311529 >> v2:172.21.15.167:3300/0 conn(0x7ffb300d0970 msgr2=0x7ffb300d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:09.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.135+0000 7ffb2effd640 1 -- 172.21.15.83:0/4256311529 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2437983407 0 0) 0x7ffb1c003530 con 0x7ffb300ce4e0 2024-04-17T10:39:09.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.135+0000 7ffb34914640 1 --2- 172.21.15.83:0/4256311529 >> v2:172.21.15.167:3300/0 conn(0x7ffb300d0970 0x7ffb300d17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:09.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.135+0000 7ffb34914640 1 -- 172.21.15.83:0/4256311529 >> v1:172.21.15.83:6789/0 conn(0x7ffb300ce4e0 legacy=0x7ffb300d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:09.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.135+0000 7ffb34914640 1 -- 172.21.15.83:0/4256311529 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb300cebc0 con 0x7ffb300d26c0 2024-04-17T10:39:09.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.135+0000 7ffb2ffff640 1 --2- 172.21.15.83:0/4256311529 >> v2:172.21.15.167:3300/0 conn(0x7ffb300d0970 0x7ffb300d17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:39:09.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.139+0000 7ffb34914640 1 --2- 172.21.15.83:0/4256311529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb300d26c0 0x7ffb3015d620 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffb28009d90 tx=0x7ffb2801c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b589296ca9db87e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:09.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.139+0000 7ffb2effd640 1 -- 172.21.15.83:0/4256311529 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb280b2070 con 0x7ffb300d26c0 2024-04-17T10:39:09.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.139+0000 7ffb2effd640 1 -- 172.21.15.83:0/4256311529 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb280a5660 con 0x7ffb300d26c0 2024-04-17T10:39:09.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.139+0000 7ffb2effd640 1 -- 172.21.15.83:0/4256311529 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb18001850 con 0x7ffb300d26c0 2024-04-17T10:39:09.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.139+0000 7ffb36377640 1 -- 172.21.15.83:0/4256311529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb300d26c0 msgr2=0x7ffb3015d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:09.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.139+0000 7ffb36377640 1 --2- 172.21.15.83:0/4256311529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb300d26c0 0x7ffb3015d620 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffb28009d90 tx=0x7ffb2801c860 comp rx=0 tx=0).stop 2024-04-17T10:39:09.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.139+0000 7ffb36377640 1 -- 172.21.15.83:0/4256311529 shutdown_connections 2024-04-17T10:39:09.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.139+0000 7ffb36377640 1 --2- 172.21.15.83:0/4256311529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb300d26c0 0x7ffb3015d620 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:09.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.139+0000 7ffb36377640 1 --2- 172.21.15.83:0/4256311529 >> v2:172.21.15.167:3300/0 conn(0x7ffb300d0970 0x7ffb300d17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:09.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.139+0000 7ffb36377640 1 -- 172.21.15.83:0/4256311529 >> 172.21.15.83:0/4256311529 conn(0x7ffb3009cd50 msgr2=0x7ffb300d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:09.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.139+0000 7ffb36377640 1 -- 172.21.15.83:0/4256311529 shutdown_connections 2024-04-17T10:39:09.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.139+0000 7ffb36377640 1 -- 172.21.15.83:0/4256311529 wait complete. 2024-04-17T10:39:09.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.139+0000 7ffb36377640 1 Processor -- start 2024-04-17T10:39:09.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.139+0000 7ffb36377640 1 -- start start 2024-04-17T10:39:09.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.143+0000 7ffb36377640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb300ce4e0 0x7ffb3014e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:09.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.143+0000 7ffb36377640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb300d0970 0x7ffb300caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:09.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.143+0000 7ffb36377640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb300d26c0 0x7ffb30156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:09.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.143+0000 7ffb36377640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3006b280 con 0x7ffb300d26c0 2024-04-17T10:39:09.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.143+0000 7ffb36377640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb300d0d50 con 0x7ffb300d0970 2024-04-17T10:39:09.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.143+0000 7ffb36377640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3006b400 con 0x7ffb300ce4e0 2024-04-17T10:39:09.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.143+0000 7ffb34914640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb300d26c0 0x7ffb30156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:09.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.143+0000 7ffb34914640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb300d26c0 0x7ffb30156550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36972/0 (socket says 172.21.15.83:36972) 2024-04-17T10:39:09.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.143+0000 7ffb34914640 1 -- 172.21.15.83:0/354797906 learned_addr learned my addr 172.21.15.83:0/354797906 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:09.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.143+0000 7ffb34914640 1 -- 172.21.15.83:0/354797906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb300ce4e0 msgr2=0x7ffb3014e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:09.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.143+0000 7ffb34914640 1 --2- 172.21.15.83:0/354797906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb300ce4e0 0x7ffb3014e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:09.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.143+0000 7ffb34914640 1 -- 172.21.15.83:0/354797906 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb300d0970 msgr2=0x7ffb300caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:09.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.143+0000 7ffb34914640 1 --2- 172.21.15.83:0/354797906 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb300d0970 0x7ffb300caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:09.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.143+0000 7ffb34914640 1 -- 172.21.15.83:0/354797906 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb301554b0 con 0x7ffb300d26c0 2024-04-17T10:39:09.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.143+0000 7ffb34914640 1 --2- 172.21.15.83:0/354797906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb300d26c0 0x7ffb30156550 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ffb2801af10 tx=0x7ffb28019600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:09.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.143+0000 7ffb2cff9640 1 -- 172.21.15.83:0/354797906 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb280b2040 con 0x7ffb300d26c0 2024-04-17T10:39:09.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.143+0000 7ffb2cff9640 1 -- 172.21.15.83:0/354797906 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb280bd070 con 0x7ffb300d26c0 2024-04-17T10:39:09.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.143+0000 7ffb36377640 1 -- 172.21.15.83:0/354797906 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb30155790 con 0x7ffb300d26c0 2024-04-17T10:39:09.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.143+0000 7ffb2cff9640 1 -- 172.21.15.83:0/354797906 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb280a5c50 con 0x7ffb300d26c0 2024-04-17T10:39:09.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.143+0000 7ffb36377640 1 -- 172.21.15.83:0/354797906 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb30155d50 con 0x7ffb300d26c0 2024-04-17T10:39:09.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.143+0000 7ffb36377640 1 -- 172.21.15.83:0/354797906 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb300681a0 con 0x7ffb300d26c0 2024-04-17T10:39:09.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.143+0000 7ffb2cff9640 1 -- 172.21.15.83:0/354797906 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffb28004040 con 0x7ffb300d26c0 2024-04-17T10:39:09.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.143+0000 7ffb2cff9640 1 --2- 172.21.15.83:0/354797906 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffb10066f60 0x7ffb10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:09.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.143+0000 7ffb2cff9640 1 -- 172.21.15.83:0/354797906 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ffb2811f3c0 con 0x7ffb300d26c0 2024-04-17T10:39:09.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.143+0000 7ffb2ffff640 1 --2- 172.21.15.83:0/354797906 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffb10066f60 0x7ffb10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:09.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.143+0000 7ffb2ffff640 1 --2- 172.21.15.83:0/354797906 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffb10066f60 0x7ffb10069420 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffb20006e90 tx=0x7ffb20006de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:09.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.147+0000 7ffb2cff9640 1 -- 172.21.15.83:0/354797906 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffb28016480 con 0x7ffb300d26c0 2024-04-17T10:39:09.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.359+0000 7ffb36377640 1 -- 172.21.15.83:0/354797906 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7ffb30074f90 con 0x7ffb300d26c0 2024-04-17T10:39:09.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.363+0000 7ffb2cff9640 1 -- 172.21.15.83:0/354797906 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7ffb280f0720 con 0x7ffb300d26c0 2024-04-17T10:39:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.367+0000 7ffb36377640 1 -- 172.21.15.83:0/354797906 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffb10066f60 msgr2=0x7ffb10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.367+0000 7ffb36377640 1 --2- 172.21.15.83:0/354797906 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffb10066f60 0x7ffb10069420 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffb20006e90 tx=0x7ffb20006de0 comp rx=0 tx=0).stop 2024-04-17T10:39:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.367+0000 7ffb36377640 1 -- 172.21.15.83:0/354797906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb300d26c0 msgr2=0x7ffb30156550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.367+0000 7ffb36377640 1 --2- 172.21.15.83:0/354797906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb300d26c0 0x7ffb30156550 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ffb2801af10 tx=0x7ffb28019600 comp rx=0 tx=0).stop 2024-04-17T10:39:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.367+0000 7ffb36377640 1 -- 172.21.15.83:0/354797906 shutdown_connections 2024-04-17T10:39:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.367+0000 7ffb36377640 1 --2- 172.21.15.83:0/354797906 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffb10066f60 0x7ffb10069420 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.367+0000 7ffb36377640 1 --2- 172.21.15.83:0/354797906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb300d26c0 0x7ffb30156550 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.367+0000 7ffb36377640 1 --2- 172.21.15.83:0/354797906 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb300d0970 0x7ffb300caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.367+0000 7ffb36377640 1 --2- 172.21.15.83:0/354797906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb300ce4e0 0x7ffb3014e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.367+0000 7ffb36377640 1 -- 172.21.15.83:0/354797906 >> 172.21.15.83:0/354797906 conn(0x7ffb3009cd50 msgr2=0x7ffb300b47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.367+0000 7ffb36377640 1 -- 172.21.15.83:0/354797906 shutdown_connections 2024-04-17T10:39:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.367+0000 7ffb36377640 1 -- 172.21.15.83:0/354797906 wait complete. 2024-04-17T10:39:09.380 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-17T10:39:09.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.491+0000 7f97b9b30640 1 Processor -- start 2024-04-17T10:39:09.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.491+0000 7f97b9b30640 1 -- start start 2024-04-17T10:39:09.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.491+0000 7f97b9b30640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f97b40b4210 0x7f97b415e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:09.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.491+0000 7f97b9b30640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b415bd70 con 0x7f97b406b520 2024-04-17T10:39:09.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.491+0000 7f97b9b30640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b415bf60 con 0x7f97b4154de0 2024-04-17T10:39:09.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.491+0000 7f97b9b30640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b40b4870 con 0x7f97b40b4210 2024-04-17T10:39:09.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.491+0000 7f97abfff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f97b406b520 0x7f97b40b3b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48620/0 (socket says 172.21.15.83:48620) 2024-04-17T10:39:09.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.491+0000 7f97abfff640 1 -- 172.21.15.83:0/3836263861 learned_addr learned my addr 172.21.15.83:0/3836263861 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:39:09.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.491+0000 7f97b3fff640 1 --2- 172.21.15.83:0/3836263861 >> v2:172.21.15.167:3300/0 conn(0x7f97b40b4210 0x7f97b415e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:09.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.491+0000 7f97b2ffd640 1 -- 172.21.15.83:0/3836263861 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2908278292 0 0) 0x7f97b415bd70 con 0x7f97b406b520 2024-04-17T10:39:09.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.491+0000 7f97b2ffd640 1 -- 172.21.15.83:0/3836263861 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97a0003530 con 0x7f97b406b520 2024-04-17T10:39:09.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b2ffd640 1 -- 172.21.15.83:0/3836263861 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2187429753 0 0) 0x7f97a0003530 con 0x7f97b406b520 2024-04-17T10:39:09.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b2ffd640 1 -- 172.21.15.83:0/3836263861 >> v2:172.21.15.167:3300/0 conn(0x7f97b40b4210 msgr2=0x7f97b415e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:09.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b2ffd640 1 --2- 172.21.15.83:0/3836263861 >> v2:172.21.15.167:3300/0 conn(0x7f97b40b4210 0x7f97b415e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:09.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b2ffd640 1 -- 172.21.15.83:0/3836263861 >> v1:172.21.15.167:6789/0 conn(0x7f97b4154de0 legacy=0x7f97b406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:09.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b2ffd640 1 -- 172.21.15.83:0/3836263861 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b415afb0 con 0x7f97b406b520 2024-04-17T10:39:09.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b2ffd640 1 -- 172.21.15.83:0/3836263861 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f979c001680 con 0x7f97b406b520 2024-04-17T10:39:09.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b2ffd640 1 -- 172.21.15.83:0/3836263861 >> v1:172.21.15.83:6789/0 conn(0x7f97b406b520 legacy=0x7f97b40b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:09.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b2ffd640 1 --2- 172.21.15.83:0/3836263861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97a0003a30 0x7f97a0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:09.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b2ffd640 1 --2- 172.21.15.83:0/3836263861 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97a0004810 0x7f97a0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:09.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b2ffd640 1 --2- 172.21.15.83:0/3836263861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97a00051d0 0x7f97a000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:09.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b2ffd640 1 -- 172.21.15.83:0/3836263861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a000c0d0 con 0x7f97a00051d0 2024-04-17T10:39:09.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b2ffd640 1 -- 172.21.15.83:0/3836263861 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a000c280 con 0x7f97a0003a30 2024-04-17T10:39:09.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b2ffd640 1 -- 172.21.15.83:0/3836263861 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a000c480 con 0x7f97a0004810 2024-04-17T10:39:09.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b3fff640 1 --2- 172.21.15.83:0/3836263861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97a00051d0 0x7f97a000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:09.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97abfff640 1 --2- 172.21.15.83:0/3836263861 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97a0004810 0x7f97a0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:09.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b37fe640 1 --2- 172.21.15.83:0/3836263861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97a0003a30 0x7f97a0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:09.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b3fff640 1 -- 172.21.15.83:0/3836263861 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97a0004810 msgr2=0x7f97a0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:09.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b3fff640 1 --2- 172.21.15.83:0/3836263861 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97a0004810 0x7f97a0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:09.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b3fff640 1 -- 172.21.15.83:0/3836263861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97a0003a30 msgr2=0x7f97a0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:09.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b3fff640 1 --2- 172.21.15.83:0/3836263861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97a0003a30 0x7f97a0003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:09.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b3fff640 1 -- 172.21.15.83:0/3836263861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a0003530 con 0x7f97a00051d0 2024-04-17T10:39:09.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b37fe640 1 --2- 172.21.15.83:0/3836263861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97a0003a30 0x7f97a0003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:09.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97abfff640 1 --2- 172.21.15.83:0/3836263861 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97a0004810 0x7f97a0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:09.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b3fff640 1 -- 172.21.15.83:0/3836263861 reap_dead start 2024-04-17T10:39:09.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b3fff640 1 --2- 172.21.15.83:0/3836263861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97a00051d0 0x7f97a000b9d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f97ac009fc0 tx=0x7f97ac01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92d5082374fdef4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:09.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b2ffd640 1 -- 172.21.15.83:0/3836263861 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ac009c10 con 0x7f97a00051d0 2024-04-17T10:39:09.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b2ffd640 1 -- 172.21.15.83:0/3836263861 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ac0a5d10 con 0x7f97a00051d0 2024-04-17T10:39:09.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b2ffd640 1 -- 172.21.15.83:0/3836263861 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ac003be0 con 0x7f97a00051d0 2024-04-17T10:39:09.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b9b30640 1 -- 172.21.15.83:0/3836263861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97a00051d0 msgr2=0x7f97a000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:09.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b9b30640 1 --2- 172.21.15.83:0/3836263861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97a00051d0 0x7f97a000b9d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f97ac009fc0 tx=0x7f97ac01ccc0 comp rx=0 tx=0).stop 2024-04-17T10:39:09.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b9b30640 1 -- 172.21.15.83:0/3836263861 shutdown_connections 2024-04-17T10:39:09.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b9b30640 1 --2- 172.21.15.83:0/3836263861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97a00051d0 0x7f97a000b9d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:09.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b9b30640 1 -- 172.21.15.83:0/3836263861 >> 172.21.15.83:0/3836263861 conn(0x7f97b40b15b0 msgr2=0x7f97b40b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:09.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b9b30640 1 -- 172.21.15.83:0/3836263861 shutdown_connections 2024-04-17T10:39:09.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.495+0000 7f97b9b30640 1 -- 172.21.15.83:0/3836263861 wait complete. 2024-04-17T10:39:09.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.499+0000 7f97b9b30640 1 Processor -- start 2024-04-17T10:39:09.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.499+0000 7f97b9b30640 1 -- start start 2024-04-17T10:39:09.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.499+0000 7f97b9b30640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97a00051d0 0x7f97b4142290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:09.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.499+0000 7f97b9b30640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97b41ef160 0x7f97b41ef5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:09.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.499+0000 7f97b9b30640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97b40a9430 0x7f97b40a98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:09.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.499+0000 7f97b9b30640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b415b570 con 0x7f97a00051d0 2024-04-17T10:39:09.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.499+0000 7f97b9b30640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b40b4670 con 0x7f97b41ef160 2024-04-17T10:39:09.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.499+0000 7f97b9b30640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b415b850 con 0x7f97b40a9430 2024-04-17T10:39:09.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.499+0000 7f97b37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97a00051d0 0x7f97b4142290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:09.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.499+0000 7f97b37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97a00051d0 0x7f97b4142290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36984/0 (socket says 172.21.15.83:36984) 2024-04-17T10:39:09.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.499+0000 7f97b37fe640 1 -- 172.21.15.83:0/2442961620 learned_addr learned my addr 172.21.15.83:0/2442961620 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:09.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.499+0000 7f97b3fff640 1 --2- 172.21.15.83:0/2442961620 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97b40a9430 0x7f97b40a98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:09.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.499+0000 7f97abfff640 1 --2- 172.21.15.83:0/2442961620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97b41ef160 0x7f97b41ef5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:09.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.499+0000 7f97b37fe640 1 -- 172.21.15.83:0/2442961620 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97b40a9430 msgr2=0x7f97b40a98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:09.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.499+0000 7f97b37fe640 1 --2- 172.21.15.83:0/2442961620 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97b40a9430 0x7f97b40a98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:09.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.499+0000 7f97b37fe640 1 -- 172.21.15.83:0/2442961620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97b41ef160 msgr2=0x7f97b41ef5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:09.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.499+0000 7f97b37fe640 1 --2- 172.21.15.83:0/2442961620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97b41ef160 0x7f97b41ef5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:09.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.499+0000 7f97b37fe640 1 -- 172.21.15.83:0/2442961620 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b41efc20 con 0x7f97a00051d0 2024-04-17T10:39:09.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.499+0000 7f97b3fff640 1 --2- 172.21.15.83:0/2442961620 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97b40a9430 0x7f97b40a98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:09.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.499+0000 7f97abfff640 1 --2- 172.21.15.83:0/2442961620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97b41ef160 0x7f97b41ef5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:09.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.499+0000 7f97b37fe640 1 --2- 172.21.15.83:0/2442961620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97a00051d0 0x7f97b4142290 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f97a400d7d0 tx=0x7f97a400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:09.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.503+0000 7f97b0ff9640 1 -- 172.21.15.83:0/2442961620 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a4005270 con 0x7f97a00051d0 2024-04-17T10:39:09.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.503+0000 7f97b0ff9640 1 -- 172.21.15.83:0/2442961620 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a4017070 con 0x7f97a00051d0 2024-04-17T10:39:09.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.503+0000 7f97b0ff9640 1 -- 172.21.15.83:0/2442961620 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a400b650 con 0x7f97a00051d0 2024-04-17T10:39:09.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.503+0000 7f97b9b30640 1 -- 172.21.15.83:0/2442961620 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b41eff00 con 0x7f97a00051d0 2024-04-17T10:39:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.503+0000 7f97b9b30640 1 -- 172.21.15.83:0/2442961620 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b4158bb0 con 0x7f97a00051d0 2024-04-17T10:39:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.503+0000 7f97b9b30640 1 -- 172.21.15.83:0/2442961620 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b4074f90 con 0x7f97a00051d0 2024-04-17T10:39:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.503+0000 7f97b0ff9640 1 -- 172.21.15.83:0/2442961620 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f97a400b810 con 0x7f97a00051d0 2024-04-17T10:39:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.503+0000 7f97b0ff9640 1 --2- 172.21.15.83:0/2442961620 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9794066f90 0x7f9794069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.503+0000 7f97b0ff9640 1 -- 172.21.15.83:0/2442961620 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f97a40849b0 con 0x7f97a00051d0 2024-04-17T10:39:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.503+0000 7f97abfff640 1 --2- 172.21.15.83:0/2442961620 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9794066f90 0x7f9794069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.503+0000 7f97abfff640 1 --2- 172.21.15.83:0/2442961620 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9794066f90 0x7f9794069450 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f979c001930 tx=0x7f979c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:09.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.507+0000 7f97b0ff9640 1 -- 172.21.15.83:0/2442961620 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f97a4055d10 con 0x7f97a00051d0 2024-04-17T10:39:09.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.707+0000 7f97b9b30640 1 -- 172.21.15.83:0/2442961620 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f97b40b3280 con 0x7f97a00051d0 2024-04-17T10:39:09.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.711+0000 7f97b0ff9640 1 -- 172.21.15.83:0/2442961620 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7f97a4001240 con 0x7f97a00051d0 2024-04-17T10:39:09.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.715+0000 7f97b9b30640 1 -- 172.21.15.83:0/2442961620 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9794066f90 msgr2=0x7f9794069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:09.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.715+0000 7f97b9b30640 1 --2- 172.21.15.83:0/2442961620 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9794066f90 0x7f9794069450 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f979c001930 tx=0x7f979c009040 comp rx=0 tx=0).stop 2024-04-17T10:39:09.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.715+0000 7f97b9b30640 1 -- 172.21.15.83:0/2442961620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97a00051d0 msgr2=0x7f97b4142290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:09.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.715+0000 7f97b9b30640 1 --2- 172.21.15.83:0/2442961620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97a00051d0 0x7f97b4142290 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f97a400d7d0 tx=0x7f97a400dc90 comp rx=0 tx=0).stop 2024-04-17T10:39:09.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.715+0000 7f97b9b30640 1 -- 172.21.15.83:0/2442961620 shutdown_connections 2024-04-17T10:39:09.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.715+0000 7f97b9b30640 1 --2- 172.21.15.83:0/2442961620 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9794066f90 0x7f9794069450 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:09.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.715+0000 7f97b9b30640 1 --2- 172.21.15.83:0/2442961620 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97b40a9430 0x7f97b40a98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:09.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.715+0000 7f97b9b30640 1 --2- 172.21.15.83:0/2442961620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97b41ef160 0x7f97b41ef5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:09.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.715+0000 7f97b9b30640 1 --2- 172.21.15.83:0/2442961620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97a00051d0 0x7f97b4142290 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:09.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.715+0000 7f97b9b30640 1 -- 172.21.15.83:0/2442961620 >> 172.21.15.83:0/2442961620 conn(0x7f97b40b15b0 msgr2=0x7f97b4153e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:09.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.715+0000 7f97b9b30640 1 -- 172.21.15.83:0/2442961620 shutdown_connections 2024-04-17T10:39:09.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.715+0000 7f97b9b30640 1 -- 172.21.15.83:0/2442961620 wait complete. 2024-04-17T10:39:09.729 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-17T10:39:09.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.847+0000 7f0188127640 1 Processor -- start 2024-04-17T10:39:09.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.847+0000 7f0188127640 1 -- start start 2024-04-17T10:39:09.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.847+0000 7f0188127640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f018006cd10 0x7f0180168b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:09.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.847+0000 7f0188127640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01800cf940 con 0x7f018015e1d0 2024-04-17T10:39:09.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.847+0000 7f0188127640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f018006b220 con 0x7f018015ddf0 2024-04-17T10:39:09.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.847+0000 7f0188127640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01800cfc20 con 0x7f018006cd10 2024-04-17T10:39:09.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.847+0000 7f01856c2640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f018015e1d0 0x7f01800d0450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48622/0 (socket says 172.21.15.83:48622) 2024-04-17T10:39:09.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.847+0000 7f01856c2640 1 -- 172.21.15.83:0/2109263879 learned_addr learned my addr 172.21.15.83:0/2109263879 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:39:09.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.847+0000 7f0184ec1640 1 -- 172.21.15.83:0/2109263879 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 188194200 0 0) 0x7f01800cf940 con 0x7f018015e1d0 2024-04-17T10:39:09.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.851+0000 7f01866c4640 1 --2- 172.21.15.83:0/2109263879 >> v2:172.21.15.167:3300/0 conn(0x7f018006cd10 0x7f0180168b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:09.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.851+0000 7f0184ec1640 1 -- 172.21.15.83:0/2109263879 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f016c003530 con 0x7f018015e1d0 2024-04-17T10:39:09.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.851+0000 7f0184ec1640 1 -- 172.21.15.83:0/2109263879 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3027547818 0 0) 0x7f016c003530 con 0x7f018015e1d0 2024-04-17T10:39:09.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.851+0000 7f0184ec1640 1 -- 172.21.15.83:0/2109263879 >> v2:172.21.15.167:3300/0 conn(0x7f018006cd10 msgr2=0x7f0180168b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:09.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.851+0000 7f0184ec1640 1 --2- 172.21.15.83:0/2109263879 >> v2:172.21.15.167:3300/0 conn(0x7f018006cd10 0x7f0180168b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:09.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.851+0000 7f0184ec1640 1 -- 172.21.15.83:0/2109263879 >> v1:172.21.15.167:6789/0 conn(0x7f018015ddf0 legacy=0x7f01800cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:09.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.851+0000 7f0184ec1640 1 -- 172.21.15.83:0/2109263879 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f018006b500 con 0x7f018015e1d0 2024-04-17T10:39:09.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.851+0000 7f0184ec1640 1 -- 172.21.15.83:0/2109263879 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f0170001680 con 0x7f018015e1d0 2024-04-17T10:39:09.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.851+0000 7f0184ec1640 1 -- 172.21.15.83:0/2109263879 >> v1:172.21.15.83:6789/0 conn(0x7f018015e1d0 legacy=0x7f01800d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:09.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.851+0000 7f0184ec1640 1 --2- 172.21.15.83:0/2109263879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f016c003a30 0x7f016c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:09.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.851+0000 7f0184ec1640 1 --2- 172.21.15.83:0/2109263879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f016c004810 0x7f016c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:09.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.851+0000 7f0184ec1640 1 --2- 172.21.15.83:0/2109263879 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f016c0052d0 0x7f016c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:09.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.851+0000 7f0184ec1640 1 -- 172.21.15.83:0/2109263879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01800cfc20 con 0x7f016c004810 2024-04-17T10:39:09.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.851+0000 7f0184ec1640 1 -- 172.21.15.83:0/2109263879 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016c00c210 con 0x7f016c003a30 2024-04-17T10:39:09.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.851+0000 7f0184ec1640 1 -- 172.21.15.83:0/2109263879 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016c00c410 con 0x7f016c0052d0 2024-04-17T10:39:09.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.851+0000 7f01856c2640 1 --2- 172.21.15.83:0/2109263879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f016c004810 0x7f016c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:09.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.851+0000 7f0185ec3640 1 --2- 172.21.15.83:0/2109263879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f016c003a30 0x7f016c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:09.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.851+0000 7f01856c2640 1 -- 172.21.15.83:0/2109263879 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f016c0052d0 msgr2=0x7f016c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:09.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.851+0000 7f01856c2640 1 --2- 172.21.15.83:0/2109263879 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f016c0052d0 0x7f016c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:09.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.851+0000 7f01856c2640 1 -- 172.21.15.83:0/2109263879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f016c003a30 msgr2=0x7f016c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:09.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.851+0000 7f01856c2640 1 --2- 172.21.15.83:0/2109263879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f016c003a30 0x7f016c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:09.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.851+0000 7f01856c2640 1 -- 172.21.15.83:0/2109263879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016c003530 con 0x7f016c004810 2024-04-17T10:39:09.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.851+0000 7f01866c4640 1 -- 172.21.15.83:0/2109263879 reap_dead start 2024-04-17T10:39:09.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.851+0000 7f0185ec3640 1 --2- 172.21.15.83:0/2109263879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f016c003a30 0x7f016c003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:09.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.851+0000 7f01856c2640 1 --2- 172.21.15.83:0/2109263879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f016c004810 0x7f016c004c90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f017000c240 tx=0x7f017001cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3485f0f039037032 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:09.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.851+0000 7f0184ec1640 1 -- 172.21.15.83:0/2109263879 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0170007d20 con 0x7f016c004810 2024-04-17T10:39:09.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.851+0000 7f0184ec1640 1 -- 172.21.15.83:0/2109263879 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01700a9070 con 0x7f016c004810 2024-04-17T10:39:09.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.851+0000 7f0184ec1640 1 -- 172.21.15.83:0/2109263879 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0170005d80 con 0x7f016c004810 2024-04-17T10:39:09.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.855+0000 7f0188127640 1 -- 172.21.15.83:0/2109263879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f016c004810 msgr2=0x7f016c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:09.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.855+0000 7f0188127640 1 --2- 172.21.15.83:0/2109263879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f016c004810 0x7f016c004c90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f017000c240 tx=0x7f017001cc40 comp rx=0 tx=0).stop 2024-04-17T10:39:09.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.855+0000 7f0188127640 1 -- 172.21.15.83:0/2109263879 shutdown_connections 2024-04-17T10:39:09.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.855+0000 7f0188127640 1 --2- 172.21.15.83:0/2109263879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f016c004810 0x7f016c004c90 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:09.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.855+0000 7f0188127640 1 -- 172.21.15.83:0/2109263879 >> 172.21.15.83:0/2109263879 conn(0x7f01800d3a60 msgr2=0x7f01800d3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:09.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.855+0000 7f0188127640 1 -- 172.21.15.83:0/2109263879 shutdown_connections 2024-04-17T10:39:09.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.855+0000 7f0188127640 1 -- 172.21.15.83:0/2109263879 wait complete. 2024-04-17T10:39:09.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.855+0000 7f0188127640 1 Processor -- start 2024-04-17T10:39:09.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.855+0000 7f0188127640 1 -- start start 2024-04-17T10:39:09.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.855+0000 7f0188127640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f016c004810 0x7f0180160820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:09.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.855+0000 7f0188127640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0180148990 0x7f0180148e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:09.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.855+0000 7f0188127640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01801fa3c0 0x7f01801fa840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:09.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.855+0000 7f0188127640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01800d1a50 con 0x7f01801fa3c0 2024-04-17T10:39:09.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.855+0000 7f0188127640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01800d18d0 con 0x7f0180148990 2024-04-17T10:39:09.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.855+0000 7f0188127640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01800d1bd0 con 0x7f016c004810 2024-04-17T10:39:09.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.855+0000 7f01866c4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01801fa3c0 0x7f01801fa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:09.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.855+0000 7f01866c4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01801fa3c0 0x7f01801fa840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36994/0 (socket says 172.21.15.83:36994) 2024-04-17T10:39:09.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.855+0000 7f01866c4640 1 -- 172.21.15.83:0/707305205 learned_addr learned my addr 172.21.15.83:0/707305205 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:09.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.855+0000 7f01856c2640 1 --2- 172.21.15.83:0/707305205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0180148990 0x7f0180148e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:09.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.855+0000 7f0185ec3640 1 --2- 172.21.15.83:0/707305205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f016c004810 0x7f0180160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:09.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.855+0000 7f01866c4640 1 -- 172.21.15.83:0/707305205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f016c004810 msgr2=0x7f0180160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:09.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.855+0000 7f01866c4640 1 --2- 172.21.15.83:0/707305205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f016c004810 0x7f0180160820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:09.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.855+0000 7f01866c4640 1 -- 172.21.15.83:0/707305205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0180148990 msgr2=0x7f0180148e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:09.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.855+0000 7f01866c4640 1 --2- 172.21.15.83:0/707305205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0180148990 0x7f0180148e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:09.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.855+0000 7f01866c4640 1 -- 172.21.15.83:0/707305205 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01800ccae0 con 0x7f01801fa3c0 2024-04-17T10:39:09.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.855+0000 7f01856c2640 1 --2- 172.21.15.83:0/707305205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0180148990 0x7f0180148e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:39:09.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.855+0000 7f01866c4640 1 --2- 172.21.15.83:0/707305205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01801fa3c0 0x7f01801fa840 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f01800ce060 tx=0x7f017800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:09.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.855+0000 7f01767fc640 1 -- 172.21.15.83:0/707305205 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0178018020 con 0x7f01801fa3c0 2024-04-17T10:39:09.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.855+0000 7f01767fc640 1 -- 172.21.15.83:0/707305205 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f017800bd10 con 0x7f01801fa3c0 2024-04-17T10:39:09.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.855+0000 7f0188127640 1 -- 172.21.15.83:0/707305205 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0180149350 con 0x7f01801fa3c0 2024-04-17T10:39:09.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.855+0000 7f01767fc640 1 -- 172.21.15.83:0/707305205 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0178014640 con 0x7f01801fa3c0 2024-04-17T10:39:09.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.855+0000 7f0188127640 1 -- 172.21.15.83:0/707305205 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01800cc000 con 0x7f01801fa3c0 2024-04-17T10:39:09.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.859+0000 7f0188127640 1 -- 172.21.15.83:0/707305205 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0180074f90 con 0x7f01801fa3c0 2024-04-17T10:39:09.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.859+0000 7f01767fc640 1 -- 172.21.15.83:0/707305205 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f017801b020 con 0x7f01801fa3c0 2024-04-17T10:39:09.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.859+0000 7f01767fc640 1 --2- 172.21.15.83:0/707305205 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0164066f90 0x7f0164069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:09.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.859+0000 7f01767fc640 1 -- 172.21.15.83:0/707305205 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0178085370 con 0x7f01801fa3c0 2024-04-17T10:39:09.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.859+0000 7f0185ec3640 1 --2- 172.21.15.83:0/707305205 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0164066f90 0x7f0164069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:09.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.859+0000 7f0185ec3640 1 --2- 172.21.15.83:0/707305205 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0164066f90 0x7f0164069450 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f017c000c10 tx=0x7f017c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:09.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:09.863+0000 7f01767fc640 1 -- 172.21.15.83:0/707305205 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0178022030 con 0x7f01801fa3c0 2024-04-17T10:39:10.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.075+0000 7f0188127640 1 -- 172.21.15.83:0/707305205 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f018006c820 con 0x7f01801fa3c0 2024-04-17T10:39:10.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.079+0000 7f01767fc640 1 -- 172.21.15.83:0/707305205 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f01780566c0 con 0x7f01801fa3c0 2024-04-17T10:39:10.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.083+0000 7f0188127640 1 -- 172.21.15.83:0/707305205 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0164066f90 msgr2=0x7f0164069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:10.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.083+0000 7f0188127640 1 --2- 172.21.15.83:0/707305205 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0164066f90 0x7f0164069450 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f017c000c10 tx=0x7f017c009040 comp rx=0 tx=0).stop 2024-04-17T10:39:10.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.083+0000 7f0188127640 1 -- 172.21.15.83:0/707305205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01801fa3c0 msgr2=0x7f01801fa840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:10.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.083+0000 7f0188127640 1 --2- 172.21.15.83:0/707305205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01801fa3c0 0x7f01801fa840 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f01800ce060 tx=0x7f017800b480 comp rx=0 tx=0).stop 2024-04-17T10:39:10.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.083+0000 7f0188127640 1 -- 172.21.15.83:0/707305205 shutdown_connections 2024-04-17T10:39:10.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.083+0000 7f0188127640 1 --2- 172.21.15.83:0/707305205 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0164066f90 0x7f0164069450 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.083+0000 7f0188127640 1 --2- 172.21.15.83:0/707305205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01801fa3c0 0x7f01801fa840 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.083+0000 7f0188127640 1 --2- 172.21.15.83:0/707305205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0180148990 0x7f0180148e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.083+0000 7f0188127640 1 --2- 172.21.15.83:0/707305205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f016c004810 0x7f0180160820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.083+0000 7f0188127640 1 -- 172.21.15.83:0/707305205 >> 172.21.15.83:0/707305205 conn(0x7f01800d3a60 msgr2=0x7f01800c7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:10.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.083+0000 7f0188127640 1 -- 172.21.15.83:0/707305205 shutdown_connections 2024-04-17T10:39:10.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.083+0000 7f0188127640 1 -- 172.21.15.83:0/707305205 wait complete. 2024-04-17T10:39:10.096 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:39:10.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.211+0000 7f8c5863e640 1 Processor -- start 2024-04-17T10:39:10.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.211+0000 7f8c5863e640 1 -- start start 2024-04-17T10:39:10.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.211+0000 7f8c5863e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c5015b7f0 0x7f8c5006b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:10.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.211+0000 7f8c5863e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8c500ae030 0x7f8c5015ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:10.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.211+0000 7f8c5863e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c5015ef40 con 0x7f8c500acec0 2024-04-17T10:39:10.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.211+0000 7f8c5863e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c5006abe0 con 0x7f8c500ae030 2024-04-17T10:39:10.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.211+0000 7f8c5863e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5006bb70 con 0x7f8c5015b7f0 2024-04-17T10:39:10.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.215+0000 7f8c55bd9640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8c500ae030 0x7f8c5015ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:10.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.215+0000 7f8c56bdb640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8c500acec0 0x7f8c500b3ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48636/0 (socket says 172.21.15.83:48636) 2024-04-17T10:39:10.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.215+0000 7f8c56bdb640 1 -- 172.21.15.83:0/937489990 learned_addr learned my addr 172.21.15.83:0/937489990 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:39:10.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.215+0000 7f8c563da640 1 --2- 172.21.15.83:0/937489990 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c5015b7f0 0x7f8c5006b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:10.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.215+0000 7f8c55bd9640 1 -- 172.21.15.83:0/937489990 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c5015b7f0 msgr2=0x7f8c5006b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:10.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.215+0000 7f8c55bd9640 1 --2- 172.21.15.83:0/937489990 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c5015b7f0 0x7f8c5006b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.215+0000 7f8c55bd9640 1 -- 172.21.15.83:0/937489990 >> v1:172.21.15.83:6789/0 conn(0x7f8c500acec0 legacy=0x7f8c500b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:10.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.215+0000 7f8c55bd9640 1 -- 172.21.15.83:0/937489990 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c5006c110 con 0x7f8c500ae030 2024-04-17T10:39:10.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.215+0000 7f8c553d8640 1 -- 172.21.15.83:0/937489990 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2205084434 0 0) 0x7f8c5015ef40 con 0x7f8c500acec0 2024-04-17T10:39:10.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.215+0000 7f8c55bd9640 1 --2- 172.21.15.83:0/937489990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c500ae030 0x7f8c5015ea00 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8c400097c0 tx=0x7f8c4001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1005ebe3acee7d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:10.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.215+0000 7f8c553d8640 1 -- 172.21.15.83:0/937489990 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c400aa600 con 0x7f8c500ae030 2024-04-17T10:39:10.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.215+0000 7f8c553d8640 1 -- 172.21.15.83:0/937489990 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c400af070 con 0x7f8c500ae030 2024-04-17T10:39:10.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.215+0000 7f8c553d8640 1 -- 172.21.15.83:0/937489990 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c400aa9c0 con 0x7f8c500ae030 2024-04-17T10:39:10.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.215+0000 7f8c5863e640 1 -- 172.21.15.83:0/937489990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c500ae030 msgr2=0x7f8c5015ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:10.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.215+0000 7f8c5863e640 1 --2- 172.21.15.83:0/937489990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c500ae030 0x7f8c5015ea00 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8c400097c0 tx=0x7f8c4001c4a0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.215+0000 7f8c5863e640 1 -- 172.21.15.83:0/937489990 shutdown_connections 2024-04-17T10:39:10.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.215+0000 7f8c5863e640 1 --2- 172.21.15.83:0/937489990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c500ae030 0x7f8c5015ea00 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.215+0000 7f8c5863e640 1 --2- 172.21.15.83:0/937489990 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c5015b7f0 0x7f8c5006b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.215+0000 7f8c5863e640 1 -- 172.21.15.83:0/937489990 >> 172.21.15.83:0/937489990 conn(0x7f8c500b1be0 msgr2=0x7f8c500b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:10.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.215+0000 7f8c5863e640 1 -- 172.21.15.83:0/937489990 shutdown_connections 2024-04-17T10:39:10.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.215+0000 7f8c5863e640 1 -- 172.21.15.83:0/937489990 wait complete. 2024-04-17T10:39:10.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.219+0000 7f8c5863e640 1 Processor -- start 2024-04-17T10:39:10.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.219+0000 7f8c5863e640 1 -- start start 2024-04-17T10:39:10.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.219+0000 7f8c5863e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c500acec0 0x7f8c50152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:10.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.219+0000 7f8c5863e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c500ae030 0x7f8c501601a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:10.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.219+0000 7f8c5863e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c5015b7f0 0x7f8c501606e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:10.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.219+0000 7f8c5863e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c500b4a10 con 0x7f8c5015b7f0 2024-04-17T10:39:10.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.219+0000 7f8c5863e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c500ad740 con 0x7f8c500ae030 2024-04-17T10:39:10.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.219+0000 7f8c5863e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c500b4b90 con 0x7f8c500acec0 2024-04-17T10:39:10.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.219+0000 7f8c56bdb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c5015b7f0 0x7f8c501606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:10.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.219+0000 7f8c56bdb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c5015b7f0 0x7f8c501606e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36998/0 (socket says 172.21.15.83:36998) 2024-04-17T10:39:10.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.219+0000 7f8c56bdb640 1 -- 172.21.15.83:0/3168741918 learned_addr learned my addr 172.21.15.83:0/3168741918 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:10.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.219+0000 7f8c56bdb640 1 -- 172.21.15.83:0/3168741918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c500acec0 msgr2=0x7f8c50152f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:10.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.219+0000 7f8c56bdb640 1 --2- 172.21.15.83:0/3168741918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c500acec0 0x7f8c50152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.219+0000 7f8c56bdb640 1 -- 172.21.15.83:0/3168741918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c500ae030 msgr2=0x7f8c501601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:10.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.219+0000 7f8c56bdb640 1 --2- 172.21.15.83:0/3168741918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c500ae030 0x7f8c501601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.219+0000 7f8c56bdb640 1 -- 172.21.15.83:0/3168741918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c501554a0 con 0x7f8c5015b7f0 2024-04-17T10:39:10.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.219+0000 7f8c56bdb640 1 --2- 172.21.15.83:0/3168741918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c5015b7f0 0x7f8c501606e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8c48001570 tx=0x7f8c4800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:10.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.219+0000 7f8c46ffd640 1 -- 172.21.15.83:0/3168741918 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c48018070 con 0x7f8c5015b7f0 2024-04-17T10:39:10.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.219+0000 7f8c46ffd640 1 -- 172.21.15.83:0/3168741918 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4800a640 con 0x7f8c5015b7f0 2024-04-17T10:39:10.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.219+0000 7f8c5863e640 1 -- 172.21.15.83:0/3168741918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c501534b0 con 0x7f8c5015b7f0 2024-04-17T10:39:10.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.219+0000 7f8c46ffd640 1 -- 172.21.15.83:0/3168741918 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4800aa30 con 0x7f8c5015b7f0 2024-04-17T10:39:10.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.219+0000 7f8c5863e640 1 -- 172.21.15.83:0/3168741918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c50155a20 con 0x7f8c5015b7f0 2024-04-17T10:39:10.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.219+0000 7f8c5863e640 1 -- 172.21.15.83:0/3168741918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c50074f90 con 0x7f8c5015b7f0 2024-04-17T10:39:10.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.219+0000 7f8c46ffd640 1 -- 172.21.15.83:0/3168741918 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8c4801b070 con 0x7f8c5015b7f0 2024-04-17T10:39:10.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.219+0000 7f8c46ffd640 1 --2- 172.21.15.83:0/3168741918 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8c34066f90 0x7f8c34069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:10.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.219+0000 7f8c563da640 1 --2- 172.21.15.83:0/3168741918 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8c34066f90 0x7f8c34069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:10.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.219+0000 7f8c46ffd640 1 -- 172.21.15.83:0/3168741918 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8c48084270 con 0x7f8c5015b7f0 2024-04-17T10:39:10.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.223+0000 7f8c563da640 1 --2- 172.21.15.83:0/3168741918 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8c34066f90 0x7f8c34069450 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c00b850 tx=0x7f8c4c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:10.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.227+0000 7f8c46ffd640 1 -- 172.21.15.83:0/3168741918 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8c48055660 con 0x7f8c5015b7f0 2024-04-17T10:39:10.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.371+0000 7f8c5863e640 1 -- 172.21.15.83:0/3168741918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c5006bd30 con 0x7f8c5015b7f0 2024-04-17T10:39:10.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.375+0000 7f8c46ffd640 1 -- 172.21.15.83:0/3168741918 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+12819 (secure 0 0 0) 0x7f8c48054f60 con 0x7f8c5015b7f0 2024-04-17T10:39:10.372 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:39:10.372 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":29,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:39:07.516701+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:39:10.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.375+0000 7f8c5863e640 1 -- 172.21.15.83:0/3168741918 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8c34066f90 msgr2=0x7f8c34069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:10.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.375+0000 7f8c5863e640 1 --2- 172.21.15.83:0/3168741918 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8c34066f90 0x7f8c34069450 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c00b850 tx=0x7f8c4c008040 comp rx=0 tx=0).stop 2024-04-17T10:39:10.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.375+0000 7f8c5863e640 1 -- 172.21.15.83:0/3168741918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c5015b7f0 msgr2=0x7f8c501606e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:10.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.375+0000 7f8c5863e640 1 --2- 172.21.15.83:0/3168741918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c5015b7f0 0x7f8c501606e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8c48001570 tx=0x7f8c4800cca0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.375+0000 7f8c5863e640 1 -- 172.21.15.83:0/3168741918 shutdown_connections 2024-04-17T10:39:10.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.375+0000 7f8c5863e640 1 --2- 172.21.15.83:0/3168741918 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8c34066f90 0x7f8c34069450 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.375+0000 7f8c5863e640 1 --2- 172.21.15.83:0/3168741918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c5015b7f0 0x7f8c501606e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.375+0000 7f8c5863e640 1 --2- 172.21.15.83:0/3168741918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c500ae030 0x7f8c501601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.375+0000 7f8c5863e640 1 --2- 172.21.15.83:0/3168741918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c500acec0 0x7f8c50152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.375+0000 7f8c5863e640 1 -- 172.21.15.83:0/3168741918 >> 172.21.15.83:0/3168741918 conn(0x7f8c500b1be0 msgr2=0x7f8c500b3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:10.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.375+0000 7f8c5863e640 1 -- 172.21.15.83:0/3168741918 shutdown_connections 2024-04-17T10:39:10.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.375+0000 7f8c5863e640 1 -- 172.21.15.83:0/3168741918 wait complete. 2024-04-17T10:39:10.389 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:39:10.390 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:39:10.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.519+0000 7f433c68d640 1 Processor -- start 2024-04-17T10:39:10.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.519+0000 7f433c68d640 1 -- start start 2024-04-17T10:39:10.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.519+0000 7f433c68d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43340b5b60 0x7f43340b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:10.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.519+0000 7f433c68d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f433415e4d0 con 0x7f43340ce400 2024-04-17T10:39:10.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.519+0000 7f433c68d640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4334162580 con 0x7f433415e0f0 2024-04-17T10:39:10.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.519+0000 7f433c68d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4334162b40 con 0x7f43340b5b60 2024-04-17T10:39:10.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.519+0000 7f4339c28640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f43340ce400 0x7f43340b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48652/0 (socket says 172.21.15.83:48652) 2024-04-17T10:39:10.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.519+0000 7f4339c28640 1 -- 172.21.15.83:0/3977498538 learned_addr learned my addr 172.21.15.83:0/3977498538 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:39:10.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.519+0000 7f4339427640 1 -- 172.21.15.83:0/3977498538 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4141132682 0 0) 0x7f433415e4d0 con 0x7f43340ce400 2024-04-17T10:39:10.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.519+0000 7f4339427640 1 -- 172.21.15.83:0/3977498538 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4320003530 con 0x7f43340ce400 2024-04-17T10:39:10.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.519+0000 7f433ac2a640 1 --2- 172.21.15.83:0/3977498538 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43340b5b60 0x7f43340b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:10.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.519+0000 7f4339427640 1 -- 172.21.15.83:0/3977498538 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2560061393 0 0) 0x7f4320003530 con 0x7f43340ce400 2024-04-17T10:39:10.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.519+0000 7f4339427640 1 -- 172.21.15.83:0/3977498538 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43340b5b60 msgr2=0x7f43340b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:10.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.519+0000 7f4339427640 1 --2- 172.21.15.83:0/3977498538 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43340b5b60 0x7f43340b6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.519+0000 7f4339427640 1 -- 172.21.15.83:0/3977498538 >> v1:172.21.15.167:6789/0 conn(0x7f433415e0f0 legacy=0x7f4334161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:10.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.519+0000 7f4339427640 1 -- 172.21.15.83:0/3977498538 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4334162860 con 0x7f43340ce400 2024-04-17T10:39:10.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.519+0000 7f4339427640 1 -- 172.21.15.83:0/3977498538 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f4324001680 con 0x7f43340ce400 2024-04-17T10:39:10.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.519+0000 7f4339427640 1 -- 172.21.15.83:0/3977498538 >> v1:172.21.15.83:6789/0 conn(0x7f43340ce400 legacy=0x7f43340b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:10.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.519+0000 7f4339427640 1 --2- 172.21.15.83:0/3977498538 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43340b5b60 0x7f4320003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:10.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.519+0000 7f4339427640 1 --2- 172.21.15.83:0/3977498538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43200040e0 0x7f4320004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:10.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.519+0000 7f4339427640 1 --2- 172.21.15.83:0/3977498538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4320004dc0 0x7f4320005240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:10.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.519+0000 7f4339427640 1 -- 172.21.15.83:0/3977498538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4334162b40 con 0x7f43200040e0 2024-04-17T10:39:10.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.519+0000 7f4339427640 1 -- 172.21.15.83:0/3977498538 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432000bcd0 con 0x7f4320004dc0 2024-04-17T10:39:10.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.519+0000 7f4339427640 1 -- 172.21.15.83:0/3977498538 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f432000bed0 con 0x7f43340b5b60 2024-04-17T10:39:10.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.519+0000 7f4339c28640 1 --2- 172.21.15.83:0/3977498538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43200040e0 0x7f4320004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:10.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.519+0000 7f433ac2a640 1 --2- 172.21.15.83:0/3977498538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4320004dc0 0x7f4320005240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:10.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.519+0000 7f433a429640 1 --2- 172.21.15.83:0/3977498538 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43340b5b60 0x7f4320003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:10.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.519+0000 7f4339c28640 1 -- 172.21.15.83:0/3977498538 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43340b5b60 msgr2=0x7f4320003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:10.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.519+0000 7f4339c28640 1 --2- 172.21.15.83:0/3977498538 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43340b5b60 0x7f4320003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.519+0000 7f4339c28640 1 -- 172.21.15.83:0/3977498538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4320004dc0 msgr2=0x7f4320005240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:10.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.519+0000 7f4339c28640 1 --2- 172.21.15.83:0/3977498538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4320004dc0 0x7f4320005240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.519+0000 7f4339c28640 1 -- 172.21.15.83:0/3977498538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4320003530 con 0x7f43200040e0 2024-04-17T10:39:10.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.523+0000 7f4339c28640 1 --2- 172.21.15.83:0/3977498538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43200040e0 0x7f4320004810 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f432400c160 tx=0x7f432401cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c8cc4704950e12 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:10.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.523+0000 7f4339427640 1 -- 172.21.15.83:0/3977498538 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4324007c70 con 0x7f43200040e0 2024-04-17T10:39:10.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.523+0000 7f4339427640 1 -- 172.21.15.83:0/3977498538 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43240a9070 con 0x7f43200040e0 2024-04-17T10:39:10.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.523+0000 7f4339427640 1 -- 172.21.15.83:0/3977498538 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4324005dc0 con 0x7f43200040e0 2024-04-17T10:39:10.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.523+0000 7f433c68d640 1 -- 172.21.15.83:0/3977498538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43200040e0 msgr2=0x7f4320004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:10.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.523+0000 7f433c68d640 1 --2- 172.21.15.83:0/3977498538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43200040e0 0x7f4320004810 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f432400c160 tx=0x7f432401cba0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.523+0000 7f433ac2a640 1 -- 172.21.15.83:0/3977498538 reap_dead start 2024-04-17T10:39:10.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.523+0000 7f433c68d640 1 -- 172.21.15.83:0/3977498538 shutdown_connections 2024-04-17T10:39:10.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.523+0000 7f433c68d640 1 -- 172.21.15.83:0/3977498538 >> 172.21.15.83:0/3977498538 conn(0x7f43340d0450 msgr2=0x7f43340d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:10.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.523+0000 7f433c68d640 1 -- 172.21.15.83:0/3977498538 shutdown_connections 2024-04-17T10:39:10.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.523+0000 7f433c68d640 1 -- 172.21.15.83:0/3977498538 wait complete. 2024-04-17T10:39:10.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.523+0000 7f433c68d640 1 Processor -- start 2024-04-17T10:39:10.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.523+0000 7f433c68d640 1 -- start start 2024-04-17T10:39:10.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.523+0000 7f433c68d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43200040e0 0x7f4334152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:10.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.523+0000 7f433c68d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43340cd430 0x7f43340cad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:10.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.523+0000 7f433c68d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4334150f00 0x7f4334151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:10.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.523+0000 7f433c68d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43340d2ed0 con 0x7f43340cd430 2024-04-17T10:39:10.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.523+0000 7f433c68d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43340d2d50 con 0x7f43200040e0 2024-04-17T10:39:10.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.523+0000 7f433c68d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43340d3050 con 0x7f4334150f00 2024-04-17T10:39:10.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.523+0000 7f4339c28640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43340cd430 0x7f43340cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:10.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.523+0000 7f4339c28640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43340cd430 0x7f43340cad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37028/0 (socket says 172.21.15.83:37028) 2024-04-17T10:39:10.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.523+0000 7f4339c28640 1 -- 172.21.15.83:0/672897529 learned_addr learned my addr 172.21.15.83:0/672897529 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:10.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.523+0000 7f433a429640 1 --2- 172.21.15.83:0/672897529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43200040e0 0x7f4334152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:10.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.527+0000 7f433ac2a640 1 --2- 172.21.15.83:0/672897529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4334150f00 0x7f4334151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:10.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.527+0000 7f4339c28640 1 -- 172.21.15.83:0/672897529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4334150f00 msgr2=0x7f4334151380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:10.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.527+0000 7f4339c28640 1 --2- 172.21.15.83:0/672897529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4334150f00 0x7f4334151380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.527+0000 7f4339c28640 1 -- 172.21.15.83:0/672897529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43200040e0 msgr2=0x7f4334152a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:10.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.527+0000 7f4339c28640 1 --2- 172.21.15.83:0/672897529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43200040e0 0x7f4334152a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.527+0000 7f4339c28640 1 -- 172.21.15.83:0/672897529 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43340b9870 con 0x7f43340cd430 2024-04-17T10:39:10.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.527+0000 7f433ac2a640 1 --2- 172.21.15.83:0/672897529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4334150f00 0x7f4334151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:10.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.527+0000 7f4339c28640 1 --2- 172.21.15.83:0/672897529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43340cd430 0x7f43340cad00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f432401c9d0 tx=0x7f4320005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:10.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.527+0000 7f432affd640 1 -- 172.21.15.83:0/672897529 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4324005e60 con 0x7f43340cd430 2024-04-17T10:39:10.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.527+0000 7f432affd640 1 -- 172.21.15.83:0/672897529 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43240047f0 con 0x7f43340cd430 2024-04-17T10:39:10.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.527+0000 7f433c68d640 1 -- 172.21.15.83:0/672897529 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43341530e0 con 0x7f43340cd430 2024-04-17T10:39:10.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.527+0000 7f432affd640 1 -- 172.21.15.83:0/672897529 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4324004ba0 con 0x7f43340cd430 2024-04-17T10:39:10.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.527+0000 7f433c68d640 1 -- 172.21.15.83:0/672897529 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43341536a0 con 0x7f43340cd430 2024-04-17T10:39:10.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.527+0000 7f433c68d640 1 -- 172.21.15.83:0/672897529 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43340681a0 con 0x7f43340cd430 2024-04-17T10:39:10.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.527+0000 7f432affd640 1 -- 172.21.15.83:0/672897529 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4324007070 con 0x7f43340cd430 2024-04-17T10:39:10.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.527+0000 7f432affd640 1 --2- 172.21.15.83:0/672897529 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4318066f60 0x7f4318069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:10.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.527+0000 7f433a429640 1 --2- 172.21.15.83:0/672897529 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4318066f60 0x7f4318069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:10.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.527+0000 7f432affd640 1 -- 172.21.15.83:0/672897529 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f432411e000 con 0x7f43340cd430 2024-04-17T10:39:10.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.527+0000 7f433a429640 1 --2- 172.21.15.83:0/672897529 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4318066f60 0x7f4318069420 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4330000c10 tx=0x7f4330009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:10.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.531+0000 7f432affd640 1 -- 172.21.15.83:0/672897529 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f43240168b0 con 0x7f43340cd430 2024-04-17T10:39:10.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.691+0000 7f433c68d640 1 -- 172.21.15.83:0/672897529 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4334074f90 con 0x7f43340cd430 2024-04-17T10:39:10.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.691+0000 7f432affd640 1 -- 172.21.15.83:0/672897529 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f43240ef360 con 0x7f43340cd430 2024-04-17T10:39:10.689 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:39:10.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.691+0000 7f433c68d640 1 -- 172.21.15.83:0/672897529 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4318066f60 msgr2=0x7f4318069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:10.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.691+0000 7f433c68d640 1 --2- 172.21.15.83:0/672897529 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4318066f60 0x7f4318069420 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4330000c10 tx=0x7f4330009040 comp rx=0 tx=0).stop 2024-04-17T10:39:10.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.691+0000 7f433c68d640 1 -- 172.21.15.83:0/672897529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43340cd430 msgr2=0x7f43340cad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:10.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.691+0000 7f433c68d640 1 --2- 172.21.15.83:0/672897529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43340cd430 0x7f43340cad00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f432401c9d0 tx=0x7f4320005840 comp rx=0 tx=0).stop 2024-04-17T10:39:10.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.691+0000 7f433c68d640 1 -- 172.21.15.83:0/672897529 shutdown_connections 2024-04-17T10:39:10.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.691+0000 7f433c68d640 1 --2- 172.21.15.83:0/672897529 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4318066f60 0x7f4318069420 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.691+0000 7f433c68d640 1 --2- 172.21.15.83:0/672897529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4334150f00 0x7f4334151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.691+0000 7f433c68d640 1 --2- 172.21.15.83:0/672897529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43340cd430 0x7f43340cad00 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.691+0000 7f433c68d640 1 --2- 172.21.15.83:0/672897529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43200040e0 0x7f4334152a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.691+0000 7f433c68d640 1 -- 172.21.15.83:0/672897529 >> 172.21.15.83:0/672897529 conn(0x7f43340d0450 msgr2=0x7f43340b7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:10.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.691+0000 7f433c68d640 1 -- 172.21.15.83:0/672897529 shutdown_connections 2024-04-17T10:39:10.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.691+0000 7f433c68d640 1 -- 172.21.15.83:0/672897529 wait complete. 2024-04-17T10:39:10.707 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-17T10:39:10.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.827+0000 7f405cc84640 1 Processor -- start 2024-04-17T10:39:10.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.831+0000 7f405cc84640 1 -- start start 2024-04-17T10:39:10.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.831+0000 7f405cc84640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f405815db50 0x7f4058070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:10.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.831+0000 7f405cc84640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40580709b0 con 0x7f405806dcb0 2024-04-17T10:39:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.831+0000 7f405cc84640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f405815e440 con 0x7f40580aedd0 2024-04-17T10:39:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.831+0000 7f405cc84640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f405815e720 con 0x7f405815db50 2024-04-17T10:39:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.831+0000 7f4056d9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f405806dcb0 0x7f40580b2e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48666/0 (socket says 172.21.15.83:48666) 2024-04-17T10:39:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.831+0000 7f4056d9d640 1 -- 172.21.15.83:0/2722819875 learned_addr learned my addr 172.21.15.83:0/2722819875 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:39:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.831+0000 7f405659c640 1 --2- 172.21.15.83:0/2722819875 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f405815db50 0x7f4058070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.831+0000 7f405559a640 1 -- 172.21.15.83:0/2722819875 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1067263012 0 0) 0x7f40500014a0 con 0x7f405806dcb0 2024-04-17T10:39:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.831+0000 7f405559a640 1 -- 172.21.15.83:0/2722819875 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4040003530 con 0x7f405806dcb0 2024-04-17T10:39:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.831+0000 7f405559a640 1 -- 172.21.15.83:0/2722819875 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 830028256 0 0) 0x7f4040003530 con 0x7f405806dcb0 2024-04-17T10:39:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.831+0000 7f405559a640 1 -- 172.21.15.83:0/2722819875 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f405815db50 msgr2=0x7f4058070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.831+0000 7f405559a640 1 --2- 172.21.15.83:0/2722819875 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f405815db50 0x7f4058070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.831+0000 7f405559a640 1 -- 172.21.15.83:0/2722819875 >> v1:172.21.15.167:6789/0 conn(0x7f40580aedd0 legacy=0x7f405806d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.831+0000 7f405559a640 1 -- 172.21.15.83:0/2722819875 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f405815ea00 con 0x7f405806dcb0 2024-04-17T10:39:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.831+0000 7f405559a640 1 -- 172.21.15.83:0/2722819875 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f4050001bd0 con 0x7f405806dcb0 2024-04-17T10:39:10.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.831+0000 7f405559a640 1 -- 172.21.15.83:0/2722819875 >> v1:172.21.15.83:6789/0 conn(0x7f405806dcb0 legacy=0x7f40580b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:10.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.831+0000 7f405559a640 1 --2- 172.21.15.83:0/2722819875 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f405815db50 0x7f40400039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:10.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.831+0000 7f405559a640 1 --2- 172.21.15.83:0/2722819875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4040003f70 0x7f4040004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:10.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.831+0000 7f405559a640 1 --2- 172.21.15.83:0/2722819875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4040004d50 0x7f40400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:10.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.831+0000 7f405559a640 1 -- 172.21.15.83:0/2722819875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405815e720 con 0x7f4040004d50 2024-04-17T10:39:10.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.831+0000 7f405559a640 1 -- 172.21.15.83:0/2722819875 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404000dcc0 con 0x7f4040003f70 2024-04-17T10:39:10.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.831+0000 7f405559a640 1 -- 172.21.15.83:0/2722819875 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f404000dec0 con 0x7f405815db50 2024-04-17T10:39:10.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.831+0000 7f4056d9d640 1 --2- 172.21.15.83:0/2722819875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4040004d50 0x7f40400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:10.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.831+0000 7f405659c640 1 --2- 172.21.15.83:0/2722819875 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f405815db50 0x7f40400039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:10.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.831+0000 7f4055d9b640 1 --2- 172.21.15.83:0/2722819875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4040003f70 0x7f4040004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:10.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.831+0000 7f4056d9d640 1 -- 172.21.15.83:0/2722819875 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f405815db50 msgr2=0x7f40400039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:10.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.831+0000 7f4056d9d640 1 --2- 172.21.15.83:0/2722819875 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f405815db50 0x7f40400039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.831+0000 7f4056d9d640 1 -- 172.21.15.83:0/2722819875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4040003f70 msgr2=0x7f4040004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:10.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.831+0000 7f4056d9d640 1 --2- 172.21.15.83:0/2722819875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4040003f70 0x7f4040004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.831+0000 7f4056d9d640 1 -- 172.21.15.83:0/2722819875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4040003530 con 0x7f4040004d50 2024-04-17T10:39:10.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.831+0000 7f4055d9b640 1 --2- 172.21.15.83:0/2722819875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4040003f70 0x7f4040004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:10.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.835+0000 7f4056d9d640 1 --2- 172.21.15.83:0/2722819875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4040004d50 0x7f40400051d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f405000aa20 tx=0x7f405001d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7219bdf927a91242 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:10.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.835+0000 7f405559a640 1 -- 172.21.15.83:0/2722819875 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405000a640 con 0x7f4040004d50 2024-04-17T10:39:10.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.835+0000 7f405559a640 1 -- 172.21.15.83:0/2722819875 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40500ad630 con 0x7f4040004d50 2024-04-17T10:39:10.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.835+0000 7f405559a640 1 -- 172.21.15.83:0/2722819875 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40500ada60 con 0x7f4040004d50 2024-04-17T10:39:10.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.835+0000 7f405cc84640 1 -- 172.21.15.83:0/2722819875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4040004d50 msgr2=0x7f40400051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:10.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.835+0000 7f405cc84640 1 --2- 172.21.15.83:0/2722819875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4040004d50 0x7f40400051d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f405000aa20 tx=0x7f405001d360 comp rx=0 tx=0).stop 2024-04-17T10:39:10.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.835+0000 7f4056d9d640 1 -- 172.21.15.83:0/2722819875 reap_dead start 2024-04-17T10:39:10.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.835+0000 7f405cc84640 1 -- 172.21.15.83:0/2722819875 shutdown_connections 2024-04-17T10:39:10.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.835+0000 7f405cc84640 1 -- 172.21.15.83:0/2722819875 >> 172.21.15.83:0/2722819875 conn(0x7f405806abe0 msgr2=0x7f405806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:10.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.835+0000 7f405cc84640 1 -- 172.21.15.83:0/2722819875 shutdown_connections 2024-04-17T10:39:10.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.835+0000 7f405cc84640 1 -- 172.21.15.83:0/2722819875 wait complete. 2024-04-17T10:39:10.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.835+0000 7f405cc84640 1 Processor -- start 2024-04-17T10:39:10.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.835+0000 7f405cc84640 1 -- start start 2024-04-17T10:39:10.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.835+0000 7f405cc84640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4040004d50 0x7f40580b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:10.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.835+0000 7f405cc84640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40580b0410 0x7f40580afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:10.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.835+0000 7f405cc84640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40581549c0 0x7f405815b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:10.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.835+0000 7f405cc84640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405815df30 con 0x7f4040004d50 2024-04-17T10:39:10.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.835+0000 7f405cc84640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405815efc0 con 0x7f40580b0410 2024-04-17T10:39:10.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.835+0000 7f405cc84640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f405815e0b0 con 0x7f40581549c0 2024-04-17T10:39:10.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.835+0000 7f405659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4040004d50 0x7f40580b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:10.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.835+0000 7f405659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4040004d50 0x7f40580b1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37040/0 (socket says 172.21.15.83:37040) 2024-04-17T10:39:10.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.835+0000 7f405659c640 1 -- 172.21.15.83:0/3568256964 learned_addr learned my addr 172.21.15.83:0/3568256964 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:10.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.835+0000 7f4056d9d640 1 --2- 172.21.15.83:0/3568256964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40581549c0 0x7f405815b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:10.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.835+0000 7f4055d9b640 1 --2- 172.21.15.83:0/3568256964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40580b0410 0x7f40580afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:10.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.835+0000 7f405659c640 1 -- 172.21.15.83:0/3568256964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40581549c0 msgr2=0x7f405815b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:10.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.835+0000 7f405659c640 1 --2- 172.21.15.83:0/3568256964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40581549c0 0x7f405815b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.835+0000 7f405659c640 1 -- 172.21.15.83:0/3568256964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40580b0410 msgr2=0x7f40580afcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:10.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.835+0000 7f405659c640 1 --2- 172.21.15.83:0/3568256964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40580b0410 0x7f40580afcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.835+0000 7f405659c640 1 -- 172.21.15.83:0/3568256964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40580b09d0 con 0x7f4040004d50 2024-04-17T10:39:10.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.839+0000 7f405659c640 1 --2- 172.21.15.83:0/3568256964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4040004d50 0x7f40580b1a70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4048011610 tx=0x7f4048011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:10.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.839+0000 7f4046ffd640 1 -- 172.21.15.83:0/3568256964 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404800f8f0 con 0x7f4040004d50 2024-04-17T10:39:10.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.839+0000 7f4046ffd640 1 -- 172.21.15.83:0/3568256964 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f404800fa90 con 0x7f4040004d50 2024-04-17T10:39:10.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.839+0000 7f4046ffd640 1 -- 172.21.15.83:0/3568256964 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404800fe40 con 0x7f4040004d50 2024-04-17T10:39:10.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.839+0000 7f405cc84640 1 -- 172.21.15.83:0/3568256964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40580b0cb0 con 0x7f4040004d50 2024-04-17T10:39:10.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.839+0000 7f405cc84640 1 -- 172.21.15.83:0/3568256964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40580b1270 con 0x7f4040004d50 2024-04-17T10:39:10.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.839+0000 7f405cc84640 1 -- 172.21.15.83:0/3568256964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4058074f90 con 0x7f4040004d50 2024-04-17T10:39:10.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.839+0000 7f4046ffd640 1 -- 172.21.15.83:0/3568256964 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4048015460 con 0x7f4040004d50 2024-04-17T10:39:10.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.839+0000 7f4046ffd640 1 --2- 172.21.15.83:0/3568256964 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4038066f60 0x7f4038069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:10.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.839+0000 7f4046ffd640 1 -- 172.21.15.83:0/3568256964 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f40480855e0 con 0x7f4040004d50 2024-04-17T10:39:10.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.839+0000 7f4055d9b640 1 --2- 172.21.15.83:0/3568256964 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4038066f60 0x7f4038069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:10.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.839+0000 7f4055d9b640 1 --2- 172.21.15.83:0/3568256964 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4038066f60 0x7f4038069420 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f40581f2960 tx=0x7f404c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:10.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.843+0000 7f4046ffd640 1 -- 172.21.15.83:0/3568256964 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4048019030 con 0x7f4040004d50 2024-04-17T10:39:10.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.991+0000 7f405cc84640 1 -- 172.21.15.83:0/3568256964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f40580b2af0 con 0x7f4040004d50 2024-04-17T10:39:10.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.991+0000 7f4046ffd640 1 -- 172.21.15.83:0/3568256964 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) v1 ==== 84+0+65 (secure 0 0 0) 0x7f40480569d0 con 0x7f4040004d50 2024-04-17T10:39:10.989 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:39:10.990 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-04-17T10:39:10.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:39:10.990 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2024-04-17T10:39:10.990 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2024-04-17T10:39:10.990 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:39:10.990 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-04-17T10:39:10.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.995+0000 7f405cc84640 1 -- 172.21.15.83:0/3568256964 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4038066f60 msgr2=0x7f4038069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:10.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.995+0000 7f405cc84640 1 --2- 172.21.15.83:0/3568256964 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4038066f60 0x7f4038069420 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f40581f2960 tx=0x7f404c009040 comp rx=0 tx=0).stop 2024-04-17T10:39:10.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.995+0000 7f405cc84640 1 -- 172.21.15.83:0/3568256964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4040004d50 msgr2=0x7f40580b1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:10.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.995+0000 7f405cc84640 1 --2- 172.21.15.83:0/3568256964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4040004d50 0x7f40580b1a70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4048011610 tx=0x7f4048011ad0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.995+0000 7f405cc84640 1 -- 172.21.15.83:0/3568256964 shutdown_connections 2024-04-17T10:39:10.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.995+0000 7f405cc84640 1 --2- 172.21.15.83:0/3568256964 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4038066f60 0x7f4038069420 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.995+0000 7f405cc84640 1 --2- 172.21.15.83:0/3568256964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40581549c0 0x7f405815b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.995+0000 7f405cc84640 1 --2- 172.21.15.83:0/3568256964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40580b0410 0x7f40580afcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.995+0000 7f405cc84640 1 --2- 172.21.15.83:0/3568256964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4040004d50 0x7f40580b1a70 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:10.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.995+0000 7f405cc84640 1 -- 172.21.15.83:0/3568256964 >> 172.21.15.83:0/3568256964 conn(0x7f405806abe0 msgr2=0x7f4058159af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:10.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.995+0000 7f405cc84640 1 -- 172.21.15.83:0/3568256964 shutdown_connections 2024-04-17T10:39:10.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:10.995+0000 7f405cc84640 1 -- 172.21.15.83:0/3568256964 wait complete. 2024-04-17T10:39:11.007 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-17T10:39:11.007 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-17T10:39:11.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.131+0000 7f82793f8640 1 Processor -- start 2024-04-17T10:39:11.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.131+0000 7f82793f8640 1 -- start start 2024-04-17T10:39:11.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.131+0000 7f82793f8640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f82740aedf0 0x7f82740b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:11.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.131+0000 7f82793f8640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f827406b2f0 0x7f8274071720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:11.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.131+0000 7f82793f8640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82740b39d0 con 0x7f827415cb40 2024-04-17T10:39:11.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.131+0000 7f82793f8640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8274071290 con 0x7f82740aedf0 2024-04-17T10:39:11.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.131+0000 7f82793f8640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8274070c40 con 0x7f827406b2f0 2024-04-17T10:39:11.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.131+0000 7f8272ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f82740aedf0 0x7f82740b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:11.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.131+0000 7f8272ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f82740aedf0 0x7f82740b30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37050/0 (socket says 172.21.15.83:37050) 2024-04-17T10:39:11.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.131+0000 7f8272ffd640 1 -- 172.21.15.83:0/1780835950 learned_addr learned my addr 172.21.15.83:0/1780835950 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:11.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.131+0000 7f82737fe640 1 --2- 172.21.15.83:0/1780835950 >> v2:172.21.15.167:3300/0 conn(0x7f827406b2f0 0x7f8274071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:11.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.131+0000 7f8272ffd640 1 -- 172.21.15.83:0/1780835950 >> v2:172.21.15.167:3300/0 conn(0x7f827406b2f0 msgr2=0x7f8274071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:11.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.131+0000 7f8272ffd640 1 --2- 172.21.15.83:0/1780835950 >> v2:172.21.15.167:3300/0 conn(0x7f827406b2f0 0x7f8274071720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:11.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.131+0000 7f8272ffd640 1 -- 172.21.15.83:0/1780835950 >> v1:172.21.15.167:6789/0 conn(0x7f827415cb40 legacy=0x7f827406abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:11.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.131+0000 7f8272ffd640 1 -- 172.21.15.83:0/1780835950 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8274070f20 con 0x7f82740aedf0 2024-04-17T10:39:11.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.135+0000 7f8272ffd640 1 --2- 172.21.15.83:0/1780835950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82740aedf0 0x7f82740b30f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f827415bf00 tx=0x7f826401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce8a66e406abce41 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:11.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.135+0000 7f8271ffb640 1 -- 172.21.15.83:0/1780835950 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82640b0070 con 0x7f82740aedf0 2024-04-17T10:39:11.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.135+0000 7f8271ffb640 1 -- 172.21.15.83:0/1780835950 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82640ab600 con 0x7f82740aedf0 2024-04-17T10:39:11.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.135+0000 7f8271ffb640 1 -- 172.21.15.83:0/1780835950 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82640ab9d0 con 0x7f82740aedf0 2024-04-17T10:39:11.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.135+0000 7f82793f8640 1 -- 172.21.15.83:0/1780835950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82740aedf0 msgr2=0x7f82740b30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:11.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.135+0000 7f82793f8640 1 --2- 172.21.15.83:0/1780835950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82740aedf0 0x7f82740b30f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f827415bf00 tx=0x7f826401c4e0 comp rx=0 tx=0).stop 2024-04-17T10:39:11.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.135+0000 7f82793f8640 1 -- 172.21.15.83:0/1780835950 shutdown_connections 2024-04-17T10:39:11.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.135+0000 7f82793f8640 1 --2- 172.21.15.83:0/1780835950 >> v2:172.21.15.167:3300/0 conn(0x7f827406b2f0 0x7f8274071720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:11.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.135+0000 7f82793f8640 1 --2- 172.21.15.83:0/1780835950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82740aedf0 0x7f82740b30f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:11.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.135+0000 7f82793f8640 1 -- 172.21.15.83:0/1780835950 >> 172.21.15.83:0/1780835950 conn(0x7f82740cc050 msgr2=0x7f82740cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:11.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.135+0000 7f82793f8640 1 -- 172.21.15.83:0/1780835950 shutdown_connections 2024-04-17T10:39:11.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.135+0000 7f82793f8640 1 -- 172.21.15.83:0/1780835950 wait complete. 2024-04-17T10:39:11.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.135+0000 7f82793f8640 1 Processor -- start 2024-04-17T10:39:11.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.135+0000 7f82793f8640 1 -- start start 2024-04-17T10:39:11.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.135+0000 7f82793f8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f827406b2f0 0x7f8274149060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:11.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.135+0000 7f82793f8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82740aedf0 0x7f8274155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:11.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.135+0000 7f82793f8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f827415cb40 0x7f82741564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:11.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.135+0000 7f82793f8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82740725f0 con 0x7f827406b2f0 2024-04-17T10:39:11.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.135+0000 7f82793f8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8274072470 con 0x7f82740aedf0 2024-04-17T10:39:11.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.135+0000 7f82793f8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8274168890 con 0x7f827415cb40 2024-04-17T10:39:11.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.135+0000 7f8272ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f827406b2f0 0x7f8274149060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:11.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.135+0000 7f8272ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f827406b2f0 0x7f8274149060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37058/0 (socket says 172.21.15.83:37058) 2024-04-17T10:39:11.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.135+0000 7f8272ffd640 1 -- 172.21.15.83:0/2668497874 learned_addr learned my addr 172.21.15.83:0/2668497874 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:11.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.135+0000 7f82727fc640 1 --2- 172.21.15.83:0/2668497874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82740aedf0 0x7f8274155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:11.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.135+0000 7f8272ffd640 1 -- 172.21.15.83:0/2668497874 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f827415cb40 msgr2=0x7f82741564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:11.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.135+0000 7f8272ffd640 1 --2- 172.21.15.83:0/2668497874 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f827415cb40 0x7f82741564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:11.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.135+0000 7f8272ffd640 1 -- 172.21.15.83:0/2668497874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82740aedf0 msgr2=0x7f8274155fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:11.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.135+0000 7f8272ffd640 1 --2- 172.21.15.83:0/2668497874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82740aedf0 0x7f8274155fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:11.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.135+0000 7f8272ffd640 1 -- 172.21.15.83:0/2668497874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82741570d0 con 0x7f827406b2f0 2024-04-17T10:39:11.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.139+0000 7f8272ffd640 1 --2- 172.21.15.83:0/2668497874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f827406b2f0 0x7f8274149060 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f82640ae040 tx=0x7f826401ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:11.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.139+0000 7f825b7fe640 1 -- 172.21.15.83:0/2668497874 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82640b0040 con 0x7f827406b2f0 2024-04-17T10:39:11.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.139+0000 7f825b7fe640 1 -- 172.21.15.83:0/2668497874 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82640bc070 con 0x7f827406b2f0 2024-04-17T10:39:11.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.139+0000 7f825b7fe640 1 -- 172.21.15.83:0/2668497874 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82640b6880 con 0x7f827406b2f0 2024-04-17T10:39:11.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.139+0000 7f82793f8640 1 -- 172.21.15.83:0/2668497874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8274156b60 con 0x7f827406b2f0 2024-04-17T10:39:11.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.139+0000 7f82793f8640 1 -- 172.21.15.83:0/2668497874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8274157670 con 0x7f827406b2f0 2024-04-17T10:39:11.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.139+0000 7f82793f8640 1 -- 172.21.15.83:0/2668497874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8274075000 con 0x7f827406b2f0 2024-04-17T10:39:11.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.139+0000 7f825b7fe640 1 -- 172.21.15.83:0/2668497874 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f82640ab4c0 con 0x7f827406b2f0 2024-04-17T10:39:11.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.139+0000 7f825b7fe640 1 --2- 172.21.15.83:0/2668497874 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8260066f60 0x7f8260069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:11.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.139+0000 7f825b7fe640 1 -- 172.21.15.83:0/2668497874 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f826411f050 con 0x7f827406b2f0 2024-04-17T10:39:11.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.139+0000 7f82727fc640 1 --2- 172.21.15.83:0/2668497874 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8260066f60 0x7f8260069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:11.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.139+0000 7f82727fc640 1 --2- 172.21.15.83:0/2668497874 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8260066f60 0x7f8260069420 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f826800a750 tx=0x7f8268008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:11.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.143+0000 7f825b7fe640 1 -- 172.21.15.83:0/2668497874 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f82640160d0 con 0x7f827406b2f0 2024-04-17T10:39:11.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:11.287+0000 7f82793f8640 1 -- 172.21.15.83:0/2668497874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f82740b2cc0 con 0x7f827406b2f0 2024-04-17T10:39:12.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.083+0000 7f825b7fe640 1 -- 172.21.15.83:0/2668497874 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7f82640f0440 con 0x7f827406b2f0 2024-04-17T10:39:12.083 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2024-04-17T10:39:12.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.087+0000 7f82793f8640 1 -- 172.21.15.83:0/2668497874 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8260066f60 msgr2=0x7f8260069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:12.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.087+0000 7f82793f8640 1 --2- 172.21.15.83:0/2668497874 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8260066f60 0x7f8260069420 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f826800a750 tx=0x7f8268008040 comp rx=0 tx=0).stop 2024-04-17T10:39:12.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.087+0000 7f82793f8640 1 -- 172.21.15.83:0/2668497874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f827406b2f0 msgr2=0x7f8274149060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:12.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.087+0000 7f82793f8640 1 --2- 172.21.15.83:0/2668497874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f827406b2f0 0x7f8274149060 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f82640ae040 tx=0x7f826401ad20 comp rx=0 tx=0).stop 2024-04-17T10:39:12.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.087+0000 7f82793f8640 1 -- 172.21.15.83:0/2668497874 shutdown_connections 2024-04-17T10:39:12.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.087+0000 7f82793f8640 1 --2- 172.21.15.83:0/2668497874 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8260066f60 0x7f8260069420 secure :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f826800a750 tx=0x7f8268008040 comp rx=0 tx=0).stop 2024-04-17T10:39:12.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.087+0000 7f82793f8640 1 --2- 172.21.15.83:0/2668497874 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f827415cb40 0x7f82741564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:12.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.087+0000 7f82793f8640 1 --2- 172.21.15.83:0/2668497874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82740aedf0 0x7f8274155fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:12.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.087+0000 7f82793f8640 1 --2- 172.21.15.83:0/2668497874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f827406b2f0 0x7f8274149060 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:12.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.087+0000 7f82793f8640 1 -- 172.21.15.83:0/2668497874 >> 172.21.15.83:0/2668497874 conn(0x7f82740cc050 msgr2=0x7f827406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:12.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.087+0000 7f82793f8640 1 -- 172.21.15.83:0/2668497874 shutdown_connections 2024-04-17T10:39:12.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.087+0000 7f82793f8640 1 -- 172.21.15.83:0/2668497874 wait complete. 2024-04-17T10:39:12.108 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-17T10:39:12.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.235+0000 7ff9d7b94640 1 Processor -- start 2024-04-17T10:39:12.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.235+0000 7ff9d7b94640 1 -- start start 2024-04-17T10:39:12.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.235+0000 7ff9d7b94640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9d00b09e0 0x7ff9d00cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:12.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.235+0000 7ff9d7b94640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9d00d0480 con 0x7ff9d015de10 2024-04-17T10:39:12.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.235+0000 7ff9d7b94640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9d006c2e0 con 0x7ff9d006ccd0 2024-04-17T10:39:12.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.235+0000 7ff9d7b94640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d00d0740 con 0x7ff9d00b09e0 2024-04-17T10:39:12.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.235+0000 7ff9d5930640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff9d015de10 0x7ff9d006c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48676/0 (socket says 172.21.15.83:48676) 2024-04-17T10:39:12.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.235+0000 7ff9d5930640 1 -- 172.21.15.83:0/2746463651 learned_addr learned my addr 172.21.15.83:0/2746463651 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:39:12.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.235+0000 7ff9d6131640 1 --2- 172.21.15.83:0/2746463651 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9d00b09e0 0x7ff9d00cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:12.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.235+0000 7ff9d492e640 1 -- 172.21.15.83:0/2746463651 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 681380850 0 0) 0x7ff9d00d0480 con 0x7ff9d015de10 2024-04-17T10:39:12.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.235+0000 7ff9d492e640 1 -- 172.21.15.83:0/2746463651 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9bc003530 con 0x7ff9d015de10 2024-04-17T10:39:12.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.235+0000 7ff9d492e640 1 -- 172.21.15.83:0/2746463651 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2310217111 0 0) 0x7ff9bc003530 con 0x7ff9d015de10 2024-04-17T10:39:12.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.235+0000 7ff9d492e640 1 -- 172.21.15.83:0/2746463651 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9d00b09e0 msgr2=0x7ff9d00cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:12.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.235+0000 7ff9d492e640 1 --2- 172.21.15.83:0/2746463651 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9d00b09e0 0x7ff9d00cc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:12.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.235+0000 7ff9d492e640 1 -- 172.21.15.83:0/2746463651 >> v1:172.21.15.167:6789/0 conn(0x7ff9d006ccd0 legacy=0x7ff9d00cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:12.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.235+0000 7ff9d492e640 1 -- 172.21.15.83:0/2746463651 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d00d0a20 con 0x7ff9d015de10 2024-04-17T10:39:12.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.235+0000 7ff9d492e640 1 -- 172.21.15.83:0/2746463651 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7ff9cc001a00 con 0x7ff9d015de10 2024-04-17T10:39:12.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.235+0000 7ff9d492e640 1 -- 172.21.15.83:0/2746463651 >> v1:172.21.15.83:6789/0 conn(0x7ff9d015de10 legacy=0x7ff9d006c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:12.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.235+0000 7ff9d492e640 1 --2- 172.21.15.83:0/2746463651 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9d00b09e0 0x7ff9bc003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:12.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.235+0000 7ff9d492e640 1 --2- 172.21.15.83:0/2746463651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9bc003fe0 0x7ff9bc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:12.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.235+0000 7ff9d492e640 1 --2- 172.21.15.83:0/2746463651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9bc004d50 0x7ff9bc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:12.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.235+0000 7ff9d492e640 1 -- 172.21.15.83:0/2746463651 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d00d0740 con 0x7ff9bc003fe0 2024-04-17T10:39:12.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.235+0000 7ff9d492e640 1 -- 172.21.15.83:0/2746463651 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc00bbc0 con 0x7ff9bc004d50 2024-04-17T10:39:12.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.235+0000 7ff9d492e640 1 -- 172.21.15.83:0/2746463651 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc00bdc0 con 0x7ff9d00b09e0 2024-04-17T10:39:12.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.235+0000 7ff9d512f640 1 --2- 172.21.15.83:0/2746463651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9bc003fe0 0x7ff9bc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:12.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.235+0000 7ff9d6131640 1 --2- 172.21.15.83:0/2746463651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9bc004d50 0x7ff9bc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:12.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.235+0000 7ff9d5930640 1 --2- 172.21.15.83:0/2746463651 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9d00b09e0 0x7ff9bc003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:12.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.235+0000 7ff9d512f640 1 -- 172.21.15.83:0/2746463651 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9d00b09e0 msgr2=0x7ff9bc003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:12.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.235+0000 7ff9d512f640 1 --2- 172.21.15.83:0/2746463651 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9d00b09e0 0x7ff9bc003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:12.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.235+0000 7ff9d512f640 1 -- 172.21.15.83:0/2746463651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9bc004d50 msgr2=0x7ff9bc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:12.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.235+0000 7ff9d512f640 1 --2- 172.21.15.83:0/2746463651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9bc004d50 0x7ff9bc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:12.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.235+0000 7ff9d512f640 1 -- 172.21.15.83:0/2746463651 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9bc003530 con 0x7ff9bc003fe0 2024-04-17T10:39:12.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.239+0000 7ff9d512f640 1 --2- 172.21.15.83:0/2746463651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9bc003fe0 0x7ff9bc004810 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0001780 tx=0x7ff9c001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92de528db884e93e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:12.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.239+0000 7ff9d492e640 1 -- 172.21.15.83:0/2746463651 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c00013a0 con 0x7ff9bc003fe0 2024-04-17T10:39:12.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.239+0000 7ff9d492e640 1 -- 172.21.15.83:0/2746463651 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c00052f0 con 0x7ff9bc003fe0 2024-04-17T10:39:12.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.239+0000 7ff9d492e640 1 -- 172.21.15.83:0/2746463651 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0003840 con 0x7ff9bc003fe0 2024-04-17T10:39:12.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.239+0000 7ff9d7b94640 1 -- 172.21.15.83:0/2746463651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9bc003fe0 msgr2=0x7ff9bc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:12.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.239+0000 7ff9d7b94640 1 --2- 172.21.15.83:0/2746463651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9bc003fe0 0x7ff9bc004810 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0001780 tx=0x7ff9c001c3f0 comp rx=0 tx=0).stop 2024-04-17T10:39:12.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.239+0000 7ff9d6131640 1 -- 172.21.15.83:0/2746463651 reap_dead start 2024-04-17T10:39:12.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.239+0000 7ff9d7b94640 1 -- 172.21.15.83:0/2746463651 shutdown_connections 2024-04-17T10:39:12.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.239+0000 7ff9d7b94640 1 -- 172.21.15.83:0/2746463651 >> 172.21.15.83:0/2746463651 conn(0x7ff9d00d1730 msgr2=0x7ff9d00d1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:12.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.239+0000 7ff9d7b94640 1 -- 172.21.15.83:0/2746463651 shutdown_connections 2024-04-17T10:39:12.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.239+0000 7ff9d7b94640 1 -- 172.21.15.83:0/2746463651 wait complete. 2024-04-17T10:39:12.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.243+0000 7ff9d7b94640 1 Processor -- start 2024-04-17T10:39:12.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.243+0000 7ff9d7b94640 1 -- start start 2024-04-17T10:39:12.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.243+0000 7ff9d7b94640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9bc003fe0 0x7ff9d014fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:12.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.243+0000 7ff9d7b94640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9d00b29d0 0x7ff9d00b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:12.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.243+0000 7ff9d7b94640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9d00b2210 0x7ff9d0152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:12.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.243+0000 7ff9d7b94640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d00cd100 con 0x7ff9d00b2210 2024-04-17T10:39:12.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.243+0000 7ff9d7b94640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d00ccf80 con 0x7ff9bc003fe0 2024-04-17T10:39:12.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.243+0000 7ff9d7b94640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d00cd280 con 0x7ff9d00b29d0 2024-04-17T10:39:12.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.243+0000 7ff9d6131640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9d00b2210 0x7ff9d0152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:12.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.243+0000 7ff9d6131640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9d00b2210 0x7ff9d0152910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37076/0 (socket says 172.21.15.83:37076) 2024-04-17T10:39:12.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.243+0000 7ff9d6131640 1 -- 172.21.15.83:0/3385985710 learned_addr learned my addr 172.21.15.83:0/3385985710 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:12.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.243+0000 7ff9d5930640 1 --2- 172.21.15.83:0/3385985710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9bc003fe0 0x7ff9d014fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:12.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.243+0000 7ff9d512f640 1 --2- 172.21.15.83:0/3385985710 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9d00b29d0 0x7ff9d00b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:12.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.243+0000 7ff9d6131640 1 -- 172.21.15.83:0/3385985710 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9d00b29d0 msgr2=0x7ff9d00b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:12.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.243+0000 7ff9d6131640 1 --2- 172.21.15.83:0/3385985710 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9d00b29d0 0x7ff9d00b1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:12.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.243+0000 7ff9d6131640 1 -- 172.21.15.83:0/3385985710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9bc003fe0 msgr2=0x7ff9d014fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:12.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.243+0000 7ff9d6131640 1 --2- 172.21.15.83:0/3385985710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9bc003fe0 0x7ff9d014fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:12.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.243+0000 7ff9d6131640 1 -- 172.21.15.83:0/3385985710 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d00cfdd0 con 0x7ff9d00b2210 2024-04-17T10:39:12.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.243+0000 7ff9d5930640 1 --2- 172.21.15.83:0/3385985710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9bc003fe0 0x7ff9d014fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:12.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.243+0000 7ff9d6131640 1 --2- 172.21.15.83:0/3385985710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9d00b2210 0x7ff9d0152910 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0071850 tx=0x7ff9c8011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:12.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.243+0000 7ff9c67fc640 1 -- 172.21.15.83:0/3385985710 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8019020 con 0x7ff9d00b2210 2024-04-17T10:39:12.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.243+0000 7ff9c67fc640 1 -- 172.21.15.83:0/3385985710 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c8003620 con 0x7ff9d00b2210 2024-04-17T10:39:12.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.243+0000 7ff9d7b94640 1 -- 172.21.15.83:0/3385985710 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9d01503a0 con 0x7ff9d00b2210 2024-04-17T10:39:12.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.243+0000 7ff9c67fc640 1 -- 172.21.15.83:0/3385985710 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8003a10 con 0x7ff9d00b2210 2024-04-17T10:39:12.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.243+0000 7ff9d7b94640 1 -- 172.21.15.83:0/3385985710 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9d0150960 con 0x7ff9d00b2210 2024-04-17T10:39:12.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.243+0000 7ff9d7b94640 1 -- 172.21.15.83:0/3385985710 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9d0074f90 con 0x7ff9d00b2210 2024-04-17T10:39:12.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.243+0000 7ff9c67fc640 1 -- 172.21.15.83:0/3385985710 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff9c801c020 con 0x7ff9d00b2210 2024-04-17T10:39:12.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.243+0000 7ff9c67fc640 1 --2- 172.21.15.83:0/3385985710 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff9b4066f60 0x7ff9b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:12.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.243+0000 7ff9c67fc640 1 -- 172.21.15.83:0/3385985710 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5227+0+0 (secure 0 0 0) 0x7ff9c8085150 con 0x7ff9d00b2210 2024-04-17T10:39:12.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.243+0000 7ff9d5930640 1 --2- 172.21.15.83:0/3385985710 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff9b4066f60 0x7ff9b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:12.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.247+0000 7ff9d5930640 1 --2- 172.21.15.83:0/3385985710 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff9b4066f60 0x7ff9b4069420 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc001590 tx=0x7ff9cc0014a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:12.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.251+0000 7ff9c67fc640 1 -- 172.21.15.83:0/3385985710 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff9c8023030 con 0x7ff9d00b2210 2024-04-17T10:39:12.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:12.395+0000 7ff9d7b94640 1 -- 172.21.15.83:0/3385985710 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff9d00ae590 con 0x7ff9d00b2210 2024-04-17T10:39:13.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.095+0000 7ff9c67fc640 1 -- 172.21.15.83:0/3385985710 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff9c80563d0 con 0x7ff9d00b2210 2024-04-17T10:39:13.093 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2024-04-17T10:39:13.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.099+0000 7ff9d7b94640 1 -- 172.21.15.83:0/3385985710 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff9b4066f60 msgr2=0x7ff9b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:13.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.099+0000 7ff9d7b94640 1 --2- 172.21.15.83:0/3385985710 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff9b4066f60 0x7ff9b4069420 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc001590 tx=0x7ff9cc0014a0 comp rx=0 tx=0).stop 2024-04-17T10:39:13.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.099+0000 7ff9d7b94640 1 -- 172.21.15.83:0/3385985710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9d00b2210 msgr2=0x7ff9d0152910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:13.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.099+0000 7ff9d7b94640 1 --2- 172.21.15.83:0/3385985710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9d00b2210 0x7ff9d0152910 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0071850 tx=0x7ff9c8011920 comp rx=0 tx=0).stop 2024-04-17T10:39:13.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.099+0000 7ff9d7b94640 1 -- 172.21.15.83:0/3385985710 shutdown_connections 2024-04-17T10:39:13.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.099+0000 7ff9d7b94640 1 --2- 172.21.15.83:0/3385985710 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff9b4066f60 0x7ff9b4069420 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:13.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.099+0000 7ff9d7b94640 1 --2- 172.21.15.83:0/3385985710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9d00b2210 0x7ff9d0152910 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:13.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.099+0000 7ff9d7b94640 1 --2- 172.21.15.83:0/3385985710 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9d00b29d0 0x7ff9d00b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:13.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.099+0000 7ff9d7b94640 1 --2- 172.21.15.83:0/3385985710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9bc003fe0 0x7ff9d014fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:13.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.099+0000 7ff9d7b94640 1 -- 172.21.15.83:0/3385985710 >> 172.21.15.83:0/3385985710 conn(0x7ff9d00d1730 msgr2=0x7ff9d00cdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:13.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.099+0000 7ff9d7b94640 1 -- 172.21.15.83:0/3385985710 shutdown_connections 2024-04-17T10:39:13.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.099+0000 7ff9d7b94640 1 -- 172.21.15.83:0/3385985710 wait complete. 2024-04-17T10:39:13.133 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-17T10:39:13.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.259+0000 7fd8967ed640 1 Processor -- start 2024-04-17T10:39:13.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.259+0000 7fd8967ed640 1 -- start start 2024-04-17T10:39:13.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.259+0000 7fd8967ed640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8900d19c0 0x7fd8900d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:13.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.259+0000 7fd8967ed640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd89015be00 0x7fd8900d0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:13.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.259+0000 7fd8967ed640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8900b18a0 con 0x7fd8900d05b0 2024-04-17T10:39:13.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.259+0000 7fd8967ed640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8900b1b80 con 0x7fd89015be00 2024-04-17T10:39:13.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.259+0000 7fd8967ed640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8900b1e60 con 0x7fd8900d19c0 2024-04-17T10:39:13.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.259+0000 7fd88f7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd89015be00 0x7fd8900d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:13.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.259+0000 7fd88f7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd89015be00 0x7fd8900d0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37082/0 (socket says 172.21.15.83:37082) 2024-04-17T10:39:13.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.259+0000 7fd88f7fe640 1 -- 172.21.15.83:0/3482697923 learned_addr learned my addr 172.21.15.83:0/3482697923 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:13.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.259+0000 7fd88ffff640 1 --2- 172.21.15.83:0/3482697923 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8900d19c0 0x7fd8900d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:13.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.259+0000 7fd88f7fe640 1 -- 172.21.15.83:0/3482697923 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8900d19c0 msgr2=0x7fd8900d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:13.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.259+0000 7fd88f7fe640 1 --2- 172.21.15.83:0/3482697923 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8900d19c0 0x7fd8900d1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:13.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.259+0000 7fd88f7fe640 1 -- 172.21.15.83:0/3482697923 >> v1:172.21.15.167:6789/0 conn(0x7fd8900d05b0 legacy=0x7fd89015d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:13.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.259+0000 7fd88f7fe640 1 -- 172.21.15.83:0/3482697923 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8900b2140 con 0x7fd89015be00 2024-04-17T10:39:13.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.263+0000 7fd88f7fe640 1 --2- 172.21.15.83:0/3482697923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd89015be00 0x7fd8900d0070 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd884009870 tx=0x7fd88401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb47317710e6feb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:13.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.263+0000 7fd88effd640 1 -- 172.21.15.83:0/3482697923 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8840ab620 con 0x7fd89015be00 2024-04-17T10:39:13.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.263+0000 7fd88effd640 1 -- 172.21.15.83:0/3482697923 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8840b0070 con 0x7fd89015be00 2024-04-17T10:39:13.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.263+0000 7fd88effd640 1 -- 172.21.15.83:0/3482697923 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8840ab9e0 con 0x7fd89015be00 2024-04-17T10:39:13.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.263+0000 7fd8967ed640 1 -- 172.21.15.83:0/3482697923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd89015be00 msgr2=0x7fd8900d0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:13.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.263+0000 7fd8967ed640 1 --2- 172.21.15.83:0/3482697923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd89015be00 0x7fd8900d0070 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd884009870 tx=0x7fd88401c460 comp rx=0 tx=0).stop 2024-04-17T10:39:13.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.263+0000 7fd8967ed640 1 -- 172.21.15.83:0/3482697923 shutdown_connections 2024-04-17T10:39:13.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.263+0000 7fd8967ed640 1 --2- 172.21.15.83:0/3482697923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd89015be00 0x7fd8900d0070 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:13.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.263+0000 7fd8967ed640 1 --2- 172.21.15.83:0/3482697923 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8900d19c0 0x7fd8900d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:13.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.263+0000 7fd8967ed640 1 -- 172.21.15.83:0/3482697923 >> 172.21.15.83:0/3482697923 conn(0x7fd89009a9d0 msgr2=0x7fd89009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:13.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.263+0000 7fd8967ed640 1 -- 172.21.15.83:0/3482697923 shutdown_connections 2024-04-17T10:39:13.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.263+0000 7fd8967ed640 1 -- 172.21.15.83:0/3482697923 wait complete. 2024-04-17T10:39:13.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.263+0000 7fd8967ed640 1 Processor -- start 2024-04-17T10:39:13.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.263+0000 7fd8967ed640 1 -- start start 2024-04-17T10:39:13.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.263+0000 7fd8967ed640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8900d05b0 0x7fd8900c53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:13.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.263+0000 7fd8967ed640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8900d19c0 0x7fd8901532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:13.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.263+0000 7fd8967ed640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8900af7b0 0x7fd8900afc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:13.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.263+0000 7fd8967ed640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89015d160 con 0x7fd8900d05b0 2024-04-17T10:39:13.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.263+0000 7fd8967ed640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89015cfe0 con 0x7fd8900af7b0 2024-04-17T10:39:13.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.263+0000 7fd8967ed640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd890168a30 con 0x7fd8900d19c0 2024-04-17T10:39:13.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.263+0000 7fd88ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8900d05b0 0x7fd8900c53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:13.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.267+0000 7fd88ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8900d05b0 0x7fd8900c53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37086/0 (socket says 172.21.15.83:37086) 2024-04-17T10:39:13.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.267+0000 7fd88ffff640 1 -- 172.21.15.83:0/1991041119 learned_addr learned my addr 172.21.15.83:0/1991041119 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:13.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.267+0000 7fd894d8a640 1 --2- 172.21.15.83:0/1991041119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8900af7b0 0x7fd8900afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:13.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.267+0000 7fd88ffff640 1 -- 172.21.15.83:0/1991041119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8900d19c0 msgr2=0x7fd8901532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:13.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.267+0000 7fd88ffff640 1 --2- 172.21.15.83:0/1991041119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8900d19c0 0x7fd8901532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:13.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.267+0000 7fd88ffff640 1 -- 172.21.15.83:0/1991041119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8900af7b0 msgr2=0x7fd8900afc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:13.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.267+0000 7fd88ffff640 1 --2- 172.21.15.83:0/1991041119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8900af7b0 0x7fd8900afc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:13.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.267+0000 7fd88ffff640 1 -- 172.21.15.83:0/1991041119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8900cd190 con 0x7fd8900d05b0 2024-04-17T10:39:13.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.267+0000 7fd894d8a640 1 --2- 172.21.15.83:0/1991041119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8900af7b0 0x7fd8900afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:13.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.267+0000 7fd88ffff640 1 --2- 172.21.15.83:0/1991041119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8900d05b0 0x7fd8900c53b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd88000c7f0 tx=0x7fd88000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:13.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.267+0000 7fd88cff9640 1 -- 172.21.15.83:0/1991041119 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd880015070 con 0x7fd8900d05b0 2024-04-17T10:39:13.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.267+0000 7fd88cff9640 1 -- 172.21.15.83:0/1991041119 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd880007820 con 0x7fd8900d05b0 2024-04-17T10:39:13.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.267+0000 7fd8967ed640 1 -- 172.21.15.83:0/1991041119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd890153de0 con 0x7fd8900d05b0 2024-04-17T10:39:13.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.267+0000 7fd88cff9640 1 -- 172.21.15.83:0/1991041119 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd880007c10 con 0x7fd8900d05b0 2024-04-17T10:39:13.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.267+0000 7fd8967ed640 1 -- 172.21.15.83:0/1991041119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd890153820 con 0x7fd8900d05b0 2024-04-17T10:39:13.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.267+0000 7fd8967ed640 1 -- 172.21.15.83:0/1991041119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd890074f90 con 0x7fd8900d05b0 2024-04-17T10:39:13.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.267+0000 7fd88cff9640 1 -- 172.21.15.83:0/1991041119 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd880018070 con 0x7fd8900d05b0 2024-04-17T10:39:13.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.267+0000 7fd88cff9640 1 --2- 172.21.15.83:0/1991041119 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd87c066f60 0x7fd87c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:13.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.267+0000 7fd88cff9640 1 -- 172.21.15.83:0/1991041119 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fd880085710 con 0x7fd8900d05b0 2024-04-17T10:39:13.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.267+0000 7fd88f7fe640 1 --2- 172.21.15.83:0/1991041119 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd87c066f60 0x7fd87c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:13.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.267+0000 7fd88f7fe640 1 --2- 172.21.15.83:0/1991041119 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd87c066f60 0x7fd87c069420 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd8840af000 tx=0x7fd884004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:13.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.271+0000 7fd88cff9640 1 -- 172.21.15.83:0/1991041119 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd8800566e0 con 0x7fd8900d05b0 2024-04-17T10:39:13.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:13.467+0000 7fd8967ed640 1 -- 172.21.15.83:0/1991041119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd8900d2900 con 0x7fd8900d05b0 2024-04-17T10:39:14.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.099+0000 7fd88cff9640 1 -- 172.21.15.83:0/1991041119 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v11) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd880055fe0 con 0x7fd8900d05b0 2024-04-17T10:39:14.095 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 4 and data pool 5 2024-04-17T10:39:14.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.103+0000 7fd8967ed640 1 -- 172.21.15.83:0/1991041119 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd87c066f60 msgr2=0x7fd87c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:14.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.103+0000 7fd8967ed640 1 --2- 172.21.15.83:0/1991041119 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd87c066f60 0x7fd87c069420 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd8840af000 tx=0x7fd884004740 comp rx=0 tx=0).stop 2024-04-17T10:39:14.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.103+0000 7fd8967ed640 1 -- 172.21.15.83:0/1991041119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8900d05b0 msgr2=0x7fd8900c53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:14.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.103+0000 7fd8967ed640 1 --2- 172.21.15.83:0/1991041119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8900d05b0 0x7fd8900c53b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd88000c7f0 tx=0x7fd88000ccb0 comp rx=0 tx=0).stop 2024-04-17T10:39:14.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.103+0000 7fd8967ed640 1 -- 172.21.15.83:0/1991041119 shutdown_connections 2024-04-17T10:39:14.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.103+0000 7fd8967ed640 1 --2- 172.21.15.83:0/1991041119 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd87c066f60 0x7fd87c069420 secure :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd8840af000 tx=0x7fd884004740 comp rx=0 tx=0).stop 2024-04-17T10:39:14.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.103+0000 7fd8967ed640 1 --2- 172.21.15.83:0/1991041119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8900af7b0 0x7fd8900afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:14.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.103+0000 7fd8967ed640 1 --2- 172.21.15.83:0/1991041119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8900d19c0 0x7fd8901532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:14.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.103+0000 7fd8967ed640 1 --2- 172.21.15.83:0/1991041119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8900d05b0 0x7fd8900c53b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:14.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.103+0000 7fd8967ed640 1 -- 172.21.15.83:0/1991041119 >> 172.21.15.83:0/1991041119 conn(0x7fd89009a9d0 msgr2=0x7fd890159560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:14.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.103+0000 7fd8967ed640 1 -- 172.21.15.83:0/1991041119 shutdown_connections 2024-04-17T10:39:14.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.103+0000 7fd8967ed640 1 -- 172.21.15.83:0/1991041119 wait complete. 2024-04-17T10:39:14.141 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:39:14.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.263+0000 7faf6b9f9640 1 Processor -- start 2024-04-17T10:39:14.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.263+0000 7faf6b9f9640 1 -- start start 2024-04-17T10:39:14.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.263+0000 7faf6b9f9640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faf640aecd0 0x7faf640cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:14.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.263+0000 7faf6b9f9640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf640b3f80 0x7faf640b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:14.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.263+0000 7faf6b9f9640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf64072760 con 0x7faf64166550 2024-04-17T10:39:14.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.263+0000 7faf6b9f9640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf640cf810 con 0x7faf640aecd0 2024-04-17T10:39:14.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.263+0000 7faf6b9f9640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf640b4950 con 0x7faf640b3f80 2024-04-17T10:39:14.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.263+0000 7faf69795640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faf640aecd0 0x7faf640cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:14.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.263+0000 7faf69f96640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf640b3f80 0x7faf640b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:14.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.263+0000 7faf69f96640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf640b3f80 0x7faf640b4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:44940/0 (socket says 172.21.15.83:44940) 2024-04-17T10:39:14.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.263+0000 7faf69f96640 1 -- 172.21.15.83:0/3005569411 learned_addr learned my addr 172.21.15.83:0/3005569411 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:14.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.263+0000 7faf68f94640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7faf64166550 0x7faf64071be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:46818/0 (socket says 172.21.15.83:46818) 2024-04-17T10:39:14.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.263+0000 7faf69795640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faf640aecd0 0x7faf640cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:40978/0 (socket says 172.21.15.83:40978) 2024-04-17T10:39:14.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.263+0000 7faf69f96640 1 -- 172.21.15.83:0/3005569411 >> v2:172.21.15.167:3300/0 conn(0x7faf640aecd0 msgr2=0x7faf640cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:14.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.263+0000 7faf69f96640 1 --2- 172.21.15.83:0/3005569411 >> v2:172.21.15.167:3300/0 conn(0x7faf640aecd0 0x7faf640cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:14.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.263+0000 7faf69f96640 1 -- 172.21.15.83:0/3005569411 >> v1:172.21.15.167:6789/0 conn(0x7faf64166550 legacy=0x7faf64071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:14.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.263+0000 7faf69f96640 1 -- 172.21.15.83:0/3005569411 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf640cf200 con 0x7faf640b3f80 2024-04-17T10:39:14.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.267+0000 7faf69f96640 1 --2- 172.21.15.83:0/3005569411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf640b3f80 0x7faf640b4410 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7faf5c009870 tx=0x7faf5c01c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6b21d2094676932 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:14.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.267+0000 7faf5bfff640 1 -- 172.21.15.83:0/3005569411 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c0b1070 con 0x7faf640b3f80 2024-04-17T10:39:14.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.267+0000 7faf5bfff640 1 -- 172.21.15.83:0/3005569411 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5c0ac620 con 0x7faf640b3f80 2024-04-17T10:39:14.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.267+0000 7faf5bfff640 1 -- 172.21.15.83:0/3005569411 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c0ac9f0 con 0x7faf640b3f80 2024-04-17T10:39:14.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.267+0000 7faf6b9f9640 1 -- 172.21.15.83:0/3005569411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf640b3f80 msgr2=0x7faf640b4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:14.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.267+0000 7faf6b9f9640 1 --2- 172.21.15.83:0/3005569411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf640b3f80 0x7faf640b4410 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7faf5c009870 tx=0x7faf5c01c4d0 comp rx=0 tx=0).stop 2024-04-17T10:39:14.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.267+0000 7faf6b9f9640 1 -- 172.21.15.83:0/3005569411 shutdown_connections 2024-04-17T10:39:14.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.267+0000 7faf6b9f9640 1 --2- 172.21.15.83:0/3005569411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf640b3f80 0x7faf640b4410 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:14.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.267+0000 7faf6b9f9640 1 --2- 172.21.15.83:0/3005569411 >> v2:172.21.15.167:3300/0 conn(0x7faf640aecd0 0x7faf640cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:14.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.267+0000 7faf6b9f9640 1 -- 172.21.15.83:0/3005569411 >> 172.21.15.83:0/3005569411 conn(0x7faf6406abe0 msgr2=0x7faf6406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:14.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.267+0000 7faf6b9f9640 1 -- 172.21.15.83:0/3005569411 shutdown_connections 2024-04-17T10:39:14.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.267+0000 7faf6b9f9640 1 -- 172.21.15.83:0/3005569411 wait complete. 2024-04-17T10:39:14.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.267+0000 7faf6b9f9640 1 Processor -- start 2024-04-17T10:39:14.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.267+0000 7faf6b9f9640 1 -- start start 2024-04-17T10:39:14.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.267+0000 7faf6b9f9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf640aecd0 0x7faf641532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:14.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.267+0000 7faf6b9f9640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf640b3f80 0x7faf64158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:14.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.267+0000 7faf6b9f9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf64166550 0x7faf64156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:14.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.267+0000 7faf6b9f9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf640b3000 con 0x7faf640aecd0 2024-04-17T10:39:14.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.267+0000 7faf6b9f9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf640b2e80 con 0x7faf64166550 2024-04-17T10:39:14.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.267+0000 7faf6b9f9640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf640b3180 con 0x7faf640b3f80 2024-04-17T10:39:14.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.267+0000 7faf69795640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf640aecd0 0x7faf641532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:14.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.267+0000 7faf69795640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf640aecd0 0x7faf641532e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37102/0 (socket says 172.21.15.83:37102) 2024-04-17T10:39:14.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.267+0000 7faf69795640 1 -- 172.21.15.83:0/1384180581 learned_addr learned my addr 172.21.15.83:0/1384180581 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:14.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.271+0000 7faf69f96640 1 --2- 172.21.15.83:0/1384180581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf64166550 0x7faf64156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:14.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.271+0000 7faf68f94640 1 --2- 172.21.15.83:0/1384180581 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf640b3f80 0x7faf64158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:14.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.271+0000 7faf69795640 1 -- 172.21.15.83:0/1384180581 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf640b3f80 msgr2=0x7faf64158fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:14.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.271+0000 7faf69795640 1 --2- 172.21.15.83:0/1384180581 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf640b3f80 0x7faf64158fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:14.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.271+0000 7faf69795640 1 -- 172.21.15.83:0/1384180581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf64166550 msgr2=0x7faf64156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:14.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.271+0000 7faf69795640 1 --2- 172.21.15.83:0/1384180581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf64166550 0x7faf64156d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:14.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.271+0000 7faf69795640 1 -- 172.21.15.83:0/1384180581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf640cd810 con 0x7faf640aecd0 2024-04-17T10:39:14.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.271+0000 7faf69795640 1 --2- 172.21.15.83:0/1384180581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf640aecd0 0x7faf641532e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7faf6000efc0 tx=0x7faf6000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:14.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.271+0000 7faf59ffb640 1 -- 172.21.15.83:0/1384180581 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf60018020 con 0x7faf640aecd0 2024-04-17T10:39:14.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.271+0000 7faf6b9f9640 1 -- 172.21.15.83:0/1384180581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf64159a20 con 0x7faf640aecd0 2024-04-17T10:39:14.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.271+0000 7faf59ffb640 1 -- 172.21.15.83:0/1384180581 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf60014480 con 0x7faf640aecd0 2024-04-17T10:39:14.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.271+0000 7faf59ffb640 1 -- 172.21.15.83:0/1384180581 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf60014850 con 0x7faf640aecd0 2024-04-17T10:39:14.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.271+0000 7faf6b9f9640 1 -- 172.21.15.83:0/1384180581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf640ccd30 con 0x7faf640aecd0 2024-04-17T10:39:14.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.271+0000 7faf6b9f9640 1 -- 172.21.15.83:0/1384180581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf64074f90 con 0x7faf640aecd0 2024-04-17T10:39:14.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.271+0000 7faf59ffb640 1 -- 172.21.15.83:0/1384180581 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faf6001b020 con 0x7faf640aecd0 2024-04-17T10:39:14.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.271+0000 7faf59ffb640 1 --2- 172.21.15.83:0/1384180581 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faf48066f60 0x7faf48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:14.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.271+0000 7faf68f94640 1 --2- 172.21.15.83:0/1384180581 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faf48066f60 0x7faf48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:14.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.271+0000 7faf59ffb640 1 -- 172.21.15.83:0/1384180581 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf60085bb0 con 0x7faf640aecd0 2024-04-17T10:39:14.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.271+0000 7faf68f94640 1 --2- 172.21.15.83:0/1384180581 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faf48066f60 0x7faf48069420 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7faf640ae220 tx=0x7faf54007cf0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:14.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.275+0000 7faf59ffb640 1 -- 172.21.15.83:0/1384180581 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faf60022070 con 0x7faf640aecd0 2024-04-17T10:39:14.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.439+0000 7faf6b9f9640 1 -- 172.21.15.83:0/1384180581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf6406b160 con 0x7faf640aecd0 2024-04-17T10:39:14.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.439+0000 7faf59ffb640 1 -- 172.21.15.83:0/1384180581 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16610 (secure 0 0 0) 0x7faf60010040 con 0x7faf640aecd0 2024-04-17T10:39:14.438 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:39:14.439 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":32,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:39:13.471670+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:39:11.330695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-17T10:39:12.438377+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:39:14.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.443+0000 7faf6b9f9640 1 -- 172.21.15.83:0/1384180581 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faf48066f60 msgr2=0x7faf48069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:14.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.443+0000 7faf6b9f9640 1 --2- 172.21.15.83:0/1384180581 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faf48066f60 0x7faf48069420 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7faf640ae220 tx=0x7faf54007cf0 comp rx=0 tx=0).stop 2024-04-17T10:39:14.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.443+0000 7faf6b9f9640 1 -- 172.21.15.83:0/1384180581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf640aecd0 msgr2=0x7faf641532e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:14.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.443+0000 7faf6b9f9640 1 --2- 172.21.15.83:0/1384180581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf640aecd0 0x7faf641532e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7faf6000efc0 tx=0x7faf6000c490 comp rx=0 tx=0).stop 2024-04-17T10:39:14.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.443+0000 7faf6b9f9640 1 -- 172.21.15.83:0/1384180581 shutdown_connections 2024-04-17T10:39:14.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.443+0000 7faf6b9f9640 1 --2- 172.21.15.83:0/1384180581 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faf48066f60 0x7faf48069420 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:14.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.443+0000 7faf6b9f9640 1 --2- 172.21.15.83:0/1384180581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf64166550 0x7faf64156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:14.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.443+0000 7faf6b9f9640 1 --2- 172.21.15.83:0/1384180581 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf640b3f80 0x7faf64158fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:14.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.443+0000 7faf6b9f9640 1 --2- 172.21.15.83:0/1384180581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf640aecd0 0x7faf641532e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:14.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.443+0000 7faf6b9f9640 1 -- 172.21.15.83:0/1384180581 >> 172.21.15.83:0/1384180581 conn(0x7faf6406abe0 msgr2=0x7faf640cfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:14.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.443+0000 7faf6b9f9640 1 -- 172.21.15.83:0/1384180581 shutdown_connections 2024-04-17T10:39:14.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.443+0000 7faf6b9f9640 1 -- 172.21.15.83:0/1384180581 wait complete. 2024-04-17T10:39:14.457 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:39:14.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.579+0000 7fe04cf86640 1 Processor -- start 2024-04-17T10:39:14.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.579+0000 7fe04cf86640 1 -- start start 2024-04-17T10:39:14.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.579+0000 7fe04cf86640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe04815ba30 0x7fe048072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:14.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.579+0000 7fe04cf86640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe04806e320 con 0x7fe04815c960 2024-04-17T10:39:14.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.579+0000 7fe04cf86640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe04806e040 con 0x7fe0480cfbe0 2024-04-17T10:39:14.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.579+0000 7fe04cf86640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe048071ea0 con 0x7fe04815ba30 2024-04-17T10:39:14.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.579+0000 7fe04659c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe04815ba30 0x7fe048072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:14.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.579+0000 7fe03ffff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe04815c960 0x7fe0480cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48680/0 (socket says 172.21.15.83:48680) 2024-04-17T10:39:14.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.579+0000 7fe03ffff640 1 -- 172.21.15.83:0/3925626208 learned_addr learned my addr 172.21.15.83:0/3925626208 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:39:14.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.579+0000 7fe04659c640 1 -- 172.21.15.83:0/3925626208 >> v1:172.21.15.167:6789/0 conn(0x7fe0480cfbe0 legacy=0x7fe04806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:14.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.579+0000 7fe04659c640 1 -- 172.21.15.83:0/3925626208 >> v1:172.21.15.83:6789/0 conn(0x7fe04815c960 legacy=0x7fe0480cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:14.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.579+0000 7fe04659c640 1 -- 172.21.15.83:0/3925626208 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe048071850 con 0x7fe04815ba30 2024-04-17T10:39:14.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.583+0000 7fe04659c640 1 --2- 172.21.15.83:0/3925626208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe04815ba30 0x7fe048072330 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe0300097c0 tx=0x7fe03001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b1a884d6972174 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:14.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.583+0000 7fe045d9b640 1 -- 172.21.15.83:0/3925626208 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0300b0070 con 0x7fe04815ba30 2024-04-17T10:39:14.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.583+0000 7fe045d9b640 1 -- 172.21.15.83:0/3925626208 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0300ab600 con 0x7fe04815ba30 2024-04-17T10:39:14.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.583+0000 7fe045d9b640 1 -- 172.21.15.83:0/3925626208 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0300ab9d0 con 0x7fe04815ba30 2024-04-17T10:39:14.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.583+0000 7fe04cf86640 1 -- 172.21.15.83:0/3925626208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe04815ba30 msgr2=0x7fe048072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:14.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.583+0000 7fe04cf86640 1 --2- 172.21.15.83:0/3925626208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe04815ba30 0x7fe048072330 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe0300097c0 tx=0x7fe03001c3d0 comp rx=0 tx=0).stop 2024-04-17T10:39:14.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.583+0000 7fe04cf86640 1 -- 172.21.15.83:0/3925626208 shutdown_connections 2024-04-17T10:39:14.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.583+0000 7fe04cf86640 1 --2- 172.21.15.83:0/3925626208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe04815ba30 0x7fe048072330 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:14.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.583+0000 7fe04cf86640 1 -- 172.21.15.83:0/3925626208 >> 172.21.15.83:0/3925626208 conn(0x7fe04806abe0 msgr2=0x7fe04806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:14.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.583+0000 7fe04cf86640 1 -- 172.21.15.83:0/3925626208 shutdown_connections 2024-04-17T10:39:14.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.583+0000 7fe04cf86640 1 -- 172.21.15.83:0/3925626208 wait complete. 2024-04-17T10:39:14.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.583+0000 7fe04cf86640 1 Processor -- start 2024-04-17T10:39:14.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.583+0000 7fe04cf86640 1 -- start start 2024-04-17T10:39:14.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.583+0000 7fe04cf86640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0480cfbe0 0x7fe0480adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:14.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.583+0000 7fe04cf86640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe04815ba30 0x7fe04814ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:14.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.583+0000 7fe04cf86640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe04815c960 0x7fe0480cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:14.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.583+0000 7fe04cf86640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04815cee0 con 0x7fe04815ba30 2024-04-17T10:39:14.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.583+0000 7fe04cf86640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04815cd60 con 0x7fe04815c960 2024-04-17T10:39:14.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.583+0000 7fe04cf86640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe048169530 con 0x7fe0480cfbe0 2024-04-17T10:39:14.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.583+0000 7fe03ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe04815ba30 0x7fe04814ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:14.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.583+0000 7fe03ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe04815ba30 0x7fe04814ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37134/0 (socket says 172.21.15.83:37134) 2024-04-17T10:39:14.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.583+0000 7fe03ffff640 1 -- 172.21.15.83:0/862102978 learned_addr learned my addr 172.21.15.83:0/862102978 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:14.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.587+0000 7fe046d9d640 1 --2- 172.21.15.83:0/862102978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe04815c960 0x7fe0480cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:14.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.587+0000 7fe03ffff640 1 -- 172.21.15.83:0/862102978 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0480cfbe0 msgr2=0x7fe0480adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:14.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.587+0000 7fe04659c640 1 --2- 172.21.15.83:0/862102978 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0480cfbe0 0x7fe0480adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:14.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.587+0000 7fe03ffff640 1 --2- 172.21.15.83:0/862102978 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0480cfbe0 0x7fe0480adde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:14.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.587+0000 7fe03ffff640 1 -- 172.21.15.83:0/862102978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe04815c960 msgr2=0x7fe0480cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:14.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.587+0000 7fe03ffff640 1 --2- 172.21.15.83:0/862102978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe04815c960 0x7fe0480cd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:14.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.587+0000 7fe03ffff640 1 -- 172.21.15.83:0/862102978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe048152410 con 0x7fe04815ba30 2024-04-17T10:39:14.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.587+0000 7fe03ffff640 1 --2- 172.21.15.83:0/862102978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe04815ba30 0x7fe04814ddf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe03800c370 tx=0x7fe03800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:14.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.587+0000 7fe03effd640 1 -- 172.21.15.83:0/862102978 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe038017070 con 0x7fe04815ba30 2024-04-17T10:39:14.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.587+0000 7fe03effd640 1 -- 172.21.15.83:0/862102978 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe038005cf0 con 0x7fe04815ba30 2024-04-17T10:39:14.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.587+0000 7fe04cf86640 1 -- 172.21.15.83:0/862102978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0480a97a0 con 0x7fe04815ba30 2024-04-17T10:39:14.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.587+0000 7fe03effd640 1 -- 172.21.15.83:0/862102978 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0380072f0 con 0x7fe04815ba30 2024-04-17T10:39:14.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.587+0000 7fe04cf86640 1 -- 172.21.15.83:0/862102978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0481526d0 con 0x7fe04815ba30 2024-04-17T10:39:14.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.587+0000 7fe04cf86640 1 -- 172.21.15.83:0/862102978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe048152b10 con 0x7fe04815ba30 2024-04-17T10:39:14.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.587+0000 7fe03effd640 1 -- 172.21.15.83:0/862102978 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe03801a070 con 0x7fe04815ba30 2024-04-17T10:39:14.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.587+0000 7fe03effd640 1 --2- 172.21.15.83:0/862102978 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe028066f60 0x7fe028069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:14.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.587+0000 7fe03effd640 1 -- 172.21.15.83:0/862102978 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0380842f0 con 0x7fe04815ba30 2024-04-17T10:39:14.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.587+0000 7fe04659c640 1 --2- 172.21.15.83:0/862102978 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe028066f60 0x7fe028069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:14.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.587+0000 7fe04659c640 1 --2- 172.21.15.83:0/862102978 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe028066f60 0x7fe028069420 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe0300ad000 tx=0x7fe030008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:14.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.595+0000 7fe03effd640 1 -- 172.21.15.83:0/862102978 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe038055390 con 0x7fe04815ba30 2024-04-17T10:39:14.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.755+0000 7fe04cf86640 1 -- 172.21.15.83:0/862102978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe048074480 con 0x7fe04815ba30 2024-04-17T10:39:14.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.755+0000 7fe03effd640 1 -- 172.21.15.83:0/862102978 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16610 (secure 0 0 0) 0x7fe038054c90 con 0x7fe04815ba30 2024-04-17T10:39:14.753 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:39:14.753 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":32,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:39:13.471670+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:39:11.330695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-17T10:39:12.438377+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:39:14.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.759+0000 7fe04cf86640 1 -- 172.21.15.83:0/862102978 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe028066f60 msgr2=0x7fe028069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:14.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.759+0000 7fe04cf86640 1 --2- 172.21.15.83:0/862102978 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe028066f60 0x7fe028069420 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe0300ad000 tx=0x7fe030008040 comp rx=0 tx=0).stop 2024-04-17T10:39:14.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.759+0000 7fe04cf86640 1 -- 172.21.15.83:0/862102978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe04815ba30 msgr2=0x7fe04814ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:14.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.759+0000 7fe04cf86640 1 --2- 172.21.15.83:0/862102978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe04815ba30 0x7fe04814ddf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe03800c370 tx=0x7fe03800c830 comp rx=0 tx=0).stop 2024-04-17T10:39:14.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.759+0000 7fe04cf86640 1 -- 172.21.15.83:0/862102978 shutdown_connections 2024-04-17T10:39:14.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.759+0000 7fe04cf86640 1 --2- 172.21.15.83:0/862102978 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe028066f60 0x7fe028069420 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:14.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.759+0000 7fe04cf86640 1 --2- 172.21.15.83:0/862102978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe04815c960 0x7fe0480cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:14.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.759+0000 7fe04cf86640 1 --2- 172.21.15.83:0/862102978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe04815ba30 0x7fe04814ddf0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:14.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.759+0000 7fe04cf86640 1 --2- 172.21.15.83:0/862102978 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0480cfbe0 0x7fe0480adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:14.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.759+0000 7fe04cf86640 1 -- 172.21.15.83:0/862102978 >> 172.21.15.83:0/862102978 conn(0x7fe04806abe0 msgr2=0x7fe048150fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:14.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.759+0000 7fe04cf86640 1 -- 172.21.15.83:0/862102978 shutdown_connections 2024-04-17T10:39:14.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.759+0000 7fe04cf86640 1 -- 172.21.15.83:0/862102978 wait complete. 2024-04-17T10:39:14.772 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-17T10:39:14.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.895+0000 7fc13bf21640 1 Processor -- start 2024-04-17T10:39:14.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.895+0000 7fc13bf21640 1 -- start start 2024-04-17T10:39:14.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.899+0000 7fc13bf21640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc13406bb80 0x7fc13406c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:14.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.899+0000 7fc13bf21640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1340d2430 con 0x7fc134161940 2024-04-17T10:39:14.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.899+0000 7fc13bf21640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1340ced00 con 0x7fc13415ed70 2024-04-17T10:39:14.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.899+0000 7fc13bf21640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1340d2710 con 0x7fc13406bb80 2024-04-17T10:39:14.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.899+0000 7fc139cbd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc134161940 0x7fc1340cfb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48682/0 (socket says 172.21.15.83:48682) 2024-04-17T10:39:14.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.899+0000 7fc139cbd640 1 -- 172.21.15.83:0/3687163647 learned_addr learned my addr 172.21.15.83:0/3687163647 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:39:14.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.899+0000 7fc138cbb640 1 -- 172.21.15.83:0/3687163647 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2373871268 0 0) 0x7fc1340d2430 con 0x7fc134161940 2024-04-17T10:39:14.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.899+0000 7fc138cbb640 1 -- 172.21.15.83:0/3687163647 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc11c003530 con 0x7fc134161940 2024-04-17T10:39:14.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.899+0000 7fc13a4be640 1 --2- 172.21.15.83:0/3687163647 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc13406bb80 0x7fc13406c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:14.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.899+0000 7fc138cbb640 1 -- 172.21.15.83:0/3687163647 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1001016140 0 0) 0x7fc11c003530 con 0x7fc134161940 2024-04-17T10:39:14.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.899+0000 7fc138cbb640 1 -- 172.21.15.83:0/3687163647 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc13406bb80 msgr2=0x7fc13406c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:14.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.899+0000 7fc138cbb640 1 --2- 172.21.15.83:0/3687163647 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc13406bb80 0x7fc13406c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:14.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.899+0000 7fc138cbb640 1 -- 172.21.15.83:0/3687163647 >> v1:172.21.15.167:6789/0 conn(0x7fc13415ed70 legacy=0x7fc134161d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:14.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.899+0000 7fc138cbb640 1 -- 172.21.15.83:0/3687163647 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1340cefe0 con 0x7fc134161940 2024-04-17T10:39:14.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.899+0000 7fc138cbb640 1 -- 172.21.15.83:0/3687163647 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fc12c001680 con 0x7fc134161940 2024-04-17T10:39:14.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.899+0000 7fc138cbb640 1 -- 172.21.15.83:0/3687163647 >> v1:172.21.15.83:6789/0 conn(0x7fc134161940 legacy=0x7fc1340cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:14.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.899+0000 7fc138cbb640 1 --2- 172.21.15.83:0/3687163647 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc13406bb80 0x7fc11c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:14.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.899+0000 7fc138cbb640 1 --2- 172.21.15.83:0/3687163647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc11c003fe0 0x7fc11c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:14.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.899+0000 7fc138cbb640 1 --2- 172.21.15.83:0/3687163647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc11c004d50 0x7fc11c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:14.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.899+0000 7fc138cbb640 1 -- 172.21.15.83:0/3687163647 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1340d2710 con 0x7fc11c003fe0 2024-04-17T10:39:14.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.899+0000 7fc138cbb640 1 -- 172.21.15.83:0/3687163647 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c00bbc0 con 0x7fc11c004d50 2024-04-17T10:39:14.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.899+0000 7fc138cbb640 1 -- 172.21.15.83:0/3687163647 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c00bdc0 con 0x7fc13406bb80 2024-04-17T10:39:14.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.899+0000 7fc1394bc640 1 --2- 172.21.15.83:0/3687163647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc11c003fe0 0x7fc11c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:14.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.899+0000 7fc13a4be640 1 --2- 172.21.15.83:0/3687163647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc11c004d50 0x7fc11c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:14.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.899+0000 7fc139cbd640 1 --2- 172.21.15.83:0/3687163647 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc13406bb80 0x7fc11c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:14.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.899+0000 7fc1394bc640 1 -- 172.21.15.83:0/3687163647 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc13406bb80 msgr2=0x7fc11c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:14.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.899+0000 7fc1394bc640 1 --2- 172.21.15.83:0/3687163647 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc13406bb80 0x7fc11c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:14.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.899+0000 7fc1394bc640 1 -- 172.21.15.83:0/3687163647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc11c004d50 msgr2=0x7fc11c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:14.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.899+0000 7fc1394bc640 1 --2- 172.21.15.83:0/3687163647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc11c004d50 0x7fc11c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:14.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.899+0000 7fc1394bc640 1 -- 172.21.15.83:0/3687163647 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11c003530 con 0x7fc11c003fe0 2024-04-17T10:39:14.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.903+0000 7fc1394bc640 1 --2- 172.21.15.83:0/3687163647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc11c003fe0 0x7fc11c004810 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc128001750 tx=0x7fc12801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8148f7b1b817a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:14.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.903+0000 7fc138cbb640 1 -- 172.21.15.83:0/3687163647 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1280013a0 con 0x7fc11c003fe0 2024-04-17T10:39:14.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.903+0000 7fc138cbb640 1 -- 172.21.15.83:0/3687163647 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1280052f0 con 0x7fc11c003fe0 2024-04-17T10:39:14.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.903+0000 7fc138cbb640 1 -- 172.21.15.83:0/3687163647 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc128003840 con 0x7fc11c003fe0 2024-04-17T10:39:14.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.903+0000 7fc13bf21640 1 -- 172.21.15.83:0/3687163647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc11c003fe0 msgr2=0x7fc11c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:14.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.903+0000 7fc13bf21640 1 --2- 172.21.15.83:0/3687163647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc11c003fe0 0x7fc11c004810 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc128001750 tx=0x7fc12801c3e0 comp rx=0 tx=0).stop 2024-04-17T10:39:14.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.903+0000 7fc13a4be640 1 -- 172.21.15.83:0/3687163647 reap_dead start 2024-04-17T10:39:14.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.903+0000 7fc13bf21640 1 -- 172.21.15.83:0/3687163647 shutdown_connections 2024-04-17T10:39:14.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.903+0000 7fc13bf21640 1 -- 172.21.15.83:0/3687163647 >> 172.21.15.83:0/3687163647 conn(0x7fc1340d17e0 msgr2=0x7fc1340d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:14.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.903+0000 7fc13bf21640 1 -- 172.21.15.83:0/3687163647 shutdown_connections 2024-04-17T10:39:14.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.903+0000 7fc13bf21640 1 -- 172.21.15.83:0/3687163647 wait complete. 2024-04-17T10:39:14.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.903+0000 7fc13bf21640 1 Processor -- start 2024-04-17T10:39:14.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.903+0000 7fc13bf21640 1 -- start start 2024-04-17T10:39:14.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.903+0000 7fc13bf21640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc11c003fe0 0x7fc13414e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:14.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.903+0000 7fc13bf21640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1340b1c00 0x7fc13406faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:14.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.903+0000 7fc13bf21640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1340c7a70 0x7fc134070030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:14.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.903+0000 7fc13bf21640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13406d130 con 0x7fc1340c7a70 2024-04-17T10:39:14.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.903+0000 7fc13bf21640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13406cfb0 con 0x7fc11c003fe0 2024-04-17T10:39:14.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.903+0000 7fc13bf21640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc134169a20 con 0x7fc1340b1c00 2024-04-17T10:39:14.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.903+0000 7fc13a4be640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1340c7a70 0x7fc134070030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:14.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.903+0000 7fc13a4be640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1340c7a70 0x7fc134070030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37156/0 (socket says 172.21.15.83:37156) 2024-04-17T10:39:14.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.903+0000 7fc13a4be640 1 -- 172.21.15.83:0/307452943 learned_addr learned my addr 172.21.15.83:0/307452943 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:14.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.903+0000 7fc139cbd640 1 --2- 172.21.15.83:0/307452943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc11c003fe0 0x7fc13414e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:14.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.903+0000 7fc1394bc640 1 --2- 172.21.15.83:0/307452943 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1340b1c00 0x7fc13406faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:14.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.903+0000 7fc13a4be640 1 -- 172.21.15.83:0/307452943 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1340b1c00 msgr2=0x7fc13406faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:14.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.903+0000 7fc13a4be640 1 --2- 172.21.15.83:0/307452943 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1340b1c00 0x7fc13406faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:14.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.903+0000 7fc13a4be640 1 -- 172.21.15.83:0/307452943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc11c003fe0 msgr2=0x7fc13414e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:14.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.903+0000 7fc13a4be640 1 --2- 172.21.15.83:0/307452943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc11c003fe0 0x7fc13414e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:14.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.903+0000 7fc13a4be640 1 -- 172.21.15.83:0/307452943 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1340b21a0 con 0x7fc1340c7a70 2024-04-17T10:39:14.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.903+0000 7fc1394bc640 1 --2- 172.21.15.83:0/307452943 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1340b1c00 0x7fc13406faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:14.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.903+0000 7fc13a4be640 1 --2- 172.21.15.83:0/307452943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1340c7a70 0x7fc134070030 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc134071840 tx=0x7fc1240115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:14.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.903+0000 7fc1327fc640 1 -- 172.21.15.83:0/307452943 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc124011e10 con 0x7fc1340c7a70 2024-04-17T10:39:14.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.907+0000 7fc1327fc640 1 -- 172.21.15.83:0/307452943 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc12401b070 con 0x7fc1340c7a70 2024-04-17T10:39:14.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.907+0000 7fc13bf21640 1 -- 172.21.15.83:0/307452943 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1340b2480 con 0x7fc1340c7a70 2024-04-17T10:39:14.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.907+0000 7fc1327fc640 1 -- 172.21.15.83:0/307452943 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1240039a0 con 0x7fc1340c7a70 2024-04-17T10:39:14.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.907+0000 7fc13bf21640 1 -- 172.21.15.83:0/307452943 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1340b5980 con 0x7fc1340c7a70 2024-04-17T10:39:14.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.907+0000 7fc13bf21640 1 -- 172.21.15.83:0/307452943 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1340b5b80 con 0x7fc1340c7a70 2024-04-17T10:39:14.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.907+0000 7fc1327fc640 1 -- 172.21.15.83:0/307452943 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc124003b60 con 0x7fc1340c7a70 2024-04-17T10:39:14.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.907+0000 7fc1327fc640 1 --2- 172.21.15.83:0/307452943 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc120066f90 0x7fc120069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:14.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.907+0000 7fc1327fc640 1 -- 172.21.15.83:0/307452943 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc124085860 con 0x7fc1340c7a70 2024-04-17T10:39:14.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.907+0000 7fc139cbd640 1 --2- 172.21.15.83:0/307452943 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc120066f90 0x7fc120069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:14.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.907+0000 7fc139cbd640 1 --2- 172.21.15.83:0/307452943 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc120066f90 0x7fc120069450 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc12c001060 tx=0x7fc12c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:14.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:14.911+0000 7fc1327fc640 1 -- 172.21.15.83:0/307452943 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc1340b5b80 con 0x7fc1340c7a70 2024-04-17T10:39:15.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.103+0000 7fc13bf21640 1 -- 172.21.15.83:0/307452943 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc134074f90 con 0x7fc1340c7a70 2024-04-17T10:39:15.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.163+0000 7fc1327fc640 1 -- 172.21.15.83:0/307452943 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v12) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc124085020 con 0x7fc1340c7a70 2024-04-17T10:39:15.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.167+0000 7fc13bf21640 1 -- 172.21.15.83:0/307452943 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc120066f90 msgr2=0x7fc120069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:15.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.167+0000 7fc13bf21640 1 --2- 172.21.15.83:0/307452943 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc120066f90 0x7fc120069450 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc12c001060 tx=0x7fc12c009040 comp rx=0 tx=0).stop 2024-04-17T10:39:15.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.167+0000 7fc13bf21640 1 -- 172.21.15.83:0/307452943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1340c7a70 msgr2=0x7fc134070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:15.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.167+0000 7fc13bf21640 1 --2- 172.21.15.83:0/307452943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1340c7a70 0x7fc134070030 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc134071840 tx=0x7fc1240115a0 comp rx=0 tx=0).stop 2024-04-17T10:39:15.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.167+0000 7fc13bf21640 1 -- 172.21.15.83:0/307452943 shutdown_connections 2024-04-17T10:39:15.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.167+0000 7fc13bf21640 1 --2- 172.21.15.83:0/307452943 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc120066f90 0x7fc120069450 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:15.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.167+0000 7fc13bf21640 1 --2- 172.21.15.83:0/307452943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1340c7a70 0x7fc134070030 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:15.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.167+0000 7fc13bf21640 1 --2- 172.21.15.83:0/307452943 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1340b1c00 0x7fc13406faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:15.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.167+0000 7fc13bf21640 1 --2- 172.21.15.83:0/307452943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc11c003fe0 0x7fc13414e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:15.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.167+0000 7fc13bf21640 1 -- 172.21.15.83:0/307452943 >> 172.21.15.83:0/307452943 conn(0x7fc1340d17e0 msgr2=0x7fc13415fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:15.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.167+0000 7fc13bf21640 1 -- 172.21.15.83:0/307452943 shutdown_connections 2024-04-17T10:39:15.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.167+0000 7fc13bf21640 1 -- 172.21.15.83:0/307452943 wait complete. 2024-04-17T10:39:15.180 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-17T10:39:15.180 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-17T10:39:15.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.307+0000 7f9238ce0640 1 Processor -- start 2024-04-17T10:39:15.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.307+0000 7f9238ce0640 1 -- start start 2024-04-17T10:39:15.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.307+0000 7f9238ce0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9234154670 0x7f92340aea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:15.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.307+0000 7f9238ce0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f92340b4090 0x7f923406c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:15.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.307+0000 7f9238ce0640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92340b2ff0 con 0x7f92340b44f0 2024-04-17T10:39:15.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.307+0000 7f9238ce0640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9234154a50 con 0x7f9234154670 2024-04-17T10:39:15.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.307+0000 7f9238ce0640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92340b32d0 con 0x7f92340b4090 2024-04-17T10:39:15.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.307+0000 7f923259c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9234154670 0x7f92340aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:15.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.307+0000 7f9232d9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f92340b44f0 0x7f9234070470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48698/0 (socket says 172.21.15.83:48698) 2024-04-17T10:39:15.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.307+0000 7f9232d9d640 1 -- 172.21.15.83:0/3071074969 learned_addr learned my addr 172.21.15.83:0/3071074969 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:39:15.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.307+0000 7f9231d9b640 1 --2- 172.21.15.83:0/3071074969 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f92340b4090 0x7f923406c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:15.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.307+0000 7f923259c640 1 -- 172.21.15.83:0/3071074969 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f92340b4090 msgr2=0x7f923406c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:15.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.307+0000 7f923259c640 1 --2- 172.21.15.83:0/3071074969 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f92340b4090 0x7f923406c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:15.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.307+0000 7f923259c640 1 -- 172.21.15.83:0/3071074969 >> v1:172.21.15.83:6789/0 conn(0x7f92340b44f0 legacy=0x7f9234070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:15.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.307+0000 7f923259c640 1 -- 172.21.15.83:0/3071074969 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92340b35b0 con 0x7f9234154670 2024-04-17T10:39:15.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.307+0000 7f923159a640 1 -- 172.21.15.83:0/3071074969 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3266502691 0 0) 0x7f92340b2ff0 con 0x7f92340b44f0 2024-04-17T10:39:15.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.307+0000 7f9231d9b640 1 --2- 172.21.15.83:0/3071074969 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f92340b4090 0x7f923406c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:39:15.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.311+0000 7f923259c640 1 --2- 172.21.15.83:0/3071074969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9234154670 0x7f92340aea50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f92240097c0 tx=0x7f922401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53dca8a5a816ccf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:15.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.311+0000 7f923159a640 1 -- 172.21.15.83:0/3071074969 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92240a7070 con 0x7f9234154670 2024-04-17T10:39:15.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.311+0000 7f923159a640 1 -- 172.21.15.83:0/3071074969 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92240b0070 con 0x7f9234154670 2024-04-17T10:39:15.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.311+0000 7f923159a640 1 -- 172.21.15.83:0/3071074969 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92240ab890 con 0x7f9234154670 2024-04-17T10:39:15.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.311+0000 7f9238ce0640 1 -- 172.21.15.83:0/3071074969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9234154670 msgr2=0x7f92340aea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:15.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.311+0000 7f9238ce0640 1 --2- 172.21.15.83:0/3071074969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9234154670 0x7f92340aea50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f92240097c0 tx=0x7f922401c470 comp rx=0 tx=0).stop 2024-04-17T10:39:15.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.311+0000 7f9238ce0640 1 -- 172.21.15.83:0/3071074969 shutdown_connections 2024-04-17T10:39:15.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.311+0000 7f9238ce0640 1 --2- 172.21.15.83:0/3071074969 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f92340b4090 0x7f923406c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:15.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.311+0000 7f9238ce0640 1 --2- 172.21.15.83:0/3071074969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9234154670 0x7f92340aea50 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:15.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.311+0000 7f9238ce0640 1 -- 172.21.15.83:0/3071074969 >> 172.21.15.83:0/3071074969 conn(0x7f92340b15b0 msgr2=0x7f92340b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:15.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.311+0000 7f9238ce0640 1 -- 172.21.15.83:0/3071074969 shutdown_connections 2024-04-17T10:39:15.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.311+0000 7f9238ce0640 1 -- 172.21.15.83:0/3071074969 wait complete. 2024-04-17T10:39:15.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.311+0000 7f9238ce0640 1 Processor -- start 2024-04-17T10:39:15.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.311+0000 7f9238ce0640 1 -- start start 2024-04-17T10:39:15.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.311+0000 7f9238ce0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92340b4090 0x7f923414a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:15.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.311+0000 7f9238ce0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92340b44f0 0x7f923414b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:15.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.311+0000 7f9238ce0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9234154670 0x7f923414b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:15.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.311+0000 7f9238ce0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92340aef90 con 0x7f92340b44f0 2024-04-17T10:39:15.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.311+0000 7f9238ce0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923406cec0 con 0x7f92340b4090 2024-04-17T10:39:15.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.311+0000 7f9238ce0640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92340af110 con 0x7f9234154670 2024-04-17T10:39:15.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.311+0000 7f9231d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92340b44f0 0x7f923414b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:15.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.311+0000 7f9231d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92340b44f0 0x7f923414b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37168/0 (socket says 172.21.15.83:37168) 2024-04-17T10:39:15.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.311+0000 7f9231d9b640 1 -- 172.21.15.83:0/364064444 learned_addr learned my addr 172.21.15.83:0/364064444 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:15.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.311+0000 7f923259c640 1 --2- 172.21.15.83:0/364064444 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92340b4090 0x7f923414a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:15.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.311+0000 7f9231d9b640 1 -- 172.21.15.83:0/364064444 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9234154670 msgr2=0x7f923414b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:15.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.311+0000 7f9231d9b640 1 --2- 172.21.15.83:0/364064444 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9234154670 0x7f923414b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:15.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.311+0000 7f9231d9b640 1 -- 172.21.15.83:0/364064444 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92340b4090 msgr2=0x7f923414a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:15.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.311+0000 7f9231d9b640 1 --2- 172.21.15.83:0/364064444 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92340b4090 0x7f923414a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:15.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.311+0000 7f9231d9b640 1 -- 172.21.15.83:0/364064444 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9234147db0 con 0x7f92340b44f0 2024-04-17T10:39:15.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.311+0000 7f9231d9b640 1 --2- 172.21.15.83:0/364064444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92340b44f0 0x7f923414b020 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f922800d830 tx=0x7f922800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:15.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.311+0000 7f9222ffd640 1 -- 172.21.15.83:0/364064444 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9228018020 con 0x7f92340b44f0 2024-04-17T10:39:15.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.315+0000 7f9222ffd640 1 -- 172.21.15.83:0/364064444 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92280096c0 con 0x7f92340b44f0 2024-04-17T10:39:15.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.315+0000 7f9222ffd640 1 -- 172.21.15.83:0/364064444 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9228009ab0 con 0x7f92340b44f0 2024-04-17T10:39:15.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.315+0000 7f9238ce0640 1 -- 172.21.15.83:0/364064444 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92341480f0 con 0x7f92340b44f0 2024-04-17T10:39:15.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.315+0000 7f9238ce0640 1 -- 172.21.15.83:0/364064444 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9234148930 con 0x7f92340b44f0 2024-04-17T10:39:15.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.315+0000 7f9238ce0640 1 -- 172.21.15.83:0/364064444 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9234074f90 con 0x7f92340b44f0 2024-04-17T10:39:15.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.315+0000 7f9222ffd640 1 -- 172.21.15.83:0/364064444 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f922801b020 con 0x7f92340b44f0 2024-04-17T10:39:15.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.315+0000 7f9222ffd640 1 --2- 172.21.15.83:0/364064444 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9214066f60 0x7f9214069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:15.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.315+0000 7f9222ffd640 1 -- 172.21.15.83:0/364064444 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f92280857e0 con 0x7f92340b44f0 2024-04-17T10:39:15.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.315+0000 7f923259c640 1 --2- 172.21.15.83:0/364064444 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9214066f60 0x7f9214069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:15.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.315+0000 7f923259c640 1 --2- 172.21.15.83:0/364064444 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9214066f60 0x7f9214069420 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f92240af000 tx=0x7f9224008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:15.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.319+0000 7f9222ffd640 1 -- 172.21.15.83:0/364064444 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9228022020 con 0x7f92340b44f0 2024-04-17T10:39:15.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:15.511+0000 7f9238ce0640 1 -- 172.21.15.83:0/364064444 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f92340b2a00 con 0x7f92340b44f0 2024-04-17T10:39:16.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.170+0000 7f9222ffd640 1 -- 172.21.15.83:0/364064444 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v13) v1 ==== 122+0+0 (secure 0 0 0) 0x7f92341480f0 con 0x7f92340b44f0 2024-04-17T10:39:16.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.174+0000 7f9238ce0640 1 -- 172.21.15.83:0/364064444 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9214066f60 msgr2=0x7f9214069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:16.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.174+0000 7f9238ce0640 1 --2- 172.21.15.83:0/364064444 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9214066f60 0x7f9214069420 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f92240af000 tx=0x7f9224008040 comp rx=0 tx=0).stop 2024-04-17T10:39:16.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.174+0000 7f9238ce0640 1 -- 172.21.15.83:0/364064444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92340b44f0 msgr2=0x7f923414b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:16.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.174+0000 7f9238ce0640 1 --2- 172.21.15.83:0/364064444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92340b44f0 0x7f923414b020 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f922800d830 tx=0x7f922800dcf0 comp rx=0 tx=0).stop 2024-04-17T10:39:16.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.174+0000 7f9238ce0640 1 -- 172.21.15.83:0/364064444 shutdown_connections 2024-04-17T10:39:16.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.174+0000 7f9238ce0640 1 --2- 172.21.15.83:0/364064444 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9214066f60 0x7f9214069420 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:16.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.174+0000 7f9238ce0640 1 --2- 172.21.15.83:0/364064444 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9234154670 0x7f923414b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:16.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.174+0000 7f9238ce0640 1 --2- 172.21.15.83:0/364064444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92340b44f0 0x7f923414b020 secure :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f922800d830 tx=0x7f922800dcf0 comp rx=0 tx=0).stop 2024-04-17T10:39:16.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.174+0000 7f9238ce0640 1 --2- 172.21.15.83:0/364064444 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92340b4090 0x7f923414a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:16.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.174+0000 7f9238ce0640 1 -- 172.21.15.83:0/364064444 >> 172.21.15.83:0/364064444 conn(0x7f92340b15b0 msgr2=0x7f92340b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:16.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.174+0000 7f9238ce0640 1 -- 172.21.15.83:0/364064444 shutdown_connections 2024-04-17T10:39:16.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.174+0000 7f9238ce0640 1 -- 172.21.15.83:0/364064444 wait complete. 2024-04-17T10:39:16.191 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-17T10:39:16.192 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:39:16.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.314+0000 7f14c09e1640 1 Processor -- start 2024-04-17T10:39:16.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.314+0000 7f14c09e1640 1 -- start start 2024-04-17T10:39:16.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.318+0000 7f14c09e1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14bc15ba30 0x7f14bc072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:16.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.318+0000 7f14c09e1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f14bc0cfbe0 0x7f14bc06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:16.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.318+0000 7f14c09e1640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14bc06e320 con 0x7f14bc15c960 2024-04-17T10:39:16.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.318+0000 7f14c09e1640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14bc071ea0 con 0x7f14bc0cfbe0 2024-04-17T10:39:16.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.318+0000 7f14c09e1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14bc06e040 con 0x7f14bc15ba30 2024-04-17T10:39:16.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.318+0000 7f14bad9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f14bc0cfbe0 0x7f14bc06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:16.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.318+0000 7f14b9d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f14bc15c960 0x7f14bc0cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48704/0 (socket says 172.21.15.83:48704) 2024-04-17T10:39:16.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.318+0000 7f14b9d9b640 1 -- 172.21.15.83:0/1577912821 learned_addr learned my addr 172.21.15.83:0/1577912821 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:39:16.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.318+0000 7f14ba59c640 1 --2- 172.21.15.83:0/1577912821 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14bc15ba30 0x7f14bc072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:16.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.318+0000 7f14bad9d640 1 -- 172.21.15.83:0/1577912821 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14bc15ba30 msgr2=0x7f14bc072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:16.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.318+0000 7f14bad9d640 1 --2- 172.21.15.83:0/1577912821 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14bc15ba30 0x7f14bc072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:16.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.318+0000 7f14b959a640 1 -- 172.21.15.83:0/1577912821 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 334148247 0 0) 0x7f14bc06e320 con 0x7f14bc15c960 2024-04-17T10:39:16.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.318+0000 7f14bad9d640 1 -- 172.21.15.83:0/1577912821 >> v1:172.21.15.83:6789/0 conn(0x7f14bc15c960 legacy=0x7f14bc0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:16.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.318+0000 7f14bad9d640 1 -- 172.21.15.83:0/1577912821 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14bc071850 con 0x7f14bc0cfbe0 2024-04-17T10:39:16.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.318+0000 7f14ba59c640 1 --2- 172.21.15.83:0/1577912821 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14bc15ba30 0x7f14bc072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:16.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.318+0000 7f14bad9d640 1 --2- 172.21.15.83:0/1577912821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14bc0cfbe0 0x7f14bc06c490 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f14b4009d90 tx=0x7f14b401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de25d86d94d7b1b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:16.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.318+0000 7f14b959a640 1 -- 172.21.15.83:0/1577912821 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b40b2070 con 0x7f14bc0cfbe0 2024-04-17T10:39:16.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.318+0000 7f14b959a640 1 -- 172.21.15.83:0/1577912821 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b40a57f0 con 0x7f14bc0cfbe0 2024-04-17T10:39:16.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.318+0000 7f14b959a640 1 -- 172.21.15.83:0/1577912821 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b40a5bc0 con 0x7f14bc0cfbe0 2024-04-17T10:39:16.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.318+0000 7f14c09e1640 1 -- 172.21.15.83:0/1577912821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14bc0cfbe0 msgr2=0x7f14bc06c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:16.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.318+0000 7f14c09e1640 1 --2- 172.21.15.83:0/1577912821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14bc0cfbe0 0x7f14bc06c490 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f14b4009d90 tx=0x7f14b401c8e0 comp rx=0 tx=0).stop 2024-04-17T10:39:16.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.322+0000 7f14c09e1640 1 -- 172.21.15.83:0/1577912821 shutdown_connections 2024-04-17T10:39:16.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.322+0000 7f14c09e1640 1 --2- 172.21.15.83:0/1577912821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14bc0cfbe0 0x7f14bc06c490 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:16.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.322+0000 7f14c09e1640 1 --2- 172.21.15.83:0/1577912821 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14bc15ba30 0x7f14bc072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:16.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.322+0000 7f14c09e1640 1 -- 172.21.15.83:0/1577912821 >> 172.21.15.83:0/1577912821 conn(0x7f14bc06abe0 msgr2=0x7f14bc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:16.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.322+0000 7f14c09e1640 1 -- 172.21.15.83:0/1577912821 shutdown_connections 2024-04-17T10:39:16.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.322+0000 7f14c09e1640 1 -- 172.21.15.83:0/1577912821 wait complete. 2024-04-17T10:39:16.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.322+0000 7f14c09e1640 1 Processor -- start 2024-04-17T10:39:16.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.322+0000 7f14c09e1640 1 -- start start 2024-04-17T10:39:16.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.322+0000 7f14c09e1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14bc0cfbe0 0x7f14bc137130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:16.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.322+0000 7f14c09e1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14bc15ba30 0x7f14bc14b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:16.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.322+0000 7f14c09e1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14bc15c960 0x7f14bc14bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:16.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.322+0000 7f14c09e1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14bc06de20 con 0x7f14bc15ba30 2024-04-17T10:39:16.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.322+0000 7f14c09e1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14bc06dca0 con 0x7f14bc0cfbe0 2024-04-17T10:39:16.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.322+0000 7f14c09e1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14bc168520 con 0x7f14bc15c960 2024-04-17T10:39:16.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.322+0000 7f14b9d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14bc15ba30 0x7f14bc14b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:16.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.322+0000 7f14b9d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14bc15ba30 0x7f14bc14b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37192/0 (socket says 172.21.15.83:37192) 2024-04-17T10:39:16.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.322+0000 7f14b9d9b640 1 -- 172.21.15.83:0/2567789689 learned_addr learned my addr 172.21.15.83:0/2567789689 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:16.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.322+0000 7f14ba59c640 1 --2- 172.21.15.83:0/2567789689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14bc0cfbe0 0x7f14bc137130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:16.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.322+0000 7f14bad9d640 1 --2- 172.21.15.83:0/2567789689 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14bc15c960 0x7f14bc14bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:16.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.322+0000 7f14b9d9b640 1 -- 172.21.15.83:0/2567789689 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14bc15c960 msgr2=0x7f14bc14bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:16.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.322+0000 7f14b9d9b640 1 --2- 172.21.15.83:0/2567789689 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14bc15c960 0x7f14bc14bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:16.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.322+0000 7f14b9d9b640 1 -- 172.21.15.83:0/2567789689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14bc0cfbe0 msgr2=0x7f14bc137130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:16.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.322+0000 7f14b9d9b640 1 --2- 172.21.15.83:0/2567789689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14bc0cfbe0 0x7f14bc137130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:16.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.322+0000 7f14b9d9b640 1 -- 172.21.15.83:0/2567789689 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14bc0726a0 con 0x7f14bc15ba30 2024-04-17T10:39:16.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.322+0000 7f14bad9d640 1 --2- 172.21.15.83:0/2567789689 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14bc15c960 0x7f14bc14bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:16.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.322+0000 7f14b9d9b640 1 --2- 172.21.15.83:0/2567789689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14bc15ba30 0x7f14bc14b9e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f14b0007fd0 tx=0x7f14b000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:16.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.322+0000 7f14aaffd640 1 -- 172.21.15.83:0/2567789689 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b0017070 con 0x7f14bc15ba30 2024-04-17T10:39:16.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.322+0000 7f14aaffd640 1 -- 172.21.15.83:0/2567789689 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b0005d10 con 0x7f14bc15ba30 2024-04-17T10:39:16.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.322+0000 7f14c09e1640 1 -- 172.21.15.83:0/2567789689 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14bc137c20 con 0x7f14bc15ba30 2024-04-17T10:39:16.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.322+0000 7f14aaffd640 1 -- 172.21.15.83:0/2567789689 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b0007310 con 0x7f14bc15ba30 2024-04-17T10:39:16.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.322+0000 7f14c09e1640 1 -- 172.21.15.83:0/2567789689 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14bc1381e0 con 0x7f14bc15ba30 2024-04-17T10:39:16.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.326+0000 7f14c09e1640 1 -- 172.21.15.83:0/2567789689 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14bc074f90 con 0x7f14bc15ba30 2024-04-17T10:39:16.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.326+0000 7f14aaffd640 1 -- 172.21.15.83:0/2567789689 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f14b001a070 con 0x7f14bc15ba30 2024-04-17T10:39:16.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.326+0000 7f14aaffd640 1 --2- 172.21.15.83:0/2567789689 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f149c066f90 0x7f149c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:16.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.326+0000 7f14ba59c640 1 --2- 172.21.15.83:0/2567789689 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f149c066f90 0x7f149c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:16.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.326+0000 7f14aaffd640 1 -- 172.21.15.83:0/2567789689 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14b00843f0 con 0x7f14bc15ba30 2024-04-17T10:39:16.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.326+0000 7f14ba59c640 1 --2- 172.21.15.83:0/2567789689 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f149c066f90 0x7f149c069450 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00a9a0 tx=0x7f14ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:16.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.330+0000 7f14aaffd640 1 -- 172.21.15.83:0/2567789689 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f14b0055400 con 0x7f14bc15ba30 2024-04-17T10:39:16.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.518+0000 7f14c09e1640 1 -- 172.21.15.83:0/2567789689 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14bc15abb0 con 0x7f14bc15ba30 2024-04-17T10:39:16.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.518+0000 7f14aaffd640 1 -- 172.21.15.83:0/2567789689 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f14b0054d00 con 0x7f14bc15ba30 2024-04-17T10:39:16.516 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:39:16.516 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1456029744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6833","nonce":1456029744},{"type":"v1","addr":"172.21.15.167:6835","nonce":1456029744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6834/648555908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":648555908},{"type":"v1","addr":"172.21.15.167:6834","nonce":648555908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5163,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1435807773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1435807773},{"type":"v1","addr":"172.21.15.83:6835","nonce":1435807773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:39:13.472135+0000","modified":"2024-04-17T10:39:15.518124+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5172},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5172":{"gid":5172,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1181902025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1181902025},{"type":"v1","addr":"172.21.15.83:6837","nonce":1181902025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-17T10:39:16.517 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 13 2024-04-17T10:39:16.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.518+0000 7f14c09e1640 1 -- 172.21.15.83:0/2567789689 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f149c066f90 msgr2=0x7f149c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:16.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.518+0000 7f14c09e1640 1 --2- 172.21.15.83:0/2567789689 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f149c066f90 0x7f149c069450 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00a9a0 tx=0x7f14ac008040 comp rx=0 tx=0).stop 2024-04-17T10:39:16.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.518+0000 7f14c09e1640 1 -- 172.21.15.83:0/2567789689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14bc15ba30 msgr2=0x7f14bc14b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:16.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.518+0000 7f14c09e1640 1 --2- 172.21.15.83:0/2567789689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14bc15ba30 0x7f14bc14b9e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f14b0007fd0 tx=0x7f14b000c8e0 comp rx=0 tx=0).stop 2024-04-17T10:39:16.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.518+0000 7f14c09e1640 1 -- 172.21.15.83:0/2567789689 shutdown_connections 2024-04-17T10:39:16.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.518+0000 7f14c09e1640 1 --2- 172.21.15.83:0/2567789689 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f149c066f90 0x7f149c069450 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:16.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.518+0000 7f14c09e1640 1 --2- 172.21.15.83:0/2567789689 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14bc15c960 0x7f14bc14bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:16.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.518+0000 7f14c09e1640 1 --2- 172.21.15.83:0/2567789689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14bc15ba30 0x7f14bc14b9e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:16.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.518+0000 7f14c09e1640 1 --2- 172.21.15.83:0/2567789689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14bc0cfbe0 0x7f14bc137130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:16.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.518+0000 7f14c09e1640 1 -- 172.21.15.83:0/2567789689 >> 172.21.15.83:0/2567789689 conn(0x7f14bc06abe0 msgr2=0x7f14bc0b1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:16.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.518+0000 7f14c09e1640 1 -- 172.21.15.83:0/2567789689 shutdown_connections 2024-04-17T10:39:16.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.518+0000 7f14c09e1640 1 -- 172.21.15.83:0/2567789689 wait complete. 2024-04-17T10:39:16.533 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:39:16.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.650+0000 7fe9a8bfc640 1 Processor -- start 2024-04-17T10:39:16.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.650+0000 7fe9a8bfc640 1 -- start start 2024-04-17T10:39:16.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe9a8bfc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9a415db50 0x7fe9a4070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:16.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe9a8bfc640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe9a406dcb0 0x7fe9a40b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:16.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe9a8bfc640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9a40709b0 con 0x7fe9a40aedd0 2024-04-17T10:39:16.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe9a8bfc640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9a415e440 con 0x7fe9a406dcb0 2024-04-17T10:39:16.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe9a8bfc640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a415e720 con 0x7fe9a415db50 2024-04-17T10:39:16.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe99bfff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe9a40aedd0 0x7fe9a406d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48706/0 (socket says 172.21.15.83:48706) 2024-04-17T10:39:16.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe99bfff640 1 -- 172.21.15.83:0/1802573059 learned_addr learned my addr 172.21.15.83:0/1802573059 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:39:16.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe9a2d9d640 1 --2- 172.21.15.83:0/1802573059 >> v2:172.21.15.167:3300/0 conn(0x7fe9a406dcb0 0x7fe9a40b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:16.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe9a259c640 1 --2- 172.21.15.83:0/1802573059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9a415db50 0x7fe9a4070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:16.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe9a1d9b640 1 -- 172.21.15.83:0/1802573059 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1505425793 0 0) 0x7fe98c001120 con 0x7fe9a40aedd0 2024-04-17T10:39:16.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe9a1d9b640 1 -- 172.21.15.83:0/1802573059 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe990003530 con 0x7fe9a40aedd0 2024-04-17T10:39:16.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe9a1d9b640 1 -- 172.21.15.83:0/1802573059 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 951233237 0 0) 0x7fe990003530 con 0x7fe9a40aedd0 2024-04-17T10:39:16.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe9a1d9b640 1 -- 172.21.15.83:0/1802573059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9a415db50 msgr2=0x7fe9a4070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:16.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe9a1d9b640 1 --2- 172.21.15.83:0/1802573059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9a415db50 0x7fe9a4070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:16.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe9a1d9b640 1 -- 172.21.15.83:0/1802573059 >> v2:172.21.15.167:3300/0 conn(0x7fe9a406dcb0 msgr2=0x7fe9a40b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:16.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe9a1d9b640 1 --2- 172.21.15.83:0/1802573059 >> v2:172.21.15.167:3300/0 conn(0x7fe9a406dcb0 0x7fe9a40b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:16.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe9a1d9b640 1 -- 172.21.15.83:0/1802573059 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a415ea00 con 0x7fe9a40aedd0 2024-04-17T10:39:16.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe9a1d9b640 1 -- 172.21.15.83:0/1802573059 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fe98c001850 con 0x7fe9a40aedd0 2024-04-17T10:39:16.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe9a1d9b640 1 -- 172.21.15.83:0/1802573059 >> v1:172.21.15.83:6789/0 conn(0x7fe9a40aedd0 legacy=0x7fe9a406d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:16.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe9a1d9b640 1 --2- 172.21.15.83:0/1802573059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9a415db50 0x7fe9900039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:16.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe9a1d9b640 1 --2- 172.21.15.83:0/1802573059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe990003f40 0x7fe990004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:16.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe9a1d9b640 1 --2- 172.21.15.83:0/1802573059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe990004d50 0x7fe9900051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:16.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe9a1d9b640 1 -- 172.21.15.83:0/1802573059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a415e440 con 0x7fe990004d50 2024-04-17T10:39:16.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe9a1d9b640 1 -- 172.21.15.83:0/1802573059 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a415e720 con 0x7fe990003f40 2024-04-17T10:39:16.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe9a1d9b640 1 -- 172.21.15.83:0/1802573059 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe99000b9d0 con 0x7fe9a415db50 2024-04-17T10:39:16.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe9a2d9d640 1 --2- 172.21.15.83:0/1802573059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe990004d50 0x7fe9900051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:16.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe99bfff640 1 --2- 172.21.15.83:0/1802573059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe990003f40 0x7fe990004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:16.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe9a259c640 1 --2- 172.21.15.83:0/1802573059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9a415db50 0x7fe9900039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:16.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe9a2d9d640 1 -- 172.21.15.83:0/1802573059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9a415db50 msgr2=0x7fe9900039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:16.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe9a2d9d640 1 --2- 172.21.15.83:0/1802573059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9a415db50 0x7fe9900039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:16.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe9a2d9d640 1 -- 172.21.15.83:0/1802573059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe990003f40 msgr2=0x7fe990004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:16.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe9a2d9d640 1 --2- 172.21.15.83:0/1802573059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe990003f40 0x7fe990004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:16.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe9a2d9d640 1 -- 172.21.15.83:0/1802573059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe990003530 con 0x7fe990004d50 2024-04-17T10:39:16.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.654+0000 7fe9a259c640 1 --2- 172.21.15.83:0/1802573059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9a415db50 0x7fe9900039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:16.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.658+0000 7fe9a2d9d640 1 --2- 172.21.15.83:0/1802573059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe990004d50 0x7fe9900051d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe99c009fc0 tx=0x7fe99c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a813abd3a69f9f3b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:16.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.658+0000 7fe9a1d9b640 1 -- 172.21.15.83:0/1802573059 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99c0b2070 con 0x7fe990004d50 2024-04-17T10:39:16.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.658+0000 7fe9a1d9b640 1 -- 172.21.15.83:0/1802573059 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe99c0a5cd0 con 0x7fe990004d50 2024-04-17T10:39:16.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.658+0000 7fe9a1d9b640 1 -- 172.21.15.83:0/1802573059 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99c009c10 con 0x7fe990004d50 2024-04-17T10:39:16.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.658+0000 7fe9a8bfc640 1 -- 172.21.15.83:0/1802573059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe990004d50 msgr2=0x7fe9900051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:16.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.658+0000 7fe9a8bfc640 1 --2- 172.21.15.83:0/1802573059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe990004d50 0x7fe9900051d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe99c009fc0 tx=0x7fe99c01cc60 comp rx=0 tx=0).stop 2024-04-17T10:39:16.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.658+0000 7fe9a2d9d640 1 -- 172.21.15.83:0/1802573059 reap_dead start 2024-04-17T10:39:16.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.658+0000 7fe9a8bfc640 1 -- 172.21.15.83:0/1802573059 shutdown_connections 2024-04-17T10:39:16.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.658+0000 7fe9a8bfc640 1 -- 172.21.15.83:0/1802573059 >> 172.21.15.83:0/1802573059 conn(0x7fe9a406abe0 msgr2=0x7fe9a406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:16.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.658+0000 7fe9a8bfc640 1 -- 172.21.15.83:0/1802573059 shutdown_connections 2024-04-17T10:39:16.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.658+0000 7fe9a8bfc640 1 -- 172.21.15.83:0/1802573059 wait complete. 2024-04-17T10:39:16.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.658+0000 7fe9a8bfc640 1 Processor -- start 2024-04-17T10:39:16.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.658+0000 7fe9a8bfc640 1 -- start start 2024-04-17T10:39:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.658+0000 7fe9a8bfc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe990004d50 0x7fe9a41516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.658+0000 7fe9a8bfc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9a4157930 0x7fe9a4143260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.658+0000 7fe9a8bfc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9a4143de0 0x7fe9a4156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.658+0000 7fe9a8bfc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a415df30 con 0x7fe990004d50 2024-04-17T10:39:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.658+0000 7fe9a8bfc640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a415efc0 con 0x7fe9a4157930 2024-04-17T10:39:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.658+0000 7fe9a8bfc640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a415e0b0 con 0x7fe9a4143de0 2024-04-17T10:39:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.658+0000 7fe9a259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe990004d50 0x7fe9a41516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.658+0000 7fe9a259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe990004d50 0x7fe9a41516b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37214/0 (socket says 172.21.15.83:37214) 2024-04-17T10:39:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.658+0000 7fe9a259c640 1 -- 172.21.15.83:0/598571144 learned_addr learned my addr 172.21.15.83:0/598571144 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.658+0000 7fe99bfff640 1 --2- 172.21.15.83:0/598571144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9a4157930 0x7fe9a4143260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.658+0000 7fe9a2d9d640 1 --2- 172.21.15.83:0/598571144 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9a4143de0 0x7fe9a4156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:16.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.658+0000 7fe9a259c640 1 -- 172.21.15.83:0/598571144 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9a4143de0 msgr2=0x7fe9a4156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:16.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.658+0000 7fe9a259c640 1 --2- 172.21.15.83:0/598571144 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9a4143de0 0x7fe9a4156d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:16.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.658+0000 7fe9a259c640 1 -- 172.21.15.83:0/598571144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9a4157930 msgr2=0x7fe9a4143260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:16.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.658+0000 7fe9a259c640 1 --2- 172.21.15.83:0/598571144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9a4157930 0x7fe9a4143260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:16.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.658+0000 7fe9a259c640 1 -- 172.21.15.83:0/598571144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a4151d40 con 0x7fe990004d50 2024-04-17T10:39:16.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.658+0000 7fe9a259c640 1 --2- 172.21.15.83:0/598571144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe990004d50 0x7fe9a41516b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe9940108d0 tx=0x7fe994010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:16.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.662+0000 7fe99affd640 1 -- 172.21.15.83:0/598571144 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99400f040 con 0x7fe990004d50 2024-04-17T10:39:16.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.662+0000 7fe99affd640 1 -- 172.21.15.83:0/598571144 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe99400eb60 con 0x7fe990004d50 2024-04-17T10:39:16.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.662+0000 7fe9a8bfc640 1 -- 172.21.15.83:0/598571144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9a4152020 con 0x7fe990004d50 2024-04-17T10:39:16.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.662+0000 7fe99affd640 1 -- 172.21.15.83:0/598571144 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99401c400 con 0x7fe990004d50 2024-04-17T10:39:16.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.662+0000 7fe9a8bfc640 1 -- 172.21.15.83:0/598571144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a41525e0 con 0x7fe990004d50 2024-04-17T10:39:16.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.662+0000 7fe9a8bfc640 1 -- 172.21.15.83:0/598571144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9a4074f90 con 0x7fe990004d50 2024-04-17T10:39:16.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.662+0000 7fe99affd640 1 -- 172.21.15.83:0/598571144 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe99401c5a0 con 0x7fe990004d50 2024-04-17T10:39:16.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.662+0000 7fe99affd640 1 --2- 172.21.15.83:0/598571144 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe984066f60 0x7fe984069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:16.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.662+0000 7fe99affd640 1 -- 172.21.15.83:0/598571144 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe994085860 con 0x7fe990004d50 2024-04-17T10:39:16.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.662+0000 7fe99bfff640 1 --2- 172.21.15.83:0/598571144 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe984066f60 0x7fe984069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:16.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.662+0000 7fe99bfff640 1 --2- 172.21.15.83:0/598571144 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe984066f60 0x7fe984069420 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0012c0 tx=0x7fe98c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:16.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.666+0000 7fe99affd640 1 -- 172.21.15.83:0/598571144 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe994018030 con 0x7fe990004d50 2024-04-17T10:39:16.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.822+0000 7fe9a8bfc640 1 -- 172.21.15.83:0/598571144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9a40b2af0 con 0x7fe990004d50 2024-04-17T10:39:16.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.826+0000 7fe99affd640 1 -- 172.21.15.83:0/598571144 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16597 (secure 0 0 0) 0x7fe994085030 con 0x7fe990004d50 2024-04-17T10:39:16.824 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:39:16.824 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":33,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:39:15.101333+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:39:11.330695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-17T10:39:12.438377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:39:16.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.826+0000 7fe9a8bfc640 1 -- 172.21.15.83:0/598571144 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe984066f60 msgr2=0x7fe984069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:16.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.826+0000 7fe9a8bfc640 1 --2- 172.21.15.83:0/598571144 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe984066f60 0x7fe984069420 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0012c0 tx=0x7fe98c009040 comp rx=0 tx=0).stop 2024-04-17T10:39:16.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.826+0000 7fe9a8bfc640 1 -- 172.21.15.83:0/598571144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe990004d50 msgr2=0x7fe9a41516b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:16.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.826+0000 7fe9a8bfc640 1 --2- 172.21.15.83:0/598571144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe990004d50 0x7fe9a41516b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe9940108d0 tx=0x7fe994010d90 comp rx=0 tx=0).stop 2024-04-17T10:39:16.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.826+0000 7fe9a8bfc640 1 -- 172.21.15.83:0/598571144 shutdown_connections 2024-04-17T10:39:16.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.826+0000 7fe9a8bfc640 1 --2- 172.21.15.83:0/598571144 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe984066f60 0x7fe984069420 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:16.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.826+0000 7fe9a8bfc640 1 --2- 172.21.15.83:0/598571144 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9a4143de0 0x7fe9a4156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:16.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.826+0000 7fe9a8bfc640 1 --2- 172.21.15.83:0/598571144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9a4157930 0x7fe9a4143260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:16.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.826+0000 7fe9a8bfc640 1 --2- 172.21.15.83:0/598571144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe990004d50 0x7fe9a41516b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:16.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.826+0000 7fe9a8bfc640 1 -- 172.21.15.83:0/598571144 >> 172.21.15.83:0/598571144 conn(0x7fe9a406abe0 msgr2=0x7fe9a406cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:16.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.826+0000 7fe9a8bfc640 1 -- 172.21.15.83:0/598571144 shutdown_connections 2024-04-17T10:39:16.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.826+0000 7fe9a8bfc640 1 -- 172.21.15.83:0/598571144 wait complete. 2024-04-17T10:39:16.841 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-17T10:39:16.842 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:39:16.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.958+0000 7f2924632640 1 Processor -- start 2024-04-17T10:39:16.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.958+0000 7f2924632640 1 -- start start 2024-04-17T10:39:16.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.962+0000 7f2924632640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f291c15f550 0x7f291c15f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:16.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.962+0000 7f2924632640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f291c06baf0 con 0x7f291c0ccbf0 2024-04-17T10:39:16.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.962+0000 7f2924632640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f291c071bf0 con 0x7f291c06fb70 2024-04-17T10:39:16.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.962+0000 7f2924632640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f291c071ed0 con 0x7f291c15f550 2024-04-17T10:39:16.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.962+0000 7f2922bcf640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f291c15f550 0x7f291c15f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:16.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.962+0000 7f2921bcd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f291c0ccbf0 0x7f291c06b3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54054/0 (socket says 172.21.15.83:54054) 2024-04-17T10:39:16.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.962+0000 7f2921bcd640 1 -- 172.21.15.83:0/511250120 learned_addr learned my addr 172.21.15.83:0/511250120 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:39:16.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.962+0000 7f2922bcf640 1 -- 172.21.15.83:0/511250120 >> v1:172.21.15.167:6789/0 conn(0x7f291c06fb70 legacy=0x7f291c0713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:16.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.962+0000 7f2922bcf640 1 -- 172.21.15.83:0/511250120 >> v1:172.21.15.83:6789/0 conn(0x7f291c0ccbf0 legacy=0x7f291c06b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:16.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.962+0000 7f29213cc640 1 -- 172.21.15.83:0/511250120 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1954329469 0 0) 0x7f291c06baf0 con 0x7f291c0ccbf0 2024-04-17T10:39:16.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.962+0000 7f2922bcf640 1 -- 172.21.15.83:0/511250120 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f291c0721b0 con 0x7f291c15f550 2024-04-17T10:39:16.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.962+0000 7f2922bcf640 1 --2- 172.21.15.83:0/511250120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f291c15f550 0x7f291c15f9e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2914009dc0 tx=0x7f291401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca1ec26a2d4c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:16.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.962+0000 7f29213cc640 1 -- 172.21.15.83:0/511250120 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29140b2070 con 0x7f291c15f550 2024-04-17T10:39:16.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.962+0000 7f29213cc640 1 -- 172.21.15.83:0/511250120 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29140a57d0 con 0x7f291c15f550 2024-04-17T10:39:16.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.962+0000 7f29213cc640 1 -- 172.21.15.83:0/511250120 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29140a5ba0 con 0x7f291c15f550 2024-04-17T10:39:16.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.966+0000 7f2924632640 1 -- 172.21.15.83:0/511250120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f291c15f550 msgr2=0x7f291c15f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:16.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.966+0000 7f2924632640 1 --2- 172.21.15.83:0/511250120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f291c15f550 0x7f291c15f9e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2914009dc0 tx=0x7f291401c910 comp rx=0 tx=0).stop 2024-04-17T10:39:16.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.966+0000 7f2924632640 1 -- 172.21.15.83:0/511250120 shutdown_connections 2024-04-17T10:39:16.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.966+0000 7f2924632640 1 --2- 172.21.15.83:0/511250120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f291c15f550 0x7f291c15f9e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:16.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.966+0000 7f2924632640 1 -- 172.21.15.83:0/511250120 >> 172.21.15.83:0/511250120 conn(0x7f291c06bcf0 msgr2=0x7f291c06c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:16.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.966+0000 7f2924632640 1 -- 172.21.15.83:0/511250120 shutdown_connections 2024-04-17T10:39:16.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.966+0000 7f2924632640 1 -- 172.21.15.83:0/511250120 wait complete. 2024-04-17T10:39:16.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.966+0000 7f2924632640 1 Processor -- start 2024-04-17T10:39:16.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.966+0000 7f2924632640 1 -- start start 2024-04-17T10:39:16.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.966+0000 7f2924632640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f291c06fb70 0x7f291c14d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:16.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.966+0000 7f2924632640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f291c0ccbf0 0x7f291c15ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:16.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.966+0000 7f2924632640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f291c15f550 0x7f291c0c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:16.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.966+0000 7f2924632640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291c169c90 con 0x7f291c06fb70 2024-04-17T10:39:16.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.966+0000 7f2924632640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291c169b10 con 0x7f291c0ccbf0 2024-04-17T10:39:16.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.966+0000 7f2924632640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f291c169e10 con 0x7f291c15f550 2024-04-17T10:39:16.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.966+0000 7f29223ce640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f291c06fb70 0x7f291c14d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:16.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.966+0000 7f29223ce640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f291c06fb70 0x7f291c14d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59884/0 (socket says 172.21.15.83:59884) 2024-04-17T10:39:16.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.966+0000 7f29223ce640 1 -- 172.21.15.83:0/4057120771 learned_addr learned my addr 172.21.15.83:0/4057120771 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:16.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.966+0000 7f2921bcd640 1 --2- 172.21.15.83:0/4057120771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f291c0ccbf0 0x7f291c15ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:16.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.966+0000 7f2922bcf640 1 --2- 172.21.15.83:0/4057120771 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f291c15f550 0x7f291c0c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:16.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.966+0000 7f29223ce640 1 -- 172.21.15.83:0/4057120771 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f291c15f550 msgr2=0x7f291c0c7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:16.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.966+0000 7f29223ce640 1 --2- 172.21.15.83:0/4057120771 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f291c15f550 0x7f291c0c7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:16.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.966+0000 7f29223ce640 1 -- 172.21.15.83:0/4057120771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f291c0ccbf0 msgr2=0x7f291c15ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:16.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.966+0000 7f29223ce640 1 --2- 172.21.15.83:0/4057120771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f291c0ccbf0 0x7f291c15ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:16.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.966+0000 7f29223ce640 1 -- 172.21.15.83:0/4057120771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f291c0c88b0 con 0x7f291c06fb70 2024-04-17T10:39:16.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.966+0000 7f29223ce640 1 --2- 172.21.15.83:0/4057120771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f291c06fb70 0x7f291c14d750 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f291800c450 tx=0x7f291800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:16.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.966+0000 7f2912ffd640 1 -- 172.21.15.83:0/4057120771 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2918012830 con 0x7f291c06fb70 2024-04-17T10:39:16.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.966+0000 7f2912ffd640 1 -- 172.21.15.83:0/4057120771 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2918017070 con 0x7f291c06fb70 2024-04-17T10:39:16.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.966+0000 7f2924632640 1 -- 172.21.15.83:0/4057120771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f291c0c8b90 con 0x7f291c06fb70 2024-04-17T10:39:16.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.966+0000 7f2912ffd640 1 -- 172.21.15.83:0/4057120771 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2918012c30 con 0x7f291c06fb70 2024-04-17T10:39:16.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.966+0000 7f2924632640 1 -- 172.21.15.83:0/4057120771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f291c15a880 con 0x7f291c06fb70 2024-04-17T10:39:16.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.970+0000 7f2924632640 1 -- 172.21.15.83:0/4057120771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f291c074f90 con 0x7f291c06fb70 2024-04-17T10:39:16.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.970+0000 7f2912ffd640 1 -- 172.21.15.83:0/4057120771 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f291801c460 con 0x7f291c06fb70 2024-04-17T10:39:16.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.970+0000 7f2912ffd640 1 --2- 172.21.15.83:0/4057120771 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2900066f60 0x7f2900069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:16.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.970+0000 7f2912ffd640 1 -- 172.21.15.83:0/4057120771 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2918084d70 con 0x7f291c06fb70 2024-04-17T10:39:16.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.970+0000 7f2921bcd640 1 --2- 172.21.15.83:0/4057120771 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2900066f60 0x7f2900069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:16.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.970+0000 7f2921bcd640 1 --2- 172.21.15.83:0/4057120771 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2900066f60 0x7f2900069420 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f291c071930 tx=0x7f290c007b80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:16.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:16.974+0000 7f2912ffd640 1 -- 172.21.15.83:0/4057120771 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2918055e10 con 0x7f291c06fb70 2024-04-17T10:39:17.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.126+0000 7f2924632640 1 -- 172.21.15.83:0/4057120771 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f291c06c9c0 con 0x7f2900066f60 2024-04-17T10:39:17.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.146+0000 7f2912ffd640 1 -- 172.21.15.83:0/4057120771 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251888 (secure 0 0 0) 0x7f291c06c9c0 con 0x7f2900066f60 2024-04-17T10:39:17.147 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:39:17.149 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:39:17.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.150+0000 7f2924632640 1 -- 172.21.15.83:0/4057120771 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2900066f60 msgr2=0x7f2900069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:17.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.150+0000 7f2924632640 1 --2- 172.21.15.83:0/4057120771 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2900066f60 0x7f2900069420 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f291c071930 tx=0x7f290c007b80 comp rx=0 tx=0).stop 2024-04-17T10:39:17.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.150+0000 7f2924632640 1 -- 172.21.15.83:0/4057120771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f291c06fb70 msgr2=0x7f291c14d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:17.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.150+0000 7f2924632640 1 --2- 172.21.15.83:0/4057120771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f291c06fb70 0x7f291c14d750 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f291800c450 tx=0x7f291800c910 comp rx=0 tx=0).stop 2024-04-17T10:39:17.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.150+0000 7f2924632640 1 -- 172.21.15.83:0/4057120771 shutdown_connections 2024-04-17T10:39:17.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.150+0000 7f2924632640 1 --2- 172.21.15.83:0/4057120771 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2900066f60 0x7f2900069420 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:17.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.150+0000 7f2924632640 1 --2- 172.21.15.83:0/4057120771 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f291c15f550 0x7f291c0c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:17.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.150+0000 7f2924632640 1 --2- 172.21.15.83:0/4057120771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f291c0ccbf0 0x7f291c15ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:17.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.150+0000 7f2924632640 1 --2- 172.21.15.83:0/4057120771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f291c06fb70 0x7f291c14d750 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:17.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.150+0000 7f2924632640 1 -- 172.21.15.83:0/4057120771 >> 172.21.15.83:0/4057120771 conn(0x7f291c06bcf0 msgr2=0x7f291c06b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:17.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.150+0000 7f2924632640 1 -- 172.21.15.83:0/4057120771 shutdown_connections 2024-04-17T10:39:17.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.150+0000 7f2924632640 1 -- 172.21.15.83:0/4057120771 wait complete. 2024-04-17T10:39:17.165 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-04-17T10:39:16.197770+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":67,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220100,"kb_used_data":3920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754620,"statfs":{"total":773094113280,"available":772868730880,"internally_reserved":0,"allocated":4014080,"data_stored":2263326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":47,"apply_latency_ms":47,"commit_latency_ns":47000000,"apply_latency_ns":47000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.106743"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.157736+0000","last_change":"2024-04-17T10:39:14.149806+0000","last_active":"2024-04-17T10:39:14.157736+0000","last_peered":"2024-04-17T10:39:14.157736+0000","last_clean":"2024-04-17T10:39:14.157736+0000","last_became_active":"2024-04-17T10:39:14.147423+0000","last_became_peered":"2024-04-17T10:39:14.147423+0000","last_unstale":"2024-04-17T10:39:14.157736+0000","last_undegraded":"2024-04-17T10:39:14.157736+0000","last_fullsized":"2024-04-17T10:39:14.157736+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:58:07.020827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.130156+0000","last_change":"2024-04-17T10:39:14.122516+0000","last_active":"2024-04-17T10:39:14.130156+0000","last_peered":"2024-04-17T10:39:14.130156+0000","last_clean":"2024-04-17T10:39:14.130156+0000","last_became_active":"2024-04-17T10:39:14.122216+0000","last_became_peered":"2024-04-17T10:39:14.122216+0000","last_unstale":"2024-04-17T10:39:14.130156+0000","last_undegraded":"2024-04-17T10:39:14.130156+0000","last_fullsized":"2024-04-17T10:39:14.130156+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:23.357477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.143512+0000","last_change":"2024-04-17T10:39:14.134180+0000","last_active":"2024-04-17T10:39:14.143512+0000","last_peered":"2024-04-17T10:39:14.143512+0000","last_clean":"2024-04-17T10:39:14.143512+0000","last_became_active":"2024-04-17T10:39:14.132417+0000","last_became_peered":"2024-04-17T10:39:14.132417+0000","last_unstale":"2024-04-17T10:39:14.143512+0000","last_undegraded":"2024-04-17T10:39:14.143512+0000","last_fullsized":"2024-04-17T10:39:14.143512+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:09:41.478596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.142129+0000","last_change":"2024-04-17T10:39:14.139185+0000","last_active":"2024-04-17T10:39:14.142129+0000","last_peered":"2024-04-17T10:39:14.142129+0000","last_clean":"2024-04-17T10:39:14.142129+0000","last_became_active":"2024-04-17T10:39:14.138869+0000","last_became_peered":"2024-04-17T10:39:14.138869+0000","last_unstale":"2024-04-17T10:39:14.142129+0000","last_undegraded":"2024-04-17T10:39:14.142129+0000","last_fullsized":"2024-04-17T10:39:14.142129+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:45:43.557201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.129884+0000","last_change":"2024-04-17T10:39:14.125870+0000","last_active":"2024-04-17T10:39:14.129884+0000","last_peered":"2024-04-17T10:39:14.129884+0000","last_clean":"2024-04-17T10:39:14.129884+0000","last_became_active":"2024-04-17T10:39:14.125594+0000","last_became_peered":"2024-04-17T10:39:14.125594+0000","last_unstale":"2024-04-17T10:39:14.129884+0000","last_undegraded":"2024-04-17T10:39:14.129884+0000","last_fullsized":"2024-04-17T10:39:14.129884+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:10:44.246912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.141788+0000","last_change":"2024-04-17T10:39:14.136456+0000","last_active":"2024-04-17T10:39:14.141788+0000","last_peered":"2024-04-17T10:39:14.141788+0000","last_clean":"2024-04-17T10:39:14.141788+0000","last_became_active":"2024-04-17T10:39:14.136208+0000","last_became_peered":"2024-04-17T10:39:14.136208+0000","last_unstale":"2024-04-17T10:39:14.141788+0000","last_undegraded":"2024-04-17T10:39:14.141788+0000","last_fullsized":"2024-04-17T10:39:14.141788+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:28:23.922532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.161348+0000","last_change":"2024-04-17T10:39:14.150874+0000","last_active":"2024-04-17T10:39:14.161348+0000","last_peered":"2024-04-17T10:39:14.161348+0000","last_clean":"2024-04-17T10:39:14.161348+0000","last_became_active":"2024-04-17T10:39:14.149282+0000","last_became_peered":"2024-04-17T10:39:14.149282+0000","last_unstale":"2024-04-17T10:39:14.161348+0000","last_undegraded":"2024-04-17T10:39:14.161348+0000","last_fullsized":"2024-04-17T10:39:14.161348+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:12:05.386724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.154613+0000","last_change":"2024-04-17T10:39:14.146974+0000","last_active":"2024-04-17T10:39:14.154613+0000","last_peered":"2024-04-17T10:39:14.154613+0000","last_clean":"2024-04-17T10:39:14.154613+0000","last_became_active":"2024-04-17T10:39:14.139506+0000","last_became_peered":"2024-04-17T10:39:14.139506+0000","last_unstale":"2024-04-17T10:39:14.154613+0000","last_undegraded":"2024-04-17T10:39:14.154613+0000","last_fullsized":"2024-04-17T10:39:14.154613+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:25:15.943792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.151385+0000","last_change":"2024-04-17T10:39:14.119197+0000","last_active":"2024-04-17T10:39:14.151385+0000","last_peered":"2024-04-17T10:39:14.151385+0000","last_clean":"2024-04-17T10:39:14.151385+0000","last_became_active":"2024-04-17T10:39:14.118452+0000","last_became_peered":"2024-04-17T10:39:14.118452+0000","last_unstale":"2024-04-17T10:39:14.151385+0000","last_undegraded":"2024-04-17T10:39:14.151385+0000","last_fullsized":"2024-04-17T10:39:14.151385+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:58:07.020827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.131190+0000","last_change":"2024-04-17T10:39:14.127149+0000","last_active":"2024-04-17T10:39:14.131190+0000","last_peered":"2024-04-17T10:39:14.131190+0000","last_clean":"2024-04-17T10:39:14.131190+0000","last_became_active":"2024-04-17T10:39:14.126573+0000","last_became_peered":"2024-04-17T10:39:14.126573+0000","last_unstale":"2024-04-17T10:39:14.131190+0000","last_undegraded":"2024-04-17T10:39:14.131190+0000","last_fullsized":"2024-04-17T10:39:14.131190+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:39.269987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.154071+0000","last_change":"2024-04-17T10:39:14.140253+0000","last_active":"2024-04-17T10:39:14.154071+0000","last_peered":"2024-04-17T10:39:14.154071+0000","last_clean":"2024-04-17T10:39:14.154071+0000","last_became_active":"2024-04-17T10:39:14.139460+0000","last_became_peered":"2024-04-17T10:39:14.139460+0000","last_unstale":"2024-04-17T10:39:14.154071+0000","last_undegraded":"2024-04-17T10:39:14.154071+0000","last_fullsized":"2024-04-17T10:39:14.154071+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:23:38.450031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.145623+0000","last_change":"2024-04-17T10:39:14.140030+0000","last_active":"2024-04-17T10:39:14.145623+0000","last_peered":"2024-04-17T10:39:14.145623+0000","last_clean":"2024-04-17T10:39:14.145623+0000","last_became_active":"2024-04-17T10:39:14.137672+0000","last_became_peered":"2024-04-17T10:39:14.137672+0000","last_unstale":"2024-04-17T10:39:14.145623+0000","last_undegraded":"2024-04-17T10:39:14.145623+0000","last_fullsized":"2024-04-17T10:39:14.145623+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:02:20.150885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.120750+0000","last_change":"2024-04-17T10:39:14.117285+0000","last_active":"2024-04-17T10:39:14.120750+0000","last_peered":"2024-04-17T10:39:14.120750+0000","last_clean":"2024-04-17T10:39:14.120750+0000","last_became_active":"2024-04-17T10:39:14.116962+0000","last_became_peered":"2024-04-17T10:39:14.116962+0000","last_unstale":"2024-04-17T10:39:14.120750+0000","last_undegraded":"2024-04-17T10:39:14.120750+0000","last_fullsized":"2024-04-17T10:39:14.120750+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:33:09.883101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.123650+0000","last_change":"2024-04-17T10:39:14.114202+0000","last_active":"2024-04-17T10:39:14.123650+0000","last_peered":"2024-04-17T10:39:14.123650+0000","last_clean":"2024-04-17T10:39:14.123650+0000","last_became_active":"2024-04-17T10:39:14.113956+0000","last_became_peered":"2024-04-17T10:39:14.113956+0000","last_unstale":"2024-04-17T10:39:14.123650+0000","last_undegraded":"2024-04-17T10:39:14.123650+0000","last_fullsized":"2024-04-17T10:39:14.123650+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:23:38.450031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.159283+0000","last_change":"2024-04-17T10:39:14.153679+0000","last_active":"2024-04-17T10:39:14.159283+0000","last_peered":"2024-04-17T10:39:14.159283+0000","last_clean":"2024-04-17T10:39:14.159283+0000","last_became_active":"2024-04-17T10:39:14.152709+0000","last_became_peered":"2024-04-17T10:39:14.152709+0000","last_unstale":"2024-04-17T10:39:14.159283+0000","last_undegraded":"2024-04-17T10:39:14.159283+0000","last_fullsized":"2024-04-17T10:39:14.159283+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:55.079731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.162488+0000","last_change":"2024-04-17T10:39:14.154314+0000","last_active":"2024-04-17T10:39:14.162488+0000","last_peered":"2024-04-17T10:39:14.162488+0000","last_clean":"2024-04-17T10:39:14.162488+0000","last_became_active":"2024-04-17T10:39:14.153667+0000","last_became_peered":"2024-04-17T10:39:14.153667+0000","last_unstale":"2024-04-17T10:39:14.162488+0000","last_undegraded":"2024-04-17T10:39:14.162488+0000","last_fullsized":"2024-04-17T10:39:14.162488+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:02:20.150885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.155597+0000","last_change":"2024-04-17T10:39:14.150086+0000","last_active":"2024-04-17T10:39:14.155597+0000","last_peered":"2024-04-17T10:39:14.155597+0000","last_clean":"2024-04-17T10:39:14.155597+0000","last_became_active":"2024-04-17T10:39:14.149415+0000","last_became_peered":"2024-04-17T10:39:14.149415+0000","last_unstale":"2024-04-17T10:39:14.155597+0000","last_undegraded":"2024-04-17T10:39:14.155597+0000","last_fullsized":"2024-04-17T10:39:14.155597+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:23.357477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.130181+0000","last_change":"2024-04-17T10:39:14.119438+0000","last_active":"2024-04-17T10:39:14.130181+0000","last_peered":"2024-04-17T10:39:14.130181+0000","last_clean":"2024-04-17T10:39:14.130181+0000","last_became_active":"2024-04-17T10:39:14.119181+0000","last_became_peered":"2024-04-17T10:39:14.119181+0000","last_unstale":"2024-04-17T10:39:14.130181+0000","last_undegraded":"2024-04-17T10:39:14.130181+0000","last_fullsized":"2024-04-17T10:39:14.130181+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:12.498524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.127011+0000","last_change":"2024-04-17T10:39:14.117915+0000","last_active":"2024-04-17T10:39:14.127011+0000","last_peered":"2024-04-17T10:39:14.127011+0000","last_clean":"2024-04-17T10:39:14.127011+0000","last_became_active":"2024-04-17T10:39:14.117554+0000","last_became_peered":"2024-04-17T10:39:14.117554+0000","last_unstale":"2024-04-17T10:39:14.127011+0000","last_undegraded":"2024-04-17T10:39:14.127011+0000","last_fullsized":"2024-04-17T10:39:14.127011+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:25:15.943792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.147590+0000","last_change":"2024-04-17T10:39:14.144468+0000","last_active":"2024-04-17T10:39:14.147590+0000","last_peered":"2024-04-17T10:39:14.147590+0000","last_clean":"2024-04-17T10:39:14.147590+0000","last_became_active":"2024-04-17T10:39:14.144169+0000","last_became_peered":"2024-04-17T10:39:14.144169+0000","last_unstale":"2024-04-17T10:39:14.147590+0000","last_undegraded":"2024-04-17T10:39:14.147590+0000","last_fullsized":"2024-04-17T10:39:14.147590+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:03:44.258454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.148694+0000","last_change":"2024-04-17T10:39:14.122477+0000","last_active":"2024-04-17T10:39:14.148694+0000","last_peered":"2024-04-17T10:39:14.148694+0000","last_clean":"2024-04-17T10:39:14.148694+0000","last_became_active":"2024-04-17T10:39:14.121344+0000","last_became_peered":"2024-04-17T10:39:14.121344+0000","last_unstale":"2024-04-17T10:39:14.148694+0000","last_undegraded":"2024-04-17T10:39:14.148694+0000","last_fullsized":"2024-04-17T10:39:14.148694+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:15.162469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.148425+0000","last_change":"2024-04-17T10:39:14.129387+0000","last_active":"2024-04-17T10:39:14.148425+0000","last_peered":"2024-04-17T10:39:14.148425+0000","last_clean":"2024-04-17T10:39:14.148425+0000","last_became_active":"2024-04-17T10:39:14.128601+0000","last_became_peered":"2024-04-17T10:39:14.128601+0000","last_unstale":"2024-04-17T10:39:14.148425+0000","last_undegraded":"2024-04-17T10:39:14.148425+0000","last_fullsized":"2024-04-17T10:39:14.148425+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:03:31.760691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.158577+0000","last_change":"2024-04-17T10:39:14.157470+0000","last_active":"2024-04-17T10:39:14.158577+0000","last_peered":"2024-04-17T10:39:14.158577+0000","last_clean":"2024-04-17T10:39:14.158577+0000","last_became_active":"2024-04-17T10:39:14.157195+0000","last_became_peered":"2024-04-17T10:39:14.157195+0000","last_unstale":"2024-04-17T10:39:14.158577+0000","last_undegraded":"2024-04-17T10:39:14.158577+0000","last_fullsized":"2024-04-17T10:39:14.158577+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:57:53.655517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.144932+0000","last_change":"2024-04-17T10:39:14.133445+0000","last_active":"2024-04-17T10:39:14.144932+0000","last_peered":"2024-04-17T10:39:14.144932+0000","last_clean":"2024-04-17T10:39:14.144932+0000","last_became_active":"2024-04-17T10:39:14.132681+0000","last_became_peered":"2024-04-17T10:39:14.132681+0000","last_unstale":"2024-04-17T10:39:14.144932+0000","last_undegraded":"2024-04-17T10:39:14.144932+0000","last_fullsized":"2024-04-17T10:39:14.144932+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:12.498524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.158350+0000","last_change":"2024-04-17T10:39:14.154140+0000","last_active":"2024-04-17T10:39:14.158350+0000","last_peered":"2024-04-17T10:39:14.158350+0000","last_clean":"2024-04-17T10:39:14.158350+0000","last_became_active":"2024-04-17T10:39:14.153811+0000","last_became_peered":"2024-04-17T10:39:14.153811+0000","last_unstale":"2024-04-17T10:39:14.158350+0000","last_undegraded":"2024-04-17T10:39:14.158350+0000","last_fullsized":"2024-04-17T10:39:14.158350+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:23:38.450031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.130123+0000","last_change":"2024-04-17T10:39:14.121358+0000","last_active":"2024-04-17T10:39:14.130123+0000","last_peered":"2024-04-17T10:39:14.130123+0000","last_clean":"2024-04-17T10:39:14.130123+0000","last_became_active":"2024-04-17T10:39:14.121069+0000","last_became_peered":"2024-04-17T10:39:14.121069+0000","last_unstale":"2024-04-17T10:39:14.130123+0000","last_undegraded":"2024-04-17T10:39:14.130123+0000","last_fullsized":"2024-04-17T10:39:14.130123+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:09:41.478596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.142150+0000","last_change":"2024-04-17T10:39:14.139462+0000","last_active":"2024-04-17T10:39:14.142150+0000","last_peered":"2024-04-17T10:39:14.142150+0000","last_clean":"2024-04-17T10:39:14.142150+0000","last_became_active":"2024-04-17T10:39:14.138997+0000","last_became_peered":"2024-04-17T10:39:14.138997+0000","last_unstale":"2024-04-17T10:39:14.142150+0000","last_undegraded":"2024-04-17T10:39:14.142150+0000","last_fullsized":"2024-04-17T10:39:14.142150+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:28:20.535333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.145568+0000","last_change":"2024-04-17T10:39:14.138417+0000","last_active":"2024-04-17T10:39:14.145568+0000","last_peered":"2024-04-17T10:39:14.145568+0000","last_clean":"2024-04-17T10:39:14.145568+0000","last_became_active":"2024-04-17T10:39:14.136784+0000","last_became_peered":"2024-04-17T10:39:14.136784+0000","last_unstale":"2024-04-17T10:39:14.145568+0000","last_undegraded":"2024-04-17T10:39:14.145568+0000","last_fullsized":"2024-04-17T10:39:14.145568+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:28:23.922532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.148991+0000","last_change":"2024-04-17T10:39:14.131698+0000","last_active":"2024-04-17T10:39:14.148991+0000","last_peered":"2024-04-17T10:39:14.148991+0000","last_clean":"2024-04-17T10:39:14.148991+0000","last_became_active":"2024-04-17T10:39:14.129585+0000","last_became_peered":"2024-04-17T10:39:14.129585+0000","last_unstale":"2024-04-17T10:39:14.148991+0000","last_undegraded":"2024-04-17T10:39:14.148991+0000","last_fullsized":"2024-04-17T10:39:14.148991+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:57:25.113157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.155188+0000","last_change":"2024-04-17T10:39:14.148859+0000","last_active":"2024-04-17T10:39:14.155188+0000","last_peered":"2024-04-17T10:39:14.155188+0000","last_clean":"2024-04-17T10:39:14.155188+0000","last_became_active":"2024-04-17T10:39:14.148147+0000","last_became_peered":"2024-04-17T10:39:14.148147+0000","last_unstale":"2024-04-17T10:39:14.155188+0000","last_undegraded":"2024-04-17T10:39:14.155188+0000","last_fullsized":"2024-04-17T10:39:14.155188+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:09:41.478596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.151209+0000","last_change":"2024-04-17T10:39:14.142935+0000","last_active":"2024-04-17T10:39:14.151209+0000","last_peered":"2024-04-17T10:39:14.151209+0000","last_clean":"2024-04-17T10:39:14.151209+0000","last_became_active":"2024-04-17T10:39:14.141904+0000","last_became_peered":"2024-04-17T10:39:14.141904+0000","last_unstale":"2024-04-17T10:39:14.151209+0000","last_undegraded":"2024-04-17T10:39:14.151209+0000","last_fullsized":"2024-04-17T10:39:14.151209+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:17:01.172533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.154655+0000","last_change":"2024-04-17T10:39:14.148267+0000","last_active":"2024-04-17T10:39:14.154655+0000","last_peered":"2024-04-17T10:39:14.154655+0000","last_clean":"2024-04-17T10:39:14.154655+0000","last_became_active":"2024-04-17T10:39:14.140489+0000","last_became_peered":"2024-04-17T10:39:14.140489+0000","last_unstale":"2024-04-17T10:39:14.154655+0000","last_undegraded":"2024-04-17T10:39:14.154655+0000","last_fullsized":"2024-04-17T10:39:14.154655+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:57:16.628519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.158160+0000","last_change":"2024-04-17T10:39:14.154234+0000","last_active":"2024-04-17T10:39:14.158160+0000","last_peered":"2024-04-17T10:39:14.158160+0000","last_clean":"2024-04-17T10:39:14.158160+0000","last_became_active":"2024-04-17T10:39:14.153893+0000","last_became_peered":"2024-04-17T10:39:14.153893+0000","last_unstale":"2024-04-17T10:39:14.158160+0000","last_undegraded":"2024-04-17T10:39:14.158160+0000","last_fullsized":"2024-04-17T10:39:14.158160+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:45:43.557201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.156988+0000","last_change":"2024-04-17T10:39:13.137675+0000","last_active":"2024-04-17T10:39:14.156988+0000","last_peered":"2024-04-17T10:39:14.156988+0000","last_clean":"2024-04-17T10:39:14.156988+0000","last_became_active":"2024-04-17T10:39:13.136910+0000","last_became_peered":"2024-04-17T10:39:13.136910+0000","last_unstale":"2024-04-17T10:39:14.156988+0000","last_undegraded":"2024-04-17T10:39:14.156988+0000","last_fullsized":"2024-04-17T10:39:14.156988+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:10:15.558571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.159194+0000","last_change":"2024-04-17T10:39:14.155480+0000","last_active":"2024-04-17T10:39:14.159194+0000","last_peered":"2024-04-17T10:39:14.159194+0000","last_clean":"2024-04-17T10:39:14.159194+0000","last_became_active":"2024-04-17T10:39:14.154666+0000","last_became_peered":"2024-04-17T10:39:14.154666+0000","last_unstale":"2024-04-17T10:39:14.159194+0000","last_undegraded":"2024-04-17T10:39:14.159194+0000","last_fullsized":"2024-04-17T10:39:14.159194+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:15.162469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.153759+0000","last_change":"2024-04-17T10:39:13.114832+0000","last_active":"2024-04-17T10:39:14.153759+0000","last_peered":"2024-04-17T10:39:14.153759+0000","last_clean":"2024-04-17T10:39:14.153759+0000","last_became_active":"2024-04-17T10:39:13.114474+0000","last_became_peered":"2024-04-17T10:39:13.114474+0000","last_unstale":"2024-04-17T10:39:14.153759+0000","last_undegraded":"2024-04-17T10:39:14.153759+0000","last_fullsized":"2024-04-17T10:39:14.153759+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:45:46.186394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.148474+0000","last_change":"2024-04-17T10:39:14.129590+0000","last_active":"2024-04-17T10:39:14.148474+0000","last_peered":"2024-04-17T10:39:14.148474+0000","last_clean":"2024-04-17T10:39:14.148474+0000","last_became_active":"2024-04-17T10:39:14.128771+0000","last_became_peered":"2024-04-17T10:39:14.128771+0000","last_unstale":"2024-04-17T10:39:14.148474+0000","last_undegraded":"2024-04-17T10:39:14.148474+0000","last_fullsized":"2024-04-17T10:39:14.148474+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:39.269987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.131849+0000","last_change":"2024-04-17T10:39:13.120615+0000","last_active":"2024-04-17T10:39:14.131849+0000","last_peered":"2024-04-17T10:39:14.131849+0000","last_clean":"2024-04-17T10:39:14.131849+0000","last_became_active":"2024-04-17T10:39:13.120338+0000","last_became_peered":"2024-04-17T10:39:13.120338+0000","last_unstale":"2024-04-17T10:39:14.131849+0000","last_undegraded":"2024-04-17T10:39:14.131849+0000","last_fullsized":"2024-04-17T10:39:14.131849+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:28:19.531406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.142176+0000","last_change":"2024-04-17T10:39:14.139943+0000","last_active":"2024-04-17T10:39:14.142176+0000","last_peered":"2024-04-17T10:39:14.142176+0000","last_clean":"2024-04-17T10:39:14.142176+0000","last_became_active":"2024-04-17T10:39:14.139474+0000","last_became_peered":"2024-04-17T10:39:14.139474+0000","last_unstale":"2024-04-17T10:39:14.142176+0000","last_undegraded":"2024-04-17T10:39:14.142176+0000","last_fullsized":"2024-04-17T10:39:14.142176+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:30:34.072845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.130286+0000","last_change":"2024-04-17T10:39:13.123599+0000","last_active":"2024-04-17T10:39:14.130286+0000","last_peered":"2024-04-17T10:39:14.130286+0000","last_clean":"2024-04-17T10:39:14.130286+0000","last_became_active":"2024-04-17T10:39:13.123010+0000","last_became_peered":"2024-04-17T10:39:13.123010+0000","last_unstale":"2024-04-17T10:39:14.130286+0000","last_undegraded":"2024-04-17T10:39:14.130286+0000","last_fullsized":"2024-04-17T10:39:14.130286+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:57:54.529742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.155140+0000","last_change":"2024-04-17T10:39:14.148468+0000","last_active":"2024-04-17T10:39:14.155140+0000","last_peered":"2024-04-17T10:39:14.155140+0000","last_clean":"2024-04-17T10:39:14.155140+0000","last_became_active":"2024-04-17T10:39:14.147709+0000","last_became_peered":"2024-04-17T10:39:14.147709+0000","last_unstale":"2024-04-17T10:39:14.155140+0000","last_undegraded":"2024-04-17T10:39:14.155140+0000","last_fullsized":"2024-04-17T10:39:14.155140+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:12.498524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.148800+0000","last_change":"2024-04-17T10:39:13.139187+0000","last_active":"2024-04-17T10:39:14.148800+0000","last_peered":"2024-04-17T10:39:14.148800+0000","last_clean":"2024-04-17T10:39:14.148800+0000","last_became_active":"2024-04-17T10:39:13.138301+0000","last_became_peered":"2024-04-17T10:39:13.138301+0000","last_unstale":"2024-04-17T10:39:14.148800+0000","last_undegraded":"2024-04-17T10:39:14.148800+0000","last_fullsized":"2024-04-17T10:39:14.148800+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:52:09.191244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.142406+0000","last_change":"2024-04-17T10:39:14.133235+0000","last_active":"2024-04-17T10:39:14.142406+0000","last_peered":"2024-04-17T10:39:14.142406+0000","last_clean":"2024-04-17T10:39:14.142406+0000","last_became_active":"2024-04-17T10:39:14.132536+0000","last_became_peered":"2024-04-17T10:39:14.132536+0000","last_unstale":"2024-04-17T10:39:14.142406+0000","last_undegraded":"2024-04-17T10:39:14.142406+0000","last_fullsized":"2024-04-17T10:39:14.142406+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:57:16.628519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.148545+0000","last_change":"2024-04-17T10:39:13.139900+0000","last_active":"2024-04-17T10:39:14.148545+0000","last_peered":"2024-04-17T10:39:14.148545+0000","last_clean":"2024-04-17T10:39:14.148545+0000","last_became_active":"2024-04-17T10:39:13.138670+0000","last_became_peered":"2024-04-17T10:39:13.138670+0000","last_unstale":"2024-04-17T10:39:14.148545+0000","last_undegraded":"2024-04-17T10:39:14.148545+0000","last_fullsized":"2024-04-17T10:39:14.148545+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:31:02.964758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.161720+0000","last_change":"2024-04-17T10:39:14.156061+0000","last_active":"2024-04-17T10:39:14.161720+0000","last_peered":"2024-04-17T10:39:14.161720+0000","last_clean":"2024-04-17T10:39:14.161720+0000","last_became_active":"2024-04-17T10:39:14.154568+0000","last_became_peered":"2024-04-17T10:39:14.154568+0000","last_unstale":"2024-04-17T10:39:14.161720+0000","last_undegraded":"2024-04-17T10:39:14.161720+0000","last_fullsized":"2024-04-17T10:39:14.161720+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:28:20.535333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.105297+0000","last_change":"2024-04-17T10:39:13.119307+0000","last_active":"2024-04-17T10:39:14.105297+0000","last_peered":"2024-04-17T10:39:14.105297+0000","last_clean":"2024-04-17T10:39:14.105297+0000","last_became_active":"2024-04-17T10:39:13.118917+0000","last_became_peered":"2024-04-17T10:39:13.118917+0000","last_unstale":"2024-04-17T10:39:14.105297+0000","last_undegraded":"2024-04-17T10:39:14.105297+0000","last_fullsized":"2024-04-17T10:39:14.105297+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:57:24.109230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.141560+0000","last_change":"2024-04-17T10:39:14.136470+0000","last_active":"2024-04-17T10:39:14.141560+0000","last_peered":"2024-04-17T10:39:14.141560+0000","last_clean":"2024-04-17T10:39:14.141560+0000","last_became_active":"2024-04-17T10:39:14.136202+0000","last_became_peered":"2024-04-17T10:39:14.136202+0000","last_unstale":"2024-04-17T10:39:14.141560+0000","last_undegraded":"2024-04-17T10:39:14.141560+0000","last_fullsized":"2024-04-17T10:39:14.141560+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:57:25.113157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.108447+0000","last_change":"2024-04-17T10:39:13.139381+0000","last_active":"2024-04-17T10:39:14.108447+0000","last_peered":"2024-04-17T10:39:14.108447+0000","last_clean":"2024-04-17T10:39:14.108447+0000","last_became_active":"2024-04-17T10:39:13.138662+0000","last_became_peered":"2024-04-17T10:39:13.138662+0000","last_unstale":"2024-04-17T10:39:14.108447+0000","last_undegraded":"2024-04-17T10:39:14.108447+0000","last_fullsized":"2024-04-17T10:39:14.108447+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:23:13.822291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.142675+0000","last_change":"2024-04-17T10:39:14.134588+0000","last_active":"2024-04-17T10:39:14.142675+0000","last_peered":"2024-04-17T10:39:14.142675+0000","last_clean":"2024-04-17T10:39:14.142675+0000","last_became_active":"2024-04-17T10:39:14.133701+0000","last_became_peered":"2024-04-17T10:39:14.133701+0000","last_unstale":"2024-04-17T10:39:14.142675+0000","last_undegraded":"2024-04-17T10:39:14.142675+0000","last_fullsized":"2024-04-17T10:39:14.142675+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:23.357477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.106772+0000","last_change":"2024-04-17T10:39:13.129003+0000","last_active":"2024-04-17T10:39:14.106772+0000","last_peered":"2024-04-17T10:39:14.106772+0000","last_clean":"2024-04-17T10:39:14.106772+0000","last_became_active":"2024-04-17T10:39:13.128775+0000","last_became_peered":"2024-04-17T10:39:13.128775+0000","last_unstale":"2024-04-17T10:39:14.106772+0000","last_undegraded":"2024-04-17T10:39:14.106772+0000","last_fullsized":"2024-04-17T10:39:14.106772+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:24:47.491838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.127035+0000","last_change":"2024-04-17T10:39:14.118323+0000","last_active":"2024-04-17T10:39:14.127035+0000","last_peered":"2024-04-17T10:39:14.127035+0000","last_clean":"2024-04-17T10:39:14.127035+0000","last_became_active":"2024-04-17T10:39:14.118097+0000","last_became_peered":"2024-04-17T10:39:14.118097+0000","last_unstale":"2024-04-17T10:39:14.127035+0000","last_undegraded":"2024-04-17T10:39:14.127035+0000","last_fullsized":"2024-04-17T10:39:14.127035+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:57:16.628519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.115834+0000","last_change":"2024-04-17T10:39:13.108220+0000","last_active":"2024-04-17T10:39:14.115834+0000","last_peered":"2024-04-17T10:39:14.115834+0000","last_clean":"2024-04-17T10:39:14.115834+0000","last_became_active":"2024-04-17T10:39:13.107899+0000","last_became_peered":"2024-04-17T10:39:13.107899+0000","last_unstale":"2024-04-17T10:39:14.115834+0000","last_undegraded":"2024-04-17T10:39:14.115834+0000","last_fullsized":"2024-04-17T10:39:14.115834+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:14.158542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.155752+0000","last_change":"2024-04-17T10:39:14.153002+0000","last_active":"2024-04-17T10:39:14.155752+0000","last_peered":"2024-04-17T10:39:14.155752+0000","last_clean":"2024-04-17T10:39:14.155752+0000","last_became_active":"2024-04-17T10:39:14.152720+0000","last_became_peered":"2024-04-17T10:39:14.152720+0000","last_unstale":"2024-04-17T10:39:14.155752+0000","last_undegraded":"2024-04-17T10:39:14.155752+0000","last_fullsized":"2024-04-17T10:39:14.155752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:02:20.150885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.165962+0000","last_change":"2024-04-17T10:39:13.133898+0000","last_active":"2024-04-17T10:39:14.165962+0000","last_peered":"2024-04-17T10:39:14.165962+0000","last_clean":"2024-04-17T10:39:14.165962+0000","last_became_active":"2024-04-17T10:39:13.133055+0000","last_became_peered":"2024-04-17T10:39:13.133055+0000","last_unstale":"2024-04-17T10:39:14.165962+0000","last_undegraded":"2024-04-17T10:39:14.165962+0000","last_fullsized":"2024-04-17T10:39:14.165962+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:25:14.939865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.143578+0000","last_change":"2024-04-17T10:39:14.134931+0000","last_active":"2024-04-17T10:39:14.143578+0000","last_peered":"2024-04-17T10:39:14.143578+0000","last_clean":"2024-04-17T10:39:14.143578+0000","last_became_active":"2024-04-17T10:39:14.133404+0000","last_became_peered":"2024-04-17T10:39:14.133404+0000","last_unstale":"2024-04-17T10:39:14.143578+0000","last_undegraded":"2024-04-17T10:39:14.143578+0000","last_fullsized":"2024-04-17T10:39:14.143578+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:12:05.386724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.153129+0000","last_change":"2024-04-17T10:39:13.118837+0000","last_active":"2024-04-17T10:39:14.153129+0000","last_peered":"2024-04-17T10:39:14.153129+0000","last_clean":"2024-04-17T10:39:14.153129+0000","last_became_active":"2024-04-17T10:39:13.118431+0000","last_became_peered":"2024-04-17T10:39:13.118431+0000","last_unstale":"2024-04-17T10:39:14.153129+0000","last_undegraded":"2024-04-17T10:39:14.153129+0000","last_fullsized":"2024-04-17T10:39:14.153129+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:45:42.553273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.147727+0000","last_change":"2024-04-17T10:39:14.143465+0000","last_active":"2024-04-17T10:39:14.147727+0000","last_peered":"2024-04-17T10:39:14.147727+0000","last_clean":"2024-04-17T10:39:14.147727+0000","last_became_active":"2024-04-17T10:39:14.143108+0000","last_became_peered":"2024-04-17T10:39:14.143108+0000","last_unstale":"2024-04-17T10:39:14.147727+0000","last_undegraded":"2024-04-17T10:39:14.147727+0000","last_fullsized":"2024-04-17T10:39:14.147727+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:24:48.495765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.153383+0000","last_change":"2024-04-17T10:39:13.139826+0000","last_active":"2024-04-17T10:39:14.153383+0000","last_peered":"2024-04-17T10:39:14.153383+0000","last_clean":"2024-04-17T10:39:14.153383+0000","last_became_active":"2024-04-17T10:39:13.138981+0000","last_became_peered":"2024-04-17T10:39:13.138981+0000","last_unstale":"2024-04-17T10:39:14.153383+0000","last_undegraded":"2024-04-17T10:39:14.153383+0000","last_fullsized":"2024-04-17T10:39:14.153383+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:10:15.558571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.129169+0000","last_change":"2024-04-17T10:39:14.122559+0000","last_active":"2024-04-17T10:39:14.129169+0000","last_peered":"2024-04-17T10:39:14.129169+0000","last_clean":"2024-04-17T10:39:14.129169+0000","last_became_active":"2024-04-17T10:39:14.122293+0000","last_became_peered":"2024-04-17T10:39:14.122293+0000","last_unstale":"2024-04-17T10:39:14.129169+0000","last_undegraded":"2024-04-17T10:39:14.129169+0000","last_fullsized":"2024-04-17T10:39:14.129169+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:12:05.386724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.113037+0000","last_change":"2024-04-17T10:39:13.133706+0000","last_active":"2024-04-17T10:39:14.113037+0000","last_peered":"2024-04-17T10:39:14.113037+0000","last_clean":"2024-04-17T10:39:14.113037+0000","last_became_active":"2024-04-17T10:39:13.133182+0000","last_became_peered":"2024-04-17T10:39:13.133182+0000","last_unstale":"2024-04-17T10:39:14.113037+0000","last_undegraded":"2024-04-17T10:39:14.113037+0000","last_fullsized":"2024-04-17T10:39:14.113037+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:45:46.186394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.162707+0000","last_change":"2024-04-17T10:39:14.160724+0000","last_active":"2024-04-17T10:39:14.162707+0000","last_peered":"2024-04-17T10:39:14.162707+0000","last_clean":"2024-04-17T10:39:14.162707+0000","last_became_active":"2024-04-17T10:39:14.160182+0000","last_became_peered":"2024-04-17T10:39:14.160182+0000","last_unstale":"2024-04-17T10:39:14.162707+0000","last_undegraded":"2024-04-17T10:39:14.162707+0000","last_fullsized":"2024-04-17T10:39:14.162707+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:13:32.981384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.105130+0000","last_change":"2024-04-17T10:39:13.131733+0000","last_active":"2024-04-17T10:39:14.105130+0000","last_peered":"2024-04-17T10:39:14.105130+0000","last_clean":"2024-04-17T10:39:14.105130+0000","last_became_active":"2024-04-17T10:39:13.131494+0000","last_became_peered":"2024-04-17T10:39:13.131494+0000","last_unstale":"2024-04-17T10:39:14.105130+0000","last_undegraded":"2024-04-17T10:39:14.105130+0000","last_fullsized":"2024-04-17T10:39:14.105130+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:38.266060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.162425+0000","last_change":"2024-04-17T10:39:14.157681+0000","last_active":"2024-04-17T10:39:14.162425+0000","last_peered":"2024-04-17T10:39:14.162425+0000","last_clean":"2024-04-17T10:39:14.162425+0000","last_became_active":"2024-04-17T10:39:14.156655+0000","last_became_peered":"2024-04-17T10:39:14.156655+0000","last_unstale":"2024-04-17T10:39:14.162425+0000","last_undegraded":"2024-04-17T10:39:14.162425+0000","last_fullsized":"2024-04-17T10:39:14.162425+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:39.269987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.132561+0000","last_change":"2024-04-17T10:39:13.120068+0000","last_active":"2024-04-17T10:39:14.132561+0000","last_peered":"2024-04-17T10:39:14.132561+0000","last_clean":"2024-04-17T10:39:14.132561+0000","last_became_active":"2024-04-17T10:39:13.119711+0000","last_became_peered":"2024-04-17T10:39:13.119711+0000","last_unstale":"2024-04-17T10:39:14.132561+0000","last_undegraded":"2024-04-17T10:39:14.132561+0000","last_fullsized":"2024-04-17T10:39:14.132561+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:30:33.068918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.153777+0000","last_change":"2024-04-17T10:39:14.145653+0000","last_active":"2024-04-17T10:39:14.153777+0000","last_peered":"2024-04-17T10:39:14.153777+0000","last_clean":"2024-04-17T10:39:14.153777+0000","last_became_active":"2024-04-17T10:39:14.144936+0000","last_became_peered":"2024-04-17T10:39:14.144936+0000","last_unstale":"2024-04-17T10:39:14.153777+0000","last_undegraded":"2024-04-17T10:39:14.153777+0000","last_fullsized":"2024-04-17T10:39:14.153777+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:57:53.655517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.131017+0000","last_change":"2024-04-17T10:39:13.128344+0000","last_active":"2024-04-17T10:39:14.131017+0000","last_peered":"2024-04-17T10:39:14.131017+0000","last_clean":"2024-04-17T10:39:14.131017+0000","last_became_active":"2024-04-17T10:39:13.127941+0000","last_became_peered":"2024-04-17T10:39:13.127941+0000","last_unstale":"2024-04-17T10:39:14.131017+0000","last_undegraded":"2024-04-17T10:39:14.131017+0000","last_fullsized":"2024-04-17T10:39:14.131017+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:17:00.168606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.159341+0000","last_change":"2024-04-17T10:39:14.156339+0000","last_active":"2024-04-17T10:39:14.159341+0000","last_peered":"2024-04-17T10:39:14.159341+0000","last_clean":"2024-04-17T10:39:14.159341+0000","last_became_active":"2024-04-17T10:39:14.155669+0000","last_became_peered":"2024-04-17T10:39:14.155669+0000","last_unstale":"2024-04-17T10:39:14.159341+0000","last_undegraded":"2024-04-17T10:39:14.159341+0000","last_fullsized":"2024-04-17T10:39:14.159341+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:41:36.240202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.153667+0000","last_change":"2024-04-17T10:39:13.145192+0000","last_active":"2024-04-17T10:39:14.153667+0000","last_peered":"2024-04-17T10:39:14.153667+0000","last_clean":"2024-04-17T10:39:14.153667+0000","last_became_active":"2024-04-17T10:39:13.144002+0000","last_became_peered":"2024-04-17T10:39:13.144002+0000","last_unstale":"2024-04-17T10:39:14.153667+0000","last_undegraded":"2024-04-17T10:39:14.153667+0000","last_fullsized":"2024-04-17T10:39:14.153667+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:03:43.254527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.142155+0000","last_change":"2024-04-17T10:39:14.139357+0000","last_active":"2024-04-17T10:39:14.142155+0000","last_peered":"2024-04-17T10:39:14.142155+0000","last_clean":"2024-04-17T10:39:14.142155+0000","last_became_active":"2024-04-17T10:39:14.139073+0000","last_became_peered":"2024-04-17T10:39:14.139073+0000","last_unstale":"2024-04-17T10:39:14.142155+0000","last_undegraded":"2024-04-17T10:39:14.142155+0000","last_fullsized":"2024-04-17T10:39:14.142155+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:57:53.655517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.148320+0000","last_change":"2024-04-17T10:39:13.128103+0000","last_active":"2024-04-17T10:39:14.148320+0000","last_peered":"2024-04-17T10:39:14.148320+0000","last_clean":"2024-04-17T10:39:14.148320+0000","last_became_active":"2024-04-17T10:39:13.127167+0000","last_became_peered":"2024-04-17T10:39:13.127167+0000","last_unstale":"2024-04-17T10:39:14.148320+0000","last_undegraded":"2024-04-17T10:39:14.148320+0000","last_fullsized":"2024-04-17T10:39:14.148320+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:18:28.175123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.158288+0000","last_change":"2024-04-17T10:39:14.153894+0000","last_active":"2024-04-17T10:39:14.158288+0000","last_peered":"2024-04-17T10:39:14.158288+0000","last_clean":"2024-04-17T10:39:14.158288+0000","last_became_active":"2024-04-17T10:39:14.153591+0000","last_became_peered":"2024-04-17T10:39:14.153591+0000","last_unstale":"2024-04-17T10:39:14.158288+0000","last_undegraded":"2024-04-17T10:39:14.158288+0000","last_fullsized":"2024-04-17T10:39:14.158288+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:52:10.195171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.108733+0000","last_change":"2024-04-17T10:39:13.139001+0000","last_active":"2024-04-17T10:39:14.108733+0000","last_peered":"2024-04-17T10:39:14.108733+0000","last_clean":"2024-04-17T10:39:14.108733+0000","last_became_active":"2024-04-17T10:39:13.138189+0000","last_became_peered":"2024-04-17T10:39:13.138189+0000","last_unstale":"2024-04-17T10:39:14.108733+0000","last_undegraded":"2024-04-17T10:39:14.108733+0000","last_fullsized":"2024-04-17T10:39:14.108733+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:53:42.646220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.141744+0000","last_change":"2024-04-17T10:39:14.139769+0000","last_active":"2024-04-17T10:39:14.141744+0000","last_peered":"2024-04-17T10:39:14.141744+0000","last_clean":"2024-04-17T10:39:14.141744+0000","last_became_active":"2024-04-17T10:39:14.139461+0000","last_became_peered":"2024-04-17T10:39:14.139461+0000","last_unstale":"2024-04-17T10:39:14.141744+0000","last_undegraded":"2024-04-17T10:39:14.141744+0000","last_fullsized":"2024-04-17T10:39:14.141744+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:23:14.826218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.131831+0000","last_change":"2024-04-17T10:39:13.120631+0000","last_active":"2024-04-17T10:39:14.131831+0000","last_peered":"2024-04-17T10:39:14.131831+0000","last_clean":"2024-04-17T10:39:14.131831+0000","last_became_active":"2024-04-17T10:39:13.120018+0000","last_became_peered":"2024-04-17T10:39:13.120018+0000","last_unstale":"2024-04-17T10:39:14.131831+0000","last_undegraded":"2024-04-17T10:39:14.131831+0000","last_fullsized":"2024-04-17T10:39:14.131831+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:17:00.168606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.129908+0000","last_change":"2024-04-17T10:39:14.127220+0000","last_active":"2024-04-17T10:39:14.129908+0000","last_peered":"2024-04-17T10:39:14.129908+0000","last_clean":"2024-04-17T10:39:14.129908+0000","last_became_active":"2024-04-17T10:39:14.126926+0000","last_became_peered":"2024-04-17T10:39:14.126926+0000","last_unstale":"2024-04-17T10:39:14.129908+0000","last_undegraded":"2024-04-17T10:39:14.129908+0000","last_fullsized":"2024-04-17T10:39:14.129908+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:18:29.179050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.136271+0000","last_change":"2024-04-17T10:39:13.135178+0000","last_active":"2024-04-17T10:39:14.136271+0000","last_peered":"2024-04-17T10:39:14.136271+0000","last_clean":"2024-04-17T10:39:14.136271+0000","last_became_active":"2024-04-17T10:39:13.134490+0000","last_became_peered":"2024-04-17T10:39:13.134490+0000","last_unstale":"2024-04-17T10:39:14.136271+0000","last_undegraded":"2024-04-17T10:39:14.136271+0000","last_fullsized":"2024-04-17T10:39:14.136271+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:45:46.186394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.120606+0000","last_change":"2024-04-17T10:39:14.118147+0000","last_active":"2024-04-17T10:39:14.120606+0000","last_peered":"2024-04-17T10:39:14.120606+0000","last_clean":"2024-04-17T10:39:14.120606+0000","last_became_active":"2024-04-17T10:39:14.117781+0000","last_became_peered":"2024-04-17T10:39:14.117781+0000","last_unstale":"2024-04-17T10:39:14.120606+0000","last_undegraded":"2024-04-17T10:39:14.120606+0000","last_fullsized":"2024-04-17T10:39:14.120606+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:40:40.684393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.111353+0000","last_change":"2024-04-17T10:39:13.126741+0000","last_active":"2024-04-17T10:39:14.111353+0000","last_peered":"2024-04-17T10:39:14.111353+0000","last_clean":"2024-04-17T10:39:14.111353+0000","last_became_active":"2024-04-17T10:39:13.124777+0000","last_became_peered":"2024-04-17T10:39:13.124777+0000","last_unstale":"2024-04-17T10:39:14.111353+0000","last_undegraded":"2024-04-17T10:39:14.111353+0000","last_fullsized":"2024-04-17T10:39:14.111353+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:45:46.186394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.162946+0000","last_change":"2024-04-17T10:39:14.153983+0000","last_active":"2024-04-17T10:39:14.162946+0000","last_peered":"2024-04-17T10:39:14.162946+0000","last_clean":"2024-04-17T10:39:14.162946+0000","last_became_active":"2024-04-17T10:39:14.150196+0000","last_became_peered":"2024-04-17T10:39:14.150196+0000","last_unstale":"2024-04-17T10:39:14.162946+0000","last_undegraded":"2024-04-17T10:39:14.162946+0000","last_fullsized":"2024-04-17T10:39:14.162946+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:28:23.922532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.119377+0000","last_change":"2024-04-17T10:39:13.133409+0000","last_active":"2024-04-17T10:39:14.119377+0000","last_peered":"2024-04-17T10:39:14.119377+0000","last_clean":"2024-04-17T10:39:14.119377+0000","last_became_active":"2024-04-17T10:39:13.132490+0000","last_became_peered":"2024-04-17T10:39:13.132490+0000","last_unstale":"2024-04-17T10:39:14.119377+0000","last_undegraded":"2024-04-17T10:39:14.119377+0000","last_fullsized":"2024-04-17T10:39:14.119377+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:14:31.765076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.147548+0000","last_change":"2024-04-17T10:39:14.143153+0000","last_active":"2024-04-17T10:39:14.147548+0000","last_peered":"2024-04-17T10:39:14.147548+0000","last_clean":"2024-04-17T10:39:14.147548+0000","last_became_active":"2024-04-17T10:39:14.142895+0000","last_became_peered":"2024-04-17T10:39:14.142895+0000","last_unstale":"2024-04-17T10:39:14.147548+0000","last_undegraded":"2024-04-17T10:39:14.147548+0000","last_fullsized":"2024-04-17T10:39:14.147548+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:53:43.650148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.146468+0000","last_change":"2024-04-17T10:39:13.133488+0000","last_active":"2024-04-17T10:39:14.146468+0000","last_peered":"2024-04-17T10:39:14.146468+0000","last_clean":"2024-04-17T10:39:14.146468+0000","last_became_active":"2024-04-17T10:39:13.133148+0000","last_became_peered":"2024-04-17T10:39:13.133148+0000","last_unstale":"2024-04-17T10:39:14.146468+0000","last_undegraded":"2024-04-17T10:39:14.146468+0000","last_fullsized":"2024-04-17T10:39:14.146468+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:14.158542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.153284+0000","last_change":"2024-04-17T10:39:14.145382+0000","last_active":"2024-04-17T10:39:14.153284+0000","last_peered":"2024-04-17T10:39:14.153284+0000","last_clean":"2024-04-17T10:39:14.153284+0000","last_became_active":"2024-04-17T10:39:14.142759+0000","last_became_peered":"2024-04-17T10:39:14.142759+0000","last_unstale":"2024-04-17T10:39:14.153284+0000","last_undegraded":"2024-04-17T10:39:14.153284+0000","last_fullsized":"2024-04-17T10:39:14.153284+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:13:32.981384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.116025+0000","last_change":"2024-04-17T10:39:13.116593+0000","last_active":"2024-04-17T10:39:14.116025+0000","last_peered":"2024-04-17T10:39:14.116025+0000","last_clean":"2024-04-17T10:39:14.116025+0000","last_became_active":"2024-04-17T10:39:13.116212+0000","last_became_peered":"2024-04-17T10:39:13.116212+0000","last_unstale":"2024-04-17T10:39:14.116025+0000","last_undegraded":"2024-04-17T10:39:14.116025+0000","last_fullsized":"2024-04-17T10:39:14.116025+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:03:30.756764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.121035+0000","last_change":"2024-04-17T10:39:14.118830+0000","last_active":"2024-04-17T10:39:14.121035+0000","last_peered":"2024-04-17T10:39:14.121035+0000","last_clean":"2024-04-17T10:39:14.121035+0000","last_became_active":"2024-04-17T10:39:14.118371+0000","last_became_peered":"2024-04-17T10:39:14.118371+0000","last_unstale":"2024-04-17T10:39:14.121035+0000","last_undegraded":"2024-04-17T10:39:14.121035+0000","last_fullsized":"2024-04-17T10:39:14.121035+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:39.269987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.114892+0000","last_change":"2024-04-17T10:39:13.122418+0000","last_active":"2024-04-17T10:39:14.114892+0000","last_peered":"2024-04-17T10:39:14.114892+0000","last_clean":"2024-04-17T10:39:14.114892+0000","last_became_active":"2024-04-17T10:39:13.121969+0000","last_became_peered":"2024-04-17T10:39:13.121969+0000","last_unstale":"2024-04-17T10:39:14.114892+0000","last_undegraded":"2024-04-17T10:39:14.114892+0000","last_fullsized":"2024-04-17T10:39:14.114892+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:03:30.756764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.159347+0000","last_change":"2024-04-17T10:39:14.156982+0000","last_active":"2024-04-17T10:39:14.159347+0000","last_peered":"2024-04-17T10:39:14.159347+0000","last_clean":"2024-04-17T10:39:14.159347+0000","last_became_active":"2024-04-17T10:39:14.155095+0000","last_became_peered":"2024-04-17T10:39:14.155095+0000","last_unstale":"2024-04-17T10:39:14.159347+0000","last_undegraded":"2024-04-17T10:39:14.159347+0000","last_fullsized":"2024-04-17T10:39:14.159347+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:03:31.760691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.114375+0000","last_change":"2024-04-17T10:39:13.126248+0000","last_active":"2024-04-17T10:39:14.114375+0000","last_peered":"2024-04-17T10:39:14.114375+0000","last_clean":"2024-04-17T10:39:14.114375+0000","last_became_active":"2024-04-17T10:39:13.125917+0000","last_became_peered":"2024-04-17T10:39:13.125917+0000","last_unstale":"2024-04-17T10:39:14.114375+0000","last_undegraded":"2024-04-17T10:39:14.114375+0000","last_fullsized":"2024-04-17T10:39:14.114375+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:58:06.016900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.147498+0000","last_change":"2024-04-17T10:39:14.142456+0000","last_active":"2024-04-17T10:39:14.147498+0000","last_peered":"2024-04-17T10:39:14.147498+0000","last_clean":"2024-04-17T10:39:14.147498+0000","last_became_active":"2024-04-17T10:39:14.142187+0000","last_became_peered":"2024-04-17T10:39:14.142187+0000","last_unstale":"2024-04-17T10:39:14.147498+0000","last_undegraded":"2024-04-17T10:39:14.147498+0000","last_fullsized":"2024-04-17T10:39:14.147498+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:57:25.113157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.113505+0000","last_change":"2024-04-17T10:39:13.133461+0000","last_active":"2024-04-17T10:39:14.113505+0000","last_peered":"2024-04-17T10:39:14.113505+0000","last_clean":"2024-04-17T10:39:14.113505+0000","last_became_active":"2024-04-17T10:39:13.133189+0000","last_became_peered":"2024-04-17T10:39:13.133189+0000","last_unstale":"2024-04-17T10:39:14.113505+0000","last_undegraded":"2024-04-17T10:39:14.113505+0000","last_fullsized":"2024-04-17T10:39:14.113505+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:58:06.016900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.127541+0000","last_change":"2024-04-17T10:39:14.117643+0000","last_active":"2024-04-17T10:39:14.127541+0000","last_peered":"2024-04-17T10:39:14.127541+0000","last_clean":"2024-04-17T10:39:14.127541+0000","last_became_active":"2024-04-17T10:39:14.117355+0000","last_became_peered":"2024-04-17T10:39:14.117355+0000","last_unstale":"2024-04-17T10:39:14.127541+0000","last_undegraded":"2024-04-17T10:39:14.127541+0000","last_fullsized":"2024-04-17T10:39:14.127541+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:14:32.769003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.111757+0000","last_change":"2024-04-17T10:39:13.138057+0000","last_active":"2024-04-17T10:39:14.111757+0000","last_peered":"2024-04-17T10:39:14.111757+0000","last_clean":"2024-04-17T10:39:14.111757+0000","last_became_active":"2024-04-17T10:39:13.130733+0000","last_became_peered":"2024-04-17T10:39:13.130733+0000","last_unstale":"2024-04-17T10:39:14.111757+0000","last_undegraded":"2024-04-17T10:39:14.111757+0000","last_fullsized":"2024-04-17T10:39:14.111757+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:17:00.168606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.153460+0000","last_change":"2024-04-17T10:39:14.137840+0000","last_active":"2024-04-17T10:39:14.153460+0000","last_peered":"2024-04-17T10:39:14.153460+0000","last_clean":"2024-04-17T10:39:14.153460+0000","last_became_active":"2024-04-17T10:39:14.128627+0000","last_became_peered":"2024-04-17T10:39:14.128627+0000","last_unstale":"2024-04-17T10:39:14.153460+0000","last_undegraded":"2024-04-17T10:39:14.153460+0000","last_fullsized":"2024-04-17T10:39:14.153460+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:45:43.557201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.105523+0000","last_change":"2024-04-17T10:39:13.130880+0000","last_active":"2024-04-17T10:39:14.105523+0000","last_peered":"2024-04-17T10:39:14.105523+0000","last_clean":"2024-04-17T10:39:14.105523+0000","last_became_active":"2024-04-17T10:39:13.130434+0000","last_became_peered":"2024-04-17T10:39:13.130434+0000","last_unstale":"2024-04-17T10:39:14.105523+0000","last_undegraded":"2024-04-17T10:39:14.105523+0000","last_fullsized":"2024-04-17T10:39:14.105523+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:57:52.651590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.154051+0000","last_change":"2024-04-17T10:39:14.140608+0000","last_active":"2024-04-17T10:39:14.154051+0000","last_peered":"2024-04-17T10:39:14.154051+0000","last_clean":"2024-04-17T10:39:14.154051+0000","last_became_active":"2024-04-17T10:39:14.139887+0000","last_became_peered":"2024-04-17T10:39:14.139887+0000","last_unstale":"2024-04-17T10:39:14.154051+0000","last_undegraded":"2024-04-17T10:39:14.154051+0000","last_fullsized":"2024-04-17T10:39:14.154051+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:14:32.769003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.132716+0000","last_change":"2024-04-17T10:39:13.144322+0000","last_active":"2024-04-17T10:39:14.132716+0000","last_peered":"2024-04-17T10:39:14.132716+0000","last_clean":"2024-04-17T10:39:14.132716+0000","last_became_active":"2024-04-17T10:39:13.143607+0000","last_became_peered":"2024-04-17T10:39:13.143607+0000","last_unstale":"2024-04-17T10:39:14.132716+0000","last_undegraded":"2024-04-17T10:39:14.132716+0000","last_fullsized":"2024-04-17T10:39:14.132716+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:31:02.964758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":194,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.115048+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:39:14.115048+0000","last_peered":"2024-04-17T10:39:14.115048+0000","last_clean":"2024-04-17T10:39:14.115048+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:39:14.115048+0000","last_undegraded":"2024-04-17T10:39:14.115048+0000","last_fullsized":"2024-04-17T10:39:14.115048+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":140980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":140412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:39:17.169 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:39:17.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.326+0000 7f794156a640 1 Processor -- start 2024-04-17T10:39:17.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.326+0000 7f794156a640 1 -- start start 2024-04-17T10:39:17.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.326+0000 7f794156a640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f793c0cfbe0 0x7f793c06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:17.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.326+0000 7f794156a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f793c06e320 con 0x7f793c15ba30 2024-04-17T10:39:17.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.326+0000 7f794156a640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f793c06e040 con 0x7f793c072330 2024-04-17T10:39:17.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.326+0000 7f794156a640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f793c071ea0 con 0x7f793c0cfbe0 2024-04-17T10:39:17.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.326+0000 7f793affd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f793c15ba30 0x7f793c15c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54060/0 (socket says 172.21.15.83:54060) 2024-04-17T10:39:17.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.326+0000 7f793affd640 1 -- 172.21.15.83:0/569399842 learned_addr learned my addr 172.21.15.83:0/569399842 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:39:17.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.326+0000 7f7939ffb640 1 -- 172.21.15.83:0/569399842 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4086289440 0 0) 0x7f793c06e320 con 0x7f793c15ba30 2024-04-17T10:39:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.326+0000 7f7939ffb640 1 -- 172.21.15.83:0/569399842 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7924003530 con 0x7f793c15ba30 2024-04-17T10:39:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.326+0000 7f793b7fe640 1 --2- 172.21.15.83:0/569399842 >> v2:172.21.15.167:3300/0 conn(0x7f793c0cfbe0 0x7f793c06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.326+0000 7f7939ffb640 1 -- 172.21.15.83:0/569399842 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2204859392 0 0) 0x7f7924003530 con 0x7f793c15ba30 2024-04-17T10:39:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.326+0000 7f7939ffb640 1 -- 172.21.15.83:0/569399842 >> v2:172.21.15.167:3300/0 conn(0x7f793c0cfbe0 msgr2=0x7f793c06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.326+0000 7f7939ffb640 1 --2- 172.21.15.83:0/569399842 >> v2:172.21.15.167:3300/0 conn(0x7f793c0cfbe0 0x7f793c06c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.326+0000 7f7939ffb640 1 -- 172.21.15.83:0/569399842 >> v1:172.21.15.167:6789/0 conn(0x7f793c072330 legacy=0x7f793c0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.326+0000 7f7939ffb640 1 -- 172.21.15.83:0/569399842 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793c071850 con 0x7f793c15ba30 2024-04-17T10:39:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.326+0000 7f7939ffb640 1 -- 172.21.15.83:0/569399842 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f79300016a0 con 0x7f793c15ba30 2024-04-17T10:39:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.326+0000 7f7939ffb640 1 -- 172.21.15.83:0/569399842 >> v1:172.21.15.83:6789/0 conn(0x7f793c15ba30 legacy=0x7f793c15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.326+0000 7f7939ffb640 1 --2- 172.21.15.83:0/569399842 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79240039c0 0x7f7924003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.326+0000 7f7939ffb640 1 --2- 172.21.15.83:0/569399842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7924004810 0x7f7924004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.326+0000 7f7939ffb640 1 --2- 172.21.15.83:0/569399842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79240052f0 0x7f792400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.326+0000 7f7939ffb640 1 -- 172.21.15.83:0/569399842 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c071ea0 con 0x7f79240052f0 2024-04-17T10:39:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.326+0000 7f7939ffb640 1 -- 172.21.15.83:0/569399842 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792400c1a0 con 0x7f7924004810 2024-04-17T10:39:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.326+0000 7f7939ffb640 1 -- 172.21.15.83:0/569399842 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f792400c3a0 con 0x7f79240039c0 2024-04-17T10:39:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.326+0000 7f793b7fe640 1 --2- 172.21.15.83:0/569399842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79240052f0 0x7f792400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.326+0000 7f793a7fc640 1 --2- 172.21.15.83:0/569399842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7924004810 0x7f7924004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.326+0000 7f793b7fe640 1 -- 172.21.15.83:0/569399842 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79240039c0 msgr2=0x7f7924003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.326+0000 7f793b7fe640 1 --2- 172.21.15.83:0/569399842 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79240039c0 0x7f7924003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.326+0000 7f793b7fe640 1 -- 172.21.15.83:0/569399842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7924004810 msgr2=0x7f7924004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.326+0000 7f793b7fe640 1 --2- 172.21.15.83:0/569399842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7924004810 0x7f7924004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.326+0000 7f793b7fe640 1 -- 172.21.15.83:0/569399842 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7924003530 con 0x7f79240052f0 2024-04-17T10:39:17.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.330+0000 7f793b7fe640 1 -- 172.21.15.83:0/569399842 reap_dead start 2024-04-17T10:39:17.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.330+0000 7f793b7fe640 1 --2- 172.21.15.83:0/569399842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79240052f0 0x7f792400b9d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7934007330 tx=0x7f793401ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8658a344312d5301 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:17.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.330+0000 7f7939ffb640 1 -- 172.21.15.83:0/569399842 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7934009be0 con 0x7f79240052f0 2024-04-17T10:39:17.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.330+0000 7f7939ffb640 1 -- 172.21.15.83:0/569399842 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79340a5d60 con 0x7f79240052f0 2024-04-17T10:39:17.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.330+0000 7f7939ffb640 1 -- 172.21.15.83:0/569399842 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7934003be0 con 0x7f79240052f0 2024-04-17T10:39:17.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.330+0000 7f794156a640 1 -- 172.21.15.83:0/569399842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79240052f0 msgr2=0x7f792400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:17.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.330+0000 7f794156a640 1 --2- 172.21.15.83:0/569399842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79240052f0 0x7f792400b9d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7934007330 tx=0x7f793401ce60 comp rx=0 tx=0).stop 2024-04-17T10:39:17.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.330+0000 7f794156a640 1 -- 172.21.15.83:0/569399842 shutdown_connections 2024-04-17T10:39:17.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.330+0000 7f794156a640 1 --2- 172.21.15.83:0/569399842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79240052f0 0x7f792400b9d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:17.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.330+0000 7f794156a640 1 -- 172.21.15.83:0/569399842 >> 172.21.15.83:0/569399842 conn(0x7f793c06abe0 msgr2=0x7f793c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:17.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.330+0000 7f794156a640 1 -- 172.21.15.83:0/569399842 shutdown_connections 2024-04-17T10:39:17.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.330+0000 7f794156a640 1 -- 172.21.15.83:0/569399842 wait complete. 2024-04-17T10:39:17.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.330+0000 7f794156a640 1 Processor -- start 2024-04-17T10:39:17.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.330+0000 7f794156a640 1 -- start start 2024-04-17T10:39:17.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.330+0000 7f794156a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79240052f0 0x7f793c15dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:17.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.330+0000 7f794156a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f793c14f020 0x7f793c14f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:17.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.330+0000 7f794156a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f793c15be10 0x7f793c15c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:17.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.330+0000 7f794156a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c06dca0 con 0x7f793c14f020 2024-04-17T10:39:17.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.330+0000 7f794156a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c0aeae0 con 0x7f793c15be10 2024-04-17T10:39:17.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.330+0000 7f794156a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f793c06de20 con 0x7f79240052f0 2024-04-17T10:39:17.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.330+0000 7f793a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f793c14f020 0x7f793c14f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:17.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.330+0000 7f793a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f793c14f020 0x7f793c14f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59910/0 (socket says 172.21.15.83:59910) 2024-04-17T10:39:17.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.330+0000 7f793a7fc640 1 -- 172.21.15.83:0/1060291077 learned_addr learned my addr 172.21.15.83:0/1060291077 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:17.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.334+0000 7f793b7fe640 1 --2- 172.21.15.83:0/1060291077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f793c15be10 0x7f793c15c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:17.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.334+0000 7f793a7fc640 1 -- 172.21.15.83:0/1060291077 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79240052f0 msgr2=0x7f793c15dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:17.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.334+0000 7f793affd640 1 --2- 172.21.15.83:0/1060291077 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79240052f0 0x7f793c15dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:17.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.334+0000 7f793a7fc640 1 --2- 172.21.15.83:0/1060291077 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79240052f0 0x7f793c15dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:17.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.334+0000 7f793a7fc640 1 -- 172.21.15.83:0/1060291077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f793c15be10 msgr2=0x7f793c15c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:17.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.334+0000 7f793a7fc640 1 --2- 172.21.15.83:0/1060291077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f793c15be10 0x7f793c15c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:17.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.334+0000 7f793a7fc640 1 -- 172.21.15.83:0/1060291077 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793c14fd90 con 0x7f793c14f020 2024-04-17T10:39:17.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.334+0000 7f793a7fc640 1 --2- 172.21.15.83:0/1060291077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f793c14f020 0x7f793c14f4a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f792c00e490 tx=0x7f792c00e950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:17.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.334+0000 7f79237fe640 1 -- 172.21.15.83:0/1060291077 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c0135f0 con 0x7f793c14f020 2024-04-17T10:39:17.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.334+0000 7f79237fe640 1 -- 172.21.15.83:0/1060291077 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f792c019070 con 0x7f793c14f020 2024-04-17T10:39:17.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.334+0000 7f794156a640 1 -- 172.21.15.83:0/1060291077 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f793c150070 con 0x7f793c14f020 2024-04-17T10:39:17.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.334+0000 7f79237fe640 1 -- 172.21.15.83:0/1060291077 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c0139d0 con 0x7f793c14f020 2024-04-17T10:39:17.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.334+0000 7f794156a640 1 -- 172.21.15.83:0/1060291077 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f793c150630 con 0x7f793c14f020 2024-04-17T10:39:17.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.334+0000 7f794156a640 1 -- 172.21.15.83:0/1060291077 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f793c074f90 con 0x7f793c14f020 2024-04-17T10:39:17.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.334+0000 7f79237fe640 1 -- 172.21.15.83:0/1060291077 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f792c013b70 con 0x7f793c14f020 2024-04-17T10:39:17.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.334+0000 7f79237fe640 1 --2- 172.21.15.83:0/1060291077 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7928066f60 0x7f7928069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:17.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.334+0000 7f793affd640 1 --2- 172.21.15.83:0/1060291077 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7928066f60 0x7f7928069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:17.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.334+0000 7f79237fe640 1 -- 172.21.15.83:0/1060291077 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f792c084ba0 con 0x7f793c14f020 2024-04-17T10:39:17.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.334+0000 7f793affd640 1 --2- 172.21.15.83:0/1060291077 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7928066f60 0x7f7928069420 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7930001890 tx=0x7f793000c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:17.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.338+0000 7f79237fe640 1 -- 172.21.15.83:0/1060291077 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f792c022020 con 0x7f793c14f020 2024-04-17T10:39:17.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.482+0000 7f794156a640 1 -- 172.21.15.83:0/1060291077 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f793c15a610 con 0x7f7928066f60 2024-04-17T10:39:17.487 INFO:tasks.ceph.mds.a.smithi083.stderr:2024-04-17T10:39:17.490+0000 7fb6cbd91640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T10:39:17.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.494+0000 7f79237fe640 1 -- 172.21.15.83:0/1060291077 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251888 (secure 0 0 0) 0x7f793c15a610 con 0x7f7928066f60 2024-04-17T10:39:17.494 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:39:17.496 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:39:17.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.498+0000 7f794156a640 1 -- 172.21.15.83:0/1060291077 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7928066f60 msgr2=0x7f7928069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:17.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.498+0000 7f794156a640 1 --2- 172.21.15.83:0/1060291077 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7928066f60 0x7f7928069420 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7930001890 tx=0x7f793000c040 comp rx=0 tx=0).stop 2024-04-17T10:39:17.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.498+0000 7f794156a640 1 -- 172.21.15.83:0/1060291077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f793c14f020 msgr2=0x7f793c14f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:17.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.498+0000 7f794156a640 1 --2- 172.21.15.83:0/1060291077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f793c14f020 0x7f793c14f4a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f792c00e490 tx=0x7f792c00e950 comp rx=0 tx=0).stop 2024-04-17T10:39:17.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.498+0000 7f794156a640 1 -- 172.21.15.83:0/1060291077 shutdown_connections 2024-04-17T10:39:17.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.498+0000 7f794156a640 1 --2- 172.21.15.83:0/1060291077 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7928066f60 0x7f7928069420 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:17.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.498+0000 7f794156a640 1 --2- 172.21.15.83:0/1060291077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f793c15be10 0x7f793c15c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:17.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.498+0000 7f794156a640 1 --2- 172.21.15.83:0/1060291077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f793c14f020 0x7f793c14f4a0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:17.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.498+0000 7f794156a640 1 --2- 172.21.15.83:0/1060291077 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79240052f0 0x7f793c15dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:17.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.498+0000 7f794156a640 1 -- 172.21.15.83:0/1060291077 >> 172.21.15.83:0/1060291077 conn(0x7f793c06abe0 msgr2=0x7f793c0cd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:17.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.498+0000 7f794156a640 1 -- 172.21.15.83:0/1060291077 shutdown_connections 2024-04-17T10:39:17.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.498+0000 7f794156a640 1 -- 172.21.15.83:0/1060291077 wait complete. 2024-04-17T10:39:17.512 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-04-17T10:39:16.197770+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":67,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220100,"kb_used_data":3920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754620,"statfs":{"total":773094113280,"available":772868730880,"internally_reserved":0,"allocated":4014080,"data_stored":2263326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":47,"apply_latency_ms":47,"commit_latency_ns":47000000,"apply_latency_ns":47000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.106743"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.157736+0000","last_change":"2024-04-17T10:39:14.149806+0000","last_active":"2024-04-17T10:39:14.157736+0000","last_peered":"2024-04-17T10:39:14.157736+0000","last_clean":"2024-04-17T10:39:14.157736+0000","last_became_active":"2024-04-17T10:39:14.147423+0000","last_became_peered":"2024-04-17T10:39:14.147423+0000","last_unstale":"2024-04-17T10:39:14.157736+0000","last_undegraded":"2024-04-17T10:39:14.157736+0000","last_fullsized":"2024-04-17T10:39:14.157736+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:58:07.020827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.130156+0000","last_change":"2024-04-17T10:39:14.122516+0000","last_active":"2024-04-17T10:39:14.130156+0000","last_peered":"2024-04-17T10:39:14.130156+0000","last_clean":"2024-04-17T10:39:14.130156+0000","last_became_active":"2024-04-17T10:39:14.122216+0000","last_became_peered":"2024-04-17T10:39:14.122216+0000","last_unstale":"2024-04-17T10:39:14.130156+0000","last_undegraded":"2024-04-17T10:39:14.130156+0000","last_fullsized":"2024-04-17T10:39:14.130156+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:23.357477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.143512+0000","last_change":"2024-04-17T10:39:14.134180+0000","last_active":"2024-04-17T10:39:14.143512+0000","last_peered":"2024-04-17T10:39:14.143512+0000","last_clean":"2024-04-17T10:39:14.143512+0000","last_became_active":"2024-04-17T10:39:14.132417+0000","last_became_peered":"2024-04-17T10:39:14.132417+0000","last_unstale":"2024-04-17T10:39:14.143512+0000","last_undegraded":"2024-04-17T10:39:14.143512+0000","last_fullsized":"2024-04-17T10:39:14.143512+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:09:41.478596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.142129+0000","last_change":"2024-04-17T10:39:14.139185+0000","last_active":"2024-04-17T10:39:14.142129+0000","last_peered":"2024-04-17T10:39:14.142129+0000","last_clean":"2024-04-17T10:39:14.142129+0000","last_became_active":"2024-04-17T10:39:14.138869+0000","last_became_peered":"2024-04-17T10:39:14.138869+0000","last_unstale":"2024-04-17T10:39:14.142129+0000","last_undegraded":"2024-04-17T10:39:14.142129+0000","last_fullsized":"2024-04-17T10:39:14.142129+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:45:43.557201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.129884+0000","last_change":"2024-04-17T10:39:14.125870+0000","last_active":"2024-04-17T10:39:14.129884+0000","last_peered":"2024-04-17T10:39:14.129884+0000","last_clean":"2024-04-17T10:39:14.129884+0000","last_became_active":"2024-04-17T10:39:14.125594+0000","last_became_peered":"2024-04-17T10:39:14.125594+0000","last_unstale":"2024-04-17T10:39:14.129884+0000","last_undegraded":"2024-04-17T10:39:14.129884+0000","last_fullsized":"2024-04-17T10:39:14.129884+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:10:44.246912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.141788+0000","last_change":"2024-04-17T10:39:14.136456+0000","last_active":"2024-04-17T10:39:14.141788+0000","last_peered":"2024-04-17T10:39:14.141788+0000","last_clean":"2024-04-17T10:39:14.141788+0000","last_became_active":"2024-04-17T10:39:14.136208+0000","last_became_peered":"2024-04-17T10:39:14.136208+0000","last_unstale":"2024-04-17T10:39:14.141788+0000","last_undegraded":"2024-04-17T10:39:14.141788+0000","last_fullsized":"2024-04-17T10:39:14.141788+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:28:23.922532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.161348+0000","last_change":"2024-04-17T10:39:14.150874+0000","last_active":"2024-04-17T10:39:14.161348+0000","last_peered":"2024-04-17T10:39:14.161348+0000","last_clean":"2024-04-17T10:39:14.161348+0000","last_became_active":"2024-04-17T10:39:14.149282+0000","last_became_peered":"2024-04-17T10:39:14.149282+0000","last_unstale":"2024-04-17T10:39:14.161348+0000","last_undegraded":"2024-04-17T10:39:14.161348+0000","last_fullsized":"2024-04-17T10:39:14.161348+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:12:05.386724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.154613+0000","last_change":"2024-04-17T10:39:14.146974+0000","last_active":"2024-04-17T10:39:14.154613+0000","last_peered":"2024-04-17T10:39:14.154613+0000","last_clean":"2024-04-17T10:39:14.154613+0000","last_became_active":"2024-04-17T10:39:14.139506+0000","last_became_peered":"2024-04-17T10:39:14.139506+0000","last_unstale":"2024-04-17T10:39:14.154613+0000","last_undegraded":"2024-04-17T10:39:14.154613+0000","last_fullsized":"2024-04-17T10:39:14.154613+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:25:15.943792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.151385+0000","last_change":"2024-04-17T10:39:14.119197+0000","last_active":"2024-04-17T10:39:14.151385+0000","last_peered":"2024-04-17T10:39:14.151385+0000","last_clean":"2024-04-17T10:39:14.151385+0000","last_became_active":"2024-04-17T10:39:14.118452+0000","last_became_peered":"2024-04-17T10:39:14.118452+0000","last_unstale":"2024-04-17T10:39:14.151385+0000","last_undegraded":"2024-04-17T10:39:14.151385+0000","last_fullsized":"2024-04-17T10:39:14.151385+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:58:07.020827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.131190+0000","last_change":"2024-04-17T10:39:14.127149+0000","last_active":"2024-04-17T10:39:14.131190+0000","last_peered":"2024-04-17T10:39:14.131190+0000","last_clean":"2024-04-17T10:39:14.131190+0000","last_became_active":"2024-04-17T10:39:14.126573+0000","last_became_peered":"2024-04-17T10:39:14.126573+0000","last_unstale":"2024-04-17T10:39:14.131190+0000","last_undegraded":"2024-04-17T10:39:14.131190+0000","last_fullsized":"2024-04-17T10:39:14.131190+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:39.269987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.154071+0000","last_change":"2024-04-17T10:39:14.140253+0000","last_active":"2024-04-17T10:39:14.154071+0000","last_peered":"2024-04-17T10:39:14.154071+0000","last_clean":"2024-04-17T10:39:14.154071+0000","last_became_active":"2024-04-17T10:39:14.139460+0000","last_became_peered":"2024-04-17T10:39:14.139460+0000","last_unstale":"2024-04-17T10:39:14.154071+0000","last_undegraded":"2024-04-17T10:39:14.154071+0000","last_fullsized":"2024-04-17T10:39:14.154071+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:23:38.450031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.145623+0000","last_change":"2024-04-17T10:39:14.140030+0000","last_active":"2024-04-17T10:39:14.145623+0000","last_peered":"2024-04-17T10:39:14.145623+0000","last_clean":"2024-04-17T10:39:14.145623+0000","last_became_active":"2024-04-17T10:39:14.137672+0000","last_became_peered":"2024-04-17T10:39:14.137672+0000","last_unstale":"2024-04-17T10:39:14.145623+0000","last_undegraded":"2024-04-17T10:39:14.145623+0000","last_fullsized":"2024-04-17T10:39:14.145623+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:02:20.150885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.120750+0000","last_change":"2024-04-17T10:39:14.117285+0000","last_active":"2024-04-17T10:39:14.120750+0000","last_peered":"2024-04-17T10:39:14.120750+0000","last_clean":"2024-04-17T10:39:14.120750+0000","last_became_active":"2024-04-17T10:39:14.116962+0000","last_became_peered":"2024-04-17T10:39:14.116962+0000","last_unstale":"2024-04-17T10:39:14.120750+0000","last_undegraded":"2024-04-17T10:39:14.120750+0000","last_fullsized":"2024-04-17T10:39:14.120750+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:33:09.883101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.123650+0000","last_change":"2024-04-17T10:39:14.114202+0000","last_active":"2024-04-17T10:39:14.123650+0000","last_peered":"2024-04-17T10:39:14.123650+0000","last_clean":"2024-04-17T10:39:14.123650+0000","last_became_active":"2024-04-17T10:39:14.113956+0000","last_became_peered":"2024-04-17T10:39:14.113956+0000","last_unstale":"2024-04-17T10:39:14.123650+0000","last_undegraded":"2024-04-17T10:39:14.123650+0000","last_fullsized":"2024-04-17T10:39:14.123650+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:23:38.450031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.159283+0000","last_change":"2024-04-17T10:39:14.153679+0000","last_active":"2024-04-17T10:39:14.159283+0000","last_peered":"2024-04-17T10:39:14.159283+0000","last_clean":"2024-04-17T10:39:14.159283+0000","last_became_active":"2024-04-17T10:39:14.152709+0000","last_became_peered":"2024-04-17T10:39:14.152709+0000","last_unstale":"2024-04-17T10:39:14.159283+0000","last_undegraded":"2024-04-17T10:39:14.159283+0000","last_fullsized":"2024-04-17T10:39:14.159283+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:55.079731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.162488+0000","last_change":"2024-04-17T10:39:14.154314+0000","last_active":"2024-04-17T10:39:14.162488+0000","last_peered":"2024-04-17T10:39:14.162488+0000","last_clean":"2024-04-17T10:39:14.162488+0000","last_became_active":"2024-04-17T10:39:14.153667+0000","last_became_peered":"2024-04-17T10:39:14.153667+0000","last_unstale":"2024-04-17T10:39:14.162488+0000","last_undegraded":"2024-04-17T10:39:14.162488+0000","last_fullsized":"2024-04-17T10:39:14.162488+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:02:20.150885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.155597+0000","last_change":"2024-04-17T10:39:14.150086+0000","last_active":"2024-04-17T10:39:14.155597+0000","last_peered":"2024-04-17T10:39:14.155597+0000","last_clean":"2024-04-17T10:39:14.155597+0000","last_became_active":"2024-04-17T10:39:14.149415+0000","last_became_peered":"2024-04-17T10:39:14.149415+0000","last_unstale":"2024-04-17T10:39:14.155597+0000","last_undegraded":"2024-04-17T10:39:14.155597+0000","last_fullsized":"2024-04-17T10:39:14.155597+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:23.357477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.130181+0000","last_change":"2024-04-17T10:39:14.119438+0000","last_active":"2024-04-17T10:39:14.130181+0000","last_peered":"2024-04-17T10:39:14.130181+0000","last_clean":"2024-04-17T10:39:14.130181+0000","last_became_active":"2024-04-17T10:39:14.119181+0000","last_became_peered":"2024-04-17T10:39:14.119181+0000","last_unstale":"2024-04-17T10:39:14.130181+0000","last_undegraded":"2024-04-17T10:39:14.130181+0000","last_fullsized":"2024-04-17T10:39:14.130181+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:12.498524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.127011+0000","last_change":"2024-04-17T10:39:14.117915+0000","last_active":"2024-04-17T10:39:14.127011+0000","last_peered":"2024-04-17T10:39:14.127011+0000","last_clean":"2024-04-17T10:39:14.127011+0000","last_became_active":"2024-04-17T10:39:14.117554+0000","last_became_peered":"2024-04-17T10:39:14.117554+0000","last_unstale":"2024-04-17T10:39:14.127011+0000","last_undegraded":"2024-04-17T10:39:14.127011+0000","last_fullsized":"2024-04-17T10:39:14.127011+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:25:15.943792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.147590+0000","last_change":"2024-04-17T10:39:14.144468+0000","last_active":"2024-04-17T10:39:14.147590+0000","last_peered":"2024-04-17T10:39:14.147590+0000","last_clean":"2024-04-17T10:39:14.147590+0000","last_became_active":"2024-04-17T10:39:14.144169+0000","last_became_peered":"2024-04-17T10:39:14.144169+0000","last_unstale":"2024-04-17T10:39:14.147590+0000","last_undegraded":"2024-04-17T10:39:14.147590+0000","last_fullsized":"2024-04-17T10:39:14.147590+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:03:44.258454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.148694+0000","last_change":"2024-04-17T10:39:14.122477+0000","last_active":"2024-04-17T10:39:14.148694+0000","last_peered":"2024-04-17T10:39:14.148694+0000","last_clean":"2024-04-17T10:39:14.148694+0000","last_became_active":"2024-04-17T10:39:14.121344+0000","last_became_peered":"2024-04-17T10:39:14.121344+0000","last_unstale":"2024-04-17T10:39:14.148694+0000","last_undegraded":"2024-04-17T10:39:14.148694+0000","last_fullsized":"2024-04-17T10:39:14.148694+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:15.162469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.148425+0000","last_change":"2024-04-17T10:39:14.129387+0000","last_active":"2024-04-17T10:39:14.148425+0000","last_peered":"2024-04-17T10:39:14.148425+0000","last_clean":"2024-04-17T10:39:14.148425+0000","last_became_active":"2024-04-17T10:39:14.128601+0000","last_became_peered":"2024-04-17T10:39:14.128601+0000","last_unstale":"2024-04-17T10:39:14.148425+0000","last_undegraded":"2024-04-17T10:39:14.148425+0000","last_fullsized":"2024-04-17T10:39:14.148425+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:03:31.760691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.158577+0000","last_change":"2024-04-17T10:39:14.157470+0000","last_active":"2024-04-17T10:39:14.158577+0000","last_peered":"2024-04-17T10:39:14.158577+0000","last_clean":"2024-04-17T10:39:14.158577+0000","last_became_active":"2024-04-17T10:39:14.157195+0000","last_became_peered":"2024-04-17T10:39:14.157195+0000","last_unstale":"2024-04-17T10:39:14.158577+0000","last_undegraded":"2024-04-17T10:39:14.158577+0000","last_fullsized":"2024-04-17T10:39:14.158577+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:57:53.655517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.144932+0000","last_change":"2024-04-17T10:39:14.133445+0000","last_active":"2024-04-17T10:39:14.144932+0000","last_peered":"2024-04-17T10:39:14.144932+0000","last_clean":"2024-04-17T10:39:14.144932+0000","last_became_active":"2024-04-17T10:39:14.132681+0000","last_became_peered":"2024-04-17T10:39:14.132681+0000","last_unstale":"2024-04-17T10:39:14.144932+0000","last_undegraded":"2024-04-17T10:39:14.144932+0000","last_fullsized":"2024-04-17T10:39:14.144932+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:12.498524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.158350+0000","last_change":"2024-04-17T10:39:14.154140+0000","last_active":"2024-04-17T10:39:14.158350+0000","last_peered":"2024-04-17T10:39:14.158350+0000","last_clean":"2024-04-17T10:39:14.158350+0000","last_became_active":"2024-04-17T10:39:14.153811+0000","last_became_peered":"2024-04-17T10:39:14.153811+0000","last_unstale":"2024-04-17T10:39:14.158350+0000","last_undegraded":"2024-04-17T10:39:14.158350+0000","last_fullsized":"2024-04-17T10:39:14.158350+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:23:38.450031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.130123+0000","last_change":"2024-04-17T10:39:14.121358+0000","last_active":"2024-04-17T10:39:14.130123+0000","last_peered":"2024-04-17T10:39:14.130123+0000","last_clean":"2024-04-17T10:39:14.130123+0000","last_became_active":"2024-04-17T10:39:14.121069+0000","last_became_peered":"2024-04-17T10:39:14.121069+0000","last_unstale":"2024-04-17T10:39:14.130123+0000","last_undegraded":"2024-04-17T10:39:14.130123+0000","last_fullsized":"2024-04-17T10:39:14.130123+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:09:41.478596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.142150+0000","last_change":"2024-04-17T10:39:14.139462+0000","last_active":"2024-04-17T10:39:14.142150+0000","last_peered":"2024-04-17T10:39:14.142150+0000","last_clean":"2024-04-17T10:39:14.142150+0000","last_became_active":"2024-04-17T10:39:14.138997+0000","last_became_peered":"2024-04-17T10:39:14.138997+0000","last_unstale":"2024-04-17T10:39:14.142150+0000","last_undegraded":"2024-04-17T10:39:14.142150+0000","last_fullsized":"2024-04-17T10:39:14.142150+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:28:20.535333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.145568+0000","last_change":"2024-04-17T10:39:14.138417+0000","last_active":"2024-04-17T10:39:14.145568+0000","last_peered":"2024-04-17T10:39:14.145568+0000","last_clean":"2024-04-17T10:39:14.145568+0000","last_became_active":"2024-04-17T10:39:14.136784+0000","last_became_peered":"2024-04-17T10:39:14.136784+0000","last_unstale":"2024-04-17T10:39:14.145568+0000","last_undegraded":"2024-04-17T10:39:14.145568+0000","last_fullsized":"2024-04-17T10:39:14.145568+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:28:23.922532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.148991+0000","last_change":"2024-04-17T10:39:14.131698+0000","last_active":"2024-04-17T10:39:14.148991+0000","last_peered":"2024-04-17T10:39:14.148991+0000","last_clean":"2024-04-17T10:39:14.148991+0000","last_became_active":"2024-04-17T10:39:14.129585+0000","last_became_peered":"2024-04-17T10:39:14.129585+0000","last_unstale":"2024-04-17T10:39:14.148991+0000","last_undegraded":"2024-04-17T10:39:14.148991+0000","last_fullsized":"2024-04-17T10:39:14.148991+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:57:25.113157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.155188+0000","last_change":"2024-04-17T10:39:14.148859+0000","last_active":"2024-04-17T10:39:14.155188+0000","last_peered":"2024-04-17T10:39:14.155188+0000","last_clean":"2024-04-17T10:39:14.155188+0000","last_became_active":"2024-04-17T10:39:14.148147+0000","last_became_peered":"2024-04-17T10:39:14.148147+0000","last_unstale":"2024-04-17T10:39:14.155188+0000","last_undegraded":"2024-04-17T10:39:14.155188+0000","last_fullsized":"2024-04-17T10:39:14.155188+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:09:41.478596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.151209+0000","last_change":"2024-04-17T10:39:14.142935+0000","last_active":"2024-04-17T10:39:14.151209+0000","last_peered":"2024-04-17T10:39:14.151209+0000","last_clean":"2024-04-17T10:39:14.151209+0000","last_became_active":"2024-04-17T10:39:14.141904+0000","last_became_peered":"2024-04-17T10:39:14.141904+0000","last_unstale":"2024-04-17T10:39:14.151209+0000","last_undegraded":"2024-04-17T10:39:14.151209+0000","last_fullsized":"2024-04-17T10:39:14.151209+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:17:01.172533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.154655+0000","last_change":"2024-04-17T10:39:14.148267+0000","last_active":"2024-04-17T10:39:14.154655+0000","last_peered":"2024-04-17T10:39:14.154655+0000","last_clean":"2024-04-17T10:39:14.154655+0000","last_became_active":"2024-04-17T10:39:14.140489+0000","last_became_peered":"2024-04-17T10:39:14.140489+0000","last_unstale":"2024-04-17T10:39:14.154655+0000","last_undegraded":"2024-04-17T10:39:14.154655+0000","last_fullsized":"2024-04-17T10:39:14.154655+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:57:16.628519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.158160+0000","last_change":"2024-04-17T10:39:14.154234+0000","last_active":"2024-04-17T10:39:14.158160+0000","last_peered":"2024-04-17T10:39:14.158160+0000","last_clean":"2024-04-17T10:39:14.158160+0000","last_became_active":"2024-04-17T10:39:14.153893+0000","last_became_peered":"2024-04-17T10:39:14.153893+0000","last_unstale":"2024-04-17T10:39:14.158160+0000","last_undegraded":"2024-04-17T10:39:14.158160+0000","last_fullsized":"2024-04-17T10:39:14.158160+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:45:43.557201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.156988+0000","last_change":"2024-04-17T10:39:13.137675+0000","last_active":"2024-04-17T10:39:14.156988+0000","last_peered":"2024-04-17T10:39:14.156988+0000","last_clean":"2024-04-17T10:39:14.156988+0000","last_became_active":"2024-04-17T10:39:13.136910+0000","last_became_peered":"2024-04-17T10:39:13.136910+0000","last_unstale":"2024-04-17T10:39:14.156988+0000","last_undegraded":"2024-04-17T10:39:14.156988+0000","last_fullsized":"2024-04-17T10:39:14.156988+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:10:15.558571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.159194+0000","last_change":"2024-04-17T10:39:14.155480+0000","last_active":"2024-04-17T10:39:14.159194+0000","last_peered":"2024-04-17T10:39:14.159194+0000","last_clean":"2024-04-17T10:39:14.159194+0000","last_became_active":"2024-04-17T10:39:14.154666+0000","last_became_peered":"2024-04-17T10:39:14.154666+0000","last_unstale":"2024-04-17T10:39:14.159194+0000","last_undegraded":"2024-04-17T10:39:14.159194+0000","last_fullsized":"2024-04-17T10:39:14.159194+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:15.162469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.153759+0000","last_change":"2024-04-17T10:39:13.114832+0000","last_active":"2024-04-17T10:39:14.153759+0000","last_peered":"2024-04-17T10:39:14.153759+0000","last_clean":"2024-04-17T10:39:14.153759+0000","last_became_active":"2024-04-17T10:39:13.114474+0000","last_became_peered":"2024-04-17T10:39:13.114474+0000","last_unstale":"2024-04-17T10:39:14.153759+0000","last_undegraded":"2024-04-17T10:39:14.153759+0000","last_fullsized":"2024-04-17T10:39:14.153759+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:45:46.186394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.148474+0000","last_change":"2024-04-17T10:39:14.129590+0000","last_active":"2024-04-17T10:39:14.148474+0000","last_peered":"2024-04-17T10:39:14.148474+0000","last_clean":"2024-04-17T10:39:14.148474+0000","last_became_active":"2024-04-17T10:39:14.128771+0000","last_became_peered":"2024-04-17T10:39:14.128771+0000","last_unstale":"2024-04-17T10:39:14.148474+0000","last_undegraded":"2024-04-17T10:39:14.148474+0000","last_fullsized":"2024-04-17T10:39:14.148474+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:39.269987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.131849+0000","last_change":"2024-04-17T10:39:13.120615+0000","last_active":"2024-04-17T10:39:14.131849+0000","last_peered":"2024-04-17T10:39:14.131849+0000","last_clean":"2024-04-17T10:39:14.131849+0000","last_became_active":"2024-04-17T10:39:13.120338+0000","last_became_peered":"2024-04-17T10:39:13.120338+0000","last_unstale":"2024-04-17T10:39:14.131849+0000","last_undegraded":"2024-04-17T10:39:14.131849+0000","last_fullsized":"2024-04-17T10:39:14.131849+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:28:19.531406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.142176+0000","last_change":"2024-04-17T10:39:14.139943+0000","last_active":"2024-04-17T10:39:14.142176+0000","last_peered":"2024-04-17T10:39:14.142176+0000","last_clean":"2024-04-17T10:39:14.142176+0000","last_became_active":"2024-04-17T10:39:14.139474+0000","last_became_peered":"2024-04-17T10:39:14.139474+0000","last_unstale":"2024-04-17T10:39:14.142176+0000","last_undegraded":"2024-04-17T10:39:14.142176+0000","last_fullsized":"2024-04-17T10:39:14.142176+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:30:34.072845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.130286+0000","last_change":"2024-04-17T10:39:13.123599+0000","last_active":"2024-04-17T10:39:14.130286+0000","last_peered":"2024-04-17T10:39:14.130286+0000","last_clean":"2024-04-17T10:39:14.130286+0000","last_became_active":"2024-04-17T10:39:13.123010+0000","last_became_peered":"2024-04-17T10:39:13.123010+0000","last_unstale":"2024-04-17T10:39:14.130286+0000","last_undegraded":"2024-04-17T10:39:14.130286+0000","last_fullsized":"2024-04-17T10:39:14.130286+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:57:54.529742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.155140+0000","last_change":"2024-04-17T10:39:14.148468+0000","last_active":"2024-04-17T10:39:14.155140+0000","last_peered":"2024-04-17T10:39:14.155140+0000","last_clean":"2024-04-17T10:39:14.155140+0000","last_became_active":"2024-04-17T10:39:14.147709+0000","last_became_peered":"2024-04-17T10:39:14.147709+0000","last_unstale":"2024-04-17T10:39:14.155140+0000","last_undegraded":"2024-04-17T10:39:14.155140+0000","last_fullsized":"2024-04-17T10:39:14.155140+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:12.498524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.148800+0000","last_change":"2024-04-17T10:39:13.139187+0000","last_active":"2024-04-17T10:39:14.148800+0000","last_peered":"2024-04-17T10:39:14.148800+0000","last_clean":"2024-04-17T10:39:14.148800+0000","last_became_active":"2024-04-17T10:39:13.138301+0000","last_became_peered":"2024-04-17T10:39:13.138301+0000","last_unstale":"2024-04-17T10:39:14.148800+0000","last_undegraded":"2024-04-17T10:39:14.148800+0000","last_fullsized":"2024-04-17T10:39:14.148800+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:52:09.191244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.142406+0000","last_change":"2024-04-17T10:39:14.133235+0000","last_active":"2024-04-17T10:39:14.142406+0000","last_peered":"2024-04-17T10:39:14.142406+0000","last_clean":"2024-04-17T10:39:14.142406+0000","last_became_active":"2024-04-17T10:39:14.132536+0000","last_became_peered":"2024-04-17T10:39:14.132536+0000","last_unstale":"2024-04-17T10:39:14.142406+0000","last_undegraded":"2024-04-17T10:39:14.142406+0000","last_fullsized":"2024-04-17T10:39:14.142406+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:57:16.628519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.148545+0000","last_change":"2024-04-17T10:39:13.139900+0000","last_active":"2024-04-17T10:39:14.148545+0000","last_peered":"2024-04-17T10:39:14.148545+0000","last_clean":"2024-04-17T10:39:14.148545+0000","last_became_active":"2024-04-17T10:39:13.138670+0000","last_became_peered":"2024-04-17T10:39:13.138670+0000","last_unstale":"2024-04-17T10:39:14.148545+0000","last_undegraded":"2024-04-17T10:39:14.148545+0000","last_fullsized":"2024-04-17T10:39:14.148545+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:31:02.964758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.161720+0000","last_change":"2024-04-17T10:39:14.156061+0000","last_active":"2024-04-17T10:39:14.161720+0000","last_peered":"2024-04-17T10:39:14.161720+0000","last_clean":"2024-04-17T10:39:14.161720+0000","last_became_active":"2024-04-17T10:39:14.154568+0000","last_became_peered":"2024-04-17T10:39:14.154568+0000","last_unstale":"2024-04-17T10:39:14.161720+0000","last_undegraded":"2024-04-17T10:39:14.161720+0000","last_fullsized":"2024-04-17T10:39:14.161720+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:28:20.535333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.105297+0000","last_change":"2024-04-17T10:39:13.119307+0000","last_active":"2024-04-17T10:39:14.105297+0000","last_peered":"2024-04-17T10:39:14.105297+0000","last_clean":"2024-04-17T10:39:14.105297+0000","last_became_active":"2024-04-17T10:39:13.118917+0000","last_became_peered":"2024-04-17T10:39:13.118917+0000","last_unstale":"2024-04-17T10:39:14.105297+0000","last_undegraded":"2024-04-17T10:39:14.105297+0000","last_fullsized":"2024-04-17T10:39:14.105297+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:57:24.109230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.141560+0000","last_change":"2024-04-17T10:39:14.136470+0000","last_active":"2024-04-17T10:39:14.141560+0000","last_peered":"2024-04-17T10:39:14.141560+0000","last_clean":"2024-04-17T10:39:14.141560+0000","last_became_active":"2024-04-17T10:39:14.136202+0000","last_became_peered":"2024-04-17T10:39:14.136202+0000","last_unstale":"2024-04-17T10:39:14.141560+0000","last_undegraded":"2024-04-17T10:39:14.141560+0000","last_fullsized":"2024-04-17T10:39:14.141560+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:57:25.113157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.108447+0000","last_change":"2024-04-17T10:39:13.139381+0000","last_active":"2024-04-17T10:39:14.108447+0000","last_peered":"2024-04-17T10:39:14.108447+0000","last_clean":"2024-04-17T10:39:14.108447+0000","last_became_active":"2024-04-17T10:39:13.138662+0000","last_became_peered":"2024-04-17T10:39:13.138662+0000","last_unstale":"2024-04-17T10:39:14.108447+0000","last_undegraded":"2024-04-17T10:39:14.108447+0000","last_fullsized":"2024-04-17T10:39:14.108447+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:23:13.822291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.142675+0000","last_change":"2024-04-17T10:39:14.134588+0000","last_active":"2024-04-17T10:39:14.142675+0000","last_peered":"2024-04-17T10:39:14.142675+0000","last_clean":"2024-04-17T10:39:14.142675+0000","last_became_active":"2024-04-17T10:39:14.133701+0000","last_became_peered":"2024-04-17T10:39:14.133701+0000","last_unstale":"2024-04-17T10:39:14.142675+0000","last_undegraded":"2024-04-17T10:39:14.142675+0000","last_fullsized":"2024-04-17T10:39:14.142675+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:23.357477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.106772+0000","last_change":"2024-04-17T10:39:13.129003+0000","last_active":"2024-04-17T10:39:14.106772+0000","last_peered":"2024-04-17T10:39:14.106772+0000","last_clean":"2024-04-17T10:39:14.106772+0000","last_became_active":"2024-04-17T10:39:13.128775+0000","last_became_peered":"2024-04-17T10:39:13.128775+0000","last_unstale":"2024-04-17T10:39:14.106772+0000","last_undegraded":"2024-04-17T10:39:14.106772+0000","last_fullsized":"2024-04-17T10:39:14.106772+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:24:47.491838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.127035+0000","last_change":"2024-04-17T10:39:14.118323+0000","last_active":"2024-04-17T10:39:14.127035+0000","last_peered":"2024-04-17T10:39:14.127035+0000","last_clean":"2024-04-17T10:39:14.127035+0000","last_became_active":"2024-04-17T10:39:14.118097+0000","last_became_peered":"2024-04-17T10:39:14.118097+0000","last_unstale":"2024-04-17T10:39:14.127035+0000","last_undegraded":"2024-04-17T10:39:14.127035+0000","last_fullsized":"2024-04-17T10:39:14.127035+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:57:16.628519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.115834+0000","last_change":"2024-04-17T10:39:13.108220+0000","last_active":"2024-04-17T10:39:14.115834+0000","last_peered":"2024-04-17T10:39:14.115834+0000","last_clean":"2024-04-17T10:39:14.115834+0000","last_became_active":"2024-04-17T10:39:13.107899+0000","last_became_peered":"2024-04-17T10:39:13.107899+0000","last_unstale":"2024-04-17T10:39:14.115834+0000","last_undegraded":"2024-04-17T10:39:14.115834+0000","last_fullsized":"2024-04-17T10:39:14.115834+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:14.158542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.155752+0000","last_change":"2024-04-17T10:39:14.153002+0000","last_active":"2024-04-17T10:39:14.155752+0000","last_peered":"2024-04-17T10:39:14.155752+0000","last_clean":"2024-04-17T10:39:14.155752+0000","last_became_active":"2024-04-17T10:39:14.152720+0000","last_became_peered":"2024-04-17T10:39:14.152720+0000","last_unstale":"2024-04-17T10:39:14.155752+0000","last_undegraded":"2024-04-17T10:39:14.155752+0000","last_fullsized":"2024-04-17T10:39:14.155752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:02:20.150885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.165962+0000","last_change":"2024-04-17T10:39:13.133898+0000","last_active":"2024-04-17T10:39:14.165962+0000","last_peered":"2024-04-17T10:39:14.165962+0000","last_clean":"2024-04-17T10:39:14.165962+0000","last_became_active":"2024-04-17T10:39:13.133055+0000","last_became_peered":"2024-04-17T10:39:13.133055+0000","last_unstale":"2024-04-17T10:39:14.165962+0000","last_undegraded":"2024-04-17T10:39:14.165962+0000","last_fullsized":"2024-04-17T10:39:14.165962+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:25:14.939865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.143578+0000","last_change":"2024-04-17T10:39:14.134931+0000","last_active":"2024-04-17T10:39:14.143578+0000","last_peered":"2024-04-17T10:39:14.143578+0000","last_clean":"2024-04-17T10:39:14.143578+0000","last_became_active":"2024-04-17T10:39:14.133404+0000","last_became_peered":"2024-04-17T10:39:14.133404+0000","last_unstale":"2024-04-17T10:39:14.143578+0000","last_undegraded":"2024-04-17T10:39:14.143578+0000","last_fullsized":"2024-04-17T10:39:14.143578+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:12:05.386724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.153129+0000","last_change":"2024-04-17T10:39:13.118837+0000","last_active":"2024-04-17T10:39:14.153129+0000","last_peered":"2024-04-17T10:39:14.153129+0000","last_clean":"2024-04-17T10:39:14.153129+0000","last_became_active":"2024-04-17T10:39:13.118431+0000","last_became_peered":"2024-04-17T10:39:13.118431+0000","last_unstale":"2024-04-17T10:39:14.153129+0000","last_undegraded":"2024-04-17T10:39:14.153129+0000","last_fullsized":"2024-04-17T10:39:14.153129+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:45:42.553273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.147727+0000","last_change":"2024-04-17T10:39:14.143465+0000","last_active":"2024-04-17T10:39:14.147727+0000","last_peered":"2024-04-17T10:39:14.147727+0000","last_clean":"2024-04-17T10:39:14.147727+0000","last_became_active":"2024-04-17T10:39:14.143108+0000","last_became_peered":"2024-04-17T10:39:14.143108+0000","last_unstale":"2024-04-17T10:39:14.147727+0000","last_undegraded":"2024-04-17T10:39:14.147727+0000","last_fullsized":"2024-04-17T10:39:14.147727+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:24:48.495765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.153383+0000","last_change":"2024-04-17T10:39:13.139826+0000","last_active":"2024-04-17T10:39:14.153383+0000","last_peered":"2024-04-17T10:39:14.153383+0000","last_clean":"2024-04-17T10:39:14.153383+0000","last_became_active":"2024-04-17T10:39:13.138981+0000","last_became_peered":"2024-04-17T10:39:13.138981+0000","last_unstale":"2024-04-17T10:39:14.153383+0000","last_undegraded":"2024-04-17T10:39:14.153383+0000","last_fullsized":"2024-04-17T10:39:14.153383+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:10:15.558571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.129169+0000","last_change":"2024-04-17T10:39:14.122559+0000","last_active":"2024-04-17T10:39:14.129169+0000","last_peered":"2024-04-17T10:39:14.129169+0000","last_clean":"2024-04-17T10:39:14.129169+0000","last_became_active":"2024-04-17T10:39:14.122293+0000","last_became_peered":"2024-04-17T10:39:14.122293+0000","last_unstale":"2024-04-17T10:39:14.129169+0000","last_undegraded":"2024-04-17T10:39:14.129169+0000","last_fullsized":"2024-04-17T10:39:14.129169+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:12:05.386724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.113037+0000","last_change":"2024-04-17T10:39:13.133706+0000","last_active":"2024-04-17T10:39:14.113037+0000","last_peered":"2024-04-17T10:39:14.113037+0000","last_clean":"2024-04-17T10:39:14.113037+0000","last_became_active":"2024-04-17T10:39:13.133182+0000","last_became_peered":"2024-04-17T10:39:13.133182+0000","last_unstale":"2024-04-17T10:39:14.113037+0000","last_undegraded":"2024-04-17T10:39:14.113037+0000","last_fullsized":"2024-04-17T10:39:14.113037+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:45:46.186394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.162707+0000","last_change":"2024-04-17T10:39:14.160724+0000","last_active":"2024-04-17T10:39:14.162707+0000","last_peered":"2024-04-17T10:39:14.162707+0000","last_clean":"2024-04-17T10:39:14.162707+0000","last_became_active":"2024-04-17T10:39:14.160182+0000","last_became_peered":"2024-04-17T10:39:14.160182+0000","last_unstale":"2024-04-17T10:39:14.162707+0000","last_undegraded":"2024-04-17T10:39:14.162707+0000","last_fullsized":"2024-04-17T10:39:14.162707+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:13:32.981384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.105130+0000","last_change":"2024-04-17T10:39:13.131733+0000","last_active":"2024-04-17T10:39:14.105130+0000","last_peered":"2024-04-17T10:39:14.105130+0000","last_clean":"2024-04-17T10:39:14.105130+0000","last_became_active":"2024-04-17T10:39:13.131494+0000","last_became_peered":"2024-04-17T10:39:13.131494+0000","last_unstale":"2024-04-17T10:39:14.105130+0000","last_undegraded":"2024-04-17T10:39:14.105130+0000","last_fullsized":"2024-04-17T10:39:14.105130+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:38.266060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.162425+0000","last_change":"2024-04-17T10:39:14.157681+0000","last_active":"2024-04-17T10:39:14.162425+0000","last_peered":"2024-04-17T10:39:14.162425+0000","last_clean":"2024-04-17T10:39:14.162425+0000","last_became_active":"2024-04-17T10:39:14.156655+0000","last_became_peered":"2024-04-17T10:39:14.156655+0000","last_unstale":"2024-04-17T10:39:14.162425+0000","last_undegraded":"2024-04-17T10:39:14.162425+0000","last_fullsized":"2024-04-17T10:39:14.162425+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:39.269987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.132561+0000","last_change":"2024-04-17T10:39:13.120068+0000","last_active":"2024-04-17T10:39:14.132561+0000","last_peered":"2024-04-17T10:39:14.132561+0000","last_clean":"2024-04-17T10:39:14.132561+0000","last_became_active":"2024-04-17T10:39:13.119711+0000","last_became_peered":"2024-04-17T10:39:13.119711+0000","last_unstale":"2024-04-17T10:39:14.132561+0000","last_undegraded":"2024-04-17T10:39:14.132561+0000","last_fullsized":"2024-04-17T10:39:14.132561+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:30:33.068918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.153777+0000","last_change":"2024-04-17T10:39:14.145653+0000","last_active":"2024-04-17T10:39:14.153777+0000","last_peered":"2024-04-17T10:39:14.153777+0000","last_clean":"2024-04-17T10:39:14.153777+0000","last_became_active":"2024-04-17T10:39:14.144936+0000","last_became_peered":"2024-04-17T10:39:14.144936+0000","last_unstale":"2024-04-17T10:39:14.153777+0000","last_undegraded":"2024-04-17T10:39:14.153777+0000","last_fullsized":"2024-04-17T10:39:14.153777+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:57:53.655517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.131017+0000","last_change":"2024-04-17T10:39:13.128344+0000","last_active":"2024-04-17T10:39:14.131017+0000","last_peered":"2024-04-17T10:39:14.131017+0000","last_clean":"2024-04-17T10:39:14.131017+0000","last_became_active":"2024-04-17T10:39:13.127941+0000","last_became_peered":"2024-04-17T10:39:13.127941+0000","last_unstale":"2024-04-17T10:39:14.131017+0000","last_undegraded":"2024-04-17T10:39:14.131017+0000","last_fullsized":"2024-04-17T10:39:14.131017+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:17:00.168606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.159341+0000","last_change":"2024-04-17T10:39:14.156339+0000","last_active":"2024-04-17T10:39:14.159341+0000","last_peered":"2024-04-17T10:39:14.159341+0000","last_clean":"2024-04-17T10:39:14.159341+0000","last_became_active":"2024-04-17T10:39:14.155669+0000","last_became_peered":"2024-04-17T10:39:14.155669+0000","last_unstale":"2024-04-17T10:39:14.159341+0000","last_undegraded":"2024-04-17T10:39:14.159341+0000","last_fullsized":"2024-04-17T10:39:14.159341+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:41:36.240202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.153667+0000","last_change":"2024-04-17T10:39:13.145192+0000","last_active":"2024-04-17T10:39:14.153667+0000","last_peered":"2024-04-17T10:39:14.153667+0000","last_clean":"2024-04-17T10:39:14.153667+0000","last_became_active":"2024-04-17T10:39:13.144002+0000","last_became_peered":"2024-04-17T10:39:13.144002+0000","last_unstale":"2024-04-17T10:39:14.153667+0000","last_undegraded":"2024-04-17T10:39:14.153667+0000","last_fullsized":"2024-04-17T10:39:14.153667+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:03:43.254527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.142155+0000","last_change":"2024-04-17T10:39:14.139357+0000","last_active":"2024-04-17T10:39:14.142155+0000","last_peered":"2024-04-17T10:39:14.142155+0000","last_clean":"2024-04-17T10:39:14.142155+0000","last_became_active":"2024-04-17T10:39:14.139073+0000","last_became_peered":"2024-04-17T10:39:14.139073+0000","last_unstale":"2024-04-17T10:39:14.142155+0000","last_undegraded":"2024-04-17T10:39:14.142155+0000","last_fullsized":"2024-04-17T10:39:14.142155+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:57:53.655517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.148320+0000","last_change":"2024-04-17T10:39:13.128103+0000","last_active":"2024-04-17T10:39:14.148320+0000","last_peered":"2024-04-17T10:39:14.148320+0000","last_clean":"2024-04-17T10:39:14.148320+0000","last_became_active":"2024-04-17T10:39:13.127167+0000","last_became_peered":"2024-04-17T10:39:13.127167+0000","last_unstale":"2024-04-17T10:39:14.148320+0000","last_undegraded":"2024-04-17T10:39:14.148320+0000","last_fullsized":"2024-04-17T10:39:14.148320+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:18:28.175123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.158288+0000","last_change":"2024-04-17T10:39:14.153894+0000","last_active":"2024-04-17T10:39:14.158288+0000","last_peered":"2024-04-17T10:39:14.158288+0000","last_clean":"2024-04-17T10:39:14.158288+0000","last_became_active":"2024-04-17T10:39:14.153591+0000","last_became_peered":"2024-04-17T10:39:14.153591+0000","last_unstale":"2024-04-17T10:39:14.158288+0000","last_undegraded":"2024-04-17T10:39:14.158288+0000","last_fullsized":"2024-04-17T10:39:14.158288+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:52:10.195171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.108733+0000","last_change":"2024-04-17T10:39:13.139001+0000","last_active":"2024-04-17T10:39:14.108733+0000","last_peered":"2024-04-17T10:39:14.108733+0000","last_clean":"2024-04-17T10:39:14.108733+0000","last_became_active":"2024-04-17T10:39:13.138189+0000","last_became_peered":"2024-04-17T10:39:13.138189+0000","last_unstale":"2024-04-17T10:39:14.108733+0000","last_undegraded":"2024-04-17T10:39:14.108733+0000","last_fullsized":"2024-04-17T10:39:14.108733+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:53:42.646220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.141744+0000","last_change":"2024-04-17T10:39:14.139769+0000","last_active":"2024-04-17T10:39:14.141744+0000","last_peered":"2024-04-17T10:39:14.141744+0000","last_clean":"2024-04-17T10:39:14.141744+0000","last_became_active":"2024-04-17T10:39:14.139461+0000","last_became_peered":"2024-04-17T10:39:14.139461+0000","last_unstale":"2024-04-17T10:39:14.141744+0000","last_undegraded":"2024-04-17T10:39:14.141744+0000","last_fullsized":"2024-04-17T10:39:14.141744+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:23:14.826218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.131831+0000","last_change":"2024-04-17T10:39:13.120631+0000","last_active":"2024-04-17T10:39:14.131831+0000","last_peered":"2024-04-17T10:39:14.131831+0000","last_clean":"2024-04-17T10:39:14.131831+0000","last_became_active":"2024-04-17T10:39:13.120018+0000","last_became_peered":"2024-04-17T10:39:13.120018+0000","last_unstale":"2024-04-17T10:39:14.131831+0000","last_undegraded":"2024-04-17T10:39:14.131831+0000","last_fullsized":"2024-04-17T10:39:14.131831+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:17:00.168606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.129908+0000","last_change":"2024-04-17T10:39:14.127220+0000","last_active":"2024-04-17T10:39:14.129908+0000","last_peered":"2024-04-17T10:39:14.129908+0000","last_clean":"2024-04-17T10:39:14.129908+0000","last_became_active":"2024-04-17T10:39:14.126926+0000","last_became_peered":"2024-04-17T10:39:14.126926+0000","last_unstale":"2024-04-17T10:39:14.129908+0000","last_undegraded":"2024-04-17T10:39:14.129908+0000","last_fullsized":"2024-04-17T10:39:14.129908+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:18:29.179050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.136271+0000","last_change":"2024-04-17T10:39:13.135178+0000","last_active":"2024-04-17T10:39:14.136271+0000","last_peered":"2024-04-17T10:39:14.136271+0000","last_clean":"2024-04-17T10:39:14.136271+0000","last_became_active":"2024-04-17T10:39:13.134490+0000","last_became_peered":"2024-04-17T10:39:13.134490+0000","last_unstale":"2024-04-17T10:39:14.136271+0000","last_undegraded":"2024-04-17T10:39:14.136271+0000","last_fullsized":"2024-04-17T10:39:14.136271+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:45:46.186394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.120606+0000","last_change":"2024-04-17T10:39:14.118147+0000","last_active":"2024-04-17T10:39:14.120606+0000","last_peered":"2024-04-17T10:39:14.120606+0000","last_clean":"2024-04-17T10:39:14.120606+0000","last_became_active":"2024-04-17T10:39:14.117781+0000","last_became_peered":"2024-04-17T10:39:14.117781+0000","last_unstale":"2024-04-17T10:39:14.120606+0000","last_undegraded":"2024-04-17T10:39:14.120606+0000","last_fullsized":"2024-04-17T10:39:14.120606+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:40:40.684393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.111353+0000","last_change":"2024-04-17T10:39:13.126741+0000","last_active":"2024-04-17T10:39:14.111353+0000","last_peered":"2024-04-17T10:39:14.111353+0000","last_clean":"2024-04-17T10:39:14.111353+0000","last_became_active":"2024-04-17T10:39:13.124777+0000","last_became_peered":"2024-04-17T10:39:13.124777+0000","last_unstale":"2024-04-17T10:39:14.111353+0000","last_undegraded":"2024-04-17T10:39:14.111353+0000","last_fullsized":"2024-04-17T10:39:14.111353+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:45:46.186394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.162946+0000","last_change":"2024-04-17T10:39:14.153983+0000","last_active":"2024-04-17T10:39:14.162946+0000","last_peered":"2024-04-17T10:39:14.162946+0000","last_clean":"2024-04-17T10:39:14.162946+0000","last_became_active":"2024-04-17T10:39:14.150196+0000","last_became_peered":"2024-04-17T10:39:14.150196+0000","last_unstale":"2024-04-17T10:39:14.162946+0000","last_undegraded":"2024-04-17T10:39:14.162946+0000","last_fullsized":"2024-04-17T10:39:14.162946+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:28:23.922532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.119377+0000","last_change":"2024-04-17T10:39:13.133409+0000","last_active":"2024-04-17T10:39:14.119377+0000","last_peered":"2024-04-17T10:39:14.119377+0000","last_clean":"2024-04-17T10:39:14.119377+0000","last_became_active":"2024-04-17T10:39:13.132490+0000","last_became_peered":"2024-04-17T10:39:13.132490+0000","last_unstale":"2024-04-17T10:39:14.119377+0000","last_undegraded":"2024-04-17T10:39:14.119377+0000","last_fullsized":"2024-04-17T10:39:14.119377+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:14:31.765076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.147548+0000","last_change":"2024-04-17T10:39:14.143153+0000","last_active":"2024-04-17T10:39:14.147548+0000","last_peered":"2024-04-17T10:39:14.147548+0000","last_clean":"2024-04-17T10:39:14.147548+0000","last_became_active":"2024-04-17T10:39:14.142895+0000","last_became_peered":"2024-04-17T10:39:14.142895+0000","last_unstale":"2024-04-17T10:39:14.147548+0000","last_undegraded":"2024-04-17T10:39:14.147548+0000","last_fullsized":"2024-04-17T10:39:14.147548+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:53:43.650148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.146468+0000","last_change":"2024-04-17T10:39:13.133488+0000","last_active":"2024-04-17T10:39:14.146468+0000","last_peered":"2024-04-17T10:39:14.146468+0000","last_clean":"2024-04-17T10:39:14.146468+0000","last_became_active":"2024-04-17T10:39:13.133148+0000","last_became_peered":"2024-04-17T10:39:13.133148+0000","last_unstale":"2024-04-17T10:39:14.146468+0000","last_undegraded":"2024-04-17T10:39:14.146468+0000","last_fullsized":"2024-04-17T10:39:14.146468+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:14.158542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.153284+0000","last_change":"2024-04-17T10:39:14.145382+0000","last_active":"2024-04-17T10:39:14.153284+0000","last_peered":"2024-04-17T10:39:14.153284+0000","last_clean":"2024-04-17T10:39:14.153284+0000","last_became_active":"2024-04-17T10:39:14.142759+0000","last_became_peered":"2024-04-17T10:39:14.142759+0000","last_unstale":"2024-04-17T10:39:14.153284+0000","last_undegraded":"2024-04-17T10:39:14.153284+0000","last_fullsized":"2024-04-17T10:39:14.153284+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:13:32.981384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.116025+0000","last_change":"2024-04-17T10:39:13.116593+0000","last_active":"2024-04-17T10:39:14.116025+0000","last_peered":"2024-04-17T10:39:14.116025+0000","last_clean":"2024-04-17T10:39:14.116025+0000","last_became_active":"2024-04-17T10:39:13.116212+0000","last_became_peered":"2024-04-17T10:39:13.116212+0000","last_unstale":"2024-04-17T10:39:14.116025+0000","last_undegraded":"2024-04-17T10:39:14.116025+0000","last_fullsized":"2024-04-17T10:39:14.116025+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:03:30.756764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.121035+0000","last_change":"2024-04-17T10:39:14.118830+0000","last_active":"2024-04-17T10:39:14.121035+0000","last_peered":"2024-04-17T10:39:14.121035+0000","last_clean":"2024-04-17T10:39:14.121035+0000","last_became_active":"2024-04-17T10:39:14.118371+0000","last_became_peered":"2024-04-17T10:39:14.118371+0000","last_unstale":"2024-04-17T10:39:14.121035+0000","last_undegraded":"2024-04-17T10:39:14.121035+0000","last_fullsized":"2024-04-17T10:39:14.121035+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:39.269987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.114892+0000","last_change":"2024-04-17T10:39:13.122418+0000","last_active":"2024-04-17T10:39:14.114892+0000","last_peered":"2024-04-17T10:39:14.114892+0000","last_clean":"2024-04-17T10:39:14.114892+0000","last_became_active":"2024-04-17T10:39:13.121969+0000","last_became_peered":"2024-04-17T10:39:13.121969+0000","last_unstale":"2024-04-17T10:39:14.114892+0000","last_undegraded":"2024-04-17T10:39:14.114892+0000","last_fullsized":"2024-04-17T10:39:14.114892+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:03:30.756764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.159347+0000","last_change":"2024-04-17T10:39:14.156982+0000","last_active":"2024-04-17T10:39:14.159347+0000","last_peered":"2024-04-17T10:39:14.159347+0000","last_clean":"2024-04-17T10:39:14.159347+0000","last_became_active":"2024-04-17T10:39:14.155095+0000","last_became_peered":"2024-04-17T10:39:14.155095+0000","last_unstale":"2024-04-17T10:39:14.159347+0000","last_undegraded":"2024-04-17T10:39:14.159347+0000","last_fullsized":"2024-04-17T10:39:14.159347+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:03:31.760691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.114375+0000","last_change":"2024-04-17T10:39:13.126248+0000","last_active":"2024-04-17T10:39:14.114375+0000","last_peered":"2024-04-17T10:39:14.114375+0000","last_clean":"2024-04-17T10:39:14.114375+0000","last_became_active":"2024-04-17T10:39:13.125917+0000","last_became_peered":"2024-04-17T10:39:13.125917+0000","last_unstale":"2024-04-17T10:39:14.114375+0000","last_undegraded":"2024-04-17T10:39:14.114375+0000","last_fullsized":"2024-04-17T10:39:14.114375+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:58:06.016900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.147498+0000","last_change":"2024-04-17T10:39:14.142456+0000","last_active":"2024-04-17T10:39:14.147498+0000","last_peered":"2024-04-17T10:39:14.147498+0000","last_clean":"2024-04-17T10:39:14.147498+0000","last_became_active":"2024-04-17T10:39:14.142187+0000","last_became_peered":"2024-04-17T10:39:14.142187+0000","last_unstale":"2024-04-17T10:39:14.147498+0000","last_undegraded":"2024-04-17T10:39:14.147498+0000","last_fullsized":"2024-04-17T10:39:14.147498+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:57:25.113157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.113505+0000","last_change":"2024-04-17T10:39:13.133461+0000","last_active":"2024-04-17T10:39:14.113505+0000","last_peered":"2024-04-17T10:39:14.113505+0000","last_clean":"2024-04-17T10:39:14.113505+0000","last_became_active":"2024-04-17T10:39:13.133189+0000","last_became_peered":"2024-04-17T10:39:13.133189+0000","last_unstale":"2024-04-17T10:39:14.113505+0000","last_undegraded":"2024-04-17T10:39:14.113505+0000","last_fullsized":"2024-04-17T10:39:14.113505+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:58:06.016900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.127541+0000","last_change":"2024-04-17T10:39:14.117643+0000","last_active":"2024-04-17T10:39:14.127541+0000","last_peered":"2024-04-17T10:39:14.127541+0000","last_clean":"2024-04-17T10:39:14.127541+0000","last_became_active":"2024-04-17T10:39:14.117355+0000","last_became_peered":"2024-04-17T10:39:14.117355+0000","last_unstale":"2024-04-17T10:39:14.127541+0000","last_undegraded":"2024-04-17T10:39:14.127541+0000","last_fullsized":"2024-04-17T10:39:14.127541+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:14:32.769003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.111757+0000","last_change":"2024-04-17T10:39:13.138057+0000","last_active":"2024-04-17T10:39:14.111757+0000","last_peered":"2024-04-17T10:39:14.111757+0000","last_clean":"2024-04-17T10:39:14.111757+0000","last_became_active":"2024-04-17T10:39:13.130733+0000","last_became_peered":"2024-04-17T10:39:13.130733+0000","last_unstale":"2024-04-17T10:39:14.111757+0000","last_undegraded":"2024-04-17T10:39:14.111757+0000","last_fullsized":"2024-04-17T10:39:14.111757+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:17:00.168606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.153460+0000","last_change":"2024-04-17T10:39:14.137840+0000","last_active":"2024-04-17T10:39:14.153460+0000","last_peered":"2024-04-17T10:39:14.153460+0000","last_clean":"2024-04-17T10:39:14.153460+0000","last_became_active":"2024-04-17T10:39:14.128627+0000","last_became_peered":"2024-04-17T10:39:14.128627+0000","last_unstale":"2024-04-17T10:39:14.153460+0000","last_undegraded":"2024-04-17T10:39:14.153460+0000","last_fullsized":"2024-04-17T10:39:14.153460+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:45:43.557201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.105523+0000","last_change":"2024-04-17T10:39:13.130880+0000","last_active":"2024-04-17T10:39:14.105523+0000","last_peered":"2024-04-17T10:39:14.105523+0000","last_clean":"2024-04-17T10:39:14.105523+0000","last_became_active":"2024-04-17T10:39:13.130434+0000","last_became_peered":"2024-04-17T10:39:13.130434+0000","last_unstale":"2024-04-17T10:39:14.105523+0000","last_undegraded":"2024-04-17T10:39:14.105523+0000","last_fullsized":"2024-04-17T10:39:14.105523+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:57:52.651590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.154051+0000","last_change":"2024-04-17T10:39:14.140608+0000","last_active":"2024-04-17T10:39:14.154051+0000","last_peered":"2024-04-17T10:39:14.154051+0000","last_clean":"2024-04-17T10:39:14.154051+0000","last_became_active":"2024-04-17T10:39:14.139887+0000","last_became_peered":"2024-04-17T10:39:14.139887+0000","last_unstale":"2024-04-17T10:39:14.154051+0000","last_undegraded":"2024-04-17T10:39:14.154051+0000","last_fullsized":"2024-04-17T10:39:14.154051+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:13.089713+0000","last_clean_scrub_stamp":"2024-04-17T10:39:13.089713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:14:32.769003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.132716+0000","last_change":"2024-04-17T10:39:13.144322+0000","last_active":"2024-04-17T10:39:14.132716+0000","last_peered":"2024-04-17T10:39:14.132716+0000","last_clean":"2024-04-17T10:39:14.132716+0000","last_became_active":"2024-04-17T10:39:13.143607+0000","last_became_peered":"2024-04-17T10:39:13.143607+0000","last_unstale":"2024-04-17T10:39:14.132716+0000","last_undegraded":"2024-04-17T10:39:14.132716+0000","last_fullsized":"2024-04-17T10:39:14.132716+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:39:12.085786+0000","last_clean_scrub_stamp":"2024-04-17T10:39:12.085786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:31:02.964758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":194,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-17T10:39:14.115048+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:39:14.115048+0000","last_peered":"2024-04-17T10:39:14.115048+0000","last_clean":"2024-04-17T10:39:14.115048+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:39:14.115048+0000","last_undegraded":"2024-04-17T10:39:14.115048+0000","last_fullsized":"2024-04-17T10:39:14.115048+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":140980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":140412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:39:17.518 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-17T10:39:17.518 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-17T10:39:17.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.686+0000 7f681d603640 1 Processor -- start 2024-04-17T10:39:17.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.686+0000 7f681d603640 1 -- start start 2024-04-17T10:39:17.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.690+0000 7f681d603640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f681815bac0 0x7f6818071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:17.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.690+0000 7f681d603640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f681806c1f0 con 0x7f681815b6e0 2024-04-17T10:39:17.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.690+0000 7f681d603640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6818070a10 con 0x7f6818070470 2024-04-17T10:39:17.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.690+0000 7f681d603640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6818070cf0 con 0x7f681815bac0 2024-04-17T10:39:17.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.690+0000 7f6816ffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f681815b6e0 0x7f68180cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54066/0 (socket says 172.21.15.83:54066) 2024-04-17T10:39:17.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.690+0000 7f6816ffd640 1 -- 172.21.15.83:0/198061292 learned_addr learned my addr 172.21.15.83:0/198061292 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:39:17.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.690+0000 7f68167fc640 1 --2- 172.21.15.83:0/198061292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f681815bac0 0x7f6818071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:17.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.690+0000 7f6815ffb640 1 -- 172.21.15.83:0/198061292 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 283127028 0 0) 0x7f681806c1f0 con 0x7f681815b6e0 2024-04-17T10:39:17.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.690+0000 7f6815ffb640 1 -- 172.21.15.83:0/198061292 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6800003530 con 0x7f681815b6e0 2024-04-17T10:39:17.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.690+0000 7f6815ffb640 1 -- 172.21.15.83:0/198061292 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1784004634 0 0) 0x7f6800003530 con 0x7f681815b6e0 2024-04-17T10:39:17.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.690+0000 7f6815ffb640 1 -- 172.21.15.83:0/198061292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f681815bac0 msgr2=0x7f6818071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:17.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.690+0000 7f6815ffb640 1 --2- 172.21.15.83:0/198061292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f681815bac0 0x7f6818071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:17.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.690+0000 7f6815ffb640 1 -- 172.21.15.83:0/198061292 >> v1:172.21.15.167:6789/0 conn(0x7f6818070470 legacy=0x7f681806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:17.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.690+0000 7f6815ffb640 1 -- 172.21.15.83:0/198061292 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6818070fd0 con 0x7f681815b6e0 2024-04-17T10:39:17.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.690+0000 7f6815ffb640 1 -- 172.21.15.83:0/198061292 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f680c001680 con 0x7f681815b6e0 2024-04-17T10:39:17.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.690+0000 7f6815ffb640 1 -- 172.21.15.83:0/198061292 >> v1:172.21.15.83:6789/0 conn(0x7f681815b6e0 legacy=0x7f68180cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:17.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.690+0000 7f6815ffb640 1 --2- 172.21.15.83:0/198061292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f681815bac0 0x7f6800003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:17.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.690+0000 7f6815ffb640 1 --2- 172.21.15.83:0/198061292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6800003fe0 0x7f6800004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:17.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.690+0000 7f6815ffb640 1 --2- 172.21.15.83:0/198061292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6800004d50 0x7f68000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.690+0000 7f6815ffb640 1 -- 172.21.15.83:0/198061292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6818070cf0 con 0x7f6800003fe0 2024-04-17T10:39:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.690+0000 7f6815ffb640 1 -- 172.21.15.83:0/198061292 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680000bb30 con 0x7f6800004d50 2024-04-17T10:39:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.690+0000 7f6815ffb640 1 -- 172.21.15.83:0/198061292 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f680000bd30 con 0x7f681815bac0 2024-04-17T10:39:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.690+0000 7f68167fc640 1 --2- 172.21.15.83:0/198061292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6800003fe0 0x7f6800004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.690+0000 7f68177fe640 1 --2- 172.21.15.83:0/198061292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6800004d50 0x7f68000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.690+0000 7f6816ffd640 1 --2- 172.21.15.83:0/198061292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f681815bac0 0x7f6800003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.690+0000 7f68167fc640 1 -- 172.21.15.83:0/198061292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f681815bac0 msgr2=0x7f6800003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.690+0000 7f68167fc640 1 --2- 172.21.15.83:0/198061292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f681815bac0 0x7f6800003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.690+0000 7f68167fc640 1 -- 172.21.15.83:0/198061292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6800004d50 msgr2=0x7f68000051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.690+0000 7f68167fc640 1 --2- 172.21.15.83:0/198061292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6800004d50 0x7f68000051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.690+0000 7f68167fc640 1 -- 172.21.15.83:0/198061292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6800003530 con 0x7f6800003fe0 2024-04-17T10:39:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.690+0000 7f6816ffd640 1 --2- 172.21.15.83:0/198061292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f681815bac0 0x7f6800003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:17.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.694+0000 7f68167fc640 1 --2- 172.21.15.83:0/198061292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6800003fe0 0x7f6800004810 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6808009ab0 tx=0x7f680801c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ee207a8f767b52 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:17.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.694+0000 7f6815ffb640 1 -- 172.21.15.83:0/198061292 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68080b2070 con 0x7f6800003fe0 2024-04-17T10:39:17.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.694+0000 7f6815ffb640 1 -- 172.21.15.83:0/198061292 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68080a5580 con 0x7f6800003fe0 2024-04-17T10:39:17.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.694+0000 7f6815ffb640 1 -- 172.21.15.83:0/198061292 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68080096b0 con 0x7f6800003fe0 2024-04-17T10:39:17.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.694+0000 7f681d603640 1 -- 172.21.15.83:0/198061292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6800003fe0 msgr2=0x7f6800004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:17.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.694+0000 7f681d603640 1 --2- 172.21.15.83:0/198061292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6800003fe0 0x7f6800004810 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6808009ab0 tx=0x7f680801c990 comp rx=0 tx=0).stop 2024-04-17T10:39:17.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.694+0000 7f68177fe640 1 -- 172.21.15.83:0/198061292 reap_dead start 2024-04-17T10:39:17.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.694+0000 7f681d603640 1 -- 172.21.15.83:0/198061292 shutdown_connections 2024-04-17T10:39:17.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.694+0000 7f681d603640 1 -- 172.21.15.83:0/198061292 >> 172.21.15.83:0/198061292 conn(0x7f68180b15b0 msgr2=0x7f68180b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:17.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.694+0000 7f681d603640 1 -- 172.21.15.83:0/198061292 shutdown_connections 2024-04-17T10:39:17.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.694+0000 7f681d603640 1 -- 172.21.15.83:0/198061292 wait complete. 2024-04-17T10:39:17.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.694+0000 7f681d603640 1 Processor -- start 2024-04-17T10:39:17.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.694+0000 7f681d603640 1 -- start start 2024-04-17T10:39:17.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.694+0000 7f681d603640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6800003fe0 0x7f681815d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:17.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.694+0000 7f681d603640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f681815bf50 0x7f6818143f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:17.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.694+0000 7f681d603640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68180b06f0 0x7f68180b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:17.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.694+0000 7f681d603640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681806b6a0 con 0x7f68180b06f0 2024-04-17T10:39:17.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.694+0000 7f681d603640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681806b520 con 0x7f6800003fe0 2024-04-17T10:39:17.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.694+0000 7f681d603640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6818071cc0 con 0x7f681815bf50 2024-04-17T10:39:17.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.694+0000 7f68177fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68180b06f0 0x7f68180b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:17.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.694+0000 7f68177fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68180b06f0 0x7f68180b0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59936/0 (socket says 172.21.15.83:59936) 2024-04-17T10:39:17.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.694+0000 7f68177fe640 1 -- 172.21.15.83:0/3817259663 learned_addr learned my addr 172.21.15.83:0/3817259663 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:17.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.694+0000 7f68177fe640 1 -- 172.21.15.83:0/3817259663 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f681815bf50 msgr2=0x7f6818143f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:17.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.694+0000 7f68177fe640 1 --2- 172.21.15.83:0/3817259663 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f681815bf50 0x7f6818143f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:17.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.694+0000 7f68177fe640 1 -- 172.21.15.83:0/3817259663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6800003fe0 msgr2=0x7f681815d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:17.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.694+0000 7f68177fe640 1 --2- 172.21.15.83:0/3817259663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6800003fe0 0x7f681815d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:17.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.694+0000 7f68177fe640 1 -- 172.21.15.83:0/3817259663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f681815df30 con 0x7f68180b06f0 2024-04-17T10:39:17.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.698+0000 7f68177fe640 1 --2- 172.21.15.83:0/3817259663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68180b06f0 0x7f68180b0b70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f68180cd5b0 tx=0x7f681000e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:17.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.698+0000 7f67ff7fe640 1 -- 172.21.15.83:0/3817259663 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6810003b90 con 0x7f68180b06f0 2024-04-17T10:39:17.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.698+0000 7f67ff7fe640 1 -- 172.21.15.83:0/3817259663 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6810018070 con 0x7f68180b06f0 2024-04-17T10:39:17.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.698+0000 7f681d603640 1 -- 172.21.15.83:0/3817259663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f681815e210 con 0x7f68180b06f0 2024-04-17T10:39:17.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.698+0000 7f67ff7fe640 1 -- 172.21.15.83:0/3817259663 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68100050d0 con 0x7f68180b06f0 2024-04-17T10:39:17.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.698+0000 7f681d603640 1 -- 172.21.15.83:0/3817259663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f681815e7d0 con 0x7f68180b06f0 2024-04-17T10:39:17.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.698+0000 7f681d603640 1 -- 172.21.15.83:0/3817259663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6818074f90 con 0x7f68180b06f0 2024-04-17T10:39:17.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.698+0000 7f67ff7fe640 1 -- 172.21.15.83:0/3817259663 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6810005270 con 0x7f68180b06f0 2024-04-17T10:39:17.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.698+0000 7f67ff7fe640 1 --2- 172.21.15.83:0/3817259663 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6804066f60 0x7f6804069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:17.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.698+0000 7f67ff7fe640 1 -- 172.21.15.83:0/3817259663 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6810084c70 con 0x7f68180b06f0 2024-04-17T10:39:17.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.698+0000 7f6816ffd640 1 --2- 172.21.15.83:0/3817259663 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6804066f60 0x7f6804069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:17.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.698+0000 7f6816ffd640 1 --2- 172.21.15.83:0/3817259663 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6804066f60 0x7f6804069420 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f680c001930 tx=0x7f680c001120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:17.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.702+0000 7f67ff7fe640 1 -- 172.21.15.83:0/3817259663 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6810055c80 con 0x7f68180b06f0 2024-04-17T10:39:17.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.898+0000 7f681d603640 1 -- 172.21.15.83:0/3817259663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f68180d00d0 con 0x7f68180b06f0 2024-04-17T10:39:17.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.902+0000 7f67ff7fe640 1 -- 172.21.15.83:0/3817259663 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+147 (secure 0 0 0) 0x7f6810055580 con 0x7f68180b06f0 2024-04-17T10:39:17.902 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-04-17T10:39:17.903 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBUph9mHGmXGBAAZF+JiAG9YrFtbW+dCy2IEQ== 2024-04-17T10:39:17.903 INFO:teuthology.orchestra.run.smithi083.stdout: caps mds = "allow" 2024-04-17T10:39:17.903 INFO:teuthology.orchestra.run.smithi083.stdout: caps mon = "allow r" 2024-04-17T10:39:17.903 INFO:teuthology.orchestra.run.smithi083.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-17T10:39:17.903 INFO:teuthology.orchestra.run.smithi083.stderr:updated caps for client.0 2024-04-17T10:39:17.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.906+0000 7f681d603640 1 -- 172.21.15.83:0/3817259663 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6804066f60 msgr2=0x7f6804069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:17.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.906+0000 7f681d603640 1 --2- 172.21.15.83:0/3817259663 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6804066f60 0x7f6804069420 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f680c001930 tx=0x7f680c001120 comp rx=0 tx=0).stop 2024-04-17T10:39:17.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.906+0000 7f681d603640 1 -- 172.21.15.83:0/3817259663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68180b06f0 msgr2=0x7f68180b0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:17.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.906+0000 7f681d603640 1 --2- 172.21.15.83:0/3817259663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68180b06f0 0x7f68180b0b70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f68180cd5b0 tx=0x7f681000e9e0 comp rx=0 tx=0).stop 2024-04-17T10:39:17.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.906+0000 7f681d603640 1 -- 172.21.15.83:0/3817259663 shutdown_connections 2024-04-17T10:39:17.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.906+0000 7f681d603640 1 --2- 172.21.15.83:0/3817259663 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6804066f60 0x7f6804069420 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:17.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.906+0000 7f681d603640 1 --2- 172.21.15.83:0/3817259663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68180b06f0 0x7f68180b0b70 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:17.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.906+0000 7f681d603640 1 --2- 172.21.15.83:0/3817259663 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f681815bf50 0x7f6818143f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:17.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.906+0000 7f681d603640 1 --2- 172.21.15.83:0/3817259663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6800003fe0 0x7f681815d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:17.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.906+0000 7f681d603640 1 -- 172.21.15.83:0/3817259663 >> 172.21.15.83:0/3817259663 conn(0x7f68180b15b0 msgr2=0x7f68180cd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:17.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.906+0000 7f681d603640 1 -- 172.21.15.83:0/3817259663 shutdown_connections 2024-04-17T10:39:17.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:17.906+0000 7f681d603640 1 -- 172.21.15.83:0/3817259663 wait complete. 2024-04-17T10:39:17.920 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:39:18.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2e01f41640 1 Processor -- start 2024-04-17T10:39:18.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2e01f41640 1 -- start start 2024-04-17T10:39:18.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2e01f41640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2dfc15ba30 0x7f2dfc072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:18.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2e01f41640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dfc06e320 con 0x7f2dfc15c960 2024-04-17T10:39:18.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2e01f41640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dfc06e040 con 0x7f2dfc0cfbe0 2024-04-17T10:39:18.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2e01f41640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dfc071ea0 con 0x7f2dfc15ba30 2024-04-17T10:39:18.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2df3fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2dfc15c960 0x7f2dfc0cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54070/0 (socket says 172.21.15.83:54070) 2024-04-17T10:39:18.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2df3fff640 1 -- 172.21.15.83:0/2249916010 learned_addr learned my addr 172.21.15.83:0/2249916010 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:39:18.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2dfb7fe640 1 --2- 172.21.15.83:0/2249916010 >> v2:172.21.15.167:3300/0 conn(0x7f2dfc15ba30 0x7f2dfc072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:18.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2dfaffd640 1 -- 172.21.15.83:0/2249916010 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2002377710 0 0) 0x7f2dfc06e320 con 0x7f2dfc15c960 2024-04-17T10:39:18.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2dfaffd640 1 -- 172.21.15.83:0/2249916010 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2de8003530 con 0x7f2dfc15c960 2024-04-17T10:39:18.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2dfaffd640 1 -- 172.21.15.83:0/2249916010 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3374545550 0 0) 0x7f2de8003530 con 0x7f2dfc15c960 2024-04-17T10:39:18.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2dfaffd640 1 -- 172.21.15.83:0/2249916010 >> v2:172.21.15.167:3300/0 conn(0x7f2dfc15ba30 msgr2=0x7f2dfc072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:18.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2dfaffd640 1 --2- 172.21.15.83:0/2249916010 >> v2:172.21.15.167:3300/0 conn(0x7f2dfc15ba30 0x7f2dfc072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2dfaffd640 1 -- 172.21.15.83:0/2249916010 >> v1:172.21.15.167:6789/0 conn(0x7f2dfc0cfbe0 legacy=0x7f2dfc06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:18.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2dfaffd640 1 -- 172.21.15.83:0/2249916010 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dfc071850 con 0x7f2dfc15c960 2024-04-17T10:39:18.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2dfaffd640 1 -- 172.21.15.83:0/2249916010 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f2de40016a0 con 0x7f2dfc15c960 2024-04-17T10:39:18.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2dfaffd640 1 -- 172.21.15.83:0/2249916010 >> v1:172.21.15.83:6789/0 conn(0x7f2dfc15c960 legacy=0x7f2dfc0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:18.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2dfaffd640 1 --2- 172.21.15.83:0/2249916010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2de80039c0 0x7f2de8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:18.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2dfaffd640 1 --2- 172.21.15.83:0/2249916010 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2de8004810 0x7f2de8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:18.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2dfaffd640 1 --2- 172.21.15.83:0/2249916010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2de80051d0 0x7f2de800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:18.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2dfaffd640 1 -- 172.21.15.83:0/2249916010 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dfc071ea0 con 0x7f2de80039c0 2024-04-17T10:39:18.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2dfaffd640 1 -- 172.21.15.83:0/2249916010 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de800c170 con 0x7f2de80051d0 2024-04-17T10:39:18.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2dfaffd640 1 -- 172.21.15.83:0/2249916010 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de800c370 con 0x7f2de8004810 2024-04-17T10:39:18.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2dfb7fe640 1 --2- 172.21.15.83:0/2249916010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2de80039c0 0x7f2de8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:18.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2dfbfff640 1 --2- 172.21.15.83:0/2249916010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2de80051d0 0x7f2de800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:18.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2df3fff640 1 --2- 172.21.15.83:0/2249916010 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2de8004810 0x7f2de8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:18.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2dfb7fe640 1 -- 172.21.15.83:0/2249916010 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2de8004810 msgr2=0x7f2de8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:18.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2dfb7fe640 1 --2- 172.21.15.83:0/2249916010 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2de8004810 0x7f2de8004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2dfb7fe640 1 -- 172.21.15.83:0/2249916010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2de80051d0 msgr2=0x7f2de800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:18.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2dfb7fe640 1 --2- 172.21.15.83:0/2249916010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2de80051d0 0x7f2de800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2dfbfff640 1 -- 172.21.15.83:0/2249916010 reap_dead start 2024-04-17T10:39:18.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.038+0000 7f2dfb7fe640 1 -- 172.21.15.83:0/2249916010 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de8003530 con 0x7f2de80039c0 2024-04-17T10:39:18.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.042+0000 7f2dfb7fe640 1 --2- 172.21.15.83:0/2249916010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2de80039c0 0x7f2de8003de0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2dec009ae0 tx=0x7f2dec01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f16cb0966b7c92 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:18.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.042+0000 7f2dfaffd640 1 -- 172.21.15.83:0/2249916010 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec0096b0 con 0x7f2de80039c0 2024-04-17T10:39:18.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.042+0000 7f2dfaffd640 1 -- 172.21.15.83:0/2249916010 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dec0a6450 con 0x7f2de80039c0 2024-04-17T10:39:18.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.042+0000 7f2dfaffd640 1 -- 172.21.15.83:0/2249916010 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec0a6840 con 0x7f2de80039c0 2024-04-17T10:39:18.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.042+0000 7f2e01f41640 1 -- 172.21.15.83:0/2249916010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2de80039c0 msgr2=0x7f2de8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:18.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.042+0000 7f2e01f41640 1 --2- 172.21.15.83:0/2249916010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2de80039c0 0x7f2de8003de0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2dec009ae0 tx=0x7f2dec01ca70 comp rx=0 tx=0).stop 2024-04-17T10:39:18.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.042+0000 7f2e01f41640 1 -- 172.21.15.83:0/2249916010 shutdown_connections 2024-04-17T10:39:18.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.042+0000 7f2e01f41640 1 --2- 172.21.15.83:0/2249916010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2de80039c0 0x7f2de8003de0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.042+0000 7f2e01f41640 1 -- 172.21.15.83:0/2249916010 >> 172.21.15.83:0/2249916010 conn(0x7f2dfc06abe0 msgr2=0x7f2dfc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:18.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.042+0000 7f2e01f41640 1 -- 172.21.15.83:0/2249916010 shutdown_connections 2024-04-17T10:39:18.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.042+0000 7f2e01f41640 1 -- 172.21.15.83:0/2249916010 wait complete. 2024-04-17T10:39:18.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.042+0000 7f2e01f41640 1 Processor -- start 2024-04-17T10:39:18.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.042+0000 7f2e01f41640 1 -- start start 2024-04-17T10:39:18.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.042+0000 7f2e01f41640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2de80039c0 0x7f2dfc1516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:18.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.042+0000 7f2e01f41640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dfc15eb50 0x7f2dfc1f7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:18.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.042+0000 7f2e01f41640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2dfc154080 0x7f2dfc0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:18.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.042+0000 7f2e01f41640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dfc15cf00 con 0x7f2dfc15eb50 2024-04-17T10:39:18.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.042+0000 7f2e01f41640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dfc15cd80 con 0x7f2de80039c0 2024-04-17T10:39:18.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.042+0000 7f2e01f41640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dfc0cf8e0 con 0x7f2dfc154080 2024-04-17T10:39:18.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.046+0000 7f2df3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dfc15eb50 0x7f2dfc1f7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:18.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.046+0000 7f2df3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dfc15eb50 0x7f2dfc1f7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59950/0 (socket says 172.21.15.83:59950) 2024-04-17T10:39:18.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.046+0000 7f2df3fff640 1 -- 172.21.15.83:0/140879090 learned_addr learned my addr 172.21.15.83:0/140879090 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:18.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.046+0000 7f2dfb7fe640 1 --2- 172.21.15.83:0/140879090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2de80039c0 0x7f2dfc1516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:18.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.046+0000 7f2dfbfff640 1 --2- 172.21.15.83:0/140879090 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2dfc154080 0x7f2dfc0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:18.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.046+0000 7f2df3fff640 1 -- 172.21.15.83:0/140879090 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2dfc154080 msgr2=0x7f2dfc0ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.046+0000 7f2df3fff640 1 --2- 172.21.15.83:0/140879090 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2dfc154080 0x7f2dfc0ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.046+0000 7f2df3fff640 1 -- 172.21.15.83:0/140879090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2de80039c0 msgr2=0x7f2dfc1516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.046+0000 7f2df3fff640 1 --2- 172.21.15.83:0/140879090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2de80039c0 0x7f2dfc1516d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.046+0000 7f2df3fff640 1 -- 172.21.15.83:0/140879090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dfc1f7e70 con 0x7f2dfc15eb50 2024-04-17T10:39:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.046+0000 7f2dfbfff640 1 --2- 172.21.15.83:0/140879090 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2dfc154080 0x7f2dfc0ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:39:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.046+0000 7f2dfb7fe640 1 --2- 172.21.15.83:0/140879090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2de80039c0 0x7f2dfc1516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:39:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.046+0000 7f2df3fff640 1 --2- 172.21.15.83:0/140879090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dfc15eb50 0x7f2dfc1f7930 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2de4007fd0 tx=0x7f2de400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.046+0000 7f2df8ff9640 1 -- 172.21.15.83:0/140879090 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de4019070 con 0x7f2dfc15eb50 2024-04-17T10:39:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.046+0000 7f2df8ff9640 1 -- 172.21.15.83:0/140879090 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de400de50 con 0x7f2dfc15eb50 2024-04-17T10:39:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.046+0000 7f2e01f41640 1 -- 172.21.15.83:0/140879090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dfc0b2810 con 0x7f2dfc15eb50 2024-04-17T10:39:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.046+0000 7f2df8ff9640 1 -- 172.21.15.83:0/140879090 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de40047f0 con 0x7f2dfc15eb50 2024-04-17T10:39:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.046+0000 7f2e01f41640 1 -- 172.21.15.83:0/140879090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dfc0b2dd0 con 0x7f2dfc15eb50 2024-04-17T10:39:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.046+0000 7f2e01f41640 1 -- 172.21.15.83:0/140879090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dfc074f90 con 0x7f2dfc15eb50 2024-04-17T10:39:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.046+0000 7f2df8ff9640 1 -- 172.21.15.83:0/140879090 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2de401c070 con 0x7f2dfc15eb50 2024-04-17T10:39:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.046+0000 7f2df8ff9640 1 --2- 172.21.15.83:0/140879090 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2ddc066f90 0x7f2ddc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.046+0000 7f2df8ff9640 1 -- 172.21.15.83:0/140879090 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2de4084a10 con 0x7f2dfc15eb50 2024-04-17T10:39:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.046+0000 7f2dfb7fe640 1 --2- 172.21.15.83:0/140879090 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2ddc066f90 0x7f2ddc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.046+0000 7f2dfb7fe640 1 --2- 172.21.15.83:0/140879090 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2ddc066f90 0x7f2ddc069450 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2dec0ad000 tx=0x7f2dec0b2040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:18.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.050+0000 7f2df8ff9640 1 -- 172.21.15.83:0/140879090 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2dfc1f7e70 con 0x7f2dfc15eb50 2024-04-17T10:39:18.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.246+0000 7f2e01f41640 1 -- 172.21.15.83:0/140879090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2dfc158560 con 0x7f2dfc15eb50 2024-04-17T10:39:18.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.246+0000 7f2df8ff9640 1 -- 172.21.15.83:0/140879090 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f2de4055a20 con 0x7f2dfc15eb50 2024-04-17T10:39:18.244 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:39:18.244 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1456029744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6833","nonce":1456029744},{"type":"v1","addr":"172.21.15.167:6835","nonce":1456029744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6834/648555908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":648555908},{"type":"v1","addr":"172.21.15.167:6834","nonce":648555908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5163,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1435807773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1435807773},{"type":"v1","addr":"172.21.15.83:6835","nonce":1435807773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:39:13.472135+0000","modified":"2024-04-17T10:39:15.518124+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5172},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5172":{"gid":5172,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1181902025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1181902025},{"type":"v1","addr":"172.21.15.83:6837","nonce":1181902025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-17T10:39:18.245 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 13 2024-04-17T10:39:18.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.246+0000 7f2e01f41640 1 -- 172.21.15.83:0/140879090 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2ddc066f90 msgr2=0x7f2ddc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:18.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.246+0000 7f2e01f41640 1 --2- 172.21.15.83:0/140879090 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2ddc066f90 0x7f2ddc069450 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2dec0ad000 tx=0x7f2dec0b2040 comp rx=0 tx=0).stop 2024-04-17T10:39:18.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.246+0000 7f2e01f41640 1 -- 172.21.15.83:0/140879090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dfc15eb50 msgr2=0x7f2dfc1f7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:18.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.246+0000 7f2e01f41640 1 --2- 172.21.15.83:0/140879090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dfc15eb50 0x7f2dfc1f7930 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2de4007fd0 tx=0x7f2de400dc00 comp rx=0 tx=0).stop 2024-04-17T10:39:18.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.246+0000 7f2e01f41640 1 -- 172.21.15.83:0/140879090 shutdown_connections 2024-04-17T10:39:18.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.246+0000 7f2e01f41640 1 --2- 172.21.15.83:0/140879090 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2ddc066f90 0x7f2ddc069450 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.246+0000 7f2e01f41640 1 --2- 172.21.15.83:0/140879090 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2dfc154080 0x7f2dfc0ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.246+0000 7f2e01f41640 1 --2- 172.21.15.83:0/140879090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dfc15eb50 0x7f2dfc1f7930 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.246+0000 7f2e01f41640 1 --2- 172.21.15.83:0/140879090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2de80039c0 0x7f2dfc1516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.246+0000 7f2e01f41640 1 -- 172.21.15.83:0/140879090 >> 172.21.15.83:0/140879090 conn(0x7f2dfc06abe0 msgr2=0x7f2dfc15f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:18.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.246+0000 7f2e01f41640 1 -- 172.21.15.83:0/140879090 shutdown_connections 2024-04-17T10:39:18.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.246+0000 7f2e01f41640 1 -- 172.21.15.83:0/140879090 wait complete. 2024-04-17T10:39:18.664 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 13, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T10:39:13.472135+0000', 'modified': '2024-04-17T10:39:15.518124+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5172}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5172': {'gid': 5172, 'name': 'a', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6837/1181902025', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6836', 'nonce': 1181902025}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 1181902025}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:39:18.664 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T10:39:18.664 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-17T10:39:18.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.794+0000 7eff1cc9b640 1 Processor -- start 2024-04-17T10:39:18.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.794+0000 7eff1cc9b640 1 -- start start 2024-04-17T10:39:18.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.794+0000 7eff1cc9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7eff1815da50 0x7eff180ae950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:18.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.794+0000 7eff1cc9b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff180af9e0 con 0x7eff180afe20 2024-04-17T10:39:18.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.794+0000 7eff1cc9b640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff180b4e80 con 0x7eff1806b2f0 2024-04-17T10:39:18.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.794+0000 7eff1cc9b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff180af3a0 con 0x7eff1815da50 2024-04-17T10:39:18.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.798+0000 7eff1659c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7eff1815da50 0x7eff180ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:18.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.798+0000 7eff0dd9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7eff180afe20 0x7eff1806abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54078/0 (socket says 172.21.15.83:54078) 2024-04-17T10:39:18.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.798+0000 7eff0dd9b640 1 -- 172.21.15.83:0/153569103 learned_addr learned my addr 172.21.15.83:0/153569103 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:39:18.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.798+0000 7eff1659c640 1 -- 172.21.15.83:0/153569103 >> v1:172.21.15.167:6789/0 conn(0x7eff1806b2f0 legacy=0x7eff1806dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:18.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.798+0000 7eff1659c640 1 -- 172.21.15.83:0/153569103 >> v1:172.21.15.83:6789/0 conn(0x7eff180afe20 legacy=0x7eff1806abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:18.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.798+0000 7eff1659c640 1 -- 172.21.15.83:0/153569103 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff1815de30 con 0x7eff1815da50 2024-04-17T10:39:18.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.798+0000 7eff1659c640 1 --2- 172.21.15.83:0/153569103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff1815da50 0x7eff180ae950 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7eff080097c0 tx=0x7eff0801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=435a29a61b6147ab server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:18.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.798+0000 7eff15d9b640 1 -- 172.21.15.83:0/153569103 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff080b0070 con 0x7eff1815da50 2024-04-17T10:39:18.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.798+0000 7eff15d9b640 1 -- 172.21.15.83:0/153569103 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff080ab600 con 0x7eff1815da50 2024-04-17T10:39:18.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.798+0000 7eff15d9b640 1 -- 172.21.15.83:0/153569103 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff080ab9d0 con 0x7eff1815da50 2024-04-17T10:39:18.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.798+0000 7eff1cc9b640 1 -- 172.21.15.83:0/153569103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff1815da50 msgr2=0x7eff180ae950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:18.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.798+0000 7eff1cc9b640 1 --2- 172.21.15.83:0/153569103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff1815da50 0x7eff180ae950 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7eff080097c0 tx=0x7eff0801c460 comp rx=0 tx=0).stop 2024-04-17T10:39:18.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.798+0000 7eff1cc9b640 1 -- 172.21.15.83:0/153569103 shutdown_connections 2024-04-17T10:39:18.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.798+0000 7eff1cc9b640 1 --2- 172.21.15.83:0/153569103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff1815da50 0x7eff180ae950 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.798+0000 7eff1cc9b640 1 -- 172.21.15.83:0/153569103 >> 172.21.15.83:0/153569103 conn(0x7eff180b24f0 msgr2=0x7eff180b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:18.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.798+0000 7eff1cc9b640 1 -- 172.21.15.83:0/153569103 shutdown_connections 2024-04-17T10:39:18.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.802+0000 7eff1cc9b640 1 -- 172.21.15.83:0/153569103 wait complete. 2024-04-17T10:39:18.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.802+0000 7eff1cc9b640 1 Processor -- start 2024-04-17T10:39:18.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.802+0000 7eff1cc9b640 1 -- start start 2024-04-17T10:39:18.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.802+0000 7eff1cc9b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff1806b2f0 0x7eff1814f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:18.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.802+0000 7eff1cc9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff180afe20 0x7eff1815ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:18.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.802+0000 7eff1cc9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff1815da50 0x7eff1815f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:18.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.802+0000 7eff1cc9b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1806ec40 con 0x7eff1815da50 2024-04-17T10:39:18.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.802+0000 7eff1cc9b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1806eac0 con 0x7eff180afe20 2024-04-17T10:39:18.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.802+0000 7eff1cc9b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff1816b650 con 0x7eff1806b2f0 2024-04-17T10:39:18.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.802+0000 7eff16d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff1815da50 0x7eff1815f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:18.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.802+0000 7eff16d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff1815da50 0x7eff1815f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59980/0 (socket says 172.21.15.83:59980) 2024-04-17T10:39:18.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.802+0000 7eff16d9d640 1 -- 172.21.15.83:0/2137742711 learned_addr learned my addr 172.21.15.83:0/2137742711 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:18.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.802+0000 7eff0dd9b640 1 --2- 172.21.15.83:0/2137742711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff180afe20 0x7eff1815ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:18.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.802+0000 7eff1659c640 1 --2- 172.21.15.83:0/2137742711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff1806b2f0 0x7eff1814f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:18.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.802+0000 7eff16d9d640 1 -- 172.21.15.83:0/2137742711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff1806b2f0 msgr2=0x7eff1814f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:18.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.802+0000 7eff16d9d640 1 --2- 172.21.15.83:0/2137742711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff1806b2f0 0x7eff1814f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.802+0000 7eff16d9d640 1 -- 172.21.15.83:0/2137742711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff180afe20 msgr2=0x7eff1815ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:18.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.802+0000 7eff16d9d640 1 --2- 172.21.15.83:0/2137742711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff180afe20 0x7eff1815ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.802+0000 7eff16d9d640 1 -- 172.21.15.83:0/2137742711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff1814f630 con 0x7eff1815da50 2024-04-17T10:39:18.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.802+0000 7eff0dd9b640 1 --2- 172.21.15.83:0/2137742711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff180afe20 0x7eff1815ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:18.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.802+0000 7eff16d9d640 1 --2- 172.21.15.83:0/2137742711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff1815da50 0x7eff1815f1b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7eff1000ba10 tx=0x7eff1000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:18.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.802+0000 7eff0f7fe640 1 -- 172.21.15.83:0/2137742711 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff10019070 con 0x7eff1815da50 2024-04-17T10:39:18.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.802+0000 7eff0f7fe640 1 -- 172.21.15.83:0/2137742711 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff1000cb40 con 0x7eff1815da50 2024-04-17T10:39:18.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.802+0000 7eff1cc9b640 1 -- 172.21.15.83:0/2137742711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff18160e90 con 0x7eff1815da50 2024-04-17T10:39:18.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.802+0000 7eff0f7fe640 1 -- 172.21.15.83:0/2137742711 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff10014440 con 0x7eff1815da50 2024-04-17T10:39:18.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.802+0000 7eff1cc9b640 1 -- 172.21.15.83:0/2137742711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff1815fb40 con 0x7eff1815da50 2024-04-17T10:39:18.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7eff0f7fe640 1 -- 172.21.15.83:0/2137742711 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7eff1001c070 con 0x7eff1815da50 2024-04-17T10:39:18.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7eff0f7fe640 1 --2- 172.21.15.83:0/2137742711 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efef8066f60 0x7efef8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:18.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7eff0f7fe640 1 -- 172.21.15.83:0/2137742711 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff10084ca0 con 0x7eff1815da50 2024-04-17T10:39:18.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7eff1659c640 1 --2- 172.21.15.83:0/2137742711 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efef8066f60 0x7efef8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:18.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7eff1659c640 1 --2- 172.21.15.83:0/2137742711 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efef8066f60 0x7efef8069420 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7eff080ad000 tx=0x7eff08008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:18.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7eff1fe08000 1 Processor -- start 2024-04-17T10:39:18.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7eff1fe08000 1 -- start start 2024-04-17T10:39:18.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7eff1fe08000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649db6ff920 0x5649db6ffd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:18.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7eff1fe08000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5649db700240 0x5649db6b59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:18.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7eff1fe08000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5649db6b5f30 0x5649db6b6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:18.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7eff1fe08000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649db59d5f0 con 0x5649db700240 2024-04-17T10:39:18.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7eff1fe08000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649db504da0 con 0x5649db6ff920 2024-04-17T10:39:18.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7eff1fe08000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5649db48d150 con 0x5649db6b5f30 2024-04-17T10:39:18.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7eff0dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5649db700240 0x5649db6b59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:18.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7eff0dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5649db700240 0x5649db6b59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59984/0 (socket says 172.21.15.83:59984) 2024-04-17T10:39:18.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7eff0dd9b640 1 -- 172.21.15.83:0/2729680492 learned_addr learned my addr 172.21.15.83:0/2729680492 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:18.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7eff0dd9b640 1 -- 172.21.15.83:0/2729680492 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5649db6b5f30 msgr2=0x5649db6b6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:18.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7eff0dd9b640 1 --2- 172.21.15.83:0/2729680492 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5649db6b5f30 0x5649db6b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7eff0dd9b640 1 -- 172.21.15.83:0/2729680492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649db6ff920 msgr2=0x5649db6ffd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:18.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7eff0dd9b640 1 --2- 172.21.15.83:0/2729680492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649db6ff920 0x5649db6ffd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7eff0dd9b640 1 -- 172.21.15.83:0/2729680492 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649db4a6ca0 con 0x5649db700240 2024-04-17T10:39:18.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7eff0dd9b640 1 --2- 172.21.15.83:0/2729680492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5649db700240 0x5649db6b59f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7eff1815e240 tx=0x7eff00001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb027eced5b8c51 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:18.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7efef7fff640 1 -- 172.21.15.83:0/2729680492 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff00018070 con 0x5649db700240 2024-04-17T10:39:18.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7efef7fff640 1 -- 172.21.15.83:0/2729680492 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff00005cf0 con 0x5649db700240 2024-04-17T10:39:18.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7efef7fff640 1 -- 172.21.15.83:0/2729680492 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff000072f0 con 0x5649db700240 2024-04-17T10:39:18.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7eff1fe08000 1 -- 172.21.15.83:0/2729680492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5649db700240 msgr2=0x5649db6b59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:18.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7eff1fe08000 1 --2- 172.21.15.83:0/2729680492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5649db700240 0x5649db6b59f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7eff1815e240 tx=0x7eff00001b80 comp rx=0 tx=0).stop 2024-04-17T10:39:18.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7eff1fe08000 1 -- 172.21.15.83:0/2729680492 shutdown_connections 2024-04-17T10:39:18.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7eff1fe08000 1 --2- 172.21.15.83:0/2729680492 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5649db6b5f30 0x5649db6b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7eff1fe08000 1 --2- 172.21.15.83:0/2729680492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5649db700240 0x5649db6b59f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7eff1fe08000 1 --2- 172.21.15.83:0/2729680492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649db6ff920 0x5649db6ffd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7eff1fe08000 1 -- 172.21.15.83:0/2729680492 >> 172.21.15.83:0/2729680492 conn(0x5649db6bc2a0 msgr2=0x5649db6b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:18.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7eff1fe08000 1 -- 172.21.15.83:0/2729680492 shutdown_connections 2024-04-17T10:39:18.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.806+0000 7eff1fe08000 1 -- 172.21.15.83:0/2729680492 wait complete. 2024-04-17T10:39:18.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff1fe08000 1 -- start start 2024-04-17T10:39:18.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff1fe08000 1 Processor -- start 2024-04-17T10:39:18.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff1fe08000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649db6b5f30 0x5649db706dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:18.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff1fe08000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5649db6ff920 0x5649db707310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:18.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff1fe08000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5649db700240 0x5649db6b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:18.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff1fe08000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649db4bf990 con 0x5649db6ff920 2024-04-17T10:39:18.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff1fe08000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649db500e40 con 0x5649db6b5f30 2024-04-17T10:39:18.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff1fe08000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5649db490900 con 0x5649db700240 2024-04-17T10:39:18.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff1fe08000 10 client.0 _ll_register_callbacks cb 0x5649db5b81c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:39:18.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff0dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5649db6ff920 0x5649db707310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:18.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff0dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5649db6ff920 0x5649db707310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59998/0 (socket says 172.21.15.83:59998) 2024-04-17T10:39:18.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff0dd9b640 1 -- 172.21.15.83:0/2819811366 learned_addr learned my addr 172.21.15.83:0/2819811366 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:18.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff1659c640 1 --2- 172.21.15.83:0/2819811366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649db6b5f30 0x5649db706dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:18.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff0dd9b640 1 -- 172.21.15.83:0/2819811366 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5649db700240 msgr2=0x5649db6b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:18.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff16d9d640 1 --2- 172.21.15.83:0/2819811366 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5649db700240 0x5649db6b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:18.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff0dd9b640 1 --2- 172.21.15.83:0/2819811366 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5649db700240 0x5649db6b9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff0dd9b640 1 -- 172.21.15.83:0/2819811366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649db6b5f30 msgr2=0x5649db706dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:18.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff0dd9b640 1 --2- 172.21.15.83:0/2819811366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649db6b5f30 0x5649db706dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff0dd9b640 1 -- 172.21.15.83:0/2819811366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5649db4bd630 con 0x5649db6ff920 2024-04-17T10:39:18.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff1659c640 1 --2- 172.21.15.83:0/2819811366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649db6b5f30 0x5649db706dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:18.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff0dd9b640 1 --2- 172.21.15.83:0/2819811366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5649db6ff920 0x5649db707310 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7eff00001bc0 tx=0x7eff00000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3ebfd0864df20d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:18.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7efef57fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:39:18.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7efef57fa640 1 -- 172.21.15.83:0/2819811366 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff00023020 con 0x5649db6ff920 2024-04-17T10:39:18.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7efef57fa640 1 -- 172.21.15.83:0/2819811366 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff00005af0 con 0x5649db6ff920 2024-04-17T10:39:18.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff1fe08000 1 -- 172.21.15.83:0/2819811366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff00005cf0 con 0x5649db6ff920 2024-04-17T10:39:18.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7efef57fa640 1 -- 172.21.15.83:0/2819811366 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff0002d050 con 0x5649db6ff920 2024-04-17T10:39:18.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff1fe08000 10 client.5352 fetch_fsmap learned FSMap version 13 2024-04-17T10:39:18.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff1fe08000 1 -- 172.21.15.83:0/2819811366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x5649db48d3a0 con 0x5649db6ff920 2024-04-17T10:39:18.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7efef57fa640 1 -- 172.21.15.83:0/2819811366 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff00005cf0 con 0x5649db6ff920 2024-04-17T10:39:18.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7efef57fa640 1 -- 172.21.15.83:0/2819811366 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 13) v1 ==== 3294+0+0 (secure 0 0 0) 0x7eff00030020 con 0x5649db6ff920 2024-04-17T10:39:18.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff1fe08000 10 client.5352 fetch_fsmap finished waiting for FSMap version 13 2024-04-17T10:39:18.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff1fe08000 10 client.5352 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T10:39:18.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff1fe08000 20 client.5352 populate_metadata read hostname 'smithi083' 2024-04-17T10:39:18.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff1fe08000 1 --2- 172.21.15.83:0/2819811366 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x5649db6bb160 0x5649db710e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:18.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff1fe08000 4 client.5352 mds_command: new command op to 5172 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:39:18.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff1fe08000 1 -- 172.21.15.83:0/2819811366 --> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5649db4a0a50 con 0x5649db6bb160 2024-04-17T10:39:18.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff1659c640 1 --2- 172.21.15.83:0/2819811366 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x5649db6bb160 0x5649db710e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:18.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7efef57fa640 10 client.5352 ms_handle_connect on v2:172.21.15.83:6836/1181902025 2024-04-17T10:39:18.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff1659c640 1 --2- 172.21.15.83:0/2819811366 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x5649db6bb160 0x5649db710e30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e566747a3d3bd6b5 server_cookie=9fb275667babd27b in_seq=0 out_seq=0 2024-04-17T10:39:18.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7efef57fa640 1 -- 172.21.15.83:0/2819811366 <== mds.0 v2:172.21.15.83:6836/1181902025 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5649db4a0a50 con 0x5649db6bb160 2024-04-17T10:39:18.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7efef57fa640 10 client.5352 handle_command_reply: tid=0 2024-04-17T10:39:18.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff1fe08000 1 client.5352 shutdown 2024-04-17T10:39:18.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff1fe08000 1 -- 172.21.15.83:0/2819811366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5649db6ff920 msgr2=0x5649db707310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:18.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.810+0000 7eff1fe08000 1 --2- 172.21.15.83:0/2819811366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5649db6ff920 0x5649db707310 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7eff00001bc0 tx=0x7eff00000bd0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.814+0000 7eff1fe08000 1 -- 172.21.15.83:0/2819811366 shutdown_connections 2024-04-17T10:39:18.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.814+0000 7eff1fe08000 1 --2- 172.21.15.83:0/2819811366 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x5649db6bb160 0x5649db710e30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.814+0000 7eff1fe08000 1 --2- 172.21.15.83:0/2819811366 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5649db700240 0x5649db6b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.814+0000 7eff1fe08000 1 --2- 172.21.15.83:0/2819811366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5649db6ff920 0x5649db707310 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.814+0000 7efef57fa640 0 client.5352 ms_handle_reset on v2:172.21.15.83:6836/1181902025 2024-04-17T10:39:18.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.814+0000 7eff1fe08000 1 --2- 172.21.15.83:0/2819811366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649db6b5f30 0x5649db706dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.814+0000 7eff1fe08000 1 -- 172.21.15.83:0/2819811366 >> 172.21.15.83:0/2819811366 conn(0x5649db6bc2a0 msgr2=0x5649db6b49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:18.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.814+0000 7eff1fe08000 1 -- 172.21.15.83:0/2819811366 shutdown_connections 2024-04-17T10:39:18.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.814+0000 7eff1fe08000 1 -- 172.21.15.83:0/2819811366 wait complete. 2024-04-17T10:39:18.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.814+0000 7eff1fe08000 20 client.5352 trim_cache size 0 max 16384 2024-04-17T10:39:18.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.826+0000 7eff1fe08000 1 Processor -- start 2024-04-17T10:39:18.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.826+0000 7eff1fe08000 1 -- start start 2024-04-17T10:39:18.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.826+0000 7eff1fe08000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5649db6bc2a0 0x5649db6ebc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:18.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.826+0000 7eff1fe08000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649db6b5f30 0x5649db6ec190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:18.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.826+0000 7eff1fe08000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5649db6ff920 0x5649db709a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:18.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.826+0000 7eff1fe08000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff18201f60 con 0x5649db6ff920 2024-04-17T10:39:18.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.826+0000 7eff1fe08000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff18201420 con 0x5649db6b5f30 2024-04-17T10:39:18.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.826+0000 7eff1fe08000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff18201960 con 0x5649db6bc2a0 2024-04-17T10:39:18.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.826+0000 7eff16d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5649db6ff920 0x5649db709a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:18.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.826+0000 7eff16d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5649db6ff920 0x5649db709a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60010/0 (socket says 172.21.15.83:60010) 2024-04-17T10:39:18.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.826+0000 7eff16d9d640 1 -- 172.21.15.83:0/2533157843 learned_addr learned my addr 172.21.15.83:0/2533157843 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:18.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff16d9d640 1 -- 172.21.15.83:0/2533157843 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5649db6bc2a0 msgr2=0x5649db6ebc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:18.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff16d9d640 1 --2- 172.21.15.83:0/2533157843 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5649db6bc2a0 0x5649db6ebc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff16d9d640 1 -- 172.21.15.83:0/2533157843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649db6b5f30 msgr2=0x5649db6ec190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:18.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff16d9d640 1 --2- 172.21.15.83:0/2533157843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649db6b5f30 0x5649db6ec190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff16d9d640 1 -- 172.21.15.83:0/2533157843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649db700070 con 0x5649db6ff920 2024-04-17T10:39:18.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff16d9d640 1 --2- 172.21.15.83:0/2533157843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5649db6ff920 0x5649db709a00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7eff10004ac0 tx=0x7eff10055d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23cf8b378e3c4c93 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:18.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7efef77fe640 1 -- 172.21.15.83:0/2533157843 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1005dcb0 con 0x5649db6ff920 2024-04-17T10:39:18.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7efef77fe640 1 -- 172.21.15.83:0/2533157843 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff100538a0 con 0x5649db6ff920 2024-04-17T10:39:18.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7efef77fe640 1 -- 172.21.15.83:0/2533157843 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1005d820 con 0x5649db6ff920 2024-04-17T10:39:18.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff1fe08000 1 -- 172.21.15.83:0/2533157843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5649db6ff920 msgr2=0x5649db709a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:18.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff1fe08000 1 --2- 172.21.15.83:0/2533157843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5649db6ff920 0x5649db709a00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7eff10004ac0 tx=0x7eff10055d60 comp rx=0 tx=0).stop 2024-04-17T10:39:18.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff1fe08000 1 -- 172.21.15.83:0/2533157843 shutdown_connections 2024-04-17T10:39:18.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff1fe08000 1 --2- 172.21.15.83:0/2533157843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5649db6ff920 0x5649db709a00 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff1fe08000 1 --2- 172.21.15.83:0/2533157843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649db6b5f30 0x5649db6ec190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff1fe08000 1 --2- 172.21.15.83:0/2533157843 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5649db6bc2a0 0x5649db6ebc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff1fe08000 1 -- 172.21.15.83:0/2533157843 >> 172.21.15.83:0/2533157843 conn(0x5649db706e40 msgr2=0x5649db6b9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:18.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff1fe08000 1 -- 172.21.15.83:0/2533157843 shutdown_connections 2024-04-17T10:39:18.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff1fe08000 1 -- 172.21.15.83:0/2533157843 wait complete. 2024-04-17T10:39:18.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff1fe08000 1 -- start start 2024-04-17T10:39:18.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff1fe08000 1 Processor -- start 2024-04-17T10:39:18.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff1fe08000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5649db6b5f30 0x5649db760f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:18.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff1fe08000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649db6bc2a0 0x5649db761490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:18.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff1fe08000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5649db6ff920 0x5649db744ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:18.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff1fe08000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649db5093d0 con 0x5649db6ff920 2024-04-17T10:39:18.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff1fe08000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649db5c0d50 con 0x5649db6bc2a0 2024-04-17T10:39:18.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff1fe08000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5649db49f980 con 0x5649db6b5f30 2024-04-17T10:39:18.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff1fe08000 10 client.0 _ll_register_callbacks cb 0x7eff00007e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:39:18.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff16d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5649db6ff920 0x5649db744ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:18.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff16d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5649db6ff920 0x5649db744ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60020/0 (socket says 172.21.15.83:60020) 2024-04-17T10:39:18.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff16d9d640 1 -- 172.21.15.83:0/2380419294 learned_addr learned my addr 172.21.15.83:0/2380419294 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:18.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff0dd9b640 1 --2- 172.21.15.83:0/2380419294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649db6bc2a0 0x5649db761490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:18.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff16d9d640 1 -- 172.21.15.83:0/2380419294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5649db6b5f30 msgr2=0x5649db760f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:18.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff16d9d640 1 --2- 172.21.15.83:0/2380419294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5649db6b5f30 0x5649db760f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff16d9d640 1 -- 172.21.15.83:0/2380419294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649db6bc2a0 msgr2=0x5649db761490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:18.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff16d9d640 1 --2- 172.21.15.83:0/2380419294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649db6bc2a0 0x5649db761490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff16d9d640 1 -- 172.21.15.83:0/2380419294 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5649db586600 con 0x5649db6ff920 2024-04-17T10:39:18.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff16d9d640 1 --2- 172.21.15.83:0/2380419294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5649db6ff920 0x5649db744ca0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7eff1000b9e0 tx=0x7eff10009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fdd328706cc646b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:18.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7efef57fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:39:18.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7efef57fa640 1 -- 172.21.15.83:0/2380419294 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1005dcb0 con 0x5649db6ff920 2024-04-17T10:39:18.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7efef57fa640 1 -- 172.21.15.83:0/2380419294 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff1005da30 con 0x5649db6ff920 2024-04-17T10:39:18.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff1fe08000 1 -- 172.21.15.83:0/2380419294 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff100538a0 con 0x5649db6ff920 2024-04-17T10:39:18.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7efef57fa640 1 -- 172.21.15.83:0/2380419294 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff100549f0 con 0x5649db6ff920 2024-04-17T10:39:18.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff1fe08000 10 client.5358 fetch_fsmap learned FSMap version 13 2024-04-17T10:39:18.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff1fe08000 1 -- 172.21.15.83:0/2380419294 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x5649db4999b0 con 0x5649db6ff920 2024-04-17T10:39:18.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7efef57fa640 1 -- 172.21.15.83:0/2380419294 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff100538a0 con 0x5649db6ff920 2024-04-17T10:39:18.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7efef57fa640 1 -- 172.21.15.83:0/2380419294 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 13) v1 ==== 3294+0+0 (secure 0 0 0) 0x7eff10054b90 con 0x5649db6ff920 2024-04-17T10:39:18.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff1fe08000 10 client.5358 fetch_fsmap finished waiting for FSMap version 13 2024-04-17T10:39:18.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff1fe08000 10 client.5358 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T10:39:18.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff1fe08000 20 client.5358 populate_metadata read hostname 'smithi083' 2024-04-17T10:39:18.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff1fe08000 1 --2- 172.21.15.83:0/2380419294 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x5649db700240 0x5649db7451e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff1fe08000 4 client.5358 mds_command: new command op to 5172 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T10:39:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff1fe08000 1 -- 172.21.15.83:0/2380419294 --> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5649db5122e0 con 0x5649db700240 2024-04-17T10:39:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.830+0000 7eff1659c640 1 --2- 172.21.15.83:0/2380419294 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x5649db700240 0x5649db7451e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.834+0000 7efef57fa640 10 client.5358 ms_handle_connect on v2:172.21.15.83:6836/1181902025 2024-04-17T10:39:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.834+0000 7eff1659c640 1 --2- 172.21.15.83:0/2380419294 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x5649db700240 0x5649db7451e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e98efb0fadf39fa server_cookie=ac99bfaf918af577 in_seq=0 out_seq=0 2024-04-17T10:39:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.834+0000 7efef57fa640 1 -- 172.21.15.83:0/2380419294 <== mds.0 v2:172.21.15.83:6836/1181902025 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x5649db5122e0 con 0x5649db700240 2024-04-17T10:39:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.834+0000 7efef57fa640 10 client.5358 handle_command_reply: tid=0 2024-04-17T10:39:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.834+0000 7eff1fe08000 1 client.5358 shutdown 2024-04-17T10:39:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.834+0000 7eff1fe08000 1 -- 172.21.15.83:0/2380419294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5649db6ff920 msgr2=0x5649db744ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.834+0000 7eff1fe08000 1 --2- 172.21.15.83:0/2380419294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5649db6ff920 0x5649db744ca0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7eff1000b9e0 tx=0x7eff10009f60 comp rx=0 tx=0).stop 2024-04-17T10:39:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.834+0000 7eff1fe08000 1 -- 172.21.15.83:0/2380419294 shutdown_connections 2024-04-17T10:39:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.834+0000 7eff1fe08000 1 --2- 172.21.15.83:0/2380419294 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x5649db700240 0x5649db7451e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.834+0000 7eff1fe08000 1 --2- 172.21.15.83:0/2380419294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5649db6ff920 0x5649db744ca0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.834+0000 7efef57fa640 0 client.5358 ms_handle_reset on v2:172.21.15.83:6836/1181902025 2024-04-17T10:39:18.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.834+0000 7eff1fe08000 1 --2- 172.21.15.83:0/2380419294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649db6bc2a0 0x5649db761490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.834+0000 7eff1fe08000 1 --2- 172.21.15.83:0/2380419294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5649db6b5f30 0x5649db760f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.834+0000 7eff1fe08000 1 -- 172.21.15.83:0/2380419294 >> 172.21.15.83:0/2380419294 conn(0x5649db706e40 msgr2=0x5649db6eac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:18.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.834+0000 7eff1fe08000 1 -- 172.21.15.83:0/2380419294 shutdown_connections 2024-04-17T10:39:18.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.834+0000 7eff1fe08000 1 -- 172.21.15.83:0/2380419294 wait complete. 2024-04-17T10:39:18.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.834+0000 7eff1fe08000 20 client.5358 trim_cache size 0 max 16384 2024-04-17T10:39:18.838 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:39:18.838 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "3de9b7be-af7f-488a-969e-2b6b8db63bcd", 2024-04-17T10:39:18.838 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-04-17T10:39:18.839 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 5172, 2024-04-17T10:39:18.839 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-17T10:39:18.839 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-17T10:39:18.839 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-17T10:39:18.839 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 4.735050771, 2024-04-17T10:39:18.839 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 13, 2024-04-17T10:39:18.839 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 33, 2024-04-17T10:39:18.839 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 33, 2024-04-17T10:39:18.839 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 12.206172867999999 2024-04-17T10:39:18.839 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:39:18.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.834+0000 7eff1cc9b640 1 -- 172.21.15.83:0/2137742711 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efef8066f60 msgr2=0x7efef8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:18.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.834+0000 7eff1cc9b640 1 --2- 172.21.15.83:0/2137742711 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efef8066f60 0x7efef8069420 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7eff080ad000 tx=0x7eff08008040 comp rx=0 tx=0).stop 2024-04-17T10:39:18.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.834+0000 7eff1cc9b640 1 -- 172.21.15.83:0/2137742711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff1815da50 msgr2=0x7eff1815f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:18.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.834+0000 7eff1cc9b640 1 --2- 172.21.15.83:0/2137742711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff1815da50 0x7eff1815f1b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7eff1000ba10 tx=0x7eff1000bed0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.834+0000 7eff1cc9b640 1 -- 172.21.15.83:0/2137742711 shutdown_connections 2024-04-17T10:39:18.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.834+0000 7eff1cc9b640 1 --2- 172.21.15.83:0/2137742711 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efef8066f60 0x7efef8069420 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.834+0000 7eff1cc9b640 1 --2- 172.21.15.83:0/2137742711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff1815da50 0x7eff1815f1b0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.834+0000 7eff1cc9b640 1 --2- 172.21.15.83:0/2137742711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff180afe20 0x7eff1815ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.834+0000 7eff1cc9b640 1 --2- 172.21.15.83:0/2137742711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff1806b2f0 0x7eff1814f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.834+0000 7eff1cc9b640 1 -- 172.21.15.83:0/2137742711 >> 172.21.15.83:0/2137742711 conn(0x7eff180b24f0 msgr2=0x7eff180b3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:18.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.834+0000 7eff1cc9b640 1 -- 172.21.15.83:0/2137742711 shutdown_connections 2024-04-17T10:39:18.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.834+0000 7eff1cc9b640 1 -- 172.21.15.83:0/2137742711 wait complete. 2024-04-17T10:39:18.847 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T10:39:18.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.950+0000 7f9b415d8640 1 Processor -- start 2024-04-17T10:39:18.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.954+0000 7f9b415d8640 1 -- start start 2024-04-17T10:39:18.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.954+0000 7f9b415d8640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9b3c0cfbe0 0x7f9b3c06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:18.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.954+0000 7f9b415d8640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b3c06e320 con 0x7f9b3c15ba30 2024-04-17T10:39:18.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.954+0000 7f9b415d8640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b3c06e040 con 0x7f9b3c072330 2024-04-17T10:39:18.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.954+0000 7f9b415d8640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b3c071ea0 con 0x7f9b3c0cfbe0 2024-04-17T10:39:18.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.954+0000 7f9b3b7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9b3c0cfbe0 0x7f9b3c06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:18.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.954+0000 7f9b3affd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9b3c15ba30 0x7f9b3c15c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54092/0 (socket says 172.21.15.83:54092) 2024-04-17T10:39:18.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.954+0000 7f9b3affd640 1 -- 172.21.15.83:0/3494824477 learned_addr learned my addr 172.21.15.83:0/3494824477 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:39:18.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.954+0000 7f9b3b7fe640 1 -- 172.21.15.83:0/3494824477 >> v1:172.21.15.167:6789/0 conn(0x7f9b3c072330 legacy=0x7f9b3c0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:18.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.954+0000 7f9b3b7fe640 1 -- 172.21.15.83:0/3494824477 >> v1:172.21.15.83:6789/0 conn(0x7f9b3c15ba30 legacy=0x7f9b3c15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:18.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.954+0000 7f9b3b7fe640 1 -- 172.21.15.83:0/3494824477 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3c071850 con 0x7f9b3c0cfbe0 2024-04-17T10:39:18.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.954+0000 7f9b39ffb640 1 -- 172.21.15.83:0/3494824477 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2882494213 0 0) 0x7f9b3c06e320 con 0x7f9b3c15ba30 2024-04-17T10:39:18.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.954+0000 7f9b3b7fe640 1 --2- 172.21.15.83:0/3494824477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b3c0cfbe0 0x7f9b3c06c490 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9b34009d90 tx=0x7f9b3401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dddf6d25778c94a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:18.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.954+0000 7f9b39ffb640 1 -- 172.21.15.83:0/3494824477 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b340b2070 con 0x7f9b3c0cfbe0 2024-04-17T10:39:18.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.954+0000 7f9b39ffb640 1 -- 172.21.15.83:0/3494824477 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b340a5870 con 0x7f9b3c0cfbe0 2024-04-17T10:39:18.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.954+0000 7f9b39ffb640 1 -- 172.21.15.83:0/3494824477 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b340a5c40 con 0x7f9b3c0cfbe0 2024-04-17T10:39:18.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b415d8640 1 -- 172.21.15.83:0/3494824477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b3c0cfbe0 msgr2=0x7f9b3c06c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:18.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b415d8640 1 --2- 172.21.15.83:0/3494824477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b3c0cfbe0 0x7f9b3c06c490 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9b34009d90 tx=0x7f9b3401c8a0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b415d8640 1 -- 172.21.15.83:0/3494824477 shutdown_connections 2024-04-17T10:39:18.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b415d8640 1 --2- 172.21.15.83:0/3494824477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b3c0cfbe0 0x7f9b3c06c490 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b415d8640 1 -- 172.21.15.83:0/3494824477 >> 172.21.15.83:0/3494824477 conn(0x7f9b3c06abe0 msgr2=0x7f9b3c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:18.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b415d8640 1 -- 172.21.15.83:0/3494824477 shutdown_connections 2024-04-17T10:39:18.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b415d8640 1 -- 172.21.15.83:0/3494824477 wait complete. 2024-04-17T10:39:18.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b415d8640 1 Processor -- start 2024-04-17T10:39:18.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b415d8640 1 -- start start 2024-04-17T10:39:18.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b415d8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b3c072330 0x7f9b3c146950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:18.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b415d8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b3c0cfbe0 0x7f9b3c146e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:18.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b415d8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b3c15ba30 0x7f9b3c14f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:18.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b415d8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c1696b0 con 0x7f9b3c072330 2024-04-17T10:39:18.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b415d8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c169530 con 0x7f9b3c15ba30 2024-04-17T10:39:18.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b415d8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c169830 con 0x7f9b3c0cfbe0 2024-04-17T10:39:18.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b3affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b3c072330 0x7f9b3c146950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:18.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b3affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b3c072330 0x7f9b3c146950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60032/0 (socket says 172.21.15.83:60032) 2024-04-17T10:39:18.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b3affd640 1 -- 172.21.15.83:0/1341304675 learned_addr learned my addr 172.21.15.83:0/1341304675 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:18.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b3b7fe640 1 --2- 172.21.15.83:0/1341304675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b3c15ba30 0x7f9b3c14f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:18.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b3a7fc640 1 --2- 172.21.15.83:0/1341304675 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b3c0cfbe0 0x7f9b3c146e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:18.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b3affd640 1 -- 172.21.15.83:0/1341304675 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b3c0cfbe0 msgr2=0x7f9b3c146e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:18.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b3affd640 1 --2- 172.21.15.83:0/1341304675 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b3c0cfbe0 0x7f9b3c146e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b3affd640 1 -- 172.21.15.83:0/1341304675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b3c15ba30 msgr2=0x7f9b3c14f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:18.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b3affd640 1 --2- 172.21.15.83:0/1341304675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b3c15ba30 0x7f9b3c14f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:18.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b3affd640 1 -- 172.21.15.83:0/1341304675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3c15cfc0 con 0x7f9b3c072330 2024-04-17T10:39:18.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b3b7fe640 1 --2- 172.21.15.83:0/1341304675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b3c15ba30 0x7f9b3c14f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:18.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b3a7fc640 1 --2- 172.21.15.83:0/1341304675 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b3c0cfbe0 0x7f9b3c146e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:39:18.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b3affd640 1 --2- 172.21.15.83:0/1341304675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b3c072330 0x7f9b3c146950 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c00c550 tx=0x7f9b2c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:18.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b237fe640 1 -- 172.21.15.83:0/1341304675 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c017070 con 0x7f9b3c072330 2024-04-17T10:39:18.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b237fe640 1 -- 172.21.15.83:0/1341304675 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b2c005d10 con 0x7f9b3c072330 2024-04-17T10:39:18.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b237fe640 1 -- 172.21.15.83:0/1341304675 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c007330 con 0x7f9b3c072330 2024-04-17T10:39:18.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b415d8640 1 -- 172.21.15.83:0/1341304675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b3c15d2a0 con 0x7f9b3c072330 2024-04-17T10:39:18.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.958+0000 7f9b415d8640 1 -- 172.21.15.83:0/1341304675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b3c15d860 con 0x7f9b3c072330 2024-04-17T10:39:18.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.962+0000 7f9b415d8640 1 -- 172.21.15.83:0/1341304675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b3c074f90 con 0x7f9b3c072330 2024-04-17T10:39:18.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.962+0000 7f9b237fe640 1 -- 172.21.15.83:0/1341304675 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9b2c01a070 con 0x7f9b3c072330 2024-04-17T10:39:18.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.962+0000 7f9b237fe640 1 --2- 172.21.15.83:0/1341304675 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9b28066f60 0x7f9b28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:18.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.962+0000 7f9b3a7fc640 1 --2- 172.21.15.83:0/1341304675 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9b28066f60 0x7f9b28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:18.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.962+0000 7f9b237fe640 1 -- 172.21.15.83:0/1341304675 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b2c084440 con 0x7f9b3c072330 2024-04-17T10:39:18.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.962+0000 7f9b3a7fc640 1 --2- 172.21.15.83:0/1341304675 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9b28066f60 0x7f9b28069420 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c1552d0 tx=0x7f9b30008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:18.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:18.966+0000 7f9b237fe640 1 -- 172.21.15.83:0/1341304675 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9b2c0554e0 con 0x7f9b3c072330 2024-04-17T10:39:19.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:19.158+0000 7f9b415d8640 1 -- 172.21.15.83:0/1341304675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9b3c15bf50 con 0x7f9b3c072330 2024-04-17T10:39:19.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:19.158+0000 7f9b237fe640 1 -- 172.21.15.83:0/1341304675 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9b2c054de0 con 0x7f9b3c072330 2024-04-17T10:39:19.159 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T10:39:19.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:19.162+0000 7f9b415d8640 1 -- 172.21.15.83:0/1341304675 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9b28066f60 msgr2=0x7f9b28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:19.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:19.162+0000 7f9b415d8640 1 --2- 172.21.15.83:0/1341304675 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9b28066f60 0x7f9b28069420 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c1552d0 tx=0x7f9b30008040 comp rx=0 tx=0).stop 2024-04-17T10:39:19.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:19.162+0000 7f9b415d8640 1 -- 172.21.15.83:0/1341304675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b3c072330 msgr2=0x7f9b3c146950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:19.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:19.162+0000 7f9b415d8640 1 --2- 172.21.15.83:0/1341304675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b3c072330 0x7f9b3c146950 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c00c550 tx=0x7f9b2c00ca10 comp rx=0 tx=0).stop 2024-04-17T10:39:19.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:19.162+0000 7f9b415d8640 1 -- 172.21.15.83:0/1341304675 shutdown_connections 2024-04-17T10:39:19.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:19.162+0000 7f9b415d8640 1 --2- 172.21.15.83:0/1341304675 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9b28066f60 0x7f9b28069420 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:19.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:19.162+0000 7f9b415d8640 1 --2- 172.21.15.83:0/1341304675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b3c15ba30 0x7f9b3c14f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:19.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:19.162+0000 7f9b415d8640 1 --2- 172.21.15.83:0/1341304675 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b3c0cfbe0 0x7f9b3c146e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:19.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:19.162+0000 7f9b415d8640 1 --2- 172.21.15.83:0/1341304675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b3c072330 0x7f9b3c146950 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:19.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:19.162+0000 7f9b415d8640 1 -- 172.21.15.83:0/1341304675 >> 172.21.15.83:0/1341304675 conn(0x7f9b3c06abe0 msgr2=0x7f9b3c0af300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:19.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:19.162+0000 7f9b415d8640 1 -- 172.21.15.83:0/1341304675 shutdown_connections 2024-04-17T10:39:19.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:19.162+0000 7f9b415d8640 1 -- 172.21.15.83:0/1341304675 wait complete. 2024-04-17T10:39:19.177 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T10:39:19.177 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T10:39:19.177 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-04-17T10:39:19.177 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T10:39:19.177 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T10:39:19.178 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T10:39:19.178 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T10:39:19.178 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T10:39:19.178 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:39:19.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:39:19.178 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-04-17T10:39:19.183 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T10:39:19.184 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T10:39:19.184 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T10:39:19.184 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:39:19.184 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-17T10:39:19.184 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:39:19.184 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:39:19.184 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-04-17T10:39:19.184 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-04-17T10:39:19.184 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:39:19.184 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-04-17T10:39:19.184 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-04-17T10:39:19.184 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:39:19.185 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:39:19.185 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T10:39:19.185 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:39:19.185 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-17T10:39:19.185 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42425sec preferred_lft 42425sec 2024-04-17T10:39:19.185 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-04-17T10:39:19.185 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:39:19.185 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T10:39:19.185 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:37:0e:23:b2 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:39:19.185 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T10:39:19.185 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:39:19.185 INFO:teuthology.orchestra.run.smithi083.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T10:39:19.185 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-17T10:39:19.185 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T10:39:19.185 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:39:19.186 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-17T10:39:19.186 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:39:19.186 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:39:19.186 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:39:19.186 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-17T10:39:19.186 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:39:19.186 INFO:teuthology.orchestra.run.smithi083.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:39:19.186 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:39:19.186 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-17T10:39:19.186 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:39:19.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:39:19.187 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T10:39:19.237 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-04-17T10:39:19.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:39:19.238 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-04-17T10:39:19.291 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-04-17T10:39:19.291 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T10:39:19.291 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-17T10:39:19.291 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-17T10:39:19.291 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-17T10:39:19.292 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T10:39:19.292 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-17T10:39:19.292 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:39:19.292 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:39:19.292 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:39:19.292 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T10:39:19.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:39:19.292 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2024-04-17T10:39:19.292 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:39:19.293 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-17T10:39:19.293 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-17T10:39:19.293 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-17T10:39:19.293 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:39:19.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:39:19.361 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-17T10:39:19.412 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-17T10:39:19.412 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T10:39:19.412 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:39:19.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:39:19.412 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-04-17T10:39:19.468 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-17T10:39:19.468 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:39:19.468 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T10:39:19.515 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:39:19.515 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:39:19.515 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:39:19.563 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:39:19.620 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:39:19.668 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:39:19.668 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:39:19.668 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:39:19.668 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:39:19.668 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:39:19.669 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:39:19.669 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:39:19.669 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:39:19.669 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:39:19.669 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:39:19.669 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:39:19.669 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:39:19.669 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:39:19.669 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:39:19.669 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:39:19.669 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:39:19.669 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:39:19.669 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:39:19.669 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:39:19.670 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:39:19.670 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:39:19.670 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:39:19.670 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:39:19.670 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:39:19.670 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:39:19.670 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:39:19.670 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:39:19.670 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:39:19.670 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:39:19.670 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:39:19.670 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:39:19.670 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:39:19.671 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:39:19.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:39:19.671 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:39:19.719 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T10:39:19.719 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-17T10:39:19.764 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:39:19.778 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:39:19.823 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:39:19.823 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:39:19.823 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:39:19.823 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:39:19.823 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:39:19.824 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:39:19.824 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:39:19.824 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:39:19.824 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:39:19.824 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:39:19.824 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:39:19.824 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:39:19.824 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:39:19.824 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:39:19.824 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:39:19.824 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:39:19.824 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:39:19.824 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:39:19.824 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:39:19.825 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:39:19.825 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:39:19.825 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:39:19.825 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:39:19.825 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:39:19.825 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:39:19.825 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:39:19.825 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:39:19.825 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:39:19.825 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:39:19.825 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:39:19.825 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:39:19.825 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:39:19.825 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:39:19.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:39:19.826 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:39:19.839 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:2024-04-17T10:39:19.842+0000 7f01f515f580 -1 init, newargv = 0x5587b346f2a0 newargc=15 2024-04-17T10:39:19.839 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[37888]: starting ceph client 2024-04-17T10:39:19.844 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[37888]: starting fuse 2024-04-17T10:39:20.945 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:39:20.961 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:39:21.010 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:39:21.010 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:39:21.010 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:39:21.010 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:39:21.010 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:39:21.010 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:39:21.010 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:39:21.011 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:39:21.011 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:39:21.011 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:39:21.011 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:39:21.011 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:39:21.011 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:39:21.011 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:39:21.011 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:39:21.011 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:39:21.011 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:39:21.011 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:39:21.011 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:39:21.011 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:39:21.012 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:39:21.012 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:39:21.012 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:39:21.012 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:39:21.012 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:39:21.012 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:39:21.012 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:39:21.012 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:39:21.012 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:39:21.012 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:39:21.012 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:39:21.012 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:39:21.012 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:39:21.013 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-17T10:39:21.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:39:21.013 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:39:21.059 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-04-17T10:39:21.060 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-17T10:39:21.060 DEBUG:teuthology.orchestra.run.smithi083:> sudo stdin-killer -- python3 -c ' 2024-04-17T10:39:21.060 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2024-04-17T10:39:21.060 DEBUG:teuthology.orchestra.run.smithi083:> import re 2024-04-17T10:39:21.060 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-04-17T10:39:21.060 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2024-04-17T10:39:21.060 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:39:21.060 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2024-04-17T10:39:21.060 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-17T10:39:21.060 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2024-04-17T10:39:21.060 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-17T10:39:21.061 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:39:21.061 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2024-04-17T10:39:21.061 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2024-04-17T10:39:21.061 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2024-04-17T10:39:21.061 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2024-04-17T10:39:21.061 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:39:21.061 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2024-04-17T10:39:21.061 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T10:39:21.061 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T10:39:21.061 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T10:39:21.061 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2024-04-17T10:39:21.061 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2024-04-17T10:39:21.061 DEBUG:teuthology.orchestra.run.smithi083:> return f 2024-04-17T10:39:21.061 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T10:39:21.061 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:39:21.061 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.0")) 2024-04-17T10:39:21.062 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-04-17T10:39:21.182 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.0.37888.asok 2024-04-17T10:39:21.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:39:21.193 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.37888.asok 2024-04-17T10:39:21.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:39:21.193 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.37888.asok status 2024-04-17T10:39:21.348 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:39:21.348 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2024-04-17T10:39:21.349 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-17T10:39:21.349 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-17T10:39:21.349 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "0", 2024-04-17T10:39:21.349 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-04-17T10:39:21.349 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-17T10:39:21.349 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "37888", 2024-04-17T10:39:21.349 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-04-17T10:39:21.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:39:21.349 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2024-04-17T10:39:21.349 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2024-04-17T10:39:21.349 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 5376, 2024-04-17T10:39:21.349 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2024-04-17T10:39:21.349 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-04-17T10:39:21.350 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-04-17T10:39:21.350 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 5376 2024-04-17T10:39:21.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:39:21.350 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-04-17T10:39:21.350 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-17T10:39:21.350 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:39:21.350 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2069701714 2024-04-17T10:39:21.350 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:39:21.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:39:21.350 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-04-17T10:39:21.350 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-17T10:39:21.350 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:39:21.350 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2069701714 2024-04-17T10:39:21.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:39:21.351 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.5376 192.168.0.1:0/2069701714", 2024-04-17T10:39:21.351 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.1:0/2069701714", 2024-04-17T10:39:21.351 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 1, 2024-04-17T10:39:21.351 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 13, 2024-04-17T10:39:21.351 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 33, 2024-04-17T10:39:21.351 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2024-04-17T10:39:21.351 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2024-04-17T10:39:21.351 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2024-04-17T10:39:21.352 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:39:21.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:39:21.363 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2024-04-17T10:39:21.363 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-17T10:39:21.370 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2024-04-17T10:39:21.370 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-17T10:39:21.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:39:21.370 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:39:21.424 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-04-17T10:39:21.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.598+0000 7f4a525d1640 1 Processor -- start 2024-04-17T10:39:21.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.598+0000 7f4a525d1640 1 -- start start 2024-04-17T10:39:21.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.598+0000 7f4a525d1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a4c06be50 0x7f4a4c072780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:21.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.598+0000 7f4a525d1640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4a4c0af440 0x7f4a4c06cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:21.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.598+0000 7f4a525d1640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a4c06f400 con 0x7f4a4c072cc0 2024-04-17T10:39:21.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.598+0000 7f4a525d1640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a4c071ca0 con 0x7f4a4c0af440 2024-04-17T10:39:21.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.598+0000 7f4a525d1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c071f80 con 0x7f4a4c06be50 2024-04-17T10:39:21.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.598+0000 7f4a4bfff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a4c06be50 0x7f4a4c072780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:21.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.598+0000 7f4a50b6e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4a4c0af440 0x7f4a4c06cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:21.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.598+0000 7f4a50b6e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4a4c0af440 0x7f4a4c06cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:50802/0 (socket says 172.21.15.83:50802) 2024-04-17T10:39:21.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.598+0000 7f4a50b6e640 1 -- 172.21.15.83:0/1905982798 learned_addr learned my addr 172.21.15.83:0/1905982798 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:21.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.598+0000 7f4a50b6e640 1 -- 172.21.15.83:0/1905982798 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a4c06be50 msgr2=0x7f4a4c072780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:39:21.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.598+0000 7f4a50b6e640 1 --2- 172.21.15.83:0/1905982798 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a4c06be50 0x7f4a4c072780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:21.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.598+0000 7f4a50b6e640 1 -- 172.21.15.83:0/1905982798 >> v1:172.21.15.167:6789/0 conn(0x7f4a4c072cc0 legacy=0x7f4a4c0aed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:21.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.598+0000 7f4a50b6e640 1 -- 172.21.15.83:0/1905982798 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4c072260 con 0x7f4a4c0af440 2024-04-17T10:39:21.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.602+0000 7f4a50b6e640 1 --2- 172.21.15.83:0/1905982798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a4c0af440 0x7f4a4c06cb60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4a44009e50 tx=0x7f4a4401ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dff9e0579c0b57ac server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:21.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.602+0000 7f4a4affd640 1 -- 172.21.15.83:0/1905982798 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a440a5920 con 0x7f4a4c0af440 2024-04-17T10:39:21.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.602+0000 7f4a4affd640 1 -- 172.21.15.83:0/1905982798 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a440a5ac0 con 0x7f4a4c0af440 2024-04-17T10:39:21.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.602+0000 7f4a4affd640 1 -- 172.21.15.83:0/1905982798 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a440ad3f0 con 0x7f4a4c0af440 2024-04-17T10:39:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.602+0000 7f4a525d1640 1 -- 172.21.15.83:0/1905982798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a4c0af440 msgr2=0x7f4a4c06cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.602+0000 7f4a525d1640 1 --2- 172.21.15.83:0/1905982798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a4c0af440 0x7f4a4c06cb60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4a44009e50 tx=0x7f4a4401ca30 comp rx=0 tx=0).stop 2024-04-17T10:39:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.602+0000 7f4a525d1640 1 -- 172.21.15.83:0/1905982798 shutdown_connections 2024-04-17T10:39:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.602+0000 7f4a525d1640 1 --2- 172.21.15.83:0/1905982798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a4c0af440 0x7f4a4c06cb60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.602+0000 7f4a525d1640 1 --2- 172.21.15.83:0/1905982798 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a4c06be50 0x7f4a4c072780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.602+0000 7f4a525d1640 1 -- 172.21.15.83:0/1905982798 >> 172.21.15.83:0/1905982798 conn(0x7f4a4c0cc910 msgr2=0x7f4a4c0ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.602+0000 7f4a525d1640 1 -- 172.21.15.83:0/1905982798 shutdown_connections 2024-04-17T10:39:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.602+0000 7f4a525d1640 1 -- 172.21.15.83:0/1905982798 wait complete. 2024-04-17T10:39:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.606+0000 7f4a525d1640 1 Processor -- start 2024-04-17T10:39:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.606+0000 7f4a525d1640 1 -- start start 2024-04-17T10:39:21.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.606+0000 7f4a525d1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a4c06be50 0x7f4a4c09b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:21.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.606+0000 7f4a525d1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a4c072cc0 0x7f4a4c15bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:21.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.606+0000 7f4a525d1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a4c0af440 0x7f4a4c09f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:21.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.606+0000 7f4a525d1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c06d0a0 con 0x7f4a4c06be50 2024-04-17T10:39:21.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.606+0000 7f4a525d1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c06c440 con 0x7f4a4c0af440 2024-04-17T10:39:21.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.606+0000 7f4a525d1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c06d220 con 0x7f4a4c072cc0 2024-04-17T10:39:21.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.606+0000 7f4a4bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a4c06be50 0x7f4a4c09b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:21.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.606+0000 7f4a4bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a4c06be50 0x7f4a4c09b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60042/0 (socket says 172.21.15.83:60042) 2024-04-17T10:39:21.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.606+0000 7f4a4bfff640 1 -- 172.21.15.83:0/4014339262 learned_addr learned my addr 172.21.15.83:0/4014339262 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:39:21.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.606+0000 7f4a50b6e640 1 --2- 172.21.15.83:0/4014339262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a4c0af440 0x7f4a4c09f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:21.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.606+0000 7f4a4b7fe640 1 --2- 172.21.15.83:0/4014339262 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a4c072cc0 0x7f4a4c15bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:21.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.606+0000 7f4a4bfff640 1 -- 172.21.15.83:0/4014339262 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a4c072cc0 msgr2=0x7f4a4c15bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:21.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.606+0000 7f4a4bfff640 1 --2- 172.21.15.83:0/4014339262 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a4c072cc0 0x7f4a4c15bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:21.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.606+0000 7f4a4bfff640 1 -- 172.21.15.83:0/4014339262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a4c0af440 msgr2=0x7f4a4c09f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:21.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.606+0000 7f4a4bfff640 1 --2- 172.21.15.83:0/4014339262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a4c0af440 0x7f4a4c09f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:21.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.606+0000 7f4a4bfff640 1 -- 172.21.15.83:0/4014339262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4c09d4e0 con 0x7f4a4c06be50 2024-04-17T10:39:21.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.606+0000 7f4a50b6e640 1 --2- 172.21.15.83:0/4014339262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a4c0af440 0x7f4a4c09f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:39:21.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.606+0000 7f4a4bfff640 1 --2- 172.21.15.83:0/4014339262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a4c06be50 0x7f4a4c09b9d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00e930 tx=0x7f4a3c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:21.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.606+0000 7f4a48ff9640 1 -- 172.21.15.83:0/4014339262 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c017070 con 0x7f4a4c06be50 2024-04-17T10:39:21.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.606+0000 7f4a48ff9640 1 -- 172.21.15.83:0/4014339262 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3c00ca00 con 0x7f4a4c06be50 2024-04-17T10:39:21.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.606+0000 7f4a48ff9640 1 -- 172.21.15.83:0/4014339262 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c00cdb0 con 0x7f4a4c06be50 2024-04-17T10:39:21.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.606+0000 7f4a525d1640 1 -- 172.21.15.83:0/4014339262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a4c09d7c0 con 0x7f4a4c06be50 2024-04-17T10:39:21.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.606+0000 7f4a525d1640 1 -- 172.21.15.83:0/4014339262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a4c0c7240 con 0x7f4a4c06be50 2024-04-17T10:39:21.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.606+0000 7f4a525d1640 1 -- 172.21.15.83:0/4014339262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a4c075000 con 0x7f4a4c06be50 2024-04-17T10:39:21.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.606+0000 7f4a48ff9640 1 -- 172.21.15.83:0/4014339262 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4a3c01a070 con 0x7f4a4c06be50 2024-04-17T10:39:21.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.610+0000 7f4a48ff9640 1 --2- 172.21.15.83:0/4014339262 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4a38066f60 0x7f4a38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:39:21.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.610+0000 7f4a48ff9640 1 -- 172.21.15.83:0/4014339262 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a3c085740 con 0x7f4a4c06be50 2024-04-17T10:39:21.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.610+0000 7f4a4b7fe640 1 --2- 172.21.15.83:0/4014339262 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4a38066f60 0x7f4a38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:39:21.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.610+0000 7f4a4b7fe640 1 --2- 172.21.15.83:0/4014339262 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4a38066f60 0x7f4a38069420 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c1fa590 tx=0x7f4a40009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:39:21.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.614+0000 7f4a48ff9640 1 -- 172.21.15.83:0/4014339262 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4a3c0568f0 con 0x7f4a4c06be50 2024-04-17T10:39:21.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:21.802+0000 7f4a525d1640 1 -- 172.21.15.83:0/4014339262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4a4c161330 con 0x7f4a4c06be50 2024-04-17T10:39:22.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:22.258+0000 7f4a48ff9640 1 -- 172.21.15.83:0/4014339262 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v14) v1 ==== 167+0+0 (secure 0 0 0) 0x7f4a3c0561f0 con 0x7f4a4c06be50 2024-04-17T10:39:22.258 INFO:teuthology.orchestra.run.smithi083.stderr:enabled new snapshots 2024-04-17T10:39:22.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:22.262+0000 7f4a525d1640 1 -- 172.21.15.83:0/4014339262 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4a38066f60 msgr2=0x7f4a38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:22.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:22.262+0000 7f4a525d1640 1 --2- 172.21.15.83:0/4014339262 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4a38066f60 0x7f4a38069420 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c1fa590 tx=0x7f4a40009040 comp rx=0 tx=0).stop 2024-04-17T10:39:22.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:22.262+0000 7f4a525d1640 1 -- 172.21.15.83:0/4014339262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a4c06be50 msgr2=0x7f4a4c09b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:39:22.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:22.262+0000 7f4a525d1640 1 --2- 172.21.15.83:0/4014339262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a4c06be50 0x7f4a4c09b9d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00e930 tx=0x7f4a3c00edf0 comp rx=0 tx=0).stop 2024-04-17T10:39:22.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:22.262+0000 7f4a525d1640 1 -- 172.21.15.83:0/4014339262 shutdown_connections 2024-04-17T10:39:22.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:22.262+0000 7f4a525d1640 1 --2- 172.21.15.83:0/4014339262 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4a38066f60 0x7f4a38069420 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:22.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:22.262+0000 7f4a525d1640 1 --2- 172.21.15.83:0/4014339262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a4c0af440 0x7f4a4c09f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:22.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:22.262+0000 7f4a525d1640 1 --2- 172.21.15.83:0/4014339262 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a4c072cc0 0x7f4a4c15bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:22.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:22.262+0000 7f4a525d1640 1 --2- 172.21.15.83:0/4014339262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a4c06be50 0x7f4a4c09b9d0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:39:22.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:22.262+0000 7f4a525d1640 1 -- 172.21.15.83:0/4014339262 >> 172.21.15.83:0/4014339262 conn(0x7f4a4c0cc910 msgr2=0x7f4a4c06e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:39:22.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:22.262+0000 7f4a525d1640 1 -- 172.21.15.83:0/4014339262 shutdown_connections 2024-04-17T10:39:22.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:22.262+0000 7f4a525d1640 1 -- 172.21.15.83:0/4014339262 wait complete. 2024-04-17T10:39:22.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:39:22.280 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stray_eval_dir) 2024-04-17T10:39:22.290 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:39:22.290 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:39:22.336 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-04-17T10:39:22.336 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-04-17T10:39:22.336 DEBUG:teuthology.orchestra.run.smithi083:> import uuid 2024-04-17T10:39:22.336 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:39:22.336 DEBUG:teuthology.orchestra.run.smithi083:> n = 2000 2024-04-17T10:39:22.336 DEBUG:teuthology.orchestra.run.smithi083:> create_hard_links = False 2024-04-17T10:39:22.336 DEBUG:teuthology.orchestra.run.smithi083:> if 3 > 0: 2024-04-17T10:39:22.336 DEBUG:teuthology.orchestra.run.smithi083:> create_hard_links = True 2024-04-17T10:39:22.336 DEBUG:teuthology.orchestra.run.smithi083:> path = "/home/ubuntu/cephtest/mnt.0/stray_eval_dir/file" 2024-04-17T10:39:22.336 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:39:22.337 DEBUG:teuthology.orchestra.run.smithi083:> dpath = os.path.dirname(path) 2024-04-17T10:39:22.337 DEBUG:teuthology.orchestra.run.smithi083:> fnameprefix = os.path.basename(path) 2024-04-17T10:39:22.337 DEBUG:teuthology.orchestra.run.smithi083:> os.makedirs(dpath, exist_ok=True) 2024-04-17T10:39:22.337 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:39:22.337 DEBUG:teuthology.orchestra.run.smithi083:> try: 2024-04-17T10:39:22.337 DEBUG:teuthology.orchestra.run.smithi083:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-17T10:39:22.337 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:39:22.337 DEBUG:teuthology.orchestra.run.smithi083:> for i in range(n): 2024-04-17T10:39:22.337 DEBUG:teuthology.orchestra.run.smithi083:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-17T10:39:22.337 DEBUG:teuthology.orchestra.run.smithi083:> with open(fpath, '"'"'w'"'"') as f: 2024-04-17T10:39:22.337 DEBUG:teuthology.orchestra.run.smithi083:> f.write(f"{i}") 2024-04-17T10:39:22.337 DEBUG:teuthology.orchestra.run.smithi083:> if False: 2024-04-17T10:39:22.337 DEBUG:teuthology.orchestra.run.smithi083:> f.flush() 2024-04-17T10:39:22.337 DEBUG:teuthology.orchestra.run.smithi083:> os.fsync(f.fileno()) 2024-04-17T10:39:22.337 DEBUG:teuthology.orchestra.run.smithi083:> if False: 2024-04-17T10:39:22.338 DEBUG:teuthology.orchestra.run.smithi083:> os.unlink(fpath) 2024-04-17T10:39:22.338 DEBUG:teuthology.orchestra.run.smithi083:> if False: 2024-04-17T10:39:22.338 DEBUG:teuthology.orchestra.run.smithi083:> os.fsync(dirfd) 2024-04-17T10:39:22.338 DEBUG:teuthology.orchestra.run.smithi083:> if create_hard_links: 2024-04-17T10:39:22.338 DEBUG:teuthology.orchestra.run.smithi083:> for j in range(3): 2024-04-17T10:39:22.338 DEBUG:teuthology.orchestra.run.smithi083:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-17T10:39:22.338 DEBUG:teuthology.orchestra.run.smithi083:> if False: 2024-04-17T10:39:22.338 DEBUG:teuthology.orchestra.run.smithi083:> os.fsync(dirfd) 2024-04-17T10:39:22.338 DEBUG:teuthology.orchestra.run.smithi083:> finally: 2024-04-17T10:39:22.338 DEBUG:teuthology.orchestra.run.smithi083:> os.close(dirfd) 2024-04-17T10:39:22.338 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-04-17T10:39:36.107 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:39:36.110 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:39:45.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:39:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:39:45.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:39:45.830 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-17T10:39:45.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:39:45.870 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec find /home/ubuntu/cephtest/mnt.0/stray_eval_dir/ -type f -delete) 2024-04-17T10:40:06.230 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:40:06.233 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:40:36.360 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:40:36.364 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:40:49.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:40:49.371 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-17T10:40:49.396 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-17T10:40:49.396 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2024-04-17T10:40:49.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.572+0000 7f60b0cf7640 1 Processor -- start 2024-04-17T10:40:49.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.572+0000 7f60b0cf7640 1 -- start start 2024-04-17T10:40:49.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.572+0000 7f60b0cf7640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f60ac158030 0x7f60ac0b3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.572+0000 7f60b0cf7640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f60ac0adf50 0x7f60ac06ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.572+0000 7f60b0cf7640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60ac0b3e70 con 0x7f60ac0b27c0 2024-04-17T10:40:49.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.572+0000 7f60b0cf7640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60ac15e830 con 0x7f60ac0adf50 2024-04-17T10:40:49.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.572+0000 7f60b0cf7640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60ac15e1e0 con 0x7f60ac158030 2024-04-17T10:40:49.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.572+0000 7f60aad9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f60ac0adf50 0x7f60ac06ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.572+0000 7f60a9d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f60ac0b27c0 0x7f60ac0ad840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33738/0 (socket says 172.21.15.83:33738) 2024-04-17T10:40:49.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.572+0000 7f60a9d9b640 1 -- 172.21.15.83:0/48899745 learned_addr learned my addr 172.21.15.83:0/48899745 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:40:49.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.572+0000 7f60aad9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f60ac0adf50 0x7f60ac06ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35696/0 (socket says 172.21.15.83:35696) 2024-04-17T10:40:49.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.572+0000 7f60aa59c640 1 --2- 172.21.15.83:0/48899745 >> v2:172.21.15.167:3300/0 conn(0x7f60ac158030 0x7f60ac0b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.572+0000 7f60a959a640 1 -- 172.21.15.83:0/48899745 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3744548825 0 0) 0x7f60ac0b3e70 con 0x7f60ac0b27c0 2024-04-17T10:40:49.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.572+0000 7f60a959a640 1 -- 172.21.15.83:0/48899745 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6094003530 con 0x7f60ac0b27c0 2024-04-17T10:40:49.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.572+0000 7f60aad9d640 1 -- 172.21.15.83:0/48899745 >> v2:172.21.15.167:3300/0 conn(0x7f60ac158030 msgr2=0x7f60ac0b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:40:49.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.572+0000 7f60aad9d640 1 --2- 172.21.15.83:0/48899745 >> v2:172.21.15.167:3300/0 conn(0x7f60ac158030 0x7f60ac0b3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.572+0000 7f60aad9d640 1 -- 172.21.15.83:0/48899745 >> v1:172.21.15.83:6789/0 conn(0x7f60ac0b27c0 legacy=0x7f60ac0ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:40:49.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.572+0000 7f60aad9d640 1 -- 172.21.15.83:0/48899745 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ac15e4c0 con 0x7f60ac0adf50 2024-04-17T10:40:49.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.576+0000 7f60aad9d640 1 --2- 172.21.15.83:0/48899745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60ac0adf50 0x7f60ac06ca90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f60ac15e9b0 tx=0x7f60a401c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e877b41a73cc80c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:40:49.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.576+0000 7f60a959a640 1 -- 172.21.15.83:0/48899745 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a40b2070 con 0x7f60ac0adf50 2024-04-17T10:40:49.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.576+0000 7f60a959a640 1 -- 172.21.15.83:0/48899745 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60a40a57c0 con 0x7f60ac0adf50 2024-04-17T10:40:49.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.576+0000 7f60a959a640 1 -- 172.21.15.83:0/48899745 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a40a5b90 con 0x7f60ac0adf50 2024-04-17T10:40:49.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.580+0000 7f60b0cf7640 1 -- 172.21.15.83:0/48899745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60ac0adf50 msgr2=0x7f60ac06ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:40:49.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.580+0000 7f60b0cf7640 1 --2- 172.21.15.83:0/48899745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60ac0adf50 0x7f60ac06ca90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f60ac15e9b0 tx=0x7f60a401c7f0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.580+0000 7f60b0cf7640 1 -- 172.21.15.83:0/48899745 shutdown_connections 2024-04-17T10:40:49.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.580+0000 7f60b0cf7640 1 --2- 172.21.15.83:0/48899745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60ac0adf50 0x7f60ac06ca90 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.580+0000 7f60b0cf7640 1 --2- 172.21.15.83:0/48899745 >> v2:172.21.15.167:3300/0 conn(0x7f60ac158030 0x7f60ac0b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.580+0000 7f60b0cf7640 1 -- 172.21.15.83:0/48899745 >> 172.21.15.83:0/48899745 conn(0x7f60ac0cf630 msgr2=0x7f60ac0cfa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:40:49.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.580+0000 7f60b0cf7640 1 -- 172.21.15.83:0/48899745 shutdown_connections 2024-04-17T10:40:49.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.580+0000 7f60b0cf7640 1 -- 172.21.15.83:0/48899745 wait complete. 2024-04-17T10:40:49.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.580+0000 7f60b0cf7640 1 Processor -- start 2024-04-17T10:40:49.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.580+0000 7f60b0cf7640 1 -- start start 2024-04-17T10:40:49.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.580+0000 7f60b0cf7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60ac0adf50 0x7f60ac0b0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.580+0000 7f60b0cf7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60ac0b27c0 0x7f60ac1598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.580+0000 7f60b0cf7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60ac158030 0x7f60ac141d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.580+0000 7f60b0cf7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac16a7a0 con 0x7f60ac158030 2024-04-17T10:40:49.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.580+0000 7f60b0cf7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac0b3c50 con 0x7f60ac0b27c0 2024-04-17T10:40:49.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.580+0000 7f60b0cf7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac16a920 con 0x7f60ac0adf50 2024-04-17T10:40:49.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.580+0000 7f60aad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60ac158030 0x7f60ac141d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.580+0000 7f60aad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60ac158030 0x7f60ac141d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35698/0 (socket says 172.21.15.83:35698) 2024-04-17T10:40:49.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.580+0000 7f60aad9d640 1 -- 172.21.15.83:0/1665442438 learned_addr learned my addr 172.21.15.83:0/1665442438 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:40:49.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.580+0000 7f60aa59c640 1 --2- 172.21.15.83:0/1665442438 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60ac0adf50 0x7f60ac0b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.580+0000 7f60aad9d640 1 -- 172.21.15.83:0/1665442438 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60ac0adf50 msgr2=0x7f60ac0b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:40:49.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.580+0000 7f60aad9d640 1 --2- 172.21.15.83:0/1665442438 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60ac0adf50 0x7f60ac0b0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.580+0000 7f60aad9d640 1 -- 172.21.15.83:0/1665442438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60ac0b27c0 msgr2=0x7f60ac1598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:40:49.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.580+0000 7f60aad9d640 1 --2- 172.21.15.83:0/1665442438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60ac0b27c0 0x7f60ac1598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.580+0000 7f60aad9d640 1 -- 172.21.15.83:0/1665442438 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ac158820 con 0x7f60ac158030 2024-04-17T10:40:49.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.584+0000 7f60aad9d640 1 --2- 172.21.15.83:0/1665442438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60ac158030 0x7f60ac141d80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f60a40b0040 tx=0x7f60a4019590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:40:49.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.584+0000 7f609affd640 1 -- 172.21.15.83:0/1665442438 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a40bd020 con 0x7f60ac158030 2024-04-17T10:40:49.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.584+0000 7f609affd640 1 -- 172.21.15.83:0/1665442438 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60a40b8470 con 0x7f60ac158030 2024-04-17T10:40:49.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.584+0000 7f609affd640 1 -- 172.21.15.83:0/1665442438 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a40b2040 con 0x7f60ac158030 2024-04-17T10:40:49.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.584+0000 7f60b0cf7640 1 -- 172.21.15.83:0/1665442438 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60ac158b00 con 0x7f60ac158030 2024-04-17T10:40:49.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.584+0000 7f60b0cf7640 1 -- 172.21.15.83:0/1665442438 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60ac1590c0 con 0x7f60ac158030 2024-04-17T10:40:49.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.588+0000 7f60b3e64000 1 Processor -- start 2024-04-17T10:40:49.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.588+0000 7f609affd640 1 -- 172.21.15.83:0/1665442438 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f60a40c0020 con 0x7f60ac158030 2024-04-17T10:40:49.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.588+0000 7f60b3e64000 1 -- start start 2024-04-17T10:40:49.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.588+0000 7f609affd640 1 --2- 172.21.15.83:0/1665442438 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f608c066f60 0x7f608c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.588+0000 7f609affd640 1 -- 172.21.15.83:0/1665442438 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f60a4120120 con 0x7f60ac158030 2024-04-17T10:40:49.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.588+0000 7f60a9d9b640 1 --2- 172.21.15.83:0/1665442438 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f608c066f60 0x7f608c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.588+0000 7f60b3e64000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c7d94e250 0x556c7d94e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.588+0000 7f60b3e64000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556c7d94eb70 0x556c7d909060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.588+0000 7f60b3e64000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c7d9095a0 0x556c7d909980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.588+0000 7f60b3e64000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c7d7e31f0 con 0x556c7d94e250 2024-04-17T10:40:49.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.588+0000 7f60b3e64000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c7d749e90 con 0x556c7d9095a0 2024-04-17T10:40:49.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.588+0000 7f60b3e64000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c7d6d1f50 con 0x556c7d94eb70 2024-04-17T10:40:49.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.588+0000 7f60aa59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c7d94e250 0x556c7d94e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.588+0000 7f60aa59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c7d94e250 0x556c7d94e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35710/0 (socket says 172.21.15.83:35710) 2024-04-17T10:40:49.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.588+0000 7f60aa59c640 1 -- 172.21.15.83:0/1047061162 learned_addr learned my addr 172.21.15.83:0/1047061162 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:40:49.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.588+0000 7f60a9d9b640 1 --2- 172.21.15.83:0/1665442438 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f608c066f60 0x7f608c069420 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f60a0007fd0 tx=0x7f60a0007900 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:40:49.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.588+0000 7f60aa59c640 1 -- 172.21.15.83:0/1047061162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556c7d94eb70 msgr2=0x556c7d909060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:40:49.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.588+0000 7f60aa59c640 1 --2- 172.21.15.83:0/1047061162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556c7d94eb70 0x556c7d909060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.588+0000 7f60aa59c640 1 -- 172.21.15.83:0/1047061162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c7d9095a0 msgr2=0x556c7d909980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:40:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.588+0000 7f60aa59c640 1 --2- 172.21.15.83:0/1047061162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c7d9095a0 0x556c7d909980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.588+0000 7f60aa59c640 1 -- 172.21.15.83:0/1047061162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c7d7218b0 con 0x556c7d94e250 2024-04-17T10:40:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.588+0000 7f60aa59c640 1 --2- 172.21.15.83:0/1047061162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c7d94e250 0x556c7d94e630 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f609c009a60 tx=0x7f609c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd9341e467a6f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:40:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.588+0000 7f608bfff640 1 -- 172.21.15.83:0/1047061162 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c018070 con 0x556c7d94e250 2024-04-17T10:40:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.588+0000 7f608bfff640 1 -- 172.21.15.83:0/1047061162 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f609c010d00 con 0x556c7d94e250 2024-04-17T10:40:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.588+0000 7f608bfff640 1 -- 172.21.15.83:0/1047061162 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c00f040 con 0x556c7d94e250 2024-04-17T10:40:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.592+0000 7f60b3e64000 1 -- 172.21.15.83:0/1047061162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c7d94e250 msgr2=0x556c7d94e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:40:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.592+0000 7f60b3e64000 1 --2- 172.21.15.83:0/1047061162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c7d94e250 0x556c7d94e630 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f609c009a60 tx=0x7f609c009190 comp rx=0 tx=0).stop 2024-04-17T10:40:49.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.592+0000 7f60b3e64000 1 -- 172.21.15.83:0/1047061162 shutdown_connections 2024-04-17T10:40:49.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.592+0000 7f60b3e64000 1 --2- 172.21.15.83:0/1047061162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c7d9095a0 0x556c7d909980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.592+0000 7f60b3e64000 1 --2- 172.21.15.83:0/1047061162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556c7d94eb70 0x556c7d909060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.592+0000 7f60b3e64000 1 --2- 172.21.15.83:0/1047061162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c7d94e250 0x556c7d94e630 secure :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f609c009a60 tx=0x7f609c009190 comp rx=0 tx=0).stop 2024-04-17T10:40:49.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.592+0000 7f60b3e64000 1 -- 172.21.15.83:0/1047061162 >> 172.21.15.83:0/1047061162 conn(0x556c7d905550 msgr2=0x556c7d907940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:40:49.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.592+0000 7f60b3e64000 1 -- 172.21.15.83:0/1047061162 shutdown_connections 2024-04-17T10:40:49.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.592+0000 7f60b3e64000 1 -- 172.21.15.83:0/1047061162 wait complete. 2024-04-17T10:40:49.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.592+0000 7f60b3e64000 1 -- start start 2024-04-17T10:40:49.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.592+0000 7f60b3e64000 1 Processor -- start 2024-04-17T10:40:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60b3e64000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c7d9095a0 0x556c7d93c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60b3e64000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c7d94eb70 0x556c7d93cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60b3e64000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556c7d93bf00 0x556c7d93d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60b3e64000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c7d704540 con 0x556c7d9095a0 2024-04-17T10:40:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60b3e64000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c7d745f30 con 0x556c7d94eb70 2024-04-17T10:40:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60b3e64000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c7d728010 con 0x556c7d93bf00 2024-04-17T10:40:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60b3e64000 10 client.0 _ll_register_callbacks cb 0x556c7d7fe180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:40:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60aa59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c7d9095a0 0x556c7d93c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60aa59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c7d9095a0 0x556c7d93c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35724/0 (socket says 172.21.15.83:35724) 2024-04-17T10:40:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60aa59c640 1 -- 172.21.15.83:0/1821166410 learned_addr learned my addr 172.21.15.83:0/1821166410 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:40:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60a9d9b640 1 --2- 172.21.15.83:0/1821166410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c7d94eb70 0x556c7d93cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60aa59c640 1 -- 172.21.15.83:0/1821166410 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556c7d93bf00 msgr2=0x556c7d93d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:40:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60aa59c640 1 --2- 172.21.15.83:0/1821166410 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556c7d93bf00 0x556c7d93d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60aa59c640 1 -- 172.21.15.83:0/1821166410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c7d94eb70 msgr2=0x556c7d93cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:40:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60aa59c640 1 --2- 172.21.15.83:0/1821166410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c7d94eb70 0x556c7d93cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60aa59c640 1 -- 172.21.15.83:0/1821166410 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c7d6d21a0 con 0x556c7d9095a0 2024-04-17T10:40:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60aa59c640 1 --2- 172.21.15.83:0/1821166410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c7d9095a0 0x556c7d93c5d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f609c017000 tx=0x7f609c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4119e33e10c5b34c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:40:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60897fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:40:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60897fa640 1 -- 172.21.15.83:0/1821166410 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c02b060 con 0x556c7d9095a0 2024-04-17T10:40:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60b3e64000 1 -- 172.21.15.83:0/1821166410 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f609c010d00 con 0x556c7d9095a0 2024-04-17T10:40:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60897fa640 1 -- 172.21.15.83:0/1821166410 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f609c010b00 con 0x556c7d9095a0 2024-04-17T10:40:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60897fa640 1 -- 172.21.15.83:0/1821166410 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f609c031050 con 0x556c7d9095a0 2024-04-17T10:40:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60b3e64000 10 client.5394 fetch_fsmap learned FSMap version 14 2024-04-17T10:40:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60b3e64000 1 -- 172.21.15.83:0/1821166410 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x556c7d7cc840 con 0x556c7d9095a0 2024-04-17T10:40:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60897fa640 1 -- 172.21.15.83:0/1821166410 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f609c010d00 con 0x556c7d9095a0 2024-04-17T10:40:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60897fa640 1 -- 172.21.15.83:0/1821166410 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 14) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f609c034020 con 0x556c7d9095a0 2024-04-17T10:40:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60b3e64000 10 client.5394 fetch_fsmap finished waiting for FSMap version 14 2024-04-17T10:40:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60b3e64000 10 client.5394 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-17T10:40:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60b3e64000 20 client.5394 populate_metadata read hostname 'smithi083' 2024-04-17T10:40:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60b3e64000 1 --2- 172.21.15.83:0/1821166410 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x556c7d95e500 0x556c7d9608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60b3e64000 4 client.5394 mds_command: new command op to 5172 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:40:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60b3e64000 1 -- 172.21.15.83:0/1821166410 --> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c7d6e5be0 con 0x556c7d95e500 2024-04-17T10:40:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60a9d9b640 1 --2- 172.21.15.83:0/1821166410 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x556c7d95e500 0x556c7d9608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60897fa640 10 client.5394 ms_handle_connect on v2:172.21.15.83:6836/1181902025 2024-04-17T10:40:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.596+0000 7f60a9d9b640 1 --2- 172.21.15.83:0/1821166410 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x556c7d95e500 0x556c7d9608f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac3153e20431a325 server_cookie=60f925d78663e282 in_seq=0 out_seq=0 2024-04-17T10:40:49.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.600+0000 7f60897fa640 1 -- 172.21.15.83:0/1821166410 <== mds.0 v2:172.21.15.83:6836/1181902025 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556c7d6e5be0 con 0x556c7d95e500 2024-04-17T10:40:49.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.600+0000 7f60897fa640 10 client.5394 handle_command_reply: tid=0 2024-04-17T10:40:49.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.600+0000 7f60b3e64000 1 client.5394 shutdown 2024-04-17T10:40:49.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.600+0000 7f60b3e64000 1 -- 172.21.15.83:0/1821166410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c7d9095a0 msgr2=0x556c7d93c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:40:49.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.600+0000 7f60b3e64000 1 --2- 172.21.15.83:0/1821166410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c7d9095a0 0x556c7d93c5d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f609c017000 tx=0x7f609c004740 comp rx=0 tx=0).stop 2024-04-17T10:40:49.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.600+0000 7f60b3e64000 1 -- 172.21.15.83:0/1821166410 shutdown_connections 2024-04-17T10:40:49.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.600+0000 7f60b3e64000 1 --2- 172.21.15.83:0/1821166410 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x556c7d95e500 0x556c7d9608f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.600+0000 7f60b3e64000 1 --2- 172.21.15.83:0/1821166410 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556c7d93bf00 0x556c7d93d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.600+0000 7f60897fa640 0 client.5394 ms_handle_reset on v2:172.21.15.83:6836/1181902025 2024-04-17T10:40:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.600+0000 7f60b3e64000 1 --2- 172.21.15.83:0/1821166410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c7d94eb70 0x556c7d93cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.600+0000 7f60b3e64000 1 --2- 172.21.15.83:0/1821166410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c7d9095a0 0x556c7d93c5d0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.600+0000 7f60b3e64000 1 -- 172.21.15.83:0/1821166410 >> 172.21.15.83:0/1821166410 conn(0x556c7d905550 msgr2=0x556c7d9081d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:40:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.600+0000 7f60b3e64000 1 -- 172.21.15.83:0/1821166410 shutdown_connections 2024-04-17T10:40:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.600+0000 7f60b3e64000 1 -- 172.21.15.83:0/1821166410 wait complete. 2024-04-17T10:40:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.600+0000 7f60b3e64000 20 client.5394 trim_cache size 0 max 16384 2024-04-17T10:40:49.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.616+0000 7f60b3e64000 1 Processor -- start 2024-04-17T10:40:49.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.616+0000 7f60b3e64000 1 -- start start 2024-04-17T10:40:49.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.616+0000 7f60b3e64000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c7d9095a0 0x556c7d908950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.616+0000 7f60b3e64000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556c7d93bf00 0x556c7d908e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.616+0000 7f60b3e64000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c7d94eb70 0x556c7d9608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.616+0000 7f60b3e64000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac0ab710 con 0x556c7d9095a0 2024-04-17T10:40:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.616+0000 7f60b3e64000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac0aa8f0 con 0x556c7d94eb70 2024-04-17T10:40:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.616+0000 7f60b3e64000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac15af30 con 0x556c7d93bf00 2024-04-17T10:40:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.616+0000 7f60aa59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c7d9095a0 0x556c7d908950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.616+0000 7f60aa59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c7d9095a0 0x556c7d908950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35740/0 (socket says 172.21.15.83:35740) 2024-04-17T10:40:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.616+0000 7f60aa59c640 1 -- 172.21.15.83:0/2621638099 learned_addr learned my addr 172.21.15.83:0/2621638099 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:40:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.616+0000 7f60a9d9b640 1 --2- 172.21.15.83:0/2621638099 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556c7d93bf00 0x556c7d908e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.616+0000 7f60aad9d640 1 --2- 172.21.15.83:0/2621638099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c7d94eb70 0x556c7d9608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.616+0000 7f60aa59c640 1 -- 172.21.15.83:0/2621638099 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556c7d93bf00 msgr2=0x556c7d908e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:40:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.616+0000 7f60aa59c640 1 --2- 172.21.15.83:0/2621638099 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556c7d93bf00 0x556c7d908e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.616+0000 7f60aa59c640 1 -- 172.21.15.83:0/2621638099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c7d94eb70 msgr2=0x556c7d9608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:40:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.616+0000 7f60aa59c640 1 --2- 172.21.15.83:0/2621638099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c7d94eb70 0x556c7d9608f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.616+0000 7f60aa59c640 1 -- 172.21.15.83:0/2621638099 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c7d7a3a10 con 0x556c7d9095a0 2024-04-17T10:40:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.616+0000 7f60aa59c640 1 --2- 172.21.15.83:0/2621638099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c7d9095a0 0x556c7d908950 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f609c009a60 tx=0x7f609c022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97f302c546dcfe4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:40:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.616+0000 7f608b7fe640 1 -- 172.21.15.83:0/2621638099 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c018040 con 0x556c7d9095a0 2024-04-17T10:40:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.616+0000 7f608b7fe640 1 -- 172.21.15.83:0/2621638099 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f609c000e50 con 0x556c7d9095a0 2024-04-17T10:40:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.616+0000 7f608b7fe640 1 -- 172.21.15.83:0/2621638099 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c0266d0 con 0x556c7d9095a0 2024-04-17T10:40:49.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60b3e64000 1 -- 172.21.15.83:0/2621638099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c7d9095a0 msgr2=0x556c7d908950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:40:49.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60b3e64000 1 --2- 172.21.15.83:0/2621638099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c7d9095a0 0x556c7d908950 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f609c009a60 tx=0x7f609c022000 comp rx=0 tx=0).stop 2024-04-17T10:40:49.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60b3e64000 1 -- 172.21.15.83:0/2621638099 shutdown_connections 2024-04-17T10:40:49.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60b3e64000 1 --2- 172.21.15.83:0/2621638099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c7d94eb70 0x556c7d9608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60b3e64000 1 --2- 172.21.15.83:0/2621638099 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556c7d93bf00 0x556c7d908e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60b3e64000 1 --2- 172.21.15.83:0/2621638099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c7d9095a0 0x556c7d908950 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60b3e64000 1 -- 172.21.15.83:0/2621638099 >> 172.21.15.83:0/2621638099 conn(0x556c7d905550 msgr2=0x556c7d93af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:40:49.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60b3e64000 1 -- 172.21.15.83:0/2621638099 shutdown_connections 2024-04-17T10:40:49.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60b3e64000 1 -- 172.21.15.83:0/2621638099 wait complete. 2024-04-17T10:40:49.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60b3e64000 1 -- start start 2024-04-17T10:40:49.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60b3e64000 1 Processor -- start 2024-04-17T10:40:49.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60b3e64000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c7d9095a0 0x556c7d962b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60b3e64000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556c7d93bf00 0x556c7d96b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60b3e64000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c7d94eb70 0x556c7d96b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60b3e64000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c7d8e75b0 con 0x556c7d9095a0 2024-04-17T10:40:49.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60b3e64000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac15fea0 con 0x556c7d94eb70 2024-04-17T10:40:49.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60b3e64000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c7d806d10 con 0x556c7d93bf00 2024-04-17T10:40:49.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60b3e64000 10 client.0 _ll_register_callbacks cb 0x556c7d70ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:40:49.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60aa59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c7d9095a0 0x556c7d962b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60aa59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c7d9095a0 0x556c7d962b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35744/0 (socket says 172.21.15.83:35744) 2024-04-17T10:40:49.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60aa59c640 1 -- 172.21.15.83:0/444214411 learned_addr learned my addr 172.21.15.83:0/444214411 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:40:49.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60aa59c640 1 -- 172.21.15.83:0/444214411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556c7d93bf00 msgr2=0x556c7d96b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:40:49.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60aa59c640 1 --2- 172.21.15.83:0/444214411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556c7d93bf00 0x556c7d96b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60aa59c640 1 -- 172.21.15.83:0/444214411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c7d94eb70 msgr2=0x556c7d96b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:40:49.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60aad9d640 1 --2- 172.21.15.83:0/444214411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c7d94eb70 0x556c7d96b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60aa59c640 1 --2- 172.21.15.83:0/444214411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c7d94eb70 0x556c7d96b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60aa59c640 1 -- 172.21.15.83:0/444214411 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c7d6deb00 con 0x556c7d9095a0 2024-04-17T10:40:49.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60aa59c640 1 --2- 172.21.15.83:0/444214411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c7d9095a0 0x556c7d962b00 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f609c009250 tx=0x7f609c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff828b3f29019715 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:40:49.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60897fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:40:49.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60897fa640 1 -- 172.21.15.83:0/444214411 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c018040 con 0x556c7d9095a0 2024-04-17T10:40:49.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60897fa640 1 -- 172.21.15.83:0/444214411 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f609c026690 con 0x556c7d9095a0 2024-04-17T10:40:49.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60b3e64000 1 -- 172.21.15.83:0/444214411 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f609c000e50 con 0x556c7d9095a0 2024-04-17T10:40:49.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60897fa640 1 -- 172.21.15.83:0/444214411 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f609c04d050 con 0x556c7d9095a0 2024-04-17T10:40:49.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60b3e64000 10 client.5400 fetch_fsmap learned FSMap version 14 2024-04-17T10:40:49.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60b3e64000 1 -- 172.21.15.83:0/444214411 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x556c7d6f0f80 con 0x556c7d9095a0 2024-04-17T10:40:49.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60897fa640 1 -- 172.21.15.83:0/444214411 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f609c000e50 con 0x556c7d9095a0 2024-04-17T10:40:49.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60897fa640 1 -- 172.21.15.83:0/444214411 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 14) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f609c050020 con 0x556c7d9095a0 2024-04-17T10:40:49.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60b3e64000 10 client.5400 fetch_fsmap finished waiting for FSMap version 14 2024-04-17T10:40:49.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60b3e64000 10 client.5400 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-17T10:40:49.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60b3e64000 20 client.5400 populate_metadata read hostname 'smithi083' 2024-04-17T10:40:49.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60b3e64000 1 --2- 172.21.15.83:0/444214411 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x556c7d96be40 0x556c7d9aee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60b3e64000 4 client.5400 mds_command: new command op to 5172 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-17T10:40:49.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60b3e64000 1 -- 172.21.15.83:0/444214411 --> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x556c7d7573d0 con 0x556c7d96be40 2024-04-17T10:40:49.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60a9d9b640 1 --2- 172.21.15.83:0/444214411 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x556c7d96be40 0x556c7d9aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60897fa640 10 client.5400 ms_handle_connect on v2:172.21.15.83:6836/1181902025 2024-04-17T10:40:49.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.620+0000 7f60a9d9b640 1 --2- 172.21.15.83:0/444214411 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x556c7d96be40 0x556c7d9aee20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95b4bbeac269f809 server_cookie=70bb91521196b395 in_seq=0 out_seq=0 2024-04-17T10:40:49.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.624+0000 7f60897fa640 1 -- 172.21.15.83:0/444214411 <== mds.0 v2:172.21.15.83:6836/1181902025 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17936 (crc 0 0 0) 0x556c7d7573d0 con 0x556c7d96be40 2024-04-17T10:40:49.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.624+0000 7f60897fa640 10 client.5400 handle_command_reply: tid=0 2024-04-17T10:40:49.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.624+0000 7f60b3e64000 1 client.5400 shutdown 2024-04-17T10:40:49.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.624+0000 7f60b3e64000 1 -- 172.21.15.83:0/444214411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c7d9095a0 msgr2=0x556c7d962b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:40:49.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.624+0000 7f60b3e64000 1 --2- 172.21.15.83:0/444214411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c7d9095a0 0x556c7d962b00 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f609c009250 tx=0x7f609c008040 comp rx=0 tx=0).stop 2024-04-17T10:40:49.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.624+0000 7f60b3e64000 1 -- 172.21.15.83:0/444214411 shutdown_connections 2024-04-17T10:40:49.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.624+0000 7f60b3e64000 1 --2- 172.21.15.83:0/444214411 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x556c7d96be40 0x556c7d9aee20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.624+0000 7f60b3e64000 1 --2- 172.21.15.83:0/444214411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c7d94eb70 0x556c7d96b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.624+0000 7f60b3e64000 1 --2- 172.21.15.83:0/444214411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556c7d93bf00 0x556c7d96b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.624+0000 7f60897fa640 0 client.5400 ms_handle_reset on v2:172.21.15.83:6836/1181902025 2024-04-17T10:40:49.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.624+0000 7f60b3e64000 1 --2- 172.21.15.83:0/444214411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c7d9095a0 0x556c7d962b00 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.624+0000 7f60b3e64000 1 -- 172.21.15.83:0/444214411 >> 172.21.15.83:0/444214411 conn(0x556c7d905550 msgr2=0x556c7d907940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:40:49.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.624+0000 7f60b3e64000 1 -- 172.21.15.83:0/444214411 shutdown_connections 2024-04-17T10:40:49.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.624+0000 7f60b3e64000 1 -- 172.21.15.83:0/444214411 wait complete. 2024-04-17T10:40:49.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.624+0000 7f60b3e64000 20 client.5400 trim_cache size 0 max 16384 2024-04-17T10:40:49.625 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:40:49.626 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T10:40:49.626 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 4148, 2024-04-17T10:40:49.626 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 4149, 2024-04-17T10:40:49.626 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1031866, 2024-04-17T10:40:49.626 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 34605659, 2024-04-17T10:40:49.626 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 9, 2024-04-17T10:40:49.626 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 3, 2024-04-17T10:40:49.626 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.473754403, 2024-04-17T10:40:49.626 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.204659391, 2024-04-17T10:40:49.626 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.089661164, 2024-04-17T10:40:49.626 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.142815583, 2024-04-17T10:40:49.626 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T10:40:49.626 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 4147, 2024-04-17T10:40:49.627 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.418815244, 2024-04-17T10:40:49.627 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000100992 2024-04-17T10:40:49.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.627 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-17T10:40:49.627 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2, 2024-04-17T10:40:49.627 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000262, 2024-04-17T10:40:49.627 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000131 2024-04-17T10:40:49.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.627 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-17T10:40:49.627 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 0 2024-04-17T10:40:49.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.627 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T10:40:49.627 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 33439, 2024-04-17T10:40:49.627 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 59646, 2024-04-17T10:40:49.628 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 12026839, 2024-04-17T10:40:49.628 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 42792308, 2024-04-17T10:40:49.628 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 8, 2024-04-17T10:40:49.628 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 4, 2024-04-17T10:40:49.628 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 4.896280256, 2024-04-17T10:40:49.628 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 2.466766648, 2024-04-17T10:40:49.628 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 2.959977998, 2024-04-17T10:40:49.628 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.053211932, 2024-04-17T10:40:49.628 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T10:40:49.628 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 59645, 2024-04-17T10:40:49.628 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 1.849825336, 2024-04-17T10:40:49.628 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000031013 2024-04-17T10:40:49.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.629 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-17T10:40:49.629 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 88152, 2024-04-17T10:40:49.629 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.021584463, 2024-04-17T10:40:49.629 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000244 2024-04-17T10:40:49.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.629 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-04-17T10:40:49.629 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 121488 2024-04-17T10:40:49.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.629 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T10:40:49.629 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 3974, 2024-04-17T10:40:49.629 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 3969, 2024-04-17T10:40:49.630 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1266414, 2024-04-17T10:40:49.630 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 39005195, 2024-04-17T10:40:49.630 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 5, 2024-04-17T10:40:49.630 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 4, 2024-04-17T10:40:49.630 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.494973122, 2024-04-17T10:40:49.630 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.206691373, 2024-04-17T10:40:49.630 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.101462087, 2024-04-17T10:40:49.630 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.166172862, 2024-04-17T10:40:49.630 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T10:40:49.631 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3969, 2024-04-17T10:40:49.631 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.529952240, 2024-04-17T10:40:49.631 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000133522 2024-04-17T10:40:49.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.631 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-17T10:40:49.631 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:40:49.631 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:40:49.631 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:40:49.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.631 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 114544, 2024-04-17T10:40:49.631 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 24096 2024-04-17T10:40:49.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.631 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-04-17T10:40:49.631 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 1, 2024-04-17T10:40:49.632 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-04-17T10:40:49.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.632 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-MDSRank": { 2024-04-17T10:40:49.632 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-04-17T10:40:49.632 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-04-17T10:40:49.632 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 9068, 2024-04-17T10:40:49.632 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 24.074249146, 2024-04-17T10:40:49.632 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.002654857 2024-04-17T10:40:49.632 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:40:49.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.632 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-PurgeQueue": { 2024-04-17T10:40:49.633 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-04-17T10:40:49.633 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-04-17T10:40:49.633 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2220, 2024-04-17T10:40:49.633 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 1.115948721, 2024-04-17T10:40:49.633 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000502679 2024-04-17T10:40:49.633 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:40:49.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.633 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": { 2024-04-17T10:40:49.633 INFO:teuthology.orchestra.run.smithi083.stdout: "request": 26730, 2024-04-17T10:40:49.633 INFO:teuthology.orchestra.run.smithi083.stdout: "reply": 26730, 2024-04-17T10:40:49.633 INFO:teuthology.orchestra.run.smithi083.stdout: "reply_latency": { 2024-04-17T10:40:49.633 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 26730, 2024-04-17T10:40:49.633 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 449.985240211, 2024-04-17T10:40:49.633 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.016834464 2024-04-17T10:40:49.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.634 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_reply": 0, 2024-04-17T10:40:49.634 INFO:teuthology.orchestra.run.smithi083.stdout: "forward": 0, 2024-04-17T10:40:49.634 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_complete": 0, 2024-04-17T10:40:49.634 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_keys": 0, 2024-04-17T10:40:49.634 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_commit": 12, 2024-04-17T10:40:49.634 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_split": 0, 2024-04-17T10:40:49.634 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_merge": 0, 2024-04-17T10:40:49.634 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes": 19851, 2024-04-17T10:40:49.634 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_top": 0, 2024-04-17T10:40:49.634 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_bottom": 19851, 2024-04-17T10:40:49.634 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pin_tail": 0, 2024-04-17T10:40:49.634 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pinned": 19851, 2024-04-17T10:40:49.635 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_expired": 0, 2024-04-17T10:40:49.635 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_with_caps": 271, 2024-04-17T10:40:49.635 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": 271, 2024-04-17T10:40:49.635 INFO:teuthology.orchestra.run.smithi083.stdout: "subtrees": 2, 2024-04-17T10:40:49.635 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse": 43728, 2024-04-17T10:40:49.635 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_hit": 43254, 2024-04-17T10:40:49.635 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_forward": 0, 2024-04-17T10:40:49.635 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_discover": 0, 2024-04-17T10:40:49.635 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_dir_fetch": 0, 2024-04-17T10:40:49.635 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_remote_ino": 0, 2024-04-17T10:40:49.635 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_lock": 149, 2024-04-17T10:40:49.635 INFO:teuthology.orchestra.run.smithi083.stdout: "load_cent": 0, 2024-04-17T10:40:49.635 INFO:teuthology.orchestra.run.smithi083.stdout: "q": 0, 2024-04-17T10:40:49.635 INFO:teuthology.orchestra.run.smithi083.stdout: "exported": 0, 2024-04-17T10:40:49.635 INFO:teuthology.orchestra.run.smithi083.stdout: "exported_inodes": 0, 2024-04-17T10:40:49.636 INFO:teuthology.orchestra.run.smithi083.stdout: "imported": 0, 2024-04-17T10:40:49.636 INFO:teuthology.orchestra.run.smithi083.stdout: "imported_inodes": 0, 2024-04-17T10:40:49.636 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_dir_fetch": 0, 2024-04-17T10:40:49.636 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_backtrace_fetch": 0, 2024-04-17T10:40:49.636 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_peer_discover": 0, 2024-04-17T10:40:49.636 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rfiles": 0, 2024-04-17T10:40:49.636 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rbytes": 0, 2024-04-17T10:40:49.636 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rsnaps": 0, 2024-04-17T10:40:49.636 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_fetch": 0, 2024-04-17T10:40:49.636 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_set_tag": 0, 2024-04-17T10:40:49.636 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_repaired": 0, 2024-04-17T10:40:49.636 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_inotable_repaired": 0, 2024-04-17T10:40:49.637 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_inodes": 0, 2024-04-17T10:40:49.637 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_base_inodes": 0, 2024-04-17T10:40:49.637 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dirfrag_rstats": 0, 2024-04-17T10:40:49.637 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_file_inodes": 0, 2024-04-17T10:40:49.637 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_inode_file_caps": 0, 2024-04-17T10:40:49.637 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_revoke": 7781, 2024-04-17T10:40:49.637 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_grant": 6321, 2024-04-17T10:40:49.637 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_trunc": 0, 2024-04-17T10:40:49.637 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-17T10:40:49.638 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-17T10:40:49.638 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps": 7781, 2024-04-17T10:40:49.638 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps_dirty": 2000, 2024-04-17T10:40:49.638 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_cap_release": 51, 2024-04-17T10:40:49.638 INFO:teuthology.orchestra.run.smithi083.stdout: "process_request_cap_release": 17966 2024-04-17T10:40:49.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.638 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache": { 2024-04-17T10:40:49.638 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update": 0, 2024-04-17T10:40:49.638 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update_receipt": 0, 2024-04-17T10:40:49.638 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_try_discover": 0, 2024-04-17T10:40:49.638 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_send_discover": 0, 2024-04-17T10:40:49.639 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_handle_discover": 0, 2024-04-17T10:40:49.639 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays": 1889, 2024-04-17T10:40:49.639 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_delayed": 0, 2024-04-17T10:40:49.639 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_enqueuing": 0, 2024-04-17T10:40:49.639 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_created": 4093, 2024-04-17T10:40:49.639 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_enqueued": 1731, 2024-04-17T10:40:49.639 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_reintegrated": 2187, 2024-04-17T10:40:49.639 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_migrated": 0, 2024-04-17T10:40:49.639 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_processing": 0, 2024-04-17T10:40:49.639 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_enqueued": 0, 2024-04-17T10:40:49.639 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_prioritized": 0, 2024-04-17T10:40:49.639 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_started": 0, 2024-04-17T10:40:49.639 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_completed": 0, 2024-04-17T10:40:49.640 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_enqueue_scrub": 0, 2024-04-17T10:40:49.640 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_exportdir": 0, 2024-04-17T10:40:49.640 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_flush": 0, 2024-04-17T10:40:49.640 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragmentdir": 0, 2024-04-17T10:40:49.640 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragstats": 0, 2024-04-17T10:40:49.640 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_inodestats": 0 2024-04-17T10:40:49.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.640 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log": { 2024-04-17T10:40:49.640 INFO:teuthology.orchestra.run.smithi083.stdout: "evlrg": 0, 2024-04-17T10:40:49.640 INFO:teuthology.orchestra.run.smithi083.stdout: "evadd": 23806, 2024-04-17T10:40:49.640 INFO:teuthology.orchestra.run.smithi083.stdout: "evex": 0, 2024-04-17T10:40:49.641 INFO:teuthology.orchestra.run.smithi083.stdout: "evtrm": 0, 2024-04-17T10:40:49.641 INFO:teuthology.orchestra.run.smithi083.stdout: "ev": 23806, 2024-04-17T10:40:49.641 INFO:teuthology.orchestra.run.smithi083.stdout: "evexg": 0, 2024-04-17T10:40:49.641 INFO:teuthology.orchestra.run.smithi083.stdout: "evexd": 0, 2024-04-17T10:40:49.641 INFO:teuthology.orchestra.run.smithi083.stdout: "segadd": 31, 2024-04-17T10:40:49.641 INFO:teuthology.orchestra.run.smithi083.stdout: "segex": 0, 2024-04-17T10:40:49.641 INFO:teuthology.orchestra.run.smithi083.stdout: "segtrm": 0, 2024-04-17T10:40:49.641 INFO:teuthology.orchestra.run.smithi083.stdout: "seg": 31, 2024-04-17T10:40:49.641 INFO:teuthology.orchestra.run.smithi083.stdout: "segmjr": 3, 2024-04-17T10:40:49.641 INFO:teuthology.orchestra.run.smithi083.stdout: "segexg": 0, 2024-04-17T10:40:49.641 INFO:teuthology.orchestra.run.smithi083.stdout: "segexd": 0, 2024-04-17T10:40:49.641 INFO:teuthology.orchestra.run.smithi083.stdout: "expos": 4194304, 2024-04-17T10:40:49.641 INFO:teuthology.orchestra.run.smithi083.stdout: "wrpos": 85327309, 2024-04-17T10:40:49.641 INFO:teuthology.orchestra.run.smithi083.stdout: "rdpos": 0, 2024-04-17T10:40:49.642 INFO:teuthology.orchestra.run.smithi083.stdout: "jlat": { 2024-04-17T10:40:49.642 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 6802, 2024-04-17T10:40:49.642 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 117.001430822, 2024-04-17T10:40:49.642 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.017201033 2024-04-17T10:40:49.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.642 INFO:teuthology.orchestra.run.smithi083.stdout: "replayed": 0 2024-04-17T10:40:49.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.642 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_mem": { 2024-04-17T10:40:49.642 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 2014, 2024-04-17T10:40:49.642 INFO:teuthology.orchestra.run.smithi083.stdout: "ino+": 2014, 2024-04-17T10:40:49.642 INFO:teuthology.orchestra.run.smithi083.stdout: "ino-": 0, 2024-04-17T10:40:49.642 INFO:teuthology.orchestra.run.smithi083.stdout: "dir": 13, 2024-04-17T10:40:49.642 INFO:teuthology.orchestra.run.smithi083.stdout: "dir+": 13, 2024-04-17T10:40:49.643 INFO:teuthology.orchestra.run.smithi083.stdout: "dir-": 0, 2024-04-17T10:40:49.643 INFO:teuthology.orchestra.run.smithi083.stdout: "dn": 19851, 2024-04-17T10:40:49.643 INFO:teuthology.orchestra.run.smithi083.stdout: "dn+": 19851, 2024-04-17T10:40:49.643 INFO:teuthology.orchestra.run.smithi083.stdout: "dn-": 0, 2024-04-17T10:40:49.643 INFO:teuthology.orchestra.run.smithi083.stdout: "cap": 271, 2024-04-17T10:40:49.643 INFO:teuthology.orchestra.run.smithi083.stdout: "cap+": 2002, 2024-04-17T10:40:49.643 INFO:teuthology.orchestra.run.smithi083.stdout: "cap-": 1731, 2024-04-17T10:40:49.643 INFO:teuthology.orchestra.run.smithi083.stdout: "rss": 101496, 2024-04-17T10:40:49.643 INFO:teuthology.orchestra.run.smithi083.stdout: "heap": 215340 2024-04-17T10:40:49.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.643 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_server": { 2024-04-17T10:40:49.643 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_client_request": 36099, 2024-04-17T10:40:49.643 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_server_request": 0, 2024-04-17T10:40:49.643 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_request": 26730, 2024-04-17T10:40:49.643 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_session": 7, 2024-04-17T10:40:49.644 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_peer_request": 0, 2024-04-17T10:40:49.644 INFO:teuthology.orchestra.run.smithi083.stdout: "req_create_latency": { 2024-04-17T10:40:49.644 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2000, 2024-04-17T10:40:49.644 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 1.389634135, 2024-04-17T10:40:49.644 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000694817 2024-04-17T10:40:49.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.644 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getattr_latency": { 2024-04-17T10:40:49.644 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 515, 2024-04-17T10:40:49.644 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.150340387, 2024-04-17T10:40:49.644 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000291923 2024-04-17T10:40:49.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.644 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getfilelock_latency": { 2024-04-17T10:40:49.644 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:40:49.644 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:40:49.644 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:40:49.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.645 INFO:teuthology.orchestra.run.smithi083.stdout: "req_link_latency": { 2024-04-17T10:40:49.645 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 6000, 2024-04-17T10:40:49.645 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 9.614716615, 2024-04-17T10:40:49.645 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.001602452 2024-04-17T10:40:49.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.645 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookup_latency": { 2024-04-17T10:40:49.645 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 8015, 2024-04-17T10:40:49.645 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 15.699825801, 2024-04-17T10:40:49.645 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.001958805 2024-04-17T10:40:49.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.645 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookuphash_latency": { 2024-04-17T10:40:49.645 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:40:49.645 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:40:49.645 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:40:49.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.646 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupino_latency": { 2024-04-17T10:40:49.646 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:40:49.646 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:40:49.646 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:40:49.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.646 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupname_latency": { 2024-04-17T10:40:49.646 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:40:49.646 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:40:49.646 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:40:49.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.646 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupparent_latency": { 2024-04-17T10:40:49.646 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:40:49.646 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:40:49.647 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:40:49.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.647 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupsnap_latency": { 2024-04-17T10:40:49.647 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3, 2024-04-17T10:40:49.647 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.003287393, 2024-04-17T10:40:49.647 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.001095797 2024-04-17T10:40:49.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.647 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lssnap_latency": { 2024-04-17T10:40:49.647 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:40:49.647 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:40:49.647 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:40:49.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.647 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mkdir_latency": { 2024-04-17T10:40:49.647 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1, 2024-04-17T10:40:49.648 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.002850512, 2024-04-17T10:40:49.648 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.002850512 2024-04-17T10:40:49.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.648 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mknod_latency": { 2024-04-17T10:40:49.648 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:40:49.648 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:40:49.648 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:40:49.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.648 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mksnap_latency": { 2024-04-17T10:40:49.648 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1, 2024-04-17T10:40:49.648 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.005833500, 2024-04-17T10:40:49.648 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.005833500 2024-04-17T10:40:49.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.649 INFO:teuthology.orchestra.run.smithi083.stdout: "req_open_latency": { 2024-04-17T10:40:49.649 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:40:49.649 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:40:49.649 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:40:49.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.649 INFO:teuthology.orchestra.run.smithi083.stdout: "req_readdir_latency": { 2024-04-17T10:40:49.649 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 6, 2024-04-17T10:40:49.649 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.219240218, 2024-04-17T10:40:49.650 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.036540036 2024-04-17T10:40:49.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.650 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rename_latency": { 2024-04-17T10:40:49.650 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2187, 2024-04-17T10:40:49.650 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 392.619088066, 2024-04-17T10:40:49.650 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.179524045 2024-04-17T10:40:49.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.650 INFO:teuthology.orchestra.run.smithi083.stdout: "req_renamesnap_latency": { 2024-04-17T10:40:49.650 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:40:49.650 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:40:49.650 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:40:49.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.650 INFO:teuthology.orchestra.run.smithi083.stdout: "req_snapdiff_latency": { 2024-04-17T10:40:49.651 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:40:49.651 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:40:49.651 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:40:49.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.651 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmdir_latency": { 2024-04-17T10:40:49.651 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:40:49.651 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:40:49.651 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:40:49.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.651 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmsnap_latency": { 2024-04-17T10:40:49.651 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1, 2024-04-17T10:40:49.651 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.003107278, 2024-04-17T10:40:49.651 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.003107278 2024-04-17T10:40:49.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.652 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmxattr_latency": { 2024-04-17T10:40:49.652 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:40:49.652 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:40:49.652 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:40:49.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.652 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setattr_latency": { 2024-04-17T10:40:49.652 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1, 2024-04-17T10:40:49.652 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000555286, 2024-04-17T10:40:49.652 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000555286 2024-04-17T10:40:49.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.652 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setdirlayout_latency": { 2024-04-17T10:40:49.652 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:40:49.652 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:40:49.652 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:40:49.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.653 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setfilelock_latency": { 2024-04-17T10:40:49.653 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:40:49.653 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:40:49.653 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:40:49.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.653 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setlayout_latency": { 2024-04-17T10:40:49.653 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:40:49.653 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:40:49.653 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:40:49.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.653 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setxattr_latency": { 2024-04-17T10:40:49.653 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:40:49.653 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:40:49.654 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:40:49.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.654 INFO:teuthology.orchestra.run.smithi083.stdout: "req_symlink_latency": { 2024-04-17T10:40:49.654 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:40:49.654 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:40:49.654 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:40:49.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.654 INFO:teuthology.orchestra.run.smithi083.stdout: "req_unlink_latency": { 2024-04-17T10:40:49.654 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 8000, 2024-04-17T10:40:49.654 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 30.276761020, 2024-04-17T10:40:49.655 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.003784595 2024-04-17T10:40:49.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.655 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_revoke_eviction": 0, 2024-04-17T10:40:49.655 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_acquisition_throttle": 0, 2024-04-17T10:40:49.655 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getvxattr_latency": { 2024-04-17T10:40:49.655 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:40:49.655 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:40:49.655 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:40:49.656 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:40:49.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.656 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_sessions": { 2024-04-17T10:40:49.656 INFO:teuthology.orchestra.run.smithi083.stdout: "session_count": 1, 2024-04-17T10:40:49.656 INFO:teuthology.orchestra.run.smithi083.stdout: "session_add": 1, 2024-04-17T10:40:49.656 INFO:teuthology.orchestra.run.smithi083.stdout: "session_remove": 0, 2024-04-17T10:40:49.656 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_open": 1, 2024-04-17T10:40:49.656 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_stale": 0, 2024-04-17T10:40:49.656 INFO:teuthology.orchestra.run.smithi083.stdout: "total_load": 15015, 2024-04-17T10:40:49.657 INFO:teuthology.orchestra.run.smithi083.stdout: "average_load": 15015, 2024-04-17T10:40:49.657 INFO:teuthology.orchestra.run.smithi083.stdout: "avg_session_uptime": 86, 2024-04-17T10:40:49.657 INFO:teuthology.orchestra.run.smithi083.stdout: "mdthresh_evicted": 0 2024-04-17T10:40:49.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.657 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-04-17T10:40:49.657 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-04-17T10:40:49.657 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-04-17T10:40:49.657 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T10:40:49.658 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-04-17T10:40:49.658 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T10:40:49.658 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-04-17T10:40:49.658 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T10:40:49.658 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T10:40:49.658 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T10:40:49.658 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T10:40:49.658 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T10:40:49.659 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-04-17T10:40:49.659 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T10:40:49.659 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T10:40:49.659 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-04-17T10:40:49.659 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-04-17T10:40:49.659 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-04-17T10:40:49.659 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-04-17T10:40:49.659 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T10:40:49.660 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T10:40:49.660 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T10:40:49.660 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T10:40:49.660 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T10:40:49.660 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-04-17T10:40:49.660 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-04-17T10:40:49.660 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-04-17T10:40:49.660 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T10:40:49.661 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T10:40:49.661 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-04-17T10:40:49.661 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-04-17T10:40:49.661 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-04-17T10:40:49.661 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-04-17T10:40:49.661 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T10:40:49.661 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-04-17T10:40:49.661 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T10:40:49.661 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-04-17T10:40:49.662 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 18242427, 2024-04-17T10:40:49.662 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 4593, 2024-04-17T10:40:49.662 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 88, 2024-04-17T10:40:49.662 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 1, 2024-04-17T10:40:49.662 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-04-17T10:40:49.662 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-04-17T10:40:49.662 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-04-17T10:40:49.662 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-04-17T10:40:49.663 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-04-17T10:40:49.663 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-04-17T10:40:49.663 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 5312, 2024-04-17T10:40:49.663 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 75, 2024-04-17T10:40:49.663 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T10:40:49.663 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-04-17T10:40:49.663 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-04-17T10:40:49.663 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-04-17T10:40:49.663 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 20849273, 2024-04-17T10:40:49.663 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 815689, 2024-04-17T10:40:49.663 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-04-17T10:40:49.663 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-04-17T10:40:49.663 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-04-17T10:40:49.664 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-04-17T10:40:49.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.664 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter": { 2024-04-17T10:40:49.664 INFO:teuthology.orchestra.run.smithi083.stdout: "op_active": 0, 2024-04-17T10:40:49.664 INFO:teuthology.orchestra.run.smithi083.stdout: "op_laggy": 0, 2024-04-17T10:40:49.664 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send": 8965, 2024-04-17T10:40:49.664 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send_bytes": 81353305, 2024-04-17T10:40:49.664 INFO:teuthology.orchestra.run.smithi083.stdout: "op_resend": 37, 2024-04-17T10:40:49.664 INFO:teuthology.orchestra.run.smithi083.stdout: "op_reply": 8965, 2024-04-17T10:40:49.664 INFO:teuthology.orchestra.run.smithi083.stdout: "op_latency": { 2024-04-17T10:40:49.664 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 8965, 2024-04-17T10:40:49.664 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 43.859072166, 2024-04-17T10:40:49.664 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.004892255 2024-04-17T10:40:49.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.665 INFO:teuthology.orchestra.run.smithi083.stdout: "op_inflight": 0, 2024-04-17T10:40:49.665 INFO:teuthology.orchestra.run.smithi083.stdout: "oplen_avg": { 2024-04-17T10:40:49.665 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 8965, 2024-04-17T10:40:49.665 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 9121 2024-04-17T10:40:49.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.665 INFO:teuthology.orchestra.run.smithi083.stdout: "op": 8965, 2024-04-17T10:40:49.665 INFO:teuthology.orchestra.run.smithi083.stdout: "op_r": 112, 2024-04-17T10:40:49.665 INFO:teuthology.orchestra.run.smithi083.stdout: "op_w": 8853, 2024-04-17T10:40:49.665 INFO:teuthology.orchestra.run.smithi083.stdout: "op_rmw": 0, 2024-04-17T10:40:49.665 INFO:teuthology.orchestra.run.smithi083.stdout: "op_pg": 0, 2024-04-17T10:40:49.665 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_stat": 0, 2024-04-17T10:40:49.665 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_create": 11, 2024-04-17T10:40:49.665 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_read": 112, 2024-04-17T10:40:49.666 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_write": 6914, 2024-04-17T10:40:49.666 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writefull": 65, 2024-04-17T10:40:49.666 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writesame": 0, 2024-04-17T10:40:49.666 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_append": 0, 2024-04-17T10:40:49.666 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_zero": 1620, 2024-04-17T10:40:49.666 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_truncate": 0, 2024-04-17T10:40:49.666 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_delete": 142, 2024-04-17T10:40:49.666 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_mapext": 0, 2024-04-17T10:40:49.666 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_sparse_read": 0, 2024-04-17T10:40:49.666 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_clonerange": 0, 2024-04-17T10:40:49.666 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_getxattr": 0, 2024-04-17T10:40:49.667 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_setxattr": 22, 2024-04-17T10:40:49.667 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_cmpxattr": 0, 2024-04-17T10:40:49.667 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_rmxattr": 0, 2024-04-17T10:40:49.667 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_resetxattrs": 0, 2024-04-17T10:40:49.667 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_call": 0, 2024-04-17T10:40:49.667 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_watch": 0, 2024-04-17T10:40:49.667 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_notify": 0, 2024-04-17T10:40:49.667 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_src_cmpxattr": 0, 2024-04-17T10:40:49.667 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls": 0, 2024-04-17T10:40:49.667 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls_filter": 0, 2024-04-17T10:40:49.667 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_other": 0, 2024-04-17T10:40:49.667 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_active": 0, 2024-04-17T10:40:49.667 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_send": 0, 2024-04-17T10:40:49.668 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_resend": 0, 2024-04-17T10:40:49.668 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_ping": 0, 2024-04-17T10:40:49.668 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_active": 0, 2024-04-17T10:40:49.668 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_send": 0, 2024-04-17T10:40:49.668 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_resend": 0, 2024-04-17T10:40:49.668 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_active": 0, 2024-04-17T10:40:49.668 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_send": 0, 2024-04-17T10:40:49.668 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_resend": 0, 2024-04-17T10:40:49.668 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_active": 0, 2024-04-17T10:40:49.668 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_send": 0, 2024-04-17T10:40:49.668 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_resend": 0, 2024-04-17T10:40:49.669 INFO:teuthology.orchestra.run.smithi083.stdout: "command_active": 0, 2024-04-17T10:40:49.669 INFO:teuthology.orchestra.run.smithi083.stdout: "command_send": 0, 2024-04-17T10:40:49.669 INFO:teuthology.orchestra.run.smithi083.stdout: "command_resend": 0, 2024-04-17T10:40:49.669 INFO:teuthology.orchestra.run.smithi083.stdout: "map_epoch": 33, 2024-04-17T10:40:49.669 INFO:teuthology.orchestra.run.smithi083.stdout: "map_full": 0, 2024-04-17T10:40:49.669 INFO:teuthology.orchestra.run.smithi083.stdout: "map_inc": 1, 2024-04-17T10:40:49.669 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_sessions": 8, 2024-04-17T10:40:49.669 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_open": 8, 2024-04-17T10:40:49.669 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_close": 0, 2024-04-17T10:40:49.670 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_laggy": 0, 2024-04-17T10:40:49.670 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_wr": 187, 2024-04-17T10:40:49.670 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_rd": 0, 2024-04-17T10:40:49.670 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_del": 48 2024-04-17T10:40:49.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.670 INFO:teuthology.orchestra.run.smithi083.stdout: "oft": { 2024-04-17T10:40:49.670 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_objs": 1, 2024-04-17T10:40:49.670 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_kv_pairs": 0, 2024-04-17T10:40:49.670 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_updates": 0, 2024-04-17T10:40:49.670 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_removes": 0 2024-04-17T10:40:49.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.670 INFO:teuthology.orchestra.run.smithi083.stdout: "osd-slow-ops": { 2024-04-17T10:40:49.671 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_ops_count": 0 2024-04-17T10:40:49.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.671 INFO:teuthology.orchestra.run.smithi083.stdout: "purge_queue": { 2024-04-17T10:40:49.671 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops": 0, 2024-04-17T10:40:49.671 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops_high_water": 32, 2024-04-17T10:40:49.671 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing": 0, 2024-04-17T10:40:49.671 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_high_water": 32, 2024-04-17T10:40:49.671 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed_ops": 1731, 2024-04-17T10:40:49.671 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed": 1731, 2024-04-17T10:40:49.671 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_item_in_journal": 0 2024-04-17T10:40:49.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.671 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T10:40:49.671 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-17T10:40:49.672 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-17T10:40:49.672 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-17T10:40:49.672 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 41556, 2024-04-17T10:40:49.672 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 10579709, 2024-04-17T10:40:49.672 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-17T10:40:49.672 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 41556, 2024-04-17T10:40:49.672 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-17T10:40:49.672 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-17T10:40:49.672 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 41556, 2024-04-17T10:40:49.672 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 10579709, 2024-04-17T10:40:49.672 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-17T10:40:49.672 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:40:49.672 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:40:49.672 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:40:49.672 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:40:49.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.673 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_bytes": { 2024-04-17T10:40:49.673 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-17T10:40:49.673 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-17T10:40:49.673 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-17T10:40:49.673 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-17T10:40:49.673 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-17T10:40:49.673 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-17T10:40:49.673 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-17T10:40:49.673 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 8965, 2024-04-17T10:40:49.673 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 81541096, 2024-04-17T10:40:49.673 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 8965, 2024-04-17T10:40:49.673 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 81541096, 2024-04-17T10:40:49.674 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-17T10:40:49.674 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:40:49.674 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:40:49.674 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:40:49.674 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:40:49.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.674 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_ops": { 2024-04-17T10:40:49.674 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-17T10:40:49.674 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 1024, 2024-04-17T10:40:49.674 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-17T10:40:49.675 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-17T10:40:49.675 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-17T10:40:49.675 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-17T10:40:49.675 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-17T10:40:49.675 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 8965, 2024-04-17T10:40:49.675 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 8965, 2024-04-17T10:40:49.675 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 8965, 2024-04-17T10:40:49.675 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 8965, 2024-04-17T10:40:49.675 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-17T10:40:49.675 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:40:49.675 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:40:49.675 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:40:49.675 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:40:49.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.676 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle": { 2024-04-17T10:40:49.676 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-17T10:40:49.676 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-04-17T10:40:49.676 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-17T10:40:49.676 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 1731, 2024-04-17T10:40:49.676 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 187791, 2024-04-17T10:40:49.676 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-17T10:40:49.676 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 1731, 2024-04-17T10:40:49.676 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-17T10:40:49.676 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-17T10:40:49.676 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 112, 2024-04-17T10:40:49.676 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 187791, 2024-04-17T10:40:49.676 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-17T10:40:49.676 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:40:49.676 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:40:49.677 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:40:49.677 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:40:49.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:40:49.677 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle-0x55812d868960": { 2024-04-17T10:40:49.677 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 66573, 2024-04-17T10:40:49.677 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-04-17T10:40:49.677 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-17T10:40:49.677 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 23806, 2024-04-17T10:40:49.677 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 81133005, 2024-04-17T10:40:49.677 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-17T10:40:49.677 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 23806, 2024-04-17T10:40:49.677 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-17T10:40:49.677 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-17T10:40:49.677 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 6802, 2024-04-17T10:40:49.678 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 81066432, 2024-04-17T10:40:49.678 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-17T10:40:49.678 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:40:49.678 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:40:49.678 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:40:49.678 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:40:49.678 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:40:49.678 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:40:49.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.624+0000 7f60b0cf7640 1 -- 172.21.15.83:0/1665442438 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f608c066f60 msgr2=0x7f608c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:40:49.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.624+0000 7f60b0cf7640 1 --2- 172.21.15.83:0/1665442438 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f608c066f60 0x7f608c069420 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f60a0007fd0 tx=0x7f60a0007900 comp rx=0 tx=0).stop 2024-04-17T10:40:49.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.624+0000 7f60b0cf7640 1 -- 172.21.15.83:0/1665442438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60ac158030 msgr2=0x7f60ac141d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:40:49.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.624+0000 7f60b0cf7640 1 --2- 172.21.15.83:0/1665442438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60ac158030 0x7f60ac141d80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f60a40b0040 tx=0x7f60a4019590 comp rx=0 tx=0).stop 2024-04-17T10:40:49.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.624+0000 7f60b0cf7640 1 -- 172.21.15.83:0/1665442438 shutdown_connections 2024-04-17T10:40:49.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.624+0000 7f60b0cf7640 1 --2- 172.21.15.83:0/1665442438 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f608c066f60 0x7f608c069420 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.624+0000 7f60b0cf7640 1 --2- 172.21.15.83:0/1665442438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60ac158030 0x7f60ac141d80 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.624+0000 7f60b0cf7640 1 --2- 172.21.15.83:0/1665442438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60ac0b27c0 0x7f60ac1598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.624+0000 7f60b0cf7640 1 --2- 172.21.15.83:0/1665442438 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60ac0adf50 0x7f60ac0b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.624+0000 7f60b0cf7640 1 -- 172.21.15.83:0/1665442438 >> 172.21.15.83:0/1665442438 conn(0x7f60ac0cf630 msgr2=0x7f60ac0ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:40:49.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.624+0000 7f60b0cf7640 1 -- 172.21.15.83:0/1665442438 shutdown_connections 2024-04-17T10:40:49.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.624+0000 7f60b0cf7640 1 -- 172.21.15.83:0/1665442438 wait complete. 2024-04-17T10:40:49.681 INFO:tasks.cephfs.test_scrub_checks:num of strays: 1889 2024-04-17T10:40:49.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:40:49.681 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub start / recursive,scrub_mdsdir 2024-04-17T10:40:49.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.812+0000 7f2424481640 1 Processor -- start 2024-04-17T10:40:49.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.812+0000 7f2424481640 1 -- start start 2024-04-17T10:40:49.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.816+0000 7f2424481640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f241c06b2f0 0x7f241c166880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.816+0000 7f2424481640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f241c0b39d0 con 0x7f241c15cb40 2024-04-17T10:40:49.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.816+0000 7f2424481640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f241c070c40 con 0x7f241c0aedf0 2024-04-17T10:40:49.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.816+0000 7f2424481640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f241c072730 con 0x7f241c06b2f0 2024-04-17T10:40:49.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.816+0000 7f2422a1e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f241c06b2f0 0x7f241c166880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.816+0000 7f2422a1e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f241c06b2f0 0x7f241c166880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35760/0 (socket says 172.21.15.83:35760) 2024-04-17T10:40:49.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.816+0000 7f2422a1e640 1 -- 172.21.15.83:0/1860243620 learned_addr learned my addr 172.21.15.83:0/1860243620 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:40:49.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.816+0000 7f2422a1e640 1 -- 172.21.15.83:0/1860243620 >> v1:172.21.15.167:6789/0 conn(0x7f241c0aedf0 legacy=0x7f241c0b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:40:49.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.816+0000 7f2422a1e640 1 -- 172.21.15.83:0/1860243620 >> v1:172.21.15.83:6789/0 conn(0x7f241c15cb40 legacy=0x7f241c06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:40:49.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.816+0000 7f242121b640 1 -- 172.21.15.83:0/1860243620 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3687998593 0 0) 0x7f241c0b39d0 con 0x7f241c15cb40 2024-04-17T10:40:49.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.816+0000 7f2422a1e640 1 -- 172.21.15.83:0/1860243620 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f241c070f20 con 0x7f241c06b2f0 2024-04-17T10:40:49.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.820+0000 7f2422a1e640 1 --2- 172.21.15.83:0/1860243620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241c06b2f0 0x7f241c166880 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2414009870 tx=0x7f241401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160122c2f11e2d97 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:40:49.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.820+0000 7f242121b640 1 -- 172.21.15.83:0/1860243620 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24140b2070 con 0x7f241c06b2f0 2024-04-17T10:40:49.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.820+0000 7f242121b640 1 -- 172.21.15.83:0/1860243620 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24140a9070 con 0x7f241c06b2f0 2024-04-17T10:40:49.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.820+0000 7f242121b640 1 -- 172.21.15.83:0/1860243620 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24140ad880 con 0x7f241c06b2f0 2024-04-17T10:40:49.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.820+0000 7f2424481640 1 -- 172.21.15.83:0/1860243620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241c06b2f0 msgr2=0x7f241c166880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:40:49.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.820+0000 7f2424481640 1 --2- 172.21.15.83:0/1860243620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241c06b2f0 0x7f241c166880 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2414009870 tx=0x7f241401c560 comp rx=0 tx=0).stop 2024-04-17T10:40:49.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.820+0000 7f2424481640 1 -- 172.21.15.83:0/1860243620 shutdown_connections 2024-04-17T10:40:49.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.820+0000 7f2424481640 1 --2- 172.21.15.83:0/1860243620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241c06b2f0 0x7f241c166880 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.820+0000 7f2424481640 1 -- 172.21.15.83:0/1860243620 >> 172.21.15.83:0/1860243620 conn(0x7f241c0cc050 msgr2=0x7f241c0cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:40:49.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.820+0000 7f2424481640 1 -- 172.21.15.83:0/1860243620 shutdown_connections 2024-04-17T10:40:49.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.820+0000 7f2424481640 1 -- 172.21.15.83:0/1860243620 wait complete. 2024-04-17T10:40:49.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.824+0000 7f2424481640 1 Processor -- start 2024-04-17T10:40:49.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.824+0000 7f2424481640 1 -- start start 2024-04-17T10:40:49.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.824+0000 7f2424481640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f241c06b2f0 0x7f241c150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.824+0000 7f2424481640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f241c0aedf0 0x7f241c155100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.824+0000 7f2424481640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241c15cb40 0x7f241c15ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.824+0000 7f2424481640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241c1697f0 con 0x7f241c15cb40 2024-04-17T10:40:49.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.824+0000 7f2424481640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241c071290 con 0x7f241c0aedf0 2024-04-17T10:40:49.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.824+0000 7f2424481640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f241c169970 con 0x7f241c06b2f0 2024-04-17T10:40:49.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.824+0000 7f2422a1e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241c15cb40 0x7f241c15ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.824+0000 7f2422a1e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241c15cb40 0x7f241c15ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35768/0 (socket says 172.21.15.83:35768) 2024-04-17T10:40:49.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.824+0000 7f2422a1e640 1 -- 172.21.15.83:0/634155655 learned_addr learned my addr 172.21.15.83:0/634155655 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:40:49.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.824+0000 7f2421a1c640 1 --2- 172.21.15.83:0/634155655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f241c0aedf0 0x7f241c155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.824+0000 7f242221d640 1 --2- 172.21.15.83:0/634155655 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f241c06b2f0 0x7f241c150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.824+0000 7f2422a1e640 1 -- 172.21.15.83:0/634155655 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f241c06b2f0 msgr2=0x7f241c150f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:40:49.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.824+0000 7f2422a1e640 1 --2- 172.21.15.83:0/634155655 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f241c06b2f0 0x7f241c150f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.824+0000 7f2422a1e640 1 -- 172.21.15.83:0/634155655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f241c0aedf0 msgr2=0x7f241c155100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:40:49.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.824+0000 7f2422a1e640 1 --2- 172.21.15.83:0/634155655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f241c0aedf0 0x7f241c155100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.824+0000 7f2422a1e640 1 -- 172.21.15.83:0/634155655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f241c158110 con 0x7f241c15cb40 2024-04-17T10:40:49.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.824+0000 7f242221d640 1 --2- 172.21.15.83:0/634155655 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f241c06b2f0 0x7f241c150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:40:49.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.824+0000 7f2422a1e640 1 --2- 172.21.15.83:0/634155655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241c15cb40 0x7f241c15ae90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2414009870 tx=0x7f241401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:40:49.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.824+0000 7f241affd640 1 -- 172.21.15.83:0/634155655 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24140bc070 con 0x7f241c15cb40 2024-04-17T10:40:49.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.824+0000 7f241affd640 1 -- 172.21.15.83:0/634155655 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24140adce0 con 0x7f241c15cb40 2024-04-17T10:40:49.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.824+0000 7f2424481640 1 -- 172.21.15.83:0/634155655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f241c1583f0 con 0x7f241c15cb40 2024-04-17T10:40:49.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.824+0000 7f241affd640 1 -- 172.21.15.83:0/634155655 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24140b2040 con 0x7f241c15cb40 2024-04-17T10:40:49.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.824+0000 7f2424481640 1 -- 172.21.15.83:0/634155655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f241c1589b0 con 0x7f241c15cb40 2024-04-17T10:40:49.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f241affd640 1 -- 172.21.15.83:0/634155655 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f24140ab070 con 0x7f241c15cb40 2024-04-17T10:40:49.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f241affd640 1 --2- 172.21.15.83:0/634155655 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2408066f60 0x7f2408069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f241affd640 1 -- 172.21.15.83:0/634155655 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f241411efb0 con 0x7f241c15cb40 2024-04-17T10:40:49.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f242221d640 1 --2- 172.21.15.83:0/634155655 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2408066f60 0x7f2408069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f242221d640 1 --2- 172.21.15.83:0/634155655 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2408066f60 0x7f2408069420 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f240c007fd0 tx=0x7f240c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:40:49.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f24275ee000 1 Processor -- start 2024-04-17T10:40:49.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f24275ee000 1 -- start start 2024-04-17T10:40:49.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f24275ee000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56286d5b2520 0x56286d5b2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f24275ee000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56286d5b2e40 0x56286d569430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f24275ee000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286d569970 0x56286d569d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f24275ee000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286d3b6d00 con 0x56286d569970 2024-04-17T10:40:49.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f24275ee000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286d33edf0 con 0x56286d5b2520 2024-04-17T10:40:49.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f24275ee000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x56286d3b2da0 con 0x56286d5b2e40 2024-04-17T10:40:49.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f2422a1e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286d569970 0x56286d569d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f2422a1e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286d569970 0x56286d569d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35782/0 (socket says 172.21.15.83:35782) 2024-04-17T10:40:49.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f2422a1e640 1 -- 172.21.15.83:0/2698937845 learned_addr learned my addr 172.21.15.83:0/2698937845 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:40:49.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f242221d640 1 --2- 172.21.15.83:0/2698937845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56286d5b2520 0x56286d5b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f2421a1c640 1 --2- 172.21.15.83:0/2698937845 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56286d5b2e40 0x56286d569430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f2422a1e640 1 -- 172.21.15.83:0/2698937845 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56286d5b2e40 msgr2=0x56286d569430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:40:49.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f2422a1e640 1 --2- 172.21.15.83:0/2698937845 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56286d5b2e40 0x56286d569430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f2422a1e640 1 -- 172.21.15.83:0/2698937845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56286d5b2520 msgr2=0x56286d5b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:40:49.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f2422a1e640 1 --2- 172.21.15.83:0/2698937845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56286d5b2520 0x56286d5b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f2422a1e640 1 -- 172.21.15.83:0/2698937845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56286d371580 con 0x56286d569970 2024-04-17T10:40:49.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f242221d640 1 --2- 172.21.15.83:0/2698937845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56286d5b2520 0x56286d5b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:40:49.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f2422a1e640 1 --2- 172.21.15.83:0/2698937845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286d569970 0x56286d569d50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f24140f7c50 tx=0x7f24140ef3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0e09caff147568 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:40:49.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f2403fff640 1 -- 172.21.15.83:0/2698937845 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24140efa70 con 0x56286d569970 2024-04-17T10:40:49.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f2403fff640 1 -- 172.21.15.83:0/2698937845 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24140a9040 con 0x56286d569970 2024-04-17T10:40:49.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f2403fff640 1 -- 172.21.15.83:0/2698937845 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24140f3bc0 con 0x56286d569970 2024-04-17T10:40:49.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f24275ee000 1 -- 172.21.15.83:0/2698937845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286d569970 msgr2=0x56286d569d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:40:49.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f24275ee000 1 --2- 172.21.15.83:0/2698937845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286d569970 0x56286d569d50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f24140f7c50 tx=0x7f24140ef3d0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f24275ee000 1 -- 172.21.15.83:0/2698937845 shutdown_connections 2024-04-17T10:40:49.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f24275ee000 1 --2- 172.21.15.83:0/2698937845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286d569970 0x56286d569d50 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f24275ee000 1 --2- 172.21.15.83:0/2698937845 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56286d5b2e40 0x56286d569430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f24275ee000 1 --2- 172.21.15.83:0/2698937845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56286d5b2520 0x56286d5b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f24275ee000 1 -- 172.21.15.83:0/2698937845 >> 172.21.15.83:0/2698937845 conn(0x56286d450c10 msgr2=0x56286d567d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:40:49.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f24275ee000 1 -- 172.21.15.83:0/2698937845 shutdown_connections 2024-04-17T10:40:49.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.828+0000 7f24275ee000 1 -- 172.21.15.83:0/2698937845 wait complete. 2024-04-17T10:40:49.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24275ee000 1 -- start start 2024-04-17T10:40:49.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24275ee000 1 Processor -- start 2024-04-17T10:40:49.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24275ee000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56286d569970 0x56286d5b8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24275ee000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56286d5b2520 0x56286d5b9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24275ee000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286d5b2e40 0x56286d56de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24275ee000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286d34f6d0 con 0x56286d5b2e40 2024-04-17T10:40:49.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24275ee000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286d371950 con 0x56286d569970 2024-04-17T10:40:49.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24275ee000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x56286d3b6010 con 0x56286d5b2520 2024-04-17T10:40:49.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24275ee000 10 client.0 _ll_register_callbacks cb 0x56286d555030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:40:49.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f2422a1e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286d5b2e40 0x56286d56de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f2422a1e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286d5b2e40 0x56286d56de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35790/0 (socket says 172.21.15.83:35790) 2024-04-17T10:40:49.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f2422a1e640 1 -- 172.21.15.83:0/2957340833 learned_addr learned my addr 172.21.15.83:0/2957340833 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:40:49.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f2421a1c640 1 --2- 172.21.15.83:0/2957340833 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56286d5b2520 0x56286d5b9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f242221d640 1 --2- 172.21.15.83:0/2957340833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56286d569970 0x56286d5b8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f2422a1e640 1 -- 172.21.15.83:0/2957340833 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56286d5b2520 msgr2=0x56286d5b9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:40:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f2422a1e640 1 --2- 172.21.15.83:0/2957340833 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56286d5b2520 0x56286d5b9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f2422a1e640 1 -- 172.21.15.83:0/2957340833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56286d569970 msgr2=0x56286d5b8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:40:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f2422a1e640 1 --2- 172.21.15.83:0/2957340833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56286d569970 0x56286d5b8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f2422a1e640 1 -- 172.21.15.83:0/2957340833 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56286d353d40 con 0x56286d5b2e40 2024-04-17T10:40:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f242221d640 1 --2- 172.21.15.83:0/2957340833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56286d569970 0x56286d5b8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:40:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f2422a1e640 1 --2- 172.21.15.83:0/2957340833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286d5b2e40 0x56286d56de70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2414097e30 tx=0x7f2414019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b44854009fe74fc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:40:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24017fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:40:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24017fa640 1 -- 172.21.15.83:0/2957340833 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241411d2a0 con 0x56286d5b2e40 2024-04-17T10:40:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24017fa640 1 -- 172.21.15.83:0/2957340833 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24140eeec0 con 0x56286d5b2e40 2024-04-17T10:40:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24275ee000 1 -- 172.21.15.83:0/2957340833 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24140a9040 con 0x56286d5b2e40 2024-04-17T10:40:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24017fa640 1 -- 172.21.15.83:0/2957340833 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24140efa70 con 0x56286d5b2e40 2024-04-17T10:40:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24275ee000 10 client.5415 fetch_fsmap learned FSMap version 14 2024-04-17T10:40:49.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24275ee000 1 -- 172.21.15.83:0/2957340833 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x56286d3b1cf0 con 0x56286d5b2e40 2024-04-17T10:40:49.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24017fa640 1 -- 172.21.15.83:0/2957340833 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24140a9040 con 0x56286d5b2e40 2024-04-17T10:40:49.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24017fa640 1 -- 172.21.15.83:0/2957340833 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 14) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f24140b5030 con 0x56286d5b2e40 2024-04-17T10:40:49.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24275ee000 10 client.5415 fetch_fsmap finished waiting for FSMap version 14 2024-04-17T10:40:49.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24275ee000 10 client.5415 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-17T10:40:49.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24275ee000 20 client.5415 populate_metadata read hostname 'smithi083' 2024-04-17T10:40:49.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24275ee000 1 --2- 172.21.15.83:0/2957340833 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x56286d56ec10 0x56286d56eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24275ee000 4 client.5415 mds_command: new command op to 5172 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:40:49.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24275ee000 1 -- 172.21.15.83:0/2957340833 --> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56286d3529d0 con 0x56286d56ec10 2024-04-17T10:40:49.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f242221d640 1 --2- 172.21.15.83:0/2957340833 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x56286d56ec10 0x56286d56eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24017fa640 10 client.5415 ms_handle_connect on v2:172.21.15.83:6836/1181902025 2024-04-17T10:40:49.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f242221d640 1 --2- 172.21.15.83:0/2957340833 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x56286d56ec10 0x56286d56eff0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b15ad16ebd831ad server_cookie=fd38e7fdd12ddf4d in_seq=0 out_seq=0 2024-04-17T10:40:49.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24017fa640 1 -- 172.21.15.83:0/2957340833 <== mds.0 v2:172.21.15.83:6836/1181902025 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56286d3529d0 con 0x56286d56ec10 2024-04-17T10:40:49.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24017fa640 10 client.5415 handle_command_reply: tid=0 2024-04-17T10:40:49.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24275ee000 1 client.5415 shutdown 2024-04-17T10:40:49.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24275ee000 1 -- 172.21.15.83:0/2957340833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286d5b2e40 msgr2=0x56286d56de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:40:49.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24275ee000 1 --2- 172.21.15.83:0/2957340833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286d5b2e40 0x56286d56de70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2414097e30 tx=0x7f2414019300 comp rx=0 tx=0).stop 2024-04-17T10:40:49.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24275ee000 1 -- 172.21.15.83:0/2957340833 shutdown_connections 2024-04-17T10:40:49.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24275ee000 1 --2- 172.21.15.83:0/2957340833 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x56286d56ec10 0x56286d56eff0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24275ee000 1 --2- 172.21.15.83:0/2957340833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286d5b2e40 0x56286d56de70 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.832+0000 7f24275ee000 1 --2- 172.21.15.83:0/2957340833 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56286d5b2520 0x56286d5b9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.836+0000 7f24275ee000 1 --2- 172.21.15.83:0/2957340833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56286d569970 0x56286d5b8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.836+0000 7f24017fa640 0 client.5415 ms_handle_reset on v2:172.21.15.83:6836/1181902025 2024-04-17T10:40:49.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.836+0000 7f24275ee000 1 -- 172.21.15.83:0/2957340833 >> 172.21.15.83:0/2957340833 conn(0x56286d450c10 msgr2=0x56286d568420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:40:49.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.836+0000 7f24275ee000 1 -- 172.21.15.83:0/2957340833 shutdown_connections 2024-04-17T10:40:49.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.836+0000 7f24275ee000 1 -- 172.21.15.83:0/2957340833 wait complete. 2024-04-17T10:40:49.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.836+0000 7f24275ee000 20 client.5415 trim_cache size 0 max 16384 2024-04-17T10:40:49.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.848+0000 7f24275ee000 1 Processor -- start 2024-04-17T10:40:49.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.848+0000 7f24275ee000 1 -- start start 2024-04-17T10:40:49.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.848+0000 7f24275ee000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286d569970 0x56286d616210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.848+0000 7f24275ee000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56286d5b2520 0x56286d616750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.848+0000 7f24275ee000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56286d5b2e40 0x56286d616c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.848+0000 7f24275ee000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241c200940 con 0x56286d569970 2024-04-17T10:40:49.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.848+0000 7f24275ee000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241c157850 con 0x56286d5b2e40 2024-04-17T10:40:49.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.848+0000 7f24275ee000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f241c1524f0 con 0x56286d5b2520 2024-04-17T10:40:49.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f242221d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286d569970 0x56286d616210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f242221d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286d569970 0x56286d616210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35796/0 (socket says 172.21.15.83:35796) 2024-04-17T10:40:49.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f242221d640 1 -- 172.21.15.83:0/3062013237 learned_addr learned my addr 172.21.15.83:0/3062013237 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:40:49.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f2422a1e640 1 --2- 172.21.15.83:0/3062013237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56286d5b2e40 0x56286d616c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f242221d640 1 -- 172.21.15.83:0/3062013237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56286d5b2520 msgr2=0x56286d616750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:40:49.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f242221d640 1 --2- 172.21.15.83:0/3062013237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56286d5b2520 0x56286d616750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f242221d640 1 -- 172.21.15.83:0/3062013237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56286d5b2e40 msgr2=0x56286d616c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:40:49.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f242221d640 1 --2- 172.21.15.83:0/3062013237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56286d5b2e40 0x56286d616c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f242221d640 1 -- 172.21.15.83:0/3062013237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56286d5b2c70 con 0x56286d569970 2024-04-17T10:40:49.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f2422a1e640 1 --2- 172.21.15.83:0/3062013237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56286d5b2e40 0x56286d616c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:40:49.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f242221d640 1 --2- 172.21.15.83:0/3062013237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286d569970 0x56286d616210 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x56286d33f8a0 tx=0x7f240c017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dca2bfa10f4534f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:40:49.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f24037fe640 1 -- 172.21.15.83:0/3062013237 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240c006b60 con 0x56286d569970 2024-04-17T10:40:49.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f24037fe640 1 -- 172.21.15.83:0/3062013237 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f240c006d00 con 0x56286d569970 2024-04-17T10:40:49.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f24037fe640 1 -- 172.21.15.83:0/3062013237 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240c0052d0 con 0x56286d569970 2024-04-17T10:40:49.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f24275ee000 1 -- 172.21.15.83:0/3062013237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286d569970 msgr2=0x56286d616210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:40:49.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f24275ee000 1 --2- 172.21.15.83:0/3062013237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286d569970 0x56286d616210 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x56286d33f8a0 tx=0x7f240c017040 comp rx=0 tx=0).stop 2024-04-17T10:40:49.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f24275ee000 1 -- 172.21.15.83:0/3062013237 shutdown_connections 2024-04-17T10:40:49.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f24275ee000 1 --2- 172.21.15.83:0/3062013237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56286d5b2e40 0x56286d616c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f24275ee000 1 --2- 172.21.15.83:0/3062013237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56286d5b2520 0x56286d616750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f24275ee000 1 --2- 172.21.15.83:0/3062013237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286d569970 0x56286d616210 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f24275ee000 1 -- 172.21.15.83:0/3062013237 >> 172.21.15.83:0/3062013237 conn(0x56286d450c10 msgr2=0x56286d5b15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:40:49.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f24275ee000 1 -- 172.21.15.83:0/3062013237 shutdown_connections 2024-04-17T10:40:49.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f24275ee000 1 -- 172.21.15.83:0/3062013237 wait complete. 2024-04-17T10:40:49.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f24275ee000 1 -- start start 2024-04-17T10:40:49.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f24275ee000 1 Processor -- start 2024-04-17T10:40:49.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f24275ee000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286d569970 0x56286d61bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f24275ee000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56286d5b2520 0x56286d61e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f24275ee000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56286d5b2e40 0x56286d620820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f24275ee000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286d3b6010 con 0x56286d569970 2024-04-17T10:40:49.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f24275ee000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241c157250 con 0x56286d5b2e40 2024-04-17T10:40:49.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f24275ee000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x56286d342b40 con 0x56286d5b2520 2024-04-17T10:40:49.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f24275ee000 10 client.0 _ll_register_callbacks cb 0x7f24140ade90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:40:49.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f242221d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286d569970 0x56286d61bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f242221d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286d569970 0x56286d61bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35810/0 (socket says 172.21.15.83:35810) 2024-04-17T10:40:49.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f242221d640 1 -- 172.21.15.83:0/439107187 learned_addr learned my addr 172.21.15.83:0/439107187 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:40:49.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f2421a1c640 1 --2- 172.21.15.83:0/439107187 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56286d5b2520 0x56286d61e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f2422a1e640 1 --2- 172.21.15.83:0/439107187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56286d5b2e40 0x56286d620820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f242221d640 1 -- 172.21.15.83:0/439107187 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56286d5b2520 msgr2=0x56286d61e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:40:49.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f242221d640 1 --2- 172.21.15.83:0/439107187 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56286d5b2520 0x56286d61e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f242221d640 1 -- 172.21.15.83:0/439107187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56286d5b2e40 msgr2=0x56286d620820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:40:49.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f242221d640 1 --2- 172.21.15.83:0/439107187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56286d5b2e40 0x56286d620820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f242221d640 1 -- 172.21.15.83:0/439107187 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56286d439d30 con 0x56286d569970 2024-04-17T10:40:49.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f2421a1c640 1 --2- 172.21.15.83:0/439107187 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56286d5b2520 0x56286d61e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:40:49.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f242221d640 1 --2- 172.21.15.83:0/439107187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286d569970 0x56286d61bd80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f240c024010 tx=0x7f240c0013e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e95737b5174fd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:40:49.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f24017fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:40:49.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f24017fa640 1 -- 172.21.15.83:0/439107187 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240c03b060 con 0x56286d569970 2024-04-17T10:40:49.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f24017fa640 1 -- 172.21.15.83:0/439107187 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f240c0052e0 con 0x56286d569970 2024-04-17T10:40:49.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f24275ee000 1 -- 172.21.15.83:0/439107187 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f240c006d00 con 0x56286d569970 2024-04-17T10:40:49.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.852+0000 7f24017fa640 1 -- 172.21.15.83:0/439107187 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f240c041050 con 0x56286d569970 2024-04-17T10:40:49.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.856+0000 7f24275ee000 10 client.5421 fetch_fsmap learned FSMap version 14 2024-04-17T10:40:49.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.856+0000 7f24275ee000 1 -- 172.21.15.83:0/439107187 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x56286d3835a0 con 0x56286d569970 2024-04-17T10:40:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.856+0000 7f24017fa640 1 -- 172.21.15.83:0/439107187 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f240c006d00 con 0x56286d569970 2024-04-17T10:40:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.856+0000 7f24017fa640 1 -- 172.21.15.83:0/439107187 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 14) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f240c0054e0 con 0x56286d569970 2024-04-17T10:40:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.856+0000 7f24275ee000 10 client.5421 fetch_fsmap finished waiting for FSMap version 14 2024-04-17T10:40:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.856+0000 7f24275ee000 10 client.5421 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-17T10:40:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.856+0000 7f24275ee000 20 client.5421 populate_metadata read hostname 'smithi083' 2024-04-17T10:40:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.856+0000 7f24275ee000 1 --2- 172.21.15.83:0/439107187 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x56286d620d60 0x56286d623150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.856+0000 7f24275ee000 4 client.5421 mds_command: new command op to 5172 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "scrub_mdsdir"]}] 2024-04-17T10:40:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.856+0000 7f24275ee000 1 -- 172.21.15.83:0/439107187 --> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "scrub_mdsdir"]}) v1 -- 0x56286d393610 con 0x56286d620d60 2024-04-17T10:40:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.856+0000 7f2421a1c640 1 --2- 172.21.15.83:0/439107187 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x56286d620d60 0x56286d623150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.856+0000 7f24017fa640 10 client.5421 ms_handle_connect on v2:172.21.15.83:6836/1181902025 2024-04-17T10:40:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.856+0000 7f2421a1c640 1 --2- 172.21.15.83:0/439107187 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x56286d620d60 0x56286d623150 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc0c4ea5c7d0ce21 server_cookie=4af59046d3d11b5d in_seq=0 out_seq=0 2024-04-17T10:40:49.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.872+0000 7f24017fa640 1 -- 172.21.15.83:0/439107187 <== mds.0 v2:172.21.15.83:6836/1181902025 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56286d393610 con 0x56286d620d60 2024-04-17T10:40:49.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.872+0000 7f24017fa640 10 client.5421 handle_command_reply: tid=0 2024-04-17T10:40:49.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.872+0000 7f24275ee000 1 client.5421 shutdown 2024-04-17T10:40:49.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.872+0000 7f24275ee000 1 -- 172.21.15.83:0/439107187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286d569970 msgr2=0x56286d61bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:40:49.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.872+0000 7f24275ee000 1 --2- 172.21.15.83:0/439107187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286d569970 0x56286d61bd80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f240c024010 tx=0x7f240c0013e0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.872+0000 7f24275ee000 1 -- 172.21.15.83:0/439107187 shutdown_connections 2024-04-17T10:40:49.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.872+0000 7f24275ee000 1 --2- 172.21.15.83:0/439107187 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x56286d620d60 0x56286d623150 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.872+0000 7f24275ee000 1 --2- 172.21.15.83:0/439107187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56286d5b2e40 0x56286d620820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.872+0000 7f24275ee000 1 --2- 172.21.15.83:0/439107187 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56286d5b2520 0x56286d61e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.872+0000 7f24275ee000 1 --2- 172.21.15.83:0/439107187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286d569970 0x56286d61bd80 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.872+0000 7f24017fa640 0 client.5421 ms_handle_reset on v2:172.21.15.83:6836/1181902025 2024-04-17T10:40:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.872+0000 7f24275ee000 1 -- 172.21.15.83:0/439107187 >> 172.21.15.83:0/439107187 conn(0x56286d450c10 msgr2=0x56286d5b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:40:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.872+0000 7f24275ee000 1 -- 172.21.15.83:0/439107187 shutdown_connections 2024-04-17T10:40:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.872+0000 7f24275ee000 1 -- 172.21.15.83:0/439107187 wait complete. 2024-04-17T10:40:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.872+0000 7f24275ee000 20 client.5421 trim_cache size 0 max 16384 2024-04-17T10:40:49.871 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:40:49.872 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-17T10:40:49.872 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "24d48d4d-ceb4-418b-a131-8c34b0bd0377", 2024-04-17T10:40:49.872 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-17T10:40:49.872 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:40:49.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.872+0000 7f2424481640 1 -- 172.21.15.83:0/634155655 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2408066f60 msgr2=0x7f2408069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:40:49.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.872+0000 7f2424481640 1 --2- 172.21.15.83:0/634155655 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2408066f60 0x7f2408069420 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f240c007fd0 tx=0x7f240c009040 comp rx=0 tx=0).stop 2024-04-17T10:40:49.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.872+0000 7f2424481640 1 -- 172.21.15.83:0/634155655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241c15cb40 msgr2=0x7f241c15ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:40:49.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.872+0000 7f2424481640 1 --2- 172.21.15.83:0/634155655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241c15cb40 0x7f241c15ae90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2414009870 tx=0x7f241401ada0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.872+0000 7f2424481640 1 -- 172.21.15.83:0/634155655 shutdown_connections 2024-04-17T10:40:49.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.872+0000 7f2424481640 1 --2- 172.21.15.83:0/634155655 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2408066f60 0x7f2408069420 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.872+0000 7f2424481640 1 --2- 172.21.15.83:0/634155655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241c15cb40 0x7f241c15ae90 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.872+0000 7f2424481640 1 --2- 172.21.15.83:0/634155655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f241c0aedf0 0x7f241c155100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.872+0000 7f2424481640 1 --2- 172.21.15.83:0/634155655 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f241c06b2f0 0x7f241c150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:49.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.872+0000 7f2424481640 1 -- 172.21.15.83:0/634155655 >> 172.21.15.83:0/634155655 conn(0x7f241c0cc050 msgr2=0x7f241c06d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:40:49.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.872+0000 7f2424481640 1 -- 172.21.15.83:0/634155655 shutdown_connections 2024-04-17T10:40:49.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:49.872+0000 7f2424481640 1 -- 172.21.15.83:0/634155655 wait complete. 2024-04-17T10:40:49.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:40:49.889 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-17T10:40:50.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.024+0000 7fce49332640 1 Processor -- start 2024-04-17T10:40:50.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.024+0000 7fce49332640 1 -- start start 2024-04-17T10:40:50.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.028+0000 7fce49332640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fce4415f1c0 0x7fce440b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:50.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.028+0000 7fce49332640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce440cffe0 0x7fce44165900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:50.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.028+0000 7fce49332640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce440b2280 con 0x7fce440b24d0 2024-04-17T10:40:50.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.028+0000 7fce49332640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce440b29d0 con 0x7fce4415f1c0 2024-04-17T10:40:50.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.028+0000 7fce49332640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce440af250 con 0x7fce440cffe0 2024-04-17T10:40:50.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.028+0000 7fce437fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce440cffe0 0x7fce44165900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:50.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.028+0000 7fce42ffd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fce4415f1c0 0x7fce440b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:50.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.028+0000 7fce437fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce440cffe0 0x7fce44165900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:51554/0 (socket says 172.21.15.83:51554) 2024-04-17T10:40:50.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.028+0000 7fce427fc640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fce440b24d0 0x7fce440cf8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:53150/0 (socket says 172.21.15.83:53150) 2024-04-17T10:40:50.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.028+0000 7fce437fe640 1 -- 172.21.15.83:0/963964006 learned_addr learned my addr 172.21.15.83:0/963964006 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:40:50.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.028+0000 7fce437fe640 1 -- 172.21.15.83:0/963964006 >> v2:172.21.15.167:3300/0 conn(0x7fce4415f1c0 msgr2=0x7fce440b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:40:50.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.028+0000 7fce437fe640 1 --2- 172.21.15.83:0/963964006 >> v2:172.21.15.167:3300/0 conn(0x7fce4415f1c0 0x7fce440b1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:50.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.028+0000 7fce41ffb640 1 -- 172.21.15.83:0/963964006 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 533725852 0 0) 0x7fce440b2280 con 0x7fce440b24d0 2024-04-17T10:40:50.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.028+0000 7fce437fe640 1 -- 172.21.15.83:0/963964006 >> v1:172.21.15.167:6789/0 conn(0x7fce440b24d0 legacy=0x7fce440cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:40:50.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.028+0000 7fce437fe640 1 -- 172.21.15.83:0/963964006 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce440af530 con 0x7fce440cffe0 2024-04-17T10:40:50.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.032+0000 7fce437fe640 1 --2- 172.21.15.83:0/963964006 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce440cffe0 0x7fce44165900 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fce3c009870 tx=0x7fce3c01c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=757cd088d4cc235c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:40:50.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.032+0000 7fce41ffb640 1 -- 172.21.15.83:0/963964006 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c0aa670 con 0x7fce440cffe0 2024-04-17T10:40:50.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.032+0000 7fce41ffb640 1 -- 172.21.15.83:0/963964006 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3c0af070 con 0x7fce440cffe0 2024-04-17T10:40:50.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.032+0000 7fce41ffb640 1 -- 172.21.15.83:0/963964006 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c0aaa30 con 0x7fce440cffe0 2024-04-17T10:40:50.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.032+0000 7fce49332640 1 -- 172.21.15.83:0/963964006 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce440cffe0 msgr2=0x7fce44165900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:40:50.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.032+0000 7fce49332640 1 --2- 172.21.15.83:0/963964006 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce440cffe0 0x7fce44165900 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fce3c009870 tx=0x7fce3c01c530 comp rx=0 tx=0).stop 2024-04-17T10:40:50.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.032+0000 7fce49332640 1 -- 172.21.15.83:0/963964006 shutdown_connections 2024-04-17T10:40:50.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.032+0000 7fce49332640 1 --2- 172.21.15.83:0/963964006 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce440cffe0 0x7fce44165900 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:50.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.032+0000 7fce49332640 1 --2- 172.21.15.83:0/963964006 >> v2:172.21.15.167:3300/0 conn(0x7fce4415f1c0 0x7fce440b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:50.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.032+0000 7fce49332640 1 -- 172.21.15.83:0/963964006 >> 172.21.15.83:0/963964006 conn(0x7fce440b3430 msgr2=0x7fce440b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:40:50.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.032+0000 7fce49332640 1 -- 172.21.15.83:0/963964006 shutdown_connections 2024-04-17T10:40:50.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.032+0000 7fce49332640 1 -- 172.21.15.83:0/963964006 wait complete. 2024-04-17T10:40:50.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.032+0000 7fce49332640 1 Processor -- start 2024-04-17T10:40:50.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.036+0000 7fce49332640 1 -- start start 2024-04-17T10:40:50.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.036+0000 7fce49332640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce440b24d0 0x7fce44156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:50.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.036+0000 7fce49332640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce440cffe0 0x7fce441575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:50.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.036+0000 7fce49332640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce4415f1c0 0x7fce44157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:50.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.036+0000 7fce49332640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce441687e0 con 0x7fce440cffe0 2024-04-17T10:40:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.036+0000 7fce49332640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce44168660 con 0x7fce440b24d0 2024-04-17T10:40:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.036+0000 7fce49332640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce44168960 con 0x7fce4415f1c0 2024-04-17T10:40:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.036+0000 7fce427fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce440cffe0 0x7fce441575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.036+0000 7fce427fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce440cffe0 0x7fce441575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35814/0 (socket says 172.21.15.83:35814) 2024-04-17T10:40:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.036+0000 7fce427fc640 1 -- 172.21.15.83:0/152963072 learned_addr learned my addr 172.21.15.83:0/152963072 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:40:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.036+0000 7fce42ffd640 1 --2- 172.21.15.83:0/152963072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce440b24d0 0x7fce44156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.036+0000 7fce437fe640 1 --2- 172.21.15.83:0/152963072 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce4415f1c0 0x7fce44157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.036+0000 7fce427fc640 1 -- 172.21.15.83:0/152963072 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce4415f1c0 msgr2=0x7fce44157ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:40:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.036+0000 7fce427fc640 1 --2- 172.21.15.83:0/152963072 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce4415f1c0 0x7fce44157ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.036+0000 7fce427fc640 1 -- 172.21.15.83:0/152963072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce440b24d0 msgr2=0x7fce44156610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:40:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.036+0000 7fce427fc640 1 --2- 172.21.15.83:0/152963072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce440b24d0 0x7fce44156610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:50.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.036+0000 7fce427fc640 1 -- 172.21.15.83:0/152963072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce440c73a0 con 0x7fce440cffe0 2024-04-17T10:40:50.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.036+0000 7fce427fc640 1 --2- 172.21.15.83:0/152963072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce440cffe0 0x7fce441575a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fce3800c480 tx=0x7fce3800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:40:50.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.036+0000 7fce2b7fe640 1 -- 172.21.15.83:0/152963072 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce38018070 con 0x7fce440cffe0 2024-04-17T10:40:50.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.036+0000 7fce2b7fe640 1 -- 172.21.15.83:0/152963072 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce380074e0 con 0x7fce440cffe0 2024-04-17T10:40:50.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.036+0000 7fce2b7fe640 1 -- 172.21.15.83:0/152963072 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce380049f0 con 0x7fce440cffe0 2024-04-17T10:40:50.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.036+0000 7fce49332640 1 -- 172.21.15.83:0/152963072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce440c7680 con 0x7fce440cffe0 2024-04-17T10:40:50.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.036+0000 7fce49332640 1 -- 172.21.15.83:0/152963072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce440c7c40 con 0x7fce440cffe0 2024-04-17T10:40:50.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.040+0000 7fce2b7fe640 1 -- 172.21.15.83:0/152963072 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fce38004b90 con 0x7fce440cffe0 2024-04-17T10:40:50.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.040+0000 7fce2b7fe640 1 --2- 172.21.15.83:0/152963072 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fce30066f60 0x7fce30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:50.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.040+0000 7fce2b7fe640 1 -- 172.21.15.83:0/152963072 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce38084380 con 0x7fce440cffe0 2024-04-17T10:40:50.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.040+0000 7fce42ffd640 1 --2- 172.21.15.83:0/152963072 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fce30066f60 0x7fce30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:50.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.040+0000 7fce42ffd640 1 --2- 172.21.15.83:0/152963072 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fce30066f60 0x7fce30069420 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fce44154f90 tx=0x7fce3400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:40:50.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.040+0000 7fce4c49f000 1 Processor -- start 2024-04-17T10:40:50.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.040+0000 7fce4c49f000 1 -- start start 2024-04-17T10:40:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.040+0000 7fce4c49f000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561098e68220 0x561098e68600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.040+0000 7fce4c49f000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561098e68b40 0x561098e15b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.040+0000 7fce4c49f000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561098e160a0 0x561098e16480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.040+0000 7fce4c49f000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x561098c62ff0 con 0x561098e160a0 2024-04-17T10:40:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.040+0000 7fce4c49f000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561098beb150 con 0x561098e68b40 2024-04-17T10:40:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.040+0000 7fce4c49f000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x561098c5f090 con 0x561098e68220 2024-04-17T10:40:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.040+0000 7fce437fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561098e160a0 0x561098e16480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.040+0000 7fce437fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561098e160a0 0x561098e16480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35820/0 (socket says 172.21.15.83:35820) 2024-04-17T10:40:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.040+0000 7fce437fe640 1 -- 172.21.15.83:0/615376957 learned_addr learned my addr 172.21.15.83:0/615376957 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:40:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.040+0000 7fce42ffd640 1 --2- 172.21.15.83:0/615376957 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561098e68220 0x561098e68600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.040+0000 7fce427fc640 1 --2- 172.21.15.83:0/615376957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561098e68b40 0x561098e15b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.040+0000 7fce437fe640 1 -- 172.21.15.83:0/615376957 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561098e68220 msgr2=0x561098e68600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:40:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.040+0000 7fce437fe640 1 --2- 172.21.15.83:0/615376957 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561098e68220 0x561098e68600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.040+0000 7fce437fe640 1 -- 172.21.15.83:0/615376957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561098e68b40 msgr2=0x561098e15b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:40:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.040+0000 7fce437fe640 1 --2- 172.21.15.83:0/615376957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561098e68b40 0x561098e15b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.040+0000 7fce437fe640 1 -- 172.21.15.83:0/615376957 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561098df90d0 con 0x561098e160a0 2024-04-17T10:40:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.040+0000 7fce437fe640 1 --2- 172.21.15.83:0/615376957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561098e160a0 0x561098e16480 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fce3c01a160 tx=0x7fce3c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297b7db4af066186 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:40:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.044+0000 7fce28ff9640 1 -- 172.21.15.83:0/615376957 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c0b9070 con 0x561098e160a0 2024-04-17T10:40:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.044+0000 7fce28ff9640 1 -- 172.21.15.83:0/615376957 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3c0af040 con 0x561098e160a0 2024-04-17T10:40:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.044+0000 7fce28ff9640 1 -- 172.21.15.83:0/615376957 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c0aaaa0 con 0x561098e160a0 2024-04-17T10:40:50.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.044+0000 7fce4c49f000 1 -- 172.21.15.83:0/615376957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561098e160a0 msgr2=0x561098e16480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:40:50.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.044+0000 7fce4c49f000 1 --2- 172.21.15.83:0/615376957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561098e160a0 0x561098e16480 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fce3c01a160 tx=0x7fce3c004740 comp rx=0 tx=0).stop 2024-04-17T10:40:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.044+0000 7fce4c49f000 1 -- 172.21.15.83:0/615376957 shutdown_connections 2024-04-17T10:40:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.044+0000 7fce4c49f000 1 --2- 172.21.15.83:0/615376957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561098e160a0 0x561098e16480 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.044+0000 7fce4c49f000 1 --2- 172.21.15.83:0/615376957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561098e68b40 0x561098e15b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.044+0000 7fce4c49f000 1 --2- 172.21.15.83:0/615376957 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561098e68220 0x561098e68600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.044+0000 7fce4c49f000 1 -- 172.21.15.83:0/615376957 >> 172.21.15.83:0/615376957 conn(0x561098e12020 msgr2=0x561098e14440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:40:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.044+0000 7fce4c49f000 1 -- 172.21.15.83:0/615376957 shutdown_connections 2024-04-17T10:40:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.044+0000 7fce4c49f000 1 -- 172.21.15.83:0/615376957 wait complete. 2024-04-17T10:40:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.044+0000 7fce4c49f000 1 -- start start 2024-04-17T10:40:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.044+0000 7fce4c49f000 1 Processor -- start 2024-04-17T10:40:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.044+0000 7fce4c49f000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561098e160a0 0x561098e60380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.044+0000 7fce4c49f000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561098e68220 0x561098e608c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.044+0000 7fce4c49f000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561098e68b40 0x561098e671e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.044+0000 7fce4c49f000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x561098c27250 con 0x561098e68220 2024-04-17T10:40:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.044+0000 7fce4c49f000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561098c1d690 con 0x561098e68b40 2024-04-17T10:40:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.044+0000 7fce4c49f000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x561098beba40 con 0x561098e160a0 2024-04-17T10:40:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.044+0000 7fce4c49f000 10 client.0 _ll_register_callbacks cb 0x561098e00e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:40:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.044+0000 7fce427fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561098e68220 0x561098e608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.044+0000 7fce427fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561098e68220 0x561098e608c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35824/0 (socket says 172.21.15.83:35824) 2024-04-17T10:40:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.044+0000 7fce427fc640 1 -- 172.21.15.83:0/2763617329 learned_addr learned my addr 172.21.15.83:0/2763617329 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:40:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.044+0000 7fce42ffd640 1 --2- 172.21.15.83:0/2763617329 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561098e160a0 0x561098e60380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.044+0000 7fce437fe640 1 --2- 172.21.15.83:0/2763617329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561098e68b40 0x561098e671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.044+0000 7fce427fc640 1 -- 172.21.15.83:0/2763617329 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561098e160a0 msgr2=0x561098e60380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:40:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.044+0000 7fce427fc640 1 --2- 172.21.15.83:0/2763617329 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561098e160a0 0x561098e60380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.044+0000 7fce427fc640 1 -- 172.21.15.83:0/2763617329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561098e68b40 msgr2=0x561098e671e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:40:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.044+0000 7fce427fc640 1 --2- 172.21.15.83:0/2763617329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561098e68b40 0x561098e671e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.044+0000 7fce427fc640 1 -- 172.21.15.83:0/2763617329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561098d101c0 con 0x561098e68220 2024-04-17T10:40:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.044+0000 7fce42ffd640 1 --2- 172.21.15.83:0/2763617329 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561098e160a0 0x561098e60380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:40:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce427fc640 1 --2- 172.21.15.83:0/2763617329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561098e68220 0x561098e608c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fce3805ce80 tx=0x7fce38054ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ee2d5e5a11d47b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:40:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:40:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce21ffb640 1 -- 172.21.15.83:0/2763617329 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3805d3b0 con 0x561098e68220 2024-04-17T10:40:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce4c49f000 1 -- 172.21.15.83:0/2763617329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce3c0af040 con 0x561098e68220 2024-04-17T10:40:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce21ffb640 1 -- 172.21.15.83:0/2763617329 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce38052fa0 con 0x561098e68220 2024-04-17T10:40:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce21ffb640 1 -- 172.21.15.83:0/2763617329 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce38051310 con 0x561098e68220 2024-04-17T10:40:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce4c49f000 10 client.5430 fetch_fsmap learned FSMap version 14 2024-04-17T10:40:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce4c49f000 1 -- 172.21.15.83:0/2763617329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x561098d0c170 con 0x561098e68220 2024-04-17T10:40:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce21ffb640 1 -- 172.21.15.83:0/2763617329 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3c0af040 con 0x561098e68220 2024-04-17T10:40:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce21ffb640 1 -- 172.21.15.83:0/2763617329 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 14) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fce380540f0 con 0x561098e68220 2024-04-17T10:40:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce4c49f000 10 client.5430 fetch_fsmap finished waiting for FSMap version 14 2024-04-17T10:40:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce4c49f000 10 client.5430 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-17T10:40:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce4c49f000 20 client.5430 populate_metadata read hostname 'smithi083' 2024-04-17T10:40:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce4c49f000 1 --2- 172.21.15.83:0/2763617329 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x561098e6b300 0x561098e6b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce4c49f000 4 client.5430 mds_command: new command op to 5172 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:40:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce4c49f000 1 -- 172.21.15.83:0/2763617329 --> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561098bfec60 con 0x561098e6b300 2024-04-17T10:40:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce42ffd640 1 --2- 172.21.15.83:0/2763617329 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x561098e6b300 0x561098e6b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce21ffb640 10 client.5430 ms_handle_connect on v2:172.21.15.83:6836/1181902025 2024-04-17T10:40:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce42ffd640 1 --2- 172.21.15.83:0/2763617329 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x561098e6b300 0x561098e6b710 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aba3c66e6c0849e server_cookie=938f3ff3f524bb6f in_seq=0 out_seq=0 2024-04-17T10:40:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce21ffb640 1 -- 172.21.15.83:0/2763617329 <== mds.0 v2:172.21.15.83:6836/1181902025 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561098bfec60 con 0x561098e6b300 2024-04-17T10:40:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce21ffb640 10 client.5430 handle_command_reply: tid=0 2024-04-17T10:40:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce4c49f000 1 client.5430 shutdown 2024-04-17T10:40:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce4c49f000 1 -- 172.21.15.83:0/2763617329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561098e68220 msgr2=0x561098e608c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:40:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce4c49f000 1 --2- 172.21.15.83:0/2763617329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561098e68220 0x561098e608c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fce3805ce80 tx=0x7fce38054ca0 comp rx=0 tx=0).stop 2024-04-17T10:40:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce4c49f000 1 -- 172.21.15.83:0/2763617329 shutdown_connections 2024-04-17T10:40:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce4c49f000 1 --2- 172.21.15.83:0/2763617329 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x561098e6b300 0x561098e6b710 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce4c49f000 1 --2- 172.21.15.83:0/2763617329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561098e68b40 0x561098e671e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce4c49f000 1 --2- 172.21.15.83:0/2763617329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561098e68220 0x561098e608c0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce4c49f000 1 --2- 172.21.15.83:0/2763617329 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561098e160a0 0x561098e60380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce4c49f000 1 -- 172.21.15.83:0/2763617329 >> 172.21.15.83:0/2763617329 conn(0x561098e12020 msgr2=0x561098e14ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:40:50.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce21ffb640 0 client.5430 ms_handle_reset on v2:172.21.15.83:6836/1181902025 2024-04-17T10:40:50.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce4c49f000 1 -- 172.21.15.83:0/2763617329 shutdown_connections 2024-04-17T10:40:50.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce4c49f000 1 -- 172.21.15.83:0/2763617329 wait complete. 2024-04-17T10:40:50.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.048+0000 7fce4c49f000 20 client.5430 trim_cache size 0 max 16384 2024-04-17T10:40:50.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.072+0000 7fce4c49f000 1 Processor -- start 2024-04-17T10:40:50.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.072+0000 7fce4c49f000 1 -- start start 2024-04-17T10:40:50.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.072+0000 7fce4c49f000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561098e160a0 0x561098e87650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:50.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.072+0000 7fce4c49f000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561098e68220 0x561098e661d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:50.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.072+0000 7fce4c49f000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561098e68b40 0x561098e66710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:50.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.072+0000 7fce4c49f000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4406dab0 con 0x561098e68b40 2024-04-17T10:40:50.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.072+0000 7fce4c49f000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce440b1030 con 0x561098e68220 2024-04-17T10:40:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.072+0000 7fce4c49f000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce44155f10 con 0x561098e160a0 2024-04-17T10:40:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.072+0000 7fce437fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561098e68b40 0x561098e66710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.072+0000 7fce437fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561098e68b40 0x561098e66710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35836/0 (socket says 172.21.15.83:35836) 2024-04-17T10:40:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.072+0000 7fce437fe640 1 -- 172.21.15.83:0/3703008357 learned_addr learned my addr 172.21.15.83:0/3703008357 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:40:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.072+0000 7fce427fc640 1 --2- 172.21.15.83:0/3703008357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561098e68220 0x561098e661d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.072+0000 7fce42ffd640 1 --2- 172.21.15.83:0/3703008357 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561098e160a0 0x561098e87650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.076+0000 7fce437fe640 1 -- 172.21.15.83:0/3703008357 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561098e160a0 msgr2=0x561098e87650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:40:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.076+0000 7fce437fe640 1 --2- 172.21.15.83:0/3703008357 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561098e160a0 0x561098e87650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.076+0000 7fce437fe640 1 -- 172.21.15.83:0/3703008357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561098e68220 msgr2=0x561098e661d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:40:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.076+0000 7fce437fe640 1 --2- 172.21.15.83:0/3703008357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561098e68220 0x561098e661d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.076+0000 7fce437fe640 1 -- 172.21.15.83:0/3703008357 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561098ca9de0 con 0x561098e68b40 2024-04-17T10:40:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.076+0000 7fce427fc640 1 --2- 172.21.15.83:0/3703008357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561098e68220 0x561098e661d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:40:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.076+0000 7fce437fe640 1 --2- 172.21.15.83:0/3703008357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561098e68b40 0x561098e66710 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fce3c01a160 tx=0x7fce3c009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a79c0cc75190e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:40:50.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.076+0000 7fce23fff640 1 -- 172.21.15.83:0/3703008357 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c0b9070 con 0x561098e68b40 2024-04-17T10:40:50.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.076+0000 7fce23fff640 1 -- 172.21.15.83:0/3703008357 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3c0af040 con 0x561098e68b40 2024-04-17T10:40:50.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.076+0000 7fce23fff640 1 -- 172.21.15.83:0/3703008357 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c0044f0 con 0x561098e68b40 2024-04-17T10:40:50.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.076+0000 7fce4c49f000 1 -- 172.21.15.83:0/3703008357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561098e68b40 msgr2=0x561098e66710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:40:50.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.076+0000 7fce4c49f000 1 --2- 172.21.15.83:0/3703008357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561098e68b40 0x561098e66710 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fce3c01a160 tx=0x7fce3c009240 comp rx=0 tx=0).stop 2024-04-17T10:40:50.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.076+0000 7fce4c49f000 1 -- 172.21.15.83:0/3703008357 shutdown_connections 2024-04-17T10:40:50.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.076+0000 7fce4c49f000 1 --2- 172.21.15.83:0/3703008357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561098e68b40 0x561098e66710 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:50.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.076+0000 7fce4c49f000 1 --2- 172.21.15.83:0/3703008357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561098e68220 0x561098e661d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:50.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.076+0000 7fce4c49f000 1 --2- 172.21.15.83:0/3703008357 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561098e160a0 0x561098e87650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:50.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.076+0000 7fce4c49f000 1 -- 172.21.15.83:0/3703008357 >> 172.21.15.83:0/3703008357 conn(0x561098e12020 msgr2=0x561098e42470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:40:50.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.076+0000 7fce4c49f000 1 -- 172.21.15.83:0/3703008357 shutdown_connections 2024-04-17T10:40:50.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.076+0000 7fce4c49f000 1 -- 172.21.15.83:0/3703008357 wait complete. 2024-04-17T10:40:50.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.076+0000 7fce4c49f000 1 -- start start 2024-04-17T10:40:50.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.076+0000 7fce4c49f000 1 Processor -- start 2024-04-17T10:40:50.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce4c49f000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561098e160a0 0x561098e83610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:50.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce4c49f000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561098e68220 0x561098e83b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:50.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce4c49f000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561098e68b40 0x561098e84090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:50.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce4c49f000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x561098bfde00 con 0x561098e68220 2024-04-17T10:40:50.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce4c49f000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce440b0a30 con 0x561098e68b40 2024-04-17T10:40:50.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce4c49f000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x561098c62300 con 0x561098e160a0 2024-04-17T10:40:50.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce4c49f000 10 client.0 _ll_register_callbacks cb 0x561098de1040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:40:50.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce427fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561098e68220 0x561098e83b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:50.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce427fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561098e68220 0x561098e83b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35850/0 (socket says 172.21.15.83:35850) 2024-04-17T10:40:50.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce427fc640 1 -- 172.21.15.83:0/171495541 learned_addr learned my addr 172.21.15.83:0/171495541 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:40:50.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce42ffd640 1 --2- 172.21.15.83:0/171495541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561098e160a0 0x561098e83610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:50.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce427fc640 1 -- 172.21.15.83:0/171495541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561098e160a0 msgr2=0x561098e83610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:40:50.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce427fc640 1 --2- 172.21.15.83:0/171495541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561098e160a0 0x561098e83610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:50.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce427fc640 1 -- 172.21.15.83:0/171495541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561098e68b40 msgr2=0x561098e84090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:40:50.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce437fe640 1 --2- 172.21.15.83:0/171495541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561098e68b40 0x561098e84090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:50.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce427fc640 1 --2- 172.21.15.83:0/171495541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561098e68b40 0x561098e84090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:50.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce427fc640 1 -- 172.21.15.83:0/171495541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561098c048e0 con 0x561098e68220 2024-04-17T10:40:50.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce437fe640 1 --2- 172.21.15.83:0/171495541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561098e68b40 0x561098e84090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:40:50.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce427fc640 1 --2- 172.21.15.83:0/171495541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561098e68220 0x561098e83b50 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fce38054ce0 tx=0x7fce38082810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d232b364b497bec server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:40:50.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:40:50.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce21ffb640 1 -- 172.21.15.83:0/171495541 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3805d3b0 con 0x561098e68220 2024-04-17T10:40:50.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce21ffb640 1 -- 172.21.15.83:0/171495541 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce38053b60 con 0x561098e68220 2024-04-17T10:40:50.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce4c49f000 1 -- 172.21.15.83:0/171495541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce3c0af040 con 0x561098e68220 2024-04-17T10:40:50.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce21ffb640 1 -- 172.21.15.83:0/171495541 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce38050f40 con 0x561098e68220 2024-04-17T10:40:50.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce4c49f000 10 client.5436 fetch_fsmap learned FSMap version 14 2024-04-17T10:40:50.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce4c49f000 1 -- 172.21.15.83:0/171495541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x561098cbc8f0 con 0x561098e68220 2024-04-17T10:40:50.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce21ffb640 1 -- 172.21.15.83:0/171495541 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3c0af040 con 0x561098e68220 2024-04-17T10:40:50.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce21ffb640 1 -- 172.21.15.83:0/171495541 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 14) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fce38053d60 con 0x561098e68220 2024-04-17T10:40:50.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce4c49f000 10 client.5436 fetch_fsmap finished waiting for FSMap version 14 2024-04-17T10:40:50.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce4c49f000 10 client.5436 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-17T10:40:50.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce4c49f000 20 client.5436 populate_metadata read hostname 'smithi083' 2024-04-17T10:40:50.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce4c49f000 1 --2- 172.21.15.83:0/171495541 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x561098e84b40 0x561098ed8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:40:50.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce4c49f000 4 client.5436 mds_command: new command op to 5172 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T10:40:50.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce4c49f000 1 -- 172.21.15.83:0/171495541 --> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561098c70530 con 0x561098e84b40 2024-04-17T10:40:50.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce42ffd640 1 --2- 172.21.15.83:0/171495541 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x561098e84b40 0x561098ed8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:40:50.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce21ffb640 10 client.5436 ms_handle_connect on v2:172.21.15.83:6836/1181902025 2024-04-17T10:40:50.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce42ffd640 1 --2- 172.21.15.83:0/171495541 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x561098e84b40 0x561098ed8fb0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61598cc4f6ecea66 server_cookie=8cd330d1c709ad0d in_seq=0 out_seq=0 2024-04-17T10:40:50.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce21ffb640 1 -- 172.21.15.83:0/171495541 <== mds.0 v2:172.21.15.83:6836/1181902025 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+274 (crc 0 0 0) 0x561098c70530 con 0x561098e84b40 2024-04-17T10:40:50.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce21ffb640 10 client.5436 handle_command_reply: tid=0 2024-04-17T10:40:50.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce4c49f000 1 client.5436 shutdown 2024-04-17T10:40:50.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce4c49f000 1 -- 172.21.15.83:0/171495541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561098e68220 msgr2=0x561098e83b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:40:50.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce4c49f000 1 --2- 172.21.15.83:0/171495541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561098e68220 0x561098e83b50 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fce38054ce0 tx=0x7fce38082810 comp rx=0 tx=0).stop 2024-04-17T10:40:50.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce4c49f000 1 -- 172.21.15.83:0/171495541 shutdown_connections 2024-04-17T10:40:50.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce4c49f000 1 --2- 172.21.15.83:0/171495541 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x561098e84b40 0x561098ed8fb0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:50.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce4c49f000 1 --2- 172.21.15.83:0/171495541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561098e68b40 0x561098e84090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:50.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce4c49f000 1 --2- 172.21.15.83:0/171495541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561098e68220 0x561098e83b50 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:50.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce21ffb640 0 client.5436 ms_handle_reset on v2:172.21.15.83:6836/1181902025 2024-04-17T10:40:50.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce4c49f000 1 --2- 172.21.15.83:0/171495541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561098e160a0 0x561098e83610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:50.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce4c49f000 1 -- 172.21.15.83:0/171495541 >> 172.21.15.83:0/171495541 conn(0x561098e12020 msgr2=0x561098e42d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:40:50.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce4c49f000 1 -- 172.21.15.83:0/171495541 shutdown_connections 2024-04-17T10:40:50.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce4c49f000 1 -- 172.21.15.83:0/171495541 wait complete. 2024-04-17T10:40:50.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.080+0000 7fce4c49f000 20 client.5436 trim_cache size 0 max 16384 2024-04-17T10:40:50.084 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:40:50.084 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "scrub active (944 inodes in the stack)", 2024-04-17T10:40:50.084 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-17T10:40:50.084 INFO:teuthology.orchestra.run.smithi083.stdout: "24d48d4d-ceb4-418b-a131-8c34b0bd0377": { 2024-04-17T10:40:50.084 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-17T10:40:50.084 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "24d48d4d-ceb4-418b-a131-8c34b0bd0377", 2024-04-17T10:40:50.084 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,scrub_mdsdir" 2024-04-17T10:40:50.084 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:40:50.084 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:40:50.084 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:40:50.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.084+0000 7fce49332640 1 -- 172.21.15.83:0/152963072 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fce30066f60 msgr2=0x7fce30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:40:50.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.084+0000 7fce49332640 1 --2- 172.21.15.83:0/152963072 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fce30066f60 0x7fce30069420 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fce44154f90 tx=0x7fce3400b040 comp rx=0 tx=0).stop 2024-04-17T10:40:50.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.084+0000 7fce49332640 1 -- 172.21.15.83:0/152963072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce440cffe0 msgr2=0x7fce441575a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:40:50.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.084+0000 7fce49332640 1 --2- 172.21.15.83:0/152963072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce440cffe0 0x7fce441575a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fce3800c480 tx=0x7fce3800c940 comp rx=0 tx=0).stop 2024-04-17T10:40:50.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.084+0000 7fce49332640 1 -- 172.21.15.83:0/152963072 shutdown_connections 2024-04-17T10:40:50.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.084+0000 7fce49332640 1 --2- 172.21.15.83:0/152963072 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fce30066f60 0x7fce30069420 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:50.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.084+0000 7fce49332640 1 --2- 172.21.15.83:0/152963072 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce4415f1c0 0x7fce44157ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:50.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.084+0000 7fce49332640 1 --2- 172.21.15.83:0/152963072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce440cffe0 0x7fce441575a0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:50.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.084+0000 7fce49332640 1 --2- 172.21.15.83:0/152963072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce440b24d0 0x7fce44156610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:40:50.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.084+0000 7fce49332640 1 -- 172.21.15.83:0/152963072 >> 172.21.15.83:0/152963072 conn(0x7fce440b3430 msgr2=0x7fce440d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:40:50.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.084+0000 7fce49332640 1 -- 172.21.15.83:0/152963072 shutdown_connections 2024-04-17T10:40:50.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:40:50.084+0000 7fce49332640 1 -- 172.21.15.83:0/152963072 wait complete. 2024-04-17T10:40:50.100 INFO:tasks.cephfs.filesystem:scrub status for tag:24d48d4d-ceb4-418b-a131-8c34b0bd0377 - {'path': '/', 'tag': '24d48d4d-ceb4-418b-a131-8c34b0bd0377', 'options': 'recursive,scrub_mdsdir'} 2024-04-17T10:41:06.458 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:41:06.461 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:41:20.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:41:20.101 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-17T10:41:20.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.239+0000 7f383f6e5640 1 Processor -- start 2024-04-17T10:41:20.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.239+0000 7f383f6e5640 1 -- start start 2024-04-17T10:41:20.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.239+0000 7f383f6e5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f38380d0970 0x7f38380d17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.239+0000 7f383f6e5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f38380d26c0 0x7f383815d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.239+0000 7f383f6e5640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38380ce940 con 0x7f38380ce4e0 2024-04-17T10:41:20.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.239+0000 7f383f6e5640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38380cf7d0 con 0x7f38380d26c0 2024-04-17T10:41:20.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.239+0000 7f383f6e5640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38380d4890 con 0x7f38380d0970 2024-04-17T10:41:20.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.239+0000 7f383dc82640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f38380d26c0 0x7f383815d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.239+0000 7f383dc82640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f38380d26c0 0x7f383815d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36306/0 (socket says 172.21.15.83:36306) 2024-04-17T10:41:20.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.239+0000 7f383dc82640 1 -- 172.21.15.83:0/2152760096 learned_addr learned my addr 172.21.15.83:0/2152760096 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:20.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.239+0000 7f383cc80640 1 --1- 172.21.15.83:0/2152760096 >> v1:172.21.15.83:6789/0 conn(0x7f38380ce4e0 0x7f38380d1fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53968/0 (socket says 172.21.15.83:53968) 2024-04-17T10:41:20.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.239+0000 7f383d481640 1 --2- 172.21.15.83:0/2152760096 >> v2:172.21.15.167:3300/0 conn(0x7f38380d0970 0x7f38380d17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.239+0000 7f383dc82640 1 -- 172.21.15.83:0/2152760096 >> v2:172.21.15.167:3300/0 conn(0x7f38380d0970 msgr2=0x7f38380d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:20.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.239+0000 7f383dc82640 1 --2- 172.21.15.83:0/2152760096 >> v2:172.21.15.167:3300/0 conn(0x7f38380d0970 0x7f38380d17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.239+0000 7f383dc82640 1 -- 172.21.15.83:0/2152760096 >> v1:172.21.15.83:6789/0 conn(0x7f38380ce4e0 legacy=0x7f38380d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:20.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.239+0000 7f383dc82640 1 -- 172.21.15.83:0/2152760096 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38380cebc0 con 0x7f38380d26c0 2024-04-17T10:41:20.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.243+0000 7f383dc82640 1 --2- 172.21.15.83:0/2152760096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38380d26c0 0x7f383815d620 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f383806af50 tx=0x7f383001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=693c853063aa964 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:20.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.243+0000 7f382ffff640 1 -- 172.21.15.83:0/2152760096 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38300a5880 con 0x7f38380d26c0 2024-04-17T10:41:20.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.243+0000 7f382ffff640 1 -- 172.21.15.83:0/2152760096 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38300a5a20 con 0x7f38380d26c0 2024-04-17T10:41:20.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.243+0000 7f382ffff640 1 -- 172.21.15.83:0/2152760096 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38300a5df0 con 0x7f38380d26c0 2024-04-17T10:41:20.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.243+0000 7f383f6e5640 1 -- 172.21.15.83:0/2152760096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38380d26c0 msgr2=0x7f383815d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:20.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.243+0000 7f383f6e5640 1 --2- 172.21.15.83:0/2152760096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38380d26c0 0x7f383815d620 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f383806af50 tx=0x7f383001c9c0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.243+0000 7f383f6e5640 1 -- 172.21.15.83:0/2152760096 shutdown_connections 2024-04-17T10:41:20.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.243+0000 7f383f6e5640 1 --2- 172.21.15.83:0/2152760096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38380d26c0 0x7f383815d620 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.243+0000 7f383f6e5640 1 --2- 172.21.15.83:0/2152760096 >> v2:172.21.15.167:3300/0 conn(0x7f38380d0970 0x7f38380d17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.243+0000 7f383f6e5640 1 -- 172.21.15.83:0/2152760096 >> 172.21.15.83:0/2152760096 conn(0x7f383809cd50 msgr2=0x7f38380d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:20.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.243+0000 7f383f6e5640 1 -- 172.21.15.83:0/2152760096 shutdown_connections 2024-04-17T10:41:20.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.243+0000 7f383f6e5640 1 -- 172.21.15.83:0/2152760096 wait complete. 2024-04-17T10:41:20.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.243+0000 7f383f6e5640 1 Processor -- start 2024-04-17T10:41:20.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.243+0000 7f383f6e5640 1 -- start start 2024-04-17T10:41:20.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.247+0000 7f383f6e5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38380ce4e0 0x7f3838152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.247+0000 7f383f6e5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38380d0970 0x7f3838160cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.247+0000 7f383f6e5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38380d26c0 0x7f38380cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.247+0000 7f383f6e5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383806b280 con 0x7f38380ce4e0 2024-04-17T10:41:20.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.247+0000 7f383f6e5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38380d0d50 con 0x7f38380d0970 2024-04-17T10:41:20.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.247+0000 7f383f6e5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f383806b400 con 0x7f38380d26c0 2024-04-17T10:41:20.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.247+0000 7f383d481640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38380ce4e0 0x7f3838152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.247+0000 7f383d481640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38380ce4e0 0x7f3838152870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36320/0 (socket says 172.21.15.83:36320) 2024-04-17T10:41:20.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.247+0000 7f383d481640 1 -- 172.21.15.83:0/1315571740 learned_addr learned my addr 172.21.15.83:0/1315571740 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:20.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.247+0000 7f383cc80640 1 --2- 172.21.15.83:0/1315571740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38380d0970 0x7f3838160cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.247+0000 7f383d481640 1 -- 172.21.15.83:0/1315571740 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38380d26c0 msgr2=0x7f38380cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:20.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.247+0000 7f383d481640 1 --2- 172.21.15.83:0/1315571740 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38380d26c0 0x7f38380cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.247+0000 7f383d481640 1 -- 172.21.15.83:0/1315571740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38380d0970 msgr2=0x7f3838160cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:20.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.247+0000 7f383d481640 1 --2- 172.21.15.83:0/1315571740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38380d0970 0x7f3838160cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.247+0000 7f383d481640 1 -- 172.21.15.83:0/1315571740 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3838152db0 con 0x7f38380ce4e0 2024-04-17T10:41:20.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.247+0000 7f383cc80640 1 --2- 172.21.15.83:0/1315571740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38380d0970 0x7f3838160cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:20.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.247+0000 7f383d481640 1 --2- 172.21.15.83:0/1315571740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38380ce4e0 0x7f3838152870 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f382800d800 tx=0x7f382800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:20.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.247+0000 7f382dffb640 1 -- 172.21.15.83:0/1315571740 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382800b6c0 con 0x7f38380ce4e0 2024-04-17T10:41:20.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.247+0000 7f382dffb640 1 -- 172.21.15.83:0/1315571740 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3828019070 con 0x7f38380ce4e0 2024-04-17T10:41:20.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.247+0000 7f383f6e5640 1 -- 172.21.15.83:0/1315571740 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3838155960 con 0x7f38380ce4e0 2024-04-17T10:41:20.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.247+0000 7f382dffb640 1 -- 172.21.15.83:0/1315571740 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382800ba60 con 0x7f38380ce4e0 2024-04-17T10:41:20.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.247+0000 7f383f6e5640 1 -- 172.21.15.83:0/1315571740 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3838155f20 con 0x7f38380ce4e0 2024-04-17T10:41:20.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.247+0000 7f382dffb640 1 -- 172.21.15.83:0/1315571740 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f382800bc00 con 0x7f38380ce4e0 2024-04-17T10:41:20.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.247+0000 7f382dffb640 1 --2- 172.21.15.83:0/1315571740 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f381c066f60 0x7f381c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.247+0000 7f382dffb640 1 -- 172.21.15.83:0/1315571740 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f3828085460 con 0x7f38380ce4e0 2024-04-17T10:41:20.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.247+0000 7f383cc80640 1 --2- 172.21.15.83:0/1315571740 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f381c066f60 0x7f381c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.247+0000 7f383cc80640 1 --2- 172.21.15.83:0/1315571740 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f381c066f60 0x7f381c069420 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3838156ab0 tx=0x7f3834009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:20.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f3842852000 1 Processor -- start 2024-04-17T10:41:20.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f3842852000 1 -- start start 2024-04-17T10:41:20.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f3842852000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557f5efb79d0 0x557f5efb7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f3842852000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f5efb82f0 0x557f5ef6db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f3842852000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5ef6e0a0 0x557f5ef6e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f3842852000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5edbaff0 con 0x557f5ef6e0a0 2024-04-17T10:41:20.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f3842852000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5ed43150 con 0x557f5efb82f0 2024-04-17T10:41:20.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f3842852000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x557f5edb7090 con 0x557f5efb79d0 2024-04-17T10:41:20.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f383dc82640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5ef6e0a0 0x557f5ef6e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f383dc82640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5ef6e0a0 0x557f5ef6e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36326/0 (socket says 172.21.15.83:36326) 2024-04-17T10:41:20.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f383dc82640 1 -- 172.21.15.83:0/2233609342 learned_addr learned my addr 172.21.15.83:0/2233609342 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:20.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f383cc80640 1 --2- 172.21.15.83:0/2233609342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f5efb82f0 0x557f5ef6db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f383dc82640 1 -- 172.21.15.83:0/2233609342 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557f5efb79d0 msgr2=0x557f5efb7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:20.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f383dc82640 1 --2- 172.21.15.83:0/2233609342 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557f5efb79d0 0x557f5efb7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f383dc82640 1 -- 172.21.15.83:0/2233609342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f5efb82f0 msgr2=0x557f5ef6db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:20.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f383dc82640 1 --2- 172.21.15.83:0/2233609342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f5efb82f0 0x557f5ef6db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f383dc82640 1 -- 172.21.15.83:0/2233609342 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f5ef510d0 con 0x557f5ef6e0a0 2024-04-17T10:41:20.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f383cc80640 1 --2- 172.21.15.83:0/2233609342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f5efb82f0 0x557f5ef6db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:20.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f383dc82640 1 --2- 172.21.15.83:0/2233609342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5ef6e0a0 0x557f5ef6e480 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f38300af040 tx=0x7f3830004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11714a3903baf305 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:20.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f3822ffd640 1 -- 172.21.15.83:0/2233609342 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38300090d0 con 0x557f5ef6e0a0 2024-04-17T10:41:20.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f3822ffd640 1 -- 172.21.15.83:0/2233609342 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38300a5880 con 0x557f5ef6e0a0 2024-04-17T10:41:20.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f3822ffd640 1 -- 172.21.15.83:0/2233609342 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38300a5a70 con 0x557f5ef6e0a0 2024-04-17T10:41:20.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f3842852000 1 -- 172.21.15.83:0/2233609342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5ef6e0a0 msgr2=0x557f5ef6e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:20.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f3842852000 1 --2- 172.21.15.83:0/2233609342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5ef6e0a0 0x557f5ef6e480 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f38300af040 tx=0x7f3830004a50 comp rx=0 tx=0).stop 2024-04-17T10:41:20.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f3842852000 1 -- 172.21.15.83:0/2233609342 shutdown_connections 2024-04-17T10:41:20.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f3842852000 1 --2- 172.21.15.83:0/2233609342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5ef6e0a0 0x557f5ef6e480 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f3842852000 1 --2- 172.21.15.83:0/2233609342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f5efb82f0 0x557f5ef6db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f3842852000 1 --2- 172.21.15.83:0/2233609342 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557f5efb79d0 0x557f5efb7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f3842852000 1 -- 172.21.15.83:0/2233609342 >> 172.21.15.83:0/2233609342 conn(0x557f5ef6a020 msgr2=0x557f5ef6c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:20.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f3842852000 1 -- 172.21.15.83:0/2233609342 shutdown_connections 2024-04-17T10:41:20.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f3842852000 1 -- 172.21.15.83:0/2233609342 wait complete. 2024-04-17T10:41:20.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f3842852000 1 -- start start 2024-04-17T10:41:20.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f3842852000 1 Processor -- start 2024-04-17T10:41:20.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f3842852000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5ef6e0a0 0x557f5ef9c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f3842852000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f5efb79d0 0x557f5ef9cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f3842852000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557f5efb82f0 0x557f5ef9d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f3842852000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5ed7f250 con 0x557f5ef6e0a0 2024-04-17T10:41:20.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f3842852000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5ed75690 con 0x557f5efb79d0 2024-04-17T10:41:20.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f3842852000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x557f5ed43a40 con 0x557f5efb82f0 2024-04-17T10:41:20.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f3842852000 10 client.0 _ll_register_callbacks cb 0x557f5ef58e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:41:20.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f383d481640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5ef6e0a0 0x557f5ef9c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f383d481640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5ef6e0a0 0x557f5ef9c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36332/0 (socket says 172.21.15.83:36332) 2024-04-17T10:41:20.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.251+0000 7f383d481640 1 -- 172.21.15.83:0/2021365493 learned_addr learned my addr 172.21.15.83:0/2021365493 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:20.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f383dc82640 1 --2- 172.21.15.83:0/2021365493 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557f5efb82f0 0x557f5ef9d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f383cc80640 1 --2- 172.21.15.83:0/2021365493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f5efb79d0 0x557f5ef9cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f383d481640 1 -- 172.21.15.83:0/2021365493 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557f5efb82f0 msgr2=0x557f5ef9d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:20.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f383d481640 1 --2- 172.21.15.83:0/2021365493 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557f5efb82f0 0x557f5ef9d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f383d481640 1 -- 172.21.15.83:0/2021365493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f5efb79d0 msgr2=0x557f5ef9cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:20.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f383d481640 1 --2- 172.21.15.83:0/2021365493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f5efb79d0 0x557f5ef9cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f383d481640 1 -- 172.21.15.83:0/2021365493 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f5ee681c0 con 0x557f5ef6e0a0 2024-04-17T10:41:20.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f383dc82640 1 --2- 172.21.15.83:0/2021365493 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557f5efb82f0 0x557f5ef9d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:20.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f383d481640 1 --2- 172.21.15.83:0/2021365493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5ef6e0a0 0x557f5ef9c7f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f382805dfc0 tx=0x7f3828055de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a8d81331638dd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:20.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f3817fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:41:20.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f3817fff640 1 -- 172.21.15.83:0/2021365493 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382805e4f0 con 0x557f5ef6e0a0 2024-04-17T10:41:20.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f3817fff640 1 -- 172.21.15.83:0/2021365493 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f3828052450 con 0x557f5ef6e0a0 2024-04-17T10:41:20.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f3842852000 1 -- 172.21.15.83:0/2021365493 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38300a5880 con 0x557f5ef6e0a0 2024-04-17T10:41:20.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f3817fff640 1 -- 172.21.15.83:0/2021365493 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38280850b0 con 0x557f5ef6e0a0 2024-04-17T10:41:20.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f3842852000 10 client.5451 fetch_fsmap learned FSMap version 14 2024-04-17T10:41:20.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f3842852000 1 -- 172.21.15.83:0/2021365493 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x557f5ee64170 con 0x557f5ef6e0a0 2024-04-17T10:41:20.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f3817fff640 1 -- 172.21.15.83:0/2021365493 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38300a5880 con 0x557f5ef6e0a0 2024-04-17T10:41:20.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f3817fff640 1 -- 172.21.15.83:0/2021365493 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 14) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3828013070 con 0x557f5ef6e0a0 2024-04-17T10:41:20.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f3842852000 10 client.5451 fetch_fsmap finished waiting for FSMap version 14 2024-04-17T10:41:20.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f3842852000 10 client.5451 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-17T10:41:20.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f3842852000 20 client.5451 populate_metadata read hostname 'smithi083' 2024-04-17T10:41:20.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f3842852000 1 --2- 172.21.15.83:0/2021365493 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x557f5efc7c80 0x557f5efca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f3842852000 4 client.5451 mds_command: new command op to 5172 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:41:20.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f3842852000 1 -- 172.21.15.83:0/2021365493 --> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f5ed56c60 con 0x557f5efc7c80 2024-04-17T10:41:20.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f383cc80640 1 --2- 172.21.15.83:0/2021365493 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x557f5efc7c80 0x557f5efca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f3817fff640 10 client.5451 ms_handle_connect on v2:172.21.15.83:6836/1181902025 2024-04-17T10:41:20.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f383cc80640 1 --2- 172.21.15.83:0/2021365493 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x557f5efc7c80 0x557f5efca070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a79506e5b39e27ed server_cookie=b67ef9423a43dd1c in_seq=0 out_seq=0 2024-04-17T10:41:20.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f3817fff640 1 -- 172.21.15.83:0/2021365493 <== mds.0 v2:172.21.15.83:6836/1181902025 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557f5ed56c60 con 0x557f5efc7c80 2024-04-17T10:41:20.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f3817fff640 10 client.5451 handle_command_reply: tid=0 2024-04-17T10:41:20.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f3842852000 1 client.5451 shutdown 2024-04-17T10:41:20.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f3842852000 1 -- 172.21.15.83:0/2021365493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5ef6e0a0 msgr2=0x557f5ef9c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:20.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f3842852000 1 --2- 172.21.15.83:0/2021365493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5ef6e0a0 0x557f5ef9c7f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f382805dfc0 tx=0x7f3828055de0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f3842852000 1 -- 172.21.15.83:0/2021365493 shutdown_connections 2024-04-17T10:41:20.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f3842852000 1 --2- 172.21.15.83:0/2021365493 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x557f5efc7c80 0x557f5efca070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f3842852000 1 --2- 172.21.15.83:0/2021365493 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557f5efb82f0 0x557f5ef9d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f3842852000 1 --2- 172.21.15.83:0/2021365493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f5efb79d0 0x557f5ef9cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f3817fff640 0 client.5451 ms_handle_reset on v2:172.21.15.83:6836/1181902025 2024-04-17T10:41:20.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f3842852000 1 --2- 172.21.15.83:0/2021365493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5ef6e0a0 0x557f5ef9c7f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f3842852000 1 -- 172.21.15.83:0/2021365493 >> 172.21.15.83:0/2021365493 conn(0x557f5ef6a020 msgr2=0x557f5ef6cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:20.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f3842852000 1 -- 172.21.15.83:0/2021365493 shutdown_connections 2024-04-17T10:41:20.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f3842852000 1 -- 172.21.15.83:0/2021365493 wait complete. 2024-04-17T10:41:20.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.255+0000 7f3842852000 20 client.5451 trim_cache size 0 max 16384 2024-04-17T10:41:20.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.271+0000 7f3842852000 1 Processor -- start 2024-04-17T10:41:20.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.271+0000 7f3842852000 1 -- start start 2024-04-17T10:41:20.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.271+0000 7f3842852000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f5ef6e0a0 0x557f5efb6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.271+0000 7f3842852000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5efb79d0 0x557f5ef6d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.271+0000 7f3842852000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557f5efb82f0 0x557f5ef6d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.271+0000 7f3842852000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3838202930 con 0x557f5efb79d0 2024-04-17T10:41:20.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.271+0000 7f3842852000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3838201b20 con 0x557f5ef6e0a0 2024-04-17T10:41:20.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.271+0000 7f3842852000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3838155660 con 0x557f5efb82f0 2024-04-17T10:41:20.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.271+0000 7f383cc80640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5efb79d0 0x557f5ef6d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.271+0000 7f383cc80640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5efb79d0 0x557f5ef6d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36336/0 (socket says 172.21.15.83:36336) 2024-04-17T10:41:20.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.271+0000 7f383cc80640 1 -- 172.21.15.83:0/2219881066 learned_addr learned my addr 172.21.15.83:0/2219881066 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:20.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.271+0000 7f383cc80640 1 -- 172.21.15.83:0/2219881066 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557f5efb82f0 msgr2=0x557f5ef6d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:20.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.271+0000 7f383cc80640 1 --2- 172.21.15.83:0/2219881066 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557f5efb82f0 0x557f5ef6d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.271+0000 7f383cc80640 1 -- 172.21.15.83:0/2219881066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f5ef6e0a0 msgr2=0x557f5efb6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:20.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.271+0000 7f383cc80640 1 --2- 172.21.15.83:0/2219881066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f5ef6e0a0 0x557f5efb6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.271+0000 7f383cc80640 1 -- 172.21.15.83:0/2219881066 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f5ee01de0 con 0x557f5efb79d0 2024-04-17T10:41:20.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.271+0000 7f383cc80640 1 --2- 172.21.15.83:0/2219881066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5efb79d0 0x557f5ef6d420 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3834000f70 tx=0x7f3834011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4be8f7f82872411 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:20.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.271+0000 7f38227fc640 1 -- 172.21.15.83:0/2219881066 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38340076c0 con 0x557f5efb79d0 2024-04-17T10:41:20.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.271+0000 7f38227fc640 1 -- 172.21.15.83:0/2219881066 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3834023850 con 0x557f5efb79d0 2024-04-17T10:41:20.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.271+0000 7f38227fc640 1 -- 172.21.15.83:0/2219881066 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383400d7a0 con 0x557f5efb79d0 2024-04-17T10:41:20.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.271+0000 7f3842852000 1 -- 172.21.15.83:0/2219881066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5efb79d0 msgr2=0x557f5ef6d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:20.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.271+0000 7f3842852000 1 --2- 172.21.15.83:0/2219881066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5efb79d0 0x557f5ef6d420 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3834000f70 tx=0x7f3834011000 comp rx=0 tx=0).stop 2024-04-17T10:41:20.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.271+0000 7f3842852000 1 -- 172.21.15.83:0/2219881066 shutdown_connections 2024-04-17T10:41:20.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.271+0000 7f3842852000 1 --2- 172.21.15.83:0/2219881066 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557f5efb82f0 0x557f5ef6d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.271+0000 7f3842852000 1 --2- 172.21.15.83:0/2219881066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5efb79d0 0x557f5ef6d420 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.271+0000 7f3842852000 1 --2- 172.21.15.83:0/2219881066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f5ef6e0a0 0x557f5efb6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.271+0000 7f3842852000 1 -- 172.21.15.83:0/2219881066 >> 172.21.15.83:0/2219881066 conn(0x557f5ef6a020 msgr2=0x557f5efb60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:20.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.271+0000 7f3842852000 1 -- 172.21.15.83:0/2219881066 shutdown_connections 2024-04-17T10:41:20.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.271+0000 7f3842852000 1 -- 172.21.15.83:0/2219881066 wait complete. 2024-04-17T10:41:20.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3842852000 1 -- start start 2024-04-17T10:41:20.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3842852000 1 Processor -- start 2024-04-17T10:41:20.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3842852000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557f5ef6e0a0 0x557f5f018520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3842852000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5efb79d0 0x557f5f018a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3842852000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f5efb82f0 0x557f5efca380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3842852000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5ed55e00 con 0x557f5efb79d0 2024-04-17T10:41:20.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3842852000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3838201520 con 0x557f5efb82f0 2024-04-17T10:41:20.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3842852000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x557f5edba300 con 0x557f5ef6e0a0 2024-04-17T10:41:20.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3842852000 10 client.0 _ll_register_callbacks cb 0x557f5ef39040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:41:20.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f383cc80640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5efb79d0 0x557f5f018a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f383cc80640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5efb79d0 0x557f5f018a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36346/0 (socket says 172.21.15.83:36346) 2024-04-17T10:41:20.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f383cc80640 1 -- 172.21.15.83:0/3411769375 learned_addr learned my addr 172.21.15.83:0/3411769375 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:20.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f383dc82640 1 --2- 172.21.15.83:0/3411769375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f5efb82f0 0x557f5efca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f383cc80640 1 -- 172.21.15.83:0/3411769375 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557f5ef6e0a0 msgr2=0x557f5f018520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:20.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f383cc80640 1 --2- 172.21.15.83:0/3411769375 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557f5ef6e0a0 0x557f5f018520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f383cc80640 1 -- 172.21.15.83:0/3411769375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f5efb82f0 msgr2=0x557f5efca380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:20.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f383cc80640 1 --2- 172.21.15.83:0/3411769375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f5efb82f0 0x557f5efca380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f383cc80640 1 -- 172.21.15.83:0/3411769375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f5ed5c8e0 con 0x557f5efb79d0 2024-04-17T10:41:20.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f383cc80640 1 --2- 172.21.15.83:0/3411769375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5efb79d0 0x557f5f018a60 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f3834005ed0 tx=0x7f3834000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c70874bade62ea server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:20.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3817fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:41:20.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3817fff640 1 -- 172.21.15.83:0/3411769375 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383400d9a0 con 0x557f5efb79d0 2024-04-17T10:41:20.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3842852000 1 -- 172.21.15.83:0/3411769375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3834023850 con 0x557f5efb79d0 2024-04-17T10:41:20.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3817fff640 1 -- 172.21.15.83:0/3411769375 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f383400db40 con 0x557f5efb79d0 2024-04-17T10:41:20.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3817fff640 1 -- 172.21.15.83:0/3411769375 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f383400dd80 con 0x557f5efb79d0 2024-04-17T10:41:20.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3842852000 10 client.5457 fetch_fsmap learned FSMap version 14 2024-04-17T10:41:20.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3842852000 1 -- 172.21.15.83:0/3411769375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x557f5ee148f0 con 0x557f5efb79d0 2024-04-17T10:41:20.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3817fff640 1 -- 172.21.15.83:0/3411769375 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3834023850 con 0x557f5efb79d0 2024-04-17T10:41:20.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3817fff640 1 -- 172.21.15.83:0/3411769375 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 14) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3834041080 con 0x557f5efb79d0 2024-04-17T10:41:20.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3842852000 10 client.5457 fetch_fsmap finished waiting for FSMap version 14 2024-04-17T10:41:20.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3842852000 10 client.5457 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-17T10:41:20.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3842852000 20 client.5457 populate_metadata read hostname 'smithi083' 2024-04-17T10:41:20.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3842852000 1 --2- 172.21.15.83:0/3411769375 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x557f5efcac80 0x557f5efcb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3842852000 4 client.5457 mds_command: new command op to 5172 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T10:41:20.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3842852000 1 -- 172.21.15.83:0/3411769375 --> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557f5edc8530 con 0x557f5efcac80 2024-04-17T10:41:20.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f383d481640 1 --2- 172.21.15.83:0/3411769375 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x557f5efcac80 0x557f5efcb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3817fff640 10 client.5457 ms_handle_connect on v2:172.21.15.83:6836/1181902025 2024-04-17T10:41:20.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f383d481640 1 --2- 172.21.15.83:0/3411769375 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x557f5efcac80 0x557f5efcb090 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72d74e1d5f376b12 server_cookie=ec6745a9ae83a41f in_seq=0 out_seq=0 2024-04-17T10:41:20.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3817fff640 1 -- 172.21.15.83:0/3411769375 <== mds.0 v2:172.21.15.83:6836/1181902025 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557f5edc8530 con 0x557f5efcac80 2024-04-17T10:41:20.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3817fff640 10 client.5457 handle_command_reply: tid=0 2024-04-17T10:41:20.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3842852000 1 client.5457 shutdown 2024-04-17T10:41:20.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3842852000 1 -- 172.21.15.83:0/3411769375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5efb79d0 msgr2=0x557f5f018a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:20.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3842852000 1 --2- 172.21.15.83:0/3411769375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5efb79d0 0x557f5f018a60 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f3834005ed0 tx=0x7f3834000ee0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3842852000 1 -- 172.21.15.83:0/3411769375 shutdown_connections 2024-04-17T10:41:20.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3842852000 1 --2- 172.21.15.83:0/3411769375 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x557f5efcac80 0x557f5efcb090 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3842852000 1 --2- 172.21.15.83:0/3411769375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f5efb82f0 0x557f5efca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3817fff640 0 client.5457 ms_handle_reset on v2:172.21.15.83:6836/1181902025 2024-04-17T10:41:20.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3842852000 1 --2- 172.21.15.83:0/3411769375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5efb79d0 0x557f5f018a60 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3842852000 1 --2- 172.21.15.83:0/3411769375 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557f5ef6e0a0 0x557f5f018520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3842852000 1 -- 172.21.15.83:0/3411769375 >> 172.21.15.83:0/3411769375 conn(0x557f5ef6a020 msgr2=0x557f5efb6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:20.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3842852000 1 -- 172.21.15.83:0/3411769375 shutdown_connections 2024-04-17T10:41:20.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3842852000 1 -- 172.21.15.83:0/3411769375 wait complete. 2024-04-17T10:41:20.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f3842852000 20 client.5457 trim_cache size 0 max 16384 2024-04-17T10:41:20.281 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:41:20.282 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-17T10:41:20.282 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-17T10:41:20.282 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:41:20.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f383f6e5640 1 -- 172.21.15.83:0/1315571740 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f381c066f60 msgr2=0x7f381c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:20.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f383f6e5640 1 --2- 172.21.15.83:0/1315571740 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f381c066f60 0x7f381c069420 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3838156ab0 tx=0x7f3834009040 comp rx=0 tx=0).stop 2024-04-17T10:41:20.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f383f6e5640 1 -- 172.21.15.83:0/1315571740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38380ce4e0 msgr2=0x7f3838152870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:20.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.275+0000 7f383f6e5640 1 --2- 172.21.15.83:0/1315571740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38380ce4e0 0x7f3838152870 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f382800d800 tx=0x7f382800dcc0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.279+0000 7f383f6e5640 1 -- 172.21.15.83:0/1315571740 shutdown_connections 2024-04-17T10:41:20.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.279+0000 7f383f6e5640 1 --2- 172.21.15.83:0/1315571740 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f381c066f60 0x7f381c069420 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.279+0000 7f383f6e5640 1 --2- 172.21.15.83:0/1315571740 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38380d26c0 0x7f38380cc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.279+0000 7f383f6e5640 1 --2- 172.21.15.83:0/1315571740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38380d0970 0x7f3838160cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.279+0000 7f383f6e5640 1 --2- 172.21.15.83:0/1315571740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38380ce4e0 0x7f3838152870 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.279+0000 7f383f6e5640 1 -- 172.21.15.83:0/1315571740 >> 172.21.15.83:0/1315571740 conn(0x7f383809cd50 msgr2=0x7f38380b1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:20.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.279+0000 7f383f6e5640 1 -- 172.21.15.83:0/1315571740 shutdown_connections 2024-04-17T10:41:20.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.279+0000 7f383f6e5640 1 -- 172.21.15.83:0/1315571740 wait complete. 2024-04-17T10:41:20.291 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T10:41:20.291 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-17T10:41:20.291 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2024-04-17T10:41:20.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.407+0000 7f5d43ee6640 1 Processor -- start 2024-04-17T10:41:20.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.407+0000 7f5d43ee6640 1 -- start start 2024-04-17T10:41:20.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.411+0000 7f5d43ee6640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5d3c15bac0 0x7f5d3c071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.411+0000 7f5d43ee6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d3c070470 0x7f5d3c06ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.411+0000 7f5d43ee6640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d3c06c1f0 con 0x7f5d3c15b6e0 2024-04-17T10:41:20.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.411+0000 7f5d43ee6640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d3c070a10 con 0x7f5d3c15bac0 2024-04-17T10:41:20.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.411+0000 7f5d43ee6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c070cf0 con 0x7f5d3c070470 2024-04-17T10:41:20.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.411+0000 7f5d41481640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5d3c15bac0 0x7f5d3c071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.411+0000 7f5d41c82640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5d3c15b6e0 0x7f5d3c0cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53984/0 (socket says 172.21.15.83:53984) 2024-04-17T10:41:20.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.411+0000 7f5d41c82640 1 -- 172.21.15.83:0/419812392 learned_addr learned my addr 172.21.15.83:0/419812392 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:20.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.411+0000 7f5d41481640 1 -- 172.21.15.83:0/419812392 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d3c070470 msgr2=0x7f5d3c06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:20.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.411+0000 7f5d41481640 1 --2- 172.21.15.83:0/419812392 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d3c070470 0x7f5d3c06ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.411+0000 7f5d41481640 1 -- 172.21.15.83:0/419812392 >> v1:172.21.15.83:6789/0 conn(0x7f5d3c15b6e0 legacy=0x7f5d3c0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:20.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.411+0000 7f5d41481640 1 -- 172.21.15.83:0/419812392 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3c070fd0 con 0x7f5d3c15bac0 2024-04-17T10:41:20.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.411+0000 7f5d41481640 1 --2- 172.21.15.83:0/419812392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d3c15bac0 0x7f5d3c071cc0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c0097c0 tx=0x7f5d2c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f4b7ed46e203c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:20.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.411+0000 7f5d40c80640 1 -- 172.21.15.83:0/419812392 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c0b0070 con 0x7f5d3c15bac0 2024-04-17T10:41:20.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.411+0000 7f5d40c80640 1 -- 172.21.15.83:0/419812392 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d2c0ab600 con 0x7f5d3c15bac0 2024-04-17T10:41:20.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.411+0000 7f5d40c80640 1 -- 172.21.15.83:0/419812392 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c0ab9d0 con 0x7f5d3c15bac0 2024-04-17T10:41:20.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.411+0000 7f5d43ee6640 1 -- 172.21.15.83:0/419812392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d3c15bac0 msgr2=0x7f5d3c071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:20.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.411+0000 7f5d43ee6640 1 --2- 172.21.15.83:0/419812392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d3c15bac0 0x7f5d3c071cc0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c0097c0 tx=0x7f5d2c01c410 comp rx=0 tx=0).stop 2024-04-17T10:41:20.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.411+0000 7f5d43ee6640 1 -- 172.21.15.83:0/419812392 shutdown_connections 2024-04-17T10:41:20.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.411+0000 7f5d43ee6640 1 --2- 172.21.15.83:0/419812392 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d3c070470 0x7f5d3c06ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.411+0000 7f5d43ee6640 1 --2- 172.21.15.83:0/419812392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d3c15bac0 0x7f5d3c071cc0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.411+0000 7f5d43ee6640 1 -- 172.21.15.83:0/419812392 >> 172.21.15.83:0/419812392 conn(0x7f5d3c0b15b0 msgr2=0x7f5d3c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:20.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.411+0000 7f5d43ee6640 1 -- 172.21.15.83:0/419812392 shutdown_connections 2024-04-17T10:41:20.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.411+0000 7f5d43ee6640 1 -- 172.21.15.83:0/419812392 wait complete. 2024-04-17T10:41:20.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.415+0000 7f5d43ee6640 1 Processor -- start 2024-04-17T10:41:20.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.415+0000 7f5d43ee6640 1 -- start start 2024-04-17T10:41:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.415+0000 7f5d43ee6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d3c070470 0x7f5d3c14dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.415+0000 7f5d43ee6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d3c15b6e0 0x7f5d3c14e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.415+0000 7f5d43ee6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d3c15bac0 0x7f5d3c158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.415+0000 7f5d43ee6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c06d2f0 con 0x7f5d3c15b6e0 2024-04-17T10:41:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.415+0000 7f5d43ee6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c06d170 con 0x7f5d3c15bac0 2024-04-17T10:41:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.415+0000 7f5d43ee6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c06b760 con 0x7f5d3c070470 2024-04-17T10:41:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.415+0000 7f5d41481640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d3c15b6e0 0x7f5d3c14e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.415+0000 7f5d41481640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d3c15b6e0 0x7f5d3c14e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36366/0 (socket says 172.21.15.83:36366) 2024-04-17T10:41:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.415+0000 7f5d41481640 1 -- 172.21.15.83:0/349125802 learned_addr learned my addr 172.21.15.83:0/349125802 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.415+0000 7f5d41c82640 1 --2- 172.21.15.83:0/349125802 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d3c070470 0x7f5d3c14dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.415+0000 7f5d42483640 1 --2- 172.21.15.83:0/349125802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d3c15bac0 0x7f5d3c158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.415+0000 7f5d41481640 1 -- 172.21.15.83:0/349125802 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d3c070470 msgr2=0x7f5d3c14dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.415+0000 7f5d41481640 1 --2- 172.21.15.83:0/349125802 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d3c070470 0x7f5d3c14dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.415+0000 7f5d41481640 1 -- 172.21.15.83:0/349125802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d3c15bac0 msgr2=0x7f5d3c158f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.415+0000 7f5d41481640 1 --2- 172.21.15.83:0/349125802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d3c15bac0 0x7f5d3c158f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.415+0000 7f5d41481640 1 -- 172.21.15.83:0/349125802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3c155fa0 con 0x7f5d3c15b6e0 2024-04-17T10:41:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.415+0000 7f5d42483640 1 --2- 172.21.15.83:0/349125802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d3c15bac0 0x7f5d3c158f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.415+0000 7f5d41481640 1 --2- 172.21.15.83:0/349125802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d3c15b6e0 0x7f5d3c14e170 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c01a040 tx=0x7f5d2c0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.415+0000 7f5d327fc640 1 -- 172.21.15.83:0/349125802 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c0ba070 con 0x7f5d3c15b6e0 2024-04-17T10:41:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.415+0000 7f5d327fc640 1 -- 172.21.15.83:0/349125802 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d2c0b5890 con 0x7f5d3c15b6e0 2024-04-17T10:41:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.415+0000 7f5d43ee6640 1 -- 172.21.15.83:0/349125802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d3c156280 con 0x7f5d3c15b6e0 2024-04-17T10:41:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.415+0000 7f5d327fc640 1 -- 172.21.15.83:0/349125802 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c0b0040 con 0x7f5d3c15b6e0 2024-04-17T10:41:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.415+0000 7f5d43ee6640 1 -- 172.21.15.83:0/349125802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d3c156840 con 0x7f5d3c15b6e0 2024-04-17T10:41:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.419+0000 7f5d327fc640 1 -- 172.21.15.83:0/349125802 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5d2c0bd070 con 0x7f5d3c15b6e0 2024-04-17T10:41:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.419+0000 7f5d327fc640 1 --2- 172.21.15.83:0/349125802 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5d20066f60 0x7f5d20069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.419+0000 7f5d327fc640 1 -- 172.21.15.83:0/349125802 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f5d2c11f890 con 0x7f5d3c15b6e0 2024-04-17T10:41:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.419+0000 7f5d41c82640 1 --2- 172.21.15.83:0/349125802 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5d20066f60 0x7f5d20069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.419+0000 7f5d41c82640 1 --2- 172.21.15.83:0/349125802 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5d20066f60 0x7f5d20069420 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5d38000f70 tx=0x7f5d38009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.419+0000 7f5d47053000 1 Processor -- start 2024-04-17T10:41:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.419+0000 7f5d47053000 1 -- start start 2024-04-17T10:41:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.419+0000 7f5d47053000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627020a8a40 0x5627020a8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.419+0000 7f5d47053000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5627020a9360 0x562702063850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.419+0000 7f5d47053000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562702063d90 0x562702064170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.419+0000 7f5d47053000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562701f461f0 con 0x562702063d90 2024-04-17T10:41:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.419+0000 7f5d47053000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562701eace90 con 0x5627020a8a40 2024-04-17T10:41:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.419+0000 7f5d47053000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x562701e34f50 con 0x5627020a9360 2024-04-17T10:41:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.419+0000 7f5d42483640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562702063d90 0x562702064170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.419+0000 7f5d42483640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562702063d90 0x562702064170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36378/0 (socket says 172.21.15.83:36378) 2024-04-17T10:41:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.419+0000 7f5d42483640 1 -- 172.21.15.83:0/2559246210 learned_addr learned my addr 172.21.15.83:0/2559246210 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.419+0000 7f5d41c82640 1 --2- 172.21.15.83:0/2559246210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627020a8a40 0x5627020a8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.419+0000 7f5d42483640 1 -- 172.21.15.83:0/2559246210 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5627020a9360 msgr2=0x562702063850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.419+0000 7f5d41481640 1 --2- 172.21.15.83:0/2559246210 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5627020a9360 0x562702063850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.419+0000 7f5d42483640 1 --2- 172.21.15.83:0/2559246210 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5627020a9360 0x562702063850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.419+0000 7f5d42483640 1 -- 172.21.15.83:0/2559246210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627020a8a40 msgr2=0x5627020a8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.419+0000 7f5d42483640 1 --2- 172.21.15.83:0/2559246210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627020a8a40 0x5627020a8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.419+0000 7f5d42483640 1 -- 172.21.15.83:0/2559246210 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562701e848b0 con 0x562702063d90 2024-04-17T10:41:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.419+0000 7f5d42483640 1 --2- 172.21.15.83:0/2559246210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562702063d90 0x562702064170 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5d34007f30 tx=0x7f5d3400c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a20b7cb4b85305 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.419+0000 7f5d1f7fe640 1 -- 172.21.15.83:0/2559246210 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d34014070 con 0x562702063d90 2024-04-17T10:41:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.419+0000 7f5d1f7fe640 1 -- 172.21.15.83:0/2559246210 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3400f850 con 0x562702063d90 2024-04-17T10:41:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.419+0000 7f5d1f7fe640 1 -- 172.21.15.83:0/2559246210 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3400fc20 con 0x562702063d90 2024-04-17T10:41:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d47053000 1 -- 172.21.15.83:0/2559246210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562702063d90 msgr2=0x562702064170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d47053000 1 --2- 172.21.15.83:0/2559246210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562702063d90 0x562702064170 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5d34007f30 tx=0x7f5d3400c750 comp rx=0 tx=0).stop 2024-04-17T10:41:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d47053000 1 -- 172.21.15.83:0/2559246210 shutdown_connections 2024-04-17T10:41:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d47053000 1 --2- 172.21.15.83:0/2559246210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562702063d90 0x562702064170 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d47053000 1 --2- 172.21.15.83:0/2559246210 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5627020a9360 0x562702063850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d47053000 1 --2- 172.21.15.83:0/2559246210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627020a8a40 0x5627020a8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d47053000 1 -- 172.21.15.83:0/2559246210 >> 172.21.15.83:0/2559246210 conn(0x56270205fd40 msgr2=0x562702062130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d47053000 1 -- 172.21.15.83:0/2559246210 shutdown_connections 2024-04-17T10:41:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d47053000 1 -- 172.21.15.83:0/2559246210 wait complete. 2024-04-17T10:41:20.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d47053000 1 -- start start 2024-04-17T10:41:20.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d47053000 1 Processor -- start 2024-04-17T10:41:20.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d47053000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562702063d90 0x5627020b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d47053000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627020a8a40 0x5627020b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d47053000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5627020a9360 0x562702069e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d47053000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562701e67540 con 0x5627020a8a40 2024-04-17T10:41:20.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d47053000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562701ea8f30 con 0x562702063d90 2024-04-17T10:41:20.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d47053000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x562701e6ee50 con 0x5627020a9360 2024-04-17T10:41:20.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d47053000 10 client.0 _ll_register_callbacks cb 0x562701f61180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:41:20.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d41481640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627020a8a40 0x5627020b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d41481640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627020a8a40 0x5627020b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36390/0 (socket says 172.21.15.83:36390) 2024-04-17T10:41:20.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d41481640 1 -- 172.21.15.83:0/3280097548 learned_addr learned my addr 172.21.15.83:0/3280097548 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:20.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d42483640 1 --2- 172.21.15.83:0/3280097548 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5627020a9360 0x562702069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d41c82640 1 --2- 172.21.15.83:0/3280097548 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562702063d90 0x5627020b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d41481640 1 -- 172.21.15.83:0/3280097548 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5627020a9360 msgr2=0x562702069e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:20.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d41481640 1 --2- 172.21.15.83:0/3280097548 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5627020a9360 0x562702069e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d41481640 1 -- 172.21.15.83:0/3280097548 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562702063d90 msgr2=0x5627020b01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:20.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d41481640 1 --2- 172.21.15.83:0/3280097548 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562702063d90 0x5627020b01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d41481640 1 -- 172.21.15.83:0/3280097548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562701e351a0 con 0x5627020a8a40 2024-04-17T10:41:20.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d41481640 1 --2- 172.21.15.83:0/3280097548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627020a8a40 0x5627020b0700 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c01c8f0 tx=0x7f5d2c0f0940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a6cd1c352cc09c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:20.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:41:20.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d1cff9640 1 -- 172.21.15.83:0/3280097548 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c0c4040 con 0x5627020a8a40 2024-04-17T10:41:20.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d1cff9640 1 -- 172.21.15.83:0/3280097548 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f5d2c118dd0 con 0x5627020a8a40 2024-04-17T10:41:20.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d47053000 1 -- 172.21.15.83:0/3280097548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d3400f850 con 0x5627020a8a40 2024-04-17T10:41:20.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d1cff9640 1 -- 172.21.15.83:0/3280097548 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d2c0f8890 con 0x5627020a8a40 2024-04-17T10:41:20.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d47053000 10 client.5472 fetch_fsmap learned FSMap version 14 2024-04-17T10:41:20.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d47053000 1 -- 172.21.15.83:0/3280097548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x562701f2f840 con 0x5627020a8a40 2024-04-17T10:41:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d1cff9640 1 -- 172.21.15.83:0/3280097548 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3400f850 con 0x5627020a8a40 2024-04-17T10:41:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d1cff9640 1 -- 172.21.15.83:0/3280097548 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 14) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5d2c0ec7f0 con 0x5627020a8a40 2024-04-17T10:41:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d47053000 10 client.5472 fetch_fsmap finished waiting for FSMap version 14 2024-04-17T10:41:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d47053000 10 client.5472 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-17T10:41:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d47053000 20 client.5472 populate_metadata read hostname 'smithi083' 2024-04-17T10:41:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d47053000 1 --2- 172.21.15.83:0/3280097548 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x56270206ab30 0x56270206af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d47053000 4 client.5472 mds_command: new command op to 5172 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:41:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d47053000 1 -- 172.21.15.83:0/3280097548 --> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562701e48be0 con 0x56270206ab30 2024-04-17T10:41:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d41c82640 1 --2- 172.21.15.83:0/3280097548 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x56270206ab30 0x56270206af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d41c82640 1 --2- 172.21.15.83:0/3280097548 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x56270206ab30 0x56270206af10 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8ab90c52cb0bacd server_cookie=2915547bca664ba3 in_seq=0 out_seq=0 2024-04-17T10:41:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d1cff9640 10 client.5472 ms_handle_connect on v2:172.21.15.83:6836/1181902025 2024-04-17T10:41:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d1cff9640 1 -- 172.21.15.83:0/3280097548 <== mds.0 v2:172.21.15.83:6836/1181902025 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562701e48be0 con 0x56270206ab30 2024-04-17T10:41:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d1cff9640 10 client.5472 handle_command_reply: tid=0 2024-04-17T10:41:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.423+0000 7f5d47053000 1 client.5472 shutdown 2024-04-17T10:41:20.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.427+0000 7f5d47053000 1 -- 172.21.15.83:0/3280097548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627020a8a40 msgr2=0x5627020b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:20.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.427+0000 7f5d47053000 1 --2- 172.21.15.83:0/3280097548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627020a8a40 0x5627020b0700 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c01c8f0 tx=0x7f5d2c0f0940 comp rx=0 tx=0).stop 2024-04-17T10:41:20.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.427+0000 7f5d47053000 1 -- 172.21.15.83:0/3280097548 shutdown_connections 2024-04-17T10:41:20.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.427+0000 7f5d47053000 1 --2- 172.21.15.83:0/3280097548 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x56270206ab30 0x56270206af10 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.427+0000 7f5d47053000 1 --2- 172.21.15.83:0/3280097548 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5627020a9360 0x562702069e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.427+0000 7f5d47053000 1 --2- 172.21.15.83:0/3280097548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627020a8a40 0x5627020b0700 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.427+0000 7f5d47053000 1 --2- 172.21.15.83:0/3280097548 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562702063d90 0x5627020b01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.427+0000 7f5d1cff9640 0 client.5472 ms_handle_reset on v2:172.21.15.83:6836/1181902025 2024-04-17T10:41:20.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.427+0000 7f5d47053000 1 -- 172.21.15.83:0/3280097548 >> 172.21.15.83:0/3280097548 conn(0x56270205fd40 msgr2=0x5627020629c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:20.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.427+0000 7f5d47053000 1 -- 172.21.15.83:0/3280097548 shutdown_connections 2024-04-17T10:41:20.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.427+0000 7f5d47053000 1 -- 172.21.15.83:0/3280097548 wait complete. 2024-04-17T10:41:20.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.427+0000 7f5d47053000 20 client.5472 trim_cache size 0 max 16384 2024-04-17T10:41:20.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.439+0000 7f5d47053000 1 Processor -- start 2024-04-17T10:41:20.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.439+0000 7f5d47053000 1 -- start start 2024-04-17T10:41:20.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.439+0000 7f5d47053000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562702063d90 0x562702104e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.439+0000 7f5d47053000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5627020a8a40 0x5627021053b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.439+0000 7f5d47053000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627020a9360 0x5627021058f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.439+0000 7f5d47053000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c200d30 con 0x562702063d90 2024-04-17T10:41:20.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.439+0000 7f5d47053000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c1fff20 con 0x5627020a9360 2024-04-17T10:41:20.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.439+0000 7f5d47053000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c14b8f0 con 0x5627020a8a40 2024-04-17T10:41:20.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.439+0000 7f5d41c82640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562702063d90 0x562702104e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.439+0000 7f5d41c82640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562702063d90 0x562702104e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36402/0 (socket says 172.21.15.83:36402) 2024-04-17T10:41:20.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.439+0000 7f5d41c82640 1 -- 172.21.15.83:0/1213880551 learned_addr learned my addr 172.21.15.83:0/1213880551 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:20.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.439+0000 7f5d41481640 1 --2- 172.21.15.83:0/1213880551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5627020a8a40 0x5627021053b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.439+0000 7f5d41c82640 1 -- 172.21.15.83:0/1213880551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5627020a8a40 msgr2=0x5627021053b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:20.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.439+0000 7f5d42483640 1 --2- 172.21.15.83:0/1213880551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627020a9360 0x5627021058f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.439+0000 7f5d41c82640 1 --2- 172.21.15.83:0/1213880551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5627020a8a40 0x5627021053b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.439+0000 7f5d41c82640 1 -- 172.21.15.83:0/1213880551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627020a9360 msgr2=0x5627021058f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:20.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.439+0000 7f5d41c82640 1 --2- 172.21.15.83:0/1213880551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627020a9360 0x5627021058f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.439+0000 7f5d41c82640 1 -- 172.21.15.83:0/1213880551 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562701f06a10 con 0x562702063d90 2024-04-17T10:41:20.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.439+0000 7f5d41c82640 1 --2- 172.21.15.83:0/1213880551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562702063d90 0x562702104e70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f5d38005d60 tx=0x7f5d38011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8ad554ff4de334 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:20.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.439+0000 7f5d1effd640 1 -- 172.21.15.83:0/1213880551 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d380247a0 con 0x562702063d90 2024-04-17T10:41:20.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.439+0000 7f5d1effd640 1 -- 172.21.15.83:0/1213880551 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d380075d0 con 0x562702063d90 2024-04-17T10:41:20.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.439+0000 7f5d1effd640 1 -- 172.21.15.83:0/1213880551 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3800d6f0 con 0x562702063d90 2024-04-17T10:41:20.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 1 -- 172.21.15.83:0/1213880551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562702063d90 msgr2=0x562702104e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:20.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 1 --2- 172.21.15.83:0/1213880551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562702063d90 0x562702104e70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f5d38005d60 tx=0x7f5d38011000 comp rx=0 tx=0).stop 2024-04-17T10:41:20.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 1 -- 172.21.15.83:0/1213880551 shutdown_connections 2024-04-17T10:41:20.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 1 --2- 172.21.15.83:0/1213880551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627020a9360 0x5627021058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 1 --2- 172.21.15.83:0/1213880551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5627020a8a40 0x5627021053b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 1 --2- 172.21.15.83:0/1213880551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562702063d90 0x562702104e70 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 1 -- 172.21.15.83:0/1213880551 >> 172.21.15.83:0/1213880551 conn(0x56270205fd40 msgr2=0x5627020a70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:20.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 1 -- 172.21.15.83:0/1213880551 shutdown_connections 2024-04-17T10:41:20.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 1 -- 172.21.15.83:0/1213880551 wait complete. 2024-04-17T10:41:20.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 1 -- start start 2024-04-17T10:41:20.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 1 Processor -- start 2024-04-17T10:41:20.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562702063d90 0x5627021092a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5627020a8a40 0x5627021097e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627020a9360 0x5627020aded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562701ea8f30 con 0x562702063d90 2024-04-17T10:41:20.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c14bef0 con 0x5627020a9360 2024-04-17T10:41:20.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x562701f69d10 con 0x5627020a8a40 2024-04-17T10:41:20.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 10 client.0 _ll_register_callbacks cb 0x562701e6fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:41:20.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d41c82640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562702063d90 0x5627021092a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d41c82640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562702063d90 0x5627021092a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36412/0 (socket says 172.21.15.83:36412) 2024-04-17T10:41:20.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d41c82640 1 -- 172.21.15.83:0/1090982611 learned_addr learned my addr 172.21.15.83:0/1090982611 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:20.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d42483640 1 --2- 172.21.15.83:0/1090982611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627020a9360 0x5627020aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d41c82640 1 -- 172.21.15.83:0/1090982611 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5627020a8a40 msgr2=0x5627021097e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:20.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d41c82640 1 --2- 172.21.15.83:0/1090982611 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5627020a8a40 0x5627021097e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d41c82640 1 -- 172.21.15.83:0/1090982611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627020a9360 msgr2=0x5627020aded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:20.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d41c82640 1 --2- 172.21.15.83:0/1090982611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627020a9360 0x5627020aded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d41c82640 1 -- 172.21.15.83:0/1090982611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562701e41b00 con 0x562702063d90 2024-04-17T10:41:20.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d42483640 1 --2- 172.21.15.83:0/1090982611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627020a9360 0x5627020aded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:20.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d41c82640 1 --2- 172.21.15.83:0/1090982611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562702063d90 0x5627021092a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x562701eae2e0 tx=0x7f5d38000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c06479740734cd5e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:20.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:41:20.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d1cff9640 1 -- 172.21.15.83:0/1090982611 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d38036060 con 0x562702063d90 2024-04-17T10:41:20.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 1 -- 172.21.15.83:0/1090982611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d380075d0 con 0x562702063d90 2024-04-17T10:41:20.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d1cff9640 1 -- 172.21.15.83:0/1090982611 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f5d3800d6f0 con 0x562702063d90 2024-04-17T10:41:20.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d1cff9640 1 -- 172.21.15.83:0/1090982611 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d3800d8f0 con 0x562702063d90 2024-04-17T10:41:20.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 10 client.5478 fetch_fsmap learned FSMap version 14 2024-04-17T10:41:20.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 1 -- 172.21.15.83:0/1090982611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x562701e53f80 con 0x562702063d90 2024-04-17T10:41:20.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d1cff9640 1 -- 172.21.15.83:0/1090982611 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d380075d0 con 0x562702063d90 2024-04-17T10:41:20.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d1cff9640 1 -- 172.21.15.83:0/1090982611 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 14) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5d3800da90 con 0x562702063d90 2024-04-17T10:41:20.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 10 client.5478 fetch_fsmap finished waiting for FSMap version 14 2024-04-17T10:41:20.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 10 client.5478 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-17T10:41:20.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 20 client.5478 populate_metadata read hostname 'smithi083' 2024-04-17T10:41:20.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 1 --2- 172.21.15.83:0/1090982611 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x5627020ae830 0x5627020aec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 4 client.5478 mds_command: new command op to 5172 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-17T10:41:20.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 1 -- 172.21.15.83:0/1090982611 --> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x562701eba3d0 con 0x5627020ae830 2024-04-17T10:41:20.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d41481640 1 --2- 172.21.15.83:0/1090982611 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x5627020ae830 0x5627020aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d41481640 1 --2- 172.21.15.83:0/1090982611 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x5627020ae830 0x5627020aec10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a4107b5bdc07452 server_cookie=dbe2b6d8131d161f in_seq=0 out_seq=0 2024-04-17T10:41:20.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d1cff9640 10 client.5478 ms_handle_connect on v2:172.21.15.83:6836/1181902025 2024-04-17T10:41:20.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d1cff9640 1 -- 172.21.15.83:0/1090982611 <== mds.0 v2:172.21.15.83:6836/1181902025 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17954 (crc 0 0 0) 0x562701eba3d0 con 0x5627020ae830 2024-04-17T10:41:20.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d1cff9640 10 client.5478 handle_command_reply: tid=0 2024-04-17T10:41:20.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 1 client.5478 shutdown 2024-04-17T10:41:20.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 1 -- 172.21.15.83:0/1090982611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562702063d90 msgr2=0x5627021092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:20.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 1 --2- 172.21.15.83:0/1090982611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562702063d90 0x5627021092a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x562701eae2e0 tx=0x7f5d38000bd0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 1 -- 172.21.15.83:0/1090982611 shutdown_connections 2024-04-17T10:41:20.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 1 --2- 172.21.15.83:0/1090982611 >> [v2:172.21.15.83:6836/1181902025,v1:172.21.15.83:6837/1181902025] conn(0x5627020ae830 0x5627020aec10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 1 --2- 172.21.15.83:0/1090982611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627020a9360 0x5627020aded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d1cff9640 0 client.5478 ms_handle_reset on v2:172.21.15.83:6836/1181902025 2024-04-17T10:41:20.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 1 --2- 172.21.15.83:0/1090982611 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5627020a8a40 0x5627021097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 1 --2- 172.21.15.83:0/1090982611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562702063d90 0x5627021092a0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.443+0000 7f5d47053000 1 -- 172.21.15.83:0/1090982611 >> 172.21.15.83:0/1090982611 conn(0x56270205fd40 msgr2=0x5627020a7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:20.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.447+0000 7f5d47053000 1 -- 172.21.15.83:0/1090982611 shutdown_connections 2024-04-17T10:41:20.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.447+0000 7f5d47053000 1 -- 172.21.15.83:0/1090982611 wait complete. 2024-04-17T10:41:20.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.447+0000 7f5d47053000 20 client.5478 trim_cache size 0 max 16384 2024-04-17T10:41:20.449 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:41:20.449 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T10:41:20.449 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 5604, 2024-04-17T10:41:20.449 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 5605, 2024-04-17T10:41:20.449 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1402046, 2024-04-17T10:41:20.449 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 35179213, 2024-04-17T10:41:20.449 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 17, 2024-04-17T10:41:20.450 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 6, 2024-04-17T10:41:20.450 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.600964317, 2024-04-17T10:41:20.450 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.261217041, 2024-04-17T10:41:20.450 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.116274665, 2024-04-17T10:41:20.450 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.177640782, 2024-04-17T10:41:20.450 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T10:41:20.450 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 5603, 2024-04-17T10:41:20.450 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.568439859, 2024-04-17T10:41:20.450 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000101452 2024-04-17T10:41:20.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.450 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-17T10:41:20.450 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 10, 2024-04-17T10:41:20.450 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000001401, 2024-04-17T10:41:20.451 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000140 2024-04-17T10:41:20.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.451 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-17T10:41:20.451 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 0 2024-04-17T10:41:20.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.451 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T10:41:20.451 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 34447, 2024-04-17T10:41:20.451 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 60627, 2024-04-17T10:41:20.451 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 12294039, 2024-04-17T10:41:20.451 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 44339036, 2024-04-17T10:41:20.451 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 8, 2024-04-17T10:41:20.451 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 4, 2024-04-17T10:41:20.451 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 5.018340060, 2024-04-17T10:41:20.451 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 2.525705130, 2024-04-17T10:41:20.452 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 2.983127209, 2024-04-17T10:41:20.452 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.086592895, 2024-04-17T10:41:20.452 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T10:41:20.452 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 60626, 2024-04-17T10:41:20.452 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 1.934145548, 2024-04-17T10:41:20.452 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000031902 2024-04-17T10:41:20.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.452 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-17T10:41:20.452 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 88187, 2024-04-17T10:41:20.452 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.021592984, 2024-04-17T10:41:20.452 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000244 2024-04-17T10:41:20.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.452 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-04-17T10:41:20.452 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 132240 2024-04-17T10:41:20.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.453 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T10:41:20.453 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 5384, 2024-04-17T10:41:20.453 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 5379, 2024-04-17T10:41:20.453 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1816463, 2024-04-17T10:41:20.453 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 39661948, 2024-04-17T10:41:20.453 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 5, 2024-04-17T10:41:20.453 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 4, 2024-04-17T10:41:20.453 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.599989428, 2024-04-17T10:41:20.453 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.245018098, 2024-04-17T10:41:20.453 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.125078440, 2024-04-17T10:41:20.453 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.200215719, 2024-04-17T10:41:20.453 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T10:41:20.453 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 5379, 2024-04-17T10:41:20.453 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.779123682, 2024-04-17T10:41:20.454 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000144845 2024-04-17T10:41:20.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.454 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-17T10:41:20.454 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:41:20.454 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:41:20.454 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:41:20.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.454 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 118688, 2024-04-17T10:41:20.454 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 30944 2024-04-17T10:41:20.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.454 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-04-17T10:41:20.454 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 1, 2024-04-17T10:41:20.454 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-04-17T10:41:20.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.455 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-MDSRank": { 2024-04-17T10:41:20.455 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-04-17T10:41:20.455 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-04-17T10:41:20.455 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 10558, 2024-04-17T10:41:20.455 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 24.499338656, 2024-04-17T10:41:20.455 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.002320452 2024-04-17T10:41:20.455 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:41:20.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.455 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-PurgeQueue": { 2024-04-17T10:41:20.455 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-04-17T10:41:20.455 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-04-17T10:41:20.455 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3882, 2024-04-17T10:41:20.455 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 1.496799959, 2024-04-17T10:41:20.456 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000385574 2024-04-17T10:41:20.456 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:41:20.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.456 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": { 2024-04-17T10:41:20.456 INFO:teuthology.orchestra.run.smithi083.stdout: "request": 26730, 2024-04-17T10:41:20.456 INFO:teuthology.orchestra.run.smithi083.stdout: "reply": 26730, 2024-04-17T10:41:20.456 INFO:teuthology.orchestra.run.smithi083.stdout: "reply_latency": { 2024-04-17T10:41:20.456 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 26730, 2024-04-17T10:41:20.456 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 449.985240211, 2024-04-17T10:41:20.456 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.016834464 2024-04-17T10:41:20.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.456 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_reply": 0, 2024-04-17T10:41:20.456 INFO:teuthology.orchestra.run.smithi083.stdout: "forward": 0, 2024-04-17T10:41:20.456 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_complete": 0, 2024-04-17T10:41:20.456 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_keys": 0, 2024-04-17T10:41:20.457 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_commit": 12, 2024-04-17T10:41:20.457 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_split": 0, 2024-04-17T10:41:20.457 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_merge": 0, 2024-04-17T10:41:20.457 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes": 17851, 2024-04-17T10:41:20.457 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_top": 0, 2024-04-17T10:41:20.457 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_bottom": 17851, 2024-04-17T10:41:20.457 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pin_tail": 0, 2024-04-17T10:41:20.457 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pinned": 17851, 2024-04-17T10:41:20.457 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_expired": 0, 2024-04-17T10:41:20.457 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_with_caps": 2, 2024-04-17T10:41:20.457 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": 2, 2024-04-17T10:41:20.457 INFO:teuthology.orchestra.run.smithi083.stdout: "subtrees": 2, 2024-04-17T10:41:20.457 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse": 43729, 2024-04-17T10:41:20.457 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_hit": 43255, 2024-04-17T10:41:20.458 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_forward": 0, 2024-04-17T10:41:20.458 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_discover": 0, 2024-04-17T10:41:20.458 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_dir_fetch": 0, 2024-04-17T10:41:20.458 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_remote_ino": 0, 2024-04-17T10:41:20.458 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_lock": 149, 2024-04-17T10:41:20.458 INFO:teuthology.orchestra.run.smithi083.stdout: "load_cent": 0, 2024-04-17T10:41:20.458 INFO:teuthology.orchestra.run.smithi083.stdout: "q": 0, 2024-04-17T10:41:20.458 INFO:teuthology.orchestra.run.smithi083.stdout: "exported": 0, 2024-04-17T10:41:20.458 INFO:teuthology.orchestra.run.smithi083.stdout: "exported_inodes": 0, 2024-04-17T10:41:20.458 INFO:teuthology.orchestra.run.smithi083.stdout: "imported": 0, 2024-04-17T10:41:20.458 INFO:teuthology.orchestra.run.smithi083.stdout: "imported_inodes": 0, 2024-04-17T10:41:20.458 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_dir_fetch": 0, 2024-04-17T10:41:20.458 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_backtrace_fetch": 1902, 2024-04-17T10:41:20.458 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_peer_discover": 0, 2024-04-17T10:41:20.459 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rfiles": 0, 2024-04-17T10:41:20.459 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rbytes": 0, 2024-04-17T10:41:20.459 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rsnaps": 0, 2024-04-17T10:41:20.459 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_fetch": 1902, 2024-04-17T10:41:20.459 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_set_tag": 0, 2024-04-17T10:41:20.459 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_repaired": 0, 2024-04-17T10:41:20.459 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_inotable_repaired": 0, 2024-04-17T10:41:20.459 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_inodes": 13, 2024-04-17T10:41:20.459 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_base_inodes": 2, 2024-04-17T10:41:20.459 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dirfrag_rstats": 11, 2024-04-17T10:41:20.459 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_file_inodes": 1889, 2024-04-17T10:41:20.459 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_inode_file_caps": 0, 2024-04-17T10:41:20.459 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_revoke": 7781, 2024-04-17T10:41:20.459 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_grant": 6321, 2024-04-17T10:41:20.460 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_trunc": 0, 2024-04-17T10:41:20.460 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-17T10:41:20.460 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-17T10:41:20.460 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps": 7781, 2024-04-17T10:41:20.460 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps_dirty": 2000, 2024-04-17T10:41:20.460 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_cap_release": 53, 2024-04-17T10:41:20.460 INFO:teuthology.orchestra.run.smithi083.stdout: "process_request_cap_release": 17966 2024-04-17T10:41:20.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.460 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache": { 2024-04-17T10:41:20.460 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update": 0, 2024-04-17T10:41:20.460 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update_receipt": 0, 2024-04-17T10:41:20.460 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_try_discover": 0, 2024-04-17T10:41:20.460 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_send_discover": 0, 2024-04-17T10:41:20.461 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_handle_discover": 0, 2024-04-17T10:41:20.461 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays": 0, 2024-04-17T10:41:20.461 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_delayed": 0, 2024-04-17T10:41:20.461 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_enqueuing": 0, 2024-04-17T10:41:20.461 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_created": 4093, 2024-04-17T10:41:20.461 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_enqueued": 3620, 2024-04-17T10:41:20.461 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_reintegrated": 2187, 2024-04-17T10:41:20.461 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_migrated": 0, 2024-04-17T10:41:20.461 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_processing": 0, 2024-04-17T10:41:20.461 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_enqueued": 0, 2024-04-17T10:41:20.461 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_prioritized": 0, 2024-04-17T10:41:20.461 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_started": 0, 2024-04-17T10:41:20.461 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_completed": 0, 2024-04-17T10:41:20.461 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_enqueue_scrub": 1, 2024-04-17T10:41:20.462 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_exportdir": 0, 2024-04-17T10:41:20.462 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_flush": 0, 2024-04-17T10:41:20.462 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragmentdir": 0, 2024-04-17T10:41:20.462 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragstats": 0, 2024-04-17T10:41:20.462 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_inodestats": 0 2024-04-17T10:41:20.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.462 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log": { 2024-04-17T10:41:20.462 INFO:teuthology.orchestra.run.smithi083.stdout: "evlrg": 0, 2024-04-17T10:41:20.462 INFO:teuthology.orchestra.run.smithi083.stdout: "evadd": 25703, 2024-04-17T10:41:20.462 INFO:teuthology.orchestra.run.smithi083.stdout: "evex": 0, 2024-04-17T10:41:20.462 INFO:teuthology.orchestra.run.smithi083.stdout: "evtrm": 0, 2024-04-17T10:41:20.462 INFO:teuthology.orchestra.run.smithi083.stdout: "ev": 25703, 2024-04-17T10:41:20.462 INFO:teuthology.orchestra.run.smithi083.stdout: "evexg": 0, 2024-04-17T10:41:20.463 INFO:teuthology.orchestra.run.smithi083.stdout: "evexd": 0, 2024-04-17T10:41:20.463 INFO:teuthology.orchestra.run.smithi083.stdout: "segadd": 34, 2024-04-17T10:41:20.463 INFO:teuthology.orchestra.run.smithi083.stdout: "segex": 0, 2024-04-17T10:41:20.463 INFO:teuthology.orchestra.run.smithi083.stdout: "segtrm": 0, 2024-04-17T10:41:20.463 INFO:teuthology.orchestra.run.smithi083.stdout: "seg": 34, 2024-04-17T10:41:20.463 INFO:teuthology.orchestra.run.smithi083.stdout: "segmjr": 4, 2024-04-17T10:41:20.463 INFO:teuthology.orchestra.run.smithi083.stdout: "segexg": 0, 2024-04-17T10:41:20.463 INFO:teuthology.orchestra.run.smithi083.stdout: "segexd": 0, 2024-04-17T10:41:20.463 INFO:teuthology.orchestra.run.smithi083.stdout: "expos": 4194304, 2024-04-17T10:41:20.463 INFO:teuthology.orchestra.run.smithi083.stdout: "wrpos": 86476202, 2024-04-17T10:41:20.463 INFO:teuthology.orchestra.run.smithi083.stdout: "rdpos": 0, 2024-04-17T10:41:20.463 INFO:teuthology.orchestra.run.smithi083.stdout: "jlat": { 2024-04-17T10:41:20.463 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 6806, 2024-04-17T10:41:20.464 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 117.042913851, 2024-04-17T10:41:20.464 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.017197019 2024-04-17T10:41:20.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.464 INFO:teuthology.orchestra.run.smithi083.stdout: "replayed": 0 2024-04-17T10:41:20.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.464 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_mem": { 2024-04-17T10:41:20.464 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 14, 2024-04-17T10:41:20.464 INFO:teuthology.orchestra.run.smithi083.stdout: "ino+": 2016, 2024-04-17T10:41:20.464 INFO:teuthology.orchestra.run.smithi083.stdout: "ino-": 2002, 2024-04-17T10:41:20.464 INFO:teuthology.orchestra.run.smithi083.stdout: "dir": 13, 2024-04-17T10:41:20.464 INFO:teuthology.orchestra.run.smithi083.stdout: "dir+": 13, 2024-04-17T10:41:20.464 INFO:teuthology.orchestra.run.smithi083.stdout: "dir-": 0, 2024-04-17T10:41:20.464 INFO:teuthology.orchestra.run.smithi083.stdout: "dn": 17851, 2024-04-17T10:41:20.465 INFO:teuthology.orchestra.run.smithi083.stdout: "dn+": 19851, 2024-04-17T10:41:20.465 INFO:teuthology.orchestra.run.smithi083.stdout: "dn-": 2000, 2024-04-17T10:41:20.465 INFO:teuthology.orchestra.run.smithi083.stdout: "cap": 2, 2024-04-17T10:41:20.465 INFO:teuthology.orchestra.run.smithi083.stdout: "cap+": 2002, 2024-04-17T10:41:20.465 INFO:teuthology.orchestra.run.smithi083.stdout: "cap-": 2000, 2024-04-17T10:41:20.465 INFO:teuthology.orchestra.run.smithi083.stdout: "rss": 91344, 2024-04-17T10:41:20.465 INFO:teuthology.orchestra.run.smithi083.stdout: "heap": 215340 2024-04-17T10:41:20.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.465 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_server": { 2024-04-17T10:41:20.465 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_client_request": 36099, 2024-04-17T10:41:20.465 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_server_request": 0, 2024-04-17T10:41:20.465 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_request": 26730, 2024-04-17T10:41:20.465 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_session": 8, 2024-04-17T10:41:20.465 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_peer_request": 0, 2024-04-17T10:41:20.466 INFO:teuthology.orchestra.run.smithi083.stdout: "req_create_latency": { 2024-04-17T10:41:20.466 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2000, 2024-04-17T10:41:20.466 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 1.389634135, 2024-04-17T10:41:20.466 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000694817 2024-04-17T10:41:20.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.466 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getattr_latency": { 2024-04-17T10:41:20.466 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 515, 2024-04-17T10:41:20.466 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.150340387, 2024-04-17T10:41:20.466 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000291923 2024-04-17T10:41:20.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.466 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getfilelock_latency": { 2024-04-17T10:41:20.466 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:41:20.466 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:41:20.467 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:41:20.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.467 INFO:teuthology.orchestra.run.smithi083.stdout: "req_link_latency": { 2024-04-17T10:41:20.467 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 6000, 2024-04-17T10:41:20.467 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 9.614716615, 2024-04-17T10:41:20.467 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.001602452 2024-04-17T10:41:20.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.467 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookup_latency": { 2024-04-17T10:41:20.467 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 8015, 2024-04-17T10:41:20.467 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 15.699825801, 2024-04-17T10:41:20.467 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.001958805 2024-04-17T10:41:20.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.467 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookuphash_latency": { 2024-04-17T10:41:20.468 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:41:20.468 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:41:20.468 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:41:20.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.468 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupino_latency": { 2024-04-17T10:41:20.468 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:41:20.468 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:41:20.468 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:41:20.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.468 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupname_latency": { 2024-04-17T10:41:20.468 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:41:20.468 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:41:20.468 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:41:20.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.469 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupparent_latency": { 2024-04-17T10:41:20.469 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:41:20.469 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:41:20.469 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:41:20.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.469 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupsnap_latency": { 2024-04-17T10:41:20.469 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3, 2024-04-17T10:41:20.469 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.003287393, 2024-04-17T10:41:20.469 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.001095797 2024-04-17T10:41:20.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.469 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lssnap_latency": { 2024-04-17T10:41:20.469 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:41:20.469 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:41:20.470 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:41:20.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.470 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mkdir_latency": { 2024-04-17T10:41:20.470 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1, 2024-04-17T10:41:20.470 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.002850512, 2024-04-17T10:41:20.470 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.002850512 2024-04-17T10:41:20.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.470 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mknod_latency": { 2024-04-17T10:41:20.470 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:41:20.470 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:41:20.470 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:41:20.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.471 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mksnap_latency": { 2024-04-17T10:41:20.471 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1, 2024-04-17T10:41:20.471 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.005833500, 2024-04-17T10:41:20.471 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.005833500 2024-04-17T10:41:20.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.471 INFO:teuthology.orchestra.run.smithi083.stdout: "req_open_latency": { 2024-04-17T10:41:20.471 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:41:20.471 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:41:20.471 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:41:20.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.471 INFO:teuthology.orchestra.run.smithi083.stdout: "req_readdir_latency": { 2024-04-17T10:41:20.471 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 6, 2024-04-17T10:41:20.471 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.219240218, 2024-04-17T10:41:20.471 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.036540036 2024-04-17T10:41:20.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.472 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rename_latency": { 2024-04-17T10:41:20.472 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2187, 2024-04-17T10:41:20.472 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 392.619088066, 2024-04-17T10:41:20.472 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.179524045 2024-04-17T10:41:20.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.472 INFO:teuthology.orchestra.run.smithi083.stdout: "req_renamesnap_latency": { 2024-04-17T10:41:20.472 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:41:20.472 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:41:20.472 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:41:20.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.472 INFO:teuthology.orchestra.run.smithi083.stdout: "req_snapdiff_latency": { 2024-04-17T10:41:20.472 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:41:20.473 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:41:20.473 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:41:20.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.473 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmdir_latency": { 2024-04-17T10:41:20.473 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:41:20.473 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:41:20.473 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:41:20.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.473 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmsnap_latency": { 2024-04-17T10:41:20.473 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1, 2024-04-17T10:41:20.473 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.003107278, 2024-04-17T10:41:20.473 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.003107278 2024-04-17T10:41:20.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.473 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmxattr_latency": { 2024-04-17T10:41:20.474 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:41:20.474 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:41:20.474 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:41:20.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.474 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setattr_latency": { 2024-04-17T10:41:20.474 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1, 2024-04-17T10:41:20.474 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000555286, 2024-04-17T10:41:20.474 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000555286 2024-04-17T10:41:20.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.474 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setdirlayout_latency": { 2024-04-17T10:41:20.474 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:41:20.474 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:41:20.474 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:41:20.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.474 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setfilelock_latency": { 2024-04-17T10:41:20.475 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:41:20.475 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:41:20.475 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:41:20.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.475 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setlayout_latency": { 2024-04-17T10:41:20.475 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:41:20.475 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:41:20.475 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:41:20.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.475 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setxattr_latency": { 2024-04-17T10:41:20.475 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:41:20.475 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:41:20.475 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:41:20.476 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.476 INFO:teuthology.orchestra.run.smithi083.stdout: "req_symlink_latency": { 2024-04-17T10:41:20.476 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:41:20.476 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:41:20.476 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:41:20.476 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.476 INFO:teuthology.orchestra.run.smithi083.stdout: "req_unlink_latency": { 2024-04-17T10:41:20.476 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 8000, 2024-04-17T10:41:20.476 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 30.276761020, 2024-04-17T10:41:20.476 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.003784595 2024-04-17T10:41:20.476 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.476 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_revoke_eviction": 0, 2024-04-17T10:41:20.476 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_acquisition_throttle": 0, 2024-04-17T10:41:20.477 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getvxattr_latency": { 2024-04-17T10:41:20.477 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:41:20.477 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:41:20.477 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:41:20.477 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:41:20.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.477 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_sessions": { 2024-04-17T10:41:20.477 INFO:teuthology.orchestra.run.smithi083.stdout: "session_count": 1, 2024-04-17T10:41:20.477 INFO:teuthology.orchestra.run.smithi083.stdout: "session_add": 1, 2024-04-17T10:41:20.477 INFO:teuthology.orchestra.run.smithi083.stdout: "session_remove": 0, 2024-04-17T10:41:20.477 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_open": 1, 2024-04-17T10:41:20.477 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_stale": 0, 2024-04-17T10:41:20.477 INFO:teuthology.orchestra.run.smithi083.stdout: "total_load": 15015, 2024-04-17T10:41:20.477 INFO:teuthology.orchestra.run.smithi083.stdout: "average_load": 15015, 2024-04-17T10:41:20.478 INFO:teuthology.orchestra.run.smithi083.stdout: "avg_session_uptime": 116, 2024-04-17T10:41:20.478 INFO:teuthology.orchestra.run.smithi083.stdout: "mdthresh_evicted": 0 2024-04-17T10:41:20.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.478 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-04-17T10:41:20.478 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-04-17T10:41:20.478 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-04-17T10:41:20.478 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T10:41:20.478 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-04-17T10:41:20.478 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T10:41:20.478 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-04-17T10:41:20.478 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T10:41:20.478 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T10:41:20.479 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T10:41:20.479 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T10:41:20.479 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T10:41:20.479 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-04-17T10:41:20.479 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T10:41:20.479 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T10:41:20.479 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-04-17T10:41:20.479 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-04-17T10:41:20.479 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-04-17T10:41:20.479 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-04-17T10:41:20.479 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T10:41:20.479 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T10:41:20.479 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T10:41:20.480 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T10:41:20.480 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T10:41:20.480 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-04-17T10:41:20.480 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-04-17T10:41:20.480 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-04-17T10:41:20.480 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T10:41:20.480 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T10:41:20.480 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-04-17T10:41:20.480 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-04-17T10:41:20.480 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-04-17T10:41:20.480 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-04-17T10:41:20.480 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T10:41:20.480 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-04-17T10:41:20.480 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T10:41:20.481 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-04-17T10:41:20.481 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 251493, 2024-04-17T10:41:20.481 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 86, 2024-04-17T10:41:20.481 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-04-17T10:41:20.481 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-04-17T10:41:20.481 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-04-17T10:41:20.481 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-04-17T10:41:20.481 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-04-17T10:41:20.481 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-04-17T10:41:20.481 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-04-17T10:41:20.481 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-04-17T10:41:20.481 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 5416, 2024-04-17T10:41:20.481 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 77, 2024-04-17T10:41:20.481 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T10:41:20.482 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-04-17T10:41:20.482 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-04-17T10:41:20.482 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-04-17T10:41:20.482 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 13441572, 2024-04-17T10:41:20.482 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 661012, 2024-04-17T10:41:20.482 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-04-17T10:41:20.482 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-04-17T10:41:20.482 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-04-17T10:41:20.482 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-04-17T10:41:20.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.482 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter": { 2024-04-17T10:41:20.482 INFO:teuthology.orchestra.run.smithi083.stdout: "op_active": 0, 2024-04-17T10:41:20.482 INFO:teuthology.orchestra.run.smithi083.stdout: "op_laggy": 0, 2024-04-17T10:41:20.483 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send": 12783, 2024-04-17T10:41:20.483 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send_bytes": 82772557, 2024-04-17T10:41:20.483 INFO:teuthology.orchestra.run.smithi083.stdout: "op_resend": 37, 2024-04-17T10:41:20.483 INFO:teuthology.orchestra.run.smithi083.stdout: "op_reply": 12783, 2024-04-17T10:41:20.483 INFO:teuthology.orchestra.run.smithi083.stdout: "op_latency": { 2024-04-17T10:41:20.483 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 12783, 2024-04-17T10:41:20.483 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 56.334878875, 2024-04-17T10:41:20.483 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.004407015 2024-04-17T10:41:20.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.483 INFO:teuthology.orchestra.run.smithi083.stdout: "op_inflight": 0, 2024-04-17T10:41:20.483 INFO:teuthology.orchestra.run.smithi083.stdout: "oplen_avg": { 2024-04-17T10:41:20.483 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 12783, 2024-04-17T10:41:20.483 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 12945 2024-04-17T10:41:20.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.484 INFO:teuthology.orchestra.run.smithi083.stdout: "op": 12783, 2024-04-17T10:41:20.484 INFO:teuthology.orchestra.run.smithi083.stdout: "op_r": 2024, 2024-04-17T10:41:20.484 INFO:teuthology.orchestra.run.smithi083.stdout: "op_w": 10759, 2024-04-17T10:41:20.484 INFO:teuthology.orchestra.run.smithi083.stdout: "op_rmw": 0, 2024-04-17T10:41:20.484 INFO:teuthology.orchestra.run.smithi083.stdout: "op_pg": 0, 2024-04-17T10:41:20.484 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_stat": 0, 2024-04-17T10:41:20.484 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_create": 11, 2024-04-17T10:41:20.484 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_read": 120, 2024-04-17T10:41:20.484 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_write": 6924, 2024-04-17T10:41:20.484 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writefull": 68, 2024-04-17T10:41:20.484 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writesame": 0, 2024-04-17T10:41:20.484 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_append": 0, 2024-04-17T10:41:20.484 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_zero": 1620, 2024-04-17T10:41:20.484 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_truncate": 0, 2024-04-17T10:41:20.485 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_delete": 2031, 2024-04-17T10:41:20.485 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_mapext": 0, 2024-04-17T10:41:20.485 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_sparse_read": 0, 2024-04-17T10:41:20.485 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_clonerange": 0, 2024-04-17T10:41:20.485 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_getxattr": 1904, 2024-04-17T10:41:20.485 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_setxattr": 22, 2024-04-17T10:41:20.485 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_cmpxattr": 0, 2024-04-17T10:41:20.485 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_rmxattr": 0, 2024-04-17T10:41:20.485 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_resetxattrs": 0, 2024-04-17T10:41:20.485 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_call": 0, 2024-04-17T10:41:20.485 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_watch": 0, 2024-04-17T10:41:20.485 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_notify": 0, 2024-04-17T10:41:20.485 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_src_cmpxattr": 0, 2024-04-17T10:41:20.486 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls": 0, 2024-04-17T10:41:20.486 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls_filter": 0, 2024-04-17T10:41:20.486 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_other": 0, 2024-04-17T10:41:20.486 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_active": 0, 2024-04-17T10:41:20.486 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_send": 0, 2024-04-17T10:41:20.486 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_resend": 0, 2024-04-17T10:41:20.486 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_ping": 0, 2024-04-17T10:41:20.486 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_active": 0, 2024-04-17T10:41:20.486 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_send": 0, 2024-04-17T10:41:20.486 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_resend": 0, 2024-04-17T10:41:20.486 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_active": 0, 2024-04-17T10:41:20.486 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_send": 0, 2024-04-17T10:41:20.486 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_resend": 0, 2024-04-17T10:41:20.487 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_active": 0, 2024-04-17T10:41:20.487 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_send": 0, 2024-04-17T10:41:20.487 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_resend": 0, 2024-04-17T10:41:20.487 INFO:teuthology.orchestra.run.smithi083.stdout: "command_active": 0, 2024-04-17T10:41:20.487 INFO:teuthology.orchestra.run.smithi083.stdout: "command_send": 0, 2024-04-17T10:41:20.487 INFO:teuthology.orchestra.run.smithi083.stdout: "command_resend": 0, 2024-04-17T10:41:20.487 INFO:teuthology.orchestra.run.smithi083.stdout: "map_epoch": 35, 2024-04-17T10:41:20.487 INFO:teuthology.orchestra.run.smithi083.stdout: "map_full": 0, 2024-04-17T10:41:20.487 INFO:teuthology.orchestra.run.smithi083.stdout: "map_inc": 3, 2024-04-17T10:41:20.487 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_sessions": 8, 2024-04-17T10:41:20.487 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_open": 8, 2024-04-17T10:41:20.487 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_close": 0, 2024-04-17T10:41:20.487 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_laggy": 0, 2024-04-17T10:41:20.488 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_wr": 193, 2024-04-17T10:41:20.488 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_rd": 0, 2024-04-17T10:41:20.488 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_del": 52 2024-04-17T10:41:20.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.488 INFO:teuthology.orchestra.run.smithi083.stdout: "oft": { 2024-04-17T10:41:20.488 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_objs": 1, 2024-04-17T10:41:20.488 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_kv_pairs": 0, 2024-04-17T10:41:20.488 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_updates": 0, 2024-04-17T10:41:20.488 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_removes": 0 2024-04-17T10:41:20.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.488 INFO:teuthology.orchestra.run.smithi083.stdout: "osd-slow-ops": { 2024-04-17T10:41:20.488 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_ops_count": 0 2024-04-17T10:41:20.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.488 INFO:teuthology.orchestra.run.smithi083.stdout: "purge_queue": { 2024-04-17T10:41:20.489 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops": 0, 2024-04-17T10:41:20.489 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops_high_water": 32, 2024-04-17T10:41:20.489 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing": 0, 2024-04-17T10:41:20.489 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_high_water": 32, 2024-04-17T10:41:20.489 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed_ops": 3620, 2024-04-17T10:41:20.489 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed": 3620, 2024-04-17T10:41:20.489 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_item_in_journal": 0 2024-04-17T10:41:20.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.489 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T10:41:20.489 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-17T10:41:20.489 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-17T10:41:20.489 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-17T10:41:20.489 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 45430, 2024-04-17T10:41:20.490 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 11417805, 2024-04-17T10:41:20.490 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-17T10:41:20.490 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 45430, 2024-04-17T10:41:20.490 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-17T10:41:20.490 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-17T10:41:20.490 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 45430, 2024-04-17T10:41:20.490 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 11417805, 2024-04-17T10:41:20.490 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-17T10:41:20.490 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:41:20.490 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:41:20.490 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:41:20.490 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:41:20.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.491 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_bytes": { 2024-04-17T10:41:20.491 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-17T10:41:20.491 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-17T10:41:20.491 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-17T10:41:20.491 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-17T10:41:20.491 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-17T10:41:20.491 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-17T10:41:20.491 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-17T10:41:20.491 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 12783, 2024-04-17T10:41:20.491 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 83151137, 2024-04-17T10:41:20.491 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 12783, 2024-04-17T10:41:20.491 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 83151137, 2024-04-17T10:41:20.492 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-17T10:41:20.492 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:41:20.492 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:41:20.492 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:41:20.492 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:41:20.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.492 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_ops": { 2024-04-17T10:41:20.492 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-17T10:41:20.492 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 1024, 2024-04-17T10:41:20.492 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-17T10:41:20.492 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-17T10:41:20.492 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-17T10:41:20.492 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-17T10:41:20.492 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-17T10:41:20.493 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 12783, 2024-04-17T10:41:20.493 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 12783, 2024-04-17T10:41:20.493 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 12783, 2024-04-17T10:41:20.493 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 12783, 2024-04-17T10:41:20.493 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-17T10:41:20.493 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:41:20.493 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:41:20.493 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:41:20.493 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:41:20.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.493 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle": { 2024-04-17T10:41:20.493 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-17T10:41:20.493 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-04-17T10:41:20.494 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-17T10:41:20.494 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 3620, 2024-04-17T10:41:20.494 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 378580, 2024-04-17T10:41:20.494 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-17T10:41:20.494 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 3620, 2024-04-17T10:41:20.494 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-17T10:41:20.494 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-17T10:41:20.494 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 118, 2024-04-17T10:41:20.494 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 378580, 2024-04-17T10:41:20.494 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-17T10:41:20.494 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:41:20.494 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:41:20.494 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:41:20.495 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:41:20.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:20.495 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle-0x55812d868960": { 2024-04-17T10:41:20.495 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-17T10:41:20.495 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-04-17T10:41:20.495 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-17T10:41:20.495 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 25703, 2024-04-17T10:41:20.495 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 82281898, 2024-04-17T10:41:20.495 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-17T10:41:20.495 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 25703, 2024-04-17T10:41:20.495 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-17T10:41:20.495 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-17T10:41:20.495 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 6806, 2024-04-17T10:41:20.496 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 82281898, 2024-04-17T10:41:20.496 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-17T10:41:20.496 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:41:20.496 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:41:20.496 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:41:20.496 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:41:20.496 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:41:20.496 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:41:20.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.447+0000 7f5d43ee6640 1 -- 172.21.15.83:0/349125802 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5d20066f60 msgr2=0x7f5d20069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:20.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.447+0000 7f5d43ee6640 1 --2- 172.21.15.83:0/349125802 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5d20066f60 0x7f5d20069420 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5d38000f70 tx=0x7f5d38009040 comp rx=0 tx=0).stop 2024-04-17T10:41:20.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.447+0000 7f5d43ee6640 1 -- 172.21.15.83:0/349125802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d3c15b6e0 msgr2=0x7f5d3c14e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:20.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.447+0000 7f5d43ee6640 1 --2- 172.21.15.83:0/349125802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d3c15b6e0 0x7f5d3c14e170 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c01a040 tx=0x7f5d2c0191b0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.447+0000 7f5d43ee6640 1 -- 172.21.15.83:0/349125802 shutdown_connections 2024-04-17T10:41:20.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.447+0000 7f5d43ee6640 1 --2- 172.21.15.83:0/349125802 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5d20066f60 0x7f5d20069420 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.447+0000 7f5d43ee6640 1 --2- 172.21.15.83:0/349125802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d3c15bac0 0x7f5d3c158f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.447+0000 7f5d43ee6640 1 --2- 172.21.15.83:0/349125802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d3c15b6e0 0x7f5d3c14e170 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.447+0000 7f5d43ee6640 1 --2- 172.21.15.83:0/349125802 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d3c070470 0x7f5d3c14dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.447+0000 7f5d43ee6640 1 -- 172.21.15.83:0/349125802 >> 172.21.15.83:0/349125802 conn(0x7f5d3c0b15b0 msgr2=0x7f5d3c0b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:20.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.447+0000 7f5d43ee6640 1 -- 172.21.15.83:0/349125802 shutdown_connections 2024-04-17T10:41:20.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.447+0000 7f5d43ee6640 1 -- 172.21.15.83:0/349125802 wait complete. 2024-04-17T10:41:20.499 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T10:41:20.499 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:41:20.548 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:41:20.599 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T10:41:20.600 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:41:20.600 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:41:20.605 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:41:20.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.771+0000 7f943242c640 1 Processor -- start 2024-04-17T10:41:20.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.771+0000 7f943242c640 1 -- start start 2024-04-17T10:41:20.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.775+0000 7f943242c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f942c15ba30 0x7f942c072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.775+0000 7f943242c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f942c0cfbe0 0x7f942c06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.775+0000 7f943242c640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f942c06e320 con 0x7f942c15c960 2024-04-17T10:41:20.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.775+0000 7f943242c640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f942c071ea0 con 0x7f942c15ba30 2024-04-17T10:41:20.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.775+0000 7f943242c640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f942c06e040 con 0x7f942c0cfbe0 2024-04-17T10:41:20.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.775+0000 7f942bfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f942c15ba30 0x7f942c072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.775+0000 7f942b7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f942c15c960 0x7f942c0cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54000/0 (socket says 172.21.15.83:54000) 2024-04-17T10:41:20.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.775+0000 7f942b7fe640 1 -- 172.21.15.83:0/570928414 learned_addr learned my addr 172.21.15.83:0/570928414 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:20.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.775+0000 7f94309c9640 1 --2- 172.21.15.83:0/570928414 >> v2:172.21.15.167:3300/0 conn(0x7f942c0cfbe0 0x7f942c06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.775+0000 7f942affd640 1 -- 172.21.15.83:0/570928414 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3564896292 0 0) 0x7f942c06e320 con 0x7f942c15c960 2024-04-17T10:41:20.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.775+0000 7f942affd640 1 -- 172.21.15.83:0/570928414 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9418003530 con 0x7f942c15c960 2024-04-17T10:41:20.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.775+0000 7f942bfff640 1 -- 172.21.15.83:0/570928414 >> v2:172.21.15.167:3300/0 conn(0x7f942c0cfbe0 msgr2=0x7f942c06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:20.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.775+0000 7f942bfff640 1 --2- 172.21.15.83:0/570928414 >> v2:172.21.15.167:3300/0 conn(0x7f942c0cfbe0 0x7f942c06c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.775+0000 7f942bfff640 1 -- 172.21.15.83:0/570928414 >> v1:172.21.15.83:6789/0 conn(0x7f942c15c960 legacy=0x7f942c0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:20.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.775+0000 7f942bfff640 1 -- 172.21.15.83:0/570928414 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c071850 con 0x7f942c15ba30 2024-04-17T10:41:20.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.775+0000 7f942bfff640 1 --2- 172.21.15.83:0/570928414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f942c15ba30 0x7f942c072330 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f942c0676a0 tx=0x7f941c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc75aa355034387 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:20.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.775+0000 7f942affd640 1 -- 172.21.15.83:0/570928414 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c0b0070 con 0x7f942c15ba30 2024-04-17T10:41:20.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.775+0000 7f942affd640 1 -- 172.21.15.83:0/570928414 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941c0ab600 con 0x7f942c15ba30 2024-04-17T10:41:20.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.775+0000 7f942affd640 1 -- 172.21.15.83:0/570928414 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c0ab9d0 con 0x7f942c15ba30 2024-04-17T10:41:20.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.779+0000 7f943242c640 1 -- 172.21.15.83:0/570928414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f942c15ba30 msgr2=0x7f942c072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:20.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.779+0000 7f943242c640 1 --2- 172.21.15.83:0/570928414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f942c15ba30 0x7f942c072330 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f942c0676a0 tx=0x7f941c01c3f0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.779+0000 7f943242c640 1 -- 172.21.15.83:0/570928414 shutdown_connections 2024-04-17T10:41:20.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.779+0000 7f943242c640 1 --2- 172.21.15.83:0/570928414 >> v2:172.21.15.167:3300/0 conn(0x7f942c0cfbe0 0x7f942c06c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.779+0000 7f943242c640 1 --2- 172.21.15.83:0/570928414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f942c15ba30 0x7f942c072330 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.779+0000 7f943242c640 1 -- 172.21.15.83:0/570928414 >> 172.21.15.83:0/570928414 conn(0x7f942c06abe0 msgr2=0x7f942c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:20.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.779+0000 7f943242c640 1 -- 172.21.15.83:0/570928414 shutdown_connections 2024-04-17T10:41:20.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.779+0000 7f943242c640 1 -- 172.21.15.83:0/570928414 wait complete. 2024-04-17T10:41:20.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.779+0000 7f943242c640 1 Processor -- start 2024-04-17T10:41:20.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.779+0000 7f943242c640 1 -- start start 2024-04-17T10:41:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.779+0000 7f943242c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f942c0cfbe0 0x7f942c1583a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.779+0000 7f943242c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f942c15ba30 0x7f942c150000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.779+0000 7f943242c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f942c15c960 0x7f942c159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.779+0000 7f943242c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c15cee0 con 0x7f942c15c960 2024-04-17T10:41:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.779+0000 7f943242c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c15cd60 con 0x7f942c15ba30 2024-04-17T10:41:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.779+0000 7f943242c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942c168520 con 0x7f942c0cfbe0 2024-04-17T10:41:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.779+0000 7f94309c9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f942c15c960 0x7f942c159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.779+0000 7f94309c9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f942c15c960 0x7f942c159d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36444/0 (socket says 172.21.15.83:36444) 2024-04-17T10:41:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.779+0000 7f94309c9640 1 -- 172.21.15.83:0/3076791543 learned_addr learned my addr 172.21.15.83:0/3076791543 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.779+0000 7f942b7fe640 1 --2- 172.21.15.83:0/3076791543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f942c15ba30 0x7f942c150000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.779+0000 7f94309c9640 1 -- 172.21.15.83:0/3076791543 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f942c0cfbe0 msgr2=0x7f942c1583a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:20.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.779+0000 7f94309c9640 1 --2- 172.21.15.83:0/3076791543 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f942c0cfbe0 0x7f942c1583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.779+0000 7f94309c9640 1 -- 172.21.15.83:0/3076791543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f942c15ba30 msgr2=0x7f942c150000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:20.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.779+0000 7f94309c9640 1 --2- 172.21.15.83:0/3076791543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f942c15ba30 0x7f942c150000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.779+0000 7f94309c9640 1 -- 172.21.15.83:0/3076791543 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c158c60 con 0x7f942c15c960 2024-04-17T10:41:20.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.779+0000 7f94309c9640 1 --2- 172.21.15.83:0/3076791543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f942c15c960 0x7f942c159d00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9424007e90 tx=0x7f942400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:20.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.779+0000 7f9428ff9640 1 -- 172.21.15.83:0/3076791543 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9424014420 con 0x7f942c15c960 2024-04-17T10:41:20.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.779+0000 7f9428ff9640 1 -- 172.21.15.83:0/3076791543 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9424019070 con 0x7f942c15c960 2024-04-17T10:41:20.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.779+0000 7f943242c640 1 -- 172.21.15.83:0/3076791543 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f942c158f40 con 0x7f942c15c960 2024-04-17T10:41:20.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.779+0000 7f9428ff9640 1 -- 172.21.15.83:0/3076791543 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9424014800 con 0x7f942c15c960 2024-04-17T10:41:20.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.779+0000 7f943242c640 1 -- 172.21.15.83:0/3076791543 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f942c159500 con 0x7f942c15c960 2024-04-17T10:41:20.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.783+0000 7f943242c640 1 -- 172.21.15.83:0/3076791543 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f942c074f90 con 0x7f942c15c960 2024-04-17T10:41:20.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.783+0000 7f9428ff9640 1 -- 172.21.15.83:0/3076791543 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f94240149a0 con 0x7f942c15c960 2024-04-17T10:41:20.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.783+0000 7f9428ff9640 1 --2- 172.21.15.83:0/3076791543 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f940c066f60 0x7f940c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:20.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.783+0000 7f9428ff9640 1 -- 172.21.15.83:0/3076791543 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f9424085810 con 0x7f942c15c960 2024-04-17T10:41:20.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.783+0000 7f942bfff640 1 --2- 172.21.15.83:0/3076791543 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f940c066f60 0x7f940c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:20.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.783+0000 7f942bfff640 1 --2- 172.21.15.83:0/3076791543 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f940c066f60 0x7f940c069420 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f941c0ad000 tx=0x7f941c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:20.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.787+0000 7f9428ff9640 1 -- 172.21.15.83:0/3076791543 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9424056890 con 0x7f942c15c960 2024-04-17T10:41:20.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.939+0000 7f943242c640 1 -- 172.21.15.83:0/3076791543 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f942c074480 con 0x7f942c15c960 2024-04-17T10:41:20.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.943+0000 7f9428ff9640 1 -- 172.21.15.83:0/3076791543 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16662 (secure 0 0 0) 0x7f9424056190 con 0x7f942c15c960 2024-04-17T10:41:20.940 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:41:20.940 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":35,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:40:57.038108+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:39:11.330695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-17T10:39:12.438377+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":34,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:41:20.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.943+0000 7f943242c640 1 -- 172.21.15.83:0/3076791543 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f940c066f60 msgr2=0x7f940c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:20.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.943+0000 7f943242c640 1 --2- 172.21.15.83:0/3076791543 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f940c066f60 0x7f940c069420 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f941c0ad000 tx=0x7f941c008040 comp rx=0 tx=0).stop 2024-04-17T10:41:20.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.943+0000 7f943242c640 1 -- 172.21.15.83:0/3076791543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f942c15c960 msgr2=0x7f942c159d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:20.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.943+0000 7f943242c640 1 --2- 172.21.15.83:0/3076791543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f942c15c960 0x7f942c159d00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9424007e90 tx=0x7f942400e4b0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.943+0000 7f943242c640 1 -- 172.21.15.83:0/3076791543 shutdown_connections 2024-04-17T10:41:20.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.943+0000 7f943242c640 1 --2- 172.21.15.83:0/3076791543 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f940c066f60 0x7f940c069420 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.943+0000 7f943242c640 1 --2- 172.21.15.83:0/3076791543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f942c15c960 0x7f942c159d00 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.943+0000 7f943242c640 1 --2- 172.21.15.83:0/3076791543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f942c15ba30 0x7f942c150000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.943+0000 7f943242c640 1 --2- 172.21.15.83:0/3076791543 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f942c0cfbe0 0x7f942c1583a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:20.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.943+0000 7f943242c640 1 -- 172.21.15.83:0/3076791543 >> 172.21.15.83:0/3076791543 conn(0x7f942c06abe0 msgr2=0x7f942c0b0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:20.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.943+0000 7f943242c640 1 -- 172.21.15.83:0/3076791543 shutdown_connections 2024-04-17T10:41:20.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:20.943+0000 7f943242c640 1 -- 172.21.15.83:0/3076791543 wait complete. 2024-04-17T10:41:20.957 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:39:11.330695+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:39:12.438377+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 34, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:41:20.958 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:41:21.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.067+0000 7f47129ef640 1 Processor -- start 2024-04-17T10:41:21.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.067+0000 7f47129ef640 1 -- start start 2024-04-17T10:41:21.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.067+0000 7f47129ef640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f470c15e0f0 0x7f470c0b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:21.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.067+0000 7f47129ef640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f470c0ce400 0x7f470c161c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:21.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.067+0000 7f47129ef640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f470c0b5450 0x7f470c0b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:21.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.067+0000 7f47129ef640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f470c0b5df0 con 0x7f470c0ce400 2024-04-17T10:41:21.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.067+0000 7f47129ef640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f470c162b40 con 0x7f470c0b5450 2024-04-17T10:41:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.067+0000 7f47129ef640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f470c15e4d0 con 0x7f470c15e0f0 2024-04-17T10:41:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.067+0000 7f470b7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f470c0ce400 0x7f470c161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.067+0000 7f470b7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f470c0ce400 0x7f470c161c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36446/0 (socket says 172.21.15.83:36446) 2024-04-17T10:41:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.067+0000 7f470b7fe640 1 -- 172.21.15.83:0/3366219881 learned_addr learned my addr 172.21.15.83:0/3366219881 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.067+0000 7f470b7fe640 1 -- 172.21.15.83:0/3366219881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f470c15e0f0 msgr2=0x7f470c0b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.067+0000 7f470b7fe640 1 --2- 172.21.15.83:0/3366219881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f470c15e0f0 0x7f470c0b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.067+0000 7f470b7fe640 1 -- 172.21.15.83:0/3366219881 >> v2:172.21.15.167:3300/0 conn(0x7f470c0b5450 msgr2=0x7f470c0b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.067+0000 7f470b7fe640 1 --2- 172.21.15.83:0/3366219881 >> v2:172.21.15.167:3300/0 conn(0x7f470c0b5450 0x7f470c0b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.067+0000 7f470b7fe640 1 -- 172.21.15.83:0/3366219881 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f470c162580 con 0x7f470c0ce400 2024-04-17T10:41:21.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.071+0000 7f470b7fe640 1 --2- 172.21.15.83:0/3366219881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c0ce400 0x7f470c161c10 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f470c0b94e0 tx=0x7f470001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cee6c305e497d71 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:21.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.071+0000 7f470affd640 1 -- 172.21.15.83:0/3366219881 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47000b0070 con 0x7f470c0ce400 2024-04-17T10:41:21.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.071+0000 7f470affd640 1 -- 172.21.15.83:0/3366219881 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47000ab600 con 0x7f470c0ce400 2024-04-17T10:41:21.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.071+0000 7f470affd640 1 -- 172.21.15.83:0/3366219881 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47000ab9d0 con 0x7f470c0ce400 2024-04-17T10:41:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.071+0000 7f47129ef640 1 -- 172.21.15.83:0/3366219881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c0ce400 msgr2=0x7f470c161c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.071+0000 7f47129ef640 1 --2- 172.21.15.83:0/3366219881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c0ce400 0x7f470c161c10 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f470c0b94e0 tx=0x7f470001c490 comp rx=0 tx=0).stop 2024-04-17T10:41:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.071+0000 7f47129ef640 1 -- 172.21.15.83:0/3366219881 shutdown_connections 2024-04-17T10:41:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.071+0000 7f47129ef640 1 --2- 172.21.15.83:0/3366219881 >> v2:172.21.15.167:3300/0 conn(0x7f470c0b5450 0x7f470c0b58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.071+0000 7f47129ef640 1 --2- 172.21.15.83:0/3366219881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c0ce400 0x7f470c161c10 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.071+0000 7f47129ef640 1 --2- 172.21.15.83:0/3366219881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f470c15e0f0 0x7f470c0b6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.071+0000 7f47129ef640 1 -- 172.21.15.83:0/3366219881 >> 172.21.15.83:0/3366219881 conn(0x7f470c0d0450 msgr2=0x7f470c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:21.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.071+0000 7f47129ef640 1 -- 172.21.15.83:0/3366219881 shutdown_connections 2024-04-17T10:41:21.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.071+0000 7f47129ef640 1 -- 172.21.15.83:0/3366219881 wait complete. 2024-04-17T10:41:21.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.071+0000 7f47129ef640 1 Processor -- start 2024-04-17T10:41:21.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.071+0000 7f47129ef640 1 -- start start 2024-04-17T10:41:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.075+0000 7f47129ef640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c0b5450 0x7f470c161100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.075+0000 7f47129ef640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f470c0ce400 0x7f470c15e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.075+0000 7f47129ef640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f470c15e0f0 0x7f470c14b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.075+0000 7f47129ef640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c1622d0 con 0x7f470c0b5450 2024-04-17T10:41:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.075+0000 7f47129ef640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c162150 con 0x7f470c0ce400 2024-04-17T10:41:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.075+0000 7f47129ef640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f470c0d2d50 con 0x7f470c15e0f0 2024-04-17T10:41:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.075+0000 7f470bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c0b5450 0x7f470c161100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.075+0000 7f470bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c0b5450 0x7f470c161100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36450/0 (socket says 172.21.15.83:36450) 2024-04-17T10:41:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.075+0000 7f470bfff640 1 -- 172.21.15.83:0/3688279195 learned_addr learned my addr 172.21.15.83:0/3688279195 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.075+0000 7f470b7fe640 1 --2- 172.21.15.83:0/3688279195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f470c0ce400 0x7f470c15e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.075+0000 7f4710f8c640 1 --2- 172.21.15.83:0/3688279195 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f470c15e0f0 0x7f470c14b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.075+0000 7f470bfff640 1 -- 172.21.15.83:0/3688279195 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f470c15e0f0 msgr2=0x7f470c14b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.075+0000 7f470bfff640 1 --2- 172.21.15.83:0/3688279195 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f470c15e0f0 0x7f470c14b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.075+0000 7f470bfff640 1 -- 172.21.15.83:0/3688279195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f470c0ce400 msgr2=0x7f470c15e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.075+0000 7f470bfff640 1 --2- 172.21.15.83:0/3688279195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f470c0ce400 0x7f470c15e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.075+0000 7f470bfff640 1 -- 172.21.15.83:0/3688279195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f470c154f90 con 0x7f470c0b5450 2024-04-17T10:41:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.075+0000 7f4710f8c640 1 --2- 172.21.15.83:0/3688279195 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f470c15e0f0 0x7f470c14b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.075+0000 7f470bfff640 1 --2- 172.21.15.83:0/3688279195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c0b5450 0x7f470c161100 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00b440 tx=0x7f46fc00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.075+0000 7f4708ff9640 1 -- 172.21.15.83:0/3688279195 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46fc015020 con 0x7f470c0b5450 2024-04-17T10:41:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.075+0000 7f4708ff9640 1 -- 172.21.15.83:0/3688279195 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46fc0118b0 con 0x7f470c0b5450 2024-04-17T10:41:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.075+0000 7f47129ef640 1 -- 172.21.15.83:0/3688279195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f470c15b890 con 0x7f470c0b5450 2024-04-17T10:41:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.075+0000 7f4708ff9640 1 -- 172.21.15.83:0/3688279195 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46fc011ca0 con 0x7f470c0b5450 2024-04-17T10:41:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.075+0000 7f47129ef640 1 -- 172.21.15.83:0/3688279195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f470c14b840 con 0x7f470c0b5450 2024-04-17T10:41:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.075+0000 7f47129ef640 1 -- 172.21.15.83:0/3688279195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f470c0681a0 con 0x7f470c0b5450 2024-04-17T10:41:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.075+0000 7f4708ff9640 1 -- 172.21.15.83:0/3688279195 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f46fc018020 con 0x7f470c0b5450 2024-04-17T10:41:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.075+0000 7f4708ff9640 1 --2- 172.21.15.83:0/3688279195 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f46f8066f90 0x7f46f8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.075+0000 7f470b7fe640 1 --2- 172.21.15.83:0/3688279195 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f46f8066f90 0x7f46f8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.075+0000 7f4708ff9640 1 -- 172.21.15.83:0/3688279195 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f46fc085120 con 0x7f470c0b5450 2024-04-17T10:41:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.075+0000 7f470b7fe640 1 --2- 172.21.15.83:0/3688279195 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f46f8066f90 0x7f46f8069450 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f470c0d4a30 tx=0x7f47000a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:21.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.079+0000 7f4708ff9640 1 -- 172.21.15.83:0/3688279195 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f46fc01f020 con 0x7f470c0b5450 2024-04-17T10:41:21.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.231+0000 7f47129ef640 1 -- 172.21.15.83:0/3688279195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f470c074f90 con 0x7f470c0b5450 2024-04-17T10:41:21.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.231+0000 7f4708ff9640 1 -- 172.21.15.83:0/3688279195 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7f46fc00d040 con 0x7f470c0b5450 2024-04-17T10:41:21.229 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:41:21.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.235+0000 7f47129ef640 1 -- 172.21.15.83:0/3688279195 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f46f8066f90 msgr2=0x7f46f8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:21.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.235+0000 7f47129ef640 1 --2- 172.21.15.83:0/3688279195 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f46f8066f90 0x7f46f8069450 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f470c0d4a30 tx=0x7f47000a6000 comp rx=0 tx=0).stop 2024-04-17T10:41:21.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.235+0000 7f47129ef640 1 -- 172.21.15.83:0/3688279195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c0b5450 msgr2=0x7f470c161100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:21.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.235+0000 7f47129ef640 1 --2- 172.21.15.83:0/3688279195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c0b5450 0x7f470c161100 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00b440 tx=0x7f46fc00b900 comp rx=0 tx=0).stop 2024-04-17T10:41:21.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.235+0000 7f47129ef640 1 -- 172.21.15.83:0/3688279195 shutdown_connections 2024-04-17T10:41:21.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.235+0000 7f47129ef640 1 --2- 172.21.15.83:0/3688279195 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f46f8066f90 0x7f46f8069450 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.235+0000 7f47129ef640 1 --2- 172.21.15.83:0/3688279195 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f470c15e0f0 0x7f470c14b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.235+0000 7f47129ef640 1 --2- 172.21.15.83:0/3688279195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f470c0ce400 0x7f470c15e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.235+0000 7f47129ef640 1 --2- 172.21.15.83:0/3688279195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c0b5450 0x7f470c161100 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.235+0000 7f47129ef640 1 -- 172.21.15.83:0/3688279195 >> 172.21.15.83:0/3688279195 conn(0x7f470c0d0450 msgr2=0x7f470c0cf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:21.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.235+0000 7f47129ef640 1 -- 172.21.15.83:0/3688279195 shutdown_connections 2024-04-17T10:41:21.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.235+0000 7f47129ef640 1 -- 172.21.15.83:0/3688279195 wait complete. 2024-04-17T10:41:21.246 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:41:21.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.371+0000 7f2845b98640 1 Processor -- start 2024-04-17T10:41:21.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.375+0000 7f2845b98640 1 -- start start 2024-04-17T10:41:21.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.375+0000 7f2845b98640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f284015bac0 0x7f2840071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:21.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.375+0000 7f2845b98640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2840070470 0x7f284006ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:21.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.375+0000 7f2845b98640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f284006c1f0 con 0x7f284015b6e0 2024-04-17T10:41:21.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.375+0000 7f2845b98640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2840070a10 con 0x7f284015bac0 2024-04-17T10:41:21.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.375+0000 7f2845b98640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2840070cf0 con 0x7f2840070470 2024-04-17T10:41:21.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.375+0000 7f283effd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f284015bac0 0x7f2840071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:21.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.375+0000 7f283effd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f284015bac0 0x7f2840071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36458/0 (socket says 172.21.15.83:36458) 2024-04-17T10:41:21.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.375+0000 7f283effd640 1 -- 172.21.15.83:0/3173729804 learned_addr learned my addr 172.21.15.83:0/3173729804 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:21.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.375+0000 7f283ffff640 1 --2- 172.21.15.83:0/3173729804 >> v2:172.21.15.167:3300/0 conn(0x7f2840070470 0x7f284006ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:21.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.375+0000 7f283effd640 1 -- 172.21.15.83:0/3173729804 >> v2:172.21.15.167:3300/0 conn(0x7f2840070470 msgr2=0x7f284006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:21.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.375+0000 7f283effd640 1 --2- 172.21.15.83:0/3173729804 >> v2:172.21.15.167:3300/0 conn(0x7f2840070470 0x7f284006ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.375+0000 7f283effd640 1 -- 172.21.15.83:0/3173729804 >> v1:172.21.15.167:6789/0 conn(0x7f284015b6e0 legacy=0x7f28400cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:21.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.375+0000 7f283effd640 1 -- 172.21.15.83:0/3173729804 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2840070fd0 con 0x7f284015bac0 2024-04-17T10:41:21.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.375+0000 7f283effd640 1 --2- 172.21.15.83:0/3173729804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f284015bac0 0x7f2840071cc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2840071c50 tx=0x7f283401c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d57e7bbb73d018 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:21.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f283e7fc640 1 -- 172.21.15.83:0/3173729804 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28340b0070 con 0x7f284015bac0 2024-04-17T10:41:21.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f283e7fc640 1 -- 172.21.15.83:0/3173729804 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28340ab600 con 0x7f284015bac0 2024-04-17T10:41:21.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f283e7fc640 1 -- 172.21.15.83:0/3173729804 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28340ab9d0 con 0x7f284015bac0 2024-04-17T10:41:21.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f2845b98640 1 -- 172.21.15.83:0/3173729804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f284015bac0 msgr2=0x7f2840071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:21.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f2845b98640 1 --2- 172.21.15.83:0/3173729804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f284015bac0 0x7f2840071cc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2840071c50 tx=0x7f283401c4c0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f2845b98640 1 -- 172.21.15.83:0/3173729804 shutdown_connections 2024-04-17T10:41:21.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f2845b98640 1 --2- 172.21.15.83:0/3173729804 >> v2:172.21.15.167:3300/0 conn(0x7f2840070470 0x7f284006ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f2845b98640 1 --2- 172.21.15.83:0/3173729804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f284015bac0 0x7f2840071cc0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f2845b98640 1 -- 172.21.15.83:0/3173729804 >> 172.21.15.83:0/3173729804 conn(0x7f28400b15b0 msgr2=0x7f28400b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:21.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f2845b98640 1 -- 172.21.15.83:0/3173729804 shutdown_connections 2024-04-17T10:41:21.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f2845b98640 1 -- 172.21.15.83:0/3173729804 wait complete. 2024-04-17T10:41:21.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f2845b98640 1 Processor -- start 2024-04-17T10:41:21.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f2845b98640 1 -- start start 2024-04-17T10:41:21.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f2845b98640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2840070470 0x7f284014db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:21.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f2845b98640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f284015b6e0 0x7f284014e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:21.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f2845b98640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f284015bac0 0x7f284014aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:21.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f2845b98640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284006d2f0 con 0x7f284015b6e0 2024-04-17T10:41:21.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f2845b98640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284006d170 con 0x7f284015bac0 2024-04-17T10:41:21.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f2845b98640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f284006b760 con 0x7f2840070470 2024-04-17T10:41:21.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f283effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f284015b6e0 0x7f284014e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:21.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f283effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f284015b6e0 0x7f284014e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36470/0 (socket says 172.21.15.83:36470) 2024-04-17T10:41:21.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f283effd640 1 -- 172.21.15.83:0/1669281607 learned_addr learned my addr 172.21.15.83:0/1669281607 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:21.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f283ffff640 1 --2- 172.21.15.83:0/1669281607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f284015bac0 0x7f284014aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:21.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f283f7fe640 1 --2- 172.21.15.83:0/1669281607 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2840070470 0x7f284014db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:21.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f283effd640 1 -- 172.21.15.83:0/1669281607 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2840070470 msgr2=0x7f284014db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:21.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f283effd640 1 --2- 172.21.15.83:0/1669281607 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2840070470 0x7f284014db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f283effd640 1 -- 172.21.15.83:0/1669281607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f284015bac0 msgr2=0x7f284014aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:21.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f283effd640 1 --2- 172.21.15.83:0/1669281607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f284015bac0 0x7f284014aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f283effd640 1 -- 172.21.15.83:0/1669281607 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2840158670 con 0x7f284015b6e0 2024-04-17T10:41:21.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f283ffff640 1 --2- 172.21.15.83:0/1669281607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f284015bac0 0x7f284014aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:21.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f283effd640 1 --2- 172.21.15.83:0/1669281607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f284015b6e0 0x7f284014e0b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f283401ab70 tx=0x7f283401ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:21.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f2827fff640 1 -- 172.21.15.83:0/1669281607 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28340ba020 con 0x7f284015b6e0 2024-04-17T10:41:21.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f2827fff640 1 -- 172.21.15.83:0/1669281607 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28340b6640 con 0x7f284015b6e0 2024-04-17T10:41:21.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f2845b98640 1 -- 172.21.15.83:0/1669281607 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2840158950 con 0x7f284015b6e0 2024-04-17T10:41:21.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f2827fff640 1 -- 172.21.15.83:0/1669281607 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28340b0040 con 0x7f284015b6e0 2024-04-17T10:41:21.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.379+0000 7f2845b98640 1 -- 172.21.15.83:0/1669281607 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2840158e80 con 0x7f284015b6e0 2024-04-17T10:41:21.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.383+0000 7f2845b98640 1 -- 172.21.15.83:0/1669281607 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2840155b10 con 0x7f284015b6e0 2024-04-17T10:41:21.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.383+0000 7f2827fff640 1 -- 172.21.15.83:0/1669281607 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f28340bd020 con 0x7f284015b6e0 2024-04-17T10:41:21.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.383+0000 7f2827fff640 1 --2- 172.21.15.83:0/1669281607 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f282c066f60 0x7f282c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:21.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.383+0000 7f2827fff640 1 -- 172.21.15.83:0/1669281607 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f283411f4a0 con 0x7f284015b6e0 2024-04-17T10:41:21.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.383+0000 7f283f7fe640 1 --2- 172.21.15.83:0/1669281607 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f282c066f60 0x7f282c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:21.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.383+0000 7f283f7fe640 1 --2- 172.21.15.83:0/1669281607 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f282c066f60 0x7f282c069420 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2830003d60 tx=0x7f2830009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:21.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.387+0000 7f2827fff640 1 -- 172.21.15.83:0/1669281607 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2840155b10 con 0x7f284015b6e0 2024-04-17T10:41:21.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.543+0000 7f2845b98640 1 -- 172.21.15.83:0/1669281607 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2840074f90 con 0x7f284015b6e0 2024-04-17T10:41:21.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.543+0000 7f2827fff640 1 -- 172.21.15.83:0/1669281607 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+11 (secure 0 0 0) 0x7f28340c4030 con 0x7f284015b6e0 2024-04-17T10:41:21.542 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:41:21.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.547+0000 7f2845b98640 1 -- 172.21.15.83:0/1669281607 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f282c066f60 msgr2=0x7f282c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:21.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.547+0000 7f2845b98640 1 --2- 172.21.15.83:0/1669281607 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f282c066f60 0x7f282c069420 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2830003d60 tx=0x7f2830009040 comp rx=0 tx=0).stop 2024-04-17T10:41:21.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.547+0000 7f2845b98640 1 -- 172.21.15.83:0/1669281607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f284015b6e0 msgr2=0x7f284014e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:21.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.547+0000 7f2845b98640 1 --2- 172.21.15.83:0/1669281607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f284015b6e0 0x7f284014e0b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f283401ab70 tx=0x7f283401ad00 comp rx=0 tx=0).stop 2024-04-17T10:41:21.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.547+0000 7f2845b98640 1 -- 172.21.15.83:0/1669281607 shutdown_connections 2024-04-17T10:41:21.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.547+0000 7f2845b98640 1 --2- 172.21.15.83:0/1669281607 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f282c066f60 0x7f282c069420 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.547+0000 7f2845b98640 1 --2- 172.21.15.83:0/1669281607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f284015bac0 0x7f284014aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.547+0000 7f2845b98640 1 --2- 172.21.15.83:0/1669281607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f284015b6e0 0x7f284014e0b0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.547+0000 7f2845b98640 1 --2- 172.21.15.83:0/1669281607 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2840070470 0x7f284014db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.547+0000 7f2845b98640 1 -- 172.21.15.83:0/1669281607 >> 172.21.15.83:0/1669281607 conn(0x7f28400b15b0 msgr2=0x7f28400b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:21.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.547+0000 7f2845b98640 1 -- 172.21.15.83:0/1669281607 shutdown_connections 2024-04-17T10:41:21.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.547+0000 7f2845b98640 1 -- 172.21.15.83:0/1669281607 wait complete. 2024-04-17T10:41:21.559 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:41:21.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.683+0000 7f9575de6640 1 Processor -- start 2024-04-17T10:41:21.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.683+0000 7f9575de6640 1 -- start start 2024-04-17T10:41:21.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.683+0000 7f9575de6640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f957016a950 0x7f957008b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:21.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.683+0000 7f9575de6640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f957016ad30 0x7f957006c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:21.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.683+0000 7f9575de6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f957006c730 0x7f95700b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:21.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.683+0000 7f9575de6640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f957006cb80 con 0x7f957016ad30 2024-04-17T10:41:21.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.683+0000 7f9575de6640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f957016b1e0 con 0x7f957016a950 2024-04-17T10:41:21.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.683+0000 7f9575de6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9570071cd0 con 0x7f957006c730 2024-04-17T10:41:21.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.683+0000 7f956effd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f957016ad30 0x7f957006c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:21.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.683+0000 7f956effd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f957016ad30 0x7f957006c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36474/0 (socket says 172.21.15.83:36474) 2024-04-17T10:41:21.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.683+0000 7f956f7fe640 1 --2- 172.21.15.83:0/357688085 >> v2:172.21.15.167:3300/0 conn(0x7f957016a950 0x7f957008b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:21.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.683+0000 7f956ffff640 1 --2- 172.21.15.83:0/357688085 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f957006c730 0x7f95700b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:21.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.683+0000 7f956effd640 1 -- 172.21.15.83:0/357688085 learned_addr learned my addr 172.21.15.83:0/357688085 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:21.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.687+0000 7f956ffff640 1 -- 172.21.15.83:0/357688085 >> v2:172.21.15.167:3300/0 conn(0x7f957016a950 msgr2=0x7f957008b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:21.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.687+0000 7f956ffff640 1 --2- 172.21.15.83:0/357688085 >> v2:172.21.15.167:3300/0 conn(0x7f957016a950 0x7f957008b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.687+0000 7f956ffff640 1 -- 172.21.15.83:0/357688085 >> v2:172.21.15.83:3300/0 conn(0x7f957016ad30 msgr2=0x7f957006c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:21.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.687+0000 7f956ffff640 1 --2- 172.21.15.83:0/357688085 >> v2:172.21.15.83:3300/0 conn(0x7f957016ad30 0x7f957006c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.687+0000 7f956ffff640 1 -- 172.21.15.83:0/357688085 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95700b2f30 con 0x7f957006c730 2024-04-17T10:41:21.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.687+0000 7f956f7fe640 1 --2- 172.21.15.83:0/357688085 >> v2:172.21.15.167:3300/0 conn(0x7f957016a950 0x7f957008b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:41:21.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.687+0000 7f956effd640 1 --2- 172.21.15.83:0/357688085 >> v2:172.21.15.83:3300/0 conn(0x7f957016ad30 0x7f957006c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:41:21.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.691+0000 7f956ffff640 1 --2- 172.21.15.83:0/357688085 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f957006c730 0x7f95700b3cf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9570072630 tx=0x7f9568020ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=205b5434167d9785 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:21.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.691+0000 7f956e7fc640 1 -- 172.21.15.83:0/357688085 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956800e2d0 con 0x7f957006c730 2024-04-17T10:41:21.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.691+0000 7f956e7fc640 1 -- 172.21.15.83:0/357688085 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9568007020 con 0x7f957006c730 2024-04-17T10:41:21.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.691+0000 7f956e7fc640 1 -- 172.21.15.83:0/357688085 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95680a9950 con 0x7f957006c730 2024-04-17T10:41:21.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.691+0000 7f9575de6640 1 -- 172.21.15.83:0/357688085 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f957006c730 msgr2=0x7f95700b3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:21.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.691+0000 7f9575de6640 1 --2- 172.21.15.83:0/357688085 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f957006c730 0x7f95700b3cf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9570072630 tx=0x7f9568020ac0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.691+0000 7f9575de6640 1 -- 172.21.15.83:0/357688085 shutdown_connections 2024-04-17T10:41:21.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.691+0000 7f9575de6640 1 --2- 172.21.15.83:0/357688085 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f957006c730 0x7f95700b3cf0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.691+0000 7f9575de6640 1 --2- 172.21.15.83:0/357688085 >> v2:172.21.15.83:3300/0 conn(0x7f957016ad30 0x7f957006c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.691+0000 7f9575de6640 1 --2- 172.21.15.83:0/357688085 >> v2:172.21.15.167:3300/0 conn(0x7f957016a950 0x7f957008b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.691+0000 7f9575de6640 1 -- 172.21.15.83:0/357688085 >> 172.21.15.83:0/357688085 conn(0x7f95700cf630 msgr2=0x7f95700cfa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:21.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.691+0000 7f9575de6640 1 -- 172.21.15.83:0/357688085 shutdown_connections 2024-04-17T10:41:21.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.691+0000 7f9575de6640 1 -- 172.21.15.83:0/357688085 wait complete. 2024-04-17T10:41:21.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.691+0000 7f9575de6640 1 Processor -- start 2024-04-17T10:41:21.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.691+0000 7f9575de6640 1 -- start start 2024-04-17T10:41:21.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.691+0000 7f9575de6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f957006c730 0x7f9570089670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:21.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.691+0000 7f9575de6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f957016a950 0x7f9570089cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:21.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.691+0000 7f9575de6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f957016ad30 0x7f957008a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:21.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.691+0000 7f9575de6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95700705f0 con 0x7f957016a950 2024-04-17T10:41:21.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.691+0000 7f9575de6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9570070470 con 0x7f957016ad30 2024-04-17T10:41:21.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.691+0000 7f9575de6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9570070770 con 0x7f957006c730 2024-04-17T10:41:21.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.691+0000 7f956effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f957016a950 0x7f9570089cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:21.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.691+0000 7f956effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f957016a950 0x7f9570089cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36476/0 (socket says 172.21.15.83:36476) 2024-04-17T10:41:21.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.691+0000 7f956effd640 1 -- 172.21.15.83:0/2481838718 learned_addr learned my addr 172.21.15.83:0/2481838718 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:21.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.691+0000 7f956effd640 1 -- 172.21.15.83:0/2481838718 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f957006c730 msgr2=0x7f9570089670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:21.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.691+0000 7f956effd640 1 --2- 172.21.15.83:0/2481838718 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f957006c730 0x7f9570089670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.691+0000 7f956effd640 1 -- 172.21.15.83:0/2481838718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f957016ad30 msgr2=0x7f957008a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:21.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.691+0000 7f956effd640 1 --2- 172.21.15.83:0/2481838718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f957016ad30 0x7f957008a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.691+0000 7f956effd640 1 -- 172.21.15.83:0/2481838718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957008b550 con 0x7f957016a950 2024-04-17T10:41:21.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.695+0000 7f956effd640 1 --2- 172.21.15.83:0/2481838718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f957016a950 0x7f9570089cf0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f956400e930 tx=0x7f956400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:21.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.695+0000 7f9557fff640 1 -- 172.21.15.83:0/2481838718 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9564017070 con 0x7f957016a950 2024-04-17T10:41:21.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.695+0000 7f9557fff640 1 -- 172.21.15.83:0/2481838718 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956400ca00 con 0x7f957016a950 2024-04-17T10:41:21.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.695+0000 7f9575de6640 1 -- 172.21.15.83:0/2481838718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f957008a860 con 0x7f957016a950 2024-04-17T10:41:21.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.695+0000 7f9557fff640 1 -- 172.21.15.83:0/2481838718 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956400cdd0 con 0x7f957016a950 2024-04-17T10:41:21.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.695+0000 7f9575de6640 1 -- 172.21.15.83:0/2481838718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f957008ae20 con 0x7f957016a950 2024-04-17T10:41:21.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.695+0000 7f9575de6640 1 -- 172.21.15.83:0/2481838718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f957008b020 con 0x7f957016a950 2024-04-17T10:41:21.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.695+0000 7f9557fff640 1 -- 172.21.15.83:0/2481838718 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f956401a070 con 0x7f957016a950 2024-04-17T10:41:21.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.695+0000 7f9557fff640 1 --2- 172.21.15.83:0/2481838718 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f955c066f60 0x7f955c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:21.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.695+0000 7f956f7fe640 1 --2- 172.21.15.83:0/2481838718 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f955c066f60 0x7f955c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:21.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.695+0000 7f9557fff640 1 -- 172.21.15.83:0/2481838718 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f9564084f80 con 0x7f957016a950 2024-04-17T10:41:21.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.695+0000 7f956f7fe640 1 --2- 172.21.15.83:0/2481838718 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f955c066f60 0x7f955c069420 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f956000b7b0 tx=0x7f9560005c70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:21.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.699+0000 7f9557fff640 1 -- 172.21.15.83:0/2481838718 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f957008b020 con 0x7f957016a950 2024-04-17T10:41:21.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.851+0000 7f9575de6640 1 -- 172.21.15.83:0/2481838718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9570074f90 con 0x7f957016a950 2024-04-17T10:41:21.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.855+0000 7f9557fff640 1 -- 172.21.15.83:0/2481838718 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9564055ee0 con 0x7f957016a950 2024-04-17T10:41:21.851 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:41:21.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.855+0000 7f9575de6640 1 -- 172.21.15.83:0/2481838718 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f955c066f60 msgr2=0x7f955c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:21.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.855+0000 7f9575de6640 1 --2- 172.21.15.83:0/2481838718 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f955c066f60 0x7f955c069420 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f956000b7b0 tx=0x7f9560005c70 comp rx=0 tx=0).stop 2024-04-17T10:41:21.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.855+0000 7f9575de6640 1 -- 172.21.15.83:0/2481838718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f957016a950 msgr2=0x7f9570089cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:21.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.855+0000 7f9575de6640 1 --2- 172.21.15.83:0/2481838718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f957016a950 0x7f9570089cf0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f956400e930 tx=0x7f956400edf0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.855+0000 7f9575de6640 1 -- 172.21.15.83:0/2481838718 shutdown_connections 2024-04-17T10:41:21.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.855+0000 7f9575de6640 1 --2- 172.21.15.83:0/2481838718 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f955c066f60 0x7f955c069420 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.855+0000 7f9575de6640 1 --2- 172.21.15.83:0/2481838718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f957016ad30 0x7f957008a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.855+0000 7f9575de6640 1 --2- 172.21.15.83:0/2481838718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f957016a950 0x7f9570089cf0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.855+0000 7f9575de6640 1 --2- 172.21.15.83:0/2481838718 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f957006c730 0x7f9570089670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.855+0000 7f9575de6640 1 -- 172.21.15.83:0/2481838718 >> 172.21.15.83:0/2481838718 conn(0x7f95700cf630 msgr2=0x7f957006b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:21.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.855+0000 7f9575de6640 1 -- 172.21.15.83:0/2481838718 shutdown_connections 2024-04-17T10:41:21.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.855+0000 7f9575de6640 1 -- 172.21.15.83:0/2481838718 wait complete. 2024-04-17T10:41:21.869 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:41:21.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.983+0000 7f61ce4b0640 1 Processor -- start 2024-04-17T10:41:21.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.983+0000 7f61ce4b0640 1 -- start start 2024-04-17T10:41:21.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.983+0000 7f61ce4b0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61c815b6e0 0x7f61c8071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:21.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.983+0000 7f61ce4b0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f61c815bac0 0x7f61c80cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:21.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.983+0000 7f61ce4b0640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c806c1f0 con 0x7f61c8070470 2024-04-17T10:41:21.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.983+0000 7f61ce4b0640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c80cdd30 con 0x7f61c815bac0 2024-04-17T10:41:21.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.983+0000 7f61ce4b0640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c8070a10 con 0x7f61c815b6e0 2024-04-17T10:41:21.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.983+0000 7f61c77fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f61c815bac0 0x7f61c80cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:21.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.983+0000 7f61cca4d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f61c8070470 0x7f61c806ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54008/0 (socket says 172.21.15.83:54008) 2024-04-17T10:41:21.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.983+0000 7f61cca4d640 1 -- 172.21.15.83:0/3223231625 learned_addr learned my addr 172.21.15.83:0/3223231625 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:21.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.983+0000 7f61c77fe640 1 -- 172.21.15.83:0/3223231625 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61c815b6e0 msgr2=0x7f61c8071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:21.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.983+0000 7f61c77fe640 1 --2- 172.21.15.83:0/3223231625 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61c815b6e0 0x7f61c8071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.983+0000 7f61c77fe640 1 -- 172.21.15.83:0/3223231625 >> v1:172.21.15.83:6789/0 conn(0x7f61c8070470 legacy=0x7f61c806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:21.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.983+0000 7f61c6ffd640 1 -- 172.21.15.83:0/3223231625 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3225793310 0 0) 0x7f61c806c1f0 con 0x7f61c8070470 2024-04-17T10:41:21.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.983+0000 7f61c77fe640 1 -- 172.21.15.83:0/3223231625 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c8070cf0 con 0x7f61c815bac0 2024-04-17T10:41:21.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.987+0000 7f61c77fe640 1 --2- 172.21.15.83:0/3223231625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61c815bac0 0x7f61c80cd7f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f61b00097c0 tx=0x7f61b001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef75b16f71d5207e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:21.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.987+0000 7f61c6ffd640 1 -- 172.21.15.83:0/3223231625 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b00b0070 con 0x7f61c815bac0 2024-04-17T10:41:21.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.987+0000 7f61c6ffd640 1 -- 172.21.15.83:0/3223231625 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b00ab600 con 0x7f61c815bac0 2024-04-17T10:41:21.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.987+0000 7f61c6ffd640 1 -- 172.21.15.83:0/3223231625 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b00ab9d0 con 0x7f61c815bac0 2024-04-17T10:41:21.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.987+0000 7f61ce4b0640 1 -- 172.21.15.83:0/3223231625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61c815bac0 msgr2=0x7f61c80cd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:21.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.987+0000 7f61ce4b0640 1 --2- 172.21.15.83:0/3223231625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61c815bac0 0x7f61c80cd7f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f61b00097c0 tx=0x7f61b001c4a0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.987+0000 7f61ce4b0640 1 -- 172.21.15.83:0/3223231625 shutdown_connections 2024-04-17T10:41:21.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.987+0000 7f61ce4b0640 1 --2- 172.21.15.83:0/3223231625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61c815bac0 0x7f61c80cd7f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.987+0000 7f61ce4b0640 1 --2- 172.21.15.83:0/3223231625 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61c815b6e0 0x7f61c8071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.987+0000 7f61ce4b0640 1 -- 172.21.15.83:0/3223231625 >> 172.21.15.83:0/3223231625 conn(0x7f61c80b15b0 msgr2=0x7f61c80b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:21.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.987+0000 7f61ce4b0640 1 -- 172.21.15.83:0/3223231625 shutdown_connections 2024-04-17T10:41:21.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.987+0000 7f61ce4b0640 1 -- 172.21.15.83:0/3223231625 wait complete. 2024-04-17T10:41:21.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.987+0000 7f61ce4b0640 1 Processor -- start 2024-04-17T10:41:21.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.987+0000 7f61ce4b0640 1 -- start start 2024-04-17T10:41:21.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.987+0000 7f61ce4b0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61c8070470 0x7f61c8145660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:21.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.987+0000 7f61ce4b0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61c815b6e0 0x7f61c8157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:21.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.987+0000 7f61ce4b0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61c815bac0 0x7f61c8145d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:21.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.987+0000 7f61ce4b0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c806cee0 con 0x7f61c815b6e0 2024-04-17T10:41:21.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.987+0000 7f61ce4b0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c806b730 con 0x7f61c815bac0 2024-04-17T10:41:21.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.987+0000 7f61ce4b0640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c806d060 con 0x7f61c8070470 2024-04-17T10:41:21.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.987+0000 7f61c77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61c815b6e0 0x7f61c8157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:21.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.987+0000 7f61c77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61c815b6e0 0x7f61c8157c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36506/0 (socket says 172.21.15.83:36506) 2024-04-17T10:41:21.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.987+0000 7f61c77fe640 1 -- 172.21.15.83:0/1014692335 learned_addr learned my addr 172.21.15.83:0/1014692335 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:21.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.987+0000 7f61cca4d640 1 --2- 172.21.15.83:0/1014692335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61c815bac0 0x7f61c8145d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:21.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.987+0000 7f61c77fe640 1 -- 172.21.15.83:0/1014692335 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61c8070470 msgr2=0x7f61c8145660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:21.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.987+0000 7f61c77fe640 1 --2- 172.21.15.83:0/1014692335 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61c8070470 0x7f61c8145660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.987+0000 7f61c77fe640 1 -- 172.21.15.83:0/1014692335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61c815bac0 msgr2=0x7f61c8145d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:21.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.987+0000 7f61c77fe640 1 --2- 172.21.15.83:0/1014692335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61c815bac0 0x7f61c8145d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:21.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.987+0000 7f61c77fe640 1 -- 172.21.15.83:0/1014692335 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c814a6f0 con 0x7f61c815b6e0 2024-04-17T10:41:21.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.991+0000 7f61c77fe640 1 --2- 172.21.15.83:0/1014692335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61c815b6e0 0x7f61c8157c20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f61b00ae040 tx=0x7f61b001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:21.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.991+0000 7f61c4ff9640 1 -- 172.21.15.83:0/1014692335 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b00b0050 con 0x7f61c815b6e0 2024-04-17T10:41:21.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.991+0000 7f61c4ff9640 1 -- 172.21.15.83:0/1014692335 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b00bc070 con 0x7f61c815b6e0 2024-04-17T10:41:21.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.991+0000 7f61ce4b0640 1 -- 172.21.15.83:0/1014692335 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c814a9d0 con 0x7f61c815b6e0 2024-04-17T10:41:21.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.991+0000 7f61c4ff9640 1 -- 172.21.15.83:0/1014692335 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b00b5a70 con 0x7f61c815b6e0 2024-04-17T10:41:21.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.991+0000 7f61ce4b0640 1 -- 172.21.15.83:0/1014692335 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c8158160 con 0x7f61c815b6e0 2024-04-17T10:41:21.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.991+0000 7f61ce4b0640 1 -- 172.21.15.83:0/1014692335 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c814ae80 con 0x7f61c815b6e0 2024-04-17T10:41:21.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.991+0000 7f61c4ff9640 1 -- 172.21.15.83:0/1014692335 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f61b00ab4c0 con 0x7f61c815b6e0 2024-04-17T10:41:21.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.991+0000 7f61c4ff9640 1 --2- 172.21.15.83:0/1014692335 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f61a8066f60 0x7f61a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:21.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.991+0000 7f61c4ff9640 1 -- 172.21.15.83:0/1014692335 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f61b011f940 con 0x7f61c815b6e0 2024-04-17T10:41:21.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.991+0000 7f61c7fff640 1 --2- 172.21.15.83:0/1014692335 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f61a8066f60 0x7f61a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:21.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.991+0000 7f61c7fff640 1 --2- 172.21.15.83:0/1014692335 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f61a8066f60 0x7f61a8069420 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f61b800ab90 tx=0x7f61b8009250 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:21.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:21.995+0000 7f61c4ff9640 1 -- 172.21.15.83:0/1014692335 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f61b0016090 con 0x7f61c815b6e0 2024-04-17T10:41:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.159+0000 7f61ce4b0640 1 -- 172.21.15.83:0/1014692335 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f61c8074f90 con 0x7f61c815b6e0 2024-04-17T10:41:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.159+0000 7f61c4ff9640 1 -- 172.21.15.83:0/1014692335 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) v1 ==== 80+0+0 (secure 0 0 0) 0x7f61b00f0930 con 0x7f61c815b6e0 2024-04-17T10:41:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-17T10:41:22.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.163+0000 7f61ce4b0640 1 -- 172.21.15.83:0/1014692335 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f61a8066f60 msgr2=0x7f61a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:22.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.163+0000 7f61ce4b0640 1 --2- 172.21.15.83:0/1014692335 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f61a8066f60 0x7f61a8069420 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f61b800ab90 tx=0x7f61b8009250 comp rx=0 tx=0).stop 2024-04-17T10:41:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.163+0000 7f61ce4b0640 1 -- 172.21.15.83:0/1014692335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61c815b6e0 msgr2=0x7f61c8157c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.163+0000 7f61ce4b0640 1 --2- 172.21.15.83:0/1014692335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61c815b6e0 0x7f61c8157c20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f61b00ae040 tx=0x7f61b001ace0 comp rx=0 tx=0).stop 2024-04-17T10:41:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.163+0000 7f61ce4b0640 1 -- 172.21.15.83:0/1014692335 shutdown_connections 2024-04-17T10:41:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.163+0000 7f61ce4b0640 1 --2- 172.21.15.83:0/1014692335 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f61a8066f60 0x7f61a8069420 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.163+0000 7f61ce4b0640 1 --2- 172.21.15.83:0/1014692335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61c815bac0 0x7f61c8145d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.163+0000 7f61ce4b0640 1 --2- 172.21.15.83:0/1014692335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61c815b6e0 0x7f61c8157c20 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.163+0000 7f61ce4b0640 1 --2- 172.21.15.83:0/1014692335 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61c8070470 0x7f61c8145660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.163+0000 7f61ce4b0640 1 -- 172.21.15.83:0/1014692335 >> 172.21.15.83:0/1014692335 conn(0x7f61c80b15b0 msgr2=0x7f61c80b0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.163+0000 7f61ce4b0640 1 -- 172.21.15.83:0/1014692335 shutdown_connections 2024-04-17T10:41:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.163+0000 7f61ce4b0640 1 -- 172.21.15.83:0/1014692335 wait complete. 2024-04-17T10:41:22.175 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2024-04-17T10:41:22.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:41:22.175 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-17T10:41:22.184 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[37888]: fuse finished with error 0 and tester_r 0 2024-04-17T10:41:22.188 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-17T10:41:22.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:41:22.188 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T10:41:22.358 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:41:22.358 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:41:22.365 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:41:22.365 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-04-17T10:41:22.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:41:22.365 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-17T10:41:22.410 INFO:teuthology.orchestra.run.smithi167.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-17T10:41:22.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:41:22.412 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:41:22.412 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:41:22.418 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16063 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:41:22.418 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-17T10:41:22.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:41:22.418 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-17T10:41:22.463 INFO:teuthology.orchestra.run.smithi083.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-17T10:41:22.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:41:22.465 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:41:22.465 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:41:22.472 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:41:22.472 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-04-17T10:41:22.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:41:22.472 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-17T10:41:22.518 INFO:teuthology.orchestra.run.smithi167.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-17T10:41:22.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:41:22.520 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:41:22.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.651+0000 7f556edeb640 1 Processor -- start 2024-04-17T10:41:22.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.651+0000 7f556edeb640 1 -- start start 2024-04-17T10:41:22.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.651+0000 7f556edeb640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55680d0040 0x7f55680d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:22.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.651+0000 7f556edeb640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f55680d04a0 0x7f55680d1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:22.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.651+0000 7f556edeb640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55680b1e10 con 0x7f556815bb40 2024-04-17T10:41:22.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.651+0000 7f556edeb640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55680b2a20 con 0x7f55680d04a0 2024-04-17T10:41:22.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.651+0000 7f556edeb640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55680b20f0 con 0x7f55680d0040 2024-04-17T10:41:22.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.651+0000 7f556cb87640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f556815bb40 0x7f55680b2eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54020/0 (socket says 172.21.15.83:54020) 2024-04-17T10:41:22.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.651+0000 7f556d388640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f55680d04a0 0x7f55680d1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:22.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.651+0000 7f556cb87640 1 -- 172.21.15.83:0/1358414856 learned_addr learned my addr 172.21.15.83:0/1358414856 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:22.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.651+0000 7f556d388640 1 -- 172.21.15.83:0/1358414856 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55680d0040 msgr2=0x7f55680d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:22.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.651+0000 7f556d388640 1 --2- 172.21.15.83:0/1358414856 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55680d0040 0x7f55680d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:22.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.651+0000 7f556d388640 1 -- 172.21.15.83:0/1358414856 >> v1:172.21.15.83:6789/0 conn(0x7f556815bb40 legacy=0x7f55680b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:22.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.651+0000 7f55677fe640 1 -- 172.21.15.83:0/1358414856 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3242597891 0 0) 0x7f55680b1e10 con 0x7f556815bb40 2024-04-17T10:41:22.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.651+0000 7f556d388640 1 -- 172.21.15.83:0/1358414856 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55680b23d0 con 0x7f55680d04a0 2024-04-17T10:41:22.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.655+0000 7f556d388640 1 --2- 172.21.15.83:0/1358414856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55680d04a0 0x7f55680d1cc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5560009d90 tx=0x7f556001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc017125b56eb0e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:22.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.655+0000 7f55677fe640 1 -- 172.21.15.83:0/1358414856 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55600b2070 con 0x7f55680d04a0 2024-04-17T10:41:22.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.655+0000 7f55677fe640 1 -- 172.21.15.83:0/1358414856 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55600a57a0 con 0x7f55680d04a0 2024-04-17T10:41:22.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.655+0000 7f55677fe640 1 -- 172.21.15.83:0/1358414856 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55600a5b70 con 0x7f55680d04a0 2024-04-17T10:41:22.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.655+0000 7f556edeb640 1 -- 172.21.15.83:0/1358414856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55680d04a0 msgr2=0x7f55680d1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:22.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.655+0000 7f556edeb640 1 --2- 172.21.15.83:0/1358414856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55680d04a0 0x7f55680d1cc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5560009d90 tx=0x7f556001c8e0 comp rx=0 tx=0).stop 2024-04-17T10:41:22.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.655+0000 7f556edeb640 1 -- 172.21.15.83:0/1358414856 shutdown_connections 2024-04-17T10:41:22.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.655+0000 7f556edeb640 1 --2- 172.21.15.83:0/1358414856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55680d04a0 0x7f55680d1cc0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:22.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.655+0000 7f556edeb640 1 --2- 172.21.15.83:0/1358414856 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55680d0040 0x7f55680d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:22.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.655+0000 7f556edeb640 1 -- 172.21.15.83:0/1358414856 >> 172.21.15.83:0/1358414856 conn(0x7f556809a9d0 msgr2=0x7f556809ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:22.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.655+0000 7f556edeb640 1 -- 172.21.15.83:0/1358414856 shutdown_connections 2024-04-17T10:41:22.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.655+0000 7f556edeb640 1 -- 172.21.15.83:0/1358414856 wait complete. 2024-04-17T10:41:22.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.655+0000 7f556edeb640 1 Processor -- start 2024-04-17T10:41:22.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.655+0000 7f556edeb640 1 -- start start 2024-04-17T10:41:22.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.655+0000 7f556edeb640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55680d0040 0x7f556806d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:22.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.655+0000 7f556edeb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55680d04a0 0x7f55681f6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:22.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.655+0000 7f556edeb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f556815bb40 0x7f556815a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:22.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.655+0000 7f556edeb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55680ccc90 con 0x7f556815bb40 2024-04-17T10:41:22.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.655+0000 7f556edeb640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55680ccb10 con 0x7f55680d04a0 2024-04-17T10:41:22.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.655+0000 7f556edeb640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55680cce10 con 0x7f55680d0040 2024-04-17T10:41:22.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.655+0000 7f556d388640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f556815bb40 0x7f556815a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:22.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.655+0000 7f556d388640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f556815bb40 0x7f556815a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36534/0 (socket says 172.21.15.83:36534) 2024-04-17T10:41:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.655+0000 7f556d388640 1 -- 172.21.15.83:0/4067691036 learned_addr learned my addr 172.21.15.83:0/4067691036 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.655+0000 7f556cb87640 1 --2- 172.21.15.83:0/4067691036 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55680d0040 0x7f556806d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.655+0000 7f5567fff640 1 --2- 172.21.15.83:0/4067691036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55680d04a0 0x7f55681f6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.655+0000 7f556d388640 1 -- 172.21.15.83:0/4067691036 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55680d0040 msgr2=0x7f556806d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.655+0000 7f556d388640 1 --2- 172.21.15.83:0/4067691036 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55680d0040 0x7f556806d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.655+0000 7f556d388640 1 -- 172.21.15.83:0/4067691036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55680d04a0 msgr2=0x7f55681f6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.655+0000 7f556d388640 1 --2- 172.21.15.83:0/4067691036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55680d04a0 0x7f55681f6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.655+0000 7f556d388640 1 -- 172.21.15.83:0/4067691036 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55680c5a50 con 0x7f556815bb40 2024-04-17T10:41:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.655+0000 7f5567fff640 1 --2- 172.21.15.83:0/4067691036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55680d04a0 0x7f55681f6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.659+0000 7f556d388640 1 --2- 172.21.15.83:0/4067691036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f556815bb40 0x7f556815a450 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f556001af90 tx=0x7f5560019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.659+0000 7f55657fa640 1 -- 172.21.15.83:0/4067691036 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55600bd020 con 0x7f556815bb40 2024-04-17T10:41:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.659+0000 7f55657fa640 1 -- 172.21.15.83:0/4067691036 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55600b8670 con 0x7f556815bb40 2024-04-17T10:41:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.659+0000 7f556edeb640 1 -- 172.21.15.83:0/4067691036 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55680c5d30 con 0x7f556815bb40 2024-04-17T10:41:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.659+0000 7f55657fa640 1 -- 172.21.15.83:0/4067691036 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55600b2040 con 0x7f556815bb40 2024-04-17T10:41:22.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.659+0000 7f556edeb640 1 -- 172.21.15.83:0/4067691036 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55680c62f0 con 0x7f556815bb40 2024-04-17T10:41:22.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.659+0000 7f556edeb640 1 -- 172.21.15.83:0/4067691036 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5568074f90 con 0x7f556815bb40 2024-04-17T10:41:22.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.659+0000 7f55657fa640 1 -- 172.21.15.83:0/4067691036 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f55600c0020 con 0x7f556815bb40 2024-04-17T10:41:22.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.659+0000 7f55657fa640 1 --2- 172.21.15.83:0/4067691036 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5554066f60 0x7f5554069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:22.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.659+0000 7f55657fa640 1 -- 172.21.15.83:0/4067691036 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f556011ffd0 con 0x7f556815bb40 2024-04-17T10:41:22.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.659+0000 7f556cb87640 1 --2- 172.21.15.83:0/4067691036 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5554066f60 0x7f5554069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:22.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.659+0000 7f556cb87640 1 --2- 172.21.15.83:0/4067691036 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5554066f60 0x7f5554069420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5558000f70 tx=0x7f5558009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:22.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.663+0000 7f55657fa640 1 -- 172.21.15.83:0/4067691036 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f55600c7030 con 0x7f556815bb40 2024-04-17T10:41:22.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.839+0000 7f556edeb640 1 -- 172.21.15.83:0/4067691036 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55680cbe50 con 0x7f556815bb40 2024-04-17T10:41:22.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.843+0000 7f55657fa640 1 -- 172.21.15.83:0/4067691036 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f5560016560 con 0x7f556815bb40 2024-04-17T10:41:22.839 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:41:22.840 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1456029744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6833","nonce":1456029744},{"type":"v1","addr":"172.21.15.167:6835","nonce":1456029744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6834/648555908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":648555908},{"type":"v1","addr":"172.21.15.167:6834","nonce":648555908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5163,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1435807773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1435807773},{"type":"v1","addr":"172.21.15.83:6835","nonce":1435807773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-17T10:39:13.472135+0000","modified":"2024-04-17T10:39:21.807600+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5172},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5172":{"gid":5172,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1181902025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1181902025},{"type":"v1","addr":"172.21.15.83:6837","nonce":1181902025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-17T10:41:22.840 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 14 2024-04-17T10:41:22.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.843+0000 7f556edeb640 1 -- 172.21.15.83:0/4067691036 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5554066f60 msgr2=0x7f5554069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:22.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.843+0000 7f556edeb640 1 --2- 172.21.15.83:0/4067691036 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5554066f60 0x7f5554069420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5558000f70 tx=0x7f5558009040 comp rx=0 tx=0).stop 2024-04-17T10:41:22.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.843+0000 7f556edeb640 1 -- 172.21.15.83:0/4067691036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f556815bb40 msgr2=0x7f556815a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:22.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.843+0000 7f556edeb640 1 --2- 172.21.15.83:0/4067691036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f556815bb40 0x7f556815a450 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f556001af90 tx=0x7f5560019680 comp rx=0 tx=0).stop 2024-04-17T10:41:22.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.843+0000 7f556edeb640 1 -- 172.21.15.83:0/4067691036 shutdown_connections 2024-04-17T10:41:22.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.843+0000 7f556edeb640 1 --2- 172.21.15.83:0/4067691036 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5554066f60 0x7f5554069420 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:22.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.843+0000 7f556edeb640 1 --2- 172.21.15.83:0/4067691036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f556815bb40 0x7f556815a450 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:22.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.843+0000 7f556edeb640 1 --2- 172.21.15.83:0/4067691036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55680d04a0 0x7f55681f6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:22.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.843+0000 7f556edeb640 1 --2- 172.21.15.83:0/4067691036 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55680d0040 0x7f556806d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:22.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.843+0000 7f556edeb640 1 -- 172.21.15.83:0/4067691036 >> 172.21.15.83:0/4067691036 conn(0x7f556809a9d0 msgr2=0x7f55680affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:22.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.843+0000 7f556edeb640 1 -- 172.21.15.83:0/4067691036 shutdown_connections 2024-04-17T10:41:22.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.843+0000 7f556edeb640 1 -- 172.21.15.83:0/4067691036 wait complete. 2024-04-17T10:41:22.857 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:41:22.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.971+0000 7f97baa4a640 1 Processor -- start 2024-04-17T10:41:22.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.971+0000 7f97baa4a640 1 -- start start 2024-04-17T10:41:22.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.971+0000 7f97baa4a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97b4072330 0x7f97b40cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:22.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.971+0000 7f97baa4a640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f97b40cfa10 0x7f97b406c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:22.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.971+0000 7f97baa4a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b40cfea0 con 0x7f97b415ba30 2024-04-17T10:41:22.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.971+0000 7f97baa4a640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b4071ea0 con 0x7f97b40cfa10 2024-04-17T10:41:22.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.971+0000 7f97baa4a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b406e320 con 0x7f97b4072330 2024-04-17T10:41:22.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.975+0000 7f97b3fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f97b415ba30 0x7f97b415c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54024/0 (socket says 172.21.15.83:54024) 2024-04-17T10:41:22.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.975+0000 7f97b3fff640 1 -- 172.21.15.83:0/2067140295 learned_addr learned my addr 172.21.15.83:0/2067140295 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:22.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.975+0000 7f97b8fe7640 1 --2- 172.21.15.83:0/2067140295 >> v2:172.21.15.167:3300/0 conn(0x7f97b40cfa10 0x7f97b406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:22.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.975+0000 7f97b37fe640 1 --2- 172.21.15.83:0/2067140295 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97b4072330 0x7f97b40cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:22.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.975+0000 7f97b2ffd640 1 -- 172.21.15.83:0/2067140295 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3169969600 0 0) 0x7f97b40cfea0 con 0x7f97b415ba30 2024-04-17T10:41:22.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.975+0000 7f97b2ffd640 1 -- 172.21.15.83:0/2067140295 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f979c003530 con 0x7f97b415ba30 2024-04-17T10:41:22.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.975+0000 7f97b2ffd640 1 -- 172.21.15.83:0/2067140295 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1162679435 0 0) 0x7f979c003530 con 0x7f97b415ba30 2024-04-17T10:41:22.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.975+0000 7f97b2ffd640 1 -- 172.21.15.83:0/2067140295 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97b4072330 msgr2=0x7f97b40cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:22.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.975+0000 7f97b2ffd640 1 --2- 172.21.15.83:0/2067140295 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97b4072330 0x7f97b40cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:22.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.975+0000 7f97b2ffd640 1 -- 172.21.15.83:0/2067140295 >> v2:172.21.15.167:3300/0 conn(0x7f97b40cfa10 msgr2=0x7f97b406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:22.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.975+0000 7f97b2ffd640 1 --2- 172.21.15.83:0/2067140295 >> v2:172.21.15.167:3300/0 conn(0x7f97b40cfa10 0x7f97b406c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:22.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.975+0000 7f97b2ffd640 1 -- 172.21.15.83:0/2067140295 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b406e040 con 0x7f97b415ba30 2024-04-17T10:41:22.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.975+0000 7f97b2ffd640 1 -- 172.21.15.83:0/2067140295 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f97a4001680 con 0x7f97b415ba30 2024-04-17T10:41:22.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.975+0000 7f97b2ffd640 1 -- 172.21.15.83:0/2067140295 >> v1:172.21.15.83:6789/0 conn(0x7f97b415ba30 legacy=0x7f97b415c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:22.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.975+0000 7f97b2ffd640 1 --2- 172.21.15.83:0/2067140295 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97b4072330 0x7f979c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:22.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.975+0000 7f97b2ffd640 1 --2- 172.21.15.83:0/2067140295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f979c003f40 0x7f979c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:22.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.975+0000 7f97b2ffd640 1 --2- 172.21.15.83:0/2067140295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f979c004d50 0x7f979c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:22.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.975+0000 7f97b2ffd640 1 -- 172.21.15.83:0/2067140295 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b4071ea0 con 0x7f979c003f40 2024-04-17T10:41:22.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.975+0000 7f97b2ffd640 1 -- 172.21.15.83:0/2067140295 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b406e320 con 0x7f979c004d50 2024-04-17T10:41:22.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.975+0000 7f97b2ffd640 1 -- 172.21.15.83:0/2067140295 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f979c00ba20 con 0x7f97b4072330 2024-04-17T10:41:22.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.975+0000 7f97b37fe640 1 --2- 172.21.15.83:0/2067140295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f979c003f40 0x7f979c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:22.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.975+0000 7f97b8fe7640 1 --2- 172.21.15.83:0/2067140295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f979c004d50 0x7f979c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:22.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.975+0000 7f97b37fe640 1 -- 172.21.15.83:0/2067140295 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97b4072330 msgr2=0x7f979c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:22.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.975+0000 7f97b37fe640 1 --2- 172.21.15.83:0/2067140295 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97b4072330 0x7f979c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:22.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.975+0000 7f97b37fe640 1 -- 172.21.15.83:0/2067140295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f979c004d50 msgr2=0x7f979c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:22.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.975+0000 7f97b37fe640 1 --2- 172.21.15.83:0/2067140295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f979c004d50 0x7f979c0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:22.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.975+0000 7f97b37fe640 1 -- 172.21.15.83:0/2067140295 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979c003530 con 0x7f979c003f40 2024-04-17T10:41:22.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.975+0000 7f97b37fe640 1 --2- 172.21.15.83:0/2067140295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f979c003f40 0x7f979c004810 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f97a8009af0 tx=0x7f97a801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da419ce2aa59841 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:22.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.975+0000 7f97b2ffd640 1 -- 172.21.15.83:0/2067140295 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a80b2070 con 0x7f979c003f40 2024-04-17T10:41:22.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.975+0000 7f97b2ffd640 1 -- 172.21.15.83:0/2067140295 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a80a5550 con 0x7f979c003f40 2024-04-17T10:41:22.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.975+0000 7f97b2ffd640 1 -- 172.21.15.83:0/2067140295 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a8009690 con 0x7f979c003f40 2024-04-17T10:41:22.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97baa4a640 1 -- 172.21.15.83:0/2067140295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f979c003f40 msgr2=0x7f979c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:22.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97baa4a640 1 --2- 172.21.15.83:0/2067140295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f979c003f40 0x7f979c004810 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f97a8009af0 tx=0x7f97a801ca70 comp rx=0 tx=0).stop 2024-04-17T10:41:22.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97b8fe7640 1 -- 172.21.15.83:0/2067140295 reap_dead start 2024-04-17T10:41:22.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97baa4a640 1 -- 172.21.15.83:0/2067140295 shutdown_connections 2024-04-17T10:41:22.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97baa4a640 1 -- 172.21.15.83:0/2067140295 >> 172.21.15.83:0/2067140295 conn(0x7f97b406abe0 msgr2=0x7f97b406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:22.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97baa4a640 1 -- 172.21.15.83:0/2067140295 shutdown_connections 2024-04-17T10:41:22.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97baa4a640 1 -- 172.21.15.83:0/2067140295 wait complete. 2024-04-17T10:41:22.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97baa4a640 1 Processor -- start 2024-04-17T10:41:22.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97baa4a640 1 -- start start 2024-04-17T10:41:22.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97baa4a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f979c003f40 0x7f97b414e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:22.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97baa4a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97b4152250 0x7f97b4161410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:22.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97baa4a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97b415a420 0x7f97b415a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:22.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97baa4a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b406de20 con 0x7f979c003f40 2024-04-17T10:41:22.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97baa4a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b406dca0 con 0x7f97b415a420 2024-04-17T10:41:22.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97baa4a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b4166510 con 0x7f97b4152250 2024-04-17T10:41:22.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97b3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f979c003f40 0x7f97b414e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:22.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97b3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f979c003f40 0x7f97b414e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36550/0 (socket says 172.21.15.83:36550) 2024-04-17T10:41:22.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97b3fff640 1 -- 172.21.15.83:0/149803771 learned_addr learned my addr 172.21.15.83:0/149803771 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:22.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97b8fe7640 1 --2- 172.21.15.83:0/149803771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97b415a420 0x7f97b415a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:22.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97b37fe640 1 --2- 172.21.15.83:0/149803771 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97b4152250 0x7f97b4161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:22.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97b3fff640 1 -- 172.21.15.83:0/149803771 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97b4152250 msgr2=0x7f97b4161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:22.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97b3fff640 1 --2- 172.21.15.83:0/149803771 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97b4152250 0x7f97b4161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:22.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97b3fff640 1 -- 172.21.15.83:0/149803771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97b415a420 msgr2=0x7f97b415a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:22.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97b3fff640 1 --2- 172.21.15.83:0/149803771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97b415a420 0x7f97b415a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:22.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97b3fff640 1 -- 172.21.15.83:0/149803771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b41527c0 con 0x7f979c003f40 2024-04-17T10:41:22.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97b37fe640 1 --2- 172.21.15.83:0/149803771 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97b4152250 0x7f97b4161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:22.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97b3fff640 1 --2- 172.21.15.83:0/149803771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f979c003f40 0x7f97b414e070 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f97a4007fd0 tx=0x7f97a400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:22.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97b0ff9640 1 -- 172.21.15.83:0/149803771 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a4018070 con 0x7f979c003f40 2024-04-17T10:41:22.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97b0ff9640 1 -- 172.21.15.83:0/149803771 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a40072d0 con 0x7f979c003f40 2024-04-17T10:41:22.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97baa4a640 1 -- 172.21.15.83:0/149803771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b4152aa0 con 0x7f979c003f40 2024-04-17T10:41:22.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97b0ff9640 1 -- 172.21.15.83:0/149803771 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a40047f0 con 0x7f979c003f40 2024-04-17T10:41:22.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97baa4a640 1 -- 172.21.15.83:0/149803771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b4153060 con 0x7f979c003f40 2024-04-17T10:41:22.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.979+0000 7f97baa4a640 1 -- 172.21.15.83:0/149803771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b4074f90 con 0x7f979c003f40 2024-04-17T10:41:22.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.983+0000 7f97b0ff9640 1 -- 172.21.15.83:0/149803771 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f97a401b070 con 0x7f979c003f40 2024-04-17T10:41:22.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.983+0000 7f97b0ff9640 1 --2- 172.21.15.83:0/149803771 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f97a0066f90 0x7f97a0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:22.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.983+0000 7f97b0ff9640 1 -- 172.21.15.83:0/149803771 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f97a40846e0 con 0x7f979c003f40 2024-04-17T10:41:22.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.983+0000 7f97b37fe640 1 --2- 172.21.15.83:0/149803771 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f97a0066f90 0x7f97a0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:22.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.983+0000 7f97b37fe640 1 --2- 172.21.15.83:0/149803771 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f97a0066f90 0x7f97a0069450 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f97a80af000 tx=0x7f97a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:22.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:22.987+0000 7f97b0ff9640 1 -- 172.21.15.83:0/149803771 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f97a40556d0 con 0x7f979c003f40 2024-04-17T10:41:23.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.139+0000 7f97baa4a640 1 -- 172.21.15.83:0/149803771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97b40cf190 con 0x7f979c003f40 2024-04-17T10:41:23.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.139+0000 7f97b0ff9640 1 -- 172.21.15.83:0/149803771 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16662 (secure 0 0 0) 0x7f97a4054fd0 con 0x7f979c003f40 2024-04-17T10:41:23.138 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:41:23.139 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":35,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:40:57.038108+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:39:11.330695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-17T10:39:12.438377+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":34,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:41:23.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.143+0000 7f97baa4a640 1 -- 172.21.15.83:0/149803771 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f97a0066f90 msgr2=0x7f97a0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:23.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.143+0000 7f97baa4a640 1 --2- 172.21.15.83:0/149803771 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f97a0066f90 0x7f97a0069450 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f97a80af000 tx=0x7f97a8008040 comp rx=0 tx=0).stop 2024-04-17T10:41:23.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.143+0000 7f97baa4a640 1 -- 172.21.15.83:0/149803771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f979c003f40 msgr2=0x7f97b414e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:23.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.143+0000 7f97baa4a640 1 --2- 172.21.15.83:0/149803771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f979c003f40 0x7f97b414e070 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f97a4007fd0 tx=0x7f97a400cb90 comp rx=0 tx=0).stop 2024-04-17T10:41:23.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.143+0000 7f97baa4a640 1 -- 172.21.15.83:0/149803771 shutdown_connections 2024-04-17T10:41:23.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.143+0000 7f97baa4a640 1 --2- 172.21.15.83:0/149803771 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f97a0066f90 0x7f97a0069450 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:23.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.143+0000 7f97baa4a640 1 --2- 172.21.15.83:0/149803771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97b415a420 0x7f97b415a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:23.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.143+0000 7f97baa4a640 1 --2- 172.21.15.83:0/149803771 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97b4152250 0x7f97b4161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:23.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.143+0000 7f97baa4a640 1 --2- 172.21.15.83:0/149803771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f979c003f40 0x7f97b414e070 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:23.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.143+0000 7f97baa4a640 1 -- 172.21.15.83:0/149803771 >> 172.21.15.83:0/149803771 conn(0x7f97b406abe0 msgr2=0x7f97b40b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:23.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.143+0000 7f97baa4a640 1 -- 172.21.15.83:0/149803771 shutdown_connections 2024-04-17T10:41:23.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.143+0000 7f97baa4a640 1 -- 172.21.15.83:0/149803771 wait complete. 2024-04-17T10:41:23.156 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:39:11.330695+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:39:12.438377+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 34, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:41:23.157 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:41:23.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.279+0000 7f68f8424640 1 Processor -- start 2024-04-17T10:41:23.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.279+0000 7f68f8424640 1 -- start start 2024-04-17T10:41:23.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.279+0000 7f68f8424640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f68f015b6e0 0x7f68f0071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:23.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.279+0000 7f68f8424640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f68f0070470 0x7f68f006ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:23.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.279+0000 7f68f8424640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68f006c1f0 con 0x7f68f015bac0 2024-04-17T10:41:23.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.279+0000 7f68f8424640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68f0070a10 con 0x7f68f0070470 2024-04-17T10:41:23.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.279+0000 7f68f8424640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68f0070cf0 con 0x7f68f015b6e0 2024-04-17T10:41:23.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.279+0000 7f68f69c1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f68f0070470 0x7f68f006ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:23.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.279+0000 7f68f59bf640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f68f015bac0 0x7f68f00cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54038/0 (socket says 172.21.15.83:54038) 2024-04-17T10:41:23.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.279+0000 7f68f59bf640 1 -- 172.21.15.83:0/2711349642 learned_addr learned my addr 172.21.15.83:0/2711349642 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:23.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.279+0000 7f68f61c0640 1 --2- 172.21.15.83:0/2711349642 >> v2:172.21.15.167:3300/0 conn(0x7f68f015b6e0 0x7f68f0071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:23.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.279+0000 7f68f69c1640 1 -- 172.21.15.83:0/2711349642 >> v2:172.21.15.167:3300/0 conn(0x7f68f015b6e0 msgr2=0x7f68f0071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:23.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.279+0000 7f68f69c1640 1 --2- 172.21.15.83:0/2711349642 >> v2:172.21.15.167:3300/0 conn(0x7f68f015b6e0 0x7f68f0071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:23.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.279+0000 7f68f69c1640 1 -- 172.21.15.83:0/2711349642 >> v1:172.21.15.83:6789/0 conn(0x7f68f015bac0 legacy=0x7f68f00cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:23.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.279+0000 7f68f51be640 1 -- 172.21.15.83:0/2711349642 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1453792337 0 0) 0x7f68f006c1f0 con 0x7f68f015bac0 2024-04-17T10:41:23.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.279+0000 7f68f69c1640 1 -- 172.21.15.83:0/2711349642 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f0070fd0 con 0x7f68f0070470 2024-04-17T10:41:23.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.283+0000 7f68f69c1640 1 --2- 172.21.15.83:0/2711349642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68f0070470 0x7f68f006ae10 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f68f00728a0 tx=0x7f68e801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b109bb0b279a61d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:23.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.283+0000 7f68f51be640 1 -- 172.21.15.83:0/2711349642 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e80b2070 con 0x7f68f0070470 2024-04-17T10:41:23.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.283+0000 7f68f51be640 1 -- 172.21.15.83:0/2711349642 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e80a57f0 con 0x7f68f0070470 2024-04-17T10:41:23.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.283+0000 7f68f51be640 1 -- 172.21.15.83:0/2711349642 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e80a5bc0 con 0x7f68f0070470 2024-04-17T10:41:23.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.283+0000 7f68f8424640 1 -- 172.21.15.83:0/2711349642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68f0070470 msgr2=0x7f68f006ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:23.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.283+0000 7f68f8424640 1 --2- 172.21.15.83:0/2711349642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68f0070470 0x7f68f006ae10 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f68f00728a0 tx=0x7f68e801c930 comp rx=0 tx=0).stop 2024-04-17T10:41:23.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.283+0000 7f68f8424640 1 -- 172.21.15.83:0/2711349642 shutdown_connections 2024-04-17T10:41:23.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.283+0000 7f68f8424640 1 --2- 172.21.15.83:0/2711349642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68f0070470 0x7f68f006ae10 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:23.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.283+0000 7f68f8424640 1 --2- 172.21.15.83:0/2711349642 >> v2:172.21.15.167:3300/0 conn(0x7f68f015b6e0 0x7f68f0071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:23.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.283+0000 7f68f8424640 1 -- 172.21.15.83:0/2711349642 >> 172.21.15.83:0/2711349642 conn(0x7f68f00b15b0 msgr2=0x7f68f00b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:23.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.283+0000 7f68f8424640 1 -- 172.21.15.83:0/2711349642 shutdown_connections 2024-04-17T10:41:23.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.283+0000 7f68f8424640 1 -- 172.21.15.83:0/2711349642 wait complete. 2024-04-17T10:41:23.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.283+0000 7f68f8424640 1 Processor -- start 2024-04-17T10:41:23.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.283+0000 7f68f8424640 1 -- start start 2024-04-17T10:41:23.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.283+0000 7f68f8424640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68f0070470 0x7f68f014fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:23.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.283+0000 7f68f8424640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68f015b6e0 0x7f68f00b0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:23.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.287+0000 7f68f8424640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68f015bac0 0x7f68f00b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:23.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.287+0000 7f68f8424640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f006d2f0 con 0x7f68f015bac0 2024-04-17T10:41:23.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.287+0000 7f68f8424640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f006d170 con 0x7f68f0070470 2024-04-17T10:41:23.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.287+0000 7f68f8424640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f006b760 con 0x7f68f015b6e0 2024-04-17T10:41:23.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.287+0000 7f68f69c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68f015bac0 0x7f68f00b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:23.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.287+0000 7f68f69c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68f015bac0 0x7f68f00b0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36574/0 (socket says 172.21.15.83:36574) 2024-04-17T10:41:23.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.287+0000 7f68f69c1640 1 -- 172.21.15.83:0/1202552255 learned_addr learned my addr 172.21.15.83:0/1202552255 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:23.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.287+0000 7f68f59bf640 1 --2- 172.21.15.83:0/1202552255 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68f015b6e0 0x7f68f00b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:23.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.287+0000 7f68f61c0640 1 --2- 172.21.15.83:0/1202552255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68f0070470 0x7f68f014fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:23.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.287+0000 7f68f69c1640 1 -- 172.21.15.83:0/1202552255 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68f015b6e0 msgr2=0x7f68f00b0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:23.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.287+0000 7f68f69c1640 1 --2- 172.21.15.83:0/1202552255 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68f015b6e0 0x7f68f00b0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:23.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.287+0000 7f68f69c1640 1 -- 172.21.15.83:0/1202552255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68f0070470 msgr2=0x7f68f014fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:23.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.287+0000 7f68f69c1640 1 --2- 172.21.15.83:0/1202552255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68f0070470 0x7f68f014fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:23.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.287+0000 7f68f69c1640 1 -- 172.21.15.83:0/1202552255 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f00b3460 con 0x7f68f015bac0 2024-04-17T10:41:23.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.287+0000 7f68f59bf640 1 --2- 172.21.15.83:0/1202552255 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68f015b6e0 0x7f68f00b0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:23.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.287+0000 7f68f61c0640 1 --2- 172.21.15.83:0/1202552255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68f0070470 0x7f68f014fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:23.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.287+0000 7f68f69c1640 1 --2- 172.21.15.83:0/1202552255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68f015bac0 0x7f68f00b0a50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f68e8019540 tx=0x7f68e80196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:23.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.287+0000 7f68e6ffd640 1 -- 172.21.15.83:0/1202552255 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e80b2050 con 0x7f68f015bac0 2024-04-17T10:41:23.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.287+0000 7f68e6ffd640 1 -- 172.21.15.83:0/1202552255 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e80b3040 con 0x7f68f015bac0 2024-04-17T10:41:23.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.287+0000 7f68f8424640 1 -- 172.21.15.83:0/1202552255 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f00b3740 con 0x7f68f015bac0 2024-04-17T10:41:23.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.287+0000 7f68e6ffd640 1 -- 172.21.15.83:0/1202552255 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e8009450 con 0x7f68f015bac0 2024-04-17T10:41:23.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.287+0000 7f68f8424640 1 -- 172.21.15.83:0/1202552255 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f00b3d00 con 0x7f68f015bac0 2024-04-17T10:41:23.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.287+0000 7f68f8424640 1 -- 172.21.15.83:0/1202552255 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68f0074f90 con 0x7f68f015bac0 2024-04-17T10:41:23.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.287+0000 7f68e6ffd640 1 -- 172.21.15.83:0/1202552255 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f68e80c0020 con 0x7f68f015bac0 2024-04-17T10:41:23.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.287+0000 7f68e6ffd640 1 --2- 172.21.15.83:0/1202552255 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f68d4066f60 0x7f68d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:23.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.287+0000 7f68f61c0640 1 --2- 172.21.15.83:0/1202552255 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f68d4066f60 0x7f68d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:23.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.287+0000 7f68e6ffd640 1 -- 172.21.15.83:0/1202552255 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f68e811fbc0 con 0x7f68f015bac0 2024-04-17T10:41:23.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.287+0000 7f68f61c0640 1 --2- 172.21.15.83:0/1202552255 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f68d4066f60 0x7f68d4069420 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009840 tx=0x7f68ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:23.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.291+0000 7f68e6ffd640 1 -- 172.21.15.83:0/1202552255 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f68e8016520 con 0x7f68f015bac0 2024-04-17T10:41:23.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.439+0000 7f68f8424640 1 -- 172.21.15.83:0/1202552255 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f68f00b1b30 con 0x7f68f015bac0 2024-04-17T10:41:23.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.439+0000 7f68e6ffd640 1 -- 172.21.15.83:0/1202552255 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7f68e80f0d60 con 0x7f68f015bac0 2024-04-17T10:41:23.439 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:41:23.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.443+0000 7f68f8424640 1 -- 172.21.15.83:0/1202552255 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f68d4066f60 msgr2=0x7f68d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:23.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.443+0000 7f68f8424640 1 --2- 172.21.15.83:0/1202552255 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f68d4066f60 0x7f68d4069420 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009840 tx=0x7f68ec008040 comp rx=0 tx=0).stop 2024-04-17T10:41:23.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.443+0000 7f68f8424640 1 -- 172.21.15.83:0/1202552255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68f015bac0 msgr2=0x7f68f00b0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:23.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.443+0000 7f68f8424640 1 --2- 172.21.15.83:0/1202552255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68f015bac0 0x7f68f00b0a50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f68e8019540 tx=0x7f68e80196d0 comp rx=0 tx=0).stop 2024-04-17T10:41:23.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.443+0000 7f68f8424640 1 -- 172.21.15.83:0/1202552255 shutdown_connections 2024-04-17T10:41:23.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.443+0000 7f68f8424640 1 --2- 172.21.15.83:0/1202552255 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f68d4066f60 0x7f68d4069420 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:23.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.443+0000 7f68f8424640 1 --2- 172.21.15.83:0/1202552255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68f015bac0 0x7f68f00b0a50 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:23.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.443+0000 7f68f8424640 1 --2- 172.21.15.83:0/1202552255 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68f015b6e0 0x7f68f00b0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:23.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.443+0000 7f68f8424640 1 --2- 172.21.15.83:0/1202552255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68f0070470 0x7f68f014fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:23.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.443+0000 7f68f8424640 1 -- 172.21.15.83:0/1202552255 >> 172.21.15.83:0/1202552255 conn(0x7f68f00b15b0 msgr2=0x7f68f00cd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:23.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.443+0000 7f68f8424640 1 -- 172.21.15.83:0/1202552255 shutdown_connections 2024-04-17T10:41:23.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.443+0000 7f68f8424640 1 -- 172.21.15.83:0/1202552255 wait complete. 2024-04-17T10:41:23.457 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:41:23.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.579+0000 7fbe12655640 1 Processor -- start 2024-04-17T10:41:23.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.579+0000 7fbe12655640 1 -- start start 2024-04-17T10:41:23.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.583+0000 7fbe12655640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbe0c15b6e0 0x7fbe0c071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:23.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.583+0000 7fbe12655640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe0c15bac0 0x7fbe0c0cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:23.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.583+0000 7fbe12655640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe0c06c1f0 con 0x7fbe0c070470 2024-04-17T10:41:23.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.583+0000 7fbe12655640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe0c0cdd30 con 0x7fbe0c15b6e0 2024-04-17T10:41:23.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.583+0000 7fbe12655640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0c070a10 con 0x7fbe0c15bac0 2024-04-17T10:41:23.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.583+0000 7fbe0b7fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe0c15bac0 0x7fbe0c0cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:23.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.583+0000 7fbe0b7fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe0c15bac0 0x7fbe0c0cd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:49868/0 (socket says 172.21.15.83:49868) 2024-04-17T10:41:23.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.583+0000 7fbe0b7fe640 1 -- 172.21.15.83:0/4247076597 learned_addr learned my addr 172.21.15.83:0/4247076597 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:23.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.583+0000 7fbe0bfff640 1 --2- 172.21.15.83:0/4247076597 >> v2:172.21.15.167:3300/0 conn(0x7fbe0c15b6e0 0x7fbe0c071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:23.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.583+0000 7fbe0b7fe640 1 -- 172.21.15.83:0/4247076597 >> v2:172.21.15.167:3300/0 conn(0x7fbe0c15b6e0 msgr2=0x7fbe0c071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:23.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.583+0000 7fbe0b7fe640 1 --2- 172.21.15.83:0/4247076597 >> v2:172.21.15.167:3300/0 conn(0x7fbe0c15b6e0 0x7fbe0c071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:23.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.583+0000 7fbe0b7fe640 1 -- 172.21.15.83:0/4247076597 >> v1:172.21.15.167:6789/0 conn(0x7fbe0c070470 legacy=0x7fbe0c06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:23.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.583+0000 7fbe0b7fe640 1 -- 172.21.15.83:0/4247076597 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe0c070cf0 con 0x7fbe0c15bac0 2024-04-17T10:41:23.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.583+0000 7fbe0b7fe640 1 --2- 172.21.15.83:0/4247076597 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe0c15bac0 0x7fbe0c0cd7f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbe00009870 tx=0x7fbe0001c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=256f8ca867394a42 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:23.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.587+0000 7fbe0affd640 1 -- 172.21.15.83:0/4247076597 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe000b2070 con 0x7fbe0c15bac0 2024-04-17T10:41:23.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.587+0000 7fbe0affd640 1 -- 172.21.15.83:0/4247076597 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe000a9070 con 0x7fbe0c15bac0 2024-04-17T10:41:23.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.587+0000 7fbe0affd640 1 -- 172.21.15.83:0/4247076597 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe000ad880 con 0x7fbe0c15bac0 2024-04-17T10:41:23.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.587+0000 7fbe12655640 1 -- 172.21.15.83:0/4247076597 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe0c15bac0 msgr2=0x7fbe0c0cd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:23.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.587+0000 7fbe12655640 1 --2- 172.21.15.83:0/4247076597 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe0c15bac0 0x7fbe0c0cd7f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbe00009870 tx=0x7fbe0001c560 comp rx=0 tx=0).stop 2024-04-17T10:41:23.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.587+0000 7fbe12655640 1 -- 172.21.15.83:0/4247076597 shutdown_connections 2024-04-17T10:41:23.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.587+0000 7fbe12655640 1 --2- 172.21.15.83:0/4247076597 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe0c15bac0 0x7fbe0c0cd7f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:23.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.587+0000 7fbe12655640 1 --2- 172.21.15.83:0/4247076597 >> v2:172.21.15.167:3300/0 conn(0x7fbe0c15b6e0 0x7fbe0c071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:23.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.587+0000 7fbe12655640 1 -- 172.21.15.83:0/4247076597 >> 172.21.15.83:0/4247076597 conn(0x7fbe0c0b15b0 msgr2=0x7fbe0c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:23.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.587+0000 7fbe12655640 1 -- 172.21.15.83:0/4247076597 shutdown_connections 2024-04-17T10:41:23.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.587+0000 7fbe12655640 1 -- 172.21.15.83:0/4247076597 wait complete. 2024-04-17T10:41:23.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.587+0000 7fbe12655640 1 Processor -- start 2024-04-17T10:41:23.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.587+0000 7fbe12655640 1 -- start start 2024-04-17T10:41:23.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.587+0000 7fbe12655640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe0c070470 0x7fbe0c1479c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:23.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.587+0000 7fbe12655640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe0c15b6e0 0x7fbe0c158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:23.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.587+0000 7fbe12655640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe0c15bac0 0x7fbe0c159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:23.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.587+0000 7fbe12655640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0c06cee0 con 0x7fbe0c15b6e0 2024-04-17T10:41:23.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.587+0000 7fbe12655640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0c06b730 con 0x7fbe0c070470 2024-04-17T10:41:23.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.587+0000 7fbe12655640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0c06d060 con 0x7fbe0c15bac0 2024-04-17T10:41:23.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.587+0000 7fbe0b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe0c15b6e0 0x7fbe0c158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:23.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.587+0000 7fbe0b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe0c15b6e0 0x7fbe0c158ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36590/0 (socket says 172.21.15.83:36590) 2024-04-17T10:41:23.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.587+0000 7fbe0b7fe640 1 -- 172.21.15.83:0/3093984928 learned_addr learned my addr 172.21.15.83:0/3093984928 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:23.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.587+0000 7fbe0bfff640 1 --2- 172.21.15.83:0/3093984928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe0c070470 0x7fbe0c1479c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:23.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.587+0000 7fbe10bf2640 1 --2- 172.21.15.83:0/3093984928 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe0c15bac0 0x7fbe0c159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:23.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.587+0000 7fbe0b7fe640 1 -- 172.21.15.83:0/3093984928 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe0c15bac0 msgr2=0x7fbe0c159220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:23.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.587+0000 7fbe0b7fe640 1 --2- 172.21.15.83:0/3093984928 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe0c15bac0 0x7fbe0c159220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:23.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.587+0000 7fbe0b7fe640 1 -- 172.21.15.83:0/3093984928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe0c070470 msgr2=0x7fbe0c1479c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:23.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.587+0000 7fbe0b7fe640 1 --2- 172.21.15.83:0/3093984928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe0c070470 0x7fbe0c1479c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:23.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.587+0000 7fbe0b7fe640 1 -- 172.21.15.83:0/3093984928 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe0c14a8b0 con 0x7fbe0c15b6e0 2024-04-17T10:41:23.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.587+0000 7fbe0b7fe640 1 --2- 172.21.15.83:0/3093984928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe0c15b6e0 0x7fbe0c158ce0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbe000b0040 tx=0x7fbe00019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:23.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.587+0000 7fbe08ff9640 1 -- 172.21.15.83:0/3093984928 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe000bb070 con 0x7fbe0c15b6e0 2024-04-17T10:41:23.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.591+0000 7fbe08ff9640 1 -- 172.21.15.83:0/3093984928 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe000add80 con 0x7fbe0c15b6e0 2024-04-17T10:41:23.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.591+0000 7fbe08ff9640 1 -- 172.21.15.83:0/3093984928 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe000b2040 con 0x7fbe0c15b6e0 2024-04-17T10:41:23.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.591+0000 7fbe12655640 1 -- 172.21.15.83:0/3093984928 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe0c147f00 con 0x7fbe0c15b6e0 2024-04-17T10:41:23.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.591+0000 7fbe12655640 1 -- 172.21.15.83:0/3093984928 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe0c14b1f0 con 0x7fbe0c15b6e0 2024-04-17T10:41:23.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.591+0000 7fbe12655640 1 -- 172.21.15.83:0/3093984928 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe0c074f90 con 0x7fbe0c15b6e0 2024-04-17T10:41:23.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.591+0000 7fbe08ff9640 1 -- 172.21.15.83:0/3093984928 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbe000ab070 con 0x7fbe0c15b6e0 2024-04-17T10:41:23.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.591+0000 7fbe08ff9640 1 --2- 172.21.15.83:0/3093984928 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbdf8066f60 0x7fbdf8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:23.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.591+0000 7fbe08ff9640 1 -- 172.21.15.83:0/3093984928 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fbe0011de20 con 0x7fbe0c15b6e0 2024-04-17T10:41:23.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.591+0000 7fbe0bfff640 1 --2- 172.21.15.83:0/3093984928 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbdf8066f60 0x7fbdf8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:23.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.591+0000 7fbe0bfff640 1 --2- 172.21.15.83:0/3093984928 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbdf8066f60 0x7fbdf8069420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc009af0 tx=0x7fbdfc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:23.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.595+0000 7fbe08ff9640 1 -- 172.21.15.83:0/3093984928 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbe00016150 con 0x7fbe0c15b6e0 2024-04-17T10:41:23.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.755+0000 7fbe12655640 1 -- 172.21.15.83:0/3093984928 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbe0c15a610 con 0x7fbe0c15b6e0 2024-04-17T10:41:23.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.755+0000 7fbe08ff9640 1 -- 172.21.15.83:0/3093984928 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbe000ee740 con 0x7fbe0c15b6e0 2024-04-17T10:41:23.752 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:41:23.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.755+0000 7fbe12655640 1 -- 172.21.15.83:0/3093984928 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbdf8066f60 msgr2=0x7fbdf8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:23.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.755+0000 7fbe12655640 1 --2- 172.21.15.83:0/3093984928 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbdf8066f60 0x7fbdf8069420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc009af0 tx=0x7fbdfc008040 comp rx=0 tx=0).stop 2024-04-17T10:41:23.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.755+0000 7fbe12655640 1 -- 172.21.15.83:0/3093984928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe0c15b6e0 msgr2=0x7fbe0c158ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:23.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.755+0000 7fbe12655640 1 --2- 172.21.15.83:0/3093984928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe0c15b6e0 0x7fbe0c158ce0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbe000b0040 tx=0x7fbe00019300 comp rx=0 tx=0).stop 2024-04-17T10:41:23.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.755+0000 7fbe12655640 1 -- 172.21.15.83:0/3093984928 shutdown_connections 2024-04-17T10:41:23.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.755+0000 7fbe12655640 1 --2- 172.21.15.83:0/3093984928 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbdf8066f60 0x7fbdf8069420 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:23.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.755+0000 7fbe12655640 1 --2- 172.21.15.83:0/3093984928 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe0c15bac0 0x7fbe0c159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:23.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.755+0000 7fbe12655640 1 --2- 172.21.15.83:0/3093984928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe0c15b6e0 0x7fbe0c158ce0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:23.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.755+0000 7fbe12655640 1 --2- 172.21.15.83:0/3093984928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe0c070470 0x7fbe0c1479c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:23.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.755+0000 7fbe12655640 1 -- 172.21.15.83:0/3093984928 >> 172.21.15.83:0/3093984928 conn(0x7fbe0c0b15b0 msgr2=0x7fbe0c0b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:23.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.755+0000 7fbe12655640 1 -- 172.21.15.83:0/3093984928 shutdown_connections 2024-04-17T10:41:23.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.755+0000 7fbe12655640 1 -- 172.21.15.83:0/3093984928 wait complete. 2024-04-17T10:41:23.771 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:41:23.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.895+0000 7f8da6647640 1 Processor -- start 2024-04-17T10:41:23.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.895+0000 7f8da6647640 1 -- start start 2024-04-17T10:41:23.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.895+0000 7f8da6647640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8da015dab0 0x7f8da0161840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:23.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.895+0000 7f8da6647640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8da015de90 0x7f8da0161d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:23.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.895+0000 7f8da6647640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8da00ce4e0 con 0x7f8da01622c0 2024-04-17T10:41:23.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.895+0000 7f8da6647640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8da00cf370 con 0x7f8da015de90 2024-04-17T10:41:23.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.895+0000 7f8da6647640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8da00d4910 con 0x7f8da015dab0 2024-04-17T10:41:23.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.895+0000 7f8d9f7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8da015de90 0x7f8da0161d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:23.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.895+0000 7f8d9f7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8da015de90 0x7f8da0161d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36592/0 (socket says 172.21.15.83:36592) 2024-04-17T10:41:23.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.895+0000 7f8d9f7fe640 1 -- 172.21.15.83:0/1309772875 learned_addr learned my addr 172.21.15.83:0/1309772875 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:23.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.895+0000 7f8d9ffff640 1 --2- 172.21.15.83:0/1309772875 >> v2:172.21.15.167:3300/0 conn(0x7f8da015dab0 0x7f8da0161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:23.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.895+0000 7f8d9f7fe640 1 -- 172.21.15.83:0/1309772875 >> v2:172.21.15.167:3300/0 conn(0x7f8da015dab0 msgr2=0x7f8da0161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:23.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.895+0000 7f8d9f7fe640 1 --2- 172.21.15.83:0/1309772875 >> v2:172.21.15.167:3300/0 conn(0x7f8da015dab0 0x7f8da0161840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:23.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.895+0000 7f8d9f7fe640 1 -- 172.21.15.83:0/1309772875 >> v1:172.21.15.167:6789/0 conn(0x7f8da01622c0 legacy=0x7f8da00d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:23.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.895+0000 7f8d9f7fe640 1 -- 172.21.15.83:0/1309772875 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da00ce760 con 0x7f8da015de90 2024-04-17T10:41:23.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.899+0000 7f8d9f7fe640 1 --2- 172.21.15.83:0/1309772875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8da015de90 0x7f8da0161d80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8da00b65c0 tx=0x7f8d9401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a249611a75fa835 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:23.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.899+0000 7f8d9effd640 1 -- 172.21.15.83:0/1309772875 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d940b0070 con 0x7f8da015de90 2024-04-17T10:41:23.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.899+0000 7f8d9effd640 1 -- 172.21.15.83:0/1309772875 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d940ab600 con 0x7f8da015de90 2024-04-17T10:41:23.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.899+0000 7f8d9effd640 1 -- 172.21.15.83:0/1309772875 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d940ab9d0 con 0x7f8da015de90 2024-04-17T10:41:23.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.899+0000 7f8da6647640 1 -- 172.21.15.83:0/1309772875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8da015de90 msgr2=0x7f8da0161d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:23.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.899+0000 7f8da6647640 1 --2- 172.21.15.83:0/1309772875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8da015de90 0x7f8da0161d80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8da00b65c0 tx=0x7f8d9401c520 comp rx=0 tx=0).stop 2024-04-17T10:41:23.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.899+0000 7f8da6647640 1 -- 172.21.15.83:0/1309772875 shutdown_connections 2024-04-17T10:41:23.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.899+0000 7f8da6647640 1 --2- 172.21.15.83:0/1309772875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8da015de90 0x7f8da0161d80 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:23.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.899+0000 7f8da6647640 1 --2- 172.21.15.83:0/1309772875 >> v2:172.21.15.167:3300/0 conn(0x7f8da015dab0 0x7f8da0161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:23.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.899+0000 7f8da6647640 1 -- 172.21.15.83:0/1309772875 >> 172.21.15.83:0/1309772875 conn(0x7f8da009cd70 msgr2=0x7f8da00d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:23.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.899+0000 7f8da6647640 1 -- 172.21.15.83:0/1309772875 shutdown_connections 2024-04-17T10:41:23.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.899+0000 7f8da6647640 1 -- 172.21.15.83:0/1309772875 wait complete. 2024-04-17T10:41:23.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.899+0000 7f8da6647640 1 Processor -- start 2024-04-17T10:41:23.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.899+0000 7f8da6647640 1 -- start start 2024-04-17T10:41:23.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.903+0000 7f8da6647640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8da015dab0 0x7f8da0155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:23.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.903+0000 7f8da6647640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8da015de90 0x7f8da00cb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:23.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.903+0000 7f8da6647640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8da01622c0 0x7f8da00c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:23.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.903+0000 7f8da6647640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da00cf960 con 0x7f8da015de90 2024-04-17T10:41:23.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.903+0000 7f8da6647640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da00cf7e0 con 0x7f8da01622c0 2024-04-17T10:41:23.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.903+0000 7f8da6647640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da00d24c0 con 0x7f8da015dab0 2024-04-17T10:41:23.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.903+0000 7f8d9f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8da015de90 0x7f8da00cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:23.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.903+0000 7f8d9f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8da015de90 0x7f8da00cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36608/0 (socket says 172.21.15.83:36608) 2024-04-17T10:41:23.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.903+0000 7f8d9f7fe640 1 -- 172.21.15.83:0/2039838917 learned_addr learned my addr 172.21.15.83:0/2039838917 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:23.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.903+0000 7f8d9ffff640 1 --2- 172.21.15.83:0/2039838917 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8da015dab0 0x7f8da0155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:23.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.903+0000 7f8da4be4640 1 --2- 172.21.15.83:0/2039838917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8da01622c0 0x7f8da00c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:23.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.903+0000 7f8d9f7fe640 1 -- 172.21.15.83:0/2039838917 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8da015dab0 msgr2=0x7f8da0155e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:23.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.903+0000 7f8d9f7fe640 1 --2- 172.21.15.83:0/2039838917 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8da015dab0 0x7f8da0155e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:23.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.903+0000 7f8d9f7fe640 1 -- 172.21.15.83:0/2039838917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8da01622c0 msgr2=0x7f8da00c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:23.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.903+0000 7f8d9f7fe640 1 --2- 172.21.15.83:0/2039838917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8da01622c0 0x7f8da00c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:23.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.903+0000 7f8d9f7fe640 1 -- 172.21.15.83:0/2039838917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da00c8cf0 con 0x7f8da015de90 2024-04-17T10:41:23.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.903+0000 7f8d9f7fe640 1 --2- 172.21.15.83:0/2039838917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8da015de90 0x7f8da00cb540 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8d9401ca00 tx=0x7f8d9401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:23.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.903+0000 7f8d9cff9640 1 -- 172.21.15.83:0/2039838917 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d940ba020 con 0x7f8da015de90 2024-04-17T10:41:23.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.903+0000 7f8d9cff9640 1 -- 172.21.15.83:0/2039838917 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d940b6420 con 0x7f8da015de90 2024-04-17T10:41:23.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.903+0000 7f8da6647640 1 -- 172.21.15.83:0/2039838917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da00c8fd0 con 0x7f8da015de90 2024-04-17T10:41:23.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.903+0000 7f8d9cff9640 1 -- 172.21.15.83:0/2039838917 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d940b0040 con 0x7f8da015de90 2024-04-17T10:41:23.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.903+0000 7f8da6647640 1 -- 172.21.15.83:0/2039838917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da00c9590 con 0x7f8da015de90 2024-04-17T10:41:23.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.903+0000 7f8da6647640 1 -- 172.21.15.83:0/2039838917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da00681a0 con 0x7f8da015de90 2024-04-17T10:41:23.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.903+0000 7f8d9cff9640 1 -- 172.21.15.83:0/2039838917 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8d940bd020 con 0x7f8da015de90 2024-04-17T10:41:23.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.903+0000 7f8d9cff9640 1 --2- 172.21.15.83:0/2039838917 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8d8c066f60 0x7f8d8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:23.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.903+0000 7f8d9cff9640 1 -- 172.21.15.83:0/2039838917 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f8d9411f0f0 con 0x7f8da015de90 2024-04-17T10:41:23.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.903+0000 7f8d9ffff640 1 --2- 172.21.15.83:0/2039838917 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8d8c066f60 0x7f8d8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:23.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.903+0000 7f8d9ffff640 1 --2- 172.21.15.83:0/2039838917 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8d8c066f60 0x7f8d8c069420 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8d90009af0 tx=0x7f8d90008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:23.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:23.907+0000 7f8d9cff9640 1 -- 172.21.15.83:0/2039838917 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8d940c4030 con 0x7f8da015de90 2024-04-17T10:41:24.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.063+0000 7f8da6647640 1 -- 172.21.15.83:0/2039838917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8da0074f90 con 0x7f8da015de90 2024-04-17T10:41:24.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.063+0000 7f8d9cff9640 1 -- 172.21.15.83:0/2039838917 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8d94016110 con 0x7f8da015de90 2024-04-17T10:41:24.063 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:41:24.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.067+0000 7f8da6647640 1 -- 172.21.15.83:0/2039838917 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8d8c066f60 msgr2=0x7f8d8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:24.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.067+0000 7f8da6647640 1 --2- 172.21.15.83:0/2039838917 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8d8c066f60 0x7f8d8c069420 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8d90009af0 tx=0x7f8d90008040 comp rx=0 tx=0).stop 2024-04-17T10:41:24.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.067+0000 7f8da6647640 1 -- 172.21.15.83:0/2039838917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8da015de90 msgr2=0x7f8da00cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:24.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.067+0000 7f8da6647640 1 --2- 172.21.15.83:0/2039838917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8da015de90 0x7f8da00cb540 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8d9401ca00 tx=0x7f8d9401ad60 comp rx=0 tx=0).stop 2024-04-17T10:41:24.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.067+0000 7f8da6647640 1 -- 172.21.15.83:0/2039838917 shutdown_connections 2024-04-17T10:41:24.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.067+0000 7f8da6647640 1 --2- 172.21.15.83:0/2039838917 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8d8c066f60 0x7f8d8c069420 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:24.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.067+0000 7f8da6647640 1 --2- 172.21.15.83:0/2039838917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8da01622c0 0x7f8da00c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:24.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.067+0000 7f8da6647640 1 --2- 172.21.15.83:0/2039838917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8da015de90 0x7f8da00cb540 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:24.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.067+0000 7f8da6647640 1 --2- 172.21.15.83:0/2039838917 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8da015dab0 0x7f8da0155e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:24.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.067+0000 7f8da6647640 1 -- 172.21.15.83:0/2039838917 >> 172.21.15.83:0/2039838917 conn(0x7f8da009cd70 msgr2=0x7f8da006bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:24.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.067+0000 7f8da6647640 1 -- 172.21.15.83:0/2039838917 shutdown_connections 2024-04-17T10:41:24.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.067+0000 7f8da6647640 1 -- 172.21.15.83:0/2039838917 wait complete. 2024-04-17T10:41:24.082 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:41:24.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.191+0000 7fcd490f5640 1 Processor -- start 2024-04-17T10:41:24.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.191+0000 7fcd490f5640 1 -- start start 2024-04-17T10:41:24.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.195+0000 7fcd490f5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcd44072330 0x7fcd440cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:24.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.195+0000 7fcd490f5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcd440cfa10 0x7fcd4406c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:24.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.195+0000 7fcd490f5640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd440cfea0 con 0x7fcd4415ba30 2024-04-17T10:41:24.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.195+0000 7fcd490f5640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd44071ea0 con 0x7fcd44072330 2024-04-17T10:41:24.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.195+0000 7fcd490f5640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd4406e320 con 0x7fcd440cfa10 2024-04-17T10:41:24.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.195+0000 7fcd4259c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcd44072330 0x7fcd440cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:24.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.195+0000 7fcd42d9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcd4415ba30 0x7fcd4415c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54054/0 (socket says 172.21.15.83:54054) 2024-04-17T10:41:24.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.195+0000 7fcd42d9d640 1 -- 172.21.15.83:0/24132875 learned_addr learned my addr 172.21.15.83:0/24132875 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:24.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.195+0000 7fcd4359e640 1 --2- 172.21.15.83:0/24132875 >> v2:172.21.15.167:3300/0 conn(0x7fcd440cfa10 0x7fcd4406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:24.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.195+0000 7fcd4259c640 1 -- 172.21.15.83:0/24132875 >> v2:172.21.15.167:3300/0 conn(0x7fcd440cfa10 msgr2=0x7fcd4406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:24.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.195+0000 7fcd4259c640 1 --2- 172.21.15.83:0/24132875 >> v2:172.21.15.167:3300/0 conn(0x7fcd440cfa10 0x7fcd4406c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:24.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.195+0000 7fcd4259c640 1 -- 172.21.15.83:0/24132875 >> v1:172.21.15.83:6789/0 conn(0x7fcd4415ba30 legacy=0x7fcd4415c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:24.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.195+0000 7fcd4259c640 1 -- 172.21.15.83:0/24132875 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd4406e040 con 0x7fcd44072330 2024-04-17T10:41:24.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.195+0000 7fcd41d9b640 1 -- 172.21.15.83:0/24132875 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 72700498 0 0) 0x7fcd440cfea0 con 0x7fcd4415ba30 2024-04-17T10:41:24.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.195+0000 7fcd4259c640 1 --2- 172.21.15.83:0/24132875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd44072330 0x7fcd440cf4d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fcd440676a0 tx=0x7fcd3801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e690c3c6bbe4d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:24.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.195+0000 7fcd41d9b640 1 -- 172.21.15.83:0/24132875 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd380b0070 con 0x7fcd44072330 2024-04-17T10:41:24.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.195+0000 7fcd41d9b640 1 -- 172.21.15.83:0/24132875 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd380ab600 con 0x7fcd44072330 2024-04-17T10:41:24.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.195+0000 7fcd41d9b640 1 -- 172.21.15.83:0/24132875 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd380ab9d0 con 0x7fcd44072330 2024-04-17T10:41:24.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.195+0000 7fcd490f5640 1 -- 172.21.15.83:0/24132875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd44072330 msgr2=0x7fcd440cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:24.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.195+0000 7fcd490f5640 1 --2- 172.21.15.83:0/24132875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd44072330 0x7fcd440cf4d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fcd440676a0 tx=0x7fcd3801c3f0 comp rx=0 tx=0).stop 2024-04-17T10:41:24.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.195+0000 7fcd490f5640 1 -- 172.21.15.83:0/24132875 shutdown_connections 2024-04-17T10:41:24.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.195+0000 7fcd490f5640 1 --2- 172.21.15.83:0/24132875 >> v2:172.21.15.167:3300/0 conn(0x7fcd440cfa10 0x7fcd4406c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:24.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.195+0000 7fcd490f5640 1 --2- 172.21.15.83:0/24132875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd44072330 0x7fcd440cf4d0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:24.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.195+0000 7fcd490f5640 1 -- 172.21.15.83:0/24132875 >> 172.21.15.83:0/24132875 conn(0x7fcd4406abe0 msgr2=0x7fcd4406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:24.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.195+0000 7fcd490f5640 1 -- 172.21.15.83:0/24132875 shutdown_connections 2024-04-17T10:41:24.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.195+0000 7fcd490f5640 1 -- 172.21.15.83:0/24132875 wait complete. 2024-04-17T10:41:24.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.199+0000 7fcd490f5640 1 Processor -- start 2024-04-17T10:41:24.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.199+0000 7fcd490f5640 1 -- start start 2024-04-17T10:41:24.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.199+0000 7fcd490f5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd44072330 0x7fcd44157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:24.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.199+0000 7fcd490f5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd440cfa10 0x7fcd4413d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:24.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.199+0000 7fcd490f5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd4415ba30 0x7fcd440af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:24.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.199+0000 7fcd490f5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4406de20 con 0x7fcd4415ba30 2024-04-17T10:41:24.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.199+0000 7fcd490f5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4406dca0 con 0x7fcd44072330 2024-04-17T10:41:24.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.199+0000 7fcd490f5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd44168520 con 0x7fcd440cfa10 2024-04-17T10:41:24.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.199+0000 7fcd4359e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd4415ba30 0x7fcd440af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:24.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.199+0000 7fcd4359e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd4415ba30 0x7fcd440af220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36622/0 (socket says 172.21.15.83:36622) 2024-04-17T10:41:24.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.199+0000 7fcd4359e640 1 -- 172.21.15.83:0/750884989 learned_addr learned my addr 172.21.15.83:0/750884989 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:24.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.199+0000 7fcd4259c640 1 --2- 172.21.15.83:0/750884989 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd440cfa10 0x7fcd4413d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:24.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.199+0000 7fcd42d9d640 1 --2- 172.21.15.83:0/750884989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd44072330 0x7fcd44157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:24.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.199+0000 7fcd4359e640 1 -- 172.21.15.83:0/750884989 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd440cfa10 msgr2=0x7fcd4413d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:24.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.199+0000 7fcd4359e640 1 --2- 172.21.15.83:0/750884989 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd440cfa10 0x7fcd4413d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:24.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.199+0000 7fcd4359e640 1 -- 172.21.15.83:0/750884989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd44072330 msgr2=0x7fcd44157960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:24.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.199+0000 7fcd4359e640 1 --2- 172.21.15.83:0/750884989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd44072330 0x7fcd44157960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:24.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.199+0000 7fcd4359e640 1 -- 172.21.15.83:0/750884989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd4413d9e0 con 0x7fcd4415ba30 2024-04-17T10:41:24.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.199+0000 7fcd42d9d640 1 --2- 172.21.15.83:0/750884989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd44072330 0x7fcd44157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:24.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.199+0000 7fcd4359e640 1 --2- 172.21.15.83:0/750884989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd4415ba30 0x7fcd440af220 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c007e40 tx=0x7fcd3c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:24.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.199+0000 7fcd2b7fe640 1 -- 172.21.15.83:0/750884989 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c014620 con 0x7fcd4415ba30 2024-04-17T10:41:24.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.199+0000 7fcd2b7fe640 1 -- 172.21.15.83:0/750884989 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3c019070 con 0x7fcd4415ba30 2024-04-17T10:41:24.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.199+0000 7fcd490f5640 1 -- 172.21.15.83:0/750884989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd441568c0 con 0x7fcd4415ba30 2024-04-17T10:41:24.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.199+0000 7fcd2b7fe640 1 -- 172.21.15.83:0/750884989 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c014a20 con 0x7fcd4415ba30 2024-04-17T10:41:24.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.199+0000 7fcd490f5640 1 -- 172.21.15.83:0/750884989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd44156e80 con 0x7fcd4415ba30 2024-04-17T10:41:24.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.203+0000 7fcd490f5640 1 -- 172.21.15.83:0/750884989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd44074f90 con 0x7fcd4415ba30 2024-04-17T10:41:24.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.203+0000 7fcd2b7fe640 1 -- 172.21.15.83:0/750884989 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcd3c014bc0 con 0x7fcd4415ba30 2024-04-17T10:41:24.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.203+0000 7fcd2b7fe640 1 --2- 172.21.15.83:0/750884989 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd30066f60 0x7fcd30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:24.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.203+0000 7fcd42d9d640 1 --2- 172.21.15.83:0/750884989 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd30066f60 0x7fcd30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:24.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.203+0000 7fcd2b7fe640 1 -- 172.21.15.83:0/750884989 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fcd3c085a00 con 0x7fcd4415ba30 2024-04-17T10:41:24.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.203+0000 7fcd42d9d640 1 --2- 172.21.15.83:0/750884989 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd30066f60 0x7fcd30069420 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcd34000f70 tx=0x7fcd34009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:24.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.207+0000 7fcd2b7fe640 1 -- 172.21.15.83:0/750884989 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcd4413d9e0 con 0x7fcd4415ba30 2024-04-17T10:41:24.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.387+0000 7fcd490f5640 1 -- 172.21.15.83:0/750884989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd440cf6c0 con 0x7fcd4415ba30 2024-04-17T10:41:24.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.391+0000 7fcd2b7fe640 1 -- 172.21.15.83:0/750884989 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fcd3c056950 con 0x7fcd4415ba30 2024-04-17T10:41:24.387 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:41:24.388 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1456029744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6833","nonce":1456029744},{"type":"v1","addr":"172.21.15.167:6835","nonce":1456029744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6834/648555908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":648555908},{"type":"v1","addr":"172.21.15.167:6834","nonce":648555908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5163,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1435807773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1435807773},{"type":"v1","addr":"172.21.15.83:6835","nonce":1435807773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-17T10:39:13.472135+0000","modified":"2024-04-17T10:39:21.807600+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5172},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5172":{"gid":5172,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1181902025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1181902025},{"type":"v1","addr":"172.21.15.83:6837","nonce":1181902025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-17T10:41:24.388 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 14 2024-04-17T10:41:24.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.391+0000 7fcd490f5640 1 -- 172.21.15.83:0/750884989 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd30066f60 msgr2=0x7fcd30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:24.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.391+0000 7fcd490f5640 1 --2- 172.21.15.83:0/750884989 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd30066f60 0x7fcd30069420 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcd34000f70 tx=0x7fcd34009040 comp rx=0 tx=0).stop 2024-04-17T10:41:24.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.391+0000 7fcd490f5640 1 -- 172.21.15.83:0/750884989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd4415ba30 msgr2=0x7fcd440af220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:24.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.391+0000 7fcd490f5640 1 --2- 172.21.15.83:0/750884989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd4415ba30 0x7fcd440af220 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c007e40 tx=0x7fcd3c00e4b0 comp rx=0 tx=0).stop 2024-04-17T10:41:24.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.391+0000 7fcd490f5640 1 -- 172.21.15.83:0/750884989 shutdown_connections 2024-04-17T10:41:24.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.391+0000 7fcd490f5640 1 --2- 172.21.15.83:0/750884989 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd30066f60 0x7fcd30069420 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:24.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.391+0000 7fcd490f5640 1 --2- 172.21.15.83:0/750884989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd4415ba30 0x7fcd440af220 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:24.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.391+0000 7fcd490f5640 1 --2- 172.21.15.83:0/750884989 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd440cfa10 0x7fcd4413d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:24.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.391+0000 7fcd490f5640 1 --2- 172.21.15.83:0/750884989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd44072330 0x7fcd44157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:24.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.391+0000 7fcd490f5640 1 -- 172.21.15.83:0/750884989 >> 172.21.15.83:0/750884989 conn(0x7fcd4406abe0 msgr2=0x7fcd44159be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:24.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.391+0000 7fcd490f5640 1 -- 172.21.15.83:0/750884989 shutdown_connections 2024-04-17T10:41:24.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.391+0000 7fcd490f5640 1 -- 172.21.15.83:0/750884989 wait complete. 2024-04-17T10:41:24.405 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:41:24.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc36bb5e640 1 Processor -- start 2024-04-17T10:41:24.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc36bb5e640 1 -- start start 2024-04-17T10:41:24.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc36bb5e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc3640ce980 0x7fc3640b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:24.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc36bb5e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc364162a00 con 0x7fc36415dfa0 2024-04-17T10:41:24.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc36bb5e640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc364161e80 con 0x7fc3640cee00 2024-04-17T10:41:24.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc36bb5e640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc364162160 con 0x7fc3640ce980 2024-04-17T10:41:24.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc3698fa640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc36415dfa0 0x7fc3640b6c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54066/0 (socket says 172.21.15.83:54066) 2024-04-17T10:41:24.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc3698fa640 1 -- 172.21.15.83:0/3447910878 learned_addr learned my addr 172.21.15.83:0/3447910878 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:24.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc3690f9640 1 --2- 172.21.15.83:0/3447910878 >> v2:172.21.15.167:3300/0 conn(0x7fc3640ce980 0x7fc3640b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:24.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc3688f8640 1 -- 172.21.15.83:0/3447910878 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2179737318 0 0) 0x7fc364162a00 con 0x7fc36415dfa0 2024-04-17T10:41:24.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc3688f8640 1 -- 172.21.15.83:0/3447910878 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc350003530 con 0x7fc36415dfa0 2024-04-17T10:41:24.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc3688f8640 1 -- 172.21.15.83:0/3447910878 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2556414050 0 0) 0x7fc350003530 con 0x7fc36415dfa0 2024-04-17T10:41:24.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc3688f8640 1 -- 172.21.15.83:0/3447910878 >> v2:172.21.15.167:3300/0 conn(0x7fc3640ce980 msgr2=0x7fc3640b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:24.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc3688f8640 1 --2- 172.21.15.83:0/3447910878 >> v2:172.21.15.167:3300/0 conn(0x7fc3640ce980 0x7fc3640b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:24.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc3688f8640 1 -- 172.21.15.83:0/3447910878 >> v1:172.21.15.167:6789/0 conn(0x7fc3640cee00 legacy=0x7fc3640d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:24.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc3688f8640 1 -- 172.21.15.83:0/3447910878 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc364162440 con 0x7fc36415dfa0 2024-04-17T10:41:24.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc3688f8640 1 -- 172.21.15.83:0/3447910878 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fc360001680 con 0x7fc36415dfa0 2024-04-17T10:41:24.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc3690f9640 1 --2- 172.21.15.83:0/3447910878 >> v2:172.21.15.167:3300/0 conn(0x7fc3640ce980 0x7fc3640b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:41:24.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc3688f8640 1 -- 172.21.15.83:0/3447910878 >> v1:172.21.15.83:6789/0 conn(0x7fc36415dfa0 legacy=0x7fc3640b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:24.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc3688f8640 1 --2- 172.21.15.83:0/3447910878 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3500039c0 0x7fc350003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:24.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc3688f8640 1 --2- 172.21.15.83:0/3447910878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc350004810 0x7fc350004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:24.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc3688f8640 1 --2- 172.21.15.83:0/3447910878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3500052f0 0x7fc35000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:24.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc3688f8640 1 -- 172.21.15.83:0/3447910878 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc364162160 con 0x7fc350004810 2024-04-17T10:41:24.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc3688f8640 1 -- 172.21.15.83:0/3447910878 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35000c1c0 con 0x7fc3500052f0 2024-04-17T10:41:24.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc3688f8640 1 -- 172.21.15.83:0/3447910878 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc35000c3c0 con 0x7fc3500039c0 2024-04-17T10:41:24.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc3690f9640 1 --2- 172.21.15.83:0/3447910878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc350004810 0x7fc350004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:24.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc36a0fb640 1 --2- 172.21.15.83:0/3447910878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3500052f0 0x7fc35000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:24.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc3698fa640 1 --2- 172.21.15.83:0/3447910878 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3500039c0 0x7fc350003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:24.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc3690f9640 1 -- 172.21.15.83:0/3447910878 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3500039c0 msgr2=0x7fc350003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:24.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc3690f9640 1 --2- 172.21.15.83:0/3447910878 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3500039c0 0x7fc350003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:24.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc3690f9640 1 -- 172.21.15.83:0/3447910878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3500052f0 msgr2=0x7fc35000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:24.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc3690f9640 1 --2- 172.21.15.83:0/3447910878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3500052f0 0x7fc35000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:24.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc36a0fb640 1 -- 172.21.15.83:0/3447910878 reap_dead start 2024-04-17T10:41:24.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.527+0000 7fc3690f9640 1 -- 172.21.15.83:0/3447910878 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc350003530 con 0x7fc350004810 2024-04-17T10:41:24.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.531+0000 7fc3690f9640 1 --2- 172.21.15.83:0/3447910878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc350004810 0x7fc350004c90 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc3540098a0 tx=0x7fc35401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c304b015c8d6a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:24.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.531+0000 7fc3688f8640 1 -- 172.21.15.83:0/3447910878 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35400f040 con 0x7fc350004810 2024-04-17T10:41:24.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.531+0000 7fc3688f8640 1 -- 172.21.15.83:0/3447910878 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3540a95f0 con 0x7fc350004810 2024-04-17T10:41:24.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.531+0000 7fc3688f8640 1 -- 172.21.15.83:0/3447910878 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3540a99a0 con 0x7fc350004810 2024-04-17T10:41:24.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.531+0000 7fc36bb5e640 1 -- 172.21.15.83:0/3447910878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc350004810 msgr2=0x7fc350004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:24.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.531+0000 7fc36bb5e640 1 --2- 172.21.15.83:0/3447910878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc350004810 0x7fc350004c90 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc3540098a0 tx=0x7fc35401c560 comp rx=0 tx=0).stop 2024-04-17T10:41:24.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.531+0000 7fc36bb5e640 1 -- 172.21.15.83:0/3447910878 shutdown_connections 2024-04-17T10:41:24.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.531+0000 7fc36bb5e640 1 --2- 172.21.15.83:0/3447910878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc350004810 0x7fc350004c90 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:24.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.531+0000 7fc36bb5e640 1 -- 172.21.15.83:0/3447910878 >> 172.21.15.83:0/3447910878 conn(0x7fc3640d0600 msgr2=0x7fc3640d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:24.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.531+0000 7fc36bb5e640 1 -- 172.21.15.83:0/3447910878 shutdown_connections 2024-04-17T10:41:24.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.531+0000 7fc36bb5e640 1 -- 172.21.15.83:0/3447910878 wait complete. 2024-04-17T10:41:24.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.531+0000 7fc36bb5e640 1 Processor -- start 2024-04-17T10:41:24.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.531+0000 7fc36bb5e640 1 -- start start 2024-04-17T10:41:24.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.531+0000 7fc36bb5e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc350004810 0x7fc3640c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:24.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.531+0000 7fc36bb5e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3641511f0 0x7fc3641f6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:24.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.531+0000 7fc36bb5e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3640c9d90 0x7fc3640cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:24.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.531+0000 7fc36bb5e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3640b71b0 con 0x7fc3640c9d90 2024-04-17T10:41:24.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.531+0000 7fc36bb5e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3640b7030 con 0x7fc350004810 2024-04-17T10:41:24.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.531+0000 7fc36bb5e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3640d1d30 con 0x7fc3641511f0 2024-04-17T10:41:24.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.535+0000 7fc36a0fb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3640c9d90 0x7fc3640cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:24.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.535+0000 7fc36a0fb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3640c9d90 0x7fc3640cbd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36640/0 (socket says 172.21.15.83:36640) 2024-04-17T10:41:24.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.535+0000 7fc36a0fb640 1 -- 172.21.15.83:0/1725902628 learned_addr learned my addr 172.21.15.83:0/1725902628 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:24.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.535+0000 7fc3698fa640 1 --2- 172.21.15.83:0/1725902628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc350004810 0x7fc3640c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:24.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.535+0000 7fc3690f9640 1 --2- 172.21.15.83:0/1725902628 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3641511f0 0x7fc3641f6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:24.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.535+0000 7fc36a0fb640 1 -- 172.21.15.83:0/1725902628 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3641511f0 msgr2=0x7fc3641f6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:24.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.535+0000 7fc36a0fb640 1 --2- 172.21.15.83:0/1725902628 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3641511f0 0x7fc3641f6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:24.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.535+0000 7fc36a0fb640 1 -- 172.21.15.83:0/1725902628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc350004810 msgr2=0x7fc3640c85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:24.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.535+0000 7fc36a0fb640 1 --2- 172.21.15.83:0/1725902628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc350004810 0x7fc3640c85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:24.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.535+0000 7fc36a0fb640 1 -- 172.21.15.83:0/1725902628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc364161550 con 0x7fc3640c9d90 2024-04-17T10:41:24.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.535+0000 7fc36a0fb640 1 --2- 172.21.15.83:0/1725902628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3640c9d90 0x7fc3640cbd60 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc36406dae0 tx=0x7fc35c00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:24.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.535+0000 7fc35a7fc640 1 -- 172.21.15.83:0/1725902628 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c019070 con 0x7fc3640c9d90 2024-04-17T10:41:24.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.535+0000 7fc35a7fc640 1 -- 172.21.15.83:0/1725902628 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35c006b60 con 0x7fc3640c9d90 2024-04-17T10:41:24.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.535+0000 7fc36bb5e640 1 -- 172.21.15.83:0/1725902628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3640c8cf0 con 0x7fc3640c9d90 2024-04-17T10:41:24.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.535+0000 7fc35a7fc640 1 -- 172.21.15.83:0/1725902628 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c003980 con 0x7fc3640c9d90 2024-04-17T10:41:24.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.535+0000 7fc36bb5e640 1 -- 172.21.15.83:0/1725902628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3640c92b0 con 0x7fc3640c9d90 2024-04-17T10:41:24.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.535+0000 7fc36bb5e640 1 -- 172.21.15.83:0/1725902628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3640681a0 con 0x7fc3640c9d90 2024-04-17T10:41:24.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.535+0000 7fc35a7fc640 1 -- 172.21.15.83:0/1725902628 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc35c01c070 con 0x7fc3640c9d90 2024-04-17T10:41:24.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.535+0000 7fc35a7fc640 1 --2- 172.21.15.83:0/1725902628 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc348066f60 0x7fc348069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:24.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.535+0000 7fc35a7fc640 1 -- 172.21.15.83:0/1725902628 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fc35c084560 con 0x7fc3640c9d90 2024-04-17T10:41:24.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.535+0000 7fc3698fa640 1 --2- 172.21.15.83:0/1725902628 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc348066f60 0x7fc348069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:24.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.535+0000 7fc3698fa640 1 --2- 172.21.15.83:0/1725902628 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc348066f60 0x7fc348069420 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc360001ce0 tx=0x7fc360009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:24.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.539+0000 7fc35a7fc640 1 -- 172.21.15.83:0/1725902628 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc35c055550 con 0x7fc3640c9d90 2024-04-17T10:41:24.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.687+0000 7fc36bb5e640 1 -- 172.21.15.83:0/1725902628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc364074f90 con 0x7fc3640c9d90 2024-04-17T10:41:24.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.687+0000 7fc35a7fc640 1 -- 172.21.15.83:0/1725902628 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16662 (secure 0 0 0) 0x7fc35c054e50 con 0x7fc3640c9d90 2024-04-17T10:41:24.686 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:41:24.686 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":35,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:40:57.038108+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:39:11.330695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-17T10:39:12.438377+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":34,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:41:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.691+0000 7fc36bb5e640 1 -- 172.21.15.83:0/1725902628 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc348066f60 msgr2=0x7fc348069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.691+0000 7fc36bb5e640 1 --2- 172.21.15.83:0/1725902628 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc348066f60 0x7fc348069420 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc360001ce0 tx=0x7fc360009040 comp rx=0 tx=0).stop 2024-04-17T10:41:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.691+0000 7fc36bb5e640 1 -- 172.21.15.83:0/1725902628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3640c9d90 msgr2=0x7fc3640cbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.691+0000 7fc36bb5e640 1 --2- 172.21.15.83:0/1725902628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3640c9d90 0x7fc3640cbd60 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc36406dae0 tx=0x7fc35c00eb90 comp rx=0 tx=0).stop 2024-04-17T10:41:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.691+0000 7fc36bb5e640 1 -- 172.21.15.83:0/1725902628 shutdown_connections 2024-04-17T10:41:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.691+0000 7fc36bb5e640 1 --2- 172.21.15.83:0/1725902628 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc348066f60 0x7fc348069420 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.691+0000 7fc36bb5e640 1 --2- 172.21.15.83:0/1725902628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3640c9d90 0x7fc3640cbd60 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.691+0000 7fc36bb5e640 1 --2- 172.21.15.83:0/1725902628 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3641511f0 0x7fc3641f6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.691+0000 7fc36bb5e640 1 --2- 172.21.15.83:0/1725902628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc350004810 0x7fc3640c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.691+0000 7fc36bb5e640 1 -- 172.21.15.83:0/1725902628 >> 172.21.15.83:0/1725902628 conn(0x7fc3640d0600 msgr2=0x7fc364158020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:24.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.691+0000 7fc36bb5e640 1 -- 172.21.15.83:0/1725902628 shutdown_connections 2024-04-17T10:41:24.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.691+0000 7fc36bb5e640 1 -- 172.21.15.83:0/1725902628 wait complete. 2024-04-17T10:41:24.704 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-17T10:41:24.704 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:41:24.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.823+0000 7f1fb03fb640 1 Processor -- start 2024-04-17T10:41:24.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.823+0000 7f1fb03fb640 1 -- start start 2024-04-17T10:41:24.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.827+0000 7f1fb03fb640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1fa80aedd0 0x7f1fa806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:24.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.827+0000 7f1fb03fb640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1fa806dae0 0x7f1fa80b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:24.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.827+0000 7f1fb03fb640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fa815e440 con 0x7f1fa815db50 2024-04-17T10:41:24.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.827+0000 7f1fb03fb640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fa815e720 con 0x7f1fa806dae0 2024-04-17T10:41:24.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.827+0000 7f1fb03fb640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fa815ea00 con 0x7f1fa80aedd0 2024-04-17T10:41:24.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.827+0000 7f1fae998640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1fa806dae0 0x7f1fa80b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:24.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.827+0000 7f1fae197640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1fa815db50 0x7f1fa8070470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54078/0 (socket says 172.21.15.83:54078) 2024-04-17T10:41:24.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.827+0000 7f1fae197640 1 -- 172.21.15.83:0/503270828 learned_addr learned my addr 172.21.15.83:0/503270828 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:24.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.827+0000 7f1fad996640 1 --2- 172.21.15.83:0/503270828 >> v2:172.21.15.167:3300/0 conn(0x7f1fa80aedd0 0x7f1fa806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:24.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.827+0000 7f1fae998640 1 -- 172.21.15.83:0/503270828 >> v2:172.21.15.167:3300/0 conn(0x7f1fa80aedd0 msgr2=0x7f1fa806d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:24.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.827+0000 7f1fae998640 1 --2- 172.21.15.83:0/503270828 >> v2:172.21.15.167:3300/0 conn(0x7f1fa80aedd0 0x7f1fa806d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:24.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.827+0000 7f1fae998640 1 -- 172.21.15.83:0/503270828 >> v1:172.21.15.83:6789/0 conn(0x7f1fa815db50 legacy=0x7f1fa8070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:24.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.827+0000 7f1fad195640 1 -- 172.21.15.83:0/503270828 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3133476780 0 0) 0x7f1fa815e440 con 0x7f1fa815db50 2024-04-17T10:41:24.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.827+0000 7f1fae998640 1 -- 172.21.15.83:0/503270828 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa815ece0 con 0x7f1fa806dae0 2024-04-17T10:41:24.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.827+0000 7f1fae998640 1 --2- 172.21.15.83:0/503270828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fa806dae0 0x7f1fa80b2e10 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009d90 tx=0x7f1fa001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8bb7d2d1743a4cf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:24.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.827+0000 7f1fad195640 1 -- 172.21.15.83:0/503270828 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa00b2070 con 0x7f1fa806dae0 2024-04-17T10:41:24.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.827+0000 7f1fad195640 1 -- 172.21.15.83:0/503270828 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa00a57f0 con 0x7f1fa806dae0 2024-04-17T10:41:24.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.827+0000 7f1fad195640 1 -- 172.21.15.83:0/503270828 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa00a5bc0 con 0x7f1fa806dae0 2024-04-17T10:41:24.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.831+0000 7f1fb03fb640 1 -- 172.21.15.83:0/503270828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fa806dae0 msgr2=0x7f1fa80b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:24.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.831+0000 7f1fb03fb640 1 --2- 172.21.15.83:0/503270828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fa806dae0 0x7f1fa80b2e10 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009d90 tx=0x7f1fa001c930 comp rx=0 tx=0).stop 2024-04-17T10:41:24.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.831+0000 7f1fb03fb640 1 -- 172.21.15.83:0/503270828 shutdown_connections 2024-04-17T10:41:24.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.831+0000 7f1fb03fb640 1 --2- 172.21.15.83:0/503270828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fa806dae0 0x7f1fa80b2e10 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:24.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.831+0000 7f1fb03fb640 1 --2- 172.21.15.83:0/503270828 >> v2:172.21.15.167:3300/0 conn(0x7f1fa80aedd0 0x7f1fa806d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:24.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.831+0000 7f1fb03fb640 1 -- 172.21.15.83:0/503270828 >> 172.21.15.83:0/503270828 conn(0x7f1fa806abe0 msgr2=0x7f1fa806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:24.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.831+0000 7f1fb03fb640 1 -- 172.21.15.83:0/503270828 shutdown_connections 2024-04-17T10:41:24.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.831+0000 7f1fb03fb640 1 -- 172.21.15.83:0/503270828 wait complete. 2024-04-17T10:41:24.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.831+0000 7f1fb03fb640 1 Processor -- start 2024-04-17T10:41:24.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.831+0000 7f1fb03fb640 1 -- start start 2024-04-17T10:41:24.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.831+0000 7f1fb03fb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fa806dae0 0x7f1fa8150980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:24.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.831+0000 7f1fb03fb640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fa80aedd0 0x7f1fa8157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:24.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.831+0000 7f1fb03fb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fa815db50 0x7f1fa80cf420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:24.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.831+0000 7f1fb03fb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa80b3ef0 con 0x7f1fa815db50 2024-04-17T10:41:24.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.831+0000 7f1fb03fb640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa80b3d70 con 0x7f1fa806dae0 2024-04-17T10:41:24.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.831+0000 7f1fb03fb640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa80b4070 con 0x7f1fa80aedd0 2024-04-17T10:41:24.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.831+0000 7f1fae998640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fa815db50 0x7f1fa80cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:24.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.831+0000 7f1fae998640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fa815db50 0x7f1fa80cf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36650/0 (socket says 172.21.15.83:36650) 2024-04-17T10:41:24.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.831+0000 7f1fae998640 1 -- 172.21.15.83:0/734124278 learned_addr learned my addr 172.21.15.83:0/734124278 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:24.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.831+0000 7f1fae197640 1 --2- 172.21.15.83:0/734124278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fa806dae0 0x7f1fa8150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:24.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.831+0000 7f1fae998640 1 -- 172.21.15.83:0/734124278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fa80aedd0 msgr2=0x7f1fa8157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:24.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.831+0000 7f1fae998640 1 --2- 172.21.15.83:0/734124278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fa80aedd0 0x7f1fa8157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:24.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.831+0000 7f1fae998640 1 -- 172.21.15.83:0/734124278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fa806dae0 msgr2=0x7f1fa8150980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:24.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.831+0000 7f1fae998640 1 --2- 172.21.15.83:0/734124278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fa806dae0 0x7f1fa8150980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:24.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.831+0000 7f1fae998640 1 -- 172.21.15.83:0/734124278 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa8158530 con 0x7f1fa815db50 2024-04-17T10:41:24.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.831+0000 7f1fae197640 1 --2- 172.21.15.83:0/734124278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fa806dae0 0x7f1fa8150980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:24.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.831+0000 7f1fae998640 1 --2- 172.21.15.83:0/734124278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fa815db50 0x7f1fa80cf420 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00b0040 tx=0x7f1fa00196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:24.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.831+0000 7f1f9effd640 1 -- 172.21.15.83:0/734124278 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa00b2040 con 0x7f1fa815db50 2024-04-17T10:41:24.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.831+0000 7f1f9effd640 1 -- 172.21.15.83:0/734124278 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa00bf070 con 0x7f1fa815db50 2024-04-17T10:41:24.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.831+0000 7f1fb03fb640 1 -- 172.21.15.83:0/734124278 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa81587a0 con 0x7f1fa815db50 2024-04-17T10:41:24.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.831+0000 7f1f9effd640 1 -- 172.21.15.83:0/734124278 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa00b8890 con 0x7f1fa815db50 2024-04-17T10:41:24.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.831+0000 7f1fb03fb640 1 -- 172.21.15.83:0/734124278 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa8158d60 con 0x7f1fa815db50 2024-04-17T10:41:24.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.835+0000 7f1fb03fb640 1 -- 172.21.15.83:0/734124278 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa8074f90 con 0x7f1fa815db50 2024-04-17T10:41:24.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.835+0000 7f1f9effd640 1 -- 172.21.15.83:0/734124278 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1fa00a5670 con 0x7f1fa815db50 2024-04-17T10:41:24.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.835+0000 7f1f9effd640 1 --2- 172.21.15.83:0/734124278 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1f8c066f90 0x7f1f8c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:24.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.835+0000 7f1fae197640 1 --2- 172.21.15.83:0/734124278 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1f8c066f90 0x7f1f8c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:24.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.835+0000 7f1f9effd640 1 -- 172.21.15.83:0/734124278 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f1fa011fa90 con 0x7f1fa815db50 2024-04-17T10:41:24.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.835+0000 7f1fae197640 1 --2- 172.21.15.83:0/734124278 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1f8c066f90 0x7f1f8c069450 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4007fd0 tx=0x7f1fa4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:24.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:24.839+0000 7f1f9effd640 1 -- 172.21.15.83:0/734124278 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1fa00c8020 con 0x7f1fa815db50 2024-04-17T10:41:25.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.031+0000 7f1fb03fb640 1 -- 172.21.15.83:0/734124278 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1fa80aea60 con 0x7f1fa815db50 2024-04-17T10:41:25.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.031+0000 7f1f9effd640 1 -- 172.21.15.83:0/734124278 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f1fa0016520 con 0x7f1fa815db50 2024-04-17T10:41:25.031 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:41:25.031 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1456029744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6833","nonce":1456029744},{"type":"v1","addr":"172.21.15.167:6835","nonce":1456029744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6834/648555908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":648555908},{"type":"v1","addr":"172.21.15.167:6834","nonce":648555908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5163,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1435807773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1435807773},{"type":"v1","addr":"172.21.15.83:6835","nonce":1435807773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-17T10:39:13.472135+0000","modified":"2024-04-17T10:39:21.807600+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5172},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5172":{"gid":5172,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1181902025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1181902025},{"type":"v1","addr":"172.21.15.83:6837","nonce":1181902025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-17T10:41:25.032 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 14 2024-04-17T10:41:25.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.035+0000 7f1fb03fb640 1 -- 172.21.15.83:0/734124278 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1f8c066f90 msgr2=0x7f1f8c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:25.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.035+0000 7f1fb03fb640 1 --2- 172.21.15.83:0/734124278 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1f8c066f90 0x7f1f8c069450 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4007fd0 tx=0x7f1fa4009040 comp rx=0 tx=0).stop 2024-04-17T10:41:25.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.035+0000 7f1fb03fb640 1 -- 172.21.15.83:0/734124278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fa815db50 msgr2=0x7f1fa80cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:25.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.035+0000 7f1fb03fb640 1 --2- 172.21.15.83:0/734124278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fa815db50 0x7f1fa80cf420 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00b0040 tx=0x7f1fa00196d0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.035+0000 7f1fb03fb640 1 -- 172.21.15.83:0/734124278 shutdown_connections 2024-04-17T10:41:25.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.035+0000 7f1fb03fb640 1 --2- 172.21.15.83:0/734124278 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1f8c066f90 0x7f1f8c069450 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.035+0000 7f1fb03fb640 1 --2- 172.21.15.83:0/734124278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fa815db50 0x7f1fa80cf420 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.035+0000 7f1fb03fb640 1 --2- 172.21.15.83:0/734124278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fa80aedd0 0x7f1fa8157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.035+0000 7f1fb03fb640 1 --2- 172.21.15.83:0/734124278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fa806dae0 0x7f1fa8150980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.035+0000 7f1fb03fb640 1 -- 172.21.15.83:0/734124278 >> 172.21.15.83:0/734124278 conn(0x7f1fa806abe0 msgr2=0x7f1fa806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:25.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.035+0000 7f1fb03fb640 1 -- 172.21.15.83:0/734124278 shutdown_connections 2024-04-17T10:41:25.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.035+0000 7f1fb03fb640 1 -- 172.21.15.83:0/734124278 wait complete. 2024-04-17T10:41:25.050 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:41:25.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.167+0000 7f513505b640 1 Processor -- start 2024-04-17T10:41:25.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.167+0000 7f513505b640 1 -- start start 2024-04-17T10:41:25.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.167+0000 7f513505b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f513015dfa0 0x7f51300b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:25.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.167+0000 7f513505b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51300b5b40 0x7f51300d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:25.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.167+0000 7f513505b640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5130162a00 con 0x7f51300ce980 2024-04-17T10:41:25.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.167+0000 7f513505b640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5130161e80 con 0x7f513015dfa0 2024-04-17T10:41:25.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.167+0000 7f513505b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5130162160 con 0x7f51300b5b40 2024-04-17T10:41:25.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.167+0000 7f512f59e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51300b5b40 0x7f51300d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:25.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.167+0000 7f512ed9d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f513015dfa0 0x7f51300b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:25.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.167+0000 7f512f59e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51300b5b40 0x7f51300d1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:49936/0 (socket says 172.21.15.83:49936) 2024-04-17T10:41:25.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.167+0000 7f512e59c640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f51300ce980 0x7f51300b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:42654/0 (socket says 172.21.15.83:42654) 2024-04-17T10:41:25.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.167+0000 7f512f59e640 1 -- 172.21.15.83:0/4017748711 learned_addr learned my addr 172.21.15.83:0/4017748711 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:25.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.167+0000 7f512ed9d640 1 -- 172.21.15.83:0/4017748711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51300b5b40 msgr2=0x7f51300d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:25.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.167+0000 7f512ed9d640 1 --2- 172.21.15.83:0/4017748711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51300b5b40 0x7f51300d1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.167+0000 7f512ed9d640 1 -- 172.21.15.83:0/4017748711 >> v1:172.21.15.167:6789/0 conn(0x7f51300ce980 legacy=0x7f51300b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:25.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.167+0000 7f512ed9d640 1 -- 172.21.15.83:0/4017748711 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5130162440 con 0x7f513015dfa0 2024-04-17T10:41:25.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.171+0000 7f512ed9d640 1 --2- 172.21.15.83:0/4017748711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f513015dfa0 0x7f51300b6c20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f51280097c0 tx=0x7f512801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10ffaf5fc0aafda server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:25.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.171+0000 7f512dd9b640 1 -- 172.21.15.83:0/4017748711 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51280b0070 con 0x7f513015dfa0 2024-04-17T10:41:25.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.171+0000 7f512dd9b640 1 -- 172.21.15.83:0/4017748711 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51280ab600 con 0x7f513015dfa0 2024-04-17T10:41:25.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.171+0000 7f512dd9b640 1 -- 172.21.15.83:0/4017748711 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51280ab9d0 con 0x7f513015dfa0 2024-04-17T10:41:25.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.171+0000 7f513505b640 1 -- 172.21.15.83:0/4017748711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f513015dfa0 msgr2=0x7f51300b6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:25.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.171+0000 7f513505b640 1 --2- 172.21.15.83:0/4017748711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f513015dfa0 0x7f51300b6c20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f51280097c0 tx=0x7f512801c3e0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.171+0000 7f513505b640 1 -- 172.21.15.83:0/4017748711 shutdown_connections 2024-04-17T10:41:25.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.171+0000 7f513505b640 1 --2- 172.21.15.83:0/4017748711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51300b5b40 0x7f51300d1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.171+0000 7f513505b640 1 --2- 172.21.15.83:0/4017748711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f513015dfa0 0x7f51300b6c20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.171+0000 7f513505b640 1 -- 172.21.15.83:0/4017748711 >> 172.21.15.83:0/4017748711 conn(0x7f51300d0600 msgr2=0x7f51300d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:25.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.171+0000 7f513505b640 1 -- 172.21.15.83:0/4017748711 shutdown_connections 2024-04-17T10:41:25.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.171+0000 7f513505b640 1 -- 172.21.15.83:0/4017748711 wait complete. 2024-04-17T10:41:25.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.175+0000 7f513505b640 1 Processor -- start 2024-04-17T10:41:25.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.175+0000 7f513505b640 1 -- start start 2024-04-17T10:41:25.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.175+0000 7f513505b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51300b5b40 0x7f51300c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:25.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.175+0000 7f513505b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51300ce980 0x7f51300c7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:25.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.175+0000 7f513505b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f513015dfa0 0x7f5130148c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:25.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.175+0000 7f513505b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513016ae50 con 0x7f51300ce980 2024-04-17T10:41:25.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.175+0000 7f513505b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5130161b10 con 0x7f513015dfa0 2024-04-17T10:41:25.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.175+0000 7f513505b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513016afd0 con 0x7f51300b5b40 2024-04-17T10:41:25.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.175+0000 7f512e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51300ce980 0x7f51300c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:25.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.175+0000 7f512e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51300ce980 0x7f51300c7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36658/0 (socket says 172.21.15.83:36658) 2024-04-17T10:41:25.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.175+0000 7f512e59c640 1 -- 172.21.15.83:0/539905643 learned_addr learned my addr 172.21.15.83:0/539905643 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:25.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.175+0000 7f512f59e640 1 --2- 172.21.15.83:0/539905643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f513015dfa0 0x7f5130148c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:25.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.175+0000 7f512ed9d640 1 --2- 172.21.15.83:0/539905643 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51300b5b40 0x7f51300c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:25.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.175+0000 7f512e59c640 1 -- 172.21.15.83:0/539905643 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51300b5b40 msgr2=0x7f51300c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:25.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.175+0000 7f512e59c640 1 --2- 172.21.15.83:0/539905643 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51300b5b40 0x7f51300c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.175+0000 7f512e59c640 1 -- 172.21.15.83:0/539905643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f513015dfa0 msgr2=0x7f5130148c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:25.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.175+0000 7f512e59c640 1 --2- 172.21.15.83:0/539905643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f513015dfa0 0x7f5130148c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.175+0000 7f512e59c640 1 -- 172.21.15.83:0/539905643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51300c5dc0 con 0x7f51300ce980 2024-04-17T10:41:25.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.175+0000 7f512e59c640 1 --2- 172.21.15.83:0/539905643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51300ce980 0x7f51300c7720 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f512400c3a0 tx=0x7f512400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:25.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.175+0000 7f51177fe640 1 -- 172.21.15.83:0/539905643 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51240074e0 con 0x7f51300ce980 2024-04-17T10:41:25.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.175+0000 7f51177fe640 1 -- 172.21.15.83:0/539905643 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5124007680 con 0x7f51300ce980 2024-04-17T10:41:25.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.175+0000 7f513505b640 1 -- 172.21.15.83:0/539905643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5130153cd0 con 0x7f51300ce980 2024-04-17T10:41:25.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.175+0000 7f51177fe640 1 -- 172.21.15.83:0/539905643 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5124004a30 con 0x7f51300ce980 2024-04-17T10:41:25.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.175+0000 7f513505b640 1 -- 172.21.15.83:0/539905643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5130154290 con 0x7f51300ce980 2024-04-17T10:41:25.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.175+0000 7f513505b640 1 -- 172.21.15.83:0/539905643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5130074f90 con 0x7f51300ce980 2024-04-17T10:41:25.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.179+0000 7f51177fe640 1 -- 172.21.15.83:0/539905643 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f512401a020 con 0x7f51300ce980 2024-04-17T10:41:25.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.179+0000 7f51177fe640 1 --2- 172.21.15.83:0/539905643 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f511c066f60 0x7f511c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:25.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.179+0000 7f512ed9d640 1 --2- 172.21.15.83:0/539905643 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f511c066f60 0x7f511c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:25.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.179+0000 7f51177fe640 1 -- 172.21.15.83:0/539905643 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f5124084800 con 0x7f51300ce980 2024-04-17T10:41:25.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.179+0000 7f512ed9d640 1 --2- 172.21.15.83:0/539905643 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f511c066f60 0x7f511c069420 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f51280ad000 tx=0x7f51280a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:25.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.183+0000 7f51177fe640 1 -- 172.21.15.83:0/539905643 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f51300c5dc0 con 0x7f51300ce980 2024-04-17T10:41:25.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.375+0000 7f513505b640 1 -- 172.21.15.83:0/539905643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5130160b30 con 0x7f51300ce980 2024-04-17T10:41:25.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.375+0000 7f51177fe640 1 -- 172.21.15.83:0/539905643 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f51240557e0 con 0x7f51300ce980 2024-04-17T10:41:25.372 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:41:25.372 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1456029744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6833","nonce":1456029744},{"type":"v1","addr":"172.21.15.167:6835","nonce":1456029744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6834/648555908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":648555908},{"type":"v1","addr":"172.21.15.167:6834","nonce":648555908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5163,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1435807773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1435807773},{"type":"v1","addr":"172.21.15.83:6835","nonce":1435807773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-17T10:39:13.472135+0000","modified":"2024-04-17T10:39:21.807600+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5172},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5172":{"gid":5172,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1181902025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1181902025},{"type":"v1","addr":"172.21.15.83:6837","nonce":1181902025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-17T10:41:25.372 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 14 2024-04-17T10:41:25.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.375+0000 7f513505b640 1 -- 172.21.15.83:0/539905643 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f511c066f60 msgr2=0x7f511c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:25.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.375+0000 7f513505b640 1 --2- 172.21.15.83:0/539905643 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f511c066f60 0x7f511c069420 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f51280ad000 tx=0x7f51280a6000 comp rx=0 tx=0).stop 2024-04-17T10:41:25.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.375+0000 7f513505b640 1 -- 172.21.15.83:0/539905643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51300ce980 msgr2=0x7f51300c7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:25.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.375+0000 7f513505b640 1 --2- 172.21.15.83:0/539905643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51300ce980 0x7f51300c7720 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f512400c3a0 tx=0x7f512400c860 comp rx=0 tx=0).stop 2024-04-17T10:41:25.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.375+0000 7f513505b640 1 -- 172.21.15.83:0/539905643 shutdown_connections 2024-04-17T10:41:25.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.375+0000 7f513505b640 1 --2- 172.21.15.83:0/539905643 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f511c066f60 0x7f511c069420 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.375+0000 7f513505b640 1 --2- 172.21.15.83:0/539905643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f513015dfa0 0x7f5130148c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.375+0000 7f513505b640 1 --2- 172.21.15.83:0/539905643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51300ce980 0x7f51300c7720 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.375+0000 7f513505b640 1 --2- 172.21.15.83:0/539905643 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51300b5b40 0x7f51300c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.375+0000 7f513505b640 1 -- 172.21.15.83:0/539905643 >> 172.21.15.83:0/539905643 conn(0x7f51300d0600 msgr2=0x7f513015e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:25.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.375+0000 7f513505b640 1 -- 172.21.15.83:0/539905643 shutdown_connections 2024-04-17T10:41:25.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.375+0000 7f513505b640 1 -- 172.21.15.83:0/539905643 wait complete. 2024-04-17T10:41:25.391 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:41:25.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.515+0000 7f4e82054640 1 Processor -- start 2024-04-17T10:41:25.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.515+0000 7f4e82054640 1 -- start start 2024-04-17T10:41:25.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.515+0000 7f4e82054640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4e7c0cd640 0x7f4e7c0ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:25.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.515+0000 7f4e82054640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e7c0cda20 0x7f4e7c06c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:25.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.515+0000 7f4e82054640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4e7c0d0b70 0x7f4e7c06cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:25.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.515+0000 7f4e82054640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e7c0ceff0 con 0x7f4e7c0d0b70 2024-04-17T10:41:25.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.515+0000 7f4e82054640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e7c0cfdb0 con 0x7f4e7c0cd640 2024-04-17T10:41:25.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.515+0000 7f4e82054640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7c0d0090 con 0x7f4e7c0cda20 2024-04-17T10:41:25.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.515+0000 7f4e7bfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4e7c0d0b70 0x7f4e7c06cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:25.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.515+0000 7f4e7bfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4e7c0d0b70 0x7f4e7c06cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36666/0 (socket says 172.21.15.83:36666) 2024-04-17T10:41:25.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.515+0000 7f4e7bfff640 1 -- 172.21.15.83:0/1876858865 learned_addr learned my addr 172.21.15.83:0/1876858865 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:25.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.515+0000 7f4e7b7fe640 1 --2- 172.21.15.83:0/1876858865 >> v2:172.21.15.167:3300/0 conn(0x7f4e7c0cd640 0x7f4e7c0ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:25.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.515+0000 7f4e73fff640 1 --2- 172.21.15.83:0/1876858865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e7c0cda20 0x7f4e7c06c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:25.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.515+0000 7f4e7bfff640 1 -- 172.21.15.83:0/1876858865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e7c0cda20 msgr2=0x7f4e7c06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:25.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.515+0000 7f4e7bfff640 1 --2- 172.21.15.83:0/1876858865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e7c0cda20 0x7f4e7c06c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.515+0000 7f4e7bfff640 1 -- 172.21.15.83:0/1876858865 >> v2:172.21.15.167:3300/0 conn(0x7f4e7c0cd640 msgr2=0x7f4e7c0ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:25.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.515+0000 7f4e7bfff640 1 --2- 172.21.15.83:0/1876858865 >> v2:172.21.15.167:3300/0 conn(0x7f4e7c0cd640 0x7f4e7c0ceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.515+0000 7f4e7bfff640 1 -- 172.21.15.83:0/1876858865 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e7c0d0370 con 0x7f4e7c0d0b70 2024-04-17T10:41:25.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.519+0000 7f4e7bfff640 1 --2- 172.21.15.83:0/1876858865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e7c0d0b70 0x7f4e7c06cc40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c06b830 tx=0x7f4e7401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a116985720a3aab server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:25.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.519+0000 7f4e7affd640 1 -- 172.21.15.83:0/1876858865 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e740a5780 con 0x7f4e7c0d0b70 2024-04-17T10:41:25.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.519+0000 7f4e7affd640 1 -- 172.21.15.83:0/1876858865 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e740a5920 con 0x7f4e7c0d0b70 2024-04-17T10:41:25.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.519+0000 7f4e7affd640 1 -- 172.21.15.83:0/1876858865 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e740a5cf0 con 0x7f4e7c0d0b70 2024-04-17T10:41:25.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.519+0000 7f4e82054640 1 -- 172.21.15.83:0/1876858865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e7c0d0b70 msgr2=0x7f4e7c06cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:25.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.519+0000 7f4e82054640 1 --2- 172.21.15.83:0/1876858865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e7c0d0b70 0x7f4e7c06cc40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c06b830 tx=0x7f4e7401c9f0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.519+0000 7f4e82054640 1 -- 172.21.15.83:0/1876858865 shutdown_connections 2024-04-17T10:41:25.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.519+0000 7f4e82054640 1 --2- 172.21.15.83:0/1876858865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e7c0d0b70 0x7f4e7c06cc40 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.519+0000 7f4e82054640 1 --2- 172.21.15.83:0/1876858865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e7c0cda20 0x7f4e7c06c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.519+0000 7f4e82054640 1 --2- 172.21.15.83:0/1876858865 >> v2:172.21.15.167:3300/0 conn(0x7f4e7c0cd640 0x7f4e7c0ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.519+0000 7f4e82054640 1 -- 172.21.15.83:0/1876858865 >> 172.21.15.83:0/1876858865 conn(0x7f4e7c0d1820 msgr2=0x7f4e7c0d1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:25.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.519+0000 7f4e82054640 1 -- 172.21.15.83:0/1876858865 shutdown_connections 2024-04-17T10:41:25.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.519+0000 7f4e82054640 1 -- 172.21.15.83:0/1876858865 wait complete. 2024-04-17T10:41:25.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.519+0000 7f4e82054640 1 Processor -- start 2024-04-17T10:41:25.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.519+0000 7f4e82054640 1 -- start start 2024-04-17T10:41:25.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.519+0000 7f4e82054640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e7c0cd640 0x7f4e7c151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:25.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.519+0000 7f4e82054640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e7c0cda20 0x7f4e7c151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:25.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.519+0000 7f4e82054640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e7c0d0b70 0x7f4e7c157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:25.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.519+0000 7f4e82054640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7c15e030 con 0x7f4e7c0cda20 2024-04-17T10:41:25.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.519+0000 7f4e82054640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7c15deb0 con 0x7f4e7c0d0b70 2024-04-17T10:41:25.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.519+0000 7f4e82054640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7c15e1b0 con 0x7f4e7c0cd640 2024-04-17T10:41:25.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.519+0000 7f4e73fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e7c0cda20 0x7f4e7c151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:25.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.519+0000 7f4e73fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e7c0cda20 0x7f4e7c151ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36680/0 (socket says 172.21.15.83:36680) 2024-04-17T10:41:25.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.519+0000 7f4e73fff640 1 -- 172.21.15.83:0/147853268 learned_addr learned my addr 172.21.15.83:0/147853268 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:25.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.519+0000 7f4e7b7fe640 1 --2- 172.21.15.83:0/147853268 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e7c0cd640 0x7f4e7c151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:25.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.519+0000 7f4e7bfff640 1 --2- 172.21.15.83:0/147853268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e7c0d0b70 0x7f4e7c157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:25.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.519+0000 7f4e73fff640 1 -- 172.21.15.83:0/147853268 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e7c0cd640 msgr2=0x7f4e7c151960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:25.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.519+0000 7f4e73fff640 1 --2- 172.21.15.83:0/147853268 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e7c0cd640 0x7f4e7c151960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.519+0000 7f4e73fff640 1 -- 172.21.15.83:0/147853268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e7c0d0b70 msgr2=0x7f4e7c157e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:25.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.519+0000 7f4e73fff640 1 --2- 172.21.15.83:0/147853268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e7c0d0b70 0x7f4e7c157e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.519+0000 7f4e73fff640 1 -- 172.21.15.83:0/147853268 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e7c1591c0 con 0x7f4e7c0cda20 2024-04-17T10:41:25.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.519+0000 7f4e7b7fe640 1 --2- 172.21.15.83:0/147853268 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e7c0cd640 0x7f4e7c151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:25.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.523+0000 7f4e73fff640 1 --2- 172.21.15.83:0/147853268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e7c0cda20 0x7f4e7c151ea0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400c7f0 tx=0x7f4e6400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:25.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.523+0000 7f4e78ff9640 1 -- 172.21.15.83:0/147853268 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64004230 con 0x7f4e7c0cda20 2024-04-17T10:41:25.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.523+0000 7f4e78ff9640 1 -- 172.21.15.83:0/147853268 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e640043d0 con 0x7f4e7c0cda20 2024-04-17T10:41:25.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.523+0000 7f4e82054640 1 -- 172.21.15.83:0/147853268 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e7c158ee0 con 0x7f4e7c0cda20 2024-04-17T10:41:25.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.523+0000 7f4e78ff9640 1 -- 172.21.15.83:0/147853268 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64003860 con 0x7f4e7c0cda20 2024-04-17T10:41:25.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.523+0000 7f4e82054640 1 -- 172.21.15.83:0/147853268 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e7c0b4cb0 con 0x7f4e7c0cda20 2024-04-17T10:41:25.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.523+0000 7f4e82054640 1 -- 172.21.15.83:0/147853268 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e7c0b4800 con 0x7f4e7c0cda20 2024-04-17T10:41:25.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.523+0000 7f4e78ff9640 1 -- 172.21.15.83:0/147853268 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4e64003c10 con 0x7f4e7c0cda20 2024-04-17T10:41:25.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.523+0000 7f4e78ff9640 1 --2- 172.21.15.83:0/147853268 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4e5c066f90 0x7f4e5c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:25.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.523+0000 7f4e7b7fe640 1 --2- 172.21.15.83:0/147853268 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4e5c066f90 0x7f4e5c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:25.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.523+0000 7f4e78ff9640 1 -- 172.21.15.83:0/147853268 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f4e64085130 con 0x7f4e7c0cda20 2024-04-17T10:41:25.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.523+0000 7f4e7b7fe640 1 --2- 172.21.15.83:0/147853268 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4e5c066f90 0x7f4e5c069450 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0c6ef0 tx=0x7f4e6c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:25.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.527+0000 7f4e78ff9640 1 -- 172.21.15.83:0/147853268 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4e64015020 con 0x7f4e7c0cda20 2024-04-17T10:41:25.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.679+0000 7f4e82054640 1 -- 172.21.15.83:0/147853268 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e7c074f90 con 0x7f4e7c0cda20 2024-04-17T10:41:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.679+0000 7f4e78ff9640 1 -- 172.21.15.83:0/147853268 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16662 (secure 0 0 0) 0x7f4e64056120 con 0x7f4e7c0cda20 2024-04-17T10:41:25.679 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:41:25.679 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":35,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:40:57.038108+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:39:11.330695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-17T10:39:12.438377+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":34,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:41:25.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.683+0000 7f4e82054640 1 -- 172.21.15.83:0/147853268 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4e5c066f90 msgr2=0x7f4e5c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:25.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.683+0000 7f4e82054640 1 --2- 172.21.15.83:0/147853268 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4e5c066f90 0x7f4e5c069450 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0c6ef0 tx=0x7f4e6c008040 comp rx=0 tx=0).stop 2024-04-17T10:41:25.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.683+0000 7f4e82054640 1 -- 172.21.15.83:0/147853268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e7c0cda20 msgr2=0x7f4e7c151ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:25.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.683+0000 7f4e82054640 1 --2- 172.21.15.83:0/147853268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e7c0cda20 0x7f4e7c151ea0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400c7f0 tx=0x7f4e6400ccb0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.683+0000 7f4e82054640 1 -- 172.21.15.83:0/147853268 shutdown_connections 2024-04-17T10:41:25.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.683+0000 7f4e82054640 1 --2- 172.21.15.83:0/147853268 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4e5c066f90 0x7f4e5c069450 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.683+0000 7f4e82054640 1 --2- 172.21.15.83:0/147853268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e7c0d0b70 0x7f4e7c157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.683+0000 7f4e82054640 1 --2- 172.21.15.83:0/147853268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e7c0cda20 0x7f4e7c151ea0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.683+0000 7f4e82054640 1 --2- 172.21.15.83:0/147853268 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e7c0cd640 0x7f4e7c151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.683+0000 7f4e82054640 1 -- 172.21.15.83:0/147853268 >> 172.21.15.83:0/147853268 conn(0x7f4e7c0d1820 msgr2=0x7f4e7c15ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:25.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.683+0000 7f4e82054640 1 -- 172.21.15.83:0/147853268 shutdown_connections 2024-04-17T10:41:25.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.683+0000 7f4e82054640 1 -- 172.21.15.83:0/147853268 wait complete. 2024-04-17T10:41:25.697 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-17T10:41:25.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.819+0000 7f92248aa640 1 Processor -- start 2024-04-17T10:41:25.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.819+0000 7f92248aa640 1 -- start start 2024-04-17T10:41:25.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.819+0000 7f92248aa640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9220154de0 0x7f922006ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:25.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.819+0000 7f92248aa640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f922006b350 0x7f92200b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:25.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.819+0000 7f92248aa640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f92200b4040 0x7f9220071580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:25.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.819+0000 7f92248aa640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92200b4870 con 0x7f922006b350 2024-04-17T10:41:25.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.819+0000 7f92248aa640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f922015bd70 con 0x7f92200b4040 2024-04-17T10:41:25.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.819+0000 7f92248aa640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f922015bf60 con 0x7f9220154de0 2024-04-17T10:41:25.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.819+0000 7f921dd9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f922006b350 0x7f92200b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:25.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.819+0000 7f921dd9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f922006b350 0x7f92200b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36686/0 (socket says 172.21.15.83:36686) 2024-04-17T10:41:25.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.819+0000 7f921dd9b640 1 -- 172.21.15.83:0/1886372294 learned_addr learned my addr 172.21.15.83:0/1886372294 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:25.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.819+0000 7f921ed9d640 1 --2- 172.21.15.83:0/1886372294 >> v2:172.21.15.167:3300/0 conn(0x7f92200b4040 0x7f9220071580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:25.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.819+0000 7f921e59c640 1 --2- 172.21.15.83:0/1886372294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9220154de0 0x7f922006ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:25.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.819+0000 7f921dd9b640 1 -- 172.21.15.83:0/1886372294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9220154de0 msgr2=0x7f922006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:25.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.819+0000 7f921dd9b640 1 --2- 172.21.15.83:0/1886372294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9220154de0 0x7f922006ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.819+0000 7f921dd9b640 1 -- 172.21.15.83:0/1886372294 >> v2:172.21.15.167:3300/0 conn(0x7f92200b4040 msgr2=0x7f9220071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:25.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.819+0000 7f921dd9b640 1 --2- 172.21.15.83:0/1886372294 >> v2:172.21.15.167:3300/0 conn(0x7f92200b4040 0x7f9220071580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.819+0000 7f921dd9b640 1 -- 172.21.15.83:0/1886372294 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f922015afb0 con 0x7f922006b350 2024-04-17T10:41:25.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.819+0000 7f921ed9d640 1 --2- 172.21.15.83:0/1886372294 >> v2:172.21.15.167:3300/0 conn(0x7f92200b4040 0x7f9220071580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:41:25.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.823+0000 7f921dd9b640 1 --2- 172.21.15.83:0/1886372294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f922006b350 0x7f92200b3b00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f922006e900 tx=0x7f921401c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93bc02b269bc822e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.823+0000 7f921d59a640 1 -- 172.21.15.83:0/1886372294 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92140b0070 con 0x7f922006b350 2024-04-17T10:41:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.823+0000 7f921d59a640 1 -- 172.21.15.83:0/1886372294 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92140ab600 con 0x7f922006b350 2024-04-17T10:41:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.823+0000 7f921d59a640 1 -- 172.21.15.83:0/1886372294 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92140ab9d0 con 0x7f922006b350 2024-04-17T10:41:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.823+0000 7f92248aa640 1 -- 172.21.15.83:0/1886372294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f922006b350 msgr2=0x7f92200b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.823+0000 7f92248aa640 1 --2- 172.21.15.83:0/1886372294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f922006b350 0x7f92200b3b00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f922006e900 tx=0x7f921401c260 comp rx=0 tx=0).stop 2024-04-17T10:41:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.823+0000 7f92248aa640 1 -- 172.21.15.83:0/1886372294 shutdown_connections 2024-04-17T10:41:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.823+0000 7f92248aa640 1 --2- 172.21.15.83:0/1886372294 >> v2:172.21.15.167:3300/0 conn(0x7f92200b4040 0x7f9220071580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.823+0000 7f92248aa640 1 --2- 172.21.15.83:0/1886372294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f922006b350 0x7f92200b3b00 secure :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f922006e900 tx=0x7f921401c260 comp rx=0 tx=0).stop 2024-04-17T10:41:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.823+0000 7f92248aa640 1 --2- 172.21.15.83:0/1886372294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9220154de0 0x7f922006ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.823+0000 7f92248aa640 1 -- 172.21.15.83:0/1886372294 >> 172.21.15.83:0/1886372294 conn(0x7f92200b15b0 msgr2=0x7f92200b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.823+0000 7f92248aa640 1 -- 172.21.15.83:0/1886372294 shutdown_connections 2024-04-17T10:41:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.823+0000 7f92248aa640 1 -- 172.21.15.83:0/1886372294 wait complete. 2024-04-17T10:41:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.823+0000 7f92248aa640 1 Processor -- start 2024-04-17T10:41:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.823+0000 7f92248aa640 1 -- start start 2024-04-17T10:41:25.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.827+0000 7f92248aa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92200b4040 0x7f92201553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:25.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.827+0000 7f92248aa640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9220154de0 0x7f9220157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:25.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.827+0000 7f92248aa640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9220157b40 0x7f9220070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:25.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.827+0000 7f92248aa640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92200b4450 con 0x7f92200b4040 2024-04-17T10:41:25.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.827+0000 7f92248aa640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92201551c0 con 0x7f9220157b40 2024-04-17T10:41:25.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.827+0000 7f92248aa640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92200b45d0 con 0x7f9220154de0 2024-04-17T10:41:25.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.827+0000 7f921e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92200b4040 0x7f92201553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:25.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.827+0000 7f921e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92200b4040 0x7f92201553d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36696/0 (socket says 172.21.15.83:36696) 2024-04-17T10:41:25.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.827+0000 7f921e59c640 1 -- 172.21.15.83:0/3735981183 learned_addr learned my addr 172.21.15.83:0/3735981183 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:25.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.827+0000 7f921ed9d640 1 --2- 172.21.15.83:0/3735981183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9220157b40 0x7f9220070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:25.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.827+0000 7f921dd9b640 1 --2- 172.21.15.83:0/3735981183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9220154de0 0x7f9220157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:25.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.827+0000 7f921e59c640 1 -- 172.21.15.83:0/3735981183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9220154de0 msgr2=0x7f9220157600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:25.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.827+0000 7f921e59c640 1 --2- 172.21.15.83:0/3735981183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9220154de0 0x7f9220157600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.827+0000 7f921e59c640 1 -- 172.21.15.83:0/3735981183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9220157b40 msgr2=0x7f9220070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:25.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.827+0000 7f921e59c640 1 --2- 172.21.15.83:0/3735981183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9220157b40 0x7f9220070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:25.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.827+0000 7f921e59c640 1 -- 172.21.15.83:0/3735981183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9220158040 con 0x7f92200b4040 2024-04-17T10:41:25.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.827+0000 7f921dd9b640 1 --2- 172.21.15.83:0/3735981183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9220154de0 0x7f9220157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:25.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.827+0000 7f921e59c640 1 --2- 172.21.15.83:0/3735981183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92200b4040 0x7f92201553d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f921000d830 tx=0x7f921000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:25.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.827+0000 7f920effd640 1 -- 172.21.15.83:0/3735981183 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921000b6c0 con 0x7f92200b4040 2024-04-17T10:41:25.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.827+0000 7f920effd640 1 -- 172.21.15.83:0/3735981183 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9210019070 con 0x7f92200b4040 2024-04-17T10:41:25.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.827+0000 7f92248aa640 1 -- 172.21.15.83:0/3735981183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9220149880 con 0x7f92200b4040 2024-04-17T10:41:25.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.827+0000 7f920effd640 1 -- 172.21.15.83:0/3735981183 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921000bae0 con 0x7f92200b4040 2024-04-17T10:41:25.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.827+0000 7f92248aa640 1 -- 172.21.15.83:0/3735981183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9220149e40 con 0x7f92200b4040 2024-04-17T10:41:25.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.827+0000 7f92248aa640 1 -- 172.21.15.83:0/3735981183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9220074f90 con 0x7f92200b4040 2024-04-17T10:41:25.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.827+0000 7f920effd640 1 -- 172.21.15.83:0/3735981183 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f921001e460 con 0x7f92200b4040 2024-04-17T10:41:25.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.827+0000 7f920effd640 1 --2- 172.21.15.83:0/3735981183 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9200066f90 0x7f9200069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:25.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.827+0000 7f920effd640 1 -- 172.21.15.83:0/3735981183 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f9210085640 con 0x7f92200b4040 2024-04-17T10:41:25.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.827+0000 7f921dd9b640 1 --2- 172.21.15.83:0/3735981183 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9200066f90 0x7f9200069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:25.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.831+0000 7f921dd9b640 1 --2- 172.21.15.83:0/3735981183 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9200066f90 0x7f9200069450 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f92140ad000 tx=0x7f9214008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:25.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:25.835+0000 7f920effd640 1 -- 172.21.15.83:0/3735981183 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f92100566c0 con 0x7f92200b4040 2024-04-17T10:41:26.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.023+0000 7f92248aa640 1 -- 172.21.15.83:0/3735981183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f92200b3280 con 0x7f92200b4040 2024-04-17T10:41:26.022 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-04-17T10:41:26.023+0000 7f35c3b81640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T10:41:26.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.027+0000 7f920effd640 1 -- 172.21.15.83:0/3735981183 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v15) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9210055fc0 con 0x7f92200b4040 2024-04-17T10:41:26.025 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-17T10:41:26.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.031+0000 7f92248aa640 1 -- 172.21.15.83:0/3735981183 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9200066f90 msgr2=0x7f9200069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:26.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.031+0000 7f92248aa640 1 --2- 172.21.15.83:0/3735981183 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9200066f90 0x7f9200069450 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f92140ad000 tx=0x7f9214008040 comp rx=0 tx=0).stop 2024-04-17T10:41:26.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.031+0000 7f92248aa640 1 -- 172.21.15.83:0/3735981183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92200b4040 msgr2=0x7f92201553d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:26.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.031+0000 7f92248aa640 1 --2- 172.21.15.83:0/3735981183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92200b4040 0x7f92201553d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f921000d830 tx=0x7f921000dcf0 comp rx=0 tx=0).stop 2024-04-17T10:41:26.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.031+0000 7f92248aa640 1 -- 172.21.15.83:0/3735981183 shutdown_connections 2024-04-17T10:41:26.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.031+0000 7f92248aa640 1 --2- 172.21.15.83:0/3735981183 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9200066f90 0x7f9200069450 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:26.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.031+0000 7f92248aa640 1 --2- 172.21.15.83:0/3735981183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9220157b40 0x7f9220070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:26.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.031+0000 7f92248aa640 1 --2- 172.21.15.83:0/3735981183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9220154de0 0x7f9220157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:26.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.031+0000 7f92248aa640 1 --2- 172.21.15.83:0/3735981183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92200b4040 0x7f92201553d0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:26.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.031+0000 7f92248aa640 1 -- 172.21.15.83:0/3735981183 >> 172.21.15.83:0/3735981183 conn(0x7f92200b15b0 msgr2=0x7f922006d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:26.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.031+0000 7f92248aa640 1 -- 172.21.15.83:0/3735981183 shutdown_connections 2024-04-17T10:41:26.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.031+0000 7f92248aa640 1 -- 172.21.15.83:0/3735981183 wait complete. 2024-04-17T10:41:26.053 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-17T10:41:26.082 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-04-17T10:41:26.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.171+0000 7f2f08bc6640 1 Processor -- start 2024-04-17T10:41:26.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.171+0000 7f2f08bc6640 1 -- start start 2024-04-17T10:41:26.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.175+0000 7f2f08bc6640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2f0415bac0 0x7f2f04071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:26.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.175+0000 7f2f08bc6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f04070470 0x7f2f0406ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:26.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.175+0000 7f2f08bc6640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f0406c1f0 con 0x7f2f0415b6e0 2024-04-17T10:41:26.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.175+0000 7f2f08bc6640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f04070a10 con 0x7f2f0415bac0 2024-04-17T10:41:26.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.175+0000 7f2f08bc6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f04070cf0 con 0x7f2f04070470 2024-04-17T10:41:26.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.175+0000 7f2f01d9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2f0415bac0 0x7f2f04071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:26.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.175+0000 7f2f0259c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2f0415b6e0 0x7f2f040cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54086/0 (socket says 172.21.15.83:54086) 2024-04-17T10:41:26.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.175+0000 7f2f01d9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2f0415bac0 0x7f2f04071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36724/0 (socket says 172.21.15.83:36724) 2024-04-17T10:41:26.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.175+0000 7f2f0259c640 1 -- 172.21.15.83:0/176130312 learned_addr learned my addr 172.21.15.83:0/176130312 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:26.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.175+0000 7f2f01d9b640 1 -- 172.21.15.83:0/176130312 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f04070470 msgr2=0x7f2f0406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:26.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.175+0000 7f2f01d9b640 1 --2- 172.21.15.83:0/176130312 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f04070470 0x7f2f0406ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:26.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.175+0000 7f2f01d9b640 1 -- 172.21.15.83:0/176130312 >> v1:172.21.15.83:6789/0 conn(0x7f2f0415b6e0 legacy=0x7f2f040cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:26.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.175+0000 7f2f01d9b640 1 -- 172.21.15.83:0/176130312 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f04070fd0 con 0x7f2f0415bac0 2024-04-17T10:41:26.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.175+0000 7f2f0159a640 1 -- 172.21.15.83:0/176130312 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3437965081 0 0) 0x7f2f0406c1f0 con 0x7f2f0415b6e0 2024-04-17T10:41:26.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.175+0000 7f2f01d9b640 1 --2- 172.21.15.83:0/176130312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f0415bac0 0x7f2f04071cc0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80097c0 tx=0x7f2ef801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d851233a8c7e8cc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:26.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.175+0000 7f2f0159a640 1 -- 172.21.15.83:0/176130312 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef80b0070 con 0x7f2f0415bac0 2024-04-17T10:41:26.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.175+0000 7f2f0159a640 1 -- 172.21.15.83:0/176130312 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef80ab600 con 0x7f2f0415bac0 2024-04-17T10:41:26.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.175+0000 7f2f0159a640 1 -- 172.21.15.83:0/176130312 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef80ab9d0 con 0x7f2f0415bac0 2024-04-17T10:41:26.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.175+0000 7f2f08bc6640 1 -- 172.21.15.83:0/176130312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f0415bac0 msgr2=0x7f2f04071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:26.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.175+0000 7f2f08bc6640 1 --2- 172.21.15.83:0/176130312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f0415bac0 0x7f2f04071cc0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80097c0 tx=0x7f2ef801c410 comp rx=0 tx=0).stop 2024-04-17T10:41:26.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.175+0000 7f2f08bc6640 1 -- 172.21.15.83:0/176130312 shutdown_connections 2024-04-17T10:41:26.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.175+0000 7f2f08bc6640 1 --2- 172.21.15.83:0/176130312 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f04070470 0x7f2f0406ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:26.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.175+0000 7f2f08bc6640 1 --2- 172.21.15.83:0/176130312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f0415bac0 0x7f2f04071cc0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:26.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.175+0000 7f2f08bc6640 1 -- 172.21.15.83:0/176130312 >> 172.21.15.83:0/176130312 conn(0x7f2f040b15b0 msgr2=0x7f2f040b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:26.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.175+0000 7f2f08bc6640 1 -- 172.21.15.83:0/176130312 shutdown_connections 2024-04-17T10:41:26.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.175+0000 7f2f08bc6640 1 -- 172.21.15.83:0/176130312 wait complete. 2024-04-17T10:41:26.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.179+0000 7f2f08bc6640 1 Processor -- start 2024-04-17T10:41:26.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.179+0000 7f2f08bc6640 1 -- start start 2024-04-17T10:41:26.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.179+0000 7f2f08bc6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f04070470 0x7f2f04155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:26.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.179+0000 7f2f08bc6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f0415b6e0 0x7f2f04155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:26.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.179+0000 7f2f08bc6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f0415bac0 0x7f2f041564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:26.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.179+0000 7f2f08bc6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0406d2f0 con 0x7f2f0415b6e0 2024-04-17T10:41:26.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.179+0000 7f2f08bc6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0406d170 con 0x7f2f04070470 2024-04-17T10:41:26.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.179+0000 7f2f08bc6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0406b760 con 0x7f2f0415bac0 2024-04-17T10:41:26.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.179+0000 7f2f01d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f0415b6e0 0x7f2f04155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:26.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.179+0000 7f2f01d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f0415b6e0 0x7f2f04155fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36728/0 (socket says 172.21.15.83:36728) 2024-04-17T10:41:26.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.179+0000 7f2f01d9b640 1 -- 172.21.15.83:0/1269202590 learned_addr learned my addr 172.21.15.83:0/1269202590 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:26.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.179+0000 7f2f01d9b640 1 -- 172.21.15.83:0/1269202590 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f0415bac0 msgr2=0x7f2f041564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:26.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.179+0000 7f2f01d9b640 1 --2- 172.21.15.83:0/1269202590 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f0415bac0 0x7f2f041564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:26.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.179+0000 7f2f01d9b640 1 -- 172.21.15.83:0/1269202590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f04070470 msgr2=0x7f2f04155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:26.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.179+0000 7f2f01d9b640 1 --2- 172.21.15.83:0/1269202590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f04070470 0x7f2f04155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:26.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.179+0000 7f2f01d9b640 1 -- 172.21.15.83:0/1269202590 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f041576a0 con 0x7f2f0415b6e0 2024-04-17T10:41:26.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.179+0000 7f2f01d9b640 1 --2- 172.21.15.83:0/1269202590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f0415b6e0 0x7f2f04155fa0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2ef801a040 tx=0x7f2ef80191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:26.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.179+0000 7f2ef2ffd640 1 -- 172.21.15.83:0/1269202590 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef80bb070 con 0x7f2f0415b6e0 2024-04-17T10:41:26.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.179+0000 7f2ef2ffd640 1 -- 172.21.15.83:0/1269202590 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef80b7890 con 0x7f2f0415b6e0 2024-04-17T10:41:26.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.179+0000 7f2f08bc6640 1 -- 172.21.15.83:0/1269202590 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f04157980 con 0x7f2f0415b6e0 2024-04-17T10:41:26.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.179+0000 7f2ef2ffd640 1 -- 172.21.15.83:0/1269202590 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef80b0040 con 0x7f2f0415b6e0 2024-04-17T10:41:26.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.179+0000 7f2f08bc6640 1 -- 172.21.15.83:0/1269202590 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f04155540 con 0x7f2f0415b6e0 2024-04-17T10:41:26.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.179+0000 7f2f08bc6640 1 -- 172.21.15.83:0/1269202590 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f04155350 con 0x7f2f0415b6e0 2024-04-17T10:41:26.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.179+0000 7f2ef2ffd640 1 -- 172.21.15.83:0/1269202590 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2ef80be070 con 0x7f2f0415b6e0 2024-04-17T10:41:26.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.183+0000 7f2ef2ffd640 1 --2- 172.21.15.83:0/1269202590 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2ee4066f60 0x7f2ee4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:26.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.183+0000 7f2ef2ffd640 1 -- 172.21.15.83:0/1269202590 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2ef811f900 con 0x7f2f0415b6e0 2024-04-17T10:41:26.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.183+0000 7f2f0259c640 1 --2- 172.21.15.83:0/1269202590 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2ee4066f60 0x7f2ee4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:26.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.183+0000 7f2f0259c640 1 --2- 172.21.15.83:0/1269202590 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2ee4066f60 0x7f2ee4069420 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4007fd0 tx=0x7f2ef4007960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:26.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.187+0000 7f2ef2ffd640 1 -- 172.21.15.83:0/1269202590 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2ef8016090 con 0x7f2f0415b6e0 2024-04-17T10:41:26.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:26.379+0000 7f2f08bc6640 1 -- 172.21.15.83:0/1269202590 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2f04074f90 con 0x7f2f0415b6e0 2024-04-17T10:41:27.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.031+0000 7f2ef2ffd640 1 -- 172.21.15.83:0/1269202590 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v16) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2ef80f0940 con 0x7f2f0415b6e0 2024-04-17T10:41:27.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.031+0000 7f2f08bc6640 1 -- 172.21.15.83:0/1269202590 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2ee4066f60 msgr2=0x7f2ee4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:27.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.031+0000 7f2f08bc6640 1 --2- 172.21.15.83:0/1269202590 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2ee4066f60 0x7f2ee4069420 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4007fd0 tx=0x7f2ef4007960 comp rx=0 tx=0).stop 2024-04-17T10:41:27.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.031+0000 7f2f08bc6640 1 -- 172.21.15.83:0/1269202590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f0415b6e0 msgr2=0x7f2f04155fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:27.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.031+0000 7f2f08bc6640 1 --2- 172.21.15.83:0/1269202590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f0415b6e0 0x7f2f04155fa0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2ef801a040 tx=0x7f2ef80191b0 comp rx=0 tx=0).stop 2024-04-17T10:41:27.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.031+0000 7f2f08bc6640 1 -- 172.21.15.83:0/1269202590 shutdown_connections 2024-04-17T10:41:27.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.031+0000 7f2f08bc6640 1 --2- 172.21.15.83:0/1269202590 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2ee4066f60 0x7f2ee4069420 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:27.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.031+0000 7f2f08bc6640 1 --2- 172.21.15.83:0/1269202590 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f0415bac0 0x7f2f041564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:27.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.031+0000 7f2f08bc6640 1 --2- 172.21.15.83:0/1269202590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f0415b6e0 0x7f2f04155fa0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:27.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.031+0000 7f2f08bc6640 1 --2- 172.21.15.83:0/1269202590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f04070470 0x7f2f04155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:27.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.031+0000 7f2f08bc6640 1 -- 172.21.15.83:0/1269202590 >> 172.21.15.83:0/1269202590 conn(0x7f2f040b15b0 msgr2=0x7f2f040cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:27.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.031+0000 7f2f08bc6640 1 -- 172.21.15.83:0/1269202590 shutdown_connections 2024-04-17T10:41:27.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.035+0000 7f2f08bc6640 1 -- 172.21.15.83:0/1269202590 wait complete. 2024-04-17T10:41:27.053 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-17T10:41:27.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.231+0000 7f68629c1640 1 Processor -- start 2024-04-17T10:41:27.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.231+0000 7f68629c1640 1 -- start start 2024-04-17T10:41:27.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.231+0000 7f68629c1640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f685c0aedf0 0x7f685c0b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:27.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.231+0000 7f68629c1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f685c15cb40 0x7f685c06abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:27.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.231+0000 7f68629c1640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685c0b39d0 con 0x7f685c06b120 2024-04-17T10:41:27.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.231+0000 7f68629c1640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f685c072730 con 0x7f685c15cb40 2024-04-17T10:41:27.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.231+0000 7f68629c1640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f685c071290 con 0x7f685c0aedf0 2024-04-17T10:41:27.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.231+0000 7f685b7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f685c15cb40 0x7f685c06abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:27.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.231+0000 7f685b7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f685c15cb40 0x7f685c06abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40438/0 (socket says 172.21.15.83:40438) 2024-04-17T10:41:27.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.231+0000 7f685b7fe640 1 -- 172.21.15.83:0/3011448800 learned_addr learned my addr 172.21.15.83:0/3011448800 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:27.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.231+0000 7f685bfff640 1 --2- 172.21.15.83:0/3011448800 >> v2:172.21.15.167:3300/0 conn(0x7f685c0aedf0 0x7f685c0b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:27.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.231+0000 7f685b7fe640 1 -- 172.21.15.83:0/3011448800 >> v2:172.21.15.167:3300/0 conn(0x7f685c0aedf0 msgr2=0x7f685c0b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:27.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.231+0000 7f685b7fe640 1 --2- 172.21.15.83:0/3011448800 >> v2:172.21.15.167:3300/0 conn(0x7f685c0aedf0 0x7f685c0b30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:27.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.231+0000 7f685b7fe640 1 -- 172.21.15.83:0/3011448800 >> v1:172.21.15.167:6789/0 conn(0x7f685c06b120 legacy=0x7f685c15f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:27.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.231+0000 7f685b7fe640 1 -- 172.21.15.83:0/3011448800 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c070c40 con 0x7f685c15cb40 2024-04-17T10:41:27.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.235+0000 7f685b7fe640 1 --2- 172.21.15.83:0/3011448800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f685c15cb40 0x7f685c06abe0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f685c0cf4a0 tx=0x7f684c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f75287d4bde3e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:27.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.235+0000 7f685affd640 1 -- 172.21.15.83:0/3011448800 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c0b0070 con 0x7f685c15cb40 2024-04-17T10:41:27.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.235+0000 7f685affd640 1 -- 172.21.15.83:0/3011448800 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684c0ab600 con 0x7f685c15cb40 2024-04-17T10:41:27.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.235+0000 7f685affd640 1 -- 172.21.15.83:0/3011448800 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c0ab9d0 con 0x7f685c15cb40 2024-04-17T10:41:27.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.235+0000 7f68629c1640 1 -- 172.21.15.83:0/3011448800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f685c15cb40 msgr2=0x7f685c06abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:27.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.235+0000 7f68629c1640 1 --2- 172.21.15.83:0/3011448800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f685c15cb40 0x7f685c06abe0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f685c0cf4a0 tx=0x7f684c01c4e0 comp rx=0 tx=0).stop 2024-04-17T10:41:27.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.235+0000 7f68629c1640 1 -- 172.21.15.83:0/3011448800 shutdown_connections 2024-04-17T10:41:27.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.235+0000 7f68629c1640 1 --2- 172.21.15.83:0/3011448800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f685c15cb40 0x7f685c06abe0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:27.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.235+0000 7f68629c1640 1 --2- 172.21.15.83:0/3011448800 >> v2:172.21.15.167:3300/0 conn(0x7f685c0aedf0 0x7f685c0b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:27.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.235+0000 7f68629c1640 1 -- 172.21.15.83:0/3011448800 >> 172.21.15.83:0/3011448800 conn(0x7f685c0cc050 msgr2=0x7f685c0cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:27.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.235+0000 7f68629c1640 1 -- 172.21.15.83:0/3011448800 shutdown_connections 2024-04-17T10:41:27.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.235+0000 7f68629c1640 1 -- 172.21.15.83:0/3011448800 wait complete. 2024-04-17T10:41:27.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.235+0000 7f68629c1640 1 Processor -- start 2024-04-17T10:41:27.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.235+0000 7f68629c1640 1 -- start start 2024-04-17T10:41:27.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.239+0000 7f68629c1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f685c06b120 0x7f685c13b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:27.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.239+0000 7f68629c1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f685c0aedf0 0x7f685c158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:27.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.239+0000 7f68629c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f685c15cb40 0x7f685c1509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:27.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.239+0000 7f68629c1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c06c2e0 con 0x7f685c15cb40 2024-04-17T10:41:27.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.239+0000 7f68629c1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c06c160 con 0x7f685c06b120 2024-04-17T10:41:27.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.239+0000 7f68629c1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685c168890 con 0x7f685c0aedf0 2024-04-17T10:41:27.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.239+0000 7f6860f5e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f685c15cb40 0x7f685c1509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:27.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.239+0000 7f6860f5e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f685c15cb40 0x7f685c1509d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40442/0 (socket says 172.21.15.83:40442) 2024-04-17T10:41:27.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.239+0000 7f6860f5e640 1 -- 172.21.15.83:0/3287862346 learned_addr learned my addr 172.21.15.83:0/3287862346 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:27.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.239+0000 7f685bfff640 1 --2- 172.21.15.83:0/3287862346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f685c06b120 0x7f685c13b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:27.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.239+0000 7f6860f5e640 1 -- 172.21.15.83:0/3287862346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f685c0aedf0 msgr2=0x7f685c158690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:27.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.239+0000 7f6860f5e640 1 --2- 172.21.15.83:0/3287862346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f685c0aedf0 0x7f685c158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:27.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.239+0000 7f6860f5e640 1 -- 172.21.15.83:0/3287862346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f685c06b120 msgr2=0x7f685c13b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:27.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.239+0000 7f6860f5e640 1 --2- 172.21.15.83:0/3287862346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f685c06b120 0x7f685c13b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:27.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.239+0000 7f6860f5e640 1 -- 172.21.15.83:0/3287862346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c13ba10 con 0x7f685c15cb40 2024-04-17T10:41:27.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.239+0000 7f685bfff640 1 --2- 172.21.15.83:0/3287862346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f685c06b120 0x7f685c13b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:27.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.239+0000 7f6860f5e640 1 --2- 172.21.15.83:0/3287862346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f685c15cb40 0x7f685c1509d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f685400b9e0 tx=0x7f685400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:27.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.239+0000 7f6858ff9640 1 -- 172.21.15.83:0/3287862346 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6854003b80 con 0x7f685c15cb40 2024-04-17T10:41:27.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.239+0000 7f6858ff9640 1 -- 172.21.15.83:0/3287862346 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6854003d20 con 0x7f685c15cb40 2024-04-17T10:41:27.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.239+0000 7f68629c1640 1 -- 172.21.15.83:0/3287862346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f685c1575f0 con 0x7f685c15cb40 2024-04-17T10:41:27.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.239+0000 7f6858ff9640 1 -- 172.21.15.83:0/3287862346 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685400cae0 con 0x7f685c15cb40 2024-04-17T10:41:27.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.239+0000 7f68629c1640 1 -- 172.21.15.83:0/3287862346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f685c157bb0 con 0x7f685c15cb40 2024-04-17T10:41:27.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.239+0000 7f68629c1640 1 -- 172.21.15.83:0/3287862346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f685c075000 con 0x7f685c15cb40 2024-04-17T10:41:27.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.239+0000 7f6858ff9640 1 -- 172.21.15.83:0/3287862346 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6854006d10 con 0x7f685c15cb40 2024-04-17T10:41:27.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.239+0000 7f6858ff9640 1 --2- 172.21.15.83:0/3287862346 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6848066f90 0x7f6848069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:27.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.239+0000 7f6858ff9640 1 -- 172.21.15.83:0/3287862346 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6854084b20 con 0x7f685c15cb40 2024-04-17T10:41:27.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.239+0000 7f685bfff640 1 --2- 172.21.15.83:0/3287862346 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6848066f90 0x7f6848069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:27.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.239+0000 7f685bfff640 1 --2- 172.21.15.83:0/3287862346 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6848066f90 0x7f6848069450 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f685c156ee0 tx=0x7f6850008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:27.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.243+0000 7f6858ff9640 1 -- 172.21.15.83:0/3287862346 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6854055ad0 con 0x7f685c15cb40 2024-04-17T10:41:27.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:27.407+0000 7f68629c1640 1 -- 172.21.15.83:0/3287862346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f685c0b2cc0 con 0x7f685c15cb40 2024-04-17T10:41:28.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.359+0000 7f6858ff9640 1 -- 172.21.15.83:0/3287862346 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v37) v1 ==== 181+0+0 (secure 0 0 0) 0x7f68540553d0 con 0x7f685c15cb40 2024-04-17T10:41:28.357 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2024-04-17T10:41:28.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.363+0000 7f68629c1640 1 -- 172.21.15.83:0/3287862346 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6848066f90 msgr2=0x7f6848069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:28.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.363+0000 7f68629c1640 1 --2- 172.21.15.83:0/3287862346 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6848066f90 0x7f6848069450 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f685c156ee0 tx=0x7f6850008040 comp rx=0 tx=0).stop 2024-04-17T10:41:28.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.363+0000 7f68629c1640 1 -- 172.21.15.83:0/3287862346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f685c15cb40 msgr2=0x7f685c1509d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:28.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.363+0000 7f68629c1640 1 --2- 172.21.15.83:0/3287862346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f685c15cb40 0x7f685c1509d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f685400b9e0 tx=0x7f685400bea0 comp rx=0 tx=0).stop 2024-04-17T10:41:28.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.363+0000 7f68629c1640 1 -- 172.21.15.83:0/3287862346 shutdown_connections 2024-04-17T10:41:28.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.363+0000 7f68629c1640 1 --2- 172.21.15.83:0/3287862346 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6848066f90 0x7f6848069450 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:28.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.363+0000 7f68629c1640 1 --2- 172.21.15.83:0/3287862346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f685c15cb40 0x7f685c1509d0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:28.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.363+0000 7f68629c1640 1 --2- 172.21.15.83:0/3287862346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f685c0aedf0 0x7f685c158690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:28.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.363+0000 7f68629c1640 1 --2- 172.21.15.83:0/3287862346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f685c06b120 0x7f685c13b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:28.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.363+0000 7f68629c1640 1 -- 172.21.15.83:0/3287862346 >> 172.21.15.83:0/3287862346 conn(0x7f685c0cc050 msgr2=0x7f685c15d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:28.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.363+0000 7f68629c1640 1 -- 172.21.15.83:0/3287862346 shutdown_connections 2024-04-17T10:41:28.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.363+0000 7f68629c1640 1 -- 172.21.15.83:0/3287862346 wait complete. 2024-04-17T10:41:28.382 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-17T10:41:28.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.515+0000 7f5ac0e3c640 1 Processor -- start 2024-04-17T10:41:28.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.515+0000 7f5ac0e3c640 1 -- start start 2024-04-17T10:41:28.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.515+0000 7f5ac0e3c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5abc0aebd0 0x7f5abc0ced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:28.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.515+0000 7f5ac0e3c640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5abc0cf3e0 con 0x7f5abc15b990 2024-04-17T10:41:28.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.515+0000 7f5ac0e3c640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5abc0cd890 con 0x7f5abc06abe0 2024-04-17T10:41:28.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.515+0000 7f5ac0e3c640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5abc15e5c0 con 0x7f5abc0aebd0 2024-04-17T10:41:28.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.515+0000 7f5ab9d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5abc15b990 0x7f5abc0cd180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53652/0 (socket says 172.21.15.83:53652) 2024-04-17T10:41:28.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.515+0000 7f5ab9d9b640 1 -- 172.21.15.83:0/1698395448 learned_addr learned my addr 172.21.15.83:0/1698395448 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:28.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.515+0000 7f5aba59c640 1 --2- 172.21.15.83:0/1698395448 >> v2:172.21.15.167:3300/0 conn(0x7f5abc0aebd0 0x7f5abc0ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:28.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.515+0000 7f5ab959a640 1 -- 172.21.15.83:0/1698395448 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 327152010 0 0) 0x7f5abc0cf3e0 con 0x7f5abc15b990 2024-04-17T10:41:28.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.515+0000 7f5ab959a640 1 -- 172.21.15.83:0/1698395448 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5aa4003530 con 0x7f5abc15b990 2024-04-17T10:41:28.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.515+0000 7f5ab959a640 1 -- 172.21.15.83:0/1698395448 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1089785856 0 0) 0x7f5aa4003530 con 0x7f5abc15b990 2024-04-17T10:41:28.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.515+0000 7f5ab959a640 1 -- 172.21.15.83:0/1698395448 >> v2:172.21.15.167:3300/0 conn(0x7f5abc0aebd0 msgr2=0x7f5abc0ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:28.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.515+0000 7f5ab959a640 1 --2- 172.21.15.83:0/1698395448 >> v2:172.21.15.167:3300/0 conn(0x7f5abc0aebd0 0x7f5abc0ced50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:28.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.515+0000 7f5ab959a640 1 -- 172.21.15.83:0/1698395448 >> v1:172.21.15.167:6789/0 conn(0x7f5abc06abe0 legacy=0x7f5abc15ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:28.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.515+0000 7f5ab959a640 1 -- 172.21.15.83:0/1698395448 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5abc0b4230 con 0x7f5abc15b990 2024-04-17T10:41:28.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.515+0000 7f5ab959a640 1 -- 172.21.15.83:0/1698395448 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f5ab0001680 con 0x7f5abc15b990 2024-04-17T10:41:28.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.515+0000 7f5ab959a640 1 -- 172.21.15.83:0/1698395448 >> v1:172.21.15.83:6789/0 conn(0x7f5abc15b990 legacy=0x7f5abc0cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:28.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.515+0000 7f5ab959a640 1 --2- 172.21.15.83:0/1698395448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5aa40039c0 0x7f5aa4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:28.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.515+0000 7f5ab959a640 1 --2- 172.21.15.83:0/1698395448 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5aa4004810 0x7f5aa4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:28.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.515+0000 7f5ab959a640 1 --2- 172.21.15.83:0/1698395448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5aa40051d0 0x7f5aa400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:28.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.515+0000 7f5ab959a640 1 -- 172.21.15.83:0/1698395448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc15e5c0 con 0x7f5aa40051d0 2024-04-17T10:41:28.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.515+0000 7f5ab959a640 1 -- 172.21.15.83:0/1698395448 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa400c0a0 con 0x7f5aa40039c0 2024-04-17T10:41:28.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.515+0000 7f5ab959a640 1 -- 172.21.15.83:0/1698395448 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa400c2a0 con 0x7f5aa4004810 2024-04-17T10:41:28.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.515+0000 7f5abad9d640 1 --2- 172.21.15.83:0/1698395448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5aa40051d0 0x7f5aa400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:28.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.519+0000 7f5ab9d9b640 1 --2- 172.21.15.83:0/1698395448 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5aa4004810 0x7f5aa4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:28.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.519+0000 7f5aba59c640 1 --2- 172.21.15.83:0/1698395448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5aa40039c0 0x7f5aa4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:28.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.519+0000 7f5abad9d640 1 -- 172.21.15.83:0/1698395448 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5aa4004810 msgr2=0x7f5aa4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:28.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.519+0000 7f5abad9d640 1 --2- 172.21.15.83:0/1698395448 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5aa4004810 0x7f5aa4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:28.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.519+0000 7f5abad9d640 1 -- 172.21.15.83:0/1698395448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5aa40039c0 msgr2=0x7f5aa4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:28.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.519+0000 7f5abad9d640 1 --2- 172.21.15.83:0/1698395448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5aa40039c0 0x7f5aa4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:28.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.519+0000 7f5abad9d640 1 -- 172.21.15.83:0/1698395448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa4003530 con 0x7f5aa40051d0 2024-04-17T10:41:28.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.519+0000 7f5ab9d9b640 1 --2- 172.21.15.83:0/1698395448 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5aa4004810 0x7f5aa4004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:28.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.519+0000 7f5abad9d640 1 -- 172.21.15.83:0/1698395448 reap_dead start 2024-04-17T10:41:28.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.519+0000 7f5abad9d640 1 --2- 172.21.15.83:0/1698395448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5aa40051d0 0x7f5aa400b9d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4001e70 tx=0x7f5ab401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=702bcbb255e6f467 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:28.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.519+0000 7f5ab959a640 1 -- 172.21.15.83:0/1698395448 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4001a90 con 0x7f5aa40051d0 2024-04-17T10:41:28.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.519+0000 7f5ab959a640 1 -- 172.21.15.83:0/1698395448 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab4003bc0 con 0x7f5aa40051d0 2024-04-17T10:41:28.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.519+0000 7f5ab959a640 1 -- 172.21.15.83:0/1698395448 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab40050d0 con 0x7f5aa40051d0 2024-04-17T10:41:28.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.519+0000 7f5ac0e3c640 1 -- 172.21.15.83:0/1698395448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5aa40051d0 msgr2=0x7f5aa400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:28.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.519+0000 7f5ac0e3c640 1 --2- 172.21.15.83:0/1698395448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5aa40051d0 0x7f5aa400b9d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4001e70 tx=0x7f5ab401c780 comp rx=0 tx=0).stop 2024-04-17T10:41:28.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.519+0000 7f5ac0e3c640 1 -- 172.21.15.83:0/1698395448 shutdown_connections 2024-04-17T10:41:28.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.519+0000 7f5ac0e3c640 1 --2- 172.21.15.83:0/1698395448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5aa40051d0 0x7f5aa400b9d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:28.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.519+0000 7f5ac0e3c640 1 -- 172.21.15.83:0/1698395448 >> 172.21.15.83:0/1698395448 conn(0x7f5abc0cc050 msgr2=0x7f5abc0cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:28.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.519+0000 7f5ac0e3c640 1 -- 172.21.15.83:0/1698395448 shutdown_connections 2024-04-17T10:41:28.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.519+0000 7f5ac0e3c640 1 -- 172.21.15.83:0/1698395448 wait complete. 2024-04-17T10:41:28.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.523+0000 7f5ac0e3c640 1 Processor -- start 2024-04-17T10:41:28.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.523+0000 7f5ac0e3c640 1 -- start start 2024-04-17T10:41:28.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.523+0000 7f5ac0e3c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5aa40051d0 0x7f5abc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:28.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.523+0000 7f5ac0e3c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5abc1540c0 0x7f5abc149db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:28.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.523+0000 7f5ac0e3c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5abc0702e0 0x7f5abc158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:28.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.523+0000 7f5ac0e3c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc15be20 con 0x7f5aa40051d0 2024-04-17T10:41:28.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.523+0000 7f5ac0e3c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc0b47f0 con 0x7f5abc1540c0 2024-04-17T10:41:28.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.523+0000 7f5ac0e3c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc0aefb0 con 0x7f5abc0702e0 2024-04-17T10:41:28.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.523+0000 7f5aba59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5aa40051d0 0x7f5abc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:28.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.523+0000 7f5aba59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5aa40051d0 0x7f5abc071940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40460/0 (socket says 172.21.15.83:40460) 2024-04-17T10:41:28.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.523+0000 7f5aba59c640 1 -- 172.21.15.83:0/787134303 learned_addr learned my addr 172.21.15.83:0/787134303 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:28.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.523+0000 7f5ab9d9b640 1 --2- 172.21.15.83:0/787134303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5abc1540c0 0x7f5abc149db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:28.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.523+0000 7f5aba59c640 1 -- 172.21.15.83:0/787134303 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5abc0702e0 msgr2=0x7f5abc158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:28.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.523+0000 7f5aba59c640 1 --2- 172.21.15.83:0/787134303 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5abc0702e0 0x7f5abc158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:28.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.523+0000 7f5aba59c640 1 -- 172.21.15.83:0/787134303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5abc1540c0 msgr2=0x7f5abc149db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:28.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.523+0000 7f5aba59c640 1 --2- 172.21.15.83:0/787134303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5abc1540c0 0x7f5abc149db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:28.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.523+0000 7f5aba59c640 1 -- 172.21.15.83:0/787134303 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5abc0708a0 con 0x7f5aa40051d0 2024-04-17T10:41:28.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.523+0000 7f5ab9d9b640 1 --2- 172.21.15.83:0/787134303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5abc1540c0 0x7f5abc149db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:28.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.523+0000 7f5aba59c640 1 --2- 172.21.15.83:0/787134303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5aa40051d0 0x7f5abc071940 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5aac011590 tx=0x7f5aac011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:28.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.523+0000 7f5aaaffd640 1 -- 172.21.15.83:0/787134303 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac015930 con 0x7f5aa40051d0 2024-04-17T10:41:28.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.523+0000 7f5aaaffd640 1 -- 172.21.15.83:0/787134303 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aac015ad0 con 0x7f5aa40051d0 2024-04-17T10:41:28.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.523+0000 7f5aaaffd640 1 -- 172.21.15.83:0/787134303 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac00f460 con 0x7f5aa40051d0 2024-04-17T10:41:28.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.523+0000 7f5ac0e3c640 1 -- 172.21.15.83:0/787134303 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5abc070b80 con 0x7f5aa40051d0 2024-04-17T10:41:28.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.523+0000 7f5ac0e3c640 1 -- 172.21.15.83:0/787134303 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5abc071140 con 0x7f5aa40051d0 2024-04-17T10:41:28.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.523+0000 7f5ac0e3c640 1 -- 172.21.15.83:0/787134303 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5abc075000 con 0x7f5aa40051d0 2024-04-17T10:41:28.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.523+0000 7f5aaaffd640 1 -- 172.21.15.83:0/787134303 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5aac00f600 con 0x7f5aa40051d0 2024-04-17T10:41:28.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.527+0000 7f5aaaffd640 1 --2- 172.21.15.83:0/787134303 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5a9c066f60 0x7f5a9c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:28.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.527+0000 7f5aaaffd640 1 -- 172.21.15.83:0/787134303 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f5aac085920 con 0x7f5aa40051d0 2024-04-17T10:41:28.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.527+0000 7f5ab9d9b640 1 --2- 172.21.15.83:0/787134303 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5a9c066f60 0x7f5a9c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:28.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.527+0000 7f5ab9d9b640 1 --2- 172.21.15.83:0/787134303 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5a9c066f60 0x7f5a9c069420 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5abc14a770 tx=0x7f5ab0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:28.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.531+0000 7f5aaaffd640 1 -- 172.21.15.83:0/787134303 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5aac019030 con 0x7f5aa40051d0 2024-04-17T10:41:28.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:28.695+0000 7f5ac0e3c640 1 -- 172.21.15.83:0/787134303 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5abc15b770 con 0x7f5aa40051d0 2024-04-17T10:41:29.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.367+0000 7f5aaaffd640 1 -- 172.21.15.83:0/787134303 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v38) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5aac056980 con 0x7f5aa40051d0 2024-04-17T10:41:29.367 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2024-04-17T10:41:29.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.375+0000 7f5ac0e3c640 1 -- 172.21.15.83:0/787134303 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5a9c066f60 msgr2=0x7f5a9c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:29.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.375+0000 7f5ac0e3c640 1 --2- 172.21.15.83:0/787134303 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5a9c066f60 0x7f5a9c069420 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5abc14a770 tx=0x7f5ab0009040 comp rx=0 tx=0).stop 2024-04-17T10:41:29.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.375+0000 7f5ac0e3c640 1 -- 172.21.15.83:0/787134303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5aa40051d0 msgr2=0x7f5abc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:29.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.375+0000 7f5ac0e3c640 1 --2- 172.21.15.83:0/787134303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5aa40051d0 0x7f5abc071940 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5aac011590 tx=0x7f5aac011a50 comp rx=0 tx=0).stop 2024-04-17T10:41:29.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.375+0000 7f5ac0e3c640 1 -- 172.21.15.83:0/787134303 shutdown_connections 2024-04-17T10:41:29.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.375+0000 7f5ac0e3c640 1 --2- 172.21.15.83:0/787134303 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5a9c066f60 0x7f5a9c069420 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:29.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.375+0000 7f5ac0e3c640 1 --2- 172.21.15.83:0/787134303 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5abc0702e0 0x7f5abc158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:29.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.375+0000 7f5ac0e3c640 1 --2- 172.21.15.83:0/787134303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5abc1540c0 0x7f5abc149db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:29.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.375+0000 7f5ac0e3c640 1 --2- 172.21.15.83:0/787134303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5aa40051d0 0x7f5abc071940 secure :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5aac011590 tx=0x7f5aac011a50 comp rx=0 tx=0).stop 2024-04-17T10:41:29.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.375+0000 7f5ac0e3c640 1 -- 172.21.15.83:0/787134303 >> 172.21.15.83:0/787134303 conn(0x7f5abc0cc050 msgr2=0x7f5abc15ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:29.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.375+0000 7f5ac0e3c640 1 -- 172.21.15.83:0/787134303 shutdown_connections 2024-04-17T10:41:29.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.375+0000 7f5ac0e3c640 1 -- 172.21.15.83:0/787134303 wait complete. 2024-04-17T10:41:29.395 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-17T10:41:29.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.515+0000 7f12e2f4a640 1 Processor -- start 2024-04-17T10:41:29.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.515+0000 7f12e2f4a640 1 -- start start 2024-04-17T10:41:29.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.519+0000 7f12e2f4a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12dc06b520 0x7f12dc0b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:29.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.519+0000 7f12e2f4a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f12dc0b4040 0x7f12dc06cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:29.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.519+0000 7f12e2f4a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12dc15bd70 con 0x7f12dc154de0 2024-04-17T10:41:29.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.519+0000 7f12e2f4a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12dc0b4870 con 0x7f12dc0b4040 2024-04-17T10:41:29.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.519+0000 7f12e2f4a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc15bf60 con 0x7f12dc06b520 2024-04-17T10:41:29.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.519+0000 7f12e0ce6640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f12dc154de0 0x7f12dc06ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53666/0 (socket says 172.21.15.83:53666) 2024-04-17T10:41:29.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.519+0000 7f12e0ce6640 1 -- 172.21.15.83:0/3344038247 learned_addr learned my addr 172.21.15.83:0/3344038247 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:29.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.519+0000 7f12e14e7640 1 --2- 172.21.15.83:0/3344038247 >> v2:172.21.15.83:3300/0 conn(0x7f12dc0b4040 0x7f12dc06cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:29.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.519+0000 7f12db7fe640 1 -- 172.21.15.83:0/3344038247 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2539280979 0 0) 0x7f12dc15bd70 con 0x7f12dc154de0 2024-04-17T10:41:29.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.519+0000 7f12db7fe640 1 -- 172.21.15.83:0/3344038247 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12c4003530 con 0x7f12dc154de0 2024-04-17T10:41:29.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.519+0000 7f12dbfff640 1 --2- 172.21.15.83:0/3344038247 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12dc06b520 0x7f12dc0b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:29.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.519+0000 7f12e14e7640 1 -- 172.21.15.83:0/3344038247 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12dc06b520 msgr2=0x7f12dc0b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:29.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.519+0000 7f12e14e7640 1 --2- 172.21.15.83:0/3344038247 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12dc06b520 0x7f12dc0b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:29.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.519+0000 7f12e14e7640 1 -- 172.21.15.83:0/3344038247 >> v1:172.21.15.83:6789/0 conn(0x7f12dc154de0 legacy=0x7f12dc06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:29.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.519+0000 7f12db7fe640 1 -- 172.21.15.83:0/3344038247 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1717621400 0 0) 0x7f12c4003530 con 0x7f12dc154de0 2024-04-17T10:41:29.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.519+0000 7f12e14e7640 1 -- 172.21.15.83:0/3344038247 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12dc15afb0 con 0x7f12dc0b4040 2024-04-17T10:41:29.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12e14e7640 1 --2- 172.21.15.83:0/3344038247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12dc0b4040 0x7f12dc06cbe0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f12d4009d90 tx=0x7f12d401c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29171d32b49ff58b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:29.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12db7fe640 1 -- 172.21.15.83:0/3344038247 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d40b2070 con 0x7f12dc0b4040 2024-04-17T10:41:29.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12db7fe640 1 -- 172.21.15.83:0/3344038247 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d40a5580 con 0x7f12dc0b4040 2024-04-17T10:41:29.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12db7fe640 1 -- 172.21.15.83:0/3344038247 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc001680 con 0x7f12dc0b4040 2024-04-17T10:41:29.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12e2f4a640 1 -- 172.21.15.83:0/3344038247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12dc0b4040 msgr2=0x7f12dc06cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:29.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12e2f4a640 1 --2- 172.21.15.83:0/3344038247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12dc0b4040 0x7f12dc06cbe0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f12d4009d90 tx=0x7f12d401c890 comp rx=0 tx=0).stop 2024-04-17T10:41:29.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12e2f4a640 1 -- 172.21.15.83:0/3344038247 shutdown_connections 2024-04-17T10:41:29.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12e2f4a640 1 --2- 172.21.15.83:0/3344038247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12dc0b4040 0x7f12dc06cbe0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:29.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12e2f4a640 1 --2- 172.21.15.83:0/3344038247 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12dc06b520 0x7f12dc0b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:29.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12e2f4a640 1 -- 172.21.15.83:0/3344038247 >> 172.21.15.83:0/3344038247 conn(0x7f12dc0b15b0 msgr2=0x7f12dc0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:29.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12e2f4a640 1 -- 172.21.15.83:0/3344038247 shutdown_connections 2024-04-17T10:41:29.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12e2f4a640 1 -- 172.21.15.83:0/3344038247 wait complete. 2024-04-17T10:41:29.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12e2f4a640 1 Processor -- start 2024-04-17T10:41:29.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12e2f4a640 1 -- start start 2024-04-17T10:41:29.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12e2f4a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12dc06b520 0x7f12dc154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:29.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12e2f4a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12dc0b4040 0x7f12dc0ab2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:29.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12e2f4a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12dc154de0 0x7f12dc0abce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:29.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12e2f4a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc0b44f0 con 0x7f12dc0b4040 2024-04-17T10:41:29.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12e2f4a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc1551c0 con 0x7f12dc06b520 2024-04-17T10:41:29.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12e2f4a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc15e390 con 0x7f12dc154de0 2024-04-17T10:41:29.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12dbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12dc0b4040 0x7f12dc0ab2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:29.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12dbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12dc0b4040 0x7f12dc0ab2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40486/0 (socket says 172.21.15.83:40486) 2024-04-17T10:41:29.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12dbfff640 1 -- 172.21.15.83:0/3319448070 learned_addr learned my addr 172.21.15.83:0/3319448070 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:29.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12e14e7640 1 --2- 172.21.15.83:0/3319448070 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12dc154de0 0x7f12dc0abce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:29.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12e0ce6640 1 --2- 172.21.15.83:0/3319448070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12dc06b520 0x7f12dc154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:29.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12dbfff640 1 -- 172.21.15.83:0/3319448070 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12dc154de0 msgr2=0x7f12dc0abce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:29.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12dbfff640 1 --2- 172.21.15.83:0/3319448070 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12dc154de0 0x7f12dc0abce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:29.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12dbfff640 1 -- 172.21.15.83:0/3319448070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12dc06b520 msgr2=0x7f12dc154100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:29.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12dbfff640 1 --2- 172.21.15.83:0/3319448070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12dc06b520 0x7f12dc154100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:29.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12dbfff640 1 -- 172.21.15.83:0/3319448070 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12dc1577d0 con 0x7f12dc0b4040 2024-04-17T10:41:29.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12e0ce6640 1 --2- 172.21.15.83:0/3319448070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12dc06b520 0x7f12dc154100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:29.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12dbfff640 1 --2- 172.21.15.83:0/3319448070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12dc0b4040 0x7f12dc0ab2a0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f12d000d800 tx=0x7f12d000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:29.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.523+0000 7f12d97fa640 1 -- 172.21.15.83:0/3319448070 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d000b6c0 con 0x7f12dc0b4040 2024-04-17T10:41:29.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.527+0000 7f12d97fa640 1 -- 172.21.15.83:0/3319448070 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d0019070 con 0x7f12dc0b4040 2024-04-17T10:41:29.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.527+0000 7f12e2f4a640 1 -- 172.21.15.83:0/3319448070 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12dc0aa4e0 con 0x7f12dc0b4040 2024-04-17T10:41:29.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.527+0000 7f12d97fa640 1 -- 172.21.15.83:0/3319448070 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d000ba60 con 0x7f12dc0b4040 2024-04-17T10:41:29.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.527+0000 7f12e2f4a640 1 -- 172.21.15.83:0/3319448070 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12dc0aaaa0 con 0x7f12dc0b4040 2024-04-17T10:41:29.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.527+0000 7f12e2f4a640 1 -- 172.21.15.83:0/3319448070 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12dc074f90 con 0x7f12dc0b4040 2024-04-17T10:41:29.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.527+0000 7f12d97fa640 1 -- 172.21.15.83:0/3319448070 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f12d000bc00 con 0x7f12dc0b4040 2024-04-17T10:41:29.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.527+0000 7f12d97fa640 1 --2- 172.21.15.83:0/3319448070 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f12c8066f60 0x7f12c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:29.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.527+0000 7f12d97fa640 1 -- 172.21.15.83:0/3319448070 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f12d00851d0 con 0x7f12dc0b4040 2024-04-17T10:41:29.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.527+0000 7f12e0ce6640 1 --2- 172.21.15.83:0/3319448070 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f12c8066f60 0x7f12c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:29.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.527+0000 7f12e0ce6640 1 --2- 172.21.15.83:0/3319448070 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f12c8066f60 0x7f12c8069420 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f12cc001ec0 tx=0x7f12cc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:29.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.531+0000 7f12d97fa640 1 -- 172.21.15.83:0/3319448070 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f12d00564e0 con 0x7f12dc0b4040 2024-04-17T10:41:29.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.683+0000 7f12e2f4a640 1 -- 172.21.15.83:0/3319448070 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f12dc0b3280 con 0x7f12dc0b4040 2024-04-17T10:41:29.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.687+0000 7f12d97fa640 1 -- 172.21.15.83:0/3319448070 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f12d0083e20 con 0x7f12dc0b4040 2024-04-17T10:41:29.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.691+0000 7f12e2f4a640 1 -- 172.21.15.83:0/3319448070 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f12c8066f60 msgr2=0x7f12c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:29.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.691+0000 7f12e2f4a640 1 --2- 172.21.15.83:0/3319448070 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f12c8066f60 0x7f12c8069420 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f12cc001ec0 tx=0x7f12cc009040 comp rx=0 tx=0).stop 2024-04-17T10:41:29.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.691+0000 7f12e2f4a640 1 -- 172.21.15.83:0/3319448070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12dc0b4040 msgr2=0x7f12dc0ab2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:29.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.691+0000 7f12e2f4a640 1 --2- 172.21.15.83:0/3319448070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12dc0b4040 0x7f12dc0ab2a0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f12d000d800 tx=0x7f12d000dcc0 comp rx=0 tx=0).stop 2024-04-17T10:41:29.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.691+0000 7f12e2f4a640 1 -- 172.21.15.83:0/3319448070 shutdown_connections 2024-04-17T10:41:29.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.691+0000 7f12e2f4a640 1 --2- 172.21.15.83:0/3319448070 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f12c8066f60 0x7f12c8069420 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:29.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.691+0000 7f12e2f4a640 1 --2- 172.21.15.83:0/3319448070 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12dc154de0 0x7f12dc0abce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:29.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.691+0000 7f12e2f4a640 1 --2- 172.21.15.83:0/3319448070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12dc0b4040 0x7f12dc0ab2a0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:29.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.691+0000 7f12e2f4a640 1 --2- 172.21.15.83:0/3319448070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12dc06b520 0x7f12dc154100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:29.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.691+0000 7f12e2f4a640 1 -- 172.21.15.83:0/3319448070 >> 172.21.15.83:0/3319448070 conn(0x7f12dc0b15b0 msgr2=0x7f12dc0af250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:29.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.691+0000 7f12e2f4a640 1 -- 172.21.15.83:0/3319448070 shutdown_connections 2024-04-17T10:41:29.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.691+0000 7f12e2f4a640 1 -- 172.21.15.83:0/3319448070 wait complete. 2024-04-17T10:41:29.704 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2024-04-17T10:41:29.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.815+0000 7fb64d8c0640 1 Processor -- start 2024-04-17T10:41:29.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.815+0000 7fb64d8c0640 1 -- start start 2024-04-17T10:41:29.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.815+0000 7fb64d8c0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb64815fd50 0x7fb648163360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:29.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.815+0000 7fb64d8c0640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6480ceb30 con 0x7fb6481669a0 2024-04-17T10:41:29.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.815+0000 7fb64d8c0640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb648160130 con 0x7fb6480b5430 2024-04-17T10:41:29.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.815+0000 7fb64d8c0640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb648166510 con 0x7fb64815fd50 2024-04-17T10:41:29.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.815+0000 7fb6467fc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb6481669a0 0x7fb6480ce000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53674/0 (socket says 172.21.15.83:53674) 2024-04-17T10:41:29.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.815+0000 7fb6467fc640 1 -- 172.21.15.83:0/3502449176 learned_addr learned my addr 172.21.15.83:0/3502449176 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:29.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.815+0000 7fb646ffd640 1 --2- 172.21.15.83:0/3502449176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb64815fd50 0x7fb648163360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:29.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.815+0000 7fb645ffb640 1 -- 172.21.15.83:0/3502449176 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3589109266 0 0) 0x7fb6480ceb30 con 0x7fb6481669a0 2024-04-17T10:41:29.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.815+0000 7fb645ffb640 1 -- 172.21.15.83:0/3502449176 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb634003530 con 0x7fb6481669a0 2024-04-17T10:41:29.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.815+0000 7fb645ffb640 1 -- 172.21.15.83:0/3502449176 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3801702735 0 0) 0x7fb634003530 con 0x7fb6481669a0 2024-04-17T10:41:29.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.815+0000 7fb645ffb640 1 -- 172.21.15.83:0/3502449176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb64815fd50 msgr2=0x7fb648163360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:29.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.815+0000 7fb645ffb640 1 --2- 172.21.15.83:0/3502449176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb64815fd50 0x7fb648163360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:29.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.815+0000 7fb645ffb640 1 -- 172.21.15.83:0/3502449176 >> v1:172.21.15.167:6789/0 conn(0x7fb6480b5430 legacy=0x7fb6480d1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:29.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.815+0000 7fb645ffb640 1 -- 172.21.15.83:0/3502449176 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6480ce850 con 0x7fb6481669a0 2024-04-17T10:41:29.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.815+0000 7fb645ffb640 1 -- 172.21.15.83:0/3502449176 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fb638001a00 con 0x7fb6481669a0 2024-04-17T10:41:29.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.815+0000 7fb645ffb640 1 -- 172.21.15.83:0/3502449176 >> v1:172.21.15.83:6789/0 conn(0x7fb6481669a0 legacy=0x7fb6480ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:29.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.815+0000 7fb645ffb640 1 --2- 172.21.15.83:0/3502449176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6340039f0 0x7fb634003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:29.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.815+0000 7fb645ffb640 1 --2- 172.21.15.83:0/3502449176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb64815fd50 0x7fb634004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:29.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.815+0000 7fb645ffb640 1 --2- 172.21.15.83:0/3502449176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb634004d50 0x7fb6340051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:29.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.815+0000 7fb645ffb640 1 -- 172.21.15.83:0/3502449176 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb648166510 con 0x7fb634004d50 2024-04-17T10:41:29.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.815+0000 7fb645ffb640 1 -- 172.21.15.83:0/3502449176 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63400ba80 con 0x7fb6340039f0 2024-04-17T10:41:29.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.815+0000 7fb645ffb640 1 -- 172.21.15.83:0/3502449176 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb63400bc80 con 0x7fb64815fd50 2024-04-17T10:41:29.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.815+0000 7fb6477fe640 1 --2- 172.21.15.83:0/3502449176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb634004d50 0x7fb6340051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:29.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.815+0000 7fb646ffd640 1 --2- 172.21.15.83:0/3502449176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6340039f0 0x7fb634003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:29.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.815+0000 7fb6467fc640 1 --2- 172.21.15.83:0/3502449176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb64815fd50 0x7fb634004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:29.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.815+0000 7fb6477fe640 1 -- 172.21.15.83:0/3502449176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb64815fd50 msgr2=0x7fb634004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:29.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.815+0000 7fb6477fe640 1 --2- 172.21.15.83:0/3502449176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb64815fd50 0x7fb634004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:29.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.815+0000 7fb6477fe640 1 -- 172.21.15.83:0/3502449176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6340039f0 msgr2=0x7fb634003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:29.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.815+0000 7fb6477fe640 1 --2- 172.21.15.83:0/3502449176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6340039f0 0x7fb634003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:29.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.815+0000 7fb6477fe640 1 -- 172.21.15.83:0/3502449176 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb634003530 con 0x7fb634004d50 2024-04-17T10:41:29.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.819+0000 7fb6477fe640 1 --2- 172.21.15.83:0/3502449176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb634004d50 0x7fb6340051d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb630001750 tx=0x7fb63001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85eedd3d7244c134 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:29.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.819+0000 7fb645ffb640 1 -- 172.21.15.83:0/3502449176 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6300013a0 con 0x7fb634004d50 2024-04-17T10:41:29.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.819+0000 7fb645ffb640 1 -- 172.21.15.83:0/3502449176 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6300052f0 con 0x7fb634004d50 2024-04-17T10:41:29.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.819+0000 7fb645ffb640 1 -- 172.21.15.83:0/3502449176 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb630003840 con 0x7fb634004d50 2024-04-17T10:41:29.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.819+0000 7fb64d8c0640 1 -- 172.21.15.83:0/3502449176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb634004d50 msgr2=0x7fb6340051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:29.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.819+0000 7fb64d8c0640 1 --2- 172.21.15.83:0/3502449176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb634004d50 0x7fb6340051d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb630001750 tx=0x7fb63001c3e0 comp rx=0 tx=0).stop 2024-04-17T10:41:29.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.819+0000 7fb6477fe640 1 -- 172.21.15.83:0/3502449176 reap_dead start 2024-04-17T10:41:29.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.819+0000 7fb64d8c0640 1 -- 172.21.15.83:0/3502449176 shutdown_connections 2024-04-17T10:41:29.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.819+0000 7fb64d8c0640 1 -- 172.21.15.83:0/3502449176 >> 172.21.15.83:0/3502449176 conn(0x7fb6480d0600 msgr2=0x7fb6480d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:29.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.819+0000 7fb64d8c0640 1 -- 172.21.15.83:0/3502449176 shutdown_connections 2024-04-17T10:41:29.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.819+0000 7fb64d8c0640 1 -- 172.21.15.83:0/3502449176 wait complete. 2024-04-17T10:41:29.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.819+0000 7fb64d8c0640 1 Processor -- start 2024-04-17T10:41:29.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.819+0000 7fb64d8c0640 1 -- start start 2024-04-17T10:41:29.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.819+0000 7fb64d8c0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb634004d50 0x7fb64815e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:29.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.819+0000 7fb64d8c0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6480cb6a0 0x7fb6480cd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:29.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.819+0000 7fb64d8c0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6480cd660 0x7fb6480c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:29.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.819+0000 7fb64d8c0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb648163360 con 0x7fb6480cb6a0 2024-04-17T10:41:29.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.819+0000 7fb64d8c0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6481661a0 con 0x7fb6480cd660 2024-04-17T10:41:29.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.819+0000 7fb64d8c0640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6481634e0 con 0x7fb634004d50 2024-04-17T10:41:29.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.819+0000 7fb6467fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6480cb6a0 0x7fb6480cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:29.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.819+0000 7fb6467fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6480cb6a0 0x7fb6480cd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40504/0 (socket says 172.21.15.83:40504) 2024-04-17T10:41:29.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.819+0000 7fb6467fc640 1 -- 172.21.15.83:0/1699840896 learned_addr learned my addr 172.21.15.83:0/1699840896 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:29.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.823+0000 7fb646ffd640 1 --2- 172.21.15.83:0/1699840896 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb634004d50 0x7fb64815e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:29.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.823+0000 7fb6477fe640 1 --2- 172.21.15.83:0/1699840896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6480cd660 0x7fb6480c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:29.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.823+0000 7fb6467fc640 1 -- 172.21.15.83:0/1699840896 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb634004d50 msgr2=0x7fb64815e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:29.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.823+0000 7fb6467fc640 1 --2- 172.21.15.83:0/1699840896 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb634004d50 0x7fb64815e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:29.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.823+0000 7fb6467fc640 1 -- 172.21.15.83:0/1699840896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6480cd660 msgr2=0x7fb6480c74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:29.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.823+0000 7fb6467fc640 1 --2- 172.21.15.83:0/1699840896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6480cd660 0x7fb6480c74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:29.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.823+0000 7fb6467fc640 1 -- 172.21.15.83:0/1699840896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6480cc080 con 0x7fb6480cb6a0 2024-04-17T10:41:29.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.823+0000 7fb6467fc640 1 --2- 172.21.15.83:0/1699840896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6480cb6a0 0x7fb6480cd120 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb638001380 tx=0x7fb63800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:29.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.823+0000 7fb63ffff640 1 -- 172.21.15.83:0/1699840896 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb638019070 con 0x7fb6480cb6a0 2024-04-17T10:41:29.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.823+0000 7fb63ffff640 1 -- 172.21.15.83:0/1699840896 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb638004d70 con 0x7fb6480cb6a0 2024-04-17T10:41:29.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.823+0000 7fb64d8c0640 1 -- 172.21.15.83:0/1699840896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6480cc360 con 0x7fb6480cb6a0 2024-04-17T10:41:29.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.823+0000 7fb63ffff640 1 -- 172.21.15.83:0/1699840896 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6380072f0 con 0x7fb6480cb6a0 2024-04-17T10:41:29.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.823+0000 7fb64d8c0640 1 -- 172.21.15.83:0/1699840896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6480cc920 con 0x7fb6480cb6a0 2024-04-17T10:41:29.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.823+0000 7fb64d8c0640 1 -- 172.21.15.83:0/1699840896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6480681a0 con 0x7fb6480cb6a0 2024-04-17T10:41:29.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.823+0000 7fb63ffff640 1 -- 172.21.15.83:0/1699840896 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb63801c070 con 0x7fb6480cb6a0 2024-04-17T10:41:29.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.823+0000 7fb63ffff640 1 --2- 172.21.15.83:0/1699840896 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb628066f90 0x7fb628069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:29.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.823+0000 7fb63ffff640 1 -- 172.21.15.83:0/1699840896 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb638084e70 con 0x7fb6480cb6a0 2024-04-17T10:41:29.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.823+0000 7fb646ffd640 1 --2- 172.21.15.83:0/1699840896 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb628066f90 0x7fb628069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:29.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.823+0000 7fb646ffd640 1 --2- 172.21.15.83:0/1699840896 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb628066f90 0x7fb628069450 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb6481fb5a0 tx=0x7fb640008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:29.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:29.827+0000 7fb63ffff640 1 -- 172.21.15.83:0/1699840896 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb638056180 con 0x7fb6480cb6a0 2024-04-17T10:41:30.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.039+0000 7fb64d8c0640 1 -- 172.21.15.83:0/1699840896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) v1 -- 0x7fb6480b97c0 con 0x7fb6480cb6a0 2024-04-17T10:41:30.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.371+0000 7fb63ffff640 1 -- 172.21.15.83:0/1699840896 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v158) v1 ==== 148+0+0 (secure 0 0 0) 0x7fb638055a80 con 0x7fb6480cb6a0 2024-04-17T10:41:30.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.375+0000 7fb64d8c0640 1 -- 172.21.15.83:0/1699840896 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb628066f90 msgr2=0x7fb628069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:30.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.375+0000 7fb64d8c0640 1 --2- 172.21.15.83:0/1699840896 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb628066f90 0x7fb628069450 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb6481fb5a0 tx=0x7fb640008040 comp rx=0 tx=0).stop 2024-04-17T10:41:30.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.375+0000 7fb64d8c0640 1 -- 172.21.15.83:0/1699840896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6480cb6a0 msgr2=0x7fb6480cd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:30.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.375+0000 7fb64d8c0640 1 --2- 172.21.15.83:0/1699840896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6480cb6a0 0x7fb6480cd120 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb638001380 tx=0x7fb63800b4a0 comp rx=0 tx=0).stop 2024-04-17T10:41:30.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.375+0000 7fb64d8c0640 1 -- 172.21.15.83:0/1699840896 shutdown_connections 2024-04-17T10:41:30.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.375+0000 7fb64d8c0640 1 --2- 172.21.15.83:0/1699840896 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb628066f90 0x7fb628069450 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:30.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.375+0000 7fb64d8c0640 1 --2- 172.21.15.83:0/1699840896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6480cd660 0x7fb6480c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:30.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.375+0000 7fb64d8c0640 1 --2- 172.21.15.83:0/1699840896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6480cb6a0 0x7fb6480cd120 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:30.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.375+0000 7fb64d8c0640 1 --2- 172.21.15.83:0/1699840896 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb634004d50 0x7fb64815e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:30.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.375+0000 7fb64d8c0640 1 -- 172.21.15.83:0/1699840896 >> 172.21.15.83:0/1699840896 conn(0x7fb6480d0600 msgr2=0x7fb6480ceb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:30.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.375+0000 7fb64d8c0640 1 -- 172.21.15.83:0/1699840896 shutdown_connections 2024-04-17T10:41:30.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.375+0000 7fb64d8c0640 1 -- 172.21.15.83:0/1699840896 wait complete. 2024-04-17T10:41:30.394 INFO:tasks.cephfs_test_runner:test flag scrub_mdsdir ... ok 2024-04-17T10:41:30.394 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-17T10:41:30.395 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:41:30.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.519+0000 7f25261da640 1 Processor -- start 2024-04-17T10:41:30.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.519+0000 7f25261da640 1 -- start start 2024-04-17T10:41:30.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.523+0000 7f25261da640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f252015ba30 0x7f2520072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:30.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.523+0000 7f25261da640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f25200cfbe0 0x7f252006c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:30.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.523+0000 7f25261da640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f252006e320 con 0x7f252015c960 2024-04-17T10:41:30.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.523+0000 7f25261da640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2520071ea0 con 0x7f25200cfbe0 2024-04-17T10:41:30.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.523+0000 7f25261da640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f252006e040 con 0x7f252015ba30 2024-04-17T10:41:30.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.523+0000 7f251ffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f25200cfbe0 0x7f252006c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:30.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.523+0000 7f2516ffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f252015c960 0x7f25200cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53684/0 (socket says 172.21.15.83:53684) 2024-04-17T10:41:30.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.523+0000 7f2516ffd640 1 -- 172.21.15.83:0/1941290846 learned_addr learned my addr 172.21.15.83:0/1941290846 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:30.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.523+0000 7f251f7fe640 1 --2- 172.21.15.83:0/1941290846 >> v2:172.21.15.167:3300/0 conn(0x7f252015ba30 0x7f2520072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:30.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.523+0000 7f251ffff640 1 -- 172.21.15.83:0/1941290846 >> v2:172.21.15.167:3300/0 conn(0x7f252015ba30 msgr2=0x7f2520072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:30.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.523+0000 7f251ffff640 1 --2- 172.21.15.83:0/1941290846 >> v2:172.21.15.167:3300/0 conn(0x7f252015ba30 0x7f2520072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:30.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.523+0000 7f251ffff640 1 -- 172.21.15.83:0/1941290846 >> v1:172.21.15.83:6789/0 conn(0x7f252015c960 legacy=0x7f25200cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:30.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.523+0000 7f251ffff640 1 -- 172.21.15.83:0/1941290846 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2520071850 con 0x7f25200cfbe0 2024-04-17T10:41:30.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.523+0000 7f251f7fe640 1 --2- 172.21.15.83:0/1941290846 >> v2:172.21.15.167:3300/0 conn(0x7f252015ba30 0x7f2520072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:30.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.523+0000 7f251ffff640 1 --2- 172.21.15.83:0/1941290846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25200cfbe0 0x7f252006c490 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f25200b2410 tx=0x7f251801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa8bc30df13a5f2e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:30.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.523+0000 7f251effd640 1 -- 172.21.15.83:0/1941290846 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25180b2070 con 0x7f25200cfbe0 2024-04-17T10:41:30.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.523+0000 7f251effd640 1 -- 172.21.15.83:0/1941290846 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25180a57b0 con 0x7f25200cfbe0 2024-04-17T10:41:30.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.523+0000 7f251effd640 1 -- 172.21.15.83:0/1941290846 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25180a5b80 con 0x7f25200cfbe0 2024-04-17T10:41:30.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.527+0000 7f25261da640 1 -- 172.21.15.83:0/1941290846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25200cfbe0 msgr2=0x7f252006c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:30.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.527+0000 7f25261da640 1 --2- 172.21.15.83:0/1941290846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25200cfbe0 0x7f252006c490 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f25200b2410 tx=0x7f251801c8a0 comp rx=0 tx=0).stop 2024-04-17T10:41:30.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.527+0000 7f25261da640 1 -- 172.21.15.83:0/1941290846 shutdown_connections 2024-04-17T10:41:30.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.527+0000 7f25261da640 1 --2- 172.21.15.83:0/1941290846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25200cfbe0 0x7f252006c490 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:30.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.527+0000 7f25261da640 1 --2- 172.21.15.83:0/1941290846 >> v2:172.21.15.167:3300/0 conn(0x7f252015ba30 0x7f2520072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:30.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.527+0000 7f25261da640 1 -- 172.21.15.83:0/1941290846 >> 172.21.15.83:0/1941290846 conn(0x7f252006abe0 msgr2=0x7f252006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:30.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.527+0000 7f25261da640 1 -- 172.21.15.83:0/1941290846 shutdown_connections 2024-04-17T10:41:30.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.527+0000 7f25261da640 1 -- 172.21.15.83:0/1941290846 wait complete. 2024-04-17T10:41:30.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.527+0000 7f25261da640 1 Processor -- start 2024-04-17T10:41:30.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.527+0000 7f25261da640 1 -- start start 2024-04-17T10:41:30.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.527+0000 7f25261da640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25200cfbe0 0x7f252014bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:30.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.527+0000 7f25261da640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f252015ba30 0x7f252013a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:30.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.527+0000 7f25261da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f252015c960 0x7f2520149f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:30.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.527+0000 7f25261da640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252015cee0 con 0x7f252015c960 2024-04-17T10:41:30.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.527+0000 7f25261da640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252015cd60 con 0x7f25200cfbe0 2024-04-17T10:41:30.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.527+0000 7f25261da640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2520168520 con 0x7f252015ba30 2024-04-17T10:41:30.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.527+0000 7f251ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f252015c960 0x7f2520149f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:30.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.527+0000 7f251ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f252015c960 0x7f2520149f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40530/0 (socket says 172.21.15.83:40530) 2024-04-17T10:41:30.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.527+0000 7f251ffff640 1 -- 172.21.15.83:0/833378318 learned_addr learned my addr 172.21.15.83:0/833378318 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:30.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.527+0000 7f251ffff640 1 -- 172.21.15.83:0/833378318 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f252015ba30 msgr2=0x7f252013a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:30.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.527+0000 7f251f7fe640 1 --2- 172.21.15.83:0/833378318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25200cfbe0 0x7f252014bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:30.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.527+0000 7f251ffff640 1 --2- 172.21.15.83:0/833378318 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f252015ba30 0x7f252013a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:30.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.527+0000 7f251ffff640 1 -- 172.21.15.83:0/833378318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25200cfbe0 msgr2=0x7f252014bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:30.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.527+0000 7f251ffff640 1 --2- 172.21.15.83:0/833378318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25200cfbe0 0x7f252014bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:30.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.527+0000 7f251ffff640 1 -- 172.21.15.83:0/833378318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f252014aa80 con 0x7f252015c960 2024-04-17T10:41:30.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.527+0000 7f251ffff640 1 --2- 172.21.15.83:0/833378318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f252015c960 0x7f2520149f10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f25180194b0 tx=0x7f2518019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:30.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.527+0000 7f251cff9640 1 -- 172.21.15.83:0/833378318 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25180b2050 con 0x7f252015c960 2024-04-17T10:41:30.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.527+0000 7f251cff9640 1 -- 172.21.15.83:0/833378318 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25180be070 con 0x7f252015c960 2024-04-17T10:41:30.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.527+0000 7f25261da640 1 -- 172.21.15.83:0/833378318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f252014ad60 con 0x7f252015c960 2024-04-17T10:41:30.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.527+0000 7f251cff9640 1 -- 172.21.15.83:0/833378318 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25180adab0 con 0x7f252015c960 2024-04-17T10:41:30.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.527+0000 7f25261da640 1 -- 172.21.15.83:0/833378318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f252014b320 con 0x7f252015c960 2024-04-17T10:41:30.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.527+0000 7f25261da640 1 -- 172.21.15.83:0/833378318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2520074f90 con 0x7f252015c960 2024-04-17T10:41:30.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.531+0000 7f251cff9640 1 -- 172.21.15.83:0/833378318 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f25180adc50 con 0x7f252015c960 2024-04-17T10:41:30.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.531+0000 7f251cff9640 1 --2- 172.21.15.83:0/833378318 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2500066f60 0x7f2500069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:30.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.531+0000 7f251cff9640 1 -- 172.21.15.83:0/833378318 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f251811ffb0 con 0x7f252015c960 2024-04-17T10:41:30.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.531+0000 7f251f7fe640 1 --2- 172.21.15.83:0/833378318 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2500066f60 0x7f2500069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:30.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.531+0000 7f251f7fe640 1 --2- 172.21.15.83:0/833378318 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2500066f60 0x7f2500069420 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2510006fd0 tx=0x7f2510008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:30.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.535+0000 7f251cff9640 1 -- 172.21.15.83:0/833378318 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2518016520 con 0x7f252015c960 2024-04-17T10:41:30.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.695+0000 7f25261da640 1 -- 172.21.15.83:0/833378318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25200b1100 con 0x7f252015c960 2024-04-17T10:41:30.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.699+0000 7f251cff9640 1 -- 172.21.15.83:0/833378318 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+12948 (secure 0 0 0) 0x7f25180f12c0 con 0x7f252015c960 2024-04-17T10:41:30.695 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:41:30.696 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":38,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:41:29.362831+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/1181902025":"2024-04-18T10:41:26.029039+0000","172.21.15.83:6836/1181902025":"2024-04-18T10:41:26.029039+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:41:30.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.699+0000 7f25261da640 1 -- 172.21.15.83:0/833378318 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2500066f60 msgr2=0x7f2500069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:30.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.699+0000 7f25261da640 1 --2- 172.21.15.83:0/833378318 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2500066f60 0x7f2500069420 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2510006fd0 tx=0x7f2510008040 comp rx=0 tx=0).stop 2024-04-17T10:41:30.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.699+0000 7f25261da640 1 -- 172.21.15.83:0/833378318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f252015c960 msgr2=0x7f2520149f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:30.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.699+0000 7f25261da640 1 --2- 172.21.15.83:0/833378318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f252015c960 0x7f2520149f10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f25180194b0 tx=0x7f2518019640 comp rx=0 tx=0).stop 2024-04-17T10:41:30.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.699+0000 7f25261da640 1 -- 172.21.15.83:0/833378318 shutdown_connections 2024-04-17T10:41:30.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.699+0000 7f25261da640 1 --2- 172.21.15.83:0/833378318 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2500066f60 0x7f2500069420 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:30.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.699+0000 7f25261da640 1 --2- 172.21.15.83:0/833378318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f252015c960 0x7f2520149f10 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:30.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.699+0000 7f25261da640 1 --2- 172.21.15.83:0/833378318 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f252015ba30 0x7f252013a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:30.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.699+0000 7f25261da640 1 --2- 172.21.15.83:0/833378318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25200cfbe0 0x7f252014bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:30.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.699+0000 7f25261da640 1 -- 172.21.15.83:0/833378318 >> 172.21.15.83:0/833378318 conn(0x7f252006abe0 msgr2=0x7f252006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:30.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.699+0000 7f25261da640 1 -- 172.21.15.83:0/833378318 shutdown_connections 2024-04-17T10:41:30.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.699+0000 7f25261da640 1 -- 172.21.15.83:0/833378318 wait complete. 2024-04-17T10:41:30.715 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:41:30.716 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:41:30.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.839+0000 7fabf9c3a640 1 Processor -- start 2024-04-17T10:41:30.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.839+0000 7fabf9c3a640 1 -- start start 2024-04-17T10:41:30.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.839+0000 7fabf9c3a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fabf415e0f0 0x7fabf40b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:30.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.839+0000 7fabf9c3a640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fabf40ce400 0x7fabf4161c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:30.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.839+0000 7fabf9c3a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabf40b5450 0x7fabf40b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:30.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.839+0000 7fabf9c3a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabf4162b40 con 0x7fabf415e0f0 2024-04-17T10:41:30.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.839+0000 7fabf9c3a640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabf415e4d0 con 0x7fabf40ce400 2024-04-17T10:41:30.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.839+0000 7fabf9c3a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf4162580 con 0x7fabf40b5450 2024-04-17T10:41:30.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.839+0000 7fabf37fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fabf415e0f0 0x7fabf40b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:30.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.839+0000 7fabf37fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fabf415e0f0 0x7fabf40b6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40540/0 (socket says 172.21.15.83:40540) 2024-04-17T10:41:30.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.839+0000 7fabf37fe640 1 -- 172.21.15.83:0/2740927024 learned_addr learned my addr 172.21.15.83:0/2740927024 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:30.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.839+0000 7fabf2ffd640 1 --2- 172.21.15.83:0/2740927024 >> v2:172.21.15.167:3300/0 conn(0x7fabf40ce400 0x7fabf4161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:30.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.839+0000 7fabf3fff640 1 --2- 172.21.15.83:0/2740927024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabf40b5450 0x7fabf40b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:30.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.839+0000 7fabf37fe640 1 -- 172.21.15.83:0/2740927024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabf40b5450 msgr2=0x7fabf40b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:30.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.839+0000 7fabf37fe640 1 --2- 172.21.15.83:0/2740927024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabf40b5450 0x7fabf40b58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:30.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.839+0000 7fabf37fe640 1 -- 172.21.15.83:0/2740927024 >> v2:172.21.15.167:3300/0 conn(0x7fabf40ce400 msgr2=0x7fabf4161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:30.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.839+0000 7fabf37fe640 1 --2- 172.21.15.83:0/2740927024 >> v2:172.21.15.167:3300/0 conn(0x7fabf40ce400 0x7fabf4161c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:30.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.839+0000 7fabf37fe640 1 -- 172.21.15.83:0/2740927024 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf4162860 con 0x7fabf415e0f0 2024-04-17T10:41:30.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.843+0000 7fabf37fe640 1 --2- 172.21.15.83:0/2740927024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabf415e0f0 0x7fabf40b6d00 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fabf4068170 tx=0x7fabe401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34d5e550ba61c64 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:30.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.843+0000 7fabf27fc640 1 -- 172.21.15.83:0/2740927024 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe40b0070 con 0x7fabf415e0f0 2024-04-17T10:41:30.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.843+0000 7fabf27fc640 1 -- 172.21.15.83:0/2740927024 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe40ab600 con 0x7fabf415e0f0 2024-04-17T10:41:30.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.843+0000 7fabf27fc640 1 -- 172.21.15.83:0/2740927024 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe40ab9d0 con 0x7fabf415e0f0 2024-04-17T10:41:30.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.843+0000 7fabf9c3a640 1 -- 172.21.15.83:0/2740927024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabf415e0f0 msgr2=0x7fabf40b6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:30.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.843+0000 7fabf9c3a640 1 --2- 172.21.15.83:0/2740927024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabf415e0f0 0x7fabf40b6d00 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fabf4068170 tx=0x7fabe401c490 comp rx=0 tx=0).stop 2024-04-17T10:41:30.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.843+0000 7fabf9c3a640 1 -- 172.21.15.83:0/2740927024 shutdown_connections 2024-04-17T10:41:30.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.843+0000 7fabf9c3a640 1 --2- 172.21.15.83:0/2740927024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabf40b5450 0x7fabf40b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:30.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.843+0000 7fabf9c3a640 1 --2- 172.21.15.83:0/2740927024 >> v2:172.21.15.167:3300/0 conn(0x7fabf40ce400 0x7fabf4161c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:30.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.843+0000 7fabf9c3a640 1 --2- 172.21.15.83:0/2740927024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabf415e0f0 0x7fabf40b6d00 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:30.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.843+0000 7fabf9c3a640 1 -- 172.21.15.83:0/2740927024 >> 172.21.15.83:0/2740927024 conn(0x7fabf40d0450 msgr2=0x7fabf40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:30.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.843+0000 7fabf9c3a640 1 -- 172.21.15.83:0/2740927024 shutdown_connections 2024-04-17T10:41:30.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.843+0000 7fabf9c3a640 1 -- 172.21.15.83:0/2740927024 wait complete. 2024-04-17T10:41:30.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.843+0000 7fabf9c3a640 1 Processor -- start 2024-04-17T10:41:30.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.843+0000 7fabf9c3a640 1 -- start start 2024-04-17T10:41:30.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.843+0000 7fabf9c3a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabf40b5450 0x7fabf414cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:30.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.843+0000 7fabf9c3a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabf40ce400 0x7fabf4153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:30.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.843+0000 7fabf9c3a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabf415e0f0 0x7fabf414eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:30.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.843+0000 7fabf9c3a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf40d2d50 con 0x7fabf40b5450 2024-04-17T10:41:30.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.843+0000 7fabf9c3a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf4162360 con 0x7fabf40ce400 2024-04-17T10:41:30.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.843+0000 7fabf9c3a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf40d2ed0 con 0x7fabf415e0f0 2024-04-17T10:41:30.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.843+0000 7fabf37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabf40b5450 0x7fabf414cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:30.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.843+0000 7fabf37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabf40b5450 0x7fabf414cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40548/0 (socket says 172.21.15.83:40548) 2024-04-17T10:41:30.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.843+0000 7fabf37fe640 1 -- 172.21.15.83:0/3803054025 learned_addr learned my addr 172.21.15.83:0/3803054025 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:30.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.843+0000 7fabf37fe640 1 -- 172.21.15.83:0/3803054025 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabf415e0f0 msgr2=0x7fabf414eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:30.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.843+0000 7fabf2ffd640 1 --2- 172.21.15.83:0/3803054025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabf40ce400 0x7fabf4153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:30.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.843+0000 7fabf37fe640 1 --2- 172.21.15.83:0/3803054025 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabf415e0f0 0x7fabf414eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:30.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.843+0000 7fabf37fe640 1 -- 172.21.15.83:0/3803054025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabf40ce400 msgr2=0x7fabf4153750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:30.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.843+0000 7fabf37fe640 1 --2- 172.21.15.83:0/3803054025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabf40ce400 0x7fabf4153750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:30.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.843+0000 7fabf37fe640 1 -- 172.21.15.83:0/3803054025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf414bcf0 con 0x7fabf40b5450 2024-04-17T10:41:30.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.843+0000 7fabf2ffd640 1 --2- 172.21.15.83:0/3803054025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabf40ce400 0x7fabf4153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:30.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.843+0000 7fabf37fe640 1 --2- 172.21.15.83:0/3803054025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabf40b5450 0x7fabf414cd90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fabe40ae040 tx=0x7fabe4019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:30.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.847+0000 7fabdbfff640 1 -- 172.21.15.83:0/3803054025 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe40b0040 con 0x7fabf40b5450 2024-04-17T10:41:30.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.847+0000 7fabdbfff640 1 -- 172.21.15.83:0/3803054025 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe40b9070 con 0x7fabf40b5450 2024-04-17T10:41:30.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.847+0000 7fabf9c3a640 1 -- 172.21.15.83:0/3803054025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf414bfd0 con 0x7fabf40b5450 2024-04-17T10:41:30.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.847+0000 7fabdbfff640 1 -- 172.21.15.83:0/3803054025 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe40b46a0 con 0x7fabf40b5450 2024-04-17T10:41:30.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.847+0000 7fabf9c3a640 1 -- 172.21.15.83:0/3803054025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf414c590 con 0x7fabf40b5450 2024-04-17T10:41:30.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.847+0000 7fabf9c3a640 1 -- 172.21.15.83:0/3803054025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf40681a0 con 0x7fabf40b5450 2024-04-17T10:41:30.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.847+0000 7fabdbfff640 1 -- 172.21.15.83:0/3803054025 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fabe40b4840 con 0x7fabf40b5450 2024-04-17T10:41:30.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.847+0000 7fabdbfff640 1 --2- 172.21.15.83:0/3803054025 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fabe0066f60 0x7fabe0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:30.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.847+0000 7fabdbfff640 1 -- 172.21.15.83:0/3803054025 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fabe411e950 con 0x7fabf40b5450 2024-04-17T10:41:30.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.847+0000 7fabf2ffd640 1 --2- 172.21.15.83:0/3803054025 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fabe0066f60 0x7fabe0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:30.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.847+0000 7fabf2ffd640 1 --2- 172.21.15.83:0/3803054025 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fabe0066f60 0x7fabe0069420 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fabf414d370 tx=0x7fabe8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:30.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:30.851+0000 7fabdbfff640 1 -- 172.21.15.83:0/3803054025 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fabe4016110 con 0x7fabf40b5450 2024-04-17T10:41:31.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.011+0000 7fabf9c3a640 1 -- 172.21.15.83:0/3803054025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fabf414e630 con 0x7fabf40b5450 2024-04-17T10:41:31.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.011+0000 7fabdbfff640 1 -- 172.21.15.83:0/3803054025 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7fabe40efd80 con 0x7fabf40b5450 2024-04-17T10:41:31.008 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:41:31.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.011+0000 7fabf9c3a640 1 -- 172.21.15.83:0/3803054025 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fabe0066f60 msgr2=0x7fabe0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:31.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.011+0000 7fabf9c3a640 1 --2- 172.21.15.83:0/3803054025 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fabe0066f60 0x7fabe0069420 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fabf414d370 tx=0x7fabe8008040 comp rx=0 tx=0).stop 2024-04-17T10:41:31.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.011+0000 7fabf9c3a640 1 -- 172.21.15.83:0/3803054025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabf40b5450 msgr2=0x7fabf414cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:31.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.011+0000 7fabf9c3a640 1 --2- 172.21.15.83:0/3803054025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabf40b5450 0x7fabf414cd90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fabe40ae040 tx=0x7fabe4019230 comp rx=0 tx=0).stop 2024-04-17T10:41:31.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.011+0000 7fabf9c3a640 1 -- 172.21.15.83:0/3803054025 shutdown_connections 2024-04-17T10:41:31.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.011+0000 7fabf9c3a640 1 --2- 172.21.15.83:0/3803054025 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fabe0066f60 0x7fabe0069420 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:31.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.011+0000 7fabf9c3a640 1 --2- 172.21.15.83:0/3803054025 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabf415e0f0 0x7fabf414eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:31.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.011+0000 7fabf9c3a640 1 --2- 172.21.15.83:0/3803054025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabf40ce400 0x7fabf4153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:31.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.011+0000 7fabf9c3a640 1 --2- 172.21.15.83:0/3803054025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabf40b5450 0x7fabf414cd90 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:31.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.011+0000 7fabf9c3a640 1 -- 172.21.15.83:0/3803054025 >> 172.21.15.83:0/3803054025 conn(0x7fabf40d0450 msgr2=0x7fabf415e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:31.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.011+0000 7fabf9c3a640 1 -- 172.21.15.83:0/3803054025 shutdown_connections 2024-04-17T10:41:31.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.011+0000 7fabf9c3a640 1 -- 172.21.15.83:0/3803054025 wait complete. 2024-04-17T10:41:31.027 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks' 2024-04-17T10:41:31.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.151+0000 7fd8b8abe640 1 Processor -- start 2024-04-17T10:41:31.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.151+0000 7fd8b8abe640 1 -- start start 2024-04-17T10:41:31.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b8abe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd8b4072330 0x7fd8b40cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:31.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b8abe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8b40cfa10 0x7fd8b406c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:31.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b8abe640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b40cfea0 con 0x7fd8b415ba30 2024-04-17T10:41:31.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b8abe640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b4071ea0 con 0x7fd8b4072330 2024-04-17T10:41:31.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b8abe640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b406e320 con 0x7fd8b40cfa10 2024-04-17T10:41:31.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b259c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd8b415ba30 0x7fd8b415c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53688/0 (socket says 172.21.15.83:53688) 2024-04-17T10:41:31.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b259c640 1 -- 172.21.15.83:0/4241259996 learned_addr learned my addr 172.21.15.83:0/4241259996 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:31.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b2d9d640 1 --2- 172.21.15.83:0/4241259996 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8b40cfa10 0x7fd8b406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:31.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b1d9b640 1 --2- 172.21.15.83:0/4241259996 >> v2:172.21.15.167:3300/0 conn(0x7fd8b4072330 0x7fd8b40cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:31.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b159a640 1 -- 172.21.15.83:0/4241259996 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1332730369 0 0) 0x7fd8b40cfea0 con 0x7fd8b415ba30 2024-04-17T10:41:31.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b159a640 1 -- 172.21.15.83:0/4241259996 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd89c003530 con 0x7fd8b415ba30 2024-04-17T10:41:31.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b159a640 1 -- 172.21.15.83:0/4241259996 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1918646133 0 0) 0x7fd89c003530 con 0x7fd8b415ba30 2024-04-17T10:41:31.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b159a640 1 -- 172.21.15.83:0/4241259996 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8b40cfa10 msgr2=0x7fd8b406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:31.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b159a640 1 --2- 172.21.15.83:0/4241259996 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8b40cfa10 0x7fd8b406c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:31.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b159a640 1 -- 172.21.15.83:0/4241259996 >> v2:172.21.15.167:3300/0 conn(0x7fd8b4072330 msgr2=0x7fd8b40cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:31.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b159a640 1 --2- 172.21.15.83:0/4241259996 >> v2:172.21.15.167:3300/0 conn(0x7fd8b4072330 0x7fd8b40cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:31.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b159a640 1 -- 172.21.15.83:0/4241259996 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b406e040 con 0x7fd8b415ba30 2024-04-17T10:41:31.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b159a640 1 -- 172.21.15.83:0/4241259996 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fd8a4001680 con 0x7fd8b415ba30 2024-04-17T10:41:31.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b159a640 1 -- 172.21.15.83:0/4241259996 >> v1:172.21.15.83:6789/0 conn(0x7fd8b415ba30 legacy=0x7fd8b415c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:31.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b159a640 1 --2- 172.21.15.83:0/4241259996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd89c0039c0 0x7fd89c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:31.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b159a640 1 --2- 172.21.15.83:0/4241259996 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8b40cfa10 0x7fd89c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:31.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b159a640 1 --2- 172.21.15.83:0/4241259996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd89c004d50 0x7fd89c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:31.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b159a640 1 -- 172.21.15.83:0/4241259996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b4071ea0 con 0x7fd89c004d50 2024-04-17T10:41:31.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b159a640 1 -- 172.21.15.83:0/4241259996 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b406e320 con 0x7fd89c0039c0 2024-04-17T10:41:31.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b159a640 1 -- 172.21.15.83:0/4241259996 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c00ba00 con 0x7fd8b40cfa10 2024-04-17T10:41:31.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b2d9d640 1 --2- 172.21.15.83:0/4241259996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd89c004d50 0x7fd89c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:31.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b259c640 1 --2- 172.21.15.83:0/4241259996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd89c0039c0 0x7fd89c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:31.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b1d9b640 1 --2- 172.21.15.83:0/4241259996 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8b40cfa10 0x7fd89c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:31.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b2d9d640 1 -- 172.21.15.83:0/4241259996 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8b40cfa10 msgr2=0x7fd89c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:31.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b2d9d640 1 --2- 172.21.15.83:0/4241259996 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8b40cfa10 0x7fd89c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:31.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b2d9d640 1 -- 172.21.15.83:0/4241259996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd89c0039c0 msgr2=0x7fd89c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:31.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b2d9d640 1 --2- 172.21.15.83:0/4241259996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd89c0039c0 0x7fd89c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:31.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b2d9d640 1 -- 172.21.15.83:0/4241259996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd89c003530 con 0x7fd89c004d50 2024-04-17T10:41:31.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b259c640 1 --2- 172.21.15.83:0/4241259996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd89c0039c0 0x7fd89c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:41:31.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b2d9d640 1 --2- 172.21.15.83:0/4241259996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd89c004d50 0x7fd89c0051d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac009fc0 tx=0x7fd8ac01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c28fd00dd51664c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:31.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b159a640 1 -- 172.21.15.83:0/4241259996 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac0b2070 con 0x7fd89c004d50 2024-04-17T10:41:31.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b159a640 1 -- 172.21.15.83:0/4241259996 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ac0a5c90 con 0x7fd89c004d50 2024-04-17T10:41:31.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.155+0000 7fd8b159a640 1 -- 172.21.15.83:0/4241259996 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac009c10 con 0x7fd89c004d50 2024-04-17T10:41:31.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.159+0000 7fd8b8abe640 1 -- 172.21.15.83:0/4241259996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd89c004d50 msgr2=0x7fd89c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:31.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.159+0000 7fd8b8abe640 1 --2- 172.21.15.83:0/4241259996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd89c004d50 0x7fd89c0051d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac009fc0 tx=0x7fd8ac01cc80 comp rx=0 tx=0).stop 2024-04-17T10:41:31.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.159+0000 7fd8b2d9d640 1 -- 172.21.15.83:0/4241259996 reap_dead start 2024-04-17T10:41:31.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.159+0000 7fd8b8abe640 1 -- 172.21.15.83:0/4241259996 shutdown_connections 2024-04-17T10:41:31.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.159+0000 7fd8b8abe640 1 -- 172.21.15.83:0/4241259996 >> 172.21.15.83:0/4241259996 conn(0x7fd8b406abe0 msgr2=0x7fd8b406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:31.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.159+0000 7fd8b8abe640 1 -- 172.21.15.83:0/4241259996 shutdown_connections 2024-04-17T10:41:31.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.159+0000 7fd8b8abe640 1 -- 172.21.15.83:0/4241259996 wait complete. 2024-04-17T10:41:31.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.159+0000 7fd8b8abe640 1 Processor -- start 2024-04-17T10:41:31.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.159+0000 7fd8b8abe640 1 -- start start 2024-04-17T10:41:31.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.159+0000 7fd8b8abe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8b40b1090 0x7fd8b414de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:31.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.159+0000 7fd8b8abe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8b40b1470 0x7fd8b414e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:31.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.159+0000 7fd8b8abe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8b40aac30 0x7fd8b40ab090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:31.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.159+0000 7fd8b8abe640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b40aeae0 con 0x7fd8b40b1470 2024-04-17T10:41:31.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.159+0000 7fd8b8abe640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b4071b30 con 0x7fd8b40aac30 2024-04-17T10:41:31.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.159+0000 7fd8b8abe640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b406dca0 con 0x7fd8b40b1090 2024-04-17T10:41:31.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.159+0000 7fd8b1d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8b40b1470 0x7fd8b414e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:31.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.159+0000 7fd8b1d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8b40b1470 0x7fd8b414e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40562/0 (socket says 172.21.15.83:40562) 2024-04-17T10:41:31.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.159+0000 7fd8b1d9b640 1 -- 172.21.15.83:0/1028272711 learned_addr learned my addr 172.21.15.83:0/1028272711 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:31.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.159+0000 7fd8b2d9d640 1 --2- 172.21.15.83:0/1028272711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8b40aac30 0x7fd8b40ab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:31.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.159+0000 7fd8b259c640 1 --2- 172.21.15.83:0/1028272711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8b40b1090 0x7fd8b414de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:31.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.159+0000 7fd8b1d9b640 1 -- 172.21.15.83:0/1028272711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8b40b1090 msgr2=0x7fd8b414de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:31.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.159+0000 7fd8b1d9b640 1 --2- 172.21.15.83:0/1028272711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8b40b1090 0x7fd8b414de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:31.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.159+0000 7fd8b1d9b640 1 -- 172.21.15.83:0/1028272711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8b40aac30 msgr2=0x7fd8b40ab090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:31.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.159+0000 7fd8b1d9b640 1 --2- 172.21.15.83:0/1028272711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8b40aac30 0x7fd8b40ab090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:31.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.159+0000 7fd8b1d9b640 1 -- 172.21.15.83:0/1028272711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b40ab920 con 0x7fd8b40b1470 2024-04-17T10:41:31.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.159+0000 7fd8b2d9d640 1 --2- 172.21.15.83:0/1028272711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8b40aac30 0x7fd8b40ab090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:41:31.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.159+0000 7fd8b1d9b640 1 --2- 172.21.15.83:0/1028272711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8b40b1470 0x7fd8b414e3c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8011690 tx=0x7fd8a8011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:31.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.159+0000 7fd8a2ffd640 1 -- 172.21.15.83:0/1028272711 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a800f9c0 con 0x7fd8b40b1470 2024-04-17T10:41:31.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.159+0000 7fd8a2ffd640 1 -- 172.21.15.83:0/1028272711 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a800fb60 con 0x7fd8b40b1470 2024-04-17T10:41:31.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.159+0000 7fd8b8abe640 1 -- 172.21.15.83:0/1028272711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b40abc00 con 0x7fd8b40b1470 2024-04-17T10:41:31.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.159+0000 7fd8a2ffd640 1 -- 172.21.15.83:0/1028272711 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8015460 con 0x7fd8b40b1470 2024-04-17T10:41:31.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.159+0000 7fd8b8abe640 1 -- 172.21.15.83:0/1028272711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b40ac1c0 con 0x7fd8b40b1470 2024-04-17T10:41:31.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.163+0000 7fd8b8abe640 1 -- 172.21.15.83:0/1028272711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b4074f90 con 0x7fd8b40b1470 2024-04-17T10:41:31.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.163+0000 7fd8a2ffd640 1 -- 172.21.15.83:0/1028272711 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd8a8015600 con 0x7fd8b40b1470 2024-04-17T10:41:31.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.163+0000 7fd8a2ffd640 1 --2- 172.21.15.83:0/1028272711 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd894066f60 0x7fd894069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:31.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.163+0000 7fd8a2ffd640 1 -- 172.21.15.83:0/1028272711 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd8a8085720 con 0x7fd8b40b1470 2024-04-17T10:41:31.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.163+0000 7fd8b259c640 1 --2- 172.21.15.83:0/1028272711 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd894066f60 0x7fd894069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:31.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.163+0000 7fd8b259c640 1 --2- 172.21.15.83:0/1028272711 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd894066f60 0x7fd894069420 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4000c10 tx=0x7fd8a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:31.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.167+0000 7fd8a2ffd640 1 -- 172.21.15.83:0/1028272711 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd8a8019030 con 0x7fd8b40b1470 2024-04-17T10:41:31.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:31.379+0000 7fd8b8abe640 1 -- 172.21.15.83:0/1028272711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]} v 0) v1 -- 0x7fd8b40cdaa0 con 0x7fd8b40b1470 2024-04-17T10:41:32.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.375+0000 7fd8a2ffd640 1 -- 172.21.15.83:0/1028272711 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]}]=0 v160) v1 ==== 146+0+0 (secure 0 0 0) 0x7fd8a8056a30 con 0x7fd8b40b1470 2024-04-17T10:41:32.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.379+0000 7fd8b8abe640 1 -- 172.21.15.83:0/1028272711 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd894066f60 msgr2=0x7fd894069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:32.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.379+0000 7fd8b8abe640 1 --2- 172.21.15.83:0/1028272711 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd894066f60 0x7fd894069420 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4000c10 tx=0x7fd8a4009040 comp rx=0 tx=0).stop 2024-04-17T10:41:32.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.379+0000 7fd8b8abe640 1 -- 172.21.15.83:0/1028272711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8b40b1470 msgr2=0x7fd8b414e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:32.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.379+0000 7fd8b8abe640 1 --2- 172.21.15.83:0/1028272711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8b40b1470 0x7fd8b414e3c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8011690 tx=0x7fd8a8011b50 comp rx=0 tx=0).stop 2024-04-17T10:41:32.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.379+0000 7fd8b8abe640 1 -- 172.21.15.83:0/1028272711 shutdown_connections 2024-04-17T10:41:32.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.379+0000 7fd8b8abe640 1 --2- 172.21.15.83:0/1028272711 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd894066f60 0x7fd894069420 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:32.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.379+0000 7fd8b8abe640 1 --2- 172.21.15.83:0/1028272711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8b40aac30 0x7fd8b40ab090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:32.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.379+0000 7fd8b8abe640 1 --2- 172.21.15.83:0/1028272711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8b40b1470 0x7fd8b414e3c0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:32.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.379+0000 7fd8b8abe640 1 --2- 172.21.15.83:0/1028272711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8b40b1090 0x7fd8b414de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:32.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.379+0000 7fd8b8abe640 1 -- 172.21.15.83:0/1028272711 >> 172.21.15.83:0/1028272711 conn(0x7fd8b406abe0 msgr2=0x7fd8b40ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:32.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.379+0000 7fd8b8abe640 1 -- 172.21.15.83:0/1028272711 shutdown_connections 2024-04-17T10:41:32.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.379+0000 7fd8b8abe640 1 -- 172.21.15.83:0/1028272711 wait complete. 2024-04-17T10:41:32.400 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-17T10:41:32.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99bd72d640 1 Processor -- start 2024-04-17T10:41:32.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99bd72d640 1 -- start start 2024-04-17T10:41:32.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99bd72d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f99b80ce400 0x7f99b80b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:32.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99bd72d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99b80b5450 0x7f99b80b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:32.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99bd72d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99b815e4d0 con 0x7f99b815e0f0 2024-04-17T10:41:32.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99bd72d640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99b8162b40 con 0x7f99b80ce400 2024-04-17T10:41:32.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99bd72d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99b8162320 con 0x7f99b80b5450 2024-04-17T10:41:32.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99b6ffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f99b815e0f0 0x7f99b8161c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53698/0 (socket says 172.21.15.83:53698) 2024-04-17T10:41:32.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99b6ffd640 1 -- 172.21.15.83:0/3341036832 learned_addr learned my addr 172.21.15.83:0/3341036832 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:32.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99affff640 1 --2- 172.21.15.83:0/3341036832 >> v2:172.21.15.167:3300/0 conn(0x7f99b80ce400 0x7f99b80b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:32.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99b77fe640 1 --2- 172.21.15.83:0/3341036832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99b80b5450 0x7f99b80b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:32.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99b67fc640 1 -- 172.21.15.83:0/3341036832 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4259234665 0 0) 0x7f99b815e4d0 con 0x7f99b815e0f0 2024-04-17T10:41:32.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99b67fc640 1 -- 172.21.15.83:0/3341036832 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99a4003530 con 0x7f99b815e0f0 2024-04-17T10:41:32.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99b67fc640 1 -- 172.21.15.83:0/3341036832 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2503603405 0 0) 0x7f99a4003530 con 0x7f99b815e0f0 2024-04-17T10:41:32.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99b67fc640 1 -- 172.21.15.83:0/3341036832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99b80b5450 msgr2=0x7f99b80b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:32.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99b67fc640 1 --2- 172.21.15.83:0/3341036832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99b80b5450 0x7f99b80b58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:32.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99b67fc640 1 -- 172.21.15.83:0/3341036832 >> v2:172.21.15.167:3300/0 conn(0x7f99b80ce400 msgr2=0x7f99b80b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:32.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99b67fc640 1 --2- 172.21.15.83:0/3341036832 >> v2:172.21.15.167:3300/0 conn(0x7f99b80ce400 0x7f99b80b6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:32.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99b67fc640 1 -- 172.21.15.83:0/3341036832 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b8162580 con 0x7f99b815e0f0 2024-04-17T10:41:32.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99b67fc640 1 -- 172.21.15.83:0/3341036832 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f99a8001680 con 0x7f99b815e0f0 2024-04-17T10:41:32.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99b67fc640 1 -- 172.21.15.83:0/3341036832 >> v1:172.21.15.83:6789/0 conn(0x7f99b815e0f0 legacy=0x7f99b8161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:32.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99b67fc640 1 --2- 172.21.15.83:0/3341036832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99a4003a60 0x7f99a4003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:32.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99b67fc640 1 --2- 172.21.15.83:0/3341036832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99b80b5450 0x7f99a4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:32.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99b67fc640 1 --2- 172.21.15.83:0/3341036832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99a4004d50 0x7f99a40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:32.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99b67fc640 1 -- 172.21.15.83:0/3341036832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b8162b40 con 0x7f99a4003a60 2024-04-17T10:41:32.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99b67fc640 1 -- 172.21.15.83:0/3341036832 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b8162320 con 0x7f99a4004d50 2024-04-17T10:41:32.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99b67fc640 1 -- 172.21.15.83:0/3341036832 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a400ba80 con 0x7f99b80b5450 2024-04-17T10:41:32.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99b6ffd640 1 --2- 172.21.15.83:0/3341036832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99a4003a60 0x7f99a4003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:32.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99b77fe640 1 --2- 172.21.15.83:0/3341036832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99a4004d50 0x7f99a40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:32.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99affff640 1 --2- 172.21.15.83:0/3341036832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99b80b5450 0x7f99a4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:32.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99b6ffd640 1 -- 172.21.15.83:0/3341036832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99b80b5450 msgr2=0x7f99a4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:32.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99b6ffd640 1 --2- 172.21.15.83:0/3341036832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99b80b5450 0x7f99a4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:32.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99b6ffd640 1 -- 172.21.15.83:0/3341036832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99a4004d50 msgr2=0x7f99a40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:32.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99b6ffd640 1 --2- 172.21.15.83:0/3341036832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99a4004d50 0x7f99a40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:32.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.531+0000 7f99b6ffd640 1 -- 172.21.15.83:0/3341036832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a4003530 con 0x7f99a4003a60 2024-04-17T10:41:32.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.539+0000 7f99b6ffd640 1 --2- 172.21.15.83:0/3341036832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99a4003a60 0x7f99a4003e80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f99a800c210 tx=0x7f99a801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8341625ae63d0fd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:32.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.539+0000 7f99b67fc640 1 -- 172.21.15.83:0/3341036832 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a8007d40 con 0x7f99a4003a60 2024-04-17T10:41:32.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.539+0000 7f99b67fc640 1 -- 172.21.15.83:0/3341036832 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a80a9070 con 0x7f99a4003a60 2024-04-17T10:41:32.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.539+0000 7f99b67fc640 1 -- 172.21.15.83:0/3341036832 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a8005d40 con 0x7f99a4003a60 2024-04-17T10:41:32.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.539+0000 7f99bd72d640 1 -- 172.21.15.83:0/3341036832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99a4003a60 msgr2=0x7f99a4003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:32.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.539+0000 7f99bd72d640 1 --2- 172.21.15.83:0/3341036832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99a4003a60 0x7f99a4003e80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f99a800c210 tx=0x7f99a801cca0 comp rx=0 tx=0).stop 2024-04-17T10:41:32.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.539+0000 7f99b77fe640 1 -- 172.21.15.83:0/3341036832 reap_dead start 2024-04-17T10:41:32.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.539+0000 7f99bd72d640 1 -- 172.21.15.83:0/3341036832 shutdown_connections 2024-04-17T10:41:32.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.539+0000 7f99bd72d640 1 -- 172.21.15.83:0/3341036832 >> 172.21.15.83:0/3341036832 conn(0x7f99b80d0450 msgr2=0x7f99b80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:32.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.539+0000 7f99bd72d640 1 -- 172.21.15.83:0/3341036832 shutdown_connections 2024-04-17T10:41:32.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.539+0000 7f99bd72d640 1 -- 172.21.15.83:0/3341036832 wait complete. 2024-04-17T10:41:32.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.539+0000 7f99bd72d640 1 Processor -- start 2024-04-17T10:41:32.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.539+0000 7f99bd72d640 1 -- start start 2024-04-17T10:41:32.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.539+0000 7f99bd72d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99b80c8e10 0x7f99b815d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:32.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.539+0000 7f99bd72d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99b80cd470 0x7f99b80cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:32.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.539+0000 7f99bd72d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99b80ccc10 0x7f99b80b7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:32.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.539+0000 7f99bd72d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b80b5c70 con 0x7f99b80ccc10 2024-04-17T10:41:32.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.539+0000 7f99bd72d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b80d2d50 con 0x7f99b80cd470 2024-04-17T10:41:32.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.539+0000 7f99bd72d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99b80cc300 con 0x7f99b80c8e10 2024-04-17T10:41:32.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.539+0000 7f99b77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99b80ccc10 0x7f99b80b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:32.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.539+0000 7f99b77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99b80ccc10 0x7f99b80b7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40574/0 (socket says 172.21.15.83:40574) 2024-04-17T10:41:32.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.539+0000 7f99b77fe640 1 -- 172.21.15.83:0/1330571950 learned_addr learned my addr 172.21.15.83:0/1330571950 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:32.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.539+0000 7f99b6ffd640 1 --2- 172.21.15.83:0/1330571950 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99b80c8e10 0x7f99b815d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:32.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.539+0000 7f99affff640 1 --2- 172.21.15.83:0/1330571950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99b80cd470 0x7f99b80cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:32.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.539+0000 7f99b77fe640 1 -- 172.21.15.83:0/1330571950 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99b80c8e10 msgr2=0x7f99b815d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:32.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.539+0000 7f99b77fe640 1 --2- 172.21.15.83:0/1330571950 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99b80c8e10 0x7f99b815d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:32.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.539+0000 7f99b77fe640 1 -- 172.21.15.83:0/1330571950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99b80cd470 msgr2=0x7f99b80cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:32.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.539+0000 7f99b77fe640 1 --2- 172.21.15.83:0/1330571950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99b80cd470 0x7f99b80cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:32.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.539+0000 7f99b77fe640 1 -- 172.21.15.83:0/1330571950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b80cbd40 con 0x7f99b80ccc10 2024-04-17T10:41:32.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.539+0000 7f99affff640 1 --2- 172.21.15.83:0/1330571950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99b80cd470 0x7f99b80cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:41:32.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.539+0000 7f99b6ffd640 1 --2- 172.21.15.83:0/1330571950 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99b80c8e10 0x7f99b815d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:41:32.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.539+0000 7f99b77fe640 1 --2- 172.21.15.83:0/1330571950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99b80ccc10 0x7f99b80b7380 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f99b808c650 tx=0x7f99b000f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:32.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.543+0000 7f99af7fe640 1 -- 172.21.15.83:0/1330571950 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b000f8d0 con 0x7f99b80ccc10 2024-04-17T10:41:32.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.543+0000 7f99af7fe640 1 -- 172.21.15.83:0/1330571950 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b001c070 con 0x7f99b80ccc10 2024-04-17T10:41:32.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.543+0000 7f99bd72d640 1 -- 172.21.15.83:0/1330571950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99b80cc020 con 0x7f99b80ccc10 2024-04-17T10:41:32.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.543+0000 7f99af7fe640 1 -- 172.21.15.83:0/1330571950 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b000fcb0 con 0x7f99b80ccc10 2024-04-17T10:41:32.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.543+0000 7f99bd72d640 1 -- 172.21.15.83:0/1330571950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b815ccc0 con 0x7f99b80ccc10 2024-04-17T10:41:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.543+0000 7f99bd72d640 1 -- 172.21.15.83:0/1330571950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99b8074f90 con 0x7f99b80ccc10 2024-04-17T10:41:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.543+0000 7f99af7fe640 1 -- 172.21.15.83:0/1330571950 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f99b0004030 con 0x7f99b80ccc10 2024-04-17T10:41:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.543+0000 7f99af7fe640 1 --2- 172.21.15.83:0/1330571950 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9998066f90 0x7f9998069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.543+0000 7f99af7fe640 1 -- 172.21.15.83:0/1330571950 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f99b0085530 con 0x7f99b80ccc10 2024-04-17T10:41:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.543+0000 7f99b6ffd640 1 --2- 172.21.15.83:0/1330571950 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9998066f90 0x7f9998069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.543+0000 7f99b6ffd640 1 --2- 172.21.15.83:0/1330571950 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9998066f90 0x7f9998069450 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f99a80ab000 tx=0x7f99a80b0040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:32.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.547+0000 7f99af7fe640 1 -- 172.21.15.83:0/1330571950 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f99b0010040 con 0x7f99b80ccc10 2024-04-17T10:41:32.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.703+0000 7f99bd72d640 1 -- 172.21.15.83:0/1330571950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f99b80d0ae0 con 0x7f99b80ccc10 2024-04-17T10:41:32.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.711+0000 7f99af7fe640 1 -- 172.21.15.83:0/1330571950 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f99b00568d0 con 0x7f99b80ccc10 2024-04-17T10:41:32.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.711+0000 7f99bd72d640 1 -- 172.21.15.83:0/1330571950 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9998066f90 msgr2=0x7f9998069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:32.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.711+0000 7f99bd72d640 1 --2- 172.21.15.83:0/1330571950 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9998066f90 0x7f9998069450 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f99a80ab000 tx=0x7f99a80b0040 comp rx=0 tx=0).stop 2024-04-17T10:41:32.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.711+0000 7f99bd72d640 1 -- 172.21.15.83:0/1330571950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99b80ccc10 msgr2=0x7f99b80b7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:32.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.711+0000 7f99bd72d640 1 --2- 172.21.15.83:0/1330571950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99b80ccc10 0x7f99b80b7380 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f99b808c650 tx=0x7f99b000f1b0 comp rx=0 tx=0).stop 2024-04-17T10:41:32.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.711+0000 7f99bd72d640 1 -- 172.21.15.83:0/1330571950 shutdown_connections 2024-04-17T10:41:32.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.711+0000 7f99bd72d640 1 --2- 172.21.15.83:0/1330571950 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9998066f90 0x7f9998069450 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:32.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.711+0000 7f99bd72d640 1 --2- 172.21.15.83:0/1330571950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99b80ccc10 0x7f99b80b7380 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:32.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.711+0000 7f99bd72d640 1 --2- 172.21.15.83:0/1330571950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99b80cd470 0x7f99b80cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:32.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.711+0000 7f99bd72d640 1 --2- 172.21.15.83:0/1330571950 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99b80c8e10 0x7f99b815d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:32.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.711+0000 7f99bd72d640 1 -- 172.21.15.83:0/1330571950 >> 172.21.15.83:0/1330571950 conn(0x7f99b80d0450 msgr2=0x7f99b815f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:32.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.711+0000 7f99bd72d640 1 -- 172.21.15.83:0/1330571950 shutdown_connections 2024-04-17T10:41:32.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:32.711+0000 7f99bd72d640 1 -- 172.21.15.83:0/1330571950 wait complete. 2024-04-17T10:41:32.726 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T10:41:32.727 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:41:32.756 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:41:32.815 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T10:41:32.816 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:41:32.816 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:41:32.821 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16063 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:41:32.821 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:41:32.821 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:41:32.827 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:41:32.827 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:41:32.827 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:41:32.868 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16063 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:41:32.868 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:41:32.868 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:41:32.874 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:41:32.874 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-17T10:41:32.874 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:41:33.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.035+0000 7efe27eb0640 1 Processor -- start 2024-04-17T10:41:33.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.035+0000 7efe27eb0640 1 -- start start 2024-04-17T10:41:33.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.035+0000 7efe27eb0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe200b2d80 0x7efe20146e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:33.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.035+0000 7efe27eb0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efe2006d690 0x7efe201620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:33.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.035+0000 7efe27eb0640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe2006db10 con 0x7efe20146a00 2024-04-17T10:41:33.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.035+0000 7efe27eb0640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe20162610 con 0x7efe2006d690 2024-04-17T10:41:33.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.035+0000 7efe27eb0640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe2006c960 con 0x7efe200b2d80 2024-04-17T10:41:33.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.035+0000 7efe2644d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efe2006d690 0x7efe201620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:33.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.035+0000 7efe2644d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efe2006d690 0x7efe201620d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40580/0 (socket says 172.21.15.83:40580) 2024-04-17T10:41:33.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.035+0000 7efe25c4c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7efe20146a00 0x7efe200b2670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53702/0 (socket says 172.21.15.83:53702) 2024-04-17T10:41:33.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.035+0000 7efe2644d640 1 -- 172.21.15.83:0/2157303115 learned_addr learned my addr 172.21.15.83:0/2157303115 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:33.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.035+0000 7efe2544b640 1 --2- 172.21.15.83:0/2157303115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe200b2d80 0x7efe20146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:33.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.035+0000 7efe2644d640 1 -- 172.21.15.83:0/2157303115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe200b2d80 msgr2=0x7efe20146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:33.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.035+0000 7efe2644d640 1 --2- 172.21.15.83:0/2157303115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe200b2d80 0x7efe20146e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:33.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.035+0000 7efe2644d640 1 -- 172.21.15.83:0/2157303115 >> v1:172.21.15.83:6789/0 conn(0x7efe20146a00 legacy=0x7efe200b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:33.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.035+0000 7efe2644d640 1 -- 172.21.15.83:0/2157303115 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe20147570 con 0x7efe2006d690 2024-04-17T10:41:33.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.039+0000 7efe2644d640 1 --2- 172.21.15.83:0/2157303115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe2006d690 0x7efe201620d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7efe18009870 tx=0x7efe1801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f56b7db6f60ce5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:33.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.039+0000 7efe24c4a640 1 -- 172.21.15.83:0/2157303115 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe180b0070 con 0x7efe2006d690 2024-04-17T10:41:33.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.039+0000 7efe24c4a640 1 -- 172.21.15.83:0/2157303115 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe180ab600 con 0x7efe2006d690 2024-04-17T10:41:33.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.039+0000 7efe24c4a640 1 -- 172.21.15.83:0/2157303115 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe180ab9d0 con 0x7efe2006d690 2024-04-17T10:41:33.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.039+0000 7efe27eb0640 1 -- 172.21.15.83:0/2157303115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe2006d690 msgr2=0x7efe201620d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:33.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.039+0000 7efe27eb0640 1 --2- 172.21.15.83:0/2157303115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe2006d690 0x7efe201620d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7efe18009870 tx=0x7efe1801c3f0 comp rx=0 tx=0).stop 2024-04-17T10:41:33.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.039+0000 7efe27eb0640 1 -- 172.21.15.83:0/2157303115 shutdown_connections 2024-04-17T10:41:33.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.039+0000 7efe27eb0640 1 --2- 172.21.15.83:0/2157303115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe2006d690 0x7efe201620d0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:33.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.039+0000 7efe27eb0640 1 --2- 172.21.15.83:0/2157303115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe200b2d80 0x7efe20146e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:33.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.039+0000 7efe27eb0640 1 -- 172.21.15.83:0/2157303115 >> 172.21.15.83:0/2157303115 conn(0x7efe2006abe0 msgr2=0x7efe2006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:33.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.039+0000 7efe27eb0640 1 -- 172.21.15.83:0/2157303115 shutdown_connections 2024-04-17T10:41:33.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.039+0000 7efe27eb0640 1 -- 172.21.15.83:0/2157303115 wait complete. 2024-04-17T10:41:33.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.039+0000 7efe27eb0640 1 Processor -- start 2024-04-17T10:41:33.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.039+0000 7efe27eb0640 1 -- start start 2024-04-17T10:41:33.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.043+0000 7efe27eb0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe2006d690 0x7efe2013af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:33.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.043+0000 7efe27eb0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe200b2d80 0x7efe201f99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.043+0000 7efe27eb0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe20146a00 0x7efe200cf930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.043+0000 7efe27eb0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe20071010 con 0x7efe2006d690 2024-04-17T10:41:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.043+0000 7efe27eb0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe20070e90 con 0x7efe200b2d80 2024-04-17T10:41:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.043+0000 7efe27eb0640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe201f9560 con 0x7efe20146a00 2024-04-17T10:41:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.043+0000 7efe25c4c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe2006d690 0x7efe2013af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.043+0000 7efe25c4c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe2006d690 0x7efe2013af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40592/0 (socket says 172.21.15.83:40592) 2024-04-17T10:41:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.043+0000 7efe25c4c640 1 -- 172.21.15.83:0/1510645895 learned_addr learned my addr 172.21.15.83:0/1510645895 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.043+0000 7efe25c4c640 1 -- 172.21.15.83:0/1510645895 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe20146a00 msgr2=0x7efe200cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.043+0000 7efe25c4c640 1 --2- 172.21.15.83:0/1510645895 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe20146a00 0x7efe200cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.043+0000 7efe25c4c640 1 -- 172.21.15.83:0/1510645895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe200b2d80 msgr2=0x7efe201f99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.043+0000 7efe25c4c640 1 --2- 172.21.15.83:0/1510645895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe200b2d80 0x7efe201f99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.043+0000 7efe25c4c640 1 -- 172.21.15.83:0/1510645895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe201f8950 con 0x7efe2006d690 2024-04-17T10:41:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.043+0000 7efe25c4c640 1 --2- 172.21.15.83:0/1510645895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe2006d690 0x7efe2013af40 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7efe1c00caf0 tx=0x7efe1c0020b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.043+0000 7efe167fc640 1 -- 172.21.15.83:0/1510645895 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1c018070 con 0x7efe2006d690 2024-04-17T10:41:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.043+0000 7efe167fc640 1 -- 172.21.15.83:0/1510645895 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe1c00a700 con 0x7efe2006d690 2024-04-17T10:41:33.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.043+0000 7efe27eb0640 1 -- 172.21.15.83:0/1510645895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe201f8c30 con 0x7efe2006d690 2024-04-17T10:41:33.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.043+0000 7efe167fc640 1 -- 172.21.15.83:0/1510645895 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1c00ab10 con 0x7efe2006d690 2024-04-17T10:41:33.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.043+0000 7efe27eb0640 1 -- 172.21.15.83:0/1510645895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe201f91f0 con 0x7efe2006d690 2024-04-17T10:41:33.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.043+0000 7efe27eb0640 1 -- 172.21.15.83:0/1510645895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe20074f90 con 0x7efe2006d690 2024-04-17T10:41:33.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.043+0000 7efe167fc640 1 -- 172.21.15.83:0/1510645895 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efe1c01b070 con 0x7efe2006d690 2024-04-17T10:41:33.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.043+0000 7efe167fc640 1 --2- 172.21.15.83:0/1510645895 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efe04066f90 0x7efe04069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:33.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.043+0000 7efe167fc640 1 -- 172.21.15.83:0/1510645895 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7efe1c084510 con 0x7efe2006d690 2024-04-17T10:41:33.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.043+0000 7efe2544b640 1 --2- 172.21.15.83:0/1510645895 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efe04066f90 0x7efe04069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:33.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.043+0000 7efe2544b640 1 --2- 172.21.15.83:0/1510645895 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efe04066f90 0x7efe04069450 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efe2015db70 tx=0x7efe10008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:33.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.047+0000 7efe167fc640 1 -- 172.21.15.83:0/1510645895 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efe1c055820 con 0x7efe2006d690 2024-04-17T10:41:33.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.231+0000 7efe27eb0640 1 -- 172.21.15.83:0/1510645895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe2015dd50 con 0x7efe2006d690 2024-04-17T10:41:33.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.235+0000 7efe167fc640 1 -- 172.21.15.83:0/1510645895 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+3702 (secure 0 0 0) 0x7efe1c055120 con 0x7efe2006d690 2024-04-17T10:41:33.232 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:41:33.232 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":16,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1456029744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6833","nonce":1456029744},{"type":"v1","addr":"172.21.15.167:6835","nonce":1456029744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6834/648555908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":648555908},{"type":"v1","addr":"172.21.15.167:6834","nonce":648555908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5163,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1435807773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1435807773},{"type":"v1","addr":"172.21.15.83:6835","nonce":1435807773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2091025524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":2091025524},{"type":"v1","addr":"172.21.15.83:6837","nonce":2091025524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":16}],"filesystems":[]} 2024-04-17T10:41:33.232 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 16 2024-04-17T10:41:33.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.235+0000 7efe27eb0640 1 -- 172.21.15.83:0/1510645895 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efe04066f90 msgr2=0x7efe04069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:33.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.235+0000 7efe27eb0640 1 --2- 172.21.15.83:0/1510645895 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efe04066f90 0x7efe04069450 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efe2015db70 tx=0x7efe10008040 comp rx=0 tx=0).stop 2024-04-17T10:41:33.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.235+0000 7efe27eb0640 1 -- 172.21.15.83:0/1510645895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe2006d690 msgr2=0x7efe2013af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:33.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.235+0000 7efe27eb0640 1 --2- 172.21.15.83:0/1510645895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe2006d690 0x7efe2013af40 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7efe1c00caf0 tx=0x7efe1c0020b0 comp rx=0 tx=0).stop 2024-04-17T10:41:33.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.235+0000 7efe27eb0640 1 -- 172.21.15.83:0/1510645895 shutdown_connections 2024-04-17T10:41:33.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.235+0000 7efe27eb0640 1 --2- 172.21.15.83:0/1510645895 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efe04066f90 0x7efe04069450 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:33.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.235+0000 7efe27eb0640 1 --2- 172.21.15.83:0/1510645895 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe20146a00 0x7efe200cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:33.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.235+0000 7efe27eb0640 1 --2- 172.21.15.83:0/1510645895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe200b2d80 0x7efe201f99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:33.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.235+0000 7efe27eb0640 1 --2- 172.21.15.83:0/1510645895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe2006d690 0x7efe2013af40 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:33.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.235+0000 7efe27eb0640 1 -- 172.21.15.83:0/1510645895 >> 172.21.15.83:0/1510645895 conn(0x7efe2006abe0 msgr2=0x7efe2015fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:33.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.235+0000 7efe27eb0640 1 -- 172.21.15.83:0/1510645895 shutdown_connections 2024-04-17T10:41:33.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.235+0000 7efe27eb0640 1 -- 172.21.15.83:0/1510645895 wait complete. 2024-04-17T10:41:33.250 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-17T10:41:33.250 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-17T10:41:33.250 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-17T10:41:33.251 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:41:33.251 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-17T10:41:33.251 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-17T10:41:33.251 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-17T10:41:33.251 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:41:33.251 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-17T10:41:33.252 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-17T10:41:33.252 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-17T10:41:33.252 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:41:33.252 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-17T10:41:33.252 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-17T10:41:33.252 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-17T10:41:33.252 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:41:33.263 INFO:tasks.ceph.mds.c:Stopped 2024-04-17T10:41:33.263 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-17T10:41:33.264 INFO:tasks.ceph.mds.a:Stopped 2024-04-17T10:41:33.264 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-17T10:41:33.266 INFO:tasks.ceph.mds.d:Stopped 2024-04-17T10:41:33.266 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-17T10:41:33.272 INFO:tasks.ceph.mds.b:Stopped 2024-04-17T10:41:33.273 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-17T10:41:33.313 INFO:tasks.ceph.mds.c:Started 2024-04-17T10:41:33.313 DEBUG:teuthology.parallel:result is None 2024-04-17T10:41:33.313 INFO:tasks.ceph.mds.a:Started 2024-04-17T10:41:33.314 DEBUG:teuthology.parallel:result is None 2024-04-17T10:41:33.315 INFO:tasks.ceph.mds.d:Started 2024-04-17T10:41:33.315 DEBUG:teuthology.parallel:result is None 2024-04-17T10:41:33.316 INFO:tasks.ceph.mds.b:Started 2024-04-17T10:41:33.317 DEBUG:teuthology.parallel:result is None 2024-04-17T10:41:33.317 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-17T10:41:33.380 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-04-17T10:41:33.380 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-04-17T10:41:33.385 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2024-04-17T10:41:33.393 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2024-04-17T10:41:33.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.439+0000 7f1b2bccb640 1 Processor -- start 2024-04-17T10:41:33.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.439+0000 7f1b2bccb640 1 -- start start 2024-04-17T10:41:33.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.439+0000 7f1b2bccb640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1b240d19c0 0x7f1b240d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:33.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.439+0000 7f1b2bccb640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b2415d4f0 0x7f1b240b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:33.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.439+0000 7f1b2bccb640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b240d0780 con 0x7f1b2415be00 2024-04-17T10:41:33.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.439+0000 7f1b2bccb640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b240b18a0 con 0x7f1b240d19c0 2024-04-17T10:41:33.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.439+0000 7f1b2bccb640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b240b1b80 con 0x7f1b2415d4f0 2024-04-17T10:41:33.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.439+0000 7f1b29266640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1b2415be00 0x7f1b240d0070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53710/0 (socket says 172.21.15.83:53710) 2024-04-17T10:41:33.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.439+0000 7f1b29266640 1 -- 172.21.15.83:0/1389644468 learned_addr learned my addr 172.21.15.83:0/1389644468 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:33.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.443+0000 7f1b29a67640 1 --2- 172.21.15.83:0/1389644468 >> v2:172.21.15.167:3300/0 conn(0x7f1b240d19c0 0x7f1b240d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:33.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.443+0000 7f1b2a268640 1 --2- 172.21.15.83:0/1389644468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b2415d4f0 0x7f1b240b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:33.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.443+0000 7f1b28a65640 1 -- 172.21.15.83:0/1389644468 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1320592223 0 0) 0x7f1b240d0780 con 0x7f1b2415be00 2024-04-17T10:41:33.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.443+0000 7f1b28a65640 1 -- 172.21.15.83:0/1389644468 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b10003530 con 0x7f1b2415be00 2024-04-17T10:41:33.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.443+0000 7f1b29a67640 1 -- 172.21.15.83:0/1389644468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b2415d4f0 msgr2=0x7f1b240b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:33.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.443+0000 7f1b29a67640 1 --2- 172.21.15.83:0/1389644468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b2415d4f0 0x7f1b240b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:33.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.443+0000 7f1b29a67640 1 -- 172.21.15.83:0/1389644468 >> v1:172.21.15.83:6789/0 conn(0x7f1b2415be00 legacy=0x7f1b240d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:33.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.443+0000 7f1b29a67640 1 -- 172.21.15.83:0/1389644468 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b240b1e60 con 0x7f1b240d19c0 2024-04-17T10:41:33.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.443+0000 7f1b2a268640 1 --2- 172.21.15.83:0/1389644468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b2415d4f0 0x7f1b240b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:41:33.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.443+0000 7f1b29a67640 1 --2- 172.21.15.83:0/1389644468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b240d19c0 0x7f1b240d1da0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c009d90 tx=0x7f1b1c01c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15fb22ea5d766f7c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:33.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.443+0000 7f1b28a65640 1 -- 172.21.15.83:0/1389644468 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c0b2070 con 0x7f1b240d19c0 2024-04-17T10:41:33.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.443+0000 7f1b28a65640 1 -- 172.21.15.83:0/1389644468 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b1c0a5880 con 0x7f1b240d19c0 2024-04-17T10:41:33.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.443+0000 7f1b28a65640 1 -- 172.21.15.83:0/1389644468 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c0a5c30 con 0x7f1b240d19c0 2024-04-17T10:41:33.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.443+0000 7f1b2bccb640 1 -- 172.21.15.83:0/1389644468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b240d19c0 msgr2=0x7f1b240d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:33.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.443+0000 7f1b2bccb640 1 --2- 172.21.15.83:0/1389644468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b240d19c0 0x7f1b240d1da0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c009d90 tx=0x7f1b1c01c8b0 comp rx=0 tx=0).stop 2024-04-17T10:41:33.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.443+0000 7f1b2bccb640 1 -- 172.21.15.83:0/1389644468 shutdown_connections 2024-04-17T10:41:33.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.443+0000 7f1b2bccb640 1 --2- 172.21.15.83:0/1389644468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b2415d4f0 0x7f1b240b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:33.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.443+0000 7f1b2bccb640 1 --2- 172.21.15.83:0/1389644468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b240d19c0 0x7f1b240d1da0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:33.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.443+0000 7f1b2bccb640 1 -- 172.21.15.83:0/1389644468 >> 172.21.15.83:0/1389644468 conn(0x7f1b2409a9d0 msgr2=0x7f1b2409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:33.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.443+0000 7f1b2bccb640 1 -- 172.21.15.83:0/1389644468 shutdown_connections 2024-04-17T10:41:33.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.443+0000 7f1b2bccb640 1 -- 172.21.15.83:0/1389644468 wait complete. 2024-04-17T10:41:33.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.447+0000 7f1b2bccb640 1 Processor -- start 2024-04-17T10:41:33.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.447+0000 7f1b2bccb640 1 -- start start 2024-04-17T10:41:33.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.447+0000 7f1b2bccb640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b240d19c0 0x7f1b2414ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:33.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.447+0000 7f1b2bccb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b2415d4f0 0x7f1b24150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:33.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.447+0000 7f1b2bccb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b24151230 0x7f1b2414b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:33.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.447+0000 7f1b2bccb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2415cfe0 con 0x7f1b2415d4f0 2024-04-17T10:41:33.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.447+0000 7f1b2bccb640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b240b3290 con 0x7f1b24151230 2024-04-17T10:41:33.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.447+0000 7f1b2bccb640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2415d160 con 0x7f1b240d19c0 2024-04-17T10:41:33.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.447+0000 7f1b29266640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b2415d4f0 0x7f1b24150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:33.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.447+0000 7f1b29266640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b2415d4f0 0x7f1b24150cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40626/0 (socket says 172.21.15.83:40626) 2024-04-17T10:41:33.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.447+0000 7f1b29266640 1 -- 172.21.15.83:0/1482726409 learned_addr learned my addr 172.21.15.83:0/1482726409 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:33.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.447+0000 7f1b29a67640 1 --2- 172.21.15.83:0/1482726409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b240d19c0 0x7f1b2414ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:33.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.447+0000 7f1b2a268640 1 --2- 172.21.15.83:0/1482726409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b24151230 0x7f1b2414b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:33.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.447+0000 7f1b29266640 1 -- 172.21.15.83:0/1482726409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b240d19c0 msgr2=0x7f1b2414ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:33.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.447+0000 7f1b29266640 1 --2- 172.21.15.83:0/1482726409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b240d19c0 0x7f1b2414ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:33.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.447+0000 7f1b29266640 1 -- 172.21.15.83:0/1482726409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b24151230 msgr2=0x7f1b2414b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:33.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.447+0000 7f1b29266640 1 --2- 172.21.15.83:0/1482726409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b24151230 0x7f1b2414b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:33.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.447+0000 7f1b29266640 1 -- 172.21.15.83:0/1482726409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b24151720 con 0x7f1b2415d4f0 2024-04-17T10:41:33.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.447+0000 7f1b29266640 1 --2- 172.21.15.83:0/1482726409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b2415d4f0 0x7f1b24150cf0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400c3f0 tx=0x7f1b1400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:33.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.447+0000 7f1b1a7fc640 1 -- 172.21.15.83:0/1482726409 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b14017070 con 0x7f1b2415d4f0 2024-04-17T10:41:33.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.447+0000 7f1b1a7fc640 1 -- 172.21.15.83:0/1482726409 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b14005cf0 con 0x7f1b2415d4f0 2024-04-17T10:41:33.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.447+0000 7f1b2bccb640 1 -- 172.21.15.83:0/1482726409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b24151a00 con 0x7f1b2415d4f0 2024-04-17T10:41:33.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.447+0000 7f1b1a7fc640 1 -- 172.21.15.83:0/1482726409 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b140072f0 con 0x7f1b2415d4f0 2024-04-17T10:41:33.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.447+0000 7f1b2bccb640 1 -- 172.21.15.83:0/1482726409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b2415a810 con 0x7f1b2415d4f0 2024-04-17T10:41:33.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.447+0000 7f1b2bccb640 1 -- 172.21.15.83:0/1482726409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b24074f90 con 0x7f1b2415d4f0 2024-04-17T10:41:33.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.447+0000 7f1b1a7fc640 1 -- 172.21.15.83:0/1482726409 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1b1401a070 con 0x7f1b2415d4f0 2024-04-17T10:41:33.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.447+0000 7f1b1a7fc640 1 --2- 172.21.15.83:0/1482726409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1b08066f60 0x7f1b08069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:33.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.451+0000 7f1b1a7fc640 1 -- 172.21.15.83:0/1482726409 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1b14084030 con 0x7f1b2415d4f0 2024-04-17T10:41:33.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.451+0000 7f1b29a67640 1 --2- 172.21.15.83:0/1482726409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1b08066f60 0x7f1b08069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:33.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.451+0000 7f1b29a67640 1 --2- 172.21.15.83:0/1482726409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1b08066f60 0x7f1b08069420 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c0af000 tx=0x7f1b1c0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:33.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.455+0000 7f1b1a7fc640 1 -- 172.21.15.83:0/1482726409 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1b14054d80 con 0x7f1b2415d4f0 2024-04-17T10:41:33.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:33.607+0000 7f1b2bccb640 1 -- 172.21.15.83:0/1482726409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1b240b7a10 con 0x7f1b2415d4f0 2024-04-17T10:41:34.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.399+0000 7f1b1a7fc640 1 -- 172.21.15.83:0/1482726409 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v40) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1b14058e20 con 0x7f1b2415d4f0 2024-04-17T10:41:34.396 INFO:teuthology.orchestra.run.smithi083.stderr: removed all blocklist entries 2024-04-17T10:41:34.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.403+0000 7f1b2bccb640 1 -- 172.21.15.83:0/1482726409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1b08066f60 msgr2=0x7f1b08069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:34.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.403+0000 7f1b2bccb640 1 --2- 172.21.15.83:0/1482726409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1b08066f60 0x7f1b08069420 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c0af000 tx=0x7f1b1c0a7070 comp rx=0 tx=0).stop 2024-04-17T10:41:34.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.403+0000 7f1b2bccb640 1 -- 172.21.15.83:0/1482726409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b2415d4f0 msgr2=0x7f1b24150cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:34.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.403+0000 7f1b2bccb640 1 --2- 172.21.15.83:0/1482726409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b2415d4f0 0x7f1b24150cf0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400c3f0 tx=0x7f1b1400c8b0 comp rx=0 tx=0).stop 2024-04-17T10:41:34.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.403+0000 7f1b2bccb640 1 -- 172.21.15.83:0/1482726409 shutdown_connections 2024-04-17T10:41:34.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.403+0000 7f1b2bccb640 1 --2- 172.21.15.83:0/1482726409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1b08066f60 0x7f1b08069420 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:34.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.403+0000 7f1b2bccb640 1 --2- 172.21.15.83:0/1482726409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b24151230 0x7f1b2414b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:34.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.403+0000 7f1b2bccb640 1 --2- 172.21.15.83:0/1482726409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b2415d4f0 0x7f1b24150cf0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:34.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.403+0000 7f1b2bccb640 1 --2- 172.21.15.83:0/1482726409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b240d19c0 0x7f1b2414ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:34.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.403+0000 7f1b2bccb640 1 -- 172.21.15.83:0/1482726409 >> 172.21.15.83:0/1482726409 conn(0x7f1b2409a9d0 msgr2=0x7f1b2415c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:34.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.403+0000 7f1b2bccb640 1 -- 172.21.15.83:0/1482726409 shutdown_connections 2024-04-17T10:41:34.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.403+0000 7f1b2bccb640 1 -- 172.21.15.83:0/1482726409 wait complete. 2024-04-17T10:41:34.421 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-17T10:41:34.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.535+0000 7ff7588c5640 1 Processor -- start 2024-04-17T10:41:34.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.535+0000 7ff7588c5640 1 -- start start 2024-04-17T10:41:34.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.535+0000 7ff7588c5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff75415db50 0x7ff754070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:34.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.535+0000 7ff7588c5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff7540aedd0 0x7ff75406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:34.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.535+0000 7ff7588c5640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7540709b0 con 0x7ff75406dae0 2024-04-17T10:41:34.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.535+0000 7ff7588c5640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff75415e440 con 0x7ff75415db50 2024-04-17T10:41:34.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.535+0000 7ff7588c5640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff75415e720 con 0x7ff7540aedd0 2024-04-17T10:41:34.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.535+0000 7ff75259c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff75415db50 0x7ff754070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:34.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.535+0000 7ff752d9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff75406dae0 0x7ff7540cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53714/0 (socket says 172.21.15.83:53714) 2024-04-17T10:41:34.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.535+0000 7ff752d9d640 1 -- 172.21.15.83:0/1647998417 learned_addr learned my addr 172.21.15.83:0/1647998417 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:34.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.535+0000 7ff749d9b640 1 --2- 172.21.15.83:0/1647998417 >> v2:172.21.15.167:3300/0 conn(0x7ff7540aedd0 0x7ff75406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:34.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.535+0000 7ff75259c640 1 -- 172.21.15.83:0/1647998417 >> v2:172.21.15.167:3300/0 conn(0x7ff7540aedd0 msgr2=0x7ff75406d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:34.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.535+0000 7ff75259c640 1 --2- 172.21.15.83:0/1647998417 >> v2:172.21.15.167:3300/0 conn(0x7ff7540aedd0 0x7ff75406d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:34.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.535+0000 7ff75259c640 1 -- 172.21.15.83:0/1647998417 >> v1:172.21.15.83:6789/0 conn(0x7ff75406dae0 legacy=0x7ff7540cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:34.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.535+0000 7ff75259c640 1 -- 172.21.15.83:0/1647998417 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75415ea00 con 0x7ff75415db50 2024-04-17T10:41:34.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.535+0000 7ff75259c640 1 --2- 172.21.15.83:0/1647998417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff75415db50 0x7ff754070470 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff7440097c0 tx=0x7ff74401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209b46e9622c5f18 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:34.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.535+0000 7ff751d9b640 1 -- 172.21.15.83:0/1647998417 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7440b0070 con 0x7ff75415db50 2024-04-17T10:41:34.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.539+0000 7ff751d9b640 1 -- 172.21.15.83:0/1647998417 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff744004230 con 0x7ff75415db50 2024-04-17T10:41:34.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.539+0000 7ff751d9b640 1 -- 172.21.15.83:0/1647998417 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff744003620 con 0x7ff75415db50 2024-04-17T10:41:34.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.539+0000 7ff7588c5640 1 -- 172.21.15.83:0/1647998417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff75415db50 msgr2=0x7ff754070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:34.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.539+0000 7ff7588c5640 1 --2- 172.21.15.83:0/1647998417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff75415db50 0x7ff754070470 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff7440097c0 tx=0x7ff74401c460 comp rx=0 tx=0).stop 2024-04-17T10:41:34.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.539+0000 7ff7588c5640 1 -- 172.21.15.83:0/1647998417 shutdown_connections 2024-04-17T10:41:34.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.539+0000 7ff7588c5640 1 --2- 172.21.15.83:0/1647998417 >> v2:172.21.15.167:3300/0 conn(0x7ff7540aedd0 0x7ff75406d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:34.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.539+0000 7ff7588c5640 1 --2- 172.21.15.83:0/1647998417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff75415db50 0x7ff754070470 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:34.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.539+0000 7ff7588c5640 1 -- 172.21.15.83:0/1647998417 >> 172.21.15.83:0/1647998417 conn(0x7ff75406abe0 msgr2=0x7ff75406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:34.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.539+0000 7ff7588c5640 1 -- 172.21.15.83:0/1647998417 shutdown_connections 2024-04-17T10:41:34.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.539+0000 7ff7588c5640 1 -- 172.21.15.83:0/1647998417 wait complete. 2024-04-17T10:41:34.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.539+0000 7ff7588c5640 1 Processor -- start 2024-04-17T10:41:34.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.539+0000 7ff7588c5640 1 -- start start 2024-04-17T10:41:34.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.539+0000 7ff7588c5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff75406dae0 0x7ff7541f7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:34.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.539+0000 7ff7588c5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7540aedd0 0x7ff75415f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:34.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.539+0000 7ff7588c5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff75415db50 0x7ff75415f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:34.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.539+0000 7ff7588c5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7541677e0 con 0x7ff75406dae0 2024-04-17T10:41:34.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.539+0000 7ff7588c5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff754167660 con 0x7ff75415db50 2024-04-17T10:41:34.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.539+0000 7ff7588c5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff754167960 con 0x7ff7540aedd0 2024-04-17T10:41:34.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.539+0000 7ff75259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff75406dae0 0x7ff7541f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:34.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.539+0000 7ff75259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff75406dae0 0x7ff7541f7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40640/0 (socket says 172.21.15.83:40640) 2024-04-17T10:41:34.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.539+0000 7ff75259c640 1 -- 172.21.15.83:0/640243264 learned_addr learned my addr 172.21.15.83:0/640243264 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:34.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.539+0000 7ff75259c640 1 -- 172.21.15.83:0/640243264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7540aedd0 msgr2=0x7ff75415f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:34.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.539+0000 7ff75259c640 1 --2- 172.21.15.83:0/640243264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7540aedd0 0x7ff75415f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:34.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.539+0000 7ff75259c640 1 -- 172.21.15.83:0/640243264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff75415db50 msgr2=0x7ff75415f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:34.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.539+0000 7ff75259c640 1 --2- 172.21.15.83:0/640243264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff75415db50 0x7ff75415f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:34.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.539+0000 7ff75259c640 1 -- 172.21.15.83:0/640243264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff754145890 con 0x7ff75406dae0 2024-04-17T10:41:34.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.539+0000 7ff75259c640 1 --2- 172.21.15.83:0/640243264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff75406dae0 0x7ff7541f7680 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff744019070 tx=0x7ff744019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:34.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.539+0000 7ff74b7fe640 1 -- 172.21.15.83:0/640243264 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7440b0040 con 0x7ff75406dae0 2024-04-17T10:41:34.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.539+0000 7ff74b7fe640 1 -- 172.21.15.83:0/640243264 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7440bc070 con 0x7ff75406dae0 2024-04-17T10:41:34.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.539+0000 7ff7588c5640 1 -- 172.21.15.83:0/640243264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7541f6ba0 con 0x7ff75406dae0 2024-04-17T10:41:34.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.539+0000 7ff74b7fe640 1 -- 172.21.15.83:0/640243264 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff744006e20 con 0x7ff75406dae0 2024-04-17T10:41:34.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.539+0000 7ff7588c5640 1 -- 172.21.15.83:0/640243264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7541f7160 con 0x7ff75406dae0 2024-04-17T10:41:34.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.543+0000 7ff7588c5640 1 -- 172.21.15.83:0/640243264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7541f69b0 con 0x7ff75406dae0 2024-04-17T10:41:34.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.543+0000 7ff74b7fe640 1 -- 172.21.15.83:0/640243264 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff7440b3070 con 0x7ff75406dae0 2024-04-17T10:41:34.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.543+0000 7ff74b7fe640 1 --2- 172.21.15.83:0/640243264 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff734066f60 0x7ff734069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:34.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.543+0000 7ff74b7fe640 1 -- 172.21.15.83:0/640243264 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff74411eaf0 con 0x7ff75406dae0 2024-04-17T10:41:34.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.543+0000 7ff749d9b640 1 --2- 172.21.15.83:0/640243264 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff734066f60 0x7ff734069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:34.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.543+0000 7ff749d9b640 1 --2- 172.21.15.83:0/640243264 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff734066f60 0x7ff734069420 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff754158f50 tx=0x7ff73c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:34.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.547+0000 7ff74b7fe640 1 -- 172.21.15.83:0/640243264 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff744016050 con 0x7ff75406dae0 2024-04-17T10:41:34.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.755+0000 7ff7588c5640 1 -- 172.21.15.83:0/640243264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff754074f90 con 0x7ff75406dae0 2024-04-17T10:41:34.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.759+0000 7ff74b7fe640 1 -- 172.21.15.83:0/640243264 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff7440aa3a0 con 0x7ff75406dae0 2024-04-17T10:41:34.756 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:41:34.756 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:41:34.756 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_dump": [ 2024-04-17T10:41:34.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:41:34.756 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.a", 2024-04-17T10:41:34.756 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mfpxsNxAANpL7rcWevXwd2KMftSQnEQ==", 2024-04-17T10:41:34.756 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:41:34.756 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:41:34.756 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:41:34.757 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:41:34.757 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:41:34.757 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:41:34.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:34.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:41:34.757 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.b", 2024-04-17T10:41:34.757 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mXlI9ChAAYdcrGVmtYdtWWgCLVmpTTg==", 2024-04-17T10:41:34.757 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:41:34.757 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:41:34.757 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:41:34.757 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:41:34.757 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:41:34.757 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:41:34.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:34.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:41:34.758 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.c", 2024-04-17T10:41:34.758 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mupo3BBAAyKH801vpFLXYs9mfzSlT0w==", 2024-04-17T10:41:34.758 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:41:34.758 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:41:34.758 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:41:34.758 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:41:34.758 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:41:34.758 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:41:34.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:34.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:41:34.758 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.d", 2024-04-17T10:41:34.758 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mbm++EhAAIBMOtQoD+kPeMqR6gK5ctw==", 2024-04-17T10:41:34.759 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:41:34.759 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:41:34.759 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:41:34.759 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:41:34.759 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:41:34.759 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:41:34.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:34.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:41:34.759 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.0", 2024-04-17T10:41:34.759 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBXph9m1HWhIRAAOq+9nGq7OGonUCE2aeiAzQ==", 2024-04-17T10:41:34.759 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:41:34.759 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:41:34.759 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:41:34.759 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:41:34.760 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:41:34.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:34.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:41:34.760 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.1", 2024-04-17T10:41:34.760 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBaph9mlMmtJRAAXHLMjXfm/+WjIFURLkbAfA==", 2024-04-17T10:41:34.760 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:41:34.760 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:41:34.760 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:41:34.760 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:41:34.760 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:41:34.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:34.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:41:34.760 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.2", 2024-04-17T10:41:34.760 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBdph9mBsNPLRAAtR5hwGdDkjLDLS19QUDDYQ==", 2024-04-17T10:41:34.761 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:41:34.761 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:41:34.761 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:41:34.761 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:41:34.761 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:41:34.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:34.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:41:34.761 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.3", 2024-04-17T10:41:34.761 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBgph9mZ4EwNRAAIyL7e0EK1523dPXmbOOkzw==", 2024-04-17T10:41:34.761 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:41:34.761 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:41:34.761 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:41:34.761 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:41:34.762 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:41:34.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:34.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:41:34.762 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.4", 2024-04-17T10:41:34.762 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBmph9mphAsMhAAtF4uV28Nb66PNjuugB12Hw==", 2024-04-17T10:41:34.762 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:41:34.762 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:41:34.762 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:41:34.762 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:41:34.762 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:41:34.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:34.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:41:34.763 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.5", 2024-04-17T10:41:34.763 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBpph9mHJQNNxAA72eOFYFbhakQac1JKRNjzw==", 2024-04-17T10:41:34.763 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:41:34.763 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:41:34.763 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:41:34.763 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:41:34.763 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:41:34.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:34.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:41:34.763 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.6", 2024-04-17T10:41:34.763 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBtph9mb6RKAxAAoVX6Cp7l7j3Lm1ES2Sq+fQ==", 2024-04-17T10:41:34.763 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:41:34.763 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:41:34.764 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:41:34.764 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:41:34.764 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:41:34.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:34.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:41:34.764 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.7", 2024-04-17T10:41:34.764 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBwph9mdrdrBxAApDwjml2ahsLbRol4uL/BxA==", 2024-04-17T10:41:34.764 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:41:34.764 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:41:34.764 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:41:34.764 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:41:34.764 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:41:34.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:34.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:41:34.765 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.0", 2024-04-17T10:41:34.765 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mHGmXGBAAZF+JiAG9YrFtbW+dCy2IEQ==", 2024-04-17T10:41:34.765 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:41:34.765 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:41:34.765 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow r", 2024-04-17T10:41:34.765 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-17T10:41:34.765 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:41:34.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:34.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:41:34.765 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.1", 2024-04-17T10:41:34.765 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mB7wgIBAABdavAo/uzVuSjfkkcYtqJg==", 2024-04-17T10:41:34.765 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:41:34.766 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:41:34.766 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-17T10:41:34.766 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-17T10:41:34.766 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-17T10:41:34.766 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:41:34.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:34.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:41:34.766 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.2", 2024-04-17T10:41:34.766 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9m+wKgHRAAe8IUFx34ItLpsaDqSYyEgA==", 2024-04-17T10:41:34.766 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:41:34.766 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:41:34.766 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-17T10:41:34.766 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-17T10:41:34.767 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-17T10:41:34.767 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:41:34.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:34.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:41:34.767 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.3", 2024-04-17T10:41:34.767 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mfJ0yJRAAHH7rRh/BUIujTT04AF2l0w==", 2024-04-17T10:41:34.767 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:41:34.767 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:41:34.767 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-17T10:41:34.767 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-17T10:41:34.767 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-17T10:41:34.767 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:41:34.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:34.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:41:34.768 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.admin", 2024-04-17T10:41:34.768 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mjJ3GIBAA/2qaCbRiLJ5X1cv3ZGdfoA==", 2024-04-17T10:41:34.768 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:41:34.768 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-17T10:41:34.768 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:41:34.768 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:41:34.768 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:41:34.768 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:41:34.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:34.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:41:34.768 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.x", 2024-04-17T10:41:34.768 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mMFptNBAA/G0zEfMZTCy9Iox+MwHG3Q==", 2024-04-17T10:41:34.769 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:41:34.769 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-17T10:41:34.769 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-04-17T10:41:34.769 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:41:34.769 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:41:34.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:34.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:41:34.769 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.y", 2024-04-17T10:41:34.769 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mjFbRMRAAjY2d8POBzp8hEp9GSO0ylQ==", 2024-04-17T10:41:34.769 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:41:34.769 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-17T10:41:34.769 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-04-17T10:41:34.769 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:41:34.769 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:41:34.769 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:41:34.770 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-17T10:41:34.770 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:41:34.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.759+0000 7ff7588c5640 1 -- 172.21.15.83:0/640243264 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff734066f60 msgr2=0x7ff734069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:34.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.759+0000 7ff7588c5640 1 --2- 172.21.15.83:0/640243264 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff734066f60 0x7ff734069420 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff754158f50 tx=0x7ff73c008040 comp rx=0 tx=0).stop 2024-04-17T10:41:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.759+0000 7ff7588c5640 1 -- 172.21.15.83:0/640243264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff75406dae0 msgr2=0x7ff7541f7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.759+0000 7ff7588c5640 1 --2- 172.21.15.83:0/640243264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff75406dae0 0x7ff7541f7680 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff744019070 tx=0x7ff744019200 comp rx=0 tx=0).stop 2024-04-17T10:41:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.759+0000 7ff7588c5640 1 -- 172.21.15.83:0/640243264 shutdown_connections 2024-04-17T10:41:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.759+0000 7ff7588c5640 1 --2- 172.21.15.83:0/640243264 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff734066f60 0x7ff734069420 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.759+0000 7ff7588c5640 1 --2- 172.21.15.83:0/640243264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff75415db50 0x7ff75415f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.759+0000 7ff7588c5640 1 --2- 172.21.15.83:0/640243264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7540aedd0 0x7ff75415f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.759+0000 7ff7588c5640 1 --2- 172.21.15.83:0/640243264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff75406dae0 0x7ff7541f7680 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.759+0000 7ff7588c5640 1 -- 172.21.15.83:0/640243264 >> 172.21.15.83:0/640243264 conn(0x7ff75406abe0 msgr2=0x7ff75406cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.759+0000 7ff7588c5640 1 -- 172.21.15.83:0/640243264 shutdown_connections 2024-04-17T10:41:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.759+0000 7ff7588c5640 1 -- 172.21.15.83:0/640243264 wait complete. 2024-04-17T10:41:34.774 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:41:34.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.891+0000 7f7c6d437640 1 Processor -- start 2024-04-17T10:41:34.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.895+0000 7f7c6d437640 1 -- start start 2024-04-17T10:41:34.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.895+0000 7f7c6d437640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c6815db50 0x7f7c68070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:34.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.895+0000 7f7c6d437640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c680709b0 con 0x7f7c6806dcb0 2024-04-17T10:41:34.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.895+0000 7f7c6d437640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c6815e440 con 0x7f7c680aedd0 2024-04-17T10:41:34.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.895+0000 7f7c6d437640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6815e720 con 0x7f7c6815db50 2024-04-17T10:41:34.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.895+0000 7f7c677fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7c6806dcb0 0x7f7c680b2e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53730/0 (socket says 172.21.15.83:53730) 2024-04-17T10:41:34.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.895+0000 7f7c677fe640 1 -- 172.21.15.83:0/1183974428 learned_addr learned my addr 172.21.15.83:0/1183974428 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:34.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.895+0000 7f7c66ffd640 1 --2- 172.21.15.83:0/1183974428 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c6815db50 0x7f7c68070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:34.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.895+0000 7f7c65ffb640 1 -- 172.21.15.83:0/1183974428 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2507149762 0 0) 0x7f7c580014a0 con 0x7f7c6806dcb0 2024-04-17T10:41:34.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.895+0000 7f7c65ffb640 1 -- 172.21.15.83:0/1183974428 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c54003530 con 0x7f7c6806dcb0 2024-04-17T10:41:34.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.895+0000 7f7c65ffb640 1 -- 172.21.15.83:0/1183974428 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3276650281 0 0) 0x7f7c54003530 con 0x7f7c6806dcb0 2024-04-17T10:41:34.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.895+0000 7f7c65ffb640 1 -- 172.21.15.83:0/1183974428 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c6815db50 msgr2=0x7f7c68070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:34.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.895+0000 7f7c65ffb640 1 --2- 172.21.15.83:0/1183974428 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c6815db50 0x7f7c68070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:34.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.895+0000 7f7c65ffb640 1 -- 172.21.15.83:0/1183974428 >> v1:172.21.15.167:6789/0 conn(0x7f7c680aedd0 legacy=0x7f7c6806d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:34.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.895+0000 7f7c65ffb640 1 -- 172.21.15.83:0/1183974428 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6815ea00 con 0x7f7c6806dcb0 2024-04-17T10:41:34.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.895+0000 7f7c65ffb640 1 -- 172.21.15.83:0/1183974428 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f7c58001bd0 con 0x7f7c6806dcb0 2024-04-17T10:41:34.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.895+0000 7f7c65ffb640 1 -- 172.21.15.83:0/1183974428 >> v1:172.21.15.83:6789/0 conn(0x7f7c6806dcb0 legacy=0x7f7c680b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:34.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.895+0000 7f7c65ffb640 1 --2- 172.21.15.83:0/1183974428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c540039f0 0x7f7c54003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:34.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.895+0000 7f7c65ffb640 1 --2- 172.21.15.83:0/1183974428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c54004810 0x7f7c54004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:34.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.895+0000 7f7c65ffb640 1 --2- 172.21.15.83:0/1183974428 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c6815db50 0x7f7c540051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:34.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.895+0000 7f7c65ffb640 1 -- 172.21.15.83:0/1183974428 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6815e720 con 0x7f7c54004810 2024-04-17T10:41:34.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.895+0000 7f7c65ffb640 1 -- 172.21.15.83:0/1183974428 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5400ba80 con 0x7f7c540039f0 2024-04-17T10:41:34.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.895+0000 7f7c65ffb640 1 -- 172.21.15.83:0/1183974428 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5400de90 con 0x7f7c6815db50 2024-04-17T10:41:34.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.895+0000 7f7c667fc640 1 --2- 172.21.15.83:0/1183974428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c54004810 0x7f7c54004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:34.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.895+0000 7f7c66ffd640 1 --2- 172.21.15.83:0/1183974428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c540039f0 0x7f7c54003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:34.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.895+0000 7f7c667fc640 1 -- 172.21.15.83:0/1183974428 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c6815db50 msgr2=0x7f7c540051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:34.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.895+0000 7f7c667fc640 1 --2- 172.21.15.83:0/1183974428 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c6815db50 0x7f7c540051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:34.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.895+0000 7f7c667fc640 1 -- 172.21.15.83:0/1183974428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c540039f0 msgr2=0x7f7c54003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:34.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.895+0000 7f7c667fc640 1 --2- 172.21.15.83:0/1183974428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c540039f0 0x7f7c54003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:34.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.895+0000 7f7c667fc640 1 -- 172.21.15.83:0/1183974428 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c54003530 con 0x7f7c54004810 2024-04-17T10:41:34.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.895+0000 7f7c66ffd640 1 --2- 172.21.15.83:0/1183974428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c540039f0 0x7f7c54003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:34.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.899+0000 7f7c667fc640 1 --2- 172.21.15.83:0/1183974428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c54004810 0x7f7c54004c90 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f7c50001750 tx=0x7f7c5001c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90299be176cdd31a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:34.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.899+0000 7f7c65ffb640 1 -- 172.21.15.83:0/1183974428 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c500013a0 con 0x7f7c54004810 2024-04-17T10:41:34.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.899+0000 7f7c65ffb640 1 -- 172.21.15.83:0/1183974428 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c500052f0 con 0x7f7c54004810 2024-04-17T10:41:34.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.899+0000 7f7c65ffb640 1 -- 172.21.15.83:0/1183974428 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c50003840 con 0x7f7c54004810 2024-04-17T10:41:34.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.899+0000 7f7c6d437640 1 -- 172.21.15.83:0/1183974428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c54004810 msgr2=0x7f7c54004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:34.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.899+0000 7f7c6d437640 1 --2- 172.21.15.83:0/1183974428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c54004810 0x7f7c54004c90 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f7c50001750 tx=0x7f7c5001c300 comp rx=0 tx=0).stop 2024-04-17T10:41:34.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.899+0000 7f7c677fe640 1 -- 172.21.15.83:0/1183974428 reap_dead start 2024-04-17T10:41:34.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.899+0000 7f7c6d437640 1 -- 172.21.15.83:0/1183974428 shutdown_connections 2024-04-17T10:41:34.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.899+0000 7f7c6d437640 1 -- 172.21.15.83:0/1183974428 >> 172.21.15.83:0/1183974428 conn(0x7f7c6806abe0 msgr2=0x7f7c6806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:34.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.899+0000 7f7c6d437640 1 -- 172.21.15.83:0/1183974428 shutdown_connections 2024-04-17T10:41:34.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.899+0000 7f7c6d437640 1 -- 172.21.15.83:0/1183974428 wait complete. 2024-04-17T10:41:34.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.899+0000 7f7c6d437640 1 Processor -- start 2024-04-17T10:41:34.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.899+0000 7f7c6d437640 1 -- start start 2024-04-17T10:41:34.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.899+0000 7f7c6d437640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c54004810 0x7f7c680c4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:34.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.899+0000 7f7c6d437640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c680af2e0 0x7f7c681f39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:34.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.899+0000 7f7c6d437640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c680ad840 0x7f7c680adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:34.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.899+0000 7f7c6d437640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6815df30 con 0x7f7c54004810 2024-04-17T10:41:34.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.899+0000 7f7c6d437640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6815efc0 con 0x7f7c680ad840 2024-04-17T10:41:34.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.899+0000 7f7c6d437640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6815e0b0 con 0x7f7c680af2e0 2024-04-17T10:41:34.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.899+0000 7f7c66ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c54004810 0x7f7c680c4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:34.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.899+0000 7f7c66ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c54004810 0x7f7c680c4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40650/0 (socket says 172.21.15.83:40650) 2024-04-17T10:41:34.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.899+0000 7f7c66ffd640 1 -- 172.21.15.83:0/1349499186 learned_addr learned my addr 172.21.15.83:0/1349499186 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:34.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.899+0000 7f7c677fe640 1 --2- 172.21.15.83:0/1349499186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c680ad840 0x7f7c680adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:34.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.899+0000 7f7c66ffd640 1 -- 172.21.15.83:0/1349499186 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c680af2e0 msgr2=0x7f7c681f39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:34.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.899+0000 7f7c66ffd640 1 --2- 172.21.15.83:0/1349499186 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c680af2e0 0x7f7c681f39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:34.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.899+0000 7f7c66ffd640 1 -- 172.21.15.83:0/1349499186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c680ad840 msgr2=0x7f7c680adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:34.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.899+0000 7f7c66ffd640 1 --2- 172.21.15.83:0/1349499186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c680ad840 0x7f7c680adcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:34.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.899+0000 7f7c66ffd640 1 -- 172.21.15.83:0/1349499186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c680afaf0 con 0x7f7c54004810 2024-04-17T10:41:34.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.899+0000 7f7c677fe640 1 --2- 172.21.15.83:0/1349499186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c680ad840 0x7f7c680adcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:34.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.899+0000 7f7c66ffd640 1 --2- 172.21.15.83:0/1349499186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c54004810 0x7f7c680c4d20 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7c60010900 tx=0x7f7c60010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:34.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.903+0000 7f7c5f7fe640 1 -- 172.21.15.83:0/1349499186 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6000f040 con 0x7f7c54004810 2024-04-17T10:41:34.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.903+0000 7f7c5f7fe640 1 -- 172.21.15.83:0/1349499186 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c60004220 con 0x7f7c54004810 2024-04-17T10:41:34.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.903+0000 7f7c5f7fe640 1 -- 172.21.15.83:0/1349499186 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6000e9c0 con 0x7f7c54004810 2024-04-17T10:41:34.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.903+0000 7f7c6d437640 1 -- 172.21.15.83:0/1349499186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c680afdd0 con 0x7f7c54004810 2024-04-17T10:41:34.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.903+0000 7f7c6d437640 1 -- 172.21.15.83:0/1349499186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c680b0390 con 0x7f7c54004810 2024-04-17T10:41:34.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.903+0000 7f7c6d437640 1 -- 172.21.15.83:0/1349499186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c68074f90 con 0x7f7c54004810 2024-04-17T10:41:34.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.903+0000 7f7c5f7fe640 1 -- 172.21.15.83:0/1349499186 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7c6000eb60 con 0x7f7c54004810 2024-04-17T10:41:34.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.903+0000 7f7c5f7fe640 1 --2- 172.21.15.83:0/1349499186 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7c48066f60 0x7f7c48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:34.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.903+0000 7f7c5f7fe640 1 -- 172.21.15.83:0/1349499186 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7c60085110 con 0x7f7c54004810 2024-04-17T10:41:34.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.903+0000 7f7c667fc640 1 --2- 172.21.15.83:0/1349499186 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7c48066f60 0x7f7c48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:34.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.903+0000 7f7c667fc640 1 --2- 172.21.15.83:0/1349499186 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7c48066f60 0x7f7c48069420 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7c540057f0 tx=0x7f7c50001130 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:34.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:34.907+0000 7f7c5f7fe640 1 -- 172.21.15.83:0/1349499186 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7c60018030 con 0x7f7c54004810 2024-04-17T10:41:35.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.067+0000 7f7c6d437640 1 -- 172.21.15.83:0/1349499186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c680cc060 con 0x7f7c54004810 2024-04-17T10:41:35.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.071+0000 7f7c5f7fe640 1 -- 172.21.15.83:0/1349499186 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+12819 (secure 0 0 0) 0x7f7c60056470 con 0x7f7c54004810 2024-04-17T10:41:35.067 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:41:35.068 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":42,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:41:34.411110+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:41:35.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.071+0000 7f7c6d437640 1 -- 172.21.15.83:0/1349499186 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7c48066f60 msgr2=0x7f7c48069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:35.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.071+0000 7f7c6d437640 1 --2- 172.21.15.83:0/1349499186 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7c48066f60 0x7f7c48069420 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7c540057f0 tx=0x7f7c50001130 comp rx=0 tx=0).stop 2024-04-17T10:41:35.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.071+0000 7f7c6d437640 1 -- 172.21.15.83:0/1349499186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c54004810 msgr2=0x7f7c680c4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:35.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.071+0000 7f7c6d437640 1 --2- 172.21.15.83:0/1349499186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c54004810 0x7f7c680c4d20 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7c60010900 tx=0x7f7c60010dc0 comp rx=0 tx=0).stop 2024-04-17T10:41:35.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.071+0000 7f7c6d437640 1 -- 172.21.15.83:0/1349499186 shutdown_connections 2024-04-17T10:41:35.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.071+0000 7f7c6d437640 1 --2- 172.21.15.83:0/1349499186 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7c48066f60 0x7f7c48069420 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:35.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.071+0000 7f7c6d437640 1 --2- 172.21.15.83:0/1349499186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c680ad840 0x7f7c680adcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:35.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.071+0000 7f7c6d437640 1 --2- 172.21.15.83:0/1349499186 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c680af2e0 0x7f7c681f39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:35.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.071+0000 7f7c6d437640 1 --2- 172.21.15.83:0/1349499186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c54004810 0x7f7c680c4d20 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:35.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.071+0000 7f7c6d437640 1 -- 172.21.15.83:0/1349499186 >> 172.21.15.83:0/1349499186 conn(0x7f7c6806abe0 msgr2=0x7f7c68159d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:35.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.071+0000 7f7c6d437640 1 -- 172.21.15.83:0/1349499186 shutdown_connections 2024-04-17T10:41:35.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.071+0000 7f7c6d437640 1 -- 172.21.15.83:0/1349499186 wait complete. 2024-04-17T10:41:35.087 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:41:35.087 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:41:35.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.211+0000 7f0601052640 1 Processor -- start 2024-04-17T10:41:35.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.211+0000 7f0601052640 1 -- start start 2024-04-17T10:41:35.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.211+0000 7f0601052640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f05fc15b6e0 0x7f05fc071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:35.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.211+0000 7f0601052640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05fc06c1f0 con 0x7f05fc070470 2024-04-17T10:41:35.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.211+0000 7f0601052640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05fc070a10 con 0x7f05fc15bac0 2024-04-17T10:41:35.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.211+0000 7f0601052640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05fc070cf0 con 0x7f05fc15b6e0 2024-04-17T10:41:35.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.211+0000 7f05fb59e640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f05fc070470 0x7f05fc06ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53734/0 (socket says 172.21.15.83:53734) 2024-04-17T10:41:35.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.211+0000 7f05fb59e640 1 -- 172.21.15.83:0/2225430699 learned_addr learned my addr 172.21.15.83:0/2225430699 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:35.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.211+0000 7f05fad9d640 1 --2- 172.21.15.83:0/2225430699 >> v2:172.21.15.167:3300/0 conn(0x7f05fc15b6e0 0x7f05fc071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:35.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.211+0000 7f05f9d9b640 1 -- 172.21.15.83:0/2225430699 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3593542917 0 0) 0x7f05fc06c1f0 con 0x7f05fc070470 2024-04-17T10:41:35.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.211+0000 7f05f9d9b640 1 -- 172.21.15.83:0/2225430699 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05e4003530 con 0x7f05fc070470 2024-04-17T10:41:35.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.211+0000 7f05f9d9b640 1 -- 172.21.15.83:0/2225430699 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 122047456 0 0) 0x7f05e4003530 con 0x7f05fc070470 2024-04-17T10:41:35.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.211+0000 7f05f9d9b640 1 -- 172.21.15.83:0/2225430699 >> v2:172.21.15.167:3300/0 conn(0x7f05fc15b6e0 msgr2=0x7f05fc071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:35.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.211+0000 7f05f9d9b640 1 --2- 172.21.15.83:0/2225430699 >> v2:172.21.15.167:3300/0 conn(0x7f05fc15b6e0 0x7f05fc071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:35.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.211+0000 7f05f9d9b640 1 -- 172.21.15.83:0/2225430699 >> v1:172.21.15.167:6789/0 conn(0x7f05fc15bac0 legacy=0x7f05fc0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:35.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.211+0000 7f05f9d9b640 1 -- 172.21.15.83:0/2225430699 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05fc070fd0 con 0x7f05fc070470 2024-04-17T10:41:35.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.211+0000 7f05f9d9b640 1 -- 172.21.15.83:0/2225430699 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f05f4001a20 con 0x7f05fc070470 2024-04-17T10:41:35.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.211+0000 7f05f9d9b640 1 -- 172.21.15.83:0/2225430699 >> v1:172.21.15.83:6789/0 conn(0x7f05fc070470 legacy=0x7f05fc06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:35.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.211+0000 7f05f9d9b640 1 --2- 172.21.15.83:0/2225430699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05e4003a30 0x7f05e4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:35.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.211+0000 7f05f9d9b640 1 --2- 172.21.15.83:0/2225430699 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05e4004810 0x7f05e4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:35.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.211+0000 7f05f9d9b640 1 --2- 172.21.15.83:0/2225430699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05e40051d0 0x7f05e400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:35.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.211+0000 7f05f9d9b640 1 -- 172.21.15.83:0/2225430699 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc070cf0 con 0x7f05e4003a30 2024-04-17T10:41:35.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.211+0000 7f05f9d9b640 1 -- 172.21.15.83:0/2225430699 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e400c100 con 0x7f05e40051d0 2024-04-17T10:41:35.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.211+0000 7f05f9d9b640 1 -- 172.21.15.83:0/2225430699 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e400c300 con 0x7f05e4004810 2024-04-17T10:41:35.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.211+0000 7f05fad9d640 1 --2- 172.21.15.83:0/2225430699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05e4003a30 0x7f05e4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:35.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.211+0000 7f05fad9d640 1 -- 172.21.15.83:0/2225430699 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05e4004810 msgr2=0x7f05e4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:35.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.211+0000 7f05fad9d640 1 --2- 172.21.15.83:0/2225430699 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05e4004810 0x7f05e4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:35.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.211+0000 7f05fad9d640 1 -- 172.21.15.83:0/2225430699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05e40051d0 msgr2=0x7f05e400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:35.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.211+0000 7f05fad9d640 1 --2- 172.21.15.83:0/2225430699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05e40051d0 0x7f05e400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:35.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.211+0000 7f05fad9d640 1 -- 172.21.15.83:0/2225430699 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e4003530 con 0x7f05e4003a30 2024-04-17T10:41:35.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.211+0000 7f05fb59e640 1 -- 172.21.15.83:0/2225430699 reap_dead start 2024-04-17T10:41:35.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.215+0000 7f05fad9d640 1 --2- 172.21.15.83:0/2225430699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05e4003a30 0x7f05e4003e50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f05f0009ae0 tx=0x7f05f001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad445ef61283017 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:35.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.215+0000 7f05f9d9b640 1 -- 172.21.15.83:0/2225430699 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f00b2070 con 0x7f05e4003a30 2024-04-17T10:41:35.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.215+0000 7f05f9d9b640 1 -- 172.21.15.83:0/2225430699 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f00a5550 con 0x7f05e4003a30 2024-04-17T10:41:35.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.215+0000 7f05f9d9b640 1 -- 172.21.15.83:0/2225430699 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f00096b0 con 0x7f05e4003a30 2024-04-17T10:41:35.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.215+0000 7f0601052640 1 -- 172.21.15.83:0/2225430699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05e4003a30 msgr2=0x7f05e4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:35.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.215+0000 7f0601052640 1 --2- 172.21.15.83:0/2225430699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05e4003a30 0x7f05e4003e50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f05f0009ae0 tx=0x7f05f001ca90 comp rx=0 tx=0).stop 2024-04-17T10:41:35.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.215+0000 7f0601052640 1 -- 172.21.15.83:0/2225430699 shutdown_connections 2024-04-17T10:41:35.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.215+0000 7f0601052640 1 --2- 172.21.15.83:0/2225430699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05e4003a30 0x7f05e4003e50 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:35.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.215+0000 7f0601052640 1 -- 172.21.15.83:0/2225430699 >> 172.21.15.83:0/2225430699 conn(0x7f05fc0b15b0 msgr2=0x7f05fc0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:35.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.215+0000 7f0601052640 1 -- 172.21.15.83:0/2225430699 shutdown_connections 2024-04-17T10:41:35.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.215+0000 7f0601052640 1 -- 172.21.15.83:0/2225430699 wait complete. 2024-04-17T10:41:35.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.215+0000 7f0601052640 1 Processor -- start 2024-04-17T10:41:35.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.215+0000 7f0601052640 1 -- start start 2024-04-17T10:41:35.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.219+0000 7f0601052640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05e4003a30 0x7f05fc15d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:35.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.219+0000 7f0601052640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05fc153890 0x7f05fc15c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:35.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.219+0000 7f0601052640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05fc15c6e0 0x7f05fc15cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:35.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.219+0000 7f0601052640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc06b7a0 con 0x7f05e4003a30 2024-04-17T10:41:35.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.219+0000 7f0601052640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc06b620 con 0x7f05fc15c6e0 2024-04-17T10:41:35.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.219+0000 7f0601052640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc06c6d0 con 0x7f05fc153890 2024-04-17T10:41:35.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.219+0000 7f05fad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05e4003a30 0x7f05fc15d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:35.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.219+0000 7f05fad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05e4003a30 0x7f05fc15d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40666/0 (socket says 172.21.15.83:40666) 2024-04-17T10:41:35.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.219+0000 7f05fad9d640 1 -- 172.21.15.83:0/14029895 learned_addr learned my addr 172.21.15.83:0/14029895 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:35.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.219+0000 7f05fad9d640 1 -- 172.21.15.83:0/14029895 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05fc153890 msgr2=0x7f05fc15c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:35.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.219+0000 7f05fa59c640 1 --2- 172.21.15.83:0/14029895 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05fc153890 0x7f05fc15c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:35.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.219+0000 7f05fb59e640 1 --2- 172.21.15.83:0/14029895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05fc15c6e0 0x7f05fc15cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:35.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.219+0000 7f05fad9d640 1 --2- 172.21.15.83:0/14029895 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05fc153890 0x7f05fc15c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:35.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.219+0000 7f05fad9d640 1 -- 172.21.15.83:0/14029895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05fc15c6e0 msgr2=0x7f05fc15cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:35.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.219+0000 7f05fad9d640 1 --2- 172.21.15.83:0/14029895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05fc15c6e0 0x7f05fc15cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:35.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.219+0000 7f05fad9d640 1 -- 172.21.15.83:0/14029895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05fc154030 con 0x7f05e4003a30 2024-04-17T10:41:35.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.219+0000 7f05fad9d640 1 --2- 172.21.15.83:0/14029895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05e4003a30 0x7f05fc15d7d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f05f001c8c0 tx=0x7f05f0019830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:35.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.219+0000 7f05e37fe640 1 -- 172.21.15.83:0/14029895 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f00b2040 con 0x7f05e4003a30 2024-04-17T10:41:35.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.219+0000 7f05e37fe640 1 -- 172.21.15.83:0/14029895 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f000f000 con 0x7f05e4003a30 2024-04-17T10:41:35.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.219+0000 7f0601052640 1 -- 172.21.15.83:0/14029895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05fc154310 con 0x7f05e4003a30 2024-04-17T10:41:35.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.219+0000 7f05e37fe640 1 -- 172.21.15.83:0/14029895 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f0004020 con 0x7f05e4003a30 2024-04-17T10:41:35.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.219+0000 7f0601052640 1 -- 172.21.15.83:0/14029895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05fc1548d0 con 0x7f05e4003a30 2024-04-17T10:41:35.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.219+0000 7f0601052640 1 -- 172.21.15.83:0/14029895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05fc074f90 con 0x7f05e4003a30 2024-04-17T10:41:35.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.219+0000 7f05e37fe640 1 -- 172.21.15.83:0/14029895 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f05f00bf030 con 0x7f05e4003a30 2024-04-17T10:41:35.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.219+0000 7f05e37fe640 1 --2- 172.21.15.83:0/14029895 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f05e8066f90 0x7f05e8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:35.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.219+0000 7f05e37fe640 1 -- 172.21.15.83:0/14029895 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f05f011e7a0 con 0x7f05e4003a30 2024-04-17T10:41:35.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.219+0000 7f05fa59c640 1 --2- 172.21.15.83:0/14029895 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f05e8066f90 0x7f05e8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:35.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.219+0000 7f05fa59c640 1 --2- 172.21.15.83:0/14029895 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f05e8066f90 0x7f05e8069450 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f05fc157410 tx=0x7f05ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:35.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.223+0000 7f05e37fe640 1 -- 172.21.15.83:0/14029895 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f05f00c6060 con 0x7f05e4003a30 2024-04-17T10:41:35.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.387+0000 7f0601052640 1 -- 172.21.15.83:0/14029895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f05fc157120 con 0x7f05e4003a30 2024-04-17T10:41:35.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.387+0000 7f05e37fe640 1 -- 172.21.15.83:0/14029895 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f05f00167a0 con 0x7f05e4003a30 2024-04-17T10:41:35.384 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:41:35.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.387+0000 7f0601052640 1 -- 172.21.15.83:0/14029895 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f05e8066f90 msgr2=0x7f05e8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:35.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.387+0000 7f0601052640 1 --2- 172.21.15.83:0/14029895 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f05e8066f90 0x7f05e8069450 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f05fc157410 tx=0x7f05ec009040 comp rx=0 tx=0).stop 2024-04-17T10:41:35.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.387+0000 7f0601052640 1 -- 172.21.15.83:0/14029895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05e4003a30 msgr2=0x7f05fc15d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:35.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.387+0000 7f0601052640 1 --2- 172.21.15.83:0/14029895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05e4003a30 0x7f05fc15d7d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f05f001c8c0 tx=0x7f05f0019830 comp rx=0 tx=0).stop 2024-04-17T10:41:35.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.387+0000 7f0601052640 1 -- 172.21.15.83:0/14029895 shutdown_connections 2024-04-17T10:41:35.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.387+0000 7f0601052640 1 --2- 172.21.15.83:0/14029895 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f05e8066f90 0x7f05e8069450 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:35.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.387+0000 7f0601052640 1 --2- 172.21.15.83:0/14029895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05fc15c6e0 0x7f05fc15cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:35.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.387+0000 7f0601052640 1 --2- 172.21.15.83:0/14029895 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05fc153890 0x7f05fc15c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:35.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.387+0000 7f0601052640 1 --2- 172.21.15.83:0/14029895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05e4003a30 0x7f05fc15d7d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:35.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.387+0000 7f0601052640 1 -- 172.21.15.83:0/14029895 >> 172.21.15.83:0/14029895 conn(0x7f05fc0b15b0 msgr2=0x7f05fc157680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:35.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.387+0000 7f0601052640 1 -- 172.21.15.83:0/14029895 shutdown_connections 2024-04-17T10:41:35.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.387+0000 7f0601052640 1 -- 172.21.15.83:0/14029895 wait complete. 2024-04-17T10:41:35.405 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-17T10:41:35.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.523+0000 7f8a29feb640 1 Processor -- start 2024-04-17T10:41:35.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.523+0000 7f8a29feb640 1 -- start start 2024-04-17T10:41:35.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.523+0000 7f8a29feb640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a2415f1d0 0x7f8a2415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:35.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.523+0000 7f8a29feb640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8a240d22d0 0x7f8a240ae520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:35.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.523+0000 7f8a29feb640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a2415f5b0 con 0x7f8a240b2de0 2024-04-17T10:41:35.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.523+0000 7f8a29feb640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a240b2620 con 0x7f8a240d22d0 2024-04-17T10:41:35.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.523+0000 7f8a29feb640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a240b2340 con 0x7f8a2415f1d0 2024-04-17T10:41:35.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.523+0000 7f8a23fff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8a240d22d0 0x7f8a240ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:35.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.523+0000 7f8a237fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a2415f1d0 0x7f8a2415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:35.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.523+0000 7f8a23fff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8a240d22d0 0x7f8a240ae520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:54888/0 (socket says 172.21.15.83:54888) 2024-04-17T10:41:35.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.523+0000 7f8a23fff640 1 -- 172.21.15.83:0/1300254218 learned_addr learned my addr 172.21.15.83:0/1300254218 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:35.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.523+0000 7f8a22ffd640 1 --1- 172.21.15.83:0/1300254218 >> v1:172.21.15.167:6789/0 conn(0x7f8a240b2de0 0x7f8a2415fd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:37504/0 (socket says 172.21.15.83:37504) 2024-04-17T10:41:35.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.523+0000 7f8a23fff640 1 -- 172.21.15.83:0/1300254218 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a2415f1d0 msgr2=0x7f8a2415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:35.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.523+0000 7f8a23fff640 1 --2- 172.21.15.83:0/1300254218 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a2415f1d0 0x7f8a2415f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:35.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.523+0000 7f8a23fff640 1 -- 172.21.15.83:0/1300254218 >> v1:172.21.15.167:6789/0 conn(0x7f8a240b2de0 legacy=0x7f8a2415fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:35.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.523+0000 7f8a23fff640 1 -- 172.21.15.83:0/1300254218 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a240b1760 con 0x7f8a240d22d0 2024-04-17T10:41:35.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.527+0000 7f8a23fff640 1 --2- 172.21.15.83:0/1300254218 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a240d22d0 0x7f8a240ae520 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c009e50 tx=0x7f8a1c01ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83b389bc716398a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:35.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.527+0000 7f8a227fc640 1 -- 172.21.15.83:0/1300254218 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0a58d0 con 0x7f8a240d22d0 2024-04-17T10:41:35.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.527+0000 7f8a227fc640 1 -- 172.21.15.83:0/1300254218 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a1c0a5a70 con 0x7f8a240d22d0 2024-04-17T10:41:35.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.527+0000 7f8a227fc640 1 -- 172.21.15.83:0/1300254218 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0a5e40 con 0x7f8a240d22d0 2024-04-17T10:41:35.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.527+0000 7f8a29feb640 1 -- 172.21.15.83:0/1300254218 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a240d22d0 msgr2=0x7f8a240ae520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:35.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.527+0000 7f8a29feb640 1 --2- 172.21.15.83:0/1300254218 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a240d22d0 0x7f8a240ae520 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c009e50 tx=0x7f8a1c01ca00 comp rx=0 tx=0).stop 2024-04-17T10:41:35.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.527+0000 7f8a29feb640 1 -- 172.21.15.83:0/1300254218 shutdown_connections 2024-04-17T10:41:35.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.527+0000 7f8a29feb640 1 --2- 172.21.15.83:0/1300254218 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a240d22d0 0x7f8a240ae520 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:35.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.527+0000 7f8a29feb640 1 --2- 172.21.15.83:0/1300254218 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a2415f1d0 0x7f8a2415f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:35.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.527+0000 7f8a29feb640 1 -- 172.21.15.83:0/1300254218 >> 172.21.15.83:0/1300254218 conn(0x7f8a240d17b0 msgr2=0x7f8a240d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:35.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.527+0000 7f8a29feb640 1 -- 172.21.15.83:0/1300254218 shutdown_connections 2024-04-17T10:41:35.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.527+0000 7f8a29feb640 1 -- 172.21.15.83:0/1300254218 wait complete. 2024-04-17T10:41:35.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.527+0000 7f8a29feb640 1 Processor -- start 2024-04-17T10:41:35.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.527+0000 7f8a29feb640 1 -- start start 2024-04-17T10:41:35.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.531+0000 7f8a29feb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a240d22d0 0x7f8a240cc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:35.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.531+0000 7f8a29feb640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a2415f1d0 0x7f8a2406d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:35.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.531+0000 7f8a29feb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a24165fc0 0x7f8a240cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:35.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.531+0000 7f8a29feb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a240af270 con 0x7f8a240d22d0 2024-04-17T10:41:35.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.531+0000 7f8a29feb640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a24160690 con 0x7f8a24165fc0 2024-04-17T10:41:35.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.531+0000 7f8a29feb640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a240af3f0 con 0x7f8a2415f1d0 2024-04-17T10:41:35.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.531+0000 7f8a237fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a240d22d0 0x7f8a240cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:35.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.531+0000 7f8a237fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a240d22d0 0x7f8a240cc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40680/0 (socket says 172.21.15.83:40680) 2024-04-17T10:41:35.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.531+0000 7f8a237fe640 1 -- 172.21.15.83:0/129976462 learned_addr learned my addr 172.21.15.83:0/129976462 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:35.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.531+0000 7f8a22ffd640 1 --2- 172.21.15.83:0/129976462 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a2415f1d0 0x7f8a2406d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:35.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.531+0000 7f8a237fe640 1 -- 172.21.15.83:0/129976462 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a2415f1d0 msgr2=0x7f8a2406d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:35.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.531+0000 7f8a237fe640 1 --2- 172.21.15.83:0/129976462 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a2415f1d0 0x7f8a2406d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:35.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.531+0000 7f8a23fff640 1 --2- 172.21.15.83:0/129976462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a24165fc0 0x7f8a240cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:35.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.531+0000 7f8a237fe640 1 -- 172.21.15.83:0/129976462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a24165fc0 msgr2=0x7f8a240cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:35.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.531+0000 7f8a237fe640 1 --2- 172.21.15.83:0/129976462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a24165fc0 0x7f8a240cf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:35.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.531+0000 7f8a237fe640 1 -- 172.21.15.83:0/129976462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a24166470 con 0x7f8a240d22d0 2024-04-17T10:41:35.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.531+0000 7f8a237fe640 1 --2- 172.21.15.83:0/129976462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a240d22d0 0x7f8a240cc150 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400e870 tx=0x7f8a1400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:35.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.531+0000 7f8a0bfff640 1 -- 172.21.15.83:0/129976462 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a14017070 con 0x7f8a240d22d0 2024-04-17T10:41:35.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.531+0000 7f8a0bfff640 1 -- 172.21.15.83:0/129976462 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a1400c940 con 0x7f8a240d22d0 2024-04-17T10:41:35.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.531+0000 7f8a0bfff640 1 -- 172.21.15.83:0/129976462 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1400ccf0 con 0x7f8a240d22d0 2024-04-17T10:41:35.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.531+0000 7f8a29feb640 1 -- 172.21.15.83:0/129976462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a24166710 con 0x7f8a240d22d0 2024-04-17T10:41:35.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.531+0000 7f8a29feb640 1 -- 172.21.15.83:0/129976462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a2415cd00 con 0x7f8a240d22d0 2024-04-17T10:41:35.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.531+0000 7f8a29feb640 1 -- 172.21.15.83:0/129976462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a2415c830 con 0x7f8a240d22d0 2024-04-17T10:41:35.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.531+0000 7f8a0bfff640 1 -- 172.21.15.83:0/129976462 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8a1401a070 con 0x7f8a240d22d0 2024-04-17T10:41:35.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.531+0000 7f8a0bfff640 1 --2- 172.21.15.83:0/129976462 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8a10066f90 0x7f8a10069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:35.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.531+0000 7f8a0bfff640 1 -- 172.21.15.83:0/129976462 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8a14084b90 con 0x7f8a240d22d0 2024-04-17T10:41:35.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.531+0000 7f8a22ffd640 1 --2- 172.21.15.83:0/129976462 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8a10066f90 0x7f8a10069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:35.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.535+0000 7f8a22ffd640 1 --2- 172.21.15.83:0/129976462 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8a10066f90 0x7f8a10069450 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8a240d0a40 tx=0x7f8a18009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:35.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.535+0000 7f8a0bfff640 1 -- 172.21.15.83:0/129976462 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8a14055ed0 con 0x7f8a240d22d0 2024-04-17T10:41:35.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.683+0000 7f8a29feb640 1 -- 172.21.15.83:0/129976462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8a24074f90 con 0x7f8a240d22d0 2024-04-17T10:41:35.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.683+0000 7f8a0bfff640 1 -- 172.21.15.83:0/129976462 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v42) v1 ==== 84+0+65 (secure 0 0 0) 0x7f8a140557d0 con 0x7f8a240d22d0 2024-04-17T10:41:35.683 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:41:35.683 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-04-17T10:41:35.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:41:35.683 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2024-04-17T10:41:35.683 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2024-04-17T10:41:35.683 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:41:35.683 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-04-17T10:41:35.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.687+0000 7f8a29feb640 1 -- 172.21.15.83:0/129976462 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8a10066f90 msgr2=0x7f8a10069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:35.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.687+0000 7f8a29feb640 1 --2- 172.21.15.83:0/129976462 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8a10066f90 0x7f8a10069450 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8a240d0a40 tx=0x7f8a18009040 comp rx=0 tx=0).stop 2024-04-17T10:41:35.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.687+0000 7f8a29feb640 1 -- 172.21.15.83:0/129976462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a240d22d0 msgr2=0x7f8a240cc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:35.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.687+0000 7f8a29feb640 1 --2- 172.21.15.83:0/129976462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a240d22d0 0x7f8a240cc150 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400e870 tx=0x7f8a1400ec70 comp rx=0 tx=0).stop 2024-04-17T10:41:35.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.687+0000 7f8a29feb640 1 -- 172.21.15.83:0/129976462 shutdown_connections 2024-04-17T10:41:35.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.687+0000 7f8a29feb640 1 --2- 172.21.15.83:0/129976462 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8a10066f90 0x7f8a10069450 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:35.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.687+0000 7f8a29feb640 1 --2- 172.21.15.83:0/129976462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a24165fc0 0x7f8a240cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:35.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.687+0000 7f8a29feb640 1 --2- 172.21.15.83:0/129976462 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a2415f1d0 0x7f8a2406d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:35.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.687+0000 7f8a29feb640 1 --2- 172.21.15.83:0/129976462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a240d22d0 0x7f8a240cc150 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:35.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.687+0000 7f8a29feb640 1 -- 172.21.15.83:0/129976462 >> 172.21.15.83:0/129976462 conn(0x7f8a240d17b0 msgr2=0x7f8a2415c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:35.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.687+0000 7f8a29feb640 1 -- 172.21.15.83:0/129976462 shutdown_connections 2024-04-17T10:41:35.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.687+0000 7f8a29feb640 1 -- 172.21.15.83:0/129976462 wait complete. 2024-04-17T10:41:35.700 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-17T10:41:35.700 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-17T10:41:35.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.827+0000 7f88d0689640 1 Processor -- start 2024-04-17T10:41:35.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.827+0000 7f88d0689640 1 -- start start 2024-04-17T10:41:35.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.827+0000 7f88d0689640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f88c815aa90 0x7f88c806cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:35.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.827+0000 7f88d0689640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f88c815ae70 0x7f88c80720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:35.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.827+0000 7f88d0689640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88c80cc350 con 0x7f88c806b4a0 2024-04-17T10:41:35.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.827+0000 7f88d0689640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88c8066eb0 con 0x7f88c815aa90 2024-04-17T10:41:35.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.827+0000 7f88d0689640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88c80cc630 con 0x7f88c815ae70 2024-04-17T10:41:35.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.827+0000 7f88ce425640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f88c815aa90 0x7f88c806cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:35.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.827+0000 7f88cdc24640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f88c806b4a0 0x7f88c806abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53744/0 (socket says 172.21.15.83:53744) 2024-04-17T10:41:35.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.827+0000 7f88cdc24640 1 -- 172.21.15.83:0/397312553 learned_addr learned my addr 172.21.15.83:0/397312553 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:35.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.827+0000 7f88cec26640 1 --2- 172.21.15.83:0/397312553 >> v2:172.21.15.167:3300/0 conn(0x7f88c815ae70 0x7f88c80720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:35.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.827+0000 7f88ce425640 1 -- 172.21.15.83:0/397312553 >> v2:172.21.15.167:3300/0 conn(0x7f88c815ae70 msgr2=0x7f88c80720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:35.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.827+0000 7f88ce425640 1 --2- 172.21.15.83:0/397312553 >> v2:172.21.15.167:3300/0 conn(0x7f88c815ae70 0x7f88c80720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:35.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.827+0000 7f88ce425640 1 -- 172.21.15.83:0/397312553 >> v1:172.21.15.83:6789/0 conn(0x7f88c806b4a0 legacy=0x7f88c806abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:35.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.827+0000 7f88ce425640 1 -- 172.21.15.83:0/397312553 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c8066b40 con 0x7f88c815aa90 2024-04-17T10:41:35.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.827+0000 7f88cd423640 1 -- 172.21.15.83:0/397312553 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3690752884 0 0) 0x7f88c80cc350 con 0x7f88c806b4a0 2024-04-17T10:41:35.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.827+0000 7f88cec26640 1 --2- 172.21.15.83:0/397312553 >> v2:172.21.15.167:3300/0 conn(0x7f88c815ae70 0x7f88c80720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:35.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88ce425640 1 --2- 172.21.15.83:0/397312553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c815aa90 0x7f88c806cd60 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f88c806efb0 tx=0x7f88c401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e8449b3e42ce3a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:35.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88cd423640 1 -- 172.21.15.83:0/397312553 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c40aa600 con 0x7f88c815aa90 2024-04-17T10:41:35.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88cd423640 1 -- 172.21.15.83:0/397312553 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c40af070 con 0x7f88c815aa90 2024-04-17T10:41:35.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88cd423640 1 -- 172.21.15.83:0/397312553 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c40aa9c0 con 0x7f88c815aa90 2024-04-17T10:41:35.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88d0689640 1 -- 172.21.15.83:0/397312553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c815aa90 msgr2=0x7f88c806cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:35.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88d0689640 1 --2- 172.21.15.83:0/397312553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c815aa90 0x7f88c806cd60 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f88c806efb0 tx=0x7f88c401c480 comp rx=0 tx=0).stop 2024-04-17T10:41:35.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88d0689640 1 -- 172.21.15.83:0/397312553 shutdown_connections 2024-04-17T10:41:35.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88d0689640 1 --2- 172.21.15.83:0/397312553 >> v2:172.21.15.167:3300/0 conn(0x7f88c815ae70 0x7f88c80720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:35.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88d0689640 1 --2- 172.21.15.83:0/397312553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c815aa90 0x7f88c806cd60 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:35.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88d0689640 1 -- 172.21.15.83:0/397312553 >> 172.21.15.83:0/397312553 conn(0x7f88c8071200 msgr2=0x7f88c80b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:35.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88d0689640 1 -- 172.21.15.83:0/397312553 shutdown_connections 2024-04-17T10:41:35.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88d0689640 1 -- 172.21.15.83:0/397312553 wait complete. 2024-04-17T10:41:35.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88d0689640 1 Processor -- start 2024-04-17T10:41:35.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88d0689640 1 -- start start 2024-04-17T10:41:35.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88d0689640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88c806b4a0 0x7f88c81577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:35.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88d0689640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88c815aa90 0x7f88c81566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:35.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88d0689640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c815ae70 0x7f88c81424f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:35.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88d0689640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c80b28f0 con 0x7f88c815ae70 2024-04-17T10:41:35.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88d0689640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c80b2770 con 0x7f88c806b4a0 2024-04-17T10:41:35.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88d0689640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c80b2a70 con 0x7f88c815aa90 2024-04-17T10:41:35.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88cec26640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c815ae70 0x7f88c81424f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:35.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88cec26640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c815ae70 0x7f88c81424f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40704/0 (socket says 172.21.15.83:40704) 2024-04-17T10:41:35.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88cec26640 1 -- 172.21.15.83:0/140967264 learned_addr learned my addr 172.21.15.83:0/140967264 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:35.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88ce425640 1 --2- 172.21.15.83:0/140967264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88c806b4a0 0x7f88c81577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:35.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88cdc24640 1 --2- 172.21.15.83:0/140967264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88c815aa90 0x7f88c81566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:35.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88cec26640 1 -- 172.21.15.83:0/140967264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88c815aa90 msgr2=0x7f88c81566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:35.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88cec26640 1 --2- 172.21.15.83:0/140967264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88c815aa90 0x7f88c81566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:35.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88cec26640 1 -- 172.21.15.83:0/140967264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88c806b4a0 msgr2=0x7f88c81577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:35.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88cec26640 1 --2- 172.21.15.83:0/140967264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88c806b4a0 0x7f88c81577d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:35.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88cec26640 1 -- 172.21.15.83:0/140967264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c8159340 con 0x7f88c815ae70 2024-04-17T10:41:35.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88cdc24640 1 --2- 172.21.15.83:0/140967264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88c815aa90 0x7f88c81566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:35.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88cec26640 1 --2- 172.21.15.83:0/140967264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c815ae70 0x7f88c81424f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f88c000cef0 tx=0x7f88c000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:35.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.831+0000 7f88beffd640 1 -- 172.21.15.83:0/140967264 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c001a070 con 0x7f88c815ae70 2024-04-17T10:41:35.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.835+0000 7f88beffd640 1 -- 172.21.15.83:0/140967264 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c000d860 con 0x7f88c815ae70 2024-04-17T10:41:35.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.835+0000 7f88d0689640 1 -- 172.21.15.83:0/140967264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c8157d10 con 0x7f88c815ae70 2024-04-17T10:41:35.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.835+0000 7f88beffd640 1 -- 172.21.15.83:0/140967264 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c000dc50 con 0x7f88c815ae70 2024-04-17T10:41:35.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.835+0000 7f88d0689640 1 -- 172.21.15.83:0/140967264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c81595e0 con 0x7f88c815ae70 2024-04-17T10:41:35.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.835+0000 7f88d0689640 1 -- 172.21.15.83:0/140967264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88c814f510 con 0x7f88c815ae70 2024-04-17T10:41:35.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.835+0000 7f88beffd640 1 -- 172.21.15.83:0/140967264 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f88c001d070 con 0x7f88c815ae70 2024-04-17T10:41:35.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.835+0000 7f88beffd640 1 --2- 172.21.15.83:0/140967264 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f88ac066f60 0x7f88ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:35.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.835+0000 7f88beffd640 1 -- 172.21.15.83:0/140967264 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f88c0085500 con 0x7f88c815ae70 2024-04-17T10:41:35.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.835+0000 7f88ce425640 1 --2- 172.21.15.83:0/140967264 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f88ac066f60 0x7f88ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:35.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.835+0000 7f88ce425640 1 --2- 172.21.15.83:0/140967264 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f88ac066f60 0x7f88ac069420 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f88c40ae000 tx=0x7f88c40a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:35.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.839+0000 7f88beffd640 1 -- 172.21.15.83:0/140967264 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f88c00569b0 con 0x7f88c815ae70 2024-04-17T10:41:35.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:35.991+0000 7f88d0689640 1 -- 172.21.15.83:0/140967264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f88c8075000 con 0x7f88c815ae70 2024-04-17T10:41:36.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.399+0000 7f88beffd640 1 -- 172.21.15.83:0/140967264 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v43) v1 ==== 138+0+0 (secure 0 0 0) 0x7f88c00562b0 con 0x7f88c815ae70 2024-04-17T10:41:36.398 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2024-04-17T10:41:36.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.403+0000 7f88d0689640 1 -- 172.21.15.83:0/140967264 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f88ac066f60 msgr2=0x7f88ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:36.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.403+0000 7f88d0689640 1 --2- 172.21.15.83:0/140967264 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f88ac066f60 0x7f88ac069420 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f88c40ae000 tx=0x7f88c40a7000 comp rx=0 tx=0).stop 2024-04-17T10:41:36.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.403+0000 7f88d0689640 1 -- 172.21.15.83:0/140967264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c815ae70 msgr2=0x7f88c81424f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:36.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.403+0000 7f88d0689640 1 --2- 172.21.15.83:0/140967264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c815ae70 0x7f88c81424f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f88c000cef0 tx=0x7f88c000a710 comp rx=0 tx=0).stop 2024-04-17T10:41:36.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.403+0000 7f88d0689640 1 -- 172.21.15.83:0/140967264 shutdown_connections 2024-04-17T10:41:36.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.403+0000 7f88d0689640 1 --2- 172.21.15.83:0/140967264 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f88ac066f60 0x7f88ac069420 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:36.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.403+0000 7f88d0689640 1 --2- 172.21.15.83:0/140967264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c815ae70 0x7f88c81424f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:36.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.403+0000 7f88d0689640 1 --2- 172.21.15.83:0/140967264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88c815aa90 0x7f88c81566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:36.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.403+0000 7f88d0689640 1 --2- 172.21.15.83:0/140967264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88c806b4a0 0x7f88c81577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:36.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.403+0000 7f88d0689640 1 -- 172.21.15.83:0/140967264 >> 172.21.15.83:0/140967264 conn(0x7f88c8071200 msgr2=0x7f88c806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:36.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.403+0000 7f88d0689640 1 -- 172.21.15.83:0/140967264 shutdown_connections 2024-04-17T10:41:36.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.403+0000 7f88d0689640 1 -- 172.21.15.83:0/140967264 wait complete. 2024-04-17T10:41:36.428 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-17T10:41:36.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.563+0000 7f4e5e3b5640 1 Processor -- start 2024-04-17T10:41:36.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.563+0000 7f4e5e3b5640 1 -- start start 2024-04-17T10:41:36.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.567+0000 7f4e5e3b5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e5806ae10 0x7f4e5806b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:36.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.567+0000 7f4e5e3b5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4e5815e330 0x7f4e5815e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:36.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.567+0000 7f4e5e3b5640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e581625d0 con 0x7f4e580ad840 2024-04-17T10:41:36.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.567+0000 7f4e5e3b5640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e5806b730 con 0x7f4e5815e330 2024-04-17T10:41:36.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.567+0000 7f4e5e3b5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e58161810 con 0x7f4e5806ae10 2024-04-17T10:41:36.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.567+0000 7f4e57fff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e5806ae10 0x7f4e5806b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:36.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.567+0000 7f4e577fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4e5815e330 0x7f4e5815e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:36.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.567+0000 7f4e5c952640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f4e580ad840 0x7f4e580adcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:37514/0 (socket says 172.21.15.83:37514) 2024-04-17T10:41:36.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.567+0000 7f4e577fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4e5815e330 0x7f4e5815e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:54922/0 (socket says 172.21.15.83:54922) 2024-04-17T10:41:36.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.567+0000 7f4e5c952640 1 -- 172.21.15.83:0/4280138299 learned_addr learned my addr 172.21.15.83:0/4280138299 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:36.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.567+0000 7f4e57fff640 1 -- 172.21.15.83:0/4280138299 >> v2:172.21.15.167:3300/0 conn(0x7f4e5815e330 msgr2=0x7f4e5815e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:36.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.567+0000 7f4e57fff640 1 --2- 172.21.15.83:0/4280138299 >> v2:172.21.15.167:3300/0 conn(0x7f4e5815e330 0x7f4e5815e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:36.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.567+0000 7f4e56ffd640 1 -- 172.21.15.83:0/4280138299 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1981633923 0 0) 0x7f4e581625d0 con 0x7f4e580ad840 2024-04-17T10:41:36.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.567+0000 7f4e57fff640 1 -- 172.21.15.83:0/4280138299 >> v1:172.21.15.167:6789/0 conn(0x7f4e580ad840 legacy=0x7f4e580adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:36.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.567+0000 7f4e57fff640 1 -- 172.21.15.83:0/4280138299 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e58161af0 con 0x7f4e5806ae10 2024-04-17T10:41:36.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.571+0000 7f4e57fff640 1 --2- 172.21.15.83:0/4280138299 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e5806ae10 0x7f4e5806b1f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4e480097c0 tx=0x7f4e4801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62ed88dba5b38fab server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:36.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.571+0000 7f4e56ffd640 1 -- 172.21.15.83:0/4280138299 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e480b0070 con 0x7f4e5806ae10 2024-04-17T10:41:36.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.571+0000 7f4e56ffd640 1 -- 172.21.15.83:0/4280138299 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e480ab600 con 0x7f4e5806ae10 2024-04-17T10:41:36.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.571+0000 7f4e56ffd640 1 -- 172.21.15.83:0/4280138299 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e480ab9d0 con 0x7f4e5806ae10 2024-04-17T10:41:36.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.571+0000 7f4e5e3b5640 1 -- 172.21.15.83:0/4280138299 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e5806ae10 msgr2=0x7f4e5806b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:36.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.571+0000 7f4e5e3b5640 1 --2- 172.21.15.83:0/4280138299 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e5806ae10 0x7f4e5806b1f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4e480097c0 tx=0x7f4e4801c410 comp rx=0 tx=0).stop 2024-04-17T10:41:36.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.571+0000 7f4e5e3b5640 1 -- 172.21.15.83:0/4280138299 shutdown_connections 2024-04-17T10:41:36.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.571+0000 7f4e5e3b5640 1 --2- 172.21.15.83:0/4280138299 >> v2:172.21.15.167:3300/0 conn(0x7f4e5815e330 0x7f4e5815e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:36.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.571+0000 7f4e5e3b5640 1 --2- 172.21.15.83:0/4280138299 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e5806ae10 0x7f4e5806b1f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:36.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.571+0000 7f4e5e3b5640 1 -- 172.21.15.83:0/4280138299 >> 172.21.15.83:0/4280138299 conn(0x7f4e580b15b0 msgr2=0x7f4e580b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:36.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.571+0000 7f4e5e3b5640 1 -- 172.21.15.83:0/4280138299 shutdown_connections 2024-04-17T10:41:36.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.571+0000 7f4e5e3b5640 1 -- 172.21.15.83:0/4280138299 wait complete. 2024-04-17T10:41:36.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.571+0000 7f4e5e3b5640 1 Processor -- start 2024-04-17T10:41:36.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.571+0000 7f4e5e3b5640 1 -- start start 2024-04-17T10:41:36.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.571+0000 7f4e5e3b5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e5806ae10 0x7f4e5813b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:36.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.571+0000 7f4e5e3b5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e580ad840 0x7f4e581431a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:36.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.571+0000 7f4e5e3b5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e5815e330 0x7f4e581436e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:36.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.571+0000 7f4e5e3b5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e580cf470 con 0x7f4e5806ae10 2024-04-17T10:41:36.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.571+0000 7f4e5e3b5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e580cf2f0 con 0x7f4e580ad840 2024-04-17T10:41:36.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.571+0000 7f4e5e3b5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5815ecb0 con 0x7f4e5815e330 2024-04-17T10:41:36.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.571+0000 7f4e57fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e5806ae10 0x7f4e5813b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:36.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.571+0000 7f4e57fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e5806ae10 0x7f4e5813b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40708/0 (socket says 172.21.15.83:40708) 2024-04-17T10:41:36.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.571+0000 7f4e57fff640 1 -- 172.21.15.83:0/2810503275 learned_addr learned my addr 172.21.15.83:0/2810503275 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:36.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.571+0000 7f4e577fe640 1 --2- 172.21.15.83:0/2810503275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e580ad840 0x7f4e581431a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:36.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.571+0000 7f4e5c952640 1 --2- 172.21.15.83:0/2810503275 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e5815e330 0x7f4e581436e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:36.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.571+0000 7f4e57fff640 1 -- 172.21.15.83:0/2810503275 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e5815e330 msgr2=0x7f4e581436e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:36.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.571+0000 7f4e57fff640 1 --2- 172.21.15.83:0/2810503275 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e5815e330 0x7f4e581436e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:36.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.571+0000 7f4e57fff640 1 -- 172.21.15.83:0/2810503275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e580ad840 msgr2=0x7f4e581431a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:36.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.571+0000 7f4e57fff640 1 --2- 172.21.15.83:0/2810503275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e580ad840 0x7f4e581431a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:36.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.571+0000 7f4e57fff640 1 -- 172.21.15.83:0/2810503275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e5813c970 con 0x7f4e5806ae10 2024-04-17T10:41:36.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.571+0000 7f4e5c952640 1 --2- 172.21.15.83:0/2810503275 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e5815e330 0x7f4e581436e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:36.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.575+0000 7f4e57fff640 1 --2- 172.21.15.83:0/2810503275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e5806ae10 0x7f4e5813b420 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4e480ae040 tx=0x7f4e4801ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:36.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.575+0000 7f4e54ff9640 1 -- 172.21.15.83:0/2810503275 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e480bb070 con 0x7f4e5806ae10 2024-04-17T10:41:36.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.575+0000 7f4e54ff9640 1 -- 172.21.15.83:0/2810503275 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e48004020 con 0x7f4e5806ae10 2024-04-17T10:41:36.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.575+0000 7f4e5e3b5640 1 -- 172.21.15.83:0/2810503275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e5813cc50 con 0x7f4e5806ae10 2024-04-17T10:41:36.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.575+0000 7f4e54ff9640 1 -- 172.21.15.83:0/2810503275 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e480b0040 con 0x7f4e5806ae10 2024-04-17T10:41:36.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.575+0000 7f4e5e3b5640 1 -- 172.21.15.83:0/2810503275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e5815fdd0 con 0x7f4e5806ae10 2024-04-17T10:41:36.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.575+0000 7f4e5e3b5640 1 -- 172.21.15.83:0/2810503275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e58074f90 con 0x7f4e5806ae10 2024-04-17T10:41:36.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.575+0000 7f4e54ff9640 1 -- 172.21.15.83:0/2810503275 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4e48003620 con 0x7f4e5806ae10 2024-04-17T10:41:36.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.575+0000 7f4e54ff9640 1 --2- 172.21.15.83:0/2810503275 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4e44066f60 0x7f4e44069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:36.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.575+0000 7f4e54ff9640 1 -- 172.21.15.83:0/2810503275 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f4e4811f3e0 con 0x7f4e5806ae10 2024-04-17T10:41:36.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.575+0000 7f4e577fe640 1 --2- 172.21.15.83:0/2810503275 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4e44066f60 0x7f4e44069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:36.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.575+0000 7f4e577fe640 1 --2- 172.21.15.83:0/2810503275 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4e44066f60 0x7f4e44069420 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009a50 tx=0x7f4e4c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:36.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.579+0000 7f4e54ff9640 1 -- 172.21.15.83:0/2810503275 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4e48016090 con 0x7f4e5806ae10 2024-04-17T10:41:36.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:36.739+0000 7f4e5e3b5640 1 -- 172.21.15.83:0/2810503275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4e58070880 con 0x7f4e5806ae10 2024-04-17T10:41:36.797 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:41:36.799 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:41:37.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.415+0000 7f4e54ff9640 1 -- 172.21.15.83:0/2810503275 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v44) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4e480f0780 con 0x7f4e5806ae10 2024-04-17T10:41:37.414 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2024-04-17T10:41:37.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.419+0000 7f4e5e3b5640 1 -- 172.21.15.83:0/2810503275 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4e44066f60 msgr2=0x7f4e44069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:37.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.419+0000 7f4e5e3b5640 1 --2- 172.21.15.83:0/2810503275 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4e44066f60 0x7f4e44069420 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009a50 tx=0x7f4e4c008040 comp rx=0 tx=0).stop 2024-04-17T10:41:37.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.419+0000 7f4e5e3b5640 1 -- 172.21.15.83:0/2810503275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e5806ae10 msgr2=0x7f4e5813b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:37.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.419+0000 7f4e5e3b5640 1 --2- 172.21.15.83:0/2810503275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e5806ae10 0x7f4e5813b420 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4e480ae040 tx=0x7f4e4801ac50 comp rx=0 tx=0).stop 2024-04-17T10:41:37.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.419+0000 7f4e5e3b5640 1 -- 172.21.15.83:0/2810503275 shutdown_connections 2024-04-17T10:41:37.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.419+0000 7f4e5e3b5640 1 --2- 172.21.15.83:0/2810503275 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4e44066f60 0x7f4e44069420 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:37.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.419+0000 7f4e5e3b5640 1 --2- 172.21.15.83:0/2810503275 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e5815e330 0x7f4e581436e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:37.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.419+0000 7f4e5e3b5640 1 --2- 172.21.15.83:0/2810503275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e580ad840 0x7f4e581431a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:37.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.419+0000 7f4e5e3b5640 1 --2- 172.21.15.83:0/2810503275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e5806ae10 0x7f4e5813b420 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:37.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.419+0000 7f4e5e3b5640 1 -- 172.21.15.83:0/2810503275 >> 172.21.15.83:0/2810503275 conn(0x7f4e580b15b0 msgr2=0x7f4e580b3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:37.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.419+0000 7f4e5e3b5640 1 -- 172.21.15.83:0/2810503275 shutdown_connections 2024-04-17T10:41:37.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.419+0000 7f4e5e3b5640 1 -- 172.21.15.83:0/2810503275 wait complete. 2024-04-17T10:41:37.449 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-17T10:41:37.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.571+0000 7fecc45de640 1 Processor -- start 2024-04-17T10:41:37.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.571+0000 7fecc45de640 1 -- start start 2024-04-17T10:41:37.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.571+0000 7fecc45de640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecbc15e0f0 0x7fecbc0b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:37.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.571+0000 7fecc45de640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fecbc0ce400 0x7fecbc161c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:37.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.571+0000 7fecc45de640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecbc15e4d0 con 0x7fecbc0b5450 2024-04-17T10:41:37.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.571+0000 7fecc45de640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecbc162b40 con 0x7fecbc0ce400 2024-04-17T10:41:37.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.571+0000 7fecc45de640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc162580 con 0x7fecbc15e0f0 2024-04-17T10:41:37.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.571+0000 7fecc1b79640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fecbc0ce400 0x7fecbc161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:37.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.571+0000 7fecc1b79640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fecbc0ce400 0x7fecbc161c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51106/0 (socket says 172.21.15.83:51106) 2024-04-17T10:41:37.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.571+0000 7fecc1b79640 1 -- 172.21.15.83:0/956357013 learned_addr learned my addr 172.21.15.83:0/956357013 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:37.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.571+0000 7fecc237a640 1 --2- 172.21.15.83:0/956357013 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecbc15e0f0 0x7fecbc0b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:37.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.571+0000 7fecc1b79640 1 -- 172.21.15.83:0/956357013 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecbc15e0f0 msgr2=0x7fecbc0b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:37.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.571+0000 7fecc1b79640 1 --2- 172.21.15.83:0/956357013 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecbc15e0f0 0x7fecbc0b6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:37.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.571+0000 7fecc1b79640 1 -- 172.21.15.83:0/956357013 >> v1:172.21.15.83:6789/0 conn(0x7fecbc0b5450 legacy=0x7fecbc0d2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:37.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.571+0000 7fecc1b79640 1 -- 172.21.15.83:0/956357013 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecbc162860 con 0x7fecbc0ce400 2024-04-17T10:41:37.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.571+0000 7fecc237a640 1 --2- 172.21.15.83:0/956357013 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecbc15e0f0 0x7fecbc0b6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:37.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.571+0000 7fecc1378640 1 -- 172.21.15.83:0/956357013 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 506892689 0 0) 0x7fecbc15e4d0 con 0x7fecbc0b5450 2024-04-17T10:41:37.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.575+0000 7fecc1b79640 1 --2- 172.21.15.83:0/956357013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecbc0ce400 0x7fecbc161c10 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fecac009870 tx=0x7fecac01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13247afeb37c6000 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:37.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.575+0000 7fecc1378640 1 -- 172.21.15.83:0/956357013 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac0b2070 con 0x7fecbc0ce400 2024-04-17T10:41:37.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.575+0000 7fecc1378640 1 -- 172.21.15.83:0/956357013 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecac0ad640 con 0x7fecbc0ce400 2024-04-17T10:41:37.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.575+0000 7fecc1378640 1 -- 172.21.15.83:0/956357013 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac0ada10 con 0x7fecbc0ce400 2024-04-17T10:41:37.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.575+0000 7fecc45de640 1 -- 172.21.15.83:0/956357013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecbc0ce400 msgr2=0x7fecbc161c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:37.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.575+0000 7fecc45de640 1 --2- 172.21.15.83:0/956357013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecbc0ce400 0x7fecbc161c10 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fecac009870 tx=0x7fecac01c4d0 comp rx=0 tx=0).stop 2024-04-17T10:41:37.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.575+0000 7fecc45de640 1 -- 172.21.15.83:0/956357013 shutdown_connections 2024-04-17T10:41:37.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.575+0000 7fecc45de640 1 --2- 172.21.15.83:0/956357013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecbc0ce400 0x7fecbc161c10 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:37.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.575+0000 7fecc45de640 1 --2- 172.21.15.83:0/956357013 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecbc15e0f0 0x7fecbc0b6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:37.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.575+0000 7fecc45de640 1 -- 172.21.15.83:0/956357013 >> 172.21.15.83:0/956357013 conn(0x7fecbc0d0450 msgr2=0x7fecbc0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:37.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.575+0000 7fecc45de640 1 -- 172.21.15.83:0/956357013 shutdown_connections 2024-04-17T10:41:37.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.575+0000 7fecc45de640 1 -- 172.21.15.83:0/956357013 wait complete. 2024-04-17T10:41:37.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.575+0000 7fecc45de640 1 Processor -- start 2024-04-17T10:41:37.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.575+0000 7fecc45de640 1 -- start start 2024-04-17T10:41:37.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.575+0000 7fecc45de640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecbc0b5450 0x7fecbc14c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:37.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.575+0000 7fecc45de640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecbc0ce400 0x7fecbc1570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:37.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.575+0000 7fecc45de640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecbc15e0f0 0x7fecbc157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:37.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.575+0000 7fecc45de640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc1622d0 con 0x7fecbc0b5450 2024-04-17T10:41:37.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.575+0000 7fecc45de640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc162150 con 0x7fecbc15e0f0 2024-04-17T10:41:37.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.575+0000 7fecc45de640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc16af50 con 0x7fecbc0ce400 2024-04-17T10:41:37.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.575+0000 7fecc237a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecbc0b5450 0x7fecbc14c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:37.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.575+0000 7fecc237a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecbc0b5450 0x7fecbc14c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51108/0 (socket says 172.21.15.83:51108) 2024-04-17T10:41:37.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.575+0000 7fecc237a640 1 -- 172.21.15.83:0/1304091121 learned_addr learned my addr 172.21.15.83:0/1304091121 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:37.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.575+0000 7fecc237a640 1 -- 172.21.15.83:0/1304091121 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecbc0ce400 msgr2=0x7fecbc1570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:37.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.575+0000 7fecc237a640 1 --2- 172.21.15.83:0/1304091121 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecbc0ce400 0x7fecbc1570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:37.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.575+0000 7fecc237a640 1 -- 172.21.15.83:0/1304091121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecbc15e0f0 msgr2=0x7fecbc157600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:37.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.575+0000 7fecc2b7b640 1 --2- 172.21.15.83:0/1304091121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecbc15e0f0 0x7fecbc157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:37.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.575+0000 7fecc237a640 1 --2- 172.21.15.83:0/1304091121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecbc15e0f0 0x7fecbc157600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:37.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.575+0000 7fecc237a640 1 -- 172.21.15.83:0/1304091121 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecbc14d310 con 0x7fecbc0b5450 2024-04-17T10:41:37.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.579+0000 7fecc237a640 1 --2- 172.21.15.83:0/1304091121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecbc0b5450 0x7fecbc14c720 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fecb800c7f0 tx=0x7fecb800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:37.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.579+0000 7fecb2ffd640 1 -- 172.21.15.83:0/1304091121 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb8007a30 con 0x7fecbc0b5450 2024-04-17T10:41:37.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.579+0000 7fecb2ffd640 1 -- 172.21.15.83:0/1304091121 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb8007bd0 con 0x7fecbc0b5450 2024-04-17T10:41:37.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.579+0000 7fecc45de640 1 -- 172.21.15.83:0/1304091121 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecbc14d5f0 con 0x7fecbc0b5450 2024-04-17T10:41:37.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.579+0000 7fecb2ffd640 1 -- 172.21.15.83:0/1304091121 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb80113f0 con 0x7fecbc0b5450 2024-04-17T10:41:37.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.579+0000 7fecc45de640 1 -- 172.21.15.83:0/1304091121 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecbc14dbb0 con 0x7fecbc0b5450 2024-04-17T10:41:37.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.579+0000 7fecc45de640 1 -- 172.21.15.83:0/1304091121 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecbc0681a0 con 0x7fecbc0b5450 2024-04-17T10:41:37.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.579+0000 7fecb2ffd640 1 -- 172.21.15.83:0/1304091121 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fecb8011590 con 0x7fecbc0b5450 2024-04-17T10:41:37.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.579+0000 7fecb2ffd640 1 --2- 172.21.15.83:0/1304091121 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feca0066f60 0x7feca0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:37.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.579+0000 7fecb2ffd640 1 -- 172.21.15.83:0/1304091121 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fecb8085180 con 0x7fecbc0b5450 2024-04-17T10:41:37.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.579+0000 7fecc1b79640 1 --2- 172.21.15.83:0/1304091121 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feca0066f60 0x7feca0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:37.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.579+0000 7fecc1b79640 1 --2- 172.21.15.83:0/1304091121 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feca0066f60 0x7feca0069420 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fecac0af000 tx=0x7fecac004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:37.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.583+0000 7fecb2ffd640 1 -- 172.21.15.83:0/1304091121 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fecb8015020 con 0x7fecbc0b5450 2024-04-17T10:41:37.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:37.767+0000 7fecc45de640 1 -- 172.21.15.83:0/1304091121 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fecbc074f90 con 0x7fecbc0b5450 2024-04-17T10:41:38.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.443+0000 7fecb2ffd640 1 -- 172.21.15.83:0/1304091121 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v19) v1 ==== 172+0+0 (secure 0 0 0) 0x7fecb8056280 con 0x7fecbc0b5450 2024-04-17T10:41:38.447 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 6 and data pool 7 2024-04-17T10:41:38.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.455+0000 7fecc45de640 1 -- 172.21.15.83:0/1304091121 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feca0066f60 msgr2=0x7feca0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:38.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.455+0000 7fecc45de640 1 --2- 172.21.15.83:0/1304091121 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feca0066f60 0x7feca0069420 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fecac0af000 tx=0x7fecac004740 comp rx=0 tx=0).stop 2024-04-17T10:41:38.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.455+0000 7fecc45de640 1 -- 172.21.15.83:0/1304091121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecbc0b5450 msgr2=0x7fecbc14c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:38.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.455+0000 7fecc45de640 1 --2- 172.21.15.83:0/1304091121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecbc0b5450 0x7fecbc14c720 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fecb800c7f0 tx=0x7fecb800ccb0 comp rx=0 tx=0).stop 2024-04-17T10:41:38.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.463+0000 7fecc45de640 1 -- 172.21.15.83:0/1304091121 shutdown_connections 2024-04-17T10:41:38.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.463+0000 7fecc45de640 1 --2- 172.21.15.83:0/1304091121 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feca0066f60 0x7feca0069420 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:38.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.463+0000 7fecc45de640 1 --2- 172.21.15.83:0/1304091121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecbc15e0f0 0x7fecbc157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:38.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.463+0000 7fecc45de640 1 --2- 172.21.15.83:0/1304091121 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecbc0ce400 0x7fecbc1570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:38.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.463+0000 7fecc45de640 1 --2- 172.21.15.83:0/1304091121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecbc0b5450 0x7fecbc14c720 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:38.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.463+0000 7fecc45de640 1 -- 172.21.15.83:0/1304091121 >> 172.21.15.83:0/1304091121 conn(0x7fecbc0d0450 msgr2=0x7fecbc0b8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:38.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.463+0000 7fecc45de640 1 -- 172.21.15.83:0/1304091121 shutdown_connections 2024-04-17T10:41:38.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.463+0000 7fecc45de640 1 -- 172.21.15.83:0/1304091121 wait complete. 2024-04-17T10:41:38.487 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:41:38.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.603+0000 7f261f385640 1 Processor -- start 2024-04-17T10:41:38.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.603+0000 7f261f385640 1 -- start start 2024-04-17T10:41:38.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.607+0000 7f261f385640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2618072330 0x7f26180cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:38.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.607+0000 7f261f385640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f26180cfa10 0x7f261806c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:38.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.607+0000 7f261f385640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26180cfea0 con 0x7f261815ba30 2024-04-17T10:41:38.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.607+0000 7f261f385640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2618071ea0 con 0x7f26180cfa10 2024-04-17T10:41:38.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.607+0000 7f261f385640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f261806e320 con 0x7f2618072330 2024-04-17T10:41:38.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.607+0000 7f261d121640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f261815ba30 0x7f261815c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:36830/0 (socket says 172.21.15.83:36830) 2024-04-17T10:41:38.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.607+0000 7f261d121640 1 -- 172.21.15.83:0/323986185 learned_addr learned my addr 172.21.15.83:0/323986185 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:38.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.607+0000 7f261d922640 1 --2- 172.21.15.83:0/323986185 >> v2:172.21.15.83:3300/0 conn(0x7f26180cfa10 0x7f261806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:38.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.607+0000 7f261c920640 1 --2- 172.21.15.83:0/323986185 >> v2:172.21.15.167:3300/0 conn(0x7f2618072330 0x7f26180cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:38.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.607+0000 7f260ffff640 1 -- 172.21.15.83:0/323986185 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2798365015 0 0) 0x7f26180cfea0 con 0x7f261815ba30 2024-04-17T10:41:38.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.607+0000 7f260ffff640 1 -- 172.21.15.83:0/323986185 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2604003530 con 0x7f261815ba30 2024-04-17T10:41:38.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.607+0000 7f261d922640 1 -- 172.21.15.83:0/323986185 >> v2:172.21.15.167:3300/0 conn(0x7f2618072330 msgr2=0x7f26180cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:38.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.607+0000 7f261d922640 1 --2- 172.21.15.83:0/323986185 >> v2:172.21.15.167:3300/0 conn(0x7f2618072330 0x7f26180cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:38.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.607+0000 7f261d922640 1 -- 172.21.15.83:0/323986185 >> v1:172.21.15.83:6789/0 conn(0x7f261815ba30 legacy=0x7f261815c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:38.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.607+0000 7f261d922640 1 -- 172.21.15.83:0/323986185 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261806e040 con 0x7f26180cfa10 2024-04-17T10:41:38.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.607+0000 7f261d922640 1 --2- 172.21.15.83:0/323986185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26180cfa10 0x7f261806c490 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f26180af2b0 tx=0x7f261001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c33e7471690999 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:38.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.607+0000 7f260ffff640 1 -- 172.21.15.83:0/323986185 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26100b2070 con 0x7f26180cfa10 2024-04-17T10:41:38.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.607+0000 7f260ffff640 1 -- 172.21.15.83:0/323986185 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26100a5870 con 0x7f26180cfa10 2024-04-17T10:41:38.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.607+0000 7f260ffff640 1 -- 172.21.15.83:0/323986185 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26100a5c40 con 0x7f26180cfa10 2024-04-17T10:41:38.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.607+0000 7f261f385640 1 -- 172.21.15.83:0/323986185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26180cfa10 msgr2=0x7f261806c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:38.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.607+0000 7f261f385640 1 --2- 172.21.15.83:0/323986185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26180cfa10 0x7f261806c490 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f26180af2b0 tx=0x7f261001c8a0 comp rx=0 tx=0).stop 2024-04-17T10:41:38.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.607+0000 7f261f385640 1 -- 172.21.15.83:0/323986185 shutdown_connections 2024-04-17T10:41:38.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.607+0000 7f261f385640 1 --2- 172.21.15.83:0/323986185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26180cfa10 0x7f261806c490 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:38.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.607+0000 7f261f385640 1 --2- 172.21.15.83:0/323986185 >> v2:172.21.15.167:3300/0 conn(0x7f2618072330 0x7f26180cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:38.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.607+0000 7f261f385640 1 -- 172.21.15.83:0/323986185 >> 172.21.15.83:0/323986185 conn(0x7f261806abe0 msgr2=0x7f261806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:38.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.607+0000 7f261f385640 1 -- 172.21.15.83:0/323986185 shutdown_connections 2024-04-17T10:41:38.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.607+0000 7f261f385640 1 -- 172.21.15.83:0/323986185 wait complete. 2024-04-17T10:41:38.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.611+0000 7f261f385640 1 Processor -- start 2024-04-17T10:41:38.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.611+0000 7f261f385640 1 -- start start 2024-04-17T10:41:38.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.611+0000 7f261f385640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2618072330 0x7f26180b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:38.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.611+0000 7f261f385640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26180cfa10 0x7f2618151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:38.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.611+0000 7f261f385640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f261815ba30 0x7f26180adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:38.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.611+0000 7f261f385640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261806de20 con 0x7f261815ba30 2024-04-17T10:41:38.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.611+0000 7f261f385640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261806dca0 con 0x7f2618072330 2024-04-17T10:41:38.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.611+0000 7f261f385640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2618168520 con 0x7f26180cfa10 2024-04-17T10:41:38.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.611+0000 7f261d922640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f261815ba30 0x7f26180adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:38.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.611+0000 7f261d922640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f261815ba30 0x7f26180adaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51130/0 (socket says 172.21.15.83:51130) 2024-04-17T10:41:38.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.611+0000 7f261d922640 1 -- 172.21.15.83:0/1987792539 learned_addr learned my addr 172.21.15.83:0/1987792539 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:38.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.611+0000 7f261d922640 1 -- 172.21.15.83:0/1987792539 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26180cfa10 msgr2=0x7f2618151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:38.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.611+0000 7f261d922640 1 --2- 172.21.15.83:0/1987792539 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26180cfa10 0x7f2618151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:38.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.611+0000 7f261d922640 1 -- 172.21.15.83:0/1987792539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2618072330 msgr2=0x7f26180b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:38.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.611+0000 7f261d922640 1 --2- 172.21.15.83:0/1987792539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2618072330 0x7f26180b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:38.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.611+0000 7f261d922640 1 -- 172.21.15.83:0/1987792539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26180b4750 con 0x7f261815ba30 2024-04-17T10:41:38.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.611+0000 7f261d922640 1 --2- 172.21.15.83:0/1987792539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f261815ba30 0x7f26180adaa0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f261001a4d0 tx=0x7f2610019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:38.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.611+0000 7f260dffb640 1 -- 172.21.15.83:0/1987792539 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26100bd020 con 0x7f261815ba30 2024-04-17T10:41:38.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.611+0000 7f260dffb640 1 -- 172.21.15.83:0/1987792539 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26100b8470 con 0x7f261815ba30 2024-04-17T10:41:38.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.611+0000 7f260dffb640 1 -- 172.21.15.83:0/1987792539 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26100b2040 con 0x7f261815ba30 2024-04-17T10:41:38.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.611+0000 7f261f385640 1 -- 172.21.15.83:0/1987792539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2618151890 con 0x7f261815ba30 2024-04-17T10:41:38.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.611+0000 7f261f385640 1 -- 172.21.15.83:0/1987792539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26180b3a10 con 0x7f261815ba30 2024-04-17T10:41:38.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.611+0000 7f261f385640 1 -- 172.21.15.83:0/1987792539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2618074f90 con 0x7f261815ba30 2024-04-17T10:41:38.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.611+0000 7f260dffb640 1 -- 172.21.15.83:0/1987792539 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f26100c0020 con 0x7f261815ba30 2024-04-17T10:41:38.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.615+0000 7f260dffb640 1 --2- 172.21.15.83:0/1987792539 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f25fc066f60 0x7f25fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:38.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.615+0000 7f261d121640 1 --2- 172.21.15.83:0/1987792539 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f25fc066f60 0x7f25fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:38.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.615+0000 7f260dffb640 1 -- 172.21.15.83:0/1987792539 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f261011fd80 con 0x7f261815ba30 2024-04-17T10:41:38.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.615+0000 7f261d121640 1 --2- 172.21.15.83:0/1987792539 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f25fc066f60 0x7f25fc069420 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2614007fd0 tx=0x7f2614007900 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:38.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.619+0000 7f260dffb640 1 -- 172.21.15.83:0/1987792539 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f26100c7030 con 0x7f261815ba30 2024-04-17T10:41:38.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.763+0000 7f261f385640 1 -- 172.21.15.83:0/1987792539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26180b37c0 con 0x7f261815ba30 2024-04-17T10:41:38.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.767+0000 7f260dffb640 1 -- 172.21.15.83:0/1987792539 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16554 (secure 0 0 0) 0x7f2610016520 con 0x7f261815ba30 2024-04-17T10:41:38.765 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:41:38.765 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":45,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:41:37.772771+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:41:36.031388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-17T10:41:36.783410+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:41:38.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.767+0000 7f261f385640 1 -- 172.21.15.83:0/1987792539 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f25fc066f60 msgr2=0x7f25fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:38.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.767+0000 7f261f385640 1 --2- 172.21.15.83:0/1987792539 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f25fc066f60 0x7f25fc069420 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2614007fd0 tx=0x7f2614007900 comp rx=0 tx=0).stop 2024-04-17T10:41:38.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.767+0000 7f261f385640 1 -- 172.21.15.83:0/1987792539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f261815ba30 msgr2=0x7f26180adaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:38.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.767+0000 7f261f385640 1 --2- 172.21.15.83:0/1987792539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f261815ba30 0x7f26180adaa0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f261001a4d0 tx=0x7f2610019640 comp rx=0 tx=0).stop 2024-04-17T10:41:38.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.767+0000 7f261f385640 1 -- 172.21.15.83:0/1987792539 shutdown_connections 2024-04-17T10:41:38.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.767+0000 7f261f385640 1 --2- 172.21.15.83:0/1987792539 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f25fc066f60 0x7f25fc069420 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:38.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.767+0000 7f261f385640 1 --2- 172.21.15.83:0/1987792539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f261815ba30 0x7f26180adaa0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:38.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.767+0000 7f261f385640 1 --2- 172.21.15.83:0/1987792539 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26180cfa10 0x7f2618151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:38.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.771+0000 7f261f385640 1 --2- 172.21.15.83:0/1987792539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2618072330 0x7f26180b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:38.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.771+0000 7f261f385640 1 -- 172.21.15.83:0/1987792539 >> 172.21.15.83:0/1987792539 conn(0x7f261806abe0 msgr2=0x7f261815acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:38.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.771+0000 7f261f385640 1 -- 172.21.15.83:0/1987792539 shutdown_connections 2024-04-17T10:41:38.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.771+0000 7f261f385640 1 -- 172.21.15.83:0/1987792539 wait complete. 2024-04-17T10:41:38.783 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:41:38.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.903+0000 7f02384cd640 1 Processor -- start 2024-04-17T10:41:38.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.907+0000 7f02384cd640 1 -- start start 2024-04-17T10:41:38.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.907+0000 7f02384cd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f02300aedd0 0x7f023006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:38.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.907+0000 7f02384cd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f023006dae0 0x7f02300b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:38.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.907+0000 7f02384cd640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f023015e440 con 0x7f023015db50 2024-04-17T10:41:38.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.907+0000 7f02384cd640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f023015e720 con 0x7f023006dae0 2024-04-17T10:41:38.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.907+0000 7f02384cd640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f023015ea00 con 0x7f02300aedd0 2024-04-17T10:41:38.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.907+0000 7f0236269640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f023015db50 0x7f0230070470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:36846/0 (socket says 172.21.15.83:36846) 2024-04-17T10:41:38.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.907+0000 7f0236269640 1 -- 172.21.15.83:0/3213346827 learned_addr learned my addr 172.21.15.83:0/3213346827 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:38.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.907+0000 7f0236a6a640 1 --2- 172.21.15.83:0/3213346827 >> v2:172.21.15.83:3300/0 conn(0x7f023006dae0 0x7f02300b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:38.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.907+0000 7f0235a68640 1 --2- 172.21.15.83:0/3213346827 >> v2:172.21.15.167:3300/0 conn(0x7f02300aedd0 0x7f023006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:38.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.907+0000 7f0235267640 1 -- 172.21.15.83:0/3213346827 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3586291576 0 0) 0x7f023015e440 con 0x7f023015db50 2024-04-17T10:41:38.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.907+0000 7f0235267640 1 -- 172.21.15.83:0/3213346827 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f021c003530 con 0x7f023015db50 2024-04-17T10:41:38.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.907+0000 7f0236a6a640 1 -- 172.21.15.83:0/3213346827 >> v2:172.21.15.167:3300/0 conn(0x7f02300aedd0 msgr2=0x7f023006d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:38.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.907+0000 7f0236a6a640 1 --2- 172.21.15.83:0/3213346827 >> v2:172.21.15.167:3300/0 conn(0x7f02300aedd0 0x7f023006d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:38.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.907+0000 7f0236a6a640 1 -- 172.21.15.83:0/3213346827 >> v1:172.21.15.83:6789/0 conn(0x7f023015db50 legacy=0x7f0230070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:38.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.907+0000 7f0236a6a640 1 -- 172.21.15.83:0/3213346827 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f023015ece0 con 0x7f023006dae0 2024-04-17T10:41:38.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.907+0000 7f0235a68640 1 --2- 172.21.15.83:0/3213346827 >> v2:172.21.15.167:3300/0 conn(0x7f02300aedd0 0x7f023006d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:41:38.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f0236a6a640 1 --2- 172.21.15.83:0/3213346827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f023006dae0 0x7f02300b2e10 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f02280097c0 tx=0x7f022801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=711f1a909d975b8c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:38.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f0235267640 1 -- 172.21.15.83:0/3213346827 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02280b0070 con 0x7f023006dae0 2024-04-17T10:41:38.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f0235267640 1 -- 172.21.15.83:0/3213346827 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02280ab600 con 0x7f023006dae0 2024-04-17T10:41:38.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f0235267640 1 -- 172.21.15.83:0/3213346827 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02280ab9d0 con 0x7f023006dae0 2024-04-17T10:41:38.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f02384cd640 1 -- 172.21.15.83:0/3213346827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f023006dae0 msgr2=0x7f02300b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:38.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f02384cd640 1 --2- 172.21.15.83:0/3213346827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f023006dae0 0x7f02300b2e10 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f02280097c0 tx=0x7f022801c460 comp rx=0 tx=0).stop 2024-04-17T10:41:38.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f02384cd640 1 -- 172.21.15.83:0/3213346827 shutdown_connections 2024-04-17T10:41:38.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f02384cd640 1 --2- 172.21.15.83:0/3213346827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f023006dae0 0x7f02300b2e10 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:38.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f02384cd640 1 --2- 172.21.15.83:0/3213346827 >> v2:172.21.15.167:3300/0 conn(0x7f02300aedd0 0x7f023006d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:38.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f02384cd640 1 -- 172.21.15.83:0/3213346827 >> 172.21.15.83:0/3213346827 conn(0x7f023006abe0 msgr2=0x7f023006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:38.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f02384cd640 1 -- 172.21.15.83:0/3213346827 shutdown_connections 2024-04-17T10:41:38.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f02384cd640 1 -- 172.21.15.83:0/3213346827 wait complete. 2024-04-17T10:41:38.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f02384cd640 1 Processor -- start 2024-04-17T10:41:38.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f02384cd640 1 -- start start 2024-04-17T10:41:38.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f02384cd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f023006dae0 0x7f02300cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:38.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f02384cd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f02300aedd0 0x7f0230153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:38.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f02384cd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f023015db50 0x7f0230153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:38.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f02384cd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02300b3ef0 con 0x7f023006dae0 2024-04-17T10:41:38.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f02384cd640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02300b3d70 con 0x7f023015db50 2024-04-17T10:41:38.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f02384cd640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02300b4070 con 0x7f02300aedd0 2024-04-17T10:41:38.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f0236269640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f023006dae0 0x7f02300cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:38.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f0236269640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f023006dae0 0x7f02300cdfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51152/0 (socket says 172.21.15.83:51152) 2024-04-17T10:41:38.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f0236269640 1 -- 172.21.15.83:0/2637247697 learned_addr learned my addr 172.21.15.83:0/2637247697 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:38.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f0235a68640 1 --2- 172.21.15.83:0/2637247697 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f02300aedd0 0x7f0230153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:38.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f0236a6a640 1 --2- 172.21.15.83:0/2637247697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f023015db50 0x7f0230153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:38.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f0236269640 1 -- 172.21.15.83:0/2637247697 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f02300aedd0 msgr2=0x7f0230153080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:38.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f0236269640 1 --2- 172.21.15.83:0/2637247697 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f02300aedd0 0x7f0230153080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:38.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f0236269640 1 -- 172.21.15.83:0/2637247697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f023015db50 msgr2=0x7f0230153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:38.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f0236269640 1 --2- 172.21.15.83:0/2637247697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f023015db50 0x7f0230153f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:38.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f0236269640 1 -- 172.21.15.83:0/2637247697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0230150b20 con 0x7f023006dae0 2024-04-17T10:41:38.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f0236269640 1 --2- 172.21.15.83:0/2637247697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f023006dae0 0x7f02300cdfd0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f022c00cad0 tx=0x7f022c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:38.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f0226ffd640 1 -- 172.21.15.83:0/2637247697 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022c017070 con 0x7f023006dae0 2024-04-17T10:41:38.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f0226ffd640 1 -- 172.21.15.83:0/2637247697 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f022c00ac30 con 0x7f023006dae0 2024-04-17T10:41:38.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f02384cd640 1 -- 172.21.15.83:0/2637247697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0230150560 con 0x7f023006dae0 2024-04-17T10:41:38.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f0226ffd640 1 -- 172.21.15.83:0/2637247697 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022c012400 con 0x7f023006dae0 2024-04-17T10:41:38.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.911+0000 7f02384cd640 1 -- 172.21.15.83:0/2637247697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0230152880 con 0x7f023006dae0 2024-04-17T10:41:38.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.915+0000 7f02384cd640 1 -- 172.21.15.83:0/2637247697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0230074f90 con 0x7f023006dae0 2024-04-17T10:41:38.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.915+0000 7f0226ffd640 1 -- 172.21.15.83:0/2637247697 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f022c01a070 con 0x7f023006dae0 2024-04-17T10:41:38.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.915+0000 7f0226ffd640 1 --2- 172.21.15.83:0/2637247697 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0214066f60 0x7f0214069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:38.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.915+0000 7f0226ffd640 1 -- 172.21.15.83:0/2637247697 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f022c084dc0 con 0x7f023006dae0 2024-04-17T10:41:38.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.915+0000 7f0235a68640 1 --2- 172.21.15.83:0/2637247697 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0214066f60 0x7f0214069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:38.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.915+0000 7f0235a68640 1 --2- 172.21.15.83:0/2637247697 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0214066f60 0x7f0214069420 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f02301f79d0 tx=0x7f0220008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:38.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:38.919+0000 7f0226ffd640 1 -- 172.21.15.83:0/2637247697 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f022c055dd0 con 0x7f023006dae0 2024-04-17T10:41:39.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.079+0000 7f02384cd640 1 -- 172.21.15.83:0/2637247697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02300b2af0 con 0x7f023006dae0 2024-04-17T10:41:39.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.083+0000 7f0226ffd640 1 -- 172.21.15.83:0/2637247697 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16554 (secure 0 0 0) 0x7f022c0556d0 con 0x7f023006dae0 2024-04-17T10:41:39.081 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:41:39.081 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":45,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:41:37.772771+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:41:36.031388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-17T10:41:36.783410+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:41:39.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.083+0000 7f02384cd640 1 -- 172.21.15.83:0/2637247697 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0214066f60 msgr2=0x7f0214069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:39.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.083+0000 7f02384cd640 1 --2- 172.21.15.83:0/2637247697 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0214066f60 0x7f0214069420 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f02301f79d0 tx=0x7f0220008040 comp rx=0 tx=0).stop 2024-04-17T10:41:39.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.083+0000 7f02384cd640 1 -- 172.21.15.83:0/2637247697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f023006dae0 msgr2=0x7f02300cdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:39.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.083+0000 7f02384cd640 1 --2- 172.21.15.83:0/2637247697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f023006dae0 0x7f02300cdfd0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f022c00cad0 tx=0x7f022c00cf90 comp rx=0 tx=0).stop 2024-04-17T10:41:39.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.083+0000 7f02384cd640 1 -- 172.21.15.83:0/2637247697 shutdown_connections 2024-04-17T10:41:39.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.083+0000 7f02384cd640 1 --2- 172.21.15.83:0/2637247697 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0214066f60 0x7f0214069420 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:39.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.083+0000 7f02384cd640 1 --2- 172.21.15.83:0/2637247697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f023015db50 0x7f0230153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:39.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.083+0000 7f02384cd640 1 --2- 172.21.15.83:0/2637247697 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f02300aedd0 0x7f0230153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:39.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.083+0000 7f02384cd640 1 --2- 172.21.15.83:0/2637247697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f023006dae0 0x7f02300cdfd0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:39.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.083+0000 7f02384cd640 1 -- 172.21.15.83:0/2637247697 >> 172.21.15.83:0/2637247697 conn(0x7f023006abe0 msgr2=0x7f0230070470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:39.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.087+0000 7f02384cd640 1 -- 172.21.15.83:0/2637247697 shutdown_connections 2024-04-17T10:41:39.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.087+0000 7f02384cd640 1 -- 172.21.15.83:0/2637247697 wait complete. 2024-04-17T10:41:39.100 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-17T10:41:39.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18bdda6640 1 Processor -- start 2024-04-17T10:41:39.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18bdda6640 1 -- start start 2024-04-17T10:41:39.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18bdda6640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f18b80d2330 0x7f18b80cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:39.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18bdda6640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18b80b1230 con 0x7f18b815bc40 2024-04-17T10:41:39.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18bdda6640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18b80d04f0 con 0x7f18b80ce8c0 2024-04-17T10:41:39.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18bdda6640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b80cd7f0 con 0x7f18b80d2330 2024-04-17T10:41:39.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18b77fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f18b815bc40 0x7f18b80ce1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:36854/0 (socket says 172.21.15.83:36854) 2024-04-17T10:41:39.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18b77fe640 1 -- 172.21.15.83:0/11291403 learned_addr learned my addr 172.21.15.83:0/11291403 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:39.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18b7fff640 1 --2- 172.21.15.83:0/11291403 >> v2:172.21.15.167:3300/0 conn(0x7f18b80d2330 0x7f18b80cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:39.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18b67fc640 1 -- 172.21.15.83:0/11291403 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4169771288 0 0) 0x7f18b80b1230 con 0x7f18b815bc40 2024-04-17T10:41:39.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18b67fc640 1 -- 172.21.15.83:0/11291403 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18a0003530 con 0x7f18b815bc40 2024-04-17T10:41:39.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18b67fc640 1 -- 172.21.15.83:0/11291403 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4061868407 0 0) 0x7f18a0003530 con 0x7f18b815bc40 2024-04-17T10:41:39.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18b67fc640 1 -- 172.21.15.83:0/11291403 >> v2:172.21.15.167:3300/0 conn(0x7f18b80d2330 msgr2=0x7f18b80cfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:39.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18b67fc640 1 --2- 172.21.15.83:0/11291403 >> v2:172.21.15.167:3300/0 conn(0x7f18b80d2330 0x7f18b80cfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:39.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18b67fc640 1 -- 172.21.15.83:0/11291403 >> v1:172.21.15.167:6789/0 conn(0x7f18b80ce8c0 legacy=0x7f18b80b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:39.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18b67fc640 1 -- 172.21.15.83:0/11291403 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b80b14d0 con 0x7f18b815bc40 2024-04-17T10:41:39.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18b67fc640 1 -- 172.21.15.83:0/11291403 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f18a8001680 con 0x7f18b815bc40 2024-04-17T10:41:39.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18b67fc640 1 -- 172.21.15.83:0/11291403 >> v1:172.21.15.83:6789/0 conn(0x7f18b815bc40 legacy=0x7f18b80ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:39.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18b67fc640 1 --2- 172.21.15.83:0/11291403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18a0003a30 0x7f18a0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:39.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18b67fc640 1 --2- 172.21.15.83:0/11291403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18a0004810 0x7f18a0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:39.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18b67fc640 1 --2- 172.21.15.83:0/11291403 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18a0005240 0x7f18a000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:39.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18b67fc640 1 -- 172.21.15.83:0/11291403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a000c170 con 0x7f18a0003a30 2024-04-17T10:41:39.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18b67fc640 1 -- 172.21.15.83:0/11291403 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a000c320 con 0x7f18a0004810 2024-04-17T10:41:39.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18b67fc640 1 -- 172.21.15.83:0/11291403 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a000c520 con 0x7f18a0005240 2024-04-17T10:41:39.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18b77fe640 1 --2- 172.21.15.83:0/11291403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18a0003a30 0x7f18a0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:39.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18b6ffd640 1 --2- 172.21.15.83:0/11291403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18a0004810 0x7f18a0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:39.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18b7fff640 1 --2- 172.21.15.83:0/11291403 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18a0005240 0x7f18a000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:39.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18b77fe640 1 -- 172.21.15.83:0/11291403 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18a0005240 msgr2=0x7f18a000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:39.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18b77fe640 1 --2- 172.21.15.83:0/11291403 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18a0005240 0x7f18a000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:39.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18b77fe640 1 -- 172.21.15.83:0/11291403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18a0004810 msgr2=0x7f18a0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:39.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18b77fe640 1 --2- 172.21.15.83:0/11291403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18a0004810 0x7f18a0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:39.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18b7fff640 1 -- 172.21.15.83:0/11291403 reap_dead start 2024-04-17T10:41:39.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18b77fe640 1 -- 172.21.15.83:0/11291403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a0003530 con 0x7f18a0003a30 2024-04-17T10:41:39.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.227+0000 7f18b6ffd640 1 --2- 172.21.15.83:0/11291403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18a0004810 0x7f18a0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:41:39.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.231+0000 7f18b77fe640 1 --2- 172.21.15.83:0/11291403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18a0003a30 0x7f18a0003e50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f18a800c210 tx=0x7f18a801cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1707f620bb4061d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:39.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.231+0000 7f18b67fc640 1 -- 172.21.15.83:0/11291403 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a8007d40 con 0x7f18a0003a30 2024-04-17T10:41:39.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.231+0000 7f18b67fc640 1 -- 172.21.15.83:0/11291403 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a80a9070 con 0x7f18a0003a30 2024-04-17T10:41:39.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.231+0000 7f18b67fc640 1 -- 172.21.15.83:0/11291403 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a8005d80 con 0x7f18a0003a30 2024-04-17T10:41:39.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.231+0000 7f18bdda6640 1 -- 172.21.15.83:0/11291403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18a0003a30 msgr2=0x7f18a0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:39.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.231+0000 7f18bdda6640 1 --2- 172.21.15.83:0/11291403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18a0003a30 0x7f18a0003e50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f18a800c210 tx=0x7f18a801cb40 comp rx=0 tx=0).stop 2024-04-17T10:41:39.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.231+0000 7f18bdda6640 1 -- 172.21.15.83:0/11291403 shutdown_connections 2024-04-17T10:41:39.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.231+0000 7f18bdda6640 1 --2- 172.21.15.83:0/11291403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18a0003a30 0x7f18a0003e50 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:39.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.231+0000 7f18bdda6640 1 -- 172.21.15.83:0/11291403 >> 172.21.15.83:0/11291403 conn(0x7f18b80d17a0 msgr2=0x7f18b80d1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:39.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.231+0000 7f18bdda6640 1 -- 172.21.15.83:0/11291403 shutdown_connections 2024-04-17T10:41:39.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.231+0000 7f18bdda6640 1 -- 172.21.15.83:0/11291403 wait complete. 2024-04-17T10:41:39.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.231+0000 7f18bdda6640 1 Processor -- start 2024-04-17T10:41:39.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.231+0000 7f18bdda6640 1 -- start start 2024-04-17T10:41:39.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.235+0000 7f18bdda6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18a0003a30 0x7f18b815a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:39.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.235+0000 7f18bdda6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18b80c5a00 0x7f18b815e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.235+0000 7f18bdda6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18b815e8e0 0x7f18b80b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.235+0000 7f18bdda6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b80cce20 con 0x7f18b80c5a00 2024-04-17T10:41:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.235+0000 7f18bdda6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b80ccca0 con 0x7f18a0003a30 2024-04-17T10:41:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.235+0000 7f18bdda6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b80ccfa0 con 0x7f18b815e8e0 2024-04-17T10:41:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.235+0000 7f18b6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18b80c5a00 0x7f18b815e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.235+0000 7f18b6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18b80c5a00 0x7f18b815e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51160/0 (socket says 172.21.15.83:51160) 2024-04-17T10:41:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.235+0000 7f18b6ffd640 1 -- 172.21.15.83:0/1386835803 learned_addr learned my addr 172.21.15.83:0/1386835803 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.235+0000 7f18b77fe640 1 --2- 172.21.15.83:0/1386835803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18a0003a30 0x7f18b815a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.235+0000 7f18b6ffd640 1 -- 172.21.15.83:0/1386835803 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18b815e8e0 msgr2=0x7f18b80b59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.235+0000 7f18b6ffd640 1 --2- 172.21.15.83:0/1386835803 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18b815e8e0 0x7f18b80b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.235+0000 7f18b6ffd640 1 -- 172.21.15.83:0/1386835803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18a0003a30 msgr2=0x7f18b815a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.235+0000 7f18b6ffd640 1 --2- 172.21.15.83:0/1386835803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18a0003a30 0x7f18b815a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.235+0000 7f18b6ffd640 1 -- 172.21.15.83:0/1386835803 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b80b5500 con 0x7f18b80c5a00 2024-04-17T10:41:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.235+0000 7f18b6ffd640 1 --2- 172.21.15.83:0/1386835803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18b80c5a00 0x7f18b815e3a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f18ac00e430 tx=0x7f18ac00e8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:39.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.235+0000 7f189ffff640 1 -- 172.21.15.83:0/1386835803 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ac019070 con 0x7f18b80c5a00 2024-04-17T10:41:39.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.235+0000 7f189ffff640 1 -- 172.21.15.83:0/1386835803 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18ac0052e0 con 0x7f18b80c5a00 2024-04-17T10:41:39.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.235+0000 7f18bdda6640 1 -- 172.21.15.83:0/1386835803 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b80c5f30 con 0x7f18b80c5a00 2024-04-17T10:41:39.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.235+0000 7f189ffff640 1 -- 172.21.15.83:0/1386835803 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ac003840 con 0x7f18b80c5a00 2024-04-17T10:41:39.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.235+0000 7f18bdda6640 1 -- 172.21.15.83:0/1386835803 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b80c64f0 con 0x7f18b80c5a00 2024-04-17T10:41:39.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.235+0000 7f18bdda6640 1 -- 172.21.15.83:0/1386835803 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b8074f90 con 0x7f18b80c5a00 2024-04-17T10:41:39.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.235+0000 7f189ffff640 1 -- 172.21.15.83:0/1386835803 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f18ac01c070 con 0x7f18b80c5a00 2024-04-17T10:41:39.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.235+0000 7f189ffff640 1 --2- 172.21.15.83:0/1386835803 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f18a4066f60 0x7f18a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:39.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.235+0000 7f189ffff640 1 -- 172.21.15.83:0/1386835803 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f18ac084520 con 0x7f18b80c5a00 2024-04-17T10:41:39.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.235+0000 7f18b77fe640 1 --2- 172.21.15.83:0/1386835803 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f18a4066f60 0x7f18a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:39.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.235+0000 7f18b77fe640 1 --2- 172.21.15.83:0/1386835803 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f18a4066f60 0x7f18a4069420 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f18a80ab000 tx=0x7f18a80b1040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:39.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.239+0000 7f189ffff640 1 -- 172.21.15.83:0/1386835803 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f18b80b5500 con 0x7f18b80c5a00 2024-04-17T10:41:39.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.435+0000 7f18bdda6640 1 -- 172.21.15.83:0/1386835803 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f18b815ed00 con 0x7f18b80c5a00 2024-04-17T10:41:39.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.439+0000 7f189ffff640 1 -- 172.21.15.83:0/1386835803 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v20) v1 ==== 118+0+0 (secure 0 0 0) 0x7f18ac054ee0 con 0x7f18b80c5a00 2024-04-17T10:41:39.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.443+0000 7f18bdda6640 1 -- 172.21.15.83:0/1386835803 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f18a4066f60 msgr2=0x7f18a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:39.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.443+0000 7f18bdda6640 1 --2- 172.21.15.83:0/1386835803 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f18a4066f60 0x7f18a4069420 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f18a80ab000 tx=0x7f18a80b1040 comp rx=0 tx=0).stop 2024-04-17T10:41:39.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.443+0000 7f18bdda6640 1 -- 172.21.15.83:0/1386835803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18b80c5a00 msgr2=0x7f18b815e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:39.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.443+0000 7f18bdda6640 1 --2- 172.21.15.83:0/1386835803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18b80c5a00 0x7f18b815e3a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f18ac00e430 tx=0x7f18ac00e8f0 comp rx=0 tx=0).stop 2024-04-17T10:41:39.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.443+0000 7f18bdda6640 1 -- 172.21.15.83:0/1386835803 shutdown_connections 2024-04-17T10:41:39.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.443+0000 7f18bdda6640 1 --2- 172.21.15.83:0/1386835803 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f18a4066f60 0x7f18a4069420 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:39.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.443+0000 7f18bdda6640 1 --2- 172.21.15.83:0/1386835803 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18b815e8e0 0x7f18b80b59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:39.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.443+0000 7f18bdda6640 1 --2- 172.21.15.83:0/1386835803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18b80c5a00 0x7f18b815e3a0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:39.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.443+0000 7f18bdda6640 1 --2- 172.21.15.83:0/1386835803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18a0003a30 0x7f18b815a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:39.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.443+0000 7f18bdda6640 1 -- 172.21.15.83:0/1386835803 >> 172.21.15.83:0/1386835803 conn(0x7f18b80d17a0 msgr2=0x7f18b806e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:39.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.443+0000 7f18bdda6640 1 -- 172.21.15.83:0/1386835803 shutdown_connections 2024-04-17T10:41:39.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.443+0000 7f18bdda6640 1 -- 172.21.15.83:0/1386835803 wait complete. 2024-04-17T10:41:39.460 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-17T10:41:39.461 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-17T10:41:39.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5c00208640 1 Processor -- start 2024-04-17T10:41:39.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5c00208640 1 -- start start 2024-04-17T10:41:39.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5c00208640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5bf806b520 0x7f5bf80b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:39.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5c00208640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bf80b4040 0x7f5bf806cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:39.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5c00208640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bf815bd70 con 0x7f5bf8154de0 2024-04-17T10:41:39.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5c00208640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bf80b4870 con 0x7f5bf806b520 2024-04-17T10:41:39.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5c00208640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf815bf60 con 0x7f5bf80b4040 2024-04-17T10:41:39.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5bfdfa4640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5bf8154de0 0x7f5bf806ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:36868/0 (socket says 172.21.15.83:36868) 2024-04-17T10:41:39.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5bfdfa4640 1 -- 172.21.15.83:0/1004075080 learned_addr learned my addr 172.21.15.83:0/1004075080 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:39.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5bfd7a3640 1 --2- 172.21.15.83:0/1004075080 >> v2:172.21.15.167:3300/0 conn(0x7f5bf806b520 0x7f5bf80b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:39.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5bfe7a5640 1 --2- 172.21.15.83:0/1004075080 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bf80b4040 0x7f5bf806cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:39.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5bfcfa2640 1 -- 172.21.15.83:0/1004075080 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1385671430 0 0) 0x7f5bf815bd70 con 0x7f5bf8154de0 2024-04-17T10:41:39.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5bfcfa2640 1 -- 172.21.15.83:0/1004075080 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5be4003530 con 0x7f5bf8154de0 2024-04-17T10:41:39.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5bfcfa2640 1 -- 172.21.15.83:0/1004075080 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 766474989 0 0) 0x7f5be4003530 con 0x7f5bf8154de0 2024-04-17T10:41:39.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5bfcfa2640 1 -- 172.21.15.83:0/1004075080 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bf80b4040 msgr2=0x7f5bf806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:39.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5bfcfa2640 1 --2- 172.21.15.83:0/1004075080 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bf80b4040 0x7f5bf806cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:39.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5bfcfa2640 1 -- 172.21.15.83:0/1004075080 >> v2:172.21.15.167:3300/0 conn(0x7f5bf806b520 msgr2=0x7f5bf80b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:39.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5bfcfa2640 1 --2- 172.21.15.83:0/1004075080 >> v2:172.21.15.167:3300/0 conn(0x7f5bf806b520 0x7f5bf80b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:39.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5bfcfa2640 1 -- 172.21.15.83:0/1004075080 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf815afb0 con 0x7f5bf8154de0 2024-04-17T10:41:39.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5bfcfa2640 1 -- 172.21.15.83:0/1004075080 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f5bf40016a0 con 0x7f5bf8154de0 2024-04-17T10:41:39.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5bfcfa2640 1 -- 172.21.15.83:0/1004075080 >> v1:172.21.15.83:6789/0 conn(0x7f5bf8154de0 legacy=0x7f5bf806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:39.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5bfcfa2640 1 --2- 172.21.15.83:0/1004075080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5be4003a30 0x7f5be4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:39.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5bfcfa2640 1 --2- 172.21.15.83:0/1004075080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5be4004810 0x7f5be4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:39.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5bfcfa2640 1 --2- 172.21.15.83:0/1004075080 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bf80b4040 0x7f5be40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:39.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5bfcfa2640 1 -- 172.21.15.83:0/1004075080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf815bf60 con 0x7f5be4003a30 2024-04-17T10:41:39.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5bfcfa2640 1 -- 172.21.15.83:0/1004075080 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be400b9d0 con 0x7f5be4004810 2024-04-17T10:41:39.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5bfcfa2640 1 -- 172.21.15.83:0/1004075080 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be400bb50 con 0x7f5bf80b4040 2024-04-17T10:41:39.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5bfdfa4640 1 --2- 172.21.15.83:0/1004075080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5be4003a30 0x7f5be4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:39.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5bfe7a5640 1 --2- 172.21.15.83:0/1004075080 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bf80b4040 0x7f5be40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:39.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5bfdfa4640 1 -- 172.21.15.83:0/1004075080 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bf80b4040 msgr2=0x7f5be40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:39.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5bfdfa4640 1 --2- 172.21.15.83:0/1004075080 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bf80b4040 0x7f5be40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:39.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5bfdfa4640 1 -- 172.21.15.83:0/1004075080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5be4004810 msgr2=0x7f5be4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:39.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5bfdfa4640 1 --2- 172.21.15.83:0/1004075080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5be4004810 0x7f5be4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:39.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5bfdfa4640 1 -- 172.21.15.83:0/1004075080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be4003530 con 0x7f5be4003a30 2024-04-17T10:41:39.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.583+0000 7f5bfe7a5640 1 --2- 172.21.15.83:0/1004075080 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bf80b4040 0x7f5be40051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:39.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.587+0000 7f5bfdfa4640 1 --2- 172.21.15.83:0/1004075080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5be4003a30 0x7f5be4003e50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5bf400c2f0 tx=0x7f5bf401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08fa45a8aa39a8d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:39.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.587+0000 7f5bfcfa2640 1 -- 172.21.15.83:0/1004075080 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4007da0 con 0x7f5be4003a30 2024-04-17T10:41:39.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.587+0000 7f5bfcfa2640 1 -- 172.21.15.83:0/1004075080 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf40a9070 con 0x7f5be4003a30 2024-04-17T10:41:39.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.587+0000 7f5bfcfa2640 1 -- 172.21.15.83:0/1004075080 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4005d20 con 0x7f5be4003a30 2024-04-17T10:41:39.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.587+0000 7f5c00208640 1 -- 172.21.15.83:0/1004075080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5be4003a30 msgr2=0x7f5be4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:39.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.587+0000 7f5c00208640 1 --2- 172.21.15.83:0/1004075080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5be4003a30 0x7f5be4003e50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5bf400c2f0 tx=0x7f5bf401cca0 comp rx=0 tx=0).stop 2024-04-17T10:41:39.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.587+0000 7f5bfe7a5640 1 -- 172.21.15.83:0/1004075080 reap_dead start 2024-04-17T10:41:39.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.587+0000 7f5c00208640 1 -- 172.21.15.83:0/1004075080 shutdown_connections 2024-04-17T10:41:39.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.587+0000 7f5c00208640 1 -- 172.21.15.83:0/1004075080 >> 172.21.15.83:0/1004075080 conn(0x7f5bf80b15b0 msgr2=0x7f5bf80b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:39.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.587+0000 7f5c00208640 1 -- 172.21.15.83:0/1004075080 shutdown_connections 2024-04-17T10:41:39.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.587+0000 7f5c00208640 1 -- 172.21.15.83:0/1004075080 wait complete. 2024-04-17T10:41:39.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.587+0000 7f5c00208640 1 Processor -- start 2024-04-17T10:41:39.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.587+0000 7f5c00208640 1 -- start start 2024-04-17T10:41:39.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.591+0000 7f5c00208640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5be4003a30 0x7f5bf8146700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:39.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.591+0000 7f5c00208640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bf8146eb0 0x7f5bf81553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:39.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.591+0000 7f5c00208640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bf814f1e0 0x7f5bf814f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:39.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.591+0000 7f5c00208640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf80b4450 con 0x7f5be4003a30 2024-04-17T10:41:39.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.591+0000 7f5c00208640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf81551c0 con 0x7f5bf814f1e0 2024-04-17T10:41:39.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.591+0000 7f5c00208640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf80b45d0 con 0x7f5bf8146eb0 2024-04-17T10:41:39.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.591+0000 7f5bfdfa4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5be4003a30 0x7f5bf8146700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:39.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.591+0000 7f5bfdfa4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5be4003a30 0x7f5bf8146700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51178/0 (socket says 172.21.15.83:51178) 2024-04-17T10:41:39.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.591+0000 7f5bfdfa4640 1 -- 172.21.15.83:0/1713201974 learned_addr learned my addr 172.21.15.83:0/1713201974 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:39.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.591+0000 7f5bfd7a3640 1 --2- 172.21.15.83:0/1713201974 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bf8146eb0 0x7f5bf81553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:39.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.591+0000 7f5bfdfa4640 1 -- 172.21.15.83:0/1713201974 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bf8146eb0 msgr2=0x7f5bf81553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:39.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.591+0000 7f5bfe7a5640 1 --2- 172.21.15.83:0/1713201974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bf814f1e0 0x7f5bf814f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:39.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.591+0000 7f5bfdfa4640 1 --2- 172.21.15.83:0/1713201974 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bf8146eb0 0x7f5bf81553d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:39.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.591+0000 7f5bfdfa4640 1 -- 172.21.15.83:0/1713201974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bf814f1e0 msgr2=0x7f5bf814f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:39.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.591+0000 7f5bfdfa4640 1 --2- 172.21.15.83:0/1713201974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bf814f1e0 0x7f5bf814f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:39.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.591+0000 7f5bfdfa4640 1 -- 172.21.15.83:0/1713201974 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf8157e80 con 0x7f5be4003a30 2024-04-17T10:41:39.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.591+0000 7f5bfdfa4640 1 --2- 172.21.15.83:0/1713201974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5be4003a30 0x7f5bf8146700 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4001ec0 tx=0x7f5be4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:39.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.591+0000 7f5bee7fc640 1 -- 172.21.15.83:0/1713201974 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4005d00 con 0x7f5be4003a30 2024-04-17T10:41:39.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.591+0000 7f5bee7fc640 1 -- 172.21.15.83:0/1713201974 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf40047f0 con 0x7f5be4003a30 2024-04-17T10:41:39.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.591+0000 7f5bee7fc640 1 -- 172.21.15.83:0/1713201974 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4004ba0 con 0x7f5be4003a30 2024-04-17T10:41:39.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.591+0000 7f5c00208640 1 -- 172.21.15.83:0/1713201974 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf8158160 con 0x7f5be4003a30 2024-04-17T10:41:39.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.591+0000 7f5c00208640 1 -- 172.21.15.83:0/1713201974 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf8146c40 con 0x7f5be4003a30 2024-04-17T10:41:39.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.591+0000 7f5c00208640 1 -- 172.21.15.83:0/1713201974 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bf8158330 con 0x7f5be4003a30 2024-04-17T10:41:39.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.591+0000 7f5bee7fc640 1 -- 172.21.15.83:0/1713201974 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5bf40a5710 con 0x7f5be4003a30 2024-04-17T10:41:39.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.591+0000 7f5bee7fc640 1 --2- 172.21.15.83:0/1713201974 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5bdc066f60 0x7f5bdc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:39.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.591+0000 7f5bfd7a3640 1 --2- 172.21.15.83:0/1713201974 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5bdc066f60 0x7f5bdc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:39.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.591+0000 7f5bee7fc640 1 -- 172.21.15.83:0/1713201974 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5bf411e5d0 con 0x7f5be4003a30 2024-04-17T10:41:39.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.591+0000 7f5bfd7a3640 1 --2- 172.21.15.83:0/1713201974 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5bdc066f60 0x7f5bdc069420 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5bf80cf510 tx=0x7f5be8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:39.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.595+0000 7f5bee7fc640 1 -- 172.21.15.83:0/1713201974 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5bf40169b0 con 0x7f5be4003a30 2024-04-17T10:41:39.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:39.783+0000 7f5c00208640 1 -- 172.21.15.83:0/1713201974 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5bf80cc7c0 con 0x7f5be4003a30 2024-04-17T10:41:40.396 INFO:tasks.ceph.mds.c.smithi083.stderr:2024-04-17T10:41:40.399+0000 7f894a955640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T10:41:40.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.455+0000 7f5bee7fc640 1 -- 172.21.15.83:0/1713201974 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v21) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5bf40ef5e0 con 0x7f5be4003a30 2024-04-17T10:41:40.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.459+0000 7f5c00208640 1 -- 172.21.15.83:0/1713201974 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5bdc066f60 msgr2=0x7f5bdc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:40.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.459+0000 7f5c00208640 1 --2- 172.21.15.83:0/1713201974 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5bdc066f60 0x7f5bdc069420 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5bf80cf510 tx=0x7f5be8008040 comp rx=0 tx=0).stop 2024-04-17T10:41:40.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.459+0000 7f5c00208640 1 -- 172.21.15.83:0/1713201974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5be4003a30 msgr2=0x7f5bf8146700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:40.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.459+0000 7f5c00208640 1 --2- 172.21.15.83:0/1713201974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5be4003a30 0x7f5bf8146700 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4001ec0 tx=0x7f5be4005840 comp rx=0 tx=0).stop 2024-04-17T10:41:40.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.459+0000 7f5c00208640 1 -- 172.21.15.83:0/1713201974 shutdown_connections 2024-04-17T10:41:40.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.459+0000 7f5c00208640 1 --2- 172.21.15.83:0/1713201974 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5bdc066f60 0x7f5bdc069420 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:40.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.459+0000 7f5c00208640 1 --2- 172.21.15.83:0/1713201974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bf814f1e0 0x7f5bf814f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:40.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.459+0000 7f5c00208640 1 --2- 172.21.15.83:0/1713201974 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bf8146eb0 0x7f5bf81553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:40.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.459+0000 7f5c00208640 1 --2- 172.21.15.83:0/1713201974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5be4003a30 0x7f5bf8146700 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:40.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.459+0000 7f5c00208640 1 -- 172.21.15.83:0/1713201974 >> 172.21.15.83:0/1713201974 conn(0x7f5bf80b15b0 msgr2=0x7f5bf80af250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:40.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.459+0000 7f5c00208640 1 -- 172.21.15.83:0/1713201974 shutdown_connections 2024-04-17T10:41:40.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.459+0000 7f5c00208640 1 -- 172.21.15.83:0/1713201974 wait complete. 2024-04-17T10:41:40.479 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-17T10:41:40.479 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:41:40.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03db11f640 1 Processor -- start 2024-04-17T10:41:40.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03db11f640 1 -- start start 2024-04-17T10:41:40.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03db11f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03d40b2de0 0x7f03d415ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:40.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03db11f640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f03d40d22d0 0x7f03d40ae520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:40.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03db11f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03d415f5b0 con 0x7f03d415f1d0 2024-04-17T10:41:40.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03db11f640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03d40b2620 con 0x7f03d40d22d0 2024-04-17T10:41:40.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03db11f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d40b2340 con 0x7f03d40b2de0 2024-04-17T10:41:40.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03d8ebb640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f03d415f1d0 0x7f03d415f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:36872/0 (socket says 172.21.15.83:36872) 2024-04-17T10:41:40.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03d8ebb640 1 -- 172.21.15.83:0/3270194157 learned_addr learned my addr 172.21.15.83:0/3270194157 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:40.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03d3fff640 1 --2- 172.21.15.83:0/3270194157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03d40b2de0 0x7f03d415ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:40.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03d96bc640 1 --2- 172.21.15.83:0/3270194157 >> v2:172.21.15.167:3300/0 conn(0x7f03d40d22d0 0x7f03d40ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:40.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03d37fe640 1 -- 172.21.15.83:0/3270194157 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1016802415 0 0) 0x7f03d415f5b0 con 0x7f03d415f1d0 2024-04-17T10:41:40.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03d37fe640 1 -- 172.21.15.83:0/3270194157 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03bc003530 con 0x7f03d415f1d0 2024-04-17T10:41:40.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03d37fe640 1 -- 172.21.15.83:0/3270194157 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1569144313 0 0) 0x7f03bc003530 con 0x7f03d415f1d0 2024-04-17T10:41:40.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03d37fe640 1 -- 172.21.15.83:0/3270194157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03d40b2de0 msgr2=0x7f03d415ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:40.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03d37fe640 1 --2- 172.21.15.83:0/3270194157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03d40b2de0 0x7f03d415ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:40.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03d37fe640 1 -- 172.21.15.83:0/3270194157 >> v2:172.21.15.167:3300/0 conn(0x7f03d40d22d0 msgr2=0x7f03d40ae520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:40.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03d37fe640 1 --2- 172.21.15.83:0/3270194157 >> v2:172.21.15.167:3300/0 conn(0x7f03d40d22d0 0x7f03d40ae520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:40.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03d37fe640 1 -- 172.21.15.83:0/3270194157 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d40b1760 con 0x7f03d415f1d0 2024-04-17T10:41:40.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03d37fe640 1 -- 172.21.15.83:0/3270194157 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f03cc001680 con 0x7f03d415f1d0 2024-04-17T10:41:40.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03d37fe640 1 -- 172.21.15.83:0/3270194157 >> v1:172.21.15.83:6789/0 conn(0x7f03d415f1d0 legacy=0x7f03d415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:40.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03d37fe640 1 --2- 172.21.15.83:0/3270194157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03bc0039c0 0x7f03bc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:40.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03d37fe640 1 --2- 172.21.15.83:0/3270194157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03bc004320 0x7f03bc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:40.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03d37fe640 1 --2- 172.21.15.83:0/3270194157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03bc004e60 0x7f03bc0052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:40.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03d37fe640 1 -- 172.21.15.83:0/3270194157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d40b2620 con 0x7f03bc004320 2024-04-17T10:41:40.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03d37fe640 1 -- 172.21.15.83:0/3270194157 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d40b2340 con 0x7f03bc004e60 2024-04-17T10:41:40.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03d37fe640 1 -- 172.21.15.83:0/3270194157 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc00bd50 con 0x7f03bc0039c0 2024-04-17T10:41:40.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03d3fff640 1 --2- 172.21.15.83:0/3270194157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03bc004320 0x7f03bc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03d8ebb640 1 --2- 172.21.15.83:0/3270194157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03bc0039c0 0x7f03bc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03d96bc640 1 --2- 172.21.15.83:0/3270194157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03bc004e60 0x7f03bc0052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03d3fff640 1 -- 172.21.15.83:0/3270194157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03bc0039c0 msgr2=0x7f03bc003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03d3fff640 1 --2- 172.21.15.83:0/3270194157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03bc0039c0 0x7f03bc003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03d3fff640 1 -- 172.21.15.83:0/3270194157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03bc004e60 msgr2=0x7f03bc0052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03d3fff640 1 --2- 172.21.15.83:0/3270194157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03bc004e60 0x7f03bc0052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03d3fff640 1 -- 172.21.15.83:0/3270194157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03bc003530 con 0x7f03bc004320 2024-04-17T10:41:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.603+0000 7f03d96bc640 1 --2- 172.21.15.83:0/3270194157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03bc004e60 0x7f03bc0052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.607+0000 7f03d3fff640 1 --2- 172.21.15.83:0/3270194157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03bc004320 0x7f03bc004810 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009f90 tx=0x7f03c801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ddc7d5ab5e396a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.607+0000 7f03d37fe640 1 -- 172.21.15.83:0/3270194157 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c80b2070 con 0x7f03bc004320 2024-04-17T10:41:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.607+0000 7f03d37fe640 1 -- 172.21.15.83:0/3270194157 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c80a5c50 con 0x7f03bc004320 2024-04-17T10:41:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.607+0000 7f03d37fe640 1 -- 172.21.15.83:0/3270194157 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c8009c10 con 0x7f03bc004320 2024-04-17T10:41:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.607+0000 7f03db11f640 1 -- 172.21.15.83:0/3270194157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03bc004320 msgr2=0x7f03bc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.607+0000 7f03db11f640 1 --2- 172.21.15.83:0/3270194157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03bc004320 0x7f03bc004810 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009f90 tx=0x7f03c801cc40 comp rx=0 tx=0).stop 2024-04-17T10:41:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.607+0000 7f03d96bc640 1 -- 172.21.15.83:0/3270194157 reap_dead start 2024-04-17T10:41:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.607+0000 7f03db11f640 1 -- 172.21.15.83:0/3270194157 shutdown_connections 2024-04-17T10:41:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.607+0000 7f03db11f640 1 -- 172.21.15.83:0/3270194157 >> 172.21.15.83:0/3270194157 conn(0x7f03d40d17b0 msgr2=0x7f03d40d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.607+0000 7f03db11f640 1 -- 172.21.15.83:0/3270194157 shutdown_connections 2024-04-17T10:41:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.607+0000 7f03db11f640 1 -- 172.21.15.83:0/3270194157 wait complete. 2024-04-17T10:41:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.607+0000 7f03db11f640 1 Processor -- start 2024-04-17T10:41:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.607+0000 7f03db11f640 1 -- start start 2024-04-17T10:41:40.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.611+0000 7f03db11f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03d414ed10 0x7f03d40aff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:40.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.611+0000 7f03db11f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03d40b04d0 0x7f03d415a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:40.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.611+0000 7f03db11f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03d415ad80 0x7f03d40ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:40.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.611+0000 7f03db11f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d415ff40 con 0x7f03d415ad80 2024-04-17T10:41:40.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.611+0000 7f03db11f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d40b1d20 con 0x7f03d414ed10 2024-04-17T10:41:40.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.611+0000 7f03db11f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d41600c0 con 0x7f03d40b04d0 2024-04-17T10:41:40.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.611+0000 7f03d96bc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03d415ad80 0x7f03d40ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:40.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.611+0000 7f03d96bc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03d415ad80 0x7f03d40ae520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51186/0 (socket says 172.21.15.83:51186) 2024-04-17T10:41:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.611+0000 7f03d96bc640 1 -- 172.21.15.83:0/1326884598 learned_addr learned my addr 172.21.15.83:0/1326884598 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.611+0000 7f03d8ebb640 1 --2- 172.21.15.83:0/1326884598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03d414ed10 0x7f03d40aff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.611+0000 7f03d96bc640 1 -- 172.21.15.83:0/1326884598 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03d40b04d0 msgr2=0x7f03d415a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.611+0000 7f03d96bc640 1 --2- 172.21.15.83:0/1326884598 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03d40b04d0 0x7f03d415a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.611+0000 7f03d96bc640 1 -- 172.21.15.83:0/1326884598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03d414ed10 msgr2=0x7f03d40aff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.611+0000 7f03d96bc640 1 --2- 172.21.15.83:0/1326884598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03d414ed10 0x7f03d40aff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.611+0000 7f03d96bc640 1 -- 172.21.15.83:0/1326884598 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d40c67c0 con 0x7f03d415ad80 2024-04-17T10:41:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.611+0000 7f03d96bc640 1 --2- 172.21.15.83:0/1326884598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03d415ad80 0x7f03d40ae520 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f03d40717e0 tx=0x7f03c4010b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.611+0000 7f03d17fa640 1 -- 172.21.15.83:0/1326884598 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c400f040 con 0x7f03d415ad80 2024-04-17T10:41:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.611+0000 7f03d17fa640 1 -- 172.21.15.83:0/1326884598 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c401c070 con 0x7f03d415ad80 2024-04-17T10:41:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.611+0000 7f03db11f640 1 -- 172.21.15.83:0/1326884598 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d40c6aa0 con 0x7f03d415ad80 2024-04-17T10:41:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.611+0000 7f03d17fa640 1 -- 172.21.15.83:0/1326884598 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c400eaa0 con 0x7f03d415ad80 2024-04-17T10:41:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.611+0000 7f03db11f640 1 -- 172.21.15.83:0/1326884598 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d40c7060 con 0x7f03d415ad80 2024-04-17T10:41:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.611+0000 7f03db11f640 1 -- 172.21.15.83:0/1326884598 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d4074f90 con 0x7f03d415ad80 2024-04-17T10:41:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.611+0000 7f03d17fa640 1 -- 172.21.15.83:0/1326884598 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f03c400ec40 con 0x7f03d415ad80 2024-04-17T10:41:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.611+0000 7f03d17fa640 1 --2- 172.21.15.83:0/1326884598 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f03c0066f90 0x7f03c0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.611+0000 7f03d17fa640 1 -- 172.21.15.83:0/1326884598 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03c4084b30 con 0x7f03d415ad80 2024-04-17T10:41:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.611+0000 7f03d8ebb640 1 --2- 172.21.15.83:0/1326884598 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f03c0066f90 0x7f03c0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.611+0000 7f03d8ebb640 1 --2- 172.21.15.83:0/1326884598 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f03c0066f90 0x7f03c0069450 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f03cc000fc0 tx=0x7f03cc0013c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:40.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.615+0000 7f03d17fa640 1 -- 172.21.15.83:0/1326884598 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f03c4026020 con 0x7f03d415ad80 2024-04-17T10:41:40.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.799+0000 7f03db11f640 1 -- 172.21.15.83:0/1326884598 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03d4072690 con 0x7f03d415ad80 2024-04-17T10:41:40.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.799+0000 7f03d17fa640 1 -- 172.21.15.83:0/1326884598 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f03c4055bd0 con 0x7f03d415ad80 2024-04-17T10:41:40.799 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:41:40.799 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/161398126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":161398126},{"type":"v1","addr":"172.21.15.167:6833","nonce":161398126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1085380618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1085380618},{"type":"v1","addr":"172.21.15.167:6835","nonce":1085380618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/836442291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":836442291},{"type":"v1","addr":"172.21.15.83:6835","nonce":836442291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:41:37.773342+0000","modified":"2024-04-17T10:41:39.788394+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5682":{"gid":5682,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1894547386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1894547386},{"type":"v1","addr":"172.21.15.83:6837","nonce":1894547386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-17T10:41:40.799 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 21 2024-04-17T10:41:40.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.803+0000 7f03db11f640 1 -- 172.21.15.83:0/1326884598 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f03c0066f90 msgr2=0x7f03c0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:40.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.803+0000 7f03db11f640 1 --2- 172.21.15.83:0/1326884598 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f03c0066f90 0x7f03c0069450 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f03cc000fc0 tx=0x7f03cc0013c0 comp rx=0 tx=0).stop 2024-04-17T10:41:40.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.803+0000 7f03db11f640 1 -- 172.21.15.83:0/1326884598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03d415ad80 msgr2=0x7f03d40ae520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:40.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.803+0000 7f03db11f640 1 --2- 172.21.15.83:0/1326884598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03d415ad80 0x7f03d40ae520 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f03d40717e0 tx=0x7f03c4010b60 comp rx=0 tx=0).stop 2024-04-17T10:41:40.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.803+0000 7f03db11f640 1 -- 172.21.15.83:0/1326884598 shutdown_connections 2024-04-17T10:41:40.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.803+0000 7f03db11f640 1 --2- 172.21.15.83:0/1326884598 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f03c0066f90 0x7f03c0069450 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:40.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.803+0000 7f03db11f640 1 --2- 172.21.15.83:0/1326884598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03d415ad80 0x7f03d40ae520 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:40.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.803+0000 7f03db11f640 1 --2- 172.21.15.83:0/1326884598 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03d40b04d0 0x7f03d415a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:40.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.803+0000 7f03db11f640 1 --2- 172.21.15.83:0/1326884598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03d414ed10 0x7f03d40aff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:40.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.803+0000 7f03db11f640 1 -- 172.21.15.83:0/1326884598 >> 172.21.15.83:0/1326884598 conn(0x7f03d40d17b0 msgr2=0x7f03d415e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:40.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.803+0000 7f03db11f640 1 -- 172.21.15.83:0/1326884598 shutdown_connections 2024-04-17T10:41:40.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.803+0000 7f03db11f640 1 -- 172.21.15.83:0/1326884598 wait complete. 2024-04-17T10:41:40.819 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:41:40.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.927+0000 7f95c27e7640 1 Processor -- start 2024-04-17T10:41:40.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.927+0000 7f95c27e7640 1 -- start start 2024-04-17T10:41:40.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.931+0000 7f95c27e7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f95bc15ba30 0x7f95bc072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:40.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.931+0000 7f95c27e7640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95bc06e320 con 0x7f95bc0cfbe0 2024-04-17T10:41:40.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.931+0000 7f95c27e7640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95bc06e040 con 0x7f95bc15c960 2024-04-17T10:41:40.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.931+0000 7f95c27e7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95bc071ea0 con 0x7f95bc15ba30 2024-04-17T10:41:40.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.931+0000 7f95c0d84640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f95bc0cfbe0 0x7f95bc06c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:36876/0 (socket says 172.21.15.83:36876) 2024-04-17T10:41:40.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.931+0000 7f95c0d84640 1 -- 172.21.15.83:0/960616754 learned_addr learned my addr 172.21.15.83:0/960616754 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:40.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.931+0000 7f95baffd640 1 -- 172.21.15.83:0/960616754 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3078692354 0 0) 0x7f95bc06e320 con 0x7f95bc0cfbe0 2024-04-17T10:41:40.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.931+0000 7f95baffd640 1 -- 172.21.15.83:0/960616754 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95a8003530 con 0x7f95bc0cfbe0 2024-04-17T10:41:40.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.931+0000 7f95bbfff640 1 --2- 172.21.15.83:0/960616754 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f95bc15ba30 0x7f95bc072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:40.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.931+0000 7f95baffd640 1 -- 172.21.15.83:0/960616754 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1266101019 0 0) 0x7f95a8003530 con 0x7f95bc0cfbe0 2024-04-17T10:41:40.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.931+0000 7f95baffd640 1 -- 172.21.15.83:0/960616754 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f95bc15ba30 msgr2=0x7f95bc072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:40.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.931+0000 7f95baffd640 1 --2- 172.21.15.83:0/960616754 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f95bc15ba30 0x7f95bc072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:40.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.931+0000 7f95baffd640 1 -- 172.21.15.83:0/960616754 >> v1:172.21.15.167:6789/0 conn(0x7f95bc15c960 legacy=0x7f95bc0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:40.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.931+0000 7f95baffd640 1 -- 172.21.15.83:0/960616754 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95bc071850 con 0x7f95bc0cfbe0 2024-04-17T10:41:40.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.931+0000 7f95baffd640 1 -- 172.21.15.83:0/960616754 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f95b4001a20 con 0x7f95bc0cfbe0 2024-04-17T10:41:40.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.931+0000 7f95baffd640 1 -- 172.21.15.83:0/960616754 >> v1:172.21.15.83:6789/0 conn(0x7f95bc0cfbe0 legacy=0x7f95bc06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:40.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.931+0000 7f95baffd640 1 --2- 172.21.15.83:0/960616754 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f95bc15ba30 0x7f95a80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:40.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.931+0000 7f95baffd640 1 --2- 172.21.15.83:0/960616754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95a8003f70 0x7f95a8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:40.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.931+0000 7f95baffd640 1 --2- 172.21.15.83:0/960616754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a8004d50 0x7f95a80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:40.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.931+0000 7f95baffd640 1 -- 172.21.15.83:0/960616754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95bc071ea0 con 0x7f95a8004d50 2024-04-17T10:41:40.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.931+0000 7f95baffd640 1 -- 172.21.15.83:0/960616754 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a800bb60 con 0x7f95a8003f70 2024-04-17T10:41:40.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.931+0000 7f95baffd640 1 -- 172.21.15.83:0/960616754 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a800bd60 con 0x7f95bc15ba30 2024-04-17T10:41:40.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.931+0000 7f95c0d84640 1 --2- 172.21.15.83:0/960616754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a8004d50 0x7f95a80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:40.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.931+0000 7f95bbfff640 1 --2- 172.21.15.83:0/960616754 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f95bc15ba30 0x7f95a80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:40.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.931+0000 7f95c0d84640 1 -- 172.21.15.83:0/960616754 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f95bc15ba30 msgr2=0x7f95a80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:40.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.931+0000 7f95bb7fe640 1 --2- 172.21.15.83:0/960616754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95a8003f70 0x7f95a8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:40.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.931+0000 7f95c0d84640 1 --2- 172.21.15.83:0/960616754 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f95bc15ba30 0x7f95a80039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:40.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.931+0000 7f95c0d84640 1 -- 172.21.15.83:0/960616754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95a8003f70 msgr2=0x7f95a8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:40.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.931+0000 7f95c0d84640 1 --2- 172.21.15.83:0/960616754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95a8003f70 0x7f95a8004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:40.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.931+0000 7f95c0d84640 1 -- 172.21.15.83:0/960616754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a8003530 con 0x7f95a8004d50 2024-04-17T10:41:40.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.931+0000 7f95bbfff640 1 --2- 172.21.15.83:0/960616754 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f95bc15ba30 0x7f95a80039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:40.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95c0d84640 1 --2- 172.21.15.83:0/960616754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a8004d50 0x7f95a80051d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f95b400a8f0 tx=0x7f95b401fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d8961c2d6f17a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:40.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95baffd640 1 -- 172.21.15.83:0/960616754 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b400a4c0 con 0x7f95a8004d50 2024-04-17T10:41:40.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95baffd640 1 -- 172.21.15.83:0/960616754 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b40aa700 con 0x7f95a8004d50 2024-04-17T10:41:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95baffd640 1 -- 172.21.15.83:0/960616754 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b40aaad0 con 0x7f95a8004d50 2024-04-17T10:41:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95c27e7640 1 -- 172.21.15.83:0/960616754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a8004d50 msgr2=0x7f95a80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95c27e7640 1 --2- 172.21.15.83:0/960616754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a8004d50 0x7f95a80051d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f95b400a8f0 tx=0x7f95b401fb60 comp rx=0 tx=0).stop 2024-04-17T10:41:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95c0d84640 1 -- 172.21.15.83:0/960616754 reap_dead start 2024-04-17T10:41:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95c27e7640 1 -- 172.21.15.83:0/960616754 shutdown_connections 2024-04-17T10:41:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95c27e7640 1 -- 172.21.15.83:0/960616754 >> 172.21.15.83:0/960616754 conn(0x7f95bc06abe0 msgr2=0x7f95bc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95c27e7640 1 -- 172.21.15.83:0/960616754 shutdown_connections 2024-04-17T10:41:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95c27e7640 1 -- 172.21.15.83:0/960616754 wait complete. 2024-04-17T10:41:40.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95c27e7640 1 Processor -- start 2024-04-17T10:41:40.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95c27e7640 1 -- start start 2024-04-17T10:41:40.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95c27e7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a8004d50 0x7f95bc1f6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:40.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95c27e7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95bc15e9c0 0x7f95bc1574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:40.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95c27e7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f95bc0b0090 0x7f95bc0c45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:40.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95c27e7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95bc0aeae0 con 0x7f95a8004d50 2024-04-17T10:41:40.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95c27e7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95bc15cde0 con 0x7f95bc15e9c0 2024-04-17T10:41:40.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95c27e7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95bc06dca0 con 0x7f95bc0b0090 2024-04-17T10:41:40.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95bbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a8004d50 0x7f95bc1f6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:40.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95bbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a8004d50 0x7f95bc1f6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51210/0 (socket says 172.21.15.83:51210) 2024-04-17T10:41:40.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95bbfff640 1 -- 172.21.15.83:0/336106218 learned_addr learned my addr 172.21.15.83:0/336106218 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:40.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95bb7fe640 1 --2- 172.21.15.83:0/336106218 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95bc15e9c0 0x7f95bc1574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:40.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95bbfff640 1 -- 172.21.15.83:0/336106218 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f95bc0b0090 msgr2=0x7f95bc0c45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:40.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95bbfff640 1 --2- 172.21.15.83:0/336106218 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f95bc0b0090 0x7f95bc0c45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:40.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95bbfff640 1 -- 172.21.15.83:0/336106218 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95bc15e9c0 msgr2=0x7f95bc1574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:40.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95bbfff640 1 --2- 172.21.15.83:0/336106218 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95bc15e9c0 0x7f95bc1574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:40.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95bbfff640 1 -- 172.21.15.83:0/336106218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95bc0c5570 con 0x7f95a8004d50 2024-04-17T10:41:40.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95bbfff640 1 --2- 172.21.15.83:0/336106218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a8004d50 0x7f95bc1f6790 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f95ac010900 tx=0x7f95ac010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:40.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95b8ff9640 1 -- 172.21.15.83:0/336106218 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac00f040 con 0x7f95a8004d50 2024-04-17T10:41:40.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95b8ff9640 1 -- 172.21.15.83:0/336106218 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95ac004220 con 0x7f95a8004d50 2024-04-17T10:41:40.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95c27e7640 1 -- 172.21.15.83:0/336106218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95bc1f6e20 con 0x7f95a8004d50 2024-04-17T10:41:40.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95b8ff9640 1 -- 172.21.15.83:0/336106218 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac00e9a0 con 0x7f95a8004d50 2024-04-17T10:41:40.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.935+0000 7f95c27e7640 1 -- 172.21.15.83:0/336106218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95bc1f73e0 con 0x7f95a8004d50 2024-04-17T10:41:40.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.939+0000 7f95c27e7640 1 -- 172.21.15.83:0/336106218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95bc074f90 con 0x7f95a8004d50 2024-04-17T10:41:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.939+0000 7f95b8ff9640 1 -- 172.21.15.83:0/336106218 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f95ac00eb40 con 0x7f95a8004d50 2024-04-17T10:41:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.939+0000 7f95b8ff9640 1 --2- 172.21.15.83:0/336106218 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f959c066f90 0x7f959c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.939+0000 7f95b8ff9640 1 -- 172.21.15.83:0/336106218 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f95ac085070 con 0x7f95a8004d50 2024-04-17T10:41:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.939+0000 7f95bb7fe640 1 --2- 172.21.15.83:0/336106218 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f959c066f90 0x7f959c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.939+0000 7f95bb7fe640 1 --2- 172.21.15.83:0/336106218 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f959c066f90 0x7f959c069450 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f95a4007fd0 tx=0x7f95a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:40.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:40.943+0000 7f95b8ff9640 1 -- 172.21.15.83:0/336106218 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f95ac018030 con 0x7f95a8004d50 2024-04-17T10:41:41.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.099+0000 7f95c27e7640 1 -- 172.21.15.83:0/336106218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95bc0b4760 con 0x7f95a8004d50 2024-04-17T10:41:41.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.103+0000 7f95b8ff9640 1 -- 172.21.15.83:0/336106218 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f95ac055fe0 con 0x7f95a8004d50 2024-04-17T10:41:41.102 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:41:41.102 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":46,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:41:39.443033+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:41:36.031388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-17T10:41:36.783410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:41:41.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.107+0000 7f95c27e7640 1 -- 172.21.15.83:0/336106218 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f959c066f90 msgr2=0x7f959c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:41.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.107+0000 7f95c27e7640 1 --2- 172.21.15.83:0/336106218 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f959c066f90 0x7f959c069450 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f95a4007fd0 tx=0x7f95a4009040 comp rx=0 tx=0).stop 2024-04-17T10:41:41.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.107+0000 7f95c27e7640 1 -- 172.21.15.83:0/336106218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a8004d50 msgr2=0x7f95bc1f6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:41.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.107+0000 7f95c27e7640 1 --2- 172.21.15.83:0/336106218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a8004d50 0x7f95bc1f6790 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f95ac010900 tx=0x7f95ac010dc0 comp rx=0 tx=0).stop 2024-04-17T10:41:41.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.107+0000 7f95c27e7640 1 -- 172.21.15.83:0/336106218 shutdown_connections 2024-04-17T10:41:41.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.107+0000 7f95c27e7640 1 --2- 172.21.15.83:0/336106218 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f959c066f90 0x7f959c069450 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:41.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.107+0000 7f95c27e7640 1 --2- 172.21.15.83:0/336106218 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f95bc0b0090 0x7f95bc0c45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:41.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.107+0000 7f95c27e7640 1 --2- 172.21.15.83:0/336106218 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95bc15e9c0 0x7f95bc1574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:41.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.107+0000 7f95c27e7640 1 --2- 172.21.15.83:0/336106218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a8004d50 0x7f95bc1f6790 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:41.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.107+0000 7f95c27e7640 1 -- 172.21.15.83:0/336106218 >> 172.21.15.83:0/336106218 conn(0x7f95bc06abe0 msgr2=0x7f95bc0ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:41.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.107+0000 7f95c27e7640 1 -- 172.21.15.83:0/336106218 shutdown_connections 2024-04-17T10:41:41.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.107+0000 7f95c27e7640 1 -- 172.21.15.83:0/336106218 wait complete. 2024-04-17T10:41:41.121 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-17T10:41:41.121 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:41:41.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.247+0000 7fc75f8b6640 1 Processor -- start 2024-04-17T10:41:41.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.247+0000 7fc75f8b6640 1 -- start start 2024-04-17T10:41:41.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.247+0000 7fc75f8b6640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc7580cf400 0x7fc7580cc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:41.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.247+0000 7fc75f8b6640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc7580cf890 0x7fc7580cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:41.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.247+0000 7fc75f8b6640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc75815bef0 con 0x7fc75815f610 2024-04-17T10:41:41.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.247+0000 7fc75f8b6640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc75815c1b0 con 0x7fc7580cf890 2024-04-17T10:41:41.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.247+0000 7fc75f8b6640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc75815c490 con 0x7fc7580cf400 2024-04-17T10:41:41.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.247+0000 7fc75de53640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc7580cf890 0x7fc7580cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:41.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.247+0000 7fc75de53640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc7580cf890 0x7fc7580cc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51220/0 (socket says 172.21.15.83:51220) 2024-04-17T10:41:41.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.247+0000 7fc75d652640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc75815f610 0x7fc75815f9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:36890/0 (socket says 172.21.15.83:36890) 2024-04-17T10:41:41.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.247+0000 7fc75de53640 1 -- 172.21.15.83:0/3721993368 learned_addr learned my addr 172.21.15.83:0/3721993368 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:41.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.247+0000 7fc75ce51640 1 --2- 172.21.15.83:0/3721993368 >> v2:172.21.15.167:3300/0 conn(0x7fc7580cf400 0x7fc7580cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:41.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.247+0000 7fc75de53640 1 -- 172.21.15.83:0/3721993368 >> v2:172.21.15.167:3300/0 conn(0x7fc7580cf400 msgr2=0x7fc7580cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:41.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.247+0000 7fc75de53640 1 --2- 172.21.15.83:0/3721993368 >> v2:172.21.15.167:3300/0 conn(0x7fc7580cf400 0x7fc7580cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:41.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.247+0000 7fc75de53640 1 -- 172.21.15.83:0/3721993368 >> v1:172.21.15.83:6789/0 conn(0x7fc75815f610 legacy=0x7fc75815f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:41.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.247+0000 7fc757fff640 1 -- 172.21.15.83:0/3721993368 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2929558524 0 0) 0x7fc75815bef0 con 0x7fc75815f610 2024-04-17T10:41:41.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.247+0000 7fc75de53640 1 -- 172.21.15.83:0/3721993368 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75815c770 con 0x7fc7580cf890 2024-04-17T10:41:41.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.251+0000 7fc75de53640 1 --2- 172.21.15.83:0/3721993368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7580cf890 0x7fc7580cc9a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc758071a80 tx=0x7fc74801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40bac71149dc86e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:41.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.251+0000 7fc757fff640 1 -- 172.21.15.83:0/3721993368 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7480a5620 con 0x7fc7580cf890 2024-04-17T10:41:41.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.251+0000 7fc757fff640 1 -- 172.21.15.83:0/3721993368 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7480a57c0 con 0x7fc7580cf890 2024-04-17T10:41:41.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.251+0000 7fc757fff640 1 -- 172.21.15.83:0/3721993368 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7480a5b90 con 0x7fc7580cf890 2024-04-17T10:41:41.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.251+0000 7fc75f8b6640 1 -- 172.21.15.83:0/3721993368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7580cf890 msgr2=0x7fc7580cc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:41.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.251+0000 7fc75f8b6640 1 --2- 172.21.15.83:0/3721993368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7580cf890 0x7fc7580cc9a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc758071a80 tx=0x7fc74801c960 comp rx=0 tx=0).stop 2024-04-17T10:41:41.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.251+0000 7fc75f8b6640 1 -- 172.21.15.83:0/3721993368 shutdown_connections 2024-04-17T10:41:41.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.251+0000 7fc75f8b6640 1 --2- 172.21.15.83:0/3721993368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7580cf890 0x7fc7580cc9a0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:41.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.251+0000 7fc75f8b6640 1 --2- 172.21.15.83:0/3721993368 >> v2:172.21.15.167:3300/0 conn(0x7fc7580cf400 0x7fc7580cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:41.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.251+0000 7fc75f8b6640 1 -- 172.21.15.83:0/3721993368 >> 172.21.15.83:0/3721993368 conn(0x7fc7580d1830 msgr2=0x7fc7580d1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:41.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.251+0000 7fc75f8b6640 1 -- 172.21.15.83:0/3721993368 shutdown_connections 2024-04-17T10:41:41.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.251+0000 7fc75f8b6640 1 -- 172.21.15.83:0/3721993368 wait complete. 2024-04-17T10:41:41.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.255+0000 7fc75f8b6640 1 Processor -- start 2024-04-17T10:41:41.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.255+0000 7fc75f8b6640 1 -- start start 2024-04-17T10:41:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.255+0000 7fc75f8b6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7580cf400 0x7fc75814a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.255+0000 7fc75f8b6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7580cf890 0x7fc7580c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.255+0000 7fc75f8b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc75815f610 0x7fc75815a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.255+0000 7fc75f8b6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc758168ce0 con 0x7fc75815f610 2024-04-17T10:41:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.255+0000 7fc75f8b6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc758168b60 con 0x7fc7580cf400 2024-04-17T10:41:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.255+0000 7fc75f8b6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc758168e60 con 0x7fc7580cf890 2024-04-17T10:41:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.255+0000 7fc75de53640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc75815f610 0x7fc75815a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.255+0000 7fc75de53640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc75815f610 0x7fc75815a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51234/0 (socket says 172.21.15.83:51234) 2024-04-17T10:41:41.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.255+0000 7fc75de53640 1 -- 172.21.15.83:0/1602408539 learned_addr learned my addr 172.21.15.83:0/1602408539 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:41.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.255+0000 7fc75d652640 1 --2- 172.21.15.83:0/1602408539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7580cf400 0x7fc75814a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:41.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.255+0000 7fc75de53640 1 -- 172.21.15.83:0/1602408539 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7580cf890 msgr2=0x7fc7580c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:41.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.255+0000 7fc75de53640 1 --2- 172.21.15.83:0/1602408539 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7580cf890 0x7fc7580c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:41.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.255+0000 7fc75de53640 1 -- 172.21.15.83:0/1602408539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7580cf400 msgr2=0x7fc75814a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:41.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.255+0000 7fc75de53640 1 --2- 172.21.15.83:0/1602408539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7580cf400 0x7fc75814a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:41.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.255+0000 7fc75de53640 1 -- 172.21.15.83:0/1602408539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7580c7f20 con 0x7fc75815f610 2024-04-17T10:41:41.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.255+0000 7fc75d652640 1 --2- 172.21.15.83:0/1602408539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7580cf400 0x7fc75814a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:41.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.255+0000 7fc75de53640 1 --2- 172.21.15.83:0/1602408539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc75815f610 0x7fc75815a500 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc7480af040 tx=0x7fc748019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:41.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.255+0000 7fc755ffb640 1 -- 172.21.15.83:0/1602408539 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7480bc020 con 0x7fc75815f610 2024-04-17T10:41:41.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.255+0000 7fc755ffb640 1 -- 172.21.15.83:0/1602408539 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7480b8670 con 0x7fc75815f610 2024-04-17T10:41:41.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.255+0000 7fc75f8b6640 1 -- 172.21.15.83:0/1602408539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7580c8aa0 con 0x7fc75815f610 2024-04-17T10:41:41.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.255+0000 7fc755ffb640 1 -- 172.21.15.83:0/1602408539 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748009510 con 0x7fc75815f610 2024-04-17T10:41:41.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.255+0000 7fc75f8b6640 1 -- 172.21.15.83:0/1602408539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc75814afa0 con 0x7fc75815f610 2024-04-17T10:41:41.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.255+0000 7fc75f8b6640 1 -- 172.21.15.83:0/1602408539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc758074f90 con 0x7fc75815f610 2024-04-17T10:41:41.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.255+0000 7fc755ffb640 1 -- 172.21.15.83:0/1602408539 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc7480bf020 con 0x7fc75815f610 2024-04-17T10:41:41.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.259+0000 7fc755ffb640 1 --2- 172.21.15.83:0/1602408539 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc744066f60 0x7fc744069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:41.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.259+0000 7fc75d652640 1 --2- 172.21.15.83:0/1602408539 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc744066f60 0x7fc744069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:41.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.259+0000 7fc755ffb640 1 -- 172.21.15.83:0/1602408539 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc74811fff0 con 0x7fc75815f610 2024-04-17T10:41:41.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.259+0000 7fc75d652640 1 --2- 172.21.15.83:0/1602408539 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc744066f60 0x7fc744069420 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc750007fd0 tx=0x7fc750009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:41.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.263+0000 7fc755ffb640 1 -- 172.21.15.83:0/1602408539 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc7480c6030 con 0x7fc75815f610 2024-04-17T10:41:41.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.415+0000 7fc75f8b6640 1 -- 172.21.15.83:0/1602408539 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc7580cdbb0 con 0x7fc744066f60 2024-04-17T10:41:41.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.435+0000 7fc755ffb640 1 -- 172.21.15.83:0/1602408539 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251887 (secure 0 0 0) 0x7fc7580cdbb0 con 0x7fc744066f60 2024-04-17T10:41:41.437 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:41:41.439 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:41:41.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.439+0000 7fc75f8b6640 1 -- 172.21.15.83:0/1602408539 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc744066f60 msgr2=0x7fc744069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:41.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.439+0000 7fc75f8b6640 1 --2- 172.21.15.83:0/1602408539 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc744066f60 0x7fc744069420 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc750007fd0 tx=0x7fc750009040 comp rx=0 tx=0).stop 2024-04-17T10:41:41.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.439+0000 7fc75f8b6640 1 -- 172.21.15.83:0/1602408539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc75815f610 msgr2=0x7fc75815a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:41.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.439+0000 7fc75f8b6640 1 --2- 172.21.15.83:0/1602408539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc75815f610 0x7fc75815a500 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc7480af040 tx=0x7fc748019700 comp rx=0 tx=0).stop 2024-04-17T10:41:41.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.439+0000 7fc75f8b6640 1 -- 172.21.15.83:0/1602408539 shutdown_connections 2024-04-17T10:41:41.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.439+0000 7fc75f8b6640 1 --2- 172.21.15.83:0/1602408539 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc744066f60 0x7fc744069420 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:41.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.439+0000 7fc75f8b6640 1 --2- 172.21.15.83:0/1602408539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc75815f610 0x7fc75815a500 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:41.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.439+0000 7fc75f8b6640 1 --2- 172.21.15.83:0/1602408539 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7580cf890 0x7fc7580c74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:41.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.439+0000 7fc75f8b6640 1 --2- 172.21.15.83:0/1602408539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7580cf400 0x7fc75814a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:41.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.439+0000 7fc75f8b6640 1 -- 172.21.15.83:0/1602408539 >> 172.21.15.83:0/1602408539 conn(0x7fc7580d1830 msgr2=0x7fc75815ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:41.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.439+0000 7fc75f8b6640 1 -- 172.21.15.83:0/1602408539 shutdown_connections 2024-04-17T10:41:41.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.439+0000 7fc75f8b6640 1 -- 172.21.15.83:0/1602408539 wait complete. 2024-04-17T10:41:41.454 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-04-17T10:41:40.243750+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":31,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":342252,"kb_used_data":5352,"kb_used_omap":12,"kb_used_meta":336755,"kb_avail":754632468,"statfs":{"total":773094113280,"available":772743647232,"internally_reserved":0,"allocated":5480448,"data_stored":2912348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":344837680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001195"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.457624+0000","last_change":"2024-04-17T10:41:38.454812+0000","last_active":"2024-04-17T10:41:38.457624+0000","last_peered":"2024-04-17T10:41:38.457624+0000","last_clean":"2024-04-17T10:41:38.457624+0000","last_became_active":"2024-04-17T10:41:38.454068+0000","last_became_peered":"2024-04-17T10:41:38.454068+0000","last_unstale":"2024-04-17T10:41:38.457624+0000","last_undegraded":"2024-04-17T10:41:38.457624+0000","last_fullsized":"2024-04-17T10:41:38.457624+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:09:29.824195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.482293+0000","last_change":"2024-04-17T10:41:38.475143+0000","last_active":"2024-04-17T10:41:38.482293+0000","last_peered":"2024-04-17T10:41:38.482293+0000","last_clean":"2024-04-17T10:41:38.482293+0000","last_became_active":"2024-04-17T10:41:38.464378+0000","last_became_peered":"2024-04-17T10:41:38.464378+0000","last_unstale":"2024-04-17T10:41:38.482293+0000","last_undegraded":"2024-04-17T10:41:38.482293+0000","last_fullsized":"2024-04-17T10:41:38.482293+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:41:48.464779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.464984+0000","last_change":"2024-04-17T10:41:38.456591+0000","last_active":"2024-04-17T10:41:38.464984+0000","last_peered":"2024-04-17T10:41:38.464984+0000","last_clean":"2024-04-17T10:41:38.464984+0000","last_became_active":"2024-04-17T10:41:38.455835+0000","last_became_peered":"2024-04-17T10:41:38.455835+0000","last_unstale":"2024-04-17T10:41:38.464984+0000","last_undegraded":"2024-04-17T10:41:38.464984+0000","last_fullsized":"2024-04-17T10:41:38.464984+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:30:16.883126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.475196+0000","last_change":"2024-04-17T10:41:38.471953+0000","last_active":"2024-04-17T10:41:38.475196+0000","last_peered":"2024-04-17T10:41:38.475196+0000","last_clean":"2024-04-17T10:41:38.475196+0000","last_became_active":"2024-04-17T10:41:38.471638+0000","last_became_peered":"2024-04-17T10:41:38.471638+0000","last_unstale":"2024-04-17T10:41:38.475196+0000","last_undegraded":"2024-04-17T10:41:38.475196+0000","last_fullsized":"2024-04-17T10:41:38.475196+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:26:30.980989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.482461+0000","last_change":"2024-04-17T10:41:38.476262+0000","last_active":"2024-04-17T10:41:38.482461+0000","last_peered":"2024-04-17T10:41:38.482461+0000","last_clean":"2024-04-17T10:41:38.482461+0000","last_became_active":"2024-04-17T10:41:38.471052+0000","last_became_peered":"2024-04-17T10:41:38.471052+0000","last_unstale":"2024-04-17T10:41:38.482461+0000","last_undegraded":"2024-04-17T10:41:38.482461+0000","last_fullsized":"2024-04-17T10:41:38.482461+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:38:10.610378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.456541+0000","last_change":"2024-04-17T10:41:38.452412+0000","last_active":"2024-04-17T10:41:38.456541+0000","last_peered":"2024-04-17T10:41:38.456541+0000","last_clean":"2024-04-17T10:41:38.456541+0000","last_became_active":"2024-04-17T10:41:38.449166+0000","last_became_peered":"2024-04-17T10:41:38.449166+0000","last_unstale":"2024-04-17T10:41:38.456541+0000","last_undegraded":"2024-04-17T10:41:38.456541+0000","last_fullsized":"2024-04-17T10:41:38.456541+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:04:11.620082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.473230+0000","last_change":"2024-04-17T10:41:38.467456+0000","last_active":"2024-04-17T10:41:38.473230+0000","last_peered":"2024-04-17T10:41:38.473230+0000","last_clean":"2024-04-17T10:41:38.473230+0000","last_became_active":"2024-04-17T10:41:38.466706+0000","last_became_peered":"2024-04-17T10:41:38.466706+0000","last_unstale":"2024-04-17T10:41:38.473230+0000","last_undegraded":"2024-04-17T10:41:38.473230+0000","last_fullsized":"2024-04-17T10:41:38.473230+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:25:44.023009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.474950+0000","last_change":"2024-04-17T10:41:38.471695+0000","last_active":"2024-04-17T10:41:38.474950+0000","last_peered":"2024-04-17T10:41:38.474950+0000","last_clean":"2024-04-17T10:41:38.474950+0000","last_became_active":"2024-04-17T10:41:38.470467+0000","last_became_peered":"2024-04-17T10:41:38.470467+0000","last_unstale":"2024-04-17T10:41:38.474950+0000","last_undegraded":"2024-04-17T10:41:38.474950+0000","last_fullsized":"2024-04-17T10:41:38.474950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:36:33.039968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.475062+0000","last_change":"2024-04-17T10:41:38.471737+0000","last_active":"2024-04-17T10:41:38.475062+0000","last_peered":"2024-04-17T10:41:38.475062+0000","last_clean":"2024-04-17T10:41:38.475062+0000","last_became_active":"2024-04-17T10:41:38.469841+0000","last_became_peered":"2024-04-17T10:41:38.469841+0000","last_unstale":"2024-04-17T10:41:38.475062+0000","last_undegraded":"2024-04-17T10:41:38.475062+0000","last_fullsized":"2024-04-17T10:41:38.475062+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:30:16.883126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.474867+0000","last_change":"2024-04-17T10:41:38.469595+0000","last_active":"2024-04-17T10:41:38.474867+0000","last_peered":"2024-04-17T10:41:38.474867+0000","last_clean":"2024-04-17T10:41:38.474867+0000","last_became_active":"2024-04-17T10:41:38.469283+0000","last_became_peered":"2024-04-17T10:41:38.469283+0000","last_unstale":"2024-04-17T10:41:38.474867+0000","last_undegraded":"2024-04-17T10:41:38.474867+0000","last_fullsized":"2024-04-17T10:41:38.474867+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:07:17.088945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.475283+0000","last_change":"2024-04-17T10:41:38.472253+0000","last_active":"2024-04-17T10:41:38.475283+0000","last_peered":"2024-04-17T10:41:38.475283+0000","last_clean":"2024-04-17T10:41:38.475283+0000","last_became_active":"2024-04-17T10:41:38.471689+0000","last_became_peered":"2024-04-17T10:41:38.471689+0000","last_unstale":"2024-04-17T10:41:38.475283+0000","last_undegraded":"2024-04-17T10:41:38.475283+0000","last_fullsized":"2024-04-17T10:41:38.475283+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:03:40.637698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.458896+0000","last_change":"2024-04-17T10:41:38.451210+0000","last_active":"2024-04-17T10:41:38.458896+0000","last_peered":"2024-04-17T10:41:38.458896+0000","last_clean":"2024-04-17T10:41:38.458896+0000","last_became_active":"2024-04-17T10:41:38.449852+0000","last_became_peered":"2024-04-17T10:41:38.449852+0000","last_unstale":"2024-04-17T10:41:38.458896+0000","last_undegraded":"2024-04-17T10:41:38.458896+0000","last_fullsized":"2024-04-17T10:41:38.458896+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:09:12.709635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.455816+0000","last_change":"2024-04-17T10:41:38.450542+0000","last_active":"2024-04-17T10:41:38.455816+0000","last_peered":"2024-04-17T10:41:38.455816+0000","last_clean":"2024-04-17T10:41:38.455816+0000","last_became_active":"2024-04-17T10:41:38.450095+0000","last_became_peered":"2024-04-17T10:41:38.450095+0000","last_unstale":"2024-04-17T10:41:38.455816+0000","last_undegraded":"2024-04-17T10:41:38.455816+0000","last_fullsized":"2024-04-17T10:41:38.455816+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:59:43.511180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.474827+0000","last_change":"2024-04-17T10:41:38.469475+0000","last_active":"2024-04-17T10:41:38.474827+0000","last_peered":"2024-04-17T10:41:38.474827+0000","last_clean":"2024-04-17T10:41:38.474827+0000","last_became_active":"2024-04-17T10:41:38.469189+0000","last_became_peered":"2024-04-17T10:41:38.469189+0000","last_unstale":"2024-04-17T10:41:38.474827+0000","last_undegraded":"2024-04-17T10:41:38.474827+0000","last_fullsized":"2024-04-17T10:41:38.474827+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:51:22.736113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.468819+0000","last_change":"2024-04-17T10:41:38.468044+0000","last_active":"2024-04-17T10:41:38.468819+0000","last_peered":"2024-04-17T10:41:38.468819+0000","last_clean":"2024-04-17T10:41:38.468819+0000","last_became_active":"2024-04-17T10:41:38.467727+0000","last_became_peered":"2024-04-17T10:41:38.467727+0000","last_unstale":"2024-04-17T10:41:38.468819+0000","last_undegraded":"2024-04-17T10:41:38.468819+0000","last_fullsized":"2024-04-17T10:41:38.468819+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:40:14.726947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.479187+0000","last_change":"2024-04-17T10:41:38.475110+0000","last_active":"2024-04-17T10:41:38.479187+0000","last_peered":"2024-04-17T10:41:38.479187+0000","last_clean":"2024-04-17T10:41:38.479187+0000","last_became_active":"2024-04-17T10:41:38.474029+0000","last_became_peered":"2024-04-17T10:41:38.474029+0000","last_unstale":"2024-04-17T10:41:38.479187+0000","last_undegraded":"2024-04-17T10:41:38.479187+0000","last_fullsized":"2024-04-17T10:41:38.479187+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:56.618475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.466053+0000","last_change":"2024-04-17T10:41:38.463242+0000","last_active":"2024-04-17T10:41:38.466053+0000","last_peered":"2024-04-17T10:41:38.466053+0000","last_clean":"2024-04-17T10:41:38.466053+0000","last_became_active":"2024-04-17T10:41:38.462351+0000","last_became_peered":"2024-04-17T10:41:38.462351+0000","last_unstale":"2024-04-17T10:41:38.466053+0000","last_undegraded":"2024-04-17T10:41:38.466053+0000","last_fullsized":"2024-04-17T10:41:38.466053+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:26:30.980989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.485559+0000","last_change":"2024-04-17T10:41:38.481775+0000","last_active":"2024-04-17T10:41:38.485559+0000","last_peered":"2024-04-17T10:41:38.485559+0000","last_clean":"2024-04-17T10:41:38.485559+0000","last_became_active":"2024-04-17T10:41:38.480895+0000","last_became_peered":"2024-04-17T10:41:38.480895+0000","last_unstale":"2024-04-17T10:41:38.485559+0000","last_undegraded":"2024-04-17T10:41:38.485559+0000","last_fullsized":"2024-04-17T10:41:38.485559+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:03:58.711500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.473309+0000","last_change":"2024-04-17T10:41:38.470081+0000","last_active":"2024-04-17T10:41:38.473309+0000","last_peered":"2024-04-17T10:41:38.473309+0000","last_clean":"2024-04-17T10:41:38.473309+0000","last_became_active":"2024-04-17T10:41:38.468955+0000","last_became_peered":"2024-04-17T10:41:38.468955+0000","last_unstale":"2024-04-17T10:41:38.473309+0000","last_undegraded":"2024-04-17T10:41:38.473309+0000","last_fullsized":"2024-04-17T10:41:38.473309+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:54:01.366180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.464980+0000","last_change":"2024-04-17T10:41:38.462523+0000","last_active":"2024-04-17T10:41:38.464980+0000","last_peered":"2024-04-17T10:41:38.464980+0000","last_clean":"2024-04-17T10:41:38.464980+0000","last_became_active":"2024-04-17T10:41:38.462274+0000","last_became_peered":"2024-04-17T10:41:38.462274+0000","last_unstale":"2024-04-17T10:41:38.464980+0000","last_undegraded":"2024-04-17T10:41:38.464980+0000","last_fullsized":"2024-04-17T10:41:38.464980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:16:02.896971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.460601+0000","last_change":"2024-04-17T10:41:38.456516+0000","last_active":"2024-04-17T10:41:38.460601+0000","last_peered":"2024-04-17T10:41:38.460601+0000","last_clean":"2024-04-17T10:41:38.460601+0000","last_became_active":"2024-04-17T10:41:38.456180+0000","last_became_peered":"2024-04-17T10:41:38.456180+0000","last_unstale":"2024-04-17T10:41:38.460601+0000","last_undegraded":"2024-04-17T10:41:38.460601+0000","last_fullsized":"2024-04-17T10:41:38.460601+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:09:29.824195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.475248+0000","last_change":"2024-04-17T10:41:38.470503+0000","last_active":"2024-04-17T10:41:38.475248+0000","last_peered":"2024-04-17T10:41:38.475248+0000","last_clean":"2024-04-17T10:41:38.475248+0000","last_became_active":"2024-04-17T10:41:38.469746+0000","last_became_peered":"2024-04-17T10:41:38.469746+0000","last_unstale":"2024-04-17T10:41:38.475248+0000","last_undegraded":"2024-04-17T10:41:38.475248+0000","last_fullsized":"2024-04-17T10:41:38.475248+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:11:57.808488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.455722+0000","last_change":"2024-04-17T10:41:38.451093+0000","last_active":"2024-04-17T10:41:38.455722+0000","last_peered":"2024-04-17T10:41:38.455722+0000","last_clean":"2024-04-17T10:41:38.455722+0000","last_became_active":"2024-04-17T10:41:38.448832+0000","last_became_peered":"2024-04-17T10:41:38.448832+0000","last_unstale":"2024-04-17T10:41:38.455722+0000","last_undegraded":"2024-04-17T10:41:38.455722+0000","last_fullsized":"2024-04-17T10:41:38.455722+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:28:38.418159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.451405+0000","last_change":"2024-04-17T10:41:38.446790+0000","last_active":"2024-04-17T10:41:38.451405+0000","last_peered":"2024-04-17T10:41:38.451405+0000","last_clean":"2024-04-17T10:41:38.451405+0000","last_became_active":"2024-04-17T10:41:38.446069+0000","last_became_peered":"2024-04-17T10:41:38.446069+0000","last_unstale":"2024-04-17T10:41:38.451405+0000","last_undegraded":"2024-04-17T10:41:38.451405+0000","last_fullsized":"2024-04-17T10:41:38.451405+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:56:44.993567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.478650+0000","last_change":"2024-04-17T10:41:38.472886+0000","last_active":"2024-04-17T10:41:38.478650+0000","last_peered":"2024-04-17T10:41:38.478650+0000","last_clean":"2024-04-17T10:41:38.478650+0000","last_became_active":"2024-04-17T10:41:38.471403+0000","last_became_peered":"2024-04-17T10:41:38.471403+0000","last_unstale":"2024-04-17T10:41:38.478650+0000","last_undegraded":"2024-04-17T10:41:38.478650+0000","last_fullsized":"2024-04-17T10:41:38.478650+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:17:35.241441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.464068+0000","last_change":"2024-04-17T10:41:38.457501+0000","last_active":"2024-04-17T10:41:38.464068+0000","last_peered":"2024-04-17T10:41:38.464068+0000","last_clean":"2024-04-17T10:41:38.464068+0000","last_became_active":"2024-04-17T10:41:38.457219+0000","last_became_peered":"2024-04-17T10:41:38.457219+0000","last_unstale":"2024-04-17T10:41:38.464068+0000","last_undegraded":"2024-04-17T10:41:38.464068+0000","last_fullsized":"2024-04-17T10:41:38.464068+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:28:38.418159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.465529+0000","last_change":"2024-04-17T10:41:38.461324+0000","last_active":"2024-04-17T10:41:38.465529+0000","last_peered":"2024-04-17T10:41:38.465529+0000","last_clean":"2024-04-17T10:41:38.465529+0000","last_became_active":"2024-04-17T10:41:38.460487+0000","last_became_peered":"2024-04-17T10:41:38.460487+0000","last_unstale":"2024-04-17T10:41:38.465529+0000","last_undegraded":"2024-04-17T10:41:38.465529+0000","last_fullsized":"2024-04-17T10:41:38.465529+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:03:03.269784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.454893+0000","last_change":"2024-04-17T10:41:38.450615+0000","last_active":"2024-04-17T10:41:38.454893+0000","last_peered":"2024-04-17T10:41:38.454893+0000","last_clean":"2024-04-17T10:41:38.454893+0000","last_became_active":"2024-04-17T10:41:38.448925+0000","last_became_peered":"2024-04-17T10:41:38.448925+0000","last_unstale":"2024-04-17T10:41:38.454893+0000","last_undegraded":"2024-04-17T10:41:38.454893+0000","last_fullsized":"2024-04-17T10:41:38.454893+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:31:26.144792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.473022+0000","last_change":"2024-04-17T10:41:38.470054+0000","last_active":"2024-04-17T10:41:38.473022+0000","last_peered":"2024-04-17T10:41:38.473022+0000","last_clean":"2024-04-17T10:41:38.473022+0000","last_became_active":"2024-04-17T10:41:38.469085+0000","last_became_peered":"2024-04-17T10:41:38.469085+0000","last_unstale":"2024-04-17T10:41:38.473022+0000","last_undegraded":"2024-04-17T10:41:38.473022+0000","last_fullsized":"2024-04-17T10:41:38.473022+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:51:59.724625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.463072+0000","last_change":"2024-04-17T10:41:38.454560+0000","last_active":"2024-04-17T10:41:38.463072+0000","last_peered":"2024-04-17T10:41:38.463072+0000","last_clean":"2024-04-17T10:41:38.463072+0000","last_became_active":"2024-04-17T10:41:38.453821+0000","last_became_peered":"2024-04-17T10:41:38.453821+0000","last_unstale":"2024-04-17T10:41:38.463072+0000","last_undegraded":"2024-04-17T10:41:38.463072+0000","last_fullsized":"2024-04-17T10:41:38.463072+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:12:30.359237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.456762+0000","last_change":"2024-04-17T10:41:38.452675+0000","last_active":"2024-04-17T10:41:38.456762+0000","last_peered":"2024-04-17T10:41:38.456762+0000","last_clean":"2024-04-17T10:41:38.456762+0000","last_became_active":"2024-04-17T10:41:38.447556+0000","last_became_peered":"2024-04-17T10:41:38.447556+0000","last_unstale":"2024-04-17T10:41:38.456762+0000","last_undegraded":"2024-04-17T10:41:38.456762+0000","last_fullsized":"2024-04-17T10:41:38.456762+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:40:09.456885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.464854+0000","last_change":"2024-04-17T10:41:38.461835+0000","last_active":"2024-04-17T10:41:38.464854+0000","last_peered":"2024-04-17T10:41:38.464854+0000","last_clean":"2024-04-17T10:41:38.464854+0000","last_became_active":"2024-04-17T10:41:38.458313+0000","last_became_peered":"2024-04-17T10:41:38.458313+0000","last_unstale":"2024-04-17T10:41:38.464854+0000","last_undegraded":"2024-04-17T10:41:38.464854+0000","last_fullsized":"2024-04-17T10:41:38.464854+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:07:53.020927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.465728+0000","last_change":"2024-04-17T10:41:38.458540+0000","last_active":"2024-04-17T10:41:38.465728+0000","last_peered":"2024-04-17T10:41:38.465728+0000","last_clean":"2024-04-17T10:41:38.465728+0000","last_became_active":"2024-04-17T10:41:38.457958+0000","last_became_peered":"2024-04-17T10:41:38.457958+0000","last_unstale":"2024-04-17T10:41:38.465728+0000","last_undegraded":"2024-04-17T10:41:38.465728+0000","last_fullsized":"2024-04-17T10:41:38.465728+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:30:16.883126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.471972+0000","last_change":"2024-04-17T10:41:37.459804+0000","last_active":"2024-04-17T10:41:38.471972+0000","last_peered":"2024-04-17T10:41:38.471972+0000","last_clean":"2024-04-17T10:41:38.471972+0000","last_became_active":"2024-04-17T10:41:37.459479+0000","last_became_peered":"2024-04-17T10:41:37.459479+0000","last_unstale":"2024-04-17T10:41:38.471972+0000","last_undegraded":"2024-04-17T10:41:38.471972+0000","last_fullsized":"2024-04-17T10:41:38.471972+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:51:46.650282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.478695+0000","last_change":"2024-04-17T10:41:38.473525+0000","last_active":"2024-04-17T10:41:38.478695+0000","last_peered":"2024-04-17T10:41:38.478695+0000","last_clean":"2024-04-17T10:41:38.478695+0000","last_became_active":"2024-04-17T10:41:38.472174+0000","last_became_peered":"2024-04-17T10:41:38.472174+0000","last_unstale":"2024-04-17T10:41:38.478695+0000","last_undegraded":"2024-04-17T10:41:38.478695+0000","last_fullsized":"2024-04-17T10:41:38.478695+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:12:39.460966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.485591+0000","last_change":"2024-04-17T10:41:37.460584+0000","last_active":"2024-04-17T10:41:38.485591+0000","last_peered":"2024-04-17T10:41:38.485591+0000","last_clean":"2024-04-17T10:41:38.485591+0000","last_became_active":"2024-04-17T10:41:37.459907+0000","last_became_peered":"2024-04-17T10:41:37.459907+0000","last_unstale":"2024-04-17T10:41:38.485591+0000","last_undegraded":"2024-04-17T10:41:38.485591+0000","last_fullsized":"2024-04-17T10:41:38.485591+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:03:39.629440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.482685+0000","last_change":"2024-04-17T10:41:38.477585+0000","last_active":"2024-04-17T10:41:38.482685+0000","last_peered":"2024-04-17T10:41:38.482685+0000","last_clean":"2024-04-17T10:41:38.482685+0000","last_became_active":"2024-04-17T10:41:38.459721+0000","last_became_peered":"2024-04-17T10:41:38.459721+0000","last_unstale":"2024-04-17T10:41:38.482685+0000","last_undegraded":"2024-04-17T10:41:38.482685+0000","last_fullsized":"2024-04-17T10:41:38.482685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:40:09.456885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.482090+0000","last_change":"2024-04-17T10:41:37.449488+0000","last_active":"2024-04-17T10:41:38.482090+0000","last_peered":"2024-04-17T10:41:38.482090+0000","last_clean":"2024-04-17T10:41:38.482090+0000","last_became_active":"2024-04-17T10:41:37.449108+0000","last_became_peered":"2024-04-17T10:41:37.449108+0000","last_unstale":"2024-04-17T10:41:38.482090+0000","last_undegraded":"2024-04-17T10:41:38.482090+0000","last_fullsized":"2024-04-17T10:41:38.482090+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:12:29.350979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.469644+0000","last_change":"2024-04-17T10:41:38.465350+0000","last_active":"2024-04-17T10:41:38.469644+0000","last_peered":"2024-04-17T10:41:38.469644+0000","last_clean":"2024-04-17T10:41:38.469644+0000","last_became_active":"2024-04-17T10:41:38.465085+0000","last_became_peered":"2024-04-17T10:41:38.465085+0000","last_unstale":"2024-04-17T10:41:38.469644+0000","last_undegraded":"2024-04-17T10:41:38.469644+0000","last_fullsized":"2024-04-17T10:41:38.469644+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:03:03.269784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.484362+0000","last_change":"2024-04-17T10:41:37.459728+0000","last_active":"2024-04-17T10:41:38.484362+0000","last_peered":"2024-04-17T10:41:38.484362+0000","last_clean":"2024-04-17T10:41:38.484362+0000","last_became_active":"2024-04-17T10:41:37.459283+0000","last_became_peered":"2024-04-17T10:41:37.459283+0000","last_unstale":"2024-04-17T10:41:38.484362+0000","last_undegraded":"2024-04-17T10:41:38.484362+0000","last_fullsized":"2024-04-17T10:41:38.484362+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:31:25.136534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.463009+0000","last_change":"2024-04-17T10:41:38.453473+0000","last_active":"2024-04-17T10:41:38.463009+0000","last_peered":"2024-04-17T10:41:38.463009+0000","last_clean":"2024-04-17T10:41:38.463009+0000","last_became_active":"2024-04-17T10:41:38.452911+0000","last_became_peered":"2024-04-17T10:41:38.452911+0000","last_unstale":"2024-04-17T10:41:38.463009+0000","last_undegraded":"2024-04-17T10:41:38.463009+0000","last_fullsized":"2024-04-17T10:41:38.463009+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:59:16.279043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.467706+0000","last_change":"2024-04-17T10:41:37.449711+0000","last_active":"2024-04-17T10:41:38.467706+0000","last_peered":"2024-04-17T10:41:38.467706+0000","last_clean":"2024-04-17T10:41:38.467706+0000","last_became_active":"2024-04-17T10:41:37.449391+0000","last_became_peered":"2024-04-17T10:41:37.449391+0000","last_unstale":"2024-04-17T10:41:38.467706+0000","last_undegraded":"2024-04-17T10:41:38.467706+0000","last_fullsized":"2024-04-17T10:41:38.467706+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:31:25.136534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.474296+0000","last_change":"2024-04-17T10:41:38.460279+0000","last_active":"2024-04-17T10:41:38.474296+0000","last_peered":"2024-04-17T10:41:38.474296+0000","last_clean":"2024-04-17T10:41:38.474296+0000","last_became_active":"2024-04-17T10:41:38.459941+0000","last_became_peered":"2024-04-17T10:41:38.459941+0000","last_unstale":"2024-04-17T10:41:38.474296+0000","last_undegraded":"2024-04-17T10:41:38.474296+0000","last_fullsized":"2024-04-17T10:41:38.474296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:56.618475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.483090+0000","last_change":"2024-04-17T10:41:37.456651+0000","last_active":"2024-04-17T10:41:38.483090+0000","last_peered":"2024-04-17T10:41:38.483090+0000","last_clean":"2024-04-17T10:41:38.483090+0000","last_became_active":"2024-04-17T10:41:37.455299+0000","last_became_peered":"2024-04-17T10:41:37.455299+0000","last_unstale":"2024-04-17T10:41:38.483090+0000","last_undegraded":"2024-04-17T10:41:38.483090+0000","last_fullsized":"2024-04-17T10:41:38.483090+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:51:21.727855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.475106+0000","last_change":"2024-04-17T10:41:38.471934+0000","last_active":"2024-04-17T10:41:38.475106+0000","last_peered":"2024-04-17T10:41:38.475106+0000","last_clean":"2024-04-17T10:41:38.475106+0000","last_became_active":"2024-04-17T10:41:38.470071+0000","last_became_peered":"2024-04-17T10:41:38.470071+0000","last_unstale":"2024-04-17T10:41:38.475106+0000","last_undegraded":"2024-04-17T10:41:38.475106+0000","last_fullsized":"2024-04-17T10:41:38.475106+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:03:03.269784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.429116+0000","last_change":"2024-04-17T10:41:37.450244+0000","last_active":"2024-04-17T10:41:38.429116+0000","last_peered":"2024-04-17T10:41:38.429116+0000","last_clean":"2024-04-17T10:41:38.429116+0000","last_became_active":"2024-04-17T10:41:37.449776+0000","last_became_peered":"2024-04-17T10:41:37.449776+0000","last_unstale":"2024-04-17T10:41:38.429116+0000","last_undegraded":"2024-04-17T10:41:38.429116+0000","last_fullsized":"2024-04-17T10:41:38.429116+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:17:21.157969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.455778+0000","last_change":"2024-04-17T10:41:38.451335+0000","last_active":"2024-04-17T10:41:38.455778+0000","last_peered":"2024-04-17T10:41:38.455778+0000","last_clean":"2024-04-17T10:41:38.455778+0000","last_became_active":"2024-04-17T10:41:38.449167+0000","last_became_peered":"2024-04-17T10:41:38.449167+0000","last_unstale":"2024-04-17T10:41:38.455778+0000","last_undegraded":"2024-04-17T10:41:38.455778+0000","last_fullsized":"2024-04-17T10:41:38.455778+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:07:53.020927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.444252+0000","last_change":"2024-04-17T10:41:37.438228+0000","last_active":"2024-04-17T10:41:38.444252+0000","last_peered":"2024-04-17T10:41:38.444252+0000","last_clean":"2024-04-17T10:41:38.444252+0000","last_became_active":"2024-04-17T10:41:37.437730+0000","last_became_peered":"2024-04-17T10:41:37.437730+0000","last_unstale":"2024-04-17T10:41:38.444252+0000","last_undegraded":"2024-04-17T10:41:38.444252+0000","last_fullsized":"2024-04-17T10:41:38.444252+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:54:00.357922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.485520+0000","last_change":"2024-04-17T10:41:38.481752+0000","last_active":"2024-04-17T10:41:38.485520+0000","last_peered":"2024-04-17T10:41:38.485520+0000","last_clean":"2024-04-17T10:41:38.485520+0000","last_became_active":"2024-04-17T10:41:38.480519+0000","last_became_peered":"2024-04-17T10:41:38.480519+0000","last_unstale":"2024-04-17T10:41:38.485520+0000","last_undegraded":"2024-04-17T10:41:38.485520+0000","last_fullsized":"2024-04-17T10:41:38.485520+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:51:41.472422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.441403+0000","last_change":"2024-04-17T10:41:37.453973+0000","last_active":"2024-04-17T10:41:38.441403+0000","last_peered":"2024-04-17T10:41:38.441403+0000","last_clean":"2024-04-17T10:41:38.441403+0000","last_became_active":"2024-04-17T10:41:37.453164+0000","last_became_peered":"2024-04-17T10:41:37.453164+0000","last_unstale":"2024-04-17T10:41:38.441403+0000","last_undegraded":"2024-04-17T10:41:38.441403+0000","last_fullsized":"2024-04-17T10:41:38.441403+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:07:16.080687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.447158+0000","last_change":"2024-04-17T10:41:38.444213+0000","last_active":"2024-04-17T10:41:38.447158+0000","last_peered":"2024-04-17T10:41:38.447158+0000","last_clean":"2024-04-17T10:41:38.447158+0000","last_became_active":"2024-04-17T10:41:38.443464+0000","last_became_peered":"2024-04-17T10:41:38.443464+0000","last_unstale":"2024-04-17T10:41:38.447158+0000","last_undegraded":"2024-04-17T10:41:38.447158+0000","last_fullsized":"2024-04-17T10:41:38.447158+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:38:10.610378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.446741+0000","last_change":"2024-04-17T10:41:37.450709+0000","last_active":"2024-04-17T10:41:38.446741+0000","last_peered":"2024-04-17T10:41:38.446741+0000","last_clean":"2024-04-17T10:41:38.446741+0000","last_became_active":"2024-04-17T10:41:37.450412+0000","last_became_peered":"2024-04-17T10:41:37.450412+0000","last_unstale":"2024-04-17T10:41:38.446741+0000","last_undegraded":"2024-04-17T10:41:38.446741+0000","last_fullsized":"2024-04-17T10:41:38.446741+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:17:34.233183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.459270+0000","last_change":"2024-04-17T10:41:38.456726+0000","last_active":"2024-04-17T10:41:38.459270+0000","last_peered":"2024-04-17T10:41:38.459270+0000","last_clean":"2024-04-17T10:41:38.459270+0000","last_became_active":"2024-04-17T10:41:38.456415+0000","last_became_peered":"2024-04-17T10:41:38.456415+0000","last_unstale":"2024-04-17T10:41:38.459270+0000","last_undegraded":"2024-04-17T10:41:38.459270+0000","last_fullsized":"2024-04-17T10:41:38.459270+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:22:35.852561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.485356+0000","last_change":"2024-04-17T10:41:37.462696+0000","last_active":"2024-04-17T10:41:38.485356+0000","last_peered":"2024-04-17T10:41:38.485356+0000","last_clean":"2024-04-17T10:41:38.485356+0000","last_became_active":"2024-04-17T10:41:37.462209+0000","last_became_peered":"2024-04-17T10:41:37.462209+0000","last_unstale":"2024-04-17T10:41:38.485356+0000","last_undegraded":"2024-04-17T10:41:38.485356+0000","last_fullsized":"2024-04-17T10:41:38.485356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:04:10.611824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.474700+0000","last_change":"2024-04-17T10:41:38.468072+0000","last_active":"2024-04-17T10:41:38.474700+0000","last_peered":"2024-04-17T10:41:38.474700+0000","last_clean":"2024-04-17T10:41:38.474700+0000","last_became_active":"2024-04-17T10:41:38.466877+0000","last_became_peered":"2024-04-17T10:41:38.466877+0000","last_unstale":"2024-04-17T10:41:38.474700+0000","last_undegraded":"2024-04-17T10:41:38.474700+0000","last_fullsized":"2024-04-17T10:41:38.474700+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:22:35.852561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":18,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.484589+0000","last_change":"2024-04-17T10:41:37.452385+0000","last_active":"2024-04-17T10:41:38.484589+0000","last_peered":"2024-04-17T10:41:38.484589+0000","last_clean":"2024-04-17T10:41:38.484589+0000","last_became_active":"2024-04-17T10:41:37.451936+0000","last_became_peered":"2024-04-17T10:41:37.451936+0000","last_unstale":"2024-04-17T10:41:38.484589+0000","last_undegraded":"2024-04-17T10:41:38.484589+0000","last_fullsized":"2024-04-17T10:41:38.484589+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:33:50.317259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.471224+0000","last_change":"2024-04-17T10:41:38.468117+0000","last_active":"2024-04-17T10:41:38.471224+0000","last_peered":"2024-04-17T10:41:38.471224+0000","last_clean":"2024-04-17T10:41:38.471224+0000","last_became_active":"2024-04-17T10:41:38.467465+0000","last_became_peered":"2024-04-17T10:41:38.467465+0000","last_unstale":"2024-04-17T10:41:38.471224+0000","last_undegraded":"2024-04-17T10:41:38.471224+0000","last_fullsized":"2024-04-17T10:41:38.471224+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:15:45.348582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.475108+0000","last_change":"2024-04-17T10:41:37.439006+0000","last_active":"2024-04-17T10:41:38.475108+0000","last_peered":"2024-04-17T10:41:38.475108+0000","last_clean":"2024-04-17T10:41:38.475108+0000","last_became_active":"2024-04-17T10:41:37.438687+0000","last_became_peered":"2024-04-17T10:41:37.438687+0000","last_unstale":"2024-04-17T10:41:38.475108+0000","last_undegraded":"2024-04-17T10:41:38.475108+0000","last_fullsized":"2024-04-17T10:41:38.475108+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:41:47.456521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.474792+0000","last_change":"2024-04-17T10:41:38.469628+0000","last_active":"2024-04-17T10:41:38.474792+0000","last_peered":"2024-04-17T10:41:38.474792+0000","last_clean":"2024-04-17T10:41:38.474792+0000","last_became_active":"2024-04-17T10:41:38.469322+0000","last_became_peered":"2024-04-17T10:41:38.469322+0000","last_unstale":"2024-04-17T10:41:38.474792+0000","last_undegraded":"2024-04-17T10:41:38.474792+0000","last_fullsized":"2024-04-17T10:41:38.474792+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:59:16.279043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.432098+0000","last_change":"2024-04-17T10:41:37.440526+0000","last_active":"2024-04-17T10:41:38.432098+0000","last_peered":"2024-04-17T10:41:38.432098+0000","last_clean":"2024-04-17T10:41:38.432098+0000","last_became_active":"2024-04-17T10:41:37.439773+0000","last_became_peered":"2024-04-17T10:41:37.439773+0000","last_unstale":"2024-04-17T10:41:38.432098+0000","last_undegraded":"2024-04-17T10:41:38.432098+0000","last_fullsized":"2024-04-17T10:41:38.432098+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:51:46.650282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.468435+0000","last_change":"2024-04-17T10:41:38.466712+0000","last_active":"2024-04-17T10:41:38.468435+0000","last_peered":"2024-04-17T10:41:38.468435+0000","last_clean":"2024-04-17T10:41:38.468435+0000","last_became_active":"2024-04-17T10:41:38.464593+0000","last_became_peered":"2024-04-17T10:41:38.464593+0000","last_unstale":"2024-04-17T10:41:38.468435+0000","last_undegraded":"2024-04-17T10:41:38.468435+0000","last_fullsized":"2024-04-17T10:41:38.468435+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:51:23.707316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.452435+0000","last_change":"2024-04-17T10:41:37.438244+0000","last_active":"2024-04-17T10:41:38.452435+0000","last_peered":"2024-04-17T10:41:38.452435+0000","last_clean":"2024-04-17T10:41:38.452435+0000","last_became_active":"2024-04-17T10:41:37.437914+0000","last_became_peered":"2024-04-17T10:41:37.437914+0000","last_unstale":"2024-04-17T10:41:38.452435+0000","last_undegraded":"2024-04-17T10:41:38.452435+0000","last_fullsized":"2024-04-17T10:41:38.452435+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:09:11.701377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.450239+0000","last_change":"2024-04-17T10:41:38.446965+0000","last_active":"2024-04-17T10:41:38.450239+0000","last_peered":"2024-04-17T10:41:38.450239+0000","last_clean":"2024-04-17T10:41:38.450239+0000","last_became_active":"2024-04-17T10:41:38.446093+0000","last_became_peered":"2024-04-17T10:41:38.446093+0000","last_unstale":"2024-04-17T10:41:38.450239+0000","last_undegraded":"2024-04-17T10:41:38.450239+0000","last_fullsized":"2024-04-17T10:41:38.450239+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:40:09.456885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.484303+0000","last_change":"2024-04-17T10:41:37.449790+0000","last_active":"2024-04-17T10:41:38.484303+0000","last_peered":"2024-04-17T10:41:38.484303+0000","last_clean":"2024-04-17T10:41:38.484303+0000","last_became_active":"2024-04-17T10:41:37.449406+0000","last_became_peered":"2024-04-17T10:41:37.449406+0000","last_unstale":"2024-04-17T10:41:38.484303+0000","last_undegraded":"2024-04-17T10:41:38.484303+0000","last_fullsized":"2024-04-17T10:41:38.484303+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:51:58.716367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.455856+0000","last_change":"2024-04-17T10:41:38.451185+0000","last_active":"2024-04-17T10:41:38.455856+0000","last_peered":"2024-04-17T10:41:38.455856+0000","last_clean":"2024-04-17T10:41:38.455856+0000","last_became_active":"2024-04-17T10:41:38.450475+0000","last_became_peered":"2024-04-17T10:41:38.450475+0000","last_unstale":"2024-04-17T10:41:38.455856+0000","last_undegraded":"2024-04-17T10:41:38.455856+0000","last_fullsized":"2024-04-17T10:41:38.455856+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:38:42.837743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.465015+0000","last_change":"2024-04-17T10:41:37.437417+0000","last_active":"2024-04-17T10:41:38.465015+0000","last_peered":"2024-04-17T10:41:38.465015+0000","last_clean":"2024-04-17T10:41:38.465015+0000","last_became_active":"2024-04-17T10:41:37.437021+0000","last_became_peered":"2024-04-17T10:41:37.437021+0000","last_unstale":"2024-04-17T10:41:38.465015+0000","last_undegraded":"2024-04-17T10:41:38.465015+0000","last_fullsized":"2024-04-17T10:41:38.465015+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:39:52.262198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.474356+0000","last_change":"2024-04-17T10:41:38.459897+0000","last_active":"2024-04-17T10:41:38.474356+0000","last_peered":"2024-04-17T10:41:38.474356+0000","last_clean":"2024-04-17T10:41:38.474356+0000","last_became_active":"2024-04-17T10:41:38.459676+0000","last_became_peered":"2024-04-17T10:41:38.459676+0000","last_unstale":"2024-04-17T10:41:38.474356+0000","last_undegraded":"2024-04-17T10:41:38.474356+0000","last_fullsized":"2024-04-17T10:41:38.474356+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:12:39.460966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.484580+0000","last_change":"2024-04-17T10:41:37.458195+0000","last_active":"2024-04-17T10:41:38.484580+0000","last_peered":"2024-04-17T10:41:38.484580+0000","last_clean":"2024-04-17T10:41:38.484580+0000","last_became_active":"2024-04-17T10:41:37.457225+0000","last_became_peered":"2024-04-17T10:41:37.457225+0000","last_unstale":"2024-04-17T10:41:38.484580+0000","last_undegraded":"2024-04-17T10:41:38.484580+0000","last_fullsized":"2024-04-17T10:41:38.484580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:11:56.800230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.467138+0000","last_change":"2024-04-17T10:41:38.461701+0000","last_active":"2024-04-17T10:41:38.467138+0000","last_peered":"2024-04-17T10:41:38.467138+0000","last_clean":"2024-04-17T10:41:38.467138+0000","last_became_active":"2024-04-17T10:41:38.461020+0000","last_became_peered":"2024-04-17T10:41:38.461020+0000","last_unstale":"2024-04-17T10:41:38.467138+0000","last_undegraded":"2024-04-17T10:41:38.467138+0000","last_fullsized":"2024-04-17T10:41:38.467138+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:16:02.896971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.482465+0000","last_change":"2024-04-17T10:41:37.451444+0000","last_active":"2024-04-17T10:41:38.482465+0000","last_peered":"2024-04-17T10:41:38.482465+0000","last_clean":"2024-04-17T10:41:38.482465+0000","last_became_active":"2024-04-17T10:41:37.450652+0000","last_became_peered":"2024-04-17T10:41:37.450652+0000","last_unstale":"2024-04-17T10:41:38.482465+0000","last_undegraded":"2024-04-17T10:41:38.482465+0000","last_fullsized":"2024-04-17T10:41:38.482465+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:07:15.434437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.475927+0000","last_change":"2024-04-17T10:41:38.472185+0000","last_active":"2024-04-17T10:41:38.475927+0000","last_peered":"2024-04-17T10:41:38.475927+0000","last_clean":"2024-04-17T10:41:38.475927+0000","last_became_active":"2024-04-17T10:41:38.471615+0000","last_became_peered":"2024-04-17T10:41:38.471615+0000","last_unstale":"2024-04-17T10:41:38.475927+0000","last_undegraded":"2024-04-17T10:41:38.475927+0000","last_fullsized":"2024-04-17T10:41:38.475927+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:10:35.305381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.451779+0000","last_change":"2024-04-17T10:41:37.455337+0000","last_active":"2024-04-17T10:41:38.451779+0000","last_peered":"2024-04-17T10:41:38.451779+0000","last_clean":"2024-04-17T10:41:38.451779+0000","last_became_active":"2024-04-17T10:41:37.454990+0000","last_became_peered":"2024-04-17T10:41:37.454990+0000","last_unstale":"2024-04-17T10:41:38.451779+0000","last_undegraded":"2024-04-17T10:41:38.451779+0000","last_fullsized":"2024-04-17T10:41:38.451779+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:57:17.192590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.459647+0000","last_change":"2024-04-17T10:41:38.456662+0000","last_active":"2024-04-17T10:41:38.459647+0000","last_peered":"2024-04-17T10:41:38.459647+0000","last_clean":"2024-04-17T10:41:38.459647+0000","last_became_active":"2024-04-17T10:41:38.456343+0000","last_became_peered":"2024-04-17T10:41:38.456343+0000","last_unstale":"2024-04-17T10:41:38.459647+0000","last_undegraded":"2024-04-17T10:41:38.459647+0000","last_fullsized":"2024-04-17T10:41:38.459647+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:25:44.023009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.467819+0000","last_change":"2024-04-17T10:41:37.447234+0000","last_active":"2024-04-17T10:41:38.467819+0000","last_peered":"2024-04-17T10:41:38.467819+0000","last_clean":"2024-04-17T10:41:38.467819+0000","last_became_active":"2024-04-17T10:41:37.446724+0000","last_became_peered":"2024-04-17T10:41:37.446724+0000","last_unstale":"2024-04-17T10:41:38.467819+0000","last_undegraded":"2024-04-17T10:41:38.467819+0000","last_fullsized":"2024-04-17T10:41:38.467819+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:38:09.602120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.458101+0000","last_change":"2024-04-17T10:41:38.454552+0000","last_active":"2024-04-17T10:41:38.458101+0000","last_peered":"2024-04-17T10:41:38.458101+0000","last_clean":"2024-04-17T10:41:38.458101+0000","last_became_active":"2024-04-17T10:41:38.454225+0000","last_became_peered":"2024-04-17T10:41:38.454225+0000","last_unstale":"2024-04-17T10:41:38.458101+0000","last_undegraded":"2024-04-17T10:41:38.458101+0000","last_fullsized":"2024-04-17T10:41:38.458101+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:36:33.039968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.430218+0000","last_change":"2024-04-17T10:41:37.460106+0000","last_active":"2024-04-17T10:41:38.430218+0000","last_peered":"2024-04-17T10:41:38.430218+0000","last_clean":"2024-04-17T10:41:38.430218+0000","last_became_active":"2024-04-17T10:41:37.459406+0000","last_became_peered":"2024-04-17T10:41:37.459406+0000","last_unstale":"2024-04-17T10:41:38.430218+0000","last_undegraded":"2024-04-17T10:41:38.430218+0000","last_fullsized":"2024-04-17T10:41:38.430218+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:51:46.650282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.464110+0000","last_change":"2024-04-17T10:41:38.456587+0000","last_active":"2024-04-17T10:41:38.464110+0000","last_peered":"2024-04-17T10:41:38.464110+0000","last_clean":"2024-04-17T10:41:38.464110+0000","last_became_active":"2024-04-17T10:41:38.437409+0000","last_became_peered":"2024-04-17T10:41:38.437409+0000","last_unstale":"2024-04-17T10:41:38.464110+0000","last_undegraded":"2024-04-17T10:41:38.464110+0000","last_fullsized":"2024-04-17T10:41:38.464110+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:56:44.993567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.441307+0000","last_change":"2024-04-17T10:41:37.450703+0000","last_active":"2024-04-17T10:41:38.441307+0000","last_peered":"2024-04-17T10:41:38.441307+0000","last_clean":"2024-04-17T10:41:38.441307+0000","last_became_active":"2024-04-17T10:41:37.450429+0000","last_became_peered":"2024-04-17T10:41:37.450429+0000","last_unstale":"2024-04-17T10:41:38.441307+0000","last_undegraded":"2024-04-17T10:41:38.441307+0000","last_fullsized":"2024-04-17T10:41:38.441307+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:43:32.663340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.474969+0000","last_change":"2024-04-17T10:41:38.471446+0000","last_active":"2024-04-17T10:41:38.474969+0000","last_peered":"2024-04-17T10:41:38.474969+0000","last_clean":"2024-04-17T10:41:38.474969+0000","last_became_active":"2024-04-17T10:41:38.470111+0000","last_became_peered":"2024-04-17T10:41:38.470111+0000","last_unstale":"2024-04-17T10:41:38.474969+0000","last_undegraded":"2024-04-17T10:41:38.474969+0000","last_fullsized":"2024-04-17T10:41:38.474969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:12:30.359237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.428972+0000","last_change":"2024-04-17T10:41:37.439030+0000","last_active":"2024-04-17T10:41:38.428972+0000","last_peered":"2024-04-17T10:41:38.428972+0000","last_clean":"2024-04-17T10:41:38.428972+0000","last_became_active":"2024-04-17T10:41:37.438367+0000","last_became_peered":"2024-04-17T10:41:37.438367+0000","last_unstale":"2024-04-17T10:41:38.428972+0000","last_undegraded":"2024-04-17T10:41:38.428972+0000","last_fullsized":"2024-04-17T10:41:38.428972+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:07:15.434437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.473244+0000","last_change":"2024-04-17T10:41:38.470029+0000","last_active":"2024-04-17T10:41:38.473244+0000","last_peered":"2024-04-17T10:41:38.473244+0000","last_clean":"2024-04-17T10:41:38.473244+0000","last_became_active":"2024-04-17T10:41:38.469123+0000","last_became_peered":"2024-04-17T10:41:38.469123+0000","last_unstale":"2024-04-17T10:41:38.473244+0000","last_undegraded":"2024-04-17T10:41:38.473244+0000","last_fullsized":"2024-04-17T10:41:38.473244+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:43:33.671598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.452078+0000","last_change":"2024-04-17T10:41:37.461421+0000","last_active":"2024-04-17T10:41:38.452078+0000","last_peered":"2024-04-17T10:41:38.452078+0000","last_clean":"2024-04-17T10:41:38.452078+0000","last_became_active":"2024-04-17T10:41:37.461139+0000","last_became_peered":"2024-04-17T10:41:37.461139+0000","last_unstale":"2024-04-17T10:41:38.452078+0000","last_undegraded":"2024-04-17T10:41:38.452078+0000","last_fullsized":"2024-04-17T10:41:38.452078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:07:15.434437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.474633+0000","last_change":"2024-04-17T10:41:38.466590+0000","last_active":"2024-04-17T10:41:38.474633+0000","last_peered":"2024-04-17T10:41:38.474633+0000","last_clean":"2024-04-17T10:41:38.474633+0000","last_became_active":"2024-04-17T10:41:38.466214+0000","last_became_peered":"2024-04-17T10:41:38.466214+0000","last_unstale":"2024-04-17T10:41:38.474633+0000","last_undegraded":"2024-04-17T10:41:38.474633+0000","last_fullsized":"2024-04-17T10:41:38.474633+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:27:11.970761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.434669+0000","last_change":"2024-04-17T10:41:37.463415+0000","last_active":"2024-04-17T10:41:38.434669+0000","last_peered":"2024-04-17T10:41:38.434669+0000","last_clean":"2024-04-17T10:41:38.434669+0000","last_became_active":"2024-04-17T10:41:37.462549+0000","last_became_peered":"2024-04-17T10:41:37.462549+0000","last_unstale":"2024-04-17T10:41:38.434669+0000","last_undegraded":"2024-04-17T10:41:38.434669+0000","last_fullsized":"2024-04-17T10:41:38.434669+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:41:47.456521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.456034+0000","last_change":"2024-04-17T10:41:38.451923+0000","last_active":"2024-04-17T10:41:38.456034+0000","last_peered":"2024-04-17T10:41:38.456034+0000","last_clean":"2024-04-17T10:41:38.456034+0000","last_became_active":"2024-04-17T10:41:38.447371+0000","last_became_peered":"2024-04-17T10:41:38.447371+0000","last_unstale":"2024-04-17T10:41:38.456034+0000","last_undegraded":"2024-04-17T10:41:38.456034+0000","last_fullsized":"2024-04-17T10:41:38.456034+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:12:50.521087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.437749+0000","last_change":"2024-04-17T10:41:37.457523+0000","last_active":"2024-04-17T10:41:38.437749+0000","last_peered":"2024-04-17T10:41:38.437749+0000","last_clean":"2024-04-17T10:41:38.437749+0000","last_became_active":"2024-04-17T10:41:37.457177+0000","last_became_peered":"2024-04-17T10:41:37.457177+0000","last_unstale":"2024-04-17T10:41:38.437749+0000","last_undegraded":"2024-04-17T10:41:38.437749+0000","last_fullsized":"2024-04-17T10:41:38.437749+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:07:16.080687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.467578+0000","last_change":"2024-04-17T10:41:38.464329+0000","last_active":"2024-04-17T10:41:38.467578+0000","last_peered":"2024-04-17T10:41:38.467578+0000","last_clean":"2024-04-17T10:41:38.467578+0000","last_became_active":"2024-04-17T10:41:38.464010+0000","last_became_peered":"2024-04-17T10:41:38.464010+0000","last_unstale":"2024-04-17T10:41:38.467578+0000","last_undegraded":"2024-04-17T10:41:38.467578+0000","last_fullsized":"2024-04-17T10:41:38.467578+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:24:09.759918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.433997+0000","last_change":"2024-04-17T10:41:37.460609+0000","last_active":"2024-04-17T10:41:38.433997+0000","last_peered":"2024-04-17T10:41:38.433997+0000","last_clean":"2024-04-17T10:41:38.433997+0000","last_became_active":"2024-04-17T10:41:37.459973+0000","last_became_peered":"2024-04-17T10:41:37.459973+0000","last_unstale":"2024-04-17T10:41:38.433997+0000","last_undegraded":"2024-04-17T10:41:38.433997+0000","last_fullsized":"2024-04-17T10:41:38.433997+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:09:11.701377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.460580+0000","last_change":"2024-04-17T10:41:38.455618+0000","last_active":"2024-04-17T10:41:38.460580+0000","last_peered":"2024-04-17T10:41:38.460580+0000","last_clean":"2024-04-17T10:41:38.460580+0000","last_became_active":"2024-04-17T10:41:38.452995+0000","last_became_peered":"2024-04-17T10:41:38.452995+0000","last_unstale":"2024-04-17T10:41:38.460580+0000","last_undegraded":"2024-04-17T10:41:38.460580+0000","last_fullsized":"2024-04-17T10:41:38.460580+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:12:50.521087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.439438+0000","last_change":"2024-04-17T10:41:37.456004+0000","last_active":"2024-04-17T10:41:38.439438+0000","last_peered":"2024-04-17T10:41:38.439438+0000","last_clean":"2024-04-17T10:41:38.439438+0000","last_became_active":"2024-04-17T10:41:37.454546+0000","last_became_peered":"2024-04-17T10:41:37.454546+0000","last_unstale":"2024-04-17T10:41:38.439438+0000","last_undegraded":"2024-04-17T10:41:38.439438+0000","last_fullsized":"2024-04-17T10:41:38.439438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:56:16.332083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.479520+0000","last_change":"2024-04-17T10:41:38.461401+0000","last_active":"2024-04-17T10:41:38.479520+0000","last_peered":"2024-04-17T10:41:38.479520+0000","last_clean":"2024-04-17T10:41:38.479520+0000","last_became_active":"2024-04-17T10:41:38.460711+0000","last_became_peered":"2024-04-17T10:41:38.460711+0000","last_unstale":"2024-04-17T10:41:38.479520+0000","last_undegraded":"2024-04-17T10:41:38.479520+0000","last_fullsized":"2024-04-17T10:41:38.479520+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:04:11.620082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.438457+0000","last_change":"2024-04-17T10:41:37.452805+0000","last_active":"2024-04-17T10:41:38.438457+0000","last_peered":"2024-04-17T10:41:38.438457+0000","last_clean":"2024-04-17T10:41:38.438457+0000","last_became_active":"2024-04-17T10:41:37.452549+0000","last_became_peered":"2024-04-17T10:41:37.452549+0000","last_unstale":"2024-04-17T10:41:38.438457+0000","last_undegraded":"2024-04-17T10:41:38.438457+0000","last_fullsized":"2024-04-17T10:41:38.438457+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:59:15.270785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.467250+0000","last_change":"2024-04-17T10:41:38.456229+0000","last_active":"2024-04-17T10:41:38.467250+0000","last_peered":"2024-04-17T10:41:38.467250+0000","last_clean":"2024-04-17T10:41:38.467250+0000","last_became_active":"2024-04-17T10:41:38.455482+0000","last_became_peered":"2024-04-17T10:41:38.455482+0000","last_unstale":"2024-04-17T10:41:38.467250+0000","last_undegraded":"2024-04-17T10:41:38.467250+0000","last_fullsized":"2024-04-17T10:41:38.467250+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:36:33.039968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.445078+0000","last_change":"2024-04-17T10:41:37.440123+0000","last_active":"2024-04-17T10:41:38.445078+0000","last_peered":"2024-04-17T10:41:38.445078+0000","last_clean":"2024-04-17T10:41:38.445078+0000","last_became_active":"2024-04-17T10:41:37.439823+0000","last_became_peered":"2024-04-17T10:41:37.439823+0000","last_unstale":"2024-04-17T10:41:38.445078+0000","last_undegraded":"2024-04-17T10:41:38.445078+0000","last_fullsized":"2024-04-17T10:41:38.445078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:10:34.297124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.474841+0000","last_change":"2024-04-17T10:41:38.469544+0000","last_active":"2024-04-17T10:41:38.474841+0000","last_peered":"2024-04-17T10:41:38.474841+0000","last_clean":"2024-04-17T10:41:38.474841+0000","last_became_active":"2024-04-17T10:41:38.469214+0000","last_became_peered":"2024-04-17T10:41:38.469214+0000","last_unstale":"2024-04-17T10:41:38.474841+0000","last_undegraded":"2024-04-17T10:41:38.474841+0000","last_fullsized":"2024-04-17T10:41:38.474841+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:56:17.340340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.479769+0000","last_change":"2024-04-17T10:41:37.446332+0000","last_active":"2024-04-17T10:41:38.479769+0000","last_peered":"2024-04-17T10:41:38.479769+0000","last_clean":"2024-04-17T10:41:38.479769+0000","last_became_active":"2024-04-17T10:41:37.445650+0000","last_became_peered":"2024-04-17T10:41:37.445650+0000","last_unstale":"2024-04-17T10:41:38.479769+0000","last_undegraded":"2024-04-17T10:41:38.479769+0000","last_fullsized":"2024-04-17T10:41:38.479769+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:57:17.192590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":221,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.428746+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:41:38.428746+0000","last_peered":"2024-04-17T10:41:38.428746+0000","last_clean":"2024-04-17T10:41:38.428746+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:41:38.428746+0000","last_undegraded":"2024-04-17T10:41:38.428746+0000","last_fullsized":"2024-04-17T10:41:38.428746+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40604,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331236,"statfs":{"total":96636764160,"available":96595185664,"internally_reserved":0,"allocated":1126400,"data_stored":806307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40028,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331812,"statfs":{"total":96636764160,"available":96595775488,"internally_reserved":0,"allocated":536576,"data_stored":216521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40024,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331816,"statfs":{"total":96636764160,"available":96595779584,"internally_reserved":0,"allocated":532480,"data_stored":215919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45604,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326236,"statfs":{"total":96636764160,"available":96590065664,"internally_reserved":0,"allocated":540672,"data_stored":217461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49764,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94322076,"statfs":{"total":96636764160,"available":96585805824,"internally_reserved":0,"allocated":540672,"data_stored":216487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574895,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35872,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335968,"statfs":{"total":96636764160,"available":96600031232,"internally_reserved":0,"allocated":544768,"data_stored":217483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49748,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94322092,"statfs":{"total":96636764160,"available":96585822208,"internally_reserved":0,"allocated":532480,"data_stored":215919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40608,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331232,"statfs":{"total":96636764160,"available":96595181568,"internally_reserved":0,"allocated":1126400,"data_stored":806251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:41:41.459 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:41:41.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.619+0000 7fedc611d640 1 Processor -- start 2024-04-17T10:41:41.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.619+0000 7fedc611d640 1 -- start start 2024-04-17T10:41:41.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.619+0000 7fedc611d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fedc015eeb0 0x7fedc006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:41.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.619+0000 7fedc611d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fedc006c300 0x7fedc00ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:41.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.619+0000 7fedc611d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc006e980 con 0x7fedc00cf780 2024-04-17T10:41:41.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.619+0000 7fedc611d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc00cdb90 con 0x7fedc015eeb0 2024-04-17T10:41:41.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.619+0000 7fedc611d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc006e0e0 con 0x7fedc006c300 2024-04-17T10:41:41.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.619+0000 7fedbf7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fedc015eeb0 0x7fedc006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:41.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.619+0000 7fedbeffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fedc00cf780 0x7fedc006bbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:36894/0 (socket says 172.21.15.83:36894) 2024-04-17T10:41:41.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.619+0000 7fedbeffd640 1 -- 172.21.15.83:0/4023456382 learned_addr learned my addr 172.21.15.83:0/4023456382 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:41.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.619+0000 7fedbffff640 1 --2- 172.21.15.83:0/4023456382 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fedc006c300 0x7fedc00ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:41.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.619+0000 7fedbf7fe640 1 -- 172.21.15.83:0/4023456382 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fedc006c300 msgr2=0x7fedc00ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:41.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.619+0000 7fedbf7fe640 1 --2- 172.21.15.83:0/4023456382 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fedc006c300 0x7fedc00ce020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:41.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.619+0000 7fedbf7fe640 1 -- 172.21.15.83:0/4023456382 >> v1:172.21.15.83:6789/0 conn(0x7fedc00cf780 legacy=0x7fedc006bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:41.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.619+0000 7fedbf7fe640 1 -- 172.21.15.83:0/4023456382 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc006e3c0 con 0x7fedc015eeb0 2024-04-17T10:41:41.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.619+0000 7fedbe7fc640 1 -- 172.21.15.83:0/4023456382 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 974950120 0 0) 0x7fedc006e980 con 0x7fedc00cf780 2024-04-17T10:41:41.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.619+0000 7fedbf7fe640 1 --2- 172.21.15.83:0/4023456382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc015eeb0 0x7fedc006d5d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fedb00097c0 tx=0x7fedb001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac25675366b25668 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:41.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.619+0000 7fedbe7fc640 1 -- 172.21.15.83:0/4023456382 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb00aa5f0 con 0x7fedc015eeb0 2024-04-17T10:41:41.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.619+0000 7fedbe7fc640 1 -- 172.21.15.83:0/4023456382 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb00af070 con 0x7fedc015eeb0 2024-04-17T10:41:41.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.619+0000 7fedbe7fc640 1 -- 172.21.15.83:0/4023456382 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb00aa9b0 con 0x7fedc015eeb0 2024-04-17T10:41:41.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.623+0000 7fedc611d640 1 -- 172.21.15.83:0/4023456382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc015eeb0 msgr2=0x7fedc006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:41.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.623+0000 7fedc611d640 1 --2- 172.21.15.83:0/4023456382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc015eeb0 0x7fedc006d5d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fedb00097c0 tx=0x7fedb001c470 comp rx=0 tx=0).stop 2024-04-17T10:41:41.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.623+0000 7fedc611d640 1 -- 172.21.15.83:0/4023456382 shutdown_connections 2024-04-17T10:41:41.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.623+0000 7fedc611d640 1 --2- 172.21.15.83:0/4023456382 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fedc006c300 0x7fedc00ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:41.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.623+0000 7fedc611d640 1 --2- 172.21.15.83:0/4023456382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc015eeb0 0x7fedc006d5d0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:41.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.623+0000 7fedc611d640 1 -- 172.21.15.83:0/4023456382 >> 172.21.15.83:0/4023456382 conn(0x7fedc00cc910 msgr2=0x7fedc00ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:41.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.623+0000 7fedc611d640 1 -- 172.21.15.83:0/4023456382 shutdown_connections 2024-04-17T10:41:41.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.623+0000 7fedc611d640 1 -- 172.21.15.83:0/4023456382 wait complete. 2024-04-17T10:41:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.623+0000 7fedc611d640 1 Processor -- start 2024-04-17T10:41:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.623+0000 7fedc611d640 1 -- start start 2024-04-17T10:41:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.623+0000 7fedc611d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fedc006c300 0x7fedc009caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.623+0000 7fedc611d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc00cf780 0x7fedc0155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.623+0000 7fedc611d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedc015eeb0 0x7fedc015add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.623+0000 7fedc611d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc0168a80 con 0x7fedc00cf780 2024-04-17T10:41:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.623+0000 7fedc611d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc00cef80 con 0x7fedc015eeb0 2024-04-17T10:41:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.623+0000 7fedc611d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc0168c00 con 0x7fedc006c300 2024-04-17T10:41:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.623+0000 7fedbeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc00cf780 0x7fedc0155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.623+0000 7fedbeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc00cf780 0x7fedc0155800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51262/0 (socket says 172.21.15.83:51262) 2024-04-17T10:41:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.623+0000 7fedbeffd640 1 -- 172.21.15.83:0/3943534601 learned_addr learned my addr 172.21.15.83:0/3943534601 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.623+0000 7fedbffff640 1 --2- 172.21.15.83:0/3943534601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedc015eeb0 0x7fedc015add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.623+0000 7fedbf7fe640 1 --2- 172.21.15.83:0/3943534601 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fedc006c300 0x7fedc009caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.623+0000 7fedbeffd640 1 -- 172.21.15.83:0/3943534601 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fedc006c300 msgr2=0x7fedc009caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.623+0000 7fedbeffd640 1 --2- 172.21.15.83:0/3943534601 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fedc006c300 0x7fedc009caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.623+0000 7fedbeffd640 1 -- 172.21.15.83:0/3943534601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedc015eeb0 msgr2=0x7fedc015add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.623+0000 7fedbeffd640 1 --2- 172.21.15.83:0/3943534601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedc015eeb0 0x7fedc015add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.623+0000 7fedbeffd640 1 -- 172.21.15.83:0/3943534601 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc00c90d0 con 0x7fedc00cf780 2024-04-17T10:41:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.623+0000 7fedbeffd640 1 --2- 172.21.15.83:0/3943534601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc00cf780 0x7fedc0155800 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7feda800c480 tx=0x7feda800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.623+0000 7fedb7fff640 1 -- 172.21.15.83:0/3943534601 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda8018070 con 0x7fedc00cf780 2024-04-17T10:41:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.623+0000 7fedb7fff640 1 -- 172.21.15.83:0/3943534601 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda80074e0 con 0x7fedc00cf780 2024-04-17T10:41:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.623+0000 7fedc611d640 1 -- 172.21.15.83:0/3943534601 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc00c93b0 con 0x7fedc00cf780 2024-04-17T10:41:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.623+0000 7fedb7fff640 1 -- 172.21.15.83:0/3943534601 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda8004a30 con 0x7fedc00cf780 2024-04-17T10:41:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.623+0000 7fedc611d640 1 -- 172.21.15.83:0/3943534601 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc00c9970 con 0x7fedc00cf780 2024-04-17T10:41:41.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.627+0000 7fedc611d640 1 -- 172.21.15.83:0/3943534601 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc0074f90 con 0x7fedc00cf780 2024-04-17T10:41:41.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.627+0000 7fedb7fff640 1 -- 172.21.15.83:0/3943534601 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feda801b070 con 0x7fedc00cf780 2024-04-17T10:41:41.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.627+0000 7fedb7fff640 1 --2- 172.21.15.83:0/3943534601 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feda0066f60 0x7feda0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:41.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.627+0000 7fedb7fff640 1 -- 172.21.15.83:0/3943534601 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feda8084380 con 0x7fedc00cf780 2024-04-17T10:41:41.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.627+0000 7fedbf7fe640 1 --2- 172.21.15.83:0/3943534601 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feda0066f60 0x7feda0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:41.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.627+0000 7fedbf7fe640 1 --2- 172.21.15.83:0/3943534601 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feda0066f60 0x7feda0069420 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fedb00ae000 tx=0x7fedb0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:41.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.631+0000 7fedb7fff640 1 -- 172.21.15.83:0/3943534601 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feda8055420 con 0x7fedc00cf780 2024-04-17T10:41:41.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.783+0000 7fedc611d640 1 -- 172.21.15.83:0/3943534601 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fedc00a12c0 con 0x7feda0066f60 2024-04-17T10:41:41.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.791+0000 7fedb7fff640 1 -- 172.21.15.83:0/3943534601 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251887 (secure 0 0 0) 0x7fedc00a12c0 con 0x7feda0066f60 2024-04-17T10:41:41.791 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:41:41.793 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:41:41.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.795+0000 7fedc611d640 1 -- 172.21.15.83:0/3943534601 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feda0066f60 msgr2=0x7feda0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:41.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.795+0000 7fedc611d640 1 --2- 172.21.15.83:0/3943534601 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feda0066f60 0x7feda0069420 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fedb00ae000 tx=0x7fedb0008040 comp rx=0 tx=0).stop 2024-04-17T10:41:41.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.795+0000 7fedc611d640 1 -- 172.21.15.83:0/3943534601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc00cf780 msgr2=0x7fedc0155800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:41.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.795+0000 7fedc611d640 1 --2- 172.21.15.83:0/3943534601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc00cf780 0x7fedc0155800 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7feda800c480 tx=0x7feda800c940 comp rx=0 tx=0).stop 2024-04-17T10:41:41.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.795+0000 7fedc611d640 1 -- 172.21.15.83:0/3943534601 shutdown_connections 2024-04-17T10:41:41.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.795+0000 7fedc611d640 1 --2- 172.21.15.83:0/3943534601 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feda0066f60 0x7feda0069420 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:41.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.795+0000 7fedc611d640 1 --2- 172.21.15.83:0/3943534601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedc015eeb0 0x7fedc015add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:41.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.795+0000 7fedc611d640 1 --2- 172.21.15.83:0/3943534601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc00cf780 0x7fedc0155800 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:41.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.795+0000 7fedc611d640 1 --2- 172.21.15.83:0/3943534601 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fedc006c300 0x7fedc009caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:41.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.795+0000 7fedc611d640 1 -- 172.21.15.83:0/3943534601 >> 172.21.15.83:0/3943534601 conn(0x7fedc00cc910 msgr2=0x7fedc0157900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:41.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.795+0000 7fedc611d640 1 -- 172.21.15.83:0/3943534601 shutdown_connections 2024-04-17T10:41:41.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.795+0000 7fedc611d640 1 -- 172.21.15.83:0/3943534601 wait complete. 2024-04-17T10:41:41.810 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-04-17T10:41:40.243750+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":31,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":342252,"kb_used_data":5352,"kb_used_omap":12,"kb_used_meta":336755,"kb_avail":754632468,"statfs":{"total":773094113280,"available":772743647232,"internally_reserved":0,"allocated":5480448,"data_stored":2912348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":344837680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001195"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.457624+0000","last_change":"2024-04-17T10:41:38.454812+0000","last_active":"2024-04-17T10:41:38.457624+0000","last_peered":"2024-04-17T10:41:38.457624+0000","last_clean":"2024-04-17T10:41:38.457624+0000","last_became_active":"2024-04-17T10:41:38.454068+0000","last_became_peered":"2024-04-17T10:41:38.454068+0000","last_unstale":"2024-04-17T10:41:38.457624+0000","last_undegraded":"2024-04-17T10:41:38.457624+0000","last_fullsized":"2024-04-17T10:41:38.457624+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:09:29.824195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.482293+0000","last_change":"2024-04-17T10:41:38.475143+0000","last_active":"2024-04-17T10:41:38.482293+0000","last_peered":"2024-04-17T10:41:38.482293+0000","last_clean":"2024-04-17T10:41:38.482293+0000","last_became_active":"2024-04-17T10:41:38.464378+0000","last_became_peered":"2024-04-17T10:41:38.464378+0000","last_unstale":"2024-04-17T10:41:38.482293+0000","last_undegraded":"2024-04-17T10:41:38.482293+0000","last_fullsized":"2024-04-17T10:41:38.482293+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:41:48.464779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.464984+0000","last_change":"2024-04-17T10:41:38.456591+0000","last_active":"2024-04-17T10:41:38.464984+0000","last_peered":"2024-04-17T10:41:38.464984+0000","last_clean":"2024-04-17T10:41:38.464984+0000","last_became_active":"2024-04-17T10:41:38.455835+0000","last_became_peered":"2024-04-17T10:41:38.455835+0000","last_unstale":"2024-04-17T10:41:38.464984+0000","last_undegraded":"2024-04-17T10:41:38.464984+0000","last_fullsized":"2024-04-17T10:41:38.464984+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:30:16.883126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.475196+0000","last_change":"2024-04-17T10:41:38.471953+0000","last_active":"2024-04-17T10:41:38.475196+0000","last_peered":"2024-04-17T10:41:38.475196+0000","last_clean":"2024-04-17T10:41:38.475196+0000","last_became_active":"2024-04-17T10:41:38.471638+0000","last_became_peered":"2024-04-17T10:41:38.471638+0000","last_unstale":"2024-04-17T10:41:38.475196+0000","last_undegraded":"2024-04-17T10:41:38.475196+0000","last_fullsized":"2024-04-17T10:41:38.475196+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:26:30.980989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.482461+0000","last_change":"2024-04-17T10:41:38.476262+0000","last_active":"2024-04-17T10:41:38.482461+0000","last_peered":"2024-04-17T10:41:38.482461+0000","last_clean":"2024-04-17T10:41:38.482461+0000","last_became_active":"2024-04-17T10:41:38.471052+0000","last_became_peered":"2024-04-17T10:41:38.471052+0000","last_unstale":"2024-04-17T10:41:38.482461+0000","last_undegraded":"2024-04-17T10:41:38.482461+0000","last_fullsized":"2024-04-17T10:41:38.482461+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:38:10.610378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.456541+0000","last_change":"2024-04-17T10:41:38.452412+0000","last_active":"2024-04-17T10:41:38.456541+0000","last_peered":"2024-04-17T10:41:38.456541+0000","last_clean":"2024-04-17T10:41:38.456541+0000","last_became_active":"2024-04-17T10:41:38.449166+0000","last_became_peered":"2024-04-17T10:41:38.449166+0000","last_unstale":"2024-04-17T10:41:38.456541+0000","last_undegraded":"2024-04-17T10:41:38.456541+0000","last_fullsized":"2024-04-17T10:41:38.456541+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:04:11.620082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.473230+0000","last_change":"2024-04-17T10:41:38.467456+0000","last_active":"2024-04-17T10:41:38.473230+0000","last_peered":"2024-04-17T10:41:38.473230+0000","last_clean":"2024-04-17T10:41:38.473230+0000","last_became_active":"2024-04-17T10:41:38.466706+0000","last_became_peered":"2024-04-17T10:41:38.466706+0000","last_unstale":"2024-04-17T10:41:38.473230+0000","last_undegraded":"2024-04-17T10:41:38.473230+0000","last_fullsized":"2024-04-17T10:41:38.473230+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:25:44.023009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.474950+0000","last_change":"2024-04-17T10:41:38.471695+0000","last_active":"2024-04-17T10:41:38.474950+0000","last_peered":"2024-04-17T10:41:38.474950+0000","last_clean":"2024-04-17T10:41:38.474950+0000","last_became_active":"2024-04-17T10:41:38.470467+0000","last_became_peered":"2024-04-17T10:41:38.470467+0000","last_unstale":"2024-04-17T10:41:38.474950+0000","last_undegraded":"2024-04-17T10:41:38.474950+0000","last_fullsized":"2024-04-17T10:41:38.474950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:36:33.039968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.475062+0000","last_change":"2024-04-17T10:41:38.471737+0000","last_active":"2024-04-17T10:41:38.475062+0000","last_peered":"2024-04-17T10:41:38.475062+0000","last_clean":"2024-04-17T10:41:38.475062+0000","last_became_active":"2024-04-17T10:41:38.469841+0000","last_became_peered":"2024-04-17T10:41:38.469841+0000","last_unstale":"2024-04-17T10:41:38.475062+0000","last_undegraded":"2024-04-17T10:41:38.475062+0000","last_fullsized":"2024-04-17T10:41:38.475062+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:30:16.883126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.474867+0000","last_change":"2024-04-17T10:41:38.469595+0000","last_active":"2024-04-17T10:41:38.474867+0000","last_peered":"2024-04-17T10:41:38.474867+0000","last_clean":"2024-04-17T10:41:38.474867+0000","last_became_active":"2024-04-17T10:41:38.469283+0000","last_became_peered":"2024-04-17T10:41:38.469283+0000","last_unstale":"2024-04-17T10:41:38.474867+0000","last_undegraded":"2024-04-17T10:41:38.474867+0000","last_fullsized":"2024-04-17T10:41:38.474867+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:07:17.088945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.475283+0000","last_change":"2024-04-17T10:41:38.472253+0000","last_active":"2024-04-17T10:41:38.475283+0000","last_peered":"2024-04-17T10:41:38.475283+0000","last_clean":"2024-04-17T10:41:38.475283+0000","last_became_active":"2024-04-17T10:41:38.471689+0000","last_became_peered":"2024-04-17T10:41:38.471689+0000","last_unstale":"2024-04-17T10:41:38.475283+0000","last_undegraded":"2024-04-17T10:41:38.475283+0000","last_fullsized":"2024-04-17T10:41:38.475283+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:03:40.637698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.458896+0000","last_change":"2024-04-17T10:41:38.451210+0000","last_active":"2024-04-17T10:41:38.458896+0000","last_peered":"2024-04-17T10:41:38.458896+0000","last_clean":"2024-04-17T10:41:38.458896+0000","last_became_active":"2024-04-17T10:41:38.449852+0000","last_became_peered":"2024-04-17T10:41:38.449852+0000","last_unstale":"2024-04-17T10:41:38.458896+0000","last_undegraded":"2024-04-17T10:41:38.458896+0000","last_fullsized":"2024-04-17T10:41:38.458896+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:09:12.709635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.455816+0000","last_change":"2024-04-17T10:41:38.450542+0000","last_active":"2024-04-17T10:41:38.455816+0000","last_peered":"2024-04-17T10:41:38.455816+0000","last_clean":"2024-04-17T10:41:38.455816+0000","last_became_active":"2024-04-17T10:41:38.450095+0000","last_became_peered":"2024-04-17T10:41:38.450095+0000","last_unstale":"2024-04-17T10:41:38.455816+0000","last_undegraded":"2024-04-17T10:41:38.455816+0000","last_fullsized":"2024-04-17T10:41:38.455816+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:59:43.511180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.474827+0000","last_change":"2024-04-17T10:41:38.469475+0000","last_active":"2024-04-17T10:41:38.474827+0000","last_peered":"2024-04-17T10:41:38.474827+0000","last_clean":"2024-04-17T10:41:38.474827+0000","last_became_active":"2024-04-17T10:41:38.469189+0000","last_became_peered":"2024-04-17T10:41:38.469189+0000","last_unstale":"2024-04-17T10:41:38.474827+0000","last_undegraded":"2024-04-17T10:41:38.474827+0000","last_fullsized":"2024-04-17T10:41:38.474827+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:51:22.736113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.468819+0000","last_change":"2024-04-17T10:41:38.468044+0000","last_active":"2024-04-17T10:41:38.468819+0000","last_peered":"2024-04-17T10:41:38.468819+0000","last_clean":"2024-04-17T10:41:38.468819+0000","last_became_active":"2024-04-17T10:41:38.467727+0000","last_became_peered":"2024-04-17T10:41:38.467727+0000","last_unstale":"2024-04-17T10:41:38.468819+0000","last_undegraded":"2024-04-17T10:41:38.468819+0000","last_fullsized":"2024-04-17T10:41:38.468819+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:40:14.726947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.479187+0000","last_change":"2024-04-17T10:41:38.475110+0000","last_active":"2024-04-17T10:41:38.479187+0000","last_peered":"2024-04-17T10:41:38.479187+0000","last_clean":"2024-04-17T10:41:38.479187+0000","last_became_active":"2024-04-17T10:41:38.474029+0000","last_became_peered":"2024-04-17T10:41:38.474029+0000","last_unstale":"2024-04-17T10:41:38.479187+0000","last_undegraded":"2024-04-17T10:41:38.479187+0000","last_fullsized":"2024-04-17T10:41:38.479187+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:56.618475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.466053+0000","last_change":"2024-04-17T10:41:38.463242+0000","last_active":"2024-04-17T10:41:38.466053+0000","last_peered":"2024-04-17T10:41:38.466053+0000","last_clean":"2024-04-17T10:41:38.466053+0000","last_became_active":"2024-04-17T10:41:38.462351+0000","last_became_peered":"2024-04-17T10:41:38.462351+0000","last_unstale":"2024-04-17T10:41:38.466053+0000","last_undegraded":"2024-04-17T10:41:38.466053+0000","last_fullsized":"2024-04-17T10:41:38.466053+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:26:30.980989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.485559+0000","last_change":"2024-04-17T10:41:38.481775+0000","last_active":"2024-04-17T10:41:38.485559+0000","last_peered":"2024-04-17T10:41:38.485559+0000","last_clean":"2024-04-17T10:41:38.485559+0000","last_became_active":"2024-04-17T10:41:38.480895+0000","last_became_peered":"2024-04-17T10:41:38.480895+0000","last_unstale":"2024-04-17T10:41:38.485559+0000","last_undegraded":"2024-04-17T10:41:38.485559+0000","last_fullsized":"2024-04-17T10:41:38.485559+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:03:58.711500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.473309+0000","last_change":"2024-04-17T10:41:38.470081+0000","last_active":"2024-04-17T10:41:38.473309+0000","last_peered":"2024-04-17T10:41:38.473309+0000","last_clean":"2024-04-17T10:41:38.473309+0000","last_became_active":"2024-04-17T10:41:38.468955+0000","last_became_peered":"2024-04-17T10:41:38.468955+0000","last_unstale":"2024-04-17T10:41:38.473309+0000","last_undegraded":"2024-04-17T10:41:38.473309+0000","last_fullsized":"2024-04-17T10:41:38.473309+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:54:01.366180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.464980+0000","last_change":"2024-04-17T10:41:38.462523+0000","last_active":"2024-04-17T10:41:38.464980+0000","last_peered":"2024-04-17T10:41:38.464980+0000","last_clean":"2024-04-17T10:41:38.464980+0000","last_became_active":"2024-04-17T10:41:38.462274+0000","last_became_peered":"2024-04-17T10:41:38.462274+0000","last_unstale":"2024-04-17T10:41:38.464980+0000","last_undegraded":"2024-04-17T10:41:38.464980+0000","last_fullsized":"2024-04-17T10:41:38.464980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:16:02.896971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.460601+0000","last_change":"2024-04-17T10:41:38.456516+0000","last_active":"2024-04-17T10:41:38.460601+0000","last_peered":"2024-04-17T10:41:38.460601+0000","last_clean":"2024-04-17T10:41:38.460601+0000","last_became_active":"2024-04-17T10:41:38.456180+0000","last_became_peered":"2024-04-17T10:41:38.456180+0000","last_unstale":"2024-04-17T10:41:38.460601+0000","last_undegraded":"2024-04-17T10:41:38.460601+0000","last_fullsized":"2024-04-17T10:41:38.460601+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:09:29.824195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.475248+0000","last_change":"2024-04-17T10:41:38.470503+0000","last_active":"2024-04-17T10:41:38.475248+0000","last_peered":"2024-04-17T10:41:38.475248+0000","last_clean":"2024-04-17T10:41:38.475248+0000","last_became_active":"2024-04-17T10:41:38.469746+0000","last_became_peered":"2024-04-17T10:41:38.469746+0000","last_unstale":"2024-04-17T10:41:38.475248+0000","last_undegraded":"2024-04-17T10:41:38.475248+0000","last_fullsized":"2024-04-17T10:41:38.475248+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:11:57.808488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.455722+0000","last_change":"2024-04-17T10:41:38.451093+0000","last_active":"2024-04-17T10:41:38.455722+0000","last_peered":"2024-04-17T10:41:38.455722+0000","last_clean":"2024-04-17T10:41:38.455722+0000","last_became_active":"2024-04-17T10:41:38.448832+0000","last_became_peered":"2024-04-17T10:41:38.448832+0000","last_unstale":"2024-04-17T10:41:38.455722+0000","last_undegraded":"2024-04-17T10:41:38.455722+0000","last_fullsized":"2024-04-17T10:41:38.455722+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:28:38.418159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.451405+0000","last_change":"2024-04-17T10:41:38.446790+0000","last_active":"2024-04-17T10:41:38.451405+0000","last_peered":"2024-04-17T10:41:38.451405+0000","last_clean":"2024-04-17T10:41:38.451405+0000","last_became_active":"2024-04-17T10:41:38.446069+0000","last_became_peered":"2024-04-17T10:41:38.446069+0000","last_unstale":"2024-04-17T10:41:38.451405+0000","last_undegraded":"2024-04-17T10:41:38.451405+0000","last_fullsized":"2024-04-17T10:41:38.451405+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:56:44.993567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.478650+0000","last_change":"2024-04-17T10:41:38.472886+0000","last_active":"2024-04-17T10:41:38.478650+0000","last_peered":"2024-04-17T10:41:38.478650+0000","last_clean":"2024-04-17T10:41:38.478650+0000","last_became_active":"2024-04-17T10:41:38.471403+0000","last_became_peered":"2024-04-17T10:41:38.471403+0000","last_unstale":"2024-04-17T10:41:38.478650+0000","last_undegraded":"2024-04-17T10:41:38.478650+0000","last_fullsized":"2024-04-17T10:41:38.478650+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:17:35.241441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.464068+0000","last_change":"2024-04-17T10:41:38.457501+0000","last_active":"2024-04-17T10:41:38.464068+0000","last_peered":"2024-04-17T10:41:38.464068+0000","last_clean":"2024-04-17T10:41:38.464068+0000","last_became_active":"2024-04-17T10:41:38.457219+0000","last_became_peered":"2024-04-17T10:41:38.457219+0000","last_unstale":"2024-04-17T10:41:38.464068+0000","last_undegraded":"2024-04-17T10:41:38.464068+0000","last_fullsized":"2024-04-17T10:41:38.464068+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:28:38.418159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.465529+0000","last_change":"2024-04-17T10:41:38.461324+0000","last_active":"2024-04-17T10:41:38.465529+0000","last_peered":"2024-04-17T10:41:38.465529+0000","last_clean":"2024-04-17T10:41:38.465529+0000","last_became_active":"2024-04-17T10:41:38.460487+0000","last_became_peered":"2024-04-17T10:41:38.460487+0000","last_unstale":"2024-04-17T10:41:38.465529+0000","last_undegraded":"2024-04-17T10:41:38.465529+0000","last_fullsized":"2024-04-17T10:41:38.465529+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:03:03.269784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.454893+0000","last_change":"2024-04-17T10:41:38.450615+0000","last_active":"2024-04-17T10:41:38.454893+0000","last_peered":"2024-04-17T10:41:38.454893+0000","last_clean":"2024-04-17T10:41:38.454893+0000","last_became_active":"2024-04-17T10:41:38.448925+0000","last_became_peered":"2024-04-17T10:41:38.448925+0000","last_unstale":"2024-04-17T10:41:38.454893+0000","last_undegraded":"2024-04-17T10:41:38.454893+0000","last_fullsized":"2024-04-17T10:41:38.454893+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:31:26.144792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.473022+0000","last_change":"2024-04-17T10:41:38.470054+0000","last_active":"2024-04-17T10:41:38.473022+0000","last_peered":"2024-04-17T10:41:38.473022+0000","last_clean":"2024-04-17T10:41:38.473022+0000","last_became_active":"2024-04-17T10:41:38.469085+0000","last_became_peered":"2024-04-17T10:41:38.469085+0000","last_unstale":"2024-04-17T10:41:38.473022+0000","last_undegraded":"2024-04-17T10:41:38.473022+0000","last_fullsized":"2024-04-17T10:41:38.473022+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:51:59.724625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.463072+0000","last_change":"2024-04-17T10:41:38.454560+0000","last_active":"2024-04-17T10:41:38.463072+0000","last_peered":"2024-04-17T10:41:38.463072+0000","last_clean":"2024-04-17T10:41:38.463072+0000","last_became_active":"2024-04-17T10:41:38.453821+0000","last_became_peered":"2024-04-17T10:41:38.453821+0000","last_unstale":"2024-04-17T10:41:38.463072+0000","last_undegraded":"2024-04-17T10:41:38.463072+0000","last_fullsized":"2024-04-17T10:41:38.463072+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:12:30.359237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.456762+0000","last_change":"2024-04-17T10:41:38.452675+0000","last_active":"2024-04-17T10:41:38.456762+0000","last_peered":"2024-04-17T10:41:38.456762+0000","last_clean":"2024-04-17T10:41:38.456762+0000","last_became_active":"2024-04-17T10:41:38.447556+0000","last_became_peered":"2024-04-17T10:41:38.447556+0000","last_unstale":"2024-04-17T10:41:38.456762+0000","last_undegraded":"2024-04-17T10:41:38.456762+0000","last_fullsized":"2024-04-17T10:41:38.456762+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:40:09.456885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.464854+0000","last_change":"2024-04-17T10:41:38.461835+0000","last_active":"2024-04-17T10:41:38.464854+0000","last_peered":"2024-04-17T10:41:38.464854+0000","last_clean":"2024-04-17T10:41:38.464854+0000","last_became_active":"2024-04-17T10:41:38.458313+0000","last_became_peered":"2024-04-17T10:41:38.458313+0000","last_unstale":"2024-04-17T10:41:38.464854+0000","last_undegraded":"2024-04-17T10:41:38.464854+0000","last_fullsized":"2024-04-17T10:41:38.464854+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:07:53.020927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.465728+0000","last_change":"2024-04-17T10:41:38.458540+0000","last_active":"2024-04-17T10:41:38.465728+0000","last_peered":"2024-04-17T10:41:38.465728+0000","last_clean":"2024-04-17T10:41:38.465728+0000","last_became_active":"2024-04-17T10:41:38.457958+0000","last_became_peered":"2024-04-17T10:41:38.457958+0000","last_unstale":"2024-04-17T10:41:38.465728+0000","last_undegraded":"2024-04-17T10:41:38.465728+0000","last_fullsized":"2024-04-17T10:41:38.465728+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:30:16.883126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.471972+0000","last_change":"2024-04-17T10:41:37.459804+0000","last_active":"2024-04-17T10:41:38.471972+0000","last_peered":"2024-04-17T10:41:38.471972+0000","last_clean":"2024-04-17T10:41:38.471972+0000","last_became_active":"2024-04-17T10:41:37.459479+0000","last_became_peered":"2024-04-17T10:41:37.459479+0000","last_unstale":"2024-04-17T10:41:38.471972+0000","last_undegraded":"2024-04-17T10:41:38.471972+0000","last_fullsized":"2024-04-17T10:41:38.471972+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:51:46.650282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.478695+0000","last_change":"2024-04-17T10:41:38.473525+0000","last_active":"2024-04-17T10:41:38.478695+0000","last_peered":"2024-04-17T10:41:38.478695+0000","last_clean":"2024-04-17T10:41:38.478695+0000","last_became_active":"2024-04-17T10:41:38.472174+0000","last_became_peered":"2024-04-17T10:41:38.472174+0000","last_unstale":"2024-04-17T10:41:38.478695+0000","last_undegraded":"2024-04-17T10:41:38.478695+0000","last_fullsized":"2024-04-17T10:41:38.478695+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:12:39.460966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.485591+0000","last_change":"2024-04-17T10:41:37.460584+0000","last_active":"2024-04-17T10:41:38.485591+0000","last_peered":"2024-04-17T10:41:38.485591+0000","last_clean":"2024-04-17T10:41:38.485591+0000","last_became_active":"2024-04-17T10:41:37.459907+0000","last_became_peered":"2024-04-17T10:41:37.459907+0000","last_unstale":"2024-04-17T10:41:38.485591+0000","last_undegraded":"2024-04-17T10:41:38.485591+0000","last_fullsized":"2024-04-17T10:41:38.485591+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:03:39.629440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.482685+0000","last_change":"2024-04-17T10:41:38.477585+0000","last_active":"2024-04-17T10:41:38.482685+0000","last_peered":"2024-04-17T10:41:38.482685+0000","last_clean":"2024-04-17T10:41:38.482685+0000","last_became_active":"2024-04-17T10:41:38.459721+0000","last_became_peered":"2024-04-17T10:41:38.459721+0000","last_unstale":"2024-04-17T10:41:38.482685+0000","last_undegraded":"2024-04-17T10:41:38.482685+0000","last_fullsized":"2024-04-17T10:41:38.482685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:40:09.456885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.482090+0000","last_change":"2024-04-17T10:41:37.449488+0000","last_active":"2024-04-17T10:41:38.482090+0000","last_peered":"2024-04-17T10:41:38.482090+0000","last_clean":"2024-04-17T10:41:38.482090+0000","last_became_active":"2024-04-17T10:41:37.449108+0000","last_became_peered":"2024-04-17T10:41:37.449108+0000","last_unstale":"2024-04-17T10:41:38.482090+0000","last_undegraded":"2024-04-17T10:41:38.482090+0000","last_fullsized":"2024-04-17T10:41:38.482090+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:12:29.350979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.469644+0000","last_change":"2024-04-17T10:41:38.465350+0000","last_active":"2024-04-17T10:41:38.469644+0000","last_peered":"2024-04-17T10:41:38.469644+0000","last_clean":"2024-04-17T10:41:38.469644+0000","last_became_active":"2024-04-17T10:41:38.465085+0000","last_became_peered":"2024-04-17T10:41:38.465085+0000","last_unstale":"2024-04-17T10:41:38.469644+0000","last_undegraded":"2024-04-17T10:41:38.469644+0000","last_fullsized":"2024-04-17T10:41:38.469644+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:03:03.269784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.484362+0000","last_change":"2024-04-17T10:41:37.459728+0000","last_active":"2024-04-17T10:41:38.484362+0000","last_peered":"2024-04-17T10:41:38.484362+0000","last_clean":"2024-04-17T10:41:38.484362+0000","last_became_active":"2024-04-17T10:41:37.459283+0000","last_became_peered":"2024-04-17T10:41:37.459283+0000","last_unstale":"2024-04-17T10:41:38.484362+0000","last_undegraded":"2024-04-17T10:41:38.484362+0000","last_fullsized":"2024-04-17T10:41:38.484362+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:31:25.136534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.463009+0000","last_change":"2024-04-17T10:41:38.453473+0000","last_active":"2024-04-17T10:41:38.463009+0000","last_peered":"2024-04-17T10:41:38.463009+0000","last_clean":"2024-04-17T10:41:38.463009+0000","last_became_active":"2024-04-17T10:41:38.452911+0000","last_became_peered":"2024-04-17T10:41:38.452911+0000","last_unstale":"2024-04-17T10:41:38.463009+0000","last_undegraded":"2024-04-17T10:41:38.463009+0000","last_fullsized":"2024-04-17T10:41:38.463009+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:59:16.279043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.467706+0000","last_change":"2024-04-17T10:41:37.449711+0000","last_active":"2024-04-17T10:41:38.467706+0000","last_peered":"2024-04-17T10:41:38.467706+0000","last_clean":"2024-04-17T10:41:38.467706+0000","last_became_active":"2024-04-17T10:41:37.449391+0000","last_became_peered":"2024-04-17T10:41:37.449391+0000","last_unstale":"2024-04-17T10:41:38.467706+0000","last_undegraded":"2024-04-17T10:41:38.467706+0000","last_fullsized":"2024-04-17T10:41:38.467706+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:31:25.136534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.474296+0000","last_change":"2024-04-17T10:41:38.460279+0000","last_active":"2024-04-17T10:41:38.474296+0000","last_peered":"2024-04-17T10:41:38.474296+0000","last_clean":"2024-04-17T10:41:38.474296+0000","last_became_active":"2024-04-17T10:41:38.459941+0000","last_became_peered":"2024-04-17T10:41:38.459941+0000","last_unstale":"2024-04-17T10:41:38.474296+0000","last_undegraded":"2024-04-17T10:41:38.474296+0000","last_fullsized":"2024-04-17T10:41:38.474296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:56.618475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.483090+0000","last_change":"2024-04-17T10:41:37.456651+0000","last_active":"2024-04-17T10:41:38.483090+0000","last_peered":"2024-04-17T10:41:38.483090+0000","last_clean":"2024-04-17T10:41:38.483090+0000","last_became_active":"2024-04-17T10:41:37.455299+0000","last_became_peered":"2024-04-17T10:41:37.455299+0000","last_unstale":"2024-04-17T10:41:38.483090+0000","last_undegraded":"2024-04-17T10:41:38.483090+0000","last_fullsized":"2024-04-17T10:41:38.483090+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:51:21.727855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.475106+0000","last_change":"2024-04-17T10:41:38.471934+0000","last_active":"2024-04-17T10:41:38.475106+0000","last_peered":"2024-04-17T10:41:38.475106+0000","last_clean":"2024-04-17T10:41:38.475106+0000","last_became_active":"2024-04-17T10:41:38.470071+0000","last_became_peered":"2024-04-17T10:41:38.470071+0000","last_unstale":"2024-04-17T10:41:38.475106+0000","last_undegraded":"2024-04-17T10:41:38.475106+0000","last_fullsized":"2024-04-17T10:41:38.475106+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:03:03.269784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.429116+0000","last_change":"2024-04-17T10:41:37.450244+0000","last_active":"2024-04-17T10:41:38.429116+0000","last_peered":"2024-04-17T10:41:38.429116+0000","last_clean":"2024-04-17T10:41:38.429116+0000","last_became_active":"2024-04-17T10:41:37.449776+0000","last_became_peered":"2024-04-17T10:41:37.449776+0000","last_unstale":"2024-04-17T10:41:38.429116+0000","last_undegraded":"2024-04-17T10:41:38.429116+0000","last_fullsized":"2024-04-17T10:41:38.429116+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:17:21.157969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.455778+0000","last_change":"2024-04-17T10:41:38.451335+0000","last_active":"2024-04-17T10:41:38.455778+0000","last_peered":"2024-04-17T10:41:38.455778+0000","last_clean":"2024-04-17T10:41:38.455778+0000","last_became_active":"2024-04-17T10:41:38.449167+0000","last_became_peered":"2024-04-17T10:41:38.449167+0000","last_unstale":"2024-04-17T10:41:38.455778+0000","last_undegraded":"2024-04-17T10:41:38.455778+0000","last_fullsized":"2024-04-17T10:41:38.455778+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:07:53.020927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.444252+0000","last_change":"2024-04-17T10:41:37.438228+0000","last_active":"2024-04-17T10:41:38.444252+0000","last_peered":"2024-04-17T10:41:38.444252+0000","last_clean":"2024-04-17T10:41:38.444252+0000","last_became_active":"2024-04-17T10:41:37.437730+0000","last_became_peered":"2024-04-17T10:41:37.437730+0000","last_unstale":"2024-04-17T10:41:38.444252+0000","last_undegraded":"2024-04-17T10:41:38.444252+0000","last_fullsized":"2024-04-17T10:41:38.444252+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:54:00.357922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.485520+0000","last_change":"2024-04-17T10:41:38.481752+0000","last_active":"2024-04-17T10:41:38.485520+0000","last_peered":"2024-04-17T10:41:38.485520+0000","last_clean":"2024-04-17T10:41:38.485520+0000","last_became_active":"2024-04-17T10:41:38.480519+0000","last_became_peered":"2024-04-17T10:41:38.480519+0000","last_unstale":"2024-04-17T10:41:38.485520+0000","last_undegraded":"2024-04-17T10:41:38.485520+0000","last_fullsized":"2024-04-17T10:41:38.485520+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:51:41.472422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.441403+0000","last_change":"2024-04-17T10:41:37.453973+0000","last_active":"2024-04-17T10:41:38.441403+0000","last_peered":"2024-04-17T10:41:38.441403+0000","last_clean":"2024-04-17T10:41:38.441403+0000","last_became_active":"2024-04-17T10:41:37.453164+0000","last_became_peered":"2024-04-17T10:41:37.453164+0000","last_unstale":"2024-04-17T10:41:38.441403+0000","last_undegraded":"2024-04-17T10:41:38.441403+0000","last_fullsized":"2024-04-17T10:41:38.441403+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:07:16.080687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.447158+0000","last_change":"2024-04-17T10:41:38.444213+0000","last_active":"2024-04-17T10:41:38.447158+0000","last_peered":"2024-04-17T10:41:38.447158+0000","last_clean":"2024-04-17T10:41:38.447158+0000","last_became_active":"2024-04-17T10:41:38.443464+0000","last_became_peered":"2024-04-17T10:41:38.443464+0000","last_unstale":"2024-04-17T10:41:38.447158+0000","last_undegraded":"2024-04-17T10:41:38.447158+0000","last_fullsized":"2024-04-17T10:41:38.447158+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:38:10.610378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.446741+0000","last_change":"2024-04-17T10:41:37.450709+0000","last_active":"2024-04-17T10:41:38.446741+0000","last_peered":"2024-04-17T10:41:38.446741+0000","last_clean":"2024-04-17T10:41:38.446741+0000","last_became_active":"2024-04-17T10:41:37.450412+0000","last_became_peered":"2024-04-17T10:41:37.450412+0000","last_unstale":"2024-04-17T10:41:38.446741+0000","last_undegraded":"2024-04-17T10:41:38.446741+0000","last_fullsized":"2024-04-17T10:41:38.446741+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:17:34.233183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.459270+0000","last_change":"2024-04-17T10:41:38.456726+0000","last_active":"2024-04-17T10:41:38.459270+0000","last_peered":"2024-04-17T10:41:38.459270+0000","last_clean":"2024-04-17T10:41:38.459270+0000","last_became_active":"2024-04-17T10:41:38.456415+0000","last_became_peered":"2024-04-17T10:41:38.456415+0000","last_unstale":"2024-04-17T10:41:38.459270+0000","last_undegraded":"2024-04-17T10:41:38.459270+0000","last_fullsized":"2024-04-17T10:41:38.459270+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:22:35.852561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.485356+0000","last_change":"2024-04-17T10:41:37.462696+0000","last_active":"2024-04-17T10:41:38.485356+0000","last_peered":"2024-04-17T10:41:38.485356+0000","last_clean":"2024-04-17T10:41:38.485356+0000","last_became_active":"2024-04-17T10:41:37.462209+0000","last_became_peered":"2024-04-17T10:41:37.462209+0000","last_unstale":"2024-04-17T10:41:38.485356+0000","last_undegraded":"2024-04-17T10:41:38.485356+0000","last_fullsized":"2024-04-17T10:41:38.485356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:04:10.611824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.474700+0000","last_change":"2024-04-17T10:41:38.468072+0000","last_active":"2024-04-17T10:41:38.474700+0000","last_peered":"2024-04-17T10:41:38.474700+0000","last_clean":"2024-04-17T10:41:38.474700+0000","last_became_active":"2024-04-17T10:41:38.466877+0000","last_became_peered":"2024-04-17T10:41:38.466877+0000","last_unstale":"2024-04-17T10:41:38.474700+0000","last_undegraded":"2024-04-17T10:41:38.474700+0000","last_fullsized":"2024-04-17T10:41:38.474700+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:22:35.852561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":18,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.484589+0000","last_change":"2024-04-17T10:41:37.452385+0000","last_active":"2024-04-17T10:41:38.484589+0000","last_peered":"2024-04-17T10:41:38.484589+0000","last_clean":"2024-04-17T10:41:38.484589+0000","last_became_active":"2024-04-17T10:41:37.451936+0000","last_became_peered":"2024-04-17T10:41:37.451936+0000","last_unstale":"2024-04-17T10:41:38.484589+0000","last_undegraded":"2024-04-17T10:41:38.484589+0000","last_fullsized":"2024-04-17T10:41:38.484589+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:33:50.317259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.471224+0000","last_change":"2024-04-17T10:41:38.468117+0000","last_active":"2024-04-17T10:41:38.471224+0000","last_peered":"2024-04-17T10:41:38.471224+0000","last_clean":"2024-04-17T10:41:38.471224+0000","last_became_active":"2024-04-17T10:41:38.467465+0000","last_became_peered":"2024-04-17T10:41:38.467465+0000","last_unstale":"2024-04-17T10:41:38.471224+0000","last_undegraded":"2024-04-17T10:41:38.471224+0000","last_fullsized":"2024-04-17T10:41:38.471224+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:15:45.348582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.475108+0000","last_change":"2024-04-17T10:41:37.439006+0000","last_active":"2024-04-17T10:41:38.475108+0000","last_peered":"2024-04-17T10:41:38.475108+0000","last_clean":"2024-04-17T10:41:38.475108+0000","last_became_active":"2024-04-17T10:41:37.438687+0000","last_became_peered":"2024-04-17T10:41:37.438687+0000","last_unstale":"2024-04-17T10:41:38.475108+0000","last_undegraded":"2024-04-17T10:41:38.475108+0000","last_fullsized":"2024-04-17T10:41:38.475108+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:41:47.456521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.474792+0000","last_change":"2024-04-17T10:41:38.469628+0000","last_active":"2024-04-17T10:41:38.474792+0000","last_peered":"2024-04-17T10:41:38.474792+0000","last_clean":"2024-04-17T10:41:38.474792+0000","last_became_active":"2024-04-17T10:41:38.469322+0000","last_became_peered":"2024-04-17T10:41:38.469322+0000","last_unstale":"2024-04-17T10:41:38.474792+0000","last_undegraded":"2024-04-17T10:41:38.474792+0000","last_fullsized":"2024-04-17T10:41:38.474792+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:59:16.279043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.432098+0000","last_change":"2024-04-17T10:41:37.440526+0000","last_active":"2024-04-17T10:41:38.432098+0000","last_peered":"2024-04-17T10:41:38.432098+0000","last_clean":"2024-04-17T10:41:38.432098+0000","last_became_active":"2024-04-17T10:41:37.439773+0000","last_became_peered":"2024-04-17T10:41:37.439773+0000","last_unstale":"2024-04-17T10:41:38.432098+0000","last_undegraded":"2024-04-17T10:41:38.432098+0000","last_fullsized":"2024-04-17T10:41:38.432098+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:51:46.650282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.468435+0000","last_change":"2024-04-17T10:41:38.466712+0000","last_active":"2024-04-17T10:41:38.468435+0000","last_peered":"2024-04-17T10:41:38.468435+0000","last_clean":"2024-04-17T10:41:38.468435+0000","last_became_active":"2024-04-17T10:41:38.464593+0000","last_became_peered":"2024-04-17T10:41:38.464593+0000","last_unstale":"2024-04-17T10:41:38.468435+0000","last_undegraded":"2024-04-17T10:41:38.468435+0000","last_fullsized":"2024-04-17T10:41:38.468435+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:51:23.707316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.452435+0000","last_change":"2024-04-17T10:41:37.438244+0000","last_active":"2024-04-17T10:41:38.452435+0000","last_peered":"2024-04-17T10:41:38.452435+0000","last_clean":"2024-04-17T10:41:38.452435+0000","last_became_active":"2024-04-17T10:41:37.437914+0000","last_became_peered":"2024-04-17T10:41:37.437914+0000","last_unstale":"2024-04-17T10:41:38.452435+0000","last_undegraded":"2024-04-17T10:41:38.452435+0000","last_fullsized":"2024-04-17T10:41:38.452435+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:09:11.701377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.450239+0000","last_change":"2024-04-17T10:41:38.446965+0000","last_active":"2024-04-17T10:41:38.450239+0000","last_peered":"2024-04-17T10:41:38.450239+0000","last_clean":"2024-04-17T10:41:38.450239+0000","last_became_active":"2024-04-17T10:41:38.446093+0000","last_became_peered":"2024-04-17T10:41:38.446093+0000","last_unstale":"2024-04-17T10:41:38.450239+0000","last_undegraded":"2024-04-17T10:41:38.450239+0000","last_fullsized":"2024-04-17T10:41:38.450239+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:40:09.456885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.484303+0000","last_change":"2024-04-17T10:41:37.449790+0000","last_active":"2024-04-17T10:41:38.484303+0000","last_peered":"2024-04-17T10:41:38.484303+0000","last_clean":"2024-04-17T10:41:38.484303+0000","last_became_active":"2024-04-17T10:41:37.449406+0000","last_became_peered":"2024-04-17T10:41:37.449406+0000","last_unstale":"2024-04-17T10:41:38.484303+0000","last_undegraded":"2024-04-17T10:41:38.484303+0000","last_fullsized":"2024-04-17T10:41:38.484303+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:51:58.716367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.455856+0000","last_change":"2024-04-17T10:41:38.451185+0000","last_active":"2024-04-17T10:41:38.455856+0000","last_peered":"2024-04-17T10:41:38.455856+0000","last_clean":"2024-04-17T10:41:38.455856+0000","last_became_active":"2024-04-17T10:41:38.450475+0000","last_became_peered":"2024-04-17T10:41:38.450475+0000","last_unstale":"2024-04-17T10:41:38.455856+0000","last_undegraded":"2024-04-17T10:41:38.455856+0000","last_fullsized":"2024-04-17T10:41:38.455856+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:38:42.837743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.465015+0000","last_change":"2024-04-17T10:41:37.437417+0000","last_active":"2024-04-17T10:41:38.465015+0000","last_peered":"2024-04-17T10:41:38.465015+0000","last_clean":"2024-04-17T10:41:38.465015+0000","last_became_active":"2024-04-17T10:41:37.437021+0000","last_became_peered":"2024-04-17T10:41:37.437021+0000","last_unstale":"2024-04-17T10:41:38.465015+0000","last_undegraded":"2024-04-17T10:41:38.465015+0000","last_fullsized":"2024-04-17T10:41:38.465015+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:39:52.262198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.474356+0000","last_change":"2024-04-17T10:41:38.459897+0000","last_active":"2024-04-17T10:41:38.474356+0000","last_peered":"2024-04-17T10:41:38.474356+0000","last_clean":"2024-04-17T10:41:38.474356+0000","last_became_active":"2024-04-17T10:41:38.459676+0000","last_became_peered":"2024-04-17T10:41:38.459676+0000","last_unstale":"2024-04-17T10:41:38.474356+0000","last_undegraded":"2024-04-17T10:41:38.474356+0000","last_fullsized":"2024-04-17T10:41:38.474356+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:12:39.460966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.484580+0000","last_change":"2024-04-17T10:41:37.458195+0000","last_active":"2024-04-17T10:41:38.484580+0000","last_peered":"2024-04-17T10:41:38.484580+0000","last_clean":"2024-04-17T10:41:38.484580+0000","last_became_active":"2024-04-17T10:41:37.457225+0000","last_became_peered":"2024-04-17T10:41:37.457225+0000","last_unstale":"2024-04-17T10:41:38.484580+0000","last_undegraded":"2024-04-17T10:41:38.484580+0000","last_fullsized":"2024-04-17T10:41:38.484580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:11:56.800230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.467138+0000","last_change":"2024-04-17T10:41:38.461701+0000","last_active":"2024-04-17T10:41:38.467138+0000","last_peered":"2024-04-17T10:41:38.467138+0000","last_clean":"2024-04-17T10:41:38.467138+0000","last_became_active":"2024-04-17T10:41:38.461020+0000","last_became_peered":"2024-04-17T10:41:38.461020+0000","last_unstale":"2024-04-17T10:41:38.467138+0000","last_undegraded":"2024-04-17T10:41:38.467138+0000","last_fullsized":"2024-04-17T10:41:38.467138+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:16:02.896971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.482465+0000","last_change":"2024-04-17T10:41:37.451444+0000","last_active":"2024-04-17T10:41:38.482465+0000","last_peered":"2024-04-17T10:41:38.482465+0000","last_clean":"2024-04-17T10:41:38.482465+0000","last_became_active":"2024-04-17T10:41:37.450652+0000","last_became_peered":"2024-04-17T10:41:37.450652+0000","last_unstale":"2024-04-17T10:41:38.482465+0000","last_undegraded":"2024-04-17T10:41:38.482465+0000","last_fullsized":"2024-04-17T10:41:38.482465+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:07:15.434437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.475927+0000","last_change":"2024-04-17T10:41:38.472185+0000","last_active":"2024-04-17T10:41:38.475927+0000","last_peered":"2024-04-17T10:41:38.475927+0000","last_clean":"2024-04-17T10:41:38.475927+0000","last_became_active":"2024-04-17T10:41:38.471615+0000","last_became_peered":"2024-04-17T10:41:38.471615+0000","last_unstale":"2024-04-17T10:41:38.475927+0000","last_undegraded":"2024-04-17T10:41:38.475927+0000","last_fullsized":"2024-04-17T10:41:38.475927+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:10:35.305381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.451779+0000","last_change":"2024-04-17T10:41:37.455337+0000","last_active":"2024-04-17T10:41:38.451779+0000","last_peered":"2024-04-17T10:41:38.451779+0000","last_clean":"2024-04-17T10:41:38.451779+0000","last_became_active":"2024-04-17T10:41:37.454990+0000","last_became_peered":"2024-04-17T10:41:37.454990+0000","last_unstale":"2024-04-17T10:41:38.451779+0000","last_undegraded":"2024-04-17T10:41:38.451779+0000","last_fullsized":"2024-04-17T10:41:38.451779+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:57:17.192590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.459647+0000","last_change":"2024-04-17T10:41:38.456662+0000","last_active":"2024-04-17T10:41:38.459647+0000","last_peered":"2024-04-17T10:41:38.459647+0000","last_clean":"2024-04-17T10:41:38.459647+0000","last_became_active":"2024-04-17T10:41:38.456343+0000","last_became_peered":"2024-04-17T10:41:38.456343+0000","last_unstale":"2024-04-17T10:41:38.459647+0000","last_undegraded":"2024-04-17T10:41:38.459647+0000","last_fullsized":"2024-04-17T10:41:38.459647+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:25:44.023009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.467819+0000","last_change":"2024-04-17T10:41:37.447234+0000","last_active":"2024-04-17T10:41:38.467819+0000","last_peered":"2024-04-17T10:41:38.467819+0000","last_clean":"2024-04-17T10:41:38.467819+0000","last_became_active":"2024-04-17T10:41:37.446724+0000","last_became_peered":"2024-04-17T10:41:37.446724+0000","last_unstale":"2024-04-17T10:41:38.467819+0000","last_undegraded":"2024-04-17T10:41:38.467819+0000","last_fullsized":"2024-04-17T10:41:38.467819+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:38:09.602120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.458101+0000","last_change":"2024-04-17T10:41:38.454552+0000","last_active":"2024-04-17T10:41:38.458101+0000","last_peered":"2024-04-17T10:41:38.458101+0000","last_clean":"2024-04-17T10:41:38.458101+0000","last_became_active":"2024-04-17T10:41:38.454225+0000","last_became_peered":"2024-04-17T10:41:38.454225+0000","last_unstale":"2024-04-17T10:41:38.458101+0000","last_undegraded":"2024-04-17T10:41:38.458101+0000","last_fullsized":"2024-04-17T10:41:38.458101+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:36:33.039968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.430218+0000","last_change":"2024-04-17T10:41:37.460106+0000","last_active":"2024-04-17T10:41:38.430218+0000","last_peered":"2024-04-17T10:41:38.430218+0000","last_clean":"2024-04-17T10:41:38.430218+0000","last_became_active":"2024-04-17T10:41:37.459406+0000","last_became_peered":"2024-04-17T10:41:37.459406+0000","last_unstale":"2024-04-17T10:41:38.430218+0000","last_undegraded":"2024-04-17T10:41:38.430218+0000","last_fullsized":"2024-04-17T10:41:38.430218+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:51:46.650282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.464110+0000","last_change":"2024-04-17T10:41:38.456587+0000","last_active":"2024-04-17T10:41:38.464110+0000","last_peered":"2024-04-17T10:41:38.464110+0000","last_clean":"2024-04-17T10:41:38.464110+0000","last_became_active":"2024-04-17T10:41:38.437409+0000","last_became_peered":"2024-04-17T10:41:38.437409+0000","last_unstale":"2024-04-17T10:41:38.464110+0000","last_undegraded":"2024-04-17T10:41:38.464110+0000","last_fullsized":"2024-04-17T10:41:38.464110+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:56:44.993567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.441307+0000","last_change":"2024-04-17T10:41:37.450703+0000","last_active":"2024-04-17T10:41:38.441307+0000","last_peered":"2024-04-17T10:41:38.441307+0000","last_clean":"2024-04-17T10:41:38.441307+0000","last_became_active":"2024-04-17T10:41:37.450429+0000","last_became_peered":"2024-04-17T10:41:37.450429+0000","last_unstale":"2024-04-17T10:41:38.441307+0000","last_undegraded":"2024-04-17T10:41:38.441307+0000","last_fullsized":"2024-04-17T10:41:38.441307+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:43:32.663340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.474969+0000","last_change":"2024-04-17T10:41:38.471446+0000","last_active":"2024-04-17T10:41:38.474969+0000","last_peered":"2024-04-17T10:41:38.474969+0000","last_clean":"2024-04-17T10:41:38.474969+0000","last_became_active":"2024-04-17T10:41:38.470111+0000","last_became_peered":"2024-04-17T10:41:38.470111+0000","last_unstale":"2024-04-17T10:41:38.474969+0000","last_undegraded":"2024-04-17T10:41:38.474969+0000","last_fullsized":"2024-04-17T10:41:38.474969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:12:30.359237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.428972+0000","last_change":"2024-04-17T10:41:37.439030+0000","last_active":"2024-04-17T10:41:38.428972+0000","last_peered":"2024-04-17T10:41:38.428972+0000","last_clean":"2024-04-17T10:41:38.428972+0000","last_became_active":"2024-04-17T10:41:37.438367+0000","last_became_peered":"2024-04-17T10:41:37.438367+0000","last_unstale":"2024-04-17T10:41:38.428972+0000","last_undegraded":"2024-04-17T10:41:38.428972+0000","last_fullsized":"2024-04-17T10:41:38.428972+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:07:15.434437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.473244+0000","last_change":"2024-04-17T10:41:38.470029+0000","last_active":"2024-04-17T10:41:38.473244+0000","last_peered":"2024-04-17T10:41:38.473244+0000","last_clean":"2024-04-17T10:41:38.473244+0000","last_became_active":"2024-04-17T10:41:38.469123+0000","last_became_peered":"2024-04-17T10:41:38.469123+0000","last_unstale":"2024-04-17T10:41:38.473244+0000","last_undegraded":"2024-04-17T10:41:38.473244+0000","last_fullsized":"2024-04-17T10:41:38.473244+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:43:33.671598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.452078+0000","last_change":"2024-04-17T10:41:37.461421+0000","last_active":"2024-04-17T10:41:38.452078+0000","last_peered":"2024-04-17T10:41:38.452078+0000","last_clean":"2024-04-17T10:41:38.452078+0000","last_became_active":"2024-04-17T10:41:37.461139+0000","last_became_peered":"2024-04-17T10:41:37.461139+0000","last_unstale":"2024-04-17T10:41:38.452078+0000","last_undegraded":"2024-04-17T10:41:38.452078+0000","last_fullsized":"2024-04-17T10:41:38.452078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:07:15.434437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.474633+0000","last_change":"2024-04-17T10:41:38.466590+0000","last_active":"2024-04-17T10:41:38.474633+0000","last_peered":"2024-04-17T10:41:38.474633+0000","last_clean":"2024-04-17T10:41:38.474633+0000","last_became_active":"2024-04-17T10:41:38.466214+0000","last_became_peered":"2024-04-17T10:41:38.466214+0000","last_unstale":"2024-04-17T10:41:38.474633+0000","last_undegraded":"2024-04-17T10:41:38.474633+0000","last_fullsized":"2024-04-17T10:41:38.474633+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:27:11.970761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.434669+0000","last_change":"2024-04-17T10:41:37.463415+0000","last_active":"2024-04-17T10:41:38.434669+0000","last_peered":"2024-04-17T10:41:38.434669+0000","last_clean":"2024-04-17T10:41:38.434669+0000","last_became_active":"2024-04-17T10:41:37.462549+0000","last_became_peered":"2024-04-17T10:41:37.462549+0000","last_unstale":"2024-04-17T10:41:38.434669+0000","last_undegraded":"2024-04-17T10:41:38.434669+0000","last_fullsized":"2024-04-17T10:41:38.434669+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:41:47.456521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.456034+0000","last_change":"2024-04-17T10:41:38.451923+0000","last_active":"2024-04-17T10:41:38.456034+0000","last_peered":"2024-04-17T10:41:38.456034+0000","last_clean":"2024-04-17T10:41:38.456034+0000","last_became_active":"2024-04-17T10:41:38.447371+0000","last_became_peered":"2024-04-17T10:41:38.447371+0000","last_unstale":"2024-04-17T10:41:38.456034+0000","last_undegraded":"2024-04-17T10:41:38.456034+0000","last_fullsized":"2024-04-17T10:41:38.456034+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:12:50.521087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.437749+0000","last_change":"2024-04-17T10:41:37.457523+0000","last_active":"2024-04-17T10:41:38.437749+0000","last_peered":"2024-04-17T10:41:38.437749+0000","last_clean":"2024-04-17T10:41:38.437749+0000","last_became_active":"2024-04-17T10:41:37.457177+0000","last_became_peered":"2024-04-17T10:41:37.457177+0000","last_unstale":"2024-04-17T10:41:38.437749+0000","last_undegraded":"2024-04-17T10:41:38.437749+0000","last_fullsized":"2024-04-17T10:41:38.437749+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:07:16.080687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.467578+0000","last_change":"2024-04-17T10:41:38.464329+0000","last_active":"2024-04-17T10:41:38.467578+0000","last_peered":"2024-04-17T10:41:38.467578+0000","last_clean":"2024-04-17T10:41:38.467578+0000","last_became_active":"2024-04-17T10:41:38.464010+0000","last_became_peered":"2024-04-17T10:41:38.464010+0000","last_unstale":"2024-04-17T10:41:38.467578+0000","last_undegraded":"2024-04-17T10:41:38.467578+0000","last_fullsized":"2024-04-17T10:41:38.467578+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:24:09.759918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.433997+0000","last_change":"2024-04-17T10:41:37.460609+0000","last_active":"2024-04-17T10:41:38.433997+0000","last_peered":"2024-04-17T10:41:38.433997+0000","last_clean":"2024-04-17T10:41:38.433997+0000","last_became_active":"2024-04-17T10:41:37.459973+0000","last_became_peered":"2024-04-17T10:41:37.459973+0000","last_unstale":"2024-04-17T10:41:38.433997+0000","last_undegraded":"2024-04-17T10:41:38.433997+0000","last_fullsized":"2024-04-17T10:41:38.433997+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:09:11.701377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.460580+0000","last_change":"2024-04-17T10:41:38.455618+0000","last_active":"2024-04-17T10:41:38.460580+0000","last_peered":"2024-04-17T10:41:38.460580+0000","last_clean":"2024-04-17T10:41:38.460580+0000","last_became_active":"2024-04-17T10:41:38.452995+0000","last_became_peered":"2024-04-17T10:41:38.452995+0000","last_unstale":"2024-04-17T10:41:38.460580+0000","last_undegraded":"2024-04-17T10:41:38.460580+0000","last_fullsized":"2024-04-17T10:41:38.460580+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:12:50.521087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.439438+0000","last_change":"2024-04-17T10:41:37.456004+0000","last_active":"2024-04-17T10:41:38.439438+0000","last_peered":"2024-04-17T10:41:38.439438+0000","last_clean":"2024-04-17T10:41:38.439438+0000","last_became_active":"2024-04-17T10:41:37.454546+0000","last_became_peered":"2024-04-17T10:41:37.454546+0000","last_unstale":"2024-04-17T10:41:38.439438+0000","last_undegraded":"2024-04-17T10:41:38.439438+0000","last_fullsized":"2024-04-17T10:41:38.439438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:56:16.332083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.479520+0000","last_change":"2024-04-17T10:41:38.461401+0000","last_active":"2024-04-17T10:41:38.479520+0000","last_peered":"2024-04-17T10:41:38.479520+0000","last_clean":"2024-04-17T10:41:38.479520+0000","last_became_active":"2024-04-17T10:41:38.460711+0000","last_became_peered":"2024-04-17T10:41:38.460711+0000","last_unstale":"2024-04-17T10:41:38.479520+0000","last_undegraded":"2024-04-17T10:41:38.479520+0000","last_fullsized":"2024-04-17T10:41:38.479520+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:04:11.620082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.438457+0000","last_change":"2024-04-17T10:41:37.452805+0000","last_active":"2024-04-17T10:41:38.438457+0000","last_peered":"2024-04-17T10:41:38.438457+0000","last_clean":"2024-04-17T10:41:38.438457+0000","last_became_active":"2024-04-17T10:41:37.452549+0000","last_became_peered":"2024-04-17T10:41:37.452549+0000","last_unstale":"2024-04-17T10:41:38.438457+0000","last_undegraded":"2024-04-17T10:41:38.438457+0000","last_fullsized":"2024-04-17T10:41:38.438457+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:59:15.270785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.467250+0000","last_change":"2024-04-17T10:41:38.456229+0000","last_active":"2024-04-17T10:41:38.467250+0000","last_peered":"2024-04-17T10:41:38.467250+0000","last_clean":"2024-04-17T10:41:38.467250+0000","last_became_active":"2024-04-17T10:41:38.455482+0000","last_became_peered":"2024-04-17T10:41:38.455482+0000","last_unstale":"2024-04-17T10:41:38.467250+0000","last_undegraded":"2024-04-17T10:41:38.467250+0000","last_fullsized":"2024-04-17T10:41:38.467250+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:36:33.039968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.445078+0000","last_change":"2024-04-17T10:41:37.440123+0000","last_active":"2024-04-17T10:41:38.445078+0000","last_peered":"2024-04-17T10:41:38.445078+0000","last_clean":"2024-04-17T10:41:38.445078+0000","last_became_active":"2024-04-17T10:41:37.439823+0000","last_became_peered":"2024-04-17T10:41:37.439823+0000","last_unstale":"2024-04-17T10:41:38.445078+0000","last_undegraded":"2024-04-17T10:41:38.445078+0000","last_fullsized":"2024-04-17T10:41:38.445078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:10:34.297124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.474841+0000","last_change":"2024-04-17T10:41:38.469544+0000","last_active":"2024-04-17T10:41:38.474841+0000","last_peered":"2024-04-17T10:41:38.474841+0000","last_clean":"2024-04-17T10:41:38.474841+0000","last_became_active":"2024-04-17T10:41:38.469214+0000","last_became_peered":"2024-04-17T10:41:38.469214+0000","last_unstale":"2024-04-17T10:41:38.474841+0000","last_undegraded":"2024-04-17T10:41:38.474841+0000","last_fullsized":"2024-04-17T10:41:38.474841+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:37.404638+0000","last_clean_scrub_stamp":"2024-04-17T10:41:37.404638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:56:17.340340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.479769+0000","last_change":"2024-04-17T10:41:37.446332+0000","last_active":"2024-04-17T10:41:38.479769+0000","last_peered":"2024-04-17T10:41:38.479769+0000","last_clean":"2024-04-17T10:41:38.479769+0000","last_became_active":"2024-04-17T10:41:37.445650+0000","last_became_peered":"2024-04-17T10:41:37.445650+0000","last_unstale":"2024-04-17T10:41:38.479769+0000","last_undegraded":"2024-04-17T10:41:38.479769+0000","last_fullsized":"2024-04-17T10:41:38.479769+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:41:36.396380+0000","last_clean_scrub_stamp":"2024-04-17T10:41:36.396380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:57:17.192590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":221,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T10:41:38.428746+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:41:38.428746+0000","last_peered":"2024-04-17T10:41:38.428746+0000","last_clean":"2024-04-17T10:41:38.428746+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:41:38.428746+0000","last_undegraded":"2024-04-17T10:41:38.428746+0000","last_fullsized":"2024-04-17T10:41:38.428746+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40604,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331236,"statfs":{"total":96636764160,"available":96595185664,"internally_reserved":0,"allocated":1126400,"data_stored":806307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40028,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331812,"statfs":{"total":96636764160,"available":96595775488,"internally_reserved":0,"allocated":536576,"data_stored":216521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40024,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331816,"statfs":{"total":96636764160,"available":96595779584,"internally_reserved":0,"allocated":532480,"data_stored":215919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45604,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326236,"statfs":{"total":96636764160,"available":96590065664,"internally_reserved":0,"allocated":540672,"data_stored":217461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49764,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94322076,"statfs":{"total":96636764160,"available":96585805824,"internally_reserved":0,"allocated":540672,"data_stored":216487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574895,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35872,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335968,"statfs":{"total":96636764160,"available":96600031232,"internally_reserved":0,"allocated":544768,"data_stored":217483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49748,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94322092,"statfs":{"total":96636764160,"available":96585822208,"internally_reserved":0,"allocated":532480,"data_stored":215919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40608,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331232,"statfs":{"total":96636764160,"available":96595181568,"internally_reserved":0,"allocated":1126400,"data_stored":806251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:41:41.815 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-17T10:41:41.815 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-17T10:41:41.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.983+0000 7ff1b866d640 1 Processor -- start 2024-04-17T10:41:41.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.983+0000 7ff1b866d640 1 -- start start 2024-04-17T10:41:41.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.983+0000 7ff1b866d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff1b0154670 0x7ff1b00aea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:41.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.983+0000 7ff1b866d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1b00b4090 0x7ff1b006c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:41.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.983+0000 7ff1b866d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff1b00b44f0 0x7ff1b006cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:41.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.983+0000 7ff1b866d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1b0154a50 con 0x7ff1b0154670 2024-04-17T10:41:41.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.983+0000 7ff1b866d640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1b00b2ff0 con 0x7ff1b00b44f0 2024-04-17T10:41:41.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.983+0000 7ff1b866d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b00b32d0 con 0x7ff1b00b4090 2024-04-17T10:41:41.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.983+0000 7ff1b6409640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff1b0154670 0x7ff1b00aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:41.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.983+0000 7ff1b6409640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff1b0154670 0x7ff1b00aea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51264/0 (socket says 172.21.15.83:51264) 2024-04-17T10:41:41.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.983+0000 7ff1b6409640 1 -- 172.21.15.83:0/1230114123 learned_addr learned my addr 172.21.15.83:0/1230114123 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:41.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.983+0000 7ff1b6c0a640 1 --2- 172.21.15.83:0/1230114123 >> v2:172.21.15.167:3300/0 conn(0x7ff1b00b44f0 0x7ff1b006cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:41.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.983+0000 7ff1b6409640 1 -- 172.21.15.83:0/1230114123 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1b00b4090 msgr2=0x7ff1b006c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:41.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.983+0000 7ff1b6409640 1 --2- 172.21.15.83:0/1230114123 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1b00b4090 0x7ff1b006c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:41.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.983+0000 7ff1b6409640 1 -- 172.21.15.83:0/1230114123 >> v2:172.21.15.167:3300/0 conn(0x7ff1b00b44f0 msgr2=0x7ff1b006cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:41.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.983+0000 7ff1b6409640 1 --2- 172.21.15.83:0/1230114123 >> v2:172.21.15.167:3300/0 conn(0x7ff1b00b44f0 0x7ff1b006cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:41.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.983+0000 7ff1b6409640 1 -- 172.21.15.83:0/1230114123 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b00b35b0 con 0x7ff1b0154670 2024-04-17T10:41:41.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.983+0000 7ff1b6c0a640 1 --2- 172.21.15.83:0/1230114123 >> v2:172.21.15.167:3300/0 conn(0x7ff1b00b44f0 0x7ff1b006cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:41.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.987+0000 7ff1b6409640 1 --2- 172.21.15.83:0/1230114123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1b0154670 0x7ff1b00aea50 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff1b006dc50 tx=0x7ff1ac01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4772c153e01aec85 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:41.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.987+0000 7ff1b5407640 1 -- 172.21.15.83:0/1230114123 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac0aa610 con 0x7ff1b0154670 2024-04-17T10:41:41.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.987+0000 7ff1b5407640 1 -- 172.21.15.83:0/1230114123 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1ac0af070 con 0x7ff1b0154670 2024-04-17T10:41:41.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.987+0000 7ff1b5407640 1 -- 172.21.15.83:0/1230114123 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac0aa9d0 con 0x7ff1b0154670 2024-04-17T10:41:41.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.987+0000 7ff1b866d640 1 -- 172.21.15.83:0/1230114123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1b0154670 msgr2=0x7ff1b00aea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:41.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.987+0000 7ff1b866d640 1 --2- 172.21.15.83:0/1230114123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1b0154670 0x7ff1b00aea50 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff1b006dc50 tx=0x7ff1ac01c4f0 comp rx=0 tx=0).stop 2024-04-17T10:41:41.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.987+0000 7ff1b866d640 1 -- 172.21.15.83:0/1230114123 shutdown_connections 2024-04-17T10:41:41.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.987+0000 7ff1b866d640 1 --2- 172.21.15.83:0/1230114123 >> v2:172.21.15.167:3300/0 conn(0x7ff1b00b44f0 0x7ff1b006cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:41.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.987+0000 7ff1b866d640 1 --2- 172.21.15.83:0/1230114123 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1b00b4090 0x7ff1b006c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:41.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.987+0000 7ff1b866d640 1 --2- 172.21.15.83:0/1230114123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1b0154670 0x7ff1b00aea50 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:41.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.987+0000 7ff1b866d640 1 -- 172.21.15.83:0/1230114123 >> 172.21.15.83:0/1230114123 conn(0x7ff1b00b15b0 msgr2=0x7ff1b00b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:41.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.987+0000 7ff1b866d640 1 -- 172.21.15.83:0/1230114123 shutdown_connections 2024-04-17T10:41:41.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.987+0000 7ff1b866d640 1 -- 172.21.15.83:0/1230114123 wait complete. 2024-04-17T10:41:41.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.987+0000 7ff1b866d640 1 Processor -- start 2024-04-17T10:41:41.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.991+0000 7ff1b866d640 1 -- start start 2024-04-17T10:41:41.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.991+0000 7ff1b866d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1b00b4090 0x7ff1b00aa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:41.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.991+0000 7ff1b866d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1b00b44f0 0x7ff1b013e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:41.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.991+0000 7ff1b866d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1b0154670 0x7ff1b00b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:41.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.991+0000 7ff1b866d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b00af110 con 0x7ff1b0154670 2024-04-17T10:41:41.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.991+0000 7ff1b866d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b00aef90 con 0x7ff1b00b44f0 2024-04-17T10:41:41.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.991+0000 7ff1b866d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b015db00 con 0x7ff1b00b4090 2024-04-17T10:41:41.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.991+0000 7ff1b6c0a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1b0154670 0x7ff1b00b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:41.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.991+0000 7ff1b6c0a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1b0154670 0x7ff1b00b02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51280/0 (socket says 172.21.15.83:51280) 2024-04-17T10:41:41.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.991+0000 7ff1b6c0a640 1 -- 172.21.15.83:0/1578110774 learned_addr learned my addr 172.21.15.83:0/1578110774 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:41.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.991+0000 7ff1b6c0a640 1 -- 172.21.15.83:0/1578110774 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1b00b4090 msgr2=0x7ff1b00aa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:41.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.991+0000 7ff1b6409640 1 --2- 172.21.15.83:0/1578110774 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1b00b4090 0x7ff1b00aa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:41.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.991+0000 7ff1b5c08640 1 --2- 172.21.15.83:0/1578110774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1b00b44f0 0x7ff1b013e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:41.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.991+0000 7ff1b6c0a640 1 --2- 172.21.15.83:0/1578110774 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1b00b4090 0x7ff1b00aa5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:41.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.991+0000 7ff1b6c0a640 1 -- 172.21.15.83:0/1578110774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1b00b44f0 msgr2=0x7ff1b013e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:41.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.991+0000 7ff1b6c0a640 1 --2- 172.21.15.83:0/1578110774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1b00b44f0 0x7ff1b013e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:41.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.991+0000 7ff1b6c0a640 1 -- 172.21.15.83:0/1578110774 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b013bc20 con 0x7ff1b0154670 2024-04-17T10:41:41.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.991+0000 7ff1b5c08640 1 --2- 172.21.15.83:0/1578110774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1b00b44f0 0x7ff1b013e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:41.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.991+0000 7ff1b6c0a640 1 --2- 172.21.15.83:0/1578110774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1b0154670 0x7ff1b00b02a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff1a800cea0 tx=0x7ff1a800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:41.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.991+0000 7ff1a6ffd640 1 -- 172.21.15.83:0/1578110774 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8003ba0 con 0x7ff1b0154670 2024-04-17T10:41:41.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.991+0000 7ff1a6ffd640 1 -- 172.21.15.83:0/1578110774 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a8003d60 con 0x7ff1b0154670 2024-04-17T10:41:41.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.991+0000 7ff1b866d640 1 -- 172.21.15.83:0/1578110774 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b013bf00 con 0x7ff1b0154670 2024-04-17T10:41:41.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.991+0000 7ff1a6ffd640 1 -- 172.21.15.83:0/1578110774 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a800d880 con 0x7ff1b0154670 2024-04-17T10:41:41.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.991+0000 7ff1b866d640 1 -- 172.21.15.83:0/1578110774 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b013c4c0 con 0x7ff1b0154670 2024-04-17T10:41:41.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.991+0000 7ff1b866d640 1 -- 172.21.15.83:0/1578110774 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1b0074f90 con 0x7ff1b0154670 2024-04-17T10:41:41.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.991+0000 7ff1a6ffd640 1 -- 172.21.15.83:0/1578110774 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff1a800da20 con 0x7ff1b0154670 2024-04-17T10:41:41.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.991+0000 7ff1a6ffd640 1 --2- 172.21.15.83:0/1578110774 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff194066f60 0x7ff194069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:41.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.991+0000 7ff1a6ffd640 1 -- 172.21.15.83:0/1578110774 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff1a80859b0 con 0x7ff1b0154670 2024-04-17T10:41:41.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.991+0000 7ff1b6409640 1 --2- 172.21.15.83:0/1578110774 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff194066f60 0x7ff194069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.995+0000 7ff1b6409640 1 --2- 172.21.15.83:0/1578110774 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff194066f60 0x7ff194069420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac0ae000 tx=0x7ff1ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:41.995+0000 7ff1a6ffd640 1 -- 172.21.15.83:0/1578110774 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff1a8056a50 con 0x7ff1b0154670 2024-04-17T10:41:42.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.199+0000 7ff1b866d640 1 -- 172.21.15.83:0/1578110774 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff1b0153b20 con 0x7ff1b0154670 2024-04-17T10:41:42.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.199+0000 7ff1a6ffd640 1 -- 172.21.15.83:0/1578110774 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) v1 ==== 190+0+147 (secure 0 0 0) 0x7ff1a8056350 con 0x7ff1b0154670 2024-04-17T10:41:42.201 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-04-17T10:41:42.202 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBUph9mHGmXGBAAZF+JiAG9YrFtbW+dCy2IEQ== 2024-04-17T10:41:42.202 INFO:teuthology.orchestra.run.smithi083.stdout: caps mds = "allow" 2024-04-17T10:41:42.202 INFO:teuthology.orchestra.run.smithi083.stdout: caps mon = "allow r" 2024-04-17T10:41:42.202 INFO:teuthology.orchestra.run.smithi083.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-17T10:41:42.206 INFO:teuthology.orchestra.run.smithi083.stderr:updated caps for client.0 2024-04-17T10:41:42.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.203+0000 7ff1b866d640 1 -- 172.21.15.83:0/1578110774 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff194066f60 msgr2=0x7ff194069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:42.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.203+0000 7ff1b866d640 1 --2- 172.21.15.83:0/1578110774 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff194066f60 0x7ff194069420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac0ae000 tx=0x7ff1ac008040 comp rx=0 tx=0).stop 2024-04-17T10:41:42.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.203+0000 7ff1b866d640 1 -- 172.21.15.83:0/1578110774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1b0154670 msgr2=0x7ff1b00b02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:42.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.203+0000 7ff1b866d640 1 --2- 172.21.15.83:0/1578110774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1b0154670 0x7ff1b00b02a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff1a800cea0 tx=0x7ff1a800cf90 comp rx=0 tx=0).stop 2024-04-17T10:41:42.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.203+0000 7ff1b866d640 1 -- 172.21.15.83:0/1578110774 shutdown_connections 2024-04-17T10:41:42.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.203+0000 7ff1b866d640 1 --2- 172.21.15.83:0/1578110774 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff194066f60 0x7ff194069420 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:42.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.203+0000 7ff1b866d640 1 --2- 172.21.15.83:0/1578110774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1b0154670 0x7ff1b00b02a0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:42.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.203+0000 7ff1b866d640 1 --2- 172.21.15.83:0/1578110774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1b00b44f0 0x7ff1b013e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:42.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.203+0000 7ff1b866d640 1 --2- 172.21.15.83:0/1578110774 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1b00b4090 0x7ff1b00aa5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:42.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.203+0000 7ff1b866d640 1 -- 172.21.15.83:0/1578110774 >> 172.21.15.83:0/1578110774 conn(0x7ff1b00b15b0 msgr2=0x7ff1b014fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:42.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.203+0000 7ff1b866d640 1 -- 172.21.15.83:0/1578110774 shutdown_connections 2024-04-17T10:41:42.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.203+0000 7ff1b866d640 1 -- 172.21.15.83:0/1578110774 wait complete. 2024-04-17T10:41:42.221 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:41:42.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.339+0000 7fc5fdd44640 1 Processor -- start 2024-04-17T10:41:42.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.339+0000 7fc5fdd44640 1 -- start start 2024-04-17T10:41:42.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.339+0000 7fc5fdd44640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc5f815dfa0 0x7fc5f80b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:42.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.339+0000 7fc5fdd44640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5f80ce980 0x7fc5f80b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:42.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.339+0000 7fc5fdd44640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f8162a00 con 0x7fc5f80cee00 2024-04-17T10:41:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.339+0000 7fc5fdd44640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f8161e80 con 0x7fc5f815dfa0 2024-04-17T10:41:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.339+0000 7fc5fdd44640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f8162160 con 0x7fc5f80ce980 2024-04-17T10:41:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.339+0000 7fc5f77fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc5f815dfa0 0x7fc5f80b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.339+0000 7fc5f77fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc5f815dfa0 0x7fc5f80b6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51288/0 (socket says 172.21.15.83:51288) 2024-04-17T10:41:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.339+0000 7fc5f77fe640 1 -- 172.21.15.83:0/2956269883 learned_addr learned my addr 172.21.15.83:0/2956269883 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.339+0000 7fc5f77fe640 1 -- 172.21.15.83:0/2956269883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5f80ce980 msgr2=0x7fc5f80b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.339+0000 7fc5f77fe640 1 --2- 172.21.15.83:0/2956269883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5f80ce980 0x7fc5f80b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.339+0000 7fc5f77fe640 1 -- 172.21.15.83:0/2956269883 >> v1:172.21.15.167:6789/0 conn(0x7fc5f80cee00 legacy=0x7fc5f80d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.339+0000 7fc5f77fe640 1 -- 172.21.15.83:0/2956269883 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f8162440 con 0x7fc5f815dfa0 2024-04-17T10:41:42.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.343+0000 7fc5f77fe640 1 --2- 172.21.15.83:0/2956269883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5f815dfa0 0x7fc5f80b6c20 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8009870 tx=0x7fc5e801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ddd0671fc64c2c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:42.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.343+0000 7fc5f67fc640 1 -- 172.21.15.83:0/2956269883 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e80b0070 con 0x7fc5f815dfa0 2024-04-17T10:41:42.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.343+0000 7fc5f67fc640 1 -- 172.21.15.83:0/2956269883 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e80ab600 con 0x7fc5f815dfa0 2024-04-17T10:41:42.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.343+0000 7fc5f67fc640 1 -- 172.21.15.83:0/2956269883 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e80ab9d0 con 0x7fc5f815dfa0 2024-04-17T10:41:42.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.343+0000 7fc5fdd44640 1 -- 172.21.15.83:0/2956269883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5f815dfa0 msgr2=0x7fc5f80b6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:42.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.343+0000 7fc5fdd44640 1 --2- 172.21.15.83:0/2956269883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5f815dfa0 0x7fc5f80b6c20 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8009870 tx=0x7fc5e801c460 comp rx=0 tx=0).stop 2024-04-17T10:41:42.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.343+0000 7fc5fdd44640 1 -- 172.21.15.83:0/2956269883 shutdown_connections 2024-04-17T10:41:42.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.343+0000 7fc5fdd44640 1 --2- 172.21.15.83:0/2956269883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5f80ce980 0x7fc5f80b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:42.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.343+0000 7fc5fdd44640 1 --2- 172.21.15.83:0/2956269883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5f815dfa0 0x7fc5f80b6c20 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:42.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.343+0000 7fc5fdd44640 1 -- 172.21.15.83:0/2956269883 >> 172.21.15.83:0/2956269883 conn(0x7fc5f80d0600 msgr2=0x7fc5f80d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:42.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.343+0000 7fc5fdd44640 1 -- 172.21.15.83:0/2956269883 shutdown_connections 2024-04-17T10:41:42.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.343+0000 7fc5fdd44640 1 -- 172.21.15.83:0/2956269883 wait complete. 2024-04-17T10:41:42.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.343+0000 7fc5fdd44640 1 Processor -- start 2024-04-17T10:41:42.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.343+0000 7fc5fdd44640 1 -- start start 2024-04-17T10:41:42.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.343+0000 7fc5fdd44640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5f80ce980 0x7fc5f8153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:42.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.343+0000 7fc5fdd44640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5f80cee00 0x7fc5f8152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:42.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.343+0000 7fc5fdd44640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5f815dfa0 0x7fc5f815c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:42.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.343+0000 7fc5fdd44640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f8161b10 con 0x7fc5f80ce980 2024-04-17T10:41:42.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.343+0000 7fc5fdd44640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f80d2850 con 0x7fc5f815dfa0 2024-04-17T10:41:42.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.343+0000 7fc5fdd44640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f816b660 con 0x7fc5f80cee00 2024-04-17T10:41:42.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.343+0000 7fc5f77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5f80ce980 0x7fc5f8153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:42.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.343+0000 7fc5f77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5f80ce980 0x7fc5f8153cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51302/0 (socket says 172.21.15.83:51302) 2024-04-17T10:41:42.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.343+0000 7fc5f77fe640 1 -- 172.21.15.83:0/1993890343 learned_addr learned my addr 172.21.15.83:0/1993890343 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:42.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.347+0000 7fc5f7fff640 1 --2- 172.21.15.83:0/1993890343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5f815dfa0 0x7fc5f815c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:42.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.347+0000 7fc5f6ffd640 1 --2- 172.21.15.83:0/1993890343 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5f80cee00 0x7fc5f8152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:42.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.347+0000 7fc5f77fe640 1 -- 172.21.15.83:0/1993890343 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5f80cee00 msgr2=0x7fc5f8152130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:42.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.347+0000 7fc5f77fe640 1 --2- 172.21.15.83:0/1993890343 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5f80cee00 0x7fc5f8152130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:42.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.347+0000 7fc5f77fe640 1 -- 172.21.15.83:0/1993890343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5f815dfa0 msgr2=0x7fc5f815c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:42.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.347+0000 7fc5f77fe640 1 --2- 172.21.15.83:0/1993890343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5f815dfa0 0x7fc5f815c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:42.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.347+0000 7fc5f77fe640 1 -- 172.21.15.83:0/1993890343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f8159cf0 con 0x7fc5f80ce980 2024-04-17T10:41:42.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.347+0000 7fc5f7fff640 1 --2- 172.21.15.83:0/1993890343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5f815dfa0 0x7fc5f815c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:42.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.347+0000 7fc5f77fe640 1 --2- 172.21.15.83:0/1993890343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5f80ce980 0x7fc5f8153cb0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc5e801ab10 tx=0x7fc5e8019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:42.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.347+0000 7fc5dffff640 1 -- 172.21.15.83:0/1993890343 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e80b0040 con 0x7fc5f80ce980 2024-04-17T10:41:42.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.347+0000 7fc5dffff640 1 -- 172.21.15.83:0/1993890343 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e80b9070 con 0x7fc5f80ce980 2024-04-17T10:41:42.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.347+0000 7fc5fdd44640 1 -- 172.21.15.83:0/1993890343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f8159fd0 con 0x7fc5f80ce980 2024-04-17T10:41:42.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.347+0000 7fc5dffff640 1 -- 172.21.15.83:0/1993890343 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e80b4880 con 0x7fc5f80ce980 2024-04-17T10:41:42.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.347+0000 7fc5fdd44640 1 -- 172.21.15.83:0/1993890343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f8153660 con 0x7fc5f80ce980 2024-04-17T10:41:42.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.347+0000 7fc5fdd44640 1 -- 172.21.15.83:0/1993890343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f80681a0 con 0x7fc5f80ce980 2024-04-17T10:41:42.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.347+0000 7fc5dffff640 1 -- 172.21.15.83:0/1993890343 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc5e80ab4c0 con 0x7fc5f80ce980 2024-04-17T10:41:42.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.347+0000 7fc5dffff640 1 --2- 172.21.15.83:0/1993890343 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc5e4066f60 0x7fc5e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:42.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.347+0000 7fc5dffff640 1 -- 172.21.15.83:0/1993890343 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5e811ede0 con 0x7fc5f80ce980 2024-04-17T10:41:42.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.347+0000 7fc5f6ffd640 1 --2- 172.21.15.83:0/1993890343 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc5e4066f60 0x7fc5e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:42.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.347+0000 7fc5f6ffd640 1 --2- 172.21.15.83:0/1993890343 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc5e4066f60 0x7fc5e4069420 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8159ab0 tx=0x7fc5ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:42.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.351+0000 7fc5dffff640 1 -- 172.21.15.83:0/1993890343 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc5e8016110 con 0x7fc5f80ce980 2024-04-17T10:41:42.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.531+0000 7fc5fdd44640 1 -- 172.21.15.83:0/1993890343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5f8074f90 con 0x7fc5f80ce980 2024-04-17T10:41:42.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.535+0000 7fc5dffff640 1 -- 172.21.15.83:0/1993890343 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4938 (secure 0 0 0) 0x7fc5e80efe60 con 0x7fc5f80ce980 2024-04-17T10:41:42.531 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:41:42.531 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/161398126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":161398126},{"type":"v1","addr":"172.21.15.167:6833","nonce":161398126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1085380618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1085380618},{"type":"v1","addr":"172.21.15.167:6835","nonce":1085380618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/836442291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":836442291},{"type":"v1","addr":"172.21.15.83:6835","nonce":836442291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:41:37.773342+0000","modified":"2024-04-17T10:41:39.788394+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5682":{"gid":5682,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1894547386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1894547386},{"type":"v1","addr":"172.21.15.83:6837","nonce":1894547386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-17T10:41:42.532 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 21 2024-04-17T10:41:42.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.535+0000 7fc5fdd44640 1 -- 172.21.15.83:0/1993890343 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc5e4066f60 msgr2=0x7fc5e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:42.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.535+0000 7fc5fdd44640 1 --2- 172.21.15.83:0/1993890343 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc5e4066f60 0x7fc5e4069420 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8159ab0 tx=0x7fc5ec008040 comp rx=0 tx=0).stop 2024-04-17T10:41:42.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.535+0000 7fc5fdd44640 1 -- 172.21.15.83:0/1993890343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5f80ce980 msgr2=0x7fc5f8153cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:42.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.535+0000 7fc5fdd44640 1 --2- 172.21.15.83:0/1993890343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5f80ce980 0x7fc5f8153cb0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc5e801ab10 tx=0x7fc5e8019200 comp rx=0 tx=0).stop 2024-04-17T10:41:42.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.535+0000 7fc5fdd44640 1 -- 172.21.15.83:0/1993890343 shutdown_connections 2024-04-17T10:41:42.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.535+0000 7fc5fdd44640 1 --2- 172.21.15.83:0/1993890343 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc5e4066f60 0x7fc5e4069420 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:42.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.535+0000 7fc5fdd44640 1 --2- 172.21.15.83:0/1993890343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5f815dfa0 0x7fc5f815c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:42.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.535+0000 7fc5fdd44640 1 --2- 172.21.15.83:0/1993890343 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5f80cee00 0x7fc5f8152130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:42.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.535+0000 7fc5fdd44640 1 --2- 172.21.15.83:0/1993890343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5f80ce980 0x7fc5f8153cb0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:42.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.535+0000 7fc5fdd44640 1 -- 172.21.15.83:0/1993890343 >> 172.21.15.83:0/1993890343 conn(0x7fc5f80d0600 msgr2=0x7fc5f80d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:42.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.535+0000 7fc5fdd44640 1 -- 172.21.15.83:0/1993890343 shutdown_connections 2024-04-17T10:41:42.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:42.535+0000 7fc5fdd44640 1 -- 172.21.15.83:0/1993890343 wait complete. 2024-04-17T10:41:42.951 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T10:41:37.773342+0000', 'modified': '2024-04-17T10:41:39.788394+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5682}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5682': {'gid': 5682, 'name': 'c', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6837/1894547386', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6836', 'nonce': 1894547386}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 1894547386}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:41:42.952 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T10:41:42.952 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-17T10:41:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.087+0000 7f86e15a0640 1 Processor -- start 2024-04-17T10:41:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.087+0000 7f86e15a0640 1 -- start start 2024-04-17T10:41:43.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.087+0000 7f86e15a0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86dc15bac0 0x7f86dc071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:43.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.087+0000 7f86e15a0640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86dc06c1f0 con 0x7f86dc070470 2024-04-17T10:41:43.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.087+0000 7f86e15a0640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86dc070a10 con 0x7f86dc15b6e0 2024-04-17T10:41:43.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.087+0000 7f86e15a0640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc070cf0 con 0x7f86dc15bac0 2024-04-17T10:41:43.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.087+0000 7f86db7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f86dc070470 0x7f86dc06ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:36898/0 (socket says 172.21.15.83:36898) 2024-04-17T10:41:43.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.087+0000 7f86db7fe640 1 -- 172.21.15.83:0/2997698185 learned_addr learned my addr 172.21.15.83:0/2997698185 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:43.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.087+0000 7f86da7fc640 1 --2- 172.21.15.83:0/2997698185 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86dc15bac0 0x7f86dc071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:43.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.087+0000 7f86d9ffb640 1 -- 172.21.15.83:0/2997698185 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2575413837 0 0) 0x7f86dc06c1f0 con 0x7f86dc070470 2024-04-17T10:41:43.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.087+0000 7f86d9ffb640 1 -- 172.21.15.83:0/2997698185 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86c4003530 con 0x7f86dc070470 2024-04-17T10:41:43.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.087+0000 7f86d9ffb640 1 -- 172.21.15.83:0/2997698185 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3387683629 0 0) 0x7f86c4003530 con 0x7f86dc070470 2024-04-17T10:41:43.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.087+0000 7f86d9ffb640 1 -- 172.21.15.83:0/2997698185 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86dc15bac0 msgr2=0x7f86dc071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:43.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.087+0000 7f86d9ffb640 1 --2- 172.21.15.83:0/2997698185 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86dc15bac0 0x7f86dc071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.087+0000 7f86d9ffb640 1 -- 172.21.15.83:0/2997698185 >> v1:172.21.15.167:6789/0 conn(0x7f86dc15b6e0 legacy=0x7f86dc0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:43.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.087+0000 7f86d9ffb640 1 -- 172.21.15.83:0/2997698185 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86dc070fd0 con 0x7f86dc070470 2024-04-17T10:41:43.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.087+0000 7f86d9ffb640 1 -- 172.21.15.83:0/2997698185 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f86d4001a00 con 0x7f86dc070470 2024-04-17T10:41:43.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.087+0000 7f86d9ffb640 1 -- 172.21.15.83:0/2997698185 >> v1:172.21.15.83:6789/0 conn(0x7f86dc070470 legacy=0x7f86dc06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:43.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.087+0000 7f86d9ffb640 1 --2- 172.21.15.83:0/2997698185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86c4003a60 0x7f86c4003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:43.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.087+0000 7f86d9ffb640 1 --2- 172.21.15.83:0/2997698185 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86dc15bac0 0x7f86c4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:43.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.087+0000 7f86d9ffb640 1 --2- 172.21.15.83:0/2997698185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86c4004d50 0x7f86c40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:43.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.087+0000 7f86d9ffb640 1 -- 172.21.15.83:0/2997698185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc070cf0 con 0x7f86c4004d50 2024-04-17T10:41:43.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.087+0000 7f86d9ffb640 1 -- 172.21.15.83:0/2997698185 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c400bae0 con 0x7f86c4003a60 2024-04-17T10:41:43.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.087+0000 7f86d9ffb640 1 -- 172.21.15.83:0/2997698185 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c400bce0 con 0x7f86dc15bac0 2024-04-17T10:41:43.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.087+0000 7f86db7fe640 1 --2- 172.21.15.83:0/2997698185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86c4004d50 0x7f86c40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:43.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.087+0000 7f86daffd640 1 --2- 172.21.15.83:0/2997698185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86c4003a60 0x7f86c4003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:43.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.087+0000 7f86da7fc640 1 --2- 172.21.15.83:0/2997698185 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86dc15bac0 0x7f86c4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:43.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.091+0000 7f86db7fe640 1 -- 172.21.15.83:0/2997698185 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86dc15bac0 msgr2=0x7f86c4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:43.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.091+0000 7f86db7fe640 1 --2- 172.21.15.83:0/2997698185 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86dc15bac0 0x7f86c4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.091+0000 7f86db7fe640 1 -- 172.21.15.83:0/2997698185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86c4003a60 msgr2=0x7f86c4003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:43.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.091+0000 7f86db7fe640 1 --2- 172.21.15.83:0/2997698185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86c4003a60 0x7f86c4003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.091+0000 7f86db7fe640 1 -- 172.21.15.83:0/2997698185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c4003530 con 0x7f86c4004d50 2024-04-17T10:41:43.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.091+0000 7f86da7fc640 1 --2- 172.21.15.83:0/2997698185 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86dc15bac0 0x7f86c4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:41:43.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.091+0000 7f86db7fe640 1 --2- 172.21.15.83:0/2997698185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86c4004d50 0x7f86c40051d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f86d400a830 tx=0x7f86d401d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9446460a522a022c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:43.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.091+0000 7f86d9ffb640 1 -- 172.21.15.83:0/2997698185 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d400a480 con 0x7f86c4004d50 2024-04-17T10:41:43.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.091+0000 7f86d9ffb640 1 -- 172.21.15.83:0/2997698185 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d40ad630 con 0x7f86c4004d50 2024-04-17T10:41:43.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.091+0000 7f86d9ffb640 1 -- 172.21.15.83:0/2997698185 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d40ada60 con 0x7f86c4004d50 2024-04-17T10:41:43.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.091+0000 7f86e15a0640 1 -- 172.21.15.83:0/2997698185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86c4004d50 msgr2=0x7f86c40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:43.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.091+0000 7f86e15a0640 1 --2- 172.21.15.83:0/2997698185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86c4004d50 0x7f86c40051d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f86d400a830 tx=0x7f86d401d360 comp rx=0 tx=0).stop 2024-04-17T10:41:43.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.091+0000 7f86db7fe640 1 -- 172.21.15.83:0/2997698185 reap_dead start 2024-04-17T10:41:43.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.091+0000 7f86e15a0640 1 -- 172.21.15.83:0/2997698185 shutdown_connections 2024-04-17T10:41:43.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.091+0000 7f86e15a0640 1 -- 172.21.15.83:0/2997698185 >> 172.21.15.83:0/2997698185 conn(0x7f86dc0b15b0 msgr2=0x7f86dc0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:43.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.091+0000 7f86e15a0640 1 -- 172.21.15.83:0/2997698185 shutdown_connections 2024-04-17T10:41:43.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.091+0000 7f86e15a0640 1 -- 172.21.15.83:0/2997698185 wait complete. 2024-04-17T10:41:43.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.095+0000 7f86e15a0640 1 Processor -- start 2024-04-17T10:41:43.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.095+0000 7f86e15a0640 1 -- start start 2024-04-17T10:41:43.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.095+0000 7f86e15a0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86dc143560 0x7f86dc143e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:43.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.095+0000 7f86e15a0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86dc157610 0x7f86dc15e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:43.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.095+0000 7f86e15a0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86dc0b0740 0x7f86dc0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:43.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.095+0000 7f86e15a0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc06b6a0 con 0x7f86dc143560 2024-04-17T10:41:43.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.095+0000 7f86e15a0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc06b520 con 0x7f86dc157610 2024-04-17T10:41:43.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.095+0000 7f86e15a0640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc071cc0 con 0x7f86dc0b0740 2024-04-17T10:41:43.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.095+0000 7f86daffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86dc143560 0x7f86dc143e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:43.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.095+0000 7f86daffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86dc143560 0x7f86dc143e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51320/0 (socket says 172.21.15.83:51320) 2024-04-17T10:41:43.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.095+0000 7f86daffd640 1 -- 172.21.15.83:0/1928375967 learned_addr learned my addr 172.21.15.83:0/1928375967 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:43.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.095+0000 7f86da7fc640 1 --2- 172.21.15.83:0/1928375967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86dc157610 0x7f86dc15e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:43.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.095+0000 7f86db7fe640 1 --2- 172.21.15.83:0/1928375967 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86dc0b0740 0x7f86dc0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:43.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.095+0000 7f86daffd640 1 -- 172.21.15.83:0/1928375967 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86dc0b0740 msgr2=0x7f86dc0b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:43.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.095+0000 7f86daffd640 1 --2- 172.21.15.83:0/1928375967 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86dc0b0740 0x7f86dc0b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.095+0000 7f86daffd640 1 -- 172.21.15.83:0/1928375967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86dc157610 msgr2=0x7f86dc15e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:43.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.095+0000 7f86daffd640 1 --2- 172.21.15.83:0/1928375967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86dc157610 0x7f86dc15e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.095+0000 7f86daffd640 1 -- 172.21.15.83:0/1928375967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86dc144370 con 0x7f86dc143560 2024-04-17T10:41:43.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.095+0000 7f86daffd640 1 --2- 172.21.15.83:0/1928375967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86dc143560 0x7f86dc143e30 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00f820 tx=0x7f86cc00fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:43.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.095+0000 7f86c37fe640 1 -- 172.21.15.83:0/1928375967 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc001cf0 con 0x7f86dc143560 2024-04-17T10:41:43.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.095+0000 7f86c37fe640 1 -- 172.21.15.83:0/1928375967 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86cc018070 con 0x7f86dc143560 2024-04-17T10:41:43.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.095+0000 7f86e15a0640 1 -- 172.21.15.83:0/1928375967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86dc157aa0 con 0x7f86dc143560 2024-04-17T10:41:43.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.095+0000 7f86c37fe640 1 -- 172.21.15.83:0/1928375967 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc00d730 con 0x7f86dc143560 2024-04-17T10:41:43.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.095+0000 7f86e15a0640 1 -- 172.21.15.83:0/1928375967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86dc158060 con 0x7f86dc143560 2024-04-17T10:41:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86c37fe640 1 -- 172.21.15.83:0/1928375967 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f86cc00d8d0 con 0x7f86dc143560 2024-04-17T10:41:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86c37fe640 1 --2- 172.21.15.83:0/1928375967 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f86c8066f60 0x7f86c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86c37fe640 1 -- 172.21.15.83:0/1928375967 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86cc084e40 con 0x7f86dc143560 2024-04-17T10:41:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86da7fc640 1 --2- 172.21.15.83:0/1928375967 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f86c8066f60 0x7f86c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86da7fc640 1 --2- 172.21.15.83:0/1928375967 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f86c8066f60 0x7f86c8069420 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f86dc1eb480 tx=0x7f86d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86e470d000 1 Processor -- start 2024-04-17T10:41:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86e470d000 1 -- start start 2024-04-17T10:41:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86e470d000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557242b96920 0x557242b96d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86e470d000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557242b97240 0x557242b4c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86e470d000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557242b4cf30 0x557242b4d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86e470d000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557242a345f0 con 0x557242b4cf30 2024-04-17T10:41:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86e470d000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55724299bda0 con 0x557242b97240 2024-04-17T10:41:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86e470d000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x557242924150 con 0x557242b96920 2024-04-17T10:41:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86db7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557242b4cf30 0x557242b4d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86db7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557242b4cf30 0x557242b4d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51332/0 (socket says 172.21.15.83:51332) 2024-04-17T10:41:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86db7fe640 1 -- 172.21.15.83:0/1409261570 learned_addr learned my addr 172.21.15.83:0/1409261570 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86da7fc640 1 --2- 172.21.15.83:0/1409261570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557242b97240 0x557242b4c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86db7fe640 1 -- 172.21.15.83:0/1409261570 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557242b96920 msgr2=0x557242b96d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86daffd640 1 --2- 172.21.15.83:0/1409261570 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557242b96920 0x557242b96d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86db7fe640 1 --2- 172.21.15.83:0/1409261570 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557242b96920 0x557242b96d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86db7fe640 1 -- 172.21.15.83:0/1409261570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557242b97240 msgr2=0x557242b4c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86db7fe640 1 --2- 172.21.15.83:0/1409261570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557242b97240 0x557242b4c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86db7fe640 1 -- 172.21.15.83:0/1409261570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55724293dca0 con 0x557242b4cf30 2024-04-17T10:41:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86da7fc640 1 --2- 172.21.15.83:0/1409261570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557242b97240 0x557242b4c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86db7fe640 1 --2- 172.21.15.83:0/1409261570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557242b4cf30 0x557242b4d310 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f86d401d190 tx=0x7f86dc070be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595e883fb50d84c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86c0ff9640 1 -- 172.21.15.83:0/1409261570 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d40011d0 con 0x557242b4cf30 2024-04-17T10:41:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86c0ff9640 1 -- 172.21.15.83:0/1409261570 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d400a480 con 0x557242b4cf30 2024-04-17T10:41:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86c0ff9640 1 -- 172.21.15.83:0/1409261570 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d40019c0 con 0x557242b4cf30 2024-04-17T10:41:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86e470d000 1 -- 172.21.15.83:0/1409261570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557242b4cf30 msgr2=0x557242b4d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86e470d000 1 --2- 172.21.15.83:0/1409261570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557242b4cf30 0x557242b4d310 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f86d401d190 tx=0x7f86dc070be0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86e470d000 1 -- 172.21.15.83:0/1409261570 shutdown_connections 2024-04-17T10:41:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86e470d000 1 --2- 172.21.15.83:0/1409261570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557242b4cf30 0x557242b4d310 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86e470d000 1 --2- 172.21.15.83:0/1409261570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557242b97240 0x557242b4c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86e470d000 1 --2- 172.21.15.83:0/1409261570 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557242b96920 0x557242b96d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.099+0000 7f86e470d000 1 -- 172.21.15.83:0/1409261570 >> 172.21.15.83:0/1409261570 conn(0x557242b532a0 msgr2=0x557242b4b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:43.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86e470d000 1 -- 172.21.15.83:0/1409261570 shutdown_connections 2024-04-17T10:41:43.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86e470d000 1 -- 172.21.15.83:0/1409261570 wait complete. 2024-04-17T10:41:43.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86e470d000 1 -- start start 2024-04-17T10:41:43.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86e470d000 1 Processor -- start 2024-04-17T10:41:43.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86e470d000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557242b4cf30 0x557242b9dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:43.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86e470d000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557242b96920 0x557242b9e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:43.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86e470d000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557242b97240 0x557242b50aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:43.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86e470d000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557242956990 con 0x557242b96920 2024-04-17T10:41:43.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86e470d000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557242997e40 con 0x557242b4cf30 2024-04-17T10:41:43.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86e470d000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x557242927900 con 0x557242b97240 2024-04-17T10:41:43.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86e470d000 10 client.0 _ll_register_callbacks cb 0x557242a4f1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:41:43.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86da7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557242b96920 0x557242b9e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86da7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557242b96920 0x557242b9e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51334/0 (socket says 172.21.15.83:51334) 2024-04-17T10:41:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86da7fc640 1 -- 172.21.15.83:0/4083060160 learned_addr learned my addr 172.21.15.83:0/4083060160 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86daffd640 1 --2- 172.21.15.83:0/4083060160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557242b4cf30 0x557242b9dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86db7fe640 1 --2- 172.21.15.83:0/4083060160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557242b97240 0x557242b50aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86da7fc640 1 -- 172.21.15.83:0/4083060160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557242b97240 msgr2=0x557242b50aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86da7fc640 1 --2- 172.21.15.83:0/4083060160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557242b97240 0x557242b50aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86da7fc640 1 -- 172.21.15.83:0/4083060160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557242b4cf30 msgr2=0x557242b9dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86da7fc640 1 --2- 172.21.15.83:0/4083060160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557242b4cf30 0x557242b9dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86da7fc640 1 -- 172.21.15.83:0/4083060160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557242954630 con 0x557242b96920 2024-04-17T10:41:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86daffd640 1 --2- 172.21.15.83:0/4083060160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557242b4cf30 0x557242b9dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86da7fc640 1 --2- 172.21.15.83:0/4083060160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557242b96920 0x557242b9e1c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f86d0013ea0 tx=0x7f86d0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2649be98ce0522c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:41:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86b9ffb640 1 -- 172.21.15.83:0/4083060160 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d0027070 con 0x557242b96920 2024-04-17T10:41:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86b9ffb640 1 -- 172.21.15.83:0/4083060160 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86d0023820 con 0x557242b96920 2024-04-17T10:41:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86e470d000 1 -- 172.21.15.83:0/4083060160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86d400a480 con 0x557242b96920 2024-04-17T10:41:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86b9ffb640 1 -- 172.21.15.83:0/4083060160 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86d0032050 con 0x557242b96920 2024-04-17T10:41:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86e470d000 10 client.5826 fetch_fsmap learned FSMap version 21 2024-04-17T10:41:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86e470d000 1 -- 172.21.15.83:0/4083060160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x5572429243a0 con 0x557242b96920 2024-04-17T10:41:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86b9ffb640 1 -- 172.21.15.83:0/4083060160 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d400a480 con 0x557242b96920 2024-04-17T10:41:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86b9ffb640 1 -- 172.21.15.83:0/4083060160 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f86d0035020 con 0x557242b96920 2024-04-17T10:41:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86e470d000 10 client.5826 fetch_fsmap finished waiting for FSMap version 21 2024-04-17T10:41:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86e470d000 10 client.5826 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-17T10:41:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86e470d000 20 client.5826 populate_metadata read hostname 'smithi083' 2024-04-17T10:41:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86e470d000 1 --2- 172.21.15.83:0/4083060160 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x557242b514b0 0x557242b51890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86e470d000 4 client.5826 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:41:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86e470d000 1 -- 172.21.15.83:0/4083060160 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557242937a50 con 0x557242b514b0 2024-04-17T10:41:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86daffd640 1 --2- 172.21.15.83:0/4083060160 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x557242b514b0 0x557242b51890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86b9ffb640 10 client.5826 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:41:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86daffd640 1 --2- 172.21.15.83:0/4083060160 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x557242b514b0 0x557242b51890 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33402f9328e6b15c server_cookie=79ece6f5f8ac6a6b in_seq=0 out_seq=0 2024-04-17T10:41:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86b9ffb640 1 -- 172.21.15.83:0/4083060160 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557242937a50 con 0x557242b514b0 2024-04-17T10:41:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86b9ffb640 10 client.5826 handle_command_reply: tid=0 2024-04-17T10:41:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.103+0000 7f86e470d000 1 client.5826 shutdown 2024-04-17T10:41:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.107+0000 7f86e470d000 1 -- 172.21.15.83:0/4083060160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557242b96920 msgr2=0x557242b9e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.107+0000 7f86e470d000 1 --2- 172.21.15.83:0/4083060160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557242b96920 0x557242b9e1c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f86d0013ea0 tx=0x7f86d0004740 comp rx=0 tx=0).stop 2024-04-17T10:41:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.107+0000 7f86e470d000 1 -- 172.21.15.83:0/4083060160 shutdown_connections 2024-04-17T10:41:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.107+0000 7f86e470d000 1 --2- 172.21.15.83:0/4083060160 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x557242b514b0 0x557242b51890 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.107+0000 7f86e470d000 1 --2- 172.21.15.83:0/4083060160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557242b97240 0x557242b50aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.107+0000 7f86e470d000 1 --2- 172.21.15.83:0/4083060160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557242b96920 0x557242b9e1c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.107+0000 7f86e470d000 1 --2- 172.21.15.83:0/4083060160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557242b4cf30 0x557242b9dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.107+0000 7f86b9ffb640 0 client.5826 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:41:43.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.107+0000 7f86e470d000 1 -- 172.21.15.83:0/4083060160 >> 172.21.15.83:0/4083060160 conn(0x557242b532a0 msgr2=0x557242b4b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:43.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.107+0000 7f86e470d000 1 -- 172.21.15.83:0/4083060160 shutdown_connections 2024-04-17T10:41:43.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.107+0000 7f86e470d000 1 -- 172.21.15.83:0/4083060160 wait complete. 2024-04-17T10:41:43.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.107+0000 7f86e470d000 20 client.5826 trim_cache size 0 max 16384 2024-04-17T10:41:43.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.119+0000 7f86e470d000 1 Processor -- start 2024-04-17T10:41:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.119+0000 7f86e470d000 1 -- start start 2024-04-17T10:41:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86e470d000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557242b4cf30 0x557242b50330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86e470d000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557242b96920 0x557242b50870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86e470d000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557242b97240 0x557242ba08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86e470d000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc1ffa60 con 0x557242b4cf30 2024-04-17T10:41:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86e470d000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc15cc70 con 0x557242b96920 2024-04-17T10:41:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86e470d000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc1ff460 con 0x557242b97240 2024-04-17T10:41:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86daffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557242b4cf30 0x557242b50330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86daffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557242b4cf30 0x557242b50330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51342/0 (socket says 172.21.15.83:51342) 2024-04-17T10:41:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86daffd640 1 -- 172.21.15.83:0/3160425823 learned_addr learned my addr 172.21.15.83:0/3160425823 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86da7fc640 1 --2- 172.21.15.83:0/3160425823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557242b96920 0x557242b50870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86daffd640 1 -- 172.21.15.83:0/3160425823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557242b97240 msgr2=0x557242ba08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86daffd640 1 --2- 172.21.15.83:0/3160425823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557242b97240 0x557242ba08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86daffd640 1 -- 172.21.15.83:0/3160425823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557242b96920 msgr2=0x557242b50870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86daffd640 1 --2- 172.21.15.83:0/3160425823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557242b96920 0x557242b50870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86daffd640 1 -- 172.21.15.83:0/3160425823 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557242b97070 con 0x557242b4cf30 2024-04-17T10:41:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86daffd640 1 --2- 172.21.15.83:0/3160425823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557242b4cf30 0x557242b50330 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f86cc051bb0 tx=0x7f86cc083240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb59fc9d544aff1b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86bbfff640 1 -- 172.21.15.83:0/3160425823 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc055750 con 0x557242b4cf30 2024-04-17T10:41:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86bbfff640 1 -- 172.21.15.83:0/3160425823 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86cc06d070 con 0x557242b4cf30 2024-04-17T10:41:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86bbfff640 1 -- 172.21.15.83:0/3160425823 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc0598a0 con 0x557242b4cf30 2024-04-17T10:41:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86e470d000 1 -- 172.21.15.83:0/3160425823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557242b4cf30 msgr2=0x557242b50330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86e470d000 1 --2- 172.21.15.83:0/3160425823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557242b4cf30 0x557242b50330 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f86cc051bb0 tx=0x7f86cc083240 comp rx=0 tx=0).stop 2024-04-17T10:41:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86e470d000 1 -- 172.21.15.83:0/3160425823 shutdown_connections 2024-04-17T10:41:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86e470d000 1 --2- 172.21.15.83:0/3160425823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557242b97240 0x557242ba08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86e470d000 1 --2- 172.21.15.83:0/3160425823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557242b96920 0x557242b50870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86e470d000 1 --2- 172.21.15.83:0/3160425823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557242b4cf30 0x557242b50330 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86e470d000 1 -- 172.21.15.83:0/3160425823 >> 172.21.15.83:0/3160425823 conn(0x557242b532a0 msgr2=0x557242b82b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86e470d000 1 -- 172.21.15.83:0/3160425823 shutdown_connections 2024-04-17T10:41:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86e470d000 1 -- 172.21.15.83:0/3160425823 wait complete. 2024-04-17T10:41:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86e470d000 1 -- start start 2024-04-17T10:41:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86e470d000 1 Processor -- start 2024-04-17T10:41:43.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86e470d000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557242b4cf30 0x557242ba3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:43.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86e470d000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557242b96920 0x557242be3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:43.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86e470d000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557242b97240 0x557242be4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:43.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86e470d000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572429a03d0 con 0x557242b97240 2024-04-17T10:41:43.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86e470d000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557242a57d50 con 0x557242b96920 2024-04-17T10:41:43.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86e470d000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x557242936980 con 0x557242b4cf30 2024-04-17T10:41:43.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86e470d000 10 client.0 _ll_register_callbacks cb 0x7f86c4003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:41:43.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86db7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557242b97240 0x557242be4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:43.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86db7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557242b97240 0x557242be4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51352/0 (socket says 172.21.15.83:51352) 2024-04-17T10:41:43.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86db7fe640 1 -- 172.21.15.83:0/3662243103 learned_addr learned my addr 172.21.15.83:0/3662243103 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:43.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86daffd640 1 --2- 172.21.15.83:0/3662243103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557242b4cf30 0x557242ba3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:43.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.123+0000 7f86da7fc640 1 --2- 172.21.15.83:0/3662243103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557242b96920 0x557242be3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:43.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86db7fe640 1 -- 172.21.15.83:0/3662243103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557242b4cf30 msgr2=0x557242ba3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:43.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86db7fe640 1 --2- 172.21.15.83:0/3662243103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557242b4cf30 0x557242ba3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86db7fe640 1 -- 172.21.15.83:0/3662243103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557242b96920 msgr2=0x557242be3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:43.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86db7fe640 1 --2- 172.21.15.83:0/3662243103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557242b96920 0x557242be3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86db7fe640 1 -- 172.21.15.83:0/3662243103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557242a1d600 con 0x557242b97240 2024-04-17T10:41:43.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86da7fc640 1 --2- 172.21.15.83:0/3662243103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557242b96920 0x557242be3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:43.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86db7fe640 1 --2- 172.21.15.83:0/3662243103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557242b97240 0x557242be4520 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f86d401cdd0 tx=0x7f86c4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f001e8ce5ab4ccc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:43.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:41:43.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86b9ffb640 1 -- 172.21.15.83:0/3662243103 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d400a480 con 0x557242b97240 2024-04-17T10:41:43.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86b9ffb640 1 -- 172.21.15.83:0/3662243103 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86d40017c0 con 0x557242b97240 2024-04-17T10:41:43.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e470d000 1 -- 172.21.15.83:0/3662243103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86cc06d070 con 0x557242b97240 2024-04-17T10:41:43.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86b9ffb640 1 -- 172.21.15.83:0/3662243103 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86d40019c0 con 0x557242b97240 2024-04-17T10:41:43.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e470d000 10 client.5832 fetch_fsmap learned FSMap version 21 2024-04-17T10:41:43.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e470d000 1 -- 172.21.15.83:0/3662243103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x5572429309b0 con 0x557242b97240 2024-04-17T10:41:43.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86b9ffb640 1 -- 172.21.15.83:0/3662243103 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86cc06d070 con 0x557242b97240 2024-04-17T10:41:43.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86b9ffb640 1 -- 172.21.15.83:0/3662243103 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f86d40b6460 con 0x557242b97240 2024-04-17T10:41:43.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e470d000 10 client.5832 fetch_fsmap finished waiting for FSMap version 21 2024-04-17T10:41:43.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e470d000 10 client.5832 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-17T10:41:43.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e470d000 20 client.5832 populate_metadata read hostname 'smithi083' 2024-04-17T10:41:43.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e470d000 1 --2- 172.21.15.83:0/3662243103 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x557242be4a60 0x557242bffff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:43.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e470d000 4 client.5832 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T10:41:43.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e470d000 1 -- 172.21.15.83:0/3662243103 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5572429a92e0 con 0x557242be4a60 2024-04-17T10:41:43.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86daffd640 1 --2- 172.21.15.83:0/3662243103 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x557242be4a60 0x557242bffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:43.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86b9ffb640 10 client.5832 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:41:43.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86daffd640 1 --2- 172.21.15.83:0/3662243103 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x557242be4a60 0x557242bffff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36d51e78e8337585 server_cookie=acc2e131a9942b35 in_seq=0 out_seq=0 2024-04-17T10:41:43.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86b9ffb640 1 -- 172.21.15.83:0/3662243103 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5572429a92e0 con 0x557242be4a60 2024-04-17T10:41:43.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86b9ffb640 10 client.5832 handle_command_reply: tid=0 2024-04-17T10:41:43.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e470d000 1 client.5832 shutdown 2024-04-17T10:41:43.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e470d000 1 -- 172.21.15.83:0/3662243103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557242b97240 msgr2=0x557242be4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:43.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e470d000 1 --2- 172.21.15.83:0/3662243103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557242b97240 0x557242be4520 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f86d401cdd0 tx=0x7f86c4005840 comp rx=0 tx=0).stop 2024-04-17T10:41:43.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e470d000 1 -- 172.21.15.83:0/3662243103 shutdown_connections 2024-04-17T10:41:43.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e470d000 1 --2- 172.21.15.83:0/3662243103 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x557242be4a60 0x557242bffff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e470d000 1 --2- 172.21.15.83:0/3662243103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557242b97240 0x557242be4520 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e470d000 1 --2- 172.21.15.83:0/3662243103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557242b96920 0x557242be3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86b9ffb640 0 client.5832 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:41:43.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e470d000 1 --2- 172.21.15.83:0/3662243103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557242b4cf30 0x557242ba3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e470d000 1 -- 172.21.15.83:0/3662243103 >> 172.21.15.83:0/3662243103 conn(0x557242b532a0 msgr2=0x557242b4f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:43.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e470d000 1 -- 172.21.15.83:0/3662243103 shutdown_connections 2024-04-17T10:41:43.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e470d000 1 -- 172.21.15.83:0/3662243103 wait complete. 2024-04-17T10:41:43.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e470d000 20 client.5832 trim_cache size 0 max 16384 2024-04-17T10:41:43.132 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:41:43.132 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "3de9b7be-af7f-488a-969e-2b6b8db63bcd", 2024-04-17T10:41:43.132 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-04-17T10:41:43.132 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 5682, 2024-04-17T10:41:43.132 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-17T10:41:43.133 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-17T10:41:43.133 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-17T10:41:43.133 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 4.6832076799999998, 2024-04-17T10:41:43.133 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 21, 2024-04-17T10:41:43.133 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 46, 2024-04-17T10:41:43.133 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 46, 2024-04-17T10:41:43.133 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 9.7423201129999999 2024-04-17T10:41:43.133 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:41:43.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e15a0640 1 -- 172.21.15.83:0/1928375967 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f86c8066f60 msgr2=0x7f86c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:43.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e15a0640 1 --2- 172.21.15.83:0/1928375967 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f86c8066f60 0x7f86c8069420 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f86dc1eb480 tx=0x7f86d0008040 comp rx=0 tx=0).stop 2024-04-17T10:41:43.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e15a0640 1 -- 172.21.15.83:0/1928375967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86dc143560 msgr2=0x7f86dc143e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:43.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e15a0640 1 --2- 172.21.15.83:0/1928375967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86dc143560 0x7f86dc143e30 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00f820 tx=0x7f86cc00fce0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e15a0640 1 -- 172.21.15.83:0/1928375967 shutdown_connections 2024-04-17T10:41:43.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e15a0640 1 --2- 172.21.15.83:0/1928375967 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f86c8066f60 0x7f86c8069420 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e15a0640 1 --2- 172.21.15.83:0/1928375967 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86dc0b0740 0x7f86dc0b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e15a0640 1 --2- 172.21.15.83:0/1928375967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86dc157610 0x7f86dc15e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e15a0640 1 --2- 172.21.15.83:0/1928375967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86dc143560 0x7f86dc143e30 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e15a0640 1 -- 172.21.15.83:0/1928375967 >> 172.21.15.83:0/1928375967 conn(0x7f86dc0b15b0 msgr2=0x7f86dc0cdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:43.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e15a0640 1 -- 172.21.15.83:0/1928375967 shutdown_connections 2024-04-17T10:41:43.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.127+0000 7f86e15a0640 1 -- 172.21.15.83:0/1928375967 wait complete. 2024-04-17T10:41:43.142 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T10:41:43.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.259+0000 7fbeab65d640 1 Processor -- start 2024-04-17T10:41:43.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.259+0000 7fbeab65d640 1 -- start start 2024-04-17T10:41:43.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbeab65d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbea4072330 0x7fbea40cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:43.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbeab65d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbea40cfa10 0x7fbea406c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:43.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbeab65d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbea40cfea0 con 0x7fbea415ba30 2024-04-17T10:41:43.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbeab65d640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbea4071ea0 con 0x7fbea40cfa10 2024-04-17T10:41:43.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbeab65d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbea406e320 con 0x7fbea4072330 2024-04-17T10:41:43.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbea93f9640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbea415ba30 0x7fbea415c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:36908/0 (socket says 172.21.15.83:36908) 2024-04-17T10:41:43.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbea93f9640 1 -- 172.21.15.83:0/142399577 learned_addr learned my addr 172.21.15.83:0/142399577 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:43.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbea9bfa640 1 --2- 172.21.15.83:0/142399577 >> v2:172.21.15.167:3300/0 conn(0x7fbea40cfa10 0x7fbea406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:43.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbea8bf8640 1 --2- 172.21.15.83:0/142399577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbea4072330 0x7fbea40cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:43.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbe9bfff640 1 -- 172.21.15.83:0/142399577 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 826924710 0 0) 0x7fbea40cfea0 con 0x7fbea415ba30 2024-04-17T10:41:43.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbe9bfff640 1 -- 172.21.15.83:0/142399577 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe90003530 con 0x7fbea415ba30 2024-04-17T10:41:43.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbe9bfff640 1 -- 172.21.15.83:0/142399577 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2958535052 0 0) 0x7fbe90003530 con 0x7fbea415ba30 2024-04-17T10:41:43.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbe9bfff640 1 -- 172.21.15.83:0/142399577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbea4072330 msgr2=0x7fbea40cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:43.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbe9bfff640 1 --2- 172.21.15.83:0/142399577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbea4072330 0x7fbea40cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbe9bfff640 1 -- 172.21.15.83:0/142399577 >> v2:172.21.15.167:3300/0 conn(0x7fbea40cfa10 msgr2=0x7fbea406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:43.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbe9bfff640 1 --2- 172.21.15.83:0/142399577 >> v2:172.21.15.167:3300/0 conn(0x7fbea40cfa10 0x7fbea406c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbe9bfff640 1 -- 172.21.15.83:0/142399577 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea406e040 con 0x7fbea415ba30 2024-04-17T10:41:43.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbe9bfff640 1 -- 172.21.15.83:0/142399577 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fbe9c001680 con 0x7fbea415ba30 2024-04-17T10:41:43.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbe9bfff640 1 -- 172.21.15.83:0/142399577 >> v1:172.21.15.83:6789/0 conn(0x7fbea415ba30 legacy=0x7fbea415c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:43.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbe9bfff640 1 --2- 172.21.15.83:0/142399577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe900039c0 0x7fbe90003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:43.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbe9bfff640 1 --2- 172.21.15.83:0/142399577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe90004810 0x7fbe90004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:43.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbe9bfff640 1 --2- 172.21.15.83:0/142399577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbea4072330 0x7fbe900051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:43.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbe9bfff640 1 -- 172.21.15.83:0/142399577 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea4071ea0 con 0x7fbe90004810 2024-04-17T10:41:43.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbe9bfff640 1 -- 172.21.15.83:0/142399577 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea406e320 con 0x7fbe900039c0 2024-04-17T10:41:43.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbe9bfff640 1 -- 172.21.15.83:0/142399577 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9000b9d0 con 0x7fbea4072330 2024-04-17T10:41:43.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbea8bf8640 1 --2- 172.21.15.83:0/142399577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe90004810 0x7fbe90004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:43.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbea8bf8640 1 -- 172.21.15.83:0/142399577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbea4072330 msgr2=0x7fbe900051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:43.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbea8bf8640 1 --2- 172.21.15.83:0/142399577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbea4072330 0x7fbe900051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbea8bf8640 1 -- 172.21.15.83:0/142399577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe900039c0 msgr2=0x7fbe90003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:43.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbea93f9640 1 --2- 172.21.15.83:0/142399577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe900039c0 0x7fbe90003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:43.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbea8bf8640 1 --2- 172.21.15.83:0/142399577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe900039c0 0x7fbe90003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbea8bf8640 1 -- 172.21.15.83:0/142399577 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe90003530 con 0x7fbe90004810 2024-04-17T10:41:43.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.263+0000 7fbea93f9640 1 --2- 172.21.15.83:0/142399577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe900039c0 0x7fbe90003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:43.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.267+0000 7fbea8bf8640 1 --2- 172.21.15.83:0/142399577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe90004810 0x7fbe90004c90 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009af0 tx=0x7fbe9401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98f77c46dc4e76b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:43.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.267+0000 7fbe9bfff640 1 -- 172.21.15.83:0/142399577 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe94009690 con 0x7fbe90004810 2024-04-17T10:41:43.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.267+0000 7fbe9bfff640 1 -- 172.21.15.83:0/142399577 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe940a6480 con 0x7fbe90004810 2024-04-17T10:41:43.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.267+0000 7fbe9bfff640 1 -- 172.21.15.83:0/142399577 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe940a6850 con 0x7fbe90004810 2024-04-17T10:41:43.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.267+0000 7fbeab65d640 1 -- 172.21.15.83:0/142399577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe90004810 msgr2=0x7fbe90004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:43.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.267+0000 7fbeab65d640 1 --2- 172.21.15.83:0/142399577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe90004810 0x7fbe90004c90 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009af0 tx=0x7fbe9401ca70 comp rx=0 tx=0).stop 2024-04-17T10:41:43.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.267+0000 7fbea9bfa640 1 -- 172.21.15.83:0/142399577 reap_dead start 2024-04-17T10:41:43.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.267+0000 7fbeab65d640 1 -- 172.21.15.83:0/142399577 shutdown_connections 2024-04-17T10:41:43.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.267+0000 7fbeab65d640 1 -- 172.21.15.83:0/142399577 >> 172.21.15.83:0/142399577 conn(0x7fbea406abe0 msgr2=0x7fbea406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:43.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.267+0000 7fbeab65d640 1 -- 172.21.15.83:0/142399577 shutdown_connections 2024-04-17T10:41:43.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.267+0000 7fbeab65d640 1 -- 172.21.15.83:0/142399577 wait complete. 2024-04-17T10:41:43.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.267+0000 7fbeab65d640 1 Processor -- start 2024-04-17T10:41:43.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.267+0000 7fbeab65d640 1 -- start start 2024-04-17T10:41:43.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.267+0000 7fbeab65d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe90004810 0x7fbea4156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:43.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.267+0000 7fbeab65d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbea415fcd0 0x7fbea40afde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:43.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.267+0000 7fbeab65d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbea40b0320 0x7fbea4155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:43.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.267+0000 7fbeab65d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea406de20 con 0x7fbe90004810 2024-04-17T10:41:43.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.267+0000 7fbeab65d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea406dca0 con 0x7fbea415fcd0 2024-04-17T10:41:43.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.267+0000 7fbeab65d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbea4166510 con 0x7fbea40b0320 2024-04-17T10:41:43.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.267+0000 7fbea93f9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe90004810 0x7fbea4156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:43.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.267+0000 7fbea93f9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe90004810 0x7fbea4156d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51368/0 (socket says 172.21.15.83:51368) 2024-04-17T10:41:43.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.267+0000 7fbea93f9640 1 -- 172.21.15.83:0/1510686515 learned_addr learned my addr 172.21.15.83:0/1510686515 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:43.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.267+0000 7fbea8bf8640 1 --2- 172.21.15.83:0/1510686515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbea415fcd0 0x7fbea40afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:43.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.267+0000 7fbea93f9640 1 -- 172.21.15.83:0/1510686515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbea40b0320 msgr2=0x7fbea4155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:43.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.267+0000 7fbea9bfa640 1 --2- 172.21.15.83:0/1510686515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbea40b0320 0x7fbea4155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:43.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.267+0000 7fbea93f9640 1 --2- 172.21.15.83:0/1510686515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbea40b0320 0x7fbea4155730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.267+0000 7fbea93f9640 1 -- 172.21.15.83:0/1510686515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbea415fcd0 msgr2=0x7fbea40afde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:43.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.267+0000 7fbea93f9640 1 --2- 172.21.15.83:0/1510686515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbea415fcd0 0x7fbea40afde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.267+0000 7fbea93f9640 1 -- 172.21.15.83:0/1510686515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea41601d0 con 0x7fbe90004810 2024-04-17T10:41:43.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.267+0000 7fbea8bf8640 1 --2- 172.21.15.83:0/1510686515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbea415fcd0 0x7fbea40afde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:41:43.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.267+0000 7fbea9bfa640 1 --2- 172.21.15.83:0/1510686515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbea40b0320 0x7fbea4155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:43.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.271+0000 7fbea93f9640 1 --2- 172.21.15.83:0/1510686515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe90004810 0x7fbea4156d60 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c007fd0 tx=0x7fbe9c00dc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:43.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.271+0000 7fbe99ffb640 1 -- 172.21.15.83:0/1510686515 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c019070 con 0x7fbe90004810 2024-04-17T10:41:43.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.271+0000 7fbe99ffb640 1 -- 172.21.15.83:0/1510686515 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe9c0074e0 con 0x7fbe90004810 2024-04-17T10:41:43.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.271+0000 7fbe99ffb640 1 -- 172.21.15.83:0/1510686515 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c0049f0 con 0x7fbe90004810 2024-04-17T10:41:43.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.271+0000 7fbeab65d640 1 -- 172.21.15.83:0/1510686515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbea41604b0 con 0x7fbe90004810 2024-04-17T10:41:43.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.271+0000 7fbeab65d640 1 -- 172.21.15.83:0/1510686515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbea4160a70 con 0x7fbe90004810 2024-04-17T10:41:43.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.271+0000 7fbeab65d640 1 -- 172.21.15.83:0/1510686515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbea40b0730 con 0x7fbe90004810 2024-04-17T10:41:43.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.271+0000 7fbe99ffb640 1 -- 172.21.15.83:0/1510686515 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbe9c004b90 con 0x7fbe90004810 2024-04-17T10:41:43.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.271+0000 7fbe99ffb640 1 --2- 172.21.15.83:0/1510686515 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbe88066f60 0x7fbe88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:43.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.271+0000 7fbe99ffb640 1 -- 172.21.15.83:0/1510686515 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe9c0848d0 con 0x7fbe90004810 2024-04-17T10:41:43.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.271+0000 7fbea8bf8640 1 --2- 172.21.15.83:0/1510686515 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbe88066f60 0x7fbe88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:43.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.271+0000 7fbea8bf8640 1 --2- 172.21.15.83:0/1510686515 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbe88066f60 0x7fbe88069420 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbe940ad000 tx=0x7fbe940b2040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:43.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.275+0000 7fbe99ffb640 1 -- 172.21.15.83:0/1510686515 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbea40b0730 con 0x7fbe90004810 2024-04-17T10:41:43.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.475+0000 7fbeab65d640 1 -- 172.21.15.83:0/1510686515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbea4074f90 con 0x7fbe90004810 2024-04-17T10:41:43.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.475+0000 7fbe99ffb640 1 -- 172.21.15.83:0/1510686515 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbe9c055320 con 0x7fbe90004810 2024-04-17T10:41:43.475 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T10:41:43.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.479+0000 7fbeab65d640 1 -- 172.21.15.83:0/1510686515 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbe88066f60 msgr2=0x7fbe88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:43.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.479+0000 7fbeab65d640 1 --2- 172.21.15.83:0/1510686515 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbe88066f60 0x7fbe88069420 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbe940ad000 tx=0x7fbe940b2040 comp rx=0 tx=0).stop 2024-04-17T10:41:43.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.479+0000 7fbeab65d640 1 -- 172.21.15.83:0/1510686515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe90004810 msgr2=0x7fbea4156d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:43.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.479+0000 7fbeab65d640 1 --2- 172.21.15.83:0/1510686515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe90004810 0x7fbea4156d60 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c007fd0 tx=0x7fbe9c00dc70 comp rx=0 tx=0).stop 2024-04-17T10:41:43.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.479+0000 7fbeab65d640 1 -- 172.21.15.83:0/1510686515 shutdown_connections 2024-04-17T10:41:43.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.479+0000 7fbeab65d640 1 --2- 172.21.15.83:0/1510686515 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbe88066f60 0x7fbe88069420 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.479+0000 7fbeab65d640 1 --2- 172.21.15.83:0/1510686515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbea40b0320 0x7fbea4155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.479+0000 7fbeab65d640 1 --2- 172.21.15.83:0/1510686515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbea415fcd0 0x7fbea40afde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.479+0000 7fbeab65d640 1 --2- 172.21.15.83:0/1510686515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe90004810 0x7fbea4156d60 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:43.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.479+0000 7fbeab65d640 1 -- 172.21.15.83:0/1510686515 >> 172.21.15.83:0/1510686515 conn(0x7fbea406abe0 msgr2=0x7fbea40ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:43.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.479+0000 7fbeab65d640 1 -- 172.21.15.83:0/1510686515 shutdown_connections 2024-04-17T10:41:43.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:43.479+0000 7fbeab65d640 1 -- 172.21.15.83:0/1510686515 wait complete. 2024-04-17T10:41:43.491 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T10:41:43.491 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T10:41:43.491 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-04-17T10:41:43.491 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T10:41:43.491 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T10:41:43.491 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T10:41:43.491 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T10:41:43.491 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T10:41:43.492 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:41:43.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:41:43.492 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-04-17T10:41:43.499 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T10:41:43.499 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T10:41:43.499 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T10:41:43.499 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:41:43.499 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-17T10:41:43.499 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:41:43.499 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:41:43.499 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-04-17T10:41:43.499 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-04-17T10:41:43.499 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:41:43.500 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-04-17T10:41:43.500 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-04-17T10:41:43.500 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:41:43.500 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:41:43.500 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T10:41:43.500 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:41:43.500 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-17T10:41:43.500 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42280sec preferred_lft 42280sec 2024-04-17T10:41:43.500 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-04-17T10:41:43.500 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:41:43.500 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T10:41:43.500 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:37:0e:23:b2 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:41:43.500 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T10:41:43.501 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:41:43.501 INFO:teuthology.orchestra.run.smithi083.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T10:41:43.501 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-17T10:41:43.501 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T10:41:43.501 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:41:43.501 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-17T10:41:43.501 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:41:43.501 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:41:43.501 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:41:43.501 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-17T10:41:43.501 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:41:43.501 INFO:teuthology.orchestra.run.smithi083.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:41:43.501 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:41:43.501 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-17T10:41:43.502 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:41:43.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:41:43.502 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T10:41:43.551 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-04-17T10:41:43.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:41:43.552 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-04-17T10:41:43.603 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-04-17T10:41:43.603 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T10:41:43.604 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-17T10:41:43.604 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-17T10:41:43.604 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-17T10:41:43.604 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T10:41:43.604 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-17T10:41:43.604 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:41:43.604 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:41:43.604 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:41:43.604 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T10:41:43.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:41:43.605 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2024-04-17T10:41:43.605 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:41:43.605 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-17T10:41:43.605 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-17T10:41:43.605 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-17T10:41:43.605 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:41:43.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:41:43.676 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-17T10:41:43.723 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-17T10:41:43.723 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T10:41:43.723 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:41:43.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:41:43.723 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-04-17T10:41:43.777 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-17T10:41:43.777 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:41:43.777 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T10:41:43.823 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:41:43.824 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:41:43.824 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:41:43.871 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:41:43.929 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:41:43.978 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:41:43.978 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:41:43.978 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:41:43.978 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:41:43.978 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:41:43.978 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:41:43.978 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:41:43.979 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:41:43.979 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:41:43.979 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:41:43.979 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:41:43.979 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:41:43.979 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:41:43.979 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:41:43.979 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:41:43.979 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:41:43.979 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:41:43.979 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:41:43.979 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:41:43.979 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:41:43.980 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:41:43.980 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:41:43.980 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:41:43.980 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:41:43.980 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:41:43.980 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:41:43.980 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:41:43.980 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:41:43.980 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:41:43.980 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:41:43.980 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:41:43.980 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:41:43.980 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:41:43.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:41:43.982 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:41:44.028 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T10:41:44.028 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-17T10:41:44.072 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:41:44.088 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:41:44.140 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:41:44.140 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:41:44.140 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:41:44.140 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:41:44.140 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:41:44.140 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:41:44.140 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:41:44.140 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:41:44.140 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:41:44.140 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:41:44.140 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:41:44.140 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:41:44.141 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:41:44.141 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:41:44.141 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:41:44.141 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:41:44.141 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:41:44.141 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:41:44.141 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:41:44.141 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:41:44.141 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:41:44.141 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:41:44.141 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:41:44.141 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:41:44.141 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:41:44.142 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:41:44.142 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:41:44.142 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:41:44.142 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:41:44.142 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:41:44.142 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:41:44.142 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:41:44.142 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:41:44.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:41:44.143 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:41:44.157 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:2024-04-17T10:41:44.159+0000 7fdc25b93580 -1 init, newargv = 0x55599d9dab30 newargc=15 2024-04-17T10:41:44.158 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[51595]: starting ceph client 2024-04-17T10:41:44.163 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[51595]: starting fuse 2024-04-17T10:41:45.263 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:41:45.278 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:41:45.330 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:41:45.330 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:41:45.330 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:41:45.330 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:41:45.330 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:41:45.330 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:41:45.330 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:41:45.331 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:41:45.331 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:41:45.331 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:41:45.331 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:41:45.331 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:41:45.331 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:41:45.331 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:41:45.331 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:41:45.331 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:41:45.331 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:41:45.331 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:41:45.332 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:41:45.332 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:41:45.332 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:41:45.332 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:41:45.332 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:41:45.332 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:41:45.332 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:41:45.332 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:41:45.332 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:41:45.332 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:41:45.332 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:41:45.332 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:41:45.333 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:41:45.333 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:41:45.333 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:41:45.333 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-17T10:41:45.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:41:45.334 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:41:45.384 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-04-17T10:41:45.384 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-17T10:41:45.384 DEBUG:teuthology.orchestra.run.smithi083:> sudo stdin-killer -- python3 -c ' 2024-04-17T10:41:45.384 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2024-04-17T10:41:45.384 DEBUG:teuthology.orchestra.run.smithi083:> import re 2024-04-17T10:41:45.385 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-04-17T10:41:45.385 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2024-04-17T10:41:45.385 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:41:45.385 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2024-04-17T10:41:45.385 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-17T10:41:45.385 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2024-04-17T10:41:45.385 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-17T10:41:45.385 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:41:45.385 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2024-04-17T10:41:45.385 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2024-04-17T10:41:45.385 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2024-04-17T10:41:45.385 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2024-04-17T10:41:45.385 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:41:45.385 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2024-04-17T10:41:45.385 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T10:41:45.386 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T10:41:45.386 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T10:41:45.386 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2024-04-17T10:41:45.386 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2024-04-17T10:41:45.386 DEBUG:teuthology.orchestra.run.smithi083:> return f 2024-04-17T10:41:45.386 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T10:41:45.386 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:41:45.386 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.0")) 2024-04-17T10:41:45.386 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-04-17T10:41:45.508 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.0.51595.asok 2024-04-17T10:41:45.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:41:45.520 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.51595.asok 2024-04-17T10:41:45.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:41:45.520 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.51595.asok status 2024-04-17T10:41:45.670 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:41:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2024-04-17T10:41:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-17T10:41:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-17T10:41:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "0", 2024-04-17T10:41:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-04-17T10:41:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-17T10:41:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "51595", 2024-04-17T10:41:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-04-17T10:41:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2024-04-17T10:41:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2024-04-17T10:41:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 5844, 2024-04-17T10:41:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2024-04-17T10:41:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-04-17T10:41:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-04-17T10:41:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 5844 2024-04-17T10:41:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-04-17T10:41:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-17T10:41:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:41:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4246116797 2024-04-17T10:41:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:41:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-04-17T10:41:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-17T10:41:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:41:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4246116797 2024-04-17T10:41:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:41:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.5844 192.168.0.1:0/4246116797", 2024-04-17T10:41:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.1:0/4246116797", 2024-04-17T10:41:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 1, 2024-04-17T10:41:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 21, 2024-04-17T10:41:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 46, 2024-04-17T10:41:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2024-04-17T10:41:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2024-04-17T10:41:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2024-04-17T10:41:45.673 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:41:45.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:41:45.684 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2024-04-17T10:41:45.684 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-17T10:41:45.690 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2024-04-17T10:41:45.690 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-17T10:41:45.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:41:45.690 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:41:45.744 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-17T10:41:45.744 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_test_path 2024-04-17T10:41:45.745 INFO:tasks.cephfs.test_scrub_checks:Cloning repo into place 2024-04-17T10:41:45.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:41:45.745 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p /home/ubuntu/cephtest/mnt.0/scrub_test_path) 2024-04-17T10:41:45.806 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-04-17T10:41:45.806 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-04-17T10:41:45.806 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-04-17T10:41:45.806 DEBUG:teuthology.orchestra.run.smithi083:> import json 2024-04-17T10:41:45.806 DEBUG:teuthology.orchestra.run.smithi083:> import sys 2024-04-17T10:41:45.806 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:41:45.806 DEBUG:teuthology.orchestra.run.smithi083:> try: 2024-04-17T10:41:45.806 DEBUG:teuthology.orchestra.run.smithi083:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'"'"') 2024-04-17T10:41:45.806 DEBUG:teuthology.orchestra.run.smithi083:> except OSError as e: 2024-04-17T10:41:45.806 DEBUG:teuthology.orchestra.run.smithi083:> sys.exit(e.errno) 2024-04-17T10:41:45.807 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:41:45.807 DEBUG:teuthology.orchestra.run.smithi083:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-17T10:41:45.807 DEBUG:teuthology.orchestra.run.smithi083:> print(json.dumps( 2024-04-17T10:41:45.807 DEBUG:teuthology.orchestra.run.smithi083:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-17T10:41:45.807 DEBUG:teuthology.orchestra.run.smithi083:> indent=2)) 2024-04-17T10:41:45.807 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-04-17T10:41:45.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:45 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-04-17T10:41:45.932 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-17T10:41:45.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:41:45.932 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone --branch giant http://github.com/ceph/ceph-qa-suite /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite) 2024-04-17T10:41:45.999 INFO:teuthology.orchestra.run.smithi083.stderr:Cloning into '/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'... 2024-04-17T10:41:46.333 INFO:teuthology.orchestra.run.smithi083.stderr:warning: redirecting to https://github.com/ceph/ceph-qa-suite/ 2024-04-17T10:41:58.074 INFO:teuthology.orchestra.run.smithi083.stderr:Updating files: 10% (142/1398) Updating files: 11% (154/1398) Updating files: 12% (168/1398) Updating files: 13% (182/1398) Updating files: 14% (196/1398) Updating files: 15% (210/1398) Updating files: 16% (224/1398) Updating files: 17% (238/1398) Updating files: 18% (252/1398) Updating files: 19% (266/1398) Updating files: 20% (280/1398) Updating files: 20% (290/1398) Updating files: 21% (294/1398) Updating files: 22% (308/1398) Updating files: 23% (322/1398) Updating files: 24% (336/1398) Updating files: 25% (350/1398) Updating files: 26% (364/1398) Updating files: 27% (378/1398) Updating files: 28% (392/1398) Updating files: 29% (406/1398) Updating files: 30% (420/1398) Updating files: 31% (434/1398) Updating files: 32% (448/1398) Updating files: 33% (462/1398) Updating files: 34% (476/1398) Updating files: 35% (490/1398) Updating files: 36% (504/1398) Updating files: 37% (518/1398) Updating files: 38% (532/1398) Updating files: 39% (546/1398) Updating files: 40% (560/1398) Updating files: 40% (572/1398) Updating files: 41% (574/1398) Updating files: 42% (588/1398) Updating files: 43% (602/1398) Updating files: 44% (616/1398) Updating files: 45% (630/1398) Updating files: 46% (644/1398) Updating files: 47% (658/1398) Updating files: 48% (672/1398) Updating files: 49% (686/1398) Updating files: 50% (699/1398) Updating files: 51% (713/1398) Updating files: 52% (727/1398) Updating files: 53% (741/1398) Updating files: 54% (755/1398) Updating files: 55% (769/1398) Updating files: 56% (783/1398) Updating files: 57% (797/1398) Updating files: 58% (811/1398) Updating files: 59% (825/1398) Updating files: 60% (839/1398) Updating files: 61% (853/1398) Updating files: 62% (867/1398) Updating files: 63% (881/1398) Updating files: 64% (895/1398) Updating files: 65% (909/1398) Updating files: 66% (923/1398) Updating files: 67% (937/1398) Updating files: 68% (951/1398) Updating files: 69% (965/1398) Updating files: 70% (979/1398) Updating files: 71% (993/1398) Updating files: 72% (1007/1398) Updating files: 73% (1021/1398) Updating files: 74% (1035/1398) Updating files: 75% (1049/1398) Updating files: 75% (1053/1398) Updating files: 76% (1063/1398) Updating files: 77% (1077/1398) Updating files: 78% (1091/1398) Updating files: 79% (1105/1398) Updating files: 80% (1119/1398) Updating files: 81% (1133/1398) Updating files: 82% (1147/1398) Updating files: 83% (1161/1398) Updating files: 84% (1175/1398) Updating files: 85% (1189/1398) Updating files: 85% (1202/1398) Updating files: 86% (1203/1398) Updating files: 87% (1217/1398) Updating files: 88% (1231/1398) Updating files: 89% (1245/1398) Updating files: 90% (1259/1398) Updating files: 91% (1273/1398) Updating files: 92% (1287/1398) Updating files: 93% (1301/1398) Updating files: 94% (1315/1398) Updating files: 95% (1329/1398) Updating files: 96% (1343/1398) Updating files: 97% (1357/1398) Updating files: 98% (1371/1398) Updating files: 99% (1385/1398) Updating files: 100% (1398/1398) Updating files: 100% (1398/1398), done. 2024-04-17T10:41:58.083 INFO:tasks.cephfs.test_scrub_checks:Initiating mds_scrub_checks on mds.0 test_path /scrub_test_path, run_seq 0 2024-04-17T10:41:58.083 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/i/dont/exist' 2024-04-17T10:41:58.083 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:41:58.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.267+0000 7fae9564f640 1 Processor -- start 2024-04-17T10:41:58.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.267+0000 7fae9564f640 1 -- start start 2024-04-17T10:41:58.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.271+0000 7fae9564f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fae900ce980 0x7fae900b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.271+0000 7fae9564f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae900cee00 0x7fae900b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.271+0000 7fae9564f640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae90162a00 con 0x7fae9015dfa0 2024-04-17T10:41:58.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.271+0000 7fae9564f640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae90161e80 con 0x7fae900ce980 2024-04-17T10:41:58.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.271+0000 7fae9564f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae90162160 con 0x7fae900cee00 2024-04-17T10:41:58.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.271+0000 7fae867fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fae900ce980 0x7fae900b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.271+0000 7fae867fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fae900ce980 0x7fae900b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41562/0 (socket says 172.21.15.83:41562) 2024-04-17T10:41:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.271+0000 7fae867fc640 1 -- 172.21.15.83:0/2545858096 learned_addr learned my addr 172.21.15.83:0/2545858096 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.271+0000 7fae8f7fe640 1 --2- 172.21.15.83:0/2545858096 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae900cee00 0x7fae900b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.271+0000 7fae867fc640 1 -- 172.21.15.83:0/2545858096 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae900cee00 msgr2=0x7fae900b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.271+0000 7fae867fc640 1 --2- 172.21.15.83:0/2545858096 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae900cee00 0x7fae900b5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.271+0000 7fae867fc640 1 -- 172.21.15.83:0/2545858096 >> v1:172.21.15.167:6789/0 conn(0x7fae9015dfa0 legacy=0x7fae900b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.271+0000 7fae867fc640 1 -- 172.21.15.83:0/2545858096 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae90162440 con 0x7fae900ce980 2024-04-17T10:41:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.271+0000 7fae867fc640 1 --2- 172.21.15.83:0/2545858096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae900ce980 0x7fae900b5430 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fae78009870 tx=0x7fae7801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d58e685f9a9947d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.271+0000 7fae8e7fc640 1 -- 172.21.15.83:0/2545858096 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae780b1070 con 0x7fae900ce980 2024-04-17T10:41:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.271+0000 7fae8e7fc640 1 -- 172.21.15.83:0/2545858096 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae780ac620 con 0x7fae900ce980 2024-04-17T10:41:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.271+0000 7fae8e7fc640 1 -- 172.21.15.83:0/2545858096 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae780ac9f0 con 0x7fae900ce980 2024-04-17T10:41:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.275+0000 7fae9564f640 1 -- 172.21.15.83:0/2545858096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae900ce980 msgr2=0x7fae900b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.275+0000 7fae9564f640 1 --2- 172.21.15.83:0/2545858096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae900ce980 0x7fae900b5430 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fae78009870 tx=0x7fae7801c460 comp rx=0 tx=0).stop 2024-04-17T10:41:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.275+0000 7fae9564f640 1 -- 172.21.15.83:0/2545858096 shutdown_connections 2024-04-17T10:41:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.275+0000 7fae9564f640 1 --2- 172.21.15.83:0/2545858096 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae900cee00 0x7fae900b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.275+0000 7fae9564f640 1 --2- 172.21.15.83:0/2545858096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae900ce980 0x7fae900b5430 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.275+0000 7fae9564f640 1 -- 172.21.15.83:0/2545858096 >> 172.21.15.83:0/2545858096 conn(0x7fae900d0600 msgr2=0x7fae900d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.275+0000 7fae9564f640 1 -- 172.21.15.83:0/2545858096 shutdown_connections 2024-04-17T10:41:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.275+0000 7fae9564f640 1 -- 172.21.15.83:0/2545858096 wait complete. 2024-04-17T10:41:58.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.275+0000 7fae9564f640 1 Processor -- start 2024-04-17T10:41:58.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.275+0000 7fae9564f640 1 -- start start 2024-04-17T10:41:58.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.275+0000 7fae9564f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae900ce980 0x7fae900c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.275+0000 7fae9564f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae900cee00 0x7fae900cd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.275+0000 7fae9564f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae9015dfa0 0x7fae900c6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.275+0000 7fae9564f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae900d26c0 con 0x7fae900ce980 2024-04-17T10:41:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.275+0000 7fae9564f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae900d2540 con 0x7fae900cee00 2024-04-17T10:41:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.275+0000 7fae9564f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae900d2840 con 0x7fae9015dfa0 2024-04-17T10:41:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.275+0000 7fae8effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae900ce980 0x7fae900c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.275+0000 7fae8effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae900ce980 0x7fae900c6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41576/0 (socket says 172.21.15.83:41576) 2024-04-17T10:41:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.275+0000 7fae8effd640 1 -- 172.21.15.83:0/3868109133 learned_addr learned my addr 172.21.15.83:0/3868109133 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.275+0000 7fae8f7fe640 1 --2- 172.21.15.83:0/3868109133 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae9015dfa0 0x7fae900c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.275+0000 7fae867fc640 1 --2- 172.21.15.83:0/3868109133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae900cee00 0x7fae900cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.275+0000 7fae8effd640 1 -- 172.21.15.83:0/3868109133 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae9015dfa0 msgr2=0x7fae900c6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.275+0000 7fae8effd640 1 --2- 172.21.15.83:0/3868109133 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae9015dfa0 0x7fae900c6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.275+0000 7fae8effd640 1 -- 172.21.15.83:0/3868109133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae900cee00 msgr2=0x7fae900cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.275+0000 7fae8effd640 1 --2- 172.21.15.83:0/3868109133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae900cee00 0x7fae900cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.275+0000 7fae8effd640 1 -- 172.21.15.83:0/3868109133 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9014b860 con 0x7fae900ce980 2024-04-17T10:41:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.275+0000 7fae8effd640 1 --2- 172.21.15.83:0/3868109133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae900ce980 0x7fae900c6f10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fae8000c450 tx=0x7fae8000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.275+0000 7fae87fff640 1 -- 172.21.15.83:0/3868109133 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae80012830 con 0x7fae900ce980 2024-04-17T10:41:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.275+0000 7fae87fff640 1 -- 172.21.15.83:0/3868109133 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae80017070 con 0x7fae900ce980 2024-04-17T10:41:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.275+0000 7fae9564f640 1 -- 172.21.15.83:0/3868109133 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae9014bb40 con 0x7fae900ce980 2024-04-17T10:41:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.275+0000 7fae87fff640 1 -- 172.21.15.83:0/3868109133 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae80012c30 con 0x7fae900ce980 2024-04-17T10:41:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.275+0000 7fae9564f640 1 -- 172.21.15.83:0/3868109133 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae9014c100 con 0x7fae900ce980 2024-04-17T10:41:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.279+0000 7fae9564f640 1 -- 172.21.15.83:0/3868109133 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae90074f90 con 0x7fae900ce980 2024-04-17T10:41:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.279+0000 7fae87fff640 1 -- 172.21.15.83:0/3868109133 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fae8001c460 con 0x7fae900ce980 2024-04-17T10:41:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.279+0000 7fae87fff640 1 --2- 172.21.15.83:0/3868109133 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fae70066f90 0x7fae70069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.279+0000 7fae87fff640 1 -- 172.21.15.83:0/3868109133 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fae80084e40 con 0x7fae900ce980 2024-04-17T10:41:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.279+0000 7fae867fc640 1 --2- 172.21.15.83:0/3868109133 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fae70066f90 0x7fae70069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.279+0000 7fae867fc640 1 --2- 172.21.15.83:0/3868109133 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fae70066f90 0x7fae70069450 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fae780ae000 tx=0x7fae78008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:58.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.283+0000 7fae87fff640 1 -- 172.21.15.83:0/3868109133 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fae80055e10 con 0x7fae900ce980 2024-04-17T10:41:58.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.499+0000 7fae9564f640 1 -- 172.21.15.83:0/3868109133 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae900d17a0 con 0x7fae900ce980 2024-04-17T10:41:58.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.499+0000 7fae87fff640 1 -- 172.21.15.83:0/3868109133 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4938 (secure 0 0 0) 0x7fae80055710 con 0x7fae900ce980 2024-04-17T10:41:58.497 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:41:58.497 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/161398126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":161398126},{"type":"v1","addr":"172.21.15.167:6833","nonce":161398126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1085380618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1085380618},{"type":"v1","addr":"172.21.15.167:6835","nonce":1085380618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/836442291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":836442291},{"type":"v1","addr":"172.21.15.83:6835","nonce":836442291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:41:37.773342+0000","modified":"2024-04-17T10:41:39.788394+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5682":{"gid":5682,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1894547386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1894547386},{"type":"v1","addr":"172.21.15.83:6837","nonce":1894547386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-17T10:41:58.497 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 21 2024-04-17T10:41:58.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.503+0000 7fae9564f640 1 -- 172.21.15.83:0/3868109133 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fae70066f90 msgr2=0x7fae70069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:58.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.503+0000 7fae9564f640 1 --2- 172.21.15.83:0/3868109133 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fae70066f90 0x7fae70069450 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fae780ae000 tx=0x7fae78008040 comp rx=0 tx=0).stop 2024-04-17T10:41:58.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.503+0000 7fae9564f640 1 -- 172.21.15.83:0/3868109133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae900ce980 msgr2=0x7fae900c6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:58.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.503+0000 7fae9564f640 1 --2- 172.21.15.83:0/3868109133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae900ce980 0x7fae900c6f10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fae8000c450 tx=0x7fae8000c910 comp rx=0 tx=0).stop 2024-04-17T10:41:58.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.503+0000 7fae9564f640 1 -- 172.21.15.83:0/3868109133 shutdown_connections 2024-04-17T10:41:58.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.503+0000 7fae9564f640 1 --2- 172.21.15.83:0/3868109133 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fae70066f90 0x7fae70069450 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.503+0000 7fae9564f640 1 --2- 172.21.15.83:0/3868109133 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae9015dfa0 0x7fae900c6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.503+0000 7fae9564f640 1 --2- 172.21.15.83:0/3868109133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae900cee00 0x7fae900cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.503+0000 7fae9564f640 1 --2- 172.21.15.83:0/3868109133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae900ce980 0x7fae900c6f10 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.503+0000 7fae9564f640 1 -- 172.21.15.83:0/3868109133 >> 172.21.15.83:0/3868109133 conn(0x7fae900d0600 msgr2=0x7fae900d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:58.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.503+0000 7fae9564f640 1 -- 172.21.15.83:0/3868109133 shutdown_connections 2024-04-17T10:41:58.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.503+0000 7fae9564f640 1 -- 172.21.15.83:0/3868109133 wait complete. 2024-04-17T10:41:58.518 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/i/dont/exist 2024-04-17T10:41:58.649 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:41:58.649 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": -2 2024-04-17T10:41:58.649 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:41:58.662 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/i/dont/exist' got response code '0' and stdout '{ "return_code": -2 } ' 2024-04-17T10:41:58.662 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/i/dont/exist' 2024-04-17T10:41:58.662 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-17T10:41:58.662 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/i/dont/exist 2024-04-17T10:41:58.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.787+0000 7f080d4d6640 1 Processor -- start 2024-04-17T10:41:58.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.787+0000 7f080d4d6640 1 -- start start 2024-04-17T10:41:58.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.787+0000 7f080d4d6640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f08080d25b0 0x7f0808168b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.787+0000 7f080d4d6640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08080ce4e0 con 0x7f080815dab0 2024-04-17T10:41:58.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.787+0000 7f080d4d6640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0808162660 con 0x7f080815de90 2024-04-17T10:41:58.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.787+0000 7f080d4d6640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08080cf370 con 0x7f08080d25b0 2024-04-17T10:41:58.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.787+0000 7f0806ffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f080815dab0 0x7f0808161840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60694/0 (socket says 172.21.15.83:60694) 2024-04-17T10:41:58.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.787+0000 7f0806ffd640 1 -- 172.21.15.83:0/622007952 learned_addr learned my addr 172.21.15.83:0/622007952 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:58.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.787+0000 7f0805ffb640 1 -- 172.21.15.83:0/622007952 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 727082299 0 0) 0x7f07f8001120 con 0x7f080815dab0 2024-04-17T10:41:58.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.787+0000 7f0805ffb640 1 -- 172.21.15.83:0/622007952 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07f0003530 con 0x7f080815dab0 2024-04-17T10:41:58.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.787+0000 7f08077fe640 1 --2- 172.21.15.83:0/622007952 >> v2:172.21.15.167:3300/0 conn(0x7f08080d25b0 0x7f0808168b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:58.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.787+0000 7f0805ffb640 1 -- 172.21.15.83:0/622007952 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 569544696 0 0) 0x7f07f0003530 con 0x7f080815dab0 2024-04-17T10:41:58.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.787+0000 7f0805ffb640 1 -- 172.21.15.83:0/622007952 >> v2:172.21.15.167:3300/0 conn(0x7f08080d25b0 msgr2=0x7f0808168b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:58.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.787+0000 7f0805ffb640 1 --2- 172.21.15.83:0/622007952 >> v2:172.21.15.167:3300/0 conn(0x7f08080d25b0 0x7f0808168b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.787+0000 7f0805ffb640 1 -- 172.21.15.83:0/622007952 >> v1:172.21.15.167:6789/0 conn(0x7f080815de90 legacy=0x7f0808161f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:58.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.787+0000 7f0805ffb640 1 -- 172.21.15.83:0/622007952 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08080ce760 con 0x7f080815dab0 2024-04-17T10:41:58.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.787+0000 7f0805ffb640 1 -- 172.21.15.83:0/622007952 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f07f8001850 con 0x7f080815dab0 2024-04-17T10:41:58.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.787+0000 7f0805ffb640 1 -- 172.21.15.83:0/622007952 >> v1:172.21.15.83:6789/0 conn(0x7f080815dab0 legacy=0x7f0808161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:58.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.787+0000 7f0805ffb640 1 --2- 172.21.15.83:0/622007952 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07f0003a30 0x7f07f0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.787+0000 7f0805ffb640 1 --2- 172.21.15.83:0/622007952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07f0004810 0x7f07f0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.787+0000 7f0805ffb640 1 --2- 172.21.15.83:0/622007952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07f0005220 0x7f07f000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.787+0000 7f0805ffb640 1 -- 172.21.15.83:0/622007952 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f000c140 con 0x7f07f0005220 2024-04-17T10:41:58.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.787+0000 7f0805ffb640 1 -- 172.21.15.83:0/622007952 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f000c2f0 con 0x7f07f0004810 2024-04-17T10:41:58.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.787+0000 7f0805ffb640 1 -- 172.21.15.83:0/622007952 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f000c4f0 con 0x7f07f0003a30 2024-04-17T10:41:58.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.787+0000 7f08077fe640 1 --2- 172.21.15.83:0/622007952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07f0005220 0x7f07f000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:58.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.787+0000 7f08067fc640 1 --2- 172.21.15.83:0/622007952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07f0004810 0x7f07f0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:58.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.787+0000 7f0806ffd640 1 --2- 172.21.15.83:0/622007952 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07f0003a30 0x7f07f0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:58.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.787+0000 7f08077fe640 1 -- 172.21.15.83:0/622007952 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07f0003a30 msgr2=0x7f07f0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:58.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.787+0000 7f08077fe640 1 --2- 172.21.15.83:0/622007952 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07f0003a30 0x7f07f0003e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.787+0000 7f08077fe640 1 -- 172.21.15.83:0/622007952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07f0004810 msgr2=0x7f07f0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:58.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.787+0000 7f08077fe640 1 --2- 172.21.15.83:0/622007952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07f0004810 0x7f07f0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.787+0000 7f08077fe640 1 -- 172.21.15.83:0/622007952 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f0003530 con 0x7f07f0005220 2024-04-17T10:41:58.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.791+0000 7f08077fe640 1 -- 172.21.15.83:0/622007952 reap_dead start 2024-04-17T10:41:58.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.791+0000 7f08077fe640 1 --2- 172.21.15.83:0/622007952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07f0005220 0x7f07f000b9d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0800007360 tx=0x7f080001ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b3d2e50ae643a9a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:58.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.791+0000 7f0805ffb640 1 -- 172.21.15.83:0/622007952 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0800009be0 con 0x7f07f0005220 2024-04-17T10:41:58.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.791+0000 7f0805ffb640 1 -- 172.21.15.83:0/622007952 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08000a5dc0 con 0x7f07f0005220 2024-04-17T10:41:58.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.791+0000 7f0805ffb640 1 -- 172.21.15.83:0/622007952 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0800003be0 con 0x7f07f0005220 2024-04-17T10:41:58.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.791+0000 7f080d4d6640 1 -- 172.21.15.83:0/622007952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07f0005220 msgr2=0x7f07f000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:58.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.791+0000 7f080d4d6640 1 --2- 172.21.15.83:0/622007952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07f0005220 0x7f07f000b9d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0800007360 tx=0x7f080001ce80 comp rx=0 tx=0).stop 2024-04-17T10:41:58.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.791+0000 7f080d4d6640 1 -- 172.21.15.83:0/622007952 shutdown_connections 2024-04-17T10:41:58.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.791+0000 7f080d4d6640 1 --2- 172.21.15.83:0/622007952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07f0005220 0x7f07f000b9d0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.791+0000 7f080d4d6640 1 -- 172.21.15.83:0/622007952 >> 172.21.15.83:0/622007952 conn(0x7f080809cd70 msgr2=0x7f08080d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:58.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.791+0000 7f080d4d6640 1 -- 172.21.15.83:0/622007952 shutdown_connections 2024-04-17T10:41:58.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.791+0000 7f080d4d6640 1 -- 172.21.15.83:0/622007952 wait complete. 2024-04-17T10:41:58.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.791+0000 7f080d4d6640 1 Processor -- start 2024-04-17T10:41:58.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.791+0000 7f080d4d6640 1 -- start start 2024-04-17T10:41:58.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.791+0000 7f080d4d6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07f0005220 0x7f08081f7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.791+0000 7f080d4d6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f080815fb20 0x7f080815ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.791+0000 7f080d4d6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08081f0820 0x7f08081f0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.791+0000 7f080d4d6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08080ced20 con 0x7f07f0005220 2024-04-17T10:41:58.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.791+0000 7f080d4d6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08080d4910 con 0x7f080815fb20 2024-04-17T10:41:58.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.791+0000 7f080d4d6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08080cf000 con 0x7f08081f0820 2024-04-17T10:41:58.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.791+0000 7f0806ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07f0005220 0x7f08081f7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:58.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.791+0000 7f0806ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07f0005220 0x7f08081f7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41594/0 (socket says 172.21.15.83:41594) 2024-04-17T10:41:58.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.791+0000 7f0806ffd640 1 -- 172.21.15.83:0/3103545153 learned_addr learned my addr 172.21.15.83:0/3103545153 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:58.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.795+0000 7f08077fe640 1 --2- 172.21.15.83:0/3103545153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08081f0820 0x7f08081f0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:58.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.795+0000 7f08067fc640 1 --2- 172.21.15.83:0/3103545153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f080815fb20 0x7f080815ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:58.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.795+0000 7f0806ffd640 1 -- 172.21.15.83:0/3103545153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08081f0820 msgr2=0x7f08081f0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:58.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.795+0000 7f0806ffd640 1 --2- 172.21.15.83:0/3103545153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08081f0820 0x7f08081f0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.795+0000 7f0806ffd640 1 -- 172.21.15.83:0/3103545153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f080815fb20 msgr2=0x7f080815ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:58.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.795+0000 7f0806ffd640 1 --2- 172.21.15.83:0/3103545153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f080815fb20 0x7f080815ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.795+0000 7f0806ffd640 1 -- 172.21.15.83:0/3103545153 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08081605c0 con 0x7f07f0005220 2024-04-17T10:41:58.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.795+0000 7f08067fc640 1 --2- 172.21.15.83:0/3103545153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f080815fb20 0x7f080815ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:58.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.795+0000 7f0806ffd640 1 --2- 172.21.15.83:0/3103545153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07f0005220 0x7f08081f7060 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f07f800d810 tx=0x7f07f800dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:58.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.795+0000 7f07ef7fe640 1 -- 172.21.15.83:0/3103545153 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f8019070 con 0x7f07f0005220 2024-04-17T10:41:58.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.795+0000 7f07ef7fe640 1 -- 172.21.15.83:0/3103545153 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f80074e0 con 0x7f07f0005220 2024-04-17T10:41:58.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.795+0000 7f080d4d6640 1 -- 172.21.15.83:0/3103545153 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08081608a0 con 0x7f07f0005220 2024-04-17T10:41:58.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.795+0000 7f07ef7fe640 1 -- 172.21.15.83:0/3103545153 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f8004a10 con 0x7f07f0005220 2024-04-17T10:41:58.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.795+0000 7f080d4d6640 1 -- 172.21.15.83:0/3103545153 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0808160e60 con 0x7f07f0005220 2024-04-17T10:41:58.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.795+0000 7f07ef7fe640 1 -- 172.21.15.83:0/3103545153 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f07f801c070 con 0x7f07f0005220 2024-04-17T10:41:58.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.795+0000 7f07ef7fe640 1 --2- 172.21.15.83:0/3103545153 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f07f4066f60 0x7f07f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.795+0000 7f07ef7fe640 1 -- 172.21.15.83:0/3103545153 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f07f8084a60 con 0x7f07f0005220 2024-04-17T10:41:58.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.795+0000 7f08067fc640 1 --2- 172.21.15.83:0/3103545153 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f07f4066f60 0x7f07f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:58.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.795+0000 7f08067fc640 1 --2- 172.21.15.83:0/3103545153 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f07f4066f60 0x7f07f4069420 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f07fc007fd0 tx=0x7f07fc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:58.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.795+0000 7f0810643000 1 Processor -- start 2024-04-17T10:41:58.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.795+0000 7f0810643000 1 -- start start 2024-04-17T10:41:58.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0810643000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc09023970 0x55cc09023d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0810643000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc09024290 0x55cc08fd9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0810643000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cc08fd9cd0 0x55cc08fda0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0810643000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08def8a0 con 0x55cc09024290 2024-04-17T10:41:58.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0810643000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08e27d10 con 0x55cc09023970 2024-04-17T10:41:58.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0810643000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc08db02d0 con 0x55cc08fd9cd0 2024-04-17T10:41:58.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f08067fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc09024290 0x55cc08fd9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:58.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f08067fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc09024290 0x55cc08fd9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41608/0 (socket says 172.21.15.83:41608) 2024-04-17T10:41:58.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f08067fc640 1 -- 172.21.15.83:0/2441745180 learned_addr learned my addr 172.21.15.83:0/2441745180 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:58.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0806ffd640 1 --2- 172.21.15.83:0/2441745180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc09023970 0x55cc09023d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:58.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f08067fc640 1 -- 172.21.15.83:0/2441745180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cc08fd9cd0 msgr2=0x55cc08fda0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:58.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f08067fc640 1 --2- 172.21.15.83:0/2441745180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cc08fd9cd0 0x55cc08fda0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f08067fc640 1 -- 172.21.15.83:0/2441745180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc09023970 msgr2=0x55cc09023d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:58.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f08067fc640 1 --2- 172.21.15.83:0/2441745180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc09023970 0x55cc09023d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f08067fc640 1 -- 172.21.15.83:0/2441745180 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc08e6e870 con 0x55cc09024290 2024-04-17T10:41:58.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f08067fc640 1 --2- 172.21.15.83:0/2441745180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc09024290 0x55cc08fd9790 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f07fc007c90 tx=0x7f07fc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b12174275231453 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:58.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f07ecff9640 1 -- 172.21.15.83:0/2441745180 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc026070 con 0x55cc09024290 2024-04-17T10:41:58.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f07ecff9640 1 -- 172.21.15.83:0/2441745180 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07fc021600 con 0x55cc09024290 2024-04-17T10:41:58.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f07ecff9640 1 -- 172.21.15.83:0/2441745180 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc010040 con 0x55cc09024290 2024-04-17T10:41:58.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0810643000 1 -- 172.21.15.83:0/2441745180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc09024290 msgr2=0x55cc08fd9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:58.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0810643000 1 --2- 172.21.15.83:0/2441745180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc09024290 0x55cc08fd9790 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f07fc007c90 tx=0x7f07fc012000 comp rx=0 tx=0).stop 2024-04-17T10:41:58.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0810643000 1 -- 172.21.15.83:0/2441745180 shutdown_connections 2024-04-17T10:41:58.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0810643000 1 --2- 172.21.15.83:0/2441745180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cc08fd9cd0 0x55cc08fda0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0810643000 1 --2- 172.21.15.83:0/2441745180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc09024290 0x55cc08fd9790 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0810643000 1 --2- 172.21.15.83:0/2441745180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc09023970 0x55cc09023d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0810643000 1 -- 172.21.15.83:0/2441745180 >> 172.21.15.83:0/2441745180 conn(0x55cc08fe0540 msgr2=0x55cc08fe0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:58.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0810643000 1 -- 172.21.15.83:0/2441745180 shutdown_connections 2024-04-17T10:41:58.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0810643000 1 -- 172.21.15.83:0/2441745180 wait complete. 2024-04-17T10:41:58.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0810643000 1 -- start start 2024-04-17T10:41:58.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0810643000 1 Processor -- start 2024-04-17T10:41:58.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0810643000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc08fd9cd0 0x55cc0902acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0810643000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc09023970 0x55cc0902b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0810643000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cc09024290 0x55cc08fdcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0810643000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08de2b50 con 0x55cc08fd9cd0 2024-04-17T10:41:58.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0810643000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08e23db0 con 0x55cc09023970 2024-04-17T10:41:58.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0810643000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc08de5740 con 0x55cc09024290 2024-04-17T10:41:58.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0806ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc08fd9cd0 0x55cc0902acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:58.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0810643000 10 client.0 _ll_register_callbacks cb 0x55cc08edbfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:41:58.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0806ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc08fd9cd0 0x55cc0902acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41614/0 (socket says 172.21.15.83:41614) 2024-04-17T10:41:58.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0806ffd640 1 -- 172.21.15.83:0/2637210254 learned_addr learned my addr 172.21.15.83:0/2637210254 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:58.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0806ffd640 1 -- 172.21.15.83:0/2637210254 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cc09024290 msgr2=0x55cc08fdcfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:58.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0806ffd640 1 --2- 172.21.15.83:0/2637210254 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cc09024290 0x55cc08fdcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0806ffd640 1 -- 172.21.15.83:0/2637210254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc09023970 msgr2=0x55cc0902b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:58.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0806ffd640 1 --2- 172.21.15.83:0/2637210254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc09023970 0x55cc0902b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.799+0000 7f0806ffd640 1 -- 172.21.15.83:0/2637210254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc08db0520 con 0x55cc08fd9cd0 2024-04-17T10:41:58.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f0806ffd640 1 --2- 172.21.15.83:0/2637210254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc08fd9cd0 0x55cc0902acd0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f07f8017040 tx=0x7f07f8054d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95512e1bb4120ccd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:58.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f07e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:41:58.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f07e5ffb640 1 -- 172.21.15.83:0/2637210254 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f8059510 con 0x55cc08fd9cd0 2024-04-17T10:41:58.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f07e5ffb640 1 -- 172.21.15.83:0/2637210254 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f07f8004bb0 con 0x55cc08fd9cd0 2024-04-17T10:41:58.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f0810643000 1 -- 172.21.15.83:0/2637210254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f07fc021600 con 0x55cc08fd9cd0 2024-04-17T10:41:58.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f07e5ffb640 1 -- 172.21.15.83:0/2637210254 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07f805dad0 con 0x55cc08fd9cd0 2024-04-17T10:41:58.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f0810643000 10 client.5862 fetch_fsmap learned FSMap version 21 2024-04-17T10:41:58.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f0810643000 1 -- 172.21.15.83:0/2637210254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55cc08e22d00 con 0x55cc08fd9cd0 2024-04-17T10:41:58.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f07e5ffb640 1 -- 172.21.15.83:0/2637210254 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07fc021600 con 0x55cc08fd9cd0 2024-04-17T10:41:58.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f07e5ffb640 1 -- 172.21.15.83:0/2637210254 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f07f8054810 con 0x55cc08fd9cd0 2024-04-17T10:41:58.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f0810643000 10 client.5862 fetch_fsmap finished waiting for FSMap version 21 2024-04-17T10:41:58.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f0810643000 10 client.5862 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:41:58.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f0810643000 20 client.5862 populate_metadata read hostname 'smithi083' 2024-04-17T10:41:58.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f0810643000 1 --2- 172.21.15.83:0/2637210254 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55cc08fddd60 0x55cc08fde170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f0810643000 4 client.5862 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:41:58.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f0810643000 1 -- 172.21.15.83:0/2637210254 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc08dc3b10 con 0x55cc08fddd60 2024-04-17T10:41:58.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f08067fc640 1 --2- 172.21.15.83:0/2637210254 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55cc08fddd60 0x55cc08fde170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:58.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f08067fc640 1 --2- 172.21.15.83:0/2637210254 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55cc08fddd60 0x55cc08fde170 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14a889c03735a9fd server_cookie=24129da3f40d25b5 in_seq=0 out_seq=0 2024-04-17T10:41:58.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f07e5ffb640 10 client.5862 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:41:58.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f07e5ffb640 1 -- 172.21.15.83:0/2637210254 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cc08dc3b10 con 0x55cc08fddd60 2024-04-17T10:41:58.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f07e5ffb640 10 client.5862 handle_command_reply: tid=0 2024-04-17T10:41:58.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f0810643000 1 client.5862 shutdown 2024-04-17T10:41:58.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f0810643000 1 -- 172.21.15.83:0/2637210254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc08fd9cd0 msgr2=0x55cc0902acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:58.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f0810643000 1 --2- 172.21.15.83:0/2637210254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc08fd9cd0 0x55cc0902acd0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f07f8017040 tx=0x7f07f8054d20 comp rx=0 tx=0).stop 2024-04-17T10:41:58.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f0810643000 1 -- 172.21.15.83:0/2637210254 shutdown_connections 2024-04-17T10:41:58.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f0810643000 1 --2- 172.21.15.83:0/2637210254 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55cc08fddd60 0x55cc08fde170 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f0810643000 1 --2- 172.21.15.83:0/2637210254 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cc09024290 0x55cc08fdcfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f0810643000 1 --2- 172.21.15.83:0/2637210254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc09023970 0x55cc0902b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f0810643000 1 --2- 172.21.15.83:0/2637210254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc08fd9cd0 0x55cc0902acd0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f0810643000 1 -- 172.21.15.83:0/2637210254 >> 172.21.15.83:0/2637210254 conn(0x55cc08fe0540 msgr2=0x55cc08fd6770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:58.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f07e5ffb640 0 client.5862 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:41:58.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f0810643000 1 -- 172.21.15.83:0/2637210254 shutdown_connections 2024-04-17T10:41:58.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f0810643000 1 -- 172.21.15.83:0/2637210254 wait complete. 2024-04-17T10:41:58.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.803+0000 7f0810643000 20 client.5862 trim_cache size 0 max 16384 2024-04-17T10:41:58.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0810643000 1 Processor -- start 2024-04-17T10:41:58.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0810643000 1 -- start start 2024-04-17T10:41:58.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0810643000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc08fd9cd0 0x55cc0907f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0810643000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc09023970 0x55cc0907fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0810643000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cc09024290 0x55cc09080400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0810643000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0808200210 con 0x55cc08fd9cd0 2024-04-17T10:41:58.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0810643000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08081ff440 con 0x55cc09023970 2024-04-17T10:41:58.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0810643000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08080cb970 con 0x55cc09024290 2024-04-17T10:41:58.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0806ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc08fd9cd0 0x55cc0907f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:58.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0806ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc08fd9cd0 0x55cc0907f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41616/0 (socket says 172.21.15.83:41616) 2024-04-17T10:41:58.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0806ffd640 1 -- 172.21.15.83:0/768921288 learned_addr learned my addr 172.21.15.83:0/768921288 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:58.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0806ffd640 1 -- 172.21.15.83:0/768921288 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cc09024290 msgr2=0x55cc09080400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:58.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0806ffd640 1 --2- 172.21.15.83:0/768921288 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cc09024290 0x55cc09080400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0806ffd640 1 -- 172.21.15.83:0/768921288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc09023970 msgr2=0x55cc0907fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:58.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0806ffd640 1 --2- 172.21.15.83:0/768921288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc09023970 0x55cc0907fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0806ffd640 1 -- 172.21.15.83:0/768921288 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc08eab920 con 0x55cc08fd9cd0 2024-04-17T10:41:58.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0806ffd640 1 --2- 172.21.15.83:0/768921288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc08fd9cd0 0x55cc0907f980 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f07f8084d80 tx=0x7f07f8058000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0219cb39ce9e83e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:58.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f07e7fff640 1 -- 172.21.15.83:0/768921288 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f8055190 con 0x55cc08fd9cd0 2024-04-17T10:41:58.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f07e7fff640 1 -- 172.21.15.83:0/768921288 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f8059510 con 0x55cc08fd9cd0 2024-04-17T10:41:58.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f07e7fff640 1 -- 172.21.15.83:0/768921288 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f80732c0 con 0x55cc08fd9cd0 2024-04-17T10:41:58.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0810643000 1 -- 172.21.15.83:0/768921288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc08fd9cd0 msgr2=0x55cc0907f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:58.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0810643000 1 --2- 172.21.15.83:0/768921288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc08fd9cd0 0x55cc0907f980 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f07f8084d80 tx=0x7f07f8058000 comp rx=0 tx=0).stop 2024-04-17T10:41:58.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0810643000 1 -- 172.21.15.83:0/768921288 shutdown_connections 2024-04-17T10:41:58.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0810643000 1 --2- 172.21.15.83:0/768921288 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cc09024290 0x55cc09080400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0810643000 1 --2- 172.21.15.83:0/768921288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc09023970 0x55cc0907fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0810643000 1 --2- 172.21.15.83:0/768921288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc08fd9cd0 0x55cc0907f980 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0810643000 1 -- 172.21.15.83:0/768921288 >> 172.21.15.83:0/768921288 conn(0x55cc08fe0540 msgr2=0x55cc08fd7ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:58.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0810643000 1 -- 172.21.15.83:0/768921288 shutdown_connections 2024-04-17T10:41:58.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0810643000 1 -- 172.21.15.83:0/768921288 wait complete. 2024-04-17T10:41:58.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0810643000 1 -- start start 2024-04-17T10:41:58.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0810643000 1 Processor -- start 2024-04-17T10:41:58.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0810643000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc08fd9cd0 0x55cc09083e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0810643000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cc09023970 0x55cc09084360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0810643000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc09024290 0x55cc090289a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0810643000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08fe3ad0 con 0x55cc09024290 2024-04-17T10:41:58.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0810643000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08080cbf70 con 0x55cc08fd9cd0 2024-04-17T10:41:58.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0810643000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc08de9700 con 0x55cc09023970 2024-04-17T10:41:58.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0810643000 10 client.0 _ll_register_callbacks cb 0x7f07fc007d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:41:58.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f08077fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc09024290 0x55cc090289a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:58.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f08077fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc09024290 0x55cc090289a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41626/0 (socket says 172.21.15.83:41626) 2024-04-17T10:41:58.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f08077fe640 1 -- 172.21.15.83:0/2111804123 learned_addr learned my addr 172.21.15.83:0/2111804123 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:58.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f08067fc640 1 --2- 172.21.15.83:0/2111804123 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cc09023970 0x55cc09084360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:58.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f0806ffd640 1 --2- 172.21.15.83:0/2111804123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc08fd9cd0 0x55cc09083e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:58.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f08077fe640 1 -- 172.21.15.83:0/2111804123 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cc09023970 msgr2=0x55cc09084360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:58.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f08077fe640 1 --2- 172.21.15.83:0/2111804123 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cc09023970 0x55cc09084360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f08077fe640 1 -- 172.21.15.83:0/2111804123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc08fd9cd0 msgr2=0x55cc09083e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:58.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f08077fe640 1 --2- 172.21.15.83:0/2111804123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc08fd9cd0 0x55cc09083e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.819+0000 7f08077fe640 1 -- 172.21.15.83:0/2111804123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc08e81a30 con 0x55cc09024290 2024-04-17T10:41:58.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f08077fe640 1 --2- 172.21.15.83:0/2111804123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc09024290 0x55cc090289a0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f080001c8f0 tx=0x7f08000a7fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d843f0cf47ea328 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:58.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f07e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:41:58.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f07e5ffb640 1 -- 172.21.15.83:0/2111804123 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08000bb060 con 0x55cc09024290 2024-04-17T10:41:58.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f07e5ffb640 1 -- 172.21.15.83:0/2111804123 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f08000a5b80 con 0x55cc09024290 2024-04-17T10:41:58.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f0810643000 1 -- 172.21.15.83:0/2111804123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f07f8059510 con 0x55cc09024290 2024-04-17T10:41:58.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f07e5ffb640 1 -- 172.21.15.83:0/2111804123 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08000c1050 con 0x55cc09024290 2024-04-17T10:41:58.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f0810643000 10 client.5868 fetch_fsmap learned FSMap version 21 2024-04-17T10:41:58.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f0810643000 1 -- 172.21.15.83:0/2111804123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55cc08dcf150 con 0x55cc09024290 2024-04-17T10:41:58.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f07e5ffb640 1 -- 172.21.15.83:0/2111804123 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f8059510 con 0x55cc09024290 2024-04-17T10:41:58.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f07e5ffb640 1 -- 172.21.15.83:0/2111804123 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f08000c4020 con 0x55cc09024290 2024-04-17T10:41:58.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f0810643000 10 client.5868 fetch_fsmap finished waiting for FSMap version 21 2024-04-17T10:41:58.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f0810643000 10 client.5868 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:41:58.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f0810643000 20 client.5868 populate_metadata read hostname 'smithi083' 2024-04-17T10:41:58.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f0810643000 1 --2- 172.21.15.83:0/2111804123 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55cc09029760 0x55cc09029b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f0810643000 4 client.5868 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}] 2024-04-17T10:41:58.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f0810643000 1 -- 172.21.15.83:0/2111804123 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}) v1 -- 0x55cc08e35250 con 0x55cc09029760 2024-04-17T10:41:58.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f0806ffd640 1 --2- 172.21.15.83:0/2111804123 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55cc09029760 0x55cc09029b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:58.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f07e5ffb640 10 client.5868 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:41:58.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f0806ffd640 1 --2- 172.21.15.83:0/2111804123 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55cc09029760 0x55cc09029b70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d30e864b70eedd4e server_cookie=1e7366c5daf42d2 in_seq=0 out_seq=0 2024-04-17T10:41:58.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f07e5ffb640 1 -- 172.21.15.83:0/2111804123 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26 (crc 0 0 0) 0x55cc08e35250 con 0x55cc09029760 2024-04-17T10:41:58.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f07e5ffb640 10 client.5868 handle_command_reply: tid=0 2024-04-17T10:41:58.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f0810643000 1 client.5868 shutdown 2024-04-17T10:41:58.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f0810643000 1 -- 172.21.15.83:0/2111804123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc09024290 msgr2=0x55cc090289a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:58.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f0810643000 1 --2- 172.21.15.83:0/2111804123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc09024290 0x55cc090289a0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f080001c8f0 tx=0x7f08000a7fc0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f0810643000 1 -- 172.21.15.83:0/2111804123 shutdown_connections 2024-04-17T10:41:58.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f0810643000 1 --2- 172.21.15.83:0/2111804123 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55cc09029760 0x55cc09029b70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f0810643000 1 --2- 172.21.15.83:0/2111804123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc09024290 0x55cc090289a0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f0810643000 1 --2- 172.21.15.83:0/2111804123 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cc09023970 0x55cc09084360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f07e5ffb640 0 client.5868 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:41:58.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f0810643000 1 --2- 172.21.15.83:0/2111804123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc08fd9cd0 0x55cc09083e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f0810643000 1 -- 172.21.15.83:0/2111804123 >> 172.21.15.83:0/2111804123 conn(0x55cc08fe0540 msgr2=0x55cc08fd8780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:58.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f0810643000 1 -- 172.21.15.83:0/2111804123 shutdown_connections 2024-04-17T10:41:58.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f0810643000 1 -- 172.21.15.83:0/2111804123 wait complete. 2024-04-17T10:41:58.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f0810643000 20 client.5868 trim_cache size 0 max 16384 2024-04-17T10:41:58.826 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:41:58.827 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": -2 2024-04-17T10:41:58.827 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:41:58.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f080d4d6640 1 -- 172.21.15.83:0/3103545153 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f07f4066f60 msgr2=0x7f07f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:58.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f080d4d6640 1 --2- 172.21.15.83:0/3103545153 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f07f4066f60 0x7f07f4069420 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f07fc007fd0 tx=0x7f07fc009040 comp rx=0 tx=0).stop 2024-04-17T10:41:58.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f080d4d6640 1 -- 172.21.15.83:0/3103545153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07f0005220 msgr2=0x7f08081f7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:58.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f080d4d6640 1 --2- 172.21.15.83:0/3103545153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07f0005220 0x7f08081f7060 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f07f800d810 tx=0x7f07f800dcd0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f080d4d6640 1 -- 172.21.15.83:0/3103545153 shutdown_connections 2024-04-17T10:41:58.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f080d4d6640 1 --2- 172.21.15.83:0/3103545153 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f07f4066f60 0x7f07f4069420 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f080d4d6640 1 --2- 172.21.15.83:0/3103545153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08081f0820 0x7f08081f0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f080d4d6640 1 --2- 172.21.15.83:0/3103545153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f080815fb20 0x7f080815ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f080d4d6640 1 --2- 172.21.15.83:0/3103545153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07f0005220 0x7f08081f7060 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f080d4d6640 1 -- 172.21.15.83:0/3103545153 >> 172.21.15.83:0/3103545153 conn(0x7f080809cd70 msgr2=0x7f0808161580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:58.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f080d4d6640 1 -- 172.21.15.83:0/3103545153 shutdown_connections 2024-04-17T10:41:58.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.823+0000 7f080d4d6640 1 -- 172.21.15.83:0/3103545153 wait complete. 2024-04-17T10:41:58.836 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/i/dont/exist' returned '{'return_code': -2}' 2024-04-17T10:41:58.836 INFO:tasks.cephfs.test_scrub_checks:First run: flushing /scrub_test_path/ceph-qa-suite/suites 2024-04-17T10:41:58.836 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/suites' 2024-04-17T10:41:58.837 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:41:58.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.943+0000 7f4a119cc640 1 Processor -- start 2024-04-17T10:41:58.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.943+0000 7f4a119cc640 1 -- start start 2024-04-17T10:41:58.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.943+0000 7f4a119cc640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4a0c15de30 0x7f4a0c0ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.943+0000 7f4a119cc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4a0c0ce860 0x7f4a0c0d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.943+0000 7f4a119cc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a0c0b6cd0 0x7f4a0c0d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.943+0000 7f4a119cc640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a0c15e210 con 0x7f4a0c0ce860 2024-04-17T10:41:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.943+0000 7f4a119cc640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a0c0ced00 con 0x7f4a0c15de30 2024-04-17T10:41:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.943+0000 7f4a119cc640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c0cefe0 con 0x7f4a0c0b6cd0 2024-04-17T10:41:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.947+0000 7f4a0a7fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4a0c0ce860 0x7f4a0c0d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.947+0000 7f4a0a7fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4a0c0ce860 0x7f4a0c0d02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41638/0 (socket says 172.21.15.83:41638) 2024-04-17T10:41:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.947+0000 7f4a0a7fc640 1 -- 172.21.15.83:0/110315946 learned_addr learned my addr 172.21.15.83:0/110315946 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.947+0000 7f4a0affd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4a0c15de30 0x7f4a0c0ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.947+0000 7f4a0b7fe640 1 --2- 172.21.15.83:0/110315946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a0c0b6cd0 0x7f4a0c0d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.947+0000 7f4a0a7fc640 1 -- 172.21.15.83:0/110315946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a0c0b6cd0 msgr2=0x7f4a0c0d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.947+0000 7f4a0a7fc640 1 --2- 172.21.15.83:0/110315946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a0c0b6cd0 0x7f4a0c0d0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.947+0000 7f4a0a7fc640 1 -- 172.21.15.83:0/110315946 >> v2:172.21.15.167:3300/0 conn(0x7f4a0c15de30 msgr2=0x7f4a0c0ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.947+0000 7f4a0a7fc640 1 --2- 172.21.15.83:0/110315946 >> v2:172.21.15.167:3300/0 conn(0x7f4a0c15de30 0x7f4a0c0ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.947+0000 7f4a0a7fc640 1 -- 172.21.15.83:0/110315946 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a0c0d2040 con 0x7f4a0c0ce860 2024-04-17T10:41:58.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.947+0000 7f4a0b7fe640 1 --2- 172.21.15.83:0/110315946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a0c0b6cd0 0x7f4a0c0d0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.947+0000 7f4a0a7fc640 1 --2- 172.21.15.83:0/110315946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a0c0ce860 0x7f4a0c0d02d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c08c650 tx=0x7f4a0001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ec40fbaf8dd22a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.947+0000 7f4a09ffb640 1 -- 172.21.15.83:0/110315946 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a000b0070 con 0x7f4a0c0ce860 2024-04-17T10:41:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.947+0000 7f4a09ffb640 1 -- 172.21.15.83:0/110315946 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a000ab600 con 0x7f4a0c0ce860 2024-04-17T10:41:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.947+0000 7f4a09ffb640 1 -- 172.21.15.83:0/110315946 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a000ab9d0 con 0x7f4a0c0ce860 2024-04-17T10:41:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.947+0000 7f4a119cc640 1 -- 172.21.15.83:0/110315946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a0c0ce860 msgr2=0x7f4a0c0d02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:58.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.947+0000 7f4a119cc640 1 --2- 172.21.15.83:0/110315946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a0c0ce860 0x7f4a0c0d02d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c08c650 tx=0x7f4a0001c490 comp rx=0 tx=0).stop 2024-04-17T10:41:58.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.947+0000 7f4a119cc640 1 -- 172.21.15.83:0/110315946 shutdown_connections 2024-04-17T10:41:58.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.947+0000 7f4a119cc640 1 --2- 172.21.15.83:0/110315946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a0c0b6cd0 0x7f4a0c0d0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.947+0000 7f4a119cc640 1 --2- 172.21.15.83:0/110315946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a0c0ce860 0x7f4a0c0d02d0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.947+0000 7f4a119cc640 1 --2- 172.21.15.83:0/110315946 >> v2:172.21.15.167:3300/0 conn(0x7f4a0c15de30 0x7f4a0c0ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.947+0000 7f4a119cc640 1 -- 172.21.15.83:0/110315946 >> 172.21.15.83:0/110315946 conn(0x7f4a0c06b0b0 msgr2=0x7f4a0c06b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:58.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.947+0000 7f4a119cc640 1 -- 172.21.15.83:0/110315946 shutdown_connections 2024-04-17T10:41:58.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.947+0000 7f4a119cc640 1 -- 172.21.15.83:0/110315946 wait complete. 2024-04-17T10:41:58.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.951+0000 7f4a119cc640 1 Processor -- start 2024-04-17T10:41:58.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.951+0000 7f4a119cc640 1 -- start start 2024-04-17T10:41:58.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.951+0000 7f4a119cc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a0c0b6cd0 0x7f4a0c0c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.951+0000 7f4a119cc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a0c0ce860 0x7f4a0c156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.951+0000 7f4a119cc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a0c15de30 0x7f4a0c0ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.951+0000 7f4a119cc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c0b5cc0 con 0x7f4a0c0ce860 2024-04-17T10:41:58.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.951+0000 7f4a119cc640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c161c00 con 0x7f4a0c0b6cd0 2024-04-17T10:41:58.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.951+0000 7f4a119cc640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c0b5e40 con 0x7f4a0c15de30 2024-04-17T10:41:58.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.951+0000 7f4a0a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a0c0ce860 0x7f4a0c156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.951+0000 7f4a0a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a0c0ce860 0x7f4a0c156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41646/0 (socket says 172.21.15.83:41646) 2024-04-17T10:41:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.951+0000 7f4a0a7fc640 1 -- 172.21.15.83:0/3146616879 learned_addr learned my addr 172.21.15.83:0/3146616879 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.951+0000 7f4a0b7fe640 1 --2- 172.21.15.83:0/3146616879 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a0c15de30 0x7f4a0c0ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.951+0000 7f4a0affd640 1 --2- 172.21.15.83:0/3146616879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a0c0b6cd0 0x7f4a0c0c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.951+0000 7f4a0a7fc640 1 -- 172.21.15.83:0/3146616879 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a0c15de30 msgr2=0x7f4a0c0ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.951+0000 7f4a0a7fc640 1 --2- 172.21.15.83:0/3146616879 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a0c15de30 0x7f4a0c0ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.951+0000 7f4a0a7fc640 1 -- 172.21.15.83:0/3146616879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a0c0b6cd0 msgr2=0x7f4a0c0c9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.951+0000 7f4a0a7fc640 1 --2- 172.21.15.83:0/3146616879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a0c0b6cd0 0x7f4a0c0c9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.951+0000 7f4a0a7fc640 1 -- 172.21.15.83:0/3146616879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a0c0cbb50 con 0x7f4a0c0ce860 2024-04-17T10:41:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.951+0000 7f4a0a7fc640 1 --2- 172.21.15.83:0/3146616879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a0c0ce860 0x7f4a0c156680 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4a0001ab40 tx=0x7f4a00019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.951+0000 7f49f3fff640 1 -- 172.21.15.83:0/3146616879 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a000b0040 con 0x7f4a0c0ce860 2024-04-17T10:41:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.951+0000 7f49f3fff640 1 -- 172.21.15.83:0/3146616879 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a000ab400 con 0x7f4a0c0ce860 2024-04-17T10:41:58.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.951+0000 7f49f3fff640 1 -- 172.21.15.83:0/3146616879 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a000b4660 con 0x7f4a0c0ce860 2024-04-17T10:41:58.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.951+0000 7f4a119cc640 1 -- 172.21.15.83:0/3146616879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a0c0cbe30 con 0x7f4a0c0ce860 2024-04-17T10:41:58.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.951+0000 7f4a119cc640 1 -- 172.21.15.83:0/3146616879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a0c0cc3f0 con 0x7f4a0c0ce860 2024-04-17T10:41:58.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.951+0000 7f4a119cc640 1 -- 172.21.15.83:0/3146616879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a0c074f90 con 0x7f4a0c0ce860 2024-04-17T10:41:58.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.951+0000 7f49f3fff640 1 -- 172.21.15.83:0/3146616879 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4a000bb020 con 0x7f4a0c0ce860 2024-04-17T10:41:58.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.955+0000 7f49f3fff640 1 --2- 172.21.15.83:0/3146616879 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f49f8066f60 0x7f49f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:58.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.955+0000 7f49f3fff640 1 -- 172.21.15.83:0/3146616879 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4a0011e520 con 0x7f4a0c0ce860 2024-04-17T10:41:58.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.955+0000 7f4a0affd640 1 --2- 172.21.15.83:0/3146616879 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f49f8066f60 0x7f49f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:58.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.955+0000 7f4a0affd640 1 --2- 172.21.15.83:0/3146616879 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f49f8066f60 0x7f49f8069420 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c0cdc00 tx=0x7f49fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:58.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:58.959+0000 7f49f3fff640 1 -- 172.21.15.83:0/3146616879 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4a000c2020 con 0x7f4a0c0ce860 2024-04-17T10:41:59.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.151+0000 7f4a119cc640 1 -- 172.21.15.83:0/3146616879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a0c0d28c0 con 0x7f4a0c0ce860 2024-04-17T10:41:59.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.151+0000 7f49f3fff640 1 -- 172.21.15.83:0/3146616879 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f4a00016110 con 0x7f4a0c0ce860 2024-04-17T10:41:59.149 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:41:59.149 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/161398126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":161398126},{"type":"v1","addr":"172.21.15.167:6833","nonce":161398126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1085380618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1085380618},{"type":"v1","addr":"172.21.15.167:6835","nonce":1085380618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/836442291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":836442291},{"type":"v1","addr":"172.21.15.83:6835","nonce":836442291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:41:37.773342+0000","modified":"2024-04-17T10:41:39.788394+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5682":{"gid":5682,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1894547386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1894547386},{"type":"v1","addr":"172.21.15.83:6837","nonce":1894547386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-17T10:41:59.149 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 21 2024-04-17T10:41:59.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.151+0000 7f4a119cc640 1 -- 172.21.15.83:0/3146616879 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f49f8066f60 msgr2=0x7f49f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:59.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.151+0000 7f4a119cc640 1 --2- 172.21.15.83:0/3146616879 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f49f8066f60 0x7f49f8069420 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c0cdc00 tx=0x7f49fc008040 comp rx=0 tx=0).stop 2024-04-17T10:41:59.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.151+0000 7f4a119cc640 1 -- 172.21.15.83:0/3146616879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a0c0ce860 msgr2=0x7f4a0c156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:59.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.151+0000 7f4a119cc640 1 --2- 172.21.15.83:0/3146616879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a0c0ce860 0x7f4a0c156680 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4a0001ab40 tx=0x7f4a00019230 comp rx=0 tx=0).stop 2024-04-17T10:41:59.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.151+0000 7f4a119cc640 1 -- 172.21.15.83:0/3146616879 shutdown_connections 2024-04-17T10:41:59.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.151+0000 7f4a119cc640 1 --2- 172.21.15.83:0/3146616879 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f49f8066f60 0x7f49f8069420 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.151+0000 7f4a119cc640 1 --2- 172.21.15.83:0/3146616879 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a0c15de30 0x7f4a0c0ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.151+0000 7f4a119cc640 1 --2- 172.21.15.83:0/3146616879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a0c0ce860 0x7f4a0c156680 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.151+0000 7f4a119cc640 1 --2- 172.21.15.83:0/3146616879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a0c0b6cd0 0x7f4a0c0c9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.151+0000 7f4a119cc640 1 -- 172.21.15.83:0/3146616879 >> 172.21.15.83:0/3146616879 conn(0x7f4a0c06b0b0 msgr2=0x7f4a0c0d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:59.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.151+0000 7f4a119cc640 1 -- 172.21.15.83:0/3146616879 shutdown_connections 2024-04-17T10:41:59.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.151+0000 7f4a119cc640 1 -- 172.21.15.83:0/3146616879 wait complete. 2024-04-17T10:41:59.168 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/ceph-qa-suite/suites 2024-04-17T10:41:59.299 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:41:59.299 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0 2024-04-17T10:41:59.300 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:41:59.319 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/suites' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-17T10:41:59.319 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites' 2024-04-17T10:41:59.319 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-17T10:41:59.319 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites 2024-04-17T10:41:59.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f8755ac0640 1 Processor -- start 2024-04-17T10:41:59.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f8755ac0640 1 -- start start 2024-04-17T10:41:59.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f8755ac0640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f875015a8a0 0x7f875006e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:59.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f8755ac0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87500b36f0 0x7f875006bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:59.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f8755ac0640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87500afbb0 con 0x7f875006c230 2024-04-17T10:41:59.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f8755ac0640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f875006d760 con 0x7f875015a8a0 2024-04-17T10:41:59.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f8755ac0640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f875006da40 con 0x7f87500b36f0 2024-04-17T10:41:59.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f874ffff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f875006c230 0x7f87500cd720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60704/0 (socket says 172.21.15.83:60704) 2024-04-17T10:41:59.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f874ffff640 1 -- 172.21.15.83:0/3866976259 learned_addr learned my addr 172.21.15.83:0/3866976259 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:41:59.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f874e7fc640 1 -- 172.21.15.83:0/3866976259 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3716853887 0 0) 0x7f87500afbb0 con 0x7f875006c230 2024-04-17T10:41:59.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f874e7fc640 1 -- 172.21.15.83:0/3866976259 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8738003530 con 0x7f875006c230 2024-04-17T10:41:59.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f874effd640 1 --2- 172.21.15.83:0/3866976259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87500b36f0 0x7f875006bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:59.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f874f7fe640 1 --2- 172.21.15.83:0/3866976259 >> v2:172.21.15.167:3300/0 conn(0x7f875015a8a0 0x7f875006e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:59.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f874e7fc640 1 -- 172.21.15.83:0/3866976259 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1310798724 0 0) 0x7f8738003530 con 0x7f875006c230 2024-04-17T10:41:59.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f874e7fc640 1 -- 172.21.15.83:0/3866976259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87500b36f0 msgr2=0x7f875006bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:59.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f874e7fc640 1 --2- 172.21.15.83:0/3866976259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87500b36f0 0x7f875006bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f874e7fc640 1 -- 172.21.15.83:0/3866976259 >> v2:172.21.15.167:3300/0 conn(0x7f875015a8a0 msgr2=0x7f875006e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f874e7fc640 1 --2- 172.21.15.83:0/3866976259 >> v2:172.21.15.167:3300/0 conn(0x7f875015a8a0 0x7f875006e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f874e7fc640 1 -- 172.21.15.83:0/3866976259 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875006dd20 con 0x7f875006c230 2024-04-17T10:41:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f874e7fc640 1 -- 172.21.15.83:0/3866976259 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f8748001a00 con 0x7f875006c230 2024-04-17T10:41:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f874e7fc640 1 -- 172.21.15.83:0/3866976259 >> v1:172.21.15.83:6789/0 conn(0x7f875006c230 legacy=0x7f87500cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f874e7fc640 1 --2- 172.21.15.83:0/3866976259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8738003a30 0x7f8738003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f874e7fc640 1 --2- 172.21.15.83:0/3866976259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8738004810 0x7f8738004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f874e7fc640 1 --2- 172.21.15.83:0/3866976259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87500b36f0 0x7f87380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f874e7fc640 1 -- 172.21.15.83:0/3866976259 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875006d760 con 0x7f8738004810 2024-04-17T10:41:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f874e7fc640 1 -- 172.21.15.83:0/3866976259 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875006da40 con 0x7f8738003a30 2024-04-17T10:41:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f874e7fc640 1 -- 172.21.15.83:0/3866976259 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f873800bb30 con 0x7f87500b36f0 2024-04-17T10:41:59.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f874effd640 1 --2- 172.21.15.83:0/3866976259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8738004810 0x7f8738004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:59.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f874effd640 1 -- 172.21.15.83:0/3866976259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87500b36f0 msgr2=0x7f87380051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:59.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f874effd640 1 --2- 172.21.15.83:0/3866976259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87500b36f0 0x7f87380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f874effd640 1 -- 172.21.15.83:0/3866976259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8738003a30 msgr2=0x7f8738003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:59.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f874effd640 1 --2- 172.21.15.83:0/3866976259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8738003a30 0x7f8738003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.439+0000 7f874effd640 1 -- 172.21.15.83:0/3866976259 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8738003530 con 0x7f8738004810 2024-04-17T10:41:59.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.443+0000 7f874effd640 1 --2- 172.21.15.83:0/3866976259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8738004810 0x7f8738004c90 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f874400ba00 tx=0x7f874401c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5498460edfdf4ab5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:59.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.443+0000 7f874e7fc640 1 -- 172.21.15.83:0/3866976259 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87440ae070 con 0x7f8738004810 2024-04-17T10:41:59.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.443+0000 7f874e7fc640 1 -- 172.21.15.83:0/3866976259 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87440a9660 con 0x7f8738004810 2024-04-17T10:41:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.443+0000 7f874e7fc640 1 -- 172.21.15.83:0/3866976259 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874400b600 con 0x7f8738004810 2024-04-17T10:41:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.443+0000 7f8755ac0640 1 -- 172.21.15.83:0/3866976259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8738004810 msgr2=0x7f8738004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.443+0000 7f8755ac0640 1 --2- 172.21.15.83:0/3866976259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8738004810 0x7f8738004c90 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f874400ba00 tx=0x7f874401c680 comp rx=0 tx=0).stop 2024-04-17T10:41:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.443+0000 7f874ffff640 1 -- 172.21.15.83:0/3866976259 reap_dead start 2024-04-17T10:41:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.443+0000 7f8755ac0640 1 -- 172.21.15.83:0/3866976259 shutdown_connections 2024-04-17T10:41:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.443+0000 7f8755ac0640 1 -- 172.21.15.83:0/3866976259 >> 172.21.15.83:0/3866976259 conn(0x7f87500b23f0 msgr2=0x7f87500b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.443+0000 7f8755ac0640 1 -- 172.21.15.83:0/3866976259 shutdown_connections 2024-04-17T10:41:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.443+0000 7f8755ac0640 1 -- 172.21.15.83:0/3866976259 wait complete. 2024-04-17T10:41:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.443+0000 7f8755ac0640 1 Processor -- start 2024-04-17T10:41:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.443+0000 7f8755ac0640 1 -- start start 2024-04-17T10:41:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.443+0000 7f8755ac0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8738004810 0x7f875013b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.443+0000 7f8755ac0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f875013bb40 0x7f8750146dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.443+0000 7f8755ac0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87501447a0 0x7f875015b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:59.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.443+0000 7f8755ac0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875006d3f0 con 0x7f8738004810 2024-04-17T10:41:59.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.443+0000 7f8755ac0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875006e2e0 con 0x7f87501447a0 2024-04-17T10:41:59.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.443+0000 7f8755ac0640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87500b3b50 con 0x7f875013bb40 2024-04-17T10:41:59.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.443+0000 7f874f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8738004810 0x7f875013b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:59.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.443+0000 7f874f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8738004810 0x7f875013b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41678/0 (socket says 172.21.15.83:41678) 2024-04-17T10:41:59.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.443+0000 7f874f7fe640 1 -- 172.21.15.83:0/1649609811 learned_addr learned my addr 172.21.15.83:0/1649609811 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:59.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.443+0000 7f874f7fe640 1 -- 172.21.15.83:0/1649609811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f875013bb40 msgr2=0x7f8750146dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:59.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.443+0000 7f874f7fe640 1 --2- 172.21.15.83:0/1649609811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f875013bb40 0x7f8750146dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.443+0000 7f874f7fe640 1 -- 172.21.15.83:0/1649609811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87501447a0 msgr2=0x7f875015b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:59.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.443+0000 7f874f7fe640 1 --2- 172.21.15.83:0/1649609811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87501447a0 0x7f875015b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.443+0000 7f874f7fe640 1 -- 172.21.15.83:0/1649609811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875013e5f0 con 0x7f8738004810 2024-04-17T10:41:59.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.447+0000 7f874f7fe640 1 --2- 172.21.15.83:0/1649609811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8738004810 0x7f875013b510 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f874000d830 tx=0x7f874000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:59.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.447+0000 7f8737fff640 1 -- 172.21.15.83:0/1649609811 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874000b6c0 con 0x7f8738004810 2024-04-17T10:41:59.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.447+0000 7f8737fff640 1 -- 172.21.15.83:0/1649609811 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8740019070 con 0x7f8738004810 2024-04-17T10:41:59.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.447+0000 7f8755ac0640 1 -- 172.21.15.83:0/1649609811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f875013e8d0 con 0x7f8738004810 2024-04-17T10:41:59.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.447+0000 7f8737fff640 1 -- 172.21.15.83:0/1649609811 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874000bac0 con 0x7f8738004810 2024-04-17T10:41:59.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.447+0000 7f8755ac0640 1 -- 172.21.15.83:0/1649609811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8750147310 con 0x7f8738004810 2024-04-17T10:41:59.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.447+0000 7f8737fff640 1 -- 172.21.15.83:0/1649609811 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f874001e460 con 0x7f8738004810 2024-04-17T10:41:59.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.447+0000 7f8737fff640 1 --2- 172.21.15.83:0/1649609811 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f873c066f60 0x7f873c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:59.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.447+0000 7f8737fff640 1 -- 172.21.15.83:0/1649609811 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8740085720 con 0x7f8738004810 2024-04-17T10:41:59.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.447+0000 7f874effd640 1 --2- 172.21.15.83:0/1649609811 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f873c066f60 0x7f873c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:59.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.447+0000 7f874effd640 1 --2- 172.21.15.83:0/1649609811 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f873c066f60 0x7f873c069420 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f87440b0000 tx=0x7f8744004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:59.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.447+0000 7f8758c2d000 1 Processor -- start 2024-04-17T10:41:59.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.447+0000 7f8758c2d000 1 -- start start 2024-04-17T10:41:59.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.447+0000 7f8758c2d000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ddb91fea0 0x561ddb920280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:59.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.447+0000 7f8758c2d000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ddb9207c0 0x561ddb8d61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:59.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.447+0000 7f8758c2d000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561ddb8d6710 0x561ddb8d6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:59.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.447+0000 7f8758c2d000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ddb724db0 con 0x561ddb91fea0 2024-04-17T10:41:59.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.447+0000 7f8758c2d000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ddb6ad230 con 0x561ddb9207c0 2024-04-17T10:41:59.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.447+0000 7f8758c2d000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x561ddb720e50 con 0x561ddb8d6710 2024-04-17T10:41:59.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.447+0000 7f874f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ddb91fea0 0x561ddb920280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:59.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.447+0000 7f874f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ddb91fea0 0x561ddb920280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41690/0 (socket says 172.21.15.83:41690) 2024-04-17T10:41:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.447+0000 7f874f7fe640 1 -- 172.21.15.83:0/1810962097 learned_addr learned my addr 172.21.15.83:0/1810962097 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.447+0000 7f874f7fe640 1 -- 172.21.15.83:0/1810962097 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561ddb8d6710 msgr2=0x561ddb8d6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.447+0000 7f874f7fe640 1 --2- 172.21.15.83:0/1810962097 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561ddb8d6710 0x561ddb8d6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.447+0000 7f874f7fe640 1 -- 172.21.15.83:0/1810962097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ddb9207c0 msgr2=0x561ddb8d61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.447+0000 7f874f7fe640 1 --2- 172.21.15.83:0/1810962097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ddb9207c0 0x561ddb8d61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.447+0000 7f874f7fe640 1 -- 172.21.15.83:0/1810962097 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ddb6e6750 con 0x561ddb91fea0 2024-04-17T10:41:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f874f7fe640 1 --2- 172.21.15.83:0/1810962097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ddb91fea0 0x561ddb920280 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f874005e1d0 tx=0x7f8740055950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=927302022f47fc57 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f87357fa640 1 -- 172.21.15.83:0/1810962097 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740082070 con 0x561ddb91fea0 2024-04-17T10:41:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f87357fa640 1 -- 172.21.15.83:0/1810962097 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874005a140 con 0x561ddb91fea0 2024-04-17T10:41:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f87357fa640 1 -- 172.21.15.83:0/1810962097 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874006d070 con 0x561ddb91fea0 2024-04-17T10:41:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f8758c2d000 1 -- 172.21.15.83:0/1810962097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ddb91fea0 msgr2=0x561ddb920280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f8758c2d000 1 --2- 172.21.15.83:0/1810962097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ddb91fea0 0x561ddb920280 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f874005e1d0 tx=0x7f8740055950 comp rx=0 tx=0).stop 2024-04-17T10:41:59.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f8758c2d000 1 -- 172.21.15.83:0/1810962097 shutdown_connections 2024-04-17T10:41:59.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f8758c2d000 1 --2- 172.21.15.83:0/1810962097 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561ddb8d6710 0x561ddb8d6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f8758c2d000 1 --2- 172.21.15.83:0/1810962097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ddb9207c0 0x561ddb8d61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f8758c2d000 1 --2- 172.21.15.83:0/1810962097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ddb91fea0 0x561ddb920280 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f8758c2d000 1 -- 172.21.15.83:0/1810962097 >> 172.21.15.83:0/1810962097 conn(0x561ddb8d26c0 msgr2=0x561ddb8d4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:59.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f8758c2d000 1 -- 172.21.15.83:0/1810962097 shutdown_connections 2024-04-17T10:41:59.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f8758c2d000 1 -- 172.21.15.83:0/1810962097 wait complete. 2024-04-17T10:41:59.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f8758c2d000 1 -- start start 2024-04-17T10:41:59.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f8758c2d000 1 Processor -- start 2024-04-17T10:41:59.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f8758c2d000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561ddb8d6710 0x561ddb904ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:59.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f8758c2d000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ddb91fea0 0x561ddb905410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:59.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f8758c2d000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ddb9207c0 0x561ddb905950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:59.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f8758c2d000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ddb6dfbf0 con 0x561ddb91fea0 2024-04-17T10:41:59.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f8758c2d000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ddb723d80 con 0x561ddb9207c0 2024-04-17T10:41:59.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f8758c2d000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x561ddb6a9990 con 0x561ddb8d6710 2024-04-17T10:41:59.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f8758c2d000 10 client.0 _ll_register_callbacks cb 0x561ddb8c1790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:41:59.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f874effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ddb91fea0 0x561ddb905410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:59.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f874effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ddb91fea0 0x561ddb905410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41700/0 (socket says 172.21.15.83:41700) 2024-04-17T10:41:59.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f874effd640 1 -- 172.21.15.83:0/1068245912 learned_addr learned my addr 172.21.15.83:0/1068245912 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:59.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f874f7fe640 1 --2- 172.21.15.83:0/1068245912 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561ddb8d6710 0x561ddb904ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:59.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f874ffff640 1 --2- 172.21.15.83:0/1068245912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ddb9207c0 0x561ddb905950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:59.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f874effd640 1 -- 172.21.15.83:0/1068245912 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561ddb8d6710 msgr2=0x561ddb904ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:59.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f874effd640 1 --2- 172.21.15.83:0/1068245912 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561ddb8d6710 0x561ddb904ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f874effd640 1 -- 172.21.15.83:0/1068245912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ddb9207c0 msgr2=0x561ddb905950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:59.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f874effd640 1 --2- 172.21.15.83:0/1068245912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ddb9207c0 0x561ddb905950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f874effd640 1 -- 172.21.15.83:0/1068245912 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ddb6c7640 con 0x561ddb91fea0 2024-04-17T10:41:59.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f874f7fe640 1 --2- 172.21.15.83:0/1068245912 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561ddb8d6710 0x561ddb904ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:59.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f874ffff640 1 --2- 172.21.15.83:0/1068245912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ddb9207c0 0x561ddb905950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:41:59.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f874effd640 1 --2- 172.21.15.83:0/1068245912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ddb91fea0 0x561ddb905410 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8744000c10 tx=0x7f8738005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58d0e4fe91c7eaa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:59.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f872e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:41:59.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f872e7fc640 1 -- 172.21.15.83:0/1068245912 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87440ae040 con 0x561ddb91fea0 2024-04-17T10:41:59.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f872e7fc640 1 -- 172.21.15.83:0/1068245912 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f87440b3b20 con 0x561ddb91fea0 2024-04-17T10:41:59.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f8758c2d000 1 -- 172.21.15.83:0/1068245912 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f874005a140 con 0x561ddb91fea0 2024-04-17T10:41:59.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f872e7fc640 1 -- 172.21.15.83:0/1068245912 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87440d0050 con 0x561ddb91fea0 2024-04-17T10:41:59.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f8758c2d000 10 client.5886 fetch_fsmap learned FSMap version 21 2024-04-17T10:41:59.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f8758c2d000 1 -- 172.21.15.83:0/1068245912 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x561ddb6ad480 con 0x561ddb91fea0 2024-04-17T10:41:59.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f872e7fc640 1 -- 172.21.15.83:0/1068245912 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874005a140 con 0x561ddb91fea0 2024-04-17T10:41:59.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f872e7fc640 1 -- 172.21.15.83:0/1068245912 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f87440d3020 con 0x561ddb91fea0 2024-04-17T10:41:59.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f8758c2d000 10 client.5886 fetch_fsmap finished waiting for FSMap version 21 2024-04-17T10:41:59.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f8758c2d000 10 client.5886 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:41:59.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f8758c2d000 20 client.5886 populate_metadata read hostname 'smithi083' 2024-04-17T10:41:59.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f8758c2d000 1 --2- 172.21.15.83:0/1068245912 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x561ddb92ff40 0x561ddb932330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:59.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f8758c2d000 4 client.5886 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:41:59.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f8758c2d000 1 -- 172.21.15.83:0/1068245912 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561ddb6c0bb0 con 0x561ddb92ff40 2024-04-17T10:41:59.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f874f7fe640 1 --2- 172.21.15.83:0/1068245912 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x561ddb92ff40 0x561ddb932330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:59.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f874f7fe640 1 --2- 172.21.15.83:0/1068245912 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x561ddb92ff40 0x561ddb932330 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=196f505d504d6f45 server_cookie=367b863f7568080a in_seq=0 out_seq=0 2024-04-17T10:41:59.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.451+0000 7f872e7fc640 10 client.5886 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:41:59.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.455+0000 7f872e7fc640 1 -- 172.21.15.83:0/1068245912 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561ddb6c0bb0 con 0x561ddb92ff40 2024-04-17T10:41:59.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.455+0000 7f872e7fc640 10 client.5886 handle_command_reply: tid=0 2024-04-17T10:41:59.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.455+0000 7f8758c2d000 1 client.5886 shutdown 2024-04-17T10:41:59.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.455+0000 7f8758c2d000 1 -- 172.21.15.83:0/1068245912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ddb91fea0 msgr2=0x561ddb905410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:59.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.455+0000 7f8758c2d000 1 --2- 172.21.15.83:0/1068245912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ddb91fea0 0x561ddb905410 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8744000c10 tx=0x7f8738005840 comp rx=0 tx=0).stop 2024-04-17T10:41:59.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.455+0000 7f8758c2d000 1 -- 172.21.15.83:0/1068245912 shutdown_connections 2024-04-17T10:41:59.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.455+0000 7f8758c2d000 1 --2- 172.21.15.83:0/1068245912 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x561ddb92ff40 0x561ddb932330 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.455+0000 7f8758c2d000 1 --2- 172.21.15.83:0/1068245912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ddb9207c0 0x561ddb905950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.455+0000 7f872e7fc640 0 client.5886 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:41:59.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.455+0000 7f8758c2d000 1 --2- 172.21.15.83:0/1068245912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ddb91fea0 0x561ddb905410 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.455+0000 7f8758c2d000 1 --2- 172.21.15.83:0/1068245912 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561ddb8d6710 0x561ddb904ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.455+0000 7f8758c2d000 1 -- 172.21.15.83:0/1068245912 >> 172.21.15.83:0/1068245912 conn(0x561ddb8d26c0 msgr2=0x561ddb8d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:59.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.455+0000 7f8758c2d000 1 -- 172.21.15.83:0/1068245912 shutdown_connections 2024-04-17T10:41:59.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.455+0000 7f8758c2d000 1 -- 172.21.15.83:0/1068245912 wait complete. 2024-04-17T10:41:59.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.455+0000 7f8758c2d000 20 client.5886 trim_cache size 0 max 16384 2024-04-17T10:41:59.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.467+0000 7f8758c2d000 1 Processor -- start 2024-04-17T10:41:59.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.467+0000 7f8758c2d000 1 -- start start 2024-04-17T10:41:59.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f8758c2d000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ddb8d6710 0x561ddb8d5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:59.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f8758c2d000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ddb91fea0 0x561ddb8d6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:59.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f8758c2d000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561ddb9207c0 0x561ddb930320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:59.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f8758c2d000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ddb8d9160 con 0x561ddb8d6710 2024-04-17T10:41:59.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f8758c2d000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87501fe5d0 con 0x561ddb91fea0 2024-04-17T10:41:59.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f8758c2d000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87501fd810 con 0x561ddb9207c0 2024-04-17T10:41:59.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f874f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ddb8d6710 0x561ddb8d5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:59.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f874f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ddb8d6710 0x561ddb8d5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41704/0 (socket says 172.21.15.83:41704) 2024-04-17T10:41:59.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f874f7fe640 1 -- 172.21.15.83:0/1497693591 learned_addr learned my addr 172.21.15.83:0/1497693591 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:59.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f874f7fe640 1 -- 172.21.15.83:0/1497693591 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561ddb9207c0 msgr2=0x561ddb930320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:59.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f874f7fe640 1 --2- 172.21.15.83:0/1497693591 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561ddb9207c0 0x561ddb930320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f874f7fe640 1 -- 172.21.15.83:0/1497693591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ddb91fea0 msgr2=0x561ddb8d6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:59.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f874effd640 1 --2- 172.21.15.83:0/1497693591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ddb91fea0 0x561ddb8d6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:59.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f874f7fe640 1 --2- 172.21.15.83:0/1497693591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ddb91fea0 0x561ddb8d6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f874f7fe640 1 -- 172.21.15.83:0/1497693591 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ddb7a6ef0 con 0x561ddb8d6710 2024-04-17T10:41:59.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f874f7fe640 1 --2- 172.21.15.83:0/1497693591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ddb8d6710 0x561ddb8d5ac0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f8740055ac0 tx=0x7f874006a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29bab8761c5e6236 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:59.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f8734ff9640 1 -- 172.21.15.83:0/1497693591 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740082040 con 0x561ddb8d6710 2024-04-17T10:41:59.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f8734ff9640 1 -- 172.21.15.83:0/1497693591 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874005a140 con 0x561ddb8d6710 2024-04-17T10:41:59.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f8734ff9640 1 -- 172.21.15.83:0/1497693591 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874006d040 con 0x561ddb8d6710 2024-04-17T10:41:59.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f8758c2d000 1 -- 172.21.15.83:0/1497693591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ddb8d6710 msgr2=0x561ddb8d5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:59.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f8758c2d000 1 --2- 172.21.15.83:0/1497693591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ddb8d6710 0x561ddb8d5ac0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f8740055ac0 tx=0x7f874006a000 comp rx=0 tx=0).stop 2024-04-17T10:41:59.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f8758c2d000 1 -- 172.21.15.83:0/1497693591 shutdown_connections 2024-04-17T10:41:59.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f8758c2d000 1 --2- 172.21.15.83:0/1497693591 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561ddb9207c0 0x561ddb930320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f8758c2d000 1 --2- 172.21.15.83:0/1497693591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ddb91fea0 0x561ddb8d6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f8758c2d000 1 --2- 172.21.15.83:0/1497693591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ddb8d6710 0x561ddb8d5ac0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f8758c2d000 1 -- 172.21.15.83:0/1497693591 >> 172.21.15.83:0/1497693591 conn(0x561ddb8d26c0 msgr2=0x561ddb91e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:59.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f8758c2d000 1 -- 172.21.15.83:0/1497693591 shutdown_connections 2024-04-17T10:41:59.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f8758c2d000 1 -- 172.21.15.83:0/1497693591 wait complete. 2024-04-17T10:41:59.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f8758c2d000 1 -- start start 2024-04-17T10:41:59.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f8758c2d000 1 Processor -- start 2024-04-17T10:41:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f8758c2d000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ddb8d6710 0x561ddb981860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f8758c2d000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561ddb91fea0 0x561ddb981da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f8758c2d000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ddb9207c0 0x561ddb933040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f8758c2d000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875013cda0 con 0x561ddb8d6710 2024-04-17T10:41:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f8758c2d000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87501f60d0 con 0x561ddb9207c0 2024-04-17T10:41:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f8758c2d000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x561ddb6dfbf0 con 0x561ddb91fea0 2024-04-17T10:41:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f8758c2d000 10 client.0 _ll_register_callbacks cb 0x7f874000be70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:41:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f874f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ddb8d6710 0x561ddb981860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f874f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ddb8d6710 0x561ddb981860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41720/0 (socket says 172.21.15.83:41720) 2024-04-17T10:41:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f874f7fe640 1 -- 172.21.15.83:0/1811100499 learned_addr learned my addr 172.21.15.83:0/1811100499 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f874f7fe640 1 -- 172.21.15.83:0/1811100499 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561ddb91fea0 msgr2=0x561ddb981da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f874f7fe640 1 --2- 172.21.15.83:0/1811100499 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561ddb91fea0 0x561ddb981da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f874f7fe640 1 -- 172.21.15.83:0/1811100499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ddb9207c0 msgr2=0x561ddb933040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f874f7fe640 1 --2- 172.21.15.83:0/1811100499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ddb9207c0 0x561ddb933040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f874f7fe640 1 -- 172.21.15.83:0/1811100499 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ddb6e49a0 con 0x561ddb8d6710 2024-04-17T10:41:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f874f7fe640 1 --2- 172.21.15.83:0/1811100499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ddb8d6710 0x561ddb981860 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f874000b4b0 tx=0x7f87400046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d086544b83fa66b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f872e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:41:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f872e7fc640 1 -- 172.21.15.83:0/1811100499 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740082070 con 0x561ddb8d6710 2024-04-17T10:41:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f872e7fc640 1 -- 172.21.15.83:0/1811100499 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f87400542f0 con 0x561ddb8d6710 2024-04-17T10:41:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.471+0000 7f8758c2d000 1 -- 172.21.15.83:0/1811100499 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f874005a140 con 0x561ddb8d6710 2024-04-17T10:41:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f872e7fc640 1 -- 172.21.15.83:0/1811100499 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f874007c050 con 0x561ddb8d6710 2024-04-17T10:41:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f8758c2d000 10 client.5892 fetch_fsmap learned FSMap version 21 2024-04-17T10:41:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f8758c2d000 1 -- 172.21.15.83:0/1811100499 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x561ddb3e15a0 con 0x561ddb8d6710 2024-04-17T10:41:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f872e7fc640 1 -- 172.21.15.83:0/1811100499 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874005a140 con 0x561ddb8d6710 2024-04-17T10:41:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f872e7fc640 1 -- 172.21.15.83:0/1811100499 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8740013070 con 0x561ddb8d6710 2024-04-17T10:41:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f8758c2d000 10 client.5892 fetch_fsmap finished waiting for FSMap version 21 2024-04-17T10:41:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f8758c2d000 10 client.5892 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:41:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f8758c2d000 20 client.5892 populate_metadata read hostname 'smithi083' 2024-04-17T10:41:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f8758c2d000 1 --2- 172.21.15.83:0/1811100499 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x561ddb933580 0x561ddb933990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f8758c2d000 4 client.5892 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}] 2024-04-17T10:41:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f8758c2d000 1 -- 172.21.15.83:0/1811100499 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}) v1 -- 0x561ddb6d4080 con 0x561ddb933580 2024-04-17T10:41:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f874effd640 1 --2- 172.21.15.83:0/1811100499 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x561ddb933580 0x561ddb933990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f872e7fc640 10 client.5892 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:41:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f874effd640 1 --2- 172.21.15.83:0/1811100499 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x561ddb933580 0x561ddb933990 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a517cd3ae70a90f server_cookie=7e2e7585bf935469 in_seq=0 out_seq=0 2024-04-17T10:41:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f872e7fc640 1 -- 172.21.15.83:0/1811100499 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561ddb6d4080 con 0x561ddb933580 2024-04-17T10:41:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f872e7fc640 10 client.5892 handle_command_reply: tid=0 2024-04-17T10:41:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f8758c2d000 1 client.5892 shutdown 2024-04-17T10:41:59.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f8758c2d000 1 -- 172.21.15.83:0/1811100499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ddb8d6710 msgr2=0x561ddb981860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:59.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f8758c2d000 1 --2- 172.21.15.83:0/1811100499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ddb8d6710 0x561ddb981860 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f874000b4b0 tx=0x7f87400046d0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f8758c2d000 1 -- 172.21.15.83:0/1811100499 shutdown_connections 2024-04-17T10:41:59.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f8758c2d000 1 --2- 172.21.15.83:0/1811100499 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x561ddb933580 0x561ddb933990 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f8758c2d000 1 --2- 172.21.15.83:0/1811100499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ddb9207c0 0x561ddb933040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f8758c2d000 1 --2- 172.21.15.83:0/1811100499 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561ddb91fea0 0x561ddb981da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f8758c2d000 1 --2- 172.21.15.83:0/1811100499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ddb8d6710 0x561ddb981860 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f8758c2d000 1 -- 172.21.15.83:0/1811100499 >> 172.21.15.83:0/1811100499 conn(0x561ddb8d26c0 msgr2=0x561ddb91f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:59.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f872e7fc640 0 client.5892 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:41:59.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f8758c2d000 1 -- 172.21.15.83:0/1811100499 shutdown_connections 2024-04-17T10:41:59.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f8758c2d000 1 -- 172.21.15.83:0/1811100499 wait complete. 2024-04-17T10:41:59.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f8758c2d000 20 client.5892 trim_cache size 0 max 16384 2024-04-17T10:41:59.479 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:41:59.479 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-17T10:41:59.479 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "e3a4cbbe-5a2a-4f32-a018-8c94320b148c", 2024-04-17T10:41:59.479 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-17T10:41:59.479 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:41:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f8755ac0640 1 -- 172.21.15.83:0/1649609811 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f873c066f60 msgr2=0x7f873c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f8755ac0640 1 --2- 172.21.15.83:0/1649609811 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f873c066f60 0x7f873c069420 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f87440b0000 tx=0x7f8744004740 comp rx=0 tx=0).stop 2024-04-17T10:41:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f8755ac0640 1 -- 172.21.15.83:0/1649609811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8738004810 msgr2=0x7f875013b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f8755ac0640 1 --2- 172.21.15.83:0/1649609811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8738004810 0x7f875013b510 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f874000d830 tx=0x7f874000dcf0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f8755ac0640 1 -- 172.21.15.83:0/1649609811 shutdown_connections 2024-04-17T10:41:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f8755ac0640 1 --2- 172.21.15.83:0/1649609811 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f873c066f60 0x7f873c069420 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f8755ac0640 1 --2- 172.21.15.83:0/1649609811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87501447a0 0x7f875015b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f8755ac0640 1 --2- 172.21.15.83:0/1649609811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f875013bb40 0x7f8750146dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f8755ac0640 1 --2- 172.21.15.83:0/1649609811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8738004810 0x7f875013b510 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f8755ac0640 1 -- 172.21.15.83:0/1649609811 >> 172.21.15.83:0/1649609811 conn(0x7f87500b23f0 msgr2=0x7f875006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f8755ac0640 1 -- 172.21.15.83:0/1649609811 shutdown_connections 2024-04-17T10:41:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.475+0000 7f8755ac0640 1 -- 172.21.15.83:0/1649609811 wait complete. 2024-04-17T10:41:59.488 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites' returned '{'return_code': 0, 'scrub_tag': 'e3a4cbbe-5a2a-4f32-a018-8c94320b148c', 'mode': 'asynchronous'}' 2024-04-17T10:41:59.488 INFO:tasks.cephfs.test_scrub_checks:First run: flushing /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-17T10:41:59.488 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-17T10:41:59.488 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:41:59.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.595+0000 7f38a9d85640 1 Processor -- start 2024-04-17T10:41:59.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.595+0000 7f38a9d85640 1 -- start start 2024-04-17T10:41:59.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.599+0000 7f38a9d85640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38a415db50 0x7f38a4070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:59.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.599+0000 7f38a9d85640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f38a406dcb0 0x7f38a40b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:59.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.599+0000 7f38a9d85640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a40709b0 con 0x7f38a40aedd0 2024-04-17T10:41:59.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.599+0000 7f38a9d85640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a415e440 con 0x7f38a406dcb0 2024-04-17T10:41:59.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.599+0000 7f38a9d85640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a415e720 con 0x7f38a415db50 2024-04-17T10:41:59.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.599+0000 7f38a3fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f38a406dcb0 0x7f38a40b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:59.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.599+0000 7f38a3fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f38a406dcb0 0x7f38a40b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41728/0 (socket says 172.21.15.83:41728) 2024-04-17T10:41:59.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.599+0000 7f38a3fff640 1 -- 172.21.15.83:0/4283616568 learned_addr learned my addr 172.21.15.83:0/4283616568 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:59.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.599+0000 7f38a3fff640 1 -- 172.21.15.83:0/4283616568 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38a415db50 msgr2=0x7f38a4070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:59.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.599+0000 7f38a3fff640 1 --2- 172.21.15.83:0/4283616568 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38a415db50 0x7f38a4070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.599+0000 7f38a3fff640 1 -- 172.21.15.83:0/4283616568 >> v1:172.21.15.167:6789/0 conn(0x7f38a40aedd0 legacy=0x7f38a406d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:41:59.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.599+0000 7f38a3fff640 1 -- 172.21.15.83:0/4283616568 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a415ea00 con 0x7f38a406dcb0 2024-04-17T10:41:59.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.599+0000 7f38a3fff640 1 --2- 172.21.15.83:0/4283616568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38a406dcb0 0x7f38a40b2e10 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f389c009e50 tx=0x7f389c01c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c085328d4f6219b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:59.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.599+0000 7f38a2ffd640 1 -- 172.21.15.83:0/4283616568 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c0a5660 con 0x7f38a406dcb0 2024-04-17T10:41:59.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.599+0000 7f38a2ffd640 1 -- 172.21.15.83:0/4283616568 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389c0a5800 con 0x7f38a406dcb0 2024-04-17T10:41:59.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.599+0000 7f38a2ffd640 1 -- 172.21.15.83:0/4283616568 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c0a5bd0 con 0x7f38a406dcb0 2024-04-17T10:41:59.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f38a9d85640 1 -- 172.21.15.83:0/4283616568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38a406dcb0 msgr2=0x7f38a40b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:59.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f38a9d85640 1 --2- 172.21.15.83:0/4283616568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38a406dcb0 0x7f38a40b2e10 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f389c009e50 tx=0x7f389c01c7e0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f38a9d85640 1 -- 172.21.15.83:0/4283616568 shutdown_connections 2024-04-17T10:41:59.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f38a9d85640 1 --2- 172.21.15.83:0/4283616568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38a406dcb0 0x7f38a40b2e10 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f38a9d85640 1 --2- 172.21.15.83:0/4283616568 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38a415db50 0x7f38a4070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f38a9d85640 1 -- 172.21.15.83:0/4283616568 >> 172.21.15.83:0/4283616568 conn(0x7f38a406abe0 msgr2=0x7f38a406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:59.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f38a9d85640 1 -- 172.21.15.83:0/4283616568 shutdown_connections 2024-04-17T10:41:59.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f38a9d85640 1 -- 172.21.15.83:0/4283616568 wait complete. 2024-04-17T10:41:59.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f38a9d85640 1 Processor -- start 2024-04-17T10:41:59.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f38a9d85640 1 -- start start 2024-04-17T10:41:59.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f38a9d85640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38a406dcb0 0x7f38a415a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:59.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f38a9d85640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38a40aedd0 0x7f38a413e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:59.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f38a9d85640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38a415db50 0x7f38a40ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:59.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f38a9d85640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a40b3ce0 con 0x7f38a415db50 2024-04-17T10:41:59.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f38a9d85640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a40b3b60 con 0x7f38a40aedd0 2024-04-17T10:41:59.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f38a9d85640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a40b3e60 con 0x7f38a406dcb0 2024-04-17T10:41:59.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f38a3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38a415db50 0x7f38a40ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:59.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f38a3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38a415db50 0x7f38a40ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41732/0 (socket says 172.21.15.83:41732) 2024-04-17T10:41:59.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f38a3fff640 1 -- 172.21.15.83:0/2589688994 learned_addr learned my addr 172.21.15.83:0/2589688994 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:41:59.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f38a37fe640 1 --2- 172.21.15.83:0/2589688994 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38a406dcb0 0x7f38a415a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:59.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f389bfff640 1 --2- 172.21.15.83:0/2589688994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38a40aedd0 0x7f38a413e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:59.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f38a3fff640 1 -- 172.21.15.83:0/2589688994 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38a406dcb0 msgr2=0x7f38a415a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:59.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f38a3fff640 1 --2- 172.21.15.83:0/2589688994 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38a406dcb0 0x7f38a415a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f38a3fff640 1 -- 172.21.15.83:0/2589688994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38a40aedd0 msgr2=0x7f38a413e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:59.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f38a3fff640 1 --2- 172.21.15.83:0/2589688994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38a40aedd0 0x7f38a413e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f38a3fff640 1 -- 172.21.15.83:0/2589688994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a4142250 con 0x7f38a415db50 2024-04-17T10:41:59.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f38a3fff640 1 --2- 172.21.15.83:0/2589688994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38a415db50 0x7f38a40ad840 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f389c0af040 tx=0x7f389c019580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:59.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f38a0ff9640 1 -- 172.21.15.83:0/2589688994 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c0bc020 con 0x7f38a415db50 2024-04-17T10:41:59.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f38a0ff9640 1 -- 172.21.15.83:0/2589688994 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389c0b8670 con 0x7f38a415db50 2024-04-17T10:41:59.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f38a9d85640 1 -- 172.21.15.83:0/2589688994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a4159ec0 con 0x7f38a415db50 2024-04-17T10:41:59.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f38a0ff9640 1 -- 172.21.15.83:0/2589688994 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c0b8a60 con 0x7f38a415db50 2024-04-17T10:41:59.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f38a9d85640 1 -- 172.21.15.83:0/2589688994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a415a480 con 0x7f38a415db50 2024-04-17T10:41:59.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.603+0000 7f38a9d85640 1 -- 172.21.15.83:0/2589688994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a4074f90 con 0x7f38a415db50 2024-04-17T10:41:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.607+0000 7f38a0ff9640 1 -- 172.21.15.83:0/2589688994 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f389c0bf020 con 0x7f38a415db50 2024-04-17T10:41:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.607+0000 7f38a0ff9640 1 --2- 172.21.15.83:0/2589688994 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3888066f90 0x7f3888069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:41:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.607+0000 7f38a0ff9640 1 -- 172.21.15.83:0/2589688994 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f389c11fa10 con 0x7f38a415db50 2024-04-17T10:41:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.607+0000 7f38a37fe640 1 --2- 172.21.15.83:0/2589688994 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3888066f90 0x7f3888069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:41:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.607+0000 7f38a37fe640 1 --2- 172.21.15.83:0/2589688994 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3888066f90 0x7f3888069450 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f38a41528d0 tx=0x7f3894008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:41:59.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.611+0000 7f38a0ff9640 1 -- 172.21.15.83:0/2589688994 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f389c0c7020 con 0x7f38a415db50 2024-04-17T10:41:59.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.803+0000 7f38a9d85640 1 -- 172.21.15.83:0/2589688994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38a406cee0 con 0x7f38a415db50 2024-04-17T10:41:59.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.803+0000 7f38a0ff9640 1 -- 172.21.15.83:0/2589688994 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f38a40709b0 con 0x7f38a415db50 2024-04-17T10:41:59.800 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:41:59.800 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/161398126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":161398126},{"type":"v1","addr":"172.21.15.167:6833","nonce":161398126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1085380618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1085380618},{"type":"v1","addr":"172.21.15.167:6835","nonce":1085380618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/836442291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":836442291},{"type":"v1","addr":"172.21.15.83:6835","nonce":836442291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:41:37.773342+0000","modified":"2024-04-17T10:41:39.788394+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5682":{"gid":5682,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1894547386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1894547386},{"type":"v1","addr":"172.21.15.83:6837","nonce":1894547386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-17T10:41:59.801 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 21 2024-04-17T10:41:59.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.803+0000 7f38a9d85640 1 -- 172.21.15.83:0/2589688994 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3888066f90 msgr2=0x7f3888069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:59.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.803+0000 7f38a9d85640 1 --2- 172.21.15.83:0/2589688994 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3888066f90 0x7f3888069450 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f38a41528d0 tx=0x7f3894008040 comp rx=0 tx=0).stop 2024-04-17T10:41:59.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.803+0000 7f38a9d85640 1 -- 172.21.15.83:0/2589688994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38a415db50 msgr2=0x7f38a40ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:41:59.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.803+0000 7f38a9d85640 1 --2- 172.21.15.83:0/2589688994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38a415db50 0x7f38a40ad840 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f389c0af040 tx=0x7f389c019580 comp rx=0 tx=0).stop 2024-04-17T10:41:59.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.803+0000 7f38a9d85640 1 -- 172.21.15.83:0/2589688994 shutdown_connections 2024-04-17T10:41:59.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.803+0000 7f38a9d85640 1 --2- 172.21.15.83:0/2589688994 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3888066f90 0x7f3888069450 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.803+0000 7f38a9d85640 1 --2- 172.21.15.83:0/2589688994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38a415db50 0x7f38a40ad840 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.803+0000 7f38a9d85640 1 --2- 172.21.15.83:0/2589688994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38a40aedd0 0x7f38a413e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.803+0000 7f38a9d85640 1 --2- 172.21.15.83:0/2589688994 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38a406dcb0 0x7f38a415a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:41:59.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.803+0000 7f38a9d85640 1 -- 172.21.15.83:0/2589688994 >> 172.21.15.83:0/2589688994 conn(0x7f38a406abe0 msgr2=0x7f38a406d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:41:59.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.803+0000 7f38a9d85640 1 -- 172.21.15.83:0/2589688994 shutdown_connections 2024-04-17T10:41:59.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:41:59.803+0000 7f38a9d85640 1 -- 172.21.15.83:0/2589688994 wait complete. 2024-04-17T10:41:59.819 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-17T10:41:59.940 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:41:59.941 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0 2024-04-17T10:41:59.941 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:41:59.954 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-17T10:41:59.954 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-17T10:41:59.955 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-17T10:41:59.955 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-17T10:42:00.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.086+0000 7fcabd38b640 1 Processor -- start 2024-04-17T10:42:00.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.086+0000 7fcabd38b640 1 -- start start 2024-04-17T10:42:00.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.086+0000 7fcabd38b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcab80cf320 0x7fcab8066ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.086+0000 7fcabd38b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcab80ca560 0x7fcab80ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.086+0000 7fcabd38b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcab8072350 con 0x7fcab806bbf0 2024-04-17T10:42:00.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.086+0000 7fcabd38b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcab8067000 con 0x7fcab80cf320 2024-04-17T10:42:00.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.086+0000 7fcabd38b640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcab8072630 con 0x7fcab80ca560 2024-04-17T10:42:00.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.086+0000 7fcaaffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcab80cf320 0x7fcab8066ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.086+0000 7fcaaffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcab80cf320 0x7fcab8066ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41744/0 (socket says 172.21.15.83:41744) 2024-04-17T10:42:00.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.086+0000 7fcab6ffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcab806bbf0 0x7fcab806bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60714/0 (socket says 172.21.15.83:60714) 2024-04-17T10:42:00.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.086+0000 7fcaaffff640 1 -- 172.21.15.83:0/1555719386 learned_addr learned my addr 172.21.15.83:0/1555719386 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:00.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.086+0000 7fcab77fe640 1 --2- 172.21.15.83:0/1555719386 >> v2:172.21.15.167:3300/0 conn(0x7fcab80ca560 0x7fcab80ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.086+0000 7fcaaffff640 1 -- 172.21.15.83:0/1555719386 >> v2:172.21.15.167:3300/0 conn(0x7fcab80ca560 msgr2=0x7fcab80ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:00.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.086+0000 7fcaaffff640 1 --2- 172.21.15.83:0/1555719386 >> v2:172.21.15.167:3300/0 conn(0x7fcab80ca560 0x7fcab80ca9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.086+0000 7fcaaffff640 1 -- 172.21.15.83:0/1555719386 >> v1:172.21.15.83:6789/0 conn(0x7fcab806bbf0 legacy=0x7fcab806bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:00.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.086+0000 7fcaaffff640 1 -- 172.21.15.83:0/1555719386 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab8072910 con 0x7fcab80cf320 2024-04-17T10:42:00.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.086+0000 7fcaaffff640 1 --2- 172.21.15.83:0/1555719386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcab80cf320 0x7fcab8066ac0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fcab80ca4e0 tx=0x7fcaa001c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38375fa4962614d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:00.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcab67fc640 1 -- 172.21.15.83:0/1555719386 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa00b0070 con 0x7fcab80cf320 2024-04-17T10:42:00.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcab67fc640 1 -- 172.21.15.83:0/1555719386 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa00ab600 con 0x7fcab80cf320 2024-04-17T10:42:00.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcab67fc640 1 -- 172.21.15.83:0/1555719386 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa00ab9d0 con 0x7fcab80cf320 2024-04-17T10:42:00.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcabd38b640 1 -- 172.21.15.83:0/1555719386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcab80cf320 msgr2=0x7fcab8066ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:00.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcabd38b640 1 --2- 172.21.15.83:0/1555719386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcab80cf320 0x7fcab8066ac0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fcab80ca4e0 tx=0x7fcaa001c320 comp rx=0 tx=0).stop 2024-04-17T10:42:00.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcabd38b640 1 -- 172.21.15.83:0/1555719386 shutdown_connections 2024-04-17T10:42:00.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcabd38b640 1 --2- 172.21.15.83:0/1555719386 >> v2:172.21.15.167:3300/0 conn(0x7fcab80ca560 0x7fcab80ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcabd38b640 1 --2- 172.21.15.83:0/1555719386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcab80cf320 0x7fcab8066ac0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcabd38b640 1 -- 172.21.15.83:0/1555719386 >> 172.21.15.83:0/1555719386 conn(0x7fcab80cc990 msgr2=0x7fcab80ccda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:00.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcabd38b640 1 -- 172.21.15.83:0/1555719386 shutdown_connections 2024-04-17T10:42:00.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcabd38b640 1 -- 172.21.15.83:0/1555719386 wait complete. 2024-04-17T10:42:00.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcabd38b640 1 Processor -- start 2024-04-17T10:42:00.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcabd38b640 1 -- start start 2024-04-17T10:42:00.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcabd38b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcab806bbf0 0x7fcab80c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcabd38b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcab80ca560 0x7fcab8157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcabd38b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcab80cf320 0x7fcab809a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcabd38b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab816b150 con 0x7fcab80ca560 2024-04-17T10:42:00.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcabd38b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab816afd0 con 0x7fcab806bbf0 2024-04-17T10:42:00.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcabd38b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab816b2d0 con 0x7fcab80cf320 2024-04-17T10:42:00.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcaaffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcab80ca560 0x7fcab8157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcaaffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcab80ca560 0x7fcab8157950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41752/0 (socket says 172.21.15.83:41752) 2024-04-17T10:42:00.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcaaffff640 1 -- 172.21.15.83:0/1319310903 learned_addr learned my addr 172.21.15.83:0/1319310903 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:00.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcab6ffd640 1 --2- 172.21.15.83:0/1319310903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcab806bbf0 0x7fcab80c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcaaffff640 1 -- 172.21.15.83:0/1319310903 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcab80cf320 msgr2=0x7fcab809a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:00.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcaaffff640 1 --2- 172.21.15.83:0/1319310903 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcab80cf320 0x7fcab809a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcaaffff640 1 -- 172.21.15.83:0/1319310903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcab806bbf0 msgr2=0x7fcab80c98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:00.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcaaffff640 1 --2- 172.21.15.83:0/1319310903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcab806bbf0 0x7fcab80c98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcaaffff640 1 -- 172.21.15.83:0/1319310903 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab80c9e00 con 0x7fcab80ca560 2024-04-17T10:42:00.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcaaffff640 1 --2- 172.21.15.83:0/1319310903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcab80ca560 0x7fcab8157950 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fcaa00ae040 tx=0x7fcaa001ab60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:00.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcaaf7fe640 1 -- 172.21.15.83:0/1319310903 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa00ba020 con 0x7fcab80ca560 2024-04-17T10:42:00.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcabd38b640 1 -- 172.21.15.83:0/1319310903 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab80a0560 con 0x7fcab80ca560 2024-04-17T10:42:00.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcabd38b640 1 -- 172.21.15.83:0/1319310903 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab80c6fe0 con 0x7fcab80ca560 2024-04-17T10:42:00.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcaaf7fe640 1 -- 172.21.15.83:0/1319310903 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa00b6620 con 0x7fcab80ca560 2024-04-17T10:42:00.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.090+0000 7fcaaf7fe640 1 -- 172.21.15.83:0/1319310903 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa00b0040 con 0x7fcab80ca560 2024-04-17T10:42:00.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.094+0000 7fcaaf7fe640 1 -- 172.21.15.83:0/1319310903 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcaa00bd020 con 0x7fcab80ca560 2024-04-17T10:42:00.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.094+0000 7fcaaf7fe640 1 --2- 172.21.15.83:0/1319310903 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fca98066f60 0x7fca98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.094+0000 7fcaaf7fe640 1 -- 172.21.15.83:0/1319310903 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fcaa011f1e0 con 0x7fcab80ca560 2024-04-17T10:42:00.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.094+0000 7fcab6ffd640 1 --2- 172.21.15.83:0/1319310903 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fca98066f60 0x7fca98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.094+0000 7fcab6ffd640 1 --2- 172.21.15.83:0/1319310903 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fca98066f60 0x7fca98069420 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8000f70 tx=0x7fcaa8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:00.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.094+0000 7fcac04f8000 1 Processor -- start 2024-04-17T10:42:00.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.094+0000 7fcac04f8000 1 -- start start 2024-04-17T10:42:00.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.094+0000 7fcac04f8000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559467d37200 0x559467d375e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.094+0000 7fcac04f8000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559467d37b20 0x559467d1ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.094+0000 7fcac04f8000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559467d1bf60 0x559467d1c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.094+0000 7fcac04f8000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559467bd44a0 con 0x559467d37b20 2024-04-17T10:42:00.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.094+0000 7fcac04f8000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559467b3c210 con 0x559467d37200 2024-04-17T10:42:00.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.094+0000 7fcac04f8000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x559467ac4980 con 0x559467d1bf60 2024-04-17T10:42:00.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.094+0000 7fcaaffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559467d37b20 0x559467d1ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.094+0000 7fcaaffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559467d37b20 0x559467d1ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41762/0 (socket says 172.21.15.83:41762) 2024-04-17T10:42:00.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.094+0000 7fcaaffff640 1 -- 172.21.15.83:0/1879062453 learned_addr learned my addr 172.21.15.83:0/1879062453 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:00.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.094+0000 7fcab6ffd640 1 --2- 172.21.15.83:0/1879062453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559467d37200 0x559467d375e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.094+0000 7fcaaffff640 1 -- 172.21.15.83:0/1879062453 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559467d1bf60 msgr2=0x559467d1c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:00.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.094+0000 7fcaaffff640 1 --2- 172.21.15.83:0/1879062453 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559467d1bf60 0x559467d1c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.094+0000 7fcaaffff640 1 -- 172.21.15.83:0/1879062453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559467d37200 msgr2=0x559467d375e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:00.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.094+0000 7fcaaffff640 1 --2- 172.21.15.83:0/1879062453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559467d37200 0x559467d375e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.094+0000 7fcaaffff640 1 -- 172.21.15.83:0/1879062453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559467b37200 con 0x559467d37b20 2024-04-17T10:42:00.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.094+0000 7fcaaffff640 1 --2- 172.21.15.83:0/1879062453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559467d37b20 0x559467d1ba20 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fcaa00ebef0 tx=0x7fcaa00190c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03f8282f22784d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:00.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.094+0000 7fcaacff9640 1 -- 172.21.15.83:0/1879062453 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa00f3be0 con 0x559467d37b20 2024-04-17T10:42:00.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.094+0000 7fcaacff9640 1 -- 172.21.15.83:0/1879062453 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa00f81a0 con 0x559467d37b20 2024-04-17T10:42:00.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.094+0000 7fcaacff9640 1 -- 172.21.15.83:0/1879062453 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa00edd90 con 0x559467d37b20 2024-04-17T10:42:00.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcac04f8000 1 -- 172.21.15.83:0/1879062453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559467d37b20 msgr2=0x559467d1ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:00.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcac04f8000 1 --2- 172.21.15.83:0/1879062453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559467d37b20 0x559467d1ba20 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fcaa00ebef0 tx=0x7fcaa00190c0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcac04f8000 1 -- 172.21.15.83:0/1879062453 shutdown_connections 2024-04-17T10:42:00.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcac04f8000 1 --2- 172.21.15.83:0/1879062453 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559467d1bf60 0x559467d1c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcac04f8000 1 --2- 172.21.15.83:0/1879062453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559467d37b20 0x559467d1ba20 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcac04f8000 1 --2- 172.21.15.83:0/1879062453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559467d37200 0x559467d375e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcac04f8000 1 -- 172.21.15.83:0/1879062453 >> 172.21.15.83:0/1879062453 conn(0x559467ce44f0 msgr2=0x559467cf3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:00.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcac04f8000 1 -- 172.21.15.83:0/1879062453 shutdown_connections 2024-04-17T10:42:00.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcac04f8000 1 -- 172.21.15.83:0/1879062453 wait complete. 2024-04-17T10:42:00.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcac04f8000 1 -- start start 2024-04-17T10:42:00.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcac04f8000 1 Processor -- start 2024-04-17T10:42:00.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcac04f8000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559467d1bf60 0x559467d3e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcac04f8000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559467d37200 0x559467d3eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcac04f8000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559467d37b20 0x559467d1c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcac04f8000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559467af6d80 con 0x559467d1bf60 2024-04-17T10:42:00.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcac04f8000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559467b382b0 con 0x559467d37200 2024-04-17T10:42:00.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcac04f8000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x559467afd830 con 0x559467d37b20 2024-04-17T10:42:00.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcac04f8000 10 client.0 _ll_register_callbacks cb 0x559467bef070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:00.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcab6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559467d1bf60 0x559467d3e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcab6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559467d1bf60 0x559467d3e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41766/0 (socket says 172.21.15.83:41766) 2024-04-17T10:42:00.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcab6ffd640 1 -- 172.21.15.83:0/3629295786 learned_addr learned my addr 172.21.15.83:0/3629295786 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:00.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcaaffff640 1 --2- 172.21.15.83:0/3629295786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559467d37200 0x559467d3eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcab6ffd640 1 -- 172.21.15.83:0/3629295786 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559467d37b20 msgr2=0x559467d1c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:00.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcab6ffd640 1 --2- 172.21.15.83:0/3629295786 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559467d37b20 0x559467d1c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcab6ffd640 1 -- 172.21.15.83:0/3629295786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559467d37200 msgr2=0x559467d3eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:00.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcab6ffd640 1 --2- 172.21.15.83:0/3629295786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559467d37200 0x559467d3eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcab6ffd640 1 -- 172.21.15.83:0/3629295786 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559467b08130 con 0x559467d1bf60 2024-04-17T10:42:00.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcaaffff640 1 --2- 172.21.15.83:0/3629295786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559467d37200 0x559467d3eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:00.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcab6ffd640 1 --2- 172.21.15.83:0/3629295786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559467d1bf60 0x559467d3e980 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8001570 tx=0x7fcaa8000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a75a5d099dda746 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:00.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fca9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:00.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fca9dffb640 1 -- 172.21.15.83:0/3629295786 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8021630 con 0x559467d1bf60 2024-04-17T10:42:00.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fca9dffb640 1 -- 172.21.15.83:0/3629295786 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fcaa8020ac0 con 0x559467d1bf60 2024-04-17T10:42:00.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcac04f8000 1 -- 172.21.15.83:0/3629295786 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcaa00f81a0 con 0x559467d1bf60 2024-04-17T10:42:00.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fca9dffb640 1 -- 172.21.15.83:0/3629295786 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcaa8020cc0 con 0x559467d1bf60 2024-04-17T10:42:00.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcac04f8000 10 client.5913 fetch_fsmap learned FSMap version 21 2024-04-17T10:42:00.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcac04f8000 1 -- 172.21.15.83:0/3629295786 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x559467bbd490 con 0x559467d1bf60 2024-04-17T10:42:00.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fca9dffb640 1 -- 172.21.15.83:0/3629295786 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa00f81a0 con 0x559467d1bf60 2024-04-17T10:42:00.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fca9dffb640 1 -- 172.21.15.83:0/3629295786 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcaa8020420 con 0x559467d1bf60 2024-04-17T10:42:00.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcac04f8000 10 client.5913 fetch_fsmap finished waiting for FSMap version 21 2024-04-17T10:42:00.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcac04f8000 10 client.5913 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:42:00.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcac04f8000 20 client.5913 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:00.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcac04f8000 1 --2- 172.21.15.83:0/3629295786 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x559467d1d0b0 0x559467d1d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcac04f8000 4 client.5913 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:42:00.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcac04f8000 1 -- 172.21.15.83:0/3629295786 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559467ad8020 con 0x559467d1d0b0 2024-04-17T10:42:00.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcaaffff640 1 --2- 172.21.15.83:0/3629295786 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x559467d1d0b0 0x559467d1d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fcaaffff640 1 --2- 172.21.15.83:0/3629295786 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x559467d1d0b0 0x559467d1d4c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c4bfbe10fdc98f5 server_cookie=c044bfa5c1f46c0c in_seq=0 out_seq=0 2024-04-17T10:42:00.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.098+0000 7fca9dffb640 10 client.5913 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:00.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.102+0000 7fca9dffb640 1 -- 172.21.15.83:0/3629295786 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559467ad8020 con 0x559467d1d0b0 2024-04-17T10:42:00.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.102+0000 7fca9dffb640 10 client.5913 handle_command_reply: tid=0 2024-04-17T10:42:00.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.102+0000 7fcac04f8000 1 client.5913 shutdown 2024-04-17T10:42:00.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.102+0000 7fcac04f8000 1 -- 172.21.15.83:0/3629295786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559467d1bf60 msgr2=0x559467d3e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:00.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.102+0000 7fcac04f8000 1 --2- 172.21.15.83:0/3629295786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559467d1bf60 0x559467d3e980 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8001570 tx=0x7fcaa8000bd0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.102+0000 7fcac04f8000 1 -- 172.21.15.83:0/3629295786 shutdown_connections 2024-04-17T10:42:00.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.102+0000 7fcac04f8000 1 --2- 172.21.15.83:0/3629295786 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x559467d1d0b0 0x559467d1d4c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.102+0000 7fcac04f8000 1 --2- 172.21.15.83:0/3629295786 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559467d37b20 0x559467d1c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.102+0000 7fcac04f8000 1 --2- 172.21.15.83:0/3629295786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559467d37200 0x559467d3eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.102+0000 7fcac04f8000 1 --2- 172.21.15.83:0/3629295786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559467d1bf60 0x559467d3e980 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.102+0000 7fcac04f8000 1 -- 172.21.15.83:0/3629295786 >> 172.21.15.83:0/3629295786 conn(0x559467ce44f0 msgr2=0x559467cebaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:00.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.102+0000 7fca9dffb640 0 client.5913 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:00.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.102+0000 7fcac04f8000 1 -- 172.21.15.83:0/3629295786 shutdown_connections 2024-04-17T10:42:00.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.102+0000 7fcac04f8000 1 -- 172.21.15.83:0/3629295786 wait complete. 2024-04-17T10:42:00.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.102+0000 7fcac04f8000 20 client.5913 trim_cache size 0 max 16384 2024-04-17T10:42:00.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.114+0000 7fcac04f8000 1 Processor -- start 2024-04-17T10:42:00.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.114+0000 7fcac04f8000 1 -- start start 2024-04-17T10:42:00.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.114+0000 7fcac04f8000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559467d1bf60 0x559467ceb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.114+0000 7fcac04f8000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559467d37200 0x559467ceb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.114+0000 7fcac04f8000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559467d37b20 0x559467cebc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.114+0000 7fcac04f8000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9400c610 con 0x559467d1bf60 2024-04-17T10:42:00.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.114+0000 7fcac04f8000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9400b730 con 0x559467d37b20 2024-04-17T10:42:00.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.114+0000 7fcac04f8000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9400aaf0 con 0x559467d37200 2024-04-17T10:42:00.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.114+0000 7fcab6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559467d1bf60 0x559467ceb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.114+0000 7fcab6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559467d1bf60 0x559467ceb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41780/0 (socket says 172.21.15.83:41780) 2024-04-17T10:42:00.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.114+0000 7fcab6ffd640 1 -- 172.21.15.83:0/976654950 learned_addr learned my addr 172.21.15.83:0/976654950 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:00.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.114+0000 7fcab6ffd640 1 -- 172.21.15.83:0/976654950 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559467d37200 msgr2=0x559467ceb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:00.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.114+0000 7fcab6ffd640 1 --2- 172.21.15.83:0/976654950 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559467d37200 0x559467ceb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.114+0000 7fcab6ffd640 1 -- 172.21.15.83:0/976654950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559467d37b20 msgr2=0x559467cebc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:00.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.114+0000 7fcab77fe640 1 --2- 172.21.15.83:0/976654950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559467d37b20 0x559467cebc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.114+0000 7fcab6ffd640 1 --2- 172.21.15.83:0/976654950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559467d37b20 0x559467cebc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.114+0000 7fcab6ffd640 1 -- 172.21.15.83:0/976654950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559467afba30 con 0x559467d1bf60 2024-04-17T10:42:00.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcab6ffd640 1 --2- 172.21.15.83:0/976654950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559467d1bf60 0x559467ceb210 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fcaa800dc70 tx=0x7fcaa801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e2a2caf4d2527a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:00.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fca9ffff640 1 -- 172.21.15.83:0/976654950 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8038070 con 0x559467d1bf60 2024-04-17T10:42:00.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fca9ffff640 1 -- 172.21.15.83:0/976654950 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa80013a0 con 0x559467d1bf60 2024-04-17T10:42:00.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fca9ffff640 1 -- 172.21.15.83:0/976654950 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa802f040 con 0x559467d1bf60 2024-04-17T10:42:00.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcac04f8000 1 -- 172.21.15.83:0/976654950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559467d1bf60 msgr2=0x559467ceb210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:00.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcac04f8000 1 --2- 172.21.15.83:0/976654950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559467d1bf60 0x559467ceb210 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fcaa800dc70 tx=0x7fcaa801d000 comp rx=0 tx=0).stop 2024-04-17T10:42:00.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcac04f8000 1 -- 172.21.15.83:0/976654950 shutdown_connections 2024-04-17T10:42:00.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcac04f8000 1 --2- 172.21.15.83:0/976654950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559467d37b20 0x559467cebc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcac04f8000 1 --2- 172.21.15.83:0/976654950 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559467d37200 0x559467ceb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcac04f8000 1 --2- 172.21.15.83:0/976654950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559467d1bf60 0x559467ceb210 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcac04f8000 1 -- 172.21.15.83:0/976654950 >> 172.21.15.83:0/976654950 conn(0x559467ce44f0 msgr2=0x559467d1a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:00.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcac04f8000 1 -- 172.21.15.83:0/976654950 shutdown_connections 2024-04-17T10:42:00.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcac04f8000 1 -- 172.21.15.83:0/976654950 wait complete. 2024-04-17T10:42:00.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcac04f8000 1 -- start start 2024-04-17T10:42:00.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcac04f8000 1 Processor -- start 2024-04-17T10:42:00.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcac04f8000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559467d1bf60 0x559467d97380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcac04f8000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559467d37200 0x559467d978c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcac04f8000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559467d37b20 0x559467d53030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcac04f8000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559467ac8190 con 0x559467d37b20 2024-04-17T10:42:00.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcac04f8000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9400b0f0 con 0x559467d37200 2024-04-17T10:42:00.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcac04f8000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x559467b3b520 con 0x559467d1bf60 2024-04-17T10:42:00.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcac04f8000 10 client.0 _ll_register_callbacks cb 0x7fcaa00ebd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:00.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcab77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559467d37b20 0x559467d53030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcab77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559467d37b20 0x559467d53030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41794/0 (socket says 172.21.15.83:41794) 2024-04-17T10:42:00.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcab77fe640 1 -- 172.21.15.83:0/338612950 learned_addr learned my addr 172.21.15.83:0/338612950 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:00.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcaaffff640 1 --2- 172.21.15.83:0/338612950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559467d37200 0x559467d978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcab6ffd640 1 --2- 172.21.15.83:0/338612950 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559467d1bf60 0x559467d97380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcab77fe640 1 -- 172.21.15.83:0/338612950 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559467d1bf60 msgr2=0x559467d97380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:00.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcab77fe640 1 --2- 172.21.15.83:0/338612950 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559467d1bf60 0x559467d97380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcab77fe640 1 -- 172.21.15.83:0/338612950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559467d37200 msgr2=0x559467d978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:00.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcab77fe640 1 --2- 172.21.15.83:0/338612950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559467d37200 0x559467d978c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcab77fe640 1 -- 172.21.15.83:0/338612950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559467b04040 con 0x559467d37b20 2024-04-17T10:42:00.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcab77fe640 1 --2- 172.21.15.83:0/338612950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559467d37b20 0x559467d53030 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fcab00098c0 tx=0x7fcab0009d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e0a2c79a3be0c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:00.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fca9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:00.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fca9dffb640 1 -- 172.21.15.83:0/338612950 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab0019060 con 0x559467d37b20 2024-04-17T10:42:00.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fca9dffb640 1 -- 172.21.15.83:0/338612950 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fcab0007c40 con 0x559467d37b20 2024-04-17T10:42:00.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcac04f8000 1 -- 172.21.15.83:0/338612950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcaa80013a0 con 0x559467d37b20 2024-04-17T10:42:00.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fca9dffb640 1 -- 172.21.15.83:0/338612950 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcab001f050 con 0x559467d37b20 2024-04-17T10:42:00.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcac04f8000 10 client.5919 fetch_fsmap learned FSMap version 21 2024-04-17T10:42:00.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcac04f8000 1 -- 172.21.15.83:0/338612950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x5594677f8900 con 0x559467d37b20 2024-04-17T10:42:00.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fca9dffb640 1 -- 172.21.15.83:0/338612950 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa80013a0 con 0x559467d37b20 2024-04-17T10:42:00.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fca9dffb640 1 -- 172.21.15.83:0/338612950 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcab0022020 con 0x559467d37b20 2024-04-17T10:42:00.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcac04f8000 10 client.5919 fetch_fsmap finished waiting for FSMap version 21 2024-04-17T10:42:00.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcac04f8000 10 client.5919 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:42:00.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcac04f8000 20 client.5919 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:00.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcac04f8000 1 --2- 172.21.15.83:0/338612950 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x559467d53f90 0x559467d543a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcac04f8000 4 client.5919 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}] 2024-04-17T10:42:00.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcac04f8000 1 -- 172.21.15.83:0/338612950 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}) v1 -- 0x559467b49750 con 0x559467d53f90 2024-04-17T10:42:00.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.118+0000 7fcab6ffd640 1 --2- 172.21.15.83:0/338612950 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x559467d53f90 0x559467d543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.122+0000 7fca9dffb640 10 client.5919 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:00.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.122+0000 7fcab6ffd640 1 --2- 172.21.15.83:0/338612950 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x559467d53f90 0x559467d543a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faf3b13d3e2787d4 server_cookie=6f35fb30452627d7 in_seq=0 out_seq=0 2024-04-17T10:42:00.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.122+0000 7fca9dffb640 1 -- 172.21.15.83:0/338612950 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559467b49750 con 0x559467d53f90 2024-04-17T10:42:00.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.122+0000 7fca9dffb640 10 client.5919 handle_command_reply: tid=0 2024-04-17T10:42:00.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.122+0000 7fcac04f8000 1 client.5919 shutdown 2024-04-17T10:42:00.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.122+0000 7fcac04f8000 1 -- 172.21.15.83:0/338612950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559467d37b20 msgr2=0x559467d53030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:00.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.122+0000 7fcac04f8000 1 --2- 172.21.15.83:0/338612950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559467d37b20 0x559467d53030 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fcab00098c0 tx=0x7fcab0009d80 comp rx=0 tx=0).stop 2024-04-17T10:42:00.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.122+0000 7fcac04f8000 1 -- 172.21.15.83:0/338612950 shutdown_connections 2024-04-17T10:42:00.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.122+0000 7fcac04f8000 1 --2- 172.21.15.83:0/338612950 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x559467d53f90 0x559467d543a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.122+0000 7fcac04f8000 1 --2- 172.21.15.83:0/338612950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559467d37b20 0x559467d53030 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.122+0000 7fcac04f8000 1 --2- 172.21.15.83:0/338612950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559467d37200 0x559467d978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.122+0000 7fcac04f8000 1 --2- 172.21.15.83:0/338612950 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559467d1bf60 0x559467d97380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.122+0000 7fcac04f8000 1 -- 172.21.15.83:0/338612950 >> 172.21.15.83:0/338612950 conn(0x559467ce44f0 msgr2=0x559467ceec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:00.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.122+0000 7fca9dffb640 0 client.5919 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:00.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.122+0000 7fcac04f8000 1 -- 172.21.15.83:0/338612950 shutdown_connections 2024-04-17T10:42:00.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.122+0000 7fcac04f8000 1 -- 172.21.15.83:0/338612950 wait complete. 2024-04-17T10:42:00.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.122+0000 7fcac04f8000 20 client.5919 trim_cache size 0 max 16384 2024-04-17T10:42:00.128 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:42:00.128 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-17T10:42:00.128 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "7c54d888-cbc0-4059-bc71-1ed816f1ab84", 2024-04-17T10:42:00.128 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-17T10:42:00.129 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:42:00.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.122+0000 7fcabd38b640 1 -- 172.21.15.83:0/1319310903 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fca98066f60 msgr2=0x7fca98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:00.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.122+0000 7fcabd38b640 1 --2- 172.21.15.83:0/1319310903 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fca98066f60 0x7fca98069420 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8000f70 tx=0x7fcaa8009040 comp rx=0 tx=0).stop 2024-04-17T10:42:00.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.122+0000 7fcabd38b640 1 -- 172.21.15.83:0/1319310903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcab80ca560 msgr2=0x7fcab8157950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:00.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.122+0000 7fcabd38b640 1 --2- 172.21.15.83:0/1319310903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcab80ca560 0x7fcab8157950 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fcaa00ae040 tx=0x7fcaa001ab60 comp rx=0 tx=0).stop 2024-04-17T10:42:00.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.122+0000 7fcabd38b640 1 -- 172.21.15.83:0/1319310903 shutdown_connections 2024-04-17T10:42:00.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.122+0000 7fcabd38b640 1 --2- 172.21.15.83:0/1319310903 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fca98066f60 0x7fca98069420 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.122+0000 7fcabd38b640 1 --2- 172.21.15.83:0/1319310903 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcab80cf320 0x7fcab809a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.122+0000 7fcabd38b640 1 --2- 172.21.15.83:0/1319310903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcab80ca560 0x7fcab8157950 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.122+0000 7fcabd38b640 1 --2- 172.21.15.83:0/1319310903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcab806bbf0 0x7fcab80c98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.122+0000 7fcabd38b640 1 -- 172.21.15.83:0/1319310903 >> 172.21.15.83:0/1319310903 conn(0x7fcab80cc990 msgr2=0x7fcab806d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:00.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.122+0000 7fcabd38b640 1 -- 172.21.15.83:0/1319310903 shutdown_connections 2024-04-17T10:42:00.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.122+0000 7fcabd38b640 1 -- 172.21.15.83:0/1319310903 wait complete. 2024-04-17T10:42:00.135 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' returned '{'return_code': 0, 'scrub_tag': '7c54d888-cbc0-4059-bc71-1ed816f1ab84', 'mode': 'asynchronous'}' 2024-04-17T10:42:00.136 INFO:tasks.cephfs.test_scrub_checks:First run: flushing base dir / 2024-04-17T10:42:00.136 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-17T10:42:00.136 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:42:00.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.250+0000 7f1e0711b640 1 Processor -- start 2024-04-17T10:42:00.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.250+0000 7f1e0711b640 1 -- start start 2024-04-17T10:42:00.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.250+0000 7f1e0711b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1e0015afa0 0x7f1e0015b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.250+0000 7f1e0711b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1e0006de90 0x7f1e0015b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.250+0000 7f1e0711b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e0006e320 con 0x7f1e000cf2e0 2024-04-17T10:42:00.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.250+0000 7f1e0711b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e000708a0 con 0x7f1e0015afa0 2024-04-17T10:42:00.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.250+0000 7f1e0711b640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e0015be00 con 0x7f1e0006de90 2024-04-17T10:42:00.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.250+0000 7f1e056b8640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1e000cf2e0 0x7f1e000712f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60718/0 (socket says 172.21.15.83:60718) 2024-04-17T10:42:00.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.250+0000 7f1e056b8640 1 -- 172.21.15.83:0/414558640 learned_addr learned my addr 172.21.15.83:0/414558640 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:00.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.250+0000 7f1dfffff640 1 --2- 172.21.15.83:0/414558640 >> v2:172.21.15.167:3300/0 conn(0x7f1e0006de90 0x7f1e0015b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.250+0000 7f1e04eb7640 1 --2- 172.21.15.83:0/414558640 >> v2:172.21.15.83:3300/0 conn(0x7f1e0015afa0 0x7f1e0015b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.250+0000 7f1dff7fe640 1 -- 172.21.15.83:0/414558640 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2200086758 0 0) 0x7f1e0006e320 con 0x7f1e000cf2e0 2024-04-17T10:42:00.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.250+0000 7f1dff7fe640 1 -- 172.21.15.83:0/414558640 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1df0003530 con 0x7f1e000cf2e0 2024-04-17T10:42:00.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.250+0000 7f1e04eb7640 1 -- 172.21.15.83:0/414558640 >> v2:172.21.15.167:3300/0 conn(0x7f1e0006de90 msgr2=0x7f1e0015b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:00.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.250+0000 7f1e04eb7640 1 --2- 172.21.15.83:0/414558640 >> v2:172.21.15.167:3300/0 conn(0x7f1e0006de90 0x7f1e0015b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.250+0000 7f1e04eb7640 1 -- 172.21.15.83:0/414558640 >> v1:172.21.15.83:6789/0 conn(0x7f1e000cf2e0 legacy=0x7f1e000712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:00.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.250+0000 7f1e04eb7640 1 -- 172.21.15.83:0/414558640 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0006e600 con 0x7f1e0015afa0 2024-04-17T10:42:00.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.254+0000 7f1e04eb7640 1 --2- 172.21.15.83:0/414558640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e0015afa0 0x7f1e0015b380 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1e000b2680 tx=0x7f1df401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11d852fedb324ce server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:00.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.254+0000 7f1dff7fe640 1 -- 172.21.15.83:0/414558640 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df40b0070 con 0x7f1e0015afa0 2024-04-17T10:42:00.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.254+0000 7f1dff7fe640 1 -- 172.21.15.83:0/414558640 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df40ab600 con 0x7f1e0015afa0 2024-04-17T10:42:00.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.254+0000 7f1dff7fe640 1 -- 172.21.15.83:0/414558640 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df40ab9b0 con 0x7f1e0015afa0 2024-04-17T10:42:00.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.254+0000 7f1e0711b640 1 -- 172.21.15.83:0/414558640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e0015afa0 msgr2=0x7f1e0015b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:00.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.254+0000 7f1e0711b640 1 --2- 172.21.15.83:0/414558640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e0015afa0 0x7f1e0015b380 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1e000b2680 tx=0x7f1df401c480 comp rx=0 tx=0).stop 2024-04-17T10:42:00.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.254+0000 7f1e0711b640 1 -- 172.21.15.83:0/414558640 shutdown_connections 2024-04-17T10:42:00.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.254+0000 7f1e0711b640 1 --2- 172.21.15.83:0/414558640 >> v2:172.21.15.167:3300/0 conn(0x7f1e0006de90 0x7f1e0015b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.254+0000 7f1e0711b640 1 --2- 172.21.15.83:0/414558640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e0015afa0 0x7f1e0015b380 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.254+0000 7f1e0711b640 1 -- 172.21.15.83:0/414558640 >> 172.21.15.83:0/414558640 conn(0x7f1e000cc450 msgr2=0x7f1e000cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:00.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.254+0000 7f1e0711b640 1 -- 172.21.15.83:0/414558640 shutdown_connections 2024-04-17T10:42:00.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.254+0000 7f1e0711b640 1 -- 172.21.15.83:0/414558640 wait complete. 2024-04-17T10:42:00.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.258+0000 7f1e0711b640 1 Processor -- start 2024-04-17T10:42:00.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.258+0000 7f1e0711b640 1 -- start start 2024-04-17T10:42:00.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.258+0000 7f1e0711b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e000cf2e0 0x7f1e00148e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.258+0000 7f1e0711b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e0015afa0 0x7f1e0014aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.258+0000 7f1e0711b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e0014b420 0x7f1e001f3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.258+0000 7f1e0711b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e00167570 con 0x7f1e0014b420 2024-04-17T10:42:00.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.258+0000 7f1e0711b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e001676f0 con 0x7f1e0015afa0 2024-04-17T10:42:00.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.258+0000 7f1e0711b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e00167870 con 0x7f1e000cf2e0 2024-04-17T10:42:00.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.258+0000 7f1e056b8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e0014b420 0x7f1e001f3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.258+0000 7f1e056b8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e0014b420 0x7f1e001f3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41824/0 (socket says 172.21.15.83:41824) 2024-04-17T10:42:00.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.258+0000 7f1e056b8640 1 -- 172.21.15.83:0/359618215 learned_addr learned my addr 172.21.15.83:0/359618215 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:00.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.258+0000 7f1e04eb7640 1 --2- 172.21.15.83:0/359618215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e000cf2e0 0x7f1e00148e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.258+0000 7f1dfffff640 1 --2- 172.21.15.83:0/359618215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e0015afa0 0x7f1e0014aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.262+0000 7f1e056b8640 1 -- 172.21.15.83:0/359618215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e000cf2e0 msgr2=0x7f1e00148e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:00.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.262+0000 7f1e056b8640 1 --2- 172.21.15.83:0/359618215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e000cf2e0 0x7f1e00148e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.262+0000 7f1e056b8640 1 -- 172.21.15.83:0/359618215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e0015afa0 msgr2=0x7f1e0014aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:00.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.262+0000 7f1e056b8640 1 --2- 172.21.15.83:0/359618215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e0015afa0 0x7f1e0014aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.262+0000 7f1e056b8640 1 -- 172.21.15.83:0/359618215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e00149620 con 0x7f1e0014b420 2024-04-17T10:42:00.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.262+0000 7f1e056b8640 1 --2- 172.21.15.83:0/359618215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e0014b420 0x7f1e001f3f30 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1df800ede0 tx=0x7f1df800c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:00.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.262+0000 7f1dfd7fa640 1 -- 172.21.15.83:0/359618215 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df8015070 con 0x7f1e0014b420 2024-04-17T10:42:00.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.262+0000 7f1dfd7fa640 1 -- 172.21.15.83:0/359618215 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df8011670 con 0x7f1e0014b420 2024-04-17T10:42:00.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.262+0000 7f1dfd7fa640 1 -- 172.21.15.83:0/359618215 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df8011a80 con 0x7f1e0014b420 2024-04-17T10:42:00.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.262+0000 7f1e0711b640 1 -- 172.21.15.83:0/359618215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e00149900 con 0x7f1e0014b420 2024-04-17T10:42:00.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.262+0000 7f1e0711b640 1 -- 172.21.15.83:0/359618215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e00149ec0 con 0x7f1e0014b420 2024-04-17T10:42:00.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.262+0000 7f1e0711b640 1 -- 172.21.15.83:0/359618215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e00074f90 con 0x7f1e0014b420 2024-04-17T10:42:00.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.262+0000 7f1dfd7fa640 1 -- 172.21.15.83:0/359618215 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1df8018070 con 0x7f1e0014b420 2024-04-17T10:42:00.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.262+0000 7f1dfd7fa640 1 --2- 172.21.15.83:0/359618215 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1de4066f90 0x7f1de4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.262+0000 7f1dfd7fa640 1 -- 172.21.15.83:0/359618215 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1df8087c50 con 0x7f1e0014b420 2024-04-17T10:42:00.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.262+0000 7f1e04eb7640 1 --2- 172.21.15.83:0/359618215 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1de4066f90 0x7f1de4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.266+0000 7f1e04eb7640 1 --2- 172.21.15.83:0/359618215 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1de4066f90 0x7f1de4069450 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1df40ad000 tx=0x7f1df40a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:00.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.270+0000 7f1dfd7fa640 1 -- 172.21.15.83:0/359618215 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1df808d020 con 0x7f1e0014b420 2024-04-17T10:42:00.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.454+0000 7f1e0711b640 1 -- 172.21.15.83:0/359618215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e0006ec80 con 0x7f1e0014b420 2024-04-17T10:42:00.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.458+0000 7f1dfd7fa640 1 -- 172.21.15.83:0/359618215 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f1df8058660 con 0x7f1e0014b420 2024-04-17T10:42:00.456 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:00.456 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/161398126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":161398126},{"type":"v1","addr":"172.21.15.167:6833","nonce":161398126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1085380618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1085380618},{"type":"v1","addr":"172.21.15.167:6835","nonce":1085380618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/836442291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":836442291},{"type":"v1","addr":"172.21.15.83:6835","nonce":836442291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:41:37.773342+0000","modified":"2024-04-17T10:41:39.788394+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5682":{"gid":5682,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1894547386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1894547386},{"type":"v1","addr":"172.21.15.83:6837","nonce":1894547386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-17T10:42:00.456 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 21 2024-04-17T10:42:00.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.458+0000 7f1e0711b640 1 -- 172.21.15.83:0/359618215 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1de4066f90 msgr2=0x7f1de4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:00.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.458+0000 7f1e0711b640 1 --2- 172.21.15.83:0/359618215 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1de4066f90 0x7f1de4069450 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1df40ad000 tx=0x7f1df40a6000 comp rx=0 tx=0).stop 2024-04-17T10:42:00.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.458+0000 7f1e0711b640 1 -- 172.21.15.83:0/359618215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e0014b420 msgr2=0x7f1e001f3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:00.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.458+0000 7f1e0711b640 1 --2- 172.21.15.83:0/359618215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e0014b420 0x7f1e001f3f30 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1df800ede0 tx=0x7f1df800c640 comp rx=0 tx=0).stop 2024-04-17T10:42:00.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.458+0000 7f1e0711b640 1 -- 172.21.15.83:0/359618215 shutdown_connections 2024-04-17T10:42:00.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.458+0000 7f1e0711b640 1 --2- 172.21.15.83:0/359618215 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1de4066f90 0x7f1de4069450 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.458+0000 7f1e0711b640 1 --2- 172.21.15.83:0/359618215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e0014b420 0x7f1e001f3f30 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.458+0000 7f1e0711b640 1 --2- 172.21.15.83:0/359618215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e0015afa0 0x7f1e0014aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.458+0000 7f1e0711b640 1 --2- 172.21.15.83:0/359618215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e000cf2e0 0x7f1e00148e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.458+0000 7f1e0711b640 1 -- 172.21.15.83:0/359618215 >> 172.21.15.83:0/359618215 conn(0x7f1e000cc450 msgr2=0x7f1e000b0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:00.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.458+0000 7f1e0711b640 1 -- 172.21.15.83:0/359618215 shutdown_connections 2024-04-17T10:42:00.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.458+0000 7f1e0711b640 1 -- 172.21.15.83:0/359618215 wait complete. 2024-04-17T10:42:00.474 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path / 2024-04-17T10:42:00.588 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:42:00.588 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0 2024-04-17T10:42:00.588 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:42:00.601 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-17T10:42:00.602 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /' 2024-04-17T10:42:00.602 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-17T10:42:00.602 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / 2024-04-17T10:42:00.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.726+0000 7ff681a41640 1 Processor -- start 2024-04-17T10:42:00.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.726+0000 7ff681a41640 1 -- start start 2024-04-17T10:42:00.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.726+0000 7ff681a41640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff67c0b4040 0x7ff67c0ae250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.726+0000 7ff681a41640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff67c0b4570 con 0x7ff67c147350 2024-04-17T10:42:00.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.726+0000 7ff681a41640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff67c0b4850 con 0x7ff67c06b2f0 2024-04-17T10:42:00.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.726+0000 7ff681a41640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff67c147730 con 0x7ff67c0b4040 2024-04-17T10:42:00.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.726+0000 7ff67b7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff67c0b4040 0x7ff67c0ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.726+0000 7ff67affd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff67c147350 0x7ff67c06abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60726/0 (socket says 172.21.15.83:60726) 2024-04-17T10:42:00.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.726+0000 7ff67affd640 1 -- 172.21.15.83:0/493128142 learned_addr learned my addr 172.21.15.83:0/493128142 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:00.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.726+0000 7ff67b7fe640 1 -- 172.21.15.83:0/493128142 >> v1:172.21.15.167:6789/0 conn(0x7ff67c06b2f0 legacy=0x7ff67c06d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:00.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.726+0000 7ff67b7fe640 1 -- 172.21.15.83:0/493128142 >> v1:172.21.15.83:6789/0 conn(0x7ff67c147350 legacy=0x7ff67c06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:00.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.726+0000 7ff67b7fe640 1 -- 172.21.15.83:0/493128142 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67c0b4b30 con 0x7ff67c0b4040 2024-04-17T10:42:00.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.726+0000 7ff67a7fc640 1 -- 172.21.15.83:0/493128142 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2705944576 0 0) 0x7ff67c0b4570 con 0x7ff67c147350 2024-04-17T10:42:00.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.730+0000 7ff67b7fe640 1 --2- 172.21.15.83:0/493128142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff67c0b4040 0x7ff67c0ae250 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff674009d90 tx=0x7ff67401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494b68dbc675f3ba server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:00.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.730+0000 7ff67a7fc640 1 -- 172.21.15.83:0/493128142 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6740b2070 con 0x7ff67c0b4040 2024-04-17T10:42:00.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.730+0000 7ff67a7fc640 1 -- 172.21.15.83:0/493128142 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6740a59c0 con 0x7ff67c0b4040 2024-04-17T10:42:00.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.730+0000 7ff67a7fc640 1 -- 172.21.15.83:0/493128142 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6740a5d90 con 0x7ff67c0b4040 2024-04-17T10:42:00.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.730+0000 7ff681a41640 1 -- 172.21.15.83:0/493128142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff67c0b4040 msgr2=0x7ff67c0ae250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:00.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.730+0000 7ff681a41640 1 --2- 172.21.15.83:0/493128142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff67c0b4040 0x7ff67c0ae250 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff674009d90 tx=0x7ff67401c900 comp rx=0 tx=0).stop 2024-04-17T10:42:00.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.730+0000 7ff681a41640 1 -- 172.21.15.83:0/493128142 shutdown_connections 2024-04-17T10:42:00.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.730+0000 7ff681a41640 1 --2- 172.21.15.83:0/493128142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff67c0b4040 0x7ff67c0ae250 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.730+0000 7ff681a41640 1 -- 172.21.15.83:0/493128142 >> 172.21.15.83:0/493128142 conn(0x7ff67c0d0740 msgr2=0x7ff67c0d0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:00.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.730+0000 7ff681a41640 1 -- 172.21.15.83:0/493128142 shutdown_connections 2024-04-17T10:42:00.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.730+0000 7ff681a41640 1 -- 172.21.15.83:0/493128142 wait complete. 2024-04-17T10:42:00.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.730+0000 7ff681a41640 1 Processor -- start 2024-04-17T10:42:00.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.730+0000 7ff681a41640 1 -- start start 2024-04-17T10:42:00.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.730+0000 7ff681a41640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff67c06b2f0 0x7ff67c15cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.730+0000 7ff681a41640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff67c0b4040 0x7ff67c15d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.730+0000 7ff681a41640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff67c147350 0x7ff67c160370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.730+0000 7ff681a41640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c16b700 con 0x7ff67c06b2f0 2024-04-17T10:42:00.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.730+0000 7ff681a41640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c0afe30 con 0x7ff67c0b4040 2024-04-17T10:42:00.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.730+0000 7ff681a41640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c16b880 con 0x7ff67c147350 2024-04-17T10:42:00.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.730+0000 7ff67affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff67c06b2f0 0x7ff67c15cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.730+0000 7ff67affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff67c06b2f0 0x7ff67c15cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41832/0 (socket says 172.21.15.83:41832) 2024-04-17T10:42:00.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.730+0000 7ff67affd640 1 -- 172.21.15.83:0/2283408362 learned_addr learned my addr 172.21.15.83:0/2283408362 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:00.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.730+0000 7ff67affd640 1 -- 172.21.15.83:0/2283408362 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff67c147350 msgr2=0x7ff67c160370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:00.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.730+0000 7ff67affd640 1 --2- 172.21.15.83:0/2283408362 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff67c147350 0x7ff67c160370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.730+0000 7ff67affd640 1 -- 172.21.15.83:0/2283408362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff67c0b4040 msgr2=0x7ff67c15d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:00.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.730+0000 7ff67affd640 1 --2- 172.21.15.83:0/2283408362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff67c0b4040 0x7ff67c15d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.730+0000 7ff67affd640 1 -- 172.21.15.83:0/2283408362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67c0ae480 con 0x7ff67c06b2f0 2024-04-17T10:42:00.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.734+0000 7ff67affd640 1 --2- 172.21.15.83:0/2283408362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff67c06b2f0 0x7ff67c15cdd0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00c4b0 tx=0x7ff66c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:00.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.734+0000 7ff678ff9640 1 -- 172.21.15.83:0/2283408362 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c018070 con 0x7ff67c06b2f0 2024-04-17T10:42:00.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.734+0000 7ff678ff9640 1 -- 172.21.15.83:0/2283408362 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66c007500 con 0x7ff67c06b2f0 2024-04-17T10:42:00.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.734+0000 7ff678ff9640 1 -- 172.21.15.83:0/2283408362 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c004a30 con 0x7ff67c06b2f0 2024-04-17T10:42:00.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.734+0000 7ff681a41640 1 -- 172.21.15.83:0/2283408362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff67c1431f0 con 0x7ff67c06b2f0 2024-04-17T10:42:00.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.734+0000 7ff681a41640 1 -- 172.21.15.83:0/2283408362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff67c1454c0 con 0x7ff67c06b2f0 2024-04-17T10:42:00.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.734+0000 7ff678ff9640 1 -- 172.21.15.83:0/2283408362 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff66c004bd0 con 0x7ff67c06b2f0 2024-04-17T10:42:00.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.734+0000 7ff678ff9640 1 --2- 172.21.15.83:0/2283408362 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff65c066f60 0x7ff65c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.734+0000 7ff678ff9640 1 -- 172.21.15.83:0/2283408362 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff66c0842d0 con 0x7ff67c06b2f0 2024-04-17T10:42:00.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.734+0000 7ff6727fc640 1 --2- 172.21.15.83:0/2283408362 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff65c066f60 0x7ff65c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.734+0000 7ff6727fc640 1 --2- 172.21.15.83:0/2283408362 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff65c066f60 0x7ff65c069420 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff67c1f9bd0 tx=0x7ff6640079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:00.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.734+0000 7ff684bae000 1 Processor -- start 2024-04-17T10:42:00.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.734+0000 7ff684bae000 1 -- start start 2024-04-17T10:42:00.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff684bae000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d59036f000 0x55d59036f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff684bae000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d59036f920 0x55d590324ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff684bae000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d590325410 0x55d5903257f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff684bae000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d590172f40 con 0x55d590325410 2024-04-17T10:42:00.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff684bae000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5900fafc0 con 0x55d59036f000 2024-04-17T10:42:00.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff684bae000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d59016efe0 con 0x55d59036f920 2024-04-17T10:42:00.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff67b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d590325410 0x55d5903257f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff67b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d590325410 0x55d5903257f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41838/0 (socket says 172.21.15.83:41838) 2024-04-17T10:42:00.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff67b7fe640 1 -- 172.21.15.83:0/2643687629 learned_addr learned my addr 172.21.15.83:0/2643687629 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:00.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff67affd640 1 --2- 172.21.15.83:0/2643687629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d59036f000 0x55d59036f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff67b7fe640 1 -- 172.21.15.83:0/2643687629 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d59036f920 msgr2=0x55d590324ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:00.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff6727fc640 1 --2- 172.21.15.83:0/2643687629 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d59036f920 0x55d590324ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff67b7fe640 1 --2- 172.21.15.83:0/2643687629 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d59036f920 0x55d590324ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff67b7fe640 1 -- 172.21.15.83:0/2643687629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d59036f000 msgr2=0x55d59036f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:00.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff67b7fe640 1 --2- 172.21.15.83:0/2643687629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d59036f000 0x55d59036f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff67b7fe640 1 -- 172.21.15.83:0/2643687629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5901baa50 con 0x55d590325410 2024-04-17T10:42:00.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff67b7fe640 1 --2- 172.21.15.83:0/2643687629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d590325410 0x55d5903257f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff6740b0040 tx=0x7ff674004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f973f7c4f3533ff5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:00.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff6717fa640 1 -- 172.21.15.83:0/2643687629 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6740b2040 con 0x55d590325410 2024-04-17T10:42:00.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff6717fa640 1 -- 172.21.15.83:0/2643687629 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6740be070 con 0x55d590325410 2024-04-17T10:42:00.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff6717fa640 1 -- 172.21.15.83:0/2643687629 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6740a5cc0 con 0x55d590325410 2024-04-17T10:42:00.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff684bae000 1 -- 172.21.15.83:0/2643687629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d590325410 msgr2=0x55d5903257f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:00.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff684bae000 1 --2- 172.21.15.83:0/2643687629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d590325410 0x55d5903257f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff6740b0040 tx=0x7ff674004a50 comp rx=0 tx=0).stop 2024-04-17T10:42:00.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff684bae000 1 -- 172.21.15.83:0/2643687629 shutdown_connections 2024-04-17T10:42:00.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff684bae000 1 --2- 172.21.15.83:0/2643687629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d590325410 0x55d5903257f0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff684bae000 1 --2- 172.21.15.83:0/2643687629 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d59036f920 0x55d590324ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff684bae000 1 --2- 172.21.15.83:0/2643687629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d59036f000 0x55d59036f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff684bae000 1 -- 172.21.15.83:0/2643687629 >> 172.21.15.83:0/2643687629 conn(0x55d5903213c0 msgr2=0x55d5903237b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:00.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff684bae000 1 -- 172.21.15.83:0/2643687629 shutdown_connections 2024-04-17T10:42:00.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff684bae000 1 -- 172.21.15.83:0/2643687629 wait complete. 2024-04-17T10:42:00.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff684bae000 1 -- start start 2024-04-17T10:42:00.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff684bae000 1 Processor -- start 2024-04-17T10:42:00.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff684bae000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d590325410 0x55d590330e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff684bae000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d59036f000 0x55d590331340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff684bae000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59036f920 0x55d590331880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff684bae000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d590177570 con 0x55d59036f920 2024-04-17T10:42:00.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff684bae000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d59012d5f0 con 0x55d590325410 2024-04-17T10:42:00.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff684bae000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d59013f870 con 0x55d59036f000 2024-04-17T10:42:00.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff684bae000 10 client.0 _ll_register_callbacks cb 0x55d5902277a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:00.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff67b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59036f920 0x55d590331880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff67b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59036f920 0x55d590331880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41854/0 (socket says 172.21.15.83:41854) 2024-04-17T10:42:00.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff67b7fe640 1 -- 172.21.15.83:0/247162396 learned_addr learned my addr 172.21.15.83:0/247162396 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:00.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff67affd640 1 --2- 172.21.15.83:0/247162396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d590325410 0x55d590330e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff67b7fe640 1 -- 172.21.15.83:0/247162396 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d59036f000 msgr2=0x55d590331340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:00.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff67b7fe640 1 --2- 172.21.15.83:0/247162396 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d59036f000 0x55d590331340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff67b7fe640 1 -- 172.21.15.83:0/247162396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d590325410 msgr2=0x55d590330e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:00.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff67b7fe640 1 --2- 172.21.15.83:0/247162396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d590325410 0x55d590330e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff67b7fe640 1 -- 172.21.15.83:0/247162396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5900fb210 con 0x55d59036f920 2024-04-17T10:42:00.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff67affd640 1 --2- 172.21.15.83:0/247162396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d590325410 0x55d590330e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:00.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff67b7fe640 1 --2- 172.21.15.83:0/247162396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59036f920 0x55d590331880 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff6740bb040 tx=0x7ff67401b140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef9c13522114fed server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:00.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff65a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:00.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff65a7fc640 1 -- 172.21.15.83:0/247162396 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6740c6070 con 0x55d59036f920 2024-04-17T10:42:00.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff65a7fc640 1 -- 172.21.15.83:0/247162396 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff6740a5a40 con 0x55d59036f920 2024-04-17T10:42:00.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff684bae000 1 -- 172.21.15.83:0/247162396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff6740be070 con 0x55d59036f920 2024-04-17T10:42:00.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff65a7fc640 1 -- 172.21.15.83:0/247162396 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6740d1050 con 0x55d59036f920 2024-04-17T10:42:00.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff684bae000 10 client.5943 fetch_fsmap learned FSMap version 21 2024-04-17T10:42:00.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff684bae000 1 -- 172.21.15.83:0/247162396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55d5901f7400 con 0x55d59036f920 2024-04-17T10:42:00.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.738+0000 7ff65a7fc640 1 -- 172.21.15.83:0/247162396 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6740be070 con 0x55d59036f920 2024-04-17T10:42:00.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.742+0000 7ff65a7fc640 1 -- 172.21.15.83:0/247162396 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff6740d4020 con 0x55d59036f920 2024-04-17T10:42:00.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.742+0000 7ff684bae000 10 client.5943 fetch_fsmap finished waiting for FSMap version 21 2024-04-17T10:42:00.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.742+0000 7ff684bae000 10 client.5943 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:42:00.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.742+0000 7ff684bae000 20 client.5943 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:00.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.742+0000 7ff684bae000 1 --2- 172.21.15.83:0/247162396 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55d59037f2b0 0x55d5903816a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.742+0000 7ff684bae000 4 client.5943 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:42:00.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.742+0000 7ff684bae000 1 -- 172.21.15.83:0/247162396 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d59010ec90 con 0x55d59037f2b0 2024-04-17T10:42:00.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.742+0000 7ff67affd640 1 --2- 172.21.15.83:0/247162396 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55d59037f2b0 0x55d5903816a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.742+0000 7ff65a7fc640 10 client.5943 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:00.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.742+0000 7ff67affd640 1 --2- 172.21.15.83:0/247162396 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55d59037f2b0 0x55d5903816a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddecf6a668b9f3c2 server_cookie=cf0543f5894faa4e in_seq=0 out_seq=0 2024-04-17T10:42:00.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.742+0000 7ff65a7fc640 1 -- 172.21.15.83:0/247162396 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d59010ec90 con 0x55d59037f2b0 2024-04-17T10:42:00.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.742+0000 7ff65a7fc640 10 client.5943 handle_command_reply: tid=0 2024-04-17T10:42:00.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.742+0000 7ff684bae000 1 client.5943 shutdown 2024-04-17T10:42:00.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.742+0000 7ff684bae000 1 -- 172.21.15.83:0/247162396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59036f920 msgr2=0x55d590331880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:00.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.742+0000 7ff684bae000 1 --2- 172.21.15.83:0/247162396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59036f920 0x55d590331880 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff6740bb040 tx=0x7ff67401b140 comp rx=0 tx=0).stop 2024-04-17T10:42:00.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.742+0000 7ff684bae000 1 -- 172.21.15.83:0/247162396 shutdown_connections 2024-04-17T10:42:00.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.742+0000 7ff684bae000 1 --2- 172.21.15.83:0/247162396 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55d59037f2b0 0x55d5903816a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.742+0000 7ff684bae000 1 --2- 172.21.15.83:0/247162396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59036f920 0x55d590331880 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.742+0000 7ff684bae000 1 --2- 172.21.15.83:0/247162396 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d59036f000 0x55d590331340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.742+0000 7ff684bae000 1 --2- 172.21.15.83:0/247162396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d590325410 0x55d590330e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.742+0000 7ff65a7fc640 0 client.5943 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:00.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.742+0000 7ff684bae000 1 -- 172.21.15.83:0/247162396 >> 172.21.15.83:0/247162396 conn(0x55d5903213c0 msgr2=0x55d590324040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:00.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.742+0000 7ff684bae000 1 -- 172.21.15.83:0/247162396 shutdown_connections 2024-04-17T10:42:00.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.742+0000 7ff684bae000 1 -- 172.21.15.83:0/247162396 wait complete. 2024-04-17T10:42:00.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.742+0000 7ff684bae000 20 client.5943 trim_cache size 0 max 16384 2024-04-17T10:42:00.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.754+0000 7ff684bae000 1 Processor -- start 2024-04-17T10:42:00.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.754+0000 7ff684bae000 1 -- start start 2024-04-17T10:42:00.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff684bae000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d590325410 0x55d5903cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff684bae000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d59036f000 0x55d5903cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff684bae000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d59036f920 0x55d5903cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff684bae000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c202c20 con 0x55d590325410 2024-04-17T10:42:00.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff684bae000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c201ec0 con 0x55d59036f000 2024-04-17T10:42:00.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff684bae000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c163aa0 con 0x55d59036f920 2024-04-17T10:42:00.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff67affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d590325410 0x55d5903cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff67affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d590325410 0x55d5903cbd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41860/0 (socket says 172.21.15.83:41860) 2024-04-17T10:42:00.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff67affd640 1 -- 172.21.15.83:0/2332274154 learned_addr learned my addr 172.21.15.83:0/2332274154 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:00.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff67affd640 1 -- 172.21.15.83:0/2332274154 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d59036f920 msgr2=0x55d5903cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:00.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff67affd640 1 --2- 172.21.15.83:0/2332274154 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d59036f920 0x55d5903cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff67affd640 1 -- 172.21.15.83:0/2332274154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d59036f000 msgr2=0x55d5903cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:00.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff67affd640 1 --2- 172.21.15.83:0/2332274154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d59036f000 0x55d5903cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff67affd640 1 -- 172.21.15.83:0/2332274154 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5901cdc10 con 0x55d590325410 2024-04-17T10:42:00.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff67affd640 1 --2- 172.21.15.83:0/2332274154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d590325410 0x55d5903cbd60 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff66c0548a0 tx=0x7ff66c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65cb795c8bed1ae6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:00.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff670ff9640 1 -- 172.21.15.83:0/2332274154 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c052c40 con 0x55d590325410 2024-04-17T10:42:00.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff670ff9640 1 -- 172.21.15.83:0/2332274154 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66c06c070 con 0x55d590325410 2024-04-17T10:42:00.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff670ff9640 1 -- 172.21.15.83:0/2332274154 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c052ea0 con 0x55d590325410 2024-04-17T10:42:00.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff684bae000 1 -- 172.21.15.83:0/2332274154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d590325410 msgr2=0x55d5903cbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:00.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff684bae000 1 --2- 172.21.15.83:0/2332274154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d590325410 0x55d5903cbd60 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff66c0548a0 tx=0x7ff66c009040 comp rx=0 tx=0).stop 2024-04-17T10:42:00.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff684bae000 1 -- 172.21.15.83:0/2332274154 shutdown_connections 2024-04-17T10:42:00.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff684bae000 1 --2- 172.21.15.83:0/2332274154 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d59036f920 0x55d5903cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff684bae000 1 --2- 172.21.15.83:0/2332274154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d59036f000 0x55d5903cc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff684bae000 1 --2- 172.21.15.83:0/2332274154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d590325410 0x55d5903cbd60 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff684bae000 1 -- 172.21.15.83:0/2332274154 >> 172.21.15.83:0/2332274154 conn(0x55d5903213c0 msgr2=0x55d590330660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:00.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff684bae000 1 -- 172.21.15.83:0/2332274154 shutdown_connections 2024-04-17T10:42:00.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff684bae000 1 -- 172.21.15.83:0/2332274154 wait complete. 2024-04-17T10:42:00.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff684bae000 1 -- start start 2024-04-17T10:42:00.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff684bae000 1 Processor -- start 2024-04-17T10:42:00.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff684bae000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d590325410 0x55d5903cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff684bae000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d59036f000 0x55d5903cfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff684bae000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59036f920 0x55d59038b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff684bae000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c142bc0 con 0x55d59036f920 2024-04-17T10:42:00.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff684bae000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d590172f40 con 0x55d59036f000 2024-04-17T10:42:00.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff684bae000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d59012bea0 con 0x55d590325410 2024-04-17T10:42:00.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff684bae000 10 client.0 _ll_register_callbacks cb 0x55d5901c52c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:00.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff67b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59036f920 0x55d59038b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff67b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59036f920 0x55d59038b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41874/0 (socket says 172.21.15.83:41874) 2024-04-17T10:42:00.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff67b7fe640 1 -- 172.21.15.83:0/1259953882 learned_addr learned my addr 172.21.15.83:0/1259953882 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:00.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff6727fc640 1 --2- 172.21.15.83:0/1259953882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d59036f000 0x55d5903cfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff67b7fe640 1 -- 172.21.15.83:0/1259953882 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d590325410 msgr2=0x55d5903cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:00.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff67affd640 1 --2- 172.21.15.83:0/1259953882 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d590325410 0x55d5903cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff67b7fe640 1 --2- 172.21.15.83:0/1259953882 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d590325410 0x55d5903cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff67b7fe640 1 -- 172.21.15.83:0/1259953882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d59036f000 msgr2=0x55d5903cfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff67b7fe640 1 --2- 172.21.15.83:0/1259953882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d59036f000 0x55d5903cfdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff67b7fe640 1 -- 172.21.15.83:0/1259953882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d590107bb0 con 0x55d59036f920 2024-04-17T10:42:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff6727fc640 1 --2- 172.21.15.83:0/1259953882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d59036f000 0x55d5903cfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff67b7fe640 1 --2- 172.21.15.83:0/1259953882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59036f920 0x55d59038b120 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7ff67401c940 tx=0x7ff6740196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3ca5e88670bf08 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff65a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff65a7fc640 1 -- 172.21.15.83:0/1259953882 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6740d9020 con 0x55d59036f920 2024-04-17T10:42:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff65a7fc640 1 -- 172.21.15.83:0/1259953882 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff6740d6ae0 con 0x55d59036f920 2024-04-17T10:42:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.758+0000 7ff684bae000 1 -- 172.21.15.83:0/1259953882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff66c06c070 con 0x55d59036f920 2024-04-17T10:42:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff65a7fc640 1 -- 172.21.15.83:0/1259953882 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6740c6040 con 0x55d59036f920 2024-04-17T10:42:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff684bae000 10 client.5949 fetch_fsmap learned FSMap version 21 2024-04-17T10:42:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff684bae000 1 -- 172.21.15.83:0/1259953882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55d59011a030 con 0x55d59036f920 2024-04-17T10:42:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff65a7fc640 1 -- 172.21.15.83:0/1259953882 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66c06c070 con 0x55d59036f920 2024-04-17T10:42:00.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff65a7fc640 1 -- 172.21.15.83:0/1259953882 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff6740d4040 con 0x55d59036f920 2024-04-17T10:42:00.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff684bae000 10 client.5949 fetch_fsmap finished waiting for FSMap version 21 2024-04-17T10:42:00.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff684bae000 10 client.5949 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:42:00.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff684bae000 20 client.5949 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:00.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff684bae000 1 --2- 172.21.15.83:0/1259953882 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55d59038bb10 0x55d59038bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff684bae000 4 client.5949 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/"}] 2024-04-17T10:42:00.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff684bae000 1 -- 172.21.15.83:0/1259953882 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "scrub start", "path": "/"}) v1 -- 0x55d590180480 con 0x55d59038bb10 2024-04-17T10:42:00.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff67affd640 1 --2- 172.21.15.83:0/1259953882 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55d59038bb10 0x55d59038bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff65a7fc640 10 client.5949 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:00.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff67affd640 1 --2- 172.21.15.83:0/1259953882 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55d59038bb10 0x55d59038bf20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aed962e6c8758c51 server_cookie=e9a17161a5b36c77 in_seq=0 out_seq=0 2024-04-17T10:42:00.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff65a7fc640 1 -- 172.21.15.83:0/1259953882 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d590180480 con 0x55d59038bb10 2024-04-17T10:42:00.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff65a7fc640 10 client.5949 handle_command_reply: tid=0 2024-04-17T10:42:00.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff684bae000 1 client.5949 shutdown 2024-04-17T10:42:00.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff684bae000 1 -- 172.21.15.83:0/1259953882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59036f920 msgr2=0x55d59038b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:00.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff684bae000 1 --2- 172.21.15.83:0/1259953882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59036f920 0x55d59038b120 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7ff67401c940 tx=0x7ff6740196a0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff684bae000 1 -- 172.21.15.83:0/1259953882 shutdown_connections 2024-04-17T10:42:00.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff684bae000 1 --2- 172.21.15.83:0/1259953882 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55d59038bb10 0x55d59038bf20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff684bae000 1 --2- 172.21.15.83:0/1259953882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59036f920 0x55d59038b120 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff684bae000 1 --2- 172.21.15.83:0/1259953882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d59036f000 0x55d5903cfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff684bae000 1 --2- 172.21.15.83:0/1259953882 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d590325410 0x55d5903cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff684bae000 1 -- 172.21.15.83:0/1259953882 >> 172.21.15.83:0/1259953882 conn(0x55d5903213c0 msgr2=0x55d590330ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:00.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff65a7fc640 0 client.5949 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:00.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff684bae000 1 -- 172.21.15.83:0/1259953882 shutdown_connections 2024-04-17T10:42:00.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff684bae000 1 -- 172.21.15.83:0/1259953882 wait complete. 2024-04-17T10:42:00.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff684bae000 20 client.5949 trim_cache size 0 max 16384 2024-04-17T10:42:00.767 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:42:00.767 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-17T10:42:00.767 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "f26bab6e-d69d-4de2-9b8c-b1e3503848d5", 2024-04-17T10:42:00.767 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-17T10:42:00.767 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:42:00.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff681a41640 1 -- 172.21.15.83:0/2283408362 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff65c066f60 msgr2=0x7ff65c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:00.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff681a41640 1 --2- 172.21.15.83:0/2283408362 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff65c066f60 0x7ff65c069420 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff67c1f9bd0 tx=0x7ff6640079c0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff681a41640 1 -- 172.21.15.83:0/2283408362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff67c06b2f0 msgr2=0x7ff67c15cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:00.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff681a41640 1 --2- 172.21.15.83:0/2283408362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff67c06b2f0 0x7ff67c15cdd0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00c4b0 tx=0x7ff66c00c970 comp rx=0 tx=0).stop 2024-04-17T10:42:00.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff681a41640 1 -- 172.21.15.83:0/2283408362 shutdown_connections 2024-04-17T10:42:00.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff681a41640 1 --2- 172.21.15.83:0/2283408362 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff65c066f60 0x7ff65c069420 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff681a41640 1 --2- 172.21.15.83:0/2283408362 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff67c147350 0x7ff67c160370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff681a41640 1 --2- 172.21.15.83:0/2283408362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff67c0b4040 0x7ff67c15d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff681a41640 1 --2- 172.21.15.83:0/2283408362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff67c06b2f0 0x7ff67c15cdd0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff681a41640 1 -- 172.21.15.83:0/2283408362 >> 172.21.15.83:0/2283408362 conn(0x7ff67c0d0740 msgr2=0x7ff67c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:00.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff681a41640 1 -- 172.21.15.83:0/2283408362 shutdown_connections 2024-04-17T10:42:00.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.762+0000 7ff681a41640 1 -- 172.21.15.83:0/2283408362 wait complete. 2024-04-17T10:42:00.777 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /' returned '{'return_code': 0, 'scrub_tag': 'f26bab6e-d69d-4de2-9b8c-b1e3503848d5', 'mode': 'asynchronous'}' 2024-04-17T10:42:00.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:42:00.778 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_dir_0) 2024-04-17T10:42:00.785 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_0' 2024-04-17T10:42:00.785 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:42:00.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.934+0000 7f2d2fae6640 1 Processor -- start 2024-04-17T10:42:00.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.934+0000 7f2d2fae6640 1 -- start start 2024-04-17T10:42:00.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.934+0000 7f2d2fae6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d280b2d80 0x7f2d28146e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.934+0000 7f2d2fae6640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d2806db10 con 0x7f2d2806d690 2024-04-17T10:42:00.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.934+0000 7f2d2fae6640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d28147570 con 0x7f2d28146a00 2024-04-17T10:42:00.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.934+0000 7f2d2fae6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2806c960 con 0x7f2d280b2d80 2024-04-17T10:42:00.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.934+0000 7f2d2e083640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2d2806d690 0x7f2d28070470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60728/0 (socket says 172.21.15.83:60728) 2024-04-17T10:42:00.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.934+0000 7f2d2e083640 1 -- 172.21.15.83:0/1986529599 learned_addr learned my addr 172.21.15.83:0/1986529599 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.934+0000 7f2d2d081640 1 --2- 172.21.15.83:0/1986529599 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d280b2d80 0x7f2d28146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.934+0000 7f2d2c880640 1 -- 172.21.15.83:0/1986529599 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2999250051 0 0) 0x7f2d2806db10 con 0x7f2d2806d690 2024-04-17T10:42:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.934+0000 7f2d2c880640 1 -- 172.21.15.83:0/1986529599 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d14003530 con 0x7f2d2806d690 2024-04-17T10:42:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.934+0000 7f2d2c880640 1 -- 172.21.15.83:0/1986529599 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2363056591 0 0) 0x7f2d14003530 con 0x7f2d2806d690 2024-04-17T10:42:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.934+0000 7f2d2c880640 1 -- 172.21.15.83:0/1986529599 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d280b2d80 msgr2=0x7f2d28146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.934+0000 7f2d2c880640 1 --2- 172.21.15.83:0/1986529599 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d280b2d80 0x7f2d28146e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.934+0000 7f2d2c880640 1 -- 172.21.15.83:0/1986529599 >> v1:172.21.15.167:6789/0 conn(0x7f2d28146a00 legacy=0x7f2d280b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.934+0000 7f2d2c880640 1 -- 172.21.15.83:0/1986529599 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2806cbb0 con 0x7f2d2806d690 2024-04-17T10:42:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.934+0000 7f2d2c880640 1 -- 172.21.15.83:0/1986529599 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f2d20001a00 con 0x7f2d2806d690 2024-04-17T10:42:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.934+0000 7f2d2c880640 1 -- 172.21.15.83:0/1986529599 >> v1:172.21.15.83:6789/0 conn(0x7f2d2806d690 legacy=0x7f2d28070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.934+0000 7f2d2c880640 1 --2- 172.21.15.83:0/1986529599 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d280b2d80 0x7f2d140039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.934+0000 7f2d2c880640 1 --2- 172.21.15.83:0/1986529599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d14003f70 0x7f2d14004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.934+0000 7f2d2c880640 1 --2- 172.21.15.83:0/1986529599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d14004d50 0x7f2d140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.934+0000 7f2d2c880640 1 -- 172.21.15.83:0/1986529599 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2806c960 con 0x7f2d14003f70 2024-04-17T10:42:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.934+0000 7f2d2c880640 1 -- 172.21.15.83:0/1986529599 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1400bab0 con 0x7f2d14004d50 2024-04-17T10:42:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.934+0000 7f2d2c880640 1 -- 172.21.15.83:0/1986529599 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1400bcb0 con 0x7f2d280b2d80 2024-04-17T10:42:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.934+0000 7f2d2d081640 1 --2- 172.21.15.83:0/1986529599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d14003f70 0x7f2d14004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.934+0000 7f2d2d081640 1 -- 172.21.15.83:0/1986529599 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d280b2d80 msgr2=0x7f2d140039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.934+0000 7f2d2d081640 1 --2- 172.21.15.83:0/1986529599 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d280b2d80 0x7f2d140039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.934+0000 7f2d2e083640 1 --2- 172.21.15.83:0/1986529599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d14004d50 0x7f2d140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.934+0000 7f2d2d081640 1 -- 172.21.15.83:0/1986529599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d14004d50 msgr2=0x7f2d140051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.934+0000 7f2d2d081640 1 --2- 172.21.15.83:0/1986529599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d14004d50 0x7f2d140051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.934+0000 7f2d2d081640 1 -- 172.21.15.83:0/1986529599 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d14003530 con 0x7f2d14003f70 2024-04-17T10:42:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.934+0000 7f2d2e083640 1 --2- 172.21.15.83:0/1986529599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d14004d50 0x7f2d140051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:00.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.938+0000 7f2d2d081640 1 --2- 172.21.15.83:0/1986529599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d14003f70 0x7f2d14004810 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2d281479a0 tx=0x7f2d1801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7aec03de4e70bb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:00.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.938+0000 7f2d2c880640 1 -- 172.21.15.83:0/1986529599 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d180b2070 con 0x7f2d14003f70 2024-04-17T10:42:00.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.938+0000 7f2d2c880640 1 -- 172.21.15.83:0/1986529599 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d180a5620 con 0x7f2d14003f70 2024-04-17T10:42:00.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.938+0000 7f2d2c880640 1 -- 172.21.15.83:0/1986529599 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d180096b0 con 0x7f2d14003f70 2024-04-17T10:42:00.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.938+0000 7f2d2fae6640 1 -- 172.21.15.83:0/1986529599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d14003f70 msgr2=0x7f2d14004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:00.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.938+0000 7f2d2fae6640 1 --2- 172.21.15.83:0/1986529599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d14003f70 0x7f2d14004810 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2d281479a0 tx=0x7f2d1801ca30 comp rx=0 tx=0).stop 2024-04-17T10:42:00.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.938+0000 7f2d2e083640 1 -- 172.21.15.83:0/1986529599 reap_dead start 2024-04-17T10:42:00.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.938+0000 7f2d2fae6640 1 -- 172.21.15.83:0/1986529599 shutdown_connections 2024-04-17T10:42:00.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.938+0000 7f2d2fae6640 1 -- 172.21.15.83:0/1986529599 >> 172.21.15.83:0/1986529599 conn(0x7f2d2806abe0 msgr2=0x7f2d2806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:00.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.938+0000 7f2d2fae6640 1 -- 172.21.15.83:0/1986529599 shutdown_connections 2024-04-17T10:42:00.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.938+0000 7f2d2fae6640 1 -- 172.21.15.83:0/1986529599 wait complete. 2024-04-17T10:42:00.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.938+0000 7f2d2fae6640 1 Processor -- start 2024-04-17T10:42:00.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.938+0000 7f2d2fae6640 1 -- start start 2024-04-17T10:42:00.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.938+0000 7f2d2fae6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d14003f70 0x7f2d28140450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.938+0000 7f2d2fae6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d280c45e0 0x7f2d280c4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.938+0000 7f2d2fae6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d280c61b0 0x7f2d280c6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.938+0000 7f2d2fae6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d28146e70 con 0x7f2d14003f70 2024-04-17T10:42:00.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.938+0000 7f2d2fae6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2806d200 con 0x7f2d280c61b0 2024-04-17T10:42:00.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.938+0000 7f2d2fae6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d28146ff0 con 0x7f2d280c45e0 2024-04-17T10:42:00.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.938+0000 7f2d2d882640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d14003f70 0x7f2d28140450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.938+0000 7f2d2d882640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d14003f70 0x7f2d28140450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41892/0 (socket says 172.21.15.83:41892) 2024-04-17T10:42:00.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.938+0000 7f2d2d882640 1 -- 172.21.15.83:0/2509835487 learned_addr learned my addr 172.21.15.83:0/2509835487 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:00.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.938+0000 7f2d2d882640 1 -- 172.21.15.83:0/2509835487 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d280c45e0 msgr2=0x7f2d280c4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:00.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.938+0000 7f2d2d882640 1 --2- 172.21.15.83:0/2509835487 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d280c45e0 0x7f2d280c4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.938+0000 7f2d2d882640 1 -- 172.21.15.83:0/2509835487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d280c61b0 msgr2=0x7f2d280c6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:00.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.938+0000 7f2d2d882640 1 --2- 172.21.15.83:0/2509835487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d280c61b0 0x7f2d280c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:00.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.938+0000 7f2d2d882640 1 -- 172.21.15.83:0/2509835487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d28140990 con 0x7f2d14003f70 2024-04-17T10:42:00.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.942+0000 7f2d2d882640 1 --2- 172.21.15.83:0/2509835487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d14003f70 0x7f2d28140450 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400c500 tx=0x7f2d2400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:00.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.942+0000 7f2d1e7fc640 1 -- 172.21.15.83:0/2509835487 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d240052d0 con 0x7f2d14003f70 2024-04-17T10:42:00.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.942+0000 7f2d1e7fc640 1 -- 172.21.15.83:0/2509835487 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d24005490 con 0x7f2d14003f70 2024-04-17T10:42:00.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.942+0000 7f2d2fae6640 1 -- 172.21.15.83:0/2509835487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d280c5150 con 0x7f2d14003f70 2024-04-17T10:42:00.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.942+0000 7f2d1e7fc640 1 -- 172.21.15.83:0/2509835487 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d24012850 con 0x7f2d14003f70 2024-04-17T10:42:00.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.942+0000 7f2d2fae6640 1 -- 172.21.15.83:0/2509835487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d280c5710 con 0x7f2d14003f70 2024-04-17T10:42:00.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.942+0000 7f2d1e7fc640 1 -- 172.21.15.83:0/2509835487 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2d240129f0 con 0x7f2d14003f70 2024-04-17T10:42:00.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.942+0000 7f2d2fae6640 1 -- 172.21.15.83:0/2509835487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d28074f90 con 0x7f2d14003f70 2024-04-17T10:42:00.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.942+0000 7f2d1e7fc640 1 --2- 172.21.15.83:0/2509835487 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2d0c066f60 0x7f2d0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:00.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.942+0000 7f2d2d081640 1 --2- 172.21.15.83:0/2509835487 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2d0c066f60 0x7f2d0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:00.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.942+0000 7f2d1e7fc640 1 -- 172.21.15.83:0/2509835487 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f2d24084d00 con 0x7f2d14003f70 2024-04-17T10:42:00.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.942+0000 7f2d2d081640 1 --2- 172.21.15.83:0/2509835487 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2d0c066f60 0x7f2d0c069420 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2d180af000 tx=0x7f2d18004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:00.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:00.946+0000 7f2d1e7fc640 1 -- 172.21.15.83:0/2509835487 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2d24055710 con 0x7f2d14003f70 2024-04-17T10:42:01.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.126+0000 7f2d2fae6640 1 -- 172.21.15.83:0/2509835487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d28070bd0 con 0x7f2d14003f70 2024-04-17T10:42:01.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.126+0000 7f2d1e7fc640 1 -- 172.21.15.83:0/2509835487 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f2d240597b0 con 0x7f2d14003f70 2024-04-17T10:42:01.124 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:01.125 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/161398126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":161398126},{"type":"v1","addr":"172.21.15.167:6833","nonce":161398126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1085380618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1085380618},{"type":"v1","addr":"172.21.15.167:6835","nonce":1085380618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/836442291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":836442291},{"type":"v1","addr":"172.21.15.83:6835","nonce":836442291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:41:37.773342+0000","modified":"2024-04-17T10:41:39.788394+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5682":{"gid":5682,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1894547386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1894547386},{"type":"v1","addr":"172.21.15.83:6837","nonce":1894547386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-17T10:42:01.125 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 21 2024-04-17T10:42:01.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.130+0000 7f2d2fae6640 1 -- 172.21.15.83:0/2509835487 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2d0c066f60 msgr2=0x7f2d0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:01.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.130+0000 7f2d2fae6640 1 --2- 172.21.15.83:0/2509835487 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2d0c066f60 0x7f2d0c069420 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2d180af000 tx=0x7f2d18004740 comp rx=0 tx=0).stop 2024-04-17T10:42:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.130+0000 7f2d2fae6640 1 -- 172.21.15.83:0/2509835487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d14003f70 msgr2=0x7f2d28140450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.130+0000 7f2d2fae6640 1 --2- 172.21.15.83:0/2509835487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d14003f70 0x7f2d28140450 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400c500 tx=0x7f2d2400c9c0 comp rx=0 tx=0).stop 2024-04-17T10:42:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.130+0000 7f2d2fae6640 1 -- 172.21.15.83:0/2509835487 shutdown_connections 2024-04-17T10:42:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.130+0000 7f2d2fae6640 1 --2- 172.21.15.83:0/2509835487 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2d0c066f60 0x7f2d0c069420 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.130+0000 7f2d2fae6640 1 --2- 172.21.15.83:0/2509835487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d280c61b0 0x7f2d280c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.130+0000 7f2d2fae6640 1 --2- 172.21.15.83:0/2509835487 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d280c45e0 0x7f2d280c4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.130+0000 7f2d2fae6640 1 --2- 172.21.15.83:0/2509835487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d14003f70 0x7f2d28140450 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.130+0000 7f2d2fae6640 1 -- 172.21.15.83:0/2509835487 >> 172.21.15.83:0/2509835487 conn(0x7f2d2806abe0 msgr2=0x7f2d280b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.130+0000 7f2d2fae6640 1 -- 172.21.15.83:0/2509835487 shutdown_connections 2024-04-17T10:42:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.130+0000 7f2d2fae6640 1 -- 172.21.15.83:0/2509835487 wait complete. 2024-04-17T10:42:01.143 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/ceph-qa-suite/new_dir_0 2024-04-17T10:42:01.273 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:42:01.274 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0 2024-04-17T10:42:01.274 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:42:01.292 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_0' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-17T10:42:01.292 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:42:01.292 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:42:01.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:42:01.297 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_0 bs=1M conv=fdatasync count=1 seek=0) 2024-04-17T10:42:01.362 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:42:01.362 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:42:01.362 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153118 s, 68.5 MB/s 2024-04-17T10:42:01.362 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_0' 2024-04-17T10:42:01.363 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:42:01.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.522+0000 7fc6c54f0640 1 Processor -- start 2024-04-17T10:42:01.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.522+0000 7fc6c54f0640 1 -- start start 2024-04-17T10:42:01.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.526+0000 7fc6c54f0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc6c00b0a70 0x7fc6c00d2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:01.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.526+0000 7fc6c54f0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6c0162150 0x7fc6c006c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:01.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.526+0000 7fc6c54f0640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c006e980 con 0x7fc6c00ce750 2024-04-17T10:42:01.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.526+0000 7fc6c54f0640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6c01625e0 con 0x7fc6c00b0a70 2024-04-17T10:42:01.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.526+0000 7fc6c54f0640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c006f290 con 0x7fc6c0162150 2024-04-17T10:42:01.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.526+0000 7fc6be7fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc6c00b0a70 0x7fc6c00d2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:01.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.526+0000 7fc6be7fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc6c00b0a70 0x7fc6c00d2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41902/0 (socket says 172.21.15.83:41902) 2024-04-17T10:42:01.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.526+0000 7fc6be7fc640 1 -- 172.21.15.83:0/948639861 learned_addr learned my addr 172.21.15.83:0/948639861 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:01.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.526+0000 7fc6be7fc640 1 -- 172.21.15.83:0/948639861 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6c0162150 msgr2=0x7fc6c006c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:01.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.526+0000 7fc6be7fc640 1 --2- 172.21.15.83:0/948639861 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6c0162150 0x7fc6c006c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:01.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.526+0000 7fc6be7fc640 1 -- 172.21.15.83:0/948639861 >> v1:172.21.15.167:6789/0 conn(0x7fc6c00ce750 legacy=0x7fc6c0161a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:01.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.526+0000 7fc6be7fc640 1 -- 172.21.15.83:0/948639861 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c0162810 con 0x7fc6c00b0a70 2024-04-17T10:42:01.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.526+0000 7fc6be7fc640 1 --2- 172.21.15.83:0/948639861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6c00b0a70 0x7fc6c00d2290 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fc6c006f410 tx=0x7fc6b401cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c040a24aa5e4946 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:01.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.526+0000 7fc6bdffb640 1 -- 172.21.15.83:0/948639861 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b40b1070 con 0x7fc6c00b0a70 2024-04-17T10:42:01.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.526+0000 7fc6bdffb640 1 -- 172.21.15.83:0/948639861 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b40ac630 con 0x7fc6c00b0a70 2024-04-17T10:42:01.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.526+0000 7fc6bdffb640 1 -- 172.21.15.83:0/948639861 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b40a8050 con 0x7fc6c00b0a70 2024-04-17T10:42:01.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.530+0000 7fc6c54f0640 1 -- 172.21.15.83:0/948639861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6c00b0a70 msgr2=0x7fc6c00d2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:01.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.530+0000 7fc6c54f0640 1 --2- 172.21.15.83:0/948639861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6c00b0a70 0x7fc6c00d2290 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fc6c006f410 tx=0x7fc6b401cf60 comp rx=0 tx=0).stop 2024-04-17T10:42:01.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.530+0000 7fc6c54f0640 1 -- 172.21.15.83:0/948639861 shutdown_connections 2024-04-17T10:42:01.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.530+0000 7fc6c54f0640 1 --2- 172.21.15.83:0/948639861 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6c0162150 0x7fc6c006c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:01.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.530+0000 7fc6c54f0640 1 --2- 172.21.15.83:0/948639861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6c00b0a70 0x7fc6c00d2290 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:01.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.530+0000 7fc6c54f0640 1 -- 172.21.15.83:0/948639861 >> 172.21.15.83:0/948639861 conn(0x7fc6c00d1770 msgr2=0x7fc6c00d1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:01.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.530+0000 7fc6c54f0640 1 -- 172.21.15.83:0/948639861 shutdown_connections 2024-04-17T10:42:01.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.530+0000 7fc6c54f0640 1 -- 172.21.15.83:0/948639861 wait complete. 2024-04-17T10:42:01.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.530+0000 7fc6c54f0640 1 Processor -- start 2024-04-17T10:42:01.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.530+0000 7fc6c54f0640 1 -- start start 2024-04-17T10:42:01.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.530+0000 7fc6c54f0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6c00b0a70 0x7fc6c0148b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:01.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.530+0000 7fc6c54f0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6c00ce750 0x7fc6c009b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:01.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.530+0000 7fc6c54f0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6c0162150 0x7fc6c00c62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:01.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.530+0000 7fc6c54f0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c006d5d0 con 0x7fc6c00b0a70 2024-04-17T10:42:01.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.530+0000 7fc6c54f0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c006d450 con 0x7fc6c00ce750 2024-04-17T10:42:01.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.530+0000 7fc6c54f0640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c006d750 con 0x7fc6c0162150 2024-04-17T10:42:01.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.530+0000 7fc6beffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6c00b0a70 0x7fc6c0148b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:01.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.530+0000 7fc6beffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6c00b0a70 0x7fc6c0148b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41904/0 (socket says 172.21.15.83:41904) 2024-04-17T10:42:01.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.530+0000 7fc6beffd640 1 -- 172.21.15.83:0/3282899657 learned_addr learned my addr 172.21.15.83:0/3282899657 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:01.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.530+0000 7fc6bf7fe640 1 --2- 172.21.15.83:0/3282899657 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6c0162150 0x7fc6c00c62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:01.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.530+0000 7fc6be7fc640 1 --2- 172.21.15.83:0/3282899657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6c00ce750 0x7fc6c009b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:01.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.530+0000 7fc6beffd640 1 -- 172.21.15.83:0/3282899657 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6c0162150 msgr2=0x7fc6c00c62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:01.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.530+0000 7fc6beffd640 1 --2- 172.21.15.83:0/3282899657 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6c0162150 0x7fc6c00c62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:01.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.530+0000 7fc6beffd640 1 -- 172.21.15.83:0/3282899657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6c00ce750 msgr2=0x7fc6c009b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:01.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.530+0000 7fc6beffd640 1 --2- 172.21.15.83:0/3282899657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6c00ce750 0x7fc6c009b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:01.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.530+0000 7fc6beffd640 1 -- 172.21.15.83:0/3282899657 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c009de30 con 0x7fc6c00b0a70 2024-04-17T10:42:01.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.530+0000 7fc6beffd640 1 --2- 172.21.15.83:0/3282899657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6c00b0a70 0x7fc6c0148b00 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fc6b000c450 tx=0x7fc6b000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:01.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.530+0000 7fc6a77fe640 1 -- 172.21.15.83:0/3282899657 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0012830 con 0x7fc6c00b0a70 2024-04-17T10:42:01.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.530+0000 7fc6a77fe640 1 -- 172.21.15.83:0/3282899657 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b0017070 con 0x7fc6c00b0a70 2024-04-17T10:42:01.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.530+0000 7fc6c54f0640 1 -- 172.21.15.83:0/3282899657 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c009e110 con 0x7fc6c00b0a70 2024-04-17T10:42:01.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.530+0000 7fc6a77fe640 1 -- 172.21.15.83:0/3282899657 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0012c30 con 0x7fc6c00b0a70 2024-04-17T10:42:01.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.530+0000 7fc6c54f0640 1 -- 172.21.15.83:0/3282899657 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c009e6d0 con 0x7fc6c00b0a70 2024-04-17T10:42:01.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.534+0000 7fc6c54f0640 1 -- 172.21.15.83:0/3282899657 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c0074f90 con 0x7fc6c00b0a70 2024-04-17T10:42:01.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.534+0000 7fc6a77fe640 1 -- 172.21.15.83:0/3282899657 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc6b001c460 con 0x7fc6c00b0a70 2024-04-17T10:42:01.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.534+0000 7fc6a77fe640 1 --2- 172.21.15.83:0/3282899657 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc6ac066f60 0x7fc6ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:01.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.534+0000 7fc6a77fe640 1 -- 172.21.15.83:0/3282899657 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc6b0084db0 con 0x7fc6c00b0a70 2024-04-17T10:42:01.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.534+0000 7fc6be7fc640 1 --2- 172.21.15.83:0/3282899657 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc6ac066f60 0x7fc6ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:01.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.534+0000 7fc6be7fc640 1 --2- 172.21.15.83:0/3282899657 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc6ac066f60 0x7fc6ac069420 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc6b40ae000 tx=0x7fc6b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:01.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.538+0000 7fc6a77fe640 1 -- 172.21.15.83:0/3282899657 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc6b0055e10 con 0x7fc6c00b0a70 2024-04-17T10:42:01.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.730+0000 7fc6c54f0640 1 -- 172.21.15.83:0/3282899657 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6c00b76f0 con 0x7fc6c00b0a70 2024-04-17T10:42:01.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.734+0000 7fc6a77fe640 1 -- 172.21.15.83:0/3282899657 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4938 (secure 0 0 0) 0x7fc6b0055710 con 0x7fc6c00b0a70 2024-04-17T10:42:01.732 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:01.732 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/161398126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":161398126},{"type":"v1","addr":"172.21.15.167:6833","nonce":161398126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1085380618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1085380618},{"type":"v1","addr":"172.21.15.167:6835","nonce":1085380618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/836442291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":836442291},{"type":"v1","addr":"172.21.15.83:6835","nonce":836442291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:41:37.773342+0000","modified":"2024-04-17T10:41:39.788394+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5682":{"gid":5682,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1894547386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1894547386},{"type":"v1","addr":"172.21.15.83:6837","nonce":1894547386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-17T10:42:01.732 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 21 2024-04-17T10:42:01.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.734+0000 7fc6c54f0640 1 -- 172.21.15.83:0/3282899657 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc6ac066f60 msgr2=0x7fc6ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:01.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.734+0000 7fc6c54f0640 1 --2- 172.21.15.83:0/3282899657 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc6ac066f60 0x7fc6ac069420 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc6b40ae000 tx=0x7fc6b4008040 comp rx=0 tx=0).stop 2024-04-17T10:42:01.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.734+0000 7fc6c54f0640 1 -- 172.21.15.83:0/3282899657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6c00b0a70 msgr2=0x7fc6c0148b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:01.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.734+0000 7fc6c54f0640 1 --2- 172.21.15.83:0/3282899657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6c00b0a70 0x7fc6c0148b00 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fc6b000c450 tx=0x7fc6b000c910 comp rx=0 tx=0).stop 2024-04-17T10:42:01.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.734+0000 7fc6c54f0640 1 -- 172.21.15.83:0/3282899657 shutdown_connections 2024-04-17T10:42:01.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.734+0000 7fc6c54f0640 1 --2- 172.21.15.83:0/3282899657 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc6ac066f60 0x7fc6ac069420 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:01.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.734+0000 7fc6c54f0640 1 --2- 172.21.15.83:0/3282899657 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6c0162150 0x7fc6c00c62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:01.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.734+0000 7fc6c54f0640 1 --2- 172.21.15.83:0/3282899657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6c00ce750 0x7fc6c009b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:01.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.734+0000 7fc6c54f0640 1 --2- 172.21.15.83:0/3282899657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6c00b0a70 0x7fc6c0148b00 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:01.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.734+0000 7fc6c54f0640 1 -- 172.21.15.83:0/3282899657 >> 172.21.15.83:0/3282899657 conn(0x7fc6c00d1770 msgr2=0x7fc6c00b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:01.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.734+0000 7fc6c54f0640 1 -- 172.21.15.83:0/3282899657 shutdown_connections 2024-04-17T10:42:01.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:01.734+0000 7fc6c54f0640 1 -- 172.21.15.83:0/3282899657 wait complete. 2024-04-17T10:42:01.750 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/ceph-qa-suite/new_file_0 2024-04-17T10:42:01.937 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:42:01.937 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0 2024-04-17T10:42:01.937 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:42:01.951 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_0' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-17T10:42:01.952 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-04-17T10:42:01.952 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-04-17T10:42:01.952 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-04-17T10:42:01.952 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:42:01.952 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_0").st_ino) 2024-04-17T10:42:01.952 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-04-17T10:42:02.012 INFO:teuthology.orchestra.run.smithi083.stdout:1099511630409 2024-04-17T10:42:02.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:42:02.021 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-17T10:42:02.021 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-17T10:42:02.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.190+0000 7f3a500b8640 1 Processor -- start 2024-04-17T10:42:02.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.190+0000 7f3a500b8640 1 -- start start 2024-04-17T10:42:02.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.194+0000 7f3a500b8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a48154670 0x7f3a480aea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.194+0000 7f3a500b8640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3a480b4090 0x7f3a4806c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.194+0000 7f3a500b8640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3a480b44f0 0x7f3a4806cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.194+0000 7f3a500b8640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a48154a50 con 0x7f3a480b44f0 2024-04-17T10:42:02.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.194+0000 7f3a500b8640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a480b2ff0 con 0x7f3a480b4090 2024-04-17T10:42:02.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.194+0000 7f3a500b8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a480b32d0 con 0x7f3a48154670 2024-04-17T10:42:02.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.194+0000 7f3a4e655640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3a480b44f0 0x7f3a4806cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.194+0000 7f3a4e655640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3a480b44f0 0x7f3a4806cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41918/0 (socket says 172.21.15.83:41918) 2024-04-17T10:42:02.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.194+0000 7f3a4e655640 1 -- 172.21.15.83:0/3078451806 learned_addr learned my addr 172.21.15.83:0/3078451806 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:02.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.194+0000 7f3a4d653640 1 --2- 172.21.15.83:0/3078451806 >> v2:172.21.15.167:3300/0 conn(0x7f3a480b4090 0x7f3a4806c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.194+0000 7f3a4e655640 1 -- 172.21.15.83:0/3078451806 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a48154670 msgr2=0x7f3a480aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:02.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.194+0000 7f3a4e655640 1 --2- 172.21.15.83:0/3078451806 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a48154670 0x7f3a480aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.194+0000 7f3a4e655640 1 -- 172.21.15.83:0/3078451806 >> v2:172.21.15.167:3300/0 conn(0x7f3a480b4090 msgr2=0x7f3a4806c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:02.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.194+0000 7f3a4e655640 1 --2- 172.21.15.83:0/3078451806 >> v2:172.21.15.167:3300/0 conn(0x7f3a480b4090 0x7f3a4806c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.194+0000 7f3a4e655640 1 -- 172.21.15.83:0/3078451806 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a480b35b0 con 0x7f3a480b44f0 2024-04-17T10:42:02.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.194+0000 7f3a4e655640 1 --2- 172.21.15.83:0/3078451806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a480b44f0 0x7f3a4806cab0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f3a4806b6a0 tx=0x7f3a4001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f7e0e74fbd764f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:02.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.194+0000 7f3a4ce52640 1 -- 172.21.15.83:0/3078451806 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a400a5840 con 0x7f3a480b44f0 2024-04-17T10:42:02.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.194+0000 7f3a4ce52640 1 -- 172.21.15.83:0/3078451806 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a400a59e0 con 0x7f3a480b44f0 2024-04-17T10:42:02.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.194+0000 7f3a4ce52640 1 -- 172.21.15.83:0/3078451806 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a400a5db0 con 0x7f3a480b44f0 2024-04-17T10:42:02.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.194+0000 7f3a500b8640 1 -- 172.21.15.83:0/3078451806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a480b44f0 msgr2=0x7f3a4806cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.194+0000 7f3a500b8640 1 --2- 172.21.15.83:0/3078451806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a480b44f0 0x7f3a4806cab0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f3a4806b6a0 tx=0x7f3a4001c9c0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.194+0000 7f3a500b8640 1 -- 172.21.15.83:0/3078451806 shutdown_connections 2024-04-17T10:42:02.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.194+0000 7f3a500b8640 1 --2- 172.21.15.83:0/3078451806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a480b44f0 0x7f3a4806cab0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.194+0000 7f3a500b8640 1 --2- 172.21.15.83:0/3078451806 >> v2:172.21.15.167:3300/0 conn(0x7f3a480b4090 0x7f3a4806c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.194+0000 7f3a500b8640 1 --2- 172.21.15.83:0/3078451806 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a48154670 0x7f3a480aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.194+0000 7f3a500b8640 1 -- 172.21.15.83:0/3078451806 >> 172.21.15.83:0/3078451806 conn(0x7f3a480b15b0 msgr2=0x7f3a480b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:02.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.198+0000 7f3a500b8640 1 -- 172.21.15.83:0/3078451806 shutdown_connections 2024-04-17T10:42:02.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.198+0000 7f3a500b8640 1 -- 172.21.15.83:0/3078451806 wait complete. 2024-04-17T10:42:02.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.198+0000 7f3a500b8640 1 Processor -- start 2024-04-17T10:42:02.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.198+0000 7f3a500b8640 1 -- start start 2024-04-17T10:42:02.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.198+0000 7f3a500b8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a480b4090 0x7f3a481f6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.198+0000 7f3a500b8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a480b44f0 0x7f3a4814d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.198+0000 7f3a500b8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a48154670 0x7f3a480ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.198+0000 7f3a500b8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a480af110 con 0x7f3a48154670 2024-04-17T10:42:02.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.198+0000 7f3a500b8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a480aef90 con 0x7f3a480b4090 2024-04-17T10:42:02.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.198+0000 7f3a500b8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4815db00 con 0x7f3a480b44f0 2024-04-17T10:42:02.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.198+0000 7f3a4e655640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a48154670 0x7f3a480ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.198+0000 7f3a4e655640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a48154670 0x7f3a480ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41924/0 (socket says 172.21.15.83:41924) 2024-04-17T10:42:02.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.198+0000 7f3a4e655640 1 -- 172.21.15.83:0/1004369419 learned_addr learned my addr 172.21.15.83:0/1004369419 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:02.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.198+0000 7f3a4e655640 1 -- 172.21.15.83:0/1004369419 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a480b44f0 msgr2=0x7f3a4814d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.198+0000 7f3a4de54640 1 --2- 172.21.15.83:0/1004369419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a480b4090 0x7f3a481f6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.198+0000 7f3a4e655640 1 --2- 172.21.15.83:0/1004369419 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a480b44f0 0x7f3a4814d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.198+0000 7f3a4e655640 1 -- 172.21.15.83:0/1004369419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a480b4090 msgr2=0x7f3a481f6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.198+0000 7f3a4e655640 1 --2- 172.21.15.83:0/1004369419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a480b4090 0x7f3a481f6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.198+0000 7f3a4e655640 1 -- 172.21.15.83:0/1004369419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a481f7030 con 0x7f3a48154670 2024-04-17T10:42:02.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.198+0000 7f3a4e655640 1 --2- 172.21.15.83:0/1004369419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a48154670 0x7f3a480ad840 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f3a400af040 tx=0x7f3a40019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:02.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.198+0000 7f3a3e7fc640 1 -- 172.21.15.83:0/1004369419 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a400b8670 con 0x7f3a48154670 2024-04-17T10:42:02.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.198+0000 7f3a3e7fc640 1 -- 172.21.15.83:0/1004369419 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a400be070 con 0x7f3a48154670 2024-04-17T10:42:02.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.198+0000 7f3a3e7fc640 1 -- 172.21.15.83:0/1004369419 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a400b8a70 con 0x7f3a48154670 2024-04-17T10:42:02.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.198+0000 7f3a500b8640 1 -- 172.21.15.83:0/1004369419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a481f7310 con 0x7f3a48154670 2024-04-17T10:42:02.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.198+0000 7f3a500b8640 1 -- 172.21.15.83:0/1004369419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a481f78d0 con 0x7f3a48154670 2024-04-17T10:42:02.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a3e7fc640 1 -- 172.21.15.83:0/1004369419 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3a400b8c10 con 0x7f3a48154670 2024-04-17T10:42:02.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a3e7fc640 1 --2- 172.21.15.83:0/1004369419 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3a2c066f60 0x7f3a2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a3e7fc640 1 -- 172.21.15.83:0/1004369419 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3a401203c0 con 0x7f3a48154670 2024-04-17T10:42:02.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a4de54640 1 --2- 172.21.15.83:0/1004369419 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3a2c066f60 0x7f3a2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a4de54640 1 --2- 172.21.15.83:0/1004369419 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3a2c066f60 0x7f3a2c069420 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400a790 tx=0x7f3a44008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:02.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a53225000 1 Processor -- start 2024-04-17T10:42:02.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a53225000 1 -- start start 2024-04-17T10:42:02.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a53225000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562c4e670180 0x562c4e670560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a53225000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c4e670aa0 0x562c4e6260c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a53225000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4e626600 0x562c4e6269e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a53225000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4e473f20 con 0x562c4e626600 2024-04-17T10:42:02.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a53225000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4e3fc000 con 0x562c4e670aa0 2024-04-17T10:42:02.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a53225000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x562c4e46ffc0 con 0x562c4e670180 2024-04-17T10:42:02.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a4e655640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4e626600 0x562c4e6269e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a4e655640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4e626600 0x562c4e6269e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41928/0 (socket says 172.21.15.83:41928) 2024-04-17T10:42:02.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a4e655640 1 -- 172.21.15.83:0/2011532192 learned_addr learned my addr 172.21.15.83:0/2011532192 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:02.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a4de54640 1 --2- 172.21.15.83:0/2011532192 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562c4e670180 0x562c4e670560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a4d653640 1 --2- 172.21.15.83:0/2011532192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c4e670aa0 0x562c4e6260c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a4e655640 1 -- 172.21.15.83:0/2011532192 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562c4e670180 msgr2=0x562c4e670560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:02.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a4e655640 1 --2- 172.21.15.83:0/2011532192 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562c4e670180 0x562c4e670560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a4e655640 1 -- 172.21.15.83:0/2011532192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c4e670aa0 msgr2=0x562c4e6260c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:02.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a4e655640 1 --2- 172.21.15.83:0/2011532192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c4e670aa0 0x562c4e6260c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a4e655640 1 -- 172.21.15.83:0/2011532192 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c4e44fba0 con 0x562c4e626600 2024-04-17T10:42:02.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a4e655640 1 --2- 172.21.15.83:0/2011532192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4e626600 0x562c4e6269e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3a400ed1a0 tx=0x7f3a400ed060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f25889ed5c38e9f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:02.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a2b7fe640 1 -- 172.21.15.83:0/2011532192 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a400f4de0 con 0x562c4e626600 2024-04-17T10:42:02.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a2b7fe640 1 -- 172.21.15.83:0/2011532192 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a400f93a0 con 0x562c4e626600 2024-04-17T10:42:02.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a2b7fe640 1 -- 172.21.15.83:0/2011532192 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a401200e0 con 0x562c4e626600 2024-04-17T10:42:02.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a53225000 1 -- 172.21.15.83:0/2011532192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4e626600 msgr2=0x562c4e6269e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a53225000 1 --2- 172.21.15.83:0/2011532192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4e626600 0x562c4e6269e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3a400ed1a0 tx=0x7f3a400ed060 comp rx=0 tx=0).stop 2024-04-17T10:42:02.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a53225000 1 -- 172.21.15.83:0/2011532192 shutdown_connections 2024-04-17T10:42:02.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a53225000 1 --2- 172.21.15.83:0/2011532192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4e626600 0x562c4e6269e0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a53225000 1 --2- 172.21.15.83:0/2011532192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c4e670aa0 0x562c4e6260c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a53225000 1 --2- 172.21.15.83:0/2011532192 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562c4e670180 0x562c4e670560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.202+0000 7f3a53225000 1 -- 172.21.15.83:0/2011532192 >> 172.21.15.83:0/2011532192 conn(0x562c4e62c760 msgr2=0x562c4e6249a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:02.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a53225000 1 -- 172.21.15.83:0/2011532192 shutdown_connections 2024-04-17T10:42:02.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a53225000 1 -- 172.21.15.83:0/2011532192 wait complete. 2024-04-17T10:42:02.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a53225000 1 -- start start 2024-04-17T10:42:02.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a53225000 1 Processor -- start 2024-04-17T10:42:02.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a53225000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562c4e626600 0x562c4e62b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a53225000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c4e670180 0x562c4e62bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a53225000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4e670aa0 0x562c4e65e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a53225000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4e5319c0 con 0x562c4e670aa0 2024-04-17T10:42:02.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a53225000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4e42e5d0 con 0x562c4e670180 2024-04-17T10:42:02.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a53225000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x562c4e473f20 con 0x562c4e626600 2024-04-17T10:42:02.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a53225000 10 client.0 _ll_register_callbacks cb 0x562c4e611a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:02.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a4e655640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4e670aa0 0x562c4e65e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a4e655640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4e670aa0 0x562c4e65e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41944/0 (socket says 172.21.15.83:41944) 2024-04-17T10:42:02.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a4e655640 1 -- 172.21.15.83:0/2227038371 learned_addr learned my addr 172.21.15.83:0/2227038371 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:02.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a4d653640 1 --2- 172.21.15.83:0/2227038371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c4e670180 0x562c4e62bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a4de54640 1 --2- 172.21.15.83:0/2227038371 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562c4e626600 0x562c4e62b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a4e655640 1 -- 172.21.15.83:0/2227038371 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562c4e626600 msgr2=0x562c4e62b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:02.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a4e655640 1 --2- 172.21.15.83:0/2227038371 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562c4e626600 0x562c4e62b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a4e655640 1 -- 172.21.15.83:0/2227038371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c4e670180 msgr2=0x562c4e62bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:02.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a4e655640 1 --2- 172.21.15.83:0/2227038371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c4e670180 0x562c4e62bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a4e655640 1 -- 172.21.15.83:0/2227038371 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c4e4ba5a0 con 0x562c4e670aa0 2024-04-17T10:42:02.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a4d653640 1 --2- 172.21.15.83:0/2227038371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c4e670180 0x562c4e62bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:02.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a4e655640 1 --2- 172.21.15.83:0/2227038371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4e670aa0 0x562c4e65e450 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f3a40098560 tx=0x7f3a40009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bdb93a66a0c60e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:02.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a28ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:02.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a28ff9640 1 -- 172.21.15.83:0/2227038371 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a40108040 con 0x562c4e670aa0 2024-04-17T10:42:02.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a28ff9640 1 -- 172.21.15.83:0/2227038371 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3a401200e0 con 0x562c4e670aa0 2024-04-17T10:42:02.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a53225000 1 -- 172.21.15.83:0/2227038371 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a400f93a0 con 0x562c4e670aa0 2024-04-17T10:42:02.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a28ff9640 1 -- 172.21.15.83:0/2227038371 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a40115050 con 0x562c4e670aa0 2024-04-17T10:42:02.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a53225000 10 client.5973 fetch_fsmap learned FSMap version 21 2024-04-17T10:42:02.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a53225000 1 -- 172.21.15.83:0/2227038371 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x562c4e44b2b0 con 0x562c4e670aa0 2024-04-17T10:42:02.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a28ff9640 1 -- 172.21.15.83:0/2227038371 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a400f93a0 con 0x562c4e670aa0 2024-04-17T10:42:02.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a28ff9640 1 -- 172.21.15.83:0/2227038371 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3a4011e450 con 0x562c4e670aa0 2024-04-17T10:42:02.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a53225000 10 client.5973 fetch_fsmap finished waiting for FSMap version 21 2024-04-17T10:42:02.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a53225000 10 client.5973 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:42:02.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a53225000 20 client.5973 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:02.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a53225000 1 --2- 172.21.15.83:0/2227038371 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562c4e65de60 0x562c4e65ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a53225000 4 client.5973 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:42:02.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a53225000 1 -- 172.21.15.83:0/2227038371 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c4e40fc70 con 0x562c4e65de60 2024-04-17T10:42:02.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a4de54640 1 --2- 172.21.15.83:0/2227038371 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562c4e65de60 0x562c4e65ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a28ff9640 10 client.5973 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:02.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a4de54640 1 --2- 172.21.15.83:0/2227038371 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562c4e65de60 0x562c4e65ee60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5b0c9a55c4d766f server_cookie=a333a9a59cc16eef in_seq=0 out_seq=0 2024-04-17T10:42:02.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a28ff9640 1 -- 172.21.15.83:0/2227038371 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562c4e40fc70 con 0x562c4e65de60 2024-04-17T10:42:02.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a28ff9640 10 client.5973 handle_command_reply: tid=0 2024-04-17T10:42:02.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a53225000 1 client.5973 shutdown 2024-04-17T10:42:02.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a53225000 1 -- 172.21.15.83:0/2227038371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4e670aa0 msgr2=0x562c4e65e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a53225000 1 --2- 172.21.15.83:0/2227038371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4e670aa0 0x562c4e65e450 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f3a40098560 tx=0x7f3a40009810 comp rx=0 tx=0).stop 2024-04-17T10:42:02.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a53225000 1 -- 172.21.15.83:0/2227038371 shutdown_connections 2024-04-17T10:42:02.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a53225000 1 --2- 172.21.15.83:0/2227038371 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562c4e65de60 0x562c4e65ee60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a53225000 1 --2- 172.21.15.83:0/2227038371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4e670aa0 0x562c4e65e450 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a28ff9640 0 client.5973 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:02.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a53225000 1 --2- 172.21.15.83:0/2227038371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c4e670180 0x562c4e62bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a53225000 1 --2- 172.21.15.83:0/2227038371 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562c4e626600 0x562c4e62b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.206+0000 7f3a53225000 1 -- 172.21.15.83:0/2227038371 >> 172.21.15.83:0/2227038371 conn(0x562c4e62c760 msgr2=0x562c4e6250b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:02.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.210+0000 7f3a53225000 1 -- 172.21.15.83:0/2227038371 shutdown_connections 2024-04-17T10:42:02.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.210+0000 7f3a53225000 1 -- 172.21.15.83:0/2227038371 wait complete. 2024-04-17T10:42:02.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.210+0000 7f3a53225000 20 client.5973 trim_cache size 0 max 16384 2024-04-17T10:42:02.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.218+0000 7f3a53225000 1 Processor -- start 2024-04-17T10:42:02.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.218+0000 7f3a53225000 1 -- start start 2024-04-17T10:42:02.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a53225000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4e626600 0x562c4e66f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a53225000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c4e670180 0x562c4e66fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a53225000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562c4e670aa0 0x562c4e625120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a53225000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a48202de0 con 0x562c4e626600 2024-04-17T10:42:02.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a53225000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a48153430 con 0x562c4e670180 2024-04-17T10:42:02.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a53225000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a480b0010 con 0x562c4e670aa0 2024-04-17T10:42:02.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a4de54640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4e626600 0x562c4e66f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a4de54640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4e626600 0x562c4e66f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41948/0 (socket says 172.21.15.83:41948) 2024-04-17T10:42:02.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a4de54640 1 -- 172.21.15.83:0/1129742000 learned_addr learned my addr 172.21.15.83:0/1129742000 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:02.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a4de54640 1 -- 172.21.15.83:0/1129742000 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562c4e670aa0 msgr2=0x562c4e625120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:02.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a4de54640 1 --2- 172.21.15.83:0/1129742000 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562c4e670aa0 0x562c4e625120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a4de54640 1 -- 172.21.15.83:0/1129742000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c4e670180 msgr2=0x562c4e66fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:02.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a4de54640 1 --2- 172.21.15.83:0/1129742000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c4e670180 0x562c4e66fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a4de54640 1 -- 172.21.15.83:0/1129742000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c4e3fc250 con 0x562c4e626600 2024-04-17T10:42:02.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a4de54640 1 --2- 172.21.15.83:0/1129742000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4e626600 0x562c4e66f5f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400aeb0 tx=0x7f3a44013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9022b72e6dc81e55 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:02.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a2affd640 1 -- 172.21.15.83:0/1129742000 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4400ce70 con 0x562c4e626600 2024-04-17T10:42:02.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a2affd640 1 -- 172.21.15.83:0/1129742000 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a44028030 con 0x562c4e626600 2024-04-17T10:42:02.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a2affd640 1 -- 172.21.15.83:0/1129742000 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4401fac0 con 0x562c4e626600 2024-04-17T10:42:02.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a53225000 1 -- 172.21.15.83:0/1129742000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4e626600 msgr2=0x562c4e66f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a53225000 1 --2- 172.21.15.83:0/1129742000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4e626600 0x562c4e66f5f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400aeb0 tx=0x7f3a44013000 comp rx=0 tx=0).stop 2024-04-17T10:42:02.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a53225000 1 -- 172.21.15.83:0/1129742000 shutdown_connections 2024-04-17T10:42:02.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a53225000 1 --2- 172.21.15.83:0/1129742000 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562c4e670aa0 0x562c4e625120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a53225000 1 --2- 172.21.15.83:0/1129742000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c4e670180 0x562c4e66fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a53225000 1 --2- 172.21.15.83:0/1129742000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4e626600 0x562c4e66f5f0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a53225000 1 -- 172.21.15.83:0/1129742000 >> 172.21.15.83:0/1129742000 conn(0x562c4e62c760 msgr2=0x562c4e66eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:02.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a53225000 1 -- 172.21.15.83:0/1129742000 shutdown_connections 2024-04-17T10:42:02.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a53225000 1 -- 172.21.15.83:0/1129742000 wait complete. 2024-04-17T10:42:02.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a53225000 1 -- start start 2024-04-17T10:42:02.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a53225000 1 Processor -- start 2024-04-17T10:42:02.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a53225000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562c4e626600 0x562c4e6d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a53225000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c4e670180 0x562c4e6d0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a53225000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4e670aa0 0x562c4e6849b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a53225000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4e62f6f0 con 0x562c4e670aa0 2024-04-17T10:42:02.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a53225000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a48152e30 con 0x562c4e670180 2024-04-17T10:42:02.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a53225000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x562c4e473f20 con 0x562c4e626600 2024-04-17T10:42:02.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a53225000 10 client.0 _ll_register_callbacks cb 0x562c4e5f2600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:02.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a4e655640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4e670aa0 0x562c4e6849b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a4e655640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4e670aa0 0x562c4e6849b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41950/0 (socket says 172.21.15.83:41950) 2024-04-17T10:42:02.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.222+0000 7f3a4e655640 1 -- 172.21.15.83:0/2901697101 learned_addr learned my addr 172.21.15.83:0/2901697101 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:02.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a4e655640 1 -- 172.21.15.83:0/2901697101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562c4e626600 msgr2=0x562c4e6d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:02.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a4e655640 1 --2- 172.21.15.83:0/2901697101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562c4e626600 0x562c4e6d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a4e655640 1 -- 172.21.15.83:0/2901697101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c4e670180 msgr2=0x562c4e6d0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:02.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a4d653640 1 --2- 172.21.15.83:0/2901697101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c4e670180 0x562c4e6d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a4e655640 1 --2- 172.21.15.83:0/2901697101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c4e670180 0x562c4e6d0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a4e655640 1 -- 172.21.15.83:0/2901697101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c4e408b90 con 0x562c4e670aa0 2024-04-17T10:42:02.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a4e655640 1 --2- 172.21.15.83:0/2901697101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4e670aa0 0x562c4e6849b0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f3a4001cea0 tx=0x7f3a4011e780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44dc805abd6cb6e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:02.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a28ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:02.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a28ff9640 1 -- 172.21.15.83:0/2901697101 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a40108040 con 0x562c4e670aa0 2024-04-17T10:42:02.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a28ff9640 1 -- 172.21.15.83:0/2901697101 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3a400f8f10 con 0x562c4e670aa0 2024-04-17T10:42:02.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a53225000 1 -- 172.21.15.83:0/2901697101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a44028030 con 0x562c4e670aa0 2024-04-17T10:42:02.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a28ff9640 1 -- 172.21.15.83:0/2901697101 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a400ecfe0 con 0x562c4e670aa0 2024-04-17T10:42:02.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a53225000 10 client.5979 fetch_fsmap learned FSMap version 21 2024-04-17T10:42:02.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a53225000 1 -- 172.21.15.83:0/2901697101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x562c4e41b010 con 0x562c4e670aa0 2024-04-17T10:42:02.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a28ff9640 1 -- 172.21.15.83:0/2901697101 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a44028030 con 0x562c4e670aa0 2024-04-17T10:42:02.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a28ff9640 1 -- 172.21.15.83:0/2901697101 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3a400f0190 con 0x562c4e670aa0 2024-04-17T10:42:02.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a53225000 10 client.5979 fetch_fsmap finished waiting for FSMap version 21 2024-04-17T10:42:02.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a53225000 10 client.5979 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:42:02.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a53225000 20 client.5979 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:02.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a53225000 1 --2- 172.21.15.83:0/2901697101 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562c4e685310 0x562c4e6856f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a53225000 4 client.5979 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T10:42:02.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a53225000 1 -- 172.21.15.83:0/2901697101 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562c4e65e7e0 con 0x562c4e685310 2024-04-17T10:42:02.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a4de54640 1 --2- 172.21.15.83:0/2901697101 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562c4e685310 0x562c4e6856f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a4de54640 1 --2- 172.21.15.83:0/2901697101 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562c4e685310 0x562c4e6856f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a229bf083cd503d7 server_cookie=60442229312e85d7 in_seq=0 out_seq=0 2024-04-17T10:42:02.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a28ff9640 10 client.5979 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:02.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a28ff9640 1 -- 172.21.15.83:0/2901697101 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562c4e65e7e0 con 0x562c4e685310 2024-04-17T10:42:02.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a28ff9640 10 client.5979 handle_command_reply: tid=0 2024-04-17T10:42:02.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a53225000 1 client.5979 shutdown 2024-04-17T10:42:02.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a53225000 1 -- 172.21.15.83:0/2901697101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4e670aa0 msgr2=0x562c4e6849b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a53225000 1 --2- 172.21.15.83:0/2901697101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4e670aa0 0x562c4e6849b0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f3a4001cea0 tx=0x7f3a4011e780 comp rx=0 tx=0).stop 2024-04-17T10:42:02.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a53225000 1 -- 172.21.15.83:0/2901697101 shutdown_connections 2024-04-17T10:42:02.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a53225000 1 --2- 172.21.15.83:0/2901697101 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562c4e685310 0x562c4e6856f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a53225000 1 --2- 172.21.15.83:0/2901697101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4e670aa0 0x562c4e6849b0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a28ff9640 0 client.5979 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:02.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a53225000 1 --2- 172.21.15.83:0/2901697101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c4e670180 0x562c4e6d0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a53225000 1 --2- 172.21.15.83:0/2901697101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562c4e626600 0x562c4e6d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a53225000 1 -- 172.21.15.83:0/2901697101 >> 172.21.15.83:0/2901697101 conn(0x562c4e62c760 msgr2=0x562c4e66f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:02.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a53225000 1 -- 172.21.15.83:0/2901697101 shutdown_connections 2024-04-17T10:42:02.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a53225000 1 -- 172.21.15.83:0/2901697101 wait complete. 2024-04-17T10:42:02.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a53225000 20 client.5979 trim_cache size 0 max 16384 2024-04-17T10:42:02.233 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-17T10:42:02.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a500b8640 1 -- 172.21.15.83:0/1004369419 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3a2c066f60 msgr2=0x7f3a2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a500b8640 1 --2- 172.21.15.83:0/1004369419 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3a2c066f60 0x7f3a2c069420 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400a790 tx=0x7f3a44008040 comp rx=0 tx=0).stop 2024-04-17T10:42:02.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a500b8640 1 -- 172.21.15.83:0/1004369419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a48154670 msgr2=0x7f3a480ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a500b8640 1 --2- 172.21.15.83:0/1004369419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a48154670 0x7f3a480ad840 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f3a400af040 tx=0x7f3a40019760 comp rx=0 tx=0).stop 2024-04-17T10:42:02.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a500b8640 1 -- 172.21.15.83:0/1004369419 shutdown_connections 2024-04-17T10:42:02.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a500b8640 1 --2- 172.21.15.83:0/1004369419 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3a2c066f60 0x7f3a2c069420 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a500b8640 1 --2- 172.21.15.83:0/1004369419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a48154670 0x7f3a480ad840 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a500b8640 1 --2- 172.21.15.83:0/1004369419 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a480b44f0 0x7f3a4814d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a500b8640 1 --2- 172.21.15.83:0/1004369419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a480b4090 0x7f3a481f6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a500b8640 1 -- 172.21.15.83:0/1004369419 >> 172.21.15.83:0/1004369419 conn(0x7f3a480b15b0 msgr2=0x7f3a48071a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:02.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a500b8640 1 -- 172.21.15.83:0/1004369419 shutdown_connections 2024-04-17T10:42:02.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.226+0000 7f3a500b8640 1 -- 172.21.15.83:0/1004369419 wait complete. 2024-04-17T10:42:02.240 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data rmxattr 10000000a49.00000000 parent 2024-04-17T10:42:02.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa6078d8080 1 Processor -- start 2024-04-17T10:42:02.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa6078d8080 1 -- start start 2024-04-17T10:42:02.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa6078d8080 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556bb40d66b0 0x556bb41a1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa6078d8080 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bb411c630 0x556bb41a0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa6078d8080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bb41a8520 0x556bb41a2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa6078d8080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bb40799f0 con 0x556bb41a8520 2024-04-17T10:42:02.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa6078d8080 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bb407d950 con 0x556bb411c630 2024-04-17T10:42:02.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa6078d8080 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x556bb41abf30 con 0x556bb40d66b0 2024-04-17T10:42:02.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa605bf7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bb41a8520 0x556bb41a2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa605bf7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bb41a8520 0x556bb41a2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41964/0 (socket says 172.21.15.83:41964) 2024-04-17T10:42:02.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa605bf7640 1 -- 172.21.15.83:0/4025772139 learned_addr learned my addr 172.21.15.83:0/4025772139 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:02.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa604bf5640 1 --2- 172.21.15.83:0/4025772139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bb411c630 0x556bb41a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa6053f6640 1 --2- 172.21.15.83:0/4025772139 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556bb40d66b0 0x556bb41a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa605bf7640 1 -- 172.21.15.83:0/4025772139 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556bb40d66b0 msgr2=0x556bb41a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:02.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa605bf7640 1 --2- 172.21.15.83:0/4025772139 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556bb40d66b0 0x556bb41a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa605bf7640 1 -- 172.21.15.83:0/4025772139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bb411c630 msgr2=0x556bb41a0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:02.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa605bf7640 1 --2- 172.21.15.83:0/4025772139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bb411c630 0x556bb41a0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa605bf7640 1 -- 172.21.15.83:0/4025772139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556bb41a0cb0 con 0x556bb41a8520 2024-04-17T10:42:02.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa604bf5640 1 --2- 172.21.15.83:0/4025772139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bb411c630 0x556bb41a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:02.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa605bf7640 1 --2- 172.21.15.83:0/4025772139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bb41a8520 0x556bb41a2080 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa6000026c0 tx=0x7fa600002b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dead86b92a29608 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:02.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa5f77fe640 1 -- 172.21.15.83:0/4025772139 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa600016070 con 0x556bb41a8520 2024-04-17T10:42:02.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa5f77fe640 1 -- 172.21.15.83:0/4025772139 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa60000f730 con 0x556bb41a8520 2024-04-17T10:42:02.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa5f77fe640 1 -- 172.21.15.83:0/4025772139 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60000fb20 con 0x556bb41a8520 2024-04-17T10:42:02.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa6078d8080 1 -- 172.21.15.83:0/4025772139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bb41a8520 msgr2=0x556bb41a2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa6078d8080 1 --2- 172.21.15.83:0/4025772139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bb41a8520 0x556bb41a2080 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa6000026c0 tx=0x7fa600002b80 comp rx=0 tx=0).stop 2024-04-17T10:42:02.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa6078d8080 1 -- 172.21.15.83:0/4025772139 shutdown_connections 2024-04-17T10:42:02.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa6078d8080 1 --2- 172.21.15.83:0/4025772139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bb41a8520 0x556bb41a2080 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa6078d8080 1 --2- 172.21.15.83:0/4025772139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bb411c630 0x556bb41a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa6078d8080 1 --2- 172.21.15.83:0/4025772139 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556bb40d66b0 0x556bb41a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa6078d8080 1 -- 172.21.15.83:0/4025772139 >> 172.21.15.83:0/4025772139 conn(0x556bb40ff1f0 msgr2=0x556bb40d5ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:02.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa6078d8080 1 -- 172.21.15.83:0/4025772139 shutdown_connections 2024-04-17T10:42:02.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa6078d8080 1 -- 172.21.15.83:0/4025772139 wait complete. 2024-04-17T10:42:02.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa6078d8080 1 Processor -- start 2024-04-17T10:42:02.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.274+0000 7fa6078d8080 1 -- start start 2024-04-17T10:42:02.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.278+0000 7fa6078d8080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bb40d66b0 0x556bb41170e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.278+0000 7fa6078d8080 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bb411c630 0x556bb419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.278+0000 7fa6078d8080 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556bb41a8520 0x556bb419f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.278+0000 7fa6078d8080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bb41a1e70 con 0x556bb40d66b0 2024-04-17T10:42:02.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.278+0000 7fa6078d8080 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bb41a8b10 con 0x556bb411c630 2024-04-17T10:42:02.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.278+0000 7fa6078d8080 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x556bb41a0f50 con 0x556bb41a8520 2024-04-17T10:42:02.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.278+0000 7fa6053f6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bb40d66b0 0x556bb41170e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.278+0000 7fa6053f6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bb40d66b0 0x556bb41170e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41966/0 (socket says 172.21.15.83:41966) 2024-04-17T10:42:02.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.278+0000 7fa6053f6640 1 -- 172.21.15.83:0/2422448565 learned_addr learned my addr 172.21.15.83:0/2422448565 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:02.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.278+0000 7fa604bf5640 1 --2- 172.21.15.83:0/2422448565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bb411c630 0x556bb419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.278+0000 7fa605bf7640 1 --2- 172.21.15.83:0/2422448565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556bb41a8520 0x556bb419f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.278+0000 7fa6053f6640 1 -- 172.21.15.83:0/2422448565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556bb41a8520 msgr2=0x556bb419f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.278+0000 7fa6053f6640 1 --2- 172.21.15.83:0/2422448565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556bb41a8520 0x556bb419f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.278+0000 7fa6053f6640 1 -- 172.21.15.83:0/2422448565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bb411c630 msgr2=0x556bb419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.278+0000 7fa6053f6640 1 --2- 172.21.15.83:0/2422448565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bb411c630 0x556bb419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.278+0000 7fa6053f6640 1 -- 172.21.15.83:0/2422448565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556bb411a1e0 con 0x556bb40d66b0 2024-04-17T10:42:02.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.278+0000 7fa6053f6640 1 --2- 172.21.15.83:0/2422448565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bb40d66b0 0x556bb41170e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800ce90 tx=0x7fa5f800a5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:02.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.278+0000 7fa5f67fc640 1 -- 172.21.15.83:0/2422448565 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f800f410 con 0x556bb40d66b0 2024-04-17T10:42:02.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.278+0000 7fa5f67fc640 1 -- 172.21.15.83:0/2422448565 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f8014020 con 0x556bb40d66b0 2024-04-17T10:42:02.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.278+0000 7fa5f67fc640 1 -- 172.21.15.83:0/2422448565 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f800f7f0 con 0x556bb40d66b0 2024-04-17T10:42:02.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.278+0000 7fa6078d8080 1 -- 172.21.15.83:0/2422448565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556bb411a480 con 0x556bb40d66b0 2024-04-17T10:42:02.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.278+0000 7fa6078d8080 1 -- 172.21.15.83:0/2422448565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556bb411a9c0 con 0x556bb40d66b0 2024-04-17T10:42:02.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.278+0000 7fa5f67fc640 1 -- 172.21.15.83:0/2422448565 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa5f800f990 con 0x556bb40d66b0 2024-04-17T10:42:02.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.278+0000 7fa5f67fc640 1 --2- 172.21.15.83:0/2422448565 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa5e8065940 0x7fa5e8067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.278+0000 7fa5f67fc640 1 -- 172.21.15.83:0/2422448565 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa5f8083dc0 con 0x556bb40d66b0 2024-04-17T10:42:02.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.278+0000 7fa604bf5640 1 --2- 172.21.15.83:0/2422448565 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa5e8065940 0x7fa5e8067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.282+0000 7fa6078d8080 1 --2- 172.21.15.83:0/2422448565 >> [v2:172.21.15.167:6812/1109787903,v1:172.21.15.167:6815/1109787903] conn(0x556bb4117620 0x556bb4114c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.282+0000 7fa6078d8080 1 -- 172.21.15.83:0/2422448565 --> [v2:172.21.15.167:6812/1109787903,v1:172.21.15.167:6815/1109787903] -- osd_op(unknown.0.0:1 7.1f 7:f95eeb57:::10000000a49.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48) v9 -- 0x556bb4119d80 con 0x556bb4117620 2024-04-17T10:42:02.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.282+0000 7fa605bf7640 1 --2- 172.21.15.83:0/2422448565 >> [v2:172.21.15.167:6812/1109787903,v1:172.21.15.167:6815/1109787903] conn(0x556bb4117620 0x556bb4114c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.282+0000 7fa604bf5640 1 --2- 172.21.15.83:0/2422448565 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa5e8065940 0x7fa5e8067e00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc0ac000 tx=0x7fa5fc0a5000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:02.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.282+0000 7fa605bf7640 1 --2- 172.21.15.83:0/2422448565 >> [v2:172.21.15.167:6812/1109787903,v1:172.21.15.167:6815/1109787903] conn(0x556bb4117620 0x556bb4114c70 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:02.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.282+0000 7fa605bf7640 1 -- 172.21.15.83:0/2422448565 <== osd.6 v2:172.21.15.167:6812/1109787903 1 ==== osd_op_reply(1 10000000a49.00000000 [rmxattr] v48'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa60000f070 con 0x556bb4117620 2024-04-17T10:42:02.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.282+0000 7fa6078d8080 1 -- 172.21.15.83:0/2422448565 >> [v2:172.21.15.167:6812/1109787903,v1:172.21.15.167:6815/1109787903] conn(0x556bb4117620 msgr2=0x556bb4114c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.282+0000 7fa6078d8080 1 --2- 172.21.15.83:0/2422448565 >> [v2:172.21.15.167:6812/1109787903,v1:172.21.15.167:6815/1109787903] conn(0x556bb4117620 0x556bb4114c70 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.282+0000 7fa6078d8080 1 -- 172.21.15.83:0/2422448565 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa5e8065940 msgr2=0x7fa5e8067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.282+0000 7fa6078d8080 1 --2- 172.21.15.83:0/2422448565 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa5e8065940 0x7fa5e8067e00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc0ac000 tx=0x7fa5fc0a5000 comp rx=0 tx=0).stop 2024-04-17T10:42:02.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.282+0000 7fa6078d8080 1 -- 172.21.15.83:0/2422448565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bb40d66b0 msgr2=0x556bb41170e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.282+0000 7fa6078d8080 1 --2- 172.21.15.83:0/2422448565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bb40d66b0 0x556bb41170e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800ce90 tx=0x7fa5f800a5e0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.282+0000 7fa605bf7640 1 -- 172.21.15.83:0/2422448565 reap_dead start 2024-04-17T10:42:02.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.286+0000 7fa6078d8080 1 -- 172.21.15.83:0/2422448565 shutdown_connections 2024-04-17T10:42:02.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.286+0000 7fa6078d8080 1 -- 172.21.15.83:0/2422448565 >> 172.21.15.83:0/2422448565 conn(0x556bb40ff1f0 msgr2=0x556bb40d5ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:02.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.286+0000 7fa6078d8080 1 -- 172.21.15.83:0/2422448565 shutdown_connections 2024-04-17T10:42:02.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.286+0000 7fa6078d8080 1 -- 172.21.15.83:0/2422448565 wait complete. 2024-04-17T10:42:02.286 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_0' 2024-04-17T10:42:02.286 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-17T10:42:02.287 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/new_file_0 2024-04-17T10:42:02.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.446+0000 7f945c8e3640 1 Processor -- start 2024-04-17T10:42:02.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.446+0000 7f945c8e3640 1 -- start start 2024-04-17T10:42:02.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.446+0000 7f945c8e3640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f94580d0bf0 0x7f94580b6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.446+0000 7f945c8e3640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f94581618c0 0x7f9458161d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.446+0000 7f945c8e3640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f945815e1b0 con 0x7f945815ddd0 2024-04-17T10:42:02.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.446+0000 7f945c8e3640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94580d1dc0 con 0x7f94581618c0 2024-04-17T10:42:02.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.446+0000 7f945c8e3640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94580d2660 con 0x7f94580d0bf0 2024-04-17T10:42:02.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.446+0000 7f945659c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f945815ddd0 0x7f94580d04e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60740/0 (socket says 172.21.15.83:60740) 2024-04-17T10:42:02.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.446+0000 7f9456d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f94581618c0 0x7f9458161d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.446+0000 7f945659c640 1 -- 172.21.15.83:0/2096657664 learned_addr learned my addr 172.21.15.83:0/2096657664 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:02.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.446+0000 7f944ffff640 1 --2- 172.21.15.83:0/2096657664 >> v2:172.21.15.167:3300/0 conn(0x7f94580d0bf0 0x7f94580b6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.446+0000 7f9456d9d640 1 -- 172.21.15.83:0/2096657664 >> v2:172.21.15.167:3300/0 conn(0x7f94580d0bf0 msgr2=0x7f94580b6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:02.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.446+0000 7f9456d9d640 1 --2- 172.21.15.83:0/2096657664 >> v2:172.21.15.167:3300/0 conn(0x7f94580d0bf0 0x7f94580b6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.446+0000 7f9456d9d640 1 -- 172.21.15.83:0/2096657664 >> v1:172.21.15.83:6789/0 conn(0x7f945815ddd0 legacy=0x7f94580d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.446+0000 7f9456d9d640 1 -- 172.21.15.83:0/2096657664 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94581625f0 con 0x7f94581618c0 2024-04-17T10:42:02.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.446+0000 7f9455d9b640 1 -- 172.21.15.83:0/2096657664 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 565479259 0 0) 0x7f945815e1b0 con 0x7f945815ddd0 2024-04-17T10:42:02.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.446+0000 7f9456d9d640 1 --2- 172.21.15.83:0/2096657664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94581618c0 0x7f9458161d10 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f94580ce0c0 tx=0x7f945001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76df934c45027bcb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:02.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f9455d9b640 1 -- 172.21.15.83:0/2096657664 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94500b2070 con 0x7f94581618c0 2024-04-17T10:42:02.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f9455d9b640 1 -- 172.21.15.83:0/2096657664 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94500a59c0 con 0x7f94581618c0 2024-04-17T10:42:02.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f9455d9b640 1 -- 172.21.15.83:0/2096657664 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94500a5d90 con 0x7f94581618c0 2024-04-17T10:42:02.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f945c8e3640 1 -- 172.21.15.83:0/2096657664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94581618c0 msgr2=0x7f9458161d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f945c8e3640 1 --2- 172.21.15.83:0/2096657664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94581618c0 0x7f9458161d10 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f94580ce0c0 tx=0x7f945001c900 comp rx=0 tx=0).stop 2024-04-17T10:42:02.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f945c8e3640 1 -- 172.21.15.83:0/2096657664 shutdown_connections 2024-04-17T10:42:02.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f945c8e3640 1 --2- 172.21.15.83:0/2096657664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94581618c0 0x7f9458161d10 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f945c8e3640 1 --2- 172.21.15.83:0/2096657664 >> v2:172.21.15.167:3300/0 conn(0x7f94580d0bf0 0x7f94580b6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f945c8e3640 1 -- 172.21.15.83:0/2096657664 >> 172.21.15.83:0/2096657664 conn(0x7f945806b0b0 msgr2=0x7f945806b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:02.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f945c8e3640 1 -- 172.21.15.83:0/2096657664 shutdown_connections 2024-04-17T10:42:02.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f945c8e3640 1 -- 172.21.15.83:0/2096657664 wait complete. 2024-04-17T10:42:02.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f945c8e3640 1 Processor -- start 2024-04-17T10:42:02.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f945c8e3640 1 -- start start 2024-04-17T10:42:02.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f945c8e3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94580d0bf0 0x7f94580c7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f945c8e3640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f945815ddd0 0x7f9458148ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f945c8e3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94581618c0 0x7f94580b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f945c8e3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94580d3480 con 0x7f94580d0bf0 2024-04-17T10:42:02.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f945c8e3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94580d3300 con 0x7f94581618c0 2024-04-17T10:42:02.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f945c8e3640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94580d3600 con 0x7f945815ddd0 2024-04-17T10:42:02.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f945659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94580d0bf0 0x7f94580c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f945659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94580d0bf0 0x7f94580c7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41974/0 (socket says 172.21.15.83:41974) 2024-04-17T10:42:02.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f945659c640 1 -- 172.21.15.83:0/3162362426 learned_addr learned my addr 172.21.15.83:0/3162362426 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:02.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f9456d9d640 1 --2- 172.21.15.83:0/3162362426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94581618c0 0x7f94580b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f944ffff640 1 --2- 172.21.15.83:0/3162362426 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f945815ddd0 0x7f9458148ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f945659c640 1 -- 172.21.15.83:0/3162362426 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f945815ddd0 msgr2=0x7f9458148ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f945659c640 1 --2- 172.21.15.83:0/3162362426 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f945815ddd0 0x7f9458148ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f945659c640 1 -- 172.21.15.83:0/3162362426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94581618c0 msgr2=0x7f94580b7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f945659c640 1 --2- 172.21.15.83:0/3162362426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94581618c0 0x7f94580b7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f945659c640 1 -- 172.21.15.83:0/3162362426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94580cb790 con 0x7f94580d0bf0 2024-04-17T10:42:02.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f944ffff640 1 --2- 172.21.15.83:0/3162362426 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f945815ddd0 0x7f9458148ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:02.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f945659c640 1 --2- 172.21.15.83:0/3162362426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94580d0bf0 0x7f94580c7640 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f944800c520 tx=0x7f944800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:02.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f944effd640 1 -- 172.21.15.83:0/3162362426 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9448018070 con 0x7f94580d0bf0 2024-04-17T10:42:02.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f944effd640 1 -- 172.21.15.83:0/3162362426 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94480074e0 con 0x7f94580d0bf0 2024-04-17T10:42:02.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f944effd640 1 -- 172.21.15.83:0/3162362426 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94480049f0 con 0x7f94580d0bf0 2024-04-17T10:42:02.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f945c8e3640 1 -- 172.21.15.83:0/3162362426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94580cba70 con 0x7f94580d0bf0 2024-04-17T10:42:02.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.450+0000 7f945c8e3640 1 -- 172.21.15.83:0/3162362426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94580c7b80 con 0x7f94580d0bf0 2024-04-17T10:42:02.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.454+0000 7f944effd640 1 -- 172.21.15.83:0/3162362426 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f944801b070 con 0x7f94580d0bf0 2024-04-17T10:42:02.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.454+0000 7f944effd640 1 --2- 172.21.15.83:0/3162362426 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9438066f60 0x7f9438069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.454+0000 7f944ffff640 1 --2- 172.21.15.83:0/3162362426 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9438066f60 0x7f9438069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.454+0000 7f944effd640 1 -- 172.21.15.83:0/3162362426 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9448084540 con 0x7f94580d0bf0 2024-04-17T10:42:02.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.454+0000 7f944ffff640 1 --2- 172.21.15.83:0/3162362426 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9438066f60 0x7f9438069420 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f945815e6a0 tx=0x7f9440008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:02.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.454+0000 7f945fa50000 1 Processor -- start 2024-04-17T10:42:02.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.454+0000 7f945fa50000 1 -- start start 2024-04-17T10:42:02.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.454+0000 7f945fa50000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555d9b8cd1f0 0x555d9b8cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.454+0000 7f945fa50000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9b8cdb10 0x555d9b8803e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.454+0000 7f945fa50000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555d9b880920 0x555d9b880d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.454+0000 7f945fa50000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d9b6d0ce0 con 0x555d9b8cdb10 2024-04-17T10:42:02.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.454+0000 7f945fa50000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d9b659530 con 0x555d9b8cd1f0 2024-04-17T10:42:02.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.454+0000 7f945fa50000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x555d9b6ccd80 con 0x555d9b880920 2024-04-17T10:42:02.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.454+0000 7f944ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9b8cdb10 0x555d9b8803e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.454+0000 7f944ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9b8cdb10 0x555d9b8803e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41976/0 (socket says 172.21.15.83:41976) 2024-04-17T10:42:02.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.454+0000 7f944ffff640 1 -- 172.21.15.83:0/28221020 learned_addr learned my addr 172.21.15.83:0/28221020 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:02.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.454+0000 7f945659c640 1 --2- 172.21.15.83:0/28221020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555d9b8cd1f0 0x555d9b8cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.454+0000 7f9456d9d640 1 --2- 172.21.15.83:0/28221020 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555d9b880920 0x555d9b880d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.454+0000 7f944ffff640 1 -- 172.21.15.83:0/28221020 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555d9b880920 msgr2=0x555d9b880d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:02.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.454+0000 7f944ffff640 1 --2- 172.21.15.83:0/28221020 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555d9b880920 0x555d9b880d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f944ffff640 1 -- 172.21.15.83:0/28221020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555d9b8cd1f0 msgr2=0x555d9b8cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:02.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f944ffff640 1 --2- 172.21.15.83:0/28221020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555d9b8cd1f0 0x555d9b8cd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f944ffff640 1 -- 172.21.15.83:0/28221020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d9b6cbcd0 con 0x555d9b8cdb10 2024-04-17T10:42:02.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f944ffff640 1 --2- 172.21.15.83:0/28221020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9b8cdb10 0x555d9b8803e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f9440004350 tx=0x7f9440011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158d97e9afaaff67 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:02.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f9437fff640 1 -- 172.21.15.83:0/28221020 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944000f040 con 0x555d9b8cdb10 2024-04-17T10:42:02.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f9437fff640 1 -- 172.21.15.83:0/28221020 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9440004380 con 0x555d9b8cdb10 2024-04-17T10:42:02.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f9437fff640 1 -- 172.21.15.83:0/28221020 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944000ee80 con 0x555d9b8cdb10 2024-04-17T10:42:02.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945fa50000 1 -- 172.21.15.83:0/28221020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9b8cdb10 msgr2=0x555d9b8803e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945fa50000 1 --2- 172.21.15.83:0/28221020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9b8cdb10 0x555d9b8803e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f9440004350 tx=0x7f9440011000 comp rx=0 tx=0).stop 2024-04-17T10:42:02.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945fa50000 1 -- 172.21.15.83:0/28221020 shutdown_connections 2024-04-17T10:42:02.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945fa50000 1 --2- 172.21.15.83:0/28221020 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555d9b880920 0x555d9b880d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945fa50000 1 --2- 172.21.15.83:0/28221020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9b8cdb10 0x555d9b8803e0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945fa50000 1 --2- 172.21.15.83:0/28221020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555d9b8cd1f0 0x555d9b8cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945fa50000 1 -- 172.21.15.83:0/28221020 >> 172.21.15.83:0/28221020 conn(0x555d9b8adf90 msgr2=0x555d9b8b0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:02.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945fa50000 1 -- 172.21.15.83:0/28221020 shutdown_connections 2024-04-17T10:42:02.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945fa50000 1 -- 172.21.15.83:0/28221020 wait complete. 2024-04-17T10:42:02.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945fa50000 1 -- start start 2024-04-17T10:42:02.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945fa50000 1 Processor -- start 2024-04-17T10:42:02.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945fa50000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9b880920 0x555d9b885ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945fa50000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555d9b8cd1f0 0x555d9b8861e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945fa50000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555d9b8cdb10 0x555d9b886720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945fa50000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d9b68bbc0 con 0x555d9b880920 2024-04-17T10:42:02.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945fa50000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d9b88c130 con 0x555d9b8cd1f0 2024-04-17T10:42:02.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945fa50000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x555d9b78a400 con 0x555d9b8cdb10 2024-04-17T10:42:02.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945fa50000 10 client.0 _ll_register_callbacks cb 0x555d9b86e4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:02.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9b880920 0x555d9b885ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9b880920 0x555d9b885ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41984/0 (socket says 172.21.15.83:41984) 2024-04-17T10:42:02.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945659c640 1 -- 172.21.15.83:0/2382256403 learned_addr learned my addr 172.21.15.83:0/2382256403 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:02.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f9456d9d640 1 --2- 172.21.15.83:0/2382256403 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555d9b8cdb10 0x555d9b886720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945659c640 1 -- 172.21.15.83:0/2382256403 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555d9b8cdb10 msgr2=0x555d9b886720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:02.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945659c640 1 --2- 172.21.15.83:0/2382256403 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555d9b8cdb10 0x555d9b886720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945659c640 1 -- 172.21.15.83:0/2382256403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555d9b8cd1f0 msgr2=0x555d9b8861e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:02.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945659c640 1 --2- 172.21.15.83:0/2382256403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555d9b8cd1f0 0x555d9b8861e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945659c640 1 -- 172.21.15.83:0/2382256403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d9b6de950 con 0x555d9b880920 2024-04-17T10:42:02.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945659c640 1 --2- 172.21.15.83:0/2382256403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9b880920 0x555d9b885ca0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9448000f70 tx=0x7f9448054e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f42cf44f96203ba server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:02.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f94357fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:02.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f94357fa640 1 -- 172.21.15.83:0/2382256403 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944805d510 con 0x555d9b880920 2024-04-17T10:42:02.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945fa50000 1 -- 172.21.15.83:0/2382256403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9440004380 con 0x555d9b880920 2024-04-17T10:42:02.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f94357fa640 1 -- 172.21.15.83:0/2382256403 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f944805b0c0 con 0x555d9b880920 2024-04-17T10:42:02.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f94357fa640 1 -- 172.21.15.83:0/2382256403 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94480840e0 con 0x555d9b880920 2024-04-17T10:42:02.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945fa50000 10 client.6006 fetch_fsmap learned FSMap version 21 2024-04-17T10:42:02.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945fa50000 1 -- 172.21.15.83:0/2382256403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x555d9b74d790 con 0x555d9b880920 2024-04-17T10:42:02.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f94357fa640 1 -- 172.21.15.83:0/2382256403 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9440004380 con 0x555d9b880920 2024-04-17T10:42:02.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f94357fa640 1 -- 172.21.15.83:0/2382256403 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f944806cb90 con 0x555d9b880920 2024-04-17T10:42:02.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945fa50000 10 client.6006 fetch_fsmap finished waiting for FSMap version 21 2024-04-17T10:42:02.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945fa50000 10 client.6006 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:42:02.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945fa50000 20 client.6006 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:02.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945fa50000 1 --2- 172.21.15.83:0/2382256403 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x555d9b887130 0x555d9b8de290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945fa50000 4 client.6006 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:42:02.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.458+0000 7f945fa50000 1 -- 172.21.15.83:0/2382256403 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d9b66cb80 con 0x555d9b887130 2024-04-17T10:42:02.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.462+0000 7f944ffff640 1 --2- 172.21.15.83:0/2382256403 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x555d9b887130 0x555d9b8de290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.462+0000 7f94357fa640 10 client.6006 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:02.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.462+0000 7f944ffff640 1 --2- 172.21.15.83:0/2382256403 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x555d9b887130 0x555d9b8de290 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f340000a5229 server_cookie=a68bede33f30a4a4 in_seq=0 out_seq=0 2024-04-17T10:42:02.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.462+0000 7f94357fa640 1 -- 172.21.15.83:0/2382256403 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555d9b66cb80 con 0x555d9b887130 2024-04-17T10:42:02.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.462+0000 7f94357fa640 10 client.6006 handle_command_reply: tid=0 2024-04-17T10:42:02.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.462+0000 7f945fa50000 1 client.6006 shutdown 2024-04-17T10:42:02.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.462+0000 7f945fa50000 1 -- 172.21.15.83:0/2382256403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9b880920 msgr2=0x555d9b885ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.462+0000 7f945fa50000 1 --2- 172.21.15.83:0/2382256403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9b880920 0x555d9b885ca0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9448000f70 tx=0x7f9448054e00 comp rx=0 tx=0).stop 2024-04-17T10:42:02.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.462+0000 7f945fa50000 1 -- 172.21.15.83:0/2382256403 shutdown_connections 2024-04-17T10:42:02.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.462+0000 7f945fa50000 1 --2- 172.21.15.83:0/2382256403 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x555d9b887130 0x555d9b8de290 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.462+0000 7f945fa50000 1 --2- 172.21.15.83:0/2382256403 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555d9b8cdb10 0x555d9b886720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.462+0000 7f94357fa640 0 client.6006 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:02.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.462+0000 7f945fa50000 1 --2- 172.21.15.83:0/2382256403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555d9b8cd1f0 0x555d9b8861e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.462+0000 7f945fa50000 1 --2- 172.21.15.83:0/2382256403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9b880920 0x555d9b885ca0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.462+0000 7f945fa50000 1 -- 172.21.15.83:0/2382256403 >> 172.21.15.83:0/2382256403 conn(0x555d9b8adf90 msgr2=0x555d9b8b0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:02.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.462+0000 7f945fa50000 1 -- 172.21.15.83:0/2382256403 shutdown_connections 2024-04-17T10:42:02.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.462+0000 7f945fa50000 1 -- 172.21.15.83:0/2382256403 wait complete. 2024-04-17T10:42:02.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.462+0000 7f945fa50000 20 client.6006 trim_cache size 0 max 16384 2024-04-17T10:42:02.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f945fa50000 1 Processor -- start 2024-04-17T10:42:02.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f945fa50000 1 -- start start 2024-04-17T10:42:02.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f945fa50000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555d9b880920 0x555d9b8cc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f945fa50000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9b8cd1f0 0x555d9b8cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f945fa50000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555d9b8cdb10 0x555d9b8de290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f945fa50000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94582029a0 con 0x555d9b8cd1f0 2024-04-17T10:42:02.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f945fa50000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9458201b90 con 0x555d9b880920 2024-04-17T10:42:02.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f945fa50000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94580cd860 con 0x555d9b8cdb10 2024-04-17T10:42:02.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f944ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9b8cd1f0 0x555d9b8cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f944ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9b8cd1f0 0x555d9b8cc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41988/0 (socket says 172.21.15.83:41988) 2024-04-17T10:42:02.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f944ffff640 1 -- 172.21.15.83:0/3448678538 learned_addr learned my addr 172.21.15.83:0/3448678538 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:02.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f945659c640 1 --2- 172.21.15.83:0/3448678538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555d9b880920 0x555d9b8cc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f9456d9d640 1 --2- 172.21.15.83:0/3448678538 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555d9b8cdb10 0x555d9b8de290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f944ffff640 1 -- 172.21.15.83:0/3448678538 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555d9b8cdb10 msgr2=0x555d9b8de290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:02.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f944ffff640 1 --2- 172.21.15.83:0/3448678538 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555d9b8cdb10 0x555d9b8de290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f944ffff640 1 -- 172.21.15.83:0/3448678538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555d9b880920 msgr2=0x555d9b8cc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:02.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f944ffff640 1 --2- 172.21.15.83:0/3448678538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555d9b880920 0x555d9b8cc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f944ffff640 1 -- 172.21.15.83:0/3448678538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d9b6966f0 con 0x555d9b8cd1f0 2024-04-17T10:42:02.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f945659c640 1 --2- 172.21.15.83:0/3448678538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555d9b880920 0x555d9b8cc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:02.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f944ffff640 1 --2- 172.21.15.83:0/3448678538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9b8cd1f0 0x555d9b8cc7f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9440008010 tx=0x7f9440013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c2e64b182f1a705 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:02.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f94377fe640 1 -- 172.21.15.83:0/3448678538 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9440028040 con 0x555d9b8cd1f0 2024-04-17T10:42:02.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f94377fe640 1 -- 172.21.15.83:0/3448678538 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9440004380 con 0x555d9b8cd1f0 2024-04-17T10:42:02.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f94377fe640 1 -- 172.21.15.83:0/3448678538 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9440003b20 con 0x555d9b8cd1f0 2024-04-17T10:42:02.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f945fa50000 1 -- 172.21.15.83:0/3448678538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9b8cd1f0 msgr2=0x555d9b8cc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f945fa50000 1 --2- 172.21.15.83:0/3448678538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9b8cd1f0 0x555d9b8cc7f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9440008010 tx=0x7f9440013000 comp rx=0 tx=0).stop 2024-04-17T10:42:02.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f945fa50000 1 -- 172.21.15.83:0/3448678538 shutdown_connections 2024-04-17T10:42:02.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f945fa50000 1 --2- 172.21.15.83:0/3448678538 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555d9b8cdb10 0x555d9b8de290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f945fa50000 1 --2- 172.21.15.83:0/3448678538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9b8cd1f0 0x555d9b8cc7f0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f945fa50000 1 --2- 172.21.15.83:0/3448678538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555d9b880920 0x555d9b8cc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f945fa50000 1 -- 172.21.15.83:0/3448678538 >> 172.21.15.83:0/3448678538 conn(0x555d9b8adf90 msgr2=0x555d9b87f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:02.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f945fa50000 1 -- 172.21.15.83:0/3448678538 shutdown_connections 2024-04-17T10:42:02.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f945fa50000 1 -- 172.21.15.83:0/3448678538 wait complete. 2024-04-17T10:42:02.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f945fa50000 1 -- start start 2024-04-17T10:42:02.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.478+0000 7f945fa50000 1 Processor -- start 2024-04-17T10:42:02.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945fa50000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555d9b880920 0x555d9b8e0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945fa50000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555d9b8cd1f0 0x555d9b8e8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945fa50000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9b8cdb10 0x555d9b8e92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945fa50000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d9b6d5310 con 0x555d9b8cdb10 2024-04-17T10:42:02.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945fa50000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9458201590 con 0x555d9b8cd1f0 2024-04-17T10:42:02.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945fa50000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x555d9b88c130 con 0x555d9b880920 2024-04-17T10:42:02.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945fa50000 10 client.0 _ll_register_callbacks cb 0x7f9440009630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:02.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f9456d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9b8cdb10 0x555d9b8e92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f9456d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9b8cdb10 0x555d9b8e92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41994/0 (socket says 172.21.15.83:41994) 2024-04-17T10:42:02.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f9456d9d640 1 -- 172.21.15.83:0/2880581954 learned_addr learned my addr 172.21.15.83:0/2880581954 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:02.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945659c640 1 --2- 172.21.15.83:0/2880581954 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555d9b880920 0x555d9b8e0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f944ffff640 1 --2- 172.21.15.83:0/2880581954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555d9b8cd1f0 0x555d9b8e8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f9456d9d640 1 -- 172.21.15.83:0/2880581954 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555d9b880920 msgr2=0x555d9b8e0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:02.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f9456d9d640 1 --2- 172.21.15.83:0/2880581954 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555d9b880920 0x555d9b8e0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f9456d9d640 1 -- 172.21.15.83:0/2880581954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555d9b8cd1f0 msgr2=0x555d9b8e8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:02.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f9456d9d640 1 --2- 172.21.15.83:0/2880581954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555d9b8cd1f0 0x555d9b8e8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f9456d9d640 1 -- 172.21.15.83:0/2880581954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d9b68cc80 con 0x555d9b8cdb10 2024-04-17T10:42:02.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f944ffff640 1 --2- 172.21.15.83:0/2880581954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555d9b8cd1f0 0x555d9b8e8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:02.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f9456d9d640 1 --2- 172.21.15.83:0/2880581954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9b8cdb10 0x555d9b8e92a0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9450019510 tx=0x7f945001b140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c515cddb210e81 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:02.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f94357fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:02.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f94357fa640 1 -- 172.21.15.83:0/2880581954 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94500b2040 con 0x555d9b8cdb10 2024-04-17T10:42:02.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f94357fa640 1 -- 172.21.15.83:0/2880581954 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f94500c4b20 con 0x555d9b8cdb10 2024-04-17T10:42:02.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945fa50000 1 -- 172.21.15.83:0/2880581954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9440004380 con 0x555d9b8cdb10 2024-04-17T10:42:02.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f94357fa640 1 -- 172.21.15.83:0/2880581954 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9450009a30 con 0x555d9b8cdb10 2024-04-17T10:42:02.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945fa50000 10 client.6012 fetch_fsmap learned FSMap version 21 2024-04-17T10:42:02.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945fa50000 1 -- 172.21.15.83:0/2880581954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x555d9b38d600 con 0x555d9b8cdb10 2024-04-17T10:42:02.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f94357fa640 1 -- 172.21.15.83:0/2880581954 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9440004380 con 0x555d9b8cdb10 2024-04-17T10:42:02.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f94357fa640 1 -- 172.21.15.83:0/2880581954 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f94500cb050 con 0x555d9b8cdb10 2024-04-17T10:42:02.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945fa50000 10 client.6012 fetch_fsmap finished waiting for FSMap version 21 2024-04-17T10:42:02.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945fa50000 10 client.6012 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:42:02.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945fa50000 20 client.6012 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:02.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945fa50000 1 --2- 172.21.15.83:0/2880581954 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x555d9b92d000 0x555d9b92d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945fa50000 4 client.6012 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_0"}] 2024-04-17T10:42:02.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945fa50000 1 -- 172.21.15.83:0/2880581954 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_0"}) v1 -- 0x555d9b6de220 con 0x555d9b92d000 2024-04-17T10:42:02.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945659c640 1 --2- 172.21.15.83:0/2880581954 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x555d9b92d000 0x555d9b92d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f94357fa640 10 client.6012 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:02.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945659c640 1 --2- 172.21.15.83:0/2880581954 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x555d9b92d000 0x555d9b92d410 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aebab5a70cefb7b1 server_cookie=9e6ccc2cf7f07a0e in_seq=0 out_seq=0 2024-04-17T10:42:02.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f94357fa640 1 -- 172.21.15.83:0/2880581954 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555d9b6de220 con 0x555d9b92d000 2024-04-17T10:42:02.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f94357fa640 10 client.6012 handle_command_reply: tid=0 2024-04-17T10:42:02.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945fa50000 1 client.6012 shutdown 2024-04-17T10:42:02.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945fa50000 1 -- 172.21.15.83:0/2880581954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9b8cdb10 msgr2=0x555d9b8e92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945fa50000 1 --2- 172.21.15.83:0/2880581954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9b8cdb10 0x555d9b8e92a0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9450019510 tx=0x7f945001b140 comp rx=0 tx=0).stop 2024-04-17T10:42:02.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945fa50000 1 -- 172.21.15.83:0/2880581954 shutdown_connections 2024-04-17T10:42:02.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945fa50000 1 --2- 172.21.15.83:0/2880581954 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x555d9b92d000 0x555d9b92d410 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945fa50000 1 --2- 172.21.15.83:0/2880581954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d9b8cdb10 0x555d9b8e92a0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f94357fa640 0 client.6012 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:02.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945fa50000 1 --2- 172.21.15.83:0/2880581954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555d9b8cd1f0 0x555d9b8e8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945fa50000 1 --2- 172.21.15.83:0/2880581954 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555d9b880920 0x555d9b8e0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945fa50000 1 -- 172.21.15.83:0/2880581954 >> 172.21.15.83:0/2880581954 conn(0x555d9b8adf90 msgr2=0x555d9b87f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:02.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945fa50000 1 -- 172.21.15.83:0/2880581954 shutdown_connections 2024-04-17T10:42:02.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945fa50000 1 -- 172.21.15.83:0/2880581954 wait complete. 2024-04-17T10:42:02.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945fa50000 20 client.6012 trim_cache size 0 max 16384 2024-04-17T10:42:02.489 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:42:02.489 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-17T10:42:02.489 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "ca55178c-5ad9-4324-923e-065575311828", 2024-04-17T10:42:02.489 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-17T10:42:02.489 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:42:02.489 INFO:tasks.ceph.mds.c.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f894694d640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a49 [2,head] /scrub_test_path/ceph-qa-suite/new_file_0 auth v561 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5844=0-4194304@1} caps={5844=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5844 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55890254c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000a49:[<0x10000000001/new_file_0 v561>,<0x10000000000/ceph-qa-suite v71>,<0x1/scrub_test_path v32>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2024-04-17T10:42:02.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945c8e3640 1 -- 172.21.15.83:0/3162362426 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9438066f60 msgr2=0x7f9438069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945c8e3640 1 --2- 172.21.15.83:0/3162362426 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9438066f60 0x7f9438069420 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f945815e6a0 tx=0x7f9440008040 comp rx=0 tx=0).stop 2024-04-17T10:42:02.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945c8e3640 1 -- 172.21.15.83:0/3162362426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94580d0bf0 msgr2=0x7f94580c7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945c8e3640 1 --2- 172.21.15.83:0/3162362426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94580d0bf0 0x7f94580c7640 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f944800c520 tx=0x7f944800c9e0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945c8e3640 1 -- 172.21.15.83:0/3162362426 shutdown_connections 2024-04-17T10:42:02.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945c8e3640 1 --2- 172.21.15.83:0/3162362426 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9438066f60 0x7f9438069420 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945c8e3640 1 --2- 172.21.15.83:0/3162362426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94581618c0 0x7f94580b7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945c8e3640 1 --2- 172.21.15.83:0/3162362426 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f945815ddd0 0x7f9458148ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945c8e3640 1 --2- 172.21.15.83:0/3162362426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94580d0bf0 0x7f94580c7640 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945c8e3640 1 -- 172.21.15.83:0/3162362426 >> 172.21.15.83:0/3162362426 conn(0x7f945806b0b0 msgr2=0x7f94580ce240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:02.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945c8e3640 1 -- 172.21.15.83:0/3162362426 shutdown_connections 2024-04-17T10:42:02.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.482+0000 7f945c8e3640 1 -- 172.21.15.83:0/3162362426 wait complete. 2024-04-17T10:42:02.497 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_0' returned '{'return_code': 0, 'scrub_tag': 'ca55178c-5ad9-4324-923e-065575311828', 'mode': 'asynchronous'}' 2024-04-17T10:42:02.497 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-17T10:42:02.497 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-17T10:42:02.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.606+0000 7febab2ed640 1 Processor -- start 2024-04-17T10:42:02.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.606+0000 7febab2ed640 1 -- start start 2024-04-17T10:42:02.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.606+0000 7febab2ed640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7feba406ccd0 0x7feba40cc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.606+0000 7febab2ed640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feba40b09e0 0x7feba40cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.606+0000 7febab2ed640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feba40d0480 con 0x7feba415de10 2024-04-17T10:42:02.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.606+0000 7febab2ed640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feba406c2e0 con 0x7feba40b09e0 2024-04-17T10:42:02.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.606+0000 7febab2ed640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feba40d0740 con 0x7feba406ccd0 2024-04-17T10:42:02.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.606+0000 7feba988a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feba40b09e0 0x7feba40cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.606+0000 7feba988a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feba40b09e0 0x7feba40cc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42010/0 (socket says 172.21.15.83:42010) 2024-04-17T10:42:02.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.606+0000 7feba988a640 1 -- 172.21.15.83:0/1940808936 learned_addr learned my addr 172.21.15.83:0/1940808936 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:02.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.606+0000 7feba9089640 1 --1- 172.21.15.83:0/1940808936 >> v1:172.21.15.83:6789/0 conn(0x7feba415de10 0x7feba406c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60752/0 (socket says 172.21.15.83:60752) 2024-04-17T10:42:02.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.606+0000 7feba8888640 1 --2- 172.21.15.83:0/1940808936 >> v2:172.21.15.167:3300/0 conn(0x7feba406ccd0 0x7feba40cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.606+0000 7feba988a640 1 -- 172.21.15.83:0/1940808936 >> v2:172.21.15.167:3300/0 conn(0x7feba406ccd0 msgr2=0x7feba40cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:02.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.606+0000 7feba988a640 1 --2- 172.21.15.83:0/1940808936 >> v2:172.21.15.167:3300/0 conn(0x7feba406ccd0 0x7feba40cc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.606+0000 7feba988a640 1 -- 172.21.15.83:0/1940808936 >> v1:172.21.15.83:6789/0 conn(0x7feba415de10 legacy=0x7feba406c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.606+0000 7feb9bfff640 1 -- 172.21.15.83:0/1940808936 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4218845101 0 0) 0x7feba40d0480 con 0x7feba415de10 2024-04-17T10:42:02.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.606+0000 7feba988a640 1 -- 172.21.15.83:0/1940808936 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba40d0a20 con 0x7feba40b09e0 2024-04-17T10:42:02.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.606+0000 7feba8888640 1 --2- 172.21.15.83:0/1940808936 >> v2:172.21.15.167:3300/0 conn(0x7feba406ccd0 0x7feba40cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:02.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.610+0000 7feba988a640 1 --2- 172.21.15.83:0/1940808936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feba40b09e0 0x7feba40cc5a0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7feba4071930 tx=0x7feb9c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19f00f81578e941 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:02.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.610+0000 7feb9bfff640 1 -- 172.21.15.83:0/1940808936 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c0a5700 con 0x7feba40b09e0 2024-04-17T10:42:02.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.610+0000 7feb9bfff640 1 -- 172.21.15.83:0/1940808936 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9c0a58a0 con 0x7feba40b09e0 2024-04-17T10:42:02.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.610+0000 7feb9bfff640 1 -- 172.21.15.83:0/1940808936 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c0a5c70 con 0x7feba40b09e0 2024-04-17T10:42:02.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.610+0000 7febab2ed640 1 -- 172.21.15.83:0/1940808936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feba40b09e0 msgr2=0x7feba40cc5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.610+0000 7febab2ed640 1 --2- 172.21.15.83:0/1940808936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feba40b09e0 0x7feba40cc5a0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7feba4071930 tx=0x7feb9c01c930 comp rx=0 tx=0).stop 2024-04-17T10:42:02.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.610+0000 7febab2ed640 1 -- 172.21.15.83:0/1940808936 shutdown_connections 2024-04-17T10:42:02.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.610+0000 7febab2ed640 1 --2- 172.21.15.83:0/1940808936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feba40b09e0 0x7feba40cc5a0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.610+0000 7febab2ed640 1 --2- 172.21.15.83:0/1940808936 >> v2:172.21.15.167:3300/0 conn(0x7feba406ccd0 0x7feba40cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.610+0000 7febab2ed640 1 -- 172.21.15.83:0/1940808936 >> 172.21.15.83:0/1940808936 conn(0x7feba40d1730 msgr2=0x7feba40d1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:02.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.610+0000 7febab2ed640 1 -- 172.21.15.83:0/1940808936 shutdown_connections 2024-04-17T10:42:02.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.610+0000 7febab2ed640 1 -- 172.21.15.83:0/1940808936 wait complete. 2024-04-17T10:42:02.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.610+0000 7febab2ed640 1 Processor -- start 2024-04-17T10:42:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.610+0000 7febab2ed640 1 -- start start 2024-04-17T10:42:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.610+0000 7febab2ed640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feba406ccd0 0x7feba40c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.610+0000 7febab2ed640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feba40b09e0 0x7feba406f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.610+0000 7febab2ed640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feba415de10 0x7feba40b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.610+0000 7febab2ed640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba416ab60 con 0x7feba40b09e0 2024-04-17T10:42:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.610+0000 7febab2ed640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba416a9e0 con 0x7feba415de10 2024-04-17T10:42:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.610+0000 7febab2ed640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba416ace0 con 0x7feba406ccd0 2024-04-17T10:42:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.610+0000 7feba8888640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feba40b09e0 0x7feba406f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.610+0000 7feba8888640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feba40b09e0 0x7feba406f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42022/0 (socket says 172.21.15.83:42022) 2024-04-17T10:42:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.610+0000 7feba8888640 1 -- 172.21.15.83:0/4019332817 learned_addr learned my addr 172.21.15.83:0/4019332817 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.610+0000 7feba9089640 1 --2- 172.21.15.83:0/4019332817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feba406ccd0 0x7feba40c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.610+0000 7feba988a640 1 --2- 172.21.15.83:0/4019332817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feba415de10 0x7feba40b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.610+0000 7feba8888640 1 -- 172.21.15.83:0/4019332817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feba406ccd0 msgr2=0x7feba40c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.610+0000 7feba8888640 1 --2- 172.21.15.83:0/4019332817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feba406ccd0 0x7feba40c75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.610+0000 7feba8888640 1 -- 172.21.15.83:0/4019332817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feba415de10 msgr2=0x7feba40b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.610+0000 7feba8888640 1 --2- 172.21.15.83:0/4019332817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feba415de10 0x7feba40b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.610+0000 7feba8888640 1 -- 172.21.15.83:0/4019332817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba40c7ff0 con 0x7feba40b09e0 2024-04-17T10:42:02.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.610+0000 7feba988a640 1 --2- 172.21.15.83:0/4019332817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feba415de10 0x7feba40b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:02.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.614+0000 7feba8888640 1 --2- 172.21.15.83:0/4019332817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feba40b09e0 0x7feba406f400 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7feba000c820 tx=0x7feba000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:02.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.614+0000 7feb99ffb640 1 -- 172.21.15.83:0/4019332817 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba0015070 con 0x7feba40b09e0 2024-04-17T10:42:02.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.614+0000 7feb99ffb640 1 -- 172.21.15.83:0/4019332817 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba0007a30 con 0x7feba40b09e0 2024-04-17T10:42:02.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.614+0000 7febab2ed640 1 -- 172.21.15.83:0/4019332817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feba40c6540 con 0x7feba40b09e0 2024-04-17T10:42:02.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.614+0000 7feb99ffb640 1 -- 172.21.15.83:0/4019332817 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba0007e40 con 0x7feba40b09e0 2024-04-17T10:42:02.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.614+0000 7febab2ed640 1 -- 172.21.15.83:0/4019332817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba40c6de0 con 0x7feba40b09e0 2024-04-17T10:42:02.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.614+0000 7feb99ffb640 1 -- 172.21.15.83:0/4019332817 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feba0018070 con 0x7feba40b09e0 2024-04-17T10:42:02.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.614+0000 7feb99ffb640 1 --2- 172.21.15.83:0/4019332817 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feb88066f60 0x7feb88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.614+0000 7feb99ffb640 1 -- 172.21.15.83:0/4019332817 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7feba0084f10 con 0x7feba40b09e0 2024-04-17T10:42:02.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.614+0000 7feba9089640 1 --2- 172.21.15.83:0/4019332817 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feb88066f60 0x7feb88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.614+0000 7feba9089640 1 --2- 172.21.15.83:0/4019332817 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feb88066f60 0x7feb88069420 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7feb94000f70 tx=0x7feb94007b60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:02.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.614+0000 7febae45a000 1 Processor -- start 2024-04-17T10:42:02.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.614+0000 7febae45a000 1 -- start start 2024-04-17T10:42:02.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.614+0000 7febae45a000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6646e29d0 0x55f6646e2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.614+0000 7febae45a000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f6646e32f0 0x55f664698b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.614+0000 7febae45a000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f6646990a0 0x55f664699480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.614+0000 7febae45a000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6644e5ff0 con 0x55f6646e29d0 2024-04-17T10:42:02.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.614+0000 7febae45a000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f66446e150 con 0x55f6646990a0 2024-04-17T10:42:02.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.614+0000 7febae45a000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f6644e2090 con 0x55f6646e32f0 2024-04-17T10:42:02.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.614+0000 7feba9089640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6646e29d0 0x55f6646e2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.614+0000 7feba9089640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6646e29d0 0x55f6646e2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42034/0 (socket says 172.21.15.83:42034) 2024-04-17T10:42:02.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.614+0000 7feba9089640 1 -- 172.21.15.83:0/523240894 learned_addr learned my addr 172.21.15.83:0/523240894 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:02.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7feba9089640 1 -- 172.21.15.83:0/523240894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f6646e32f0 msgr2=0x55f664698b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:02.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7feba9089640 1 --2- 172.21.15.83:0/523240894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f6646e32f0 0x55f664698b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7feba9089640 1 -- 172.21.15.83:0/523240894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f6646990a0 msgr2=0x55f664699480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:02.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7feba9089640 1 --2- 172.21.15.83:0/523240894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f6646990a0 0x55f664699480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7feba9089640 1 -- 172.21.15.83:0/523240894 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f66467c0d0 con 0x55f6646e29d0 2024-04-17T10:42:02.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7feba9089640 1 --2- 172.21.15.83:0/523240894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6646e29d0 0x55f6646e2db0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7feb94005d10 tx=0x7feb94012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466a2e863081538f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:02.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7feb8effd640 1 -- 172.21.15.83:0/523240894 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb94015110 con 0x55f6646e29d0 2024-04-17T10:42:02.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7feb8effd640 1 -- 172.21.15.83:0/523240894 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb940159c0 con 0x55f6646e29d0 2024-04-17T10:42:02.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7feb8effd640 1 -- 172.21.15.83:0/523240894 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb94015d70 con 0x55f6646e29d0 2024-04-17T10:42:02.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7febae45a000 1 -- 172.21.15.83:0/523240894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6646e29d0 msgr2=0x55f6646e2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7febae45a000 1 --2- 172.21.15.83:0/523240894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6646e29d0 0x55f6646e2db0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7feb94005d10 tx=0x7feb94012000 comp rx=0 tx=0).stop 2024-04-17T10:42:02.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7febae45a000 1 -- 172.21.15.83:0/523240894 shutdown_connections 2024-04-17T10:42:02.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7febae45a000 1 --2- 172.21.15.83:0/523240894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f6646990a0 0x55f664699480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7febae45a000 1 --2- 172.21.15.83:0/523240894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f6646e32f0 0x55f664698b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7febae45a000 1 --2- 172.21.15.83:0/523240894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6646e29d0 0x55f6646e2db0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7febae45a000 1 -- 172.21.15.83:0/523240894 >> 172.21.15.83:0/523240894 conn(0x55f664695020 msgr2=0x55f664697440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:02.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7febae45a000 1 -- 172.21.15.83:0/523240894 shutdown_connections 2024-04-17T10:42:02.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7febae45a000 1 -- 172.21.15.83:0/523240894 wait complete. 2024-04-17T10:42:02.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7febae45a000 1 -- start start 2024-04-17T10:42:02.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7febae45a000 1 Processor -- start 2024-04-17T10:42:02.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7febae45a000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f6646990a0 0x55f6646c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7febae45a000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f6646e29d0 0x55f6646c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7febae45a000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6646e32f0 0x55f6646c8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7febae45a000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6644aa250 con 0x55f6646e32f0 2024-04-17T10:42:02.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7febae45a000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6644a0690 con 0x55f6646e29d0 2024-04-17T10:42:02.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7febae45a000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f66446ea40 con 0x55f6646990a0 2024-04-17T10:42:02.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7febae45a000 10 client.0 _ll_register_callbacks cb 0x55f664683e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:02.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7feba988a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6646e32f0 0x55f6646c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7feba988a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6646e32f0 0x55f6646c8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42038/0 (socket says 172.21.15.83:42038) 2024-04-17T10:42:02.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7feba988a640 1 -- 172.21.15.83:0/1541920571 learned_addr learned my addr 172.21.15.83:0/1541920571 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:02.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7feba8888640 1 --2- 172.21.15.83:0/1541920571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f6646e29d0 0x55f6646c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7feba9089640 1 --2- 172.21.15.83:0/1541920571 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f6646990a0 0x55f6646c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7feba988a640 1 -- 172.21.15.83:0/1541920571 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f6646990a0 msgr2=0x55f6646c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:02.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7feba988a640 1 --2- 172.21.15.83:0/1541920571 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f6646990a0 0x55f6646c77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7feba988a640 1 -- 172.21.15.83:0/1541920571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f6646e29d0 msgr2=0x55f6646c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:02.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7feba988a640 1 --2- 172.21.15.83:0/1541920571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f6646e29d0 0x55f6646c7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7feba988a640 1 -- 172.21.15.83:0/1541920571 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f6645931c0 con 0x55f6646e32f0 2024-04-17T10:42:02.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7feba8888640 1 --2- 172.21.15.83:0/1541920571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f6646e29d0 0x55f6646c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:02.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7feba988a640 1 --2- 172.21.15.83:0/1541920571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6646e32f0 0x55f6646c8270 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7feb9c019540 tx=0x7feb9c0196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b9f1c28277ca4e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:02.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7feb83fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:02.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7feb83fff640 1 -- 172.21.15.83:0/1541920571 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c0bc070 con 0x55f6646e32f0 2024-04-17T10:42:02.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7febae45a000 1 -- 172.21.15.83:0/1541920571 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feb940159c0 con 0x55f6646e32f0 2024-04-17T10:42:02.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7feb83fff640 1 -- 172.21.15.83:0/1541920571 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7feb9c0a5af0 con 0x55f6646e32f0 2024-04-17T10:42:02.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7feb83fff640 1 -- 172.21.15.83:0/1541920571 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb9c0c6050 con 0x55f6646e32f0 2024-04-17T10:42:02.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7febae45a000 10 client.6027 fetch_fsmap learned FSMap version 21 2024-04-17T10:42:02.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7febae45a000 1 -- 172.21.15.83:0/1541920571 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55f66458f170 con 0x55f6646e32f0 2024-04-17T10:42:02.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7feb83fff640 1 -- 172.21.15.83:0/1541920571 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb940159c0 con 0x55f6646e32f0 2024-04-17T10:42:02.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7feb83fff640 1 -- 172.21.15.83:0/1541920571 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7feb9c008040 con 0x55f6646e32f0 2024-04-17T10:42:02.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7febae45a000 10 client.6027 fetch_fsmap finished waiting for FSMap version 21 2024-04-17T10:42:02.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7febae45a000 10 client.6027 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:42:02.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7febae45a000 20 client.6027 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:02.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7febae45a000 1 --2- 172.21.15.83:0/1541920571 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55f6646f2c80 0x55f6646f5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7febae45a000 4 client.6027 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:42:02.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7febae45a000 1 -- 172.21.15.83:0/1541920571 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f664481c60 con 0x55f6646f2c80 2024-04-17T10:42:02.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.618+0000 7feba9089640 1 --2- 172.21.15.83:0/1541920571 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55f6646f2c80 0x55f6646f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.622+0000 7feb83fff640 10 client.6027 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:02.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.622+0000 7feba9089640 1 --2- 172.21.15.83:0/1541920571 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55f6646f2c80 0x55f6646f5070 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fb17f113d557431 server_cookie=41ca19ba6f96e7d8 in_seq=0 out_seq=0 2024-04-17T10:42:02.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.622+0000 7feb83fff640 1 -- 172.21.15.83:0/1541920571 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f664481c60 con 0x55f6646f2c80 2024-04-17T10:42:02.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.622+0000 7feb83fff640 10 client.6027 handle_command_reply: tid=0 2024-04-17T10:42:02.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.622+0000 7febae45a000 1 client.6027 shutdown 2024-04-17T10:42:02.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.622+0000 7febae45a000 1 -- 172.21.15.83:0/1541920571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6646e32f0 msgr2=0x55f6646c8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.622+0000 7febae45a000 1 --2- 172.21.15.83:0/1541920571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6646e32f0 0x55f6646c8270 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7feb9c019540 tx=0x7feb9c0196d0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.622+0000 7febae45a000 1 -- 172.21.15.83:0/1541920571 shutdown_connections 2024-04-17T10:42:02.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.622+0000 7febae45a000 1 --2- 172.21.15.83:0/1541920571 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55f6646f2c80 0x55f6646f5070 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.622+0000 7febae45a000 1 --2- 172.21.15.83:0/1541920571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6646e32f0 0x55f6646c8270 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.622+0000 7febae45a000 1 --2- 172.21.15.83:0/1541920571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f6646e29d0 0x55f6646c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.622+0000 7feb83fff640 0 client.6027 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:02.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.622+0000 7febae45a000 1 --2- 172.21.15.83:0/1541920571 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f6646990a0 0x55f6646c77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.622+0000 7febae45a000 1 -- 172.21.15.83:0/1541920571 >> 172.21.15.83:0/1541920571 conn(0x55f664695020 msgr2=0x55f664697ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:02.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.622+0000 7febae45a000 1 -- 172.21.15.83:0/1541920571 shutdown_connections 2024-04-17T10:42:02.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.622+0000 7febae45a000 1 -- 172.21.15.83:0/1541920571 wait complete. 2024-04-17T10:42:02.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.622+0000 7febae45a000 20 client.6027 trim_cache size 0 max 16384 2024-04-17T10:42:02.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.634+0000 7febae45a000 1 Processor -- start 2024-04-17T10:42:02.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.634+0000 7febae45a000 1 -- start start 2024-04-17T10:42:02.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f6646990a0 0x55f664697410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f6646e29d0 0x55f664697950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6646e32f0 0x55f664697e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba4202f40 con 0x55f6646e32f0 2024-04-17T10:42:02.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba41fac20 con 0x55f6646990a0 2024-04-17T10:42:02.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba40c5f00 con 0x55f6646e29d0 2024-04-17T10:42:02.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7feba988a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6646e32f0 0x55f664697e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7feba988a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6646e32f0 0x55f664697e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42050/0 (socket says 172.21.15.83:42050) 2024-04-17T10:42:02.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7feba988a640 1 -- 172.21.15.83:0/2874179150 learned_addr learned my addr 172.21.15.83:0/2874179150 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:02.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7feba988a640 1 -- 172.21.15.83:0/2874179150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f6646e29d0 msgr2=0x55f664697950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:02.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7feba988a640 1 --2- 172.21.15.83:0/2874179150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f6646e29d0 0x55f664697950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7feba988a640 1 -- 172.21.15.83:0/2874179150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f6646990a0 msgr2=0x55f664697410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:02.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7feba988a640 1 --2- 172.21.15.83:0/2874179150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f6646990a0 0x55f664697410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7feba988a640 1 -- 172.21.15.83:0/2874179150 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f66452cde0 con 0x55f6646e32f0 2024-04-17T10:42:02.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7feba988a640 1 --2- 172.21.15.83:0/2874179150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6646e32f0 0x55f664697e90 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7feb9c01d1d0 tx=0x7feb9c0b9000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf6960b0e11ae60 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:02.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7feb8e7fc640 1 -- 172.21.15.83:0/2874179150 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c0cf070 con 0x55f6646e32f0 2024-04-17T10:42:02.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7feb8e7fc640 1 -- 172.21.15.83:0/2874179150 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9c0a5700 con 0x55f6646e32f0 2024-04-17T10:42:02.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7feb8e7fc640 1 -- 172.21.15.83:0/2874179150 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c0bc040 con 0x55f6646e32f0 2024-04-17T10:42:02.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 1 -- 172.21.15.83:0/2874179150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6646e32f0 msgr2=0x55f664697e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 1 --2- 172.21.15.83:0/2874179150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6646e32f0 0x55f664697e90 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7feb9c01d1d0 tx=0x7feb9c0b9000 comp rx=0 tx=0).stop 2024-04-17T10:42:02.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 1 -- 172.21.15.83:0/2874179150 shutdown_connections 2024-04-17T10:42:02.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 1 --2- 172.21.15.83:0/2874179150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6646e32f0 0x55f664697e90 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 1 --2- 172.21.15.83:0/2874179150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f6646e29d0 0x55f664697950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 1 --2- 172.21.15.83:0/2874179150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f6646990a0 0x55f664697410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 1 -- 172.21.15.83:0/2874179150 >> 172.21.15.83:0/2874179150 conn(0x55f664695020 msgr2=0x55f6646e10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:02.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 1 -- 172.21.15.83:0/2874179150 shutdown_connections 2024-04-17T10:42:02.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 1 -- 172.21.15.83:0/2874179150 wait complete. 2024-04-17T10:42:02.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 1 -- start start 2024-04-17T10:42:02.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 1 Processor -- start 2024-04-17T10:42:02.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6646990a0 0x55f6646f5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f6646e29d0 0x55f6646f58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f6646e32f0 0x55f6646f5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f664480e00 con 0x55f6646990a0 2024-04-17T10:42:02.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba41fa620 con 0x55f6646e29d0 2024-04-17T10:42:02.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f6644e5300 con 0x55f6646e32f0 2024-04-17T10:42:02.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 10 client.0 _ll_register_callbacks cb 0x55f664664040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:02.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7feba9089640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6646990a0 0x55f6646f5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7feba9089640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6646990a0 0x55f6646f5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42060/0 (socket says 172.21.15.83:42060) 2024-04-17T10:42:02.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7feba9089640 1 -- 172.21.15.83:0/366760350 learned_addr learned my addr 172.21.15.83:0/366760350 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:02.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7feba9089640 1 -- 172.21.15.83:0/366760350 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f6646e32f0 msgr2=0x55f6646f5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:02.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7feba9089640 1 --2- 172.21.15.83:0/366760350 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f6646e32f0 0x55f6646f5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7feba9089640 1 -- 172.21.15.83:0/366760350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f6646e29d0 msgr2=0x55f6646f58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:02.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7feba9089640 1 --2- 172.21.15.83:0/366760350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f6646e29d0 0x55f6646f58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7feba9089640 1 -- 172.21.15.83:0/366760350 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f6644878e0 con 0x55f6646990a0 2024-04-17T10:42:02.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7feba9089640 1 --2- 172.21.15.83:0/366760350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6646990a0 0x55f6646f5380 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7feb940011f0 tx=0x7feb94001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7742446c622d0f70 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:02.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7feb83fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:02.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7feb83fff640 1 -- 172.21.15.83:0/366760350 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb94015240 con 0x55f6646990a0 2024-04-17T10:42:02.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7feb83fff640 1 -- 172.21.15.83:0/366760350 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7feb94015b50 con 0x55f6646990a0 2024-04-17T10:42:02.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 1 -- 172.21.15.83:0/366760350 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feb9c0a5700 con 0x55f6646990a0 2024-04-17T10:42:02.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7feb83fff640 1 -- 172.21.15.83:0/366760350 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb94033050 con 0x55f6646990a0 2024-04-17T10:42:02.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 10 client.6033 fetch_fsmap learned FSMap version 21 2024-04-17T10:42:02.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 1 -- 172.21.15.83:0/366760350 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55f66453f8f0 con 0x55f6646990a0 2024-04-17T10:42:02.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7feb83fff640 1 -- 172.21.15.83:0/366760350 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9c0a5700 con 0x55f6646990a0 2024-04-17T10:42:02.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7feb83fff640 1 -- 172.21.15.83:0/366760350 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7feb94036080 con 0x55f6646990a0 2024-04-17T10:42:02.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 10 client.6033 fetch_fsmap finished waiting for FSMap version 21 2024-04-17T10:42:02.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 10 client.6033 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:42:02.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 20 client.6033 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:02.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 1 --2- 172.21.15.83:0/366760350 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55f6646f6340 0x55f6646f6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:02.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 4 client.6033 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T10:42:02.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.638+0000 7febae45a000 1 -- 172.21.15.83:0/366760350 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f6644f3530 con 0x55f6646f6340 2024-04-17T10:42:02.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.642+0000 7feba8888640 1 --2- 172.21.15.83:0/366760350 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55f6646f6340 0x55f6646f6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:02.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.642+0000 7feb83fff640 10 client.6033 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:02.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.642+0000 7feba8888640 1 --2- 172.21.15.83:0/366760350 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55f6646f6340 0x55f6646f6750 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5d3f3bf6501e97f server_cookie=24b41b70461ae752 in_seq=0 out_seq=0 2024-04-17T10:42:02.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.642+0000 7feb83fff640 1 -- 172.21.15.83:0/366760350 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+903 (crc 0 0 0) 0x55f6644f3530 con 0x55f6646f6340 2024-04-17T10:42:02.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.642+0000 7feb83fff640 10 client.6033 handle_command_reply: tid=0 2024-04-17T10:42:02.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.642+0000 7febae45a000 1 client.6033 shutdown 2024-04-17T10:42:02.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.642+0000 7febae45a000 1 -- 172.21.15.83:0/366760350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6646990a0 msgr2=0x55f6646f5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.642+0000 7febae45a000 1 --2- 172.21.15.83:0/366760350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6646990a0 0x55f6646f5380 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7feb940011f0 tx=0x7feb94001090 comp rx=0 tx=0).stop 2024-04-17T10:42:02.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.642+0000 7febae45a000 1 -- 172.21.15.83:0/366760350 shutdown_connections 2024-04-17T10:42:02.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.642+0000 7febae45a000 1 --2- 172.21.15.83:0/366760350 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55f6646f6340 0x55f6646f6750 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.642+0000 7febae45a000 1 --2- 172.21.15.83:0/366760350 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f6646e32f0 0x55f6646f5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.642+0000 7febae45a000 1 --2- 172.21.15.83:0/366760350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f6646e29d0 0x55f6646f58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.642+0000 7febae45a000 1 --2- 172.21.15.83:0/366760350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6646990a0 0x55f6646f5380 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.642+0000 7febae45a000 1 -- 172.21.15.83:0/366760350 >> 172.21.15.83:0/366760350 conn(0x55f664695020 msgr2=0x55f6646e1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:02.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.642+0000 7feb83fff640 0 client.6033 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:02.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.642+0000 7febae45a000 1 -- 172.21.15.83:0/366760350 shutdown_connections 2024-04-17T10:42:02.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.642+0000 7febae45a000 1 -- 172.21.15.83:0/366760350 wait complete. 2024-04-17T10:42:02.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.642+0000 7febae45a000 20 client.6033 trim_cache size 0 max 16384 2024-04-17T10:42:02.646 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:42:02.646 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T10:42:02.646 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-17T10:42:02.646 INFO:teuthology.orchestra.run.smithi083.stdout: "7c54d888-cbc0-4059-bc71-1ed816f1ab84": { 2024-04-17T10:42:02.646 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml", 2024-04-17T10:42:02.646 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "7c54d888-cbc0-4059-bc71-1ed816f1ab84", 2024-04-17T10:42:02.646 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "" 2024-04-17T10:42:02.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:42:02.646 INFO:teuthology.orchestra.run.smithi083.stdout: "ca55178c-5ad9-4324-923e-065575311828": { 2024-04-17T10:42:02.646 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_0", 2024-04-17T10:42:02.647 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "ca55178c-5ad9-4324-923e-065575311828", 2024-04-17T10:42:02.647 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "" 2024-04-17T10:42:02.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:42:02.647 INFO:teuthology.orchestra.run.smithi083.stdout: "e3a4cbbe-5a2a-4f32-a018-8c94320b148c": { 2024-04-17T10:42:02.647 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/scrub_test_path/ceph-qa-suite/suites", 2024-04-17T10:42:02.647 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "e3a4cbbe-5a2a-4f32-a018-8c94320b148c", 2024-04-17T10:42:02.647 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "" 2024-04-17T10:42:02.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:42:02.647 INFO:teuthology.orchestra.run.smithi083.stdout: "f26bab6e-d69d-4de2-9b8c-b1e3503848d5": { 2024-04-17T10:42:02.647 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-17T10:42:02.647 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "f26bab6e-d69d-4de2-9b8c-b1e3503848d5", 2024-04-17T10:42:02.647 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "" 2024-04-17T10:42:02.647 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:42:02.647 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:42:02.648 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:42:02.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.642+0000 7febab2ed640 1 -- 172.21.15.83:0/4019332817 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feb88066f60 msgr2=0x7feb88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.642+0000 7febab2ed640 1 --2- 172.21.15.83:0/4019332817 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feb88066f60 0x7feb88069420 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7feb94000f70 tx=0x7feb94007b60 comp rx=0 tx=0).stop 2024-04-17T10:42:02.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.642+0000 7febab2ed640 1 -- 172.21.15.83:0/4019332817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feba40b09e0 msgr2=0x7feba406f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:02.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.642+0000 7febab2ed640 1 --2- 172.21.15.83:0/4019332817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feba40b09e0 0x7feba406f400 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7feba000c820 tx=0x7feba000cce0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.642+0000 7febab2ed640 1 -- 172.21.15.83:0/4019332817 shutdown_connections 2024-04-17T10:42:02.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.642+0000 7febab2ed640 1 --2- 172.21.15.83:0/4019332817 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feb88066f60 0x7feb88069420 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.642+0000 7febab2ed640 1 --2- 172.21.15.83:0/4019332817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feba415de10 0x7feba40b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.642+0000 7febab2ed640 1 --2- 172.21.15.83:0/4019332817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feba40b09e0 0x7feba406f400 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.642+0000 7febab2ed640 1 --2- 172.21.15.83:0/4019332817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feba406ccd0 0x7feba40c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:02.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.642+0000 7febab2ed640 1 -- 172.21.15.83:0/4019332817 >> 172.21.15.83:0/4019332817 conn(0x7feba40d1730 msgr2=0x7feba415bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:02.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.642+0000 7febab2ed640 1 -- 172.21.15.83:0/4019332817 shutdown_connections 2024-04-17T10:42:02.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:02.642+0000 7febab2ed640 1 -- 172.21.15.83:0/4019332817 wait complete. 2024-04-17T10:42:06.919 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:42:06.922 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:42:07.360 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-04-17T10:42:07.362+0000 7f35c6386640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-17T10:42:07.656 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-17T10:42:07.656 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-17T10:42:07.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.790+0000 7f0fec73b640 1 Processor -- start 2024-04-17T10:42:07.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.790+0000 7f0fec73b640 1 -- start start 2024-04-17T10:42:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.794+0000 7f0fec73b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0fe415bc10 0x7f0fe40d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.794+0000 7f0fec73b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe40d0490 0x7f0fe40cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.794+0000 7f0fec73b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0fe40cd3d0 0x7f0fe406e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.794+0000 7f0fec73b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe40ced00 con 0x7f0fe415bc10 2024-04-17T10:42:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.794+0000 7f0fec73b640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe40cefe0 con 0x7f0fe40cd3d0 2024-04-17T10:42:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.794+0000 7f0fec73b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe40cc690 con 0x7f0fe40d0490 2024-04-17T10:42:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.794+0000 7f0fea4d7640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0fe415bc10 0x7f0fe40d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.794+0000 7f0fea4d7640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0fe415bc10 0x7f0fe40d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40880/0 (socket says 172.21.15.83:40880) 2024-04-17T10:42:07.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.794+0000 7f0fea4d7640 1 -- 172.21.15.83:0/1292251527 learned_addr learned my addr 172.21.15.83:0/1292251527 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:07.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.794+0000 7f0feacd8640 1 --2- 172.21.15.83:0/1292251527 >> v2:172.21.15.167:3300/0 conn(0x7f0fe40cd3d0 0x7f0fe406e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.794+0000 7f0fe9cd6640 1 --2- 172.21.15.83:0/1292251527 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe40d0490 0x7f0fe40cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.794+0000 7f0fea4d7640 1 -- 172.21.15.83:0/1292251527 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe40d0490 msgr2=0x7f0fe40cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:07.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.794+0000 7f0fea4d7640 1 --2- 172.21.15.83:0/1292251527 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe40d0490 0x7f0fe40cce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.794+0000 7f0fea4d7640 1 -- 172.21.15.83:0/1292251527 >> v2:172.21.15.167:3300/0 conn(0x7f0fe40cd3d0 msgr2=0x7f0fe406e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:07.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.794+0000 7f0fea4d7640 1 --2- 172.21.15.83:0/1292251527 >> v2:172.21.15.167:3300/0 conn(0x7f0fe40cd3d0 0x7f0fe406e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.794+0000 7f0fea4d7640 1 -- 172.21.15.83:0/1292251527 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe40cc970 con 0x7f0fe415bc10 2024-04-17T10:42:07.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.794+0000 7f0feacd8640 1 --2- 172.21.15.83:0/1292251527 >> v2:172.21.15.167:3300/0 conn(0x7f0fe40cd3d0 0x7f0fe406e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:07.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.794+0000 7f0fea4d7640 1 --2- 172.21.15.83:0/1292251527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe415bc10 0x7f0fe40d2300 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f0fe40b7710 tx=0x7f0fe001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed30ced28a21615 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:07.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.794+0000 7f0fe94d5640 1 -- 172.21.15.83:0/1292251527 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe00b0070 con 0x7f0fe415bc10 2024-04-17T10:42:07.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.794+0000 7f0fe94d5640 1 -- 172.21.15.83:0/1292251527 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe00ab600 con 0x7f0fe415bc10 2024-04-17T10:42:07.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.794+0000 7f0fe94d5640 1 -- 172.21.15.83:0/1292251527 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe00ab9d0 con 0x7f0fe415bc10 2024-04-17T10:42:07.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0fec73b640 1 -- 172.21.15.83:0/1292251527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe415bc10 msgr2=0x7f0fe40d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:07.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0fec73b640 1 --2- 172.21.15.83:0/1292251527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe415bc10 0x7f0fe40d2300 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f0fe40b7710 tx=0x7f0fe001c490 comp rx=0 tx=0).stop 2024-04-17T10:42:07.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0fec73b640 1 -- 172.21.15.83:0/1292251527 shutdown_connections 2024-04-17T10:42:07.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0fec73b640 1 --2- 172.21.15.83:0/1292251527 >> v2:172.21.15.167:3300/0 conn(0x7f0fe40cd3d0 0x7f0fe406e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0fec73b640 1 --2- 172.21.15.83:0/1292251527 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe40d0490 0x7f0fe40cce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0fec73b640 1 --2- 172.21.15.83:0/1292251527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe415bc10 0x7f0fe40d2300 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0fec73b640 1 -- 172.21.15.83:0/1292251527 >> 172.21.15.83:0/1292251527 conn(0x7f0fe40d17e0 msgr2=0x7f0fe40d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:07.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0fec73b640 1 -- 172.21.15.83:0/1292251527 shutdown_connections 2024-04-17T10:42:07.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0fec73b640 1 -- 172.21.15.83:0/1292251527 wait complete. 2024-04-17T10:42:07.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0fec73b640 1 Processor -- start 2024-04-17T10:42:07.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0fec73b640 1 -- start start 2024-04-17T10:42:07.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0fec73b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fe40cd3d0 0x7f0fe40c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0fec73b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe40d0490 0x7f0fe41646d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0fec73b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe415bc10 0x7f0fe4155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0fec73b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe415f9b0 con 0x7f0fe415bc10 2024-04-17T10:42:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0fec73b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe415f830 con 0x7f0fe40cd3d0 2024-04-17T10:42:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0fec73b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe415fb30 con 0x7f0fe40d0490 2024-04-17T10:42:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0feacd8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe415bc10 0x7f0fe4155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0feacd8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe415bc10 0x7f0fe4155820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40888/0 (socket says 172.21.15.83:40888) 2024-04-17T10:42:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0feacd8640 1 -- 172.21.15.83:0/3945752470 learned_addr learned my addr 172.21.15.83:0/3945752470 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0fea4d7640 1 --2- 172.21.15.83:0/3945752470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fe40cd3d0 0x7f0fe40c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0fe9cd6640 1 --2- 172.21.15.83:0/3945752470 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe40d0490 0x7f0fe41646d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0feacd8640 1 -- 172.21.15.83:0/3945752470 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe40d0490 msgr2=0x7f0fe41646d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0feacd8640 1 --2- 172.21.15.83:0/3945752470 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe40d0490 0x7f0fe41646d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0feacd8640 1 -- 172.21.15.83:0/3945752470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fe40cd3d0 msgr2=0x7f0fe40c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0feacd8640 1 --2- 172.21.15.83:0/3945752470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fe40cd3d0 0x7f0fe40c5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0feacd8640 1 -- 172.21.15.83:0/3945752470 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe40b2160 con 0x7f0fe415bc10 2024-04-17T10:42:07.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0fea4d7640 1 --2- 172.21.15.83:0/3945752470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fe40cd3d0 0x7f0fe40c5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:42:07.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0feacd8640 1 --2- 172.21.15.83:0/3945752470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe415bc10 0x7f0fe4155820 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc00cef0 tx=0x7f0fdc00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:07.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0fdaffd640 1 -- 172.21.15.83:0/3945752470 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc01a070 con 0x7f0fe415bc10 2024-04-17T10:42:07.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0fdaffd640 1 -- 172.21.15.83:0/3945752470 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fdc00d840 con 0x7f0fe415bc10 2024-04-17T10:42:07.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0fec73b640 1 -- 172.21.15.83:0/3945752470 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe40b2440 con 0x7f0fe415bc10 2024-04-17T10:42:07.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0fdaffd640 1 -- 172.21.15.83:0/3945752470 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc00dc30 con 0x7f0fe415bc10 2024-04-17T10:42:07.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.798+0000 7f0fec73b640 1 -- 172.21.15.83:0/3945752470 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe40b2a00 con 0x7f0fe415bc10 2024-04-17T10:42:07.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.802+0000 7f0fdaffd640 1 -- 172.21.15.83:0/3945752470 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0fdc01d070 con 0x7f0fe415bc10 2024-04-17T10:42:07.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.802+0000 7f0fdaffd640 1 --2- 172.21.15.83:0/3945752470 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0fc8066f60 0x7f0fc8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.802+0000 7f0fdaffd640 1 -- 172.21.15.83:0/3945752470 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0fdc085810 con 0x7f0fe415bc10 2024-04-17T10:42:07.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.802+0000 7f0fea4d7640 1 --2- 172.21.15.83:0/3945752470 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0fc8066f60 0x7f0fc8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.802+0000 7f0fea4d7640 1 --2- 172.21.15.83:0/3945752470 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0fc8066f60 0x7f0fc8069420 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0fe00ad000 tx=0x7f0fe00a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:07.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.802+0000 7f0fef8a8000 1 Processor -- start 2024-04-17T10:42:07.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.802+0000 7f0fef8a8000 1 -- start start 2024-04-17T10:42:07.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.802+0000 7f0fef8a8000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559351ac99d0 0x559351ac9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.802+0000 7f0fef8a8000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559351aca2f0 0x559351a7fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.802+0000 7f0fef8a8000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559351a800a0 0x559351a80480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.802+0000 7f0fef8a8000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593518ccff0 con 0x559351ac99d0 2024-04-17T10:42:07.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.802+0000 7f0fef8a8000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559351855150 con 0x559351a800a0 2024-04-17T10:42:07.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.802+0000 7f0fef8a8000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5593518c9090 con 0x559351aca2f0 2024-04-17T10:42:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.802+0000 7f0fea4d7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559351ac99d0 0x559351ac9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.802+0000 7f0fea4d7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559351ac99d0 0x559351ac9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40898/0 (socket says 172.21.15.83:40898) 2024-04-17T10:42:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.802+0000 7f0fea4d7640 1 -- 172.21.15.83:0/2342510284 learned_addr learned my addr 172.21.15.83:0/2342510284 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.802+0000 7f0fe9cd6640 1 --2- 172.21.15.83:0/2342510284 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559351aca2f0 0x559351a7fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.802+0000 7f0fea4d7640 1 -- 172.21.15.83:0/2342510284 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559351aca2f0 msgr2=0x559351a7fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.802+0000 7f0feacd8640 1 --2- 172.21.15.83:0/2342510284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559351a800a0 0x559351a80480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.802+0000 7f0fea4d7640 1 --2- 172.21.15.83:0/2342510284 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559351aca2f0 0x559351a7fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.802+0000 7f0fea4d7640 1 -- 172.21.15.83:0/2342510284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559351a800a0 msgr2=0x559351a80480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.802+0000 7f0fea4d7640 1 --2- 172.21.15.83:0/2342510284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559351a800a0 0x559351a80480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.802+0000 7f0fea4d7640 1 -- 172.21.15.83:0/2342510284 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559351a630d0 con 0x559351ac99d0 2024-04-17T10:42:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.802+0000 7f0fea4d7640 1 --2- 172.21.15.83:0/2342510284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559351ac99d0 0x559351ac9db0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f0fe001c4d0 tx=0x7f0fe0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=376e16495b3ac9e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.802+0000 7f0fc7fff640 1 -- 172.21.15.83:0/2342510284 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe00c7070 con 0x559351ac99d0 2024-04-17T10:42:07.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.802+0000 7f0fc7fff640 1 -- 172.21.15.83:0/2342510284 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe00c2470 con 0x559351ac99d0 2024-04-17T10:42:07.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.802+0000 7f0fc7fff640 1 -- 172.21.15.83:0/2342510284 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe00b0040 con 0x559351ac99d0 2024-04-17T10:42:07.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fef8a8000 1 -- 172.21.15.83:0/2342510284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559351ac99d0 msgr2=0x559351ac9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:07.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fef8a8000 1 --2- 172.21.15.83:0/2342510284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559351ac99d0 0x559351ac9db0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f0fe001c4d0 tx=0x7f0fe0004740 comp rx=0 tx=0).stop 2024-04-17T10:42:07.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fef8a8000 1 -- 172.21.15.83:0/2342510284 shutdown_connections 2024-04-17T10:42:07.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fef8a8000 1 --2- 172.21.15.83:0/2342510284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559351a800a0 0x559351a80480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fef8a8000 1 --2- 172.21.15.83:0/2342510284 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559351aca2f0 0x559351a7fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fef8a8000 1 --2- 172.21.15.83:0/2342510284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559351ac99d0 0x559351ac9db0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fef8a8000 1 -- 172.21.15.83:0/2342510284 >> 172.21.15.83:0/2342510284 conn(0x559351a7c020 msgr2=0x559351a7e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:07.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fef8a8000 1 -- 172.21.15.83:0/2342510284 shutdown_connections 2024-04-17T10:42:07.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fef8a8000 1 -- 172.21.15.83:0/2342510284 wait complete. 2024-04-17T10:42:07.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fef8a8000 1 -- start start 2024-04-17T10:42:07.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fef8a8000 1 Processor -- start 2024-04-17T10:42:07.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fef8a8000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559351a800a0 0x559351aae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fef8a8000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559351ac99d0 0x559351aaed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fef8a8000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559351aca2f0 0x559351aaf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fef8a8000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559351891250 con 0x559351aca2f0 2024-04-17T10:42:07.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fef8a8000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559351887690 con 0x559351a800a0 2024-04-17T10:42:07.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fef8a8000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x559351855a40 con 0x559351ac99d0 2024-04-17T10:42:07.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fef8a8000 10 client.0 _ll_register_callbacks cb 0x559351a6ae90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:07.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0feacd8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559351aca2f0 0x559351aaf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0feacd8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559351aca2f0 0x559351aaf270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40908/0 (socket says 172.21.15.83:40908) 2024-04-17T10:42:07.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0feacd8640 1 -- 172.21.15.83:0/3334868073 learned_addr learned my addr 172.21.15.83:0/3334868073 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:07.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fea4d7640 1 --2- 172.21.15.83:0/3334868073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559351a800a0 0x559351aae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fe9cd6640 1 --2- 172.21.15.83:0/3334868073 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559351ac99d0 0x559351aaed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0feacd8640 1 -- 172.21.15.83:0/3334868073 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559351ac99d0 msgr2=0x559351aaed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0feacd8640 1 --2- 172.21.15.83:0/3334868073 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559351ac99d0 0x559351aaed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0feacd8640 1 -- 172.21.15.83:0/3334868073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559351a800a0 msgr2=0x559351aae7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0feacd8640 1 --2- 172.21.15.83:0/3334868073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559351a800a0 0x559351aae7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0feacd8640 1 -- 172.21.15.83:0/3334868073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55935197a1c0 con 0x559351aca2f0 2024-04-17T10:42:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0feacd8640 1 --2- 172.21.15.83:0/3334868073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559351aca2f0 0x559351aaf270 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc05e350 tx=0x7f0fdc056930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d1f9a479719a92 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fc57fa640 1 -- 172.21.15.83:0/3334868073 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc05e880 con 0x559351aca2f0 2024-04-17T10:42:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fc57fa640 1 -- 172.21.15.83:0/3334868073 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0fdc054210 con 0x559351aca2f0 2024-04-17T10:42:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fef8a8000 1 -- 172.21.15.83:0/3334868073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0fe00c2470 con 0x559351aca2f0 2024-04-17T10:42:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fc57fa640 1 -- 172.21.15.83:0/3334868073 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fdc054470 con 0x559351aca2f0 2024-04-17T10:42:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fef8a8000 10 client.6045 fetch_fsmap learned FSMap version 22 2024-04-17T10:42:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fef8a8000 1 -- 172.21.15.83:0/3334868073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x559351976170 con 0x559351aca2f0 2024-04-17T10:42:07.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fc57fa640 1 -- 172.21.15.83:0/3334868073 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe00c2470 con 0x559351aca2f0 2024-04-17T10:42:07.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fc57fa640 1 -- 172.21.15.83:0/3334868073 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0fdc05e3f0 con 0x559351aca2f0 2024-04-17T10:42:07.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fef8a8000 10 client.6045 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T10:42:07.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fef8a8000 10 client.6045 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:42:07.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fef8a8000 20 client.6045 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:07.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fef8a8000 1 --2- 172.21.15.83:0/3334868073 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x559351ad9c80 0x559351adc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fef8a8000 4 client.6045 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:42:07.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fef8a8000 1 -- 172.21.15.83:0/3334868073 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559351868c60 con 0x559351ad9c80 2024-04-17T10:42:07.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fea4d7640 1 --2- 172.21.15.83:0/3334868073 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x559351ad9c80 0x559351adc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fea4d7640 1 --2- 172.21.15.83:0/3334868073 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x559351ad9c80 0x559351adc070 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc6e8e150aada623 server_cookie=b89dee67aea8eca0 in_seq=0 out_seq=0 2024-04-17T10:42:07.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.806+0000 7f0fc57fa640 10 client.6045 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:07.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.810+0000 7f0fc57fa640 1 -- 172.21.15.83:0/3334868073 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559351868c60 con 0x559351ad9c80 2024-04-17T10:42:07.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.810+0000 7f0fc57fa640 10 client.6045 handle_command_reply: tid=0 2024-04-17T10:42:07.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.810+0000 7f0fef8a8000 1 client.6045 shutdown 2024-04-17T10:42:07.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.810+0000 7f0fef8a8000 1 -- 172.21.15.83:0/3334868073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559351aca2f0 msgr2=0x559351aaf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:07.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.810+0000 7f0fef8a8000 1 --2- 172.21.15.83:0/3334868073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559351aca2f0 0x559351aaf270 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc05e350 tx=0x7f0fdc056930 comp rx=0 tx=0).stop 2024-04-17T10:42:07.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.810+0000 7f0fef8a8000 1 -- 172.21.15.83:0/3334868073 shutdown_connections 2024-04-17T10:42:07.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.810+0000 7f0fef8a8000 1 --2- 172.21.15.83:0/3334868073 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x559351ad9c80 0x559351adc070 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.810+0000 7f0fef8a8000 1 --2- 172.21.15.83:0/3334868073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559351aca2f0 0x559351aaf270 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.810+0000 7f0fef8a8000 1 --2- 172.21.15.83:0/3334868073 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559351ac99d0 0x559351aaed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.810+0000 7f0fc57fa640 0 client.6045 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:07.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.810+0000 7f0fef8a8000 1 --2- 172.21.15.83:0/3334868073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559351a800a0 0x559351aae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.810+0000 7f0fef8a8000 1 -- 172.21.15.83:0/3334868073 >> 172.21.15.83:0/3334868073 conn(0x559351a7c020 msgr2=0x559351a7eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:07.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.810+0000 7f0fef8a8000 1 -- 172.21.15.83:0/3334868073 shutdown_connections 2024-04-17T10:42:07.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.810+0000 7f0fef8a8000 1 -- 172.21.15.83:0/3334868073 wait complete. 2024-04-17T10:42:07.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.810+0000 7f0fef8a8000 20 client.6045 trim_cache size 0 max 16384 2024-04-17T10:42:07.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.822+0000 7f0fef8a8000 1 Processor -- start 2024-04-17T10:42:07.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.822+0000 7f0fef8a8000 1 -- start start 2024-04-17T10:42:07.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fef8a8000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559351a800a0 0x559351b26300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fef8a8000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559351ac99d0 0x559351b26840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fef8a8000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559351aca2f0 0x559351b26d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fef8a8000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe4203250 con 0x559351aca2f0 2024-04-17T10:42:07.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fef8a8000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe41fb1d0 con 0x559351a800a0 2024-04-17T10:42:07.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fef8a8000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe415cd80 con 0x559351ac99d0 2024-04-17T10:42:07.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0feacd8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559351aca2f0 0x559351b26d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0feacd8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559351aca2f0 0x559351b26d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40912/0 (socket says 172.21.15.83:40912) 2024-04-17T10:42:07.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0feacd8640 1 -- 172.21.15.83:0/999516299 learned_addr learned my addr 172.21.15.83:0/999516299 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:07.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0feacd8640 1 -- 172.21.15.83:0/999516299 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559351ac99d0 msgr2=0x559351b26840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:07.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0feacd8640 1 --2- 172.21.15.83:0/999516299 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559351ac99d0 0x559351b26840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0feacd8640 1 -- 172.21.15.83:0/999516299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559351a800a0 msgr2=0x559351b26300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:07.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fea4d7640 1 --2- 172.21.15.83:0/999516299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559351a800a0 0x559351b26300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0feacd8640 1 --2- 172.21.15.83:0/999516299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559351a800a0 0x559351b26300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0feacd8640 1 -- 172.21.15.83:0/999516299 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559351913de0 con 0x559351aca2f0 2024-04-17T10:42:07.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0feacd8640 1 --2- 172.21.15.83:0/999516299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559351aca2f0 0x559351b26d80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc052af0 tx=0x7f0fdc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e592737c4fe078 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:07.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fc77fe640 1 -- 172.21.15.83:0/999516299 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc055cb0 con 0x559351aca2f0 2024-04-17T10:42:07.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fc77fe640 1 -- 172.21.15.83:0/999516299 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fdc0555c0 con 0x559351aca2f0 2024-04-17T10:42:07.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fc77fe640 1 -- 172.21.15.83:0/999516299 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc05e880 con 0x559351aca2f0 2024-04-17T10:42:07.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fef8a8000 1 -- 172.21.15.83:0/999516299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559351aca2f0 msgr2=0x559351b26d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:07.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fef8a8000 1 --2- 172.21.15.83:0/999516299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559351aca2f0 0x559351b26d80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc052af0 tx=0x7f0fdc017000 comp rx=0 tx=0).stop 2024-04-17T10:42:07.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fef8a8000 1 -- 172.21.15.83:0/999516299 shutdown_connections 2024-04-17T10:42:07.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fef8a8000 1 --2- 172.21.15.83:0/999516299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559351aca2f0 0x559351b26d80 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fef8a8000 1 --2- 172.21.15.83:0/999516299 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559351ac99d0 0x559351b26840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fef8a8000 1 --2- 172.21.15.83:0/999516299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559351a800a0 0x559351b26300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fef8a8000 1 -- 172.21.15.83:0/999516299 >> 172.21.15.83:0/999516299 conn(0x559351a7c020 msgr2=0x559351ac80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:07.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fef8a8000 1 -- 172.21.15.83:0/999516299 shutdown_connections 2024-04-17T10:42:07.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fef8a8000 1 -- 172.21.15.83:0/999516299 wait complete. 2024-04-17T10:42:07.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fef8a8000 1 -- start start 2024-04-17T10:42:07.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fef8a8000 1 Processor -- start 2024-04-17T10:42:07.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fef8a8000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559351a800a0 0x559351adc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fef8a8000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559351ac99d0 0x559351adccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fef8a8000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559351aca2f0 0x559351add1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fef8a8000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559351867e00 con 0x559351a800a0 2024-04-17T10:42:07.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fef8a8000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe41fabd0 con 0x559351aca2f0 2024-04-17T10:42:07.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fef8a8000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5593518cc300 con 0x559351ac99d0 2024-04-17T10:42:07.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fef8a8000 10 client.0 _ll_register_callbacks cb 0x559351a4b040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:07.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fea4d7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559351a800a0 0x559351adc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fea4d7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559351a800a0 0x559351adc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40922/0 (socket says 172.21.15.83:40922) 2024-04-17T10:42:07.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fea4d7640 1 -- 172.21.15.83:0/2935729456 learned_addr learned my addr 172.21.15.83:0/2935729456 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:07.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fe9cd6640 1 --2- 172.21.15.83:0/2935729456 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559351ac99d0 0x559351adccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fea4d7640 1 -- 172.21.15.83:0/2935729456 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559351ac99d0 msgr2=0x559351adccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:07.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fea4d7640 1 --2- 172.21.15.83:0/2935729456 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559351ac99d0 0x559351adccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0feacd8640 1 --2- 172.21.15.83:0/2935729456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559351aca2f0 0x559351add1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fea4d7640 1 -- 172.21.15.83:0/2935729456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559351aca2f0 msgr2=0x559351add1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:07.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fea4d7640 1 --2- 172.21.15.83:0/2935729456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559351aca2f0 0x559351add1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.826+0000 7f0fea4d7640 1 -- 172.21.15.83:0/2935729456 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55935186e8e0 con 0x559351a800a0 2024-04-17T10:42:07.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fea4d7640 1 --2- 172.21.15.83:0/2935729456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559351a800a0 0x559351adc770 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f0fe001a0c0 tx=0x7f0fe001acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c265c4824542f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:07.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:07.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fc57fa640 1 -- 172.21.15.83:0/2935729456 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe00c7040 con 0x559351a800a0 2024-04-17T10:42:07.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fc57fa640 1 -- 172.21.15.83:0/2935729456 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0fe00c2470 con 0x559351a800a0 2024-04-17T10:42:07.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fef8a8000 1 -- 172.21.15.83:0/2935729456 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5593518da530 con 0x559351a800a0 2024-04-17T10:42:07.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fc57fa640 1 -- 172.21.15.83:0/2935729456 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fe00d8050 con 0x559351a800a0 2024-04-17T10:42:07.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fef8a8000 10 client.6051 fetch_fsmap learned FSMap version 22 2024-04-17T10:42:07.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fef8a8000 1 -- 172.21.15.83:0/2935729456 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x5593519268f0 con 0x559351a800a0 2024-04-17T10:42:07.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fc57fa640 1 -- 172.21.15.83:0/2935729456 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5593518da530 con 0x559351a800a0 2024-04-17T10:42:07.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fc57fa640 1 -- 172.21.15.83:0/2935729456 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0fe00db020 con 0x559351a800a0 2024-04-17T10:42:07.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fef8a8000 10 client.6051 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T10:42:07.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fef8a8000 10 client.6051 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:42:07.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fef8a8000 20 client.6051 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:07.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fef8a8000 1 --2- 172.21.15.83:0/2935729456 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x559351b2ff50 0x559351b32340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fef8a8000 4 client.6051 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T10:42:07.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fef8a8000 1 -- 172.21.15.83:0/2935729456 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5593518d4880 con 0x559351b2ff50 2024-04-17T10:42:07.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fe9cd6640 1 --2- 172.21.15.83:0/2935729456 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x559351b2ff50 0x559351b32340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fc57fa640 10 client.6051 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:07.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fe9cd6640 1 --2- 172.21.15.83:0/2935729456 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x559351b2ff50 0x559351b32340 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec9d4ad2e2c55ed6 server_cookie=5aa9e2a1a415cc6d in_seq=0 out_seq=0 2024-04-17T10:42:07.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fc57fa640 1 -- 172.21.15.83:0/2935729456 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5593518d4880 con 0x559351b2ff50 2024-04-17T10:42:07.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fc57fa640 10 client.6051 handle_command_reply: tid=0 2024-04-17T10:42:07.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fef8a8000 1 client.6051 shutdown 2024-04-17T10:42:07.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fef8a8000 1 -- 172.21.15.83:0/2935729456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559351a800a0 msgr2=0x559351adc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:07.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fef8a8000 1 --2- 172.21.15.83:0/2935729456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559351a800a0 0x559351adc770 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f0fe001a0c0 tx=0x7f0fe001acd0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fef8a8000 1 -- 172.21.15.83:0/2935729456 shutdown_connections 2024-04-17T10:42:07.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fef8a8000 1 --2- 172.21.15.83:0/2935729456 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x559351b2ff50 0x559351b32340 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fef8a8000 1 --2- 172.21.15.83:0/2935729456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559351aca2f0 0x559351add1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fc57fa640 0 client.6051 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:07.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fef8a8000 1 --2- 172.21.15.83:0/2935729456 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559351ac99d0 0x559351adccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fef8a8000 1 --2- 172.21.15.83:0/2935729456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559351a800a0 0x559351adc770 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fef8a8000 1 -- 172.21.15.83:0/2935729456 >> 172.21.15.83:0/2935729456 conn(0x559351a7c020 msgr2=0x559351ac8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:07.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fef8a8000 1 -- 172.21.15.83:0/2935729456 shutdown_connections 2024-04-17T10:42:07.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fef8a8000 1 -- 172.21.15.83:0/2935729456 wait complete. 2024-04-17T10:42:07.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fef8a8000 20 client.6051 trim_cache size 0 max 16384 2024-04-17T10:42:07.835 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:42:07.835 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-17T10:42:07.835 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-17T10:42:07.835 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:42:07.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fec73b640 1 -- 172.21.15.83:0/3945752470 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0fc8066f60 msgr2=0x7f0fc8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:07.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fec73b640 1 --2- 172.21.15.83:0/3945752470 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0fc8066f60 0x7f0fc8069420 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0fe00ad000 tx=0x7f0fe00a6000 comp rx=0 tx=0).stop 2024-04-17T10:42:07.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fec73b640 1 -- 172.21.15.83:0/3945752470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe415bc10 msgr2=0x7f0fe4155820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:07.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fec73b640 1 --2- 172.21.15.83:0/3945752470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe415bc10 0x7f0fe4155820 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc00cef0 tx=0x7f0fdc00a710 comp rx=0 tx=0).stop 2024-04-17T10:42:07.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fec73b640 1 -- 172.21.15.83:0/3945752470 shutdown_connections 2024-04-17T10:42:07.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fec73b640 1 --2- 172.21.15.83:0/3945752470 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0fc8066f60 0x7f0fc8069420 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fec73b640 1 --2- 172.21.15.83:0/3945752470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe415bc10 0x7f0fe4155820 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fec73b640 1 --2- 172.21.15.83:0/3945752470 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe40d0490 0x7f0fe41646d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fec73b640 1 --2- 172.21.15.83:0/3945752470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fe40cd3d0 0x7f0fe40c5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fec73b640 1 -- 172.21.15.83:0/3945752470 >> 172.21.15.83:0/3945752470 conn(0x7f0fe40d17e0 msgr2=0x7f0fe4156500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:07.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fec73b640 1 -- 172.21.15.83:0/3945752470 shutdown_connections 2024-04-17T10:42:07.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.830+0000 7f0fec73b640 1 -- 172.21.15.83:0/3945752470 wait complete. 2024-04-17T10:42:07.845 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T10:42:07.845 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-17T10:42:07.845 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-17T10:42:07.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.962+0000 7f83096ef640 1 Processor -- start 2024-04-17T10:42:07.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.962+0000 7f83096ef640 1 -- start start 2024-04-17T10:42:07.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.966+0000 7f83096ef640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f830415b6e0 0x7f8304071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.966+0000 7f83096ef640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f830415bac0 0x7f83040cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.966+0000 7f83096ef640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8304070470 0x7f830406ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.966+0000 7f83096ef640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f830406c1f0 con 0x7f830415b6e0 2024-04-17T10:42:07.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.966+0000 7f83096ef640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83040cdd30 con 0x7f830415bac0 2024-04-17T10:42:07.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.966+0000 7f83096ef640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8304070a10 con 0x7f8304070470 2024-04-17T10:42:07.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.966+0000 7f8302ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f830415b6e0 0x7f8304071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.966+0000 7f8302ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f830415b6e0 0x7f8304071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40926/0 (socket says 172.21.15.83:40926) 2024-04-17T10:42:07.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.966+0000 7f8302ffd640 1 -- 172.21.15.83:0/3176963344 learned_addr learned my addr 172.21.15.83:0/3176963344 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:07.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.966+0000 7f83027fc640 1 --2- 172.21.15.83:0/3176963344 >> v2:172.21.15.167:3300/0 conn(0x7f830415bac0 0x7f83040cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.966+0000 7f83037fe640 1 --2- 172.21.15.83:0/3176963344 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8304070470 0x7f830406ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.966+0000 7f8302ffd640 1 -- 172.21.15.83:0/3176963344 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8304070470 msgr2=0x7f830406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:07.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.966+0000 7f8302ffd640 1 --2- 172.21.15.83:0/3176963344 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8304070470 0x7f830406ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.966+0000 7f8302ffd640 1 -- 172.21.15.83:0/3176963344 >> v2:172.21.15.167:3300/0 conn(0x7f830415bac0 msgr2=0x7f83040cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:07.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.966+0000 7f8302ffd640 1 --2- 172.21.15.83:0/3176963344 >> v2:172.21.15.167:3300/0 conn(0x7f830415bac0 0x7f83040cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.966+0000 7f8302ffd640 1 -- 172.21.15.83:0/3176963344 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8304070cf0 con 0x7f830415b6e0 2024-04-17T10:42:07.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.970+0000 7f8302ffd640 1 --2- 172.21.15.83:0/3176963344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f830415b6e0 0x7f8304071cc0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f8304071c50 tx=0x7f82f401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b950f1e9fd3376eb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:07.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.970+0000 7f8301ffb640 1 -- 172.21.15.83:0/3176963344 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f40ab5f0 con 0x7f830415b6e0 2024-04-17T10:42:07.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.970+0000 7f8301ffb640 1 -- 172.21.15.83:0/3176963344 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f40b0070 con 0x7f830415b6e0 2024-04-17T10:42:07.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.970+0000 7f8301ffb640 1 -- 172.21.15.83:0/3176963344 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f40ab9b0 con 0x7f830415b6e0 2024-04-17T10:42:07.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.970+0000 7f83096ef640 1 -- 172.21.15.83:0/3176963344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f830415b6e0 msgr2=0x7f8304071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:07.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.970+0000 7f83096ef640 1 --2- 172.21.15.83:0/3176963344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f830415b6e0 0x7f8304071cc0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f8304071c50 tx=0x7f82f401c520 comp rx=0 tx=0).stop 2024-04-17T10:42:07.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.970+0000 7f83096ef640 1 -- 172.21.15.83:0/3176963344 shutdown_connections 2024-04-17T10:42:07.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.970+0000 7f83096ef640 1 --2- 172.21.15.83:0/3176963344 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8304070470 0x7f830406ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.970+0000 7f83096ef640 1 --2- 172.21.15.83:0/3176963344 >> v2:172.21.15.167:3300/0 conn(0x7f830415bac0 0x7f83040cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.970+0000 7f83096ef640 1 --2- 172.21.15.83:0/3176963344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f830415b6e0 0x7f8304071cc0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.970+0000 7f83096ef640 1 -- 172.21.15.83:0/3176963344 >> 172.21.15.83:0/3176963344 conn(0x7f83040b15b0 msgr2=0x7f83040b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:07.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.970+0000 7f83096ef640 1 -- 172.21.15.83:0/3176963344 shutdown_connections 2024-04-17T10:42:07.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.970+0000 7f83096ef640 1 -- 172.21.15.83:0/3176963344 wait complete. 2024-04-17T10:42:07.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.970+0000 7f83096ef640 1 Processor -- start 2024-04-17T10:42:07.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.970+0000 7f83096ef640 1 -- start start 2024-04-17T10:42:07.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.974+0000 7f83096ef640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8304070470 0x7f8304149150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.974+0000 7f83096ef640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f830415b6e0 0x7f830415ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.974+0000 7f83096ef640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f830415bac0 0x7f83041605c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.974+0000 7f83096ef640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830406d270 con 0x7f8304070470 2024-04-17T10:42:07.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.974+0000 7f83096ef640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830406d0f0 con 0x7f830415bac0 2024-04-17T10:42:07.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.974+0000 7f83096ef640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f830406b760 con 0x7f830415b6e0 2024-04-17T10:42:07.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.974+0000 7f8302ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8304070470 0x7f8304149150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.974+0000 7f8302ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8304070470 0x7f8304149150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40932/0 (socket says 172.21.15.83:40932) 2024-04-17T10:42:07.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.974+0000 7f8302ffd640 1 -- 172.21.15.83:0/2314270795 learned_addr learned my addr 172.21.15.83:0/2314270795 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:07.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.974+0000 7f83037fe640 1 --2- 172.21.15.83:0/2314270795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f830415bac0 0x7f83041605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.974+0000 7f83027fc640 1 --2- 172.21.15.83:0/2314270795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f830415b6e0 0x7f830415ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.974+0000 7f8302ffd640 1 -- 172.21.15.83:0/2314270795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f830415b6e0 msgr2=0x7f830415ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:07.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.974+0000 7f8302ffd640 1 --2- 172.21.15.83:0/2314270795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f830415b6e0 0x7f830415ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.974+0000 7f8302ffd640 1 -- 172.21.15.83:0/2314270795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f830415bac0 msgr2=0x7f83041605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:07.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.974+0000 7f8302ffd640 1 --2- 172.21.15.83:0/2314270795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f830415bac0 0x7f83041605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.974+0000 7f8302ffd640 1 -- 172.21.15.83:0/2314270795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f830415d300 con 0x7f8304070470 2024-04-17T10:42:07.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.974+0000 7f83027fc640 1 --2- 172.21.15.83:0/2314270795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f830415b6e0 0x7f830415ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:07.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.974+0000 7f8302ffd640 1 --2- 172.21.15.83:0/2314270795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8304070470 0x7f8304149150 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f82f40ad040 tx=0x7f82f401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:07.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.974+0000 7f82eb7fe640 1 -- 172.21.15.83:0/2314270795 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f40be3f0 con 0x7f8304070470 2024-04-17T10:42:07.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.974+0000 7f82eb7fe640 1 -- 172.21.15.83:0/2314270795 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f40be5b0 con 0x7f8304070470 2024-04-17T10:42:07.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.974+0000 7f83096ef640 1 -- 172.21.15.83:0/2314270795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f830415d5e0 con 0x7f8304070470 2024-04-17T10:42:07.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.974+0000 7f82eb7fe640 1 -- 172.21.15.83:0/2314270795 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f40be9a0 con 0x7f8304070470 2024-04-17T10:42:07.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.974+0000 7f83096ef640 1 -- 172.21.15.83:0/2314270795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83040ac530 con 0x7f8304070470 2024-04-17T10:42:07.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.974+0000 7f82eb7fe640 1 -- 172.21.15.83:0/2314270795 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f82f40b43f0 con 0x7f8304070470 2024-04-17T10:42:07.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.974+0000 7f82eb7fe640 1 --2- 172.21.15.83:0/2314270795 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f82f0066f60 0x7f82f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.974+0000 7f82eb7fe640 1 -- 172.21.15.83:0/2314270795 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f82f411f320 con 0x7f8304070470 2024-04-17T10:42:07.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.974+0000 7f83027fc640 1 --2- 172.21.15.83:0/2314270795 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f82f0066f60 0x7f82f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f83027fc640 1 --2- 172.21.15.83:0/2314270795 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f82f0066f60 0x7f82f0069420 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f83041f79c0 tx=0x7f82f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:07.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f830c85c000 1 Processor -- start 2024-04-17T10:42:07.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f830c85c000 1 -- start start 2024-04-17T10:42:07.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f830c85c000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562401e4a180 0x562401e4a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f830c85c000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562401e4aaa0 0x562401e000c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f830c85c000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562401e00600 0x562401e009e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f830c85c000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562401c4df20 con 0x562401e00600 2024-04-17T10:42:07.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f830c85c000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562401bd6000 con 0x562401e4aaa0 2024-04-17T10:42:07.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f830c85c000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x562401c49fc0 con 0x562401e4a180 2024-04-17T10:42:07.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f83037fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562401e00600 0x562401e009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f83037fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562401e00600 0x562401e009e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40940/0 (socket says 172.21.15.83:40940) 2024-04-17T10:42:07.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f83037fe640 1 -- 172.21.15.83:0/2270963478 learned_addr learned my addr 172.21.15.83:0/2270963478 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:07.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f83027fc640 1 --2- 172.21.15.83:0/2270963478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562401e4aaa0 0x562401e000c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f83037fe640 1 -- 172.21.15.83:0/2270963478 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562401e4a180 msgr2=0x562401e4a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:07.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f83037fe640 1 --2- 172.21.15.83:0/2270963478 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562401e4a180 0x562401e4a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f83037fe640 1 -- 172.21.15.83:0/2270963478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562401e4aaa0 msgr2=0x562401e000c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:07.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f83037fe640 1 --2- 172.21.15.83:0/2270963478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562401e4aaa0 0x562401e000c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f83037fe640 1 -- 172.21.15.83:0/2270963478 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562401c29ba0 con 0x562401e00600 2024-04-17T10:42:07.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f83037fe640 1 --2- 172.21.15.83:0/2270963478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562401e00600 0x562401e009e0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f82fc007680 tx=0x7f82fc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4371877a972624b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:07.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f82e8ff9640 1 -- 172.21.15.83:0/2270963478 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc019020 con 0x562401e00600 2024-04-17T10:42:07.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f82e8ff9640 1 -- 172.21.15.83:0/2270963478 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82fc015600 con 0x562401e00600 2024-04-17T10:42:07.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f82e8ff9640 1 -- 172.21.15.83:0/2270963478 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc0159d0 con 0x562401e00600 2024-04-17T10:42:07.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f830c85c000 1 -- 172.21.15.83:0/2270963478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562401e00600 msgr2=0x562401e009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:07.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f830c85c000 1 --2- 172.21.15.83:0/2270963478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562401e00600 0x562401e009e0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f82fc007680 tx=0x7f82fc009750 comp rx=0 tx=0).stop 2024-04-17T10:42:07.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f830c85c000 1 -- 172.21.15.83:0/2270963478 shutdown_connections 2024-04-17T10:42:07.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f830c85c000 1 --2- 172.21.15.83:0/2270963478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562401e00600 0x562401e009e0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f830c85c000 1 --2- 172.21.15.83:0/2270963478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562401e4aaa0 0x562401e000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f830c85c000 1 --2- 172.21.15.83:0/2270963478 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562401e4a180 0x562401e4a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f830c85c000 1 -- 172.21.15.83:0/2270963478 >> 172.21.15.83:0/2270963478 conn(0x562401e06760 msgr2=0x562401dfe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:07.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f830c85c000 1 -- 172.21.15.83:0/2270963478 shutdown_connections 2024-04-17T10:42:07.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f830c85c000 1 -- 172.21.15.83:0/2270963478 wait complete. 2024-04-17T10:42:07.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f830c85c000 1 -- start start 2024-04-17T10:42:07.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.978+0000 7f830c85c000 1 Processor -- start 2024-04-17T10:42:07.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f830c85c000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562401e00600 0x562401e056a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f830c85c000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562401e4a180 0x562401e05be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f830c85c000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562401e4aaa0 0x562401e38450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f830c85c000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562401d0b9c0 con 0x562401e4aaa0 2024-04-17T10:42:07.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f830c85c000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562401c085d0 con 0x562401e4a180 2024-04-17T10:42:07.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f830c85c000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x562401c0fee0 con 0x562401e00600 2024-04-17T10:42:07.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f830c85c000 10 client.0 _ll_register_callbacks cb 0x562401deba60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:07.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f83037fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562401e4aaa0 0x562401e38450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f83037fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562401e4aaa0 0x562401e38450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40956/0 (socket says 172.21.15.83:40956) 2024-04-17T10:42:07.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f83037fe640 1 -- 172.21.15.83:0/3387766969 learned_addr learned my addr 172.21.15.83:0/3387766969 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:07.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f83027fc640 1 --2- 172.21.15.83:0/3387766969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562401e4a180 0x562401e05be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f8302ffd640 1 --2- 172.21.15.83:0/3387766969 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562401e00600 0x562401e056a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f83037fe640 1 -- 172.21.15.83:0/3387766969 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562401e00600 msgr2=0x562401e056a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:07.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f83037fe640 1 --2- 172.21.15.83:0/3387766969 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562401e00600 0x562401e056a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f83037fe640 1 -- 172.21.15.83:0/3387766969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562401e4a180 msgr2=0x562401e05be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:07.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f83037fe640 1 --2- 172.21.15.83:0/3387766969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562401e4a180 0x562401e05be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f83037fe640 1 -- 172.21.15.83:0/3387766969 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562401c945a0 con 0x562401e4aaa0 2024-04-17T10:42:07.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f8302ffd640 1 --2- 172.21.15.83:0/3387766969 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562401e00600 0x562401e056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:07.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f83037fe640 1 --2- 172.21.15.83:0/3387766969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562401e4aaa0 0x562401e38450 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x562401c53f00 tx=0x7f82fc004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c1507f0a12fe33 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:07.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f82e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:07.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f82e1ffb640 1 -- 172.21.15.83:0/3387766969 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc01e830 con 0x562401e4aaa0 2024-04-17T10:42:07.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f82e1ffb640 1 -- 172.21.15.83:0/3387766969 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f82fc01e9d0 con 0x562401e4aaa0 2024-04-17T10:42:07.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f830c85c000 1 -- 172.21.15.83:0/3387766969 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82fc015600 con 0x562401e4aaa0 2024-04-17T10:42:07.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f82e1ffb640 1 -- 172.21.15.83:0/3387766969 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82fc031050 con 0x562401e4aaa0 2024-04-17T10:42:07.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f830c85c000 10 client.6063 fetch_fsmap learned FSMap version 22 2024-04-17T10:42:07.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f830c85c000 1 -- 172.21.15.83:0/3387766969 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x562401c252b0 con 0x562401e4aaa0 2024-04-17T10:42:07.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f82e1ffb640 1 -- 172.21.15.83:0/3387766969 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82fc015600 con 0x562401e4aaa0 2024-04-17T10:42:07.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f82e1ffb640 1 -- 172.21.15.83:0/3387766969 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f82fc034020 con 0x562401e4aaa0 2024-04-17T10:42:07.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f830c85c000 10 client.6063 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T10:42:07.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f830c85c000 10 client.6063 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:42:07.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f830c85c000 20 client.6063 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:07.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f830c85c000 1 --2- 172.21.15.83:0/3387766969 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562401e37e60 0x562401e38e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f830c85c000 4 client.6063 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:42:07.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f830c85c000 1 -- 172.21.15.83:0/3387766969 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562401be9c70 con 0x562401e37e60 2024-04-17T10:42:07.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f8302ffd640 1 --2- 172.21.15.83:0/3387766969 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562401e37e60 0x562401e38e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f82e1ffb640 10 client.6063 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:07.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f8302ffd640 1 --2- 172.21.15.83:0/3387766969 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562401e37e60 0x562401e38e60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91694f626c12baac server_cookie=4604d04d33fcebad in_seq=0 out_seq=0 2024-04-17T10:42:07.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f82e1ffb640 1 -- 172.21.15.83:0/3387766969 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562401be9c70 con 0x562401e37e60 2024-04-17T10:42:07.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f82e1ffb640 10 client.6063 handle_command_reply: tid=0 2024-04-17T10:42:07.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f830c85c000 1 client.6063 shutdown 2024-04-17T10:42:07.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f830c85c000 1 -- 172.21.15.83:0/3387766969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562401e4aaa0 msgr2=0x562401e38450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:07.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f830c85c000 1 --2- 172.21.15.83:0/3387766969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562401e4aaa0 0x562401e38450 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x562401c53f00 tx=0x7f82fc004ac0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f830c85c000 1 -- 172.21.15.83:0/3387766969 shutdown_connections 2024-04-17T10:42:07.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f830c85c000 1 --2- 172.21.15.83:0/3387766969 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562401e37e60 0x562401e38e60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f830c85c000 1 --2- 172.21.15.83:0/3387766969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562401e4aaa0 0x562401e38450 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f830c85c000 1 --2- 172.21.15.83:0/3387766969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562401e4a180 0x562401e05be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f830c85c000 1 --2- 172.21.15.83:0/3387766969 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562401e00600 0x562401e056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f82e1ffb640 0 client.6063 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:07.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f830c85c000 1 -- 172.21.15.83:0/3387766969 >> 172.21.15.83:0/3387766969 conn(0x562401e06760 msgr2=0x562401dff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:07.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f830c85c000 1 -- 172.21.15.83:0/3387766969 shutdown_connections 2024-04-17T10:42:07.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f830c85c000 1 -- 172.21.15.83:0/3387766969 wait complete. 2024-04-17T10:42:07.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.982+0000 7f830c85c000 20 client.6063 trim_cache size 0 max 16384 2024-04-17T10:42:07.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.994+0000 7f830c85c000 1 Processor -- start 2024-04-17T10:42:07.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.994+0000 7f830c85c000 1 -- start start 2024-04-17T10:42:07.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.994+0000 7f830c85c000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562401e06760 0x562401e60690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.994+0000 7f830c85c000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562401e00600 0x562401dff120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.994+0000 7f830c85c000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562401e4a180 0x562401dff660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:07.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.994+0000 7f830c85c000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8304200d30 con 0x562401e06760 2024-04-17T10:42:07.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.994+0000 7f830c85c000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83041ffee0 con 0x562401e4a180 2024-04-17T10:42:07.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.994+0000 7f830c85c000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83040ad5a0 con 0x562401e00600 2024-04-17T10:42:07.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.994+0000 7f8302ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562401e06760 0x562401e60690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:07.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.994+0000 7f8302ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562401e06760 0x562401e60690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40970/0 (socket says 172.21.15.83:40970) 2024-04-17T10:42:07.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.994+0000 7f8302ffd640 1 -- 172.21.15.83:0/92880791 learned_addr learned my addr 172.21.15.83:0/92880791 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:07.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f8302ffd640 1 -- 172.21.15.83:0/92880791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562401e00600 msgr2=0x562401dff120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:07.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f8302ffd640 1 --2- 172.21.15.83:0/92880791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562401e00600 0x562401dff120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f8302ffd640 1 -- 172.21.15.83:0/92880791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562401e4a180 msgr2=0x562401dff660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:07.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f8302ffd640 1 --2- 172.21.15.83:0/92880791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562401e4a180 0x562401dff660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f8302ffd640 1 -- 172.21.15.83:0/92880791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562401e391d0 con 0x562401e06760 2024-04-17T10:42:07.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f8302ffd640 1 --2- 172.21.15.83:0/92880791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562401e06760 0x562401e60690 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f82f401ca00 tx=0x7f82f411f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673c504c3487cc1b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:07.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f82e3fff640 1 -- 172.21.15.83:0/92880791 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4118070 con 0x562401e06760 2024-04-17T10:42:07.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f82e3fff640 1 -- 172.21.15.83:0/92880791 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f40b0040 con 0x562401e06760 2024-04-17T10:42:07.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f82e3fff640 1 -- 172.21.15.83:0/92880791 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f411ca90 con 0x562401e06760 2024-04-17T10:42:07.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f830c85c000 1 -- 172.21.15.83:0/92880791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562401e06760 msgr2=0x562401e60690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:07.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f830c85c000 1 --2- 172.21.15.83:0/92880791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562401e06760 0x562401e60690 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f82f401ca00 tx=0x7f82f411f040 comp rx=0 tx=0).stop 2024-04-17T10:42:07.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f830c85c000 1 -- 172.21.15.83:0/92880791 shutdown_connections 2024-04-17T10:42:07.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f830c85c000 1 --2- 172.21.15.83:0/92880791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562401e4a180 0x562401dff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f830c85c000 1 --2- 172.21.15.83:0/92880791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562401e00600 0x562401dff120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f830c85c000 1 --2- 172.21.15.83:0/92880791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562401e06760 0x562401e60690 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:07.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f830c85c000 1 -- 172.21.15.83:0/92880791 >> 172.21.15.83:0/92880791 conn(0x562401e385b0 msgr2=0x562401e57770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:07.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f830c85c000 1 -- 172.21.15.83:0/92880791 shutdown_connections 2024-04-17T10:42:07.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f830c85c000 1 -- 172.21.15.83:0/92880791 wait complete. 2024-04-17T10:42:07.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f830c85c000 1 -- start start 2024-04-17T10:42:07.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f830c85c000 1 Processor -- start 2024-04-17T10:42:08.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f830c85c000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562401e00600 0x562401eaa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f830c85c000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562401e06760 0x562401eaad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f830c85c000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562401e4a180 0x562401e5ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f830c85c000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562401d039d0 con 0x562401e00600 2024-04-17T10:42:08.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f830c85c000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83040adba0 con 0x562401e06760 2024-04-17T10:42:08.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f830c85c000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x562401e096f0 con 0x562401e4a180 2024-04-17T10:42:08.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f830c85c000 10 client.0 _ll_register_callbacks cb 0x562401dcc600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:08.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f8302ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562401e00600 0x562401eaa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f8302ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562401e00600 0x562401eaa7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40980/0 (socket says 172.21.15.83:40980) 2024-04-17T10:42:08.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f8302ffd640 1 -- 172.21.15.83:0/1069605556 learned_addr learned my addr 172.21.15.83:0/1069605556 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:08.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f83027fc640 1 --2- 172.21.15.83:0/1069605556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562401e06760 0x562401eaad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f8302ffd640 1 -- 172.21.15.83:0/1069605556 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562401e4a180 msgr2=0x562401e5ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:08.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f8302ffd640 1 --2- 172.21.15.83:0/1069605556 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562401e4a180 0x562401e5ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f8302ffd640 1 -- 172.21.15.83:0/1069605556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562401e06760 msgr2=0x562401eaad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:08.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f8302ffd640 1 --2- 172.21.15.83:0/1069605556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562401e06760 0x562401eaad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f8302ffd640 1 -- 172.21.15.83:0/1069605556 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562401bd6250 con 0x562401e00600 2024-04-17T10:42:08.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f83027fc640 1 --2- 172.21.15.83:0/1069605556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562401e06760 0x562401eaad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:08.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f8302ffd640 1 --2- 172.21.15.83:0/1069605556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562401e00600 0x562401eaa7c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f82f401ca90 tx=0x7f82f40f6040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27e5c89e1e4743c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:08.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f82e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:08.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f82e1ffb640 1 -- 172.21.15.83:0/1069605556 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f40fc030 con 0x562401e00600 2024-04-17T10:42:08.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f82e1ffb640 1 -- 172.21.15.83:0/1069605556 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f82f40eddb0 con 0x562401e00600 2024-04-17T10:42:08.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f830c85c000 1 -- 172.21.15.83:0/1069605556 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82f40b0040 con 0x562401e00600 2024-04-17T10:42:08.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f82e1ffb640 1 -- 172.21.15.83:0/1069605556 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82f4118050 con 0x562401e00600 2024-04-17T10:42:08.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f830c85c000 10 client.6069 fetch_fsmap learned FSMap version 22 2024-04-17T10:42:08.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f830c85c000 1 -- 172.21.15.83:0/1069605556 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x562401be2b90 con 0x562401e00600 2024-04-17T10:42:08.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f82e1ffb640 1 -- 172.21.15.83:0/1069605556 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f40b0040 con 0x562401e00600 2024-04-17T10:42:08.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f82e1ffb640 1 -- 172.21.15.83:0/1069605556 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f82f40ec380 con 0x562401e00600 2024-04-17T10:42:08.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f830c85c000 10 client.6069 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T10:42:08.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f830c85c000 10 client.6069 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:42:08.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f830c85c000 20 client.6069 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:08.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f830c85c000 1 --2- 172.21.15.83:0/1069605556 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562401e4aaa0 0x562401e5f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f830c85c000 4 client.6069 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T10:42:08.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f830c85c000 1 -- 172.21.15.83:0/1069605556 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562401c5b460 con 0x562401e4aaa0 2024-04-17T10:42:08.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:07.998+0000 7f83027fc640 1 --2- 172.21.15.83:0/1069605556 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562401e4aaa0 0x562401e5f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.002+0000 7f82e1ffb640 10 client.6069 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:08.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.002+0000 7f83027fc640 1 --2- 172.21.15.83:0/1069605556 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562401e4aaa0 0x562401e5f630 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e9f20f0c7dd46ea server_cookie=936abc9cbb613f24 in_seq=0 out_seq=0 2024-04-17T10:42:08.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.002+0000 7f82e1ffb640 1 -- 172.21.15.83:0/1069605556 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+168 (crc 0 0 0) 0x562401c5b460 con 0x562401e4aaa0 2024-04-17T10:42:08.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.002+0000 7f82e1ffb640 10 client.6069 handle_command_reply: tid=0 2024-04-17T10:42:08.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.002+0000 7f830c85c000 1 client.6069 shutdown 2024-04-17T10:42:08.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.002+0000 7f830c85c000 1 -- 172.21.15.83:0/1069605556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562401e00600 msgr2=0x562401eaa7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:08.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.002+0000 7f830c85c000 1 --2- 172.21.15.83:0/1069605556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562401e00600 0x562401eaa7c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f82f401ca90 tx=0x7f82f40f6040 comp rx=0 tx=0).stop 2024-04-17T10:42:08.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.002+0000 7f830c85c000 1 -- 172.21.15.83:0/1069605556 shutdown_connections 2024-04-17T10:42:08.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.002+0000 7f830c85c000 1 --2- 172.21.15.83:0/1069605556 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562401e4aaa0 0x562401e5f630 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.002+0000 7f830c85c000 1 --2- 172.21.15.83:0/1069605556 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562401e4a180 0x562401e5ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.002+0000 7f830c85c000 1 --2- 172.21.15.83:0/1069605556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562401e06760 0x562401eaad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.002+0000 7f82e1ffb640 0 client.6069 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:08.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.002+0000 7f830c85c000 1 --2- 172.21.15.83:0/1069605556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562401e00600 0x562401eaa7c0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.002+0000 7f830c85c000 1 -- 172.21.15.83:0/1069605556 >> 172.21.15.83:0/1069605556 conn(0x562401e385b0 msgr2=0x562401dfe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:08.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.002+0000 7f830c85c000 1 -- 172.21.15.83:0/1069605556 shutdown_connections 2024-04-17T10:42:08.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.002+0000 7f830c85c000 1 -- 172.21.15.83:0/1069605556 wait complete. 2024-04-17T10:42:08.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.002+0000 7f830c85c000 20 client.6069 trim_cache size 0 max 16384 2024-04-17T10:42:08.006 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-04-17T10:42:08.006 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:42:08.006 INFO:teuthology.orchestra.run.smithi083.stdout: "damage_type": "backtrace", 2024-04-17T10:42:08.006 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 2789176921, 2024-04-17T10:42:08.006 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 1099511630409, 2024-04-17T10:42:08.007 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_0" 2024-04-17T10:42:08.007 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:42:08.007 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-04-17T10:42:08.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.002+0000 7f83096ef640 1 -- 172.21.15.83:0/2314270795 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f82f0066f60 msgr2=0x7f82f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:08.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.002+0000 7f83096ef640 1 --2- 172.21.15.83:0/2314270795 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f82f0066f60 0x7f82f0069420 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f83041f79c0 tx=0x7f82f8008040 comp rx=0 tx=0).stop 2024-04-17T10:42:08.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.002+0000 7f83096ef640 1 -- 172.21.15.83:0/2314270795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8304070470 msgr2=0x7f8304149150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:08.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.002+0000 7f83096ef640 1 --2- 172.21.15.83:0/2314270795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8304070470 0x7f8304149150 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f82f40ad040 tx=0x7f82f401ad60 comp rx=0 tx=0).stop 2024-04-17T10:42:08.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.002+0000 7f83096ef640 1 -- 172.21.15.83:0/2314270795 shutdown_connections 2024-04-17T10:42:08.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.002+0000 7f83096ef640 1 --2- 172.21.15.83:0/2314270795 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f82f0066f60 0x7f82f0069420 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.002+0000 7f83096ef640 1 --2- 172.21.15.83:0/2314270795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f830415bac0 0x7f83041605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.002+0000 7f83096ef640 1 --2- 172.21.15.83:0/2314270795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f830415b6e0 0x7f830415ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.002+0000 7f83096ef640 1 --2- 172.21.15.83:0/2314270795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8304070470 0x7f8304149150 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.002+0000 7f83096ef640 1 -- 172.21.15.83:0/2314270795 >> 172.21.15.83:0/2314270795 conn(0x7f83040b15b0 msgr2=0x7f83040b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:08.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.002+0000 7f83096ef640 1 -- 172.21.15.83:0/2314270795 shutdown_connections 2024-04-17T10:42:08.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.002+0000 7f83096ef640 1 -- 172.21.15.83:0/2314270795 wait complete. 2024-04-17T10:42:08.016 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:42:08.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.122+0000 7fe8a31ac640 1 Processor -- start 2024-04-17T10:42:08.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.122+0000 7fe8a31ac640 1 -- start start 2024-04-17T10:42:08.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.126+0000 7fe8a31ac640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe89c15ba30 0x7fe89c072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.126+0000 7fe8a31ac640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe89c15c960 0x7fe89c0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.126+0000 7fe8a31ac640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe89c0cfa10 0x7fe89c06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.126+0000 7fe8a31ac640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe89c071ea0 con 0x7fe89c15c960 2024-04-17T10:42:08.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.126+0000 7fe8a31ac640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe89c0cfea0 con 0x7fe89c0cfa10 2024-04-17T10:42:08.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.126+0000 7fe8a31ac640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c06e320 con 0x7fe89c15ba30 2024-04-17T10:42:08.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.126+0000 7fe89bfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe89c15c960 0x7fe89c0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.126+0000 7fe89bfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe89c15c960 0x7fe89c0cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40992/0 (socket says 172.21.15.83:40992) 2024-04-17T10:42:08.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.126+0000 7fe89bfff640 1 -- 172.21.15.83:0/1623919153 learned_addr learned my addr 172.21.15.83:0/1623919153 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:08.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.126+0000 7fe89bfff640 1 -- 172.21.15.83:0/1623919153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe89c15ba30 msgr2=0x7fe89c072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:08.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.126+0000 7fe8a0f48640 1 --2- 172.21.15.83:0/1623919153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe89c15ba30 0x7fe89c072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.126+0000 7fe89bfff640 1 --2- 172.21.15.83:0/1623919153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe89c15ba30 0x7fe89c072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.126+0000 7fe89bfff640 1 -- 172.21.15.83:0/1623919153 >> v2:172.21.15.167:3300/0 conn(0x7fe89c0cfa10 msgr2=0x7fe89c06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:08.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.126+0000 7fe8a1749640 1 --2- 172.21.15.83:0/1623919153 >> v2:172.21.15.167:3300/0 conn(0x7fe89c0cfa10 0x7fe89c06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.126+0000 7fe89bfff640 1 --2- 172.21.15.83:0/1623919153 >> v2:172.21.15.167:3300/0 conn(0x7fe89c0cfa10 0x7fe89c06c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.126+0000 7fe89bfff640 1 -- 172.21.15.83:0/1623919153 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89c06e040 con 0x7fe89c15c960 2024-04-17T10:42:08.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.126+0000 7fe8a1749640 1 --2- 172.21.15.83:0/1623919153 >> v2:172.21.15.167:3300/0 conn(0x7fe89c0cfa10 0x7fe89c06c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:08.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.126+0000 7fe89bfff640 1 --2- 172.21.15.83:0/1623919153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe89c15c960 0x7fe89c0cf4d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe89c06d810 tx=0x7fe88801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acbe7b42e825a8c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:08.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.126+0000 7fe89b7fe640 1 -- 172.21.15.83:0/1623919153 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8880b0070 con 0x7fe89c15c960 2024-04-17T10:42:08.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.126+0000 7fe89b7fe640 1 -- 172.21.15.83:0/1623919153 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8880ab600 con 0x7fe89c15c960 2024-04-17T10:42:08.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.126+0000 7fe89b7fe640 1 -- 172.21.15.83:0/1623919153 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8880ab9d0 con 0x7fe89c15c960 2024-04-17T10:42:08.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.126+0000 7fe8a31ac640 1 -- 172.21.15.83:0/1623919153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe89c15c960 msgr2=0x7fe89c0cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:08.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.126+0000 7fe8a31ac640 1 --2- 172.21.15.83:0/1623919153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe89c15c960 0x7fe89c0cf4d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe89c06d810 tx=0x7fe88801c490 comp rx=0 tx=0).stop 2024-04-17T10:42:08.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.126+0000 7fe8a31ac640 1 -- 172.21.15.83:0/1623919153 shutdown_connections 2024-04-17T10:42:08.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.126+0000 7fe8a31ac640 1 --2- 172.21.15.83:0/1623919153 >> v2:172.21.15.167:3300/0 conn(0x7fe89c0cfa10 0x7fe89c06c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.126+0000 7fe8a31ac640 1 --2- 172.21.15.83:0/1623919153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe89c15c960 0x7fe89c0cf4d0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.126+0000 7fe8a31ac640 1 --2- 172.21.15.83:0/1623919153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe89c15ba30 0x7fe89c072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.126+0000 7fe8a31ac640 1 -- 172.21.15.83:0/1623919153 >> 172.21.15.83:0/1623919153 conn(0x7fe89c06abe0 msgr2=0x7fe89c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:08.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.126+0000 7fe8a31ac640 1 -- 172.21.15.83:0/1623919153 shutdown_connections 2024-04-17T10:42:08.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.126+0000 7fe8a31ac640 1 -- 172.21.15.83:0/1623919153 wait complete. 2024-04-17T10:42:08.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.130+0000 7fe8a31ac640 1 Processor -- start 2024-04-17T10:42:08.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.130+0000 7fe8a31ac640 1 -- start start 2024-04-17T10:42:08.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.130+0000 7fe8a31ac640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe89c0cfa10 0x7fe89c14de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.130+0000 7fe8a31ac640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe89c15ba30 0x7fe89c1f6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.130+0000 7fe8a31ac640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe89c15c960 0x7fe89c0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.130+0000 7fe8a31ac640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c06dca0 con 0x7fe89c15ba30 2024-04-17T10:42:08.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.130+0000 7fe8a31ac640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c0aeae0 con 0x7fe89c0cfa10 2024-04-17T10:42:08.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.130+0000 7fe8a31ac640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c06de20 con 0x7fe89c15c960 2024-04-17T10:42:08.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.130+0000 7fe8a0f48640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe89c0cfa10 0x7fe89c14de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.130+0000 7fe89bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe89c15ba30 0x7fe89c1f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.130+0000 7fe8a1749640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe89c15c960 0x7fe89c0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.130+0000 7fe89bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe89c15ba30 0x7fe89c1f6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40994/0 (socket says 172.21.15.83:40994) 2024-04-17T10:42:08.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.130+0000 7fe89bfff640 1 -- 172.21.15.83:0/617656094 learned_addr learned my addr 172.21.15.83:0/617656094 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:08.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.130+0000 7fe89bfff640 1 -- 172.21.15.83:0/617656094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe89c15c960 msgr2=0x7fe89c0ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:08.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.130+0000 7fe89bfff640 1 --2- 172.21.15.83:0/617656094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe89c15c960 0x7fe89c0ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.130+0000 7fe89bfff640 1 -- 172.21.15.83:0/617656094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe89c0cfa10 msgr2=0x7fe89c14de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:08.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.130+0000 7fe89bfff640 1 --2- 172.21.15.83:0/617656094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe89c0cfa10 0x7fe89c14de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.130+0000 7fe89bfff640 1 -- 172.21.15.83:0/617656094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89c0b2820 con 0x7fe89c15ba30 2024-04-17T10:42:08.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.130+0000 7fe89bfff640 1 --2- 172.21.15.83:0/617656094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe89c15ba30 0x7fe89c1f6580 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fe88801ab40 tx=0x7fe888019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:08.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.130+0000 7fe8997fa640 1 -- 172.21.15.83:0/617656094 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8880b9070 con 0x7fe89c15ba30 2024-04-17T10:42:08.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.130+0000 7fe8997fa640 1 -- 172.21.15.83:0/617656094 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8880b4620 con 0x7fe89c15ba30 2024-04-17T10:42:08.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.130+0000 7fe8a31ac640 1 -- 172.21.15.83:0/617656094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe89c0b2ac0 con 0x7fe89c15ba30 2024-04-17T10:42:08.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.130+0000 7fe8997fa640 1 -- 172.21.15.83:0/617656094 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8880b0040 con 0x7fe89c15ba30 2024-04-17T10:42:08.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.130+0000 7fe8a31ac640 1 -- 172.21.15.83:0/617656094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe89c0b3080 con 0x7fe89c15ba30 2024-04-17T10:42:08.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.130+0000 7fe8a31ac640 1 -- 172.21.15.83:0/617656094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe89c074f90 con 0x7fe89c15ba30 2024-04-17T10:42:08.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.134+0000 7fe8997fa640 1 -- 172.21.15.83:0/617656094 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe8880bc070 con 0x7fe89c15ba30 2024-04-17T10:42:08.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.134+0000 7fe8997fa640 1 --2- 172.21.15.83:0/617656094 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe880066f60 0x7fe880069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.134+0000 7fe8997fa640 1 -- 172.21.15.83:0/617656094 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe88811e3d0 con 0x7fe89c15ba30 2024-04-17T10:42:08.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.134+0000 7fe8a0f48640 1 --2- 172.21.15.83:0/617656094 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe880066f60 0x7fe880069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.134+0000 7fe8a0f48640 1 --2- 172.21.15.83:0/617656094 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe880066f60 0x7fe880069420 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe89c155960 tx=0x7fe890008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:08.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.138+0000 7fe8997fa640 1 -- 172.21.15.83:0/617656094 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe888016110 con 0x7fe89c15ba30 2024-04-17T10:42:08.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.330+0000 7fe8a31ac640 1 -- 172.21.15.83:0/617656094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe89c0aeeb0 con 0x7fe89c15ba30 2024-04-17T10:42:08.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.334+0000 7fe8997fa640 1 -- 172.21.15.83:0/617656094 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4939 (secure 0 0 0) 0x7fe8880eede0 con 0x7fe89c15ba30 2024-04-17T10:42:08.331 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:08.332 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/161398126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":161398126},{"type":"v1","addr":"172.21.15.167:6833","nonce":161398126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1085380618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1085380618},{"type":"v1","addr":"172.21.15.167:6835","nonce":1085380618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/836442291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":836442291},{"type":"v1","addr":"172.21.15.83:6835","nonce":836442291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:41:37.773342+0000","modified":"2024-04-17T10:42:06.483516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5682":{"gid":5682,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.83:6837/1894547386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1894547386},{"type":"v1","addr":"172.21.15.83:6837","nonce":1894547386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-17T10:42:08.332 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 22 2024-04-17T10:42:08.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.334+0000 7fe8a31ac640 1 -- 172.21.15.83:0/617656094 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe880066f60 msgr2=0x7fe880069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:08.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.334+0000 7fe8a31ac640 1 --2- 172.21.15.83:0/617656094 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe880066f60 0x7fe880069420 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe89c155960 tx=0x7fe890008040 comp rx=0 tx=0).stop 2024-04-17T10:42:08.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.334+0000 7fe8a31ac640 1 -- 172.21.15.83:0/617656094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe89c15ba30 msgr2=0x7fe89c1f6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:08.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.334+0000 7fe8a31ac640 1 --2- 172.21.15.83:0/617656094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe89c15ba30 0x7fe89c1f6580 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fe88801ab40 tx=0x7fe888019230 comp rx=0 tx=0).stop 2024-04-17T10:42:08.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.334+0000 7fe8a31ac640 1 -- 172.21.15.83:0/617656094 shutdown_connections 2024-04-17T10:42:08.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.334+0000 7fe8a31ac640 1 --2- 172.21.15.83:0/617656094 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe880066f60 0x7fe880069420 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.334+0000 7fe8a31ac640 1 --2- 172.21.15.83:0/617656094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe89c15c960 0x7fe89c0ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.334+0000 7fe8a31ac640 1 --2- 172.21.15.83:0/617656094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe89c15ba30 0x7fe89c1f6580 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.334+0000 7fe8a31ac640 1 --2- 172.21.15.83:0/617656094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe89c0cfa10 0x7fe89c14de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.334+0000 7fe8a31ac640 1 -- 172.21.15.83:0/617656094 >> 172.21.15.83:0/617656094 conn(0x7fe89c06abe0 msgr2=0x7fe89c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:08.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.334+0000 7fe8a31ac640 1 -- 172.21.15.83:0/617656094 shutdown_connections 2024-04-17T10:42:08.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.334+0000 7fe8a31ac640 1 -- 172.21.15.83:0/617656094 wait complete. 2024-04-17T10:42:08.348 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage rm 2789176921 2024-04-17T10:42:08.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.478+0000 7f23c064c640 1 Processor -- start 2024-04-17T10:42:08.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.478+0000 7f23c064c640 1 -- start start 2024-04-17T10:42:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.482+0000 7f23c064c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23b80ce910 0x7f23b80ae490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.482+0000 7f23c064c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f23b80b2e30 0x7f23b80d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.482+0000 7f23c064c640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b806c150 con 0x7f23b80ae9d0 2024-04-17T10:42:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.482+0000 7f23c064c640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b80d1020 con 0x7f23b80b2e30 2024-04-17T10:42:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.482+0000 7f23c064c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b806bf00 con 0x7f23b80ce910 2024-04-17T10:42:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.482+0000 7f23bebe9640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f23b80b2e30 0x7f23b80d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.482+0000 7f23bdbe7640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f23b80ae9d0 0x7f23b80b2720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46706/0 (socket says 172.21.15.83:46706) 2024-04-17T10:42:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.482+0000 7f23bdbe7640 1 -- 172.21.15.83:0/3155948113 learned_addr learned my addr 172.21.15.83:0/3155948113 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.482+0000 7f23bebe9640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f23b80b2e30 0x7f23b80d0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41000/0 (socket says 172.21.15.83:41000) 2024-04-17T10:42:08.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.482+0000 7f23be3e8640 1 --2- 172.21.15.83:0/3155948113 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23b80ce910 0x7f23b80ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.482+0000 7f23bebe9640 1 -- 172.21.15.83:0/3155948113 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23b80ce910 msgr2=0x7f23b80ae490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:08.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.482+0000 7f23bebe9640 1 --2- 172.21.15.83:0/3155948113 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23b80ce910 0x7f23b80ae490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.482+0000 7f23bebe9640 1 -- 172.21.15.83:0/3155948113 >> v1:172.21.15.83:6789/0 conn(0x7f23b80ae9d0 legacy=0x7f23b80b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:08.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.482+0000 7f23bd3e6640 1 -- 172.21.15.83:0/3155948113 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2684471599 0 0) 0x7f23b806c150 con 0x7f23b80ae9d0 2024-04-17T10:42:08.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.482+0000 7f23bebe9640 1 -- 172.21.15.83:0/3155948113 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b80b1c40 con 0x7f23b80b2e30 2024-04-17T10:42:08.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.482+0000 7f23bebe9640 1 --2- 172.21.15.83:0/3155948113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23b80b2e30 0x7f23b80d0ae0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f23b0009d90 tx=0x7f23b001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93aeb2b51625a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:08.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.482+0000 7f23bd3e6640 1 -- 172.21.15.83:0/3155948113 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b00b2070 con 0x7f23b80b2e30 2024-04-17T10:42:08.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.482+0000 7f23bd3e6640 1 -- 172.21.15.83:0/3155948113 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b00a57a0 con 0x7f23b80b2e30 2024-04-17T10:42:08.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.482+0000 7f23bd3e6640 1 -- 172.21.15.83:0/3155948113 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b00a5b70 con 0x7f23b80b2e30 2024-04-17T10:42:08.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23c064c640 1 -- 172.21.15.83:0/3155948113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23b80b2e30 msgr2=0x7f23b80d0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:08.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23c064c640 1 --2- 172.21.15.83:0/3155948113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23b80b2e30 0x7f23b80d0ae0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f23b0009d90 tx=0x7f23b001c8e0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23c064c640 1 -- 172.21.15.83:0/3155948113 shutdown_connections 2024-04-17T10:42:08.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23c064c640 1 --2- 172.21.15.83:0/3155948113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23b80b2e30 0x7f23b80d0ae0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23c064c640 1 --2- 172.21.15.83:0/3155948113 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23b80ce910 0x7f23b80ae490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23c064c640 1 -- 172.21.15.83:0/3155948113 >> 172.21.15.83:0/3155948113 conn(0x7f23b80d17a0 msgr2=0x7f23b80d1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:08.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23c064c640 1 -- 172.21.15.83:0/3155948113 shutdown_connections 2024-04-17T10:42:08.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23c064c640 1 -- 172.21.15.83:0/3155948113 wait complete. 2024-04-17T10:42:08.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23c064c640 1 Processor -- start 2024-04-17T10:42:08.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23c064c640 1 -- start start 2024-04-17T10:42:08.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23c064c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23b80ae9d0 0x7f23b80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23c064c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23b80b2e30 0x7f23b8151a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23c064c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23b80ce910 0x7f23b815c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23c064c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b815d820 con 0x7f23b80ce910 2024-04-17T10:42:08.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23c064c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b815d6a0 con 0x7f23b80ae9d0 2024-04-17T10:42:08.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23c064c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b815d9a0 con 0x7f23b80b2e30 2024-04-17T10:42:08.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23bebe9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23b80ce910 0x7f23b815c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23bebe9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23b80ce910 0x7f23b815c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41012/0 (socket says 172.21.15.83:41012) 2024-04-17T10:42:08.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23bebe9640 1 -- 172.21.15.83:0/1824905403 learned_addr learned my addr 172.21.15.83:0/1824905403 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:08.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23bdbe7640 1 --2- 172.21.15.83:0/1824905403 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23b80b2e30 0x7f23b8151a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23be3e8640 1 --2- 172.21.15.83:0/1824905403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23b80ae9d0 0x7f23b80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23bebe9640 1 -- 172.21.15.83:0/1824905403 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23b80b2e30 msgr2=0x7f23b8151a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:08.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23bebe9640 1 --2- 172.21.15.83:0/1824905403 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23b80b2e30 0x7f23b8151a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23bebe9640 1 -- 172.21.15.83:0/1824905403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23b80ae9d0 msgr2=0x7f23b80b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:08.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23bebe9640 1 --2- 172.21.15.83:0/1824905403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23b80ae9d0 0x7f23b80b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23bebe9640 1 -- 172.21.15.83:0/1824905403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b80b62b0 con 0x7f23b80ce910 2024-04-17T10:42:08.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23bdbe7640 1 --2- 172.21.15.83:0/1824905403 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23b80b2e30 0x7f23b8151a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:08.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23be3e8640 1 --2- 172.21.15.83:0/1824905403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23b80ae9d0 0x7f23b80b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:08.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23bebe9640 1 --2- 172.21.15.83:0/1824905403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23b80ce910 0x7f23b815c490 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f23b001a510 tx=0x7f23b0019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:08.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23aeffd640 1 -- 172.21.15.83:0/1824905403 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b00b2050 con 0x7f23b80ce910 2024-04-17T10:42:08.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23aeffd640 1 -- 172.21.15.83:0/1824905403 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b00b3040 con 0x7f23b80ce910 2024-04-17T10:42:08.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23c064c640 1 -- 172.21.15.83:0/1824905403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b80c6d70 con 0x7f23b80ce910 2024-04-17T10:42:08.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23aeffd640 1 -- 172.21.15.83:0/1824905403 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0009450 con 0x7f23b80ce910 2024-04-17T10:42:08.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.486+0000 7f23c064c640 1 -- 172.21.15.83:0/1824905403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b80c7330 con 0x7f23b80ce910 2024-04-17T10:42:08.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.490+0000 7f23aeffd640 1 -- 172.21.15.83:0/1824905403 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f23b00c1060 con 0x7f23b80ce910 2024-04-17T10:42:08.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.490+0000 7f23aeffd640 1 --2- 172.21.15.83:0/1824905403 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f239c066f60 0x7f239c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.490+0000 7f23aeffd640 1 -- 172.21.15.83:0/1824905403 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f23b011f5a0 con 0x7f23b80ce910 2024-04-17T10:42:08.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.490+0000 7f23be3e8640 1 --2- 172.21.15.83:0/1824905403 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f239c066f60 0x7f239c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.490+0000 7f23c37b9000 1 Processor -- start 2024-04-17T10:42:08.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.490+0000 7f23c37b9000 1 -- start start 2024-04-17T10:42:08.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.490+0000 7f23be3e8640 1 --2- 172.21.15.83:0/1824905403 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f239c066f60 0x7f239c069420 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f23a80097a0 tx=0x7f23a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:08.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.490+0000 7f23c37b9000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562567a94070 0x562567a94450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.490+0000 7f23c37b9000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562567a94990 0x562567a476a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.490+0000 7f23c37b9000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562567a47be0 0x562567a47fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.490+0000 7f23c37b9000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562567897f00 con 0x562567a47be0 2024-04-17T10:42:08.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.490+0000 7f23c37b9000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625678204a0 con 0x562567a94990 2024-04-17T10:42:08.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.490+0000 7f23c37b9000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x562567893fa0 con 0x562567a94070 2024-04-17T10:42:08.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.490+0000 7f23bebe9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562567a47be0 0x562567a47fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.490+0000 7f23bebe9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562567a47be0 0x562567a47fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41022/0 (socket says 172.21.15.83:41022) 2024-04-17T10:42:08.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.490+0000 7f23bebe9640 1 -- 172.21.15.83:0/2187257775 learned_addr learned my addr 172.21.15.83:0/2187257775 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:08.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.490+0000 7f23bdbe7640 1 --2- 172.21.15.83:0/2187257775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562567a94990 0x562567a476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.490+0000 7f23be3e8640 1 --2- 172.21.15.83:0/2187257775 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562567a94070 0x562567a94450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.490+0000 7f23bebe9640 1 -- 172.21.15.83:0/2187257775 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562567a94070 msgr2=0x562567a94450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:08.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.490+0000 7f23bebe9640 1 --2- 172.21.15.83:0/2187257775 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562567a94070 0x562567a94450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.490+0000 7f23bebe9640 1 -- 172.21.15.83:0/2187257775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562567a94990 msgr2=0x562567a476a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:08.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.490+0000 7f23bebe9640 1 --2- 172.21.15.83:0/2187257775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562567a94990 0x562567a476a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.490+0000 7f23bebe9640 1 -- 172.21.15.83:0/2187257775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56256781dd00 con 0x562567a47be0 2024-04-17T10:42:08.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.490+0000 7f23be3e8640 1 --2- 172.21.15.83:0/2187257775 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562567a94070 0x562567a94450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:08.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.490+0000 7f23bebe9640 1 --2- 172.21.15.83:0/2187257775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562567a47be0 0x562567a47fc0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f23b00b0040 tx=0x7f23b00f04e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=329ec3875ff392e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:08.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.490+0000 7f239bfff640 1 -- 172.21.15.83:0/2187257775 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0107030 con 0x562567a47be0 2024-04-17T10:42:08.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.490+0000 7f239bfff640 1 -- 172.21.15.83:0/2187257775 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b00f3070 con 0x562567a47be0 2024-04-17T10:42:08.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.490+0000 7f239bfff640 1 -- 172.21.15.83:0/2187257775 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b00f8580 con 0x562567a47be0 2024-04-17T10:42:08.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23c37b9000 1 -- 172.21.15.83:0/2187257775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562567a47be0 msgr2=0x562567a47fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:08.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23c37b9000 1 --2- 172.21.15.83:0/2187257775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562567a47be0 0x562567a47fc0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f23b00b0040 tx=0x7f23b00f04e0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23c37b9000 1 -- 172.21.15.83:0/2187257775 shutdown_connections 2024-04-17T10:42:08.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23c37b9000 1 --2- 172.21.15.83:0/2187257775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562567a47be0 0x562567a47fc0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23c37b9000 1 --2- 172.21.15.83:0/2187257775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562567a94990 0x562567a476a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23c37b9000 1 --2- 172.21.15.83:0/2187257775 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562567a94070 0x562567a94450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23c37b9000 1 -- 172.21.15.83:0/2187257775 >> 172.21.15.83:0/2187257775 conn(0x562567a74e10 msgr2=0x562567a77200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:08.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23c37b9000 1 -- 172.21.15.83:0/2187257775 shutdown_connections 2024-04-17T10:42:08.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23c37b9000 1 -- 172.21.15.83:0/2187257775 wait complete. 2024-04-17T10:42:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23c37b9000 1 -- start start 2024-04-17T10:42:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23c37b9000 1 Processor -- start 2024-04-17T10:42:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23c37b9000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562567a47be0 0x562567a4d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23c37b9000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562567a94070 0x562567a4dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23c37b9000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562567a94990 0x562567a4e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23c37b9000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56256789c530 con 0x562567a94070 2024-04-17T10:42:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23c37b9000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562567852ae0 con 0x562567a47be0 2024-04-17T10:42:08.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23c37b9000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x562567856e10 con 0x562567a94990 2024-04-17T10:42:08.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23c37b9000 10 client.0 _ll_register_callbacks cb 0x562567a35750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:08.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23bdbe7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562567a94070 0x562567a4dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23bdbe7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562567a94070 0x562567a4dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41034/0 (socket says 172.21.15.83:41034) 2024-04-17T10:42:08.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23bdbe7640 1 -- 172.21.15.83:0/3249635044 learned_addr learned my addr 172.21.15.83:0/3249635044 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:08.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23be3e8640 1 --2- 172.21.15.83:0/3249635044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562567a47be0 0x562567a4d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23bebe9640 1 --2- 172.21.15.83:0/3249635044 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562567a94990 0x562567a4e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23bdbe7640 1 -- 172.21.15.83:0/3249635044 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562567a94990 msgr2=0x562567a4e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:08.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23bdbe7640 1 --2- 172.21.15.83:0/3249635044 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562567a94990 0x562567a4e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23bdbe7640 1 -- 172.21.15.83:0/3249635044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562567a47be0 msgr2=0x562567a4d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:08.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23bdbe7640 1 --2- 172.21.15.83:0/3249635044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562567a47be0 0x562567a4d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23bdbe7640 1 -- 172.21.15.83:0/3249635044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56256783aa10 con 0x562567a94070 2024-04-17T10:42:08.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23be3e8640 1 --2- 172.21.15.83:0/3249635044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562567a47be0 0x562567a4d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:08.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23bdbe7640 1 --2- 172.21.15.83:0/3249635044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562567a94070 0x562567a4dbf0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f23b80aedd0 tx=0x7f23b4001690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139a42f39ada4aa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:08.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23997fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23997fa640 1 -- 172.21.15.83:0/3249635044 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b4012560 con 0x562567a94070 2024-04-17T10:42:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23c37b9000 1 -- 172.21.15.83:0/3249635044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23b00f3070 con 0x562567a94070 2024-04-17T10:42:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23997fa640 1 -- 172.21.15.83:0/3249635044 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f23b4012700 con 0x562567a94070 2024-04-17T10:42:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23997fa640 1 -- 172.21.15.83:0/3249635044 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23b4020050 con 0x562567a94070 2024-04-17T10:42:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23c37b9000 10 client.6090 fetch_fsmap learned FSMap version 22 2024-04-17T10:42:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23c37b9000 1 -- 172.21.15.83:0/3249635044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x562567892ef0 con 0x562567a94070 2024-04-17T10:42:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23997fa640 1 -- 172.21.15.83:0/3249635044 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b00f3070 con 0x562567a94070 2024-04-17T10:42:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23997fa640 1 -- 172.21.15.83:0/3249635044 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f23b4023020 con 0x562567a94070 2024-04-17T10:42:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23c37b9000 10 client.6090 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T10:42:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23c37b9000 10 client.6090 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-17T10:42:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23c37b9000 20 client.6090 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23c37b9000 1 --2- 172.21.15.83:0/3249635044 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562567aa3100 0x562567aa54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23c37b9000 4 client.6090 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:42:08.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23c37b9000 1 -- 172.21.15.83:0/3249635044 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562567833b20 con 0x562567aa3100 2024-04-17T10:42:08.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23be3e8640 1 --2- 172.21.15.83:0/3249635044 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562567aa3100 0x562567aa54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23997fa640 10 client.6090 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:08.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.494+0000 7f23be3e8640 1 --2- 172.21.15.83:0/3249635044 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562567aa3100 0x562567aa54f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=534f9328cd4fe0e8 server_cookie=f44cfb6402e8a3a2 in_seq=0 out_seq=0 2024-04-17T10:42:08.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.498+0000 7f23997fa640 1 -- 172.21.15.83:0/3249635044 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562567833b20 con 0x562567aa3100 2024-04-17T10:42:08.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.498+0000 7f23997fa640 10 client.6090 handle_command_reply: tid=0 2024-04-17T10:42:08.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.498+0000 7f23c37b9000 1 client.6090 shutdown 2024-04-17T10:42:08.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.498+0000 7f23c37b9000 1 -- 172.21.15.83:0/3249635044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562567a94070 msgr2=0x562567a4dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:08.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.498+0000 7f23c37b9000 1 --2- 172.21.15.83:0/3249635044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562567a94070 0x562567a4dbf0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f23b80aedd0 tx=0x7f23b4001690 comp rx=0 tx=0).stop 2024-04-17T10:42:08.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.498+0000 7f23c37b9000 1 -- 172.21.15.83:0/3249635044 shutdown_connections 2024-04-17T10:42:08.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.498+0000 7f23c37b9000 1 --2- 172.21.15.83:0/3249635044 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562567aa3100 0x562567aa54f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.498+0000 7f23c37b9000 1 --2- 172.21.15.83:0/3249635044 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562567a94990 0x562567a4e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.498+0000 7f23997fa640 0 client.6090 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:08.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.498+0000 7f23c37b9000 1 --2- 172.21.15.83:0/3249635044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562567a94070 0x562567a4dbf0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.498+0000 7f23c37b9000 1 --2- 172.21.15.83:0/3249635044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562567a47be0 0x562567a4d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.498+0000 7f23c37b9000 1 -- 172.21.15.83:0/3249635044 >> 172.21.15.83:0/3249635044 conn(0x562567a74e10 msgr2=0x562567a46da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:08.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.498+0000 7f23c37b9000 1 -- 172.21.15.83:0/3249635044 shutdown_connections 2024-04-17T10:42:08.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.498+0000 7f23c37b9000 1 -- 172.21.15.83:0/3249635044 wait complete. 2024-04-17T10:42:08.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.498+0000 7f23c37b9000 20 client.6090 trim_cache size 0 max 16384 2024-04-17T10:42:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.510+0000 7f23c37b9000 1 Processor -- start 2024-04-17T10:42:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.510+0000 7f23c37b9000 1 -- start start 2024-04-17T10:42:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.510+0000 7f23c37b9000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562567a47be0 0x562567af0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.510+0000 7f23c37b9000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562567a94070 0x562567af08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.510+0000 7f23c37b9000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562567a94990 0x562567af0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.510+0000 7f23c37b9000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b80c5f10 con 0x562567a94070 2024-04-17T10:42:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.510+0000 7f23c37b9000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b8152de0 con 0x562567a94990 2024-04-17T10:42:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.510+0000 7f23c37b9000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x562567864b90 con 0x562567a47be0 2024-04-17T10:42:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.510+0000 7f23bdbe7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562567a94070 0x562567af08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.510+0000 7f23bdbe7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562567a94070 0x562567af08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41040/0 (socket says 172.21.15.83:41040) 2024-04-17T10:42:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.510+0000 7f23bdbe7640 1 -- 172.21.15.83:0/840420431 learned_addr learned my addr 172.21.15.83:0/840420431 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.510+0000 7f23bebe9640 1 --2- 172.21.15.83:0/840420431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562567a94990 0x562567af0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.510+0000 7f23bdbe7640 1 -- 172.21.15.83:0/840420431 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562567a47be0 msgr2=0x562567af0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.510+0000 7f23bdbe7640 1 --2- 172.21.15.83:0/840420431 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562567a47be0 0x562567af0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.510+0000 7f23bdbe7640 1 -- 172.21.15.83:0/840420431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562567a94990 msgr2=0x562567af0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.510+0000 7f23bdbe7640 1 --2- 172.21.15.83:0/840420431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562567a94990 0x562567af0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.510+0000 7f23bdbe7640 1 -- 172.21.15.83:0/840420431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562567857790 con 0x562567a94070 2024-04-17T10:42:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.510+0000 7f23bdbe7640 1 --2- 172.21.15.83:0/840420431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562567a94070 0x562567af08c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f23b40016d0 tx=0x7f23b400d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38ae1fc77cc4dd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.510+0000 7f239b7fe640 1 -- 172.21.15.83:0/840420431 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b4012560 con 0x562567a94070 2024-04-17T10:42:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.510+0000 7f239b7fe640 1 -- 172.21.15.83:0/840420431 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b4028070 con 0x562567a94070 2024-04-17T10:42:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.510+0000 7f239b7fe640 1 -- 172.21.15.83:0/840420431 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b4012b40 con 0x562567a94070 2024-04-17T10:42:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.510+0000 7f23c37b9000 1 -- 172.21.15.83:0/840420431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562567a94070 msgr2=0x562567af08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.510+0000 7f23c37b9000 1 --2- 172.21.15.83:0/840420431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562567a94070 0x562567af08c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f23b40016d0 tx=0x7f23b400d000 comp rx=0 tx=0).stop 2024-04-17T10:42:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.510+0000 7f23c37b9000 1 -- 172.21.15.83:0/840420431 shutdown_connections 2024-04-17T10:42:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.510+0000 7f23c37b9000 1 --2- 172.21.15.83:0/840420431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562567a94990 0x562567af0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.510+0000 7f23c37b9000 1 --2- 172.21.15.83:0/840420431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562567a94070 0x562567af08c0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.510+0000 7f23c37b9000 1 --2- 172.21.15.83:0/840420431 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562567a47be0 0x562567af0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.510+0000 7f23c37b9000 1 -- 172.21.15.83:0/840420431 >> 172.21.15.83:0/840420431 conn(0x562567a74e10 msgr2=0x562567a4eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.510+0000 7f23c37b9000 1 -- 172.21.15.83:0/840420431 shutdown_connections 2024-04-17T10:42:08.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.510+0000 7f23c37b9000 1 -- 172.21.15.83:0/840420431 wait complete. 2024-04-17T10:42:08.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.510+0000 7f23c37b9000 1 -- start start 2024-04-17T10:42:08.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.510+0000 7f23c37b9000 1 Processor -- start 2024-04-17T10:42:08.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c37b9000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562567a47be0 0x562567aa67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c37b9000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562567a94070 0x562567aa6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c37b9000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562567a94990 0x562567af0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c37b9000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562567897210 con 0x562567a94990 2024-04-17T10:42:08.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c37b9000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625678d7ca0 con 0x562567a94070 2024-04-17T10:42:08.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c37b9000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x562567897f00 con 0x562567a47be0 2024-04-17T10:42:08.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c37b9000 10 client.0 _ll_register_callbacks cb 0x7f23b0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:08.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23bebe9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562567a94990 0x562567af0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23bebe9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562567a94990 0x562567af0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41048/0 (socket says 172.21.15.83:41048) 2024-04-17T10:42:08.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23bebe9640 1 -- 172.21.15.83:0/2220139567 learned_addr learned my addr 172.21.15.83:0/2220139567 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:08.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23be3e8640 1 --2- 172.21.15.83:0/2220139567 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562567a47be0 0x562567aa67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23bdbe7640 1 --2- 172.21.15.83:0/2220139567 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562567a94070 0x562567aa6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23bebe9640 1 -- 172.21.15.83:0/2220139567 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562567a47be0 msgr2=0x562567aa67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:08.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23bebe9640 1 --2- 172.21.15.83:0/2220139567 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562567a47be0 0x562567aa67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23bebe9640 1 -- 172.21.15.83:0/2220139567 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562567a94070 msgr2=0x562567aa6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:08.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23bebe9640 1 --2- 172.21.15.83:0/2220139567 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562567a94070 0x562567aa6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23bebe9640 1 -- 172.21.15.83:0/2220139567 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56256785fda0 con 0x562567a94990 2024-04-17T10:42:08.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23bdbe7640 1 --2- 172.21.15.83:0/2220139567 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562567a94070 0x562567aa6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:08.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23bebe9640 1 --2- 172.21.15.83:0/2220139567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562567a94990 0x562567af0700 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f23b001af90 tx=0x7f23b0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74dd8d492758f238 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:08.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23997fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:08.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23997fa640 1 -- 172.21.15.83:0/2220139567 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0107040 con 0x562567a94990 2024-04-17T10:42:08.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23997fa640 1 -- 172.21.15.83:0/2220139567 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f23b00adc80 con 0x562567a94990 2024-04-17T10:42:08.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c37b9000 1 -- 172.21.15.83:0/2220139567 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23b4028070 con 0x562567a94990 2024-04-17T10:42:08.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23997fa640 1 -- 172.21.15.83:0/2220139567 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23b00f0040 con 0x562567a94990 2024-04-17T10:42:08.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c37b9000 10 client.6096 fetch_fsmap learned FSMap version 22 2024-04-17T10:42:08.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c37b9000 1 -- 172.21.15.83:0/2220139567 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x562567554360 con 0x562567a94990 2024-04-17T10:42:08.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23997fa640 1 -- 172.21.15.83:0/2220139567 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b4028070 con 0x562567a94990 2024-04-17T10:42:08.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23997fa640 1 -- 172.21.15.83:0/2220139567 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f23b806ce10 con 0x562567a94990 2024-04-17T10:42:08.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c37b9000 10 client.6096 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T10:42:08.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c37b9000 10 client.6096 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-17T10:42:08.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c37b9000 20 client.6096 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:08.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c37b9000 1 --2- 172.21.15.83:0/2220139567 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562567af10c0 0x562567af14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c37b9000 4 client.6096 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 2789176921}] 2024-04-17T10:42:08.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c37b9000 1 -- 172.21.15.83:0/2220139567 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "damage rm", "damage_id": 2789176921}) v1 -- 0x5625678a5440 con 0x562567af10c0 2024-04-17T10:42:08.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23be3e8640 1 --2- 172.21.15.83:0/2220139567 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562567af10c0 0x562567af14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23be3e8640 1 --2- 172.21.15.83:0/2220139567 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562567af10c0 0x562567af14a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=743a1ae85492802c server_cookie=4f2ecaf0556112d7 in_seq=0 out_seq=0 2024-04-17T10:42:08.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23997fa640 10 client.6096 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:08.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23997fa640 1 -- 172.21.15.83:0/2220139567 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x5625678a5440 con 0x562567af10c0 2024-04-17T10:42:08.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23997fa640 10 client.6096 handle_command_reply: tid=0 2024-04-17T10:42:08.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c37b9000 1 client.6096 shutdown 2024-04-17T10:42:08.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c37b9000 1 -- 172.21.15.83:0/2220139567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562567a94990 msgr2=0x562567af0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:08.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c37b9000 1 --2- 172.21.15.83:0/2220139567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562567a94990 0x562567af0700 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f23b001af90 tx=0x7f23b0009750 comp rx=0 tx=0).stop 2024-04-17T10:42:08.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c37b9000 1 -- 172.21.15.83:0/2220139567 shutdown_connections 2024-04-17T10:42:08.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c37b9000 1 --2- 172.21.15.83:0/2220139567 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562567af10c0 0x562567af14a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c37b9000 1 --2- 172.21.15.83:0/2220139567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562567a94990 0x562567af0700 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c37b9000 1 --2- 172.21.15.83:0/2220139567 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562567a94070 0x562567aa6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23997fa640 0 client.6096 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:08.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c37b9000 1 --2- 172.21.15.83:0/2220139567 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562567a47be0 0x562567aa67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c37b9000 1 -- 172.21.15.83:0/2220139567 >> 172.21.15.83:0/2220139567 conn(0x562567a74e10 msgr2=0x562567a4f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:08.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c37b9000 1 -- 172.21.15.83:0/2220139567 shutdown_connections 2024-04-17T10:42:08.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c37b9000 1 -- 172.21.15.83:0/2220139567 wait complete. 2024-04-17T10:42:08.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c37b9000 20 client.6096 trim_cache size 0 max 16384 2024-04-17T10:42:08.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c064c640 1 -- 172.21.15.83:0/1824905403 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f239c066f60 msgr2=0x7f239c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:08.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c064c640 1 --2- 172.21.15.83:0/1824905403 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f239c066f60 0x7f239c069420 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f23a80097a0 tx=0x7f23a8008040 comp rx=0 tx=0).stop 2024-04-17T10:42:08.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c064c640 1 -- 172.21.15.83:0/1824905403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23b80ce910 msgr2=0x7f23b815c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:08.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c064c640 1 --2- 172.21.15.83:0/1824905403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23b80ce910 0x7f23b815c490 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f23b001a510 tx=0x7f23b0019680 comp rx=0 tx=0).stop 2024-04-17T10:42:08.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c064c640 1 -- 172.21.15.83:0/1824905403 shutdown_connections 2024-04-17T10:42:08.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c064c640 1 --2- 172.21.15.83:0/1824905403 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f239c066f60 0x7f239c069420 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c064c640 1 --2- 172.21.15.83:0/1824905403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23b80ce910 0x7f23b815c490 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c064c640 1 --2- 172.21.15.83:0/1824905403 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23b80b2e30 0x7f23b8151a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c064c640 1 --2- 172.21.15.83:0/1824905403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23b80ae9d0 0x7f23b80b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c064c640 1 -- 172.21.15.83:0/1824905403 >> 172.21.15.83:0/1824905403 conn(0x7f23b80d17a0 msgr2=0x7f23b80b7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:08.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c064c640 1 -- 172.21.15.83:0/1824905403 shutdown_connections 2024-04-17T10:42:08.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.514+0000 7f23c064c640 1 -- 172.21.15.83:0/1824905403 wait complete. 2024-04-17T10:42:08.529 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-17T10:42:08.529 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:42:08.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.638+0000 7f88748d6640 1 Processor -- start 2024-04-17T10:42:08.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.638+0000 7f88748d6640 1 -- start start 2024-04-17T10:42:08.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f88748d6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88700ce980 0x7f88700b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f88748d6640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f88700cee00 0x7f88700b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f88748d6640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8870162a00 con 0x7f887015dfa0 2024-04-17T10:42:08.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f88748d6640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8870161e80 con 0x7f88700cee00 2024-04-17T10:42:08.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f88748d6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8870162160 con 0x7f88700ce980 2024-04-17T10:42:08.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f886e59c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f887015dfa0 0x7f88700b6c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46720/0 (socket says 172.21.15.83:46720) 2024-04-17T10:42:08.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f886e59c640 1 -- 172.21.15.83:0/1570958611 learned_addr learned my addr 172.21.15.83:0/1570958611 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:08.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f886d59a640 1 -- 172.21.15.83:0/1570958611 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3518099865 0 0) 0x7f8870162a00 con 0x7f887015dfa0 2024-04-17T10:42:08.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f886dd9b640 1 --2- 172.21.15.83:0/1570958611 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88700ce980 0x7f88700b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f886ed9d640 1 --2- 172.21.15.83:0/1570958611 >> v2:172.21.15.167:3300/0 conn(0x7f88700cee00 0x7f88700b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f886d59a640 1 -- 172.21.15.83:0/1570958611 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f885c003530 con 0x7f887015dfa0 2024-04-17T10:42:08.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f886d59a640 1 -- 172.21.15.83:0/1570958611 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2955850879 0 0) 0x7f885c003530 con 0x7f887015dfa0 2024-04-17T10:42:08.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f886d59a640 1 -- 172.21.15.83:0/1570958611 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88700ce980 msgr2=0x7f88700b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:08.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f886d59a640 1 --2- 172.21.15.83:0/1570958611 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88700ce980 0x7f88700b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f886d59a640 1 -- 172.21.15.83:0/1570958611 >> v2:172.21.15.167:3300/0 conn(0x7f88700cee00 msgr2=0x7f88700b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:08.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f886d59a640 1 --2- 172.21.15.83:0/1570958611 >> v2:172.21.15.167:3300/0 conn(0x7f88700cee00 0x7f88700b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f886d59a640 1 -- 172.21.15.83:0/1570958611 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8870162440 con 0x7f887015dfa0 2024-04-17T10:42:08.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f886d59a640 1 -- 172.21.15.83:0/1570958611 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f8858001680 con 0x7f887015dfa0 2024-04-17T10:42:08.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f886d59a640 1 -- 172.21.15.83:0/1570958611 >> v1:172.21.15.83:6789/0 conn(0x7f887015dfa0 legacy=0x7f88700b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:08.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f886d59a640 1 --2- 172.21.15.83:0/1570958611 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88700ce980 0x7f885c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f886d59a640 1 --2- 172.21.15.83:0/1570958611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f885c003ff0 0x7f885c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f886d59a640 1 --2- 172.21.15.83:0/1570958611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f885c004d50 0x7f885c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f886d59a640 1 -- 172.21.15.83:0/1570958611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8870161e80 con 0x7f885c004d50 2024-04-17T10:42:08.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f886d59a640 1 -- 172.21.15.83:0/1570958611 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8870162160 con 0x7f885c003ff0 2024-04-17T10:42:08.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f886d59a640 1 -- 172.21.15.83:0/1570958611 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f885c00bb90 con 0x7f88700ce980 2024-04-17T10:42:08.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f886ed9d640 1 --2- 172.21.15.83:0/1570958611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f885c004d50 0x7f885c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f886e59c640 1 --2- 172.21.15.83:0/1570958611 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88700ce980 0x7f885c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f886dd9b640 1 --2- 172.21.15.83:0/1570958611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f885c003ff0 0x7f885c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f886ed9d640 1 -- 172.21.15.83:0/1570958611 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88700ce980 msgr2=0x7f885c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:08.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f886ed9d640 1 --2- 172.21.15.83:0/1570958611 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88700ce980 0x7f885c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f886ed9d640 1 -- 172.21.15.83:0/1570958611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f885c003ff0 msgr2=0x7f885c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:08.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f886ed9d640 1 --2- 172.21.15.83:0/1570958611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f885c003ff0 0x7f885c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.642+0000 7f886ed9d640 1 -- 172.21.15.83:0/1570958611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f885c003530 con 0x7f885c004d50 2024-04-17T10:42:08.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f886ed9d640 1 --2- 172.21.15.83:0/1570958611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f885c004d50 0x7f885c0051d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f886000ba30 tx=0x7f886001c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb7fc1ddf32667df server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:08.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f886d59a640 1 -- 172.21.15.83:0/1570958611 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886000b600 con 0x7f885c004d50 2024-04-17T10:42:08.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f886d59a640 1 -- 172.21.15.83:0/1570958611 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88600b1070 con 0x7f885c004d50 2024-04-17T10:42:08.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f886d59a640 1 -- 172.21.15.83:0/1570958611 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88600ac8c0 con 0x7f885c004d50 2024-04-17T10:42:08.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f88748d6640 1 -- 172.21.15.83:0/1570958611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f885c004d50 msgr2=0x7f885c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:08.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f88748d6640 1 --2- 172.21.15.83:0/1570958611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f885c004d50 0x7f885c0051d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f886000ba30 tx=0x7f886001c6e0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f886ed9d640 1 -- 172.21.15.83:0/1570958611 reap_dead start 2024-04-17T10:42:08.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f88748d6640 1 -- 172.21.15.83:0/1570958611 shutdown_connections 2024-04-17T10:42:08.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f88748d6640 1 -- 172.21.15.83:0/1570958611 >> 172.21.15.83:0/1570958611 conn(0x7f88700d0600 msgr2=0x7f88700d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:08.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f88748d6640 1 -- 172.21.15.83:0/1570958611 shutdown_connections 2024-04-17T10:42:08.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f88748d6640 1 -- 172.21.15.83:0/1570958611 wait complete. 2024-04-17T10:42:08.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f88748d6640 1 Processor -- start 2024-04-17T10:42:08.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f88748d6640 1 -- start start 2024-04-17T10:42:08.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f88748d6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f885c004d50 0x7f887014e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f88748d6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8870154b30 0x7f88700cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f88748d6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8870154fb0 0x7f88700ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f88748d6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8870161b10 con 0x7f8870154fb0 2024-04-17T10:42:08.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f88748d6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887015e380 con 0x7f8870154b30 2024-04-17T10:42:08.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f88748d6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88700b5430 con 0x7f885c004d50 2024-04-17T10:42:08.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f886ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8870154fb0 0x7f88700ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f886ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8870154fb0 0x7f88700ccdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41062/0 (socket says 172.21.15.83:41062) 2024-04-17T10:42:08.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f886ed9d640 1 -- 172.21.15.83:0/3954037867 learned_addr learned my addr 172.21.15.83:0/3954037867 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:08.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f886e59c640 1 --2- 172.21.15.83:0/3954037867 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f885c004d50 0x7f887014e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f886dd9b640 1 --2- 172.21.15.83:0/3954037867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8870154b30 0x7f88700cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f886ed9d640 1 -- 172.21.15.83:0/3954037867 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f885c004d50 msgr2=0x7f887014e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:08.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f886ed9d640 1 --2- 172.21.15.83:0/3954037867 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f885c004d50 0x7f887014e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f886ed9d640 1 -- 172.21.15.83:0/3954037867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8870154b30 msgr2=0x7f88700cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:08.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f886ed9d640 1 --2- 172.21.15.83:0/3954037867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8870154b30 0x7f88700cc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f886ed9d640 1 -- 172.21.15.83:0/3954037867 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88700c8870 con 0x7f8870154fb0 2024-04-17T10:42:08.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f886ed9d640 1 --2- 172.21.15.83:0/3954037867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8870154fb0 0x7f88700ccdb0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f88600ae040 tx=0x7f885c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:08.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f8866ffd640 1 -- 172.21.15.83:0/3954037867 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886000b600 con 0x7f8870154fb0 2024-04-17T10:42:08.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f8866ffd640 1 -- 172.21.15.83:0/3954037867 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8860000e50 con 0x7f8870154fb0 2024-04-17T10:42:08.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f88748d6640 1 -- 172.21.15.83:0/3954037867 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f887014ee30 con 0x7f8870154fb0 2024-04-17T10:42:08.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f8866ffd640 1 -- 172.21.15.83:0/3954037867 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8860004420 con 0x7f8870154fb0 2024-04-17T10:42:08.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.646+0000 7f88748d6640 1 -- 172.21.15.83:0/3954037867 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8870155f40 con 0x7f8870154fb0 2024-04-17T10:42:08.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.650+0000 7f88748d6640 1 -- 172.21.15.83:0/3954037867 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88700681a0 con 0x7f8870154fb0 2024-04-17T10:42:08.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.650+0000 7f8866ffd640 1 -- 172.21.15.83:0/3954037867 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f88600be030 con 0x7f8870154fb0 2024-04-17T10:42:08.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.650+0000 7f8866ffd640 1 --2- 172.21.15.83:0/3954037867 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8850066f90 0x7f8850069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:08.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.650+0000 7f886e59c640 1 --2- 172.21.15.83:0/3954037867 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8850066f90 0x7f8850069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:08.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.650+0000 7f8866ffd640 1 -- 172.21.15.83:0/3954037867 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f886011faa0 con 0x7f8870154fb0 2024-04-17T10:42:08.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.650+0000 7f886e59c640 1 --2- 172.21.15.83:0/3954037867 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8850066f90 0x7f8850069450 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8858001060 tx=0x7f8858009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:08.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.654+0000 7f8866ffd640 1 -- 172.21.15.83:0/3954037867 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f88600163f0 con 0x7f8870154fb0 2024-04-17T10:42:08.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.830+0000 7f88748d6640 1 -- 172.21.15.83:0/3954037867 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8870074f90 con 0x7f8870154fb0 2024-04-17T10:42:08.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.830+0000 7f8866ffd640 1 -- 172.21.15.83:0/3954037867 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f88600ace20 con 0x7f8870154fb0 2024-04-17T10:42:08.829 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:08.830 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/161398126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":161398126},{"type":"v1","addr":"172.21.15.167:6833","nonce":161398126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1085380618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1085380618},{"type":"v1","addr":"172.21.15.167:6835","nonce":1085380618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/836442291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":836442291},{"type":"v1","addr":"172.21.15.83:6835","nonce":836442291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:41:37.773342+0000","modified":"2024-04-17T10:42:06.483516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5682":{"gid":5682,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.83:6837/1894547386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1894547386},{"type":"v1","addr":"172.21.15.83:6837","nonce":1894547386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-17T10:42:08.830 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 22 2024-04-17T10:42:08.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.834+0000 7f88748d6640 1 -- 172.21.15.83:0/3954037867 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8850066f90 msgr2=0x7f8850069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:08.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.834+0000 7f88748d6640 1 --2- 172.21.15.83:0/3954037867 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8850066f90 0x7f8850069450 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8858001060 tx=0x7f8858009040 comp rx=0 tx=0).stop 2024-04-17T10:42:08.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.834+0000 7f88748d6640 1 -- 172.21.15.83:0/3954037867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8870154fb0 msgr2=0x7f88700ccdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:08.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.834+0000 7f88748d6640 1 --2- 172.21.15.83:0/3954037867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8870154fb0 0x7f88700ccdb0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f88600ae040 tx=0x7f885c005840 comp rx=0 tx=0).stop 2024-04-17T10:42:08.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.834+0000 7f88748d6640 1 -- 172.21.15.83:0/3954037867 shutdown_connections 2024-04-17T10:42:08.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.834+0000 7f88748d6640 1 --2- 172.21.15.83:0/3954037867 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8850066f90 0x7f8850069450 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.834+0000 7f88748d6640 1 --2- 172.21.15.83:0/3954037867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8870154fb0 0x7f88700ccdb0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.834+0000 7f88748d6640 1 --2- 172.21.15.83:0/3954037867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8870154b30 0x7f88700cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.834+0000 7f88748d6640 1 --2- 172.21.15.83:0/3954037867 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f885c004d50 0x7f887014e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:08.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.834+0000 7f88748d6640 1 -- 172.21.15.83:0/3954037867 >> 172.21.15.83:0/3954037867 conn(0x7f88700d0600 msgr2=0x7f88700d2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:08.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.834+0000 7f88748d6640 1 -- 172.21.15.83:0/3954037867 shutdown_connections 2024-04-17T10:42:08.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:08.834+0000 7f88748d6640 1 -- 172.21.15.83:0/3954037867 wait complete. 2024-04-17T10:42:08.846 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path / 2024-04-17T10:42:08.966 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:42:08.966 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0 2024-04-17T10:42:08.967 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:42:08.980 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-17T10:42:08.980 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-17T10:42:08.980 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_test_path 2024-04-17T10:42:08.980 INFO:tasks.cephfs.test_scrub_checks:Cloning repo into place 2024-04-17T10:42:08.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:42:08.980 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p /home/ubuntu/cephtest/mnt.0/scrub_test_path) 2024-04-17T10:42:08.988 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-04-17T10:42:08.988 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-04-17T10:42:08.988 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-04-17T10:42:08.988 DEBUG:teuthology.orchestra.run.smithi083:> import json 2024-04-17T10:42:08.988 DEBUG:teuthology.orchestra.run.smithi083:> import sys 2024-04-17T10:42:08.988 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:42:08.988 DEBUG:teuthology.orchestra.run.smithi083:> try: 2024-04-17T10:42:08.988 DEBUG:teuthology.orchestra.run.smithi083:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'"'"') 2024-04-17T10:42:08.988 DEBUG:teuthology.orchestra.run.smithi083:> except OSError as e: 2024-04-17T10:42:08.989 DEBUG:teuthology.orchestra.run.smithi083:> sys.exit(e.errno) 2024-04-17T10:42:08.989 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:42:08.989 DEBUG:teuthology.orchestra.run.smithi083:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-17T10:42:08.989 DEBUG:teuthology.orchestra.run.smithi083:> print(json.dumps( 2024-04-17T10:42:08.989 DEBUG:teuthology.orchestra.run.smithi083:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-17T10:42:08.989 DEBUG:teuthology.orchestra.run.smithi083:> indent=2)) 2024-04-17T10:42:08.989 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-04-17T10:42:09.095 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:42:09.095 INFO:teuthology.orchestra.run.smithi083.stdout: "st_mode": 16893, 2024-04-17T10:42:09.095 INFO:teuthology.orchestra.run.smithi083.stdout: "st_ino": 1099511627777, 2024-04-17T10:42:09.095 INFO:teuthology.orchestra.run.smithi083.stdout: "st_dev": 45, 2024-04-17T10:42:09.095 INFO:teuthology.orchestra.run.smithi083.stdout: "st_nlink": 16, 2024-04-17T10:42:09.096 INFO:teuthology.orchestra.run.smithi083.stdout: "st_uid": 1000, 2024-04-17T10:42:09.096 INFO:teuthology.orchestra.run.smithi083.stdout: "st_gid": 1271, 2024-04-17T10:42:09.096 INFO:teuthology.orchestra.run.smithi083.stdout: "st_size": 9190641, 2024-04-17T10:42:09.096 INFO:teuthology.orchestra.run.smithi083.stdout: "st_atime": 1713350505.9999828, 2024-04-17T10:42:09.096 INFO:teuthology.orchestra.run.smithi083.stdout: "st_mtime": 1713350521.3518276, 2024-04-17T10:42:09.096 INFO:teuthology.orchestra.run.smithi083.stdout: "st_ctime": 1713350521.3518276 2024-04-17T10:42:09.096 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:42:09.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:42:09.105 INFO:tasks.cephfs.test_scrub_checks:Initiating mds_scrub_checks on mds.0 test_path /scrub_test_path, run_seq 1 2024-04-17T10:42:09.105 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/i/dont/exist' 2024-04-17T10:42:09.106 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:42:09.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.266+0000 7efee5c0d640 1 Processor -- start 2024-04-17T10:42:09.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.266+0000 7efee5c0d640 1 -- start start 2024-04-17T10:42:09.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.270+0000 7efee5c0d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efee015eeb0 0x7efee006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.270+0000 7efee5c0d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efee006c300 0x7efee00ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.270+0000 7efee5c0d640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efee006e980 con 0x7efee00cf780 2024-04-17T10:42:09.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.270+0000 7efee5c0d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efee00cdb90 con 0x7efee015eeb0 2024-04-17T10:42:09.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.270+0000 7efee5c0d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee006e0e0 con 0x7efee006c300 2024-04-17T10:42:09.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.270+0000 7efedf7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efee015eeb0 0x7efee006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.270+0000 7efedf7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efee015eeb0 0x7efee006d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41072/0 (socket says 172.21.15.83:41072) 2024-04-17T10:42:09.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.270+0000 7efedf7fe640 1 -- 172.21.15.83:0/3353229897 learned_addr learned my addr 172.21.15.83:0/3353229897 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:09.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.270+0000 7efedffff640 1 --2- 172.21.15.83:0/3353229897 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efee006c300 0x7efee00ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.270+0000 7efedf7fe640 1 -- 172.21.15.83:0/3353229897 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efee006c300 msgr2=0x7efee00ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:09.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.270+0000 7efedf7fe640 1 --2- 172.21.15.83:0/3353229897 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efee006c300 0x7efee00ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.270+0000 7efedf7fe640 1 -- 172.21.15.83:0/3353229897 >> v1:172.21.15.167:6789/0 conn(0x7efee00cf780 legacy=0x7efee006bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:09.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.270+0000 7efedf7fe640 1 -- 172.21.15.83:0/3353229897 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee006e3c0 con 0x7efee015eeb0 2024-04-17T10:42:09.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.270+0000 7efedffff640 1 --2- 172.21.15.83:0/3353229897 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efee006c300 0x7efee00ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:09.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.270+0000 7efedf7fe640 1 --2- 172.21.15.83:0/3353229897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efee015eeb0 0x7efee006d5d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7efed8009e50 tx=0x7efed801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf177e27e25c547a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:09.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.270+0000 7efede7fc640 1 -- 172.21.15.83:0/3353229897 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed80a5860 con 0x7efee015eeb0 2024-04-17T10:42:09.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efede7fc640 1 -- 172.21.15.83:0/3353229897 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed80a5a00 con 0x7efee015eeb0 2024-04-17T10:42:09.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efede7fc640 1 -- 172.21.15.83:0/3353229897 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed80a5dd0 con 0x7efee015eeb0 2024-04-17T10:42:09.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efee5c0d640 1 -- 172.21.15.83:0/3353229897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efee015eeb0 msgr2=0x7efee006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:09.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efee5c0d640 1 --2- 172.21.15.83:0/3353229897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efee015eeb0 0x7efee006d5d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7efed8009e50 tx=0x7efed801c9c0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efee5c0d640 1 -- 172.21.15.83:0/3353229897 shutdown_connections 2024-04-17T10:42:09.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efee5c0d640 1 --2- 172.21.15.83:0/3353229897 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efee006c300 0x7efee00ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efee5c0d640 1 --2- 172.21.15.83:0/3353229897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efee015eeb0 0x7efee006d5d0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efee5c0d640 1 -- 172.21.15.83:0/3353229897 >> 172.21.15.83:0/3353229897 conn(0x7efee00cc910 msgr2=0x7efee00ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:09.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efee5c0d640 1 -- 172.21.15.83:0/3353229897 shutdown_connections 2024-04-17T10:42:09.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efee5c0d640 1 -- 172.21.15.83:0/3353229897 wait complete. 2024-04-17T10:42:09.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efee5c0d640 1 Processor -- start 2024-04-17T10:42:09.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efee5c0d640 1 -- start start 2024-04-17T10:42:09.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efee5c0d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efee006c300 0x7efee00b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efee5c0d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efee00cf780 0x7efee00b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efee5c0d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efee015eeb0 0x7efee009bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efee5c0d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee0168a80 con 0x7efee015eeb0 2024-04-17T10:42:09.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efee5c0d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee00cef80 con 0x7efee006c300 2024-04-17T10:42:09.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efee5c0d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee0168c00 con 0x7efee00cf780 2024-04-17T10:42:09.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efedffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efee015eeb0 0x7efee009bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efedffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efee015eeb0 0x7efee009bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41086/0 (socket says 172.21.15.83:41086) 2024-04-17T10:42:09.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efedffff640 1 -- 172.21.15.83:0/1346510760 learned_addr learned my addr 172.21.15.83:0/1346510760 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:09.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efedf7fe640 1 --2- 172.21.15.83:0/1346510760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efee006c300 0x7efee00b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efedffff640 1 -- 172.21.15.83:0/1346510760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efee00cf780 msgr2=0x7efee00b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:09.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efedeffd640 1 --2- 172.21.15.83:0/1346510760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efee00cf780 0x7efee00b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efedffff640 1 --2- 172.21.15.83:0/1346510760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efee00cf780 0x7efee00b2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efedffff640 1 -- 172.21.15.83:0/1346510760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efee006c300 msgr2=0x7efee00b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:09.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efedffff640 1 --2- 172.21.15.83:0/1346510760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efee006c300 0x7efee00b13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efedffff640 1 -- 172.21.15.83:0/1346510760 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee01f4ec0 con 0x7efee015eeb0 2024-04-17T10:42:09.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efedf7fe640 1 --2- 172.21.15.83:0/1346510760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efee006c300 0x7efee00b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:09.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efedffff640 1 --2- 172.21.15.83:0/1346510760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efee015eeb0 0x7efee009bad0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7efed000c820 tx=0x7efed000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:09.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efec7fff640 1 -- 172.21.15.83:0/1346510760 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed0015070 con 0x7efee015eeb0 2024-04-17T10:42:09.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efec7fff640 1 -- 172.21.15.83:0/1346510760 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed0007a70 con 0x7efee015eeb0 2024-04-17T10:42:09.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efee5c0d640 1 -- 172.21.15.83:0/1346510760 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee00c7460 con 0x7efee015eeb0 2024-04-17T10:42:09.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efec7fff640 1 -- 172.21.15.83:0/1346510760 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed0007e60 con 0x7efee015eeb0 2024-04-17T10:42:09.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.274+0000 7efee5c0d640 1 -- 172.21.15.83:0/1346510760 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee00c7a20 con 0x7efee015eeb0 2024-04-17T10:42:09.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.278+0000 7efee5c0d640 1 -- 172.21.15.83:0/1346510760 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efee0074f90 con 0x7efee015eeb0 2024-04-17T10:42:09.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.278+0000 7efec7fff640 1 -- 172.21.15.83:0/1346510760 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efed0018070 con 0x7efee015eeb0 2024-04-17T10:42:09.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.278+0000 7efec7fff640 1 --2- 172.21.15.83:0/1346510760 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efecc066f60 0x7efecc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.278+0000 7efec7fff640 1 -- 172.21.15.83:0/1346510760 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7efed0085570 con 0x7efee015eeb0 2024-04-17T10:42:09.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.278+0000 7efedf7fe640 1 --2- 172.21.15.83:0/1346510760 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efecc066f60 0x7efecc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.278+0000 7efedf7fe640 1 --2- 172.21.15.83:0/1346510760 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efecc066f60 0x7efecc069420 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7efed80ae000 tx=0x7efed80a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:09.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.282+0000 7efec7fff640 1 -- 172.21.15.83:0/1346510760 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efed0056490 con 0x7efee015eeb0 2024-04-17T10:42:09.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.466+0000 7efee5c0d640 1 -- 172.21.15.83:0/1346510760 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efee0068150 con 0x7efee015eeb0 2024-04-17T10:42:09.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.466+0000 7efec7fff640 1 -- 172.21.15.83:0/1346510760 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4939 (secure 0 0 0) 0x7efed0055d90 con 0x7efee015eeb0 2024-04-17T10:42:09.467 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:09.467 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/161398126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":161398126},{"type":"v1","addr":"172.21.15.167:6833","nonce":161398126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1085380618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1085380618},{"type":"v1","addr":"172.21.15.167:6835","nonce":1085380618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/836442291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":836442291},{"type":"v1","addr":"172.21.15.83:6835","nonce":836442291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:41:37.773342+0000","modified":"2024-04-17T10:42:06.483516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5682":{"gid":5682,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.83:6837/1894547386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1894547386},{"type":"v1","addr":"172.21.15.83:6837","nonce":1894547386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-17T10:42:09.467 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 22 2024-04-17T10:42:09.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.470+0000 7efee5c0d640 1 -- 172.21.15.83:0/1346510760 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efecc066f60 msgr2=0x7efecc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:09.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.470+0000 7efee5c0d640 1 --2- 172.21.15.83:0/1346510760 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efecc066f60 0x7efecc069420 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7efed80ae000 tx=0x7efed80a7070 comp rx=0 tx=0).stop 2024-04-17T10:42:09.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.470+0000 7efee5c0d640 1 -- 172.21.15.83:0/1346510760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efee015eeb0 msgr2=0x7efee009bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:09.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.470+0000 7efee5c0d640 1 --2- 172.21.15.83:0/1346510760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efee015eeb0 0x7efee009bad0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7efed000c820 tx=0x7efed000cce0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.470+0000 7efee5c0d640 1 -- 172.21.15.83:0/1346510760 shutdown_connections 2024-04-17T10:42:09.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.470+0000 7efee5c0d640 1 --2- 172.21.15.83:0/1346510760 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efecc066f60 0x7efecc069420 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.470+0000 7efee5c0d640 1 --2- 172.21.15.83:0/1346510760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efee015eeb0 0x7efee009bad0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.470+0000 7efee5c0d640 1 --2- 172.21.15.83:0/1346510760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efee00cf780 0x7efee00b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.470+0000 7efee5c0d640 1 --2- 172.21.15.83:0/1346510760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efee006c300 0x7efee00b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.470+0000 7efee5c0d640 1 -- 172.21.15.83:0/1346510760 >> 172.21.15.83:0/1346510760 conn(0x7efee00cc910 msgr2=0x7efee0070f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:09.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.470+0000 7efee5c0d640 1 -- 172.21.15.83:0/1346510760 shutdown_connections 2024-04-17T10:42:09.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.470+0000 7efee5c0d640 1 -- 172.21.15.83:0/1346510760 wait complete. 2024-04-17T10:42:09.485 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/i/dont/exist 2024-04-17T10:42:09.599 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:42:09.599 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": -2 2024-04-17T10:42:09.599 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:42:09.613 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/i/dont/exist' got response code '0' and stdout '{ "return_code": -2 } ' 2024-04-17T10:42:09.613 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/i/dont/exist' 2024-04-17T10:42:09.613 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-17T10:42:09.613 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/i/dont/exist 2024-04-17T10:42:09.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.742+0000 7fc162d70640 1 Processor -- start 2024-04-17T10:42:09.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.742+0000 7fc162d70640 1 -- start start 2024-04-17T10:42:09.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.742+0000 7fc162d70640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc15c15bc10 0x7fc15c0d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.742+0000 7fc162d70640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc15c0d0490 0x7fc15c0cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.742+0000 7fc162d70640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc15c0cd3d0 0x7fc15c06e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.742+0000 7fc162d70640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc15c0ced00 con 0x7fc15c15bc10 2024-04-17T10:42:09.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.742+0000 7fc162d70640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc15c0cefe0 con 0x7fc15c0cd3d0 2024-04-17T10:42:09.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.742+0000 7fc162d70640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c0cc690 con 0x7fc15c0d0490 2024-04-17T10:42:09.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.742+0000 7fc160b0c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc15c15bc10 0x7fc15c0d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.742+0000 7fc160b0c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc15c15bc10 0x7fc15c0d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41098/0 (socket says 172.21.15.83:41098) 2024-04-17T10:42:09.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.742+0000 7fc160b0c640 1 -- 172.21.15.83:0/4116262530 learned_addr learned my addr 172.21.15.83:0/4116262530 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:09.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.742+0000 7fc16130d640 1 --2- 172.21.15.83:0/4116262530 >> v2:172.21.15.167:3300/0 conn(0x7fc15c0cd3d0 0x7fc15c06e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.742+0000 7fc15bfff640 1 --2- 172.21.15.83:0/4116262530 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc15c0d0490 0x7fc15c0cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.742+0000 7fc160b0c640 1 -- 172.21.15.83:0/4116262530 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc15c0d0490 msgr2=0x7fc15c0cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:09.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.742+0000 7fc160b0c640 1 --2- 172.21.15.83:0/4116262530 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc15c0d0490 0x7fc15c0cce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.742+0000 7fc160b0c640 1 -- 172.21.15.83:0/4116262530 >> v2:172.21.15.167:3300/0 conn(0x7fc15c0cd3d0 msgr2=0x7fc15c06e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:09.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.742+0000 7fc160b0c640 1 --2- 172.21.15.83:0/4116262530 >> v2:172.21.15.167:3300/0 conn(0x7fc15c0cd3d0 0x7fc15c06e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.742+0000 7fc160b0c640 1 -- 172.21.15.83:0/4116262530 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc15c0cc970 con 0x7fc15c15bc10 2024-04-17T10:42:09.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.746+0000 7fc160b0c640 1 --2- 172.21.15.83:0/4116262530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc15c15bc10 0x7fc15c0d2300 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0b7710 tx=0x7fc15001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2f96a015d7c490 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:09.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.746+0000 7fc15b7fe640 1 -- 172.21.15.83:0/4116262530 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1500b0070 con 0x7fc15c15bc10 2024-04-17T10:42:09.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.746+0000 7fc15b7fe640 1 -- 172.21.15.83:0/4116262530 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1500ab600 con 0x7fc15c15bc10 2024-04-17T10:42:09.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.746+0000 7fc15b7fe640 1 -- 172.21.15.83:0/4116262530 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1500ab9d0 con 0x7fc15c15bc10 2024-04-17T10:42:09.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.746+0000 7fc162d70640 1 -- 172.21.15.83:0/4116262530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc15c15bc10 msgr2=0x7fc15c0d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:09.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.746+0000 7fc162d70640 1 --2- 172.21.15.83:0/4116262530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc15c15bc10 0x7fc15c0d2300 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0b7710 tx=0x7fc15001c490 comp rx=0 tx=0).stop 2024-04-17T10:42:09.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.746+0000 7fc162d70640 1 -- 172.21.15.83:0/4116262530 shutdown_connections 2024-04-17T10:42:09.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.746+0000 7fc162d70640 1 --2- 172.21.15.83:0/4116262530 >> v2:172.21.15.167:3300/0 conn(0x7fc15c0cd3d0 0x7fc15c06e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.746+0000 7fc162d70640 1 --2- 172.21.15.83:0/4116262530 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc15c0d0490 0x7fc15c0cce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.746+0000 7fc162d70640 1 --2- 172.21.15.83:0/4116262530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc15c15bc10 0x7fc15c0d2300 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.746+0000 7fc162d70640 1 -- 172.21.15.83:0/4116262530 >> 172.21.15.83:0/4116262530 conn(0x7fc15c0d17e0 msgr2=0x7fc15c0d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:09.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.746+0000 7fc162d70640 1 -- 172.21.15.83:0/4116262530 shutdown_connections 2024-04-17T10:42:09.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.746+0000 7fc162d70640 1 -- 172.21.15.83:0/4116262530 wait complete. 2024-04-17T10:42:09.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.746+0000 7fc162d70640 1 Processor -- start 2024-04-17T10:42:09.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.746+0000 7fc162d70640 1 -- start start 2024-04-17T10:42:09.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.746+0000 7fc162d70640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc15c0cd3d0 0x7fc15c0c73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.746+0000 7fc162d70640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc15c0d0490 0x7fc15c0cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.746+0000 7fc162d70640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc15c15bc10 0x7fc15c0cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.746+0000 7fc162d70640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c15f9b0 con 0x7fc15c15bc10 2024-04-17T10:42:09.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.746+0000 7fc162d70640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c15f830 con 0x7fc15c0cd3d0 2024-04-17T10:42:09.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.746+0000 7fc162d70640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c15fb30 con 0x7fc15c0d0490 2024-04-17T10:42:09.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.750+0000 7fc16130d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc15c15bc10 0x7fc15c0cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.750+0000 7fc16130d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc15c15bc10 0x7fc15c0cb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41102/0 (socket says 172.21.15.83:41102) 2024-04-17T10:42:09.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.750+0000 7fc16130d640 1 -- 172.21.15.83:0/2046931297 learned_addr learned my addr 172.21.15.83:0/2046931297 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:09.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.750+0000 7fc160b0c640 1 --2- 172.21.15.83:0/2046931297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc15c0cd3d0 0x7fc15c0c73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.750+0000 7fc15bfff640 1 --2- 172.21.15.83:0/2046931297 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc15c0d0490 0x7fc15c0cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.750+0000 7fc16130d640 1 -- 172.21.15.83:0/2046931297 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc15c0d0490 msgr2=0x7fc15c0cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:09.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.750+0000 7fc16130d640 1 --2- 172.21.15.83:0/2046931297 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc15c0d0490 0x7fc15c0cab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.750+0000 7fc16130d640 1 -- 172.21.15.83:0/2046931297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc15c0cd3d0 msgr2=0x7fc15c0c73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:09.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.750+0000 7fc16130d640 1 --2- 172.21.15.83:0/2046931297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc15c0cd3d0 0x7fc15c0c73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.750+0000 7fc16130d640 1 -- 172.21.15.83:0/2046931297 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc15c1644a0 con 0x7fc15c15bc10 2024-04-17T10:42:09.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.750+0000 7fc15bfff640 1 --2- 172.21.15.83:0/2046931297 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc15c0d0490 0x7fc15c0cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:09.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.750+0000 7fc16130d640 1 --2- 172.21.15.83:0/2046931297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc15c15bc10 0x7fc15c0cb040 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fc154007e60 tx=0x7fc15400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:09.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.750+0000 7fc1597fa640 1 -- 172.21.15.83:0/2046931297 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc154014620 con 0x7fc15c15bc10 2024-04-17T10:42:09.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.750+0000 7fc1597fa640 1 -- 172.21.15.83:0/2046931297 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc154019070 con 0x7fc15c15bc10 2024-04-17T10:42:09.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.750+0000 7fc162d70640 1 -- 172.21.15.83:0/2046931297 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc15c164780 con 0x7fc15c15bc10 2024-04-17T10:42:09.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.750+0000 7fc1597fa640 1 -- 172.21.15.83:0/2046931297 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc154014a20 con 0x7fc15c15bc10 2024-04-17T10:42:09.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.750+0000 7fc162d70640 1 -- 172.21.15.83:0/2046931297 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc15c0c7e00 con 0x7fc15c15bc10 2024-04-17T10:42:09.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.750+0000 7fc1597fa640 1 -- 172.21.15.83:0/2046931297 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc154014bc0 con 0x7fc15c15bc10 2024-04-17T10:42:09.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.750+0000 7fc1597fa640 1 --2- 172.21.15.83:0/2046931297 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc140066f60 0x7fc140069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.750+0000 7fc1597fa640 1 -- 172.21.15.83:0/2046931297 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc1540858b0 con 0x7fc15c15bc10 2024-04-17T10:42:09.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.750+0000 7fc160b0c640 1 --2- 172.21.15.83:0/2046931297 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc140066f60 0x7fc140069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.750+0000 7fc160b0c640 1 --2- 172.21.15.83:0/2046931297 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc140066f60 0x7fc140069420 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc1500ad000 tx=0x7fc1500a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:09.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.750+0000 7fc165edd000 1 Processor -- start 2024-04-17T10:42:09.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.750+0000 7fc165edd000 1 -- start start 2024-04-17T10:42:09.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc165edd000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5562517e7970 0x5562517e7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc165edd000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562517e8290 0x55625179d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc165edd000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55625179dcd0 0x55625179e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc165edd000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562515b38a0 con 0x55625179dcd0 2024-04-17T10:42:09.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc165edd000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562515ebd10 con 0x5562517e8290 2024-04-17T10:42:09.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc165edd000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5562515742d0 con 0x5562517e7970 2024-04-17T10:42:09.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc16130d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55625179dcd0 0x55625179e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc16130d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55625179dcd0 0x55625179e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41110/0 (socket says 172.21.15.83:41110) 2024-04-17T10:42:09.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc16130d640 1 -- 172.21.15.83:0/2452750919 learned_addr learned my addr 172.21.15.83:0/2452750919 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:09.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc15bfff640 1 --2- 172.21.15.83:0/2452750919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562517e8290 0x55625179d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc16130d640 1 -- 172.21.15.83:0/2452750919 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5562517e7970 msgr2=0x5562517e7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:09.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc16130d640 1 --2- 172.21.15.83:0/2452750919 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5562517e7970 0x5562517e7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc16130d640 1 -- 172.21.15.83:0/2452750919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562517e8290 msgr2=0x55625179d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:09.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc16130d640 1 --2- 172.21.15.83:0/2452750919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562517e8290 0x55625179d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc16130d640 1 -- 172.21.15.83:0/2452750919 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556251632870 con 0x55625179dcd0 2024-04-17T10:42:09.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc16130d640 1 --2- 172.21.15.83:0/2452750919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55625179dcd0 0x55625179e0b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc15405e360 tx=0x7fc154056880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d365f8fc4a0bc6e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:09.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc1467fc640 1 -- 172.21.15.83:0/2452750919 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15405e890 con 0x55625179dcd0 2024-04-17T10:42:09.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc1467fc640 1 -- 172.21.15.83:0/2452750919 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc154054480 con 0x55625179dcd0 2024-04-17T10:42:09.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc1467fc640 1 -- 172.21.15.83:0/2452750919 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1540563e0 con 0x55625179dcd0 2024-04-17T10:42:09.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc165edd000 1 -- 172.21.15.83:0/2452750919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55625179dcd0 msgr2=0x55625179e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:09.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc165edd000 1 --2- 172.21.15.83:0/2452750919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55625179dcd0 0x55625179e0b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc15405e360 tx=0x7fc154056880 comp rx=0 tx=0).stop 2024-04-17T10:42:09.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc165edd000 1 -- 172.21.15.83:0/2452750919 shutdown_connections 2024-04-17T10:42:09.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc165edd000 1 --2- 172.21.15.83:0/2452750919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55625179dcd0 0x55625179e0b0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc165edd000 1 --2- 172.21.15.83:0/2452750919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562517e8290 0x55625179d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc165edd000 1 --2- 172.21.15.83:0/2452750919 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5562517e7970 0x5562517e7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc165edd000 1 -- 172.21.15.83:0/2452750919 >> 172.21.15.83:0/2452750919 conn(0x5562517a4540 msgr2=0x5562517a4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:09.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc165edd000 1 -- 172.21.15.83:0/2452750919 shutdown_connections 2024-04-17T10:42:09.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc165edd000 1 -- 172.21.15.83:0/2452750919 wait complete. 2024-04-17T10:42:09.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc165edd000 1 -- start start 2024-04-17T10:42:09.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc165edd000 1 Processor -- start 2024-04-17T10:42:09.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc165edd000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55625179dcd0 0x5562517edcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc165edd000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5562517e7970 0x5562517ee200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc165edd000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562517e8290 0x5562517a27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc165edd000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562515a6b50 con 0x5562517e8290 2024-04-17T10:42:09.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc165edd000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562515e7db0 con 0x55625179dcd0 2024-04-17T10:42:09.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc165edd000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x556251592870 con 0x5562517e7970 2024-04-17T10:42:09.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc165edd000 10 client.0 _ll_register_callbacks cb 0x55625169ffb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:09.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc16130d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562517e8290 0x5562517a27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc16130d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562517e8290 0x5562517a27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41120/0 (socket says 172.21.15.83:41120) 2024-04-17T10:42:09.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc16130d640 1 -- 172.21.15.83:0/318563908 learned_addr learned my addr 172.21.15.83:0/318563908 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:09.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc160b0c640 1 --2- 172.21.15.83:0/318563908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55625179dcd0 0x5562517edcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc15bfff640 1 --2- 172.21.15.83:0/318563908 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5562517e7970 0x5562517ee200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc16130d640 1 -- 172.21.15.83:0/318563908 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5562517e7970 msgr2=0x5562517ee200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:09.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc16130d640 1 --2- 172.21.15.83:0/318563908 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5562517e7970 0x5562517ee200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc16130d640 1 -- 172.21.15.83:0/318563908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55625179dcd0 msgr2=0x5562517edcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:09.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc16130d640 1 --2- 172.21.15.83:0/318563908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55625179dcd0 0x5562517edcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc16130d640 1 -- 172.21.15.83:0/318563908 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556251574520 con 0x5562517e8290 2024-04-17T10:42:09.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc160b0c640 1 --2- 172.21.15.83:0/318563908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55625179dcd0 0x5562517edcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:09.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.754+0000 7fc15bfff640 1 --2- 172.21.15.83:0/318563908 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5562517e7970 0x5562517ee200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:09.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc16130d640 1 --2- 172.21.15.83:0/318563908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562517e8290 0x5562517a27e0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc154056110 tx=0x7fc154007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21bf96664b585166 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:09.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc13b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:09.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc13b7fe640 1 -- 172.21.15.83:0/318563908 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc154071030 con 0x5562517e8290 2024-04-17T10:42:09.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc13b7fe640 1 -- 172.21.15.83:0/318563908 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc154082070 con 0x5562517e8290 2024-04-17T10:42:09.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc165edd000 1 -- 172.21.15.83:0/318563908 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc154054480 con 0x5562517e8290 2024-04-17T10:42:09.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc13b7fe640 1 -- 172.21.15.83:0/318563908 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc15407c050 con 0x5562517e8290 2024-04-17T10:42:09.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc165edd000 10 client.6120 fetch_fsmap learned FSMap version 22 2024-04-17T10:42:09.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc165edd000 1 -- 172.21.15.83:0/318563908 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x5562515e6d00 con 0x5562517e8290 2024-04-17T10:42:09.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc13b7fe640 1 -- 172.21.15.83:0/318563908 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc154054480 con 0x5562517e8290 2024-04-17T10:42:09.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc13b7fe640 1 -- 172.21.15.83:0/318563908 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc1540527f0 con 0x5562517e8290 2024-04-17T10:42:09.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc165edd000 10 client.6120 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T10:42:09.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc165edd000 10 client.6120 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:42:09.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc165edd000 20 client.6120 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:09.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc165edd000 1 --2- 172.21.15.83:0/318563908 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x5562517a3550 0x5562517a3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc165edd000 4 client.6120 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:42:09.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc165edd000 1 -- 172.21.15.83:0/318563908 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556251587b10 con 0x5562517a3550 2024-04-17T10:42:09.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc160b0c640 1 --2- 172.21.15.83:0/318563908 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x5562517a3550 0x5562517a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc160b0c640 1 --2- 172.21.15.83:0/318563908 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x5562517a3550 0x5562517a3930 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c98758f483ac8f7 server_cookie=157085e4470af932 in_seq=0 out_seq=0 2024-04-17T10:42:09.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc13b7fe640 10 client.6120 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:09.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc13b7fe640 1 -- 172.21.15.83:0/318563908 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556251587b10 con 0x5562517a3550 2024-04-17T10:42:09.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc13b7fe640 10 client.6120 handle_command_reply: tid=0 2024-04-17T10:42:09.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc165edd000 1 client.6120 shutdown 2024-04-17T10:42:09.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc165edd000 1 -- 172.21.15.83:0/318563908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562517e8290 msgr2=0x5562517a27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:09.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc165edd000 1 --2- 172.21.15.83:0/318563908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562517e8290 0x5562517a27e0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc154056110 tx=0x7fc154007f50 comp rx=0 tx=0).stop 2024-04-17T10:42:09.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc165edd000 1 -- 172.21.15.83:0/318563908 shutdown_connections 2024-04-17T10:42:09.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc165edd000 1 --2- 172.21.15.83:0/318563908 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x5562517a3550 0x5562517a3930 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc165edd000 1 --2- 172.21.15.83:0/318563908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562517e8290 0x5562517a27e0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc165edd000 1 --2- 172.21.15.83:0/318563908 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5562517e7970 0x5562517ee200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc13b7fe640 0 client.6120 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:09.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc165edd000 1 --2- 172.21.15.83:0/318563908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55625179dcd0 0x5562517edcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc165edd000 1 -- 172.21.15.83:0/318563908 >> 172.21.15.83:0/318563908 conn(0x5562517a4540 msgr2=0x55625179c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:09.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc165edd000 1 -- 172.21.15.83:0/318563908 shutdown_connections 2024-04-17T10:42:09.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc165edd000 1 -- 172.21.15.83:0/318563908 wait complete. 2024-04-17T10:42:09.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.758+0000 7fc165edd000 20 client.6120 trim_cache size 0 max 16384 2024-04-17T10:42:09.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc165edd000 1 Processor -- start 2024-04-17T10:42:09.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc165edd000 1 -- start start 2024-04-17T10:42:09.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc165edd000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55625179dcd0 0x5562517ea680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc165edd000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562517e7970 0x5562517eabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc165edd000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5562517e8290 0x5562517eb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc165edd000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c202890 con 0x5562517e7970 2024-04-17T10:42:09.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc165edd000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c06cbc0 con 0x55625179dcd0 2024-04-17T10:42:09.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc165edd000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c06d1c0 con 0x5562517e8290 2024-04-17T10:42:09.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc15bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562517e7970 0x5562517eabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc15bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562517e7970 0x5562517eabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41130/0 (socket says 172.21.15.83:41130) 2024-04-17T10:42:09.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc15bfff640 1 -- 172.21.15.83:0/2445661669 learned_addr learned my addr 172.21.15.83:0/2445661669 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:09.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc160b0c640 1 --2- 172.21.15.83:0/2445661669 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55625179dcd0 0x5562517ea680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc16130d640 1 --2- 172.21.15.83:0/2445661669 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5562517e8290 0x5562517eb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc15bfff640 1 -- 172.21.15.83:0/2445661669 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5562517e8290 msgr2=0x5562517eb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:09.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc15bfff640 1 --2- 172.21.15.83:0/2445661669 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5562517e8290 0x5562517eb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc15bfff640 1 -- 172.21.15.83:0/2445661669 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55625179dcd0 msgr2=0x5562517ea680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:09.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc15bfff640 1 --2- 172.21.15.83:0/2445661669 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55625179dcd0 0x5562517ea680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc15bfff640 1 -- 172.21.15.83:0/2445661669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55625166f920 con 0x5562517e7970 2024-04-17T10:42:09.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc15bfff640 1 --2- 172.21.15.83:0/2445661669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562517e7970 0x5562517eabc0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fc14800e960 tx=0x7fc14800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67f9c766df79c53 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:09.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc145ffb640 1 -- 172.21.15.83:0/2445661669 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc148010b00 con 0x5562517e7970 2024-04-17T10:42:09.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc145ffb640 1 -- 172.21.15.83:0/2445661669 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc148010cc0 con 0x5562517e7970 2024-04-17T10:42:09.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc145ffb640 1 -- 172.21.15.83:0/2445661669 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc148004270 con 0x5562517e7970 2024-04-17T10:42:09.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc165edd000 1 -- 172.21.15.83:0/2445661669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562517e7970 msgr2=0x5562517eabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:09.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc165edd000 1 --2- 172.21.15.83:0/2445661669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562517e7970 0x5562517eabc0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fc14800e960 tx=0x7fc14800ee20 comp rx=0 tx=0).stop 2024-04-17T10:42:09.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc165edd000 1 -- 172.21.15.83:0/2445661669 shutdown_connections 2024-04-17T10:42:09.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc165edd000 1 --2- 172.21.15.83:0/2445661669 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5562517e8290 0x5562517eb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc165edd000 1 --2- 172.21.15.83:0/2445661669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562517e7970 0x5562517eabc0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc165edd000 1 --2- 172.21.15.83:0/2445661669 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55625179dcd0 0x5562517ea680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc165edd000 1 -- 172.21.15.83:0/2445661669 >> 172.21.15.83:0/2445661669 conn(0x5562517a4540 msgr2=0x5562517a2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:09.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc165edd000 1 -- 172.21.15.83:0/2445661669 shutdown_connections 2024-04-17T10:42:09.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc165edd000 1 -- 172.21.15.83:0/2445661669 wait complete. 2024-04-17T10:42:09.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc165edd000 1 -- start start 2024-04-17T10:42:09.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.774+0000 7fc165edd000 1 Processor -- start 2024-04-17T10:42:09.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc165edd000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55625179dcd0 0x556251850580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc165edd000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562517e7970 0x556251850ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc165edd000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562517e8290 0x55625180ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc165edd000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562515f0340 con 0x5562517e7970 2024-04-17T10:42:09.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc165edd000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562515f9e70 con 0x5562517e8290 2024-04-17T10:42:09.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc165edd000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55625166f490 con 0x55625179dcd0 2024-04-17T10:42:09.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc165edd000 10 client.0 _ll_register_callbacks cb 0x7fc1540099a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:09.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc15bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562517e7970 0x556251850ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc15bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562517e7970 0x556251850ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41138/0 (socket says 172.21.15.83:41138) 2024-04-17T10:42:09.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc15bfff640 1 -- 172.21.15.83:0/1827843117 learned_addr learned my addr 172.21.15.83:0/1827843117 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:09.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc15bfff640 1 -- 172.21.15.83:0/1827843117 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55625179dcd0 msgr2=0x556251850580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:09.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc160b0c640 1 --2- 172.21.15.83:0/1827843117 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55625179dcd0 0x556251850580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc16130d640 1 --2- 172.21.15.83:0/1827843117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562517e8290 0x55625180ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc15bfff640 1 --2- 172.21.15.83:0/1827843117 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55625179dcd0 0x556251850580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc15bfff640 1 -- 172.21.15.83:0/1827843117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562517e8290 msgr2=0x55625180ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:09.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc15bfff640 1 --2- 172.21.15.83:0/1827843117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562517e8290 0x55625180ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc15bfff640 1 -- 172.21.15.83:0/1827843117 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556251645a30 con 0x5562517e7970 2024-04-17T10:42:09.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc15bfff640 1 --2- 172.21.15.83:0/1827843117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562517e7970 0x556251850ac0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc148000c10 tx=0x7fc148004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac541b0ada293023 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:09.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc13b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:09.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc13b7fe640 1 -- 172.21.15.83:0/1827843117 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14802b060 con 0x5562517e7970 2024-04-17T10:42:09.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc13b7fe640 1 -- 172.21.15.83:0/1827843117 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc148003830 con 0x5562517e7970 2024-04-17T10:42:09.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc165edd000 1 -- 172.21.15.83:0/1827843117 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc148010cc0 con 0x5562517e7970 2024-04-17T10:42:09.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc13b7fe640 1 -- 172.21.15.83:0/1827843117 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc148031050 con 0x5562517e7970 2024-04-17T10:42:09.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc165edd000 10 client.6126 fetch_fsmap learned FSMap version 22 2024-04-17T10:42:09.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc165edd000 1 -- 172.21.15.83:0/1827843117 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x556251593150 con 0x5562517e7970 2024-04-17T10:42:09.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc13b7fe640 1 -- 172.21.15.83:0/1827843117 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc148010cc0 con 0x5562517e7970 2024-04-17T10:42:09.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc13b7fe640 1 -- 172.21.15.83:0/1827843117 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc148034020 con 0x5562517e7970 2024-04-17T10:42:09.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc165edd000 10 client.6126 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T10:42:09.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc165edd000 10 client.6126 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:42:09.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc165edd000 20 client.6126 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:09.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc165edd000 1 --2- 172.21.15.83:0/1827843117 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55625180b540 0x55625180b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc165edd000 4 client.6126 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}] 2024-04-17T10:42:09.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc165edd000 1 -- 172.21.15.83:0/1827843117 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}) v1 -- 0x5562515f9250 con 0x55625180b540 2024-04-17T10:42:09.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc160b0c640 1 --2- 172.21.15.83:0/1827843117 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55625180b540 0x55625180b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc13b7fe640 10 client.6126 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:09.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc160b0c640 1 --2- 172.21.15.83:0/1827843117 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55625180b540 0x55625180b950 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc3c7d115d80322 server_cookie=ed3d83548dfff969 in_seq=0 out_seq=0 2024-04-17T10:42:09.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc13b7fe640 1 -- 172.21.15.83:0/1827843117 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26 (crc 0 0 0) 0x5562515f9250 con 0x55625180b540 2024-04-17T10:42:09.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc13b7fe640 10 client.6126 handle_command_reply: tid=0 2024-04-17T10:42:09.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc165edd000 1 client.6126 shutdown 2024-04-17T10:42:09.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc165edd000 1 -- 172.21.15.83:0/1827843117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562517e7970 msgr2=0x556251850ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:09.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc165edd000 1 --2- 172.21.15.83:0/1827843117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562517e7970 0x556251850ac0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc148000c10 tx=0x7fc148004740 comp rx=0 tx=0).stop 2024-04-17T10:42:09.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc165edd000 1 -- 172.21.15.83:0/1827843117 shutdown_connections 2024-04-17T10:42:09.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc165edd000 1 --2- 172.21.15.83:0/1827843117 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55625180b540 0x55625180b950 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc165edd000 1 --2- 172.21.15.83:0/1827843117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5562517e8290 0x55625180ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc165edd000 1 --2- 172.21.15.83:0/1827843117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5562517e7970 0x556251850ac0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc13b7fe640 0 client.6126 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:09.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc165edd000 1 --2- 172.21.15.83:0/1827843117 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55625179dcd0 0x556251850580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc165edd000 1 -- 172.21.15.83:0/1827843117 >> 172.21.15.83:0/1827843117 conn(0x5562517a4540 msgr2=0x55625179bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:09.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc165edd000 1 -- 172.21.15.83:0/1827843117 shutdown_connections 2024-04-17T10:42:09.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc165edd000 1 -- 172.21.15.83:0/1827843117 wait complete. 2024-04-17T10:42:09.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc165edd000 20 client.6126 trim_cache size 0 max 16384 2024-04-17T10:42:09.784 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:42:09.785 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": -2 2024-04-17T10:42:09.785 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:42:09.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc162d70640 1 -- 172.21.15.83:0/2046931297 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc140066f60 msgr2=0x7fc140069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:09.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc162d70640 1 --2- 172.21.15.83:0/2046931297 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc140066f60 0x7fc140069420 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc1500ad000 tx=0x7fc1500a6000 comp rx=0 tx=0).stop 2024-04-17T10:42:09.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc162d70640 1 -- 172.21.15.83:0/2046931297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc15c15bc10 msgr2=0x7fc15c0cb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:09.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc162d70640 1 --2- 172.21.15.83:0/2046931297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc15c15bc10 0x7fc15c0cb040 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fc154007e60 tx=0x7fc15400e4b0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc162d70640 1 -- 172.21.15.83:0/2046931297 shutdown_connections 2024-04-17T10:42:09.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc162d70640 1 --2- 172.21.15.83:0/2046931297 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc140066f60 0x7fc140069420 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc162d70640 1 --2- 172.21.15.83:0/2046931297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc15c15bc10 0x7fc15c0cb040 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc162d70640 1 --2- 172.21.15.83:0/2046931297 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc15c0d0490 0x7fc15c0cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc162d70640 1 --2- 172.21.15.83:0/2046931297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc15c0cd3d0 0x7fc15c0c73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc162d70640 1 -- 172.21.15.83:0/2046931297 >> 172.21.15.83:0/2046931297 conn(0x7fc15c0d17e0 msgr2=0x7fc15c159410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:09.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc162d70640 1 -- 172.21.15.83:0/2046931297 shutdown_connections 2024-04-17T10:42:09.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.778+0000 7fc162d70640 1 -- 172.21.15.83:0/2046931297 wait complete. 2024-04-17T10:42:09.793 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/i/dont/exist' returned '{'return_code': -2}' 2024-04-17T10:42:09.793 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites' 2024-04-17T10:42:09.793 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-17T10:42:09.794 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites 2024-04-17T10:42:09.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.906+0000 7f76560b4640 1 Processor -- start 2024-04-17T10:42:09.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.906+0000 7f76560b4640 1 -- start start 2024-04-17T10:42:09.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f76560b4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f765006b520 0x7f76500b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f76560b4640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f76500b4040 0x7f765006cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f76560b4640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f765015bd70 con 0x7f7650154de0 2024-04-17T10:42:09.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f76560b4640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76500b4870 con 0x7f76500b4040 2024-04-17T10:42:09.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f76560b4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f765015bf60 con 0x7f765006b520 2024-04-17T10:42:09.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f764f7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7650154de0 0x7f765006ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46724/0 (socket says 172.21.15.83:46724) 2024-04-17T10:42:09.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f764f7fe640 1 -- 172.21.15.83:0/2102408828 learned_addr learned my addr 172.21.15.83:0/2102408828 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:09.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f764ffff640 1 --2- 172.21.15.83:0/2102408828 >> v2:172.21.15.167:3300/0 conn(0x7f76500b4040 0x7f765006cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f764effd640 1 -- 172.21.15.83:0/2102408828 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2409350878 0 0) 0x7f765015bd70 con 0x7f7650154de0 2024-04-17T10:42:09.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f764effd640 1 -- 172.21.15.83:0/2102408828 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f763c003530 con 0x7f7650154de0 2024-04-17T10:42:09.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f7646ffd640 1 --2- 172.21.15.83:0/2102408828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f765006b520 0x7f76500b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f764effd640 1 -- 172.21.15.83:0/2102408828 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 592754300 0 0) 0x7f763c003530 con 0x7f7650154de0 2024-04-17T10:42:09.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f764effd640 1 -- 172.21.15.83:0/2102408828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f765006b520 msgr2=0x7f76500b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:09.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f764effd640 1 --2- 172.21.15.83:0/2102408828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f765006b520 0x7f76500b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f764effd640 1 -- 172.21.15.83:0/2102408828 >> v2:172.21.15.167:3300/0 conn(0x7f76500b4040 msgr2=0x7f765006cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:09.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f764effd640 1 --2- 172.21.15.83:0/2102408828 >> v2:172.21.15.167:3300/0 conn(0x7f76500b4040 0x7f765006cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f764effd640 1 -- 172.21.15.83:0/2102408828 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765015afb0 con 0x7f7650154de0 2024-04-17T10:42:09.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f764effd640 1 -- 172.21.15.83:0/2102408828 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f7640001680 con 0x7f7650154de0 2024-04-17T10:42:09.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f764effd640 1 -- 172.21.15.83:0/2102408828 >> v1:172.21.15.83:6789/0 conn(0x7f7650154de0 legacy=0x7f765006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:09.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f764effd640 1 --2- 172.21.15.83:0/2102408828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f763c003a30 0x7f763c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f764effd640 1 --2- 172.21.15.83:0/2102408828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f763c004810 0x7f763c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f764effd640 1 --2- 172.21.15.83:0/2102408828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f763c0051d0 0x7f763c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f764effd640 1 -- 172.21.15.83:0/2102408828 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765015bf60 con 0x7f763c004810 2024-04-17T10:42:09.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f764effd640 1 -- 172.21.15.83:0/2102408828 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c00c020 con 0x7f763c0051d0 2024-04-17T10:42:09.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f764effd640 1 -- 172.21.15.83:0/2102408828 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f763c00c220 con 0x7f763c003a30 2024-04-17T10:42:09.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f7646ffd640 1 --2- 172.21.15.83:0/2102408828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f765006b520 0x7f76500b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:09.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f7646ffd640 1 --2- 172.21.15.83:0/2102408828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f763c004810 0x7f763c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f7646ffd640 1 -- 172.21.15.83:0/2102408828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f763c003a30 msgr2=0x7f763c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:09.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f7646ffd640 1 --2- 172.21.15.83:0/2102408828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f763c003a30 0x7f763c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f7646ffd640 1 -- 172.21.15.83:0/2102408828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f763c0051d0 msgr2=0x7f763c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:09.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f7646ffd640 1 --2- 172.21.15.83:0/2102408828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f763c0051d0 0x7f763c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f7646ffd640 1 -- 172.21.15.83:0/2102408828 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763c003530 con 0x7f763c004810 2024-04-17T10:42:09.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f7646ffd640 1 --2- 172.21.15.83:0/2102408828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f763c004810 0x7f763c004c90 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f76500b2db0 tx=0x7f763801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4fc258a981c663 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:09.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f764effd640 1 -- 172.21.15.83:0/2102408828 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763800a4d0 con 0x7f763c004810 2024-04-17T10:42:09.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f764effd640 1 -- 172.21.15.83:0/2102408828 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7638004220 con 0x7f763c004810 2024-04-17T10:42:09.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.910+0000 7f764effd640 1 -- 172.21.15.83:0/2102408828 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7638003620 con 0x7f763c004810 2024-04-17T10:42:09.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.914+0000 7f76560b4640 1 -- 172.21.15.83:0/2102408828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f763c004810 msgr2=0x7f763c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:09.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.914+0000 7f76560b4640 1 --2- 172.21.15.83:0/2102408828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f763c004810 0x7f763c004c90 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f76500b2db0 tx=0x7f763801c470 comp rx=0 tx=0).stop 2024-04-17T10:42:09.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.914+0000 7f764ffff640 1 -- 172.21.15.83:0/2102408828 reap_dead start 2024-04-17T10:42:09.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.914+0000 7f76560b4640 1 -- 172.21.15.83:0/2102408828 shutdown_connections 2024-04-17T10:42:09.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.914+0000 7f76560b4640 1 -- 172.21.15.83:0/2102408828 >> 172.21.15.83:0/2102408828 conn(0x7f76500b15b0 msgr2=0x7f76500b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:09.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.914+0000 7f76560b4640 1 -- 172.21.15.83:0/2102408828 shutdown_connections 2024-04-17T10:42:09.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.914+0000 7f76560b4640 1 -- 172.21.15.83:0/2102408828 wait complete. 2024-04-17T10:42:09.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.914+0000 7f76560b4640 1 Processor -- start 2024-04-17T10:42:09.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.914+0000 7f76560b4640 1 -- start start 2024-04-17T10:42:09.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.914+0000 7f76560b4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f763c004810 0x7f765014a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.914+0000 7f76560b4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7650155460 0x7f7650147b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.914+0000 7f76560b4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7650149a60 0x7f765015a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.914+0000 7f76560b4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765015b850 con 0x7f7650155460 2024-04-17T10:42:09.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.914+0000 7f76560b4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765015b570 con 0x7f763c004810 2024-04-17T10:42:09.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.914+0000 7f76560b4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76501551c0 con 0x7f7650149a60 2024-04-17T10:42:09.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.914+0000 7f7646ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7650155460 0x7f7650147b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.914+0000 7f7646ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7650155460 0x7f7650147b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41158/0 (socket says 172.21.15.83:41158) 2024-04-17T10:42:09.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.914+0000 7f7646ffd640 1 -- 172.21.15.83:0/2445945160 learned_addr learned my addr 172.21.15.83:0/2445945160 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:09.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.914+0000 7f764f7fe640 1 --2- 172.21.15.83:0/2445945160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f763c004810 0x7f765014a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.914+0000 7f7646ffd640 1 -- 172.21.15.83:0/2445945160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7650149a60 msgr2=0x7f765015a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:09.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.914+0000 7f764ffff640 1 --2- 172.21.15.83:0/2445945160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7650149a60 0x7f765015a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.914+0000 7f7646ffd640 1 --2- 172.21.15.83:0/2445945160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7650149a60 0x7f765015a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.914+0000 7f7646ffd640 1 -- 172.21.15.83:0/2445945160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f763c004810 msgr2=0x7f765014a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:09.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.914+0000 7f7646ffd640 1 --2- 172.21.15.83:0/2445945160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f763c004810 0x7f765014a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.914+0000 7f7646ffd640 1 -- 172.21.15.83:0/2445945160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7650155920 con 0x7f7650155460 2024-04-17T10:42:09.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.914+0000 7f764f7fe640 1 --2- 172.21.15.83:0/2445945160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f763c004810 0x7f765014a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:09.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.914+0000 7f7646ffd640 1 --2- 172.21.15.83:0/2445945160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7650155460 0x7f7650147b20 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f763801c950 tx=0x7f763c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:09.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.914+0000 7f764cff9640 1 -- 172.21.15.83:0/2445945160 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76380043d0 con 0x7f7650155460 2024-04-17T10:42:09.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.914+0000 7f764cff9640 1 -- 172.21.15.83:0/2445945160 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76380be070 con 0x7f7650155460 2024-04-17T10:42:09.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.914+0000 7f76560b4640 1 -- 172.21.15.83:0/2445945160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7650155c00 con 0x7f7650155460 2024-04-17T10:42:09.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.914+0000 7f764cff9640 1 -- 172.21.15.83:0/2445945160 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76380acd00 con 0x7f7650155460 2024-04-17T10:42:09.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.914+0000 7f76560b4640 1 -- 172.21.15.83:0/2445945160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76501561c0 con 0x7f7650155460 2024-04-17T10:42:09.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.918+0000 7f7659221000 1 Processor -- start 2024-04-17T10:42:09.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.918+0000 7f7659221000 1 -- start start 2024-04-17T10:42:09.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.918+0000 7f7659221000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977370eea0 0x55977370f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.918+0000 7f7659221000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55977370f7c0 0x5597736c51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.918+0000 7f7659221000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5597736c5710 0x5597736c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.918+0000 7f7659221000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559773513db0 con 0x55977370eea0 2024-04-17T10:42:09.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.918+0000 7f7659221000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55977349c230 con 0x55977370f7c0 2024-04-17T10:42:09.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.918+0000 7f7659221000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55977350fe50 con 0x5597736c5710 2024-04-17T10:42:09.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.918+0000 7f764ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977370eea0 0x55977370f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.918+0000 7f764cff9640 1 -- 172.21.15.83:0/2445945160 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f76380b5040 con 0x7f7650155460 2024-04-17T10:42:09.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.918+0000 7f764ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977370eea0 0x55977370f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41172/0 (socket says 172.21.15.83:41172) 2024-04-17T10:42:09.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.918+0000 7f764ffff640 1 -- 172.21.15.83:0/1893949309 learned_addr learned my addr 172.21.15.83:0/1893949309 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:09.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.918+0000 7f764ffff640 1 -- 172.21.15.83:0/1893949309 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5597736c5710 msgr2=0x5597736c5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:09.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.918+0000 7f764ffff640 1 --2- 172.21.15.83:0/1893949309 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5597736c5710 0x5597736c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.918+0000 7f764ffff640 1 -- 172.21.15.83:0/1893949309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55977370f7c0 msgr2=0x5597736c51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:09.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.918+0000 7f764ffff640 1 --2- 172.21.15.83:0/1893949309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55977370f7c0 0x5597736c51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.918+0000 7f764ffff640 1 -- 172.21.15.83:0/1893949309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597734d5750 con 0x55977370eea0 2024-04-17T10:42:09.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.918+0000 7f764cff9640 1 --2- 172.21.15.83:0/2445945160 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7630066f60 0x7f7630069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.918+0000 7f764ffff640 1 --2- 172.21.15.83:0/1893949309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977370eea0 0x55977370f280 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f76500cc3f0 tx=0x7f7648004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b87d994d9dac3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:09.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.918+0000 7f764cff9640 1 -- 172.21.15.83:0/2445945160 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f763811f360 con 0x7f7650155460 2024-04-17T10:42:09.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.918+0000 7f764f7fe640 1 --2- 172.21.15.83:0/2445945160 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7630066f60 0x7f7630069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.918+0000 7f76457fa640 1 -- 172.21.15.83:0/1893949309 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76480155f0 con 0x55977370eea0 2024-04-17T10:42:09.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.918+0000 7f76457fa640 1 -- 172.21.15.83:0/1893949309 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764801a070 con 0x55977370eea0 2024-04-17T10:42:09.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.918+0000 7f76457fa640 1 -- 172.21.15.83:0/1893949309 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76480159d0 con 0x55977370eea0 2024-04-17T10:42:09.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 1 -- 172.21.15.83:0/1893949309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977370eea0 msgr2=0x55977370f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:09.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f764f7fe640 1 --2- 172.21.15.83:0/2445945160 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7630066f60 0x7f7630069420 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7640004880 tx=0x7f764000c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:09.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 1 --2- 172.21.15.83:0/1893949309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977370eea0 0x55977370f280 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f76500cc3f0 tx=0x7f7648004a50 comp rx=0 tx=0).stop 2024-04-17T10:42:09.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 1 -- 172.21.15.83:0/1893949309 shutdown_connections 2024-04-17T10:42:09.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 1 --2- 172.21.15.83:0/1893949309 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5597736c5710 0x5597736c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 1 --2- 172.21.15.83:0/1893949309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55977370f7c0 0x5597736c51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 1 --2- 172.21.15.83:0/1893949309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977370eea0 0x55977370f280 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 1 -- 172.21.15.83:0/1893949309 >> 172.21.15.83:0/1893949309 conn(0x5597736c16c0 msgr2=0x5597736c3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:09.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 1 -- 172.21.15.83:0/1893949309 shutdown_connections 2024-04-17T10:42:09.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 1 -- 172.21.15.83:0/1893949309 wait complete. 2024-04-17T10:42:09.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 1 -- start start 2024-04-17T10:42:09.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 1 Processor -- start 2024-04-17T10:42:09.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597736c5710 0x5597736f3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977370eea0 0x5597736f4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55977370f7c0 0x5597736f4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597734cebf0 con 0x55977370eea0 2024-04-17T10:42:09.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559773498990 con 0x5597736c5710 2024-04-17T10:42:09.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x559773512d80 con 0x55977370f7c0 2024-04-17T10:42:09.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 10 client.0 _ll_register_callbacks cb 0x5597736b0790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:09.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7646ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977370eea0 0x5597736f4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7646ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977370eea0 0x5597736f4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41182/0 (socket says 172.21.15.83:41182) 2024-04-17T10:42:09.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7646ffd640 1 -- 172.21.15.83:0/2774446498 learned_addr learned my addr 172.21.15.83:0/2774446498 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:09.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7646ffd640 1 -- 172.21.15.83:0/2774446498 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55977370f7c0 msgr2=0x5597736f4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:09.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7646ffd640 1 --2- 172.21.15.83:0/2774446498 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55977370f7c0 0x5597736f4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7646ffd640 1 -- 172.21.15.83:0/2774446498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597736c5710 msgr2=0x5597736f3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:09.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7646ffd640 1 --2- 172.21.15.83:0/2774446498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597736c5710 0x5597736f3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7646ffd640 1 -- 172.21.15.83:0/2774446498 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5597734b6640 con 0x55977370eea0 2024-04-17T10:42:09.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7646ffd640 1 --2- 172.21.15.83:0/2774446498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977370eea0 0x5597736f4410 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f76380ec090 tx=0x7f76380ef680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a141533b0a7c760 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:09.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f762e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:09.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f762e7fc640 1 -- 172.21.15.83:0/2774446498 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76380f6070 con 0x55977370eea0 2024-04-17T10:42:09.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f762e7fc640 1 -- 172.21.15.83:0/2774446498 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f76380ef080 con 0x55977370eea0 2024-04-17T10:42:09.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 1 -- 172.21.15.83:0/2774446498 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f764801a070 con 0x55977370eea0 2024-04-17T10:42:09.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f762e7fc640 1 -- 172.21.15.83:0/2774446498 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76380efc30 con 0x55977370eea0 2024-04-17T10:42:09.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 10 client.6138 fetch_fsmap learned FSMap version 22 2024-04-17T10:42:09.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 1 -- 172.21.15.83:0/2774446498 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55977349c480 con 0x55977370eea0 2024-04-17T10:42:09.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f762e7fc640 1 -- 172.21.15.83:0/2774446498 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764801a070 con 0x55977370eea0 2024-04-17T10:42:09.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f762e7fc640 1 -- 172.21.15.83:0/2774446498 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f76380eb980 con 0x55977370eea0 2024-04-17T10:42:09.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 10 client.6138 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T10:42:09.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 10 client.6138 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:42:09.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 20 client.6138 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:09.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 1 --2- 172.21.15.83:0/2774446498 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55977371ef40 0x559773721330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 4 client.6138 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:42:09.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 1 -- 172.21.15.83:0/2774446498 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5597734afbb0 con 0x55977371ef40 2024-04-17T10:42:09.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f764f7fe640 1 --2- 172.21.15.83:0/2774446498 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55977371ef40 0x559773721330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f762e7fc640 10 client.6138 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:09.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f764f7fe640 1 --2- 172.21.15.83:0/2774446498 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55977371ef40 0x559773721330 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1eb2bb4655b156e server_cookie=45fd601b636ce731 in_seq=0 out_seq=0 2024-04-17T10:42:09.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f762e7fc640 1 -- 172.21.15.83:0/2774446498 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5597734afbb0 con 0x55977371ef40 2024-04-17T10:42:09.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f762e7fc640 10 client.6138 handle_command_reply: tid=0 2024-04-17T10:42:09.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 1 client.6138 shutdown 2024-04-17T10:42:09.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 1 -- 172.21.15.83:0/2774446498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977370eea0 msgr2=0x5597736f4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:09.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 1 --2- 172.21.15.83:0/2774446498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977370eea0 0x5597736f4410 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f76380ec090 tx=0x7f76380ef680 comp rx=0 tx=0).stop 2024-04-17T10:42:09.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 1 -- 172.21.15.83:0/2774446498 shutdown_connections 2024-04-17T10:42:09.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 1 --2- 172.21.15.83:0/2774446498 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55977371ef40 0x559773721330 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 1 --2- 172.21.15.83:0/2774446498 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55977370f7c0 0x5597736f4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f762e7fc640 0 client.6138 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:09.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 1 --2- 172.21.15.83:0/2774446498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977370eea0 0x5597736f4410 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 1 --2- 172.21.15.83:0/2774446498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597736c5710 0x5597736f3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.922+0000 7f7659221000 1 -- 172.21.15.83:0/2774446498 >> 172.21.15.83:0/2774446498 conn(0x5597736c16c0 msgr2=0x5597736c4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:09.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.926+0000 7f7659221000 1 -- 172.21.15.83:0/2774446498 shutdown_connections 2024-04-17T10:42:09.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.926+0000 7f7659221000 1 -- 172.21.15.83:0/2774446498 wait complete. 2024-04-17T10:42:09.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.926+0000 7f7659221000 20 client.6138 trim_cache size 0 max 16384 2024-04-17T10:42:09.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.938+0000 7f7659221000 1 Processor -- start 2024-04-17T10:42:09.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.938+0000 7f7659221000 1 -- start start 2024-04-17T10:42:09.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.938+0000 7f7659221000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5597736c5710 0x559773723090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.938+0000 7f7659221000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977370eea0 0x5597737235d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.938+0000 7f7659221000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55977370f7c0 0x55977371f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.938+0000 7f7659221000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7650200290 con 0x55977370eea0 2024-04-17T10:42:09.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.938+0000 7f7659221000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76501ff490 con 0x55977370f7c0 2024-04-17T10:42:09.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.938+0000 7f7659221000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76501514a0 con 0x5597736c5710 2024-04-17T10:42:09.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.938+0000 7f7646ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977370eea0 0x5597737235d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.938+0000 7f7646ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977370eea0 0x5597737235d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41184/0 (socket says 172.21.15.83:41184) 2024-04-17T10:42:09.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.938+0000 7f7646ffd640 1 -- 172.21.15.83:0/978602072 learned_addr learned my addr 172.21.15.83:0/978602072 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:09.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7646ffd640 1 -- 172.21.15.83:0/978602072 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5597736c5710 msgr2=0x559773723090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:09.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7646ffd640 1 --2- 172.21.15.83:0/978602072 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5597736c5710 0x559773723090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7646ffd640 1 -- 172.21.15.83:0/978602072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55977370f7c0 msgr2=0x55977371f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:09.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7646ffd640 1 --2- 172.21.15.83:0/978602072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55977370f7c0 0x55977371f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7646ffd640 1 -- 172.21.15.83:0/978602072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597736c95c0 con 0x55977370eea0 2024-04-17T10:42:09.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7646ffd640 1 --2- 172.21.15.83:0/978602072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977370eea0 0x5597737235d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f763801c9e0 tx=0x7f76380c4000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab2bc768e1b51c1f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:09.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7644ff9640 1 -- 172.21.15.83:0/978602072 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76380f6070 con 0x55977370eea0 2024-04-17T10:42:09.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7644ff9640 1 -- 172.21.15.83:0/978602072 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76380f8340 con 0x55977370eea0 2024-04-17T10:42:09.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7644ff9640 1 -- 172.21.15.83:0/978602072 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76380ef590 con 0x55977370eea0 2024-04-17T10:42:09.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7659221000 1 -- 172.21.15.83:0/978602072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977370eea0 msgr2=0x5597737235d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:09.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7659221000 1 --2- 172.21.15.83:0/978602072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977370eea0 0x5597737235d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f763801c9e0 tx=0x7f76380c4000 comp rx=0 tx=0).stop 2024-04-17T10:42:09.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7659221000 1 -- 172.21.15.83:0/978602072 shutdown_connections 2024-04-17T10:42:09.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7659221000 1 --2- 172.21.15.83:0/978602072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55977370f7c0 0x55977371f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7659221000 1 --2- 172.21.15.83:0/978602072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977370eea0 0x5597737235d0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7659221000 1 --2- 172.21.15.83:0/978602072 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5597736c5710 0x559773723090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7659221000 1 -- 172.21.15.83:0/978602072 >> 172.21.15.83:0/978602072 conn(0x5597736c16c0 msgr2=0x55977370dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:09.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7659221000 1 -- 172.21.15.83:0/978602072 shutdown_connections 2024-04-17T10:42:09.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7659221000 1 -- 172.21.15.83:0/978602072 wait complete. 2024-04-17T10:42:09.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7659221000 1 -- start start 2024-04-17T10:42:09.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7659221000 1 Processor -- start 2024-04-17T10:42:09.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7659221000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597736c5710 0x559773722a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7659221000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55977370eea0 0x55977376fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7659221000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977370f7c0 0x5597737700c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7659221000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76501feff0 con 0x55977370f7c0 2024-04-17T10:42:09.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7659221000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559773512d80 con 0x5597736c5710 2024-04-17T10:42:09.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7659221000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5597734da940 con 0x55977370eea0 2024-04-17T10:42:09.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7659221000 10 client.0 _ll_register_callbacks cb 0x7f7648009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:09.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f764ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977370f7c0 0x5597737700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f764ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977370f7c0 0x5597737700c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41198/0 (socket says 172.21.15.83:41198) 2024-04-17T10:42:09.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f764ffff640 1 -- 172.21.15.83:0/3537917334 learned_addr learned my addr 172.21.15.83:0/3537917334 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:09.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f764ffff640 1 -- 172.21.15.83:0/3537917334 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55977370eea0 msgr2=0x55977376fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:09.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f764ffff640 1 --2- 172.21.15.83:0/3537917334 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55977370eea0 0x55977376fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f764ffff640 1 -- 172.21.15.83:0/3537917334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597736c5710 msgr2=0x559773722a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:09.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f764f7fe640 1 --2- 172.21.15.83:0/3537917334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597736c5710 0x559773722a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f764ffff640 1 --2- 172.21.15.83:0/3537917334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597736c5710 0x559773722a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f764ffff640 1 -- 172.21.15.83:0/3537917334 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5597736c1450 con 0x55977370f7c0 2024-04-17T10:42:09.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f764ffff640 1 --2- 172.21.15.83:0/3537917334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977370f7c0 0x5597737700c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x559773519d90 tx=0x7f7648004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b58b3c8eba482dd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:09.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f762e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:09.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f762e7fc640 1 -- 172.21.15.83:0/3537917334 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7648024020 con 0x55977370f7c0 2024-04-17T10:42:09.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f762e7fc640 1 -- 172.21.15.83:0/3537917334 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f76480047f0 con 0x55977370f7c0 2024-04-17T10:42:09.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7659221000 1 -- 172.21.15.83:0/3537917334 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f76380f8340 con 0x55977370f7c0 2024-04-17T10:42:09.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f762e7fc640 1 -- 172.21.15.83:0/3537917334 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f764802e050 con 0x55977370f7c0 2024-04-17T10:42:09.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7659221000 10 client.6144 fetch_fsmap learned FSMap version 22 2024-04-17T10:42:09.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7659221000 1 -- 172.21.15.83:0/3537917334 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x559773595ef0 con 0x55977370f7c0 2024-04-17T10:42:09.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f762e7fc640 1 -- 172.21.15.83:0/3537917334 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76380f8340 con 0x55977370f7c0 2024-04-17T10:42:09.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f762e7fc640 1 -- 172.21.15.83:0/3537917334 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7648031020 con 0x55977370f7c0 2024-04-17T10:42:09.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7659221000 10 client.6144 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T10:42:09.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7659221000 10 client.6144 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:42:09.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7659221000 20 client.6144 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:09.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7659221000 1 --2- 172.21.15.83:0/3537917334 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55977372cdc0 0x55977372d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:09.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7659221000 4 client.6144 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}] 2024-04-17T10:42:09.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f7659221000 1 -- 172.21.15.83:0/3537917334 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}) v1 -- 0x5597734c3080 con 0x55977372cdc0 2024-04-17T10:42:09.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.942+0000 7f764f7fe640 1 --2- 172.21.15.83:0/3537917334 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55977372cdc0 0x55977372d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:09.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.946+0000 7f764f7fe640 1 --2- 172.21.15.83:0/3537917334 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55977372cdc0 0x55977372d1d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=157e00b130c75f82 server_cookie=2936541eb6c099e3 in_seq=0 out_seq=0 2024-04-17T10:42:09.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.946+0000 7f762e7fc640 10 client.6144 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:09.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.946+0000 7f762e7fc640 1 -- 172.21.15.83:0/3537917334 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5597734c3080 con 0x55977372cdc0 2024-04-17T10:42:09.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.946+0000 7f762e7fc640 10 client.6144 handle_command_reply: tid=0 2024-04-17T10:42:09.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.946+0000 7f7659221000 1 client.6144 shutdown 2024-04-17T10:42:09.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.946+0000 7f7659221000 1 -- 172.21.15.83:0/3537917334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977370f7c0 msgr2=0x5597737700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:09.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.946+0000 7f7659221000 1 --2- 172.21.15.83:0/3537917334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977370f7c0 0x5597737700c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x559773519d90 tx=0x7f7648004270 comp rx=0 tx=0).stop 2024-04-17T10:42:09.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.946+0000 7f7659221000 1 -- 172.21.15.83:0/3537917334 shutdown_connections 2024-04-17T10:42:09.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.946+0000 7f7659221000 1 --2- 172.21.15.83:0/3537917334 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55977372cdc0 0x55977372d1d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.946+0000 7f7659221000 1 --2- 172.21.15.83:0/3537917334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977370f7c0 0x5597737700c0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.946+0000 7f7659221000 1 --2- 172.21.15.83:0/3537917334 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55977370eea0 0x55977376fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.946+0000 7f7659221000 1 --2- 172.21.15.83:0/3537917334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597736c5710 0x559773722a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.946+0000 7f7659221000 1 -- 172.21.15.83:0/3537917334 >> 172.21.15.83:0/3537917334 conn(0x5597736c16c0 msgr2=0x5597736c3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:09.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.946+0000 7f762e7fc640 0 client.6144 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:09.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.946+0000 7f7659221000 1 -- 172.21.15.83:0/3537917334 shutdown_connections 2024-04-17T10:42:09.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.946+0000 7f7659221000 1 -- 172.21.15.83:0/3537917334 wait complete. 2024-04-17T10:42:09.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.946+0000 7f7659221000 20 client.6144 trim_cache size 0 max 16384 2024-04-17T10:42:09.951 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:42:09.951 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-17T10:42:09.951 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "2d1e3bb3-41c9-4c4e-97b9-84c4f42ea491", 2024-04-17T10:42:09.951 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-17T10:42:09.951 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:42:09.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.946+0000 7f76560b4640 1 -- 172.21.15.83:0/2445945160 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7630066f60 msgr2=0x7f7630069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:09.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.946+0000 7f76560b4640 1 --2- 172.21.15.83:0/2445945160 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7630066f60 0x7f7630069420 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7640004880 tx=0x7f764000c040 comp rx=0 tx=0).stop 2024-04-17T10:42:09.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.946+0000 7f76560b4640 1 -- 172.21.15.83:0/2445945160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7650155460 msgr2=0x7f7650147b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:09.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.946+0000 7f76560b4640 1 --2- 172.21.15.83:0/2445945160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7650155460 0x7f7650147b20 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f763801c950 tx=0x7f763c005840 comp rx=0 tx=0).stop 2024-04-17T10:42:09.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.946+0000 7f76560b4640 1 -- 172.21.15.83:0/2445945160 shutdown_connections 2024-04-17T10:42:09.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.946+0000 7f76560b4640 1 --2- 172.21.15.83:0/2445945160 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7630066f60 0x7f7630069420 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.946+0000 7f76560b4640 1 --2- 172.21.15.83:0/2445945160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7650149a60 0x7f765015a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.946+0000 7f76560b4640 1 --2- 172.21.15.83:0/2445945160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7650155460 0x7f7650147b20 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.946+0000 7f76560b4640 1 --2- 172.21.15.83:0/2445945160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f763c004810 0x7f765014a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:09.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.946+0000 7f76560b4640 1 -- 172.21.15.83:0/2445945160 >> 172.21.15.83:0/2445945160 conn(0x7f76500b15b0 msgr2=0x7f76500b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:09.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.946+0000 7f76560b4640 1 -- 172.21.15.83:0/2445945160 shutdown_connections 2024-04-17T10:42:09.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:09.946+0000 7f76560b4640 1 -- 172.21.15.83:0/2445945160 wait complete. 2024-04-17T10:42:09.959 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites' returned '{'return_code': 0, 'scrub_tag': '2d1e3bb3-41c9-4c4e-97b9-84c4f42ea491', 'mode': 'asynchronous'}' 2024-04-17T10:42:09.959 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-17T10:42:09.959 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-17T10:42:09.959 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-17T10:42:10.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.078+0000 7fb15a9c7640 1 Processor -- start 2024-04-17T10:42:10.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.078+0000 7fb15a9c7640 1 -- start start 2024-04-17T10:42:10.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.078+0000 7fb15a9c7640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb1540b5b60 0x7fb1540b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.078+0000 7fb15a9c7640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb15415e4d0 con 0x7fb15415e0f0 2024-04-17T10:42:10.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.078+0000 7fb15a9c7640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb154162580 con 0x7fb1540ce400 2024-04-17T10:42:10.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.078+0000 7fb15a9c7640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb154162b40 con 0x7fb1540b5b60 2024-04-17T10:42:10.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.078+0000 7fb158f64640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb1540b5b60 0x7fb1540b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.078+0000 7fb153fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb15415e0f0 0x7fb154161c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46726/0 (socket says 172.21.15.83:46726) 2024-04-17T10:42:10.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.078+0000 7fb153fff640 1 -- 172.21.15.83:0/1827594855 learned_addr learned my addr 172.21.15.83:0/1827594855 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:10.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.078+0000 7fb152ffd640 1 -- 172.21.15.83:0/1827594855 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3297013246 0 0) 0x7fb15415e4d0 con 0x7fb15415e0f0 2024-04-17T10:42:10.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.078+0000 7fb152ffd640 1 -- 172.21.15.83:0/1827594855 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb13c003530 con 0x7fb15415e0f0 2024-04-17T10:42:10.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.078+0000 7fb158f64640 1 -- 172.21.15.83:0/1827594855 >> v1:172.21.15.167:6789/0 conn(0x7fb1540ce400 legacy=0x7fb1540b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:10.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.078+0000 7fb158f64640 1 -- 172.21.15.83:0/1827594855 >> v1:172.21.15.83:6789/0 conn(0x7fb15415e0f0 legacy=0x7fb154161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:10.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.078+0000 7fb158f64640 1 -- 172.21.15.83:0/1827594855 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb154162860 con 0x7fb1540b5b60 2024-04-17T10:42:10.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.082+0000 7fb158f64640 1 --2- 172.21.15.83:0/1827594855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1540b5b60 0x7fb1540b6d00 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fb14c009d90 tx=0x7fb14c01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e4ec9bf94197cc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:10.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.082+0000 7fb152ffd640 1 -- 172.21.15.83:0/1827594855 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14c0b2070 con 0x7fb1540b5b60 2024-04-17T10:42:10.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.082+0000 7fb152ffd640 1 -- 172.21.15.83:0/1827594855 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb14c0a57a0 con 0x7fb1540b5b60 2024-04-17T10:42:10.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.082+0000 7fb152ffd640 1 -- 172.21.15.83:0/1827594855 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14c0a5b70 con 0x7fb1540b5b60 2024-04-17T10:42:10.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.082+0000 7fb15a9c7640 1 -- 172.21.15.83:0/1827594855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1540b5b60 msgr2=0x7fb1540b6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:10.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.082+0000 7fb15a9c7640 1 --2- 172.21.15.83:0/1827594855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1540b5b60 0x7fb1540b6d00 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fb14c009d90 tx=0x7fb14c01c900 comp rx=0 tx=0).stop 2024-04-17T10:42:10.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.082+0000 7fb15a9c7640 1 -- 172.21.15.83:0/1827594855 shutdown_connections 2024-04-17T10:42:10.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.082+0000 7fb15a9c7640 1 --2- 172.21.15.83:0/1827594855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1540b5b60 0x7fb1540b6d00 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.082+0000 7fb15a9c7640 1 -- 172.21.15.83:0/1827594855 >> 172.21.15.83:0/1827594855 conn(0x7fb1540d0450 msgr2=0x7fb1540d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:10.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.082+0000 7fb15a9c7640 1 -- 172.21.15.83:0/1827594855 shutdown_connections 2024-04-17T10:42:10.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.082+0000 7fb15a9c7640 1 -- 172.21.15.83:0/1827594855 wait complete. 2024-04-17T10:42:10.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.082+0000 7fb15a9c7640 1 Processor -- start 2024-04-17T10:42:10.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.082+0000 7fb15a9c7640 1 -- start start 2024-04-17T10:42:10.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.082+0000 7fb15a9c7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1540b5b60 0x7fb1540c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.082+0000 7fb15a9c7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1540ce400 0x7fb15415e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.082+0000 7fb15a9c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb15415e0f0 0x7fb1541568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.082+0000 7fb15a9c7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1540d30e0 con 0x7fb15415e0f0 2024-04-17T10:42:10.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.082+0000 7fb15a9c7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1540d2f60 con 0x7fb1540ce400 2024-04-17T10:42:10.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.082+0000 7fb15a9c7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1540d3260 con 0x7fb1540b5b60 2024-04-17T10:42:10.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.082+0000 7fb158f64640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb15415e0f0 0x7fb1541568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.082+0000 7fb158f64640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb15415e0f0 0x7fb1541568d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41216/0 (socket says 172.21.15.83:41216) 2024-04-17T10:42:10.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.082+0000 7fb158f64640 1 -- 172.21.15.83:0/1458939429 learned_addr learned my addr 172.21.15.83:0/1458939429 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:10.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.082+0000 7fb1537fe640 1 --2- 172.21.15.83:0/1458939429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1540ce400 0x7fb15415e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.082+0000 7fb153fff640 1 --2- 172.21.15.83:0/1458939429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1540b5b60 0x7fb1540c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.086+0000 7fb158f64640 1 -- 172.21.15.83:0/1458939429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1540b5b60 msgr2=0x7fb1540c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:10.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.086+0000 7fb158f64640 1 --2- 172.21.15.83:0/1458939429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1540b5b60 0x7fb1540c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.086+0000 7fb158f64640 1 -- 172.21.15.83:0/1458939429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1540ce400 msgr2=0x7fb15415e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:10.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.086+0000 7fb158f64640 1 --2- 172.21.15.83:0/1458939429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1540ce400 0x7fb15415e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.086+0000 7fb158f64640 1 -- 172.21.15.83:0/1458939429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb15415eff0 con 0x7fb15415e0f0 2024-04-17T10:42:10.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.086+0000 7fb1537fe640 1 --2- 172.21.15.83:0/1458939429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1540ce400 0x7fb15415e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:10.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.086+0000 7fb158f64640 1 --2- 172.21.15.83:0/1458939429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb15415e0f0 0x7fb1541568d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fb14c01a530 tx=0x7fb14c0196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:10.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.086+0000 7fb150ff9640 1 -- 172.21.15.83:0/1458939429 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14c0bd020 con 0x7fb15415e0f0 2024-04-17T10:42:10.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.086+0000 7fb150ff9640 1 -- 172.21.15.83:0/1458939429 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb14c0b8470 con 0x7fb15415e0f0 2024-04-17T10:42:10.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.086+0000 7fb15a9c7640 1 -- 172.21.15.83:0/1458939429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb15415f2d0 con 0x7fb15415e0f0 2024-04-17T10:42:10.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.086+0000 7fb150ff9640 1 -- 172.21.15.83:0/1458939429 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14c0b2040 con 0x7fb15415e0f0 2024-04-17T10:42:10.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.086+0000 7fb15a9c7640 1 -- 172.21.15.83:0/1458939429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb15415f890 con 0x7fb15415e0f0 2024-04-17T10:42:10.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.086+0000 7fb150ff9640 1 -- 172.21.15.83:0/1458939429 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb14c0c0020 con 0x7fb15415e0f0 2024-04-17T10:42:10.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.086+0000 7fb150ff9640 1 --2- 172.21.15.83:0/1458939429 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb140066f60 0x7fb140069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.086+0000 7fb150ff9640 1 -- 172.21.15.83:0/1458939429 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb14c11fe30 con 0x7fb15415e0f0 2024-04-17T10:42:10.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.086+0000 7fb153fff640 1 --2- 172.21.15.83:0/1458939429 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb140066f60 0x7fb140069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.086+0000 7fb153fff640 1 --2- 172.21.15.83:0/1458939429 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb140066f60 0x7fb140069420 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb1440011f0 tx=0x7fb144009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:10.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.086+0000 7fb15db34000 1 Processor -- start 2024-04-17T10:42:10.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.086+0000 7fb15db34000 1 -- start start 2024-04-17T10:42:10.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.086+0000 7fb15db34000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560dcba90200 0x560dcba905e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.086+0000 7fb15db34000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dcba90b20 0x560dcba74a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.086+0000 7fb15db34000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dcba74f60 0x560dcba75340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.086+0000 7fb15db34000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dcb92d4a0 con 0x560dcba90b20 2024-04-17T10:42:10.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.086+0000 7fb15db34000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dcb895210 con 0x560dcba74f60 2024-04-17T10:42:10.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.086+0000 7fb15db34000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x560dcb81d980 con 0x560dcba90200 2024-04-17T10:42:10.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb1537fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dcba90b20 0x560dcba74a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb1537fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dcba90b20 0x560dcba74a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41220/0 (socket says 172.21.15.83:41220) 2024-04-17T10:42:10.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb1537fe640 1 -- 172.21.15.83:0/1694478302 learned_addr learned my addr 172.21.15.83:0/1694478302 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:10.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb158f64640 1 --2- 172.21.15.83:0/1694478302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dcba74f60 0x560dcba75340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb153fff640 1 --2- 172.21.15.83:0/1694478302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560dcba90200 0x560dcba905e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb1537fe640 1 -- 172.21.15.83:0/1694478302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560dcba90200 msgr2=0x560dcba905e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:10.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb1537fe640 1 --2- 172.21.15.83:0/1694478302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560dcba90200 0x560dcba905e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb1537fe640 1 -- 172.21.15.83:0/1694478302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dcba74f60 msgr2=0x560dcba75340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:10.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb1537fe640 1 --2- 172.21.15.83:0/1694478302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dcba74f60 0x560dcba75340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb1537fe640 1 -- 172.21.15.83:0/1694478302 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560dcb890200 con 0x560dcba90b20 2024-04-17T10:42:10.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb1537fe640 1 --2- 172.21.15.83:0/1694478302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dcba90b20 0x560dcba74a20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fb148001760 tx=0x7fb148001c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92129d95c03200c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:10.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb139ffb640 1 -- 172.21.15.83:0/1694478302 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb148018070 con 0x560dcba90b20 2024-04-17T10:42:10.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb139ffb640 1 -- 172.21.15.83:0/1694478302 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb14800d730 con 0x560dcba90b20 2024-04-17T10:42:10.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb139ffb640 1 -- 172.21.15.83:0/1694478302 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14800db00 con 0x560dcba90b20 2024-04-17T10:42:10.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb15db34000 1 -- 172.21.15.83:0/1694478302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dcba90b20 msgr2=0x560dcba74a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:10.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb15db34000 1 --2- 172.21.15.83:0/1694478302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dcba90b20 0x560dcba74a20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fb148001760 tx=0x7fb148001c20 comp rx=0 tx=0).stop 2024-04-17T10:42:10.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb15db34000 1 -- 172.21.15.83:0/1694478302 shutdown_connections 2024-04-17T10:42:10.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb15db34000 1 --2- 172.21.15.83:0/1694478302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dcba74f60 0x560dcba75340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb15db34000 1 --2- 172.21.15.83:0/1694478302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dcba90b20 0x560dcba74a20 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb15db34000 1 --2- 172.21.15.83:0/1694478302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560dcba90200 0x560dcba905e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb15db34000 1 -- 172.21.15.83:0/1694478302 >> 172.21.15.83:0/1694478302 conn(0x560dcba3d4f0 msgr2=0x560dcba4ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:10.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb15db34000 1 -- 172.21.15.83:0/1694478302 shutdown_connections 2024-04-17T10:42:10.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb15db34000 1 -- 172.21.15.83:0/1694478302 wait complete. 2024-04-17T10:42:10.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb15db34000 1 -- start start 2024-04-17T10:42:10.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb15db34000 1 Processor -- start 2024-04-17T10:42:10.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb15db34000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560dcba74f60 0x560dcba97980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb15db34000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dcba90200 0x560dcba97ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb15db34000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dcba90b20 0x560dcba75340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb15db34000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dcb84fd80 con 0x560dcba90b20 2024-04-17T10:42:10.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb15db34000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dcb8912b0 con 0x560dcba90200 2024-04-17T10:42:10.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb15db34000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x560dcb856830 con 0x560dcba74f60 2024-04-17T10:42:10.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb15db34000 10 client.0 _ll_register_callbacks cb 0x560dcb948070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:10.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb158f64640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dcba90b20 0x560dcba75340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb158f64640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dcba90b20 0x560dcba75340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41232/0 (socket says 172.21.15.83:41232) 2024-04-17T10:42:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb158f64640 1 -- 172.21.15.83:0/2411592 learned_addr learned my addr 172.21.15.83:0/2411592 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb1537fe640 1 --2- 172.21.15.83:0/2411592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dcba90200 0x560dcba97ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb153fff640 1 --2- 172.21.15.83:0/2411592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560dcba74f60 0x560dcba97980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb158f64640 1 -- 172.21.15.83:0/2411592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560dcba74f60 msgr2=0x560dcba97980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb158f64640 1 --2- 172.21.15.83:0/2411592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560dcba74f60 0x560dcba97980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb158f64640 1 -- 172.21.15.83:0/2411592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dcba90200 msgr2=0x560dcba97ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb158f64640 1 --2- 172.21.15.83:0/2411592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dcba90200 0x560dcba97ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb158f64640 1 -- 172.21.15.83:0/2411592 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560dcb861130 con 0x560dcba90b20 2024-04-17T10:42:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb158f64640 1 --2- 172.21.15.83:0/2411592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dcba90b20 0x560dcba75340 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fb14c0f0840 tx=0x7fb14c0f0970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ef1a225f7d98e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb132ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb132ffd640 1 -- 172.21.15.83:0/2411592 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14c1081f0 con 0x560dcba90b20 2024-04-17T10:42:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb132ffd640 1 -- 172.21.15.83:0/2411592 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb14c108390 con 0x560dcba90b20 2024-04-17T10:42:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb15db34000 1 -- 172.21.15.83:0/2411592 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb14800d730 con 0x560dcba90b20 2024-04-17T10:42:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb132ffd640 1 -- 172.21.15.83:0/2411592 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb14c0f1be0 con 0x560dcba90b20 2024-04-17T10:42:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb15db34000 10 client.6159 fetch_fsmap learned FSMap version 22 2024-04-17T10:42:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb15db34000 1 -- 172.21.15.83:0/2411592 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x560dcb916490 con 0x560dcba90b20 2024-04-17T10:42:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.090+0000 7fb132ffd640 1 -- 172.21.15.83:0/2411592 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb14800d730 con 0x560dcba90b20 2024-04-17T10:42:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.094+0000 7fb132ffd640 1 -- 172.21.15.83:0/2411592 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb14c0f8b30 con 0x560dcba90b20 2024-04-17T10:42:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.094+0000 7fb15db34000 10 client.6159 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T10:42:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.094+0000 7fb15db34000 10 client.6159 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:42:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.094+0000 7fb15db34000 20 client.6159 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.094+0000 7fb15db34000 1 --2- 172.21.15.83:0/2411592 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x560dcba76080 0x560dcba76460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.094+0000 7fb15db34000 4 client.6159 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:42:10.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.094+0000 7fb15db34000 1 -- 172.21.15.83:0/2411592 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560dcb831020 con 0x560dcba76080 2024-04-17T10:42:10.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.094+0000 7fb153fff640 1 --2- 172.21.15.83:0/2411592 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x560dcba76080 0x560dcba76460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.094+0000 7fb153fff640 1 --2- 172.21.15.83:0/2411592 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x560dcba76080 0x560dcba76460 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f55d184f105d41a7 server_cookie=c13adb187923f3a1 in_seq=0 out_seq=0 2024-04-17T10:42:10.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.094+0000 7fb132ffd640 10 client.6159 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:10.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.094+0000 7fb132ffd640 1 -- 172.21.15.83:0/2411592 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560dcb831020 con 0x560dcba76080 2024-04-17T10:42:10.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.094+0000 7fb132ffd640 10 client.6159 handle_command_reply: tid=0 2024-04-17T10:42:10.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.094+0000 7fb15db34000 1 client.6159 shutdown 2024-04-17T10:42:10.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.094+0000 7fb15db34000 1 -- 172.21.15.83:0/2411592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dcba90b20 msgr2=0x560dcba75340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:10.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.094+0000 7fb15db34000 1 --2- 172.21.15.83:0/2411592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dcba90b20 0x560dcba75340 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fb14c0f0840 tx=0x7fb14c0f0970 comp rx=0 tx=0).stop 2024-04-17T10:42:10.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.094+0000 7fb15db34000 1 -- 172.21.15.83:0/2411592 shutdown_connections 2024-04-17T10:42:10.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.094+0000 7fb15db34000 1 --2- 172.21.15.83:0/2411592 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x560dcba76080 0x560dcba76460 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.094+0000 7fb15db34000 1 --2- 172.21.15.83:0/2411592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dcba90b20 0x560dcba75340 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.094+0000 7fb15db34000 1 --2- 172.21.15.83:0/2411592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dcba90200 0x560dcba97ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.094+0000 7fb132ffd640 0 client.6159 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:10.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.094+0000 7fb15db34000 1 --2- 172.21.15.83:0/2411592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560dcba74f60 0x560dcba97980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.094+0000 7fb15db34000 1 -- 172.21.15.83:0/2411592 >> 172.21.15.83:0/2411592 conn(0x560dcba3d4f0 msgr2=0x560dcba44aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:10.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.094+0000 7fb15db34000 1 -- 172.21.15.83:0/2411592 shutdown_connections 2024-04-17T10:42:10.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.094+0000 7fb15db34000 1 -- 172.21.15.83:0/2411592 wait complete. 2024-04-17T10:42:10.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.094+0000 7fb15db34000 20 client.6159 trim_cache size 0 max 16384 2024-04-17T10:42:10.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.106+0000 7fb15db34000 1 Processor -- start 2024-04-17T10:42:10.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.106+0000 7fb15db34000 1 -- start start 2024-04-17T10:42:10.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb15db34000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dcba74f60 0x560dcba8f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb15db34000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dcba90200 0x560dcba8e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb15db34000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560dcba90b20 0x560dcba8ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb15db34000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb154202bd0 con 0x560dcba74f60 2024-04-17T10:42:10.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb15db34000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb154201d90 con 0x560dcba90200 2024-04-17T10:42:10.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb15db34000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb154155870 con 0x560dcba90b20 2024-04-17T10:42:10.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb153fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dcba74f60 0x560dcba8f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb153fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dcba74f60 0x560dcba8f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41242/0 (socket says 172.21.15.83:41242) 2024-04-17T10:42:10.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb153fff640 1 -- 172.21.15.83:0/2544516541 learned_addr learned my addr 172.21.15.83:0/2544516541 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:10.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb153fff640 1 -- 172.21.15.83:0/2544516541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560dcba90b20 msgr2=0x560dcba8ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:10.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb153fff640 1 --2- 172.21.15.83:0/2544516541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560dcba90b20 0x560dcba8ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb153fff640 1 -- 172.21.15.83:0/2544516541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dcba90200 msgr2=0x560dcba8e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:10.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb153fff640 1 --2- 172.21.15.83:0/2544516541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dcba90200 0x560dcba8e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb153fff640 1 -- 172.21.15.83:0/2544516541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560dcb854a30 con 0x560dcba74f60 2024-04-17T10:42:10.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb153fff640 1 --2- 172.21.15.83:0/2544516541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dcba74f60 0x560dcba8f950 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb144005df0 tx=0x7fb144011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2778145fdbf574 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:10.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb1397fa640 1 -- 172.21.15.83:0/2544516541 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb144023850 con 0x560dcba74f60 2024-04-17T10:42:10.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb1397fa640 1 -- 172.21.15.83:0/2544516541 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb144007550 con 0x560dcba74f60 2024-04-17T10:42:10.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb1397fa640 1 -- 172.21.15.83:0/2544516541 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14400d6a0 con 0x560dcba74f60 2024-04-17T10:42:10.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb15db34000 1 -- 172.21.15.83:0/2544516541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dcba74f60 msgr2=0x560dcba8f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:10.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb15db34000 1 --2- 172.21.15.83:0/2544516541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dcba74f60 0x560dcba8f950 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb144005df0 tx=0x7fb144011000 comp rx=0 tx=0).stop 2024-04-17T10:42:10.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb15db34000 1 -- 172.21.15.83:0/2544516541 shutdown_connections 2024-04-17T10:42:10.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb15db34000 1 --2- 172.21.15.83:0/2544516541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560dcba90b20 0x560dcba8ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb15db34000 1 --2- 172.21.15.83:0/2544516541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dcba90200 0x560dcba8e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb15db34000 1 --2- 172.21.15.83:0/2544516541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dcba74f60 0x560dcba8f950 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb15db34000 1 -- 172.21.15.83:0/2544516541 >> 172.21.15.83:0/2544516541 conn(0x560dcba3d4f0 msgr2=0x560dcba736f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:10.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb15db34000 1 -- 172.21.15.83:0/2544516541 shutdown_connections 2024-04-17T10:42:10.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb15db34000 1 -- 172.21.15.83:0/2544516541 wait complete. 2024-04-17T10:42:10.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb15db34000 1 -- start start 2024-04-17T10:42:10.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb15db34000 1 Processor -- start 2024-04-17T10:42:10.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb15db34000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560dcba74f60 0x560dcbaae430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb15db34000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dcba90200 0x560dcbaae970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb15db34000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dcba90b20 0x560dcba45260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb15db34000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dcb821190 con 0x560dcba90b20 2024-04-17T10:42:10.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb15db34000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb154201790 con 0x560dcba90200 2024-04-17T10:42:10.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb15db34000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x560dcb899840 con 0x560dcba74f60 2024-04-17T10:42:10.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb15db34000 10 client.0 _ll_register_callbacks cb 0x7fb148007ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:10.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb158f64640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dcba90b20 0x560dcba45260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb158f64640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dcba90b20 0x560dcba45260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41244/0 (socket says 172.21.15.83:41244) 2024-04-17T10:42:10.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb158f64640 1 -- 172.21.15.83:0/2656100675 learned_addr learned my addr 172.21.15.83:0/2656100675 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:10.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb1537fe640 1 --2- 172.21.15.83:0/2656100675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dcba90200 0x560dcbaae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb153fff640 1 --2- 172.21.15.83:0/2656100675 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560dcba74f60 0x560dcbaae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb158f64640 1 -- 172.21.15.83:0/2656100675 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560dcba74f60 msgr2=0x560dcbaae430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:10.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb158f64640 1 --2- 172.21.15.83:0/2656100675 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560dcba74f60 0x560dcbaae430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb158f64640 1 -- 172.21.15.83:0/2656100675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dcba90200 msgr2=0x560dcbaae970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:10.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb158f64640 1 --2- 172.21.15.83:0/2656100675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dcba90200 0x560dcbaae970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb158f64640 1 -- 172.21.15.83:0/2656100675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560dcb85d040 con 0x560dcba90b20 2024-04-17T10:42:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb158f64640 1 --2- 172.21.15.83:0/2656100675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dcba90b20 0x560dcba45260 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb14c0f05b0 tx=0x7fb14c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1bda01caa011a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb132ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb132ffd640 1 -- 172.21.15.83:0/2656100675 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14c0f0210 con 0x560dcba90b20 2024-04-17T10:42:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb15db34000 1 -- 172.21.15.83:0/2656100675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb144007550 con 0x560dcba90b20 2024-04-17T10:42:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.110+0000 7fb132ffd640 1 -- 172.21.15.83:0/2656100675 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb14c0eff10 con 0x560dcba90b20 2024-04-17T10:42:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb132ffd640 1 -- 172.21.15.83:0/2656100675 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb14c10d050 con 0x560dcba90b20 2024-04-17T10:42:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb15db34000 10 client.6165 fetch_fsmap learned FSMap version 22 2024-04-17T10:42:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb15db34000 1 -- 172.21.15.83:0/2656100675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x560dcb551900 con 0x560dcba90b20 2024-04-17T10:42:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb132ffd640 1 -- 172.21.15.83:0/2656100675 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb144007550 con 0x560dcba90b20 2024-04-17T10:42:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb132ffd640 1 -- 172.21.15.83:0/2656100675 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb14c104310 con 0x560dcba90b20 2024-04-17T10:42:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb15db34000 10 client.6165 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T10:42:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb15db34000 10 client.6165 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:42:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb15db34000 20 client.6165 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:10.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb15db34000 1 --2- 172.21.15.83:0/2656100675 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x560dcba45ce0 0x560dcba460f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb15db34000 4 client.6165 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}] 2024-04-17T10:42:10.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb15db34000 1 -- 172.21.15.83:0/2656100675 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}) v1 -- 0x560dcb8a2750 con 0x560dcba45ce0 2024-04-17T10:42:10.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb153fff640 1 --2- 172.21.15.83:0/2656100675 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x560dcba45ce0 0x560dcba460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb153fff640 1 --2- 172.21.15.83:0/2656100675 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x560dcba45ce0 0x560dcba460f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30b7174ac5d8d1f3 server_cookie=69d7a2fc70be074d in_seq=0 out_seq=0 2024-04-17T10:42:10.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb132ffd640 10 client.6165 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:10.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb132ffd640 1 -- 172.21.15.83:0/2656100675 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560dcb8a2750 con 0x560dcba45ce0 2024-04-17T10:42:10.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb132ffd640 10 client.6165 handle_command_reply: tid=0 2024-04-17T10:42:10.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb15db34000 1 client.6165 shutdown 2024-04-17T10:42:10.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb15db34000 1 -- 172.21.15.83:0/2656100675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dcba90b20 msgr2=0x560dcba45260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:10.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb15db34000 1 --2- 172.21.15.83:0/2656100675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dcba90b20 0x560dcba45260 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb14c0f05b0 tx=0x7fb14c009750 comp rx=0 tx=0).stop 2024-04-17T10:42:10.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb15db34000 1 -- 172.21.15.83:0/2656100675 shutdown_connections 2024-04-17T10:42:10.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb15db34000 1 --2- 172.21.15.83:0/2656100675 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x560dcba45ce0 0x560dcba460f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb15db34000 1 --2- 172.21.15.83:0/2656100675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dcba90b20 0x560dcba45260 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb15db34000 1 --2- 172.21.15.83:0/2656100675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dcba90200 0x560dcbaae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb132ffd640 0 client.6165 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:10.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb15db34000 1 --2- 172.21.15.83:0/2656100675 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560dcba74f60 0x560dcbaae430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb15db34000 1 -- 172.21.15.83:0/2656100675 >> 172.21.15.83:0/2656100675 conn(0x560dcba3d4f0 msgr2=0x560dcba47c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:10.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb15db34000 1 -- 172.21.15.83:0/2656100675 shutdown_connections 2024-04-17T10:42:10.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb15db34000 1 -- 172.21.15.83:0/2656100675 wait complete. 2024-04-17T10:42:10.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb15db34000 20 client.6165 trim_cache size 0 max 16384 2024-04-17T10:42:10.118 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:42:10.119 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-17T10:42:10.119 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "25fb7779-79d1-45a3-847e-b7d449495e3f", 2024-04-17T10:42:10.119 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-17T10:42:10.119 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:42:10.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb15a9c7640 1 -- 172.21.15.83:0/1458939429 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb140066f60 msgr2=0x7fb140069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:10.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb15a9c7640 1 --2- 172.21.15.83:0/1458939429 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb140066f60 0x7fb140069420 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb1440011f0 tx=0x7fb144009040 comp rx=0 tx=0).stop 2024-04-17T10:42:10.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb15a9c7640 1 -- 172.21.15.83:0/1458939429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb15415e0f0 msgr2=0x7fb1541568d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:10.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb15a9c7640 1 --2- 172.21.15.83:0/1458939429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb15415e0f0 0x7fb1541568d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fb14c01a530 tx=0x7fb14c0196a0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb15a9c7640 1 -- 172.21.15.83:0/1458939429 shutdown_connections 2024-04-17T10:42:10.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb15a9c7640 1 --2- 172.21.15.83:0/1458939429 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb140066f60 0x7fb140069420 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb15a9c7640 1 --2- 172.21.15.83:0/1458939429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb15415e0f0 0x7fb1541568d0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb15a9c7640 1 --2- 172.21.15.83:0/1458939429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1540ce400 0x7fb15415e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb15a9c7640 1 --2- 172.21.15.83:0/1458939429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1540b5b60 0x7fb1540c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb15a9c7640 1 -- 172.21.15.83:0/1458939429 >> 172.21.15.83:0/1458939429 conn(0x7fb1540d0450 msgr2=0x7fb1540b8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:10.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb15a9c7640 1 -- 172.21.15.83:0/1458939429 shutdown_connections 2024-04-17T10:42:10.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.114+0000 7fb15a9c7640 1 -- 172.21.15.83:0/1458939429 wait complete. 2024-04-17T10:42:10.128 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' returned '{'return_code': 0, 'scrub_tag': '25fb7779-79d1-45a3-847e-b7d449495e3f', 'mode': 'asynchronous'}' 2024-04-17T10:42:10.128 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /' 2024-04-17T10:42:10.128 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-17T10:42:10.128 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / 2024-04-17T10:42:10.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a97b5f640 1 Processor -- start 2024-04-17T10:42:10.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a97b5f640 1 -- start start 2024-04-17T10:42:10.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a97b5f640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4a90071bb0 0x7f4a9015e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a97b5f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a9006d7d0 con 0x7f4a9006e110 2024-04-17T10:42:10.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a97b5f640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a900af600 con 0x7f4a9006d340 2024-04-17T10:42:10.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a97b5f640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a9015bb60 con 0x7f4a90071bb0 2024-04-17T10:42:10.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a958fb640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f4a9006e110 0x7f4a9006c540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46732/0 (socket says 172.21.15.83:46732) 2024-04-17T10:42:10.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a958fb640 1 -- 172.21.15.83:0/4011954093 learned_addr learned my addr 172.21.15.83:0/4011954093 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:10.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a948f9640 1 -- 172.21.15.83:0/4011954093 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 278119986 0 0) 0x7f4a9006d7d0 con 0x7f4a9006e110 2024-04-17T10:42:10.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a948f9640 1 -- 172.21.15.83:0/4011954093 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a7c003530 con 0x7f4a9006e110 2024-04-17T10:42:10.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a960fc640 1 --2- 172.21.15.83:0/4011954093 >> v2:172.21.15.167:3300/0 conn(0x7f4a90071bb0 0x7f4a9015e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a948f9640 1 -- 172.21.15.83:0/4011954093 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 722872434 0 0) 0x7f4a7c003530 con 0x7f4a9006e110 2024-04-17T10:42:10.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a948f9640 1 -- 172.21.15.83:0/4011954093 >> v2:172.21.15.167:3300/0 conn(0x7f4a90071bb0 msgr2=0x7f4a9015e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:10.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a948f9640 1 --2- 172.21.15.83:0/4011954093 >> v2:172.21.15.167:3300/0 conn(0x7f4a90071bb0 0x7f4a9015e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a948f9640 1 -- 172.21.15.83:0/4011954093 >> v1:172.21.15.167:6789/0 conn(0x7f4a9006d340 legacy=0x7f4a900714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:10.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a948f9640 1 -- 172.21.15.83:0/4011954093 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a900b0140 con 0x7f4a9006e110 2024-04-17T10:42:10.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a948f9640 1 -- 172.21.15.83:0/4011954093 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f4a8c001680 con 0x7f4a9006e110 2024-04-17T10:42:10.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a960fc640 1 --2- 172.21.15.83:0/4011954093 >> v2:172.21.15.167:3300/0 conn(0x7f4a90071bb0 0x7f4a9015e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:42:10.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a948f9640 1 -- 172.21.15.83:0/4011954093 >> v1:172.21.15.83:6789/0 conn(0x7f4a9006e110 legacy=0x7f4a9006c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:10.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a948f9640 1 --2- 172.21.15.83:0/4011954093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a7c003a30 0x7f4a7c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a948f9640 1 --2- 172.21.15.83:0/4011954093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a7c004810 0x7f4a7c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a948f9640 1 --2- 172.21.15.83:0/4011954093 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a7c005240 0x7f4a7c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a948f9640 1 -- 172.21.15.83:0/4011954093 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7c00c170 con 0x7f4a7c003a30 2024-04-17T10:42:10.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a948f9640 1 -- 172.21.15.83:0/4011954093 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7c00c320 con 0x7f4a7c004810 2024-04-17T10:42:10.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a948f9640 1 -- 172.21.15.83:0/4011954093 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7c00c520 con 0x7f4a7c005240 2024-04-17T10:42:10.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a958fb640 1 --2- 172.21.15.83:0/4011954093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a7c003a30 0x7f4a7c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a950fa640 1 --2- 172.21.15.83:0/4011954093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a7c004810 0x7f4a7c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a960fc640 1 --2- 172.21.15.83:0/4011954093 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a7c005240 0x7f4a7c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a958fb640 1 -- 172.21.15.83:0/4011954093 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a7c005240 msgr2=0x7f4a7c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:10.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a958fb640 1 --2- 172.21.15.83:0/4011954093 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a7c005240 0x7f4a7c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a958fb640 1 -- 172.21.15.83:0/4011954093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a7c004810 msgr2=0x7f4a7c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:10.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a958fb640 1 --2- 172.21.15.83:0/4011954093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a7c004810 0x7f4a7c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a960fc640 1 -- 172.21.15.83:0/4011954093 reap_dead start 2024-04-17T10:42:10.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.238+0000 7f4a958fb640 1 -- 172.21.15.83:0/4011954093 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a7c003530 con 0x7f4a7c003a30 2024-04-17T10:42:10.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.242+0000 7f4a958fb640 1 --2- 172.21.15.83:0/4011954093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a7c003a30 0x7f4a7c003e50 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c00c210 tx=0x7f4a8c01cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b1fc8fe1b0bb923 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:10.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.242+0000 7f4a948f9640 1 -- 172.21.15.83:0/4011954093 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c007d40 con 0x7f4a7c003a30 2024-04-17T10:42:10.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.242+0000 7f4a948f9640 1 -- 172.21.15.83:0/4011954093 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a8c0a9070 con 0x7f4a7c003a30 2024-04-17T10:42:10.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.242+0000 7f4a948f9640 1 -- 172.21.15.83:0/4011954093 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c0a5e00 con 0x7f4a7c003a30 2024-04-17T10:42:10.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a97b5f640 1 -- 172.21.15.83:0/4011954093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a7c003a30 msgr2=0x7f4a7c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:10.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a97b5f640 1 --2- 172.21.15.83:0/4011954093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a7c003a30 0x7f4a7c003e50 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c00c210 tx=0x7f4a8c01cc30 comp rx=0 tx=0).stop 2024-04-17T10:42:10.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a97b5f640 1 -- 172.21.15.83:0/4011954093 shutdown_connections 2024-04-17T10:42:10.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a97b5f640 1 --2- 172.21.15.83:0/4011954093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a7c003a30 0x7f4a7c003e50 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a97b5f640 1 -- 172.21.15.83:0/4011954093 >> 172.21.15.83:0/4011954093 conn(0x7f4a900b23f0 msgr2=0x7f4a900b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:10.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a97b5f640 1 -- 172.21.15.83:0/4011954093 shutdown_connections 2024-04-17T10:42:10.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a97b5f640 1 -- 172.21.15.83:0/4011954093 wait complete. 2024-04-17T10:42:10.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a97b5f640 1 Processor -- start 2024-04-17T10:42:10.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a97b5f640 1 -- start start 2024-04-17T10:42:10.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a97b5f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a7c003a30 0x7f4a9014fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a97b5f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a900b0da0 0x7f4a9015bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a97b5f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a9015c0a0 0x7f4a901f9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a97b5f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a900718b0 con 0x7f4a900b0da0 2024-04-17T10:42:10.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a97b5f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a900afe60 con 0x7f4a9015c0a0 2024-04-17T10:42:10.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a97b5f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a90071a30 con 0x7f4a7c003a30 2024-04-17T10:42:10.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a950fa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a900b0da0 0x7f4a9015bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a950fa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a900b0da0 0x7f4a9015bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41258/0 (socket says 172.21.15.83:41258) 2024-04-17T10:42:10.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a950fa640 1 -- 172.21.15.83:0/1731153863 learned_addr learned my addr 172.21.15.83:0/1731153863 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:10.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a960fc640 1 --2- 172.21.15.83:0/1731153863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a9015c0a0 0x7f4a901f9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a958fb640 1 --2- 172.21.15.83:0/1731153863 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a7c003a30 0x7f4a9014fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a950fa640 1 -- 172.21.15.83:0/1731153863 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a7c003a30 msgr2=0x7f4a9014fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:10.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a950fa640 1 --2- 172.21.15.83:0/1731153863 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a7c003a30 0x7f4a9014fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a950fa640 1 -- 172.21.15.83:0/1731153863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a9015c0a0 msgr2=0x7f4a901f9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:10.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a950fa640 1 --2- 172.21.15.83:0/1731153863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a9015c0a0 0x7f4a901f9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a950fa640 1 -- 172.21.15.83:0/1731153863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a901505f0 con 0x7f4a900b0da0 2024-04-17T10:42:10.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a958fb640 1 --2- 172.21.15.83:0/1731153863 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a7c003a30 0x7f4a9014fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:10.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a960fc640 1 --2- 172.21.15.83:0/1731153863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a9015c0a0 0x7f4a901f9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:42:10.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a950fa640 1 --2- 172.21.15.83:0/1731153863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a900b0da0 0x7f4a9015bb60 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4a8000f570 tx=0x7f4a8000fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:10.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a867fc640 1 -- 172.21.15.83:0/1731153863 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a800052e0 con 0x7f4a900b0da0 2024-04-17T10:42:10.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a867fc640 1 -- 172.21.15.83:0/1731153863 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a80018070 con 0x7f4a900b0da0 2024-04-17T10:42:10.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a97b5f640 1 -- 172.21.15.83:0/1731153863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a900c7460 con 0x7f4a900b0da0 2024-04-17T10:42:10.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a867fc640 1 -- 172.21.15.83:0/1731153863 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a80003840 con 0x7f4a900b0da0 2024-04-17T10:42:10.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.246+0000 7f4a97b5f640 1 -- 172.21.15.83:0/1731153863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a900c7a20 con 0x7f4a900b0da0 2024-04-17T10:42:10.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.250+0000 7f4a867fc640 1 -- 172.21.15.83:0/1731153863 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4a800039e0 con 0x7f4a900b0da0 2024-04-17T10:42:10.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.250+0000 7f4a867fc640 1 --2- 172.21.15.83:0/1731153863 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4a74066f60 0x7f4a74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.250+0000 7f4a867fc640 1 -- 172.21.15.83:0/1731153863 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4a80084d90 con 0x7f4a900b0da0 2024-04-17T10:42:10.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.250+0000 7f4a958fb640 1 --2- 172.21.15.83:0/1731153863 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4a74066f60 0x7f4a74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.250+0000 7f4a958fb640 1 --2- 172.21.15.83:0/1731153863 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4a74066f60 0x7f4a74069420 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0ab000 tx=0x7f4a8c0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:10.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.250+0000 7f4a9accc000 1 Processor -- start 2024-04-17T10:42:10.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.250+0000 7f4a9accc000 1 -- start start 2024-04-17T10:42:10.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.250+0000 7f4a9accc000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5582c981d000 0x5582c981d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.250+0000 7f4a9accc000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582c981d920 0x5582c97d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.250+0000 7f4a9accc000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582c97d3410 0x5582c97d37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.250+0000 7f4a9accc000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582c9620f40 con 0x5582c97d3410 2024-04-17T10:42:10.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.250+0000 7f4a9accc000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582c95a8fc0 con 0x5582c981d920 2024-04-17T10:42:10.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.250+0000 7f4a9accc000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5582c961cfe0 con 0x5582c981d000 2024-04-17T10:42:10.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.250+0000 7f4a960fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582c97d3410 0x5582c97d37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.250+0000 7f4a960fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582c97d3410 0x5582c97d37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41268/0 (socket says 172.21.15.83:41268) 2024-04-17T10:42:10.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.250+0000 7f4a960fc640 1 -- 172.21.15.83:0/3728246579 learned_addr learned my addr 172.21.15.83:0/3728246579 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:10.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.250+0000 7f4a950fa640 1 --2- 172.21.15.83:0/3728246579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582c981d920 0x5582c97d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.250+0000 7f4a958fb640 1 --2- 172.21.15.83:0/3728246579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5582c981d000 0x5582c981d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.250+0000 7f4a960fc640 1 -- 172.21.15.83:0/3728246579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5582c981d000 msgr2=0x5582c981d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:10.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.250+0000 7f4a960fc640 1 --2- 172.21.15.83:0/3728246579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5582c981d000 0x5582c981d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.250+0000 7f4a960fc640 1 -- 172.21.15.83:0/3728246579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582c981d920 msgr2=0x5582c97d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:10.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.250+0000 7f4a960fc640 1 --2- 172.21.15.83:0/3728246579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582c981d920 0x5582c97d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.250+0000 7f4a960fc640 1 -- 172.21.15.83:0/3728246579 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582c9668a50 con 0x5582c97d3410 2024-04-17T10:42:10.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.250+0000 7f4a960fc640 1 --2- 172.21.15.83:0/3728246579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582c97d3410 0x5582c97d37f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f4a900b31f0 tx=0x7f4a88009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e6eb2c7d197436 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:10.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.250+0000 7f4a737fe640 1 -- 172.21.15.83:0/3728246579 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8801b020 con 0x5582c97d3410 2024-04-17T10:42:10.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.250+0000 7f4a737fe640 1 -- 172.21.15.83:0/3728246579 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a88007c70 con 0x5582c97d3410 2024-04-17T10:42:10.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.250+0000 7f4a737fe640 1 -- 172.21.15.83:0/3728246579 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a88007e60 con 0x5582c97d3410 2024-04-17T10:42:10.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.250+0000 7f4a9accc000 1 -- 172.21.15.83:0/3728246579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582c97d3410 msgr2=0x5582c97d37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:10.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.250+0000 7f4a9accc000 1 --2- 172.21.15.83:0/3728246579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582c97d3410 0x5582c97d37f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f4a900b31f0 tx=0x7f4a88009820 comp rx=0 tx=0).stop 2024-04-17T10:42:10.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 1 -- 172.21.15.83:0/3728246579 shutdown_connections 2024-04-17T10:42:10.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 1 --2- 172.21.15.83:0/3728246579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582c97d3410 0x5582c97d37f0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 1 --2- 172.21.15.83:0/3728246579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582c981d920 0x5582c97d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 1 --2- 172.21.15.83:0/3728246579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5582c981d000 0x5582c981d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 1 -- 172.21.15.83:0/3728246579 >> 172.21.15.83:0/3728246579 conn(0x5582c97cf3c0 msgr2=0x5582c97d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:10.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 1 -- 172.21.15.83:0/3728246579 shutdown_connections 2024-04-17T10:42:10.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 1 -- 172.21.15.83:0/3728246579 wait complete. 2024-04-17T10:42:10.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 1 -- start start 2024-04-17T10:42:10.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 1 Processor -- start 2024-04-17T10:42:10.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582c97d3410 0x5582c97dee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582c981d000 0x5582c97df340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5582c981d920 0x5582c97df880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582c9625570 con 0x5582c97d3410 2024-04-17T10:42:10.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582c95db5f0 con 0x5582c981d000 2024-04-17T10:42:10.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5582c961dfb0 con 0x5582c981d920 2024-04-17T10:42:10.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 10 client.0 _ll_register_callbacks cb 0x5582c96d57a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:10.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a958fb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582c97d3410 0x5582c97dee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a958fb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582c97d3410 0x5582c97dee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41278/0 (socket says 172.21.15.83:41278) 2024-04-17T10:42:10.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a958fb640 1 -- 172.21.15.83:0/2654648118 learned_addr learned my addr 172.21.15.83:0/2654648118 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:10.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a950fa640 1 --2- 172.21.15.83:0/2654648118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582c981d000 0x5582c97df340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a958fb640 1 -- 172.21.15.83:0/2654648118 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5582c981d920 msgr2=0x5582c97df880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:10.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a958fb640 1 --2- 172.21.15.83:0/2654648118 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5582c981d920 0x5582c97df880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a958fb640 1 -- 172.21.15.83:0/2654648118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582c981d000 msgr2=0x5582c97df340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:10.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a958fb640 1 --2- 172.21.15.83:0/2654648118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582c981d000 0x5582c97df340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a958fb640 1 -- 172.21.15.83:0/2654648118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582c95a9210 con 0x5582c97d3410 2024-04-17T10:42:10.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a958fb640 1 --2- 172.21.15.83:0/2654648118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582c97d3410 0x5582c97dee00 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c01c6a0 tx=0x7f4a8c0199d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f64a99895e7927 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:10.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:10.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a70ff9640 1 -- 172.21.15.83:0/2654648118 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c004970 con 0x5582c97d3410 2024-04-17T10:42:10.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a70ff9640 1 -- 172.21.15.83:0/2654648118 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4a8c0b4070 con 0x5582c97d3410 2024-04-17T10:42:10.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 1 -- 172.21.15.83:0/2654648118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a88007c70 con 0x5582c97d3410 2024-04-17T10:42:10.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a70ff9640 1 -- 172.21.15.83:0/2654648118 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a8c004b10 con 0x5582c97d3410 2024-04-17T10:42:10.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 10 client.6177 fetch_fsmap learned FSMap version 22 2024-04-17T10:42:10.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 1 -- 172.21.15.83:0/2654648118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x5582c96a5400 con 0x5582c97d3410 2024-04-17T10:42:10.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a70ff9640 1 -- 172.21.15.83:0/2654648118 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a88007c70 con 0x5582c97d3410 2024-04-17T10:42:10.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a70ff9640 1 -- 172.21.15.83:0/2654648118 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4a8c004cb0 con 0x5582c97d3410 2024-04-17T10:42:10.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 10 client.6177 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T10:42:10.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 10 client.6177 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:42:10.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 20 client.6177 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:10.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 1 --2- 172.21.15.83:0/2654648118 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x5582c97e0290 0x5582c982f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 4 client.6177 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:42:10.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 1 -- 172.21.15.83:0/2654648118 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5582c95bcc90 con 0x5582c97e0290 2024-04-17T10:42:10.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a950fa640 1 --2- 172.21.15.83:0/2654648118 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x5582c97e0290 0x5582c982f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a70ff9640 10 client.6177 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:10.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a950fa640 1 --2- 172.21.15.83:0/2654648118 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x5582c97e0290 0x5582c982f2c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2df6d39097c8225 server_cookie=cc37b9598bf7c256 in_seq=0 out_seq=0 2024-04-17T10:42:10.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a70ff9640 1 -- 172.21.15.83:0/2654648118 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5582c95bcc90 con 0x5582c97e0290 2024-04-17T10:42:10.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a70ff9640 10 client.6177 handle_command_reply: tid=0 2024-04-17T10:42:10.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 1 client.6177 shutdown 2024-04-17T10:42:10.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 1 -- 172.21.15.83:0/2654648118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582c97d3410 msgr2=0x5582c97dee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:10.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 1 --2- 172.21.15.83:0/2654648118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582c97d3410 0x5582c97dee00 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c01c6a0 tx=0x7f4a8c0199d0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 1 -- 172.21.15.83:0/2654648118 shutdown_connections 2024-04-17T10:42:10.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 1 --2- 172.21.15.83:0/2654648118 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x5582c97e0290 0x5582c982f2c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 1 --2- 172.21.15.83:0/2654648118 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5582c981d920 0x5582c97df880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a70ff9640 0 client.6177 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:10.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 1 --2- 172.21.15.83:0/2654648118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582c981d000 0x5582c97df340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 1 --2- 172.21.15.83:0/2654648118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582c97d3410 0x5582c97dee00 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 1 -- 172.21.15.83:0/2654648118 >> 172.21.15.83:0/2654648118 conn(0x5582c97cf3c0 msgr2=0x5582c97d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:10.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 1 -- 172.21.15.83:0/2654648118 shutdown_connections 2024-04-17T10:42:10.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.254+0000 7f4a9accc000 1 -- 172.21.15.83:0/2654648118 wait complete. 2024-04-17T10:42:10.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.258+0000 7f4a9accc000 20 client.6177 trim_cache size 0 max 16384 2024-04-17T10:42:10.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.270+0000 7f4a9accc000 1 Processor -- start 2024-04-17T10:42:10.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.270+0000 7f4a9accc000 1 -- start start 2024-04-17T10:42:10.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.270+0000 7f4a9accc000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582c97d3410 0x5582c97d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.270+0000 7f4a9accc000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5582c981d000 0x5582c97d2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.270+0000 7f4a9accc000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582c981d920 0x5582c982f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.270+0000 7f4a9accc000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a90203460 con 0x5582c97d3410 2024-04-17T10:42:10.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.270+0000 7f4a9accc000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a90202620 con 0x5582c981d920 2024-04-17T10:42:10.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.270+0000 7f4a9accc000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a900c96a0 con 0x5582c981d000 2024-04-17T10:42:10.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.270+0000 7f4a958fb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582c97d3410 0x5582c97d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.270+0000 7f4a958fb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582c97d3410 0x5582c97d27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41286/0 (socket says 172.21.15.83:41286) 2024-04-17T10:42:10.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.270+0000 7f4a958fb640 1 -- 172.21.15.83:0/4208585549 learned_addr learned my addr 172.21.15.83:0/4208585549 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:10.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.270+0000 7f4a958fb640 1 -- 172.21.15.83:0/4208585549 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5582c981d000 msgr2=0x5582c97d2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:10.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.270+0000 7f4a958fb640 1 --2- 172.21.15.83:0/4208585549 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5582c981d000 0x5582c97d2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.270+0000 7f4a958fb640 1 -- 172.21.15.83:0/4208585549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582c981d920 msgr2=0x5582c982f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:10.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.270+0000 7f4a958fb640 1 --2- 172.21.15.83:0/4208585549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582c981d920 0x5582c982f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.270+0000 7f4a958fb640 1 -- 172.21.15.83:0/4208585549 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582c967bc10 con 0x5582c97d3410 2024-04-17T10:42:10.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a958fb640 1 --2- 172.21.15.83:0/4208585549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582c97d3410 0x5582c97d27c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c001ec0 tx=0x7f4a8c0bf000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12b161722c4da28 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:10.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a72ffd640 1 -- 172.21.15.83:0/4208585549 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c0bd040 con 0x5582c97d3410 2024-04-17T10:42:10.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a72ffd640 1 -- 172.21.15.83:0/4208585549 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a8c0a9040 con 0x5582c97d3410 2024-04-17T10:42:10.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a72ffd640 1 -- 172.21.15.83:0/4208585549 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c0072d0 con 0x5582c97d3410 2024-04-17T10:42:10.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a9accc000 1 -- 172.21.15.83:0/4208585549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582c97d3410 msgr2=0x5582c97d27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:10.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a9accc000 1 --2- 172.21.15.83:0/4208585549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582c97d3410 0x5582c97d27c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c001ec0 tx=0x7f4a8c0bf000 comp rx=0 tx=0).stop 2024-04-17T10:42:10.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a9accc000 1 -- 172.21.15.83:0/4208585549 shutdown_connections 2024-04-17T10:42:10.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a9accc000 1 --2- 172.21.15.83:0/4208585549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582c981d920 0x5582c982f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a9accc000 1 --2- 172.21.15.83:0/4208585549 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5582c981d000 0x5582c97d2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a9accc000 1 --2- 172.21.15.83:0/4208585549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582c97d3410 0x5582c97d27c0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a9accc000 1 -- 172.21.15.83:0/4208585549 >> 172.21.15.83:0/4208585549 conn(0x5582c97cf3c0 msgr2=0x5582c97df560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:10.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a9accc000 1 -- 172.21.15.83:0/4208585549 shutdown_connections 2024-04-17T10:42:10.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a9accc000 1 -- 172.21.15.83:0/4208585549 wait complete. 2024-04-17T10:42:10.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a9accc000 1 -- start start 2024-04-17T10:42:10.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a9accc000 1 Processor -- start 2024-04-17T10:42:10.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a9accc000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582c97d3410 0x5582c98312f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a9accc000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582c981d000 0x5582c98425a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a9accc000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5582c981d920 0x5582c9842ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a9accc000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582c97cf240 con 0x5582c981d000 2024-04-17T10:42:10.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a9accc000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a90202080 con 0x5582c97d3410 2024-04-17T10:42:10.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a9accc000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5582c95e2f00 con 0x5582c981d920 2024-04-17T10:42:10.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a9accc000 10 client.0 _ll_register_callbacks cb 0x5582c96732c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:10.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a950fa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582c981d000 0x5582c98425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a950fa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582c981d000 0x5582c98425a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41288/0 (socket says 172.21.15.83:41288) 2024-04-17T10:42:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a950fa640 1 -- 172.21.15.83:0/1535632304 learned_addr learned my addr 172.21.15.83:0/1535632304 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a950fa640 1 -- 172.21.15.83:0/1535632304 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5582c981d920 msgr2=0x5582c9842ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a950fa640 1 --2- 172.21.15.83:0/1535632304 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5582c981d920 0x5582c9842ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a950fa640 1 -- 172.21.15.83:0/1535632304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582c97d3410 msgr2=0x5582c98312f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a950fa640 1 --2- 172.21.15.83:0/1535632304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582c97d3410 0x5582c98312f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a950fa640 1 -- 172.21.15.83:0/1535632304 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582c95b5bb0 con 0x5582c981d000 2024-04-17T10:42:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a950fa640 1 --2- 172.21.15.83:0/1535632304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582c981d000 0x5582c98425a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f4a800555f0 tx=0x7f4a800550e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c23e6250125ab2dc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a70ff9640 1 -- 172.21.15.83:0/1535632304 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8005dd70 con 0x5582c981d000 2024-04-17T10:42:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a70ff9640 1 -- 172.21.15.83:0/1535632304 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4a80053960 con 0x5582c981d000 2024-04-17T10:42:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a9accc000 1 -- 172.21.15.83:0/1535632304 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a8c0a9040 con 0x5582c981d000 2024-04-17T10:42:10.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a70ff9640 1 -- 172.21.15.83:0/1535632304 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a80003cd0 con 0x5582c981d000 2024-04-17T10:42:10.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a9accc000 10 client.6183 fetch_fsmap learned FSMap version 22 2024-04-17T10:42:10.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a9accc000 1 -- 172.21.15.83:0/1535632304 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x5582c95c8030 con 0x5582c981d000 2024-04-17T10:42:10.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a70ff9640 1 -- 172.21.15.83:0/1535632304 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a8c0a9040 con 0x5582c981d000 2024-04-17T10:42:10.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a70ff9640 1 -- 172.21.15.83:0/1535632304 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4a80051cd0 con 0x5582c981d000 2024-04-17T10:42:10.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a9accc000 10 client.6183 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T10:42:10.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a9accc000 10 client.6183 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:42:10.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a9accc000 20 client.6183 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:10.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a9accc000 1 --2- 172.21.15.83:0/1535632304 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x5582c9886630 0x5582c9886a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a9accc000 4 client.6183 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/"}] 2024-04-17T10:42:10.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a9accc000 1 -- 172.21.15.83:0/1535632304 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "scrub start", "path": "/"}) v1 -- 0x5582c962e480 con 0x5582c9886630 2024-04-17T10:42:10.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a958fb640 1 --2- 172.21.15.83:0/1535632304 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x5582c9886630 0x5582c9886a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a70ff9640 10 client.6183 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:10.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.274+0000 7f4a958fb640 1 --2- 172.21.15.83:0/1535632304 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x5582c9886630 0x5582c9886a40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d271e4339cb5b9bd server_cookie=58a3f590060f0d25 in_seq=0 out_seq=0 2024-04-17T10:42:10.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.278+0000 7f4a70ff9640 1 -- 172.21.15.83:0/1535632304 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5582c962e480 con 0x5582c9886630 2024-04-17T10:42:10.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.278+0000 7f4a70ff9640 10 client.6183 handle_command_reply: tid=0 2024-04-17T10:42:10.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.278+0000 7f4a9accc000 1 client.6183 shutdown 2024-04-17T10:42:10.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.278+0000 7f4a9accc000 1 -- 172.21.15.83:0/1535632304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582c981d000 msgr2=0x5582c98425a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:10.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.278+0000 7f4a9accc000 1 --2- 172.21.15.83:0/1535632304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582c981d000 0x5582c98425a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f4a800555f0 tx=0x7f4a800550e0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.278+0000 7f4a9accc000 1 -- 172.21.15.83:0/1535632304 shutdown_connections 2024-04-17T10:42:10.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.278+0000 7f4a9accc000 1 --2- 172.21.15.83:0/1535632304 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x5582c9886630 0x5582c9886a40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.278+0000 7f4a9accc000 1 --2- 172.21.15.83:0/1535632304 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5582c981d920 0x5582c9842ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.278+0000 7f4a9accc000 1 --2- 172.21.15.83:0/1535632304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582c981d000 0x5582c98425a0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.278+0000 7f4a70ff9640 0 client.6183 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:10.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.278+0000 7f4a9accc000 1 --2- 172.21.15.83:0/1535632304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582c97d3410 0x5582c98312f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.278+0000 7f4a9accc000 1 -- 172.21.15.83:0/1535632304 >> 172.21.15.83:0/1535632304 conn(0x5582c97cf3c0 msgr2=0x5582c97df560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:10.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.278+0000 7f4a9accc000 1 -- 172.21.15.83:0/1535632304 shutdown_connections 2024-04-17T10:42:10.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.278+0000 7f4a9accc000 1 -- 172.21.15.83:0/1535632304 wait complete. 2024-04-17T10:42:10.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.278+0000 7f4a9accc000 20 client.6183 trim_cache size 0 max 16384 2024-04-17T10:42:10.282 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:42:10.282 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-17T10:42:10.282 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "31af5ab9-e3c8-4446-b651-082662ab3d6b", 2024-04-17T10:42:10.283 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-17T10:42:10.283 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:42:10.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.278+0000 7f4a97b5f640 1 -- 172.21.15.83:0/1731153863 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4a74066f60 msgr2=0x7f4a74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:10.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.278+0000 7f4a97b5f640 1 --2- 172.21.15.83:0/1731153863 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4a74066f60 0x7f4a74069420 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0ab000 tx=0x7f4a8c0a7070 comp rx=0 tx=0).stop 2024-04-17T10:42:10.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.278+0000 7f4a97b5f640 1 -- 172.21.15.83:0/1731153863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a900b0da0 msgr2=0x7f4a9015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:10.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.278+0000 7f4a97b5f640 1 --2- 172.21.15.83:0/1731153863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a900b0da0 0x7f4a9015bb60 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4a8000f570 tx=0x7f4a8000fa30 comp rx=0 tx=0).stop 2024-04-17T10:42:10.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.278+0000 7f4a97b5f640 1 -- 172.21.15.83:0/1731153863 shutdown_connections 2024-04-17T10:42:10.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.278+0000 7f4a97b5f640 1 --2- 172.21.15.83:0/1731153863 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4a74066f60 0x7f4a74069420 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.278+0000 7f4a97b5f640 1 --2- 172.21.15.83:0/1731153863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a9015c0a0 0x7f4a901f9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.278+0000 7f4a97b5f640 1 --2- 172.21.15.83:0/1731153863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a900b0da0 0x7f4a9015bb60 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.278+0000 7f4a97b5f640 1 --2- 172.21.15.83:0/1731153863 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a7c003a30 0x7f4a9014fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.278+0000 7f4a97b5f640 1 -- 172.21.15.83:0/1731153863 >> 172.21.15.83:0/1731153863 conn(0x7f4a900b23f0 msgr2=0x7f4a900b0640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:10.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.278+0000 7f4a97b5f640 1 -- 172.21.15.83:0/1731153863 shutdown_connections 2024-04-17T10:42:10.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.278+0000 7f4a97b5f640 1 -- 172.21.15.83:0/1731153863 wait complete. 2024-04-17T10:42:10.291 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /' returned '{'return_code': 0, 'scrub_tag': '31af5ab9-e3c8-4446-b651-082662ab3d6b', 'mode': 'asynchronous'}' 2024-04-17T10:42:10.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:42:10.291 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_dir_1) 2024-04-17T10:42:10.298 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_1' 2024-04-17T10:42:10.298 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:42:10.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.442+0000 7ff5bc873640 1 Processor -- start 2024-04-17T10:42:10.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.446+0000 7ff5bc873640 1 -- start start 2024-04-17T10:42:10.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.446+0000 7ff5bc873640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5b80aedd0 0x7ff5b806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.446+0000 7ff5bc873640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff5b806dae0 0x7ff5b80b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.446+0000 7ff5bc873640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5b815e440 con 0x7ff5b815db50 2024-04-17T10:42:10.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.446+0000 7ff5bc873640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5b815e720 con 0x7ff5b806dae0 2024-04-17T10:42:10.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.446+0000 7ff5bc873640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b815ea00 con 0x7ff5b80aedd0 2024-04-17T10:42:10.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.446+0000 7ff5b6d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff5b806dae0 0x7ff5b80b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.446+0000 7ff5b6d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff5b806dae0 0x7ff5b80b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41298/0 (socket says 172.21.15.83:41298) 2024-04-17T10:42:10.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.446+0000 7ff5b6d9d640 1 -- 172.21.15.83:0/3611765215 learned_addr learned my addr 172.21.15.83:0/3611765215 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:10.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.446+0000 7ff5affff640 1 --2- 172.21.15.83:0/3611765215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5b80aedd0 0x7ff5b806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.446+0000 7ff5b6d9d640 1 -- 172.21.15.83:0/3611765215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5b80aedd0 msgr2=0x7ff5b806d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:10.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.446+0000 7ff5b6d9d640 1 --2- 172.21.15.83:0/3611765215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5b80aedd0 0x7ff5b806d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.446+0000 7ff5b6d9d640 1 -- 172.21.15.83:0/3611765215 >> v1:172.21.15.167:6789/0 conn(0x7ff5b815db50 legacy=0x7ff5b8070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:10.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.446+0000 7ff5b6d9d640 1 -- 172.21.15.83:0/3611765215 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b815ece0 con 0x7ff5b806dae0 2024-04-17T10:42:10.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.446+0000 7ff5b6d9d640 1 --2- 172.21.15.83:0/3611765215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5b806dae0 0x7ff5b80b2e10 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009e50 tx=0x7ff5b001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c80f555fad3b4d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:10.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.446+0000 7ff5b5d9b640 1 -- 172.21.15.83:0/3611765215 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b00a5700 con 0x7ff5b806dae0 2024-04-17T10:42:10.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.446+0000 7ff5b5d9b640 1 -- 172.21.15.83:0/3611765215 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b00a58a0 con 0x7ff5b806dae0 2024-04-17T10:42:10.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.446+0000 7ff5b5d9b640 1 -- 172.21.15.83:0/3611765215 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b00a5c70 con 0x7ff5b806dae0 2024-04-17T10:42:10.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.450+0000 7ff5bc873640 1 -- 172.21.15.83:0/3611765215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5b806dae0 msgr2=0x7ff5b80b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:10.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.450+0000 7ff5bc873640 1 --2- 172.21.15.83:0/3611765215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5b806dae0 0x7ff5b80b2e10 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009e50 tx=0x7ff5b001ca30 comp rx=0 tx=0).stop 2024-04-17T10:42:10.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.450+0000 7ff5bc873640 1 -- 172.21.15.83:0/3611765215 shutdown_connections 2024-04-17T10:42:10.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.450+0000 7ff5bc873640 1 --2- 172.21.15.83:0/3611765215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5b806dae0 0x7ff5b80b2e10 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.450+0000 7ff5bc873640 1 --2- 172.21.15.83:0/3611765215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5b80aedd0 0x7ff5b806d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.450+0000 7ff5bc873640 1 -- 172.21.15.83:0/3611765215 >> 172.21.15.83:0/3611765215 conn(0x7ff5b806abe0 msgr2=0x7ff5b806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:10.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.450+0000 7ff5bc873640 1 -- 172.21.15.83:0/3611765215 shutdown_connections 2024-04-17T10:42:10.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.450+0000 7ff5bc873640 1 -- 172.21.15.83:0/3611765215 wait complete. 2024-04-17T10:42:10.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.450+0000 7ff5bc873640 1 Processor -- start 2024-04-17T10:42:10.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.450+0000 7ff5bc873640 1 -- start start 2024-04-17T10:42:10.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.450+0000 7ff5bc873640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5b806dae0 0x7ff5b8156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.450+0000 7ff5bc873640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5b80aedd0 0x7ff5b814de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.450+0000 7ff5bc873640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5b815db50 0x7ff5b814e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.450+0000 7ff5bc873640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b80b3ef0 con 0x7ff5b815db50 2024-04-17T10:42:10.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.450+0000 7ff5bc873640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b80b3d70 con 0x7ff5b80aedd0 2024-04-17T10:42:10.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.450+0000 7ff5bc873640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b80b4070 con 0x7ff5b806dae0 2024-04-17T10:42:10.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.450+0000 7ff5b6d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5b815db50 0x7ff5b814e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.450+0000 7ff5b6d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5b815db50 0x7ff5b814e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41302/0 (socket says 172.21.15.83:41302) 2024-04-17T10:42:10.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.450+0000 7ff5b6d9d640 1 -- 172.21.15.83:0/1896664656 learned_addr learned my addr 172.21.15.83:0/1896664656 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:10.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.450+0000 7ff5affff640 1 --2- 172.21.15.83:0/1896664656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5b80aedd0 0x7ff5b814de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.450+0000 7ff5b659c640 1 --2- 172.21.15.83:0/1896664656 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5b806dae0 0x7ff5b8156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.454+0000 7ff5b6d9d640 1 -- 172.21.15.83:0/1896664656 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5b806dae0 msgr2=0x7ff5b8156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:10.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.454+0000 7ff5b6d9d640 1 --2- 172.21.15.83:0/1896664656 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5b806dae0 0x7ff5b8156320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.454+0000 7ff5b6d9d640 1 -- 172.21.15.83:0/1896664656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5b80aedd0 msgr2=0x7ff5b814de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:10.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.454+0000 7ff5b6d9d640 1 --2- 172.21.15.83:0/1896664656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5b80aedd0 0x7ff5b814de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.454+0000 7ff5b6d9d640 1 -- 172.21.15.83:0/1896664656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b8154ad0 con 0x7ff5b815db50 2024-04-17T10:42:10.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.454+0000 7ff5b6d9d640 1 --2- 172.21.15.83:0/1896664656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5b815db50 0x7ff5b814e3c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff5b00af040 tx=0x7ff5b00197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:10.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.454+0000 7ff5aeffd640 1 -- 172.21.15.83:0/1896664656 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b00b8670 con 0x7ff5b815db50 2024-04-17T10:42:10.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.454+0000 7ff5aeffd640 1 -- 172.21.15.83:0/1896664656 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b00be070 con 0x7ff5b815db50 2024-04-17T10:42:10.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.454+0000 7ff5aeffd640 1 -- 172.21.15.83:0/1896664656 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b00b8a30 con 0x7ff5b815db50 2024-04-17T10:42:10.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.454+0000 7ff5bc873640 1 -- 172.21.15.83:0/1896664656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5b8154db0 con 0x7ff5b815db50 2024-04-17T10:42:10.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.454+0000 7ff5bc873640 1 -- 172.21.15.83:0/1896664656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b8155370 con 0x7ff5b815db50 2024-04-17T10:42:10.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.454+0000 7ff5bc873640 1 -- 172.21.15.83:0/1896664656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5b8074f90 con 0x7ff5b815db50 2024-04-17T10:42:10.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.454+0000 7ff5aeffd640 1 -- 172.21.15.83:0/1896664656 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff5b00b8bd0 con 0x7ff5b815db50 2024-04-17T10:42:10.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.454+0000 7ff5aeffd640 1 --2- 172.21.15.83:0/1896664656 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff598066f60 0x7ff598069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:10.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.454+0000 7ff5aeffd640 1 -- 172.21.15.83:0/1896664656 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff5b0120150 con 0x7ff5b815db50 2024-04-17T10:42:10.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.454+0000 7ff5b659c640 1 --2- 172.21.15.83:0/1896664656 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff598066f60 0x7ff598069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:10.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.454+0000 7ff5b659c640 1 --2- 172.21.15.83:0/1896664656 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff598066f60 0x7ff598069420 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8000f70 tx=0x7ff5a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:10.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.458+0000 7ff5aeffd640 1 -- 172.21.15.83:0/1896664656 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff5b0016620 con 0x7ff5b815db50 2024-04-17T10:42:10.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.638+0000 7ff5bc873640 1 -- 172.21.15.83:0/1896664656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5b80b2af0 con 0x7ff5b815db50 2024-04-17T10:42:10.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.638+0000 7ff5aeffd640 1 -- 172.21.15.83:0/1896664656 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4939 (secure 0 0 0) 0x7ff5b00f1120 con 0x7ff5b815db50 2024-04-17T10:42:10.638 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:10.638 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/161398126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":161398126},{"type":"v1","addr":"172.21.15.167:6833","nonce":161398126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1085380618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1085380618},{"type":"v1","addr":"172.21.15.167:6835","nonce":1085380618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/836442291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":836442291},{"type":"v1","addr":"172.21.15.83:6835","nonce":836442291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:41:37.773342+0000","modified":"2024-04-17T10:42:06.483516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5682":{"gid":5682,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.83:6837/1894547386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1894547386},{"type":"v1","addr":"172.21.15.83:6837","nonce":1894547386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-17T10:42:10.639 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 22 2024-04-17T10:42:10.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.642+0000 7ff5bc873640 1 -- 172.21.15.83:0/1896664656 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff598066f60 msgr2=0x7ff598069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:10.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.642+0000 7ff5bc873640 1 --2- 172.21.15.83:0/1896664656 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff598066f60 0x7ff598069420 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8000f70 tx=0x7ff5a8009040 comp rx=0 tx=0).stop 2024-04-17T10:42:10.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.642+0000 7ff5bc873640 1 -- 172.21.15.83:0/1896664656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5b815db50 msgr2=0x7ff5b814e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:10.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.642+0000 7ff5bc873640 1 --2- 172.21.15.83:0/1896664656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5b815db50 0x7ff5b814e3c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff5b00af040 tx=0x7ff5b00197d0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.642+0000 7ff5bc873640 1 -- 172.21.15.83:0/1896664656 shutdown_connections 2024-04-17T10:42:10.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.642+0000 7ff5bc873640 1 --2- 172.21.15.83:0/1896664656 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff598066f60 0x7ff598069420 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.642+0000 7ff5bc873640 1 --2- 172.21.15.83:0/1896664656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5b815db50 0x7ff5b814e3c0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.642+0000 7ff5bc873640 1 --2- 172.21.15.83:0/1896664656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5b80aedd0 0x7ff5b814de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.642+0000 7ff5bc873640 1 --2- 172.21.15.83:0/1896664656 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5b806dae0 0x7ff5b8156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:10.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.642+0000 7ff5bc873640 1 -- 172.21.15.83:0/1896664656 >> 172.21.15.83:0/1896664656 conn(0x7ff5b806abe0 msgr2=0x7ff5b80cf420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:10.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.642+0000 7ff5bc873640 1 -- 172.21.15.83:0/1896664656 shutdown_connections 2024-04-17T10:42:10.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:10.642+0000 7ff5bc873640 1 -- 172.21.15.83:0/1896664656 wait complete. 2024-04-17T10:42:10.656 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/ceph-qa-suite/new_dir_1 2024-04-17T10:42:10.778 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:42:10.778 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0 2024-04-17T10:42:10.779 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:42:10.791 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_1' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-17T10:42:10.792 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:42:10.792 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:42:10.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:42:10.798 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_1 bs=1M conv=fdatasync count=1 seek=0) 2024-04-17T10:42:10.860 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:42:10.860 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:42:10.860 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138083 s, 75.9 MB/s 2024-04-17T10:42:10.860 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_1' 2024-04-17T10:42:10.860 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:42:11.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f877ef77640 1 Processor -- start 2024-04-17T10:42:11.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f877ef77640 1 -- start start 2024-04-17T10:42:11.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f877ef77640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f877815f1d0 0x7f877815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f877ef77640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f87780d22d0 0x7f87780ae520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f877ef77640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f877815f5b0 con 0x7f87780b2de0 2024-04-17T10:42:11.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f877ef77640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87780b2620 con 0x7f87780d22d0 2024-04-17T10:42:11.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f877ef77640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87780b2340 con 0x7f877815f1d0 2024-04-17T10:42:11.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f8777fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f87780b2de0 0x7f877815fd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46744/0 (socket says 172.21.15.83:46744) 2024-04-17T10:42:11.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f8777fff640 1 -- 172.21.15.83:0/277392992 learned_addr learned my addr 172.21.15.83:0/277392992 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:11.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f877d514640 1 --2- 172.21.15.83:0/277392992 >> v2:172.21.15.167:3300/0 conn(0x7f87780d22d0 0x7f87780ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f877cd13640 1 --2- 172.21.15.83:0/277392992 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f877815f1d0 0x7f877815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f87777fe640 1 -- 172.21.15.83:0/277392992 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1526346362 0 0) 0x7f877815f5b0 con 0x7f87780b2de0 2024-04-17T10:42:11.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f87777fe640 1 -- 172.21.15.83:0/277392992 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8760003530 con 0x7f87780b2de0 2024-04-17T10:42:11.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f87777fe640 1 -- 172.21.15.83:0/277392992 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1596649487 0 0) 0x7f8760003530 con 0x7f87780b2de0 2024-04-17T10:42:11.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f87777fe640 1 -- 172.21.15.83:0/277392992 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f877815f1d0 msgr2=0x7f877815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:11.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f87777fe640 1 --2- 172.21.15.83:0/277392992 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f877815f1d0 0x7f877815f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f87777fe640 1 -- 172.21.15.83:0/277392992 >> v2:172.21.15.167:3300/0 conn(0x7f87780d22d0 msgr2=0x7f87780ae520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:11.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f87777fe640 1 --2- 172.21.15.83:0/277392992 >> v2:172.21.15.167:3300/0 conn(0x7f87780d22d0 0x7f87780ae520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f87777fe640 1 -- 172.21.15.83:0/277392992 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87780b1760 con 0x7f87780b2de0 2024-04-17T10:42:11.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f87777fe640 1 -- 172.21.15.83:0/277392992 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f876c001680 con 0x7f87780b2de0 2024-04-17T10:42:11.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f87777fe640 1 -- 172.21.15.83:0/277392992 >> v1:172.21.15.83:6789/0 conn(0x7f87780b2de0 legacy=0x7f877815fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f87777fe640 1 --2- 172.21.15.83:0/277392992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87600039c0 0x7f8760003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f87777fe640 1 --2- 172.21.15.83:0/277392992 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f877815f1d0 0x7f8760004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f87777fe640 1 --2- 172.21.15.83:0/277392992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8760004d50 0x7f87600051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f87777fe640 1 -- 172.21.15.83:0/277392992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87780b2620 con 0x7f87600039c0 2024-04-17T10:42:11.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f87777fe640 1 -- 172.21.15.83:0/277392992 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87780b2340 con 0x7f8760004d50 2024-04-17T10:42:11.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f87777fe640 1 -- 172.21.15.83:0/277392992 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f876000b9d0 con 0x7f877815f1d0 2024-04-17T10:42:11.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f877cd13640 1 --2- 172.21.15.83:0/277392992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87600039c0 0x7f8760003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f877cd13640 1 -- 172.21.15.83:0/277392992 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f877815f1d0 msgr2=0x7f8760004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:11.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f877cd13640 1 --2- 172.21.15.83:0/277392992 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f877815f1d0 0x7f8760004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f877d514640 1 --2- 172.21.15.83:0/277392992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8760004d50 0x7f87600051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f877cd13640 1 -- 172.21.15.83:0/277392992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8760004d50 msgr2=0x7f87600051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:11.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f877cd13640 1 --2- 172.21.15.83:0/277392992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8760004d50 0x7f87600051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.022+0000 7f877cd13640 1 -- 172.21.15.83:0/277392992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8760003530 con 0x7f87600039c0 2024-04-17T10:42:11.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.026+0000 7f877cd13640 1 --2- 172.21.15.83:0/277392992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87600039c0 0x7f8760003de0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8768009ae0 tx=0x7f876801ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e700e5383e236e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:11.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.026+0000 7f87777fe640 1 -- 172.21.15.83:0/277392992 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87680b2070 con 0x7f87600039c0 2024-04-17T10:42:11.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.026+0000 7f87777fe640 1 -- 172.21.15.83:0/277392992 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87680a5560 con 0x7f87600039c0 2024-04-17T10:42:11.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.026+0000 7f87777fe640 1 -- 172.21.15.83:0/277392992 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87680096b0 con 0x7f87600039c0 2024-04-17T10:42:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.026+0000 7f877ef77640 1 -- 172.21.15.83:0/277392992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87600039c0 msgr2=0x7f8760003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.026+0000 7f877ef77640 1 --2- 172.21.15.83:0/277392992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87600039c0 0x7f8760003de0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8768009ae0 tx=0x7f876801ca80 comp rx=0 tx=0).stop 2024-04-17T10:42:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.026+0000 7f877d514640 1 -- 172.21.15.83:0/277392992 reap_dead start 2024-04-17T10:42:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.026+0000 7f877ef77640 1 -- 172.21.15.83:0/277392992 shutdown_connections 2024-04-17T10:42:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.026+0000 7f877ef77640 1 -- 172.21.15.83:0/277392992 >> 172.21.15.83:0/277392992 conn(0x7f87780d17b0 msgr2=0x7f87780d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.026+0000 7f877ef77640 1 -- 172.21.15.83:0/277392992 shutdown_connections 2024-04-17T10:42:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.026+0000 7f877ef77640 1 -- 172.21.15.83:0/277392992 wait complete. 2024-04-17T10:42:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.026+0000 7f877ef77640 1 Processor -- start 2024-04-17T10:42:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.026+0000 7f877ef77640 1 -- start start 2024-04-17T10:42:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.030+0000 7f877ef77640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87600039c0 0x7f8778159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.030+0000 7f877ef77640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8778159a30 0x7f8778158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.030+0000 7f877ef77640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f877806eb50 0x7f877806efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.030+0000 7f877ef77640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87780af3f0 con 0x7f87600039c0 2024-04-17T10:42:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.030+0000 7f877ef77640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87780af270 con 0x7f877806eb50 2024-04-17T10:42:11.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.030+0000 7f877ef77640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87780af570 con 0x7f8778159a30 2024-04-17T10:42:11.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.030+0000 7f877cd13640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87600039c0 0x7f8778159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.030+0000 7f877cd13640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87600039c0 0x7f8778159470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41330/0 (socket says 172.21.15.83:41330) 2024-04-17T10:42:11.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.030+0000 7f877cd13640 1 -- 172.21.15.83:0/1621648205 learned_addr learned my addr 172.21.15.83:0/1621648205 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:11.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.030+0000 7f8777fff640 1 --2- 172.21.15.83:0/1621648205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8778159a30 0x7f8778158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.030+0000 7f877d514640 1 --2- 172.21.15.83:0/1621648205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f877806eb50 0x7f877806efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.030+0000 7f877cd13640 1 -- 172.21.15.83:0/1621648205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8778159a30 msgr2=0x7f8778158d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.030+0000 7f877cd13640 1 --2- 172.21.15.83:0/1621648205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8778159a30 0x7f8778158d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.030+0000 7f877cd13640 1 -- 172.21.15.83:0/1621648205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f877806eb50 msgr2=0x7f877806efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.030+0000 7f877cd13640 1 --2- 172.21.15.83:0/1621648205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f877806eb50 0x7f877806efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.030+0000 7f877cd13640 1 -- 172.21.15.83:0/1621648205 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8778159280 con 0x7f87600039c0 2024-04-17T10:42:11.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.030+0000 7f8777fff640 1 --2- 172.21.15.83:0/1621648205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8778159a30 0x7f8778158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:11.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.030+0000 7f877d514640 1 --2- 172.21.15.83:0/1621648205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f877806eb50 0x7f877806efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:11.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.030+0000 7f877cd13640 1 --2- 172.21.15.83:0/1621648205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87600039c0 0x7f8778159470 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f87680b0040 tx=0x7f8760005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:11.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.030+0000 7f87757fa640 1 -- 172.21.15.83:0/1621648205 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87680bc030 con 0x7f87600039c0 2024-04-17T10:42:11.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.030+0000 7f87757fa640 1 -- 172.21.15.83:0/1621648205 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876800f000 con 0x7f87600039c0 2024-04-17T10:42:11.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.030+0000 7f87757fa640 1 -- 172.21.15.83:0/1621648205 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87680b2040 con 0x7f87600039c0 2024-04-17T10:42:11.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.030+0000 7f877ef77640 1 -- 172.21.15.83:0/1621648205 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87780b7270 con 0x7f87600039c0 2024-04-17T10:42:11.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.030+0000 7f877ef77640 1 -- 172.21.15.83:0/1621648205 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87781fa910 con 0x7f87600039c0 2024-04-17T10:42:11.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.030+0000 7f877ef77640 1 -- 172.21.15.83:0/1621648205 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8778074f90 con 0x7f87600039c0 2024-04-17T10:42:11.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.030+0000 7f87757fa640 1 -- 172.21.15.83:0/1621648205 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f87680bf070 con 0x7f87600039c0 2024-04-17T10:42:11.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.030+0000 7f87757fa640 1 --2- 172.21.15.83:0/1621648205 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8764066f90 0x7f8764069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.030+0000 7f87757fa640 1 -- 172.21.15.83:0/1621648205 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f876811ea90 con 0x7f87600039c0 2024-04-17T10:42:11.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.030+0000 7f8777fff640 1 --2- 172.21.15.83:0/1621648205 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8764066f90 0x7f8764069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.030+0000 7f8777fff640 1 --2- 172.21.15.83:0/1621648205 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8764066f90 0x7f8764069450 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f876c001ec0 tx=0x7f876c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:11.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.034+0000 7f87757fa640 1 -- 172.21.15.83:0/1621648205 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f87680167c0 con 0x7f87600039c0 2024-04-17T10:42:11.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.226+0000 7f877ef77640 1 -- 172.21.15.83:0/1621648205 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8778072690 con 0x7f87600039c0 2024-04-17T10:42:11.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.226+0000 7f87757fa640 1 -- 172.21.15.83:0/1621648205 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f87680efaf0 con 0x7f87600039c0 2024-04-17T10:42:11.225 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:11.226 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/161398126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":161398126},{"type":"v1","addr":"172.21.15.167:6833","nonce":161398126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1085380618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1085380618},{"type":"v1","addr":"172.21.15.167:6835","nonce":1085380618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/836442291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":836442291},{"type":"v1","addr":"172.21.15.83:6835","nonce":836442291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:41:37.773342+0000","modified":"2024-04-17T10:42:06.483516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5682":{"gid":5682,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.83:6837/1894547386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1894547386},{"type":"v1","addr":"172.21.15.83:6837","nonce":1894547386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-17T10:42:11.226 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 22 2024-04-17T10:42:11.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.230+0000 7f877ef77640 1 -- 172.21.15.83:0/1621648205 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8764066f90 msgr2=0x7f8764069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.230+0000 7f877ef77640 1 --2- 172.21.15.83:0/1621648205 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8764066f90 0x7f8764069450 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f876c001ec0 tx=0x7f876c009040 comp rx=0 tx=0).stop 2024-04-17T10:42:11.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.230+0000 7f877ef77640 1 -- 172.21.15.83:0/1621648205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87600039c0 msgr2=0x7f8778159470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.230+0000 7f877ef77640 1 --2- 172.21.15.83:0/1621648205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87600039c0 0x7f8778159470 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f87680b0040 tx=0x7f8760005840 comp rx=0 tx=0).stop 2024-04-17T10:42:11.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.230+0000 7f877ef77640 1 -- 172.21.15.83:0/1621648205 shutdown_connections 2024-04-17T10:42:11.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.230+0000 7f877ef77640 1 --2- 172.21.15.83:0/1621648205 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8764066f90 0x7f8764069450 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.230+0000 7f877ef77640 1 --2- 172.21.15.83:0/1621648205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f877806eb50 0x7f877806efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.230+0000 7f877ef77640 1 --2- 172.21.15.83:0/1621648205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8778159a30 0x7f8778158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.230+0000 7f877ef77640 1 --2- 172.21.15.83:0/1621648205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87600039c0 0x7f8778159470 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.230+0000 7f877ef77640 1 -- 172.21.15.83:0/1621648205 >> 172.21.15.83:0/1621648205 conn(0x7f87780d17b0 msgr2=0x7f87780d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:11.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.230+0000 7f877ef77640 1 -- 172.21.15.83:0/1621648205 shutdown_connections 2024-04-17T10:42:11.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.230+0000 7f877ef77640 1 -- 172.21.15.83:0/1621648205 wait complete. 2024-04-17T10:42:11.246 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/ceph-qa-suite/new_file_1 2024-04-17T10:42:11.373 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:42:11.373 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0 2024-04-17T10:42:11.373 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:42:11.387 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_1' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-17T10:42:11.388 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-04-17T10:42:11.388 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-04-17T10:42:11.388 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-04-17T10:42:11.388 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:42:11.388 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_1").st_ino) 2024-04-17T10:42:11.388 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-04-17T10:42:11.454 INFO:teuthology.orchestra.run.smithi083.stdout:1099511630411 2024-04-17T10:42:11.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:42:11.464 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-17T10:42:11.464 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-17T10:42:11.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe603afe640 1 Processor -- start 2024-04-17T10:42:11.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe603afe640 1 -- start start 2024-04-17T10:42:11.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe603afe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe5fc0b4210 0x7fe5fc15e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe603afe640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5fc15bd70 con 0x7fe5fc154de0 2024-04-17T10:42:11.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe603afe640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5fc15bf60 con 0x7fe5fc06b520 2024-04-17T10:42:11.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe603afe640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5fc0b4870 con 0x7fe5fc0b4210 2024-04-17T10:42:11.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe60189a640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe5fc154de0 0x7fe5fc06ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46760/0 (socket says 172.21.15.83:46760) 2024-04-17T10:42:11.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe60189a640 1 -- 172.21.15.83:0/1545720177 learned_addr learned my addr 172.21.15.83:0/1545720177 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:11.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe600898640 1 -- 172.21.15.83:0/1545720177 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2985396021 0 0) 0x7fe5fc15bd70 con 0x7fe5fc154de0 2024-04-17T10:42:11.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe600898640 1 -- 172.21.15.83:0/1545720177 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5e8003530 con 0x7fe5fc154de0 2024-04-17T10:42:11.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe60209b640 1 --2- 172.21.15.83:0/1545720177 >> v2:172.21.15.167:3300/0 conn(0x7fe5fc0b4210 0x7fe5fc15e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe600898640 1 -- 172.21.15.83:0/1545720177 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2898483918 0 0) 0x7fe5e8003530 con 0x7fe5fc154de0 2024-04-17T10:42:11.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe600898640 1 -- 172.21.15.83:0/1545720177 >> v2:172.21.15.167:3300/0 conn(0x7fe5fc0b4210 msgr2=0x7fe5fc15e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:11.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe600898640 1 --2- 172.21.15.83:0/1545720177 >> v2:172.21.15.167:3300/0 conn(0x7fe5fc0b4210 0x7fe5fc15e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe600898640 1 -- 172.21.15.83:0/1545720177 >> v1:172.21.15.167:6789/0 conn(0x7fe5fc06b520 legacy=0x7fe5fc0b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe600898640 1 -- 172.21.15.83:0/1545720177 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5fc15afb0 con 0x7fe5fc154de0 2024-04-17T10:42:11.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe600898640 1 -- 172.21.15.83:0/1545720177 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fe5f8001680 con 0x7fe5fc154de0 2024-04-17T10:42:11.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe600898640 1 -- 172.21.15.83:0/1545720177 >> v1:172.21.15.83:6789/0 conn(0x7fe5fc154de0 legacy=0x7fe5fc06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe600898640 1 --2- 172.21.15.83:0/1545720177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5e8003a80 0x7fe5e8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe600898640 1 --2- 172.21.15.83:0/1545720177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5e8004810 0x7fe5e8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe600898640 1 --2- 172.21.15.83:0/1545720177 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5e80052f0 0x7fe5e800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe600898640 1 -- 172.21.15.83:0/1545720177 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e800c1c0 con 0x7fe5e8004810 2024-04-17T10:42:11.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe600898640 1 -- 172.21.15.83:0/1545720177 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e800c370 con 0x7fe5e8003a80 2024-04-17T10:42:11.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe600898640 1 -- 172.21.15.83:0/1545720177 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e800c570 con 0x7fe5e80052f0 2024-04-17T10:42:11.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe601099640 1 --2- 172.21.15.83:0/1545720177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5e8004810 0x7fe5e8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe60189a640 1 --2- 172.21.15.83:0/1545720177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5e8003a80 0x7fe5e8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe60209b640 1 --2- 172.21.15.83:0/1545720177 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5e80052f0 0x7fe5e800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe601099640 1 -- 172.21.15.83:0/1545720177 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5e80052f0 msgr2=0x7fe5e800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:11.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe601099640 1 --2- 172.21.15.83:0/1545720177 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5e80052f0 0x7fe5e800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe601099640 1 -- 172.21.15.83:0/1545720177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5e8003a80 msgr2=0x7fe5e8003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:11.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe601099640 1 --2- 172.21.15.83:0/1545720177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5e8003a80 0x7fe5e8003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe60209b640 1 -- 172.21.15.83:0/1545720177 reap_dead start 2024-04-17T10:42:11.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.630+0000 7fe601099640 1 -- 172.21.15.83:0/1545720177 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e8003530 con 0x7fe5e8004810 2024-04-17T10:42:11.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.634+0000 7fe601099640 1 --2- 172.21.15.83:0/1545720177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5e8004810 0x7fe5e8004c90 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec001750 tx=0x7fe5ec01c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26ede5dafd9d9ed server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:11.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.634+0000 7fe600898640 1 -- 172.21.15.83:0/1545720177 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec0013a0 con 0x7fe5e8004810 2024-04-17T10:42:11.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.634+0000 7fe600898640 1 -- 172.21.15.83:0/1545720177 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5ec0052f0 con 0x7fe5e8004810 2024-04-17T10:42:11.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.634+0000 7fe600898640 1 -- 172.21.15.83:0/1545720177 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec003840 con 0x7fe5e8004810 2024-04-17T10:42:11.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.634+0000 7fe603afe640 1 -- 172.21.15.83:0/1545720177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5e8004810 msgr2=0x7fe5e8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.634+0000 7fe603afe640 1 --2- 172.21.15.83:0/1545720177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5e8004810 0x7fe5e8004c90 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec001750 tx=0x7fe5ec01c370 comp rx=0 tx=0).stop 2024-04-17T10:42:11.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.634+0000 7fe603afe640 1 -- 172.21.15.83:0/1545720177 shutdown_connections 2024-04-17T10:42:11.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.634+0000 7fe603afe640 1 --2- 172.21.15.83:0/1545720177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5e8004810 0x7fe5e8004c90 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.634+0000 7fe603afe640 1 -- 172.21.15.83:0/1545720177 >> 172.21.15.83:0/1545720177 conn(0x7fe5fc0b15b0 msgr2=0x7fe5fc0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:11.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.634+0000 7fe603afe640 1 -- 172.21.15.83:0/1545720177 shutdown_connections 2024-04-17T10:42:11.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.634+0000 7fe603afe640 1 -- 172.21.15.83:0/1545720177 wait complete. 2024-04-17T10:42:11.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.634+0000 7fe603afe640 1 Processor -- start 2024-04-17T10:42:11.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.634+0000 7fe603afe640 1 -- start start 2024-04-17T10:42:11.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.638+0000 7fe603afe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5e8004810 0x7fe5fc155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.638+0000 7fe603afe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5fc14e780 0x7fe5fc0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.638+0000 7fe603afe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5fc1557f0 0x7fe5fc0add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.638+0000 7fe603afe640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc15b850 con 0x7fe5e8004810 2024-04-17T10:42:11.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.638+0000 7fe603afe640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc15b570 con 0x7fe5fc1557f0 2024-04-17T10:42:11.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.638+0000 7fe603afe640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc1551c0 con 0x7fe5fc14e780 2024-04-17T10:42:11.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.638+0000 7fe60189a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5e8004810 0x7fe5fc155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.638+0000 7fe60189a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5e8004810 0x7fe5fc155d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41344/0 (socket says 172.21.15.83:41344) 2024-04-17T10:42:11.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.638+0000 7fe60189a640 1 -- 172.21.15.83:0/1786313104 learned_addr learned my addr 172.21.15.83:0/1786313104 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:11.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.638+0000 7fe60209b640 1 --2- 172.21.15.83:0/1786313104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5fc1557f0 0x7fe5fc0add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.638+0000 7fe60189a640 1 -- 172.21.15.83:0/1786313104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5fc14e780 msgr2=0x7fe5fc0ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.638+0000 7fe601099640 1 --2- 172.21.15.83:0/1786313104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5fc14e780 0x7fe5fc0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.638+0000 7fe60189a640 1 --2- 172.21.15.83:0/1786313104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5fc14e780 0x7fe5fc0ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.638+0000 7fe60189a640 1 -- 172.21.15.83:0/1786313104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5fc1557f0 msgr2=0x7fe5fc0add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.638+0000 7fe60189a640 1 --2- 172.21.15.83:0/1786313104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5fc1557f0 0x7fe5fc0add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.638+0000 7fe60189a640 1 -- 172.21.15.83:0/1786313104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5fc1f3a50 con 0x7fe5e8004810 2024-04-17T10:42:11.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.638+0000 7fe60209b640 1 --2- 172.21.15.83:0/1786313104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5fc1557f0 0x7fe5fc0add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:11.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.638+0000 7fe60189a640 1 --2- 172.21.15.83:0/1786313104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5e8004810 0x7fe5fc155d40 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8001060 tx=0x7fe5f8011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:11.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.638+0000 7fe5f27fc640 1 -- 172.21.15.83:0/1786313104 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8010040 con 0x7fe5e8004810 2024-04-17T10:42:11.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.638+0000 7fe5f27fc640 1 -- 172.21.15.83:0/1786313104 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f801a070 con 0x7fe5e8004810 2024-04-17T10:42:11.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.638+0000 7fe603afe640 1 -- 172.21.15.83:0/1786313104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5fc1f3d30 con 0x7fe5e8004810 2024-04-17T10:42:11.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.638+0000 7fe5f27fc640 1 -- 172.21.15.83:0/1786313104 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f800fd60 con 0x7fe5e8004810 2024-04-17T10:42:11.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.638+0000 7fe603afe640 1 -- 172.21.15.83:0/1786313104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5fc1f42f0 con 0x7fe5e8004810 2024-04-17T10:42:11.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.638+0000 7fe5f27fc640 1 -- 172.21.15.83:0/1786313104 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe5f801f460 con 0x7fe5e8004810 2024-04-17T10:42:11.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.638+0000 7fe5f27fc640 1 --2- 172.21.15.83:0/1786313104 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe5e0066f60 0x7fe5e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.638+0000 7fe5f27fc640 1 -- 172.21.15.83:0/1786313104 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe5f8088270 con 0x7fe5e8004810 2024-04-17T10:42:11.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.638+0000 7fe601099640 1 --2- 172.21.15.83:0/1786313104 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe5e0066f60 0x7fe5e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.638+0000 7fe601099640 1 --2- 172.21.15.83:0/1786313104 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe5e0066f60 0x7fe5e0069420 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe5e80057f0 tx=0x7fe5ec0011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:11.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.638+0000 7fe606c6b000 1 Processor -- start 2024-04-17T10:42:11.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.638+0000 7fe606c6b000 1 -- start start 2024-04-17T10:42:11.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d5f9db180 0x562d5f9db560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d5f9dbaa0 0x562d5f9910c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562d5f991600 0x562d5f9919e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5f7def20 con 0x562d5f9db180 2024-04-17T10:42:11.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5f767000 con 0x562d5f9dbaa0 2024-04-17T10:42:11.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d5f7dafc0 con 0x562d5f991600 2024-04-17T10:42:11.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe60189a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d5f9db180 0x562d5f9db560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe60189a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d5f9db180 0x562d5f9db560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41346/0 (socket says 172.21.15.83:41346) 2024-04-17T10:42:11.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe60189a640 1 -- 172.21.15.83:0/2582750188 learned_addr learned my addr 172.21.15.83:0/2582750188 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:11.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe60189a640 1 -- 172.21.15.83:0/2582750188 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562d5f991600 msgr2=0x562d5f9919e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:11.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe60189a640 1 --2- 172.21.15.83:0/2582750188 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562d5f991600 0x562d5f9919e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe60189a640 1 -- 172.21.15.83:0/2582750188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d5f9dbaa0 msgr2=0x562d5f9910c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:11.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe60189a640 1 --2- 172.21.15.83:0/2582750188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d5f9dbaa0 0x562d5f9910c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe60189a640 1 -- 172.21.15.83:0/2582750188 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d5f7baba0 con 0x562d5f9db180 2024-04-17T10:42:11.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe60189a640 1 --2- 172.21.15.83:0/2582750188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d5f9db180 0x562d5f9db560 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8086700 tx=0x7fe5f8059300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae67bbcbff186d0e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:11.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe5df7fe640 1 -- 172.21.15.83:0/2582750188 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f80880c0 con 0x562d5f9db180 2024-04-17T10:42:11.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe5df7fe640 1 -- 172.21.15.83:0/2582750188 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f8061250 con 0x562d5f9db180 2024-04-17T10:42:11.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe5df7fe640 1 -- 172.21.15.83:0/2582750188 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f80551b0 con 0x562d5f9db180 2024-04-17T10:42:11.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 1 -- 172.21.15.83:0/2582750188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d5f9db180 msgr2=0x562d5f9db560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 1 --2- 172.21.15.83:0/2582750188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d5f9db180 0x562d5f9db560 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8086700 tx=0x7fe5f8059300 comp rx=0 tx=0).stop 2024-04-17T10:42:11.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 1 -- 172.21.15.83:0/2582750188 shutdown_connections 2024-04-17T10:42:11.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 1 --2- 172.21.15.83:0/2582750188 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562d5f991600 0x562d5f9919e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 1 --2- 172.21.15.83:0/2582750188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d5f9dbaa0 0x562d5f9910c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 1 --2- 172.21.15.83:0/2582750188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d5f9db180 0x562d5f9db560 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 1 -- 172.21.15.83:0/2582750188 >> 172.21.15.83:0/2582750188 conn(0x562d5f997760 msgr2=0x562d5f98f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:11.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 1 -- 172.21.15.83:0/2582750188 shutdown_connections 2024-04-17T10:42:11.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 1 -- 172.21.15.83:0/2582750188 wait complete. 2024-04-17T10:42:11.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 1 -- start start 2024-04-17T10:42:11.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 1 Processor -- start 2024-04-17T10:42:11.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d5f991600 0x562d5f9966a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562d5f9db180 0x562d5f996be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d5f9dbaa0 0x562d5f9c9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5f89c9c0 con 0x562d5f991600 2024-04-17T10:42:11.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5f7995d0 con 0x562d5f9dbaa0 2024-04-17T10:42:11.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d5f79b080 con 0x562d5f9db180 2024-04-17T10:42:11.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 10 client.0 _ll_register_callbacks cb 0x562d5f97ca60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:11.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe60189a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d5f991600 0x562d5f9966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe60189a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d5f991600 0x562d5f9966a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41362/0 (socket says 172.21.15.83:41362) 2024-04-17T10:42:11.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe60189a640 1 -- 172.21.15.83:0/900634687 learned_addr learned my addr 172.21.15.83:0/900634687 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:11.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe60189a640 1 -- 172.21.15.83:0/900634687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562d5f9db180 msgr2=0x562d5f996be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:11.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe60189a640 1 --2- 172.21.15.83:0/900634687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562d5f9db180 0x562d5f996be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe60189a640 1 -- 172.21.15.83:0/900634687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d5f9dbaa0 msgr2=0x562d5f9c9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:11.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe60189a640 1 --2- 172.21.15.83:0/900634687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d5f9dbaa0 0x562d5f9c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe60189a640 1 -- 172.21.15.83:0/900634687 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d5f8255a0 con 0x562d5f991600 2024-04-17T10:42:11.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe60189a640 1 --2- 172.21.15.83:0/900634687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d5f991600 0x562d5f9966a0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8058840 tx=0x7fe5f8000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2409e28a79c903cd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:11.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe5dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:11.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe5dcff9640 1 -- 172.21.15.83:0/900634687 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f805ddb0 con 0x562d5f991600 2024-04-17T10:42:11.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe5dcff9640 1 -- 172.21.15.83:0/900634687 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe5f8086b80 con 0x562d5f991600 2024-04-17T10:42:11.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 1 -- 172.21.15.83:0/900634687 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe5f8061250 con 0x562d5f991600 2024-04-17T10:42:11.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe5dcff9640 1 -- 172.21.15.83:0/900634687 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5f8086da0 con 0x562d5f991600 2024-04-17T10:42:11.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 10 client.6207 fetch_fsmap learned FSMap version 22 2024-04-17T10:42:11.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 1 -- 172.21.15.83:0/900634687 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x562d5f7b62b0 con 0x562d5f991600 2024-04-17T10:42:11.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe5dcff9640 1 -- 172.21.15.83:0/900634687 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f8061250 con 0x562d5f991600 2024-04-17T10:42:11.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe5dcff9640 1 -- 172.21.15.83:0/900634687 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe5f8086070 con 0x562d5f991600 2024-04-17T10:42:11.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 10 client.6207 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T10:42:11.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 10 client.6207 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:42:11.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 20 client.6207 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:11.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 1 --2- 172.21.15.83:0/900634687 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562d5f9c8e60 0x562d5f9c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 4 client.6207 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:42:11.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.642+0000 7fe606c6b000 1 -- 172.21.15.83:0/900634687 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d5f77ac70 con 0x562d5f9c8e60 2024-04-17T10:42:11.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.646+0000 7fe601099640 1 --2- 172.21.15.83:0/900634687 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562d5f9c8e60 0x562d5f9c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.646+0000 7fe5dcff9640 10 client.6207 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:11.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.646+0000 7fe601099640 1 --2- 172.21.15.83:0/900634687 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562d5f9c8e60 0x562d5f9c9e60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2db8c764c5aa688 server_cookie=1a516b0ac54feed4 in_seq=0 out_seq=0 2024-04-17T10:42:11.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.646+0000 7fe5dcff9640 1 -- 172.21.15.83:0/900634687 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562d5f77ac70 con 0x562d5f9c8e60 2024-04-17T10:42:11.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.646+0000 7fe5dcff9640 10 client.6207 handle_command_reply: tid=0 2024-04-17T10:42:11.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.646+0000 7fe606c6b000 1 client.6207 shutdown 2024-04-17T10:42:11.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.646+0000 7fe606c6b000 1 -- 172.21.15.83:0/900634687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d5f991600 msgr2=0x562d5f9966a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.646+0000 7fe606c6b000 1 --2- 172.21.15.83:0/900634687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d5f991600 0x562d5f9966a0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8058840 tx=0x7fe5f8000bd0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.646+0000 7fe606c6b000 1 -- 172.21.15.83:0/900634687 shutdown_connections 2024-04-17T10:42:11.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.646+0000 7fe606c6b000 1 --2- 172.21.15.83:0/900634687 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562d5f9c8e60 0x562d5f9c9e60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.646+0000 7fe606c6b000 1 --2- 172.21.15.83:0/900634687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d5f9dbaa0 0x562d5f9c9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.646+0000 7fe606c6b000 1 --2- 172.21.15.83:0/900634687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562d5f9db180 0x562d5f996be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.646+0000 7fe5dcff9640 0 client.6207 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:11.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.646+0000 7fe606c6b000 1 --2- 172.21.15.83:0/900634687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d5f991600 0x562d5f9966a0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.646+0000 7fe606c6b000 1 -- 172.21.15.83:0/900634687 >> 172.21.15.83:0/900634687 conn(0x562d5f997760 msgr2=0x562d5f9900b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:11.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.646+0000 7fe606c6b000 1 -- 172.21.15.83:0/900634687 shutdown_connections 2024-04-17T10:42:11.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.646+0000 7fe606c6b000 1 -- 172.21.15.83:0/900634687 wait complete. 2024-04-17T10:42:11.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.646+0000 7fe606c6b000 20 client.6207 trim_cache size 0 max 16384 2024-04-17T10:42:11.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.658+0000 7fe606c6b000 1 Processor -- start 2024-04-17T10:42:11.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.658+0000 7fe606c6b000 1 -- start start 2024-04-17T10:42:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.658+0000 7fe606c6b000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d5f997760 0x562d5f9f1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.658+0000 7fe606c6b000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562d5f991600 0x562d5f990120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.658+0000 7fe606c6b000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d5f9db180 0x562d5f990660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.658+0000 7fe606c6b000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc2006f0 con 0x562d5f997760 2024-04-17T10:42:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.658+0000 7fe606c6b000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc1ff8c0 con 0x562d5f9db180 2024-04-17T10:42:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.658+0000 7fe606c6b000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc1530e0 con 0x562d5f991600 2024-04-17T10:42:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.658+0000 7fe60189a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d5f997760 0x562d5f9f1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.658+0000 7fe60189a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d5f997760 0x562d5f9f1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41374/0 (socket says 172.21.15.83:41374) 2024-04-17T10:42:11.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.658+0000 7fe60189a640 1 -- 172.21.15.83:0/96699560 learned_addr learned my addr 172.21.15.83:0/96699560 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:11.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.658+0000 7fe60189a640 1 -- 172.21.15.83:0/96699560 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562d5f991600 msgr2=0x562d5f990120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:11.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.658+0000 7fe60189a640 1 --2- 172.21.15.83:0/96699560 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562d5f991600 0x562d5f990120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.658+0000 7fe60189a640 1 -- 172.21.15.83:0/96699560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d5f9db180 msgr2=0x562d5f990660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:11.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.658+0000 7fe60189a640 1 --2- 172.21.15.83:0/96699560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d5f9db180 0x562d5f990660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.658+0000 7fe60189a640 1 -- 172.21.15.83:0/96699560 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d5f9ca1d0 con 0x562d5f997760 2024-04-17T10:42:11.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.658+0000 7fe60189a640 1 --2- 172.21.15.83:0/96699560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d5f997760 0x562d5f9f1690 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8011fd0 tx=0x7fe5f806e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abccdf94ffa49d7c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:11.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.658+0000 7fe5deffd640 1 -- 172.21.15.83:0/96699560 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8061250 con 0x562d5f997760 2024-04-17T10:42:11.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.658+0000 7fe5deffd640 1 -- 172.21.15.83:0/96699560 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f8070020 con 0x562d5f997760 2024-04-17T10:42:11.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.658+0000 7fe5deffd640 1 -- 172.21.15.83:0/96699560 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f805dbb0 con 0x562d5f997760 2024-04-17T10:42:11.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.658+0000 7fe606c6b000 1 -- 172.21.15.83:0/96699560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d5f997760 msgr2=0x562d5f9f1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.658+0000 7fe606c6b000 1 --2- 172.21.15.83:0/96699560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d5f997760 0x562d5f9f1690 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8011fd0 tx=0x7fe5f806e070 comp rx=0 tx=0).stop 2024-04-17T10:42:11.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.658+0000 7fe606c6b000 1 -- 172.21.15.83:0/96699560 shutdown_connections 2024-04-17T10:42:11.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.658+0000 7fe606c6b000 1 --2- 172.21.15.83:0/96699560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d5f9db180 0x562d5f990660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.658+0000 7fe606c6b000 1 --2- 172.21.15.83:0/96699560 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562d5f991600 0x562d5f990120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.658+0000 7fe606c6b000 1 --2- 172.21.15.83:0/96699560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d5f997760 0x562d5f9f1690 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.658+0000 7fe606c6b000 1 -- 172.21.15.83:0/96699560 >> 172.21.15.83:0/96699560 conn(0x562d5f9c95b0 msgr2=0x562d5f9e8770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:11.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.658+0000 7fe606c6b000 1 -- 172.21.15.83:0/96699560 shutdown_connections 2024-04-17T10:42:11.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.658+0000 7fe606c6b000 1 -- 172.21.15.83:0/96699560 wait complete. 2024-04-17T10:42:11.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.658+0000 7fe606c6b000 1 -- start start 2024-04-17T10:42:11.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.658+0000 7fe606c6b000 1 Processor -- start 2024-04-17T10:42:11.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe606c6b000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562d5f991600 0x562d5fa439b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe606c6b000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d5f997760 0x562d5fa43ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe606c6b000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d5f9db180 0x562d5f9efd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe606c6b000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5f7de230 con 0x562d5f997760 2024-04-17T10:42:11.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe606c6b000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc1ff2c0 con 0x562d5f9db180 2024-04-17T10:42:11.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe606c6b000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d5f7a27b0 con 0x562d5f991600 2024-04-17T10:42:11.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe606c6b000 10 client.0 _ll_register_callbacks cb 0x562d5f95d600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:11.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe601099640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d5f997760 0x562d5fa43ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe601099640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d5f997760 0x562d5fa43ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41386/0 (socket says 172.21.15.83:41386) 2024-04-17T10:42:11.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe601099640 1 -- 172.21.15.83:0/3080162910 learned_addr learned my addr 172.21.15.83:0/3080162910 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:11.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe60209b640 1 --2- 172.21.15.83:0/3080162910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d5f9db180 0x562d5f9efd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe601099640 1 -- 172.21.15.83:0/3080162910 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562d5f991600 msgr2=0x562d5fa439b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:11.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe601099640 1 --2- 172.21.15.83:0/3080162910 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562d5f991600 0x562d5fa439b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe601099640 1 -- 172.21.15.83:0/3080162910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d5f9db180 msgr2=0x562d5f9efd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:11.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe601099640 1 --2- 172.21.15.83:0/3080162910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d5f9db180 0x562d5f9efd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe601099640 1 -- 172.21.15.83:0/3080162910 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d5f767250 con 0x562d5f997760 2024-04-17T10:42:11.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe601099640 1 --2- 172.21.15.83:0/3080162910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d5f997760 0x562d5fa43ef0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec01c850 tx=0x7fe5ec0050d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec373ce20c2303c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:11.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe5dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:11.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe5dcff9640 1 -- 172.21.15.83:0/3080162910 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec0c3520 con 0x562d5f997760 2024-04-17T10:42:11.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe5dcff9640 1 -- 172.21.15.83:0/3080162910 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe5ec0c36c0 con 0x562d5f997760 2024-04-17T10:42:11.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe606c6b000 1 -- 172.21.15.83:0/3080162910 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe5f8070020 con 0x562d5f997760 2024-04-17T10:42:11.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe5dcff9640 1 -- 172.21.15.83:0/3080162910 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5ec0d1050 con 0x562d5f997760 2024-04-17T10:42:11.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe606c6b000 10 client.6213 fetch_fsmap learned FSMap version 22 2024-04-17T10:42:11.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe606c6b000 1 -- 172.21.15.83:0/3080162910 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x562d5f773b90 con 0x562d5f997760 2024-04-17T10:42:11.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe5dcff9640 1 -- 172.21.15.83:0/3080162910 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f8070020 con 0x562d5f997760 2024-04-17T10:42:11.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe5dcff9640 1 -- 172.21.15.83:0/3080162910 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe5ec0d4020 con 0x562d5f997760 2024-04-17T10:42:11.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe606c6b000 10 client.6213 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T10:42:11.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe606c6b000 10 client.6213 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:42:11.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe606c6b000 20 client.6213 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:11.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe606c6b000 1 --2- 172.21.15.83:0/3080162910 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562d5f9dbaa0 0x562d5f9f0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe606c6b000 4 client.6213 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T10:42:11.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe606c6b000 1 -- 172.21.15.83:0/3080162910 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562d5f7ec460 con 0x562d5f9dbaa0 2024-04-17T10:42:11.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe60189a640 1 --2- 172.21.15.83:0/3080162910 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562d5f9dbaa0 0x562d5f9f0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe60189a640 1 --2- 172.21.15.83:0/3080162910 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562d5f9dbaa0 0x562d5f9f0b90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdb07c3124940a93 server_cookie=ec0acaf240fc4435 in_seq=0 out_seq=0 2024-04-17T10:42:11.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe5dcff9640 10 client.6213 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:11.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe5dcff9640 1 -- 172.21.15.83:0/3080162910 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562d5f7ec460 con 0x562d5f9dbaa0 2024-04-17T10:42:11.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe5dcff9640 10 client.6213 handle_command_reply: tid=0 2024-04-17T10:42:11.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe606c6b000 1 client.6213 shutdown 2024-04-17T10:42:11.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe606c6b000 1 -- 172.21.15.83:0/3080162910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d5f997760 msgr2=0x562d5fa43ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe606c6b000 1 --2- 172.21.15.83:0/3080162910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d5f997760 0x562d5fa43ef0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec01c850 tx=0x7fe5ec0050d0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe606c6b000 1 -- 172.21.15.83:0/3080162910 shutdown_connections 2024-04-17T10:42:11.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe606c6b000 1 --2- 172.21.15.83:0/3080162910 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562d5f9dbaa0 0x562d5f9f0b90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe606c6b000 1 --2- 172.21.15.83:0/3080162910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d5f9db180 0x562d5f9efd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe606c6b000 1 --2- 172.21.15.83:0/3080162910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d5f997760 0x562d5fa43ef0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe606c6b000 1 --2- 172.21.15.83:0/3080162910 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562d5f991600 0x562d5fa439b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe606c6b000 1 -- 172.21.15.83:0/3080162910 >> 172.21.15.83:0/3080162910 conn(0x562d5f9c95b0 msgr2=0x562d5f98f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:11.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe5dcff9640 0 client.6213 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:11.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe606c6b000 1 -- 172.21.15.83:0/3080162910 shutdown_connections 2024-04-17T10:42:11.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe606c6b000 1 -- 172.21.15.83:0/3080162910 wait complete. 2024-04-17T10:42:11.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe606c6b000 20 client.6213 trim_cache size 0 max 16384 2024-04-17T10:42:11.668 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-17T10:42:11.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe603afe640 1 -- 172.21.15.83:0/1786313104 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe5e0066f60 msgr2=0x7fe5e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe603afe640 1 --2- 172.21.15.83:0/1786313104 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe5e0066f60 0x7fe5e0069420 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe5e80057f0 tx=0x7fe5ec0011a0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe603afe640 1 -- 172.21.15.83:0/1786313104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5e8004810 msgr2=0x7fe5fc155d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe603afe640 1 --2- 172.21.15.83:0/1786313104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5e8004810 0x7fe5fc155d40 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8001060 tx=0x7fe5f8011ca0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe603afe640 1 -- 172.21.15.83:0/1786313104 shutdown_connections 2024-04-17T10:42:11.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe603afe640 1 --2- 172.21.15.83:0/1786313104 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe5e0066f60 0x7fe5e0069420 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe603afe640 1 --2- 172.21.15.83:0/1786313104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5fc1557f0 0x7fe5fc0add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe603afe640 1 --2- 172.21.15.83:0/1786313104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5fc14e780 0x7fe5fc0ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe603afe640 1 --2- 172.21.15.83:0/1786313104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5e8004810 0x7fe5fc155d40 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe603afe640 1 -- 172.21.15.83:0/1786313104 >> 172.21.15.83:0/1786313104 conn(0x7fe5fc0b15b0 msgr2=0x7fe5fc15c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:11.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe603afe640 1 -- 172.21.15.83:0/1786313104 shutdown_connections 2024-04-17T10:42:11.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.662+0000 7fe603afe640 1 -- 172.21.15.83:0/1786313104 wait complete. 2024-04-17T10:42:11.676 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data rmxattr 10000000a4b.00000000 parent 2024-04-17T10:42:11.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.706+0000 7f392c04b080 1 Processor -- start 2024-04-17T10:42:11.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.706+0000 7f392c04b080 1 -- start start 2024-04-17T10:42:11.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.706+0000 7f392c04b080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53c62660 0x559b53caa380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.706+0000 7f392c04b080 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559b53c8e650 0x559b53c63f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.706+0000 7f392c04b080 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b53c92d30 0x559b53c65230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.706+0000 7f392c04b080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b53c62cd0 con 0x559b53c62660 2024-04-17T10:42:11.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.706+0000 7f392c04b080 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b53c079f0 con 0x559b53c92d30 2024-04-17T10:42:11.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.706+0000 7f392c04b080 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b53c62a40 con 0x559b53c8e650 2024-04-17T10:42:11.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.706+0000 7f3929b69640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53c62660 0x559b53caa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.706+0000 7f3929b69640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53c62660 0x559b53caa380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41416/0 (socket says 172.21.15.83:41416) 2024-04-17T10:42:11.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.706+0000 7f3929b69640 1 -- 172.21.15.83:0/3724742478 learned_addr learned my addr 172.21.15.83:0/3724742478 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:11.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f3929b69640 1 -- 172.21.15.83:0/3724742478 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559b53c8e650 msgr2=0x559b53c63f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:11.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f3929b69640 1 --2- 172.21.15.83:0/3724742478 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559b53c8e650 0x559b53c63f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f3929b69640 1 -- 172.21.15.83:0/3724742478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b53c92d30 msgr2=0x559b53c65230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:11.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f3929b69640 1 --2- 172.21.15.83:0/3724742478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b53c92d30 0x559b53c65230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f3929b69640 1 -- 172.21.15.83:0/3724742478 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b53c90320 con 0x559b53c62660 2024-04-17T10:42:11.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f3929b69640 1 --2- 172.21.15.83:0/3724742478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53c62660 0x559b53caa380 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f391c00a4b0 tx=0x7f391c0033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756405f49df796d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:11.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f391bfff640 1 -- 172.21.15.83:0/3724742478 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391c0ac6e0 con 0x559b53c62660 2024-04-17T10:42:11.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f391bfff640 1 -- 172.21.15.83:0/3724742478 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f391c0ac8a0 con 0x559b53c62660 2024-04-17T10:42:11.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f391bfff640 1 -- 172.21.15.83:0/3724742478 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391c0acc90 con 0x559b53c62660 2024-04-17T10:42:11.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f392c04b080 1 -- 172.21.15.83:0/3724742478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53c62660 msgr2=0x559b53caa380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f392c04b080 1 --2- 172.21.15.83:0/3724742478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53c62660 0x559b53caa380 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f391c00a4b0 tx=0x7f391c0033b0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f392c04b080 1 -- 172.21.15.83:0/3724742478 shutdown_connections 2024-04-17T10:42:11.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f392c04b080 1 --2- 172.21.15.83:0/3724742478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b53c92d30 0x559b53c65230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f392c04b080 1 --2- 172.21.15.83:0/3724742478 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559b53c8e650 0x559b53c63f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f392c04b080 1 --2- 172.21.15.83:0/3724742478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53c62660 0x559b53caa380 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f392c04b080 1 -- 172.21.15.83:0/3724742478 >> 172.21.15.83:0/3724742478 conn(0x559b53c8dac0 msgr2=0x559b53d1d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:11.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f392c04b080 1 -- 172.21.15.83:0/3724742478 shutdown_connections 2024-04-17T10:42:11.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f392c04b080 1 -- 172.21.15.83:0/3724742478 wait complete. 2024-04-17T10:42:11.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f392c04b080 1 Processor -- start 2024-04-17T10:42:11.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f392c04b080 1 -- start start 2024-04-17T10:42:11.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f392c04b080 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559b53c62660 0x559b53d1b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f392c04b080 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b53c8e650 0x559b53d3aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f392c04b080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53c92d30 0x559b53d3b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f392c04b080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b53c905c0 con 0x559b53c92d30 2024-04-17T10:42:11.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f392c04b080 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b53c90860 con 0x559b53c8e650 2024-04-17T10:42:11.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f392c04b080 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b53c63a90 con 0x559b53c62660 2024-04-17T10:42:11.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f392a36a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53c92d30 0x559b53d3b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f392a36a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53c92d30 0x559b53d3b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41428/0 (socket says 172.21.15.83:41428) 2024-04-17T10:42:11.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f392a36a640 1 -- 172.21.15.83:0/1025457337 learned_addr learned my addr 172.21.15.83:0/1025457337 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:11.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f3929b69640 1 --2- 172.21.15.83:0/1025457337 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559b53c62660 0x559b53d1b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f3929368640 1 --2- 172.21.15.83:0/1025457337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b53c8e650 0x559b53d3aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f392a36a640 1 -- 172.21.15.83:0/1025457337 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559b53c62660 msgr2=0x559b53d1b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f392a36a640 1 --2- 172.21.15.83:0/1025457337 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559b53c62660 0x559b53d1b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f392a36a640 1 -- 172.21.15.83:0/1025457337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b53c8e650 msgr2=0x559b53d3aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f392a36a640 1 --2- 172.21.15.83:0/1025457337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b53c8e650 0x559b53d3aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f392a36a640 1 -- 172.21.15.83:0/1025457337 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b53d1db20 con 0x559b53c92d30 2024-04-17T10:42:11.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f392a36a640 1 --2- 172.21.15.83:0/1025457337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53c92d30 0x559b53d3b530 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f392400af90 tx=0x7f3924008800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:11.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f391affd640 1 -- 172.21.15.83:0/1025457337 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3924006760 con 0x559b53c92d30 2024-04-17T10:42:11.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f391affd640 1 -- 172.21.15.83:0/1025457337 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3924006920 con 0x559b53c92d30 2024-04-17T10:42:11.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f391affd640 1 -- 172.21.15.83:0/1025457337 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3924006d10 con 0x559b53c92d30 2024-04-17T10:42:11.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f392c04b080 1 -- 172.21.15.83:0/1025457337 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559b53dcb3f0 con 0x559b53c92d30 2024-04-17T10:42:11.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.710+0000 7f392c04b080 1 -- 172.21.15.83:0/1025457337 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b53d1bbc0 con 0x559b53c92d30 2024-04-17T10:42:11.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.714+0000 7f391affd640 1 -- 172.21.15.83:0/1025457337 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f392401c440 con 0x559b53c92d30 2024-04-17T10:42:11.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.714+0000 7f391affd640 1 --2- 172.21.15.83:0/1025457337 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f390c065970 0x7f390c067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.714+0000 7f391affd640 1 -- 172.21.15.83:0/1025457337 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3924083aa0 con 0x559b53c92d30 2024-04-17T10:42:11.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.714+0000 7f3929b69640 1 --2- 172.21.15.83:0/1025457337 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f390c065970 0x7f390c067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.714+0000 7f392c04b080 1 --2- 172.21.15.83:0/1025457337 >> [v2:172.21.15.167:6800/1776227116,v1:172.21.15.167:6801/1776227116] conn(0x559b53c658b0 0x559b53d35e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.714+0000 7f392c04b080 1 -- 172.21.15.83:0/1025457337 --> [v2:172.21.15.167:6800/1776227116,v1:172.21.15.167:6801/1776227116] -- osd_op(unknown.0.0:1 7.24 7:256cebcc:::10000000a4b.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48) v9 -- 0x559b53cac1a0 con 0x559b53c658b0 2024-04-17T10:42:11.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.714+0000 7f3929b69640 1 --2- 172.21.15.83:0/1025457337 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f390c065970 0x7f390c067e30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f391c01b990 tx=0x7f391c0a5000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:11.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.714+0000 7f3929368640 1 --2- 172.21.15.83:0/1025457337 >> [v2:172.21.15.167:6800/1776227116,v1:172.21.15.167:6801/1776227116] conn(0x559b53c658b0 0x559b53d35e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.714+0000 7f3929368640 1 --2- 172.21.15.83:0/1025457337 >> [v2:172.21.15.167:6800/1776227116,v1:172.21.15.167:6801/1776227116] conn(0x559b53c658b0 0x559b53d35e60 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:11.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.718+0000 7f3929368640 1 -- 172.21.15.83:0/1025457337 <== osd.5 v2:172.21.15.167:6800/1776227116 1 ==== osd_op_reply(1 10000000a4b.00000000 [rmxattr] v48'22 uv22 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f3920009590 con 0x559b53c658b0 2024-04-17T10:42:11.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.718+0000 7f392c04b080 1 -- 172.21.15.83:0/1025457337 >> [v2:172.21.15.167:6800/1776227116,v1:172.21.15.167:6801/1776227116] conn(0x559b53c658b0 msgr2=0x559b53d35e60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.718+0000 7f392c04b080 1 --2- 172.21.15.83:0/1025457337 >> [v2:172.21.15.167:6800/1776227116,v1:172.21.15.167:6801/1776227116] conn(0x559b53c658b0 0x559b53d35e60 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.718+0000 7f392c04b080 1 -- 172.21.15.83:0/1025457337 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f390c065970 msgr2=0x7f390c067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.718+0000 7f392c04b080 1 --2- 172.21.15.83:0/1025457337 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f390c065970 0x7f390c067e30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f391c01b990 tx=0x7f391c0a5000 comp rx=0 tx=0).stop 2024-04-17T10:42:11.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.718+0000 7f392c04b080 1 -- 172.21.15.83:0/1025457337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53c92d30 msgr2=0x559b53d3b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.718+0000 7f392c04b080 1 --2- 172.21.15.83:0/1025457337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53c92d30 0x559b53d3b530 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f392400af90 tx=0x7f3924008800 comp rx=0 tx=0).stop 2024-04-17T10:42:11.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.718+0000 7f392a36a640 1 -- 172.21.15.83:0/1025457337 reap_dead start 2024-04-17T10:42:11.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.718+0000 7f392c04b080 1 -- 172.21.15.83:0/1025457337 shutdown_connections 2024-04-17T10:42:11.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.718+0000 7f392c04b080 1 -- 172.21.15.83:0/1025457337 >> 172.21.15.83:0/1025457337 conn(0x559b53c8dac0 msgr2=0x559b53ca8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:11.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.718+0000 7f392c04b080 1 -- 172.21.15.83:0/1025457337 shutdown_connections 2024-04-17T10:42:11.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.718+0000 7f392c04b080 1 -- 172.21.15.83:0/1025457337 wait complete. 2024-04-17T10:42:11.721 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_1' 2024-04-17T10:42:11.721 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-17T10:42:11.721 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/new_file_1 2024-04-17T10:42:11.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.878+0000 7ff1623b2640 1 Processor -- start 2024-04-17T10:42:11.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.878+0000 7ff1623b2640 1 -- start start 2024-04-17T10:42:11.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.878+0000 7ff1623b2640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff15c154de0 0x7ff15c06ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.878+0000 7ff1623b2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff15c0b4210 0x7ff15c06cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.878+0000 7ff1623b2640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff15c15bd70 con 0x7ff15c06b350 2024-04-17T10:42:11.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.878+0000 7ff1623b2640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff15c0b4870 con 0x7ff15c0b4210 2024-04-17T10:42:11.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.878+0000 7ff1623b2640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c15bf60 con 0x7ff15c154de0 2024-04-17T10:42:11.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.878+0000 7ff15b7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff15c06b350 0x7ff15c0b3b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46762/0 (socket says 172.21.15.83:46762) 2024-04-17T10:42:11.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.878+0000 7ff15b7fe640 1 -- 172.21.15.83:0/2798057143 learned_addr learned my addr 172.21.15.83:0/2798057143 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:11.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.878+0000 7ff16094f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff15c0b4210 0x7ff15c06cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.878+0000 7ff16094f640 1 -- 172.21.15.83:0/2798057143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff15c154de0 msgr2=0x7ff15c06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:11.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.878+0000 7ff15bfff640 1 --2- 172.21.15.83:0/2798057143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff15c154de0 0x7ff15c06ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.878+0000 7ff16094f640 1 --2- 172.21.15.83:0/2798057143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff15c154de0 0x7ff15c06ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.878+0000 7ff15affd640 1 -- 172.21.15.83:0/2798057143 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 587738617 0 0) 0x7ff15c15bd70 con 0x7ff15c06b350 2024-04-17T10:42:11.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.878+0000 7ff16094f640 1 -- 172.21.15.83:0/2798057143 >> v1:172.21.15.83:6789/0 conn(0x7ff15c06b350 legacy=0x7ff15c0b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.878+0000 7ff16094f640 1 -- 172.21.15.83:0/2798057143 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff15c15afb0 con 0x7ff15c0b4210 2024-04-17T10:42:11.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.882+0000 7ff16094f640 1 --2- 172.21.15.83:0/2798057143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff15c0b4210 0x7ff15c06cbe0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff1540097c0 tx=0x7ff15401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a0519156167559 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:11.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.882+0000 7ff15affd640 1 -- 172.21.15.83:0/2798057143 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1540b0070 con 0x7ff15c0b4210 2024-04-17T10:42:11.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.882+0000 7ff15affd640 1 -- 172.21.15.83:0/2798057143 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1540ab600 con 0x7ff15c0b4210 2024-04-17T10:42:11.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.882+0000 7ff15affd640 1 -- 172.21.15.83:0/2798057143 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1540ab9d0 con 0x7ff15c0b4210 2024-04-17T10:42:11.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.882+0000 7ff1623b2640 1 -- 172.21.15.83:0/2798057143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff15c0b4210 msgr2=0x7ff15c06cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.882+0000 7ff1623b2640 1 --2- 172.21.15.83:0/2798057143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff15c0b4210 0x7ff15c06cbe0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff1540097c0 tx=0x7ff15401c4a0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.882+0000 7ff1623b2640 1 -- 172.21.15.83:0/2798057143 shutdown_connections 2024-04-17T10:42:11.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.882+0000 7ff1623b2640 1 --2- 172.21.15.83:0/2798057143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff15c0b4210 0x7ff15c06cbe0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.882+0000 7ff1623b2640 1 --2- 172.21.15.83:0/2798057143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff15c154de0 0x7ff15c06ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.882+0000 7ff1623b2640 1 -- 172.21.15.83:0/2798057143 >> 172.21.15.83:0/2798057143 conn(0x7ff15c0b15b0 msgr2=0x7ff15c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:11.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.882+0000 7ff1623b2640 1 -- 172.21.15.83:0/2798057143 shutdown_connections 2024-04-17T10:42:11.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.882+0000 7ff1623b2640 1 -- 172.21.15.83:0/2798057143 wait complete. 2024-04-17T10:42:11.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.882+0000 7ff1623b2640 1 Processor -- start 2024-04-17T10:42:11.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.882+0000 7ff1623b2640 1 -- start start 2024-04-17T10:42:11.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.882+0000 7ff1623b2640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff15c06b350 0x7ff15c0adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.882+0000 7ff1623b2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff15c0b4210 0x7ff15c13dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.882+0000 7ff1623b2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff15c154de0 0x7ff15c13e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.882+0000 7ff1623b2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c1551c0 con 0x7ff15c0b4210 2024-04-17T10:42:11.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.882+0000 7ff1623b2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c06b790 con 0x7ff15c154de0 2024-04-17T10:42:11.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.882+0000 7ff1623b2640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c15e390 con 0x7ff15c06b350 2024-04-17T10:42:11.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.882+0000 7ff15b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff15c0b4210 0x7ff15c13dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.882+0000 7ff15b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff15c0b4210 0x7ff15c13dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41456/0 (socket says 172.21.15.83:41456) 2024-04-17T10:42:11.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.882+0000 7ff15b7fe640 1 -- 172.21.15.83:0/2076496454 learned_addr learned my addr 172.21.15.83:0/2076496454 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:11.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.882+0000 7ff15b7fe640 1 -- 172.21.15.83:0/2076496454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff15c06b350 msgr2=0x7ff15c0adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.882+0000 7ff15b7fe640 1 --2- 172.21.15.83:0/2076496454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff15c06b350 0x7ff15c0adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.882+0000 7ff15b7fe640 1 -- 172.21.15.83:0/2076496454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff15c154de0 msgr2=0x7ff15c13e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.882+0000 7ff15b7fe640 1 --2- 172.21.15.83:0/2076496454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff15c154de0 0x7ff15c13e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.882+0000 7ff15b7fe640 1 -- 172.21.15.83:0/2076496454 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff15c0ce540 con 0x7ff15c0b4210 2024-04-17T10:42:11.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.882+0000 7ff15b7fe640 1 --2- 172.21.15.83:0/2076496454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff15c0b4210 0x7ff15c13dca0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff1440011f0 tx=0x7ff14400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:11.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.882+0000 7ff158ff9640 1 -- 172.21.15.83:0/2076496454 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff144018070 con 0x7ff15c0b4210 2024-04-17T10:42:11.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.886+0000 7ff158ff9640 1 -- 172.21.15.83:0/2076496454 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1440074e0 con 0x7ff15c0b4210 2024-04-17T10:42:11.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.886+0000 7ff1623b2640 1 -- 172.21.15.83:0/2076496454 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff15c0ce820 con 0x7ff15c0b4210 2024-04-17T10:42:11.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.886+0000 7ff158ff9640 1 -- 172.21.15.83:0/2076496454 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff144004a10 con 0x7ff15c0b4210 2024-04-17T10:42:11.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.886+0000 7ff1623b2640 1 -- 172.21.15.83:0/2076496454 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff15c0adfe0 con 0x7ff15c0b4210 2024-04-17T10:42:11.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.886+0000 7ff158ff9640 1 -- 172.21.15.83:0/2076496454 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff14401b070 con 0x7ff15c0b4210 2024-04-17T10:42:11.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.886+0000 7ff158ff9640 1 --2- 172.21.15.83:0/2076496454 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff13c066f60 0x7ff13c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.886+0000 7ff158ff9640 1 -- 172.21.15.83:0/2076496454 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff1440843a0 con 0x7ff15c0b4210 2024-04-17T10:42:11.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.886+0000 7ff15bfff640 1 --2- 172.21.15.83:0/2076496454 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff13c066f60 0x7ff13c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.886+0000 7ff15bfff640 1 --2- 172.21.15.83:0/2076496454 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff13c066f60 0x7ff13c069420 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff14c007dc0 tx=0x7ff14c0072f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:11.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.886+0000 7ff16551f000 1 Processor -- start 2024-04-17T10:42:11.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.886+0000 7ff16551f000 1 -- start start 2024-04-17T10:42:11.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.886+0000 7ff16551f000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c5fbf1f0 0x5574c5fbf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.886+0000 7ff16551f000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5574c5fbfb10 0x5574c5f723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.886+0000 7ff16551f000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c5f72920 0x5574c5f72d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.886+0000 7ff16551f000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574c5dc2ce0 con 0x5574c5f72920 2024-04-17T10:42:11.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.886+0000 7ff16551f000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574c5d4b530 con 0x5574c5fbf1f0 2024-04-17T10:42:11.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.886+0000 7ff16551f000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574c5dbed80 con 0x5574c5fbfb10 2024-04-17T10:42:11.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.886+0000 7ff16094f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c5f72920 0x5574c5f72d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.886+0000 7ff16094f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c5f72920 0x5574c5f72d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41462/0 (socket says 172.21.15.83:41462) 2024-04-17T10:42:11.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.886+0000 7ff16094f640 1 -- 172.21.15.83:0/1762577089 learned_addr learned my addr 172.21.15.83:0/1762577089 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:11.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff15bfff640 1 --2- 172.21.15.83:0/1762577089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c5fbf1f0 0x5574c5fbf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16094f640 1 -- 172.21.15.83:0/1762577089 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5574c5fbfb10 msgr2=0x5574c5f723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:11.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff15b7fe640 1 --2- 172.21.15.83:0/1762577089 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5574c5fbfb10 0x5574c5f723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16094f640 1 --2- 172.21.15.83:0/1762577089 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5574c5fbfb10 0x5574c5f723e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16094f640 1 -- 172.21.15.83:0/1762577089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c5fbf1f0 msgr2=0x5574c5fbf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:11.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16094f640 1 --2- 172.21.15.83:0/1762577089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c5fbf1f0 0x5574c5fbf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16094f640 1 -- 172.21.15.83:0/1762577089 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574c5dbdcd0 con 0x5574c5f72920 2024-04-17T10:42:11.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff15bfff640 1 --2- 172.21.15.83:0/1762577089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c5fbf1f0 0x5574c5fbf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:11.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16094f640 1 --2- 172.21.15.83:0/1762577089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c5f72920 0x5574c5f72d00 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff15401ab50 tx=0x7ff1540046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbfcbdd92c3901fc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:11.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff151ffb640 1 -- 172.21.15.83:0/1762577089 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1540b0040 con 0x5574c5f72920 2024-04-17T10:42:11.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff151ffb640 1 -- 172.21.15.83:0/1762577089 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1540b4410 con 0x5574c5f72920 2024-04-17T10:42:11.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff151ffb640 1 -- 172.21.15.83:0/1762577089 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1540b47e0 con 0x5574c5f72920 2024-04-17T10:42:11.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16551f000 1 -- 172.21.15.83:0/1762577089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c5f72920 msgr2=0x5574c5f72d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16551f000 1 --2- 172.21.15.83:0/1762577089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c5f72920 0x5574c5f72d00 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff15401ab50 tx=0x7ff1540046d0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16551f000 1 -- 172.21.15.83:0/1762577089 shutdown_connections 2024-04-17T10:42:11.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16551f000 1 --2- 172.21.15.83:0/1762577089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c5f72920 0x5574c5f72d00 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16551f000 1 --2- 172.21.15.83:0/1762577089 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5574c5fbfb10 0x5574c5f723e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16551f000 1 --2- 172.21.15.83:0/1762577089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c5fbf1f0 0x5574c5fbf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16551f000 1 -- 172.21.15.83:0/1762577089 >> 172.21.15.83:0/1762577089 conn(0x5574c5f9ff90 msgr2=0x5574c5fa2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:11.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16551f000 1 -- 172.21.15.83:0/1762577089 shutdown_connections 2024-04-17T10:42:11.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16551f000 1 -- 172.21.15.83:0/1762577089 wait complete. 2024-04-17T10:42:11.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16551f000 1 -- start start 2024-04-17T10:42:11.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16551f000 1 Processor -- start 2024-04-17T10:42:11.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16551f000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c5f72920 0x5574c5fc6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16551f000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5574c5fbf1f0 0x5574c5fc6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16551f000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c5fbfb10 0x5574c5f78a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16551f000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574c5d7dbc0 con 0x5574c5f72920 2024-04-17T10:42:11.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16551f000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574c5dc7310 con 0x5574c5fbfb10 2024-04-17T10:42:11.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16551f000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574c5d8f0b0 con 0x5574c5fbf1f0 2024-04-17T10:42:11.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16551f000 10 client.0 _ll_register_callbacks cb 0x5574c5f604a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:11.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff15bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c5f72920 0x5574c5fc6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff15bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c5f72920 0x5574c5fc6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41470/0 (socket says 172.21.15.83:41470) 2024-04-17T10:42:11.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff15bfff640 1 -- 172.21.15.83:0/3589251630 learned_addr learned my addr 172.21.15.83:0/3589251630 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:11.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff15bfff640 1 -- 172.21.15.83:0/3589251630 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5574c5fbf1f0 msgr2=0x5574c5fc6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:11.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff15bfff640 1 --2- 172.21.15.83:0/3589251630 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5574c5fbf1f0 0x5574c5fc6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff15bfff640 1 -- 172.21.15.83:0/3589251630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c5fbfb10 msgr2=0x5574c5f78a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:11.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16094f640 1 --2- 172.21.15.83:0/3589251630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c5fbfb10 0x5574c5f78a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff15bfff640 1 --2- 172.21.15.83:0/3589251630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c5fbfb10 0x5574c5f78a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff15bfff640 1 -- 172.21.15.83:0/3589251630 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574c5dd0950 con 0x5574c5f72920 2024-04-17T10:42:11.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff15bfff640 1 --2- 172.21.15.83:0/3589251630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c5f72920 0x5574c5fc6760 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00f650 tx=0x7ff14c00fa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5623663f03c5046 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:11.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff13affd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:11.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff13affd640 1 -- 172.21.15.83:0/3589251630 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c020070 con 0x5574c5f72920 2024-04-17T10:42:11.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff13affd640 1 -- 172.21.15.83:0/3589251630 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff14c00e8a0 con 0x5574c5f72920 2024-04-17T10:42:11.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16551f000 1 -- 172.21.15.83:0/3589251630 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff1540b4410 con 0x5574c5f72920 2024-04-17T10:42:11.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff13affd640 1 -- 172.21.15.83:0/3589251630 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff14c02b050 con 0x5574c5f72920 2024-04-17T10:42:11.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16551f000 10 client.6237 fetch_fsmap learned FSMap version 22 2024-04-17T10:42:11.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16551f000 1 -- 172.21.15.83:0/3589251630 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x5574c5e3f790 con 0x5574c5f72920 2024-04-17T10:42:11.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff13affd640 1 -- 172.21.15.83:0/3589251630 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1540b4410 con 0x5574c5f72920 2024-04-17T10:42:11.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff13affd640 1 -- 172.21.15.83:0/3589251630 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff14c02e020 con 0x5574c5f72920 2024-04-17T10:42:11.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16551f000 10 client.6237 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T10:42:11.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16551f000 10 client.6237 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:42:11.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16551f000 20 client.6237 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:11.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16551f000 1 --2- 172.21.15.83:0/3589251630 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x5574c5f797c0 0x5574c5f79bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16551f000 4 client.6237 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:42:11.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff16551f000 1 -- 172.21.15.83:0/3589251630 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574c5d5eb80 con 0x5574c5f797c0 2024-04-17T10:42:11.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.890+0000 7ff15b7fe640 1 --2- 172.21.15.83:0/3589251630 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x5574c5f797c0 0x5574c5f79bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.894+0000 7ff15b7fe640 1 --2- 172.21.15.83:0/3589251630 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x5574c5f797c0 0x5574c5f79bd0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9109a7e5dde1ead server_cookie=eecba42c3be3bfcc in_seq=0 out_seq=0 2024-04-17T10:42:11.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.894+0000 7ff13affd640 10 client.6237 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:11.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.894+0000 7ff13affd640 1 -- 172.21.15.83:0/3589251630 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5574c5d5eb80 con 0x5574c5f797c0 2024-04-17T10:42:11.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.894+0000 7ff13affd640 10 client.6237 handle_command_reply: tid=0 2024-04-17T10:42:11.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.894+0000 7ff16551f000 1 client.6237 shutdown 2024-04-17T10:42:11.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.894+0000 7ff16551f000 1 -- 172.21.15.83:0/3589251630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c5f72920 msgr2=0x5574c5fc6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.894+0000 7ff16551f000 1 --2- 172.21.15.83:0/3589251630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c5f72920 0x5574c5fc6760 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00f650 tx=0x7ff14c00fa50 comp rx=0 tx=0).stop 2024-04-17T10:42:11.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.894+0000 7ff16551f000 1 -- 172.21.15.83:0/3589251630 shutdown_connections 2024-04-17T10:42:11.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.894+0000 7ff16551f000 1 --2- 172.21.15.83:0/3589251630 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x5574c5f797c0 0x5574c5f79bd0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.894+0000 7ff16551f000 1 --2- 172.21.15.83:0/3589251630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c5fbfb10 0x5574c5f78a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.894+0000 7ff16551f000 1 --2- 172.21.15.83:0/3589251630 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5574c5fbf1f0 0x5574c5fc6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.894+0000 7ff13affd640 0 client.6237 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:11.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.894+0000 7ff16551f000 1 --2- 172.21.15.83:0/3589251630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c5f72920 0x5574c5fc6760 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.894+0000 7ff16551f000 1 -- 172.21.15.83:0/3589251630 >> 172.21.15.83:0/3589251630 conn(0x5574c5f9ff90 msgr2=0x5574c5f71ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:11.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.894+0000 7ff16551f000 1 -- 172.21.15.83:0/3589251630 shutdown_connections 2024-04-17T10:42:11.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.894+0000 7ff16551f000 1 -- 172.21.15.83:0/3589251630 wait complete. 2024-04-17T10:42:11.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.894+0000 7ff16551f000 20 client.6237 trim_cache size 0 max 16384 2024-04-17T10:42:11.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.906+0000 7ff16551f000 1 Processor -- start 2024-04-17T10:42:11.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.906+0000 7ff16551f000 1 -- start start 2024-04-17T10:42:11.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff16551f000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c5f72920 0x5574c5f70b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff16551f000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c5fbf1f0 0x5574c5f71080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff16551f000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5574c5fbfb10 0x5574c5f715c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff16551f000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c202b40 con 0x5574c5f72920 2024-04-17T10:42:11.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff16551f000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c201d00 con 0x5574c5fbf1f0 2024-04-17T10:42:11.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff16551f000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c201100 con 0x5574c5fbfb10 2024-04-17T10:42:11.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff15bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c5f72920 0x5574c5f70b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff15bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c5f72920 0x5574c5f70b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41474/0 (socket says 172.21.15.83:41474) 2024-04-17T10:42:11.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff15bfff640 1 -- 172.21.15.83:0/3637673337 learned_addr learned my addr 172.21.15.83:0/3637673337 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:11.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff15bfff640 1 -- 172.21.15.83:0/3637673337 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5574c5fbfb10 msgr2=0x5574c5f715c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:11.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff15bfff640 1 --2- 172.21.15.83:0/3637673337 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5574c5fbfb10 0x5574c5f715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff15bfff640 1 -- 172.21.15.83:0/3637673337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c5fbf1f0 msgr2=0x5574c5f71080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:11.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff15bfff640 1 --2- 172.21.15.83:0/3637673337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c5fbf1f0 0x5574c5f71080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff15bfff640 1 -- 172.21.15.83:0/3637673337 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574c5d886f0 con 0x5574c5f72920 2024-04-17T10:42:11.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff15bfff640 1 --2- 172.21.15.83:0/3637673337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c5f72920 0x5574c5f70b40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff14c007fd0 tx=0x7ff14c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483ed540094ab534 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:11.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff1517fa640 1 -- 172.21.15.83:0/3637673337 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c020070 con 0x5574c5f72920 2024-04-17T10:42:11.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff1517fa640 1 -- 172.21.15.83:0/3637673337 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff14c015040 con 0x5574c5f72920 2024-04-17T10:42:11.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff1517fa640 1 -- 172.21.15.83:0/3637673337 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c00ce10 con 0x5574c5f72920 2024-04-17T10:42:11.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff16551f000 1 -- 172.21.15.83:0/3637673337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c5f72920 msgr2=0x5574c5f70b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff16551f000 1 --2- 172.21.15.83:0/3637673337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c5f72920 0x5574c5f70b40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff14c007fd0 tx=0x7ff14c01d000 comp rx=0 tx=0).stop 2024-04-17T10:42:11.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff16551f000 1 -- 172.21.15.83:0/3637673337 shutdown_connections 2024-04-17T10:42:11.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff16551f000 1 --2- 172.21.15.83:0/3637673337 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5574c5fbfb10 0x5574c5f715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff16551f000 1 --2- 172.21.15.83:0/3637673337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c5fbf1f0 0x5574c5f71080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff16551f000 1 --2- 172.21.15.83:0/3637673337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c5f72920 0x5574c5f70b40 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff16551f000 1 -- 172.21.15.83:0/3637673337 >> 172.21.15.83:0/3637673337 conn(0x5574c5f9ff90 msgr2=0x5574c5f79bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:11.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff16551f000 1 -- 172.21.15.83:0/3637673337 shutdown_connections 2024-04-17T10:42:11.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff16551f000 1 -- 172.21.15.83:0/3637673337 wait complete. 2024-04-17T10:42:11.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff16551f000 1 -- start start 2024-04-17T10:42:11.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff16551f000 1 Processor -- start 2024-04-17T10:42:11.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff16551f000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5574c5f72920 0x5574c6028460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff16551f000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c5fbf1f0 0x5574c60289a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff16551f000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c5fbfb10 0x5574c5fc1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff16551f000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574c5d80960 con 0x5574c5fbf1f0 2024-04-17T10:42:11.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff16551f000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c201700 con 0x5574c5fbfb10 2024-04-17T10:42:11.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff16551f000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574c5dc1cb0 con 0x5574c5f72920 2024-04-17T10:42:11.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff16551f000 10 client.0 _ll_register_callbacks cb 0x7ff1540ae070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:11.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff15b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c5fbf1f0 0x5574c60289a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff15b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c5fbf1f0 0x5574c60289a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41484/0 (socket says 172.21.15.83:41484) 2024-04-17T10:42:11.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff15b7fe640 1 -- 172.21.15.83:0/3291036180 learned_addr learned my addr 172.21.15.83:0/3291036180 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:11.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff15b7fe640 1 -- 172.21.15.83:0/3291036180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5574c5f72920 msgr2=0x5574c6028460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:11.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff15b7fe640 1 --2- 172.21.15.83:0/3291036180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5574c5f72920 0x5574c6028460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff15b7fe640 1 -- 172.21.15.83:0/3291036180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c5fbfb10 msgr2=0x5574c5fc1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:11.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff15b7fe640 1 --2- 172.21.15.83:0/3291036180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c5fbfb10 0x5574c5fc1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff15b7fe640 1 -- 172.21.15.83:0/3291036180 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574c5d7ec80 con 0x5574c5fbf1f0 2024-04-17T10:42:11.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff15b7fe640 1 --2- 172.21.15.83:0/3291036180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c5fbf1f0 0x5574c60289a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff14405cee0 tx=0x7ff144054930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=429f867e63f0a55b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:11.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff13affd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:11.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff13affd640 1 -- 172.21.15.83:0/3291036180 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff144053000 con 0x5574c5fbf1f0 2024-04-17T10:42:11.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff13affd640 1 -- 172.21.15.83:0/3291036180 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff1440840b0 con 0x5574c5fbf1f0 2024-04-17T10:42:11.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff16551f000 1 -- 172.21.15.83:0/3291036180 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff14c015040 con 0x5574c5fbf1f0 2024-04-17T10:42:11.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff13affd640 1 -- 172.21.15.83:0/3291036180 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff144071050 con 0x5574c5fbf1f0 2024-04-17T10:42:11.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff16551f000 10 client.6243 fetch_fsmap learned FSMap version 22 2024-04-17T10:42:11.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.910+0000 7ff16551f000 1 -- 172.21.15.83:0/3291036180 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x5574c5a7f600 con 0x5574c5fbf1f0 2024-04-17T10:42:11.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff13affd640 1 -- 172.21.15.83:0/3291036180 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff14c015040 con 0x5574c5fbf1f0 2024-04-17T10:42:11.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff13affd640 1 -- 172.21.15.83:0/3291036180 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff144004bb0 con 0x5574c5fbf1f0 2024-04-17T10:42:11.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff16551f000 10 client.6243 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T10:42:11.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff16551f000 10 client.6243 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:42:11.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff16551f000 20 client.6243 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:11.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff16551f000 1 --2- 172.21.15.83:0/3291036180 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x5574c5fc2db0 0x5574c5fc31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:11.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff16551f000 4 client.6243 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_1"}] 2024-04-17T10:42:11.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff16551f000 1 -- 172.21.15.83:0/3291036180 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_1"}) v1 -- 0x5574c5dd0220 con 0x5574c5fc2db0 2024-04-17T10:42:11.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff15bfff640 1 --2- 172.21.15.83:0/3291036180 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x5574c5fc2db0 0x5574c5fc31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:11.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff13affd640 10 client.6243 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:11.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff15bfff640 1 --2- 172.21.15.83:0/3291036180 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x5574c5fc2db0 0x5574c5fc31c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd4fe840dcf5d94a server_cookie=e0c154b0663c438d in_seq=0 out_seq=0 2024-04-17T10:42:11.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff13affd640 1 -- 172.21.15.83:0/3291036180 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5574c5dd0220 con 0x5574c5fc2db0 2024-04-17T10:42:11.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff13affd640 10 client.6243 handle_command_reply: tid=0 2024-04-17T10:42:11.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff16551f000 1 client.6243 shutdown 2024-04-17T10:42:11.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff16551f000 1 -- 172.21.15.83:0/3291036180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c5fbf1f0 msgr2=0x5574c60289a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff16551f000 1 --2- 172.21.15.83:0/3291036180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c5fbf1f0 0x5574c60289a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff14405cee0 tx=0x7ff144054930 comp rx=0 tx=0).stop 2024-04-17T10:42:11.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff16551f000 1 -- 172.21.15.83:0/3291036180 shutdown_connections 2024-04-17T10:42:11.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff16551f000 1 --2- 172.21.15.83:0/3291036180 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x5574c5fc2db0 0x5574c5fc31c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff16551f000 1 --2- 172.21.15.83:0/3291036180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c5fbfb10 0x5574c5fc1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff13affd640 0 client.6243 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:11.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff16551f000 1 --2- 172.21.15.83:0/3291036180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c5fbf1f0 0x5574c60289a0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff16551f000 1 --2- 172.21.15.83:0/3291036180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5574c5f72920 0x5574c6028460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff16551f000 1 -- 172.21.15.83:0/3291036180 >> 172.21.15.83:0/3291036180 conn(0x5574c5f9ff90 msgr2=0x5574c5fccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:11.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff16551f000 1 -- 172.21.15.83:0/3291036180 shutdown_connections 2024-04-17T10:42:11.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff16551f000 1 -- 172.21.15.83:0/3291036180 wait complete. 2024-04-17T10:42:11.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff16551f000 20 client.6243 trim_cache size 0 max 16384 2024-04-17T10:42:11.919 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:42:11.919 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-17T10:42:11.919 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "643c8418-ed65-4681-8d66-99229480f719", 2024-04-17T10:42:11.919 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-17T10:42:11.919 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:42:11.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff1623b2640 1 -- 172.21.15.83:0/2076496454 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff13c066f60 msgr2=0x7ff13c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff1623b2640 1 --2- 172.21.15.83:0/2076496454 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff13c066f60 0x7ff13c069420 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff14c007dc0 tx=0x7ff14c0072f0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff1623b2640 1 -- 172.21.15.83:0/2076496454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff15c0b4210 msgr2=0x7ff15c13dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:11.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff1623b2640 1 --2- 172.21.15.83:0/2076496454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff15c0b4210 0x7ff15c13dca0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff1440011f0 tx=0x7ff14400c9b0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.920 INFO:tasks.ceph.mds.c.smithi083.stderr:2024-04-17T10:42:11.914+0000 7f894694d640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a4b [2,head] /scrub_test_path/ceph-qa-suite/new_file_1 auth v634 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5844=0-4194304@1} caps={5844=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@3},l=5844 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55890254cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000a4b:[<0x10000000001/new_file_1 v634>,<0x10000000000/ceph-qa-suite v92>,<0x1/scrub_test_path v44>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2024-04-17T10:42:11.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff1623b2640 1 -- 172.21.15.83:0/2076496454 shutdown_connections 2024-04-17T10:42:11.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff1623b2640 1 --2- 172.21.15.83:0/2076496454 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff13c066f60 0x7ff13c069420 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff1623b2640 1 --2- 172.21.15.83:0/2076496454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff15c154de0 0x7ff15c13e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff1623b2640 1 --2- 172.21.15.83:0/2076496454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff15c0b4210 0x7ff15c13dca0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff1623b2640 1 --2- 172.21.15.83:0/2076496454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff15c06b350 0x7ff15c0adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:11.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff1623b2640 1 -- 172.21.15.83:0/2076496454 >> 172.21.15.83:0/2076496454 conn(0x7ff15c0b15b0 msgr2=0x7ff15c071e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:11.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff1623b2640 1 -- 172.21.15.83:0/2076496454 shutdown_connections 2024-04-17T10:42:11.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:11.914+0000 7ff1623b2640 1 -- 172.21.15.83:0/2076496454 wait complete. 2024-04-17T10:42:11.927 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_1' returned '{'return_code': 0, 'scrub_tag': '643c8418-ed65-4681-8d66-99229480f719', 'mode': 'asynchronous'}' 2024-04-17T10:42:11.927 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-17T10:42:11.927 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-17T10:42:12.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.042+0000 7f52ef271640 1 Processor -- start 2024-04-17T10:42:12.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.042+0000 7f52ef271640 1 -- start start 2024-04-17T10:42:12.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.042+0000 7f52ef271640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f52e815b6e0 0x7f52e8071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:12.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.042+0000 7f52ef271640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f52e8070470 0x7f52e806ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:12.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.042+0000 7f52ef271640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52e806c1f0 con 0x7f52e815bac0 2024-04-17T10:42:12.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.042+0000 7f52ef271640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52e8070a10 con 0x7f52e8070470 2024-04-17T10:42:12.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.042+0000 7f52ef271640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52e8070cf0 con 0x7f52e815b6e0 2024-04-17T10:42:12.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.042+0000 7f52ed80e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f52e8070470 0x7f52e806ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:12.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.042+0000 7f52ed80e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f52e8070470 0x7f52e806ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41490/0 (socket says 172.21.15.83:41490) 2024-04-17T10:42:12.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.042+0000 7f52ed80e640 1 -- 172.21.15.83:0/3801703122 learned_addr learned my addr 172.21.15.83:0/3801703122 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:12.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.042+0000 7f52ed00d640 1 --2- 172.21.15.83:0/3801703122 >> v2:172.21.15.167:3300/0 conn(0x7f52e815b6e0 0x7f52e8071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:12.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.042+0000 7f52ed80e640 1 -- 172.21.15.83:0/3801703122 >> v2:172.21.15.167:3300/0 conn(0x7f52e815b6e0 msgr2=0x7f52e8071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:12.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.042+0000 7f52ed80e640 1 --2- 172.21.15.83:0/3801703122 >> v2:172.21.15.167:3300/0 conn(0x7f52e815b6e0 0x7f52e8071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.042+0000 7f52ed80e640 1 -- 172.21.15.83:0/3801703122 >> v1:172.21.15.167:6789/0 conn(0x7f52e815bac0 legacy=0x7f52e80cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:12.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.042+0000 7f52ed80e640 1 -- 172.21.15.83:0/3801703122 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e8070fd0 con 0x7f52e8070470 2024-04-17T10:42:12.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.046+0000 7f52ed80e640 1 --2- 172.21.15.83:0/3801703122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52e8070470 0x7f52e806ae10 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f52e80cd5e0 tx=0x7f52e001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370bf77f091c4379 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:12.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.046+0000 7f52dffff640 1 -- 172.21.15.83:0/3801703122 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e00b0070 con 0x7f52e8070470 2024-04-17T10:42:12.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.046+0000 7f52dffff640 1 -- 172.21.15.83:0/3801703122 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52e00ab600 con 0x7f52e8070470 2024-04-17T10:42:12.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.046+0000 7f52dffff640 1 -- 172.21.15.83:0/3801703122 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e00ab9d0 con 0x7f52e8070470 2024-04-17T10:42:12.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.046+0000 7f52ef271640 1 -- 172.21.15.83:0/3801703122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52e8070470 msgr2=0x7f52e806ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:12.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.046+0000 7f52ef271640 1 --2- 172.21.15.83:0/3801703122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52e8070470 0x7f52e806ae10 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f52e80cd5e0 tx=0x7f52e001c4e0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.046+0000 7f52ef271640 1 -- 172.21.15.83:0/3801703122 shutdown_connections 2024-04-17T10:42:12.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.046+0000 7f52ef271640 1 --2- 172.21.15.83:0/3801703122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52e8070470 0x7f52e806ae10 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.046+0000 7f52ef271640 1 --2- 172.21.15.83:0/3801703122 >> v2:172.21.15.167:3300/0 conn(0x7f52e815b6e0 0x7f52e8071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.046+0000 7f52ef271640 1 -- 172.21.15.83:0/3801703122 >> 172.21.15.83:0/3801703122 conn(0x7f52e80b15b0 msgr2=0x7f52e80b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:12.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.046+0000 7f52ef271640 1 -- 172.21.15.83:0/3801703122 shutdown_connections 2024-04-17T10:42:12.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.046+0000 7f52ef271640 1 -- 172.21.15.83:0/3801703122 wait complete. 2024-04-17T10:42:12.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.046+0000 7f52ef271640 1 Processor -- start 2024-04-17T10:42:12.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.046+0000 7f52ef271640 1 -- start start 2024-04-17T10:42:12.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.046+0000 7f52ef271640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52e8070470 0x7f52e81596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:12.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.046+0000 7f52ef271640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52e815b6e0 0x7f52e8159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:12.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.046+0000 7f52ef271640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52e815bac0 0x7f52e81569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:12.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.046+0000 7f52ef271640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e806d2f0 con 0x7f52e815b6e0 2024-04-17T10:42:12.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.046+0000 7f52ef271640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e806d170 con 0x7f52e815bac0 2024-04-17T10:42:12.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.046+0000 7f52ef271640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e806b760 con 0x7f52e8070470 2024-04-17T10:42:12.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.046+0000 7f52ec80c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52e815b6e0 0x7f52e8159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:12.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.046+0000 7f52ec80c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52e815b6e0 0x7f52e8159c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41502/0 (socket says 172.21.15.83:41502) 2024-04-17T10:42:12.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.046+0000 7f52ec80c640 1 -- 172.21.15.83:0/1951459872 learned_addr learned my addr 172.21.15.83:0/1951459872 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:12.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.046+0000 7f52ec80c640 1 -- 172.21.15.83:0/1951459872 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52e8070470 msgr2=0x7f52e81596f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:12.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.050+0000 7f52ec80c640 1 --2- 172.21.15.83:0/1951459872 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52e8070470 0x7f52e81596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.050+0000 7f52ec80c640 1 -- 172.21.15.83:0/1951459872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52e815bac0 msgr2=0x7f52e81569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:12.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.050+0000 7f52ed80e640 1 --2- 172.21.15.83:0/1951459872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52e815bac0 0x7f52e81569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:12.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.050+0000 7f52ec80c640 1 --2- 172.21.15.83:0/1951459872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52e815bac0 0x7f52e81569e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.050+0000 7f52ec80c640 1 -- 172.21.15.83:0/1951459872 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e8157d40 con 0x7f52e815b6e0 2024-04-17T10:42:12.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.050+0000 7f52ec80c640 1 --2- 172.21.15.83:0/1951459872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52e815b6e0 0x7f52e8159c30 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f52d800b410 tx=0x7f52d800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:12.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.050+0000 7f52ddffb640 1 -- 172.21.15.83:0/1951459872 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d8015020 con 0x7f52e815b6e0 2024-04-17T10:42:12.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.050+0000 7f52ddffb640 1 -- 172.21.15.83:0/1951459872 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d8011830 con 0x7f52e815b6e0 2024-04-17T10:42:12.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.050+0000 7f52ef271640 1 -- 172.21.15.83:0/1951459872 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e815a2d0 con 0x7f52e815b6e0 2024-04-17T10:42:12.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.050+0000 7f52ddffb640 1 -- 172.21.15.83:0/1951459872 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d8011c20 con 0x7f52e815b6e0 2024-04-17T10:42:12.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.050+0000 7f52ef271640 1 -- 172.21.15.83:0/1951459872 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e81f5a90 con 0x7f52e815b6e0 2024-04-17T10:42:12.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.050+0000 7f52ddffb640 1 -- 172.21.15.83:0/1951459872 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f52d8018020 con 0x7f52e815b6e0 2024-04-17T10:42:12.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.050+0000 7f52ddffb640 1 --2- 172.21.15.83:0/1951459872 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f52cc066f60 0x7f52cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:12.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.050+0000 7f52ddffb640 1 -- 172.21.15.83:0/1951459872 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f52d80850d0 con 0x7f52e815b6e0 2024-04-17T10:42:12.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.050+0000 7f52ed00d640 1 --2- 172.21.15.83:0/1951459872 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f52cc066f60 0x7f52cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:12.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.050+0000 7f52ed00d640 1 --2- 172.21.15.83:0/1951459872 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f52cc066f60 0x7f52cc069420 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f52e4007e60 tx=0x7f52e4007390 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:12.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.050+0000 7f52f23de000 1 Processor -- start 2024-04-17T10:42:12.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.050+0000 7f52f23de000 1 -- start start 2024-04-17T10:42:12.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52f23de000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562cb0d5f9d0 0x562cb0d5fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:12.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52f23de000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cb0d602f0 0x562cb0d15b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:12.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52f23de000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562cb0d160a0 0x562cb0d16480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:12.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52f23de000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb0b62ff0 con 0x562cb0d5f9d0 2024-04-17T10:42:12.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52f23de000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb0aeb150 con 0x562cb0d602f0 2024-04-17T10:42:12.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52f23de000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x562cb0b5f090 con 0x562cb0d160a0 2024-04-17T10:42:12.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52ed00d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562cb0d5f9d0 0x562cb0d5fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:12.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52ed00d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562cb0d5f9d0 0x562cb0d5fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41518/0 (socket says 172.21.15.83:41518) 2024-04-17T10:42:12.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52ed00d640 1 -- 172.21.15.83:0/1663153657 learned_addr learned my addr 172.21.15.83:0/1663153657 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:12.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52ed00d640 1 -- 172.21.15.83:0/1663153657 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562cb0d160a0 msgr2=0x562cb0d16480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:12.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52ed00d640 1 --2- 172.21.15.83:0/1663153657 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562cb0d160a0 0x562cb0d16480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52ed00d640 1 -- 172.21.15.83:0/1663153657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cb0d602f0 msgr2=0x562cb0d15b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:12.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52ed00d640 1 --2- 172.21.15.83:0/1663153657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cb0d602f0 0x562cb0d15b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52ed00d640 1 -- 172.21.15.83:0/1663153657 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cb0cf90d0 con 0x562cb0d5f9d0 2024-04-17T10:42:12.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52ed00d640 1 --2- 172.21.15.83:0/1663153657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562cb0d5f9d0 0x562cb0d5fdb0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f52e400e580 tx=0x7f52e400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95721ebe98bac367 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:12.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52d2ffd640 1 -- 172.21.15.83:0/1663153657 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e401f070 con 0x562cb0d5f9d0 2024-04-17T10:42:12.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52d2ffd640 1 -- 172.21.15.83:0/1663153657 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52e401a880 con 0x562cb0d5f9d0 2024-04-17T10:42:12.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52d2ffd640 1 -- 172.21.15.83:0/1663153657 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e401ac30 con 0x562cb0d5f9d0 2024-04-17T10:42:12.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52f23de000 1 -- 172.21.15.83:0/1663153657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562cb0d5f9d0 msgr2=0x562cb0d5fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:12.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52f23de000 1 --2- 172.21.15.83:0/1663153657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562cb0d5f9d0 0x562cb0d5fdb0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f52e400e580 tx=0x7f52e400f000 comp rx=0 tx=0).stop 2024-04-17T10:42:12.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52f23de000 1 -- 172.21.15.83:0/1663153657 shutdown_connections 2024-04-17T10:42:12.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52f23de000 1 --2- 172.21.15.83:0/1663153657 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562cb0d160a0 0x562cb0d16480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52f23de000 1 --2- 172.21.15.83:0/1663153657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cb0d602f0 0x562cb0d15b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52f23de000 1 --2- 172.21.15.83:0/1663153657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562cb0d5f9d0 0x562cb0d5fdb0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52f23de000 1 -- 172.21.15.83:0/1663153657 >> 172.21.15.83:0/1663153657 conn(0x562cb0d12020 msgr2=0x562cb0d14440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:12.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52f23de000 1 -- 172.21.15.83:0/1663153657 shutdown_connections 2024-04-17T10:42:12.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52f23de000 1 -- 172.21.15.83:0/1663153657 wait complete. 2024-04-17T10:42:12.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52f23de000 1 -- start start 2024-04-17T10:42:12.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52f23de000 1 Processor -- start 2024-04-17T10:42:12.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52f23de000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562cb0d160a0 0x562cb0d447f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:12.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52f23de000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562cb0d5f9d0 0x562cb0d44d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:12.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52f23de000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cb0d602f0 0x562cb0d45270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:12.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52f23de000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb0b27250 con 0x562cb0d160a0 2024-04-17T10:42:12.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52f23de000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb0b1d690 con 0x562cb0d602f0 2024-04-17T10:42:12.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52f23de000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x562cb0aeba40 con 0x562cb0d5f9d0 2024-04-17T10:42:12.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52f23de000 10 client.0 _ll_register_callbacks cb 0x562cb0d00e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:12.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52ed00d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562cb0d160a0 0x562cb0d447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:12.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52ed00d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562cb0d160a0 0x562cb0d447f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41522/0 (socket says 172.21.15.83:41522) 2024-04-17T10:42:12.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52ed00d640 1 -- 172.21.15.83:0/4089161241 learned_addr learned my addr 172.21.15.83:0/4089161241 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:12.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52ed00d640 1 -- 172.21.15.83:0/4089161241 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562cb0d5f9d0 msgr2=0x562cb0d44d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:12.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52ed00d640 1 --2- 172.21.15.83:0/4089161241 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562cb0d5f9d0 0x562cb0d44d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52ed00d640 1 -- 172.21.15.83:0/4089161241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cb0d602f0 msgr2=0x562cb0d45270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:12.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52ed00d640 1 --2- 172.21.15.83:0/4089161241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cb0d602f0 0x562cb0d45270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52ed00d640 1 -- 172.21.15.83:0/4089161241 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562cb0c101c0 con 0x562cb0d160a0 2024-04-17T10:42:12.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52ed00d640 1 --2- 172.21.15.83:0/4089161241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562cb0d160a0 0x562cb0d447f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f52e400ffd0 tx=0x7f52e4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4d6cf9ab26d69f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:12.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:12.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52c7fff640 1 -- 172.21.15.83:0/4089161241 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e4033060 con 0x562cb0d160a0 2024-04-17T10:42:12.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52c7fff640 1 -- 172.21.15.83:0/4089161241 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f52e401a680 con 0x562cb0d160a0 2024-04-17T10:42:12.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52f23de000 1 -- 172.21.15.83:0/4089161241 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f52e401a880 con 0x562cb0d160a0 2024-04-17T10:42:12.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52c7fff640 1 -- 172.21.15.83:0/4089161241 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52e4039050 con 0x562cb0d160a0 2024-04-17T10:42:12.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52f23de000 10 client.6255 fetch_fsmap learned FSMap version 22 2024-04-17T10:42:12.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.054+0000 7f52f23de000 1 -- 172.21.15.83:0/4089161241 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x562cb0c0c170 con 0x562cb0d160a0 2024-04-17T10:42:12.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.058+0000 7f52c7fff640 1 -- 172.21.15.83:0/4089161241 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52e401a880 con 0x562cb0d160a0 2024-04-17T10:42:12.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.058+0000 7f52c7fff640 1 -- 172.21.15.83:0/4089161241 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f52e403c020 con 0x562cb0d160a0 2024-04-17T10:42:12.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.058+0000 7f52f23de000 10 client.6255 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T10:42:12.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.058+0000 7f52f23de000 10 client.6255 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:42:12.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.058+0000 7f52f23de000 20 client.6255 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:12.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.058+0000 7f52f23de000 1 --2- 172.21.15.83:0/4089161241 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562cb0d6fc80 0x562cb0d72070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:12.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.058+0000 7f52f23de000 4 client.6255 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:42:12.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.058+0000 7f52f23de000 1 -- 172.21.15.83:0/4089161241 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562cb0afec60 con 0x562cb0d6fc80 2024-04-17T10:42:12.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.058+0000 7f52ec80c640 1 --2- 172.21.15.83:0/4089161241 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562cb0d6fc80 0x562cb0d72070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:12.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.058+0000 7f52c7fff640 10 client.6255 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:12.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.058+0000 7f52ec80c640 1 --2- 172.21.15.83:0/4089161241 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562cb0d6fc80 0x562cb0d72070 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54fb91a2cc3fdfe3 server_cookie=f05cc34372bfc4b3 in_seq=0 out_seq=0 2024-04-17T10:42:12.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.058+0000 7f52c7fff640 1 -- 172.21.15.83:0/4089161241 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562cb0afec60 con 0x562cb0d6fc80 2024-04-17T10:42:12.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.058+0000 7f52c7fff640 10 client.6255 handle_command_reply: tid=0 2024-04-17T10:42:12.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.058+0000 7f52f23de000 1 client.6255 shutdown 2024-04-17T10:42:12.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.058+0000 7f52f23de000 1 -- 172.21.15.83:0/4089161241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562cb0d160a0 msgr2=0x562cb0d447f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:12.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.058+0000 7f52f23de000 1 --2- 172.21.15.83:0/4089161241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562cb0d160a0 0x562cb0d447f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f52e400ffd0 tx=0x7f52e4013000 comp rx=0 tx=0).stop 2024-04-17T10:42:12.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.058+0000 7f52f23de000 1 -- 172.21.15.83:0/4089161241 shutdown_connections 2024-04-17T10:42:12.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.058+0000 7f52f23de000 1 --2- 172.21.15.83:0/4089161241 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562cb0d6fc80 0x562cb0d72070 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.058+0000 7f52f23de000 1 --2- 172.21.15.83:0/4089161241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cb0d602f0 0x562cb0d45270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.058+0000 7f52f23de000 1 --2- 172.21.15.83:0/4089161241 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562cb0d5f9d0 0x562cb0d44d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.058+0000 7f52f23de000 1 --2- 172.21.15.83:0/4089161241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562cb0d160a0 0x562cb0d447f0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.058+0000 7f52f23de000 1 -- 172.21.15.83:0/4089161241 >> 172.21.15.83:0/4089161241 conn(0x562cb0d12020 msgr2=0x562cb0d14ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:12.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.058+0000 7f52c7fff640 0 client.6255 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:12.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.058+0000 7f52f23de000 1 -- 172.21.15.83:0/4089161241 shutdown_connections 2024-04-17T10:42:12.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.058+0000 7f52f23de000 1 -- 172.21.15.83:0/4089161241 wait complete. 2024-04-17T10:42:12.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.058+0000 7f52f23de000 20 client.6255 trim_cache size 0 max 16384 2024-04-17T10:42:12.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.070+0000 7f52f23de000 1 Processor -- start 2024-04-17T10:42:12.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.070+0000 7f52f23de000 1 -- start start 2024-04-17T10:42:12.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52f23de000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562cb0d160a0 0x562cb0d6ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:12.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52f23de000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562cb0d5f9d0 0x562cb0d6d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:12.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52f23de000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cb0d602f0 0x562cb0d14410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:12.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52f23de000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e8200e00 con 0x562cb0d5f9d0 2024-04-17T10:42:12.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52f23de000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e81fffc0 con 0x562cb0d602f0 2024-04-17T10:42:12.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52f23de000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e81f7470 con 0x562cb0d160a0 2024-04-17T10:42:12.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52ec80c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562cb0d5f9d0 0x562cb0d6d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:12.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52ec80c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562cb0d5f9d0 0x562cb0d6d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41538/0 (socket says 172.21.15.83:41538) 2024-04-17T10:42:12.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52ec80c640 1 -- 172.21.15.83:0/3605983860 learned_addr learned my addr 172.21.15.83:0/3605983860 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:12.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52ed80e640 1 --2- 172.21.15.83:0/3605983860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cb0d602f0 0x562cb0d14410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:12.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52ec80c640 1 -- 172.21.15.83:0/3605983860 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562cb0d160a0 msgr2=0x562cb0d6ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:12.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52ec80c640 1 --2- 172.21.15.83:0/3605983860 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562cb0d160a0 0x562cb0d6ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52ec80c640 1 -- 172.21.15.83:0/3605983860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cb0d602f0 msgr2=0x562cb0d14410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:12.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52ec80c640 1 --2- 172.21.15.83:0/3605983860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cb0d602f0 0x562cb0d14410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52ec80c640 1 -- 172.21.15.83:0/3605983860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cb0ba9de0 con 0x562cb0d5f9d0 2024-04-17T10:42:12.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52ed80e640 1 --2- 172.21.15.83:0/3605983860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cb0d602f0 0x562cb0d14410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:12.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52ec80c640 1 --2- 172.21.15.83:0/3605983860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562cb0d5f9d0 0x562cb0d6d380 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f52d8055600 tx=0x7f52d8009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a709dfcb56624a35 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:12.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52d27fc640 1 -- 172.21.15.83:0/3605983860 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d8011dc0 con 0x562cb0d5f9d0 2024-04-17T10:42:12.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52d27fc640 1 -- 172.21.15.83:0/3605983860 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d806d030 con 0x562cb0d5f9d0 2024-04-17T10:42:12.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52d27fc640 1 -- 172.21.15.83:0/3605983860 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d8053c00 con 0x562cb0d5f9d0 2024-04-17T10:42:12.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52f23de000 1 -- 172.21.15.83:0/3605983860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562cb0d5f9d0 msgr2=0x562cb0d6d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:12.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52f23de000 1 --2- 172.21.15.83:0/3605983860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562cb0d5f9d0 0x562cb0d6d380 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f52d8055600 tx=0x7f52d8009300 comp rx=0 tx=0).stop 2024-04-17T10:42:12.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52f23de000 1 -- 172.21.15.83:0/3605983860 shutdown_connections 2024-04-17T10:42:12.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52f23de000 1 --2- 172.21.15.83:0/3605983860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cb0d602f0 0x562cb0d14410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52f23de000 1 --2- 172.21.15.83:0/3605983860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562cb0d5f9d0 0x562cb0d6d380 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52f23de000 1 --2- 172.21.15.83:0/3605983860 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562cb0d160a0 0x562cb0d6ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52f23de000 1 -- 172.21.15.83:0/3605983860 >> 172.21.15.83:0/3605983860 conn(0x562cb0d12020 msgr2=0x562cb0d6c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:12.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52f23de000 1 -- 172.21.15.83:0/3605983860 shutdown_connections 2024-04-17T10:42:12.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52f23de000 1 -- 172.21.15.83:0/3605983860 wait complete. 2024-04-17T10:42:12.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52f23de000 1 -- start start 2024-04-17T10:42:12.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52f23de000 1 Processor -- start 2024-04-17T10:42:12.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52f23de000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562cb0d160a0 0x562cb0d71820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:12.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52f23de000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562cb0d5f9d0 0x562cb0d71d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:12.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52f23de000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cb0d602f0 0x562cb0d722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:12.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52f23de000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb0afde00 con 0x562cb0d160a0 2024-04-17T10:42:12.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52f23de000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e81ff9c0 con 0x562cb0d602f0 2024-04-17T10:42:12.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52f23de000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x562cb0b62300 con 0x562cb0d5f9d0 2024-04-17T10:42:12.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52f23de000 10 client.0 _ll_register_callbacks cb 0x562cb0ce1040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:12.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52ed00d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562cb0d160a0 0x562cb0d71820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:12.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52ed00d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562cb0d160a0 0x562cb0d71820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41544/0 (socket says 172.21.15.83:41544) 2024-04-17T10:42:12.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52ed00d640 1 -- 172.21.15.83:0/1738036627 learned_addr learned my addr 172.21.15.83:0/1738036627 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:12.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52ed00d640 1 -- 172.21.15.83:0/1738036627 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562cb0d5f9d0 msgr2=0x562cb0d71d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:12.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52ed00d640 1 --2- 172.21.15.83:0/1738036627 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562cb0d5f9d0 0x562cb0d71d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52ed80e640 1 --2- 172.21.15.83:0/1738036627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cb0d602f0 0x562cb0d722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:12.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52ed00d640 1 -- 172.21.15.83:0/1738036627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cb0d602f0 msgr2=0x562cb0d722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:12.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52ed00d640 1 --2- 172.21.15.83:0/1738036627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cb0d602f0 0x562cb0d722a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52ed00d640 1 -- 172.21.15.83:0/1738036627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562cb0b048e0 con 0x562cb0d160a0 2024-04-17T10:42:12.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52ed00d640 1 --2- 172.21.15.83:0/1738036627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562cb0d160a0 0x562cb0d71820 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f52e40249b0 tx=0x7f52e4004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc58b2fd15791e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:12.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:12.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52c7fff640 1 -- 172.21.15.83:0/1738036627 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e401f040 con 0x562cb0d160a0 2024-04-17T10:42:12.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52c7fff640 1 -- 172.21.15.83:0/1738036627 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f52e402e680 con 0x562cb0d160a0 2024-04-17T10:42:12.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52f23de000 1 -- 172.21.15.83:0/1738036627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f52d806d030 con 0x562cb0d160a0 2024-04-17T10:42:12.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52c7fff640 1 -- 172.21.15.83:0/1738036627 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52e404a050 con 0x562cb0d160a0 2024-04-17T10:42:12.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52f23de000 10 client.6261 fetch_fsmap learned FSMap version 22 2024-04-17T10:42:12.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.074+0000 7f52f23de000 1 -- 172.21.15.83:0/1738036627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x562cb0bbc8f0 con 0x562cb0d160a0 2024-04-17T10:42:12.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52c7fff640 1 -- 172.21.15.83:0/1738036627 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d806d030 con 0x562cb0d160a0 2024-04-17T10:42:12.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52c7fff640 1 -- 172.21.15.83:0/1738036627 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f52e404d020 con 0x562cb0d160a0 2024-04-17T10:42:12.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52f23de000 10 client.6261 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T10:42:12.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52f23de000 10 client.6261 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:42:12.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52f23de000 20 client.6261 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:12.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52f23de000 1 --2- 172.21.15.83:0/1738036627 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562cb0d72c00 0x562cb0dc6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:12.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52f23de000 4 client.6261 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T10:42:12.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52f23de000 1 -- 172.21.15.83:0/1738036627 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562cb0b70530 con 0x562cb0d72c00 2024-04-17T10:42:12.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52ec80c640 1 --2- 172.21.15.83:0/1738036627 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562cb0d72c00 0x562cb0dc6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:12.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52c7fff640 10 client.6261 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:12.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52ec80c640 1 --2- 172.21.15.83:0/1738036627 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562cb0d72c00 0x562cb0dc6f50 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94ede3fcc870c271 server_cookie=63bbfdea0747d230 in_seq=0 out_seq=0 2024-04-17T10:42:12.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52c7fff640 1 -- 172.21.15.83:0/1738036627 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+903 (crc 0 0 0) 0x562cb0b70530 con 0x562cb0d72c00 2024-04-17T10:42:12.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52c7fff640 10 client.6261 handle_command_reply: tid=0 2024-04-17T10:42:12.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52f23de000 1 client.6261 shutdown 2024-04-17T10:42:12.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52f23de000 1 -- 172.21.15.83:0/1738036627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562cb0d160a0 msgr2=0x562cb0d71820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:12.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52f23de000 1 --2- 172.21.15.83:0/1738036627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562cb0d160a0 0x562cb0d71820 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f52e40249b0 tx=0x7f52e4004a50 comp rx=0 tx=0).stop 2024-04-17T10:42:12.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52f23de000 1 -- 172.21.15.83:0/1738036627 shutdown_connections 2024-04-17T10:42:12.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52f23de000 1 --2- 172.21.15.83:0/1738036627 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x562cb0d72c00 0x562cb0dc6f50 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52f23de000 1 --2- 172.21.15.83:0/1738036627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cb0d602f0 0x562cb0d722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52c7fff640 0 client.6261 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:12.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52f23de000 1 --2- 172.21.15.83:0/1738036627 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562cb0d5f9d0 0x562cb0d71d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52f23de000 1 --2- 172.21.15.83:0/1738036627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562cb0d160a0 0x562cb0d71820 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52f23de000 1 -- 172.21.15.83:0/1738036627 >> 172.21.15.83:0/1738036627 conn(0x562cb0d12020 msgr2=0x562cb0d6cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:12.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52f23de000 1 -- 172.21.15.83:0/1738036627 shutdown_connections 2024-04-17T10:42:12.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52f23de000 1 -- 172.21.15.83:0/1738036627 wait complete. 2024-04-17T10:42:12.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52f23de000 20 client.6261 trim_cache size 0 max 16384 2024-04-17T10:42:12.083 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:42:12.084 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T10:42:12.084 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-17T10:42:12.084 INFO:teuthology.orchestra.run.smithi083.stdout: "25fb7779-79d1-45a3-847e-b7d449495e3f": { 2024-04-17T10:42:12.084 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml", 2024-04-17T10:42:12.084 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "25fb7779-79d1-45a3-847e-b7d449495e3f", 2024-04-17T10:42:12.084 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "" 2024-04-17T10:42:12.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:42:12.084 INFO:teuthology.orchestra.run.smithi083.stdout: "2d1e3bb3-41c9-4c4e-97b9-84c4f42ea491": { 2024-04-17T10:42:12.084 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/scrub_test_path/ceph-qa-suite/suites", 2024-04-17T10:42:12.084 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "2d1e3bb3-41c9-4c4e-97b9-84c4f42ea491", 2024-04-17T10:42:12.084 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "" 2024-04-17T10:42:12.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:42:12.084 INFO:teuthology.orchestra.run.smithi083.stdout: "31af5ab9-e3c8-4446-b651-082662ab3d6b": { 2024-04-17T10:42:12.085 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-17T10:42:12.085 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "31af5ab9-e3c8-4446-b651-082662ab3d6b", 2024-04-17T10:42:12.085 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "" 2024-04-17T10:42:12.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:42:12.085 INFO:teuthology.orchestra.run.smithi083.stdout: "643c8418-ed65-4681-8d66-99229480f719": { 2024-04-17T10:42:12.085 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_1", 2024-04-17T10:42:12.085 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "643c8418-ed65-4681-8d66-99229480f719", 2024-04-17T10:42:12.085 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "" 2024-04-17T10:42:12.085 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:42:12.085 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:42:12.085 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:42:12.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52ef271640 1 -- 172.21.15.83:0/1951459872 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f52cc066f60 msgr2=0x7f52cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:12.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52ef271640 1 --2- 172.21.15.83:0/1951459872 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f52cc066f60 0x7f52cc069420 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f52e4007e60 tx=0x7f52e4007390 comp rx=0 tx=0).stop 2024-04-17T10:42:12.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52ef271640 1 -- 172.21.15.83:0/1951459872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52e815b6e0 msgr2=0x7f52e8159c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:12.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52ef271640 1 --2- 172.21.15.83:0/1951459872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52e815b6e0 0x7f52e8159c30 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f52d800b410 tx=0x7f52d800b8d0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52ef271640 1 -- 172.21.15.83:0/1951459872 shutdown_connections 2024-04-17T10:42:12.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52ef271640 1 --2- 172.21.15.83:0/1951459872 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f52cc066f60 0x7f52cc069420 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52ef271640 1 --2- 172.21.15.83:0/1951459872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52e815bac0 0x7f52e81569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52ef271640 1 --2- 172.21.15.83:0/1951459872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52e815b6e0 0x7f52e8159c30 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52ef271640 1 --2- 172.21.15.83:0/1951459872 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52e8070470 0x7f52e81596f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:12.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52ef271640 1 -- 172.21.15.83:0/1951459872 >> 172.21.15.83:0/1951459872 conn(0x7f52e80b15b0 msgr2=0x7f52e80af630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:12.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52ef271640 1 -- 172.21.15.83:0/1951459872 shutdown_connections 2024-04-17T10:42:12.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:12.078+0000 7f52ef271640 1 -- 172.21.15.83:0/1951459872 wait complete. 2024-04-17T10:42:17.092 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-17T10:42:17.092 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-17T10:42:17.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.230+0000 7f14beb2e640 1 Processor -- start 2024-04-17T10:42:17.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.230+0000 7f14beb2e640 1 -- start start 2024-04-17T10:42:17.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.230+0000 7f14beb2e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f14b815b6e0 0x7f14b8071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.230+0000 7f14beb2e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b806c1f0 con 0x7f14b815bac0 2024-04-17T10:42:17.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.230+0000 7f14beb2e640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b8070a10 con 0x7f14b8070470 2024-04-17T10:42:17.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.230+0000 7f14beb2e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b8070cf0 con 0x7f14b815b6e0 2024-04-17T10:42:17.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.230+0000 7f14bc8ca640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f14b815b6e0 0x7f14b8071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.230+0000 7f14b7fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f14b815bac0 0x7f14b80cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41552/0 (socket says 172.21.15.83:41552) 2024-04-17T10:42:17.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.230+0000 7f14b7fff640 1 -- 172.21.15.83:0/765810007 learned_addr learned my addr 172.21.15.83:0/765810007 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:17.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.230+0000 7f14bc8ca640 1 -- 172.21.15.83:0/765810007 >> v1:172.21.15.167:6789/0 conn(0x7f14b8070470 legacy=0x7f14b806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:17.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.230+0000 7f14bc8ca640 1 -- 172.21.15.83:0/765810007 >> v1:172.21.15.83:6789/0 conn(0x7f14b815bac0 legacy=0x7f14b80cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.230+0000 7f14bc8ca640 1 -- 172.21.15.83:0/765810007 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b8070fd0 con 0x7f14b815b6e0 2024-04-17T10:42:17.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.234+0000 7f14bc8ca640 1 --2- 172.21.15.83:0/765810007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b815b6e0 0x7f14b8071cc0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009d90 tx=0x7f14a801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae69a8007b8591bd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:17.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.234+0000 7f14b77fe640 1 -- 172.21.15.83:0/765810007 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a80b2070 con 0x7f14b815b6e0 2024-04-17T10:42:17.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.234+0000 7f14b77fe640 1 -- 172.21.15.83:0/765810007 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a80a57f0 con 0x7f14b815b6e0 2024-04-17T10:42:17.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.234+0000 7f14b77fe640 1 -- 172.21.15.83:0/765810007 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a80a5bc0 con 0x7f14b815b6e0 2024-04-17T10:42:17.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.234+0000 7f14beb2e640 1 -- 172.21.15.83:0/765810007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b815b6e0 msgr2=0x7f14b8071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.234+0000 7f14beb2e640 1 --2- 172.21.15.83:0/765810007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b815b6e0 0x7f14b8071cc0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009d90 tx=0x7f14a801c930 comp rx=0 tx=0).stop 2024-04-17T10:42:17.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.234+0000 7f14beb2e640 1 -- 172.21.15.83:0/765810007 shutdown_connections 2024-04-17T10:42:17.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.234+0000 7f14beb2e640 1 --2- 172.21.15.83:0/765810007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b815b6e0 0x7f14b8071cc0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.234+0000 7f14beb2e640 1 -- 172.21.15.83:0/765810007 >> 172.21.15.83:0/765810007 conn(0x7f14b80b15b0 msgr2=0x7f14b80b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:17.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.234+0000 7f14beb2e640 1 -- 172.21.15.83:0/765810007 shutdown_connections 2024-04-17T10:42:17.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.234+0000 7f14beb2e640 1 -- 172.21.15.83:0/765810007 wait complete. 2024-04-17T10:42:17.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.234+0000 7f14beb2e640 1 Processor -- start 2024-04-17T10:42:17.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.234+0000 7f14beb2e640 1 -- start start 2024-04-17T10:42:17.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.234+0000 7f14beb2e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14b8070470 0x7f14b8145a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.234+0000 7f14beb2e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14b815b6e0 0x7f14b81f4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.234+0000 7f14beb2e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b815bac0 0x7f14b814b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.234+0000 7f14beb2e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b806cee0 con 0x7f14b815bac0 2024-04-17T10:42:17.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.234+0000 7f14beb2e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b806b770 con 0x7f14b8070470 2024-04-17T10:42:17.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.234+0000 7f14beb2e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b806d060 con 0x7f14b815b6e0 2024-04-17T10:42:17.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.234+0000 7f14bd0cb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b815bac0 0x7f14b814b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.238+0000 7f14bd0cb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b815bac0 0x7f14b814b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57292/0 (socket says 172.21.15.83:57292) 2024-04-17T10:42:17.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.238+0000 7f14bd0cb640 1 -- 172.21.15.83:0/695377395 learned_addr learned my addr 172.21.15.83:0/695377395 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:17.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.238+0000 7f14bc8ca640 1 --2- 172.21.15.83:0/695377395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14b8070470 0x7f14b8145a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.238+0000 7f14b7fff640 1 --2- 172.21.15.83:0/695377395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14b815b6e0 0x7f14b81f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.238+0000 7f14bd0cb640 1 -- 172.21.15.83:0/695377395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14b815b6e0 msgr2=0x7f14b81f4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.238+0000 7f14bd0cb640 1 --2- 172.21.15.83:0/695377395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14b815b6e0 0x7f14b81f4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.238+0000 7f14bd0cb640 1 -- 172.21.15.83:0/695377395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14b8070470 msgr2=0x7f14b8145a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.238+0000 7f14bd0cb640 1 --2- 172.21.15.83:0/695377395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14b8070470 0x7f14b8145a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.238+0000 7f14bd0cb640 1 -- 172.21.15.83:0/695377395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b8146d50 con 0x7f14b815bac0 2024-04-17T10:42:17.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.238+0000 7f14b7fff640 1 --2- 172.21.15.83:0/695377395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14b815b6e0 0x7f14b81f4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:17.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.238+0000 7f14bd0cb640 1 --2- 172.21.15.83:0/695377395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b815bac0 0x7f14b814b8c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f14b000c3c0 tx=0x7f14b000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:17.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.238+0000 7f14b57fa640 1 -- 172.21.15.83:0/695377395 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b0011790 con 0x7f14b815bac0 2024-04-17T10:42:17.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.238+0000 7f14b57fa640 1 -- 172.21.15.83:0/695377395 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b0017070 con 0x7f14b815bac0 2024-04-17T10:42:17.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.238+0000 7f14beb2e640 1 -- 172.21.15.83:0/695377395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b8147030 con 0x7f14b815bac0 2024-04-17T10:42:17.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.238+0000 7f14b57fa640 1 -- 172.21.15.83:0/695377395 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b0011b70 con 0x7f14b815bac0 2024-04-17T10:42:17.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.238+0000 7f14beb2e640 1 -- 172.21.15.83:0/695377395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b81475f0 con 0x7f14b815bac0 2024-04-17T10:42:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.238+0000 7f14b57fa640 1 -- 172.21.15.83:0/695377395 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f14b0011d10 con 0x7f14b815bac0 2024-04-17T10:42:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.238+0000 7f14b57fa640 1 --2- 172.21.15.83:0/695377395 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f14a4066f60 0x7f14a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.238+0000 7f14b57fa640 1 -- 172.21.15.83:0/695377395 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f14b0084d70 con 0x7f14b815bac0 2024-04-17T10:42:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.238+0000 7f14bc8ca640 1 --2- 172.21.15.83:0/695377395 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f14a4066f60 0x7f14a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.238+0000 7f14bc8ca640 1 --2- 172.21.15.83:0/695377395 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f14a4066f60 0x7f14a4069420 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f14a80af000 tx=0x7f14a80a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.238+0000 7f14c1c9b000 1 Processor -- start 2024-04-17T10:42:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.238+0000 7f14c1c9b000 1 -- start start 2024-04-17T10:42:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14c1c9b000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599f42cf9d0 0x5599f42cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14c1c9b000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599f42d02f0 0x5599f4285b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14c1c9b000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5599f42860a0 0x5599f4286480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14c1c9b000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599f40d2ff0 con 0x5599f42cf9d0 2024-04-17T10:42:17.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14c1c9b000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599f405b150 con 0x5599f42d02f0 2024-04-17T10:42:17.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14c1c9b000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599f40cf090 con 0x5599f42860a0 2024-04-17T10:42:17.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14bc8ca640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599f42cf9d0 0x5599f42cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14bc8ca640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599f42cf9d0 0x5599f42cfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57306/0 (socket says 172.21.15.83:57306) 2024-04-17T10:42:17.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14bc8ca640 1 -- 172.21.15.83:0/888822554 learned_addr learned my addr 172.21.15.83:0/888822554 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:17.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14b7fff640 1 --2- 172.21.15.83:0/888822554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599f42d02f0 0x5599f4285b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14bd0cb640 1 --2- 172.21.15.83:0/888822554 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5599f42860a0 0x5599f4286480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14bc8ca640 1 -- 172.21.15.83:0/888822554 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5599f42860a0 msgr2=0x5599f4286480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:17.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14bc8ca640 1 --2- 172.21.15.83:0/888822554 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5599f42860a0 0x5599f4286480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14bc8ca640 1 -- 172.21.15.83:0/888822554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599f42d02f0 msgr2=0x5599f4285b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:17.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14bc8ca640 1 --2- 172.21.15.83:0/888822554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599f42d02f0 0x5599f4285b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14bc8ca640 1 -- 172.21.15.83:0/888822554 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599f42690d0 con 0x5599f42cf9d0 2024-04-17T10:42:17.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14bc8ca640 1 --2- 172.21.15.83:0/888822554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599f42cf9d0 0x5599f42cfdb0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009bd0 tx=0x7f14a8004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92a70785dd58b2b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:17.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f149e7fc640 1 -- 172.21.15.83:0/888822554 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a80b2040 con 0x5599f42cf9d0 2024-04-17T10:42:17.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f149e7fc640 1 -- 172.21.15.83:0/888822554 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a80b8710 con 0x5599f42cf9d0 2024-04-17T10:42:17.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f149e7fc640 1 -- 172.21.15.83:0/888822554 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a80b8ac0 con 0x5599f42cf9d0 2024-04-17T10:42:17.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14c1c9b000 1 -- 172.21.15.83:0/888822554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599f42cf9d0 msgr2=0x5599f42cfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14c1c9b000 1 --2- 172.21.15.83:0/888822554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599f42cf9d0 0x5599f42cfdb0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009bd0 tx=0x7f14a8004ac0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14c1c9b000 1 -- 172.21.15.83:0/888822554 shutdown_connections 2024-04-17T10:42:17.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14c1c9b000 1 --2- 172.21.15.83:0/888822554 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5599f42860a0 0x5599f4286480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14c1c9b000 1 --2- 172.21.15.83:0/888822554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599f42d02f0 0x5599f4285b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14c1c9b000 1 --2- 172.21.15.83:0/888822554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599f42cf9d0 0x5599f42cfdb0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14c1c9b000 1 -- 172.21.15.83:0/888822554 >> 172.21.15.83:0/888822554 conn(0x5599f4282020 msgr2=0x5599f4284440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:17.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14c1c9b000 1 -- 172.21.15.83:0/888822554 shutdown_connections 2024-04-17T10:42:17.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14c1c9b000 1 -- 172.21.15.83:0/888822554 wait complete. 2024-04-17T10:42:17.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14c1c9b000 1 -- start start 2024-04-17T10:42:17.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14c1c9b000 1 Processor -- start 2024-04-17T10:42:17.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14c1c9b000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599f42860a0 0x5599f42b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14c1c9b000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599f42cf9d0 0x5599f42b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14c1c9b000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5599f42d02f0 0x5599f42b5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14c1c9b000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599f4097250 con 0x5599f42cf9d0 2024-04-17T10:42:17.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14c1c9b000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599f408d690 con 0x5599f42860a0 2024-04-17T10:42:17.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14c1c9b000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599f405ba40 con 0x5599f42d02f0 2024-04-17T10:42:17.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14c1c9b000 10 client.0 _ll_register_callbacks cb 0x5599f4270e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:17.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14b7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599f42cf9d0 0x5599f42b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14b7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599f42cf9d0 0x5599f42b4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57322/0 (socket says 172.21.15.83:57322) 2024-04-17T10:42:17.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14b7fff640 1 -- 172.21.15.83:0/3830614734 learned_addr learned my addr 172.21.15.83:0/3830614734 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:17.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14bc8ca640 1 --2- 172.21.15.83:0/3830614734 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599f42860a0 0x5599f42b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14bd0cb640 1 --2- 172.21.15.83:0/3830614734 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5599f42d02f0 0x5599f42b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14b7fff640 1 -- 172.21.15.83:0/3830614734 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5599f42d02f0 msgr2=0x5599f42b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:17.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14b7fff640 1 --2- 172.21.15.83:0/3830614734 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5599f42d02f0 0x5599f42b5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14b7fff640 1 -- 172.21.15.83:0/3830614734 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599f42860a0 msgr2=0x5599f42b47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:17.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14b7fff640 1 --2- 172.21.15.83:0/3830614734 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599f42860a0 0x5599f42b47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14b7fff640 1 -- 172.21.15.83:0/3830614734 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599f41801c0 con 0x5599f42cf9d0 2024-04-17T10:42:17.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.242+0000 7f14bd0cb640 1 --2- 172.21.15.83:0/3830614734 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5599f42d02f0 0x5599f42b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:17.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14b7fff640 1 --2- 172.21.15.83:0/3830614734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599f42cf9d0 0x5599f42b4d30 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00edf0 tx=0x7f14ac001420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa5971cf034f626 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:17.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14977fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:17.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14977fe640 1 -- 172.21.15.83:0/3830614734 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac019060 con 0x5599f42cf9d0 2024-04-17T10:42:17.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14977fe640 1 -- 172.21.15.83:0/3830614734 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f14ac014b30 con 0x5599f42cf9d0 2024-04-17T10:42:17.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14c1c9b000 1 -- 172.21.15.83:0/3830614734 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14a80b8710 con 0x5599f42cf9d0 2024-04-17T10:42:17.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14977fe640 1 -- 172.21.15.83:0/3830614734 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14ac014d30 con 0x5599f42cf9d0 2024-04-17T10:42:17.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14c1c9b000 10 client.6276 fetch_fsmap learned FSMap version 22 2024-04-17T10:42:17.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14c1c9b000 1 -- 172.21.15.83:0/3830614734 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x5599f417c170 con 0x5599f42cf9d0 2024-04-17T10:42:17.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14977fe640 1 -- 172.21.15.83:0/3830614734 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a80b8710 con 0x5599f42cf9d0 2024-04-17T10:42:17.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14977fe640 1 -- 172.21.15.83:0/3830614734 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f14ac005860 con 0x5599f42cf9d0 2024-04-17T10:42:17.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14c1c9b000 10 client.6276 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T10:42:17.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14c1c9b000 10 client.6276 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:42:17.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14c1c9b000 20 client.6276 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:17.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14c1c9b000 1 --2- 172.21.15.83:0/3830614734 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x5599f42dfc80 0x5599f42e2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14c1c9b000 4 client.6276 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:42:17.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14c1c9b000 1 -- 172.21.15.83:0/3830614734 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599f406ec60 con 0x5599f42dfc80 2024-04-17T10:42:17.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14bc8ca640 1 --2- 172.21.15.83:0/3830614734 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x5599f42dfc80 0x5599f42e2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14977fe640 10 client.6276 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:17.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14bc8ca640 1 --2- 172.21.15.83:0/3830614734 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x5599f42dfc80 0x5599f42e2070 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=472904a20f3b8c6b server_cookie=5860bf5a8de400e9 in_seq=0 out_seq=0 2024-04-17T10:42:17.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14977fe640 1 -- 172.21.15.83:0/3830614734 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5599f406ec60 con 0x5599f42dfc80 2024-04-17T10:42:17.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14977fe640 10 client.6276 handle_command_reply: tid=0 2024-04-17T10:42:17.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14c1c9b000 1 client.6276 shutdown 2024-04-17T10:42:17.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14c1c9b000 1 -- 172.21.15.83:0/3830614734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599f42cf9d0 msgr2=0x5599f42b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14c1c9b000 1 --2- 172.21.15.83:0/3830614734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599f42cf9d0 0x5599f42b4d30 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00edf0 tx=0x7f14ac001420 comp rx=0 tx=0).stop 2024-04-17T10:42:17.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14c1c9b000 1 -- 172.21.15.83:0/3830614734 shutdown_connections 2024-04-17T10:42:17.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14c1c9b000 1 --2- 172.21.15.83:0/3830614734 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x5599f42dfc80 0x5599f42e2070 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14c1c9b000 1 --2- 172.21.15.83:0/3830614734 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5599f42d02f0 0x5599f42b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14c1c9b000 1 --2- 172.21.15.83:0/3830614734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599f42cf9d0 0x5599f42b4d30 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14977fe640 0 client.6276 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:17.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14c1c9b000 1 --2- 172.21.15.83:0/3830614734 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599f42860a0 0x5599f42b47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14c1c9b000 1 -- 172.21.15.83:0/3830614734 >> 172.21.15.83:0/3830614734 conn(0x5599f4282020 msgr2=0x5599f4284ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:17.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14c1c9b000 1 -- 172.21.15.83:0/3830614734 shutdown_connections 2024-04-17T10:42:17.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14c1c9b000 1 -- 172.21.15.83:0/3830614734 wait complete. 2024-04-17T10:42:17.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.246+0000 7f14c1c9b000 20 client.6276 trim_cache size 0 max 16384 2024-04-17T10:42:17.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.262+0000 7f14c1c9b000 1 Processor -- start 2024-04-17T10:42:17.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.262+0000 7f14c1c9b000 1 -- start start 2024-04-17T10:42:17.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.262+0000 7f14c1c9b000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5599f42860a0 0x5599f432bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.262+0000 7f14c1c9b000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599f42cf9d0 0x5599f432c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.262+0000 7f14c1c9b000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599f42d02f0 0x5599f432c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.262+0000 7f14c1c9b000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b82008e0 con 0x5599f42d02f0 2024-04-17T10:42:17.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.262+0000 7f14c1c9b000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b81ffb40 con 0x5599f42cf9d0 2024-04-17T10:42:17.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.262+0000 7f14c1c9b000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b8149350 con 0x5599f42860a0 2024-04-17T10:42:17.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.262+0000 7f14bd0cb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599f42d02f0 0x5599f432c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.262+0000 7f14bd0cb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599f42d02f0 0x5599f432c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57326/0 (socket says 172.21.15.83:57326) 2024-04-17T10:42:17.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.262+0000 7f14bd0cb640 1 -- 172.21.15.83:0/4294697243 learned_addr learned my addr 172.21.15.83:0/4294697243 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:17.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.262+0000 7f14bd0cb640 1 -- 172.21.15.83:0/4294697243 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5599f42860a0 msgr2=0x5599f432bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:17.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.262+0000 7f14bd0cb640 1 --2- 172.21.15.83:0/4294697243 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5599f42860a0 0x5599f432bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.262+0000 7f14bd0cb640 1 -- 172.21.15.83:0/4294697243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599f42cf9d0 msgr2=0x5599f432c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:17.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.262+0000 7f14bd0cb640 1 --2- 172.21.15.83:0/4294697243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599f42cf9d0 0x5599f432c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.262+0000 7f14bd0cb640 1 -- 172.21.15.83:0/4294697243 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599f4119de0 con 0x5599f42d02f0 2024-04-17T10:42:17.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.262+0000 7f14bd0cb640 1 --2- 172.21.15.83:0/4294697243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599f42d02f0 0x5599f432c960 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f14b00832f0 tx=0x7f14b0055dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d214a2874961fe server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:17.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.262+0000 7f149dffb640 1 -- 172.21.15.83:0/4294697243 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b006d030 con 0x5599f42d02f0 2024-04-17T10:42:17.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.262+0000 7f149dffb640 1 -- 172.21.15.83:0/4294697243 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b0059820 con 0x5599f42d02f0 2024-04-17T10:42:17.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.262+0000 7f149dffb640 1 -- 172.21.15.83:0/4294697243 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b005dde0 con 0x5599f42d02f0 2024-04-17T10:42:17.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.262+0000 7f14c1c9b000 1 -- 172.21.15.83:0/4294697243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599f42d02f0 msgr2=0x5599f432c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.262+0000 7f14c1c9b000 1 --2- 172.21.15.83:0/4294697243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599f42d02f0 0x5599f432c960 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f14b00832f0 tx=0x7f14b0055dd0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.262+0000 7f14c1c9b000 1 -- 172.21.15.83:0/4294697243 shutdown_connections 2024-04-17T10:42:17.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.262+0000 7f14c1c9b000 1 --2- 172.21.15.83:0/4294697243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599f42d02f0 0x5599f432c960 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.262+0000 7f14c1c9b000 1 --2- 172.21.15.83:0/4294697243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599f42cf9d0 0x5599f432c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.262+0000 7f14c1c9b000 1 --2- 172.21.15.83:0/4294697243 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5599f42860a0 0x5599f432bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.262+0000 7f14c1c9b000 1 -- 172.21.15.83:0/4294697243 >> 172.21.15.83:0/4294697243 conn(0x5599f4282020 msgr2=0x5599f42dcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:17.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.262+0000 7f14c1c9b000 1 -- 172.21.15.83:0/4294697243 shutdown_connections 2024-04-17T10:42:17.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.262+0000 7f14c1c9b000 1 -- 172.21.15.83:0/4294697243 wait complete. 2024-04-17T10:42:17.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.262+0000 7f14c1c9b000 1 -- start start 2024-04-17T10:42:17.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.262+0000 7f14c1c9b000 1 Processor -- start 2024-04-17T10:42:17.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14c1c9b000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5599f42860a0 0x5599f432f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14c1c9b000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599f42cf9d0 0x5599f432fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14c1c9b000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599f42d02f0 0x5599f42e1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14c1c9b000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599f406de00 con 0x5599f42cf9d0 2024-04-17T10:42:17.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14c1c9b000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b8149950 con 0x5599f42d02f0 2024-04-17T10:42:17.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14c1c9b000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599f40d2300 con 0x5599f42860a0 2024-04-17T10:42:17.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14c1c9b000 10 client.0 _ll_register_callbacks cb 0x5599f4251040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:17.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14b7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599f42cf9d0 0x5599f432fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14b7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599f42cf9d0 0x5599f432fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57328/0 (socket says 172.21.15.83:57328) 2024-04-17T10:42:17.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14b7fff640 1 -- 172.21.15.83:0/3780258302 learned_addr learned my addr 172.21.15.83:0/3780258302 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:17.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14bd0cb640 1 --2- 172.21.15.83:0/3780258302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599f42d02f0 0x5599f42e1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14bc8ca640 1 --2- 172.21.15.83:0/3780258302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5599f42860a0 0x5599f432f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14b7fff640 1 -- 172.21.15.83:0/3780258302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5599f42860a0 msgr2=0x5599f432f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:17.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14b7fff640 1 --2- 172.21.15.83:0/3780258302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5599f42860a0 0x5599f432f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14b7fff640 1 -- 172.21.15.83:0/3780258302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599f42d02f0 msgr2=0x5599f42e1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:17.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14b7fff640 1 --2- 172.21.15.83:0/3780258302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599f42d02f0 0x5599f42e1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14b7fff640 1 -- 172.21.15.83:0/3780258302 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599f40748e0 con 0x5599f42cf9d0 2024-04-17T10:42:17.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14b7fff640 1 --2- 172.21.15.83:0/3780258302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599f42cf9d0 0x5599f432fe70 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f14ac001020 tx=0x7f14ac000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46bcf518240b7018 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:17.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14977fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:17.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14977fe640 1 -- 172.21.15.83:0/3780258302 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac001920 con 0x5599f42cf9d0 2024-04-17T10:42:17.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14977fe640 1 -- 172.21.15.83:0/3780258302 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f14ac005e00 con 0x5599f42cf9d0 2024-04-17T10:42:17.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14c1c9b000 1 -- 172.21.15.83:0/3780258302 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14b0059820 con 0x5599f42cf9d0 2024-04-17T10:42:17.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14977fe640 1 -- 172.21.15.83:0/3780258302 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14ac007160 con 0x5599f42cf9d0 2024-04-17T10:42:17.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14c1c9b000 10 client.6282 fetch_fsmap learned FSMap version 22 2024-04-17T10:42:17.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14c1c9b000 1 -- 172.21.15.83:0/3780258302 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x5599f412c8f0 con 0x5599f42cf9d0 2024-04-17T10:42:17.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14977fe640 1 -- 172.21.15.83:0/3780258302 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b0059820 con 0x5599f42cf9d0 2024-04-17T10:42:17.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14977fe640 1 -- 172.21.15.83:0/3780258302 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f14ac007300 con 0x5599f42cf9d0 2024-04-17T10:42:17.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14c1c9b000 10 client.6282 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T10:42:17.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14c1c9b000 10 client.6282 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:42:17.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14c1c9b000 20 client.6282 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:17.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14c1c9b000 1 --2- 172.21.15.83:0/3780258302 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x5599f42e2d50 0x5599f42e3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14c1c9b000 4 client.6282 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T10:42:17.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14c1c9b000 1 -- 172.21.15.83:0/3780258302 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5599f40e0530 con 0x5599f42e2d50 2024-04-17T10:42:17.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14bc8ca640 1 --2- 172.21.15.83:0/3780258302 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x5599f42e2d50 0x5599f42e3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14977fe640 10 client.6282 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:17.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14bc8ca640 1 --2- 172.21.15.83:0/3780258302 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x5599f42e2d50 0x5599f42e3160 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7e3ff34b38362ac server_cookie=e87b3c90c3a133df in_seq=0 out_seq=0 2024-04-17T10:42:17.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14977fe640 1 -- 172.21.15.83:0/3780258302 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5599f40e0530 con 0x5599f42e2d50 2024-04-17T10:42:17.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14977fe640 10 client.6282 handle_command_reply: tid=0 2024-04-17T10:42:17.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14c1c9b000 1 client.6282 shutdown 2024-04-17T10:42:17.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14c1c9b000 1 -- 172.21.15.83:0/3780258302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599f42cf9d0 msgr2=0x5599f432fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14c1c9b000 1 --2- 172.21.15.83:0/3780258302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599f42cf9d0 0x5599f432fe70 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f14ac001020 tx=0x7f14ac000bd0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14c1c9b000 1 -- 172.21.15.83:0/3780258302 shutdown_connections 2024-04-17T10:42:17.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14c1c9b000 1 --2- 172.21.15.83:0/3780258302 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x5599f42e2d50 0x5599f42e3160 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14c1c9b000 1 --2- 172.21.15.83:0/3780258302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599f42d02f0 0x5599f42e1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14977fe640 0 client.6282 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:17.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14c1c9b000 1 --2- 172.21.15.83:0/3780258302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599f42cf9d0 0x5599f432fe70 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14c1c9b000 1 --2- 172.21.15.83:0/3780258302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5599f42860a0 0x5599f432f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14c1c9b000 1 -- 172.21.15.83:0/3780258302 >> 172.21.15.83:0/3780258302 conn(0x5599f4282020 msgr2=0x5599f42ce070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:17.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14c1c9b000 1 -- 172.21.15.83:0/3780258302 shutdown_connections 2024-04-17T10:42:17.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14c1c9b000 1 -- 172.21.15.83:0/3780258302 wait complete. 2024-04-17T10:42:17.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14c1c9b000 20 client.6282 trim_cache size 0 max 16384 2024-04-17T10:42:17.272 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:42:17.272 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-17T10:42:17.273 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-17T10:42:17.273 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:42:17.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14beb2e640 1 -- 172.21.15.83:0/695377395 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f14a4066f60 msgr2=0x7f14a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14beb2e640 1 --2- 172.21.15.83:0/695377395 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f14a4066f60 0x7f14a4069420 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f14a80af000 tx=0x7f14a80a7070 comp rx=0 tx=0).stop 2024-04-17T10:42:17.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14beb2e640 1 -- 172.21.15.83:0/695377395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b815bac0 msgr2=0x7f14b814b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14beb2e640 1 --2- 172.21.15.83:0/695377395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b815bac0 0x7f14b814b8c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f14b000c3c0 tx=0x7f14b000c880 comp rx=0 tx=0).stop 2024-04-17T10:42:17.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14beb2e640 1 -- 172.21.15.83:0/695377395 shutdown_connections 2024-04-17T10:42:17.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14beb2e640 1 --2- 172.21.15.83:0/695377395 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f14a4066f60 0x7f14a4069420 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14beb2e640 1 --2- 172.21.15.83:0/695377395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b815bac0 0x7f14b814b8c0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14beb2e640 1 --2- 172.21.15.83:0/695377395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14b815b6e0 0x7f14b81f4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14beb2e640 1 --2- 172.21.15.83:0/695377395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14b8070470 0x7f14b8145a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14beb2e640 1 -- 172.21.15.83:0/695377395 >> 172.21.15.83:0/695377395 conn(0x7f14b80b15b0 msgr2=0x7f14b80cd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:17.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14beb2e640 1 -- 172.21.15.83:0/695377395 shutdown_connections 2024-04-17T10:42:17.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.266+0000 7f14beb2e640 1 -- 172.21.15.83:0/695377395 wait complete. 2024-04-17T10:42:17.283 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T10:42:17.283 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-17T10:42:17.283 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-17T10:42:17.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.398+0000 7fa8a35f9640 1 Processor -- start 2024-04-17T10:42:17.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.398+0000 7fa8a35f9640 1 -- start start 2024-04-17T10:42:17.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.398+0000 7fa8a35f9640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa89c15b6e0 0x7fa89c071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.398+0000 7fa8a35f9640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa89c15bac0 0x7fa89c0cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.398+0000 7fa8a35f9640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa89c070470 0x7fa89c06ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.398+0000 7fa8a35f9640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa89c06c1f0 con 0x7fa89c15b6e0 2024-04-17T10:42:17.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.398+0000 7fa8a35f9640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa89c0cdd30 con 0x7fa89c070470 2024-04-17T10:42:17.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.398+0000 7fa8a35f9640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa89c070a10 con 0x7fa89c15bac0 2024-04-17T10:42:17.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.398+0000 7fa8a1395640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa89c15b6e0 0x7fa89c071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.398+0000 7fa8a1395640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa89c15b6e0 0x7fa89c071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57340/0 (socket says 172.21.15.83:57340) 2024-04-17T10:42:17.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.398+0000 7fa8a1395640 1 -- 172.21.15.83:0/623635529 learned_addr learned my addr 172.21.15.83:0/623635529 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:17.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.398+0000 7fa8a0b94640 1 --2- 172.21.15.83:0/623635529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa89c15bac0 0x7fa89c0cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.398+0000 7fa8a1b96640 1 --2- 172.21.15.83:0/623635529 >> v2:172.21.15.167:3300/0 conn(0x7fa89c070470 0x7fa89c06ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.398+0000 7fa8a1395640 1 -- 172.21.15.83:0/623635529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa89c15bac0 msgr2=0x7fa89c0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:17.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.398+0000 7fa8a1395640 1 --2- 172.21.15.83:0/623635529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa89c15bac0 0x7fa89c0cd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.398+0000 7fa8a1395640 1 -- 172.21.15.83:0/623635529 >> v2:172.21.15.167:3300/0 conn(0x7fa89c070470 msgr2=0x7fa89c06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:17.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.398+0000 7fa8a1395640 1 --2- 172.21.15.83:0/623635529 >> v2:172.21.15.167:3300/0 conn(0x7fa89c070470 0x7fa89c06ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.398+0000 7fa8a1395640 1 -- 172.21.15.83:0/623635529 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89c070cf0 con 0x7fa89c15b6e0 2024-04-17T10:42:17.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.402+0000 7fa8a1395640 1 --2- 172.21.15.83:0/623635529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa89c15b6e0 0x7fa89c071cc0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fa89c0cd050 tx=0x7fa89801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa93e84d9ea4e3b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:17.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.402+0000 7fa893fff640 1 -- 172.21.15.83:0/623635529 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8980b0070 con 0x7fa89c15b6e0 2024-04-17T10:42:17.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.402+0000 7fa893fff640 1 -- 172.21.15.83:0/623635529 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8980ab600 con 0x7fa89c15b6e0 2024-04-17T10:42:17.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.402+0000 7fa893fff640 1 -- 172.21.15.83:0/623635529 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8980ab9d0 con 0x7fa89c15b6e0 2024-04-17T10:42:17.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.402+0000 7fa8a35f9640 1 -- 172.21.15.83:0/623635529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa89c15b6e0 msgr2=0x7fa89c071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.402+0000 7fa8a35f9640 1 --2- 172.21.15.83:0/623635529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa89c15b6e0 0x7fa89c071cc0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fa89c0cd050 tx=0x7fa89801c520 comp rx=0 tx=0).stop 2024-04-17T10:42:17.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.402+0000 7fa8a35f9640 1 -- 172.21.15.83:0/623635529 shutdown_connections 2024-04-17T10:42:17.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.402+0000 7fa8a35f9640 1 --2- 172.21.15.83:0/623635529 >> v2:172.21.15.167:3300/0 conn(0x7fa89c070470 0x7fa89c06ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.402+0000 7fa8a35f9640 1 --2- 172.21.15.83:0/623635529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa89c15bac0 0x7fa89c0cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.402+0000 7fa8a35f9640 1 --2- 172.21.15.83:0/623635529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa89c15b6e0 0x7fa89c071cc0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.402+0000 7fa8a35f9640 1 -- 172.21.15.83:0/623635529 >> 172.21.15.83:0/623635529 conn(0x7fa89c0b15b0 msgr2=0x7fa89c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:17.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.402+0000 7fa8a35f9640 1 -- 172.21.15.83:0/623635529 shutdown_connections 2024-04-17T10:42:17.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.402+0000 7fa8a35f9640 1 -- 172.21.15.83:0/623635529 wait complete. 2024-04-17T10:42:17.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.402+0000 7fa8a35f9640 1 Processor -- start 2024-04-17T10:42:17.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.402+0000 7fa8a35f9640 1 -- start start 2024-04-17T10:42:17.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.406+0000 7fa8a35f9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa89c070470 0x7fa89c157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.406+0000 7fa8a35f9640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa89c15b6e0 0x7fa89c154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.406+0000 7fa8a35f9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa89c15bac0 0x7fa89c0ab060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.406+0000 7fa8a35f9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89c06d270 con 0x7fa89c070470 2024-04-17T10:42:17.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.406+0000 7fa8a35f9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89c06d0f0 con 0x7fa89c15bac0 2024-04-17T10:42:17.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.406+0000 7fa8a35f9640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa89c06b760 con 0x7fa89c15b6e0 2024-04-17T10:42:17.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.406+0000 7fa8a1395640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa89c070470 0x7fa89c157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.406+0000 7fa8a1395640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa89c070470 0x7fa89c157560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57352/0 (socket says 172.21.15.83:57352) 2024-04-17T10:42:17.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.406+0000 7fa8a1395640 1 -- 172.21.15.83:0/95108579 learned_addr learned my addr 172.21.15.83:0/95108579 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:17.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.406+0000 7fa8a1b96640 1 --2- 172.21.15.83:0/95108579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa89c15bac0 0x7fa89c0ab060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.406+0000 7fa8a0b94640 1 --2- 172.21.15.83:0/95108579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa89c15b6e0 0x7fa89c154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.406+0000 7fa8a1395640 1 -- 172.21.15.83:0/95108579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa89c15b6e0 msgr2=0x7fa89c154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.406+0000 7fa8a1395640 1 --2- 172.21.15.83:0/95108579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa89c15b6e0 0x7fa89c154a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.406+0000 7fa8a1395640 1 -- 172.21.15.83:0/95108579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa89c15bac0 msgr2=0x7fa89c0ab060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.406+0000 7fa8a1395640 1 --2- 172.21.15.83:0/95108579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa89c15bac0 0x7fa89c0ab060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.406+0000 7fa8a1395640 1 -- 172.21.15.83:0/95108579 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89c157aa0 con 0x7fa89c070470 2024-04-17T10:42:17.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.406+0000 7fa8a1395640 1 --2- 172.21.15.83:0/95108579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa89c070470 0x7fa89c157560 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fa8980ae040 tx=0x7fa89801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.406+0000 7fa891ffb640 1 -- 172.21.15.83:0/95108579 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8980b0040 con 0x7fa89c070470 2024-04-17T10:42:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.406+0000 7fa891ffb640 1 -- 172.21.15.83:0/95108579 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8980bc070 con 0x7fa89c070470 2024-04-17T10:42:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.406+0000 7fa8a35f9640 1 -- 172.21.15.83:0/95108579 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa89c0a99a0 con 0x7fa89c070470 2024-04-17T10:42:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.406+0000 7fa891ffb640 1 -- 172.21.15.83:0/95108579 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8980b6880 con 0x7fa89c070470 2024-04-17T10:42:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.406+0000 7fa8a35f9640 1 -- 172.21.15.83:0/95108579 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa89c0a9f60 con 0x7fa89c070470 2024-04-17T10:42:17.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.406+0000 7fa891ffb640 1 -- 172.21.15.83:0/95108579 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa8980b6a20 con 0x7fa89c070470 2024-04-17T10:42:17.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.406+0000 7fa891ffb640 1 --2- 172.21.15.83:0/95108579 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa880066f60 0x7fa880069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.406+0000 7fa8a0b94640 1 --2- 172.21.15.83:0/95108579 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa880066f60 0x7fa880069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.406+0000 7fa891ffb640 1 -- 172.21.15.83:0/95108579 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa89811f520 con 0x7fa89c070470 2024-04-17T10:42:17.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.406+0000 7fa8a0b94640 1 --2- 172.21.15.83:0/95108579 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa880066f60 0x7fa880069420 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa89c1f7a60 tx=0x7fa88c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:17.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.406+0000 7fa8a6766000 1 Processor -- start 2024-04-17T10:42:17.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.406+0000 7fa8a6766000 1 -- start start 2024-04-17T10:42:17.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a6766000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f905a180 0x55d7f905a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a6766000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d7f905aaa0 0x55d7f90100c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a6766000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7f9010600 0x55d7f90109e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a6766000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f8e5df20 con 0x55d7f905a180 2024-04-17T10:42:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a6766000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f8de6000 con 0x55d7f9010600 2024-04-17T10:42:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a6766000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d7f8e59fc0 con 0x55d7f905aaa0 2024-04-17T10:42:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a1395640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f905a180 0x55d7f905a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a1395640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f905a180 0x55d7f905a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57354/0 (socket says 172.21.15.83:57354) 2024-04-17T10:42:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a1395640 1 -- 172.21.15.83:0/4055651881 learned_addr learned my addr 172.21.15.83:0/4055651881 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a0b94640 1 --2- 172.21.15.83:0/4055651881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d7f905aaa0 0x55d7f90100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a1395640 1 -- 172.21.15.83:0/4055651881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d7f905aaa0 msgr2=0x55d7f90100c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a1395640 1 --2- 172.21.15.83:0/4055651881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d7f905aaa0 0x55d7f90100c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a1395640 1 -- 172.21.15.83:0/4055651881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7f9010600 msgr2=0x55d7f90109e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a1395640 1 --2- 172.21.15.83:0/4055651881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7f9010600 0x55d7f90109e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a1395640 1 -- 172.21.15.83:0/4055651881 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7f8e39ba0 con 0x55d7f905a180 2024-04-17T10:42:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a1395640 1 --2- 172.21.15.83:0/4055651881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f905a180 0x55d7f905a560 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fa8980ec1c0 tx=0x7fa898000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c71ccecd501228 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa886ffd640 1 -- 172.21.15.83:0/4055651881 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8980efd60 con 0x55d7f905a180 2024-04-17T10:42:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa886ffd640 1 -- 172.21.15.83:0/4055651881 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8980f3eb0 con 0x55d7f905a180 2024-04-17T10:42:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa886ffd640 1 -- 172.21.15.83:0/4055651881 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8980f8500 con 0x55d7f905a180 2024-04-17T10:42:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a6766000 1 -- 172.21.15.83:0/4055651881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f905a180 msgr2=0x55d7f905a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a6766000 1 --2- 172.21.15.83:0/4055651881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f905a180 0x55d7f905a560 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fa8980ec1c0 tx=0x7fa898000bd0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a6766000 1 -- 172.21.15.83:0/4055651881 shutdown_connections 2024-04-17T10:42:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a6766000 1 --2- 172.21.15.83:0/4055651881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7f9010600 0x55d7f90109e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a6766000 1 --2- 172.21.15.83:0/4055651881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d7f905aaa0 0x55d7f90100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a6766000 1 --2- 172.21.15.83:0/4055651881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f905a180 0x55d7f905a560 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a6766000 1 -- 172.21.15.83:0/4055651881 >> 172.21.15.83:0/4055651881 conn(0x55d7f9016760 msgr2=0x55d7f900e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a6766000 1 -- 172.21.15.83:0/4055651881 shutdown_connections 2024-04-17T10:42:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a6766000 1 -- 172.21.15.83:0/4055651881 wait complete. 2024-04-17T10:42:17.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a6766000 1 -- start start 2024-04-17T10:42:17.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a6766000 1 Processor -- start 2024-04-17T10:42:17.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a6766000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d7f9010600 0x55d7f9015660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a6766000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f905a180 0x55d7f9015ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a6766000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7f905aaa0 0x55d7f9048450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a6766000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f8f1b9c0 con 0x55d7f905a180 2024-04-17T10:42:17.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a6766000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f8e185d0 con 0x55d7f905aaa0 2024-04-17T10:42:17.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a6766000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d7f8e5df20 con 0x55d7f9010600 2024-04-17T10:42:17.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a6766000 10 client.0 _ll_register_callbacks cb 0x55d7f8ffba60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:17.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a0b94640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f905a180 0x55d7f9015ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a0b94640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f905a180 0x55d7f9015ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57370/0 (socket says 172.21.15.83:57370) 2024-04-17T10:42:17.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a0b94640 1 -- 172.21.15.83:0/2644575162 learned_addr learned my addr 172.21.15.83:0/2644575162 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:17.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a1395640 1 --2- 172.21.15.83:0/2644575162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d7f9010600 0x55d7f9015660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a1b96640 1 --2- 172.21.15.83:0/2644575162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7f905aaa0 0x55d7f9048450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a0b94640 1 -- 172.21.15.83:0/2644575162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d7f9010600 msgr2=0x55d7f9015660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:17.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a0b94640 1 --2- 172.21.15.83:0/2644575162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d7f9010600 0x55d7f9015660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a0b94640 1 -- 172.21.15.83:0/2644575162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7f905aaa0 msgr2=0x55d7f9048450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:17.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a0b94640 1 --2- 172.21.15.83:0/2644575162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7f905aaa0 0x55d7f9048450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a0b94640 1 -- 172.21.15.83:0/2644575162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d7f8ea45a0 con 0x55d7f905a180 2024-04-17T10:42:17.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.410+0000 7fa8a1b96640 1 --2- 172.21.15.83:0/2644575162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7f905aaa0 0x55d7f9048450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:17.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa8a0b94640 1 --2- 172.21.15.83:0/2644575162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f905a180 0x55d7f9015ba0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa88c012720 tx=0x7fa88c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc4c3fabf2c34ec server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:17.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa87bfff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:17.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa87bfff640 1 -- 172.21.15.83:0/2644575162 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c025060 con 0x55d7f905a180 2024-04-17T10:42:17.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa87bfff640 1 -- 172.21.15.83:0/2644575162 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa88c004230 con 0x55d7f905a180 2024-04-17T10:42:17.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa8a6766000 1 -- 172.21.15.83:0/2644575162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa8980f3eb0 con 0x55d7f905a180 2024-04-17T10:42:17.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa87bfff640 1 -- 172.21.15.83:0/2644575162 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa88c02b050 con 0x55d7f905a180 2024-04-17T10:42:17.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa8a6766000 10 client.6294 fetch_fsmap learned FSMap version 22 2024-04-17T10:42:17.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa8a6766000 1 -- 172.21.15.83:0/2644575162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55d7f8e352b0 con 0x55d7f905a180 2024-04-17T10:42:17.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa87bfff640 1 -- 172.21.15.83:0/2644575162 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8980f3eb0 con 0x55d7f905a180 2024-04-17T10:42:17.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa87bfff640 1 -- 172.21.15.83:0/2644575162 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa88c02e020 con 0x55d7f905a180 2024-04-17T10:42:17.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa8a6766000 10 client.6294 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T10:42:17.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa8a6766000 10 client.6294 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:42:17.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa8a6766000 20 client.6294 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:17.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa8a6766000 1 --2- 172.21.15.83:0/2644575162 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55d7f9047e60 0x55d7f9048e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa8a6766000 4 client.6294 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:42:17.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa8a6766000 1 -- 172.21.15.83:0/2644575162 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d7f8df9c70 con 0x55d7f9047e60 2024-04-17T10:42:17.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa8a1395640 1 --2- 172.21.15.83:0/2644575162 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55d7f9047e60 0x55d7f9048e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa87bfff640 10 client.6294 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:17.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa8a1395640 1 --2- 172.21.15.83:0/2644575162 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55d7f9047e60 0x55d7f9048e60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7008085a7e9a74d2 server_cookie=dc0f2ab15f3e9e06 in_seq=0 out_seq=0 2024-04-17T10:42:17.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa87bfff640 1 -- 172.21.15.83:0/2644575162 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d7f8df9c70 con 0x55d7f9047e60 2024-04-17T10:42:17.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa87bfff640 10 client.6294 handle_command_reply: tid=0 2024-04-17T10:42:17.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa8a6766000 1 client.6294 shutdown 2024-04-17T10:42:17.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa8a6766000 1 -- 172.21.15.83:0/2644575162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f905a180 msgr2=0x55d7f9015ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa8a6766000 1 --2- 172.21.15.83:0/2644575162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f905a180 0x55d7f9015ba0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa88c012720 tx=0x7fa88c004740 comp rx=0 tx=0).stop 2024-04-17T10:42:17.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa8a6766000 1 -- 172.21.15.83:0/2644575162 shutdown_connections 2024-04-17T10:42:17.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa8a6766000 1 --2- 172.21.15.83:0/2644575162 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55d7f9047e60 0x55d7f9048e60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa8a6766000 1 --2- 172.21.15.83:0/2644575162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7f905aaa0 0x55d7f9048450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa8a6766000 1 --2- 172.21.15.83:0/2644575162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f905a180 0x55d7f9015ba0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa8a6766000 1 --2- 172.21.15.83:0/2644575162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d7f9010600 0x55d7f9015660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa87bfff640 0 client.6294 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:17.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa8a6766000 1 -- 172.21.15.83:0/2644575162 >> 172.21.15.83:0/2644575162 conn(0x55d7f9016760 msgr2=0x55d7f900f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:17.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa8a6766000 1 -- 172.21.15.83:0/2644575162 shutdown_connections 2024-04-17T10:42:17.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa8a6766000 1 -- 172.21.15.83:0/2644575162 wait complete. 2024-04-17T10:42:17.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.414+0000 7fa8a6766000 20 client.6294 trim_cache size 0 max 16384 2024-04-17T10:42:17.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.426+0000 7fa8a6766000 1 Processor -- start 2024-04-17T10:42:17.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.426+0000 7fa8a6766000 1 -- start start 2024-04-17T10:42:17.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.426+0000 7fa8a6766000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f9016760 0x55d7f9070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.426+0000 7fa8a6766000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7f9010600 0x55d7f900f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.426+0000 7fa8a6766000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d7f905a180 0x55d7f900f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.426+0000 7fa8a6766000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89c200d70 con 0x55d7f9016760 2024-04-17T10:42:17.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.426+0000 7fa8a6766000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89c1fff10 con 0x55d7f9010600 2024-04-17T10:42:17.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.426+0000 7fa8a6766000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa89c159090 con 0x55d7f905a180 2024-04-17T10:42:17.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.426+0000 7fa8a1395640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f9016760 0x55d7f9070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.426+0000 7fa8a1395640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f9016760 0x55d7f9070690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57376/0 (socket says 172.21.15.83:57376) 2024-04-17T10:42:17.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.426+0000 7fa8a1395640 1 -- 172.21.15.83:0/38466504 learned_addr learned my addr 172.21.15.83:0/38466504 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:17.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.426+0000 7fa8a1395640 1 -- 172.21.15.83:0/38466504 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d7f905a180 msgr2=0x55d7f900f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:17.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.426+0000 7fa8a1395640 1 --2- 172.21.15.83:0/38466504 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d7f905a180 0x55d7f900f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.426+0000 7fa8a1395640 1 -- 172.21.15.83:0/38466504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7f9010600 msgr2=0x55d7f900f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:17.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.426+0000 7fa8a1395640 1 --2- 172.21.15.83:0/38466504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7f9010600 0x55d7f900f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.426+0000 7fa8a1395640 1 -- 172.21.15.83:0/38466504 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7f90491d0 con 0x55d7f9016760 2024-04-17T10:42:17.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.426+0000 7fa8a1395640 1 --2- 172.21.15.83:0/38466504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f9016760 0x55d7f9070690 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa898000c10 tx=0x7fa898104000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41b7c52e093d30d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:17.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.426+0000 7fa8867fc640 1 -- 172.21.15.83:0/38466504 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8980b3df0 con 0x55d7f9016760 2024-04-17T10:42:17.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.426+0000 7fa8867fc640 1 -- 172.21.15.83:0/38466504 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8980ee060 con 0x55d7f9016760 2024-04-17T10:42:17.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.426+0000 7fa8867fc640 1 -- 172.21.15.83:0/38466504 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8980b6d10 con 0x55d7f9016760 2024-04-17T10:42:17.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.426+0000 7fa8a6766000 1 -- 172.21.15.83:0/38466504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f9016760 msgr2=0x55d7f9070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.426+0000 7fa8a6766000 1 --2- 172.21.15.83:0/38466504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f9016760 0x55d7f9070690 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa898000c10 tx=0x7fa898104000 comp rx=0 tx=0).stop 2024-04-17T10:42:17.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.426+0000 7fa8a6766000 1 -- 172.21.15.83:0/38466504 shutdown_connections 2024-04-17T10:42:17.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.426+0000 7fa8a6766000 1 --2- 172.21.15.83:0/38466504 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d7f905a180 0x55d7f900f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.426+0000 7fa8a6766000 1 --2- 172.21.15.83:0/38466504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7f9010600 0x55d7f900f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.426+0000 7fa8a6766000 1 --2- 172.21.15.83:0/38466504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f9016760 0x55d7f9070690 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.426+0000 7fa8a6766000 1 -- 172.21.15.83:0/38466504 >> 172.21.15.83:0/38466504 conn(0x55d7f90485b0 msgr2=0x55d7f9067770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:17.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.426+0000 7fa8a6766000 1 -- 172.21.15.83:0/38466504 shutdown_connections 2024-04-17T10:42:17.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.426+0000 7fa8a6766000 1 -- 172.21.15.83:0/38466504 wait complete. 2024-04-17T10:42:17.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a6766000 1 -- start start 2024-04-17T10:42:17.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a6766000 1 Processor -- start 2024-04-17T10:42:17.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a6766000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d7f9010600 0x55d7f90b9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a6766000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f9016760 0x55d7f906eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a6766000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7f905a180 0x55d7f906f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a6766000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f8e5df20 con 0x55d7f9016760 2024-04-17T10:42:17.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a6766000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89c159690 con 0x55d7f905a180 2024-04-17T10:42:17.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a6766000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d7f8e04c90 con 0x55d7f9010600 2024-04-17T10:42:17.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a6766000 10 client.0 _ll_register_callbacks cb 0x55d7f8fdc600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:17.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a0b94640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f9016760 0x55d7f906eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a0b94640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f9016760 0x55d7f906eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57388/0 (socket says 172.21.15.83:57388) 2024-04-17T10:42:17.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a0b94640 1 -- 172.21.15.83:0/2943362407 learned_addr learned my addr 172.21.15.83:0/2943362407 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:17.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a0b94640 1 -- 172.21.15.83:0/2943362407 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d7f9010600 msgr2=0x55d7f90b9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:17.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a0b94640 1 --2- 172.21.15.83:0/2943362407 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d7f9010600 0x55d7f90b9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a0b94640 1 -- 172.21.15.83:0/2943362407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7f905a180 msgr2=0x55d7f906f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:17.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a0b94640 1 --2- 172.21.15.83:0/2943362407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7f905a180 0x55d7f906f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a0b94640 1 -- 172.21.15.83:0/2943362407 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d7f8de6250 con 0x55d7f9016760 2024-04-17T10:42:17.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a0b94640 1 --2- 172.21.15.83:0/2943362407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f9016760 0x55d7f906eaf0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00dc30 tx=0x7fa88c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17edd8f220c527cc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:17.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa87bfff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:17.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa87bfff640 1 -- 172.21.15.83:0/2943362407 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c033020 con 0x55d7f9016760 2024-04-17T10:42:17.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa87bfff640 1 -- 172.21.15.83:0/2943362407 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa88c021dd0 con 0x55d7f9016760 2024-04-17T10:42:17.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a6766000 1 -- 172.21.15.83:0/2943362407 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa8980ee060 con 0x55d7f9016760 2024-04-17T10:42:17.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa87bfff640 1 -- 172.21.15.83:0/2943362407 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa88c03d050 con 0x55d7f9016760 2024-04-17T10:42:17.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a6766000 10 client.6300 fetch_fsmap learned FSMap version 22 2024-04-17T10:42:17.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a6766000 1 -- 172.21.15.83:0/2943362407 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55d7f8df2b90 con 0x55d7f9016760 2024-04-17T10:42:17.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa87bfff640 1 -- 172.21.15.83:0/2943362407 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8980ee060 con 0x55d7f9016760 2024-04-17T10:42:17.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa87bfff640 1 -- 172.21.15.83:0/2943362407 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa88c040020 con 0x55d7f9016760 2024-04-17T10:42:17.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a6766000 10 client.6300 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T10:42:17.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a6766000 10 client.6300 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-17T10:42:17.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a6766000 20 client.6300 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:17.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a6766000 1 --2- 172.21.15.83:0/2943362407 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55d7f905aaa0 0x55d7f906f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a6766000 4 client.6300 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T10:42:17.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a6766000 1 -- 172.21.15.83:0/2943362407 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d7f8e6b460 con 0x55d7f905aaa0 2024-04-17T10:42:17.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a1395640 1 --2- 172.21.15.83:0/2943362407 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55d7f905aaa0 0x55d7f906f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa87bfff640 10 client.6300 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:17.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a1395640 1 --2- 172.21.15.83:0/2943362407 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55d7f905aaa0 0x55d7f906f570 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79b1095844f6c227 server_cookie=584a799221be7d03 in_seq=0 out_seq=0 2024-04-17T10:42:17.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa87bfff640 1 -- 172.21.15.83:0/2943362407 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+167 (crc 0 0 0) 0x55d7f8e6b460 con 0x55d7f905aaa0 2024-04-17T10:42:17.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa87bfff640 10 client.6300 handle_command_reply: tid=0 2024-04-17T10:42:17.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a6766000 1 client.6300 shutdown 2024-04-17T10:42:17.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a6766000 1 -- 172.21.15.83:0/2943362407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f9016760 msgr2=0x55d7f906eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a6766000 1 --2- 172.21.15.83:0/2943362407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f9016760 0x55d7f906eaf0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00dc30 tx=0x7fa88c0046d0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a6766000 1 -- 172.21.15.83:0/2943362407 shutdown_connections 2024-04-17T10:42:17.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a6766000 1 --2- 172.21.15.83:0/2943362407 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55d7f905aaa0 0x55d7f906f570 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a6766000 1 --2- 172.21.15.83:0/2943362407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7f905a180 0x55d7f906f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a6766000 1 --2- 172.21.15.83:0/2943362407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f9016760 0x55d7f906eaf0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a6766000 1 --2- 172.21.15.83:0/2943362407 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d7f9010600 0x55d7f90b9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa87bfff640 0 client.6300 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:17.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a6766000 1 -- 172.21.15.83:0/2943362407 >> 172.21.15.83:0/2943362407 conn(0x55d7f90485b0 msgr2=0x55d7f900e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:17.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a6766000 1 -- 172.21.15.83:0/2943362407 shutdown_connections 2024-04-17T10:42:17.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a6766000 1 -- 172.21.15.83:0/2943362407 wait complete. 2024-04-17T10:42:17.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a6766000 20 client.6300 trim_cache size 0 max 16384 2024-04-17T10:42:17.437 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-04-17T10:42:17.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:42:17.438 INFO:teuthology.orchestra.run.smithi083.stdout: "damage_type": "backtrace", 2024-04-17T10:42:17.438 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 410990365, 2024-04-17T10:42:17.438 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 1099511630411, 2024-04-17T10:42:17.438 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_1" 2024-04-17T10:42:17.438 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:42:17.438 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-04-17T10:42:17.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a35f9640 1 -- 172.21.15.83:0/95108579 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa880066f60 msgr2=0x7fa880069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a35f9640 1 --2- 172.21.15.83:0/95108579 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa880066f60 0x7fa880069420 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa89c1f7a60 tx=0x7fa88c008040 comp rx=0 tx=0).stop 2024-04-17T10:42:17.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a35f9640 1 -- 172.21.15.83:0/95108579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa89c070470 msgr2=0x7fa89c157560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a35f9640 1 --2- 172.21.15.83:0/95108579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa89c070470 0x7fa89c157560 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fa8980ae040 tx=0x7fa89801ad60 comp rx=0 tx=0).stop 2024-04-17T10:42:17.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a35f9640 1 -- 172.21.15.83:0/95108579 shutdown_connections 2024-04-17T10:42:17.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a35f9640 1 --2- 172.21.15.83:0/95108579 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa880066f60 0x7fa880069420 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a35f9640 1 --2- 172.21.15.83:0/95108579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa89c15bac0 0x7fa89c0ab060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a35f9640 1 --2- 172.21.15.83:0/95108579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa89c15b6e0 0x7fa89c154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a35f9640 1 --2- 172.21.15.83:0/95108579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa89c070470 0x7fa89c157560 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a35f9640 1 -- 172.21.15.83:0/95108579 >> 172.21.15.83:0/95108579 conn(0x7fa89c0b15b0 msgr2=0x7fa89c0b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:17.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a35f9640 1 -- 172.21.15.83:0/95108579 shutdown_connections 2024-04-17T10:42:17.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.430+0000 7fa8a35f9640 1 -- 172.21.15.83:0/95108579 wait complete. 2024-04-17T10:42:17.446 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:42:17.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.554+0000 7fa1f5321640 1 Processor -- start 2024-04-17T10:42:17.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.558+0000 7fa1f5321640 1 -- start start 2024-04-17T10:42:17.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.558+0000 7fa1f5321640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1f0072330 0x7fa1f00cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.558+0000 7fa1f5321640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa1f00cfa10 0x7fa1f006c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.558+0000 7fa1f5321640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f00cfea0 con 0x7fa1f015ba30 2024-04-17T10:42:17.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.558+0000 7fa1f5321640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1f0071ea0 con 0x7fa1f00cfa10 2024-04-17T10:42:17.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.558+0000 7fa1f5321640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f006e320 con 0x7fa1f0072330 2024-04-17T10:42:17.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.558+0000 7fa1ef7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa1f00cfa10 0x7fa1f006c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.558+0000 7fa1ef7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa1f00cfa10 0x7fa1f006c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57390/0 (socket says 172.21.15.83:57390) 2024-04-17T10:42:17.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.558+0000 7fa1ef7fe640 1 -- 172.21.15.83:0/2178102257 learned_addr learned my addr 172.21.15.83:0/2178102257 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:17.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.558+0000 7fa1ee7fc640 1 --2- 172.21.15.83:0/2178102257 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1f0072330 0x7fa1f00cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.558+0000 7fa1ef7fe640 1 -- 172.21.15.83:0/2178102257 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1f0072330 msgr2=0x7fa1f00cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:17.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.558+0000 7fa1ef7fe640 1 --2- 172.21.15.83:0/2178102257 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1f0072330 0x7fa1f00cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.558+0000 7fa1ef7fe640 1 -- 172.21.15.83:0/2178102257 >> v1:172.21.15.167:6789/0 conn(0x7fa1f015ba30 legacy=0x7fa1f015c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.558+0000 7fa1ef7fe640 1 -- 172.21.15.83:0/2178102257 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f006e040 con 0x7fa1f00cfa10 2024-04-17T10:42:17.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.558+0000 7fa1ef7fe640 1 --2- 172.21.15.83:0/2178102257 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1f00cfa10 0x7fa1f006c490 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8009870 tx=0x7fa1e801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81bd45925b5222d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:17.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.558+0000 7fa1edffb640 1 -- 172.21.15.83:0/2178102257 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e80b1070 con 0x7fa1f00cfa10 2024-04-17T10:42:17.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.558+0000 7fa1edffb640 1 -- 172.21.15.83:0/2178102257 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e80a8050 con 0x7fa1f00cfa10 2024-04-17T10:42:17.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.558+0000 7fa1edffb640 1 -- 172.21.15.83:0/2178102257 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e80ac860 con 0x7fa1f00cfa10 2024-04-17T10:42:17.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.562+0000 7fa1f5321640 1 -- 172.21.15.83:0/2178102257 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1f00cfa10 msgr2=0x7fa1f006c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.562+0000 7fa1f5321640 1 --2- 172.21.15.83:0/2178102257 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1f00cfa10 0x7fa1f006c490 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8009870 tx=0x7fa1e801c4d0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.562+0000 7fa1f5321640 1 -- 172.21.15.83:0/2178102257 shutdown_connections 2024-04-17T10:42:17.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.562+0000 7fa1f5321640 1 --2- 172.21.15.83:0/2178102257 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1f00cfa10 0x7fa1f006c490 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.562+0000 7fa1f5321640 1 --2- 172.21.15.83:0/2178102257 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1f0072330 0x7fa1f00cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.562+0000 7fa1f5321640 1 -- 172.21.15.83:0/2178102257 >> 172.21.15.83:0/2178102257 conn(0x7fa1f006abe0 msgr2=0x7fa1f006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:17.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.562+0000 7fa1f5321640 1 -- 172.21.15.83:0/2178102257 shutdown_connections 2024-04-17T10:42:17.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.562+0000 7fa1f5321640 1 -- 172.21.15.83:0/2178102257 wait complete. 2024-04-17T10:42:17.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.562+0000 7fa1f5321640 1 Processor -- start 2024-04-17T10:42:17.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.562+0000 7fa1f5321640 1 -- start start 2024-04-17T10:42:17.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.562+0000 7fa1f5321640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1f0072330 0x7fa1f014caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.562+0000 7fa1f5321640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1f00cfa10 0x7fa1f015ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.562+0000 7fa1f5321640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1f015ba30 0x7fa1f00ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.562+0000 7fa1f5321640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f006de20 con 0x7fa1f00cfa10 2024-04-17T10:42:17.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.562+0000 7fa1f5321640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f006dca0 con 0x7fa1f015ba30 2024-04-17T10:42:17.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.562+0000 7fa1f5321640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f0168520 con 0x7fa1f0072330 2024-04-17T10:42:17.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.562+0000 7fa1ee7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1f00cfa10 0x7fa1f015ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.562+0000 7fa1ee7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1f00cfa10 0x7fa1f015ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57394/0 (socket says 172.21.15.83:57394) 2024-04-17T10:42:17.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.562+0000 7fa1ee7fc640 1 -- 172.21.15.83:0/2008816865 learned_addr learned my addr 172.21.15.83:0/2008816865 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:17.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.562+0000 7fa1ef7fe640 1 --2- 172.21.15.83:0/2008816865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1f015ba30 0x7fa1f00ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.562+0000 7fa1eeffd640 1 --2- 172.21.15.83:0/2008816865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1f0072330 0x7fa1f014caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.562+0000 7fa1ee7fc640 1 -- 172.21.15.83:0/2008816865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1f0072330 msgr2=0x7fa1f014caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.562+0000 7fa1ee7fc640 1 --2- 172.21.15.83:0/2008816865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1f0072330 0x7fa1f014caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.562+0000 7fa1ee7fc640 1 -- 172.21.15.83:0/2008816865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1f015ba30 msgr2=0x7fa1f00ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.562+0000 7fa1ee7fc640 1 --2- 172.21.15.83:0/2008816865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1f015ba30 0x7fa1f00ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.562+0000 7fa1ee7fc640 1 -- 172.21.15.83:0/2008816865 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f00b3540 con 0x7fa1f00cfa10 2024-04-17T10:42:17.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.562+0000 7fa1ee7fc640 1 --2- 172.21.15.83:0/2008816865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1f00cfa10 0x7fa1f015ce40 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4007e60 tx=0x7fa1e400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:17.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.562+0000 7fa1d77fe640 1 -- 172.21.15.83:0/2008816865 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4014620 con 0x7fa1f00cfa10 2024-04-17T10:42:17.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.562+0000 7fa1d77fe640 1 -- 172.21.15.83:0/2008816865 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e4019070 con 0x7fa1f00cfa10 2024-04-17T10:42:17.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.562+0000 7fa1f5321640 1 -- 172.21.15.83:0/2008816865 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f00b3820 con 0x7fa1f00cfa10 2024-04-17T10:42:17.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.562+0000 7fa1d77fe640 1 -- 172.21.15.83:0/2008816865 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4014a20 con 0x7fa1f00cfa10 2024-04-17T10:42:17.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.562+0000 7fa1f5321640 1 -- 172.21.15.83:0/2008816865 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f00b3de0 con 0x7fa1f00cfa10 2024-04-17T10:42:17.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.566+0000 7fa1f5321640 1 -- 172.21.15.83:0/2008816865 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1f0074f90 con 0x7fa1f00cfa10 2024-04-17T10:42:17.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.566+0000 7fa1d77fe640 1 -- 172.21.15.83:0/2008816865 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa1e4014bc0 con 0x7fa1f00cfa10 2024-04-17T10:42:17.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.566+0000 7fa1d77fe640 1 --2- 172.21.15.83:0/2008816865 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa1dc066f90 0x7fa1dc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.566+0000 7fa1d77fe640 1 -- 172.21.15.83:0/2008816865 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa1e40858a0 con 0x7fa1f00cfa10 2024-04-17T10:42:17.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.566+0000 7fa1eeffd640 1 --2- 172.21.15.83:0/2008816865 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa1dc066f90 0x7fa1dc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.566+0000 7fa1eeffd640 1 --2- 172.21.15.83:0/2008816865 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa1dc066f90 0x7fa1dc069450 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0007fd0 tx=0x7fa1e0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:17.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.570+0000 7fa1d77fe640 1 -- 172.21.15.83:0/2008816865 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa1e4056870 con 0x7fa1f00cfa10 2024-04-17T10:42:17.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.750+0000 7fa1f5321640 1 -- 172.21.15.83:0/2008816865 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1f0156490 con 0x7fa1f00cfa10 2024-04-17T10:42:17.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.750+0000 7fa1d77fe640 1 -- 172.21.15.83:0/2008816865 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4939 (secure 0 0 0) 0x7fa1e4085020 con 0x7fa1f00cfa10 2024-04-17T10:42:17.751 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:17.751 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/161398126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":161398126},{"type":"v1","addr":"172.21.15.167:6833","nonce":161398126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1085380618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1085380618},{"type":"v1","addr":"172.21.15.167:6835","nonce":1085380618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/836442291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":836442291},{"type":"v1","addr":"172.21.15.83:6835","nonce":836442291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:41:37.773342+0000","modified":"2024-04-17T10:42:06.483516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5682":{"gid":5682,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.83:6837/1894547386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1894547386},{"type":"v1","addr":"172.21.15.83:6837","nonce":1894547386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-17T10:42:17.751 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 22 2024-04-17T10:42:17.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.754+0000 7fa1f5321640 1 -- 172.21.15.83:0/2008816865 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa1dc066f90 msgr2=0x7fa1dc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.754+0000 7fa1f5321640 1 --2- 172.21.15.83:0/2008816865 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa1dc066f90 0x7fa1dc069450 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0007fd0 tx=0x7fa1e0009040 comp rx=0 tx=0).stop 2024-04-17T10:42:17.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.754+0000 7fa1f5321640 1 -- 172.21.15.83:0/2008816865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1f00cfa10 msgr2=0x7fa1f015ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.754+0000 7fa1f5321640 1 --2- 172.21.15.83:0/2008816865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1f00cfa10 0x7fa1f015ce40 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4007e60 tx=0x7fa1e400e4b0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.754+0000 7fa1f5321640 1 -- 172.21.15.83:0/2008816865 shutdown_connections 2024-04-17T10:42:17.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.754+0000 7fa1f5321640 1 --2- 172.21.15.83:0/2008816865 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa1dc066f90 0x7fa1dc069450 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.754+0000 7fa1f5321640 1 --2- 172.21.15.83:0/2008816865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1f015ba30 0x7fa1f00ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.754+0000 7fa1f5321640 1 --2- 172.21.15.83:0/2008816865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1f00cfa10 0x7fa1f015ce40 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.754+0000 7fa1f5321640 1 --2- 172.21.15.83:0/2008816865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1f0072330 0x7fa1f014caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.754+0000 7fa1f5321640 1 -- 172.21.15.83:0/2008816865 >> 172.21.15.83:0/2008816865 conn(0x7fa1f006abe0 msgr2=0x7fa1f00cd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:17.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.754+0000 7fa1f5321640 1 -- 172.21.15.83:0/2008816865 shutdown_connections 2024-04-17T10:42:17.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.754+0000 7fa1f5321640 1 -- 172.21.15.83:0/2008816865 wait complete. 2024-04-17T10:42:17.768 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage rm 410990365 2024-04-17T10:42:17.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.886+0000 7f38e78fc640 1 Processor -- start 2024-04-17T10:42:17.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.886+0000 7f38e78fc640 1 -- start start 2024-04-17T10:42:17.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.890+0000 7f38e78fc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38e015b6e0 0x7f38e0071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.890+0000 7f38e78fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f38e015bac0 0x7f38e00cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.890+0000 7f38e78fc640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e006c1f0 con 0x7f38e0070470 2024-04-17T10:42:17.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.890+0000 7f38e78fc640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38e00cdd30 con 0x7f38e015bac0 2024-04-17T10:42:17.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.890+0000 7f38e78fc640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e0070a10 con 0x7f38e015b6e0 2024-04-17T10:42:17.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.890+0000 7f38e4e97640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f38e015bac0 0x7f38e00cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.890+0000 7f38e4e97640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f38e015bac0 0x7f38e00cd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57396/0 (socket says 172.21.15.83:57396) 2024-04-17T10:42:17.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.890+0000 7f38e4e97640 1 -- 172.21.15.83:0/2057516710 learned_addr learned my addr 172.21.15.83:0/2057516710 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:17.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.890+0000 7f38e5698640 1 --2- 172.21.15.83:0/2057516710 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38e015b6e0 0x7f38e0071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.890+0000 7f38e4e97640 1 -- 172.21.15.83:0/2057516710 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38e015b6e0 msgr2=0x7f38e0071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:17.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.890+0000 7f38e4e97640 1 --2- 172.21.15.83:0/2057516710 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38e015b6e0 0x7f38e0071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.890+0000 7f38e4e97640 1 -- 172.21.15.83:0/2057516710 >> v1:172.21.15.167:6789/0 conn(0x7f38e0070470 legacy=0x7f38e006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.890+0000 7f38e4e97640 1 -- 172.21.15.83:0/2057516710 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e0070cf0 con 0x7f38e015bac0 2024-04-17T10:42:17.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.890+0000 7f38e5698640 1 --2- 172.21.15.83:0/2057516710 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38e015b6e0 0x7f38e0071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:17.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.890+0000 7f38e4e97640 1 --2- 172.21.15.83:0/2057516710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e015bac0 0x7f38e00cd7f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f38d0009870 tx=0x7f38d001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e8d8725211901c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:17.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.890+0000 7f38d7fff640 1 -- 172.21.15.83:0/2057516710 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d00b2070 con 0x7f38e015bac0 2024-04-17T10:42:17.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.890+0000 7f38d7fff640 1 -- 172.21.15.83:0/2057516710 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d00a9070 con 0x7f38e015bac0 2024-04-17T10:42:17.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.890+0000 7f38d7fff640 1 -- 172.21.15.83:0/2057516710 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d00ad880 con 0x7f38e015bac0 2024-04-17T10:42:17.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.894+0000 7f38e78fc640 1 -- 172.21.15.83:0/2057516710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e015bac0 msgr2=0x7f38e00cd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.894+0000 7f38e78fc640 1 --2- 172.21.15.83:0/2057516710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e015bac0 0x7f38e00cd7f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f38d0009870 tx=0x7f38d001c560 comp rx=0 tx=0).stop 2024-04-17T10:42:17.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.894+0000 7f38e78fc640 1 -- 172.21.15.83:0/2057516710 shutdown_connections 2024-04-17T10:42:17.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.894+0000 7f38e78fc640 1 --2- 172.21.15.83:0/2057516710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e015bac0 0x7f38e00cd7f0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.894+0000 7f38e78fc640 1 --2- 172.21.15.83:0/2057516710 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38e015b6e0 0x7f38e0071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.894+0000 7f38e78fc640 1 -- 172.21.15.83:0/2057516710 >> 172.21.15.83:0/2057516710 conn(0x7f38e00b15b0 msgr2=0x7f38e00b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:17.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.894+0000 7f38e78fc640 1 -- 172.21.15.83:0/2057516710 shutdown_connections 2024-04-17T10:42:17.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.894+0000 7f38e78fc640 1 -- 172.21.15.83:0/2057516710 wait complete. 2024-04-17T10:42:17.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.894+0000 7f38e78fc640 1 Processor -- start 2024-04-17T10:42:17.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.894+0000 7f38e78fc640 1 -- start start 2024-04-17T10:42:17.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.894+0000 7f38e78fc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38e0070470 0x7f38e014c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.894+0000 7f38e78fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38e015b6e0 0x7f38e00b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.894+0000 7f38e78fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e015bac0 0x7f38e00b0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.894+0000 7f38e78fc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e006cee0 con 0x7f38e015bac0 2024-04-17T10:42:17.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.894+0000 7f38e78fc640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e006b730 con 0x7f38e015b6e0 2024-04-17T10:42:17.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.894+0000 7f38e78fc640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e006d060 con 0x7f38e0070470 2024-04-17T10:42:17.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.894+0000 7f38e5e99640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e015bac0 0x7f38e00b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.894+0000 7f38e5e99640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e015bac0 0x7f38e00b0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57398/0 (socket says 172.21.15.83:57398) 2024-04-17T10:42:17.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.894+0000 7f38e5e99640 1 -- 172.21.15.83:0/675166236 learned_addr learned my addr 172.21.15.83:0/675166236 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:17.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.894+0000 7f38e5698640 1 --2- 172.21.15.83:0/675166236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38e0070470 0x7f38e014c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.894+0000 7f38e4e97640 1 --2- 172.21.15.83:0/675166236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38e015b6e0 0x7f38e00b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.894+0000 7f38e5e99640 1 -- 172.21.15.83:0/675166236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38e0070470 msgr2=0x7f38e014c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.894+0000 7f38e5e99640 1 --2- 172.21.15.83:0/675166236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38e0070470 0x7f38e014c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.894+0000 7f38e5e99640 1 -- 172.21.15.83:0/675166236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38e015b6e0 msgr2=0x7f38e00b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.894+0000 7f38e5e99640 1 --2- 172.21.15.83:0/675166236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38e015b6e0 0x7f38e00b03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.894+0000 7f38e5e99640 1 -- 172.21.15.83:0/675166236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e014cf40 con 0x7f38e015bac0 2024-04-17T10:42:17.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.894+0000 7f38e5e99640 1 --2- 172.21.15.83:0/675166236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e015bac0 0x7f38e00b0920 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f38d800c730 tx=0x7f38d800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:17.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.894+0000 7f38d5ffb640 1 -- 172.21.15.83:0/675166236 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d8003ba0 con 0x7f38e015bac0 2024-04-17T10:42:17.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.894+0000 7f38d5ffb640 1 -- 172.21.15.83:0/675166236 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d8003d60 con 0x7f38e015bac0 2024-04-17T10:42:17.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.894+0000 7f38e78fc640 1 -- 172.21.15.83:0/675166236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e014d220 con 0x7f38e015bac0 2024-04-17T10:42:17.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.894+0000 7f38d5ffb640 1 -- 172.21.15.83:0/675166236 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d800aa90 con 0x7f38e015bac0 2024-04-17T10:42:17.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.894+0000 7f38e78fc640 1 -- 172.21.15.83:0/675166236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e014d7e0 con 0x7f38e015bac0 2024-04-17T10:42:17.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.898+0000 7f38d5ffb640 1 -- 172.21.15.83:0/675166236 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f38d800ac30 con 0x7f38e015bac0 2024-04-17T10:42:17.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.898+0000 7f38d5ffb640 1 --2- 172.21.15.83:0/675166236 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f38c4066f60 0x7f38c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.898+0000 7f38e5698640 1 --2- 172.21.15.83:0/675166236 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f38c4066f60 0x7f38c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.898+0000 7f38d5ffb640 1 -- 172.21.15.83:0/675166236 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f38d8084fc0 con 0x7f38e015bac0 2024-04-17T10:42:17.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.898+0000 7f38eaa69000 1 Processor -- start 2024-04-17T10:42:17.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.898+0000 7f38eaa69000 1 -- start start 2024-04-17T10:42:17.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.898+0000 7f38e5698640 1 --2- 172.21.15.83:0/675166236 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f38c4066f60 0x7f38c4069420 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f38e00b0fd0 tx=0x7f38dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:17.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.898+0000 7f38eaa69000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e4bc349980 0x55e4bc349d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.898+0000 7f38eaa69000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4bc34a2a0 0x55e4bc300440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.898+0000 7f38eaa69000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e4bc300980 0x55e4bc300d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.898+0000 7f38eaa69000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4bc1e3a00 con 0x55e4bc34a2a0 2024-04-17T10:42:17.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.898+0000 7f38eaa69000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4bc1117d0 con 0x55e4bc349980 2024-04-17T10:42:17.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.898+0000 7f38eaa69000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e4bc149c40 con 0x55e4bc300980 2024-04-17T10:42:17.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.898+0000 7f38e4e97640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4bc34a2a0 0x55e4bc300440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.898+0000 7f38e4e97640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4bc34a2a0 0x55e4bc300440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57410/0 (socket says 172.21.15.83:57410) 2024-04-17T10:42:17.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.898+0000 7f38e4e97640 1 -- 172.21.15.83:0/285578637 learned_addr learned my addr 172.21.15.83:0/285578637 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:17.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.898+0000 7f38e5698640 1 --2- 172.21.15.83:0/285578637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e4bc349980 0x55e4bc349d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.898+0000 7f38e5e99640 1 --2- 172.21.15.83:0/285578637 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e4bc300980 0x55e4bc300d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.898+0000 7f38e4e97640 1 -- 172.21.15.83:0/285578637 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e4bc300980 msgr2=0x55e4bc300d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:17.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.898+0000 7f38e4e97640 1 --2- 172.21.15.83:0/285578637 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e4bc300980 0x55e4bc300d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.898+0000 7f38e4e97640 1 -- 172.21.15.83:0/285578637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e4bc349980 msgr2=0x55e4bc349d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:17.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.898+0000 7f38e4e97640 1 --2- 172.21.15.83:0/285578637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e4bc349980 0x55e4bc349d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.898+0000 7f38e4e97640 1 -- 172.21.15.83:0/285578637 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4bc1247b0 con 0x55e4bc34a2a0 2024-04-17T10:42:17.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.898+0000 7f38e4e97640 1 --2- 172.21.15.83:0/285578637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4bc34a2a0 0x55e4bc300440 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f38d001a190 tx=0x7f38d00a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e96ab61314238c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:17.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.898+0000 7f38caffd640 1 -- 172.21.15.83:0/285578637 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d00b2040 con 0x55e4bc34a2a0 2024-04-17T10:42:17.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.898+0000 7f38caffd640 1 -- 172.21.15.83:0/285578637 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d00bd070 con 0x55e4bc34a2a0 2024-04-17T10:42:17.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.898+0000 7f38caffd640 1 -- 172.21.15.83:0/285578637 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d00adbf0 con 0x55e4bc34a2a0 2024-04-17T10:42:17.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38eaa69000 1 -- 172.21.15.83:0/285578637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4bc34a2a0 msgr2=0x55e4bc300440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38eaa69000 1 --2- 172.21.15.83:0/285578637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4bc34a2a0 0x55e4bc300440 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f38d001a190 tx=0x7f38d00a7000 comp rx=0 tx=0).stop 2024-04-17T10:42:17.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38eaa69000 1 -- 172.21.15.83:0/285578637 shutdown_connections 2024-04-17T10:42:17.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38eaa69000 1 --2- 172.21.15.83:0/285578637 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e4bc300980 0x55e4bc300d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38eaa69000 1 --2- 172.21.15.83:0/285578637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4bc34a2a0 0x55e4bc300440 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38eaa69000 1 --2- 172.21.15.83:0/285578637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e4bc349980 0x55e4bc349d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38eaa69000 1 -- 172.21.15.83:0/285578637 >> 172.21.15.83:0/285578637 conn(0x55e4bc2f8520 msgr2=0x55e4bc2fa910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:17.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38eaa69000 1 -- 172.21.15.83:0/285578637 shutdown_connections 2024-04-17T10:42:17.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38eaa69000 1 -- 172.21.15.83:0/285578637 wait complete. 2024-04-17T10:42:17.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38eaa69000 1 -- start start 2024-04-17T10:42:17.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38eaa69000 1 Processor -- start 2024-04-17T10:42:17.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38eaa69000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e4bc300980 0x55e4bc336d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38eaa69000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e4bc349980 0x55e4bc3372d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38eaa69000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4bc34a2a0 0x55e4bc337810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38eaa69000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4bc145ce0 con 0x55e4bc34a2a0 2024-04-17T10:42:17.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38eaa69000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4bc0d2160 con 0x55e4bc300980 2024-04-17T10:42:17.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38eaa69000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e4bc104a80 con 0x55e4bc349980 2024-04-17T10:42:17.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38eaa69000 10 client.0 _ll_register_callbacks cb 0x55e4bc1fe5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:17.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38e5e99640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4bc34a2a0 0x55e4bc337810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38e5e99640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4bc34a2a0 0x55e4bc337810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57420/0 (socket says 172.21.15.83:57420) 2024-04-17T10:42:17.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38e5e99640 1 -- 172.21.15.83:0/3939744537 learned_addr learned my addr 172.21.15.83:0/3939744537 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:17.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38e4e97640 1 --2- 172.21.15.83:0/3939744537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e4bc349980 0x55e4bc3372d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38e5698640 1 --2- 172.21.15.83:0/3939744537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e4bc300980 0x55e4bc336d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38e5e99640 1 -- 172.21.15.83:0/3939744537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e4bc349980 msgr2=0x55e4bc3372d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:17.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38e5e99640 1 --2- 172.21.15.83:0/3939744537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e4bc349980 0x55e4bc3372d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38e5e99640 1 -- 172.21.15.83:0/3939744537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e4bc300980 msgr2=0x55e4bc336d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:17.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38e5e99640 1 --2- 172.21.15.83:0/3939744537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e4bc300980 0x55e4bc336d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38e5e99640 1 -- 172.21.15.83:0/3939744537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e4bc1907a0 con 0x55e4bc34a2a0 2024-04-17T10:42:17.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38e4e97640 1 --2- 172.21.15.83:0/3939744537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e4bc349980 0x55e4bc3372d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:17.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38e5698640 1 --2- 172.21.15.83:0/3939744537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e4bc300980 0x55e4bc336d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:17.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38e5e99640 1 --2- 172.21.15.83:0/3939744537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4bc34a2a0 0x55e4bc337810 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f38d8051c50 tx=0x7f38d8055fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51872dc9bd82f49e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:17.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38bffff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:17.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38bffff640 1 -- 172.21.15.83:0/3939744537 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d806d020 con 0x55e4bc34a2a0 2024-04-17T10:42:17.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38bffff640 1 -- 172.21.15.83:0/3939744537 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f38d806d1c0 con 0x55e4bc34a2a0 2024-04-17T10:42:17.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38eaa69000 1 -- 172.21.15.83:0/3939744537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38d00bd070 con 0x55e4bc34a2a0 2024-04-17T10:42:17.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38bffff640 1 -- 172.21.15.83:0/3939744537 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38d805df00 con 0x55e4bc34a2a0 2024-04-17T10:42:17.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38eaa69000 10 client.6318 fetch_fsmap learned FSMap version 22 2024-04-17T10:42:17.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38eaa69000 1 -- 172.21.15.83:0/3939744537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55e4bc0d23b0 con 0x55e4bc34a2a0 2024-04-17T10:42:17.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38bffff640 1 -- 172.21.15.83:0/3939744537 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d00bd070 con 0x55e4bc34a2a0 2024-04-17T10:42:17.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38bffff640 1 -- 172.21.15.83:0/3939744537 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f38d805da70 con 0x55e4bc34a2a0 2024-04-17T10:42:17.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38eaa69000 10 client.6318 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T10:42:17.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38eaa69000 10 client.6318 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-17T10:42:17.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38eaa69000 20 client.6318 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:17.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38eaa69000 1 --2- 172.21.15.83:0/3939744537 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55e4bc338220 0x55e4bc35aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38eaa69000 4 client.6318 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:42:17.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38eaa69000 1 -- 172.21.15.83:0/3939744537 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e4bc0e5a40 con 0x55e4bc338220 2024-04-17T10:42:17.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38e5698640 1 --2- 172.21.15.83:0/3939744537 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55e4bc338220 0x55e4bc35aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38bffff640 10 client.6318 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:17.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38e5698640 1 --2- 172.21.15.83:0/3939744537 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55e4bc338220 0x55e4bc35aa20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5414399da7559cde server_cookie=6d54c76bfc8fa19c in_seq=0 out_seq=0 2024-04-17T10:42:17.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38bffff640 1 -- 172.21.15.83:0/3939744537 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e4bc0e5a40 con 0x55e4bc338220 2024-04-17T10:42:17.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38bffff640 10 client.6318 handle_command_reply: tid=0 2024-04-17T10:42:17.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.902+0000 7f38eaa69000 1 client.6318 shutdown 2024-04-17T10:42:17.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.906+0000 7f38eaa69000 1 -- 172.21.15.83:0/3939744537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4bc34a2a0 msgr2=0x55e4bc337810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.906+0000 7f38eaa69000 1 --2- 172.21.15.83:0/3939744537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4bc34a2a0 0x55e4bc337810 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f38d8051c50 tx=0x7f38d8055fb0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.906+0000 7f38eaa69000 1 -- 172.21.15.83:0/3939744537 shutdown_connections 2024-04-17T10:42:17.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.906+0000 7f38eaa69000 1 --2- 172.21.15.83:0/3939744537 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55e4bc338220 0x55e4bc35aa20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.906+0000 7f38eaa69000 1 --2- 172.21.15.83:0/3939744537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4bc34a2a0 0x55e4bc337810 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.906+0000 7f38bffff640 0 client.6318 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:17.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.906+0000 7f38eaa69000 1 --2- 172.21.15.83:0/3939744537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e4bc349980 0x55e4bc3372d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.906+0000 7f38eaa69000 1 --2- 172.21.15.83:0/3939744537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e4bc300980 0x55e4bc336d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.906+0000 7f38eaa69000 1 -- 172.21.15.83:0/3939744537 >> 172.21.15.83:0/3939744537 conn(0x55e4bc2f8520 msgr2=0x55e4bc301800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:17.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.906+0000 7f38eaa69000 1 -- 172.21.15.83:0/3939744537 shutdown_connections 2024-04-17T10:42:17.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.906+0000 7f38eaa69000 1 -- 172.21.15.83:0/3939744537 wait complete. 2024-04-17T10:42:17.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.906+0000 7f38eaa69000 20 client.6318 trim_cache size 0 max 16384 2024-04-17T10:42:17.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.914+0000 7f38eaa69000 1 Processor -- start 2024-04-17T10:42:17.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.914+0000 7f38eaa69000 1 -- start start 2024-04-17T10:42:17.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38eaa69000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e4bc300980 0x55e4bc35ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38eaa69000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e4bc349980 0x55e4bc35bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38eaa69000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4bc34a2a0 0x55e4bc35c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38eaa69000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4bc1cf320 con 0x55e4bc34a2a0 2024-04-17T10:42:17.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38eaa69000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4bc10b630 con 0x55e4bc300980 2024-04-17T10:42:17.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38eaa69000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e4bc207160 con 0x55e4bc349980 2024-04-17T10:42:17.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38e5e99640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4bc34a2a0 0x55e4bc35c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38e5e99640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4bc34a2a0 0x55e4bc35c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57428/0 (socket says 172.21.15.83:57428) 2024-04-17T10:42:17.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38e5e99640 1 -- 172.21.15.83:0/335698631 learned_addr learned my addr 172.21.15.83:0/335698631 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:17.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38e5e99640 1 -- 172.21.15.83:0/335698631 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e4bc349980 msgr2=0x55e4bc35bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:17.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38e5e99640 1 --2- 172.21.15.83:0/335698631 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e4bc349980 0x55e4bc35bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38e5e99640 1 -- 172.21.15.83:0/335698631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e4bc300980 msgr2=0x55e4bc35ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:17.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38e5e99640 1 --2- 172.21.15.83:0/335698631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e4bc300980 0x55e4bc35ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38e5e99640 1 -- 172.21.15.83:0/335698631 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4bc34a0d0 con 0x55e4bc34a2a0 2024-04-17T10:42:17.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38e5e99640 1 --2- 172.21.15.83:0/335698631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4bc34a2a0 0x55e4bc35c4b0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f38d80554f0 tx=0x7f38d8058000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99441adb27707eb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:17.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38ca7fc640 1 -- 172.21.15.83:0/335698631 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d806d040 con 0x55e4bc34a2a0 2024-04-17T10:42:17.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38ca7fc640 1 -- 172.21.15.83:0/335698631 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d806d1e0 con 0x55e4bc34a2a0 2024-04-17T10:42:17.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38ca7fc640 1 -- 172.21.15.83:0/335698631 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d8051a90 con 0x55e4bc34a2a0 2024-04-17T10:42:17.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38eaa69000 1 -- 172.21.15.83:0/335698631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4bc34a2a0 msgr2=0x55e4bc35c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38eaa69000 1 --2- 172.21.15.83:0/335698631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4bc34a2a0 0x55e4bc35c4b0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f38d80554f0 tx=0x7f38d8058000 comp rx=0 tx=0).stop 2024-04-17T10:42:17.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38eaa69000 1 -- 172.21.15.83:0/335698631 shutdown_connections 2024-04-17T10:42:17.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38eaa69000 1 --2- 172.21.15.83:0/335698631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4bc34a2a0 0x55e4bc35c4b0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38eaa69000 1 --2- 172.21.15.83:0/335698631 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e4bc349980 0x55e4bc35bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38eaa69000 1 --2- 172.21.15.83:0/335698631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e4bc300980 0x55e4bc35ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38eaa69000 1 -- 172.21.15.83:0/335698631 >> 172.21.15.83:0/335698631 conn(0x55e4bc2f8520 msgr2=0x55e4bc348f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:17.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38eaa69000 1 -- 172.21.15.83:0/335698631 shutdown_connections 2024-04-17T10:42:17.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38eaa69000 1 -- 172.21.15.83:0/335698631 wait complete. 2024-04-17T10:42:17.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38eaa69000 1 -- start start 2024-04-17T10:42:17.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38eaa69000 1 Processor -- start 2024-04-17T10:42:17.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38eaa69000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4bc300980 0x55e4bc302060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38eaa69000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e4bc349980 0x55e4bc3a9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38eaa69000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e4bc34a2a0 0x55e4bc3a9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38eaa69000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4bc1107d0 con 0x55e4bc300980 2024-04-17T10:42:17.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38eaa69000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4bc1137a0 con 0x55e4bc349980 2024-04-17T10:42:17.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38eaa69000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e4bc148f50 con 0x55e4bc34a2a0 2024-04-17T10:42:17.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38eaa69000 10 client.0 _ll_register_callbacks cb 0x7f38d00b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:17.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38e5698640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4bc300980 0x55e4bc302060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38e5698640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4bc300980 0x55e4bc302060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57442/0 (socket says 172.21.15.83:57442) 2024-04-17T10:42:17.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38e5698640 1 -- 172.21.15.83:0/1679709896 learned_addr learned my addr 172.21.15.83:0/1679709896 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:17.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38e5698640 1 -- 172.21.15.83:0/1679709896 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e4bc34a2a0 msgr2=0x55e4bc3a9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:17.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38e5698640 1 --2- 172.21.15.83:0/1679709896 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e4bc34a2a0 0x55e4bc3a9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38e5698640 1 -- 172.21.15.83:0/1679709896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e4bc349980 msgr2=0x55e4bc3a9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:17.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38e5698640 1 --2- 172.21.15.83:0/1679709896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e4bc349980 0x55e4bc3a9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38e5698640 1 -- 172.21.15.83:0/1679709896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e4bc144c30 con 0x55e4bc300980 2024-04-17T10:42:17.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38e5698640 1 --2- 172.21.15.83:0/1679709896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4bc300980 0x55e4bc302060 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00f9a0 tx=0x7f38dc0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd9513681611df1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:17.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38bffff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:17.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38bffff640 1 -- 172.21.15.83:0/1679709896 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc026350 con 0x55e4bc300980 2024-04-17T10:42:17.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38bffff640 1 -- 172.21.15.83:0/1679709896 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f38dc0264f0 con 0x55e4bc300980 2024-04-17T10:42:17.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.918+0000 7f38eaa69000 1 -- 172.21.15.83:0/1679709896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38d806d1e0 con 0x55e4bc300980 2024-04-17T10:42:17.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38bffff640 1 -- 172.21.15.83:0/1679709896 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38dc034050 con 0x55e4bc300980 2024-04-17T10:42:17.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38eaa69000 10 client.6324 fetch_fsmap learned FSMap version 22 2024-04-17T10:42:17.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38eaa69000 1 -- 172.21.15.83:0/1679709896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55e4bc1a3960 con 0x55e4bc300980 2024-04-17T10:42:17.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38bffff640 1 -- 172.21.15.83:0/1679709896 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d806d1e0 con 0x55e4bc300980 2024-04-17T10:42:17.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38bffff640 1 -- 172.21.15.83:0/1679709896 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f38dc037020 con 0x55e4bc300980 2024-04-17T10:42:17.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38eaa69000 10 client.6324 fetch_fsmap finished waiting for FSMap version 22 2024-04-17T10:42:17.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38eaa69000 10 client.6324 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-17T10:42:17.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38eaa69000 20 client.6324 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:17.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38eaa69000 1 --2- 172.21.15.83:0/1679709896 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55e4bc3664b0 0x55e4bc366890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:17.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38eaa69000 4 client.6324 mds_command: new command op to 5682 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 410990365}] 2024-04-17T10:42:17.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38eaa69000 1 -- 172.21.15.83:0/1679709896 --> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] -- command(tid 0: {"prefix": "damage rm", "damage_id": 410990365}) v1 -- 0x55e4bc157180 con 0x55e4bc3664b0 2024-04-17T10:42:17.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38e4e97640 1 --2- 172.21.15.83:0/1679709896 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55e4bc3664b0 0x55e4bc366890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:17.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38e4e97640 1 --2- 172.21.15.83:0/1679709896 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55e4bc3664b0 0x55e4bc366890 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad011ea49503c7b3 server_cookie=75754312f539dd38 in_seq=0 out_seq=0 2024-04-17T10:42:17.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38bffff640 10 client.6324 ms_handle_connect on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:17.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38bffff640 1 -- 172.21.15.83:0/1679709896 <== mds.0 v2:172.21.15.83:6836/1894547386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x55e4bc157180 con 0x55e4bc3664b0 2024-04-17T10:42:17.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38bffff640 10 client.6324 handle_command_reply: tid=0 2024-04-17T10:42:17.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38eaa69000 1 client.6324 shutdown 2024-04-17T10:42:17.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38eaa69000 1 -- 172.21.15.83:0/1679709896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4bc300980 msgr2=0x55e4bc302060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38eaa69000 1 --2- 172.21.15.83:0/1679709896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4bc300980 0x55e4bc302060 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00f9a0 tx=0x7f38dc0046d0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38eaa69000 1 -- 172.21.15.83:0/1679709896 shutdown_connections 2024-04-17T10:42:17.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38eaa69000 1 --2- 172.21.15.83:0/1679709896 >> [v2:172.21.15.83:6836/1894547386,v1:172.21.15.83:6837/1894547386] conn(0x55e4bc3664b0 0x55e4bc366890 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38eaa69000 1 --2- 172.21.15.83:0/1679709896 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e4bc34a2a0 0x55e4bc3a9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38eaa69000 1 --2- 172.21.15.83:0/1679709896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e4bc349980 0x55e4bc3a9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38bffff640 0 client.6324 ms_handle_reset on v2:172.21.15.83:6836/1894547386 2024-04-17T10:42:17.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38eaa69000 1 --2- 172.21.15.83:0/1679709896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4bc300980 0x55e4bc302060 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38eaa69000 1 -- 172.21.15.83:0/1679709896 >> 172.21.15.83:0/1679709896 conn(0x55e4bc2f8520 msgr2=0x55e4bc2fa910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:17.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38eaa69000 1 -- 172.21.15.83:0/1679709896 shutdown_connections 2024-04-17T10:42:17.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38eaa69000 1 -- 172.21.15.83:0/1679709896 wait complete. 2024-04-17T10:42:17.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38eaa69000 20 client.6324 trim_cache size 0 max 16384 2024-04-17T10:42:17.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38e78fc640 1 -- 172.21.15.83:0/675166236 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f38c4066f60 msgr2=0x7f38c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38e78fc640 1 --2- 172.21.15.83:0/675166236 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f38c4066f60 0x7f38c4069420 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f38e00b0fd0 tx=0x7f38dc008040 comp rx=0 tx=0).stop 2024-04-17T10:42:17.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38e78fc640 1 -- 172.21.15.83:0/675166236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e015bac0 msgr2=0x7f38e00b0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:17.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38e78fc640 1 --2- 172.21.15.83:0/675166236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e015bac0 0x7f38e00b0920 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f38d800c730 tx=0x7f38d800cbf0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38e78fc640 1 -- 172.21.15.83:0/675166236 shutdown_connections 2024-04-17T10:42:17.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38e78fc640 1 --2- 172.21.15.83:0/675166236 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f38c4066f60 0x7f38c4069420 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38e78fc640 1 --2- 172.21.15.83:0/675166236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e015bac0 0x7f38e00b0920 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38e78fc640 1 --2- 172.21.15.83:0/675166236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38e015b6e0 0x7f38e00b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38e78fc640 1 --2- 172.21.15.83:0/675166236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38e0070470 0x7f38e014c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:17.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38e78fc640 1 -- 172.21.15.83:0/675166236 >> 172.21.15.83:0/675166236 conn(0x7f38e00b15b0 msgr2=0x7f38e00b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:17.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38e78fc640 1 -- 172.21.15.83:0/675166236 shutdown_connections 2024-04-17T10:42:17.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:17.922+0000 7f38e78fc640 1 -- 172.21.15.83:0/675166236 wait complete. 2024-04-17T10:42:17.935 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-17T10:42:17.935 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:42:18.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.038+0000 7f4e2bdb3640 1 Processor -- start 2024-04-17T10:42:18.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.038+0000 7f4e2bdb3640 1 -- start start 2024-04-17T10:42:18.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.042+0000 7f4e2bdb3640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e2415ba30 0x7f4e24072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:18.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.042+0000 7f4e2bdb3640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4e2415c960 0x7f4e240cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:18.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.042+0000 7f4e2bdb3640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e240cfea0 con 0x7f4e240cfa10 2024-04-17T10:42:18.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.042+0000 7f4e2bdb3640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e24071ea0 con 0x7f4e2415c960 2024-04-17T10:42:18.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.042+0000 7f4e2bdb3640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2406e320 con 0x7f4e2415ba30 2024-04-17T10:42:18.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.042+0000 7f4e2934e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4e2415c960 0x7f4e240cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:18.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.042+0000 7f4e2934e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4e2415c960 0x7f4e240cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57456/0 (socket says 172.21.15.83:57456) 2024-04-17T10:42:18.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.042+0000 7f4e2934e640 1 -- 172.21.15.83:0/1357973292 learned_addr learned my addr 172.21.15.83:0/1357973292 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:18.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.042+0000 7f4e29b4f640 1 --2- 172.21.15.83:0/1357973292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e2415ba30 0x7f4e24072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:18.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.042+0000 7f4e2934e640 1 -- 172.21.15.83:0/1357973292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e2415ba30 msgr2=0x7f4e24072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:18.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.042+0000 7f4e2934e640 1 --2- 172.21.15.83:0/1357973292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e2415ba30 0x7f4e24072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:18.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.042+0000 7f4e2934e640 1 -- 172.21.15.83:0/1357973292 >> v1:172.21.15.167:6789/0 conn(0x7f4e240cfa10 legacy=0x7f4e2406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:18.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.042+0000 7f4e2934e640 1 -- 172.21.15.83:0/1357973292 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e2406e040 con 0x7f4e2415c960 2024-04-17T10:42:18.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.042+0000 7f4e2934e640 1 --2- 172.21.15.83:0/1357973292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e2415c960 0x7f4e240cf4d0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f4e14009870 tx=0x7f4e1401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85d8737273e4d71 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:18.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.042+0000 7f4e28b4d640 1 -- 172.21.15.83:0/1357973292 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e140b1070 con 0x7f4e2415c960 2024-04-17T10:42:18.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.042+0000 7f4e28b4d640 1 -- 172.21.15.83:0/1357973292 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e140ac620 con 0x7f4e2415c960 2024-04-17T10:42:18.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e28b4d640 1 -- 172.21.15.83:0/1357973292 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e140ac9f0 con 0x7f4e2415c960 2024-04-17T10:42:18.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e2bdb3640 1 -- 172.21.15.83:0/1357973292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e2415c960 msgr2=0x7f4e240cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:18.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e2bdb3640 1 --2- 172.21.15.83:0/1357973292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e2415c960 0x7f4e240cf4d0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f4e14009870 tx=0x7f4e1401c4d0 comp rx=0 tx=0).stop 2024-04-17T10:42:18.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e2bdb3640 1 -- 172.21.15.83:0/1357973292 shutdown_connections 2024-04-17T10:42:18.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e2bdb3640 1 --2- 172.21.15.83:0/1357973292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e2415c960 0x7f4e240cf4d0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:18.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e2bdb3640 1 --2- 172.21.15.83:0/1357973292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e2415ba30 0x7f4e24072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:18.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e2bdb3640 1 -- 172.21.15.83:0/1357973292 >> 172.21.15.83:0/1357973292 conn(0x7f4e2406abe0 msgr2=0x7f4e2406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:18.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e2bdb3640 1 -- 172.21.15.83:0/1357973292 shutdown_connections 2024-04-17T10:42:18.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e2bdb3640 1 -- 172.21.15.83:0/1357973292 wait complete. 2024-04-17T10:42:18.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e2bdb3640 1 Processor -- start 2024-04-17T10:42:18.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e2bdb3640 1 -- start start 2024-04-17T10:42:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e2bdb3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e240cfa10 0x7f4e2414cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e2bdb3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e2415ba30 0x7f4e241504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e2bdb3640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e2414d480 0x7f4e24159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e2bdb3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2406dca0 con 0x7f4e240cfa10 2024-04-17T10:42:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e2bdb3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e240aeae0 con 0x7f4e2415ba30 2024-04-17T10:42:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e2bdb3640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2406de20 con 0x7f4e2414d480 2024-04-17T10:42:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e29b4f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e240cfa10 0x7f4e2414cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e29b4f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e240cfa10 0x7f4e2414cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57458/0 (socket says 172.21.15.83:57458) 2024-04-17T10:42:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e29b4f640 1 -- 172.21.15.83:0/2696027059 learned_addr learned my addr 172.21.15.83:0/2696027059 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e2a350640 1 --2- 172.21.15.83:0/2696027059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e2414d480 0x7f4e24159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e2934e640 1 --2- 172.21.15.83:0/2696027059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e2415ba30 0x7f4e241504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e29b4f640 1 -- 172.21.15.83:0/2696027059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e2414d480 msgr2=0x7f4e24159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e29b4f640 1 --2- 172.21.15.83:0/2696027059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e2414d480 0x7f4e24159b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e29b4f640 1 -- 172.21.15.83:0/2696027059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e2415ba30 msgr2=0x7f4e241504d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e29b4f640 1 --2- 172.21.15.83:0/2696027059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e2415ba30 0x7f4e241504d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e29b4f640 1 -- 172.21.15.83:0/2696027059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e24151e70 con 0x7f4e240cfa10 2024-04-17T10:42:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e29b4f640 1 --2- 172.21.15.83:0/2696027059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e240cfa10 0x7f4e2414cd00 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f4e2000d830 tx=0x7f4e2000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e1a7fc640 1 -- 172.21.15.83:0/2696027059 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2000b6c0 con 0x7f4e240cfa10 2024-04-17T10:42:18.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e1a7fc640 1 -- 172.21.15.83:0/2696027059 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e20019070 con 0x7f4e240cfa10 2024-04-17T10:42:18.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e2bdb3640 1 -- 172.21.15.83:0/2696027059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e24150dd0 con 0x7f4e240cfa10 2024-04-17T10:42:18.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e1a7fc640 1 -- 172.21.15.83:0/2696027059 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2000ba60 con 0x7f4e240cfa10 2024-04-17T10:42:18.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.046+0000 7f4e2bdb3640 1 -- 172.21.15.83:0/2696027059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e24151390 con 0x7f4e240cfa10 2024-04-17T10:42:18.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.050+0000 7f4e2bdb3640 1 -- 172.21.15.83:0/2696027059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e24074f90 con 0x7f4e240cfa10 2024-04-17T10:42:18.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.050+0000 7f4e1a7fc640 1 -- 172.21.15.83:0/2696027059 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4e2001e460 con 0x7f4e240cfa10 2024-04-17T10:42:18.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.050+0000 7f4e1a7fc640 1 --2- 172.21.15.83:0/2696027059 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4e08066f60 0x7f4e08069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:18.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.050+0000 7f4e1a7fc640 1 -- 172.21.15.83:0/2696027059 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4e20085740 con 0x7f4e240cfa10 2024-04-17T10:42:18.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.050+0000 7f4e2934e640 1 --2- 172.21.15.83:0/2696027059 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4e08066f60 0x7f4e08069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:18.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.050+0000 7f4e2934e640 1 --2- 172.21.15.83:0/2696027059 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4e08066f60 0x7f4e08069420 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4e140ae000 tx=0x7f4e14008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:18.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.054+0000 7f4e1a7fc640 1 -- 172.21.15.83:0/2696027059 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4e20056710 con 0x7f4e240cfa10 2024-04-17T10:42:18.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.230+0000 7f4e2bdb3640 1 -- 172.21.15.83:0/2696027059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e2406b160 con 0x7f4e240cfa10 2024-04-17T10:42:18.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.234+0000 7f4e1a7fc640 1 -- 172.21.15.83:0/2696027059 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f4e20056010 con 0x7f4e240cfa10 2024-04-17T10:42:18.232 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:18.232 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/161398126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":161398126},{"type":"v1","addr":"172.21.15.167:6833","nonce":161398126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1085380618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1085380618},{"type":"v1","addr":"172.21.15.167:6835","nonce":1085380618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/836442291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":836442291},{"type":"v1","addr":"172.21.15.83:6835","nonce":836442291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:41:37.773342+0000","modified":"2024-04-17T10:42:06.483516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5682":{"gid":5682,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.83:6837/1894547386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1894547386},{"type":"v1","addr":"172.21.15.83:6837","nonce":1894547386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-17T10:42:18.232 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 22 2024-04-17T10:42:18.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.234+0000 7f4e2bdb3640 1 -- 172.21.15.83:0/2696027059 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4e08066f60 msgr2=0x7f4e08069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.234+0000 7f4e2bdb3640 1 --2- 172.21.15.83:0/2696027059 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4e08066f60 0x7f4e08069420 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4e140ae000 tx=0x7f4e14008040 comp rx=0 tx=0).stop 2024-04-17T10:42:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.234+0000 7f4e2bdb3640 1 -- 172.21.15.83:0/2696027059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e240cfa10 msgr2=0x7f4e2414cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.234+0000 7f4e2bdb3640 1 --2- 172.21.15.83:0/2696027059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e240cfa10 0x7f4e2414cd00 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f4e2000d830 tx=0x7f4e2000dcf0 comp rx=0 tx=0).stop 2024-04-17T10:42:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.234+0000 7f4e2bdb3640 1 -- 172.21.15.83:0/2696027059 shutdown_connections 2024-04-17T10:42:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.234+0000 7f4e2bdb3640 1 --2- 172.21.15.83:0/2696027059 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4e08066f60 0x7f4e08069420 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.234+0000 7f4e2bdb3640 1 --2- 172.21.15.83:0/2696027059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e2414d480 0x7f4e24159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.234+0000 7f4e2bdb3640 1 --2- 172.21.15.83:0/2696027059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e2415ba30 0x7f4e241504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.234+0000 7f4e2bdb3640 1 --2- 172.21.15.83:0/2696027059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e240cfa10 0x7f4e2414cd00 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.234+0000 7f4e2bdb3640 1 -- 172.21.15.83:0/2696027059 >> 172.21.15.83:0/2696027059 conn(0x7f4e2406abe0 msgr2=0x7f4e24155520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.234+0000 7f4e2bdb3640 1 -- 172.21.15.83:0/2696027059 shutdown_connections 2024-04-17T10:42:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.234+0000 7f4e2bdb3640 1 -- 172.21.15.83:0/2696027059 wait complete. 2024-04-17T10:42:18.249 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path / 2024-04-17T10:42:18.362 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:42:18.362 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0 2024-04-17T10:42:18.362 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:42:18.375 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-17T10:42:18.375 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T10:42:18.376 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:42:18.432 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:42:18.479 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T10:42:18.480 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:42:18.480 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:42:18.485 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:42:18.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.642+0000 7f85d94fd640 1 Processor -- start 2024-04-17T10:42:18.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.642+0000 7f85d94fd640 1 -- start start 2024-04-17T10:42:18.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.646+0000 7f85d94fd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85d40cf060 0x7f85d40b4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:18.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.646+0000 7f85d94fd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f85d40b3990 0x7f85d4070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:18.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.646+0000 7f85d94fd640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85d40b2bd0 con 0x7f85d40b45a0 2024-04-17T10:42:18.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.646+0000 7f85d94fd640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85d40b2980 con 0x7f85d40b3990 2024-04-17T10:42:18.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.646+0000 7f85d94fd640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d40cf440 con 0x7f85d40cf060 2024-04-17T10:42:18.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.646+0000 7f85d27fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f85d40b3990 0x7f85d4070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:18.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.646+0000 7f85d37fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f85d40b45a0 0x7f85d415d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41564/0 (socket says 172.21.15.83:41564) 2024-04-17T10:42:18.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.646+0000 7f85d37fe640 1 -- 172.21.15.83:0/1159667247 learned_addr learned my addr 172.21.15.83:0/1159667247 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:18.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.646+0000 7f85d27fc640 1 -- 172.21.15.83:0/1159667247 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85d40cf060 msgr2=0x7f85d40b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:18.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.646+0000 7f85d27fc640 1 --2- 172.21.15.83:0/1159667247 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85d40cf060 0x7f85d40b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:18.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.646+0000 7f85d27fc640 1 -- 172.21.15.83:0/1159667247 >> v1:172.21.15.83:6789/0 conn(0x7f85d40b45a0 legacy=0x7f85d415d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:18.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.646+0000 7f85d27fc640 1 -- 172.21.15.83:0/1159667247 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d40b2eb0 con 0x7f85d40b3990 2024-04-17T10:42:18.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.646+0000 7f85d1ffb640 1 -- 172.21.15.83:0/1159667247 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2071113918 0 0) 0x7f85d40b2bd0 con 0x7f85d40b45a0 2024-04-17T10:42:18.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.646+0000 7f85d27fc640 1 --2- 172.21.15.83:0/1159667247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85d40b3990 0x7f85d4070470 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f85c800aea0 tx=0x7f85c801da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e140c38c40cce3a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:18.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.646+0000 7f85d1ffb640 1 -- 172.21.15.83:0/1159667247 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c80a6770 con 0x7f85d40b3990 2024-04-17T10:42:18.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.646+0000 7f85d1ffb640 1 -- 172.21.15.83:0/1159667247 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c80a6910 con 0x7f85d40b3990 2024-04-17T10:42:18.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.646+0000 7f85d1ffb640 1 -- 172.21.15.83:0/1159667247 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c80a6ce0 con 0x7f85d40b3990 2024-04-17T10:42:18.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.650+0000 7f85d94fd640 1 -- 172.21.15.83:0/1159667247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85d40b3990 msgr2=0x7f85d4070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:18.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.650+0000 7f85d94fd640 1 --2- 172.21.15.83:0/1159667247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85d40b3990 0x7f85d4070470 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f85c800aea0 tx=0x7f85c801da40 comp rx=0 tx=0).stop 2024-04-17T10:42:18.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.650+0000 7f85d94fd640 1 -- 172.21.15.83:0/1159667247 shutdown_connections 2024-04-17T10:42:18.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.650+0000 7f85d94fd640 1 --2- 172.21.15.83:0/1159667247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85d40b3990 0x7f85d4070470 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:18.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.650+0000 7f85d94fd640 1 --2- 172.21.15.83:0/1159667247 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85d40cf060 0x7f85d40b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:18.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.650+0000 7f85d94fd640 1 -- 172.21.15.83:0/1159667247 >> 172.21.15.83:0/1159667247 conn(0x7f85d406abe0 msgr2=0x7f85d406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:18.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.650+0000 7f85d94fd640 1 -- 172.21.15.83:0/1159667247 shutdown_connections 2024-04-17T10:42:18.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.650+0000 7f85d94fd640 1 -- 172.21.15.83:0/1159667247 wait complete. 2024-04-17T10:42:18.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.650+0000 7f85d94fd640 1 Processor -- start 2024-04-17T10:42:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.650+0000 7f85d94fd640 1 -- start start 2024-04-17T10:42:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.650+0000 7f85d94fd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85d40b3990 0x7f85d4154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.650+0000 7f85d94fd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85d40b45a0 0x7f85d4152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.650+0000 7f85d94fd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85d40cf060 0x7f85d4157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.650+0000 7f85d94fd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d415e280 con 0x7f85d40cf060 2024-04-17T10:42:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.650+0000 7f85d94fd640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d415e100 con 0x7f85d40b3990 2024-04-17T10:42:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.650+0000 7f85d94fd640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d415e400 con 0x7f85d40b45a0 2024-04-17T10:42:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.650+0000 7f85d37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85d40cf060 0x7f85d4157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.650+0000 7f85d37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85d40cf060 0x7f85d4157190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57482/0 (socket says 172.21.15.83:57482) 2024-04-17T10:42:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.650+0000 7f85d37fe640 1 -- 172.21.15.83:0/493894764 learned_addr learned my addr 172.21.15.83:0/493894764 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.650+0000 7f85d37fe640 1 -- 172.21.15.83:0/493894764 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85d40b45a0 msgr2=0x7f85d4152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.650+0000 7f85d2ffd640 1 --2- 172.21.15.83:0/493894764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85d40b3990 0x7f85d4154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.650+0000 7f85d27fc640 1 --2- 172.21.15.83:0/493894764 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85d40b45a0 0x7f85d4152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.650+0000 7f85d37fe640 1 --2- 172.21.15.83:0/493894764 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85d40b45a0 0x7f85d4152e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.650+0000 7f85d37fe640 1 -- 172.21.15.83:0/493894764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85d40b3990 msgr2=0x7f85d4154530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.650+0000 7f85d37fe640 1 --2- 172.21.15.83:0/493894764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85d40b3990 0x7f85d4154530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.650+0000 7f85d37fe640 1 -- 172.21.15.83:0/493894764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d4153500 con 0x7f85d40cf060 2024-04-17T10:42:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.650+0000 7f85d37fe640 1 --2- 172.21.15.83:0/493894764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85d40cf060 0x7f85d4157190 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f85c400c480 tx=0x7f85c400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.650+0000 7f85bb7fe640 1 -- 172.21.15.83:0/493894764 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c4018070 con 0x7f85d40cf060 2024-04-17T10:42:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.650+0000 7f85bb7fe640 1 -- 172.21.15.83:0/493894764 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c40074e0 con 0x7f85d40cf060 2024-04-17T10:42:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.650+0000 7f85d94fd640 1 -- 172.21.15.83:0/493894764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85d41537e0 con 0x7f85d40cf060 2024-04-17T10:42:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.650+0000 7f85bb7fe640 1 -- 172.21.15.83:0/493894764 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c4004a30 con 0x7f85d40cf060 2024-04-17T10:42:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.650+0000 7f85d94fd640 1 -- 172.21.15.83:0/493894764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85d414f470 con 0x7f85d40cf060 2024-04-17T10:42:18.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.654+0000 7f85d94fd640 1 -- 172.21.15.83:0/493894764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85d4074f90 con 0x7f85d40cf060 2024-04-17T10:42:18.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.654+0000 7f85bb7fe640 1 -- 172.21.15.83:0/493894764 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f85c401b070 con 0x7f85d40cf060 2024-04-17T10:42:18.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.654+0000 7f85bb7fe640 1 --2- 172.21.15.83:0/493894764 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f85c0066f60 0x7f85c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:18.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.654+0000 7f85bb7fe640 1 -- 172.21.15.83:0/493894764 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f85c4084300 con 0x7f85d40cf060 2024-04-17T10:42:18.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.654+0000 7f85d2ffd640 1 --2- 172.21.15.83:0/493894764 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f85c0066f60 0x7f85c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:18.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.654+0000 7f85d2ffd640 1 --2- 172.21.15.83:0/493894764 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f85c0066f60 0x7f85c0069420 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f85cc00a790 tx=0x7f85cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:18.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.658+0000 7f85bb7fe640 1 -- 172.21.15.83:0/493894764 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f85c4055340 con 0x7f85d40cf060 2024-04-17T10:42:18.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.814+0000 7f85d94fd640 1 -- 172.21.15.83:0/493894764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85d4151810 con 0x7f85d40cf060 2024-04-17T10:42:18.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.818+0000 7f85bb7fe640 1 -- 172.21.15.83:0/493894764 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16693 (secure 0 0 0) 0x7f85c4054c40 con 0x7f85d40cf060 2024-04-17T10:42:18.816 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:18.816 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":48,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:41:54.319551+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:41:36.031388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-17T10:41:36.783410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.13","mappings":[{"from":0,"to":4}]},{"pgid":"7.16","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:42:18.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.818+0000 7f85d94fd640 1 -- 172.21.15.83:0/493894764 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f85c0066f60 msgr2=0x7f85c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:18.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.818+0000 7f85d94fd640 1 --2- 172.21.15.83:0/493894764 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f85c0066f60 0x7f85c0069420 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f85cc00a790 tx=0x7f85cc008040 comp rx=0 tx=0).stop 2024-04-17T10:42:18.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.818+0000 7f85d94fd640 1 -- 172.21.15.83:0/493894764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85d40cf060 msgr2=0x7f85d4157190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:18.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.818+0000 7f85d94fd640 1 --2- 172.21.15.83:0/493894764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85d40cf060 0x7f85d4157190 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f85c400c480 tx=0x7f85c400c940 comp rx=0 tx=0).stop 2024-04-17T10:42:18.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.818+0000 7f85d94fd640 1 -- 172.21.15.83:0/493894764 shutdown_connections 2024-04-17T10:42:18.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.818+0000 7f85d94fd640 1 --2- 172.21.15.83:0/493894764 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f85c0066f60 0x7f85c0069420 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:18.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.818+0000 7f85d94fd640 1 --2- 172.21.15.83:0/493894764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85d40cf060 0x7f85d4157190 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:18.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.818+0000 7f85d94fd640 1 --2- 172.21.15.83:0/493894764 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85d40b45a0 0x7f85d4152e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:18.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.818+0000 7f85d94fd640 1 --2- 172.21.15.83:0/493894764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85d40b3990 0x7f85d4154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:18.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.818+0000 7f85d94fd640 1 -- 172.21.15.83:0/493894764 >> 172.21.15.83:0/493894764 conn(0x7f85d406abe0 msgr2=0x7f85d40b1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:18.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.818+0000 7f85d94fd640 1 -- 172.21.15.83:0/493894764 shutdown_connections 2024-04-17T10:42:18.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.818+0000 7f85d94fd640 1 -- 172.21.15.83:0/493894764 wait complete. 2024-04-17T10:42:18.834 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:41:36.031388+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:41:36.783410+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:42:18.835 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:42:18.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.954+0000 7fbde58e0640 1 Processor -- start 2024-04-17T10:42:18.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.954+0000 7fbde58e0640 1 -- start start 2024-04-17T10:42:18.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.954+0000 7fbde58e0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbde00ce400 0x7fbde00b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:18.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.954+0000 7fbde58e0640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbde015e4d0 con 0x7fbde00b5450 2024-04-17T10:42:18.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.954+0000 7fbde58e0640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbde0162580 con 0x7fbde015e0f0 2024-04-17T10:42:18.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.954+0000 7fbde58e0640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde0162b40 con 0x7fbde00ce400 2024-04-17T10:42:18.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.958+0000 7fbddf7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbde00b5450 0x7fbde00d2540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41568/0 (socket says 172.21.15.83:41568) 2024-04-17T10:42:18.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.958+0000 7fbddf7fe640 1 -- 172.21.15.83:0/2776121809 learned_addr learned my addr 172.21.15.83:0/2776121809 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:18.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.958+0000 7fbdde7fc640 1 --2- 172.21.15.83:0/2776121809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbde00ce400 0x7fbde00b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:18.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.958+0000 7fbdddffb640 1 -- 172.21.15.83:0/2776121809 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2729774685 0 0) 0x7fbde015e4d0 con 0x7fbde00b5450 2024-04-17T10:42:18.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.958+0000 7fbdddffb640 1 -- 172.21.15.83:0/2776121809 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdc8003530 con 0x7fbde00b5450 2024-04-17T10:42:18.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.958+0000 7fbdddffb640 1 -- 172.21.15.83:0/2776121809 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 325044784 0 0) 0x7fbdc8003530 con 0x7fbde00b5450 2024-04-17T10:42:18.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.958+0000 7fbdddffb640 1 -- 172.21.15.83:0/2776121809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbde00ce400 msgr2=0x7fbde00b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:18.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.958+0000 7fbdddffb640 1 --2- 172.21.15.83:0/2776121809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbde00ce400 0x7fbde00b6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:18.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.958+0000 7fbdddffb640 1 -- 172.21.15.83:0/2776121809 >> v1:172.21.15.167:6789/0 conn(0x7fbde015e0f0 legacy=0x7fbde0161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:18.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.958+0000 7fbdddffb640 1 -- 172.21.15.83:0/2776121809 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde0162860 con 0x7fbde00b5450 2024-04-17T10:42:18.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.958+0000 7fbdddffb640 1 -- 172.21.15.83:0/2776121809 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fbdd8001a00 con 0x7fbde00b5450 2024-04-17T10:42:18.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.958+0000 7fbdddffb640 1 -- 172.21.15.83:0/2776121809 >> v1:172.21.15.83:6789/0 conn(0x7fbde00b5450 legacy=0x7fbde00d2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:18.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.958+0000 7fbdddffb640 1 --2- 172.21.15.83:0/2776121809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdc8003a60 0x7fbdc8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:18.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.958+0000 7fbdddffb640 1 --2- 172.21.15.83:0/2776121809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdc8004810 0x7fbdc8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:18.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.958+0000 7fbdddffb640 1 --2- 172.21.15.83:0/2776121809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbde00ce400 0x7fbdc80051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:18.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.958+0000 7fbdddffb640 1 -- 172.21.15.83:0/2776121809 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde0162b40 con 0x7fbdc8004810 2024-04-17T10:42:18.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.958+0000 7fbdddffb640 1 -- 172.21.15.83:0/2776121809 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc800bae0 con 0x7fbdc8003a60 2024-04-17T10:42:18.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.958+0000 7fbdddffb640 1 -- 172.21.15.83:0/2776121809 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc800bce0 con 0x7fbde00ce400 2024-04-17T10:42:18.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.958+0000 7fbdde7fc640 1 --2- 172.21.15.83:0/2776121809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdc8004810 0x7fbdc8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:18.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.958+0000 7fbddf7fe640 1 --2- 172.21.15.83:0/2776121809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbde00ce400 0x7fbdc80051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:18.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.958+0000 7fbddeffd640 1 --2- 172.21.15.83:0/2776121809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdc8003a60 0x7fbdc8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:18.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.958+0000 7fbdde7fc640 1 -- 172.21.15.83:0/2776121809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbde00ce400 msgr2=0x7fbdc80051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:18.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.958+0000 7fbdde7fc640 1 --2- 172.21.15.83:0/2776121809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbde00ce400 0x7fbdc80051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:18.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.958+0000 7fbdde7fc640 1 -- 172.21.15.83:0/2776121809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdc8003a60 msgr2=0x7fbdc8003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:18.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.958+0000 7fbdde7fc640 1 --2- 172.21.15.83:0/2776121809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdc8003a60 0x7fbdc8003e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:18.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.958+0000 7fbdde7fc640 1 -- 172.21.15.83:0/2776121809 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc8003530 con 0x7fbdc8004810 2024-04-17T10:42:18.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.958+0000 7fbddf7fe640 1 --2- 172.21.15.83:0/2776121809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbde00ce400 0x7fbdc80051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:18.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.958+0000 7fbdde7fc640 1 --2- 172.21.15.83:0/2776121809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdc8004810 0x7fbdc8004c90 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0009ab0 tx=0x7fbdd001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd23c23b38d353e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:18.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.958+0000 7fbdddffb640 1 -- 172.21.15.83:0/2776121809 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd00b2070 con 0x7fbdc8004810 2024-04-17T10:42:18.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.958+0000 7fbdddffb640 1 -- 172.21.15.83:0/2776121809 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd00a5640 con 0x7fbdc8004810 2024-04-17T10:42:18.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.958+0000 7fbdddffb640 1 -- 172.21.15.83:0/2776121809 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd00096b0 con 0x7fbdc8004810 2024-04-17T10:42:18.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbde58e0640 1 -- 172.21.15.83:0/2776121809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdc8004810 msgr2=0x7fbdc8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:18.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbde58e0640 1 --2- 172.21.15.83:0/2776121809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdc8004810 0x7fbdc8004c90 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0009ab0 tx=0x7fbdd001ca50 comp rx=0 tx=0).stop 2024-04-17T10:42:18.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbddf7fe640 1 -- 172.21.15.83:0/2776121809 reap_dead start 2024-04-17T10:42:18.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbde58e0640 1 -- 172.21.15.83:0/2776121809 shutdown_connections 2024-04-17T10:42:18.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbde58e0640 1 -- 172.21.15.83:0/2776121809 >> 172.21.15.83:0/2776121809 conn(0x7fbde00d0450 msgr2=0x7fbde00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:18.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbde58e0640 1 -- 172.21.15.83:0/2776121809 shutdown_connections 2024-04-17T10:42:18.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbde58e0640 1 -- 172.21.15.83:0/2776121809 wait complete. 2024-04-17T10:42:18.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbde58e0640 1 Processor -- start 2024-04-17T10:42:18.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbde58e0640 1 -- start start 2024-04-17T10:42:18.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbde58e0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdc8004810 0x7fbde015e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:18.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbde58e0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbde006cce0 0x7fbde00c9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:18.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbde58e0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbde006d160 0x7fbde00c9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:18.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbde58e0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde00d2020 con 0x7fbdc8004810 2024-04-17T10:42:18.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbde58e0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde0162320 con 0x7fbde006cce0 2024-04-17T10:42:18.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbde58e0640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde00b6d00 con 0x7fbde006d160 2024-04-17T10:42:18.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbddeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdc8004810 0x7fbde015e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:18.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbddeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdc8004810 0x7fbde015e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57506/0 (socket says 172.21.15.83:57506) 2024-04-17T10:42:18.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbddeffd640 1 -- 172.21.15.83:0/4122486149 learned_addr learned my addr 172.21.15.83:0/4122486149 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:18.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbdde7fc640 1 --2- 172.21.15.83:0/4122486149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbde006cce0 0x7fbde00c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:18.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbddeffd640 1 -- 172.21.15.83:0/4122486149 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbde006d160 msgr2=0x7fbde00c9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:18.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbddf7fe640 1 --2- 172.21.15.83:0/4122486149 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbde006d160 0x7fbde00c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:18.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbddeffd640 1 --2- 172.21.15.83:0/4122486149 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbde006d160 0x7fbde00c9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:18.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbddeffd640 1 -- 172.21.15.83:0/4122486149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbde006cce0 msgr2=0x7fbde00c9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:18.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbddeffd640 1 --2- 172.21.15.83:0/4122486149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbde006cce0 0x7fbde00c9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:18.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbddeffd640 1 -- 172.21.15.83:0/4122486149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde015eff0 con 0x7fbdc8004810 2024-04-17T10:42:18.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbdde7fc640 1 --2- 172.21.15.83:0/4122486149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbde006cce0 0x7fbde00c9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:18.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbddf7fe640 1 --2- 172.21.15.83:0/4122486149 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbde006d160 0x7fbde00c9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:18.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbddeffd640 1 --2- 172.21.15.83:0/4122486149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdc8004810 0x7fbde015e680 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fbdd400f940 tx=0x7fbdd400fe00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:18.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbdc7fff640 1 -- 172.21.15.83:0/4122486149 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4005280 con 0x7fbdc8004810 2024-04-17T10:42:18.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbdc7fff640 1 -- 172.21.15.83:0/4122486149 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd4019070 con 0x7fbdc8004810 2024-04-17T10:42:18.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbde58e0640 1 -- 172.21.15.83:0/4122486149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde015f2d0 con 0x7fbdc8004810 2024-04-17T10:42:18.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbdc7fff640 1 -- 172.21.15.83:0/4122486149 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd400d780 con 0x7fbdc8004810 2024-04-17T10:42:18.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbde58e0640 1 -- 172.21.15.83:0/4122486149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde015f890 con 0x7fbdc8004810 2024-04-17T10:42:18.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.962+0000 7fbde58e0640 1 -- 172.21.15.83:0/4122486149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde00681a0 con 0x7fbdc8004810 2024-04-17T10:42:18.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.966+0000 7fbdc7fff640 1 -- 172.21.15.83:0/4122486149 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbdd400d920 con 0x7fbdc8004810 2024-04-17T10:42:18.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.966+0000 7fbdc7fff640 1 --2- 172.21.15.83:0/4122486149 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbdcc066f90 0x7fbdcc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:18.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.966+0000 7fbdc7fff640 1 -- 172.21.15.83:0/4122486149 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fbdd4088040 con 0x7fbdc8004810 2024-04-17T10:42:18.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.966+0000 7fbdde7fc640 1 --2- 172.21.15.83:0/4122486149 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbdcc066f90 0x7fbdcc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:18.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.966+0000 7fbdde7fc640 1 --2- 172.21.15.83:0/4122486149 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbdcc066f90 0x7fbdcc069450 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbdd00af000 tx=0x7fbdd00046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:18.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:18.970+0000 7fbdc7fff640 1 -- 172.21.15.83:0/4122486149 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbdd4059010 con 0x7fbdc8004810 2024-04-17T10:42:19.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.126+0000 7fbde58e0640 1 -- 172.21.15.83:0/4122486149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbde00d17c0 con 0x7fbdc8004810 2024-04-17T10:42:19.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.126+0000 7fbdc7fff640 1 -- 172.21.15.83:0/4122486149 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbdd4058910 con 0x7fbdc8004810 2024-04-17T10:42:19.125 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:42:19.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.130+0000 7fbde58e0640 1 -- 172.21.15.83:0/4122486149 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbdcc066f90 msgr2=0x7fbdcc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:19.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.130+0000 7fbde58e0640 1 --2- 172.21.15.83:0/4122486149 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbdcc066f90 0x7fbdcc069450 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbdd00af000 tx=0x7fbdd00046d0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.130+0000 7fbde58e0640 1 -- 172.21.15.83:0/4122486149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdc8004810 msgr2=0x7fbde015e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:19.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.130+0000 7fbde58e0640 1 --2- 172.21.15.83:0/4122486149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdc8004810 0x7fbde015e680 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fbdd400f940 tx=0x7fbdd400fe00 comp rx=0 tx=0).stop 2024-04-17T10:42:19.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.130+0000 7fbde58e0640 1 -- 172.21.15.83:0/4122486149 shutdown_connections 2024-04-17T10:42:19.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.130+0000 7fbde58e0640 1 --2- 172.21.15.83:0/4122486149 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbdcc066f90 0x7fbdcc069450 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.130+0000 7fbde58e0640 1 --2- 172.21.15.83:0/4122486149 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbde006d160 0x7fbde00c9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.130+0000 7fbde58e0640 1 --2- 172.21.15.83:0/4122486149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbde006cce0 0x7fbde00c9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.130+0000 7fbde58e0640 1 --2- 172.21.15.83:0/4122486149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdc8004810 0x7fbde015e680 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.130+0000 7fbde58e0640 1 -- 172.21.15.83:0/4122486149 >> 172.21.15.83:0/4122486149 conn(0x7fbde00d0450 msgr2=0x7fbde015c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:19.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.130+0000 7fbde58e0640 1 -- 172.21.15.83:0/4122486149 shutdown_connections 2024-04-17T10:42:19.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.130+0000 7fbde58e0640 1 -- 172.21.15.83:0/4122486149 wait complete. 2024-04-17T10:42:19.144 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:42:19.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.258+0000 7f2ef2235640 1 Processor -- start 2024-04-17T10:42:19.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.258+0000 7f2ef2235640 1 -- start start 2024-04-17T10:42:19.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.258+0000 7f2ef2235640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2eec15e0f0 0x7f2eec0b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:19.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.258+0000 7f2ef2235640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2eec0ce400 0x7f2eec161c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:19.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.258+0000 7f2ef2235640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2eec0b5450 0x7f2eec0b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:19.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.258+0000 7f2ef2235640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eec162b40 con 0x7f2eec15e0f0 2024-04-17T10:42:19.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.258+0000 7f2ef2235640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eec15e4d0 con 0x7f2eec0ce400 2024-04-17T10:42:19.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.258+0000 7f2ef2235640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eec162580 con 0x7f2eec0b5450 2024-04-17T10:42:19.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.258+0000 7f2eeb7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2eec15e0f0 0x7f2eec0b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:19.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.258+0000 7f2eeb7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2eec15e0f0 0x7f2eec0b6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57522/0 (socket says 172.21.15.83:57522) 2024-04-17T10:42:19.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.258+0000 7f2eeb7fe640 1 -- 172.21.15.83:0/3973490253 learned_addr learned my addr 172.21.15.83:0/3973490253 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:19.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.258+0000 7f2eeb7fe640 1 -- 172.21.15.83:0/3973490253 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2eec0b5450 msgr2=0x7f2eec0b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:19.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.258+0000 7f2eeb7fe640 1 --2- 172.21.15.83:0/3973490253 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2eec0b5450 0x7f2eec0b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.258+0000 7f2ee2ffd640 1 --2- 172.21.15.83:0/3973490253 >> v2:172.21.15.167:3300/0 conn(0x7f2eec0ce400 0x7f2eec161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:19.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.258+0000 7f2eeb7fe640 1 -- 172.21.15.83:0/3973490253 >> v2:172.21.15.167:3300/0 conn(0x7f2eec0ce400 msgr2=0x7f2eec161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:19.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.258+0000 7f2eeb7fe640 1 --2- 172.21.15.83:0/3973490253 >> v2:172.21.15.167:3300/0 conn(0x7f2eec0ce400 0x7f2eec161c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.258+0000 7f2eeb7fe640 1 -- 172.21.15.83:0/3973490253 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eec162860 con 0x7f2eec15e0f0 2024-04-17T10:42:19.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.258+0000 7f2eeb7fe640 1 --2- 172.21.15.83:0/3973490253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2eec15e0f0 0x7f2eec0b6d00 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f2eec068170 tx=0x7f2edc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3a7b8db621d011 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:19.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2eeaffd640 1 -- 172.21.15.83:0/3973490253 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc0b0070 con 0x7f2eec15e0f0 2024-04-17T10:42:19.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2eeaffd640 1 -- 172.21.15.83:0/3973490253 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2edc0ab600 con 0x7f2eec15e0f0 2024-04-17T10:42:19.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2eeaffd640 1 -- 172.21.15.83:0/3973490253 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc0ab9d0 con 0x7f2eec15e0f0 2024-04-17T10:42:19.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2ef2235640 1 -- 172.21.15.83:0/3973490253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2eec15e0f0 msgr2=0x7f2eec0b6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:19.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2ef2235640 1 --2- 172.21.15.83:0/3973490253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2eec15e0f0 0x7f2eec0b6d00 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f2eec068170 tx=0x7f2edc01c490 comp rx=0 tx=0).stop 2024-04-17T10:42:19.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2ef2235640 1 -- 172.21.15.83:0/3973490253 shutdown_connections 2024-04-17T10:42:19.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2ef2235640 1 --2- 172.21.15.83:0/3973490253 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2eec0b5450 0x7f2eec0b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2ef2235640 1 --2- 172.21.15.83:0/3973490253 >> v2:172.21.15.167:3300/0 conn(0x7f2eec0ce400 0x7f2eec161c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2ef2235640 1 --2- 172.21.15.83:0/3973490253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2eec15e0f0 0x7f2eec0b6d00 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2ef2235640 1 -- 172.21.15.83:0/3973490253 >> 172.21.15.83:0/3973490253 conn(0x7f2eec0d0450 msgr2=0x7f2eec0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:19.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2ef2235640 1 -- 172.21.15.83:0/3973490253 shutdown_connections 2024-04-17T10:42:19.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2ef2235640 1 -- 172.21.15.83:0/3973490253 wait complete. 2024-04-17T10:42:19.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2ef2235640 1 Processor -- start 2024-04-17T10:42:19.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2ef2235640 1 -- start start 2024-04-17T10:42:19.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2ef2235640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2eec0b5450 0x7f2eec154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:19.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2ef2235640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2eec0ce400 0x7f2eec150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:19.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2ef2235640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2eec15e0f0 0x7f2eec14cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:19.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2ef2235640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eec0d2d50 con 0x7f2eec0ce400 2024-04-17T10:42:19.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2ef2235640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eec162360 con 0x7f2eec15e0f0 2024-04-17T10:42:19.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2ef2235640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eec0d2ed0 con 0x7f2eec0b5450 2024-04-17T10:42:19.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2ee2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2eec0ce400 0x7f2eec150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:19.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2ee2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2eec0ce400 0x7f2eec150150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57528/0 (socket says 172.21.15.83:57528) 2024-04-17T10:42:19.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2ee2ffd640 1 -- 172.21.15.83:0/3270207991 learned_addr learned my addr 172.21.15.83:0/3270207991 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:19.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2eeb7fe640 1 --2- 172.21.15.83:0/3270207991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2eec0b5450 0x7f2eec154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:19.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2eebfff640 1 --2- 172.21.15.83:0/3270207991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2eec15e0f0 0x7f2eec14cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:19.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2ee2ffd640 1 -- 172.21.15.83:0/3270207991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2eec0b5450 msgr2=0x7f2eec154210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:19.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2ee2ffd640 1 --2- 172.21.15.83:0/3270207991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2eec0b5450 0x7f2eec154210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2ee2ffd640 1 -- 172.21.15.83:0/3270207991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2eec15e0f0 msgr2=0x7f2eec14cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:19.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2ee2ffd640 1 --2- 172.21.15.83:0/3270207991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2eec15e0f0 0x7f2eec14cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2ee2ffd640 1 -- 172.21.15.83:0/3270207991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eec0cae80 con 0x7f2eec0ce400 2024-04-17T10:42:19.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2eeb7fe640 1 --2- 172.21.15.83:0/3270207991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2eec0b5450 0x7f2eec154210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:19.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2eebfff640 1 --2- 172.21.15.83:0/3270207991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2eec15e0f0 0x7f2eec14cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:19.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2ee2ffd640 1 --2- 172.21.15.83:0/3270207991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2eec0ce400 0x7f2eec150150 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f2ed400c7f0 tx=0x7f2ed400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:19.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2ee8ff9640 1 -- 172.21.15.83:0/3270207991 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4015070 con 0x7f2eec0ce400 2024-04-17T10:42:19.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2ee8ff9640 1 -- 172.21.15.83:0/3270207991 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed4004220 con 0x7f2eec0ce400 2024-04-17T10:42:19.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2ef2235640 1 -- 172.21.15.83:0/3270207991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eec0cb160 con 0x7f2eec0ce400 2024-04-17T10:42:19.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2ee8ff9640 1 -- 172.21.15.83:0/3270207991 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4003620 con 0x7f2eec0ce400 2024-04-17T10:42:19.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.262+0000 7f2ef2235640 1 -- 172.21.15.83:0/3270207991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eec0cb720 con 0x7f2eec0ce400 2024-04-17T10:42:19.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.266+0000 7f2ef2235640 1 -- 172.21.15.83:0/3270207991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eec0681a0 con 0x7f2eec0ce400 2024-04-17T10:42:19.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.266+0000 7f2ee8ff9640 1 -- 172.21.15.83:0/3270207991 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2ed40039d0 con 0x7f2eec0ce400 2024-04-17T10:42:19.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.266+0000 7f2ee8ff9640 1 --2- 172.21.15.83:0/3270207991 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2ecc066f60 0x7f2ecc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:19.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.266+0000 7f2ee8ff9640 1 -- 172.21.15.83:0/3270207991 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f2ed4085120 con 0x7f2eec0ce400 2024-04-17T10:42:19.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.266+0000 7f2eeb7fe640 1 --2- 172.21.15.83:0/3270207991 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2ecc066f60 0x7f2ecc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:19.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.266+0000 7f2eeb7fe640 1 --2- 172.21.15.83:0/3270207991 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2ecc066f60 0x7f2ecc069420 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2edc0ad000 tx=0x7f2edc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:19.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.270+0000 7f2ee8ff9640 1 -- 172.21.15.83:0/3270207991 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2ed40560f0 con 0x7f2eec0ce400 2024-04-17T10:42:19.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.418+0000 7f2ef2235640 1 -- 172.21.15.83:0/3270207991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2eec074f90 con 0x7f2eec0ce400 2024-04-17T10:42:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.418+0000 7f2ee8ff9640 1 -- 172.21.15.83:0/3270207991 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v48) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2ed40559f0 con 0x7f2eec0ce400 2024-04-17T10:42:19.416 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:42:19.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.418+0000 7f2ef2235640 1 -- 172.21.15.83:0/3270207991 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2ecc066f60 msgr2=0x7f2ecc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:19.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.418+0000 7f2ef2235640 1 --2- 172.21.15.83:0/3270207991 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2ecc066f60 0x7f2ecc069420 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2edc0ad000 tx=0x7f2edc008040 comp rx=0 tx=0).stop 2024-04-17T10:42:19.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.418+0000 7f2ef2235640 1 -- 172.21.15.83:0/3270207991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2eec0ce400 msgr2=0x7f2eec150150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:19.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.418+0000 7f2ef2235640 1 --2- 172.21.15.83:0/3270207991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2eec0ce400 0x7f2eec150150 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f2ed400c7f0 tx=0x7f2ed400ccb0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.418+0000 7f2ef2235640 1 -- 172.21.15.83:0/3270207991 shutdown_connections 2024-04-17T10:42:19.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.418+0000 7f2ef2235640 1 --2- 172.21.15.83:0/3270207991 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2ecc066f60 0x7f2ecc069420 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.418+0000 7f2ef2235640 1 --2- 172.21.15.83:0/3270207991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2eec15e0f0 0x7f2eec14cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.418+0000 7f2ef2235640 1 --2- 172.21.15.83:0/3270207991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2eec0ce400 0x7f2eec150150 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.418+0000 7f2ef2235640 1 --2- 172.21.15.83:0/3270207991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2eec0b5450 0x7f2eec154210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.418+0000 7f2ef2235640 1 -- 172.21.15.83:0/3270207991 >> 172.21.15.83:0/3270207991 conn(0x7f2eec0d0450 msgr2=0x7f2eec15e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:19.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.418+0000 7f2ef2235640 1 -- 172.21.15.83:0/3270207991 shutdown_connections 2024-04-17T10:42:19.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.418+0000 7f2ef2235640 1 -- 172.21.15.83:0/3270207991 wait complete. 2024-04-17T10:42:19.434 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:42:19.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.546+0000 7fdc90ed7640 1 Processor -- start 2024-04-17T10:42:19.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.546+0000 7fdc90ed7640 1 -- start start 2024-04-17T10:42:19.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc90ed7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc8c06b520 0x7fdc8c071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:19.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc90ed7640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc8c06c1f0 con 0x7fdc8c15bac0 2024-04-17T10:42:19.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc90ed7640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc8c070a10 con 0x7fdc8c15b6e0 2024-04-17T10:42:19.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc90ed7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c070cf0 con 0x7fdc8c06b520 2024-04-17T10:42:19.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc89d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fdc8c15bac0 0x7fdc8c06ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41578/0 (socket says 172.21.15.83:41578) 2024-04-17T10:42:19.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc89d9b640 1 -- 172.21.15.83:0/2763022788 learned_addr learned my addr 172.21.15.83:0/2763022788 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:19.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc8ad9d640 1 --2- 172.21.15.83:0/2763022788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc8c06b520 0x7fdc8c071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:19.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc8959a640 1 -- 172.21.15.83:0/2763022788 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 498354635 0 0) 0x7fdc8c06c1f0 con 0x7fdc8c15bac0 2024-04-17T10:42:19.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc8959a640 1 -- 172.21.15.83:0/2763022788 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc74003530 con 0x7fdc8c15bac0 2024-04-17T10:42:19.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc8959a640 1 -- 172.21.15.83:0/2763022788 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3999097433 0 0) 0x7fdc74003530 con 0x7fdc8c15bac0 2024-04-17T10:42:19.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc8959a640 1 -- 172.21.15.83:0/2763022788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc8c06b520 msgr2=0x7fdc8c071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:19.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc8959a640 1 --2- 172.21.15.83:0/2763022788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc8c06b520 0x7fdc8c071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc8959a640 1 -- 172.21.15.83:0/2763022788 >> v1:172.21.15.167:6789/0 conn(0x7fdc8c15b6e0 legacy=0x7fdc8c0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:19.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc8959a640 1 -- 172.21.15.83:0/2763022788 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc8c070fd0 con 0x7fdc8c15bac0 2024-04-17T10:42:19.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc8959a640 1 -- 172.21.15.83:0/2763022788 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fdc80001680 con 0x7fdc8c15bac0 2024-04-17T10:42:19.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc8959a640 1 -- 172.21.15.83:0/2763022788 >> v1:172.21.15.83:6789/0 conn(0x7fdc8c15bac0 legacy=0x7fdc8c06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:19.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc8959a640 1 --2- 172.21.15.83:0/2763022788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc74003a60 0x7fdc74003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:19.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc8959a640 1 --2- 172.21.15.83:0/2763022788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc74004810 0x7fdc74004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:19.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc8959a640 1 --2- 172.21.15.83:0/2763022788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc8c06b520 0x7fdc740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:19.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc8959a640 1 -- 172.21.15.83:0/2763022788 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c070cf0 con 0x7fdc74003a60 2024-04-17T10:42:19.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc8959a640 1 -- 172.21.15.83:0/2763022788 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7400bb00 con 0x7fdc74004810 2024-04-17T10:42:19.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc8959a640 1 -- 172.21.15.83:0/2763022788 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7400bd00 con 0x7fdc8c06b520 2024-04-17T10:42:19.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc8a59c640 1 --2- 172.21.15.83:0/2763022788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc74003a60 0x7fdc74003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:19.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc89d9b640 1 --2- 172.21.15.83:0/2763022788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc74004810 0x7fdc74004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:19.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc8ad9d640 1 --2- 172.21.15.83:0/2763022788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc8c06b520 0x7fdc740051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:19.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc8a59c640 1 -- 172.21.15.83:0/2763022788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc8c06b520 msgr2=0x7fdc740051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:19.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc8a59c640 1 --2- 172.21.15.83:0/2763022788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc8c06b520 0x7fdc740051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc8a59c640 1 -- 172.21.15.83:0/2763022788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc74004810 msgr2=0x7fdc74004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:19.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc8a59c640 1 --2- 172.21.15.83:0/2763022788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc74004810 0x7fdc74004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc8a59c640 1 -- 172.21.15.83:0/2763022788 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc74003530 con 0x7fdc74003a60 2024-04-17T10:42:19.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc8a59c640 1 --2- 172.21.15.83:0/2763022788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc74003a60 0x7fdc74003e80 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c001780 tx=0x7fdc7c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dbc272ce071c759 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:19.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc8959a640 1 -- 172.21.15.83:0/2763022788 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c0013a0 con 0x7fdc74003a60 2024-04-17T10:42:19.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc8959a640 1 -- 172.21.15.83:0/2763022788 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc7c0050d0 con 0x7fdc74003a60 2024-04-17T10:42:19.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.550+0000 7fdc8959a640 1 -- 172.21.15.83:0/2763022788 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c0054a0 con 0x7fdc74003a60 2024-04-17T10:42:19.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.554+0000 7fdc90ed7640 1 -- 172.21.15.83:0/2763022788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc74003a60 msgr2=0x7fdc74003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:19.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.554+0000 7fdc90ed7640 1 --2- 172.21.15.83:0/2763022788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc74003a60 0x7fdc74003e80 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c001780 tx=0x7fdc7c01c420 comp rx=0 tx=0).stop 2024-04-17T10:42:19.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.554+0000 7fdc8ad9d640 1 -- 172.21.15.83:0/2763022788 reap_dead start 2024-04-17T10:42:19.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.554+0000 7fdc90ed7640 1 -- 172.21.15.83:0/2763022788 shutdown_connections 2024-04-17T10:42:19.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.554+0000 7fdc90ed7640 1 -- 172.21.15.83:0/2763022788 >> 172.21.15.83:0/2763022788 conn(0x7fdc8c0b15b0 msgr2=0x7fdc8c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:19.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.554+0000 7fdc90ed7640 1 -- 172.21.15.83:0/2763022788 shutdown_connections 2024-04-17T10:42:19.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.554+0000 7fdc90ed7640 1 -- 172.21.15.83:0/2763022788 wait complete. 2024-04-17T10:42:19.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.554+0000 7fdc90ed7640 1 Processor -- start 2024-04-17T10:42:19.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.554+0000 7fdc90ed7640 1 -- start start 2024-04-17T10:42:19.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.554+0000 7fdc90ed7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc74003a60 0x7fdc8c155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:19.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.554+0000 7fdc90ed7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc8c160090 0x7fdc8c157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:19.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.554+0000 7fdc90ed7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc8c1f80f0 0x7fdc8c1f8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:19.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.554+0000 7fdc90ed7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c06d2b0 con 0x7fdc8c1f80f0 2024-04-17T10:42:19.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.554+0000 7fdc90ed7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c06d130 con 0x7fdc8c160090 2024-04-17T10:42:19.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.554+0000 7fdc90ed7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c167600 con 0x7fdc74003a60 2024-04-17T10:42:19.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.554+0000 7fdc8ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc8c1f80f0 0x7fdc8c1f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:19.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.554+0000 7fdc8ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc8c1f80f0 0x7fdc8c1f8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57542/0 (socket says 172.21.15.83:57542) 2024-04-17T10:42:19.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.554+0000 7fdc8ad9d640 1 -- 172.21.15.83:0/3537550765 learned_addr learned my addr 172.21.15.83:0/3537550765 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:19.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.554+0000 7fdc8a59c640 1 --2- 172.21.15.83:0/3537550765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc74003a60 0x7fdc8c155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:19.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.554+0000 7fdc89d9b640 1 --2- 172.21.15.83:0/3537550765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc8c160090 0x7fdc8c157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:19.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.554+0000 7fdc8ad9d640 1 -- 172.21.15.83:0/3537550765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc74003a60 msgr2=0x7fdc8c155330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:19.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.554+0000 7fdc8ad9d640 1 --2- 172.21.15.83:0/3537550765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc74003a60 0x7fdc8c155330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.554+0000 7fdc8ad9d640 1 -- 172.21.15.83:0/3537550765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc8c160090 msgr2=0x7fdc8c157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:19.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.554+0000 7fdc8ad9d640 1 --2- 172.21.15.83:0/3537550765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc8c160090 0x7fdc8c157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.554+0000 7fdc8ad9d640 1 -- 172.21.15.83:0/3537550765 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc8c155870 con 0x7fdc8c1f80f0 2024-04-17T10:42:19.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.554+0000 7fdc8ad9d640 1 --2- 172.21.15.83:0/3537550765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc8c1f80f0 0x7fdc8c1f8580 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c0cdf00 tx=0x7fdc840115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:19.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.554+0000 7fdc7affd640 1 -- 172.21.15.83:0/3537550765 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc840039a0 con 0x7fdc8c1f80f0 2024-04-17T10:42:19.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.554+0000 7fdc7affd640 1 -- 172.21.15.83:0/3537550765 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8401b070 con 0x7fdc8c1f80f0 2024-04-17T10:42:19.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.554+0000 7fdc90ed7640 1 -- 172.21.15.83:0/3537550765 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc8c1f6d90 con 0x7fdc8c1f80f0 2024-04-17T10:42:19.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.554+0000 7fdc7affd640 1 -- 172.21.15.83:0/3537550765 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc84003b80 con 0x7fdc8c1f80f0 2024-04-17T10:42:19.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.554+0000 7fdc90ed7640 1 -- 172.21.15.83:0/3537550765 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc8c160870 con 0x7fdc8c1f80f0 2024-04-17T10:42:19.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.554+0000 7fdc90ed7640 1 -- 172.21.15.83:0/3537550765 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc8c074f90 con 0x7fdc8c1f80f0 2024-04-17T10:42:19.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.558+0000 7fdc7affd640 1 -- 172.21.15.83:0/3537550765 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdc84004030 con 0x7fdc8c1f80f0 2024-04-17T10:42:19.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.558+0000 7fdc7affd640 1 --2- 172.21.15.83:0/3537550765 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdc6c066f60 0x7fdc6c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:19.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.558+0000 7fdc7affd640 1 -- 172.21.15.83:0/3537550765 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fdc84085700 con 0x7fdc8c1f80f0 2024-04-17T10:42:19.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.558+0000 7fdc8a59c640 1 --2- 172.21.15.83:0/3537550765 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdc6c066f60 0x7fdc6c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:19.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.558+0000 7fdc8a59c640 1 --2- 172.21.15.83:0/3537550765 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdc6c066f60 0x7fdc6c069420 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c001750 tx=0x7fdc7c0011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:19.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.562+0000 7fdc7affd640 1 -- 172.21.15.83:0/3537550765 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdc84056760 con 0x7fdc8c1f80f0 2024-04-17T10:42:19.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.706+0000 7fdc90ed7640 1 -- 172.21.15.83:0/3537550765 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdc8c06b120 con 0x7fdc8c1f80f0 2024-04-17T10:42:19.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.706+0000 7fdc7affd640 1 -- 172.21.15.83:0/3537550765 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v48) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdc84056060 con 0x7fdc8c1f80f0 2024-04-17T10:42:19.705 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:42:19.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.706+0000 7fdc90ed7640 1 -- 172.21.15.83:0/3537550765 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdc6c066f60 msgr2=0x7fdc6c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:19.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.706+0000 7fdc90ed7640 1 --2- 172.21.15.83:0/3537550765 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdc6c066f60 0x7fdc6c069420 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c001750 tx=0x7fdc7c0011a0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.706+0000 7fdc90ed7640 1 -- 172.21.15.83:0/3537550765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc8c1f80f0 msgr2=0x7fdc8c1f8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:19.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.706+0000 7fdc90ed7640 1 --2- 172.21.15.83:0/3537550765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc8c1f80f0 0x7fdc8c1f8580 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c0cdf00 tx=0x7fdc840115a0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.706+0000 7fdc90ed7640 1 -- 172.21.15.83:0/3537550765 shutdown_connections 2024-04-17T10:42:19.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.706+0000 7fdc90ed7640 1 --2- 172.21.15.83:0/3537550765 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdc6c066f60 0x7fdc6c069420 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.706+0000 7fdc90ed7640 1 --2- 172.21.15.83:0/3537550765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc8c1f80f0 0x7fdc8c1f8580 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.706+0000 7fdc90ed7640 1 --2- 172.21.15.83:0/3537550765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc8c160090 0x7fdc8c157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.706+0000 7fdc90ed7640 1 --2- 172.21.15.83:0/3537550765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc74003a60 0x7fdc8c155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.706+0000 7fdc90ed7640 1 -- 172.21.15.83:0/3537550765 >> 172.21.15.83:0/3537550765 conn(0x7fdc8c0b15b0 msgr2=0x7fdc8c0cf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:19.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.706+0000 7fdc90ed7640 1 -- 172.21.15.83:0/3537550765 shutdown_connections 2024-04-17T10:42:19.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.706+0000 7fdc90ed7640 1 -- 172.21.15.83:0/3537550765 wait complete. 2024-04-17T10:42:19.723 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:42:19.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f55e54640 1 Processor -- start 2024-04-17T10:42:19.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f55e54640 1 -- start start 2024-04-17T10:42:19.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f55e54640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0f5015dc70 0x7f0f5006bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:19.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f55e54640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f5006cd80 0x7f0f500cc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:19.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f55e54640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f500d0850 con 0x7f0f5015e050 2024-04-17T10:42:19.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f55e54640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f500d0b30 con 0x7f0f5015dc70 2024-04-17T10:42:19.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f55e54640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f500cbf80 con 0x7f0f5006cd80 2024-04-17T10:42:19.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f4effd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0f5015e050 0x7f0f5006c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41584/0 (socket says 172.21.15.83:41584) 2024-04-17T10:42:19.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f4effd640 1 -- 172.21.15.83:0/2822371641 learned_addr learned my addr 172.21.15.83:0/2822371641 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:19.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f4f7fe640 1 --2- 172.21.15.83:0/2822371641 >> v2:172.21.15.167:3300/0 conn(0x7f0f5015dc70 0x7f0f5006bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:19.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f4e7fc640 1 -- 172.21.15.83:0/2822371641 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2305448343 0 0) 0x7f0f500d0850 con 0x7f0f5015e050 2024-04-17T10:42:19.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f4ffff640 1 --2- 172.21.15.83:0/2822371641 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f5006cd80 0x7f0f500cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:19.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f4e7fc640 1 -- 172.21.15.83:0/2822371641 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f38003530 con 0x7f0f5015e050 2024-04-17T10:42:19.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f4e7fc640 1 -- 172.21.15.83:0/2822371641 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 539926113 0 0) 0x7f0f38003530 con 0x7f0f5015e050 2024-04-17T10:42:19.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f4e7fc640 1 -- 172.21.15.83:0/2822371641 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f5006cd80 msgr2=0x7f0f500cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:19.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f4e7fc640 1 --2- 172.21.15.83:0/2822371641 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f5006cd80 0x7f0f500cc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f4e7fc640 1 -- 172.21.15.83:0/2822371641 >> v2:172.21.15.167:3300/0 conn(0x7f0f5015dc70 msgr2=0x7f0f5006bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:19.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f4e7fc640 1 --2- 172.21.15.83:0/2822371641 >> v2:172.21.15.167:3300/0 conn(0x7f0f5015dc70 0x7f0f5006bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f4e7fc640 1 -- 172.21.15.83:0/2822371641 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f500cc260 con 0x7f0f5015e050 2024-04-17T10:42:19.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f4e7fc640 1 -- 172.21.15.83:0/2822371641 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f0f44001680 con 0x7f0f5015e050 2024-04-17T10:42:19.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f4e7fc640 1 -- 172.21.15.83:0/2822371641 >> v1:172.21.15.83:6789/0 conn(0x7f0f5015e050 legacy=0x7f0f5006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:19.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f4e7fc640 1 --2- 172.21.15.83:0/2822371641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f38003a30 0x7f0f38003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:19.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f4e7fc640 1 --2- 172.21.15.83:0/2822371641 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f5006cd80 0x7f0f38004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:19.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f4e7fc640 1 --2- 172.21.15.83:0/2822371641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f38004d50 0x7f0f380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:19.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f4e7fc640 1 -- 172.21.15.83:0/2822371641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f500d0b30 con 0x7f0f38004d50 2024-04-17T10:42:19.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f4e7fc640 1 -- 172.21.15.83:0/2822371641 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f500cbf80 con 0x7f0f38003a30 2024-04-17T10:42:19.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f4e7fc640 1 -- 172.21.15.83:0/2822371641 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3800bb90 con 0x7f0f5006cd80 2024-04-17T10:42:19.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f4ffff640 1 --2- 172.21.15.83:0/2822371641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f38004d50 0x7f0f380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:19.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f4f7fe640 1 --2- 172.21.15.83:0/2822371641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f38003a30 0x7f0f38003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:19.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f4effd640 1 --2- 172.21.15.83:0/2822371641 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f5006cd80 0x7f0f38004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:19.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f4ffff640 1 -- 172.21.15.83:0/2822371641 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f5006cd80 msgr2=0x7f0f38004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:19.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f4ffff640 1 --2- 172.21.15.83:0/2822371641 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f5006cd80 0x7f0f38004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f4ffff640 1 -- 172.21.15.83:0/2822371641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f38003a30 msgr2=0x7f0f38003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:19.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f4ffff640 1 --2- 172.21.15.83:0/2822371641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f38003a30 0x7f0f38003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.838+0000 7f0f4ffff640 1 -- 172.21.15.83:0/2822371641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f38003530 con 0x7f0f38004d50 2024-04-17T10:42:19.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.842+0000 7f0f4ffff640 1 --2- 172.21.15.83:0/2822371641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f38004d50 0x7f0f380051d0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f0f48007330 tx=0x7f0f4801ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd49d4f63f55263b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:19.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.842+0000 7f0f4e7fc640 1 -- 172.21.15.83:0/2822371641 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f480b2070 con 0x7f0f38004d50 2024-04-17T10:42:19.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.842+0000 7f0f4e7fc640 1 -- 172.21.15.83:0/2822371641 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f480a5d20 con 0x7f0f38004d50 2024-04-17T10:42:19.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.842+0000 7f0f4e7fc640 1 -- 172.21.15.83:0/2822371641 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f48009be0 con 0x7f0f38004d50 2024-04-17T10:42:19.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.842+0000 7f0f55e54640 1 -- 172.21.15.83:0/2822371641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f38004d50 msgr2=0x7f0f380051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:19.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.842+0000 7f0f55e54640 1 --2- 172.21.15.83:0/2822371641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f38004d50 0x7f0f380051d0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f0f48007330 tx=0x7f0f4801ce20 comp rx=0 tx=0).stop 2024-04-17T10:42:19.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.842+0000 7f0f4ffff640 1 -- 172.21.15.83:0/2822371641 reap_dead start 2024-04-17T10:42:19.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.842+0000 7f0f55e54640 1 -- 172.21.15.83:0/2822371641 shutdown_connections 2024-04-17T10:42:19.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.842+0000 7f0f55e54640 1 -- 172.21.15.83:0/2822371641 >> 172.21.15.83:0/2822371641 conn(0x7f0f500d1790 msgr2=0x7f0f500d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:19.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.842+0000 7f0f55e54640 1 -- 172.21.15.83:0/2822371641 shutdown_connections 2024-04-17T10:42:19.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.842+0000 7f0f55e54640 1 -- 172.21.15.83:0/2822371641 wait complete. 2024-04-17T10:42:19.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.842+0000 7f0f55e54640 1 Processor -- start 2024-04-17T10:42:19.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.842+0000 7f0f55e54640 1 -- start start 2024-04-17T10:42:19.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.846+0000 7f0f55e54640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f50156a40 0x7f0f500c52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:19.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.846+0000 7f0f55e54640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f500b5840 0x7f0f500b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:19.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.846+0000 7f0f55e54640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f500b5c60 0x7f0f500b3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:19.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.846+0000 7f0f55e54640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f500d2550 con 0x7f0f500b5c60 2024-04-17T10:42:19.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.846+0000 7f0f55e54640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f500d23d0 con 0x7f0f50156a40 2024-04-17T10:42:19.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.846+0000 7f0f55e54640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f500cc780 con 0x7f0f500b5840 2024-04-17T10:42:19.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.846+0000 7f0f4ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f500b5c60 0x7f0f500b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:19.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.846+0000 7f0f4ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f500b5c60 0x7f0f500b3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57558/0 (socket says 172.21.15.83:57558) 2024-04-17T10:42:19.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.846+0000 7f0f4ffff640 1 -- 172.21.15.83:0/1279634899 learned_addr learned my addr 172.21.15.83:0/1279634899 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:19.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.846+0000 7f0f4f7fe640 1 --2- 172.21.15.83:0/1279634899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f50156a40 0x7f0f500c52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:19.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.846+0000 7f0f4effd640 1 --2- 172.21.15.83:0/1279634899 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f500b5840 0x7f0f500b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:19.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.846+0000 7f0f4ffff640 1 -- 172.21.15.83:0/1279634899 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f500b5840 msgr2=0x7f0f500b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:19.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.846+0000 7f0f4ffff640 1 --2- 172.21.15.83:0/1279634899 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f500b5840 0x7f0f500b2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.846+0000 7f0f4ffff640 1 -- 172.21.15.83:0/1279634899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f50156a40 msgr2=0x7f0f500c52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:19.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.846+0000 7f0f4ffff640 1 --2- 172.21.15.83:0/1279634899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f50156a40 0x7f0f500c52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:19.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.846+0000 7f0f4ffff640 1 -- 172.21.15.83:0/1279634899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f50156e20 con 0x7f0f500b5c60 2024-04-17T10:42:19.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.846+0000 7f0f4ffff640 1 --2- 172.21.15.83:0/1279634899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f500b5c60 0x7f0f500b3460 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f0f48002240 tx=0x7f0f38005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:19.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.846+0000 7f0f37fff640 1 -- 172.21.15.83:0/1279634899 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f480bb070 con 0x7f0f500b5c60 2024-04-17T10:42:19.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.846+0000 7f0f37fff640 1 -- 172.21.15.83:0/1279634899 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f480a5b20 con 0x7f0f500b5c60 2024-04-17T10:42:19.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.846+0000 7f0f55e54640 1 -- 172.21.15.83:0/1279634899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f500c9650 con 0x7f0f500b5c60 2024-04-17T10:42:19.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.846+0000 7f0f37fff640 1 -- 172.21.15.83:0/1279634899 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f480b2040 con 0x7f0f500b5c60 2024-04-17T10:42:19.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.846+0000 7f0f55e54640 1 -- 172.21.15.83:0/1279634899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f500c9c10 con 0x7f0f500b5c60 2024-04-17T10:42:19.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.846+0000 7f0f55e54640 1 -- 172.21.15.83:0/1279634899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f50074f90 con 0x7f0f500b5c60 2024-04-17T10:42:19.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.846+0000 7f0f37fff640 1 -- 172.21.15.83:0/1279634899 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0f480be070 con 0x7f0f500b5c60 2024-04-17T10:42:19.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.846+0000 7f0f37fff640 1 --2- 172.21.15.83:0/1279634899 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0f3c066f60 0x7f0f3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:19.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.846+0000 7f0f37fff640 1 -- 172.21.15.83:0/1279634899 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0f4811eaa0 con 0x7f0f500b5c60 2024-04-17T10:42:19.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.846+0000 7f0f4f7fe640 1 --2- 172.21.15.83:0/1279634899 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0f3c066f60 0x7f0f3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:19.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.846+0000 7f0f4f7fe640 1 --2- 172.21.15.83:0/1279634899 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0f3c066f60 0x7f0f3c069420 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0f4000ba30 tx=0x7f0f40008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:19.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:19.850+0000 7f0f37fff640 1 -- 172.21.15.83:0/1279634899 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0f48016b30 con 0x7f0f500b5c60 2024-04-17T10:42:20.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.006+0000 7f0f55e54640 1 -- 172.21.15.83:0/1279634899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0f50070df0 con 0x7f0f500b5c60 2024-04-17T10:42:20.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.006+0000 7f0f37fff640 1 -- 172.21.15.83:0/1279634899 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v48) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0f480ef2f0 con 0x7f0f500b5c60 2024-04-17T10:42:20.007 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-17T10:42:20.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.010+0000 7f0f55e54640 1 -- 172.21.15.83:0/1279634899 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0f3c066f60 msgr2=0x7f0f3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:20.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.010+0000 7f0f55e54640 1 --2- 172.21.15.83:0/1279634899 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0f3c066f60 0x7f0f3c069420 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0f4000ba30 tx=0x7f0f40008040 comp rx=0 tx=0).stop 2024-04-17T10:42:20.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.010+0000 7f0f55e54640 1 -- 172.21.15.83:0/1279634899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f500b5c60 msgr2=0x7f0f500b3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:20.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.010+0000 7f0f55e54640 1 --2- 172.21.15.83:0/1279634899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f500b5c60 0x7f0f500b3460 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f0f48002240 tx=0x7f0f38005840 comp rx=0 tx=0).stop 2024-04-17T10:42:20.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.010+0000 7f0f55e54640 1 -- 172.21.15.83:0/1279634899 shutdown_connections 2024-04-17T10:42:20.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.010+0000 7f0f55e54640 1 --2- 172.21.15.83:0/1279634899 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0f3c066f60 0x7f0f3c069420 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:20.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.010+0000 7f0f55e54640 1 --2- 172.21.15.83:0/1279634899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f500b5c60 0x7f0f500b3460 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:20.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.010+0000 7f0f55e54640 1 --2- 172.21.15.83:0/1279634899 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f500b5840 0x7f0f500b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:20.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.010+0000 7f0f55e54640 1 --2- 172.21.15.83:0/1279634899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f50156a40 0x7f0f500c52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:20.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.010+0000 7f0f55e54640 1 -- 172.21.15.83:0/1279634899 >> 172.21.15.83:0/1279634899 conn(0x7f0f500d1790 msgr2=0x7f0f500af740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:20.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.010+0000 7f0f55e54640 1 -- 172.21.15.83:0/1279634899 shutdown_connections 2024-04-17T10:42:20.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.010+0000 7f0f55e54640 1 -- 172.21.15.83:0/1279634899 wait complete. 2024-04-17T10:42:20.024 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2024-04-17T10:42:20.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:42:20.024 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-17T10:42:20.035 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[51595]: fuse finished with error 0 and tester_r 0 2024-04-17T10:42:20.040 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-17T10:42:20.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:42:20.040 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T10:42:20.225 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:42:20.225 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:42:20.233 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:42:20.234 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-04-17T10:42:20.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:42:20.234 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-17T10:42:20.282 INFO:teuthology.orchestra.run.smithi167.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-17T10:42:20.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:42:20.284 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:42:20.284 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:42:20.288 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16063 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:42:20.288 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-17T10:42:20.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:42:20.289 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-17T10:42:20.339 INFO:teuthology.orchestra.run.smithi083.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-17T10:42:20.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:42:20.340 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:42:20.340 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:42:20.346 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:42:20.347 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-04-17T10:42:20.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:42:20.347 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-17T10:42:20.394 INFO:teuthology.orchestra.run.smithi167.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-17T10:42:20.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:42:20.396 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:42:20.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.518+0000 7efd12049640 1 Processor -- start 2024-04-17T10:42:20.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.522+0000 7efd12049640 1 -- start start 2024-04-17T10:42:20.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.522+0000 7efd12049640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efd0c15db50 0x7efd0c070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:20.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.522+0000 7efd12049640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd0c0aedd0 0x7efd0c06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:20.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.522+0000 7efd12049640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd0c0709b0 con 0x7efd0c06dae0 2024-04-17T10:42:20.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.522+0000 7efd12049640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd0c15e440 con 0x7efd0c15db50 2024-04-17T10:42:20.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.522+0000 7efd12049640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd0c15e720 con 0x7efd0c0aedd0 2024-04-17T10:42:20.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.522+0000 7efd0b7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efd0c15db50 0x7efd0c070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:20.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.522+0000 7efd0bfff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7efd0c06dae0 0x7efd0c0cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41592/0 (socket says 172.21.15.83:41592) 2024-04-17T10:42:20.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.522+0000 7efd0bfff640 1 -- 172.21.15.83:0/2447792715 learned_addr learned my addr 172.21.15.83:0/2447792715 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:20.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.522+0000 7efd0b7fe640 1 -- 172.21.15.83:0/2447792715 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd0c0aedd0 msgr2=0x7efd0c06d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:20.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.522+0000 7efd0affd640 1 -- 172.21.15.83:0/2447792715 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3424751075 0 0) 0x7efd040014a0 con 0x7efd0c06dae0 2024-04-17T10:42:20.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.522+0000 7efd03fff640 1 --2- 172.21.15.83:0/2447792715 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd0c0aedd0 0x7efd0c06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:20.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.522+0000 7efd0b7fe640 1 --2- 172.21.15.83:0/2447792715 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd0c0aedd0 0x7efd0c06d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:20.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.522+0000 7efd0b7fe640 1 -- 172.21.15.83:0/2447792715 >> v1:172.21.15.83:6789/0 conn(0x7efd0c06dae0 legacy=0x7efd0c0cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:20.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.522+0000 7efd0b7fe640 1 -- 172.21.15.83:0/2447792715 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd0c15ea00 con 0x7efd0c15db50 2024-04-17T10:42:20.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.522+0000 7efd0b7fe640 1 --2- 172.21.15.83:0/2447792715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c15db50 0x7efd0c070470 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7efcfc0097c0 tx=0x7efcfc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edff2dc060308168 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:20.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd0affd640 1 -- 172.21.15.83:0/2447792715 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc0b2070 con 0x7efd0c15db50 2024-04-17T10:42:20.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd0affd640 1 -- 172.21.15.83:0/2447792715 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcfc0ad640 con 0x7efd0c15db50 2024-04-17T10:42:20.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd0affd640 1 -- 172.21.15.83:0/2447792715 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc0ada10 con 0x7efd0c15db50 2024-04-17T10:42:20.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd12049640 1 -- 172.21.15.83:0/2447792715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c15db50 msgr2=0x7efd0c070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:20.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd12049640 1 --2- 172.21.15.83:0/2447792715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c15db50 0x7efd0c070470 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7efcfc0097c0 tx=0x7efcfc01c4a0 comp rx=0 tx=0).stop 2024-04-17T10:42:20.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd12049640 1 -- 172.21.15.83:0/2447792715 shutdown_connections 2024-04-17T10:42:20.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd12049640 1 --2- 172.21.15.83:0/2447792715 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd0c0aedd0 0x7efd0c06d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:20.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd12049640 1 --2- 172.21.15.83:0/2447792715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c15db50 0x7efd0c070470 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:20.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd12049640 1 -- 172.21.15.83:0/2447792715 >> 172.21.15.83:0/2447792715 conn(0x7efd0c06abe0 msgr2=0x7efd0c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:20.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd12049640 1 -- 172.21.15.83:0/2447792715 shutdown_connections 2024-04-17T10:42:20.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd12049640 1 -- 172.21.15.83:0/2447792715 wait complete. 2024-04-17T10:42:20.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd12049640 1 Processor -- start 2024-04-17T10:42:20.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd12049640 1 -- start start 2024-04-17T10:42:20.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd12049640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd0c06dae0 0x7efd0c144140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:20.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd12049640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd0c0aedd0 0x7efd0c0cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:20.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd12049640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c15db50 0x7efd0c06c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:20.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd12049640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd0c1677e0 con 0x7efd0c15db50 2024-04-17T10:42:20.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd12049640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd0c167660 con 0x7efd0c06dae0 2024-04-17T10:42:20.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd12049640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd0c167960 con 0x7efd0c0aedd0 2024-04-17T10:42:20.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd0bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c15db50 0x7efd0c06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:20.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd0bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c15db50 0x7efd0c06c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57586/0 (socket says 172.21.15.83:57586) 2024-04-17T10:42:20.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd0bfff640 1 -- 172.21.15.83:0/1707234236 learned_addr learned my addr 172.21.15.83:0/1707234236 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:20.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd0b7fe640 1 --2- 172.21.15.83:0/1707234236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd0c06dae0 0x7efd0c144140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:20.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd03fff640 1 --2- 172.21.15.83:0/1707234236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd0c0aedd0 0x7efd0c0cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:20.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd0bfff640 1 -- 172.21.15.83:0/1707234236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd0c0aedd0 msgr2=0x7efd0c0cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:20.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd0bfff640 1 --2- 172.21.15.83:0/1707234236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd0c0aedd0 0x7efd0c0cdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:20.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd0bfff640 1 -- 172.21.15.83:0/1707234236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd0c06dae0 msgr2=0x7efd0c144140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:20.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd0bfff640 1 --2- 172.21.15.83:0/1707234236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd0c06dae0 0x7efd0c144140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:20.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd0bfff640 1 -- 172.21.15.83:0/1707234236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd0c0ada90 con 0x7efd0c15db50 2024-04-17T10:42:20.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd0b7fe640 1 --2- 172.21.15.83:0/1707234236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd0c06dae0 0x7efd0c144140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:20.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd0bfff640 1 --2- 172.21.15.83:0/1707234236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c15db50 0x7efd0c06c490 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7efd0400c7f0 tx=0x7efd0400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:20.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd08ff9640 1 -- 172.21.15.83:0/1707234236 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd04018070 con 0x7efd0c15db50 2024-04-17T10:42:20.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd08ff9640 1 -- 172.21.15.83:0/1707234236 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd0400a640 con 0x7efd0c15db50 2024-04-17T10:42:20.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd08ff9640 1 -- 172.21.15.83:0/1707234236 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0400aa30 con 0x7efd0c15db50 2024-04-17T10:42:20.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd12049640 1 -- 172.21.15.83:0/1707234236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd0c0add70 con 0x7efd0c15db50 2024-04-17T10:42:20.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.526+0000 7efd12049640 1 -- 172.21.15.83:0/1707234236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd0c0ae330 con 0x7efd0c15db50 2024-04-17T10:42:20.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.530+0000 7efd12049640 1 -- 172.21.15.83:0/1707234236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd0c074f90 con 0x7efd0c15db50 2024-04-17T10:42:20.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.530+0000 7efd08ff9640 1 -- 172.21.15.83:0/1707234236 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efd0401b070 con 0x7efd0c15db50 2024-04-17T10:42:20.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.530+0000 7efd08ff9640 1 --2- 172.21.15.83:0/1707234236 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efcec066f60 0x7efcec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:20.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.530+0000 7efd08ff9640 1 -- 172.21.15.83:0/1707234236 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7efd04084750 con 0x7efd0c15db50 2024-04-17T10:42:20.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.530+0000 7efd0b7fe640 1 --2- 172.21.15.83:0/1707234236 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efcec066f60 0x7efcec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:20.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.530+0000 7efd0b7fe640 1 --2- 172.21.15.83:0/1707234236 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efcec066f60 0x7efcec069420 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efcfc0af000 tx=0x7efcfc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:20.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.534+0000 7efd08ff9640 1 -- 172.21.15.83:0/1707234236 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efd0c0709b0 con 0x7efd0c15db50 2024-04-17T10:42:20.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.726+0000 7efd12049640 1 -- 172.21.15.83:0/1707234236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd0c0b27c0 con 0x7efd0c15db50 2024-04-17T10:42:20.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.726+0000 7efd08ff9640 1 -- 172.21.15.83:0/1707234236 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4939 (secure 0 0 0) 0x7efd04055680 con 0x7efd0c15db50 2024-04-17T10:42:20.726 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:20.727 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/161398126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":161398126},{"type":"v1","addr":"172.21.15.167:6833","nonce":161398126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1085380618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1085380618},{"type":"v1","addr":"172.21.15.167:6835","nonce":1085380618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/836442291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":836442291},{"type":"v1","addr":"172.21.15.83:6835","nonce":836442291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:41:37.773342+0000","modified":"2024-04-17T10:42:18.483648+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5682":{"gid":5682,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.83:6837/1894547386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1894547386},{"type":"v1","addr":"172.21.15.83:6837","nonce":1894547386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-17T10:42:20.727 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 23 2024-04-17T10:42:20.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.730+0000 7efd12049640 1 -- 172.21.15.83:0/1707234236 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efcec066f60 msgr2=0x7efcec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:20.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.730+0000 7efd12049640 1 --2- 172.21.15.83:0/1707234236 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efcec066f60 0x7efcec069420 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efcfc0af000 tx=0x7efcfc008040 comp rx=0 tx=0).stop 2024-04-17T10:42:20.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.730+0000 7efd12049640 1 -- 172.21.15.83:0/1707234236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c15db50 msgr2=0x7efd0c06c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:20.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.730+0000 7efd12049640 1 --2- 172.21.15.83:0/1707234236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c15db50 0x7efd0c06c490 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7efd0400c7f0 tx=0x7efd0400ccb0 comp rx=0 tx=0).stop 2024-04-17T10:42:20.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.730+0000 7efd12049640 1 -- 172.21.15.83:0/1707234236 shutdown_connections 2024-04-17T10:42:20.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.730+0000 7efd12049640 1 --2- 172.21.15.83:0/1707234236 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efcec066f60 0x7efcec069420 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:20.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.730+0000 7efd12049640 1 --2- 172.21.15.83:0/1707234236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c15db50 0x7efd0c06c490 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:20.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.730+0000 7efd12049640 1 --2- 172.21.15.83:0/1707234236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd0c0aedd0 0x7efd0c0cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:20.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.730+0000 7efd12049640 1 --2- 172.21.15.83:0/1707234236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd0c06dae0 0x7efd0c144140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:20.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.730+0000 7efd12049640 1 -- 172.21.15.83:0/1707234236 >> 172.21.15.83:0/1707234236 conn(0x7efd0c06abe0 msgr2=0x7efd0c0b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:20.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.730+0000 7efd12049640 1 -- 172.21.15.83:0/1707234236 shutdown_connections 2024-04-17T10:42:20.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.730+0000 7efd12049640 1 -- 172.21.15.83:0/1707234236 wait complete. 2024-04-17T10:42:20.746 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:42:20.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.866+0000 7f8228ed7640 1 Processor -- start 2024-04-17T10:42:20.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.866+0000 7f8228ed7640 1 -- start start 2024-04-17T10:42:20.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.870+0000 7f8228ed7640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f82240cfbe0 0x7f822406c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:20.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.870+0000 7f8228ed7640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f822406e320 con 0x7f8224072330 2024-04-17T10:42:20.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.870+0000 7f8228ed7640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f822406e040 con 0x7f822415ba30 2024-04-17T10:42:20.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.870+0000 7f8228ed7640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8224071ea0 con 0x7f82240cfbe0 2024-04-17T10:42:20.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.870+0000 7f8221d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8224072330 0x7f82240cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41596/0 (socket says 172.21.15.83:41596) 2024-04-17T10:42:20.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.870+0000 7f8221d9b640 1 -- 172.21.15.83:0/1817102905 learned_addr learned my addr 172.21.15.83:0/1817102905 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:20.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.870+0000 7f822159a640 1 -- 172.21.15.83:0/1817102905 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1970112953 0 0) 0x7f822406e320 con 0x7f8224072330 2024-04-17T10:42:20.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.870+0000 7f8222d9d640 1 --2- 172.21.15.83:0/1817102905 >> v2:172.21.15.167:3300/0 conn(0x7f82240cfbe0 0x7f822406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:20.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.870+0000 7f822159a640 1 -- 172.21.15.83:0/1817102905 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f820c003530 con 0x7f8224072330 2024-04-17T10:42:20.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.870+0000 7f822159a640 1 -- 172.21.15.83:0/1817102905 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 925412740 0 0) 0x7f820c003530 con 0x7f8224072330 2024-04-17T10:42:20.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.870+0000 7f822159a640 1 -- 172.21.15.83:0/1817102905 >> v2:172.21.15.167:3300/0 conn(0x7f82240cfbe0 msgr2=0x7f822406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:20.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.870+0000 7f822159a640 1 --2- 172.21.15.83:0/1817102905 >> v2:172.21.15.167:3300/0 conn(0x7f82240cfbe0 0x7f822406c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:20.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.870+0000 7f822159a640 1 -- 172.21.15.83:0/1817102905 >> v1:172.21.15.167:6789/0 conn(0x7f822415ba30 legacy=0x7f822415c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:20.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.870+0000 7f822159a640 1 -- 172.21.15.83:0/1817102905 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8224071850 con 0x7f8224072330 2024-04-17T10:42:20.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.870+0000 7f822159a640 1 -- 172.21.15.83:0/1817102905 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f82180016a0 con 0x7f8224072330 2024-04-17T10:42:20.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.870+0000 7f822159a640 1 -- 172.21.15.83:0/1817102905 >> v1:172.21.15.83:6789/0 conn(0x7f8224072330 legacy=0x7f82240cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:20.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.870+0000 7f822159a640 1 --2- 172.21.15.83:0/1817102905 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f820c0039c0 0x7f820c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:20.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.870+0000 7f822159a640 1 --2- 172.21.15.83:0/1817102905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f820c004810 0x7f820c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:20.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.870+0000 7f822159a640 1 --2- 172.21.15.83:0/1817102905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f820c0051d0 0x7f820c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:20.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.870+0000 7f822159a640 1 -- 172.21.15.83:0/1817102905 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8224071ea0 con 0x7f820c0051d0 2024-04-17T10:42:20.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.870+0000 7f822159a640 1 -- 172.21.15.83:0/1817102905 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820c00c170 con 0x7f820c004810 2024-04-17T10:42:20.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.870+0000 7f822159a640 1 -- 172.21.15.83:0/1817102905 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820c00c370 con 0x7f820c0039c0 2024-04-17T10:42:20.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.870+0000 7f8222d9d640 1 --2- 172.21.15.83:0/1817102905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f820c0051d0 0x7f820c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:20.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.870+0000 7f8221d9b640 1 --2- 172.21.15.83:0/1817102905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f820c004810 0x7f820c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:20.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.870+0000 7f8222d9d640 1 -- 172.21.15.83:0/1817102905 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f820c0039c0 msgr2=0x7f820c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:20.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.870+0000 7f8222d9d640 1 --2- 172.21.15.83:0/1817102905 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f820c0039c0 0x7f820c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:20.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.870+0000 7f8222d9d640 1 -- 172.21.15.83:0/1817102905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f820c004810 msgr2=0x7f820c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:20.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.870+0000 7f8222d9d640 1 --2- 172.21.15.83:0/1817102905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f820c004810 0x7f820c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:20.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.870+0000 7f8222d9d640 1 -- 172.21.15.83:0/1817102905 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f820c003530 con 0x7f820c0051d0 2024-04-17T10:42:20.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.870+0000 7f8221d9b640 1 --2- 172.21.15.83:0/1817102905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f820c004810 0x7f820c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:20.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.870+0000 7f8222d9d640 1 -- 172.21.15.83:0/1817102905 reap_dead start 2024-04-17T10:42:20.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f8222d9d640 1 --2- 172.21.15.83:0/1817102905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f820c0051d0 0x7f820c00b9d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f821c009fc0 tx=0x7f821c01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e7f262e4172201 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:20.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f822159a640 1 -- 172.21.15.83:0/1817102905 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c009c10 con 0x7f820c0051d0 2024-04-17T10:42:20.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f822159a640 1 -- 172.21.15.83:0/1817102905 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f821c0a5d10 con 0x7f820c0051d0 2024-04-17T10:42:20.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f822159a640 1 -- 172.21.15.83:0/1817102905 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c003be0 con 0x7f820c0051d0 2024-04-17T10:42:20.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f8228ed7640 1 -- 172.21.15.83:0/1817102905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f820c0051d0 msgr2=0x7f820c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:20.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f8228ed7640 1 --2- 172.21.15.83:0/1817102905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f820c0051d0 0x7f820c00b9d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f821c009fc0 tx=0x7f821c01ccc0 comp rx=0 tx=0).stop 2024-04-17T10:42:20.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f8228ed7640 1 -- 172.21.15.83:0/1817102905 shutdown_connections 2024-04-17T10:42:20.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f8228ed7640 1 --2- 172.21.15.83:0/1817102905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f820c0051d0 0x7f820c00b9d0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:20.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f8228ed7640 1 -- 172.21.15.83:0/1817102905 >> 172.21.15.83:0/1817102905 conn(0x7f822406abe0 msgr2=0x7f822406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:20.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f8228ed7640 1 -- 172.21.15.83:0/1817102905 shutdown_connections 2024-04-17T10:42:20.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f8228ed7640 1 -- 172.21.15.83:0/1817102905 wait complete. 2024-04-17T10:42:20.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f8228ed7640 1 Processor -- start 2024-04-17T10:42:20.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f8228ed7640 1 -- start start 2024-04-17T10:42:20.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f8228ed7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f820c0051d0 0x7f82240a9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:20.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f8228ed7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82240adc50 0x7f822415da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:20.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f8228ed7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f822415dfc0 0x7f82240afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:20.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f8228ed7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822406dca0 con 0x7f820c0051d0 2024-04-17T10:42:20.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f8228ed7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82240aeae0 con 0x7f82240adc50 2024-04-17T10:42:20.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f8228ed7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f822406de20 con 0x7f822415dfc0 2024-04-17T10:42:20.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f822259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f820c0051d0 0x7f82240a9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:20.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f822259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f820c0051d0 0x7f82240a9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57604/0 (socket says 172.21.15.83:57604) 2024-04-17T10:42:20.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f822259c640 1 -- 172.21.15.83:0/463919268 learned_addr learned my addr 172.21.15.83:0/463919268 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f8221d9b640 1 --2- 172.21.15.83:0/463919268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82240adc50 0x7f822415da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f8222d9d640 1 --2- 172.21.15.83:0/463919268 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f822415dfc0 0x7f82240afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f822259c640 1 -- 172.21.15.83:0/463919268 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f822415dfc0 msgr2=0x7f82240afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f822259c640 1 --2- 172.21.15.83:0/463919268 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f822415dfc0 0x7f82240afcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f822259c640 1 -- 172.21.15.83:0/463919268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82240adc50 msgr2=0x7f822415da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f822259c640 1 --2- 172.21.15.83:0/463919268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82240adc50 0x7f822415da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f822259c640 1 -- 172.21.15.83:0/463919268 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82241f12b0 con 0x7f820c0051d0 2024-04-17T10:42:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f822259c640 1 --2- 172.21.15.83:0/463919268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f820c0051d0 0x7f82240a9430 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f821400c500 tx=0x7f821400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f8212ffd640 1 -- 172.21.15.83:0/463919268 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82140052d0 con 0x7f820c0051d0 2024-04-17T10:42:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f8212ffd640 1 -- 172.21.15.83:0/463919268 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8214005470 con 0x7f820c0051d0 2024-04-17T10:42:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f8228ed7640 1 -- 172.21.15.83:0/463919268 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82241f1590 con 0x7f820c0051d0 2024-04-17T10:42:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f8212ffd640 1 -- 172.21.15.83:0/463919268 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8214012870 con 0x7f820c0051d0 2024-04-17T10:42:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.874+0000 7f8228ed7640 1 -- 172.21.15.83:0/463919268 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82241f1b50 con 0x7f820c0051d0 2024-04-17T10:42:20.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.878+0000 7f8228ed7640 1 -- 172.21.15.83:0/463919268 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8224074f90 con 0x7f820c0051d0 2024-04-17T10:42:20.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.878+0000 7f8212ffd640 1 -- 172.21.15.83:0/463919268 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8214012a10 con 0x7f820c0051d0 2024-04-17T10:42:20.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.878+0000 7f8212ffd640 1 --2- 172.21.15.83:0/463919268 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8204066f90 0x7f8204069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:20.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.878+0000 7f8212ffd640 1 -- 172.21.15.83:0/463919268 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8214084c60 con 0x7f820c0051d0 2024-04-17T10:42:20.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.878+0000 7f8221d9b640 1 --2- 172.21.15.83:0/463919268 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8204066f90 0x7f8204069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:20.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.878+0000 7f8221d9b640 1 --2- 172.21.15.83:0/463919268 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8204066f90 0x7f8204069450 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8218000fe0 tx=0x7f82180014e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:20.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:20.882+0000 7f8212ffd640 1 -- 172.21.15.83:0/463919268 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8214055cc0 con 0x7f820c0051d0 2024-04-17T10:42:21.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.038+0000 7f8228ed7640 1 -- 172.21.15.83:0/463919268 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8224074480 con 0x7f820c0051d0 2024-04-17T10:42:21.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.042+0000 7f8212ffd640 1 -- 172.21.15.83:0/463919268 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16693 (secure 0 0 0) 0x7f82140555c0 con 0x7f820c0051d0 2024-04-17T10:42:21.040 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:21.040 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":48,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:41:54.319551+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:41:36.031388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-17T10:41:36.783410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.13","mappings":[{"from":0,"to":4}]},{"pgid":"7.16","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:42:21.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.042+0000 7f8228ed7640 1 -- 172.21.15.83:0/463919268 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8204066f90 msgr2=0x7f8204069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:21.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.042+0000 7f8228ed7640 1 --2- 172.21.15.83:0/463919268 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8204066f90 0x7f8204069450 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8218000fe0 tx=0x7f82180014e0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.042+0000 7f8228ed7640 1 -- 172.21.15.83:0/463919268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f820c0051d0 msgr2=0x7f82240a9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:21.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.042+0000 7f8228ed7640 1 --2- 172.21.15.83:0/463919268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f820c0051d0 0x7f82240a9430 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f821400c500 tx=0x7f821400c9c0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.042+0000 7f8228ed7640 1 -- 172.21.15.83:0/463919268 shutdown_connections 2024-04-17T10:42:21.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.042+0000 7f8228ed7640 1 --2- 172.21.15.83:0/463919268 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8204066f90 0x7f8204069450 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.042+0000 7f8228ed7640 1 --2- 172.21.15.83:0/463919268 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f822415dfc0 0x7f82240afcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.042+0000 7f8228ed7640 1 --2- 172.21.15.83:0/463919268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82240adc50 0x7f822415da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.042+0000 7f8228ed7640 1 --2- 172.21.15.83:0/463919268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f820c0051d0 0x7f82240a9430 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.042+0000 7f8228ed7640 1 -- 172.21.15.83:0/463919268 >> 172.21.15.83:0/463919268 conn(0x7f822406abe0 msgr2=0x7f82240b1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:21.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.042+0000 7f8228ed7640 1 -- 172.21.15.83:0/463919268 shutdown_connections 2024-04-17T10:42:21.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.042+0000 7f8228ed7640 1 -- 172.21.15.83:0/463919268 wait complete. 2024-04-17T10:42:21.058 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:41:36.031388+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:41:36.783410+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:42:21.059 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:42:21.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.182+0000 7f2651e58640 1 Processor -- start 2024-04-17T10:42:21.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.182+0000 7f2651e58640 1 -- start start 2024-04-17T10:42:21.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.186+0000 7f2651e58640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f264c15b6e0 0x7f264c071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:21.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.186+0000 7f2651e58640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f264c15bac0 0x7f264c0cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:21.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.186+0000 7f2651e58640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f264c06c1f0 con 0x7f264c070470 2024-04-17T10:42:21.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.186+0000 7f2651e58640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f264c0cdd30 con 0x7f264c15bac0 2024-04-17T10:42:21.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.186+0000 7f2651e58640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f264c070a10 con 0x7f264c15b6e0 2024-04-17T10:42:21.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.186+0000 7f264affd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f264c15bac0 0x7f264c0cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:21.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.186+0000 7f264bfff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f264c070470 0x7f264c06ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41598/0 (socket says 172.21.15.83:41598) 2024-04-17T10:42:21.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.186+0000 7f264bfff640 1 -- 172.21.15.83:0/3833629945 learned_addr learned my addr 172.21.15.83:0/3833629945 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:21.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.186+0000 7f264b7fe640 1 --2- 172.21.15.83:0/3833629945 >> v2:172.21.15.167:3300/0 conn(0x7f264c15b6e0 0x7f264c071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:21.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.186+0000 7f264affd640 1 -- 172.21.15.83:0/3833629945 >> v2:172.21.15.167:3300/0 conn(0x7f264c15b6e0 msgr2=0x7f264c071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:21.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.186+0000 7f264affd640 1 --2- 172.21.15.83:0/3833629945 >> v2:172.21.15.167:3300/0 conn(0x7f264c15b6e0 0x7f264c071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.186+0000 7f264affd640 1 -- 172.21.15.83:0/3833629945 >> v1:172.21.15.83:6789/0 conn(0x7f264c070470 legacy=0x7f264c06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:21.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.186+0000 7f264affd640 1 -- 172.21.15.83:0/3833629945 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f264c070cf0 con 0x7f264c15bac0 2024-04-17T10:42:21.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.186+0000 7f264a7fc640 1 -- 172.21.15.83:0/3833629945 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3481558102 0 0) 0x7f264c06c1f0 con 0x7f264c070470 2024-04-17T10:42:21.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.186+0000 7f264b7fe640 1 --2- 172.21.15.83:0/3833629945 >> v2:172.21.15.167:3300/0 conn(0x7f264c15b6e0 0x7f264c071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:21.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.186+0000 7f264affd640 1 --2- 172.21.15.83:0/3833629945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f264c15bac0 0x7f264c0cd7f0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f264c0728a0 tx=0x7f264001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b079a809442b79c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:21.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.186+0000 7f264a7fc640 1 -- 172.21.15.83:0/3833629945 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26400b0070 con 0x7f264c15bac0 2024-04-17T10:42:21.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.186+0000 7f264a7fc640 1 -- 172.21.15.83:0/3833629945 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26400ab600 con 0x7f264c15bac0 2024-04-17T10:42:21.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.186+0000 7f264a7fc640 1 -- 172.21.15.83:0/3833629945 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26400ab9d0 con 0x7f264c15bac0 2024-04-17T10:42:21.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.186+0000 7f2651e58640 1 -- 172.21.15.83:0/3833629945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f264c15bac0 msgr2=0x7f264c0cd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:21.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.186+0000 7f2651e58640 1 --2- 172.21.15.83:0/3833629945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f264c15bac0 0x7f264c0cd7f0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f264c0728a0 tx=0x7f264001c480 comp rx=0 tx=0).stop 2024-04-17T10:42:21.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.190+0000 7f2651e58640 1 -- 172.21.15.83:0/3833629945 shutdown_connections 2024-04-17T10:42:21.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.190+0000 7f2651e58640 1 --2- 172.21.15.83:0/3833629945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f264c15bac0 0x7f264c0cd7f0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.190+0000 7f2651e58640 1 --2- 172.21.15.83:0/3833629945 >> v2:172.21.15.167:3300/0 conn(0x7f264c15b6e0 0x7f264c071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.190+0000 7f2651e58640 1 -- 172.21.15.83:0/3833629945 >> 172.21.15.83:0/3833629945 conn(0x7f264c0b15b0 msgr2=0x7f264c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:21.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.190+0000 7f2651e58640 1 -- 172.21.15.83:0/3833629945 shutdown_connections 2024-04-17T10:42:21.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.190+0000 7f2651e58640 1 -- 172.21.15.83:0/3833629945 wait complete. 2024-04-17T10:42:21.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.190+0000 7f2651e58640 1 Processor -- start 2024-04-17T10:42:21.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.190+0000 7f2651e58640 1 -- start start 2024-04-17T10:42:21.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.190+0000 7f2651e58640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f264c070470 0x7f264c145810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:21.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.190+0000 7f2651e58640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f264c15b6e0 0x7f264c1485c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:21.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.190+0000 7f2651e58640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f264c15bac0 0x7f264c152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:21.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.190+0000 7f2651e58640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c06cee0 con 0x7f264c15b6e0 2024-04-17T10:42:21.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.190+0000 7f2651e58640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c06d060 con 0x7f264c15bac0 2024-04-17T10:42:21.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.190+0000 7f2651e58640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f264c06d1e0 con 0x7f264c070470 2024-04-17T10:42:21.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.190+0000 7f264affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f264c15b6e0 0x7f264c1485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:21.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.190+0000 7f264affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f264c15b6e0 0x7f264c1485c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57630/0 (socket says 172.21.15.83:57630) 2024-04-17T10:42:21.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.190+0000 7f264affd640 1 -- 172.21.15.83:0/2290737285 learned_addr learned my addr 172.21.15.83:0/2290737285 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:21.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.190+0000 7f264bfff640 1 --2- 172.21.15.83:0/2290737285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f264c15bac0 0x7f264c152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:21.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.190+0000 7f264affd640 1 -- 172.21.15.83:0/2290737285 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f264c070470 msgr2=0x7f264c145810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:21.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.190+0000 7f264affd640 1 --2- 172.21.15.83:0/2290737285 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f264c070470 0x7f264c145810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.190+0000 7f264affd640 1 -- 172.21.15.83:0/2290737285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f264c15bac0 msgr2=0x7f264c152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:21.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.190+0000 7f264affd640 1 --2- 172.21.15.83:0/2290737285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f264c15bac0 0x7f264c152dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.190+0000 7f264affd640 1 -- 172.21.15.83:0/2290737285 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f264c146220 con 0x7f264c15b6e0 2024-04-17T10:42:21.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.190+0000 7f264affd640 1 --2- 172.21.15.83:0/2290737285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f264c15b6e0 0x7f264c1485c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f264001a0b0 tx=0x7f2640019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:21.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.190+0000 7f2633fff640 1 -- 172.21.15.83:0/2290737285 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26400b0040 con 0x7f264c15b6e0 2024-04-17T10:42:21.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.190+0000 7f2633fff640 1 -- 172.21.15.83:0/2290737285 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26400bd070 con 0x7f264c15b6e0 2024-04-17T10:42:21.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.190+0000 7f2651e58640 1 -- 172.21.15.83:0/2290737285 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f264c146500 con 0x7f264c15b6e0 2024-04-17T10:42:21.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.190+0000 7f2633fff640 1 -- 172.21.15.83:0/2290737285 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26400b58a0 con 0x7f264c15b6e0 2024-04-17T10:42:21.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.190+0000 7f2651e58640 1 -- 172.21.15.83:0/2290737285 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f264c146ac0 con 0x7f264c15b6e0 2024-04-17T10:42:21.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.190+0000 7f2651e58640 1 -- 172.21.15.83:0/2290737285 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f264c06b7b0 con 0x7f264c15b6e0 2024-04-17T10:42:21.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.194+0000 7f2633fff640 1 -- 172.21.15.83:0/2290737285 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f26400b3070 con 0x7f264c15b6e0 2024-04-17T10:42:21.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.194+0000 7f2633fff640 1 --2- 172.21.15.83:0/2290737285 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2638066f60 0x7f2638069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:21.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.194+0000 7f2633fff640 1 -- 172.21.15.83:0/2290737285 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f264011f480 con 0x7f264c15b6e0 2024-04-17T10:42:21.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.194+0000 7f264b7fe640 1 --2- 172.21.15.83:0/2290737285 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2638066f60 0x7f2638069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:21.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.194+0000 7f264b7fe640 1 --2- 172.21.15.83:0/2290737285 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2638066f60 0x7f2638069420 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f263c008ea0 tx=0x7f263c0083d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:21.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.198+0000 7f2633fff640 1 -- 172.21.15.83:0/2290737285 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2640016070 con 0x7f264c15b6e0 2024-04-17T10:42:21.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.358+0000 7f2651e58640 1 -- 172.21.15.83:0/2290737285 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f264c074f90 con 0x7f264c15b6e0 2024-04-17T10:42:21.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.358+0000 7f2633fff640 1 -- 172.21.15.83:0/2290737285 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) v1 ==== 93+0+10 (secure 0 0 0) 0x7f26400f0450 con 0x7f264c15b6e0 2024-04-17T10:42:21.356 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:42:21.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.358+0000 7f2651e58640 1 -- 172.21.15.83:0/2290737285 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2638066f60 msgr2=0x7f2638069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:21.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.358+0000 7f2651e58640 1 --2- 172.21.15.83:0/2290737285 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2638066f60 0x7f2638069420 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f263c008ea0 tx=0x7f263c0083d0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.358+0000 7f2651e58640 1 -- 172.21.15.83:0/2290737285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f264c15b6e0 msgr2=0x7f264c1485c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:21.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.358+0000 7f2651e58640 1 --2- 172.21.15.83:0/2290737285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f264c15b6e0 0x7f264c1485c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f264001a0b0 tx=0x7f2640019220 comp rx=0 tx=0).stop 2024-04-17T10:42:21.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.358+0000 7f2651e58640 1 -- 172.21.15.83:0/2290737285 shutdown_connections 2024-04-17T10:42:21.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.358+0000 7f2651e58640 1 --2- 172.21.15.83:0/2290737285 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2638066f60 0x7f2638069420 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.358+0000 7f2651e58640 1 --2- 172.21.15.83:0/2290737285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f264c15bac0 0x7f264c152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.358+0000 7f2651e58640 1 --2- 172.21.15.83:0/2290737285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f264c15b6e0 0x7f264c1485c0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.358+0000 7f2651e58640 1 --2- 172.21.15.83:0/2290737285 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f264c070470 0x7f264c145810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.358+0000 7f2651e58640 1 -- 172.21.15.83:0/2290737285 >> 172.21.15.83:0/2290737285 conn(0x7f264c0b15b0 msgr2=0x7f264c0af4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:21.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.358+0000 7f2651e58640 1 -- 172.21.15.83:0/2290737285 shutdown_connections 2024-04-17T10:42:21.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.358+0000 7f2651e58640 1 -- 172.21.15.83:0/2290737285 wait complete. 2024-04-17T10:42:21.375 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:42:21.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.498+0000 7fad8fef3640 1 Processor -- start 2024-04-17T10:42:21.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.498+0000 7fad8fef3640 1 -- start start 2024-04-17T10:42:21.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.502+0000 7fad8fef3640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fad8815b6e0 0x7fad88071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:21.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.502+0000 7fad8fef3640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad8806c1f0 con 0x7fad88070470 2024-04-17T10:42:21.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.502+0000 7fad8fef3640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad88070a10 con 0x7fad8815bac0 2024-04-17T10:42:21.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.502+0000 7fad8fef3640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad88070cf0 con 0x7fad8815b6e0 2024-04-17T10:42:21.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.502+0000 7fad8dc8f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fad8815b6e0 0x7fad88071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:21.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.502+0000 7fad8e490640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fad88070470 0x7fad8806ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41608/0 (socket says 172.21.15.83:41608) 2024-04-17T10:42:21.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.502+0000 7fad8e490640 1 -- 172.21.15.83:0/1283176277 learned_addr learned my addr 172.21.15.83:0/1283176277 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:21.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.502+0000 7fad8dc8f640 1 -- 172.21.15.83:0/1283176277 >> v1:172.21.15.167:6789/0 conn(0x7fad8815bac0 legacy=0x7fad880cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:21.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.502+0000 7fad8dc8f640 1 -- 172.21.15.83:0/1283176277 >> v1:172.21.15.83:6789/0 conn(0x7fad88070470 legacy=0x7fad8806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:21.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.502+0000 7fad8dc8f640 1 -- 172.21.15.83:0/1283176277 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad88070fd0 con 0x7fad8815b6e0 2024-04-17T10:42:21.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.502+0000 7fad8dc8f640 1 --2- 172.21.15.83:0/1283176277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad8815b6e0 0x7fad88071cc0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fad840097c0 tx=0x7fad8401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe3f2bf4d1c4eac0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:21.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.502+0000 7fad8cc8d640 1 -- 172.21.15.83:0/1283176277 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad840b0070 con 0x7fad8815b6e0 2024-04-17T10:42:21.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.502+0000 7fad8cc8d640 1 -- 172.21.15.83:0/1283176277 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad840ab600 con 0x7fad8815b6e0 2024-04-17T10:42:21.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.502+0000 7fad8cc8d640 1 -- 172.21.15.83:0/1283176277 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad840ab9d0 con 0x7fad8815b6e0 2024-04-17T10:42:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.506+0000 7fad8fef3640 1 -- 172.21.15.83:0/1283176277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad8815b6e0 msgr2=0x7fad88071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.506+0000 7fad8fef3640 1 --2- 172.21.15.83:0/1283176277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad8815b6e0 0x7fad88071cc0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fad840097c0 tx=0x7fad8401c460 comp rx=0 tx=0).stop 2024-04-17T10:42:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.506+0000 7fad8fef3640 1 -- 172.21.15.83:0/1283176277 shutdown_connections 2024-04-17T10:42:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.506+0000 7fad8fef3640 1 --2- 172.21.15.83:0/1283176277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad8815b6e0 0x7fad88071cc0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.506+0000 7fad8fef3640 1 -- 172.21.15.83:0/1283176277 >> 172.21.15.83:0/1283176277 conn(0x7fad880b15b0 msgr2=0x7fad880b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.506+0000 7fad8fef3640 1 -- 172.21.15.83:0/1283176277 shutdown_connections 2024-04-17T10:42:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.506+0000 7fad8fef3640 1 -- 172.21.15.83:0/1283176277 wait complete. 2024-04-17T10:42:21.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.506+0000 7fad8fef3640 1 Processor -- start 2024-04-17T10:42:21.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.506+0000 7fad8fef3640 1 -- start start 2024-04-17T10:42:21.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.506+0000 7fad8fef3640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad88070470 0x7fad8814c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:21.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.506+0000 7fad8fef3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad8815b6e0 0x7fad88151a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:21.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.506+0000 7fad8fef3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad8815bac0 0x7fad88151f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:21.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.506+0000 7fad8fef3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8806cee0 con 0x7fad8815b6e0 2024-04-17T10:42:21.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.506+0000 7fad8fef3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8806d060 con 0x7fad8815bac0 2024-04-17T10:42:21.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.506+0000 7fad8fef3640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad8806d1e0 con 0x7fad88070470 2024-04-17T10:42:21.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.506+0000 7fad8d48e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad8815b6e0 0x7fad88151a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:21.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.506+0000 7fad8d48e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad8815b6e0 0x7fad88151a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57648/0 (socket says 172.21.15.83:57648) 2024-04-17T10:42:21.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.506+0000 7fad8d48e640 1 -- 172.21.15.83:0/3660165900 learned_addr learned my addr 172.21.15.83:0/3660165900 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:21.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.506+0000 7fad8dc8f640 1 --2- 172.21.15.83:0/3660165900 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad88070470 0x7fad8814c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:21.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.506+0000 7fad8e490640 1 --2- 172.21.15.83:0/3660165900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad8815bac0 0x7fad88151f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:21.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.506+0000 7fad8d48e640 1 -- 172.21.15.83:0/3660165900 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad88070470 msgr2=0x7fad8814c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:21.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.506+0000 7fad8d48e640 1 --2- 172.21.15.83:0/3660165900 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad88070470 0x7fad8814c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.506+0000 7fad8d48e640 1 -- 172.21.15.83:0/3660165900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad8815bac0 msgr2=0x7fad88151f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:21.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.506+0000 7fad8d48e640 1 --2- 172.21.15.83:0/3660165900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad8815bac0 0x7fad88151f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.506+0000 7fad8d48e640 1 -- 172.21.15.83:0/3660165900 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8815f720 con 0x7fad8815b6e0 2024-04-17T10:42:21.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.506+0000 7fad8dc8f640 1 --2- 172.21.15.83:0/3660165900 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad88070470 0x7fad8814c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:21.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.510+0000 7fad8d48e640 1 --2- 172.21.15.83:0/3660165900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad8815b6e0 0x7fad88151a00 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fad7800c450 tx=0x7fad7800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:21.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.510+0000 7fad7e7fc640 1 -- 172.21.15.83:0/3660165900 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad78012830 con 0x7fad8815b6e0 2024-04-17T10:42:21.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.510+0000 7fad7e7fc640 1 -- 172.21.15.83:0/3660165900 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad78017070 con 0x7fad8815b6e0 2024-04-17T10:42:21.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.510+0000 7fad7e7fc640 1 -- 172.21.15.83:0/3660165900 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad78012c30 con 0x7fad8815b6e0 2024-04-17T10:42:21.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.510+0000 7fad8fef3640 1 -- 172.21.15.83:0/3660165900 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad8815fa00 con 0x7fad8815b6e0 2024-04-17T10:42:21.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.510+0000 7fad8fef3640 1 -- 172.21.15.83:0/3660165900 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad8815ffc0 con 0x7fad8815b6e0 2024-04-17T10:42:21.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.510+0000 7fad8fef3640 1 -- 172.21.15.83:0/3660165900 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad88074f90 con 0x7fad8815b6e0 2024-04-17T10:42:21.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.510+0000 7fad7e7fc640 1 -- 172.21.15.83:0/3660165900 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fad7801c460 con 0x7fad8815b6e0 2024-04-17T10:42:21.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.510+0000 7fad7e7fc640 1 --2- 172.21.15.83:0/3660165900 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fad6c066f60 0x7fad6c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:21.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.510+0000 7fad7e7fc640 1 -- 172.21.15.83:0/3660165900 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fad78085900 con 0x7fad8815b6e0 2024-04-17T10:42:21.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.510+0000 7fad8dc8f640 1 --2- 172.21.15.83:0/3660165900 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fad6c066f60 0x7fad6c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:21.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.510+0000 7fad8dc8f640 1 --2- 172.21.15.83:0/3660165900 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fad6c066f60 0x7fad6c069420 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fad840ad000 tx=0x7fad84008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:21.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.514+0000 7fad7e7fc640 1 -- 172.21.15.83:0/3660165900 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fad780569b0 con 0x7fad8815b6e0 2024-04-17T10:42:21.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.670+0000 7fad8fef3640 1 -- 172.21.15.83:0/3660165900 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fad8814f9f0 con 0x7fad8815b6e0 2024-04-17T10:42:21.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.674+0000 7fad7e7fc640 1 -- 172.21.15.83:0/3660165900 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v48) v1 ==== 104+0+11 (secure 0 0 0) 0x7fad780562b0 con 0x7fad8815b6e0 2024-04-17T10:42:21.674 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:42:21.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.674+0000 7fad8fef3640 1 -- 172.21.15.83:0/3660165900 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fad6c066f60 msgr2=0x7fad6c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:21.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.674+0000 7fad8fef3640 1 --2- 172.21.15.83:0/3660165900 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fad6c066f60 0x7fad6c069420 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fad840ad000 tx=0x7fad84008040 comp rx=0 tx=0).stop 2024-04-17T10:42:21.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.674+0000 7fad8fef3640 1 -- 172.21.15.83:0/3660165900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad8815b6e0 msgr2=0x7fad88151a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:21.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.674+0000 7fad8fef3640 1 --2- 172.21.15.83:0/3660165900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad8815b6e0 0x7fad88151a00 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fad7800c450 tx=0x7fad7800c910 comp rx=0 tx=0).stop 2024-04-17T10:42:21.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.674+0000 7fad8fef3640 1 -- 172.21.15.83:0/3660165900 shutdown_connections 2024-04-17T10:42:21.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.674+0000 7fad8fef3640 1 --2- 172.21.15.83:0/3660165900 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fad6c066f60 0x7fad6c069420 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.674+0000 7fad8fef3640 1 --2- 172.21.15.83:0/3660165900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad8815bac0 0x7fad88151f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.674+0000 7fad8fef3640 1 --2- 172.21.15.83:0/3660165900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad8815b6e0 0x7fad88151a00 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.674+0000 7fad8fef3640 1 --2- 172.21.15.83:0/3660165900 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad88070470 0x7fad8814c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.674+0000 7fad8fef3640 1 -- 172.21.15.83:0/3660165900 >> 172.21.15.83:0/3660165900 conn(0x7fad880b15b0 msgr2=0x7fad880af350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:21.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.674+0000 7fad8fef3640 1 -- 172.21.15.83:0/3660165900 shutdown_connections 2024-04-17T10:42:21.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.674+0000 7fad8fef3640 1 -- 172.21.15.83:0/3660165900 wait complete. 2024-04-17T10:42:21.691 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:42:21.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.814+0000 7f01724fe640 1 Processor -- start 2024-04-17T10:42:21.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.814+0000 7f01724fe640 1 -- start start 2024-04-17T10:42:21.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.818+0000 7f01724fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f016c06b520 0x7f016c0b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:21.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.818+0000 7f01724fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f016c0b4040 0x7f016c06cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:21.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.818+0000 7f01724fe640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f016c15bd70 con 0x7f016c154de0 2024-04-17T10:42:21.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.818+0000 7f01724fe640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f016c0b4870 con 0x7f016c0b4040 2024-04-17T10:42:21.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.818+0000 7f01724fe640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016c15bf60 con 0x7f016c06b520 2024-04-17T10:42:21.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.818+0000 7f0170a9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f016c0b4040 0x7f016c06cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:21.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.818+0000 7f0170a9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f016c0b4040 0x7f016c06cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57658/0 (socket says 172.21.15.83:57658) 2024-04-17T10:42:21.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.818+0000 7f0170a9b640 1 -- 172.21.15.83:0/4082951845 learned_addr learned my addr 172.21.15.83:0/4082951845 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:21.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.818+0000 7f016b7fe640 1 --2- 172.21.15.83:0/4082951845 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f016c06b520 0x7f016c0b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:21.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.818+0000 7f0170a9b640 1 -- 172.21.15.83:0/4082951845 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f016c06b520 msgr2=0x7f016c0b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:21.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.818+0000 7f0170a9b640 1 --2- 172.21.15.83:0/4082951845 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f016c06b520 0x7f016c0b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.818+0000 7f0170a9b640 1 -- 172.21.15.83:0/4082951845 >> v1:172.21.15.167:6789/0 conn(0x7f016c154de0 legacy=0x7f016c06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:21.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.818+0000 7f0170a9b640 1 -- 172.21.15.83:0/4082951845 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016c15afb0 con 0x7f016c0b4040 2024-04-17T10:42:21.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.818+0000 7f0170a9b640 1 --2- 172.21.15.83:0/4082951845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f016c0b4040 0x7f016c06cbe0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0164009e50 tx=0x7f016401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca51739429c88c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:21.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f016affd640 1 -- 172.21.15.83:0/4082951845 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01640a5760 con 0x7f016c0b4040 2024-04-17T10:42:21.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f016affd640 1 -- 172.21.15.83:0/4082951845 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01640a5900 con 0x7f016c0b4040 2024-04-17T10:42:21.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f016affd640 1 -- 172.21.15.83:0/4082951845 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01640a5cd0 con 0x7f016c0b4040 2024-04-17T10:42:21.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f01724fe640 1 -- 172.21.15.83:0/4082951845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f016c0b4040 msgr2=0x7f016c06cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:21.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f01724fe640 1 --2- 172.21.15.83:0/4082951845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f016c0b4040 0x7f016c06cbe0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0164009e50 tx=0x7f016401c9f0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f01724fe640 1 -- 172.21.15.83:0/4082951845 shutdown_connections 2024-04-17T10:42:21.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f01724fe640 1 --2- 172.21.15.83:0/4082951845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f016c0b4040 0x7f016c06cbe0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f01724fe640 1 --2- 172.21.15.83:0/4082951845 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f016c06b520 0x7f016c0b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f01724fe640 1 -- 172.21.15.83:0/4082951845 >> 172.21.15.83:0/4082951845 conn(0x7f016c0b15b0 msgr2=0x7f016c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:21.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f01724fe640 1 -- 172.21.15.83:0/4082951845 shutdown_connections 2024-04-17T10:42:21.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f01724fe640 1 -- 172.21.15.83:0/4082951845 wait complete. 2024-04-17T10:42:21.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f01724fe640 1 Processor -- start 2024-04-17T10:42:21.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f01724fe640 1 -- start start 2024-04-17T10:42:21.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f01724fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f016c06b520 0x7f016c14b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:21.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f01724fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f016c0b4040 0x7f016c143a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:21.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f01724fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f016c154de0 0x7f016c0ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:21.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f01724fe640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016c0b44f0 con 0x7f016c06b520 2024-04-17T10:42:21.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f01724fe640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016c1551c0 con 0x7f016c154de0 2024-04-17T10:42:21.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f01724fe640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016c070470 con 0x7f016c0b4040 2024-04-17T10:42:21.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f016bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f016c06b520 0x7f016c14b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:21.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f016bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f016c06b520 0x7f016c14b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57660/0 (socket says 172.21.15.83:57660) 2024-04-17T10:42:21.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f016bfff640 1 -- 172.21.15.83:0/2177442197 learned_addr learned my addr 172.21.15.83:0/2177442197 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:21.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f0170a9b640 1 --2- 172.21.15.83:0/2177442197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f016c154de0 0x7f016c0ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:21.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f016b7fe640 1 --2- 172.21.15.83:0/2177442197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f016c0b4040 0x7f016c143a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:21.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f016bfff640 1 -- 172.21.15.83:0/2177442197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f016c0b4040 msgr2=0x7f016c143a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:21.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f016bfff640 1 --2- 172.21.15.83:0/2177442197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f016c0b4040 0x7f016c143a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f016bfff640 1 -- 172.21.15.83:0/2177442197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f016c154de0 msgr2=0x7f016c0ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:21.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f016bfff640 1 --2- 172.21.15.83:0/2177442197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f016c154de0 0x7f016c0ccf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f016bfff640 1 -- 172.21.15.83:0/2177442197 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016c14be10 con 0x7f016c06b520 2024-04-17T10:42:21.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f016b7fe640 1 --2- 172.21.15.83:0/2177442197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f016c0b4040 0x7f016c143a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:21.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f016bfff640 1 --2- 172.21.15.83:0/2177442197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f016c06b520 0x7f016c14b710 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f015c00b410 tx=0x7f015c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:21.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f0168ff9640 1 -- 172.21.15.83:0/2177442197 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015c015030 con 0x7f016c06b520 2024-04-17T10:42:21.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f0168ff9640 1 -- 172.21.15.83:0/2177442197 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f015c004020 con 0x7f016c06b520 2024-04-17T10:42:21.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f01724fe640 1 -- 172.21.15.83:0/2177442197 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f016c14c0f0 con 0x7f016c06b520 2024-04-17T10:42:21.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f0168ff9640 1 -- 172.21.15.83:0/2177442197 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015c004450 con 0x7f016c06b520 2024-04-17T10:42:21.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.822+0000 7f01724fe640 1 -- 172.21.15.83:0/2177442197 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f016c14c6b0 con 0x7f016c06b520 2024-04-17T10:42:21.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.826+0000 7f01724fe640 1 -- 172.21.15.83:0/2177442197 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f016c068ff0 con 0x7f016c06b520 2024-04-17T10:42:21.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.826+0000 7f0168ff9640 1 -- 172.21.15.83:0/2177442197 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f015c018070 con 0x7f016c06b520 2024-04-17T10:42:21.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.826+0000 7f0168ff9640 1 --2- 172.21.15.83:0/2177442197 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0158066f90 0x7f0158069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:21.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.826+0000 7f0168ff9640 1 -- 172.21.15.83:0/2177442197 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f015c084c50 con 0x7f016c06b520 2024-04-17T10:42:21.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.826+0000 7f016b7fe640 1 --2- 172.21.15.83:0/2177442197 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0158066f90 0x7f0158069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:21.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.826+0000 7f016b7fe640 1 --2- 172.21.15.83:0/2177442197 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0158066f90 0x7f0158069450 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f016c15ad70 tx=0x7f0160008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:21.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.830+0000 7f0168ff9640 1 -- 172.21.15.83:0/2177442197 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f015c055b80 con 0x7f016c06b520 2024-04-17T10:42:21.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.990+0000 7f01724fe640 1 -- 172.21.15.83:0/2177442197 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f016c074f90 con 0x7f016c06b520 2024-04-17T10:42:21.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.990+0000 7f0168ff9640 1 -- 172.21.15.83:0/2177442197 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v48) v1 ==== 100+0+11 (secure 0 0 0) 0x7f015c055480 con 0x7f016c06b520 2024-04-17T10:42:21.989 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:42:21.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.990+0000 7f01724fe640 1 -- 172.21.15.83:0/2177442197 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0158066f90 msgr2=0x7f0158069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:21.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.990+0000 7f01724fe640 1 --2- 172.21.15.83:0/2177442197 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0158066f90 0x7f0158069450 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f016c15ad70 tx=0x7f0160008040 comp rx=0 tx=0).stop 2024-04-17T10:42:21.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.990+0000 7f01724fe640 1 -- 172.21.15.83:0/2177442197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f016c06b520 msgr2=0x7f016c14b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:21.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.990+0000 7f01724fe640 1 --2- 172.21.15.83:0/2177442197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f016c06b520 0x7f016c14b710 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f015c00b410 tx=0x7f015c00b8d0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.994+0000 7f01724fe640 1 -- 172.21.15.83:0/2177442197 shutdown_connections 2024-04-17T10:42:21.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.994+0000 7f01724fe640 1 --2- 172.21.15.83:0/2177442197 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0158066f90 0x7f0158069450 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.994+0000 7f01724fe640 1 --2- 172.21.15.83:0/2177442197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f016c154de0 0x7f016c0ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.994+0000 7f01724fe640 1 --2- 172.21.15.83:0/2177442197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f016c0b4040 0x7f016c143a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.994+0000 7f01724fe640 1 --2- 172.21.15.83:0/2177442197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f016c06b520 0x7f016c14b710 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:21.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.994+0000 7f01724fe640 1 -- 172.21.15.83:0/2177442197 >> 172.21.15.83:0/2177442197 conn(0x7f016c0b15b0 msgr2=0x7f016c071d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:21.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.994+0000 7f01724fe640 1 -- 172.21.15.83:0/2177442197 shutdown_connections 2024-04-17T10:42:21.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:21.994+0000 7f01724fe640 1 -- 172.21.15.83:0/2177442197 wait complete. 2024-04-17T10:42:22.008 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:42:22.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.122+0000 7f3bf4ef4640 1 Processor -- start 2024-04-17T10:42:22.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.122+0000 7f3bf4ef4640 1 -- start start 2024-04-17T10:42:22.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.122+0000 7f3bf4ef4640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3bf015db50 0x7f3bf0070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:22.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.122+0000 7f3bf4ef4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3bf00aedd0 0x7f3bf006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:22.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.122+0000 7f3bf4ef4640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf00709b0 con 0x7f3bf006dae0 2024-04-17T10:42:22.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.122+0000 7f3bf4ef4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bf015e440 con 0x7f3bf00aedd0 2024-04-17T10:42:22.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.122+0000 7f3bf4ef4640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bf015e720 con 0x7f3bf015db50 2024-04-17T10:42:22.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.122+0000 7f3be5d9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3bf00aedd0 0x7f3bf006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:22.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.122+0000 7f3beed9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3bf006dae0 0x7f3bf00cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41624/0 (socket says 172.21.15.83:41624) 2024-04-17T10:42:22.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.122+0000 7f3beed9d640 1 -- 172.21.15.83:0/4179885961 learned_addr learned my addr 172.21.15.83:0/4179885961 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:22.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.122+0000 7f3bee59c640 1 --2- 172.21.15.83:0/4179885961 >> v2:172.21.15.167:3300/0 conn(0x7f3bf015db50 0x7f3bf0070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:22.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.122+0000 7f3be5d9b640 1 -- 172.21.15.83:0/4179885961 >> v2:172.21.15.167:3300/0 conn(0x7f3bf015db50 msgr2=0x7f3bf0070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:22.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.122+0000 7f3be5d9b640 1 --2- 172.21.15.83:0/4179885961 >> v2:172.21.15.167:3300/0 conn(0x7f3bf015db50 0x7f3bf0070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:22.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.122+0000 7f3be5d9b640 1 -- 172.21.15.83:0/4179885961 >> v1:172.21.15.83:6789/0 conn(0x7f3bf006dae0 legacy=0x7f3bf00cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:22.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.122+0000 7f3be5d9b640 1 -- 172.21.15.83:0/4179885961 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf015ea00 con 0x7f3bf00aedd0 2024-04-17T10:42:22.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.122+0000 7f3bee59c640 1 --2- 172.21.15.83:0/4179885961 >> v2:172.21.15.167:3300/0 conn(0x7f3bf015db50 0x7f3bf0070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.126+0000 7f3be5d9b640 1 --2- 172.21.15.83:0/4179885961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf00aedd0 0x7f3bf006d5a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80097c0 tx=0x7f3bd801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=987a86c24662e3ac server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.126+0000 7f3bedd9b640 1 -- 172.21.15.83:0/4179885961 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd80b0070 con 0x7f3bf00aedd0 2024-04-17T10:42:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.126+0000 7f3bedd9b640 1 -- 172.21.15.83:0/4179885961 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd8004230 con 0x7f3bf00aedd0 2024-04-17T10:42:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.126+0000 7f3bedd9b640 1 -- 172.21.15.83:0/4179885961 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8003620 con 0x7f3bf00aedd0 2024-04-17T10:42:22.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.126+0000 7f3bf4ef4640 1 -- 172.21.15.83:0/4179885961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf00aedd0 msgr2=0x7f3bf006d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:22.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.126+0000 7f3bf4ef4640 1 --2- 172.21.15.83:0/4179885961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf00aedd0 0x7f3bf006d5a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80097c0 tx=0x7f3bd801c460 comp rx=0 tx=0).stop 2024-04-17T10:42:22.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.126+0000 7f3bf4ef4640 1 -- 172.21.15.83:0/4179885961 shutdown_connections 2024-04-17T10:42:22.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.126+0000 7f3bf4ef4640 1 --2- 172.21.15.83:0/4179885961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf00aedd0 0x7f3bf006d5a0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:22.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.126+0000 7f3bf4ef4640 1 --2- 172.21.15.83:0/4179885961 >> v2:172.21.15.167:3300/0 conn(0x7f3bf015db50 0x7f3bf0070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:22.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.126+0000 7f3bf4ef4640 1 -- 172.21.15.83:0/4179885961 >> 172.21.15.83:0/4179885961 conn(0x7f3bf006abe0 msgr2=0x7f3bf006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:22.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.126+0000 7f3bf4ef4640 1 -- 172.21.15.83:0/4179885961 shutdown_connections 2024-04-17T10:42:22.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.126+0000 7f3bf4ef4640 1 -- 172.21.15.83:0/4179885961 wait complete. 2024-04-17T10:42:22.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.130+0000 7f3bf4ef4640 1 Processor -- start 2024-04-17T10:42:22.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.130+0000 7f3bf4ef4640 1 -- start start 2024-04-17T10:42:22.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.130+0000 7f3bf4ef4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bf006dae0 0x7f3bf00a9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:22.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.130+0000 7f3bf4ef4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bf00aedd0 0x7f3bf006c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:22.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.130+0000 7f3bf4ef4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf015db50 0x7f3bf00aaf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:22.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.130+0000 7f3bf4ef4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf01677e0 con 0x7f3bf015db50 2024-04-17T10:42:22.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.130+0000 7f3bf4ef4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf0167660 con 0x7f3bf00aedd0 2024-04-17T10:42:22.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.130+0000 7f3bf4ef4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf0167960 con 0x7f3bf006dae0 2024-04-17T10:42:22.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.130+0000 7f3beed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf015db50 0x7f3bf00aaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:22.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.130+0000 7f3beed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf015db50 0x7f3bf00aaf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57668/0 (socket says 172.21.15.83:57668) 2024-04-17T10:42:22.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.130+0000 7f3beed9d640 1 -- 172.21.15.83:0/2170384068 learned_addr learned my addr 172.21.15.83:0/2170384068 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:22.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.130+0000 7f3bee59c640 1 --2- 172.21.15.83:0/2170384068 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bf006dae0 0x7f3bf00a9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:22.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.130+0000 7f3be5d9b640 1 --2- 172.21.15.83:0/2170384068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bf00aedd0 0x7f3bf006c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:22.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.130+0000 7f3beed9d640 1 -- 172.21.15.83:0/2170384068 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bf006dae0 msgr2=0x7f3bf00a9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:22.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.130+0000 7f3beed9d640 1 --2- 172.21.15.83:0/2170384068 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bf006dae0 0x7f3bf00a9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:22.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.130+0000 7f3beed9d640 1 -- 172.21.15.83:0/2170384068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bf00aedd0 msgr2=0x7f3bf006c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:22.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.130+0000 7f3beed9d640 1 --2- 172.21.15.83:0/2170384068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bf00aedd0 0x7f3bf006c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:22.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.130+0000 7f3beed9d640 1 -- 172.21.15.83:0/2170384068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf00cd840 con 0x7f3bf015db50 2024-04-17T10:42:22.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.130+0000 7f3beed9d640 1 --2- 172.21.15.83:0/2170384068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf015db50 0x7f3bf00aaf20 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f3be800cbc0 tx=0x7f3be800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:22.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.130+0000 7f3be77fe640 1 -- 172.21.15.83:0/2170384068 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be8017070 con 0x7f3bf015db50 2024-04-17T10:42:22.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.130+0000 7f3be77fe640 1 -- 172.21.15.83:0/2170384068 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be800ad40 con 0x7f3bf015db50 2024-04-17T10:42:22.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.130+0000 7f3bf4ef4640 1 -- 172.21.15.83:0/2170384068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf00cdb20 con 0x7f3bf015db50 2024-04-17T10:42:22.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.130+0000 7f3be77fe640 1 -- 172.21.15.83:0/2170384068 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be8012600 con 0x7f3bf015db50 2024-04-17T10:42:22.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.130+0000 7f3bf4ef4640 1 -- 172.21.15.83:0/2170384068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf00ce0e0 con 0x7f3bf015db50 2024-04-17T10:42:22.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.130+0000 7f3bf4ef4640 1 -- 172.21.15.83:0/2170384068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bf0074f90 con 0x7f3bf015db50 2024-04-17T10:42:22.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.130+0000 7f3be77fe640 1 -- 172.21.15.83:0/2170384068 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3be801a070 con 0x7f3bf015db50 2024-04-17T10:42:22.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.130+0000 7f3be77fe640 1 --2- 172.21.15.83:0/2170384068 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3bd0066f60 0x7f3bd0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:22.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.130+0000 7f3be77fe640 1 -- 172.21.15.83:0/2170384068 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3be8085120 con 0x7f3bf015db50 2024-04-17T10:42:22.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.130+0000 7f3bee59c640 1 --2- 172.21.15.83:0/2170384068 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3bd0066f60 0x7f3bd0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:22.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.134+0000 7f3bee59c640 1 --2- 172.21.15.83:0/2170384068 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3bd0066f60 0x7f3bd0069420 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3be0006fd0 tx=0x7f3be0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:22.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.134+0000 7f3be77fe640 1 -- 172.21.15.83:0/2170384068 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3bf00709b0 con 0x7f3bf015db50 2024-04-17T10:42:22.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.318+0000 7f3bf4ef4640 1 -- 172.21.15.83:0/2170384068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3bf00b2af0 con 0x7f3bf015db50 2024-04-17T10:42:22.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.318+0000 7f3be77fe640 1 -- 172.21.15.83:0/2170384068 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f3be8055b80 con 0x7f3bf015db50 2024-04-17T10:42:22.316 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:22.317 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/161398126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":161398126},{"type":"v1","addr":"172.21.15.167:6833","nonce":161398126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1085380618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1085380618},{"type":"v1","addr":"172.21.15.167:6835","nonce":1085380618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/836442291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":836442291},{"type":"v1","addr":"172.21.15.83:6835","nonce":836442291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:41:37.773342+0000","modified":"2024-04-17T10:42:18.483648+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5682":{"gid":5682,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.83:6837/1894547386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1894547386},{"type":"v1","addr":"172.21.15.83:6837","nonce":1894547386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-17T10:42:22.317 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 23 2024-04-17T10:42:22.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.318+0000 7f3bf4ef4640 1 -- 172.21.15.83:0/2170384068 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3bd0066f60 msgr2=0x7f3bd0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:22.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.318+0000 7f3bf4ef4640 1 --2- 172.21.15.83:0/2170384068 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3bd0066f60 0x7f3bd0069420 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3be0006fd0 tx=0x7f3be0008040 comp rx=0 tx=0).stop 2024-04-17T10:42:22.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.318+0000 7f3bf4ef4640 1 -- 172.21.15.83:0/2170384068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf015db50 msgr2=0x7f3bf00aaf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:22.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.318+0000 7f3bf4ef4640 1 --2- 172.21.15.83:0/2170384068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf015db50 0x7f3bf00aaf20 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f3be800cbc0 tx=0x7f3be800a430 comp rx=0 tx=0).stop 2024-04-17T10:42:22.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.318+0000 7f3bf4ef4640 1 -- 172.21.15.83:0/2170384068 shutdown_connections 2024-04-17T10:42:22.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.318+0000 7f3bf4ef4640 1 --2- 172.21.15.83:0/2170384068 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3bd0066f60 0x7f3bd0069420 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:22.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.318+0000 7f3bf4ef4640 1 --2- 172.21.15.83:0/2170384068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf015db50 0x7f3bf00aaf20 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:22.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.318+0000 7f3bf4ef4640 1 --2- 172.21.15.83:0/2170384068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bf00aedd0 0x7f3bf006c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:22.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.318+0000 7f3bf4ef4640 1 --2- 172.21.15.83:0/2170384068 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bf006dae0 0x7f3bf00a9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:22.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.318+0000 7f3bf4ef4640 1 -- 172.21.15.83:0/2170384068 >> 172.21.15.83:0/2170384068 conn(0x7f3bf006abe0 msgr2=0x7f3bf006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:22.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.318+0000 7f3bf4ef4640 1 -- 172.21.15.83:0/2170384068 shutdown_connections 2024-04-17T10:42:22.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.318+0000 7f3bf4ef4640 1 -- 172.21.15.83:0/2170384068 wait complete. 2024-04-17T10:42:22.334 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:42:22.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.450+0000 7fd110333640 1 Processor -- start 2024-04-17T10:42:22.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.450+0000 7fd110333640 1 -- start start 2024-04-17T10:42:22.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.450+0000 7fd110333640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd10815ba30 0x7fd108072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:22.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.450+0000 7fd110333640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd10815c960 0x7fd1080cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:22.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.450+0000 7fd110333640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1080cfea0 con 0x7fd1080cfa10 2024-04-17T10:42:22.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.450+0000 7fd110333640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd108071ea0 con 0x7fd10815ba30 2024-04-17T10:42:22.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.450+0000 7fd110333640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd10806e320 con 0x7fd10815c960 2024-04-17T10:42:22.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.454+0000 7fd10e0cf640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd10815ba30 0x7fd108072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:22.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.454+0000 7fd10d8ce640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd10815c960 0x7fd1080cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:22.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.454+0000 7fd10e0cf640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd10815ba30 0x7fd108072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:39298/0 (socket says 172.21.15.83:39298) 2024-04-17T10:42:22.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.454+0000 7fd10e0cf640 1 -- 172.21.15.83:0/1057001285 learned_addr learned my addr 172.21.15.83:0/1057001285 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:22.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.454+0000 7fd10e8d0640 1 --1- 172.21.15.83:0/1057001285 >> v1:172.21.15.167:6789/0 conn(0x7fd1080cfa10 0x7fd10806c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:40826/0 (socket says 172.21.15.83:40826) 2024-04-17T10:42:22.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.454+0000 7fd10d8ce640 1 -- 172.21.15.83:0/1057001285 >> v2:172.21.15.167:3300/0 conn(0x7fd10815ba30 msgr2=0x7fd108072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:22.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.454+0000 7fd10d8ce640 1 --2- 172.21.15.83:0/1057001285 >> v2:172.21.15.167:3300/0 conn(0x7fd10815ba30 0x7fd108072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:22.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.454+0000 7fd10d8ce640 1 -- 172.21.15.83:0/1057001285 >> v1:172.21.15.167:6789/0 conn(0x7fd1080cfa10 legacy=0x7fd10806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:22.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.454+0000 7fd10d8ce640 1 -- 172.21.15.83:0/1057001285 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd10806e040 con 0x7fd10815c960 2024-04-17T10:42:22.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.454+0000 7fd10e0cf640 1 --2- 172.21.15.83:0/1057001285 >> v2:172.21.15.167:3300/0 conn(0x7fd10815ba30 0x7fd108072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:42:22.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.454+0000 7fd10d8ce640 1 --2- 172.21.15.83:0/1057001285 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd10815c960 0x7fd1080cf4d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd0f80097c0 tx=0x7fd0f801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26dd647206660a1e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:22.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.454+0000 7fd10d0cd640 1 -- 172.21.15.83:0/1057001285 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f80b0070 con 0x7fd10815c960 2024-04-17T10:42:22.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.454+0000 7fd10d0cd640 1 -- 172.21.15.83:0/1057001285 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f80ab600 con 0x7fd10815c960 2024-04-17T10:42:22.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.454+0000 7fd10d0cd640 1 -- 172.21.15.83:0/1057001285 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f80ab9d0 con 0x7fd10815c960 2024-04-17T10:42:22.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.454+0000 7fd110333640 1 -- 172.21.15.83:0/1057001285 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd10815c960 msgr2=0x7fd1080cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:22.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.454+0000 7fd110333640 1 --2- 172.21.15.83:0/1057001285 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd10815c960 0x7fd1080cf4d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd0f80097c0 tx=0x7fd0f801c410 comp rx=0 tx=0).stop 2024-04-17T10:42:22.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.458+0000 7fd110333640 1 -- 172.21.15.83:0/1057001285 shutdown_connections 2024-04-17T10:42:22.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.458+0000 7fd110333640 1 --2- 172.21.15.83:0/1057001285 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd10815c960 0x7fd1080cf4d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:22.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.458+0000 7fd110333640 1 --2- 172.21.15.83:0/1057001285 >> v2:172.21.15.167:3300/0 conn(0x7fd10815ba30 0x7fd108072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:22.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.458+0000 7fd110333640 1 -- 172.21.15.83:0/1057001285 >> 172.21.15.83:0/1057001285 conn(0x7fd10806abe0 msgr2=0x7fd10806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:22.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.458+0000 7fd110333640 1 -- 172.21.15.83:0/1057001285 shutdown_connections 2024-04-17T10:42:22.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.458+0000 7fd110333640 1 -- 172.21.15.83:0/1057001285 wait complete. 2024-04-17T10:42:22.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.458+0000 7fd110333640 1 Processor -- start 2024-04-17T10:42:22.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.458+0000 7fd110333640 1 -- start start 2024-04-17T10:42:22.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.458+0000 7fd110333640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1080cfa10 0x7fd10814e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:22.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.458+0000 7fd110333640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd10815ba30 0x7fd10814d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:22.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.458+0000 7fd110333640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd10815c960 0x7fd10815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:22.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.458+0000 7fd110333640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10806dca0 con 0x7fd10815c960 2024-04-17T10:42:22.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.458+0000 7fd110333640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1080aeae0 con 0x7fd1080cfa10 2024-04-17T10:42:22.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.458+0000 7fd110333640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd10806de20 con 0x7fd10815ba30 2024-04-17T10:42:22.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.458+0000 7fd10e8d0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd10815c960 0x7fd10815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:22.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.458+0000 7fd10e8d0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd10815c960 0x7fd10815a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57684/0 (socket says 172.21.15.83:57684) 2024-04-17T10:42:22.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.458+0000 7fd10e8d0640 1 -- 172.21.15.83:0/528652870 learned_addr learned my addr 172.21.15.83:0/528652870 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:22.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.458+0000 7fd10e0cf640 1 --2- 172.21.15.83:0/528652870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1080cfa10 0x7fd10814e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:22.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.458+0000 7fd10d8ce640 1 --2- 172.21.15.83:0/528652870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd10815ba30 0x7fd10814d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:22.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.458+0000 7fd10e8d0640 1 -- 172.21.15.83:0/528652870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd10815ba30 msgr2=0x7fd10814d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:22.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.458+0000 7fd10e8d0640 1 --2- 172.21.15.83:0/528652870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd10815ba30 0x7fd10814d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:22.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.458+0000 7fd10e8d0640 1 -- 172.21.15.83:0/528652870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1080cfa10 msgr2=0x7fd10814e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:22.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.458+0000 7fd10e8d0640 1 --2- 172.21.15.83:0/528652870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1080cfa10 0x7fd10814e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:22.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.458+0000 7fd10e8d0640 1 -- 172.21.15.83:0/528652870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd10814e7e0 con 0x7fd10815c960 2024-04-17T10:42:22.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.458+0000 7fd10e8d0640 1 --2- 172.21.15.83:0/528652870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd10815c960 0x7fd10815a5e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd10000c700 tx=0x7fd10000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:22.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.458+0000 7fd0feffd640 1 -- 172.21.15.83:0/528652870 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd100017070 con 0x7fd10815c960 2024-04-17T10:42:22.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.458+0000 7fd0feffd640 1 -- 172.21.15.83:0/528652870 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd10000ce10 con 0x7fd10815c960 2024-04-17T10:42:22.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.458+0000 7fd110333640 1 -- 172.21.15.83:0/528652870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd10814eac0 con 0x7fd10815c960 2024-04-17T10:42:22.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.458+0000 7fd0feffd640 1 -- 172.21.15.83:0/528652870 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10000a840 con 0x7fd10815c960 2024-04-17T10:42:22.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.458+0000 7fd110333640 1 -- 172.21.15.83:0/528652870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd10814f080 con 0x7fd10815c960 2024-04-17T10:42:22.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.462+0000 7fd110333640 1 -- 172.21.15.83:0/528652870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd108074f90 con 0x7fd10815c960 2024-04-17T10:42:22.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.462+0000 7fd0feffd640 1 -- 172.21.15.83:0/528652870 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd10001a070 con 0x7fd10815c960 2024-04-17T10:42:22.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.462+0000 7fd0feffd640 1 --2- 172.21.15.83:0/528652870 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd0ec066f90 0x7fd0ec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:22.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.462+0000 7fd10e0cf640 1 --2- 172.21.15.83:0/528652870 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd0ec066f90 0x7fd0ec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:22.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.462+0000 7fd0feffd640 1 -- 172.21.15.83:0/528652870 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd100084550 con 0x7fd10815c960 2024-04-17T10:42:22.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.462+0000 7fd10e0cf640 1 --2- 172.21.15.83:0/528652870 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd0ec066f90 0x7fd0ec069450 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd108157dd0 tx=0x7fd104008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:22.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.466+0000 7fd0feffd640 1 -- 172.21.15.83:0/528652870 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd100054ec0 con 0x7fd10815c960 2024-04-17T10:42:22.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.610+0000 7fd110333640 1 -- 172.21.15.83:0/528652870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd10806b3e0 con 0x7fd10815c960 2024-04-17T10:42:22.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.614+0000 7fd0feffd640 1 -- 172.21.15.83:0/528652870 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16693 (secure 0 0 0) 0x7fd100058f60 con 0x7fd10815c960 2024-04-17T10:42:22.614 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:22.614 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":48,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:41:54.319551+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:41:36.031388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-17T10:41:36.783410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.13","mappings":[{"from":0,"to":4}]},{"pgid":"7.16","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:42:22.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.618+0000 7fd110333640 1 -- 172.21.15.83:0/528652870 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd0ec066f90 msgr2=0x7fd0ec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:22.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.618+0000 7fd110333640 1 --2- 172.21.15.83:0/528652870 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd0ec066f90 0x7fd0ec069450 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd108157dd0 tx=0x7fd104008040 comp rx=0 tx=0).stop 2024-04-17T10:42:22.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.618+0000 7fd110333640 1 -- 172.21.15.83:0/528652870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd10815c960 msgr2=0x7fd10815a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:22.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.618+0000 7fd110333640 1 --2- 172.21.15.83:0/528652870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd10815c960 0x7fd10815a5e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd10000c700 tx=0x7fd10000cbc0 comp rx=0 tx=0).stop 2024-04-17T10:42:22.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.618+0000 7fd110333640 1 -- 172.21.15.83:0/528652870 shutdown_connections 2024-04-17T10:42:22.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.618+0000 7fd110333640 1 --2- 172.21.15.83:0/528652870 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd0ec066f90 0x7fd0ec069450 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:22.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.618+0000 7fd110333640 1 --2- 172.21.15.83:0/528652870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd10815c960 0x7fd10815a5e0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:22.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.618+0000 7fd110333640 1 --2- 172.21.15.83:0/528652870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd10815ba30 0x7fd10814d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:22.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.618+0000 7fd110333640 1 --2- 172.21.15.83:0/528652870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1080cfa10 0x7fd10814e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:22.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.618+0000 7fd110333640 1 -- 172.21.15.83:0/528652870 >> 172.21.15.83:0/528652870 conn(0x7fd10806abe0 msgr2=0x7fd10806c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:22.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.618+0000 7fd110333640 1 -- 172.21.15.83:0/528652870 shutdown_connections 2024-04-17T10:42:22.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.618+0000 7fd110333640 1 -- 172.21.15.83:0/528652870 wait complete. 2024-04-17T10:42:22.632 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-17T10:42:22.632 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:42:22.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6f38bf640 1 Processor -- start 2024-04-17T10:42:22.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6f38bf640 1 -- start start 2024-04-17T10:42:22.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6f38bf640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff6ec0ce980 0x7ff6ec0b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:22.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6f38bf640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6ec162a00 con 0x7ff6ec15dfa0 2024-04-17T10:42:22.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6f38bf640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6ec161e80 con 0x7ff6ec0cee00 2024-04-17T10:42:22.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6f38bf640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6ec162160 con 0x7ff6ec0ce980 2024-04-17T10:42:22.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6f165b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff6ec15dfa0 0x7ff6ec0b6c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41628/0 (socket says 172.21.15.83:41628) 2024-04-17T10:42:22.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6f165b640 1 -- 172.21.15.83:0/3163479559 learned_addr learned my addr 172.21.15.83:0/3163479559 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:22.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6e3fff640 1 -- 172.21.15.83:0/3163479559 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 536148490 0 0) 0x7ff6ec162a00 con 0x7ff6ec15dfa0 2024-04-17T10:42:22.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6e3fff640 1 -- 172.21.15.83:0/3163479559 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6d8003530 con 0x7ff6ec15dfa0 2024-04-17T10:42:22.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6f0e5a640 1 --2- 172.21.15.83:0/3163479559 >> v2:172.21.15.167:3300/0 conn(0x7ff6ec0ce980 0x7ff6ec0b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:22.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6e3fff640 1 -- 172.21.15.83:0/3163479559 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1574246072 0 0) 0x7ff6d8003530 con 0x7ff6ec15dfa0 2024-04-17T10:42:22.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6e3fff640 1 -- 172.21.15.83:0/3163479559 >> v2:172.21.15.167:3300/0 conn(0x7ff6ec0ce980 msgr2=0x7ff6ec0b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:22.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6e3fff640 1 --2- 172.21.15.83:0/3163479559 >> v2:172.21.15.167:3300/0 conn(0x7ff6ec0ce980 0x7ff6ec0b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:22.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6e3fff640 1 -- 172.21.15.83:0/3163479559 >> v1:172.21.15.167:6789/0 conn(0x7ff6ec0cee00 legacy=0x7ff6ec0d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:22.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6e3fff640 1 -- 172.21.15.83:0/3163479559 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ec162440 con 0x7ff6ec15dfa0 2024-04-17T10:42:22.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6e3fff640 1 -- 172.21.15.83:0/3163479559 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7ff6e8001680 con 0x7ff6ec15dfa0 2024-04-17T10:42:22.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6e3fff640 1 -- 172.21.15.83:0/3163479559 >> v1:172.21.15.83:6789/0 conn(0x7ff6ec15dfa0 legacy=0x7ff6ec0b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:22.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6e3fff640 1 --2- 172.21.15.83:0/3163479559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6d80039c0 0x7ff6d8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:22.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6e3fff640 1 --2- 172.21.15.83:0/3163479559 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6d8004810 0x7ff6d8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:22.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6e3fff640 1 --2- 172.21.15.83:0/3163479559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6d80051d0 0x7ff6d800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:22.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6e3fff640 1 -- 172.21.15.83:0/3163479559 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec162160 con 0x7ff6d80051d0 2024-04-17T10:42:22.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6e3fff640 1 -- 172.21.15.83:0/3163479559 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d800c170 con 0x7ff6d80039c0 2024-04-17T10:42:22.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6e3fff640 1 -- 172.21.15.83:0/3163479559 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d800c370 con 0x7ff6d8004810 2024-04-17T10:42:22.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6f1e5c640 1 --2- 172.21.15.83:0/3163479559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6d80051d0 0x7ff6d800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:22.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6f165b640 1 --2- 172.21.15.83:0/3163479559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6d80039c0 0x7ff6d8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:22.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6f0e5a640 1 --2- 172.21.15.83:0/3163479559 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6d8004810 0x7ff6d8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:22.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6f1e5c640 1 -- 172.21.15.83:0/3163479559 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6d8004810 msgr2=0x7ff6d8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:22.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6f1e5c640 1 --2- 172.21.15.83:0/3163479559 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6d8004810 0x7ff6d8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:22.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6f1e5c640 1 -- 172.21.15.83:0/3163479559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6d80039c0 msgr2=0x7ff6d8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:22.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6f1e5c640 1 --2- 172.21.15.83:0/3163479559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6d80039c0 0x7ff6d8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:22.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6f1e5c640 1 -- 172.21.15.83:0/3163479559 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d8003530 con 0x7ff6d80051d0 2024-04-17T10:42:22.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.746+0000 7ff6f0e5a640 1 --2- 172.21.15.83:0/3163479559 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6d8004810 0x7ff6d8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:22.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.750+0000 7ff6f1e5c640 1 -- 172.21.15.83:0/3163479559 reap_dead start 2024-04-17T10:42:22.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.750+0000 7ff6f1e5c640 1 --2- 172.21.15.83:0/3163479559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6d80051d0 0x7ff6d800b9d0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4001e70 tx=0x7ff6e401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=decd0d5b551f23fa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:22.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.750+0000 7ff6e3fff640 1 -- 172.21.15.83:0/3163479559 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4001a90 con 0x7ff6d80051d0 2024-04-17T10:42:22.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.750+0000 7ff6e3fff640 1 -- 172.21.15.83:0/3163479559 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e4003bc0 con 0x7ff6d80051d0 2024-04-17T10:42:22.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.750+0000 7ff6e3fff640 1 -- 172.21.15.83:0/3163479559 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e40050d0 con 0x7ff6d80051d0 2024-04-17T10:42:22.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.750+0000 7ff6f38bf640 1 -- 172.21.15.83:0/3163479559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6d80051d0 msgr2=0x7ff6d800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:22.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.750+0000 7ff6f38bf640 1 --2- 172.21.15.83:0/3163479559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6d80051d0 0x7ff6d800b9d0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4001e70 tx=0x7ff6e401c780 comp rx=0 tx=0).stop 2024-04-17T10:42:22.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.750+0000 7ff6f38bf640 1 -- 172.21.15.83:0/3163479559 shutdown_connections 2024-04-17T10:42:22.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.750+0000 7ff6f38bf640 1 --2- 172.21.15.83:0/3163479559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6d80051d0 0x7ff6d800b9d0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:22.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.750+0000 7ff6f38bf640 1 -- 172.21.15.83:0/3163479559 >> 172.21.15.83:0/3163479559 conn(0x7ff6ec0d0600 msgr2=0x7ff6ec0d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:22.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.750+0000 7ff6f38bf640 1 -- 172.21.15.83:0/3163479559 shutdown_connections 2024-04-17T10:42:22.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.750+0000 7ff6f38bf640 1 -- 172.21.15.83:0/3163479559 wait complete. 2024-04-17T10:42:22.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.750+0000 7ff6f38bf640 1 Processor -- start 2024-04-17T10:42:22.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.750+0000 7ff6f38bf640 1 -- start start 2024-04-17T10:42:22.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.750+0000 7ff6f38bf640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6d80051d0 0x7ff6ec0c9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:22.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.750+0000 7ff6f38bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6ec161030 0x7ff6ec1f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:22.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.750+0000 7ff6f38bf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6ec1f29c0 0x7ff6ec1f2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:22.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.750+0000 7ff6f38bf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec161b10 con 0x7ff6ec161030 2024-04-17T10:42:22.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.750+0000 7ff6f38bf640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec15e380 con 0x7ff6ec1f29c0 2024-04-17T10:42:22.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.750+0000 7ff6f38bf640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec161c90 con 0x7ff6d80051d0 2024-04-17T10:42:22.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.750+0000 7ff6f0e5a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6ec161030 0x7ff6ec1f3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:22.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.750+0000 7ff6f0e5a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6ec161030 0x7ff6ec1f3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57694/0 (socket says 172.21.15.83:57694) 2024-04-17T10:42:22.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.750+0000 7ff6f0e5a640 1 -- 172.21.15.83:0/266155005 learned_addr learned my addr 172.21.15.83:0/266155005 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:22.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.750+0000 7ff6f165b640 1 --2- 172.21.15.83:0/266155005 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6d80051d0 0x7ff6ec0c9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:22.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.750+0000 7ff6f1e5c640 1 --2- 172.21.15.83:0/266155005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6ec1f29c0 0x7ff6ec1f2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:22.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.754+0000 7ff6f0e5a640 1 -- 172.21.15.83:0/266155005 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6d80051d0 msgr2=0x7ff6ec0c9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:22.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.754+0000 7ff6f0e5a640 1 --2- 172.21.15.83:0/266155005 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6d80051d0 0x7ff6ec0c9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:22.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.754+0000 7ff6f0e5a640 1 -- 172.21.15.83:0/266155005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6ec1f29c0 msgr2=0x7ff6ec1f2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:22.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.754+0000 7ff6f0e5a640 1 --2- 172.21.15.83:0/266155005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6ec1f29c0 0x7ff6ec1f2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:22.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.754+0000 7ff6f0e5a640 1 -- 172.21.15.83:0/266155005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ec0c9fc0 con 0x7ff6ec161030 2024-04-17T10:42:22.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.754+0000 7ff6f1e5c640 1 --2- 172.21.15.83:0/266155005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6ec1f29c0 0x7ff6ec1f2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:22.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.754+0000 7ff6f0e5a640 1 --2- 172.21.15.83:0/266155005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6ec161030 0x7ff6ec1f3c00 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc010890 tx=0x7ff6dc010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:22.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.754+0000 7ff6e1ffb640 1 -- 172.21.15.83:0/266155005 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc00f040 con 0x7ff6ec161030 2024-04-17T10:42:22.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.754+0000 7ff6e1ffb640 1 -- 172.21.15.83:0/266155005 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6dc004220 con 0x7ff6ec161030 2024-04-17T10:42:22.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.754+0000 7ff6f38bf640 1 -- 172.21.15.83:0/266155005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6ec0ca2a0 con 0x7ff6ec161030 2024-04-17T10:42:22.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.754+0000 7ff6e1ffb640 1 -- 172.21.15.83:0/266155005 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc010e80 con 0x7ff6ec161030 2024-04-17T10:42:22.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.754+0000 7ff6f38bf640 1 -- 172.21.15.83:0/266155005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6ec1614b0 con 0x7ff6ec161030 2024-04-17T10:42:22.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.754+0000 7ff6f38bf640 1 -- 172.21.15.83:0/266155005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6ec0681a0 con 0x7ff6ec161030 2024-04-17T10:42:22.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.754+0000 7ff6e1ffb640 1 -- 172.21.15.83:0/266155005 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff6dc00e960 con 0x7ff6ec161030 2024-04-17T10:42:22.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.754+0000 7ff6e1ffb640 1 --2- 172.21.15.83:0/266155005 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff6d0066f60 0x7ff6d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:22.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.754+0000 7ff6e1ffb640 1 -- 172.21.15.83:0/266155005 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff6dc085360 con 0x7ff6ec161030 2024-04-17T10:42:22.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.754+0000 7ff6f165b640 1 --2- 172.21.15.83:0/266155005 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff6d0066f60 0x7ff6d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:22.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.754+0000 7ff6f165b640 1 --2- 172.21.15.83:0/266155005 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff6d0066f60 0x7ff6d0069420 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8001060 tx=0x7ff6e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:22.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.758+0000 7ff6e1ffb640 1 -- 172.21.15.83:0/266155005 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff6dc018030 con 0x7ff6ec161030 2024-04-17T10:42:22.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.946+0000 7ff6f38bf640 1 -- 172.21.15.83:0/266155005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6ec074f90 con 0x7ff6ec161030 2024-04-17T10:42:22.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.946+0000 7ff6e1ffb640 1 -- 172.21.15.83:0/266155005 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4939 (secure 0 0 0) 0x7ff6dc056330 con 0x7ff6ec161030 2024-04-17T10:42:22.946 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:22.946 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/161398126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":161398126},{"type":"v1","addr":"172.21.15.167:6833","nonce":161398126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1085380618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1085380618},{"type":"v1","addr":"172.21.15.167:6835","nonce":1085380618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/836442291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":836442291},{"type":"v1","addr":"172.21.15.83:6835","nonce":836442291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:41:37.773342+0000","modified":"2024-04-17T10:42:18.483648+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5682":{"gid":5682,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.83:6837/1894547386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1894547386},{"type":"v1","addr":"172.21.15.83:6837","nonce":1894547386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-17T10:42:22.946 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 23 2024-04-17T10:42:22.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.950+0000 7ff6f38bf640 1 -- 172.21.15.83:0/266155005 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff6d0066f60 msgr2=0x7ff6d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:22.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.950+0000 7ff6f38bf640 1 --2- 172.21.15.83:0/266155005 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff6d0066f60 0x7ff6d0069420 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8001060 tx=0x7ff6e8009040 comp rx=0 tx=0).stop 2024-04-17T10:42:22.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.950+0000 7ff6f38bf640 1 -- 172.21.15.83:0/266155005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6ec161030 msgr2=0x7ff6ec1f3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:22.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.950+0000 7ff6f38bf640 1 --2- 172.21.15.83:0/266155005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6ec161030 0x7ff6ec1f3c00 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc010890 tx=0x7ff6dc010d50 comp rx=0 tx=0).stop 2024-04-17T10:42:22.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.950+0000 7ff6f38bf640 1 -- 172.21.15.83:0/266155005 shutdown_connections 2024-04-17T10:42:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.950+0000 7ff6f38bf640 1 --2- 172.21.15.83:0/266155005 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff6d0066f60 0x7ff6d0069420 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.950+0000 7ff6f38bf640 1 --2- 172.21.15.83:0/266155005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6ec1f29c0 0x7ff6ec1f2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.950+0000 7ff6f38bf640 1 --2- 172.21.15.83:0/266155005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6ec161030 0x7ff6ec1f3c00 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.950+0000 7ff6f38bf640 1 --2- 172.21.15.83:0/266155005 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6d80051d0 0x7ff6ec0c9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.950+0000 7ff6f38bf640 1 -- 172.21.15.83:0/266155005 >> 172.21.15.83:0/266155005 conn(0x7ff6ec0d0600 msgr2=0x7ff6ec15bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.950+0000 7ff6f38bf640 1 -- 172.21.15.83:0/266155005 shutdown_connections 2024-04-17T10:42:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:22.950+0000 7ff6f38bf640 1 -- 172.21.15.83:0/266155005 wait complete. 2024-04-17T10:42:22.963 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:42:23.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.078+0000 7fa65db92640 1 Processor -- start 2024-04-17T10:42:23.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.078+0000 7fa65db92640 1 -- start start 2024-04-17T10:42:23.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.082+0000 7fa65db92640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa6580ce6f0 0x7fa65806abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:23.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.082+0000 7fa65db92640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6581625b0 con 0x7fa6580713a0 2024-04-17T10:42:23.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.082+0000 7fa65db92640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6580718f0 con 0x7fa6580cead0 2024-04-17T10:42:23.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.082+0000 7fa65db92640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa658072470 con 0x7fa6580ce6f0 2024-04-17T10:42:23.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.082+0000 7fa657fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa6580713a0 0x7fa658161ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41642/0 (socket says 172.21.15.83:41642) 2024-04-17T10:42:23.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.082+0000 7fa657fff640 1 -- 172.21.15.83:0/1434517209 learned_addr learned my addr 172.21.15.83:0/1434517209 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:23.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.082+0000 7fa6577fe640 1 --2- 172.21.15.83:0/1434517209 >> v2:172.21.15.83:3300/0 conn(0x7fa6580ce6f0 0x7fa65806abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:23.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.082+0000 7fa656ffd640 1 -- 172.21.15.83:0/1434517209 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 44355856 0 0) 0x7fa6581625b0 con 0x7fa6580713a0 2024-04-17T10:42:23.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.082+0000 7fa656ffd640 1 -- 172.21.15.83:0/1434517209 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa644003530 con 0x7fa6580713a0 2024-04-17T10:42:23.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.082+0000 7fa656ffd640 1 -- 172.21.15.83:0/1434517209 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 287729899 0 0) 0x7fa644003530 con 0x7fa6580713a0 2024-04-17T10:42:23.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.082+0000 7fa6577fe640 1 -- 172.21.15.83:0/1434517209 >> v1:172.21.15.167:6789/0 conn(0x7fa6580cead0 legacy=0x7fa658161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:23.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.082+0000 7fa6577fe640 1 -- 172.21.15.83:0/1434517209 >> v1:172.21.15.83:6789/0 conn(0x7fa6580713a0 legacy=0x7fa658161ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:23.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.082+0000 7fa6577fe640 1 -- 172.21.15.83:0/1434517209 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa658071bd0 con 0x7fa6580ce6f0 2024-04-17T10:42:23.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.082+0000 7fa6577fe640 1 --2- 172.21.15.83:0/1434517209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6580ce6f0 0x7fa65806abe0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa6400097c0 tx=0x7fa64001c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd724f4ee1d80d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:23.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.082+0000 7fa656ffd640 1 -- 172.21.15.83:0/1434517209 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6400b0070 con 0x7fa6580ce6f0 2024-04-17T10:42:23.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.082+0000 7fa656ffd640 1 -- 172.21.15.83:0/1434517209 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6400ab400 con 0x7fa6580ce6f0 2024-04-17T10:42:23.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.082+0000 7fa656ffd640 1 -- 172.21.15.83:0/1434517209 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa650001a00 con 0x7fa6580ce6f0 2024-04-17T10:42:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.082+0000 7fa65db92640 1 -- 172.21.15.83:0/1434517209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6580ce6f0 msgr2=0x7fa65806abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.082+0000 7fa65db92640 1 --2- 172.21.15.83:0/1434517209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6580ce6f0 0x7fa65806abe0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa6400097c0 tx=0x7fa64001c300 comp rx=0 tx=0).stop 2024-04-17T10:42:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.086+0000 7fa65db92640 1 -- 172.21.15.83:0/1434517209 shutdown_connections 2024-04-17T10:42:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.086+0000 7fa65db92640 1 --2- 172.21.15.83:0/1434517209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6580ce6f0 0x7fa65806abe0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.086+0000 7fa65db92640 1 -- 172.21.15.83:0/1434517209 >> 172.21.15.83:0/1434517209 conn(0x7fa65806bcf0 msgr2=0x7fa65806c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.086+0000 7fa65db92640 1 -- 172.21.15.83:0/1434517209 shutdown_connections 2024-04-17T10:42:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.086+0000 7fa65db92640 1 -- 172.21.15.83:0/1434517209 wait complete. 2024-04-17T10:42:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.086+0000 7fa65db92640 1 Processor -- start 2024-04-17T10:42:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.086+0000 7fa65db92640 1 -- start start 2024-04-17T10:42:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.086+0000 7fa65db92640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6580713a0 0x7fa65815d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.086+0000 7fa65db92640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6580ce6f0 0x7fa65813d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.086+0000 7fa65db92640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6580cead0 0x7fa65813da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.086+0000 7fa65db92640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65806b2a0 con 0x7fa6580ce6f0 2024-04-17T10:42:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.086+0000 7fa65db92640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65806b120 con 0x7fa6580cead0 2024-04-17T10:42:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.086+0000 7fa65db92640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa65806b420 con 0x7fa6580713a0 2024-04-17T10:42:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.086+0000 7fa64ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6580ce6f0 0x7fa65813d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.086+0000 7fa64ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6580ce6f0 0x7fa65813d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57700/0 (socket says 172.21.15.83:57700) 2024-04-17T10:42:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.086+0000 7fa64ffff640 1 -- 172.21.15.83:0/2887746179 learned_addr learned my addr 172.21.15.83:0/2887746179 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.086+0000 7fa64ffff640 1 -- 172.21.15.83:0/2887746179 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6580713a0 msgr2=0x7fa65815d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.086+0000 7fa64ffff640 1 --2- 172.21.15.83:0/2887746179 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6580713a0 0x7fa65815d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.086+0000 7fa64ffff640 1 -- 172.21.15.83:0/2887746179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6580cead0 msgr2=0x7fa65813da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.086+0000 7fa64ffff640 1 --2- 172.21.15.83:0/2887746179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6580cead0 0x7fa65813da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.086+0000 7fa64ffff640 1 -- 172.21.15.83:0/2887746179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6580c7850 con 0x7fa6580ce6f0 2024-04-17T10:42:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.086+0000 7fa64ffff640 1 --2- 172.21.15.83:0/2887746179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6580ce6f0 0x7fa65813d540 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fa64800c3c0 tx=0x7fa64800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.086+0000 7fa654ff9640 1 -- 172.21.15.83:0/2887746179 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa648017070 con 0x7fa6580ce6f0 2024-04-17T10:42:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.086+0000 7fa654ff9640 1 -- 172.21.15.83:0/2887746179 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6480052d0 con 0x7fa6580ce6f0 2024-04-17T10:42:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.086+0000 7fa65db92640 1 -- 172.21.15.83:0/2887746179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6580c7b30 con 0x7fa6580ce6f0 2024-04-17T10:42:23.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.086+0000 7fa654ff9640 1 -- 172.21.15.83:0/2887746179 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa648003840 con 0x7fa6580ce6f0 2024-04-17T10:42:23.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.086+0000 7fa65db92640 1 -- 172.21.15.83:0/2887746179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6580c80f0 con 0x7fa6580ce6f0 2024-04-17T10:42:23.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.086+0000 7fa65db92640 1 -- 172.21.15.83:0/2887746179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6580c82f0 con 0x7fa6580ce6f0 2024-04-17T10:42:23.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.090+0000 7fa654ff9640 1 -- 172.21.15.83:0/2887746179 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa64801a070 con 0x7fa6580ce6f0 2024-04-17T10:42:23.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.090+0000 7fa654ff9640 1 --2- 172.21.15.83:0/2887746179 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa638066f60 0x7fa638069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:23.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.090+0000 7fa654ff9640 1 -- 172.21.15.83:0/2887746179 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa648084330 con 0x7fa6580ce6f0 2024-04-17T10:42:23.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.090+0000 7fa6577fe640 1 --2- 172.21.15.83:0/2887746179 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa638066f60 0x7fa638069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:23.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.090+0000 7fa6577fe640 1 --2- 172.21.15.83:0/2887746179 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa638066f60 0x7fa638069420 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa6400ad000 tx=0x7fa640008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:23.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.094+0000 7fa654ff9640 1 -- 172.21.15.83:0/2887746179 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa6580c82f0 con 0x7fa6580ce6f0 2024-04-17T10:42:23.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.270+0000 7fa65db92640 1 -- 172.21.15.83:0/2887746179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa658074f90 con 0x7fa6580ce6f0 2024-04-17T10:42:23.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.270+0000 7fa654ff9640 1 -- 172.21.15.83:0/2887746179 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4939 (secure 0 0 0) 0x7fa648005480 con 0x7fa6580ce6f0 2024-04-17T10:42:23.271 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:23.271 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/161398126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":161398126},{"type":"v1","addr":"172.21.15.167:6833","nonce":161398126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1085380618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1085380618},{"type":"v1","addr":"172.21.15.167:6835","nonce":1085380618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/836442291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":836442291},{"type":"v1","addr":"172.21.15.83:6835","nonce":836442291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:41:37.773342+0000","modified":"2024-04-17T10:42:18.483648+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5682":{"gid":5682,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.83:6837/1894547386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1894547386},{"type":"v1","addr":"172.21.15.83:6837","nonce":1894547386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-17T10:42:23.271 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 23 2024-04-17T10:42:23.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.274+0000 7fa65db92640 1 -- 172.21.15.83:0/2887746179 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa638066f60 msgr2=0x7fa638069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:23.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.274+0000 7fa65db92640 1 --2- 172.21.15.83:0/2887746179 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa638066f60 0x7fa638069420 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa6400ad000 tx=0x7fa640008040 comp rx=0 tx=0).stop 2024-04-17T10:42:23.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.274+0000 7fa65db92640 1 -- 172.21.15.83:0/2887746179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6580ce6f0 msgr2=0x7fa65813d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:23.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.274+0000 7fa65db92640 1 --2- 172.21.15.83:0/2887746179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6580ce6f0 0x7fa65813d540 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fa64800c3c0 tx=0x7fa64800c880 comp rx=0 tx=0).stop 2024-04-17T10:42:23.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.274+0000 7fa65db92640 1 -- 172.21.15.83:0/2887746179 shutdown_connections 2024-04-17T10:42:23.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.274+0000 7fa65db92640 1 --2- 172.21.15.83:0/2887746179 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa638066f60 0x7fa638069420 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:23.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.274+0000 7fa65db92640 1 --2- 172.21.15.83:0/2887746179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6580cead0 0x7fa65813da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:23.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.274+0000 7fa65db92640 1 --2- 172.21.15.83:0/2887746179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6580ce6f0 0x7fa65813d540 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:23.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.274+0000 7fa65db92640 1 --2- 172.21.15.83:0/2887746179 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6580713a0 0x7fa65815d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:23.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.274+0000 7fa65db92640 1 -- 172.21.15.83:0/2887746179 >> 172.21.15.83:0/2887746179 conn(0x7fa65806bcf0 msgr2=0x7fa658161d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:23.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.274+0000 7fa65db92640 1 -- 172.21.15.83:0/2887746179 shutdown_connections 2024-04-17T10:42:23.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.274+0000 7fa65db92640 1 -- 172.21.15.83:0/2887746179 wait complete. 2024-04-17T10:42:23.288 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:42:23.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.394+0000 7fdfcda22640 1 Processor -- start 2024-04-17T10:42:23.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.394+0000 7fdfcda22640 1 -- start start 2024-04-17T10:42:23.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.394+0000 7fdfcda22640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdfc815eeb0 0x7fdfc806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:23.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.394+0000 7fdfcda22640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfc80cf780 0x7fdfc806bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:23.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.394+0000 7fdfcda22640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdfc815f290 0x7fdfc806c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:23.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.394+0000 7fdfcda22640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfc80cdb90 con 0x7fdfc815eeb0 2024-04-17T10:42:23.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.394+0000 7fdfcda22640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfc806e980 con 0x7fdfc815f290 2024-04-17T10:42:23.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.394+0000 7fdfcda22640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc806e0e0 con 0x7fdfc80cf780 2024-04-17T10:42:23.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.394+0000 7fdfc6ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdfc815eeb0 0x7fdfc806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:23.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.394+0000 7fdfc6ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdfc815eeb0 0x7fdfc806d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57714/0 (socket says 172.21.15.83:57714) 2024-04-17T10:42:23.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.394+0000 7fdfc6ffd640 1 -- 172.21.15.83:0/4038373444 learned_addr learned my addr 172.21.15.83:0/4038373444 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:23.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.394+0000 7fdfc77fe640 1 --2- 172.21.15.83:0/4038373444 >> v2:172.21.15.167:3300/0 conn(0x7fdfc815f290 0x7fdfc806c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:23.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.394+0000 7fdfc67fc640 1 --2- 172.21.15.83:0/4038373444 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfc80cf780 0x7fdfc806bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:23.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.394+0000 7fdfc6ffd640 1 -- 172.21.15.83:0/4038373444 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfc80cf780 msgr2=0x7fdfc806bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:23.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.394+0000 7fdfc6ffd640 1 --2- 172.21.15.83:0/4038373444 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfc80cf780 0x7fdfc806bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:23.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.394+0000 7fdfc6ffd640 1 -- 172.21.15.83:0/4038373444 >> v2:172.21.15.167:3300/0 conn(0x7fdfc815f290 msgr2=0x7fdfc806c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:23.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.394+0000 7fdfc6ffd640 1 --2- 172.21.15.83:0/4038373444 >> v2:172.21.15.167:3300/0 conn(0x7fdfc815f290 0x7fdfc806c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:23.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.394+0000 7fdfc6ffd640 1 -- 172.21.15.83:0/4038373444 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc806e3c0 con 0x7fdfc815eeb0 2024-04-17T10:42:23.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.394+0000 7fdfc77fe640 1 --2- 172.21.15.83:0/4038373444 >> v2:172.21.15.167:3300/0 conn(0x7fdfc815f290 0x7fdfc806c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:23.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.398+0000 7fdfc6ffd640 1 --2- 172.21.15.83:0/4038373444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfc815eeb0 0x7fdfc806d5d0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fdfc80cd870 tx=0x7fdfb801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c33f6fd1ea77ae server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:23.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.398+0000 7fdfc5ffb640 1 -- 172.21.15.83:0/4038373444 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb80a5860 con 0x7fdfc815eeb0 2024-04-17T10:42:23.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.398+0000 7fdfc5ffb640 1 -- 172.21.15.83:0/4038373444 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb80a5a00 con 0x7fdfc815eeb0 2024-04-17T10:42:23.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.398+0000 7fdfc5ffb640 1 -- 172.21.15.83:0/4038373444 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb80a5dd0 con 0x7fdfc815eeb0 2024-04-17T10:42:23.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.398+0000 7fdfcda22640 1 -- 172.21.15.83:0/4038373444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfc815eeb0 msgr2=0x7fdfc806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:23.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.398+0000 7fdfcda22640 1 --2- 172.21.15.83:0/4038373444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfc815eeb0 0x7fdfc806d5d0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fdfc80cd870 tx=0x7fdfb801c9c0 comp rx=0 tx=0).stop 2024-04-17T10:42:23.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.398+0000 7fdfcda22640 1 -- 172.21.15.83:0/4038373444 shutdown_connections 2024-04-17T10:42:23.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.398+0000 7fdfcda22640 1 --2- 172.21.15.83:0/4038373444 >> v2:172.21.15.167:3300/0 conn(0x7fdfc815f290 0x7fdfc806c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:23.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.398+0000 7fdfcda22640 1 --2- 172.21.15.83:0/4038373444 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfc80cf780 0x7fdfc806bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:23.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.398+0000 7fdfcda22640 1 --2- 172.21.15.83:0/4038373444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfc815eeb0 0x7fdfc806d5d0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:23.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.398+0000 7fdfcda22640 1 -- 172.21.15.83:0/4038373444 >> 172.21.15.83:0/4038373444 conn(0x7fdfc80cc910 msgr2=0x7fdfc80ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:23.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.398+0000 7fdfcda22640 1 -- 172.21.15.83:0/4038373444 shutdown_connections 2024-04-17T10:42:23.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.398+0000 7fdfcda22640 1 -- 172.21.15.83:0/4038373444 wait complete. 2024-04-17T10:42:23.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.398+0000 7fdfcda22640 1 Processor -- start 2024-04-17T10:42:23.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.398+0000 7fdfcda22640 1 -- start start 2024-04-17T10:42:23.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.402+0000 7fdfcda22640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfc80cf780 0x7fdfc80b0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:23.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.402+0000 7fdfcda22640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfc815eeb0 0x7fdfc80c9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:23.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.402+0000 7fdfcda22640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfc815f290 0x7fdfc80c7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:23.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.402+0000 7fdfcda22640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc81602e0 con 0x7fdfc815eeb0 2024-04-17T10:42:23.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.402+0000 7fdfcda22640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc8160160 con 0x7fdfc80cf780 2024-04-17T10:42:23.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.402+0000 7fdfcda22640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc8160460 con 0x7fdfc815f290 2024-04-17T10:42:23.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.402+0000 7fdfc67fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfc815eeb0 0x7fdfc80c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:23.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.402+0000 7fdfc67fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfc815eeb0 0x7fdfc80c9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57716/0 (socket says 172.21.15.83:57716) 2024-04-17T10:42:23.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.402+0000 7fdfc67fc640 1 -- 172.21.15.83:0/2068898938 learned_addr learned my addr 172.21.15.83:0/2068898938 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:23.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.402+0000 7fdfc6ffd640 1 --2- 172.21.15.83:0/2068898938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfc80cf780 0x7fdfc80b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:23.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.402+0000 7fdfc77fe640 1 --2- 172.21.15.83:0/2068898938 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfc815f290 0x7fdfc80c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:23.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.402+0000 7fdfc67fc640 1 -- 172.21.15.83:0/2068898938 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfc815f290 msgr2=0x7fdfc80c7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:23.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.402+0000 7fdfc67fc640 1 --2- 172.21.15.83:0/2068898938 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfc815f290 0x7fdfc80c7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:23.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.402+0000 7fdfc67fc640 1 -- 172.21.15.83:0/2068898938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfc80cf780 msgr2=0x7fdfc80b0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:23.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.402+0000 7fdfc67fc640 1 --2- 172.21.15.83:0/2068898938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfc80cf780 0x7fdfc80b0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:23.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.402+0000 7fdfc67fc640 1 -- 172.21.15.83:0/2068898938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc80c9be0 con 0x7fdfc815eeb0 2024-04-17T10:42:23.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.402+0000 7fdfc67fc640 1 --2- 172.21.15.83:0/2068898938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfc815eeb0 0x7fdfc80c9300 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00c610 tx=0x7fdfbc00cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:23.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.402+0000 7fdfaffff640 1 -- 172.21.15.83:0/2068898938 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc015070 con 0x7fdfc815eeb0 2024-04-17T10:42:23.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.402+0000 7fdfaffff640 1 -- 172.21.15.83:0/2068898938 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfbc004220 con 0x7fdfc815eeb0 2024-04-17T10:42:23.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.402+0000 7fdfcda22640 1 -- 172.21.15.83:0/2068898938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc8158c40 con 0x7fdfc815eeb0 2024-04-17T10:42:23.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.402+0000 7fdfaffff640 1 -- 172.21.15.83:0/2068898938 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc003620 con 0x7fdfc815eeb0 2024-04-17T10:42:23.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.402+0000 7fdfcda22640 1 -- 172.21.15.83:0/2068898938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc80c9ec0 con 0x7fdfc815eeb0 2024-04-17T10:42:23.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.402+0000 7fdfadffb640 1 -- 172.21.15.83:0/2068898938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfa00052a0 con 0x7fdfc815eeb0 2024-04-17T10:42:23.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.402+0000 7fdfaffff640 1 -- 172.21.15.83:0/2068898938 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdfbc0037c0 con 0x7fdfc815eeb0 2024-04-17T10:42:23.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.402+0000 7fdfaffff640 1 --2- 172.21.15.83:0/2068898938 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdfb4066f60 0x7fdfb4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:23.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.402+0000 7fdfaffff640 1 -- 172.21.15.83:0/2068898938 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fdfbc084e70 con 0x7fdfc815eeb0 2024-04-17T10:42:23.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.402+0000 7fdfc6ffd640 1 --2- 172.21.15.83:0/2068898938 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdfb4066f60 0x7fdfb4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:23.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.402+0000 7fdfc6ffd640 1 --2- 172.21.15.83:0/2068898938 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdfb4066f60 0x7fdfb4069420 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80ae000 tx=0x7fdfb80a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:23.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.406+0000 7fdfaffff640 1 -- 172.21.15.83:0/2068898938 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdfbc055e40 con 0x7fdfc815eeb0 2024-04-17T10:42:23.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.566+0000 7fdfadffb640 1 -- 172.21.15.83:0/2068898938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdfa0005860 con 0x7fdfc815eeb0 2024-04-17T10:42:23.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.566+0000 7fdfaffff640 1 -- 172.21.15.83:0/2068898938 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16693 (secure 0 0 0) 0x7fdfbc055740 con 0x7fdfc815eeb0 2024-04-17T10:42:23.567 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:23.567 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":48,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:41:54.319551+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:41:36.031388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-17T10:41:36.783410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.13","mappings":[{"from":0,"to":4}]},{"pgid":"7.16","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:42:23.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.570+0000 7fdfadffb640 1 -- 172.21.15.83:0/2068898938 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdfb4066f60 msgr2=0x7fdfb4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:23.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.570+0000 7fdfadffb640 1 --2- 172.21.15.83:0/2068898938 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdfb4066f60 0x7fdfb4069420 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80ae000 tx=0x7fdfb80a7070 comp rx=0 tx=0).stop 2024-04-17T10:42:23.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.570+0000 7fdfadffb640 1 -- 172.21.15.83:0/2068898938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfc815eeb0 msgr2=0x7fdfc80c9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:23.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.570+0000 7fdfadffb640 1 --2- 172.21.15.83:0/2068898938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfc815eeb0 0x7fdfc80c9300 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00c610 tx=0x7fdfbc00cad0 comp rx=0 tx=0).stop 2024-04-17T10:42:23.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.570+0000 7fdfadffb640 1 -- 172.21.15.83:0/2068898938 shutdown_connections 2024-04-17T10:42:23.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.570+0000 7fdfadffb640 1 --2- 172.21.15.83:0/2068898938 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdfb4066f60 0x7fdfb4069420 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:23.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.570+0000 7fdfadffb640 1 --2- 172.21.15.83:0/2068898938 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfc815f290 0x7fdfc80c7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:23.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.570+0000 7fdfadffb640 1 --2- 172.21.15.83:0/2068898938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfc815eeb0 0x7fdfc80c9300 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:23.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.570+0000 7fdfadffb640 1 --2- 172.21.15.83:0/2068898938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfc80cf780 0x7fdfc80b0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:23.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.570+0000 7fdfadffb640 1 -- 172.21.15.83:0/2068898938 >> 172.21.15.83:0/2068898938 conn(0x7fdfc80cc910 msgr2=0x7fdfc809a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:23.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.570+0000 7fdfadffb640 1 -- 172.21.15.83:0/2068898938 shutdown_connections 2024-04-17T10:42:23.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.570+0000 7fdfadffb640 1 -- 172.21.15.83:0/2068898938 wait complete. 2024-04-17T10:42:23.585 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-17T10:42:23.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.698+0000 7f43c830f640 1 Processor -- start 2024-04-17T10:42:23.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.698+0000 7f43c830f640 1 -- start start 2024-04-17T10:42:23.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.698+0000 7f43c830f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43c015b6e0 0x7f43c0071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:23.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.698+0000 7f43c830f640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f43c015bac0 0x7f43c00cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:23.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.698+0000 7f43c830f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f43c0070470 0x7f43c006ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:23.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.698+0000 7f43c830f640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43c006c1f0 con 0x7f43c0070470 2024-04-17T10:42:23.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.698+0000 7f43c830f640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43c00cdd30 con 0x7f43c015bac0 2024-04-17T10:42:23.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.698+0000 7f43c830f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c0070a10 con 0x7f43c015b6e0 2024-04-17T10:42:23.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.698+0000 7f43c68ac640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f43c0070470 0x7f43c006ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:23.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.698+0000 7f43c68ac640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f43c0070470 0x7f43c006ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57724/0 (socket says 172.21.15.83:57724) 2024-04-17T10:42:23.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.698+0000 7f43c68ac640 1 -- 172.21.15.83:0/2552390504 learned_addr learned my addr 172.21.15.83:0/2552390504 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:23.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.698+0000 7f43c60ab640 1 --2- 172.21.15.83:0/2552390504 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43c015b6e0 0x7f43c0071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:23.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.698+0000 7f43c58aa640 1 --2- 172.21.15.83:0/2552390504 >> v2:172.21.15.167:3300/0 conn(0x7f43c015bac0 0x7f43c00cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:23.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.698+0000 7f43c68ac640 1 -- 172.21.15.83:0/2552390504 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43c015b6e0 msgr2=0x7f43c0071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:23.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.698+0000 7f43c68ac640 1 --2- 172.21.15.83:0/2552390504 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43c015b6e0 0x7f43c0071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:23.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.698+0000 7f43c68ac640 1 -- 172.21.15.83:0/2552390504 >> v2:172.21.15.167:3300/0 conn(0x7f43c015bac0 msgr2=0x7f43c00cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:23.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.698+0000 7f43c68ac640 1 --2- 172.21.15.83:0/2552390504 >> v2:172.21.15.167:3300/0 conn(0x7f43c015bac0 0x7f43c00cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:23.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.698+0000 7f43c68ac640 1 -- 172.21.15.83:0/2552390504 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c0070cf0 con 0x7f43c0070470 2024-04-17T10:42:23.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43c68ac640 1 --2- 172.21.15.83:0/2552390504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43c0070470 0x7f43c006ae10 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f43c00b49f0 tx=0x7f43b801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6121570fa331c4db server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:23.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43c50a9640 1 -- 172.21.15.83:0/2552390504 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b80b0070 con 0x7f43c0070470 2024-04-17T10:42:23.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43c50a9640 1 -- 172.21.15.83:0/2552390504 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b80ab600 con 0x7f43c0070470 2024-04-17T10:42:23.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43c50a9640 1 -- 172.21.15.83:0/2552390504 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b80ab9d0 con 0x7f43c0070470 2024-04-17T10:42:23.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43c830f640 1 -- 172.21.15.83:0/2552390504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43c0070470 msgr2=0x7f43c006ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:23.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43c830f640 1 --2- 172.21.15.83:0/2552390504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43c0070470 0x7f43c006ae10 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f43c00b49f0 tx=0x7f43b801c520 comp rx=0 tx=0).stop 2024-04-17T10:42:23.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43c830f640 1 -- 172.21.15.83:0/2552390504 shutdown_connections 2024-04-17T10:42:23.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43c830f640 1 --2- 172.21.15.83:0/2552390504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43c0070470 0x7f43c006ae10 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:23.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43c830f640 1 --2- 172.21.15.83:0/2552390504 >> v2:172.21.15.167:3300/0 conn(0x7f43c015bac0 0x7f43c00cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:23.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43c830f640 1 --2- 172.21.15.83:0/2552390504 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43c015b6e0 0x7f43c0071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:23.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43c830f640 1 -- 172.21.15.83:0/2552390504 >> 172.21.15.83:0/2552390504 conn(0x7f43c00b15b0 msgr2=0x7f43c00b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:23.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43c830f640 1 -- 172.21.15.83:0/2552390504 shutdown_connections 2024-04-17T10:42:23.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43c830f640 1 -- 172.21.15.83:0/2552390504 wait complete. 2024-04-17T10:42:23.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43c830f640 1 Processor -- start 2024-04-17T10:42:23.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43c830f640 1 -- start start 2024-04-17T10:42:23.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43c830f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43c0070470 0x7f43c015f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:23.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43c830f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43c015b6e0 0x7f43c01f4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:23.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43c830f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43c015bac0 0x7f43c015ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:23.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43c830f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c006d270 con 0x7f43c015b6e0 2024-04-17T10:42:23.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43c830f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c006d0f0 con 0x7f43c015bac0 2024-04-17T10:42:23.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43c830f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c006b760 con 0x7f43c0070470 2024-04-17T10:42:23.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43c58aa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43c015b6e0 0x7f43c01f4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:23.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43c58aa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43c015b6e0 0x7f43c01f4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57728/0 (socket says 172.21.15.83:57728) 2024-04-17T10:42:23.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43c58aa640 1 -- 172.21.15.83:0/472290516 learned_addr learned my addr 172.21.15.83:0/472290516 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:23.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43c58aa640 1 -- 172.21.15.83:0/472290516 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43c0070470 msgr2=0x7f43c015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:23.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43c58aa640 1 --2- 172.21.15.83:0/472290516 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43c0070470 0x7f43c015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:23.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43c58aa640 1 -- 172.21.15.83:0/472290516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43c015bac0 msgr2=0x7f43c015ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:23.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43c58aa640 1 --2- 172.21.15.83:0/472290516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43c015bac0 0x7f43c015ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:23.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43c58aa640 1 -- 172.21.15.83:0/472290516 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c0151600 con 0x7f43c015b6e0 2024-04-17T10:42:23.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43c58aa640 1 --2- 172.21.15.83:0/472290516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43c015b6e0 0x7f43c01f4b60 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f43b400d7d0 tx=0x7f43b400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:23.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43beffd640 1 -- 172.21.15.83:0/472290516 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b4004020 con 0x7f43c015b6e0 2024-04-17T10:42:23.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43beffd640 1 -- 172.21.15.83:0/472290516 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b40041c0 con 0x7f43c015b6e0 2024-04-17T10:42:23.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.702+0000 7f43c830f640 1 -- 172.21.15.83:0/472290516 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c0151bc0 con 0x7f43c015b6e0 2024-04-17T10:42:23.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.706+0000 7f43beffd640 1 -- 172.21.15.83:0/472290516 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b40043b0 con 0x7f43c015b6e0 2024-04-17T10:42:23.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.706+0000 7f43c830f640 1 -- 172.21.15.83:0/472290516 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c0152180 con 0x7f43c015b6e0 2024-04-17T10:42:23.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.706+0000 7f43c830f640 1 -- 172.21.15.83:0/472290516 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c0074f90 con 0x7f43c015b6e0 2024-04-17T10:42:23.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.706+0000 7f43beffd640 1 -- 172.21.15.83:0/472290516 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f43b401b020 con 0x7f43c015b6e0 2024-04-17T10:42:23.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.706+0000 7f43beffd640 1 --2- 172.21.15.83:0/472290516 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f43ac066f90 0x7f43ac069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:23.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.706+0000 7f43beffd640 1 -- 172.21.15.83:0/472290516 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f43b4084e70 con 0x7f43c015b6e0 2024-04-17T10:42:23.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.706+0000 7f43c60ab640 1 --2- 172.21.15.83:0/472290516 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f43ac066f90 0x7f43ac069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:23.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.706+0000 7f43c60ab640 1 --2- 172.21.15.83:0/472290516 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f43ac066f90 0x7f43ac069450 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f43b00070b0 tx=0x7f43b0007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:23.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.710+0000 7f43beffd640 1 -- 172.21.15.83:0/472290516 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f43b4055db0 con 0x7f43c015b6e0 2024-04-17T10:42:23.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.886+0000 7f43c830f640 1 -- 172.21.15.83:0/472290516 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f43c00ae690 con 0x7f43c015b6e0 2024-04-17T10:42:23.885 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-04-17T10:42:23.886+0000 7f35c3b81640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T10:42:23.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.890+0000 7f43beffd640 1 -- 172.21.15.83:0/472290516 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) v1 ==== 161+0+0 (secure 0 0 0) 0x7f43b40556b0 con 0x7f43c015b6e0 2024-04-17T10:42:23.888 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-17T10:42:23.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.890+0000 7f43c830f640 1 -- 172.21.15.83:0/472290516 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f43ac066f90 msgr2=0x7f43ac069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:23.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.890+0000 7f43c830f640 1 --2- 172.21.15.83:0/472290516 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f43ac066f90 0x7f43ac069450 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f43b00070b0 tx=0x7f43b0007040 comp rx=0 tx=0).stop 2024-04-17T10:42:23.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.890+0000 7f43c830f640 1 -- 172.21.15.83:0/472290516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43c015b6e0 msgr2=0x7f43c01f4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:23.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.890+0000 7f43c830f640 1 --2- 172.21.15.83:0/472290516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43c015b6e0 0x7f43c01f4b60 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f43b400d7d0 tx=0x7f43b400dc90 comp rx=0 tx=0).stop 2024-04-17T10:42:23.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.890+0000 7f43c830f640 1 -- 172.21.15.83:0/472290516 shutdown_connections 2024-04-17T10:42:23.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.890+0000 7f43c830f640 1 --2- 172.21.15.83:0/472290516 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f43ac066f90 0x7f43ac069450 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:23.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.890+0000 7f43c830f640 1 --2- 172.21.15.83:0/472290516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43c015bac0 0x7f43c015ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:23.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.890+0000 7f43c830f640 1 --2- 172.21.15.83:0/472290516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43c015b6e0 0x7f43c01f4b60 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:23.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.890+0000 7f43c830f640 1 --2- 172.21.15.83:0/472290516 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43c0070470 0x7f43c015f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:23.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.890+0000 7f43c830f640 1 -- 172.21.15.83:0/472290516 >> 172.21.15.83:0/472290516 conn(0x7f43c00b15b0 msgr2=0x7f43c0158ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:23.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.894+0000 7f43c830f640 1 -- 172.21.15.83:0/472290516 shutdown_connections 2024-04-17T10:42:23.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:23.894+0000 7f43c830f640 1 -- 172.21.15.83:0/472290516 wait complete. 2024-04-17T10:42:23.909 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-17T10:42:23.945 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-04-17T10:42:24.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.018+0000 7faa57937640 1 Processor -- start 2024-04-17T10:42:24.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.018+0000 7faa57937640 1 -- start start 2024-04-17T10:42:24.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.022+0000 7faa57937640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa50154de0 0x7faa5006ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:24.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.022+0000 7faa57937640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faa5006b350 0x7faa500b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:24.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.022+0000 7faa57937640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa5015bd70 con 0x7faa500b4040 2024-04-17T10:42:24.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.022+0000 7faa57937640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa500b4870 con 0x7faa5006b350 2024-04-17T10:42:24.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.022+0000 7faa57937640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa5015bf60 con 0x7faa50154de0 2024-04-17T10:42:24.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.022+0000 7faa54ed2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faa5006b350 0x7faa500b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:24.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.022+0000 7faa54ed2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faa5006b350 0x7faa500b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57758/0 (socket says 172.21.15.83:57758) 2024-04-17T10:42:24.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.022+0000 7faa54ed2640 1 -- 172.21.15.83:0/625948155 learned_addr learned my addr 172.21.15.83:0/625948155 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:24.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.022+0000 7faa556d3640 1 --2- 172.21.15.83:0/625948155 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa50154de0 0x7faa5006ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:24.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.022+0000 7faa54ed2640 1 -- 172.21.15.83:0/625948155 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa50154de0 msgr2=0x7faa5006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:24.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.022+0000 7faa54ed2640 1 --2- 172.21.15.83:0/625948155 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa50154de0 0x7faa5006ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:24.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.022+0000 7faa54ed2640 1 -- 172.21.15.83:0/625948155 >> v1:172.21.15.167:6789/0 conn(0x7faa500b4040 legacy=0x7faa50071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:24.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.022+0000 7faa54ed2640 1 -- 172.21.15.83:0/625948155 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5015afb0 con 0x7faa5006b350 2024-04-17T10:42:24.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.022+0000 7faa54ed2640 1 --2- 172.21.15.83:0/625948155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa5006b350 0x7faa500b3b00 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7faa4000bb10 tx=0x7faa4001ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2831d0511c2a655 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:24.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.022+0000 7faa47fff640 1 -- 172.21.15.83:0/625948155 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa400b4070 con 0x7faa5006b350 2024-04-17T10:42:24.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.022+0000 7faa47fff640 1 -- 172.21.15.83:0/625948155 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa400a79a0 con 0x7faa5006b350 2024-04-17T10:42:24.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.022+0000 7faa47fff640 1 -- 172.21.15.83:0/625948155 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa400a7d70 con 0x7faa5006b350 2024-04-17T10:42:24.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.022+0000 7faa57937640 1 -- 172.21.15.83:0/625948155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa5006b350 msgr2=0x7faa500b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:24.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.022+0000 7faa57937640 1 --2- 172.21.15.83:0/625948155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa5006b350 0x7faa500b3b00 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7faa4000bb10 tx=0x7faa4001ebf0 comp rx=0 tx=0).stop 2024-04-17T10:42:24.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.022+0000 7faa57937640 1 -- 172.21.15.83:0/625948155 shutdown_connections 2024-04-17T10:42:24.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.022+0000 7faa57937640 1 --2- 172.21.15.83:0/625948155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa5006b350 0x7faa500b3b00 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:24.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.022+0000 7faa57937640 1 --2- 172.21.15.83:0/625948155 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa50154de0 0x7faa5006ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:24.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.022+0000 7faa57937640 1 -- 172.21.15.83:0/625948155 >> 172.21.15.83:0/625948155 conn(0x7faa500b15b0 msgr2=0x7faa500b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:24.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.026+0000 7faa57937640 1 -- 172.21.15.83:0/625948155 shutdown_connections 2024-04-17T10:42:24.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.026+0000 7faa57937640 1 -- 172.21.15.83:0/625948155 wait complete. 2024-04-17T10:42:24.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.026+0000 7faa57937640 1 Processor -- start 2024-04-17T10:42:24.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.026+0000 7faa57937640 1 -- start start 2024-04-17T10:42:24.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.026+0000 7faa57937640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa5006b350 0x7faa50149c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:24.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.026+0000 7faa57937640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa500b4040 0x7faa5014ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:24.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.026+0000 7faa57937640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa50154de0 0x7faa500ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.026+0000 7faa57937640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa500b44d0 con 0x7faa5006b350 2024-04-17T10:42:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.026+0000 7faa57937640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa501551c0 con 0x7faa50154de0 2024-04-17T10:42:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.026+0000 7faa57937640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa500b4650 con 0x7faa500b4040 2024-04-17T10:42:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.026+0000 7faa556d3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa5006b350 0x7faa50149c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.026+0000 7faa556d3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa5006b350 0x7faa50149c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57768/0 (socket says 172.21.15.83:57768) 2024-04-17T10:42:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.026+0000 7faa556d3640 1 -- 172.21.15.83:0/1675269988 learned_addr learned my addr 172.21.15.83:0/1675269988 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.026+0000 7faa54ed2640 1 --2- 172.21.15.83:0/1675269988 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa500b4040 0x7faa5014ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.026+0000 7faa556d3640 1 -- 172.21.15.83:0/1675269988 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa500b4040 msgr2=0x7faa5014ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.026+0000 7faa556d3640 1 --2- 172.21.15.83:0/1675269988 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa500b4040 0x7faa5014ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.026+0000 7faa556d3640 1 -- 172.21.15.83:0/1675269988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa50154de0 msgr2=0x7faa500ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.026+0000 7faa556d3640 1 --2- 172.21.15.83:0/1675269988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa50154de0 0x7faa500ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.026+0000 7faa556d3640 1 -- 172.21.15.83:0/1675269988 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5014a150 con 0x7faa5006b350 2024-04-17T10:42:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.026+0000 7faa54ed2640 1 --2- 172.21.15.83:0/1675269988 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa500b4040 0x7faa5014ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:24.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.026+0000 7faa556d3640 1 --2- 172.21.15.83:0/1675269988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa5006b350 0x7faa50149c10 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00c820 tx=0x7faa4c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:24.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.026+0000 7faa45ffb640 1 -- 172.21.15.83:0/1675269988 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c015070 con 0x7faa5006b350 2024-04-17T10:42:24.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.026+0000 7faa45ffb640 1 -- 172.21.15.83:0/1675269988 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa4c004240 con 0x7faa5006b350 2024-04-17T10:42:24.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.026+0000 7faa57937640 1 -- 172.21.15.83:0/1675269988 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa5014c3b0 con 0x7faa5006b350 2024-04-17T10:42:24.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.026+0000 7faa45ffb640 1 -- 172.21.15.83:0/1675269988 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c00ce10 con 0x7faa5006b350 2024-04-17T10:42:24.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.026+0000 7faa57937640 1 -- 172.21.15.83:0/1675269988 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa5014c970 con 0x7faa5006b350 2024-04-17T10:42:24.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.026+0000 7faa57937640 1 -- 172.21.15.83:0/1675269988 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa50074f90 con 0x7faa5006b350 2024-04-17T10:42:24.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.030+0000 7faa45ffb640 1 -- 172.21.15.83:0/1675269988 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faa4c0077d0 con 0x7faa5006b350 2024-04-17T10:42:24.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.030+0000 7faa45ffb640 1 --2- 172.21.15.83:0/1675269988 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faa34066f60 0x7faa34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:24.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.030+0000 7faa45ffb640 1 -- 172.21.15.83:0/1675269988 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5853+0+0 (secure 0 0 0) 0x7faa4c084e60 con 0x7faa5006b350 2024-04-17T10:42:24.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.030+0000 7faa54ed2640 1 --2- 172.21.15.83:0/1675269988 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faa34066f60 0x7faa34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:24.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.030+0000 7faa54ed2640 1 --2- 172.21.15.83:0/1675269988 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faa34066f60 0x7faa34069420 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7faa400b1000 tx=0x7faa40008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:24.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.034+0000 7faa45ffb640 1 -- 172.21.15.83:0/1675269988 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faa4c055e70 con 0x7faa5006b350 2024-04-17T10:42:24.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.210+0000 7faa57937640 1 -- 172.21.15.83:0/1675269988 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7faa500713d0 con 0x7faa5006b350 2024-04-17T10:42:24.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.890+0000 7faa45ffb640 1 -- 172.21.15.83:0/1675269988 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7faa4c055770 con 0x7faa5006b350 2024-04-17T10:42:24.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.894+0000 7faa57937640 1 -- 172.21.15.83:0/1675269988 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faa34066f60 msgr2=0x7faa34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:24.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.894+0000 7faa57937640 1 --2- 172.21.15.83:0/1675269988 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faa34066f60 0x7faa34069420 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7faa400b1000 tx=0x7faa40008040 comp rx=0 tx=0).stop 2024-04-17T10:42:24.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.894+0000 7faa57937640 1 -- 172.21.15.83:0/1675269988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa5006b350 msgr2=0x7faa50149c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:24.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.894+0000 7faa57937640 1 --2- 172.21.15.83:0/1675269988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa5006b350 0x7faa50149c10 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00c820 tx=0x7faa4c00cce0 comp rx=0 tx=0).stop 2024-04-17T10:42:24.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.894+0000 7faa57937640 1 -- 172.21.15.83:0/1675269988 shutdown_connections 2024-04-17T10:42:24.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.894+0000 7faa57937640 1 --2- 172.21.15.83:0/1675269988 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faa34066f60 0x7faa34069420 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:24.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.894+0000 7faa57937640 1 --2- 172.21.15.83:0/1675269988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa50154de0 0x7faa500ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:24.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.894+0000 7faa57937640 1 --2- 172.21.15.83:0/1675269988 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa500b4040 0x7faa5014ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:24.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.894+0000 7faa57937640 1 --2- 172.21.15.83:0/1675269988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa5006b350 0x7faa50149c10 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:24.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.894+0000 7faa57937640 1 -- 172.21.15.83:0/1675269988 >> 172.21.15.83:0/1675269988 conn(0x7faa500b15b0 msgr2=0x7faa5006d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:24.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.894+0000 7faa57937640 1 -- 172.21.15.83:0/1675269988 shutdown_connections 2024-04-17T10:42:24.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:24.894+0000 7faa57937640 1 -- 172.21.15.83:0/1675269988 wait complete. 2024-04-17T10:42:24.914 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-17T10:42:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.078+0000 7f8cdf625640 1 Processor -- start 2024-04-17T10:42:25.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.078+0000 7f8cdf625640 1 -- start start 2024-04-17T10:42:25.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.078+0000 7f8cdf625640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cd80afef0 0x7f8cd815e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:25.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.078+0000 7f8cdf625640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8cd806dfc0 0x7f8cd80b4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:25.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.078+0000 7f8cdf625640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cd80b4fd0 con 0x7f8cd80b47b0 2024-04-17T10:42:25.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.078+0000 7f8cdf625640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cd815e8f0 con 0x7f8cd806dfc0 2024-04-17T10:42:25.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.078+0000 7f8cdf625640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd815b060 con 0x7f8cd80afef0 2024-04-17T10:42:25.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.078+0000 7f8cddbc2640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8cd80b47b0 0x7f8cd8071510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41654/0 (socket says 172.21.15.83:41654) 2024-04-17T10:42:25.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.078+0000 7f8cddbc2640 1 -- 172.21.15.83:0/63658610 learned_addr learned my addr 172.21.15.83:0/63658610 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:25.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8cdcbc0640 1 --2- 172.21.15.83:0/63658610 >> v2:172.21.15.167:3300/0 conn(0x7f8cd806dfc0 0x7f8cd80b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:25.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8ccffff640 1 -- 172.21.15.83:0/63658610 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2581468496 0 0) 0x7f8cd80b4fd0 con 0x7f8cd80b47b0 2024-04-17T10:42:25.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8cdd3c1640 1 --2- 172.21.15.83:0/63658610 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cd80afef0 0x7f8cd815e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:25.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8ccffff640 1 -- 172.21.15.83:0/63658610 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cc4003530 con 0x7f8cd80b47b0 2024-04-17T10:42:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8ccffff640 1 -- 172.21.15.83:0/63658610 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 379633284 0 0) 0x7f8cc4003530 con 0x7f8cd80b47b0 2024-04-17T10:42:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8ccffff640 1 -- 172.21.15.83:0/63658610 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cd80afef0 msgr2=0x7f8cd815e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8ccffff640 1 --2- 172.21.15.83:0/63658610 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cd80afef0 0x7f8cd815e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8ccffff640 1 -- 172.21.15.83:0/63658610 >> v2:172.21.15.167:3300/0 conn(0x7f8cd806dfc0 msgr2=0x7f8cd80b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8ccffff640 1 --2- 172.21.15.83:0/63658610 >> v2:172.21.15.167:3300/0 conn(0x7f8cd806dfc0 0x7f8cd80b4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8ccffff640 1 -- 172.21.15.83:0/63658610 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd815eb20 con 0x7f8cd80b47b0 2024-04-17T10:42:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8ccffff640 1 -- 172.21.15.83:0/63658610 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f8cd0001a00 con 0x7f8cd80b47b0 2024-04-17T10:42:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8ccffff640 1 -- 172.21.15.83:0/63658610 >> v1:172.21.15.83:6789/0 conn(0x7f8cd80b47b0 legacy=0x7f8cd8071510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8ccffff640 1 --2- 172.21.15.83:0/63658610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8cc40039c0 0x7f8cc4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8ccffff640 1 --2- 172.21.15.83:0/63658610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cc4004810 0x7f8cc4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8ccffff640 1 --2- 172.21.15.83:0/63658610 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cd80afef0 0x7f8cc40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8ccffff640 1 -- 172.21.15.83:0/63658610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd815b060 con 0x7f8cc4004810 2024-04-17T10:42:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8ccffff640 1 -- 172.21.15.83:0/63658610 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc400b9d0 con 0x7f8cc40039c0 2024-04-17T10:42:25.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8ccffff640 1 -- 172.21.15.83:0/63658610 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc400bb50 con 0x7f8cd80afef0 2024-04-17T10:42:25.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8cdcbc0640 1 --2- 172.21.15.83:0/63658610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cc4004810 0x7f8cc4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:25.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8cdd3c1640 1 --2- 172.21.15.83:0/63658610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8cc40039c0 0x7f8cc4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:25.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8cdcbc0640 1 -- 172.21.15.83:0/63658610 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cd80afef0 msgr2=0x7f8cc40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:25.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8cddbc2640 1 --2- 172.21.15.83:0/63658610 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cd80afef0 0x7f8cc40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:25.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8cdcbc0640 1 --2- 172.21.15.83:0/63658610 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cd80afef0 0x7f8cc40051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:25.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8cdcbc0640 1 -- 172.21.15.83:0/63658610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8cc40039c0 msgr2=0x7f8cc4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:25.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8cdcbc0640 1 --2- 172.21.15.83:0/63658610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8cc40039c0 0x7f8cc4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:25.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8cdcbc0640 1 -- 172.21.15.83:0/63658610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc4003530 con 0x7f8cc4004810 2024-04-17T10:42:25.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8cdcbc0640 1 --2- 172.21.15.83:0/63658610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cc4004810 0x7f8cc4004c90 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8009ae0 tx=0x7f8cc801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c6e351f1d30078 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:25.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8ccffff640 1 -- 172.21.15.83:0/63658610 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc80b2070 con 0x7f8cc4004810 2024-04-17T10:42:25.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8ccffff640 1 -- 172.21.15.83:0/63658610 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc80a55c0 con 0x7f8cc4004810 2024-04-17T10:42:25.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8ccffff640 1 -- 172.21.15.83:0/63658610 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc80096b0 con 0x7f8cc4004810 2024-04-17T10:42:25.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8cdf625640 1 -- 172.21.15.83:0/63658610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cc4004810 msgr2=0x7f8cc4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:25.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8cdf625640 1 --2- 172.21.15.83:0/63658610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cc4004810 0x7f8cc4004c90 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8009ae0 tx=0x7f8cc801cab0 comp rx=0 tx=0).stop 2024-04-17T10:42:25.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8cddbc2640 1 -- 172.21.15.83:0/63658610 reap_dead start 2024-04-17T10:42:25.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8cdf625640 1 -- 172.21.15.83:0/63658610 shutdown_connections 2024-04-17T10:42:25.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8cdf625640 1 -- 172.21.15.83:0/63658610 >> 172.21.15.83:0/63658610 conn(0x7f8cd806bcf0 msgr2=0x7f8cd806c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:25.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8cdf625640 1 -- 172.21.15.83:0/63658610 shutdown_connections 2024-04-17T10:42:25.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.082+0000 7f8cdf625640 1 -- 172.21.15.83:0/63658610 wait complete. 2024-04-17T10:42:25.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.086+0000 7f8cdf625640 1 Processor -- start 2024-04-17T10:42:25.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.086+0000 7f8cdf625640 1 -- start start 2024-04-17T10:42:25.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.086+0000 7f8cdf625640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cc4004810 0x7f8cd814df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:25.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.086+0000 7f8cdf625640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cd814b2c0 0x7f8cd814cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:25.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.086+0000 7f8cdf625640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8cd814a920 0x7f8cd815c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:25.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.086+0000 7f8cdf625640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd815c500 con 0x7f8cc4004810 2024-04-17T10:42:25.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.086+0000 7f8cdf625640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd815f0e0 con 0x7f8cd814a920 2024-04-17T10:42:25.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.086+0000 7f8cdf625640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd815c680 con 0x7f8cd814b2c0 2024-04-17T10:42:25.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.086+0000 7f8cdd3c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cc4004810 0x7f8cd814df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:25.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.086+0000 7f8cdd3c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cc4004810 0x7f8cd814df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57790/0 (socket says 172.21.15.83:57790) 2024-04-17T10:42:25.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.086+0000 7f8cdd3c1640 1 -- 172.21.15.83:0/2020918387 learned_addr learned my addr 172.21.15.83:0/2020918387 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:25.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.086+0000 7f8cdcbc0640 1 --2- 172.21.15.83:0/2020918387 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cd814b2c0 0x7f8cd814cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:25.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.086+0000 7f8cddbc2640 1 --2- 172.21.15.83:0/2020918387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8cd814a920 0x7f8cd815c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:25.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.086+0000 7f8cdd3c1640 1 -- 172.21.15.83:0/2020918387 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cd814b2c0 msgr2=0x7f8cd814cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:25.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.086+0000 7f8cdd3c1640 1 --2- 172.21.15.83:0/2020918387 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cd814b2c0 0x7f8cd814cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:25.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.086+0000 7f8cdd3c1640 1 -- 172.21.15.83:0/2020918387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8cd814a920 msgr2=0x7f8cd815c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:25.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.086+0000 7f8cdd3c1640 1 --2- 172.21.15.83:0/2020918387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8cd814a920 0x7f8cd815c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:25.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.086+0000 7f8cdd3c1640 1 -- 172.21.15.83:0/2020918387 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd814d0b0 con 0x7f8cc4004810 2024-04-17T10:42:25.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.086+0000 7f8cdd3c1640 1 --2- 172.21.15.83:0/2020918387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cc4004810 0x7f8cd814df20 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f8cd400f580 tx=0x7f8cd400fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:25.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.086+0000 7f8ccdffb640 1 -- 172.21.15.83:0/2020918387 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4019020 con 0x7f8cc4004810 2024-04-17T10:42:25.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.086+0000 7f8cdf625640 1 -- 172.21.15.83:0/2020918387 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cd814bad0 con 0x7f8cc4004810 2024-04-17T10:42:25.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.086+0000 7f8ccdffb640 1 -- 172.21.15.83:0/2020918387 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cd4014870 con 0x7f8cc4004810 2024-04-17T10:42:25.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.086+0000 7f8ccdffb640 1 -- 172.21.15.83:0/2020918387 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4014c40 con 0x7f8cc4004810 2024-04-17T10:42:25.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.086+0000 7f8cdf625640 1 -- 172.21.15.83:0/2020918387 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd814c090 con 0x7f8cc4004810 2024-04-17T10:42:25.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.090+0000 7f8cdf625640 1 -- 172.21.15.83:0/2020918387 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cd8075000 con 0x7f8cc4004810 2024-04-17T10:42:25.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.090+0000 7f8ccdffb640 1 -- 172.21.15.83:0/2020918387 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8cd401c020 con 0x7f8cc4004810 2024-04-17T10:42:25.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.090+0000 7f8ccdffb640 1 --2- 172.21.15.83:0/2020918387 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8cbc066f60 0x7f8cbc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:25.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.090+0000 7f8cdcbc0640 1 --2- 172.21.15.83:0/2020918387 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8cbc066f60 0x7f8cbc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:25.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.090+0000 7f8ccdffb640 1 -- 172.21.15.83:0/2020918387 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f8cd40857f0 con 0x7f8cc4004810 2024-04-17T10:42:25.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.090+0000 7f8cdcbc0640 1 --2- 172.21.15.83:0/2020918387 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8cbc066f60 0x7f8cbc069420 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8cc80af000 tx=0x7f8cc80046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:25.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.094+0000 7f8ccdffb640 1 -- 172.21.15.83:0/2020918387 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8cd4023020 con 0x7f8cc4004810 2024-04-17T10:42:25.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.242+0000 7f8cdf625640 1 -- 172.21.15.83:0/2020918387 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8cd806d850 con 0x7f8cc4004810 2024-04-17T10:42:25.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.902+0000 7f8ccdffb640 1 -- 172.21.15.83:0/2020918387 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v50) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8cd4056780 con 0x7f8cc4004810 2024-04-17T10:42:25.901 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2024-04-17T10:42:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.906+0000 7f8cdf625640 1 -- 172.21.15.83:0/2020918387 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8cbc066f60 msgr2=0x7f8cbc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.906+0000 7f8cdf625640 1 --2- 172.21.15.83:0/2020918387 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8cbc066f60 0x7f8cbc069420 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8cc80af000 tx=0x7f8cc80046d0 comp rx=0 tx=0).stop 2024-04-17T10:42:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.906+0000 7f8cdf625640 1 -- 172.21.15.83:0/2020918387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cc4004810 msgr2=0x7f8cd814df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.906+0000 7f8cdf625640 1 --2- 172.21.15.83:0/2020918387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cc4004810 0x7f8cd814df20 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f8cd400f580 tx=0x7f8cd400fa40 comp rx=0 tx=0).stop 2024-04-17T10:42:25.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.906+0000 7f8cdf625640 1 -- 172.21.15.83:0/2020918387 shutdown_connections 2024-04-17T10:42:25.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.906+0000 7f8cdf625640 1 --2- 172.21.15.83:0/2020918387 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8cbc066f60 0x7f8cbc069420 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:25.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.906+0000 7f8cdf625640 1 --2- 172.21.15.83:0/2020918387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8cd814a920 0x7f8cd815c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:25.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.906+0000 7f8cdf625640 1 --2- 172.21.15.83:0/2020918387 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cd814b2c0 0x7f8cd814cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:25.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.906+0000 7f8cdf625640 1 --2- 172.21.15.83:0/2020918387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cc4004810 0x7f8cd814df20 secure :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f8cd400f580 tx=0x7f8cd400fa40 comp rx=0 tx=0).stop 2024-04-17T10:42:25.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.906+0000 7f8cdf625640 1 -- 172.21.15.83:0/2020918387 >> 172.21.15.83:0/2020918387 conn(0x7f8cd806bcf0 msgr2=0x7f8cd8159af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:25.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.910+0000 7f8cdf625640 1 -- 172.21.15.83:0/2020918387 shutdown_connections 2024-04-17T10:42:25.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:25.910+0000 7f8cdf625640 1 -- 172.21.15.83:0/2020918387 wait complete. 2024-04-17T10:42:25.929 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-17T10:42:26.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.058+0000 7f8491ea5640 1 Processor -- start 2024-04-17T10:42:26.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.058+0000 7f8491ea5640 1 -- start start 2024-04-17T10:42:26.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.058+0000 7f8491ea5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f848c06fce0 0x7f848c15eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:26.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.058+0000 7f8491ea5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f848c071580 0x7f848c0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:26.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.058+0000 7f8491ea5640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f848c15f0e0 con 0x7f848c0add80 2024-04-17T10:42:26.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.058+0000 7f8491ea5640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f848c15f2e0 con 0x7f848c071580 2024-04-17T10:42:26.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.058+0000 7f8491ea5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f848c071b20 con 0x7f848c06fce0 2024-04-17T10:42:26.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.058+0000 7f848affd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f848c071580 0x7f848c0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:26.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.058+0000 7f848affd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f848c071580 0x7f848c0ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57792/0 (socket says 172.21.15.83:57792) 2024-04-17T10:42:26.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.058+0000 7f848affd640 1 -- 172.21.15.83:0/1903286489 learned_addr learned my addr 172.21.15.83:0/1903286489 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:26.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.058+0000 7f848b7fe640 1 --2- 172.21.15.83:0/1903286489 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f848c06fce0 0x7f848c15eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:26.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.058+0000 7f848affd640 1 -- 172.21.15.83:0/1903286489 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f848c06fce0 msgr2=0x7f848c15eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:26.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.058+0000 7f848affd640 1 --2- 172.21.15.83:0/1903286489 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f848c06fce0 0x7f848c15eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:26.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.058+0000 7f848affd640 1 -- 172.21.15.83:0/1903286489 >> v1:172.21.15.167:6789/0 conn(0x7f848c0add80 legacy=0x7f848c06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:26.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.058+0000 7f848affd640 1 -- 172.21.15.83:0/1903286489 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848c071e00 con 0x7f848c071580 2024-04-17T10:42:26.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.062+0000 7f848affd640 1 --2- 172.21.15.83:0/1903286489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f848c071580 0x7f848c0ad840 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f8480009870 tx=0x7f848001c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa903f8f1e65da2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:26.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.062+0000 7f848a7fc640 1 -- 172.21.15.83:0/1903286489 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84800b0070 con 0x7f848c071580 2024-04-17T10:42:26.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.062+0000 7f848a7fc640 1 -- 172.21.15.83:0/1903286489 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84800ab600 con 0x7f848c071580 2024-04-17T10:42:26.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.062+0000 7f848a7fc640 1 -- 172.21.15.83:0/1903286489 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84800ab9d0 con 0x7f848c071580 2024-04-17T10:42:26.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.062+0000 7f8491ea5640 1 -- 172.21.15.83:0/1903286489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f848c071580 msgr2=0x7f848c0ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:26.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.062+0000 7f8491ea5640 1 --2- 172.21.15.83:0/1903286489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f848c071580 0x7f848c0ad840 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f8480009870 tx=0x7f848001c340 comp rx=0 tx=0).stop 2024-04-17T10:42:26.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.062+0000 7f8491ea5640 1 -- 172.21.15.83:0/1903286489 shutdown_connections 2024-04-17T10:42:26.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.062+0000 7f8491ea5640 1 --2- 172.21.15.83:0/1903286489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f848c071580 0x7f848c0ad840 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:26.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.062+0000 7f8491ea5640 1 --2- 172.21.15.83:0/1903286489 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f848c06fce0 0x7f848c15eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:26.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.062+0000 7f8491ea5640 1 -- 172.21.15.83:0/1903286489 >> 172.21.15.83:0/1903286489 conn(0x7f848c0b2410 msgr2=0x7f848c0b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:26.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.062+0000 7f8491ea5640 1 -- 172.21.15.83:0/1903286489 shutdown_connections 2024-04-17T10:42:26.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.062+0000 7f8491ea5640 1 -- 172.21.15.83:0/1903286489 wait complete. 2024-04-17T10:42:26.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.066+0000 7f8491ea5640 1 Processor -- start 2024-04-17T10:42:26.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.066+0000 7f8491ea5640 1 -- start start 2024-04-17T10:42:26.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.066+0000 7f8491ea5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f848c06fce0 0x7f848c0cd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:26.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.066+0000 7f8491ea5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f848c071580 0x7f848c0cfa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:26.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.066+0000 7f8491ea5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f848c0add80 0x7f848c15c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:26.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.066+0000 7f8491ea5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848c16a760 con 0x7f848c06fce0 2024-04-17T10:42:26.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.066+0000 7f8491ea5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848c0b5120 con 0x7f848c0add80 2024-04-17T10:42:26.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.066+0000 7f8491ea5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f848c16a8e0 con 0x7f848c071580 2024-04-17T10:42:26.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.066+0000 7f848b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f848c06fce0 0x7f848c0cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:26.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.066+0000 7f848b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f848c06fce0 0x7f848c0cd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57802/0 (socket says 172.21.15.83:57802) 2024-04-17T10:42:26.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.066+0000 7f848b7fe640 1 -- 172.21.15.83:0/1906959876 learned_addr learned my addr 172.21.15.83:0/1906959876 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:26.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.066+0000 7f848bfff640 1 --2- 172.21.15.83:0/1906959876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f848c0add80 0x7f848c15c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:26.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.066+0000 7f848affd640 1 --2- 172.21.15.83:0/1906959876 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f848c071580 0x7f848c0cfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:26.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.066+0000 7f848b7fe640 1 -- 172.21.15.83:0/1906959876 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f848c071580 msgr2=0x7f848c0cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:26.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.066+0000 7f848b7fe640 1 --2- 172.21.15.83:0/1906959876 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f848c071580 0x7f848c0cfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:26.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.066+0000 7f848b7fe640 1 -- 172.21.15.83:0/1906959876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f848c0add80 msgr2=0x7f848c15c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:26.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.066+0000 7f848b7fe640 1 --2- 172.21.15.83:0/1906959876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f848c0add80 0x7f848c15c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:26.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.066+0000 7f848b7fe640 1 -- 172.21.15.83:0/1906959876 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848c158940 con 0x7f848c06fce0 2024-04-17T10:42:26.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.066+0000 7f848b7fe640 1 --2- 172.21.15.83:0/1906959876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f848c06fce0 0x7f848c0cd130 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f847c00c7f0 tx=0x7f847c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:26.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.066+0000 7f8473fff640 1 -- 172.21.15.83:0/1906959876 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c015070 con 0x7f848c06fce0 2024-04-17T10:42:26.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.066+0000 7f8473fff640 1 -- 172.21.15.83:0/1906959876 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847c007820 con 0x7f848c06fce0 2024-04-17T10:42:26.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.066+0000 7f8491ea5640 1 -- 172.21.15.83:0/1906959876 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f848c158c20 con 0x7f848c06fce0 2024-04-17T10:42:26.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.066+0000 7f8473fff640 1 -- 172.21.15.83:0/1906959876 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c007c10 con 0x7f848c06fce0 2024-04-17T10:42:26.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.066+0000 7f8491ea5640 1 -- 172.21.15.83:0/1906959876 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f848c1594c0 con 0x7f848c06fce0 2024-04-17T10:42:26.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.066+0000 7f8491ea5640 1 -- 172.21.15.83:0/1906959876 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f848c15bd60 con 0x7f848c06fce0 2024-04-17T10:42:26.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.070+0000 7f8473fff640 1 -- 172.21.15.83:0/1906959876 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f847c018070 con 0x7f848c06fce0 2024-04-17T10:42:26.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.070+0000 7f8473fff640 1 --2- 172.21.15.83:0/1906959876 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8478066f60 0x7f8478069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:26.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.070+0000 7f8473fff640 1 -- 172.21.15.83:0/1906959876 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5415+0+0 (secure 0 0 0) 0x7f847c084da0 con 0x7f848c06fce0 2024-04-17T10:42:26.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.070+0000 7f848affd640 1 --2- 172.21.15.83:0/1906959876 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8478066f60 0x7f8478069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:26.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.070+0000 7f848affd640 1 --2- 172.21.15.83:0/1906959876 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8478066f60 0x7f8478069420 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f84800ad000 tx=0x7f8480008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:26.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.074+0000 7f8473fff640 1 -- 172.21.15.83:0/1906959876 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f847c055ed0 con 0x7f848c06fce0 2024-04-17T10:42:26.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.234+0000 7f8491ea5640 1 -- 172.21.15.83:0/1906959876 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f848c075000 con 0x7f848c06fce0 2024-04-17T10:42:26.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.910+0000 7f8473fff640 1 -- 172.21.15.83:0/1906959876 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v51) v1 ==== 169+0+0 (secure 0 0 0) 0x7f847c0557d0 con 0x7f848c06fce0 2024-04-17T10:42:26.910 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2024-04-17T10:42:26.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.914+0000 7f8491ea5640 1 -- 172.21.15.83:0/1906959876 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8478066f60 msgr2=0x7f8478069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:26.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.914+0000 7f8491ea5640 1 --2- 172.21.15.83:0/1906959876 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8478066f60 0x7f8478069420 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f84800ad000 tx=0x7f8480008040 comp rx=0 tx=0).stop 2024-04-17T10:42:26.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.914+0000 7f8491ea5640 1 -- 172.21.15.83:0/1906959876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f848c06fce0 msgr2=0x7f848c0cd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:26.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.914+0000 7f8491ea5640 1 --2- 172.21.15.83:0/1906959876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f848c06fce0 0x7f848c0cd130 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f847c00c7f0 tx=0x7f847c00ccb0 comp rx=0 tx=0).stop 2024-04-17T10:42:26.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.918+0000 7f8491ea5640 1 -- 172.21.15.83:0/1906959876 shutdown_connections 2024-04-17T10:42:26.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.918+0000 7f8491ea5640 1 --2- 172.21.15.83:0/1906959876 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8478066f60 0x7f8478069420 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:26.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.918+0000 7f8491ea5640 1 --2- 172.21.15.83:0/1906959876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f848c0add80 0x7f848c15c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:26.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.918+0000 7f8491ea5640 1 --2- 172.21.15.83:0/1906959876 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f848c071580 0x7f848c0cfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:26.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.918+0000 7f8491ea5640 1 --2- 172.21.15.83:0/1906959876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f848c06fce0 0x7f848c0cd130 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:26.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.918+0000 7f8491ea5640 1 -- 172.21.15.83:0/1906959876 >> 172.21.15.83:0/1906959876 conn(0x7f848c0b2410 msgr2=0x7f848c157f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:26.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.918+0000 7f8491ea5640 1 -- 172.21.15.83:0/1906959876 shutdown_connections 2024-04-17T10:42:26.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:26.918+0000 7f8491ea5640 1 -- 172.21.15.83:0/1906959876 wait complete. 2024-04-17T10:42:26.942 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-17T10:42:27.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.066+0000 7f4cd6501640 1 Processor -- start 2024-04-17T10:42:27.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.066+0000 7f4cd6501640 1 -- start start 2024-04-17T10:42:27.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.066+0000 7f4cd6501640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4cd015bc10 0x7f4cd00d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:27.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.066+0000 7f4cd6501640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4cd006e500 0x7f4cd006e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:27.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.066+0000 7f4cd6501640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cd00ced00 con 0x7f4cd00d0490 2024-04-17T10:42:27.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.066+0000 7f4cd6501640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cd00cefe0 con 0x7f4cd006e500 2024-04-17T10:42:27.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.066+0000 7f4cd6501640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cd00cc690 con 0x7f4cd015bc10 2024-04-17T10:42:27.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.066+0000 7f4ccf7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f4cd00d0490 0x7f4cd00cce90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39718/0 (socket says 172.21.15.83:39718) 2024-04-17T10:42:27.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.066+0000 7f4ccf7fe640 1 -- 172.21.15.83:0/498596008 learned_addr learned my addr 172.21.15.83:0/498596008 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:27.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.066+0000 7f4cd4a9e640 1 --2- 172.21.15.83:0/498596008 >> v2:172.21.15.83:3300/0 conn(0x7f4cd006e500 0x7f4cd006e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:27.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.066+0000 7f4cceffd640 1 -- 172.21.15.83:0/498596008 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3924008164 0 0) 0x7f4cd00ced00 con 0x7f4cd00d0490 2024-04-17T10:42:27.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.066+0000 7f4cceffd640 1 -- 172.21.15.83:0/498596008 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cb8003530 con 0x7f4cd00d0490 2024-04-17T10:42:27.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.066+0000 7f4ccffff640 1 --2- 172.21.15.83:0/498596008 >> v2:172.21.15.167:3300/0 conn(0x7f4cd015bc10 0x7f4cd00d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:27.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.066+0000 7f4cd4a9e640 1 -- 172.21.15.83:0/498596008 >> v2:172.21.15.167:3300/0 conn(0x7f4cd015bc10 msgr2=0x7f4cd00d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:27.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.066+0000 7f4cd4a9e640 1 --2- 172.21.15.83:0/498596008 >> v2:172.21.15.167:3300/0 conn(0x7f4cd015bc10 0x7f4cd00d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:27.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.066+0000 7f4cceffd640 1 -- 172.21.15.83:0/498596008 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 805446325 0 0) 0x7f4cb8003530 con 0x7f4cd00d0490 2024-04-17T10:42:27.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.066+0000 7f4cd4a9e640 1 -- 172.21.15.83:0/498596008 >> v1:172.21.15.83:6789/0 conn(0x7f4cd00d0490 legacy=0x7f4cd00cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:27.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.066+0000 7f4cd4a9e640 1 -- 172.21.15.83:0/498596008 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd00cc970 con 0x7f4cd006e500 2024-04-17T10:42:27.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.070+0000 7f4cd4a9e640 1 --2- 172.21.15.83:0/498596008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cd006e500 0x7f4cd006e950 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f4cd00678a0 tx=0x7f4cc801c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87fdf385c3080c20 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:27.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.070+0000 7f4cceffd640 1 -- 172.21.15.83:0/498596008 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc80ac070 con 0x7f4cd006e500 2024-04-17T10:42:27.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.070+0000 7f4cceffd640 1 -- 172.21.15.83:0/498596008 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc80a6410 con 0x7f4cd006e500 2024-04-17T10:42:27.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.070+0000 7f4cceffd640 1 -- 172.21.15.83:0/498596008 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4001680 con 0x7f4cd006e500 2024-04-17T10:42:27.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.070+0000 7f4cd6501640 1 -- 172.21.15.83:0/498596008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cd006e500 msgr2=0x7f4cd006e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:27.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.070+0000 7f4cd6501640 1 --2- 172.21.15.83:0/498596008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cd006e500 0x7f4cd006e950 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f4cd00678a0 tx=0x7f4cc801c7c0 comp rx=0 tx=0).stop 2024-04-17T10:42:27.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.070+0000 7f4cd6501640 1 -- 172.21.15.83:0/498596008 shutdown_connections 2024-04-17T10:42:27.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.070+0000 7f4cd6501640 1 --2- 172.21.15.83:0/498596008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cd006e500 0x7f4cd006e950 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:27.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.070+0000 7f4cd6501640 1 --2- 172.21.15.83:0/498596008 >> v2:172.21.15.167:3300/0 conn(0x7f4cd015bc10 0x7f4cd00d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:27.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.070+0000 7f4cd6501640 1 -- 172.21.15.83:0/498596008 >> 172.21.15.83:0/498596008 conn(0x7f4cd00d17e0 msgr2=0x7f4cd00d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:27.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.070+0000 7f4cd6501640 1 -- 172.21.15.83:0/498596008 shutdown_connections 2024-04-17T10:42:27.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.070+0000 7f4cd6501640 1 -- 172.21.15.83:0/498596008 wait complete. 2024-04-17T10:42:27.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.070+0000 7f4cd6501640 1 Processor -- start 2024-04-17T10:42:27.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.070+0000 7f4cd6501640 1 -- start start 2024-04-17T10:42:27.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.074+0000 7f4cd6501640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4cd006e500 0x7f4cd00ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:27.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.074+0000 7f4cd6501640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cd00d0490 0x7f4cd0155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:27.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.074+0000 7f4cd6501640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cd015bc10 0x7f4cd00b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:27.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.074+0000 7f4cd6501640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd0168b10 con 0x7f4cd015bc10 2024-04-17T10:42:27.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.074+0000 7f4cd6501640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd0168990 con 0x7f4cd00d0490 2024-04-17T10:42:27.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.074+0000 7f4cd6501640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd0168c90 con 0x7f4cd006e500 2024-04-17T10:42:27.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.074+0000 7f4cd4a9e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cd015bc10 0x7f4cd00b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:27.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.074+0000 7f4cd4a9e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cd015bc10 0x7f4cd00b1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55168/0 (socket says 172.21.15.83:55168) 2024-04-17T10:42:27.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.074+0000 7f4cd4a9e640 1 -- 172.21.15.83:0/960200453 learned_addr learned my addr 172.21.15.83:0/960200453 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:27.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.074+0000 7f4cd4a9e640 1 -- 172.21.15.83:0/960200453 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4cd006e500 msgr2=0x7f4cd00ca9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:27.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.074+0000 7f4cd4a9e640 1 --2- 172.21.15.83:0/960200453 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4cd006e500 0x7f4cd00ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:27.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.074+0000 7f4cd4a9e640 1 -- 172.21.15.83:0/960200453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cd00d0490 msgr2=0x7f4cd0155c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:27.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.074+0000 7f4cd4a9e640 1 --2- 172.21.15.83:0/960200453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cd00d0490 0x7f4cd0155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:27.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.074+0000 7f4cd4a9e640 1 -- 172.21.15.83:0/960200453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd01561b0 con 0x7f4cd015bc10 2024-04-17T10:42:27.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.074+0000 7f4cd4a9e640 1 --2- 172.21.15.83:0/960200453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cd015bc10 0x7f4cd00b1f10 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4cc801a3f0 tx=0x7f4cc8019560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:27.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.074+0000 7f4cccff9640 1 -- 172.21.15.83:0/960200453 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc80ac070 con 0x7f4cd015bc10 2024-04-17T10:42:27.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.074+0000 7f4cccff9640 1 -- 172.21.15.83:0/960200453 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc80b1660 con 0x7f4cd015bc10 2024-04-17T10:42:27.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.074+0000 7f4cccff9640 1 -- 172.21.15.83:0/960200453 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8009450 con 0x7f4cd015bc10 2024-04-17T10:42:27.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.074+0000 7f4cd6501640 1 -- 172.21.15.83:0/960200453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cd00cb060 con 0x7f4cd015bc10 2024-04-17T10:42:27.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.074+0000 7f4cd6501640 1 -- 172.21.15.83:0/960200453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cd00cb620 con 0x7f4cd015bc10 2024-04-17T10:42:27.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.074+0000 7f4cd6501640 1 -- 172.21.15.83:0/960200453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cd0074f90 con 0x7f4cd015bc10 2024-04-17T10:42:27.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.074+0000 7f4cccff9640 1 -- 172.21.15.83:0/960200453 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4cc80af070 con 0x7f4cd015bc10 2024-04-17T10:42:27.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.074+0000 7f4cccff9640 1 --2- 172.21.15.83:0/960200453 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4cbc066f60 0x7f4cbc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:27.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.074+0000 7f4cccff9640 1 -- 172.21.15.83:0/960200453 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4cc811ec90 con 0x7f4cd015bc10 2024-04-17T10:42:27.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.074+0000 7f4ccffff640 1 --2- 172.21.15.83:0/960200453 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4cbc066f60 0x7f4cbc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:27.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.078+0000 7f4ccffff640 1 --2- 172.21.15.83:0/960200453 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4cbc066f60 0x7f4cbc069420 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4cc00098f0 tx=0x7f4cc0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:27.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.082+0000 7f4cccff9640 1 -- 172.21.15.83:0/960200453 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4cc8016440 con 0x7f4cd015bc10 2024-04-17T10:42:27.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.238+0000 7f4cd6501640 1 -- 172.21.15.83:0/960200453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f4cd00b1a90 con 0x7f4cd015bc10 2024-04-17T10:42:27.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.242+0000 7f4cccff9640 1 -- 172.21.15.83:0/960200453 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4cc80f0030 con 0x7f4cd015bc10 2024-04-17T10:42:27.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.246+0000 7f4cd6501640 1 -- 172.21.15.83:0/960200453 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4cbc066f60 msgr2=0x7f4cbc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:27.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.246+0000 7f4cd6501640 1 --2- 172.21.15.83:0/960200453 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4cbc066f60 0x7f4cbc069420 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4cc00098f0 tx=0x7f4cc0008040 comp rx=0 tx=0).stop 2024-04-17T10:42:27.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.246+0000 7f4cd6501640 1 -- 172.21.15.83:0/960200453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cd015bc10 msgr2=0x7f4cd00b1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:27.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.246+0000 7f4cd6501640 1 --2- 172.21.15.83:0/960200453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cd015bc10 0x7f4cd00b1f10 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4cc801a3f0 tx=0x7f4cc8019560 comp rx=0 tx=0).stop 2024-04-17T10:42:27.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.246+0000 7f4cd6501640 1 -- 172.21.15.83:0/960200453 shutdown_connections 2024-04-17T10:42:27.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.246+0000 7f4cd6501640 1 --2- 172.21.15.83:0/960200453 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4cbc066f60 0x7f4cbc069420 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:27.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.246+0000 7f4cd6501640 1 --2- 172.21.15.83:0/960200453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cd015bc10 0x7f4cd00b1f10 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:27.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.246+0000 7f4cd6501640 1 --2- 172.21.15.83:0/960200453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cd00d0490 0x7f4cd0155c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:27.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.246+0000 7f4cd6501640 1 --2- 172.21.15.83:0/960200453 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4cd006e500 0x7f4cd00ca9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:27.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.246+0000 7f4cd6501640 1 -- 172.21.15.83:0/960200453 >> 172.21.15.83:0/960200453 conn(0x7f4cd00d17e0 msgr2=0x7f4cd01583a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:27.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.246+0000 7f4cd6501640 1 -- 172.21.15.83:0/960200453 shutdown_connections 2024-04-17T10:42:27.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.246+0000 7f4cd6501640 1 -- 172.21.15.83:0/960200453 wait complete. 2024-04-17T10:42:27.263 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks' 2024-04-17T10:42:27.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.382+0000 7f2430465640 1 Processor -- start 2024-04-17T10:42:27.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.382+0000 7f2430465640 1 -- start start 2024-04-17T10:42:27.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.382+0000 7f2430465640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242815db50 0x7f2428070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:27.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.382+0000 7f2430465640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f242806dcb0 0x7f24280b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:27.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.382+0000 7f2430465640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24280709b0 con 0x7f24280aedd0 2024-04-17T10:42:27.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.382+0000 7f2430465640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f242815e440 con 0x7f242806dcb0 2024-04-17T10:42:27.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.382+0000 7f2430465640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242815e720 con 0x7f242815db50 2024-04-17T10:42:27.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.382+0000 7f242e201640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242815db50 0x7f2428070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:27.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.382+0000 7f242ea02640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f242806dcb0 0x7f24280b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:27.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.382+0000 7f242e201640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242815db50 0x7f2428070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:54122/0 (socket says 172.21.15.83:54122) 2024-04-17T10:42:27.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.382+0000 7f242e201640 1 -- 172.21.15.83:0/3129513105 learned_addr learned my addr 172.21.15.83:0/3129513105 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:27.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.382+0000 7f242da00640 1 --1- 172.21.15.83:0/3129513105 >> v1:172.21.15.167:6789/0 conn(0x7f24280aedd0 0x7f242806d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:48854/0 (socket says 172.21.15.83:48854) 2024-04-17T10:42:27.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.382+0000 7f242ea02640 1 -- 172.21.15.83:0/3129513105 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242815db50 msgr2=0x7f2428070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:27.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.382+0000 7f242ea02640 1 --2- 172.21.15.83:0/3129513105 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242815db50 0x7f2428070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:27.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.382+0000 7f242ea02640 1 -- 172.21.15.83:0/3129513105 >> v1:172.21.15.167:6789/0 conn(0x7f24280aedd0 legacy=0x7f242806d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:27.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.382+0000 7f242ea02640 1 -- 172.21.15.83:0/3129513105 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242815ea00 con 0x7f242806dcb0 2024-04-17T10:42:27.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.386+0000 7f242ea02640 1 --2- 172.21.15.83:0/3129513105 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f242806dcb0 0x7f24280b2e10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2420009d90 tx=0x7f242001c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e65faade0c1abc9d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:27.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.386+0000 7f242d1ff640 1 -- 172.21.15.83:0/3129513105 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24200b2070 con 0x7f242806dcb0 2024-04-17T10:42:27.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.386+0000 7f242d1ff640 1 -- 172.21.15.83:0/3129513105 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24200a5a00 con 0x7f242806dcb0 2024-04-17T10:42:27.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.386+0000 7f242d1ff640 1 -- 172.21.15.83:0/3129513105 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24200a5dd0 con 0x7f242806dcb0 2024-04-17T10:42:27.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.386+0000 7f2430465640 1 -- 172.21.15.83:0/3129513105 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f242806dcb0 msgr2=0x7f24280b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:27.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.386+0000 7f2430465640 1 --2- 172.21.15.83:0/3129513105 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f242806dcb0 0x7f24280b2e10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2420009d90 tx=0x7f242001c940 comp rx=0 tx=0).stop 2024-04-17T10:42:27.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.386+0000 7f2430465640 1 -- 172.21.15.83:0/3129513105 shutdown_connections 2024-04-17T10:42:27.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.386+0000 7f2430465640 1 --2- 172.21.15.83:0/3129513105 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f242806dcb0 0x7f24280b2e10 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:27.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.386+0000 7f2430465640 1 --2- 172.21.15.83:0/3129513105 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242815db50 0x7f2428070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:27.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.386+0000 7f2430465640 1 -- 172.21.15.83:0/3129513105 >> 172.21.15.83:0/3129513105 conn(0x7f242806abe0 msgr2=0x7f242806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:27.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.386+0000 7f2430465640 1 -- 172.21.15.83:0/3129513105 shutdown_connections 2024-04-17T10:42:27.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.386+0000 7f2430465640 1 -- 172.21.15.83:0/3129513105 wait complete. 2024-04-17T10:42:27.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.386+0000 7f2430465640 1 Processor -- start 2024-04-17T10:42:27.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.386+0000 7f2430465640 1 -- start start 2024-04-17T10:42:27.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.386+0000 7f2430465640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f242806dcb0 0x7f24280b0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:27.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.386+0000 7f2430465640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24280aedd0 0x7f24280b0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:27.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.386+0000 7f2430465640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242815db50 0x7f24281f9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:27.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.386+0000 7f2430465640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24280b3ce0 con 0x7f24280aedd0 2024-04-17T10:42:27.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.386+0000 7f2430465640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24280b3b60 con 0x7f242806dcb0 2024-04-17T10:42:27.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.386+0000 7f2430465640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24280b3e60 con 0x7f242815db50 2024-04-17T10:42:27.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.386+0000 7f242da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24280aedd0 0x7f24280b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:27.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.386+0000 7f242da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24280aedd0 0x7f24280b0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55184/0 (socket says 172.21.15.83:55184) 2024-04-17T10:42:27.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.386+0000 7f242da00640 1 -- 172.21.15.83:0/2039765843 learned_addr learned my addr 172.21.15.83:0/2039765843 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:27.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.386+0000 7f242da00640 1 -- 172.21.15.83:0/2039765843 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242815db50 msgr2=0x7f24281f9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:27.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.386+0000 7f242da00640 1 --2- 172.21.15.83:0/2039765843 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242815db50 0x7f24281f9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:27.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.386+0000 7f242da00640 1 -- 172.21.15.83:0/2039765843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f242806dcb0 msgr2=0x7f24280b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:27.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.386+0000 7f242e201640 1 --2- 172.21.15.83:0/2039765843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f242806dcb0 0x7f24280b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:27.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.386+0000 7f242da00640 1 --2- 172.21.15.83:0/2039765843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f242806dcb0 0x7f24280b0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:27.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.386+0000 7f242da00640 1 -- 172.21.15.83:0/2039765843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24280cdbd0 con 0x7f24280aedd0 2024-04-17T10:42:27.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.390+0000 7f242da00640 1 --2- 172.21.15.83:0/2039765843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24280aedd0 0x7f24280b0890 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f241800c420 tx=0x7f241800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:27.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.390+0000 7f241effd640 1 -- 172.21.15.83:0/2039765843 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24180074e0 con 0x7f24280aedd0 2024-04-17T10:42:27.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.390+0000 7f241effd640 1 -- 172.21.15.83:0/2039765843 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2418007680 con 0x7f24280aedd0 2024-04-17T10:42:27.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.390+0000 7f241effd640 1 -- 172.21.15.83:0/2039765843 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418004a30 con 0x7f24280aedd0 2024-04-17T10:42:27.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.390+0000 7f2430465640 1 -- 172.21.15.83:0/2039765843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24280cc620 con 0x7f24280aedd0 2024-04-17T10:42:27.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.390+0000 7f2430465640 1 -- 172.21.15.83:0/2039765843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24280cd7f0 con 0x7f24280aedd0 2024-04-17T10:42:27.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.390+0000 7f2430465640 1 -- 172.21.15.83:0/2039765843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24280cc410 con 0x7f24280aedd0 2024-04-17T10:42:27.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.390+0000 7f241effd640 1 -- 172.21.15.83:0/2039765843 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f241801a020 con 0x7f24280aedd0 2024-04-17T10:42:27.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.390+0000 7f241effd640 1 --2- 172.21.15.83:0/2039765843 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f240c066f60 0x7f240c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:27.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.390+0000 7f241effd640 1 -- 172.21.15.83:0/2039765843 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2418084cc0 con 0x7f24280aedd0 2024-04-17T10:42:27.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.390+0000 7f242e201640 1 --2- 172.21.15.83:0/2039765843 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f240c066f60 0x7f240c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:27.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.390+0000 7f242e201640 1 --2- 172.21.15.83:0/2039765843 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f240c066f60 0x7f240c069420 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2424009980 tx=0x7f2424008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:27.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.394+0000 7f241effd640 1 -- 172.21.15.83:0/2039765843 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2418021070 con 0x7f24280aedd0 2024-04-17T10:42:27.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.602+0000 7f2430465640 1 -- 172.21.15.83:0/2039765843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]} v 0) v1 -- 0x7f24280cdf60 con 0x7f24280aedd0 2024-04-17T10:42:27.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.914+0000 7f241effd640 1 -- 172.21.15.83:0/2039765843 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]}]=0 v204) v1 ==== 143+0+0 (secure 0 0 0) 0x7f24280709b0 con 0x7f24280aedd0 2024-04-17T10:42:27.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.914+0000 7f2430465640 1 -- 172.21.15.83:0/2039765843 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f240c066f60 msgr2=0x7f240c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:27.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.914+0000 7f2430465640 1 --2- 172.21.15.83:0/2039765843 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f240c066f60 0x7f240c069420 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2424009980 tx=0x7f2424008040 comp rx=0 tx=0).stop 2024-04-17T10:42:27.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.914+0000 7f2430465640 1 -- 172.21.15.83:0/2039765843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24280aedd0 msgr2=0x7f24280b0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:27.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.914+0000 7f2430465640 1 --2- 172.21.15.83:0/2039765843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24280aedd0 0x7f24280b0890 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f241800c420 tx=0x7f241800c8e0 comp rx=0 tx=0).stop 2024-04-17T10:42:27.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.918+0000 7f2430465640 1 -- 172.21.15.83:0/2039765843 shutdown_connections 2024-04-17T10:42:27.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.918+0000 7f2430465640 1 --2- 172.21.15.83:0/2039765843 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f240c066f60 0x7f240c069420 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:27.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.918+0000 7f2430465640 1 --2- 172.21.15.83:0/2039765843 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242815db50 0x7f24281f9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:27.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.918+0000 7f2430465640 1 --2- 172.21.15.83:0/2039765843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24280aedd0 0x7f24280b0890 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:27.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.918+0000 7f2430465640 1 --2- 172.21.15.83:0/2039765843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f242806dcb0 0x7f24280b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:27.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.918+0000 7f2430465640 1 -- 172.21.15.83:0/2039765843 >> 172.21.15.83:0/2039765843 conn(0x7f242806abe0 msgr2=0x7f24280cf420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:27.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.918+0000 7f2430465640 1 -- 172.21.15.83:0/2039765843 shutdown_connections 2024-04-17T10:42:27.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:27.918+0000 7f2430465640 1 -- 172.21.15.83:0/2039765843 wait complete. 2024-04-17T10:42:27.936 INFO:tasks.cephfs_test_runner:test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ok 2024-04-17T10:42:27.937 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_repair (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-17T10:42:27.937 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:42:28.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.074+0000 7f6b08f7e640 1 Processor -- start 2024-04-17T10:42:28.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.074+0000 7f6b08f7e640 1 -- start start 2024-04-17T10:42:28.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.074+0000 7f6b08f7e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b04161710 0x7f6b04161b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:28.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.074+0000 7f6b08f7e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b0406bd90 con 0x7f6b040cf8a0 2024-04-17T10:42:28.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.074+0000 7f6b08f7e640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b0406c070 con 0x7f6b0415dac0 2024-04-17T10:42:28.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.074+0000 7f6b08f7e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b041620b0 con 0x7f6b04161710 2024-04-17T10:42:28.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.074+0000 7f6b02d9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6b040cf8a0 0x7f6b040cfd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39724/0 (socket says 172.21.15.83:39724) 2024-04-17T10:42:28.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.074+0000 7f6b02d9d640 1 -- 172.21.15.83:0/719677274 learned_addr learned my addr 172.21.15.83:0/719677274 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:28.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b0159a640 1 -- 172.21.15.83:0/719677274 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 366230636 0 0) 0x7f6b0406bd90 con 0x7f6b040cf8a0 2024-04-17T10:42:28.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b0159a640 1 -- 172.21.15.83:0/719677274 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6aec003530 con 0x7f6b040cf8a0 2024-04-17T10:42:28.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b01d9b640 1 --2- 172.21.15.83:0/719677274 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b04161710 0x7f6b04161b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:28.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b0159a640 1 -- 172.21.15.83:0/719677274 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3057208813 0 0) 0x7f6aec003530 con 0x7f6b040cf8a0 2024-04-17T10:42:28.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b0159a640 1 -- 172.21.15.83:0/719677274 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b04161710 msgr2=0x7f6b04161b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:28.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b0159a640 1 --2- 172.21.15.83:0/719677274 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b04161710 0x7f6b04161b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:28.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b0159a640 1 -- 172.21.15.83:0/719677274 >> v1:172.21.15.167:6789/0 conn(0x7f6b0415dac0 legacy=0x7f6b0406e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:28.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b0159a640 1 -- 172.21.15.83:0/719677274 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0406c350 con 0x7f6b040cf8a0 2024-04-17T10:42:28.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b0159a640 1 -- 172.21.15.83:0/719677274 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f6afc001a00 con 0x7f6b040cf8a0 2024-04-17T10:42:28.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b0159a640 1 -- 172.21.15.83:0/719677274 >> v1:172.21.15.83:6789/0 conn(0x7f6b040cf8a0 legacy=0x7f6b040cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:28.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b0159a640 1 --2- 172.21.15.83:0/719677274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aec0039f0 0x7f6aec003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:28.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b0159a640 1 --2- 172.21.15.83:0/719677274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6aec004810 0x7f6aec004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:28.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b0159a640 1 --2- 172.21.15.83:0/719677274 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b04161710 0x7f6aec005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:28.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b0159a640 1 -- 172.21.15.83:0/719677274 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec00bad0 con 0x7f6aec0039f0 2024-04-17T10:42:28.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b0159a640 1 -- 172.21.15.83:0/719677274 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec00bc80 con 0x7f6aec004810 2024-04-17T10:42:28.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b0159a640 1 -- 172.21.15.83:0/719677274 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec00be80 con 0x7f6b04161710 2024-04-17T10:42:28.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b0259c640 1 --2- 172.21.15.83:0/719677274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aec0039f0 0x7f6aec003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:28.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b01d9b640 1 --2- 172.21.15.83:0/719677274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6aec004810 0x7f6aec004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:28.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b0259c640 1 -- 172.21.15.83:0/719677274 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b04161710 msgr2=0x7f6aec005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:28.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b0259c640 1 --2- 172.21.15.83:0/719677274 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b04161710 0x7f6aec005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:28.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b0259c640 1 -- 172.21.15.83:0/719677274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6aec004810 msgr2=0x7f6aec004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:28.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b0259c640 1 --2- 172.21.15.83:0/719677274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6aec004810 0x7f6aec004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:28.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b0259c640 1 -- 172.21.15.83:0/719677274 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aec003530 con 0x7f6aec0039f0 2024-04-17T10:42:28.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b01d9b640 1 --2- 172.21.15.83:0/719677274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6aec004810 0x7f6aec004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:28.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b0259c640 1 --2- 172.21.15.83:0/719677274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aec0039f0 0x7f6aec003e10 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f6af4001780 tx=0x7f6af401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a9879e4567b486 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:28.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b0159a640 1 -- 172.21.15.83:0/719677274 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af40013a0 con 0x7f6aec0039f0 2024-04-17T10:42:28.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b0159a640 1 -- 172.21.15.83:0/719677274 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af40052f0 con 0x7f6aec0039f0 2024-04-17T10:42:28.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b0159a640 1 -- 172.21.15.83:0/719677274 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af4003840 con 0x7f6aec0039f0 2024-04-17T10:42:28.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b08f7e640 1 -- 172.21.15.83:0/719677274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aec0039f0 msgr2=0x7f6aec003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:28.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b08f7e640 1 --2- 172.21.15.83:0/719677274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aec0039f0 0x7f6aec003e10 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f6af4001780 tx=0x7f6af401c420 comp rx=0 tx=0).stop 2024-04-17T10:42:28.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b02d9d640 1 -- 172.21.15.83:0/719677274 reap_dead start 2024-04-17T10:42:28.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b08f7e640 1 -- 172.21.15.83:0/719677274 shutdown_connections 2024-04-17T10:42:28.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b08f7e640 1 -- 172.21.15.83:0/719677274 >> 172.21.15.83:0/719677274 conn(0x7f6b040b3220 msgr2=0x7f6b040b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:28.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b08f7e640 1 -- 172.21.15.83:0/719677274 shutdown_connections 2024-04-17T10:42:28.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.078+0000 7f6b08f7e640 1 -- 172.21.15.83:0/719677274 wait complete. 2024-04-17T10:42:28.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.082+0000 7f6b08f7e640 1 Processor -- start 2024-04-17T10:42:28.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.082+0000 7f6b08f7e640 1 -- start start 2024-04-17T10:42:28.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.082+0000 7f6b08f7e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6aec0039f0 0x7f6b0415cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:28.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.082+0000 7f6b08f7e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b040d1710 0x7f6b040d1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:28.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.082+0000 7f6b08f7e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b040b5c50 0x7f6b040b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:28.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.082+0000 7f6b08f7e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b040ae500 con 0x7f6b040b5c50 2024-04-17T10:42:28.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.082+0000 7f6b08f7e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b04161f00 con 0x7f6aec0039f0 2024-04-17T10:42:28.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.082+0000 7f6b08f7e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b040ae680 con 0x7f6b040d1710 2024-04-17T10:42:28.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.082+0000 7f6b02d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b040b5c50 0x7f6b040b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:28.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.082+0000 7f6b02d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b040b5c50 0x7f6b040b60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55194/0 (socket says 172.21.15.83:55194) 2024-04-17T10:42:28.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.082+0000 7f6b02d9d640 1 -- 172.21.15.83:0/2117424395 learned_addr learned my addr 172.21.15.83:0/2117424395 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:28.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.082+0000 7f6b0259c640 1 --2- 172.21.15.83:0/2117424395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6aec0039f0 0x7f6b0415cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:28.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.082+0000 7f6b01d9b640 1 --2- 172.21.15.83:0/2117424395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b040d1710 0x7f6b040d1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:28.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.082+0000 7f6b02d9d640 1 -- 172.21.15.83:0/2117424395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b040d1710 msgr2=0x7f6b040d1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:28.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.082+0000 7f6b02d9d640 1 --2- 172.21.15.83:0/2117424395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b040d1710 0x7f6b040d1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:28.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.082+0000 7f6b02d9d640 1 -- 172.21.15.83:0/2117424395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6aec0039f0 msgr2=0x7f6b0415cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:28.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.082+0000 7f6b02d9d640 1 --2- 172.21.15.83:0/2117424395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6aec0039f0 0x7f6b0415cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:28.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.082+0000 7f6b02d9d640 1 -- 172.21.15.83:0/2117424395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b040b43d0 con 0x7f6b040b5c50 2024-04-17T10:42:28.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.082+0000 7f6b02d9d640 1 --2- 172.21.15.83:0/2117424395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b040b5c50 0x7f6b040b60d0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f6afc001340 tx=0x7f6afc00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:28.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.082+0000 7f6af2ffd640 1 -- 172.21.15.83:0/2117424395 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc007110 con 0x7f6b040b5c50 2024-04-17T10:42:28.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.082+0000 7f6af2ffd640 1 -- 172.21.15.83:0/2117424395 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6afc0072b0 con 0x7f6b040b5c50 2024-04-17T10:42:28.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.082+0000 7f6af2ffd640 1 -- 172.21.15.83:0/2117424395 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc0076a0 con 0x7f6b040b5c50 2024-04-17T10:42:28.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.082+0000 7f6b08f7e640 1 -- 172.21.15.83:0/2117424395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b040b46b0 con 0x7f6b040b5c50 2024-04-17T10:42:28.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.082+0000 7f6b08f7e640 1 -- 172.21.15.83:0/2117424395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b040b4c70 con 0x7f6b040b5c50 2024-04-17T10:42:28.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.082+0000 7f6b08f7e640 1 -- 172.21.15.83:0/2117424395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b04074f90 con 0x7f6b040b5c50 2024-04-17T10:42:28.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.086+0000 7f6af2ffd640 1 -- 172.21.15.83:0/2117424395 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6afc011070 con 0x7f6b040b5c50 2024-04-17T10:42:28.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.086+0000 7f6af2ffd640 1 --2- 172.21.15.83:0/2117424395 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6ae4066f90 0x7f6ae4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:28.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.086+0000 7f6af2ffd640 1 -- 172.21.15.83:0/2117424395 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6afc084350 con 0x7f6b040b5c50 2024-04-17T10:42:28.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.086+0000 7f6b0259c640 1 --2- 172.21.15.83:0/2117424395 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6ae4066f90 0x7f6ae4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:28.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.086+0000 7f6b0259c640 1 --2- 172.21.15.83:0/2117424395 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6ae4066f90 0x7f6ae4069450 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6af4001750 tx=0x7f6af40011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:28.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.090+0000 7f6af2ffd640 1 -- 172.21.15.83:0/2117424395 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6afc0556f0 con 0x7f6b040b5c50 2024-04-17T10:42:28.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.234+0000 7f6b08f7e640 1 -- 172.21.15.83:0/2117424395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b040730a0 con 0x7f6b040b5c50 2024-04-17T10:42:28.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.238+0000 7f6af2ffd640 1 -- 172.21.15.83:0/2117424395 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+12948 (secure 0 0 0) 0x7f6afc054ff0 con 0x7f6b040b5c50 2024-04-17T10:42:28.237 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:28.237 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":51,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:42:26.907710+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/1894547386":"2024-04-18T10:42:23.891666+0000","172.21.15.83:6836/1894547386":"2024-04-18T10:42:23.891666+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:42:28.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.238+0000 7f6b08f7e640 1 -- 172.21.15.83:0/2117424395 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6ae4066f90 msgr2=0x7f6ae4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:28.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.238+0000 7f6b08f7e640 1 --2- 172.21.15.83:0/2117424395 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6ae4066f90 0x7f6ae4069450 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6af4001750 tx=0x7f6af40011a0 comp rx=0 tx=0).stop 2024-04-17T10:42:28.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.238+0000 7f6b08f7e640 1 -- 172.21.15.83:0/2117424395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b040b5c50 msgr2=0x7f6b040b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:28.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.238+0000 7f6b08f7e640 1 --2- 172.21.15.83:0/2117424395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b040b5c50 0x7f6b040b60d0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f6afc001340 tx=0x7f6afc00ccd0 comp rx=0 tx=0).stop 2024-04-17T10:42:28.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.238+0000 7f6b08f7e640 1 -- 172.21.15.83:0/2117424395 shutdown_connections 2024-04-17T10:42:28.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.238+0000 7f6b08f7e640 1 --2- 172.21.15.83:0/2117424395 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6ae4066f90 0x7f6ae4069450 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:28.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.238+0000 7f6b08f7e640 1 --2- 172.21.15.83:0/2117424395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b040b5c50 0x7f6b040b60d0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:28.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.238+0000 7f6b08f7e640 1 --2- 172.21.15.83:0/2117424395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b040d1710 0x7f6b040d1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:28.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.238+0000 7f6b08f7e640 1 --2- 172.21.15.83:0/2117424395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6aec0039f0 0x7f6b0415cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:28.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.238+0000 7f6b08f7e640 1 -- 172.21.15.83:0/2117424395 >> 172.21.15.83:0/2117424395 conn(0x7f6b040b3220 msgr2=0x7f6b04168830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:28.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.238+0000 7f6b08f7e640 1 -- 172.21.15.83:0/2117424395 shutdown_connections 2024-04-17T10:42:28.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.242+0000 7f6b08f7e640 1 -- 172.21.15.83:0/2117424395 wait complete. 2024-04-17T10:42:28.255 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:42:28.255 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:42:28.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.378+0000 7fb098866640 1 Processor -- start 2024-04-17T10:42:28.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.378+0000 7fb098866640 1 -- start start 2024-04-17T10:42:28.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.378+0000 7fb098866640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb0940af150 0x7fb0940af530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:28.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.378+0000 7fb098866640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb094072210 0x7fb09415e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:28.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.378+0000 7fb098866640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb094072940 0x7fb09415dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:28.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.378+0000 7fb098866640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb094071d80 con 0x7fb094072940 2024-04-17T10:42:28.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.378+0000 7fb098866640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb094071170 con 0x7fb094072210 2024-04-17T10:42:28.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.378+0000 7fb098866640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb094071450 con 0x7fb0940af150 2024-04-17T10:42:28.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.378+0000 7fb092d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb094072940 0x7fb09415dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:28.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.378+0000 7fb092d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb094072940 0x7fb09415dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55204/0 (socket says 172.21.15.83:55204) 2024-04-17T10:42:28.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.378+0000 7fb092d9d640 1 -- 172.21.15.83:0/1337736685 learned_addr learned my addr 172.21.15.83:0/1337736685 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:28.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.378+0000 7fb092d9d640 1 -- 172.21.15.83:0/1337736685 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb0940af150 msgr2=0x7fb0940af530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:28.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.378+0000 7fb092d9d640 1 --2- 172.21.15.83:0/1337736685 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb0940af150 0x7fb0940af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:28.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.378+0000 7fb092d9d640 1 -- 172.21.15.83:0/1337736685 >> v2:172.21.15.167:3300/0 conn(0x7fb094072210 msgr2=0x7fb09415e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:28.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.378+0000 7fb092d9d640 1 --2- 172.21.15.83:0/1337736685 >> v2:172.21.15.167:3300/0 conn(0x7fb094072210 0x7fb09415e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:28.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.378+0000 7fb092d9d640 1 -- 172.21.15.83:0/1337736685 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb094071730 con 0x7fb094072940 2024-04-17T10:42:28.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb092d9d640 1 --2- 172.21.15.83:0/1337736685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb094072940 0x7fb09415dd50 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb0940684e0 tx=0x7fb08c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd5560636b29cc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:28.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb09159a640 1 -- 172.21.15.83:0/1337736685 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c0a5750 con 0x7fb094072940 2024-04-17T10:42:28.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb09159a640 1 -- 172.21.15.83:0/1337736685 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08c0a58f0 con 0x7fb094072940 2024-04-17T10:42:28.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb09159a640 1 -- 172.21.15.83:0/1337736685 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c0a5cc0 con 0x7fb094072940 2024-04-17T10:42:28.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb098866640 1 -- 172.21.15.83:0/1337736685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb094072940 msgr2=0x7fb09415dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:28.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb098866640 1 --2- 172.21.15.83:0/1337736685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb094072940 0x7fb09415dd50 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb0940684e0 tx=0x7fb08c01c9c0 comp rx=0 tx=0).stop 2024-04-17T10:42:28.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb098866640 1 -- 172.21.15.83:0/1337736685 shutdown_connections 2024-04-17T10:42:28.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb098866640 1 --2- 172.21.15.83:0/1337736685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb094072940 0x7fb09415dd50 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:28.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb098866640 1 --2- 172.21.15.83:0/1337736685 >> v2:172.21.15.167:3300/0 conn(0x7fb094072210 0x7fb09415e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:28.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb098866640 1 --2- 172.21.15.83:0/1337736685 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb0940af150 0x7fb0940af530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:28.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb098866640 1 -- 172.21.15.83:0/1337736685 >> 172.21.15.83:0/1337736685 conn(0x7fb09406abe0 msgr2=0x7fb09406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:28.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb098866640 1 -- 172.21.15.83:0/1337736685 shutdown_connections 2024-04-17T10:42:28.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb098866640 1 -- 172.21.15.83:0/1337736685 wait complete. 2024-04-17T10:42:28.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb098866640 1 Processor -- start 2024-04-17T10:42:28.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb098866640 1 -- start start 2024-04-17T10:42:28.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb098866640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb094072210 0x7fb0941fb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:28.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb098866640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb094072940 0x7fb09414ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:28.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb098866640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0940af150 0x7fb094153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:28.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb098866640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0940c67a0 con 0x7fb0940af150 2024-04-17T10:42:28.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb098866640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0940ce3f0 con 0x7fb094072940 2024-04-17T10:42:28.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb098866640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0941fb700 con 0x7fb094072210 2024-04-17T10:42:28.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb092d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0940af150 0x7fb094153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:28.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb092d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0940af150 0x7fb094153cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55208/0 (socket says 172.21.15.83:55208) 2024-04-17T10:42:28.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb092d9d640 1 -- 172.21.15.83:0/130684419 learned_addr learned my addr 172.21.15.83:0/130684419 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:28.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb091d9b640 1 --2- 172.21.15.83:0/130684419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb094072940 0x7fb09414ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:28.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb092d9d640 1 -- 172.21.15.83:0/130684419 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb094072210 msgr2=0x7fb0941fb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:28.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb09259c640 1 --2- 172.21.15.83:0/130684419 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb094072210 0x7fb0941fb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:28.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb092d9d640 1 --2- 172.21.15.83:0/130684419 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb094072210 0x7fb0941fb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:28.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb092d9d640 1 -- 172.21.15.83:0/130684419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb094072940 msgr2=0x7fb09414ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:28.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb092d9d640 1 --2- 172.21.15.83:0/130684419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb094072940 0x7fb09414ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:28.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb092d9d640 1 -- 172.21.15.83:0/130684419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb094154720 con 0x7fb0940af150 2024-04-17T10:42:28.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb091d9b640 1 --2- 172.21.15.83:0/130684419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb094072940 0x7fb09414ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:28.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.382+0000 7fb092d9d640 1 --2- 172.21.15.83:0/130684419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0940af150 0x7fb094153cb0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb08c01b070 tx=0x7fb08c019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:28.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.386+0000 7fb082ffd640 1 -- 172.21.15.83:0/130684419 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c0bc020 con 0x7fb0940af150 2024-04-17T10:42:28.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.386+0000 7fb082ffd640 1 -- 172.21.15.83:0/130684419 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08c0b8670 con 0x7fb0940af150 2024-04-17T10:42:28.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.386+0000 7fb082ffd640 1 -- 172.21.15.83:0/130684419 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c0b8a60 con 0x7fb0940af150 2024-04-17T10:42:28.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.386+0000 7fb098866640 1 -- 172.21.15.83:0/130684419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb094154a00 con 0x7fb0940af150 2024-04-17T10:42:28.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.386+0000 7fb098866640 1 -- 172.21.15.83:0/130684419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0941faca0 con 0x7fb0940af150 2024-04-17T10:42:28.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.386+0000 7fb098866640 1 -- 172.21.15.83:0/130684419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb094074f90 con 0x7fb0940af150 2024-04-17T10:42:28.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.386+0000 7fb082ffd640 1 -- 172.21.15.83:0/130684419 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb08c0bf020 con 0x7fb0940af150 2024-04-17T10:42:28.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.386+0000 7fb082ffd640 1 --2- 172.21.15.83:0/130684419 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb074066f60 0x7fb074069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:28.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.386+0000 7fb082ffd640 1 -- 172.21.15.83:0/130684419 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb08c11feb0 con 0x7fb0940af150 2024-04-17T10:42:28.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.386+0000 7fb09259c640 1 --2- 172.21.15.83:0/130684419 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb074066f60 0x7fb074069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:28.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.386+0000 7fb09259c640 1 --2- 172.21.15.83:0/130684419 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb074066f60 0x7fb074069420 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb08400a830 tx=0x7fb084008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:28.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.390+0000 7fb082ffd640 1 -- 172.21.15.83:0/130684419 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb08c0c6030 con 0x7fb0940af150 2024-04-17T10:42:28.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.538+0000 7fb098866640 1 -- 172.21.15.83:0/130684419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb0941fae70 con 0x7fb0940af150 2024-04-17T10:42:28.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.538+0000 7fb082ffd640 1 -- 172.21.15.83:0/130684419 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb08c0165e0 con 0x7fb0940af150 2024-04-17T10:42:28.538 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:42:28.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.542+0000 7fb098866640 1 -- 172.21.15.83:0/130684419 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb074066f60 msgr2=0x7fb074069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:28.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.542+0000 7fb098866640 1 --2- 172.21.15.83:0/130684419 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb074066f60 0x7fb074069420 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb08400a830 tx=0x7fb084008040 comp rx=0 tx=0).stop 2024-04-17T10:42:28.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.542+0000 7fb098866640 1 -- 172.21.15.83:0/130684419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0940af150 msgr2=0x7fb094153cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:28.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.542+0000 7fb098866640 1 --2- 172.21.15.83:0/130684419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0940af150 0x7fb094153cb0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb08c01b070 tx=0x7fb08c019760 comp rx=0 tx=0).stop 2024-04-17T10:42:28.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.542+0000 7fb098866640 1 -- 172.21.15.83:0/130684419 shutdown_connections 2024-04-17T10:42:28.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.542+0000 7fb098866640 1 --2- 172.21.15.83:0/130684419 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb074066f60 0x7fb074069420 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:28.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.542+0000 7fb098866640 1 --2- 172.21.15.83:0/130684419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0940af150 0x7fb094153cb0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:28.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.542+0000 7fb098866640 1 --2- 172.21.15.83:0/130684419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb094072940 0x7fb09414ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:28.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.542+0000 7fb098866640 1 --2- 172.21.15.83:0/130684419 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb094072210 0x7fb0941fb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:28.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.542+0000 7fb098866640 1 -- 172.21.15.83:0/130684419 >> 172.21.15.83:0/130684419 conn(0x7fb09406abe0 msgr2=0x7fb094159970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:28.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.542+0000 7fb098866640 1 -- 172.21.15.83:0/130684419 shutdown_connections 2024-04-17T10:42:28.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.542+0000 7fb098866640 1 -- 172.21.15.83:0/130684419 wait complete. 2024-04-17T10:42:28.555 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair' 2024-04-17T10:42:28.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.674+0000 7f0cea1f5640 1 Processor -- start 2024-04-17T10:42:28.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.678+0000 7f0cea1f5640 1 -- start start 2024-04-17T10:42:28.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.678+0000 7f0cea1f5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0ce415ba30 0x7f0ce4072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:28.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.678+0000 7f0cea1f5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ce40cfbe0 0x7f0ce406c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:28.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.678+0000 7f0cea1f5640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce406e320 con 0x7f0ce415c960 2024-04-17T10:42:28.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.678+0000 7f0cea1f5640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce4071ea0 con 0x7f0ce415ba30 2024-04-17T10:42:28.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.678+0000 7f0cea1f5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce406e040 con 0x7f0ce40cfbe0 2024-04-17T10:42:28.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.678+0000 7f0cdaffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0ce415c960 0x7f0ce40cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39730/0 (socket says 172.21.15.83:39730) 2024-04-17T10:42:28.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.678+0000 7f0ce37fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0ce415ba30 0x7f0ce4072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:28.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.678+0000 7f0cdaffd640 1 -- 172.21.15.83:0/398664750 learned_addr learned my addr 172.21.15.83:0/398664750 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:28.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.678+0000 7f0ce3fff640 1 --2- 172.21.15.83:0/398664750 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ce40cfbe0 0x7f0ce406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:28.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.678+0000 7f0ce2ffd640 1 -- 172.21.15.83:0/398664750 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3890005123 0 0) 0x7f0ce406e320 con 0x7f0ce415c960 2024-04-17T10:42:28.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.678+0000 7f0ce2ffd640 1 -- 172.21.15.83:0/398664750 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cd0003530 con 0x7f0ce415c960 2024-04-17T10:42:28.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.678+0000 7f0ce37fe640 1 -- 172.21.15.83:0/398664750 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ce40cfbe0 msgr2=0x7f0ce406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:28.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.678+0000 7f0ce37fe640 1 --2- 172.21.15.83:0/398664750 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ce40cfbe0 0x7f0ce406c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:28.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.678+0000 7f0ce37fe640 1 -- 172.21.15.83:0/398664750 >> v1:172.21.15.83:6789/0 conn(0x7f0ce415c960 legacy=0x7f0ce40cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:28.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.678+0000 7f0ce37fe640 1 -- 172.21.15.83:0/398664750 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce4071850 con 0x7f0ce415ba30 2024-04-17T10:42:28.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.678+0000 7f0ce37fe640 1 --2- 172.21.15.83:0/398664750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ce415ba30 0x7f0ce4072330 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40097c0 tx=0x7f0cd401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69600a5b1b1143a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:28.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0ce2ffd640 1 -- 172.21.15.83:0/398664750 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd40b0070 con 0x7f0ce415ba30 2024-04-17T10:42:28.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0ce2ffd640 1 -- 172.21.15.83:0/398664750 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd40ab600 con 0x7f0ce415ba30 2024-04-17T10:42:28.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0ce2ffd640 1 -- 172.21.15.83:0/398664750 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd40ab9b0 con 0x7f0ce415ba30 2024-04-17T10:42:28.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0cea1f5640 1 -- 172.21.15.83:0/398664750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ce415ba30 msgr2=0x7f0ce4072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:28.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0cea1f5640 1 --2- 172.21.15.83:0/398664750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ce415ba30 0x7f0ce4072330 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40097c0 tx=0x7f0cd401c410 comp rx=0 tx=0).stop 2024-04-17T10:42:28.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0cea1f5640 1 -- 172.21.15.83:0/398664750 shutdown_connections 2024-04-17T10:42:28.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0cea1f5640 1 --2- 172.21.15.83:0/398664750 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ce40cfbe0 0x7f0ce406c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:28.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0cea1f5640 1 --2- 172.21.15.83:0/398664750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ce415ba30 0x7f0ce4072330 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:28.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0cea1f5640 1 -- 172.21.15.83:0/398664750 >> 172.21.15.83:0/398664750 conn(0x7f0ce406abe0 msgr2=0x7f0ce406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:28.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0cea1f5640 1 -- 172.21.15.83:0/398664750 shutdown_connections 2024-04-17T10:42:28.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0cea1f5640 1 -- 172.21.15.83:0/398664750 wait complete. 2024-04-17T10:42:28.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0cea1f5640 1 Processor -- start 2024-04-17T10:42:28.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0cea1f5640 1 -- start start 2024-04-17T10:42:28.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0cea1f5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ce40cfbe0 0x7f0ce414f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:28.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0cea1f5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ce415ba30 0x7f0ce414a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:28.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0cea1f5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ce415c960 0x7f0ce415d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:28.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0cea1f5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce415cee0 con 0x7f0ce415c960 2024-04-17T10:42:28.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0cea1f5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce415cd60 con 0x7f0ce415ba30 2024-04-17T10:42:28.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0cea1f5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce4168520 con 0x7f0ce40cfbe0 2024-04-17T10:42:28.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0ce3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ce415c960 0x7f0ce415d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:28.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0ce3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ce415c960 0x7f0ce415d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55222/0 (socket says 172.21.15.83:55222) 2024-04-17T10:42:28.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0ce3fff640 1 -- 172.21.15.83:0/3301697141 learned_addr learned my addr 172.21.15.83:0/3301697141 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:28.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0ce37fe640 1 --2- 172.21.15.83:0/3301697141 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ce40cfbe0 0x7f0ce414f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:28.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0cdaffd640 1 --2- 172.21.15.83:0/3301697141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ce415ba30 0x7f0ce414a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:28.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0ce3fff640 1 -- 172.21.15.83:0/3301697141 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ce40cfbe0 msgr2=0x7f0ce414f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:28.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0ce3fff640 1 --2- 172.21.15.83:0/3301697141 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ce40cfbe0 0x7f0ce414f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:28.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0ce3fff640 1 -- 172.21.15.83:0/3301697141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ce415ba30 msgr2=0x7f0ce414a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:28.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0ce3fff640 1 --2- 172.21.15.83:0/3301697141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ce415ba30 0x7f0ce414a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:28.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0ce3fff640 1 -- 172.21.15.83:0/3301697141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce414a960 con 0x7f0ce415c960 2024-04-17T10:42:28.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0cdaffd640 1 --2- 172.21.15.83:0/3301697141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ce415ba30 0x7f0ce414a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:28.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0ce3fff640 1 --2- 172.21.15.83:0/3301697141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ce415c960 0x7f0ce415d2e0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc007e60 tx=0x7f0cdc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:28.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0ce0ff9640 1 -- 172.21.15.83:0/3301697141 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc014420 con 0x7f0ce415c960 2024-04-17T10:42:28.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0ce0ff9640 1 -- 172.21.15.83:0/3301697141 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cdc019070 con 0x7f0ce415c960 2024-04-17T10:42:28.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0cea1f5640 1 -- 172.21.15.83:0/3301697141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce414ac40 con 0x7f0ce415c960 2024-04-17T10:42:28.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0ce0ff9640 1 -- 172.21.15.83:0/3301697141 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc014820 con 0x7f0ce415c960 2024-04-17T10:42:28.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.682+0000 7f0cea1f5640 1 -- 172.21.15.83:0/3301697141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce415d820 con 0x7f0ce415c960 2024-04-17T10:42:28.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.686+0000 7f0cea1f5640 1 -- 172.21.15.83:0/3301697141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce4074f90 con 0x7f0ce415c960 2024-04-17T10:42:28.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.686+0000 7f0ce0ff9640 1 -- 172.21.15.83:0/3301697141 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0cdc0149c0 con 0x7f0ce415c960 2024-04-17T10:42:28.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.686+0000 7f0ce0ff9640 1 --2- 172.21.15.83:0/3301697141 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0cc4066f60 0x7f0cc4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:28.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.686+0000 7f0ce0ff9640 1 -- 172.21.15.83:0/3301697141 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0cdc085540 con 0x7f0ce415c960 2024-04-17T10:42:28.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.686+0000 7f0ce37fe640 1 --2- 172.21.15.83:0/3301697141 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0cc4066f60 0x7f0cc4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:28.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.686+0000 7f0ce37fe640 1 --2- 172.21.15.83:0/3301697141 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0cc4066f60 0x7f0cc4069420 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40ad000 tx=0x7f0cd40a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:28.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.690+0000 7f0ce0ff9640 1 -- 172.21.15.83:0/3301697141 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0cdc0568e0 con 0x7f0ce415c960 2024-04-17T10:42:28.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.890+0000 7f0cea1f5640 1 -- 172.21.15.83:0/3301697141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]} v 0) v1 -- 0x7f0ce414b0f0 con 0x7f0ce415c960 2024-04-17T10:42:29.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.922+0000 7f0ce0ff9640 1 -- 172.21.15.83:0/3301697141 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]}]=0 v205) v1 ==== 146+0+0 (secure 0 0 0) 0x7f0cdc0561e0 con 0x7f0ce415c960 2024-04-17T10:42:29.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.922+0000 7f0cea1f5640 1 -- 172.21.15.83:0/3301697141 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0cc4066f60 msgr2=0x7f0cc4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:29.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.922+0000 7f0cea1f5640 1 --2- 172.21.15.83:0/3301697141 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0cc4066f60 0x7f0cc4069420 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40ad000 tx=0x7f0cd40a6000 comp rx=0 tx=0).stop 2024-04-17T10:42:29.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.922+0000 7f0cea1f5640 1 -- 172.21.15.83:0/3301697141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ce415c960 msgr2=0x7f0ce415d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:29.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.926+0000 7f0cea1f5640 1 --2- 172.21.15.83:0/3301697141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ce415c960 0x7f0ce415d2e0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc007e60 tx=0x7f0cdc00e4b0 comp rx=0 tx=0).stop 2024-04-17T10:42:29.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.926+0000 7f0cea1f5640 1 -- 172.21.15.83:0/3301697141 shutdown_connections 2024-04-17T10:42:29.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.926+0000 7f0cea1f5640 1 --2- 172.21.15.83:0/3301697141 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0cc4066f60 0x7f0cc4069420 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:29.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.926+0000 7f0cea1f5640 1 --2- 172.21.15.83:0/3301697141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ce415c960 0x7f0ce415d2e0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:29.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.926+0000 7f0cea1f5640 1 --2- 172.21.15.83:0/3301697141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ce415ba30 0x7f0ce414a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:29.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.926+0000 7f0cea1f5640 1 --2- 172.21.15.83:0/3301697141 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ce40cfbe0 0x7f0ce414f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:29.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.926+0000 7f0cea1f5640 1 -- 172.21.15.83:0/3301697141 >> 172.21.15.83:0/3301697141 conn(0x7f0ce406abe0 msgr2=0x7f0ce415e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:29.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.926+0000 7f0cea1f5640 1 -- 172.21.15.83:0/3301697141 shutdown_connections 2024-04-17T10:42:29.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:28.926+0000 7f0cea1f5640 1 -- 172.21.15.83:0/3301697141 wait complete. 2024-04-17T10:42:29.126 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-17T10:42:29.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.258+0000 7f9085815640 1 Processor -- start 2024-04-17T10:42:29.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.258+0000 7f9085815640 1 -- start start 2024-04-17T10:42:29.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.258+0000 7f9085815640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f908006ae10 0x7f908006b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:29.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.258+0000 7f9085815640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f908015e330 0x7f908015e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:29.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.258+0000 7f9085815640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90801625d0 con 0x7f90800ad840 2024-04-17T10:42:29.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.258+0000 7f9085815640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f908006b730 con 0x7f908015e330 2024-04-17T10:42:29.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.258+0000 7f9085815640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9080161810 con 0x7f908006ae10 2024-04-17T10:42:29.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.258+0000 7f907effd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f908006ae10 0x7f908006b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:29.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.258+0000 7f907effd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f908006ae10 0x7f908006b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:54212/0 (socket says 172.21.15.83:54212) 2024-04-17T10:42:29.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.258+0000 7f907effd640 1 -- 172.21.15.83:0/1234879193 learned_addr learned my addr 172.21.15.83:0/1234879193 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:29.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.258+0000 7f907e7fc640 1 --2- 172.21.15.83:0/1234879193 >> v2:172.21.15.167:3300/0 conn(0x7f908015e330 0x7f908015e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:29.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.258+0000 7f907e7fc640 1 -- 172.21.15.83:0/1234879193 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f908006ae10 msgr2=0x7f908006b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:29.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.258+0000 7f907e7fc640 1 --2- 172.21.15.83:0/1234879193 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f908006ae10 0x7f908006b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:29.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.258+0000 7f907e7fc640 1 -- 172.21.15.83:0/1234879193 >> v1:172.21.15.167:6789/0 conn(0x7f90800ad840 legacy=0x7f90800adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:29.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.258+0000 7f907e7fc640 1 -- 172.21.15.83:0/1234879193 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9080161af0 con 0x7f908015e330 2024-04-17T10:42:29.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.262+0000 7f907e7fc640 1 --2- 172.21.15.83:0/1234879193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f908015e330 0x7f908015e770 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f90740097c0 tx=0x7f907401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c17f714aa5d33601 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:29.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.262+0000 7f907dffb640 1 -- 172.21.15.83:0/1234879193 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90740b0070 con 0x7f908015e330 2024-04-17T10:42:29.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.262+0000 7f907dffb640 1 -- 172.21.15.83:0/1234879193 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90740ab600 con 0x7f908015e330 2024-04-17T10:42:29.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.262+0000 7f907dffb640 1 -- 172.21.15.83:0/1234879193 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90740ab9d0 con 0x7f908015e330 2024-04-17T10:42:29.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.262+0000 7f9085815640 1 -- 172.21.15.83:0/1234879193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f908015e330 msgr2=0x7f908015e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:29.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.262+0000 7f9085815640 1 --2- 172.21.15.83:0/1234879193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f908015e330 0x7f908015e770 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f90740097c0 tx=0x7f907401c410 comp rx=0 tx=0).stop 2024-04-17T10:42:29.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.262+0000 7f9085815640 1 -- 172.21.15.83:0/1234879193 shutdown_connections 2024-04-17T10:42:29.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.262+0000 7f9085815640 1 --2- 172.21.15.83:0/1234879193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f908015e330 0x7f908015e770 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:29.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.262+0000 7f9085815640 1 --2- 172.21.15.83:0/1234879193 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f908006ae10 0x7f908006b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:29.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.262+0000 7f9085815640 1 -- 172.21.15.83:0/1234879193 >> 172.21.15.83:0/1234879193 conn(0x7f90800b15b0 msgr2=0x7f90800b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:29.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.262+0000 7f9085815640 1 -- 172.21.15.83:0/1234879193 shutdown_connections 2024-04-17T10:42:29.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.262+0000 7f9085815640 1 -- 172.21.15.83:0/1234879193 wait complete. 2024-04-17T10:42:29.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.262+0000 7f9085815640 1 Processor -- start 2024-04-17T10:42:29.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.262+0000 7f9085815640 1 -- start start 2024-04-17T10:42:29.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.266+0000 7f9085815640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f908006ae10 0x7f908013af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:29.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.266+0000 7f9085815640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90800ad840 0x7f908013b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:29.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.266+0000 7f9085815640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f908015e330 0x7f908013bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:29.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.266+0000 7f9085815640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90800cf470 con 0x7f908006ae10 2024-04-17T10:42:29.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.266+0000 7f9085815640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90800cf2f0 con 0x7f908015e330 2024-04-17T10:42:29.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.266+0000 7f9085815640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f908015ecb0 con 0x7f90800ad840 2024-04-17T10:42:29.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.266+0000 7f907effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f908006ae10 0x7f908013af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:29.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.266+0000 7f907effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f908006ae10 0x7f908013af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55226/0 (socket says 172.21.15.83:55226) 2024-04-17T10:42:29.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.266+0000 7f907effd640 1 -- 172.21.15.83:0/3646984982 learned_addr learned my addr 172.21.15.83:0/3646984982 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:29.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.266+0000 7f907f7fe640 1 --2- 172.21.15.83:0/3646984982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f908015e330 0x7f908013bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:29.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.266+0000 7f907e7fc640 1 --2- 172.21.15.83:0/3646984982 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90800ad840 0x7f908013b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:29.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.266+0000 7f907effd640 1 -- 172.21.15.83:0/3646984982 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90800ad840 msgr2=0x7f908013b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:29.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.266+0000 7f907effd640 1 --2- 172.21.15.83:0/3646984982 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90800ad840 0x7f908013b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:29.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.266+0000 7f907effd640 1 -- 172.21.15.83:0/3646984982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f908015e330 msgr2=0x7f908013bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:29.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.266+0000 7f907effd640 1 --2- 172.21.15.83:0/3646984982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f908015e330 0x7f908013bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:29.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.266+0000 7f907effd640 1 -- 172.21.15.83:0/3646984982 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9080145970 con 0x7f908006ae10 2024-04-17T10:42:29.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.266+0000 7f907effd640 1 --2- 172.21.15.83:0/3646984982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f908006ae10 0x7f908013af70 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f907000efc0 tx=0x7f907000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:29.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.266+0000 7f9067fff640 1 -- 172.21.15.83:0/3646984982 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907000cda0 con 0x7f908006ae10 2024-04-17T10:42:29.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.266+0000 7f9067fff640 1 -- 172.21.15.83:0/3646984982 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9070019070 con 0x7f908006ae10 2024-04-17T10:42:29.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.266+0000 7f9067fff640 1 -- 172.21.15.83:0/3646984982 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90700146a0 con 0x7f908006ae10 2024-04-17T10:42:29.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.266+0000 7f9085815640 1 -- 172.21.15.83:0/3646984982 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9080145c50 con 0x7f908006ae10 2024-04-17T10:42:29.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.266+0000 7f9085815640 1 -- 172.21.15.83:0/3646984982 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9080146210 con 0x7f908006ae10 2024-04-17T10:42:29.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.266+0000 7f9085815640 1 -- 172.21.15.83:0/3646984982 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9080074f90 con 0x7f908006ae10 2024-04-17T10:42:29.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.266+0000 7f9067fff640 1 -- 172.21.15.83:0/3646984982 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9070012050 con 0x7f908006ae10 2024-04-17T10:42:29.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.266+0000 7f9067fff640 1 --2- 172.21.15.83:0/3646984982 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f906c066f60 0x7f906c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:29.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.266+0000 7f9067fff640 1 -- 172.21.15.83:0/3646984982 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9070085170 con 0x7f908006ae10 2024-04-17T10:42:29.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.266+0000 7f907e7fc640 1 --2- 172.21.15.83:0/3646984982 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f906c066f60 0x7f906c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:29.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.266+0000 7f907e7fc640 1 --2- 172.21.15.83:0/3646984982 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f906c066f60 0x7f906c069420 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f90740ad000 tx=0x7f9074008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:29.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.270+0000 7f9067fff640 1 -- 172.21.15.83:0/3646984982 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9070010040 con 0x7f908006ae10 2024-04-17T10:42:29.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.430+0000 7f9085815640 1 -- 172.21.15.83:0/3646984982 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9080072740 con 0x7f908006ae10 2024-04-17T10:42:29.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.438+0000 7f9067fff640 1 -- 172.21.15.83:0/3646984982 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9070056510 con 0x7f908006ae10 2024-04-17T10:42:29.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.438+0000 7f9085815640 1 -- 172.21.15.83:0/3646984982 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f906c066f60 msgr2=0x7f906c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:29.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.438+0000 7f9085815640 1 --2- 172.21.15.83:0/3646984982 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f906c066f60 0x7f906c069420 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f90740ad000 tx=0x7f9074008040 comp rx=0 tx=0).stop 2024-04-17T10:42:29.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.438+0000 7f9085815640 1 -- 172.21.15.83:0/3646984982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f908006ae10 msgr2=0x7f908013af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:29.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.438+0000 7f9085815640 1 --2- 172.21.15.83:0/3646984982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f908006ae10 0x7f908013af70 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f907000efc0 tx=0x7f907000c490 comp rx=0 tx=0).stop 2024-04-17T10:42:29.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.438+0000 7f9085815640 1 -- 172.21.15.83:0/3646984982 shutdown_connections 2024-04-17T10:42:29.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.438+0000 7f9085815640 1 --2- 172.21.15.83:0/3646984982 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f906c066f60 0x7f906c069420 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:29.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.438+0000 7f9085815640 1 --2- 172.21.15.83:0/3646984982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f908015e330 0x7f908013bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:29.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.438+0000 7f9085815640 1 --2- 172.21.15.83:0/3646984982 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90800ad840 0x7f908013b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:29.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.438+0000 7f9085815640 1 --2- 172.21.15.83:0/3646984982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f908006ae10 0x7f908013af70 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:29.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.438+0000 7f9085815640 1 -- 172.21.15.83:0/3646984982 >> 172.21.15.83:0/3646984982 conn(0x7f90800b15b0 msgr2=0x7f90800cf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:29.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.438+0000 7f9085815640 1 -- 172.21.15.83:0/3646984982 shutdown_connections 2024-04-17T10:42:29.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.438+0000 7f9085815640 1 -- 172.21.15.83:0/3646984982 wait complete. 2024-04-17T10:42:29.456 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T10:42:29.456 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:42:29.505 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:42:29.539 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T10:42:29.540 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:42:29.540 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:42:29.545 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16063 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:42:29.545 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:42:29.545 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:42:29.590 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:42:29.591 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:42:29.591 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:42:29.596 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16063 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:42:29.596 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:42:29.596 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:42:29.639 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:42:29.639 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-17T10:42:29.639 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:42:29.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.770+0000 7f2fe80b3640 1 Processor -- start 2024-04-17T10:42:29.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.770+0000 7f2fe80b3640 1 -- start start 2024-04-17T10:42:29.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.770+0000 7f2fe80b3640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fe015db50 0x7f2fe0070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:29.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.770+0000 7f2fe80b3640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2fe006dcb0 0x7f2fe00b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:29.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.770+0000 7f2fe80b3640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe00709b0 con 0x7f2fe00aedd0 2024-04-17T10:42:29.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.770+0000 7f2fe80b3640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe015e440 con 0x7f2fe006dcb0 2024-04-17T10:42:29.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.770+0000 7f2fe80b3640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe015e720 con 0x7f2fe015db50 2024-04-17T10:42:29.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.770+0000 7f2fe6650640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2fe006dcb0 0x7f2fe00b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:29.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.770+0000 7f2fe6650640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2fe006dcb0 0x7f2fe00b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55230/0 (socket says 172.21.15.83:55230) 2024-04-17T10:42:29.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.770+0000 7f2fe6650640 1 -- 172.21.15.83:0/3457541553 learned_addr learned my addr 172.21.15.83:0/3457541553 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:29.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.770+0000 7f2fe5e4f640 1 --2- 172.21.15.83:0/3457541553 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fe015db50 0x7f2fe0070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:29.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.770+0000 7f2fe6650640 1 -- 172.21.15.83:0/3457541553 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fe015db50 msgr2=0x7f2fe0070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:29.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.770+0000 7f2fe6650640 1 --2- 172.21.15.83:0/3457541553 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fe015db50 0x7f2fe0070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:29.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.770+0000 7f2fe6650640 1 -- 172.21.15.83:0/3457541553 >> v1:172.21.15.167:6789/0 conn(0x7f2fe00aedd0 legacy=0x7f2fe006d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:29.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.770+0000 7f2fe6650640 1 -- 172.21.15.83:0/3457541553 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe015ea00 con 0x7f2fe006dcb0 2024-04-17T10:42:29.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.770+0000 7f2fe6650640 1 --2- 172.21.15.83:0/3457541553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fe006dcb0 0x7f2fe00b2e10 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8009e50 tx=0x7f2fd801c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a229be063eb0fa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:29.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fe4e4d640 1 -- 172.21.15.83:0/3457541553 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd80a5660 con 0x7f2fe006dcb0 2024-04-17T10:42:29.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fe4e4d640 1 -- 172.21.15.83:0/3457541553 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd80a5800 con 0x7f2fe006dcb0 2024-04-17T10:42:29.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fe4e4d640 1 -- 172.21.15.83:0/3457541553 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd80a5bd0 con 0x7f2fe006dcb0 2024-04-17T10:42:29.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fe80b3640 1 -- 172.21.15.83:0/3457541553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fe006dcb0 msgr2=0x7f2fe00b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:29.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fe80b3640 1 --2- 172.21.15.83:0/3457541553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fe006dcb0 0x7f2fe00b2e10 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8009e50 tx=0x7f2fd801c7e0 comp rx=0 tx=0).stop 2024-04-17T10:42:29.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fe80b3640 1 -- 172.21.15.83:0/3457541553 shutdown_connections 2024-04-17T10:42:29.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fe80b3640 1 --2- 172.21.15.83:0/3457541553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fe006dcb0 0x7f2fe00b2e10 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:29.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fe80b3640 1 --2- 172.21.15.83:0/3457541553 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fe015db50 0x7f2fe0070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:29.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fe80b3640 1 -- 172.21.15.83:0/3457541553 >> 172.21.15.83:0/3457541553 conn(0x7f2fe006abe0 msgr2=0x7f2fe006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:29.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fe80b3640 1 -- 172.21.15.83:0/3457541553 shutdown_connections 2024-04-17T10:42:29.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fe80b3640 1 -- 172.21.15.83:0/3457541553 wait complete. 2024-04-17T10:42:29.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fe80b3640 1 Processor -- start 2024-04-17T10:42:29.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fe80b3640 1 -- start start 2024-04-17T10:42:29.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fe80b3640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fe006dcb0 0x7f2fe01585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:29.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fe80b3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fe00aedd0 0x7f2fe0154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:29.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fe80b3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fe015db50 0x7f2fe015c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:29.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fe80b3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe00b3ce0 con 0x7f2fe00aedd0 2024-04-17T10:42:29.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fe80b3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe00b3b60 con 0x7f2fe015db50 2024-04-17T10:42:29.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fe80b3640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe00b3e60 con 0x7f2fe006dcb0 2024-04-17T10:42:29.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fe564e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fe00aedd0 0x7f2fe0154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:29.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fe564e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fe00aedd0 0x7f2fe0154090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55240/0 (socket says 172.21.15.83:55240) 2024-04-17T10:42:29.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fe564e640 1 -- 172.21.15.83:0/193309376 learned_addr learned my addr 172.21.15.83:0/193309376 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:29.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fe6650640 1 --2- 172.21.15.83:0/193309376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fe015db50 0x7f2fe015c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:29.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fe5e4f640 1 --2- 172.21.15.83:0/193309376 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fe006dcb0 0x7f2fe01585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:29.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fe564e640 1 -- 172.21.15.83:0/193309376 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fe006dcb0 msgr2=0x7f2fe01585e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:29.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fe564e640 1 --2- 172.21.15.83:0/193309376 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fe006dcb0 0x7f2fe01585e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:29.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fe564e640 1 -- 172.21.15.83:0/193309376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fe015db50 msgr2=0x7f2fe015c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:29.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fe564e640 1 --2- 172.21.15.83:0/193309376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fe015db50 0x7f2fe015c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:29.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fe564e640 1 -- 172.21.15.83:0/193309376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe00cd750 con 0x7f2fe00aedd0 2024-04-17T10:42:29.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fe564e640 1 --2- 172.21.15.83:0/193309376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fe00aedd0 0x7f2fe0154090 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000c450 tx=0x7f2fd000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:29.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fd67fc640 1 -- 172.21.15.83:0/193309376 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0012830 con 0x7f2fe00aedd0 2024-04-17T10:42:29.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fd67fc640 1 -- 172.21.15.83:0/193309376 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd0017070 con 0x7f2fe00aedd0 2024-04-17T10:42:29.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fe80b3640 1 -- 172.21.15.83:0/193309376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe0158c50 con 0x7f2fe00aedd0 2024-04-17T10:42:29.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fd67fc640 1 -- 172.21.15.83:0/193309376 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0012c30 con 0x7f2fe00aedd0 2024-04-17T10:42:29.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.774+0000 7f2fe80b3640 1 -- 172.21.15.83:0/193309376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe0159210 con 0x7f2fe00aedd0 2024-04-17T10:42:29.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.778+0000 7f2fe80b3640 1 -- 172.21.15.83:0/193309376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe0074f90 con 0x7f2fe00aedd0 2024-04-17T10:42:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.778+0000 7f2fd67fc640 1 -- 172.21.15.83:0/193309376 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2fd001c460 con 0x7f2fe00aedd0 2024-04-17T10:42:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.778+0000 7f2fd67fc640 1 --2- 172.21.15.83:0/193309376 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2fc4066f60 0x7f2fc4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.778+0000 7f2fd67fc640 1 -- 172.21.15.83:0/193309376 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2fd0084b30 con 0x7f2fe00aedd0 2024-04-17T10:42:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.778+0000 7f2fe5e4f640 1 --2- 172.21.15.83:0/193309376 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2fc4066f60 0x7f2fc4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.778+0000 7f2fe5e4f640 1 --2- 172.21.15.83:0/193309376 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2fc4066f60 0x7f2fc4069420 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc00b930 tx=0x7f2fdc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:29.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.782+0000 7f2fd67fc640 1 -- 172.21.15.83:0/193309376 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2fd0055e20 con 0x7f2fe00aedd0 2024-04-17T10:42:29.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.970+0000 7f2fe80b3640 1 -- 172.21.15.83:0/193309376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2fe00b2af0 con 0x7f2fe00aedd0 2024-04-17T10:42:29.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.970+0000 7f2fd67fc640 1 -- 172.21.15.83:0/193309376 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f2fd0055720 con 0x7f2fe00aedd0 2024-04-17T10:42:29.971 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:29.971 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":25,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/161398126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":161398126},{"type":"v1","addr":"172.21.15.167:6833","nonce":161398126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4811,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1085380618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1085380618},{"type":"v1","addr":"172.21.15.167:6835","nonce":1085380618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/836442291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":836442291},{"type":"v1","addr":"172.21.15.83:6835","nonce":836442291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":6450,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3693904165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3693904165},{"type":"v1","addr":"172.21.15.83:6837","nonce":3693904165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[]} 2024-04-17T10:42:29.972 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 25 2024-04-17T10:42:29.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.974+0000 7f2fe80b3640 1 -- 172.21.15.83:0/193309376 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2fc4066f60 msgr2=0x7f2fc4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:29.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.974+0000 7f2fe80b3640 1 --2- 172.21.15.83:0/193309376 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2fc4066f60 0x7f2fc4069420 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc00b930 tx=0x7f2fdc008040 comp rx=0 tx=0).stop 2024-04-17T10:42:29.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.974+0000 7f2fe80b3640 1 -- 172.21.15.83:0/193309376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fe00aedd0 msgr2=0x7f2fe0154090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:29.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.974+0000 7f2fe80b3640 1 --2- 172.21.15.83:0/193309376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fe00aedd0 0x7f2fe0154090 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000c450 tx=0x7f2fd000c910 comp rx=0 tx=0).stop 2024-04-17T10:42:29.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.974+0000 7f2fe80b3640 1 -- 172.21.15.83:0/193309376 shutdown_connections 2024-04-17T10:42:29.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.974+0000 7f2fe80b3640 1 --2- 172.21.15.83:0/193309376 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2fc4066f60 0x7f2fc4069420 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:29.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.974+0000 7f2fe80b3640 1 --2- 172.21.15.83:0/193309376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fe015db50 0x7f2fe015c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:29.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.974+0000 7f2fe80b3640 1 --2- 172.21.15.83:0/193309376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fe00aedd0 0x7f2fe0154090 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:29.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.974+0000 7f2fe80b3640 1 --2- 172.21.15.83:0/193309376 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fe006dcb0 0x7f2fe01585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:29.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.974+0000 7f2fe80b3640 1 -- 172.21.15.83:0/193309376 >> 172.21.15.83:0/193309376 conn(0x7f2fe006abe0 msgr2=0x7f2fe006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:29.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.974+0000 7f2fe80b3640 1 -- 172.21.15.83:0/193309376 shutdown_connections 2024-04-17T10:42:29.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:29.974+0000 7f2fe80b3640 1 -- 172.21.15.83:0/193309376 wait complete. 2024-04-17T10:42:29.989 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-17T10:42:29.989 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-17T10:42:29.989 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-17T10:42:29.989 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:42:29.990 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-17T10:42:29.990 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-17T10:42:29.990 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-17T10:42:29.990 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:42:29.990 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-17T10:42:29.990 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-17T10:42:29.991 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-17T10:42:29.991 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:42:29.991 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-17T10:42:29.991 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-17T10:42:29.991 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-17T10:42:29.991 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:42:29.999 INFO:tasks.ceph.mds.c:Stopped 2024-04-17T10:42:29.999 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-17T10:42:30.001 INFO:tasks.ceph.mds.a:Stopped 2024-04-17T10:42:30.001 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-17T10:42:30.003 INFO:tasks.ceph.mds.b:Stopped 2024-04-17T10:42:30.003 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-17T10:42:30.012 INFO:tasks.ceph.mds.d:Stopped 2024-04-17T10:42:30.013 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-17T10:42:30.049 INFO:tasks.ceph.mds.c:Started 2024-04-17T10:42:30.049 DEBUG:teuthology.parallel:result is None 2024-04-17T10:42:30.050 INFO:tasks.ceph.mds.a:Started 2024-04-17T10:42:30.051 DEBUG:teuthology.parallel:result is None 2024-04-17T10:42:30.055 INFO:tasks.ceph.mds.b:Started 2024-04-17T10:42:30.056 DEBUG:teuthology.parallel:result is None 2024-04-17T10:42:30.057 INFO:tasks.ceph.mds.d:Started 2024-04-17T10:42:30.057 DEBUG:teuthology.parallel:result is None 2024-04-17T10:42:30.058 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-17T10:42:30.107 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-04-17T10:42:30.107 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-04-17T10:42:30.125 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2024-04-17T10:42:30.128 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2024-04-17T10:42:30.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.165+0000 7f3d4e930640 1 Processor -- start 2024-04-17T10:42:30.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.165+0000 7f3d4e930640 1 -- start start 2024-04-17T10:42:30.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.169+0000 7f3d4e930640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3d480afd60 0x7f3d480b4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:30.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.169+0000 7f3d4e930640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3d4815cb20 0x7f3d480b5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:30.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.169+0000 7f3d4e930640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d480b0140 con 0x7f3d4806abe0 2024-04-17T10:42:30.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.169+0000 7f3d4e930640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d480b37f0 con 0x7f3d480afd60 2024-04-17T10:42:30.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.169+0000 7f3d4e930640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d480b5860 con 0x7f3d4815cb20 2024-04-17T10:42:30.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.169+0000 7f3d47fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3d480afd60 0x7f3d480b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:30.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.169+0000 7f3d4cecd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3d4806abe0 0x7f3d4815d0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39740/0 (socket says 172.21.15.83:39740) 2024-04-17T10:42:30.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.169+0000 7f3d4cecd640 1 -- 172.21.15.83:0/3777265225 learned_addr learned my addr 172.21.15.83:0/3777265225 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:30.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.169+0000 7f3d47fff640 1 -- 172.21.15.83:0/3777265225 >> v2:172.21.15.167:3300/0 conn(0x7f3d4815cb20 msgr2=0x7f3d480b5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:30.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.169+0000 7f3d47fff640 1 --2- 172.21.15.83:0/3777265225 >> v2:172.21.15.167:3300/0 conn(0x7f3d4815cb20 0x7f3d480b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:30.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.169+0000 7f3d47fff640 1 -- 172.21.15.83:0/3777265225 >> v1:172.21.15.83:6789/0 conn(0x7f3d4806abe0 legacy=0x7f3d4815d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:30.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.169+0000 7f3d46ffd640 1 -- 172.21.15.83:0/3777265225 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2339980624 0 0) 0x7f3d480b0140 con 0x7f3d4806abe0 2024-04-17T10:42:30.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.169+0000 7f3d47fff640 1 -- 172.21.15.83:0/3777265225 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d480b5b00 con 0x7f3d480afd60 2024-04-17T10:42:30.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.169+0000 7f3d47fff640 1 --2- 172.21.15.83:0/3777265225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d480afd60 0x7f3d480b4520 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f3d48071ac0 tx=0x7f3d3801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0f281711e60e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:30.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.169+0000 7f3d46ffd640 1 -- 172.21.15.83:0/3777265225 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d380b0070 con 0x7f3d480afd60 2024-04-17T10:42:30.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.169+0000 7f3d46ffd640 1 -- 172.21.15.83:0/3777265225 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d380ab600 con 0x7f3d480afd60 2024-04-17T10:42:30.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.169+0000 7f3d46ffd640 1 -- 172.21.15.83:0/3777265225 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d380ab9d0 con 0x7f3d480afd60 2024-04-17T10:42:30.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.169+0000 7f3d4e930640 1 -- 172.21.15.83:0/3777265225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d480afd60 msgr2=0x7f3d480b4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:30.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.169+0000 7f3d4e930640 1 --2- 172.21.15.83:0/3777265225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d480afd60 0x7f3d480b4520 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f3d48071ac0 tx=0x7f3d3801c3f0 comp rx=0 tx=0).stop 2024-04-17T10:42:30.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.169+0000 7f3d4e930640 1 -- 172.21.15.83:0/3777265225 shutdown_connections 2024-04-17T10:42:30.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.169+0000 7f3d4e930640 1 --2- 172.21.15.83:0/3777265225 >> v2:172.21.15.167:3300/0 conn(0x7f3d4815cb20 0x7f3d480b5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:30.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.169+0000 7f3d4e930640 1 --2- 172.21.15.83:0/3777265225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d480afd60 0x7f3d480b4520 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:30.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.169+0000 7f3d4e930640 1 -- 172.21.15.83:0/3777265225 >> 172.21.15.83:0/3777265225 conn(0x7f3d4806bcf0 msgr2=0x7f3d4806c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:30.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.173+0000 7f3d4e930640 1 -- 172.21.15.83:0/3777265225 shutdown_connections 2024-04-17T10:42:30.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.173+0000 7f3d4e930640 1 -- 172.21.15.83:0/3777265225 wait complete. 2024-04-17T10:42:30.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.173+0000 7f3d4e930640 1 Processor -- start 2024-04-17T10:42:30.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.173+0000 7f3d4e930640 1 -- start start 2024-04-17T10:42:30.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.173+0000 7f3d4e930640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d4806abe0 0x7f3d4806ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:30.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.173+0000 7f3d4e930640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d480afd60 0x7f3d4814ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:30.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.173+0000 7f3d4e930640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d4815cb20 0x7f3d4806d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:30.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.173+0000 7f3d4e930640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4815dbf0 con 0x7f3d480afd60 2024-04-17T10:42:30.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.173+0000 7f3d4e930640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4806b440 con 0x7f3d4815cb20 2024-04-17T10:42:30.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.173+0000 7f3d4e930640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4815dd70 con 0x7f3d4806abe0 2024-04-17T10:42:30.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.173+0000 7f3d477fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d480afd60 0x7f3d4814ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:30.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.173+0000 7f3d477fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d480afd60 0x7f3d4814ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55298/0 (socket says 172.21.15.83:55298) 2024-04-17T10:42:30.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.173+0000 7f3d477fe640 1 -- 172.21.15.83:0/708236655 learned_addr learned my addr 172.21.15.83:0/708236655 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:30.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.173+0000 7f3d477fe640 1 -- 172.21.15.83:0/708236655 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d4806abe0 msgr2=0x7f3d4806ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:30.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.173+0000 7f3d477fe640 1 --2- 172.21.15.83:0/708236655 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d4806abe0 0x7f3d4806ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:30.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.173+0000 7f3d477fe640 1 -- 172.21.15.83:0/708236655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d4815cb20 msgr2=0x7f3d4806d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:30.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.173+0000 7f3d4cecd640 1 --2- 172.21.15.83:0/708236655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d4815cb20 0x7f3d4806d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:30.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.173+0000 7f3d477fe640 1 --2- 172.21.15.83:0/708236655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d4815cb20 0x7f3d4806d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:30.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.173+0000 7f3d477fe640 1 -- 172.21.15.83:0/708236655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4806dcb0 con 0x7f3d480afd60 2024-04-17T10:42:30.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.173+0000 7f3d477fe640 1 --2- 172.21.15.83:0/708236655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d480afd60 0x7f3d4814ab10 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00b410 tx=0x7f3d3c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:30.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.173+0000 7f3d44ff9640 1 -- 172.21.15.83:0/708236655 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c015020 con 0x7f3d480afd60 2024-04-17T10:42:30.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.173+0000 7f3d44ff9640 1 -- 172.21.15.83:0/708236655 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d3c011830 con 0x7f3d480afd60 2024-04-17T10:42:30.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.173+0000 7f3d44ff9640 1 -- 172.21.15.83:0/708236655 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c011c20 con 0x7f3d480afd60 2024-04-17T10:42:30.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.173+0000 7f3d4e930640 1 -- 172.21.15.83:0/708236655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d4806df90 con 0x7f3d480afd60 2024-04-17T10:42:30.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.173+0000 7f3d4e930640 1 -- 172.21.15.83:0/708236655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d4806e550 con 0x7f3d480afd60 2024-04-17T10:42:30.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.173+0000 7f3d4e930640 1 -- 172.21.15.83:0/708236655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d48074f90 con 0x7f3d480afd60 2024-04-17T10:42:30.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.177+0000 7f3d44ff9640 1 -- 172.21.15.83:0/708236655 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3d3c018020 con 0x7f3d480afd60 2024-04-17T10:42:30.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.177+0000 7f3d44ff9640 1 --2- 172.21.15.83:0/708236655 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3d34066f60 0x7f3d34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:30.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.177+0000 7f3d44ff9640 1 -- 172.21.15.83:0/708236655 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3d3c084bc0 con 0x7f3d480afd60 2024-04-17T10:42:30.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.177+0000 7f3d47fff640 1 --2- 172.21.15.83:0/708236655 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3d34066f60 0x7f3d34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:30.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.177+0000 7f3d47fff640 1 --2- 172.21.15.83:0/708236655 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3d34066f60 0x7f3d34069420 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3d380ad000 tx=0x7f3d38008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:30.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.181+0000 7f3d44ff9640 1 -- 172.21.15.83:0/708236655 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3d3c01f020 con 0x7f3d480afd60 2024-04-17T10:42:30.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.345+0000 7f3d4e930640 1 -- 172.21.15.83:0/708236655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3d480cf1c0 con 0x7f3d480afd60 2024-04-17T10:42:30.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.941+0000 7f3d44ff9640 1 -- 172.21.15.83:0/708236655 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v53) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3d3c0172c0 con 0x7f3d480afd60 2024-04-17T10:42:30.941 INFO:teuthology.orchestra.run.smithi083.stderr: removed all blocklist entries 2024-04-17T10:42:30.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.945+0000 7f3d4e930640 1 -- 172.21.15.83:0/708236655 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3d34066f60 msgr2=0x7f3d34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:30.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.945+0000 7f3d4e930640 1 --2- 172.21.15.83:0/708236655 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3d34066f60 0x7f3d34069420 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3d380ad000 tx=0x7f3d38008040 comp rx=0 tx=0).stop 2024-04-17T10:42:30.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.945+0000 7f3d4e930640 1 -- 172.21.15.83:0/708236655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d480afd60 msgr2=0x7f3d4814ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:30.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.945+0000 7f3d4e930640 1 --2- 172.21.15.83:0/708236655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d480afd60 0x7f3d4814ab10 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00b410 tx=0x7f3d3c00b8d0 comp rx=0 tx=0).stop 2024-04-17T10:42:30.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.945+0000 7f3d4e930640 1 -- 172.21.15.83:0/708236655 shutdown_connections 2024-04-17T10:42:30.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.945+0000 7f3d4e930640 1 --2- 172.21.15.83:0/708236655 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3d34066f60 0x7f3d34069420 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:30.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.945+0000 7f3d4e930640 1 --2- 172.21.15.83:0/708236655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d4815cb20 0x7f3d4806d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:30.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.945+0000 7f3d4e930640 1 --2- 172.21.15.83:0/708236655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d480afd60 0x7f3d4814ab10 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:30.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.945+0000 7f3d4e930640 1 --2- 172.21.15.83:0/708236655 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d4806abe0 0x7f3d4806ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:30.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.945+0000 7f3d4e930640 1 -- 172.21.15.83:0/708236655 >> 172.21.15.83:0/708236655 conn(0x7f3d4806bcf0 msgr2=0x7f3d48154ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:30.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.945+0000 7f3d4e930640 1 -- 172.21.15.83:0/708236655 shutdown_connections 2024-04-17T10:42:30.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:30.945+0000 7f3d4e930640 1 -- 172.21.15.83:0/708236655 wait complete. 2024-04-17T10:42:30.968 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-17T10:42:31.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.089+0000 7f94edb34640 1 Processor -- start 2024-04-17T10:42:31.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.089+0000 7f94edb34640 1 -- start start 2024-04-17T10:42:31.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.089+0000 7f94edb34640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f94e8072330 0x7f94e80cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:31.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.089+0000 7f94edb34640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f94e80cfa10 0x7f94e806c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:31.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.089+0000 7f94edb34640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94e80cfea0 con 0x7f94e815ba30 2024-04-17T10:42:31.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.089+0000 7f94edb34640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94e8071ea0 con 0x7f94e80cfa10 2024-04-17T10:42:31.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.089+0000 7f94edb34640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94e806e320 con 0x7f94e8072330 2024-04-17T10:42:31.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.089+0000 7f94e7fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f94e80cfa10 0x7f94e806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:31.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.089+0000 7f94e7fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f94e80cfa10 0x7f94e806c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55300/0 (socket says 172.21.15.83:55300) 2024-04-17T10:42:31.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.089+0000 7f94e7fff640 1 -- 172.21.15.83:0/1465403124 learned_addr learned my addr 172.21.15.83:0/1465403124 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:31.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.089+0000 7f94e6ffd640 1 --2- 172.21.15.83:0/1465403124 >> v2:172.21.15.167:3300/0 conn(0x7f94e8072330 0x7f94e80cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:31.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.089+0000 7f94e7fff640 1 -- 172.21.15.83:0/1465403124 >> v2:172.21.15.167:3300/0 conn(0x7f94e8072330 msgr2=0x7f94e80cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:31.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.089+0000 7f94e7fff640 1 --2- 172.21.15.83:0/1465403124 >> v2:172.21.15.167:3300/0 conn(0x7f94e8072330 0x7f94e80cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:31.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.089+0000 7f94e7fff640 1 -- 172.21.15.83:0/1465403124 >> v1:172.21.15.167:6789/0 conn(0x7f94e815ba30 legacy=0x7f94e815c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:31.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.089+0000 7f94e7fff640 1 -- 172.21.15.83:0/1465403124 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e806e040 con 0x7f94e80cfa10 2024-04-17T10:42:31.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.093+0000 7f94e7fff640 1 --2- 172.21.15.83:0/1465403124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94e80cfa10 0x7f94e806c490 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f94e80af2b0 tx=0x7f94e001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5847f6aa583d04 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:31.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.093+0000 7f94e67fc640 1 -- 172.21.15.83:0/1465403124 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e00a56f0 con 0x7f94e80cfa10 2024-04-17T10:42:31.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.093+0000 7f94e67fc640 1 -- 172.21.15.83:0/1465403124 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e00a5890 con 0x7f94e80cfa10 2024-04-17T10:42:31.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.093+0000 7f94e67fc640 1 -- 172.21.15.83:0/1465403124 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e00a5c60 con 0x7f94e80cfa10 2024-04-17T10:42:31.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.093+0000 7f94edb34640 1 -- 172.21.15.83:0/1465403124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94e80cfa10 msgr2=0x7f94e806c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:31.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.093+0000 7f94edb34640 1 --2- 172.21.15.83:0/1465403124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94e80cfa10 0x7f94e806c490 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f94e80af2b0 tx=0x7f94e001c920 comp rx=0 tx=0).stop 2024-04-17T10:42:31.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.093+0000 7f94edb34640 1 -- 172.21.15.83:0/1465403124 shutdown_connections 2024-04-17T10:42:31.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.093+0000 7f94edb34640 1 --2- 172.21.15.83:0/1465403124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94e80cfa10 0x7f94e806c490 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:31.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.093+0000 7f94edb34640 1 --2- 172.21.15.83:0/1465403124 >> v2:172.21.15.167:3300/0 conn(0x7f94e8072330 0x7f94e80cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:31.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.093+0000 7f94edb34640 1 -- 172.21.15.83:0/1465403124 >> 172.21.15.83:0/1465403124 conn(0x7f94e806abe0 msgr2=0x7f94e806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:31.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.093+0000 7f94edb34640 1 -- 172.21.15.83:0/1465403124 shutdown_connections 2024-04-17T10:42:31.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.093+0000 7f94edb34640 1 -- 172.21.15.83:0/1465403124 wait complete. 2024-04-17T10:42:31.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.093+0000 7f94edb34640 1 Processor -- start 2024-04-17T10:42:31.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.093+0000 7f94edb34640 1 -- start start 2024-04-17T10:42:31.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.093+0000 7f94edb34640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94e8072330 0x7f94e815d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:31.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.093+0000 7f94edb34640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94e80cfa10 0x7f94e814f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:31.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.093+0000 7f94edb34640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94e815ba30 0x7f94e815db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:31.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.093+0000 7f94edb34640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e806de20 con 0x7f94e815ba30 2024-04-17T10:42:31.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.093+0000 7f94edb34640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e806dca0 con 0x7f94e80cfa10 2024-04-17T10:42:31.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.093+0000 7f94edb34640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e8168520 con 0x7f94e8072330 2024-04-17T10:42:31.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.093+0000 7f94e7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94e815ba30 0x7f94e815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:31.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.093+0000 7f94e7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94e815ba30 0x7f94e815db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55304/0 (socket says 172.21.15.83:55304) 2024-04-17T10:42:31.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.093+0000 7f94e7fff640 1 -- 172.21.15.83:0/2838003582 learned_addr learned my addr 172.21.15.83:0/2838003582 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:31.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.093+0000 7f94e7fff640 1 -- 172.21.15.83:0/2838003582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94e8072330 msgr2=0x7f94e815d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:31.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.093+0000 7f94e77fe640 1 --2- 172.21.15.83:0/2838003582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94e8072330 0x7f94e815d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:31.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.093+0000 7f94e6ffd640 1 --2- 172.21.15.83:0/2838003582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94e80cfa10 0x7f94e814f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:31.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.093+0000 7f94e7fff640 1 --2- 172.21.15.83:0/2838003582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94e8072330 0x7f94e815d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:31.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.093+0000 7f94e7fff640 1 -- 172.21.15.83:0/2838003582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94e80cfa10 msgr2=0x7f94e814f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:31.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.093+0000 7f94e7fff640 1 --2- 172.21.15.83:0/2838003582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94e80cfa10 0x7f94e814f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:31.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.093+0000 7f94e7fff640 1 -- 172.21.15.83:0/2838003582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e815e050 con 0x7f94e815ba30 2024-04-17T10:42:31.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.093+0000 7f94e7fff640 1 --2- 172.21.15.83:0/2838003582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94e815ba30 0x7f94e815db10 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f94e001a550 tx=0x7f94e00196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:31.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.097+0000 7f94cffff640 1 -- 172.21.15.83:0/2838003582 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e00bc020 con 0x7f94e815ba30 2024-04-17T10:42:31.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.097+0000 7f94cffff640 1 -- 172.21.15.83:0/2838003582 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e00b8670 con 0x7f94e815ba30 2024-04-17T10:42:31.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.097+0000 7f94edb34640 1 -- 172.21.15.83:0/2838003582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94e8152540 con 0x7f94e815ba30 2024-04-17T10:42:31.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.097+0000 7f94cffff640 1 -- 172.21.15.83:0/2838003582 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e0009510 con 0x7f94e815ba30 2024-04-17T10:42:31.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.097+0000 7f94edb34640 1 -- 172.21.15.83:0/2838003582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94e8152b00 con 0x7f94e815ba30 2024-04-17T10:42:31.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.097+0000 7f94edb34640 1 -- 172.21.15.83:0/2838003582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94e8074f90 con 0x7f94e815ba30 2024-04-17T10:42:31.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.097+0000 7f94cffff640 1 -- 172.21.15.83:0/2838003582 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f94e00bf020 con 0x7f94e815ba30 2024-04-17T10:42:31.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.097+0000 7f94cffff640 1 --2- 172.21.15.83:0/2838003582 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f94d4066f60 0x7f94d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:31.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.097+0000 7f94cffff640 1 -- 172.21.15.83:0/2838003582 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f94e011fec0 con 0x7f94e815ba30 2024-04-17T10:42:31.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.097+0000 7f94e77fe640 1 --2- 172.21.15.83:0/2838003582 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f94d4066f60 0x7f94d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:31.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.097+0000 7f94e77fe640 1 --2- 172.21.15.83:0/2838003582 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f94d4066f60 0x7f94d4069420 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f94d8003d60 tx=0x7f94d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:31.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.101+0000 7f94cffff640 1 -- 172.21.15.83:0/2838003582 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f94e00c6030 con 0x7f94e815ba30 2024-04-17T10:42:31.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.317+0000 7f94edb34640 1 -- 172.21.15.83:0/2838003582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f94e80b1080 con 0x7f94e815ba30 2024-04-17T10:42:31.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.317+0000 7f94cffff640 1 -- 172.21.15.83:0/2838003582 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f94e00165a0 con 0x7f94e815ba30 2024-04-17T10:42:31.316 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:31.316 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:42:31.316 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_dump": [ 2024-04-17T10:42:31.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:42:31.317 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.a", 2024-04-17T10:42:31.317 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mfpxsNxAANpL7rcWevXwd2KMftSQnEQ==", 2024-04-17T10:42:31.317 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:42:31.317 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:42:31.317 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:42:31.317 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:42:31.317 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:42:31.317 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:42:31.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:42:31.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:42:31.317 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.b", 2024-04-17T10:42:31.317 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mXlI9ChAAYdcrGVmtYdtWWgCLVmpTTg==", 2024-04-17T10:42:31.317 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:42:31.318 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:42:31.318 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:42:31.318 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:42:31.318 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:42:31.318 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:42:31.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:42:31.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:42:31.318 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.c", 2024-04-17T10:42:31.318 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mupo3BBAAyKH801vpFLXYs9mfzSlT0w==", 2024-04-17T10:42:31.318 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:42:31.318 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:42:31.319 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:42:31.319 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:42:31.319 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:42:31.319 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:42:31.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:42:31.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:42:31.319 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.d", 2024-04-17T10:42:31.319 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mbm++EhAAIBMOtQoD+kPeMqR6gK5ctw==", 2024-04-17T10:42:31.319 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:42:31.319 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:42:31.319 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:42:31.319 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:42:31.320 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:42:31.320 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:42:31.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:42:31.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:42:31.320 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.0", 2024-04-17T10:42:31.320 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBXph9m1HWhIRAAOq+9nGq7OGonUCE2aeiAzQ==", 2024-04-17T10:42:31.322 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:42:31.322 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:42:31.322 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:42:31.322 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:42:31.322 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:42:31.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:42:31.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:42:31.322 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.1", 2024-04-17T10:42:31.322 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBaph9mlMmtJRAAXHLMjXfm/+WjIFURLkbAfA==", 2024-04-17T10:42:31.322 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:42:31.322 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:42:31.323 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:42:31.323 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:42:31.323 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:42:31.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:42:31.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:42:31.323 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.2", 2024-04-17T10:42:31.323 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBdph9mBsNPLRAAtR5hwGdDkjLDLS19QUDDYQ==", 2024-04-17T10:42:31.323 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:42:31.323 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:42:31.323 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:42:31.323 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:42:31.323 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:42:31.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:42:31.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:42:31.324 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.3", 2024-04-17T10:42:31.324 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBgph9mZ4EwNRAAIyL7e0EK1523dPXmbOOkzw==", 2024-04-17T10:42:31.324 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:42:31.324 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:42:31.324 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:42:31.324 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:42:31.324 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:42:31.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:42:31.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:42:31.324 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.4", 2024-04-17T10:42:31.324 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBmph9mphAsMhAAtF4uV28Nb66PNjuugB12Hw==", 2024-04-17T10:42:31.324 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:42:31.325 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:42:31.325 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:42:31.325 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:42:31.325 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:42:31.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:42:31.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:42:31.325 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.5", 2024-04-17T10:42:31.325 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBpph9mHJQNNxAA72eOFYFbhakQac1JKRNjzw==", 2024-04-17T10:42:31.325 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:42:31.325 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:42:31.325 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:42:31.325 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:42:31.325 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:42:31.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:42:31.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:42:31.326 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.6", 2024-04-17T10:42:31.326 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBtph9mb6RKAxAAoVX6Cp7l7j3Lm1ES2Sq+fQ==", 2024-04-17T10:42:31.326 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:42:31.326 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:42:31.326 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:42:31.326 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:42:31.326 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:42:31.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:42:31.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:42:31.326 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.7", 2024-04-17T10:42:31.326 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBwph9mdrdrBxAApDwjml2ahsLbRol4uL/BxA==", 2024-04-17T10:42:31.327 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:42:31.327 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:42:31.327 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:42:31.327 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:42:31.327 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:42:31.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:42:31.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:42:31.327 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.0", 2024-04-17T10:42:31.327 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mHGmXGBAAZF+JiAG9YrFtbW+dCy2IEQ==", 2024-04-17T10:42:31.327 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:42:31.327 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:42:31.327 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow r", 2024-04-17T10:42:31.327 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-17T10:42:31.328 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:42:31.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:42:31.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:42:31.328 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.1", 2024-04-17T10:42:31.328 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mB7wgIBAABdavAo/uzVuSjfkkcYtqJg==", 2024-04-17T10:42:31.328 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:42:31.328 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:42:31.328 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-17T10:42:31.328 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-17T10:42:31.328 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-17T10:42:31.328 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:42:31.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:42:31.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:42:31.328 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.2", 2024-04-17T10:42:31.329 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9m+wKgHRAAe8IUFx34ItLpsaDqSYyEgA==", 2024-04-17T10:42:31.329 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:42:31.329 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:42:31.329 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-17T10:42:31.329 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-17T10:42:31.329 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-17T10:42:31.329 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:42:31.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:42:31.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:42:31.329 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.3", 2024-04-17T10:42:31.329 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mfJ0yJRAAHH7rRh/BUIujTT04AF2l0w==", 2024-04-17T10:42:31.329 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:42:31.329 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:42:31.329 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-17T10:42:31.330 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-17T10:42:31.330 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-17T10:42:31.330 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:42:31.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:42:31.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:42:31.330 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.admin", 2024-04-17T10:42:31.330 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mjJ3GIBAA/2qaCbRiLJ5X1cv3ZGdfoA==", 2024-04-17T10:42:31.330 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:42:31.330 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-17T10:42:31.330 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:42:31.330 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:42:31.330 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:42:31.331 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:42:31.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:42:31.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:42:31.331 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.x", 2024-04-17T10:42:31.331 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mMFptNBAA/G0zEfMZTCy9Iox+MwHG3Q==", 2024-04-17T10:42:31.331 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:42:31.331 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-17T10:42:31.331 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-04-17T10:42:31.331 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:42:31.331 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:42:31.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:42:31.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:42:31.332 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.y", 2024-04-17T10:42:31.332 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mjFbRMRAAjY2d8POBzp8hEp9GSO0ylQ==", 2024-04-17T10:42:31.332 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:42:31.332 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-17T10:42:31.332 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-04-17T10:42:31.332 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:42:31.332 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:42:31.333 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:42:31.333 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-17T10:42:31.333 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:42:31.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.321+0000 7f94edb34640 1 -- 172.21.15.83:0/2838003582 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f94d4066f60 msgr2=0x7f94d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:31.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.321+0000 7f94edb34640 1 --2- 172.21.15.83:0/2838003582 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f94d4066f60 0x7f94d4069420 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f94d8003d60 tx=0x7f94d8009040 comp rx=0 tx=0).stop 2024-04-17T10:42:31.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.321+0000 7f94edb34640 1 -- 172.21.15.83:0/2838003582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94e815ba30 msgr2=0x7f94e815db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:31.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.321+0000 7f94edb34640 1 --2- 172.21.15.83:0/2838003582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94e815ba30 0x7f94e815db10 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f94e001a550 tx=0x7f94e00196c0 comp rx=0 tx=0).stop 2024-04-17T10:42:31.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.321+0000 7f94edb34640 1 -- 172.21.15.83:0/2838003582 shutdown_connections 2024-04-17T10:42:31.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.321+0000 7f94edb34640 1 --2- 172.21.15.83:0/2838003582 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f94d4066f60 0x7f94d4069420 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:31.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.321+0000 7f94edb34640 1 --2- 172.21.15.83:0/2838003582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94e815ba30 0x7f94e815db10 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:31.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.321+0000 7f94edb34640 1 --2- 172.21.15.83:0/2838003582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94e80cfa10 0x7f94e814f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:31.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.321+0000 7f94edb34640 1 --2- 172.21.15.83:0/2838003582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94e8072330 0x7f94e815d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:31.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.321+0000 7f94edb34640 1 -- 172.21.15.83:0/2838003582 >> 172.21.15.83:0/2838003582 conn(0x7f94e806abe0 msgr2=0x7f94e8155530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:31.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.321+0000 7f94edb34640 1 -- 172.21.15.83:0/2838003582 shutdown_connections 2024-04-17T10:42:31.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.321+0000 7f94edb34640 1 -- 172.21.15.83:0/2838003582 wait complete. 2024-04-17T10:42:31.337 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:42:31.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.441+0000 7f14754e8640 1 Processor -- start 2024-04-17T10:42:31.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.441+0000 7f14754e8640 1 -- start start 2024-04-17T10:42:31.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.445+0000 7f14754e8640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f14700aedd0 0x7f147006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:31.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.445+0000 7f14754e8640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f147015e440 con 0x7f147006dae0 2024-04-17T10:42:31.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.445+0000 7f14754e8640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f147015e720 con 0x7f147015db50 2024-04-17T10:42:31.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.445+0000 7f14754e8640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f147015ea00 con 0x7f14700aedd0 2024-04-17T10:42:31.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.445+0000 7f146e7fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f14700aedd0 0x7f147006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:31.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.445+0000 7f146f7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f147006dae0 0x7f14700b2e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39748/0 (socket says 172.21.15.83:39748) 2024-04-17T10:42:31.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.445+0000 7f146f7fe640 1 -- 172.21.15.83:0/2702523269 learned_addr learned my addr 172.21.15.83:0/2702523269 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:31.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.445+0000 7f146e7fc640 1 -- 172.21.15.83:0/2702523269 >> v1:172.21.15.167:6789/0 conn(0x7f147015db50 legacy=0x7f1470070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:31.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.445+0000 7f146e7fc640 1 -- 172.21.15.83:0/2702523269 >> v1:172.21.15.83:6789/0 conn(0x7f147006dae0 legacy=0x7f14700b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:31.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.445+0000 7f146dffb640 1 -- 172.21.15.83:0/2702523269 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1915022718 0 0) 0x7f147015e440 con 0x7f147006dae0 2024-04-17T10:42:31.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.445+0000 7f146e7fc640 1 -- 172.21.15.83:0/2702523269 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f147015ece0 con 0x7f14700aedd0 2024-04-17T10:42:31.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.445+0000 7f146e7fc640 1 --2- 172.21.15.83:0/2702523269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14700aedd0 0x7f147006d5a0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f14580097f0 tx=0x7f145801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213cfd07511c5e6a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:31.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.445+0000 7f146dffb640 1 -- 172.21.15.83:0/2702523269 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14580b0070 con 0x7f14700aedd0 2024-04-17T10:42:31.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.445+0000 7f146dffb640 1 -- 172.21.15.83:0/2702523269 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14580ab600 con 0x7f14700aedd0 2024-04-17T10:42:31.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.445+0000 7f146dffb640 1 -- 172.21.15.83:0/2702523269 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14580ab9d0 con 0x7f14700aedd0 2024-04-17T10:42:31.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.445+0000 7f14754e8640 1 -- 172.21.15.83:0/2702523269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14700aedd0 msgr2=0x7f147006d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:31.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.445+0000 7f14754e8640 1 --2- 172.21.15.83:0/2702523269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14700aedd0 0x7f147006d5a0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f14580097f0 tx=0x7f145801c460 comp rx=0 tx=0).stop 2024-04-17T10:42:31.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.445+0000 7f14754e8640 1 -- 172.21.15.83:0/2702523269 shutdown_connections 2024-04-17T10:42:31.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.445+0000 7f14754e8640 1 --2- 172.21.15.83:0/2702523269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14700aedd0 0x7f147006d5a0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:31.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.445+0000 7f14754e8640 1 -- 172.21.15.83:0/2702523269 >> 172.21.15.83:0/2702523269 conn(0x7f147006abe0 msgr2=0x7f147006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:31.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.445+0000 7f14754e8640 1 -- 172.21.15.83:0/2702523269 shutdown_connections 2024-04-17T10:42:31.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.445+0000 7f14754e8640 1 -- 172.21.15.83:0/2702523269 wait complete. 2024-04-17T10:42:31.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.449+0000 7f14754e8640 1 Processor -- start 2024-04-17T10:42:31.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.449+0000 7f14754e8640 1 -- start start 2024-04-17T10:42:31.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.449+0000 7f14754e8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f147006dae0 0x7f1470155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:31.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.449+0000 7f14754e8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14700aedd0 0x7f14700ac820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:31.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.449+0000 7f14754e8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f147015db50 0x7f147006c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:31.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.449+0000 7f14754e8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14700b40c0 con 0x7f147006dae0 2024-04-17T10:42:31.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.449+0000 7f14754e8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14700b3f40 con 0x7f147015db50 2024-04-17T10:42:31.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.449+0000 7f14754e8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1470168670 con 0x7f14700aedd0 2024-04-17T10:42:31.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.449+0000 7f146effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f147006dae0 0x7f1470155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:31.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.449+0000 7f146effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f147006dae0 0x7f1470155710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55334/0 (socket says 172.21.15.83:55334) 2024-04-17T10:42:31.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.449+0000 7f146effd640 1 -- 172.21.15.83:0/2315501958 learned_addr learned my addr 172.21.15.83:0/2315501958 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:31.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.449+0000 7f146f7fe640 1 --2- 172.21.15.83:0/2315501958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f147015db50 0x7f147006c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:31.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.449+0000 7f146e7fc640 1 --2- 172.21.15.83:0/2315501958 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14700aedd0 0x7f14700ac820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:31.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.449+0000 7f146effd640 1 -- 172.21.15.83:0/2315501958 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14700aedd0 msgr2=0x7f14700ac820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:31.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.449+0000 7f146effd640 1 --2- 172.21.15.83:0/2315501958 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14700aedd0 0x7f14700ac820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:31.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.449+0000 7f146effd640 1 -- 172.21.15.83:0/2315501958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f147015db50 msgr2=0x7f147006c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:31.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.449+0000 7f146effd640 1 --2- 172.21.15.83:0/2315501958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f147015db50 0x7f147006c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:31.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.449+0000 7f146effd640 1 -- 172.21.15.83:0/2315501958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14700acd60 con 0x7f147006dae0 2024-04-17T10:42:31.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.449+0000 7f146e7fc640 1 --2- 172.21.15.83:0/2315501958 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14700aedd0 0x7f14700ac820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:31.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.449+0000 7f146effd640 1 --2- 172.21.15.83:0/2315501958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f147006dae0 0x7f1470155710 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f146000ba50 tx=0x7f146000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:31.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.449+0000 7f14677fe640 1 -- 172.21.15.83:0/2315501958 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1460019070 con 0x7f147006dae0 2024-04-17T10:42:31.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.449+0000 7f14677fe640 1 -- 172.21.15.83:0/2315501958 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f146000cc10 con 0x7f147006dae0 2024-04-17T10:42:31.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.449+0000 7f14754e8640 1 -- 172.21.15.83:0/2315501958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1470156270 con 0x7f147006dae0 2024-04-17T10:42:31.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.449+0000 7f14677fe640 1 -- 172.21.15.83:0/2315501958 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1460014440 con 0x7f147006dae0 2024-04-17T10:42:31.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.449+0000 7f14754e8640 1 -- 172.21.15.83:0/2315501958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1470156830 con 0x7f147006dae0 2024-04-17T10:42:31.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.449+0000 7f14754e8640 1 -- 172.21.15.83:0/2315501958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1470074f90 con 0x7f147006dae0 2024-04-17T10:42:31.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.449+0000 7f14677fe640 1 -- 172.21.15.83:0/2315501958 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1460007370 con 0x7f147006dae0 2024-04-17T10:42:31.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.453+0000 7f14677fe640 1 --2- 172.21.15.83:0/2315501958 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1450066f90 0x7f1450069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:31.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.453+0000 7f146e7fc640 1 --2- 172.21.15.83:0/2315501958 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1450066f90 0x7f1450069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:31.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.453+0000 7f14677fe640 1 -- 172.21.15.83:0/2315501958 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f14600848c0 con 0x7f147006dae0 2024-04-17T10:42:31.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.453+0000 7f146e7fc640 1 --2- 172.21.15.83:0/2315501958 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1450066f90 0x7f1450069450 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f14580ad000 tx=0x7f1458008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:31.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.457+0000 7f14677fe640 1 -- 172.21.15.83:0/2315501958 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1460055c20 con 0x7f147006dae0 2024-04-17T10:42:31.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.601+0000 7f14754e8640 1 -- 172.21.15.83:0/2315501958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14700cc060 con 0x7f147006dae0 2024-04-17T10:42:31.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.605+0000 7f14677fe640 1 -- 172.21.15.83:0/2315501958 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+12819 (secure 0 0 0) 0x7f1460055520 con 0x7f147006dae0 2024-04-17T10:42:31.604 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:31.604 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":55,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:42:30.956306+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:42:31.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.605+0000 7f14754e8640 1 -- 172.21.15.83:0/2315501958 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1450066f90 msgr2=0x7f1450069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:31.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.605+0000 7f14754e8640 1 --2- 172.21.15.83:0/2315501958 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1450066f90 0x7f1450069450 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f14580ad000 tx=0x7f1458008040 comp rx=0 tx=0).stop 2024-04-17T10:42:31.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.605+0000 7f14754e8640 1 -- 172.21.15.83:0/2315501958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f147006dae0 msgr2=0x7f1470155710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:31.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.605+0000 7f14754e8640 1 --2- 172.21.15.83:0/2315501958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f147006dae0 0x7f1470155710 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f146000ba50 tx=0x7f146000bf10 comp rx=0 tx=0).stop 2024-04-17T10:42:31.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.605+0000 7f14754e8640 1 -- 172.21.15.83:0/2315501958 shutdown_connections 2024-04-17T10:42:31.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.605+0000 7f14754e8640 1 --2- 172.21.15.83:0/2315501958 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1450066f90 0x7f1450069450 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:31.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.605+0000 7f14754e8640 1 --2- 172.21.15.83:0/2315501958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f147015db50 0x7f147006c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:31.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.605+0000 7f14754e8640 1 --2- 172.21.15.83:0/2315501958 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14700aedd0 0x7f14700ac820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:31.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.605+0000 7f14754e8640 1 --2- 172.21.15.83:0/2315501958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f147006dae0 0x7f1470155710 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:31.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.605+0000 7f14754e8640 1 -- 172.21.15.83:0/2315501958 >> 172.21.15.83:0/2315501958 conn(0x7f147006abe0 msgr2=0x7f14700cf420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:31.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.605+0000 7f14754e8640 1 -- 172.21.15.83:0/2315501958 shutdown_connections 2024-04-17T10:42:31.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.605+0000 7f14754e8640 1 -- 172.21.15.83:0/2315501958 wait complete. 2024-04-17T10:42:31.621 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:42:31.622 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:42:31.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.733+0000 7fb5cae3a640 1 Processor -- start 2024-04-17T10:42:31.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.733+0000 7fb5cae3a640 1 -- start start 2024-04-17T10:42:31.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.737+0000 7fb5cae3a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5c406b520 0x7fb5c40b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:31.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.737+0000 7fb5cae3a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5c415bd70 con 0x7fb5c40b4040 2024-04-17T10:42:31.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.737+0000 7fb5cae3a640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5c415bf60 con 0x7fb5c4154de0 2024-04-17T10:42:31.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.737+0000 7fb5cae3a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c40b4870 con 0x7fb5c406b520 2024-04-17T10:42:31.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.737+0000 7fb5c93d7640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb5c40b4040 0x7fb5c406cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39760/0 (socket says 172.21.15.83:39760) 2024-04-17T10:42:31.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.737+0000 7fb5c93d7640 1 -- 172.21.15.83:0/511874178 learned_addr learned my addr 172.21.15.83:0/511874178 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:31.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.737+0000 7fb5c3fff640 1 --2- 172.21.15.83:0/511874178 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5c406b520 0x7fb5c40b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:31.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.737+0000 7fb5c37fe640 1 -- 172.21.15.83:0/511874178 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1232763943 0 0) 0x7fb5c415bd70 con 0x7fb5c40b4040 2024-04-17T10:42:31.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.737+0000 7fb5c37fe640 1 -- 172.21.15.83:0/511874178 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5b4003530 con 0x7fb5c40b4040 2024-04-17T10:42:31.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.737+0000 7fb5c37fe640 1 -- 172.21.15.83:0/511874178 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1427485418 0 0) 0x7fb5b4003530 con 0x7fb5c40b4040 2024-04-17T10:42:31.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.737+0000 7fb5c37fe640 1 -- 172.21.15.83:0/511874178 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5c406b520 msgr2=0x7fb5c40b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:31.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.737+0000 7fb5c37fe640 1 --2- 172.21.15.83:0/511874178 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5c406b520 0x7fb5c40b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:31.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.737+0000 7fb5c37fe640 1 -- 172.21.15.83:0/511874178 >> v1:172.21.15.167:6789/0 conn(0x7fb5c4154de0 legacy=0x7fb5c406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:31.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.737+0000 7fb5c37fe640 1 -- 172.21.15.83:0/511874178 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c415afb0 con 0x7fb5c40b4040 2024-04-17T10:42:31.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.737+0000 7fb5c37fe640 1 -- 172.21.15.83:0/511874178 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fb5bc001a00 con 0x7fb5c40b4040 2024-04-17T10:42:31.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.737+0000 7fb5c37fe640 1 -- 172.21.15.83:0/511874178 >> v1:172.21.15.83:6789/0 conn(0x7fb5c40b4040 legacy=0x7fb5c406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:31.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.737+0000 7fb5c37fe640 1 --2- 172.21.15.83:0/511874178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5b4003a60 0x7fb5b4003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:31.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.737+0000 7fb5c37fe640 1 --2- 172.21.15.83:0/511874178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5b4004810 0x7fb5b4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:31.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.737+0000 7fb5c37fe640 1 --2- 172.21.15.83:0/511874178 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5c406b520 0x7fb5b40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:31.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.737+0000 7fb5c37fe640 1 -- 172.21.15.83:0/511874178 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b400baa0 con 0x7fb5b4004810 2024-04-17T10:42:31.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.737+0000 7fb5c37fe640 1 -- 172.21.15.83:0/511874178 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b400bc50 con 0x7fb5b4003a60 2024-04-17T10:42:31.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.737+0000 7fb5c37fe640 1 -- 172.21.15.83:0/511874178 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b400be50 con 0x7fb5c406b520 2024-04-17T10:42:31.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.737+0000 7fb5c3fff640 1 --2- 172.21.15.83:0/511874178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5b4004810 0x7fb5b4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:31.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.737+0000 7fb5c8bd6640 1 --2- 172.21.15.83:0/511874178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5b4003a60 0x7fb5b4003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:31.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.737+0000 7fb5c3fff640 1 -- 172.21.15.83:0/511874178 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5c406b520 msgr2=0x7fb5b40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:31.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.737+0000 7fb5c3fff640 1 --2- 172.21.15.83:0/511874178 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5c406b520 0x7fb5b40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:31.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.737+0000 7fb5c3fff640 1 -- 172.21.15.83:0/511874178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5b4003a60 msgr2=0x7fb5b4003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:31.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.737+0000 7fb5c3fff640 1 --2- 172.21.15.83:0/511874178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5b4003a60 0x7fb5b4003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:31.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.737+0000 7fb5c3fff640 1 -- 172.21.15.83:0/511874178 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b4003530 con 0x7fb5b4004810 2024-04-17T10:42:31.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.737+0000 7fb5c8bd6640 1 --2- 172.21.15.83:0/511874178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5b4003a60 0x7fb5b4003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5c3fff640 1 --2- 172.21.15.83:0/511874178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5b4004810 0x7fb5b4004c90 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0009ac0 tx=0x7fb5b001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492078b1c127096a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5c37fe640 1 -- 172.21.15.83:0/511874178 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b00b2070 con 0x7fb5b4004810 2024-04-17T10:42:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5c37fe640 1 -- 172.21.15.83:0/511874178 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b00a55a0 con 0x7fb5b4004810 2024-04-17T10:42:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5c37fe640 1 -- 172.21.15.83:0/511874178 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b0009690 con 0x7fb5b4004810 2024-04-17T10:42:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5cae3a640 1 -- 172.21.15.83:0/511874178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5b4004810 msgr2=0x7fb5b4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5cae3a640 1 --2- 172.21.15.83:0/511874178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5b4004810 0x7fb5b4004c90 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0009ac0 tx=0x7fb5b001ca90 comp rx=0 tx=0).stop 2024-04-17T10:42:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5c93d7640 1 -- 172.21.15.83:0/511874178 reap_dead start 2024-04-17T10:42:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5cae3a640 1 -- 172.21.15.83:0/511874178 shutdown_connections 2024-04-17T10:42:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5cae3a640 1 -- 172.21.15.83:0/511874178 >> 172.21.15.83:0/511874178 conn(0x7fb5c40b15b0 msgr2=0x7fb5c40b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5cae3a640 1 -- 172.21.15.83:0/511874178 shutdown_connections 2024-04-17T10:42:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5cae3a640 1 -- 172.21.15.83:0/511874178 wait complete. 2024-04-17T10:42:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5cae3a640 1 Processor -- start 2024-04-17T10:42:31.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5cae3a640 1 -- start start 2024-04-17T10:42:31.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5cae3a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5b4004810 0x7fb5c41f22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:31.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5cae3a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5c40adee0 0x7fb5c40ab0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:31.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5cae3a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5c40ab5f0 0x7fb5c4157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:31.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5cae3a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c415b850 con 0x7fb5c40ab5f0 2024-04-17T10:42:31.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5cae3a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c415b570 con 0x7fb5c40adee0 2024-04-17T10:42:31.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5cae3a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c41551c0 con 0x7fb5b4004810 2024-04-17T10:42:31.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5c93d7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5c40ab5f0 0x7fb5c4157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:31.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5c93d7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5c40ab5f0 0x7fb5c4157670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55354/0 (socket says 172.21.15.83:55354) 2024-04-17T10:42:31.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5c93d7640 1 -- 172.21.15.83:0/3559187555 learned_addr learned my addr 172.21.15.83:0/3559187555 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:31.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5c3fff640 1 --2- 172.21.15.83:0/3559187555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5c40adee0 0x7fb5c40ab0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:31.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5c8bd6640 1 --2- 172.21.15.83:0/3559187555 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5b4004810 0x7fb5c41f22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:31.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5c93d7640 1 -- 172.21.15.83:0/3559187555 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5b4004810 msgr2=0x7fb5c41f22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:31.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5c93d7640 1 --2- 172.21.15.83:0/3559187555 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5b4004810 0x7fb5c41f22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:31.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5c93d7640 1 -- 172.21.15.83:0/3559187555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5c40adee0 msgr2=0x7fb5c40ab0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:31.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5c93d7640 1 --2- 172.21.15.83:0/3559187555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5c40adee0 0x7fb5c40ab0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:31.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5c93d7640 1 -- 172.21.15.83:0/3559187555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c40ae360 con 0x7fb5c40ab5f0 2024-04-17T10:42:31.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5c8bd6640 1 --2- 172.21.15.83:0/3559187555 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5b4004810 0x7fb5c41f22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:31.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5c93d7640 1 --2- 172.21.15.83:0/3559187555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5c40ab5f0 0x7fb5c4157670 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb5c406e900 tx=0x7fb5bc00fb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:31.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5c17fa640 1 -- 172.21.15.83:0/3559187555 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc00d840 con 0x7fb5c40ab5f0 2024-04-17T10:42:31.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5c17fa640 1 -- 172.21.15.83:0/3559187555 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5bc00da00 con 0x7fb5c40ab5f0 2024-04-17T10:42:31.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5cae3a640 1 -- 172.21.15.83:0/3559187555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c40abb50 con 0x7fb5c40ab5f0 2024-04-17T10:42:31.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5c17fa640 1 -- 172.21.15.83:0/3559187555 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc00ddf0 con 0x7fb5c40ab5f0 2024-04-17T10:42:31.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.741+0000 7fb5cae3a640 1 -- 172.21.15.83:0/3559187555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c41f1ab0 con 0x7fb5c40ab5f0 2024-04-17T10:42:31.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.745+0000 7fb5cae3a640 1 -- 172.21.15.83:0/3559187555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c4074f90 con 0x7fb5c40ab5f0 2024-04-17T10:42:31.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.745+0000 7fb5c17fa640 1 -- 172.21.15.83:0/3559187555 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb5bc014070 con 0x7fb5c40ab5f0 2024-04-17T10:42:31.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.745+0000 7fb5c17fa640 1 --2- 172.21.15.83:0/3559187555 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb5a8066f90 0x7fb5a8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:31.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.745+0000 7fb5c17fa640 1 -- 172.21.15.83:0/3559187555 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb5bc087930 con 0x7fb5c40ab5f0 2024-04-17T10:42:31.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.745+0000 7fb5c8bd6640 1 --2- 172.21.15.83:0/3559187555 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb5a8066f90 0x7fb5a8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:31.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.745+0000 7fb5c8bd6640 1 --2- 172.21.15.83:0/3559187555 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb5a8066f90 0x7fb5a8069450 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4153fd0 tx=0x7fb5b8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:31.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.749+0000 7fb5c17fa640 1 -- 172.21.15.83:0/3559187555 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb5bc058c90 con 0x7fb5c40ab5f0 2024-04-17T10:42:31.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.893+0000 7fb5cae3a640 1 -- 172.21.15.83:0/3559187555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb5c40b24b0 con 0x7fb5c40ab5f0 2024-04-17T10:42:31.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.897+0000 7fb5c17fa640 1 -- 172.21.15.83:0/3559187555 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb5bc058590 con 0x7fb5c40ab5f0 2024-04-17T10:42:31.895 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:42:31.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.897+0000 7fb5cae3a640 1 -- 172.21.15.83:0/3559187555 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb5a8066f90 msgr2=0x7fb5a8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:31.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.897+0000 7fb5cae3a640 1 --2- 172.21.15.83:0/3559187555 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb5a8066f90 0x7fb5a8069450 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4153fd0 tx=0x7fb5b8009040 comp rx=0 tx=0).stop 2024-04-17T10:42:31.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.897+0000 7fb5cae3a640 1 -- 172.21.15.83:0/3559187555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5c40ab5f0 msgr2=0x7fb5c4157670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:31.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.897+0000 7fb5cae3a640 1 --2- 172.21.15.83:0/3559187555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5c40ab5f0 0x7fb5c4157670 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb5c406e900 tx=0x7fb5bc00fb90 comp rx=0 tx=0).stop 2024-04-17T10:42:31.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.897+0000 7fb5cae3a640 1 -- 172.21.15.83:0/3559187555 shutdown_connections 2024-04-17T10:42:31.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.897+0000 7fb5cae3a640 1 --2- 172.21.15.83:0/3559187555 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb5a8066f90 0x7fb5a8069450 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:31.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.897+0000 7fb5cae3a640 1 --2- 172.21.15.83:0/3559187555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5c40ab5f0 0x7fb5c4157670 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:31.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.897+0000 7fb5cae3a640 1 --2- 172.21.15.83:0/3559187555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5c40adee0 0x7fb5c40ab0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:31.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.897+0000 7fb5cae3a640 1 --2- 172.21.15.83:0/3559187555 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5b4004810 0x7fb5c41f22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:31.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.897+0000 7fb5cae3a640 1 -- 172.21.15.83:0/3559187555 >> 172.21.15.83:0/3559187555 conn(0x7fb5c40b15b0 msgr2=0x7fb5c41591f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:31.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.897+0000 7fb5cae3a640 1 -- 172.21.15.83:0/3559187555 shutdown_connections 2024-04-17T10:42:31.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:31.897+0000 7fb5cae3a640 1 -- 172.21.15.83:0/3559187555 wait complete. 2024-04-17T10:42:31.913 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-17T10:42:32.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.021+0000 7f5b99e5e640 1 Processor -- start 2024-04-17T10:42:32.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.021+0000 7f5b99e5e640 1 -- start start 2024-04-17T10:42:32.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.025+0000 7f5b99e5e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5b940afea0 0x7f5b940cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:32.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.025+0000 7f5b99e5e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b940d0880 con 0x7f5b9406abe0 2024-04-17T10:42:32.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.025+0000 7f5b99e5e640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b940d0b60 con 0x7f5b94071ca0 2024-04-17T10:42:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.025+0000 7f5b99e5e640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b940d1640 con 0x7f5b940afea0 2024-04-17T10:42:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.025+0000 7f5b937fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5b9406abe0 0x7f5b9406afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39774/0 (socket says 172.21.15.83:39774) 2024-04-17T10:42:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.025+0000 7f5b937fe640 1 -- 172.21.15.83:0/311019703 learned_addr learned my addr 172.21.15.83:0/311019703 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.025+0000 7f5b927fc640 1 -- 172.21.15.83:0/311019703 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 307159532 0 0) 0x7f5b940d0880 con 0x7f5b9406abe0 2024-04-17T10:42:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.025+0000 7f5b927fc640 1 -- 172.21.15.83:0/311019703 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b7c003530 con 0x7f5b9406abe0 2024-04-17T10:42:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.025+0000 7f5b93fff640 1 --2- 172.21.15.83:0/311019703 >> v2:172.21.15.167:3300/0 conn(0x7f5b940afea0 0x7f5b940cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.025+0000 7f5b927fc640 1 -- 172.21.15.83:0/311019703 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4173631456 0 0) 0x7f5b7c003530 con 0x7f5b9406abe0 2024-04-17T10:42:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.025+0000 7f5b927fc640 1 -- 172.21.15.83:0/311019703 >> v2:172.21.15.167:3300/0 conn(0x7f5b940afea0 msgr2=0x7f5b940cede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.025+0000 7f5b927fc640 1 --2- 172.21.15.83:0/311019703 >> v2:172.21.15.167:3300/0 conn(0x7f5b940afea0 0x7f5b940cede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.025+0000 7f5b927fc640 1 -- 172.21.15.83:0/311019703 >> v1:172.21.15.167:6789/0 conn(0x7f5b94071ca0 legacy=0x7f5b940ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.025+0000 7f5b927fc640 1 -- 172.21.15.83:0/311019703 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b940d0e40 con 0x7f5b9406abe0 2024-04-17T10:42:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.025+0000 7f5b927fc640 1 -- 172.21.15.83:0/311019703 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f5b8c001a00 con 0x7f5b9406abe0 2024-04-17T10:42:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.025+0000 7f5b927fc640 1 -- 172.21.15.83:0/311019703 >> v1:172.21.15.83:6789/0 conn(0x7f5b9406abe0 legacy=0x7f5b9406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:32.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.025+0000 7f5b927fc640 1 --2- 172.21.15.83:0/311019703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b7c0039c0 0x7f5b7c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:32.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.025+0000 7f5b927fc640 1 --2- 172.21.15.83:0/311019703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b7c004810 0x7f5b7c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:32.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.025+0000 7f5b927fc640 1 --2- 172.21.15.83:0/311019703 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b7c0051d0 0x7f5b7c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:32.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.025+0000 7f5b927fc640 1 -- 172.21.15.83:0/311019703 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b940d1640 con 0x7f5b7c004810 2024-04-17T10:42:32.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.025+0000 7f5b927fc640 1 -- 172.21.15.83:0/311019703 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c00c170 con 0x7f5b7c0039c0 2024-04-17T10:42:32.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.025+0000 7f5b927fc640 1 -- 172.21.15.83:0/311019703 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c00c370 con 0x7f5b7c0051d0 2024-04-17T10:42:32.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.025+0000 7f5b92ffd640 1 --2- 172.21.15.83:0/311019703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b7c004810 0x7f5b7c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:32.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.025+0000 7f5b937fe640 1 --2- 172.21.15.83:0/311019703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b7c0039c0 0x7f5b7c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:32.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.025+0000 7f5b92ffd640 1 -- 172.21.15.83:0/311019703 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b7c0051d0 msgr2=0x7f5b7c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:32.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.025+0000 7f5b92ffd640 1 --2- 172.21.15.83:0/311019703 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b7c0051d0 0x7f5b7c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:32.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.025+0000 7f5b92ffd640 1 -- 172.21.15.83:0/311019703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b7c0039c0 msgr2=0x7f5b7c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:32.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.025+0000 7f5b92ffd640 1 --2- 172.21.15.83:0/311019703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b7c0039c0 0x7f5b7c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:32.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.025+0000 7f5b92ffd640 1 -- 172.21.15.83:0/311019703 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7c003530 con 0x7f5b7c004810 2024-04-17T10:42:32.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.025+0000 7f5b93fff640 1 -- 172.21.15.83:0/311019703 reap_dead start 2024-04-17T10:42:32.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b92ffd640 1 --2- 172.21.15.83:0/311019703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b7c004810 0x7f5b7c004c90 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f5b88001750 tx=0x7f5b8801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c270f1500036fc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:32.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b927fc640 1 -- 172.21.15.83:0/311019703 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b880013a0 con 0x7f5b7c004810 2024-04-17T10:42:32.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b927fc640 1 -- 172.21.15.83:0/311019703 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b880052f0 con 0x7f5b7c004810 2024-04-17T10:42:32.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b927fc640 1 -- 172.21.15.83:0/311019703 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b88003840 con 0x7f5b7c004810 2024-04-17T10:42:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b99e5e640 1 -- 172.21.15.83:0/311019703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b7c004810 msgr2=0x7f5b7c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b99e5e640 1 --2- 172.21.15.83:0/311019703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b7c004810 0x7f5b7c004c90 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f5b88001750 tx=0x7f5b8801c3c0 comp rx=0 tx=0).stop 2024-04-17T10:42:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b99e5e640 1 -- 172.21.15.83:0/311019703 shutdown_connections 2024-04-17T10:42:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b99e5e640 1 --2- 172.21.15.83:0/311019703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b7c004810 0x7f5b7c004c90 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b99e5e640 1 -- 172.21.15.83:0/311019703 >> 172.21.15.83:0/311019703 conn(0x7f5b9406bcf0 msgr2=0x7f5b9406c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b99e5e640 1 -- 172.21.15.83:0/311019703 shutdown_connections 2024-04-17T10:42:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b99e5e640 1 -- 172.21.15.83:0/311019703 wait complete. 2024-04-17T10:42:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b99e5e640 1 Processor -- start 2024-04-17T10:42:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b99e5e640 1 -- start start 2024-04-17T10:42:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b99e5e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b7c004810 0x7f5b9414c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b99e5e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b94150560 0x7f5b9414d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b99e5e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b9414ca60 0x7f5b94157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b99e5e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9406b140 con 0x7f5b9414ca60 2024-04-17T10:42:32.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b99e5e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9406afc0 con 0x7f5b94150560 2024-04-17T10:42:32.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b99e5e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9406b2c0 con 0x7f5b7c004810 2024-04-17T10:42:32.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b93fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b9414ca60 0x7f5b94157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:32.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b93fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b9414ca60 0x7f5b94157ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55370/0 (socket says 172.21.15.83:55370) 2024-04-17T10:42:32.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b93fff640 1 -- 172.21.15.83:0/726419251 learned_addr learned my addr 172.21.15.83:0/726419251 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:32.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b937fe640 1 --2- 172.21.15.83:0/726419251 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b7c004810 0x7f5b9414c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:32.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b92ffd640 1 --2- 172.21.15.83:0/726419251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b94150560 0x7f5b9414d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:32.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b93fff640 1 -- 172.21.15.83:0/726419251 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b7c004810 msgr2=0x7f5b9414c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:32.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b93fff640 1 --2- 172.21.15.83:0/726419251 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b7c004810 0x7f5b9414c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:32.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b93fff640 1 -- 172.21.15.83:0/726419251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b94150560 msgr2=0x7f5b9414d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:32.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b93fff640 1 --2- 172.21.15.83:0/726419251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b94150560 0x7f5b9414d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:32.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b93fff640 1 -- 172.21.15.83:0/726419251 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b94158e60 con 0x7f5b9414ca60 2024-04-17T10:42:32.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b937fe640 1 --2- 172.21.15.83:0/726419251 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b7c004810 0x7f5b9414c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:32.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b92ffd640 1 --2- 172.21.15.83:0/726419251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b94150560 0x7f5b9414d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:32.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b93fff640 1 --2- 172.21.15.83:0/726419251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b9414ca60 0x7f5b94157ca0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f5b940af900 tx=0x7f5b8400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:32.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b7bfff640 1 -- 172.21.15.83:0/726419251 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8400c620 con 0x7f5b9414ca60 2024-04-17T10:42:32.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.029+0000 7f5b7bfff640 1 -- 172.21.15.83:0/726419251 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b8401a070 con 0x7f5b9414ca60 2024-04-17T10:42:32.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.033+0000 7f5b99e5e640 1 -- 172.21.15.83:0/726419251 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b9415d120 con 0x7f5b9414ca60 2024-04-17T10:42:32.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.033+0000 7f5b7bfff640 1 -- 172.21.15.83:0/726419251 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8400ca20 con 0x7f5b9414ca60 2024-04-17T10:42:32.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.033+0000 7f5b99e5e640 1 -- 172.21.15.83:0/726419251 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b9415d9c0 con 0x7f5b9414ca60 2024-04-17T10:42:32.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.033+0000 7f5b99e5e640 1 -- 172.21.15.83:0/726419251 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b94074f90 con 0x7f5b9414ca60 2024-04-17T10:42:32.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.033+0000 7f5b7bfff640 1 -- 172.21.15.83:0/726419251 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5b8400cbc0 con 0x7f5b9414ca60 2024-04-17T10:42:32.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.033+0000 7f5b7bfff640 1 --2- 172.21.15.83:0/726419251 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5b80066f60 0x7f5b80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:32.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.033+0000 7f5b937fe640 1 --2- 172.21.15.83:0/726419251 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5b80066f60 0x7f5b80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:32.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.033+0000 7f5b7bfff640 1 -- 172.21.15.83:0/726419251 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5b84084c00 con 0x7f5b9414ca60 2024-04-17T10:42:32.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.033+0000 7f5b937fe640 1 --2- 172.21.15.83:0/726419251 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5b80066f60 0x7f5b80069420 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c001590 tx=0x7f5b8c0012e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:32.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.037+0000 7f5b7bfff640 1 -- 172.21.15.83:0/726419251 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5b8408b080 con 0x7f5b9414ca60 2024-04-17T10:42:32.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.185+0000 7f5b99e5e640 1 -- 172.21.15.83:0/726419251 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5b9414fd80 con 0x7f5b9414ca60 2024-04-17T10:42:32.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.185+0000 7f5b7bfff640 1 -- 172.21.15.83:0/726419251 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v55) v1 ==== 84+0+65 (secure 0 0 0) 0x7f5b84055f60 con 0x7f5b9414ca60 2024-04-17T10:42:32.186 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:32.186 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-04-17T10:42:32.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:42:32.186 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2024-04-17T10:42:32.186 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2024-04-17T10:42:32.186 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:42:32.186 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-04-17T10:42:32.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.189+0000 7f5b99e5e640 1 -- 172.21.15.83:0/726419251 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5b80066f60 msgr2=0x7f5b80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:32.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.189+0000 7f5b99e5e640 1 --2- 172.21.15.83:0/726419251 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5b80066f60 0x7f5b80069420 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c001590 tx=0x7f5b8c0012e0 comp rx=0 tx=0).stop 2024-04-17T10:42:32.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.189+0000 7f5b99e5e640 1 -- 172.21.15.83:0/726419251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b9414ca60 msgr2=0x7f5b94157ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:32.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.189+0000 7f5b99e5e640 1 --2- 172.21.15.83:0/726419251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b9414ca60 0x7f5b94157ca0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f5b940af900 tx=0x7f5b8400ec00 comp rx=0 tx=0).stop 2024-04-17T10:42:32.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.189+0000 7f5b99e5e640 1 -- 172.21.15.83:0/726419251 shutdown_connections 2024-04-17T10:42:32.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.189+0000 7f5b99e5e640 1 --2- 172.21.15.83:0/726419251 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5b80066f60 0x7f5b80069420 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:32.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.189+0000 7f5b99e5e640 1 --2- 172.21.15.83:0/726419251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b9414ca60 0x7f5b94157ca0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:32.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.189+0000 7f5b99e5e640 1 --2- 172.21.15.83:0/726419251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b94150560 0x7f5b9414d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:32.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.189+0000 7f5b99e5e640 1 --2- 172.21.15.83:0/726419251 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b7c004810 0x7f5b9414c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:32.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.189+0000 7f5b99e5e640 1 -- 172.21.15.83:0/726419251 >> 172.21.15.83:0/726419251 conn(0x7f5b9406bcf0 msgr2=0x7f5b9406d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:32.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.189+0000 7f5b99e5e640 1 -- 172.21.15.83:0/726419251 shutdown_connections 2024-04-17T10:42:32.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.189+0000 7f5b99e5e640 1 -- 172.21.15.83:0/726419251 wait complete. 2024-04-17T10:42:32.204 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-17T10:42:32.204 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-17T10:42:32.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.317+0000 7f595db70640 1 Processor -- start 2024-04-17T10:42:32.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.317+0000 7f595db70640 1 -- start start 2024-04-17T10:42:32.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.317+0000 7f595db70640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f59580aebd0 0x7f59580ced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:32.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.317+0000 7f595db70640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f595806abe0 0x7f595806b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:32.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.317+0000 7f595db70640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59580cf3e0 con 0x7f595815b990 2024-04-17T10:42:32.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.317+0000 7f595db70640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f595806b570 con 0x7f595806abe0 2024-04-17T10:42:32.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.317+0000 7f595db70640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f595815e5c0 con 0x7f59580aebd0 2024-04-17T10:42:32.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.317+0000 7f5957fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f595806abe0 0x7f595806b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:32.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.317+0000 7f5956ffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f595815b990 0x7f59580cd180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39788/0 (socket says 172.21.15.83:39788) 2024-04-17T10:42:32.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.317+0000 7f5956ffd640 1 -- 172.21.15.83:0/2655026364 learned_addr learned my addr 172.21.15.83:0/2655026364 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:32.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.317+0000 7f59577fe640 1 --2- 172.21.15.83:0/2655026364 >> v2:172.21.15.167:3300/0 conn(0x7f59580aebd0 0x7f59580ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:32.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.317+0000 7f5957fff640 1 -- 172.21.15.83:0/2655026364 >> v2:172.21.15.167:3300/0 conn(0x7f59580aebd0 msgr2=0x7f59580ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:32.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.317+0000 7f5957fff640 1 --2- 172.21.15.83:0/2655026364 >> v2:172.21.15.167:3300/0 conn(0x7f59580aebd0 0x7f59580ced50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:32.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.317+0000 7f5957fff640 1 -- 172.21.15.83:0/2655026364 >> v1:172.21.15.83:6789/0 conn(0x7f595815b990 legacy=0x7f59580cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:32.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.317+0000 7f59567fc640 1 -- 172.21.15.83:0/2655026364 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2376010322 0 0) 0x7f59580cf3e0 con 0x7f595815b990 2024-04-17T10:42:32.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.317+0000 7f5957fff640 1 -- 172.21.15.83:0/2655026364 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59580cd890 con 0x7f595806abe0 2024-04-17T10:42:32.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.317+0000 7f59577fe640 1 --2- 172.21.15.83:0/2655026364 >> v2:172.21.15.167:3300/0 conn(0x7f59580aebd0 0x7f59580ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:32.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.321+0000 7f5957fff640 1 --2- 172.21.15.83:0/2655026364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f595806abe0 0x7f595806b030 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f595806fb10 tx=0x7f595001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bccff804000a8c05 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:32.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.321+0000 7f59567fc640 1 -- 172.21.15.83:0/2655026364 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59500b2070 con 0x7f595806abe0 2024-04-17T10:42:32.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.321+0000 7f59567fc640 1 -- 172.21.15.83:0/2655026364 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59500a58e0 con 0x7f595806abe0 2024-04-17T10:42:32.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.321+0000 7f59567fc640 1 -- 172.21.15.83:0/2655026364 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59500a5cb0 con 0x7f595806abe0 2024-04-17T10:42:32.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.321+0000 7f595db70640 1 -- 172.21.15.83:0/2655026364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f595806abe0 msgr2=0x7f595806b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:32.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.321+0000 7f595db70640 1 --2- 172.21.15.83:0/2655026364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f595806abe0 0x7f595806b030 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f595806fb10 tx=0x7f595001c930 comp rx=0 tx=0).stop 2024-04-17T10:42:32.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.321+0000 7f595db70640 1 -- 172.21.15.83:0/2655026364 shutdown_connections 2024-04-17T10:42:32.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.321+0000 7f595db70640 1 --2- 172.21.15.83:0/2655026364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f595806abe0 0x7f595806b030 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:32.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.321+0000 7f595db70640 1 --2- 172.21.15.83:0/2655026364 >> v2:172.21.15.167:3300/0 conn(0x7f59580aebd0 0x7f59580ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:32.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.321+0000 7f595db70640 1 -- 172.21.15.83:0/2655026364 >> 172.21.15.83:0/2655026364 conn(0x7f59580cc050 msgr2=0x7f59580cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:32.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.321+0000 7f595db70640 1 -- 172.21.15.83:0/2655026364 shutdown_connections 2024-04-17T10:42:32.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.321+0000 7f595db70640 1 -- 172.21.15.83:0/2655026364 wait complete. 2024-04-17T10:42:32.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.321+0000 7f595db70640 1 Processor -- start 2024-04-17T10:42:32.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.321+0000 7f595db70640 1 -- start start 2024-04-17T10:42:32.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.321+0000 7f595db70640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f595806abe0 0x7f5958152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:32.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.321+0000 7f595db70640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59580aebd0 0x7f59581f5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:32.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.321+0000 7f595db70640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f595815b990 0x7f595806c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:32.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.321+0000 7f595db70640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59580aefb0 con 0x7f595806abe0 2024-04-17T10:42:32.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.321+0000 7f595db70640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595815bde0 con 0x7f595815b990 2024-04-17T10:42:32.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.321+0000 7f595db70640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59581678d0 con 0x7f59580aebd0 2024-04-17T10:42:32.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.321+0000 7f59577fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f595806abe0 0x7f5958152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:32.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.321+0000 7f59577fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f595806abe0 0x7f5958152720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55392/0 (socket says 172.21.15.83:55392) 2024-04-17T10:42:32.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.321+0000 7f59577fe640 1 -- 172.21.15.83:0/4021050902 learned_addr learned my addr 172.21.15.83:0/4021050902 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:32.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.325+0000 7f59577fe640 1 -- 172.21.15.83:0/4021050902 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59580aebd0 msgr2=0x7f59581f5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:32.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.325+0000 7f59577fe640 1 --2- 172.21.15.83:0/4021050902 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59580aebd0 0x7f59581f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:32.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.325+0000 7f59577fe640 1 -- 172.21.15.83:0/4021050902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f595815b990 msgr2=0x7f595806c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:32.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.325+0000 7f59577fe640 1 --2- 172.21.15.83:0/4021050902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f595815b990 0x7f595806c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:32.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.325+0000 7f59577fe640 1 -- 172.21.15.83:0/4021050902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5958155d30 con 0x7f595806abe0 2024-04-17T10:42:32.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.325+0000 7f59577fe640 1 --2- 172.21.15.83:0/4021050902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f595806abe0 0x7f5958152720 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f594800c7f0 tx=0x7f594800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:32.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.325+0000 7f593ffff640 1 -- 172.21.15.83:0/4021050902 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5948007a30 con 0x7f595806abe0 2024-04-17T10:42:32.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.325+0000 7f593ffff640 1 -- 172.21.15.83:0/4021050902 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5948007bd0 con 0x7f595806abe0 2024-04-17T10:42:32.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.325+0000 7f595db70640 1 -- 172.21.15.83:0/4021050902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5958155fd0 con 0x7f595806abe0 2024-04-17T10:42:32.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.325+0000 7f593ffff640 1 -- 172.21.15.83:0/4021050902 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59480113f0 con 0x7f595806abe0 2024-04-17T10:42:32.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.325+0000 7f595db70640 1 -- 172.21.15.83:0/4021050902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59580acf30 con 0x7f595806abe0 2024-04-17T10:42:32.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.325+0000 7f595db70640 1 -- 172.21.15.83:0/4021050902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5958075000 con 0x7f595806abe0 2024-04-17T10:42:32.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.325+0000 7f593ffff640 1 -- 172.21.15.83:0/4021050902 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5948011590 con 0x7f595806abe0 2024-04-17T10:42:32.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.325+0000 7f593ffff640 1 --2- 172.21.15.83:0/4021050902 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5944066f60 0x7f5944069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:32.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.325+0000 7f593ffff640 1 -- 172.21.15.83:0/4021050902 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5948084f30 con 0x7f595806abe0 2024-04-17T10:42:32.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.325+0000 7f5956ffd640 1 --2- 172.21.15.83:0/4021050902 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5944066f60 0x7f5944069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:32.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.325+0000 7f5956ffd640 1 --2- 172.21.15.83:0/4021050902 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5944066f60 0x7f5944069420 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f595815aa50 tx=0x7f594c007970 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:32.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.329+0000 7f593ffff640 1 -- 172.21.15.83:0/4021050902 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5948015020 con 0x7f595806abe0 2024-04-17T10:42:32.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.485+0000 7f595db70640 1 -- 172.21.15.83:0/4021050902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f59581656a0 con 0x7f595806abe0 2024-04-17T10:42:32.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.953+0000 7f593ffff640 1 -- 172.21.15.83:0/4021050902 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v56) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5948056280 con 0x7f595806abe0 2024-04-17T10:42:32.955 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2024-04-17T10:42:32.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.957+0000 7f595db70640 1 -- 172.21.15.83:0/4021050902 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5944066f60 msgr2=0x7f5944069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:32.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.957+0000 7f595db70640 1 --2- 172.21.15.83:0/4021050902 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5944066f60 0x7f5944069420 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f595815aa50 tx=0x7f594c007970 comp rx=0 tx=0).stop 2024-04-17T10:42:32.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.957+0000 7f595db70640 1 -- 172.21.15.83:0/4021050902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f595806abe0 msgr2=0x7f5958152720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:32.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.957+0000 7f595db70640 1 --2- 172.21.15.83:0/4021050902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f595806abe0 0x7f5958152720 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f594800c7f0 tx=0x7f594800ccb0 comp rx=0 tx=0).stop 2024-04-17T10:42:32.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.957+0000 7f595db70640 1 -- 172.21.15.83:0/4021050902 shutdown_connections 2024-04-17T10:42:32.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.957+0000 7f595db70640 1 --2- 172.21.15.83:0/4021050902 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5944066f60 0x7f5944069420 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:32.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.957+0000 7f595db70640 1 --2- 172.21.15.83:0/4021050902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f595815b990 0x7f595806c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:32.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.957+0000 7f595db70640 1 --2- 172.21.15.83:0/4021050902 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59580aebd0 0x7f59581f5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:32.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.957+0000 7f595db70640 1 --2- 172.21.15.83:0/4021050902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f595806abe0 0x7f5958152720 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:32.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.957+0000 7f595db70640 1 -- 172.21.15.83:0/4021050902 >> 172.21.15.83:0/4021050902 conn(0x7f59580cc050 msgr2=0x7f595815afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:32.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.957+0000 7f595db70640 1 -- 172.21.15.83:0/4021050902 shutdown_connections 2024-04-17T10:42:32.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:32.957+0000 7f595db70640 1 -- 172.21.15.83:0/4021050902 wait complete. 2024-04-17T10:42:32.985 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-17T10:42:33.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.117+0000 7f6ca9bf0640 1 Processor -- start 2024-04-17T10:42:33.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.117+0000 7f6ca9bf0640 1 -- start start 2024-04-17T10:42:33.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.117+0000 7f6ca9bf0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6ca415b6e0 0x7f6ca4071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:33.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.117+0000 7f6ca9bf0640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6ca4070470 0x7f6ca406ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:33.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.117+0000 7f6ca9bf0640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca406c1f0 con 0x7f6ca415bac0 2024-04-17T10:42:33.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.117+0000 7f6ca9bf0640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ca4070a10 con 0x7f6ca415b6e0 2024-04-17T10:42:33.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.117+0000 7f6ca9bf0640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ca4070cf0 con 0x7f6ca4070470 2024-04-17T10:42:33.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.117+0000 7f6ca37fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6ca415b6e0 0x7f6ca4071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:33.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.117+0000 7f6ca37fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6ca415b6e0 0x7f6ca4071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55402/0 (socket says 172.21.15.83:55402) 2024-04-17T10:42:33.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.117+0000 7f6ca37fe640 1 -- 172.21.15.83:0/445159313 learned_addr learned my addr 172.21.15.83:0/445159313 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:33.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.117+0000 7f6ca3fff640 1 --2- 172.21.15.83:0/445159313 >> v2:172.21.15.167:3300/0 conn(0x7f6ca4070470 0x7f6ca406ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:33.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.117+0000 7f6ca37fe640 1 -- 172.21.15.83:0/445159313 >> v2:172.21.15.167:3300/0 conn(0x7f6ca4070470 msgr2=0x7f6ca406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:33.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.117+0000 7f6ca37fe640 1 --2- 172.21.15.83:0/445159313 >> v2:172.21.15.167:3300/0 conn(0x7f6ca4070470 0x7f6ca406ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:33.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.117+0000 7f6ca37fe640 1 -- 172.21.15.83:0/445159313 >> v1:172.21.15.83:6789/0 conn(0x7f6ca415bac0 legacy=0x7f6ca40cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:33.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.117+0000 7f6ca27fc640 1 -- 172.21.15.83:0/445159313 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1655204888 0 0) 0x7f6ca406c1f0 con 0x7f6ca415bac0 2024-04-17T10:42:33.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.117+0000 7f6ca37fe640 1 -- 172.21.15.83:0/445159313 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca4070fd0 con 0x7f6ca415b6e0 2024-04-17T10:42:33.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.121+0000 7f6ca37fe640 1 --2- 172.21.15.83:0/445159313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca415b6e0 0x7f6ca4071cc0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40cd5e0 tx=0x7f6c9801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c9d2018ace1ee4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:33.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.121+0000 7f6ca27fc640 1 -- 172.21.15.83:0/445159313 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c980b0070 con 0x7f6ca415b6e0 2024-04-17T10:42:33.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.121+0000 7f6ca27fc640 1 -- 172.21.15.83:0/445159313 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c980ab600 con 0x7f6ca415b6e0 2024-04-17T10:42:33.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.121+0000 7f6ca27fc640 1 -- 172.21.15.83:0/445159313 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c980ab9d0 con 0x7f6ca415b6e0 2024-04-17T10:42:33.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.121+0000 7f6ca9bf0640 1 -- 172.21.15.83:0/445159313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca415b6e0 msgr2=0x7f6ca4071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:33.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.121+0000 7f6ca9bf0640 1 --2- 172.21.15.83:0/445159313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca415b6e0 0x7f6ca4071cc0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40cd5e0 tx=0x7f6c9801c520 comp rx=0 tx=0).stop 2024-04-17T10:42:33.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.121+0000 7f6ca9bf0640 1 -- 172.21.15.83:0/445159313 shutdown_connections 2024-04-17T10:42:33.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.121+0000 7f6ca9bf0640 1 --2- 172.21.15.83:0/445159313 >> v2:172.21.15.167:3300/0 conn(0x7f6ca4070470 0x7f6ca406ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:33.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.121+0000 7f6ca9bf0640 1 --2- 172.21.15.83:0/445159313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca415b6e0 0x7f6ca4071cc0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:33.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.121+0000 7f6ca9bf0640 1 -- 172.21.15.83:0/445159313 >> 172.21.15.83:0/445159313 conn(0x7f6ca40b15b0 msgr2=0x7f6ca40b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:33.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.121+0000 7f6ca9bf0640 1 -- 172.21.15.83:0/445159313 shutdown_connections 2024-04-17T10:42:33.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.121+0000 7f6ca9bf0640 1 -- 172.21.15.83:0/445159313 wait complete. 2024-04-17T10:42:33.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.121+0000 7f6ca9bf0640 1 Processor -- start 2024-04-17T10:42:33.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.125+0000 7f6ca9bf0640 1 -- start start 2024-04-17T10:42:33.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.125+0000 7f6ca9bf0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca4070470 0x7f6ca414d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:33.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.125+0000 7f6ca9bf0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ca415b6e0 0x7f6ca415aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:33.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.125+0000 7f6ca9bf0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ca415bac0 0x7f6ca414a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:33.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.125+0000 7f6ca9bf0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca406d2b0 con 0x7f6ca4070470 2024-04-17T10:42:33.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.125+0000 7f6ca9bf0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca406d130 con 0x7f6ca415b6e0 2024-04-17T10:42:33.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.125+0000 7f6ca9bf0640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca406b760 con 0x7f6ca415bac0 2024-04-17T10:42:33.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.125+0000 7f6ca37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca4070470 0x7f6ca414d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:33.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.125+0000 7f6ca37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca4070470 0x7f6ca414d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55408/0 (socket says 172.21.15.83:55408) 2024-04-17T10:42:33.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.125+0000 7f6ca37fe640 1 -- 172.21.15.83:0/3851307506 learned_addr learned my addr 172.21.15.83:0/3851307506 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:33.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.125+0000 7f6ca2ffd640 1 --2- 172.21.15.83:0/3851307506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ca415b6e0 0x7f6ca415aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:33.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.125+0000 7f6ca3fff640 1 --2- 172.21.15.83:0/3851307506 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ca415bac0 0x7f6ca414a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:33.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.125+0000 7f6ca37fe640 1 -- 172.21.15.83:0/3851307506 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ca415bac0 msgr2=0x7f6ca414a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:33.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.125+0000 7f6ca37fe640 1 --2- 172.21.15.83:0/3851307506 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ca415bac0 0x7f6ca414a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:33.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.125+0000 7f6ca37fe640 1 -- 172.21.15.83:0/3851307506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ca415b6e0 msgr2=0x7f6ca415aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:33.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.125+0000 7f6ca37fe640 1 --2- 172.21.15.83:0/3851307506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ca415b6e0 0x7f6ca415aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:33.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.125+0000 7f6ca37fe640 1 -- 172.21.15.83:0/3851307506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca415c0b0 con 0x7f6ca4070470 2024-04-17T10:42:33.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.125+0000 7f6ca3fff640 1 --2- 172.21.15.83:0/3851307506 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ca415bac0 0x7f6ca414a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:33.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.125+0000 7f6ca2ffd640 1 --2- 172.21.15.83:0/3851307506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ca415b6e0 0x7f6ca415aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:33.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.125+0000 7f6ca37fe640 1 --2- 172.21.15.83:0/3851307506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca4070470 0x7f6ca414d650 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f6c9801abd0 tx=0x7f6c9801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:33.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.125+0000 7f6c8bfff640 1 -- 172.21.15.83:0/3851307506 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c980b0040 con 0x7f6ca4070470 2024-04-17T10:42:33.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.125+0000 7f6c8bfff640 1 -- 172.21.15.83:0/3851307506 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c980b6420 con 0x7f6ca4070470 2024-04-17T10:42:33.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.125+0000 7f6ca9bf0640 1 -- 172.21.15.83:0/3851307506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca414dd50 con 0x7f6ca4070470 2024-04-17T10:42:33.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.125+0000 7f6c8bfff640 1 -- 172.21.15.83:0/3851307506 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c980b67f0 con 0x7f6ca4070470 2024-04-17T10:42:33.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.125+0000 7f6ca9bf0640 1 -- 172.21.15.83:0/3851307506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca414a750 con 0x7f6ca4070470 2024-04-17T10:42:33.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.125+0000 7f6ca9bf0640 1 -- 172.21.15.83:0/3851307506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca4074f90 con 0x7f6ca4070470 2024-04-17T10:42:33.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.125+0000 7f6c8bfff640 1 -- 172.21.15.83:0/3851307506 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6c980be030 con 0x7f6ca4070470 2024-04-17T10:42:33.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.129+0000 7f6c8bfff640 1 --2- 172.21.15.83:0/3851307506 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6c90066f90 0x7f6c90069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:33.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.129+0000 7f6c8bfff640 1 -- 172.21.15.83:0/3851307506 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f6c9811f230 con 0x7f6ca4070470 2024-04-17T10:42:33.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.129+0000 7f6ca2ffd640 1 --2- 172.21.15.83:0/3851307506 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6c90066f90 0x7f6c90069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:33.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.129+0000 7f6ca2ffd640 1 --2- 172.21.15.83:0/3851307506 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6c90066f90 0x7f6c90069450 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6ca41665c0 tx=0x7f6c94009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:33.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.129+0000 7f6c8bfff640 1 -- 172.21.15.83:0/3851307506 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6c980c5060 con 0x7f6ca4070470 2024-04-17T10:42:33.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.289+0000 7f6ca9bf0640 1 -- 172.21.15.83:0/3851307506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6ca40b1ac0 con 0x7f6ca4070470 2024-04-17T10:42:33.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.965+0000 7f6c8bfff640 1 -- 172.21.15.83:0/3851307506 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v57) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6c98016110 con 0x7f6ca4070470 2024-04-17T10:42:33.967 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2024-04-17T10:42:33.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.973+0000 7f6ca9bf0640 1 -- 172.21.15.83:0/3851307506 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6c90066f90 msgr2=0x7f6c90069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:33.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.973+0000 7f6ca9bf0640 1 --2- 172.21.15.83:0/3851307506 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6c90066f90 0x7f6c90069450 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6ca41665c0 tx=0x7f6c94009040 comp rx=0 tx=0).stop 2024-04-17T10:42:33.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.973+0000 7f6ca9bf0640 1 -- 172.21.15.83:0/3851307506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca4070470 msgr2=0x7f6ca414d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:33.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.973+0000 7f6ca9bf0640 1 --2- 172.21.15.83:0/3851307506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca4070470 0x7f6ca414d650 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f6c9801abd0 tx=0x7f6c9801ad60 comp rx=0 tx=0).stop 2024-04-17T10:42:33.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.973+0000 7f6ca9bf0640 1 -- 172.21.15.83:0/3851307506 shutdown_connections 2024-04-17T10:42:33.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.973+0000 7f6ca9bf0640 1 --2- 172.21.15.83:0/3851307506 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6c90066f90 0x7f6c90069450 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:33.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.973+0000 7f6ca9bf0640 1 --2- 172.21.15.83:0/3851307506 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ca415bac0 0x7f6ca414a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:33.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.973+0000 7f6ca9bf0640 1 --2- 172.21.15.83:0/3851307506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ca415b6e0 0x7f6ca415aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:33.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.973+0000 7f6ca9bf0640 1 --2- 172.21.15.83:0/3851307506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca4070470 0x7f6ca414d650 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:33.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.973+0000 7f6ca9bf0640 1 -- 172.21.15.83:0/3851307506 >> 172.21.15.83:0/3851307506 conn(0x7f6ca40b15b0 msgr2=0x7f6ca40cd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:33.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.973+0000 7f6ca9bf0640 1 -- 172.21.15.83:0/3851307506 shutdown_connections 2024-04-17T10:42:33.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:33.973+0000 7f6ca9bf0640 1 -- 172.21.15.83:0/3851307506 wait complete. 2024-04-17T10:42:34.010 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-17T10:42:34.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.129+0000 7fb40bcd9640 1 Processor -- start 2024-04-17T10:42:34.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.129+0000 7fb40bcd9640 1 -- start start 2024-04-17T10:42:34.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.129+0000 7fb40bcd9640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb40406ccd0 0x7fb4040cc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:34.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.129+0000 7fb40bcd9640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb4040b09e0 0x7fb4040cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:34.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.129+0000 7fb40bcd9640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4040d0480 con 0x7fb40415de10 2024-04-17T10:42:34.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.129+0000 7fb40bcd9640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb40406c2e0 con 0x7fb40406ccd0 2024-04-17T10:42:34.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.129+0000 7fb40bcd9640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4040d0740 con 0x7fb4040b09e0 2024-04-17T10:42:34.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.129+0000 7fb409274640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb40406ccd0 0x7fb4040cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:34.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.129+0000 7fb409274640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb40406ccd0 0x7fb4040cc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55412/0 (socket says 172.21.15.83:55412) 2024-04-17T10:42:34.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.129+0000 7fb409274640 1 -- 172.21.15.83:0/3301251620 learned_addr learned my addr 172.21.15.83:0/3301251620 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:34.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.129+0000 7fb40a276640 1 --2- 172.21.15.83:0/3301251620 >> v2:172.21.15.167:3300/0 conn(0x7fb4040b09e0 0x7fb4040cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:34.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.129+0000 7fb409274640 1 -- 172.21.15.83:0/3301251620 >> v2:172.21.15.167:3300/0 conn(0x7fb4040b09e0 msgr2=0x7fb4040cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:34.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.129+0000 7fb409274640 1 --2- 172.21.15.83:0/3301251620 >> v2:172.21.15.167:3300/0 conn(0x7fb4040b09e0 0x7fb4040cc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:34.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.129+0000 7fb409274640 1 -- 172.21.15.83:0/3301251620 >> v1:172.21.15.83:6789/0 conn(0x7fb40415de10 legacy=0x7fb40406c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:34.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.129+0000 7fb409274640 1 -- 172.21.15.83:0/3301251620 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4040d0a20 con 0x7fb40406ccd0 2024-04-17T10:42:34.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.129+0000 7fb409274640 1 --2- 172.21.15.83:0/3301251620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb40406ccd0 0x7fb4040cc060 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fb4040cbd10 tx=0x7fb3f801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4939a17ba87995b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:34.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.129+0000 7fb408a73640 1 -- 172.21.15.83:0/3301251620 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f80a5700 con 0x7fb40406ccd0 2024-04-17T10:42:34.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb408a73640 1 -- 172.21.15.83:0/3301251620 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f80a58a0 con 0x7fb40406ccd0 2024-04-17T10:42:34.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb408a73640 1 -- 172.21.15.83:0/3301251620 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f80a5c70 con 0x7fb40406ccd0 2024-04-17T10:42:34.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb40bcd9640 1 -- 172.21.15.83:0/3301251620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb40406ccd0 msgr2=0x7fb4040cc060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:34.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb40bcd9640 1 --2- 172.21.15.83:0/3301251620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb40406ccd0 0x7fb4040cc060 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fb4040cbd10 tx=0x7fb3f801c930 comp rx=0 tx=0).stop 2024-04-17T10:42:34.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb40bcd9640 1 -- 172.21.15.83:0/3301251620 shutdown_connections 2024-04-17T10:42:34.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb40bcd9640 1 --2- 172.21.15.83:0/3301251620 >> v2:172.21.15.167:3300/0 conn(0x7fb4040b09e0 0x7fb4040cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:34.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb40bcd9640 1 --2- 172.21.15.83:0/3301251620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb40406ccd0 0x7fb4040cc060 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:34.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb40bcd9640 1 -- 172.21.15.83:0/3301251620 >> 172.21.15.83:0/3301251620 conn(0x7fb4040d1730 msgr2=0x7fb4040d1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:34.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb40bcd9640 1 -- 172.21.15.83:0/3301251620 shutdown_connections 2024-04-17T10:42:34.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb40bcd9640 1 -- 172.21.15.83:0/3301251620 wait complete. 2024-04-17T10:42:34.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb40bcd9640 1 Processor -- start 2024-04-17T10:42:34.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb40bcd9640 1 -- start start 2024-04-17T10:42:34.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb40bcd9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb40406ccd0 0x7fb4040c7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:34.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb40bcd9640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4040b09e0 0x7fb404154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:34.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb40bcd9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb40415de10 0x7fb4041588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:34.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb40bcd9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40416ab60 con 0x7fb40415de10 2024-04-17T10:42:34.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb40bcd9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40416a9e0 con 0x7fb40406ccd0 2024-04-17T10:42:34.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb40bcd9640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb40416ace0 con 0x7fb4040b09e0 2024-04-17T10:42:34.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb40a276640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb40415de10 0x7fb4041588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:34.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb40a276640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb40415de10 0x7fb4041588d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55424/0 (socket says 172.21.15.83:55424) 2024-04-17T10:42:34.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb40a276640 1 -- 172.21.15.83:0/903923502 learned_addr learned my addr 172.21.15.83:0/903923502 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:34.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb409274640 1 --2- 172.21.15.83:0/903923502 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4040b09e0 0x7fb404154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:34.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb409a75640 1 --2- 172.21.15.83:0/903923502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb40406ccd0 0x7fb4040c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:34.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb40a276640 1 -- 172.21.15.83:0/903923502 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4040b09e0 msgr2=0x7fb404154b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:34.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb40a276640 1 --2- 172.21.15.83:0/903923502 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4040b09e0 0x7fb404154b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:34.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb40a276640 1 -- 172.21.15.83:0/903923502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb40406ccd0 msgr2=0x7fb4040c7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:34.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb40a276640 1 --2- 172.21.15.83:0/903923502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb40406ccd0 0x7fb4040c7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:34.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb40a276640 1 -- 172.21.15.83:0/903923502 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb404159d00 con 0x7fb40415de10 2024-04-17T10:42:34.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb40a276640 1 --2- 172.21.15.83:0/903923502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb40415de10 0x7fb4041588d0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00c7f0 tx=0x7fb3fc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:34.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb4027fc640 1 -- 172.21.15.83:0/903923502 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc015070 con 0x7fb40415de10 2024-04-17T10:42:34.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb4027fc640 1 -- 172.21.15.83:0/903923502 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3fc004220 con 0x7fb40415de10 2024-04-17T10:42:34.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb40bcd9640 1 -- 172.21.15.83:0/903923502 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4040c7810 con 0x7fb40415de10 2024-04-17T10:42:34.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb4027fc640 1 -- 172.21.15.83:0/903923502 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc003620 con 0x7fb40415de10 2024-04-17T10:42:34.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.133+0000 7fb40bcd9640 1 -- 172.21.15.83:0/903923502 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4040c7dd0 con 0x7fb40415de10 2024-04-17T10:42:34.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.137+0000 7fb40bcd9640 1 -- 172.21.15.83:0/903923502 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb404159ed0 con 0x7fb40415de10 2024-04-17T10:42:34.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.137+0000 7fb4027fc640 1 -- 172.21.15.83:0/903923502 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb3fc0039d0 con 0x7fb40415de10 2024-04-17T10:42:34.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.137+0000 7fb4027fc640 1 --2- 172.21.15.83:0/903923502 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb3f0066f60 0x7fb3f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:34.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.137+0000 7fb4027fc640 1 -- 172.21.15.83:0/903923502 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fb3fc085480 con 0x7fb40415de10 2024-04-17T10:42:34.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.137+0000 7fb409a75640 1 --2- 172.21.15.83:0/903923502 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb3f0066f60 0x7fb3f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:34.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.137+0000 7fb409a75640 1 --2- 172.21.15.83:0/903923502 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb3f0066f60 0x7fb3f0069420 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4007fd0 tx=0x7fb3f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:34.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.141+0000 7fb4027fc640 1 -- 172.21.15.83:0/903923502 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb3fc056600 con 0x7fb40415de10 2024-04-17T10:42:34.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:34.321+0000 7fb40bcd9640 1 -- 172.21.15.83:0/903923502 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb404074f90 con 0x7fb40415de10 2024-04-17T10:42:35.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.001+0000 7fb4027fc640 1 -- 172.21.15.83:0/903923502 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v28) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb3fc055f00 con 0x7fb40415de10 2024-04-17T10:42:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 8 and data pool 9 2024-04-17T10:42:35.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.017+0000 7fb40bcd9640 1 -- 172.21.15.83:0/903923502 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb3f0066f60 msgr2=0x7fb3f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:35.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.017+0000 7fb40bcd9640 1 --2- 172.21.15.83:0/903923502 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb3f0066f60 0x7fb3f0069420 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4007fd0 tx=0x7fb3f4009040 comp rx=0 tx=0).stop 2024-04-17T10:42:35.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.017+0000 7fb40bcd9640 1 -- 172.21.15.83:0/903923502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb40415de10 msgr2=0x7fb4041588d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:35.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.017+0000 7fb40bcd9640 1 --2- 172.21.15.83:0/903923502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb40415de10 0x7fb4041588d0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00c7f0 tx=0x7fb3fc00ccb0 comp rx=0 tx=0).stop 2024-04-17T10:42:35.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.017+0000 7fb40bcd9640 1 -- 172.21.15.83:0/903923502 shutdown_connections 2024-04-17T10:42:35.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.017+0000 7fb40bcd9640 1 --2- 172.21.15.83:0/903923502 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb3f0066f60 0x7fb3f0069420 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:35.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.017+0000 7fb40bcd9640 1 --2- 172.21.15.83:0/903923502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb40415de10 0x7fb4041588d0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:35.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.017+0000 7fb40bcd9640 1 --2- 172.21.15.83:0/903923502 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4040b09e0 0x7fb404154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:35.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.017+0000 7fb40bcd9640 1 --2- 172.21.15.83:0/903923502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb40406ccd0 0x7fb4040c7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:35.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.017+0000 7fb40bcd9640 1 -- 172.21.15.83:0/903923502 >> 172.21.15.83:0/903923502 conn(0x7fb4040d1730 msgr2=0x7fb40406c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:35.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.021+0000 7fb40bcd9640 1 -- 172.21.15.83:0/903923502 shutdown_connections 2024-04-17T10:42:35.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.021+0000 7fb40bcd9640 1 -- 172.21.15.83:0/903923502 wait complete. 2024-04-17T10:42:35.044 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:42:35.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.165+0000 7fcd6dff4640 1 Processor -- start 2024-04-17T10:42:35.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.165+0000 7fcd6dff4640 1 -- start start 2024-04-17T10:42:35.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.165+0000 7fcd6dff4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcd6815ba30 0x7fcd68072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:35.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.165+0000 7fcd6dff4640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcd680cfbe0 0x7fcd6806c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:35.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.165+0000 7fcd6dff4640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd6806e320 con 0x7fcd6815c960 2024-04-17T10:42:35.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.165+0000 7fcd6dff4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd68071ea0 con 0x7fcd6815ba30 2024-04-17T10:42:35.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.165+0000 7fcd6dff4640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd6806e040 con 0x7fcd680cfbe0 2024-04-17T10:42:35.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.165+0000 7fcd677fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcd6815ba30 0x7fcd68072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:35.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.165+0000 7fcd677fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcd6815ba30 0x7fcd68072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55430/0 (socket says 172.21.15.83:55430) 2024-04-17T10:42:35.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.165+0000 7fcd677fe640 1 -- 172.21.15.83:0/1822656309 learned_addr learned my addr 172.21.15.83:0/1822656309 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:35.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.165+0000 7fcd67fff640 1 --2- 172.21.15.83:0/1822656309 >> v2:172.21.15.167:3300/0 conn(0x7fcd680cfbe0 0x7fcd6806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:35.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.165+0000 7fcd677fe640 1 -- 172.21.15.83:0/1822656309 >> v2:172.21.15.167:3300/0 conn(0x7fcd680cfbe0 msgr2=0x7fcd6806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:35.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.165+0000 7fcd677fe640 1 --2- 172.21.15.83:0/1822656309 >> v2:172.21.15.167:3300/0 conn(0x7fcd680cfbe0 0x7fcd6806c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:35.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.165+0000 7fcd677fe640 1 -- 172.21.15.83:0/1822656309 >> v1:172.21.15.167:6789/0 conn(0x7fcd6815c960 legacy=0x7fcd680cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:35.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.165+0000 7fcd677fe640 1 -- 172.21.15.83:0/1822656309 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd68071850 con 0x7fcd6815ba30 2024-04-17T10:42:35.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.165+0000 7fcd67fff640 1 --2- 172.21.15.83:0/1822656309 >> v2:172.21.15.167:3300/0 conn(0x7fcd680cfbe0 0x7fcd6806c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:35.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.169+0000 7fcd677fe640 1 --2- 172.21.15.83:0/1822656309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd6815ba30 0x7fcd68072330 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fcd680b2410 tx=0x7fcd5001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77272553799e8a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:35.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.169+0000 7fcd667fc640 1 -- 172.21.15.83:0/1822656309 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd500b0070 con 0x7fcd6815ba30 2024-04-17T10:42:35.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.169+0000 7fcd667fc640 1 -- 172.21.15.83:0/1822656309 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd500ab600 con 0x7fcd6815ba30 2024-04-17T10:42:35.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.169+0000 7fcd667fc640 1 -- 172.21.15.83:0/1822656309 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd500ab9d0 con 0x7fcd6815ba30 2024-04-17T10:42:35.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.169+0000 7fcd6dff4640 1 -- 172.21.15.83:0/1822656309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd6815ba30 msgr2=0x7fcd68072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:35.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.169+0000 7fcd6dff4640 1 --2- 172.21.15.83:0/1822656309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd6815ba30 0x7fcd68072330 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fcd680b2410 tx=0x7fcd5001c430 comp rx=0 tx=0).stop 2024-04-17T10:42:35.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.169+0000 7fcd6dff4640 1 -- 172.21.15.83:0/1822656309 shutdown_connections 2024-04-17T10:42:35.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.169+0000 7fcd6dff4640 1 --2- 172.21.15.83:0/1822656309 >> v2:172.21.15.167:3300/0 conn(0x7fcd680cfbe0 0x7fcd6806c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:35.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.169+0000 7fcd6dff4640 1 --2- 172.21.15.83:0/1822656309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd6815ba30 0x7fcd68072330 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:35.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.169+0000 7fcd6dff4640 1 -- 172.21.15.83:0/1822656309 >> 172.21.15.83:0/1822656309 conn(0x7fcd6806abe0 msgr2=0x7fcd6806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:35.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.169+0000 7fcd6dff4640 1 -- 172.21.15.83:0/1822656309 shutdown_connections 2024-04-17T10:42:35.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.169+0000 7fcd6dff4640 1 -- 172.21.15.83:0/1822656309 wait complete. 2024-04-17T10:42:35.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.169+0000 7fcd6dff4640 1 Processor -- start 2024-04-17T10:42:35.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.173+0000 7fcd6dff4640 1 -- start start 2024-04-17T10:42:35.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.173+0000 7fcd6dff4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd680cfbe0 0x7fcd68149150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:35.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.173+0000 7fcd6dff4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd6815ba30 0x7fcd68147f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:35.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.173+0000 7fcd6dff4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd6815c960 0x7fcd68149ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:35.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.173+0000 7fcd6dff4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6806de20 con 0x7fcd680cfbe0 2024-04-17T10:42:35.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.173+0000 7fcd6dff4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6806dca0 con 0x7fcd6815c960 2024-04-17T10:42:35.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.173+0000 7fcd6dff4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd68168520 con 0x7fcd6815ba30 2024-04-17T10:42:35.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.173+0000 7fcd677fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd680cfbe0 0x7fcd68149150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:35.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.173+0000 7fcd677fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd680cfbe0 0x7fcd68149150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55440/0 (socket says 172.21.15.83:55440) 2024-04-17T10:42:35.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.173+0000 7fcd677fe640 1 -- 172.21.15.83:0/1650170253 learned_addr learned my addr 172.21.15.83:0/1650170253 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:35.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.173+0000 7fcd66ffd640 1 --2- 172.21.15.83:0/1650170253 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd6815ba30 0x7fcd68147f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:35.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.173+0000 7fcd67fff640 1 --2- 172.21.15.83:0/1650170253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd6815c960 0x7fcd68149ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:35.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.173+0000 7fcd677fe640 1 -- 172.21.15.83:0/1650170253 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd6815ba30 msgr2=0x7fcd68147f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:35.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.173+0000 7fcd677fe640 1 --2- 172.21.15.83:0/1650170253 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd6815ba30 0x7fcd68147f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:35.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.173+0000 7fcd677fe640 1 -- 172.21.15.83:0/1650170253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd6815c960 msgr2=0x7fcd68149ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:35.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.173+0000 7fcd677fe640 1 --2- 172.21.15.83:0/1650170253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd6815c960 0x7fcd68149ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:35.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.173+0000 7fcd677fe640 1 -- 172.21.15.83:0/1650170253 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd6814ba70 con 0x7fcd680cfbe0 2024-04-17T10:42:35.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.173+0000 7fcd67fff640 1 --2- 172.21.15.83:0/1650170253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd6815c960 0x7fcd68149ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:35.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.173+0000 7fcd677fe640 1 --2- 172.21.15.83:0/1650170253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd680cfbe0 0x7fcd68149150 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fcd500ae040 tx=0x7fcd5001ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:35.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.173+0000 7fcd5ffff640 1 -- 172.21.15.83:0/1650170253 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd500ba020 con 0x7fcd680cfbe0 2024-04-17T10:42:35.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.173+0000 7fcd5ffff640 1 -- 172.21.15.83:0/1650170253 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd500b6640 con 0x7fcd680cfbe0 2024-04-17T10:42:35.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.173+0000 7fcd6dff4640 1 -- 172.21.15.83:0/1650170253 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd6814bd10 con 0x7fcd680cfbe0 2024-04-17T10:42:35.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.173+0000 7fcd5ffff640 1 -- 172.21.15.83:0/1650170253 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd500b0040 con 0x7fcd680cfbe0 2024-04-17T10:42:35.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.173+0000 7fcd6dff4640 1 -- 172.21.15.83:0/1650170253 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd6814c2d0 con 0x7fcd680cfbe0 2024-04-17T10:42:35.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.173+0000 7fcd6dff4640 1 -- 172.21.15.83:0/1650170253 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd68074f90 con 0x7fcd680cfbe0 2024-04-17T10:42:35.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.173+0000 7fcd5ffff640 1 -- 172.21.15.83:0/1650170253 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcd500bd020 con 0x7fcd680cfbe0 2024-04-17T10:42:35.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.173+0000 7fcd5ffff640 1 --2- 172.21.15.83:0/1650170253 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd48066f60 0x7fcd48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:35.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.173+0000 7fcd5ffff640 1 -- 172.21.15.83:0/1650170253 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd5011f2f0 con 0x7fcd680cfbe0 2024-04-17T10:42:35.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.173+0000 7fcd66ffd640 1 --2- 172.21.15.83:0/1650170253 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd48066f60 0x7fcd48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:35.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.177+0000 7fcd66ffd640 1 --2- 172.21.15.83:0/1650170253 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd48066f60 0x7fcd48069420 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fcd6814fc00 tx=0x7fcd58009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:35.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.177+0000 7fcd5ffff640 1 -- 172.21.15.83:0/1650170253 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcd500c4030 con 0x7fcd680cfbe0 2024-04-17T10:42:35.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.337+0000 7fcd6dff4640 1 -- 172.21.15.83:0/1650170253 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd68152e30 con 0x7fcd680cfbe0 2024-04-17T10:42:35.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.337+0000 7fcd5ffff640 1 -- 172.21.15.83:0/1650170253 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16606 (secure 0 0 0) 0x7fcd500160b0 con 0x7fcd680cfbe0 2024-04-17T10:42:35.339 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:35.339 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":58,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:42:34.327690+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:42:32.530700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-17T10:42:33.334905+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:42:35.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.341+0000 7fcd6dff4640 1 -- 172.21.15.83:0/1650170253 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd48066f60 msgr2=0x7fcd48069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:35.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.341+0000 7fcd6dff4640 1 --2- 172.21.15.83:0/1650170253 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd48066f60 0x7fcd48069420 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fcd6814fc00 tx=0x7fcd58009040 comp rx=0 tx=0).stop 2024-04-17T10:42:35.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.341+0000 7fcd6dff4640 1 -- 172.21.15.83:0/1650170253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd680cfbe0 msgr2=0x7fcd68149150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:35.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.341+0000 7fcd6dff4640 1 --2- 172.21.15.83:0/1650170253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd680cfbe0 0x7fcd68149150 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fcd500ae040 tx=0x7fcd5001ac70 comp rx=0 tx=0).stop 2024-04-17T10:42:35.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.341+0000 7fcd6dff4640 1 -- 172.21.15.83:0/1650170253 shutdown_connections 2024-04-17T10:42:35.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.341+0000 7fcd6dff4640 1 --2- 172.21.15.83:0/1650170253 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd48066f60 0x7fcd48069420 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:35.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.341+0000 7fcd6dff4640 1 --2- 172.21.15.83:0/1650170253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd6815c960 0x7fcd68149ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:35.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.341+0000 7fcd6dff4640 1 --2- 172.21.15.83:0/1650170253 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd6815ba30 0x7fcd68147f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:35.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.341+0000 7fcd6dff4640 1 --2- 172.21.15.83:0/1650170253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd680cfbe0 0x7fcd68149150 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:35.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.341+0000 7fcd6dff4640 1 -- 172.21.15.83:0/1650170253 >> 172.21.15.83:0/1650170253 conn(0x7fcd6806abe0 msgr2=0x7fcd6815aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:35.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.341+0000 7fcd6dff4640 1 -- 172.21.15.83:0/1650170253 shutdown_connections 2024-04-17T10:42:35.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.341+0000 7fcd6dff4640 1 -- 172.21.15.83:0/1650170253 wait complete. 2024-04-17T10:42:35.359 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:42:35.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f4508021640 1 Processor -- start 2024-04-17T10:42:35.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f4508021640 1 -- start start 2024-04-17T10:42:35.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f4508021640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4500146e70 0x7f450006d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:35.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f4508021640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4500147570 con 0x7f4500146a00 2024-04-17T10:42:35.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f4508021640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f450006cbb0 con 0x7f45000b2d80 2024-04-17T10:42:35.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f4508021640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f450006c960 con 0x7f4500146e70 2024-04-17T10:42:35.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f4505dbd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f4500146a00 0x7f45000b2670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39814/0 (socket says 172.21.15.83:39814) 2024-04-17T10:42:35.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f4505dbd640 1 -- 172.21.15.83:0/2222406965 learned_addr learned my addr 172.21.15.83:0/2222406965 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:35.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f45065be640 1 --2- 172.21.15.83:0/2222406965 >> v2:172.21.15.167:3300/0 conn(0x7f4500146e70 0x7f450006d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:35.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f4504dbb640 1 -- 172.21.15.83:0/2222406965 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1982004902 0 0) 0x7f4500147570 con 0x7f4500146a00 2024-04-17T10:42:35.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f4504dbb640 1 -- 172.21.15.83:0/2222406965 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44e8003530 con 0x7f4500146a00 2024-04-17T10:42:35.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f4504dbb640 1 -- 172.21.15.83:0/2222406965 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1838591985 0 0) 0x7f44e8003530 con 0x7f4500146a00 2024-04-17T10:42:35.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f4504dbb640 1 -- 172.21.15.83:0/2222406965 >> v2:172.21.15.167:3300/0 conn(0x7f4500146e70 msgr2=0x7f450006d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:35.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f4504dbb640 1 --2- 172.21.15.83:0/2222406965 >> v2:172.21.15.167:3300/0 conn(0x7f4500146e70 0x7f450006d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:35.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f4504dbb640 1 -- 172.21.15.83:0/2222406965 >> v1:172.21.15.167:6789/0 conn(0x7f45000b2d80 legacy=0x7f4500070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:35.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f4504dbb640 1 -- 172.21.15.83:0/2222406965 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f450006ce90 con 0x7f4500146a00 2024-04-17T10:42:35.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f4504dbb640 1 -- 172.21.15.83:0/2222406965 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f44f8001680 con 0x7f4500146a00 2024-04-17T10:42:35.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f4504dbb640 1 -- 172.21.15.83:0/2222406965 >> v1:172.21.15.83:6789/0 conn(0x7f4500146a00 legacy=0x7f45000b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:35.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f4504dbb640 1 --2- 172.21.15.83:0/2222406965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44e80039c0 0x7f44e8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:35.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f4504dbb640 1 --2- 172.21.15.83:0/2222406965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44e8004810 0x7f44e8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:35.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f4504dbb640 1 --2- 172.21.15.83:0/2222406965 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44e80051d0 0x7f44e800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:35.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f4504dbb640 1 -- 172.21.15.83:0/2222406965 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450006c960 con 0x7f44e8004810 2024-04-17T10:42:35.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f4504dbb640 1 -- 172.21.15.83:0/2222406965 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e800c070 con 0x7f44e80039c0 2024-04-17T10:42:35.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f4504dbb640 1 -- 172.21.15.83:0/2222406965 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e800c270 con 0x7f44e80051d0 2024-04-17T10:42:35.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f45055bc640 1 --2- 172.21.15.83:0/2222406965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44e8004810 0x7f44e8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:35.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f45065be640 1 --2- 172.21.15.83:0/2222406965 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44e80051d0 0x7f44e800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:35.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f4505dbd640 1 --2- 172.21.15.83:0/2222406965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44e80039c0 0x7f44e8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:35.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f45055bc640 1 -- 172.21.15.83:0/2222406965 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44e80051d0 msgr2=0x7f44e800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:35.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f45055bc640 1 --2- 172.21.15.83:0/2222406965 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44e80051d0 0x7f44e800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:35.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f45055bc640 1 -- 172.21.15.83:0/2222406965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44e80039c0 msgr2=0x7f44e8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:35.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f45055bc640 1 --2- 172.21.15.83:0/2222406965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44e80039c0 0x7f44e8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:35.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f45055bc640 1 -- 172.21.15.83:0/2222406965 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e8003530 con 0x7f44e8004810 2024-04-17T10:42:35.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f45065be640 1 -- 172.21.15.83:0/2222406965 reap_dead start 2024-04-17T10:42:35.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.477+0000 7f4505dbd640 1 --2- 172.21.15.83:0/2222406965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44e80039c0 0x7f44e8003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:35.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.481+0000 7f45055bc640 1 --2- 172.21.15.83:0/2222406965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44e8004810 0x7f44e8004c90 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f45001479a0 tx=0x7f44f001c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805c4735531f0387 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:35.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.481+0000 7f4504dbb640 1 -- 172.21.15.83:0/2222406965 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f0001a90 con 0x7f44e8004810 2024-04-17T10:42:35.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.481+0000 7f4504dbb640 1 -- 172.21.15.83:0/2222406965 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f0003b80 con 0x7f44e8004810 2024-04-17T10:42:35.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.481+0000 7f4504dbb640 1 -- 172.21.15.83:0/2222406965 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f00050d0 con 0x7f44e8004810 2024-04-17T10:42:35.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.481+0000 7f4508021640 1 -- 172.21.15.83:0/2222406965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44e8004810 msgr2=0x7f44e8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:35.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.481+0000 7f4508021640 1 --2- 172.21.15.83:0/2222406965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44e8004810 0x7f44e8004c90 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f45001479a0 tx=0x7f44f001c6e0 comp rx=0 tx=0).stop 2024-04-17T10:42:35.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.481+0000 7f4508021640 1 -- 172.21.15.83:0/2222406965 shutdown_connections 2024-04-17T10:42:35.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.481+0000 7f4508021640 1 --2- 172.21.15.83:0/2222406965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44e8004810 0x7f44e8004c90 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:35.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.481+0000 7f4508021640 1 -- 172.21.15.83:0/2222406965 >> 172.21.15.83:0/2222406965 conn(0x7f450006abe0 msgr2=0x7f450006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:35.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.481+0000 7f4508021640 1 -- 172.21.15.83:0/2222406965 shutdown_connections 2024-04-17T10:42:35.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.481+0000 7f4508021640 1 -- 172.21.15.83:0/2222406965 wait complete. 2024-04-17T10:42:35.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.481+0000 7f4508021640 1 Processor -- start 2024-04-17T10:42:35.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.481+0000 7f4508021640 1 -- start start 2024-04-17T10:42:35.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.481+0000 7f4508021640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44e8004810 0x7f450015d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:35.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.481+0000 7f4508021640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45001e8000 0x7f4500070d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:35.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.481+0000 7f4508021640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f45000712a0 0x7f450015fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:35.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.481+0000 7f4508021640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450006da20 con 0x7f44e8004810 2024-04-17T10:42:35.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.481+0000 7f4508021640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450006d8a0 con 0x7f45001e8000 2024-04-17T10:42:35.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.481+0000 7f4508021640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f450006d200 con 0x7f45000712a0 2024-04-17T10:42:35.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.481+0000 7f4505dbd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44e8004810 0x7f450015d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:35.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.481+0000 7f4505dbd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44e8004810 0x7f450015d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55456/0 (socket says 172.21.15.83:55456) 2024-04-17T10:42:35.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.481+0000 7f4505dbd640 1 -- 172.21.15.83:0/1080680559 learned_addr learned my addr 172.21.15.83:0/1080680559 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:35.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.485+0000 7f45055bc640 1 --2- 172.21.15.83:0/1080680559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45001e8000 0x7f4500070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:35.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.485+0000 7f4505dbd640 1 -- 172.21.15.83:0/1080680559 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f45000712a0 msgr2=0x7f450015fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:35.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.485+0000 7f4505dbd640 1 --2- 172.21.15.83:0/1080680559 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f45000712a0 0x7f450015fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:35.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.485+0000 7f4505dbd640 1 -- 172.21.15.83:0/1080680559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45001e8000 msgr2=0x7f4500070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:35.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.485+0000 7f4505dbd640 1 --2- 172.21.15.83:0/1080680559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45001e8000 0x7f4500070d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:35.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.485+0000 7f4505dbd640 1 -- 172.21.15.83:0/1080680559 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45001e7240 con 0x7f44e8004810 2024-04-17T10:42:35.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.485+0000 7f4505dbd640 1 --2- 172.21.15.83:0/1080680559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44e8004810 0x7f450015d370 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f44f8010b10 tx=0x7f44f80057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:35.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.485+0000 7f44fe7fc640 1 -- 172.21.15.83:0/1080680559 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f801b070 con 0x7f44e8004810 2024-04-17T10:42:35.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.485+0000 7f44fe7fc640 1 -- 172.21.15.83:0/1080680559 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f8011040 con 0x7f44e8004810 2024-04-17T10:42:35.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.485+0000 7f4508021640 1 -- 172.21.15.83:0/1080680559 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45001e7520 con 0x7f44e8004810 2024-04-17T10:42:35.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.485+0000 7f44fe7fc640 1 -- 172.21.15.83:0/1080680559 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f800ed60 con 0x7f44e8004810 2024-04-17T10:42:35.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.485+0000 7f4508021640 1 -- 172.21.15.83:0/1080680559 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f450015d8b0 con 0x7f44e8004810 2024-04-17T10:42:35.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.485+0000 7f4508021640 1 -- 172.21.15.83:0/1080680559 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45000716c0 con 0x7f44e8004810 2024-04-17T10:42:35.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.485+0000 7f44fe7fc640 1 -- 172.21.15.83:0/1080680559 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f44f801e070 con 0x7f44e8004810 2024-04-17T10:42:35.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.485+0000 7f44fe7fc640 1 --2- 172.21.15.83:0/1080680559 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f44ec066f60 0x7f44ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:35.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.485+0000 7f45055bc640 1 --2- 172.21.15.83:0/1080680559 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f44ec066f60 0x7f44ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:35.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.485+0000 7f44fe7fc640 1 -- 172.21.15.83:0/1080680559 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f44f80879b0 con 0x7f44e8004810 2024-04-17T10:42:35.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.485+0000 7f45055bc640 1 --2- 172.21.15.83:0/1080680559 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f44ec066f60 0x7f44ec069420 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f44f00ad000 tx=0x7f44f0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:35.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.489+0000 7f44fe7fc640 1 -- 172.21.15.83:0/1080680559 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f44f8058a50 con 0x7f44e8004810 2024-04-17T10:42:35.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.649+0000 7f4508021640 1 -- 172.21.15.83:0/1080680559 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4500074f90 con 0x7f44e8004810 2024-04-17T10:42:35.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.649+0000 7f44fe7fc640 1 -- 172.21.15.83:0/1080680559 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16606 (secure 0 0 0) 0x7f44f8058350 con 0x7f44e8004810 2024-04-17T10:42:35.650 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:35.651 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":58,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:42:34.327690+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:42:32.530700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-17T10:42:33.334905+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:42:35.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.653+0000 7f4508021640 1 -- 172.21.15.83:0/1080680559 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f44ec066f60 msgr2=0x7f44ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:35.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.653+0000 7f4508021640 1 --2- 172.21.15.83:0/1080680559 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f44ec066f60 0x7f44ec069420 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f44f00ad000 tx=0x7f44f0009040 comp rx=0 tx=0).stop 2024-04-17T10:42:35.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.653+0000 7f4508021640 1 -- 172.21.15.83:0/1080680559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44e8004810 msgr2=0x7f450015d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:35.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.653+0000 7f4508021640 1 --2- 172.21.15.83:0/1080680559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44e8004810 0x7f450015d370 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f44f8010b10 tx=0x7f44f80057d0 comp rx=0 tx=0).stop 2024-04-17T10:42:35.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.653+0000 7f4508021640 1 -- 172.21.15.83:0/1080680559 shutdown_connections 2024-04-17T10:42:35.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.653+0000 7f4508021640 1 --2- 172.21.15.83:0/1080680559 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f44ec066f60 0x7f44ec069420 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:35.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.653+0000 7f4508021640 1 --2- 172.21.15.83:0/1080680559 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f45000712a0 0x7f450015fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:35.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.653+0000 7f4508021640 1 --2- 172.21.15.83:0/1080680559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45001e8000 0x7f4500070d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:35.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.653+0000 7f4508021640 1 --2- 172.21.15.83:0/1080680559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44e8004810 0x7f450015d370 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:35.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.653+0000 7f4508021640 1 -- 172.21.15.83:0/1080680559 >> 172.21.15.83:0/1080680559 conn(0x7f450006abe0 msgr2=0x7f4500071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:35.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.653+0000 7f4508021640 1 -- 172.21.15.83:0/1080680559 shutdown_connections 2024-04-17T10:42:35.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.653+0000 7f4508021640 1 -- 172.21.15.83:0/1080680559 wait complete. 2024-04-17T10:42:35.670 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-17T10:42:35.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.789+0000 7f1ec1815640 1 Processor -- start 2024-04-17T10:42:35.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.789+0000 7f1ec1815640 1 -- start start 2024-04-17T10:42:35.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.793+0000 7f1ec1815640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1ebc0d0970 0x7f1ebc0d17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:35.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.793+0000 7f1ec1815640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1ebc0d26c0 0x7f1ebc15d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:35.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.793+0000 7f1ec1815640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ebc0ce940 con 0x7f1ebc0ce4e0 2024-04-17T10:42:35.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.793+0000 7f1ec1815640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ebc0cf7d0 con 0x7f1ebc0d26c0 2024-04-17T10:42:35.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.793+0000 7f1ec1815640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ebc0d4890 con 0x7f1ebc0d0970 2024-04-17T10:42:35.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.793+0000 7f1ebb7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1ebc0d26c0 0x7f1ebc15d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:35.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.793+0000 7f1ebb7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1ebc0d26c0 0x7f1ebc15d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55458/0 (socket says 172.21.15.83:55458) 2024-04-17T10:42:35.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.793+0000 7f1ebb7fe640 1 -- 172.21.15.83:0/3410818397 learned_addr learned my addr 172.21.15.83:0/3410818397 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:35.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.793+0000 7f1ebaffd640 1 --2- 172.21.15.83:0/3410818397 >> v2:172.21.15.167:3300/0 conn(0x7f1ebc0d0970 0x7f1ebc0d17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:35.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.793+0000 7f1ebb7fe640 1 -- 172.21.15.83:0/3410818397 >> v2:172.21.15.167:3300/0 conn(0x7f1ebc0d0970 msgr2=0x7f1ebc0d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:35.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.793+0000 7f1ebb7fe640 1 --2- 172.21.15.83:0/3410818397 >> v2:172.21.15.167:3300/0 conn(0x7f1ebc0d0970 0x7f1ebc0d17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:35.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.793+0000 7f1ebb7fe640 1 -- 172.21.15.83:0/3410818397 >> v1:172.21.15.167:6789/0 conn(0x7f1ebc0ce4e0 legacy=0x7f1ebc0d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:35.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.793+0000 7f1ebb7fe640 1 -- 172.21.15.83:0/3410818397 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ebc0cebc0 con 0x7f1ebc0d26c0 2024-04-17T10:42:35.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.793+0000 7f1ebb7fe640 1 --2- 172.21.15.83:0/3410818397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ebc0d26c0 0x7f1ebc15d620 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc06af50 tx=0x7f1eb401c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290ef9cd0828c9ac server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:35.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.793+0000 7f1eba7fc640 1 -- 172.21.15.83:0/3410818397 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb40a5660 con 0x7f1ebc0d26c0 2024-04-17T10:42:35.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.793+0000 7f1eba7fc640 1 -- 172.21.15.83:0/3410818397 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb40a5800 con 0x7f1ebc0d26c0 2024-04-17T10:42:35.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.793+0000 7f1eba7fc640 1 -- 172.21.15.83:0/3410818397 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb40a5bd0 con 0x7f1ebc0d26c0 2024-04-17T10:42:35.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.797+0000 7f1ec1815640 1 -- 172.21.15.83:0/3410818397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ebc0d26c0 msgr2=0x7f1ebc15d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:35.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.797+0000 7f1ec1815640 1 --2- 172.21.15.83:0/3410818397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ebc0d26c0 0x7f1ebc15d620 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc06af50 tx=0x7f1eb401c7a0 comp rx=0 tx=0).stop 2024-04-17T10:42:35.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.797+0000 7f1ec1815640 1 -- 172.21.15.83:0/3410818397 shutdown_connections 2024-04-17T10:42:35.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.797+0000 7f1ec1815640 1 --2- 172.21.15.83:0/3410818397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ebc0d26c0 0x7f1ebc15d620 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:35.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.797+0000 7f1ec1815640 1 --2- 172.21.15.83:0/3410818397 >> v2:172.21.15.167:3300/0 conn(0x7f1ebc0d0970 0x7f1ebc0d17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:35.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.797+0000 7f1ec1815640 1 -- 172.21.15.83:0/3410818397 >> 172.21.15.83:0/3410818397 conn(0x7f1ebc09cd50 msgr2=0x7f1ebc0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:35.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.797+0000 7f1ec1815640 1 -- 172.21.15.83:0/3410818397 shutdown_connections 2024-04-17T10:42:35.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.797+0000 7f1ec1815640 1 -- 172.21.15.83:0/3410818397 wait complete. 2024-04-17T10:42:35.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.797+0000 7f1ec1815640 1 Processor -- start 2024-04-17T10:42:35.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.797+0000 7f1ec1815640 1 -- start start 2024-04-17T10:42:35.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.797+0000 7f1ec1815640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ebc0ce4e0 0x7f1ebc0b65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:35.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.797+0000 7f1ec1815640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ebc0d0970 0x7f1ebc14fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:35.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.797+0000 7f1ec1815640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ebc0d26c0 0x7f1ebc0cce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:35.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.797+0000 7f1ec1815640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc06b280 con 0x7f1ebc0d26c0 2024-04-17T10:42:35.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.797+0000 7f1ec1815640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc0d0d50 con 0x7f1ebc0ce4e0 2024-04-17T10:42:35.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.797+0000 7f1ec1815640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc06b400 con 0x7f1ebc0d0970 2024-04-17T10:42:35.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.797+0000 7f1ebb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ebc0d26c0 0x7f1ebc0cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:35.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.797+0000 7f1ebb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ebc0d26c0 0x7f1ebc0cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55462/0 (socket says 172.21.15.83:55462) 2024-04-17T10:42:35.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.797+0000 7f1ebb7fe640 1 -- 172.21.15.83:0/2797572292 learned_addr learned my addr 172.21.15.83:0/2797572292 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:35.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.797+0000 7f1eb27fc640 1 --2- 172.21.15.83:0/2797572292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ebc0d0970 0x7f1ebc14fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:35.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.797+0000 7f1ebb7fe640 1 -- 172.21.15.83:0/2797572292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ebc0d0970 msgr2=0x7f1ebc14fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:35.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.797+0000 7f1ebb7fe640 1 --2- 172.21.15.83:0/2797572292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ebc0d0970 0x7f1ebc14fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:35.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.797+0000 7f1ebb7fe640 1 -- 172.21.15.83:0/2797572292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ebc0ce4e0 msgr2=0x7f1ebc0b65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:35.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.797+0000 7f1ebb7fe640 1 --2- 172.21.15.83:0/2797572292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ebc0ce4e0 0x7f1ebc0b65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:35.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.797+0000 7f1ebb7fe640 1 -- 172.21.15.83:0/2797572292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ebc0cb5e0 con 0x7f1ebc0d26c0 2024-04-17T10:42:35.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.797+0000 7f1eb27fc640 1 --2- 172.21.15.83:0/2797572292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ebc0d0970 0x7f1ebc14fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:35.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.797+0000 7f1ebb7fe640 1 --2- 172.21.15.83:0/2797572292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ebc0d26c0 0x7f1ebc0cce30 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f1eb40af040 tx=0x7f1eb4019540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:35.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.797+0000 7f1eb8ff9640 1 -- 172.21.15.83:0/2797572292 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb40bc020 con 0x7f1ebc0d26c0 2024-04-17T10:42:35.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.797+0000 7f1eb8ff9640 1 -- 172.21.15.83:0/2797572292 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb40b8670 con 0x7f1ebc0d26c0 2024-04-17T10:42:35.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.797+0000 7f1ec1815640 1 -- 172.21.15.83:0/2797572292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ebc0cb8c0 con 0x7f1ebc0d26c0 2024-04-17T10:42:35.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.797+0000 7f1eb8ff9640 1 -- 172.21.15.83:0/2797572292 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4009510 con 0x7f1ebc0d26c0 2024-04-17T10:42:35.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.801+0000 7f1ec1815640 1 -- 172.21.15.83:0/2797572292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ebc0cbe80 con 0x7f1ebc0d26c0 2024-04-17T10:42:35.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.801+0000 7f1ec1815640 1 -- 172.21.15.83:0/2797572292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ebc0681a0 con 0x7f1ebc0d26c0 2024-04-17T10:42:35.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.801+0000 7f1eb8ff9640 1 -- 172.21.15.83:0/2797572292 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1eb40bf020 con 0x7f1ebc0d26c0 2024-04-17T10:42:35.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.801+0000 7f1eb8ff9640 1 --2- 172.21.15.83:0/2797572292 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1e9c066f90 0x7f1e9c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:35.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.801+0000 7f1ebaffd640 1 --2- 172.21.15.83:0/2797572292 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1e9c066f90 0x7f1e9c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:35.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.801+0000 7f1eb8ff9640 1 -- 172.21.15.83:0/2797572292 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1eb411f990 con 0x7f1ebc0d26c0 2024-04-17T10:42:35.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.801+0000 7f1ebaffd640 1 --2- 172.21.15.83:0/2797572292 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1e9c066f90 0x7f1e9c069450 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1eac0098f0 tx=0x7f1eac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:35.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:35.805+0000 7f1eb8ff9640 1 -- 172.21.15.83:0/2797572292 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1ebc0ce940 con 0x7f1ebc0d26c0 2024-04-17T10:42:36.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:36.001+0000 7f1ec1815640 1 -- 172.21.15.83:0/2797572292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1ebc074f90 con 0x7f1ebc0d26c0 2024-04-17T10:42:36.950 INFO:tasks.ceph.mds.b.smithi167.stderr:2024-04-17T10:42:36.954+0000 7fc719f61640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T10:42:36.967 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:42:36.969 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:42:37.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.005+0000 7f1eb8ff9640 1 -- 172.21.15.83:0/2797572292 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v31) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1eb40f0a30 con 0x7f1ebc0d26c0 2024-04-17T10:42:37.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.009+0000 7f1ec1815640 1 -- 172.21.15.83:0/2797572292 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1e9c066f90 msgr2=0x7f1e9c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:37.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.009+0000 7f1ec1815640 1 --2- 172.21.15.83:0/2797572292 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1e9c066f90 0x7f1e9c069450 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1eac0098f0 tx=0x7f1eac008040 comp rx=0 tx=0).stop 2024-04-17T10:42:37.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.009+0000 7f1ec1815640 1 -- 172.21.15.83:0/2797572292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ebc0d26c0 msgr2=0x7f1ebc0cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:37.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.009+0000 7f1ec1815640 1 --2- 172.21.15.83:0/2797572292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ebc0d26c0 0x7f1ebc0cce30 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f1eb40af040 tx=0x7f1eb4019540 comp rx=0 tx=0).stop 2024-04-17T10:42:37.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.009+0000 7f1ec1815640 1 -- 172.21.15.83:0/2797572292 shutdown_connections 2024-04-17T10:42:37.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.009+0000 7f1ec1815640 1 --2- 172.21.15.83:0/2797572292 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1e9c066f90 0x7f1e9c069450 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:37.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.009+0000 7f1ec1815640 1 --2- 172.21.15.83:0/2797572292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ebc0d26c0 0x7f1ebc0cce30 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:37.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.009+0000 7f1ec1815640 1 --2- 172.21.15.83:0/2797572292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ebc0d0970 0x7f1ebc14fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:37.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.009+0000 7f1ec1815640 1 --2- 172.21.15.83:0/2797572292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ebc0ce4e0 0x7f1ebc0b65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:37.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.009+0000 7f1ec1815640 1 -- 172.21.15.83:0/2797572292 >> 172.21.15.83:0/2797572292 conn(0x7f1ebc09cd50 msgr2=0x7f1ebc0b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:37.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.009+0000 7f1ec1815640 1 -- 172.21.15.83:0/2797572292 shutdown_connections 2024-04-17T10:42:37.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.009+0000 7f1ec1815640 1 -- 172.21.15.83:0/2797572292 wait complete. 2024-04-17T10:42:37.028 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-17T10:42:37.028 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-17T10:42:37.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f52098c9640 1 Processor -- start 2024-04-17T10:42:37.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f52098c9640 1 -- start start 2024-04-17T10:42:37.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f52098c9640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f520406ae10 0x7f520406b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:37.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f52098c9640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5204070aa0 0x7f5204070f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:37.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f52098c9640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52041625d0 con 0x7f520415e330 2024-04-17T10:42:37.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f52098c9640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f520415ee80 con 0x7f5204070aa0 2024-04-17T10:42:37.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f52098c9640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f520406b730 con 0x7f520406ae10 2024-04-17T10:42:37.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f52027fc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f520415e330 0x7f520415e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57416/0 (socket says 172.21.15.83:57416) 2024-04-17T10:42:37.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f52027fc640 1 -- 172.21.15.83:0/4106569427 learned_addr learned my addr 172.21.15.83:0/4106569427 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:37.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f5202ffd640 1 --2- 172.21.15.83:0/4106569427 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f520406ae10 0x7f520406b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:37.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f52037fe640 1 --2- 172.21.15.83:0/4106569427 >> v2:172.21.15.167:3300/0 conn(0x7f5204070aa0 0x7f5204070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:37.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f5201ffb640 1 -- 172.21.15.83:0/4106569427 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181996693 0 0) 0x7f52041625d0 con 0x7f520415e330 2024-04-17T10:42:37.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f5201ffb640 1 -- 172.21.15.83:0/4106569427 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51f0003530 con 0x7f520415e330 2024-04-17T10:42:37.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f5201ffb640 1 -- 172.21.15.83:0/4106569427 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3761128533 0 0) 0x7f51f0003530 con 0x7f520415e330 2024-04-17T10:42:37.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f5201ffb640 1 -- 172.21.15.83:0/4106569427 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f520406ae10 msgr2=0x7f520406b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:37.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f5201ffb640 1 --2- 172.21.15.83:0/4106569427 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f520406ae10 0x7f520406b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:37.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f5201ffb640 1 -- 172.21.15.83:0/4106569427 >> v2:172.21.15.167:3300/0 conn(0x7f5204070aa0 msgr2=0x7f5204070f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:37.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f5201ffb640 1 --2- 172.21.15.83:0/4106569427 >> v2:172.21.15.167:3300/0 conn(0x7f5204070aa0 0x7f5204070f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:37.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f5201ffb640 1 -- 172.21.15.83:0/4106569427 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5204161810 con 0x7f520415e330 2024-04-17T10:42:37.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f5201ffb640 1 -- 172.21.15.83:0/4106569427 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f51ec001680 con 0x7f520415e330 2024-04-17T10:42:37.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f52037fe640 1 --2- 172.21.15.83:0/4106569427 >> v2:172.21.15.167:3300/0 conn(0x7f5204070aa0 0x7f5204070f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:42:37.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f5201ffb640 1 -- 172.21.15.83:0/4106569427 >> v1:172.21.15.83:6789/0 conn(0x7f520415e330 legacy=0x7f520415e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:37.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f5201ffb640 1 --2- 172.21.15.83:0/4106569427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51f0003a30 0x7f51f0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:37.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f5201ffb640 1 --2- 172.21.15.83:0/4106569427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51f0004810 0x7f51f0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:37.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f5201ffb640 1 --2- 172.21.15.83:0/4106569427 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f520406ae10 0x7f51f00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:37.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f5201ffb640 1 -- 172.21.15.83:0/4106569427 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520415ee80 con 0x7f51f0004810 2024-04-17T10:42:37.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f5201ffb640 1 -- 172.21.15.83:0/4106569427 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520406b730 con 0x7f51f0003a30 2024-04-17T10:42:37.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f5201ffb640 1 -- 172.21.15.83:0/4106569427 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f000bb00 con 0x7f520406ae10 2024-04-17T10:42:37.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f52027fc640 1 --2- 172.21.15.83:0/4106569427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51f0004810 0x7f51f0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:37.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f5202ffd640 1 --2- 172.21.15.83:0/4106569427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51f0003a30 0x7f51f0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:37.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f52037fe640 1 --2- 172.21.15.83:0/4106569427 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f520406ae10 0x7f51f00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:37.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f52027fc640 1 -- 172.21.15.83:0/4106569427 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f520406ae10 msgr2=0x7f51f00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:37.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f52027fc640 1 --2- 172.21.15.83:0/4106569427 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f520406ae10 0x7f51f00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:37.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f52027fc640 1 -- 172.21.15.83:0/4106569427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51f0003a30 msgr2=0x7f51f0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:37.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f52027fc640 1 --2- 172.21.15.83:0/4106569427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51f0003a30 0x7f51f0003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:37.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.241+0000 7f52027fc640 1 -- 172.21.15.83:0/4106569427 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f0003530 con 0x7f51f0004810 2024-04-17T10:42:37.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.245+0000 7f52027fc640 1 --2- 172.21.15.83:0/4106569427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51f0004810 0x7f51f0004c90 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f51ec001ec0 tx=0x7f51ec01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6a322f786288b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:37.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.245+0000 7f5201ffb640 1 -- 172.21.15.83:0/4106569427 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec007d80 con 0x7f51f0004810 2024-04-17T10:42:37.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.245+0000 7f5201ffb640 1 -- 172.21.15.83:0/4106569427 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ec0a9020 con 0x7f51f0004810 2024-04-17T10:42:37.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.245+0000 7f5201ffb640 1 -- 172.21.15.83:0/4106569427 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec005b00 con 0x7f51f0004810 2024-04-17T10:42:37.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.245+0000 7f52098c9640 1 -- 172.21.15.83:0/4106569427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51f0004810 msgr2=0x7f51f0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:37.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.245+0000 7f52098c9640 1 --2- 172.21.15.83:0/4106569427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51f0004810 0x7f51f0004c90 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f51ec001ec0 tx=0x7f51ec01cc60 comp rx=0 tx=0).stop 2024-04-17T10:42:37.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.245+0000 7f52037fe640 1 -- 172.21.15.83:0/4106569427 reap_dead start 2024-04-17T10:42:37.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.245+0000 7f52098c9640 1 -- 172.21.15.83:0/4106569427 shutdown_connections 2024-04-17T10:42:37.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.245+0000 7f52098c9640 1 -- 172.21.15.83:0/4106569427 >> 172.21.15.83:0/4106569427 conn(0x7f52040b15b0 msgr2=0x7f52040b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:37.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.245+0000 7f52098c9640 1 -- 172.21.15.83:0/4106569427 shutdown_connections 2024-04-17T10:42:37.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.245+0000 7f52098c9640 1 -- 172.21.15.83:0/4106569427 wait complete. 2024-04-17T10:42:37.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.245+0000 7f52098c9640 1 Processor -- start 2024-04-17T10:42:37.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.245+0000 7f52098c9640 1 -- start start 2024-04-17T10:42:37.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.245+0000 7f52098c9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51f0004810 0x7f52041649e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:37.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.245+0000 7f52098c9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f520415aac0 0x7f52040af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:37.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.245+0000 7f52098c9640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5204146ca0 0x7f5204147120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:37.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.245+0000 7f52098c9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52040cf470 con 0x7f520415aac0 2024-04-17T10:42:37.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.245+0000 7f52098c9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52040cf2f0 con 0x7f51f0004810 2024-04-17T10:42:37.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.245+0000 7f52098c9640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5204168a00 con 0x7f5204146ca0 2024-04-17T10:42:37.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.249+0000 7f52027fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f520415aac0 0x7f52040af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:37.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.249+0000 7f52027fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f520415aac0 0x7f52040af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36694/0 (socket says 172.21.15.83:36694) 2024-04-17T10:42:37.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.249+0000 7f52027fc640 1 -- 172.21.15.83:0/2109229865 learned_addr learned my addr 172.21.15.83:0/2109229865 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:37.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.249+0000 7f5202ffd640 1 --2- 172.21.15.83:0/2109229865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51f0004810 0x7f52041649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:37.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.249+0000 7f52037fe640 1 --2- 172.21.15.83:0/2109229865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5204146ca0 0x7f5204147120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:37.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.249+0000 7f52027fc640 1 -- 172.21.15.83:0/2109229865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5204146ca0 msgr2=0x7f5204147120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:37.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.249+0000 7f52027fc640 1 --2- 172.21.15.83:0/2109229865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5204146ca0 0x7f5204147120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:37.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.249+0000 7f52027fc640 1 -- 172.21.15.83:0/2109229865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51f0004810 msgr2=0x7f52041649e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:37.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.249+0000 7f52027fc640 1 --2- 172.21.15.83:0/2109229865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51f0004810 0x7f52041649e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:37.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.249+0000 7f52027fc640 1 -- 172.21.15.83:0/2109229865 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f520415af40 con 0x7f520415aac0 2024-04-17T10:42:37.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.249+0000 7f52027fc640 1 --2- 172.21.15.83:0/2109229865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f520415aac0 0x7f52040af790 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f51ec0ab040 tx=0x7f51f0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:37.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.249+0000 7f51fbfff640 1 -- 172.21.15.83:0/2109229865 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec0049f0 con 0x7f520415aac0 2024-04-17T10:42:37.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.249+0000 7f51fbfff640 1 -- 172.21.15.83:0/2109229865 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ec004b90 con 0x7f520415aac0 2024-04-17T10:42:37.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.249+0000 7f52098c9640 1 -- 172.21.15.83:0/2109229865 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5204165080 con 0x7f520415aac0 2024-04-17T10:42:37.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.249+0000 7f51fbfff640 1 -- 172.21.15.83:0/2109229865 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec0070d0 con 0x7f520415aac0 2024-04-17T10:42:37.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.249+0000 7f52098c9640 1 -- 172.21.15.83:0/2109229865 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5204165640 con 0x7f520415aac0 2024-04-17T10:42:37.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.249+0000 7f52098c9640 1 -- 172.21.15.83:0/2109229865 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5204074f90 con 0x7f520415aac0 2024-04-17T10:42:37.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.249+0000 7f51fbfff640 1 -- 172.21.15.83:0/2109229865 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f51ec007270 con 0x7f520415aac0 2024-04-17T10:42:37.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.249+0000 7f51fbfff640 1 --2- 172.21.15.83:0/2109229865 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f51e4066f60 0x7f51e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:37.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.249+0000 7f5202ffd640 1 --2- 172.21.15.83:0/2109229865 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f51e4066f60 0x7f51e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:37.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.249+0000 7f51fbfff640 1 -- 172.21.15.83:0/2109229865 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51ec11e070 con 0x7f520415aac0 2024-04-17T10:42:37.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.249+0000 7f5202ffd640 1 --2- 172.21.15.83:0/2109229865 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f51e4066f60 0x7f51e4069420 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f51f4007440 tx=0x7f51f400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:37.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.253+0000 7f51fbfff640 1 -- 172.21.15.83:0/2109229865 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f51ec0b4070 con 0x7f520415aac0 2024-04-17T10:42:37.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:37.449+0000 7f52098c9640 1 -- 172.21.15.83:0/2109229865 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5204072740 con 0x7f520415aac0 2024-04-17T10:42:38.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.009+0000 7f51fbfff640 1 -- 172.21.15.83:0/2109229865 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7f51ec016970 con 0x7f520415aac0 2024-04-17T10:42:38.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.009+0000 7f52098c9640 1 -- 172.21.15.83:0/2109229865 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f51e4066f60 msgr2=0x7f51e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:38.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.009+0000 7f52098c9640 1 --2- 172.21.15.83:0/2109229865 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f51e4066f60 0x7f51e4069420 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f51f4007440 tx=0x7f51f400b040 comp rx=0 tx=0).stop 2024-04-17T10:42:38.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.009+0000 7f52098c9640 1 -- 172.21.15.83:0/2109229865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f520415aac0 msgr2=0x7f52040af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:38.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.009+0000 7f52098c9640 1 --2- 172.21.15.83:0/2109229865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f520415aac0 0x7f52040af790 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f51ec0ab040 tx=0x7f51f0005840 comp rx=0 tx=0).stop 2024-04-17T10:42:38.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.009+0000 7f52098c9640 1 -- 172.21.15.83:0/2109229865 shutdown_connections 2024-04-17T10:42:38.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.009+0000 7f52098c9640 1 --2- 172.21.15.83:0/2109229865 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f51e4066f60 0x7f51e4069420 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:38.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.009+0000 7f52098c9640 1 --2- 172.21.15.83:0/2109229865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5204146ca0 0x7f5204147120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:38.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.009+0000 7f52098c9640 1 --2- 172.21.15.83:0/2109229865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f520415aac0 0x7f52040af790 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:38.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.009+0000 7f52098c9640 1 --2- 172.21.15.83:0/2109229865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51f0004810 0x7f52041649e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:38.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.009+0000 7f52098c9640 1 -- 172.21.15.83:0/2109229865 >> 172.21.15.83:0/2109229865 conn(0x7f52040b15b0 msgr2=0x7f52040ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:38.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.009+0000 7f52098c9640 1 -- 172.21.15.83:0/2109229865 shutdown_connections 2024-04-17T10:42:38.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.009+0000 7f52098c9640 1 -- 172.21.15.83:0/2109229865 wait complete. 2024-04-17T10:42:38.030 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-17T10:42:38.031 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:42:38.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.153+0000 7fe71449c640 1 Processor -- start 2024-04-17T10:42:38.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.153+0000 7fe71449c640 1 -- start start 2024-04-17T10:42:38.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.153+0000 7fe71449c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe70c072330 0x7fe70c0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:38.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.153+0000 7fe71449c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe70c0cfa10 0x7fe70c06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:38.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.153+0000 7fe71449c640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe70c0cfea0 con 0x7fe70c15ba30 2024-04-17T10:42:38.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.153+0000 7fe71449c640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe70c071ea0 con 0x7fe70c0cfa10 2024-04-17T10:42:38.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.153+0000 7fe71449c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c06e320 con 0x7fe70c072330 2024-04-17T10:42:38.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.153+0000 7fe712a39640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe70c0cfa10 0x7fe70c06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:38.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.153+0000 7fe712a39640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe70c0cfa10 0x7fe70c06c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36710/0 (socket says 172.21.15.83:36710) 2024-04-17T10:42:38.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.153+0000 7fe712a39640 1 -- 172.21.15.83:0/3212836994 learned_addr learned my addr 172.21.15.83:0/3212836994 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:38.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.153+0000 7fe711a37640 1 --2- 172.21.15.83:0/3212836994 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe70c072330 0x7fe70c0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:38.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.153+0000 7fe712a39640 1 -- 172.21.15.83:0/3212836994 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe70c072330 msgr2=0x7fe70c0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:38.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.153+0000 7fe712a39640 1 --2- 172.21.15.83:0/3212836994 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe70c072330 0x7fe70c0cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:38.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.153+0000 7fe712a39640 1 -- 172.21.15.83:0/3212836994 >> v1:172.21.15.167:6789/0 conn(0x7fe70c15ba30 legacy=0x7fe70c15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:38.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.153+0000 7fe712a39640 1 -- 172.21.15.83:0/3212836994 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70c06e040 con 0x7fe70c0cfa10 2024-04-17T10:42:38.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.153+0000 7fe711a37640 1 --2- 172.21.15.83:0/3212836994 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe70c072330 0x7fe70c0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:38.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.157+0000 7fe712a39640 1 --2- 172.21.15.83:0/3212836994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe70c0cfa10 0x7fe70c06c490 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fe704009e50 tx=0x7fe70401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63fe57600e711f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:38.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.157+0000 7fe711236640 1 -- 172.21.15.83:0/3212836994 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7040a5770 con 0x7fe70c0cfa10 2024-04-17T10:42:38.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.157+0000 7fe711236640 1 -- 172.21.15.83:0/3212836994 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7040a5910 con 0x7fe70c0cfa10 2024-04-17T10:42:38.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.157+0000 7fe711236640 1 -- 172.21.15.83:0/3212836994 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7040a5ce0 con 0x7fe70c0cfa10 2024-04-17T10:42:38.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.157+0000 7fe71449c640 1 -- 172.21.15.83:0/3212836994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe70c0cfa10 msgr2=0x7fe70c06c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:38.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.157+0000 7fe71449c640 1 --2- 172.21.15.83:0/3212836994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe70c0cfa10 0x7fe70c06c490 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fe704009e50 tx=0x7fe70401c9a0 comp rx=0 tx=0).stop 2024-04-17T10:42:38.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.157+0000 7fe71449c640 1 -- 172.21.15.83:0/3212836994 shutdown_connections 2024-04-17T10:42:38.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.157+0000 7fe71449c640 1 --2- 172.21.15.83:0/3212836994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe70c0cfa10 0x7fe70c06c490 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:38.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.157+0000 7fe71449c640 1 --2- 172.21.15.83:0/3212836994 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe70c072330 0x7fe70c0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:38.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.157+0000 7fe71449c640 1 -- 172.21.15.83:0/3212836994 >> 172.21.15.83:0/3212836994 conn(0x7fe70c06abe0 msgr2=0x7fe70c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:38.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.157+0000 7fe71449c640 1 -- 172.21.15.83:0/3212836994 shutdown_connections 2024-04-17T10:42:38.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.157+0000 7fe71449c640 1 -- 172.21.15.83:0/3212836994 wait complete. 2024-04-17T10:42:38.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.157+0000 7fe71449c640 1 Processor -- start 2024-04-17T10:42:38.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.157+0000 7fe71449c640 1 -- start start 2024-04-17T10:42:38.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.157+0000 7fe71449c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe70c072330 0x7fe70c14ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:38.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.157+0000 7fe71449c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe70c0cfa10 0x7fe70c145b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:38.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.157+0000 7fe71449c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe70c15ba30 0x7fe70c14c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:38.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.157+0000 7fe71449c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c06de20 con 0x7fe70c072330 2024-04-17T10:42:38.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.157+0000 7fe71449c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c06dca0 con 0x7fe70c0cfa10 2024-04-17T10:42:38.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.157+0000 7fe71449c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c168520 con 0x7fe70c15ba30 2024-04-17T10:42:38.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.157+0000 7fe712238640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe70c072330 0x7fe70c14ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:38.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.161+0000 7fe712238640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe70c072330 0x7fe70c14ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36716/0 (socket says 172.21.15.83:36716) 2024-04-17T10:42:38.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.161+0000 7fe712238640 1 -- 172.21.15.83:0/2175660689 learned_addr learned my addr 172.21.15.83:0/2175660689 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:38.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.161+0000 7fe711a37640 1 --2- 172.21.15.83:0/2175660689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe70c0cfa10 0x7fe70c145b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:38.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.161+0000 7fe712a39640 1 --2- 172.21.15.83:0/2175660689 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe70c15ba30 0x7fe70c14c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:38.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.161+0000 7fe712238640 1 -- 172.21.15.83:0/2175660689 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe70c15ba30 msgr2=0x7fe70c14c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:38.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.161+0000 7fe712238640 1 --2- 172.21.15.83:0/2175660689 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe70c15ba30 0x7fe70c14c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:38.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.161+0000 7fe712238640 1 -- 172.21.15.83:0/2175660689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe70c0cfa10 msgr2=0x7fe70c145b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:38.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.161+0000 7fe712238640 1 --2- 172.21.15.83:0/2175660689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe70c0cfa10 0x7fe70c145b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:38.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.161+0000 7fe712238640 1 -- 172.21.15.83:0/2175660689 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70c14dbd0 con 0x7fe70c072330 2024-04-17T10:42:38.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.161+0000 7fe712238640 1 --2- 172.21.15.83:0/2175660689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe70c072330 0x7fe70c14ad30 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fe70800c500 tx=0x7fe70800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:38.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.161+0000 7fe702ffd640 1 -- 172.21.15.83:0/2175660689 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708016020 con 0x7fe70c072330 2024-04-17T10:42:38.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.161+0000 7fe702ffd640 1 -- 172.21.15.83:0/2175660689 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe708005220 con 0x7fe70c072330 2024-04-17T10:42:38.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.161+0000 7fe702ffd640 1 -- 172.21.15.83:0/2175660689 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708007030 con 0x7fe70c072330 2024-04-17T10:42:38.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.161+0000 7fe71449c640 1 -- 172.21.15.83:0/2175660689 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe70c14deb0 con 0x7fe70c072330 2024-04-17T10:42:38.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.161+0000 7fe71449c640 1 -- 172.21.15.83:0/2175660689 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe70c14e470 con 0x7fe70c072330 2024-04-17T10:42:38.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.161+0000 7fe71449c640 1 -- 172.21.15.83:0/2175660689 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe70c074f90 con 0x7fe70c072330 2024-04-17T10:42:38.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.161+0000 7fe702ffd640 1 -- 172.21.15.83:0/2175660689 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe7080071f0 con 0x7fe70c072330 2024-04-17T10:42:38.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.161+0000 7fe702ffd640 1 --2- 172.21.15.83:0/2175660689 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe6f0066f60 0x7fe6f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:38.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.161+0000 7fe702ffd640 1 -- 172.21.15.83:0/2175660689 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe708084a40 con 0x7fe70c072330 2024-04-17T10:42:38.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.161+0000 7fe711a37640 1 --2- 172.21.15.83:0/2175660689 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe6f0066f60 0x7fe6f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:38.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.161+0000 7fe711a37640 1 --2- 172.21.15.83:0/2175660689 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe6f0066f60 0x7fe6f0069420 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc006fd0 tx=0x7fe6fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:38.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.165+0000 7fe702ffd640 1 -- 172.21.15.83:0/2175660689 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe708020020 con 0x7fe70c072330 2024-04-17T10:42:38.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.357+0000 7fe71449c640 1 -- 172.21.15.83:0/2175660689 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe70c14e890 con 0x7fe70c072330 2024-04-17T10:42:38.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.361+0000 7fe702ffd640 1 -- 172.21.15.83:0/2175660689 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4938 (secure 0 0 0) 0x7fe708055a60 con 0x7fe70c072330 2024-04-17T10:42:38.360 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:38.360 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5398,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1991041077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1991041077},{"type":"v1","addr":"172.21.15.167:6835","nonce":1991041077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6522,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/320723747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":320723747},{"type":"v1","addr":"172.21.15.83:6836","nonce":320723747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6525,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/4066270849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":4066270849},{"type":"v1","addr":"172.21.15.83:6837","nonce":4066270849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:42:34.328397+0000","modified":"2024-04-17T10:42:37.455022+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6534},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6534":{"gid":6534,"name":"b","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6833/271994303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":271994303},{"type":"v1","addr":"172.21.15.167:6833","nonce":271994303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-17T10:42:38.360 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 32 2024-04-17T10:42:38.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.361+0000 7fe71449c640 1 -- 172.21.15.83:0/2175660689 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe6f0066f60 msgr2=0x7fe6f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:38.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.361+0000 7fe71449c640 1 --2- 172.21.15.83:0/2175660689 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe6f0066f60 0x7fe6f0069420 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc006fd0 tx=0x7fe6fc008040 comp rx=0 tx=0).stop 2024-04-17T10:42:38.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.361+0000 7fe71449c640 1 -- 172.21.15.83:0/2175660689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe70c072330 msgr2=0x7fe70c14ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:38.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.361+0000 7fe71449c640 1 --2- 172.21.15.83:0/2175660689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe70c072330 0x7fe70c14ad30 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fe70800c500 tx=0x7fe70800c9c0 comp rx=0 tx=0).stop 2024-04-17T10:42:38.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.361+0000 7fe71449c640 1 -- 172.21.15.83:0/2175660689 shutdown_connections 2024-04-17T10:42:38.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.361+0000 7fe71449c640 1 --2- 172.21.15.83:0/2175660689 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe6f0066f60 0x7fe6f0069420 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:38.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.361+0000 7fe71449c640 1 --2- 172.21.15.83:0/2175660689 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe70c15ba30 0x7fe70c14c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:38.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.361+0000 7fe71449c640 1 --2- 172.21.15.83:0/2175660689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe70c0cfa10 0x7fe70c145b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:38.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.361+0000 7fe71449c640 1 --2- 172.21.15.83:0/2175660689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe70c072330 0x7fe70c14ad30 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:38.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.361+0000 7fe71449c640 1 -- 172.21.15.83:0/2175660689 >> 172.21.15.83:0/2175660689 conn(0x7fe70c06abe0 msgr2=0x7fe70c0aed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:38.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.361+0000 7fe71449c640 1 -- 172.21.15.83:0/2175660689 shutdown_connections 2024-04-17T10:42:38.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.361+0000 7fe71449c640 1 -- 172.21.15.83:0/2175660689 wait complete. 2024-04-17T10:42:38.380 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:42:38.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.493+0000 7f3bf4caf640 1 Processor -- start 2024-04-17T10:42:38.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.497+0000 7f3bf4caf640 1 -- start start 2024-04-17T10:42:38.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.497+0000 7f3bf4caf640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3bf01540d0 0x7f3bf0070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:38.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.497+0000 7f3bf4caf640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bf015b270 0x7f3bf015d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:38.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.497+0000 7f3bf4caf640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf006ce70 con 0x7f3bf015dbe0 2024-04-17T10:42:38.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.497+0000 7f3bf4caf640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bf015e050 con 0x7f3bf01540d0 2024-04-17T10:42:38.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.497+0000 7f3bf4caf640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf0072730 con 0x7f3bf015b270 2024-04-17T10:42:38.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.497+0000 7f3bee59c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3bf01540d0 0x7f3bf0070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:38.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.497+0000 7f3bee59c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3bf01540d0 0x7f3bf0070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36728/0 (socket says 172.21.15.83:36728) 2024-04-17T10:42:38.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.497+0000 7f3bee59c640 1 -- 172.21.15.83:0/1162330302 learned_addr learned my addr 172.21.15.83:0/1162330302 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:38.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.497+0000 7f3bedd9b640 1 --2- 172.21.15.83:0/1162330302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bf015b270 0x7f3bf015d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:38.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.497+0000 7f3bee59c640 1 -- 172.21.15.83:0/1162330302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bf015b270 msgr2=0x7f3bf015d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:38.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.497+0000 7f3bee59c640 1 --2- 172.21.15.83:0/1162330302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bf015b270 0x7f3bf015d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:38.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.497+0000 7f3bee59c640 1 -- 172.21.15.83:0/1162330302 >> v1:172.21.15.167:6789/0 conn(0x7f3bf015dbe0 legacy=0x7f3bf015a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:38.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.497+0000 7f3bee59c640 1 -- 172.21.15.83:0/1162330302 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf006d140 con 0x7f3bf01540d0 2024-04-17T10:42:38.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.497+0000 7f3bee59c640 1 --2- 172.21.15.83:0/1162330302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf01540d0 0x7f3bf0070470 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009870 tx=0x7f3be001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d14473d2873390 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:38.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.497+0000 7f3bed59a640 1 -- 172.21.15.83:0/1162330302 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be00aa610 con 0x7f3bf01540d0 2024-04-17T10:42:38.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.501+0000 7f3bed59a640 1 -- 172.21.15.83:0/1162330302 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be00af070 con 0x7f3bf01540d0 2024-04-17T10:42:38.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.501+0000 7f3bed59a640 1 -- 172.21.15.83:0/1162330302 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be00aa9d0 con 0x7f3bf01540d0 2024-04-17T10:42:38.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.501+0000 7f3bf4caf640 1 -- 172.21.15.83:0/1162330302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf01540d0 msgr2=0x7f3bf0070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:38.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.501+0000 7f3bf4caf640 1 --2- 172.21.15.83:0/1162330302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf01540d0 0x7f3bf0070470 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009870 tx=0x7f3be001c4f0 comp rx=0 tx=0).stop 2024-04-17T10:42:38.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.501+0000 7f3bf4caf640 1 -- 172.21.15.83:0/1162330302 shutdown_connections 2024-04-17T10:42:38.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.501+0000 7f3bf4caf640 1 --2- 172.21.15.83:0/1162330302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bf015b270 0x7f3bf015d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:38.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.501+0000 7f3bf4caf640 1 --2- 172.21.15.83:0/1162330302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf01540d0 0x7f3bf0070470 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:38.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.501+0000 7f3bf4caf640 1 -- 172.21.15.83:0/1162330302 >> 172.21.15.83:0/1162330302 conn(0x7f3bf006abe0 msgr2=0x7f3bf006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:38.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.501+0000 7f3bf4caf640 1 -- 172.21.15.83:0/1162330302 shutdown_connections 2024-04-17T10:42:38.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.501+0000 7f3bf4caf640 1 -- 172.21.15.83:0/1162330302 wait complete. 2024-04-17T10:42:38.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.501+0000 7f3bf4caf640 1 Processor -- start 2024-04-17T10:42:38.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.501+0000 7f3bf4caf640 1 -- start start 2024-04-17T10:42:38.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.501+0000 7f3bf4caf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf01540d0 0x7f3bf0138120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:38.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.501+0000 7f3bf4caf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bf015b270 0x7f3bf015c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:38.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.501+0000 7f3bf4caf640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bf015dbe0 0x7f3bf015cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:38.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.501+0000 7f3bf4caf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf015e380 con 0x7f3bf01540d0 2024-04-17T10:42:38.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.501+0000 7f3bf4caf640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf01544b0 con 0x7f3bf015b270 2024-04-17T10:42:38.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.501+0000 7f3bf4caf640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf00709b0 con 0x7f3bf015dbe0 2024-04-17T10:42:38.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.501+0000 7f3bee59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf01540d0 0x7f3bf0138120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:38.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.501+0000 7f3bee59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf01540d0 0x7f3bf0138120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36738/0 (socket says 172.21.15.83:36738) 2024-04-17T10:42:38.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.501+0000 7f3bee59c640 1 -- 172.21.15.83:0/2354829081 learned_addr learned my addr 172.21.15.83:0/2354829081 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:38.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.501+0000 7f3beed9d640 1 --2- 172.21.15.83:0/2354829081 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bf015dbe0 0x7f3bf015cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:38.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.501+0000 7f3bedd9b640 1 --2- 172.21.15.83:0/2354829081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bf015b270 0x7f3bf015c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:38.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.505+0000 7f3bee59c640 1 -- 172.21.15.83:0/2354829081 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bf015dbe0 msgr2=0x7f3bf015cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:38.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.505+0000 7f3bee59c640 1 --2- 172.21.15.83:0/2354829081 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bf015dbe0 0x7f3bf015cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:38.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.505+0000 7f3bee59c640 1 -- 172.21.15.83:0/2354829081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bf015b270 msgr2=0x7f3bf015c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:38.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.505+0000 7f3bee59c640 1 --2- 172.21.15.83:0/2354829081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bf015b270 0x7f3bf015c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:38.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.505+0000 7f3bee59c640 1 -- 172.21.15.83:0/2354829081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf0152560 con 0x7f3bf01540d0 2024-04-17T10:42:38.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.505+0000 7f3bee59c640 1 --2- 172.21.15.83:0/2354829081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf01540d0 0x7f3bf0138120 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f3be00ac040 tx=0x7f3be001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:38.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.505+0000 7f3be6ffd640 1 -- 172.21.15.83:0/2354829081 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be00bd5f0 con 0x7f3bf01540d0 2024-04-17T10:42:38.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.505+0000 7f3be6ffd640 1 -- 172.21.15.83:0/2354829081 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be00bd7b0 con 0x7f3bf01540d0 2024-04-17T10:42:38.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.505+0000 7f3bf4caf640 1 -- 172.21.15.83:0/2354829081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf0152840 con 0x7f3bf01540d0 2024-04-17T10:42:38.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.505+0000 7f3be6ffd640 1 -- 172.21.15.83:0/2354829081 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be00bdba0 con 0x7f3bf01540d0 2024-04-17T10:42:38.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.505+0000 7f3bf4caf640 1 -- 172.21.15.83:0/2354829081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf0152e00 con 0x7f3bf01540d0 2024-04-17T10:42:38.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.505+0000 7f3bf4caf640 1 -- 172.21.15.83:0/2354829081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bf0074f90 con 0x7f3bf01540d0 2024-04-17T10:42:38.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.505+0000 7f3be6ffd640 1 -- 172.21.15.83:0/2354829081 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3be00b33f0 con 0x7f3bf01540d0 2024-04-17T10:42:38.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.505+0000 7f3be6ffd640 1 --2- 172.21.15.83:0/2354829081 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3bd0066f60 0x7f3bd0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:38.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.505+0000 7f3be6ffd640 1 -- 172.21.15.83:0/2354829081 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3be011faf0 con 0x7f3bf01540d0 2024-04-17T10:42:38.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.505+0000 7f3bedd9b640 1 --2- 172.21.15.83:0/2354829081 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3bd0066f60 0x7f3bd0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:38.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.505+0000 7f3bedd9b640 1 --2- 172.21.15.83:0/2354829081 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3bd0066f60 0x7f3bd0069420 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0146170 tx=0x7f3bd8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:38.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.509+0000 7f3be6ffd640 1 -- 172.21.15.83:0/2354829081 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3be00c3060 con 0x7f3bf01540d0 2024-04-17T10:42:38.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.669+0000 7f3bf4caf640 1 -- 172.21.15.83:0/2354829081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3bf0074f90 con 0x7f3bf01540d0 2024-04-17T10:42:38.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.669+0000 7f3be6ffd640 1 -- 172.21.15.83:0/2354829081 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+16593 (secure 0 0 0) 0x7f3be00a7070 con 0x7f3bf01540d0 2024-04-17T10:42:38.671 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:38.671 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":59,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:42:35.975462+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:42:32.530700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-17T10:42:33.334905+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:42:38.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.673+0000 7f3bf4caf640 1 -- 172.21.15.83:0/2354829081 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3bd0066f60 msgr2=0x7f3bd0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:38.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.673+0000 7f3bf4caf640 1 --2- 172.21.15.83:0/2354829081 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3bd0066f60 0x7f3bd0069420 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0146170 tx=0x7f3bd8008040 comp rx=0 tx=0).stop 2024-04-17T10:42:38.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.673+0000 7f3bf4caf640 1 -- 172.21.15.83:0/2354829081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf01540d0 msgr2=0x7f3bf0138120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:38.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.673+0000 7f3bf4caf640 1 --2- 172.21.15.83:0/2354829081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf01540d0 0x7f3bf0138120 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f3be00ac040 tx=0x7f3be001ad30 comp rx=0 tx=0).stop 2024-04-17T10:42:38.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.673+0000 7f3bf4caf640 1 -- 172.21.15.83:0/2354829081 shutdown_connections 2024-04-17T10:42:38.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.673+0000 7f3bf4caf640 1 --2- 172.21.15.83:0/2354829081 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3bd0066f60 0x7f3bd0069420 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:38.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.673+0000 7f3bf4caf640 1 --2- 172.21.15.83:0/2354829081 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bf015dbe0 0x7f3bf015cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:38.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.673+0000 7f3bf4caf640 1 --2- 172.21.15.83:0/2354829081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bf015b270 0x7f3bf015c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:38.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.673+0000 7f3bf4caf640 1 --2- 172.21.15.83:0/2354829081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf01540d0 0x7f3bf0138120 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:38.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.673+0000 7f3bf4caf640 1 -- 172.21.15.83:0/2354829081 >> 172.21.15.83:0/2354829081 conn(0x7f3bf006abe0 msgr2=0x7f3bf014ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:38.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.673+0000 7f3bf4caf640 1 -- 172.21.15.83:0/2354829081 shutdown_connections 2024-04-17T10:42:38.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.673+0000 7f3bf4caf640 1 -- 172.21.15.83:0/2354829081 wait complete. 2024-04-17T10:42:38.694 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-17T10:42:38.694 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:42:38.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.817+0000 7f1f7f8eb640 1 Processor -- start 2024-04-17T10:42:38.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.817+0000 7f1f7f8eb640 1 -- start start 2024-04-17T10:42:38.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.817+0000 7f1f7f8eb640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f7815db50 0x7f1f78070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:38.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.817+0000 7f1f7f8eb640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1f7806dcb0 0x7f1f780b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:38.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.817+0000 7f1f7f8eb640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f780709b0 con 0x7f1f780aedd0 2024-04-17T10:42:38.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.817+0000 7f1f7f8eb640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f7815e440 con 0x7f1f7806dcb0 2024-04-17T10:42:38.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.817+0000 7f1f7f8eb640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7815e720 con 0x7f1f7815db50 2024-04-17T10:42:38.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.817+0000 7f1f7de88640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1f7806dcb0 0x7f1f780b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:38.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.817+0000 7f1f7de88640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1f7806dcb0 0x7f1f780b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36752/0 (socket says 172.21.15.83:36752) 2024-04-17T10:42:38.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.817+0000 7f1f7ce86640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1f780aedd0 0x7f1f7806d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57424/0 (socket says 172.21.15.83:57424) 2024-04-17T10:42:38.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.817+0000 7f1f7de88640 1 -- 172.21.15.83:0/4021312469 learned_addr learned my addr 172.21.15.83:0/4021312469 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:38.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.817+0000 7f1f7d687640 1 --2- 172.21.15.83:0/4021312469 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f7815db50 0x7f1f78070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:38.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.817+0000 7f1f7de88640 1 -- 172.21.15.83:0/4021312469 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f7815db50 msgr2=0x7f1f78070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:38.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.817+0000 7f1f7de88640 1 --2- 172.21.15.83:0/4021312469 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f7815db50 0x7f1f78070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:38.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.817+0000 7f1f7de88640 1 -- 172.21.15.83:0/4021312469 >> v1:172.21.15.83:6789/0 conn(0x7f1f780aedd0 legacy=0x7f1f7806d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:38.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.817+0000 7f1f7de88640 1 -- 172.21.15.83:0/4021312469 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7815ea00 con 0x7f1f7806dcb0 2024-04-17T10:42:38.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.817+0000 7f1f6ffff640 1 -- 172.21.15.83:0/4021312469 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 871311175 0 0) 0x7f1f74001120 con 0x7f1f780aedd0 2024-04-17T10:42:38.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.821+0000 7f1f7de88640 1 --2- 172.21.15.83:0/4021312469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f7806dcb0 0x7f1f780b2e10 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f1f7000d150 tx=0x7f1f7001ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b70b0ce9924396 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:38.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.821+0000 7f1f6ffff640 1 -- 172.21.15.83:0/4021312469 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f700a7cd0 con 0x7f1f7806dcb0 2024-04-17T10:42:38.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.821+0000 7f1f6ffff640 1 -- 172.21.15.83:0/4021312469 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7000b0d0 con 0x7f1f7806dcb0 2024-04-17T10:42:38.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.821+0000 7f1f6ffff640 1 -- 172.21.15.83:0/4021312469 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70009600 con 0x7f1f7806dcb0 2024-04-17T10:42:38.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.821+0000 7f1f7f8eb640 1 -- 172.21.15.83:0/4021312469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f7806dcb0 msgr2=0x7f1f780b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:38.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.821+0000 7f1f7f8eb640 1 --2- 172.21.15.83:0/4021312469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f7806dcb0 0x7f1f780b2e10 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f1f7000d150 tx=0x7f1f7001ec00 comp rx=0 tx=0).stop 2024-04-17T10:42:38.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.821+0000 7f1f7f8eb640 1 -- 172.21.15.83:0/4021312469 shutdown_connections 2024-04-17T10:42:38.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.821+0000 7f1f7f8eb640 1 --2- 172.21.15.83:0/4021312469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f7806dcb0 0x7f1f780b2e10 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:38.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.821+0000 7f1f7f8eb640 1 --2- 172.21.15.83:0/4021312469 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f7815db50 0x7f1f78070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:38.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.821+0000 7f1f7f8eb640 1 -- 172.21.15.83:0/4021312469 >> 172.21.15.83:0/4021312469 conn(0x7f1f7806abe0 msgr2=0x7f1f7806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:38.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.821+0000 7f1f7f8eb640 1 -- 172.21.15.83:0/4021312469 shutdown_connections 2024-04-17T10:42:38.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.821+0000 7f1f7f8eb640 1 -- 172.21.15.83:0/4021312469 wait complete. 2024-04-17T10:42:38.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.821+0000 7f1f7f8eb640 1 Processor -- start 2024-04-17T10:42:38.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.821+0000 7f1f7f8eb640 1 -- start start 2024-04-17T10:42:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.821+0000 7f1f7f8eb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f7806dcb0 0x7f1f7815a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.821+0000 7f1f7f8eb640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f780aedd0 0x7f1f78157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.821+0000 7f1f7f8eb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f7815db50 0x7f1f7814f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.821+0000 7f1f7f8eb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f780b3ce0 con 0x7f1f7815db50 2024-04-17T10:42:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.821+0000 7f1f7f8eb640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f780b3b60 con 0x7f1f7806dcb0 2024-04-17T10:42:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.821+0000 7f1f7f8eb640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f780b3e60 con 0x7f1f780aedd0 2024-04-17T10:42:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.821+0000 7f1f7de88640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f7815db50 0x7f1f7814f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.821+0000 7f1f7de88640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f7815db50 0x7f1f7814f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36762/0 (socket says 172.21.15.83:36762) 2024-04-17T10:42:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.821+0000 7f1f7de88640 1 -- 172.21.15.83:0/1114186681 learned_addr learned my addr 172.21.15.83:0/1114186681 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.821+0000 7f1f7ce86640 1 --2- 172.21.15.83:0/1114186681 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f780aedd0 0x7f1f78157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.821+0000 7f1f7d687640 1 --2- 172.21.15.83:0/1114186681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f7806dcb0 0x7f1f7815a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.821+0000 7f1f7de88640 1 -- 172.21.15.83:0/1114186681 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f780aedd0 msgr2=0x7f1f78157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.821+0000 7f1f7de88640 1 --2- 172.21.15.83:0/1114186681 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f780aedd0 0x7f1f78157bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:38.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.821+0000 7f1f7de88640 1 -- 172.21.15.83:0/1114186681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f7806dcb0 msgr2=0x7f1f7815a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:38.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.821+0000 7f1f7de88640 1 --2- 172.21.15.83:0/1114186681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f7806dcb0 0x7f1f7815a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:38.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.821+0000 7f1f7de88640 1 -- 172.21.15.83:0/1114186681 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f781582d0 con 0x7f1f7815db50 2024-04-17T10:42:38.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.821+0000 7f1f7ce86640 1 --2- 172.21.15.83:0/1114186681 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f780aedd0 0x7f1f78157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:38.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.821+0000 7f1f7de88640 1 --2- 172.21.15.83:0/1114186681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f7815db50 0x7f1f7814f290 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f1f700b1040 tx=0x7f1f7001b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:38.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.825+0000 7f1f6dffb640 1 -- 172.21.15.83:0/1114186681 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f700be020 con 0x7f1f7815db50 2024-04-17T10:42:38.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.825+0000 7f1f6dffb640 1 -- 172.21.15.83:0/1114186681 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f70009840 con 0x7f1f7815db50 2024-04-17T10:42:38.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.825+0000 7f1f7f8eb640 1 -- 172.21.15.83:0/1114186681 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f781585b0 con 0x7f1f7815db50 2024-04-17T10:42:38.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.825+0000 7f1f6dffb640 1 -- 172.21.15.83:0/1114186681 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70009c30 con 0x7f1f7815db50 2024-04-17T10:42:38.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.825+0000 7f1f7f8eb640 1 -- 172.21.15.83:0/1114186681 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f78158b70 con 0x7f1f7815db50 2024-04-17T10:42:38.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.825+0000 7f1f7f8eb640 1 -- 172.21.15.83:0/1114186681 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f78074f90 con 0x7f1f7815db50 2024-04-17T10:42:38.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.825+0000 7f1f6dffb640 1 -- 172.21.15.83:0/1114186681 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1f700c1020 con 0x7f1f7815db50 2024-04-17T10:42:38.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.825+0000 7f1f6dffb640 1 --2- 172.21.15.83:0/1114186681 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1f5c066f60 0x7f1f5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:38.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.825+0000 7f1f6dffb640 1 -- 172.21.15.83:0/1114186681 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f701221c0 con 0x7f1f7815db50 2024-04-17T10:42:38.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.825+0000 7f1f7d687640 1 --2- 172.21.15.83:0/1114186681 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1f5c066f60 0x7f1f5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:38.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.825+0000 7f1f7d687640 1 --2- 172.21.15.83:0/1114186681 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1f5c066f60 0x7f1f5c069420 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1f680096f0 tx=0x7f1f68008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.829+0000 7f1f6dffb640 1 -- 172.21.15.83:0/1114186681 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1f70018820 con 0x7f1f7815db50 2024-04-17T10:42:38.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.989+0000 7f1f7f8eb640 1 -- 172.21.15.83:0/1114186681 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1f780d01b0 con 0x7f1f5c066f60 2024-04-17T10:42:38.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.993+0000 7f1f6dffb640 1 -- 172.21.15.83:0/1114186681 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246636 (secure 0 0 0) 0x7f1f780d01b0 con 0x7f1f5c066f60 2024-04-17T10:42:38.998 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:39.000 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:42:39.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.997+0000 7f1f7f8eb640 1 -- 172.21.15.83:0/1114186681 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1f5c066f60 msgr2=0x7f1f5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:39.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.997+0000 7f1f7f8eb640 1 --2- 172.21.15.83:0/1114186681 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1f5c066f60 0x7f1f5c069420 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1f680096f0 tx=0x7f1f68008040 comp rx=0 tx=0).stop 2024-04-17T10:42:39.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.997+0000 7f1f7f8eb640 1 -- 172.21.15.83:0/1114186681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f7815db50 msgr2=0x7f1f7814f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:39.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:38.997+0000 7f1f7f8eb640 1 --2- 172.21.15.83:0/1114186681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f7815db50 0x7f1f7814f290 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f1f700b1040 tx=0x7f1f7001b9a0 comp rx=0 tx=0).stop 2024-04-17T10:42:39.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.001+0000 7f1f7f8eb640 1 -- 172.21.15.83:0/1114186681 shutdown_connections 2024-04-17T10:42:39.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.001+0000 7f1f7f8eb640 1 --2- 172.21.15.83:0/1114186681 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1f5c066f60 0x7f1f5c069420 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:39.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.001+0000 7f1f7f8eb640 1 --2- 172.21.15.83:0/1114186681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f7815db50 0x7f1f7814f290 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:39.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.001+0000 7f1f7f8eb640 1 --2- 172.21.15.83:0/1114186681 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f780aedd0 0x7f1f78157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:39.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.001+0000 7f1f7f8eb640 1 --2- 172.21.15.83:0/1114186681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f7806dcb0 0x7f1f7815a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:39.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.001+0000 7f1f7f8eb640 1 -- 172.21.15.83:0/1114186681 >> 172.21.15.83:0/1114186681 conn(0x7f1f7806abe0 msgr2=0x7f1f7806cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:39.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.001+0000 7f1f7f8eb640 1 -- 172.21.15.83:0/1114186681 shutdown_connections 2024-04-17T10:42:39.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.001+0000 7f1f7f8eb640 1 -- 172.21.15.83:0/1114186681 wait complete. 2024-04-17T10:42:39.018 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2024-04-17T10:42:38.259280+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":355828,"kb_used_data":6444,"kb_used_omap":12,"kb_used_meta":349235,"kb_avail":754618892,"statfs":{"total":773094113280,"available":772729745408,"internally_reserved":0,"allocated":6598656,"data_stored":3424729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":357617200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.296682"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.988554+0000","last_change":"2024-04-17T10:42:33.988554+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.988554+0000","last_undegraded":"2024-04-17T10:42:33.988554+0000","last_fullsized":"2024-04-17T10:42:33.988554+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.992506+0000","last_change":"2024-04-17T10:42:33.992506+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.992506+0000","last_undegraded":"2024-04-17T10:42:33.992506+0000","last_fullsized":"2024-04-17T10:42:33.992506+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.991262+0000","last_change":"2024-04-17T10:42:33.991262+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.991262+0000","last_undegraded":"2024-04-17T10:42:33.991262+0000","last_fullsized":"2024-04-17T10:42:33.991262+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.999359+0000","last_change":"2024-04-17T10:42:33.999359+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.999359+0000","last_undegraded":"2024-04-17T10:42:33.999359+0000","last_fullsized":"2024-04-17T10:42:33.999359+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.985818+0000","last_change":"2024-04-17T10:42:33.985818+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.985818+0000","last_undegraded":"2024-04-17T10:42:33.985818+0000","last_fullsized":"2024-04-17T10:42:33.985818+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982596+0000","last_change":"2024-04-17T10:42:33.982596+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982596+0000","last_undegraded":"2024-04-17T10:42:33.982596+0000","last_fullsized":"2024-04-17T10:42:33.982596+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.984830+0000","last_change":"2024-04-17T10:42:33.984830+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.984830+0000","last_undegraded":"2024-04-17T10:42:33.984830+0000","last_fullsized":"2024-04-17T10:42:33.984830+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.994100+0000","last_change":"2024-04-17T10:42:33.994100+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.994100+0000","last_undegraded":"2024-04-17T10:42:33.994100+0000","last_fullsized":"2024-04-17T10:42:33.994100+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982038+0000","last_change":"2024-04-17T10:42:33.982038+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982038+0000","last_undegraded":"2024-04-17T10:42:33.982038+0000","last_fullsized":"2024-04-17T10:42:33.982038+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.980127+0000","last_change":"2024-04-17T10:42:33.980127+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.980127+0000","last_undegraded":"2024-04-17T10:42:33.980127+0000","last_fullsized":"2024-04-17T10:42:33.980127+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.980448+0000","last_change":"2024-04-17T10:42:33.980448+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.980448+0000","last_undegraded":"2024-04-17T10:42:33.980448+0000","last_fullsized":"2024-04-17T10:42:33.980448+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.995007+0000","last_change":"2024-04-17T10:42:33.995007+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.995007+0000","last_undegraded":"2024-04-17T10:42:33.995007+0000","last_fullsized":"2024-04-17T10:42:33.995007+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.990877+0000","last_change":"2024-04-17T10:42:33.990877+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.990877+0000","last_undegraded":"2024-04-17T10:42:33.990877+0000","last_fullsized":"2024-04-17T10:42:33.990877+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.983487+0000","last_change":"2024-04-17T10:42:33.983487+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.983487+0000","last_undegraded":"2024-04-17T10:42:33.983487+0000","last_fullsized":"2024-04-17T10:42:33.983487+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982041+0000","last_change":"2024-04-17T10:42:33.982041+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982041+0000","last_undegraded":"2024-04-17T10:42:33.982041+0000","last_fullsized":"2024-04-17T10:42:33.982041+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.985005+0000","last_change":"2024-04-17T10:42:33.985005+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.985005+0000","last_undegraded":"2024-04-17T10:42:33.985005+0000","last_fullsized":"2024-04-17T10:42:33.985005+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.994197+0000","last_change":"2024-04-17T10:42:33.994197+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.994197+0000","last_undegraded":"2024-04-17T10:42:33.994197+0000","last_fullsized":"2024-04-17T10:42:33.994197+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.988273+0000","last_change":"2024-04-17T10:42:33.988273+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.988273+0000","last_undegraded":"2024-04-17T10:42:33.988273+0000","last_fullsized":"2024-04-17T10:42:33.988273+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.985480+0000","last_change":"2024-04-17T10:42:33.985480+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.985480+0000","last_undegraded":"2024-04-17T10:42:33.985480+0000","last_fullsized":"2024-04-17T10:42:33.985480+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982918+0000","last_change":"2024-04-17T10:42:33.982918+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982918+0000","last_undegraded":"2024-04-17T10:42:33.982918+0000","last_fullsized":"2024-04-17T10:42:33.982918+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.984586+0000","last_change":"2024-04-17T10:42:33.984586+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.984586+0000","last_undegraded":"2024-04-17T10:42:33.984586+0000","last_fullsized":"2024-04-17T10:42:33.984586+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.994276+0000","last_change":"2024-04-17T10:42:33.994276+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.994276+0000","last_undegraded":"2024-04-17T10:42:33.994276+0000","last_fullsized":"2024-04-17T10:42:33.994276+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.983711+0000","last_change":"2024-04-17T10:42:33.983711+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.983711+0000","last_undegraded":"2024-04-17T10:42:33.983711+0000","last_fullsized":"2024-04-17T10:42:33.983711+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.992224+0000","last_change":"2024-04-17T10:42:33.992224+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.992224+0000","last_undegraded":"2024-04-17T10:42:33.992224+0000","last_fullsized":"2024-04-17T10:42:33.992224+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.988095+0000","last_change":"2024-04-17T10:42:33.988095+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.988095+0000","last_undegraded":"2024-04-17T10:42:33.988095+0000","last_fullsized":"2024-04-17T10:42:33.988095+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.996585+0000","last_change":"2024-04-17T10:42:33.996585+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.996585+0000","last_undegraded":"2024-04-17T10:42:33.996585+0000","last_fullsized":"2024-04-17T10:42:33.996585+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.987477+0000","last_change":"2024-04-17T10:42:33.987477+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.987477+0000","last_undegraded":"2024-04-17T10:42:33.987477+0000","last_fullsized":"2024-04-17T10:42:33.987477+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.985065+0000","last_change":"2024-04-17T10:42:33.985065+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.985065+0000","last_undegraded":"2024-04-17T10:42:33.985065+0000","last_fullsized":"2024-04-17T10:42:33.985065+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.991668+0000","last_change":"2024-04-17T10:42:33.991668+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.991668+0000","last_undegraded":"2024-04-17T10:42:33.991668+0000","last_fullsized":"2024-04-17T10:42:33.991668+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.985954+0000","last_change":"2024-04-17T10:42:33.985954+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.985954+0000","last_undegraded":"2024-04-17T10:42:33.985954+0000","last_fullsized":"2024-04-17T10:42:33.985954+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.984858+0000","last_change":"2024-04-17T10:42:33.984858+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.984858+0000","last_undegraded":"2024-04-17T10:42:33.984858+0000","last_fullsized":"2024-04-17T10:42:33.984858+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.984313+0000","last_change":"2024-04-17T10:42:33.984313+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.984313+0000","last_undegraded":"2024-04-17T10:42:33.984313+0000","last_fullsized":"2024-04-17T10:42:33.984313+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.987311+0000","last_change":"2024-04-17T10:42:33.987311+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.987311+0000","last_undegraded":"2024-04-17T10:42:33.987311+0000","last_fullsized":"2024-04-17T10:42:33.987311+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.015861+0000","last_change":"2024-04-17T10:42:34.015019+0000","last_active":"2024-04-17T10:42:34.015861+0000","last_peered":"2024-04-17T10:42:34.015861+0000","last_clean":"2024-04-17T10:42:34.015861+0000","last_became_active":"2024-04-17T10:42:34.014716+0000","last_became_peered":"2024-04-17T10:42:34.014716+0000","last_unstale":"2024-04-17T10:42:34.015861+0000","last_undegraded":"2024-04-17T10:42:34.015861+0000","last_fullsized":"2024-04-17T10:42:34.015861+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:26:57.986949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.980974+0000","last_change":"2024-04-17T10:42:33.980974+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.980974+0000","last_undegraded":"2024-04-17T10:42:33.980974+0000","last_fullsized":"2024-04-17T10:42:33.980974+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.006810+0000","last_change":"2024-04-17T10:42:34.006170+0000","last_active":"2024-04-17T10:42:34.006810+0000","last_peered":"2024-04-17T10:42:34.006810+0000","last_clean":"2024-04-17T10:42:34.006810+0000","last_became_active":"2024-04-17T10:42:34.005757+0000","last_became_peered":"2024-04-17T10:42:34.005757+0000","last_unstale":"2024-04-17T10:42:34.006810+0000","last_undegraded":"2024-04-17T10:42:34.006810+0000","last_fullsized":"2024-04-17T10:42:34.006810+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:25:59.364150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982188+0000","last_change":"2024-04-17T10:42:33.982188+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982188+0000","last_undegraded":"2024-04-17T10:42:33.982188+0000","last_fullsized":"2024-04-17T10:42:33.982188+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:33.996540+0000","last_change":"2024-04-17T10:42:33.993426+0000","last_active":"2024-04-17T10:42:33.996540+0000","last_peered":"2024-04-17T10:42:33.996540+0000","last_clean":"2024-04-17T10:42:33.996540+0000","last_became_active":"2024-04-17T10:42:33.992971+0000","last_became_peered":"2024-04-17T10:42:33.992971+0000","last_unstale":"2024-04-17T10:42:33.996540+0000","last_undegraded":"2024-04-17T10:42:33.996540+0000","last_fullsized":"2024-04-17T10:42:33.996540+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:42:12.531359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.987048+0000","last_change":"2024-04-17T10:42:33.987048+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.987048+0000","last_undegraded":"2024-04-17T10:42:33.987048+0000","last_fullsized":"2024-04-17T10:42:33.987048+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.021952+0000","last_change":"2024-04-17T10:42:34.018234+0000","last_active":"2024-04-17T10:42:34.021952+0000","last_peered":"2024-04-17T10:42:34.021952+0000","last_clean":"2024-04-17T10:42:34.021952+0000","last_became_active":"2024-04-17T10:42:34.017497+0000","last_became_peered":"2024-04-17T10:42:34.017497+0000","last_unstale":"2024-04-17T10:42:34.021952+0000","last_undegraded":"2024-04-17T10:42:34.021952+0000","last_fullsized":"2024-04-17T10:42:34.021952+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:32:27.523313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.992516+0000","last_change":"2024-04-17T10:42:33.992516+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.992516+0000","last_undegraded":"2024-04-17T10:42:33.992516+0000","last_fullsized":"2024-04-17T10:42:33.992516+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.013446+0000","last_change":"2024-04-17T10:42:34.004483+0000","last_active":"2024-04-17T10:42:34.013446+0000","last_peered":"2024-04-17T10:42:34.013446+0000","last_clean":"2024-04-17T10:42:34.013446+0000","last_became_active":"2024-04-17T10:42:33.998883+0000","last_became_peered":"2024-04-17T10:42:33.998883+0000","last_unstale":"2024-04-17T10:42:34.013446+0000","last_undegraded":"2024-04-17T10:42:34.013446+0000","last_fullsized":"2024-04-17T10:42:34.013446+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:42:12.531359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.992432+0000","last_change":"2024-04-17T10:42:33.992432+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.992432+0000","last_undegraded":"2024-04-17T10:42:33.992432+0000","last_fullsized":"2024-04-17T10:42:33.992432+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.002170+0000","last_change":"2024-04-17T10:42:34.000321+0000","last_active":"2024-04-17T10:42:34.002170+0000","last_peered":"2024-04-17T10:42:34.002170+0000","last_clean":"2024-04-17T10:42:34.002170+0000","last_became_active":"2024-04-17T10:42:34.000062+0000","last_became_peered":"2024-04-17T10:42:34.000062+0000","last_unstale":"2024-04-17T10:42:34.002170+0000","last_undegraded":"2024-04-17T10:42:34.002170+0000","last_fullsized":"2024-04-17T10:42:34.002170+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:58:18.204240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.991126+0000","last_change":"2024-04-17T10:42:33.991126+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.991126+0000","last_undegraded":"2024-04-17T10:42:33.991126+0000","last_fullsized":"2024-04-17T10:42:33.991126+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.021947+0000","last_change":"2024-04-17T10:42:34.018055+0000","last_active":"2024-04-17T10:42:34.021947+0000","last_peered":"2024-04-17T10:42:34.021947+0000","last_clean":"2024-04-17T10:42:34.021947+0000","last_became_active":"2024-04-17T10:42:34.016719+0000","last_became_peered":"2024-04-17T10:42:34.016719+0000","last_unstale":"2024-04-17T10:42:34.021947+0000","last_undegraded":"2024-04-17T10:42:34.021947+0000","last_fullsized":"2024-04-17T10:42:34.021947+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:20:46.478088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.990753+0000","last_change":"2024-04-17T10:42:33.990753+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.990753+0000","last_undegraded":"2024-04-17T10:42:33.990753+0000","last_fullsized":"2024-04-17T10:42:33.990753+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.013153+0000","last_change":"2024-04-17T10:42:34.008774+0000","last_active":"2024-04-17T10:42:34.013153+0000","last_peered":"2024-04-17T10:42:34.013153+0000","last_clean":"2024-04-17T10:42:34.013153+0000","last_became_active":"2024-04-17T10:42:34.005106+0000","last_became_peered":"2024-04-17T10:42:34.005106+0000","last_unstale":"2024-04-17T10:42:34.013153+0000","last_undegraded":"2024-04-17T10:42:34.013153+0000","last_fullsized":"2024-04-17T10:42:34.013153+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:58:18.204240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.984503+0000","last_change":"2024-04-17T10:42:33.984503+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.984503+0000","last_undegraded":"2024-04-17T10:42:33.984503+0000","last_fullsized":"2024-04-17T10:42:33.984503+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.014222+0000","last_change":"2024-04-17T10:42:34.006222+0000","last_active":"2024-04-17T10:42:34.014222+0000","last_peered":"2024-04-17T10:42:34.014222+0000","last_clean":"2024-04-17T10:42:34.014222+0000","last_became_active":"2024-04-17T10:42:34.005353+0000","last_became_peered":"2024-04-17T10:42:34.005353+0000","last_unstale":"2024-04-17T10:42:34.014222+0000","last_undegraded":"2024-04-17T10:42:34.014222+0000","last_fullsized":"2024-04-17T10:42:34.014222+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:20:46.478088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.985320+0000","last_change":"2024-04-17T10:42:33.985320+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.985320+0000","last_undegraded":"2024-04-17T10:42:33.985320+0000","last_fullsized":"2024-04-17T10:42:33.985320+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.000258+0000","last_change":"2024-04-17T10:42:33.995027+0000","last_active":"2024-04-17T10:42:34.000258+0000","last_peered":"2024-04-17T10:42:34.000258+0000","last_clean":"2024-04-17T10:42:34.000258+0000","last_became_active":"2024-04-17T10:42:33.994748+0000","last_became_peered":"2024-04-17T10:42:33.994748+0000","last_unstale":"2024-04-17T10:42:34.000258+0000","last_undegraded":"2024-04-17T10:42:34.000258+0000","last_fullsized":"2024-04-17T10:42:34.000258+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:41:52.745779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.981639+0000","last_change":"2024-04-17T10:42:33.981639+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.981639+0000","last_undegraded":"2024-04-17T10:42:33.981639+0000","last_fullsized":"2024-04-17T10:42:33.981639+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.016297+0000","last_change":"2024-04-17T10:42:34.010793+0000","last_active":"2024-04-17T10:42:34.016297+0000","last_peered":"2024-04-17T10:42:34.016297+0000","last_clean":"2024-04-17T10:42:34.016297+0000","last_became_active":"2024-04-17T10:42:34.009786+0000","last_became_peered":"2024-04-17T10:42:34.009786+0000","last_unstale":"2024-04-17T10:42:34.016297+0000","last_undegraded":"2024-04-17T10:42:34.016297+0000","last_fullsized":"2024-04-17T10:42:34.016297+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:32:27.523313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.990052+0000","last_change":"2024-04-17T10:42:33.990052+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.990052+0000","last_undegraded":"2024-04-17T10:42:33.990052+0000","last_fullsized":"2024-04-17T10:42:33.990052+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.013294+0000","last_change":"2024-04-17T10:42:34.008738+0000","last_active":"2024-04-17T10:42:34.013294+0000","last_peered":"2024-04-17T10:42:34.013294+0000","last_clean":"2024-04-17T10:42:34.013294+0000","last_became_active":"2024-04-17T10:42:34.004730+0000","last_became_peered":"2024-04-17T10:42:34.004730+0000","last_unstale":"2024-04-17T10:42:34.013294+0000","last_undegraded":"2024-04-17T10:42:34.013294+0000","last_fullsized":"2024-04-17T10:42:34.013294+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:14:14.289936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.993209+0000","last_change":"2024-04-17T10:42:33.993209+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.993209+0000","last_undegraded":"2024-04-17T10:42:33.993209+0000","last_fullsized":"2024-04-17T10:42:33.993209+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.000845+0000","last_change":"2024-04-17T10:42:34.000022+0000","last_active":"2024-04-17T10:42:34.000845+0000","last_peered":"2024-04-17T10:42:34.000845+0000","last_clean":"2024-04-17T10:42:34.000845+0000","last_became_active":"2024-04-17T10:42:33.999505+0000","last_became_peered":"2024-04-17T10:42:33.999505+0000","last_unstale":"2024-04-17T10:42:34.000845+0000","last_undegraded":"2024-04-17T10:42:34.000845+0000","last_fullsized":"2024-04-17T10:42:34.000845+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:01:33.910102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.983396+0000","last_change":"2024-04-17T10:42:33.983396+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.983396+0000","last_undegraded":"2024-04-17T10:42:33.983396+0000","last_fullsized":"2024-04-17T10:42:33.983396+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.015295+0000","last_change":"2024-04-17T10:42:34.010051+0000","last_active":"2024-04-17T10:42:34.015295+0000","last_peered":"2024-04-17T10:42:34.015295+0000","last_clean":"2024-04-17T10:42:34.015295+0000","last_became_active":"2024-04-17T10:42:34.009227+0000","last_became_peered":"2024-04-17T10:42:34.009227+0000","last_unstale":"2024-04-17T10:42:34.015295+0000","last_undegraded":"2024-04-17T10:42:34.015295+0000","last_fullsized":"2024-04-17T10:42:34.015295+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:41:05.001485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982143+0000","last_change":"2024-04-17T10:42:33.982143+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982143+0000","last_undegraded":"2024-04-17T10:42:33.982143+0000","last_fullsized":"2024-04-17T10:42:33.982143+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.023975+0000","last_change":"2024-04-17T10:42:34.019631+0000","last_active":"2024-04-17T10:42:34.023975+0000","last_peered":"2024-04-17T10:42:34.023975+0000","last_clean":"2024-04-17T10:42:34.023975+0000","last_became_active":"2024-04-17T10:42:34.018768+0000","last_became_peered":"2024-04-17T10:42:34.018768+0000","last_unstale":"2024-04-17T10:42:34.023975+0000","last_undegraded":"2024-04-17T10:42:34.023975+0000","last_fullsized":"2024-04-17T10:42:34.023975+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:37:08.284805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982708+0000","last_change":"2024-04-17T10:42:33.982708+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982708+0000","last_undegraded":"2024-04-17T10:42:33.982708+0000","last_fullsized":"2024-04-17T10:42:33.982708+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.019298+0000","last_change":"2024-04-17T10:42:34.015072+0000","last_active":"2024-04-17T10:42:34.019298+0000","last_peered":"2024-04-17T10:42:34.019298+0000","last_clean":"2024-04-17T10:42:34.019298+0000","last_became_active":"2024-04-17T10:42:34.014406+0000","last_became_peered":"2024-04-17T10:42:34.014406+0000","last_unstale":"2024-04-17T10:42:34.019298+0000","last_undegraded":"2024-04-17T10:42:34.019298+0000","last_fullsized":"2024-04-17T10:42:34.019298+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:37:08.284805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.984738+0000","last_change":"2024-04-17T10:42:33.984738+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.984738+0000","last_undegraded":"2024-04-17T10:42:33.984738+0000","last_fullsized":"2024-04-17T10:42:33.984738+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.000855+0000","last_change":"2024-04-17T10:42:33.998582+0000","last_active":"2024-04-17T10:42:34.000855+0000","last_peered":"2024-04-17T10:42:34.000855+0000","last_clean":"2024-04-17T10:42:34.000855+0000","last_became_active":"2024-04-17T10:42:33.997341+0000","last_became_peered":"2024-04-17T10:42:33.997341+0000","last_unstale":"2024-04-17T10:42:34.000855+0000","last_undegraded":"2024-04-17T10:42:34.000855+0000","last_fullsized":"2024-04-17T10:42:34.000855+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:03:51.699883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.999265+0000","last_change":"2024-04-17T10:42:33.999265+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.999265+0000","last_undegraded":"2024-04-17T10:42:33.999265+0000","last_fullsized":"2024-04-17T10:42:33.999265+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:33.999215+0000","last_change":"2024-04-17T10:42:33.995762+0000","last_active":"2024-04-17T10:42:33.999215+0000","last_peered":"2024-04-17T10:42:33.999215+0000","last_clean":"2024-04-17T10:42:33.999215+0000","last_became_active":"2024-04-17T10:42:33.995562+0000","last_became_peered":"2024-04-17T10:42:33.995562+0000","last_unstale":"2024-04-17T10:42:33.999215+0000","last_undegraded":"2024-04-17T10:42:33.999215+0000","last_fullsized":"2024-04-17T10:42:33.999215+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:14:14.289936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982369+0000","last_change":"2024-04-17T10:42:33.982369+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982369+0000","last_undegraded":"2024-04-17T10:42:33.982369+0000","last_fullsized":"2024-04-17T10:42:33.982369+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.021531+0000","last_change":"2024-04-17T10:42:34.017202+0000","last_active":"2024-04-17T10:42:34.021531+0000","last_peered":"2024-04-17T10:42:34.021531+0000","last_clean":"2024-04-17T10:42:34.021531+0000","last_became_active":"2024-04-17T10:42:34.016613+0000","last_became_peered":"2024-04-17T10:42:34.016613+0000","last_unstale":"2024-04-17T10:42:34.021531+0000","last_undegraded":"2024-04-17T10:42:34.021531+0000","last_fullsized":"2024-04-17T10:42:34.021531+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:09:32.979808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.985866+0000","last_change":"2024-04-17T10:42:33.985866+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.985866+0000","last_undegraded":"2024-04-17T10:42:33.985866+0000","last_fullsized":"2024-04-17T10:42:33.985866+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.002835+0000","last_change":"2024-04-17T10:42:34.002003+0000","last_active":"2024-04-17T10:42:34.002835+0000","last_peered":"2024-04-17T10:42:34.002835+0000","last_clean":"2024-04-17T10:42:34.002835+0000","last_became_active":"2024-04-17T10:42:34.001413+0000","last_became_peered":"2024-04-17T10:42:34.001413+0000","last_unstale":"2024-04-17T10:42:34.002835+0000","last_undegraded":"2024-04-17T10:42:34.002835+0000","last_fullsized":"2024-04-17T10:42:34.002835+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:44:30.520002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.991626+0000","last_change":"2024-04-17T10:42:33.991626+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.991626+0000","last_undegraded":"2024-04-17T10:42:33.991626+0000","last_fullsized":"2024-04-17T10:42:33.991626+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.011040+0000","last_change":"2024-04-17T10:42:34.010407+0000","last_active":"2024-04-17T10:42:34.011040+0000","last_peered":"2024-04-17T10:42:34.011040+0000","last_clean":"2024-04-17T10:42:34.011040+0000","last_became_active":"2024-04-17T10:42:34.009954+0000","last_became_peered":"2024-04-17T10:42:34.009954+0000","last_unstale":"2024-04-17T10:42:34.011040+0000","last_undegraded":"2024-04-17T10:42:34.011040+0000","last_fullsized":"2024-04-17T10:42:34.011040+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:44:50.147806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982157+0000","last_change":"2024-04-17T10:42:33.982157+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982157+0000","last_undegraded":"2024-04-17T10:42:33.982157+0000","last_fullsized":"2024-04-17T10:42:33.982157+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:33.996210+0000","last_change":"2024-04-17T10:42:33.991455+0000","last_active":"2024-04-17T10:42:33.996210+0000","last_peered":"2024-04-17T10:42:33.996210+0000","last_clean":"2024-04-17T10:42:33.996210+0000","last_became_active":"2024-04-17T10:42:33.990906+0000","last_became_peered":"2024-04-17T10:42:33.990906+0000","last_unstale":"2024-04-17T10:42:33.996210+0000","last_undegraded":"2024-04-17T10:42:33.996210+0000","last_fullsized":"2024-04-17T10:42:33.996210+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:13:46.065686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.984120+0000","last_change":"2024-04-17T10:42:33.984120+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.984120+0000","last_undegraded":"2024-04-17T10:42:33.984120+0000","last_fullsized":"2024-04-17T10:42:33.984120+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.016150+0000","last_change":"2024-04-17T10:42:34.007999+0000","last_active":"2024-04-17T10:42:34.016150+0000","last_peered":"2024-04-17T10:42:34.016150+0000","last_clean":"2024-04-17T10:42:34.016150+0000","last_became_active":"2024-04-17T10:42:34.007259+0000","last_became_peered":"2024-04-17T10:42:34.007259+0000","last_unstale":"2024-04-17T10:42:34.016150+0000","last_undegraded":"2024-04-17T10:42:34.016150+0000","last_fullsized":"2024-04-17T10:42:34.016150+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:10:36.224186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.995538+0000","last_change":"2024-04-17T10:42:33.995538+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.995538+0000","last_undegraded":"2024-04-17T10:42:33.995538+0000","last_fullsized":"2024-04-17T10:42:33.995538+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.002602+0000","last_change":"2024-04-17T10:42:34.002043+0000","last_active":"2024-04-17T10:42:34.002602+0000","last_peered":"2024-04-17T10:42:34.002602+0000","last_clean":"2024-04-17T10:42:34.002602+0000","last_became_active":"2024-04-17T10:42:34.001536+0000","last_became_peered":"2024-04-17T10:42:34.001536+0000","last_unstale":"2024-04-17T10:42:34.002602+0000","last_undegraded":"2024-04-17T10:42:34.002602+0000","last_fullsized":"2024-04-17T10:42:34.002602+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:10:25.368795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.998164+0000","last_change":"2024-04-17T10:42:33.998164+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.998164+0000","last_undegraded":"2024-04-17T10:42:33.998164+0000","last_fullsized":"2024-04-17T10:42:33.998164+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.003492+0000","last_change":"2024-04-17T10:42:34.003042+0000","last_active":"2024-04-17T10:42:34.003492+0000","last_peered":"2024-04-17T10:42:34.003492+0000","last_clean":"2024-04-17T10:42:34.003492+0000","last_became_active":"2024-04-17T10:42:34.002726+0000","last_became_peered":"2024-04-17T10:42:34.002726+0000","last_unstale":"2024-04-17T10:42:34.003492+0000","last_undegraded":"2024-04-17T10:42:34.003492+0000","last_fullsized":"2024-04-17T10:42:34.003492+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:34:51.462597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.979958+0000","last_change":"2024-04-17T10:42:33.979958+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.979958+0000","last_undegraded":"2024-04-17T10:42:33.979958+0000","last_fullsized":"2024-04-17T10:42:33.979958+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.018861+0000","last_change":"2024-04-17T10:42:34.012933+0000","last_active":"2024-04-17T10:42:34.018861+0000","last_peered":"2024-04-17T10:42:34.018861+0000","last_clean":"2024-04-17T10:42:34.018861+0000","last_became_active":"2024-04-17T10:42:34.012150+0000","last_became_peered":"2024-04-17T10:42:34.012150+0000","last_unstale":"2024-04-17T10:42:34.018861+0000","last_undegraded":"2024-04-17T10:42:34.018861+0000","last_fullsized":"2024-04-17T10:42:34.018861+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:13:46.065686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982616+0000","last_change":"2024-04-17T10:42:33.982616+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982616+0000","last_undegraded":"2024-04-17T10:42:33.982616+0000","last_fullsized":"2024-04-17T10:42:33.982616+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.016283+0000","last_change":"2024-04-17T10:42:34.012790+0000","last_active":"2024-04-17T10:42:34.016283+0000","last_peered":"2024-04-17T10:42:34.016283+0000","last_clean":"2024-04-17T10:42:34.016283+0000","last_became_active":"2024-04-17T10:42:34.012177+0000","last_became_peered":"2024-04-17T10:42:34.012177+0000","last_unstale":"2024-04-17T10:42:34.016283+0000","last_undegraded":"2024-04-17T10:42:34.016283+0000","last_fullsized":"2024-04-17T10:42:34.016283+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:42:38.783160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982460+0000","last_change":"2024-04-17T10:42:33.982460+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982460+0000","last_undegraded":"2024-04-17T10:42:33.982460+0000","last_fullsized":"2024-04-17T10:42:33.982460+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.015716+0000","last_change":"2024-04-17T10:42:34.015303+0000","last_active":"2024-04-17T10:42:34.015716+0000","last_peered":"2024-04-17T10:42:34.015716+0000","last_clean":"2024-04-17T10:42:34.015716+0000","last_became_active":"2024-04-17T10:42:34.014919+0000","last_became_peered":"2024-04-17T10:42:34.014919+0000","last_unstale":"2024-04-17T10:42:34.015716+0000","last_undegraded":"2024-04-17T10:42:34.015716+0000","last_fullsized":"2024-04-17T10:42:34.015716+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:26:39.567608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982097+0000","last_change":"2024-04-17T10:42:33.982097+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982097+0000","last_undegraded":"2024-04-17T10:42:33.982097+0000","last_fullsized":"2024-04-17T10:42:33.982097+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.013335+0000","last_change":"2024-04-17T10:42:34.009833+0000","last_active":"2024-04-17T10:42:34.013335+0000","last_peered":"2024-04-17T10:42:34.013335+0000","last_clean":"2024-04-17T10:42:34.013335+0000","last_became_active":"2024-04-17T10:42:34.005236+0000","last_became_peered":"2024-04-17T10:42:34.005236+0000","last_unstale":"2024-04-17T10:42:34.013335+0000","last_undegraded":"2024-04-17T10:42:34.013335+0000","last_fullsized":"2024-04-17T10:42:34.013335+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:44:30.520002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.979789+0000","last_change":"2024-04-17T10:42:33.979789+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.979789+0000","last_undegraded":"2024-04-17T10:42:33.979789+0000","last_fullsized":"2024-04-17T10:42:33.979789+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:33.988519+0000","last_change":"2024-04-17T10:42:33.985466+0000","last_active":"2024-04-17T10:42:33.988519+0000","last_peered":"2024-04-17T10:42:33.988519+0000","last_clean":"2024-04-17T10:42:33.988519+0000","last_became_active":"2024-04-17T10:42:33.984982+0000","last_became_peered":"2024-04-17T10:42:33.984982+0000","last_unstale":"2024-04-17T10:42:33.988519+0000","last_undegraded":"2024-04-17T10:42:33.988519+0000","last_fullsized":"2024-04-17T10:42:33.988519+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:41:05.001485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982429+0000","last_change":"2024-04-17T10:42:33.982429+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982429+0000","last_undegraded":"2024-04-17T10:42:33.982429+0000","last_fullsized":"2024-04-17T10:42:33.982429+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.016130+0000","last_change":"2024-04-17T10:42:34.015593+0000","last_active":"2024-04-17T10:42:34.016130+0000","last_peered":"2024-04-17T10:42:34.016130+0000","last_clean":"2024-04-17T10:42:34.016130+0000","last_became_active":"2024-04-17T10:42:34.015222+0000","last_became_peered":"2024-04-17T10:42:34.015222+0000","last_unstale":"2024-04-17T10:42:34.016130+0000","last_undegraded":"2024-04-17T10:42:34.016130+0000","last_fullsized":"2024-04-17T10:42:34.016130+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:44:06.265361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.992302+0000","last_change":"2024-04-17T10:42:33.992302+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.992302+0000","last_undegraded":"2024-04-17T10:42:33.992302+0000","last_fullsized":"2024-04-17T10:42:33.992302+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.026745+0000","last_change":"2024-04-17T10:42:34.021417+0000","last_active":"2024-04-17T10:42:34.026745+0000","last_peered":"2024-04-17T10:42:34.026745+0000","last_clean":"2024-04-17T10:42:34.026745+0000","last_became_active":"2024-04-17T10:42:34.020795+0000","last_became_peered":"2024-04-17T10:42:34.020795+0000","last_unstale":"2024-04-17T10:42:34.026745+0000","last_undegraded":"2024-04-17T10:42:34.026745+0000","last_fullsized":"2024-04-17T10:42:34.026745+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:10:25.368795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":246,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:33.992698+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:42:33.992698+0000","last_peered":"2024-04-17T10:42:33.992698+0000","last_clean":"2024-04-17T10:42:33.992698+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:42:33.992698+0000","last_undegraded":"2024-04-17T10:42:33.992698+0000","last_fullsized":"2024-04-17T10:42:33.992698+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574907,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44912,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326928,"statfs":{"total":96636764160,"available":96590774272,"internally_reserved":0,"allocated":1265664,"data_stored":868517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40160,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331680,"statfs":{"total":96636764160,"available":96595640320,"internally_reserved":0,"allocated":671744,"data_stored":278149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40164,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331676,"statfs":{"total":96636764160,"available":96595636224,"internally_reserved":0,"allocated":671744,"data_stored":278149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49884,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321956,"statfs":{"total":96636764160,"available":96585682944,"internally_reserved":0,"allocated":671744,"data_stored":278149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54044,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317796,"statfs":{"total":96636764160,"available":96581423104,"internally_reserved":0,"allocated":671744,"data_stored":278149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36012,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335828,"statfs":{"total":96636764160,"available":96599887872,"internally_reserved":0,"allocated":684032,"data_stored":284416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49900,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321940,"statfs":{"total":96636764160,"available":96585666560,"internally_reserved":0,"allocated":684032,"data_stored":284416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574906,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40752,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331088,"statfs":{"total":96636764160,"available":96595034112,"internally_reserved":0,"allocated":1277952,"data_stored":874784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:42:39.023 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:42:39.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.185+0000 7f0b94bf8640 1 Processor -- start 2024-04-17T10:42:39.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.185+0000 7f0b94bf8640 1 -- start start 2024-04-17T10:42:39.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.185+0000 7f0b94bf8640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0b900cf060 0x7f0b900b4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:39.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.185+0000 7f0b94bf8640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0b900b3990 0x7f0b90070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:39.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.185+0000 7f0b94bf8640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b900b2bd0 con 0x7f0b900b45a0 2024-04-17T10:42:39.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.185+0000 7f0b94bf8640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b900b2980 con 0x7f0b900b3990 2024-04-17T10:42:39.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.185+0000 7f0b94bf8640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b900cf440 con 0x7f0b900cf060 2024-04-17T10:42:39.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.185+0000 7f0b8ed9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0b900b45a0 0x7f0b9015d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57440/0 (socket says 172.21.15.83:57440) 2024-04-17T10:42:39.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.185+0000 7f0b8ed9d640 1 -- 172.21.15.83:0/541873979 learned_addr learned my addr 172.21.15.83:0/541873979 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:39.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.185+0000 7f0b8dd9b640 1 --2- 172.21.15.83:0/541873979 >> v2:172.21.15.83:3300/0 conn(0x7f0b900b3990 0x7f0b90070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:39.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.185+0000 7f0b8e59c640 1 --2- 172.21.15.83:0/541873979 >> v2:172.21.15.167:3300/0 conn(0x7f0b900cf060 0x7f0b900b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:39.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.185+0000 7f0b8d59a640 1 -- 172.21.15.83:0/541873979 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4081404580 0 0) 0x7f0b900b2bd0 con 0x7f0b900b45a0 2024-04-17T10:42:39.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.185+0000 7f0b8d59a640 1 -- 172.21.15.83:0/541873979 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b78003530 con 0x7f0b900b45a0 2024-04-17T10:42:39.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.185+0000 7f0b8d59a640 1 -- 172.21.15.83:0/541873979 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3026421148 0 0) 0x7f0b78003530 con 0x7f0b900b45a0 2024-04-17T10:42:39.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.185+0000 7f0b8dd9b640 1 -- 172.21.15.83:0/541873979 >> v2:172.21.15.167:3300/0 conn(0x7f0b900cf060 msgr2=0x7f0b900b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:39.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.185+0000 7f0b8dd9b640 1 --2- 172.21.15.83:0/541873979 >> v2:172.21.15.167:3300/0 conn(0x7f0b900cf060 0x7f0b900b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:39.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.185+0000 7f0b8dd9b640 1 -- 172.21.15.83:0/541873979 >> v1:172.21.15.83:6789/0 conn(0x7f0b900b45a0 legacy=0x7f0b9015d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:39.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.185+0000 7f0b8dd9b640 1 -- 172.21.15.83:0/541873979 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b900b2eb0 con 0x7f0b900b3990 2024-04-17T10:42:39.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.189+0000 7f0b8dd9b640 1 --2- 172.21.15.83:0/541873979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b900b3990 0x7f0b90070470 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f0b9006e6c0 tx=0x7f0b8401d740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf47d37b2bfe77a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:39.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.189+0000 7f0b8d59a640 1 -- 172.21.15.83:0/541873979 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b88001a00 con 0x7f0b900b3990 2024-04-17T10:42:39.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.189+0000 7f0b8d59a640 1 -- 172.21.15.83:0/541873979 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b840a9450 con 0x7f0b900b3990 2024-04-17T10:42:39.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.189+0000 7f0b8d59a640 1 -- 172.21.15.83:0/541873979 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8400b040 con 0x7f0b900b3990 2024-04-17T10:42:39.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.189+0000 7f0b94bf8640 1 -- 172.21.15.83:0/541873979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b900b3990 msgr2=0x7f0b90070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:39.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.189+0000 7f0b94bf8640 1 --2- 172.21.15.83:0/541873979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b900b3990 0x7f0b90070470 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f0b9006e6c0 tx=0x7f0b8401d740 comp rx=0 tx=0).stop 2024-04-17T10:42:39.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.189+0000 7f0b94bf8640 1 -- 172.21.15.83:0/541873979 shutdown_connections 2024-04-17T10:42:39.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.189+0000 7f0b94bf8640 1 --2- 172.21.15.83:0/541873979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b900b3990 0x7f0b90070470 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:39.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.189+0000 7f0b94bf8640 1 --2- 172.21.15.83:0/541873979 >> v2:172.21.15.167:3300/0 conn(0x7f0b900cf060 0x7f0b900b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:39.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.189+0000 7f0b94bf8640 1 -- 172.21.15.83:0/541873979 >> 172.21.15.83:0/541873979 conn(0x7f0b9006abe0 msgr2=0x7f0b9006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:39.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.189+0000 7f0b94bf8640 1 -- 172.21.15.83:0/541873979 shutdown_connections 2024-04-17T10:42:39.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.189+0000 7f0b94bf8640 1 -- 172.21.15.83:0/541873979 wait complete. 2024-04-17T10:42:39.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.189+0000 7f0b94bf8640 1 Processor -- start 2024-04-17T10:42:39.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.189+0000 7f0b94bf8640 1 -- start start 2024-04-17T10:42:39.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.193+0000 7f0b94bf8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b900b3990 0x7f0b90138080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:39.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.193+0000 7f0b94bf8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b900b45a0 0x7f0b90154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:39.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.193+0000 7f0b94bf8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b900cf060 0x7f0b90155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:39.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.193+0000 7f0b94bf8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9015e280 con 0x7f0b900b45a0 2024-04-17T10:42:39.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.193+0000 7f0b94bf8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9015e100 con 0x7f0b900b3990 2024-04-17T10:42:39.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.193+0000 7f0b94bf8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9015e400 con 0x7f0b900cf060 2024-04-17T10:42:39.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.193+0000 7f0b8dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b900b45a0 0x7f0b90154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:39.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.193+0000 7f0b8dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b900b45a0 0x7f0b90154ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36786/0 (socket says 172.21.15.83:36786) 2024-04-17T10:42:39.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.193+0000 7f0b8dd9b640 1 -- 172.21.15.83:0/3679440638 learned_addr learned my addr 172.21.15.83:0/3679440638 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:39.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.193+0000 7f0b8dd9b640 1 -- 172.21.15.83:0/3679440638 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b900cf060 msgr2=0x7f0b90155000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:39.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.193+0000 7f0b8dd9b640 1 --2- 172.21.15.83:0/3679440638 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b900cf060 0x7f0b90155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:39.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.193+0000 7f0b8dd9b640 1 -- 172.21.15.83:0/3679440638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b900b3990 msgr2=0x7f0b90138080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:39.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.193+0000 7f0b8dd9b640 1 --2- 172.21.15.83:0/3679440638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b900b3990 0x7f0b90138080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:39.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.193+0000 7f0b8dd9b640 1 -- 172.21.15.83:0/3679440638 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b90155bd0 con 0x7f0b900b45a0 2024-04-17T10:42:39.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.193+0000 7f0b8dd9b640 1 --2- 172.21.15.83:0/3679440638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b900b45a0 0x7f0b90154ac0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f0b8401b370 tx=0x7f0b8401a4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:39.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.193+0000 7f0b7effd640 1 -- 172.21.15.83:0/3679440638 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b840c0070 con 0x7f0b900b45a0 2024-04-17T10:42:39.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.193+0000 7f0b7effd640 1 -- 172.21.15.83:0/3679440638 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b840bb410 con 0x7f0b900b45a0 2024-04-17T10:42:39.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.193+0000 7f0b94bf8640 1 -- 172.21.15.83:0/3679440638 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b90156140 con 0x7f0b900b45a0 2024-04-17T10:42:39.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.193+0000 7f0b7effd640 1 -- 172.21.15.83:0/3679440638 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b84006e20 con 0x7f0b900b45a0 2024-04-17T10:42:39.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.193+0000 7f0b94bf8640 1 -- 172.21.15.83:0/3679440638 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b901558f0 con 0x7f0b900b45a0 2024-04-17T10:42:39.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.193+0000 7f0b94bf8640 1 -- 172.21.15.83:0/3679440638 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b90074f90 con 0x7f0b900b45a0 2024-04-17T10:42:39.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.193+0000 7f0b7effd640 1 -- 172.21.15.83:0/3679440638 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0b840c3070 con 0x7f0b900b45a0 2024-04-17T10:42:39.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.193+0000 7f0b7effd640 1 --2- 172.21.15.83:0/3679440638 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0b70066f60 0x7f0b70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:39.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.193+0000 7f0b7effd640 1 -- 172.21.15.83:0/3679440638 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b84120820 con 0x7f0b900b45a0 2024-04-17T10:42:39.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.193+0000 7f0b8e59c640 1 --2- 172.21.15.83:0/3679440638 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0b70066f60 0x7f0b70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:39.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.197+0000 7f0b8e59c640 1 --2- 172.21.15.83:0/3679440638 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0b70066f60 0x7f0b70069420 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0b80006aa0 tx=0x7f0b80006a30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:39.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.197+0000 7f0b7effd640 1 -- 172.21.15.83:0/3679440638 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0b840ca070 con 0x7f0b900b45a0 2024-04-17T10:42:39.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.353+0000 7f0b94bf8640 1 -- 172.21.15.83:0/3679440638 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0b9015d770 con 0x7f0b70066f60 2024-04-17T10:42:39.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.361+0000 7f0b7effd640 1 -- 172.21.15.83:0/3679440638 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246636 (secure 0 0 0) 0x7f0b9015d770 con 0x7f0b70066f60 2024-04-17T10:42:39.362 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:39.363 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:42:39.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.361+0000 7f0b94bf8640 1 -- 172.21.15.83:0/3679440638 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0b70066f60 msgr2=0x7f0b70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:39.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.361+0000 7f0b94bf8640 1 --2- 172.21.15.83:0/3679440638 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0b70066f60 0x7f0b70069420 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0b80006aa0 tx=0x7f0b80006a30 comp rx=0 tx=0).stop 2024-04-17T10:42:39.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.361+0000 7f0b94bf8640 1 -- 172.21.15.83:0/3679440638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b900b45a0 msgr2=0x7f0b90154ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:39.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.361+0000 7f0b94bf8640 1 --2- 172.21.15.83:0/3679440638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b900b45a0 0x7f0b90154ac0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f0b8401b370 tx=0x7f0b8401a4e0 comp rx=0 tx=0).stop 2024-04-17T10:42:39.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.361+0000 7f0b94bf8640 1 -- 172.21.15.83:0/3679440638 shutdown_connections 2024-04-17T10:42:39.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.361+0000 7f0b94bf8640 1 --2- 172.21.15.83:0/3679440638 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0b70066f60 0x7f0b70069420 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:39.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.361+0000 7f0b94bf8640 1 --2- 172.21.15.83:0/3679440638 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b900cf060 0x7f0b90155000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:39.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.361+0000 7f0b94bf8640 1 --2- 172.21.15.83:0/3679440638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b900b45a0 0x7f0b90154ac0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:39.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.361+0000 7f0b94bf8640 1 --2- 172.21.15.83:0/3679440638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b900b3990 0x7f0b90138080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:39.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.361+0000 7f0b94bf8640 1 -- 172.21.15.83:0/3679440638 >> 172.21.15.83:0/3679440638 conn(0x7f0b9006abe0 msgr2=0x7f0b90073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:39.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.361+0000 7f0b94bf8640 1 -- 172.21.15.83:0/3679440638 shutdown_connections 2024-04-17T10:42:39.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.361+0000 7f0b94bf8640 1 -- 172.21.15.83:0/3679440638 wait complete. 2024-04-17T10:42:39.381 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2024-04-17T10:42:38.259280+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":355828,"kb_used_data":6444,"kb_used_omap":12,"kb_used_meta":349235,"kb_avail":754618892,"statfs":{"total":773094113280,"available":772729745408,"internally_reserved":0,"allocated":6598656,"data_stored":3424729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":357617200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.296682"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.988554+0000","last_change":"2024-04-17T10:42:33.988554+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.988554+0000","last_undegraded":"2024-04-17T10:42:33.988554+0000","last_fullsized":"2024-04-17T10:42:33.988554+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.992506+0000","last_change":"2024-04-17T10:42:33.992506+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.992506+0000","last_undegraded":"2024-04-17T10:42:33.992506+0000","last_fullsized":"2024-04-17T10:42:33.992506+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.991262+0000","last_change":"2024-04-17T10:42:33.991262+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.991262+0000","last_undegraded":"2024-04-17T10:42:33.991262+0000","last_fullsized":"2024-04-17T10:42:33.991262+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.999359+0000","last_change":"2024-04-17T10:42:33.999359+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.999359+0000","last_undegraded":"2024-04-17T10:42:33.999359+0000","last_fullsized":"2024-04-17T10:42:33.999359+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.985818+0000","last_change":"2024-04-17T10:42:33.985818+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.985818+0000","last_undegraded":"2024-04-17T10:42:33.985818+0000","last_fullsized":"2024-04-17T10:42:33.985818+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982596+0000","last_change":"2024-04-17T10:42:33.982596+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982596+0000","last_undegraded":"2024-04-17T10:42:33.982596+0000","last_fullsized":"2024-04-17T10:42:33.982596+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.984830+0000","last_change":"2024-04-17T10:42:33.984830+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.984830+0000","last_undegraded":"2024-04-17T10:42:33.984830+0000","last_fullsized":"2024-04-17T10:42:33.984830+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.994100+0000","last_change":"2024-04-17T10:42:33.994100+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.994100+0000","last_undegraded":"2024-04-17T10:42:33.994100+0000","last_fullsized":"2024-04-17T10:42:33.994100+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982038+0000","last_change":"2024-04-17T10:42:33.982038+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982038+0000","last_undegraded":"2024-04-17T10:42:33.982038+0000","last_fullsized":"2024-04-17T10:42:33.982038+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.980127+0000","last_change":"2024-04-17T10:42:33.980127+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.980127+0000","last_undegraded":"2024-04-17T10:42:33.980127+0000","last_fullsized":"2024-04-17T10:42:33.980127+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.980448+0000","last_change":"2024-04-17T10:42:33.980448+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.980448+0000","last_undegraded":"2024-04-17T10:42:33.980448+0000","last_fullsized":"2024-04-17T10:42:33.980448+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.995007+0000","last_change":"2024-04-17T10:42:33.995007+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.995007+0000","last_undegraded":"2024-04-17T10:42:33.995007+0000","last_fullsized":"2024-04-17T10:42:33.995007+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.990877+0000","last_change":"2024-04-17T10:42:33.990877+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.990877+0000","last_undegraded":"2024-04-17T10:42:33.990877+0000","last_fullsized":"2024-04-17T10:42:33.990877+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.983487+0000","last_change":"2024-04-17T10:42:33.983487+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.983487+0000","last_undegraded":"2024-04-17T10:42:33.983487+0000","last_fullsized":"2024-04-17T10:42:33.983487+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982041+0000","last_change":"2024-04-17T10:42:33.982041+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982041+0000","last_undegraded":"2024-04-17T10:42:33.982041+0000","last_fullsized":"2024-04-17T10:42:33.982041+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.985005+0000","last_change":"2024-04-17T10:42:33.985005+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.985005+0000","last_undegraded":"2024-04-17T10:42:33.985005+0000","last_fullsized":"2024-04-17T10:42:33.985005+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.994197+0000","last_change":"2024-04-17T10:42:33.994197+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.994197+0000","last_undegraded":"2024-04-17T10:42:33.994197+0000","last_fullsized":"2024-04-17T10:42:33.994197+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.988273+0000","last_change":"2024-04-17T10:42:33.988273+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.988273+0000","last_undegraded":"2024-04-17T10:42:33.988273+0000","last_fullsized":"2024-04-17T10:42:33.988273+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.985480+0000","last_change":"2024-04-17T10:42:33.985480+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.985480+0000","last_undegraded":"2024-04-17T10:42:33.985480+0000","last_fullsized":"2024-04-17T10:42:33.985480+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982918+0000","last_change":"2024-04-17T10:42:33.982918+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982918+0000","last_undegraded":"2024-04-17T10:42:33.982918+0000","last_fullsized":"2024-04-17T10:42:33.982918+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.984586+0000","last_change":"2024-04-17T10:42:33.984586+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.984586+0000","last_undegraded":"2024-04-17T10:42:33.984586+0000","last_fullsized":"2024-04-17T10:42:33.984586+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.994276+0000","last_change":"2024-04-17T10:42:33.994276+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.994276+0000","last_undegraded":"2024-04-17T10:42:33.994276+0000","last_fullsized":"2024-04-17T10:42:33.994276+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.983711+0000","last_change":"2024-04-17T10:42:33.983711+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.983711+0000","last_undegraded":"2024-04-17T10:42:33.983711+0000","last_fullsized":"2024-04-17T10:42:33.983711+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.992224+0000","last_change":"2024-04-17T10:42:33.992224+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.992224+0000","last_undegraded":"2024-04-17T10:42:33.992224+0000","last_fullsized":"2024-04-17T10:42:33.992224+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.988095+0000","last_change":"2024-04-17T10:42:33.988095+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.988095+0000","last_undegraded":"2024-04-17T10:42:33.988095+0000","last_fullsized":"2024-04-17T10:42:33.988095+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.996585+0000","last_change":"2024-04-17T10:42:33.996585+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.996585+0000","last_undegraded":"2024-04-17T10:42:33.996585+0000","last_fullsized":"2024-04-17T10:42:33.996585+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.987477+0000","last_change":"2024-04-17T10:42:33.987477+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.987477+0000","last_undegraded":"2024-04-17T10:42:33.987477+0000","last_fullsized":"2024-04-17T10:42:33.987477+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.985065+0000","last_change":"2024-04-17T10:42:33.985065+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.985065+0000","last_undegraded":"2024-04-17T10:42:33.985065+0000","last_fullsized":"2024-04-17T10:42:33.985065+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.991668+0000","last_change":"2024-04-17T10:42:33.991668+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.991668+0000","last_undegraded":"2024-04-17T10:42:33.991668+0000","last_fullsized":"2024-04-17T10:42:33.991668+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.985954+0000","last_change":"2024-04-17T10:42:33.985954+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.985954+0000","last_undegraded":"2024-04-17T10:42:33.985954+0000","last_fullsized":"2024-04-17T10:42:33.985954+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.984858+0000","last_change":"2024-04-17T10:42:33.984858+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.984858+0000","last_undegraded":"2024-04-17T10:42:33.984858+0000","last_fullsized":"2024-04-17T10:42:33.984858+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.984313+0000","last_change":"2024-04-17T10:42:33.984313+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.984313+0000","last_undegraded":"2024-04-17T10:42:33.984313+0000","last_fullsized":"2024-04-17T10:42:33.984313+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.987311+0000","last_change":"2024-04-17T10:42:33.987311+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.987311+0000","last_undegraded":"2024-04-17T10:42:33.987311+0000","last_fullsized":"2024-04-17T10:42:33.987311+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.015861+0000","last_change":"2024-04-17T10:42:34.015019+0000","last_active":"2024-04-17T10:42:34.015861+0000","last_peered":"2024-04-17T10:42:34.015861+0000","last_clean":"2024-04-17T10:42:34.015861+0000","last_became_active":"2024-04-17T10:42:34.014716+0000","last_became_peered":"2024-04-17T10:42:34.014716+0000","last_unstale":"2024-04-17T10:42:34.015861+0000","last_undegraded":"2024-04-17T10:42:34.015861+0000","last_fullsized":"2024-04-17T10:42:34.015861+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:26:57.986949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.980974+0000","last_change":"2024-04-17T10:42:33.980974+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.980974+0000","last_undegraded":"2024-04-17T10:42:33.980974+0000","last_fullsized":"2024-04-17T10:42:33.980974+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.006810+0000","last_change":"2024-04-17T10:42:34.006170+0000","last_active":"2024-04-17T10:42:34.006810+0000","last_peered":"2024-04-17T10:42:34.006810+0000","last_clean":"2024-04-17T10:42:34.006810+0000","last_became_active":"2024-04-17T10:42:34.005757+0000","last_became_peered":"2024-04-17T10:42:34.005757+0000","last_unstale":"2024-04-17T10:42:34.006810+0000","last_undegraded":"2024-04-17T10:42:34.006810+0000","last_fullsized":"2024-04-17T10:42:34.006810+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:25:59.364150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982188+0000","last_change":"2024-04-17T10:42:33.982188+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982188+0000","last_undegraded":"2024-04-17T10:42:33.982188+0000","last_fullsized":"2024-04-17T10:42:33.982188+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:33.996540+0000","last_change":"2024-04-17T10:42:33.993426+0000","last_active":"2024-04-17T10:42:33.996540+0000","last_peered":"2024-04-17T10:42:33.996540+0000","last_clean":"2024-04-17T10:42:33.996540+0000","last_became_active":"2024-04-17T10:42:33.992971+0000","last_became_peered":"2024-04-17T10:42:33.992971+0000","last_unstale":"2024-04-17T10:42:33.996540+0000","last_undegraded":"2024-04-17T10:42:33.996540+0000","last_fullsized":"2024-04-17T10:42:33.996540+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:42:12.531359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.987048+0000","last_change":"2024-04-17T10:42:33.987048+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.987048+0000","last_undegraded":"2024-04-17T10:42:33.987048+0000","last_fullsized":"2024-04-17T10:42:33.987048+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.021952+0000","last_change":"2024-04-17T10:42:34.018234+0000","last_active":"2024-04-17T10:42:34.021952+0000","last_peered":"2024-04-17T10:42:34.021952+0000","last_clean":"2024-04-17T10:42:34.021952+0000","last_became_active":"2024-04-17T10:42:34.017497+0000","last_became_peered":"2024-04-17T10:42:34.017497+0000","last_unstale":"2024-04-17T10:42:34.021952+0000","last_undegraded":"2024-04-17T10:42:34.021952+0000","last_fullsized":"2024-04-17T10:42:34.021952+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:32:27.523313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.992516+0000","last_change":"2024-04-17T10:42:33.992516+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.992516+0000","last_undegraded":"2024-04-17T10:42:33.992516+0000","last_fullsized":"2024-04-17T10:42:33.992516+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.013446+0000","last_change":"2024-04-17T10:42:34.004483+0000","last_active":"2024-04-17T10:42:34.013446+0000","last_peered":"2024-04-17T10:42:34.013446+0000","last_clean":"2024-04-17T10:42:34.013446+0000","last_became_active":"2024-04-17T10:42:33.998883+0000","last_became_peered":"2024-04-17T10:42:33.998883+0000","last_unstale":"2024-04-17T10:42:34.013446+0000","last_undegraded":"2024-04-17T10:42:34.013446+0000","last_fullsized":"2024-04-17T10:42:34.013446+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:42:12.531359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.992432+0000","last_change":"2024-04-17T10:42:33.992432+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.992432+0000","last_undegraded":"2024-04-17T10:42:33.992432+0000","last_fullsized":"2024-04-17T10:42:33.992432+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.002170+0000","last_change":"2024-04-17T10:42:34.000321+0000","last_active":"2024-04-17T10:42:34.002170+0000","last_peered":"2024-04-17T10:42:34.002170+0000","last_clean":"2024-04-17T10:42:34.002170+0000","last_became_active":"2024-04-17T10:42:34.000062+0000","last_became_peered":"2024-04-17T10:42:34.000062+0000","last_unstale":"2024-04-17T10:42:34.002170+0000","last_undegraded":"2024-04-17T10:42:34.002170+0000","last_fullsized":"2024-04-17T10:42:34.002170+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:58:18.204240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.991126+0000","last_change":"2024-04-17T10:42:33.991126+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.991126+0000","last_undegraded":"2024-04-17T10:42:33.991126+0000","last_fullsized":"2024-04-17T10:42:33.991126+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.021947+0000","last_change":"2024-04-17T10:42:34.018055+0000","last_active":"2024-04-17T10:42:34.021947+0000","last_peered":"2024-04-17T10:42:34.021947+0000","last_clean":"2024-04-17T10:42:34.021947+0000","last_became_active":"2024-04-17T10:42:34.016719+0000","last_became_peered":"2024-04-17T10:42:34.016719+0000","last_unstale":"2024-04-17T10:42:34.021947+0000","last_undegraded":"2024-04-17T10:42:34.021947+0000","last_fullsized":"2024-04-17T10:42:34.021947+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:20:46.478088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.990753+0000","last_change":"2024-04-17T10:42:33.990753+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.990753+0000","last_undegraded":"2024-04-17T10:42:33.990753+0000","last_fullsized":"2024-04-17T10:42:33.990753+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.013153+0000","last_change":"2024-04-17T10:42:34.008774+0000","last_active":"2024-04-17T10:42:34.013153+0000","last_peered":"2024-04-17T10:42:34.013153+0000","last_clean":"2024-04-17T10:42:34.013153+0000","last_became_active":"2024-04-17T10:42:34.005106+0000","last_became_peered":"2024-04-17T10:42:34.005106+0000","last_unstale":"2024-04-17T10:42:34.013153+0000","last_undegraded":"2024-04-17T10:42:34.013153+0000","last_fullsized":"2024-04-17T10:42:34.013153+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:58:18.204240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.984503+0000","last_change":"2024-04-17T10:42:33.984503+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.984503+0000","last_undegraded":"2024-04-17T10:42:33.984503+0000","last_fullsized":"2024-04-17T10:42:33.984503+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.014222+0000","last_change":"2024-04-17T10:42:34.006222+0000","last_active":"2024-04-17T10:42:34.014222+0000","last_peered":"2024-04-17T10:42:34.014222+0000","last_clean":"2024-04-17T10:42:34.014222+0000","last_became_active":"2024-04-17T10:42:34.005353+0000","last_became_peered":"2024-04-17T10:42:34.005353+0000","last_unstale":"2024-04-17T10:42:34.014222+0000","last_undegraded":"2024-04-17T10:42:34.014222+0000","last_fullsized":"2024-04-17T10:42:34.014222+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:20:46.478088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.985320+0000","last_change":"2024-04-17T10:42:33.985320+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.985320+0000","last_undegraded":"2024-04-17T10:42:33.985320+0000","last_fullsized":"2024-04-17T10:42:33.985320+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.000258+0000","last_change":"2024-04-17T10:42:33.995027+0000","last_active":"2024-04-17T10:42:34.000258+0000","last_peered":"2024-04-17T10:42:34.000258+0000","last_clean":"2024-04-17T10:42:34.000258+0000","last_became_active":"2024-04-17T10:42:33.994748+0000","last_became_peered":"2024-04-17T10:42:33.994748+0000","last_unstale":"2024-04-17T10:42:34.000258+0000","last_undegraded":"2024-04-17T10:42:34.000258+0000","last_fullsized":"2024-04-17T10:42:34.000258+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:41:52.745779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.981639+0000","last_change":"2024-04-17T10:42:33.981639+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.981639+0000","last_undegraded":"2024-04-17T10:42:33.981639+0000","last_fullsized":"2024-04-17T10:42:33.981639+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.016297+0000","last_change":"2024-04-17T10:42:34.010793+0000","last_active":"2024-04-17T10:42:34.016297+0000","last_peered":"2024-04-17T10:42:34.016297+0000","last_clean":"2024-04-17T10:42:34.016297+0000","last_became_active":"2024-04-17T10:42:34.009786+0000","last_became_peered":"2024-04-17T10:42:34.009786+0000","last_unstale":"2024-04-17T10:42:34.016297+0000","last_undegraded":"2024-04-17T10:42:34.016297+0000","last_fullsized":"2024-04-17T10:42:34.016297+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:32:27.523313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.990052+0000","last_change":"2024-04-17T10:42:33.990052+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.990052+0000","last_undegraded":"2024-04-17T10:42:33.990052+0000","last_fullsized":"2024-04-17T10:42:33.990052+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.013294+0000","last_change":"2024-04-17T10:42:34.008738+0000","last_active":"2024-04-17T10:42:34.013294+0000","last_peered":"2024-04-17T10:42:34.013294+0000","last_clean":"2024-04-17T10:42:34.013294+0000","last_became_active":"2024-04-17T10:42:34.004730+0000","last_became_peered":"2024-04-17T10:42:34.004730+0000","last_unstale":"2024-04-17T10:42:34.013294+0000","last_undegraded":"2024-04-17T10:42:34.013294+0000","last_fullsized":"2024-04-17T10:42:34.013294+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:14:14.289936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.993209+0000","last_change":"2024-04-17T10:42:33.993209+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.993209+0000","last_undegraded":"2024-04-17T10:42:33.993209+0000","last_fullsized":"2024-04-17T10:42:33.993209+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.000845+0000","last_change":"2024-04-17T10:42:34.000022+0000","last_active":"2024-04-17T10:42:34.000845+0000","last_peered":"2024-04-17T10:42:34.000845+0000","last_clean":"2024-04-17T10:42:34.000845+0000","last_became_active":"2024-04-17T10:42:33.999505+0000","last_became_peered":"2024-04-17T10:42:33.999505+0000","last_unstale":"2024-04-17T10:42:34.000845+0000","last_undegraded":"2024-04-17T10:42:34.000845+0000","last_fullsized":"2024-04-17T10:42:34.000845+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:01:33.910102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.983396+0000","last_change":"2024-04-17T10:42:33.983396+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.983396+0000","last_undegraded":"2024-04-17T10:42:33.983396+0000","last_fullsized":"2024-04-17T10:42:33.983396+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.015295+0000","last_change":"2024-04-17T10:42:34.010051+0000","last_active":"2024-04-17T10:42:34.015295+0000","last_peered":"2024-04-17T10:42:34.015295+0000","last_clean":"2024-04-17T10:42:34.015295+0000","last_became_active":"2024-04-17T10:42:34.009227+0000","last_became_peered":"2024-04-17T10:42:34.009227+0000","last_unstale":"2024-04-17T10:42:34.015295+0000","last_undegraded":"2024-04-17T10:42:34.015295+0000","last_fullsized":"2024-04-17T10:42:34.015295+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:41:05.001485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982143+0000","last_change":"2024-04-17T10:42:33.982143+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982143+0000","last_undegraded":"2024-04-17T10:42:33.982143+0000","last_fullsized":"2024-04-17T10:42:33.982143+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.023975+0000","last_change":"2024-04-17T10:42:34.019631+0000","last_active":"2024-04-17T10:42:34.023975+0000","last_peered":"2024-04-17T10:42:34.023975+0000","last_clean":"2024-04-17T10:42:34.023975+0000","last_became_active":"2024-04-17T10:42:34.018768+0000","last_became_peered":"2024-04-17T10:42:34.018768+0000","last_unstale":"2024-04-17T10:42:34.023975+0000","last_undegraded":"2024-04-17T10:42:34.023975+0000","last_fullsized":"2024-04-17T10:42:34.023975+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:37:08.284805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982708+0000","last_change":"2024-04-17T10:42:33.982708+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982708+0000","last_undegraded":"2024-04-17T10:42:33.982708+0000","last_fullsized":"2024-04-17T10:42:33.982708+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.019298+0000","last_change":"2024-04-17T10:42:34.015072+0000","last_active":"2024-04-17T10:42:34.019298+0000","last_peered":"2024-04-17T10:42:34.019298+0000","last_clean":"2024-04-17T10:42:34.019298+0000","last_became_active":"2024-04-17T10:42:34.014406+0000","last_became_peered":"2024-04-17T10:42:34.014406+0000","last_unstale":"2024-04-17T10:42:34.019298+0000","last_undegraded":"2024-04-17T10:42:34.019298+0000","last_fullsized":"2024-04-17T10:42:34.019298+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:37:08.284805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.984738+0000","last_change":"2024-04-17T10:42:33.984738+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.984738+0000","last_undegraded":"2024-04-17T10:42:33.984738+0000","last_fullsized":"2024-04-17T10:42:33.984738+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.000855+0000","last_change":"2024-04-17T10:42:33.998582+0000","last_active":"2024-04-17T10:42:34.000855+0000","last_peered":"2024-04-17T10:42:34.000855+0000","last_clean":"2024-04-17T10:42:34.000855+0000","last_became_active":"2024-04-17T10:42:33.997341+0000","last_became_peered":"2024-04-17T10:42:33.997341+0000","last_unstale":"2024-04-17T10:42:34.000855+0000","last_undegraded":"2024-04-17T10:42:34.000855+0000","last_fullsized":"2024-04-17T10:42:34.000855+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:03:51.699883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.999265+0000","last_change":"2024-04-17T10:42:33.999265+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.999265+0000","last_undegraded":"2024-04-17T10:42:33.999265+0000","last_fullsized":"2024-04-17T10:42:33.999265+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:33.999215+0000","last_change":"2024-04-17T10:42:33.995762+0000","last_active":"2024-04-17T10:42:33.999215+0000","last_peered":"2024-04-17T10:42:33.999215+0000","last_clean":"2024-04-17T10:42:33.999215+0000","last_became_active":"2024-04-17T10:42:33.995562+0000","last_became_peered":"2024-04-17T10:42:33.995562+0000","last_unstale":"2024-04-17T10:42:33.999215+0000","last_undegraded":"2024-04-17T10:42:33.999215+0000","last_fullsized":"2024-04-17T10:42:33.999215+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:14:14.289936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982369+0000","last_change":"2024-04-17T10:42:33.982369+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982369+0000","last_undegraded":"2024-04-17T10:42:33.982369+0000","last_fullsized":"2024-04-17T10:42:33.982369+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.021531+0000","last_change":"2024-04-17T10:42:34.017202+0000","last_active":"2024-04-17T10:42:34.021531+0000","last_peered":"2024-04-17T10:42:34.021531+0000","last_clean":"2024-04-17T10:42:34.021531+0000","last_became_active":"2024-04-17T10:42:34.016613+0000","last_became_peered":"2024-04-17T10:42:34.016613+0000","last_unstale":"2024-04-17T10:42:34.021531+0000","last_undegraded":"2024-04-17T10:42:34.021531+0000","last_fullsized":"2024-04-17T10:42:34.021531+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:09:32.979808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.985866+0000","last_change":"2024-04-17T10:42:33.985866+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.985866+0000","last_undegraded":"2024-04-17T10:42:33.985866+0000","last_fullsized":"2024-04-17T10:42:33.985866+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.002835+0000","last_change":"2024-04-17T10:42:34.002003+0000","last_active":"2024-04-17T10:42:34.002835+0000","last_peered":"2024-04-17T10:42:34.002835+0000","last_clean":"2024-04-17T10:42:34.002835+0000","last_became_active":"2024-04-17T10:42:34.001413+0000","last_became_peered":"2024-04-17T10:42:34.001413+0000","last_unstale":"2024-04-17T10:42:34.002835+0000","last_undegraded":"2024-04-17T10:42:34.002835+0000","last_fullsized":"2024-04-17T10:42:34.002835+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:44:30.520002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.991626+0000","last_change":"2024-04-17T10:42:33.991626+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.991626+0000","last_undegraded":"2024-04-17T10:42:33.991626+0000","last_fullsized":"2024-04-17T10:42:33.991626+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.011040+0000","last_change":"2024-04-17T10:42:34.010407+0000","last_active":"2024-04-17T10:42:34.011040+0000","last_peered":"2024-04-17T10:42:34.011040+0000","last_clean":"2024-04-17T10:42:34.011040+0000","last_became_active":"2024-04-17T10:42:34.009954+0000","last_became_peered":"2024-04-17T10:42:34.009954+0000","last_unstale":"2024-04-17T10:42:34.011040+0000","last_undegraded":"2024-04-17T10:42:34.011040+0000","last_fullsized":"2024-04-17T10:42:34.011040+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:44:50.147806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982157+0000","last_change":"2024-04-17T10:42:33.982157+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982157+0000","last_undegraded":"2024-04-17T10:42:33.982157+0000","last_fullsized":"2024-04-17T10:42:33.982157+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:33.996210+0000","last_change":"2024-04-17T10:42:33.991455+0000","last_active":"2024-04-17T10:42:33.996210+0000","last_peered":"2024-04-17T10:42:33.996210+0000","last_clean":"2024-04-17T10:42:33.996210+0000","last_became_active":"2024-04-17T10:42:33.990906+0000","last_became_peered":"2024-04-17T10:42:33.990906+0000","last_unstale":"2024-04-17T10:42:33.996210+0000","last_undegraded":"2024-04-17T10:42:33.996210+0000","last_fullsized":"2024-04-17T10:42:33.996210+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:13:46.065686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.984120+0000","last_change":"2024-04-17T10:42:33.984120+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.984120+0000","last_undegraded":"2024-04-17T10:42:33.984120+0000","last_fullsized":"2024-04-17T10:42:33.984120+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.016150+0000","last_change":"2024-04-17T10:42:34.007999+0000","last_active":"2024-04-17T10:42:34.016150+0000","last_peered":"2024-04-17T10:42:34.016150+0000","last_clean":"2024-04-17T10:42:34.016150+0000","last_became_active":"2024-04-17T10:42:34.007259+0000","last_became_peered":"2024-04-17T10:42:34.007259+0000","last_unstale":"2024-04-17T10:42:34.016150+0000","last_undegraded":"2024-04-17T10:42:34.016150+0000","last_fullsized":"2024-04-17T10:42:34.016150+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:10:36.224186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.995538+0000","last_change":"2024-04-17T10:42:33.995538+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.995538+0000","last_undegraded":"2024-04-17T10:42:33.995538+0000","last_fullsized":"2024-04-17T10:42:33.995538+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.002602+0000","last_change":"2024-04-17T10:42:34.002043+0000","last_active":"2024-04-17T10:42:34.002602+0000","last_peered":"2024-04-17T10:42:34.002602+0000","last_clean":"2024-04-17T10:42:34.002602+0000","last_became_active":"2024-04-17T10:42:34.001536+0000","last_became_peered":"2024-04-17T10:42:34.001536+0000","last_unstale":"2024-04-17T10:42:34.002602+0000","last_undegraded":"2024-04-17T10:42:34.002602+0000","last_fullsized":"2024-04-17T10:42:34.002602+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:10:25.368795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.998164+0000","last_change":"2024-04-17T10:42:33.998164+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.998164+0000","last_undegraded":"2024-04-17T10:42:33.998164+0000","last_fullsized":"2024-04-17T10:42:33.998164+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.003492+0000","last_change":"2024-04-17T10:42:34.003042+0000","last_active":"2024-04-17T10:42:34.003492+0000","last_peered":"2024-04-17T10:42:34.003492+0000","last_clean":"2024-04-17T10:42:34.003492+0000","last_became_active":"2024-04-17T10:42:34.002726+0000","last_became_peered":"2024-04-17T10:42:34.002726+0000","last_unstale":"2024-04-17T10:42:34.003492+0000","last_undegraded":"2024-04-17T10:42:34.003492+0000","last_fullsized":"2024-04-17T10:42:34.003492+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:34:51.462597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.979958+0000","last_change":"2024-04-17T10:42:33.979958+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.979958+0000","last_undegraded":"2024-04-17T10:42:33.979958+0000","last_fullsized":"2024-04-17T10:42:33.979958+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.018861+0000","last_change":"2024-04-17T10:42:34.012933+0000","last_active":"2024-04-17T10:42:34.018861+0000","last_peered":"2024-04-17T10:42:34.018861+0000","last_clean":"2024-04-17T10:42:34.018861+0000","last_became_active":"2024-04-17T10:42:34.012150+0000","last_became_peered":"2024-04-17T10:42:34.012150+0000","last_unstale":"2024-04-17T10:42:34.018861+0000","last_undegraded":"2024-04-17T10:42:34.018861+0000","last_fullsized":"2024-04-17T10:42:34.018861+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:13:46.065686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982616+0000","last_change":"2024-04-17T10:42:33.982616+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982616+0000","last_undegraded":"2024-04-17T10:42:33.982616+0000","last_fullsized":"2024-04-17T10:42:33.982616+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.016283+0000","last_change":"2024-04-17T10:42:34.012790+0000","last_active":"2024-04-17T10:42:34.016283+0000","last_peered":"2024-04-17T10:42:34.016283+0000","last_clean":"2024-04-17T10:42:34.016283+0000","last_became_active":"2024-04-17T10:42:34.012177+0000","last_became_peered":"2024-04-17T10:42:34.012177+0000","last_unstale":"2024-04-17T10:42:34.016283+0000","last_undegraded":"2024-04-17T10:42:34.016283+0000","last_fullsized":"2024-04-17T10:42:34.016283+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:42:38.783160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982460+0000","last_change":"2024-04-17T10:42:33.982460+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982460+0000","last_undegraded":"2024-04-17T10:42:33.982460+0000","last_fullsized":"2024-04-17T10:42:33.982460+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.015716+0000","last_change":"2024-04-17T10:42:34.015303+0000","last_active":"2024-04-17T10:42:34.015716+0000","last_peered":"2024-04-17T10:42:34.015716+0000","last_clean":"2024-04-17T10:42:34.015716+0000","last_became_active":"2024-04-17T10:42:34.014919+0000","last_became_peered":"2024-04-17T10:42:34.014919+0000","last_unstale":"2024-04-17T10:42:34.015716+0000","last_undegraded":"2024-04-17T10:42:34.015716+0000","last_fullsized":"2024-04-17T10:42:34.015716+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:26:39.567608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982097+0000","last_change":"2024-04-17T10:42:33.982097+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982097+0000","last_undegraded":"2024-04-17T10:42:33.982097+0000","last_fullsized":"2024-04-17T10:42:33.982097+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.013335+0000","last_change":"2024-04-17T10:42:34.009833+0000","last_active":"2024-04-17T10:42:34.013335+0000","last_peered":"2024-04-17T10:42:34.013335+0000","last_clean":"2024-04-17T10:42:34.013335+0000","last_became_active":"2024-04-17T10:42:34.005236+0000","last_became_peered":"2024-04-17T10:42:34.005236+0000","last_unstale":"2024-04-17T10:42:34.013335+0000","last_undegraded":"2024-04-17T10:42:34.013335+0000","last_fullsized":"2024-04-17T10:42:34.013335+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:44:30.520002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.979789+0000","last_change":"2024-04-17T10:42:33.979789+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.979789+0000","last_undegraded":"2024-04-17T10:42:33.979789+0000","last_fullsized":"2024-04-17T10:42:33.979789+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:33.988519+0000","last_change":"2024-04-17T10:42:33.985466+0000","last_active":"2024-04-17T10:42:33.988519+0000","last_peered":"2024-04-17T10:42:33.988519+0000","last_clean":"2024-04-17T10:42:33.988519+0000","last_became_active":"2024-04-17T10:42:33.984982+0000","last_became_peered":"2024-04-17T10:42:33.984982+0000","last_unstale":"2024-04-17T10:42:33.988519+0000","last_undegraded":"2024-04-17T10:42:33.988519+0000","last_fullsized":"2024-04-17T10:42:33.988519+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:41:05.001485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982429+0000","last_change":"2024-04-17T10:42:33.982429+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982429+0000","last_undegraded":"2024-04-17T10:42:33.982429+0000","last_fullsized":"2024-04-17T10:42:33.982429+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.016130+0000","last_change":"2024-04-17T10:42:34.015593+0000","last_active":"2024-04-17T10:42:34.016130+0000","last_peered":"2024-04-17T10:42:34.016130+0000","last_clean":"2024-04-17T10:42:34.016130+0000","last_became_active":"2024-04-17T10:42:34.015222+0000","last_became_peered":"2024-04-17T10:42:34.015222+0000","last_unstale":"2024-04-17T10:42:34.016130+0000","last_undegraded":"2024-04-17T10:42:34.016130+0000","last_fullsized":"2024-04-17T10:42:34.016130+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:44:06.265361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.992302+0000","last_change":"2024-04-17T10:42:33.992302+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.992302+0000","last_undegraded":"2024-04-17T10:42:33.992302+0000","last_fullsized":"2024-04-17T10:42:33.992302+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.026745+0000","last_change":"2024-04-17T10:42:34.021417+0000","last_active":"2024-04-17T10:42:34.026745+0000","last_peered":"2024-04-17T10:42:34.026745+0000","last_clean":"2024-04-17T10:42:34.026745+0000","last_became_active":"2024-04-17T10:42:34.020795+0000","last_became_peered":"2024-04-17T10:42:34.020795+0000","last_unstale":"2024-04-17T10:42:34.026745+0000","last_undegraded":"2024-04-17T10:42:34.026745+0000","last_fullsized":"2024-04-17T10:42:34.026745+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:10:25.368795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":246,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:33.992698+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:42:33.992698+0000","last_peered":"2024-04-17T10:42:33.992698+0000","last_clean":"2024-04-17T10:42:33.992698+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:42:33.992698+0000","last_undegraded":"2024-04-17T10:42:33.992698+0000","last_fullsized":"2024-04-17T10:42:33.992698+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574907,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44912,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326928,"statfs":{"total":96636764160,"available":96590774272,"internally_reserved":0,"allocated":1265664,"data_stored":868517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40160,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331680,"statfs":{"total":96636764160,"available":96595640320,"internally_reserved":0,"allocated":671744,"data_stored":278149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40164,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331676,"statfs":{"total":96636764160,"available":96595636224,"internally_reserved":0,"allocated":671744,"data_stored":278149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49884,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321956,"statfs":{"total":96636764160,"available":96585682944,"internally_reserved":0,"allocated":671744,"data_stored":278149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54044,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317796,"statfs":{"total":96636764160,"available":96581423104,"internally_reserved":0,"allocated":671744,"data_stored":278149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36012,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335828,"statfs":{"total":96636764160,"available":96599887872,"internally_reserved":0,"allocated":684032,"data_stored":284416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49900,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321940,"statfs":{"total":96636764160,"available":96585666560,"internally_reserved":0,"allocated":684032,"data_stored":284416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574906,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40752,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331088,"statfs":{"total":96636764160,"available":96595034112,"internally_reserved":0,"allocated":1277952,"data_stored":874784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:42:39.386 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:42:39.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.549+0000 7f3a0e034640 1 Processor -- start 2024-04-17T10:42:39.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.549+0000 7f3a0e034640 1 -- start start 2024-04-17T10:42:39.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.553+0000 7f3a0e034640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a08146e70 0x7f3a0806d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:39.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.553+0000 7f3a0e034640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a08147570 con 0x7f3a08146a00 2024-04-17T10:42:39.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.553+0000 7f3a0e034640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a0806cbb0 con 0x7f3a080b2d80 2024-04-17T10:42:39.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.553+0000 7f3a0e034640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0806c960 con 0x7f3a08146e70 2024-04-17T10:42:39.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.553+0000 7f3a077fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3a08146a00 0x7f3a080b2670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57456/0 (socket says 172.21.15.83:57456) 2024-04-17T10:42:39.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.553+0000 7f3a077fe640 1 -- 172.21.15.83:0/3840007784 learned_addr learned my addr 172.21.15.83:0/3840007784 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:39.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.553+0000 7f3a07fff640 1 --2- 172.21.15.83:0/3840007784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a08146e70 0x7f3a0806d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:39.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.553+0000 7f3a06ffd640 1 -- 172.21.15.83:0/3840007784 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181837473 0 0) 0x7f3a08147570 con 0x7f3a08146a00 2024-04-17T10:42:39.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.553+0000 7f3a06ffd640 1 -- 172.21.15.83:0/3840007784 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39f4003530 con 0x7f3a08146a00 2024-04-17T10:42:39.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.553+0000 7f3a06ffd640 1 -- 172.21.15.83:0/3840007784 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3038626518 0 0) 0x7f39f4003530 con 0x7f3a08146a00 2024-04-17T10:42:39.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.553+0000 7f3a06ffd640 1 -- 172.21.15.83:0/3840007784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a08146e70 msgr2=0x7f3a0806d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:39.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.553+0000 7f3a06ffd640 1 --2- 172.21.15.83:0/3840007784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a08146e70 0x7f3a0806d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:39.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.553+0000 7f3a06ffd640 1 -- 172.21.15.83:0/3840007784 >> v1:172.21.15.167:6789/0 conn(0x7f3a080b2d80 legacy=0x7f3a08070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:39.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.553+0000 7f3a06ffd640 1 -- 172.21.15.83:0/3840007784 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a0806ce90 con 0x7f3a08146a00 2024-04-17T10:42:39.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.553+0000 7f3a06ffd640 1 -- 172.21.15.83:0/3840007784 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f39f8001680 con 0x7f3a08146a00 2024-04-17T10:42:39.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.553+0000 7f3a06ffd640 1 -- 172.21.15.83:0/3840007784 >> v1:172.21.15.83:6789/0 conn(0x7f3a08146a00 legacy=0x7f3a080b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:39.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.553+0000 7f3a06ffd640 1 --2- 172.21.15.83:0/3840007784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a08146e70 0x7f39f40039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:39.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.553+0000 7f3a06ffd640 1 --2- 172.21.15.83:0/3840007784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39f4003f70 0x7f39f4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:39.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.553+0000 7f3a06ffd640 1 --2- 172.21.15.83:0/3840007784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39f4004d50 0x7f39f40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:39.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.553+0000 7f3a06ffd640 1 -- 172.21.15.83:0/3840007784 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0806c960 con 0x7f39f4003f70 2024-04-17T10:42:39.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.553+0000 7f3a06ffd640 1 -- 172.21.15.83:0/3840007784 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f400bab0 con 0x7f39f4004d50 2024-04-17T10:42:39.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.553+0000 7f3a06ffd640 1 -- 172.21.15.83:0/3840007784 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f400bcb0 con 0x7f3a08146e70 2024-04-17T10:42:39.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.553+0000 7f39fffff640 1 --2- 172.21.15.83:0/3840007784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39f4003f70 0x7f39f4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:39.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.553+0000 7f3a077fe640 1 --2- 172.21.15.83:0/3840007784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a08146e70 0x7f39f40039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:39.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.553+0000 7f3a07fff640 1 --2- 172.21.15.83:0/3840007784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39f4004d50 0x7f39f40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:39.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.553+0000 7f39fffff640 1 -- 172.21.15.83:0/3840007784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a08146e70 msgr2=0x7f39f40039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:39.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.553+0000 7f39fffff640 1 --2- 172.21.15.83:0/3840007784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a08146e70 0x7f39f40039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:39.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.553+0000 7f39fffff640 1 -- 172.21.15.83:0/3840007784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39f4004d50 msgr2=0x7f39f40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:39.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.553+0000 7f39fffff640 1 --2- 172.21.15.83:0/3840007784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39f4004d50 0x7f39f40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:39.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.553+0000 7f39fffff640 1 -- 172.21.15.83:0/3840007784 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f4003530 con 0x7f39f4003f70 2024-04-17T10:42:39.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.557+0000 7f39fffff640 1 --2- 172.21.15.83:0/3840007784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39f4003f70 0x7f39f4004810 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f3a081479a0 tx=0x7f39f001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506ba36653bc0838 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:39.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.557+0000 7f3a06ffd640 1 -- 172.21.15.83:0/3840007784 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f00013a0 con 0x7f39f4003f70 2024-04-17T10:42:39.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.557+0000 7f3a06ffd640 1 -- 172.21.15.83:0/3840007784 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f00052f0 con 0x7f39f4003f70 2024-04-17T10:42:39.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.557+0000 7f3a06ffd640 1 -- 172.21.15.83:0/3840007784 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f0003840 con 0x7f39f4003f70 2024-04-17T10:42:39.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.557+0000 7f3a0e034640 1 -- 172.21.15.83:0/3840007784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39f4003f70 msgr2=0x7f39f4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:39.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.557+0000 7f3a0e034640 1 --2- 172.21.15.83:0/3840007784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39f4003f70 0x7f39f4004810 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f3a081479a0 tx=0x7f39f001c380 comp rx=0 tx=0).stop 2024-04-17T10:42:39.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.557+0000 7f3a07fff640 1 -- 172.21.15.83:0/3840007784 reap_dead start 2024-04-17T10:42:39.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.557+0000 7f3a0e034640 1 -- 172.21.15.83:0/3840007784 shutdown_connections 2024-04-17T10:42:39.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.557+0000 7f3a0e034640 1 -- 172.21.15.83:0/3840007784 >> 172.21.15.83:0/3840007784 conn(0x7f3a0806abe0 msgr2=0x7f3a0806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:39.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.557+0000 7f3a0e034640 1 -- 172.21.15.83:0/3840007784 shutdown_connections 2024-04-17T10:42:39.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.557+0000 7f3a0e034640 1 -- 172.21.15.83:0/3840007784 wait complete. 2024-04-17T10:42:39.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.557+0000 7f3a0e034640 1 Processor -- start 2024-04-17T10:42:39.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.557+0000 7f3a0e034640 1 -- start start 2024-04-17T10:42:39.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.557+0000 7f3a0e034640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39f4003f70 0x7f3a0815d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:39.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.557+0000 7f3a0e034640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a080b13b0 0x7f3a080b1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:39.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.557+0000 7f3a0e034640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a081e7290 0x7f3a081e7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:39.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.557+0000 7f3a0e034640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0806da20 con 0x7f3a080b13b0 2024-04-17T10:42:39.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.557+0000 7f3a0e034640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0806d8a0 con 0x7f39f4003f70 2024-04-17T10:42:39.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.557+0000 7f3a0e034640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0806d200 con 0x7f3a081e7290 2024-04-17T10:42:39.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.561+0000 7f39fffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a080b13b0 0x7f3a080b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:39.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.561+0000 7f39fffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a080b13b0 0x7f3a080b1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36814/0 (socket says 172.21.15.83:36814) 2024-04-17T10:42:39.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.561+0000 7f39fffff640 1 -- 172.21.15.83:0/3278511158 learned_addr learned my addr 172.21.15.83:0/3278511158 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:39.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.561+0000 7f3a07fff640 1 --2- 172.21.15.83:0/3278511158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a081e7290 0x7f3a081e7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:39.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.561+0000 7f3a077fe640 1 --2- 172.21.15.83:0/3278511158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39f4003f70 0x7f3a0815d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:39.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.561+0000 7f39fffff640 1 -- 172.21.15.83:0/3278511158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a081e7290 msgr2=0x7f3a081e7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:39.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.561+0000 7f39fffff640 1 --2- 172.21.15.83:0/3278511158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a081e7290 0x7f3a081e7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:39.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.561+0000 7f39fffff640 1 -- 172.21.15.83:0/3278511158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39f4003f70 msgr2=0x7f3a0815d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:39.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.561+0000 7f39fffff640 1 --2- 172.21.15.83:0/3278511158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39f4003f70 0x7f3a0815d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:39.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.561+0000 7f39fffff640 1 -- 172.21.15.83:0/3278511158 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a081e82a0 con 0x7f3a080b13b0 2024-04-17T10:42:39.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.561+0000 7f3a077fe640 1 --2- 172.21.15.83:0/3278511158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39f4003f70 0x7f3a0815d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:39.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.561+0000 7f3a07fff640 1 --2- 172.21.15.83:0/3278511158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a081e7290 0x7f3a081e7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:42:39.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.561+0000 7f39fffff640 1 --2- 172.21.15.83:0/3278511158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a080b13b0 0x7f3a080b1830 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f39f0005040 tx=0x7f39f4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:39.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.561+0000 7f3a04ff9640 1 -- 172.21.15.83:0/3278511158 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f00054a0 con 0x7f3a080b13b0 2024-04-17T10:42:39.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.561+0000 7f3a04ff9640 1 -- 172.21.15.83:0/3278511158 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f00bb070 con 0x7f3a080b13b0 2024-04-17T10:42:39.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.561+0000 7f3a0e034640 1 -- 172.21.15.83:0/3278511158 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a081e7fc0 con 0x7f3a080b13b0 2024-04-17T10:42:39.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.561+0000 7f3a04ff9640 1 -- 172.21.15.83:0/3278511158 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f00069e0 con 0x7f3a080b13b0 2024-04-17T10:42:39.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.561+0000 7f3a0e034640 1 -- 172.21.15.83:0/3278511158 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a081306b0 con 0x7f3a080b13b0 2024-04-17T10:42:39.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.561+0000 7f3a0e034640 1 -- 172.21.15.83:0/3278511158 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a08074f90 con 0x7f3a080b13b0 2024-04-17T10:42:39.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.561+0000 7f3a04ff9640 1 -- 172.21.15.83:0/3278511158 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f39f00b4040 con 0x7f3a080b13b0 2024-04-17T10:42:39.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.561+0000 7f3a04ff9640 1 --2- 172.21.15.83:0/3278511158 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f39e8066f90 0x7f39e8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:39.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.561+0000 7f3a04ff9640 1 -- 172.21.15.83:0/3278511158 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f39f011f590 con 0x7f3a080b13b0 2024-04-17T10:42:39.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.561+0000 7f3a077fe640 1 --2- 172.21.15.83:0/3278511158 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f39e8066f90 0x7f39e8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:39.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.561+0000 7f3a077fe640 1 --2- 172.21.15.83:0/3278511158 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f39e8066f90 0x7f39e8069450 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f39f8000c10 tx=0x7f39f80014d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:39.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.565+0000 7f3a04ff9640 1 -- 172.21.15.83:0/3278511158 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f39f0016090 con 0x7f3a080b13b0 2024-04-17T10:42:39.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.721+0000 7f3a0e034640 1 -- 172.21.15.83:0/3278511158 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3a080b1d70 con 0x7f39e8066f90 2024-04-17T10:42:39.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.729+0000 7f3a04ff9640 1 -- 172.21.15.83:0/3278511158 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246636 (secure 0 0 0) 0x7f39f80128e0 con 0x7f39e8066f90 2024-04-17T10:42:39.732 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:39.733 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:42:39.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.733+0000 7f3a0e034640 1 -- 172.21.15.83:0/3278511158 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f39e8066f90 msgr2=0x7f39e8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:39.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.733+0000 7f3a0e034640 1 --2- 172.21.15.83:0/3278511158 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f39e8066f90 0x7f39e8069450 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f39f8000c10 tx=0x7f39f80014d0 comp rx=0 tx=0).stop 2024-04-17T10:42:39.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.733+0000 7f3a0e034640 1 -- 172.21.15.83:0/3278511158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a080b13b0 msgr2=0x7f3a080b1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:39.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.733+0000 7f3a0e034640 1 --2- 172.21.15.83:0/3278511158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a080b13b0 0x7f3a080b1830 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f39f0005040 tx=0x7f39f4005840 comp rx=0 tx=0).stop 2024-04-17T10:42:39.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.733+0000 7f3a0e034640 1 -- 172.21.15.83:0/3278511158 shutdown_connections 2024-04-17T10:42:39.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.733+0000 7f3a0e034640 1 --2- 172.21.15.83:0/3278511158 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f39e8066f90 0x7f39e8069450 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:39.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.733+0000 7f3a0e034640 1 --2- 172.21.15.83:0/3278511158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a081e7290 0x7f3a081e7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:39.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.733+0000 7f3a0e034640 1 --2- 172.21.15.83:0/3278511158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a080b13b0 0x7f3a080b1830 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:39.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.733+0000 7f3a0e034640 1 --2- 172.21.15.83:0/3278511158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39f4003f70 0x7f3a0815d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:39.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.733+0000 7f3a0e034640 1 -- 172.21.15.83:0/3278511158 >> 172.21.15.83:0/3278511158 conn(0x7f3a0806abe0 msgr2=0x7f3a08070810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:39.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.733+0000 7f3a0e034640 1 -- 172.21.15.83:0/3278511158 shutdown_connections 2024-04-17T10:42:39.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.733+0000 7f3a0e034640 1 -- 172.21.15.83:0/3278511158 wait complete. 2024-04-17T10:42:39.750 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2024-04-17T10:42:38.259280+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":355828,"kb_used_data":6444,"kb_used_omap":12,"kb_used_meta":349235,"kb_avail":754618892,"statfs":{"total":773094113280,"available":772729745408,"internally_reserved":0,"allocated":6598656,"data_stored":3424729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":357617200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.296682"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.988554+0000","last_change":"2024-04-17T10:42:33.988554+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.988554+0000","last_undegraded":"2024-04-17T10:42:33.988554+0000","last_fullsized":"2024-04-17T10:42:33.988554+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.992506+0000","last_change":"2024-04-17T10:42:33.992506+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.992506+0000","last_undegraded":"2024-04-17T10:42:33.992506+0000","last_fullsized":"2024-04-17T10:42:33.992506+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.991262+0000","last_change":"2024-04-17T10:42:33.991262+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.991262+0000","last_undegraded":"2024-04-17T10:42:33.991262+0000","last_fullsized":"2024-04-17T10:42:33.991262+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.999359+0000","last_change":"2024-04-17T10:42:33.999359+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.999359+0000","last_undegraded":"2024-04-17T10:42:33.999359+0000","last_fullsized":"2024-04-17T10:42:33.999359+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.985818+0000","last_change":"2024-04-17T10:42:33.985818+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.985818+0000","last_undegraded":"2024-04-17T10:42:33.985818+0000","last_fullsized":"2024-04-17T10:42:33.985818+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982596+0000","last_change":"2024-04-17T10:42:33.982596+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982596+0000","last_undegraded":"2024-04-17T10:42:33.982596+0000","last_fullsized":"2024-04-17T10:42:33.982596+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.984830+0000","last_change":"2024-04-17T10:42:33.984830+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.984830+0000","last_undegraded":"2024-04-17T10:42:33.984830+0000","last_fullsized":"2024-04-17T10:42:33.984830+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.994100+0000","last_change":"2024-04-17T10:42:33.994100+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.994100+0000","last_undegraded":"2024-04-17T10:42:33.994100+0000","last_fullsized":"2024-04-17T10:42:33.994100+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982038+0000","last_change":"2024-04-17T10:42:33.982038+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982038+0000","last_undegraded":"2024-04-17T10:42:33.982038+0000","last_fullsized":"2024-04-17T10:42:33.982038+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.980127+0000","last_change":"2024-04-17T10:42:33.980127+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.980127+0000","last_undegraded":"2024-04-17T10:42:33.980127+0000","last_fullsized":"2024-04-17T10:42:33.980127+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.980448+0000","last_change":"2024-04-17T10:42:33.980448+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.980448+0000","last_undegraded":"2024-04-17T10:42:33.980448+0000","last_fullsized":"2024-04-17T10:42:33.980448+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.995007+0000","last_change":"2024-04-17T10:42:33.995007+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.995007+0000","last_undegraded":"2024-04-17T10:42:33.995007+0000","last_fullsized":"2024-04-17T10:42:33.995007+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.990877+0000","last_change":"2024-04-17T10:42:33.990877+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.990877+0000","last_undegraded":"2024-04-17T10:42:33.990877+0000","last_fullsized":"2024-04-17T10:42:33.990877+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.983487+0000","last_change":"2024-04-17T10:42:33.983487+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.983487+0000","last_undegraded":"2024-04-17T10:42:33.983487+0000","last_fullsized":"2024-04-17T10:42:33.983487+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982041+0000","last_change":"2024-04-17T10:42:33.982041+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982041+0000","last_undegraded":"2024-04-17T10:42:33.982041+0000","last_fullsized":"2024-04-17T10:42:33.982041+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.985005+0000","last_change":"2024-04-17T10:42:33.985005+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.985005+0000","last_undegraded":"2024-04-17T10:42:33.985005+0000","last_fullsized":"2024-04-17T10:42:33.985005+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.994197+0000","last_change":"2024-04-17T10:42:33.994197+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.994197+0000","last_undegraded":"2024-04-17T10:42:33.994197+0000","last_fullsized":"2024-04-17T10:42:33.994197+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.988273+0000","last_change":"2024-04-17T10:42:33.988273+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.988273+0000","last_undegraded":"2024-04-17T10:42:33.988273+0000","last_fullsized":"2024-04-17T10:42:33.988273+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.985480+0000","last_change":"2024-04-17T10:42:33.985480+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.985480+0000","last_undegraded":"2024-04-17T10:42:33.985480+0000","last_fullsized":"2024-04-17T10:42:33.985480+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982918+0000","last_change":"2024-04-17T10:42:33.982918+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982918+0000","last_undegraded":"2024-04-17T10:42:33.982918+0000","last_fullsized":"2024-04-17T10:42:33.982918+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.984586+0000","last_change":"2024-04-17T10:42:33.984586+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.984586+0000","last_undegraded":"2024-04-17T10:42:33.984586+0000","last_fullsized":"2024-04-17T10:42:33.984586+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.994276+0000","last_change":"2024-04-17T10:42:33.994276+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.994276+0000","last_undegraded":"2024-04-17T10:42:33.994276+0000","last_fullsized":"2024-04-17T10:42:33.994276+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.983711+0000","last_change":"2024-04-17T10:42:33.983711+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.983711+0000","last_undegraded":"2024-04-17T10:42:33.983711+0000","last_fullsized":"2024-04-17T10:42:33.983711+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.992224+0000","last_change":"2024-04-17T10:42:33.992224+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.992224+0000","last_undegraded":"2024-04-17T10:42:33.992224+0000","last_fullsized":"2024-04-17T10:42:33.992224+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.988095+0000","last_change":"2024-04-17T10:42:33.988095+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.988095+0000","last_undegraded":"2024-04-17T10:42:33.988095+0000","last_fullsized":"2024-04-17T10:42:33.988095+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.996585+0000","last_change":"2024-04-17T10:42:33.996585+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.996585+0000","last_undegraded":"2024-04-17T10:42:33.996585+0000","last_fullsized":"2024-04-17T10:42:33.996585+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.987477+0000","last_change":"2024-04-17T10:42:33.987477+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.987477+0000","last_undegraded":"2024-04-17T10:42:33.987477+0000","last_fullsized":"2024-04-17T10:42:33.987477+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.985065+0000","last_change":"2024-04-17T10:42:33.985065+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.985065+0000","last_undegraded":"2024-04-17T10:42:33.985065+0000","last_fullsized":"2024-04-17T10:42:33.985065+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.991668+0000","last_change":"2024-04-17T10:42:33.991668+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.991668+0000","last_undegraded":"2024-04-17T10:42:33.991668+0000","last_fullsized":"2024-04-17T10:42:33.991668+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.985954+0000","last_change":"2024-04-17T10:42:33.985954+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.985954+0000","last_undegraded":"2024-04-17T10:42:33.985954+0000","last_fullsized":"2024-04-17T10:42:33.985954+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.984858+0000","last_change":"2024-04-17T10:42:33.984858+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.984858+0000","last_undegraded":"2024-04-17T10:42:33.984858+0000","last_fullsized":"2024-04-17T10:42:33.984858+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.984313+0000","last_change":"2024-04-17T10:42:33.984313+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.984313+0000","last_undegraded":"2024-04-17T10:42:33.984313+0000","last_fullsized":"2024-04-17T10:42:33.984313+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.987311+0000","last_change":"2024-04-17T10:42:33.987311+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.987311+0000","last_undegraded":"2024-04-17T10:42:33.987311+0000","last_fullsized":"2024-04-17T10:42:33.987311+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.015861+0000","last_change":"2024-04-17T10:42:34.015019+0000","last_active":"2024-04-17T10:42:34.015861+0000","last_peered":"2024-04-17T10:42:34.015861+0000","last_clean":"2024-04-17T10:42:34.015861+0000","last_became_active":"2024-04-17T10:42:34.014716+0000","last_became_peered":"2024-04-17T10:42:34.014716+0000","last_unstale":"2024-04-17T10:42:34.015861+0000","last_undegraded":"2024-04-17T10:42:34.015861+0000","last_fullsized":"2024-04-17T10:42:34.015861+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:26:57.986949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.980974+0000","last_change":"2024-04-17T10:42:33.980974+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.980974+0000","last_undegraded":"2024-04-17T10:42:33.980974+0000","last_fullsized":"2024-04-17T10:42:33.980974+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.006810+0000","last_change":"2024-04-17T10:42:34.006170+0000","last_active":"2024-04-17T10:42:34.006810+0000","last_peered":"2024-04-17T10:42:34.006810+0000","last_clean":"2024-04-17T10:42:34.006810+0000","last_became_active":"2024-04-17T10:42:34.005757+0000","last_became_peered":"2024-04-17T10:42:34.005757+0000","last_unstale":"2024-04-17T10:42:34.006810+0000","last_undegraded":"2024-04-17T10:42:34.006810+0000","last_fullsized":"2024-04-17T10:42:34.006810+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:25:59.364150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982188+0000","last_change":"2024-04-17T10:42:33.982188+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982188+0000","last_undegraded":"2024-04-17T10:42:33.982188+0000","last_fullsized":"2024-04-17T10:42:33.982188+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:33.996540+0000","last_change":"2024-04-17T10:42:33.993426+0000","last_active":"2024-04-17T10:42:33.996540+0000","last_peered":"2024-04-17T10:42:33.996540+0000","last_clean":"2024-04-17T10:42:33.996540+0000","last_became_active":"2024-04-17T10:42:33.992971+0000","last_became_peered":"2024-04-17T10:42:33.992971+0000","last_unstale":"2024-04-17T10:42:33.996540+0000","last_undegraded":"2024-04-17T10:42:33.996540+0000","last_fullsized":"2024-04-17T10:42:33.996540+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:42:12.531359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.987048+0000","last_change":"2024-04-17T10:42:33.987048+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.987048+0000","last_undegraded":"2024-04-17T10:42:33.987048+0000","last_fullsized":"2024-04-17T10:42:33.987048+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.021952+0000","last_change":"2024-04-17T10:42:34.018234+0000","last_active":"2024-04-17T10:42:34.021952+0000","last_peered":"2024-04-17T10:42:34.021952+0000","last_clean":"2024-04-17T10:42:34.021952+0000","last_became_active":"2024-04-17T10:42:34.017497+0000","last_became_peered":"2024-04-17T10:42:34.017497+0000","last_unstale":"2024-04-17T10:42:34.021952+0000","last_undegraded":"2024-04-17T10:42:34.021952+0000","last_fullsized":"2024-04-17T10:42:34.021952+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:32:27.523313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.992516+0000","last_change":"2024-04-17T10:42:33.992516+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.992516+0000","last_undegraded":"2024-04-17T10:42:33.992516+0000","last_fullsized":"2024-04-17T10:42:33.992516+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.013446+0000","last_change":"2024-04-17T10:42:34.004483+0000","last_active":"2024-04-17T10:42:34.013446+0000","last_peered":"2024-04-17T10:42:34.013446+0000","last_clean":"2024-04-17T10:42:34.013446+0000","last_became_active":"2024-04-17T10:42:33.998883+0000","last_became_peered":"2024-04-17T10:42:33.998883+0000","last_unstale":"2024-04-17T10:42:34.013446+0000","last_undegraded":"2024-04-17T10:42:34.013446+0000","last_fullsized":"2024-04-17T10:42:34.013446+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:42:12.531359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.992432+0000","last_change":"2024-04-17T10:42:33.992432+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.992432+0000","last_undegraded":"2024-04-17T10:42:33.992432+0000","last_fullsized":"2024-04-17T10:42:33.992432+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.002170+0000","last_change":"2024-04-17T10:42:34.000321+0000","last_active":"2024-04-17T10:42:34.002170+0000","last_peered":"2024-04-17T10:42:34.002170+0000","last_clean":"2024-04-17T10:42:34.002170+0000","last_became_active":"2024-04-17T10:42:34.000062+0000","last_became_peered":"2024-04-17T10:42:34.000062+0000","last_unstale":"2024-04-17T10:42:34.002170+0000","last_undegraded":"2024-04-17T10:42:34.002170+0000","last_fullsized":"2024-04-17T10:42:34.002170+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:58:18.204240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.991126+0000","last_change":"2024-04-17T10:42:33.991126+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.991126+0000","last_undegraded":"2024-04-17T10:42:33.991126+0000","last_fullsized":"2024-04-17T10:42:33.991126+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.021947+0000","last_change":"2024-04-17T10:42:34.018055+0000","last_active":"2024-04-17T10:42:34.021947+0000","last_peered":"2024-04-17T10:42:34.021947+0000","last_clean":"2024-04-17T10:42:34.021947+0000","last_became_active":"2024-04-17T10:42:34.016719+0000","last_became_peered":"2024-04-17T10:42:34.016719+0000","last_unstale":"2024-04-17T10:42:34.021947+0000","last_undegraded":"2024-04-17T10:42:34.021947+0000","last_fullsized":"2024-04-17T10:42:34.021947+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:20:46.478088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.990753+0000","last_change":"2024-04-17T10:42:33.990753+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.990753+0000","last_undegraded":"2024-04-17T10:42:33.990753+0000","last_fullsized":"2024-04-17T10:42:33.990753+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.013153+0000","last_change":"2024-04-17T10:42:34.008774+0000","last_active":"2024-04-17T10:42:34.013153+0000","last_peered":"2024-04-17T10:42:34.013153+0000","last_clean":"2024-04-17T10:42:34.013153+0000","last_became_active":"2024-04-17T10:42:34.005106+0000","last_became_peered":"2024-04-17T10:42:34.005106+0000","last_unstale":"2024-04-17T10:42:34.013153+0000","last_undegraded":"2024-04-17T10:42:34.013153+0000","last_fullsized":"2024-04-17T10:42:34.013153+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:58:18.204240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.984503+0000","last_change":"2024-04-17T10:42:33.984503+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.984503+0000","last_undegraded":"2024-04-17T10:42:33.984503+0000","last_fullsized":"2024-04-17T10:42:33.984503+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.014222+0000","last_change":"2024-04-17T10:42:34.006222+0000","last_active":"2024-04-17T10:42:34.014222+0000","last_peered":"2024-04-17T10:42:34.014222+0000","last_clean":"2024-04-17T10:42:34.014222+0000","last_became_active":"2024-04-17T10:42:34.005353+0000","last_became_peered":"2024-04-17T10:42:34.005353+0000","last_unstale":"2024-04-17T10:42:34.014222+0000","last_undegraded":"2024-04-17T10:42:34.014222+0000","last_fullsized":"2024-04-17T10:42:34.014222+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:20:46.478088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.985320+0000","last_change":"2024-04-17T10:42:33.985320+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.985320+0000","last_undegraded":"2024-04-17T10:42:33.985320+0000","last_fullsized":"2024-04-17T10:42:33.985320+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.000258+0000","last_change":"2024-04-17T10:42:33.995027+0000","last_active":"2024-04-17T10:42:34.000258+0000","last_peered":"2024-04-17T10:42:34.000258+0000","last_clean":"2024-04-17T10:42:34.000258+0000","last_became_active":"2024-04-17T10:42:33.994748+0000","last_became_peered":"2024-04-17T10:42:33.994748+0000","last_unstale":"2024-04-17T10:42:34.000258+0000","last_undegraded":"2024-04-17T10:42:34.000258+0000","last_fullsized":"2024-04-17T10:42:34.000258+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:41:52.745779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.981639+0000","last_change":"2024-04-17T10:42:33.981639+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.981639+0000","last_undegraded":"2024-04-17T10:42:33.981639+0000","last_fullsized":"2024-04-17T10:42:33.981639+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.016297+0000","last_change":"2024-04-17T10:42:34.010793+0000","last_active":"2024-04-17T10:42:34.016297+0000","last_peered":"2024-04-17T10:42:34.016297+0000","last_clean":"2024-04-17T10:42:34.016297+0000","last_became_active":"2024-04-17T10:42:34.009786+0000","last_became_peered":"2024-04-17T10:42:34.009786+0000","last_unstale":"2024-04-17T10:42:34.016297+0000","last_undegraded":"2024-04-17T10:42:34.016297+0000","last_fullsized":"2024-04-17T10:42:34.016297+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:32:27.523313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.990052+0000","last_change":"2024-04-17T10:42:33.990052+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.990052+0000","last_undegraded":"2024-04-17T10:42:33.990052+0000","last_fullsized":"2024-04-17T10:42:33.990052+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.013294+0000","last_change":"2024-04-17T10:42:34.008738+0000","last_active":"2024-04-17T10:42:34.013294+0000","last_peered":"2024-04-17T10:42:34.013294+0000","last_clean":"2024-04-17T10:42:34.013294+0000","last_became_active":"2024-04-17T10:42:34.004730+0000","last_became_peered":"2024-04-17T10:42:34.004730+0000","last_unstale":"2024-04-17T10:42:34.013294+0000","last_undegraded":"2024-04-17T10:42:34.013294+0000","last_fullsized":"2024-04-17T10:42:34.013294+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:14:14.289936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.993209+0000","last_change":"2024-04-17T10:42:33.993209+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.993209+0000","last_undegraded":"2024-04-17T10:42:33.993209+0000","last_fullsized":"2024-04-17T10:42:33.993209+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.000845+0000","last_change":"2024-04-17T10:42:34.000022+0000","last_active":"2024-04-17T10:42:34.000845+0000","last_peered":"2024-04-17T10:42:34.000845+0000","last_clean":"2024-04-17T10:42:34.000845+0000","last_became_active":"2024-04-17T10:42:33.999505+0000","last_became_peered":"2024-04-17T10:42:33.999505+0000","last_unstale":"2024-04-17T10:42:34.000845+0000","last_undegraded":"2024-04-17T10:42:34.000845+0000","last_fullsized":"2024-04-17T10:42:34.000845+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:01:33.910102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.983396+0000","last_change":"2024-04-17T10:42:33.983396+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.983396+0000","last_undegraded":"2024-04-17T10:42:33.983396+0000","last_fullsized":"2024-04-17T10:42:33.983396+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.015295+0000","last_change":"2024-04-17T10:42:34.010051+0000","last_active":"2024-04-17T10:42:34.015295+0000","last_peered":"2024-04-17T10:42:34.015295+0000","last_clean":"2024-04-17T10:42:34.015295+0000","last_became_active":"2024-04-17T10:42:34.009227+0000","last_became_peered":"2024-04-17T10:42:34.009227+0000","last_unstale":"2024-04-17T10:42:34.015295+0000","last_undegraded":"2024-04-17T10:42:34.015295+0000","last_fullsized":"2024-04-17T10:42:34.015295+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:41:05.001485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982143+0000","last_change":"2024-04-17T10:42:33.982143+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982143+0000","last_undegraded":"2024-04-17T10:42:33.982143+0000","last_fullsized":"2024-04-17T10:42:33.982143+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.023975+0000","last_change":"2024-04-17T10:42:34.019631+0000","last_active":"2024-04-17T10:42:34.023975+0000","last_peered":"2024-04-17T10:42:34.023975+0000","last_clean":"2024-04-17T10:42:34.023975+0000","last_became_active":"2024-04-17T10:42:34.018768+0000","last_became_peered":"2024-04-17T10:42:34.018768+0000","last_unstale":"2024-04-17T10:42:34.023975+0000","last_undegraded":"2024-04-17T10:42:34.023975+0000","last_fullsized":"2024-04-17T10:42:34.023975+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:37:08.284805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982708+0000","last_change":"2024-04-17T10:42:33.982708+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982708+0000","last_undegraded":"2024-04-17T10:42:33.982708+0000","last_fullsized":"2024-04-17T10:42:33.982708+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.019298+0000","last_change":"2024-04-17T10:42:34.015072+0000","last_active":"2024-04-17T10:42:34.019298+0000","last_peered":"2024-04-17T10:42:34.019298+0000","last_clean":"2024-04-17T10:42:34.019298+0000","last_became_active":"2024-04-17T10:42:34.014406+0000","last_became_peered":"2024-04-17T10:42:34.014406+0000","last_unstale":"2024-04-17T10:42:34.019298+0000","last_undegraded":"2024-04-17T10:42:34.019298+0000","last_fullsized":"2024-04-17T10:42:34.019298+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:37:08.284805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.984738+0000","last_change":"2024-04-17T10:42:33.984738+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.984738+0000","last_undegraded":"2024-04-17T10:42:33.984738+0000","last_fullsized":"2024-04-17T10:42:33.984738+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.000855+0000","last_change":"2024-04-17T10:42:33.998582+0000","last_active":"2024-04-17T10:42:34.000855+0000","last_peered":"2024-04-17T10:42:34.000855+0000","last_clean":"2024-04-17T10:42:34.000855+0000","last_became_active":"2024-04-17T10:42:33.997341+0000","last_became_peered":"2024-04-17T10:42:33.997341+0000","last_unstale":"2024-04-17T10:42:34.000855+0000","last_undegraded":"2024-04-17T10:42:34.000855+0000","last_fullsized":"2024-04-17T10:42:34.000855+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:03:51.699883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.999265+0000","last_change":"2024-04-17T10:42:33.999265+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.999265+0000","last_undegraded":"2024-04-17T10:42:33.999265+0000","last_fullsized":"2024-04-17T10:42:33.999265+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:33.999215+0000","last_change":"2024-04-17T10:42:33.995762+0000","last_active":"2024-04-17T10:42:33.999215+0000","last_peered":"2024-04-17T10:42:33.999215+0000","last_clean":"2024-04-17T10:42:33.999215+0000","last_became_active":"2024-04-17T10:42:33.995562+0000","last_became_peered":"2024-04-17T10:42:33.995562+0000","last_unstale":"2024-04-17T10:42:33.999215+0000","last_undegraded":"2024-04-17T10:42:33.999215+0000","last_fullsized":"2024-04-17T10:42:33.999215+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:14:14.289936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982369+0000","last_change":"2024-04-17T10:42:33.982369+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982369+0000","last_undegraded":"2024-04-17T10:42:33.982369+0000","last_fullsized":"2024-04-17T10:42:33.982369+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.021531+0000","last_change":"2024-04-17T10:42:34.017202+0000","last_active":"2024-04-17T10:42:34.021531+0000","last_peered":"2024-04-17T10:42:34.021531+0000","last_clean":"2024-04-17T10:42:34.021531+0000","last_became_active":"2024-04-17T10:42:34.016613+0000","last_became_peered":"2024-04-17T10:42:34.016613+0000","last_unstale":"2024-04-17T10:42:34.021531+0000","last_undegraded":"2024-04-17T10:42:34.021531+0000","last_fullsized":"2024-04-17T10:42:34.021531+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:09:32.979808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.985866+0000","last_change":"2024-04-17T10:42:33.985866+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.985866+0000","last_undegraded":"2024-04-17T10:42:33.985866+0000","last_fullsized":"2024-04-17T10:42:33.985866+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.002835+0000","last_change":"2024-04-17T10:42:34.002003+0000","last_active":"2024-04-17T10:42:34.002835+0000","last_peered":"2024-04-17T10:42:34.002835+0000","last_clean":"2024-04-17T10:42:34.002835+0000","last_became_active":"2024-04-17T10:42:34.001413+0000","last_became_peered":"2024-04-17T10:42:34.001413+0000","last_unstale":"2024-04-17T10:42:34.002835+0000","last_undegraded":"2024-04-17T10:42:34.002835+0000","last_fullsized":"2024-04-17T10:42:34.002835+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:44:30.520002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.991626+0000","last_change":"2024-04-17T10:42:33.991626+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.991626+0000","last_undegraded":"2024-04-17T10:42:33.991626+0000","last_fullsized":"2024-04-17T10:42:33.991626+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.011040+0000","last_change":"2024-04-17T10:42:34.010407+0000","last_active":"2024-04-17T10:42:34.011040+0000","last_peered":"2024-04-17T10:42:34.011040+0000","last_clean":"2024-04-17T10:42:34.011040+0000","last_became_active":"2024-04-17T10:42:34.009954+0000","last_became_peered":"2024-04-17T10:42:34.009954+0000","last_unstale":"2024-04-17T10:42:34.011040+0000","last_undegraded":"2024-04-17T10:42:34.011040+0000","last_fullsized":"2024-04-17T10:42:34.011040+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:44:50.147806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982157+0000","last_change":"2024-04-17T10:42:33.982157+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982157+0000","last_undegraded":"2024-04-17T10:42:33.982157+0000","last_fullsized":"2024-04-17T10:42:33.982157+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:33.996210+0000","last_change":"2024-04-17T10:42:33.991455+0000","last_active":"2024-04-17T10:42:33.996210+0000","last_peered":"2024-04-17T10:42:33.996210+0000","last_clean":"2024-04-17T10:42:33.996210+0000","last_became_active":"2024-04-17T10:42:33.990906+0000","last_became_peered":"2024-04-17T10:42:33.990906+0000","last_unstale":"2024-04-17T10:42:33.996210+0000","last_undegraded":"2024-04-17T10:42:33.996210+0000","last_fullsized":"2024-04-17T10:42:33.996210+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:13:46.065686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.984120+0000","last_change":"2024-04-17T10:42:33.984120+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.984120+0000","last_undegraded":"2024-04-17T10:42:33.984120+0000","last_fullsized":"2024-04-17T10:42:33.984120+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.016150+0000","last_change":"2024-04-17T10:42:34.007999+0000","last_active":"2024-04-17T10:42:34.016150+0000","last_peered":"2024-04-17T10:42:34.016150+0000","last_clean":"2024-04-17T10:42:34.016150+0000","last_became_active":"2024-04-17T10:42:34.007259+0000","last_became_peered":"2024-04-17T10:42:34.007259+0000","last_unstale":"2024-04-17T10:42:34.016150+0000","last_undegraded":"2024-04-17T10:42:34.016150+0000","last_fullsized":"2024-04-17T10:42:34.016150+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:10:36.224186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.995538+0000","last_change":"2024-04-17T10:42:33.995538+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.995538+0000","last_undegraded":"2024-04-17T10:42:33.995538+0000","last_fullsized":"2024-04-17T10:42:33.995538+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.002602+0000","last_change":"2024-04-17T10:42:34.002043+0000","last_active":"2024-04-17T10:42:34.002602+0000","last_peered":"2024-04-17T10:42:34.002602+0000","last_clean":"2024-04-17T10:42:34.002602+0000","last_became_active":"2024-04-17T10:42:34.001536+0000","last_became_peered":"2024-04-17T10:42:34.001536+0000","last_unstale":"2024-04-17T10:42:34.002602+0000","last_undegraded":"2024-04-17T10:42:34.002602+0000","last_fullsized":"2024-04-17T10:42:34.002602+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:10:25.368795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.998164+0000","last_change":"2024-04-17T10:42:33.998164+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.998164+0000","last_undegraded":"2024-04-17T10:42:33.998164+0000","last_fullsized":"2024-04-17T10:42:33.998164+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.003492+0000","last_change":"2024-04-17T10:42:34.003042+0000","last_active":"2024-04-17T10:42:34.003492+0000","last_peered":"2024-04-17T10:42:34.003492+0000","last_clean":"2024-04-17T10:42:34.003492+0000","last_became_active":"2024-04-17T10:42:34.002726+0000","last_became_peered":"2024-04-17T10:42:34.002726+0000","last_unstale":"2024-04-17T10:42:34.003492+0000","last_undegraded":"2024-04-17T10:42:34.003492+0000","last_fullsized":"2024-04-17T10:42:34.003492+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:34:51.462597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.979958+0000","last_change":"2024-04-17T10:42:33.979958+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.979958+0000","last_undegraded":"2024-04-17T10:42:33.979958+0000","last_fullsized":"2024-04-17T10:42:33.979958+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.018861+0000","last_change":"2024-04-17T10:42:34.012933+0000","last_active":"2024-04-17T10:42:34.018861+0000","last_peered":"2024-04-17T10:42:34.018861+0000","last_clean":"2024-04-17T10:42:34.018861+0000","last_became_active":"2024-04-17T10:42:34.012150+0000","last_became_peered":"2024-04-17T10:42:34.012150+0000","last_unstale":"2024-04-17T10:42:34.018861+0000","last_undegraded":"2024-04-17T10:42:34.018861+0000","last_fullsized":"2024-04-17T10:42:34.018861+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:13:46.065686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982616+0000","last_change":"2024-04-17T10:42:33.982616+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982616+0000","last_undegraded":"2024-04-17T10:42:33.982616+0000","last_fullsized":"2024-04-17T10:42:33.982616+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.016283+0000","last_change":"2024-04-17T10:42:34.012790+0000","last_active":"2024-04-17T10:42:34.016283+0000","last_peered":"2024-04-17T10:42:34.016283+0000","last_clean":"2024-04-17T10:42:34.016283+0000","last_became_active":"2024-04-17T10:42:34.012177+0000","last_became_peered":"2024-04-17T10:42:34.012177+0000","last_unstale":"2024-04-17T10:42:34.016283+0000","last_undegraded":"2024-04-17T10:42:34.016283+0000","last_fullsized":"2024-04-17T10:42:34.016283+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:42:38.783160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982460+0000","last_change":"2024-04-17T10:42:33.982460+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982460+0000","last_undegraded":"2024-04-17T10:42:33.982460+0000","last_fullsized":"2024-04-17T10:42:33.982460+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.015716+0000","last_change":"2024-04-17T10:42:34.015303+0000","last_active":"2024-04-17T10:42:34.015716+0000","last_peered":"2024-04-17T10:42:34.015716+0000","last_clean":"2024-04-17T10:42:34.015716+0000","last_became_active":"2024-04-17T10:42:34.014919+0000","last_became_peered":"2024-04-17T10:42:34.014919+0000","last_unstale":"2024-04-17T10:42:34.015716+0000","last_undegraded":"2024-04-17T10:42:34.015716+0000","last_fullsized":"2024-04-17T10:42:34.015716+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:26:39.567608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982097+0000","last_change":"2024-04-17T10:42:33.982097+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982097+0000","last_undegraded":"2024-04-17T10:42:33.982097+0000","last_fullsized":"2024-04-17T10:42:33.982097+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.013335+0000","last_change":"2024-04-17T10:42:34.009833+0000","last_active":"2024-04-17T10:42:34.013335+0000","last_peered":"2024-04-17T10:42:34.013335+0000","last_clean":"2024-04-17T10:42:34.013335+0000","last_became_active":"2024-04-17T10:42:34.005236+0000","last_became_peered":"2024-04-17T10:42:34.005236+0000","last_unstale":"2024-04-17T10:42:34.013335+0000","last_undegraded":"2024-04-17T10:42:34.013335+0000","last_fullsized":"2024-04-17T10:42:34.013335+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:44:30.520002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.979789+0000","last_change":"2024-04-17T10:42:33.979789+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.979789+0000","last_undegraded":"2024-04-17T10:42:33.979789+0000","last_fullsized":"2024-04-17T10:42:33.979789+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:33.988519+0000","last_change":"2024-04-17T10:42:33.985466+0000","last_active":"2024-04-17T10:42:33.988519+0000","last_peered":"2024-04-17T10:42:33.988519+0000","last_clean":"2024-04-17T10:42:33.988519+0000","last_became_active":"2024-04-17T10:42:33.984982+0000","last_became_peered":"2024-04-17T10:42:33.984982+0000","last_unstale":"2024-04-17T10:42:33.988519+0000","last_undegraded":"2024-04-17T10:42:33.988519+0000","last_fullsized":"2024-04-17T10:42:33.988519+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:41:05.001485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.982429+0000","last_change":"2024-04-17T10:42:33.982429+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.982429+0000","last_undegraded":"2024-04-17T10:42:33.982429+0000","last_fullsized":"2024-04-17T10:42:33.982429+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.016130+0000","last_change":"2024-04-17T10:42:34.015593+0000","last_active":"2024-04-17T10:42:34.016130+0000","last_peered":"2024-04-17T10:42:34.016130+0000","last_clean":"2024-04-17T10:42:34.016130+0000","last_became_active":"2024-04-17T10:42:34.015222+0000","last_became_peered":"2024-04-17T10:42:34.015222+0000","last_unstale":"2024-04-17T10:42:34.016130+0000","last_undegraded":"2024-04-17T10:42:34.016130+0000","last_fullsized":"2024-04-17T10:42:34.016130+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:44:06.265361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-17T10:42:33.992302+0000","last_change":"2024-04-17T10:42:33.992302+0000","last_active":"2024-04-17T10:42:33.961433+0000","last_peered":"2024-04-17T10:42:33.961433+0000","last_clean":"2024-04-17T10:42:33.961433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:42:33.992302+0000","last_undegraded":"2024-04-17T10:42:33.992302+0000","last_fullsized":"2024-04-17T10:42:33.992302+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:34.026745+0000","last_change":"2024-04-17T10:42:34.021417+0000","last_active":"2024-04-17T10:42:34.026745+0000","last_peered":"2024-04-17T10:42:34.026745+0000","last_clean":"2024-04-17T10:42:34.026745+0000","last_became_active":"2024-04-17T10:42:34.020795+0000","last_became_peered":"2024-04-17T10:42:34.020795+0000","last_unstale":"2024-04-17T10:42:34.026745+0000","last_undegraded":"2024-04-17T10:42:34.026745+0000","last_fullsized":"2024-04-17T10:42:34.026745+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:10:25.368795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":246,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T10:42:33.992698+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:42:33.992698+0000","last_peered":"2024-04-17T10:42:33.992698+0000","last_clean":"2024-04-17T10:42:33.992698+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:42:33.992698+0000","last_undegraded":"2024-04-17T10:42:33.992698+0000","last_fullsized":"2024-04-17T10:42:33.992698+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574907,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44912,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326928,"statfs":{"total":96636764160,"available":96590774272,"internally_reserved":0,"allocated":1265664,"data_stored":868517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40160,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331680,"statfs":{"total":96636764160,"available":96595640320,"internally_reserved":0,"allocated":671744,"data_stored":278149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40164,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331676,"statfs":{"total":96636764160,"available":96595636224,"internally_reserved":0,"allocated":671744,"data_stored":278149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49884,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321956,"statfs":{"total":96636764160,"available":96585682944,"internally_reserved":0,"allocated":671744,"data_stored":278149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54044,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317796,"statfs":{"total":96636764160,"available":96581423104,"internally_reserved":0,"allocated":671744,"data_stored":278149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36012,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335828,"statfs":{"total":96636764160,"available":96599887872,"internally_reserved":0,"allocated":684032,"data_stored":284416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49900,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321940,"statfs":{"total":96636764160,"available":96585666560,"internally_reserved":0,"allocated":684032,"data_stored":284416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574906,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40752,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331088,"statfs":{"total":96636764160,"available":96595034112,"internally_reserved":0,"allocated":1277952,"data_stored":874784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:42:39.754 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3f is not active+clean 2024-04-17T10:42:39.754 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.988554+0000', 'last_change': '2024-04-17T10:42:33.988554+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.988554+0000', 'last_undegraded': '2024-04-17T10:42:33.988554+0000', 'last_fullsized': '2024-04-17T10:42:33.988554+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:42:39.754 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3e is not active+clean 2024-04-17T10:42:39.755 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.992506+0000', 'last_change': '2024-04-17T10:42:33.992506+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.992506+0000', 'last_undegraded': '2024-04-17T10:42:33.992506+0000', 'last_fullsized': '2024-04-17T10:42:33.992506+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:42:39.755 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3d is not active+clean 2024-04-17T10:42:39.755 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.991262+0000', 'last_change': '2024-04-17T10:42:33.991262+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.991262+0000', 'last_undegraded': '2024-04-17T10:42:33.991262+0000', 'last_fullsized': '2024-04-17T10:42:33.991262+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:42:39.755 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3c is not active+clean 2024-04-17T10:42:39.755 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.999359+0000', 'last_change': '2024-04-17T10:42:33.999359+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.999359+0000', 'last_undegraded': '2024-04-17T10:42:33.999359+0000', 'last_fullsized': '2024-04-17T10:42:33.999359+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:42:39.755 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3b is not active+clean 2024-04-17T10:42:39.755 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.985818+0000', 'last_change': '2024-04-17T10:42:33.985818+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.985818+0000', 'last_undegraded': '2024-04-17T10:42:33.985818+0000', 'last_fullsized': '2024-04-17T10:42:33.985818+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:42:39.755 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3a is not active+clean 2024-04-17T10:42:39.755 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.982596+0000', 'last_change': '2024-04-17T10:42:33.982596+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.982596+0000', 'last_undegraded': '2024-04-17T10:42:33.982596+0000', 'last_fullsized': '2024-04-17T10:42:33.982596+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:42:39.756 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.39 is not active+clean 2024-04-17T10:42:39.756 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.984830+0000', 'last_change': '2024-04-17T10:42:33.984830+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.984830+0000', 'last_undegraded': '2024-04-17T10:42:33.984830+0000', 'last_fullsized': '2024-04-17T10:42:33.984830+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:42:39.756 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.38 is not active+clean 2024-04-17T10:42:39.756 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.994100+0000', 'last_change': '2024-04-17T10:42:33.994100+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.994100+0000', 'last_undegraded': '2024-04-17T10:42:33.994100+0000', 'last_fullsized': '2024-04-17T10:42:33.994100+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:42:39.756 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.37 is not active+clean 2024-04-17T10:42:39.756 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.982038+0000', 'last_change': '2024-04-17T10:42:33.982038+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.982038+0000', 'last_undegraded': '2024-04-17T10:42:33.982038+0000', 'last_fullsized': '2024-04-17T10:42:33.982038+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:42:39.756 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.36 is not active+clean 2024-04-17T10:42:39.756 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.980127+0000', 'last_change': '2024-04-17T10:42:33.980127+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.980127+0000', 'last_undegraded': '2024-04-17T10:42:33.980127+0000', 'last_fullsized': '2024-04-17T10:42:33.980127+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:42:39.756 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.35 is not active+clean 2024-04-17T10:42:39.756 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.980448+0000', 'last_change': '2024-04-17T10:42:33.980448+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.980448+0000', 'last_undegraded': '2024-04-17T10:42:33.980448+0000', 'last_fullsized': '2024-04-17T10:42:33.980448+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:42:39.757 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.34 is not active+clean 2024-04-17T10:42:39.757 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.995007+0000', 'last_change': '2024-04-17T10:42:33.995007+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.995007+0000', 'last_undegraded': '2024-04-17T10:42:33.995007+0000', 'last_fullsized': '2024-04-17T10:42:33.995007+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:42:39.757 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.33 is not active+clean 2024-04-17T10:42:39.757 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.990877+0000', 'last_change': '2024-04-17T10:42:33.990877+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.990877+0000', 'last_undegraded': '2024-04-17T10:42:33.990877+0000', 'last_fullsized': '2024-04-17T10:42:33.990877+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:42:39.757 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.32 is not active+clean 2024-04-17T10:42:39.757 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.983487+0000', 'last_change': '2024-04-17T10:42:33.983487+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.983487+0000', 'last_undegraded': '2024-04-17T10:42:33.983487+0000', 'last_fullsized': '2024-04-17T10:42:33.983487+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:42:39.757 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.31 is not active+clean 2024-04-17T10:42:39.757 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.982041+0000', 'last_change': '2024-04-17T10:42:33.982041+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.982041+0000', 'last_undegraded': '2024-04-17T10:42:33.982041+0000', 'last_fullsized': '2024-04-17T10:42:33.982041+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:42:39.757 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.30 is not active+clean 2024-04-17T10:42:39.757 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.985005+0000', 'last_change': '2024-04-17T10:42:33.985005+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.985005+0000', 'last_undegraded': '2024-04-17T10:42:33.985005+0000', 'last_fullsized': '2024-04-17T10:42:33.985005+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:42:39.758 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2f is not active+clean 2024-04-17T10:42:39.758 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.994197+0000', 'last_change': '2024-04-17T10:42:33.994197+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.994197+0000', 'last_undegraded': '2024-04-17T10:42:33.994197+0000', 'last_fullsized': '2024-04-17T10:42:33.994197+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:42:39.758 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2e is not active+clean 2024-04-17T10:42:39.758 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.988273+0000', 'last_change': '2024-04-17T10:42:33.988273+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.988273+0000', 'last_undegraded': '2024-04-17T10:42:33.988273+0000', 'last_fullsized': '2024-04-17T10:42:33.988273+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:42:39.758 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2d is not active+clean 2024-04-17T10:42:39.758 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.985480+0000', 'last_change': '2024-04-17T10:42:33.985480+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.985480+0000', 'last_undegraded': '2024-04-17T10:42:33.985480+0000', 'last_fullsized': '2024-04-17T10:42:33.985480+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:42:39.758 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2c is not active+clean 2024-04-17T10:42:39.758 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.982918+0000', 'last_change': '2024-04-17T10:42:33.982918+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.982918+0000', 'last_undegraded': '2024-04-17T10:42:33.982918+0000', 'last_fullsized': '2024-04-17T10:42:33.982918+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:42:39.758 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2b is not active+clean 2024-04-17T10:42:39.758 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.984586+0000', 'last_change': '2024-04-17T10:42:33.984586+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.984586+0000', 'last_undegraded': '2024-04-17T10:42:33.984586+0000', 'last_fullsized': '2024-04-17T10:42:33.984586+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:42:39.759 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2a is not active+clean 2024-04-17T10:42:39.759 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.994276+0000', 'last_change': '2024-04-17T10:42:33.994276+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.994276+0000', 'last_undegraded': '2024-04-17T10:42:33.994276+0000', 'last_fullsized': '2024-04-17T10:42:33.994276+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:42:39.759 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.29 is not active+clean 2024-04-17T10:42:39.759 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.983711+0000', 'last_change': '2024-04-17T10:42:33.983711+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.983711+0000', 'last_undegraded': '2024-04-17T10:42:33.983711+0000', 'last_fullsized': '2024-04-17T10:42:33.983711+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:42:39.759 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.28 is not active+clean 2024-04-17T10:42:39.759 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.992224+0000', 'last_change': '2024-04-17T10:42:33.992224+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.992224+0000', 'last_undegraded': '2024-04-17T10:42:33.992224+0000', 'last_fullsized': '2024-04-17T10:42:33.992224+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:42:39.759 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.27 is not active+clean 2024-04-17T10:42:39.759 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.988095+0000', 'last_change': '2024-04-17T10:42:33.988095+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.988095+0000', 'last_undegraded': '2024-04-17T10:42:33.988095+0000', 'last_fullsized': '2024-04-17T10:42:33.988095+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:42:39.759 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.26 is not active+clean 2024-04-17T10:42:39.759 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.996585+0000', 'last_change': '2024-04-17T10:42:33.996585+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.996585+0000', 'last_undegraded': '2024-04-17T10:42:33.996585+0000', 'last_fullsized': '2024-04-17T10:42:33.996585+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:42:39.759 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.25 is not active+clean 2024-04-17T10:42:39.760 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.987477+0000', 'last_change': '2024-04-17T10:42:33.987477+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.987477+0000', 'last_undegraded': '2024-04-17T10:42:33.987477+0000', 'last_fullsized': '2024-04-17T10:42:33.987477+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:42:39.760 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.24 is not active+clean 2024-04-17T10:42:39.760 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.985065+0000', 'last_change': '2024-04-17T10:42:33.985065+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.985065+0000', 'last_undegraded': '2024-04-17T10:42:33.985065+0000', 'last_fullsized': '2024-04-17T10:42:33.985065+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:42:39.760 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.23 is not active+clean 2024-04-17T10:42:39.760 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.991668+0000', 'last_change': '2024-04-17T10:42:33.991668+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.991668+0000', 'last_undegraded': '2024-04-17T10:42:33.991668+0000', 'last_fullsized': '2024-04-17T10:42:33.991668+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:42:39.760 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.22 is not active+clean 2024-04-17T10:42:39.760 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.985954+0000', 'last_change': '2024-04-17T10:42:33.985954+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.985954+0000', 'last_undegraded': '2024-04-17T10:42:33.985954+0000', 'last_fullsized': '2024-04-17T10:42:33.985954+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:42:39.760 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.21 is not active+clean 2024-04-17T10:42:39.760 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.984858+0000', 'last_change': '2024-04-17T10:42:33.984858+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.984858+0000', 'last_undegraded': '2024-04-17T10:42:33.984858+0000', 'last_fullsized': '2024-04-17T10:42:33.984858+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:42:39.760 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.20 is not active+clean 2024-04-17T10:42:39.761 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.984313+0000', 'last_change': '2024-04-17T10:42:33.984313+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.984313+0000', 'last_undegraded': '2024-04-17T10:42:33.984313+0000', 'last_fullsized': '2024-04-17T10:42:33.984313+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:42:39.761 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1e is not active+clean 2024-04-17T10:42:39.761 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.987311+0000', 'last_change': '2024-04-17T10:42:33.987311+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.987311+0000', 'last_undegraded': '2024-04-17T10:42:33.987311+0000', 'last_fullsized': '2024-04-17T10:42:33.987311+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:42:39.761 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1f is not active+clean 2024-04-17T10:42:39.761 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.980974+0000', 'last_change': '2024-04-17T10:42:33.980974+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.980974+0000', 'last_undegraded': '2024-04-17T10:42:33.980974+0000', 'last_fullsized': '2024-04-17T10:42:33.980974+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:42:39.761 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1c is not active+clean 2024-04-17T10:42:39.761 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.982188+0000', 'last_change': '2024-04-17T10:42:33.982188+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.982188+0000', 'last_undegraded': '2024-04-17T10:42:33.982188+0000', 'last_fullsized': '2024-04-17T10:42:33.982188+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:42:39.761 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1d is not active+clean 2024-04-17T10:42:39.761 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.987048+0000', 'last_change': '2024-04-17T10:42:33.987048+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.987048+0000', 'last_undegraded': '2024-04-17T10:42:33.987048+0000', 'last_fullsized': '2024-04-17T10:42:33.987048+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:42:39.761 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1a is not active+clean 2024-04-17T10:42:39.761 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.992516+0000', 'last_change': '2024-04-17T10:42:33.992516+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.992516+0000', 'last_undegraded': '2024-04-17T10:42:33.992516+0000', 'last_fullsized': '2024-04-17T10:42:33.992516+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:42:39.762 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1b is not active+clean 2024-04-17T10:42:39.762 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.992432+0000', 'last_change': '2024-04-17T10:42:33.992432+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.992432+0000', 'last_undegraded': '2024-04-17T10:42:33.992432+0000', 'last_fullsized': '2024-04-17T10:42:33.992432+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:42:39.762 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.18 is not active+clean 2024-04-17T10:42:39.762 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.991126+0000', 'last_change': '2024-04-17T10:42:33.991126+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.991126+0000', 'last_undegraded': '2024-04-17T10:42:33.991126+0000', 'last_fullsized': '2024-04-17T10:42:33.991126+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:42:39.762 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.19 is not active+clean 2024-04-17T10:42:39.762 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.990753+0000', 'last_change': '2024-04-17T10:42:33.990753+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.990753+0000', 'last_undegraded': '2024-04-17T10:42:33.990753+0000', 'last_fullsized': '2024-04-17T10:42:33.990753+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:42:39.762 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.16 is not active+clean 2024-04-17T10:42:39.762 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.984503+0000', 'last_change': '2024-04-17T10:42:33.984503+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.984503+0000', 'last_undegraded': '2024-04-17T10:42:33.984503+0000', 'last_fullsized': '2024-04-17T10:42:33.984503+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:42:39.762 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.17 is not active+clean 2024-04-17T10:42:39.762 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.985320+0000', 'last_change': '2024-04-17T10:42:33.985320+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.985320+0000', 'last_undegraded': '2024-04-17T10:42:33.985320+0000', 'last_fullsized': '2024-04-17T10:42:33.985320+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:42:39.763 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.14 is not active+clean 2024-04-17T10:42:39.763 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.981639+0000', 'last_change': '2024-04-17T10:42:33.981639+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.981639+0000', 'last_undegraded': '2024-04-17T10:42:33.981639+0000', 'last_fullsized': '2024-04-17T10:42:33.981639+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:42:39.763 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.15 is not active+clean 2024-04-17T10:42:39.763 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.990052+0000', 'last_change': '2024-04-17T10:42:33.990052+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.990052+0000', 'last_undegraded': '2024-04-17T10:42:33.990052+0000', 'last_fullsized': '2024-04-17T10:42:33.990052+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:42:39.763 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.12 is not active+clean 2024-04-17T10:42:39.763 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.993209+0000', 'last_change': '2024-04-17T10:42:33.993209+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.993209+0000', 'last_undegraded': '2024-04-17T10:42:33.993209+0000', 'last_fullsized': '2024-04-17T10:42:33.993209+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:42:39.763 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.13 is not active+clean 2024-04-17T10:42:39.763 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.983396+0000', 'last_change': '2024-04-17T10:42:33.983396+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.983396+0000', 'last_undegraded': '2024-04-17T10:42:33.983396+0000', 'last_fullsized': '2024-04-17T10:42:33.983396+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:42:39.763 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.10 is not active+clean 2024-04-17T10:42:39.763 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.982143+0000', 'last_change': '2024-04-17T10:42:33.982143+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.982143+0000', 'last_undegraded': '2024-04-17T10:42:33.982143+0000', 'last_fullsized': '2024-04-17T10:42:33.982143+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:42:39.764 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.11 is not active+clean 2024-04-17T10:42:39.764 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.982708+0000', 'last_change': '2024-04-17T10:42:33.982708+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.982708+0000', 'last_undegraded': '2024-04-17T10:42:33.982708+0000', 'last_fullsized': '2024-04-17T10:42:33.982708+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:42:39.764 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.e is not active+clean 2024-04-17T10:42:39.764 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.984738+0000', 'last_change': '2024-04-17T10:42:33.984738+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.984738+0000', 'last_undegraded': '2024-04-17T10:42:33.984738+0000', 'last_fullsized': '2024-04-17T10:42:33.984738+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:42:39.764 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.f is not active+clean 2024-04-17T10:42:39.764 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.999265+0000', 'last_change': '2024-04-17T10:42:33.999265+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.999265+0000', 'last_undegraded': '2024-04-17T10:42:33.999265+0000', 'last_fullsized': '2024-04-17T10:42:33.999265+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:42:39.764 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.c is not active+clean 2024-04-17T10:42:39.764 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.982369+0000', 'last_change': '2024-04-17T10:42:33.982369+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.982369+0000', 'last_undegraded': '2024-04-17T10:42:33.982369+0000', 'last_fullsized': '2024-04-17T10:42:33.982369+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:42:39.764 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.d is not active+clean 2024-04-17T10:42:39.764 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.985866+0000', 'last_change': '2024-04-17T10:42:33.985866+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.985866+0000', 'last_undegraded': '2024-04-17T10:42:33.985866+0000', 'last_fullsized': '2024-04-17T10:42:33.985866+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:42:39.765 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.a is not active+clean 2024-04-17T10:42:39.765 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.991626+0000', 'last_change': '2024-04-17T10:42:33.991626+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.991626+0000', 'last_undegraded': '2024-04-17T10:42:33.991626+0000', 'last_fullsized': '2024-04-17T10:42:33.991626+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:42:39.765 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.b is not active+clean 2024-04-17T10:42:39.765 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.982157+0000', 'last_change': '2024-04-17T10:42:33.982157+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.982157+0000', 'last_undegraded': '2024-04-17T10:42:33.982157+0000', 'last_fullsized': '2024-04-17T10:42:33.982157+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:42:39.765 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.9 is not active+clean 2024-04-17T10:42:39.765 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.984120+0000', 'last_change': '2024-04-17T10:42:33.984120+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.984120+0000', 'last_undegraded': '2024-04-17T10:42:33.984120+0000', 'last_fullsized': '2024-04-17T10:42:33.984120+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:42:39.765 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.6 is not active+clean 2024-04-17T10:42:39.765 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.995538+0000', 'last_change': '2024-04-17T10:42:33.995538+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.995538+0000', 'last_undegraded': '2024-04-17T10:42:33.995538+0000', 'last_fullsized': '2024-04-17T10:42:33.995538+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:42:39.765 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.7 is not active+clean 2024-04-17T10:42:39.765 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.998164+0000', 'last_change': '2024-04-17T10:42:33.998164+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.998164+0000', 'last_undegraded': '2024-04-17T10:42:33.998164+0000', 'last_fullsized': '2024-04-17T10:42:33.998164+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:42:39.766 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.4 is not active+clean 2024-04-17T10:42:39.766 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.979958+0000', 'last_change': '2024-04-17T10:42:33.979958+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.979958+0000', 'last_undegraded': '2024-04-17T10:42:33.979958+0000', 'last_fullsized': '2024-04-17T10:42:33.979958+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:42:39.766 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.5 is not active+clean 2024-04-17T10:42:39.766 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.982616+0000', 'last_change': '2024-04-17T10:42:33.982616+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.982616+0000', 'last_undegraded': '2024-04-17T10:42:33.982616+0000', 'last_fullsized': '2024-04-17T10:42:33.982616+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:42:39.766 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2 is not active+clean 2024-04-17T10:42:39.766 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.982460+0000', 'last_change': '2024-04-17T10:42:33.982460+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.982460+0000', 'last_undegraded': '2024-04-17T10:42:33.982460+0000', 'last_fullsized': '2024-04-17T10:42:33.982460+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:42:39.766 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3 is not active+clean 2024-04-17T10:42:39.766 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.982097+0000', 'last_change': '2024-04-17T10:42:33.982097+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.982097+0000', 'last_undegraded': '2024-04-17T10:42:33.982097+0000', 'last_fullsized': '2024-04-17T10:42:33.982097+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:42:39.766 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.0 is not active+clean 2024-04-17T10:42:39.766 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.979789+0000', 'last_change': '2024-04-17T10:42:33.979789+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.979789+0000', 'last_undegraded': '2024-04-17T10:42:33.979789+0000', 'last_fullsized': '2024-04-17T10:42:33.979789+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:42:39.767 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1 is not active+clean 2024-04-17T10:42:39.767 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.982429+0000', 'last_change': '2024-04-17T10:42:33.982429+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.982429+0000', 'last_undegraded': '2024-04-17T10:42:33.982429+0000', 'last_fullsized': '2024-04-17T10:42:33.982429+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:42:39.767 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.8 is not active+clean 2024-04-17T10:42:39.767 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:42:33.992302+0000', 'last_change': '2024-04-17T10:42:33.992302+0000', 'last_active': '2024-04-17T10:42:33.961433+0000', 'last_peered': '2024-04-17T10:42:33.961433+0000', 'last_clean': '2024-04-17T10:42:33.961433+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:42:33.992302+0000', 'last_undegraded': '2024-04-17T10:42:33.992302+0000', 'last_fullsized': '2024-04-17T10:42:33.992302+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'last_clean_scrub_stamp': '2024-04-17T10:42:33.961433+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:42:39.767 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-17T10:42:39.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.909+0000 7f89fee23640 1 Processor -- start 2024-04-17T10:42:39.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.909+0000 7f89fee23640 1 -- start start 2024-04-17T10:42:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.909+0000 7f89fee23640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f89f80ce6f0 0x7f89f806abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.909+0000 7f89fee23640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f89f80cead0 0x7f89f806b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.909+0000 7f89fee23640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89f80713a0 0x7f89f8161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:39.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.909+0000 7f89fee23640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f8072470 con 0x7f89f80ce6f0 2024-04-17T10:42:39.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.909+0000 7f89fee23640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f80718f0 con 0x7f89f80cead0 2024-04-17T10:42:39.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.909+0000 7f89fee23640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f8071bd0 con 0x7f89f80713a0 2024-04-17T10:42:39.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.909+0000 7f89fcbbf640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f89f80ce6f0 0x7f89f806abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:39.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.909+0000 7f89fcbbf640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f89f80ce6f0 0x7f89f806abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36822/0 (socket says 172.21.15.83:36822) 2024-04-17T10:42:39.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.909+0000 7f89fcbbf640 1 -- 172.21.15.83:0/1657515377 learned_addr learned my addr 172.21.15.83:0/1657515377 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:39.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.909+0000 7f89fd3c0640 1 --2- 172.21.15.83:0/1657515377 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89f80713a0 0x7f89f8161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:39.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.909+0000 7f89f7fff640 1 --2- 172.21.15.83:0/1657515377 >> v2:172.21.15.167:3300/0 conn(0x7f89f80cead0 0x7f89f806b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:39.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.909+0000 7f89fcbbf640 1 -- 172.21.15.83:0/1657515377 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89f80713a0 msgr2=0x7f89f8161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:39.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.909+0000 7f89fcbbf640 1 --2- 172.21.15.83:0/1657515377 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89f80713a0 0x7f89f8161790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:39.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.909+0000 7f89fcbbf640 1 -- 172.21.15.83:0/1657515377 >> v2:172.21.15.167:3300/0 conn(0x7f89f80cead0 msgr2=0x7f89f806b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:39.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.909+0000 7f89fcbbf640 1 --2- 172.21.15.83:0/1657515377 >> v2:172.21.15.167:3300/0 conn(0x7f89f80cead0 0x7f89f806b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:39.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.909+0000 7f89fcbbf640 1 -- 172.21.15.83:0/1657515377 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f8071eb0 con 0x7f89f80ce6f0 2024-04-17T10:42:39.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.909+0000 7f89f7fff640 1 --2- 172.21.15.83:0/1657515377 >> v2:172.21.15.167:3300/0 conn(0x7f89f80cead0 0x7f89f806b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:39.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.909+0000 7f89fcbbf640 1 --2- 172.21.15.83:0/1657515377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89f80ce6f0 0x7f89f806abe0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f89f80b68a0 tx=0x7f89e801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab4e062571b5b43f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:39.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.909+0000 7f89f77fe640 1 -- 172.21.15.83:0/1657515377 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e80ab640 con 0x7f89f80ce6f0 2024-04-17T10:42:39.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.909+0000 7f89f77fe640 1 -- 172.21.15.83:0/1657515377 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e80b0070 con 0x7f89f80ce6f0 2024-04-17T10:42:39.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.909+0000 7f89f77fe640 1 -- 172.21.15.83:0/1657515377 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e80aba00 con 0x7f89f80ce6f0 2024-04-17T10:42:39.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89fee23640 1 -- 172.21.15.83:0/1657515377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89f80ce6f0 msgr2=0x7f89f806abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:39.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89fee23640 1 --2- 172.21.15.83:0/1657515377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89f80ce6f0 0x7f89f806abe0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f89f80b68a0 tx=0x7f89e801c460 comp rx=0 tx=0).stop 2024-04-17T10:42:39.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89fee23640 1 -- 172.21.15.83:0/1657515377 shutdown_connections 2024-04-17T10:42:39.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89fee23640 1 --2- 172.21.15.83:0/1657515377 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89f80713a0 0x7f89f8161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:39.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89fee23640 1 --2- 172.21.15.83:0/1657515377 >> v2:172.21.15.167:3300/0 conn(0x7f89f80cead0 0x7f89f806b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:39.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89fee23640 1 --2- 172.21.15.83:0/1657515377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89f80ce6f0 0x7f89f806abe0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:39.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89fee23640 1 -- 172.21.15.83:0/1657515377 >> 172.21.15.83:0/1657515377 conn(0x7f89f806bcf0 msgr2=0x7f89f806c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:39.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89fee23640 1 -- 172.21.15.83:0/1657515377 shutdown_connections 2024-04-17T10:42:39.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89fee23640 1 -- 172.21.15.83:0/1657515377 wait complete. 2024-04-17T10:42:39.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89fee23640 1 Processor -- start 2024-04-17T10:42:39.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89fee23640 1 -- start start 2024-04-17T10:42:39.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89fee23640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89f80713a0 0x7f89f814e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:39.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89fee23640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89f80ce6f0 0x7f89f8140650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:39.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89fee23640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89f80cead0 0x7f89f8140b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:39.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89fee23640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f80c5c50 con 0x7f89f80cead0 2024-04-17T10:42:39.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89fee23640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f80c5ad0 con 0x7f89f80ce6f0 2024-04-17T10:42:39.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89fee23640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f80c5dd0 con 0x7f89f80713a0 2024-04-17T10:42:39.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89fd3c0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89f80cead0 0x7f89f8140b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:39.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89fd3c0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89f80cead0 0x7f89f8140b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36838/0 (socket says 172.21.15.83:36838) 2024-04-17T10:42:39.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89fd3c0640 1 -- 172.21.15.83:0/1938817942 learned_addr learned my addr 172.21.15.83:0/1938817942 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:39.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89fcbbf640 1 --2- 172.21.15.83:0/1938817942 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89f80713a0 0x7f89f814e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:39.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89f7fff640 1 --2- 172.21.15.83:0/1938817942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89f80ce6f0 0x7f89f8140650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:39.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89fd3c0640 1 -- 172.21.15.83:0/1938817942 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89f80713a0 msgr2=0x7f89f814e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:39.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89fd3c0640 1 --2- 172.21.15.83:0/1938817942 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89f80713a0 0x7f89f814e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:39.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89fd3c0640 1 -- 172.21.15.83:0/1938817942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89f80ce6f0 msgr2=0x7f89f8140650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:39.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89fd3c0640 1 --2- 172.21.15.83:0/1938817942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89f80ce6f0 0x7f89f8140650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:39.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89fd3c0640 1 -- 172.21.15.83:0/1938817942 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f814e9b0 con 0x7f89f80cead0 2024-04-17T10:42:39.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89fd3c0640 1 --2- 172.21.15.83:0/1938817942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89f80cead0 0x7f89f8140b90 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f89f0007e70 tx=0x7f89f000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:39.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89f57fa640 1 -- 172.21.15.83:0/1938817942 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f0014620 con 0x7f89f80cead0 2024-04-17T10:42:39.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89f57fa640 1 -- 172.21.15.83:0/1938817942 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f0019070 con 0x7f89f80cead0 2024-04-17T10:42:39.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89fee23640 1 -- 172.21.15.83:0/1938817942 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f80cc020 con 0x7f89f80cead0 2024-04-17T10:42:39.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89f57fa640 1 -- 172.21.15.83:0/1938817942 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f0014a20 con 0x7f89f80cead0 2024-04-17T10:42:39.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.913+0000 7f89fee23640 1 -- 172.21.15.83:0/1938817942 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f80cc5e0 con 0x7f89f80cead0 2024-04-17T10:42:39.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.917+0000 7f89fee23640 1 -- 172.21.15.83:0/1938817942 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f8074f90 con 0x7f89f80cead0 2024-04-17T10:42:39.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.917+0000 7f89f57fa640 1 -- 172.21.15.83:0/1938817942 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f89f0014bc0 con 0x7f89f80cead0 2024-04-17T10:42:39.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.917+0000 7f89f57fa640 1 --2- 172.21.15.83:0/1938817942 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f89e4066f60 0x7f89e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:39.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.917+0000 7f89f57fa640 1 -- 172.21.15.83:0/1938817942 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f89f0085960 con 0x7f89f80cead0 2024-04-17T10:42:39.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.917+0000 7f89fcbbf640 1 --2- 172.21.15.83:0/1938817942 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f89e4066f60 0x7f89e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:39.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.917+0000 7f89fcbbf640 1 --2- 172.21.15.83:0/1938817942 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f89e4066f60 0x7f89e4069420 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f89e80af000 tx=0x7f89e80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:39.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:39.921+0000 7f89f57fa640 1 -- 172.21.15.83:0/1938817942 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f89f0011260 con 0x7f89f80cead0 2024-04-17T10:42:40.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.121+0000 7f89fee23640 1 -- 172.21.15.83:0/1938817942 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f89f80b0f00 con 0x7f89f80cead0 2024-04-17T10:42:40.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.121+0000 7f89f57fa640 1 -- 172.21.15.83:0/1938817942 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1454 (secure 0 0 0) 0x7f89f00568e0 con 0x7f89f80cead0 2024-04-17T10:42:40.121 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:40.121 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":292,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":8,"num_up_osds":8,"osd_up_since":1713350279,"num_in_osds":8,"osd_in_since":1713350270,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":364367872,"bytes_avail":772729745408,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":32,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"b","status":"up:active","gid":6534}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-04-17T10:42:24.256057+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-17T10:42:40.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.121+0000 7f89fee23640 1 -- 172.21.15.83:0/1938817942 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f89e4066f60 msgr2=0x7f89e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:40.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.121+0000 7f89fee23640 1 --2- 172.21.15.83:0/1938817942 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f89e4066f60 0x7f89e4069420 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f89e80af000 tx=0x7f89e80a7000 comp rx=0 tx=0).stop 2024-04-17T10:42:40.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.121+0000 7f89fee23640 1 -- 172.21.15.83:0/1938817942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89f80cead0 msgr2=0x7f89f8140b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:40.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.121+0000 7f89fee23640 1 --2- 172.21.15.83:0/1938817942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89f80cead0 0x7f89f8140b90 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f89f0007e70 tx=0x7f89f000e4b0 comp rx=0 tx=0).stop 2024-04-17T10:42:40.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.121+0000 7f89fee23640 1 -- 172.21.15.83:0/1938817942 shutdown_connections 2024-04-17T10:42:40.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.121+0000 7f89fee23640 1 --2- 172.21.15.83:0/1938817942 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f89e4066f60 0x7f89e4069420 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:40.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.121+0000 7f89fee23640 1 --2- 172.21.15.83:0/1938817942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89f80cead0 0x7f89f8140b90 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:40.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.121+0000 7f89fee23640 1 --2- 172.21.15.83:0/1938817942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89f80ce6f0 0x7f89f8140650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:40.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.121+0000 7f89fee23640 1 --2- 172.21.15.83:0/1938817942 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89f80713a0 0x7f89f814e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:40.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.121+0000 7f89fee23640 1 -- 172.21.15.83:0/1938817942 >> 172.21.15.83:0/1938817942 conn(0x7f89f806bcf0 msgr2=0x7f89f80b1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:40.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.121+0000 7f89fee23640 1 -- 172.21.15.83:0/1938817942 shutdown_connections 2024-04-17T10:42:40.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.121+0000 7f89fee23640 1 -- 172.21.15.83:0/1938817942 wait complete. 2024-04-17T10:42:40.139 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-17T10:42:40.139 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:42:40.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.261+0000 7f9ffaaf3640 1 Processor -- start 2024-04-17T10:42:40.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.261+0000 7f9ffaaf3640 1 -- start start 2024-04-17T10:42:40.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.261+0000 7f9ffaaf3640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9ff415f1d0 0x7f9ff415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:40.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.261+0000 7f9ffaaf3640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ff40d22d0 0x7f9ff40ae520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:40.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.261+0000 7f9ffaaf3640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ff415f5b0 con 0x7f9ff40b2de0 2024-04-17T10:42:40.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.261+0000 7f9ffaaf3640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ff40b2620 con 0x7f9ff415f1d0 2024-04-17T10:42:40.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.261+0000 7f9ffaaf3640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff40b2340 con 0x7f9ff40d22d0 2024-04-17T10:42:40.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.261+0000 7f9ff888f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9ff415f1d0 0x7f9ff415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:40.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.261+0000 7f9ff888f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9ff415f1d0 0x7f9ff415f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36842/0 (socket says 172.21.15.83:36842) 2024-04-17T10:42:40.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.261+0000 7f9ff888f640 1 -- 172.21.15.83:0/181110963 learned_addr learned my addr 172.21.15.83:0/181110963 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:40.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.261+0000 7f9ff9090640 1 --2- 172.21.15.83:0/181110963 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ff40d22d0 0x7f9ff40ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:40.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.261+0000 7f9ff888f640 1 -- 172.21.15.83:0/181110963 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ff40d22d0 msgr2=0x7f9ff40ae520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:40.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.261+0000 7f9ff888f640 1 --2- 172.21.15.83:0/181110963 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ff40d22d0 0x7f9ff40ae520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:40.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.261+0000 7f9ff888f640 1 -- 172.21.15.83:0/181110963 >> v1:172.21.15.167:6789/0 conn(0x7f9ff40b2de0 legacy=0x7f9ff415fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:40.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.261+0000 7f9ff888f640 1 -- 172.21.15.83:0/181110963 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff40b1760 con 0x7f9ff415f1d0 2024-04-17T10:42:40.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ff888f640 1 --2- 172.21.15.83:0/181110963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ff415f1d0 0x7f9ff415f830 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4009870 tx=0x7f9fe401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b1e479175962b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:40.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ff37fe640 1 -- 172.21.15.83:0/181110963 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe40aa610 con 0x7f9ff415f1d0 2024-04-17T10:42:40.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ff37fe640 1 -- 172.21.15.83:0/181110963 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe40af070 con 0x7f9ff415f1d0 2024-04-17T10:42:40.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ff37fe640 1 -- 172.21.15.83:0/181110963 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe40aa9d0 con 0x7f9ff415f1d0 2024-04-17T10:42:40.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ffaaf3640 1 -- 172.21.15.83:0/181110963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ff415f1d0 msgr2=0x7f9ff415f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:40.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ffaaf3640 1 --2- 172.21.15.83:0/181110963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ff415f1d0 0x7f9ff415f830 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4009870 tx=0x7f9fe401c4f0 comp rx=0 tx=0).stop 2024-04-17T10:42:40.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ffaaf3640 1 -- 172.21.15.83:0/181110963 shutdown_connections 2024-04-17T10:42:40.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ffaaf3640 1 --2- 172.21.15.83:0/181110963 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ff40d22d0 0x7f9ff40ae520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:40.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ffaaf3640 1 --2- 172.21.15.83:0/181110963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ff415f1d0 0x7f9ff415f830 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:40.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ffaaf3640 1 -- 172.21.15.83:0/181110963 >> 172.21.15.83:0/181110963 conn(0x7f9ff40d17b0 msgr2=0x7f9ff40d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:40.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ffaaf3640 1 -- 172.21.15.83:0/181110963 shutdown_connections 2024-04-17T10:42:40.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ffaaf3640 1 -- 172.21.15.83:0/181110963 wait complete. 2024-04-17T10:42:40.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ffaaf3640 1 Processor -- start 2024-04-17T10:42:40.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ffaaf3640 1 -- start start 2024-04-17T10:42:40.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ffaaf3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ff40d22d0 0x7f9ff415e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:40.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ffaaf3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ff415f1d0 0x7f9ff4150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:40.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ffaaf3640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ff40c6950 0x7f9ff415ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:40.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ffaaf3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff40af270 con 0x7f9ff40d22d0 2024-04-17T10:42:40.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ffaaf3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff4160690 con 0x7f9ff415f1d0 2024-04-17T10:42:40.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ffaaf3640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff40af3f0 con 0x7f9ff40c6950 2024-04-17T10:42:40.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ff888f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ff40d22d0 0x7f9ff415e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:40.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ff888f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ff40d22d0 0x7f9ff415e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36846/0 (socket says 172.21.15.83:36846) 2024-04-17T10:42:40.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ff888f640 1 -- 172.21.15.83:0/2184067773 learned_addr learned my addr 172.21.15.83:0/2184067773 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:40.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ff9090640 1 --2- 172.21.15.83:0/2184067773 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ff40c6950 0x7f9ff415ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:40.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ff3fff640 1 --2- 172.21.15.83:0/2184067773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ff415f1d0 0x7f9ff4150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:40.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ff888f640 1 -- 172.21.15.83:0/2184067773 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ff40c6950 msgr2=0x7f9ff415ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:40.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ff888f640 1 --2- 172.21.15.83:0/2184067773 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ff40c6950 0x7f9ff415ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:40.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ff888f640 1 -- 172.21.15.83:0/2184067773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ff415f1d0 msgr2=0x7f9ff4150dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:40.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ff888f640 1 --2- 172.21.15.83:0/2184067773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ff415f1d0 0x7f9ff4150dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:40.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ff888f640 1 -- 172.21.15.83:0/2184067773 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff406db00 con 0x7f9ff40d22d0 2024-04-17T10:42:40.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ff9090640 1 --2- 172.21.15.83:0/2184067773 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ff40c6950 0x7f9ff415ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:40.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ff888f640 1 --2- 172.21.15.83:0/2184067773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ff40d22d0 0x7f9ff415e130 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f9fe40097c0 tx=0x7f9fe401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:40.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.265+0000 7f9ff17fa640 1 -- 172.21.15.83:0/2184067773 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe40bd5f0 con 0x7f9ff40d22d0 2024-04-17T10:42:40.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.269+0000 7f9ff17fa640 1 -- 172.21.15.83:0/2184067773 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe40bd7b0 con 0x7f9ff40d22d0 2024-04-17T10:42:40.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.269+0000 7f9ffaaf3640 1 -- 172.21.15.83:0/2184067773 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff40c5e70 con 0x7f9ff40d22d0 2024-04-17T10:42:40.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.269+0000 7f9ff17fa640 1 -- 172.21.15.83:0/2184067773 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe40bdba0 con 0x7f9ff40d22d0 2024-04-17T10:42:40.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.269+0000 7f9ffaaf3640 1 -- 172.21.15.83:0/2184067773 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff4151300 con 0x7f9ff40d22d0 2024-04-17T10:42:40.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.269+0000 7f9ffaaf3640 1 -- 172.21.15.83:0/2184067773 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ff4074f90 con 0x7f9ff40d22d0 2024-04-17T10:42:40.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.269+0000 7f9ff17fa640 1 -- 172.21.15.83:0/2184067773 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9fe40b33f0 con 0x7f9ff40d22d0 2024-04-17T10:42:40.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.269+0000 7f9ff17fa640 1 --2- 172.21.15.83:0/2184067773 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9fe0066f90 0x7f9fe0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:40.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.269+0000 7f9ff3fff640 1 --2- 172.21.15.83:0/2184067773 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9fe0066f90 0x7f9fe0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:40.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.269+0000 7f9ff17fa640 1 -- 172.21.15.83:0/2184067773 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9fe411f4d0 con 0x7f9ff40d22d0 2024-04-17T10:42:40.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.269+0000 7f9ff3fff640 1 --2- 172.21.15.83:0/2184067773 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9fe0066f90 0x7f9fe0069450 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9ff41fa160 tx=0x7f9fe8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:40.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.273+0000 7f9ff17fa640 1 -- 172.21.15.83:0/2184067773 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9fe40c3060 con 0x7f9ff40d22d0 2024-04-17T10:42:40.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.429+0000 7f9ffaaf3640 1 -- 172.21.15.83:0/2184067773 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9ff4072690 con 0x7f9fe0066f90 2024-04-17T10:42:40.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.445+0000 7f9ff17fa640 1 -- 172.21.15.83:0/2184067773 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251965 (secure 0 0 0) 0x7f9ff4072690 con 0x7f9fe0066f90 2024-04-17T10:42:40.451 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:40.452 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:42:40.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.449+0000 7f9ffaaf3640 1 -- 172.21.15.83:0/2184067773 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9fe0066f90 msgr2=0x7f9fe0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:40.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.449+0000 7f9ffaaf3640 1 --2- 172.21.15.83:0/2184067773 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9fe0066f90 0x7f9fe0069450 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9ff41fa160 tx=0x7f9fe8009040 comp rx=0 tx=0).stop 2024-04-17T10:42:40.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.449+0000 7f9ffaaf3640 1 -- 172.21.15.83:0/2184067773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ff40d22d0 msgr2=0x7f9ff415e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:40.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.449+0000 7f9ffaaf3640 1 --2- 172.21.15.83:0/2184067773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ff40d22d0 0x7f9ff415e130 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f9fe40097c0 tx=0x7f9fe401ad30 comp rx=0 tx=0).stop 2024-04-17T10:42:40.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.449+0000 7f9ffaaf3640 1 -- 172.21.15.83:0/2184067773 shutdown_connections 2024-04-17T10:42:40.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.449+0000 7f9ffaaf3640 1 --2- 172.21.15.83:0/2184067773 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9fe0066f90 0x7f9fe0069450 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:40.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.449+0000 7f9ffaaf3640 1 --2- 172.21.15.83:0/2184067773 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ff40c6950 0x7f9ff415ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:40.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.449+0000 7f9ffaaf3640 1 --2- 172.21.15.83:0/2184067773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ff415f1d0 0x7f9ff4150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:40.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.449+0000 7f9ffaaf3640 1 --2- 172.21.15.83:0/2184067773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ff40d22d0 0x7f9ff415e130 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:40.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.449+0000 7f9ffaaf3640 1 -- 172.21.15.83:0/2184067773 >> 172.21.15.83:0/2184067773 conn(0x7f9ff40d17b0 msgr2=0x7f9ff40d1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:40.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.449+0000 7f9ffaaf3640 1 -- 172.21.15.83:0/2184067773 shutdown_connections 2024-04-17T10:42:40.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:40.449+0000 7f9ffaaf3640 1 -- 172.21.15.83:0/2184067773 wait complete. 2024-04-17T10:42:40.466 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2024-04-17T10:42:40.259830+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":180,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":356140,"kb_used_data":6760,"kb_used_omap":12,"kb_used_meta":349235,"kb_avail":754618580,"statfs":{"total":773094113280,"available":772729425920,"internally_reserved":0,"allocated":6922240,"data_stored":3565484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":357617200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.283398"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.411826+0000","last_change":"2024-04-17T10:42:35.025913+0000","last_active":"2024-04-17T10:42:36.411826+0000","last_peered":"2024-04-17T10:42:36.411826+0000","last_clean":"2024-04-17T10:42:36.411826+0000","last_became_active":"2024-04-17T10:42:35.025299+0000","last_became_peered":"2024-04-17T10:42:35.025299+0000","last_unstale":"2024-04-17T10:42:36.411826+0000","last_undegraded":"2024-04-17T10:42:36.411826+0000","last_fullsized":"2024-04-17T10:42:36.411826+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:05:08.176877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.340051+0000","last_change":"2024-04-17T10:42:35.028596+0000","last_active":"2024-04-17T10:42:36.340051+0000","last_peered":"2024-04-17T10:42:36.340051+0000","last_clean":"2024-04-17T10:42:36.340051+0000","last_became_active":"2024-04-17T10:42:35.027377+0000","last_became_peered":"2024-04-17T10:42:35.027377+0000","last_unstale":"2024-04-17T10:42:36.340051+0000","last_undegraded":"2024-04-17T10:42:36.340051+0000","last_fullsized":"2024-04-17T10:42:36.340051+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:53:55.930528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.992426+0000","last_change":"2024-04-17T10:42:35.010816+0000","last_active":"2024-04-17T10:42:35.992426+0000","last_peered":"2024-04-17T10:42:35.992426+0000","last_clean":"2024-04-17T10:42:35.992426+0000","last_became_active":"2024-04-17T10:42:35.010365+0000","last_became_peered":"2024-04-17T10:42:35.010365+0000","last_unstale":"2024-04-17T10:42:35.992426+0000","last_undegraded":"2024-04-17T10:42:35.992426+0000","last_fullsized":"2024-04-17T10:42:35.992426+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:04:05.170981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.412599+0000","last_change":"2024-04-17T10:42:35.031696+0000","last_active":"2024-04-17T10:42:36.412599+0000","last_peered":"2024-04-17T10:42:36.412599+0000","last_clean":"2024-04-17T10:42:36.412599+0000","last_became_active":"2024-04-17T10:42:35.031205+0000","last_became_peered":"2024-04-17T10:42:35.031205+0000","last_unstale":"2024-04-17T10:42:36.412599+0000","last_undegraded":"2024-04-17T10:42:36.412599+0000","last_fullsized":"2024-04-17T10:42:36.412599+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:37:35.341273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.412273+0000","last_change":"2024-04-17T10:42:35.022055+0000","last_active":"2024-04-17T10:42:36.412273+0000","last_peered":"2024-04-17T10:42:36.412273+0000","last_clean":"2024-04-17T10:42:36.412273+0000","last_became_active":"2024-04-17T10:42:35.021773+0000","last_became_peered":"2024-04-17T10:42:35.021773+0000","last_unstale":"2024-04-17T10:42:36.412273+0000","last_undegraded":"2024-04-17T10:42:36.412273+0000","last_fullsized":"2024-04-17T10:42:36.412273+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:39:07.167173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.986747+0000","last_change":"2024-04-17T10:42:35.010720+0000","last_active":"2024-04-17T10:42:35.986747+0000","last_peered":"2024-04-17T10:42:35.986747+0000","last_clean":"2024-04-17T10:42:35.986747+0000","last_became_active":"2024-04-17T10:42:35.010119+0000","last_became_peered":"2024-04-17T10:42:35.010119+0000","last_unstale":"2024-04-17T10:42:35.986747+0000","last_undegraded":"2024-04-17T10:42:35.986747+0000","last_fullsized":"2024-04-17T10:42:35.986747+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:43:28.482310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.989458+0000","last_change":"2024-04-17T10:42:35.030532+0000","last_active":"2024-04-17T10:42:35.989458+0000","last_peered":"2024-04-17T10:42:35.989458+0000","last_clean":"2024-04-17T10:42:35.989458+0000","last_became_active":"2024-04-17T10:42:35.029381+0000","last_became_peered":"2024-04-17T10:42:35.029381+0000","last_unstale":"2024-04-17T10:42:35.989458+0000","last_undegraded":"2024-04-17T10:42:35.989458+0000","last_fullsized":"2024-04-17T10:42:35.989458+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:32:39.595630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.339360+0000","last_change":"2024-04-17T10:42:35.018376+0000","last_active":"2024-04-17T10:42:36.339360+0000","last_peered":"2024-04-17T10:42:36.339360+0000","last_clean":"2024-04-17T10:42:36.339360+0000","last_became_active":"2024-04-17T10:42:35.018017+0000","last_became_peered":"2024-04-17T10:42:35.018017+0000","last_unstale":"2024-04-17T10:42:36.339360+0000","last_undegraded":"2024-04-17T10:42:36.339360+0000","last_fullsized":"2024-04-17T10:42:36.339360+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:27:46.041944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.990864+0000","last_change":"2024-04-17T10:42:35.024624+0000","last_active":"2024-04-17T10:42:35.990864+0000","last_peered":"2024-04-17T10:42:35.990864+0000","last_clean":"2024-04-17T10:42:35.990864+0000","last_became_active":"2024-04-17T10:42:35.022047+0000","last_became_peered":"2024-04-17T10:42:35.022047+0000","last_unstale":"2024-04-17T10:42:35.990864+0000","last_undegraded":"2024-04-17T10:42:35.990864+0000","last_fullsized":"2024-04-17T10:42:35.990864+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:01:34.922297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.411613+0000","last_change":"2024-04-17T10:42:35.006487+0000","last_active":"2024-04-17T10:42:36.411613+0000","last_peered":"2024-04-17T10:42:36.411613+0000","last_clean":"2024-04-17T10:42:36.411613+0000","last_became_active":"2024-04-17T10:42:35.005168+0000","last_became_peered":"2024-04-17T10:42:35.005168+0000","last_unstale":"2024-04-17T10:42:36.411613+0000","last_undegraded":"2024-04-17T10:42:36.411613+0000","last_fullsized":"2024-04-17T10:42:36.411613+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:26:58.999145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.411768+0000","last_change":"2024-04-17T10:42:35.026150+0000","last_active":"2024-04-17T10:42:36.411768+0000","last_peered":"2024-04-17T10:42:36.411768+0000","last_clean":"2024-04-17T10:42:36.411768+0000","last_became_active":"2024-04-17T10:42:35.025867+0000","last_became_peered":"2024-04-17T10:42:35.025867+0000","last_unstale":"2024-04-17T10:42:36.411768+0000","last_undegraded":"2024-04-17T10:42:36.411768+0000","last_fullsized":"2024-04-17T10:42:36.411768+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:58:19.216435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.412734+0000","last_change":"2024-04-17T10:42:35.029601+0000","last_active":"2024-04-17T10:42:36.412734+0000","last_peered":"2024-04-17T10:42:36.412734+0000","last_clean":"2024-04-17T10:42:36.412734+0000","last_became_active":"2024-04-17T10:42:35.027716+0000","last_became_peered":"2024-04-17T10:42:35.027716+0000","last_unstale":"2024-04-17T10:42:36.412734+0000","last_undegraded":"2024-04-17T10:42:36.412734+0000","last_fullsized":"2024-04-17T10:42:36.412734+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:29:43.713428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.007304+0000","last_change":"2024-04-17T10:42:35.016481+0000","last_active":"2024-04-17T10:42:36.007304+0000","last_peered":"2024-04-17T10:42:36.007304+0000","last_clean":"2024-04-17T10:42:36.007304+0000","last_became_active":"2024-04-17T10:42:35.016153+0000","last_became_peered":"2024-04-17T10:42:35.016153+0000","last_unstale":"2024-04-17T10:42:36.007304+0000","last_undegraded":"2024-04-17T10:42:36.007304+0000","last_fullsized":"2024-04-17T10:42:36.007304+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:47:15.899420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.989807+0000","last_change":"2024-04-17T10:42:35.008905+0000","last_active":"2024-04-17T10:42:35.989807+0000","last_peered":"2024-04-17T10:42:35.989807+0000","last_clean":"2024-04-17T10:42:35.989807+0000","last_became_active":"2024-04-17T10:42:35.008293+0000","last_became_peered":"2024-04-17T10:42:35.008293+0000","last_unstale":"2024-04-17T10:42:35.989807+0000","last_undegraded":"2024-04-17T10:42:35.989807+0000","last_fullsized":"2024-04-17T10:42:35.989807+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:41:53.757975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.989763+0000","last_change":"2024-04-17T10:42:35.024840+0000","last_active":"2024-04-17T10:42:35.989763+0000","last_peered":"2024-04-17T10:42:35.989763+0000","last_clean":"2024-04-17T10:42:35.989763+0000","last_became_active":"2024-04-17T10:42:35.022239+0000","last_became_peered":"2024-04-17T10:42:35.022239+0000","last_unstale":"2024-04-17T10:42:35.989763+0000","last_undegraded":"2024-04-17T10:42:35.989763+0000","last_fullsized":"2024-04-17T10:42:35.989763+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:33:42.502740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.986677+0000","last_change":"2024-04-17T10:42:35.018664+0000","last_active":"2024-04-17T10:42:35.986677+0000","last_peered":"2024-04-17T10:42:35.986677+0000","last_clean":"2024-04-17T10:42:35.986677+0000","last_became_active":"2024-04-17T10:42:35.018257+0000","last_became_peered":"2024-04-17T10:42:35.018257+0000","last_unstale":"2024-04-17T10:42:35.986677+0000","last_undegraded":"2024-04-17T10:42:35.986677+0000","last_fullsized":"2024-04-17T10:42:35.986677+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:35:46.679023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.339968+0000","last_change":"2024-04-17T10:42:35.025373+0000","last_active":"2024-04-17T10:42:36.339968+0000","last_peered":"2024-04-17T10:42:36.339968+0000","last_clean":"2024-04-17T10:42:36.339968+0000","last_became_active":"2024-04-17T10:42:35.025055+0000","last_became_peered":"2024-04-17T10:42:35.025055+0000","last_unstale":"2024-04-17T10:42:36.339968+0000","last_undegraded":"2024-04-17T10:42:36.339968+0000","last_fullsized":"2024-04-17T10:42:36.339968+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:04:47.145034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.992756+0000","last_change":"2024-04-17T10:42:35.017543+0000","last_active":"2024-04-17T10:42:35.992756+0000","last_peered":"2024-04-17T10:42:35.992756+0000","last_clean":"2024-04-17T10:42:35.992756+0000","last_became_active":"2024-04-17T10:42:35.017003+0000","last_became_peered":"2024-04-17T10:42:35.017003+0000","last_unstale":"2024-04-17T10:42:35.992756+0000","last_undegraded":"2024-04-17T10:42:35.992756+0000","last_fullsized":"2024-04-17T10:42:35.992756+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:33:02.276726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.411690+0000","last_change":"2024-04-17T10:42:35.015960+0000","last_active":"2024-04-17T10:42:36.411690+0000","last_peered":"2024-04-17T10:42:36.411690+0000","last_clean":"2024-04-17T10:42:36.411690+0000","last_became_active":"2024-04-17T10:42:35.015589+0000","last_became_peered":"2024-04-17T10:42:35.015589+0000","last_unstale":"2024-04-17T10:42:36.411690+0000","last_undegraded":"2024-04-17T10:42:36.411690+0000","last_fullsized":"2024-04-17T10:42:36.411690+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:10:09.266430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.340151+0000","last_change":"2024-04-17T10:42:35.027129+0000","last_active":"2024-04-17T10:42:36.340151+0000","last_peered":"2024-04-17T10:42:36.340151+0000","last_clean":"2024-04-17T10:42:36.340151+0000","last_became_active":"2024-04-17T10:42:35.026663+0000","last_became_peered":"2024-04-17T10:42:35.026663+0000","last_unstale":"2024-04-17T10:42:36.340151+0000","last_undegraded":"2024-04-17T10:42:36.340151+0000","last_fullsized":"2024-04-17T10:42:36.340151+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:41:27.625355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.992079+0000","last_change":"2024-04-17T10:42:35.024264+0000","last_active":"2024-04-17T10:42:35.992079+0000","last_peered":"2024-04-17T10:42:35.992079+0000","last_clean":"2024-04-17T10:42:35.992079+0000","last_became_active":"2024-04-17T10:42:35.023746+0000","last_became_peered":"2024-04-17T10:42:35.023746+0000","last_unstale":"2024-04-17T10:42:35.992079+0000","last_undegraded":"2024-04-17T10:42:35.992079+0000","last_fullsized":"2024-04-17T10:42:35.992079+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:53:20.596275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.339478+0000","last_change":"2024-04-17T10:42:35.027645+0000","last_active":"2024-04-17T10:42:36.339478+0000","last_peered":"2024-04-17T10:42:36.339478+0000","last_clean":"2024-04-17T10:42:36.339478+0000","last_became_active":"2024-04-17T10:42:35.023180+0000","last_became_peered":"2024-04-17T10:42:35.023180+0000","last_unstale":"2024-04-17T10:42:36.339478+0000","last_undegraded":"2024-04-17T10:42:36.339478+0000","last_fullsized":"2024-04-17T10:42:36.339478+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:45:46.184170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.991418+0000","last_change":"2024-04-17T10:42:35.018399+0000","last_active":"2024-04-17T10:42:35.991418+0000","last_peered":"2024-04-17T10:42:35.991418+0000","last_clean":"2024-04-17T10:42:35.991418+0000","last_became_active":"2024-04-17T10:42:35.018087+0000","last_became_peered":"2024-04-17T10:42:35.018087+0000","last_unstale":"2024-04-17T10:42:35.991418+0000","last_undegraded":"2024-04-17T10:42:35.991418+0000","last_fullsized":"2024-04-17T10:42:35.991418+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:40:05.316943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.007746+0000","last_change":"2024-04-17T10:42:35.016286+0000","last_active":"2024-04-17T10:42:36.007746+0000","last_peered":"2024-04-17T10:42:36.007746+0000","last_clean":"2024-04-17T10:42:36.007746+0000","last_became_active":"2024-04-17T10:42:35.015899+0000","last_became_peered":"2024-04-17T10:42:35.015899+0000","last_unstale":"2024-04-17T10:42:36.007746+0000","last_undegraded":"2024-04-17T10:42:36.007746+0000","last_fullsized":"2024-04-17T10:42:36.007746+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:29:50.762209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.412205+0000","last_change":"2024-04-17T10:42:35.000543+0000","last_active":"2024-04-17T10:42:36.412205+0000","last_peered":"2024-04-17T10:42:36.412205+0000","last_clean":"2024-04-17T10:42:36.412205+0000","last_became_active":"2024-04-17T10:42:34.999131+0000","last_became_peered":"2024-04-17T10:42:34.999131+0000","last_unstale":"2024-04-17T10:42:36.412205+0000","last_undegraded":"2024-04-17T10:42:36.412205+0000","last_fullsized":"2024-04-17T10:42:36.412205+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:42:45.021574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.412784+0000","last_change":"2024-04-17T10:42:35.028556+0000","last_active":"2024-04-17T10:42:36.412784+0000","last_peered":"2024-04-17T10:42:36.412784+0000","last_clean":"2024-04-17T10:42:36.412784+0000","last_became_active":"2024-04-17T10:42:35.027301+0000","last_became_peered":"2024-04-17T10:42:35.027301+0000","last_unstale":"2024-04-17T10:42:36.412784+0000","last_undegraded":"2024-04-17T10:42:36.412784+0000","last_fullsized":"2024-04-17T10:42:36.412784+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:11:46.657622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.987320+0000","last_change":"2024-04-17T10:42:35.008932+0000","last_active":"2024-04-17T10:42:35.987320+0000","last_peered":"2024-04-17T10:42:35.987320+0000","last_clean":"2024-04-17T10:42:35.987320+0000","last_became_active":"2024-04-17T10:42:35.008359+0000","last_became_peered":"2024-04-17T10:42:35.008359+0000","last_unstale":"2024-04-17T10:42:35.987320+0000","last_undegraded":"2024-04-17T10:42:35.987320+0000","last_fullsized":"2024-04-17T10:42:35.987320+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:32:23.762419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.411683+0000","last_change":"2024-04-17T10:42:35.026972+0000","last_active":"2024-04-17T10:42:36.411683+0000","last_peered":"2024-04-17T10:42:36.411683+0000","last_clean":"2024-04-17T10:42:36.411683+0000","last_became_active":"2024-04-17T10:42:35.026597+0000","last_became_peered":"2024-04-17T10:42:35.026597+0000","last_unstale":"2024-04-17T10:42:36.411683+0000","last_undegraded":"2024-04-17T10:42:36.411683+0000","last_fullsized":"2024-04-17T10:42:36.411683+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:32:22.701587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.340364+0000","last_change":"2024-04-17T10:42:35.024054+0000","last_active":"2024-04-17T10:42:36.340364+0000","last_peered":"2024-04-17T10:42:36.340364+0000","last_clean":"2024-04-17T10:42:36.340364+0000","last_became_active":"2024-04-17T10:42:35.023243+0000","last_became_peered":"2024-04-17T10:42:35.023243+0000","last_unstale":"2024-04-17T10:42:36.340364+0000","last_undegraded":"2024-04-17T10:42:36.340364+0000","last_fullsized":"2024-04-17T10:42:36.340364+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:47:08.648118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.991827+0000","last_change":"2024-04-17T10:42:35.010569+0000","last_active":"2024-04-17T10:42:35.991827+0000","last_peered":"2024-04-17T10:42:35.991827+0000","last_clean":"2024-04-17T10:42:35.991827+0000","last_became_active":"2024-04-17T10:42:35.009896+0000","last_became_peered":"2024-04-17T10:42:35.009896+0000","last_unstale":"2024-04-17T10:42:35.991827+0000","last_undegraded":"2024-04-17T10:42:35.991827+0000","last_fullsized":"2024-04-17T10:42:35.991827+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:29:43.713428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.410618+0000","last_change":"2024-04-17T10:42:35.004653+0000","last_active":"2024-04-17T10:42:36.410618+0000","last_peered":"2024-04-17T10:42:36.410618+0000","last_clean":"2024-04-17T10:42:36.410618+0000","last_became_active":"2024-04-17T10:42:35.003785+0000","last_became_peered":"2024-04-17T10:42:35.003785+0000","last_unstale":"2024-04-17T10:42:36.410618+0000","last_undegraded":"2024-04-17T10:42:36.410618+0000","last_fullsized":"2024-04-17T10:42:36.410618+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:52:44.215335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.986791+0000","last_change":"2024-04-17T10:42:35.030273+0000","last_active":"2024-04-17T10:42:35.986791+0000","last_peered":"2024-04-17T10:42:35.986791+0000","last_clean":"2024-04-17T10:42:35.986791+0000","last_became_active":"2024-04-17T10:42:35.029890+0000","last_became_peered":"2024-04-17T10:42:35.029890+0000","last_unstale":"2024-04-17T10:42:35.986791+0000","last_undegraded":"2024-04-17T10:42:35.986791+0000","last_fullsized":"2024-04-17T10:42:35.986791+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:40:28.617010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.987620+0000","last_change":"2024-04-17T10:42:35.009772+0000","last_active":"2024-04-17T10:42:35.987620+0000","last_peered":"2024-04-17T10:42:35.987620+0000","last_clean":"2024-04-17T10:42:35.987620+0000","last_became_active":"2024-04-17T10:42:35.009014+0000","last_became_peered":"2024-04-17T10:42:35.009014+0000","last_unstale":"2024-04-17T10:42:35.987620+0000","last_undegraded":"2024-04-17T10:42:35.987620+0000","last_fullsized":"2024-04-17T10:42:35.987620+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:40:05.316943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"58'4","reported_seq":18,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.987646+0000","last_change":"2024-04-17T10:42:34.015019+0000","last_active":"2024-04-17T10:42:35.987646+0000","last_peered":"2024-04-17T10:42:35.987646+0000","last_clean":"2024-04-17T10:42:35.987646+0000","last_became_active":"2024-04-17T10:42:34.014716+0000","last_became_peered":"2024-04-17T10:42:34.014716+0000","last_unstale":"2024-04-17T10:42:35.987646+0000","last_undegraded":"2024-04-17T10:42:35.987646+0000","last_fullsized":"2024-04-17T10:42:35.987646+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:43:53.941771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.412270+0000","last_change":"2024-04-17T10:42:35.002129+0000","last_active":"2024-04-17T10:42:36.412270+0000","last_peered":"2024-04-17T10:42:36.412270+0000","last_clean":"2024-04-17T10:42:36.412270+0000","last_became_active":"2024-04-17T10:42:35.001174+0000","last_became_peered":"2024-04-17T10:42:35.001174+0000","last_unstale":"2024-04-17T10:42:36.412270+0000","last_undegraded":"2024-04-17T10:42:36.412270+0000","last_fullsized":"2024-04-17T10:42:36.412270+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:34:52.474792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.339797+0000","last_change":"2024-04-17T10:42:34.006170+0000","last_active":"2024-04-17T10:42:36.339797+0000","last_peered":"2024-04-17T10:42:36.339797+0000","last_clean":"2024-04-17T10:42:36.339797+0000","last_became_active":"2024-04-17T10:42:34.005757+0000","last_became_peered":"2024-04-17T10:42:34.005757+0000","last_unstale":"2024-04-17T10:42:36.339797+0000","last_undegraded":"2024-04-17T10:42:36.339797+0000","last_fullsized":"2024-04-17T10:42:36.339797+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:35:45.666827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.987870+0000","last_change":"2024-04-17T10:42:35.013753+0000","last_active":"2024-04-17T10:42:35.987870+0000","last_peered":"2024-04-17T10:42:35.987870+0000","last_clean":"2024-04-17T10:42:35.987870+0000","last_became_active":"2024-04-17T10:42:35.013440+0000","last_became_peered":"2024-04-17T10:42:35.013440+0000","last_unstale":"2024-04-17T10:42:35.987870+0000","last_undegraded":"2024-04-17T10:42:35.987870+0000","last_fullsized":"2024-04-17T10:42:35.987870+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:37:35.341273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.987433+0000","last_change":"2024-04-17T10:42:33.993426+0000","last_active":"2024-04-17T10:42:35.987433+0000","last_peered":"2024-04-17T10:42:35.987433+0000","last_clean":"2024-04-17T10:42:35.987433+0000","last_became_active":"2024-04-17T10:42:33.992971+0000","last_became_peered":"2024-04-17T10:42:33.992971+0000","last_unstale":"2024-04-17T10:42:35.987433+0000","last_undegraded":"2024-04-17T10:42:35.987433+0000","last_fullsized":"2024-04-17T10:42:35.987433+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:32:27.523313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.987034+0000","last_change":"2024-04-17T10:42:35.027423+0000","last_active":"2024-04-17T10:42:35.987034+0000","last_peered":"2024-04-17T10:42:35.987034+0000","last_clean":"2024-04-17T10:42:35.987034+0000","last_became_active":"2024-04-17T10:42:35.027017+0000","last_became_peered":"2024-04-17T10:42:35.027017+0000","last_unstale":"2024-04-17T10:42:35.987034+0000","last_undegraded":"2024-04-17T10:42:35.987034+0000","last_fullsized":"2024-04-17T10:42:35.987034+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:30:45.283107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.412995+0000","last_change":"2024-04-17T10:42:34.018234+0000","last_active":"2024-04-17T10:42:36.412995+0000","last_peered":"2024-04-17T10:42:36.412995+0000","last_clean":"2024-04-17T10:42:36.412995+0000","last_became_active":"2024-04-17T10:42:34.017497+0000","last_became_peered":"2024-04-17T10:42:34.017497+0000","last_unstale":"2024-04-17T10:42:36.412995+0000","last_undegraded":"2024-04-17T10:42:36.412995+0000","last_fullsized":"2024-04-17T10:42:36.412995+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:25:59.364150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.339385+0000","last_change":"2024-04-17T10:42:35.023936+0000","last_active":"2024-04-17T10:42:36.339385+0000","last_peered":"2024-04-17T10:42:36.339385+0000","last_clean":"2024-04-17T10:42:36.339385+0000","last_became_active":"2024-04-17T10:42:35.022583+0000","last_became_peered":"2024-04-17T10:42:35.022583+0000","last_unstale":"2024-04-17T10:42:36.339385+0000","last_undegraded":"2024-04-17T10:42:36.339385+0000","last_fullsized":"2024-04-17T10:42:36.339385+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:54:37.642863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.990342+0000","last_change":"2024-04-17T10:42:34.004483+0000","last_active":"2024-04-17T10:42:35.990342+0000","last_peered":"2024-04-17T10:42:35.990342+0000","last_clean":"2024-04-17T10:42:35.990342+0000","last_became_active":"2024-04-17T10:42:33.998883+0000","last_became_peered":"2024-04-17T10:42:33.998883+0000","last_unstale":"2024-04-17T10:42:35.990342+0000","last_undegraded":"2024-04-17T10:42:35.990342+0000","last_fullsized":"2024-04-17T10:42:35.990342+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:34:51.462597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.339435+0000","last_change":"2024-04-17T10:42:35.028013+0000","last_active":"2024-04-17T10:42:36.339435+0000","last_peered":"2024-04-17T10:42:36.339435+0000","last_clean":"2024-04-17T10:42:36.339435+0000","last_became_active":"2024-04-17T10:42:35.025898+0000","last_became_peered":"2024-04-17T10:42:35.025898+0000","last_unstale":"2024-04-17T10:42:36.339435+0000","last_undegraded":"2024-04-17T10:42:36.339435+0000","last_fullsized":"2024-04-17T10:42:36.339435+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:11:09.719074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.987023+0000","last_change":"2024-04-17T10:42:34.000321+0000","last_active":"2024-04-17T10:42:35.987023+0000","last_peered":"2024-04-17T10:42:35.987023+0000","last_clean":"2024-04-17T10:42:35.987023+0000","last_became_active":"2024-04-17T10:42:34.000062+0000","last_became_peered":"2024-04-17T10:42:34.000062+0000","last_unstale":"2024-04-17T10:42:35.987023+0000","last_undegraded":"2024-04-17T10:42:35.987023+0000","last_fullsized":"2024-04-17T10:42:35.987023+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:42:38.783160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.007759+0000","last_change":"2024-04-17T10:42:34.989852+0000","last_active":"2024-04-17T10:42:36.007759+0000","last_peered":"2024-04-17T10:42:36.007759+0000","last_clean":"2024-04-17T10:42:36.007759+0000","last_became_active":"2024-04-17T10:42:34.989121+0000","last_became_peered":"2024-04-17T10:42:34.989121+0000","last_unstale":"2024-04-17T10:42:36.007759+0000","last_undegraded":"2024-04-17T10:42:36.007759+0000","last_fullsized":"2024-04-17T10:42:36.007759+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:17:25.136437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.412216+0000","last_change":"2024-04-17T10:42:34.018055+0000","last_active":"2024-04-17T10:42:36.412216+0000","last_peered":"2024-04-17T10:42:36.412216+0000","last_clean":"2024-04-17T10:42:36.412216+0000","last_became_active":"2024-04-17T10:42:34.016719+0000","last_became_peered":"2024-04-17T10:42:34.016719+0000","last_unstale":"2024-04-17T10:42:36.412216+0000","last_undegraded":"2024-04-17T10:42:36.412216+0000","last_fullsized":"2024-04-17T10:42:36.412216+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:41:52.745779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.339561+0000","last_change":"2024-04-17T10:42:35.026185+0000","last_active":"2024-04-17T10:42:36.339561+0000","last_peered":"2024-04-17T10:42:36.339561+0000","last_clean":"2024-04-17T10:42:36.339561+0000","last_became_active":"2024-04-17T10:42:35.025681+0000","last_became_peered":"2024-04-17T10:42:35.025681+0000","last_unstale":"2024-04-17T10:42:36.339561+0000","last_undegraded":"2024-04-17T10:42:36.339561+0000","last_fullsized":"2024-04-17T10:42:36.339561+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:10:29.515616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.989337+0000","last_change":"2024-04-17T10:42:34.008774+0000","last_active":"2024-04-17T10:42:35.989337+0000","last_peered":"2024-04-17T10:42:35.989337+0000","last_clean":"2024-04-17T10:42:35.989337+0000","last_became_active":"2024-04-17T10:42:34.005106+0000","last_became_peered":"2024-04-17T10:42:34.005106+0000","last_unstale":"2024-04-17T10:42:35.989337+0000","last_undegraded":"2024-04-17T10:42:35.989337+0000","last_fullsized":"2024-04-17T10:42:35.989337+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:26:57.986949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.991108+0000","last_change":"2024-04-17T10:42:35.019578+0000","last_active":"2024-04-17T10:42:35.991108+0000","last_peered":"2024-04-17T10:42:35.991108+0000","last_clean":"2024-04-17T10:42:35.991108+0000","last_became_active":"2024-04-17T10:42:35.013952+0000","last_became_peered":"2024-04-17T10:42:35.013952+0000","last_unstale":"2024-04-17T10:42:35.991108+0000","last_undegraded":"2024-04-17T10:42:35.991108+0000","last_fullsized":"2024-04-17T10:42:35.991108+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:26:00.376345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.992745+0000","last_change":"2024-04-17T10:42:34.006222+0000","last_active":"2024-04-17T10:42:35.992745+0000","last_peered":"2024-04-17T10:42:35.992745+0000","last_clean":"2024-04-17T10:42:35.992745+0000","last_became_active":"2024-04-17T10:42:34.005353+0000","last_became_peered":"2024-04-17T10:42:34.005353+0000","last_unstale":"2024-04-17T10:42:35.992745+0000","last_undegraded":"2024-04-17T10:42:35.992745+0000","last_fullsized":"2024-04-17T10:42:35.992745+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:41:52.745779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.992681+0000","last_change":"2024-04-17T10:42:35.010552+0000","last_active":"2024-04-17T10:42:35.992681+0000","last_peered":"2024-04-17T10:42:35.992681+0000","last_clean":"2024-04-17T10:42:35.992681+0000","last_became_active":"2024-04-17T10:42:35.010242+0000","last_became_peered":"2024-04-17T10:42:35.010242+0000","last_unstale":"2024-04-17T10:42:35.992681+0000","last_undegraded":"2024-04-17T10:42:35.992681+0000","last_fullsized":"2024-04-17T10:42:35.992681+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:37:45.633967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.340089+0000","last_change":"2024-04-17T10:42:33.995027+0000","last_active":"2024-04-17T10:42:36.340089+0000","last_peered":"2024-04-17T10:42:36.340089+0000","last_clean":"2024-04-17T10:42:36.340089+0000","last_became_active":"2024-04-17T10:42:33.994748+0000","last_became_peered":"2024-04-17T10:42:33.994748+0000","last_unstale":"2024-04-17T10:42:36.340089+0000","last_undegraded":"2024-04-17T10:42:36.340089+0000","last_fullsized":"2024-04-17T10:42:36.340089+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:57:31.575219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.411404+0000","last_change":"2024-04-17T10:42:35.014305+0000","last_active":"2024-04-17T10:42:36.411404+0000","last_peered":"2024-04-17T10:42:36.411404+0000","last_clean":"2024-04-17T10:42:36.411404+0000","last_became_active":"2024-04-17T10:42:35.011452+0000","last_became_peered":"2024-04-17T10:42:35.011452+0000","last_unstale":"2024-04-17T10:42:36.411404+0000","last_undegraded":"2024-04-17T10:42:36.411404+0000","last_fullsized":"2024-04-17T10:42:36.411404+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:14:15.302132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"58'3","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.993111+0000","last_change":"2024-04-17T10:42:34.010793+0000","last_active":"2024-04-17T10:42:35.993111+0000","last_peered":"2024-04-17T10:42:35.993111+0000","last_clean":"2024-04-17T10:42:35.993111+0000","last_became_active":"2024-04-17T10:42:34.009786+0000","last_became_peered":"2024-04-17T10:42:34.009786+0000","last_unstale":"2024-04-17T10:42:35.993111+0000","last_undegraded":"2024-04-17T10:42:35.993111+0000","last_fullsized":"2024-04-17T10:42:35.993111+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:33:41.490545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.992902+0000","last_change":"2024-04-17T10:42:35.010650+0000","last_active":"2024-04-17T10:42:35.992902+0000","last_peered":"2024-04-17T10:42:35.992902+0000","last_clean":"2024-04-17T10:42:35.992902+0000","last_became_active":"2024-04-17T10:42:35.009392+0000","last_became_peered":"2024-04-17T10:42:35.009392+0000","last_unstale":"2024-04-17T10:42:35.992902+0000","last_undegraded":"2024-04-17T10:42:35.992902+0000","last_fullsized":"2024-04-17T10:42:35.992902+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:37:51.467889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"58'6","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.991000+0000","last_change":"2024-04-17T10:42:34.008738+0000","last_active":"2024-04-17T10:42:35.991000+0000","last_peered":"2024-04-17T10:42:35.991000+0000","last_clean":"2024-04-17T10:42:35.991000+0000","last_became_active":"2024-04-17T10:42:34.004730+0000","last_became_peered":"2024-04-17T10:42:34.004730+0000","last_unstale":"2024-04-17T10:42:35.991000+0000","last_undegraded":"2024-04-17T10:42:35.991000+0000","last_fullsized":"2024-04-17T10:42:35.991000+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:10:36.224186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.412233+0000","last_change":"2024-04-17T10:42:35.023535+0000","last_active":"2024-04-17T10:42:36.412233+0000","last_peered":"2024-04-17T10:42:36.412233+0000","last_clean":"2024-04-17T10:42:36.412233+0000","last_became_active":"2024-04-17T10:42:35.023146+0000","last_became_peered":"2024-04-17T10:42:35.023146+0000","last_unstale":"2024-04-17T10:42:36.412233+0000","last_undegraded":"2024-04-17T10:42:36.412233+0000","last_fullsized":"2024-04-17T10:42:36.412233+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:30:04.131306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.339827+0000","last_change":"2024-04-17T10:42:34.000022+0000","last_active":"2024-04-17T10:42:36.339827+0000","last_peered":"2024-04-17T10:42:36.339827+0000","last_clean":"2024-04-17T10:42:36.339827+0000","last_became_active":"2024-04-17T10:42:33.999505+0000","last_became_peered":"2024-04-17T10:42:33.999505+0000","last_unstale":"2024-04-17T10:42:36.339827+0000","last_undegraded":"2024-04-17T10:42:36.339827+0000","last_fullsized":"2024-04-17T10:42:36.339827+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:43:27.470115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.991994+0000","last_change":"2024-04-17T10:42:34.993799+0000","last_active":"2024-04-17T10:42:35.991994+0000","last_peered":"2024-04-17T10:42:35.991994+0000","last_clean":"2024-04-17T10:42:35.991994+0000","last_became_active":"2024-04-17T10:42:34.992986+0000","last_became_peered":"2024-04-17T10:42:34.992986+0000","last_unstale":"2024-04-17T10:42:35.991994+0000","last_undegraded":"2024-04-17T10:42:35.991994+0000","last_fullsized":"2024-04-17T10:42:35.991994+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:11:46.657622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.007266+0000","last_change":"2024-04-17T10:42:34.010051+0000","last_active":"2024-04-17T10:42:36.007266+0000","last_peered":"2024-04-17T10:42:36.007266+0000","last_clean":"2024-04-17T10:42:36.007266+0000","last_became_active":"2024-04-17T10:42:34.009227+0000","last_became_peered":"2024-04-17T10:42:34.009227+0000","last_unstale":"2024-04-17T10:42:36.007266+0000","last_undegraded":"2024-04-17T10:42:36.007266+0000","last_fullsized":"2024-04-17T10:42:36.007266+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:31.397161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.990373+0000","last_change":"2024-04-17T10:42:35.008635+0000","last_active":"2024-04-17T10:42:35.990373+0000","last_peered":"2024-04-17T10:42:35.990373+0000","last_clean":"2024-04-17T10:42:35.990373+0000","last_became_active":"2024-04-17T10:42:35.008288+0000","last_became_peered":"2024-04-17T10:42:35.008288+0000","last_unstale":"2024-04-17T10:42:35.990373+0000","last_undegraded":"2024-04-17T10:42:35.990373+0000","last_fullsized":"2024-04-17T10:42:35.990373+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:03:52.712078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.412237+0000","last_change":"2024-04-17T10:42:34.019631+0000","last_active":"2024-04-17T10:42:36.412237+0000","last_peered":"2024-04-17T10:42:36.412237+0000","last_clean":"2024-04-17T10:42:36.412237+0000","last_became_active":"2024-04-17T10:42:34.018768+0000","last_became_peered":"2024-04-17T10:42:34.018768+0000","last_unstale":"2024-04-17T10:42:36.412237+0000","last_undegraded":"2024-04-17T10:42:36.412237+0000","last_fullsized":"2024-04-17T10:42:36.412237+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:03:51.699883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.991712+0000","last_change":"2024-04-17T10:42:35.030930+0000","last_active":"2024-04-17T10:42:35.991712+0000","last_peered":"2024-04-17T10:42:35.991712+0000","last_clean":"2024-04-17T10:42:35.991712+0000","last_became_active":"2024-04-17T10:42:35.029914+0000","last_became_peered":"2024-04-17T10:42:35.029914+0000","last_unstale":"2024-04-17T10:42:35.991712+0000","last_undegraded":"2024-04-17T10:42:35.991712+0000","last_fullsized":"2024-04-17T10:42:35.991712+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:37:35.341273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"58'8","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.990785+0000","last_change":"2024-04-17T10:42:34.015072+0000","last_active":"2024-04-17T10:42:35.990785+0000","last_peered":"2024-04-17T10:42:35.990785+0000","last_clean":"2024-04-17T10:42:35.990785+0000","last_became_active":"2024-04-17T10:42:34.014406+0000","last_became_peered":"2024-04-17T10:42:34.014406+0000","last_unstale":"2024-04-17T10:42:35.990785+0000","last_undegraded":"2024-04-17T10:42:35.990785+0000","last_fullsized":"2024-04-17T10:42:35.990785+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:32:38.583434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.992839+0000","last_change":"2024-04-17T10:42:35.012444+0000","last_active":"2024-04-17T10:42:35.992839+0000","last_peered":"2024-04-17T10:42:35.992839+0000","last_clean":"2024-04-17T10:42:35.992839+0000","last_became_active":"2024-04-17T10:42:35.011922+0000","last_became_peered":"2024-04-17T10:42:35.011922+0000","last_unstale":"2024-04-17T10:42:35.992839+0000","last_undegraded":"2024-04-17T10:42:35.992839+0000","last_fullsized":"2024-04-17T10:42:35.992839+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:32:08.445909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.340498+0000","last_change":"2024-04-17T10:42:33.998582+0000","last_active":"2024-04-17T10:42:36.340498+0000","last_peered":"2024-04-17T10:42:36.340498+0000","last_clean":"2024-04-17T10:42:36.340498+0000","last_became_active":"2024-04-17T10:42:33.997341+0000","last_became_peered":"2024-04-17T10:42:33.997341+0000","last_unstale":"2024-04-17T10:42:36.340498+0000","last_undegraded":"2024-04-17T10:42:36.340498+0000","last_fullsized":"2024-04-17T10:42:36.340498+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:32:22.750224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.412893+0000","last_change":"2024-04-17T10:42:35.029792+0000","last_active":"2024-04-17T10:42:36.412893+0000","last_peered":"2024-04-17T10:42:36.412893+0000","last_clean":"2024-04-17T10:42:36.412893+0000","last_became_active":"2024-04-17T10:42:35.029384+0000","last_became_peered":"2024-04-17T10:42:35.029384+0000","last_unstale":"2024-04-17T10:42:36.412893+0000","last_undegraded":"2024-04-17T10:42:36.412893+0000","last_fullsized":"2024-04-17T10:42:36.412893+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:53:20.596275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.987878+0000","last_change":"2024-04-17T10:42:33.995762+0000","last_active":"2024-04-17T10:42:35.987878+0000","last_peered":"2024-04-17T10:42:35.987878+0000","last_clean":"2024-04-17T10:42:35.987878+0000","last_became_active":"2024-04-17T10:42:33.995562+0000","last_became_peered":"2024-04-17T10:42:33.995562+0000","last_unstale":"2024-04-17T10:42:35.987878+0000","last_undegraded":"2024-04-17T10:42:35.987878+0000","last_fullsized":"2024-04-17T10:42:35.987878+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:20:46.478088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.340201+0000","last_change":"2024-04-17T10:42:35.022270+0000","last_active":"2024-04-17T10:42:36.340201+0000","last_peered":"2024-04-17T10:42:36.340201+0000","last_clean":"2024-04-17T10:42:36.340201+0000","last_became_active":"2024-04-17T10:42:35.021592+0000","last_became_peered":"2024-04-17T10:42:35.021592+0000","last_unstale":"2024-04-17T10:42:36.340201+0000","last_undegraded":"2024-04-17T10:42:36.340201+0000","last_fullsized":"2024-04-17T10:42:36.340201+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:06:20.461158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.412678+0000","last_change":"2024-04-17T10:42:34.017202+0000","last_active":"2024-04-17T10:42:36.412678+0000","last_peered":"2024-04-17T10:42:36.412678+0000","last_clean":"2024-04-17T10:42:36.412678+0000","last_became_active":"2024-04-17T10:42:34.016613+0000","last_became_peered":"2024-04-17T10:42:34.016613+0000","last_unstale":"2024-04-17T10:42:36.412678+0000","last_undegraded":"2024-04-17T10:42:36.412678+0000","last_fullsized":"2024-04-17T10:42:36.412678+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:32:38.583434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.992946+0000","last_change":"2024-04-17T10:42:35.010024+0000","last_active":"2024-04-17T10:42:35.992946+0000","last_peered":"2024-04-17T10:42:35.992946+0000","last_clean":"2024-04-17T10:42:35.992946+0000","last_became_active":"2024-04-17T10:42:35.008998+0000","last_became_peered":"2024-04-17T10:42:35.008998+0000","last_unstale":"2024-04-17T10:42:35.992946+0000","last_undegraded":"2024-04-17T10:42:35.992946+0000","last_fullsized":"2024-04-17T10:42:35.992946+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:00:02.307316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.988087+0000","last_change":"2024-04-17T10:42:34.002003+0000","last_active":"2024-04-17T10:42:35.988087+0000","last_peered":"2024-04-17T10:42:35.988087+0000","last_clean":"2024-04-17T10:42:35.988087+0000","last_became_active":"2024-04-17T10:42:34.001413+0000","last_became_peered":"2024-04-17T10:42:34.001413+0000","last_unstale":"2024-04-17T10:42:35.988087+0000","last_undegraded":"2024-04-17T10:42:35.988087+0000","last_fullsized":"2024-04-17T10:42:35.988087+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:10:36.224186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.412285+0000","last_change":"2024-04-17T10:42:35.026741+0000","last_active":"2024-04-17T10:42:36.412285+0000","last_peered":"2024-04-17T10:42:36.412285+0000","last_clean":"2024-04-17T10:42:36.412285+0000","last_became_active":"2024-04-17T10:42:35.023488+0000","last_became_peered":"2024-04-17T10:42:35.023488+0000","last_unstale":"2024-04-17T10:42:36.412285+0000","last_undegraded":"2024-04-17T10:42:36.412285+0000","last_fullsized":"2024-04-17T10:42:36.412285+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:40:05.316943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.340176+0000","last_change":"2024-04-17T10:42:34.010407+0000","last_active":"2024-04-17T10:42:36.340176+0000","last_peered":"2024-04-17T10:42:36.340176+0000","last_clean":"2024-04-17T10:42:36.340176+0000","last_became_active":"2024-04-17T10:42:34.009954+0000","last_became_peered":"2024-04-17T10:42:34.009954+0000","last_unstale":"2024-04-17T10:42:36.340176+0000","last_undegraded":"2024-04-17T10:42:36.340176+0000","last_fullsized":"2024-04-17T10:42:36.340176+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:40:04.304748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.988476+0000","last_change":"2024-04-17T10:42:35.014822+0000","last_active":"2024-04-17T10:42:35.988476+0000","last_peered":"2024-04-17T10:42:35.988476+0000","last_clean":"2024-04-17T10:42:35.988476+0000","last_became_active":"2024-04-17T10:42:35.014526+0000","last_became_peered":"2024-04-17T10:42:35.014526+0000","last_unstale":"2024-04-17T10:42:35.988476+0000","last_undegraded":"2024-04-17T10:42:35.988476+0000","last_fullsized":"2024-04-17T10:42:35.988476+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:29:43.713428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.411183+0000","last_change":"2024-04-17T10:42:33.991455+0000","last_active":"2024-04-17T10:42:36.411183+0000","last_peered":"2024-04-17T10:42:36.411183+0000","last_clean":"2024-04-17T10:42:36.411183+0000","last_became_active":"2024-04-17T10:42:33.990906+0000","last_became_peered":"2024-04-17T10:42:33.990906+0000","last_unstale":"2024-04-17T10:42:36.411183+0000","last_undegraded":"2024-04-17T10:42:36.411183+0000","last_fullsized":"2024-04-17T10:42:36.411183+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:25:05.304517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.986773+0000","last_change":"2024-04-17T10:42:35.008786+0000","last_active":"2024-04-17T10:42:35.986773+0000","last_peered":"2024-04-17T10:42:35.986773+0000","last_clean":"2024-04-17T10:42:35.986773+0000","last_became_active":"2024-04-17T10:42:35.008219+0000","last_became_peered":"2024-04-17T10:42:35.008219+0000","last_unstale":"2024-04-17T10:42:35.986773+0000","last_undegraded":"2024-04-17T10:42:35.986773+0000","last_fullsized":"2024-04-17T10:42:35.986773+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:57:32.587415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.412013+0000","last_change":"2024-04-17T10:42:34.007999+0000","last_active":"2024-04-17T10:42:36.412013+0000","last_peered":"2024-04-17T10:42:36.412013+0000","last_clean":"2024-04-17T10:42:36.412013+0000","last_became_active":"2024-04-17T10:42:34.007259+0000","last_became_peered":"2024-04-17T10:42:34.007259+0000","last_unstale":"2024-04-17T10:42:36.412013+0000","last_undegraded":"2024-04-17T10:42:36.412013+0000","last_fullsized":"2024-04-17T10:42:36.412013+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:33:41.490545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.412992+0000","last_change":"2024-04-17T10:42:35.033669+0000","last_active":"2024-04-17T10:42:36.412992+0000","last_peered":"2024-04-17T10:42:36.412992+0000","last_clean":"2024-04-17T10:42:36.412992+0000","last_became_active":"2024-04-17T10:42:35.033109+0000","last_became_peered":"2024-04-17T10:42:35.033109+0000","last_unstale":"2024-04-17T10:42:36.412992+0000","last_undegraded":"2024-04-17T10:42:36.412992+0000","last_fullsized":"2024-04-17T10:42:36.412992+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:31:41.284193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.412529+0000","last_change":"2024-04-17T10:42:34.002043+0000","last_active":"2024-04-17T10:42:36.412529+0000","last_peered":"2024-04-17T10:42:36.412529+0000","last_clean":"2024-04-17T10:42:36.412529+0000","last_became_active":"2024-04-17T10:42:34.001536+0000","last_became_peered":"2024-04-17T10:42:34.001536+0000","last_unstale":"2024-04-17T10:42:36.412529+0000","last_undegraded":"2024-04-17T10:42:36.412529+0000","last_fullsized":"2024-04-17T10:42:36.412529+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:29:49.750014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.413193+0000","last_change":"2024-04-17T10:42:35.028984+0000","last_active":"2024-04-17T10:42:36.413193+0000","last_peered":"2024-04-17T10:42:36.413193+0000","last_clean":"2024-04-17T10:42:36.413193+0000","last_became_active":"2024-04-17T10:42:35.026253+0000","last_became_peered":"2024-04-17T10:42:35.026253+0000","last_unstale":"2024-04-17T10:42:36.413193+0000","last_undegraded":"2024-04-17T10:42:36.413193+0000","last_fullsized":"2024-04-17T10:42:36.413193+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:04:05.170981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"59'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.988273+0000","last_change":"2024-04-17T10:42:34.003042+0000","last_active":"2024-04-17T10:42:35.988273+0000","last_peered":"2024-04-17T10:42:35.988273+0000","last_clean":"2024-04-17T10:42:35.988273+0000","last_became_active":"2024-04-17T10:42:34.002726+0000","last_became_peered":"2024-04-17T10:42:34.002726+0000","last_unstale":"2024-04-17T10:42:35.988273+0000","last_undegraded":"2024-04-17T10:42:35.988273+0000","last_fullsized":"2024-04-17T10:42:35.988273+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:09:32.979808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.411822+0000","last_change":"2024-04-17T10:42:35.011542+0000","last_active":"2024-04-17T10:42:36.411822+0000","last_peered":"2024-04-17T10:42:36.411822+0000","last_clean":"2024-04-17T10:42:36.411822+0000","last_became_active":"2024-04-17T10:42:35.011176+0000","last_became_peered":"2024-04-17T10:42:35.011176+0000","last_unstale":"2024-04-17T10:42:36.411822+0000","last_undegraded":"2024-04-17T10:42:36.411822+0000","last_fullsized":"2024-04-17T10:42:36.411822+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:44:31.532198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.007072+0000","last_change":"2024-04-17T10:42:34.012933+0000","last_active":"2024-04-17T10:42:36.007072+0000","last_peered":"2024-04-17T10:42:36.007072+0000","last_clean":"2024-04-17T10:42:36.007072+0000","last_became_active":"2024-04-17T10:42:34.012150+0000","last_became_peered":"2024-04-17T10:42:34.012150+0000","last_unstale":"2024-04-17T10:42:36.007072+0000","last_undegraded":"2024-04-17T10:42:36.007072+0000","last_fullsized":"2024-04-17T10:42:36.007072+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:25:05.304517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.987280+0000","last_change":"2024-04-17T10:42:35.027139+0000","last_active":"2024-04-17T10:42:35.987280+0000","last_peered":"2024-04-17T10:42:35.987280+0000","last_clean":"2024-04-17T10:42:35.987280+0000","last_became_active":"2024-04-17T10:42:35.026861+0000","last_became_peered":"2024-04-17T10:42:35.026861+0000","last_unstale":"2024-04-17T10:42:35.987280+0000","last_undegraded":"2024-04-17T10:42:35.987280+0000","last_fullsized":"2024-04-17T10:42:35.987280+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:30:04.131306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.413175+0000","last_change":"2024-04-17T10:42:34.012790+0000","last_active":"2024-04-17T10:42:36.413175+0000","last_peered":"2024-04-17T10:42:36.413175+0000","last_clean":"2024-04-17T10:42:36.413175+0000","last_became_active":"2024-04-17T10:42:34.012177+0000","last_became_peered":"2024-04-17T10:42:34.012177+0000","last_unstale":"2024-04-17T10:42:36.413175+0000","last_undegraded":"2024-04-17T10:42:36.413175+0000","last_fullsized":"2024-04-17T10:42:36.413175+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:01:33.910102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.339788+0000","last_change":"2024-04-17T10:42:35.026981+0000","last_active":"2024-04-17T10:42:36.339788+0000","last_peered":"2024-04-17T10:42:36.339788+0000","last_clean":"2024-04-17T10:42:36.339788+0000","last_became_active":"2024-04-17T10:42:35.026700+0000","last_became_peered":"2024-04-17T10:42:35.026700+0000","last_unstale":"2024-04-17T10:42:36.339788+0000","last_undegraded":"2024-04-17T10:42:36.339788+0000","last_fullsized":"2024-04-17T10:42:36.339788+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:39:10.426786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.410836+0000","last_change":"2024-04-17T10:42:34.015303+0000","last_active":"2024-04-17T10:42:36.410836+0000","last_peered":"2024-04-17T10:42:36.410836+0000","last_clean":"2024-04-17T10:42:36.410836+0000","last_became_active":"2024-04-17T10:42:34.014919+0000","last_became_peered":"2024-04-17T10:42:34.014919+0000","last_unstale":"2024-04-17T10:42:36.410836+0000","last_undegraded":"2024-04-17T10:42:36.410836+0000","last_fullsized":"2024-04-17T10:42:36.410836+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:41:10.271546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.992332+0000","last_change":"2024-04-17T10:42:35.025934+0000","last_active":"2024-04-17T10:42:35.992332+0000","last_peered":"2024-04-17T10:42:35.992332+0000","last_clean":"2024-04-17T10:42:35.992332+0000","last_became_active":"2024-04-17T10:42:35.025571+0000","last_became_peered":"2024-04-17T10:42:35.025571+0000","last_unstale":"2024-04-17T10:42:35.992332+0000","last_undegraded":"2024-04-17T10:42:35.992332+0000","last_fullsized":"2024-04-17T10:42:35.992332+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:31:41.284193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.989801+0000","last_change":"2024-04-17T10:42:34.009833+0000","last_active":"2024-04-17T10:42:35.989801+0000","last_peered":"2024-04-17T10:42:35.989801+0000","last_clean":"2024-04-17T10:42:35.989801+0000","last_became_active":"2024-04-17T10:42:34.005236+0000","last_became_peered":"2024-04-17T10:42:34.005236+0000","last_unstale":"2024-04-17T10:42:35.989801+0000","last_undegraded":"2024-04-17T10:42:35.989801+0000","last_fullsized":"2024-04-17T10:42:35.989801+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:43:53.941771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.410654+0000","last_change":"2024-04-17T10:42:35.005484+0000","last_active":"2024-04-17T10:42:36.410654+0000","last_peered":"2024-04-17T10:42:36.410654+0000","last_clean":"2024-04-17T10:42:36.410654+0000","last_became_active":"2024-04-17T10:42:35.004816+0000","last_became_peered":"2024-04-17T10:42:35.004816+0000","last_unstale":"2024-04-17T10:42:36.410654+0000","last_undegraded":"2024-04-17T10:42:36.410654+0000","last_fullsized":"2024-04-17T10:42:36.410654+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:42:13.543554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.410792+0000","last_change":"2024-04-17T10:42:33.985466+0000","last_active":"2024-04-17T10:42:36.410792+0000","last_peered":"2024-04-17T10:42:36.410792+0000","last_clean":"2024-04-17T10:42:36.410792+0000","last_became_active":"2024-04-17T10:42:33.984982+0000","last_became_peered":"2024-04-17T10:42:33.984982+0000","last_unstale":"2024-04-17T10:42:36.410792+0000","last_undegraded":"2024-04-17T10:42:36.410792+0000","last_fullsized":"2024-04-17T10:42:36.410792+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:52:19.251915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.339469+0000","last_change":"2024-04-17T10:42:35.017346+0000","last_active":"2024-04-17T10:42:36.339469+0000","last_peered":"2024-04-17T10:42:36.339469+0000","last_clean":"2024-04-17T10:42:36.339469+0000","last_became_active":"2024-04-17T10:42:35.013548+0000","last_became_peered":"2024-04-17T10:42:35.013548+0000","last_unstale":"2024-04-17T10:42:36.339469+0000","last_undegraded":"2024-04-17T10:42:36.339469+0000","last_fullsized":"2024-04-17T10:42:36.339469+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:17:51.676141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.340536+0000","last_change":"2024-04-17T10:42:34.015593+0000","last_active":"2024-04-17T10:42:36.340536+0000","last_peered":"2024-04-17T10:42:36.340536+0000","last_clean":"2024-04-17T10:42:36.340536+0000","last_became_active":"2024-04-17T10:42:34.015222+0000","last_became_peered":"2024-04-17T10:42:34.015222+0000","last_unstale":"2024-04-17T10:42:36.340536+0000","last_undegraded":"2024-04-17T10:42:36.340536+0000","last_fullsized":"2024-04-17T10:42:36.340536+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:30:44.270911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.340410+0000","last_change":"2024-04-17T10:42:35.026305+0000","last_active":"2024-04-17T10:42:36.340410+0000","last_peered":"2024-04-17T10:42:36.340410+0000","last_clean":"2024-04-17T10:42:36.340410+0000","last_became_active":"2024-04-17T10:42:35.025776+0000","last_became_peered":"2024-04-17T10:42:35.025776+0000","last_unstale":"2024-04-17T10:42:36.340410+0000","last_undegraded":"2024-04-17T10:42:36.340410+0000","last_fullsized":"2024-04-17T10:42:36.340410+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:42:22.146232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.007036+0000","last_change":"2024-04-17T10:42:34.021417+0000","last_active":"2024-04-17T10:42:36.007036+0000","last_peered":"2024-04-17T10:42:36.007036+0000","last_clean":"2024-04-17T10:42:36.007036+0000","last_became_active":"2024-04-17T10:42:34.020795+0000","last_became_peered":"2024-04-17T10:42:34.020795+0000","last_unstale":"2024-04-17T10:42:36.007036+0000","last_undegraded":"2024-04-17T10:42:36.007036+0000","last_fullsized":"2024-04-17T10:42:36.007036+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:26:39.567608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":250,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.991083+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:42:35.991083+0000","last_peered":"2024-04-17T10:42:35.991083+0000","last_clean":"2024-04-17T10:42:35.991083+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:42:35.991083+0000","last_undegraded":"2024-04-17T10:42:35.991083+0000","last_fullsized":"2024-04-17T10:42:35.991083+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574908,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44948,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326892,"statfs":{"total":96636764160,"available":96590737408,"internally_reserved":0,"allocated":1310720,"data_stored":888019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574907,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40200,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331640,"statfs":{"total":96636764160,"available":96595599360,"internally_reserved":0,"allocated":708608,"data_stored":297491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574908,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40208,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331632,"statfs":{"total":96636764160,"available":96595591168,"internally_reserved":0,"allocated":716800,"data_stored":298143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574907,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49932,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321908,"statfs":{"total":96636764160,"available":96585633792,"internally_reserved":0,"allocated":720896,"data_stored":299103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574909,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54092,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317748,"statfs":{"total":96636764160,"available":96581373952,"internally_reserved":0,"allocated":716800,"data_stored":297547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36044,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335796,"statfs":{"total":96636764160,"available":96599855104,"internally_reserved":0,"allocated":716800,"data_stored":297547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49928,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321912,"statfs":{"total":96636764160,"available":96585637888,"internally_reserved":0,"allocated":712704,"data_stored":297581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574907,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40788,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331052,"statfs":{"total":96636764160,"available":96594997248,"internally_reserved":0,"allocated":1318912,"data_stored":890053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:42:43.472 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:42:43.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.609+0000 7f0ca8e95640 1 Processor -- start 2024-04-17T10:42:43.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.609+0000 7f0ca8e95640 1 -- start start 2024-04-17T10:42:43.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.613+0000 7f0ca8e95640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ca40af530 0x7f0ca4166b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:43.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.613+0000 7f0ca8e95640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ca4160650 con 0x7f0ca415f1d0 2024-04-17T10:42:43.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.613+0000 7f0ca8e95640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ca415f5b0 con 0x7f0ca40b2de0 2024-04-17T10:42:43.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.613+0000 7f0ca8e95640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca40b2620 con 0x7f0ca40af530 2024-04-17T10:42:43.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.613+0000 7f0ca259c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0ca415f1d0 0x7f0ca415f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57470/0 (socket says 172.21.15.83:57470) 2024-04-17T10:42:43.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.613+0000 7f0ca259c640 1 -- 172.21.15.83:0/218724782 learned_addr learned my addr 172.21.15.83:0/218724782 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:43.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.613+0000 7f0ca2d9d640 1 --2- 172.21.15.83:0/218724782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ca40af530 0x7f0ca4166b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:43.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.613+0000 7f0ca159a640 1 -- 172.21.15.83:0/218724782 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3303124369 0 0) 0x7f0ca4160650 con 0x7f0ca415f1d0 2024-04-17T10:42:43.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.613+0000 7f0ca159a640 1 -- 172.21.15.83:0/218724782 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c8c003530 con 0x7f0ca415f1d0 2024-04-17T10:42:43.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.613+0000 7f0ca159a640 1 -- 172.21.15.83:0/218724782 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 960519922 0 0) 0x7f0c8c003530 con 0x7f0ca415f1d0 2024-04-17T10:42:43.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.613+0000 7f0ca159a640 1 -- 172.21.15.83:0/218724782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ca40af530 msgr2=0x7f0ca4166b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:43.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.613+0000 7f0ca159a640 1 --2- 172.21.15.83:0/218724782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ca40af530 0x7f0ca4166b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:43.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.613+0000 7f0ca159a640 1 -- 172.21.15.83:0/218724782 >> v1:172.21.15.167:6789/0 conn(0x7f0ca40b2de0 legacy=0x7f0ca415ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:43.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.613+0000 7f0ca159a640 1 -- 172.21.15.83:0/218724782 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca40b2340 con 0x7f0ca415f1d0 2024-04-17T10:42:43.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.613+0000 7f0ca159a640 1 -- 172.21.15.83:0/218724782 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f0c94001680 con 0x7f0ca415f1d0 2024-04-17T10:42:43.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.613+0000 7f0ca159a640 1 -- 172.21.15.83:0/218724782 >> v1:172.21.15.83:6789/0 conn(0x7f0ca415f1d0 legacy=0x7f0ca415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:43.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.613+0000 7f0ca159a640 1 --2- 172.21.15.83:0/218724782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c8c0039f0 0x7f0c8c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:43.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.613+0000 7f0ca159a640 1 --2- 172.21.15.83:0/218724782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c8c004810 0x7f0c8c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:43.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.613+0000 7f0ca159a640 1 --2- 172.21.15.83:0/218724782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ca40af530 0x7f0c8c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:43.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.613+0000 7f0ca159a640 1 -- 172.21.15.83:0/218724782 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca40b2620 con 0x7f0c8c0039f0 2024-04-17T10:42:43.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.613+0000 7f0ca159a640 1 -- 172.21.15.83:0/218724782 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c00bab0 con 0x7f0c8c004810 2024-04-17T10:42:43.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.613+0000 7f0ca159a640 1 -- 172.21.15.83:0/218724782 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c00bcb0 con 0x7f0ca40af530 2024-04-17T10:42:43.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.613+0000 7f0ca259c640 1 --2- 172.21.15.83:0/218724782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c8c0039f0 0x7f0c8c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:43.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.613+0000 7f0ca1d9b640 1 --2- 172.21.15.83:0/218724782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c8c004810 0x7f0c8c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:43.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.613+0000 7f0ca2d9d640 1 --2- 172.21.15.83:0/218724782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ca40af530 0x7f0c8c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:43.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.613+0000 7f0ca259c640 1 -- 172.21.15.83:0/218724782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ca40af530 msgr2=0x7f0c8c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:43.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.613+0000 7f0ca259c640 1 --2- 172.21.15.83:0/218724782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ca40af530 0x7f0c8c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:43.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.613+0000 7f0ca259c640 1 -- 172.21.15.83:0/218724782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c8c004810 msgr2=0x7f0c8c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:43.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.613+0000 7f0ca259c640 1 --2- 172.21.15.83:0/218724782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c8c004810 0x7f0c8c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:43.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.613+0000 7f0ca259c640 1 -- 172.21.15.83:0/218724782 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c8c003530 con 0x7f0c8c0039f0 2024-04-17T10:42:43.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.617+0000 7f0ca259c640 1 --2- 172.21.15.83:0/218724782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c8c0039f0 0x7f0c8c003e10 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f0c94001ec0 tx=0x7f0c9401cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c838334da282dac4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:43.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.617+0000 7f0ca159a640 1 -- 172.21.15.83:0/218724782 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c94007d40 con 0x7f0c8c0039f0 2024-04-17T10:42:43.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.617+0000 7f0ca159a640 1 -- 172.21.15.83:0/218724782 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c940a9070 con 0x7f0c8c0039f0 2024-04-17T10:42:43.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.617+0000 7f0ca159a640 1 -- 172.21.15.83:0/218724782 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c940a5d50 con 0x7f0c8c0039f0 2024-04-17T10:42:43.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.617+0000 7f0ca8e95640 1 -- 172.21.15.83:0/218724782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c8c0039f0 msgr2=0x7f0c8c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:43.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.617+0000 7f0ca8e95640 1 --2- 172.21.15.83:0/218724782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c8c0039f0 0x7f0c8c003e10 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f0c94001ec0 tx=0x7f0c9401cc10 comp rx=0 tx=0).stop 2024-04-17T10:42:43.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.617+0000 7f0ca2d9d640 1 -- 172.21.15.83:0/218724782 reap_dead start 2024-04-17T10:42:43.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.617+0000 7f0ca8e95640 1 -- 172.21.15.83:0/218724782 shutdown_connections 2024-04-17T10:42:43.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.617+0000 7f0ca8e95640 1 -- 172.21.15.83:0/218724782 >> 172.21.15.83:0/218724782 conn(0x7f0ca40d17b0 msgr2=0x7f0ca40d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:43.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.617+0000 7f0ca8e95640 1 -- 172.21.15.83:0/218724782 shutdown_connections 2024-04-17T10:42:43.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.617+0000 7f0ca8e95640 1 -- 172.21.15.83:0/218724782 wait complete. 2024-04-17T10:42:43.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.617+0000 7f0ca8e95640 1 Processor -- start 2024-04-17T10:42:43.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.617+0000 7f0ca8e95640 1 -- start start 2024-04-17T10:42:43.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.621+0000 7f0ca8e95640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c8c0039f0 0x7f0ca40c7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:43.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.621+0000 7f0ca8e95640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ca4150ff0 0x7f0ca4154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:43.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.621+0000 7f0ca8e95640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ca4150590 0x7f0ca4155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:43.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.621+0000 7f0ca8e95640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca41674f0 con 0x7f0c8c0039f0 2024-04-17T10:42:43.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.621+0000 7f0ca8e95640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca4167370 con 0x7f0ca4150590 2024-04-17T10:42:43.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.621+0000 7f0ca8e95640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca4167670 con 0x7f0ca4150ff0 2024-04-17T10:42:43.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.621+0000 7f0ca259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c8c0039f0 0x7f0ca40c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:43.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.621+0000 7f0ca259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c8c0039f0 0x7f0ca40c7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36874/0 (socket says 172.21.15.83:36874) 2024-04-17T10:42:43.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.621+0000 7f0ca259c640 1 -- 172.21.15.83:0/1710844160 learned_addr learned my addr 172.21.15.83:0/1710844160 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:43.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.621+0000 7f0ca1d9b640 1 --2- 172.21.15.83:0/1710844160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ca4150ff0 0x7f0ca4154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:43.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.621+0000 7f0ca2d9d640 1 --2- 172.21.15.83:0/1710844160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ca4150590 0x7f0ca4155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:43.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.621+0000 7f0ca259c640 1 -- 172.21.15.83:0/1710844160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ca4150ff0 msgr2=0x7f0ca4154d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:43.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.621+0000 7f0ca259c640 1 --2- 172.21.15.83:0/1710844160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ca4150ff0 0x7f0ca4154d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:43.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.621+0000 7f0ca259c640 1 -- 172.21.15.83:0/1710844160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ca4150590 msgr2=0x7f0ca4155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:43.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.621+0000 7f0ca259c640 1 --2- 172.21.15.83:0/1710844160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ca4150590 0x7f0ca4155260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:43.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.621+0000 7f0ca259c640 1 -- 172.21.15.83:0/1710844160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca40c6700 con 0x7f0c8c0039f0 2024-04-17T10:42:43.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.621+0000 7f0ca259c640 1 --2- 172.21.15.83:0/1710844160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c8c0039f0 0x7f0ca40c7740 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f0c94000c10 tx=0x7f0c8c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:43.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.621+0000 7f0c92ffd640 1 -- 172.21.15.83:0/1710844160 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c94001440 con 0x7f0c8c0039f0 2024-04-17T10:42:43.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.621+0000 7f0c92ffd640 1 -- 172.21.15.83:0/1710844160 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c94001600 con 0x7f0c8c0039f0 2024-04-17T10:42:43.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.621+0000 7f0ca8e95640 1 -- 172.21.15.83:0/1710844160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca40c6980 con 0x7f0c8c0039f0 2024-04-17T10:42:43.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.621+0000 7f0c92ffd640 1 -- 172.21.15.83:0/1710844160 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c940047f0 con 0x7f0c8c0039f0 2024-04-17T10:42:43.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.621+0000 7f0ca8e95640 1 -- 172.21.15.83:0/1710844160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca40c6f40 con 0x7f0c8c0039f0 2024-04-17T10:42:43.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.621+0000 7f0ca8e95640 1 -- 172.21.15.83:0/1710844160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ca4074f90 con 0x7f0c8c0039f0 2024-04-17T10:42:43.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.621+0000 7f0c92ffd640 1 -- 172.21.15.83:0/1710844160 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0c94004990 con 0x7f0c8c0039f0 2024-04-17T10:42:43.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.621+0000 7f0c92ffd640 1 --2- 172.21.15.83:0/1710844160 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0c84066f60 0x7f0c84069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:43.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.621+0000 7f0c92ffd640 1 -- 172.21.15.83:0/1710844160 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c9411ee30 con 0x7f0c8c0039f0 2024-04-17T10:42:43.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.621+0000 7f0ca1d9b640 1 --2- 172.21.15.83:0/1710844160 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0c84066f60 0x7f0c84069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:43.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.621+0000 7f0ca1d9b640 1 --2- 172.21.15.83:0/1710844160 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0c84066f60 0x7f0c84069420 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0ca40cac00 tx=0x7f0c98009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:43.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.625+0000 7f0c92ffd640 1 -- 172.21.15.83:0/1710844160 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0ca40c6700 con 0x7f0c8c0039f0 2024-04-17T10:42:43.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.781+0000 7f0ca8e95640 1 -- 172.21.15.83:0/1710844160 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0ca4072690 con 0x7f0c84066f60 2024-04-17T10:42:43.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.801+0000 7f0c92ffd640 1 -- 172.21.15.83:0/1710844160 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251965 (secure 0 0 0) 0x7f0ca4072690 con 0x7f0c84066f60 2024-04-17T10:42:43.803 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:43.804 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:42:43.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.805+0000 7f0ca8e95640 1 -- 172.21.15.83:0/1710844160 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0c84066f60 msgr2=0x7f0c84069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:43.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.805+0000 7f0ca8e95640 1 --2- 172.21.15.83:0/1710844160 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0c84066f60 0x7f0c84069420 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0ca40cac00 tx=0x7f0c98009040 comp rx=0 tx=0).stop 2024-04-17T10:42:43.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.805+0000 7f0ca8e95640 1 -- 172.21.15.83:0/1710844160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c8c0039f0 msgr2=0x7f0ca40c7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:43.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.805+0000 7f0ca8e95640 1 --2- 172.21.15.83:0/1710844160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c8c0039f0 0x7f0ca40c7740 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f0c94000c10 tx=0x7f0c8c005840 comp rx=0 tx=0).stop 2024-04-17T10:42:43.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.805+0000 7f0ca8e95640 1 -- 172.21.15.83:0/1710844160 shutdown_connections 2024-04-17T10:42:43.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.805+0000 7f0ca8e95640 1 --2- 172.21.15.83:0/1710844160 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0c84066f60 0x7f0c84069420 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:43.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.805+0000 7f0ca8e95640 1 --2- 172.21.15.83:0/1710844160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ca4150590 0x7f0ca4155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:43.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.805+0000 7f0ca8e95640 1 --2- 172.21.15.83:0/1710844160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ca4150ff0 0x7f0ca4154d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:43.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.805+0000 7f0ca8e95640 1 --2- 172.21.15.83:0/1710844160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c8c0039f0 0x7f0ca40c7740 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:43.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.805+0000 7f0ca8e95640 1 -- 172.21.15.83:0/1710844160 >> 172.21.15.83:0/1710844160 conn(0x7f0ca40d17b0 msgr2=0x7f0ca406e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:43.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.805+0000 7f0ca8e95640 1 -- 172.21.15.83:0/1710844160 shutdown_connections 2024-04-17T10:42:43.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.805+0000 7f0ca8e95640 1 -- 172.21.15.83:0/1710844160 wait complete. 2024-04-17T10:42:43.822 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2024-04-17T10:42:42.260786+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":180,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":356140,"kb_used_data":6760,"kb_used_omap":12,"kb_used_meta":349235,"kb_avail":754618580,"statfs":{"total":773094113280,"available":772729425920,"internally_reserved":0,"allocated":6922240,"data_stored":3565484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":357617200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002622"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.411826+0000","last_change":"2024-04-17T10:42:35.025913+0000","last_active":"2024-04-17T10:42:36.411826+0000","last_peered":"2024-04-17T10:42:36.411826+0000","last_clean":"2024-04-17T10:42:36.411826+0000","last_became_active":"2024-04-17T10:42:35.025299+0000","last_became_peered":"2024-04-17T10:42:35.025299+0000","last_unstale":"2024-04-17T10:42:36.411826+0000","last_undegraded":"2024-04-17T10:42:36.411826+0000","last_fullsized":"2024-04-17T10:42:36.411826+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:05:08.176877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.340051+0000","last_change":"2024-04-17T10:42:35.028596+0000","last_active":"2024-04-17T10:42:36.340051+0000","last_peered":"2024-04-17T10:42:36.340051+0000","last_clean":"2024-04-17T10:42:36.340051+0000","last_became_active":"2024-04-17T10:42:35.027377+0000","last_became_peered":"2024-04-17T10:42:35.027377+0000","last_unstale":"2024-04-17T10:42:36.340051+0000","last_undegraded":"2024-04-17T10:42:36.340051+0000","last_fullsized":"2024-04-17T10:42:36.340051+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:53:55.930528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.992426+0000","last_change":"2024-04-17T10:42:35.010816+0000","last_active":"2024-04-17T10:42:35.992426+0000","last_peered":"2024-04-17T10:42:35.992426+0000","last_clean":"2024-04-17T10:42:35.992426+0000","last_became_active":"2024-04-17T10:42:35.010365+0000","last_became_peered":"2024-04-17T10:42:35.010365+0000","last_unstale":"2024-04-17T10:42:35.992426+0000","last_undegraded":"2024-04-17T10:42:35.992426+0000","last_fullsized":"2024-04-17T10:42:35.992426+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:04:05.170981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.412599+0000","last_change":"2024-04-17T10:42:35.031696+0000","last_active":"2024-04-17T10:42:36.412599+0000","last_peered":"2024-04-17T10:42:36.412599+0000","last_clean":"2024-04-17T10:42:36.412599+0000","last_became_active":"2024-04-17T10:42:35.031205+0000","last_became_peered":"2024-04-17T10:42:35.031205+0000","last_unstale":"2024-04-17T10:42:36.412599+0000","last_undegraded":"2024-04-17T10:42:36.412599+0000","last_fullsized":"2024-04-17T10:42:36.412599+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:37:35.341273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.412273+0000","last_change":"2024-04-17T10:42:35.022055+0000","last_active":"2024-04-17T10:42:36.412273+0000","last_peered":"2024-04-17T10:42:36.412273+0000","last_clean":"2024-04-17T10:42:36.412273+0000","last_became_active":"2024-04-17T10:42:35.021773+0000","last_became_peered":"2024-04-17T10:42:35.021773+0000","last_unstale":"2024-04-17T10:42:36.412273+0000","last_undegraded":"2024-04-17T10:42:36.412273+0000","last_fullsized":"2024-04-17T10:42:36.412273+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:39:07.167173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.986747+0000","last_change":"2024-04-17T10:42:35.010720+0000","last_active":"2024-04-17T10:42:35.986747+0000","last_peered":"2024-04-17T10:42:35.986747+0000","last_clean":"2024-04-17T10:42:35.986747+0000","last_became_active":"2024-04-17T10:42:35.010119+0000","last_became_peered":"2024-04-17T10:42:35.010119+0000","last_unstale":"2024-04-17T10:42:35.986747+0000","last_undegraded":"2024-04-17T10:42:35.986747+0000","last_fullsized":"2024-04-17T10:42:35.986747+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:43:28.482310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.989458+0000","last_change":"2024-04-17T10:42:35.030532+0000","last_active":"2024-04-17T10:42:35.989458+0000","last_peered":"2024-04-17T10:42:35.989458+0000","last_clean":"2024-04-17T10:42:35.989458+0000","last_became_active":"2024-04-17T10:42:35.029381+0000","last_became_peered":"2024-04-17T10:42:35.029381+0000","last_unstale":"2024-04-17T10:42:35.989458+0000","last_undegraded":"2024-04-17T10:42:35.989458+0000","last_fullsized":"2024-04-17T10:42:35.989458+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:32:39.595630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.339360+0000","last_change":"2024-04-17T10:42:35.018376+0000","last_active":"2024-04-17T10:42:36.339360+0000","last_peered":"2024-04-17T10:42:36.339360+0000","last_clean":"2024-04-17T10:42:36.339360+0000","last_became_active":"2024-04-17T10:42:35.018017+0000","last_became_peered":"2024-04-17T10:42:35.018017+0000","last_unstale":"2024-04-17T10:42:36.339360+0000","last_undegraded":"2024-04-17T10:42:36.339360+0000","last_fullsized":"2024-04-17T10:42:36.339360+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:27:46.041944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.990864+0000","last_change":"2024-04-17T10:42:35.024624+0000","last_active":"2024-04-17T10:42:35.990864+0000","last_peered":"2024-04-17T10:42:35.990864+0000","last_clean":"2024-04-17T10:42:35.990864+0000","last_became_active":"2024-04-17T10:42:35.022047+0000","last_became_peered":"2024-04-17T10:42:35.022047+0000","last_unstale":"2024-04-17T10:42:35.990864+0000","last_undegraded":"2024-04-17T10:42:35.990864+0000","last_fullsized":"2024-04-17T10:42:35.990864+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:01:34.922297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.411613+0000","last_change":"2024-04-17T10:42:35.006487+0000","last_active":"2024-04-17T10:42:36.411613+0000","last_peered":"2024-04-17T10:42:36.411613+0000","last_clean":"2024-04-17T10:42:36.411613+0000","last_became_active":"2024-04-17T10:42:35.005168+0000","last_became_peered":"2024-04-17T10:42:35.005168+0000","last_unstale":"2024-04-17T10:42:36.411613+0000","last_undegraded":"2024-04-17T10:42:36.411613+0000","last_fullsized":"2024-04-17T10:42:36.411613+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:26:58.999145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.411768+0000","last_change":"2024-04-17T10:42:35.026150+0000","last_active":"2024-04-17T10:42:36.411768+0000","last_peered":"2024-04-17T10:42:36.411768+0000","last_clean":"2024-04-17T10:42:36.411768+0000","last_became_active":"2024-04-17T10:42:35.025867+0000","last_became_peered":"2024-04-17T10:42:35.025867+0000","last_unstale":"2024-04-17T10:42:36.411768+0000","last_undegraded":"2024-04-17T10:42:36.411768+0000","last_fullsized":"2024-04-17T10:42:36.411768+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:58:19.216435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.412734+0000","last_change":"2024-04-17T10:42:35.029601+0000","last_active":"2024-04-17T10:42:36.412734+0000","last_peered":"2024-04-17T10:42:36.412734+0000","last_clean":"2024-04-17T10:42:36.412734+0000","last_became_active":"2024-04-17T10:42:35.027716+0000","last_became_peered":"2024-04-17T10:42:35.027716+0000","last_unstale":"2024-04-17T10:42:36.412734+0000","last_undegraded":"2024-04-17T10:42:36.412734+0000","last_fullsized":"2024-04-17T10:42:36.412734+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:29:43.713428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.007304+0000","last_change":"2024-04-17T10:42:35.016481+0000","last_active":"2024-04-17T10:42:36.007304+0000","last_peered":"2024-04-17T10:42:36.007304+0000","last_clean":"2024-04-17T10:42:36.007304+0000","last_became_active":"2024-04-17T10:42:35.016153+0000","last_became_peered":"2024-04-17T10:42:35.016153+0000","last_unstale":"2024-04-17T10:42:36.007304+0000","last_undegraded":"2024-04-17T10:42:36.007304+0000","last_fullsized":"2024-04-17T10:42:36.007304+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:47:15.899420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.989807+0000","last_change":"2024-04-17T10:42:35.008905+0000","last_active":"2024-04-17T10:42:35.989807+0000","last_peered":"2024-04-17T10:42:35.989807+0000","last_clean":"2024-04-17T10:42:35.989807+0000","last_became_active":"2024-04-17T10:42:35.008293+0000","last_became_peered":"2024-04-17T10:42:35.008293+0000","last_unstale":"2024-04-17T10:42:35.989807+0000","last_undegraded":"2024-04-17T10:42:35.989807+0000","last_fullsized":"2024-04-17T10:42:35.989807+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:41:53.757975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.989763+0000","last_change":"2024-04-17T10:42:35.024840+0000","last_active":"2024-04-17T10:42:35.989763+0000","last_peered":"2024-04-17T10:42:35.989763+0000","last_clean":"2024-04-17T10:42:35.989763+0000","last_became_active":"2024-04-17T10:42:35.022239+0000","last_became_peered":"2024-04-17T10:42:35.022239+0000","last_unstale":"2024-04-17T10:42:35.989763+0000","last_undegraded":"2024-04-17T10:42:35.989763+0000","last_fullsized":"2024-04-17T10:42:35.989763+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:33:42.502740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.986677+0000","last_change":"2024-04-17T10:42:35.018664+0000","last_active":"2024-04-17T10:42:35.986677+0000","last_peered":"2024-04-17T10:42:35.986677+0000","last_clean":"2024-04-17T10:42:35.986677+0000","last_became_active":"2024-04-17T10:42:35.018257+0000","last_became_peered":"2024-04-17T10:42:35.018257+0000","last_unstale":"2024-04-17T10:42:35.986677+0000","last_undegraded":"2024-04-17T10:42:35.986677+0000","last_fullsized":"2024-04-17T10:42:35.986677+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:35:46.679023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.339968+0000","last_change":"2024-04-17T10:42:35.025373+0000","last_active":"2024-04-17T10:42:36.339968+0000","last_peered":"2024-04-17T10:42:36.339968+0000","last_clean":"2024-04-17T10:42:36.339968+0000","last_became_active":"2024-04-17T10:42:35.025055+0000","last_became_peered":"2024-04-17T10:42:35.025055+0000","last_unstale":"2024-04-17T10:42:36.339968+0000","last_undegraded":"2024-04-17T10:42:36.339968+0000","last_fullsized":"2024-04-17T10:42:36.339968+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:04:47.145034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.992756+0000","last_change":"2024-04-17T10:42:35.017543+0000","last_active":"2024-04-17T10:42:35.992756+0000","last_peered":"2024-04-17T10:42:35.992756+0000","last_clean":"2024-04-17T10:42:35.992756+0000","last_became_active":"2024-04-17T10:42:35.017003+0000","last_became_peered":"2024-04-17T10:42:35.017003+0000","last_unstale":"2024-04-17T10:42:35.992756+0000","last_undegraded":"2024-04-17T10:42:35.992756+0000","last_fullsized":"2024-04-17T10:42:35.992756+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:33:02.276726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.411690+0000","last_change":"2024-04-17T10:42:35.015960+0000","last_active":"2024-04-17T10:42:36.411690+0000","last_peered":"2024-04-17T10:42:36.411690+0000","last_clean":"2024-04-17T10:42:36.411690+0000","last_became_active":"2024-04-17T10:42:35.015589+0000","last_became_peered":"2024-04-17T10:42:35.015589+0000","last_unstale":"2024-04-17T10:42:36.411690+0000","last_undegraded":"2024-04-17T10:42:36.411690+0000","last_fullsized":"2024-04-17T10:42:36.411690+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:10:09.266430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.340151+0000","last_change":"2024-04-17T10:42:35.027129+0000","last_active":"2024-04-17T10:42:36.340151+0000","last_peered":"2024-04-17T10:42:36.340151+0000","last_clean":"2024-04-17T10:42:36.340151+0000","last_became_active":"2024-04-17T10:42:35.026663+0000","last_became_peered":"2024-04-17T10:42:35.026663+0000","last_unstale":"2024-04-17T10:42:36.340151+0000","last_undegraded":"2024-04-17T10:42:36.340151+0000","last_fullsized":"2024-04-17T10:42:36.340151+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:41:27.625355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.992079+0000","last_change":"2024-04-17T10:42:35.024264+0000","last_active":"2024-04-17T10:42:35.992079+0000","last_peered":"2024-04-17T10:42:35.992079+0000","last_clean":"2024-04-17T10:42:35.992079+0000","last_became_active":"2024-04-17T10:42:35.023746+0000","last_became_peered":"2024-04-17T10:42:35.023746+0000","last_unstale":"2024-04-17T10:42:35.992079+0000","last_undegraded":"2024-04-17T10:42:35.992079+0000","last_fullsized":"2024-04-17T10:42:35.992079+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:53:20.596275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.339478+0000","last_change":"2024-04-17T10:42:35.027645+0000","last_active":"2024-04-17T10:42:36.339478+0000","last_peered":"2024-04-17T10:42:36.339478+0000","last_clean":"2024-04-17T10:42:36.339478+0000","last_became_active":"2024-04-17T10:42:35.023180+0000","last_became_peered":"2024-04-17T10:42:35.023180+0000","last_unstale":"2024-04-17T10:42:36.339478+0000","last_undegraded":"2024-04-17T10:42:36.339478+0000","last_fullsized":"2024-04-17T10:42:36.339478+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:45:46.184170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.991418+0000","last_change":"2024-04-17T10:42:35.018399+0000","last_active":"2024-04-17T10:42:35.991418+0000","last_peered":"2024-04-17T10:42:35.991418+0000","last_clean":"2024-04-17T10:42:35.991418+0000","last_became_active":"2024-04-17T10:42:35.018087+0000","last_became_peered":"2024-04-17T10:42:35.018087+0000","last_unstale":"2024-04-17T10:42:35.991418+0000","last_undegraded":"2024-04-17T10:42:35.991418+0000","last_fullsized":"2024-04-17T10:42:35.991418+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:40:05.316943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.007746+0000","last_change":"2024-04-17T10:42:35.016286+0000","last_active":"2024-04-17T10:42:36.007746+0000","last_peered":"2024-04-17T10:42:36.007746+0000","last_clean":"2024-04-17T10:42:36.007746+0000","last_became_active":"2024-04-17T10:42:35.015899+0000","last_became_peered":"2024-04-17T10:42:35.015899+0000","last_unstale":"2024-04-17T10:42:36.007746+0000","last_undegraded":"2024-04-17T10:42:36.007746+0000","last_fullsized":"2024-04-17T10:42:36.007746+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:29:50.762209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.412205+0000","last_change":"2024-04-17T10:42:35.000543+0000","last_active":"2024-04-17T10:42:36.412205+0000","last_peered":"2024-04-17T10:42:36.412205+0000","last_clean":"2024-04-17T10:42:36.412205+0000","last_became_active":"2024-04-17T10:42:34.999131+0000","last_became_peered":"2024-04-17T10:42:34.999131+0000","last_unstale":"2024-04-17T10:42:36.412205+0000","last_undegraded":"2024-04-17T10:42:36.412205+0000","last_fullsized":"2024-04-17T10:42:36.412205+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:42:45.021574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.412784+0000","last_change":"2024-04-17T10:42:35.028556+0000","last_active":"2024-04-17T10:42:36.412784+0000","last_peered":"2024-04-17T10:42:36.412784+0000","last_clean":"2024-04-17T10:42:36.412784+0000","last_became_active":"2024-04-17T10:42:35.027301+0000","last_became_peered":"2024-04-17T10:42:35.027301+0000","last_unstale":"2024-04-17T10:42:36.412784+0000","last_undegraded":"2024-04-17T10:42:36.412784+0000","last_fullsized":"2024-04-17T10:42:36.412784+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:11:46.657622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.987320+0000","last_change":"2024-04-17T10:42:35.008932+0000","last_active":"2024-04-17T10:42:35.987320+0000","last_peered":"2024-04-17T10:42:35.987320+0000","last_clean":"2024-04-17T10:42:35.987320+0000","last_became_active":"2024-04-17T10:42:35.008359+0000","last_became_peered":"2024-04-17T10:42:35.008359+0000","last_unstale":"2024-04-17T10:42:35.987320+0000","last_undegraded":"2024-04-17T10:42:35.987320+0000","last_fullsized":"2024-04-17T10:42:35.987320+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:32:23.762419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.411683+0000","last_change":"2024-04-17T10:42:35.026972+0000","last_active":"2024-04-17T10:42:36.411683+0000","last_peered":"2024-04-17T10:42:36.411683+0000","last_clean":"2024-04-17T10:42:36.411683+0000","last_became_active":"2024-04-17T10:42:35.026597+0000","last_became_peered":"2024-04-17T10:42:35.026597+0000","last_unstale":"2024-04-17T10:42:36.411683+0000","last_undegraded":"2024-04-17T10:42:36.411683+0000","last_fullsized":"2024-04-17T10:42:36.411683+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:32:22.701587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.340364+0000","last_change":"2024-04-17T10:42:35.024054+0000","last_active":"2024-04-17T10:42:36.340364+0000","last_peered":"2024-04-17T10:42:36.340364+0000","last_clean":"2024-04-17T10:42:36.340364+0000","last_became_active":"2024-04-17T10:42:35.023243+0000","last_became_peered":"2024-04-17T10:42:35.023243+0000","last_unstale":"2024-04-17T10:42:36.340364+0000","last_undegraded":"2024-04-17T10:42:36.340364+0000","last_fullsized":"2024-04-17T10:42:36.340364+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:47:08.648118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.991827+0000","last_change":"2024-04-17T10:42:35.010569+0000","last_active":"2024-04-17T10:42:35.991827+0000","last_peered":"2024-04-17T10:42:35.991827+0000","last_clean":"2024-04-17T10:42:35.991827+0000","last_became_active":"2024-04-17T10:42:35.009896+0000","last_became_peered":"2024-04-17T10:42:35.009896+0000","last_unstale":"2024-04-17T10:42:35.991827+0000","last_undegraded":"2024-04-17T10:42:35.991827+0000","last_fullsized":"2024-04-17T10:42:35.991827+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:29:43.713428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.410618+0000","last_change":"2024-04-17T10:42:35.004653+0000","last_active":"2024-04-17T10:42:36.410618+0000","last_peered":"2024-04-17T10:42:36.410618+0000","last_clean":"2024-04-17T10:42:36.410618+0000","last_became_active":"2024-04-17T10:42:35.003785+0000","last_became_peered":"2024-04-17T10:42:35.003785+0000","last_unstale":"2024-04-17T10:42:36.410618+0000","last_undegraded":"2024-04-17T10:42:36.410618+0000","last_fullsized":"2024-04-17T10:42:36.410618+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:52:44.215335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.986791+0000","last_change":"2024-04-17T10:42:35.030273+0000","last_active":"2024-04-17T10:42:35.986791+0000","last_peered":"2024-04-17T10:42:35.986791+0000","last_clean":"2024-04-17T10:42:35.986791+0000","last_became_active":"2024-04-17T10:42:35.029890+0000","last_became_peered":"2024-04-17T10:42:35.029890+0000","last_unstale":"2024-04-17T10:42:35.986791+0000","last_undegraded":"2024-04-17T10:42:35.986791+0000","last_fullsized":"2024-04-17T10:42:35.986791+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:40:28.617010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.987620+0000","last_change":"2024-04-17T10:42:35.009772+0000","last_active":"2024-04-17T10:42:35.987620+0000","last_peered":"2024-04-17T10:42:35.987620+0000","last_clean":"2024-04-17T10:42:35.987620+0000","last_became_active":"2024-04-17T10:42:35.009014+0000","last_became_peered":"2024-04-17T10:42:35.009014+0000","last_unstale":"2024-04-17T10:42:35.987620+0000","last_undegraded":"2024-04-17T10:42:35.987620+0000","last_fullsized":"2024-04-17T10:42:35.987620+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:40:05.316943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"58'4","reported_seq":18,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.987646+0000","last_change":"2024-04-17T10:42:34.015019+0000","last_active":"2024-04-17T10:42:35.987646+0000","last_peered":"2024-04-17T10:42:35.987646+0000","last_clean":"2024-04-17T10:42:35.987646+0000","last_became_active":"2024-04-17T10:42:34.014716+0000","last_became_peered":"2024-04-17T10:42:34.014716+0000","last_unstale":"2024-04-17T10:42:35.987646+0000","last_undegraded":"2024-04-17T10:42:35.987646+0000","last_fullsized":"2024-04-17T10:42:35.987646+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:43:53.941771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.412270+0000","last_change":"2024-04-17T10:42:35.002129+0000","last_active":"2024-04-17T10:42:36.412270+0000","last_peered":"2024-04-17T10:42:36.412270+0000","last_clean":"2024-04-17T10:42:36.412270+0000","last_became_active":"2024-04-17T10:42:35.001174+0000","last_became_peered":"2024-04-17T10:42:35.001174+0000","last_unstale":"2024-04-17T10:42:36.412270+0000","last_undegraded":"2024-04-17T10:42:36.412270+0000","last_fullsized":"2024-04-17T10:42:36.412270+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:34:52.474792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.339797+0000","last_change":"2024-04-17T10:42:34.006170+0000","last_active":"2024-04-17T10:42:36.339797+0000","last_peered":"2024-04-17T10:42:36.339797+0000","last_clean":"2024-04-17T10:42:36.339797+0000","last_became_active":"2024-04-17T10:42:34.005757+0000","last_became_peered":"2024-04-17T10:42:34.005757+0000","last_unstale":"2024-04-17T10:42:36.339797+0000","last_undegraded":"2024-04-17T10:42:36.339797+0000","last_fullsized":"2024-04-17T10:42:36.339797+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:35:45.666827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.987870+0000","last_change":"2024-04-17T10:42:35.013753+0000","last_active":"2024-04-17T10:42:35.987870+0000","last_peered":"2024-04-17T10:42:35.987870+0000","last_clean":"2024-04-17T10:42:35.987870+0000","last_became_active":"2024-04-17T10:42:35.013440+0000","last_became_peered":"2024-04-17T10:42:35.013440+0000","last_unstale":"2024-04-17T10:42:35.987870+0000","last_undegraded":"2024-04-17T10:42:35.987870+0000","last_fullsized":"2024-04-17T10:42:35.987870+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:37:35.341273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.987433+0000","last_change":"2024-04-17T10:42:33.993426+0000","last_active":"2024-04-17T10:42:35.987433+0000","last_peered":"2024-04-17T10:42:35.987433+0000","last_clean":"2024-04-17T10:42:35.987433+0000","last_became_active":"2024-04-17T10:42:33.992971+0000","last_became_peered":"2024-04-17T10:42:33.992971+0000","last_unstale":"2024-04-17T10:42:35.987433+0000","last_undegraded":"2024-04-17T10:42:35.987433+0000","last_fullsized":"2024-04-17T10:42:35.987433+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:32:27.523313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.987034+0000","last_change":"2024-04-17T10:42:35.027423+0000","last_active":"2024-04-17T10:42:35.987034+0000","last_peered":"2024-04-17T10:42:35.987034+0000","last_clean":"2024-04-17T10:42:35.987034+0000","last_became_active":"2024-04-17T10:42:35.027017+0000","last_became_peered":"2024-04-17T10:42:35.027017+0000","last_unstale":"2024-04-17T10:42:35.987034+0000","last_undegraded":"2024-04-17T10:42:35.987034+0000","last_fullsized":"2024-04-17T10:42:35.987034+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:30:45.283107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.412995+0000","last_change":"2024-04-17T10:42:34.018234+0000","last_active":"2024-04-17T10:42:36.412995+0000","last_peered":"2024-04-17T10:42:36.412995+0000","last_clean":"2024-04-17T10:42:36.412995+0000","last_became_active":"2024-04-17T10:42:34.017497+0000","last_became_peered":"2024-04-17T10:42:34.017497+0000","last_unstale":"2024-04-17T10:42:36.412995+0000","last_undegraded":"2024-04-17T10:42:36.412995+0000","last_fullsized":"2024-04-17T10:42:36.412995+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:25:59.364150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.339385+0000","last_change":"2024-04-17T10:42:35.023936+0000","last_active":"2024-04-17T10:42:36.339385+0000","last_peered":"2024-04-17T10:42:36.339385+0000","last_clean":"2024-04-17T10:42:36.339385+0000","last_became_active":"2024-04-17T10:42:35.022583+0000","last_became_peered":"2024-04-17T10:42:35.022583+0000","last_unstale":"2024-04-17T10:42:36.339385+0000","last_undegraded":"2024-04-17T10:42:36.339385+0000","last_fullsized":"2024-04-17T10:42:36.339385+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:54:37.642863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.990342+0000","last_change":"2024-04-17T10:42:34.004483+0000","last_active":"2024-04-17T10:42:35.990342+0000","last_peered":"2024-04-17T10:42:35.990342+0000","last_clean":"2024-04-17T10:42:35.990342+0000","last_became_active":"2024-04-17T10:42:33.998883+0000","last_became_peered":"2024-04-17T10:42:33.998883+0000","last_unstale":"2024-04-17T10:42:35.990342+0000","last_undegraded":"2024-04-17T10:42:35.990342+0000","last_fullsized":"2024-04-17T10:42:35.990342+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:34:51.462597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.339435+0000","last_change":"2024-04-17T10:42:35.028013+0000","last_active":"2024-04-17T10:42:36.339435+0000","last_peered":"2024-04-17T10:42:36.339435+0000","last_clean":"2024-04-17T10:42:36.339435+0000","last_became_active":"2024-04-17T10:42:35.025898+0000","last_became_peered":"2024-04-17T10:42:35.025898+0000","last_unstale":"2024-04-17T10:42:36.339435+0000","last_undegraded":"2024-04-17T10:42:36.339435+0000","last_fullsized":"2024-04-17T10:42:36.339435+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:11:09.719074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.987023+0000","last_change":"2024-04-17T10:42:34.000321+0000","last_active":"2024-04-17T10:42:35.987023+0000","last_peered":"2024-04-17T10:42:35.987023+0000","last_clean":"2024-04-17T10:42:35.987023+0000","last_became_active":"2024-04-17T10:42:34.000062+0000","last_became_peered":"2024-04-17T10:42:34.000062+0000","last_unstale":"2024-04-17T10:42:35.987023+0000","last_undegraded":"2024-04-17T10:42:35.987023+0000","last_fullsized":"2024-04-17T10:42:35.987023+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:42:38.783160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.007759+0000","last_change":"2024-04-17T10:42:34.989852+0000","last_active":"2024-04-17T10:42:36.007759+0000","last_peered":"2024-04-17T10:42:36.007759+0000","last_clean":"2024-04-17T10:42:36.007759+0000","last_became_active":"2024-04-17T10:42:34.989121+0000","last_became_peered":"2024-04-17T10:42:34.989121+0000","last_unstale":"2024-04-17T10:42:36.007759+0000","last_undegraded":"2024-04-17T10:42:36.007759+0000","last_fullsized":"2024-04-17T10:42:36.007759+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:17:25.136437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.412216+0000","last_change":"2024-04-17T10:42:34.018055+0000","last_active":"2024-04-17T10:42:36.412216+0000","last_peered":"2024-04-17T10:42:36.412216+0000","last_clean":"2024-04-17T10:42:36.412216+0000","last_became_active":"2024-04-17T10:42:34.016719+0000","last_became_peered":"2024-04-17T10:42:34.016719+0000","last_unstale":"2024-04-17T10:42:36.412216+0000","last_undegraded":"2024-04-17T10:42:36.412216+0000","last_fullsized":"2024-04-17T10:42:36.412216+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:41:52.745779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.339561+0000","last_change":"2024-04-17T10:42:35.026185+0000","last_active":"2024-04-17T10:42:36.339561+0000","last_peered":"2024-04-17T10:42:36.339561+0000","last_clean":"2024-04-17T10:42:36.339561+0000","last_became_active":"2024-04-17T10:42:35.025681+0000","last_became_peered":"2024-04-17T10:42:35.025681+0000","last_unstale":"2024-04-17T10:42:36.339561+0000","last_undegraded":"2024-04-17T10:42:36.339561+0000","last_fullsized":"2024-04-17T10:42:36.339561+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:10:29.515616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.989337+0000","last_change":"2024-04-17T10:42:34.008774+0000","last_active":"2024-04-17T10:42:35.989337+0000","last_peered":"2024-04-17T10:42:35.989337+0000","last_clean":"2024-04-17T10:42:35.989337+0000","last_became_active":"2024-04-17T10:42:34.005106+0000","last_became_peered":"2024-04-17T10:42:34.005106+0000","last_unstale":"2024-04-17T10:42:35.989337+0000","last_undegraded":"2024-04-17T10:42:35.989337+0000","last_fullsized":"2024-04-17T10:42:35.989337+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:26:57.986949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.991108+0000","last_change":"2024-04-17T10:42:35.019578+0000","last_active":"2024-04-17T10:42:35.991108+0000","last_peered":"2024-04-17T10:42:35.991108+0000","last_clean":"2024-04-17T10:42:35.991108+0000","last_became_active":"2024-04-17T10:42:35.013952+0000","last_became_peered":"2024-04-17T10:42:35.013952+0000","last_unstale":"2024-04-17T10:42:35.991108+0000","last_undegraded":"2024-04-17T10:42:35.991108+0000","last_fullsized":"2024-04-17T10:42:35.991108+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:26:00.376345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.992745+0000","last_change":"2024-04-17T10:42:34.006222+0000","last_active":"2024-04-17T10:42:35.992745+0000","last_peered":"2024-04-17T10:42:35.992745+0000","last_clean":"2024-04-17T10:42:35.992745+0000","last_became_active":"2024-04-17T10:42:34.005353+0000","last_became_peered":"2024-04-17T10:42:34.005353+0000","last_unstale":"2024-04-17T10:42:35.992745+0000","last_undegraded":"2024-04-17T10:42:35.992745+0000","last_fullsized":"2024-04-17T10:42:35.992745+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:41:52.745779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.992681+0000","last_change":"2024-04-17T10:42:35.010552+0000","last_active":"2024-04-17T10:42:35.992681+0000","last_peered":"2024-04-17T10:42:35.992681+0000","last_clean":"2024-04-17T10:42:35.992681+0000","last_became_active":"2024-04-17T10:42:35.010242+0000","last_became_peered":"2024-04-17T10:42:35.010242+0000","last_unstale":"2024-04-17T10:42:35.992681+0000","last_undegraded":"2024-04-17T10:42:35.992681+0000","last_fullsized":"2024-04-17T10:42:35.992681+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:37:45.633967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.340089+0000","last_change":"2024-04-17T10:42:33.995027+0000","last_active":"2024-04-17T10:42:36.340089+0000","last_peered":"2024-04-17T10:42:36.340089+0000","last_clean":"2024-04-17T10:42:36.340089+0000","last_became_active":"2024-04-17T10:42:33.994748+0000","last_became_peered":"2024-04-17T10:42:33.994748+0000","last_unstale":"2024-04-17T10:42:36.340089+0000","last_undegraded":"2024-04-17T10:42:36.340089+0000","last_fullsized":"2024-04-17T10:42:36.340089+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:57:31.575219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.411404+0000","last_change":"2024-04-17T10:42:35.014305+0000","last_active":"2024-04-17T10:42:36.411404+0000","last_peered":"2024-04-17T10:42:36.411404+0000","last_clean":"2024-04-17T10:42:36.411404+0000","last_became_active":"2024-04-17T10:42:35.011452+0000","last_became_peered":"2024-04-17T10:42:35.011452+0000","last_unstale":"2024-04-17T10:42:36.411404+0000","last_undegraded":"2024-04-17T10:42:36.411404+0000","last_fullsized":"2024-04-17T10:42:36.411404+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:14:15.302132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"58'3","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.993111+0000","last_change":"2024-04-17T10:42:34.010793+0000","last_active":"2024-04-17T10:42:35.993111+0000","last_peered":"2024-04-17T10:42:35.993111+0000","last_clean":"2024-04-17T10:42:35.993111+0000","last_became_active":"2024-04-17T10:42:34.009786+0000","last_became_peered":"2024-04-17T10:42:34.009786+0000","last_unstale":"2024-04-17T10:42:35.993111+0000","last_undegraded":"2024-04-17T10:42:35.993111+0000","last_fullsized":"2024-04-17T10:42:35.993111+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:33:41.490545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.992902+0000","last_change":"2024-04-17T10:42:35.010650+0000","last_active":"2024-04-17T10:42:35.992902+0000","last_peered":"2024-04-17T10:42:35.992902+0000","last_clean":"2024-04-17T10:42:35.992902+0000","last_became_active":"2024-04-17T10:42:35.009392+0000","last_became_peered":"2024-04-17T10:42:35.009392+0000","last_unstale":"2024-04-17T10:42:35.992902+0000","last_undegraded":"2024-04-17T10:42:35.992902+0000","last_fullsized":"2024-04-17T10:42:35.992902+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:37:51.467889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"58'6","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.991000+0000","last_change":"2024-04-17T10:42:34.008738+0000","last_active":"2024-04-17T10:42:35.991000+0000","last_peered":"2024-04-17T10:42:35.991000+0000","last_clean":"2024-04-17T10:42:35.991000+0000","last_became_active":"2024-04-17T10:42:34.004730+0000","last_became_peered":"2024-04-17T10:42:34.004730+0000","last_unstale":"2024-04-17T10:42:35.991000+0000","last_undegraded":"2024-04-17T10:42:35.991000+0000","last_fullsized":"2024-04-17T10:42:35.991000+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:10:36.224186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.412233+0000","last_change":"2024-04-17T10:42:35.023535+0000","last_active":"2024-04-17T10:42:36.412233+0000","last_peered":"2024-04-17T10:42:36.412233+0000","last_clean":"2024-04-17T10:42:36.412233+0000","last_became_active":"2024-04-17T10:42:35.023146+0000","last_became_peered":"2024-04-17T10:42:35.023146+0000","last_unstale":"2024-04-17T10:42:36.412233+0000","last_undegraded":"2024-04-17T10:42:36.412233+0000","last_fullsized":"2024-04-17T10:42:36.412233+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:30:04.131306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.339827+0000","last_change":"2024-04-17T10:42:34.000022+0000","last_active":"2024-04-17T10:42:36.339827+0000","last_peered":"2024-04-17T10:42:36.339827+0000","last_clean":"2024-04-17T10:42:36.339827+0000","last_became_active":"2024-04-17T10:42:33.999505+0000","last_became_peered":"2024-04-17T10:42:33.999505+0000","last_unstale":"2024-04-17T10:42:36.339827+0000","last_undegraded":"2024-04-17T10:42:36.339827+0000","last_fullsized":"2024-04-17T10:42:36.339827+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:43:27.470115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.991994+0000","last_change":"2024-04-17T10:42:34.993799+0000","last_active":"2024-04-17T10:42:35.991994+0000","last_peered":"2024-04-17T10:42:35.991994+0000","last_clean":"2024-04-17T10:42:35.991994+0000","last_became_active":"2024-04-17T10:42:34.992986+0000","last_became_peered":"2024-04-17T10:42:34.992986+0000","last_unstale":"2024-04-17T10:42:35.991994+0000","last_undegraded":"2024-04-17T10:42:35.991994+0000","last_fullsized":"2024-04-17T10:42:35.991994+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:11:46.657622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.007266+0000","last_change":"2024-04-17T10:42:34.010051+0000","last_active":"2024-04-17T10:42:36.007266+0000","last_peered":"2024-04-17T10:42:36.007266+0000","last_clean":"2024-04-17T10:42:36.007266+0000","last_became_active":"2024-04-17T10:42:34.009227+0000","last_became_peered":"2024-04-17T10:42:34.009227+0000","last_unstale":"2024-04-17T10:42:36.007266+0000","last_undegraded":"2024-04-17T10:42:36.007266+0000","last_fullsized":"2024-04-17T10:42:36.007266+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:31.397161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.990373+0000","last_change":"2024-04-17T10:42:35.008635+0000","last_active":"2024-04-17T10:42:35.990373+0000","last_peered":"2024-04-17T10:42:35.990373+0000","last_clean":"2024-04-17T10:42:35.990373+0000","last_became_active":"2024-04-17T10:42:35.008288+0000","last_became_peered":"2024-04-17T10:42:35.008288+0000","last_unstale":"2024-04-17T10:42:35.990373+0000","last_undegraded":"2024-04-17T10:42:35.990373+0000","last_fullsized":"2024-04-17T10:42:35.990373+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:03:52.712078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.412237+0000","last_change":"2024-04-17T10:42:34.019631+0000","last_active":"2024-04-17T10:42:36.412237+0000","last_peered":"2024-04-17T10:42:36.412237+0000","last_clean":"2024-04-17T10:42:36.412237+0000","last_became_active":"2024-04-17T10:42:34.018768+0000","last_became_peered":"2024-04-17T10:42:34.018768+0000","last_unstale":"2024-04-17T10:42:36.412237+0000","last_undegraded":"2024-04-17T10:42:36.412237+0000","last_fullsized":"2024-04-17T10:42:36.412237+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:03:51.699883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.991712+0000","last_change":"2024-04-17T10:42:35.030930+0000","last_active":"2024-04-17T10:42:35.991712+0000","last_peered":"2024-04-17T10:42:35.991712+0000","last_clean":"2024-04-17T10:42:35.991712+0000","last_became_active":"2024-04-17T10:42:35.029914+0000","last_became_peered":"2024-04-17T10:42:35.029914+0000","last_unstale":"2024-04-17T10:42:35.991712+0000","last_undegraded":"2024-04-17T10:42:35.991712+0000","last_fullsized":"2024-04-17T10:42:35.991712+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:37:35.341273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"58'8","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.990785+0000","last_change":"2024-04-17T10:42:34.015072+0000","last_active":"2024-04-17T10:42:35.990785+0000","last_peered":"2024-04-17T10:42:35.990785+0000","last_clean":"2024-04-17T10:42:35.990785+0000","last_became_active":"2024-04-17T10:42:34.014406+0000","last_became_peered":"2024-04-17T10:42:34.014406+0000","last_unstale":"2024-04-17T10:42:35.990785+0000","last_undegraded":"2024-04-17T10:42:35.990785+0000","last_fullsized":"2024-04-17T10:42:35.990785+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:32:38.583434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.992839+0000","last_change":"2024-04-17T10:42:35.012444+0000","last_active":"2024-04-17T10:42:35.992839+0000","last_peered":"2024-04-17T10:42:35.992839+0000","last_clean":"2024-04-17T10:42:35.992839+0000","last_became_active":"2024-04-17T10:42:35.011922+0000","last_became_peered":"2024-04-17T10:42:35.011922+0000","last_unstale":"2024-04-17T10:42:35.992839+0000","last_undegraded":"2024-04-17T10:42:35.992839+0000","last_fullsized":"2024-04-17T10:42:35.992839+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:32:08.445909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.340498+0000","last_change":"2024-04-17T10:42:33.998582+0000","last_active":"2024-04-17T10:42:36.340498+0000","last_peered":"2024-04-17T10:42:36.340498+0000","last_clean":"2024-04-17T10:42:36.340498+0000","last_became_active":"2024-04-17T10:42:33.997341+0000","last_became_peered":"2024-04-17T10:42:33.997341+0000","last_unstale":"2024-04-17T10:42:36.340498+0000","last_undegraded":"2024-04-17T10:42:36.340498+0000","last_fullsized":"2024-04-17T10:42:36.340498+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:32:22.750224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.412893+0000","last_change":"2024-04-17T10:42:35.029792+0000","last_active":"2024-04-17T10:42:36.412893+0000","last_peered":"2024-04-17T10:42:36.412893+0000","last_clean":"2024-04-17T10:42:36.412893+0000","last_became_active":"2024-04-17T10:42:35.029384+0000","last_became_peered":"2024-04-17T10:42:35.029384+0000","last_unstale":"2024-04-17T10:42:36.412893+0000","last_undegraded":"2024-04-17T10:42:36.412893+0000","last_fullsized":"2024-04-17T10:42:36.412893+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:53:20.596275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.987878+0000","last_change":"2024-04-17T10:42:33.995762+0000","last_active":"2024-04-17T10:42:35.987878+0000","last_peered":"2024-04-17T10:42:35.987878+0000","last_clean":"2024-04-17T10:42:35.987878+0000","last_became_active":"2024-04-17T10:42:33.995562+0000","last_became_peered":"2024-04-17T10:42:33.995562+0000","last_unstale":"2024-04-17T10:42:35.987878+0000","last_undegraded":"2024-04-17T10:42:35.987878+0000","last_fullsized":"2024-04-17T10:42:35.987878+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:20:46.478088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.340201+0000","last_change":"2024-04-17T10:42:35.022270+0000","last_active":"2024-04-17T10:42:36.340201+0000","last_peered":"2024-04-17T10:42:36.340201+0000","last_clean":"2024-04-17T10:42:36.340201+0000","last_became_active":"2024-04-17T10:42:35.021592+0000","last_became_peered":"2024-04-17T10:42:35.021592+0000","last_unstale":"2024-04-17T10:42:36.340201+0000","last_undegraded":"2024-04-17T10:42:36.340201+0000","last_fullsized":"2024-04-17T10:42:36.340201+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:06:20.461158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.412678+0000","last_change":"2024-04-17T10:42:34.017202+0000","last_active":"2024-04-17T10:42:36.412678+0000","last_peered":"2024-04-17T10:42:36.412678+0000","last_clean":"2024-04-17T10:42:36.412678+0000","last_became_active":"2024-04-17T10:42:34.016613+0000","last_became_peered":"2024-04-17T10:42:34.016613+0000","last_unstale":"2024-04-17T10:42:36.412678+0000","last_undegraded":"2024-04-17T10:42:36.412678+0000","last_fullsized":"2024-04-17T10:42:36.412678+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:32:38.583434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.992946+0000","last_change":"2024-04-17T10:42:35.010024+0000","last_active":"2024-04-17T10:42:35.992946+0000","last_peered":"2024-04-17T10:42:35.992946+0000","last_clean":"2024-04-17T10:42:35.992946+0000","last_became_active":"2024-04-17T10:42:35.008998+0000","last_became_peered":"2024-04-17T10:42:35.008998+0000","last_unstale":"2024-04-17T10:42:35.992946+0000","last_undegraded":"2024-04-17T10:42:35.992946+0000","last_fullsized":"2024-04-17T10:42:35.992946+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:00:02.307316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.988087+0000","last_change":"2024-04-17T10:42:34.002003+0000","last_active":"2024-04-17T10:42:35.988087+0000","last_peered":"2024-04-17T10:42:35.988087+0000","last_clean":"2024-04-17T10:42:35.988087+0000","last_became_active":"2024-04-17T10:42:34.001413+0000","last_became_peered":"2024-04-17T10:42:34.001413+0000","last_unstale":"2024-04-17T10:42:35.988087+0000","last_undegraded":"2024-04-17T10:42:35.988087+0000","last_fullsized":"2024-04-17T10:42:35.988087+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:10:36.224186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.412285+0000","last_change":"2024-04-17T10:42:35.026741+0000","last_active":"2024-04-17T10:42:36.412285+0000","last_peered":"2024-04-17T10:42:36.412285+0000","last_clean":"2024-04-17T10:42:36.412285+0000","last_became_active":"2024-04-17T10:42:35.023488+0000","last_became_peered":"2024-04-17T10:42:35.023488+0000","last_unstale":"2024-04-17T10:42:36.412285+0000","last_undegraded":"2024-04-17T10:42:36.412285+0000","last_fullsized":"2024-04-17T10:42:36.412285+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:40:05.316943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.340176+0000","last_change":"2024-04-17T10:42:34.010407+0000","last_active":"2024-04-17T10:42:36.340176+0000","last_peered":"2024-04-17T10:42:36.340176+0000","last_clean":"2024-04-17T10:42:36.340176+0000","last_became_active":"2024-04-17T10:42:34.009954+0000","last_became_peered":"2024-04-17T10:42:34.009954+0000","last_unstale":"2024-04-17T10:42:36.340176+0000","last_undegraded":"2024-04-17T10:42:36.340176+0000","last_fullsized":"2024-04-17T10:42:36.340176+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:40:04.304748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.988476+0000","last_change":"2024-04-17T10:42:35.014822+0000","last_active":"2024-04-17T10:42:35.988476+0000","last_peered":"2024-04-17T10:42:35.988476+0000","last_clean":"2024-04-17T10:42:35.988476+0000","last_became_active":"2024-04-17T10:42:35.014526+0000","last_became_peered":"2024-04-17T10:42:35.014526+0000","last_unstale":"2024-04-17T10:42:35.988476+0000","last_undegraded":"2024-04-17T10:42:35.988476+0000","last_fullsized":"2024-04-17T10:42:35.988476+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:29:43.713428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.411183+0000","last_change":"2024-04-17T10:42:33.991455+0000","last_active":"2024-04-17T10:42:36.411183+0000","last_peered":"2024-04-17T10:42:36.411183+0000","last_clean":"2024-04-17T10:42:36.411183+0000","last_became_active":"2024-04-17T10:42:33.990906+0000","last_became_peered":"2024-04-17T10:42:33.990906+0000","last_unstale":"2024-04-17T10:42:36.411183+0000","last_undegraded":"2024-04-17T10:42:36.411183+0000","last_fullsized":"2024-04-17T10:42:36.411183+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:25:05.304517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.986773+0000","last_change":"2024-04-17T10:42:35.008786+0000","last_active":"2024-04-17T10:42:35.986773+0000","last_peered":"2024-04-17T10:42:35.986773+0000","last_clean":"2024-04-17T10:42:35.986773+0000","last_became_active":"2024-04-17T10:42:35.008219+0000","last_became_peered":"2024-04-17T10:42:35.008219+0000","last_unstale":"2024-04-17T10:42:35.986773+0000","last_undegraded":"2024-04-17T10:42:35.986773+0000","last_fullsized":"2024-04-17T10:42:35.986773+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:57:32.587415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.412013+0000","last_change":"2024-04-17T10:42:34.007999+0000","last_active":"2024-04-17T10:42:36.412013+0000","last_peered":"2024-04-17T10:42:36.412013+0000","last_clean":"2024-04-17T10:42:36.412013+0000","last_became_active":"2024-04-17T10:42:34.007259+0000","last_became_peered":"2024-04-17T10:42:34.007259+0000","last_unstale":"2024-04-17T10:42:36.412013+0000","last_undegraded":"2024-04-17T10:42:36.412013+0000","last_fullsized":"2024-04-17T10:42:36.412013+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:33:41.490545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.412992+0000","last_change":"2024-04-17T10:42:35.033669+0000","last_active":"2024-04-17T10:42:36.412992+0000","last_peered":"2024-04-17T10:42:36.412992+0000","last_clean":"2024-04-17T10:42:36.412992+0000","last_became_active":"2024-04-17T10:42:35.033109+0000","last_became_peered":"2024-04-17T10:42:35.033109+0000","last_unstale":"2024-04-17T10:42:36.412992+0000","last_undegraded":"2024-04-17T10:42:36.412992+0000","last_fullsized":"2024-04-17T10:42:36.412992+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:31:41.284193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.412529+0000","last_change":"2024-04-17T10:42:34.002043+0000","last_active":"2024-04-17T10:42:36.412529+0000","last_peered":"2024-04-17T10:42:36.412529+0000","last_clean":"2024-04-17T10:42:36.412529+0000","last_became_active":"2024-04-17T10:42:34.001536+0000","last_became_peered":"2024-04-17T10:42:34.001536+0000","last_unstale":"2024-04-17T10:42:36.412529+0000","last_undegraded":"2024-04-17T10:42:36.412529+0000","last_fullsized":"2024-04-17T10:42:36.412529+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:29:49.750014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.413193+0000","last_change":"2024-04-17T10:42:35.028984+0000","last_active":"2024-04-17T10:42:36.413193+0000","last_peered":"2024-04-17T10:42:36.413193+0000","last_clean":"2024-04-17T10:42:36.413193+0000","last_became_active":"2024-04-17T10:42:35.026253+0000","last_became_peered":"2024-04-17T10:42:35.026253+0000","last_unstale":"2024-04-17T10:42:36.413193+0000","last_undegraded":"2024-04-17T10:42:36.413193+0000","last_fullsized":"2024-04-17T10:42:36.413193+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:04:05.170981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"59'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.988273+0000","last_change":"2024-04-17T10:42:34.003042+0000","last_active":"2024-04-17T10:42:35.988273+0000","last_peered":"2024-04-17T10:42:35.988273+0000","last_clean":"2024-04-17T10:42:35.988273+0000","last_became_active":"2024-04-17T10:42:34.002726+0000","last_became_peered":"2024-04-17T10:42:34.002726+0000","last_unstale":"2024-04-17T10:42:35.988273+0000","last_undegraded":"2024-04-17T10:42:35.988273+0000","last_fullsized":"2024-04-17T10:42:35.988273+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:09:32.979808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.411822+0000","last_change":"2024-04-17T10:42:35.011542+0000","last_active":"2024-04-17T10:42:36.411822+0000","last_peered":"2024-04-17T10:42:36.411822+0000","last_clean":"2024-04-17T10:42:36.411822+0000","last_became_active":"2024-04-17T10:42:35.011176+0000","last_became_peered":"2024-04-17T10:42:35.011176+0000","last_unstale":"2024-04-17T10:42:36.411822+0000","last_undegraded":"2024-04-17T10:42:36.411822+0000","last_fullsized":"2024-04-17T10:42:36.411822+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:44:31.532198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.007072+0000","last_change":"2024-04-17T10:42:34.012933+0000","last_active":"2024-04-17T10:42:36.007072+0000","last_peered":"2024-04-17T10:42:36.007072+0000","last_clean":"2024-04-17T10:42:36.007072+0000","last_became_active":"2024-04-17T10:42:34.012150+0000","last_became_peered":"2024-04-17T10:42:34.012150+0000","last_unstale":"2024-04-17T10:42:36.007072+0000","last_undegraded":"2024-04-17T10:42:36.007072+0000","last_fullsized":"2024-04-17T10:42:36.007072+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:25:05.304517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.987280+0000","last_change":"2024-04-17T10:42:35.027139+0000","last_active":"2024-04-17T10:42:35.987280+0000","last_peered":"2024-04-17T10:42:35.987280+0000","last_clean":"2024-04-17T10:42:35.987280+0000","last_became_active":"2024-04-17T10:42:35.026861+0000","last_became_peered":"2024-04-17T10:42:35.026861+0000","last_unstale":"2024-04-17T10:42:35.987280+0000","last_undegraded":"2024-04-17T10:42:35.987280+0000","last_fullsized":"2024-04-17T10:42:35.987280+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:30:04.131306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.413175+0000","last_change":"2024-04-17T10:42:34.012790+0000","last_active":"2024-04-17T10:42:36.413175+0000","last_peered":"2024-04-17T10:42:36.413175+0000","last_clean":"2024-04-17T10:42:36.413175+0000","last_became_active":"2024-04-17T10:42:34.012177+0000","last_became_peered":"2024-04-17T10:42:34.012177+0000","last_unstale":"2024-04-17T10:42:36.413175+0000","last_undegraded":"2024-04-17T10:42:36.413175+0000","last_fullsized":"2024-04-17T10:42:36.413175+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:01:33.910102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.339788+0000","last_change":"2024-04-17T10:42:35.026981+0000","last_active":"2024-04-17T10:42:36.339788+0000","last_peered":"2024-04-17T10:42:36.339788+0000","last_clean":"2024-04-17T10:42:36.339788+0000","last_became_active":"2024-04-17T10:42:35.026700+0000","last_became_peered":"2024-04-17T10:42:35.026700+0000","last_unstale":"2024-04-17T10:42:36.339788+0000","last_undegraded":"2024-04-17T10:42:36.339788+0000","last_fullsized":"2024-04-17T10:42:36.339788+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:39:10.426786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.410836+0000","last_change":"2024-04-17T10:42:34.015303+0000","last_active":"2024-04-17T10:42:36.410836+0000","last_peered":"2024-04-17T10:42:36.410836+0000","last_clean":"2024-04-17T10:42:36.410836+0000","last_became_active":"2024-04-17T10:42:34.014919+0000","last_became_peered":"2024-04-17T10:42:34.014919+0000","last_unstale":"2024-04-17T10:42:36.410836+0000","last_undegraded":"2024-04-17T10:42:36.410836+0000","last_fullsized":"2024-04-17T10:42:36.410836+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:41:10.271546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.992332+0000","last_change":"2024-04-17T10:42:35.025934+0000","last_active":"2024-04-17T10:42:35.992332+0000","last_peered":"2024-04-17T10:42:35.992332+0000","last_clean":"2024-04-17T10:42:35.992332+0000","last_became_active":"2024-04-17T10:42:35.025571+0000","last_became_peered":"2024-04-17T10:42:35.025571+0000","last_unstale":"2024-04-17T10:42:35.992332+0000","last_undegraded":"2024-04-17T10:42:35.992332+0000","last_fullsized":"2024-04-17T10:42:35.992332+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:31:41.284193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.989801+0000","last_change":"2024-04-17T10:42:34.009833+0000","last_active":"2024-04-17T10:42:35.989801+0000","last_peered":"2024-04-17T10:42:35.989801+0000","last_clean":"2024-04-17T10:42:35.989801+0000","last_became_active":"2024-04-17T10:42:34.005236+0000","last_became_peered":"2024-04-17T10:42:34.005236+0000","last_unstale":"2024-04-17T10:42:35.989801+0000","last_undegraded":"2024-04-17T10:42:35.989801+0000","last_fullsized":"2024-04-17T10:42:35.989801+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:43:53.941771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.410654+0000","last_change":"2024-04-17T10:42:35.005484+0000","last_active":"2024-04-17T10:42:36.410654+0000","last_peered":"2024-04-17T10:42:36.410654+0000","last_clean":"2024-04-17T10:42:36.410654+0000","last_became_active":"2024-04-17T10:42:35.004816+0000","last_became_peered":"2024-04-17T10:42:35.004816+0000","last_unstale":"2024-04-17T10:42:36.410654+0000","last_undegraded":"2024-04-17T10:42:36.410654+0000","last_fullsized":"2024-04-17T10:42:36.410654+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:42:13.543554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.410792+0000","last_change":"2024-04-17T10:42:33.985466+0000","last_active":"2024-04-17T10:42:36.410792+0000","last_peered":"2024-04-17T10:42:36.410792+0000","last_clean":"2024-04-17T10:42:36.410792+0000","last_became_active":"2024-04-17T10:42:33.984982+0000","last_became_peered":"2024-04-17T10:42:33.984982+0000","last_unstale":"2024-04-17T10:42:36.410792+0000","last_undegraded":"2024-04-17T10:42:36.410792+0000","last_fullsized":"2024-04-17T10:42:36.410792+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:52:19.251915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.339469+0000","last_change":"2024-04-17T10:42:35.017346+0000","last_active":"2024-04-17T10:42:36.339469+0000","last_peered":"2024-04-17T10:42:36.339469+0000","last_clean":"2024-04-17T10:42:36.339469+0000","last_became_active":"2024-04-17T10:42:35.013548+0000","last_became_peered":"2024-04-17T10:42:35.013548+0000","last_unstale":"2024-04-17T10:42:36.339469+0000","last_undegraded":"2024-04-17T10:42:36.339469+0000","last_fullsized":"2024-04-17T10:42:36.339469+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:17:51.676141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.340536+0000","last_change":"2024-04-17T10:42:34.015593+0000","last_active":"2024-04-17T10:42:36.340536+0000","last_peered":"2024-04-17T10:42:36.340536+0000","last_clean":"2024-04-17T10:42:36.340536+0000","last_became_active":"2024-04-17T10:42:34.015222+0000","last_became_peered":"2024-04-17T10:42:34.015222+0000","last_unstale":"2024-04-17T10:42:36.340536+0000","last_undegraded":"2024-04-17T10:42:36.340536+0000","last_fullsized":"2024-04-17T10:42:36.340536+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:30:44.270911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.340410+0000","last_change":"2024-04-17T10:42:35.026305+0000","last_active":"2024-04-17T10:42:36.340410+0000","last_peered":"2024-04-17T10:42:36.340410+0000","last_clean":"2024-04-17T10:42:36.340410+0000","last_became_active":"2024-04-17T10:42:35.025776+0000","last_became_peered":"2024-04-17T10:42:35.025776+0000","last_unstale":"2024-04-17T10:42:36.340410+0000","last_undegraded":"2024-04-17T10:42:36.340410+0000","last_fullsized":"2024-04-17T10:42:36.340410+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:33.961433+0000","last_clean_scrub_stamp":"2024-04-17T10:42:33.961433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:42:22.146232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:36.007036+0000","last_change":"2024-04-17T10:42:34.021417+0000","last_active":"2024-04-17T10:42:36.007036+0000","last_peered":"2024-04-17T10:42:36.007036+0000","last_clean":"2024-04-17T10:42:36.007036+0000","last_became_active":"2024-04-17T10:42:34.020795+0000","last_became_peered":"2024-04-17T10:42:34.020795+0000","last_unstale":"2024-04-17T10:42:36.007036+0000","last_undegraded":"2024-04-17T10:42:36.007036+0000","last_fullsized":"2024-04-17T10:42:36.007036+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:42:32.949238+0000","last_clean_scrub_stamp":"2024-04-17T10:42:32.949238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:26:39.567608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":250,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-17T10:42:35.991083+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:42:35.991083+0000","last_peered":"2024-04-17T10:42:35.991083+0000","last_clean":"2024-04-17T10:42:35.991083+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:42:35.991083+0000","last_undegraded":"2024-04-17T10:42:35.991083+0000","last_fullsized":"2024-04-17T10:42:35.991083+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574908,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44948,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326892,"statfs":{"total":96636764160,"available":96590737408,"internally_reserved":0,"allocated":1310720,"data_stored":888019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574907,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40200,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331640,"statfs":{"total":96636764160,"available":96595599360,"internally_reserved":0,"allocated":708608,"data_stored":297491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574908,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40208,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331632,"statfs":{"total":96636764160,"available":96595591168,"internally_reserved":0,"allocated":716800,"data_stored":298143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574907,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49932,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321908,"statfs":{"total":96636764160,"available":96585633792,"internally_reserved":0,"allocated":720896,"data_stored":299103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574909,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54092,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317748,"statfs":{"total":96636764160,"available":96581373952,"internally_reserved":0,"allocated":716800,"data_stored":297547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36044,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335796,"statfs":{"total":96636764160,"available":96599855104,"internally_reserved":0,"allocated":716800,"data_stored":297547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49928,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321912,"statfs":{"total":96636764160,"available":96585637888,"internally_reserved":0,"allocated":712704,"data_stored":297581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574907,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40788,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331052,"statfs":{"total":96636764160,"available":96594997248,"internally_reserved":0,"allocated":1318912,"data_stored":890053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:42:43.826 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-17T10:42:43.826 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-17T10:42:43.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.993+0000 7fa8b2fee640 1 Processor -- start 2024-04-17T10:42:43.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.993+0000 7fa8b2fee640 1 -- start start 2024-04-17T10:42:43.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.993+0000 7fa8b2fee640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa8ac0ce400 0x7fa8ac0b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:43.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.993+0000 7fa8b2fee640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8ac0b5450 0x7fa8ac0b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:43.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.993+0000 7fa8b2fee640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8ac15e4d0 con 0x7fa8ac15e0f0 2024-04-17T10:42:43.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.993+0000 7fa8b2fee640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8ac162b40 con 0x7fa8ac0ce400 2024-04-17T10:42:43.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.993+0000 7fa8b2fee640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac162320 con 0x7fa8ac0b5450 2024-04-17T10:42:43.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.997+0000 7fa8b158b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8ac0b5450 0x7fa8ac0b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:43.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.997+0000 7fa8b158b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8ac0b5450 0x7fa8ac0b58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:47422/0 (socket says 172.21.15.83:47422) 2024-04-17T10:42:43.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.997+0000 7fa8abfff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa8ac0ce400 0x7fa8ac0b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:43.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.997+0000 7fa8b158b640 1 -- 172.21.15.83:0/2630990278 learned_addr learned my addr 172.21.15.83:0/2630990278 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:43.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.997+0000 7fa8b158b640 1 -- 172.21.15.83:0/2630990278 >> v2:172.21.15.167:3300/0 conn(0x7fa8ac0ce400 msgr2=0x7fa8ac0b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:43.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.997+0000 7fa8b158b640 1 --2- 172.21.15.83:0/2630990278 >> v2:172.21.15.167:3300/0 conn(0x7fa8ac0ce400 0x7fa8ac0b6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:43.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.997+0000 7fa8b158b640 1 -- 172.21.15.83:0/2630990278 >> v1:172.21.15.167:6789/0 conn(0x7fa8ac15e0f0 legacy=0x7fa8ac161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:43.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.997+0000 7fa8b158b640 1 -- 172.21.15.83:0/2630990278 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8ac162580 con 0x7fa8ac0b5450 2024-04-17T10:42:43.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.997+0000 7fa8b158b640 1 --2- 172.21.15.83:0/2630990278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8ac0b5450 0x7fa8ac0b58e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4009e50 tx=0x7fa8a401c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=975d27b86c4558e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:43.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.997+0000 7fa8ab7fe640 1 -- 172.21.15.83:0/2630990278 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a40a5660 con 0x7fa8ac0b5450 2024-04-17T10:42:43.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.997+0000 7fa8ab7fe640 1 -- 172.21.15.83:0/2630990278 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8a40a5800 con 0x7fa8ac0b5450 2024-04-17T10:42:43.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:43.997+0000 7fa8ab7fe640 1 -- 172.21.15.83:0/2630990278 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a40a5bd0 con 0x7fa8ac0b5450 2024-04-17T10:42:43.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.001+0000 7fa8b2fee640 1 -- 172.21.15.83:0/2630990278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8ac0b5450 msgr2=0x7fa8ac0b58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:44.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.001+0000 7fa8b2fee640 1 --2- 172.21.15.83:0/2630990278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8ac0b5450 0x7fa8ac0b58e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4009e50 tx=0x7fa8a401c9a0 comp rx=0 tx=0).stop 2024-04-17T10:42:44.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.001+0000 7fa8b2fee640 1 -- 172.21.15.83:0/2630990278 shutdown_connections 2024-04-17T10:42:44.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.001+0000 7fa8b2fee640 1 --2- 172.21.15.83:0/2630990278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8ac0b5450 0x7fa8ac0b58e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:44.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.001+0000 7fa8b2fee640 1 --2- 172.21.15.83:0/2630990278 >> v2:172.21.15.167:3300/0 conn(0x7fa8ac0ce400 0x7fa8ac0b6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:44.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.001+0000 7fa8b2fee640 1 -- 172.21.15.83:0/2630990278 >> 172.21.15.83:0/2630990278 conn(0x7fa8ac0d0450 msgr2=0x7fa8ac0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:44.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.001+0000 7fa8b2fee640 1 -- 172.21.15.83:0/2630990278 shutdown_connections 2024-04-17T10:42:44.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.001+0000 7fa8b2fee640 1 -- 172.21.15.83:0/2630990278 wait complete. 2024-04-17T10:42:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.001+0000 7fa8b2fee640 1 Processor -- start 2024-04-17T10:42:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.001+0000 7fa8b2fee640 1 -- start start 2024-04-17T10:42:44.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.001+0000 7fa8b2fee640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8ac0b5450 0x7fa8ac0c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:44.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.001+0000 7fa8b2fee640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8ac0ce400 0x7fa8ac0c9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:44.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.001+0000 7fa8b2fee640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8ac15e0f0 0x7fa8ac15cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:44.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.001+0000 7fa8b2fee640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac0d2ed0 con 0x7fa8ac15e0f0 2024-04-17T10:42:44.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.001+0000 7fa8b2fee640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac0d2d50 con 0x7fa8ac0ce400 2024-04-17T10:42:44.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.001+0000 7fa8b2fee640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac0d3050 con 0x7fa8ac0b5450 2024-04-17T10:42:44.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.001+0000 7fa8b158b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8ac15e0f0 0x7fa8ac15cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:44.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.001+0000 7fa8b158b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8ac15e0f0 0x7fa8ac15cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36888/0 (socket says 172.21.15.83:36888) 2024-04-17T10:42:44.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.001+0000 7fa8b158b640 1 -- 172.21.15.83:0/3078147395 learned_addr learned my addr 172.21.15.83:0/3078147395 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:44.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.001+0000 7fa8b0d8a640 1 --2- 172.21.15.83:0/3078147395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8ac0b5450 0x7fa8ac0c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:44.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.001+0000 7fa8abfff640 1 --2- 172.21.15.83:0/3078147395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8ac0ce400 0x7fa8ac0c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:44.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.001+0000 7fa8b158b640 1 -- 172.21.15.83:0/3078147395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8ac0b5450 msgr2=0x7fa8ac0c7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:44.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.001+0000 7fa8b158b640 1 --2- 172.21.15.83:0/3078147395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8ac0b5450 0x7fa8ac0c7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:44.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.001+0000 7fa8b158b640 1 -- 172.21.15.83:0/3078147395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8ac0ce400 msgr2=0x7fa8ac0c9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:44.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.001+0000 7fa8b158b640 1 --2- 172.21.15.83:0/3078147395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8ac0ce400 0x7fa8ac0c9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:44.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.001+0000 7fa8b158b640 1 -- 172.21.15.83:0/3078147395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8ac0c85d0 con 0x7fa8ac15e0f0 2024-04-17T10:42:44.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.001+0000 7fa8b158b640 1 --2- 172.21.15.83:0/3078147395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8ac15e0f0 0x7fa8ac15cf20 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fa8a401a5d0 tx=0x7fa8a4019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:44.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.001+0000 7fa8a97fa640 1 -- 172.21.15.83:0/3078147395 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4009510 con 0x7fa8ac15e0f0 2024-04-17T10:42:44.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.001+0000 7fa8a97fa640 1 -- 172.21.15.83:0/3078147395 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8a40b8470 con 0x7fa8ac15e0f0 2024-04-17T10:42:44.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.001+0000 7fa8b2fee640 1 -- 172.21.15.83:0/3078147395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8ac0c88b0 con 0x7fa8ac15e0f0 2024-04-17T10:42:44.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.001+0000 7fa8a97fa640 1 -- 172.21.15.83:0/3078147395 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a40b8880 con 0x7fa8ac15e0f0 2024-04-17T10:42:44.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.001+0000 7fa8b2fee640 1 -- 172.21.15.83:0/3078147395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8ac0c8e70 con 0x7fa8ac15e0f0 2024-04-17T10:42:44.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.005+0000 7fa8b2fee640 1 -- 172.21.15.83:0/3078147395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8ac0681a0 con 0x7fa8ac15e0f0 2024-04-17T10:42:44.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.005+0000 7fa8a97fa640 1 -- 172.21.15.83:0/3078147395 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa8a40c0020 con 0x7fa8ac15e0f0 2024-04-17T10:42:44.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.005+0000 7fa8a97fa640 1 --2- 172.21.15.83:0/3078147395 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa890066f60 0x7fa890069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:44.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.005+0000 7fa8a97fa640 1 -- 172.21.15.83:0/3078147395 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa8a411fa30 con 0x7fa8ac15e0f0 2024-04-17T10:42:44.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.005+0000 7fa8b0d8a640 1 --2- 172.21.15.83:0/3078147395 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa890066f60 0x7fa890069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:44.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.005+0000 7fa8b0d8a640 1 --2- 172.21.15.83:0/3078147395 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa890066f60 0x7fa890069420 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00015b0 tx=0x7fa8a0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:44.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.009+0000 7fa8a97fa640 1 -- 172.21.15.83:0/3078147395 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa8a40c7020 con 0x7fa8ac15e0f0 2024-04-17T10:42:44.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.217+0000 7fa8b2fee640 1 -- 172.21.15.83:0/3078147395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa8ac074f90 con 0x7fa8ac15e0f0 2024-04-17T10:42:44.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.221+0000 7fa8a97fa640 1 -- 172.21.15.83:0/3078147395 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7fa8a4016620 con 0x7fa8ac15e0f0 2024-04-17T10:42:44.222 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-04-17T10:42:44.223 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBUph9mHGmXGBAAZF+JiAG9YrFtbW+dCy2IEQ== 2024-04-17T10:42:44.223 INFO:teuthology.orchestra.run.smithi083.stdout: caps mds = "allow" 2024-04-17T10:42:44.223 INFO:teuthology.orchestra.run.smithi083.stdout: caps mon = "allow r" 2024-04-17T10:42:44.223 INFO:teuthology.orchestra.run.smithi083.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-17T10:42:44.224 INFO:teuthology.orchestra.run.smithi083.stderr:updated caps for client.0 2024-04-17T10:42:44.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.221+0000 7fa8b2fee640 1 -- 172.21.15.83:0/3078147395 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa890066f60 msgr2=0x7fa890069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:44.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.221+0000 7fa8b2fee640 1 --2- 172.21.15.83:0/3078147395 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa890066f60 0x7fa890069420 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00015b0 tx=0x7fa8a0009040 comp rx=0 tx=0).stop 2024-04-17T10:42:44.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.221+0000 7fa8b2fee640 1 -- 172.21.15.83:0/3078147395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8ac15e0f0 msgr2=0x7fa8ac15cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:44.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.221+0000 7fa8b2fee640 1 --2- 172.21.15.83:0/3078147395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8ac15e0f0 0x7fa8ac15cf20 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fa8a401a5d0 tx=0x7fa8a4019740 comp rx=0 tx=0).stop 2024-04-17T10:42:44.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.225+0000 7fa8b2fee640 1 -- 172.21.15.83:0/3078147395 shutdown_connections 2024-04-17T10:42:44.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.225+0000 7fa8b2fee640 1 --2- 172.21.15.83:0/3078147395 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa890066f60 0x7fa890069420 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:44.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.225+0000 7fa8b2fee640 1 --2- 172.21.15.83:0/3078147395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8ac15e0f0 0x7fa8ac15cf20 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:44.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.225+0000 7fa8b2fee640 1 --2- 172.21.15.83:0/3078147395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8ac0ce400 0x7fa8ac0c9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:44.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.225+0000 7fa8b2fee640 1 --2- 172.21.15.83:0/3078147395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8ac0b5450 0x7fa8ac0c7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:44.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.225+0000 7fa8b2fee640 1 -- 172.21.15.83:0/3078147395 >> 172.21.15.83:0/3078147395 conn(0x7fa8ac0d0450 msgr2=0x7fa8ac0c9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:44.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.225+0000 7fa8b2fee640 1 -- 172.21.15.83:0/3078147395 shutdown_connections 2024-04-17T10:42:44.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.225+0000 7fa8b2fee640 1 -- 172.21.15.83:0/3078147395 wait complete. 2024-04-17T10:42:44.239 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:42:44.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.357+0000 7f027b6a8640 1 Processor -- start 2024-04-17T10:42:44.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.357+0000 7f027b6a8640 1 -- start start 2024-04-17T10:42:44.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.361+0000 7f027b6a8640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0274072330 0x7f02740cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:44.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.361+0000 7f027b6a8640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02740cfea0 con 0x7f02740cfa10 2024-04-17T10:42:44.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.361+0000 7f027b6a8640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f027406e320 con 0x7f027415ba30 2024-04-17T10:42:44.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.361+0000 7f027b6a8640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0274071ea0 con 0x7f0274072330 2024-04-17T10:42:44.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.361+0000 7f0279c45640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f02740cfa10 0x7f027406c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57480/0 (socket says 172.21.15.83:57480) 2024-04-17T10:42:44.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.361+0000 7f0279c45640 1 -- 172.21.15.83:0/2942852152 learned_addr learned my addr 172.21.15.83:0/2942852152 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:44.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.361+0000 7f0278c43640 1 --2- 172.21.15.83:0/2942852152 >> v2:172.21.15.167:3300/0 conn(0x7f0274072330 0x7f02740cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:44.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.361+0000 7f026bfff640 1 -- 172.21.15.83:0/2942852152 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 331928171 0 0) 0x7f02740cfea0 con 0x7f02740cfa10 2024-04-17T10:42:44.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.361+0000 7f026bfff640 1 -- 172.21.15.83:0/2942852152 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0260003530 con 0x7f02740cfa10 2024-04-17T10:42:44.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.361+0000 7f0278c43640 1 -- 172.21.15.83:0/2942852152 >> v1:172.21.15.167:6789/0 conn(0x7f027415ba30 legacy=0x7f027415c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:44.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.361+0000 7f0278c43640 1 -- 172.21.15.83:0/2942852152 >> v1:172.21.15.83:6789/0 conn(0x7f02740cfa10 legacy=0x7f027406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:44.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.361+0000 7f0278c43640 1 -- 172.21.15.83:0/2942852152 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f027406e040 con 0x7f0274072330 2024-04-17T10:42:44.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f0278c43640 1 --2- 172.21.15.83:0/2942852152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0274072330 0x7f02740cf4d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f02640097c0 tx=0x7f026401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6509437572345059 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:44.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f026bfff640 1 -- 172.21.15.83:0/2942852152 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02640b0070 con 0x7f0274072330 2024-04-17T10:42:44.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f026bfff640 1 -- 172.21.15.83:0/2942852152 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02640ab600 con 0x7f0274072330 2024-04-17T10:42:44.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f026bfff640 1 -- 172.21.15.83:0/2942852152 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02640ab9b0 con 0x7f0274072330 2024-04-17T10:42:44.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f027b6a8640 1 -- 172.21.15.83:0/2942852152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0274072330 msgr2=0x7f02740cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:44.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f027b6a8640 1 --2- 172.21.15.83:0/2942852152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0274072330 0x7f02740cf4d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f02640097c0 tx=0x7f026401c410 comp rx=0 tx=0).stop 2024-04-17T10:42:44.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f027b6a8640 1 -- 172.21.15.83:0/2942852152 shutdown_connections 2024-04-17T10:42:44.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f027b6a8640 1 --2- 172.21.15.83:0/2942852152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0274072330 0x7f02740cf4d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:44.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f027b6a8640 1 -- 172.21.15.83:0/2942852152 >> 172.21.15.83:0/2942852152 conn(0x7f027406abe0 msgr2=0x7f027406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:44.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f027b6a8640 1 -- 172.21.15.83:0/2942852152 shutdown_connections 2024-04-17T10:42:44.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f027b6a8640 1 -- 172.21.15.83:0/2942852152 wait complete. 2024-04-17T10:42:44.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f027b6a8640 1 Processor -- start 2024-04-17T10:42:44.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f027b6a8640 1 -- start start 2024-04-17T10:42:44.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f027b6a8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0274072330 0x7f0274154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:44.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f027b6a8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02740cfa10 0x7f0274147170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:44.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f027b6a8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f027415ba30 0x7f027415d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:44.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f027b6a8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027406de20 con 0x7f0274072330 2024-04-17T10:42:44.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f027b6a8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027406dca0 con 0x7f02740cfa10 2024-04-17T10:42:44.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f027b6a8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0274169530 con 0x7f027415ba30 2024-04-17T10:42:44.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f0279444640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0274072330 0x7f0274154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:44.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f0279444640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0274072330 0x7f0274154a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36900/0 (socket says 172.21.15.83:36900) 2024-04-17T10:42:44.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f0279444640 1 -- 172.21.15.83:0/2152698066 learned_addr learned my addr 172.21.15.83:0/2152698066 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:44.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f0279c45640 1 --2- 172.21.15.83:0/2152698066 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f027415ba30 0x7f027415d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:44.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f0279444640 1 -- 172.21.15.83:0/2152698066 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f027415ba30 msgr2=0x7f027415d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:44.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f0279444640 1 --2- 172.21.15.83:0/2152698066 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f027415ba30 0x7f027415d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:44.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f0279444640 1 -- 172.21.15.83:0/2152698066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02740cfa10 msgr2=0x7f0274147170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:44.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f0278c43640 1 --2- 172.21.15.83:0/2152698066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02740cfa10 0x7f0274147170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:44.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f0279444640 1 --2- 172.21.15.83:0/2152698066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02740cfa10 0x7f0274147170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:44.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f0279444640 1 -- 172.21.15.83:0/2152698066 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0274154fb0 con 0x7f0274072330 2024-04-17T10:42:44.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f0279444640 1 --2- 172.21.15.83:0/2152698066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0274072330 0x7f0274154a70 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f027000c3f0 tx=0x7f027000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:44.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f0269ffb640 1 -- 172.21.15.83:0/2152698066 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0270017070 con 0x7f0274072330 2024-04-17T10:42:44.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f0269ffb640 1 -- 172.21.15.83:0/2152698066 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02700052d0 con 0x7f0274072330 2024-04-17T10:42:44.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f027b6a8640 1 -- 172.21.15.83:0/2152698066 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0274152840 con 0x7f0274072330 2024-04-17T10:42:44.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f0269ffb640 1 -- 172.21.15.83:0/2152698066 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0270003840 con 0x7f0274072330 2024-04-17T10:42:44.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.365+0000 7f027b6a8640 1 -- 172.21.15.83:0/2152698066 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0274152e00 con 0x7f0274072330 2024-04-17T10:42:44.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.369+0000 7f027b6a8640 1 -- 172.21.15.83:0/2152698066 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0274074f90 con 0x7f0274072330 2024-04-17T10:42:44.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.369+0000 7f0269ffb640 1 -- 172.21.15.83:0/2152698066 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f027001a070 con 0x7f0274072330 2024-04-17T10:42:44.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.369+0000 7f0269ffb640 1 --2- 172.21.15.83:0/2152698066 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0258066f90 0x7f0258069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:44.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.369+0000 7f0269ffb640 1 -- 172.21.15.83:0/2152698066 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0270084300 con 0x7f0274072330 2024-04-17T10:42:44.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.369+0000 7f0278c43640 1 --2- 172.21.15.83:0/2152698066 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0258066f90 0x7f0258069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:44.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.369+0000 7f0278c43640 1 --2- 172.21.15.83:0/2152698066 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0258066f90 0x7f0258069450 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f02640ad000 tx=0x7f0264004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:44.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.373+0000 7f0269ffb640 1 -- 172.21.15.83:0/2152698066 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0274154fb0 con 0x7f0274072330 2024-04-17T10:42:44.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.553+0000 7f027b6a8640 1 -- 172.21.15.83:0/2152698066 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f027406c980 con 0x7f0274072330 2024-04-17T10:42:44.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.553+0000 7f0269ffb640 1 -- 172.21.15.83:0/2152698066 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f0270005480 con 0x7f0274072330 2024-04-17T10:42:44.554 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:44.554 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5398,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1991041077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1991041077},{"type":"v1","addr":"172.21.15.167:6835","nonce":1991041077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6522,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/320723747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":320723747},{"type":"v1","addr":"172.21.15.83:6836","nonce":320723747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6525,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/4066270849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":4066270849},{"type":"v1","addr":"172.21.15.83:6837","nonce":4066270849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:42:34.328397+0000","modified":"2024-04-17T10:42:37.455022+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6534},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6534":{"gid":6534,"name":"b","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6833/271994303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":271994303},{"type":"v1","addr":"172.21.15.167:6833","nonce":271994303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-17T10:42:44.554 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 32 2024-04-17T10:42:44.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.557+0000 7f027b6a8640 1 -- 172.21.15.83:0/2152698066 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0258066f90 msgr2=0x7f0258069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:44.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.557+0000 7f027b6a8640 1 --2- 172.21.15.83:0/2152698066 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0258066f90 0x7f0258069450 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f02640ad000 tx=0x7f0264004740 comp rx=0 tx=0).stop 2024-04-17T10:42:44.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.557+0000 7f027b6a8640 1 -- 172.21.15.83:0/2152698066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0274072330 msgr2=0x7f0274154a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:44.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.557+0000 7f027b6a8640 1 --2- 172.21.15.83:0/2152698066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0274072330 0x7f0274154a70 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f027000c3f0 tx=0x7f027000c8b0 comp rx=0 tx=0).stop 2024-04-17T10:42:44.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.557+0000 7f027b6a8640 1 -- 172.21.15.83:0/2152698066 shutdown_connections 2024-04-17T10:42:44.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.557+0000 7f027b6a8640 1 --2- 172.21.15.83:0/2152698066 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0258066f90 0x7f0258069450 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:44.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.557+0000 7f027b6a8640 1 --2- 172.21.15.83:0/2152698066 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f027415ba30 0x7f027415d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:44.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.557+0000 7f027b6a8640 1 --2- 172.21.15.83:0/2152698066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02740cfa10 0x7f0274147170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:44.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.557+0000 7f027b6a8640 1 --2- 172.21.15.83:0/2152698066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0274072330 0x7f0274154a70 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:44.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.557+0000 7f027b6a8640 1 -- 172.21.15.83:0/2152698066 >> 172.21.15.83:0/2152698066 conn(0x7f027406abe0 msgr2=0x7f0274159da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:44.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.557+0000 7f027b6a8640 1 -- 172.21.15.83:0/2152698066 shutdown_connections 2024-04-17T10:42:44.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:44.557+0000 7f027b6a8640 1 -- 172.21.15.83:0/2152698066 wait complete. 2024-04-17T10:42:44.974 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T10:42:34.328397+0000', 'modified': '2024-04-17T10:42:37.455022+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6534}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6534': {'gid': 6534, 'name': 'b', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.167:6833/271994303', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6832', 'nonce': 271994303}, {'type': 'v1', 'addr': '172.21.15.167:6833', 'nonce': 271994303}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:42:44.975 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T10:42:44.975 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-17T10:42:45.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.109+0000 7f171e7e5640 1 Processor -- start 2024-04-17T10:42:45.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.109+0000 7f171e7e5640 1 -- start start 2024-04-17T10:42:45.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.109+0000 7f171e7e5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f171815ed70 0x7f171806bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:45.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.109+0000 7f171e7e5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1718161d20 0x7f1718162190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:45.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.109+0000 7f171e7e5640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17180d2430 con 0x7f1718161940 2024-04-17T10:42:45.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.109+0000 7f171e7e5640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17180d2710 con 0x7f1718161d20 2024-04-17T10:42:45.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.109+0000 7f171e7e5640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17180ced00 con 0x7f171815ed70 2024-04-17T10:42:45.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.109+0000 7f171cd82640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1718161d20 0x7f1718162190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:45.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.109+0000 7f171cd82640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1718161d20 0x7f1718162190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36910/0 (socket says 172.21.15.83:36910) 2024-04-17T10:42:45.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.109+0000 7f171cd82640 1 -- 172.21.15.83:0/3428522531 learned_addr learned my addr 172.21.15.83:0/3428522531 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:45.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.109+0000 7f17177fe640 1 --2- 172.21.15.83:0/3428522531 >> v2:172.21.15.167:3300/0 conn(0x7f171815ed70 0x7f171806bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:45.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.109+0000 7f171cd82640 1 -- 172.21.15.83:0/3428522531 >> v2:172.21.15.167:3300/0 conn(0x7f171815ed70 msgr2=0x7f171806bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:45.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.109+0000 7f171cd82640 1 --2- 172.21.15.83:0/3428522531 >> v2:172.21.15.167:3300/0 conn(0x7f171815ed70 0x7f171806bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.109+0000 7f171cd82640 1 -- 172.21.15.83:0/3428522531 >> v1:172.21.15.167:6789/0 conn(0x7f1718161940 legacy=0x7f17180cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:45.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.109+0000 7f171cd82640 1 -- 172.21.15.83:0/3428522531 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17180cefe0 con 0x7f1718161d20 2024-04-17T10:42:45.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.113+0000 7f171cd82640 1 --2- 172.21.15.83:0/3428522531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1718161d20 0x7f1718162190 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f17180cbb70 tx=0x7f171001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0843854af0f4ec5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:45.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.113+0000 7f1716ffd640 1 -- 172.21.15.83:0/3428522531 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17100b0070 con 0x7f1718161d20 2024-04-17T10:42:45.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.113+0000 7f1716ffd640 1 -- 172.21.15.83:0/3428522531 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17100ab600 con 0x7f1718161d20 2024-04-17T10:42:45.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.113+0000 7f1716ffd640 1 -- 172.21.15.83:0/3428522531 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17100ab9d0 con 0x7f1718161d20 2024-04-17T10:42:45.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.113+0000 7f171e7e5640 1 -- 172.21.15.83:0/3428522531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1718161d20 msgr2=0x7f1718162190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:45.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.113+0000 7f171e7e5640 1 --2- 172.21.15.83:0/3428522531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1718161d20 0x7f1718162190 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f17180cbb70 tx=0x7f171001c490 comp rx=0 tx=0).stop 2024-04-17T10:42:45.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.113+0000 7f171e7e5640 1 -- 172.21.15.83:0/3428522531 shutdown_connections 2024-04-17T10:42:45.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.113+0000 7f171e7e5640 1 --2- 172.21.15.83:0/3428522531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1718161d20 0x7f1718162190 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.113+0000 7f171e7e5640 1 --2- 172.21.15.83:0/3428522531 >> v2:172.21.15.167:3300/0 conn(0x7f171815ed70 0x7f171806bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.113+0000 7f171e7e5640 1 -- 172.21.15.83:0/3428522531 >> 172.21.15.83:0/3428522531 conn(0x7f17180d17e0 msgr2=0x7f17180d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:45.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.113+0000 7f171e7e5640 1 -- 172.21.15.83:0/3428522531 shutdown_connections 2024-04-17T10:42:45.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.113+0000 7f171e7e5640 1 -- 172.21.15.83:0/3428522531 wait complete. 2024-04-17T10:42:45.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.113+0000 7f171e7e5640 1 Processor -- start 2024-04-17T10:42:45.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.113+0000 7f171e7e5640 1 -- start start 2024-04-17T10:42:45.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.117+0000 7f171e7e5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f171815ed70 0x7f1718154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:45.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.117+0000 7f171e7e5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1718161940 0x7f17180b1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:45.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.117+0000 7f171e7e5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1718161d20 0x7f17180b6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:45.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.117+0000 7f171e7e5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171806d170 con 0x7f171815ed70 2024-04-17T10:42:45.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.117+0000 7f171e7e5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171806cff0 con 0x7f1718161d20 2024-04-17T10:42:45.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.117+0000 7f171e7e5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f171816aa20 con 0x7f1718161940 2024-04-17T10:42:45.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.117+0000 7f1717fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f171815ed70 0x7f1718154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:45.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.117+0000 7f1717fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f171815ed70 0x7f1718154a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36916/0 (socket says 172.21.15.83:36916) 2024-04-17T10:42:45.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.117+0000 7f1717fff640 1 -- 172.21.15.83:0/907032307 learned_addr learned my addr 172.21.15.83:0/907032307 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.117+0000 7f17177fe640 1 --2- 172.21.15.83:0/907032307 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1718161940 0x7f17180b1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.117+0000 7f171cd82640 1 --2- 172.21.15.83:0/907032307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1718161d20 0x7f17180b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.117+0000 7f1717fff640 1 -- 172.21.15.83:0/907032307 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1718161940 msgr2=0x7f17180b1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.117+0000 7f1717fff640 1 --2- 172.21.15.83:0/907032307 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1718161940 0x7f17180b1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.117+0000 7f1717fff640 1 -- 172.21.15.83:0/907032307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1718161d20 msgr2=0x7f17180b6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.117+0000 7f1717fff640 1 --2- 172.21.15.83:0/907032307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1718161d20 0x7f17180b6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.117+0000 7f1717fff640 1 -- 172.21.15.83:0/907032307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17180b3790 con 0x7f171815ed70 2024-04-17T10:42:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.117+0000 7f1717fff640 1 --2- 172.21.15.83:0/907032307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f171815ed70 0x7f1718154a00 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f170800c420 tx=0x7f170800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.117+0000 7f1714ff9640 1 -- 172.21.15.83:0/907032307 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17080052d0 con 0x7f171815ed70 2024-04-17T10:42:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.117+0000 7f1714ff9640 1 -- 172.21.15.83:0/907032307 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1708005470 con 0x7f171815ed70 2024-04-17T10:42:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.117+0000 7f171e7e5640 1 -- 172.21.15.83:0/907032307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17180b3a30 con 0x7f171815ed70 2024-04-17T10:42:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.117+0000 7f1714ff9640 1 -- 172.21.15.83:0/907032307 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1708007260 con 0x7f171815ed70 2024-04-17T10:42:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.117+0000 7f171e7e5640 1 -- 172.21.15.83:0/907032307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17180b3ff0 con 0x7f171815ed70 2024-04-17T10:42:45.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.117+0000 7f1714ff9640 1 -- 172.21.15.83:0/907032307 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1708007400 con 0x7f171815ed70 2024-04-17T10:42:45.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.117+0000 7f1714ff9640 1 --2- 172.21.15.83:0/907032307 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f16f8066fb0 0x7f16f8069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:45.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.117+0000 7f17177fe640 1 --2- 172.21.15.83:0/907032307 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f16f8066fb0 0x7f16f8069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:45.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.117+0000 7f1714ff9640 1 -- 172.21.15.83:0/907032307 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f17080844d0 con 0x7f171815ed70 2024-04-17T10:42:45.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.117+0000 7f17177fe640 1 --2- 172.21.15.83:0/907032307 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f16f8066fb0 0x7f16f8069470 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f17180b0900 tx=0x7f170000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:45.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1721952000 1 Processor -- start 2024-04-17T10:42:45.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1721952000 1 -- start start 2024-04-17T10:42:45.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1721952000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556360b4b920 0x556360b4bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:45.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1721952000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556360b4c240 0x556360b019f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:45.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1721952000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556360b01f30 0x556360b02310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:45.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1721952000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563609e95f0 con 0x556360b4b920 2024-04-17T10:42:45.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1721952000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556360950da0 con 0x556360b4c240 2024-04-17T10:42:45.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1721952000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5563608d9150 con 0x556360b01f30 2024-04-17T10:42:45.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1717fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556360b4b920 0x556360b4bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:45.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1717fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556360b4b920 0x556360b4bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36932/0 (socket says 172.21.15.83:36932) 2024-04-17T10:42:45.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1717fff640 1 -- 172.21.15.83:0/3131268935 learned_addr learned my addr 172.21.15.83:0/3131268935 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:45.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f17177fe640 1 --2- 172.21.15.83:0/3131268935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556360b4c240 0x556360b019f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:45.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f171cd82640 1 --2- 172.21.15.83:0/3131268935 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556360b01f30 0x556360b02310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:45.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1717fff640 1 -- 172.21.15.83:0/3131268935 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556360b01f30 msgr2=0x556360b02310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:45.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1717fff640 1 --2- 172.21.15.83:0/3131268935 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556360b01f30 0x556360b02310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1717fff640 1 -- 172.21.15.83:0/3131268935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556360b4c240 msgr2=0x556360b019f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:45.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1717fff640 1 --2- 172.21.15.83:0/3131268935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556360b4c240 0x556360b019f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1717fff640 1 -- 172.21.15.83:0/3131268935 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563608f2ca0 con 0x556360b4b920 2024-04-17T10:42:45.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f17177fe640 1 --2- 172.21.15.83:0/3131268935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556360b4c240 0x556360b019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:45.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1717fff640 1 --2- 172.21.15.83:0/3131268935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556360b4b920 0x556360b4bd00 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f1708054de0 tx=0x7f170800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f59b48bf7544377 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:45.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f170dffb640 1 -- 172.21.15.83:0/3131268935 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1708053150 con 0x556360b4b920 2024-04-17T10:42:45.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f170dffb640 1 -- 172.21.15.83:0/3131268935 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1708057070 con 0x556360b4b920 2024-04-17T10:42:45.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f170dffb640 1 -- 172.21.15.83:0/3131268935 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170805b0d0 con 0x556360b4b920 2024-04-17T10:42:45.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1721952000 1 -- 172.21.15.83:0/3131268935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556360b4b920 msgr2=0x556360b4bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:45.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1721952000 1 --2- 172.21.15.83:0/3131268935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556360b4b920 0x556360b4bd00 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f1708054de0 tx=0x7f170800cf90 comp rx=0 tx=0).stop 2024-04-17T10:42:45.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1721952000 1 -- 172.21.15.83:0/3131268935 shutdown_connections 2024-04-17T10:42:45.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1721952000 1 --2- 172.21.15.83:0/3131268935 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556360b01f30 0x556360b02310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1721952000 1 --2- 172.21.15.83:0/3131268935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556360b4c240 0x556360b019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1721952000 1 --2- 172.21.15.83:0/3131268935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556360b4b920 0x556360b4bd00 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1721952000 1 -- 172.21.15.83:0/3131268935 >> 172.21.15.83:0/3131268935 conn(0x556360b082a0 msgr2=0x556360b002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:45.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1721952000 1 -- 172.21.15.83:0/3131268935 shutdown_connections 2024-04-17T10:42:45.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1721952000 1 -- 172.21.15.83:0/3131268935 wait complete. 2024-04-17T10:42:45.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1721952000 1 -- start start 2024-04-17T10:42:45.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1721952000 1 Processor -- start 2024-04-17T10:42:45.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1721952000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556360b01f30 0x556360b52c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:45.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1721952000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556360b4b920 0x556360b531c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:45.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1721952000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556360b4c240 0x556360b05aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:45.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1721952000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636090b990 con 0x556360b01f30 2024-04-17T10:42:45.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1721952000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636094ce40 con 0x556360b4b920 2024-04-17T10:42:45.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1721952000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5563608dc900 con 0x556360b4c240 2024-04-17T10:42:45.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.121+0000 7f1721952000 10 client.0 _ll_register_callbacks cb 0x556360a041c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:45.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f1717fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556360b01f30 0x556360b52c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:45.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f1717fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556360b01f30 0x556360b52c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36934/0 (socket says 172.21.15.83:36934) 2024-04-17T10:42:45.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f1717fff640 1 -- 172.21.15.83:0/2068942286 learned_addr learned my addr 172.21.15.83:0/2068942286 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:45.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f1717fff640 1 -- 172.21.15.83:0/2068942286 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556360b4c240 msgr2=0x556360b05aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:45.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f1717fff640 1 --2- 172.21.15.83:0/2068942286 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556360b4c240 0x556360b05aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f1717fff640 1 -- 172.21.15.83:0/2068942286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556360b4b920 msgr2=0x556360b531c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:45.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f1717fff640 1 --2- 172.21.15.83:0/2068942286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556360b4b920 0x556360b531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f1717fff640 1 -- 172.21.15.83:0/2068942286 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556360909630 con 0x556360b01f30 2024-04-17T10:42:45.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f1717fff640 1 --2- 172.21.15.83:0/2068942286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556360b01f30 0x556360b52c80 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f1708055700 tx=0x7f1708000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72186b2a79e77710 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:45.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f16f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:45.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f16f6ffd640 1 -- 172.21.15.83:0/2068942286 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17080514c0 con 0x556360b01f30 2024-04-17T10:42:45.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f16f6ffd640 1 -- 172.21.15.83:0/2068942286 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f170805b0d0 con 0x556360b01f30 2024-04-17T10:42:45.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f1721952000 1 -- 172.21.15.83:0/2068942286 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1708057070 con 0x556360b01f30 2024-04-17T10:42:45.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f16f6ffd640 1 -- 172.21.15.83:0/2068942286 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1708051660 con 0x556360b01f30 2024-04-17T10:42:45.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f1721952000 10 client.6699 fetch_fsmap learned FSMap version 32 2024-04-17T10:42:45.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f1721952000 1 -- 172.21.15.83:0/2068942286 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x5563608d93a0 con 0x556360b01f30 2024-04-17T10:42:45.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f16f6ffd640 1 -- 172.21.15.83:0/2068942286 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1708057070 con 0x556360b01f30 2024-04-17T10:42:45.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f16f6ffd640 1 -- 172.21.15.83:0/2068942286 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1708075070 con 0x556360b01f30 2024-04-17T10:42:45.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f1721952000 10 client.6699 fetch_fsmap finished waiting for FSMap version 32 2024-04-17T10:42:45.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f1721952000 10 client.6699 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-17T10:42:45.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f1721952000 20 client.6699 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:45.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f1721952000 1 --2- 172.21.15.83:0/2068942286 >> [v2:172.21.15.167:6832/271994303,v1:172.21.15.167:6833/271994303] conn(0x556360b07160 0x556360b5cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:45.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f1721952000 4 client.6699 mds_command: new command op to 6534 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:42:45.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f1721952000 1 -- 172.21.15.83:0/2068942286 --> [v2:172.21.15.167:6832/271994303,v1:172.21.15.167:6833/271994303] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5563608eca50 con 0x556360b07160 2024-04-17T10:42:45.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f17177fe640 1 --2- 172.21.15.83:0/2068942286 >> [v2:172.21.15.167:6832/271994303,v1:172.21.15.167:6833/271994303] conn(0x556360b07160 0x556360b5cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:45.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f17177fe640 1 --2- 172.21.15.83:0/2068942286 >> [v2:172.21.15.167:6832/271994303,v1:172.21.15.167:6833/271994303] conn(0x556360b07160 0x556360b5cce0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b26d15a7659fbd21 server_cookie=aa7ae238a94b09e4 in_seq=0 out_seq=0 2024-04-17T10:42:45.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f16f6ffd640 10 client.6699 ms_handle_connect on v2:172.21.15.167:6832/271994303 2024-04-17T10:42:45.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f16f6ffd640 1 -- 172.21.15.83:0/2068942286 <== mds.0 v2:172.21.15.167:6832/271994303 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5563608eca50 con 0x556360b07160 2024-04-17T10:42:45.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f16f6ffd640 10 client.6699 handle_command_reply: tid=0 2024-04-17T10:42:45.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f1721952000 1 client.6699 shutdown 2024-04-17T10:42:45.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f1721952000 1 -- 172.21.15.83:0/2068942286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556360b01f30 msgr2=0x556360b52c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:45.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f1721952000 1 --2- 172.21.15.83:0/2068942286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556360b01f30 0x556360b52c80 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f1708055700 tx=0x7f1708000bd0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f1721952000 1 -- 172.21.15.83:0/2068942286 shutdown_connections 2024-04-17T10:42:45.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f1721952000 1 --2- 172.21.15.83:0/2068942286 >> [v2:172.21.15.167:6832/271994303,v1:172.21.15.167:6833/271994303] conn(0x556360b07160 0x556360b5cce0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f1721952000 1 --2- 172.21.15.83:0/2068942286 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556360b4c240 0x556360b05aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f16f6ffd640 0 client.6699 ms_handle_reset on v2:172.21.15.167:6832/271994303 2024-04-17T10:42:45.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f1721952000 1 --2- 172.21.15.83:0/2068942286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556360b4b920 0x556360b531c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f1721952000 1 --2- 172.21.15.83:0/2068942286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556360b01f30 0x556360b52c80 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f1721952000 1 -- 172.21.15.83:0/2068942286 >> 172.21.15.83:0/2068942286 conn(0x556360b082a0 msgr2=0x556360b009e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:45.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f1721952000 1 -- 172.21.15.83:0/2068942286 shutdown_connections 2024-04-17T10:42:45.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f1721952000 1 -- 172.21.15.83:0/2068942286 wait complete. 2024-04-17T10:42:45.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.125+0000 7f1721952000 20 client.6699 trim_cache size 0 max 16384 2024-04-17T10:42:45.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.141+0000 7f1721952000 1 Processor -- start 2024-04-17T10:42:45.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.141+0000 7f1721952000 1 -- start start 2024-04-17T10:42:45.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.141+0000 7f1721952000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556360b01f30 0x556360b052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:45.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.141+0000 7f1721952000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556360b4b920 0x556360b05800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:45.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.141+0000 7f1721952000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556360b4c240 0x556360b548b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:45.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.141+0000 7f1721952000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1718201a50 con 0x556360b4c240 2024-04-17T10:42:45.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.141+0000 7f1721952000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1718200eb0 con 0x556360b01f30 2024-04-17T10:42:45.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.141+0000 7f1721952000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1718201450 con 0x556360b4b920 2024-04-17T10:42:45.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.141+0000 7f171cd82640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556360b4c240 0x556360b548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:45.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f171cd82640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556360b4c240 0x556360b548b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36942/0 (socket says 172.21.15.83:36942) 2024-04-17T10:42:45.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f171cd82640 1 -- 172.21.15.83:0/234338468 learned_addr learned my addr 172.21.15.83:0/234338468 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:45.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f171cd82640 1 -- 172.21.15.83:0/234338468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556360b4b920 msgr2=0x556360b05800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:45.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f171cd82640 1 --2- 172.21.15.83:0/234338468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556360b4b920 0x556360b05800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f171cd82640 1 -- 172.21.15.83:0/234338468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556360b01f30 msgr2=0x556360b052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:45.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f1717fff640 1 --2- 172.21.15.83:0/234338468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556360b01f30 0x556360b052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:45.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f171cd82640 1 --2- 172.21.15.83:0/234338468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556360b01f30 0x556360b052c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f171cd82640 1 -- 172.21.15.83:0/234338468 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556360b4c070 con 0x556360b4c240 2024-04-17T10:42:45.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f171cd82640 1 --2- 172.21.15.83:0/234338468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556360b4c240 0x556360b548b0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f171001a0c0 tx=0x7f17100046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f4487c67b4fc88 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:45.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f170d7fa640 1 -- 172.21.15.83:0/234338468 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17100ba070 con 0x556360b4c240 2024-04-17T10:42:45.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f170d7fa640 1 -- 172.21.15.83:0/234338468 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17100b5660 con 0x556360b4c240 2024-04-17T10:42:45.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f170d7fa640 1 -- 172.21.15.83:0/234338468 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17100b0040 con 0x556360b4c240 2024-04-17T10:42:45.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f1721952000 1 -- 172.21.15.83:0/234338468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556360b4c240 msgr2=0x556360b548b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:45.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f1721952000 1 --2- 172.21.15.83:0/234338468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556360b4c240 0x556360b548b0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f171001a0c0 tx=0x7f17100046d0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f1721952000 1 -- 172.21.15.83:0/234338468 shutdown_connections 2024-04-17T10:42:45.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f1721952000 1 --2- 172.21.15.83:0/234338468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556360b4c240 0x556360b548b0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f1721952000 1 --2- 172.21.15.83:0/234338468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556360b4b920 0x556360b05800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f1721952000 1 --2- 172.21.15.83:0/234338468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556360b01f30 0x556360b052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f1721952000 1 -- 172.21.15.83:0/234338468 >> 172.21.15.83:0/234338468 conn(0x556360b082a0 msgr2=0x556360b04bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:45.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f1721952000 1 -- 172.21.15.83:0/234338468 shutdown_connections 2024-04-17T10:42:45.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f1721952000 1 -- 172.21.15.83:0/234338468 wait complete. 2024-04-17T10:42:45.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f1721952000 1 -- start start 2024-04-17T10:42:45.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f1721952000 1 Processor -- start 2024-04-17T10:42:45.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f1721952000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556360b01f30 0x556360b57e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:45.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f1721952000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556360b4b920 0x556360b8f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:45.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f1721952000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556360b4c240 0x556360b8f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:45.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f1721952000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563609553d0 con 0x556360b4b920 2024-04-17T10:42:45.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f1721952000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556360a0cd50 con 0x556360b01f30 2024-04-17T10:42:45.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f1721952000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5563608eb980 con 0x556360b4c240 2024-04-17T10:42:45.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f1721952000 10 client.0 _ll_register_callbacks cb 0x7f1708054c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:42:45.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f17177fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556360b4b920 0x556360b8f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:45.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f17177fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556360b4b920 0x556360b8f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36954/0 (socket says 172.21.15.83:36954) 2024-04-17T10:42:45.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f17177fe640 1 -- 172.21.15.83:0/1751020771 learned_addr learned my addr 172.21.15.83:0/1751020771 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:45.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f171cd82640 1 --2- 172.21.15.83:0/1751020771 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556360b4c240 0x556360b8f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:45.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f1717fff640 1 --2- 172.21.15.83:0/1751020771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556360b01f30 0x556360b57e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:45.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f17177fe640 1 -- 172.21.15.83:0/1751020771 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556360b4c240 msgr2=0x556360b8f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:45.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f17177fe640 1 --2- 172.21.15.83:0/1751020771 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556360b4c240 0x556360b8f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f17177fe640 1 -- 172.21.15.83:0/1751020771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556360b01f30 msgr2=0x556360b57e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:45.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f17177fe640 1 --2- 172.21.15.83:0/1751020771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556360b01f30 0x556360b57e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f17177fe640 1 -- 172.21.15.83:0/1751020771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5563609d2600 con 0x556360b4b920 2024-04-17T10:42:45.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f17177fe640 1 --2- 172.21.15.83:0/1751020771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556360b4b920 0x556360b8f2d0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f17180c87e0 tx=0x7f1700024710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c363c7bc8dfd04 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:45.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f16f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:42:45.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f16f6ffd640 1 -- 172.21.15.83:0/1751020771 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1700004700 con 0x556360b4b920 2024-04-17T10:42:45.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f1721952000 1 -- 172.21.15.83:0/1751020771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f17100b5660 con 0x556360b4b920 2024-04-17T10:42:45.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f16f6ffd640 1 -- 172.21.15.83:0/1751020771 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f170002d430 con 0x556360b4b920 2024-04-17T10:42:45.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f16f6ffd640 1 -- 172.21.15.83:0/1751020771 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1700034050 con 0x556360b4b920 2024-04-17T10:42:45.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f1721952000 10 client.6705 fetch_fsmap learned FSMap version 32 2024-04-17T10:42:45.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f1721952000 1 -- 172.21.15.83:0/1751020771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x5563608e59b0 con 0x556360b4b920 2024-04-17T10:42:45.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f16f6ffd640 1 -- 172.21.15.83:0/1751020771 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17100b5660 con 0x556360b4b920 2024-04-17T10:42:45.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f16f6ffd640 1 -- 172.21.15.83:0/1751020771 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1700037020 con 0x556360b4b920 2024-04-17T10:42:45.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f1721952000 10 client.6705 fetch_fsmap finished waiting for FSMap version 32 2024-04-17T10:42:45.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f1721952000 10 client.6705 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-17T10:42:45.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f1721952000 20 client.6705 populate_metadata read hostname 'smithi083' 2024-04-17T10:42:45.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f1721952000 1 --2- 172.21.15.83:0/1751020771 >> [v2:172.21.15.167:6832/271994303,v1:172.21.15.167:6833/271994303] conn(0x556360b90020 0x556360b90400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:45.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f1721952000 4 client.6705 mds_command: new command op to 6534 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T10:42:45.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.145+0000 7f1721952000 1 -- 172.21.15.83:0/1751020771 --> [v2:172.21.15.167:6832/271994303,v1:172.21.15.167:6833/271994303] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55636095e2e0 con 0x556360b90020 2024-04-17T10:42:45.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.149+0000 7f1717fff640 1 --2- 172.21.15.83:0/1751020771 >> [v2:172.21.15.167:6832/271994303,v1:172.21.15.167:6833/271994303] conn(0x556360b90020 0x556360b90400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:45.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.149+0000 7f1717fff640 1 --2- 172.21.15.83:0/1751020771 >> [v2:172.21.15.167:6832/271994303,v1:172.21.15.167:6833/271994303] conn(0x556360b90020 0x556360b90400 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2161e7b4c01cca8 server_cookie=7c3ab6d8f48bad66 in_seq=0 out_seq=0 2024-04-17T10:42:45.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.149+0000 7f16f6ffd640 10 client.6705 ms_handle_connect on v2:172.21.15.167:6832/271994303 2024-04-17T10:42:45.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.149+0000 7f16f6ffd640 1 -- 172.21.15.83:0/1751020771 <== mds.0 v2:172.21.15.167:6832/271994303 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55636095e2e0 con 0x556360b90020 2024-04-17T10:42:45.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.149+0000 7f16f6ffd640 10 client.6705 handle_command_reply: tid=0 2024-04-17T10:42:45.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.149+0000 7f1721952000 1 client.6705 shutdown 2024-04-17T10:42:45.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.149+0000 7f1721952000 1 -- 172.21.15.83:0/1751020771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556360b4b920 msgr2=0x556360b8f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:45.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.149+0000 7f1721952000 1 --2- 172.21.15.83:0/1751020771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556360b4b920 0x556360b8f2d0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f17180c87e0 tx=0x7f1700024710 comp rx=0 tx=0).stop 2024-04-17T10:42:45.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.149+0000 7f1721952000 1 -- 172.21.15.83:0/1751020771 shutdown_connections 2024-04-17T10:42:45.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.149+0000 7f1721952000 1 --2- 172.21.15.83:0/1751020771 >> [v2:172.21.15.167:6832/271994303,v1:172.21.15.167:6833/271994303] conn(0x556360b90020 0x556360b90400 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.149+0000 7f1721952000 1 --2- 172.21.15.83:0/1751020771 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556360b4c240 0x556360b8f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.149+0000 7f1721952000 1 --2- 172.21.15.83:0/1751020771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556360b4b920 0x556360b8f2d0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.149+0000 7f16f6ffd640 0 client.6705 ms_handle_reset on v2:172.21.15.167:6832/271994303 2024-04-17T10:42:45.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.149+0000 7f1721952000 1 --2- 172.21.15.83:0/1751020771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556360b01f30 0x556360b57e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.149+0000 7f1721952000 1 -- 172.21.15.83:0/1751020771 >> 172.21.15.83:0/1751020771 conn(0x556360b082a0 msgr2=0x556360b052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:45.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.149+0000 7f1721952000 1 -- 172.21.15.83:0/1751020771 shutdown_connections 2024-04-17T10:42:45.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.149+0000 7f1721952000 1 -- 172.21.15.83:0/1751020771 wait complete. 2024-04-17T10:42:45.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.149+0000 7f1721952000 20 client.6705 trim_cache size 0 max 16384 2024-04-17T10:42:45.155 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:42:45.155 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "3de9b7be-af7f-488a-969e-2b6b8db63bcd", 2024-04-17T10:42:45.155 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-04-17T10:42:45.155 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 6534, 2024-04-17T10:42:45.155 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-17T10:42:45.155 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-17T10:42:45.155 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-17T10:42:45.155 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 10.157114422999999, 2024-04-17T10:42:45.156 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 32, 2024-04-17T10:42:45.156 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 59, 2024-04-17T10:42:45.156 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 59, 2024-04-17T10:42:45.156 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 15.022068973 2024-04-17T10:42:45.156 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:42:45.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.149+0000 7f171e7e5640 1 -- 172.21.15.83:0/907032307 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f16f8066fb0 msgr2=0x7f16f8069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:45.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.149+0000 7f171e7e5640 1 --2- 172.21.15.83:0/907032307 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f16f8066fb0 0x7f16f8069470 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f17180b0900 tx=0x7f170000b040 comp rx=0 tx=0).stop 2024-04-17T10:42:45.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.149+0000 7f171e7e5640 1 -- 172.21.15.83:0/907032307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f171815ed70 msgr2=0x7f1718154a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:45.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.149+0000 7f171e7e5640 1 --2- 172.21.15.83:0/907032307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f171815ed70 0x7f1718154a00 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f170800c420 tx=0x7f170800c8e0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.149+0000 7f171e7e5640 1 -- 172.21.15.83:0/907032307 shutdown_connections 2024-04-17T10:42:45.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.149+0000 7f171e7e5640 1 --2- 172.21.15.83:0/907032307 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f16f8066fb0 0x7f16f8069470 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.149+0000 7f171e7e5640 1 --2- 172.21.15.83:0/907032307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1718161d20 0x7f17180b6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.149+0000 7f171e7e5640 1 --2- 172.21.15.83:0/907032307 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1718161940 0x7f17180b1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.149+0000 7f171e7e5640 1 --2- 172.21.15.83:0/907032307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f171815ed70 0x7f1718154a00 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.149+0000 7f171e7e5640 1 -- 172.21.15.83:0/907032307 >> 172.21.15.83:0/907032307 conn(0x7f17180d17e0 msgr2=0x7f1718160d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:45.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.149+0000 7f171e7e5640 1 -- 172.21.15.83:0/907032307 shutdown_connections 2024-04-17T10:42:45.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.149+0000 7f171e7e5640 1 -- 172.21.15.83:0/907032307 wait complete. 2024-04-17T10:42:45.165 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T10:42:45.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.269+0000 7fd798f4f640 1 Processor -- start 2024-04-17T10:42:45.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.269+0000 7fd798f4f640 1 -- start start 2024-04-17T10:42:45.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.273+0000 7fd798f4f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd79415db50 0x7fd794070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:45.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.273+0000 7fd798f4f640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd79406dcb0 0x7fd7940b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:45.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.273+0000 7fd798f4f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7940709b0 con 0x7fd7940aedd0 2024-04-17T10:42:45.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.273+0000 7fd798f4f640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd79415e440 con 0x7fd79406dcb0 2024-04-17T10:42:45.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.273+0000 7fd798f4f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd79415e720 con 0x7fd79415db50 2024-04-17T10:42:45.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.273+0000 7fd791d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd7940aedd0 0x7fd79406d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57482/0 (socket says 172.21.15.83:57482) 2024-04-17T10:42:45.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.273+0000 7fd791d9b640 1 -- 172.21.15.83:0/86839516 learned_addr learned my addr 172.21.15.83:0/86839516 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:45.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.273+0000 7fd79259c640 1 --2- 172.21.15.83:0/86839516 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd79415db50 0x7fd794070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:45.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.273+0000 7fd792d9d640 1 --2- 172.21.15.83:0/86839516 >> v2:172.21.15.167:3300/0 conn(0x7fd79406dcb0 0x7fd7940b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:45.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.273+0000 7fd79259c640 1 -- 172.21.15.83:0/86839516 >> v2:172.21.15.167:3300/0 conn(0x7fd79406dcb0 msgr2=0x7fd7940b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:45.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.273+0000 7fd79259c640 1 --2- 172.21.15.83:0/86839516 >> v2:172.21.15.167:3300/0 conn(0x7fd79406dcb0 0x7fd7940b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.273+0000 7fd79259c640 1 -- 172.21.15.83:0/86839516 >> v1:172.21.15.83:6789/0 conn(0x7fd7940aedd0 legacy=0x7fd79406d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:45.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.273+0000 7fd79259c640 1 -- 172.21.15.83:0/86839516 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd79415ea00 con 0x7fd79415db50 2024-04-17T10:42:45.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.277+0000 7fd79259c640 1 --2- 172.21.15.83:0/86839516 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd79415db50 0x7fd794070470 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd78c0097c0 tx=0x7fd78c01c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ae1bb8568363460 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:45.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.277+0000 7fd79159a640 1 -- 172.21.15.83:0/86839516 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c0a7070 con 0x7fd79415db50 2024-04-17T10:42:45.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.277+0000 7fd79159a640 1 -- 172.21.15.83:0/86839516 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78c0b0070 con 0x7fd79415db50 2024-04-17T10:42:45.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.277+0000 7fd79159a640 1 -- 172.21.15.83:0/86839516 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c0ab890 con 0x7fd79415db50 2024-04-17T10:42:45.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.277+0000 7fd798f4f640 1 -- 172.21.15.83:0/86839516 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd79415db50 msgr2=0x7fd794070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:45.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.277+0000 7fd798f4f640 1 --2- 172.21.15.83:0/86839516 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd79415db50 0x7fd794070470 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd78c0097c0 tx=0x7fd78c01c470 comp rx=0 tx=0).stop 2024-04-17T10:42:45.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.277+0000 7fd798f4f640 1 -- 172.21.15.83:0/86839516 shutdown_connections 2024-04-17T10:42:45.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.277+0000 7fd798f4f640 1 --2- 172.21.15.83:0/86839516 >> v2:172.21.15.167:3300/0 conn(0x7fd79406dcb0 0x7fd7940b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.277+0000 7fd798f4f640 1 --2- 172.21.15.83:0/86839516 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd79415db50 0x7fd794070470 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.277+0000 7fd798f4f640 1 -- 172.21.15.83:0/86839516 >> 172.21.15.83:0/86839516 conn(0x7fd79406abe0 msgr2=0x7fd79406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:45.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.277+0000 7fd798f4f640 1 -- 172.21.15.83:0/86839516 shutdown_connections 2024-04-17T10:42:45.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.277+0000 7fd798f4f640 1 -- 172.21.15.83:0/86839516 wait complete. 2024-04-17T10:42:45.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.281+0000 7fd798f4f640 1 Processor -- start 2024-04-17T10:42:45.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.281+0000 7fd798f4f640 1 -- start start 2024-04-17T10:42:45.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.281+0000 7fd798f4f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd79406dcb0 0x7fd79414eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:45.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.281+0000 7fd798f4f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7940aedd0 0x7fd79414f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:45.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.281+0000 7fd798f4f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd79415db50 0x7fd794155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:45.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.281+0000 7fd798f4f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7940b3ce0 con 0x7fd7940aedd0 2024-04-17T10:42:45.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.281+0000 7fd798f4f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7940b3b60 con 0x7fd79415db50 2024-04-17T10:42:45.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.281+0000 7fd798f4f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7940b3e60 con 0x7fd79406dcb0 2024-04-17T10:42:45.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.281+0000 7fd791d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7940aedd0 0x7fd79414f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:45.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.281+0000 7fd791d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7940aedd0 0x7fd79414f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36958/0 (socket says 172.21.15.83:36958) 2024-04-17T10:42:45.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.281+0000 7fd791d9b640 1 -- 172.21.15.83:0/1805875796 learned_addr learned my addr 172.21.15.83:0/1805875796 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:45.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.281+0000 7fd79259c640 1 --2- 172.21.15.83:0/1805875796 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd79406dcb0 0x7fd79414eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:45.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.281+0000 7fd792d9d640 1 --2- 172.21.15.83:0/1805875796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd79415db50 0x7fd794155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:45.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.281+0000 7fd791d9b640 1 -- 172.21.15.83:0/1805875796 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd79406dcb0 msgr2=0x7fd79414eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:45.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.281+0000 7fd791d9b640 1 --2- 172.21.15.83:0/1805875796 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd79406dcb0 0x7fd79414eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.281+0000 7fd791d9b640 1 -- 172.21.15.83:0/1805875796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd79415db50 msgr2=0x7fd794155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:45.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.281+0000 7fd791d9b640 1 --2- 172.21.15.83:0/1805875796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd79415db50 0x7fd794155bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.281+0000 7fd791d9b640 1 -- 172.21.15.83:0/1805875796 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd79414f3f0 con 0x7fd7940aedd0 2024-04-17T10:42:45.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.281+0000 7fd791d9b640 1 --2- 172.21.15.83:0/1805875796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7940aedd0 0x7fd79414f820 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fd78400cba0 tx=0x7fd78400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:45.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.281+0000 7fd78affd640 1 -- 172.21.15.83:0/1805875796 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784017070 con 0x7fd7940aedd0 2024-04-17T10:42:45.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.281+0000 7fd78affd640 1 -- 172.21.15.83:0/1805875796 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78400ad40 con 0x7fd7940aedd0 2024-04-17T10:42:45.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.281+0000 7fd798f4f640 1 -- 172.21.15.83:0/1805875796 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd79414fd60 con 0x7fd7940aedd0 2024-04-17T10:42:45.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.281+0000 7fd78affd640 1 -- 172.21.15.83:0/1805875796 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784012600 con 0x7fd7940aedd0 2024-04-17T10:42:45.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.281+0000 7fd798f4f640 1 -- 172.21.15.83:0/1805875796 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd794150840 con 0x7fd7940aedd0 2024-04-17T10:42:45.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.281+0000 7fd798f4f640 1 -- 172.21.15.83:0/1805875796 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd794074f90 con 0x7fd7940aedd0 2024-04-17T10:42:45.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.281+0000 7fd78affd640 1 -- 172.21.15.83:0/1805875796 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd78401a070 con 0x7fd7940aedd0 2024-04-17T10:42:45.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.281+0000 7fd78affd640 1 --2- 172.21.15.83:0/1805875796 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd774066f90 0x7fd774069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:45.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.281+0000 7fd78affd640 1 -- 172.21.15.83:0/1805875796 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd784084ca0 con 0x7fd7940aedd0 2024-04-17T10:42:45.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.285+0000 7fd79259c640 1 --2- 172.21.15.83:0/1805875796 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd774066f90 0x7fd774069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:45.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.285+0000 7fd79259c640 1 --2- 172.21.15.83:0/1805875796 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd774066f90 0x7fd774069450 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd78c0af000 tx=0x7fd78c0b6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:45.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.289+0000 7fd78affd640 1 -- 172.21.15.83:0/1805875796 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd79414f3f0 con 0x7fd7940aedd0 2024-04-17T10:42:45.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.465+0000 7fd798f4f640 1 -- 172.21.15.83:0/1805875796 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd7940b2af0 con 0x7fd7940aedd0 2024-04-17T10:42:45.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.465+0000 7fd78affd640 1 -- 172.21.15.83:0/1805875796 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd7940709b0 con 0x7fd7940aedd0 2024-04-17T10:42:45.465 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T10:42:45.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.465+0000 7fd798f4f640 1 -- 172.21.15.83:0/1805875796 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd774066f90 msgr2=0x7fd774069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:45.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.465+0000 7fd798f4f640 1 --2- 172.21.15.83:0/1805875796 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd774066f90 0x7fd774069450 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd78c0af000 tx=0x7fd78c0b6040 comp rx=0 tx=0).stop 2024-04-17T10:42:45.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.465+0000 7fd798f4f640 1 -- 172.21.15.83:0/1805875796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7940aedd0 msgr2=0x7fd79414f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:45.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.465+0000 7fd798f4f640 1 --2- 172.21.15.83:0/1805875796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7940aedd0 0x7fd79414f820 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fd78400cba0 tx=0x7fd78400a430 comp rx=0 tx=0).stop 2024-04-17T10:42:45.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.465+0000 7fd798f4f640 1 -- 172.21.15.83:0/1805875796 shutdown_connections 2024-04-17T10:42:45.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.465+0000 7fd798f4f640 1 --2- 172.21.15.83:0/1805875796 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd774066f90 0x7fd774069450 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.465+0000 7fd798f4f640 1 --2- 172.21.15.83:0/1805875796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd79415db50 0x7fd794155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.465+0000 7fd798f4f640 1 --2- 172.21.15.83:0/1805875796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7940aedd0 0x7fd79414f820 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.465+0000 7fd798f4f640 1 --2- 172.21.15.83:0/1805875796 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd79406dcb0 0x7fd79414eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:45.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.465+0000 7fd798f4f640 1 -- 172.21.15.83:0/1805875796 >> 172.21.15.83:0/1805875796 conn(0x7fd79406abe0 msgr2=0x7fd7940cc410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:45.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.465+0000 7fd798f4f640 1 -- 172.21.15.83:0/1805875796 shutdown_connections 2024-04-17T10:42:45.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:45.465+0000 7fd798f4f640 1 -- 172.21.15.83:0/1805875796 wait complete. 2024-04-17T10:42:45.482 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T10:42:45.482 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T10:42:45.483 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-04-17T10:42:45.483 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T10:42:45.483 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T10:42:45.483 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T10:42:45.483 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T10:42:45.483 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T10:42:45.483 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:42:45.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:42:45.483 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-04-17T10:42:45.489 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T10:42:45.490 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T10:42:45.490 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T10:42:45.490 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:42:45.490 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-17T10:42:45.490 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:42:45.490 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:42:45.490 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-04-17T10:42:45.490 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-04-17T10:42:45.490 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:42:45.490 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-04-17T10:42:45.491 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-04-17T10:42:45.491 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:42:45.491 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:42:45.491 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T10:42:45.491 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:42:45.491 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-17T10:42:45.491 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42219sec preferred_lft 42219sec 2024-04-17T10:42:45.491 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-04-17T10:42:45.491 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:42:45.491 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T10:42:45.491 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:37:0e:23:b2 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:42:45.491 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T10:42:45.491 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:42:45.492 INFO:teuthology.orchestra.run.smithi083.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T10:42:45.492 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-17T10:42:45.492 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T10:42:45.492 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:42:45.492 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-17T10:42:45.492 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:42:45.492 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:42:45.492 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:42:45.492 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-17T10:42:45.492 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:42:45.492 INFO:teuthology.orchestra.run.smithi083.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:42:45.492 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:42:45.492 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-17T10:42:45.492 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:42:45.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:42:45.493 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T10:42:45.540 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-04-17T10:42:45.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:42:45.541 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-04-17T10:42:45.591 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-04-17T10:42:45.591 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T10:42:45.591 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-17T10:42:45.591 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-17T10:42:45.591 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-17T10:42:45.591 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T10:42:45.591 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-17T10:42:45.591 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:42:45.591 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:42:45.592 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:42:45.592 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T10:42:45.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:42:45.592 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2024-04-17T10:42:45.592 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:42:45.592 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-17T10:42:45.592 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-17T10:42:45.592 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-17T10:42:45.592 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:42:45.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:42:45.668 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-17T10:42:45.720 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-17T10:42:45.720 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T10:42:45.720 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:42:45.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:42:45.721 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-04-17T10:42:45.776 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-17T10:42:45.776 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:42:45.776 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T10:42:45.824 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:42:45.825 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:42:45.825 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:42:45.876 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:42:45.938 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:42:45.985 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:42:45.985 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:42:45.985 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:42:45.985 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:42:45.985 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:42:45.985 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:42:45.986 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:42:45.986 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:42:45.986 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:42:45.986 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:42:45.986 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:42:45.986 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:42:45.986 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:42:45.986 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:42:45.986 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:42:45.986 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:42:45.986 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:42:45.986 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:42:45.986 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:42:45.987 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:42:45.987 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:42:45.987 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:42:45.987 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:42:45.987 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:42:45.987 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:42:45.987 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:42:45.987 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:42:45.987 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:42:45.987 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:42:45.987 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:42:45.987 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:42:45.987 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:42:45.987 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:42:45.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:42:45.988 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:42:46.036 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T10:42:46.037 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-17T10:42:46.081 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:42:46.095 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:42:46.144 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:42:46.144 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:42:46.144 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:42:46.144 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:42:46.144 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:42:46.145 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:42:46.145 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:42:46.145 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:42:46.145 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:42:46.145 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:42:46.145 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:42:46.145 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:42:46.145 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:42:46.145 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:42:46.145 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:42:46.145 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:42:46.145 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:42:46.145 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:42:46.146 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:42:46.146 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:42:46.146 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:42:46.146 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:42:46.146 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:42:46.146 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:42:46.146 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:42:46.146 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:42:46.146 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:42:46.146 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:42:46.146 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:42:46.146 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:42:46.147 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:42:46.147 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:42:46.147 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:42:46.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:42:46.148 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:42:46.160 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:2024-04-17T10:42:46.161+0000 7f7dae34c580 -1 init, newargv = 0x559202533300 newargc=15 2024-04-17T10:42:46.160 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[54562]: starting ceph client 2024-04-17T10:42:46.167 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[54562]: starting fuse 2024-04-17T10:42:47.265 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:42:47.281 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:42:47.331 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:42:47.331 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:42:47.331 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:42:47.331 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:42:47.331 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:42:47.332 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:42:47.332 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:42:47.332 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:42:47.332 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:42:47.332 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:42:47.332 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:42:47.332 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:42:47.332 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:42:47.332 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:42:47.332 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:42:47.332 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:42:47.332 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:42:47.332 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:42:47.332 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:42:47.332 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:42:47.333 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:42:47.333 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:42:47.333 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:42:47.333 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:42:47.333 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:42:47.333 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:42:47.333 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:42:47.333 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:42:47.333 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:42:47.333 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:42:47.333 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:42:47.333 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:42:47.333 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:42:47.333 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-17T10:42:47.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:42:47.335 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:42:47.380 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-04-17T10:42:47.381 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-17T10:42:47.381 DEBUG:teuthology.orchestra.run.smithi083:> sudo stdin-killer -- python3 -c ' 2024-04-17T10:42:47.381 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2024-04-17T10:42:47.381 DEBUG:teuthology.orchestra.run.smithi083:> import re 2024-04-17T10:42:47.381 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-04-17T10:42:47.381 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2024-04-17T10:42:47.381 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:42:47.381 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2024-04-17T10:42:47.381 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-17T10:42:47.382 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2024-04-17T10:42:47.382 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-17T10:42:47.382 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:42:47.382 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2024-04-17T10:42:47.382 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2024-04-17T10:42:47.382 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2024-04-17T10:42:47.382 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2024-04-17T10:42:47.382 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:42:47.382 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2024-04-17T10:42:47.382 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T10:42:47.382 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T10:42:47.382 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T10:42:47.382 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2024-04-17T10:42:47.382 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2024-04-17T10:42:47.383 DEBUG:teuthology.orchestra.run.smithi083:> return f 2024-04-17T10:42:47.383 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T10:42:47.383 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:42:47.383 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.0")) 2024-04-17T10:42:47.383 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-04-17T10:42:47.504 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.0.54562.asok 2024-04-17T10:42:47.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:42:47.516 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.54562.asok 2024-04-17T10:42:47.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:42:47.517 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.54562.asok status 2024-04-17T10:42:47.678 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:42:47.678 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2024-04-17T10:42:47.678 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-17T10:42:47.678 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-17T10:42:47.678 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "0", 2024-04-17T10:42:47.678 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-04-17T10:42:47.679 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-17T10:42:47.679 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "54562", 2024-04-17T10:42:47.679 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-04-17T10:42:47.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:42:47.679 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2024-04-17T10:42:47.679 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2024-04-17T10:42:47.679 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 6714, 2024-04-17T10:42:47.679 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2024-04-17T10:42:47.679 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-04-17T10:42:47.679 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-04-17T10:42:47.679 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 6714 2024-04-17T10:42:47.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:42:47.680 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-04-17T10:42:47.680 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-17T10:42:47.680 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:42:47.680 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2611495324 2024-04-17T10:42:47.680 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:42:47.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:42:47.680 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-04-17T10:42:47.680 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-17T10:42:47.680 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:42:47.680 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2611495324 2024-04-17T10:42:47.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:42:47.680 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.6714 192.168.0.1:0/2611495324", 2024-04-17T10:42:47.680 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.1:0/2611495324", 2024-04-17T10:42:47.681 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 1, 2024-04-17T10:42:47.681 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 32, 2024-04-17T10:42:47.681 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 59, 2024-04-17T10:42:47.681 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2024-04-17T10:42:47.681 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2024-04-17T10:42:47.681 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2024-04-17T10:42:47.681 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:42:47.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:42:47.692 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2024-04-17T10:42:47.692 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-17T10:42:47.698 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2024-04-17T10:42:47.698 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-17T10:42:47.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:42:47.698 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:42:47.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:42:47.752 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scrub_repair_path) 2024-04-17T10:42:47.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:42:47.815 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch scrub_repair_path/file) 2024-04-17T10:42:47.873 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-04-17T10:42:47.873 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-04-17T10:42:47.873 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-04-17T10:42:47.873 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:42:47.873 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_repair_path").st_ino) 2024-04-17T10:42:47.873 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-04-17T10:42:47.982 INFO:teuthology.orchestra.run.smithi083.stdout:1099511627776 2024-04-17T10:42:47.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:42:47.992 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:42:47.992 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:42:48.040 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:42:48.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.209+0000 7f7eab561640 1 Processor -- start 2024-04-17T10:42:48.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.209+0000 7f7eab561640 1 -- start start 2024-04-17T10:42:48.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.213+0000 7f7eab561640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7ea40cfbe0 0x7f7ea406c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:48.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.213+0000 7f7eab561640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ea406e320 con 0x7f7ea4072330 2024-04-17T10:42:48.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.213+0000 7f7eab561640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ea406e040 con 0x7f7ea415ba30 2024-04-17T10:42:48.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.213+0000 7f7eab561640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea4071ea0 con 0x7f7ea40cfbe0 2024-04-17T10:42:48.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.213+0000 7f7ea8afc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7ea4072330 0x7f7ea40cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52772/0 (socket says 172.21.15.83:52772) 2024-04-17T10:42:48.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.213+0000 7f7ea8afc640 1 -- 172.21.15.83:0/1556369628 learned_addr learned my addr 172.21.15.83:0/1556369628 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:48.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.213+0000 7f7ea9afe640 1 --2- 172.21.15.83:0/1556369628 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7ea40cfbe0 0x7f7ea406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:48.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.213+0000 7f7e9bfff640 1 -- 172.21.15.83:0/1556369628 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3130597600 0 0) 0x7f7ea406e320 con 0x7f7ea4072330 2024-04-17T10:42:48.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.213+0000 7f7e9bfff640 1 -- 172.21.15.83:0/1556369628 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e90003530 con 0x7f7ea4072330 2024-04-17T10:42:48.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.213+0000 7f7e9bfff640 1 -- 172.21.15.83:0/1556369628 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1530845540 0 0) 0x7f7e90003530 con 0x7f7ea4072330 2024-04-17T10:42:48.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.213+0000 7f7e9bfff640 1 -- 172.21.15.83:0/1556369628 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7ea40cfbe0 msgr2=0x7f7ea406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:48.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.213+0000 7f7e9bfff640 1 --2- 172.21.15.83:0/1556369628 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7ea40cfbe0 0x7f7ea406c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:48.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.213+0000 7f7e9bfff640 1 -- 172.21.15.83:0/1556369628 >> v1:172.21.15.167:6789/0 conn(0x7f7ea415ba30 legacy=0x7f7ea415c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:48.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.213+0000 7f7e9bfff640 1 -- 172.21.15.83:0/1556369628 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea4071850 con 0x7f7ea4072330 2024-04-17T10:42:48.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.213+0000 7f7e9bfff640 1 -- 172.21.15.83:0/1556369628 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f7e94001680 con 0x7f7ea4072330 2024-04-17T10:42:48.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.213+0000 7f7e9bfff640 1 -- 172.21.15.83:0/1556369628 >> v1:172.21.15.83:6789/0 conn(0x7f7ea4072330 legacy=0x7f7ea40cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:48.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.213+0000 7f7e9bfff640 1 --2- 172.21.15.83:0/1556369628 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7ea40cfbe0 0x7f7e900039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:48.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.213+0000 7f7e9bfff640 1 --2- 172.21.15.83:0/1556369628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e90003f70 0x7f7e90004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:48.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.213+0000 7f7e9bfff640 1 --2- 172.21.15.83:0/1556369628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e90004d50 0x7f7e900051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:48.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.213+0000 7f7e9bfff640 1 -- 172.21.15.83:0/1556369628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea4071ea0 con 0x7f7e90003f70 2024-04-17T10:42:48.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.213+0000 7f7e9bfff640 1 -- 172.21.15.83:0/1556369628 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9000bb60 con 0x7f7e90004d50 2024-04-17T10:42:48.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.213+0000 7f7e9bfff640 1 -- 172.21.15.83:0/1556369628 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9000bd60 con 0x7f7ea40cfbe0 2024-04-17T10:42:48.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.213+0000 7f7ea8afc640 1 --2- 172.21.15.83:0/1556369628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e90003f70 0x7f7e90004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:48.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.213+0000 7f7ea92fd640 1 --2- 172.21.15.83:0/1556369628 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7ea40cfbe0 0x7f7e900039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:48.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.213+0000 7f7ea9afe640 1 --2- 172.21.15.83:0/1556369628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e90004d50 0x7f7e900051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:48.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.213+0000 7f7ea8afc640 1 -- 172.21.15.83:0/1556369628 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7ea40cfbe0 msgr2=0x7f7e900039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:48.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.213+0000 7f7ea8afc640 1 --2- 172.21.15.83:0/1556369628 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7ea40cfbe0 0x7f7e900039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:48.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.213+0000 7f7ea8afc640 1 -- 172.21.15.83:0/1556369628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e90004d50 msgr2=0x7f7e900051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:48.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.213+0000 7f7ea8afc640 1 --2- 172.21.15.83:0/1556369628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e90004d50 0x7f7e900051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:48.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.213+0000 7f7ea8afc640 1 -- 172.21.15.83:0/1556369628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e90003530 con 0x7f7e90003f70 2024-04-17T10:42:48.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7ea8afc640 1 --2- 172.21.15.83:0/1556369628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e90003f70 0x7f7e90004810 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400c240 tx=0x7f7e9401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b4c269bdbcd4de server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:48.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7e9bfff640 1 -- 172.21.15.83:0/1556369628 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e94007d20 con 0x7f7e90003f70 2024-04-17T10:42:48.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7e9bfff640 1 -- 172.21.15.83:0/1556369628 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e940a9070 con 0x7f7e90003f70 2024-04-17T10:42:48.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7e9bfff640 1 -- 172.21.15.83:0/1556369628 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e94005d40 con 0x7f7e90003f70 2024-04-17T10:42:48.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7eab561640 1 -- 172.21.15.83:0/1556369628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e90003f70 msgr2=0x7f7e90004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:48.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7eab561640 1 --2- 172.21.15.83:0/1556369628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e90003f70 0x7f7e90004810 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400c240 tx=0x7f7e9401cc80 comp rx=0 tx=0).stop 2024-04-17T10:42:48.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7ea9afe640 1 -- 172.21.15.83:0/1556369628 reap_dead start 2024-04-17T10:42:48.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7eab561640 1 -- 172.21.15.83:0/1556369628 shutdown_connections 2024-04-17T10:42:48.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7eab561640 1 -- 172.21.15.83:0/1556369628 >> 172.21.15.83:0/1556369628 conn(0x7f7ea406abe0 msgr2=0x7f7ea406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:48.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7eab561640 1 -- 172.21.15.83:0/1556369628 shutdown_connections 2024-04-17T10:42:48.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7eab561640 1 -- 172.21.15.83:0/1556369628 wait complete. 2024-04-17T10:42:48.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7eab561640 1 Processor -- start 2024-04-17T10:42:48.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7eab561640 1 -- start start 2024-04-17T10:42:48.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7eab561640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e90003f70 0x7f7ea415cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:48.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7eab561640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ea40af050 0x7f7ea41484d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:48.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7eab561640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ea40b0660 0x7f7ea40b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:48.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7eab561640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea406ce20 con 0x7f7ea40af050 2024-04-17T10:42:48.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7eab561640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea406cca0 con 0x7f7ea40b0660 2024-04-17T10:42:48.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7eab561640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea406cfa0 con 0x7f7e90003f70 2024-04-17T10:42:48.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7ea8afc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ea40af050 0x7f7ea41484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:48.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7ea8afc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ea40af050 0x7f7ea41484d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34664/0 (socket says 172.21.15.83:34664) 2024-04-17T10:42:48.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7ea8afc640 1 -- 172.21.15.83:0/1901433938 learned_addr learned my addr 172.21.15.83:0/1901433938 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:48.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7ea92fd640 1 --2- 172.21.15.83:0/1901433938 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e90003f70 0x7f7ea415cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:48.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7ea9afe640 1 --2- 172.21.15.83:0/1901433938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ea40b0660 0x7f7ea40b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:48.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7ea8afc640 1 -- 172.21.15.83:0/1901433938 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e90003f70 msgr2=0x7f7ea415cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:48.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7ea8afc640 1 --2- 172.21.15.83:0/1901433938 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e90003f70 0x7f7ea415cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:48.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7ea8afc640 1 -- 172.21.15.83:0/1901433938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ea40b0660 msgr2=0x7f7ea40b0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:48.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7ea8afc640 1 --2- 172.21.15.83:0/1901433938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ea40b0660 0x7f7ea40b0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:48.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7ea8afc640 1 -- 172.21.15.83:0/1901433938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea40af5c0 con 0x7f7ea40af050 2024-04-17T10:42:48.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7ea9afe640 1 --2- 172.21.15.83:0/1901433938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ea40b0660 0x7f7ea40b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:48.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7ea8afc640 1 --2- 172.21.15.83:0/1901433938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ea40af050 0x7f7ea41484d0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f7e94098590 tx=0x7f7e90005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:48.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7e99ffb640 1 -- 172.21.15.83:0/1901433938 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e94007d20 con 0x7f7ea40af050 2024-04-17T10:42:48.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7e99ffb640 1 -- 172.21.15.83:0/1901433938 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e940a5870 con 0x7f7ea40af050 2024-04-17T10:42:48.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7eab561640 1 -- 172.21.15.83:0/1901433938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea40af8a0 con 0x7f7ea40af050 2024-04-17T10:42:48.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7e99ffb640 1 -- 172.21.15.83:0/1901433938 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e940a5c60 con 0x7f7ea40af050 2024-04-17T10:42:48.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.217+0000 7f7eab561640 1 -- 172.21.15.83:0/1901433938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea40afe60 con 0x7f7ea40af050 2024-04-17T10:42:48.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.221+0000 7f7eab561640 1 -- 172.21.15.83:0/1901433938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ea4074f90 con 0x7f7ea40af050 2024-04-17T10:42:48.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.221+0000 7f7e99ffb640 1 -- 172.21.15.83:0/1901433938 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7e94007140 con 0x7f7ea40af050 2024-04-17T10:42:48.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.221+0000 7f7e99ffb640 1 --2- 172.21.15.83:0/1901433938 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7e88066f90 0x7f7e88069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:48.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.221+0000 7f7ea92fd640 1 --2- 172.21.15.83:0/1901433938 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7e88066f90 0x7f7e88069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:48.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.221+0000 7f7e99ffb640 1 -- 172.21.15.83:0/1901433938 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e9411e9b0 con 0x7f7ea40af050 2024-04-17T10:42:48.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.221+0000 7f7ea92fd640 1 --2- 172.21.15.83:0/1901433938 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7e88066f90 0x7f7e88069450 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0000c10 tx=0x7f7ea0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:48.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.225+0000 7f7e99ffb640 1 -- 172.21.15.83:0/1901433938 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7e94016990 con 0x7f7ea40af050 2024-04-17T10:42:48.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.385+0000 7f7eab561640 1 -- 172.21.15.83:0/1901433938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ea4158730 con 0x7f7ea40af050 2024-04-17T10:42:48.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.385+0000 7f7e99ffb640 1 -- 172.21.15.83:0/1901433938 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+16593 (secure 0 0 0) 0x7f7e940ef9d0 con 0x7f7ea40af050 2024-04-17T10:42:48.387 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:48.387 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":59,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:42:35.975462+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:42:32.530700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-17T10:42:33.334905+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:42:48.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.389+0000 7f7eab561640 1 -- 172.21.15.83:0/1901433938 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7e88066f90 msgr2=0x7f7e88069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:48.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.389+0000 7f7eab561640 1 --2- 172.21.15.83:0/1901433938 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7e88066f90 0x7f7e88069450 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0000c10 tx=0x7f7ea0009040 comp rx=0 tx=0).stop 2024-04-17T10:42:48.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.389+0000 7f7eab561640 1 -- 172.21.15.83:0/1901433938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ea40af050 msgr2=0x7f7ea41484d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:48.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.389+0000 7f7eab561640 1 --2- 172.21.15.83:0/1901433938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ea40af050 0x7f7ea41484d0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f7e94098590 tx=0x7f7e90005840 comp rx=0 tx=0).stop 2024-04-17T10:42:48.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.389+0000 7f7eab561640 1 -- 172.21.15.83:0/1901433938 shutdown_connections 2024-04-17T10:42:48.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.389+0000 7f7eab561640 1 --2- 172.21.15.83:0/1901433938 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7e88066f90 0x7f7e88069450 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:48.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.389+0000 7f7eab561640 1 --2- 172.21.15.83:0/1901433938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ea40b0660 0x7f7ea40b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:48.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.389+0000 7f7eab561640 1 --2- 172.21.15.83:0/1901433938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ea40af050 0x7f7ea41484d0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:48.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.389+0000 7f7eab561640 1 --2- 172.21.15.83:0/1901433938 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e90003f70 0x7f7ea415cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:48.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.389+0000 7f7eab561640 1 -- 172.21.15.83:0/1901433938 >> 172.21.15.83:0/1901433938 conn(0x7f7ea406abe0 msgr2=0x7f7ea415a290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:48.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.389+0000 7f7eab561640 1 -- 172.21.15.83:0/1901433938 shutdown_connections 2024-04-17T10:42:48.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.389+0000 7f7eab561640 1 -- 172.21.15.83:0/1901433938 wait complete. 2024-04-17T10:42:48.406 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:42:32.530700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:42:33.334905+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:42:48.407 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:42:48.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d2c790640 1 Processor -- start 2024-04-17T10:42:48.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d2c790640 1 -- start start 2024-04-17T10:42:48.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d2c790640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d240b5b60 0x7f8d240d17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:48.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d2c790640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d24166cb0 con 0x7f8d240ce5b0 2024-04-17T10:42:48.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d2c790640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d241666f0 con 0x7f8d24165d50 2024-04-17T10:42:48.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d2c790640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d24162c50 con 0x7f8d240b5b60 2024-04-17T10:42:48.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d2a52c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8d240ce5b0 0x7f8d241630e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52788/0 (socket says 172.21.15.83:52788) 2024-04-17T10:42:48.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d2a52c640 1 -- 172.21.15.83:0/1528226777 learned_addr learned my addr 172.21.15.83:0/1528226777 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:48.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d2ad2d640 1 --2- 172.21.15.83:0/1528226777 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d240b5b60 0x7f8d240d17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:48.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d2952a640 1 -- 172.21.15.83:0/1528226777 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 718976547 0 0) 0x7f8d24166cb0 con 0x7f8d240ce5b0 2024-04-17T10:42:48.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d2952a640 1 -- 172.21.15.83:0/1528226777 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d10003530 con 0x7f8d240ce5b0 2024-04-17T10:42:48.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d2952a640 1 -- 172.21.15.83:0/1528226777 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 737601966 0 0) 0x7f8d10003530 con 0x7f8d240ce5b0 2024-04-17T10:42:48.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d2952a640 1 -- 172.21.15.83:0/1528226777 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d240b5b60 msgr2=0x7f8d240d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:48.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d2952a640 1 --2- 172.21.15.83:0/1528226777 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d240b5b60 0x7f8d240d17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:48.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d2952a640 1 -- 172.21.15.83:0/1528226777 >> v1:172.21.15.167:6789/0 conn(0x7f8d24165d50 legacy=0x7f8d240b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:48.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d2952a640 1 -- 172.21.15.83:0/1528226777 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d241669d0 con 0x7f8d240ce5b0 2024-04-17T10:42:48.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d2952a640 1 -- 172.21.15.83:0/1528226777 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f8d20001680 con 0x7f8d240ce5b0 2024-04-17T10:42:48.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d2952a640 1 -- 172.21.15.83:0/1528226777 >> v1:172.21.15.83:6789/0 conn(0x7f8d240ce5b0 legacy=0x7f8d241630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:48.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d2952a640 1 --2- 172.21.15.83:0/1528226777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d10003a60 0x7f8d10003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:48.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d2952a640 1 --2- 172.21.15.83:0/1528226777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d10004810 0x7f8d10004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:48.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d2952a640 1 --2- 172.21.15.83:0/1528226777 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d240b5b60 0x7f8d10005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:48.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d2952a640 1 -- 172.21.15.83:0/1528226777 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d24162c50 con 0x7f8d10003a60 2024-04-17T10:42:48.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d2952a640 1 -- 172.21.15.83:0/1528226777 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1000bbc0 con 0x7f8d10004810 2024-04-17T10:42:48.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d2952a640 1 -- 172.21.15.83:0/1528226777 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1000bdc0 con 0x7f8d240b5b60 2024-04-17T10:42:48.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d2a52c640 1 --2- 172.21.15.83:0/1528226777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d10003a60 0x7f8d10003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:48.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d2ad2d640 1 --2- 172.21.15.83:0/1528226777 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d240b5b60 0x7f8d10005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:48.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d29d2b640 1 --2- 172.21.15.83:0/1528226777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d10004810 0x7f8d10004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:48.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d2a52c640 1 -- 172.21.15.83:0/1528226777 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d240b5b60 msgr2=0x7f8d10005280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:48.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d2a52c640 1 --2- 172.21.15.83:0/1528226777 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d240b5b60 0x7f8d10005280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:48.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d2a52c640 1 -- 172.21.15.83:0/1528226777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d10004810 msgr2=0x7f8d10004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:48.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d2a52c640 1 --2- 172.21.15.83:0/1528226777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d10004810 0x7f8d10004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:48.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d2a52c640 1 -- 172.21.15.83:0/1528226777 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d10003530 con 0x7f8d10003a60 2024-04-17T10:42:48.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.537+0000 7f8d29d2b640 1 --2- 172.21.15.83:0/1528226777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d10004810 0x7f8d10004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:48.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.541+0000 7f8d2a52c640 1 --2- 172.21.15.83:0/1528226777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d10003a60 0x7f8d10003e80 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f8d20001ec0 tx=0x7f8d2001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22bc8bab85f9c901 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:48.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.541+0000 7f8d2952a640 1 -- 172.21.15.83:0/1528226777 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d20007d40 con 0x7f8d10003a60 2024-04-17T10:42:48.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.541+0000 7f8d2952a640 1 -- 172.21.15.83:0/1528226777 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d200a9070 con 0x7f8d10003a60 2024-04-17T10:42:48.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.541+0000 7f8d2952a640 1 -- 172.21.15.83:0/1528226777 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d20005d40 con 0x7f8d10003a60 2024-04-17T10:42:48.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.541+0000 7f8d2c790640 1 -- 172.21.15.83:0/1528226777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d10003a60 msgr2=0x7f8d10003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:48.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.541+0000 7f8d2c790640 1 --2- 172.21.15.83:0/1528226777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d10003a60 0x7f8d10003e80 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f8d20001ec0 tx=0x7f8d2001cc60 comp rx=0 tx=0).stop 2024-04-17T10:42:48.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.541+0000 7f8d2ad2d640 1 -- 172.21.15.83:0/1528226777 reap_dead start 2024-04-17T10:42:48.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.541+0000 7f8d2c790640 1 -- 172.21.15.83:0/1528226777 shutdown_connections 2024-04-17T10:42:48.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.541+0000 7f8d2c790640 1 -- 172.21.15.83:0/1528226777 >> 172.21.15.83:0/1528226777 conn(0x7f8d240d0450 msgr2=0x7f8d240d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:48.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.541+0000 7f8d2c790640 1 -- 172.21.15.83:0/1528226777 shutdown_connections 2024-04-17T10:42:48.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.541+0000 7f8d2c790640 1 -- 172.21.15.83:0/1528226777 wait complete. 2024-04-17T10:42:48.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.541+0000 7f8d2c790640 1 Processor -- start 2024-04-17T10:42:48.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.541+0000 7f8d2c790640 1 -- start start 2024-04-17T10:42:48.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.545+0000 7f8d2c790640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d10003a60 0x7f8d24153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:48.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.545+0000 7f8d2c790640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d240c96e0 0x7f8d240c6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:48.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.545+0000 7f8d2c790640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d240c9b60 0x7f8d240c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:48.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.545+0000 7f8d2c790640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d240d2140 con 0x7f8d240c96e0 2024-04-17T10:42:48.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.545+0000 7f8d2c790640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d240d1fc0 con 0x7f8d10003a60 2024-04-17T10:42:48.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.545+0000 7f8d2c790640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2416e0f0 con 0x7f8d240c9b60 2024-04-17T10:42:48.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.545+0000 7f8d29d2b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d240c96e0 0x7f8d240c6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:48.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.545+0000 7f8d29d2b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d240c96e0 0x7f8d240c6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34688/0 (socket says 172.21.15.83:34688) 2024-04-17T10:42:48.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.545+0000 7f8d29d2b640 1 -- 172.21.15.83:0/2708306695 learned_addr learned my addr 172.21.15.83:0/2708306695 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:48.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.545+0000 7f8d2ad2d640 1 --2- 172.21.15.83:0/2708306695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d240c9b60 0x7f8d240c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:48.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.545+0000 7f8d2a52c640 1 --2- 172.21.15.83:0/2708306695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d10003a60 0x7f8d24153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:48.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.545+0000 7f8d29d2b640 1 -- 172.21.15.83:0/2708306695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d240c9b60 msgr2=0x7f8d240c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:48.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.545+0000 7f8d29d2b640 1 --2- 172.21.15.83:0/2708306695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d240c9b60 0x7f8d240c6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:48.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.545+0000 7f8d29d2b640 1 -- 172.21.15.83:0/2708306695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d10003a60 msgr2=0x7f8d24153160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:48.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.545+0000 7f8d29d2b640 1 --2- 172.21.15.83:0/2708306695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d10003a60 0x7f8d24153160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:48.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.545+0000 7f8d29d2b640 1 -- 172.21.15.83:0/2708306695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2406e300 con 0x7f8d240c96e0 2024-04-17T10:42:48.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.545+0000 7f8d2ad2d640 1 --2- 172.21.15.83:0/2708306695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d240c9b60 0x7f8d240c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:42:48.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.545+0000 7f8d2a52c640 1 --2- 172.21.15.83:0/2708306695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d10003a60 0x7f8d24153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:48.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.545+0000 7f8d29d2b640 1 --2- 172.21.15.83:0/2708306695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d240c96e0 0x7f8d240c6880 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f8d1400d5a0 tx=0x7f8d1400da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:48.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.545+0000 7f8d1affd640 1 -- 172.21.15.83:0/2708306695 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d14005480 con 0x7f8d240c96e0 2024-04-17T10:42:48.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.545+0000 7f8d1affd640 1 -- 172.21.15.83:0/2708306695 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d14017070 con 0x7f8d240c96e0 2024-04-17T10:42:48.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.545+0000 7f8d2c790640 1 -- 172.21.15.83:0/2708306695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d240c8680 con 0x7f8d240c96e0 2024-04-17T10:42:48.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.545+0000 7f8d1affd640 1 -- 172.21.15.83:0/2708306695 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d14003840 con 0x7f8d240c96e0 2024-04-17T10:42:48.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.545+0000 7f8d2c790640 1 -- 172.21.15.83:0/2708306695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d240c8c00 con 0x7f8d240c96e0 2024-04-17T10:42:48.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.545+0000 7f8d2c790640 1 -- 172.21.15.83:0/2708306695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d240681a0 con 0x7f8d240c96e0 2024-04-17T10:42:48.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.545+0000 7f8d1affd640 1 -- 172.21.15.83:0/2708306695 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8d140039e0 con 0x7f8d240c96e0 2024-04-17T10:42:48.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.545+0000 7f8d1affd640 1 --2- 172.21.15.83:0/2708306695 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8d08066f60 0x7f8d08069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:48.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.545+0000 7f8d1affd640 1 -- 172.21.15.83:0/2708306695 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d14084d50 con 0x7f8d240c96e0 2024-04-17T10:42:48.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.545+0000 7f8d2a52c640 1 --2- 172.21.15.83:0/2708306695 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8d08066f60 0x7f8d08069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:48.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.545+0000 7f8d2a52c640 1 --2- 172.21.15.83:0/2708306695 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8d08066f60 0x7f8d08069420 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8d200ab000 tx=0x7f8d200a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:48.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.549+0000 7f8d1affd640 1 -- 172.21.15.83:0/2708306695 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8d2406e300 con 0x7f8d240c96e0 2024-04-17T10:42:48.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.709+0000 7f8d2c790640 1 -- 172.21.15.83:0/2708306695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8d24074f90 con 0x7f8d240c96e0 2024-04-17T10:42:48.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.709+0000 7f8d1affd640 1 -- 172.21.15.83:0/2708306695 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8d14019310 con 0x7f8d240c96e0 2024-04-17T10:42:48.711 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:42:48.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.713+0000 7f8d2c790640 1 -- 172.21.15.83:0/2708306695 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8d08066f60 msgr2=0x7f8d08069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:48.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.713+0000 7f8d2c790640 1 --2- 172.21.15.83:0/2708306695 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8d08066f60 0x7f8d08069420 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8d200ab000 tx=0x7f8d200a7070 comp rx=0 tx=0).stop 2024-04-17T10:42:48.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.713+0000 7f8d2c790640 1 -- 172.21.15.83:0/2708306695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d240c96e0 msgr2=0x7f8d240c6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:48.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.713+0000 7f8d2c790640 1 --2- 172.21.15.83:0/2708306695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d240c96e0 0x7f8d240c6880 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f8d1400d5a0 tx=0x7f8d1400da60 comp rx=0 tx=0).stop 2024-04-17T10:42:48.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.713+0000 7f8d2c790640 1 -- 172.21.15.83:0/2708306695 shutdown_connections 2024-04-17T10:42:48.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.713+0000 7f8d2c790640 1 --2- 172.21.15.83:0/2708306695 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8d08066f60 0x7f8d08069420 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:48.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.713+0000 7f8d2c790640 1 --2- 172.21.15.83:0/2708306695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d240c9b60 0x7f8d240c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:48.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.713+0000 7f8d2c790640 1 --2- 172.21.15.83:0/2708306695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d240c96e0 0x7f8d240c6880 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:48.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.713+0000 7f8d2c790640 1 --2- 172.21.15.83:0/2708306695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d10003a60 0x7f8d24153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:48.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.713+0000 7f8d2c790640 1 -- 172.21.15.83:0/2708306695 >> 172.21.15.83:0/2708306695 conn(0x7f8d240d0450 msgr2=0x7f8d241643f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:48.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.713+0000 7f8d2c790640 1 -- 172.21.15.83:0/2708306695 shutdown_connections 2024-04-17T10:42:48.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.713+0000 7f8d2c790640 1 -- 172.21.15.83:0/2708306695 wait complete. 2024-04-17T10:42:48.730 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:42:48.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.853+0000 7f8830c28640 1 Processor -- start 2024-04-17T10:42:48.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.853+0000 7f8830c28640 1 -- start start 2024-04-17T10:42:48.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.853+0000 7f8830c28640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f882c15b6e0 0x7f882c071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:48.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.853+0000 7f8830c28640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f882c070470 0x7f882c06ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:48.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.853+0000 7f8830c28640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f882c06c1f0 con 0x7f882c15bac0 2024-04-17T10:42:48.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.853+0000 7f8830c28640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f882c070a10 con 0x7f882c070470 2024-04-17T10:42:48.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.853+0000 7f8830c28640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f882c070cf0 con 0x7f882c15b6e0 2024-04-17T10:42:48.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.853+0000 7f8829d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f882c15bac0 0x7f882c0cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52800/0 (socket says 172.21.15.83:52800) 2024-04-17T10:42:48.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.853+0000 7f8829d9b640 1 -- 172.21.15.83:0/3839318869 learned_addr learned my addr 172.21.15.83:0/3839318869 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:48.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.853+0000 7f882ad9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f882c070470 0x7f882c06ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:48.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.853+0000 7f882a59c640 1 --2- 172.21.15.83:0/3839318869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f882c15b6e0 0x7f882c071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:48.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.853+0000 7f882959a640 1 -- 172.21.15.83:0/3839318869 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3352422281 0 0) 0x7f882c06c1f0 con 0x7f882c15bac0 2024-04-17T10:42:48.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.853+0000 7f882959a640 1 -- 172.21.15.83:0/3839318869 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8814003530 con 0x7f882c15bac0 2024-04-17T10:42:48.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.853+0000 7f882ad9d640 1 -- 172.21.15.83:0/3839318869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f882c15b6e0 msgr2=0x7f882c071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:48.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.853+0000 7f882ad9d640 1 --2- 172.21.15.83:0/3839318869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f882c15b6e0 0x7f882c071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:48.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.853+0000 7f882ad9d640 1 -- 172.21.15.83:0/3839318869 >> v1:172.21.15.83:6789/0 conn(0x7f882c15bac0 legacy=0x7f882c0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:48.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.853+0000 7f882ad9d640 1 -- 172.21.15.83:0/3839318869 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f882c070fd0 con 0x7f882c070470 2024-04-17T10:42:48.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.857+0000 7f882ad9d640 1 --2- 172.21.15.83:0/3839318869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f882c070470 0x7f882c06ae10 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f8824009d90 tx=0x7f882401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf220e69ad905e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:48.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.857+0000 7f882959a640 1 -- 172.21.15.83:0/3839318869 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88240b2070 con 0x7f882c070470 2024-04-17T10:42:48.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.857+0000 7f882959a640 1 -- 172.21.15.83:0/3839318869 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88240a5830 con 0x7f882c070470 2024-04-17T10:42:48.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.857+0000 7f882959a640 1 -- 172.21.15.83:0/3839318869 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88240a5c00 con 0x7f882c070470 2024-04-17T10:42:48.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.857+0000 7f8830c28640 1 -- 172.21.15.83:0/3839318869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f882c070470 msgr2=0x7f882c06ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:48.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.857+0000 7f8830c28640 1 --2- 172.21.15.83:0/3839318869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f882c070470 0x7f882c06ae10 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f8824009d90 tx=0x7f882401c970 comp rx=0 tx=0).stop 2024-04-17T10:42:48.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.857+0000 7f8830c28640 1 -- 172.21.15.83:0/3839318869 shutdown_connections 2024-04-17T10:42:48.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.857+0000 7f8830c28640 1 --2- 172.21.15.83:0/3839318869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f882c070470 0x7f882c06ae10 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:48.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.857+0000 7f8830c28640 1 --2- 172.21.15.83:0/3839318869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f882c15b6e0 0x7f882c071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:48.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.857+0000 7f8830c28640 1 -- 172.21.15.83:0/3839318869 >> 172.21.15.83:0/3839318869 conn(0x7f882c0b15b0 msgr2=0x7f882c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:48.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.857+0000 7f8830c28640 1 -- 172.21.15.83:0/3839318869 shutdown_connections 2024-04-17T10:42:48.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.857+0000 7f8830c28640 1 -- 172.21.15.83:0/3839318869 wait complete. 2024-04-17T10:42:48.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.857+0000 7f8830c28640 1 Processor -- start 2024-04-17T10:42:48.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.857+0000 7f8830c28640 1 -- start start 2024-04-17T10:42:48.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.861+0000 7f8830c28640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f882c070470 0x7f882c14d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:48.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.861+0000 7f8830c28640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f882c15b6e0 0x7f882c15dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:48.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.861+0000 7f8830c28640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f882c15bac0 0x7f882c15e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:48.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.861+0000 7f8830c28640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882c06d270 con 0x7f882c070470 2024-04-17T10:42:48.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.861+0000 7f8830c28640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882c06d0f0 con 0x7f882c15bac0 2024-04-17T10:42:48.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.861+0000 7f8830c28640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f882c06b760 con 0x7f882c15b6e0 2024-04-17T10:42:48.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.861+0000 7f882a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f882c070470 0x7f882c14d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:48.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.861+0000 7f882a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f882c070470 0x7f882c14d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34706/0 (socket says 172.21.15.83:34706) 2024-04-17T10:42:48.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.861+0000 7f882a59c640 1 -- 172.21.15.83:0/1521614635 learned_addr learned my addr 172.21.15.83:0/1521614635 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:48.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.861+0000 7f8829d9b640 1 --2- 172.21.15.83:0/1521614635 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f882c15b6e0 0x7f882c15dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:48.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.861+0000 7f882a59c640 1 -- 172.21.15.83:0/1521614635 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f882c15b6e0 msgr2=0x7f882c15dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:48.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.861+0000 7f882a59c640 1 --2- 172.21.15.83:0/1521614635 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f882c15b6e0 0x7f882c15dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:48.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.861+0000 7f882a59c640 1 -- 172.21.15.83:0/1521614635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f882c15bac0 msgr2=0x7f882c15e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:48.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.861+0000 7f882ad9d640 1 --2- 172.21.15.83:0/1521614635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f882c15bac0 0x7f882c15e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:48.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.861+0000 7f882a59c640 1 --2- 172.21.15.83:0/1521614635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f882c15bac0 0x7f882c15e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:48.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.861+0000 7f882a59c640 1 -- 172.21.15.83:0/1521614635 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f882c15cd70 con 0x7f882c070470 2024-04-17T10:42:48.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.861+0000 7f8829d9b640 1 --2- 172.21.15.83:0/1521614635 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f882c15b6e0 0x7f882c15dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:48.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.861+0000 7f882ad9d640 1 --2- 172.21.15.83:0/1521614635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f882c15bac0 0x7f882c15e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:48.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.861+0000 7f882a59c640 1 --2- 172.21.15.83:0/1521614635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f882c070470 0x7f882c14d610 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f881c00e960 tx=0x7f881c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:48.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.861+0000 7f881affd640 1 -- 172.21.15.83:0/1521614635 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881c017070 con 0x7f882c070470 2024-04-17T10:42:48.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.861+0000 7f881affd640 1 -- 172.21.15.83:0/1521614635 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f881c00ca00 con 0x7f882c070470 2024-04-17T10:42:48.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.861+0000 7f8830c28640 1 -- 172.21.15.83:0/1521614635 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f882c15d010 con 0x7f882c070470 2024-04-17T10:42:48.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.861+0000 7f881affd640 1 -- 172.21.15.83:0/1521614635 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881c00cdb0 con 0x7f882c070470 2024-04-17T10:42:48.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.861+0000 7f8830c28640 1 -- 172.21.15.83:0/1521614635 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f882c15d590 con 0x7f882c070470 2024-04-17T10:42:48.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.861+0000 7f8830c28640 1 -- 172.21.15.83:0/1521614635 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f882c074f90 con 0x7f882c070470 2024-04-17T10:42:48.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.861+0000 7f881affd640 1 -- 172.21.15.83:0/1521614635 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f881c01a070 con 0x7f882c070470 2024-04-17T10:42:48.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.861+0000 7f881affd640 1 --2- 172.21.15.83:0/1521614635 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f880c066f60 0x7f880c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:48.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.861+0000 7f881affd640 1 -- 172.21.15.83:0/1521614635 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f881c0850f0 con 0x7f882c070470 2024-04-17T10:42:48.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.861+0000 7f8829d9b640 1 --2- 172.21.15.83:0/1521614635 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f880c066f60 0x7f880c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:48.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.861+0000 7f8829d9b640 1 --2- 172.21.15.83:0/1521614635 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f880c066f60 0x7f880c069420 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f882c14bc80 tx=0x7f8820009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:48.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:48.865+0000 7f881affd640 1 -- 172.21.15.83:0/1521614635 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f881c056100 con 0x7f882c070470 2024-04-17T10:42:49.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.025+0000 7f8830c28640 1 -- 172.21.15.83:0/1521614635 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f882c074480 con 0x7f882c070470 2024-04-17T10:42:49.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.025+0000 7f881affd640 1 -- 172.21.15.83:0/1521614635 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v59) v1 ==== 104+0+11 (secure 0 0 0) 0x7f881c055a00 con 0x7f882c070470 2024-04-17T10:42:49.024 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:42:49.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.025+0000 7f8830c28640 1 -- 172.21.15.83:0/1521614635 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f880c066f60 msgr2=0x7f880c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:49.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.025+0000 7f8830c28640 1 --2- 172.21.15.83:0/1521614635 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f880c066f60 0x7f880c069420 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f882c14bc80 tx=0x7f8820009040 comp rx=0 tx=0).stop 2024-04-17T10:42:49.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.025+0000 7f8830c28640 1 -- 172.21.15.83:0/1521614635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f882c070470 msgr2=0x7f882c14d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:49.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.025+0000 7f8830c28640 1 --2- 172.21.15.83:0/1521614635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f882c070470 0x7f882c14d610 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f881c00e960 tx=0x7f881c00ee20 comp rx=0 tx=0).stop 2024-04-17T10:42:49.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.025+0000 7f8830c28640 1 -- 172.21.15.83:0/1521614635 shutdown_connections 2024-04-17T10:42:49.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.025+0000 7f8830c28640 1 --2- 172.21.15.83:0/1521614635 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f880c066f60 0x7f880c069420 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:49.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.025+0000 7f8830c28640 1 --2- 172.21.15.83:0/1521614635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f882c15bac0 0x7f882c15e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:49.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.025+0000 7f8830c28640 1 --2- 172.21.15.83:0/1521614635 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f882c15b6e0 0x7f882c15dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:49.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.025+0000 7f8830c28640 1 --2- 172.21.15.83:0/1521614635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f882c070470 0x7f882c14d610 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:49.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.025+0000 7f8830c28640 1 -- 172.21.15.83:0/1521614635 >> 172.21.15.83:0/1521614635 conn(0x7f882c0b15b0 msgr2=0x7f882c0cdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:49.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.025+0000 7f8830c28640 1 -- 172.21.15.83:0/1521614635 shutdown_connections 2024-04-17T10:42:49.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.025+0000 7f8830c28640 1 -- 172.21.15.83:0/1521614635 wait complete. 2024-04-17T10:42:49.043 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:42:49.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.169+0000 7f6dfaa45640 1 Processor -- start 2024-04-17T10:42:49.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.169+0000 7f6dfaa45640 1 -- start start 2024-04-17T10:42:49.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.173+0000 7f6dfaa45640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6df415e0f0 0x7f6df40b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:49.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.173+0000 7f6dfaa45640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6df40b5450 0x7f6df40b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:49.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.173+0000 7f6dfaa45640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6df415e4d0 con 0x7f6df40ce400 2024-04-17T10:42:49.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.173+0000 7f6dfaa45640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df4162b40 con 0x7f6df40b5450 2024-04-17T10:42:49.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.173+0000 7f6dfaa45640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df4162320 con 0x7f6df415e0f0 2024-04-17T10:42:49.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.173+0000 7f6df8fe2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6df40b5450 0x7f6df40b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:49.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.173+0000 7f6df8fe2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6df40b5450 0x7f6df40b58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34712/0 (socket says 172.21.15.83:34712) 2024-04-17T10:42:49.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.173+0000 7f6df8fe2640 1 -- 172.21.15.83:0/1603419110 learned_addr learned my addr 172.21.15.83:0/1603419110 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:49.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.173+0000 7f6df3fff640 1 --2- 172.21.15.83:0/1603419110 >> v2:172.21.15.167:3300/0 conn(0x7f6df415e0f0 0x7f6df40b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:49.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.173+0000 7f6df8fe2640 1 -- 172.21.15.83:0/1603419110 >> v2:172.21.15.167:3300/0 conn(0x7f6df415e0f0 msgr2=0x7f6df40b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:49.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.173+0000 7f6df8fe2640 1 --2- 172.21.15.83:0/1603419110 >> v2:172.21.15.167:3300/0 conn(0x7f6df415e0f0 0x7f6df40b6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:49.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.173+0000 7f6df8fe2640 1 -- 172.21.15.83:0/1603419110 >> v1:172.21.15.167:6789/0 conn(0x7f6df40ce400 legacy=0x7f6df4161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:49.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.173+0000 7f6df8fe2640 1 -- 172.21.15.83:0/1603419110 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df4162580 con 0x7f6df40b5450 2024-04-17T10:42:49.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.173+0000 7f6df8fe2640 1 --2- 172.21.15.83:0/1603419110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6df40b5450 0x7f6df40b58e0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f6df408c3f0 tx=0x7f6dec01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a70a681d9c4be98f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:49.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.173+0000 7f6df2ffd640 1 -- 172.21.15.83:0/1603419110 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec0a5620 con 0x7f6df40b5450 2024-04-17T10:42:49.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.173+0000 7f6df2ffd640 1 -- 172.21.15.83:0/1603419110 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dec0a57c0 con 0x7f6df40b5450 2024-04-17T10:42:49.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.173+0000 7f6df2ffd640 1 -- 172.21.15.83:0/1603419110 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec0a5b90 con 0x7f6df40b5450 2024-04-17T10:42:49.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6dfaa45640 1 -- 172.21.15.83:0/1603419110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6df40b5450 msgr2=0x7f6df40b58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:49.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6dfaa45640 1 --2- 172.21.15.83:0/1603419110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6df40b5450 0x7f6df40b58e0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f6df408c3f0 tx=0x7f6dec01c960 comp rx=0 tx=0).stop 2024-04-17T10:42:49.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6dfaa45640 1 -- 172.21.15.83:0/1603419110 shutdown_connections 2024-04-17T10:42:49.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6dfaa45640 1 --2- 172.21.15.83:0/1603419110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6df40b5450 0x7f6df40b58e0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:49.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6dfaa45640 1 --2- 172.21.15.83:0/1603419110 >> v2:172.21.15.167:3300/0 conn(0x7f6df415e0f0 0x7f6df40b6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:49.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6dfaa45640 1 -- 172.21.15.83:0/1603419110 >> 172.21.15.83:0/1603419110 conn(0x7f6df40d0450 msgr2=0x7f6df40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:49.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6dfaa45640 1 -- 172.21.15.83:0/1603419110 shutdown_connections 2024-04-17T10:42:49.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6dfaa45640 1 -- 172.21.15.83:0/1603419110 wait complete. 2024-04-17T10:42:49.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6dfaa45640 1 Processor -- start 2024-04-17T10:42:49.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6dfaa45640 1 -- start start 2024-04-17T10:42:49.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6dfaa45640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6df40b5450 0x7f6df414d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:49.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6dfaa45640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6df40ce400 0x7f6df4152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:49.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6dfaa45640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6df415e0f0 0x7f6df4159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:49.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6dfaa45640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df40d2ed0 con 0x7f6df40ce400 2024-04-17T10:42:49.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6dfaa45640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df40d2d50 con 0x7f6df40b5450 2024-04-17T10:42:49.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6dfaa45640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df40d3050 con 0x7f6df415e0f0 2024-04-17T10:42:49.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6df37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6df40ce400 0x7f6df4152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:49.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6df37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6df40ce400 0x7f6df4152160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34716/0 (socket says 172.21.15.83:34716) 2024-04-17T10:42:49.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6df37fe640 1 -- 172.21.15.83:0/2010015000 learned_addr learned my addr 172.21.15.83:0/2010015000 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:49.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6df3fff640 1 --2- 172.21.15.83:0/2010015000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6df40b5450 0x7f6df414d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:49.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6df8fe2640 1 --2- 172.21.15.83:0/2010015000 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6df415e0f0 0x7f6df4159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:49.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6df37fe640 1 -- 172.21.15.83:0/2010015000 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6df415e0f0 msgr2=0x7f6df4159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:49.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6df37fe640 1 --2- 172.21.15.83:0/2010015000 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6df415e0f0 0x7f6df4159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:49.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6df37fe640 1 -- 172.21.15.83:0/2010015000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6df40b5450 msgr2=0x7f6df414d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:49.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6df37fe640 1 --2- 172.21.15.83:0/2010015000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6df40b5450 0x7f6df414d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:49.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6df37fe640 1 -- 172.21.15.83:0/2010015000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df414e9f0 con 0x7f6df40ce400 2024-04-17T10:42:49.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6df8fe2640 1 --2- 172.21.15.83:0/2010015000 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6df415e0f0 0x7f6df4159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:49.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6df3fff640 1 --2- 172.21.15.83:0/2010015000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6df40b5450 0x7f6df414d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:49.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6df37fe640 1 --2- 172.21.15.83:0/2010015000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6df40ce400 0x7f6df4152160 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f6de800c430 tx=0x7f6de800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:49.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6df0ff9640 1 -- 172.21.15.83:0/2010015000 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de80052d0 con 0x7f6df40ce400 2024-04-17T10:42:49.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6df0ff9640 1 -- 172.21.15.83:0/2010015000 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de8005470 con 0x7f6df40ce400 2024-04-17T10:42:49.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6dfaa45640 1 -- 172.21.15.83:0/2010015000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df4153e40 con 0x7f6df40ce400 2024-04-17T10:42:49.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6df0ff9640 1 -- 172.21.15.83:0/2010015000 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de8007260 con 0x7f6df40ce400 2024-04-17T10:42:49.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.177+0000 7f6dfaa45640 1 -- 172.21.15.83:0/2010015000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df41543a0 con 0x7f6df40ce400 2024-04-17T10:42:49.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.181+0000 7f6dfaa45640 1 -- 172.21.15.83:0/2010015000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df40681a0 con 0x7f6df40ce400 2024-04-17T10:42:49.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.181+0000 7f6df0ff9640 1 -- 172.21.15.83:0/2010015000 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6de8007400 con 0x7f6df40ce400 2024-04-17T10:42:49.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.181+0000 7f6df0ff9640 1 --2- 172.21.15.83:0/2010015000 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6de0066f60 0x7f6de0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:49.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.181+0000 7f6df0ff9640 1 -- 172.21.15.83:0/2010015000 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6de8084540 con 0x7f6df40ce400 2024-04-17T10:42:49.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.181+0000 7f6df3fff640 1 --2- 172.21.15.83:0/2010015000 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6de0066f60 0x7f6de0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:49.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.181+0000 7f6df3fff640 1 --2- 172.21.15.83:0/2010015000 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6de0066f60 0x7f6de0069420 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6df415a1f0 tx=0x7f6de4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:49.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.185+0000 7f6df0ff9640 1 -- 172.21.15.83:0/2010015000 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6df4153e40 con 0x7f6df40ce400 2024-04-17T10:42:49.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.341+0000 7f6dfaa45640 1 -- 172.21.15.83:0/2010015000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6df4074f90 con 0x7f6df40ce400 2024-04-17T10:42:49.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.341+0000 7f6df0ff9640 1 -- 172.21.15.83:0/2010015000 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v59) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6de8055550 con 0x7f6df40ce400 2024-04-17T10:42:49.343 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:42:49.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.345+0000 7f6dfaa45640 1 -- 172.21.15.83:0/2010015000 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6de0066f60 msgr2=0x7f6de0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:49.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.345+0000 7f6dfaa45640 1 --2- 172.21.15.83:0/2010015000 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6de0066f60 0x7f6de0069420 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6df415a1f0 tx=0x7f6de4008040 comp rx=0 tx=0).stop 2024-04-17T10:42:49.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.345+0000 7f6dfaa45640 1 -- 172.21.15.83:0/2010015000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6df40ce400 msgr2=0x7f6df4152160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:49.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.345+0000 7f6dfaa45640 1 --2- 172.21.15.83:0/2010015000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6df40ce400 0x7f6df4152160 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f6de800c430 tx=0x7f6de800c8f0 comp rx=0 tx=0).stop 2024-04-17T10:42:49.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.345+0000 7f6dfaa45640 1 -- 172.21.15.83:0/2010015000 shutdown_connections 2024-04-17T10:42:49.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.345+0000 7f6dfaa45640 1 --2- 172.21.15.83:0/2010015000 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6de0066f60 0x7f6de0069420 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:49.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.345+0000 7f6dfaa45640 1 --2- 172.21.15.83:0/2010015000 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6df415e0f0 0x7f6df4159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:49.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.345+0000 7f6dfaa45640 1 --2- 172.21.15.83:0/2010015000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6df40ce400 0x7f6df4152160 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:49.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.345+0000 7f6dfaa45640 1 --2- 172.21.15.83:0/2010015000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6df40b5450 0x7f6df414d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:49.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.345+0000 7f6dfaa45640 1 -- 172.21.15.83:0/2010015000 >> 172.21.15.83:0/2010015000 conn(0x7f6df40d0450 msgr2=0x7f6df40cf140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:49.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.345+0000 7f6dfaa45640 1 -- 172.21.15.83:0/2010015000 shutdown_connections 2024-04-17T10:42:49.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.345+0000 7f6dfaa45640 1 -- 172.21.15.83:0/2010015000 wait complete. 2024-04-17T10:42:49.363 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:42:49.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.481+0000 7f5b9d4b3640 1 Processor -- start 2024-04-17T10:42:49.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.481+0000 7f5b9d4b3640 1 -- start start 2024-04-17T10:42:49.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.481+0000 7f5b9d4b3640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5b9815bc10 0x7f5b980d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:49.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.481+0000 7f5b9d4b3640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b9806e500 0x7f5b9806e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:49.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.481+0000 7f5b9d4b3640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b980ced00 con 0x7f5b980d0490 2024-04-17T10:42:49.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.485+0000 7f5b9d4b3640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b980cefe0 con 0x7f5b9815bc10 2024-04-17T10:42:49.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.485+0000 7f5b9d4b3640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b980cc690 con 0x7f5b9806e500 2024-04-17T10:42:49.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.485+0000 7f5b977fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b9806e500 0x7f5b9806e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:49.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.485+0000 7f5b96ffd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5b9815bc10 0x7f5b980d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:49.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.485+0000 7f5b977fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b9806e500 0x7f5b9806e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:36374/0 (socket says 172.21.15.83:36374) 2024-04-17T10:42:49.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.485+0000 7f5b977fe640 1 -- 172.21.15.83:0/529655169 learned_addr learned my addr 172.21.15.83:0/529655169 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:49.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.485+0000 7f5b977fe640 1 -- 172.21.15.83:0/529655169 >> v2:172.21.15.167:3300/0 conn(0x7f5b9815bc10 msgr2=0x7f5b980d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:49.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.485+0000 7f5b977fe640 1 --2- 172.21.15.83:0/529655169 >> v2:172.21.15.167:3300/0 conn(0x7f5b9815bc10 0x7f5b980d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:49.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.485+0000 7f5b977fe640 1 -- 172.21.15.83:0/529655169 >> v1:172.21.15.167:6789/0 conn(0x7f5b980d0490 legacy=0x7f5b980cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:49.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.485+0000 7f5b977fe640 1 -- 172.21.15.83:0/529655169 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b980cc970 con 0x7f5b9806e500 2024-04-17T10:42:49.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.485+0000 7f5b977fe640 1 --2- 172.21.15.83:0/529655169 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b9806e500 0x7f5b9806e990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5b90009e50 tx=0x7f5b9001c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7c89505a440a3fa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:49.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.485+0000 7f5b967fc640 1 -- 172.21.15.83:0/529655169 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b900a5660 con 0x7f5b9806e500 2024-04-17T10:42:49.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.485+0000 7f5b967fc640 1 -- 172.21.15.83:0/529655169 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b900a5800 con 0x7f5b9806e500 2024-04-17T10:42:49.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.485+0000 7f5b967fc640 1 -- 172.21.15.83:0/529655169 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b900a5bd0 con 0x7f5b9806e500 2024-04-17T10:42:49.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.489+0000 7f5b9d4b3640 1 -- 172.21.15.83:0/529655169 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b9806e500 msgr2=0x7f5b9806e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:49.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.489+0000 7f5b9d4b3640 1 --2- 172.21.15.83:0/529655169 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b9806e500 0x7f5b9806e990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5b90009e50 tx=0x7f5b9001c9a0 comp rx=0 tx=0).stop 2024-04-17T10:42:49.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.489+0000 7f5b9d4b3640 1 -- 172.21.15.83:0/529655169 shutdown_connections 2024-04-17T10:42:49.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.489+0000 7f5b9d4b3640 1 --2- 172.21.15.83:0/529655169 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b9806e500 0x7f5b9806e990 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:49.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.489+0000 7f5b9d4b3640 1 --2- 172.21.15.83:0/529655169 >> v2:172.21.15.167:3300/0 conn(0x7f5b9815bc10 0x7f5b980d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:49.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.489+0000 7f5b9d4b3640 1 -- 172.21.15.83:0/529655169 >> 172.21.15.83:0/529655169 conn(0x7f5b980d17e0 msgr2=0x7f5b980d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:49.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.489+0000 7f5b9d4b3640 1 -- 172.21.15.83:0/529655169 shutdown_connections 2024-04-17T10:42:49.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.489+0000 7f5b9d4b3640 1 -- 172.21.15.83:0/529655169 wait complete. 2024-04-17T10:42:49.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.489+0000 7f5b9d4b3640 1 Processor -- start 2024-04-17T10:42:49.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.489+0000 7f5b9d4b3640 1 -- start start 2024-04-17T10:42:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.489+0000 7f5b9d4b3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b9806e500 0x7f5b980b6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.489+0000 7f5b9d4b3640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b980d0490 0x7f5b981534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.489+0000 7f5b9d4b3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b9815bc10 0x7f5b9815aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.489+0000 7f5b9d4b3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b98168b10 con 0x7f5b9806e500 2024-04-17T10:42:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.489+0000 7f5b9d4b3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b98168990 con 0x7f5b9815bc10 2024-04-17T10:42:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.489+0000 7f5b9d4b3640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b98168c90 con 0x7f5b980d0490 2024-04-17T10:42:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.489+0000 7f5b96ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b9806e500 0x7f5b980b6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.489+0000 7f5b96ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b9806e500 0x7f5b980b6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34732/0 (socket says 172.21.15.83:34732) 2024-04-17T10:42:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.489+0000 7f5b96ffd640 1 -- 172.21.15.83:0/2390791692 learned_addr learned my addr 172.21.15.83:0/2390791692 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.489+0000 7f5b8e7fc640 1 --2- 172.21.15.83:0/2390791692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b980d0490 0x7f5b981534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.489+0000 7f5b977fe640 1 --2- 172.21.15.83:0/2390791692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b9815bc10 0x7f5b9815aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.489+0000 7f5b96ffd640 1 -- 172.21.15.83:0/2390791692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b980d0490 msgr2=0x7f5b981534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:49.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.489+0000 7f5b96ffd640 1 --2- 172.21.15.83:0/2390791692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b980d0490 0x7f5b981534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:49.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.489+0000 7f5b96ffd640 1 -- 172.21.15.83:0/2390791692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b9815bc10 msgr2=0x7f5b9815aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:49.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.489+0000 7f5b96ffd640 1 --2- 172.21.15.83:0/2390791692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b9815bc10 0x7f5b9815aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:49.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.489+0000 7f5b96ffd640 1 -- 172.21.15.83:0/2390791692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9815b530 con 0x7f5b9806e500 2024-04-17T10:42:49.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.489+0000 7f5b96ffd640 1 --2- 172.21.15.83:0/2390791692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b9806e500 0x7f5b980b6e20 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800e870 tx=0x7f5b8800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:49.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.493+0000 7f5b8ffff640 1 -- 172.21.15.83:0/2390791692 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b88017070 con 0x7f5b9806e500 2024-04-17T10:42:49.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.493+0000 7f5b8ffff640 1 -- 172.21.15.83:0/2390791692 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b8800c940 con 0x7f5b9806e500 2024-04-17T10:42:49.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.493+0000 7f5b9d4b3640 1 -- 172.21.15.83:0/2390791692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b98152410 con 0x7f5b9806e500 2024-04-17T10:42:49.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.493+0000 7f5b8ffff640 1 -- 172.21.15.83:0/2390791692 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8800ccf0 con 0x7f5b9806e500 2024-04-17T10:42:49.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.493+0000 7f5b9d4b3640 1 -- 172.21.15.83:0/2390791692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b981529d0 con 0x7f5b9806e500 2024-04-17T10:42:49.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.493+0000 7f5b9d4b3640 1 -- 172.21.15.83:0/2390791692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b981539f0 con 0x7f5b9806e500 2024-04-17T10:42:49.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.493+0000 7f5b8ffff640 1 -- 172.21.15.83:0/2390791692 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5b8801a070 con 0x7f5b9806e500 2024-04-17T10:42:49.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.493+0000 7f5b8ffff640 1 --2- 172.21.15.83:0/2390791692 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5b78066f60 0x7f5b78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:49.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.493+0000 7f5b8ffff640 1 -- 172.21.15.83:0/2390791692 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5b88085530 con 0x7f5b9806e500 2024-04-17T10:42:49.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.493+0000 7f5b8e7fc640 1 --2- 172.21.15.83:0/2390791692 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5b78066f60 0x7f5b78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:49.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.493+0000 7f5b8e7fc640 1 --2- 172.21.15.83:0/2390791692 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5b78066f60 0x7f5b78069420 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5b980cb9b0 tx=0x7f5b80009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:49.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.497+0000 7f5b8ffff640 1 -- 172.21.15.83:0/2390791692 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5b88056060 con 0x7f5b9806e500 2024-04-17T10:42:49.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.669+0000 7f5b9d4b3640 1 -- 172.21.15.83:0/2390791692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5b98074f90 con 0x7f5b9806e500 2024-04-17T10:42:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.669+0000 7f5b8ffff640 1 -- 172.21.15.83:0/2390791692 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v59) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5b8805a100 con 0x7f5b9806e500 2024-04-17T10:42:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-17T10:42:49.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.673+0000 7f5b9d4b3640 1 -- 172.21.15.83:0/2390791692 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5b78066f60 msgr2=0x7f5b78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:49.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.673+0000 7f5b9d4b3640 1 --2- 172.21.15.83:0/2390791692 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5b78066f60 0x7f5b78069420 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5b980cb9b0 tx=0x7f5b80009040 comp rx=0 tx=0).stop 2024-04-17T10:42:49.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.673+0000 7f5b9d4b3640 1 -- 172.21.15.83:0/2390791692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b9806e500 msgr2=0x7f5b980b6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:49.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.673+0000 7f5b9d4b3640 1 --2- 172.21.15.83:0/2390791692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b9806e500 0x7f5b980b6e20 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800e870 tx=0x7f5b8800ec70 comp rx=0 tx=0).stop 2024-04-17T10:42:49.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.673+0000 7f5b9d4b3640 1 -- 172.21.15.83:0/2390791692 shutdown_connections 2024-04-17T10:42:49.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.673+0000 7f5b9d4b3640 1 --2- 172.21.15.83:0/2390791692 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5b78066f60 0x7f5b78069420 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:49.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.673+0000 7f5b9d4b3640 1 --2- 172.21.15.83:0/2390791692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b9815bc10 0x7f5b9815aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:49.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.673+0000 7f5b9d4b3640 1 --2- 172.21.15.83:0/2390791692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b980d0490 0x7f5b981534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:49.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.673+0000 7f5b9d4b3640 1 --2- 172.21.15.83:0/2390791692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b9806e500 0x7f5b980b6e20 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:49.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.673+0000 7f5b9d4b3640 1 -- 172.21.15.83:0/2390791692 >> 172.21.15.83:0/2390791692 conn(0x7f5b980d17e0 msgr2=0x7f5b980b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:49.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.673+0000 7f5b9d4b3640 1 -- 172.21.15.83:0/2390791692 shutdown_connections 2024-04-17T10:42:49.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.673+0000 7f5b9d4b3640 1 -- 172.21.15.83:0/2390791692 wait complete. 2024-04-17T10:42:49.690 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:42:49.691 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:42:49.696 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:42:49.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.861+0000 7fef0087c640 1 Processor -- start 2024-04-17T10:42:49.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.861+0000 7fef0087c640 1 -- start start 2024-04-17T10:42:49.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.865+0000 7fef0087c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feefc15ba30 0x7feefc072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:49.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.865+0000 7fef0087c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feefc0cfbe0 0x7feefc06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:49.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.865+0000 7fef0087c640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feefc06e320 con 0x7feefc15c960 2024-04-17T10:42:49.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.865+0000 7fef0087c640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feefc071ea0 con 0x7feefc15ba30 2024-04-17T10:42:49.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.865+0000 7fef0087c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feefc06e040 con 0x7feefc0cfbe0 2024-04-17T10:42:49.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.865+0000 7feefa59c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feefc15ba30 0x7feefc072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:49.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.865+0000 7feef9d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7feefc15c960 0x7feefc0cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52802/0 (socket says 172.21.15.83:52802) 2024-04-17T10:42:49.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.865+0000 7feef9d9b640 1 -- 172.21.15.83:0/2889762668 learned_addr learned my addr 172.21.15.83:0/2889762668 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:49.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.865+0000 7feefad9d640 1 --2- 172.21.15.83:0/2889762668 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feefc0cfbe0 0x7feefc06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:49.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.865+0000 7feefa59c640 1 -- 172.21.15.83:0/2889762668 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feefc0cfbe0 msgr2=0x7feefc06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:49.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.865+0000 7feefa59c640 1 --2- 172.21.15.83:0/2889762668 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feefc0cfbe0 0x7feefc06c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:49.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.865+0000 7feefa59c640 1 -- 172.21.15.83:0/2889762668 >> v1:172.21.15.83:6789/0 conn(0x7feefc15c960 legacy=0x7feefc0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:49.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.865+0000 7feef959a640 1 -- 172.21.15.83:0/2889762668 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3789666679 0 0) 0x7feefc06e320 con 0x7feefc15c960 2024-04-17T10:42:49.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.865+0000 7feefa59c640 1 -- 172.21.15.83:0/2889762668 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc071850 con 0x7feefc15ba30 2024-04-17T10:42:49.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.865+0000 7feefa59c640 1 --2- 172.21.15.83:0/2889762668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feefc15ba30 0x7feefc072330 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7feeec0097c0 tx=0x7feeec01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c8e367367448c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:49.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.865+0000 7feef959a640 1 -- 172.21.15.83:0/2889762668 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec0b0070 con 0x7feefc15ba30 2024-04-17T10:42:49.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.865+0000 7feef959a640 1 -- 172.21.15.83:0/2889762668 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeec0ab600 con 0x7feefc15ba30 2024-04-17T10:42:49.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.865+0000 7feef959a640 1 -- 172.21.15.83:0/2889762668 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec0ab9d0 con 0x7feefc15ba30 2024-04-17T10:42:49.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.869+0000 7fef0087c640 1 -- 172.21.15.83:0/2889762668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feefc15ba30 msgr2=0x7feefc072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:49.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.869+0000 7fef0087c640 1 --2- 172.21.15.83:0/2889762668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feefc15ba30 0x7feefc072330 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7feeec0097c0 tx=0x7feeec01c410 comp rx=0 tx=0).stop 2024-04-17T10:42:49.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.869+0000 7fef0087c640 1 -- 172.21.15.83:0/2889762668 shutdown_connections 2024-04-17T10:42:49.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.869+0000 7fef0087c640 1 --2- 172.21.15.83:0/2889762668 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feefc0cfbe0 0x7feefc06c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:49.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.869+0000 7fef0087c640 1 --2- 172.21.15.83:0/2889762668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feefc15ba30 0x7feefc072330 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:49.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.869+0000 7fef0087c640 1 -- 172.21.15.83:0/2889762668 >> 172.21.15.83:0/2889762668 conn(0x7feefc06abe0 msgr2=0x7feefc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:49.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.869+0000 7fef0087c640 1 -- 172.21.15.83:0/2889762668 shutdown_connections 2024-04-17T10:42:49.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.869+0000 7fef0087c640 1 -- 172.21.15.83:0/2889762668 wait complete. 2024-04-17T10:42:49.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.869+0000 7fef0087c640 1 Processor -- start 2024-04-17T10:42:49.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.869+0000 7fef0087c640 1 -- start start 2024-04-17T10:42:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.869+0000 7fef0087c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feefc0cfbe0 0x7feefc154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.869+0000 7fef0087c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feefc15ba30 0x7feefc151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.869+0000 7fef0087c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feefc15c960 0x7feefc0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.869+0000 7fef0087c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc06de20 con 0x7feefc15c960 2024-04-17T10:42:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.869+0000 7fef0087c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc06dca0 con 0x7feefc0cfbe0 2024-04-17T10:42:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.869+0000 7fef0087c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feefc168520 con 0x7feefc15ba30 2024-04-17T10:42:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.869+0000 7feefad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feefc15c960 0x7feefc0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.869+0000 7feefad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feefc15c960 0x7feefc0ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34748/0 (socket says 172.21.15.83:34748) 2024-04-17T10:42:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.869+0000 7feefad9d640 1 -- 172.21.15.83:0/1571171373 learned_addr learned my addr 172.21.15.83:0/1571171373 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.869+0000 7feefa59c640 1 --2- 172.21.15.83:0/1571171373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feefc0cfbe0 0x7feefc154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.869+0000 7feef9d9b640 1 --2- 172.21.15.83:0/1571171373 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feefc15ba30 0x7feefc151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.869+0000 7feefad9d640 1 -- 172.21.15.83:0/1571171373 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feefc15ba30 msgr2=0x7feefc151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.869+0000 7feefad9d640 1 --2- 172.21.15.83:0/1571171373 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feefc15ba30 0x7feefc151290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.869+0000 7feefad9d640 1 -- 172.21.15.83:0/1571171373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feefc0cfbe0 msgr2=0x7feefc154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.869+0000 7feefad9d640 1 --2- 172.21.15.83:0/1571171373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feefc0cfbe0 0x7feefc154270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.869+0000 7feefad9d640 1 -- 172.21.15.83:0/1571171373 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc15a170 con 0x7feefc15c960 2024-04-17T10:42:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.869+0000 7feefad9d640 1 --2- 172.21.15.83:0/1571171373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feefc15c960 0x7feefc0ad840 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7feef4007e60 tx=0x7feef400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.869+0000 7feef2ffd640 1 -- 172.21.15.83:0/1571171373 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef4014420 con 0x7feefc15c960 2024-04-17T10:42:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.869+0000 7feef2ffd640 1 -- 172.21.15.83:0/1571171373 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef4019070 con 0x7feefc15c960 2024-04-17T10:42:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.869+0000 7fef0087c640 1 -- 172.21.15.83:0/1571171373 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feefc15a450 con 0x7feefc15c960 2024-04-17T10:42:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.869+0000 7feef2ffd640 1 -- 172.21.15.83:0/1571171373 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef4014820 con 0x7feefc15c960 2024-04-17T10:42:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.869+0000 7fef0087c640 1 -- 172.21.15.83:0/1571171373 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feefc157870 con 0x7feefc15c960 2024-04-17T10:42:49.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.873+0000 7fef0087c640 1 -- 172.21.15.83:0/1571171373 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feefc074f90 con 0x7feefc15c960 2024-04-17T10:42:49.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.873+0000 7feef2ffd640 1 -- 172.21.15.83:0/1571171373 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feef40149c0 con 0x7feefc15c960 2024-04-17T10:42:49.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.873+0000 7feef2ffd640 1 --2- 172.21.15.83:0/1571171373 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feedc066f60 0x7feedc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:49.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.873+0000 7feef2ffd640 1 -- 172.21.15.83:0/1571171373 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feef4085920 con 0x7feefc15c960 2024-04-17T10:42:49.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.873+0000 7feefa59c640 1 --2- 172.21.15.83:0/1571171373 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feedc066f60 0x7feedc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:49.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.873+0000 7feefa59c640 1 --2- 172.21.15.83:0/1571171373 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feedc066f60 0x7feedc069420 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7feeec0ad000 tx=0x7feeec0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:49.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:49.877+0000 7feef2ffd640 1 -- 172.21.15.83:0/1571171373 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feef4056930 con 0x7feefc15c960 2024-04-17T10:42:50.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.037+0000 7fef0087c640 1 -- 172.21.15.83:0/1571171373 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feefc1549c0 con 0x7feefc15c960 2024-04-17T10:42:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.037+0000 7feef2ffd640 1 -- 172.21.15.83:0/1571171373 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+16593 (secure 0 0 0) 0x7feef4056230 con 0x7feefc15c960 2024-04-17T10:42:50.039 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:50.040 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":59,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:42:35.975462+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:42:32.530700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-17T10:42:33.334905+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:42:50.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.041+0000 7fef0087c640 1 -- 172.21.15.83:0/1571171373 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feedc066f60 msgr2=0x7feedc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:50.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.041+0000 7fef0087c640 1 --2- 172.21.15.83:0/1571171373 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feedc066f60 0x7feedc069420 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7feeec0ad000 tx=0x7feeec0a6000 comp rx=0 tx=0).stop 2024-04-17T10:42:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.041+0000 7fef0087c640 1 -- 172.21.15.83:0/1571171373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feefc15c960 msgr2=0x7feefc0ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.041+0000 7fef0087c640 1 --2- 172.21.15.83:0/1571171373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feefc15c960 0x7feefc0ad840 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7feef4007e60 tx=0x7feef400e4b0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.041+0000 7fef0087c640 1 -- 172.21.15.83:0/1571171373 shutdown_connections 2024-04-17T10:42:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.041+0000 7fef0087c640 1 --2- 172.21.15.83:0/1571171373 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feedc066f60 0x7feedc069420 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.041+0000 7fef0087c640 1 --2- 172.21.15.83:0/1571171373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feefc15c960 0x7feefc0ad840 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.041+0000 7fef0087c640 1 --2- 172.21.15.83:0/1571171373 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feefc15ba30 0x7feefc151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.041+0000 7fef0087c640 1 --2- 172.21.15.83:0/1571171373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feefc0cfbe0 0x7feefc154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.041+0000 7fef0087c640 1 -- 172.21.15.83:0/1571171373 >> 172.21.15.83:0/1571171373 conn(0x7feefc06abe0 msgr2=0x7feefc15d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.041+0000 7fef0087c640 1 -- 172.21.15.83:0/1571171373 shutdown_connections 2024-04-17T10:42:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.041+0000 7fef0087c640 1 -- 172.21.15.83:0/1571171373 wait complete. 2024-04-17T10:42:50.057 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:42:32.530700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:42:33.334905+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:42:50.058 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:42:50.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd676e0e640 1 Processor -- start 2024-04-17T10:42:50.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd676e0e640 1 -- start start 2024-04-17T10:42:50.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd676e0e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd670158490 0x7fd67015ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:50.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd676e0e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd6700ceec0 0x7fd67006ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:50.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd676e0e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd67006f360 con 0x7fd6700cac80 2024-04-17T10:42:50.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd676e0e640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd67006f5e0 con 0x7fd6700ceec0 2024-04-17T10:42:50.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd676e0e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6700cf320 con 0x7fd670158490 2024-04-17T10:42:50.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd6753ab640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd6700cac80 0x7fd67006abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52818/0 (socket says 172.21.15.83:52818) 2024-04-17T10:42:50.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd6753ab640 1 -- 172.21.15.83:0/1479734150 learned_addr learned my addr 172.21.15.83:0/1479734150 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:50.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd674baa640 1 --2- 172.21.15.83:0/1479734150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd670158490 0x7fd67015ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:50.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd66ffff640 1 --2- 172.21.15.83:0/1479734150 >> v2:172.21.15.167:3300/0 conn(0x7fd6700ceec0 0x7fd67006ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:50.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd66f7fe640 1 -- 172.21.15.83:0/1479734150 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302769956 0 0) 0x7fd6680014a0 con 0x7fd6700cac80 2024-04-17T10:42:50.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd66f7fe640 1 -- 172.21.15.83:0/1479734150 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd658003530 con 0x7fd6700cac80 2024-04-17T10:42:50.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd66f7fe640 1 -- 172.21.15.83:0/1479734150 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1539609774 0 0) 0x7fd658003530 con 0x7fd6700cac80 2024-04-17T10:42:50.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd66f7fe640 1 -- 172.21.15.83:0/1479734150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd670158490 msgr2=0x7fd67015ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:50.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd66f7fe640 1 --2- 172.21.15.83:0/1479734150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd670158490 0x7fd67015ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd66f7fe640 1 -- 172.21.15.83:0/1479734150 >> v2:172.21.15.167:3300/0 conn(0x7fd6700ceec0 msgr2=0x7fd67006ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:50.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd66f7fe640 1 --2- 172.21.15.83:0/1479734150 >> v2:172.21.15.167:3300/0 conn(0x7fd6700ceec0 0x7fd67006ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd66f7fe640 1 -- 172.21.15.83:0/1479734150 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6700ce100 con 0x7fd6700cac80 2024-04-17T10:42:50.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd66f7fe640 1 -- 172.21.15.83:0/1479734150 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fd668001bd0 con 0x7fd6700cac80 2024-04-17T10:42:50.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd66f7fe640 1 -- 172.21.15.83:0/1479734150 >> v1:172.21.15.83:6789/0 conn(0x7fd6700cac80 legacy=0x7fd67006abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:50.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd66f7fe640 1 --2- 172.21.15.83:0/1479734150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd658003a30 0x7fd658003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:50.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd66f7fe640 1 --2- 172.21.15.83:0/1479734150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd670158490 0x7fd658004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:50.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd66f7fe640 1 --2- 172.21.15.83:0/1479734150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd658004d50 0x7fd6580051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:50.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd66f7fe640 1 -- 172.21.15.83:0/1479734150 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67006f5e0 con 0x7fd658003a30 2024-04-17T10:42:50.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd66f7fe640 1 -- 172.21.15.83:0/1479734150 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6700cf320 con 0x7fd658004d50 2024-04-17T10:42:50.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd66f7fe640 1 -- 172.21.15.83:0/1479734150 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd65800ba20 con 0x7fd670158490 2024-04-17T10:42:50.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd674baa640 1 --2- 172.21.15.83:0/1479734150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd658003a30 0x7fd658003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:50.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd6753ab640 1 --2- 172.21.15.83:0/1479734150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd658004d50 0x7fd6580051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:50.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd66ffff640 1 --2- 172.21.15.83:0/1479734150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd670158490 0x7fd658004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:50.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd674baa640 1 -- 172.21.15.83:0/1479734150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd670158490 msgr2=0x7fd658004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:50.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd674baa640 1 --2- 172.21.15.83:0/1479734150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd670158490 0x7fd658004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd674baa640 1 -- 172.21.15.83:0/1479734150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd658004d50 msgr2=0x7fd6580051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:50.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd674baa640 1 --2- 172.21.15.83:0/1479734150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd658004d50 0x7fd6580051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.181+0000 7fd674baa640 1 -- 172.21.15.83:0/1479734150 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd658003530 con 0x7fd658003a30 2024-04-17T10:42:50.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.185+0000 7fd66ffff640 1 --2- 172.21.15.83:0/1479734150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd670158490 0x7fd658004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:50.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.185+0000 7fd674baa640 1 --2- 172.21.15.83:0/1479734150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd658003a30 0x7fd658003e50 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fd660009ae0 tx=0x7fd66001caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a843c2315011108 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:50.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.185+0000 7fd66f7fe640 1 -- 172.21.15.83:0/1479734150 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6600b2070 con 0x7fd658003a30 2024-04-17T10:42:50.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.185+0000 7fd66f7fe640 1 -- 172.21.15.83:0/1479734150 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6600a5690 con 0x7fd658003a30 2024-04-17T10:42:50.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.185+0000 7fd66f7fe640 1 -- 172.21.15.83:0/1479734150 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6600096b0 con 0x7fd658003a30 2024-04-17T10:42:50.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.185+0000 7fd676e0e640 1 -- 172.21.15.83:0/1479734150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd658003a30 msgr2=0x7fd658003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:50.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.185+0000 7fd676e0e640 1 --2- 172.21.15.83:0/1479734150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd658003a30 0x7fd658003e50 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fd660009ae0 tx=0x7fd66001caa0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.185+0000 7fd6753ab640 1 -- 172.21.15.83:0/1479734150 reap_dead start 2024-04-17T10:42:50.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.185+0000 7fd676e0e640 1 -- 172.21.15.83:0/1479734150 shutdown_connections 2024-04-17T10:42:50.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.185+0000 7fd676e0e640 1 -- 172.21.15.83:0/1479734150 >> 172.21.15.83:0/1479734150 conn(0x7fd6700b1e20 msgr2=0x7fd6700b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:50.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.185+0000 7fd676e0e640 1 -- 172.21.15.83:0/1479734150 shutdown_connections 2024-04-17T10:42:50.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.185+0000 7fd676e0e640 1 -- 172.21.15.83:0/1479734150 wait complete. 2024-04-17T10:42:50.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.185+0000 7fd676e0e640 1 Processor -- start 2024-04-17T10:42:50.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.185+0000 7fd676e0e640 1 -- start start 2024-04-17T10:42:50.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.189+0000 7fd676e0e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd670163da0 0x7fd6701467e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:50.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.189+0000 7fd676e0e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd670164180 0x7fd670146d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:50.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.189+0000 7fd676e0e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6701619c0 0x7fd670161e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:50.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.189+0000 7fd676e0e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd670158870 con 0x7fd6701619c0 2024-04-17T10:42:50.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.189+0000 7fd676e0e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6700ce9a0 con 0x7fd670163da0 2024-04-17T10:42:50.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.189+0000 7fd676e0e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6700cb4a0 con 0x7fd670164180 2024-04-17T10:42:50.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.189+0000 7fd6753ab640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6701619c0 0x7fd670161e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:50.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.189+0000 7fd6753ab640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6701619c0 0x7fd670161e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34768/0 (socket says 172.21.15.83:34768) 2024-04-17T10:42:50.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.189+0000 7fd6753ab640 1 -- 172.21.15.83:0/1374629409 learned_addr learned my addr 172.21.15.83:0/1374629409 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:50.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.189+0000 7fd674baa640 1 --2- 172.21.15.83:0/1374629409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd670163da0 0x7fd6701467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:50.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.189+0000 7fd66ffff640 1 --2- 172.21.15.83:0/1374629409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd670164180 0x7fd670146d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:50.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.189+0000 7fd6753ab640 1 -- 172.21.15.83:0/1374629409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd670164180 msgr2=0x7fd670146d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:50.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.189+0000 7fd6753ab640 1 --2- 172.21.15.83:0/1374629409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd670164180 0x7fd670146d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.189+0000 7fd6753ab640 1 -- 172.21.15.83:0/1374629409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd670163da0 msgr2=0x7fd6701467e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:50.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.189+0000 7fd6753ab640 1 --2- 172.21.15.83:0/1374629409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd670163da0 0x7fd6701467e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.189+0000 7fd6753ab640 1 -- 172.21.15.83:0/1374629409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd670149850 con 0x7fd6701619c0 2024-04-17T10:42:50.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.189+0000 7fd6753ab640 1 --2- 172.21.15.83:0/1374629409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6701619c0 0x7fd670161e40 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fd6700ae470 tx=0x7fd66800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:50.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.189+0000 7fd66d7fa640 1 -- 172.21.15.83:0/1374629409 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd668017030 con 0x7fd6701619c0 2024-04-17T10:42:50.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.189+0000 7fd66d7fa640 1 -- 172.21.15.83:0/1374629409 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd668013420 con 0x7fd6701619c0 2024-04-17T10:42:50.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.189+0000 7fd66d7fa640 1 -- 172.21.15.83:0/1374629409 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd668013610 con 0x7fd6701619c0 2024-04-17T10:42:50.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.189+0000 7fd676e0e640 1 -- 172.21.15.83:0/1374629409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd670149b30 con 0x7fd6701619c0 2024-04-17T10:42:50.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.189+0000 7fd676e0e640 1 -- 172.21.15.83:0/1374629409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd67014a0f0 con 0x7fd6701619c0 2024-04-17T10:42:50.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.189+0000 7fd676e0e640 1 -- 172.21.15.83:0/1374629409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd670074f90 con 0x7fd6701619c0 2024-04-17T10:42:50.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.189+0000 7fd66d7fa640 1 -- 172.21.15.83:0/1374629409 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd66801a070 con 0x7fd6701619c0 2024-04-17T10:42:50.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.189+0000 7fd66d7fa640 1 --2- 172.21.15.83:0/1374629409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd65c066f60 0x7fd65c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:50.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.189+0000 7fd66d7fa640 1 -- 172.21.15.83:0/1374629409 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd668085730 con 0x7fd6701619c0 2024-04-17T10:42:50.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.189+0000 7fd674baa640 1 --2- 172.21.15.83:0/1374629409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd65c066f60 0x7fd65c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:50.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.193+0000 7fd674baa640 1 --2- 172.21.15.83:0/1374629409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd65c066f60 0x7fd65c069420 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd6600af000 tx=0x7fd660008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:50.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.193+0000 7fd66d7fa640 1 -- 172.21.15.83:0/1374629409 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd668022020 con 0x7fd6701619c0 2024-04-17T10:42:50.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.345+0000 7fd676e0e640 1 -- 172.21.15.83:0/1374629409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd6700cddb0 con 0x7fd6701619c0 2024-04-17T10:42:50.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.345+0000 7fd66d7fa640 1 -- 172.21.15.83:0/1374629409 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd668083e20 con 0x7fd6701619c0 2024-04-17T10:42:50.345 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:42:50.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.349+0000 7fd676e0e640 1 -- 172.21.15.83:0/1374629409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd65c066f60 msgr2=0x7fd65c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:50.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.349+0000 7fd676e0e640 1 --2- 172.21.15.83:0/1374629409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd65c066f60 0x7fd65c069420 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd6600af000 tx=0x7fd660008040 comp rx=0 tx=0).stop 2024-04-17T10:42:50.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.349+0000 7fd676e0e640 1 -- 172.21.15.83:0/1374629409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6701619c0 msgr2=0x7fd670161e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:50.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.349+0000 7fd676e0e640 1 --2- 172.21.15.83:0/1374629409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6701619c0 0x7fd670161e40 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fd6700ae470 tx=0x7fd66800e530 comp rx=0 tx=0).stop 2024-04-17T10:42:50.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.349+0000 7fd676e0e640 1 -- 172.21.15.83:0/1374629409 shutdown_connections 2024-04-17T10:42:50.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.349+0000 7fd676e0e640 1 --2- 172.21.15.83:0/1374629409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd65c066f60 0x7fd65c069420 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.349+0000 7fd676e0e640 1 --2- 172.21.15.83:0/1374629409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6701619c0 0x7fd670161e40 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.349+0000 7fd676e0e640 1 --2- 172.21.15.83:0/1374629409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd670164180 0x7fd670146d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.349+0000 7fd676e0e640 1 --2- 172.21.15.83:0/1374629409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd670163da0 0x7fd6701467e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.349+0000 7fd676e0e640 1 -- 172.21.15.83:0/1374629409 >> 172.21.15.83:0/1374629409 conn(0x7fd6700b1e20 msgr2=0x7fd67006b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:50.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.349+0000 7fd676e0e640 1 -- 172.21.15.83:0/1374629409 shutdown_connections 2024-04-17T10:42:50.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.349+0000 7fd676e0e640 1 -- 172.21.15.83:0/1374629409 wait complete. 2024-04-17T10:42:50.363 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:42:50.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46fbe8640 1 Processor -- start 2024-04-17T10:42:50.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46fbe8640 1 -- start start 2024-04-17T10:42:50.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46fbe8640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb4680ce400 0x7fb4680b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:50.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46fbe8640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb4680b5450 0x7fb4680b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:50.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46fbe8640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb46815e4d0 con 0x7fb46815e0f0 2024-04-17T10:42:50.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46fbe8640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb468162b40 con 0x7fb4680b5450 2024-04-17T10:42:50.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46fbe8640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb468162320 con 0x7fb4680ce400 2024-04-17T10:42:50.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46e185640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb4680b5450 0x7fb4680b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:50.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46d984640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb46815e0f0 0x7fb468161c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52826/0 (socket says 172.21.15.83:52826) 2024-04-17T10:42:50.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46d984640 1 -- 172.21.15.83:0/3865466094 learned_addr learned my addr 172.21.15.83:0/3865466094 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:50.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46e185640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb4680b5450 0x7fb4680b58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34780/0 (socket says 172.21.15.83:34780) 2024-04-17T10:42:50.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46d183640 1 --2- 172.21.15.83:0/3865466094 >> v2:172.21.15.167:3300/0 conn(0x7fb4680ce400 0x7fb4680b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:50.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46e185640 1 -- 172.21.15.83:0/3865466094 >> v2:172.21.15.167:3300/0 conn(0x7fb4680ce400 msgr2=0x7fb4680b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:50.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46e185640 1 --2- 172.21.15.83:0/3865466094 >> v2:172.21.15.167:3300/0 conn(0x7fb4680ce400 0x7fb4680b6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46e185640 1 -- 172.21.15.83:0/3865466094 >> v1:172.21.15.83:6789/0 conn(0x7fb46815e0f0 legacy=0x7fb468161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:50.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46c982640 1 -- 172.21.15.83:0/3865466094 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2232846789 0 0) 0x7fb46815e4d0 con 0x7fb46815e0f0 2024-04-17T10:42:50.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46e185640 1 -- 172.21.15.83:0/3865466094 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb468162580 con 0x7fb4680b5450 2024-04-17T10:42:50.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46d183640 1 --2- 172.21.15.83:0/3865466094 >> v2:172.21.15.167:3300/0 conn(0x7fb4680ce400 0x7fb4680b6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:50.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46e185640 1 --2- 172.21.15.83:0/3865466094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4680b5450 0x7fb4680b58e0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fb468068170 tx=0x7fb46001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02039c406e35438 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:50.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46c982640 1 -- 172.21.15.83:0/3865466094 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4600b2070 con 0x7fb4680b5450 2024-04-17T10:42:50.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46c982640 1 -- 172.21.15.83:0/3865466094 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4600a5760 con 0x7fb4680b5450 2024-04-17T10:42:50.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46c982640 1 -- 172.21.15.83:0/3865466094 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4600a5b30 con 0x7fb4680b5450 2024-04-17T10:42:50.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46fbe8640 1 -- 172.21.15.83:0/3865466094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4680b5450 msgr2=0x7fb4680b58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:50.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46fbe8640 1 --2- 172.21.15.83:0/3865466094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4680b5450 0x7fb4680b58e0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fb468068170 tx=0x7fb46001c8a0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46fbe8640 1 -- 172.21.15.83:0/3865466094 shutdown_connections 2024-04-17T10:42:50.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46fbe8640 1 --2- 172.21.15.83:0/3865466094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4680b5450 0x7fb4680b58e0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46fbe8640 1 --2- 172.21.15.83:0/3865466094 >> v2:172.21.15.167:3300/0 conn(0x7fb4680ce400 0x7fb4680b6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46fbe8640 1 -- 172.21.15.83:0/3865466094 >> 172.21.15.83:0/3865466094 conn(0x7fb4680d0450 msgr2=0x7fb4680d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:50.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46fbe8640 1 -- 172.21.15.83:0/3865466094 shutdown_connections 2024-04-17T10:42:50.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46fbe8640 1 -- 172.21.15.83:0/3865466094 wait complete. 2024-04-17T10:42:50.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46fbe8640 1 Processor -- start 2024-04-17T10:42:50.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46fbe8640 1 -- start start 2024-04-17T10:42:50.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46fbe8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4680b5450 0x7fb46814e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:50.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46fbe8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4680ce400 0x7fb468157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:50.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46fbe8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb46815e0f0 0x7fb468155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:50.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46fbe8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4680d2ed0 con 0x7fb4680ce400 2024-04-17T10:42:50.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46fbe8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4680d2d50 con 0x7fb46815e0f0 2024-04-17T10:42:50.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46fbe8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4680d3050 con 0x7fb4680b5450 2024-04-17T10:42:50.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46d183640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4680ce400 0x7fb468157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:50.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46d183640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4680ce400 0x7fb468157300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34796/0 (socket says 172.21.15.83:34796) 2024-04-17T10:42:50.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46d183640 1 -- 172.21.15.83:0/743037983 learned_addr learned my addr 172.21.15.83:0/743037983 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:50.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46d984640 1 --2- 172.21.15.83:0/743037983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4680b5450 0x7fb46814e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:50.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46e185640 1 --2- 172.21.15.83:0/743037983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb46815e0f0 0x7fb468155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:50.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46d183640 1 -- 172.21.15.83:0/743037983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4680b5450 msgr2=0x7fb46814e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:50.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46d183640 1 --2- 172.21.15.83:0/743037983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4680b5450 0x7fb46814e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46d183640 1 -- 172.21.15.83:0/743037983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb46815e0f0 msgr2=0x7fb468155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:50.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46d183640 1 --2- 172.21.15.83:0/743037983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb46815e0f0 0x7fb468155540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46d183640 1 -- 172.21.15.83:0/743037983 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb468153730 con 0x7fb4680ce400 2024-04-17T10:42:50.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb46d183640 1 --2- 172.21.15.83:0/743037983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4680ce400 0x7fb468157300 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fb45800c820 tx=0x7fb45800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:50.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb45e7fc640 1 -- 172.21.15.83:0/743037983 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb458015070 con 0x7fb4680ce400 2024-04-17T10:42:50.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb45e7fc640 1 -- 172.21.15.83:0/743037983 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb458007a30 con 0x7fb4680ce400 2024-04-17T10:42:50.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.493+0000 7fb45e7fc640 1 -- 172.21.15.83:0/743037983 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb458007e00 con 0x7fb4680ce400 2024-04-17T10:42:50.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.497+0000 7fb46fbe8640 1 -- 172.21.15.83:0/743037983 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb468157840 con 0x7fb4680ce400 2024-04-17T10:42:50.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.497+0000 7fb46fbe8640 1 -- 172.21.15.83:0/743037983 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb468152c50 con 0x7fb4680ce400 2024-04-17T10:42:50.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.497+0000 7fb46fbe8640 1 -- 172.21.15.83:0/743037983 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4680681a0 con 0x7fb4680ce400 2024-04-17T10:42:50.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.497+0000 7fb45e7fc640 1 -- 172.21.15.83:0/743037983 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb458018070 con 0x7fb4680ce400 2024-04-17T10:42:50.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.497+0000 7fb45e7fc640 1 --2- 172.21.15.83:0/743037983 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb44c066f60 0x7fb44c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:50.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.497+0000 7fb46d984640 1 --2- 172.21.15.83:0/743037983 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb44c066f60 0x7fb44c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:50.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.497+0000 7fb45e7fc640 1 -- 172.21.15.83:0/743037983 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb458084fc0 con 0x7fb4680ce400 2024-04-17T10:42:50.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.497+0000 7fb46d984640 1 --2- 172.21.15.83:0/743037983 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb44c066f60 0x7fb44c069420 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb464000f70 tx=0x7fb464007c10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:50.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.501+0000 7fb45e7fc640 1 -- 172.21.15.83:0/743037983 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb458055fd0 con 0x7fb4680ce400 2024-04-17T10:42:50.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.649+0000 7fb46fbe8640 1 -- 172.21.15.83:0/743037983 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb468074f90 con 0x7fb4680ce400 2024-04-17T10:42:50.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.649+0000 7fb45e7fc640 1 -- 172.21.15.83:0/743037983 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v59) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb4580558d0 con 0x7fb4680ce400 2024-04-17T10:42:50.648 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:42:50.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.649+0000 7fb46fbe8640 1 -- 172.21.15.83:0/743037983 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb44c066f60 msgr2=0x7fb44c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:50.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.649+0000 7fb46fbe8640 1 --2- 172.21.15.83:0/743037983 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb44c066f60 0x7fb44c069420 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb464000f70 tx=0x7fb464007c10 comp rx=0 tx=0).stop 2024-04-17T10:42:50.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.649+0000 7fb46fbe8640 1 -- 172.21.15.83:0/743037983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4680ce400 msgr2=0x7fb468157300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:50.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.649+0000 7fb46fbe8640 1 --2- 172.21.15.83:0/743037983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4680ce400 0x7fb468157300 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fb45800c820 tx=0x7fb45800cce0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.649+0000 7fb46fbe8640 1 -- 172.21.15.83:0/743037983 shutdown_connections 2024-04-17T10:42:50.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.649+0000 7fb46fbe8640 1 --2- 172.21.15.83:0/743037983 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb44c066f60 0x7fb44c069420 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.649+0000 7fb46fbe8640 1 --2- 172.21.15.83:0/743037983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb46815e0f0 0x7fb468155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.649+0000 7fb46fbe8640 1 --2- 172.21.15.83:0/743037983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4680ce400 0x7fb468157300 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.649+0000 7fb46fbe8640 1 --2- 172.21.15.83:0/743037983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4680b5450 0x7fb46814e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.649+0000 7fb46fbe8640 1 -- 172.21.15.83:0/743037983 >> 172.21.15.83:0/743037983 conn(0x7fb4680d0450 msgr2=0x7fb4680b8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:50.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.649+0000 7fb46fbe8640 1 -- 172.21.15.83:0/743037983 shutdown_connections 2024-04-17T10:42:50.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.649+0000 7fb46fbe8640 1 -- 172.21.15.83:0/743037983 wait complete. 2024-04-17T10:42:50.667 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:42:50.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.793+0000 7fdb75d5f640 1 Processor -- start 2024-04-17T10:42:50.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.793+0000 7fdb75d5f640 1 -- start start 2024-04-17T10:42:50.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.793+0000 7fdb75d5f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdb700b2e30 0x7fdb700d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:50.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.793+0000 7fdb75d5f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb7006c150 con 0x7fdb700b0900 2024-04-17T10:42:50.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.793+0000 7fdb75d5f640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb700b1c40 con 0x7fdb700ce910 2024-04-17T10:42:50.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.793+0000 7fdb75d5f640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb700d1020 con 0x7fdb700b2e30 2024-04-17T10:42:50.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.793+0000 7fdb6ffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdb700b2e30 0x7fdb700d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:50.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.793+0000 7fdb6effd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fdb700b0900 0x7fdb700b2720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52838/0 (socket says 172.21.15.83:52838) 2024-04-17T10:42:50.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.793+0000 7fdb6effd640 1 -- 172.21.15.83:0/582277427 learned_addr learned my addr 172.21.15.83:0/582277427 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:50.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.793+0000 7fdb6ffff640 1 -- 172.21.15.83:0/582277427 >> v1:172.21.15.167:6789/0 conn(0x7fdb700ce910 legacy=0x7fdb700ae490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:50.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.793+0000 7fdb6ffff640 1 -- 172.21.15.83:0/582277427 >> v1:172.21.15.83:6789/0 conn(0x7fdb700b0900 legacy=0x7fdb700b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:50.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.793+0000 7fdb6ffff640 1 -- 172.21.15.83:0/582277427 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb700b1f20 con 0x7fdb700b2e30 2024-04-17T10:42:50.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.793+0000 7fdb6e7fc640 1 -- 172.21.15.83:0/582277427 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4254058992 0 0) 0x7fdb7006c150 con 0x7fdb700b0900 2024-04-17T10:42:50.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.793+0000 7fdb6ffff640 1 --2- 172.21.15.83:0/582277427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb700b2e30 0x7fdb700d0ae0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fdb68009d90 tx=0x7fdb6801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43c975f5afd0a4c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:50.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb6e7fc640 1 -- 172.21.15.83:0/582277427 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb680b2070 con 0x7fdb700b2e30 2024-04-17T10:42:50.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb6e7fc640 1 -- 172.21.15.83:0/582277427 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb680a5760 con 0x7fdb700b2e30 2024-04-17T10:42:50.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb6e7fc640 1 -- 172.21.15.83:0/582277427 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb680a5b30 con 0x7fdb700b2e30 2024-04-17T10:42:50.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb75d5f640 1 -- 172.21.15.83:0/582277427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb700b2e30 msgr2=0x7fdb700d0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:50.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb75d5f640 1 --2- 172.21.15.83:0/582277427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb700b2e30 0x7fdb700d0ae0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fdb68009d90 tx=0x7fdb6801c8a0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb75d5f640 1 -- 172.21.15.83:0/582277427 shutdown_connections 2024-04-17T10:42:50.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb75d5f640 1 --2- 172.21.15.83:0/582277427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb700b2e30 0x7fdb700d0ae0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb75d5f640 1 -- 172.21.15.83:0/582277427 >> 172.21.15.83:0/582277427 conn(0x7fdb700d17a0 msgr2=0x7fdb700d1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:50.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb75d5f640 1 -- 172.21.15.83:0/582277427 shutdown_connections 2024-04-17T10:42:50.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb75d5f640 1 -- 172.21.15.83:0/582277427 wait complete. 2024-04-17T10:42:50.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb75d5f640 1 Processor -- start 2024-04-17T10:42:50.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb75d5f640 1 -- start start 2024-04-17T10:42:50.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb75d5f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb700b0900 0x7fdb700c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:50.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb75d5f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb700b2e30 0x7fdb70152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:50.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb75d5f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb700ce910 0x7fdb700af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:50.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb75d5f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7015dff0 con 0x7fdb700ce910 2024-04-17T10:42:50.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb75d5f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7015de70 con 0x7fdb700b0900 2024-04-17T10:42:50.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb75d5f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7015e170 con 0x7fdb700b2e30 2024-04-17T10:42:50.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb6ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb700ce910 0x7fdb700af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:50.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb6ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb700ce910 0x7fdb700af2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34816/0 (socket says 172.21.15.83:34816) 2024-04-17T10:42:50.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb6ffff640 1 -- 172.21.15.83:0/3319994020 learned_addr learned my addr 172.21.15.83:0/3319994020 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:50.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb6effd640 1 --2- 172.21.15.83:0/3319994020 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb700b2e30 0x7fdb70152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:50.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb6ffff640 1 -- 172.21.15.83:0/3319994020 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb700b2e30 msgr2=0x7fdb70152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:50.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb6ffff640 1 --2- 172.21.15.83:0/3319994020 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb700b2e30 0x7fdb70152400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb6ffff640 1 -- 172.21.15.83:0/3319994020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb700b0900 msgr2=0x7fdb700c6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:50.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb6f7fe640 1 --2- 172.21.15.83:0/3319994020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb700b0900 0x7fdb700c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:50.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb6ffff640 1 --2- 172.21.15.83:0/3319994020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb700b0900 0x7fdb700c6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb6ffff640 1 -- 172.21.15.83:0/3319994020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb700c7420 con 0x7fdb700ce910 2024-04-17T10:42:50.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb6ffff640 1 --2- 172.21.15.83:0/3319994020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb700ce910 0x7fdb700af2c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fdb6801a4d0 tx=0x7fdb68019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:50.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb57fff640 1 -- 172.21.15.83:0/3319994020 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb680b2050 con 0x7fdb700ce910 2024-04-17T10:42:50.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb57fff640 1 -- 172.21.15.83:0/3319994020 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb680be070 con 0x7fdb700ce910 2024-04-17T10:42:50.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb75d5f640 1 -- 172.21.15.83:0/3319994020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb700c7700 con 0x7fdb700ce910 2024-04-17T10:42:50.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb57fff640 1 -- 172.21.15.83:0/3319994020 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb680adba0 con 0x7fdb700ce910 2024-04-17T10:42:50.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.797+0000 7fdb75d5f640 1 -- 172.21.15.83:0/3319994020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb700c7cc0 con 0x7fdb700ce910 2024-04-17T10:42:50.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.801+0000 7fdb75d5f640 1 -- 172.21.15.83:0/3319994020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb70074f90 con 0x7fdb700ce910 2024-04-17T10:42:50.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.801+0000 7fdb57fff640 1 -- 172.21.15.83:0/3319994020 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdb680c3460 con 0x7fdb700ce910 2024-04-17T10:42:50.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.801+0000 7fdb57fff640 1 --2- 172.21.15.83:0/3319994020 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdb5c066f60 0x7fdb5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:50.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.801+0000 7fdb57fff640 1 -- 172.21.15.83:0/3319994020 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb68120390 con 0x7fdb700ce910 2024-04-17T10:42:50.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.801+0000 7fdb6f7fe640 1 --2- 172.21.15.83:0/3319994020 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdb5c066f60 0x7fdb5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:50.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.801+0000 7fdb6f7fe640 1 --2- 172.21.15.83:0/3319994020 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdb5c066f60 0x7fdb5c069420 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdb6400a750 tx=0x7fdb64008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:50.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.805+0000 7fdb57fff640 1 -- 172.21.15.83:0/3319994020 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdb68016520 con 0x7fdb700ce910 2024-04-17T10:42:50.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.961+0000 7fdb75d5f640 1 -- 172.21.15.83:0/3319994020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdb700686f0 con 0x7fdb700ce910 2024-04-17T10:42:50.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.961+0000 7fdb57fff640 1 -- 172.21.15.83:0/3319994020 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v59) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdb680f1390 con 0x7fdb700ce910 2024-04-17T10:42:50.961 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:42:50.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.961+0000 7fdb75d5f640 1 -- 172.21.15.83:0/3319994020 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdb5c066f60 msgr2=0x7fdb5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:50.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.961+0000 7fdb75d5f640 1 --2- 172.21.15.83:0/3319994020 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdb5c066f60 0x7fdb5c069420 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdb6400a750 tx=0x7fdb64008040 comp rx=0 tx=0).stop 2024-04-17T10:42:50.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.961+0000 7fdb75d5f640 1 -- 172.21.15.83:0/3319994020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb700ce910 msgr2=0x7fdb700af2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:50.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.961+0000 7fdb75d5f640 1 --2- 172.21.15.83:0/3319994020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb700ce910 0x7fdb700af2c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fdb6801a4d0 tx=0x7fdb68019640 comp rx=0 tx=0).stop 2024-04-17T10:42:50.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.961+0000 7fdb75d5f640 1 -- 172.21.15.83:0/3319994020 shutdown_connections 2024-04-17T10:42:50.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.961+0000 7fdb75d5f640 1 --2- 172.21.15.83:0/3319994020 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdb5c066f60 0x7fdb5c069420 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.961+0000 7fdb75d5f640 1 --2- 172.21.15.83:0/3319994020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb700ce910 0x7fdb700af2c0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.961+0000 7fdb75d5f640 1 --2- 172.21.15.83:0/3319994020 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb700b2e30 0x7fdb70152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.961+0000 7fdb75d5f640 1 --2- 172.21.15.83:0/3319994020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb700b0900 0x7fdb700c6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:50.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.961+0000 7fdb75d5f640 1 -- 172.21.15.83:0/3319994020 >> 172.21.15.83:0/3319994020 conn(0x7fdb700d17a0 msgr2=0x7fdb700d1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:50.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.961+0000 7fdb75d5f640 1 -- 172.21.15.83:0/3319994020 shutdown_connections 2024-04-17T10:42:50.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:50.961+0000 7fdb75d5f640 1 -- 172.21.15.83:0/3319994020 wait complete. 2024-04-17T10:42:50.978 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:42:51.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.101+0000 7f5fc6ebe640 1 Processor -- start 2024-04-17T10:42:51.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.101+0000 7f5fc6ebe640 1 -- start start 2024-04-17T10:42:51.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.101+0000 7f5fc6ebe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fc015e0f0 0x7f5fc00b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:51.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.101+0000 7f5fc6ebe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5fc00b5450 0x7f5fc00b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:51.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.101+0000 7f5fc6ebe640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fc015e4d0 con 0x7f5fc00ce400 2024-04-17T10:42:51.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.101+0000 7f5fc6ebe640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fc00b5df0 con 0x7f5fc00b5450 2024-04-17T10:42:51.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.101+0000 7f5fc6ebe640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc0162b40 con 0x7f5fc015e0f0 2024-04-17T10:42:51.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.101+0000 7f5fc4c5a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fc015e0f0 0x7f5fc00b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:51.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.101+0000 7f5fc545b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5fc00b5450 0x7f5fc00b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:51.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.101+0000 7f5fbffff640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f5fc00ce400 0x7f5fc0161c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:47354/0 (socket says 172.21.15.83:47354) 2024-04-17T10:42:51.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.101+0000 7f5fbffff640 1 -- 172.21.15.83:0/2970799611 learned_addr learned my addr 172.21.15.83:0/2970799611 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:51.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.101+0000 7f5fc4c5a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fc015e0f0 0x7f5fc00b6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:36452/0 (socket says 172.21.15.83:36452) 2024-04-17T10:42:51.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.101+0000 7f5fc4c5a640 1 -- 172.21.15.83:0/2970799611 >> v2:172.21.15.167:3300/0 conn(0x7f5fc00b5450 msgr2=0x7f5fc00b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:51.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.101+0000 7f5fc4c5a640 1 --2- 172.21.15.83:0/2970799611 >> v2:172.21.15.167:3300/0 conn(0x7f5fc00b5450 0x7f5fc00b58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:51.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.101+0000 7f5fc4c5a640 1 -- 172.21.15.83:0/2970799611 >> v1:172.21.15.167:6789/0 conn(0x7f5fc00ce400 legacy=0x7f5fc0161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:51.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.101+0000 7f5fc4c5a640 1 -- 172.21.15.83:0/2970799611 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc0162580 con 0x7f5fc015e0f0 2024-04-17T10:42:51.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.105+0000 7f5fc4c5a640 1 --2- 172.21.15.83:0/2970799611 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fc015e0f0 0x7f5fc00b6d00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00097c0 tx=0x7f5fb001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52fdb3b8d04efa34 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:51.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.105+0000 7f5fbf7fe640 1 -- 172.21.15.83:0/2970799611 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb00b0070 con 0x7f5fc015e0f0 2024-04-17T10:42:51.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.105+0000 7f5fbf7fe640 1 -- 172.21.15.83:0/2970799611 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb00ab600 con 0x7f5fc015e0f0 2024-04-17T10:42:51.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.105+0000 7f5fbf7fe640 1 -- 172.21.15.83:0/2970799611 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb00ab9d0 con 0x7f5fc015e0f0 2024-04-17T10:42:51.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.105+0000 7f5fc6ebe640 1 -- 172.21.15.83:0/2970799611 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fc015e0f0 msgr2=0x7f5fc00b6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:51.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.105+0000 7f5fc6ebe640 1 --2- 172.21.15.83:0/2970799611 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fc015e0f0 0x7f5fc00b6d00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00097c0 tx=0x7f5fb001c410 comp rx=0 tx=0).stop 2024-04-17T10:42:51.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.105+0000 7f5fc6ebe640 1 -- 172.21.15.83:0/2970799611 shutdown_connections 2024-04-17T10:42:51.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.105+0000 7f5fc6ebe640 1 --2- 172.21.15.83:0/2970799611 >> v2:172.21.15.167:3300/0 conn(0x7f5fc00b5450 0x7f5fc00b58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:51.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.105+0000 7f5fc6ebe640 1 --2- 172.21.15.83:0/2970799611 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fc015e0f0 0x7f5fc00b6d00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:51.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.105+0000 7f5fc6ebe640 1 -- 172.21.15.83:0/2970799611 >> 172.21.15.83:0/2970799611 conn(0x7f5fc00d0450 msgr2=0x7f5fc00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:51.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.105+0000 7f5fc6ebe640 1 -- 172.21.15.83:0/2970799611 shutdown_connections 2024-04-17T10:42:51.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.105+0000 7f5fc6ebe640 1 -- 172.21.15.83:0/2970799611 wait complete. 2024-04-17T10:42:51.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.109+0000 7f5fc6ebe640 1 Processor -- start 2024-04-17T10:42:51.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.109+0000 7f5fc6ebe640 1 -- start start 2024-04-17T10:42:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.109+0000 7f5fc6ebe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fc00b5450 0x7f5fc00c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.109+0000 7f5fc6ebe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fc00ce400 0x7f5fc00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.109+0000 7f5fc6ebe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fc015e0f0 0x7f5fc014c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.109+0000 7f5fc6ebe640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc00d2ed0 con 0x7f5fc00b5450 2024-04-17T10:42:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.109+0000 7f5fc6ebe640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc00d2d50 con 0x7f5fc015e0f0 2024-04-17T10:42:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.109+0000 7f5fc6ebe640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc00d3050 con 0x7f5fc00ce400 2024-04-17T10:42:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.109+0000 7f5fc4c5a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fc00b5450 0x7f5fc00c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.109+0000 7f5fc4c5a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fc00b5450 0x7f5fc00c9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34826/0 (socket says 172.21.15.83:34826) 2024-04-17T10:42:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.109+0000 7f5fc4c5a640 1 -- 172.21.15.83:0/2580613491 learned_addr learned my addr 172.21.15.83:0/2580613491 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.109+0000 7f5fbffff640 1 --2- 172.21.15.83:0/2580613491 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fc00ce400 0x7f5fc00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.109+0000 7f5fc545b640 1 --2- 172.21.15.83:0/2580613491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fc015e0f0 0x7f5fc014c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.109+0000 7f5fc4c5a640 1 -- 172.21.15.83:0/2580613491 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fc00ce400 msgr2=0x7f5fc00b8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.109+0000 7f5fc4c5a640 1 --2- 172.21.15.83:0/2580613491 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fc00ce400 0x7f5fc00b8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.109+0000 7f5fc4c5a640 1 -- 172.21.15.83:0/2580613491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fc015e0f0 msgr2=0x7f5fc014c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.109+0000 7f5fc4c5a640 1 --2- 172.21.15.83:0/2580613491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fc015e0f0 0x7f5fc014c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.109+0000 7f5fc4c5a640 1 -- 172.21.15.83:0/2580613491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc015c490 con 0x7f5fc00b5450 2024-04-17T10:42:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.109+0000 7f5fc4c5a640 1 --2- 172.21.15.83:0/2580613491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fc00b5450 0x7f5fc00c9f10 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00ae040 tx=0x7f5fb00191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.109+0000 7f5fbd7fa640 1 -- 172.21.15.83:0/2580613491 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb00b0040 con 0x7f5fc00b5450 2024-04-17T10:42:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.109+0000 7f5fc6ebe640 1 -- 172.21.15.83:0/2580613491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fc015c770 con 0x7f5fc00b5450 2024-04-17T10:42:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.109+0000 7f5fbd7fa640 1 -- 172.21.15.83:0/2580613491 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb00b9070 con 0x7f5fc00b5450 2024-04-17T10:42:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.109+0000 7f5fbd7fa640 1 -- 172.21.15.83:0/2580613491 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb00b4670 con 0x7f5fc00b5450 2024-04-17T10:42:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.109+0000 7f5fc6ebe640 1 -- 172.21.15.83:0/2580613491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc015cd30 con 0x7f5fc00b5450 2024-04-17T10:42:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.109+0000 7f5fc6ebe640 1 -- 172.21.15.83:0/2580613491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fc00681a0 con 0x7f5fc00b5450 2024-04-17T10:42:51.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.113+0000 7f5fbd7fa640 1 -- 172.21.15.83:0/2580613491 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5fb00b4810 con 0x7f5fc00b5450 2024-04-17T10:42:51.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.113+0000 7f5fbd7fa640 1 --2- 172.21.15.83:0/2580613491 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5fac066f60 0x7f5fac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:51.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.113+0000 7f5fbd7fa640 1 -- 172.21.15.83:0/2580613491 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5fb011f9d0 con 0x7f5fc00b5450 2024-04-17T10:42:51.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.113+0000 7f5fbffff640 1 --2- 172.21.15.83:0/2580613491 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5fac066f60 0x7f5fac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:51.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.113+0000 7f5fbffff640 1 --2- 172.21.15.83:0/2580613491 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5fac066f60 0x7f5fac069420 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5fc01fa130 tx=0x7f5fb4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:51.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.117+0000 7f5fbd7fa640 1 -- 172.21.15.83:0/2580613491 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5fb0016090 con 0x7f5fc00b5450 2024-04-17T10:42:51.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.273+0000 7f5fc6ebe640 1 -- 172.21.15.83:0/2580613491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5fc00caa80 con 0x7f5fc00b5450 2024-04-17T10:42:51.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.273+0000 7f5fbd7fa640 1 -- 172.21.15.83:0/2580613491 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v59) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5fb00f0940 con 0x7f5fc00b5450 2024-04-17T10:42:51.275 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-17T10:42:51.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.277+0000 7f5fc6ebe640 1 -- 172.21.15.83:0/2580613491 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5fac066f60 msgr2=0x7f5fac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:51.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.277+0000 7f5fc6ebe640 1 --2- 172.21.15.83:0/2580613491 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5fac066f60 0x7f5fac069420 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5fc01fa130 tx=0x7f5fb4009040 comp rx=0 tx=0).stop 2024-04-17T10:42:51.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.277+0000 7f5fc6ebe640 1 -- 172.21.15.83:0/2580613491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fc00b5450 msgr2=0x7f5fc00c9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:51.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.277+0000 7f5fc6ebe640 1 --2- 172.21.15.83:0/2580613491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fc00b5450 0x7f5fc00c9f10 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00ae040 tx=0x7f5fb00191b0 comp rx=0 tx=0).stop 2024-04-17T10:42:51.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.277+0000 7f5fc6ebe640 1 -- 172.21.15.83:0/2580613491 shutdown_connections 2024-04-17T10:42:51.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.277+0000 7f5fc6ebe640 1 --2- 172.21.15.83:0/2580613491 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5fac066f60 0x7f5fac069420 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:51.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.277+0000 7f5fc6ebe640 1 --2- 172.21.15.83:0/2580613491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fc015e0f0 0x7f5fc014c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:51.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.277+0000 7f5fc6ebe640 1 --2- 172.21.15.83:0/2580613491 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fc00ce400 0x7f5fc00b8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:51.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.277+0000 7f5fc6ebe640 1 --2- 172.21.15.83:0/2580613491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fc00b5450 0x7f5fc00c9f10 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:51.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.277+0000 7f5fc6ebe640 1 -- 172.21.15.83:0/2580613491 >> 172.21.15.83:0/2580613491 conn(0x7f5fc00d0450 msgr2=0x7f5fc014d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:51.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.277+0000 7f5fc6ebe640 1 -- 172.21.15.83:0/2580613491 shutdown_connections 2024-04-17T10:42:51.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:51.277+0000 7f5fc6ebe640 1 -- 172.21.15.83:0/2580613491 wait complete. 2024-04-17T10:42:51.292 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2024-04-17T10:42:51.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:42:51.293 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-17T10:42:51.303 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[54562]: fuse finished with error 0 and tester_r 0 2024-04-17T10:42:51.304 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:42:57.406 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-17T10:42:57.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:42:57.406 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T10:42:57.414 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:42:57.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.589+0000 7f3097a8c640 1 Processor -- start 2024-04-17T10:42:57.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.589+0000 7f3097a8c640 1 -- start start 2024-04-17T10:42:57.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.593+0000 7f3097a8c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f309015db50 0x7f3090070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:57.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.593+0000 7f3097a8c640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30900709b0 con 0x7f30900aedd0 2024-04-17T10:42:57.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.593+0000 7f3097a8c640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f309015e440 con 0x7f309006dcb0 2024-04-17T10:42:57.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.593+0000 7f3097a8c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f309015e720 con 0x7f309015db50 2024-04-17T10:42:57.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.593+0000 7f3095027640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f30900aedd0 0x7f309006d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53652/0 (socket says 172.21.15.83:53652) 2024-04-17T10:42:57.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.593+0000 7f3095027640 1 -- 172.21.15.83:0/3468314661 learned_addr learned my addr 172.21.15.83:0/3468314661 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:57.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.593+0000 7f3095828640 1 --2- 172.21.15.83:0/3468314661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f309015db50 0x7f3090070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:57.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.593+0000 7f3094826640 1 -- 172.21.15.83:0/3468314661 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4159214281 0 0) 0x7f3080001120 con 0x7f30900aedd0 2024-04-17T10:42:57.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.593+0000 7f3094826640 1 -- 172.21.15.83:0/3468314661 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f307c003530 con 0x7f30900aedd0 2024-04-17T10:42:57.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.593+0000 7f3094826640 1 -- 172.21.15.83:0/3468314661 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1383426547 0 0) 0x7f307c003530 con 0x7f30900aedd0 2024-04-17T10:42:57.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.593+0000 7f3094826640 1 -- 172.21.15.83:0/3468314661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f309015db50 msgr2=0x7f3090070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:57.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.593+0000 7f3094826640 1 --2- 172.21.15.83:0/3468314661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f309015db50 0x7f3090070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:57.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.593+0000 7f3094826640 1 -- 172.21.15.83:0/3468314661 >> v1:172.21.15.167:6789/0 conn(0x7f309006dcb0 legacy=0x7f30900b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:57.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.593+0000 7f3094826640 1 -- 172.21.15.83:0/3468314661 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309015ea00 con 0x7f30900aedd0 2024-04-17T10:42:57.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.593+0000 7f3094826640 1 -- 172.21.15.83:0/3468314661 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f3080001850 con 0x7f30900aedd0 2024-04-17T10:42:57.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.593+0000 7f3094826640 1 -- 172.21.15.83:0/3468314661 >> v1:172.21.15.83:6789/0 conn(0x7f30900aedd0 legacy=0x7f309006d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:57.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.593+0000 7f3094826640 1 --2- 172.21.15.83:0/3468314661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f307c0039f0 0x7f307c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:57.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.593+0000 7f3094826640 1 --2- 172.21.15.83:0/3468314661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f309015db50 0x7f307c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:57.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.593+0000 7f3094826640 1 --2- 172.21.15.83:0/3468314661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f307c004d50 0x7f307c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:57.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.593+0000 7f3094826640 1 -- 172.21.15.83:0/3468314661 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309015e720 con 0x7f307c004d50 2024-04-17T10:42:57.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.593+0000 7f3094826640 1 -- 172.21.15.83:0/3468314661 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c00ba80 con 0x7f307c0039f0 2024-04-17T10:42:57.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.593+0000 7f3094826640 1 -- 172.21.15.83:0/3468314661 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307c00bc80 con 0x7f309015db50 2024-04-17T10:42:57.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.593+0000 7f3096029640 1 --2- 172.21.15.83:0/3468314661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f307c004d50 0x7f307c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:57.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.593+0000 7f3095828640 1 --2- 172.21.15.83:0/3468314661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f307c0039f0 0x7f307c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:57.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.593+0000 7f3095027640 1 --2- 172.21.15.83:0/3468314661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f309015db50 0x7f307c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:57.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.593+0000 7f3096029640 1 -- 172.21.15.83:0/3468314661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f309015db50 msgr2=0x7f307c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:57.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.593+0000 7f3096029640 1 --2- 172.21.15.83:0/3468314661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f309015db50 0x7f307c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:57.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.593+0000 7f3096029640 1 -- 172.21.15.83:0/3468314661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f307c0039f0 msgr2=0x7f307c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:57.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.593+0000 7f3096029640 1 --2- 172.21.15.83:0/3468314661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f307c0039f0 0x7f307c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:57.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.593+0000 7f3096029640 1 -- 172.21.15.83:0/3468314661 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307c003530 con 0x7f307c004d50 2024-04-17T10:42:57.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.597+0000 7f3096029640 1 --2- 172.21.15.83:0/3468314661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f307c004d50 0x7f307c0051d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f3088001e40 tx=0x7f308801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add93218656f8fb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:57.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.597+0000 7f3094826640 1 -- 172.21.15.83:0/3468314661 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3088001a90 con 0x7f307c004d50 2024-04-17T10:42:57.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.597+0000 7f3094826640 1 -- 172.21.15.83:0/3468314661 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3088003b80 con 0x7f307c004d50 2024-04-17T10:42:57.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.597+0000 7f3094826640 1 -- 172.21.15.83:0/3468314661 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3088005080 con 0x7f307c004d50 2024-04-17T10:42:57.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.597+0000 7f3097a8c640 1 -- 172.21.15.83:0/3468314661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f307c004d50 msgr2=0x7f307c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:57.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.597+0000 7f3097a8c640 1 --2- 172.21.15.83:0/3468314661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f307c004d50 0x7f307c0051d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f3088001e40 tx=0x7f308801c760 comp rx=0 tx=0).stop 2024-04-17T10:42:57.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.597+0000 7f3096029640 1 -- 172.21.15.83:0/3468314661 reap_dead start 2024-04-17T10:42:57.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.597+0000 7f3097a8c640 1 -- 172.21.15.83:0/3468314661 shutdown_connections 2024-04-17T10:42:57.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.597+0000 7f3097a8c640 1 -- 172.21.15.83:0/3468314661 >> 172.21.15.83:0/3468314661 conn(0x7f309006abe0 msgr2=0x7f309006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:57.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.597+0000 7f3097a8c640 1 -- 172.21.15.83:0/3468314661 shutdown_connections 2024-04-17T10:42:57.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.597+0000 7f3097a8c640 1 -- 172.21.15.83:0/3468314661 wait complete. 2024-04-17T10:42:57.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.597+0000 7f3097a8c640 1 Processor -- start 2024-04-17T10:42:57.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.597+0000 7f3097a8c640 1 -- start start 2024-04-17T10:42:57.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.597+0000 7f3097a8c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30901e4350 0x7f309015b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:57.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.597+0000 7f3097a8c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30901e4730 0x7f30900b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:57.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.597+0000 7f3097a8c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30901f4380 0x7f30900b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:57.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.597+0000 7f3097a8c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309015df30 con 0x7f30901e4730 2024-04-17T10:42:57.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.597+0000 7f3097a8c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309015efc0 con 0x7f30901e4350 2024-04-17T10:42:57.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.597+0000 7f3097a8c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f309015e0b0 con 0x7f30901f4380 2024-04-17T10:42:57.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.597+0000 7f3095027640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30901e4730 0x7f30900b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:57.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.597+0000 7f3095027640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30901e4730 0x7f30900b18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58204/0 (socket says 172.21.15.83:58204) 2024-04-17T10:42:57.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.597+0000 7f3095027640 1 -- 172.21.15.83:0/3981047849 learned_addr learned my addr 172.21.15.83:0/3981047849 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:57.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.597+0000 7f3095828640 1 --2- 172.21.15.83:0/3981047849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30901e4350 0x7f309015b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:57.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.597+0000 7f3096029640 1 --2- 172.21.15.83:0/3981047849 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30901f4380 0x7f30900b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:57.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.597+0000 7f3095027640 1 -- 172.21.15.83:0/3981047849 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30901f4380 msgr2=0x7f30900b1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:57.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.597+0000 7f3095027640 1 --2- 172.21.15.83:0/3981047849 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30901f4380 0x7f30900b1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:57.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.597+0000 7f3095027640 1 -- 172.21.15.83:0/3981047849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30901e4350 msgr2=0x7f309015b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:57.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.597+0000 7f3095027640 1 --2- 172.21.15.83:0/3981047849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30901e4350 0x7f309015b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:57.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.597+0000 7f3095027640 1 -- 172.21.15.83:0/3981047849 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3090145140 con 0x7f30901e4730 2024-04-17T10:42:57.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.597+0000 7f3095027640 1 --2- 172.21.15.83:0/3981047849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30901e4730 0x7f30900b18b0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f3080007360 tx=0x7f3080007820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:57.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.601+0000 7f30867fc640 1 -- 172.21.15.83:0/3981047849 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308000e8f0 con 0x7f30901e4730 2024-04-17T10:42:57.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.601+0000 7f30867fc640 1 -- 172.21.15.83:0/3981047849 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3080018070 con 0x7f30901e4730 2024-04-17T10:42:57.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.601+0000 7f3097a8c640 1 -- 172.21.15.83:0/3981047849 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f309015c1e0 con 0x7f30901e4730 2024-04-17T10:42:57.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.601+0000 7f30867fc640 1 -- 172.21.15.83:0/3981047849 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308000ecd0 con 0x7f30901e4730 2024-04-17T10:42:57.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.601+0000 7f3097a8c640 1 -- 172.21.15.83:0/3981047849 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f309015c7a0 con 0x7f30901e4730 2024-04-17T10:42:57.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.601+0000 7f3097a8c640 1 -- 172.21.15.83:0/3981047849 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3090074f90 con 0x7f30901e4730 2024-04-17T10:42:57.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.601+0000 7f30867fc640 1 -- 172.21.15.83:0/3981047849 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f308001d460 con 0x7f30901e4730 2024-04-17T10:42:57.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.601+0000 7f30867fc640 1 --2- 172.21.15.83:0/3981047849 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3074066f60 0x7f3074069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:57.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.601+0000 7f30867fc640 1 -- 172.21.15.83:0/3981047849 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f30800850e0 con 0x7f30901e4730 2024-04-17T10:42:57.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.601+0000 7f3095828640 1 --2- 172.21.15.83:0/3981047849 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3074066f60 0x7f3074069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:57.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.601+0000 7f3095828640 1 --2- 172.21.15.83:0/3981047849 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3074066f60 0x7f3074069420 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f30900ade50 tx=0x7f308c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:57.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.605+0000 7f30867fc640 1 -- 172.21.15.83:0/3981047849 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f30900709b0 con 0x7f30901e4730 2024-04-17T10:42:57.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.801+0000 7f3097a8c640 1 -- 172.21.15.83:0/3981047849 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30900b2af0 con 0x7f30901e4730 2024-04-17T10:42:57.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.801+0000 7f30867fc640 1 -- 172.21.15.83:0/3981047849 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f3080056180 con 0x7f30901e4730 2024-04-17T10:42:57.796 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:42:57.797 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5398,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1991041077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1991041077},{"type":"v1","addr":"172.21.15.167:6835","nonce":1991041077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6522,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/320723747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":320723747},{"type":"v1","addr":"172.21.15.83:6836","nonce":320723747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6525,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/4066270849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":4066270849},{"type":"v1","addr":"172.21.15.83:6837","nonce":4066270849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:42:34.328397+0000","modified":"2024-04-17T10:42:37.455022+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6534},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6534":{"gid":6534,"name":"b","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6833/271994303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":271994303},{"type":"v1","addr":"172.21.15.167:6833","nonce":271994303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-17T10:42:57.797 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 32 2024-04-17T10:42:57.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.801+0000 7f3097a8c640 1 -- 172.21.15.83:0/3981047849 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3074066f60 msgr2=0x7f3074069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:57.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.801+0000 7f3097a8c640 1 --2- 172.21.15.83:0/3981047849 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3074066f60 0x7f3074069420 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f30900ade50 tx=0x7f308c008040 comp rx=0 tx=0).stop 2024-04-17T10:42:57.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.801+0000 7f3097a8c640 1 -- 172.21.15.83:0/3981047849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30901e4730 msgr2=0x7f30900b18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:57.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.801+0000 7f3097a8c640 1 --2- 172.21.15.83:0/3981047849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30901e4730 0x7f30900b18b0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f3080007360 tx=0x7f3080007820 comp rx=0 tx=0).stop 2024-04-17T10:42:57.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.801+0000 7f3097a8c640 1 -- 172.21.15.83:0/3981047849 shutdown_connections 2024-04-17T10:42:57.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.801+0000 7f3097a8c640 1 --2- 172.21.15.83:0/3981047849 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3074066f60 0x7f3074069420 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:57.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.801+0000 7f3097a8c640 1 --2- 172.21.15.83:0/3981047849 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30901f4380 0x7f30900b1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:57.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.801+0000 7f3097a8c640 1 --2- 172.21.15.83:0/3981047849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30901e4730 0x7f30900b18b0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:57.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.801+0000 7f3097a8c640 1 --2- 172.21.15.83:0/3981047849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30901e4350 0x7f309015b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:57.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.801+0000 7f3097a8c640 1 -- 172.21.15.83:0/3981047849 >> 172.21.15.83:0/3981047849 conn(0x7f309006abe0 msgr2=0x7f30900cc450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:57.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.801+0000 7f3097a8c640 1 -- 172.21.15.83:0/3981047849 shutdown_connections 2024-04-17T10:42:57.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:57.801+0000 7f3097a8c640 1 -- 172.21.15.83:0/3981047849 wait complete. 2024-04-17T10:42:57.816 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-04-17T10:42:57.976 INFO:teuthology.orchestra.run.smithi167.stdout:{"message":"","return_code":0} 2024-04-17T10:42:57.977 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-17T10:42:57.978 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-17T10:42:57.978 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:42:57.978 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-17T10:42:57.978 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:42:57.979 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-17T10:42:57.979 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:42:57.979 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-17T10:42:57.979 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:42:57.990 INFO:tasks.ceph.mds.b:Stopped 2024-04-17T10:42:57.990 DEBUG:teuthology.parallel:result is None 2024-04-17T10:42:57.991 INFO:tasks.ceph.mds.d:Stopped 2024-04-17T10:42:57.991 DEBUG:teuthology.parallel:result is None 2024-04-17T10:42:58.003 INFO:tasks.ceph.mds.c:Stopped 2024-04-17T10:42:58.004 DEBUG:teuthology.parallel:result is None 2024-04-17T10:42:58.004 INFO:tasks.ceph.mds.a:Stopped 2024-04-17T10:42:58.004 DEBUG:teuthology.parallel:result is None 2024-04-17T10:42:58.004 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000000.00000000 file_head 2024-04-17T10:42:58.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.089+0000 7f5cceb1d080 1 Processor -- start 2024-04-17T10:42:58.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.089+0000 7f5cceb1d080 1 -- start start 2024-04-17T10:42:58.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5cceb1d080 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560c12bdac50 0x560c12bf7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:58.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5cceb1d080 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c12bdb030 0x560c12bd73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:58.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5cceb1d080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c12bdc8d0 0x560c12bd7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:58.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5cceb1d080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c12b213c0 con 0x560c12bdc8d0 2024-04-17T10:42:58.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5cceb1d080 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c12ac59f0 con 0x560c12bdb030 2024-04-17T10:42:58.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5cceb1d080 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c12b21540 con 0x560c12bdac50 2024-04-17T10:42:58.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5ccce3c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c12bdc8d0 0x560c12bd7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:58.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5ccce3c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c12bdc8d0 0x560c12bd7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58218/0 (socket says 172.21.15.83:58218) 2024-04-17T10:42:58.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5ccce3c640 1 -- 172.21.15.83:0/2282972381 learned_addr learned my addr 172.21.15.83:0/2282972381 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:58.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5ccc63b640 1 --2- 172.21.15.83:0/2282972381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560c12bdac50 0x560c12bf7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:58.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5ccbe3a640 1 --2- 172.21.15.83:0/2282972381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c12bdb030 0x560c12bd73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:58.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5ccce3c640 1 -- 172.21.15.83:0/2282972381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560c12bdac50 msgr2=0x560c12bf7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:58.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5ccce3c640 1 --2- 172.21.15.83:0/2282972381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560c12bdac50 0x560c12bf7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:58.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5ccce3c640 1 -- 172.21.15.83:0/2282972381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c12bdb030 msgr2=0x560c12bd73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:58.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5ccce3c640 1 --2- 172.21.15.83:0/2282972381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c12bdb030 0x560c12bd73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:58.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5ccce3c640 1 -- 172.21.15.83:0/2282972381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c12b4d750 con 0x560c12bdc8d0 2024-04-17T10:42:58.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5ccbe3a640 1 --2- 172.21.15.83:0/2282972381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c12bdb030 0x560c12bd73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:42:58.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5ccce3c640 1 --2- 172.21.15.83:0/2282972381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c12bdc8d0 0x560c12bd7920 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4018fc0 tx=0x7f5cc4003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=983e4938e17bbd2f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:58.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5ccae38640 1 -- 172.21.15.83:0/2282972381 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc40af040 con 0x560c12bdc8d0 2024-04-17T10:42:58.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5ccae38640 1 -- 172.21.15.83:0/2282972381 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc40b9070 con 0x560c12bdc8d0 2024-04-17T10:42:58.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5ccae38640 1 -- 172.21.15.83:0/2282972381 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc40aad90 con 0x560c12bdc8d0 2024-04-17T10:42:58.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5cceb1d080 1 -- 172.21.15.83:0/2282972381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c12bdc8d0 msgr2=0x560c12bd7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:58.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5cceb1d080 1 --2- 172.21.15.83:0/2282972381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c12bdc8d0 0x560c12bd7920 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4018fc0 tx=0x7f5cc4003030 comp rx=0 tx=0).stop 2024-04-17T10:42:58.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5cceb1d080 1 -- 172.21.15.83:0/2282972381 shutdown_connections 2024-04-17T10:42:58.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5cceb1d080 1 --2- 172.21.15.83:0/2282972381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c12bdc8d0 0x560c12bd7920 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:58.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5cceb1d080 1 --2- 172.21.15.83:0/2282972381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c12bdb030 0x560c12bd73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:58.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5cceb1d080 1 --2- 172.21.15.83:0/2282972381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560c12bdac50 0x560c12bf7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:58.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5cceb1d080 1 -- 172.21.15.83:0/2282972381 >> 172.21.15.83:0/2282972381 conn(0x560c12b4c850 msgr2=0x560c12b66000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:58.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5cceb1d080 1 -- 172.21.15.83:0/2282972381 shutdown_connections 2024-04-17T10:42:58.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5cceb1d080 1 -- 172.21.15.83:0/2282972381 wait complete. 2024-04-17T10:42:58.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5cceb1d080 1 Processor -- start 2024-04-17T10:42:58.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5cceb1d080 1 -- start start 2024-04-17T10:42:58.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5cceb1d080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c12bdac50 0x560c12c8ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:58.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5cceb1d080 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560c12bdb030 0x560c12bd5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:58.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5cceb1d080 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c12bdc8d0 0x560c12bfea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:58.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5cceb1d080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c12b66960 con 0x560c12bdac50 2024-04-17T10:42:58.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5cceb1d080 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c12b4d9f0 con 0x560c12bdc8d0 2024-04-17T10:42:58.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5cceb1d080 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c12bd6000 con 0x560c12bdb030 2024-04-17T10:42:58.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5ccc63b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c12bdac50 0x560c12c8ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:58.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5ccc63b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c12bdac50 0x560c12c8ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58224/0 (socket says 172.21.15.83:58224) 2024-04-17T10:42:58.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.093+0000 7f5ccc63b640 1 -- 172.21.15.83:0/4143339152 learned_addr learned my addr 172.21.15.83:0/4143339152 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:58.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.097+0000 7f5ccce3c640 1 --2- 172.21.15.83:0/4143339152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c12bdc8d0 0x560c12bfea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:58.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.097+0000 7f5ccbe3a640 1 --2- 172.21.15.83:0/4143339152 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560c12bdb030 0x560c12bd5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:58.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.097+0000 7f5ccc63b640 1 -- 172.21.15.83:0/4143339152 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560c12bdb030 msgr2=0x560c12bd5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:58.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.097+0000 7f5ccc63b640 1 --2- 172.21.15.83:0/4143339152 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560c12bdb030 0x560c12bd5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:58.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.097+0000 7f5ccc63b640 1 -- 172.21.15.83:0/4143339152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c12bdc8d0 msgr2=0x560c12bfea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:58.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.097+0000 7f5ccc63b640 1 --2- 172.21.15.83:0/4143339152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c12bdc8d0 0x560c12bfea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:58.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.097+0000 7f5ccc63b640 1 -- 172.21.15.83:0/4143339152 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c12c8f5e0 con 0x560c12bdac50 2024-04-17T10:42:58.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.097+0000 7f5ccc63b640 1 --2- 172.21.15.83:0/4143339152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c12bdac50 0x560c12c8ef80 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc008780 tx=0x7f5cbc008c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:58.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.097+0000 7f5cc9e36640 1 -- 172.21.15.83:0/4143339152 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc006d20 con 0x560c12bdac50 2024-04-17T10:42:58.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.097+0000 7f5cc9e36640 1 -- 172.21.15.83:0/4143339152 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cbc01a070 con 0x560c12bdac50 2024-04-17T10:42:58.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.097+0000 7f5cc9e36640 1 -- 172.21.15.83:0/4143339152 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc015680 con 0x560c12bdac50 2024-04-17T10:42:58.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.097+0000 7f5cceb1d080 1 -- 172.21.15.83:0/4143339152 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560c12c8f880 con 0x560c12bdac50 2024-04-17T10:42:58.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.097+0000 7f5cceb1d080 1 -- 172.21.15.83:0/4143339152 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c12b60ab0 con 0x560c12bdac50 2024-04-17T10:42:58.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.097+0000 7f5cc9e36640 1 -- 172.21.15.83:0/4143339152 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5cbc013030 con 0x560c12bdac50 2024-04-17T10:42:58.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.097+0000 7f5cc9e36640 1 --2- 172.21.15.83:0/4143339152 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5cac065970 0x7f5cac067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:58.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.097+0000 7f5cc9e36640 1 -- 172.21.15.83:0/4143339152 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5cbc0843e0 con 0x560c12bdac50 2024-04-17T10:42:58.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.097+0000 7f5ccbe3a640 1 --2- 172.21.15.83:0/4143339152 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5cac065970 0x7f5cac067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:58.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.097+0000 7f5cceb1d080 1 --2- 172.21.15.83:0/4143339152 >> [v2:172.21.15.167:6808/285324536,v1:172.21.15.167:6809/285324536] conn(0x560c12b61910 0x560c12b61cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:58.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.097+0000 7f5cceb1d080 1 -- 172.21.15.83:0/4143339152 --> [v2:172.21.15.167:6808/285324536,v1:172.21.15.167:6809/285324536] -- osd_op(unknown.0.0:1 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-rm-keys in=17b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59) v9 -- 0x560c12c8c9c0 con 0x560c12b61910 2024-04-17T10:42:58.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.097+0000 7f5ccce3c640 1 --2- 172.21.15.83:0/4143339152 >> [v2:172.21.15.167:6808/285324536,v1:172.21.15.167:6809/285324536] conn(0x560c12b61910 0x560c12b61cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:58.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.097+0000 7f5ccbe3a640 1 --2- 172.21.15.83:0/4143339152 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5cac065970 0x7f5cac067e30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4004500 tx=0x7f5cb4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:58.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.101+0000 7f5ccce3c640 1 --2- 172.21.15.83:0/4143339152 >> [v2:172.21.15.167:6808/285324536,v1:172.21.15.167:6809/285324536] conn(0x560c12b61910 0x560c12b61cf0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:58.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.101+0000 7f5ccce3c640 1 -- 172.21.15.83:0/4143339152 <== osd.7 v2:172.21.15.167:6808/285324536 1 ==== osd_op_reply(1 10000000000.00000000 [omap-rm-keys] v59'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5cc40b2040 con 0x560c12b61910 2024-04-17T10:42:58.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.101+0000 7f5cceb1d080 1 -- 172.21.15.83:0/4143339152 >> [v2:172.21.15.167:6808/285324536,v1:172.21.15.167:6809/285324536] conn(0x560c12b61910 msgr2=0x560c12b61cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:58.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.101+0000 7f5cceb1d080 1 --2- 172.21.15.83:0/4143339152 >> [v2:172.21.15.167:6808/285324536,v1:172.21.15.167:6809/285324536] conn(0x560c12b61910 0x560c12b61cf0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:58.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.101+0000 7f5cceb1d080 1 -- 172.21.15.83:0/4143339152 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5cac065970 msgr2=0x7f5cac067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:58.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.101+0000 7f5cceb1d080 1 --2- 172.21.15.83:0/4143339152 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5cac065970 0x7f5cac067e30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4004500 tx=0x7f5cb4008040 comp rx=0 tx=0).stop 2024-04-17T10:42:58.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.101+0000 7f5cceb1d080 1 -- 172.21.15.83:0/4143339152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c12bdac50 msgr2=0x560c12c8ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:58.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.101+0000 7f5cceb1d080 1 --2- 172.21.15.83:0/4143339152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c12bdac50 0x560c12c8ef80 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc008780 tx=0x7f5cbc008c40 comp rx=0 tx=0).stop 2024-04-17T10:42:58.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.101+0000 7f5ccce3c640 1 -- 172.21.15.83:0/4143339152 reap_dead start 2024-04-17T10:42:58.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.101+0000 7f5cceb1d080 1 -- 172.21.15.83:0/4143339152 shutdown_connections 2024-04-17T10:42:58.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.101+0000 7f5cceb1d080 1 -- 172.21.15.83:0/4143339152 >> 172.21.15.83:0/4143339152 conn(0x560c12b4c850 msgr2=0x560c12bdb850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:58.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.101+0000 7f5cceb1d080 1 -- 172.21.15.83:0/4143339152 shutdown_connections 2024-04-17T10:42:58.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.101+0000 7f5cceb1d080 1 -- 172.21.15.83:0/4143339152 wait complete. 2024-04-17T10:42:58.102 ERROR:tasks.ceph.mds.a:tried to stop a non-running daemon 2024-04-17T10:42:58.102 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2024-04-17T10:42:58.102 ERROR:tasks.ceph.mds.c:tried to stop a non-running daemon 2024-04-17T10:42:58.102 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2024-04-17T10:42:58.103 ERROR:tasks.ceph.mds.b:tried to stop a non-running daemon 2024-04-17T10:42:58.103 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2024-04-17T10:42:58.103 ERROR:tasks.ceph.mds.d:tried to stop a non-running daemon 2024-04-17T10:42:58.103 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail d 2024-04-17T10:42:58.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.265+0000 7f3092302640 1 Processor -- start 2024-04-17T10:42:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.265+0000 7f3092302640 1 -- start start 2024-04-17T10:42:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.265+0000 7f4fa0d96640 1 Processor -- start 2024-04-17T10:42:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.265+0000 7f4fa0d96640 1 -- start start 2024-04-17T10:42:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.265+0000 7fccf1ea7640 1 Processor -- start 2024-04-17T10:42:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.265+0000 7fccf1ea7640 1 -- start start 2024-04-17T10:42:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.265+0000 7f3001878640 1 Processor -- start 2024-04-17T10:42:58.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.265+0000 7f3001878640 1 -- start start 2024-04-17T10:42:58.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.265+0000 7f3092302640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f308c15e0f0 0x7f308c0b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:58.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.265+0000 7f3092302640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f308c0b5450 0x7f308c0b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:58.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.265+0000 7f3092302640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f308c15e4d0 con 0x7f308c0ce400 2024-04-17T10:42:58.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.265+0000 7f3092302640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f308c0b5df0 con 0x7f308c0b5450 2024-04-17T10:42:58.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.265+0000 7f3092302640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f308c162b40 con 0x7f308c15e0f0 2024-04-17T10:42:58.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f4fa0d96640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4f9c0cf320 0x7f4f9c066ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:58.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f4fa0d96640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4f9c0ca560 0x7f4f9c0ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:58.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f4fa0d96640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f9c072350 con 0x7f4f9c06bbf0 2024-04-17T10:42:58.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f4f99d9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4f9c0cf320 0x7f4f9c066ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:58.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f308b7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f308c0ce400 0x7f308c161c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53682/0 (socket says 172.21.15.83:53682) 2024-04-17T10:42:58.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f308b7fe640 1 -- 172.21.15.83:0/13764074 learned_addr learned my addr 172.21.15.83:0/13764074 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:58.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f308bfff640 1 --2- 172.21.15.83:0/13764074 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f308c15e0f0 0x7f308c0b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:58.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f4f99d9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4f9c0cf320 0x7f4f9c066ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58232/0 (socket says 172.21.15.83:58232) 2024-04-17T10:42:58.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f4f99d9b640 1 -- 172.21.15.83:0/4090556574 learned_addr learned my addr 172.21.15.83:0/4090556574 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:58.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f4fa0d96640 1 -- 172.21.15.83:0/4090556574 --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f9c067000 con 0x7f4f9c0cf320 2024-04-17T10:42:58.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f4fa0d96640 1 -- 172.21.15.83:0/4090556574 --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f9c072630 con 0x7f4f9c0ca560 2024-04-17T10:42:58.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f4f9ad9d640 1 --2- 172.21.15.83:0/4090556574 >> v2:172.21.15.167:3300/0 conn(0x7f4f9c0ca560 0x7f4f9c0ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f309089f640 1 --2- 172.21.15.83:0/13764074 >> v2:172.21.15.167:3300/0 conn(0x7f308c0b5450 0x7f308c0b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f4f99d9b640 1 -- 172.21.15.83:0/4090556574 >> v2:172.21.15.167:3300/0 conn(0x7f4f9c0ca560 msgr2=0x7f4f9c0ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f4f99d9b640 1 --2- 172.21.15.83:0/4090556574 >> v2:172.21.15.167:3300/0 conn(0x7f4f9c0ca560 0x7f4f9c0ca9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f4f99d9b640 1 -- 172.21.15.83:0/4090556574 >> v1:172.21.15.167:6789/0 conn(0x7f4f9c06bbf0 legacy=0x7f4f9c06bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f308affd640 1 -- 172.21.15.83:0/13764074 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2628441039 0 0) 0x7f308c15e4d0 con 0x7f308c0ce400 2024-04-17T10:42:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f308affd640 1 -- 172.21.15.83:0/13764074 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3074003530 con 0x7f308c0ce400 2024-04-17T10:42:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f4f99d9b640 1 -- 172.21.15.83:0/4090556574 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f9c072910 con 0x7f4f9c0cf320 2024-04-17T10:42:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7fccf1ea7640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fccec15df90 0x7fccec0d0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f308bfff640 1 -- 172.21.15.83:0/13764074 >> v2:172.21.15.167:3300/0 conn(0x7f308c0b5450 msgr2=0x7f308c0b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f308bfff640 1 --2- 172.21.15.83:0/13764074 >> v2:172.21.15.167:3300/0 conn(0x7f308c0b5450 0x7f308c0b58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f308bfff640 1 -- 172.21.15.83:0/13764074 >> v1:172.21.15.83:6789/0 conn(0x7f308c0ce400 legacy=0x7f308c161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f308bfff640 1 -- 172.21.15.83:0/13764074 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308c162580 con 0x7f308c15e0f0 2024-04-17T10:42:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7fccf1ea7640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fccec06cd30 0x7fccec0cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7fccf1ea7640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccec0d2200 con 0x7fccec06c220 2024-04-17T10:42:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7fccf1ea7640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccec0d4580 con 0x7fccec06cd30 2024-04-17T10:42:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7fccebfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fccec06cd30 0x7fccec0cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7fccf1ea7640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccec0d2810 con 0x7fccec15df90 2024-04-17T10:42:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7fccebfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fccec06cd30 0x7fccec0cf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58244/0 (socket says 172.21.15.83:58244) 2024-04-17T10:42:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7fccebfff640 1 -- 172.21.15.83:0/608965287 learned_addr learned my addr 172.21.15.83:0/608965287 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f3001878640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ffc15bba0 0x7f2ffc0b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:58.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7fcceb7fe640 1 --2- 172.21.15.83:0/608965287 >> v2:172.21.15.167:3300/0 conn(0x7fccec15df90 0x7fccec0d0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:58.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f2ffa7fc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2ffc0d2300 0x7f2ffc0cc280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53698/0 (socket says 172.21.15.83:53698) 2024-04-17T10:42:58.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7fccebfff640 1 -- 172.21.15.83:0/608965287 >> v2:172.21.15.167:3300/0 conn(0x7fccec15df90 msgr2=0x7fccec0d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:58.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7fccebfff640 1 --2- 172.21.15.83:0/608965287 >> v2:172.21.15.167:3300/0 conn(0x7fccec15df90 0x7fccec0d0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:58.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7fccebfff640 1 -- 172.21.15.83:0/608965287 >> v1:172.21.15.167:6789/0 conn(0x7fccec06c220 legacy=0x7fccec06b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:58.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7fccebfff640 1 -- 172.21.15.83:0/608965287 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccec0d47d0 con 0x7fccec06cd30 2024-04-17T10:42:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f3001878640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2ffc0cfaa0 0x7f2ffc0cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f2ffa7fc640 1 -- 172.21.15.83:0/1026490694 learned_addr learned my addr 172.21.15.83:0/1026490694 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:42:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f3001878640 1 -- 172.21.15.83:0/1026490694 --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ffc0b0a30 con 0x7f2ffc0d2300 2024-04-17T10:42:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f3001878640 1 -- 172.21.15.83:0/1026490694 --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ffc0d0430 con 0x7f2ffc0cfaa0 2024-04-17T10:42:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f3001878640 1 -- 172.21.15.83:0/1026490694 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc0cf3e0 con 0x7f2ffc15bba0 2024-04-17T10:42:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f2ffaffd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ffc15bba0 0x7f2ffc0b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f2ffb7fe640 1 --2- 172.21.15.83:0/1026490694 >> v2:172.21.15.83:3300/0 conn(0x7f2ffc0cfaa0 0x7f2ffc0cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f2ff9ffb640 1 -- 172.21.15.83:0/1026490694 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1745610693 0 0) 0x7f2ffc0b0a30 con 0x7f2ffc0d2300 2024-04-17T10:42:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f2ff9ffb640 1 -- 172.21.15.83:0/1026490694 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fe4003530 con 0x7f2ffc0d2300 2024-04-17T10:42:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f2ffaffd640 1 -- 172.21.15.83:0/1026490694 >> v2:172.21.15.83:3300/0 conn(0x7f2ffc0cfaa0 msgr2=0x7f2ffc0cfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:42:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f2ffaffd640 1 --2- 172.21.15.83:0/1026490694 >> v2:172.21.15.83:3300/0 conn(0x7f2ffc0cfaa0 0x7f2ffc0cfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f2ffaffd640 1 -- 172.21.15.83:0/1026490694 >> v1:172.21.15.83:6789/0 conn(0x7f2ffc0d2300 legacy=0x7f2ffc0cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f2ffaffd640 1 -- 172.21.15.83:0/1026490694 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ffc0cf630 con 0x7f2ffc15bba0 2024-04-17T10:42:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f2ffb7fe640 1 --2- 172.21.15.83:0/1026490694 >> v2:172.21.15.83:3300/0 conn(0x7f2ffc0cfaa0 0x7f2ffc0cfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:42:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f4f99d9b640 1 --2- 172.21.15.83:0/4090556574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f9c0cf320 0x7f4f9c066ac0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c0ca4e0 tx=0x7f4f9001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681270a304f9ab1d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.269+0000 7f308bfff640 1 --2- 172.21.15.83:0/13764074 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f308c15e0f0 0x7f308c0b6d00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f307c0097c0 tx=0x7f307c01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9a59848c6cdd9a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f308affd640 1 -- 172.21.15.83:0/13764074 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c0b0070 con 0x7f308c15e0f0 2024-04-17T10:42:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f308affd640 1 -- 172.21.15.83:0/13764074 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307c0ab600 con 0x7f308c15e0f0 2024-04-17T10:42:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f4f9959a640 1 -- 172.21.15.83:0/4090556574 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f900b2070 con 0x7f4f9c0cf320 2024-04-17T10:42:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f4f9959a640 1 -- 172.21.15.83:0/4090556574 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f900a9070 con 0x7f4f9c0cf320 2024-04-17T10:42:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f4f9959a640 1 -- 172.21.15.83:0/4090556574 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f900ad880 con 0x7f4f9c0cf320 2024-04-17T10:42:58.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7fccebfff640 1 --2- 172.21.15.83:0/608965287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccec06cd30 0x7fccec0cf680 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fccec08c3c0 tx=0x7fcce401f200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b70876163ce26ef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:58.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7fccea7fc640 1 -- 172.21.15.83:0/608965287 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce40b4070 con 0x7fccec06cd30 2024-04-17T10:42:58.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7fccea7fc640 1 -- 172.21.15.83:0/608965287 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce40af400 con 0x7fccec06cd30 2024-04-17T10:42:58.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7fccea7fc640 1 -- 172.21.15.83:0/608965287 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce40af810 con 0x7fccec06cd30 2024-04-17T10:42:58.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f3092302640 1 -- 172.21.15.83:0/13764074 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f308c15e0f0 msgr2=0x7f308c0b6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:58.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f3092302640 1 --2- 172.21.15.83:0/13764074 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f308c15e0f0 0x7f308c0b6d00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f307c0097c0 tx=0x7f307c01c410 comp rx=0 tx=0).stop 2024-04-17T10:42:58.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f4fa0d96640 1 -- 172.21.15.83:0/4090556574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f9c0cf320 msgr2=0x7f4f9c066ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:58.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f4fa0d96640 1 --2- 172.21.15.83:0/4090556574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f9c0cf320 0x7f4f9c066ac0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c0ca4e0 tx=0x7f4f9001c450 comp rx=0 tx=0).stop 2024-04-17T10:42:58.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f2ffaffd640 1 --2- 172.21.15.83:0/1026490694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ffc15bba0 0x7f2ffc0b0c70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2fec0097c0 tx=0x7f2fec01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f54b0739cc669283 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:58.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f3092302640 1 -- 172.21.15.83:0/13764074 shutdown_connections 2024-04-17T10:42:58.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f3092302640 1 --2- 172.21.15.83:0/13764074 >> v2:172.21.15.167:3300/0 conn(0x7f308c0b5450 0x7f308c0b58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:58.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f3092302640 1 --2- 172.21.15.83:0/13764074 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f308c15e0f0 0x7f308c0b6d00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:58.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f3092302640 1 -- 172.21.15.83:0/13764074 >> 172.21.15.83:0/13764074 conn(0x7f308c0d0450 msgr2=0x7f308c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:58.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f4fa0d96640 1 -- 172.21.15.83:0/4090556574 shutdown_connections 2024-04-17T10:42:58.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f4fa0d96640 1 --2- 172.21.15.83:0/4090556574 >> v2:172.21.15.167:3300/0 conn(0x7f4f9c0ca560 0x7f4f9c0ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:58.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f4fa0d96640 1 --2- 172.21.15.83:0/4090556574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f9c0cf320 0x7f4f9c066ac0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:58.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f4fa0d96640 1 -- 172.21.15.83:0/4090556574 >> 172.21.15.83:0/4090556574 conn(0x7f4f9c0cc990 msgr2=0x7f4f9c0ccda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f3092302640 1 -- 172.21.15.83:0/13764074 shutdown_connections 2024-04-17T10:42:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f3092302640 1 -- 172.21.15.83:0/13764074 wait complete. 2024-04-17T10:42:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f4fa0d96640 1 -- 172.21.15.83:0/4090556574 shutdown_connections 2024-04-17T10:42:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f4fa0d96640 1 -- 172.21.15.83:0/4090556574 wait complete. 2024-04-17T10:42:58.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f2ff9ffb640 1 -- 172.21.15.83:0/1026490694 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec0b0070 con 0x7f2ffc15bba0 2024-04-17T10:42:58.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7fccf1ea7640 1 -- 172.21.15.83:0/608965287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccec06cd30 msgr2=0x7fccec0cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:58.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7fccf1ea7640 1 --2- 172.21.15.83:0/608965287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccec06cd30 0x7fccec0cf680 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fccec08c3c0 tx=0x7fcce401f200 comp rx=0 tx=0).stop 2024-04-17T10:42:58.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f2ff9ffb640 1 -- 172.21.15.83:0/1026490694 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fec0ab600 con 0x7f2ffc15bba0 2024-04-17T10:42:58.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f2ff9ffb640 1 -- 172.21.15.83:0/1026490694 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec0b0070 con 0x7f2ffc15bba0 2024-04-17T10:42:58.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f3001878640 1 -- 172.21.15.83:0/1026490694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ffc15bba0 msgr2=0x7f2ffc0b0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:58.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f3001878640 1 --2- 172.21.15.83:0/1026490694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ffc15bba0 0x7f2ffc0b0c70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2fec0097c0 tx=0x7f2fec01c4a0 comp rx=0 tx=0).stop 2024-04-17T10:42:58.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f3092302640 1 Processor -- start 2024-04-17T10:42:58.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f3092302640 1 -- start start 2024-04-17T10:42:58.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f4fa0d96640 1 Processor -- start 2024-04-17T10:42:58.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7fccf1ea7640 1 -- 172.21.15.83:0/608965287 shutdown_connections 2024-04-17T10:42:58.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7fccf1ea7640 1 --2- 172.21.15.83:0/608965287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccec06cd30 0x7fccec0cf680 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:58.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7fccf1ea7640 1 --2- 172.21.15.83:0/608965287 >> v2:172.21.15.167:3300/0 conn(0x7fccec15df90 0x7fccec0d0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:58.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7fccf1ea7640 1 -- 172.21.15.83:0/608965287 >> 172.21.15.83:0/608965287 conn(0x7fccec0d3a60 msgr2=0x7fccec0d3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:58.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f4fa0d96640 1 -- start start 2024-04-17T10:42:58.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f3092302640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f308c0b5450 0x7f308c0cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:58.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f3092302640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f308c0ce400 0x7f308c14e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:58.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f3092302640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f308c15e0f0 0x7f308c14eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:58.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f3092302640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308c0d2ed0 con 0x7f308c0b5450 2024-04-17T10:42:58.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f3092302640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308c0d2d50 con 0x7f308c0ce400 2024-04-17T10:42:58.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f3092302640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f308c0d3050 con 0x7f308c15e0f0 2024-04-17T10:42:58.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f4fa0d96640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f9c06bbf0 0x7f4f9c0a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:58.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f4fa0d96640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4f9c0ca560 0x7f4f9c15af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:58.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f4fa0d96640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f9c0cf320 0x7f4f9c166c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:58.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f4fa0d96640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c16b150 con 0x7f4f9c06bbf0 2024-04-17T10:42:58.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f4fa0d96640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c16afd0 con 0x7f4f9c0cf320 2024-04-17T10:42:58.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f4fa0d96640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c16b2d0 con 0x7f4f9c0ca560 2024-04-17T10:42:58.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f309089f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f308c15e0f0 0x7f308c14eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:58.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f309089f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f308c15e0f0 0x7f308c14eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:39354/0 (socket says 172.21.15.83:39354) 2024-04-17T10:42:58.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f309089f640 1 -- 172.21.15.83:0/2224480277 learned_addr learned my addr 172.21.15.83:0/2224480277 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:58.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f308b7fe640 1 --2- 172.21.15.83:0/2224480277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f308c0ce400 0x7f308c14e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:58.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7fccf1ea7640 1 -- 172.21.15.83:0/608965287 shutdown_connections 2024-04-17T10:42:58.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f308bfff640 1 --2- 172.21.15.83:0/2224480277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f308c0b5450 0x7f308c0cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:58.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7fccf1ea7640 1 -- 172.21.15.83:0/608965287 wait complete. 2024-04-17T10:42:58.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f4f9a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f9c06bbf0 0x7f4f9c0a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:58.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f4f9ad9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f9c0cf320 0x7f4f9c166c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:58.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f4f9a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f9c06bbf0 0x7f4f9c0a0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58270/0 (socket says 172.21.15.83:58270) 2024-04-17T10:42:58.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.273+0000 7f4f9a59c640 1 -- 172.21.15.83:0/3838552539 learned_addr learned my addr 172.21.15.83:0/3838552539 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:58.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f4f99d9b640 1 --2- 172.21.15.83:0/3838552539 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4f9c0ca560 0x7f4f9c15af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:58.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f309089f640 1 -- 172.21.15.83:0/2224480277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f308c0ce400 msgr2=0x7f308c14e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:58.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f309089f640 1 --2- 172.21.15.83:0/2224480277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f308c0ce400 0x7f308c14e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:58.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f309089f640 1 -- 172.21.15.83:0/2224480277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f308c0b5450 msgr2=0x7f308c0cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:58.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f309089f640 1 --2- 172.21.15.83:0/2224480277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f308c0b5450 0x7f308c0cd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:58.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f309089f640 1 -- 172.21.15.83:0/2224480277 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308c0b8510 con 0x7f308c15e0f0 2024-04-17T10:42:58.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f3001878640 1 -- 172.21.15.83:0/1026490694 shutdown_connections 2024-04-17T10:42:58.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f3001878640 1 --2- 172.21.15.83:0/1026490694 >> v2:172.21.15.83:3300/0 conn(0x7f2ffc0cfaa0 0x7f2ffc0cfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f3001878640 1 --2- 172.21.15.83:0/1026490694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ffc15bba0 0x7f2ffc0b0c70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f3001878640 1 -- 172.21.15.83:0/1026490694 >> 172.21.15.83:0/1026490694 conn(0x7f2ffc0d1810 msgr2=0x7f2ffc0d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f4f9a59c640 1 -- 172.21.15.83:0/3838552539 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4f9c0ca560 msgr2=0x7f4f9c15af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f4f9a59c640 1 --2- 172.21.15.83:0/3838552539 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4f9c0ca560 0x7f4f9c15af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f4f9a59c640 1 -- 172.21.15.83:0/3838552539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f9c0cf320 msgr2=0x7f4f9c166c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f4f9a59c640 1 --2- 172.21.15.83:0/3838552539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f9c0cf320 0x7f4f9c166c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f4f9a59c640 1 -- 172.21.15.83:0/3838552539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f9c15fda0 con 0x7f4f9c06bbf0 2024-04-17T10:42:58.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f3001878640 1 -- 172.21.15.83:0/1026490694 shutdown_connections 2024-04-17T10:42:58.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f3001878640 1 -- 172.21.15.83:0/1026490694 wait complete. 2024-04-17T10:42:58.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f309089f640 1 --2- 172.21.15.83:0/2224480277 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f308c15e0f0 0x7f308c14eb40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f308400efc0 tx=0x7f308400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:58.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f4f9a59c640 1 --2- 172.21.15.83:0/3838552539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f9c06bbf0 0x7f4f9c0a0d00 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00b410 tx=0x7f4f8c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:58.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f3088ff9640 1 -- 172.21.15.83:0/2224480277 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3084018020 con 0x7f308c15e0f0 2024-04-17T10:42:58.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f3092302640 1 -- 172.21.15.83:0/2224480277 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f308c0b87f0 con 0x7f308c15e0f0 2024-04-17T10:42:58.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f3088ff9640 1 -- 172.21.15.83:0/2224480277 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3084014480 con 0x7f308c15e0f0 2024-04-17T10:42:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f4f8affd640 1 -- 172.21.15.83:0/3838552539 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c015030 con 0x7f4f9c06bbf0 2024-04-17T10:42:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f4fa0d96640 1 -- 172.21.15.83:0/3838552539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f9c09f860 con 0x7f4f9c06bbf0 2024-04-17T10:42:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f4fa0d96640 1 -- 172.21.15.83:0/3838552539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f9c160040 con 0x7f4f9c06bbf0 2024-04-17T10:42:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f3088ff9640 1 -- 172.21.15.83:0/2224480277 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3084014870 con 0x7f308c15e0f0 2024-04-17T10:42:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f3092302640 1 -- 172.21.15.83:0/2224480277 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f308c14e870 con 0x7f308c15e0f0 2024-04-17T10:42:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f3092302640 1 -- 172.21.15.83:0/2224480277 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f308c0681a0 con 0x7f308c15e0f0 2024-04-17T10:42:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f3088ff9640 1 -- 172.21.15.83:0/2224480277 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f308401b020 con 0x7f308c15e0f0 2024-04-17T10:42:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.281+0000 7f3088ff9640 1 --2- 172.21.15.83:0/2224480277 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3078066f90 0x7f3078069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.281+0000 7f3088ff9640 1 -- 172.21.15.83:0/2224480277 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3084085750 con 0x7f308c15e0f0 2024-04-17T10:42:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.281+0000 7f308bfff640 1 --2- 172.21.15.83:0/2224480277 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3078066f90 0x7f3078069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.281+0000 7f308bfff640 1 --2- 172.21.15.83:0/2224480277 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3078066f90 0x7f3078069450 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f307c0097c0 tx=0x7f307c0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f4f8affd640 1 -- 172.21.15.83:0/3838552539 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f8c004020 con 0x7f4f9c06bbf0 2024-04-17T10:42:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f4f8affd640 1 -- 172.21.15.83:0/3838552539 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c004410 con 0x7f4f9c06bbf0 2024-04-17T10:42:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f4fa0d96640 1 -- 172.21.15.83:0/3838552539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f800052a0 con 0x7f4f9c06bbf0 2024-04-17T10:42:58.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f4f8affd640 1 -- 172.21.15.83:0/3838552539 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4f8c018070 con 0x7f4f9c06bbf0 2024-04-17T10:42:58.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f4f8affd640 1 --2- 172.21.15.83:0/3838552539 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4f7c066fb0 0x7f4f7c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:58.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.281+0000 7f4f99d9b640 1 --2- 172.21.15.83:0/3838552539 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4f7c066fb0 0x7f4f7c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:58.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.281+0000 7f4f99d9b640 1 --2- 172.21.15.83:0/3838552539 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4f7c066fb0 0x7f4f7c069470 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4f900af000 tx=0x7f4f900a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:58.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7fccf1ea7640 1 Processor -- start 2024-04-17T10:42:58.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f3001878640 1 Processor -- start 2024-04-17T10:42:58.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f3001878640 1 -- start start 2024-04-17T10:42:58.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f3001878640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ffc0cfaa0 0x7f2ffc15fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:58.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f3001878640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ffc0d2300 0x7f2ffc0ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:58.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f3001878640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ffc15bba0 0x7f2ffc151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:58.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f3001878640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc0cee00 con 0x7f2ffc0d2300 2024-04-17T10:42:58.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f3001878640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc0cec80 con 0x7f2ffc0cfaa0 2024-04-17T10:42:58.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f3001878640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc0cef80 con 0x7f2ffc15bba0 2024-04-17T10:42:58.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f2ffa7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ffc0d2300 0x7f2ffc0ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:58.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f2ffa7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ffc0d2300 0x7f2ffc0ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58286/0 (socket says 172.21.15.83:58286) 2024-04-17T10:42:58.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f2ffa7fc640 1 -- 172.21.15.83:0/3752092180 learned_addr learned my addr 172.21.15.83:0/3752092180 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:58.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f2ffaffd640 1 --2- 172.21.15.83:0/3752092180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ffc0cfaa0 0x7f2ffc15fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:58.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.277+0000 7f2ffb7fe640 1 --2- 172.21.15.83:0/3752092180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ffc15bba0 0x7f2ffc151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:58.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.281+0000 7f2ffa7fc640 1 -- 172.21.15.83:0/3752092180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ffc15bba0 msgr2=0x7f2ffc151a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:58.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.281+0000 7f2ffa7fc640 1 --2- 172.21.15.83:0/3752092180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ffc15bba0 0x7f2ffc151a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:58.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.281+0000 7f2ffa7fc640 1 -- 172.21.15.83:0/3752092180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ffc0cfaa0 msgr2=0x7f2ffc15fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:58.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.281+0000 7f2ffa7fc640 1 --2- 172.21.15.83:0/3752092180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ffc0cfaa0 0x7f2ffc15fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:58.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.281+0000 7f2ffa7fc640 1 -- 172.21.15.83:0/3752092180 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ffc0cac20 con 0x7f2ffc0d2300 2024-04-17T10:42:58.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.281+0000 7f2ffa7fc640 1 --2- 172.21.15.83:0/3752092180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ffc0d2300 0x7f2ffc0ca6e0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000c480 tx=0x7f2ff000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:58.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.285+0000 7fccf1ea7640 1 -- start start 2024-04-17T10:42:58.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.285+0000 7f4f8affd640 1 -- 172.21.15.83:0/3838552539 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4f8c083050 con 0x7f4f9c06bbf0 2024-04-17T10:42:58.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.285+0000 7f2fe3fff640 1 -- 172.21.15.83:0/3752092180 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0017070 con 0x7f2ffc0d2300 2024-04-17T10:42:58.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.285+0000 7f3001878640 1 -- 172.21.15.83:0/3752092180 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ffc0caee0 con 0x7f2ffc0d2300 2024-04-17T10:42:58.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.285+0000 7f2fe3fff640 1 -- 172.21.15.83:0/3752092180 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff0005db0 con 0x7f2ffc0d2300 2024-04-17T10:42:58.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.285+0000 7fccf1ea7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccec06cd30 0x7fccec0b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:58.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.285+0000 7fccf1ea7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fccec15df90 0x7fccec0b9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:58.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.285+0000 7f4f8affd640 1 -- 172.21.15.83:0/3838552539 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4f8c055a20 con 0x7f4f9c06bbf0 2024-04-17T10:42:58.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.285+0000 7fccf1ea7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccec15c8e0 0x7fccec15fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:58.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.285+0000 7fccf1ea7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec15e4f0 con 0x7fccec06cd30 2024-04-17T10:42:58.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.285+0000 7fccf1ea7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec15e370 con 0x7fccec15c8e0 2024-04-17T10:42:58.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.285+0000 7fccf1ea7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccec06c770 con 0x7fccec15df90 2024-04-17T10:42:58.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.285+0000 7fcceb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccec06cd30 0x7fccec0b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:58.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.285+0000 7fcceb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccec06cd30 0x7fccec0b8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58296/0 (socket says 172.21.15.83:58296) 2024-04-17T10:42:58.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.285+0000 7fcceb7fe640 1 -- 172.21.15.83:0/2289983565 learned_addr learned my addr 172.21.15.83:0/2289983565 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:42:58.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.285+0000 7fcceaffd640 1 --2- 172.21.15.83:0/2289983565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fccec15df90 0x7fccec0b9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:58.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.285+0000 7fccebfff640 1 --2- 172.21.15.83:0/2289983565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccec15c8e0 0x7fccec15fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:58.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.285+0000 7fcceb7fe640 1 -- 172.21.15.83:0/2289983565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fccec15df90 msgr2=0x7fccec0b9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:58.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.285+0000 7fcceb7fe640 1 --2- 172.21.15.83:0/2289983565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fccec15df90 0x7fccec0b9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:58.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.285+0000 7fcceb7fe640 1 -- 172.21.15.83:0/2289983565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccec15c8e0 msgr2=0x7fccec15fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:58.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.285+0000 7fcceb7fe640 1 --2- 172.21.15.83:0/2289983565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccec15c8e0 0x7fccec15fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:58.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.285+0000 7fcceb7fe640 1 -- 172.21.15.83:0/2289983565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccec1609d0 con 0x7fccec06cd30 2024-04-17T10:42:58.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.285+0000 7fcceb7fe640 1 --2- 172.21.15.83:0/2289983565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccec06cd30 0x7fccec0b8ae0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00e870 tx=0x7fccdc00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:58.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.285+0000 7f3001878640 1 -- 172.21.15.83:0/3752092180 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ffc0cb4a0 con 0x7f2ffc0d2300 2024-04-17T10:42:58.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.285+0000 7f3001878640 1 -- 172.21.15.83:0/3752092180 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ffc074f90 con 0x7f2ffc0d2300 2024-04-17T10:42:58.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.285+0000 7f2fe3fff640 1 -- 172.21.15.83:0/3752092180 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0021060 con 0x7f2ffc0d2300 2024-04-17T10:42:58.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.285+0000 7f2fe3fff640 1 -- 172.21.15.83:0/3752092180 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2ff001a070 con 0x7f2ffc0d2300 2024-04-17T10:42:58.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.285+0000 7fccd3fff640 1 -- 172.21.15.83:0/2289983565 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc017070 con 0x7fccec06cd30 2024-04-17T10:42:58.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.285+0000 7fccd3fff640 1 -- 172.21.15.83:0/2289983565 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccdc00c940 con 0x7fccec06cd30 2024-04-17T10:42:58.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.285+0000 7fccd3fff640 1 -- 172.21.15.83:0/2289983565 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc00ccf0 con 0x7fccec06cd30 2024-04-17T10:42:58.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.289+0000 7fccf1ea7640 1 -- 172.21.15.83:0/2289983565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccec160cb0 con 0x7fccec06cd30 2024-04-17T10:42:58.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.289+0000 7fccf1ea7640 1 -- 172.21.15.83:0/2289983565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccec0b4600 con 0x7fccec06cd30 2024-04-17T10:42:58.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.289+0000 7f2fe3fff640 1 --2- 172.21.15.83:0/3752092180 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2fe8066f90 0x7f2fe8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:58.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.289+0000 7f2fe3fff640 1 -- 172.21.15.83:0/3752092180 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ff0084f30 con 0x7f2ffc0d2300 2024-04-17T10:42:58.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.289+0000 7f2ffaffd640 1 --2- 172.21.15.83:0/3752092180 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2fe8066f90 0x7f2fe8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:58.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.289+0000 7f2ffaffd640 1 --2- 172.21.15.83:0/3752092180 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2fe8066f90 0x7f2fe8069450 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2fec0ad000 tx=0x7f2fec0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:58.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.289+0000 7fccf1ea7640 1 -- 172.21.15.83:0/2289983565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccec0681a0 con 0x7fccec06cd30 2024-04-17T10:42:58.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.293+0000 7f2fe3fff640 1 -- 172.21.15.83:0/3752092180 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2ff00558f0 con 0x7f2ffc0d2300 2024-04-17T10:42:58.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.293+0000 7f3088ff9640 1 -- 172.21.15.83:0/2224480277 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3084010040 con 0x7f308c15e0f0 2024-04-17T10:42:58.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.293+0000 7fccd3fff640 1 -- 172.21.15.83:0/2289983565 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fccdc01a070 con 0x7fccec06cd30 2024-04-17T10:42:58.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.293+0000 7fccd3fff640 1 --2- 172.21.15.83:0/2289983565 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fccd8066fb0 0x7fccd8069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:42:58.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.293+0000 7fccd3fff640 1 -- 172.21.15.83:0/2289983565 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fccdc085610 con 0x7fccec06cd30 2024-04-17T10:42:58.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.293+0000 7fcceaffd640 1 --2- 172.21.15.83:0/2289983565 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fccd8066fb0 0x7fccd8069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:42:58.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.293+0000 7fcceaffd640 1 --2- 172.21.15.83:0/2289983565 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fccd8066fb0 0x7fccd8069470 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fccec0c72a0 tx=0x7fcce0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:42:58.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.301+0000 7fccd3fff640 1 -- 172.21.15.83:0/2289983565 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fccdc056040 con 0x7fccec06cd30 2024-04-17T10:42:58.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.509+0000 7f4fa0d96640 1 -- 172.21.15.83:0/3838552539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) v1 -- 0x7f4f80005590 con 0x7f4f9c06bbf0 2024-04-17T10:42:58.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.513+0000 7f3092302640 1 -- 172.21.15.83:0/2224480277 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) v1 -- 0x7f308c074f90 con 0x7f308c15e0f0 2024-04-17T10:42:58.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.621+0000 7fccf1ea7640 1 -- 172.21.15.83:0/2289983565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) v1 -- 0x7fccec074f90 con 0x7fccec06cd30 2024-04-17T10:42:58.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:58.629+0000 7f3001878640 1 -- 172.21.15.83:0/3752092180 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "d"} v 0) v1 -- 0x7f2ffc0cdda0 con 0x7f2ffc0d2300 2024-04-17T10:42:59.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.325+0000 7f4f8affd640 1 -- 172.21.15.83:0/3838552539 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 6522 v33) v1 ==== 95+0+0 (secure 0 0 0) 0x7f4f8c055320 con 0x7f4f9c06bbf0 2024-04-17T10:42:59.321 INFO:teuthology.orchestra.run.smithi083.stderr:failed mds gid 6522 2024-04-17T10:42:59.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.329+0000 7f4fa0d96640 1 -- 172.21.15.83:0/3838552539 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4f7c066fb0 msgr2=0x7f4f7c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:59.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.329+0000 7f4fa0d96640 1 --2- 172.21.15.83:0/3838552539 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4f7c066fb0 0x7f4f7c069470 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4f900af000 tx=0x7f4f900a7000 comp rx=0 tx=0).stop 2024-04-17T10:42:59.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.329+0000 7f4fa0d96640 1 -- 172.21.15.83:0/3838552539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f9c06bbf0 msgr2=0x7f4f9c0a0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:59.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.329+0000 7f4fa0d96640 1 --2- 172.21.15.83:0/3838552539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f9c06bbf0 0x7f4f9c0a0d00 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00b410 tx=0x7f4f8c00b8d0 comp rx=0 tx=0).stop 2024-04-17T10:42:59.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.329+0000 7f4fa0d96640 1 -- 172.21.15.83:0/3838552539 shutdown_connections 2024-04-17T10:42:59.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.329+0000 7f4fa0d96640 1 --2- 172.21.15.83:0/3838552539 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4f7c066fb0 0x7f4f7c069470 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:59.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.329+0000 7f4fa0d96640 1 --2- 172.21.15.83:0/3838552539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f9c0cf320 0x7f4f9c166c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:59.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.329+0000 7f4fa0d96640 1 --2- 172.21.15.83:0/3838552539 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4f9c0ca560 0x7f4f9c15af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:59.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.329+0000 7f4fa0d96640 1 --2- 172.21.15.83:0/3838552539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f9c06bbf0 0x7f4f9c0a0d00 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:59.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.329+0000 7f4fa0d96640 1 -- 172.21.15.83:0/3838552539 >> 172.21.15.83:0/3838552539 conn(0x7f4f9c0cc990 msgr2=0x7f4f9c0b0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:59.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.329+0000 7f4fa0d96640 1 -- 172.21.15.83:0/3838552539 shutdown_connections 2024-04-17T10:42:59.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.329+0000 7f4fa0d96640 1 -- 172.21.15.83:0/3838552539 wait complete. 2024-04-17T10:42:59.341 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-04-17T10:42:59.345+0000 7f35c137c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T10:42:59.347 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-17T10:42:59.348 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-17T10:42:59.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.353+0000 7fccd3fff640 1 -- 172.21.15.83:0/2289983565 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 6534 v34) v1 ==== 95+0+0 (secure 0 0 0) 0x7fccdc05a0e0 con 0x7fccec06cd30 2024-04-17T10:42:59.349 INFO:teuthology.orchestra.run.smithi083.stderr:failed mds gid 6534 2024-04-17T10:42:59.350 INFO:tasks.ceph.mds.c:Started 2024-04-17T10:42:59.350 DEBUG:teuthology.parallel:result is None 2024-04-17T10:42:59.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.357+0000 7fccf1ea7640 1 -- 172.21.15.83:0/2289983565 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fccd8066fb0 msgr2=0x7fccd8069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:59.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.357+0000 7fccf1ea7640 1 --2- 172.21.15.83:0/2289983565 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fccd8066fb0 0x7fccd8069470 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fccec0c72a0 tx=0x7fcce0009040 comp rx=0 tx=0).stop 2024-04-17T10:42:59.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.357+0000 7fccf1ea7640 1 -- 172.21.15.83:0/2289983565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccec06cd30 msgr2=0x7fccec0b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:59.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.357+0000 7fccf1ea7640 1 --2- 172.21.15.83:0/2289983565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccec06cd30 0x7fccec0b8ae0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00e870 tx=0x7fccdc00ec70 comp rx=0 tx=0).stop 2024-04-17T10:42:59.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.357+0000 7fccf1ea7640 1 -- 172.21.15.83:0/2289983565 shutdown_connections 2024-04-17T10:42:59.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.357+0000 7fccf1ea7640 1 --2- 172.21.15.83:0/2289983565 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fccd8066fb0 0x7fccd8069470 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:59.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.357+0000 7fccf1ea7640 1 --2- 172.21.15.83:0/2289983565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccec15c8e0 0x7fccec15fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:59.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.357+0000 7fccf1ea7640 1 --2- 172.21.15.83:0/2289983565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fccec15df90 0x7fccec0b9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:59.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.357+0000 7fccf1ea7640 1 --2- 172.21.15.83:0/2289983565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccec06cd30 0x7fccec0b8ae0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:59.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.357+0000 7fccf1ea7640 1 -- 172.21.15.83:0/2289983565 >> 172.21.15.83:0/2289983565 conn(0x7fccec0d3a60 msgr2=0x7fccec0c7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:59.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.357+0000 7fccf1ea7640 1 -- 172.21.15.83:0/2289983565 shutdown_connections 2024-04-17T10:42:59.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.357+0000 7fccf1ea7640 1 -- 172.21.15.83:0/2289983565 wait complete. 2024-04-17T10:42:59.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.361+0000 7f3088ff9640 1 -- 172.21.15.83:0/2224480277 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 6525 v34) v1 ==== 95+0+0 (secure 0 0 0) 0x7f3084056080 con 0x7f308c15e0f0 2024-04-17T10:42:59.356 INFO:teuthology.orchestra.run.smithi083.stderr:failed mds gid 6525 2024-04-17T10:42:59.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.361+0000 7f3092302640 1 -- 172.21.15.83:0/2224480277 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3078066f90 msgr2=0x7f3078069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:59.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.361+0000 7f3092302640 1 --2- 172.21.15.83:0/2224480277 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3078066f90 0x7f3078069450 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f307c0097c0 tx=0x7f307c0a6000 comp rx=0 tx=0).stop 2024-04-17T10:42:59.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.361+0000 7f3092302640 1 -- 172.21.15.83:0/2224480277 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f308c15e0f0 msgr2=0x7f308c14eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:42:59.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.361+0000 7f3092302640 1 --2- 172.21.15.83:0/2224480277 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f308c15e0f0 0x7f308c14eb40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f308400efc0 tx=0x7f308400c490 comp rx=0 tx=0).stop 2024-04-17T10:42:59.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.361+0000 7f3092302640 1 -- 172.21.15.83:0/2224480277 shutdown_connections 2024-04-17T10:42:59.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.361+0000 7f3092302640 1 --2- 172.21.15.83:0/2224480277 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3078066f90 0x7f3078069450 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:59.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.361+0000 7f3092302640 1 --2- 172.21.15.83:0/2224480277 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f308c15e0f0 0x7f308c14eb40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:59.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.361+0000 7f3092302640 1 --2- 172.21.15.83:0/2224480277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f308c0ce400 0x7f308c14e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:59.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.361+0000 7f3092302640 1 --2- 172.21.15.83:0/2224480277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f308c0b5450 0x7f308c0cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:42:59.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.361+0000 7f3092302640 1 -- 172.21.15.83:0/2224480277 >> 172.21.15.83:0/2224480277 conn(0x7f308c0d0450 msgr2=0x7f308c153130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:42:59.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.365+0000 7f3092302640 1 -- 172.21.15.83:0/2224480277 shutdown_connections 2024-04-17T10:42:59.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:42:59.365+0000 7f3092302640 1 -- 172.21.15.83:0/2224480277 wait complete. 2024-04-17T10:42:59.376 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-17T10:42:59.376 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-17T10:42:59.382 INFO:tasks.ceph.mds.b:Started 2024-04-17T10:42:59.383 DEBUG:teuthology.parallel:result is None 2024-04-17T10:42:59.384 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-17T10:42:59.384 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-17T10:42:59.415 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-04-17T10:42:59.416 INFO:tasks.ceph.mds.a:Started 2024-04-17T10:42:59.417 DEBUG:teuthology.parallel:result is None 2024-04-17T10:42:59.454 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2024-04-17T10:42:59.471 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-04-17T10:43:00.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.357+0000 7f2fe3fff640 1 -- 172.21.15.83:0/3752092180 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "d"}]=0 failed mds gid 5398 v35) v1 ==== 95+0+0 (secure 0 0 0) 0x7f2ff0059990 con 0x7f2ffc0d2300 2024-04-17T10:43:00.449 INFO:teuthology.orchestra.run.smithi083.stderr:failed mds gid 5398 2024-04-17T10:43:00.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.357+0000 7f3001878640 1 -- 172.21.15.83:0/3752092180 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2fe8066f90 msgr2=0x7f2fe8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:00.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.357+0000 7f3001878640 1 --2- 172.21.15.83:0/3752092180 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2fe8066f90 0x7f2fe8069450 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2fec0ad000 tx=0x7f2fec0a6000 comp rx=0 tx=0).stop 2024-04-17T10:43:00.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.357+0000 7f3001878640 1 -- 172.21.15.83:0/3752092180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ffc0d2300 msgr2=0x7f2ffc0ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:00.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.357+0000 7f3001878640 1 --2- 172.21.15.83:0/3752092180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ffc0d2300 0x7f2ffc0ca6e0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000c480 tx=0x7f2ff000c940 comp rx=0 tx=0).stop 2024-04-17T10:43:00.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.357+0000 7f3001878640 1 -- 172.21.15.83:0/3752092180 shutdown_connections 2024-04-17T10:43:00.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.357+0000 7f3001878640 1 --2- 172.21.15.83:0/3752092180 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2fe8066f90 0x7f2fe8069450 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:00.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.357+0000 7f3001878640 1 --2- 172.21.15.83:0/3752092180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ffc15bba0 0x7f2ffc151a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:00.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.357+0000 7f3001878640 1 --2- 172.21.15.83:0/3752092180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ffc0d2300 0x7f2ffc0ca6e0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:00.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.357+0000 7f3001878640 1 --2- 172.21.15.83:0/3752092180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ffc0cfaa0 0x7f2ffc15fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:00.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.357+0000 7f3001878640 1 -- 172.21.15.83:0/3752092180 >> 172.21.15.83:0/3752092180 conn(0x7f2ffc0d1810 msgr2=0x7f2ffc0ccf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:00.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.357+0000 7f3001878640 1 -- 172.21.15.83:0/3752092180 shutdown_connections 2024-04-17T10:43:00.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.357+0000 7f3001878640 1 -- 172.21.15.83:0/3752092180 wait complete. 2024-04-17T10:43:00.453 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-17T10:43:00.453 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-17T10:43:00.461 INFO:tasks.ceph.mds.d:Started 2024-04-17T10:43:00.462 DEBUG:teuthology.parallel:result is None 2024-04-17T10:43:00.462 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:43:00.537 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2024-04-17T10:43:00.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.621+0000 7f87b2498640 1 Processor -- start 2024-04-17T10:43:00.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.621+0000 7f87b2498640 1 -- start start 2024-04-17T10:43:00.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.621+0000 7f87b2498640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f87ac15db50 0x7f87ac070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:00.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.621+0000 7f87b2498640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f87ac0aedd0 0x7f87ac06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:00.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.621+0000 7f87b2498640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87ac0709b0 con 0x7f87ac06dae0 2024-04-17T10:43:00.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.621+0000 7f87b2498640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87ac15e440 con 0x7f87ac15db50 2024-04-17T10:43:00.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.621+0000 7f87b2498640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87ac15e720 con 0x7f87ac0aedd0 2024-04-17T10:43:00.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.621+0000 7f87abfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f87ac15db50 0x7f87ac070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:00.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.621+0000 7f87abfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f87ac15db50 0x7f87ac070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58342/0 (socket says 172.21.15.83:58342) 2024-04-17T10:43:00.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.621+0000 7f87abfff640 1 -- 172.21.15.83:0/1057512767 learned_addr learned my addr 172.21.15.83:0/1057512767 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:00.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.621+0000 7f87ab7fe640 1 --2- 172.21.15.83:0/1057512767 >> v2:172.21.15.167:3300/0 conn(0x7f87ac0aedd0 0x7f87ac06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:00.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.621+0000 7f87abfff640 1 -- 172.21.15.83:0/1057512767 >> v2:172.21.15.167:3300/0 conn(0x7f87ac0aedd0 msgr2=0x7f87ac06d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:00.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.621+0000 7f87abfff640 1 --2- 172.21.15.83:0/1057512767 >> v2:172.21.15.167:3300/0 conn(0x7f87ac0aedd0 0x7f87ac06d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:00.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.621+0000 7f87abfff640 1 -- 172.21.15.83:0/1057512767 >> v1:172.21.15.167:6789/0 conn(0x7f87ac06dae0 legacy=0x7f87ac0cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:00.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.621+0000 7f87abfff640 1 -- 172.21.15.83:0/1057512767 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ac15ea00 con 0x7f87ac15db50 2024-04-17T10:43:00.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.625+0000 7f87abfff640 1 --2- 172.21.15.83:0/1057512767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87ac15db50 0x7f87ac070470 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f879c009870 tx=0x7f879c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1856dbc77bca5ce server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:00.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.625+0000 7f87aaffd640 1 -- 172.21.15.83:0/1057512767 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879c0ab610 con 0x7f87ac15db50 2024-04-17T10:43:00.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.625+0000 7f87aaffd640 1 -- 172.21.15.83:0/1057512767 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f879c0b0070 con 0x7f87ac15db50 2024-04-17T10:43:00.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.625+0000 7f87aaffd640 1 -- 172.21.15.83:0/1057512767 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879c0ab9d0 con 0x7f87ac15db50 2024-04-17T10:43:00.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.625+0000 7f87b2498640 1 -- 172.21.15.83:0/1057512767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87ac15db50 msgr2=0x7f87ac070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:00.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.625+0000 7f87b2498640 1 --2- 172.21.15.83:0/1057512767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87ac15db50 0x7f87ac070470 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f879c009870 tx=0x7f879c01c520 comp rx=0 tx=0).stop 2024-04-17T10:43:00.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.625+0000 7f87b2498640 1 -- 172.21.15.83:0/1057512767 shutdown_connections 2024-04-17T10:43:00.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.625+0000 7f87b2498640 1 --2- 172.21.15.83:0/1057512767 >> v2:172.21.15.167:3300/0 conn(0x7f87ac0aedd0 0x7f87ac06d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:00.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.625+0000 7f87b2498640 1 --2- 172.21.15.83:0/1057512767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87ac15db50 0x7f87ac070470 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:00.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.625+0000 7f87b2498640 1 -- 172.21.15.83:0/1057512767 >> 172.21.15.83:0/1057512767 conn(0x7f87ac06abe0 msgr2=0x7f87ac06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:00.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.625+0000 7f87b2498640 1 -- 172.21.15.83:0/1057512767 shutdown_connections 2024-04-17T10:43:00.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.625+0000 7f87b2498640 1 -- 172.21.15.83:0/1057512767 wait complete. 2024-04-17T10:43:00.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.625+0000 7f87b2498640 1 Processor -- start 2024-04-17T10:43:00.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.625+0000 7f87b2498640 1 -- start start 2024-04-17T10:43:00.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.625+0000 7f87b2498640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87ac06dae0 0x7f87ac0a9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:00.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.625+0000 7f87b2498640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87ac0aedd0 0x7f87ac14e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:00.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.625+0000 7f87b2498640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87ac15db50 0x7f87ac1568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:00.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.625+0000 7f87b2498640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac1677e0 con 0x7f87ac0aedd0 2024-04-17T10:43:00.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.625+0000 7f87b2498640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac167660 con 0x7f87ac06dae0 2024-04-17T10:43:00.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.625+0000 7f87b2498640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac167960 con 0x7f87ac15db50 2024-04-17T10:43:00.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.625+0000 7f87ab7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87ac0aedd0 0x7f87ac14e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:00.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.629+0000 7f87ab7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87ac0aedd0 0x7f87ac14e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58346/0 (socket says 172.21.15.83:58346) 2024-04-17T10:43:00.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.629+0000 7f87ab7fe640 1 -- 172.21.15.83:0/3126567814 learned_addr learned my addr 172.21.15.83:0/3126567814 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:00.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.629+0000 7f87abfff640 1 --2- 172.21.15.83:0/3126567814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87ac06dae0 0x7f87ac0a9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:00.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.629+0000 7f87b0a35640 1 --2- 172.21.15.83:0/3126567814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87ac15db50 0x7f87ac1568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:00.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.629+0000 7f87ab7fe640 1 -- 172.21.15.83:0/3126567814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87ac15db50 msgr2=0x7f87ac1568c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:00.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.629+0000 7f87ab7fe640 1 --2- 172.21.15.83:0/3126567814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87ac15db50 0x7f87ac1568c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:00.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.629+0000 7f87ab7fe640 1 -- 172.21.15.83:0/3126567814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87ac06dae0 msgr2=0x7f87ac0a9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:00.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.629+0000 7f87ab7fe640 1 --2- 172.21.15.83:0/3126567814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87ac06dae0 0x7f87ac0a9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:00.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.629+0000 7f87ab7fe640 1 -- 172.21.15.83:0/3126567814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ac0ab8c0 con 0x7f87ac0aedd0 2024-04-17T10:43:00.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.629+0000 7f87abfff640 1 --2- 172.21.15.83:0/3126567814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87ac06dae0 0x7f87ac0a9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:00.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.629+0000 7f87ab7fe640 1 --2- 172.21.15.83:0/3126567814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87ac0aedd0 0x7f87ac14e840 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f87a000d800 tx=0x7f87a000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:00.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.629+0000 7f87a8ff9640 1 -- 172.21.15.83:0/3126567814 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a000b6c0 con 0x7f87ac0aedd0 2024-04-17T10:43:00.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.629+0000 7f87a8ff9640 1 -- 172.21.15.83:0/3126567814 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a0019070 con 0x7f87ac0aedd0 2024-04-17T10:43:00.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.629+0000 7f87b2498640 1 -- 172.21.15.83:0/3126567814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87ac0abba0 con 0x7f87ac0aedd0 2024-04-17T10:43:00.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.629+0000 7f87a8ff9640 1 -- 172.21.15.83:0/3126567814 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a000bae0 con 0x7f87ac0aedd0 2024-04-17T10:43:00.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.629+0000 7f87b2498640 1 -- 172.21.15.83:0/3126567814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ac156e00 con 0x7f87ac0aedd0 2024-04-17T10:43:00.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.629+0000 7f87b2498640 1 -- 172.21.15.83:0/3126567814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87ac074f90 con 0x7f87ac0aedd0 2024-04-17T10:43:00.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.629+0000 7f87a8ff9640 1 -- 172.21.15.83:0/3126567814 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f87a000bc80 con 0x7f87ac0aedd0 2024-04-17T10:43:00.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.629+0000 7f87a8ff9640 1 --2- 172.21.15.83:0/3126567814 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8798066f60 0x7f8798069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:00.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.629+0000 7f87abfff640 1 --2- 172.21.15.83:0/3126567814 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8798066f60 0x7f8798069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:00.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.629+0000 7f87a8ff9640 1 -- 172.21.15.83:0/3126567814 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f87a00854f0 con 0x7f87ac0aedd0 2024-04-17T10:43:00.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.629+0000 7f87abfff640 1 --2- 172.21.15.83:0/3126567814 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8798066f60 0x7f8798069420 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f879c0af000 tx=0x7f879c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:00.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.633+0000 7f87a8ff9640 1 -- 172.21.15.83:0/3126567814 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f87a0056530 con 0x7f87ac0aedd0 2024-04-17T10:43:00.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.813+0000 7f87b2498640 1 -- 172.21.15.83:0/3126567814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87ac0b2af0 con 0x7f87ac0aedd0 2024-04-17T10:43:00.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.813+0000 7f87a8ff9640 1 -- 172.21.15.83:0/3126567814 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4148 (secure 0 0 0) 0x7f87a0055e30 con 0x7f87ac0aedd0 2024-04-17T10:43:00.811 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:43:00.811 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5165,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1924992370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1924992370},{"type":"v1","addr":"172.21.15.167:6833","nonce":1924992370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6831,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/338926944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":338926944},{"type":"v1","addr":"172.21.15.83:6835","nonce":338926944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:42:34.328397+0000","modified":"2024-04-17T10:43:00.362453+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6843},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6843":{"gid":6843,"name":"a","rank":0,"incarnation":36,"state":"up:replay","state_seq":1,"addr":"172.21.15.83:6837/3985646487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3985646487},{"type":"v1","addr":"172.21.15.83:6837","nonce":3985646487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-17T10:43:00.812 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 36 2024-04-17T10:43:00.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.817+0000 7f87b2498640 1 -- 172.21.15.83:0/3126567814 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8798066f60 msgr2=0x7f8798069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:00.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.817+0000 7f87b2498640 1 --2- 172.21.15.83:0/3126567814 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8798066f60 0x7f8798069420 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f879c0af000 tx=0x7f879c008040 comp rx=0 tx=0).stop 2024-04-17T10:43:00.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.817+0000 7f87b2498640 1 -- 172.21.15.83:0/3126567814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87ac0aedd0 msgr2=0x7f87ac14e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:00.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.817+0000 7f87b2498640 1 --2- 172.21.15.83:0/3126567814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87ac0aedd0 0x7f87ac14e840 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f87a000d800 tx=0x7f87a000dcc0 comp rx=0 tx=0).stop 2024-04-17T10:43:00.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.817+0000 7f87b2498640 1 -- 172.21.15.83:0/3126567814 shutdown_connections 2024-04-17T10:43:00.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.817+0000 7f87b2498640 1 --2- 172.21.15.83:0/3126567814 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8798066f60 0x7f8798069420 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:00.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.817+0000 7f87b2498640 1 --2- 172.21.15.83:0/3126567814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87ac15db50 0x7f87ac1568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:00.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.817+0000 7f87b2498640 1 --2- 172.21.15.83:0/3126567814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87ac0aedd0 0x7f87ac14e840 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:00.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.817+0000 7f87b2498640 1 --2- 172.21.15.83:0/3126567814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87ac06dae0 0x7f87ac0a9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:00.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.817+0000 7f87b2498640 1 -- 172.21.15.83:0/3126567814 >> 172.21.15.83:0/3126567814 conn(0x7f87ac06abe0 msgr2=0x7f87ac0b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:00.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.817+0000 7f87b2498640 1 -- 172.21.15.83:0/3126567814 shutdown_connections 2024-04-17T10:43:00.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:00.817+0000 7f87b2498640 1 -- 172.21.15.83:0/3126567814 wait complete. 2024-04-17T10:43:01.232 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T10:42:34.328397+0000', 'modified': '2024-04-17T10:43:00.362453+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 62, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6843}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6843': {'gid': 6843, 'name': 'a', 'rank': 0, 'incarnation': 36, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.83:6837/3985646487', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6836', 'nonce': 3985646487}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 3985646487}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:43:01.232 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6843:up:replay 2024-04-17T10:43:02.232 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:43:02.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.369+0000 7f4277853640 1 Processor -- start 2024-04-17T10:43:02.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.369+0000 7f4277853640 1 -- start start 2024-04-17T10:43:02.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.369+0000 7f4277853640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f427015db50 0x7f4270070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:02.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.369+0000 7f4277853640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42700709b0 con 0x7f42700aedd0 2024-04-17T10:43:02.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.369+0000 7f4277853640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f427015e440 con 0x7f427006dcb0 2024-04-17T10:43:02.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.369+0000 7f4277853640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f427015e720 con 0x7f427015db50 2024-04-17T10:43:02.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.369+0000 7f42755ef640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f427015db50 0x7f4270070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:02.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.369+0000 7f42755ef640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f427015db50 0x7f4270070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58362/0 (socket says 172.21.15.83:58362) 2024-04-17T10:43:02.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.369+0000 7f42755ef640 1 -- 172.21.15.83:0/1730629028 learned_addr learned my addr 172.21.15.83:0/1730629028 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:02.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.369+0000 7f42755ef640 1 -- 172.21.15.83:0/1730629028 >> v1:172.21.15.167:6789/0 conn(0x7f427006dcb0 legacy=0x7f42700b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:02.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.369+0000 7f42755ef640 1 -- 172.21.15.83:0/1730629028 >> v1:172.21.15.83:6789/0 conn(0x7f42700aedd0 legacy=0x7f427006d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:02.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.369+0000 7f42755ef640 1 -- 172.21.15.83:0/1730629028 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f427015ea00 con 0x7f427015db50 2024-04-17T10:43:02.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.373+0000 7f42755ef640 1 --2- 172.21.15.83:0/1730629028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f427015db50 0x7f4270070470 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f4260009e80 tx=0x7f426001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbfede370365d30c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:02.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.373+0000 7f426ffff640 1 -- 172.21.15.83:0/1730629028 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42600a5860 con 0x7f427015db50 2024-04-17T10:43:02.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.373+0000 7f426ffff640 1 -- 172.21.15.83:0/1730629028 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42600a5a00 con 0x7f427015db50 2024-04-17T10:43:02.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.373+0000 7f426ffff640 1 -- 172.21.15.83:0/1730629028 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42600a5dd0 con 0x7f427015db50 2024-04-17T10:43:02.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.373+0000 7f4277853640 1 -- 172.21.15.83:0/1730629028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f427015db50 msgr2=0x7f4270070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:02.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.373+0000 7f4277853640 1 --2- 172.21.15.83:0/1730629028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f427015db50 0x7f4270070470 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f4260009e80 tx=0x7f426001c9a0 comp rx=0 tx=0).stop 2024-04-17T10:43:02.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.373+0000 7f4277853640 1 -- 172.21.15.83:0/1730629028 shutdown_connections 2024-04-17T10:43:02.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.373+0000 7f4277853640 1 --2- 172.21.15.83:0/1730629028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f427015db50 0x7f4270070470 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:02.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.373+0000 7f4277853640 1 -- 172.21.15.83:0/1730629028 >> 172.21.15.83:0/1730629028 conn(0x7f427006abe0 msgr2=0x7f427006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:02.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.373+0000 7f4277853640 1 -- 172.21.15.83:0/1730629028 shutdown_connections 2024-04-17T10:43:02.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.373+0000 7f4277853640 1 -- 172.21.15.83:0/1730629028 wait complete. 2024-04-17T10:43:02.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.373+0000 7f4277853640 1 Processor -- start 2024-04-17T10:43:02.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.373+0000 7f4277853640 1 -- start start 2024-04-17T10:43:02.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.377+0000 7f4277853640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f427006dcb0 0x7f42700b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:02.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.377+0000 7f4277853640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42700aedd0 0x7f4270159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:02.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.377+0000 7f4277853640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f427015db50 0x7f427014c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:02.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.377+0000 7f4277853640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42700b3eb0 con 0x7f427006dcb0 2024-04-17T10:43:02.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.377+0000 7f4277853640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42700b3d30 con 0x7f427015db50 2024-04-17T10:43:02.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.377+0000 7f4277853640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42700b4030 con 0x7f42700aedd0 2024-04-17T10:43:02.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.377+0000 7f42755ef640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f427006dcb0 0x7f42700b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:02.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.377+0000 7f42755ef640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f427006dcb0 0x7f42700b04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58366/0 (socket says 172.21.15.83:58366) 2024-04-17T10:43:02.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.377+0000 7f42755ef640 1 -- 172.21.15.83:0/3701279712 learned_addr learned my addr 172.21.15.83:0/3701279712 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:02.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.377+0000 7f4274dee640 1 --2- 172.21.15.83:0/3701279712 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42700aedd0 0x7f4270159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:02.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.377+0000 7f4275df0640 1 --2- 172.21.15.83:0/3701279712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f427015db50 0x7f427014c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:02.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.377+0000 7f42755ef640 1 -- 172.21.15.83:0/3701279712 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42700aedd0 msgr2=0x7f4270159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:02.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.377+0000 7f42755ef640 1 --2- 172.21.15.83:0/3701279712 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42700aedd0 0x7f4270159660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:02.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.377+0000 7f42755ef640 1 -- 172.21.15.83:0/3701279712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f427015db50 msgr2=0x7f427014c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:02.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.377+0000 7f42755ef640 1 --2- 172.21.15.83:0/3701279712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f427015db50 0x7f427014c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:02.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.377+0000 7f42755ef640 1 -- 172.21.15.83:0/3701279712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4270153390 con 0x7f427006dcb0 2024-04-17T10:43:02.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.377+0000 7f4275df0640 1 --2- 172.21.15.83:0/3701279712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f427015db50 0x7f427014c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:02.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.377+0000 7f42755ef640 1 --2- 172.21.15.83:0/3701279712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f427006dcb0 0x7f42700b04e0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f42600af040 tx=0x7f4260019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:02.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.377+0000 7f426dffb640 1 -- 172.21.15.83:0/3701279712 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42600b8880 con 0x7f427006dcb0 2024-04-17T10:43:02.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.377+0000 7f426dffb640 1 -- 172.21.15.83:0/3701279712 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42600be070 con 0x7f427006dcb0 2024-04-17T10:43:02.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.377+0000 7f4277853640 1 -- 172.21.15.83:0/3701279712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4270153630 con 0x7f427006dcb0 2024-04-17T10:43:02.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.377+0000 7f426dffb640 1 -- 172.21.15.83:0/3701279712 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42600b8c60 con 0x7f427006dcb0 2024-04-17T10:43:02.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.377+0000 7f4277853640 1 -- 172.21.15.83:0/3701279712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4270153bf0 con 0x7f427006dcb0 2024-04-17T10:43:02.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.377+0000 7f4277853640 1 -- 172.21.15.83:0/3701279712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4270074f90 con 0x7f427006dcb0 2024-04-17T10:43:02.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.381+0000 7f426dffb640 1 -- 172.21.15.83:0/3701279712 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f42600c3460 con 0x7f427006dcb0 2024-04-17T10:43:02.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.381+0000 7f426dffb640 1 --2- 172.21.15.83:0/3701279712 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f425c066f60 0x7f425c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:02.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.381+0000 7f426dffb640 1 -- 172.21.15.83:0/3701279712 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4260120470 con 0x7f427006dcb0 2024-04-17T10:43:02.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.381+0000 7f4274dee640 1 --2- 172.21.15.83:0/3701279712 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f425c066f60 0x7f425c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:02.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.381+0000 7f4274dee640 1 --2- 172.21.15.83:0/3701279712 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f425c066f60 0x7f425c069420 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4264007fd0 tx=0x7f4264009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:02.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.385+0000 7f426dffb640 1 -- 172.21.15.83:0/3701279712 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f42600165c0 con 0x7f427006dcb0 2024-04-17T10:43:02.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.569+0000 7f4277853640 1 -- 172.21.15.83:0/3701279712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42700b2af0 con 0x7f427006dcb0 2024-04-17T10:43:02.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.573+0000 7f426dffb640 1 -- 172.21.15.83:0/3701279712 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f42600f1420 con 0x7f427006dcb0 2024-04-17T10:43:02.568 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:43:02.568 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":38,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5165,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1924992370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1924992370},{"type":"v1","addr":"172.21.15.167:6833","nonce":1924992370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5174,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3876293847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3876293847},{"type":"v1","addr":"172.21.15.167:6835","nonce":3876293847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6831,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/338926944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":338926944},{"type":"v1","addr":"172.21.15.83:6835","nonce":338926944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:42:34.328397+0000","modified":"2024-04-17T10:43:01.380376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6843},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6843":{"gid":6843,"name":"a","rank":0,"incarnation":36,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.83:6837/3985646487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3985646487},{"type":"v1","addr":"172.21.15.83:6837","nonce":3985646487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-17T10:43:02.568 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 38 2024-04-17T10:43:02.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.573+0000 7f4277853640 1 -- 172.21.15.83:0/3701279712 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f425c066f60 msgr2=0x7f425c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:02.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.573+0000 7f4277853640 1 --2- 172.21.15.83:0/3701279712 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f425c066f60 0x7f425c069420 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4264007fd0 tx=0x7f4264009040 comp rx=0 tx=0).stop 2024-04-17T10:43:02.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.573+0000 7f4277853640 1 -- 172.21.15.83:0/3701279712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f427006dcb0 msgr2=0x7f42700b04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:02.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.573+0000 7f4277853640 1 --2- 172.21.15.83:0/3701279712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f427006dcb0 0x7f42700b04e0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f42600af040 tx=0x7f4260019740 comp rx=0 tx=0).stop 2024-04-17T10:43:02.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.573+0000 7f4277853640 1 -- 172.21.15.83:0/3701279712 shutdown_connections 2024-04-17T10:43:02.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.573+0000 7f4277853640 1 --2- 172.21.15.83:0/3701279712 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f425c066f60 0x7f425c069420 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:02.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.573+0000 7f4277853640 1 --2- 172.21.15.83:0/3701279712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f427015db50 0x7f427014c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:02.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.573+0000 7f4277853640 1 --2- 172.21.15.83:0/3701279712 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42700aedd0 0x7f4270159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:02.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.573+0000 7f4277853640 1 --2- 172.21.15.83:0/3701279712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f427006dcb0 0x7f42700b04e0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:02.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.573+0000 7f4277853640 1 -- 172.21.15.83:0/3701279712 >> 172.21.15.83:0/3701279712 conn(0x7f427006abe0 msgr2=0x7f42700cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:02.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.573+0000 7f4277853640 1 -- 172.21.15.83:0/3701279712 shutdown_connections 2024-04-17T10:43:02.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:02.573+0000 7f4277853640 1 -- 172.21.15.83:0/3701279712 wait complete. 2024-04-17T10:43:02.988 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 38, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T10:42:34.328397+0000', 'modified': '2024-04-17T10:43:01.380376+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 62, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6843}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6843': {'gid': 6843, 'name': 'a', 'rank': 0, 'incarnation': 36, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.83:6837/3985646487', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6836', 'nonce': 3985646487}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 3985646487}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:43:02.988 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6843:up:rejoin 2024-04-17T10:43:03.988 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:43:04.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f3cc36640 1 Processor -- start 2024-04-17T10:43:04.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f3cc36640 1 -- start start 2024-04-17T10:43:04.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f3cc36640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7f380b2de0 0x7f7f3815ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:04.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f3cc36640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f3815f5b0 con 0x7f7f380d22d0 2024-04-17T10:43:04.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f3cc36640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f380b2620 con 0x7f7f3815f1d0 2024-04-17T10:43:04.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f3cc36640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f380b2340 con 0x7f7f380b2de0 2024-04-17T10:43:04.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f36d9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7f380d22d0 0x7f7f380ae520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53716/0 (socket says 172.21.15.83:53716) 2024-04-17T10:43:04.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f36d9d640 1 -- 172.21.15.83:0/351983779 learned_addr learned my addr 172.21.15.83:0/351983779 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:43:04.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f3559a640 1 -- 172.21.15.83:0/351983779 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2845113683 0 0) 0x7f7f3815f5b0 con 0x7f7f380d22d0 2024-04-17T10:43:04.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f35d9b640 1 --2- 172.21.15.83:0/351983779 >> v2:172.21.15.167:3300/0 conn(0x7f7f380b2de0 0x7f7f3815ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:04.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f3559a640 1 -- 172.21.15.83:0/351983779 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f20003530 con 0x7f7f380d22d0 2024-04-17T10:43:04.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f3559a640 1 -- 172.21.15.83:0/351983779 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 848701571 0 0) 0x7f7f20003530 con 0x7f7f380d22d0 2024-04-17T10:43:04.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f3559a640 1 -- 172.21.15.83:0/351983779 >> v2:172.21.15.167:3300/0 conn(0x7f7f380b2de0 msgr2=0x7f7f3815ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:04.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f3559a640 1 --2- 172.21.15.83:0/351983779 >> v2:172.21.15.167:3300/0 conn(0x7f7f380b2de0 0x7f7f3815ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f3559a640 1 -- 172.21.15.83:0/351983779 >> v1:172.21.15.167:6789/0 conn(0x7f7f3815f1d0 legacy=0x7f7f3815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:04.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f3559a640 1 -- 172.21.15.83:0/351983779 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f380b1760 con 0x7f7f380d22d0 2024-04-17T10:43:04.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f3559a640 1 -- 172.21.15.83:0/351983779 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f7f30001a00 con 0x7f7f380d22d0 2024-04-17T10:43:04.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f3559a640 1 -- 172.21.15.83:0/351983779 >> v1:172.21.15.83:6789/0 conn(0x7f7f380d22d0 legacy=0x7f7f380ae520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:04.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f3559a640 1 --2- 172.21.15.83:0/351983779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f200039c0 0x7f7f20003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:04.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f3559a640 1 --2- 172.21.15.83:0/351983779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f20004810 0x7f7f20004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:04.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f3559a640 1 --2- 172.21.15.83:0/351983779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f200051d0 0x7f7f2000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:04.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f3559a640 1 -- 172.21.15.83:0/351983779 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f380b2340 con 0x7f7f200051d0 2024-04-17T10:43:04.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f3559a640 1 -- 172.21.15.83:0/351983779 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2000c120 con 0x7f7f200039c0 2024-04-17T10:43:04.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f3559a640 1 -- 172.21.15.83:0/351983779 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2000c320 con 0x7f7f20004810 2024-04-17T10:43:04.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f36d9d640 1 --2- 172.21.15.83:0/351983779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f200051d0 0x7f7f2000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:04.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f3659c640 1 --2- 172.21.15.83:0/351983779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f200039c0 0x7f7f20003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:04.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f35d9b640 1 --2- 172.21.15.83:0/351983779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f20004810 0x7f7f20004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:04.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f36d9d640 1 -- 172.21.15.83:0/351983779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f20004810 msgr2=0x7f7f20004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:04.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f36d9d640 1 --2- 172.21.15.83:0/351983779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f20004810 0x7f7f20004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f36d9d640 1 -- 172.21.15.83:0/351983779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f200039c0 msgr2=0x7f7f20003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:04.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f36d9d640 1 --2- 172.21.15.83:0/351983779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f200039c0 0x7f7f20003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f36d9d640 1 -- 172.21.15.83:0/351983779 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f20003530 con 0x7f7f200051d0 2024-04-17T10:43:04.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f35d9b640 1 --2- 172.21.15.83:0/351983779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f20004810 0x7f7f20004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:04.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.121+0000 7f7f3659c640 1 --2- 172.21.15.83:0/351983779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f200039c0 0x7f7f20003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:43:04.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.125+0000 7f7f36d9d640 1 -- 172.21.15.83:0/351983779 reap_dead start 2024-04-17T10:43:04.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.125+0000 7f7f36d9d640 1 --2- 172.21.15.83:0/351983779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f200051d0 0x7f7f2000b9d0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000a850 tx=0x7f7f3001d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7eeddace67f6b76 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:04.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.125+0000 7f7f3559a640 1 -- 172.21.15.83:0/351983779 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3000a4a0 con 0x7f7f200051d0 2024-04-17T10:43:04.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.125+0000 7f7f3559a640 1 -- 172.21.15.83:0/351983779 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f300ad650 con 0x7f7f200051d0 2024-04-17T10:43:04.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.125+0000 7f7f3559a640 1 -- 172.21.15.83:0/351983779 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f300ada40 con 0x7f7f200051d0 2024-04-17T10:43:04.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.125+0000 7f7f3cc36640 1 -- 172.21.15.83:0/351983779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f200051d0 msgr2=0x7f7f2000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:04.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.125+0000 7f7f3cc36640 1 --2- 172.21.15.83:0/351983779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f200051d0 0x7f7f2000b9d0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000a850 tx=0x7f7f3001d3a0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.125+0000 7f7f3cc36640 1 -- 172.21.15.83:0/351983779 shutdown_connections 2024-04-17T10:43:04.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.125+0000 7f7f3cc36640 1 --2- 172.21.15.83:0/351983779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f200051d0 0x7f7f2000b9d0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.125+0000 7f7f3cc36640 1 -- 172.21.15.83:0/351983779 >> 172.21.15.83:0/351983779 conn(0x7f7f380d17b0 msgr2=0x7f7f380d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:04.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.125+0000 7f7f3cc36640 1 -- 172.21.15.83:0/351983779 shutdown_connections 2024-04-17T10:43:04.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.125+0000 7f7f3cc36640 1 -- 172.21.15.83:0/351983779 wait complete. 2024-04-17T10:43:04.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.125+0000 7f7f3cc36640 1 Processor -- start 2024-04-17T10:43:04.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.125+0000 7f7f3cc36640 1 -- start start 2024-04-17T10:43:04.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.129+0000 7f7f3cc36640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f200051d0 0x7f7f3814fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:04.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.129+0000 7f7f3cc36640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f380c6330 0x7f7f380c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:04.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.129+0000 7f7f3cc36640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f3806d060 0x7f7f3806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:04.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.129+0000 7f7f3cc36640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3815f830 con 0x7f7f200051d0 2024-04-17T10:43:04.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.129+0000 7f7f3cc36640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f380b1d20 con 0x7f7f380c6330 2024-04-17T10:43:04.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.129+0000 7f7f3cc36640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3815f9b0 con 0x7f7f3806d060 2024-04-17T10:43:04.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.129+0000 7f7f3659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f200051d0 0x7f7f3814fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:04.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.129+0000 7f7f3659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f200051d0 0x7f7f3814fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58384/0 (socket says 172.21.15.83:58384) 2024-04-17T10:43:04.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.129+0000 7f7f3659c640 1 -- 172.21.15.83:0/2539758367 learned_addr learned my addr 172.21.15.83:0/2539758367 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:04.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.129+0000 7f7f35d9b640 1 --2- 172.21.15.83:0/2539758367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f380c6330 0x7f7f380c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:04.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.129+0000 7f7f3659c640 1 -- 172.21.15.83:0/2539758367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f3806d060 msgr2=0x7f7f3806d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:04.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.129+0000 7f7f3659c640 1 --2- 172.21.15.83:0/2539758367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f3806d060 0x7f7f3806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.129+0000 7f7f3659c640 1 -- 172.21.15.83:0/2539758367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f380c6330 msgr2=0x7f7f380c67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:04.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.129+0000 7f7f3659c640 1 --2- 172.21.15.83:0/2539758367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f380c6330 0x7f7f380c67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.129+0000 7f7f3659c640 1 -- 172.21.15.83:0/2539758367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f3806da20 con 0x7f7f200051d0 2024-04-17T10:43:04.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.129+0000 7f7f35d9b640 1 --2- 172.21.15.83:0/2539758367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f380c6330 0x7f7f380c67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:04.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.129+0000 7f7f3659c640 1 --2- 172.21.15.83:0/2539758367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f200051d0 0x7f7f3814fb90 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f7f2800e4d0 tx=0x7f7f2800e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:04.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.129+0000 7f7f26ffd640 1 -- 172.21.15.83:0/2539758367 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f280052e0 con 0x7f7f200051d0 2024-04-17T10:43:04.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.129+0000 7f7f26ffd640 1 -- 172.21.15.83:0/2539758367 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f28018070 con 0x7f7f200051d0 2024-04-17T10:43:04.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.129+0000 7f7f3cc36640 1 -- 172.21.15.83:0/2539758367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f380cd510 con 0x7f7f200051d0 2024-04-17T10:43:04.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.129+0000 7f7f26ffd640 1 -- 172.21.15.83:0/2539758367 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f28003820 con 0x7f7f200051d0 2024-04-17T10:43:04.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.129+0000 7f7f3cc36640 1 -- 172.21.15.83:0/2539758367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f380cdad0 con 0x7f7f200051d0 2024-04-17T10:43:04.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.129+0000 7f7f3cc36640 1 -- 172.21.15.83:0/2539758367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f38074f90 con 0x7f7f200051d0 2024-04-17T10:43:04.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.129+0000 7f7f26ffd640 1 -- 172.21.15.83:0/2539758367 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7f280039c0 con 0x7f7f200051d0 2024-04-17T10:43:04.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.129+0000 7f7f26ffd640 1 --2- 172.21.15.83:0/2539758367 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7f18066f60 0x7f7f18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:04.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.129+0000 7f7f26ffd640 1 -- 172.21.15.83:0/2539758367 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7f28084dd0 con 0x7f7f200051d0 2024-04-17T10:43:04.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.129+0000 7f7f35d9b640 1 --2- 172.21.15.83:0/2539758367 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7f18066f60 0x7f7f18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:04.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.129+0000 7f7f35d9b640 1 --2- 172.21.15.83:0/2539758367 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7f18066f60 0x7f7f18069420 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7f3806dc50 tx=0x7f7f2c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:04.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.133+0000 7f7f26ffd640 1 -- 172.21.15.83:0/2539758367 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7f28055cf0 con 0x7f7f200051d0 2024-04-17T10:43:04.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.313+0000 7f7f3cc36640 1 -- 172.21.15.83:0/2539758367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f38072690 con 0x7f7f200051d0 2024-04-17T10:43:04.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.313+0000 7f7f26ffd640 1 -- 172.21.15.83:0/2539758367 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f7f28086070 con 0x7f7f200051d0 2024-04-17T10:43:04.310 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:43:04.310 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":39,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5165,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1924992370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1924992370},{"type":"v1","addr":"172.21.15.167:6833","nonce":1924992370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5174,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3876293847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3876293847},{"type":"v1","addr":"172.21.15.167:6835","nonce":3876293847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6831,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/338926944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":338926944},{"type":"v1","addr":"172.21.15.83:6835","nonce":338926944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:42:34.328397+0000","modified":"2024-04-17T10:43:02.389233+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6843},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6843":{"gid":6843,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6837/3985646487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3985646487},{"type":"v1","addr":"172.21.15.83:6837","nonce":3985646487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-17T10:43:04.311 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 39 2024-04-17T10:43:04.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.313+0000 7f7f3cc36640 1 -- 172.21.15.83:0/2539758367 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7f18066f60 msgr2=0x7f7f18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:04.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.313+0000 7f7f3cc36640 1 --2- 172.21.15.83:0/2539758367 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7f18066f60 0x7f7f18069420 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7f3806dc50 tx=0x7f7f2c008040 comp rx=0 tx=0).stop 2024-04-17T10:43:04.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.313+0000 7f7f3cc36640 1 -- 172.21.15.83:0/2539758367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f200051d0 msgr2=0x7f7f3814fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:04.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.313+0000 7f7f3cc36640 1 --2- 172.21.15.83:0/2539758367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f200051d0 0x7f7f3814fb90 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f7f2800e4d0 tx=0x7f7f2800e990 comp rx=0 tx=0).stop 2024-04-17T10:43:04.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.313+0000 7f7f3cc36640 1 -- 172.21.15.83:0/2539758367 shutdown_connections 2024-04-17T10:43:04.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.313+0000 7f7f3cc36640 1 --2- 172.21.15.83:0/2539758367 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7f18066f60 0x7f7f18069420 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.313+0000 7f7f3cc36640 1 --2- 172.21.15.83:0/2539758367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f3806d060 0x7f7f3806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.313+0000 7f7f3cc36640 1 --2- 172.21.15.83:0/2539758367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f380c6330 0x7f7f380c67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.313+0000 7f7f3cc36640 1 --2- 172.21.15.83:0/2539758367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f200051d0 0x7f7f3814fb90 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.313+0000 7f7f3cc36640 1 -- 172.21.15.83:0/2539758367 >> 172.21.15.83:0/2539758367 conn(0x7f7f380d17b0 msgr2=0x7f7f380cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:04.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.313+0000 7f7f3cc36640 1 -- 172.21.15.83:0/2539758367 shutdown_connections 2024-04-17T10:43:04.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.313+0000 7f7f3cc36640 1 -- 172.21.15.83:0/2539758367 wait complete. 2024-04-17T10:43:04.729 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 39, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T10:42:34.328397+0000', 'modified': '2024-04-17T10:43:02.389233+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 62, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6843}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6843': {'gid': 6843, 'name': 'a', 'rank': 0, 'incarnation': 36, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.83:6837/3985646487', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6836', 'nonce': 3985646487}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 3985646487}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:43:04.730 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T10:43:04.730 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-17T10:43:04.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.869+0000 7fe6d1c85640 1 Processor -- start 2024-04-17T10:43:04.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.869+0000 7fe6d1c85640 1 -- start start 2024-04-17T10:43:04.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.869+0000 7fe6d1c85640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6cc15e0f0 0x7fe6cc0b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:04.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.869+0000 7fe6d1c85640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6cc15e4d0 con 0x7fe6cc0b5450 2024-04-17T10:43:04.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.869+0000 7fe6d1c85640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6cc162580 con 0x7fe6cc0ce400 2024-04-17T10:43:04.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.869+0000 7fe6d1c85640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc162b40 con 0x7fe6cc15e0f0 2024-04-17T10:43:04.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.869+0000 7fe6cbfff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe6cc0b5450 0x7fe6cc0d2540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53726/0 (socket says 172.21.15.83:53726) 2024-04-17T10:43:04.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.869+0000 7fe6cbfff640 1 -- 172.21.15.83:0/598340312 learned_addr learned my addr 172.21.15.83:0/598340312 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:43:04.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.869+0000 7fe6cb7fe640 1 --2- 172.21.15.83:0/598340312 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6cc15e0f0 0x7fe6cc0b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:04.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.869+0000 7fe6ca7fc640 1 -- 172.21.15.83:0/598340312 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4005516107 0 0) 0x7fe6cc15e4d0 con 0x7fe6cc0b5450 2024-04-17T10:43:04.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.869+0000 7fe6ca7fc640 1 -- 172.21.15.83:0/598340312 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6b8003530 con 0x7fe6cc0b5450 2024-04-17T10:43:04.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.869+0000 7fe6ca7fc640 1 -- 172.21.15.83:0/598340312 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2389711104 0 0) 0x7fe6b8003530 con 0x7fe6cc0b5450 2024-04-17T10:43:04.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.869+0000 7fe6ca7fc640 1 -- 172.21.15.83:0/598340312 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6cc15e0f0 msgr2=0x7fe6cc0b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:04.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.869+0000 7fe6ca7fc640 1 --2- 172.21.15.83:0/598340312 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6cc15e0f0 0x7fe6cc0b6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.869+0000 7fe6ca7fc640 1 -- 172.21.15.83:0/598340312 >> v1:172.21.15.167:6789/0 conn(0x7fe6cc0ce400 legacy=0x7fe6cc161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:04.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.869+0000 7fe6ca7fc640 1 -- 172.21.15.83:0/598340312 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6cc162860 con 0x7fe6cc0b5450 2024-04-17T10:43:04.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.869+0000 7fe6ca7fc640 1 -- 172.21.15.83:0/598340312 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fe6c4001a00 con 0x7fe6cc0b5450 2024-04-17T10:43:04.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.869+0000 7fe6ca7fc640 1 -- 172.21.15.83:0/598340312 >> v1:172.21.15.83:6789/0 conn(0x7fe6cc0b5450 legacy=0x7fe6cc0d2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:04.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.869+0000 7fe6ca7fc640 1 --2- 172.21.15.83:0/598340312 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6cc15e0f0 0x7fe6b8003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:04.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.869+0000 7fe6ca7fc640 1 --2- 172.21.15.83:0/598340312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6b8003fe0 0x7fe6b8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:04.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.869+0000 7fe6ca7fc640 1 --2- 172.21.15.83:0/598340312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6b8004d50 0x7fe6b80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:04.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.869+0000 7fe6ca7fc640 1 -- 172.21.15.83:0/598340312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc162b40 con 0x7fe6b8003fe0 2024-04-17T10:43:04.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.869+0000 7fe6ca7fc640 1 -- 172.21.15.83:0/598340312 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b800bbc0 con 0x7fe6b8004d50 2024-04-17T10:43:04.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.869+0000 7fe6ca7fc640 1 -- 172.21.15.83:0/598340312 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b800bdc0 con 0x7fe6cc15e0f0 2024-04-17T10:43:04.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.869+0000 7fe6caffd640 1 --2- 172.21.15.83:0/598340312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6b8003fe0 0x7fe6b8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:04.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.869+0000 7fe6cb7fe640 1 --2- 172.21.15.83:0/598340312 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6cc15e0f0 0x7fe6b8003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:04.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.869+0000 7fe6cbfff640 1 --2- 172.21.15.83:0/598340312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6b8004d50 0x7fe6b80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:04.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.869+0000 7fe6caffd640 1 -- 172.21.15.83:0/598340312 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6cc15e0f0 msgr2=0x7fe6b8003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:04.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.869+0000 7fe6caffd640 1 --2- 172.21.15.83:0/598340312 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6cc15e0f0 0x7fe6b8003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.869+0000 7fe6caffd640 1 -- 172.21.15.83:0/598340312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6b8004d50 msgr2=0x7fe6b80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:04.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.869+0000 7fe6caffd640 1 --2- 172.21.15.83:0/598340312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6b8004d50 0x7fe6b80051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.869+0000 7fe6caffd640 1 -- 172.21.15.83:0/598340312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b8003530 con 0x7fe6b8003fe0 2024-04-17T10:43:04.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.873+0000 7fe6caffd640 1 --2- 172.21.15.83:0/598340312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6b8003fe0 0x7fe6b8004810 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc001750 tx=0x7fe6bc01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b4c54b1b3a38e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:04.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.873+0000 7fe6ca7fc640 1 -- 172.21.15.83:0/598340312 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc0013a0 con 0x7fe6b8003fe0 2024-04-17T10:43:04.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.873+0000 7fe6ca7fc640 1 -- 172.21.15.83:0/598340312 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6bc0052f0 con 0x7fe6b8003fe0 2024-04-17T10:43:04.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.873+0000 7fe6ca7fc640 1 -- 172.21.15.83:0/598340312 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc003840 con 0x7fe6b8003fe0 2024-04-17T10:43:04.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.873+0000 7fe6d1c85640 1 -- 172.21.15.83:0/598340312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6b8003fe0 msgr2=0x7fe6b8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:04.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.873+0000 7fe6d1c85640 1 --2- 172.21.15.83:0/598340312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6b8003fe0 0x7fe6b8004810 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc001750 tx=0x7fe6bc01c3e0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.873+0000 7fe6cbfff640 1 -- 172.21.15.83:0/598340312 reap_dead start 2024-04-17T10:43:04.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.873+0000 7fe6d1c85640 1 -- 172.21.15.83:0/598340312 shutdown_connections 2024-04-17T10:43:04.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.873+0000 7fe6d1c85640 1 -- 172.21.15.83:0/598340312 >> 172.21.15.83:0/598340312 conn(0x7fe6cc0d0450 msgr2=0x7fe6cc0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:04.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.873+0000 7fe6d1c85640 1 -- 172.21.15.83:0/598340312 shutdown_connections 2024-04-17T10:43:04.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.873+0000 7fe6d1c85640 1 -- 172.21.15.83:0/598340312 wait complete. 2024-04-17T10:43:04.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.877+0000 7fe6d1c85640 1 Processor -- start 2024-04-17T10:43:04.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.877+0000 7fe6d1c85640 1 -- start start 2024-04-17T10:43:04.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.877+0000 7fe6d1c85640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6b8003fe0 0x7fe6cc14d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:04.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.877+0000 7fe6d1c85640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6cc152590 0x7fe6cc158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:04.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.877+0000 7fe6d1c85640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6cc1591c0 0x7fe6cc1556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:04.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.877+0000 7fe6d1c85640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc0b5de0 con 0x7fe6cc1591c0 2024-04-17T10:43:04.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.877+0000 7fe6d1c85640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc0b5c60 con 0x7fe6b8003fe0 2024-04-17T10:43:04.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.877+0000 7fe6d1c85640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc169f50 con 0x7fe6cc152590 2024-04-17T10:43:04.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.877+0000 7fe6cbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6cc1591c0 0x7fe6cc1556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:04.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.877+0000 7fe6cbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6cc1591c0 0x7fe6cc1556d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58412/0 (socket says 172.21.15.83:58412) 2024-04-17T10:43:04.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.877+0000 7fe6cbfff640 1 -- 172.21.15.83:0/924682431 learned_addr learned my addr 172.21.15.83:0/924682431 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:04.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.877+0000 7fe6cb7fe640 1 --2- 172.21.15.83:0/924682431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6b8003fe0 0x7fe6cc14d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:04.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.877+0000 7fe6caffd640 1 --2- 172.21.15.83:0/924682431 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6cc152590 0x7fe6cc158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:04.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.877+0000 7fe6cbfff640 1 -- 172.21.15.83:0/924682431 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6cc152590 msgr2=0x7fe6cc158c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:04.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.877+0000 7fe6cbfff640 1 --2- 172.21.15.83:0/924682431 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6cc152590 0x7fe6cc158c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.877+0000 7fe6cbfff640 1 -- 172.21.15.83:0/924682431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6b8003fe0 msgr2=0x7fe6cc14d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:04.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.877+0000 7fe6cbfff640 1 --2- 172.21.15.83:0/924682431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6b8003fe0 0x7fe6cc14d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.877+0000 7fe6cbfff640 1 -- 172.21.15.83:0/924682431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6cc158210 con 0x7fe6cc1591c0 2024-04-17T10:43:04.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.877+0000 7fe6caffd640 1 --2- 172.21.15.83:0/924682431 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6cc152590 0x7fe6cc158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:43:04.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.877+0000 7fe6cbfff640 1 --2- 172.21.15.83:0/924682431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6cc1591c0 0x7fe6cc1556d0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc0d1480 tx=0x7fe6c400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:04.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.877+0000 7fe6c3fff640 1 -- 172.21.15.83:0/924682431 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4004d70 con 0x7fe6cc1591c0 2024-04-17T10:43:04.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.877+0000 7fe6c3fff640 1 -- 172.21.15.83:0/924682431 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c400bb80 con 0x7fe6cc1591c0 2024-04-17T10:43:04.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.877+0000 7fe6d1c85640 1 -- 172.21.15.83:0/924682431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6cc152fc0 con 0x7fe6cc1591c0 2024-04-17T10:43:04.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.877+0000 7fe6c3fff640 1 -- 172.21.15.83:0/924682431 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4007110 con 0x7fe6cc1591c0 2024-04-17T10:43:04.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.877+0000 7fe6d1c85640 1 -- 172.21.15.83:0/924682431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6cc1532a0 con 0x7fe6cc1591c0 2024-04-17T10:43:04.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.877+0000 7fe6c3fff640 1 -- 172.21.15.83:0/924682431 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe6c401b020 con 0x7fe6cc1591c0 2024-04-17T10:43:04.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6c3fff640 1 --2- 172.21.15.83:0/924682431 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe6ac066f60 0x7fe6ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:04.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6c3fff640 1 -- 172.21.15.83:0/924682431 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe6c40851e0 con 0x7fe6cc1591c0 2024-04-17T10:43:04.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6cb7fe640 1 --2- 172.21.15.83:0/924682431 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe6ac066f60 0x7fe6ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:04.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6cb7fe640 1 --2- 172.21.15.83:0/924682431 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe6ac066f60 0x7fe6ac069420 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009ae0 tx=0x7fe6b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:04.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6d4df2000 1 Processor -- start 2024-04-17T10:43:04.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6d4df2000 1 -- start start 2024-04-17T10:43:04.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6d4df2000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647edec2920 0x5647edec2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:04.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6d4df2000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647edec3240 0x5647ede789f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:04.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6d4df2000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5647ede78f30 0x5647ede79310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:04.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6d4df2000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647edd605f0 con 0x5647edec2920 2024-04-17T10:43:04.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6d4df2000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647edcc7da0 con 0x5647edec3240 2024-04-17T10:43:04.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6d4df2000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5647edc50150 con 0x5647ede78f30 2024-04-17T10:43:04.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6cb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647edec2920 0x5647edec2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:04.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6cb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647edec2920 0x5647edec2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58418/0 (socket says 172.21.15.83:58418) 2024-04-17T10:43:04.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6cb7fe640 1 -- 172.21.15.83:0/3947660154 learned_addr learned my addr 172.21.15.83:0/3947660154 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:04.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6caffd640 1 --2- 172.21.15.83:0/3947660154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647edec3240 0x5647ede789f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:04.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6cbfff640 1 --2- 172.21.15.83:0/3947660154 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5647ede78f30 0x5647ede79310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:04.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6cb7fe640 1 -- 172.21.15.83:0/3947660154 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5647ede78f30 msgr2=0x5647ede79310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:04.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6cb7fe640 1 --2- 172.21.15.83:0/3947660154 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5647ede78f30 0x5647ede79310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6cb7fe640 1 -- 172.21.15.83:0/3947660154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647edec3240 msgr2=0x5647ede789f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:04.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6cb7fe640 1 --2- 172.21.15.83:0/3947660154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647edec3240 0x5647ede789f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6cb7fe640 1 -- 172.21.15.83:0/3947660154 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647edc69ca0 con 0x5647edec2920 2024-04-17T10:43:04.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6cb7fe640 1 --2- 172.21.15.83:0/3947660154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647edec2920 0x5647edec2d00 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4014dd0 tx=0x7fe6b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b671ceb3f9aea45d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:04.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6c17fa640 1 -- 172.21.15.83:0/3947660154 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4027070 con 0x5647edec2920 2024-04-17T10:43:04.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6c17fa640 1 -- 172.21.15.83:0/3947660154 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b4014e00 con 0x5647edec2920 2024-04-17T10:43:04.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6c17fa640 1 -- 172.21.15.83:0/3947660154 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4023690 con 0x5647edec2920 2024-04-17T10:43:04.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6d4df2000 1 -- 172.21.15.83:0/3947660154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647edec2920 msgr2=0x5647edec2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:04.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6d4df2000 1 --2- 172.21.15.83:0/3947660154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647edec2920 0x5647edec2d00 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4014dd0 tx=0x7fe6b4017000 comp rx=0 tx=0).stop 2024-04-17T10:43:04.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6d4df2000 1 -- 172.21.15.83:0/3947660154 shutdown_connections 2024-04-17T10:43:04.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6d4df2000 1 --2- 172.21.15.83:0/3947660154 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5647ede78f30 0x5647ede79310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6d4df2000 1 --2- 172.21.15.83:0/3947660154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647edec3240 0x5647ede789f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6d4df2000 1 --2- 172.21.15.83:0/3947660154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647edec2920 0x5647edec2d00 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6d4df2000 1 -- 172.21.15.83:0/3947660154 >> 172.21.15.83:0/3947660154 conn(0x5647ede7f2a0 msgr2=0x5647ede772d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:04.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6d4df2000 1 -- 172.21.15.83:0/3947660154 shutdown_connections 2024-04-17T10:43:04.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.881+0000 7fe6d4df2000 1 -- 172.21.15.83:0/3947660154 wait complete. 2024-04-17T10:43:04.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6d4df2000 1 -- start start 2024-04-17T10:43:04.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6d4df2000 1 Processor -- start 2024-04-17T10:43:04.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6d4df2000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647ede78f30 0x5647edec9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:04.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6d4df2000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5647edec2920 0x5647edeca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:04.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6d4df2000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647edec3240 0x5647ede7caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:04.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6d4df2000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647edc82990 con 0x5647ede78f30 2024-04-17T10:43:04.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6d4df2000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647edcc3e40 con 0x5647edec3240 2024-04-17T10:43:04.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6d4df2000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5647edc53900 con 0x5647edec2920 2024-04-17T10:43:04.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6d4df2000 10 client.0 _ll_register_callbacks cb 0x5647edd7b1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:43:04.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6cb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647ede78f30 0x5647edec9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:04.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6cb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647ede78f30 0x5647edec9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58426/0 (socket says 172.21.15.83:58426) 2024-04-17T10:43:04.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6cb7fe640 1 -- 172.21.15.83:0/3199706258 learned_addr learned my addr 172.21.15.83:0/3199706258 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:04.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6cb7fe640 1 -- 172.21.15.83:0/3199706258 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5647edec2920 msgr2=0x5647edeca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:04.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6cb7fe640 1 --2- 172.21.15.83:0/3199706258 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5647edec2920 0x5647edeca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6cb7fe640 1 -- 172.21.15.83:0/3199706258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647edec3240 msgr2=0x5647ede7caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:04.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6cb7fe640 1 --2- 172.21.15.83:0/3199706258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647edec3240 0x5647ede7caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6cb7fe640 1 -- 172.21.15.83:0/3199706258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5647edc80630 con 0x5647ede78f30 2024-04-17T10:43:04.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6cb7fe640 1 --2- 172.21.15.83:0/3199706258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647ede78f30 0x5647edec9c80 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4017fd0 tx=0x7fe6b4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5334da2298e8116b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:04.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6aa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:43:04.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6aa7fc640 1 -- 172.21.15.83:0/3199706258 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4027040 con 0x5647ede78f30 2024-04-17T10:43:04.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6aa7fc640 1 -- 172.21.15.83:0/3199706258 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe6b402c660 con 0x5647ede78f30 2024-04-17T10:43:04.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6d4df2000 1 -- 172.21.15.83:0/3199706258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6b4014e00 con 0x5647ede78f30 2024-04-17T10:43:04.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6aa7fc640 1 -- 172.21.15.83:0/3199706258 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6b402c860 con 0x5647ede78f30 2024-04-17T10:43:04.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6d4df2000 10 client.6876 fetch_fsmap learned FSMap version 39 2024-04-17T10:43:04.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6d4df2000 1 -- 172.21.15.83:0/3199706258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x5647edc503a0 con 0x5647ede78f30 2024-04-17T10:43:04.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6aa7fc640 1 -- 172.21.15.83:0/3199706258 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b4014e00 con 0x5647ede78f30 2024-04-17T10:43:04.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6aa7fc640 1 -- 172.21.15.83:0/3199706258 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 39) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe6b402ca00 con 0x5647ede78f30 2024-04-17T10:43:04.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6d4df2000 10 client.6876 fetch_fsmap finished waiting for FSMap version 39 2024-04-17T10:43:04.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6d4df2000 10 client.6876 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T10:43:04.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6d4df2000 20 client.6876 populate_metadata read hostname 'smithi083' 2024-04-17T10:43:04.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6d4df2000 1 --2- 172.21.15.83:0/3199706258 >> [v2:172.21.15.83:6836/3985646487,v1:172.21.15.83:6837/3985646487] conn(0x5647ede7e160 0x5647eded3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:04.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6d4df2000 4 client.6876 mds_command: new command op to 6843 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:43:04.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6d4df2000 1 -- 172.21.15.83:0/3199706258 --> [v2:172.21.15.83:6836/3985646487,v1:172.21.15.83:6837/3985646487] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647edc63a50 con 0x5647ede7e160 2024-04-17T10:43:04.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6caffd640 1 --2- 172.21.15.83:0/3199706258 >> [v2:172.21.15.83:6836/3985646487,v1:172.21.15.83:6837/3985646487] conn(0x5647ede7e160 0x5647eded3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:04.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6caffd640 1 --2- 172.21.15.83:0/3199706258 >> [v2:172.21.15.83:6836/3985646487,v1:172.21.15.83:6837/3985646487] conn(0x5647ede7e160 0x5647eded3ce0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d397d84fddb0fba server_cookie=8a7ef67d80d7d262 in_seq=0 out_seq=0 2024-04-17T10:43:04.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6aa7fc640 10 client.6876 ms_handle_connect on v2:172.21.15.83:6836/3985646487 2024-04-17T10:43:04.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6aa7fc640 1 -- 172.21.15.83:0/3199706258 <== mds.0 v2:172.21.15.83:6836/3985646487 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5647edc63a50 con 0x5647ede7e160 2024-04-17T10:43:04.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6aa7fc640 10 client.6876 handle_command_reply: tid=0 2024-04-17T10:43:04.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6d4df2000 1 client.6876 shutdown 2024-04-17T10:43:04.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6d4df2000 1 -- 172.21.15.83:0/3199706258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647ede78f30 msgr2=0x5647edec9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:04.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6d4df2000 1 --2- 172.21.15.83:0/3199706258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647ede78f30 0x5647edec9c80 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4017fd0 tx=0x7fe6b4004740 comp rx=0 tx=0).stop 2024-04-17T10:43:04.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6d4df2000 1 -- 172.21.15.83:0/3199706258 shutdown_connections 2024-04-17T10:43:04.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6d4df2000 1 --2- 172.21.15.83:0/3199706258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647edec3240 0x5647ede7caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6d4df2000 1 --2- 172.21.15.83:0/3199706258 >> [v2:172.21.15.83:6836/3985646487,v1:172.21.15.83:6837/3985646487] conn(0x5647ede7e160 0x5647eded3ce0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6d4df2000 1 --2- 172.21.15.83:0/3199706258 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5647edec2920 0x5647edeca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6d4df2000 1 --2- 172.21.15.83:0/3199706258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647ede78f30 0x5647edec9c80 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6aa7fc640 0 client.6876 ms_handle_reset on v2:172.21.15.83:6836/3985646487 2024-04-17T10:43:04.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6d4df2000 1 -- 172.21.15.83:0/3199706258 >> 172.21.15.83:0/3199706258 conn(0x5647ede7f2a0 msgr2=0x5647ede779e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:04.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6d4df2000 1 -- 172.21.15.83:0/3199706258 shutdown_connections 2024-04-17T10:43:04.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6d4df2000 1 -- 172.21.15.83:0/3199706258 wait complete. 2024-04-17T10:43:04.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.885+0000 7fe6d4df2000 20 client.6876 trim_cache size 0 max 16384 2024-04-17T10:43:04.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.901+0000 7fe6d4df2000 1 Processor -- start 2024-04-17T10:43:04.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.901+0000 7fe6d4df2000 1 -- start start 2024-04-17T10:43:04.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.901+0000 7fe6d4df2000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5647ede78f30 0x5647edf1dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:04.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.901+0000 7fe6d4df2000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647edec2920 0x5647edf1e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:04.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.901+0000 7fe6d4df2000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647edec3240 0x5647edf1e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:04.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.901+0000 7fe6d4df2000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc202df0 con 0x5647edec3240 2024-04-17T10:43:04.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.901+0000 7fe6d4df2000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc0cbed0 con 0x5647edec2920 2024-04-17T10:43:04.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.901+0000 7fe6d4df2000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc151e40 con 0x5647ede78f30 2024-04-17T10:43:04.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.901+0000 7fe6cbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647edec3240 0x5647edf1e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:04.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.901+0000 7fe6cbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647edec3240 0x5647edf1e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58430/0 (socket says 172.21.15.83:58430) 2024-04-17T10:43:04.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.901+0000 7fe6cbfff640 1 -- 172.21.15.83:0/3685762394 learned_addr learned my addr 172.21.15.83:0/3685762394 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:04.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.901+0000 7fe6cbfff640 1 -- 172.21.15.83:0/3685762394 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5647ede78f30 msgr2=0x5647edf1dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:04.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.901+0000 7fe6cb7fe640 1 --2- 172.21.15.83:0/3685762394 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5647ede78f30 0x5647edf1dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:04.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.901+0000 7fe6caffd640 1 --2- 172.21.15.83:0/3685762394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647edec2920 0x5647edf1e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:04.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.901+0000 7fe6cbfff640 1 --2- 172.21.15.83:0/3685762394 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5647ede78f30 0x5647edf1dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.901+0000 7fe6cbfff640 1 -- 172.21.15.83:0/3685762394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647edec2920 msgr2=0x5647edf1e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:04.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.901+0000 7fe6cbfff640 1 --2- 172.21.15.83:0/3685762394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647edec2920 0x5647edf1e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.901+0000 7fe6cbfff640 1 -- 172.21.15.83:0/3685762394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647edec3070 con 0x5647edec3240 2024-04-17T10:43:04.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.901+0000 7fe6cbfff640 1 --2- 172.21.15.83:0/3685762394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647edec3240 0x5647edf1e650 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4083760 tx=0x7fe6c40559b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd48be6f70b1bdb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:04.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.901+0000 7fe6c0ff9640 1 -- 172.21.15.83:0/3685762394 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4055b20 con 0x5647edec3240 2024-04-17T10:43:04.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.901+0000 7fe6c0ff9640 1 -- 172.21.15.83:0/3685762394 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c4056e50 con 0x5647edec3240 2024-04-17T10:43:04.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.901+0000 7fe6c0ff9640 1 -- 172.21.15.83:0/3685762394 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4059c70 con 0x5647edec3240 2024-04-17T10:43:04.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6d4df2000 1 -- 172.21.15.83:0/3685762394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647edec3240 msgr2=0x5647edf1e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:04.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6d4df2000 1 --2- 172.21.15.83:0/3685762394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647edec3240 0x5647edf1e650 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4083760 tx=0x7fe6c40559b0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6d4df2000 1 -- 172.21.15.83:0/3685762394 shutdown_connections 2024-04-17T10:43:04.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6d4df2000 1 --2- 172.21.15.83:0/3685762394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647edec3240 0x5647edf1e650 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6d4df2000 1 --2- 172.21.15.83:0/3685762394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647edec2920 0x5647edf1e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6d4df2000 1 --2- 172.21.15.83:0/3685762394 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5647ede78f30 0x5647edf1dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6d4df2000 1 -- 172.21.15.83:0/3685762394 >> 172.21.15.83:0/3685762394 conn(0x5647ede7f2a0 msgr2=0x5647edec0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:04.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6d4df2000 1 -- 172.21.15.83:0/3685762394 shutdown_connections 2024-04-17T10:43:04.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6d4df2000 1 -- 172.21.15.83:0/3685762394 wait complete. 2024-04-17T10:43:04.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6d4df2000 1 -- start start 2024-04-17T10:43:04.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6d4df2000 1 Processor -- start 2024-04-17T10:43:04.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6d4df2000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647ede78f30 0x5647edf06940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:04.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6d4df2000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5647edec2920 0x5647eded7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:04.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6d4df2000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647edec3240 0x5647eded8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:04.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6d4df2000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647edccc3d0 con 0x5647ede78f30 2024-04-17T10:43:04.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6d4df2000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647edd83d50 con 0x5647edec3240 2024-04-17T10:43:04.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6d4df2000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5647edc62980 con 0x5647edec2920 2024-04-17T10:43:04.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6d4df2000 10 client.0 _ll_register_callbacks cb 0x7fe6b401a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:43:04.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6cb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647ede78f30 0x5647edf06940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:04.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6cb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647ede78f30 0x5647edf06940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58438/0 (socket says 172.21.15.83:58438) 2024-04-17T10:43:04.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6cb7fe640 1 -- 172.21.15.83:0/446385463 learned_addr learned my addr 172.21.15.83:0/446385463 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:04.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6cbfff640 1 --2- 172.21.15.83:0/446385463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647edec3240 0x5647eded8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:04.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6caffd640 1 --2- 172.21.15.83:0/446385463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5647edec2920 0x5647eded7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:04.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6cb7fe640 1 -- 172.21.15.83:0/446385463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5647edec2920 msgr2=0x5647eded7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:04.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6cb7fe640 1 --2- 172.21.15.83:0/446385463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5647edec2920 0x5647eded7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6cb7fe640 1 -- 172.21.15.83:0/446385463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647edec3240 msgr2=0x5647eded8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:04.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6cb7fe640 1 --2- 172.21.15.83:0/446385463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647edec3240 0x5647eded8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6cb7fe640 1 -- 172.21.15.83:0/446385463 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5647edd49600 con 0x5647ede78f30 2024-04-17T10:43:04.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6cb7fe640 1 --2- 172.21.15.83:0/446385463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647ede78f30 0x5647edf06940 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4014fc0 tx=0x7fe6b401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83e9e4e6d1f4c15 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:04.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6aa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:43:04.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6aa7fc640 1 -- 172.21.15.83:0/446385463 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4027040 con 0x5647ede78f30 2024-04-17T10:43:04.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6aa7fc640 1 -- 172.21.15.83:0/446385463 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe6b402c660 con 0x5647ede78f30 2024-04-17T10:43:04.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6d4df2000 1 -- 172.21.15.83:0/446385463 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6c4056e50 con 0x5647ede78f30 2024-04-17T10:43:04.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6aa7fc640 1 -- 172.21.15.83:0/446385463 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6b4004250 con 0x5647ede78f30 2024-04-17T10:43:04.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6d4df2000 10 client.6882 fetch_fsmap learned FSMap version 39 2024-04-17T10:43:04.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6d4df2000 1 -- 172.21.15.83:0/446385463 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x5647edc5c9b0 con 0x5647ede78f30 2024-04-17T10:43:04.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6aa7fc640 1 -- 172.21.15.83:0/446385463 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c4056e50 con 0x5647ede78f30 2024-04-17T10:43:04.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6aa7fc640 1 -- 172.21.15.83:0/446385463 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 39) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe6b40043f0 con 0x5647ede78f30 2024-04-17T10:43:04.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6d4df2000 10 client.6882 fetch_fsmap finished waiting for FSMap version 39 2024-04-17T10:43:04.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6d4df2000 10 client.6882 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T10:43:04.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6d4df2000 20 client.6882 populate_metadata read hostname 'smithi083' 2024-04-17T10:43:04.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6d4df2000 1 --2- 172.21.15.83:0/446385463 >> [v2:172.21.15.83:6836/3985646487,v1:172.21.15.83:6837/3985646487] conn(0x5647edf07230 0x5647edf07610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:04.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6d4df2000 4 client.6882 mds_command: new command op to 6843 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T10:43:04.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6d4df2000 1 -- 172.21.15.83:0/446385463 --> [v2:172.21.15.83:6836/3985646487,v1:172.21.15.83:6837/3985646487] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5647edcd52e0 con 0x5647edf07230 2024-04-17T10:43:04.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6caffd640 1 --2- 172.21.15.83:0/446385463 >> [v2:172.21.15.83:6836/3985646487,v1:172.21.15.83:6837/3985646487] conn(0x5647edf07230 0x5647edf07610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:04.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6aa7fc640 10 client.6882 ms_handle_connect on v2:172.21.15.83:6836/3985646487 2024-04-17T10:43:04.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6caffd640 1 --2- 172.21.15.83:0/446385463 >> [v2:172.21.15.83:6836/3985646487,v1:172.21.15.83:6837/3985646487] conn(0x5647edf07230 0x5647edf07610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5476cbaeb492cad3 server_cookie=721a94c115cf8013 in_seq=0 out_seq=0 2024-04-17T10:43:04.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6aa7fc640 1 -- 172.21.15.83:0/446385463 <== mds.0 v2:172.21.15.83:6836/3985646487 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5647edcd52e0 con 0x5647edf07230 2024-04-17T10:43:04.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6aa7fc640 10 client.6882 handle_command_reply: tid=0 2024-04-17T10:43:04.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.905+0000 7fe6d4df2000 1 client.6882 shutdown 2024-04-17T10:43:04.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.909+0000 7fe6d4df2000 1 -- 172.21.15.83:0/446385463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647ede78f30 msgr2=0x5647edf06940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:04.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.909+0000 7fe6d4df2000 1 --2- 172.21.15.83:0/446385463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647ede78f30 0x5647edf06940 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4014fc0 tx=0x7fe6b401a000 comp rx=0 tx=0).stop 2024-04-17T10:43:04.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.909+0000 7fe6d4df2000 1 -- 172.21.15.83:0/446385463 shutdown_connections 2024-04-17T10:43:04.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.909+0000 7fe6d4df2000 1 --2- 172.21.15.83:0/446385463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647edec3240 0x5647eded8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.909+0000 7fe6d4df2000 1 --2- 172.21.15.83:0/446385463 >> [v2:172.21.15.83:6836/3985646487,v1:172.21.15.83:6837/3985646487] conn(0x5647edf07230 0x5647edf07610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.909+0000 7fe6d4df2000 1 --2- 172.21.15.83:0/446385463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5647edec2920 0x5647eded7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.909+0000 7fe6d4df2000 1 --2- 172.21.15.83:0/446385463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647ede78f30 0x5647edf06940 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.909+0000 7fe6aa7fc640 0 client.6882 ms_handle_reset on v2:172.21.15.83:6836/3985646487 2024-04-17T10:43:04.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.909+0000 7fe6d4df2000 1 -- 172.21.15.83:0/446385463 >> 172.21.15.83:0/446385463 conn(0x5647ede7f2a0 msgr2=0x5647edec1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:04.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.909+0000 7fe6d4df2000 1 -- 172.21.15.83:0/446385463 shutdown_connections 2024-04-17T10:43:04.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.909+0000 7fe6d4df2000 1 -- 172.21.15.83:0/446385463 wait complete. 2024-04-17T10:43:04.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.909+0000 7fe6d4df2000 20 client.6882 trim_cache size 0 max 16384 2024-04-17T10:43:04.911 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:43:04.911 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "3de9b7be-af7f-488a-969e-2b6b8db63bcd", 2024-04-17T10:43:04.911 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-04-17T10:43:04.911 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 6843, 2024-04-17T10:43:04.911 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-17T10:43:04.911 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-17T10:43:04.911 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-17T10:43:04.911 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 4.5393656509999998, 2024-04-17T10:43:04.911 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 39, 2024-04-17T10:43:04.912 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 63, 2024-04-17T10:43:04.912 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 63, 2024-04-17T10:43:04.912 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 5.4306245530000004 2024-04-17T10:43:04.912 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:43:04.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.909+0000 7fe6d1c85640 1 -- 172.21.15.83:0/924682431 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe6ac066f60 msgr2=0x7fe6ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:04.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.909+0000 7fe6d1c85640 1 --2- 172.21.15.83:0/924682431 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe6ac066f60 0x7fe6ac069420 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009ae0 tx=0x7fe6b4008040 comp rx=0 tx=0).stop 2024-04-17T10:43:04.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.909+0000 7fe6d1c85640 1 -- 172.21.15.83:0/924682431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6cc1591c0 msgr2=0x7fe6cc1556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:04.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.909+0000 7fe6d1c85640 1 --2- 172.21.15.83:0/924682431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6cc1591c0 0x7fe6cc1556d0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc0d1480 tx=0x7fe6c400b4a0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.909+0000 7fe6d1c85640 1 -- 172.21.15.83:0/924682431 shutdown_connections 2024-04-17T10:43:04.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.909+0000 7fe6d1c85640 1 --2- 172.21.15.83:0/924682431 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe6ac066f60 0x7fe6ac069420 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.909+0000 7fe6d1c85640 1 --2- 172.21.15.83:0/924682431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6cc1591c0 0x7fe6cc1556d0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.909+0000 7fe6d1c85640 1 --2- 172.21.15.83:0/924682431 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6cc152590 0x7fe6cc158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.909+0000 7fe6d1c85640 1 --2- 172.21.15.83:0/924682431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6b8003fe0 0x7fe6cc14d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:04.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.909+0000 7fe6d1c85640 1 -- 172.21.15.83:0/924682431 >> 172.21.15.83:0/924682431 conn(0x7fe6cc0d0450 msgr2=0x7fe6cc0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:04.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.909+0000 7fe6d1c85640 1 -- 172.21.15.83:0/924682431 shutdown_connections 2024-04-17T10:43:04.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:04.909+0000 7fe6d1c85640 1 -- 172.21.15.83:0/924682431 wait complete. 2024-04-17T10:43:04.919 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T10:43:05.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.021+0000 7ffa6f4ee640 1 Processor -- start 2024-04-17T10:43:05.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.021+0000 7ffa6f4ee640 1 -- start start 2024-04-17T10:43:05.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.025+0000 7ffa6f4ee640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffa68146e70 0x7ffa6806d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:05.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.025+0000 7ffa6f4ee640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa68147570 con 0x7ffa68146a00 2024-04-17T10:43:05.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.025+0000 7ffa6f4ee640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa6806cbb0 con 0x7ffa680b2d80 2024-04-17T10:43:05.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.025+0000 7ffa6f4ee640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa6806c960 con 0x7ffa68146e70 2024-04-17T10:43:05.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.025+0000 7ffa6d28a640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ffa68146a00 0x7ffa680b2670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53730/0 (socket says 172.21.15.83:53730) 2024-04-17T10:43:05.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.025+0000 7ffa6d28a640 1 -- 172.21.15.83:0/490310252 learned_addr learned my addr 172.21.15.83:0/490310252 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:43:05.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.025+0000 7ffa6da8b640 1 --2- 172.21.15.83:0/490310252 >> v2:172.21.15.83:3300/0 conn(0x7ffa68146e70 0x7ffa6806d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:05.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.025+0000 7ffa6da8b640 1 -- 172.21.15.83:0/490310252 >> v1:172.21.15.167:6789/0 conn(0x7ffa680b2d80 legacy=0x7ffa68070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:05.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.025+0000 7ffa6da8b640 1 -- 172.21.15.83:0/490310252 >> v1:172.21.15.83:6789/0 conn(0x7ffa68146a00 legacy=0x7ffa680b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:05.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.025+0000 7ffa6da8b640 1 -- 172.21.15.83:0/490310252 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa6806ce90 con 0x7ffa68146e70 2024-04-17T10:43:05.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.025+0000 7ffa6da8b640 1 --2- 172.21.15.83:0/490310252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa68146e70 0x7ffa6806d690 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009d90 tx=0x7ffa6001c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35313a0990ee5d1b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:05.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.025+0000 7ffa5ffff640 1 -- 172.21.15.83:0/490310252 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa600ad070 con 0x7ffa68146e70 2024-04-17T10:43:05.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.025+0000 7ffa5ffff640 1 -- 172.21.15.83:0/490310252 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa600a9640 con 0x7ffa68146e70 2024-04-17T10:43:05.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.025+0000 7ffa5ffff640 1 -- 172.21.15.83:0/490310252 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa600a9a10 con 0x7ffa68146e70 2024-04-17T10:43:05.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.025+0000 7ffa6f4ee640 1 -- 172.21.15.83:0/490310252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa68146e70 msgr2=0x7ffa6806d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:05.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.025+0000 7ffa6f4ee640 1 --2- 172.21.15.83:0/490310252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa68146e70 0x7ffa6806d690 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009d90 tx=0x7ffa6001c670 comp rx=0 tx=0).stop 2024-04-17T10:43:05.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.025+0000 7ffa6f4ee640 1 -- 172.21.15.83:0/490310252 shutdown_connections 2024-04-17T10:43:05.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.025+0000 7ffa6f4ee640 1 --2- 172.21.15.83:0/490310252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa68146e70 0x7ffa6806d690 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:05.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.025+0000 7ffa6f4ee640 1 -- 172.21.15.83:0/490310252 >> 172.21.15.83:0/490310252 conn(0x7ffa6806abe0 msgr2=0x7ffa6806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:05.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.025+0000 7ffa6f4ee640 1 -- 172.21.15.83:0/490310252 shutdown_connections 2024-04-17T10:43:05.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.025+0000 7ffa6f4ee640 1 -- 172.21.15.83:0/490310252 wait complete. 2024-04-17T10:43:05.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.029+0000 7ffa6f4ee640 1 Processor -- start 2024-04-17T10:43:05.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.029+0000 7ffa6f4ee640 1 -- start start 2024-04-17T10:43:05.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.029+0000 7ffa6f4ee640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa680b2d80 0x7ffa681627a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:05.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.029+0000 7ffa6f4ee640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa68146a00 0x7ffa6815f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:05.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.029+0000 7ffa6f4ee640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa68146e70 0x7ffa681418c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:05.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.029+0000 7ffa6f4ee640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6816c420 con 0x7ffa680b2d80 2024-04-17T10:43:05.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.029+0000 7ffa6f4ee640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa68147320 con 0x7ffa68146e70 2024-04-17T10:43:05.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.029+0000 7ffa6f4ee640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6816c5a0 con 0x7ffa68146a00 2024-04-17T10:43:05.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.029+0000 7ffa6d28a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa680b2d80 0x7ffa681627a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:05.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.029+0000 7ffa6d28a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa680b2d80 0x7ffa681627a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58458/0 (socket says 172.21.15.83:58458) 2024-04-17T10:43:05.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.029+0000 7ffa6d28a640 1 -- 172.21.15.83:0/2539147329 learned_addr learned my addr 172.21.15.83:0/2539147329 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:05.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.029+0000 7ffa6d28a640 1 -- 172.21.15.83:0/2539147329 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa68146a00 msgr2=0x7ffa6815f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:05.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.029+0000 7ffa6d28a640 1 --2- 172.21.15.83:0/2539147329 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa68146a00 0x7ffa6815f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:05.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.029+0000 7ffa6d28a640 1 -- 172.21.15.83:0/2539147329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa68146e70 msgr2=0x7ffa681418c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:05.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.029+0000 7ffa6d28a640 1 --2- 172.21.15.83:0/2539147329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa68146e70 0x7ffa681418c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:05.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.029+0000 7ffa6d28a640 1 -- 172.21.15.83:0/2539147329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa68163110 con 0x7ffa680b2d80 2024-04-17T10:43:05.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.029+0000 7ffa6d28a640 1 --2- 172.21.15.83:0/2539147329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa680b2d80 0x7ffa681627a0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7ffa5800c420 tx=0x7ffa5800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:05.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.029+0000 7ffa5dffb640 1 -- 172.21.15.83:0/2539147329 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa58017070 con 0x7ffa680b2d80 2024-04-17T10:43:05.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.029+0000 7ffa5dffb640 1 -- 172.21.15.83:0/2539147329 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa58005cf0 con 0x7ffa680b2d80 2024-04-17T10:43:05.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.029+0000 7ffa6f4ee640 1 -- 172.21.15.83:0/2539147329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa681633f0 con 0x7ffa680b2d80 2024-04-17T10:43:05.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.029+0000 7ffa5dffb640 1 -- 172.21.15.83:0/2539147329 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa580072d0 con 0x7ffa680b2d80 2024-04-17T10:43:05.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.029+0000 7ffa6f4ee640 1 -- 172.21.15.83:0/2539147329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa68162ce0 con 0x7ffa680b2d80 2024-04-17T10:43:05.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.029+0000 7ffa6f4ee640 1 -- 172.21.15.83:0/2539147329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa68074f90 con 0x7ffa680b2d80 2024-04-17T10:43:05.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.029+0000 7ffa5dffb640 1 -- 172.21.15.83:0/2539147329 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffa5801a070 con 0x7ffa680b2d80 2024-04-17T10:43:05.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.029+0000 7ffa5dffb640 1 --2- 172.21.15.83:0/2539147329 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffa4c066f60 0x7ffa4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:05.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.033+0000 7ffa5dffb640 1 -- 172.21.15.83:0/2539147329 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ffa580842c0 con 0x7ffa680b2d80 2024-04-17T10:43:05.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.033+0000 7ffa6ca89640 1 --2- 172.21.15.83:0/2539147329 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffa4c066f60 0x7ffa4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:05.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.033+0000 7ffa6ca89640 1 --2- 172.21.15.83:0/2539147329 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffa4c066f60 0x7ffa4c069420 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ffa6815d290 tx=0x7ffa640079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:05.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.037+0000 7ffa5dffb640 1 -- 172.21.15.83:0/2539147329 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffa58054cb0 con 0x7ffa680b2d80 2024-04-17T10:43:05.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.213+0000 7ffa6f4ee640 1 -- 172.21.15.83:0/2539147329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ffa680ae6f0 con 0x7ffa680b2d80 2024-04-17T10:43:05.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.217+0000 7ffa5dffb640 1 -- 172.21.15.83:0/2539147329 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v39) v1 ==== 53+0+73 (secure 0 0 0) 0x7ffa58058d50 con 0x7ffa680b2d80 2024-04-17T10:43:05.212 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T10:43:05.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.217+0000 7ffa6f4ee640 1 -- 172.21.15.83:0/2539147329 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffa4c066f60 msgr2=0x7ffa4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:05.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.217+0000 7ffa6f4ee640 1 --2- 172.21.15.83:0/2539147329 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffa4c066f60 0x7ffa4c069420 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ffa6815d290 tx=0x7ffa640079c0 comp rx=0 tx=0).stop 2024-04-17T10:43:05.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.217+0000 7ffa6f4ee640 1 -- 172.21.15.83:0/2539147329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa680b2d80 msgr2=0x7ffa681627a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:05.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.217+0000 7ffa6f4ee640 1 --2- 172.21.15.83:0/2539147329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa680b2d80 0x7ffa681627a0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7ffa5800c420 tx=0x7ffa5800c8e0 comp rx=0 tx=0).stop 2024-04-17T10:43:05.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.217+0000 7ffa6f4ee640 1 -- 172.21.15.83:0/2539147329 shutdown_connections 2024-04-17T10:43:05.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.217+0000 7ffa6f4ee640 1 --2- 172.21.15.83:0/2539147329 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffa4c066f60 0x7ffa4c069420 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:05.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.217+0000 7ffa6f4ee640 1 --2- 172.21.15.83:0/2539147329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa68146e70 0x7ffa681418c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:05.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.217+0000 7ffa6f4ee640 1 --2- 172.21.15.83:0/2539147329 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa68146a00 0x7ffa6815f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:05.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.217+0000 7ffa6f4ee640 1 --2- 172.21.15.83:0/2539147329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa680b2d80 0x7ffa681627a0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:05.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.217+0000 7ffa6f4ee640 1 -- 172.21.15.83:0/2539147329 >> 172.21.15.83:0/2539147329 conn(0x7ffa6806abe0 msgr2=0x7ffa680afb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:05.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.217+0000 7ffa6f4ee640 1 -- 172.21.15.83:0/2539147329 shutdown_connections 2024-04-17T10:43:05.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:05.217+0000 7ffa6f4ee640 1 -- 172.21.15.83:0/2539147329 wait complete. 2024-04-17T10:43:05.229 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T10:43:05.229 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T10:43:05.229 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-04-17T10:43:05.230 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T10:43:05.230 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T10:43:05.230 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T10:43:05.230 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T10:43:05.230 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T10:43:05.230 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:43:05.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:43:05.230 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-04-17T10:43:05.237 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T10:43:05.237 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T10:43:05.237 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T10:43:05.237 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:43:05.238 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-17T10:43:05.238 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:43:05.238 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:43:05.238 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-04-17T10:43:05.238 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-04-17T10:43:05.238 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:43:05.238 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-04-17T10:43:05.238 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-04-17T10:43:05.238 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:43:05.238 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:43:05.238 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T10:43:05.238 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:43:05.239 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-17T10:43:05.239 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42199sec preferred_lft 42199sec 2024-04-17T10:43:05.239 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-04-17T10:43:05.239 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:43:05.239 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T10:43:05.239 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:37:0e:23:b2 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:43:05.239 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T10:43:05.239 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:43:05.239 INFO:teuthology.orchestra.run.smithi083.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T10:43:05.239 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-17T10:43:05.239 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T10:43:05.240 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:43:05.240 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-17T10:43:05.240 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:43:05.240 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:43:05.240 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:43:05.240 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-17T10:43:05.240 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:43:05.240 INFO:teuthology.orchestra.run.smithi083.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:43:05.240 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:43:05.240 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-17T10:43:05.240 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:43:05.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:43:05.241 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T10:43:05.289 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-04-17T10:43:05.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:43:05.291 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-04-17T10:43:05.344 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-04-17T10:43:05.344 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T10:43:05.344 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-17T10:43:05.344 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-17T10:43:05.344 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-17T10:43:05.344 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T10:43:05.344 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-17T10:43:05.344 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:43:05.344 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:43:05.345 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:43:05.345 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T10:43:05.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:43:05.345 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2024-04-17T10:43:05.345 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:43:05.345 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-17T10:43:05.345 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-17T10:43:05.345 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-17T10:43:05.345 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:43:05.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:43:05.419 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-17T10:43:05.468 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-17T10:43:05.468 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T10:43:05.469 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:43:05.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:43:05.469 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-04-17T10:43:05.523 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-17T10:43:05.524 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:43:05.524 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T10:43:05.572 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:43:05.572 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:43:05.573 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:43:05.619 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:43:05.677 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:43:05.726 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:43:05.726 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:43:05.726 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:43:05.727 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:43:05.727 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:43:05.727 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:43:05.727 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:43:05.727 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:43:05.727 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:43:05.727 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:43:05.727 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:43:05.727 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:43:05.727 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:43:05.727 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:43:05.727 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:43:05.727 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:43:05.727 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:43:05.728 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:43:05.728 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:43:05.728 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:43:05.728 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:43:05.728 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:43:05.728 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:43:05.728 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:43:05.728 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:43:05.728 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:43:05.728 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:43:05.728 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:43:05.728 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:43:05.728 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:43:05.729 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:43:05.729 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:43:05.729 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:43:05.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:43:05.730 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:43:05.776 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T10:43:05.776 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-17T10:43:05.821 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:43:05.837 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:43:05.888 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:43:05.888 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:43:05.888 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:43:05.888 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:43:05.888 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:43:05.888 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:43:05.888 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:43:05.888 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:43:05.889 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:43:05.889 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:43:05.889 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:43:05.889 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:43:05.889 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:43:05.889 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:43:05.889 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:43:05.889 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:43:05.889 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:43:05.889 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:43:05.889 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:43:05.889 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:43:05.889 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:43:05.889 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:43:05.890 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:43:05.890 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:43:05.890 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:43:05.890 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:43:05.890 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:43:05.890 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:43:05.890 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:43:05.890 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:43:05.890 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:43:05.890 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:43:05.890 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:43:05.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:43:05.891 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:43:05.902 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:2024-04-17T10:43:05.905+0000 7fdb77c21580 -1 init, newargv = 0x559f74c48250 newargc=15 2024-04-17T10:43:05.902 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[55284]: starting ceph client 2024-04-17T10:43:05.908 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[55284]: starting fuse 2024-04-17T10:43:06.363 INFO:tasks.ceph.mds.a.smithi083.stderr:2024-04-17T10:43:06.369+0000 7f04fa2b8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T10:43:07.006 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:43:07.021 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:43:07.066 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:43:07.068 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:43:07.069 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:43:07.069 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:43:07.069 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:43:07.069 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:43:07.069 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:43:07.069 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:43:07.069 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:43:07.069 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:43:07.069 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:43:07.069 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:43:07.069 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:43:07.070 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:43:07.070 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:43:07.070 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:43:07.070 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:43:07.070 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:43:07.070 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:43:07.070 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:43:07.070 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:43:07.070 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:43:07.070 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:43:07.071 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:43:07.071 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:43:07.071 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:43:07.071 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:43:07.071 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:43:07.071 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:43:07.071 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:43:07.071 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:43:07.071 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:43:07.071 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:43:07.071 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:43:07.071 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-17T10:43:07.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:43:07.073 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:43:07.073 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:43:07.124 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-04-17T10:43:07.124 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-17T10:43:07.124 DEBUG:teuthology.orchestra.run.smithi083:> sudo stdin-killer -- python3 -c ' 2024-04-17T10:43:07.124 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2024-04-17T10:43:07.125 DEBUG:teuthology.orchestra.run.smithi083:> import re 2024-04-17T10:43:07.125 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-04-17T10:43:07.125 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2024-04-17T10:43:07.125 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:43:07.125 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2024-04-17T10:43:07.125 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-17T10:43:07.125 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2024-04-17T10:43:07.125 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-17T10:43:07.125 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:43:07.125 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2024-04-17T10:43:07.125 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2024-04-17T10:43:07.125 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2024-04-17T10:43:07.125 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2024-04-17T10:43:07.126 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:43:07.126 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2024-04-17T10:43:07.126 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T10:43:07.126 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T10:43:07.126 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T10:43:07.126 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2024-04-17T10:43:07.126 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2024-04-17T10:43:07.126 DEBUG:teuthology.orchestra.run.smithi083:> return f 2024-04-17T10:43:07.126 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T10:43:07.126 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:43:07.126 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.0")) 2024-04-17T10:43:07.126 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-04-17T10:43:07.287 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.0.55284.asok 2024-04-17T10:43:07.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:43:07.299 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.55284.asok 2024-04-17T10:43:07.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:43:07.299 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.55284.asok status 2024-04-17T10:43:07.454 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:43:07.454 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2024-04-17T10:43:07.454 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-17T10:43:07.455 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-17T10:43:07.455 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "0", 2024-04-17T10:43:07.455 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-04-17T10:43:07.455 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-17T10:43:07.455 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "55284", 2024-04-17T10:43:07.455 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-04-17T10:43:07.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:43:07.455 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2024-04-17T10:43:07.455 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2024-04-17T10:43:07.455 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 6897, 2024-04-17T10:43:07.455 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2024-04-17T10:43:07.455 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-04-17T10:43:07.455 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-04-17T10:43:07.455 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 6897 2024-04-17T10:43:07.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:43:07.456 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-04-17T10:43:07.456 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-17T10:43:07.456 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:43:07.456 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 194370595 2024-04-17T10:43:07.456 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:43:07.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:43:07.456 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-04-17T10:43:07.457 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-17T10:43:07.457 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:43:07.457 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 194370595 2024-04-17T10:43:07.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:43:07.457 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.6897 192.168.0.1:0/194370595", 2024-04-17T10:43:07.457 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.1:0/194370595", 2024-04-17T10:43:07.457 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 1, 2024-04-17T10:43:07.457 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 39, 2024-04-17T10:43:07.457 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 63, 2024-04-17T10:43:07.457 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2024-04-17T10:43:07.457 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2024-04-17T10:43:07.457 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2024-04-17T10:43:07.457 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:43:07.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:43:07.468 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2024-04-17T10:43:07.468 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-17T10:43:07.475 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2024-04-17T10:43:07.475 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-17T10:43:07.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:43:07.475 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:43:07.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:43:07.529 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir scrub_repair_path) 2024-04-17T10:43:07.579 INFO:teuthology.orchestra.run.smithi083.stderr:rmdir: failed to remove 'scrub_repair_path': Directory not empty 2024-04-17T10:43:07.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:43:07.580 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_repair_path repair' 2024-04-17T10:43:07.580 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-17T10:43:07.580 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start /scrub_repair_path repair 2024-04-17T10:43:07.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.761+0000 7f9d4d771640 1 Processor -- start 2024-04-17T10:43:07.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.761+0000 7f9d4d771640 1 -- start start 2024-04-17T10:43:07.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.761+0000 7f9d4d771640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9d480af190 0x7f9d4806f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:07.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.761+0000 7f9d4d771640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9d480af570 0x7f9d480726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:07.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.761+0000 7f9d4d771640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d48071950 con 0x7f9d48072c10 2024-04-17T10:43:07.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.761+0000 7f9d4d771640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d480cf3e0 con 0x7f9d480af570 2024-04-17T10:43:07.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.761+0000 7f9d4d771640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d48071bf0 con 0x7f9d480af190 2024-04-17T10:43:07.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.761+0000 7f9d467fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9d480af570 0x7f9d480726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:07.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.761+0000 7f9d467fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9d480af570 0x7f9d480726d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34616/0 (socket says 172.21.15.83:34616) 2024-04-17T10:43:07.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.761+0000 7f9d467fc640 1 -- 172.21.15.83:0/2768981869 learned_addr learned my addr 172.21.15.83:0/2768981869 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:07.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.761+0000 7f9d46ffd640 1 --2- 172.21.15.83:0/2768981869 >> v2:172.21.15.167:3300/0 conn(0x7f9d480af190 0x7f9d4806f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:07.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.761+0000 7f9d467fc640 1 -- 172.21.15.83:0/2768981869 >> v2:172.21.15.167:3300/0 conn(0x7f9d480af190 msgr2=0x7f9d4806f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:07.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.761+0000 7f9d467fc640 1 --2- 172.21.15.83:0/2768981869 >> v2:172.21.15.167:3300/0 conn(0x7f9d480af190 0x7f9d4806f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.761+0000 7f9d467fc640 1 -- 172.21.15.83:0/2768981869 >> v1:172.21.15.167:6789/0 conn(0x7f9d48072c10 legacy=0x7f9d4806cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:07.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.761+0000 7f9d467fc640 1 -- 172.21.15.83:0/2768981869 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d48071ed0 con 0x7f9d480af570 2024-04-17T10:43:07.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.765+0000 7f9d467fc640 1 --2- 172.21.15.83:0/2768981869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d480af570 0x7f9d480726d0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f9d480b6050 tx=0x7f9d3c01c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7a5a1c377cbbbb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:07.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.765+0000 7f9d45ffb640 1 -- 172.21.15.83:0/2768981869 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c0b0070 con 0x7f9d480af570 2024-04-17T10:43:07.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.765+0000 7f9d45ffb640 1 -- 172.21.15.83:0/2768981869 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3c004230 con 0x7f9d480af570 2024-04-17T10:43:07.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.765+0000 7f9d45ffb640 1 -- 172.21.15.83:0/2768981869 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c003620 con 0x7f9d480af570 2024-04-17T10:43:07.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.765+0000 7f9d4d771640 1 -- 172.21.15.83:0/2768981869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d480af570 msgr2=0x7f9d480726d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:07.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.765+0000 7f9d4d771640 1 --2- 172.21.15.83:0/2768981869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d480af570 0x7f9d480726d0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f9d480b6050 tx=0x7f9d3c01c4c0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.765+0000 7f9d4d771640 1 -- 172.21.15.83:0/2768981869 shutdown_connections 2024-04-17T10:43:07.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.765+0000 7f9d4d771640 1 --2- 172.21.15.83:0/2768981869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d480af570 0x7f9d480726d0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.765+0000 7f9d4d771640 1 --2- 172.21.15.83:0/2768981869 >> v2:172.21.15.167:3300/0 conn(0x7f9d480af190 0x7f9d4806f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.765+0000 7f9d4d771640 1 -- 172.21.15.83:0/2768981869 >> 172.21.15.83:0/2768981869 conn(0x7f9d480cc920 msgr2=0x7f9d480ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:07.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.765+0000 7f9d4d771640 1 -- 172.21.15.83:0/2768981869 shutdown_connections 2024-04-17T10:43:07.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.765+0000 7f9d4d771640 1 -- 172.21.15.83:0/2768981869 wait complete. 2024-04-17T10:43:07.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.765+0000 7f9d4d771640 1 Processor -- start 2024-04-17T10:43:07.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.765+0000 7f9d4d771640 1 -- start start 2024-04-17T10:43:07.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.765+0000 7f9d4d771640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d48072c10 0x7f9d480b1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:07.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.765+0000 7f9d4d771640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d480af190 0x7f9d4815b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:07.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.765+0000 7f9d4d771640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d480c44c0 0x7f9d480b0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:07.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.765+0000 7f9d4d771640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d480cf170 con 0x7f9d480af190 2024-04-17T10:43:07.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.765+0000 7f9d4d771640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d480ceff0 con 0x7f9d48072c10 2024-04-17T10:43:07.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.765+0000 7f9d4d771640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4806d300 con 0x7f9d480c44c0 2024-04-17T10:43:07.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.765+0000 7f9d467fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d480af190 0x7f9d4815b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:07.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.765+0000 7f9d467fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d480af190 0x7f9d4815b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34628/0 (socket says 172.21.15.83:34628) 2024-04-17T10:43:07.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.765+0000 7f9d467fc640 1 -- 172.21.15.83:0/4174430025 learned_addr learned my addr 172.21.15.83:0/4174430025 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:07.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.765+0000 7f9d46ffd640 1 --2- 172.21.15.83:0/4174430025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d48072c10 0x7f9d480b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:07.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.765+0000 7f9d477fe640 1 --2- 172.21.15.83:0/4174430025 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d480c44c0 0x7f9d480b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:07.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.765+0000 7f9d467fc640 1 -- 172.21.15.83:0/4174430025 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d480c44c0 msgr2=0x7f9d480b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:07.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.765+0000 7f9d467fc640 1 --2- 172.21.15.83:0/4174430025 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d480c44c0 0x7f9d480b0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.765+0000 7f9d467fc640 1 -- 172.21.15.83:0/4174430025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d48072c10 msgr2=0x7f9d480b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:07.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.765+0000 7f9d467fc640 1 --2- 172.21.15.83:0/4174430025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d48072c10 0x7f9d480b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.765+0000 7f9d467fc640 1 -- 172.21.15.83:0/4174430025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d480c4980 con 0x7f9d480af190 2024-04-17T10:43:07.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.765+0000 7f9d467fc640 1 --2- 172.21.15.83:0/4174430025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d480af190 0x7f9d4815b790 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c01c9a0 tx=0x7f9d3c019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:07.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.769+0000 7f9d2f7fe640 1 -- 172.21.15.83:0/4174430025 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c0b0040 con 0x7f9d480af190 2024-04-17T10:43:07.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.769+0000 7f9d2f7fe640 1 -- 172.21.15.83:0/4174430025 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3c004030 con 0x7f9d480af190 2024-04-17T10:43:07.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.769+0000 7f9d4d771640 1 -- 172.21.15.83:0/4174430025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d480c4c60 con 0x7f9d480af190 2024-04-17T10:43:07.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.769+0000 7f9d2f7fe640 1 -- 172.21.15.83:0/4174430025 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c0043e0 con 0x7f9d480af190 2024-04-17T10:43:07.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.769+0000 7f9d4d771640 1 -- 172.21.15.83:0/4174430025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d480c5220 con 0x7f9d480af190 2024-04-17T10:43:07.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.769+0000 7f9d2f7fe640 1 -- 172.21.15.83:0/4174430025 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9d3c0be020 con 0x7f9d480af190 2024-04-17T10:43:07.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.769+0000 7f9d2f7fe640 1 --2- 172.21.15.83:0/4174430025 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9d34066f60 0x7f9d34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:07.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.769+0000 7f9d2f7fe640 1 -- 172.21.15.83:0/4174430025 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9d3c11ed40 con 0x7f9d480af190 2024-04-17T10:43:07.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.769+0000 7f9d46ffd640 1 --2- 172.21.15.83:0/4174430025 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9d34066f60 0x7f9d34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:07.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.769+0000 7f9d46ffd640 1 --2- 172.21.15.83:0/4174430025 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9d34066f60 0x7f9d34069420 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9d3800b930 tx=0x7f9d38008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:07.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.769+0000 7f9d508de000 1 Processor -- start 2024-04-17T10:43:07.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.769+0000 7f9d508de000 1 -- start start 2024-04-17T10:43:07.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.769+0000 7f9d508de000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555ac1eae60 0x5555ac1eb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:07.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.769+0000 7f9d508de000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555ac1eb780 0x5555ac1a3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:07.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.769+0000 7f9d508de000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5555ac1a3d80 0x5555ac1a4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:07.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d508de000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ac088c70 con 0x5555ac1eb780 2024-04-17T10:43:07.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d508de000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555abfef040 con 0x5555ac1eae60 2024-04-17T10:43:07.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d508de000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5555abf77040 con 0x5555ac1a3d80 2024-04-17T10:43:07.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d467fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555ac1eb780 0x5555ac1a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:07.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d467fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555ac1eb780 0x5555ac1a3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34630/0 (socket says 172.21.15.83:34630) 2024-04-17T10:43:07.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d467fc640 1 -- 172.21.15.83:0/2532130811 learned_addr learned my addr 172.21.15.83:0/2532130811 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:07.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d477fe640 1 --2- 172.21.15.83:0/2532130811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5555ac1a3d80 0x5555ac1a4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:07.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d46ffd640 1 --2- 172.21.15.83:0/2532130811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555ac1eae60 0x5555ac1eb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:07.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d467fc640 1 -- 172.21.15.83:0/2532130811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5555ac1a3d80 msgr2=0x5555ac1a4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:07.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d467fc640 1 --2- 172.21.15.83:0/2532130811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5555ac1a3d80 0x5555ac1a4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d467fc640 1 -- 172.21.15.83:0/2532130811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555ac1eae60 msgr2=0x5555ac1eb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:07.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d467fc640 1 --2- 172.21.15.83:0/2532130811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555ac1eae60 0x5555ac1eb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d467fc640 1 -- 172.21.15.83:0/2532130811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555abfa6240 con 0x5555ac1eb780 2024-04-17T10:43:07.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d46ffd640 1 --2- 172.21.15.83:0/2532130811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555ac1eae60 0x5555ac1eb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:07.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d467fc640 1 --2- 172.21.15.83:0/2532130811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555ac1eb780 0x5555ac1a3840 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0f78f0 tx=0x7f9d3c0ef310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3cd9b198fe148e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:07.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d2cff9640 1 -- 172.21.15.83:0/2532130811 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c0efe10 con 0x5555ac1eb780 2024-04-17T10:43:07.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d2cff9640 1 -- 172.21.15.83:0/2532130811 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3c0ef710 con 0x5555ac1eb780 2024-04-17T10:43:07.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d2cff9640 1 -- 172.21.15.83:0/2532130811 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c0f3860 con 0x5555ac1eb780 2024-04-17T10:43:07.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d508de000 1 -- 172.21.15.83:0/2532130811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555ac1eb780 msgr2=0x5555ac1a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:07.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d508de000 1 --2- 172.21.15.83:0/2532130811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555ac1eb780 0x5555ac1a3840 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0f78f0 tx=0x7f9d3c0ef310 comp rx=0 tx=0).stop 2024-04-17T10:43:07.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d508de000 1 -- 172.21.15.83:0/2532130811 shutdown_connections 2024-04-17T10:43:07.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d508de000 1 --2- 172.21.15.83:0/2532130811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5555ac1a3d80 0x5555ac1a4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d508de000 1 --2- 172.21.15.83:0/2532130811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555ac1eb780 0x5555ac1a3840 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d508de000 1 --2- 172.21.15.83:0/2532130811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555ac1eae60 0x5555ac1eb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d508de000 1 -- 172.21.15.83:0/2532130811 >> 172.21.15.83:0/2532130811 conn(0x5555ac19b100 msgr2=0x5555ac1a2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:07.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d508de000 1 -- 172.21.15.83:0/2532130811 shutdown_connections 2024-04-17T10:43:07.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d508de000 1 -- 172.21.15.83:0/2532130811 wait complete. 2024-04-17T10:43:07.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d508de000 1 -- start start 2024-04-17T10:43:07.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d508de000 1 Processor -- start 2024-04-17T10:43:07.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d508de000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555ac1a3d80 0x5555ac1ac210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:07.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d508de000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555ac1eae60 0x5555ac1ac750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:07.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d508de000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5555ac1eb780 0x5555ac1acc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:07.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d508de000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555abfa9540 con 0x5555ac1a3d80 2024-04-17T10:43:07.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d508de000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555abfeb0e0 con 0x5555ac1eae60 2024-04-17T10:43:07.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d508de000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5555ac088c70 con 0x5555ac1eb780 2024-04-17T10:43:07.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d508de000 10 client.0 _ll_register_callbacks cb 0x5555ac0a33a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:43:07.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d46ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555ac1a3d80 0x5555ac1ac210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:07.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d46ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555ac1a3d80 0x5555ac1ac210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34644/0 (socket says 172.21.15.83:34644) 2024-04-17T10:43:07.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d46ffd640 1 -- 172.21.15.83:0/2187877429 learned_addr learned my addr 172.21.15.83:0/2187877429 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:07.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d467fc640 1 --2- 172.21.15.83:0/2187877429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555ac1eae60 0x5555ac1ac750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:07.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d46ffd640 1 -- 172.21.15.83:0/2187877429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5555ac1eb780 msgr2=0x5555ac1acc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:07.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d46ffd640 1 --2- 172.21.15.83:0/2187877429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5555ac1eb780 0x5555ac1acc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d46ffd640 1 -- 172.21.15.83:0/2187877429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555ac1eae60 msgr2=0x5555ac1ac750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:07.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d46ffd640 1 --2- 172.21.15.83:0/2187877429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555ac1eae60 0x5555ac1ac750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d46ffd640 1 -- 172.21.15.83:0/2187877429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5555abfb7410 con 0x5555ac1a3d80 2024-04-17T10:43:07.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.773+0000 7f9d467fc640 1 --2- 172.21.15.83:0/2187877429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555ac1eae60 0x5555ac1ac750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:07.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d46ffd640 1 --2- 172.21.15.83:0/2187877429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555ac1a3d80 0x5555ac1ac210 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f9d38010010 tx=0x7f9d38004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e137b0949992667 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:07.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:43:07.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d25ffb640 1 -- 172.21.15.83:0/2187877429 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d38015820 con 0x5555ac1a3d80 2024-04-17T10:43:07.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d25ffb640 1 -- 172.21.15.83:0/2187877429 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9d380159c0 con 0x5555ac1a3d80 2024-04-17T10:43:07.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d508de000 1 -- 172.21.15.83:0/2187877429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9d3c0ef710 con 0x5555ac1a3d80 2024-04-17T10:43:07.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d25ffb640 1 -- 172.21.15.83:0/2187877429 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d3802e050 con 0x5555ac1a3d80 2024-04-17T10:43:07.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d508de000 10 client.6909 fetch_fsmap learned FSMap version 39 2024-04-17T10:43:07.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d508de000 1 -- 172.21.15.83:0/2187877429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x5555abf77290 con 0x5555ac1a3d80 2024-04-17T10:43:07.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d25ffb640 1 -- 172.21.15.83:0/2187877429 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3c0ef710 con 0x5555ac1a3d80 2024-04-17T10:43:07.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d25ffb640 1 -- 172.21.15.83:0/2187877429 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 39) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9d38031020 con 0x5555ac1a3d80 2024-04-17T10:43:07.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d508de000 10 client.6909 fetch_fsmap finished waiting for FSMap version 39 2024-04-17T10:43:07.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d508de000 10 client.6909 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-17T10:43:07.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d508de000 20 client.6909 populate_metadata read hostname 'smithi083' 2024-04-17T10:43:07.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d508de000 1 --2- 172.21.15.83:0/2187877429 >> [v2:172.21.15.83:6836/3985646487,v1:172.21.15.83:6837/3985646487] conn(0x5555ac1ad560 0x5555ac1fc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:07.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d508de000 4 client.6909 mds_command: new command op to 6843 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:43:07.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d508de000 1 -- 172.21.15.83:0/2187877429 --> [v2:172.21.15.83:6836/3985646487,v1:172.21.15.83:6837/3985646487] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5555abf8ad70 con 0x5555ac1ad560 2024-04-17T10:43:07.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d467fc640 1 --2- 172.21.15.83:0/2187877429 >> [v2:172.21.15.83:6836/3985646487,v1:172.21.15.83:6837/3985646487] conn(0x5555ac1ad560 0x5555ac1fc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:07.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d467fc640 1 --2- 172.21.15.83:0/2187877429 >> [v2:172.21.15.83:6836/3985646487,v1:172.21.15.83:6837/3985646487] conn(0x5555ac1ad560 0x5555ac1fc5e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f741cda5a54533bf server_cookie=bb838972835ee52 in_seq=0 out_seq=0 2024-04-17T10:43:07.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d25ffb640 10 client.6909 ms_handle_connect on v2:172.21.15.83:6836/3985646487 2024-04-17T10:43:07.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d25ffb640 1 -- 172.21.15.83:0/2187877429 <== mds.0 v2:172.21.15.83:6836/3985646487 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5555abf8ad70 con 0x5555ac1ad560 2024-04-17T10:43:07.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d25ffb640 10 client.6909 handle_command_reply: tid=0 2024-04-17T10:43:07.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d508de000 1 client.6909 shutdown 2024-04-17T10:43:07.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d508de000 1 -- 172.21.15.83:0/2187877429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555ac1a3d80 msgr2=0x5555ac1ac210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:07.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d508de000 1 --2- 172.21.15.83:0/2187877429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555ac1a3d80 0x5555ac1ac210 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f9d38010010 tx=0x7f9d38004740 comp rx=0 tx=0).stop 2024-04-17T10:43:07.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d508de000 1 -- 172.21.15.83:0/2187877429 shutdown_connections 2024-04-17T10:43:07.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d508de000 1 --2- 172.21.15.83:0/2187877429 >> [v2:172.21.15.83:6836/3985646487,v1:172.21.15.83:6837/3985646487] conn(0x5555ac1ad560 0x5555ac1fc5e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d508de000 1 --2- 172.21.15.83:0/2187877429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5555ac1eb780 0x5555ac1acc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d25ffb640 0 client.6909 ms_handle_reset on v2:172.21.15.83:6836/3985646487 2024-04-17T10:43:07.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d508de000 1 --2- 172.21.15.83:0/2187877429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555ac1eae60 0x5555ac1ac750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d508de000 1 --2- 172.21.15.83:0/2187877429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555ac1a3d80 0x5555ac1ac210 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d508de000 1 -- 172.21.15.83:0/2187877429 >> 172.21.15.83:0/2187877429 conn(0x5555ac19b100 msgr2=0x5555ac1a2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:07.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d508de000 1 -- 172.21.15.83:0/2187877429 shutdown_connections 2024-04-17T10:43:07.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d508de000 1 -- 172.21.15.83:0/2187877429 wait complete. 2024-04-17T10:43:07.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.777+0000 7f9d508de000 20 client.6909 trim_cache size 0 max 16384 2024-04-17T10:43:07.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.793+0000 7f9d508de000 1 Processor -- start 2024-04-17T10:43:07.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.793+0000 7f9d508de000 1 -- start start 2024-04-17T10:43:07.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.793+0000 7f9d508de000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555ac1a3d80 0x5555ac1fe750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:07.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.793+0000 7f9d508de000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5555ac1eae60 0x5555ac1fc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:07.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.793+0000 7f9d508de000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555ac1eb780 0x5555ac1fcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:07.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.793+0000 7f9d508de000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d482034c0 con 0x5555ac1eb780 2024-04-17T10:43:07.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.793+0000 7f9d508de000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d482026c0 con 0x5555ac1a3d80 2024-04-17T10:43:07.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.793+0000 7f9d508de000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4815a980 con 0x5555ac1eae60 2024-04-17T10:43:07.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.793+0000 7f9d477fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555ac1eb780 0x5555ac1fcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:07.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.793+0000 7f9d477fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555ac1eb780 0x5555ac1fcb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34646/0 (socket says 172.21.15.83:34646) 2024-04-17T10:43:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.793+0000 7f9d477fe640 1 -- 172.21.15.83:0/2997822682 learned_addr learned my addr 172.21.15.83:0/2997822682 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.793+0000 7f9d477fe640 1 -- 172.21.15.83:0/2997822682 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5555ac1eae60 msgr2=0x5555ac1fc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.793+0000 7f9d477fe640 1 --2- 172.21.15.83:0/2997822682 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5555ac1eae60 0x5555ac1fc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.793+0000 7f9d477fe640 1 -- 172.21.15.83:0/2997822682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555ac1a3d80 msgr2=0x5555ac1fe750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.793+0000 7f9d477fe640 1 --2- 172.21.15.83:0/2997822682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555ac1a3d80 0x5555ac1fe750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.793+0000 7f9d477fe640 1 -- 172.21.15.83:0/2997822682 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555ac071890 con 0x5555ac1eb780 2024-04-17T10:43:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.793+0000 7f9d477fe640 1 --2- 172.21.15.83:0/2997822682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555ac1eb780 0x5555ac1fcb20 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f9d40004280 tx=0x7f9d40004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a20959459f7ef88 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.793+0000 7f9d27fff640 1 -- 172.21.15.83:0/2997822682 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d40017020 con 0x5555ac1eb780 2024-04-17T10:43:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.793+0000 7f9d27fff640 1 -- 172.21.15.83:0/2997822682 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4000c070 con 0x5555ac1eb780 2024-04-17T10:43:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.793+0000 7f9d27fff640 1 -- 172.21.15.83:0/2997822682 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4000edf0 con 0x5555ac1eb780 2024-04-17T10:43:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.793+0000 7f9d508de000 1 -- 172.21.15.83:0/2997822682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555ac1eb780 msgr2=0x5555ac1fcb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.793+0000 7f9d508de000 1 --2- 172.21.15.83:0/2997822682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555ac1eb780 0x5555ac1fcb20 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f9d40004280 tx=0x7f9d40004740 comp rx=0 tx=0).stop 2024-04-17T10:43:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.793+0000 7f9d508de000 1 -- 172.21.15.83:0/2997822682 shutdown_connections 2024-04-17T10:43:07.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.793+0000 7f9d508de000 1 --2- 172.21.15.83:0/2997822682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555ac1eb780 0x5555ac1fcb20 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.793+0000 7f9d508de000 1 --2- 172.21.15.83:0/2997822682 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5555ac1eae60 0x5555ac1fc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.793+0000 7f9d508de000 1 --2- 172.21.15.83:0/2997822682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555ac1a3d80 0x5555ac1fe750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.793+0000 7f9d508de000 1 -- 172.21.15.83:0/2997822682 >> 172.21.15.83:0/2997822682 conn(0x5555ac19b100 msgr2=0x5555ac1e8a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:07.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.793+0000 7f9d508de000 1 -- 172.21.15.83:0/2997822682 shutdown_connections 2024-04-17T10:43:07.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.793+0000 7f9d508de000 1 -- 172.21.15.83:0/2997822682 wait complete. 2024-04-17T10:43:07.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d508de000 1 -- start start 2024-04-17T10:43:07.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d508de000 1 Processor -- start 2024-04-17T10:43:07.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d508de000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555ac1a3d80 0x5555ac24be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:07.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d508de000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555ac1eae60 0x5555ac24c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:07.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d508de000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5555ac1eb780 0x5555ac2070b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:07.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d508de000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d48202160 con 0x5555ac1a3d80 2024-04-17T10:43:07.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d508de000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ac1a7df0 con 0x5555ac1eae60 2024-04-17T10:43:07.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d508de000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5555abfa5d30 con 0x5555ac1eb780 2024-04-17T10:43:07.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d508de000 10 client.0 _ll_register_callbacks cb 0x7f9d3c004580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:43:07.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d46ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555ac1a3d80 0x5555ac24be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:07.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d46ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555ac1a3d80 0x5555ac24be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34660/0 (socket says 172.21.15.83:34660) 2024-04-17T10:43:07.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d46ffd640 1 -- 172.21.15.83:0/2110024454 learned_addr learned my addr 172.21.15.83:0/2110024454 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:07.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d46ffd640 1 -- 172.21.15.83:0/2110024454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5555ac1eb780 msgr2=0x5555ac2070b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:07.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d46ffd640 1 --2- 172.21.15.83:0/2110024454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5555ac1eb780 0x5555ac2070b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d467fc640 1 --2- 172.21.15.83:0/2110024454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555ac1eae60 0x5555ac24c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:07.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d46ffd640 1 -- 172.21.15.83:0/2110024454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555ac1eae60 msgr2=0x5555ac24c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:07.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d46ffd640 1 --2- 172.21.15.83:0/2110024454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555ac1eae60 0x5555ac24c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d46ffd640 1 -- 172.21.15.83:0/2110024454 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5555abf83e50 con 0x5555ac1a3d80 2024-04-17T10:43:07.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d46ffd640 1 --2- 172.21.15.83:0/2110024454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555ac1a3d80 0x5555ac24be40 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f9d38021640 tx=0x7f9d380046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b1d9001d73f00d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:07.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:43:07.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d25ffb640 1 -- 172.21.15.83:0/2110024454 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3800fc40 con 0x5555ac1a3d80 2024-04-17T10:43:07.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d508de000 1 -- 172.21.15.83:0/2110024454 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9d4000c070 con 0x5555ac1a3d80 2024-04-17T10:43:07.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d25ffb640 1 -- 172.21.15.83:0/2110024454 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9d38021c30 con 0x5555ac1a3d80 2024-04-17T10:43:07.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d25ffb640 1 -- 172.21.15.83:0/2110024454 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d38043050 con 0x5555ac1a3d80 2024-04-17T10:43:07.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d508de000 10 client.6915 fetch_fsmap learned FSMap version 39 2024-04-17T10:43:07.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d508de000 1 -- 172.21.15.83:0/2110024454 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x5555abf95f80 con 0x5555ac1a3d80 2024-04-17T10:43:07.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d25ffb640 1 -- 172.21.15.83:0/2110024454 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4000c070 con 0x5555ac1a3d80 2024-04-17T10:43:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d25ffb640 1 -- 172.21.15.83:0/2110024454 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 39) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9d38046020 con 0x5555ac1a3d80 2024-04-17T10:43:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d508de000 10 client.6915 fetch_fsmap finished waiting for FSMap version 39 2024-04-17T10:43:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d508de000 10 client.6915 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-17T10:43:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d508de000 20 client.6915 populate_metadata read hostname 'smithi083' 2024-04-17T10:43:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d508de000 1 --2- 172.21.15.83:0/2110024454 >> [v2:172.21.15.83:6836/3985646487,v1:172.21.15.83:6837/3985646487] conn(0x5555ac207d90 0x5555ac2081a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d508de000 4 client.6915 mds_command: new command op to 6843 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_repair_path", "scrubops": ["repair"]}] 2024-04-17T10:43:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d508de000 1 -- 172.21.15.83:0/2110024454 --> [v2:172.21.15.83:6836/3985646487,v1:172.21.15.83:6837/3985646487] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_repair_path", "scrubops": ["repair"]}) v1 -- 0x5555abffc580 con 0x5555ac207d90 2024-04-17T10:43:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d467fc640 1 --2- 172.21.15.83:0/2110024454 >> [v2:172.21.15.83:6836/3985646487,v1:172.21.15.83:6837/3985646487] conn(0x5555ac207d90 0x5555ac2081a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d467fc640 1 --2- 172.21.15.83:0/2110024454 >> [v2:172.21.15.83:6836/3985646487,v1:172.21.15.83:6837/3985646487] conn(0x5555ac207d90 0x5555ac2081a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88fe8882d9709549 server_cookie=42553e0f4b66e7f9 in_seq=0 out_seq=0 2024-04-17T10:43:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d25ffb640 10 client.6915 ms_handle_connect on v2:172.21.15.83:6836/3985646487 2024-04-17T10:43:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d25ffb640 1 -- 172.21.15.83:0/2110024454 <== mds.0 v2:172.21.15.83:6836/3985646487 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5555abffc580 con 0x5555ac207d90 2024-04-17T10:43:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d25ffb640 10 client.6915 handle_command_reply: tid=0 2024-04-17T10:43:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d508de000 1 client.6915 shutdown 2024-04-17T10:43:07.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d508de000 1 -- 172.21.15.83:0/2110024454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555ac1a3d80 msgr2=0x5555ac24be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:07.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d508de000 1 --2- 172.21.15.83:0/2110024454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555ac1a3d80 0x5555ac24be40 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f9d38021640 tx=0x7f9d380046d0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d508de000 1 -- 172.21.15.83:0/2110024454 shutdown_connections 2024-04-17T10:43:07.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d508de000 1 --2- 172.21.15.83:0/2110024454 >> [v2:172.21.15.83:6836/3985646487,v1:172.21.15.83:6837/3985646487] conn(0x5555ac207d90 0x5555ac2081a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d508de000 1 --2- 172.21.15.83:0/2110024454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5555ac1eb780 0x5555ac2070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d508de000 1 --2- 172.21.15.83:0/2110024454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555ac1eae60 0x5555ac24c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d25ffb640 0 client.6915 ms_handle_reset on v2:172.21.15.83:6836/3985646487 2024-04-17T10:43:07.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d508de000 1 --2- 172.21.15.83:0/2110024454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555ac1a3d80 0x5555ac24be40 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d508de000 1 -- 172.21.15.83:0/2110024454 >> 172.21.15.83:0/2110024454 conn(0x5555ac19b100 msgr2=0x5555ac1e9120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:07.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d508de000 1 -- 172.21.15.83:0/2110024454 shutdown_connections 2024-04-17T10:43:07.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d508de000 1 -- 172.21.15.83:0/2110024454 wait complete. 2024-04-17T10:43:07.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d508de000 20 client.6915 trim_cache size 0 max 16384 2024-04-17T10:43:07.801 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:43:07.801 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-17T10:43:07.801 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "781c6ed0-9cd1-4b43-b32c-0b5efe688638", 2024-04-17T10:43:07.801 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-17T10:43:07.801 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:43:07.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d4d771640 1 -- 172.21.15.83:0/4174430025 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9d34066f60 msgr2=0x7f9d34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:07.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d4d771640 1 --2- 172.21.15.83:0/4174430025 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9d34066f60 0x7f9d34069420 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9d3800b930 tx=0x7f9d38008040 comp rx=0 tx=0).stop 2024-04-17T10:43:07.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d4d771640 1 -- 172.21.15.83:0/4174430025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d480af190 msgr2=0x7f9d4815b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:07.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.797+0000 7f9d4d771640 1 --2- 172.21.15.83:0/4174430025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d480af190 0x7f9d4815b790 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c01c9a0 tx=0x7f9d3c019260 comp rx=0 tx=0).stop 2024-04-17T10:43:07.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.801+0000 7f9d4d771640 1 -- 172.21.15.83:0/4174430025 shutdown_connections 2024-04-17T10:43:07.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.801+0000 7f9d4d771640 1 --2- 172.21.15.83:0/4174430025 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9d34066f60 0x7f9d34069420 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.801+0000 7f9d4d771640 1 --2- 172.21.15.83:0/4174430025 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d480c44c0 0x7f9d480b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.801+0000 7f9d4d771640 1 --2- 172.21.15.83:0/4174430025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d480af190 0x7f9d4815b790 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.801+0000 7f9d4d771640 1 --2- 172.21.15.83:0/4174430025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d48072c10 0x7f9d480b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:07.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.801+0000 7f9d4d771640 1 -- 172.21.15.83:0/4174430025 >> 172.21.15.83:0/4174430025 conn(0x7f9d480cc920 msgr2=0x7f9d480cb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:07.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.801+0000 7f9d4d771640 1 -- 172.21.15.83:0/4174430025 shutdown_connections 2024-04-17T10:43:07.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:07.801+0000 7f9d4d771640 1 -- 172.21.15.83:0/4174430025 wait complete. 2024-04-17T10:43:07.812 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_repair_path repair' returned '{'return_code': 0, 'scrub_tag': '781c6ed0-9cd1-4b43-b32c-0b5efe688638', 'mode': 'asynchronous'}' 2024-04-17T10:43:17.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:43:17.812 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir scrub_repair_path) 2024-04-17T10:43:17.825 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T10:43:17.825 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:43:17.901 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:43:17.948 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T10:43:17.948 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:43:17.948 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:43:17.955 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:43:18.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc659259640 1 Processor -- start 2024-04-17T10:43:18.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc659259640 1 -- start start 2024-04-17T10:43:18.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc659259640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6540ce980 0x7fc6540b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:18.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc659259640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc654162a00 con 0x7fc65415dfa0 2024-04-17T10:43:18.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc659259640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc654161e80 con 0x7fc6540cee00 2024-04-17T10:43:18.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc659259640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc654162160 con 0x7fc6540ce980 2024-04-17T10:43:18.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc652d9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc65415dfa0 0x7fc6540b6c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52196/0 (socket says 172.21.15.83:52196) 2024-04-17T10:43:18.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc652d9d640 1 -- 172.21.15.83:0/1243263349 learned_addr learned my addr 172.21.15.83:0/1243263349 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:43:18.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc65259c640 1 --2- 172.21.15.83:0/1243263349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6540ce980 0x7fc6540b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:18.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc651d9b640 1 -- 172.21.15.83:0/1243263349 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3468507249 0 0) 0x7fc654162a00 con 0x7fc65415dfa0 2024-04-17T10:43:18.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc651d9b640 1 -- 172.21.15.83:0/1243263349 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc640003530 con 0x7fc65415dfa0 2024-04-17T10:43:18.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc651d9b640 1 -- 172.21.15.83:0/1243263349 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2250499047 0 0) 0x7fc640003530 con 0x7fc65415dfa0 2024-04-17T10:43:18.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc651d9b640 1 -- 172.21.15.83:0/1243263349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6540ce980 msgr2=0x7fc6540b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:18.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc651d9b640 1 --2- 172.21.15.83:0/1243263349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6540ce980 0x7fc6540b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:18.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc651d9b640 1 -- 172.21.15.83:0/1243263349 >> v1:172.21.15.167:6789/0 conn(0x7fc6540cee00 legacy=0x7fc6540d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:18.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc651d9b640 1 -- 172.21.15.83:0/1243263349 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc654162440 con 0x7fc65415dfa0 2024-04-17T10:43:18.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc651d9b640 1 -- 172.21.15.83:0/1243263349 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fc64c001680 con 0x7fc65415dfa0 2024-04-17T10:43:18.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc651d9b640 1 -- 172.21.15.83:0/1243263349 >> v1:172.21.15.83:6789/0 conn(0x7fc65415dfa0 legacy=0x7fc6540b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:18.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc651d9b640 1 --2- 172.21.15.83:0/1243263349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6400039f0 0x7fc640003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:18.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc651d9b640 1 --2- 172.21.15.83:0/1243263349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc640004810 0x7fc640004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:18.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc651d9b640 1 --2- 172.21.15.83:0/1243263349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6540ce980 0x7fc640005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:18.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc651d9b640 1 -- 172.21.15.83:0/1243263349 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc654162160 con 0x7fc640004810 2024-04-17T10:43:18.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc651d9b640 1 -- 172.21.15.83:0/1243263349 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64000bb00 con 0x7fc6400039f0 2024-04-17T10:43:18.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc651d9b640 1 -- 172.21.15.83:0/1243263349 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc64000bd00 con 0x7fc6540ce980 2024-04-17T10:43:18.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc65259c640 1 --2- 172.21.15.83:0/1243263349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc640004810 0x7fc640004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:18.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc652d9d640 1 --2- 172.21.15.83:0/1243263349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6400039f0 0x7fc640003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:18.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc65359e640 1 --2- 172.21.15.83:0/1243263349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6540ce980 0x7fc640005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:18.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc65259c640 1 -- 172.21.15.83:0/1243263349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6540ce980 msgr2=0x7fc640005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:18.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc65259c640 1 --2- 172.21.15.83:0/1243263349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6540ce980 0x7fc640005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:18.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc65259c640 1 -- 172.21.15.83:0/1243263349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6400039f0 msgr2=0x7fc640003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:18.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc65259c640 1 --2- 172.21.15.83:0/1243263349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6400039f0 0x7fc640003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:18.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc65259c640 1 -- 172.21.15.83:0/1243263349 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc640003530 con 0x7fc640004810 2024-04-17T10:43:18.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.133+0000 7fc65359e640 1 --2- 172.21.15.83:0/1243263349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6540ce980 0x7fc640005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:18.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.137+0000 7fc65259c640 1 --2- 172.21.15.83:0/1243263349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc640004810 0x7fc640004c90 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fc63c009ab0 tx=0x7fc63c01ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72730ecff5d875c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:18.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.137+0000 7fc651d9b640 1 -- 172.21.15.83:0/1243263349 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63c0b2070 con 0x7fc640004810 2024-04-17T10:43:18.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.137+0000 7fc651d9b640 1 -- 172.21.15.83:0/1243263349 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc63c0a5590 con 0x7fc640004810 2024-04-17T10:43:18.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.137+0000 7fc651d9b640 1 -- 172.21.15.83:0/1243263349 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63c0096b0 con 0x7fc640004810 2024-04-17T10:43:18.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.137+0000 7fc659259640 1 -- 172.21.15.83:0/1243263349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc640004810 msgr2=0x7fc640004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:18.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.137+0000 7fc659259640 1 --2- 172.21.15.83:0/1243263349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc640004810 0x7fc640004c90 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fc63c009ab0 tx=0x7fc63c01ca20 comp rx=0 tx=0).stop 2024-04-17T10:43:18.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.137+0000 7fc65359e640 1 -- 172.21.15.83:0/1243263349 reap_dead start 2024-04-17T10:43:18.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.137+0000 7fc659259640 1 -- 172.21.15.83:0/1243263349 shutdown_connections 2024-04-17T10:43:18.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.137+0000 7fc659259640 1 -- 172.21.15.83:0/1243263349 >> 172.21.15.83:0/1243263349 conn(0x7fc6540d0600 msgr2=0x7fc6540d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:18.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.137+0000 7fc659259640 1 -- 172.21.15.83:0/1243263349 shutdown_connections 2024-04-17T10:43:18.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.137+0000 7fc659259640 1 -- 172.21.15.83:0/1243263349 wait complete. 2024-04-17T10:43:18.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.137+0000 7fc659259640 1 Processor -- start 2024-04-17T10:43:18.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.137+0000 7fc659259640 1 -- start start 2024-04-17T10:43:18.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.137+0000 7fc659259640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc640004810 0x7fc6540c2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:18.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.137+0000 7fc659259640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6540c6800 0x7fc6540c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:18.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.137+0000 7fc659259640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6540cb230 0x7fc6540cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:18.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.137+0000 7fc659259640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc654169e50 con 0x7fc640004810 2024-04-17T10:43:18.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.137+0000 7fc659259640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc654161b10 con 0x7fc6540c6800 2024-04-17T10:43:18.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.137+0000 7fc659259640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc654169fd0 con 0x7fc6540cb230 2024-04-17T10:43:18.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.137+0000 7fc652d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc640004810 0x7fc6540c2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:18.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.137+0000 7fc652d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc640004810 0x7fc6540c2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51718/0 (socket says 172.21.15.83:51718) 2024-04-17T10:43:18.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.137+0000 7fc652d9d640 1 -- 172.21.15.83:0/1896679890 learned_addr learned my addr 172.21.15.83:0/1896679890 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:18.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.137+0000 7fc65259c640 1 --2- 172.21.15.83:0/1896679890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6540c6800 0x7fc6540c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:18.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.137+0000 7fc65359e640 1 --2- 172.21.15.83:0/1896679890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6540cb230 0x7fc6540cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:18.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.137+0000 7fc652d9d640 1 -- 172.21.15.83:0/1896679890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6540cb230 msgr2=0x7fc6540cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:18.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.137+0000 7fc652d9d640 1 --2- 172.21.15.83:0/1896679890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6540cb230 0x7fc6540cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:18.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.137+0000 7fc652d9d640 1 -- 172.21.15.83:0/1896679890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6540c6800 msgr2=0x7fc6540c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:18.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.137+0000 7fc652d9d640 1 --2- 172.21.15.83:0/1896679890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6540c6800 0x7fc6540c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:18.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.137+0000 7fc652d9d640 1 -- 172.21.15.83:0/1896679890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6540c5190 con 0x7fc640004810 2024-04-17T10:43:18.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.141+0000 7fc652d9d640 1 --2- 172.21.15.83:0/1896679890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc640004810 0x7fc6540c2160 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fc64c000f70 tx=0x7fc64c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:18.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.141+0000 7fc64b7fe640 1 -- 172.21.15.83:0/1896679890 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64c00c8d0 con 0x7fc640004810 2024-04-17T10:43:18.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.141+0000 7fc64b7fe640 1 -- 172.21.15.83:0/1896679890 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64c018070 con 0x7fc640004810 2024-04-17T10:43:18.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.141+0000 7fc659259640 1 -- 172.21.15.83:0/1896679890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6540cbe10 con 0x7fc640004810 2024-04-17T10:43:18.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.141+0000 7fc64b7fe640 1 -- 172.21.15.83:0/1896679890 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64c00ccd0 con 0x7fc640004810 2024-04-17T10:43:18.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.141+0000 7fc659259640 1 -- 172.21.15.83:0/1896679890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6540cc3d0 con 0x7fc640004810 2024-04-17T10:43:18.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.141+0000 7fc659259640 1 -- 172.21.15.83:0/1896679890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6540681a0 con 0x7fc640004810 2024-04-17T10:43:18.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.141+0000 7fc64b7fe640 1 -- 172.21.15.83:0/1896679890 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc64c01d460 con 0x7fc640004810 2024-04-17T10:43:18.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.141+0000 7fc64b7fe640 1 --2- 172.21.15.83:0/1896679890 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc634066f60 0x7fc634069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:18.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.141+0000 7fc65259c640 1 --2- 172.21.15.83:0/1896679890 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc634066f60 0x7fc634069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:18.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.141+0000 7fc64b7fe640 1 -- 172.21.15.83:0/1896679890 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc64c084fc0 con 0x7fc640004810 2024-04-17T10:43:18.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.141+0000 7fc65259c640 1 --2- 172.21.15.83:0/1896679890 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc634066f60 0x7fc634069420 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc63c0af000 tx=0x7fc63c0046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:18.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.145+0000 7fc64b7fe640 1 -- 172.21.15.83:0/1896679890 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc64c056000 con 0x7fc640004810 2024-04-17T10:43:18.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.305+0000 7fc659259640 1 -- 172.21.15.83:0/1896679890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc654074f90 con 0x7fc640004810 2024-04-17T10:43:18.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.305+0000 7fc64b7fe640 1 -- 172.21.15.83:0/1896679890 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16722 (secure 0 0 0) 0x7fc64c055900 con 0x7fc640004810 2024-04-17T10:43:18.302 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:43:18.303 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":63,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:42:59.360496+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:42:32.530700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-17T10:42:33.334905+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.167:6833/271994303":"2024-04-18T10:42:59.346779+0000","172.21.15.167:6832/271994303":"2024-04-18T10:42:59.346779+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:43:18.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.309+0000 7fc659259640 1 -- 172.21.15.83:0/1896679890 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc634066f60 msgr2=0x7fc634069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:18.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.309+0000 7fc659259640 1 --2- 172.21.15.83:0/1896679890 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc634066f60 0x7fc634069420 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc63c0af000 tx=0x7fc63c0046d0 comp rx=0 tx=0).stop 2024-04-17T10:43:18.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.309+0000 7fc659259640 1 -- 172.21.15.83:0/1896679890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc640004810 msgr2=0x7fc6540c2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:18.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.309+0000 7fc659259640 1 --2- 172.21.15.83:0/1896679890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc640004810 0x7fc6540c2160 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fc64c000f70 tx=0x7fc64c00ece0 comp rx=0 tx=0).stop 2024-04-17T10:43:18.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.309+0000 7fc659259640 1 -- 172.21.15.83:0/1896679890 shutdown_connections 2024-04-17T10:43:18.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.309+0000 7fc659259640 1 --2- 172.21.15.83:0/1896679890 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc634066f60 0x7fc634069420 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:18.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.309+0000 7fc659259640 1 --2- 172.21.15.83:0/1896679890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6540cb230 0x7fc6540cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:18.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.309+0000 7fc659259640 1 --2- 172.21.15.83:0/1896679890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6540c6800 0x7fc6540c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:18.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.309+0000 7fc659259640 1 --2- 172.21.15.83:0/1896679890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc640004810 0x7fc6540c2160 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:18.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.309+0000 7fc659259640 1 -- 172.21.15.83:0/1896679890 >> 172.21.15.83:0/1896679890 conn(0x7fc6540d0600 msgr2=0x7fc6540d2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:18.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.309+0000 7fc659259640 1 -- 172.21.15.83:0/1896679890 shutdown_connections 2024-04-17T10:43:18.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.309+0000 7fc659259640 1 -- 172.21.15.83:0/1896679890 wait complete. 2024-04-17T10:43:18.320 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:42:32.530700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:42:33.334905+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:43:18.321 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:43:18.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.437+0000 7f175175a640 1 Processor -- start 2024-04-17T10:43:18.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.437+0000 7f175175a640 1 -- start start 2024-04-17T10:43:18.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.441+0000 7f175175a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f174c15e0f0 0x7f174c0b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:18.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.441+0000 7f175175a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f174c0ce400 0x7f174c161c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:18.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.441+0000 7f175175a640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f174c0b5450 0x7f174c0b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:18.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.441+0000 7f175175a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f174c0b5df0 con 0x7f174c15e0f0 2024-04-17T10:43:18.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.441+0000 7f175175a640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f174c162b40 con 0x7f174c0b5450 2024-04-17T10:43:18.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.441+0000 7f175175a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f174c15e4d0 con 0x7f174c0ce400 2024-04-17T10:43:18.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.441+0000 7f174affd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f174c15e0f0 0x7f174c0b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:18.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.441+0000 7f174affd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f174c15e0f0 0x7f174c0b6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51734/0 (socket says 172.21.15.83:51734) 2024-04-17T10:43:18.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.441+0000 7f174affd640 1 -- 172.21.15.83:0/2775252260 learned_addr learned my addr 172.21.15.83:0/2775252260 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:18.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.441+0000 7f174b7fe640 1 --2- 172.21.15.83:0/2775252260 >> v2:172.21.15.167:3300/0 conn(0x7f174c0b5450 0x7f174c0b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:18.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.441+0000 7f174affd640 1 -- 172.21.15.83:0/2775252260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f174c0ce400 msgr2=0x7f174c161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:18.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.441+0000 7f174affd640 1 --2- 172.21.15.83:0/2775252260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f174c0ce400 0x7f174c161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:18.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.441+0000 7f174affd640 1 -- 172.21.15.83:0/2775252260 >> v2:172.21.15.167:3300/0 conn(0x7f174c0b5450 msgr2=0x7f174c0b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:18.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.441+0000 7f174affd640 1 --2- 172.21.15.83:0/2775252260 >> v2:172.21.15.167:3300/0 conn(0x7f174c0b5450 0x7f174c0b58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:18.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.441+0000 7f174affd640 1 -- 172.21.15.83:0/2775252260 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f174c162580 con 0x7f174c15e0f0 2024-04-17T10:43:18.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.441+0000 7f174b7fe640 1 --2- 172.21.15.83:0/2775252260 >> v2:172.21.15.167:3300/0 conn(0x7f174c0b5450 0x7f174c0b58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:18.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.441+0000 7f174affd640 1 --2- 172.21.15.83:0/2775252260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f174c15e0f0 0x7f174c0b6d00 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f174c0b94e0 tx=0x7f173c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b39bf5faba13865 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:18.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.441+0000 7f1749ffb640 1 -- 172.21.15.83:0/2775252260 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c0b0070 con 0x7f174c15e0f0 2024-04-17T10:43:18.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f1749ffb640 1 -- 172.21.15.83:0/2775252260 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173c0ab600 con 0x7f174c15e0f0 2024-04-17T10:43:18.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f1749ffb640 1 -- 172.21.15.83:0/2775252260 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c0ab9d0 con 0x7f174c15e0f0 2024-04-17T10:43:18.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f175175a640 1 -- 172.21.15.83:0/2775252260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f174c15e0f0 msgr2=0x7f174c0b6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:18.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f175175a640 1 --2- 172.21.15.83:0/2775252260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f174c15e0f0 0x7f174c0b6d00 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f174c0b94e0 tx=0x7f173c01c490 comp rx=0 tx=0).stop 2024-04-17T10:43:18.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f175175a640 1 -- 172.21.15.83:0/2775252260 shutdown_connections 2024-04-17T10:43:18.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f175175a640 1 --2- 172.21.15.83:0/2775252260 >> v2:172.21.15.167:3300/0 conn(0x7f174c0b5450 0x7f174c0b58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:18.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f175175a640 1 --2- 172.21.15.83:0/2775252260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f174c0ce400 0x7f174c161c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:18.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f175175a640 1 --2- 172.21.15.83:0/2775252260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f174c15e0f0 0x7f174c0b6d00 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:18.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f175175a640 1 -- 172.21.15.83:0/2775252260 >> 172.21.15.83:0/2775252260 conn(0x7f174c0d0450 msgr2=0x7f174c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:18.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f175175a640 1 -- 172.21.15.83:0/2775252260 shutdown_connections 2024-04-17T10:43:18.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f175175a640 1 -- 172.21.15.83:0/2775252260 wait complete. 2024-04-17T10:43:18.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f175175a640 1 Processor -- start 2024-04-17T10:43:18.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f175175a640 1 -- start start 2024-04-17T10:43:18.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f175175a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f174c0b5450 0x7f174c0cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:18.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f175175a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f174c0ce400 0x7f174c14f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:18.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f175175a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f174c15e0f0 0x7f174c0b8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:18.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f175175a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174c1622d0 con 0x7f174c15e0f0 2024-04-17T10:43:18.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f175175a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174c162150 con 0x7f174c0ce400 2024-04-17T10:43:18.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f175175a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f174c0d2d50 con 0x7f174c0b5450 2024-04-17T10:43:18.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f174b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f174c15e0f0 0x7f174c0b8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:18.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f174b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f174c15e0f0 0x7f174c0b8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51746/0 (socket says 172.21.15.83:51746) 2024-04-17T10:43:18.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f174b7fe640 1 -- 172.21.15.83:0/2327394397 learned_addr learned my addr 172.21.15.83:0/2327394397 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:18.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f174a7fc640 1 --2- 172.21.15.83:0/2327394397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f174c0ce400 0x7f174c14f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:18.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f174b7fe640 1 -- 172.21.15.83:0/2327394397 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f174c0b5450 msgr2=0x7f174c0cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:18.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f174b7fe640 1 --2- 172.21.15.83:0/2327394397 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f174c0b5450 0x7f174c0cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:18.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f174b7fe640 1 -- 172.21.15.83:0/2327394397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f174c0ce400 msgr2=0x7f174c14f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:18.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f174b7fe640 1 --2- 172.21.15.83:0/2327394397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f174c0ce400 0x7f174c14f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:18.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f174b7fe640 1 -- 172.21.15.83:0/2327394397 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f174c0cb210 con 0x7f174c15e0f0 2024-04-17T10:43:18.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f174b7fe640 1 --2- 172.21.15.83:0/2327394397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f174c15e0f0 0x7f174c0b8630 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f174400cef0 tx=0x7f174400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:18.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f17337fe640 1 -- 172.21.15.83:0/2327394397 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174401a070 con 0x7f174c15e0f0 2024-04-17T10:43:18.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f17337fe640 1 -- 172.21.15.83:0/2327394397 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174400d860 con 0x7f174c15e0f0 2024-04-17T10:43:18.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f175175a640 1 -- 172.21.15.83:0/2327394397 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f174c0cb4f0 con 0x7f174c15e0f0 2024-04-17T10:43:18.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f17337fe640 1 -- 172.21.15.83:0/2327394397 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174400dc50 con 0x7f174c15e0f0 2024-04-17T10:43:18.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.445+0000 7f175175a640 1 -- 172.21.15.83:0/2327394397 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174c14fa50 con 0x7f174c15e0f0 2024-04-17T10:43:18.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.449+0000 7f175175a640 1 -- 172.21.15.83:0/2327394397 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f174c0681a0 con 0x7f174c15e0f0 2024-04-17T10:43:18.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.449+0000 7f17337fe640 1 -- 172.21.15.83:0/2327394397 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f174401d070 con 0x7f174c15e0f0 2024-04-17T10:43:18.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.449+0000 7f17337fe640 1 --2- 172.21.15.83:0/2327394397 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1738066f60 0x7f1738069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:18.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.449+0000 7f17337fe640 1 -- 172.21.15.83:0/2327394397 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1744085870 con 0x7f174c15e0f0 2024-04-17T10:43:18.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.449+0000 7f174affd640 1 --2- 172.21.15.83:0/2327394397 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1738066f60 0x7f1738069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:18.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.449+0000 7f174affd640 1 --2- 172.21.15.83:0/2327394397 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1738066f60 0x7f1738069420 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f174c08c3f0 tx=0x7f173c004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:18.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.453+0000 7f17337fe640 1 -- 172.21.15.83:0/2327394397 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1744056970 con 0x7f174c15e0f0 2024-04-17T10:43:18.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.613+0000 7f175175a640 1 -- 172.21.15.83:0/2327394397 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f174c074f90 con 0x7f174c15e0f0 2024-04-17T10:43:18.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.613+0000 7f17337fe640 1 -- 172.21.15.83:0/2327394397 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1744056270 con 0x7f174c15e0f0 2024-04-17T10:43:18.608 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:43:18.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.613+0000 7f175175a640 1 -- 172.21.15.83:0/2327394397 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1738066f60 msgr2=0x7f1738069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:18.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.613+0000 7f175175a640 1 --2- 172.21.15.83:0/2327394397 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1738066f60 0x7f1738069420 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f174c08c3f0 tx=0x7f173c004740 comp rx=0 tx=0).stop 2024-04-17T10:43:18.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.613+0000 7f175175a640 1 -- 172.21.15.83:0/2327394397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f174c15e0f0 msgr2=0x7f174c0b8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:18.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.613+0000 7f175175a640 1 --2- 172.21.15.83:0/2327394397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f174c15e0f0 0x7f174c0b8630 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f174400cef0 tx=0x7f174400a710 comp rx=0 tx=0).stop 2024-04-17T10:43:18.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.613+0000 7f175175a640 1 -- 172.21.15.83:0/2327394397 shutdown_connections 2024-04-17T10:43:18.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.613+0000 7f175175a640 1 --2- 172.21.15.83:0/2327394397 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1738066f60 0x7f1738069420 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:18.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.613+0000 7f175175a640 1 --2- 172.21.15.83:0/2327394397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f174c15e0f0 0x7f174c0b8630 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:18.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.613+0000 7f175175a640 1 --2- 172.21.15.83:0/2327394397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f174c0ce400 0x7f174c14f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:18.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.613+0000 7f175175a640 1 --2- 172.21.15.83:0/2327394397 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f174c0b5450 0x7f174c0cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:18.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.613+0000 7f175175a640 1 -- 172.21.15.83:0/2327394397 >> 172.21.15.83:0/2327394397 conn(0x7f174c0d0450 msgr2=0x7f174c15e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:18.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.613+0000 7f175175a640 1 -- 172.21.15.83:0/2327394397 shutdown_connections 2024-04-17T10:43:18.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.613+0000 7f175175a640 1 -- 172.21.15.83:0/2327394397 wait complete. 2024-04-17T10:43:18.626 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:43:18.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc09acc4640 1 Processor -- start 2024-04-17T10:43:18.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc09acc4640 1 -- start start 2024-04-17T10:43:18.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc09acc4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc094154670 0x7fc0940aea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:18.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc09acc4640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0940b2ff0 con 0x7fc09406cc80 2024-04-17T10:43:18.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc09acc4640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0940b32d0 con 0x7fc0940b4090 2024-04-17T10:43:18.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc09acc4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc094154a50 con 0x7fc094154670 2024-04-17T10:43:18.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc099261640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc09406cc80 0x7fc094070470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52208/0 (socket says 172.21.15.83:52208) 2024-04-17T10:43:18.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc099261640 1 -- 172.21.15.83:0/3075402342 learned_addr learned my addr 172.21.15.83:0/3075402342 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:43:18.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc098a60640 1 --2- 172.21.15.83:0/3075402342 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc094154670 0x7fc0940aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:18.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc0937fe640 1 -- 172.21.15.83:0/3075402342 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1630774062 0 0) 0x7fc0940b2ff0 con 0x7fc09406cc80 2024-04-17T10:43:18.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc0937fe640 1 -- 172.21.15.83:0/3075402342 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc084003530 con 0x7fc09406cc80 2024-04-17T10:43:18.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc0937fe640 1 -- 172.21.15.83:0/3075402342 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2134691197 0 0) 0x7fc084003530 con 0x7fc09406cc80 2024-04-17T10:43:18.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc0937fe640 1 -- 172.21.15.83:0/3075402342 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc094154670 msgr2=0x7fc0940aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:18.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc0937fe640 1 --2- 172.21.15.83:0/3075402342 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc094154670 0x7fc0940aea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:18.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc0937fe640 1 -- 172.21.15.83:0/3075402342 >> v1:172.21.15.167:6789/0 conn(0x7fc0940b4090 legacy=0x7fc09406c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:18.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc0937fe640 1 -- 172.21.15.83:0/3075402342 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0940b35b0 con 0x7fc09406cc80 2024-04-17T10:43:18.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc0937fe640 1 -- 172.21.15.83:0/3075402342 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fc08c001a00 con 0x7fc09406cc80 2024-04-17T10:43:18.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc0937fe640 1 -- 172.21.15.83:0/3075402342 >> v1:172.21.15.83:6789/0 conn(0x7fc09406cc80 legacy=0x7fc094070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:18.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc0937fe640 1 --2- 172.21.15.83:0/3075402342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc084003a60 0x7fc084003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:18.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc0937fe640 1 --2- 172.21.15.83:0/3075402342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc084004810 0x7fc084004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:18.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc0937fe640 1 --2- 172.21.15.83:0/3075402342 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc094154670 0x7fc084005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:18.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc0937fe640 1 -- 172.21.15.83:0/3075402342 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc094154a50 con 0x7fc084003a60 2024-04-17T10:43:18.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc0937fe640 1 -- 172.21.15.83:0/3075402342 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08400cca0 con 0x7fc084004810 2024-04-17T10:43:18.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc0937fe640 1 -- 172.21.15.83:0/3075402342 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc08400cea0 con 0x7fc094154670 2024-04-17T10:43:18.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc098a60640 1 --2- 172.21.15.83:0/3075402342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc084003a60 0x7fc084003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:18.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc093fff640 1 --2- 172.21.15.83:0/3075402342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc084004810 0x7fc084004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:18.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc099261640 1 --2- 172.21.15.83:0/3075402342 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc094154670 0x7fc084005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:18.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc098a60640 1 -- 172.21.15.83:0/3075402342 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc094154670 msgr2=0x7fc084005280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:18.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc098a60640 1 --2- 172.21.15.83:0/3075402342 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc094154670 0x7fc084005280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:18.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc098a60640 1 -- 172.21.15.83:0/3075402342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc084004810 msgr2=0x7fc084004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:18.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc098a60640 1 --2- 172.21.15.83:0/3075402342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc084004810 0x7fc084004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:18.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc098a60640 1 -- 172.21.15.83:0/3075402342 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc084003530 con 0x7fc084003a60 2024-04-17T10:43:18.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.741+0000 7fc099261640 1 --2- 172.21.15.83:0/3075402342 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc094154670 0x7fc084005280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:43:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.745+0000 7fc098a60640 1 --2- 172.21.15.83:0/3075402342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc084003a60 0x7fc084003e80 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fc088009670 tx=0x7fc08801ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3373419ada1749a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.745+0000 7fc0937fe640 1 -- 172.21.15.83:0/3075402342 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0880b4070 con 0x7fc084003a60 2024-04-17T10:43:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.745+0000 7fc0937fe640 1 -- 172.21.15.83:0/3075402342 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0880a7810 con 0x7fc084003a60 2024-04-17T10:43:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.745+0000 7fc0937fe640 1 -- 172.21.15.83:0/3075402342 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0880096b0 con 0x7fc084003a60 2024-04-17T10:43:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.745+0000 7fc09acc4640 1 -- 172.21.15.83:0/3075402342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc084003a60 msgr2=0x7fc084003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.745+0000 7fc09acc4640 1 --2- 172.21.15.83:0/3075402342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc084003a60 0x7fc084003e80 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fc088009670 tx=0x7fc08801ec30 comp rx=0 tx=0).stop 2024-04-17T10:43:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.745+0000 7fc099261640 1 -- 172.21.15.83:0/3075402342 reap_dead start 2024-04-17T10:43:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.745+0000 7fc09acc4640 1 -- 172.21.15.83:0/3075402342 shutdown_connections 2024-04-17T10:43:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.745+0000 7fc09acc4640 1 -- 172.21.15.83:0/3075402342 >> 172.21.15.83:0/3075402342 conn(0x7fc0940b15b0 msgr2=0x7fc0940b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.745+0000 7fc09acc4640 1 -- 172.21.15.83:0/3075402342 shutdown_connections 2024-04-17T10:43:18.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.745+0000 7fc09acc4640 1 -- 172.21.15.83:0/3075402342 wait complete. 2024-04-17T10:43:18.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.745+0000 7fc09acc4640 1 Processor -- start 2024-04-17T10:43:18.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.745+0000 7fc09acc4640 1 -- start start 2024-04-17T10:43:18.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.745+0000 7fc09acc4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc084003a60 0x7fc0941427a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:18.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.745+0000 7fc09acc4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0940ad840 0x7fc0940adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:18.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.745+0000 7fc09acc4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc094145160 0x7fc094145610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:18.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.749+0000 7fc09acc4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0940aebd0 con 0x7fc0940ad840 2024-04-17T10:43:18.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.749+0000 7fc09acc4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0940aea50 con 0x7fc084003a60 2024-04-17T10:43:18.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.749+0000 7fc09acc4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0940aed50 con 0x7fc094145160 2024-04-17T10:43:18.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.749+0000 7fc093fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0940ad840 0x7fc0940adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:18.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.749+0000 7fc093fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0940ad840 0x7fc0940adcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51756/0 (socket says 172.21.15.83:51756) 2024-04-17T10:43:18.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.749+0000 7fc093fff640 1 -- 172.21.15.83:0/4188058151 learned_addr learned my addr 172.21.15.83:0/4188058151 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:18.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.749+0000 7fc098a60640 1 --2- 172.21.15.83:0/4188058151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc084003a60 0x7fc0941427a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:18.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.749+0000 7fc099261640 1 --2- 172.21.15.83:0/4188058151 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc094145160 0x7fc094145610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:18.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.749+0000 7fc093fff640 1 -- 172.21.15.83:0/4188058151 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc094145160 msgr2=0x7fc094145610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:18.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.749+0000 7fc093fff640 1 --2- 172.21.15.83:0/4188058151 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc094145160 0x7fc094145610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:18.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.749+0000 7fc093fff640 1 -- 172.21.15.83:0/4188058151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc084003a60 msgr2=0x7fc0941427a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:18.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.749+0000 7fc093fff640 1 --2- 172.21.15.83:0/4188058151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc084003a60 0x7fc0941427a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:18.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.749+0000 7fc093fff640 1 -- 172.21.15.83:0/4188058151 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc094146090 con 0x7fc0940ad840 2024-04-17T10:43:18.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.749+0000 7fc098a60640 1 --2- 172.21.15.83:0/4188058151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc084003a60 0x7fc0941427a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:18.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.749+0000 7fc093fff640 1 --2- 172.21.15.83:0/4188058151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0940ad840 0x7fc0940adcc0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fc08000f850 tx=0x7fc08000fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:18.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.749+0000 7fc0917fa640 1 -- 172.21.15.83:0/4188058151 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc080001cf0 con 0x7fc0940ad840 2024-04-17T10:43:18.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.749+0000 7fc0917fa640 1 -- 172.21.15.83:0/4188058151 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc080018070 con 0x7fc0940ad840 2024-04-17T10:43:18.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.749+0000 7fc09acc4640 1 -- 172.21.15.83:0/4188058151 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc094145db0 con 0x7fc0940ad840 2024-04-17T10:43:18.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.749+0000 7fc0917fa640 1 -- 172.21.15.83:0/4188058151 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08000d6d0 con 0x7fc0940ad840 2024-04-17T10:43:18.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.749+0000 7fc09acc4640 1 -- 172.21.15.83:0/4188058151 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0941434e0 con 0x7fc0940ad840 2024-04-17T10:43:18.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.749+0000 7fc09acc4640 1 -- 172.21.15.83:0/4188058151 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0940ae200 con 0x7fc0940ad840 2024-04-17T10:43:18.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.749+0000 7fc0917fa640 1 -- 172.21.15.83:0/4188058151 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc08000d870 con 0x7fc0940ad840 2024-04-17T10:43:18.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.749+0000 7fc0917fa640 1 --2- 172.21.15.83:0/4188058151 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc078066f60 0x7fc078069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:18.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.749+0000 7fc0917fa640 1 -- 172.21.15.83:0/4188058151 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc080084ea0 con 0x7fc0940ad840 2024-04-17T10:43:18.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.749+0000 7fc098a60640 1 --2- 172.21.15.83:0/4188058151 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc078066f60 0x7fc078069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:18.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.749+0000 7fc098a60640 1 --2- 172.21.15.83:0/4188058151 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc078066f60 0x7fc078069420 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc0880b1000 tx=0x7fc0880046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:18.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.753+0000 7fc0917fa640 1 -- 172.21.15.83:0/4188058151 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc080055e40 con 0x7fc0940ad840 2024-04-17T10:43:18.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.905+0000 7fc09acc4640 1 -- 172.21.15.83:0/4188058151 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc094074f90 con 0x7fc0940ad840 2024-04-17T10:43:18.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.905+0000 7fc0917fa640 1 -- 172.21.15.83:0/4188058151 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v63) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc080055740 con 0x7fc0940ad840 2024-04-17T10:43:18.903 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:43:18.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.909+0000 7fc09acc4640 1 -- 172.21.15.83:0/4188058151 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc078066f60 msgr2=0x7fc078069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:18.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.909+0000 7fc09acc4640 1 --2- 172.21.15.83:0/4188058151 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc078066f60 0x7fc078069420 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc0880b1000 tx=0x7fc0880046d0 comp rx=0 tx=0).stop 2024-04-17T10:43:18.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.909+0000 7fc09acc4640 1 -- 172.21.15.83:0/4188058151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0940ad840 msgr2=0x7fc0940adcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:18.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.909+0000 7fc09acc4640 1 --2- 172.21.15.83:0/4188058151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0940ad840 0x7fc0940adcc0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fc08000f850 tx=0x7fc08000fd10 comp rx=0 tx=0).stop 2024-04-17T10:43:18.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.909+0000 7fc09acc4640 1 -- 172.21.15.83:0/4188058151 shutdown_connections 2024-04-17T10:43:18.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.909+0000 7fc09acc4640 1 --2- 172.21.15.83:0/4188058151 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc078066f60 0x7fc078069420 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:18.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.909+0000 7fc09acc4640 1 --2- 172.21.15.83:0/4188058151 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc094145160 0x7fc094145610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:18.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.909+0000 7fc09acc4640 1 --2- 172.21.15.83:0/4188058151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0940ad840 0x7fc0940adcc0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:18.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.909+0000 7fc09acc4640 1 --2- 172.21.15.83:0/4188058151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc084003a60 0x7fc0941427a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:18.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.909+0000 7fc09acc4640 1 -- 172.21.15.83:0/4188058151 >> 172.21.15.83:0/4188058151 conn(0x7fc0940b15b0 msgr2=0x7fc0940cebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:18.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.909+0000 7fc09acc4640 1 -- 172.21.15.83:0/4188058151 shutdown_connections 2024-04-17T10:43:18.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:18.909+0000 7fc09acc4640 1 -- 172.21.15.83:0/4188058151 wait complete. 2024-04-17T10:43:18.920 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:43:19.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.033+0000 7f336cd80640 1 Processor -- start 2024-04-17T10:43:19.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.033+0000 7f336cd80640 1 -- start start 2024-04-17T10:43:19.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.037+0000 7f336cd80640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f336806b520 0x7f3368162400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:19.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.037+0000 7f336cd80640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f336806c1f0 con 0x7f3368162020 2024-04-17T10:43:19.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.037+0000 7f336cd80640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f336806c490 con 0x7f33680b3440 2024-04-17T10:43:19.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.037+0000 7f336cd80640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33680b29f0 con 0x7f336806b520 2024-04-17T10:43:19.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.037+0000 7f336659c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3368162020 0x7f33680b45d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52214/0 (socket says 172.21.15.83:52214) 2024-04-17T10:43:19.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.037+0000 7f3366d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f336806b520 0x7f3368162400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:19.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.037+0000 7f336659c640 1 -- 172.21.15.83:0/3352889225 learned_addr learned my addr 172.21.15.83:0/3352889225 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:43:19.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.037+0000 7f3366d9d640 1 -- 172.21.15.83:0/3352889225 >> v1:172.21.15.167:6789/0 conn(0x7f33680b3440 legacy=0x7f336806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:19.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.037+0000 7f3366d9d640 1 -- 172.21.15.83:0/3352889225 >> v1:172.21.15.83:6789/0 conn(0x7f3368162020 legacy=0x7f33680b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:19.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.037+0000 7f3366d9d640 1 -- 172.21.15.83:0/3352889225 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f336806c770 con 0x7f336806b520 2024-04-17T10:43:19.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.037+0000 7f3366d9d640 1 --2- 172.21.15.83:0/3352889225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f336806b520 0x7f3368162400 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f3360009d90 tx=0x7f336001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a17dcaee0d40610 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:19.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.037+0000 7f3365d9b640 1 -- 172.21.15.83:0/3352889225 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33600b2070 con 0x7f336806b520 2024-04-17T10:43:19.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.037+0000 7f3365d9b640 1 -- 172.21.15.83:0/3352889225 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33600a5900 con 0x7f336806b520 2024-04-17T10:43:19.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.037+0000 7f3365d9b640 1 -- 172.21.15.83:0/3352889225 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33600a5cd0 con 0x7f336806b520 2024-04-17T10:43:19.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.037+0000 7f336cd80640 1 -- 172.21.15.83:0/3352889225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f336806b520 msgr2=0x7f3368162400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:19.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.037+0000 7f336cd80640 1 --2- 172.21.15.83:0/3352889225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f336806b520 0x7f3368162400 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f3360009d90 tx=0x7f336001c930 comp rx=0 tx=0).stop 2024-04-17T10:43:19.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.037+0000 7f336cd80640 1 -- 172.21.15.83:0/3352889225 shutdown_connections 2024-04-17T10:43:19.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.037+0000 7f336cd80640 1 --2- 172.21.15.83:0/3352889225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f336806b520 0x7f3368162400 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:19.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.037+0000 7f336cd80640 1 -- 172.21.15.83:0/3352889225 >> 172.21.15.83:0/3352889225 conn(0x7f33680b15b0 msgr2=0x7f33680b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:19.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.037+0000 7f336cd80640 1 -- 172.21.15.83:0/3352889225 shutdown_connections 2024-04-17T10:43:19.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.037+0000 7f336cd80640 1 -- 172.21.15.83:0/3352889225 wait complete. 2024-04-17T10:43:19.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.041+0000 7f336cd80640 1 Processor -- start 2024-04-17T10:43:19.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.041+0000 7f336cd80640 1 -- start start 2024-04-17T10:43:19.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.041+0000 7f336cd80640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f336806b520 0x7f336815e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:19.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.041+0000 7f336cd80640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33680b3440 0x7f3368151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:19.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.041+0000 7f336cd80640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3368162020 0x7f33680ac720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:19.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.041+0000 7f336cd80640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3368170400 con 0x7f3368162020 2024-04-17T10:43:19.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.041+0000 7f336cd80640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3368170280 con 0x7f336806b520 2024-04-17T10:43:19.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.041+0000 7f336cd80640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3368170580 con 0x7f33680b3440 2024-04-17T10:43:19.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.041+0000 7f3366d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3368162020 0x7f33680ac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:19.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.041+0000 7f3366d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3368162020 0x7f33680ac720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51774/0 (socket says 172.21.15.83:51774) 2024-04-17T10:43:19.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.041+0000 7f3366d9d640 1 -- 172.21.15.83:0/1351131462 learned_addr learned my addr 172.21.15.83:0/1351131462 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:19.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.041+0000 7f336659c640 1 --2- 172.21.15.83:0/1351131462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f336806b520 0x7f336815e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:19.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.041+0000 7f335dd9b640 1 --2- 172.21.15.83:0/1351131462 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33680b3440 0x7f3368151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:19.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.041+0000 7f3366d9d640 1 -- 172.21.15.83:0/1351131462 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33680b3440 msgr2=0x7f3368151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:19.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.041+0000 7f3366d9d640 1 --2- 172.21.15.83:0/1351131462 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33680b3440 0x7f3368151010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:19.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.041+0000 7f3366d9d640 1 -- 172.21.15.83:0/1351131462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f336806b520 msgr2=0x7f336815e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:19.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.041+0000 7f3366d9d640 1 --2- 172.21.15.83:0/1351131462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f336806b520 0x7f336815e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:19.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.041+0000 7f3366d9d640 1 -- 172.21.15.83:0/1351131462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33680aacd0 con 0x7f3368162020 2024-04-17T10:43:19.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.041+0000 7f335dd9b640 1 --2- 172.21.15.83:0/1351131462 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33680b3440 0x7f3368151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:19.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.041+0000 7f3366d9d640 1 --2- 172.21.15.83:0/1351131462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3368162020 0x7f33680ac720 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f336001a560 tx=0x7f33600196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:19.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.041+0000 7f335f7fe640 1 -- 172.21.15.83:0/1351131462 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33600b2050 con 0x7f3368162020 2024-04-17T10:43:19.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.041+0000 7f335f7fe640 1 -- 172.21.15.83:0/1351131462 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33600be070 con 0x7f3368162020 2024-04-17T10:43:19.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.041+0000 7f336cd80640 1 -- 172.21.15.83:0/1351131462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33680aafb0 con 0x7f3368162020 2024-04-17T10:43:19.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.041+0000 7f335f7fe640 1 -- 172.21.15.83:0/1351131462 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33600adad0 con 0x7f3368162020 2024-04-17T10:43:19.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.041+0000 7f336cd80640 1 -- 172.21.15.83:0/1351131462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33680ab570 con 0x7f3368162020 2024-04-17T10:43:19.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.041+0000 7f336cd80640 1 -- 172.21.15.83:0/1351131462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3368074f90 con 0x7f3368162020 2024-04-17T10:43:19.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.045+0000 7f335f7fe640 1 -- 172.21.15.83:0/1351131462 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f33600adc70 con 0x7f3368162020 2024-04-17T10:43:19.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.045+0000 7f335f7fe640 1 --2- 172.21.15.83:0/1351131462 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f334c066f60 0x7f334c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:19.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.045+0000 7f335f7fe640 1 -- 172.21.15.83:0/1351131462 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3360120360 con 0x7f3368162020 2024-04-17T10:43:19.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.045+0000 7f336659c640 1 --2- 172.21.15.83:0/1351131462 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f334c066f60 0x7f334c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:19.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.045+0000 7f336659c640 1 --2- 172.21.15.83:0/1351131462 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f334c066f60 0x7f334c069420 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3358000f70 tx=0x7f3358009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:19.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.049+0000 7f335f7fe640 1 -- 172.21.15.83:0/1351131462 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3360016520 con 0x7f3368162020 2024-04-17T10:43:19.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.201+0000 7f336cd80640 1 -- 172.21.15.83:0/1351131462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3368165620 con 0x7f3368162020 2024-04-17T10:43:19.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.201+0000 7f335f7fe640 1 -- 172.21.15.83:0/1351131462 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v63) v1 ==== 100+0+11 (secure 0 0 0) 0x7f33600ab340 con 0x7f3368162020 2024-04-17T10:43:19.196 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:43:19.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.201+0000 7f336cd80640 1 -- 172.21.15.83:0/1351131462 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f334c066f60 msgr2=0x7f334c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:19.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.201+0000 7f336cd80640 1 --2- 172.21.15.83:0/1351131462 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f334c066f60 0x7f334c069420 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3358000f70 tx=0x7f3358009040 comp rx=0 tx=0).stop 2024-04-17T10:43:19.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.201+0000 7f336cd80640 1 -- 172.21.15.83:0/1351131462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3368162020 msgr2=0x7f33680ac720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:19.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.201+0000 7f336cd80640 1 --2- 172.21.15.83:0/1351131462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3368162020 0x7f33680ac720 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f336001a560 tx=0x7f33600196d0 comp rx=0 tx=0).stop 2024-04-17T10:43:19.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.201+0000 7f336cd80640 1 -- 172.21.15.83:0/1351131462 shutdown_connections 2024-04-17T10:43:19.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.201+0000 7f336cd80640 1 --2- 172.21.15.83:0/1351131462 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f334c066f60 0x7f334c069420 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:19.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.201+0000 7f336cd80640 1 --2- 172.21.15.83:0/1351131462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3368162020 0x7f33680ac720 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:19.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.201+0000 7f336cd80640 1 --2- 172.21.15.83:0/1351131462 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33680b3440 0x7f3368151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:19.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.201+0000 7f336cd80640 1 --2- 172.21.15.83:0/1351131462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f336806b520 0x7f336815e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:19.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.201+0000 7f336cd80640 1 -- 172.21.15.83:0/1351131462 >> 172.21.15.83:0/1351131462 conn(0x7f33680b15b0 msgr2=0x7f33681607f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:19.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.201+0000 7f336cd80640 1 -- 172.21.15.83:0/1351131462 shutdown_connections 2024-04-17T10:43:19.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.201+0000 7f336cd80640 1 -- 172.21.15.83:0/1351131462 wait complete. 2024-04-17T10:43:19.214 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:43:19.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.333+0000 7f3742480640 1 Processor -- start 2024-04-17T10:43:19.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.333+0000 7f3742480640 1 -- start start 2024-04-17T10:43:19.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.333+0000 7f3742480640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f373c15b6e0 0x7f373c071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:19.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.333+0000 7f3742480640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f373c15bac0 0x7f373c0cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:19.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.333+0000 7f3742480640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f373c070470 0x7f373c06ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:19.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.333+0000 7f3742480640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f373c06c1f0 con 0x7f373c15b6e0 2024-04-17T10:43:19.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.333+0000 7f3742480640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f373c0cdd30 con 0x7f373c070470 2024-04-17T10:43:19.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.333+0000 7f3742480640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f373c070a10 con 0x7f373c15bac0 2024-04-17T10:43:19.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.333+0000 7f373bfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f373c15b6e0 0x7f373c071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:19.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.333+0000 7f373bfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f373c15b6e0 0x7f373c071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51776/0 (socket says 172.21.15.83:51776) 2024-04-17T10:43:19.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.333+0000 7f373bfff640 1 -- 172.21.15.83:0/471958202 learned_addr learned my addr 172.21.15.83:0/471958202 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:19.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.333+0000 7f373b7fe640 1 --2- 172.21.15.83:0/471958202 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f373c15bac0 0x7f373c0cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:19.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.333+0000 7f3740a1d640 1 --2- 172.21.15.83:0/471958202 >> v2:172.21.15.167:3300/0 conn(0x7f373c070470 0x7f373c06ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:19.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.333+0000 7f373bfff640 1 -- 172.21.15.83:0/471958202 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f373c15bac0 msgr2=0x7f373c0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:19.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.333+0000 7f373bfff640 1 --2- 172.21.15.83:0/471958202 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f373c15bac0 0x7f373c0cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:19.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.333+0000 7f373bfff640 1 -- 172.21.15.83:0/471958202 >> v2:172.21.15.167:3300/0 conn(0x7f373c070470 msgr2=0x7f373c06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:19.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.333+0000 7f373bfff640 1 --2- 172.21.15.83:0/471958202 >> v2:172.21.15.167:3300/0 conn(0x7f373c070470 0x7f373c06ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:19.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.333+0000 7f373bfff640 1 -- 172.21.15.83:0/471958202 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f373c070cf0 con 0x7f373c15b6e0 2024-04-17T10:43:19.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.333+0000 7f373b7fe640 1 --2- 172.21.15.83:0/471958202 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f373c15bac0 0x7f373c0cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:19.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.337+0000 7f373bfff640 1 --2- 172.21.15.83:0/471958202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f373c15b6e0 0x7f373c071cc0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f373c0cd050 tx=0x7f372c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44194a78a0bf52 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:19.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.337+0000 7f373affd640 1 -- 172.21.15.83:0/471958202 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c0b0070 con 0x7f373c15b6e0 2024-04-17T10:43:19.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.337+0000 7f373affd640 1 -- 172.21.15.83:0/471958202 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372c0ab600 con 0x7f373c15b6e0 2024-04-17T10:43:19.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.337+0000 7f373affd640 1 -- 172.21.15.83:0/471958202 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c0ab9d0 con 0x7f373c15b6e0 2024-04-17T10:43:19.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.337+0000 7f3742480640 1 -- 172.21.15.83:0/471958202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f373c15b6e0 msgr2=0x7f373c071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:19.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.337+0000 7f3742480640 1 --2- 172.21.15.83:0/471958202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f373c15b6e0 0x7f373c071cc0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f373c0cd050 tx=0x7f372c01c520 comp rx=0 tx=0).stop 2024-04-17T10:43:19.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.337+0000 7f3742480640 1 -- 172.21.15.83:0/471958202 shutdown_connections 2024-04-17T10:43:19.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.337+0000 7f3742480640 1 --2- 172.21.15.83:0/471958202 >> v2:172.21.15.167:3300/0 conn(0x7f373c070470 0x7f373c06ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:19.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.337+0000 7f3742480640 1 --2- 172.21.15.83:0/471958202 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f373c15bac0 0x7f373c0cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:19.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.337+0000 7f3742480640 1 --2- 172.21.15.83:0/471958202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f373c15b6e0 0x7f373c071cc0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:19.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.337+0000 7f3742480640 1 -- 172.21.15.83:0/471958202 >> 172.21.15.83:0/471958202 conn(0x7f373c0b15b0 msgr2=0x7f373c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:19.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.337+0000 7f3742480640 1 -- 172.21.15.83:0/471958202 shutdown_connections 2024-04-17T10:43:19.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.337+0000 7f3742480640 1 -- 172.21.15.83:0/471958202 wait complete. 2024-04-17T10:43:19.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.337+0000 7f3742480640 1 Processor -- start 2024-04-17T10:43:19.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.337+0000 7f3742480640 1 -- start start 2024-04-17T10:43:19.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.341+0000 7f3742480640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f373c070470 0x7f373c153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:19.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.341+0000 7f3742480640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f373c15b6e0 0x7f373c154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:19.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.341+0000 7f3742480640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f373c15bac0 0x7f373c15d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:19.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.341+0000 7f3742480640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373c06d270 con 0x7f373c15bac0 2024-04-17T10:43:19.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.341+0000 7f3742480640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373c06d0f0 con 0x7f373c15b6e0 2024-04-17T10:43:19.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.341+0000 7f3742480640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f373c06b760 con 0x7f373c070470 2024-04-17T10:43:19.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.341+0000 7f3740a1d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f373c15bac0 0x7f373c15d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:19.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.341+0000 7f3740a1d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f373c15bac0 0x7f373c15d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51782/0 (socket says 172.21.15.83:51782) 2024-04-17T10:43:19.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.341+0000 7f3740a1d640 1 -- 172.21.15.83:0/1564968743 learned_addr learned my addr 172.21.15.83:0/1564968743 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:19.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.341+0000 7f373b7fe640 1 --2- 172.21.15.83:0/1564968743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f373c15b6e0 0x7f373c154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:19.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.341+0000 7f373bfff640 1 --2- 172.21.15.83:0/1564968743 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f373c070470 0x7f373c153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:19.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.341+0000 7f3740a1d640 1 -- 172.21.15.83:0/1564968743 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f373c070470 msgr2=0x7f373c153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:19.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.341+0000 7f3740a1d640 1 --2- 172.21.15.83:0/1564968743 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f373c070470 0x7f373c153640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:19.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.341+0000 7f3740a1d640 1 -- 172.21.15.83:0/1564968743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f373c15b6e0 msgr2=0x7f373c154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:19.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.341+0000 7f3740a1d640 1 --2- 172.21.15.83:0/1564968743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f373c15b6e0 0x7f373c154bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:19.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.341+0000 7f3740a1d640 1 -- 172.21.15.83:0/1564968743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f373c15d970 con 0x7f373c15bac0 2024-04-17T10:43:19.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.341+0000 7f373b7fe640 1 --2- 172.21.15.83:0/1564968743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f373c15b6e0 0x7f373c154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:19.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.341+0000 7f3740a1d640 1 --2- 172.21.15.83:0/1564968743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f373c15bac0 0x7f373c15d430 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f373400cc30 tx=0x7f373400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:19.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.341+0000 7f3738ff9640 1 -- 172.21.15.83:0/1564968743 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373400ce40 con 0x7f373c15bac0 2024-04-17T10:43:19.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.341+0000 7f3738ff9640 1 -- 172.21.15.83:0/1564968743 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37340039a0 con 0x7f373c15bac0 2024-04-17T10:43:19.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.341+0000 7f3738ff9640 1 -- 172.21.15.83:0/1564968743 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3734003d50 con 0x7f373c15bac0 2024-04-17T10:43:19.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.341+0000 7f3742480640 1 -- 172.21.15.83:0/1564968743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f373c15cfd0 con 0x7f373c15bac0 2024-04-17T10:43:19.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.341+0000 7f3742480640 1 -- 172.21.15.83:0/1564968743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f373c1540d0 con 0x7f373c15bac0 2024-04-17T10:43:19.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.341+0000 7f3742480640 1 -- 172.21.15.83:0/1564968743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f373c074f90 con 0x7f373c15bac0 2024-04-17T10:43:19.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.341+0000 7f3738ff9640 1 -- 172.21.15.83:0/1564968743 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3734003f70 con 0x7f373c15bac0 2024-04-17T10:43:19.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.341+0000 7f3738ff9640 1 --2- 172.21.15.83:0/1564968743 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3728066f60 0x7f3728069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:19.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.341+0000 7f3738ff9640 1 -- 172.21.15.83:0/1564968743 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3734085530 con 0x7f373c15bac0 2024-04-17T10:43:19.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.341+0000 7f373bfff640 1 --2- 172.21.15.83:0/1564968743 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3728066f60 0x7f3728069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:19.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.341+0000 7f373bfff640 1 --2- 172.21.15.83:0/1564968743 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3728066f60 0x7f3728069420 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f372c0ad000 tx=0x7f372c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:19.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.345+0000 7f3738ff9640 1 -- 172.21.15.83:0/1564968743 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3734014030 con 0x7f373c15bac0 2024-04-17T10:43:19.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.509+0000 7f3742480640 1 -- 172.21.15.83:0/1564968743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f373c06e390 con 0x7f373c15bac0 2024-04-17T10:43:19.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.509+0000 7f3738ff9640 1 -- 172.21.15.83:0/1564968743 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v63) v1 ==== 80+0+122 (secure 0 0 0) 0x7f3734083e20 con 0x7f373c15bac0 2024-04-17T10:43:19.506 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.167:6833/271994303 2024-04-18T10:42:59.346779+0000 2024-04-17T10:43:19.506 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.167:6832/271994303 2024-04-18T10:42:59.346779+0000 2024-04-17T10:43:19.506 INFO:teuthology.orchestra.run.smithi083.stderr:listed 2 entries 2024-04-17T10:43:19.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.513+0000 7f3742480640 1 -- 172.21.15.83:0/1564968743 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3728066f60 msgr2=0x7f3728069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:19.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.513+0000 7f3742480640 1 --2- 172.21.15.83:0/1564968743 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3728066f60 0x7f3728069420 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f372c0ad000 tx=0x7f372c008040 comp rx=0 tx=0).stop 2024-04-17T10:43:19.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.513+0000 7f3742480640 1 -- 172.21.15.83:0/1564968743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f373c15bac0 msgr2=0x7f373c15d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:19.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.513+0000 7f3742480640 1 --2- 172.21.15.83:0/1564968743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f373c15bac0 0x7f373c15d430 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f373400cc30 tx=0x7f373400a5c0 comp rx=0 tx=0).stop 2024-04-17T10:43:19.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.513+0000 7f3742480640 1 -- 172.21.15.83:0/1564968743 shutdown_connections 2024-04-17T10:43:19.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.513+0000 7f3742480640 1 --2- 172.21.15.83:0/1564968743 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3728066f60 0x7f3728069420 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:19.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.513+0000 7f3742480640 1 --2- 172.21.15.83:0/1564968743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f373c15bac0 0x7f373c15d430 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:19.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.513+0000 7f3742480640 1 --2- 172.21.15.83:0/1564968743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f373c15b6e0 0x7f373c154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:19.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.513+0000 7f3742480640 1 --2- 172.21.15.83:0/1564968743 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f373c070470 0x7f373c153640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:19.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.513+0000 7f3742480640 1 -- 172.21.15.83:0/1564968743 >> 172.21.15.83:0/1564968743 conn(0x7f373c0b15b0 msgr2=0x7f373c0b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:19.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.513+0000 7f3742480640 1 -- 172.21.15.83:0/1564968743 shutdown_connections 2024-04-17T10:43:19.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:19.513+0000 7f3742480640 1 -- 172.21.15.83:0/1564968743 wait complete. 2024-04-17T10:43:19.523 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2024-04-17T10:43:19.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:43:19.523 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-17T10:43:19.534 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[55284]: fuse finished with error 0 and tester_r 0 2024-04-17T10:43:19.535 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-17T10:43:19.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:43:19.536 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T10:43:19.702 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:43:19.702 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:43:19.709 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:43:19.709 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-04-17T10:43:19.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:43:19.710 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-17T10:43:19.759 INFO:teuthology.orchestra.run.smithi167.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-17T10:43:19.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:43:19.760 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:43:19.760 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:43:19.766 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16063 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:43:19.767 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-17T10:43:19.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:43:19.767 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-17T10:43:19.816 INFO:teuthology.orchestra.run.smithi083.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-17T10:43:19.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:43:19.817 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:43:19.818 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:43:19.826 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:43:19.826 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-04-17T10:43:19.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:43:19.826 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-17T10:43:19.874 INFO:teuthology.orchestra.run.smithi167.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-17T10:43:19.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:43:19.875 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:43:20.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.009+0000 7f29d3b0e640 1 Processor -- start 2024-04-17T10:43:20.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.009+0000 7f29d3b0e640 1 -- start start 2024-04-17T10:43:20.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.009+0000 7f29d3b0e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29cc0b5b40 0x7f29cc0d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:20.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.009+0000 7f29d3b0e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29cc162a00 con 0x7f29cc15dfa0 2024-04-17T10:43:20.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.009+0000 7f29d3b0e640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29cc161e80 con 0x7f29cc0ce980 2024-04-17T10:43:20.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.009+0000 7f29d3b0e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc162160 con 0x7f29cc0b5b40 2024-04-17T10:43:20.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.009+0000 7f29d18aa640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f29cc15dfa0 0x7f29cc0b6c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52226/0 (socket says 172.21.15.83:52226) 2024-04-17T10:43:20.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.009+0000 7f29d18aa640 1 -- 172.21.15.83:0/2853470552 learned_addr learned my addr 172.21.15.83:0/2853470552 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:43:20.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.009+0000 7f29d20ab640 1 --2- 172.21.15.83:0/2853470552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29cc0b5b40 0x7f29cc0d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:20.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.009+0000 7f29d08a8640 1 -- 172.21.15.83:0/2853470552 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3554377207 0 0) 0x7f29cc162a00 con 0x7f29cc15dfa0 2024-04-17T10:43:20.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.009+0000 7f29d08a8640 1 -- 172.21.15.83:0/2853470552 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29b8003530 con 0x7f29cc15dfa0 2024-04-17T10:43:20.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.009+0000 7f29d08a8640 1 -- 172.21.15.83:0/2853470552 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3211064221 0 0) 0x7f29b8003530 con 0x7f29cc15dfa0 2024-04-17T10:43:20.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.009+0000 7f29d08a8640 1 -- 172.21.15.83:0/2853470552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29cc0b5b40 msgr2=0x7f29cc0d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:20.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.009+0000 7f29d08a8640 1 --2- 172.21.15.83:0/2853470552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29cc0b5b40 0x7f29cc0d1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.009+0000 7f29d08a8640 1 -- 172.21.15.83:0/2853470552 >> v1:172.21.15.167:6789/0 conn(0x7f29cc0ce980 legacy=0x7f29cc0b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:20.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.009+0000 7f29d08a8640 1 -- 172.21.15.83:0/2853470552 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29cc162440 con 0x7f29cc15dfa0 2024-04-17T10:43:20.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.009+0000 7f29d08a8640 1 -- 172.21.15.83:0/2853470552 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f29c8001680 con 0x7f29cc15dfa0 2024-04-17T10:43:20.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.009+0000 7f29d08a8640 1 -- 172.21.15.83:0/2853470552 >> v1:172.21.15.83:6789/0 conn(0x7f29cc15dfa0 legacy=0x7f29cc0b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:20.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.009+0000 7f29d08a8640 1 --2- 172.21.15.83:0/2853470552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29cc0b5b40 0x7f29b80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:20.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.009+0000 7f29d08a8640 1 --2- 172.21.15.83:0/2853470552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29b8003f70 0x7f29b8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:20.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.009+0000 7f29d08a8640 1 --2- 172.21.15.83:0/2853470552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29b8004d50 0x7f29b80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:20.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.009+0000 7f29d08a8640 1 -- 172.21.15.83:0/2853470552 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc162160 con 0x7f29b8004d50 2024-04-17T10:43:20.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.009+0000 7f29d08a8640 1 -- 172.21.15.83:0/2853470552 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b800bb60 con 0x7f29b8003f70 2024-04-17T10:43:20.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.009+0000 7f29d08a8640 1 -- 172.21.15.83:0/2853470552 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b800bd60 con 0x7f29cc0b5b40 2024-04-17T10:43:20.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.009+0000 7f29d20ab640 1 --2- 172.21.15.83:0/2853470552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29b8004d50 0x7f29b80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:20.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.009+0000 7f29d10a9640 1 --2- 172.21.15.83:0/2853470552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29b8003f70 0x7f29b8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:20.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.009+0000 7f29d18aa640 1 --2- 172.21.15.83:0/2853470552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29cc0b5b40 0x7f29b80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:20.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.009+0000 7f29d20ab640 1 -- 172.21.15.83:0/2853470552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29cc0b5b40 msgr2=0x7f29b80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:20.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.009+0000 7f29d20ab640 1 --2- 172.21.15.83:0/2853470552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29cc0b5b40 0x7f29b80039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.009+0000 7f29d20ab640 1 -- 172.21.15.83:0/2853470552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29b8003f70 msgr2=0x7f29b8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:20.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.009+0000 7f29d20ab640 1 --2- 172.21.15.83:0/2853470552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29b8003f70 0x7f29b8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.009+0000 7f29d20ab640 1 -- 172.21.15.83:0/2853470552 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b8003530 con 0x7f29b8004d50 2024-04-17T10:43:20.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.013+0000 7f29d20ab640 1 --2- 172.21.15.83:0/2853470552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29b8004d50 0x7f29b80051d0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f29c4009f90 tx=0x7f29c401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cab930715d7c2eb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:20.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.013+0000 7f29d08a8640 1 -- 172.21.15.83:0/2853470552 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c40b2070 con 0x7f29b8004d50 2024-04-17T10:43:20.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.013+0000 7f29d08a8640 1 -- 172.21.15.83:0/2853470552 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c40a5b40 con 0x7f29b8004d50 2024-04-17T10:43:20.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.013+0000 7f29d08a8640 1 -- 172.21.15.83:0/2853470552 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c4009c10 con 0x7f29b8004d50 2024-04-17T10:43:20.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.013+0000 7f29d3b0e640 1 -- 172.21.15.83:0/2853470552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29b8004d50 msgr2=0x7f29b80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:20.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.013+0000 7f29d3b0e640 1 --2- 172.21.15.83:0/2853470552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29b8004d50 0x7f29b80051d0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f29c4009f90 tx=0x7f29c401cc40 comp rx=0 tx=0).stop 2024-04-17T10:43:20.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.013+0000 7f29d20ab640 1 -- 172.21.15.83:0/2853470552 reap_dead start 2024-04-17T10:43:20.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.013+0000 7f29d3b0e640 1 -- 172.21.15.83:0/2853470552 shutdown_connections 2024-04-17T10:43:20.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.013+0000 7f29d3b0e640 1 -- 172.21.15.83:0/2853470552 >> 172.21.15.83:0/2853470552 conn(0x7f29cc0d0600 msgr2=0x7f29cc0d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:20.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.013+0000 7f29d3b0e640 1 -- 172.21.15.83:0/2853470552 shutdown_connections 2024-04-17T10:43:20.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.013+0000 7f29d3b0e640 1 -- 172.21.15.83:0/2853470552 wait complete. 2024-04-17T10:43:20.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.013+0000 7f29d3b0e640 1 Processor -- start 2024-04-17T10:43:20.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.013+0000 7f29d3b0e640 1 -- start start 2024-04-17T10:43:20.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.013+0000 7f29d3b0e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29b8004d50 0x7f29cc0cc250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:20.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.013+0000 7f29d3b0e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29cc0c9f70 0x7f29cc0ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:20.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.013+0000 7f29d3b0e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29cc160270 0x7f29cc1606f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:20.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.013+0000 7f29d3b0e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc161b10 con 0x7f29cc0c9f70 2024-04-17T10:43:20.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.013+0000 7f29d3b0e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc15e380 con 0x7f29cc160270 2024-04-17T10:43:20.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.013+0000 7f29d3b0e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc0cee00 con 0x7f29b8004d50 2024-04-17T10:43:20.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.013+0000 7f29d10a9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29cc0c9f70 0x7f29cc0ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:20.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.017+0000 7f29d10a9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29cc0c9f70 0x7f29cc0ca3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51800/0 (socket says 172.21.15.83:51800) 2024-04-17T10:43:20.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.017+0000 7f29d10a9640 1 -- 172.21.15.83:0/1941515584 learned_addr learned my addr 172.21.15.83:0/1941515584 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:20.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.017+0000 7f29d18aa640 1 --2- 172.21.15.83:0/1941515584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29b8004d50 0x7f29cc0cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:20.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.017+0000 7f29d20ab640 1 --2- 172.21.15.83:0/1941515584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29cc160270 0x7f29cc1606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:20.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.017+0000 7f29d10a9640 1 -- 172.21.15.83:0/1941515584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29b8004d50 msgr2=0x7f29cc0cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:20.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.017+0000 7f29d10a9640 1 --2- 172.21.15.83:0/1941515584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29b8004d50 0x7f29cc0cc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.017+0000 7f29d10a9640 1 -- 172.21.15.83:0/1941515584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29cc160270 msgr2=0x7f29cc1606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:20.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.017+0000 7f29d10a9640 1 --2- 172.21.15.83:0/1941515584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29cc160270 0x7f29cc1606f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.017+0000 7f29d10a9640 1 -- 172.21.15.83:0/1941515584 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29cc0c74c0 con 0x7f29cc0c9f70 2024-04-17T10:43:20.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.017+0000 7f29d18aa640 1 --2- 172.21.15.83:0/1941515584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29b8004d50 0x7f29cc0cc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:43:20.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.017+0000 7f29d10a9640 1 --2- 172.21.15.83:0/1941515584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29cc0c9f70 0x7f29cc0ca3d0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f29bc00e550 tx=0x7f29bc00ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:20.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.017+0000 7f29c27fc640 1 -- 172.21.15.83:0/1941515584 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc0052e0 con 0x7f29cc0c9f70 2024-04-17T10:43:20.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.017+0000 7f29c27fc640 1 -- 172.21.15.83:0/1941515584 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29bc018070 con 0x7f29cc0c9f70 2024-04-17T10:43:20.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.017+0000 7f29c27fc640 1 -- 172.21.15.83:0/1941515584 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc003820 con 0x7f29cc0c9f70 2024-04-17T10:43:20.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.017+0000 7f29d3b0e640 1 -- 172.21.15.83:0/1941515584 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29cc0c77a0 con 0x7f29cc0c9f70 2024-04-17T10:43:20.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.017+0000 7f29d3b0e640 1 -- 172.21.15.83:0/1941515584 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29cc15fa70 con 0x7f29cc0c9f70 2024-04-17T10:43:20.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.017+0000 7f29d3b0e640 1 -- 172.21.15.83:0/1941515584 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29cc0681a0 con 0x7f29cc0c9f70 2024-04-17T10:43:20.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.017+0000 7f29c27fc640 1 -- 172.21.15.83:0/1941515584 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f29bc0039c0 con 0x7f29cc0c9f70 2024-04-17T10:43:20.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.017+0000 7f29c27fc640 1 --2- 172.21.15.83:0/1941515584 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f29b0066f60 0x7f29b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:20.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.017+0000 7f29d18aa640 1 --2- 172.21.15.83:0/1941515584 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f29b0066f60 0x7f29b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:20.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.017+0000 7f29c27fc640 1 -- 172.21.15.83:0/1941515584 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f29bc084cd0 con 0x7f29cc0c9f70 2024-04-17T10:43:20.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.017+0000 7f29d18aa640 1 --2- 172.21.15.83:0/1941515584 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f29b0066f60 0x7f29b0069420 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f29c8001ce0 tx=0x7f29c8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:20.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.021+0000 7f29c27fc640 1 -- 172.21.15.83:0/1941515584 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f29bc055c80 con 0x7f29cc0c9f70 2024-04-17T10:43:20.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.217+0000 7f29d3b0e640 1 -- 172.21.15.83:0/1941515584 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29cc074f90 con 0x7f29cc0c9f70 2024-04-17T10:43:20.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.217+0000 7f29c27fc640 1 -- 172.21.15.83:0/1941515584 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f29bc005480 con 0x7f29cc0c9f70 2024-04-17T10:43:20.214 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:43:20.214 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":39,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5165,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1924992370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1924992370},{"type":"v1","addr":"172.21.15.167:6833","nonce":1924992370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5174,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3876293847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3876293847},{"type":"v1","addr":"172.21.15.167:6835","nonce":3876293847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6831,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/338926944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":338926944},{"type":"v1","addr":"172.21.15.83:6835","nonce":338926944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:42:34.328397+0000","modified":"2024-04-17T10:43:02.389233+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6843},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6843":{"gid":6843,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6837/3985646487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3985646487},{"type":"v1","addr":"172.21.15.83:6837","nonce":3985646487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-17T10:43:20.214 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 39 2024-04-17T10:43:20.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.221+0000 7f29d3b0e640 1 -- 172.21.15.83:0/1941515584 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f29b0066f60 msgr2=0x7f29b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:20.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.221+0000 7f29d3b0e640 1 --2- 172.21.15.83:0/1941515584 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f29b0066f60 0x7f29b0069420 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f29c8001ce0 tx=0x7f29c8009040 comp rx=0 tx=0).stop 2024-04-17T10:43:20.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.221+0000 7f29d3b0e640 1 -- 172.21.15.83:0/1941515584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29cc0c9f70 msgr2=0x7f29cc0ca3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:20.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.221+0000 7f29d3b0e640 1 --2- 172.21.15.83:0/1941515584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29cc0c9f70 0x7f29cc0ca3d0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f29bc00e550 tx=0x7f29bc00ea10 comp rx=0 tx=0).stop 2024-04-17T10:43:20.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.221+0000 7f29d3b0e640 1 -- 172.21.15.83:0/1941515584 shutdown_connections 2024-04-17T10:43:20.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.221+0000 7f29d3b0e640 1 --2- 172.21.15.83:0/1941515584 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f29b0066f60 0x7f29b0069420 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.221+0000 7f29d3b0e640 1 --2- 172.21.15.83:0/1941515584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29cc160270 0x7f29cc1606f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.221+0000 7f29d3b0e640 1 --2- 172.21.15.83:0/1941515584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29cc0c9f70 0x7f29cc0ca3d0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.221+0000 7f29d3b0e640 1 --2- 172.21.15.83:0/1941515584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29b8004d50 0x7f29cc0cc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.221+0000 7f29d3b0e640 1 -- 172.21.15.83:0/1941515584 >> 172.21.15.83:0/1941515584 conn(0x7f29cc0d0600 msgr2=0x7f29cc15c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:20.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.221+0000 7f29d3b0e640 1 -- 172.21.15.83:0/1941515584 shutdown_connections 2024-04-17T10:43:20.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.221+0000 7f29d3b0e640 1 -- 172.21.15.83:0/1941515584 wait complete. 2024-04-17T10:43:20.233 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:43:20.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.361+0000 7f71f9ee1640 1 Processor -- start 2024-04-17T10:43:20.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.361+0000 7f71f9ee1640 1 -- start start 2024-04-17T10:43:20.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.361+0000 7f71f9ee1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f71f40cf060 0x7f71f40b4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:20.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.361+0000 7f71f9ee1640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f71f40b3990 0x7f71f4070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:20.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.361+0000 7f71f9ee1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71f40b45a0 0x7f71f40709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:20.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.361+0000 7f71f9ee1640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f40b2980 con 0x7f71f40cf060 2024-04-17T10:43:20.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.361+0000 7f71f9ee1640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f40cf440 con 0x7f71f40b3990 2024-04-17T10:43:20.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.361+0000 7f71f9ee1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f40b2bd0 con 0x7f71f40b45a0 2024-04-17T10:43:20.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.361+0000 7f71f37fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f71f40cf060 0x7f71f40b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:20.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.361+0000 7f71f37fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f71f40cf060 0x7f71f40b4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51808/0 (socket says 172.21.15.83:51808) 2024-04-17T10:43:20.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.361+0000 7f71f37fe640 1 -- 172.21.15.83:0/3485045732 learned_addr learned my addr 172.21.15.83:0/3485045732 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:20.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.361+0000 7f71ebfff640 1 --2- 172.21.15.83:0/3485045732 >> v2:172.21.15.167:3300/0 conn(0x7f71f40b3990 0x7f71f4070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:20.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.361+0000 7f71f3fff640 1 --2- 172.21.15.83:0/3485045732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71f40b45a0 0x7f71f40709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:20.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.361+0000 7f71f37fe640 1 -- 172.21.15.83:0/3485045732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71f40b45a0 msgr2=0x7f71f40709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:20.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.361+0000 7f71f37fe640 1 --2- 172.21.15.83:0/3485045732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71f40b45a0 0x7f71f40709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.361+0000 7f71f37fe640 1 -- 172.21.15.83:0/3485045732 >> v2:172.21.15.167:3300/0 conn(0x7f71f40b3990 msgr2=0x7f71f4070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:20.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.361+0000 7f71f37fe640 1 --2- 172.21.15.83:0/3485045732 >> v2:172.21.15.167:3300/0 conn(0x7f71f40b3990 0x7f71f4070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.361+0000 7f71f37fe640 1 -- 172.21.15.83:0/3485045732 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f40b2eb0 con 0x7f71f40cf060 2024-04-17T10:43:20.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.365+0000 7f71f37fe640 1 --2- 172.21.15.83:0/3485045732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71f40cf060 0x7f71f40b4060 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f71f40b2380 tx=0x7f71e401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df701a8b74008481 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:20.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.365+0000 7f71f2ffd640 1 -- 172.21.15.83:0/3485045732 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e40b2070 con 0x7f71f40cf060 2024-04-17T10:43:20.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.365+0000 7f71f2ffd640 1 -- 172.21.15.83:0/3485045732 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e40a9070 con 0x7f71f40cf060 2024-04-17T10:43:20.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.365+0000 7f71f2ffd640 1 -- 172.21.15.83:0/3485045732 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e40ad880 con 0x7f71f40cf060 2024-04-17T10:43:20.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.365+0000 7f71f9ee1640 1 -- 172.21.15.83:0/3485045732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71f40cf060 msgr2=0x7f71f40b4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:20.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.365+0000 7f71f9ee1640 1 --2- 172.21.15.83:0/3485045732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71f40cf060 0x7f71f40b4060 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f71f40b2380 tx=0x7f71e401c3a0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.365+0000 7f71f9ee1640 1 -- 172.21.15.83:0/3485045732 shutdown_connections 2024-04-17T10:43:20.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.365+0000 7f71f9ee1640 1 --2- 172.21.15.83:0/3485045732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71f40b45a0 0x7f71f40709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.365+0000 7f71f9ee1640 1 --2- 172.21.15.83:0/3485045732 >> v2:172.21.15.167:3300/0 conn(0x7f71f40b3990 0x7f71f4070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.365+0000 7f71f9ee1640 1 --2- 172.21.15.83:0/3485045732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71f40cf060 0x7f71f40b4060 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.365+0000 7f71f9ee1640 1 -- 172.21.15.83:0/3485045732 >> 172.21.15.83:0/3485045732 conn(0x7f71f406abe0 msgr2=0x7f71f406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:20.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.365+0000 7f71f9ee1640 1 -- 172.21.15.83:0/3485045732 shutdown_connections 2024-04-17T10:43:20.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.365+0000 7f71f9ee1640 1 -- 172.21.15.83:0/3485045732 wait complete. 2024-04-17T10:43:20.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.369+0000 7f71f9ee1640 1 Processor -- start 2024-04-17T10:43:20.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.369+0000 7f71f9ee1640 1 -- start start 2024-04-17T10:43:20.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.369+0000 7f71f9ee1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71f40b3990 0x7f71f40af910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:20.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.369+0000 7f71f9ee1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71f40b45a0 0x7f71f4144c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:20.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.369+0000 7f71f9ee1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71f40cf060 0x7f71f40b0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:20.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.369+0000 7f71f9ee1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f415db70 con 0x7f71f40cf060 2024-04-17T10:43:20.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.369+0000 7f71f9ee1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f415d9f0 con 0x7f71f40b3990 2024-04-17T10:43:20.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.369+0000 7f71f9ee1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f415dcf0 con 0x7f71f40b45a0 2024-04-17T10:43:20.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.369+0000 7f71f3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71f40cf060 0x7f71f40b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:20.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.369+0000 7f71f3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71f40cf060 0x7f71f40b0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51814/0 (socket says 172.21.15.83:51814) 2024-04-17T10:43:20.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.369+0000 7f71f3fff640 1 -- 172.21.15.83:0/3990466119 learned_addr learned my addr 172.21.15.83:0/3990466119 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:20.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.369+0000 7f71f37fe640 1 --2- 172.21.15.83:0/3990466119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71f40b3990 0x7f71f40af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:20.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.369+0000 7f71ebfff640 1 --2- 172.21.15.83:0/3990466119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71f40b45a0 0x7f71f4144c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:20.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.369+0000 7f71f3fff640 1 -- 172.21.15.83:0/3990466119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71f40b45a0 msgr2=0x7f71f4144c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:20.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.369+0000 7f71f3fff640 1 --2- 172.21.15.83:0/3990466119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71f40b45a0 0x7f71f4144c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.369+0000 7f71f3fff640 1 -- 172.21.15.83:0/3990466119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71f40b3990 msgr2=0x7f71f40af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:20.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.369+0000 7f71f3fff640 1 --2- 172.21.15.83:0/3990466119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71f40b3990 0x7f71f40af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.369+0000 7f71f3fff640 1 -- 172.21.15.83:0/3990466119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f40b0b80 con 0x7f71f40cf060 2024-04-17T10:43:20.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.369+0000 7f71ebfff640 1 --2- 172.21.15.83:0/3990466119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71f40b45a0 0x7f71f4144c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:20.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.369+0000 7f71f3fff640 1 --2- 172.21.15.83:0/3990466119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71f40cf060 0x7f71f40b0220 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f71ec007e40 tx=0x7f71ec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:20.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.369+0000 7f71f0ff9640 1 -- 172.21.15.83:0/3990466119 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ec014620 con 0x7f71f40cf060 2024-04-17T10:43:20.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.369+0000 7f71f0ff9640 1 -- 172.21.15.83:0/3990466119 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71ec019070 con 0x7f71f40cf060 2024-04-17T10:43:20.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.369+0000 7f71f9ee1640 1 -- 172.21.15.83:0/3990466119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f40aee30 con 0x7f71f40cf060 2024-04-17T10:43:20.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.369+0000 7f71f0ff9640 1 -- 172.21.15.83:0/3990466119 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ec014a20 con 0x7f71f40cf060 2024-04-17T10:43:20.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.369+0000 7f71f9ee1640 1 -- 172.21.15.83:0/3990466119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f40af3f0 con 0x7f71f40cf060 2024-04-17T10:43:20.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.369+0000 7f71f9ee1640 1 -- 172.21.15.83:0/3990466119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f40af5f0 con 0x7f71f40cf060 2024-04-17T10:43:20.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.369+0000 7f71f0ff9640 1 -- 172.21.15.83:0/3990466119 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f71ec014bc0 con 0x7f71f40cf060 2024-04-17T10:43:20.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.369+0000 7f71f0ff9640 1 --2- 172.21.15.83:0/3990466119 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f71d4066f60 0x7f71d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:20.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.369+0000 7f71f0ff9640 1 -- 172.21.15.83:0/3990466119 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f71ec085950 con 0x7f71f40cf060 2024-04-17T10:43:20.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.373+0000 7f71f37fe640 1 --2- 172.21.15.83:0/3990466119 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f71d4066f60 0x7f71d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:20.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.373+0000 7f71f37fe640 1 --2- 172.21.15.83:0/3990466119 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f71d4066f60 0x7f71d4069420 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f71e40af000 tx=0x7f71e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:20.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.377+0000 7f71f0ff9640 1 -- 172.21.15.83:0/3990466119 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f71f40af5f0 con 0x7f71f40cf060 2024-04-17T10:43:20.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.537+0000 7f71f9ee1640 1 -- 172.21.15.83:0/3990466119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71f4074f90 con 0x7f71f40cf060 2024-04-17T10:43:20.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.541+0000 7f71f0ff9640 1 -- 172.21.15.83:0/3990466119 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16722 (secure 0 0 0) 0x7f71ec056900 con 0x7f71f40cf060 2024-04-17T10:43:20.536 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:43:20.536 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":63,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:42:59.360496+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:42:32.530700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-17T10:42:33.334905+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.167:6833/271994303":"2024-04-18T10:42:59.346779+0000","172.21.15.167:6832/271994303":"2024-04-18T10:42:59.346779+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:43:20.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.541+0000 7f71f9ee1640 1 -- 172.21.15.83:0/3990466119 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f71d4066f60 msgr2=0x7f71d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:20.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.541+0000 7f71f9ee1640 1 --2- 172.21.15.83:0/3990466119 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f71d4066f60 0x7f71d4069420 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f71e40af000 tx=0x7f71e4008040 comp rx=0 tx=0).stop 2024-04-17T10:43:20.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.541+0000 7f71f9ee1640 1 -- 172.21.15.83:0/3990466119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71f40cf060 msgr2=0x7f71f40b0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:20.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.541+0000 7f71f9ee1640 1 --2- 172.21.15.83:0/3990466119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71f40cf060 0x7f71f40b0220 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f71ec007e40 tx=0x7f71ec00e4b0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.541+0000 7f71f9ee1640 1 -- 172.21.15.83:0/3990466119 shutdown_connections 2024-04-17T10:43:20.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.541+0000 7f71f9ee1640 1 --2- 172.21.15.83:0/3990466119 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f71d4066f60 0x7f71d4069420 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.541+0000 7f71f9ee1640 1 --2- 172.21.15.83:0/3990466119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71f40cf060 0x7f71f40b0220 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.541+0000 7f71f9ee1640 1 --2- 172.21.15.83:0/3990466119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71f40b45a0 0x7f71f4144c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.541+0000 7f71f9ee1640 1 --2- 172.21.15.83:0/3990466119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71f40b3990 0x7f71f40af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.541+0000 7f71f9ee1640 1 -- 172.21.15.83:0/3990466119 >> 172.21.15.83:0/3990466119 conn(0x7f71f406abe0 msgr2=0x7f71f4159130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:20.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.541+0000 7f71f9ee1640 1 -- 172.21.15.83:0/3990466119 shutdown_connections 2024-04-17T10:43:20.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.541+0000 7f71f9ee1640 1 -- 172.21.15.83:0/3990466119 wait complete. 2024-04-17T10:43:20.555 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:42:32.530700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:42:33.334905+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:43:20.556 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:43:20.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.685+0000 7f82d3a1f640 1 Processor -- start 2024-04-17T10:43:20.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.685+0000 7f82d3a1f640 1 -- start start 2024-04-17T10:43:20.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.685+0000 7f82d3a1f640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f82cc161940 0x7f82cc06bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:20.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.685+0000 7f82d3a1f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82cc15ed70 0x7f82cc06c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:20.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.685+0000 7f82d3a1f640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82cc0d2430 con 0x7f82cc0cfb80 2024-04-17T10:43:20.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.685+0000 7f82d3a1f640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82cc0d2710 con 0x7f82cc161940 2024-04-17T10:43:20.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.685+0000 7f82d3a1f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc0ced00 con 0x7f82cc15ed70 2024-04-17T10:43:20.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.685+0000 7f82d17bb640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f82cc161940 0x7f82cc06bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:20.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.685+0000 7f82d1fbc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82cc15ed70 0x7f82cc06c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:20.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.685+0000 7f82d0fba640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f82cc0cfb80 0x7f82cc161d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:59424/0 (socket says 172.21.15.83:59424) 2024-04-17T10:43:20.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.685+0000 7f82d0fba640 1 -- 172.21.15.83:0/3892842892 learned_addr learned my addr 172.21.15.83:0/3892842892 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:43:20.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.685+0000 7f82d1fbc640 1 -- 172.21.15.83:0/3892842892 >> v2:172.21.15.167:3300/0 conn(0x7f82cc161940 msgr2=0x7f82cc06bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:20.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.685+0000 7f82d1fbc640 1 --2- 172.21.15.83:0/3892842892 >> v2:172.21.15.167:3300/0 conn(0x7f82cc161940 0x7f82cc06bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.685+0000 7f82d1fbc640 1 -- 172.21.15.83:0/3892842892 >> v1:172.21.15.167:6789/0 conn(0x7f82cc0cfb80 legacy=0x7f82cc161d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:20.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.685+0000 7f82d1fbc640 1 -- 172.21.15.83:0/3892842892 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82cc0cefe0 con 0x7f82cc15ed70 2024-04-17T10:43:20.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.685+0000 7f82d17bb640 1 --2- 172.21.15.83:0/3892842892 >> v2:172.21.15.167:3300/0 conn(0x7f82cc161940 0x7f82cc06bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:43:20.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.689+0000 7f82d1fbc640 1 --2- 172.21.15.83:0/3892842892 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82cc15ed70 0x7f82cc06c590 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009d90 tx=0x7f82c401c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a42049c577181922 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:20.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.689+0000 7f82c3fff640 1 -- 172.21.15.83:0/3892842892 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c40a66b0 con 0x7f82cc15ed70 2024-04-17T10:43:20.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.689+0000 7f82c3fff640 1 -- 172.21.15.83:0/3892842892 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c40a6850 con 0x7f82cc15ed70 2024-04-17T10:43:20.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.689+0000 7f82c3fff640 1 -- 172.21.15.83:0/3892842892 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c40a6c20 con 0x7f82cc15ed70 2024-04-17T10:43:20.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.689+0000 7f82d3a1f640 1 -- 172.21.15.83:0/3892842892 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82cc15ed70 msgr2=0x7f82cc06c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:20.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.689+0000 7f82d3a1f640 1 --2- 172.21.15.83:0/3892842892 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82cc15ed70 0x7f82cc06c590 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009d90 tx=0x7f82c401c8e0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.689+0000 7f82d3a1f640 1 -- 172.21.15.83:0/3892842892 shutdown_connections 2024-04-17T10:43:20.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.689+0000 7f82d3a1f640 1 --2- 172.21.15.83:0/3892842892 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82cc15ed70 0x7f82cc06c590 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.689+0000 7f82d3a1f640 1 --2- 172.21.15.83:0/3892842892 >> v2:172.21.15.167:3300/0 conn(0x7f82cc161940 0x7f82cc06bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.689+0000 7f82d3a1f640 1 -- 172.21.15.83:0/3892842892 >> 172.21.15.83:0/3892842892 conn(0x7f82cc0d17e0 msgr2=0x7f82cc0d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:20.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.689+0000 7f82d3a1f640 1 -- 172.21.15.83:0/3892842892 shutdown_connections 2024-04-17T10:43:20.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.689+0000 7f82d3a1f640 1 -- 172.21.15.83:0/3892842892 wait complete. 2024-04-17T10:43:20.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.693+0000 7f82d3a1f640 1 Processor -- start 2024-04-17T10:43:20.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.693+0000 7f82d3a1f640 1 -- start start 2024-04-17T10:43:20.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.693+0000 7f82d3a1f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82cc0cfb80 0x7f82cc151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:20.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.693+0000 7f82d3a1f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82cc15ed70 0x7f82cc15cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:20.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.693+0000 7f82d3a1f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82cc161940 0x7f82cc158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:20.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.693+0000 7f82d3a1f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc16ac30 con 0x7f82cc161940 2024-04-17T10:43:20.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.693+0000 7f82d3a1f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc0d01a0 con 0x7f82cc15ed70 2024-04-17T10:43:20.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.693+0000 7f82d3a1f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc16adb0 con 0x7f82cc0cfb80 2024-04-17T10:43:20.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.693+0000 7f82d1fbc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82cc161940 0x7f82cc158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:20.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.693+0000 7f82d1fbc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82cc161940 0x7f82cc158a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51830/0 (socket says 172.21.15.83:51830) 2024-04-17T10:43:20.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.693+0000 7f82d1fbc640 1 -- 172.21.15.83:0/805451721 learned_addr learned my addr 172.21.15.83:0/805451721 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:20.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.693+0000 7f82d0fba640 1 --2- 172.21.15.83:0/805451721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82cc15ed70 0x7f82cc15cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:20.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.693+0000 7f82d17bb640 1 --2- 172.21.15.83:0/805451721 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82cc0cfb80 0x7f82cc151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:20.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.693+0000 7f82d1fbc640 1 -- 172.21.15.83:0/805451721 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82cc0cfb80 msgr2=0x7f82cc151120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:20.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.693+0000 7f82d1fbc640 1 --2- 172.21.15.83:0/805451721 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82cc0cfb80 0x7f82cc151120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.693+0000 7f82d1fbc640 1 -- 172.21.15.83:0/805451721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82cc15ed70 msgr2=0x7f82cc15cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:20.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.693+0000 7f82d1fbc640 1 --2- 172.21.15.83:0/805451721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82cc15ed70 0x7f82cc15cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.693+0000 7f82d1fbc640 1 -- 172.21.15.83:0/805451721 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82cc15d1a0 con 0x7f82cc161940 2024-04-17T10:43:20.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.693+0000 7f82d1fbc640 1 --2- 172.21.15.83:0/805451721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82cc161940 0x7f82cc158a90 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f82c401a510 tx=0x7f82c4019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:20.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.693+0000 7f82c1ffb640 1 -- 172.21.15.83:0/805451721 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c40a6e10 con 0x7f82cc161940 2024-04-17T10:43:20.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.693+0000 7f82c1ffb640 1 -- 172.21.15.83:0/805451721 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c40c1070 con 0x7f82cc161940 2024-04-17T10:43:20.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.693+0000 7f82d3a1f640 1 -- 172.21.15.83:0/805451721 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82cc15b910 con 0x7f82cc161940 2024-04-17T10:43:20.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.693+0000 7f82c1ffb640 1 -- 172.21.15.83:0/805451721 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c40b8670 con 0x7f82cc161940 2024-04-17T10:43:20.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.693+0000 7f82d3a1f640 1 -- 172.21.15.83:0/805451721 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82cc15bed0 con 0x7f82cc161940 2024-04-17T10:43:20.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.697+0000 7f82d3a1f640 1 -- 172.21.15.83:0/805451721 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82cc074f90 con 0x7f82cc161940 2024-04-17T10:43:20.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.697+0000 7f82c1ffb640 1 -- 172.21.15.83:0/805451721 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f82c40b8810 con 0x7f82cc161940 2024-04-17T10:43:20.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.697+0000 7f82c1ffb640 1 --2- 172.21.15.83:0/805451721 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f82b0066f60 0x7f82b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:20.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.697+0000 7f82c1ffb640 1 -- 172.21.15.83:0/805451721 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f82c411fe70 con 0x7f82cc161940 2024-04-17T10:43:20.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.697+0000 7f82d17bb640 1 --2- 172.21.15.83:0/805451721 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f82b0066f60 0x7f82b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:20.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.697+0000 7f82d17bb640 1 --2- 172.21.15.83:0/805451721 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f82b0066f60 0x7f82b0069420 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f82c8009a50 tx=0x7f82c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:20.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.701+0000 7f82c1ffb640 1 -- 172.21.15.83:0/805451721 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f82c4016560 con 0x7f82cc161940 2024-04-17T10:43:20.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.861+0000 7f82d3a1f640 1 -- 172.21.15.83:0/805451721 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f82cc06b060 con 0x7f82cc161940 2024-04-17T10:43:20.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.861+0000 7f82c1ffb640 1 -- 172.21.15.83:0/805451721 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7f82c40f0da0 con 0x7f82cc161940 2024-04-17T10:43:20.858 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:43:20.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.865+0000 7f82d3a1f640 1 -- 172.21.15.83:0/805451721 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f82b0066f60 msgr2=0x7f82b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:20.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.865+0000 7f82d3a1f640 1 --2- 172.21.15.83:0/805451721 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f82b0066f60 0x7f82b0069420 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f82c8009a50 tx=0x7f82c8008040 comp rx=0 tx=0).stop 2024-04-17T10:43:20.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.865+0000 7f82d3a1f640 1 -- 172.21.15.83:0/805451721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82cc161940 msgr2=0x7f82cc158a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:20.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.865+0000 7f82d3a1f640 1 --2- 172.21.15.83:0/805451721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82cc161940 0x7f82cc158a90 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f82c401a510 tx=0x7f82c4019680 comp rx=0 tx=0).stop 2024-04-17T10:43:20.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.865+0000 7f82d3a1f640 1 -- 172.21.15.83:0/805451721 shutdown_connections 2024-04-17T10:43:20.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.865+0000 7f82d3a1f640 1 --2- 172.21.15.83:0/805451721 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f82b0066f60 0x7f82b0069420 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.865+0000 7f82d3a1f640 1 --2- 172.21.15.83:0/805451721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82cc161940 0x7f82cc158a90 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.865+0000 7f82d3a1f640 1 --2- 172.21.15.83:0/805451721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82cc15ed70 0x7f82cc15cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.865+0000 7f82d3a1f640 1 --2- 172.21.15.83:0/805451721 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82cc0cfb80 0x7f82cc151120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.865+0000 7f82d3a1f640 1 -- 172.21.15.83:0/805451721 >> 172.21.15.83:0/805451721 conn(0x7f82cc0d17e0 msgr2=0x7f82cc1571a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:20.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.865+0000 7f82d3a1f640 1 -- 172.21.15.83:0/805451721 shutdown_connections 2024-04-17T10:43:20.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.865+0000 7f82d3a1f640 1 -- 172.21.15.83:0/805451721 wait complete. 2024-04-17T10:43:20.877 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:43:20.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.997+0000 7f88cf338640 1 Processor -- start 2024-04-17T10:43:20.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:20.997+0000 7f88cf338640 1 -- start start 2024-04-17T10:43:20.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.001+0000 7f88cf338640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88c815b6e0 0x7f88c8071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:20.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.001+0000 7f88cf338640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f88c815bac0 0x7f88c80cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:20.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.001+0000 7f88cf338640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88c806c1f0 con 0x7f88c8070470 2024-04-17T10:43:20.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.001+0000 7f88cf338640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88c80cdd30 con 0x7f88c815bac0 2024-04-17T10:43:20.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.001+0000 7f88cf338640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c8070a10 con 0x7f88c815b6e0 2024-04-17T10:43:20.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.001+0000 7f88cc8d3640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f88c815bac0 0x7f88c80cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:20.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.001+0000 7f88cc8d3640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f88c815bac0 0x7f88c80cd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51836/0 (socket says 172.21.15.83:51836) 2024-04-17T10:43:20.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.001+0000 7f88cc8d3640 1 -- 172.21.15.83:0/1076696115 learned_addr learned my addr 172.21.15.83:0/1076696115 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:20.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.001+0000 7f88cd0d4640 1 --2- 172.21.15.83:0/1076696115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88c815b6e0 0x7f88c8071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:20.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.001+0000 7f88cc8d3640 1 -- 172.21.15.83:0/1076696115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88c815b6e0 msgr2=0x7f88c8071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:20.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.001+0000 7f88cc8d3640 1 --2- 172.21.15.83:0/1076696115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88c815b6e0 0x7f88c8071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:20.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.001+0000 7f88cc8d3640 1 -- 172.21.15.83:0/1076696115 >> v1:172.21.15.167:6789/0 conn(0x7f88c8070470 legacy=0x7f88c806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:20.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.001+0000 7f88cc8d3640 1 -- 172.21.15.83:0/1076696115 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c8070cf0 con 0x7f88c815bac0 2024-04-17T10:43:20.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.001+0000 7f88cc8d3640 1 --2- 172.21.15.83:0/1076696115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c815bac0 0x7f88c80cd7f0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f88b8009870 tx=0x7f88b801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea3bcb084e640f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:20.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.001+0000 7f88bffff640 1 -- 172.21.15.83:0/1076696115 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b80b2070 con 0x7f88c815bac0 2024-04-17T10:43:20.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.001+0000 7f88bffff640 1 -- 172.21.15.83:0/1076696115 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b80a9070 con 0x7f88c815bac0 2024-04-17T10:43:20.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.001+0000 7f88bffff640 1 -- 172.21.15.83:0/1076696115 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b80ad880 con 0x7f88c815bac0 2024-04-17T10:43:21.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88cf338640 1 -- 172.21.15.83:0/1076696115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c815bac0 msgr2=0x7f88c80cd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:21.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88cf338640 1 --2- 172.21.15.83:0/1076696115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c815bac0 0x7f88c80cd7f0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f88b8009870 tx=0x7f88b801c560 comp rx=0 tx=0).stop 2024-04-17T10:43:21.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88cf338640 1 -- 172.21.15.83:0/1076696115 shutdown_connections 2024-04-17T10:43:21.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88cf338640 1 --2- 172.21.15.83:0/1076696115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c815bac0 0x7f88c80cd7f0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:21.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88cf338640 1 --2- 172.21.15.83:0/1076696115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88c815b6e0 0x7f88c8071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:21.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88cf338640 1 -- 172.21.15.83:0/1076696115 >> 172.21.15.83:0/1076696115 conn(0x7f88c80b15b0 msgr2=0x7f88c80b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:21.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88cf338640 1 -- 172.21.15.83:0/1076696115 shutdown_connections 2024-04-17T10:43:21.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88cf338640 1 -- 172.21.15.83:0/1076696115 wait complete. 2024-04-17T10:43:21.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88cf338640 1 Processor -- start 2024-04-17T10:43:21.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88cf338640 1 -- start start 2024-04-17T10:43:21.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88cf338640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88c8070470 0x7f88c814e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:21.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88cf338640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c815b6e0 0x7f88c8156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:21.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88cf338640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88c8157470 0x7f88c80b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:21.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88cf338640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c806cee0 con 0x7f88c815b6e0 2024-04-17T10:43:21.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88cf338640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c806b730 con 0x7f88c8070470 2024-04-17T10:43:21.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88cf338640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c806d060 con 0x7f88c8157470 2024-04-17T10:43:21.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88cc8d3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c815b6e0 0x7f88c8156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:21.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88cc8d3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c815b6e0 0x7f88c8156f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51848/0 (socket says 172.21.15.83:51848) 2024-04-17T10:43:21.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88cc8d3640 1 -- 172.21.15.83:0/172373464 learned_addr learned my addr 172.21.15.83:0/172373464 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:21.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88cd8d5640 1 --2- 172.21.15.83:0/172373464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88c8157470 0x7f88c80b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:21.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88cd0d4640 1 --2- 172.21.15.83:0/172373464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88c8070470 0x7f88c814e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:21.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88cc8d3640 1 -- 172.21.15.83:0/172373464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88c8157470 msgr2=0x7f88c80b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:21.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88cc8d3640 1 --2- 172.21.15.83:0/172373464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88c8157470 0x7f88c80b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:21.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88cc8d3640 1 -- 172.21.15.83:0/172373464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88c8070470 msgr2=0x7f88c814e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:21.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88cc8d3640 1 --2- 172.21.15.83:0/172373464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88c8070470 0x7f88c814e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:21.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88cc8d3640 1 -- 172.21.15.83:0/172373464 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c81f64d0 con 0x7f88c815b6e0 2024-04-17T10:43:21.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88cd8d5640 1 --2- 172.21.15.83:0/172373464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88c8157470 0x7f88c80b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:21.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88cd0d4640 1 --2- 172.21.15.83:0/172373464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88c8070470 0x7f88c814e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:21.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88cc8d3640 1 --2- 172.21.15.83:0/172373464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c815b6e0 0x7f88c8156f30 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f88b80b0040 tx=0x7f88b8019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:21.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88bdffb640 1 -- 172.21.15.83:0/172373464 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b80b2040 con 0x7f88c815b6e0 2024-04-17T10:43:21.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88bdffb640 1 -- 172.21.15.83:0/172373464 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b80bb070 con 0x7f88c815b6e0 2024-04-17T10:43:21.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88cf338640 1 -- 172.21.15.83:0/172373464 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c81f67b0 con 0x7f88c815b6e0 2024-04-17T10:43:21.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88bdffb640 1 -- 172.21.15.83:0/172373464 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b80b6440 con 0x7f88c815b6e0 2024-04-17T10:43:21.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88cf338640 1 -- 172.21.15.83:0/172373464 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c81f6120 con 0x7f88c815b6e0 2024-04-17T10:43:21.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.005+0000 7f88cf338640 1 -- 172.21.15.83:0/172373464 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88c8157880 con 0x7f88c815b6e0 2024-04-17T10:43:21.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.009+0000 7f88bdffb640 1 -- 172.21.15.83:0/172373464 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f88b80ab070 con 0x7f88c815b6e0 2024-04-17T10:43:21.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.009+0000 7f88bdffb640 1 --2- 172.21.15.83:0/172373464 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f88ac066f60 0x7f88ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:21.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.009+0000 7f88bdffb640 1 -- 172.21.15.83:0/172373464 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f88b811ea30 con 0x7f88c815b6e0 2024-04-17T10:43:21.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.009+0000 7f88cd0d4640 1 --2- 172.21.15.83:0/172373464 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f88ac066f60 0x7f88ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:21.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.009+0000 7f88cd0d4640 1 --2- 172.21.15.83:0/172373464 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f88ac066f60 0x7f88ac069420 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f88c40097a0 tx=0x7f88c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:21.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.013+0000 7f88bdffb640 1 -- 172.21.15.83:0/172373464 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f88b8016150 con 0x7f88c815b6e0 2024-04-17T10:43:21.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.157+0000 7f88cf338640 1 -- 172.21.15.83:0/172373464 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f88c8074f90 con 0x7f88c815b6e0 2024-04-17T10:43:21.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.157+0000 7f88bdffb640 1 -- 172.21.15.83:0/172373464 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v63) v1 ==== 104+0+11 (secure 0 0 0) 0x7f88b80ef9e0 con 0x7f88c815b6e0 2024-04-17T10:43:21.154 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:43:21.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.161+0000 7f88cf338640 1 -- 172.21.15.83:0/172373464 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f88ac066f60 msgr2=0x7f88ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:21.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.161+0000 7f88cf338640 1 --2- 172.21.15.83:0/172373464 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f88ac066f60 0x7f88ac069420 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f88c40097a0 tx=0x7f88c4008040 comp rx=0 tx=0).stop 2024-04-17T10:43:21.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.161+0000 7f88cf338640 1 -- 172.21.15.83:0/172373464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c815b6e0 msgr2=0x7f88c8156f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:21.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.161+0000 7f88cf338640 1 --2- 172.21.15.83:0/172373464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c815b6e0 0x7f88c8156f30 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f88b80b0040 tx=0x7f88b8019300 comp rx=0 tx=0).stop 2024-04-17T10:43:21.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.161+0000 7f88cf338640 1 -- 172.21.15.83:0/172373464 shutdown_connections 2024-04-17T10:43:21.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.161+0000 7f88cf338640 1 --2- 172.21.15.83:0/172373464 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f88ac066f60 0x7f88ac069420 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:21.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.161+0000 7f88cf338640 1 --2- 172.21.15.83:0/172373464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88c8157470 0x7f88c80b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:21.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.161+0000 7f88cf338640 1 --2- 172.21.15.83:0/172373464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c815b6e0 0x7f88c8156f30 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:21.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.161+0000 7f88cf338640 1 --2- 172.21.15.83:0/172373464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88c8070470 0x7f88c814e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:21.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.161+0000 7f88cf338640 1 -- 172.21.15.83:0/172373464 >> 172.21.15.83:0/172373464 conn(0x7f88c80b15b0 msgr2=0x7f88c80b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:21.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.161+0000 7f88cf338640 1 -- 172.21.15.83:0/172373464 shutdown_connections 2024-04-17T10:43:21.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.161+0000 7f88cf338640 1 -- 172.21.15.83:0/172373464 wait complete. 2024-04-17T10:43:21.172 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:43:21.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.289+0000 7f4b9b8b4640 1 Processor -- start 2024-04-17T10:43:21.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.289+0000 7f4b9b8b4640 1 -- start start 2024-04-17T10:43:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.289+0000 7f4b9b8b4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b94071ae0 0x7f4b9415bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.289+0000 7f4b9b8b4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4b94156080 0x7f4b9406abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.289+0000 7f4b9b8b4640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b940cd390 con 0x7f4b94071700 2024-04-17T10:43:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.289+0000 7f4b9b8b4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b940b3e40 con 0x7f4b94156080 2024-04-17T10:43:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.289+0000 7f4b9b8b4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b940b4120 con 0x7f4b94071ae0 2024-04-17T10:43:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.289+0000 7f4b99e51640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4b94156080 0x7f4b9406abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.289+0000 7f4b99e51640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4b94156080 0x7f4b9406abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51858/0 (socket says 172.21.15.83:51858) 2024-04-17T10:43:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.289+0000 7f4b99e51640 1 -- 172.21.15.83:0/4251959417 learned_addr learned my addr 172.21.15.83:0/4251959417 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.289+0000 7f4b98e4f640 1 --2- 172.21.15.83:0/4251959417 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b94071ae0 0x7f4b9415bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.289+0000 7f4b99e51640 1 -- 172.21.15.83:0/4251959417 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b94071ae0 msgr2=0x7f4b9415bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.289+0000 7f4b99e51640 1 --2- 172.21.15.83:0/4251959417 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b94071ae0 0x7f4b9415bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.289+0000 7f4b99e51640 1 -- 172.21.15.83:0/4251959417 >> v1:172.21.15.167:6789/0 conn(0x7f4b94071700 legacy=0x7f4b94155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.289+0000 7f4b99e51640 1 -- 172.21.15.83:0/4251959417 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b940b4400 con 0x7f4b94156080 2024-04-17T10:43:21.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.293+0000 7f4b99e51640 1 --2- 172.21.15.83:0/4251959417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b94156080 0x7f4b9406abe0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f4b84009e50 tx=0x7f4b8401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e49e694f294660 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:21.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.293+0000 7f4b93fff640 1 -- 172.21.15.83:0/4251959417 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b840a5860 con 0x7f4b94156080 2024-04-17T10:43:21.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.293+0000 7f4b93fff640 1 -- 172.21.15.83:0/4251959417 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b840a5a00 con 0x7f4b94156080 2024-04-17T10:43:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.293+0000 7f4b93fff640 1 -- 172.21.15.83:0/4251959417 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b840a5dd0 con 0x7f4b94156080 2024-04-17T10:43:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.293+0000 7f4b9b8b4640 1 -- 172.21.15.83:0/4251959417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b94156080 msgr2=0x7f4b9406abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:21.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.293+0000 7f4b9b8b4640 1 --2- 172.21.15.83:0/4251959417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b94156080 0x7f4b9406abe0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f4b84009e50 tx=0x7f4b8401ca00 comp rx=0 tx=0).stop 2024-04-17T10:43:21.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.293+0000 7f4b9b8b4640 1 -- 172.21.15.83:0/4251959417 shutdown_connections 2024-04-17T10:43:21.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.293+0000 7f4b9b8b4640 1 --2- 172.21.15.83:0/4251959417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b94156080 0x7f4b9406abe0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:21.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.293+0000 7f4b9b8b4640 1 --2- 172.21.15.83:0/4251959417 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b94071ae0 0x7f4b9415bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:21.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.293+0000 7f4b9b8b4640 1 -- 172.21.15.83:0/4251959417 >> 172.21.15.83:0/4251959417 conn(0x7f4b940b23f0 msgr2=0x7f4b940b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:21.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.293+0000 7f4b9b8b4640 1 -- 172.21.15.83:0/4251959417 shutdown_connections 2024-04-17T10:43:21.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.293+0000 7f4b9b8b4640 1 -- 172.21.15.83:0/4251959417 wait complete. 2024-04-17T10:43:21.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.293+0000 7f4b9b8b4640 1 Processor -- start 2024-04-17T10:43:21.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.293+0000 7f4b9b8b4640 1 -- start start 2024-04-17T10:43:21.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.293+0000 7f4b9b8b4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b94071700 0x7f4b9415e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:21.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.293+0000 7f4b9b8b4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b94071ae0 0x7f4b94136780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:21.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.293+0000 7f4b9b8b4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b94156080 0x7f4b9414f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:21.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.293+0000 7f4b9b8b4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b940afe20 con 0x7f4b94156080 2024-04-17T10:43:21.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.293+0000 7f4b9b8b4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b940afca0 con 0x7f4b94071700 2024-04-17T10:43:21.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.293+0000 7f4b9b8b4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b940affa0 con 0x7f4b94071ae0 2024-04-17T10:43:21.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.293+0000 7f4b99e51640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b94156080 0x7f4b9414f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:21.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.293+0000 7f4b99e51640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b94156080 0x7f4b9414f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51860/0 (socket says 172.21.15.83:51860) 2024-04-17T10:43:21.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.293+0000 7f4b99e51640 1 -- 172.21.15.83:0/1998946709 learned_addr learned my addr 172.21.15.83:0/1998946709 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.293+0000 7f4b99650640 1 --2- 172.21.15.83:0/1998946709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b94071700 0x7f4b9415e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.293+0000 7f4b98e4f640 1 --2- 172.21.15.83:0/1998946709 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b94071ae0 0x7f4b94136780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.293+0000 7f4b99e51640 1 -- 172.21.15.83:0/1998946709 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b94071ae0 msgr2=0x7f4b94136780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.293+0000 7f4b99e51640 1 --2- 172.21.15.83:0/1998946709 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b94071ae0 0x7f4b94136780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.293+0000 7f4b99e51640 1 -- 172.21.15.83:0/1998946709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b94071700 msgr2=0x7f4b9415e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.293+0000 7f4b99e51640 1 --2- 172.21.15.83:0/1998946709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b94071700 0x7f4b9415e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.293+0000 7f4b99e51640 1 -- 172.21.15.83:0/1998946709 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b9415ef30 con 0x7f4b94156080 2024-04-17T10:43:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.293+0000 7f4b98e4f640 1 --2- 172.21.15.83:0/1998946709 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b94071ae0 0x7f4b94136780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.293+0000 7f4b99e51640 1 --2- 172.21.15.83:0/1998946709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b94156080 0x7f4b9414f690 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f4b8401a630 tx=0x7f4b840197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.297+0000 7f4b91ffb640 1 -- 172.21.15.83:0/1998946709 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b840b8670 con 0x7f4b94156080 2024-04-17T10:43:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.297+0000 7f4b91ffb640 1 -- 172.21.15.83:0/1998946709 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b840be070 con 0x7f4b94156080 2024-04-17T10:43:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.297+0000 7f4b9b8b4640 1 -- 172.21.15.83:0/1998946709 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b94139400 con 0x7f4b94156080 2024-04-17T10:43:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.297+0000 7f4b91ffb640 1 -- 172.21.15.83:0/1998946709 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b840b8a70 con 0x7f4b94156080 2024-04-17T10:43:21.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.297+0000 7f4b9b8b4640 1 -- 172.21.15.83:0/1998946709 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b941399c0 con 0x7f4b94156080 2024-04-17T10:43:21.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.297+0000 7f4b9b8b4640 1 -- 172.21.15.83:0/1998946709 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b94074f90 con 0x7f4b94156080 2024-04-17T10:43:21.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.297+0000 7f4b91ffb640 1 -- 172.21.15.83:0/1998946709 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4b840b8c10 con 0x7f4b94156080 2024-04-17T10:43:21.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.297+0000 7f4b91ffb640 1 --2- 172.21.15.83:0/1998946709 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4b80066f60 0x7f4b80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:21.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.297+0000 7f4b91ffb640 1 -- 172.21.15.83:0/1998946709 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4b841205b0 con 0x7f4b94156080 2024-04-17T10:43:21.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.297+0000 7f4b99650640 1 --2- 172.21.15.83:0/1998946709 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4b80066f60 0x7f4b80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:21.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.297+0000 7f4b99650640 1 --2- 172.21.15.83:0/1998946709 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4b80066f60 0x7f4b80069420 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c001650 tx=0x7f4b8c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:21.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.301+0000 7f4b91ffb640 1 -- 172.21.15.83:0/1998946709 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4b84016620 con 0x7f4b94156080 2024-04-17T10:43:21.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.449+0000 7f4b9b8b4640 1 -- 172.21.15.83:0/1998946709 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4b940b4e50 con 0x7f4b94156080 2024-04-17T10:43:21.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.449+0000 7f4b91ffb640 1 -- 172.21.15.83:0/1998946709 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v63) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4b840f1690 con 0x7f4b94156080 2024-04-17T10:43:21.445 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:43:21.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.449+0000 7f4b9b8b4640 1 -- 172.21.15.83:0/1998946709 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4b80066f60 msgr2=0x7f4b80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:21.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.449+0000 7f4b9b8b4640 1 --2- 172.21.15.83:0/1998946709 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4b80066f60 0x7f4b80069420 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c001650 tx=0x7f4b8c009040 comp rx=0 tx=0).stop 2024-04-17T10:43:21.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.449+0000 7f4b9b8b4640 1 -- 172.21.15.83:0/1998946709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b94156080 msgr2=0x7f4b9414f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:21.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.449+0000 7f4b9b8b4640 1 --2- 172.21.15.83:0/1998946709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b94156080 0x7f4b9414f690 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f4b8401a630 tx=0x7f4b840197a0 comp rx=0 tx=0).stop 2024-04-17T10:43:21.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.449+0000 7f4b9b8b4640 1 -- 172.21.15.83:0/1998946709 shutdown_connections 2024-04-17T10:43:21.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.449+0000 7f4b9b8b4640 1 --2- 172.21.15.83:0/1998946709 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4b80066f60 0x7f4b80069420 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:21.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.449+0000 7f4b9b8b4640 1 --2- 172.21.15.83:0/1998946709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b94156080 0x7f4b9414f690 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:21.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.449+0000 7f4b9b8b4640 1 --2- 172.21.15.83:0/1998946709 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b94071ae0 0x7f4b94136780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:21.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.449+0000 7f4b9b8b4640 1 --2- 172.21.15.83:0/1998946709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b94071700 0x7f4b9415e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:21.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.449+0000 7f4b9b8b4640 1 -- 172.21.15.83:0/1998946709 >> 172.21.15.83:0/1998946709 conn(0x7f4b940b23f0 msgr2=0x7f4b9415b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:21.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.449+0000 7f4b9b8b4640 1 -- 172.21.15.83:0/1998946709 shutdown_connections 2024-04-17T10:43:21.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.449+0000 7f4b9b8b4640 1 -- 172.21.15.83:0/1998946709 wait complete. 2024-04-17T10:43:21.463 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:43:21.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.581+0000 7fab99537640 1 Processor -- start 2024-04-17T10:43:21.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.581+0000 7fab99537640 1 -- start start 2024-04-17T10:43:21.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.585+0000 7fab99537640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fab940aedd0 0x7fab9406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:21.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.585+0000 7fab99537640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab9406dae0 0x7fab940b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:21.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.585+0000 7fab99537640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab9415e440 con 0x7fab9415db50 2024-04-17T10:43:21.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.585+0000 7fab99537640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab9415e720 con 0x7fab940aedd0 2024-04-17T10:43:21.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.585+0000 7fab99537640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9415ea00 con 0x7fab9406dae0 2024-04-17T10:43:21.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.585+0000 7fab927fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fab940aedd0 0x7fab9406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:21.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.585+0000 7fab927fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fab940aedd0 0x7fab9406d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51876/0 (socket says 172.21.15.83:51876) 2024-04-17T10:43:21.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.585+0000 7fab927fc640 1 -- 172.21.15.83:0/1561966482 learned_addr learned my addr 172.21.15.83:0/1561966482 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:21.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.585+0000 7fab937fe640 1 --2- 172.21.15.83:0/1561966482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab9406dae0 0x7fab940b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:21.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.585+0000 7fab927fc640 1 -- 172.21.15.83:0/1561966482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab9406dae0 msgr2=0x7fab940b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:21.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.585+0000 7fab927fc640 1 --2- 172.21.15.83:0/1561966482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab9406dae0 0x7fab940b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:21.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.585+0000 7fab927fc640 1 -- 172.21.15.83:0/1561966482 >> v1:172.21.15.167:6789/0 conn(0x7fab9415db50 legacy=0x7fab94070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:21.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.585+0000 7fab927fc640 1 -- 172.21.15.83:0/1561966482 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9415ece0 con 0x7fab940aedd0 2024-04-17T10:43:21.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.585+0000 7fab927fc640 1 --2- 172.21.15.83:0/1561966482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab940aedd0 0x7fab9406d5a0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fab7c009870 tx=0x7fab7c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18ea77732f1004e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:21.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.585+0000 7fab91ffb640 1 -- 172.21.15.83:0/1561966482 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c0b0070 con 0x7fab940aedd0 2024-04-17T10:43:21.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.585+0000 7fab91ffb640 1 -- 172.21.15.83:0/1561966482 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7c0ab600 con 0x7fab940aedd0 2024-04-17T10:43:21.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.585+0000 7fab91ffb640 1 -- 172.21.15.83:0/1561966482 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c0ab9d0 con 0x7fab940aedd0 2024-04-17T10:43:21.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.589+0000 7fab99537640 1 -- 172.21.15.83:0/1561966482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab940aedd0 msgr2=0x7fab9406d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:21.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.589+0000 7fab99537640 1 --2- 172.21.15.83:0/1561966482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab940aedd0 0x7fab9406d5a0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fab7c009870 tx=0x7fab7c01c520 comp rx=0 tx=0).stop 2024-04-17T10:43:21.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.589+0000 7fab99537640 1 -- 172.21.15.83:0/1561966482 shutdown_connections 2024-04-17T10:43:21.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.589+0000 7fab99537640 1 --2- 172.21.15.83:0/1561966482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab9406dae0 0x7fab940b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:21.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.589+0000 7fab99537640 1 --2- 172.21.15.83:0/1561966482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab940aedd0 0x7fab9406d5a0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:21.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.589+0000 7fab99537640 1 -- 172.21.15.83:0/1561966482 >> 172.21.15.83:0/1561966482 conn(0x7fab9406abe0 msgr2=0x7fab9406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:21.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.589+0000 7fab99537640 1 -- 172.21.15.83:0/1561966482 shutdown_connections 2024-04-17T10:43:21.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.589+0000 7fab99537640 1 -- 172.21.15.83:0/1561966482 wait complete. 2024-04-17T10:43:21.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.589+0000 7fab99537640 1 Processor -- start 2024-04-17T10:43:21.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.589+0000 7fab99537640 1 -- start start 2024-04-17T10:43:21.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.589+0000 7fab99537640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab9406dae0 0x7fab940cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:21.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.589+0000 7fab99537640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab940aedd0 0x7fab9414ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:21.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.589+0000 7fab99537640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab9415db50 0x7fab9406c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:21.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.589+0000 7fab99537640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab940b3ef0 con 0x7fab9415db50 2024-04-17T10:43:21.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.589+0000 7fab99537640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab940b3d70 con 0x7fab940aedd0 2024-04-17T10:43:21.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.589+0000 7fab99537640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab940b4070 con 0x7fab9406dae0 2024-04-17T10:43:21.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.589+0000 7fab937fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab9415db50 0x7fab9406c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:21.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.589+0000 7fab937fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab9415db50 0x7fab9406c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51892/0 (socket says 172.21.15.83:51892) 2024-04-17T10:43:21.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.589+0000 7fab937fe640 1 -- 172.21.15.83:0/3436436223 learned_addr learned my addr 172.21.15.83:0/3436436223 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:21.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.589+0000 7fab927fc640 1 --2- 172.21.15.83:0/3436436223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab940aedd0 0x7fab9414ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:21.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.589+0000 7fab937fe640 1 -- 172.21.15.83:0/3436436223 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab9406dae0 msgr2=0x7fab940cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:21.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.589+0000 7fab937fe640 1 --2- 172.21.15.83:0/3436436223 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab9406dae0 0x7fab940cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:21.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.589+0000 7fab937fe640 1 -- 172.21.15.83:0/3436436223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab940aedd0 msgr2=0x7fab9414ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:21.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.589+0000 7fab937fe640 1 --2- 172.21.15.83:0/3436436223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab940aedd0 0x7fab9414ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:21.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.589+0000 7fab937fe640 1 -- 172.21.15.83:0/3436436223 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9415cc40 con 0x7fab9415db50 2024-04-17T10:43:21.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.589+0000 7fab937fe640 1 --2- 172.21.15.83:0/3436436223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab9415db50 0x7fab9406c390 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fab8400cc30 tx=0x7fab8400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:21.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.589+0000 7fab8b7fe640 1 -- 172.21.15.83:0/3436436223 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8400ce40 con 0x7fab9415db50 2024-04-17T10:43:21.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.589+0000 7fab8b7fe640 1 -- 172.21.15.83:0/3436436223 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab840039a0 con 0x7fab9415db50 2024-04-17T10:43:21.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.589+0000 7fab99537640 1 -- 172.21.15.83:0/3436436223 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab9415b530 con 0x7fab9415db50 2024-04-17T10:43:21.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.589+0000 7fab8b7fe640 1 -- 172.21.15.83:0/3436436223 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab84003d50 con 0x7fab9415db50 2024-04-17T10:43:21.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.589+0000 7fab99537640 1 -- 172.21.15.83:0/3436436223 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab9415d1c0 con 0x7fab9415db50 2024-04-17T10:43:21.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.589+0000 7fab99537640 1 -- 172.21.15.83:0/3436436223 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab94074f90 con 0x7fab9415db50 2024-04-17T10:43:21.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.593+0000 7fab8b7fe640 1 -- 172.21.15.83:0/3436436223 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fab84003fb0 con 0x7fab9415db50 2024-04-17T10:43:21.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.593+0000 7fab8b7fe640 1 --2- 172.21.15.83:0/3436436223 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fab74066f60 0x7fab74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:21.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.593+0000 7fab8b7fe640 1 -- 172.21.15.83:0/3436436223 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fab84085520 con 0x7fab9415db50 2024-04-17T10:43:21.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.593+0000 7fab92ffd640 1 --2- 172.21.15.83:0/3436436223 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fab74066f60 0x7fab74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:21.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.593+0000 7fab92ffd640 1 --2- 172.21.15.83:0/3436436223 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fab74066f60 0x7fab74069420 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fab94070b00 tx=0x7fab8c0079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:21.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.597+0000 7fab8b7fe640 1 -- 172.21.15.83:0/3436436223 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fab84014030 con 0x7fab9415db50 2024-04-17T10:43:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.773+0000 7fab99537640 1 -- 172.21.15.83:0/3436436223 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab940b2af0 con 0x7fab9415db50 2024-04-17T10:43:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.773+0000 7fab8b7fe640 1 -- 172.21.15.83:0/3436436223 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4942 (secure 0 0 0) 0x7fab840564d0 con 0x7fab9415db50 2024-04-17T10:43:21.769 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:43:21.769 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":39,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5165,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1924992370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1924992370},{"type":"v1","addr":"172.21.15.167:6833","nonce":1924992370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5174,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3876293847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3876293847},{"type":"v1","addr":"172.21.15.167:6835","nonce":3876293847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6831,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/338926944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":338926944},{"type":"v1","addr":"172.21.15.83:6835","nonce":338926944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:42:34.328397+0000","modified":"2024-04-17T10:43:02.389233+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6843},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6843":{"gid":6843,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6837/3985646487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3985646487},{"type":"v1","addr":"172.21.15.83:6837","nonce":3985646487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-17T10:43:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 39 2024-04-17T10:43:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.773+0000 7fab99537640 1 -- 172.21.15.83:0/3436436223 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fab74066f60 msgr2=0x7fab74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.773+0000 7fab99537640 1 --2- 172.21.15.83:0/3436436223 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fab74066f60 0x7fab74069420 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fab94070b00 tx=0x7fab8c0079c0 comp rx=0 tx=0).stop 2024-04-17T10:43:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.773+0000 7fab99537640 1 -- 172.21.15.83:0/3436436223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab9415db50 msgr2=0x7fab9406c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.773+0000 7fab99537640 1 --2- 172.21.15.83:0/3436436223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab9415db50 0x7fab9406c390 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fab8400cc30 tx=0x7fab8400a5c0 comp rx=0 tx=0).stop 2024-04-17T10:43:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.773+0000 7fab99537640 1 -- 172.21.15.83:0/3436436223 shutdown_connections 2024-04-17T10:43:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.773+0000 7fab99537640 1 --2- 172.21.15.83:0/3436436223 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fab74066f60 0x7fab74069420 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.773+0000 7fab99537640 1 --2- 172.21.15.83:0/3436436223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab9415db50 0x7fab9406c390 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.773+0000 7fab99537640 1 --2- 172.21.15.83:0/3436436223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab940aedd0 0x7fab9414ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.773+0000 7fab99537640 1 --2- 172.21.15.83:0/3436436223 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab9406dae0 0x7fab940cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.773+0000 7fab99537640 1 -- 172.21.15.83:0/3436436223 >> 172.21.15.83:0/3436436223 conn(0x7fab9406abe0 msgr2=0x7fab9406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.773+0000 7fab99537640 1 -- 172.21.15.83:0/3436436223 shutdown_connections 2024-04-17T10:43:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.773+0000 7fab99537640 1 -- 172.21.15.83:0/3436436223 wait complete. 2024-04-17T10:43:21.786 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:43:21.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.897+0000 7f1ab17f8640 1 Processor -- start 2024-04-17T10:43:21.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.897+0000 7f1ab17f8640 1 -- start start 2024-04-17T10:43:21.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.897+0000 7f1ab17f8640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1aac0ce980 0x7f1aac0b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:21.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.897+0000 7f1ab17f8640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aac162a00 con 0x7f1aac15dfa0 2024-04-17T10:43:21.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.897+0000 7f1ab17f8640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aac161e80 con 0x7f1aac0cee00 2024-04-17T10:43:21.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.897+0000 7f1ab17f8640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aac162160 con 0x7f1aac0ce980 2024-04-17T10:43:21.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.897+0000 7f1aaa7fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1aac0ce980 0x7f1aac0b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:21.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.897+0000 7f1aaa7fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1aac0ce980 0x7f1aac0b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51902/0 (socket says 172.21.15.83:51902) 2024-04-17T10:43:21.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.897+0000 7f1aaa7fc640 1 -- 172.21.15.83:0/741605355 learned_addr learned my addr 172.21.15.83:0/741605355 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:21.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.897+0000 7f1aaa7fc640 1 -- 172.21.15.83:0/741605355 >> v1:172.21.15.167:6789/0 conn(0x7f1aac0cee00 legacy=0x7f1aac0d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:21.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.897+0000 7f1aaa7fc640 1 -- 172.21.15.83:0/741605355 >> v1:172.21.15.83:6789/0 conn(0x7f1aac15dfa0 legacy=0x7f1aac0b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:21.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.897+0000 7f1aaa7fc640 1 -- 172.21.15.83:0/741605355 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aac162440 con 0x7f1aac0ce980 2024-04-17T10:43:21.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.901+0000 7f1aaa7fc640 1 --2- 172.21.15.83:0/741605355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1aac0ce980 0x7f1aac0b5430 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009870 tx=0x7f1aa001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d89f580a284676e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:21.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.901+0000 7f1aa9ffb640 1 -- 172.21.15.83:0/741605355 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa00b0070 con 0x7f1aac0ce980 2024-04-17T10:43:21.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.901+0000 7f1aa9ffb640 1 -- 172.21.15.83:0/741605355 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa00ab600 con 0x7f1aac0ce980 2024-04-17T10:43:21.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.901+0000 7f1aa9ffb640 1 -- 172.21.15.83:0/741605355 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa00ab9d0 con 0x7f1aac0ce980 2024-04-17T10:43:21.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.901+0000 7f1ab17f8640 1 -- 172.21.15.83:0/741605355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1aac0ce980 msgr2=0x7f1aac0b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:21.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.901+0000 7f1ab17f8640 1 --2- 172.21.15.83:0/741605355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1aac0ce980 0x7f1aac0b5430 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009870 tx=0x7f1aa001c460 comp rx=0 tx=0).stop 2024-04-17T10:43:21.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.901+0000 7f1ab17f8640 1 -- 172.21.15.83:0/741605355 shutdown_connections 2024-04-17T10:43:21.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.901+0000 7f1ab17f8640 1 --2- 172.21.15.83:0/741605355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1aac0ce980 0x7f1aac0b5430 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:21.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.901+0000 7f1ab17f8640 1 -- 172.21.15.83:0/741605355 >> 172.21.15.83:0/741605355 conn(0x7f1aac0d0600 msgr2=0x7f1aac0d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:21.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.901+0000 7f1ab17f8640 1 -- 172.21.15.83:0/741605355 shutdown_connections 2024-04-17T10:43:21.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.901+0000 7f1ab17f8640 1 -- 172.21.15.83:0/741605355 wait complete. 2024-04-17T10:43:21.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.901+0000 7f1ab17f8640 1 Processor -- start 2024-04-17T10:43:21.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.901+0000 7f1ab17f8640 1 -- start start 2024-04-17T10:43:21.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.901+0000 7f1ab17f8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1aac0ce980 0x7f1aac157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:21.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.901+0000 7f1ab17f8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1aac0cee00 0x7f1aac0c69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:21.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.901+0000 7f1ab17f8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1aac15dfa0 0x7f1aac15e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:21.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.901+0000 7f1ab17f8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac16c670 con 0x7f1aac15dfa0 2024-04-17T10:43:21.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.901+0000 7f1ab17f8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac161b10 con 0x7f1aac0cee00 2024-04-17T10:43:21.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.901+0000 7f1ab17f8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac16c7f0 con 0x7f1aac0ce980 2024-04-17T10:43:21.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.901+0000 7f1aab7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1aac15dfa0 0x7f1aac15e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:21.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.901+0000 7f1aab7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1aac15dfa0 0x7f1aac15e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51916/0 (socket says 172.21.15.83:51916) 2024-04-17T10:43:21.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.901+0000 7f1aab7fe640 1 -- 172.21.15.83:0/3640256122 learned_addr learned my addr 172.21.15.83:0/3640256122 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:21.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.901+0000 7f1aaa7fc640 1 --2- 172.21.15.83:0/3640256122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1aac0cee00 0x7f1aac0c69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:21.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.905+0000 7f1aaaffd640 1 --2- 172.21.15.83:0/3640256122 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1aac0ce980 0x7f1aac157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:21.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.905+0000 7f1aab7fe640 1 -- 172.21.15.83:0/3640256122 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1aac0ce980 msgr2=0x7f1aac157590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:21.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.905+0000 7f1aab7fe640 1 --2- 172.21.15.83:0/3640256122 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1aac0ce980 0x7f1aac157590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:21.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.905+0000 7f1aab7fe640 1 -- 172.21.15.83:0/3640256122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1aac0cee00 msgr2=0x7f1aac0c69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:21.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.905+0000 7f1aab7fe640 1 --2- 172.21.15.83:0/3640256122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1aac0cee00 0x7f1aac0c69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:21.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.905+0000 7f1aab7fe640 1 -- 172.21.15.83:0/3640256122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aac0d4880 con 0x7f1aac15dfa0 2024-04-17T10:43:21.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.905+0000 7f1aab7fe640 1 --2- 172.21.15.83:0/3640256122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1aac15dfa0 0x7f1aac15e610 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f1aa400c780 tx=0x7f1aa400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:21.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.905+0000 7f1a937fe640 1 -- 172.21.15.83:0/3640256122 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4004b70 con 0x7f1aac15dfa0 2024-04-17T10:43:21.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.905+0000 7f1a937fe640 1 -- 172.21.15.83:0/3640256122 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa4004d10 con 0x7f1aac15dfa0 2024-04-17T10:43:21.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.905+0000 7f1a937fe640 1 -- 172.21.15.83:0/3640256122 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa400aa90 con 0x7f1aac15dfa0 2024-04-17T10:43:21.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.905+0000 7f1ab17f8640 1 -- 172.21.15.83:0/3640256122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aac155990 con 0x7f1aac15dfa0 2024-04-17T10:43:21.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.905+0000 7f1ab17f8640 1 -- 172.21.15.83:0/3640256122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aac0cbc70 con 0x7f1aac15dfa0 2024-04-17T10:43:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.905+0000 7f1ab17f8640 1 -- 172.21.15.83:0/3640256122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aac0681a0 con 0x7f1aac15dfa0 2024-04-17T10:43:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.905+0000 7f1a937fe640 1 -- 172.21.15.83:0/3640256122 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1aa400ac30 con 0x7f1aac15dfa0 2024-04-17T10:43:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.905+0000 7f1a937fe640 1 --2- 172.21.15.83:0/3640256122 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1a98066f60 0x7f1a98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.905+0000 7f1aaaffd640 1 --2- 172.21.15.83:0/3640256122 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1a98066f60 0x7f1a98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:21.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.905+0000 7f1a937fe640 1 -- 172.21.15.83:0/3640256122 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1aa4085280 con 0x7f1aac15dfa0 2024-04-17T10:43:21.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.905+0000 7f1aaaffd640 1 --2- 172.21.15.83:0/3640256122 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1a98066f60 0x7f1a98069420 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c007fd0 tx=0x7f1a9c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:21.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:21.909+0000 7f1a937fe640 1 -- 172.21.15.83:0/3640256122 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1aa4020070 con 0x7f1aac15dfa0 2024-04-17T10:43:22.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.065+0000 7f1ab17f8640 1 -- 172.21.15.83:0/3640256122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1aac074f90 con 0x7f1aac15dfa0 2024-04-17T10:43:22.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.069+0000 7f1a937fe640 1 -- 172.21.15.83:0/3640256122 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16722 (secure 0 0 0) 0x7f1aa40562c0 con 0x7f1aac15dfa0 2024-04-17T10:43:22.065 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:43:22.065 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":63,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:42:59.360496+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:42:32.530700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-17T10:42:33.334905+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.167:6833/271994303":"2024-04-18T10:42:59.346779+0000","172.21.15.167:6832/271994303":"2024-04-18T10:42:59.346779+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:43:22.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.069+0000 7f1ab17f8640 1 -- 172.21.15.83:0/3640256122 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1a98066f60 msgr2=0x7f1a98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:22.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.069+0000 7f1ab17f8640 1 --2- 172.21.15.83:0/3640256122 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1a98066f60 0x7f1a98069420 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c007fd0 tx=0x7f1a9c009040 comp rx=0 tx=0).stop 2024-04-17T10:43:22.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.069+0000 7f1ab17f8640 1 -- 172.21.15.83:0/3640256122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1aac15dfa0 msgr2=0x7f1aac15e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:22.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.069+0000 7f1ab17f8640 1 --2- 172.21.15.83:0/3640256122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1aac15dfa0 0x7f1aac15e610 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f1aa400c780 tx=0x7f1aa400cc40 comp rx=0 tx=0).stop 2024-04-17T10:43:22.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.069+0000 7f1ab17f8640 1 -- 172.21.15.83:0/3640256122 shutdown_connections 2024-04-17T10:43:22.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.069+0000 7f1ab17f8640 1 --2- 172.21.15.83:0/3640256122 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1a98066f60 0x7f1a98069420 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:22.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.069+0000 7f1ab17f8640 1 --2- 172.21.15.83:0/3640256122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1aac15dfa0 0x7f1aac15e610 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:22.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.069+0000 7f1ab17f8640 1 --2- 172.21.15.83:0/3640256122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1aac0cee00 0x7f1aac0c69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:22.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.069+0000 7f1ab17f8640 1 --2- 172.21.15.83:0/3640256122 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1aac0ce980 0x7f1aac157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:22.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.069+0000 7f1ab17f8640 1 -- 172.21.15.83:0/3640256122 >> 172.21.15.83:0/3640256122 conn(0x7f1aac0d0600 msgr2=0x7f1aac0d39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:22.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.069+0000 7f1ab17f8640 1 -- 172.21.15.83:0/3640256122 shutdown_connections 2024-04-17T10:43:22.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.073+0000 7f1ab17f8640 1 -- 172.21.15.83:0/3640256122 wait complete. 2024-04-17T10:43:22.083 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-17T10:43:22.083 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:43:22.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.193+0000 7fcb5987e640 1 Processor -- start 2024-04-17T10:43:22.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.193+0000 7fcb5987e640 1 -- start start 2024-04-17T10:43:22.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.197+0000 7fcb5987e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb540b2de0 0x7fcb5415ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:22.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.197+0000 7fcb5987e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb5415f5b0 con 0x7fcb5415f1d0 2024-04-17T10:43:22.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.197+0000 7fcb5987e640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb540b2620 con 0x7fcb540d22d0 2024-04-17T10:43:22.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.197+0000 7fcb5987e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb540b2340 con 0x7fcb540b2de0 2024-04-17T10:43:22.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.197+0000 7fcb52ffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcb5415f1d0 0x7fcb5415f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52244/0 (socket says 172.21.15.83:52244) 2024-04-17T10:43:22.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.197+0000 7fcb52ffd640 1 -- 172.21.15.83:0/1848981582 learned_addr learned my addr 172.21.15.83:0/1848981582 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:43:22.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.197+0000 7fcb51ffb640 1 -- 172.21.15.83:0/1848981582 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2339093758 0 0) 0x7fcb5415f5b0 con 0x7fcb5415f1d0 2024-04-17T10:43:22.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.197+0000 7fcb51ffb640 1 -- 172.21.15.83:0/1848981582 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb3c003530 con 0x7fcb5415f1d0 2024-04-17T10:43:22.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.197+0000 7fcb527fc640 1 --2- 172.21.15.83:0/1848981582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb540b2de0 0x7fcb5415ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:22.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.197+0000 7fcb51ffb640 1 -- 172.21.15.83:0/1848981582 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1484148746 0 0) 0x7fcb3c003530 con 0x7fcb5415f1d0 2024-04-17T10:43:22.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.197+0000 7fcb51ffb640 1 -- 172.21.15.83:0/1848981582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb540b2de0 msgr2=0x7fcb5415ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:22.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.197+0000 7fcb51ffb640 1 --2- 172.21.15.83:0/1848981582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb540b2de0 0x7fcb5415ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:22.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.197+0000 7fcb51ffb640 1 -- 172.21.15.83:0/1848981582 >> v1:172.21.15.167:6789/0 conn(0x7fcb540d22d0 legacy=0x7fcb540ae520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:22.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.197+0000 7fcb51ffb640 1 -- 172.21.15.83:0/1848981582 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb540b1760 con 0x7fcb5415f1d0 2024-04-17T10:43:22.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.197+0000 7fcb51ffb640 1 -- 172.21.15.83:0/1848981582 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fcb44001680 con 0x7fcb5415f1d0 2024-04-17T10:43:22.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.197+0000 7fcb51ffb640 1 -- 172.21.15.83:0/1848981582 >> v1:172.21.15.83:6789/0 conn(0x7fcb5415f1d0 legacy=0x7fcb5415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:22.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.197+0000 7fcb51ffb640 1 --2- 172.21.15.83:0/1848981582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb3c0039f0 0x7fcb3c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:22.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.197+0000 7fcb51ffb640 1 --2- 172.21.15.83:0/1848981582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb3c004810 0x7fcb3c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:22.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.197+0000 7fcb51ffb640 1 --2- 172.21.15.83:0/1848981582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb3c005190 0x7fcb3c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:22.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.197+0000 7fcb51ffb640 1 -- 172.21.15.83:0/1848981582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb540b2340 con 0x7fcb3c005190 2024-04-17T10:43:22.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.197+0000 7fcb51ffb640 1 -- 172.21.15.83:0/1848981582 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c00bf40 con 0x7fcb3c0039f0 2024-04-17T10:43:22.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.197+0000 7fcb51ffb640 1 -- 172.21.15.83:0/1848981582 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c00c140 con 0x7fcb3c004810 2024-04-17T10:43:22.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.197+0000 7fcb537fe640 1 --2- 172.21.15.83:0/1848981582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb3c005190 0x7fcb3c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:22.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.197+0000 7fcb52ffd640 1 --2- 172.21.15.83:0/1848981582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb3c0039f0 0x7fcb3c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:22.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.197+0000 7fcb537fe640 1 -- 172.21.15.83:0/1848981582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb3c004810 msgr2=0x7fcb3c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:22.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.197+0000 7fcb527fc640 1 --2- 172.21.15.83:0/1848981582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb3c004810 0x7fcb3c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:22.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.197+0000 7fcb537fe640 1 --2- 172.21.15.83:0/1848981582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb3c004810 0x7fcb3c004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:22.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.197+0000 7fcb537fe640 1 -- 172.21.15.83:0/1848981582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb3c0039f0 msgr2=0x7fcb3c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:22.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.197+0000 7fcb537fe640 1 --2- 172.21.15.83:0/1848981582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb3c0039f0 0x7fcb3c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:22.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.197+0000 7fcb537fe640 1 -- 172.21.15.83:0/1848981582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb3c003530 con 0x7fcb3c005190 2024-04-17T10:43:22.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.197+0000 7fcb537fe640 1 --2- 172.21.15.83:0/1848981582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb3c005190 0x7fcb3c00b9d0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c001e40 tx=0x7fcb4c01c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f9d9bf8aa79fad server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:22.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb51ffb640 1 -- 172.21.15.83:0/1848981582 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c001a90 con 0x7fcb3c005190 2024-04-17T10:43:22.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb51ffb640 1 -- 172.21.15.83:0/1848981582 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4c003b80 con 0x7fcb3c005190 2024-04-17T10:43:22.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb51ffb640 1 -- 172.21.15.83:0/1848981582 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c005080 con 0x7fcb3c005190 2024-04-17T10:43:22.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb5987e640 1 -- 172.21.15.83:0/1848981582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb3c005190 msgr2=0x7fcb3c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:22.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb5987e640 1 --2- 172.21.15.83:0/1848981582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb3c005190 0x7fcb3c00b9d0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c001e40 tx=0x7fcb4c01c760 comp rx=0 tx=0).stop 2024-04-17T10:43:22.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb537fe640 1 -- 172.21.15.83:0/1848981582 reap_dead start 2024-04-17T10:43:22.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb5987e640 1 -- 172.21.15.83:0/1848981582 shutdown_connections 2024-04-17T10:43:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb5987e640 1 -- 172.21.15.83:0/1848981582 >> 172.21.15.83:0/1848981582 conn(0x7fcb540d17b0 msgr2=0x7fcb540d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb5987e640 1 -- 172.21.15.83:0/1848981582 shutdown_connections 2024-04-17T10:43:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb5987e640 1 -- 172.21.15.83:0/1848981582 wait complete. 2024-04-17T10:43:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb5987e640 1 Processor -- start 2024-04-17T10:43:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb5987e640 1 -- start start 2024-04-17T10:43:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb5987e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb5414fcb0 0x7fcb540c6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb5987e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb540c6960 0x7fcb540cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb5987e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb540cd720 0x7fcb540c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb5987e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5415ff40 con 0x7fcb5414fcb0 2024-04-17T10:43:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb5987e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb540b1d20 con 0x7fcb540c6960 2024-04-17T10:43:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb5987e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb541600c0 con 0x7fcb540cd720 2024-04-17T10:43:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb52ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb5414fcb0 0x7fcb540c6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb52ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb5414fcb0 0x7fcb540c6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51928/0 (socket says 172.21.15.83:51928) 2024-04-17T10:43:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb52ffd640 1 -- 172.21.15.83:0/1477755892 learned_addr learned my addr 172.21.15.83:0/1477755892 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:22.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb527fc640 1 --2- 172.21.15.83:0/1477755892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb540c6960 0x7fcb540cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:22.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb537fe640 1 --2- 172.21.15.83:0/1477755892 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb540cd720 0x7fcb540c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:22.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb52ffd640 1 -- 172.21.15.83:0/1477755892 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb540cd720 msgr2=0x7fcb540c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:22.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb52ffd640 1 --2- 172.21.15.83:0/1477755892 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb540cd720 0x7fcb540c4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:22.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb52ffd640 1 -- 172.21.15.83:0/1477755892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb540c6960 msgr2=0x7fcb540cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:22.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb52ffd640 1 --2- 172.21.15.83:0/1477755892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb540c6960 0x7fcb540cd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:22.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb52ffd640 1 -- 172.21.15.83:0/1477755892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb54150090 con 0x7fcb5414fcb0 2024-04-17T10:43:22.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb52ffd640 1 --2- 172.21.15.83:0/1477755892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb5414fcb0 0x7fcb540c6420 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400eaf0 tx=0x7fcb44001720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:22.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb3bfff640 1 -- 172.21.15.83:0/1477755892 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4400c8d0 con 0x7fcb5414fcb0 2024-04-17T10:43:22.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb3bfff640 1 -- 172.21.15.83:0/1477755892 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb44018070 con 0x7fcb5414fcb0 2024-04-17T10:43:22.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb5987e640 1 -- 172.21.15.83:0/1477755892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb540c6e70 con 0x7fcb5414fcb0 2024-04-17T10:43:22.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb3bfff640 1 -- 172.21.15.83:0/1477755892 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4400ccb0 con 0x7fcb5414fcb0 2024-04-17T10:43:22.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.201+0000 7fcb5987e640 1 -- 172.21.15.83:0/1477755892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb540c7430 con 0x7fcb5414fcb0 2024-04-17T10:43:22.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.205+0000 7fcb5987e640 1 -- 172.21.15.83:0/1477755892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb54074f90 con 0x7fcb5414fcb0 2024-04-17T10:43:22.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.205+0000 7fcb3bfff640 1 -- 172.21.15.83:0/1477755892 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcb4401d460 con 0x7fcb5414fcb0 2024-04-17T10:43:22.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.205+0000 7fcb3bfff640 1 --2- 172.21.15.83:0/1477755892 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcb40066f90 0x7fcb40069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:22.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.205+0000 7fcb3bfff640 1 -- 172.21.15.83:0/1477755892 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fcb44085090 con 0x7fcb5414fcb0 2024-04-17T10:43:22.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.205+0000 7fcb527fc640 1 --2- 172.21.15.83:0/1477755892 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcb40066f90 0x7fcb40069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:22.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.205+0000 7fcb527fc640 1 --2- 172.21.15.83:0/1477755892 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcb40066f90 0x7fcb40069450 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcb540d0080 tx=0x7fcb48008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:22.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.209+0000 7fcb3bfff640 1 -- 172.21.15.83:0/1477755892 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcb54150090 con 0x7fcb5414fcb0 2024-04-17T10:43:22.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.393+0000 7fcb5987e640 1 -- 172.21.15.83:0/1477755892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb540b7610 con 0x7fcb5414fcb0 2024-04-17T10:43:22.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.393+0000 7fcb3bfff640 1 -- 172.21.15.83:0/1477755892 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4942 (secure 0 0 0) 0x7fcb44055fa0 con 0x7fcb5414fcb0 2024-04-17T10:43:22.389 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:43:22.389 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":39,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5165,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1924992370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1924992370},{"type":"v1","addr":"172.21.15.167:6833","nonce":1924992370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5174,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3876293847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3876293847},{"type":"v1","addr":"172.21.15.167:6835","nonce":3876293847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6831,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/338926944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":338926944},{"type":"v1","addr":"172.21.15.83:6835","nonce":338926944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:42:34.328397+0000","modified":"2024-04-17T10:43:02.389233+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6843},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6843":{"gid":6843,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6837/3985646487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3985646487},{"type":"v1","addr":"172.21.15.83:6837","nonce":3985646487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-17T10:43:22.389 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 39 2024-04-17T10:43:22.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.393+0000 7fcb5987e640 1 -- 172.21.15.83:0/1477755892 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcb40066f90 msgr2=0x7fcb40069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:22.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.393+0000 7fcb5987e640 1 --2- 172.21.15.83:0/1477755892 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcb40066f90 0x7fcb40069450 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcb540d0080 tx=0x7fcb48008040 comp rx=0 tx=0).stop 2024-04-17T10:43:22.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.393+0000 7fcb5987e640 1 -- 172.21.15.83:0/1477755892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb5414fcb0 msgr2=0x7fcb540c6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:22.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.393+0000 7fcb5987e640 1 --2- 172.21.15.83:0/1477755892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb5414fcb0 0x7fcb540c6420 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400eaf0 tx=0x7fcb44001720 comp rx=0 tx=0).stop 2024-04-17T10:43:22.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.393+0000 7fcb5987e640 1 -- 172.21.15.83:0/1477755892 shutdown_connections 2024-04-17T10:43:22.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.393+0000 7fcb5987e640 1 --2- 172.21.15.83:0/1477755892 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcb40066f90 0x7fcb40069450 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:22.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.393+0000 7fcb5987e640 1 --2- 172.21.15.83:0/1477755892 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb540cd720 0x7fcb540c4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:22.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.393+0000 7fcb5987e640 1 --2- 172.21.15.83:0/1477755892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb540c6960 0x7fcb540cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:22.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.393+0000 7fcb5987e640 1 --2- 172.21.15.83:0/1477755892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb5414fcb0 0x7fcb540c6420 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:22.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.393+0000 7fcb5987e640 1 -- 172.21.15.83:0/1477755892 >> 172.21.15.83:0/1477755892 conn(0x7fcb540d17b0 msgr2=0x7fcb5415dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:22.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.393+0000 7fcb5987e640 1 -- 172.21.15.83:0/1477755892 shutdown_connections 2024-04-17T10:43:22.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.397+0000 7fcb5987e640 1 -- 172.21.15.83:0/1477755892 wait complete. 2024-04-17T10:43:22.407 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:43:22.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.529+0000 7f5b20a72640 1 Processor -- start 2024-04-17T10:43:22.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.529+0000 7f5b20a72640 1 -- start start 2024-04-17T10:43:22.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.533+0000 7f5b20a72640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5b1c15f1d0 0x7f5b1c15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:22.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.533+0000 7f5b20a72640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5b1c0d22d0 0x7f5b1c0ae520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:22.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.533+0000 7f5b20a72640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b1c15f5b0 con 0x7f5b1c0b2de0 2024-04-17T10:43:22.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.533+0000 7f5b20a72640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b1c0b2620 con 0x7f5b1c15f1d0 2024-04-17T10:43:22.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.533+0000 7f5b20a72640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b1c0b2340 con 0x7f5b1c0d22d0 2024-04-17T10:43:22.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.533+0000 7f5b1a59c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5b1c15f1d0 0x7f5b1c15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:22.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.533+0000 7f5b1a59c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5b1c15f1d0 0x7f5b1c15f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51942/0 (socket says 172.21.15.83:51942) 2024-04-17T10:43:22.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.533+0000 7f5b1a59c640 1 -- 172.21.15.83:0/2727227880 learned_addr learned my addr 172.21.15.83:0/2727227880 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:22.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.533+0000 7f5b1ad9d640 1 --2- 172.21.15.83:0/2727227880 >> v2:172.21.15.167:3300/0 conn(0x7f5b1c0d22d0 0x7f5b1c0ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:22.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.533+0000 7f5b1a59c640 1 -- 172.21.15.83:0/2727227880 >> v2:172.21.15.167:3300/0 conn(0x7f5b1c0d22d0 msgr2=0x7f5b1c0ae520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:22.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.533+0000 7f5b1a59c640 1 --2- 172.21.15.83:0/2727227880 >> v2:172.21.15.167:3300/0 conn(0x7f5b1c0d22d0 0x7f5b1c0ae520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:22.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.533+0000 7f5b1a59c640 1 -- 172.21.15.83:0/2727227880 >> v1:172.21.15.83:6789/0 conn(0x7f5b1c0b2de0 legacy=0x7f5b1c15fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:22.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.533+0000 7f5b1a59c640 1 -- 172.21.15.83:0/2727227880 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b1c0b1760 con 0x7f5b1c15f1d0 2024-04-17T10:43:22.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.533+0000 7f5b1ad9d640 1 --2- 172.21.15.83:0/2727227880 >> v2:172.21.15.167:3300/0 conn(0x7f5b1c0d22d0 0x7f5b1c0ae520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:22.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.533+0000 7f5b1a59c640 1 --2- 172.21.15.83:0/2727227880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b1c15f1d0 0x7f5b1c15f830 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c1608d0 tx=0x7f5b0c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65be31dbbcddcc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:22.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.533+0000 7f5b19d9b640 1 -- 172.21.15.83:0/2727227880 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c0aa650 con 0x7f5b1c15f1d0 2024-04-17T10:43:22.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.533+0000 7f5b19d9b640 1 -- 172.21.15.83:0/2727227880 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0c0af070 con 0x7f5b1c15f1d0 2024-04-17T10:43:22.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.533+0000 7f5b19d9b640 1 -- 172.21.15.83:0/2727227880 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c0aaa10 con 0x7f5b1c15f1d0 2024-04-17T10:43:22.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.533+0000 7f5b20a72640 1 -- 172.21.15.83:0/2727227880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b1c15f1d0 msgr2=0x7f5b1c15f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:22.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.533+0000 7f5b20a72640 1 --2- 172.21.15.83:0/2727227880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b1c15f1d0 0x7f5b1c15f830 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c1608d0 tx=0x7f5b0c01c4f0 comp rx=0 tx=0).stop 2024-04-17T10:43:22.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.537+0000 7f5b20a72640 1 -- 172.21.15.83:0/2727227880 shutdown_connections 2024-04-17T10:43:22.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.537+0000 7f5b20a72640 1 --2- 172.21.15.83:0/2727227880 >> v2:172.21.15.167:3300/0 conn(0x7f5b1c0d22d0 0x7f5b1c0ae520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:22.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.537+0000 7f5b20a72640 1 --2- 172.21.15.83:0/2727227880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b1c15f1d0 0x7f5b1c15f830 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:22.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.537+0000 7f5b20a72640 1 -- 172.21.15.83:0/2727227880 >> 172.21.15.83:0/2727227880 conn(0x7f5b1c0d17b0 msgr2=0x7f5b1c0d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:22.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.537+0000 7f5b20a72640 1 -- 172.21.15.83:0/2727227880 shutdown_connections 2024-04-17T10:43:22.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.537+0000 7f5b20a72640 1 -- 172.21.15.83:0/2727227880 wait complete. 2024-04-17T10:43:22.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.537+0000 7f5b20a72640 1 Processor -- start 2024-04-17T10:43:22.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.537+0000 7f5b20a72640 1 -- start start 2024-04-17T10:43:22.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.537+0000 7f5b20a72640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b1c0d22d0 0x7f5b1c0c5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:22.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.537+0000 7f5b20a72640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b1c15f1d0 0x7f5b1c0c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:22.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.537+0000 7f5b20a72640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b1c1f7310 0x7f5b1c152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:22.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.537+0000 7f5b20a72640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c0af270 con 0x7f5b1c1f7310 2024-04-17T10:43:22.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.537+0000 7f5b20a72640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c160690 con 0x7f5b1c0d22d0 2024-04-17T10:43:22.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.537+0000 7f5b20a72640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c0af3f0 con 0x7f5b1c15f1d0 2024-04-17T10:43:22.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.537+0000 7f5b1ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b1c1f7310 0x7f5b1c152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:22.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.537+0000 7f5b1ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b1c1f7310 0x7f5b1c152810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51952/0 (socket says 172.21.15.83:51952) 2024-04-17T10:43:22.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.537+0000 7f5b1ad9d640 1 -- 172.21.15.83:0/2027774603 learned_addr learned my addr 172.21.15.83:0/2027774603 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:22.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.537+0000 7f5b1a59c640 1 --2- 172.21.15.83:0/2027774603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b1c0d22d0 0x7f5b1c0c5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:22.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.537+0000 7f5b1ad9d640 1 -- 172.21.15.83:0/2027774603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b1c15f1d0 msgr2=0x7f5b1c0c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:22.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.537+0000 7f5b11d9b640 1 --2- 172.21.15.83:0/2027774603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b1c15f1d0 0x7f5b1c0c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:22.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.537+0000 7f5b1ad9d640 1 --2- 172.21.15.83:0/2027774603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b1c15f1d0 0x7f5b1c0c5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:22.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.537+0000 7f5b1ad9d640 1 -- 172.21.15.83:0/2027774603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b1c0d22d0 msgr2=0x7f5b1c0c5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:22.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.537+0000 7f5b1ad9d640 1 --2- 172.21.15.83:0/2027774603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b1c0d22d0 0x7f5b1c0c5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:22.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.537+0000 7f5b1ad9d640 1 -- 172.21.15.83:0/2027774603 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b1c153420 con 0x7f5b1c1f7310 2024-04-17T10:43:22.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.537+0000 7f5b1ad9d640 1 --2- 172.21.15.83:0/2027774603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b1c1f7310 0x7f5b1c152810 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f5b1400cef0 tx=0x7f5b1400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:22.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.537+0000 7f5b137fe640 1 -- 172.21.15.83:0/2027774603 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1401a070 con 0x7f5b1c1f7310 2024-04-17T10:43:22.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.537+0000 7f5b137fe640 1 -- 172.21.15.83:0/2027774603 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1400d860 con 0x7f5b1c1f7310 2024-04-17T10:43:22.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.537+0000 7f5b137fe640 1 -- 172.21.15.83:0/2027774603 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1400dc50 con 0x7f5b1c1f7310 2024-04-17T10:43:22.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.537+0000 7f5b20a72640 1 -- 172.21.15.83:0/2027774603 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b1c153700 con 0x7f5b1c1f7310 2024-04-17T10:43:22.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.537+0000 7f5b20a72640 1 -- 172.21.15.83:0/2027774603 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b1c0cca30 con 0x7f5b1c1f7310 2024-04-17T10:43:22.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.537+0000 7f5b20a72640 1 -- 172.21.15.83:0/2027774603 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b1c0ccc30 con 0x7f5b1c1f7310 2024-04-17T10:43:22.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.541+0000 7f5b137fe640 1 -- 172.21.15.83:0/2027774603 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5b1401d070 con 0x7f5b1c1f7310 2024-04-17T10:43:22.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.541+0000 7f5b137fe640 1 --2- 172.21.15.83:0/2027774603 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5afc066f60 0x7f5afc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:22.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.541+0000 7f5b137fe640 1 -- 172.21.15.83:0/2027774603 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f5b14085830 con 0x7f5b1c1f7310 2024-04-17T10:43:22.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.541+0000 7f5b1a59c640 1 --2- 172.21.15.83:0/2027774603 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5afc066f60 0x7f5afc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:22.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.541+0000 7f5b1a59c640 1 --2- 172.21.15.83:0/2027774603 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5afc066f60 0x7f5afc069420 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c0ae000 tx=0x7f5b0c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:22.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.545+0000 7f5b137fe640 1 -- 172.21.15.83:0/2027774603 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5b1c0ccc30 con 0x7f5b1c1f7310 2024-04-17T10:43:22.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.721+0000 7f5b20a72640 1 -- 172.21.15.83:0/2027774603 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b1c074f90 con 0x7f5b1c1f7310 2024-04-17T10:43:22.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.721+0000 7f5b137fe640 1 -- 172.21.15.83:0/2027774603 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f5b14056930 con 0x7f5b1c1f7310 2024-04-17T10:43:22.718 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:43:22.718 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":39,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5165,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1924992370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1924992370},{"type":"v1","addr":"172.21.15.167:6833","nonce":1924992370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5174,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3876293847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3876293847},{"type":"v1","addr":"172.21.15.167:6835","nonce":3876293847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6831,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/338926944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":338926944},{"type":"v1","addr":"172.21.15.83:6835","nonce":338926944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:42:34.328397+0000","modified":"2024-04-17T10:43:02.389233+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6843},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6843":{"gid":6843,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6837/3985646487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3985646487},{"type":"v1","addr":"172.21.15.83:6837","nonce":3985646487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-17T10:43:22.718 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 39 2024-04-17T10:43:22.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.725+0000 7f5b20a72640 1 -- 172.21.15.83:0/2027774603 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5afc066f60 msgr2=0x7f5afc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:22.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.725+0000 7f5b20a72640 1 --2- 172.21.15.83:0/2027774603 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5afc066f60 0x7f5afc069420 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c0ae000 tx=0x7f5b0c0a7000 comp rx=0 tx=0).stop 2024-04-17T10:43:22.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.725+0000 7f5b20a72640 1 -- 172.21.15.83:0/2027774603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b1c1f7310 msgr2=0x7f5b1c152810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:22.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.725+0000 7f5b20a72640 1 --2- 172.21.15.83:0/2027774603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b1c1f7310 0x7f5b1c152810 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f5b1400cef0 tx=0x7f5b1400a710 comp rx=0 tx=0).stop 2024-04-17T10:43:22.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.725+0000 7f5b20a72640 1 -- 172.21.15.83:0/2027774603 shutdown_connections 2024-04-17T10:43:22.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.725+0000 7f5b20a72640 1 --2- 172.21.15.83:0/2027774603 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5afc066f60 0x7f5afc069420 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:22.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.725+0000 7f5b20a72640 1 --2- 172.21.15.83:0/2027774603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b1c1f7310 0x7f5b1c152810 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:22.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.725+0000 7f5b20a72640 1 --2- 172.21.15.83:0/2027774603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b1c15f1d0 0x7f5b1c0c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:22.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.725+0000 7f5b20a72640 1 --2- 172.21.15.83:0/2027774603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b1c0d22d0 0x7f5b1c0c5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:22.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.725+0000 7f5b20a72640 1 -- 172.21.15.83:0/2027774603 >> 172.21.15.83:0/2027774603 conn(0x7f5b1c0d17b0 msgr2=0x7f5b1c157db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:22.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.725+0000 7f5b20a72640 1 -- 172.21.15.83:0/2027774603 shutdown_connections 2024-04-17T10:43:22.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.725+0000 7f5b20a72640 1 -- 172.21.15.83:0/2027774603 wait complete. 2024-04-17T10:43:22.736 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:43:22.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.849+0000 7f993eb60640 1 Processor -- start 2024-04-17T10:43:22.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.849+0000 7f993eb60640 1 -- start start 2024-04-17T10:43:22.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.853+0000 7f993eb60640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f993815db50 0x7f9938070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:22.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.853+0000 7f993eb60640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f993806dcb0 0x7f99380b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:22.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.853+0000 7f993eb60640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99380709b0 con 0x7f99380aedd0 2024-04-17T10:43:22.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.853+0000 7f993eb60640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f993815e440 con 0x7f993815db50 2024-04-17T10:43:22.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.853+0000 7f993eb60640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f993815e720 con 0x7f993806dcb0 2024-04-17T10:43:22.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.853+0000 7f993c8fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f993815db50 0x7f9938070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:22.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.853+0000 7f993c8fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f993815db50 0x7f9938070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51960/0 (socket says 172.21.15.83:51960) 2024-04-17T10:43:22.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.853+0000 7f993c8fc640 1 -- 172.21.15.83:0/4118276649 learned_addr learned my addr 172.21.15.83:0/4118276649 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:22.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.853+0000 7f993d0fd640 1 --2- 172.21.15.83:0/4118276649 >> v2:172.21.15.167:3300/0 conn(0x7f993806dcb0 0x7f99380b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:22.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.853+0000 7f993c8fc640 1 -- 172.21.15.83:0/4118276649 >> v2:172.21.15.167:3300/0 conn(0x7f993806dcb0 msgr2=0x7f99380b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:22.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.853+0000 7f993c8fc640 1 --2- 172.21.15.83:0/4118276649 >> v2:172.21.15.167:3300/0 conn(0x7f993806dcb0 0x7f99380b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:22.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.853+0000 7f993c8fc640 1 -- 172.21.15.83:0/4118276649 >> v1:172.21.15.83:6789/0 conn(0x7f99380aedd0 legacy=0x7f993806d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:22.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.853+0000 7f993c8fc640 1 -- 172.21.15.83:0/4118276649 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993815ea00 con 0x7f993815db50 2024-04-17T10:43:22.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.853+0000 7f993d0fd640 1 --2- 172.21.15.83:0/4118276649 >> v2:172.21.15.167:3300/0 conn(0x7f993806dcb0 0x7f99380b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:22.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.853+0000 7f993c8fc640 1 --2- 172.21.15.83:0/4118276649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f993815db50 0x7f9938070470 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f992800bb10 tx=0x7f992801ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f97fe0e68f13dc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:22.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.853+0000 7f99377fe640 1 -- 172.21.15.83:0/4118276649 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99280b4070 con 0x7f993815db50 2024-04-17T10:43:22.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.853+0000 7f99377fe640 1 -- 172.21.15.83:0/4118276649 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99280a7c90 con 0x7f993815db50 2024-04-17T10:43:22.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.853+0000 7f99377fe640 1 -- 172.21.15.83:0/4118276649 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99280af660 con 0x7f993815db50 2024-04-17T10:43:22.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.857+0000 7f993eb60640 1 -- 172.21.15.83:0/4118276649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f993815db50 msgr2=0x7f9938070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:22.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.857+0000 7f993eb60640 1 --2- 172.21.15.83:0/4118276649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f993815db50 0x7f9938070470 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f992800bb10 tx=0x7f992801ebc0 comp rx=0 tx=0).stop 2024-04-17T10:43:22.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.857+0000 7f993eb60640 1 -- 172.21.15.83:0/4118276649 shutdown_connections 2024-04-17T10:43:22.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.857+0000 7f993eb60640 1 --2- 172.21.15.83:0/4118276649 >> v2:172.21.15.167:3300/0 conn(0x7f993806dcb0 0x7f99380b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:22.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.857+0000 7f993eb60640 1 --2- 172.21.15.83:0/4118276649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f993815db50 0x7f9938070470 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:22.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.857+0000 7f993eb60640 1 -- 172.21.15.83:0/4118276649 >> 172.21.15.83:0/4118276649 conn(0x7f993806abe0 msgr2=0x7f993806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:22.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.857+0000 7f993eb60640 1 -- 172.21.15.83:0/4118276649 shutdown_connections 2024-04-17T10:43:22.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.857+0000 7f993eb60640 1 -- 172.21.15.83:0/4118276649 wait complete. 2024-04-17T10:43:22.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.857+0000 7f993eb60640 1 Processor -- start 2024-04-17T10:43:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.857+0000 7f993eb60640 1 -- start start 2024-04-17T10:43:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.857+0000 7f993eb60640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f993806dcb0 0x7f993814d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.857+0000 7f993eb60640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99380aedd0 0x7f9938155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.857+0000 7f993eb60640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f993815db50 0x7f9938156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.857+0000 7f993eb60640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99380b3ce0 con 0x7f993806dcb0 2024-04-17T10:43:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.857+0000 7f993eb60640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99380b3b60 con 0x7f993815db50 2024-04-17T10:43:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.857+0000 7f993eb60640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99380b3e60 con 0x7f99380aedd0 2024-04-17T10:43:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.857+0000 7f993c8fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f993806dcb0 0x7f993814d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.857+0000 7f993c8fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f993806dcb0 0x7f993814d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51964/0 (socket says 172.21.15.83:51964) 2024-04-17T10:43:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.857+0000 7f993c8fc640 1 -- 172.21.15.83:0/2267112975 learned_addr learned my addr 172.21.15.83:0/2267112975 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.857+0000 7f993c8fc640 1 -- 172.21.15.83:0/2267112975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99380aedd0 msgr2=0x7f9938155f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.857+0000 7f993c8fc640 1 --2- 172.21.15.83:0/2267112975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99380aedd0 0x7f9938155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:22.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.857+0000 7f993c8fc640 1 -- 172.21.15.83:0/2267112975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f993815db50 msgr2=0x7f9938156fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:22.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.857+0000 7f993c8fc640 1 --2- 172.21.15.83:0/2267112975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f993815db50 0x7f9938156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:22.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.857+0000 7f993c8fc640 1 -- 172.21.15.83:0/2267112975 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993815c410 con 0x7f993806dcb0 2024-04-17T10:43:22.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.857+0000 7f993c8fc640 1 --2- 172.21.15.83:0/2267112975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f993806dcb0 0x7f993814d500 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f992800b2c0 tx=0x7f992801b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:22.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.857+0000 7f99357fa640 1 -- 172.21.15.83:0/2267112975 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99280b4040 con 0x7f993806dcb0 2024-04-17T10:43:22.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.857+0000 7f99357fa640 1 -- 172.21.15.83:0/2267112975 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99280c1070 con 0x7f993806dcb0 2024-04-17T10:43:22.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.857+0000 7f99357fa640 1 -- 172.21.15.83:0/2267112975 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99280b9460 con 0x7f993806dcb0 2024-04-17T10:43:22.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.857+0000 7f993eb60640 1 -- 172.21.15.83:0/2267112975 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f993815c6f0 con 0x7f993806dcb0 2024-04-17T10:43:22.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.857+0000 7f993eb60640 1 -- 172.21.15.83:0/2267112975 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f993815ccb0 con 0x7f993806dcb0 2024-04-17T10:43:22.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.857+0000 7f993eb60640 1 -- 172.21.15.83:0/2267112975 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9938074f90 con 0x7f993806dcb0 2024-04-17T10:43:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.861+0000 7f99357fa640 1 -- 172.21.15.83:0/2267112975 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f99280b7070 con 0x7f993806dcb0 2024-04-17T10:43:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.861+0000 7f99357fa640 1 --2- 172.21.15.83:0/2267112975 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9924066f60 0x7f9924069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.861+0000 7f99357fa640 1 -- 172.21.15.83:0/2267112975 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9928122160 con 0x7f993806dcb0 2024-04-17T10:43:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.861+0000 7f9937fff640 1 --2- 172.21.15.83:0/2267112975 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9924066f60 0x7f9924069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.861+0000 7f9937fff640 1 --2- 172.21.15.83:0/2267112975 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9924066f60 0x7f9924069420 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f992c000f70 tx=0x7f992c0079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:22.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:22.865+0000 7f99357fa640 1 -- 172.21.15.83:0/2267112975 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f99280187e0 con 0x7f993806dcb0 2024-04-17T10:43:23.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.017+0000 7f993eb60640 1 -- 172.21.15.83:0/2267112975 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9938074480 con 0x7f993806dcb0 2024-04-17T10:43:23.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.021+0000 7f99357fa640 1 -- 172.21.15.83:0/2267112975 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16722 (secure 0 0 0) 0x7f99280ad340 con 0x7f993806dcb0 2024-04-17T10:43:23.018 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:43:23.018 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":63,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:42:59.360496+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:42:32.530700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-17T10:42:33.334905+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.167:6833/271994303":"2024-04-18T10:42:59.346779+0000","172.21.15.167:6832/271994303":"2024-04-18T10:42:59.346779+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:43:23.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.025+0000 7f993eb60640 1 -- 172.21.15.83:0/2267112975 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9924066f60 msgr2=0x7f9924069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:23.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.025+0000 7f993eb60640 1 --2- 172.21.15.83:0/2267112975 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9924066f60 0x7f9924069420 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f992c000f70 tx=0x7f992c0079c0 comp rx=0 tx=0).stop 2024-04-17T10:43:23.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.025+0000 7f993eb60640 1 -- 172.21.15.83:0/2267112975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f993806dcb0 msgr2=0x7f993814d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:23.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.025+0000 7f993eb60640 1 --2- 172.21.15.83:0/2267112975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f993806dcb0 0x7f993814d500 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f992800b2c0 tx=0x7f992801b960 comp rx=0 tx=0).stop 2024-04-17T10:43:23.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.025+0000 7f993eb60640 1 -- 172.21.15.83:0/2267112975 shutdown_connections 2024-04-17T10:43:23.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.025+0000 7f993eb60640 1 --2- 172.21.15.83:0/2267112975 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9924066f60 0x7f9924069420 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:23.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.025+0000 7f993eb60640 1 --2- 172.21.15.83:0/2267112975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f993815db50 0x7f9938156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:23.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.025+0000 7f993eb60640 1 --2- 172.21.15.83:0/2267112975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99380aedd0 0x7f9938155f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:23.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.025+0000 7f993eb60640 1 --2- 172.21.15.83:0/2267112975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f993806dcb0 0x7f993814d500 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:23.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.025+0000 7f993eb60640 1 -- 172.21.15.83:0/2267112975 >> 172.21.15.83:0/2267112975 conn(0x7f993806abe0 msgr2=0x7f99380b00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:23.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.025+0000 7f993eb60640 1 -- 172.21.15.83:0/2267112975 shutdown_connections 2024-04-17T10:43:23.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.025+0000 7f993eb60640 1 -- 172.21.15.83:0/2267112975 wait complete. 2024-04-17T10:43:23.038 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-17T10:43:23.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f482c8a5640 1 Processor -- start 2024-04-17T10:43:23.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f482c8a5640 1 -- start start 2024-04-17T10:43:23.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f482c8a5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48280cac30 0x7f48280ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:23.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f482c8a5640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48280cdbd0 con 0x7f482815eba0 2024-04-17T10:43:23.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f482c8a5640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48280cdeb0 con 0x7f482815da30 2024-04-17T10:43:23.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f482c8a5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f482806d2e0 con 0x7f48280cac30 2024-04-17T10:43:23.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f4825d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f482815eba0 0x7f48280ca520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52260/0 (socket says 172.21.15.83:52260) 2024-04-17T10:43:23.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f4825d9b640 1 -- 172.21.15.83:0/2931932920 learned_addr learned my addr 172.21.15.83:0/2931932920 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:43:23.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f4826d9d640 1 --2- 172.21.15.83:0/2931932920 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48280cac30 0x7f48280ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:23.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f482559a640 1 -- 172.21.15.83:0/2931932920 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 684931297 0 0) 0x7f48280cdbd0 con 0x7f482815eba0 2024-04-17T10:43:23.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f482559a640 1 -- 172.21.15.83:0/2931932920 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4814003530 con 0x7f482815eba0 2024-04-17T10:43:23.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f482559a640 1 -- 172.21.15.83:0/2931932920 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1229241090 0 0) 0x7f4814003530 con 0x7f482815eba0 2024-04-17T10:43:23.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f482559a640 1 -- 172.21.15.83:0/2931932920 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48280cac30 msgr2=0x7f48280ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:23.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f482559a640 1 --2- 172.21.15.83:0/2931932920 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48280cac30 0x7f48280ce990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:23.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f482559a640 1 -- 172.21.15.83:0/2931932920 >> v1:172.21.15.167:6789/0 conn(0x7f482815da30 legacy=0x7f482806cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:23.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f482559a640 1 -- 172.21.15.83:0/2931932920 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48280ce190 con 0x7f482815eba0 2024-04-17T10:43:23.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f482559a640 1 -- 172.21.15.83:0/2931932920 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f4810001680 con 0x7f482815eba0 2024-04-17T10:43:23.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f482559a640 1 -- 172.21.15.83:0/2931932920 >> v1:172.21.15.83:6789/0 conn(0x7f482815eba0 legacy=0x7f48280ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:23.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f482559a640 1 --2- 172.21.15.83:0/2931932920 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48280cac30 0x7f4814003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:23.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f482559a640 1 --2- 172.21.15.83:0/2931932920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4814004050 0x7f4814004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:23.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f482559a640 1 --2- 172.21.15.83:0/2931932920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4814004d50 0x7f48140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:23.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f482559a640 1 -- 172.21.15.83:0/2931932920 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482806d2e0 con 0x7f4814004050 2024-04-17T10:43:23.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f482559a640 1 -- 172.21.15.83:0/2931932920 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481400bbe0 con 0x7f4814004d50 2024-04-17T10:43:23.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f482559a640 1 -- 172.21.15.83:0/2931932920 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f481400bde0 con 0x7f48280cac30 2024-04-17T10:43:23.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f4825d9b640 1 --2- 172.21.15.83:0/2931932920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4814004050 0x7f4814004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:23.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f482659c640 1 --2- 172.21.15.83:0/2931932920 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48280cac30 0x7f4814003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:23.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f4826d9d640 1 --2- 172.21.15.83:0/2931932920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4814004d50 0x7f48140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:23.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f4825d9b640 1 -- 172.21.15.83:0/2931932920 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48280cac30 msgr2=0x7f4814003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:23.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f4825d9b640 1 --2- 172.21.15.83:0/2931932920 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48280cac30 0x7f4814003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:23.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f4825d9b640 1 -- 172.21.15.83:0/2931932920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4814004d50 msgr2=0x7f48140051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:23.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f4825d9b640 1 --2- 172.21.15.83:0/2931932920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4814004d50 0x7f48140051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:23.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f4825d9b640 1 -- 172.21.15.83:0/2931932920 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4814003530 con 0x7f4814004050 2024-04-17T10:43:23.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.161+0000 7f4826d9d640 1 --2- 172.21.15.83:0/2931932920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4814004d50 0x7f48140051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:43:23.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.165+0000 7f4825d9b640 1 --2- 172.21.15.83:0/2931932920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4814004050 0x7f4814004810 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f481000c240 tx=0x7f481001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cabc4e92f6bcda7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:23.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.165+0000 7f482559a640 1 -- 172.21.15.83:0/2931932920 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4810007d20 con 0x7f4814004050 2024-04-17T10:43:23.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.165+0000 7f482559a640 1 -- 172.21.15.83:0/2931932920 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48100a9020 con 0x7f4814004050 2024-04-17T10:43:23.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.165+0000 7f482559a640 1 -- 172.21.15.83:0/2931932920 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4810005ae0 con 0x7f4814004050 2024-04-17T10:43:23.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.165+0000 7f482c8a5640 1 -- 172.21.15.83:0/2931932920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4814004050 msgr2=0x7f4814004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:23.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.165+0000 7f482c8a5640 1 --2- 172.21.15.83:0/2931932920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4814004050 0x7f4814004810 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f481000c240 tx=0x7f481001cc60 comp rx=0 tx=0).stop 2024-04-17T10:43:23.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.165+0000 7f4826d9d640 1 -- 172.21.15.83:0/2931932920 reap_dead start 2024-04-17T10:43:23.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.165+0000 7f482c8a5640 1 -- 172.21.15.83:0/2931932920 shutdown_connections 2024-04-17T10:43:23.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.165+0000 7f482c8a5640 1 -- 172.21.15.83:0/2931932920 >> 172.21.15.83:0/2931932920 conn(0x7f482806abe0 msgr2=0x7f482806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:23.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.165+0000 7f482c8a5640 1 -- 172.21.15.83:0/2931932920 shutdown_connections 2024-04-17T10:43:23.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.165+0000 7f482c8a5640 1 -- 172.21.15.83:0/2931932920 wait complete. 2024-04-17T10:43:23.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.169+0000 7f482c8a5640 1 Processor -- start 2024-04-17T10:43:23.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.169+0000 7f482c8a5640 1 -- start start 2024-04-17T10:43:23.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.169+0000 7f482c8a5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4814004050 0x7f48281556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:23.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.169+0000 7f482c8a5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4828155e70 0x7f48280c65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:23.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.169+0000 7f482c8a5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48280c6af0 0x7f4828156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:23.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.169+0000 7f482c8a5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48280ce500 con 0x7f4814004050 2024-04-17T10:43:23.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.169+0000 7f482c8a5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48280cf3b0 con 0x7f4828155e70 2024-04-17T10:43:23.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.169+0000 7f482c8a5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4828067070 con 0x7f48280c6af0 2024-04-17T10:43:23.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.169+0000 7f482659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4814004050 0x7f48281556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:23.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.169+0000 7f482659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4814004050 0x7f48281556f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51984/0 (socket says 172.21.15.83:51984) 2024-04-17T10:43:23.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.169+0000 7f482659c640 1 -- 172.21.15.83:0/867601898 learned_addr learned my addr 172.21.15.83:0/867601898 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:23.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.169+0000 7f4825d9b640 1 --2- 172.21.15.83:0/867601898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4828155e70 0x7f48280c65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:23.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.169+0000 7f482659c640 1 -- 172.21.15.83:0/867601898 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48280c6af0 msgr2=0x7f4828156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:23.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.169+0000 7f482659c640 1 --2- 172.21.15.83:0/867601898 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48280c6af0 0x7f4828156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:23.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.169+0000 7f482659c640 1 -- 172.21.15.83:0/867601898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4828155e70 msgr2=0x7f48280c65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:23.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.169+0000 7f482659c640 1 --2- 172.21.15.83:0/867601898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4828155e70 0x7f48280c65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:23.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.169+0000 7f482659c640 1 -- 172.21.15.83:0/867601898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48280b3f80 con 0x7f4814004050 2024-04-17T10:43:23.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.169+0000 7f4825d9b640 1 --2- 172.21.15.83:0/867601898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4828155e70 0x7f48280c65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:23.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.169+0000 7f482659c640 1 --2- 172.21.15.83:0/867601898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4814004050 0x7f48281556f0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f482000e5d0 tx=0x7f482000ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:23.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.169+0000 7f481effd640 1 -- 172.21.15.83:0/867601898 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48200053a0 con 0x7f4814004050 2024-04-17T10:43:23.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.169+0000 7f481effd640 1 -- 172.21.15.83:0/867601898 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4820018070 con 0x7f4814004050 2024-04-17T10:43:23.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.169+0000 7f481effd640 1 -- 172.21.15.83:0/867601898 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4820003840 con 0x7f4814004050 2024-04-17T10:43:23.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.169+0000 7f482c8a5640 1 -- 172.21.15.83:0/867601898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48280b4260 con 0x7f4814004050 2024-04-17T10:43:23.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.169+0000 7f482c8a5640 1 -- 172.21.15.83:0/867601898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48280b4820 con 0x7f4814004050 2024-04-17T10:43:23.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.169+0000 7f482c8a5640 1 -- 172.21.15.83:0/867601898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4828074f90 con 0x7f4814004050 2024-04-17T10:43:23.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.169+0000 7f481effd640 1 -- 172.21.15.83:0/867601898 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f48200039e0 con 0x7f4814004050 2024-04-17T10:43:23.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.169+0000 7f481effd640 1 --2- 172.21.15.83:0/867601898 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4808066f60 0x7f4808069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:23.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.173+0000 7f4825d9b640 1 --2- 172.21.15.83:0/867601898 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4808066f60 0x7f4808069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:23.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.173+0000 7f481effd640 1 -- 172.21.15.83:0/867601898 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4820084ee0 con 0x7f4814004050 2024-04-17T10:43:23.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.173+0000 7f4825d9b640 1 --2- 172.21.15.83:0/867601898 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4808066f60 0x7f4808069420 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f48100aa000 tx=0x7f48280cdda0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:23.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.177+0000 7f481effd640 1 -- 172.21.15.83:0/867601898 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f482001a310 con 0x7f4814004050 2024-04-17T10:43:23.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.369+0000 7f482c8a5640 1 -- 172.21.15.83:0/867601898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f482806ceb0 con 0x7f4814004050 2024-04-17T10:43:23.367 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-04-17T10:43:23.369+0000 7f35c3b81640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T10:43:23.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.373+0000 7f481effd640 1 -- 172.21.15.83:0/867601898 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v40) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4820055e90 con 0x7f4814004050 2024-04-17T10:43:23.369 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-17T10:43:23.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.377+0000 7f482c8a5640 1 -- 172.21.15.83:0/867601898 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4808066f60 msgr2=0x7f4808069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:23.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.377+0000 7f482c8a5640 1 --2- 172.21.15.83:0/867601898 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4808066f60 0x7f4808069420 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f48100aa000 tx=0x7f48280cdda0 comp rx=0 tx=0).stop 2024-04-17T10:43:23.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.377+0000 7f482c8a5640 1 -- 172.21.15.83:0/867601898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4814004050 msgr2=0x7f48281556f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:23.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.377+0000 7f482c8a5640 1 --2- 172.21.15.83:0/867601898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4814004050 0x7f48281556f0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f482000e5d0 tx=0x7f482000ea90 comp rx=0 tx=0).stop 2024-04-17T10:43:23.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.377+0000 7f482c8a5640 1 -- 172.21.15.83:0/867601898 shutdown_connections 2024-04-17T10:43:23.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.377+0000 7f482c8a5640 1 --2- 172.21.15.83:0/867601898 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4808066f60 0x7f4808069420 secure :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f48100aa000 tx=0x7f48280cdda0 comp rx=0 tx=0).stop 2024-04-17T10:43:23.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.377+0000 7f482c8a5640 1 --2- 172.21.15.83:0/867601898 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48280c6af0 0x7f4828156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:23.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.377+0000 7f482c8a5640 1 --2- 172.21.15.83:0/867601898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4828155e70 0x7f48280c65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:23.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.377+0000 7f482c8a5640 1 --2- 172.21.15.83:0/867601898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4814004050 0x7f48281556f0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:23.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.377+0000 7f482c8a5640 1 -- 172.21.15.83:0/867601898 >> 172.21.15.83:0/867601898 conn(0x7f482806abe0 msgr2=0x7f4828070bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:23.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.381+0000 7f482c8a5640 1 -- 172.21.15.83:0/867601898 shutdown_connections 2024-04-17T10:43:23.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.381+0000 7f482c8a5640 1 -- 172.21.15.83:0/867601898 wait complete. 2024-04-17T10:43:23.377 INFO:tasks.ceph.mds.a.smithi083.stderr: -2156> 2024-04-17T10:43:06.369+0000 7f04fa2b8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T10:43:23.393 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-17T10:43:23.409 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-04-17T10:43:23.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.517+0000 7f7004ea4640 1 Processor -- start 2024-04-17T10:43:23.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.517+0000 7f7004ea4640 1 -- start start 2024-04-17T10:43:23.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.517+0000 7f7004ea4640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f700015e0f0 0x7f70000b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:23.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.517+0000 7f7004ea4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f70000ce400 0x7f7000161c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:23.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.517+0000 7f7004ea4640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f700015e4d0 con 0x7f70000b5450 2024-04-17T10:43:23.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.517+0000 7f7004ea4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7000162b40 con 0x7f70000ce400 2024-04-17T10:43:23.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.517+0000 7f7004ea4640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7000162580 con 0x7f700015e0f0 2024-04-17T10:43:23.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.517+0000 7f6ffdd9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f70000ce400 0x7f7000161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:23.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.517+0000 7f6ffdd9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f70000ce400 0x7f7000161c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52026/0 (socket says 172.21.15.83:52026) 2024-04-17T10:43:23.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.517+0000 7f6ffdd9b640 1 -- 172.21.15.83:0/2987920475 learned_addr learned my addr 172.21.15.83:0/2987920475 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:23.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.521+0000 7f6ffdd9b640 1 -- 172.21.15.83:0/2987920475 >> v2:172.21.15.167:3300/0 conn(0x7f700015e0f0 msgr2=0x7f70000b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:23.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.521+0000 7f6ffe59c640 1 --2- 172.21.15.83:0/2987920475 >> v2:172.21.15.167:3300/0 conn(0x7f700015e0f0 0x7f70000b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:23.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.521+0000 7f6ffdd9b640 1 --2- 172.21.15.83:0/2987920475 >> v2:172.21.15.167:3300/0 conn(0x7f700015e0f0 0x7f70000b6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:23.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.521+0000 7f6ffdd9b640 1 -- 172.21.15.83:0/2987920475 >> v1:172.21.15.167:6789/0 conn(0x7f70000b5450 legacy=0x7f70000d2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:23.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.521+0000 7f6ffdd9b640 1 -- 172.21.15.83:0/2987920475 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7000162860 con 0x7f70000ce400 2024-04-17T10:43:23.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.521+0000 7f6ffe59c640 1 --2- 172.21.15.83:0/2987920475 >> v2:172.21.15.167:3300/0 conn(0x7f700015e0f0 0x7f70000b6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:23.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.521+0000 7f6ffdd9b640 1 --2- 172.21.15.83:0/2987920475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70000ce400 0x7f7000161c10 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f700008c3f0 tx=0x7f6fe801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f297948a30d758dd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:23.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.521+0000 7f6ffd59a640 1 -- 172.21.15.83:0/2987920475 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe80b1070 con 0x7f70000ce400 2024-04-17T10:43:23.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.521+0000 7f6ffd59a640 1 -- 172.21.15.83:0/2987920475 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe80ac640 con 0x7f70000ce400 2024-04-17T10:43:23.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.521+0000 7f6ffd59a640 1 -- 172.21.15.83:0/2987920475 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe80aca10 con 0x7f70000ce400 2024-04-17T10:43:23.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.521+0000 7f7004ea4640 1 -- 172.21.15.83:0/2987920475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70000ce400 msgr2=0x7f7000161c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:23.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.521+0000 7f7004ea4640 1 --2- 172.21.15.83:0/2987920475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70000ce400 0x7f7000161c10 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f700008c3f0 tx=0x7f6fe801c450 comp rx=0 tx=0).stop 2024-04-17T10:43:23.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.521+0000 7f7004ea4640 1 -- 172.21.15.83:0/2987920475 shutdown_connections 2024-04-17T10:43:23.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.521+0000 7f7004ea4640 1 --2- 172.21.15.83:0/2987920475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70000ce400 0x7f7000161c10 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:23.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.521+0000 7f7004ea4640 1 --2- 172.21.15.83:0/2987920475 >> v2:172.21.15.167:3300/0 conn(0x7f700015e0f0 0x7f70000b6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:23.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.521+0000 7f7004ea4640 1 -- 172.21.15.83:0/2987920475 >> 172.21.15.83:0/2987920475 conn(0x7f70000d0450 msgr2=0x7f70000d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:23.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.521+0000 7f7004ea4640 1 -- 172.21.15.83:0/2987920475 shutdown_connections 2024-04-17T10:43:23.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.521+0000 7f7004ea4640 1 -- 172.21.15.83:0/2987920475 wait complete. 2024-04-17T10:43:23.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.525+0000 7f7004ea4640 1 Processor -- start 2024-04-17T10:43:23.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.525+0000 7f7004ea4640 1 -- start start 2024-04-17T10:43:23.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.525+0000 7f7004ea4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70000b5450 0x7f7000160440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:23.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.525+0000 7f7004ea4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70000ce400 0x7f700014b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:23.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.525+0000 7f7004ea4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f700015e0f0 0x7f7000156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:23.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.525+0000 7f7004ea4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70001622d0 con 0x7f70000ce400 2024-04-17T10:43:23.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.525+0000 7f7004ea4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7000162150 con 0x7f70000b5450 2024-04-17T10:43:23.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.525+0000 7f7004ea4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700016af50 con 0x7f700015e0f0 2024-04-17T10:43:23.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.525+0000 7f6ffdd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70000ce400 0x7f700014b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:23.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.525+0000 7f6ffdd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70000ce400 0x7f700014b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52028/0 (socket says 172.21.15.83:52028) 2024-04-17T10:43:23.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.525+0000 7f6ffdd9b640 1 -- 172.21.15.83:0/175015338 learned_addr learned my addr 172.21.15.83:0/175015338 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:23.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.525+0000 7f6ffe59c640 1 --2- 172.21.15.83:0/175015338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70000b5450 0x7f7000160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:23.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.525+0000 7f6ffdd9b640 1 -- 172.21.15.83:0/175015338 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f700015e0f0 msgr2=0x7f7000156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:23.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.525+0000 7f6ffed9d640 1 --2- 172.21.15.83:0/175015338 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f700015e0f0 0x7f7000156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:23.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.525+0000 7f6ffdd9b640 1 --2- 172.21.15.83:0/175015338 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f700015e0f0 0x7f7000156c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:23.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.525+0000 7f6ffdd9b640 1 -- 172.21.15.83:0/175015338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70000b5450 msgr2=0x7f7000160440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:23.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.525+0000 7f6ffdd9b640 1 --2- 172.21.15.83:0/175015338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70000b5450 0x7f7000160440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:23.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.525+0000 7f6ffdd9b640 1 -- 172.21.15.83:0/175015338 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7000155c30 con 0x7f70000ce400 2024-04-17T10:43:23.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.525+0000 7f6ffe59c640 1 --2- 172.21.15.83:0/175015338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70000b5450 0x7f7000160440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:23.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.525+0000 7f6ffdd9b640 1 --2- 172.21.15.83:0/175015338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70000ce400 0x7f700014b910 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f6fe80af040 tx=0x7f6fe80191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:23.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.525+0000 7f6ff6ffd640 1 -- 172.21.15.83:0/175015338 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe80b1040 con 0x7f70000ce400 2024-04-17T10:43:23.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.525+0000 7f6ff6ffd640 1 -- 172.21.15.83:0/175015338 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe80ba070 con 0x7f70000ce400 2024-04-17T10:43:23.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.525+0000 7f7004ea4640 1 -- 172.21.15.83:0/175015338 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7000155eb0 con 0x7f70000ce400 2024-04-17T10:43:23.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.525+0000 7f6ff6ffd640 1 -- 172.21.15.83:0/175015338 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe80b58a0 con 0x7f70000ce400 2024-04-17T10:43:23.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.525+0000 7f7004ea4640 1 -- 172.21.15.83:0/175015338 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7000156470 con 0x7f70000ce400 2024-04-17T10:43:23.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.529+0000 7f7004ea4640 1 -- 172.21.15.83:0/175015338 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70000681a0 con 0x7f70000ce400 2024-04-17T10:43:23.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.529+0000 7f6ff6ffd640 1 -- 172.21.15.83:0/175015338 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6fe80ac500 con 0x7f70000ce400 2024-04-17T10:43:23.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.529+0000 7f6ff6ffd640 1 --2- 172.21.15.83:0/175015338 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6fe0066f60 0x7f6fe0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:23.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.529+0000 7f6ff6ffd640 1 -- 172.21.15.83:0/175015338 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f6fe811ef70 con 0x7f70000ce400 2024-04-17T10:43:23.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.529+0000 7f6ffe59c640 1 --2- 172.21.15.83:0/175015338 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6fe0066f60 0x7f6fe0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:23.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.529+0000 7f6ffe59c640 1 --2- 172.21.15.83:0/175015338 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6fe0066f60 0x7f6fe0069420 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7000150080 tx=0x7f6ff0007550 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:23.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.533+0000 7f6ff6ffd640 1 -- 172.21.15.83:0/175015338 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7000155c30 con 0x7f70000ce400 2024-04-17T10:43:23.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:23.725+0000 7f7004ea4640 1 -- 172.21.15.83:0/175015338 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7000074f90 con 0x7f70000ce400 2024-04-17T10:43:24.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.385+0000 7f6ff6ffd640 1 -- 172.21.15.83:0/175015338 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v41) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6fe80160d0 con 0x7f70000ce400 2024-04-17T10:43:24.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.389+0000 7f7004ea4640 1 -- 172.21.15.83:0/175015338 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6fe0066f60 msgr2=0x7f6fe0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:24.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.389+0000 7f7004ea4640 1 --2- 172.21.15.83:0/175015338 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6fe0066f60 0x7f6fe0069420 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7000150080 tx=0x7f6ff0007550 comp rx=0 tx=0).stop 2024-04-17T10:43:24.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.389+0000 7f7004ea4640 1 -- 172.21.15.83:0/175015338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70000ce400 msgr2=0x7f700014b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:24.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.389+0000 7f7004ea4640 1 --2- 172.21.15.83:0/175015338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70000ce400 0x7f700014b910 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f6fe80af040 tx=0x7f6fe80191f0 comp rx=0 tx=0).stop 2024-04-17T10:43:24.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.389+0000 7f7004ea4640 1 -- 172.21.15.83:0/175015338 shutdown_connections 2024-04-17T10:43:24.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.389+0000 7f7004ea4640 1 --2- 172.21.15.83:0/175015338 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6fe0066f60 0x7f6fe0069420 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:24.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.389+0000 7f7004ea4640 1 --2- 172.21.15.83:0/175015338 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f700015e0f0 0x7f7000156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:24.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.389+0000 7f7004ea4640 1 --2- 172.21.15.83:0/175015338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70000ce400 0x7f700014b910 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:24.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.389+0000 7f7004ea4640 1 --2- 172.21.15.83:0/175015338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70000b5450 0x7f7000160440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:24.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.389+0000 7f7004ea4640 1 -- 172.21.15.83:0/175015338 >> 172.21.15.83:0/175015338 conn(0x7f70000d0450 msgr2=0x7f700015c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:24.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.389+0000 7f7004ea4640 1 -- 172.21.15.83:0/175015338 shutdown_connections 2024-04-17T10:43:24.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.389+0000 7f7004ea4640 1 -- 172.21.15.83:0/175015338 wait complete. 2024-04-17T10:43:24.406 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-17T10:43:24.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.577+0000 7f20b7409640 1 Processor -- start 2024-04-17T10:43:24.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.577+0000 7f20b7409640 1 -- start start 2024-04-17T10:43:24.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.581+0000 7f20b7409640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f20b00d4510 0x7f20b00d17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:24.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.581+0000 7f20b7409640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20b00d1fd0 0x7f20b00d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:24.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.581+0000 7f20b7409640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20b0074800 con 0x7f20b015e020 2024-04-17T10:43:24.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.581+0000 7f20b7409640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20b00cf550 con 0x7f20b00d4510 2024-04-17T10:43:24.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.581+0000 7f20b7409640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b00d2920 con 0x7f20b00d1fd0 2024-04-17T10:43:24.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.581+0000 7f20b59a6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20b00d1fd0 0x7f20b00d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:24.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.581+0000 7f20b49a4640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f20b00d4510 0x7f20b00d17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:24.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.581+0000 7f20b59a6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20b00d1fd0 0x7f20b00d23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:55820/0 (socket says 172.21.15.83:55820) 2024-04-17T10:43:24.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.581+0000 7f20b59a6640 1 -- 172.21.15.83:0/2392290095 learned_addr learned my addr 172.21.15.83:0/2392290095 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:24.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.581+0000 7f20b59a6640 1 -- 172.21.15.83:0/2392290095 >> v2:172.21.15.167:3300/0 conn(0x7f20b00d4510 msgr2=0x7f20b00d17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:24.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.581+0000 7f20b59a6640 1 --2- 172.21.15.83:0/2392290095 >> v2:172.21.15.167:3300/0 conn(0x7f20b00d4510 0x7f20b00d17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:24.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.581+0000 7f20b59a6640 1 -- 172.21.15.83:0/2392290095 >> v1:172.21.15.167:6789/0 conn(0x7f20b015e020 legacy=0x7f20b00cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:24.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.581+0000 7f20b59a6640 1 -- 172.21.15.83:0/2392290095 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b00ce980 con 0x7f20b00d1fd0 2024-04-17T10:43:24.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.581+0000 7f20b59a6640 1 --2- 172.21.15.83:0/2392290095 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20b00d1fd0 0x7f20b00d23e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f20a8009e50 tx=0x7f20a801ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52da907d4b1f585d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:24.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.581+0000 7f20a7fff640 1 -- 172.21.15.83:0/2392290095 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a80a58d0 con 0x7f20b00d1fd0 2024-04-17T10:43:24.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.581+0000 7f20a7fff640 1 -- 172.21.15.83:0/2392290095 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a80a5a70 con 0x7f20b00d1fd0 2024-04-17T10:43:24.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.581+0000 7f20a7fff640 1 -- 172.21.15.83:0/2392290095 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a80a5e40 con 0x7f20b00d1fd0 2024-04-17T10:43:24.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.585+0000 7f20b7409640 1 -- 172.21.15.83:0/2392290095 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20b00d1fd0 msgr2=0x7f20b00d23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:24.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.585+0000 7f20b7409640 1 --2- 172.21.15.83:0/2392290095 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20b00d1fd0 0x7f20b00d23e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f20a8009e50 tx=0x7f20a801ca00 comp rx=0 tx=0).stop 2024-04-17T10:43:24.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.585+0000 7f20b7409640 1 -- 172.21.15.83:0/2392290095 shutdown_connections 2024-04-17T10:43:24.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.585+0000 7f20b7409640 1 --2- 172.21.15.83:0/2392290095 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20b00d1fd0 0x7f20b00d23e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:24.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.585+0000 7f20b7409640 1 --2- 172.21.15.83:0/2392290095 >> v2:172.21.15.167:3300/0 conn(0x7f20b00d4510 0x7f20b00d17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:24.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.585+0000 7f20b7409640 1 -- 172.21.15.83:0/2392290095 >> 172.21.15.83:0/2392290095 conn(0x7f20b00d39f0 msgr2=0x7f20b00d3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:24.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.585+0000 7f20b7409640 1 -- 172.21.15.83:0/2392290095 shutdown_connections 2024-04-17T10:43:24.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.585+0000 7f20b7409640 1 -- 172.21.15.83:0/2392290095 wait complete. 2024-04-17T10:43:24.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.585+0000 7f20b7409640 1 Processor -- start 2024-04-17T10:43:24.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.585+0000 7f20b7409640 1 -- start start 2024-04-17T10:43:24.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.585+0000 7f20b7409640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20b00d1fd0 0x7f20b0141fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:24.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.585+0000 7f20b7409640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20b00d4510 0x7f20b0142520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:24.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.585+0000 7f20b7409640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20b015e020 0x7f20b0155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:24.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.585+0000 7f20b7409640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b006b900 con 0x7f20b015e020 2024-04-17T10:43:24.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.585+0000 7f20b7409640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b00d1d10 con 0x7f20b00d1fd0 2024-04-17T10:43:24.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.585+0000 7f20b7409640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b006ba80 con 0x7f20b00d4510 2024-04-17T10:43:24.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.585+0000 7f20b59a6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20b015e020 0x7f20b0155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:24.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.585+0000 7f20b59a6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20b015e020 0x7f20b0155dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52040/0 (socket says 172.21.15.83:52040) 2024-04-17T10:43:24.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.585+0000 7f20b59a6640 1 -- 172.21.15.83:0/3661036797 learned_addr learned my addr 172.21.15.83:0/3661036797 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:24.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.585+0000 7f20b49a4640 1 --2- 172.21.15.83:0/3661036797 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20b00d4510 0x7f20b0142520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:24.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.585+0000 7f20b51a5640 1 --2- 172.21.15.83:0/3661036797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20b00d1fd0 0x7f20b0141fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:24.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.585+0000 7f20b59a6640 1 -- 172.21.15.83:0/3661036797 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20b00d4510 msgr2=0x7f20b0142520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:24.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.585+0000 7f20b59a6640 1 --2- 172.21.15.83:0/3661036797 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20b00d4510 0x7f20b0142520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:24.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.585+0000 7f20b59a6640 1 -- 172.21.15.83:0/3661036797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20b00d1fd0 msgr2=0x7f20b0141fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:24.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.585+0000 7f20b59a6640 1 --2- 172.21.15.83:0/3661036797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20b00d1fd0 0x7f20b0141fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:24.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.585+0000 7f20b59a6640 1 -- 172.21.15.83:0/3661036797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b0142d80 con 0x7f20b015e020 2024-04-17T10:43:24.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.585+0000 7f20b59a6640 1 --2- 172.21.15.83:0/3661036797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20b015e020 0x7f20b0155dc0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f20a80af040 tx=0x7f20a80197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:24.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.585+0000 7f20a5ffb640 1 -- 172.21.15.83:0/3661036797 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a80bc020 con 0x7f20b015e020 2024-04-17T10:43:24.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.589+0000 7f20a5ffb640 1 -- 172.21.15.83:0/3661036797 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a8007e50 con 0x7f20b015e020 2024-04-17T10:43:24.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.589+0000 7f20a5ffb640 1 -- 172.21.15.83:0/3661036797 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a80a58d0 con 0x7f20b015e020 2024-04-17T10:43:24.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.589+0000 7f20b7409640 1 -- 172.21.15.83:0/3661036797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b0143060 con 0x7f20b015e020 2024-04-17T10:43:24.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.589+0000 7f20b7409640 1 -- 172.21.15.83:0/3661036797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b0143620 con 0x7f20b015e020 2024-04-17T10:43:24.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.589+0000 7f20b7409640 1 -- 172.21.15.83:0/3661036797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20b00681a0 con 0x7f20b015e020 2024-04-17T10:43:24.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.589+0000 7f20a5ffb640 1 -- 172.21.15.83:0/3661036797 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f20a80bf020 con 0x7f20b015e020 2024-04-17T10:43:24.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.589+0000 7f20a5ffb640 1 --2- 172.21.15.83:0/3661036797 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2094066f60 0x7f2094069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:24.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.589+0000 7f20a5ffb640 1 -- 172.21.15.83:0/3661036797 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f20a8120630 con 0x7f20b015e020 2024-04-17T10:43:24.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.589+0000 7f20b51a5640 1 --2- 172.21.15.83:0/3661036797 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2094066f60 0x7f2094069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:24.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.589+0000 7f20b51a5640 1 --2- 172.21.15.83:0/3661036797 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2094066f60 0x7f2094069420 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f20ac0015b0 tx=0x7f20ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:24.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.593+0000 7f20a5ffb640 1 -- 172.21.15.83:0/3661036797 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f20a80c6030 con 0x7f20b015e020 2024-04-17T10:43:24.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:24.737+0000 7f20b7409640 1 -- 172.21.15.83:0/3661036797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f20b015f7f0 con 0x7f20b015e020 2024-04-17T10:43:25.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.405+0000 7f20a5ffb640 1 -- 172.21.15.83:0/3661036797 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v65) v1 ==== 181+0+0 (secure 0 0 0) 0x7f20a8016620 con 0x7f20b015e020 2024-04-17T10:43:25.400 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2024-04-17T10:43:25.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.409+0000 7f20b7409640 1 -- 172.21.15.83:0/3661036797 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2094066f60 msgr2=0x7f2094069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:25.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.409+0000 7f20b7409640 1 --2- 172.21.15.83:0/3661036797 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2094066f60 0x7f2094069420 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f20ac0015b0 tx=0x7f20ac009040 comp rx=0 tx=0).stop 2024-04-17T10:43:25.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.409+0000 7f20b7409640 1 -- 172.21.15.83:0/3661036797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20b015e020 msgr2=0x7f20b0155dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:25.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.409+0000 7f20b7409640 1 --2- 172.21.15.83:0/3661036797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20b015e020 0x7f20b0155dc0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f20a80af040 tx=0x7f20a80197a0 comp rx=0 tx=0).stop 2024-04-17T10:43:25.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.409+0000 7f20b7409640 1 -- 172.21.15.83:0/3661036797 shutdown_connections 2024-04-17T10:43:25.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.409+0000 7f20b7409640 1 --2- 172.21.15.83:0/3661036797 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2094066f60 0x7f2094069420 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:25.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.409+0000 7f20b7409640 1 --2- 172.21.15.83:0/3661036797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20b015e020 0x7f20b0155dc0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:25.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.409+0000 7f20b7409640 1 --2- 172.21.15.83:0/3661036797 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20b00d4510 0x7f20b0142520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:25.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.409+0000 7f20b7409640 1 --2- 172.21.15.83:0/3661036797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20b00d1fd0 0x7f20b0141fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:25.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.409+0000 7f20b7409640 1 -- 172.21.15.83:0/3661036797 >> 172.21.15.83:0/3661036797 conn(0x7f20b00d39f0 msgr2=0x7f20b00d2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.413+0000 7f20b7409640 1 -- 172.21.15.83:0/3661036797 shutdown_connections 2024-04-17T10:43:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.413+0000 7f20b7409640 1 -- 172.21.15.83:0/3661036797 wait complete. 2024-04-17T10:43:25.429 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-17T10:43:25.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.557+0000 7f703ee6b640 1 Processor -- start 2024-04-17T10:43:25.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.557+0000 7f703ee6b640 1 -- start start 2024-04-17T10:43:25.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.557+0000 7f703ee6b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f703806b4a0 0x7f703806cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:25.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.557+0000 7f703ee6b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70380cc350 con 0x7f703815aa90 2024-04-17T10:43:25.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.557+0000 7f703ee6b640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70380cc630 con 0x7f703815ae70 2024-04-17T10:43:25.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.557+0000 7f703ee6b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7038066eb0 con 0x7f703806b4a0 2024-04-17T10:43:25.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.557+0000 7f7037fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f703806b4a0 0x7f703806cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:25.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.557+0000 7f7037fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f703806b4a0 0x7f703806cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52044/0 (socket says 172.21.15.83:52044) 2024-04-17T10:43:25.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.557+0000 7f7037fff640 1 -- 172.21.15.83:0/277887402 learned_addr learned my addr 172.21.15.83:0/277887402 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:25.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.557+0000 7f7037fff640 1 -- 172.21.15.83:0/277887402 >> v1:172.21.15.167:6789/0 conn(0x7f703815ae70 legacy=0x7f70380720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:25.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.557+0000 7f7037fff640 1 -- 172.21.15.83:0/277887402 >> v1:172.21.15.83:6789/0 conn(0x7f703815aa90 legacy=0x7f703806abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:25.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.557+0000 7f7037fff640 1 -- 172.21.15.83:0/277887402 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7038066b40 con 0x7f703806b4a0 2024-04-17T10:43:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.561+0000 7f7037fff640 1 --2- 172.21.15.83:0/277887402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f703806b4a0 0x7f703806cd60 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f7020009870 tx=0x7f702001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ad3d1a625bb13 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.561+0000 7f70377fe640 1 -- 172.21.15.83:0/277887402 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70200a9400 con 0x7f703806b4a0 2024-04-17T10:43:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.561+0000 7f70377fe640 1 -- 172.21.15.83:0/277887402 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70200a95a0 con 0x7f703806b4a0 2024-04-17T10:43:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.561+0000 7f70377fe640 1 -- 172.21.15.83:0/277887402 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70200a9970 con 0x7f703806b4a0 2024-04-17T10:43:25.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.561+0000 7f703ee6b640 1 -- 172.21.15.83:0/277887402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f703806b4a0 msgr2=0x7f703806cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:25.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.561+0000 7f703ee6b640 1 --2- 172.21.15.83:0/277887402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f703806b4a0 0x7f703806cd60 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f7020009870 tx=0x7f702001c520 comp rx=0 tx=0).stop 2024-04-17T10:43:25.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.561+0000 7f703ee6b640 1 -- 172.21.15.83:0/277887402 shutdown_connections 2024-04-17T10:43:25.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.561+0000 7f703ee6b640 1 --2- 172.21.15.83:0/277887402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f703806b4a0 0x7f703806cd60 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:25.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.561+0000 7f703ee6b640 1 -- 172.21.15.83:0/277887402 >> 172.21.15.83:0/277887402 conn(0x7f7038071200 msgr2=0x7f70380b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:25.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.561+0000 7f703ee6b640 1 -- 172.21.15.83:0/277887402 shutdown_connections 2024-04-17T10:43:25.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.561+0000 7f703ee6b640 1 -- 172.21.15.83:0/277887402 wait complete. 2024-04-17T10:43:25.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.561+0000 7f703ee6b640 1 Processor -- start 2024-04-17T10:43:25.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.561+0000 7f703ee6b640 1 -- start start 2024-04-17T10:43:25.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.561+0000 7f703ee6b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f703806b4a0 0x7f703809bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:25.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.561+0000 7f703ee6b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f703815aa90 0x7f7038152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:25.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.561+0000 7f703ee6b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f703815ae70 0x7f7038155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:25.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.561+0000 7f703ee6b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70380b28d0 con 0x7f703815ae70 2024-04-17T10:43:25.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.561+0000 7f703ee6b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70380b2750 con 0x7f703806b4a0 2024-04-17T10:43:25.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.561+0000 7f703ee6b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70380b2a50 con 0x7f703815aa90 2024-04-17T10:43:25.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.561+0000 7f703d408640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f703815ae70 0x7f7038155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:25.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.565+0000 7f703d408640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f703815ae70 0x7f7038155da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52050/0 (socket says 172.21.15.83:52050) 2024-04-17T10:43:25.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.565+0000 7f703d408640 1 -- 172.21.15.83:0/3608373347 learned_addr learned my addr 172.21.15.83:0/3608373347 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:25.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.565+0000 7f703cc07640 1 --2- 172.21.15.83:0/3608373347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f703806b4a0 0x7f703809bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:25.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.565+0000 7f7037fff640 1 --2- 172.21.15.83:0/3608373347 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f703815aa90 0x7f7038152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:25.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.565+0000 7f703d408640 1 -- 172.21.15.83:0/3608373347 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f703815aa90 msgr2=0x7f7038152ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:25.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.565+0000 7f703d408640 1 --2- 172.21.15.83:0/3608373347 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f703815aa90 0x7f7038152ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:25.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.565+0000 7f703d408640 1 -- 172.21.15.83:0/3608373347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f703806b4a0 msgr2=0x7f703809bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:25.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.565+0000 7f703d408640 1 --2- 172.21.15.83:0/3608373347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f703806b4a0 0x7f703809bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:25.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.565+0000 7f703d408640 1 -- 172.21.15.83:0/3608373347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f703809c230 con 0x7f703815ae70 2024-04-17T10:43:25.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.565+0000 7f703d408640 1 --2- 172.21.15.83:0/3608373347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f703815ae70 0x7f7038155da0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f702800b440 tx=0x7f702800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:25.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.565+0000 7f70357fa640 1 -- 172.21.15.83:0/3608373347 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7028015020 con 0x7f703815ae70 2024-04-17T10:43:25.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.565+0000 7f70357fa640 1 -- 172.21.15.83:0/3608373347 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7028011850 con 0x7f703815ae70 2024-04-17T10:43:25.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.565+0000 7f703ee6b640 1 -- 172.21.15.83:0/3608373347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f703809c510 con 0x7f703815ae70 2024-04-17T10:43:25.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.565+0000 7f70357fa640 1 -- 172.21.15.83:0/3608373347 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7028011c40 con 0x7f703815ae70 2024-04-17T10:43:25.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.565+0000 7f703ee6b640 1 -- 172.21.15.83:0/3608373347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f703809cad0 con 0x7f703815ae70 2024-04-17T10:43:25.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.565+0000 7f703ee6b640 1 -- 172.21.15.83:0/3608373347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7038075000 con 0x7f703815ae70 2024-04-17T10:43:25.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.565+0000 7f70357fa640 1 -- 172.21.15.83:0/3608373347 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7028018020 con 0x7f703815ae70 2024-04-17T10:43:25.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.565+0000 7f70357fa640 1 --2- 172.21.15.83:0/3608373347 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7018066f60 0x7f7018069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:25.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.565+0000 7f70357fa640 1 -- 172.21.15.83:0/3608373347 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5443+0+0 (secure 0 0 0) 0x7f7028084ac0 con 0x7f703815ae70 2024-04-17T10:43:25.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.565+0000 7f703cc07640 1 --2- 172.21.15.83:0/3608373347 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7018066f60 0x7f7018069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:25.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.565+0000 7f703cc07640 1 --2- 172.21.15.83:0/3608373347 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7018066f60 0x7f7018069420 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7030001cb0 tx=0x7f7030009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:25.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.569+0000 7f70357fa640 1 -- 172.21.15.83:0/3608373347 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f702801f030 con 0x7f703815ae70 2024-04-17T10:43:25.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:25.717+0000 7f703ee6b640 1 -- 172.21.15.83:0/3608373347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7038153010 con 0x7f703815ae70 2024-04-17T10:43:26.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.413+0000 7f70357fa640 1 -- 172.21.15.83:0/3608373347 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v66) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7028018310 con 0x7f703815ae70 2024-04-17T10:43:26.410 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2024-04-17T10:43:26.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.425+0000 7f703ee6b640 1 -- 172.21.15.83:0/3608373347 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7018066f60 msgr2=0x7f7018069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:26.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.425+0000 7f703ee6b640 1 --2- 172.21.15.83:0/3608373347 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7018066f60 0x7f7018069420 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7030001cb0 tx=0x7f7030009040 comp rx=0 tx=0).stop 2024-04-17T10:43:26.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.425+0000 7f703ee6b640 1 -- 172.21.15.83:0/3608373347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f703815ae70 msgr2=0x7f7038155da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:26.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.425+0000 7f703ee6b640 1 --2- 172.21.15.83:0/3608373347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f703815ae70 0x7f7038155da0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f702800b440 tx=0x7f702800b900 comp rx=0 tx=0).stop 2024-04-17T10:43:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.425+0000 7f703ee6b640 1 -- 172.21.15.83:0/3608373347 shutdown_connections 2024-04-17T10:43:26.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.425+0000 7f703ee6b640 1 --2- 172.21.15.83:0/3608373347 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7018066f60 0x7f7018069420 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:26.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.425+0000 7f703ee6b640 1 --2- 172.21.15.83:0/3608373347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f703815ae70 0x7f7038155da0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:26.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.425+0000 7f703ee6b640 1 --2- 172.21.15.83:0/3608373347 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f703815aa90 0x7f7038152ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:26.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.425+0000 7f703ee6b640 1 --2- 172.21.15.83:0/3608373347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f703806b4a0 0x7f703809bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:26.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.425+0000 7f703ee6b640 1 -- 172.21.15.83:0/3608373347 >> 172.21.15.83:0/3608373347 conn(0x7f7038071200 msgr2=0x7f703806b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:26.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.425+0000 7f703ee6b640 1 -- 172.21.15.83:0/3608373347 shutdown_connections 2024-04-17T10:43:26.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.425+0000 7f703ee6b640 1 -- 172.21.15.83:0/3608373347 wait complete. 2024-04-17T10:43:26.441 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-17T10:43:26.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.557+0000 7f8b47340640 1 Processor -- start 2024-04-17T10:43:26.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.557+0000 7f8b47340640 1 -- start start 2024-04-17T10:43:26.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.557+0000 7f8b47340640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b4015bac0 0x7f8b40071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:26.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.557+0000 7f8b47340640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8b40070470 0x7f8b4006ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:26.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.557+0000 7f8b47340640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b4006c1f0 con 0x7f8b4015b6e0 2024-04-17T10:43:26.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.557+0000 7f8b47340640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b40070a10 con 0x7f8b40070470 2024-04-17T10:43:26.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.557+0000 7f8b47340640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b40070cf0 con 0x7f8b4015bac0 2024-04-17T10:43:26.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.557+0000 7f8b450dc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8b4015b6e0 0x7f8b400cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52274/0 (socket says 172.21.15.83:52274) 2024-04-17T10:43:26.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.557+0000 7f8b450dc640 1 -- 172.21.15.83:0/3864991865 learned_addr learned my addr 172.21.15.83:0/3864991865 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:43:26.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.557+0000 7f8b458dd640 1 --2- 172.21.15.83:0/3864991865 >> v2:172.21.15.83:3300/0 conn(0x7f8b40070470 0x7f8b4006ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:26.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.557+0000 7f8b3ffff640 1 -- 172.21.15.83:0/3864991865 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 764846636 0 0) 0x7f8b4006c1f0 con 0x7f8b4015b6e0 2024-04-17T10:43:26.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.557+0000 7f8b3ffff640 1 -- 172.21.15.83:0/3864991865 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b28003530 con 0x7f8b4015b6e0 2024-04-17T10:43:26.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.557+0000 7f8b458dd640 1 -- 172.21.15.83:0/3864991865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b4015bac0 msgr2=0x7f8b40071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:26.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.557+0000 7f8b448db640 1 --2- 172.21.15.83:0/3864991865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b4015bac0 0x7f8b40071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:26.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.557+0000 7f8b458dd640 1 --2- 172.21.15.83:0/3864991865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b4015bac0 0x7f8b40071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:26.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.557+0000 7f8b458dd640 1 -- 172.21.15.83:0/3864991865 >> v1:172.21.15.83:6789/0 conn(0x7f8b4015b6e0 legacy=0x7f8b400cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:26.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.557+0000 7f8b458dd640 1 -- 172.21.15.83:0/3864991865 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b40070fd0 con 0x7f8b40070470 2024-04-17T10:43:26.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.561+0000 7f8b458dd640 1 --2- 172.21.15.83:0/3864991865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b40070470 0x7f8b4006ae10 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f8b380097c0 tx=0x7f8b3801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974aa6956ed50e72 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:26.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.561+0000 7f8b3ffff640 1 -- 172.21.15.83:0/3864991865 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b380b0070 con 0x7f8b40070470 2024-04-17T10:43:26.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.561+0000 7f8b3ffff640 1 -- 172.21.15.83:0/3864991865 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b380ab600 con 0x7f8b40070470 2024-04-17T10:43:26.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.561+0000 7f8b3ffff640 1 -- 172.21.15.83:0/3864991865 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b380ab9d0 con 0x7f8b40070470 2024-04-17T10:43:26.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.561+0000 7f8b47340640 1 -- 172.21.15.83:0/3864991865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b40070470 msgr2=0x7f8b4006ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:26.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.561+0000 7f8b47340640 1 --2- 172.21.15.83:0/3864991865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b40070470 0x7f8b4006ae10 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f8b380097c0 tx=0x7f8b3801c4a0 comp rx=0 tx=0).stop 2024-04-17T10:43:26.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.561+0000 7f8b47340640 1 -- 172.21.15.83:0/3864991865 shutdown_connections 2024-04-17T10:43:26.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.561+0000 7f8b47340640 1 --2- 172.21.15.83:0/3864991865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b40070470 0x7f8b4006ae10 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:26.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.561+0000 7f8b47340640 1 --2- 172.21.15.83:0/3864991865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b4015bac0 0x7f8b40071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:26.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.561+0000 7f8b47340640 1 -- 172.21.15.83:0/3864991865 >> 172.21.15.83:0/3864991865 conn(0x7f8b400b15b0 msgr2=0x7f8b400b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:26.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.561+0000 7f8b47340640 1 -- 172.21.15.83:0/3864991865 shutdown_connections 2024-04-17T10:43:26.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.561+0000 7f8b47340640 1 -- 172.21.15.83:0/3864991865 wait complete. 2024-04-17T10:43:26.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.561+0000 7f8b47340640 1 Processor -- start 2024-04-17T10:43:26.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.561+0000 7f8b47340640 1 -- start start 2024-04-17T10:43:26.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.561+0000 7f8b47340640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b40070470 0x7f8b400b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:26.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.561+0000 7f8b47340640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b4015b6e0 0x7f8b401377f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:26.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.561+0000 7f8b47340640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b4015bac0 0x7f8b40139840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:26.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.561+0000 7f8b47340640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4006d270 con 0x7f8b4015b6e0 2024-04-17T10:43:26.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.561+0000 7f8b47340640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4006d0f0 con 0x7f8b40070470 2024-04-17T10:43:26.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.561+0000 7f8b47340640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4006b760 con 0x7f8b4015bac0 2024-04-17T10:43:26.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.561+0000 7f8b448db640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b4015b6e0 0x7f8b401377f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:26.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.561+0000 7f8b448db640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b4015b6e0 0x7f8b401377f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52064/0 (socket says 172.21.15.83:52064) 2024-04-17T10:43:26.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.561+0000 7f8b448db640 1 -- 172.21.15.83:0/2363150147 learned_addr learned my addr 172.21.15.83:0/2363150147 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:26.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.561+0000 7f8b448db640 1 -- 172.21.15.83:0/2363150147 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b4015bac0 msgr2=0x7f8b40139840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:26.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.565+0000 7f8b448db640 1 --2- 172.21.15.83:0/2363150147 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b4015bac0 0x7f8b40139840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:26.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.565+0000 7f8b448db640 1 -- 172.21.15.83:0/2363150147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b40070470 msgr2=0x7f8b400b44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:26.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.565+0000 7f8b448db640 1 --2- 172.21.15.83:0/2363150147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b40070470 0x7f8b400b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:26.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.565+0000 7f8b448db640 1 -- 172.21.15.83:0/2363150147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b40157470 con 0x7f8b4015b6e0 2024-04-17T10:43:26.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.565+0000 7f8b448db640 1 --2- 172.21.15.83:0/2363150147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b4015b6e0 0x7f8b401377f0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f8b3400ccc0 tx=0x7f8b3400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:26.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.565+0000 7f8b3dffb640 1 -- 172.21.15.83:0/2363150147 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b340039a0 con 0x7f8b4015b6e0 2024-04-17T10:43:26.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.565+0000 7f8b3dffb640 1 -- 172.21.15.83:0/2363150147 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b34003b40 con 0x7f8b4015b6e0 2024-04-17T10:43:26.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.565+0000 7f8b47340640 1 -- 172.21.15.83:0/2363150147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b40139fd0 con 0x7f8b4015b6e0 2024-04-17T10:43:26.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.565+0000 7f8b3dffb640 1 -- 172.21.15.83:0/2363150147 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b34004030 con 0x7f8b4015b6e0 2024-04-17T10:43:26.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.565+0000 7f8b47340640 1 -- 172.21.15.83:0/2363150147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b4013a590 con 0x7f8b4015b6e0 2024-04-17T10:43:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.565+0000 7f8b47340640 1 -- 172.21.15.83:0/2363150147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b40074f90 con 0x7f8b4015b6e0 2024-04-17T10:43:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.565+0000 7f8b3dffb640 1 -- 172.21.15.83:0/2363150147 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8b340041d0 con 0x7f8b4015b6e0 2024-04-17T10:43:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.565+0000 7f8b3dffb640 1 --2- 172.21.15.83:0/2363150147 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8b2c066f60 0x7f8b2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.565+0000 7f8b450dc640 1 --2- 172.21.15.83:0/2363150147 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8b2c066f60 0x7f8b2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.565+0000 7f8b3dffb640 1 -- 172.21.15.83:0/2363150147 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f8b340851c0 con 0x7f8b4015b6e0 2024-04-17T10:43:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.565+0000 7f8b450dc640 1 --2- 172.21.15.83:0/2363150147 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8b2c066f60 0x7f8b2c069420 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8b30007fd0 tx=0x7f8b30007900 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:26.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.569+0000 7f8b3dffb640 1 -- 172.21.15.83:0/2363150147 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8b34014070 con 0x7f8b4015b6e0 2024-04-17T10:43:26.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.717+0000 7f8b47340640 1 -- 172.21.15.83:0/2363150147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f8b400b1130 con 0x7f8b4015b6e0 2024-04-17T10:43:26.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.721+0000 7f8b3dffb640 1 -- 172.21.15.83:0/2363150147 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8b34056470 con 0x7f8b4015b6e0 2024-04-17T10:43:26.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.721+0000 7f8b47340640 1 -- 172.21.15.83:0/2363150147 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8b2c066f60 msgr2=0x7f8b2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:26.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.721+0000 7f8b47340640 1 --2- 172.21.15.83:0/2363150147 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8b2c066f60 0x7f8b2c069420 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8b30007fd0 tx=0x7f8b30007900 comp rx=0 tx=0).stop 2024-04-17T10:43:26.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.721+0000 7f8b47340640 1 -- 172.21.15.83:0/2363150147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b4015b6e0 msgr2=0x7f8b401377f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:26.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.721+0000 7f8b47340640 1 --2- 172.21.15.83:0/2363150147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b4015b6e0 0x7f8b401377f0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f8b3400ccc0 tx=0x7f8b3400a5c0 comp rx=0 tx=0).stop 2024-04-17T10:43:26.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.721+0000 7f8b47340640 1 -- 172.21.15.83:0/2363150147 shutdown_connections 2024-04-17T10:43:26.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.721+0000 7f8b47340640 1 --2- 172.21.15.83:0/2363150147 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8b2c066f60 0x7f8b2c069420 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:26.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.721+0000 7f8b47340640 1 --2- 172.21.15.83:0/2363150147 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b4015bac0 0x7f8b40139840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:26.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.721+0000 7f8b47340640 1 --2- 172.21.15.83:0/2363150147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b4015b6e0 0x7f8b401377f0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:26.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.721+0000 7f8b47340640 1 --2- 172.21.15.83:0/2363150147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b40070470 0x7f8b400b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:26.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.721+0000 7f8b47340640 1 -- 172.21.15.83:0/2363150147 >> 172.21.15.83:0/2363150147 conn(0x7f8b400b15b0 msgr2=0x7f8b400ce7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:26.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.721+0000 7f8b47340640 1 -- 172.21.15.83:0/2363150147 shutdown_connections 2024-04-17T10:43:26.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.721+0000 7f8b47340640 1 -- 172.21.15.83:0/2363150147 wait complete. 2024-04-17T10:43:26.734 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair' 2024-04-17T10:43:26.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.857+0000 7f2551301640 1 Processor -- start 2024-04-17T10:43:26.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.857+0000 7f2551301640 1 -- start start 2024-04-17T10:43:26.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.861+0000 7f2551301640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f254c0b9820 0x7f254c15eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:26.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.861+0000 7f2551301640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f254c161670 0x7f254c0d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:26.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.861+0000 7f2551301640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f254c0d0df0 0x7f254c0b5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:26.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.861+0000 7f2551301640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f254c0d1260 con 0x7f254c161670 2024-04-17T10:43:26.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.861+0000 7f2551301640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f254c15f040 con 0x7f254c0b9820 2024-04-17T10:43:26.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.861+0000 7f2551301640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f254c0cee30 con 0x7f254c0d0df0 2024-04-17T10:43:26.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.861+0000 7f254a7fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f254c161670 0x7f254c0d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:26.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.861+0000 7f254a7fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f254c161670 0x7f254c0d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50898/0 (socket says 172.21.15.83:50898) 2024-04-17T10:43:26.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.861+0000 7f254a7fc640 1 -- 172.21.15.83:0/3054598569 learned_addr learned my addr 172.21.15.83:0/3054598569 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:26.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.861+0000 7f254affd640 1 --2- 172.21.15.83:0/3054598569 >> v2:172.21.15.167:3300/0 conn(0x7f254c0b9820 0x7f254c15eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:26.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.861+0000 7f254a7fc640 1 -- 172.21.15.83:0/3054598569 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f254c0d0df0 msgr2=0x7f254c0b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:26.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.861+0000 7f254a7fc640 1 --2- 172.21.15.83:0/3054598569 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f254c0d0df0 0x7f254c0b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:26.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.861+0000 7f254a7fc640 1 -- 172.21.15.83:0/3054598569 >> v2:172.21.15.167:3300/0 conn(0x7f254c0b9820 msgr2=0x7f254c15eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:26.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.861+0000 7f254a7fc640 1 --2- 172.21.15.83:0/3054598569 >> v2:172.21.15.167:3300/0 conn(0x7f254c0b9820 0x7f254c15eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:26.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.861+0000 7f254a7fc640 1 -- 172.21.15.83:0/3054598569 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254c161b30 con 0x7f254c161670 2024-04-17T10:43:26.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.861+0000 7f254a7fc640 1 --2- 172.21.15.83:0/3054598569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f254c161670 0x7f254c0d08b0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f254c08c5e0 tx=0x7f253c01c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e750ad3b26104c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:26.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.861+0000 7f2549ffb640 1 -- 172.21.15.83:0/3054598569 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c0b0070 con 0x7f254c161670 2024-04-17T10:43:26.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.861+0000 7f2549ffb640 1 -- 172.21.15.83:0/3054598569 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f253c0ab600 con 0x7f254c161670 2024-04-17T10:43:26.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.861+0000 7f2549ffb640 1 -- 172.21.15.83:0/3054598569 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c0ab9d0 con 0x7f254c161670 2024-04-17T10:43:26.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f2551301640 1 -- 172.21.15.83:0/3054598569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f254c161670 msgr2=0x7f254c0d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:26.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f2551301640 1 --2- 172.21.15.83:0/3054598569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f254c161670 0x7f254c0d08b0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f254c08c5e0 tx=0x7f253c01c1d0 comp rx=0 tx=0).stop 2024-04-17T10:43:26.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f2551301640 1 -- 172.21.15.83:0/3054598569 shutdown_connections 2024-04-17T10:43:26.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f2551301640 1 --2- 172.21.15.83:0/3054598569 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f254c0d0df0 0x7f254c0b5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:26.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f2551301640 1 --2- 172.21.15.83:0/3054598569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f254c161670 0x7f254c0d08b0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:26.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f2551301640 1 --2- 172.21.15.83:0/3054598569 >> v2:172.21.15.167:3300/0 conn(0x7f254c0b9820 0x7f254c15eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:26.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f2551301640 1 -- 172.21.15.83:0/3054598569 >> 172.21.15.83:0/3054598569 conn(0x7f254c06b280 msgr2=0x7f254c06b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:26.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f2551301640 1 -- 172.21.15.83:0/3054598569 shutdown_connections 2024-04-17T10:43:26.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f2551301640 1 -- 172.21.15.83:0/3054598569 wait complete. 2024-04-17T10:43:26.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f2551301640 1 Processor -- start 2024-04-17T10:43:26.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f2551301640 1 -- start start 2024-04-17T10:43:26.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f2551301640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f254c0b9820 0x7f254c14ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:26.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f2551301640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f254c0d0df0 0x7f254c14cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:26.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f2551301640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f254c161670 0x7f254c0cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:26.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f2551301640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c0d1ff0 con 0x7f254c0d0df0 2024-04-17T10:43:26.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f2551301640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c0d1e70 con 0x7f254c0b9820 2024-04-17T10:43:26.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f2551301640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f254c0d2170 con 0x7f254c161670 2024-04-17T10:43:26.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f254a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f254c0d0df0 0x7f254c14cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:26.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f254a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f254c0d0df0 0x7f254c14cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50902/0 (socket says 172.21.15.83:50902) 2024-04-17T10:43:26.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f254a7fc640 1 -- 172.21.15.83:0/3375098277 learned_addr learned my addr 172.21.15.83:0/3375098277 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:26.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f254affd640 1 --2- 172.21.15.83:0/3375098277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f254c0b9820 0x7f254c14ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:26.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f254b7fe640 1 --2- 172.21.15.83:0/3375098277 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f254c161670 0x7f254c0cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:26.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f254a7fc640 1 -- 172.21.15.83:0/3375098277 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f254c161670 msgr2=0x7f254c0cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:26.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f254a7fc640 1 --2- 172.21.15.83:0/3375098277 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f254c161670 0x7f254c0cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:26.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f254a7fc640 1 -- 172.21.15.83:0/3375098277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f254c0b9820 msgr2=0x7f254c14ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:26.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f254a7fc640 1 --2- 172.21.15.83:0/3375098277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f254c0b9820 0x7f254c14ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:26.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f254a7fc640 1 -- 172.21.15.83:0/3375098277 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254c14e310 con 0x7f254c0d0df0 2024-04-17T10:43:26.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f254affd640 1 --2- 172.21.15.83:0/3375098277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f254c0b9820 0x7f254c14ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:26.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f254b7fe640 1 --2- 172.21.15.83:0/3375098277 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f254c161670 0x7f254c0cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:26.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f254a7fc640 1 --2- 172.21.15.83:0/3375098277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f254c0d0df0 0x7f254c14cc40 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f253c01ca70 tx=0x7f253c018f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:26.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f25337fe640 1 -- 172.21.15.83:0/3375098277 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c0ba020 con 0x7f254c0d0df0 2024-04-17T10:43:26.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f25337fe640 1 -- 172.21.15.83:0/3375098277 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f253c0b6640 con 0x7f254c0d0df0 2024-04-17T10:43:26.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f2551301640 1 -- 172.21.15.83:0/3375098277 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f254c14e5f0 con 0x7f254c0d0df0 2024-04-17T10:43:26.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f25337fe640 1 -- 172.21.15.83:0/3375098277 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c0b0040 con 0x7f254c0d0df0 2024-04-17T10:43:26.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.865+0000 7f2551301640 1 -- 172.21.15.83:0/3375098277 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f254c14ebb0 con 0x7f254c0d0df0 2024-04-17T10:43:26.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.869+0000 7f2551301640 1 -- 172.21.15.83:0/3375098277 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f254c074f90 con 0x7f254c0d0df0 2024-04-17T10:43:26.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.869+0000 7f25337fe640 1 -- 172.21.15.83:0/3375098277 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f253c0bd020 con 0x7f254c0d0df0 2024-04-17T10:43:26.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.869+0000 7f25337fe640 1 --2- 172.21.15.83:0/3375098277 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2538066f60 0x7f2538069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:26.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.869+0000 7f254affd640 1 --2- 172.21.15.83:0/3375098277 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2538066f60 0x7f2538069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:26.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.869+0000 7f25337fe640 1 -- 172.21.15.83:0/3375098277 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f253c11f0f0 con 0x7f254c0d0df0 2024-04-17T10:43:26.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.869+0000 7f254affd640 1 --2- 172.21.15.83:0/3375098277 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2538066f60 0x7f2538069420 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2540006fd0 tx=0x7f2540008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:26.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:26.873+0000 7f25337fe640 1 -- 172.21.15.83:0/3375098277 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f253c016110 con 0x7f254c0d0df0 2024-04-17T10:43:27.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.093+0000 7f2551301640 1 -- 172.21.15.83:0/3375098277 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]} v 0) v1 -- 0x7f254c0d1870 con 0x7f254c0d0df0 2024-04-17T10:43:27.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.417+0000 7f25337fe640 1 -- 172.21.15.83:0/3375098277 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]}]=0 v255) v1 ==== 143+0+0 (secure 0 0 0) 0x7f253c0f0410 con 0x7f254c0d0df0 2024-04-17T10:43:27.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.421+0000 7f2551301640 1 -- 172.21.15.83:0/3375098277 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2538066f60 msgr2=0x7f2538069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:27.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.421+0000 7f2551301640 1 --2- 172.21.15.83:0/3375098277 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2538066f60 0x7f2538069420 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2540006fd0 tx=0x7f2540008040 comp rx=0 tx=0).stop 2024-04-17T10:43:27.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.421+0000 7f2551301640 1 -- 172.21.15.83:0/3375098277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f254c0d0df0 msgr2=0x7f254c14cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:27.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.421+0000 7f2551301640 1 --2- 172.21.15.83:0/3375098277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f254c0d0df0 0x7f254c14cc40 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f253c01ca70 tx=0x7f253c018f70 comp rx=0 tx=0).stop 2024-04-17T10:43:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.421+0000 7f2551301640 1 -- 172.21.15.83:0/3375098277 shutdown_connections 2024-04-17T10:43:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.421+0000 7f2551301640 1 --2- 172.21.15.83:0/3375098277 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2538066f60 0x7f2538069420 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.421+0000 7f2551301640 1 --2- 172.21.15.83:0/3375098277 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f254c161670 0x7f254c0cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.421+0000 7f2551301640 1 --2- 172.21.15.83:0/3375098277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f254c0d0df0 0x7f254c14cc40 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.421+0000 7f2551301640 1 --2- 172.21.15.83:0/3375098277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f254c0b9820 0x7f254c14ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.421+0000 7f2551301640 1 -- 172.21.15.83:0/3375098277 >> 172.21.15.83:0/3375098277 conn(0x7f254c06b280 msgr2=0x7f254c0d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.421+0000 7f2551301640 1 -- 172.21.15.83:0/3375098277 shutdown_connections 2024-04-17T10:43:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.421+0000 7f2551301640 1 -- 172.21.15.83:0/3375098277 wait complete. 2024-04-17T10:43:27.438 INFO:tasks.cephfs_test_runner:test_scrub_repair (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ok 2024-04-17T10:43:27.439 INFO:tasks.cephfs_test_runner:Starting test: test_stray_evaluation_with_scrub (tasks.cephfs.test_scrub_checks.TestScrubChecks) test that scrub can iterate over ~mdsdir and evaluate strays 2024-04-17T10:43:27.439 INFO:tasks.cephfs_test_runner:test_stray_evaluation_with_scrub (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-17T10:43:27.439 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:43:27.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.561+0000 7f31f72f8640 1 Processor -- start 2024-04-17T10:43:27.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.561+0000 7f31f72f8640 1 -- start start 2024-04-17T10:43:27.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.565+0000 7f31f72f8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31f015ba30 0x7f31f0072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:27.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.565+0000 7f31f72f8640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f31f015c960 0x7f31f00cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:27.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.565+0000 7f31f72f8640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31f00cfea0 con 0x7f31f00cfa10 2024-04-17T10:43:27.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.565+0000 7f31f72f8640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31f0071ea0 con 0x7f31f015c960 2024-04-17T10:43:27.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.565+0000 7f31f72f8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f006e320 con 0x7f31f015ba30 2024-04-17T10:43:27.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.565+0000 7f31f4893640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f31f015c960 0x7f31f00cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:27.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.565+0000 7f31f5895640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f31f00cfa10 0x7f31f006c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:43440/0 (socket says 172.21.15.83:43440) 2024-04-17T10:43:27.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.565+0000 7f31f5895640 1 -- 172.21.15.83:0/2174925861 learned_addr learned my addr 172.21.15.83:0/2174925861 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:43:27.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.565+0000 7f31f4893640 1 -- 172.21.15.83:0/2174925861 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31f015ba30 msgr2=0x7f31f0072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:27.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.565+0000 7f31f5094640 1 --2- 172.21.15.83:0/2174925861 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31f015ba30 0x7f31f0072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:27.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.565+0000 7f31f4893640 1 --2- 172.21.15.83:0/2174925861 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31f015ba30 0x7f31f0072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:27.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.565+0000 7f31f4893640 1 -- 172.21.15.83:0/2174925861 >> v1:172.21.15.83:6789/0 conn(0x7f31f00cfa10 legacy=0x7f31f006c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:27.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.565+0000 7f31f4893640 1 -- 172.21.15.83:0/2174925861 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f006e040 con 0x7f31f015c960 2024-04-17T10:43:27.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.565+0000 7f31f4893640 1 --2- 172.21.15.83:0/2174925861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31f015c960 0x7f31f00cf4d0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f31e00097c0 tx=0x7f31e001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14adc7c8e7b3b21 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:27.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.565+0000 7f31e7fff640 1 -- 172.21.15.83:0/2174925861 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e00b0070 con 0x7f31f015c960 2024-04-17T10:43:27.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.565+0000 7f31e7fff640 1 -- 172.21.15.83:0/2174925861 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e00ab600 con 0x7f31f015c960 2024-04-17T10:43:27.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.565+0000 7f31e7fff640 1 -- 172.21.15.83:0/2174925861 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e00ab9d0 con 0x7f31f015c960 2024-04-17T10:43:27.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.569+0000 7f31f72f8640 1 -- 172.21.15.83:0/2174925861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31f015c960 msgr2=0x7f31f00cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:27.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.569+0000 7f31f72f8640 1 --2- 172.21.15.83:0/2174925861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31f015c960 0x7f31f00cf4d0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f31e00097c0 tx=0x7f31e001c410 comp rx=0 tx=0).stop 2024-04-17T10:43:27.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.569+0000 7f31f72f8640 1 -- 172.21.15.83:0/2174925861 shutdown_connections 2024-04-17T10:43:27.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.569+0000 7f31f72f8640 1 --2- 172.21.15.83:0/2174925861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31f015c960 0x7f31f00cf4d0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:27.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.569+0000 7f31f72f8640 1 --2- 172.21.15.83:0/2174925861 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31f015ba30 0x7f31f0072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:27.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.569+0000 7f31f72f8640 1 -- 172.21.15.83:0/2174925861 >> 172.21.15.83:0/2174925861 conn(0x7f31f006abe0 msgr2=0x7f31f006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:27.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.569+0000 7f31f72f8640 1 -- 172.21.15.83:0/2174925861 shutdown_connections 2024-04-17T10:43:27.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.569+0000 7f31f72f8640 1 -- 172.21.15.83:0/2174925861 wait complete. 2024-04-17T10:43:27.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.569+0000 7f31f72f8640 1 Processor -- start 2024-04-17T10:43:27.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.569+0000 7f31f72f8640 1 -- start start 2024-04-17T10:43:27.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.569+0000 7f31f72f8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31f00cfa10 0x7f31f014b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:27.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.569+0000 7f31f72f8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31f015ba30 0x7f31f0152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:27.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.569+0000 7f31f72f8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31f015c960 0x7f31f0153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:27.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.569+0000 7f31f72f8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f006dca0 con 0x7f31f015c960 2024-04-17T10:43:27.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.569+0000 7f31f72f8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f00aeae0 con 0x7f31f00cfa10 2024-04-17T10:43:27.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.569+0000 7f31f72f8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f006de20 con 0x7f31f015ba30 2024-04-17T10:43:27.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.569+0000 7f31f5895640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31f015c960 0x7f31f0153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:27.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.569+0000 7f31f5895640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31f015c960 0x7f31f0153510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50914/0 (socket says 172.21.15.83:50914) 2024-04-17T10:43:27.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.569+0000 7f31f5895640 1 -- 172.21.15.83:0/3520920800 learned_addr learned my addr 172.21.15.83:0/3520920800 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:27.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.569+0000 7f31f5895640 1 -- 172.21.15.83:0/3520920800 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31f015ba30 msgr2=0x7f31f0152fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:27.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.569+0000 7f31f5895640 1 --2- 172.21.15.83:0/3520920800 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31f015ba30 0x7f31f0152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:27.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.569+0000 7f31f5895640 1 -- 172.21.15.83:0/3520920800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31f00cfa10 msgr2=0x7f31f014b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:27.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.569+0000 7f31f5895640 1 --2- 172.21.15.83:0/3520920800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31f00cfa10 0x7f31f014b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:27.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.569+0000 7f31f5895640 1 -- 172.21.15.83:0/3520920800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f0155ac0 con 0x7f31f015c960 2024-04-17T10:43:27.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.569+0000 7f31f5895640 1 --2- 172.21.15.83:0/3520920800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31f015c960 0x7f31f0153510 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f31e800ca80 tx=0x7f31e800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:27.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.569+0000 7f31e5ffb640 1 -- 172.21.15.83:0/3520920800 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e8017070 con 0x7f31f015c960 2024-04-17T10:43:27.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.569+0000 7f31e5ffb640 1 -- 172.21.15.83:0/3520920800 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e800ac30 con 0x7f31f015c960 2024-04-17T10:43:27.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.569+0000 7f31f72f8640 1 -- 172.21.15.83:0/3520920800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f0155500 con 0x7f31f015c960 2024-04-17T10:43:27.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.569+0000 7f31e5ffb640 1 -- 172.21.15.83:0/3520920800 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e8012600 con 0x7f31f015c960 2024-04-17T10:43:27.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.569+0000 7f31f72f8640 1 -- 172.21.15.83:0/3520920800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f014c550 con 0x7f31f015c960 2024-04-17T10:43:27.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.573+0000 7f31f72f8640 1 -- 172.21.15.83:0/3520920800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f0074f90 con 0x7f31f015c960 2024-04-17T10:43:27.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.573+0000 7f31e5ffb640 1 -- 172.21.15.83:0/3520920800 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f31e801a070 con 0x7f31f015c960 2024-04-17T10:43:27.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.573+0000 7f31e5ffb640 1 --2- 172.21.15.83:0/3520920800 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f31d4066f60 0x7f31d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:27.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.573+0000 7f31e5ffb640 1 -- 172.21.15.83:0/3520920800 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f31e8084960 con 0x7f31f015c960 2024-04-17T10:43:27.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.573+0000 7f31f5094640 1 --2- 172.21.15.83:0/3520920800 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f31d4066f60 0x7f31d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:27.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.573+0000 7f31f5094640 1 --2- 172.21.15.83:0/3520920800 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f31d4066f60 0x7f31d4069420 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f31f015dec0 tx=0x7f31ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:27.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.577+0000 7f31e5ffb640 1 -- 172.21.15.83:0/3520920800 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f31e800ae20 con 0x7f31f015c960 2024-04-17T10:43:27.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.733+0000 7f31f72f8640 1 -- 172.21.15.83:0/3520920800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31f006da80 con 0x7f31f015c960 2024-04-17T10:43:27.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.737+0000 7f31e5ffb640 1 -- 172.21.15.83:0/3520920800 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+13078 (secure 0 0 0) 0x7f31e80555c0 con 0x7f31f015c960 2024-04-17T10:43:27.731 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:43:27.731 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":66,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:43:26.406589+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/3985646487":"2024-04-18T10:43:23.373037+0000","172.21.15.83:6836/3985646487":"2024-04-18T10:43:23.373037+0000","172.21.15.167:6833/271994303":"2024-04-18T10:42:59.346779+0000","172.21.15.167:6832/271994303":"2024-04-18T10:42:59.346779+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:43:27.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.737+0000 7f31f72f8640 1 -- 172.21.15.83:0/3520920800 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f31d4066f60 msgr2=0x7f31d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:27.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.737+0000 7f31f72f8640 1 --2- 172.21.15.83:0/3520920800 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f31d4066f60 0x7f31d4069420 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f31f015dec0 tx=0x7f31ec008040 comp rx=0 tx=0).stop 2024-04-17T10:43:27.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.737+0000 7f31f72f8640 1 -- 172.21.15.83:0/3520920800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31f015c960 msgr2=0x7f31f0153510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:27.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.737+0000 7f31f72f8640 1 --2- 172.21.15.83:0/3520920800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31f015c960 0x7f31f0153510 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f31e800ca80 tx=0x7f31e800cf40 comp rx=0 tx=0).stop 2024-04-17T10:43:27.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.737+0000 7f31f72f8640 1 -- 172.21.15.83:0/3520920800 shutdown_connections 2024-04-17T10:43:27.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.737+0000 7f31f72f8640 1 --2- 172.21.15.83:0/3520920800 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f31d4066f60 0x7f31d4069420 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:27.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.737+0000 7f31f72f8640 1 --2- 172.21.15.83:0/3520920800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31f015c960 0x7f31f0153510 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:27.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.737+0000 7f31f72f8640 1 --2- 172.21.15.83:0/3520920800 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31f015ba30 0x7f31f0152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.737+0000 7f31f72f8640 1 --2- 172.21.15.83:0/3520920800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31f00cfa10 0x7f31f014b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.737+0000 7f31f72f8640 1 -- 172.21.15.83:0/3520920800 >> 172.21.15.83:0/3520920800 conn(0x7f31f006abe0 msgr2=0x7f31f015e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.737+0000 7f31f72f8640 1 -- 172.21.15.83:0/3520920800 shutdown_connections 2024-04-17T10:43:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.737+0000 7f31f72f8640 1 -- 172.21.15.83:0/3520920800 wait complete. 2024-04-17T10:43:27.751 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:43:27.751 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:43:27.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.869+0000 7ff3061d7640 1 Processor -- start 2024-04-17T10:43:27.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.869+0000 7ff3061d7640 1 -- start start 2024-04-17T10:43:27.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.869+0000 7ff3061d7640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff3000b5b60 0x7ff3000b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:27.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.869+0000 7ff3061d7640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff30015e4d0 con 0x7ff30015e0f0 2024-04-17T10:43:27.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.869+0000 7ff3061d7640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff300162580 con 0x7ff3000ce400 2024-04-17T10:43:27.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.869+0000 7ff3061d7640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff300162b40 con 0x7ff3000b5b60 2024-04-17T10:43:27.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.869+0000 7ff2ff7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff30015e0f0 0x7ff300161c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:43446/0 (socket says 172.21.15.83:43446) 2024-04-17T10:43:27.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.869+0000 7ff2ff7fe640 1 -- 172.21.15.83:0/367778878 learned_addr learned my addr 172.21.15.83:0/367778878 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:43:27.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.869+0000 7ff2fe7fc640 1 -- 172.21.15.83:0/367778878 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1892535853 0 0) 0x7ff30015e4d0 con 0x7ff30015e0f0 2024-04-17T10:43:27.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.869+0000 7ff2fe7fc640 1 -- 172.21.15.83:0/367778878 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2e8003530 con 0x7ff30015e0f0 2024-04-17T10:43:27.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.869+0000 7ff2fffff640 1 --2- 172.21.15.83:0/367778878 >> v2:172.21.15.167:3300/0 conn(0x7ff3000b5b60 0x7ff3000b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:27.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.869+0000 7ff2fe7fc640 1 -- 172.21.15.83:0/367778878 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3708624533 0 0) 0x7ff2e8003530 con 0x7ff30015e0f0 2024-04-17T10:43:27.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.869+0000 7ff2fe7fc640 1 -- 172.21.15.83:0/367778878 >> v2:172.21.15.167:3300/0 conn(0x7ff3000b5b60 msgr2=0x7ff3000b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:27.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.869+0000 7ff2fe7fc640 1 --2- 172.21.15.83:0/367778878 >> v2:172.21.15.167:3300/0 conn(0x7ff3000b5b60 0x7ff3000b6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:27.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.869+0000 7ff2fe7fc640 1 -- 172.21.15.83:0/367778878 >> v1:172.21.15.167:6789/0 conn(0x7ff3000ce400 legacy=0x7ff3000b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:27.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.869+0000 7ff2fe7fc640 1 -- 172.21.15.83:0/367778878 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff300162860 con 0x7ff30015e0f0 2024-04-17T10:43:27.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.869+0000 7ff2fe7fc640 1 -- 172.21.15.83:0/367778878 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7ff2f0001680 con 0x7ff30015e0f0 2024-04-17T10:43:27.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.869+0000 7ff2fe7fc640 1 -- 172.21.15.83:0/367778878 >> v1:172.21.15.83:6789/0 conn(0x7ff30015e0f0 legacy=0x7ff300161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:27.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.869+0000 7ff2fe7fc640 1 --2- 172.21.15.83:0/367778878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2e8003a30 0x7ff2e8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:27.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.869+0000 7ff2fe7fc640 1 --2- 172.21.15.83:0/367778878 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2e8004810 0x7ff2e8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:27.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.869+0000 7ff2fe7fc640 1 --2- 172.21.15.83:0/367778878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2e80052f0 0x7ff2e800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:27.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.869+0000 7ff2fe7fc640 1 -- 172.21.15.83:0/367778878 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff300162b40 con 0x7ff2e80052f0 2024-04-17T10:43:27.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.869+0000 7ff2fe7fc640 1 -- 172.21.15.83:0/367778878 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e800c210 con 0x7ff2e8003a30 2024-04-17T10:43:27.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.869+0000 7ff2fe7fc640 1 -- 172.21.15.83:0/367778878 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e800c410 con 0x7ff2e8004810 2024-04-17T10:43:27.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.869+0000 7ff2fffff640 1 --2- 172.21.15.83:0/367778878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2e80052f0 0x7ff2e800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:27.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.873+0000 7ff2ff7fe640 1 --2- 172.21.15.83:0/367778878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2e8003a30 0x7ff2e8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:27.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.873+0000 7ff2feffd640 1 --2- 172.21.15.83:0/367778878 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2e8004810 0x7ff2e8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:27.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.873+0000 7ff2fffff640 1 -- 172.21.15.83:0/367778878 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2e8004810 msgr2=0x7ff2e8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:27.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.873+0000 7ff2fffff640 1 --2- 172.21.15.83:0/367778878 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2e8004810 0x7ff2e8004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:27.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.873+0000 7ff2fffff640 1 -- 172.21.15.83:0/367778878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2e8003a30 msgr2=0x7ff2e8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:27.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.873+0000 7ff2fffff640 1 --2- 172.21.15.83:0/367778878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2e8003a30 0x7ff2e8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:27.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.873+0000 7ff2fffff640 1 -- 172.21.15.83:0/367778878 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e8003530 con 0x7ff2e80052f0 2024-04-17T10:43:27.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.873+0000 7ff2ff7fe640 1 --2- 172.21.15.83:0/367778878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2e8003a30 0x7ff2e8003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:27.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.873+0000 7ff2fffff640 1 -- 172.21.15.83:0/367778878 reap_dead start 2024-04-17T10:43:27.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.873+0000 7ff2fffff640 1 --2- 172.21.15.83:0/367778878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2e80052f0 0x7ff2e800b9d0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7ff3000b6430 tx=0x7ff2f801ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27febc60ea33d85 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:27.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.873+0000 7ff2fe7fc640 1 -- 172.21.15.83:0/367778878 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8009be0 con 0x7ff2e80052f0 2024-04-17T10:43:27.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.873+0000 7ff2fe7fc640 1 -- 172.21.15.83:0/367778878 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f80a5d20 con 0x7ff2e80052f0 2024-04-17T10:43:27.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.873+0000 7ff2fe7fc640 1 -- 172.21.15.83:0/367778878 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8003be0 con 0x7ff2e80052f0 2024-04-17T10:43:27.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.873+0000 7ff3061d7640 1 -- 172.21.15.83:0/367778878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2e80052f0 msgr2=0x7ff2e800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:27.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.873+0000 7ff3061d7640 1 --2- 172.21.15.83:0/367778878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2e80052f0 0x7ff2e800b9d0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7ff3000b6430 tx=0x7ff2f801ce20 comp rx=0 tx=0).stop 2024-04-17T10:43:27.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.873+0000 7ff3061d7640 1 -- 172.21.15.83:0/367778878 shutdown_connections 2024-04-17T10:43:27.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.873+0000 7ff3061d7640 1 --2- 172.21.15.83:0/367778878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2e80052f0 0x7ff2e800b9d0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:27.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.873+0000 7ff3061d7640 1 -- 172.21.15.83:0/367778878 >> 172.21.15.83:0/367778878 conn(0x7ff3000d0450 msgr2=0x7ff3000d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:27.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.873+0000 7ff3061d7640 1 -- 172.21.15.83:0/367778878 shutdown_connections 2024-04-17T10:43:27.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.873+0000 7ff3061d7640 1 -- 172.21.15.83:0/367778878 wait complete. 2024-04-17T10:43:27.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.877+0000 7ff3061d7640 1 Processor -- start 2024-04-17T10:43:27.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.877+0000 7ff3061d7640 1 -- start start 2024-04-17T10:43:27.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.877+0000 7ff3061d7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2e80052f0 0x7ff300144d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:27.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.877+0000 7ff3061d7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3000ccfa0 0x7ff3000c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:27.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.877+0000 7ff3061d7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff300154bf0 0x7ff300155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:27.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.877+0000 7ff3061d7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3000b6d00 con 0x7ff2e80052f0 2024-04-17T10:43:27.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.877+0000 7ff3061d7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3000d2020 con 0x7ff3000ccfa0 2024-04-17T10:43:27.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.877+0000 7ff3061d7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3000b6e80 con 0x7ff300154bf0 2024-04-17T10:43:27.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.877+0000 7ff2ff7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2e80052f0 0x7ff300144d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:27.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.877+0000 7ff2ff7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2e80052f0 0x7ff300144d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50928/0 (socket says 172.21.15.83:50928) 2024-04-17T10:43:27.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.877+0000 7ff2ff7fe640 1 -- 172.21.15.83:0/159098235 learned_addr learned my addr 172.21.15.83:0/159098235 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:27.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.877+0000 7ff2ff7fe640 1 -- 172.21.15.83:0/159098235 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff300154bf0 msgr2=0x7ff300155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:27.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.877+0000 7ff2ff7fe640 1 --2- 172.21.15.83:0/159098235 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff300154bf0 0x7ff300155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:27.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.877+0000 7ff2ff7fe640 1 -- 172.21.15.83:0/159098235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3000ccfa0 msgr2=0x7ff3000c9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:27.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.877+0000 7ff2ff7fe640 1 --2- 172.21.15.83:0/159098235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3000ccfa0 0x7ff3000c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:27.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.877+0000 7ff2ff7fe640 1 -- 172.21.15.83:0/159098235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff300145690 con 0x7ff2e80052f0 2024-04-17T10:43:27.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.877+0000 7ff2ff7fe640 1 --2- 172.21.15.83:0/159098235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2e80052f0 0x7ff300144d90 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0010b70 tx=0x7ff2f000e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:27.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.877+0000 7ff2e7fff640 1 -- 172.21.15.83:0/159098235 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f001b070 con 0x7ff2e80052f0 2024-04-17T10:43:27.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.877+0000 7ff2e7fff640 1 -- 172.21.15.83:0/159098235 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f0011040 con 0x7ff2e80052f0 2024-04-17T10:43:27.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.877+0000 7ff3061d7640 1 -- 172.21.15.83:0/159098235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff300145970 con 0x7ff2e80052f0 2024-04-17T10:43:27.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.877+0000 7ff2e7fff640 1 -- 172.21.15.83:0/159098235 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f00143a0 con 0x7ff2e80052f0 2024-04-17T10:43:27.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.877+0000 7ff3061d7640 1 -- 172.21.15.83:0/159098235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff300145f30 con 0x7ff2e80052f0 2024-04-17T10:43:27.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.877+0000 7ff3061d7640 1 -- 172.21.15.83:0/159098235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3000681a0 con 0x7ff2e80052f0 2024-04-17T10:43:27.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.877+0000 7ff2e7fff640 1 -- 172.21.15.83:0/159098235 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff2f001e070 con 0x7ff2e80052f0 2024-04-17T10:43:27.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.877+0000 7ff2e7fff640 1 --2- 172.21.15.83:0/159098235 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff2ec066f90 0x7ff2ec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:27.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.877+0000 7ff2feffd640 1 --2- 172.21.15.83:0/159098235 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff2ec066f90 0x7ff2ec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:27.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.877+0000 7ff2e7fff640 1 -- 172.21.15.83:0/159098235 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5029+0+0 (secure 0 0 0) 0x7ff2f00877c0 con 0x7ff2e80052f0 2024-04-17T10:43:27.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.881+0000 7ff2feffd640 1 --2- 172.21.15.83:0/159098235 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff2ec066f90 0x7ff2ec069450 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff30015cfd0 tx=0x7ff2f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:27.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:27.881+0000 7ff2e7fff640 1 -- 172.21.15.83:0/159098235 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff2f0058a70 con 0x7ff2e80052f0 2024-04-17T10:43:28.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.025+0000 7ff3061d7640 1 -- 172.21.15.83:0/159098235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff3000cd4b0 con 0x7ff2e80052f0 2024-04-17T10:43:28.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.029+0000 7ff2e7fff640 1 -- 172.21.15.83:0/159098235 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff2f0058370 con 0x7ff2e80052f0 2024-04-17T10:43:28.024 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:43:28.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.029+0000 7ff3061d7640 1 -- 172.21.15.83:0/159098235 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff2ec066f90 msgr2=0x7ff2ec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:28.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.029+0000 7ff3061d7640 1 --2- 172.21.15.83:0/159098235 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff2ec066f90 0x7ff2ec069450 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff30015cfd0 tx=0x7ff2f4009040 comp rx=0 tx=0).stop 2024-04-17T10:43:28.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.029+0000 7ff3061d7640 1 -- 172.21.15.83:0/159098235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2e80052f0 msgr2=0x7ff300144d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:28.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.029+0000 7ff3061d7640 1 --2- 172.21.15.83:0/159098235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2e80052f0 0x7ff300144d90 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0010b70 tx=0x7ff2f000e4d0 comp rx=0 tx=0).stop 2024-04-17T10:43:28.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.029+0000 7ff3061d7640 1 -- 172.21.15.83:0/159098235 shutdown_connections 2024-04-17T10:43:28.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.029+0000 7ff3061d7640 1 --2- 172.21.15.83:0/159098235 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff2ec066f90 0x7ff2ec069450 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:28.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.029+0000 7ff3061d7640 1 --2- 172.21.15.83:0/159098235 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff300154bf0 0x7ff300155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:28.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.029+0000 7ff3061d7640 1 --2- 172.21.15.83:0/159098235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3000ccfa0 0x7ff3000c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:28.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.029+0000 7ff3061d7640 1 --2- 172.21.15.83:0/159098235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2e80052f0 0x7ff300144d90 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:28.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.029+0000 7ff3061d7640 1 -- 172.21.15.83:0/159098235 >> 172.21.15.83:0/159098235 conn(0x7ff3000d0450 msgr2=0x7ff3001540a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:28.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.029+0000 7ff3061d7640 1 -- 172.21.15.83:0/159098235 shutdown_connections 2024-04-17T10:43:28.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.029+0000 7ff3061d7640 1 -- 172.21.15.83:0/159098235 wait complete. 2024-04-17T10:43:28.041 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub' 2024-04-17T10:43:28.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.161+0000 7f62cb252640 1 Processor -- start 2024-04-17T10:43:28.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.161+0000 7f62cb252640 1 -- start start 2024-04-17T10:43:28.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.165+0000 7f62cb252640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f62c415db50 0x7f62c4070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:28.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.165+0000 7f62cb252640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62c406dcb0 0x7f62c40b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:28.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.165+0000 7f62cb252640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62c40709b0 con 0x7f62c40aedd0 2024-04-17T10:43:28.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.165+0000 7f62cb252640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62c415e440 con 0x7f62c415db50 2024-04-17T10:43:28.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.165+0000 7f62cb252640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62c415e720 con 0x7f62c406dcb0 2024-04-17T10:43:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.165+0000 7f62c8fee640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f62c415db50 0x7f62c4070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.165+0000 7f62c97ef640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62c406dcb0 0x7f62c40b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.165+0000 7f62c3fff640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f62c40aedd0 0x7f62c406d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:52202/0 (socket says 172.21.15.83:52202) 2024-04-17T10:43:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.165+0000 7f62c3fff640 1 -- 172.21.15.83:0/1482771700 learned_addr learned my addr 172.21.15.83:0/1482771700 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:43:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.165+0000 7f62c97ef640 1 -- 172.21.15.83:0/1482771700 >> v2:172.21.15.167:3300/0 conn(0x7f62c415db50 msgr2=0x7f62c4070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.165+0000 7f62c97ef640 1 --2- 172.21.15.83:0/1482771700 >> v2:172.21.15.167:3300/0 conn(0x7f62c415db50 0x7f62c4070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.165+0000 7f62c97ef640 1 -- 172.21.15.83:0/1482771700 >> v1:172.21.15.167:6789/0 conn(0x7f62c40aedd0 legacy=0x7f62c406d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.165+0000 7f62c97ef640 1 -- 172.21.15.83:0/1482771700 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c415ea00 con 0x7f62c406dcb0 2024-04-17T10:43:28.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.169+0000 7f62c97ef640 1 --2- 172.21.15.83:0/1482771700 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62c406dcb0 0x7f62c40b2e10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f62b40097c0 tx=0x7f62b401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=720df1179a687487 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:28.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.169+0000 7f62c37fe640 1 -- 172.21.15.83:0/1482771700 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b40a7070 con 0x7f62c406dcb0 2024-04-17T10:43:28.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.169+0000 7f62c37fe640 1 -- 172.21.15.83:0/1482771700 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62b40b0070 con 0x7f62c406dcb0 2024-04-17T10:43:28.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.169+0000 7f62c37fe640 1 -- 172.21.15.83:0/1482771700 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b40ab890 con 0x7f62c406dcb0 2024-04-17T10:43:28.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.169+0000 7f62cb252640 1 -- 172.21.15.83:0/1482771700 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62c406dcb0 msgr2=0x7f62c40b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:28.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.169+0000 7f62cb252640 1 --2- 172.21.15.83:0/1482771700 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62c406dcb0 0x7f62c40b2e10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f62b40097c0 tx=0x7f62b401c470 comp rx=0 tx=0).stop 2024-04-17T10:43:28.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.169+0000 7f62cb252640 1 -- 172.21.15.83:0/1482771700 shutdown_connections 2024-04-17T10:43:28.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.169+0000 7f62cb252640 1 --2- 172.21.15.83:0/1482771700 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62c406dcb0 0x7f62c40b2e10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:28.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.169+0000 7f62cb252640 1 --2- 172.21.15.83:0/1482771700 >> v2:172.21.15.167:3300/0 conn(0x7f62c415db50 0x7f62c4070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:28.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.169+0000 7f62cb252640 1 -- 172.21.15.83:0/1482771700 >> 172.21.15.83:0/1482771700 conn(0x7f62c406abe0 msgr2=0x7f62c406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:28.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.169+0000 7f62cb252640 1 -- 172.21.15.83:0/1482771700 shutdown_connections 2024-04-17T10:43:28.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.169+0000 7f62cb252640 1 -- 172.21.15.83:0/1482771700 wait complete. 2024-04-17T10:43:28.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.169+0000 7f62cb252640 1 Processor -- start 2024-04-17T10:43:28.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.169+0000 7f62cb252640 1 -- start start 2024-04-17T10:43:28.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.169+0000 7f62cb252640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62c406dcb0 0x7f62c4150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:28.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.169+0000 7f62cb252640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62c40aedd0 0x7f62c40a9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:28.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.169+0000 7f62cb252640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62c415db50 0x7f62c4151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:28.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.169+0000 7f62cb252640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c40b3ce0 con 0x7f62c415db50 2024-04-17T10:43:28.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.169+0000 7f62cb252640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c40b3b60 con 0x7f62c40aedd0 2024-04-17T10:43:28.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.169+0000 7f62cb252640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62c40b3e60 con 0x7f62c406dcb0 2024-04-17T10:43:28.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.169+0000 7f62c97ef640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62c415db50 0x7f62c4151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:28.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.169+0000 7f62c97ef640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62c415db50 0x7f62c4151b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50932/0 (socket says 172.21.15.83:50932) 2024-04-17T10:43:28.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.169+0000 7f62c97ef640 1 -- 172.21.15.83:0/1163297565 learned_addr learned my addr 172.21.15.83:0/1163297565 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:28.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.169+0000 7f62c8fee640 1 --2- 172.21.15.83:0/1163297565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62c406dcb0 0x7f62c4150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:28.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.169+0000 7f62c3fff640 1 --2- 172.21.15.83:0/1163297565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62c40aedd0 0x7f62c40a9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:28.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.169+0000 7f62c97ef640 1 -- 172.21.15.83:0/1163297565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62c406dcb0 msgr2=0x7f62c4150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:28.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.169+0000 7f62c97ef640 1 --2- 172.21.15.83:0/1163297565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62c406dcb0 0x7f62c4150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:28.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.169+0000 7f62c97ef640 1 -- 172.21.15.83:0/1163297565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62c40aedd0 msgr2=0x7f62c40a9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:28.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.169+0000 7f62c97ef640 1 --2- 172.21.15.83:0/1163297565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62c40aedd0 0x7f62c40a9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:28.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.169+0000 7f62c97ef640 1 -- 172.21.15.83:0/1163297565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c41f82f0 con 0x7f62c415db50 2024-04-17T10:43:28.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.173+0000 7f62c3fff640 1 --2- 172.21.15.83:0/1163297565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62c40aedd0 0x7f62c40a9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:28.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.173+0000 7f62c97ef640 1 --2- 172.21.15.83:0/1163297565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62c415db50 0x7f62c4151b30 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f62b40ad040 tx=0x7f62b401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:28.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.173+0000 7f62c17fa640 1 -- 172.21.15.83:0/1163297565 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b40abe60 con 0x7f62c415db50 2024-04-17T10:43:28.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.173+0000 7f62c17fa640 1 -- 172.21.15.83:0/1163297565 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62b40bf610 con 0x7f62c415db50 2024-04-17T10:43:28.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.173+0000 7f62cb252640 1 -- 172.21.15.83:0/1163297565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62c41f85d0 con 0x7f62c415db50 2024-04-17T10:43:28.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.173+0000 7f62c17fa640 1 -- 172.21.15.83:0/1163297565 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b40bf9e0 con 0x7f62c415db50 2024-04-17T10:43:28.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.173+0000 7f62cb252640 1 -- 172.21.15.83:0/1163297565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62c41f8b90 con 0x7f62c415db50 2024-04-17T10:43:28.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.173+0000 7f62cb252640 1 -- 172.21.15.83:0/1163297565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62c4074f90 con 0x7f62c415db50 2024-04-17T10:43:28.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.173+0000 7f62c17fa640 1 -- 172.21.15.83:0/1163297565 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f62b40bd020 con 0x7f62c415db50 2024-04-17T10:43:28.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.173+0000 7f62c17fa640 1 --2- 172.21.15.83:0/1163297565 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f62b0066f90 0x7f62b0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:28.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.173+0000 7f62c8fee640 1 --2- 172.21.15.83:0/1163297565 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f62b0066f90 0x7f62b0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:28.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.173+0000 7f62c17fa640 1 -- 172.21.15.83:0/1163297565 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f62b411f030 con 0x7f62c415db50 2024-04-17T10:43:28.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.173+0000 7f62c8fee640 1 --2- 172.21.15.83:0/1163297565 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f62b0066f90 0x7f62b0069450 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f62bc0073f0 tx=0x7f62bc00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:28.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.177+0000 7f62c17fa640 1 -- 172.21.15.83:0/1163297565 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f62b40c6030 con 0x7f62c415db50 2024-04-17T10:43:28.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.393+0000 7f62cb252640 1 -- 172.21.15.83:0/1163297565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]} v 0) v1 -- 0x7f62c40b2af0 con 0x7f62c415db50 2024-04-17T10:43:28.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.425+0000 7f62c17fa640 1 -- 172.21.15.83:0/1163297565 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]}]=0 v256) v1 ==== 161+0+0 (secure 0 0 0) 0x7f62b4016090 con 0x7f62c415db50 2024-04-17T10:43:28.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.429+0000 7f62cb252640 1 -- 172.21.15.83:0/1163297565 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f62b0066f90 msgr2=0x7f62b0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:28.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.429+0000 7f62cb252640 1 --2- 172.21.15.83:0/1163297565 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f62b0066f90 0x7f62b0069450 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f62bc0073f0 tx=0x7f62bc00b040 comp rx=0 tx=0).stop 2024-04-17T10:43:28.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.429+0000 7f62cb252640 1 -- 172.21.15.83:0/1163297565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62c415db50 msgr2=0x7f62c4151b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:28.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.429+0000 7f62cb252640 1 --2- 172.21.15.83:0/1163297565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62c415db50 0x7f62c4151b30 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f62b40ad040 tx=0x7f62b401acb0 comp rx=0 tx=0).stop 2024-04-17T10:43:28.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.429+0000 7f62cb252640 1 -- 172.21.15.83:0/1163297565 shutdown_connections 2024-04-17T10:43:28.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.429+0000 7f62cb252640 1 --2- 172.21.15.83:0/1163297565 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f62b0066f90 0x7f62b0069450 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:28.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.429+0000 7f62cb252640 1 --2- 172.21.15.83:0/1163297565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62c415db50 0x7f62c4151b30 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:28.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.429+0000 7f62cb252640 1 --2- 172.21.15.83:0/1163297565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62c40aedd0 0x7f62c40a9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:28.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.429+0000 7f62cb252640 1 --2- 172.21.15.83:0/1163297565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62c406dcb0 0x7f62c4150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:28.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.429+0000 7f62cb252640 1 -- 172.21.15.83:0/1163297565 >> 172.21.15.83:0/1163297565 conn(0x7f62c406abe0 msgr2=0x7f62c4155860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:28.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.429+0000 7f62cb252640 1 -- 172.21.15.83:0/1163297565 shutdown_connections 2024-04-17T10:43:28.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.429+0000 7f62cb252640 1 -- 172.21.15.83:0/1163297565 wait complete. 2024-04-17T10:43:28.442 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-17T10:43:28.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.569+0000 7f424e14c640 1 Processor -- start 2024-04-17T10:43:28.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.569+0000 7f424e14c640 1 -- start start 2024-04-17T10:43:28.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.569+0000 7f424e14c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f424815b6e0 0x7f4248071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:28.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.569+0000 7f424e14c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f424815bac0 0x7f42480cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:28.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.569+0000 7f424e14c640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f424806c1f0 con 0x7f4248070470 2024-04-17T10:43:28.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.569+0000 7f424e14c640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42480cdd30 con 0x7f424815b6e0 2024-04-17T10:43:28.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.569+0000 7f424e14c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4248070a10 con 0x7f424815bac0 2024-04-17T10:43:28.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.569+0000 7f4246ffd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f424815bac0 0x7f42480cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:28.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.569+0000 7f42477fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f424815b6e0 0x7f4248071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:28.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.569+0000 7f4247fff640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f4248070470 0x7f424806ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:52204/0 (socket says 172.21.15.83:52204) 2024-04-17T10:43:28.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.569+0000 7f4246ffd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f424815bac0 0x7f42480cd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:35204/0 (socket says 172.21.15.83:35204) 2024-04-17T10:43:28.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.569+0000 7f4247fff640 1 -- 172.21.15.83:0/4237742822 learned_addr learned my addr 172.21.15.83:0/4237742822 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:43:28.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.569+0000 7f42477fe640 1 -- 172.21.15.83:0/4237742822 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f424815bac0 msgr2=0x7f42480cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:28.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.569+0000 7f42477fe640 1 --2- 172.21.15.83:0/4237742822 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f424815bac0 0x7f42480cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:28.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.569+0000 7f42477fe640 1 -- 172.21.15.83:0/4237742822 >> v1:172.21.15.167:6789/0 conn(0x7f4248070470 legacy=0x7f424806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:28.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.569+0000 7f42477fe640 1 -- 172.21.15.83:0/4237742822 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4248070cf0 con 0x7f424815b6e0 2024-04-17T10:43:28.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.569+0000 7f4246ffd640 1 --2- 172.21.15.83:0/4237742822 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f424815bac0 0x7f42480cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:43:28.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.573+0000 7f42477fe640 1 --2- 172.21.15.83:0/4237742822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f424815b6e0 0x7f4248071cc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f42380097c0 tx=0x7f423801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab0b3ad0305a2cc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:28.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.573+0000 7f42467fc640 1 -- 172.21.15.83:0/4237742822 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42380b0070 con 0x7f424815b6e0 2024-04-17T10:43:28.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.573+0000 7f42467fc640 1 -- 172.21.15.83:0/4237742822 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42380ab600 con 0x7f424815b6e0 2024-04-17T10:43:28.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.573+0000 7f42467fc640 1 -- 172.21.15.83:0/4237742822 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42380ab9d0 con 0x7f424815b6e0 2024-04-17T10:43:28.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.573+0000 7f424e14c640 1 -- 172.21.15.83:0/4237742822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f424815b6e0 msgr2=0x7f4248071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:28.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.573+0000 7f424e14c640 1 --2- 172.21.15.83:0/4237742822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f424815b6e0 0x7f4248071cc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f42380097c0 tx=0x7f423801c4a0 comp rx=0 tx=0).stop 2024-04-17T10:43:28.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.573+0000 7f424e14c640 1 -- 172.21.15.83:0/4237742822 shutdown_connections 2024-04-17T10:43:28.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.573+0000 7f424e14c640 1 --2- 172.21.15.83:0/4237742822 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f424815bac0 0x7f42480cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:28.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.573+0000 7f424e14c640 1 --2- 172.21.15.83:0/4237742822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f424815b6e0 0x7f4248071cc0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:28.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.573+0000 7f424e14c640 1 -- 172.21.15.83:0/4237742822 >> 172.21.15.83:0/4237742822 conn(0x7f42480b15b0 msgr2=0x7f42480b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:28.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.573+0000 7f424e14c640 1 -- 172.21.15.83:0/4237742822 shutdown_connections 2024-04-17T10:43:28.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.573+0000 7f424e14c640 1 -- 172.21.15.83:0/4237742822 wait complete. 2024-04-17T10:43:28.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.573+0000 7f424e14c640 1 Processor -- start 2024-04-17T10:43:28.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.573+0000 7f424e14c640 1 -- start start 2024-04-17T10:43:28.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.577+0000 7f424e14c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4248070470 0x7f42480b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:28.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.577+0000 7f424e14c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f424815b6e0 0x7f424814b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:28.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.577+0000 7f424e14c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f424815bac0 0x7f424814f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:28.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.577+0000 7f424e14c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424806cee0 con 0x7f424815b6e0 2024-04-17T10:43:28.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.577+0000 7f424e14c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424806b730 con 0x7f4248070470 2024-04-17T10:43:28.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.577+0000 7f424e14c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f424806d060 con 0x7f424815bac0 2024-04-17T10:43:28.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.577+0000 7f4246ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f424815b6e0 0x7f424814b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:28.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.577+0000 7f4246ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f424815b6e0 0x7f424814b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50946/0 (socket says 172.21.15.83:50946) 2024-04-17T10:43:28.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.577+0000 7f4246ffd640 1 -- 172.21.15.83:0/3029711403 learned_addr learned my addr 172.21.15.83:0/3029711403 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:28.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.577+0000 7f42477fe640 1 --2- 172.21.15.83:0/3029711403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4248070470 0x7f42480b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:28.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.577+0000 7f4247fff640 1 --2- 172.21.15.83:0/3029711403 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f424815bac0 0x7f424814f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:28.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.577+0000 7f4246ffd640 1 -- 172.21.15.83:0/3029711403 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f424815bac0 msgr2=0x7f424814f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:28.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.577+0000 7f4246ffd640 1 --2- 172.21.15.83:0/3029711403 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f424815bac0 0x7f424814f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:28.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.577+0000 7f4246ffd640 1 -- 172.21.15.83:0/3029711403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4248070470 msgr2=0x7f42480b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:28.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.577+0000 7f4246ffd640 1 --2- 172.21.15.83:0/3029711403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4248070470 0x7f42480b2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:28.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.577+0000 7f4246ffd640 1 -- 172.21.15.83:0/3029711403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f424814f6e0 con 0x7f424815b6e0 2024-04-17T10:43:28.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.577+0000 7f4247fff640 1 --2- 172.21.15.83:0/3029711403 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f424815bac0 0x7f424814f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:28.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.577+0000 7f4246ffd640 1 --2- 172.21.15.83:0/3029711403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f424815b6e0 0x7f424814b310 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f423c00e960 tx=0x7f423c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:28.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.577+0000 7f422ffff640 1 -- 172.21.15.83:0/3029711403 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c017070 con 0x7f424815b6e0 2024-04-17T10:43:28.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.577+0000 7f422ffff640 1 -- 172.21.15.83:0/3029711403 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423c00ca00 con 0x7f424815b6e0 2024-04-17T10:43:28.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.577+0000 7f424e14c640 1 -- 172.21.15.83:0/3029711403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42480b34d0 con 0x7f424815b6e0 2024-04-17T10:43:28.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.577+0000 7f422ffff640 1 -- 172.21.15.83:0/3029711403 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c00cdd0 con 0x7f424815b6e0 2024-04-17T10:43:28.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.577+0000 7f424e14c640 1 -- 172.21.15.83:0/3029711403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42480b3a90 con 0x7f424815b6e0 2024-04-17T10:43:28.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.577+0000 7f424e14c640 1 -- 172.21.15.83:0/3029711403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4248074f90 con 0x7f424815b6e0 2024-04-17T10:43:28.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.577+0000 7f422ffff640 1 -- 172.21.15.83:0/3029711403 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f423c01a070 con 0x7f424815b6e0 2024-04-17T10:43:28.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.577+0000 7f422ffff640 1 --2- 172.21.15.83:0/3029711403 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4234066f60 0x7f4234069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:28.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.577+0000 7f422ffff640 1 -- 172.21.15.83:0/3029711403 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f423c084bf0 con 0x7f424815b6e0 2024-04-17T10:43:28.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.577+0000 7f42477fe640 1 --2- 172.21.15.83:0/3029711403 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4234066f60 0x7f4234069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:28.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.581+0000 7f42477fe640 1 --2- 172.21.15.83:0/3029711403 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4234066f60 0x7f4234069420 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f42380ad000 tx=0x7f42380a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:28.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.585+0000 7f422ffff640 1 -- 172.21.15.83:0/3029711403 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f423c055f30 con 0x7f424815b6e0 2024-04-17T10:43:28.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.741+0000 7f424e14c640 1 -- 172.21.15.83:0/3029711403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f4248153d60 con 0x7f424815b6e0 2024-04-17T10:43:28.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.749+0000 7f422ffff640 1 -- 172.21.15.83:0/3029711403 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f423c055830 con 0x7f424815b6e0 2024-04-17T10:43:28.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.749+0000 7f424e14c640 1 -- 172.21.15.83:0/3029711403 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4234066f60 msgr2=0x7f4234069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:28.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.749+0000 7f424e14c640 1 --2- 172.21.15.83:0/3029711403 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4234066f60 0x7f4234069420 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f42380ad000 tx=0x7f42380a6000 comp rx=0 tx=0).stop 2024-04-17T10:43:28.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.749+0000 7f424e14c640 1 -- 172.21.15.83:0/3029711403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f424815b6e0 msgr2=0x7f424814b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:28.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.749+0000 7f424e14c640 1 --2- 172.21.15.83:0/3029711403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f424815b6e0 0x7f424814b310 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f423c00e960 tx=0x7f423c00ee20 comp rx=0 tx=0).stop 2024-04-17T10:43:28.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.749+0000 7f424e14c640 1 -- 172.21.15.83:0/3029711403 shutdown_connections 2024-04-17T10:43:28.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.749+0000 7f424e14c640 1 --2- 172.21.15.83:0/3029711403 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4234066f60 0x7f4234069420 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:28.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.749+0000 7f424e14c640 1 --2- 172.21.15.83:0/3029711403 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f424815bac0 0x7f424814f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:28.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.749+0000 7f424e14c640 1 --2- 172.21.15.83:0/3029711403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f424815b6e0 0x7f424814b310 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:28.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.749+0000 7f424e14c640 1 --2- 172.21.15.83:0/3029711403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4248070470 0x7f42480b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:28.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.749+0000 7f424e14c640 1 -- 172.21.15.83:0/3029711403 >> 172.21.15.83:0/3029711403 conn(0x7f42480b15b0 msgr2=0x7f424806ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:28.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.749+0000 7f424e14c640 1 -- 172.21.15.83:0/3029711403 shutdown_connections 2024-04-17T10:43:28.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:28.749+0000 7f424e14c640 1 -- 172.21.15.83:0/3029711403 wait complete. 2024-04-17T10:43:28.763 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T10:43:28.763 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:43:28.797 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:43:28.847 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T10:43:28.848 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:43:28.848 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:43:28.853 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16063 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:43:28.853 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:43:28.853 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:43:28.859 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:43:28.859 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:43:28.859 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:43:28.899 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16063 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:43:28.900 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:43:28.900 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:43:28.908 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:43:28.908 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-17T10:43:28.908 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:43:29.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.077+0000 7fe2dc82b640 1 Processor -- start 2024-04-17T10:43:29.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.077+0000 7fe2dc82b640 1 -- start start 2024-04-17T10:43:29.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.077+0000 7fe2dc82b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe2d806dcb0 0x7fe2d8165650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:29.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.077+0000 7fe2dc82b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2d815e440 con 0x7fe2d815db50 2024-04-17T10:43:29.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.077+0000 7fe2dc82b640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2d815e720 con 0x7fe2d80aedd0 2024-04-17T10:43:29.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.077+0000 7fe2dc82b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2d815ea00 con 0x7fe2d806dcb0 2024-04-17T10:43:29.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.077+0000 7fe2d6d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe2d806dcb0 0x7fe2d8165650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:29.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.077+0000 7fe2d659c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe2d815db50 0x7fe2d8070470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:43456/0 (socket says 172.21.15.83:43456) 2024-04-17T10:43:29.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.077+0000 7fe2d659c640 1 -- 172.21.15.83:0/93705867 learned_addr learned my addr 172.21.15.83:0/93705867 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:43:29.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.077+0000 7fe2d6d9d640 1 -- 172.21.15.83:0/93705867 >> v1:172.21.15.167:6789/0 conn(0x7fe2d80aedd0 legacy=0x7fe2d806d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:29.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.077+0000 7fe2d6d9d640 1 -- 172.21.15.83:0/93705867 >> v1:172.21.15.83:6789/0 conn(0x7fe2d815db50 legacy=0x7fe2d8070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:29.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.077+0000 7fe2d5d9b640 1 -- 172.21.15.83:0/93705867 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2456760030 0 0) 0x7fe2d815e440 con 0x7fe2d815db50 2024-04-17T10:43:29.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.077+0000 7fe2d6d9d640 1 -- 172.21.15.83:0/93705867 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d815ece0 con 0x7fe2d806dcb0 2024-04-17T10:43:29.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2d6d9d640 1 --2- 172.21.15.83:0/93705867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2d806dcb0 0x7fe2d8165650 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0009dc0 tx=0x7fe2d001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f87fb3f92e17708 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:29.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2d5d9b640 1 -- 172.21.15.83:0/93705867 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d00b2070 con 0x7fe2d806dcb0 2024-04-17T10:43:29.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2d5d9b640 1 -- 172.21.15.83:0/93705867 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d00a58f0 con 0x7fe2d806dcb0 2024-04-17T10:43:29.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2d5d9b640 1 -- 172.21.15.83:0/93705867 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d00a5cc0 con 0x7fe2d806dcb0 2024-04-17T10:43:29.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2dc82b640 1 -- 172.21.15.83:0/93705867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2d806dcb0 msgr2=0x7fe2d8165650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:29.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2dc82b640 1 --2- 172.21.15.83:0/93705867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2d806dcb0 0x7fe2d8165650 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0009dc0 tx=0x7fe2d001c920 comp rx=0 tx=0).stop 2024-04-17T10:43:29.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2dc82b640 1 -- 172.21.15.83:0/93705867 shutdown_connections 2024-04-17T10:43:29.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2dc82b640 1 --2- 172.21.15.83:0/93705867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2d806dcb0 0x7fe2d8165650 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:29.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2dc82b640 1 -- 172.21.15.83:0/93705867 >> 172.21.15.83:0/93705867 conn(0x7fe2d806abe0 msgr2=0x7fe2d806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:29.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2dc82b640 1 -- 172.21.15.83:0/93705867 shutdown_connections 2024-04-17T10:43:29.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2dc82b640 1 -- 172.21.15.83:0/93705867 wait complete. 2024-04-17T10:43:29.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2dc82b640 1 Processor -- start 2024-04-17T10:43:29.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2dc82b640 1 -- start start 2024-04-17T10:43:29.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2dc82b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2d806dcb0 0x7fe2d815be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:29.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2dc82b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2d80aedd0 0x7fe2d8156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:29.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2dc82b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2d815db50 0x7fe2d815fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:29.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2dc82b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d8168b50 con 0x7fe2d815db50 2024-04-17T10:43:29.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2dc82b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d81689d0 con 0x7fe2d80aedd0 2024-04-17T10:43:29.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2dc82b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d8168cd0 con 0x7fe2d806dcb0 2024-04-17T10:43:29.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2d6d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2d815db50 0x7fe2d815fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:29.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2d6d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2d815db50 0x7fe2d815fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50976/0 (socket says 172.21.15.83:50976) 2024-04-17T10:43:29.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2d6d9d640 1 -- 172.21.15.83:0/2823703252 learned_addr learned my addr 172.21.15.83:0/2823703252 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:29.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2d6d9d640 1 -- 172.21.15.83:0/2823703252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2d806dcb0 msgr2=0x7fe2d815be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:29.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2d6d9d640 1 --2- 172.21.15.83:0/2823703252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2d806dcb0 0x7fe2d815be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:29.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2d6d9d640 1 -- 172.21.15.83:0/2823703252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2d80aedd0 msgr2=0x7fe2d8156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:29.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2cffff640 1 --2- 172.21.15.83:0/2823703252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2d80aedd0 0x7fe2d8156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:29.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2d6d9d640 1 --2- 172.21.15.83:0/2823703252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2d80aedd0 0x7fe2d8156400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:29.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2d6d9d640 1 -- 172.21.15.83:0/2823703252 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d81f6d60 con 0x7fe2d815db50 2024-04-17T10:43:29.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2d6d9d640 1 --2- 172.21.15.83:0/2823703252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2d815db50 0x7fe2d815fc60 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0019530 tx=0x7fe2d00196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:29.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2ceffd640 1 -- 172.21.15.83:0/2823703252 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d00b2040 con 0x7fe2d815db50 2024-04-17T10:43:29.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2ceffd640 1 -- 172.21.15.83:0/2823703252 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d00bf070 con 0x7fe2d815db50 2024-04-17T10:43:29.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2ceffd640 1 -- 172.21.15.83:0/2823703252 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d00ad900 con 0x7fe2d815db50 2024-04-17T10:43:29.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.081+0000 7fe2dc82b640 1 -- 172.21.15.83:0/2823703252 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d815c680 con 0x7fe2d815db50 2024-04-17T10:43:29.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.085+0000 7fe2dc82b640 1 -- 172.21.15.83:0/2823703252 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d815cf20 con 0x7fe2d815db50 2024-04-17T10:43:29.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.085+0000 7fe2dc82b640 1 -- 172.21.15.83:0/2823703252 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2d8074f90 con 0x7fe2d815db50 2024-04-17T10:43:29.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.085+0000 7fe2ceffd640 1 -- 172.21.15.83:0/2823703252 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe2d00adaa0 con 0x7fe2d815db50 2024-04-17T10:43:29.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.085+0000 7fe2ceffd640 1 --2- 172.21.15.83:0/2823703252 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe2b8066f60 0x7fe2b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:29.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.085+0000 7fe2ceffd640 1 -- 172.21.15.83:0/2823703252 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fe2d011ff80 con 0x7fe2d815db50 2024-04-17T10:43:29.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.085+0000 7fe2d659c640 1 --2- 172.21.15.83:0/2823703252 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe2b8066f60 0x7fe2b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:29.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.085+0000 7fe2d659c640 1 --2- 172.21.15.83:0/2823703252 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe2b8066f60 0x7fe2b8069420 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8000c10 tx=0x7fe2c80079d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:29.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.089+0000 7fe2ceffd640 1 -- 172.21.15.83:0/2823703252 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe2d0016540 con 0x7fe2d815db50 2024-04-17T10:43:29.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.281+0000 7fe2dc82b640 1 -- 172.21.15.83:0/2823703252 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2d80aebb0 con 0x7fe2d815db50 2024-04-17T10:43:29.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.281+0000 7fe2ceffd640 1 -- 172.21.15.83:0/2823703252 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+3699 (secure 0 0 0) 0x7fe2d00f1350 con 0x7fe2d815db50 2024-04-17T10:43:29.278 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:43:29.278 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":41,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5165,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1924992370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1924992370},{"type":"v1","addr":"172.21.15.167:6833","nonce":1924992370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5174,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3876293847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3876293847},{"type":"v1","addr":"172.21.15.167:6835","nonce":3876293847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6831,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/338926944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":338926944},{"type":"v1","addr":"172.21.15.83:6835","nonce":338926944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":7020,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/661048582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":661048582},{"type":"v1","addr":"172.21.15.83:6837","nonce":661048582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[]} 2024-04-17T10:43:29.278 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 41 2024-04-17T10:43:29.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.285+0000 7fe2dc82b640 1 -- 172.21.15.83:0/2823703252 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe2b8066f60 msgr2=0x7fe2b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:29.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.285+0000 7fe2dc82b640 1 --2- 172.21.15.83:0/2823703252 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe2b8066f60 0x7fe2b8069420 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8000c10 tx=0x7fe2c80079d0 comp rx=0 tx=0).stop 2024-04-17T10:43:29.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.285+0000 7fe2dc82b640 1 -- 172.21.15.83:0/2823703252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2d815db50 msgr2=0x7fe2d815fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:29.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.285+0000 7fe2dc82b640 1 --2- 172.21.15.83:0/2823703252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2d815db50 0x7fe2d815fc60 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0019530 tx=0x7fe2d00196c0 comp rx=0 tx=0).stop 2024-04-17T10:43:29.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.285+0000 7fe2dc82b640 1 -- 172.21.15.83:0/2823703252 shutdown_connections 2024-04-17T10:43:29.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.285+0000 7fe2dc82b640 1 --2- 172.21.15.83:0/2823703252 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe2b8066f60 0x7fe2b8069420 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:29.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.285+0000 7fe2dc82b640 1 --2- 172.21.15.83:0/2823703252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2d815db50 0x7fe2d815fc60 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:29.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.285+0000 7fe2dc82b640 1 --2- 172.21.15.83:0/2823703252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2d80aedd0 0x7fe2d8156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:29.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.285+0000 7fe2dc82b640 1 --2- 172.21.15.83:0/2823703252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2d806dcb0 0x7fe2d815be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:29.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.285+0000 7fe2dc82b640 1 -- 172.21.15.83:0/2823703252 >> 172.21.15.83:0/2823703252 conn(0x7fe2d806abe0 msgr2=0x7fe2d81573f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:29.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.285+0000 7fe2dc82b640 1 -- 172.21.15.83:0/2823703252 shutdown_connections 2024-04-17T10:43:29.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.285+0000 7fe2dc82b640 1 -- 172.21.15.83:0/2823703252 wait complete. 2024-04-17T10:43:29.296 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-17T10:43:29.296 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-17T10:43:29.297 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-17T10:43:29.297 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:43:29.297 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-17T10:43:29.297 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-17T10:43:29.297 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-17T10:43:29.297 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:43:29.298 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-17T10:43:29.298 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-17T10:43:29.298 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-17T10:43:29.298 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:43:29.298 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-17T10:43:29.302 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-17T10:43:29.302 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-17T10:43:29.302 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:43:29.306 INFO:tasks.ceph.mds.a:Stopped 2024-04-17T10:43:29.306 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-17T10:43:29.307 INFO:tasks.ceph.mds.c:Stopped 2024-04-17T10:43:29.308 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-17T10:43:29.315 INFO:tasks.ceph.mds.d:Stopped 2024-04-17T10:43:29.316 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-17T10:43:29.317 INFO:tasks.ceph.mds.b:Stopped 2024-04-17T10:43:29.317 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-17T10:43:29.356 INFO:tasks.ceph.mds.a:Started 2024-04-17T10:43:29.356 DEBUG:teuthology.parallel:result is None 2024-04-17T10:43:29.357 INFO:tasks.ceph.mds.c:Started 2024-04-17T10:43:29.357 DEBUG:teuthology.parallel:result is None 2024-04-17T10:43:29.359 INFO:tasks.ceph.mds.d:Started 2024-04-17T10:43:29.359 DEBUG:teuthology.parallel:result is None 2024-04-17T10:43:29.360 INFO:tasks.ceph.mds.b:Started 2024-04-17T10:43:29.360 DEBUG:teuthology.parallel:result is None 2024-04-17T10:43:29.361 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-17T10:43:29.418 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-04-17T10:43:29.418 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-04-17T10:43:29.429 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2024-04-17T10:43:29.436 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2024-04-17T10:43:29.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.481+0000 7f75c1449640 1 Processor -- start 2024-04-17T10:43:29.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.481+0000 7f75c1449640 1 -- start start 2024-04-17T10:43:29.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.481+0000 7f75c1449640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f75bc0ce400 0x7f75bc0b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:29.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.481+0000 7f75c1449640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75bc0b5450 0x7f75bc0b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:29.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.481+0000 7f75c1449640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75bc15e4d0 con 0x7f75bc15e0f0 2024-04-17T10:43:29.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.481+0000 7f75c1449640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75bc162b40 con 0x7f75bc0ce400 2024-04-17T10:43:29.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.481+0000 7f75c1449640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc162320 con 0x7f75bc0b5450 2024-04-17T10:43:29.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.481+0000 7f75ba7fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f75bc0ce400 0x7f75bc0b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:29.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.481+0000 7f75ba7fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f75bc0ce400 0x7f75bc0b6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51020/0 (socket says 172.21.15.83:51020) 2024-04-17T10:43:29.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.481+0000 7f75ba7fc640 1 -- 172.21.15.83:0/912800079 learned_addr learned my addr 172.21.15.83:0/912800079 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:29.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.481+0000 7f75bb7fe640 1 --2- 172.21.15.83:0/912800079 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75bc0b5450 0x7f75bc0b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:29.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.481+0000 7f75ba7fc640 1 -- 172.21.15.83:0/912800079 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75bc0b5450 msgr2=0x7f75bc0b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:29.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.481+0000 7f75ba7fc640 1 --2- 172.21.15.83:0/912800079 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75bc0b5450 0x7f75bc0b58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:29.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.481+0000 7f75ba7fc640 1 -- 172.21.15.83:0/912800079 >> v1:172.21.15.83:6789/0 conn(0x7f75bc15e0f0 legacy=0x7f75bc161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:29.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.481+0000 7f75ba7fc640 1 -- 172.21.15.83:0/912800079 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75bc162580 con 0x7f75bc0ce400 2024-04-17T10:43:29.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.481+0000 7f75b9ffb640 1 -- 172.21.15.83:0/912800079 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1623830325 0 0) 0x7f75bc15e4d0 con 0x7f75bc15e0f0 2024-04-17T10:43:29.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.485+0000 7f75ba7fc640 1 --2- 172.21.15.83:0/912800079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75bc0ce400 0x7f75bc0b6d00 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f75b00098a0 tx=0x7f75b001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1331bd480040bcf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:29.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.485+0000 7f75b9ffb640 1 -- 172.21.15.83:0/912800079 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b00b0070 con 0x7f75bc0ce400 2024-04-17T10:43:29.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.485+0000 7f75b9ffb640 1 -- 172.21.15.83:0/912800079 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b00ab600 con 0x7f75bc0ce400 2024-04-17T10:43:29.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.485+0000 7f75b9ffb640 1 -- 172.21.15.83:0/912800079 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b00ab9d0 con 0x7f75bc0ce400 2024-04-17T10:43:29.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.485+0000 7f75c1449640 1 -- 172.21.15.83:0/912800079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75bc0ce400 msgr2=0x7f75bc0b6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:29.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.485+0000 7f75c1449640 1 --2- 172.21.15.83:0/912800079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75bc0ce400 0x7f75bc0b6d00 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f75b00098a0 tx=0x7f75b001c4b0 comp rx=0 tx=0).stop 2024-04-17T10:43:29.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.485+0000 7f75c1449640 1 -- 172.21.15.83:0/912800079 shutdown_connections 2024-04-17T10:43:29.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.485+0000 7f75c1449640 1 --2- 172.21.15.83:0/912800079 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75bc0b5450 0x7f75bc0b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:29.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.485+0000 7f75c1449640 1 --2- 172.21.15.83:0/912800079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75bc0ce400 0x7f75bc0b6d00 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:29.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.485+0000 7f75c1449640 1 -- 172.21.15.83:0/912800079 >> 172.21.15.83:0/912800079 conn(0x7f75bc0d0450 msgr2=0x7f75bc0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:29.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.485+0000 7f75c1449640 1 -- 172.21.15.83:0/912800079 shutdown_connections 2024-04-17T10:43:29.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.485+0000 7f75c1449640 1 -- 172.21.15.83:0/912800079 wait complete. 2024-04-17T10:43:29.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.485+0000 7f75c1449640 1 Processor -- start 2024-04-17T10:43:29.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.485+0000 7f75c1449640 1 -- start start 2024-04-17T10:43:29.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.485+0000 7f75c1449640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75bc0b5450 0x7f75bc14bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:29.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.485+0000 7f75c1449640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75bc0ce400 0x7f75bc1541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:29.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.485+0000 7f75c1449640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75bc15e0f0 0x7f75bc1562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:29.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.485+0000 7f75c1449640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc0d2ed0 con 0x7f75bc0ce400 2024-04-17T10:43:29.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.485+0000 7f75c1449640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc0d2d50 con 0x7f75bc0b5450 2024-04-17T10:43:29.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.485+0000 7f75c1449640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc0d3050 con 0x7f75bc15e0f0 2024-04-17T10:43:29.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.485+0000 7f75ba7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75bc0ce400 0x7f75bc1541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:29.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.485+0000 7f75ba7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75bc0ce400 0x7f75bc1541b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51034/0 (socket says 172.21.15.83:51034) 2024-04-17T10:43:29.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.485+0000 7f75ba7fc640 1 -- 172.21.15.83:0/459501958 learned_addr learned my addr 172.21.15.83:0/459501958 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:29.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.485+0000 7f75bb7fe640 1 --2- 172.21.15.83:0/459501958 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75bc15e0f0 0x7f75bc1562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:29.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.485+0000 7f75baffd640 1 --2- 172.21.15.83:0/459501958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75bc0b5450 0x7f75bc14bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:29.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.485+0000 7f75ba7fc640 1 -- 172.21.15.83:0/459501958 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75bc15e0f0 msgr2=0x7f75bc1562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:29.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.485+0000 7f75ba7fc640 1 --2- 172.21.15.83:0/459501958 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75bc15e0f0 0x7f75bc1562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:29.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.485+0000 7f75ba7fc640 1 -- 172.21.15.83:0/459501958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75bc0b5450 msgr2=0x7f75bc14bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:29.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.485+0000 7f75ba7fc640 1 --2- 172.21.15.83:0/459501958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75bc0b5450 0x7f75bc14bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:29.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.485+0000 7f75ba7fc640 1 -- 172.21.15.83:0/459501958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75bc14b4d0 con 0x7f75bc0ce400 2024-04-17T10:43:29.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.489+0000 7f75ba7fc640 1 --2- 172.21.15.83:0/459501958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75bc0ce400 0x7f75bc1541b0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f75b00098a0 tx=0x7f75b0019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:29.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.489+0000 7f75a37fe640 1 -- 172.21.15.83:0/459501958 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b00b0040 con 0x7f75bc0ce400 2024-04-17T10:43:29.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.489+0000 7f75a37fe640 1 -- 172.21.15.83:0/459501958 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b00b9070 con 0x7f75bc0ce400 2024-04-17T10:43:29.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.489+0000 7f75c1449640 1 -- 172.21.15.83:0/459501958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75bc1598c0 con 0x7f75bc0ce400 2024-04-17T10:43:29.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.489+0000 7f75a37fe640 1 -- 172.21.15.83:0/459501958 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b00b4660 con 0x7f75bc0ce400 2024-04-17T10:43:29.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.489+0000 7f75c1449640 1 -- 172.21.15.83:0/459501958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75bc0c9440 con 0x7f75bc0ce400 2024-04-17T10:43:29.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.489+0000 7f75c1449640 1 -- 172.21.15.83:0/459501958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75bc0681a0 con 0x7f75bc0ce400 2024-04-17T10:43:29.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.489+0000 7f75a37fe640 1 -- 172.21.15.83:0/459501958 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f75b00b4800 con 0x7f75bc0ce400 2024-04-17T10:43:29.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.489+0000 7f75a37fe640 1 --2- 172.21.15.83:0/459501958 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f75a8066f60 0x7f75a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:29.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.489+0000 7f75a37fe640 1 -- 172.21.15.83:0/459501958 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f75b011e920 con 0x7f75bc0ce400 2024-04-17T10:43:29.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.489+0000 7f75baffd640 1 --2- 172.21.15.83:0/459501958 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f75a8066f60 0x7f75a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:29.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.489+0000 7f75baffd640 1 --2- 172.21.15.83:0/459501958 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f75a8066f60 0x7f75a8069420 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f75ac000f70 tx=0x7f75ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:29.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.493+0000 7f75a37fe640 1 -- 172.21.15.83:0/459501958 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f75b0016130 con 0x7f75bc0ce400 2024-04-17T10:43:29.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:29.657+0000 7f75c1449640 1 -- 172.21.15.83:0/459501958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f75bc074f90 con 0x7f75bc0ce400 2024-04-17T10:43:30.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.445+0000 7f75a37fe640 1 -- 172.21.15.83:0/459501958 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v68) v1 ==== 97+0+0 (secure 0 0 0) 0x7f75b00efc60 con 0x7f75bc0ce400 2024-04-17T10:43:30.441 INFO:teuthology.orchestra.run.smithi083.stderr: removed all blocklist entries 2024-04-17T10:43:30.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.449+0000 7f75c1449640 1 -- 172.21.15.83:0/459501958 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f75a8066f60 msgr2=0x7f75a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:30.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.449+0000 7f75c1449640 1 --2- 172.21.15.83:0/459501958 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f75a8066f60 0x7f75a8069420 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f75ac000f70 tx=0x7f75ac009040 comp rx=0 tx=0).stop 2024-04-17T10:43:30.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.449+0000 7f75c1449640 1 -- 172.21.15.83:0/459501958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75bc0ce400 msgr2=0x7f75bc1541b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:30.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.449+0000 7f75c1449640 1 --2- 172.21.15.83:0/459501958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75bc0ce400 0x7f75bc1541b0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f75b00098a0 tx=0x7f75b0019250 comp rx=0 tx=0).stop 2024-04-17T10:43:30.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.449+0000 7f75c1449640 1 -- 172.21.15.83:0/459501958 shutdown_connections 2024-04-17T10:43:30.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.449+0000 7f75c1449640 1 --2- 172.21.15.83:0/459501958 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f75a8066f60 0x7f75a8069420 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:30.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.449+0000 7f75c1449640 1 --2- 172.21.15.83:0/459501958 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75bc15e0f0 0x7f75bc1562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:30.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.449+0000 7f75c1449640 1 --2- 172.21.15.83:0/459501958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75bc0ce400 0x7f75bc1541b0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:30.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.449+0000 7f75c1449640 1 --2- 172.21.15.83:0/459501958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75bc0b5450 0x7f75bc14bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:30.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.449+0000 7f75c1449640 1 -- 172.21.15.83:0/459501958 >> 172.21.15.83:0/459501958 conn(0x7f75bc0d0450 msgr2=0x7f75bc0c9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:30.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.449+0000 7f75c1449640 1 -- 172.21.15.83:0/459501958 shutdown_connections 2024-04-17T10:43:30.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.449+0000 7f75c1449640 1 -- 172.21.15.83:0/459501958 wait complete. 2024-04-17T10:43:30.468 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-17T10:43:30.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.589+0000 7f0d35698640 1 Processor -- start 2024-04-17T10:43:30.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.589+0000 7f0d35698640 1 -- start start 2024-04-17T10:43:30.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.589+0000 7f0d35698640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0d300d2730 0x7f0d3006b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:30.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.589+0000 7f0d35698640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d3015e1b0 0x7f0d3006b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:30.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.589+0000 7f0d35698640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d300d1940 con 0x7f0d3015ddd0 2024-04-17T10:43:30.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.589+0000 7f0d35698640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d300cfc70 con 0x7f0d300d2730 2024-04-17T10:43:30.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.589+0000 7f0d35698640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d300d1c20 con 0x7f0d3015e1b0 2024-04-17T10:43:30.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.589+0000 7f0d2e7fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0d300d2730 0x7f0d3006b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:30.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.589+0000 7f0d2e7fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0d300d2730 0x7f0d3006b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51040/0 (socket says 172.21.15.83:51040) 2024-04-17T10:43:30.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.589+0000 7f0d2e7fc640 1 -- 172.21.15.83:0/3000609539 learned_addr learned my addr 172.21.15.83:0/3000609539 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:30.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.589+0000 7f0d2f7fe640 1 --2- 172.21.15.83:0/3000609539 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d3015e1b0 0x7f0d3006b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:30.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.589+0000 7f0d2e7fc640 1 -- 172.21.15.83:0/3000609539 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d3015e1b0 msgr2=0x7f0d3006b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:30.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.589+0000 7f0d2e7fc640 1 --2- 172.21.15.83:0/3000609539 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d3015e1b0 0x7f0d3006b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:30.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.589+0000 7f0d2e7fc640 1 -- 172.21.15.83:0/3000609539 >> v1:172.21.15.167:6789/0 conn(0x7f0d3015ddd0 legacy=0x7f0d300d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:30.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.589+0000 7f0d2e7fc640 1 -- 172.21.15.83:0/3000609539 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d300cf900 con 0x7f0d300d2730 2024-04-17T10:43:30.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.593+0000 7f0d2e7fc640 1 --2- 172.21.15.83:0/3000609539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d300d2730 0x7f0d3006b3d0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f0d24009870 tx=0x7f0d2401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a907443796edce server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:30.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.593+0000 7f0d2dffb640 1 -- 172.21.15.83:0/3000609539 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d240a7070 con 0x7f0d300d2730 2024-04-17T10:43:30.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.593+0000 7f0d2dffb640 1 -- 172.21.15.83:0/3000609539 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d240b0070 con 0x7f0d300d2730 2024-04-17T10:43:30.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.593+0000 7f0d2dffb640 1 -- 172.21.15.83:0/3000609539 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d240ab890 con 0x7f0d300d2730 2024-04-17T10:43:30.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.593+0000 7f0d35698640 1 -- 172.21.15.83:0/3000609539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d300d2730 msgr2=0x7f0d3006b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:30.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.593+0000 7f0d35698640 1 --2- 172.21.15.83:0/3000609539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d300d2730 0x7f0d3006b3d0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f0d24009870 tx=0x7f0d2401c490 comp rx=0 tx=0).stop 2024-04-17T10:43:30.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.593+0000 7f0d35698640 1 -- 172.21.15.83:0/3000609539 shutdown_connections 2024-04-17T10:43:30.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.593+0000 7f0d35698640 1 --2- 172.21.15.83:0/3000609539 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d3015e1b0 0x7f0d3006b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:30.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.593+0000 7f0d35698640 1 --2- 172.21.15.83:0/3000609539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d300d2730 0x7f0d3006b3d0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:30.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.593+0000 7f0d35698640 1 -- 172.21.15.83:0/3000609539 >> 172.21.15.83:0/3000609539 conn(0x7f0d300d3a40 msgr2=0x7f0d300d3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:30.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.593+0000 7f0d35698640 1 -- 172.21.15.83:0/3000609539 shutdown_connections 2024-04-17T10:43:30.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.593+0000 7f0d35698640 1 -- 172.21.15.83:0/3000609539 wait complete. 2024-04-17T10:43:30.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.593+0000 7f0d35698640 1 Processor -- start 2024-04-17T10:43:30.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.593+0000 7f0d35698640 1 -- start start 2024-04-17T10:43:30.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.597+0000 7f0d35698640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d300d2730 0x7f0d301597e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:30.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.597+0000 7f0d35698640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d3015ddd0 0x7f0d30158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:30.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.597+0000 7f0d35698640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d3015e1b0 0x7f0d300cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:30.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.597+0000 7f0d35698640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3016af50 con 0x7f0d300d2730 2024-04-17T10:43:30.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.597+0000 7f0d35698640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3016add0 con 0x7f0d3015e1b0 2024-04-17T10:43:30.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.597+0000 7f0d35698640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3016b0d0 con 0x7f0d3015ddd0 2024-04-17T10:43:30.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.597+0000 7f0d2effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d300d2730 0x7f0d301597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:30.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.597+0000 7f0d2effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d300d2730 0x7f0d301597e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51050/0 (socket says 172.21.15.83:51050) 2024-04-17T10:43:30.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.597+0000 7f0d2effd640 1 -- 172.21.15.83:0/4169408017 learned_addr learned my addr 172.21.15.83:0/4169408017 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:30.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.597+0000 7f0d2f7fe640 1 --2- 172.21.15.83:0/4169408017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d3015e1b0 0x7f0d300cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:30.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.597+0000 7f0d2e7fc640 1 --2- 172.21.15.83:0/4169408017 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d3015ddd0 0x7f0d30158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:30.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.597+0000 7f0d2effd640 1 -- 172.21.15.83:0/4169408017 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d3015ddd0 msgr2=0x7f0d30158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:30.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.597+0000 7f0d2effd640 1 --2- 172.21.15.83:0/4169408017 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d3015ddd0 0x7f0d30158880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:30.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.597+0000 7f0d2effd640 1 -- 172.21.15.83:0/4169408017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d3015e1b0 msgr2=0x7f0d300cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:30.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.597+0000 7f0d2effd640 1 --2- 172.21.15.83:0/4169408017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d3015e1b0 0x7f0d300cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:30.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.597+0000 7f0d2effd640 1 -- 172.21.15.83:0/4169408017 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d3015b9a0 con 0x7f0d300d2730 2024-04-17T10:43:30.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.597+0000 7f0d2f7fe640 1 --2- 172.21.15.83:0/4169408017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d3015e1b0 0x7f0d300cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:30.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.597+0000 7f0d2effd640 1 --2- 172.21.15.83:0/4169408017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d300d2730 0x7f0d301597e0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f0d2000c480 tx=0x7f0d2000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:30.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.597+0000 7f0d177fe640 1 -- 172.21.15.83:0/4169408017 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d20012830 con 0x7f0d300d2730 2024-04-17T10:43:30.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.597+0000 7f0d177fe640 1 -- 172.21.15.83:0/4169408017 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d20017070 con 0x7f0d300d2730 2024-04-17T10:43:30.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.597+0000 7f0d35698640 1 -- 172.21.15.83:0/4169408017 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d3015ad90 con 0x7f0d300d2730 2024-04-17T10:43:30.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.597+0000 7f0d177fe640 1 -- 172.21.15.83:0/4169408017 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d20012c30 con 0x7f0d300d2730 2024-04-17T10:43:30.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.597+0000 7f0d35698640 1 -- 172.21.15.83:0/4169408017 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d3015b350 con 0x7f0d300d2730 2024-04-17T10:43:30.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.597+0000 7f0d35698640 1 -- 172.21.15.83:0/4169408017 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d300681a0 con 0x7f0d300d2730 2024-04-17T10:43:30.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.597+0000 7f0d177fe640 1 -- 172.21.15.83:0/4169408017 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0d2001c460 con 0x7f0d300d2730 2024-04-17T10:43:30.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.597+0000 7f0d177fe640 1 --2- 172.21.15.83:0/4169408017 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0d1c066f60 0x7f0d1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:30.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.597+0000 7f0d177fe640 1 -- 172.21.15.83:0/4169408017 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0d20084b00 con 0x7f0d300d2730 2024-04-17T10:43:30.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.597+0000 7f0d2e7fc640 1 --2- 172.21.15.83:0/4169408017 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0d1c066f60 0x7f0d1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:30.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.597+0000 7f0d2e7fc640 1 --2- 172.21.15.83:0/4169408017 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0d1c066f60 0x7f0d1c069420 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0d240af000 tx=0x7f0d24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:30.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.601+0000 7f0d177fe640 1 -- 172.21.15.83:0/4169408017 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0d20055ef0 con 0x7f0d300d2730 2024-04-17T10:43:30.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.801+0000 7f0d35698640 1 -- 172.21.15.83:0/4169408017 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0d30074f90 con 0x7f0d300d2730 2024-04-17T10:43:30.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.805+0000 7f0d177fe640 1 -- 172.21.15.83:0/4169408017 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f0d200557f0 con 0x7f0d300d2730 2024-04-17T10:43:30.799 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:43:30.799 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:43:30.799 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_dump": [ 2024-04-17T10:43:30.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:43:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.a", 2024-04-17T10:43:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mfpxsNxAANpL7rcWevXwd2KMftSQnEQ==", 2024-04-17T10:43:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:43:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:43:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:43:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:43:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:43:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:43:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:43:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:43:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.b", 2024-04-17T10:43:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mXlI9ChAAYdcrGVmtYdtWWgCLVmpTTg==", 2024-04-17T10:43:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:43:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:43:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:43:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:43:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:43:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:43:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:43:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:43:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.c", 2024-04-17T10:43:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mupo3BBAAyKH801vpFLXYs9mfzSlT0w==", 2024-04-17T10:43:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:43:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:43:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:43:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:43:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:43:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:43:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:43:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:43:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.d", 2024-04-17T10:43:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mbm++EhAAIBMOtQoD+kPeMqR6gK5ctw==", 2024-04-17T10:43:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:43:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:43:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:43:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:43:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:43:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:43:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:43:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:43:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.0", 2024-04-17T10:43:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBXph9m1HWhIRAAOq+9nGq7OGonUCE2aeiAzQ==", 2024-04-17T10:43:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:43:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:43:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:43:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:43:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:43:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:43:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:43:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.1", 2024-04-17T10:43:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBaph9mlMmtJRAAXHLMjXfm/+WjIFURLkbAfA==", 2024-04-17T10:43:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:43:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:43:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:43:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:43:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:43:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:43:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:43:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.2", 2024-04-17T10:43:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBdph9mBsNPLRAAtR5hwGdDkjLDLS19QUDDYQ==", 2024-04-17T10:43:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:43:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:43:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:43:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:43:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:43:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:43:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:43:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.3", 2024-04-17T10:43:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBgph9mZ4EwNRAAIyL7e0EK1523dPXmbOOkzw==", 2024-04-17T10:43:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:43:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:43:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:43:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:43:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:43:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:43:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:43:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.4", 2024-04-17T10:43:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBmph9mphAsMhAAtF4uV28Nb66PNjuugB12Hw==", 2024-04-17T10:43:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:43:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:43:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:43:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:43:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:43:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:43:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:43:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.5", 2024-04-17T10:43:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBpph9mHJQNNxAA72eOFYFbhakQac1JKRNjzw==", 2024-04-17T10:43:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:43:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:43:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:43:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:43:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:43:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:43:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:43:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.6", 2024-04-17T10:43:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBtph9mb6RKAxAAoVX6Cp7l7j3Lm1ES2Sq+fQ==", 2024-04-17T10:43:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:43:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:43:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:43:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:43:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:43:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:43:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:43:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.7", 2024-04-17T10:43:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBwph9mdrdrBxAApDwjml2ahsLbRol4uL/BxA==", 2024-04-17T10:43:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:43:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:43:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:43:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:43:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:43:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:43:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:43:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.0", 2024-04-17T10:43:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mHGmXGBAAZF+JiAG9YrFtbW+dCy2IEQ==", 2024-04-17T10:43:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:43:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:43:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow r", 2024-04-17T10:43:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-17T10:43:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:43:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:43:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:43:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.1", 2024-04-17T10:43:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mB7wgIBAABdavAo/uzVuSjfkkcYtqJg==", 2024-04-17T10:43:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:43:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:43:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-17T10:43:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-17T10:43:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-17T10:43:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:43:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:43:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:43:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.2", 2024-04-17T10:43:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9m+wKgHRAAe8IUFx34ItLpsaDqSYyEgA==", 2024-04-17T10:43:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:43:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:43:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-17T10:43:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-17T10:43:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-17T10:43:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:43:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:43:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:43:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.3", 2024-04-17T10:43:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mfJ0yJRAAHH7rRh/BUIujTT04AF2l0w==", 2024-04-17T10:43:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:43:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:43:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-17T10:43:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-17T10:43:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-17T10:43:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:43:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:43:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:43:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.admin", 2024-04-17T10:43:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mjJ3GIBAA/2qaCbRiLJ5X1cv3ZGdfoA==", 2024-04-17T10:43:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:43:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-17T10:43:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:43:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:43:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:43:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:43:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:43:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:43:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.x", 2024-04-17T10:43:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mMFptNBAA/G0zEfMZTCy9Iox+MwHG3Q==", 2024-04-17T10:43:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:43:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-17T10:43:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-04-17T10:43:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:43:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:43:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:43:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:43:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.y", 2024-04-17T10:43:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mjFbRMRAAjY2d8POBzp8hEp9GSO0ylQ==", 2024-04-17T10:43:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:43:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-17T10:43:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-04-17T10:43:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:43:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:43:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:43:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-17T10:43:30.814 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:43:30.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.805+0000 7f0d35698640 1 -- 172.21.15.83:0/4169408017 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0d1c066f60 msgr2=0x7f0d1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:30.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.805+0000 7f0d35698640 1 --2- 172.21.15.83:0/4169408017 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0d1c066f60 0x7f0d1c069420 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0d240af000 tx=0x7f0d24008040 comp rx=0 tx=0).stop 2024-04-17T10:43:30.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.805+0000 7f0d35698640 1 -- 172.21.15.83:0/4169408017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d300d2730 msgr2=0x7f0d301597e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:30.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.805+0000 7f0d35698640 1 --2- 172.21.15.83:0/4169408017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d300d2730 0x7f0d301597e0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f0d2000c480 tx=0x7f0d2000c940 comp rx=0 tx=0).stop 2024-04-17T10:43:30.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.805+0000 7f0d35698640 1 -- 172.21.15.83:0/4169408017 shutdown_connections 2024-04-17T10:43:30.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.805+0000 7f0d35698640 1 --2- 172.21.15.83:0/4169408017 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0d1c066f60 0x7f0d1c069420 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:30.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.805+0000 7f0d35698640 1 --2- 172.21.15.83:0/4169408017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d3015e1b0 0x7f0d300cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:30.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.805+0000 7f0d35698640 1 --2- 172.21.15.83:0/4169408017 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d3015ddd0 0x7f0d30158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:30.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.805+0000 7f0d35698640 1 --2- 172.21.15.83:0/4169408017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d300d2730 0x7f0d301597e0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:30.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.805+0000 7f0d35698640 1 -- 172.21.15.83:0/4169408017 >> 172.21.15.83:0/4169408017 conn(0x7f0d300d3a40 msgr2=0x7f0d300b34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:30.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.805+0000 7f0d35698640 1 -- 172.21.15.83:0/4169408017 shutdown_connections 2024-04-17T10:43:30.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.805+0000 7f0d35698640 1 -- 172.21.15.83:0/4169408017 wait complete. 2024-04-17T10:43:30.817 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:43:30.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa5628c1640 1 Processor -- start 2024-04-17T10:43:30.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa5628c1640 1 -- start start 2024-04-17T10:43:30.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa5628c1640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa55c15db50 0x7fa55c070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:30.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa5628c1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa55c0aedd0 0x7fa55c06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:30.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa5628c1640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa55c0709b0 con 0x7fa55c06dae0 2024-04-17T10:43:30.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa5628c1640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa55c15e440 con 0x7fa55c15db50 2024-04-17T10:43:30.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa5628c1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c15e720 con 0x7fa55c0aedd0 2024-04-17T10:43:30.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa560e5e640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa55c06dae0 0x7fa55c0cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:43494/0 (socket says 172.21.15.83:43494) 2024-04-17T10:43:30.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa560e5e640 1 -- 172.21.15.83:0/1004086650 learned_addr learned my addr 172.21.15.83:0/1004086650 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:43:30.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa55bfff640 1 --2- 172.21.15.83:0/1004086650 >> v2:172.21.15.167:3300/0 conn(0x7fa55c15db50 0x7fa55c070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:30.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa5537fe640 1 --2- 172.21.15.83:0/1004086650 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa55c0aedd0 0x7fa55c06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:30.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa55b7fe640 1 -- 172.21.15.83:0/1004086650 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2434300483 0 0) 0x7fa5540014a0 con 0x7fa55c06dae0 2024-04-17T10:43:30.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa55b7fe640 1 -- 172.21.15.83:0/1004086650 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa548003530 con 0x7fa55c06dae0 2024-04-17T10:43:30.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa55b7fe640 1 -- 172.21.15.83:0/1004086650 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4098897667 0 0) 0x7fa548003530 con 0x7fa55c06dae0 2024-04-17T10:43:30.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa55b7fe640 1 -- 172.21.15.83:0/1004086650 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa55c0aedd0 msgr2=0x7fa55c06d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:30.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa55b7fe640 1 --2- 172.21.15.83:0/1004086650 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa55c0aedd0 0x7fa55c06d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:30.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa55b7fe640 1 -- 172.21.15.83:0/1004086650 >> v2:172.21.15.167:3300/0 conn(0x7fa55c15db50 msgr2=0x7fa55c070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:30.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa55b7fe640 1 --2- 172.21.15.83:0/1004086650 >> v2:172.21.15.167:3300/0 conn(0x7fa55c15db50 0x7fa55c070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:30.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa55b7fe640 1 -- 172.21.15.83:0/1004086650 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa55c15ea00 con 0x7fa55c06dae0 2024-04-17T10:43:30.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa55b7fe640 1 -- 172.21.15.83:0/1004086650 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fa554001bd0 con 0x7fa55c06dae0 2024-04-17T10:43:30.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa55b7fe640 1 -- 172.21.15.83:0/1004086650 >> v1:172.21.15.83:6789/0 conn(0x7fa55c06dae0 legacy=0x7fa55c0cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:30.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa55b7fe640 1 --2- 172.21.15.83:0/1004086650 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa55c0aedd0 0x7fa5480039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:30.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa55b7fe640 1 --2- 172.21.15.83:0/1004086650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa548003f40 0x7fa548004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:30.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa55b7fe640 1 --2- 172.21.15.83:0/1004086650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa548004d50 0x7fa5480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:30.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa55b7fe640 1 -- 172.21.15.83:0/1004086650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c15e440 con 0x7fa548003f40 2024-04-17T10:43:30.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa55b7fe640 1 -- 172.21.15.83:0/1004086650 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c15e720 con 0x7fa548004d50 2024-04-17T10:43:30.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa55b7fe640 1 -- 172.21.15.83:0/1004086650 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa54800b9d0 con 0x7fa55c0aedd0 2024-04-17T10:43:30.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa5537fe640 1 --2- 172.21.15.83:0/1004086650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa548003f40 0x7fa548004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:30.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa560e5e640 1 --2- 172.21.15.83:0/1004086650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa548004d50 0x7fa5480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa55bfff640 1 --2- 172.21.15.83:0/1004086650 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa55c0aedd0 0x7fa5480039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa5537fe640 1 -- 172.21.15.83:0/1004086650 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa55c0aedd0 msgr2=0x7fa5480039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa5537fe640 1 --2- 172.21.15.83:0/1004086650 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa55c0aedd0 0x7fa5480039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa5537fe640 1 -- 172.21.15.83:0/1004086650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa548004d50 msgr2=0x7fa5480051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa5537fe640 1 --2- 172.21.15.83:0/1004086650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa548004d50 0x7fa5480051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa5537fe640 1 -- 172.21.15.83:0/1004086650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa548003530 con 0x7fa548003f40 2024-04-17T10:43:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.937+0000 7fa560e5e640 1 --2- 172.21.15.83:0/1004086650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa548004d50 0x7fa5480051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:43:30.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.941+0000 7fa5537fe640 1 --2- 172.21.15.83:0/1004086650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa548003f40 0x7fa548004810 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fa544009ae0 tx=0x7fa54401ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a001fd534537644f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:30.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.941+0000 7fa55b7fe640 1 -- 172.21.15.83:0/1004086650 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5440b2070 con 0x7fa548003f40 2024-04-17T10:43:30.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.941+0000 7fa55b7fe640 1 -- 172.21.15.83:0/1004086650 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5440a5660 con 0x7fa548003f40 2024-04-17T10:43:30.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.941+0000 7fa55b7fe640 1 -- 172.21.15.83:0/1004086650 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5440096b0 con 0x7fa548003f40 2024-04-17T10:43:30.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.941+0000 7fa5628c1640 1 -- 172.21.15.83:0/1004086650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa548003f40 msgr2=0x7fa548004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:30.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.941+0000 7fa5628c1640 1 --2- 172.21.15.83:0/1004086650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa548003f40 0x7fa548004810 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fa544009ae0 tx=0x7fa54401ca10 comp rx=0 tx=0).stop 2024-04-17T10:43:30.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.941+0000 7fa560e5e640 1 -- 172.21.15.83:0/1004086650 reap_dead start 2024-04-17T10:43:30.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.941+0000 7fa5628c1640 1 -- 172.21.15.83:0/1004086650 shutdown_connections 2024-04-17T10:43:30.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.941+0000 7fa5628c1640 1 -- 172.21.15.83:0/1004086650 >> 172.21.15.83:0/1004086650 conn(0x7fa55c06abe0 msgr2=0x7fa55c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:30.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.941+0000 7fa5628c1640 1 -- 172.21.15.83:0/1004086650 shutdown_connections 2024-04-17T10:43:30.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.941+0000 7fa5628c1640 1 -- 172.21.15.83:0/1004086650 wait complete. 2024-04-17T10:43:30.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.945+0000 7fa5628c1640 1 Processor -- start 2024-04-17T10:43:30.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.945+0000 7fa5628c1640 1 -- start start 2024-04-17T10:43:30.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.945+0000 7fa5628c1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa55c14c0a0 0x7fa55c150360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:30.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.945+0000 7fa5628c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa55c14c480 0x7fa55c1508a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:30.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.945+0000 7fa5628c1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa55c0aaf10 0x7fa55c0ab3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:30.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.945+0000 7fa5628c1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c15df30 con 0x7fa55c14c480 2024-04-17T10:43:30.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.945+0000 7fa5628c1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c15efc0 con 0x7fa55c14c0a0 2024-04-17T10:43:30.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.945+0000 7fa5628c1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c15e0b0 con 0x7fa55c0aaf10 2024-04-17T10:43:30.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.945+0000 7fa5537fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa55c14c480 0x7fa55c1508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:30.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.945+0000 7fa5537fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa55c14c480 0x7fa55c1508a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51074/0 (socket says 172.21.15.83:51074) 2024-04-17T10:43:30.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.945+0000 7fa5537fe640 1 -- 172.21.15.83:0/2424355146 learned_addr learned my addr 172.21.15.83:0/2424355146 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:30.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.945+0000 7fa55bfff640 1 --2- 172.21.15.83:0/2424355146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa55c14c0a0 0x7fa55c150360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:30.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.945+0000 7fa560e5e640 1 --2- 172.21.15.83:0/2424355146 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa55c0aaf10 0x7fa55c0ab3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:30.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.945+0000 7fa5537fe640 1 -- 172.21.15.83:0/2424355146 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa55c0aaf10 msgr2=0x7fa55c0ab3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:30.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.945+0000 7fa5537fe640 1 --2- 172.21.15.83:0/2424355146 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa55c0aaf10 0x7fa55c0ab3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:30.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.945+0000 7fa5537fe640 1 -- 172.21.15.83:0/2424355146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa55c14c0a0 msgr2=0x7fa55c150360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:30.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.945+0000 7fa5537fe640 1 --2- 172.21.15.83:0/2424355146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa55c14c0a0 0x7fa55c150360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:30.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.945+0000 7fa5537fe640 1 -- 172.21.15.83:0/2424355146 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa55c14c930 con 0x7fa55c14c480 2024-04-17T10:43:30.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.945+0000 7fa55bfff640 1 --2- 172.21.15.83:0/2424355146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa55c14c0a0 0x7fa55c150360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:43:30.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.945+0000 7fa5537fe640 1 --2- 172.21.15.83:0/2424355146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa55c14c480 0x7fa55c1508a0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fa5440b0040 tx=0x7fa544019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:30.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.945+0000 7fa5597fa640 1 -- 172.21.15.83:0/2424355146 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5440bc020 con 0x7fa55c14c480 2024-04-17T10:43:30.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.945+0000 7fa5597fa640 1 -- 172.21.15.83:0/2424355146 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa544004020 con 0x7fa55c14c480 2024-04-17T10:43:30.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.945+0000 7fa5628c1640 1 -- 172.21.15.83:0/2424355146 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa55c14cc10 con 0x7fa55c14c480 2024-04-17T10:43:30.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.945+0000 7fa5597fa640 1 -- 172.21.15.83:0/2424355146 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5440b2040 con 0x7fa55c14c480 2024-04-17T10:43:30.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.945+0000 7fa5628c1640 1 -- 172.21.15.83:0/2424355146 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa55c14d1d0 con 0x7fa55c14c480 2024-04-17T10:43:30.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.945+0000 7fa5628c1640 1 -- 172.21.15.83:0/2424355146 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa55c074f90 con 0x7fa55c14c480 2024-04-17T10:43:30.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.945+0000 7fa5597fa640 1 -- 172.21.15.83:0/2424355146 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa544003620 con 0x7fa55c14c480 2024-04-17T10:43:30.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.945+0000 7fa5597fa640 1 --2- 172.21.15.83:0/2424355146 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa53c066f60 0x7fa53c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:30.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.949+0000 7fa5597fa640 1 -- 172.21.15.83:0/2424355146 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa54411e860 con 0x7fa55c14c480 2024-04-17T10:43:30.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.949+0000 7fa55bfff640 1 --2- 172.21.15.83:0/2424355146 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa53c066f60 0x7fa53c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:30.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.949+0000 7fa55bfff640 1 --2- 172.21.15.83:0/2424355146 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa53c066f60 0x7fa53c069420 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa55c150fa0 tx=0x7fa54c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:30.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:30.953+0000 7fa5597fa640 1 -- 172.21.15.83:0/2424355146 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa544016780 con 0x7fa55c14c480 2024-04-17T10:43:31.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.113+0000 7fa5628c1640 1 -- 172.21.15.83:0/2424355146 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa55c0b2af0 con 0x7fa55c14c480 2024-04-17T10:43:31.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.113+0000 7fa5597fa640 1 -- 172.21.15.83:0/2424355146 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+12819 (secure 0 0 0) 0x7fa5440efc50 con 0x7fa55c14c480 2024-04-17T10:43:31.127 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:43:31.127 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":70,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:43:30.459827+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:43:31.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.117+0000 7fa5628c1640 1 -- 172.21.15.83:0/2424355146 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa53c066f60 msgr2=0x7fa53c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:31.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.117+0000 7fa5628c1640 1 --2- 172.21.15.83:0/2424355146 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa53c066f60 0x7fa53c069420 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa55c150fa0 tx=0x7fa54c008040 comp rx=0 tx=0).stop 2024-04-17T10:43:31.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.117+0000 7fa5628c1640 1 -- 172.21.15.83:0/2424355146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa55c14c480 msgr2=0x7fa55c1508a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:31.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.117+0000 7fa5628c1640 1 --2- 172.21.15.83:0/2424355146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa55c14c480 0x7fa55c1508a0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fa5440b0040 tx=0x7fa544019810 comp rx=0 tx=0).stop 2024-04-17T10:43:31.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.117+0000 7fa5628c1640 1 -- 172.21.15.83:0/2424355146 shutdown_connections 2024-04-17T10:43:31.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.117+0000 7fa5628c1640 1 --2- 172.21.15.83:0/2424355146 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa53c066f60 0x7fa53c069420 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:31.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.117+0000 7fa5628c1640 1 --2- 172.21.15.83:0/2424355146 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa55c0aaf10 0x7fa55c0ab3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:31.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.117+0000 7fa5628c1640 1 --2- 172.21.15.83:0/2424355146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa55c14c480 0x7fa55c1508a0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:31.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.117+0000 7fa5628c1640 1 --2- 172.21.15.83:0/2424355146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa55c14c0a0 0x7fa55c150360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:31.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.117+0000 7fa5628c1640 1 -- 172.21.15.83:0/2424355146 >> 172.21.15.83:0/2424355146 conn(0x7fa55c06abe0 msgr2=0x7fa55c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:31.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.117+0000 7fa5628c1640 1 -- 172.21.15.83:0/2424355146 shutdown_connections 2024-04-17T10:43:31.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.117+0000 7fa5628c1640 1 -- 172.21.15.83:0/2424355146 wait complete. 2024-04-17T10:43:31.132 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:43:31.132 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:43:31.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.261+0000 7f0fb0c76640 1 Processor -- start 2024-04-17T10:43:31.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.261+0000 7f0fb0c76640 1 -- start start 2024-04-17T10:43:31.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.261+0000 7f0fb0c76640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0fac15b6e0 0x7f0fac071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:31.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.261+0000 7f0fb0c76640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0fac15bac0 0x7f0fac0cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:31.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.261+0000 7f0fb0c76640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fac06c1f0 con 0x7f0fac070470 2024-04-17T10:43:31.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.261+0000 7f0fb0c76640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fac0cdd30 con 0x7f0fac15bac0 2024-04-17T10:43:31.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.261+0000 7f0fb0c76640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fac070a10 con 0x7f0fac15b6e0 2024-04-17T10:43:31.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.261+0000 7f0fa3fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0fac15bac0 0x7f0fac0cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:31.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.261+0000 7f0faad9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0fac070470 0x7f0fac06ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:43504/0 (socket says 172.21.15.83:43504) 2024-04-17T10:43:31.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.261+0000 7f0faad9d640 1 -- 172.21.15.83:0/947694489 learned_addr learned my addr 172.21.15.83:0/947694489 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:43:31.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.261+0000 7f0faa59c640 1 --2- 172.21.15.83:0/947694489 >> v2:172.21.15.167:3300/0 conn(0x7f0fac15b6e0 0x7f0fac071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:31.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.261+0000 7f0fa3fff640 1 -- 172.21.15.83:0/947694489 >> v2:172.21.15.167:3300/0 conn(0x7f0fac15b6e0 msgr2=0x7f0fac071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:31.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.261+0000 7f0fa3fff640 1 --2- 172.21.15.83:0/947694489 >> v2:172.21.15.167:3300/0 conn(0x7f0fac15b6e0 0x7f0fac071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:31.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.261+0000 7f0fa3fff640 1 -- 172.21.15.83:0/947694489 >> v1:172.21.15.83:6789/0 conn(0x7f0fac070470 legacy=0x7f0fac06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:31.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.261+0000 7f0fa9d9b640 1 -- 172.21.15.83:0/947694489 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3580535751 0 0) 0x7f0fac06c1f0 con 0x7f0fac070470 2024-04-17T10:43:31.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.261+0000 7f0fa3fff640 1 -- 172.21.15.83:0/947694489 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fac070cf0 con 0x7f0fac15bac0 2024-04-17T10:43:31.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.265+0000 7f0fa3fff640 1 --2- 172.21.15.83:0/947694489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fac15bac0 0x7f0fac0cd7f0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0728a0 tx=0x7f0f9401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6665c5f62db9cc63 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:31.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.265+0000 7f0fa9d9b640 1 -- 172.21.15.83:0/947694489 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f940b0070 con 0x7f0fac15bac0 2024-04-17T10:43:31.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.265+0000 7f0fa9d9b640 1 -- 172.21.15.83:0/947694489 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f940ab600 con 0x7f0fac15bac0 2024-04-17T10:43:31.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.265+0000 7f0fa9d9b640 1 -- 172.21.15.83:0/947694489 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f940ab9d0 con 0x7f0fac15bac0 2024-04-17T10:43:31.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.265+0000 7f0fb0c76640 1 -- 172.21.15.83:0/947694489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fac15bac0 msgr2=0x7f0fac0cd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:31.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.265+0000 7f0fb0c76640 1 --2- 172.21.15.83:0/947694489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fac15bac0 0x7f0fac0cd7f0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0728a0 tx=0x7f0f9401c480 comp rx=0 tx=0).stop 2024-04-17T10:43:31.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.265+0000 7f0fb0c76640 1 -- 172.21.15.83:0/947694489 shutdown_connections 2024-04-17T10:43:31.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.265+0000 7f0fb0c76640 1 --2- 172.21.15.83:0/947694489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fac15bac0 0x7f0fac0cd7f0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:31.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.265+0000 7f0fb0c76640 1 --2- 172.21.15.83:0/947694489 >> v2:172.21.15.167:3300/0 conn(0x7f0fac15b6e0 0x7f0fac071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:31.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.265+0000 7f0fb0c76640 1 -- 172.21.15.83:0/947694489 >> 172.21.15.83:0/947694489 conn(0x7f0fac0b15b0 msgr2=0x7f0fac0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:31.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.265+0000 7f0fb0c76640 1 -- 172.21.15.83:0/947694489 shutdown_connections 2024-04-17T10:43:31.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.265+0000 7f0fb0c76640 1 -- 172.21.15.83:0/947694489 wait complete. 2024-04-17T10:43:31.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.265+0000 7f0fb0c76640 1 Processor -- start 2024-04-17T10:43:31.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.265+0000 7f0fb0c76640 1 -- start start 2024-04-17T10:43:31.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.269+0000 7f0fb0c76640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fac070470 0x7f0fac146c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:31.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.269+0000 7f0fb0c76640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fac15b6e0 0x7f0fac1f6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:31.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.269+0000 7f0fb0c76640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fac15bac0 0x7f0fac152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:31.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.269+0000 7f0fb0c76640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac06cee0 con 0x7f0fac070470 2024-04-17T10:43:31.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.269+0000 7f0fb0c76640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac06d060 con 0x7f0fac15b6e0 2024-04-17T10:43:31.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.269+0000 7f0fb0c76640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac06d1e0 con 0x7f0fac15bac0 2024-04-17T10:43:31.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.269+0000 7f0faa59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fac070470 0x7f0fac146c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:31.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.269+0000 7f0faa59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fac070470 0x7f0fac146c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51084/0 (socket says 172.21.15.83:51084) 2024-04-17T10:43:31.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.269+0000 7f0faa59c640 1 -- 172.21.15.83:0/4086348875 learned_addr learned my addr 172.21.15.83:0/4086348875 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:31.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.269+0000 7f0fa3fff640 1 --2- 172.21.15.83:0/4086348875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fac15b6e0 0x7f0fac1f6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:31.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.269+0000 7f0faa59c640 1 -- 172.21.15.83:0/4086348875 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fac15bac0 msgr2=0x7f0fac152fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:31.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.269+0000 7f0faa59c640 1 --2- 172.21.15.83:0/4086348875 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fac15bac0 0x7f0fac152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:31.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.269+0000 7f0faa59c640 1 -- 172.21.15.83:0/4086348875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fac15b6e0 msgr2=0x7f0fac1f6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:31.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.269+0000 7f0faa59c640 1 --2- 172.21.15.83:0/4086348875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fac15b6e0 0x7f0fac1f6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:31.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.269+0000 7f0faa59c640 1 -- 172.21.15.83:0/4086348875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fac155f60 con 0x7f0fac070470 2024-04-17T10:43:31.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.269+0000 7f0faa59c640 1 --2- 172.21.15.83:0/4086348875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fac070470 0x7f0fac146c70 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c00d800 tx=0x7f0f9c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:31.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.269+0000 7f0fa2ffd640 1 -- 172.21.15.83:0/4086348875 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c00b6c0 con 0x7f0fac070470 2024-04-17T10:43:31.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.269+0000 7f0fa2ffd640 1 -- 172.21.15.83:0/4086348875 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f9c019070 con 0x7f0fac070470 2024-04-17T10:43:31.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.269+0000 7f0fb0c76640 1 -- 172.21.15.83:0/4086348875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fac0b2fb0 con 0x7f0fac070470 2024-04-17T10:43:31.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.269+0000 7f0fa2ffd640 1 -- 172.21.15.83:0/4086348875 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c00ba60 con 0x7f0fac070470 2024-04-17T10:43:31.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.269+0000 7f0fb0c76640 1 -- 172.21.15.83:0/4086348875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fac155bc0 con 0x7f0fac070470 2024-04-17T10:43:31.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.269+0000 7f0fb0c76640 1 -- 172.21.15.83:0/4086348875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fac074f90 con 0x7f0fac070470 2024-04-17T10:43:31.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.269+0000 7f0fa2ffd640 1 -- 172.21.15.83:0/4086348875 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0f9c00bc00 con 0x7f0fac070470 2024-04-17T10:43:31.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.269+0000 7f0fa2ffd640 1 --2- 172.21.15.83:0/4086348875 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0f8c066f60 0x7f0f8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:31.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.269+0000 7f0fa2ffd640 1 -- 172.21.15.83:0/4086348875 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0f9c0851d0 con 0x7f0fac070470 2024-04-17T10:43:31.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.269+0000 7f0fa3fff640 1 --2- 172.21.15.83:0/4086348875 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0f8c066f60 0x7f0f8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:31.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.273+0000 7f0fa3fff640 1 --2- 172.21.15.83:0/4086348875 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0f8c066f60 0x7f0f8c069420 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0f940ad000 tx=0x7f0f940a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:31.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.273+0000 7f0fa2ffd640 1 -- 172.21.15.83:0/4086348875 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0f9c056520 con 0x7f0fac070470 2024-04-17T10:43:31.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.429+0000 7f0fb0c76640 1 -- 172.21.15.83:0/4086348875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0fac06b220 con 0x7f0fac070470 2024-04-17T10:43:31.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.429+0000 7f0fa2ffd640 1 -- 172.21.15.83:0/4086348875 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0f9c055e20 con 0x7f0fac070470 2024-04-17T10:43:31.426 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:43:31.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.433+0000 7f0fb0c76640 1 -- 172.21.15.83:0/4086348875 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0f8c066f60 msgr2=0x7f0f8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:31.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.433+0000 7f0fb0c76640 1 --2- 172.21.15.83:0/4086348875 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0f8c066f60 0x7f0f8c069420 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0f940ad000 tx=0x7f0f940a6000 comp rx=0 tx=0).stop 2024-04-17T10:43:31.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.433+0000 7f0fb0c76640 1 -- 172.21.15.83:0/4086348875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fac070470 msgr2=0x7f0fac146c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:31.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.433+0000 7f0fb0c76640 1 --2- 172.21.15.83:0/4086348875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fac070470 0x7f0fac146c70 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c00d800 tx=0x7f0f9c00dcc0 comp rx=0 tx=0).stop 2024-04-17T10:43:31.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.433+0000 7f0fb0c76640 1 -- 172.21.15.83:0/4086348875 shutdown_connections 2024-04-17T10:43:31.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.433+0000 7f0fb0c76640 1 --2- 172.21.15.83:0/4086348875 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0f8c066f60 0x7f0f8c069420 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:31.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.433+0000 7f0fb0c76640 1 --2- 172.21.15.83:0/4086348875 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fac15bac0 0x7f0fac152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:31.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.433+0000 7f0fb0c76640 1 --2- 172.21.15.83:0/4086348875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fac15b6e0 0x7f0fac1f6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:31.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.433+0000 7f0fb0c76640 1 --2- 172.21.15.83:0/4086348875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fac070470 0x7f0fac146c70 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:31.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.433+0000 7f0fb0c76640 1 -- 172.21.15.83:0/4086348875 >> 172.21.15.83:0/4086348875 conn(0x7f0fac0b15b0 msgr2=0x7f0fac0af4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:31.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.433+0000 7f0fb0c76640 1 -- 172.21.15.83:0/4086348875 shutdown_connections 2024-04-17T10:43:31.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.433+0000 7f0fb0c76640 1 -- 172.21.15.83:0/4086348875 wait complete. 2024-04-17T10:43:31.445 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-17T10:43:31.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.569+0000 7f03f16b4640 1 Processor -- start 2024-04-17T10:43:31.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.569+0000 7f03f16b4640 1 -- start start 2024-04-17T10:43:31.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.569+0000 7f03f16b4640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f03ec15dfa0 0x7f03ec0b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:31.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.569+0000 7f03f16b4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f03ec0b5b40 0x7f03ec0d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:31.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.569+0000 7f03f16b4640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03ec162a00 con 0x7f03ec0ce980 2024-04-17T10:43:31.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.569+0000 7f03f16b4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03ec161e80 con 0x7f03ec0b5b40 2024-04-17T10:43:31.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.569+0000 7f03f16b4640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03ec162160 con 0x7f03ec15dfa0 2024-04-17T10:43:31.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.569+0000 7f03e27fc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f03ec0ce980 0x7f03ec0b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:43510/0 (socket says 172.21.15.83:43510) 2024-04-17T10:43:31.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.569+0000 7f03e27fc640 1 -- 172.21.15.83:0/1862151893 learned_addr learned my addr 172.21.15.83:0/1862151893 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:43:31.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.569+0000 7f03eb7fe640 1 --2- 172.21.15.83:0/1862151893 >> v2:172.21.15.83:3300/0 conn(0x7f03ec0b5b40 0x7f03ec0d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:31.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.569+0000 7f03eaffd640 1 --2- 172.21.15.83:0/1862151893 >> v2:172.21.15.167:3300/0 conn(0x7f03ec15dfa0 0x7f03ec0b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:31.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.569+0000 7f03ea7fc640 1 -- 172.21.15.83:0/1862151893 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3258714110 0 0) 0x7f03ec162a00 con 0x7f03ec0ce980 2024-04-17T10:43:31.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.569+0000 7f03ea7fc640 1 -- 172.21.15.83:0/1862151893 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03d8003530 con 0x7f03ec0ce980 2024-04-17T10:43:31.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.569+0000 7f03eb7fe640 1 -- 172.21.15.83:0/1862151893 >> v2:172.21.15.167:3300/0 conn(0x7f03ec15dfa0 msgr2=0x7f03ec0b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:31.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.569+0000 7f03eb7fe640 1 --2- 172.21.15.83:0/1862151893 >> v2:172.21.15.167:3300/0 conn(0x7f03ec15dfa0 0x7f03ec0b6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:31.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.569+0000 7f03eb7fe640 1 -- 172.21.15.83:0/1862151893 >> v1:172.21.15.83:6789/0 conn(0x7f03ec0ce980 legacy=0x7f03ec0b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:31.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.569+0000 7f03eb7fe640 1 -- 172.21.15.83:0/1862151893 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ec162440 con 0x7f03ec0b5b40 2024-04-17T10:43:31.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03eb7fe640 1 --2- 172.21.15.83:0/1862151893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03ec0b5b40 0x7f03ec0d1d30 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f03ec08c3f0 tx=0x7f03e401c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630a690cf4c5b298 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:31.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03ea7fc640 1 -- 172.21.15.83:0/1862151893 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e40b2070 con 0x7f03ec0b5b40 2024-04-17T10:43:31.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03ea7fc640 1 -- 172.21.15.83:0/1862151893 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e40a57b0 con 0x7f03ec0b5b40 2024-04-17T10:43:31.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03ea7fc640 1 -- 172.21.15.83:0/1862151893 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e40a5b80 con 0x7f03ec0b5b40 2024-04-17T10:43:31.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03f16b4640 1 -- 172.21.15.83:0/1862151893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03ec0b5b40 msgr2=0x7f03ec0d1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:31.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03f16b4640 1 --2- 172.21.15.83:0/1862151893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03ec0b5b40 0x7f03ec0d1d30 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f03ec08c3f0 tx=0x7f03e401c870 comp rx=0 tx=0).stop 2024-04-17T10:43:31.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03f16b4640 1 -- 172.21.15.83:0/1862151893 shutdown_connections 2024-04-17T10:43:31.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03f16b4640 1 --2- 172.21.15.83:0/1862151893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03ec0b5b40 0x7f03ec0d1d30 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:31.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03f16b4640 1 --2- 172.21.15.83:0/1862151893 >> v2:172.21.15.167:3300/0 conn(0x7f03ec15dfa0 0x7f03ec0b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:31.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03f16b4640 1 -- 172.21.15.83:0/1862151893 >> 172.21.15.83:0/1862151893 conn(0x7f03ec0d0600 msgr2=0x7f03ec0d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:31.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03f16b4640 1 -- 172.21.15.83:0/1862151893 shutdown_connections 2024-04-17T10:43:31.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03f16b4640 1 -- 172.21.15.83:0/1862151893 wait complete. 2024-04-17T10:43:31.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03f16b4640 1 Processor -- start 2024-04-17T10:43:31.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03f16b4640 1 -- start start 2024-04-17T10:43:31.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03f16b4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03ec0b5b40 0x7f03ec14b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:31.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03f16b4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03ec0ce980 0x7f03ec14bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:31.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03f16b4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03ec15dfa0 0x7f03ec15c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:31.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03f16b4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec16ae50 con 0x7f03ec15dfa0 2024-04-17T10:43:31.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03f16b4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec161b10 con 0x7f03ec0ce980 2024-04-17T10:43:31.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03f16b4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec16afd0 con 0x7f03ec0b5b40 2024-04-17T10:43:31.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03eb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03ec15dfa0 0x7f03ec15c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:31.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03eb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03ec15dfa0 0x7f03ec15c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51112/0 (socket says 172.21.15.83:51112) 2024-04-17T10:43:31.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03eb7fe640 1 -- 172.21.15.83:0/1090667282 learned_addr learned my addr 172.21.15.83:0/1090667282 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:31.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03eaffd640 1 --2- 172.21.15.83:0/1090667282 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03ec0b5b40 0x7f03ec14b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:31.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03e27fc640 1 --2- 172.21.15.83:0/1090667282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03ec0ce980 0x7f03ec14bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:31.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03eb7fe640 1 -- 172.21.15.83:0/1090667282 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03ec0b5b40 msgr2=0x7f03ec14b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:31.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03eb7fe640 1 --2- 172.21.15.83:0/1090667282 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03ec0b5b40 0x7f03ec14b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:31.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03eb7fe640 1 -- 172.21.15.83:0/1090667282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03ec0ce980 msgr2=0x7f03ec14bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:31.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03eb7fe640 1 --2- 172.21.15.83:0/1090667282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03ec0ce980 0x7f03ec14bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:31.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03eb7fe640 1 -- 172.21.15.83:0/1090667282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ec15e740 con 0x7f03ec15dfa0 2024-04-17T10:43:31.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03eaffd640 1 --2- 172.21.15.83:0/1090667282 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03ec0b5b40 0x7f03ec14b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:31.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03eb7fe640 1 --2- 172.21.15.83:0/1090667282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03ec15dfa0 0x7f03ec15c340 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f03e401a4a0 tx=0x7f03e4019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:31.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03e3fff640 1 -- 172.21.15.83:0/1090667282 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e40b2050 con 0x7f03ec15dfa0 2024-04-17T10:43:31.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03e3fff640 1 -- 172.21.15.83:0/1090667282 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e40be070 con 0x7f03ec15dfa0 2024-04-17T10:43:31.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03f16b4640 1 -- 172.21.15.83:0/1090667282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03ec15ea20 con 0x7f03ec15dfa0 2024-04-17T10:43:31.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03e3fff640 1 -- 172.21.15.83:0/1090667282 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e4009450 con 0x7f03ec15dfa0 2024-04-17T10:43:31.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.573+0000 7f03f16b4640 1 -- 172.21.15.83:0/1090667282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03ec15efe0 con 0x7f03ec15dfa0 2024-04-17T10:43:31.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.577+0000 7f03f16b4640 1 -- 172.21.15.83:0/1090667282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03ec074f90 con 0x7f03ec15dfa0 2024-04-17T10:43:31.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.577+0000 7f03e3fff640 1 -- 172.21.15.83:0/1090667282 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f03e40ad870 con 0x7f03ec15dfa0 2024-04-17T10:43:31.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.577+0000 7f03e3fff640 1 --2- 172.21.15.83:0/1090667282 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f03cc066f60 0x7f03cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:31.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.577+0000 7f03e3fff640 1 -- 172.21.15.83:0/1090667282 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f03e411fb60 con 0x7f03ec15dfa0 2024-04-17T10:43:31.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.577+0000 7f03eaffd640 1 --2- 172.21.15.83:0/1090667282 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f03cc066f60 0x7f03cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:31.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.577+0000 7f03eaffd640 1 --2- 172.21.15.83:0/1090667282 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f03cc066f60 0x7f03cc069420 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f03dc009ae0 tx=0x7f03dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:31.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.581+0000 7f03e3fff640 1 -- 172.21.15.83:0/1090667282 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f03e4016520 con 0x7f03ec15dfa0 2024-04-17T10:43:31.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.741+0000 7f03f16b4640 1 -- 172.21.15.83:0/1090667282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f03ec0b5430 con 0x7f03ec15dfa0 2024-04-17T10:43:31.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.745+0000 7f03e3fff640 1 -- 172.21.15.83:0/1090667282 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v70) v1 ==== 84+0+65 (secure 0 0 0) 0x7f03e40f0f50 con 0x7f03ec15dfa0 2024-04-17T10:43:31.739 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:43:31.739 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-04-17T10:43:31.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:43:31.740 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2024-04-17T10:43:31.740 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2024-04-17T10:43:31.740 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:43:31.740 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-04-17T10:43:31.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.745+0000 7f03f16b4640 1 -- 172.21.15.83:0/1090667282 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f03cc066f60 msgr2=0x7f03cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:31.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.745+0000 7f03f16b4640 1 --2- 172.21.15.83:0/1090667282 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f03cc066f60 0x7f03cc069420 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f03dc009ae0 tx=0x7f03dc008040 comp rx=0 tx=0).stop 2024-04-17T10:43:31.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.745+0000 7f03f16b4640 1 -- 172.21.15.83:0/1090667282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03ec15dfa0 msgr2=0x7f03ec15c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:31.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.745+0000 7f03f16b4640 1 --2- 172.21.15.83:0/1090667282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03ec15dfa0 0x7f03ec15c340 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f03e401a4a0 tx=0x7f03e4019610 comp rx=0 tx=0).stop 2024-04-17T10:43:31.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.745+0000 7f03f16b4640 1 -- 172.21.15.83:0/1090667282 shutdown_connections 2024-04-17T10:43:31.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.745+0000 7f03f16b4640 1 --2- 172.21.15.83:0/1090667282 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f03cc066f60 0x7f03cc069420 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.745+0000 7f03f16b4640 1 --2- 172.21.15.83:0/1090667282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03ec15dfa0 0x7f03ec15c340 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.745+0000 7f03f16b4640 1 --2- 172.21.15.83:0/1090667282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03ec0ce980 0x7f03ec14bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.745+0000 7f03f16b4640 1 --2- 172.21.15.83:0/1090667282 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03ec0b5b40 0x7f03ec14b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.745+0000 7f03f16b4640 1 -- 172.21.15.83:0/1090667282 >> 172.21.15.83:0/1090667282 conn(0x7f03ec0d0600 msgr2=0x7f03ec0d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.749+0000 7f03f16b4640 1 -- 172.21.15.83:0/1090667282 shutdown_connections 2024-04-17T10:43:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.749+0000 7f03f16b4640 1 -- 172.21.15.83:0/1090667282 wait complete. 2024-04-17T10:43:31.760 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-17T10:43:31.760 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-17T10:43:31.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.885+0000 7f20fec0b640 1 Processor -- start 2024-04-17T10:43:31.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.885+0000 7f20fec0b640 1 -- start start 2024-04-17T10:43:31.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.885+0000 7f20fec0b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f20f8161740 0x7f20f806abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:31.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.885+0000 7f20fec0b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f20f806b120 0x7f20f80b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:31.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.885+0000 7f20fec0b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20f815bb40 con 0x7f20f815b760 2024-04-17T10:43:31.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.885+0000 7f20fec0b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f80ad840 con 0x7f20f806b120 2024-04-17T10:43:31.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.885+0000 7f20fec0b640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f806b570 con 0x7f20f8161740 2024-04-17T10:43:31.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.885+0000 7f20fd1a8640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f20f806b120 0x7f20f80b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:31.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.885+0000 7f20fc9a7640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f20f815b760 0x7f20f815eb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:43526/0 (socket says 172.21.15.83:43526) 2024-04-17T10:43:31.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.885+0000 7f20fc9a7640 1 -- 172.21.15.83:0/2018686008 learned_addr learned my addr 172.21.15.83:0/2018686008 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:43:31.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.885+0000 7f20f7fff640 1 --2- 172.21.15.83:0/2018686008 >> v2:172.21.15.167:3300/0 conn(0x7f20f8161740 0x7f20f806abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:31.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.885+0000 7f20fd1a8640 1 -- 172.21.15.83:0/2018686008 >> v2:172.21.15.167:3300/0 conn(0x7f20f8161740 msgr2=0x7f20f806abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:31.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.885+0000 7f20fd1a8640 1 --2- 172.21.15.83:0/2018686008 >> v2:172.21.15.167:3300/0 conn(0x7f20f8161740 0x7f20f806abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:31.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.885+0000 7f20fd1a8640 1 -- 172.21.15.83:0/2018686008 >> v1:172.21.15.83:6789/0 conn(0x7f20f815b760 legacy=0x7f20f815eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:31.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.885+0000 7f20fd1a8640 1 -- 172.21.15.83:0/2018686008 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f8161c70 con 0x7f20f806b120 2024-04-17T10:43:31.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.885+0000 7f20f77fe640 1 -- 172.21.15.83:0/2018686008 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4290766463 0 0) 0x7f20f815bb40 con 0x7f20f815b760 2024-04-17T10:43:31.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.889+0000 7f20fd1a8640 1 --2- 172.21.15.83:0/2018686008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20f806b120 0x7f20f80b2a90 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f20e80097f0 tx=0x7f20e801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3e66b8e29bc2b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:31.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.889+0000 7f20f77fe640 1 -- 172.21.15.83:0/2018686008 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e80b0070 con 0x7f20f806b120 2024-04-17T10:43:31.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.889+0000 7f20f77fe640 1 -- 172.21.15.83:0/2018686008 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e80ab600 con 0x7f20f806b120 2024-04-17T10:43:31.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.889+0000 7f20f77fe640 1 -- 172.21.15.83:0/2018686008 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e80ab9d0 con 0x7f20f806b120 2024-04-17T10:43:31.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.889+0000 7f20fec0b640 1 -- 172.21.15.83:0/2018686008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20f806b120 msgr2=0x7f20f80b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:31.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.889+0000 7f20fec0b640 1 --2- 172.21.15.83:0/2018686008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20f806b120 0x7f20f80b2a90 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f20e80097f0 tx=0x7f20e801c460 comp rx=0 tx=0).stop 2024-04-17T10:43:31.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.889+0000 7f20fec0b640 1 -- 172.21.15.83:0/2018686008 shutdown_connections 2024-04-17T10:43:31.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.889+0000 7f20fec0b640 1 --2- 172.21.15.83:0/2018686008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20f806b120 0x7f20f80b2a90 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:31.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.889+0000 7f20fec0b640 1 --2- 172.21.15.83:0/2018686008 >> v2:172.21.15.167:3300/0 conn(0x7f20f8161740 0x7f20f806abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:31.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.889+0000 7f20fec0b640 1 -- 172.21.15.83:0/2018686008 >> 172.21.15.83:0/2018686008 conn(0x7f20f80cc050 msgr2=0x7f20f80cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:31.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.889+0000 7f20fec0b640 1 -- 172.21.15.83:0/2018686008 shutdown_connections 2024-04-17T10:43:31.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.889+0000 7f20fec0b640 1 -- 172.21.15.83:0/2018686008 wait complete. 2024-04-17T10:43:31.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.889+0000 7f20fec0b640 1 Processor -- start 2024-04-17T10:43:31.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.889+0000 7f20fec0b640 1 -- start start 2024-04-17T10:43:31.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.893+0000 7f20fec0b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20f806b120 0x7f20f8071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:31.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.893+0000 7f20fec0b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20f815b760 0x7f20f8160320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:31.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.893+0000 7f20fec0b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20f8161740 0x7f20f8160860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:31.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.893+0000 7f20fec0b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f80b3960 con 0x7f20f8161740 2024-04-17T10:43:31.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.893+0000 7f20fec0b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f80b37e0 con 0x7f20f806b120 2024-04-17T10:43:31.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.893+0000 7f20fec0b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f80b3ae0 con 0x7f20f815b760 2024-04-17T10:43:31.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.893+0000 7f20fd1a8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20f8161740 0x7f20f8160860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:31.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.893+0000 7f20fd1a8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20f8161740 0x7f20f8160860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51130/0 (socket says 172.21.15.83:51130) 2024-04-17T10:43:31.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.893+0000 7f20fd1a8640 1 -- 172.21.15.83:0/469503196 learned_addr learned my addr 172.21.15.83:0/469503196 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:31.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.893+0000 7f20fc9a7640 1 --2- 172.21.15.83:0/469503196 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20f806b120 0x7f20f8071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:31.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.893+0000 7f20fd1a8640 1 -- 172.21.15.83:0/469503196 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20f815b760 msgr2=0x7f20f8160320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:31.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.893+0000 7f20f7fff640 1 --2- 172.21.15.83:0/469503196 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20f815b760 0x7f20f8160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:31.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.893+0000 7f20fd1a8640 1 --2- 172.21.15.83:0/469503196 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20f815b760 0x7f20f8160320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:31.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.893+0000 7f20fd1a8640 1 -- 172.21.15.83:0/469503196 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20f806b120 msgr2=0x7f20f8071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:31.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.893+0000 7f20fd1a8640 1 --2- 172.21.15.83:0/469503196 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20f806b120 0x7f20f8071b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:31.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.893+0000 7f20fd1a8640 1 -- 172.21.15.83:0/469503196 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f814cc20 con 0x7f20f8161740 2024-04-17T10:43:31.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.893+0000 7f20fc9a7640 1 --2- 172.21.15.83:0/469503196 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20f806b120 0x7f20f8071b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:31.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.893+0000 7f20fd1a8640 1 --2- 172.21.15.83:0/469503196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20f8161740 0x7f20f8160860 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f20e80ae040 tx=0x7f20e801aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:31.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.893+0000 7f20f57fa640 1 -- 172.21.15.83:0/469503196 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e80b0050 con 0x7f20f8161740 2024-04-17T10:43:31.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.893+0000 7f20f57fa640 1 -- 172.21.15.83:0/469503196 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e80bc070 con 0x7f20f8161740 2024-04-17T10:43:31.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.893+0000 7f20f57fa640 1 -- 172.21.15.83:0/469503196 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8006e20 con 0x7f20f8161740 2024-04-17T10:43:31.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.893+0000 7f20fec0b640 1 -- 172.21.15.83:0/469503196 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f814cf00 con 0x7f20f8161740 2024-04-17T10:43:31.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.893+0000 7f20fec0b640 1 -- 172.21.15.83:0/469503196 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f814d4c0 con 0x7f20f8161740 2024-04-17T10:43:31.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.893+0000 7f20fec0b640 1 -- 172.21.15.83:0/469503196 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f8075000 con 0x7f20f8161740 2024-04-17T10:43:31.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.893+0000 7f20f57fa640 1 -- 172.21.15.83:0/469503196 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f20e80b5850 con 0x7f20f8161740 2024-04-17T10:43:31.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.893+0000 7f20f57fa640 1 --2- 172.21.15.83:0/469503196 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f20e4066f60 0x7f20e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:31.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.893+0000 7f20f57fa640 1 -- 172.21.15.83:0/469503196 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f20e811f5e0 con 0x7f20f8161740 2024-04-17T10:43:31.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.893+0000 7f20fc9a7640 1 --2- 172.21.15.83:0/469503196 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f20e4066f60 0x7f20e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:31.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.893+0000 7f20fc9a7640 1 --2- 172.21.15.83:0/469503196 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f20e4066f60 0x7f20e4069420 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f20f0007fd0 tx=0x7f20f0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:31.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:31.897+0000 7f20f57fa640 1 -- 172.21.15.83:0/469503196 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f20e8016050 con 0x7f20f8161740 2024-04-17T10:43:32.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.053+0000 7f20fec0b640 1 -- 172.21.15.83:0/469503196 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f20f81688b0 con 0x7f20f8161740 2024-04-17T10:43:32.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.453+0000 7f20f57fa640 1 -- 172.21.15.83:0/469503196 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v71) v1 ==== 138+0+0 (secure 0 0 0) 0x7f20e80f09d0 con 0x7f20f8161740 2024-04-17T10:43:32.449 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2024-04-17T10:43:32.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.457+0000 7f20fec0b640 1 -- 172.21.15.83:0/469503196 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f20e4066f60 msgr2=0x7f20e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:32.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.457+0000 7f20fec0b640 1 --2- 172.21.15.83:0/469503196 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f20e4066f60 0x7f20e4069420 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f20f0007fd0 tx=0x7f20f0009040 comp rx=0 tx=0).stop 2024-04-17T10:43:32.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.457+0000 7f20fec0b640 1 -- 172.21.15.83:0/469503196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20f8161740 msgr2=0x7f20f8160860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:32.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.457+0000 7f20fec0b640 1 --2- 172.21.15.83:0/469503196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20f8161740 0x7f20f8160860 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f20e80ae040 tx=0x7f20e801aca0 comp rx=0 tx=0).stop 2024-04-17T10:43:32.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.457+0000 7f20fec0b640 1 -- 172.21.15.83:0/469503196 shutdown_connections 2024-04-17T10:43:32.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.457+0000 7f20fec0b640 1 --2- 172.21.15.83:0/469503196 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f20e4066f60 0x7f20e4069420 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:32.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.457+0000 7f20fec0b640 1 --2- 172.21.15.83:0/469503196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20f8161740 0x7f20f8160860 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:32.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.457+0000 7f20fec0b640 1 --2- 172.21.15.83:0/469503196 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20f815b760 0x7f20f8160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:32.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.457+0000 7f20fec0b640 1 --2- 172.21.15.83:0/469503196 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20f806b120 0x7f20f8071b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:32.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.457+0000 7f20fec0b640 1 -- 172.21.15.83:0/469503196 >> 172.21.15.83:0/469503196 conn(0x7f20f80cc050 msgr2=0x7f20f80cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:32.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.457+0000 7f20fec0b640 1 -- 172.21.15.83:0/469503196 shutdown_connections 2024-04-17T10:43:32.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.457+0000 7f20fec0b640 1 -- 172.21.15.83:0/469503196 wait complete. 2024-04-17T10:43:32.476 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-17T10:43:32.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.601+0000 7fb8e0d88640 1 Processor -- start 2024-04-17T10:43:32.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.601+0000 7fb8e0d88640 1 -- start start 2024-04-17T10:43:32.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.605+0000 7fb8e0d88640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb8dc154de0 0x7fb8dc06ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:32.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.605+0000 7fb8e0d88640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8dc06b350 0x7fb8dc0b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:32.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.605+0000 7fb8e0d88640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8dc15bd70 con 0x7fb8dc0b4040 2024-04-17T10:43:32.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.605+0000 7fb8e0d88640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8dc0b4870 con 0x7fb8dc154de0 2024-04-17T10:43:32.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.605+0000 7fb8e0d88640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc15bf60 con 0x7fb8dc06b350 2024-04-17T10:43:32.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.605+0000 7fb8d3fff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8dc06b350 0x7fb8dc0b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:32.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.605+0000 7fb8da59c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb8dc154de0 0x7fb8dc06ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:32.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.605+0000 7fb8dad9d640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fb8dc0b4040 0x7fb8dc071580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:52232/0 (socket says 172.21.15.83:52232) 2024-04-17T10:43:32.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.605+0000 7fb8d3fff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8dc06b350 0x7fb8dc0b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:35344/0 (socket says 172.21.15.83:35344) 2024-04-17T10:43:32.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.605+0000 7fb8dad9d640 1 -- 172.21.15.83:0/3520696358 learned_addr learned my addr 172.21.15.83:0/3520696358 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:43:32.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.605+0000 7fb8da59c640 1 -- 172.21.15.83:0/3520696358 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8dc06b350 msgr2=0x7fb8dc0b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:32.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.605+0000 7fb8da59c640 1 --2- 172.21.15.83:0/3520696358 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8dc06b350 0x7fb8dc0b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:32.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.605+0000 7fb8da59c640 1 -- 172.21.15.83:0/3520696358 >> v1:172.21.15.167:6789/0 conn(0x7fb8dc0b4040 legacy=0x7fb8dc071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:32.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.605+0000 7fb8da59c640 1 -- 172.21.15.83:0/3520696358 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8dc15afb0 con 0x7fb8dc154de0 2024-04-17T10:43:32.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.609+0000 7fb8da59c640 1 --2- 172.21.15.83:0/3520696358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8dc154de0 0x7fb8dc06ae10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc0097c0 tx=0x7fb8cc01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6084f689709909d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:32.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.609+0000 7fb8d9d9b640 1 -- 172.21.15.83:0/3520696358 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc0b0070 con 0x7fb8dc154de0 2024-04-17T10:43:32.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.609+0000 7fb8d9d9b640 1 -- 172.21.15.83:0/3520696358 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8cc0ab600 con 0x7fb8dc154de0 2024-04-17T10:43:32.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.609+0000 7fb8d9d9b640 1 -- 172.21.15.83:0/3520696358 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc0ab9d0 con 0x7fb8dc154de0 2024-04-17T10:43:32.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.609+0000 7fb8e0d88640 1 -- 172.21.15.83:0/3520696358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8dc154de0 msgr2=0x7fb8dc06ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:32.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.609+0000 7fb8e0d88640 1 --2- 172.21.15.83:0/3520696358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8dc154de0 0x7fb8dc06ae10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc0097c0 tx=0x7fb8cc01c4a0 comp rx=0 tx=0).stop 2024-04-17T10:43:32.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.609+0000 7fb8e0d88640 1 -- 172.21.15.83:0/3520696358 shutdown_connections 2024-04-17T10:43:32.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.609+0000 7fb8e0d88640 1 --2- 172.21.15.83:0/3520696358 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8dc06b350 0x7fb8dc0b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:32.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.609+0000 7fb8e0d88640 1 --2- 172.21.15.83:0/3520696358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8dc154de0 0x7fb8dc06ae10 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:32.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.609+0000 7fb8e0d88640 1 -- 172.21.15.83:0/3520696358 >> 172.21.15.83:0/3520696358 conn(0x7fb8dc0b15b0 msgr2=0x7fb8dc0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:32.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.609+0000 7fb8e0d88640 1 -- 172.21.15.83:0/3520696358 shutdown_connections 2024-04-17T10:43:32.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.609+0000 7fb8e0d88640 1 -- 172.21.15.83:0/3520696358 wait complete. 2024-04-17T10:43:32.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.609+0000 7fb8e0d88640 1 Processor -- start 2024-04-17T10:43:32.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.609+0000 7fb8e0d88640 1 -- start start 2024-04-17T10:43:32.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.609+0000 7fb8e0d88640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8dc06b350 0x7fb8dc150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:32.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.609+0000 7fb8e0d88640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8dc0b4040 0x7fb8dc13dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:32.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.609+0000 7fb8e0d88640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8dc154de0 0x7fb8dc0aefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:32.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.613+0000 7fb8e0d88640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc1551c0 con 0x7fb8dc154de0 2024-04-17T10:43:32.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.613+0000 7fb8e0d88640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc15b850 con 0x7fb8dc0b4040 2024-04-17T10:43:32.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.613+0000 7fb8e0d88640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc0b4450 con 0x7fb8dc06b350 2024-04-17T10:43:32.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.613+0000 7fb8dad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8dc154de0 0x7fb8dc0aefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:32.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.613+0000 7fb8dad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8dc154de0 0x7fb8dc0aefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51146/0 (socket says 172.21.15.83:51146) 2024-04-17T10:43:32.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.613+0000 7fb8dad9d640 1 -- 172.21.15.83:0/3491403622 learned_addr learned my addr 172.21.15.83:0/3491403622 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:32.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.613+0000 7fb8da59c640 1 --2- 172.21.15.83:0/3491403622 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8dc06b350 0x7fb8dc150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:32.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.613+0000 7fb8d3fff640 1 --2- 172.21.15.83:0/3491403622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8dc0b4040 0x7fb8dc13dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:32.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.613+0000 7fb8dad9d640 1 -- 172.21.15.83:0/3491403622 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8dc06b350 msgr2=0x7fb8dc150bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:32.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.613+0000 7fb8dad9d640 1 --2- 172.21.15.83:0/3491403622 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8dc06b350 0x7fb8dc150bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:32.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.613+0000 7fb8dad9d640 1 -- 172.21.15.83:0/3491403622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8dc0b4040 msgr2=0x7fb8dc13dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:32.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.613+0000 7fb8dad9d640 1 --2- 172.21.15.83:0/3491403622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8dc0b4040 0x7fb8dc13dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:32.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.613+0000 7fb8dad9d640 1 -- 172.21.15.83:0/3491403622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8dc14f370 con 0x7fb8dc154de0 2024-04-17T10:43:32.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.613+0000 7fb8d3fff640 1 --2- 172.21.15.83:0/3491403622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8dc0b4040 0x7fb8dc13dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:43:32.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.613+0000 7fb8dad9d640 1 --2- 172.21.15.83:0/3491403622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8dc154de0 0x7fb8dc0aefb0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fb8d400ca80 tx=0x7fb8d400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:32.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.613+0000 7fb8d2ffd640 1 -- 172.21.15.83:0/3491403622 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4017070 con 0x7fb8dc154de0 2024-04-17T10:43:32.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.613+0000 7fb8d2ffd640 1 -- 172.21.15.83:0/3491403622 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d400ac30 con 0x7fb8dc154de0 2024-04-17T10:43:32.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.613+0000 7fb8e0d88640 1 -- 172.21.15.83:0/3491403622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8dc14f650 con 0x7fb8dc154de0 2024-04-17T10:43:32.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.613+0000 7fb8d2ffd640 1 -- 172.21.15.83:0/3491403622 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4012600 con 0x7fb8dc154de0 2024-04-17T10:43:32.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.613+0000 7fb8e0d88640 1 -- 172.21.15.83:0/3491403622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8dc14fc10 con 0x7fb8dc154de0 2024-04-17T10:43:32.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.613+0000 7fb8e0d88640 1 -- 172.21.15.83:0/3491403622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8dc074f90 con 0x7fb8dc154de0 2024-04-17T10:43:32.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.613+0000 7fb8d2ffd640 1 -- 172.21.15.83:0/3491403622 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb8d401a070 con 0x7fb8dc154de0 2024-04-17T10:43:32.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.613+0000 7fb8d2ffd640 1 --2- 172.21.15.83:0/3491403622 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb8c0066f60 0x7fb8c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:32.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.613+0000 7fb8d2ffd640 1 -- 172.21.15.83:0/3491403622 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fb8d4084a60 con 0x7fb8dc154de0 2024-04-17T10:43:32.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.613+0000 7fb8da59c640 1 --2- 172.21.15.83:0/3491403622 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb8c0066f60 0x7fb8c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:32.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.613+0000 7fb8da59c640 1 --2- 172.21.15.83:0/3491403622 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb8c0066f60 0x7fb8c0069420 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc0ad000 tx=0x7fb8cc0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:32.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.617+0000 7fb8d2ffd640 1 -- 172.21.15.83:0/3491403622 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb8d400ae20 con 0x7fb8dc154de0 2024-04-17T10:43:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:32.777+0000 7fb8e0d88640 1 -- 172.21.15.83:0/3491403622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb8dc15a1e0 con 0x7fb8dc154de0 2024-04-17T10:43:33.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.457+0000 7fb8d2ffd640 1 -- 172.21.15.83:0/3491403622 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v72) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb8d4055600 con 0x7fb8dc154de0 2024-04-17T10:43:33.453 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2024-04-17T10:43:33.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.461+0000 7fb8e0d88640 1 -- 172.21.15.83:0/3491403622 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb8c0066f60 msgr2=0x7fb8c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:33.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.461+0000 7fb8e0d88640 1 --2- 172.21.15.83:0/3491403622 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb8c0066f60 0x7fb8c0069420 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc0ad000 tx=0x7fb8cc0a6000 comp rx=0 tx=0).stop 2024-04-17T10:43:33.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.461+0000 7fb8e0d88640 1 -- 172.21.15.83:0/3491403622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8dc154de0 msgr2=0x7fb8dc0aefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:33.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.461+0000 7fb8e0d88640 1 --2- 172.21.15.83:0/3491403622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8dc154de0 0x7fb8dc0aefb0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fb8d400ca80 tx=0x7fb8d400cf40 comp rx=0 tx=0).stop 2024-04-17T10:43:33.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.461+0000 7fb8e0d88640 1 -- 172.21.15.83:0/3491403622 shutdown_connections 2024-04-17T10:43:33.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.461+0000 7fb8e0d88640 1 --2- 172.21.15.83:0/3491403622 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb8c0066f60 0x7fb8c0069420 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:33.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.461+0000 7fb8e0d88640 1 --2- 172.21.15.83:0/3491403622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8dc154de0 0x7fb8dc0aefb0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:33.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.461+0000 7fb8e0d88640 1 --2- 172.21.15.83:0/3491403622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8dc0b4040 0x7fb8dc13dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:33.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.461+0000 7fb8e0d88640 1 --2- 172.21.15.83:0/3491403622 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8dc06b350 0x7fb8dc150bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:33.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.461+0000 7fb8e0d88640 1 -- 172.21.15.83:0/3491403622 >> 172.21.15.83:0/3491403622 conn(0x7fb8dc0b15b0 msgr2=0x7fb8dc0ae5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:33.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.461+0000 7fb8e0d88640 1 -- 172.21.15.83:0/3491403622 shutdown_connections 2024-04-17T10:43:33.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.461+0000 7fb8e0d88640 1 -- 172.21.15.83:0/3491403622 wait complete. 2024-04-17T10:43:33.488 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-17T10:43:33.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.617+0000 7f65a9514640 1 Processor -- start 2024-04-17T10:43:33.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.617+0000 7f65a9514640 1 -- start start 2024-04-17T10:43:33.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a9514640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65a4154de0 0x7f65a406ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:33.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a9514640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f65a40b4210 0x7f65a406cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:33.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a9514640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65a415bd70 con 0x7f65a406b350 2024-04-17T10:43:33.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a9514640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65a40b4870 con 0x7f65a40b4210 2024-04-17T10:43:33.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a9514640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a415bf60 con 0x7f65a4154de0 2024-04-17T10:43:33.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a27fc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f65a406b350 0x7f65a40b3b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:43530/0 (socket says 172.21.15.83:43530) 2024-04-17T10:43:33.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a27fc640 1 -- 172.21.15.83:0/1631217272 learned_addr learned my addr 172.21.15.83:0/1631217272 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:43:33.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a37fe640 1 --2- 172.21.15.83:0/1631217272 >> v2:172.21.15.167:3300/0 conn(0x7f65a40b4210 0x7f65a406cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:33.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a1ffb640 1 -- 172.21.15.83:0/1631217272 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2975777317 0 0) 0x7f65a415bd70 con 0x7f65a406b350 2024-04-17T10:43:33.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a1ffb640 1 -- 172.21.15.83:0/1631217272 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f658c003530 con 0x7f65a406b350 2024-04-17T10:43:33.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a2ffd640 1 --2- 172.21.15.83:0/1631217272 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65a4154de0 0x7f65a406ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:33.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a1ffb640 1 -- 172.21.15.83:0/1631217272 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1250485398 0 0) 0x7f658c003530 con 0x7f65a406b350 2024-04-17T10:43:33.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a1ffb640 1 -- 172.21.15.83:0/1631217272 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65a4154de0 msgr2=0x7f65a406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:33.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a1ffb640 1 --2- 172.21.15.83:0/1631217272 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65a4154de0 0x7f65a406ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:33.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a1ffb640 1 -- 172.21.15.83:0/1631217272 >> v2:172.21.15.167:3300/0 conn(0x7f65a40b4210 msgr2=0x7f65a406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:33.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a1ffb640 1 --2- 172.21.15.83:0/1631217272 >> v2:172.21.15.167:3300/0 conn(0x7f65a40b4210 0x7f65a406cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:33.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a1ffb640 1 -- 172.21.15.83:0/1631217272 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a415afb0 con 0x7f65a406b350 2024-04-17T10:43:33.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a1ffb640 1 -- 172.21.15.83:0/1631217272 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f6598001680 con 0x7f65a406b350 2024-04-17T10:43:33.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a1ffb640 1 -- 172.21.15.83:0/1631217272 >> v1:172.21.15.83:6789/0 conn(0x7f65a406b350 legacy=0x7f65a40b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:33.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a1ffb640 1 --2- 172.21.15.83:0/1631217272 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65a4154de0 0x7f658c003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:33.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a1ffb640 1 --2- 172.21.15.83:0/1631217272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f658c003fb0 0x7f658c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:33.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a1ffb640 1 --2- 172.21.15.83:0/1631217272 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f658c004d50 0x7f658c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:33.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a1ffb640 1 -- 172.21.15.83:0/1631217272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a415bf60 con 0x7f658c003fb0 2024-04-17T10:43:33.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a1ffb640 1 -- 172.21.15.83:0/1631217272 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658c00b9d0 con 0x7f658c004d50 2024-04-17T10:43:33.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a1ffb640 1 -- 172.21.15.83:0/1631217272 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f658c00bb80 con 0x7f65a4154de0 2024-04-17T10:43:33.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a27fc640 1 --2- 172.21.15.83:0/1631217272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f658c003fb0 0x7f658c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:33.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a2ffd640 1 --2- 172.21.15.83:0/1631217272 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65a4154de0 0x7f658c003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:33.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a27fc640 1 -- 172.21.15.83:0/1631217272 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65a4154de0 msgr2=0x7f658c003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:33.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a27fc640 1 --2- 172.21.15.83:0/1631217272 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65a4154de0 0x7f658c003a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:33.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a27fc640 1 -- 172.21.15.83:0/1631217272 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f658c004d50 msgr2=0x7f658c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:33.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a37fe640 1 --2- 172.21.15.83:0/1631217272 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f658c004d50 0x7f658c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:33.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a27fc640 1 --2- 172.21.15.83:0/1631217272 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f658c004d50 0x7f658c0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:33.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.621+0000 7f65a27fc640 1 -- 172.21.15.83:0/1631217272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f658c003530 con 0x7f658c003fb0 2024-04-17T10:43:33.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a27fc640 1 --2- 172.21.15.83:0/1631217272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f658c003fb0 0x7f658c004810 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f659800c240 tx=0x7f659801f430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d21f920ce726e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:33.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a1ffb640 1 -- 172.21.15.83:0/1631217272 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6598007d60 con 0x7f658c003fb0 2024-04-17T10:43:33.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a1ffb640 1 -- 172.21.15.83:0/1631217272 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6598005b00 con 0x7f658c003fb0 2024-04-17T10:43:33.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a1ffb640 1 -- 172.21.15.83:0/1631217272 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6598007070 con 0x7f658c003fb0 2024-04-17T10:43:33.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a9514640 1 -- 172.21.15.83:0/1631217272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f658c003fb0 msgr2=0x7f658c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:33.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a9514640 1 --2- 172.21.15.83:0/1631217272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f658c003fb0 0x7f658c004810 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f659800c240 tx=0x7f659801f430 comp rx=0 tx=0).stop 2024-04-17T10:43:33.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a37fe640 1 -- 172.21.15.83:0/1631217272 reap_dead start 2024-04-17T10:43:33.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a9514640 1 -- 172.21.15.83:0/1631217272 shutdown_connections 2024-04-17T10:43:33.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a9514640 1 -- 172.21.15.83:0/1631217272 >> 172.21.15.83:0/1631217272 conn(0x7f65a40b15b0 msgr2=0x7f65a40b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:33.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a9514640 1 -- 172.21.15.83:0/1631217272 shutdown_connections 2024-04-17T10:43:33.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a9514640 1 -- 172.21.15.83:0/1631217272 wait complete. 2024-04-17T10:43:33.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a9514640 1 Processor -- start 2024-04-17T10:43:33.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a9514640 1 -- start start 2024-04-17T10:43:33.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a9514640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f658c003fb0 0x7f65a4157a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:33.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a9514640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65a41571e0 0x7f65a4145c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:33.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a9514640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65a4146160 0x7f65a4155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:33.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a9514640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a415b850 con 0x7f65a4146160 2024-04-17T10:43:33.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a9514640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a415b570 con 0x7f65a41571e0 2024-04-17T10:43:33.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a9514640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a41551c0 con 0x7f658c003fb0 2024-04-17T10:43:33.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65a4146160 0x7f65a4155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:33.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65a4146160 0x7f65a4155430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51160/0 (socket says 172.21.15.83:51160) 2024-04-17T10:43:33.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a37fe640 1 -- 172.21.15.83:0/3061992990 learned_addr learned my addr 172.21.15.83:0/3061992990 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:33.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a27fc640 1 --2- 172.21.15.83:0/3061992990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65a41571e0 0x7f65a4145c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:33.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a2ffd640 1 --2- 172.21.15.83:0/3061992990 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f658c003fb0 0x7f65a4157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:33.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a37fe640 1 -- 172.21.15.83:0/3061992990 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f658c003fb0 msgr2=0x7f65a4157a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:33.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a37fe640 1 --2- 172.21.15.83:0/3061992990 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f658c003fb0 0x7f65a4157a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:33.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a37fe640 1 -- 172.21.15.83:0/3061992990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65a41571e0 msgr2=0x7f65a4145c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:33.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a37fe640 1 --2- 172.21.15.83:0/3061992990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65a41571e0 0x7f65a4145c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:33.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a37fe640 1 -- 172.21.15.83:0/3061992990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a4146610 con 0x7f65a4146160 2024-04-17T10:43:33.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a27fc640 1 --2- 172.21.15.83:0/3061992990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65a41571e0 0x7f65a4145c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:33.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a37fe640 1 --2- 172.21.15.83:0/3061992990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65a4146160 0x7f65a4155430 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f65a415e2f0 tx=0x7f659c007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:33.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f658b7fe640 1 -- 172.21.15.83:0/3061992990 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c003ba0 con 0x7f65a4146160 2024-04-17T10:43:33.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f658b7fe640 1 -- 172.21.15.83:0/3061992990 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c003d60 con 0x7f65a4146160 2024-04-17T10:43:33.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a9514640 1 -- 172.21.15.83:0/3061992990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a4156140 con 0x7f65a4146160 2024-04-17T10:43:33.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f658b7fe640 1 -- 172.21.15.83:0/3061992990 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c013890 con 0x7f65a4146160 2024-04-17T10:43:33.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.625+0000 7f65a9514640 1 -- 172.21.15.83:0/3061992990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a4156700 con 0x7f65a4146160 2024-04-17T10:43:33.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.629+0000 7f65a9514640 1 -- 172.21.15.83:0/3061992990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a4074f90 con 0x7f65a4146160 2024-04-17T10:43:33.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.629+0000 7f658b7fe640 1 -- 172.21.15.83:0/3061992990 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f659c01b030 con 0x7f65a4146160 2024-04-17T10:43:33.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.629+0000 7f658b7fe640 1 --2- 172.21.15.83:0/3061992990 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6590066f90 0x7f6590069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:33.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.629+0000 7f658b7fe640 1 -- 172.21.15.83:0/3061992990 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f659c087df0 con 0x7f65a4146160 2024-04-17T10:43:33.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.629+0000 7f65a2ffd640 1 --2- 172.21.15.83:0/3061992990 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6590066f90 0x7f6590069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:33.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.629+0000 7f65a2ffd640 1 --2- 172.21.15.83:0/3061992990 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6590066f90 0x7f6590069450 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6594009850 tx=0x7f6594008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:33.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.633+0000 7f658b7fe640 1 -- 172.21.15.83:0/3061992990 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f659c022030 con 0x7f65a4146160 2024-04-17T10:43:33.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:33.825+0000 7f65a9514640 1 -- 172.21.15.83:0/3061992990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f65a41595b0 con 0x7f65a4146160 2024-04-17T10:43:34.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.497+0000 7f658b7fe640 1 -- 172.21.15.83:0/3061992990 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v44) v1 ==== 174+0+0 (secure 0 0 0) 0x7f659c007e20 con 0x7f65a4146160 2024-04-17T10:43:34.501 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 10 and data pool 11 2024-04-17T10:43:34.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.509+0000 7f65a9514640 1 -- 172.21.15.83:0/3061992990 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6590066f90 msgr2=0x7f6590069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:34.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.509+0000 7f65a9514640 1 --2- 172.21.15.83:0/3061992990 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6590066f90 0x7f6590069450 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6594009850 tx=0x7f6594008040 comp rx=0 tx=0).stop 2024-04-17T10:43:34.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.509+0000 7f65a9514640 1 -- 172.21.15.83:0/3061992990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65a4146160 msgr2=0x7f65a4155430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:34.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.509+0000 7f65a9514640 1 --2- 172.21.15.83:0/3061992990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65a4146160 0x7f65a4155430 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f65a415e2f0 tx=0x7f659c007700 comp rx=0 tx=0).stop 2024-04-17T10:43:34.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.517+0000 7f65a9514640 1 -- 172.21.15.83:0/3061992990 shutdown_connections 2024-04-17T10:43:34.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.517+0000 7f65a9514640 1 --2- 172.21.15.83:0/3061992990 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6590066f90 0x7f6590069450 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:34.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.517+0000 7f65a9514640 1 --2- 172.21.15.83:0/3061992990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65a4146160 0x7f65a4155430 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:34.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.517+0000 7f65a9514640 1 --2- 172.21.15.83:0/3061992990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65a41571e0 0x7f65a4145c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:34.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.517+0000 7f65a9514640 1 --2- 172.21.15.83:0/3061992990 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f658c003fb0 0x7f65a4157a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:34.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.517+0000 7f65a9514640 1 -- 172.21.15.83:0/3061992990 >> 172.21.15.83:0/3061992990 conn(0x7f65a40b15b0 msgr2=0x7f65a415db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:34.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.517+0000 7f65a9514640 1 -- 172.21.15.83:0/3061992990 shutdown_connections 2024-04-17T10:43:34.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.517+0000 7f65a9514640 1 -- 172.21.15.83:0/3061992990 wait complete. 2024-04-17T10:43:34.539 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:43:34.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.661+0000 7f0525ced640 1 Processor -- start 2024-04-17T10:43:34.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.661+0000 7f0525ced640 1 -- start start 2024-04-17T10:43:34.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.661+0000 7f0525ced640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0520072330 0x7f05200cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:34.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.661+0000 7f0525ced640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05200cfa10 0x7f052006c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:34.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.661+0000 7f0525ced640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05200cfea0 con 0x7f052015ba30 2024-04-17T10:43:34.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.661+0000 7f0525ced640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0520071ea0 con 0x7f0520072330 2024-04-17T10:43:34.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.661+0000 7f0525ced640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052006e320 con 0x7f05200cfa10 2024-04-17T10:43:34.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.661+0000 7f0516ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0520072330 0x7f05200cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:34.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.661+0000 7f0516ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0520072330 0x7f05200cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51172/0 (socket says 172.21.15.83:51172) 2024-04-17T10:43:34.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.661+0000 7f0516ffd640 1 -- 172.21.15.83:0/1685486955 learned_addr learned my addr 172.21.15.83:0/1685486955 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:34.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.661+0000 7f0516ffd640 1 -- 172.21.15.83:0/1685486955 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05200cfa10 msgr2=0x7f052006c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:34.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.661+0000 7f051ffff640 1 --2- 172.21.15.83:0/1685486955 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05200cfa10 0x7f052006c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:34.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.661+0000 7f0516ffd640 1 --2- 172.21.15.83:0/1685486955 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05200cfa10 0x7f052006c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:34.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.661+0000 7f0516ffd640 1 -- 172.21.15.83:0/1685486955 >> v1:172.21.15.83:6789/0 conn(0x7f052015ba30 legacy=0x7f052015c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:34.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.661+0000 7f0516ffd640 1 -- 172.21.15.83:0/1685486955 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052006e040 con 0x7f0520072330 2024-04-17T10:43:34.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.661+0000 7f051effd640 1 -- 172.21.15.83:0/1685486955 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2325777391 0 0) 0x7f05200cfea0 con 0x7f052015ba30 2024-04-17T10:43:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f0516ffd640 1 --2- 172.21.15.83:0/1685486955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0520072330 0x7f05200cf4d0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f0520072020 tx=0x7f050801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551373750bb59caf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f051effd640 1 -- 172.21.15.83:0/1685486955 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05080b1070 con 0x7f0520072330 2024-04-17T10:43:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f051effd640 1 -- 172.21.15.83:0/1685486955 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05080ac620 con 0x7f0520072330 2024-04-17T10:43:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f051effd640 1 -- 172.21.15.83:0/1685486955 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05080ac9f0 con 0x7f0520072330 2024-04-17T10:43:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f0525ced640 1 -- 172.21.15.83:0/1685486955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0520072330 msgr2=0x7f05200cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f0525ced640 1 --2- 172.21.15.83:0/1685486955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0520072330 0x7f05200cf4d0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f0520072020 tx=0x7f050801c4b0 comp rx=0 tx=0).stop 2024-04-17T10:43:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f0525ced640 1 -- 172.21.15.83:0/1685486955 shutdown_connections 2024-04-17T10:43:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f0525ced640 1 --2- 172.21.15.83:0/1685486955 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05200cfa10 0x7f052006c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f0525ced640 1 --2- 172.21.15.83:0/1685486955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0520072330 0x7f05200cf4d0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f0525ced640 1 -- 172.21.15.83:0/1685486955 >> 172.21.15.83:0/1685486955 conn(0x7f052006abe0 msgr2=0x7f052006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f0525ced640 1 -- 172.21.15.83:0/1685486955 shutdown_connections 2024-04-17T10:43:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f0525ced640 1 -- 172.21.15.83:0/1685486955 wait complete. 2024-04-17T10:43:34.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f0525ced640 1 Processor -- start 2024-04-17T10:43:34.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f0525ced640 1 -- start start 2024-04-17T10:43:34.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f0525ced640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0520072330 0x7f0520150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:34.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f0525ced640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05200cfa10 0x7f052015db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:34.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f0525ced640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f052015ba30 0x7f05201541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:34.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f0525ced640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052006de20 con 0x7f052015ba30 2024-04-17T10:43:34.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f0525ced640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052006dca0 con 0x7f05200cfa10 2024-04-17T10:43:34.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f0525ced640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0520168520 con 0x7f0520072330 2024-04-17T10:43:34.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f051ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f052015ba30 0x7f05201541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:34.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f051ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f052015ba30 0x7f05201541e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51178/0 (socket says 172.21.15.83:51178) 2024-04-17T10:43:34.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f051ffff640 1 -- 172.21.15.83:0/2908166675 learned_addr learned my addr 172.21.15.83:0/2908166675 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:34.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f051f7fe640 1 --2- 172.21.15.83:0/2908166675 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0520072330 0x7f0520150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:34.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f0516ffd640 1 --2- 172.21.15.83:0/2908166675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05200cfa10 0x7f052015db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:34.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f051ffff640 1 -- 172.21.15.83:0/2908166675 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0520072330 msgr2=0x7f0520150550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:34.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f051ffff640 1 --2- 172.21.15.83:0/2908166675 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0520072330 0x7f0520150550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:34.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f051ffff640 1 -- 172.21.15.83:0/2908166675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05200cfa10 msgr2=0x7f052015db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:34.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f051ffff640 1 --2- 172.21.15.83:0/2908166675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05200cfa10 0x7f052015db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:34.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f051ffff640 1 -- 172.21.15.83:0/2908166675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0520150a90 con 0x7f052015ba30 2024-04-17T10:43:34.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f0516ffd640 1 --2- 172.21.15.83:0/2908166675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05200cfa10 0x7f052015db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:34.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f051ffff640 1 --2- 172.21.15.83:0/2908166675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f052015ba30 0x7f05201541e0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f051800cc30 tx=0x7f051800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:34.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f051cff9640 1 -- 172.21.15.83:0/2908166675 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051800ce40 con 0x7f052015ba30 2024-04-17T10:43:34.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f051cff9640 1 -- 172.21.15.83:0/2908166675 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05180039a0 con 0x7f052015ba30 2024-04-17T10:43:34.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f051cff9640 1 -- 172.21.15.83:0/2908166675 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0518003d50 con 0x7f052015ba30 2024-04-17T10:43:34.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.665+0000 7f0525ced640 1 -- 172.21.15.83:0/2908166675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05201f9d20 con 0x7f052015ba30 2024-04-17T10:43:34.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.669+0000 7f0525ced640 1 -- 172.21.15.83:0/2908166675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05201fa2e0 con 0x7f052015ba30 2024-04-17T10:43:34.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.669+0000 7f0525ced640 1 -- 172.21.15.83:0/2908166675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0520074f90 con 0x7f052015ba30 2024-04-17T10:43:34.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.669+0000 7f051cff9640 1 -- 172.21.15.83:0/2908166675 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0518003ef0 con 0x7f052015ba30 2024-04-17T10:43:34.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.669+0000 7f051cff9640 1 --2- 172.21.15.83:0/2908166675 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0500066f90 0x7f0500069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:34.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.669+0000 7f051cff9640 1 -- 172.21.15.83:0/2908166675 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05180858c0 con 0x7f052015ba30 2024-04-17T10:43:34.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.669+0000 7f051f7fe640 1 --2- 172.21.15.83:0/2908166675 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0500066f90 0x7f0500069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:34.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.669+0000 7f051f7fe640 1 --2- 172.21.15.83:0/2908166675 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0500066f90 0x7f0500069450 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0510000f70 tx=0x7f0510009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:34.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.673+0000 7f051cff9640 1 -- 172.21.15.83:0/2908166675 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0518014030 con 0x7f052015ba30 2024-04-17T10:43:34.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.833+0000 7f0525ced640 1 -- 172.21.15.83:0/2908166675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0520074480 con 0x7f052015ba30 2024-04-17T10:43:34.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.833+0000 7f051cff9640 1 -- 172.21.15.83:0/2908166675 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16609 (secure 0 0 0) 0x7f05180568d0 con 0x7f052015ba30 2024-04-17T10:43:34.831 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:43:34.832 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":73,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:43:33.829998+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:43:32.090869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-17T10:43:32.815921+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:43:34.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.837+0000 7f0525ced640 1 -- 172.21.15.83:0/2908166675 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0500066f90 msgr2=0x7f0500069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:34.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.837+0000 7f0525ced640 1 --2- 172.21.15.83:0/2908166675 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0500066f90 0x7f0500069450 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0510000f70 tx=0x7f0510009040 comp rx=0 tx=0).stop 2024-04-17T10:43:34.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.837+0000 7f0525ced640 1 -- 172.21.15.83:0/2908166675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f052015ba30 msgr2=0x7f05201541e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:34.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.837+0000 7f0525ced640 1 --2- 172.21.15.83:0/2908166675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f052015ba30 0x7f05201541e0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f051800cc30 tx=0x7f051800a5c0 comp rx=0 tx=0).stop 2024-04-17T10:43:34.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.837+0000 7f0525ced640 1 -- 172.21.15.83:0/2908166675 shutdown_connections 2024-04-17T10:43:34.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.837+0000 7f0525ced640 1 --2- 172.21.15.83:0/2908166675 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0500066f90 0x7f0500069450 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:34.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.837+0000 7f0525ced640 1 --2- 172.21.15.83:0/2908166675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f052015ba30 0x7f05201541e0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:34.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.837+0000 7f0525ced640 1 --2- 172.21.15.83:0/2908166675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05200cfa10 0x7f052015db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:34.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.837+0000 7f0525ced640 1 --2- 172.21.15.83:0/2908166675 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0520072330 0x7f0520150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:34.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.837+0000 7f0525ced640 1 -- 172.21.15.83:0/2908166675 >> 172.21.15.83:0/2908166675 conn(0x7f052006abe0 msgr2=0x7f0520070470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:34.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.837+0000 7f0525ced640 1 -- 172.21.15.83:0/2908166675 shutdown_connections 2024-04-17T10:43:34.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.837+0000 7f0525ced640 1 -- 172.21.15.83:0/2908166675 wait complete. 2024-04-17T10:43:34.851 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:43:34.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.969+0000 7fa603156640 1 Processor -- start 2024-04-17T10:43:34.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.969+0000 7fa603156640 1 -- start start 2024-04-17T10:43:34.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.973+0000 7fa603156640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5fc070b80 0x7fa5fc164d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:34.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.973+0000 7fa603156640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5fc0b2bd0 con 0x7fa5fc0b3990 2024-04-17T10:43:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.973+0000 7fa603156640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5fc0b2eb0 con 0x7fa5fc0cf060 2024-04-17T10:43:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.973+0000 7fa603156640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc0b2980 con 0x7fa5fc070b80 2024-04-17T10:43:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.973+0000 7fa5fbfff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa5fc0b3990 0x7fa5fc070470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:43548/0 (socket says 172.21.15.83:43548) 2024-04-17T10:43:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.973+0000 7fa5fbfff640 1 -- 172.21.15.83:0/4013516283 learned_addr learned my addr 172.21.15.83:0/4013516283 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:43:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.973+0000 7fa6016f3640 1 --2- 172.21.15.83:0/4013516283 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5fc070b80 0x7fa5fc164d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.973+0000 7fa5fb7fe640 1 -- 172.21.15.83:0/4013516283 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 332717433 0 0) 0x7fa5fc0b2bd0 con 0x7fa5fc0b3990 2024-04-17T10:43:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.973+0000 7fa5fb7fe640 1 -- 172.21.15.83:0/4013516283 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5e8003530 con 0x7fa5fc0b3990 2024-04-17T10:43:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.973+0000 7fa5fb7fe640 1 -- 172.21.15.83:0/4013516283 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 169593110 0 0) 0x7fa5e8003530 con 0x7fa5fc0b3990 2024-04-17T10:43:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.973+0000 7fa5fb7fe640 1 -- 172.21.15.83:0/4013516283 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5fc070b80 msgr2=0x7fa5fc164d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.973+0000 7fa5fb7fe640 1 --2- 172.21.15.83:0/4013516283 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5fc070b80 0x7fa5fc164d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.973+0000 7fa5fb7fe640 1 -- 172.21.15.83:0/4013516283 >> v1:172.21.15.167:6789/0 conn(0x7fa5fc0cf060 legacy=0x7fa5fc0b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.973+0000 7fa5fb7fe640 1 -- 172.21.15.83:0/4013516283 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5fc0b3190 con 0x7fa5fc0b3990 2024-04-17T10:43:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.973+0000 7fa5fb7fe640 1 -- 172.21.15.83:0/4013516283 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fa5e4001680 con 0x7fa5fc0b3990 2024-04-17T10:43:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.973+0000 7fa5fb7fe640 1 -- 172.21.15.83:0/4013516283 >> v1:172.21.15.83:6789/0 conn(0x7fa5fc0b3990 legacy=0x7fa5fc070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.973+0000 7fa5fb7fe640 1 --2- 172.21.15.83:0/4013516283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5e80039f0 0x7fa5e8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.973+0000 7fa5fb7fe640 1 --2- 172.21.15.83:0/4013516283 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5fc070b80 0x7fa5e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.973+0000 7fa5fb7fe640 1 --2- 172.21.15.83:0/4013516283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5e8004d50 0x7fa5e80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.973+0000 7fa5fb7fe640 1 -- 172.21.15.83:0/4013516283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc0b2980 con 0x7fa5e80039f0 2024-04-17T10:43:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.973+0000 7fa5fb7fe640 1 -- 172.21.15.83:0/4013516283 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e800cbc0 con 0x7fa5e8004d50 2024-04-17T10:43:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.973+0000 7fa5fb7fe640 1 -- 172.21.15.83:0/4013516283 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e800cdc0 con 0x7fa5fc070b80 2024-04-17T10:43:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.973+0000 7fa600ef2640 1 --2- 172.21.15.83:0/4013516283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5e80039f0 0x7fa5e8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.973+0000 7fa5fbfff640 1 --2- 172.21.15.83:0/4013516283 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5fc070b80 0x7fa5e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.973+0000 7fa6016f3640 1 --2- 172.21.15.83:0/4013516283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5e8004d50 0x7fa5e80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.973+0000 7fa600ef2640 1 -- 172.21.15.83:0/4013516283 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5fc070b80 msgr2=0x7fa5e8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.973+0000 7fa600ef2640 1 --2- 172.21.15.83:0/4013516283 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5fc070b80 0x7fa5e8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.973+0000 7fa600ef2640 1 -- 172.21.15.83:0/4013516283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5e8004d50 msgr2=0x7fa5e80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:34.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.973+0000 7fa600ef2640 1 --2- 172.21.15.83:0/4013516283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5e8004d50 0x7fa5e80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:34.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.973+0000 7fa600ef2640 1 -- 172.21.15.83:0/4013516283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e8003530 con 0x7fa5e80039f0 2024-04-17T10:43:34.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.981+0000 7fa600ef2640 1 --2- 172.21.15.83:0/4013516283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5e80039f0 0x7fa5e8003e10 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4001e40 tx=0x7fa5f401c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a577547364b7f928 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:34.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.981+0000 7fa5fb7fe640 1 -- 172.21.15.83:0/4013516283 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4001a90 con 0x7fa5e80039f0 2024-04-17T10:43:34.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.981+0000 7fa5fb7fe640 1 -- 172.21.15.83:0/4013516283 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f4003b80 con 0x7fa5e80039f0 2024-04-17T10:43:34.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.981+0000 7fa5fb7fe640 1 -- 172.21.15.83:0/4013516283 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f40050d0 con 0x7fa5e80039f0 2024-04-17T10:43:34.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.981+0000 7fa603156640 1 -- 172.21.15.83:0/4013516283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5e80039f0 msgr2=0x7fa5e8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:34.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.981+0000 7fa603156640 1 --2- 172.21.15.83:0/4013516283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5e80039f0 0x7fa5e8003e10 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4001e40 tx=0x7fa5f401c730 comp rx=0 tx=0).stop 2024-04-17T10:43:34.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.981+0000 7fa6016f3640 1 -- 172.21.15.83:0/4013516283 reap_dead start 2024-04-17T10:43:34.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.981+0000 7fa603156640 1 -- 172.21.15.83:0/4013516283 shutdown_connections 2024-04-17T10:43:34.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.981+0000 7fa603156640 1 -- 172.21.15.83:0/4013516283 >> 172.21.15.83:0/4013516283 conn(0x7fa5fc06abe0 msgr2=0x7fa5fc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:34.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.981+0000 7fa603156640 1 -- 172.21.15.83:0/4013516283 shutdown_connections 2024-04-17T10:43:34.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.981+0000 7fa603156640 1 -- 172.21.15.83:0/4013516283 wait complete. 2024-04-17T10:43:34.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.981+0000 7fa603156640 1 Processor -- start 2024-04-17T10:43:34.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.981+0000 7fa603156640 1 -- start start 2024-04-17T10:43:34.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.981+0000 7fa603156640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5fc1479d0 0x7fa5fc071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:34.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.981+0000 7fa603156640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5fc159520 0x7fa5fc0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:34.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.981+0000 7fa603156640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5fc147db0 0x7fa5fc0add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:34.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.981+0000 7fa603156640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc164f00 con 0x7fa5fc159520 2024-04-17T10:43:34.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.981+0000 7fa603156640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc164d80 con 0x7fa5fc147db0 2024-04-17T10:43:34.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.981+0000 7fa603156640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc165080 con 0x7fa5fc1479d0 2024-04-17T10:43:34.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.981+0000 7fa5fbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5fc159520 0x7fa5fc0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:34.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.981+0000 7fa5fbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5fc159520 0x7fa5fc0ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51202/0 (socket says 172.21.15.83:51202) 2024-04-17T10:43:34.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.981+0000 7fa5fbfff640 1 -- 172.21.15.83:0/180309811 learned_addr learned my addr 172.21.15.83:0/180309811 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:34.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.981+0000 7fa5fbfff640 1 -- 172.21.15.83:0/180309811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5fc1479d0 msgr2=0x7fa5fc071180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:34.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.981+0000 7fa5fbfff640 1 --2- 172.21.15.83:0/180309811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5fc1479d0 0x7fa5fc071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:34.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.981+0000 7fa5fbfff640 1 -- 172.21.15.83:0/180309811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5fc147db0 msgr2=0x7fa5fc0add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:34.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.981+0000 7fa5fbfff640 1 --2- 172.21.15.83:0/180309811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5fc147db0 0x7fa5fc0add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:34.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.981+0000 7fa5fbfff640 1 -- 172.21.15.83:0/180309811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5fc159a20 con 0x7fa5fc159520 2024-04-17T10:43:34.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.985+0000 7fa5fbfff640 1 --2- 172.21.15.83:0/180309811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5fc159520 0x7fa5fc0ad840 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4007340 tx=0x7fa5e4004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:34.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.985+0000 7fa5f97fa640 1 -- 172.21.15.83:0/180309811 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e401a070 con 0x7fa5fc159520 2024-04-17T10:43:34.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.985+0000 7fa5f97fa640 1 -- 172.21.15.83:0/180309811 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e400f9c0 con 0x7fa5fc159520 2024-04-17T10:43:34.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.985+0000 7fa603156640 1 -- 172.21.15.83:0/180309811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5fc15d7a0 con 0x7fa5fc159520 2024-04-17T10:43:34.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.985+0000 7fa5f97fa640 1 -- 172.21.15.83:0/180309811 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e400fd90 con 0x7fa5fc159520 2024-04-17T10:43:34.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.985+0000 7fa603156640 1 -- 172.21.15.83:0/180309811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5fc15dd60 con 0x7fa5fc159520 2024-04-17T10:43:34.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.985+0000 7fa603156640 1 -- 172.21.15.83:0/180309811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5fc074f90 con 0x7fa5fc159520 2024-04-17T10:43:34.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.985+0000 7fa5f97fa640 1 -- 172.21.15.83:0/180309811 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa5e401d070 con 0x7fa5fc159520 2024-04-17T10:43:34.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.985+0000 7fa5f97fa640 1 --2- 172.21.15.83:0/180309811 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa5dc066f90 0x7fa5dc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:34.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.985+0000 7fa600ef2640 1 --2- 172.21.15.83:0/180309811 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa5dc066f90 0x7fa5dc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:34.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.985+0000 7fa5f97fa640 1 -- 172.21.15.83:0/180309811 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa5e4084900 con 0x7fa5fc159520 2024-04-17T10:43:34.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.985+0000 7fa600ef2640 1 --2- 172.21.15.83:0/180309811 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa5dc066f90 0x7fa5dc069450 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa5f40ad000 tx=0x7fa5f40b2040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:34.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:34.989+0000 7fa5f97fa640 1 -- 172.21.15.83:0/180309811 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa5e40559a0 con 0x7fa5fc159520 2024-04-17T10:43:35.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.149+0000 7fa603156640 1 -- 172.21.15.83:0/180309811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5fc158a60 con 0x7fa5fc159520 2024-04-17T10:43:35.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.153+0000 7fa5f97fa640 1 -- 172.21.15.83:0/180309811 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16609 (secure 0 0 0) 0x7fa5e40552a0 con 0x7fa5fc159520 2024-04-17T10:43:35.148 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:43:35.148 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":73,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:43:33.829998+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:43:32.090869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-17T10:43:32.815921+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:43:35.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.153+0000 7fa603156640 1 -- 172.21.15.83:0/180309811 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa5dc066f90 msgr2=0x7fa5dc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:35.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.153+0000 7fa603156640 1 --2- 172.21.15.83:0/180309811 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa5dc066f90 0x7fa5dc069450 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa5f40ad000 tx=0x7fa5f40b2040 comp rx=0 tx=0).stop 2024-04-17T10:43:35.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.153+0000 7fa603156640 1 -- 172.21.15.83:0/180309811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5fc159520 msgr2=0x7fa5fc0ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:35.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.153+0000 7fa603156640 1 --2- 172.21.15.83:0/180309811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5fc159520 0x7fa5fc0ad840 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4007340 tx=0x7fa5e4004f90 comp rx=0 tx=0).stop 2024-04-17T10:43:35.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.153+0000 7fa603156640 1 -- 172.21.15.83:0/180309811 shutdown_connections 2024-04-17T10:43:35.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.153+0000 7fa603156640 1 --2- 172.21.15.83:0/180309811 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa5dc066f90 0x7fa5dc069450 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:35.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.153+0000 7fa603156640 1 --2- 172.21.15.83:0/180309811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5fc147db0 0x7fa5fc0add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:35.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.153+0000 7fa603156640 1 --2- 172.21.15.83:0/180309811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5fc159520 0x7fa5fc0ad840 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:35.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.153+0000 7fa603156640 1 --2- 172.21.15.83:0/180309811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5fc1479d0 0x7fa5fc071180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:35.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.153+0000 7fa603156640 1 -- 172.21.15.83:0/180309811 >> 172.21.15.83:0/180309811 conn(0x7fa5fc06abe0 msgr2=0x7fa5fc15a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:35.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.153+0000 7fa603156640 1 -- 172.21.15.83:0/180309811 shutdown_connections 2024-04-17T10:43:35.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.153+0000 7fa603156640 1 -- 172.21.15.83:0/180309811 wait complete. 2024-04-17T10:43:35.168 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-17T10:43:35.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.293+0000 7faf02fca640 1 Processor -- start 2024-04-17T10:43:35.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.293+0000 7faf02fca640 1 -- start start 2024-04-17T10:43:35.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.293+0000 7faf02fca640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faefc15b6e0 0x7faefc071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:35.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.293+0000 7faf02fca640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faefc070470 0x7faefc06ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:35.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.293+0000 7faf02fca640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faefc06c1f0 con 0x7faefc15bac0 2024-04-17T10:43:35.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.293+0000 7faf02fca640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faefc070a10 con 0x7faefc15b6e0 2024-04-17T10:43:35.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.293+0000 7faf02fca640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faefc070cf0 con 0x7faefc070470 2024-04-17T10:43:35.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.293+0000 7faf00d66640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faefc15b6e0 0x7faefc071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:35.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.293+0000 7faf00d66640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faefc15b6e0 0x7faefc071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51204/0 (socket says 172.21.15.83:51204) 2024-04-17T10:43:35.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.293+0000 7faf00d66640 1 -- 172.21.15.83:0/4104941212 learned_addr learned my addr 172.21.15.83:0/4104941212 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:35.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.293+0000 7faf01567640 1 --2- 172.21.15.83:0/4104941212 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faefc070470 0x7faefc06ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:35.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.293+0000 7faf00d66640 1 -- 172.21.15.83:0/4104941212 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faefc070470 msgr2=0x7faefc06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:35.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.293+0000 7faf00d66640 1 --2- 172.21.15.83:0/4104941212 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faefc070470 0x7faefc06ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:35.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.293+0000 7faf00d66640 1 -- 172.21.15.83:0/4104941212 >> v1:172.21.15.167:6789/0 conn(0x7faefc15bac0 legacy=0x7faefc0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:35.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.293+0000 7faf00d66640 1 -- 172.21.15.83:0/4104941212 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faefc070fd0 con 0x7faefc15b6e0 2024-04-17T10:43:35.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.293+0000 7faf01567640 1 --2- 172.21.15.83:0/4104941212 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faefc070470 0x7faefc06ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:35.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.297+0000 7faf00d66640 1 --2- 172.21.15.83:0/4104941212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faefc15b6e0 0x7faefc071cc0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7faeec009870 tx=0x7faeec01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d34e34c3999eb43f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:35.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.297+0000 7faefb7fe640 1 -- 172.21.15.83:0/4104941212 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec0b0070 con 0x7faefc15b6e0 2024-04-17T10:43:35.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.297+0000 7faefb7fe640 1 -- 172.21.15.83:0/4104941212 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeec0ab600 con 0x7faefc15b6e0 2024-04-17T10:43:35.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.297+0000 7faefb7fe640 1 -- 172.21.15.83:0/4104941212 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec0ab9d0 con 0x7faefc15b6e0 2024-04-17T10:43:35.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.297+0000 7faf02fca640 1 -- 172.21.15.83:0/4104941212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faefc15b6e0 msgr2=0x7faefc071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:35.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.297+0000 7faf02fca640 1 --2- 172.21.15.83:0/4104941212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faefc15b6e0 0x7faefc071cc0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7faeec009870 tx=0x7faeec01c4d0 comp rx=0 tx=0).stop 2024-04-17T10:43:35.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.297+0000 7faf02fca640 1 -- 172.21.15.83:0/4104941212 shutdown_connections 2024-04-17T10:43:35.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.297+0000 7faf02fca640 1 --2- 172.21.15.83:0/4104941212 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faefc070470 0x7faefc06ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:35.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.297+0000 7faf02fca640 1 --2- 172.21.15.83:0/4104941212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faefc15b6e0 0x7faefc071cc0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:35.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.297+0000 7faf02fca640 1 -- 172.21.15.83:0/4104941212 >> 172.21.15.83:0/4104941212 conn(0x7faefc0b15b0 msgr2=0x7faefc0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:35.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.297+0000 7faf02fca640 1 -- 172.21.15.83:0/4104941212 shutdown_connections 2024-04-17T10:43:35.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.297+0000 7faf02fca640 1 -- 172.21.15.83:0/4104941212 wait complete. 2024-04-17T10:43:35.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.297+0000 7faf02fca640 1 Processor -- start 2024-04-17T10:43:35.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.297+0000 7faf02fca640 1 -- start start 2024-04-17T10:43:35.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.297+0000 7faf02fca640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faefc070470 0x7faefc154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:35.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.297+0000 7faf02fca640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faefc15b6e0 0x7faefc152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:35.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.297+0000 7faf02fca640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faefc15bac0 0x7faefc152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:35.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.297+0000 7faf02fca640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faefc06d2b0 con 0x7faefc070470 2024-04-17T10:43:35.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.297+0000 7faf02fca640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faefc06d130 con 0x7faefc15bac0 2024-04-17T10:43:35.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.297+0000 7faf02fca640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faefc06b760 con 0x7faefc15b6e0 2024-04-17T10:43:35.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.297+0000 7faf00d66640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faefc070470 0x7faefc154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:35.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.297+0000 7faf00d66640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faefc070470 0x7faefc154e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51220/0 (socket says 172.21.15.83:51220) 2024-04-17T10:43:35.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.297+0000 7faf00d66640 1 -- 172.21.15.83:0/2281720510 learned_addr learned my addr 172.21.15.83:0/2281720510 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:35.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.301+0000 7faf01567640 1 --2- 172.21.15.83:0/2281720510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faefc15bac0 0x7faefc152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:35.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.301+0000 7faefbfff640 1 --2- 172.21.15.83:0/2281720510 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faefc15b6e0 0x7faefc152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:35.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.301+0000 7faf00d66640 1 -- 172.21.15.83:0/2281720510 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faefc15b6e0 msgr2=0x7faefc152030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:35.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.301+0000 7faf00d66640 1 --2- 172.21.15.83:0/2281720510 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faefc15b6e0 0x7faefc152030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:35.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.301+0000 7faf00d66640 1 -- 172.21.15.83:0/2281720510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faefc15bac0 msgr2=0x7faefc152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:35.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.301+0000 7faf00d66640 1 --2- 172.21.15.83:0/2281720510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faefc15bac0 0x7faefc152570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:35.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.301+0000 7faf00d66640 1 -- 172.21.15.83:0/2281720510 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faefc153270 con 0x7faefc070470 2024-04-17T10:43:35.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.301+0000 7faf00d66640 1 --2- 172.21.15.83:0/2281720510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faefc070470 0x7faefc154e80 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7faeec0ae040 tx=0x7faeec01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:35.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.301+0000 7faef97fa640 1 -- 172.21.15.83:0/2281720510 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec0ba020 con 0x7faefc070470 2024-04-17T10:43:35.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.301+0000 7faef97fa640 1 -- 172.21.15.83:0/2281720510 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeec0aa6a0 con 0x7faefc070470 2024-04-17T10:43:35.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.301+0000 7faf02fca640 1 -- 172.21.15.83:0/2281720510 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faefc157e00 con 0x7faefc070470 2024-04-17T10:43:35.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.301+0000 7faef97fa640 1 -- 172.21.15.83:0/2281720510 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec0b0040 con 0x7faefc070470 2024-04-17T10:43:35.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.301+0000 7faf02fca640 1 -- 172.21.15.83:0/2281720510 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faefc1588c0 con 0x7faefc070470 2024-04-17T10:43:35.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.301+0000 7faf02fca640 1 -- 172.21.15.83:0/2281720510 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faefc074f90 con 0x7faefc070470 2024-04-17T10:43:35.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.301+0000 7faef97fa640 1 -- 172.21.15.83:0/2281720510 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faeec0bd020 con 0x7faefc070470 2024-04-17T10:43:35.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.301+0000 7faef97fa640 1 --2- 172.21.15.83:0/2281720510 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faee8066f60 0x7faee8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:35.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.301+0000 7faef97fa640 1 -- 172.21.15.83:0/2281720510 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faeec11f5a0 con 0x7faefc070470 2024-04-17T10:43:35.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.301+0000 7faefbfff640 1 --2- 172.21.15.83:0/2281720510 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faee8066f60 0x7faee8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:35.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.301+0000 7faefbfff640 1 --2- 172.21.15.83:0/2281720510 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faee8066f60 0x7faee8069420 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7faefc156520 tx=0x7faef0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:35.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.305+0000 7faef97fa640 1 -- 172.21.15.83:0/2281720510 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faeec0c4030 con 0x7faefc070470 2024-04-17T10:43:35.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:35.505+0000 7faf02fca640 1 -- 172.21.15.83:0/2281720510 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7faefc158b00 con 0x7faefc070470 2024-04-17T10:43:36.448 INFO:tasks.ceph.mds.d.smithi167.stderr:2024-04-17T10:43:36.449+0000 7f57ccf97640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T10:43:36.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.505+0000 7faef97fa640 1 -- 172.21.15.83:0/2281720510 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v47) v1 ==== 118+0+0 (secure 0 0 0) 0x7faeec016150 con 0x7faefc070470 2024-04-17T10:43:36.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.509+0000 7faf02fca640 1 -- 172.21.15.83:0/2281720510 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faee8066f60 msgr2=0x7faee8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:36.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.509+0000 7faf02fca640 1 --2- 172.21.15.83:0/2281720510 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faee8066f60 0x7faee8069420 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7faefc156520 tx=0x7faef0009040 comp rx=0 tx=0).stop 2024-04-17T10:43:36.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.509+0000 7faf02fca640 1 -- 172.21.15.83:0/2281720510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faefc070470 msgr2=0x7faefc154e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:36.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.509+0000 7faf02fca640 1 --2- 172.21.15.83:0/2281720510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faefc070470 0x7faefc154e80 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7faeec0ae040 tx=0x7faeec01ad10 comp rx=0 tx=0).stop 2024-04-17T10:43:36.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.509+0000 7faf02fca640 1 -- 172.21.15.83:0/2281720510 shutdown_connections 2024-04-17T10:43:36.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.509+0000 7faf02fca640 1 --2- 172.21.15.83:0/2281720510 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faee8066f60 0x7faee8069420 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:36.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.509+0000 7faf02fca640 1 --2- 172.21.15.83:0/2281720510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faefc15bac0 0x7faefc152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:36.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.509+0000 7faf02fca640 1 --2- 172.21.15.83:0/2281720510 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faefc15b6e0 0x7faefc152030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:36.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.509+0000 7faf02fca640 1 --2- 172.21.15.83:0/2281720510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faefc070470 0x7faefc154e80 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:36.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.509+0000 7faf02fca640 1 -- 172.21.15.83:0/2281720510 >> 172.21.15.83:0/2281720510 conn(0x7faefc0b15b0 msgr2=0x7faefc0b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:36.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.509+0000 7faf02fca640 1 -- 172.21.15.83:0/2281720510 shutdown_connections 2024-04-17T10:43:36.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.509+0000 7faf02fca640 1 -- 172.21.15.83:0/2281720510 wait complete. 2024-04-17T10:43:36.527 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-17T10:43:36.527 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-17T10:43:36.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.657+0000 7fe9748bf640 1 Processor -- start 2024-04-17T10:43:36.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.657+0000 7fe9748bf640 1 -- start start 2024-04-17T10:43:36.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.661+0000 7fe9748bf640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe97015b6e0 0x7fe970071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:36.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.661+0000 7fe9748bf640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe97006c1f0 con 0x7fe970070470 2024-04-17T10:43:36.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.661+0000 7fe9748bf640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe970070a10 con 0x7fe97015bac0 2024-04-17T10:43:36.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.661+0000 7fe9748bf640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe970070cf0 con 0x7fe97015b6e0 2024-04-17T10:43:36.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.661+0000 7fe96ed9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe970070470 0x7fe97006ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:43562/0 (socket says 172.21.15.83:43562) 2024-04-17T10:43:36.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.661+0000 7fe96ed9d640 1 -- 172.21.15.83:0/1945580248 learned_addr learned my addr 172.21.15.83:0/1945580248 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:43:36.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.661+0000 7fe96d59a640 1 -- 172.21.15.83:0/1945580248 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3011129885 0 0) 0x7fe97006c1f0 con 0x7fe970070470 2024-04-17T10:43:36.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.661+0000 7fe96d59a640 1 -- 172.21.15.83:0/1945580248 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe958003530 con 0x7fe970070470 2024-04-17T10:43:36.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.661+0000 7fe96e59c640 1 --2- 172.21.15.83:0/1945580248 >> v2:172.21.15.167:3300/0 conn(0x7fe97015b6e0 0x7fe970071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:36.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.661+0000 7fe96d59a640 1 -- 172.21.15.83:0/1945580248 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3689171409 0 0) 0x7fe958003530 con 0x7fe970070470 2024-04-17T10:43:36.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.661+0000 7fe96d59a640 1 -- 172.21.15.83:0/1945580248 >> v2:172.21.15.167:3300/0 conn(0x7fe97015b6e0 msgr2=0x7fe970071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:36.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.661+0000 7fe96d59a640 1 --2- 172.21.15.83:0/1945580248 >> v2:172.21.15.167:3300/0 conn(0x7fe97015b6e0 0x7fe970071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:36.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.661+0000 7fe96d59a640 1 -- 172.21.15.83:0/1945580248 >> v1:172.21.15.167:6789/0 conn(0x7fe97015bac0 legacy=0x7fe9700cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:36.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.661+0000 7fe96d59a640 1 -- 172.21.15.83:0/1945580248 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe970070fd0 con 0x7fe970070470 2024-04-17T10:43:36.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.661+0000 7fe96d59a640 1 -- 172.21.15.83:0/1945580248 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fe968001a20 con 0x7fe970070470 2024-04-17T10:43:36.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.661+0000 7fe96d59a640 1 -- 172.21.15.83:0/1945580248 >> v1:172.21.15.83:6789/0 conn(0x7fe970070470 legacy=0x7fe97006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:36.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.661+0000 7fe96d59a640 1 --2- 172.21.15.83:0/1945580248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe958003a30 0x7fe958003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:36.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.661+0000 7fe96d59a640 1 --2- 172.21.15.83:0/1945580248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe958004810 0x7fe958004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:36.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.661+0000 7fe96d59a640 1 --2- 172.21.15.83:0/1945580248 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9580051d0 0x7fe95800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:36.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.661+0000 7fe96d59a640 1 -- 172.21.15.83:0/1945580248 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe970070cf0 con 0x7fe958004810 2024-04-17T10:43:36.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.661+0000 7fe96d59a640 1 -- 172.21.15.83:0/1945580248 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95800c100 con 0x7fe958003a30 2024-04-17T10:43:36.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.661+0000 7fe96d59a640 1 -- 172.21.15.83:0/1945580248 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95800c300 con 0x7fe9580051d0 2024-04-17T10:43:36.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.661+0000 7fe96dd9b640 1 --2- 172.21.15.83:0/1945580248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe958004810 0x7fe958004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:36.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.661+0000 7fe96ed9d640 1 --2- 172.21.15.83:0/1945580248 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9580051d0 0x7fe95800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:36.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.661+0000 7fe96e59c640 1 --2- 172.21.15.83:0/1945580248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe958003a30 0x7fe958003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:36.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.661+0000 7fe96dd9b640 1 -- 172.21.15.83:0/1945580248 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9580051d0 msgr2=0x7fe95800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:36.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.661+0000 7fe96dd9b640 1 --2- 172.21.15.83:0/1945580248 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9580051d0 0x7fe95800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:36.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.661+0000 7fe96dd9b640 1 -- 172.21.15.83:0/1945580248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe958003a30 msgr2=0x7fe958003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:36.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.661+0000 7fe96dd9b640 1 --2- 172.21.15.83:0/1945580248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe958003a30 0x7fe958003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:36.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.661+0000 7fe96ed9d640 1 -- 172.21.15.83:0/1945580248 reap_dead start 2024-04-17T10:43:36.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.661+0000 7fe96dd9b640 1 -- 172.21.15.83:0/1945580248 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe958003530 con 0x7fe958004810 2024-04-17T10:43:36.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.661+0000 7fe96e59c640 1 --2- 172.21.15.83:0/1945580248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe958003a30 0x7fe958003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:36.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.665+0000 7fe96dd9b640 1 --2- 172.21.15.83:0/1945580248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe958004810 0x7fe958004c90 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fe964001780 tx=0x7fe96401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7868f93205de6235 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:36.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.665+0000 7fe96d59a640 1 -- 172.21.15.83:0/1945580248 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9640013a0 con 0x7fe958004810 2024-04-17T10:43:36.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.665+0000 7fe96d59a640 1 -- 172.21.15.83:0/1945580248 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9640a9070 con 0x7fe958004810 2024-04-17T10:43:36.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.665+0000 7fe96d59a640 1 -- 172.21.15.83:0/1945580248 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe964003600 con 0x7fe958004810 2024-04-17T10:43:36.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.665+0000 7fe9748bf640 1 -- 172.21.15.83:0/1945580248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe958004810 msgr2=0x7fe958004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:36.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.665+0000 7fe9748bf640 1 --2- 172.21.15.83:0/1945580248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe958004810 0x7fe958004c90 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fe964001780 tx=0x7fe96401c420 comp rx=0 tx=0).stop 2024-04-17T10:43:36.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.665+0000 7fe9748bf640 1 -- 172.21.15.83:0/1945580248 shutdown_connections 2024-04-17T10:43:36.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.665+0000 7fe9748bf640 1 --2- 172.21.15.83:0/1945580248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe958004810 0x7fe958004c90 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:36.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.665+0000 7fe9748bf640 1 -- 172.21.15.83:0/1945580248 >> 172.21.15.83:0/1945580248 conn(0x7fe9700b15b0 msgr2=0x7fe9700b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:36.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.665+0000 7fe9748bf640 1 -- 172.21.15.83:0/1945580248 shutdown_connections 2024-04-17T10:43:36.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.665+0000 7fe9748bf640 1 -- 172.21.15.83:0/1945580248 wait complete. 2024-04-17T10:43:36.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.665+0000 7fe9748bf640 1 Processor -- start 2024-04-17T10:43:36.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.665+0000 7fe9748bf640 1 -- start start 2024-04-17T10:43:36.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.665+0000 7fe9748bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe958004810 0x7fe97014e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:36.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.665+0000 7fe9748bf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe97013f870 0x7fe97015aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:36.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.665+0000 7fe9748bf640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe970152c40 0x7fe9700b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:36.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.665+0000 7fe9748bf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe970071e40 con 0x7fe958004810 2024-04-17T10:43:36.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.665+0000 7fe9748bf640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe970071cc0 con 0x7fe97013f870 2024-04-17T10:43:36.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.665+0000 7fe9748bf640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe970071fc0 con 0x7fe970152c40 2024-04-17T10:43:36.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.665+0000 7fe96e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe958004810 0x7fe97014e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:36.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.665+0000 7fe96e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe958004810 0x7fe97014e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51238/0 (socket says 172.21.15.83:51238) 2024-04-17T10:43:36.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.665+0000 7fe96e59c640 1 -- 172.21.15.83:0/3939932656 learned_addr learned my addr 172.21.15.83:0/3939932656 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:36.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.665+0000 7fe96e59c640 1 -- 172.21.15.83:0/3939932656 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe970152c40 msgr2=0x7fe9700b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:36.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.665+0000 7fe96e59c640 1 --2- 172.21.15.83:0/3939932656 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe970152c40 0x7fe9700b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:36.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.665+0000 7fe96e59c640 1 -- 172.21.15.83:0/3939932656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe97013f870 msgr2=0x7fe97015aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:36.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.665+0000 7fe96e59c640 1 --2- 172.21.15.83:0/3939932656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe97013f870 0x7fe97015aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:36.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.665+0000 7fe96e59c640 1 -- 172.21.15.83:0/3939932656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe970153830 con 0x7fe958004810 2024-04-17T10:43:36.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.665+0000 7fe96e59c640 1 --2- 172.21.15.83:0/3939932656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe958004810 0x7fe97014e100 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fe960010820 tx=0x7fe960010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:36.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.669+0000 7fe95effd640 1 -- 172.21.15.83:0/3939932656 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96000f040 con 0x7fe958004810 2024-04-17T10:43:36.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.669+0000 7fe95effd640 1 -- 172.21.15.83:0/3939932656 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe960004220 con 0x7fe958004810 2024-04-17T10:43:36.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.669+0000 7fe9748bf640 1 -- 172.21.15.83:0/3939932656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe970153550 con 0x7fe958004810 2024-04-17T10:43:36.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.669+0000 7fe95effd640 1 -- 172.21.15.83:0/3939932656 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96000e960 con 0x7fe958004810 2024-04-17T10:43:36.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.669+0000 7fe9748bf640 1 -- 172.21.15.83:0/3939932656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe97014ea60 con 0x7fe958004810 2024-04-17T10:43:36.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.669+0000 7fe9748bf640 1 -- 172.21.15.83:0/3939932656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe970074f90 con 0x7fe958004810 2024-04-17T10:43:36.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.669+0000 7fe95effd640 1 -- 172.21.15.83:0/3939932656 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe96000eb00 con 0x7fe958004810 2024-04-17T10:43:36.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.669+0000 7fe95effd640 1 --2- 172.21.15.83:0/3939932656 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe950066f60 0x7fe950069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:36.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.669+0000 7fe96dd9b640 1 --2- 172.21.15.83:0/3939932656 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe950066f60 0x7fe950069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:36.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.669+0000 7fe95effd640 1 -- 172.21.15.83:0/3939932656 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe960085190 con 0x7fe958004810 2024-04-17T10:43:36.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.669+0000 7fe96dd9b640 1 --2- 172.21.15.83:0/3939932656 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe950066f60 0x7fe950069420 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe964001750 tx=0x7fe964001130 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:36.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.673+0000 7fe95effd640 1 -- 172.21.15.83:0/3939932656 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe960018030 con 0x7fe958004810 2024-04-17T10:43:36.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:36.865+0000 7fe9748bf640 1 -- 172.21.15.83:0/3939932656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe9700aedf0 con 0x7fe958004810 2024-04-17T10:43:37.166 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:43:37.169 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:43:37.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.513+0000 7fe95effd640 1 -- 172.21.15.83:0/3939932656 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v48) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe960056230 con 0x7fe958004810 2024-04-17T10:43:37.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.517+0000 7fe9748bf640 1 -- 172.21.15.83:0/3939932656 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe950066f60 msgr2=0x7fe950069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:37.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.517+0000 7fe9748bf640 1 --2- 172.21.15.83:0/3939932656 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe950066f60 0x7fe950069420 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe964001750 tx=0x7fe964001130 comp rx=0 tx=0).stop 2024-04-17T10:43:37.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.517+0000 7fe9748bf640 1 -- 172.21.15.83:0/3939932656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe958004810 msgr2=0x7fe97014e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:37.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.517+0000 7fe9748bf640 1 --2- 172.21.15.83:0/3939932656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe958004810 0x7fe97014e100 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fe960010820 tx=0x7fe960010ce0 comp rx=0 tx=0).stop 2024-04-17T10:43:37.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.517+0000 7fe9748bf640 1 -- 172.21.15.83:0/3939932656 shutdown_connections 2024-04-17T10:43:37.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.517+0000 7fe9748bf640 1 --2- 172.21.15.83:0/3939932656 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe950066f60 0x7fe950069420 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:37.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.517+0000 7fe9748bf640 1 --2- 172.21.15.83:0/3939932656 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe970152c40 0x7fe9700b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:37.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.517+0000 7fe9748bf640 1 --2- 172.21.15.83:0/3939932656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe97013f870 0x7fe97015aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:37.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.517+0000 7fe9748bf640 1 --2- 172.21.15.83:0/3939932656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe958004810 0x7fe97014e100 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:37.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.517+0000 7fe9748bf640 1 -- 172.21.15.83:0/3939932656 >> 172.21.15.83:0/3939932656 conn(0x7fe9700b15b0 msgr2=0x7fe9700af9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:37.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.517+0000 7fe9748bf640 1 -- 172.21.15.83:0/3939932656 shutdown_connections 2024-04-17T10:43:37.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.521+0000 7fe9748bf640 1 -- 172.21.15.83:0/3939932656 wait complete. 2024-04-17T10:43:37.537 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-17T10:43:37.537 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:43:37.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.665+0000 7fb4f5cf8640 1 Processor -- start 2024-04-17T10:43:37.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.665+0000 7fb4f5cf8640 1 -- start start 2024-04-17T10:43:37.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.665+0000 7fb4f5cf8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4f015dfa0 0x7fb4f00b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:37.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.665+0000 7fb4f5cf8640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb4f00b5b40 0x7fb4f00d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:37.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.665+0000 7fb4f5cf8640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4f0162a00 con 0x7fb4f00ce980 2024-04-17T10:43:37.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.665+0000 7fb4f5cf8640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4f0161e80 con 0x7fb4f00b5b40 2024-04-17T10:43:37.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.665+0000 7fb4f5cf8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f0162160 con 0x7fb4f015dfa0 2024-04-17T10:43:37.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.665+0000 7fb4ef7fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4f015dfa0 0x7fb4f00b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:37.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.665+0000 7fb4effff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb4f00b5b40 0x7fb4f00d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:37.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.665+0000 7fb4effff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb4f00b5b40 0x7fb4f00d1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:54020/0 (socket says 172.21.15.83:54020) 2024-04-17T10:43:37.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.665+0000 7fb4ef7fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4f015dfa0 0x7fb4f00b6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:36286/0 (socket says 172.21.15.83:36286) 2024-04-17T10:43:37.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.665+0000 7fb4effff640 1 -- 172.21.15.83:0/2341964423 learned_addr learned my addr 172.21.15.83:0/2341964423 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:37.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.665+0000 7fb4effff640 1 -- 172.21.15.83:0/2341964423 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4f015dfa0 msgr2=0x7fb4f00b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:37.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.665+0000 7fb4effff640 1 --2- 172.21.15.83:0/2341964423 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4f015dfa0 0x7fb4f00b6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:37.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.665+0000 7fb4effff640 1 -- 172.21.15.83:0/2341964423 >> v1:172.21.15.167:6789/0 conn(0x7fb4f00ce980 legacy=0x7fb4f00b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:37.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.665+0000 7fb4effff640 1 -- 172.21.15.83:0/2341964423 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f0162440 con 0x7fb4f00b5b40 2024-04-17T10:43:37.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.669+0000 7fb4effff640 1 --2- 172.21.15.83:0/2341964423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4f00b5b40 0x7fb4f00d1d30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8009e50 tx=0x7fb4e801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c59317466170e762 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:37.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.669+0000 7fb4ee7fc640 1 -- 172.21.15.83:0/2341964423 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e80a56b0 con 0x7fb4f00b5b40 2024-04-17T10:43:37.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.669+0000 7fb4ee7fc640 1 -- 172.21.15.83:0/2341964423 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e80a5850 con 0x7fb4f00b5b40 2024-04-17T10:43:37.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.669+0000 7fb4ee7fc640 1 -- 172.21.15.83:0/2341964423 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e80a5c20 con 0x7fb4f00b5b40 2024-04-17T10:43:37.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.669+0000 7fb4f5cf8640 1 -- 172.21.15.83:0/2341964423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4f00b5b40 msgr2=0x7fb4f00d1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:37.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.669+0000 7fb4f5cf8640 1 --2- 172.21.15.83:0/2341964423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4f00b5b40 0x7fb4f00d1d30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8009e50 tx=0x7fb4e801c970 comp rx=0 tx=0).stop 2024-04-17T10:43:37.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.669+0000 7fb4f5cf8640 1 -- 172.21.15.83:0/2341964423 shutdown_connections 2024-04-17T10:43:37.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.669+0000 7fb4f5cf8640 1 --2- 172.21.15.83:0/2341964423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4f00b5b40 0x7fb4f00d1d30 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:37.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.669+0000 7fb4f5cf8640 1 --2- 172.21.15.83:0/2341964423 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4f015dfa0 0x7fb4f00b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:37.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.669+0000 7fb4f5cf8640 1 -- 172.21.15.83:0/2341964423 >> 172.21.15.83:0/2341964423 conn(0x7fb4f00d0600 msgr2=0x7fb4f00d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:37.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.669+0000 7fb4f5cf8640 1 -- 172.21.15.83:0/2341964423 shutdown_connections 2024-04-17T10:43:37.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.669+0000 7fb4f5cf8640 1 -- 172.21.15.83:0/2341964423 wait complete. 2024-04-17T10:43:37.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.669+0000 7fb4f5cf8640 1 Processor -- start 2024-04-17T10:43:37.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.669+0000 7fb4f5cf8640 1 -- start start 2024-04-17T10:43:37.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.673+0000 7fb4f5cf8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4f00b5b40 0x7fb4f01520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:37.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.673+0000 7fb4f5cf8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4f00ce980 0x7fb4f0150430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:37.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.673+0000 7fb4f5cf8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4f015dfa0 0x7fb4f0150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:37.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.673+0000 7fb4f5cf8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f016ae50 con 0x7fb4f00b5b40 2024-04-17T10:43:37.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.673+0000 7fb4f5cf8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f0161b10 con 0x7fb4f00ce980 2024-04-17T10:43:37.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.673+0000 7fb4f5cf8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f016afd0 con 0x7fb4f015dfa0 2024-04-17T10:43:37.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.673+0000 7fb4ef7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4f00b5b40 0x7fb4f01520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:37.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.673+0000 7fb4ef7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4f00b5b40 0x7fb4f01520e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52300/0 (socket says 172.21.15.83:52300) 2024-04-17T10:43:37.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.673+0000 7fb4ef7fe640 1 -- 172.21.15.83:0/1834309374 learned_addr learned my addr 172.21.15.83:0/1834309374 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:37.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.673+0000 7fb4eeffd640 1 --2- 172.21.15.83:0/1834309374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4f00ce980 0x7fb4f0150430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:37.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.673+0000 7fb4ef7fe640 1 -- 172.21.15.83:0/1834309374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4f015dfa0 msgr2=0x7fb4f0150bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:37.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.673+0000 7fb4effff640 1 --2- 172.21.15.83:0/1834309374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4f015dfa0 0x7fb4f0150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:37.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.673+0000 7fb4ef7fe640 1 --2- 172.21.15.83:0/1834309374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4f015dfa0 0x7fb4f0150bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:37.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.673+0000 7fb4ef7fe640 1 -- 172.21.15.83:0/1834309374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4f00ce980 msgr2=0x7fb4f0150430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:37.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.673+0000 7fb4ef7fe640 1 --2- 172.21.15.83:0/1834309374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4f00ce980 0x7fb4f0150430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:37.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.673+0000 7fb4ef7fe640 1 -- 172.21.15.83:0/1834309374 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f0151400 con 0x7fb4f00b5b40 2024-04-17T10:43:37.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.673+0000 7fb4ef7fe640 1 --2- 172.21.15.83:0/1834309374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4f00b5b40 0x7fb4f01520e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fb4e000e930 tx=0x7fb4e000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:37.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.673+0000 7fb4d7fff640 1 -- 172.21.15.83:0/1834309374 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0017070 con 0x7fb4f00b5b40 2024-04-17T10:43:37.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.673+0000 7fb4d7fff640 1 -- 172.21.15.83:0/1834309374 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e000ca00 con 0x7fb4f00b5b40 2024-04-17T10:43:37.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.673+0000 7fb4f5cf8640 1 -- 172.21.15.83:0/1834309374 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f01516e0 con 0x7fb4f00b5b40 2024-04-17T10:43:37.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.673+0000 7fb4d7fff640 1 -- 172.21.15.83:0/1834309374 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e000cdb0 con 0x7fb4f00b5b40 2024-04-17T10:43:37.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.673+0000 7fb4f5cf8640 1 -- 172.21.15.83:0/1834309374 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f0151ca0 con 0x7fb4f00b5b40 2024-04-17T10:43:37.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.673+0000 7fb4f5cf8640 1 -- 172.21.15.83:0/1834309374 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4f0074f90 con 0x7fb4f00b5b40 2024-04-17T10:43:37.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.673+0000 7fb4d7fff640 1 -- 172.21.15.83:0/1834309374 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb4e001a070 con 0x7fb4f00b5b40 2024-04-17T10:43:37.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.673+0000 7fb4d7fff640 1 --2- 172.21.15.83:0/1834309374 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb4dc066f60 0x7fb4dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:37.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.673+0000 7fb4d7fff640 1 -- 172.21.15.83:0/1834309374 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb4e0085430 con 0x7fb4f00b5b40 2024-04-17T10:43:37.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.673+0000 7fb4eeffd640 1 --2- 172.21.15.83:0/1834309374 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb4dc066f60 0x7fb4dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:37.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.673+0000 7fb4eeffd640 1 --2- 172.21.15.83:0/1834309374 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb4dc066f60 0x7fb4dc069420 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4007fd0 tx=0x7fb4e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:37.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.677+0000 7fb4d7fff640 1 -- 172.21.15.83:0/1834309374 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb4e0056500 con 0x7fb4f00b5b40 2024-04-17T10:43:37.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.865+0000 7fb4f5cf8640 1 -- 172.21.15.83:0/1834309374 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4f00d17a0 con 0x7fb4f00b5b40 2024-04-17T10:43:37.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.865+0000 7fb4d7fff640 1 -- 172.21.15.83:0/1834309374 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fb4e0055e00 con 0x7fb4f00b5b40 2024-04-17T10:43:37.862 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:43:37.862 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":48,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5258,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2259892875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2259892875},{"type":"v1","addr":"172.21.15.167:6835","nonce":2259892875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":7095,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/3769102527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3769102527},{"type":"v1","addr":"172.21.15.83:6836","nonce":3769102527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":7098,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/395594830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":395594830},{"type":"v1","addr":"172.21.15.83:6837","nonce":395594830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:43:33.830720+0000","modified":"2024-04-17T10:43:36.867500+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7104},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7104":{"gid":7104,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6833/2796648400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2796648400},{"type":"v1","addr":"172.21.15.167:6833","nonce":2796648400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-17T10:43:37.862 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 48 2024-04-17T10:43:37.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.869+0000 7fb4f5cf8640 1 -- 172.21.15.83:0/1834309374 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb4dc066f60 msgr2=0x7fb4dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:37.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.869+0000 7fb4f5cf8640 1 --2- 172.21.15.83:0/1834309374 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb4dc066f60 0x7fb4dc069420 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4007fd0 tx=0x7fb4e4009040 comp rx=0 tx=0).stop 2024-04-17T10:43:37.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.869+0000 7fb4f5cf8640 1 -- 172.21.15.83:0/1834309374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4f00b5b40 msgr2=0x7fb4f01520e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:37.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.869+0000 7fb4f5cf8640 1 --2- 172.21.15.83:0/1834309374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4f00b5b40 0x7fb4f01520e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fb4e000e930 tx=0x7fb4e000edf0 comp rx=0 tx=0).stop 2024-04-17T10:43:37.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.869+0000 7fb4f5cf8640 1 -- 172.21.15.83:0/1834309374 shutdown_connections 2024-04-17T10:43:37.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.869+0000 7fb4f5cf8640 1 --2- 172.21.15.83:0/1834309374 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb4dc066f60 0x7fb4dc069420 secure :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4007fd0 tx=0x7fb4e4009040 comp rx=0 tx=0).stop 2024-04-17T10:43:37.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.869+0000 7fb4f5cf8640 1 --2- 172.21.15.83:0/1834309374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4f015dfa0 0x7fb4f0150bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:37.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.869+0000 7fb4f5cf8640 1 --2- 172.21.15.83:0/1834309374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4f00ce980 0x7fb4f0150430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:37.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.869+0000 7fb4f5cf8640 1 --2- 172.21.15.83:0/1834309374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4f00b5b40 0x7fb4f01520e0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:37.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.869+0000 7fb4f5cf8640 1 -- 172.21.15.83:0/1834309374 >> 172.21.15.83:0/1834309374 conn(0x7fb4f00d0600 msgr2=0x7fb4f015cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:37.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.869+0000 7fb4f5cf8640 1 -- 172.21.15.83:0/1834309374 shutdown_connections 2024-04-17T10:43:37.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:37.869+0000 7fb4f5cf8640 1 -- 172.21.15.83:0/1834309374 wait complete. 2024-04-17T10:43:37.880 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:43:38.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.005+0000 7fe8433da640 1 Processor -- start 2024-04-17T10:43:38.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.005+0000 7fe8433da640 1 -- start start 2024-04-17T10:43:38.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.005+0000 7fe8433da640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe83c0b9820 0x7fe83c15eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:38.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.005+0000 7fe8433da640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe83c161670 0x7fe83c0d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:38.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.005+0000 7fe8433da640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe83c15f040 con 0x7fe83c0d0df0 2024-04-17T10:43:38.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.005+0000 7fe8433da640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe83c0cee30 con 0x7fe83c0b9820 2024-04-17T10:43:38.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.005+0000 7fe8433da640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c161b30 con 0x7fe83c161670 2024-04-17T10:43:38.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.005+0000 7fe841176640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe83c0b9820 0x7fe83c15eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:38.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.005+0000 7fe840975640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe83c161670 0x7fe83c0d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:38.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.005+0000 7fe841977640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fe83c0d0df0 0x7fe83c0b5570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:35974/0 (socket says 172.21.15.83:35974) 2024-04-17T10:43:38.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.005+0000 7fe841977640 1 -- 172.21.15.83:0/2193062483 learned_addr learned my addr 172.21.15.83:0/2193062483 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:43:38.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.005+0000 7fe840975640 1 -- 172.21.15.83:0/2193062483 >> v2:172.21.15.167:3300/0 conn(0x7fe83c0b9820 msgr2=0x7fe83c15eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:38.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.005+0000 7fe840975640 1 --2- 172.21.15.83:0/2193062483 >> v2:172.21.15.167:3300/0 conn(0x7fe83c0b9820 0x7fe83c15eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.005+0000 7fe840975640 1 -- 172.21.15.83:0/2193062483 >> v1:172.21.15.167:6789/0 conn(0x7fe83c0d0df0 legacy=0x7fe83c0b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:38.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.005+0000 7fe840975640 1 -- 172.21.15.83:0/2193062483 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83c161e10 con 0x7fe83c161670 2024-04-17T10:43:38.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.009+0000 7fe840975640 1 --2- 172.21.15.83:0/2193062483 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe83c161670 0x7fe83c0d08b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0097c0 tx=0x7fe82c01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef0d92a70bfba09e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:38.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.009+0000 7fe833fff640 1 -- 172.21.15.83:0/2193062483 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82c0b0070 con 0x7fe83c161670 2024-04-17T10:43:38.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.009+0000 7fe833fff640 1 -- 172.21.15.83:0/2193062483 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82c0ab600 con 0x7fe83c161670 2024-04-17T10:43:38.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.009+0000 7fe833fff640 1 -- 172.21.15.83:0/2193062483 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82c0ab9d0 con 0x7fe83c161670 2024-04-17T10:43:38.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.009+0000 7fe8433da640 1 -- 172.21.15.83:0/2193062483 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe83c161670 msgr2=0x7fe83c0d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:38.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.009+0000 7fe8433da640 1 --2- 172.21.15.83:0/2193062483 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe83c161670 0x7fe83c0d08b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0097c0 tx=0x7fe82c01c410 comp rx=0 tx=0).stop 2024-04-17T10:43:38.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.009+0000 7fe8433da640 1 -- 172.21.15.83:0/2193062483 shutdown_connections 2024-04-17T10:43:38.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.009+0000 7fe8433da640 1 --2- 172.21.15.83:0/2193062483 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe83c161670 0x7fe83c0d08b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.009+0000 7fe8433da640 1 --2- 172.21.15.83:0/2193062483 >> v2:172.21.15.167:3300/0 conn(0x7fe83c0b9820 0x7fe83c15eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.009+0000 7fe8433da640 1 -- 172.21.15.83:0/2193062483 >> 172.21.15.83:0/2193062483 conn(0x7fe83c06b280 msgr2=0x7fe83c06b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:38.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.009+0000 7fe8433da640 1 -- 172.21.15.83:0/2193062483 shutdown_connections 2024-04-17T10:43:38.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.009+0000 7fe8433da640 1 -- 172.21.15.83:0/2193062483 wait complete. 2024-04-17T10:43:38.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.009+0000 7fe8433da640 1 Processor -- start 2024-04-17T10:43:38.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.009+0000 7fe8433da640 1 -- start start 2024-04-17T10:43:38.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.013+0000 7fe8433da640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe83c0b9820 0x7fe83c159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:38.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.013+0000 7fe8433da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe83c0d0df0 0x7fe83c0cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:38.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.013+0000 7fe8433da640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe83c161670 0x7fe83c1603a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:38.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.013+0000 7fe8433da640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c16ada0 con 0x7fe83c0d0df0 2024-04-17T10:43:38.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.013+0000 7fe8433da640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c16ac20 con 0x7fe83c161670 2024-04-17T10:43:38.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.013+0000 7fe8433da640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c16af20 con 0x7fe83c0b9820 2024-04-17T10:43:38.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.013+0000 7fe840975640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe83c0d0df0 0x7fe83c0cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:38.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.013+0000 7fe840975640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe83c0d0df0 0x7fe83c0cd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52306/0 (socket says 172.21.15.83:52306) 2024-04-17T10:43:38.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.013+0000 7fe840975640 1 -- 172.21.15.83:0/3708650038 learned_addr learned my addr 172.21.15.83:0/3708650038 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:38.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.013+0000 7fe841977640 1 --2- 172.21.15.83:0/3708650038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe83c161670 0x7fe83c1603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:38.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.013+0000 7fe841176640 1 --2- 172.21.15.83:0/3708650038 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe83c0b9820 0x7fe83c159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:38.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.013+0000 7fe840975640 1 -- 172.21.15.83:0/3708650038 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe83c0b9820 msgr2=0x7fe83c159590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:38.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.013+0000 7fe840975640 1 --2- 172.21.15.83:0/3708650038 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe83c0b9820 0x7fe83c159590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.013+0000 7fe840975640 1 -- 172.21.15.83:0/3708650038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe83c161670 msgr2=0x7fe83c1603a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:38.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.013+0000 7fe840975640 1 --2- 172.21.15.83:0/3708650038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe83c161670 0x7fe83c1603a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.013+0000 7fe840975640 1 -- 172.21.15.83:0/3708650038 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83c0cda50 con 0x7fe83c0d0df0 2024-04-17T10:43:38.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.013+0000 7fe840975640 1 --2- 172.21.15.83:0/3708650038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe83c0d0df0 0x7fe83c0cd000 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0097c0 tx=0x7fe82c0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:38.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.013+0000 7fe831ffb640 1 -- 172.21.15.83:0/3708650038 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82c0b0040 con 0x7fe83c0d0df0 2024-04-17T10:43:38.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.013+0000 7fe831ffb640 1 -- 172.21.15.83:0/3708650038 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82c0b9070 con 0x7fe83c0d0df0 2024-04-17T10:43:38.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.013+0000 7fe8433da640 1 -- 172.21.15.83:0/3708650038 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe83c162a10 con 0x7fe83c0d0df0 2024-04-17T10:43:38.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.013+0000 7fe831ffb640 1 -- 172.21.15.83:0/3708650038 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82c006e20 con 0x7fe83c0d0df0 2024-04-17T10:43:38.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.013+0000 7fe8433da640 1 -- 172.21.15.83:0/3708650038 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe83c162fd0 con 0x7fe83c0d0df0 2024-04-17T10:43:38.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.013+0000 7fe8433da640 1 -- 172.21.15.83:0/3708650038 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe83c074f90 con 0x7fe83c0d0df0 2024-04-17T10:43:38.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.013+0000 7fe831ffb640 1 -- 172.21.15.83:0/3708650038 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe82c0b4630 con 0x7fe83c0d0df0 2024-04-17T10:43:38.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.013+0000 7fe831ffb640 1 --2- 172.21.15.83:0/3708650038 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe820066f60 0x7fe820069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:38.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.013+0000 7fe831ffb640 1 -- 172.21.15.83:0/3708650038 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe82c11fa10 con 0x7fe83c0d0df0 2024-04-17T10:43:38.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.013+0000 7fe841176640 1 --2- 172.21.15.83:0/3708650038 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe820066f60 0x7fe820069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:38.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.013+0000 7fe841176640 1 --2- 172.21.15.83:0/3708650038 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe820066f60 0x7fe820069420 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe838009a50 tx=0x7fe838008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:38.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.017+0000 7fe831ffb640 1 -- 172.21.15.83:0/3708650038 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe82c016090 con 0x7fe83c0d0df0 2024-04-17T10:43:38.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.165+0000 7fe8433da640 1 -- 172.21.15.83:0/3708650038 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe83c0b66e0 con 0x7fe83c0d0df0 2024-04-17T10:43:38.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.169+0000 7fe831ffb640 1 -- 172.21.15.83:0/3708650038 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16596 (secure 0 0 0) 0x7fe82c0f0b40 con 0x7fe83c0d0df0 2024-04-17T10:43:38.164 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:43:38.165 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":74,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:43:35.498518+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:43:32.090869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-17T10:43:32.815921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:43:38.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.169+0000 7fe8433da640 1 -- 172.21.15.83:0/3708650038 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe820066f60 msgr2=0x7fe820069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:38.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.169+0000 7fe8433da640 1 --2- 172.21.15.83:0/3708650038 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe820066f60 0x7fe820069420 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe838009a50 tx=0x7fe838008040 comp rx=0 tx=0).stop 2024-04-17T10:43:38.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.169+0000 7fe8433da640 1 -- 172.21.15.83:0/3708650038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe83c0d0df0 msgr2=0x7fe83c0cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:38.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.169+0000 7fe8433da640 1 --2- 172.21.15.83:0/3708650038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe83c0d0df0 0x7fe83c0cd000 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0097c0 tx=0x7fe82c0191b0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.169+0000 7fe8433da640 1 -- 172.21.15.83:0/3708650038 shutdown_connections 2024-04-17T10:43:38.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.169+0000 7fe8433da640 1 --2- 172.21.15.83:0/3708650038 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe820066f60 0x7fe820069420 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.169+0000 7fe8433da640 1 --2- 172.21.15.83:0/3708650038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe83c161670 0x7fe83c1603a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.169+0000 7fe8433da640 1 --2- 172.21.15.83:0/3708650038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe83c0d0df0 0x7fe83c0cd000 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.169+0000 7fe8433da640 1 --2- 172.21.15.83:0/3708650038 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe83c0b9820 0x7fe83c159590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.169+0000 7fe8433da640 1 -- 172.21.15.83:0/3708650038 >> 172.21.15.83:0/3708650038 conn(0x7fe83c06b280 msgr2=0x7fe83c06b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:38.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.169+0000 7fe8433da640 1 -- 172.21.15.83:0/3708650038 shutdown_connections 2024-04-17T10:43:38.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.169+0000 7fe8433da640 1 -- 172.21.15.83:0/3708650038 wait complete. 2024-04-17T10:43:38.182 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-17T10:43:38.182 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:43:38.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.297+0000 7f1d014c2640 1 Processor -- start 2024-04-17T10:43:38.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.297+0000 7f1d014c2640 1 -- start start 2024-04-17T10:43:38.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.297+0000 7f1d014c2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1cfc15dfa0 0x7f1cfc0b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:38.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.297+0000 7f1d014c2640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1cfc0b5b40 0x7f1cfc0d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:38.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.297+0000 7f1d014c2640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cfc162a00 con 0x7f1cfc0ce980 2024-04-17T10:43:38.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.297+0000 7f1d014c2640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cfc161e80 con 0x7f1cfc15dfa0 2024-04-17T10:43:38.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.297+0000 7f1d014c2640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cfc162160 con 0x7f1cfc0b5b40 2024-04-17T10:43:38.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.297+0000 7f1cfaffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1cfc15dfa0 0x7f1cfc0b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:38.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.297+0000 7f1cf27fc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1cfc0ce980 0x7f1cfc0b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55078/0 (socket says 172.21.15.83:55078) 2024-04-17T10:43:38.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.297+0000 7f1cf27fc640 1 -- 172.21.15.83:0/3451016190 learned_addr learned my addr 172.21.15.83:0/3451016190 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:43:38.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.297+0000 7f1cfb7fe640 1 --2- 172.21.15.83:0/3451016190 >> v2:172.21.15.167:3300/0 conn(0x7f1cfc0b5b40 0x7f1cfc0d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:38.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.297+0000 7f1cfaffd640 1 -- 172.21.15.83:0/3451016190 >> v2:172.21.15.167:3300/0 conn(0x7f1cfc0b5b40 msgr2=0x7f1cfc0d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:38.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.297+0000 7f1cfaffd640 1 --2- 172.21.15.83:0/3451016190 >> v2:172.21.15.167:3300/0 conn(0x7f1cfc0b5b40 0x7f1cfc0d1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.297+0000 7f1cfaffd640 1 -- 172.21.15.83:0/3451016190 >> v1:172.21.15.83:6789/0 conn(0x7f1cfc0ce980 legacy=0x7f1cfc0b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:38.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.297+0000 7f1cfa7fc640 1 -- 172.21.15.83:0/3451016190 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2468561875 0 0) 0x7f1cfc162a00 con 0x7f1cfc0ce980 2024-04-17T10:43:38.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.297+0000 7f1cfaffd640 1 -- 172.21.15.83:0/3451016190 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cfc162440 con 0x7f1cfc15dfa0 2024-04-17T10:43:38.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.297+0000 7f1cfaffd640 1 --2- 172.21.15.83:0/3451016190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cfc15dfa0 0x7f1cfc0b6c20 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc0ca980 tx=0x7f1cec01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a89a1f979afe58 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:38.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1cfa7fc640 1 -- 172.21.15.83:0/3451016190 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0b0070 con 0x7f1cfc15dfa0 2024-04-17T10:43:38.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1cfa7fc640 1 -- 172.21.15.83:0/3451016190 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cec0ab600 con 0x7f1cfc15dfa0 2024-04-17T10:43:38.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1cfa7fc640 1 -- 172.21.15.83:0/3451016190 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0ab9d0 con 0x7f1cfc15dfa0 2024-04-17T10:43:38.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1d014c2640 1 -- 172.21.15.83:0/3451016190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cfc15dfa0 msgr2=0x7f1cfc0b6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:38.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1d014c2640 1 --2- 172.21.15.83:0/3451016190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cfc15dfa0 0x7f1cfc0b6c20 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc0ca980 tx=0x7f1cec01c3c0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1d014c2640 1 -- 172.21.15.83:0/3451016190 shutdown_connections 2024-04-17T10:43:38.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1d014c2640 1 --2- 172.21.15.83:0/3451016190 >> v2:172.21.15.167:3300/0 conn(0x7f1cfc0b5b40 0x7f1cfc0d1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1d014c2640 1 --2- 172.21.15.83:0/3451016190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cfc15dfa0 0x7f1cfc0b6c20 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1d014c2640 1 -- 172.21.15.83:0/3451016190 >> 172.21.15.83:0/3451016190 conn(0x7f1cfc0d0600 msgr2=0x7f1cfc0d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:38.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1d014c2640 1 -- 172.21.15.83:0/3451016190 shutdown_connections 2024-04-17T10:43:38.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1d014c2640 1 -- 172.21.15.83:0/3451016190 wait complete. 2024-04-17T10:43:38.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1d014c2640 1 Processor -- start 2024-04-17T10:43:38.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1d014c2640 1 -- start start 2024-04-17T10:43:38.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1d014c2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cfc0b5b40 0x7f1cfc159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:38.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1d014c2640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1cfc0ce980 0x7f1cfc15fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:38.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1d014c2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cfc15dfa0 0x7f1cfc160190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:38.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1d014c2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc16ae50 con 0x7f1cfc15dfa0 2024-04-17T10:43:38.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1d014c2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc161b10 con 0x7f1cfc0b5b40 2024-04-17T10:43:38.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1d014c2640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc16afd0 con 0x7f1cfc0ce980 2024-04-17T10:43:38.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1cfb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cfc15dfa0 0x7f1cfc160190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:38.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1cfb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cfc15dfa0 0x7f1cfc160190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52326/0 (socket says 172.21.15.83:52326) 2024-04-17T10:43:38.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1cfb7fe640 1 -- 172.21.15.83:0/1741923142 learned_addr learned my addr 172.21.15.83:0/1741923142 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:38.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1cfaffd640 1 --2- 172.21.15.83:0/1741923142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cfc0b5b40 0x7f1cfc159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:38.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1cf27fc640 1 --2- 172.21.15.83:0/1741923142 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1cfc0ce980 0x7f1cfc15fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:38.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1cfb7fe640 1 -- 172.21.15.83:0/1741923142 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1cfc0ce980 msgr2=0x7f1cfc15fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:38.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1cfb7fe640 1 --2- 172.21.15.83:0/1741923142 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1cfc0ce980 0x7f1cfc15fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1cfb7fe640 1 -- 172.21.15.83:0/1741923142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cfc0b5b40 msgr2=0x7f1cfc159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:38.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1cfb7fe640 1 --2- 172.21.15.83:0/1741923142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cfc0b5b40 0x7f1cfc159a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1cfb7fe640 1 -- 172.21.15.83:0/1741923142 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cfc1546d0 con 0x7f1cfc15dfa0 2024-04-17T10:43:38.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1cfaffd640 1 --2- 172.21.15.83:0/1741923142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cfc0b5b40 0x7f1cfc159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:38.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1cfb7fe640 1 --2- 172.21.15.83:0/1741923142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cfc15dfa0 0x7f1cfc160190 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4007d70 tx=0x7f1cf400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:38.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1cf3fff640 1 -- 172.21.15.83:0/1741923142 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4014420 con 0x7f1cfc15dfa0 2024-04-17T10:43:38.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1cf3fff640 1 -- 172.21.15.83:0/1741923142 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf4019070 con 0x7f1cfc15dfa0 2024-04-17T10:43:38.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1cf3fff640 1 -- 172.21.15.83:0/1741923142 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4014820 con 0x7f1cfc15dfa0 2024-04-17T10:43:38.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1d014c2640 1 -- 172.21.15.83:0/1741923142 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cfc154110 con 0x7f1cfc15dfa0 2024-04-17T10:43:38.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.301+0000 7f1d014c2640 1 -- 172.21.15.83:0/1741923142 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cfc1f8320 con 0x7f1cfc15dfa0 2024-04-17T10:43:38.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.305+0000 7f1d014c2640 1 -- 172.21.15.83:0/1741923142 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cfc157b30 con 0x7f1cfc15dfa0 2024-04-17T10:43:38.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.305+0000 7f1cf3fff640 1 -- 172.21.15.83:0/1741923142 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1cf40149c0 con 0x7f1cfc15dfa0 2024-04-17T10:43:38.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.305+0000 7f1cf3fff640 1 --2- 172.21.15.83:0/1741923142 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1cdc066f60 0x7f1cdc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:38.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.305+0000 7f1cf3fff640 1 -- 172.21.15.83:0/1741923142 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1cf4085760 con 0x7f1cfc15dfa0 2024-04-17T10:43:38.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.305+0000 7f1cfaffd640 1 --2- 172.21.15.83:0/1741923142 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1cdc066f60 0x7f1cdc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:38.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.305+0000 7f1cfaffd640 1 --2- 172.21.15.83:0/1741923142 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1cdc066f60 0x7f1cdc069420 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1cec0ad000 tx=0x7f1cec0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:38.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.309+0000 7f1cf3fff640 1 -- 172.21.15.83:0/1741923142 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1cf4011260 con 0x7f1cfc15dfa0 2024-04-17T10:43:38.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.453+0000 7f1d014c2640 1 -- 172.21.15.83:0/1741923142 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1cfc074f90 con 0x7f1cdc066f60 2024-04-17T10:43:38.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.457+0000 7f1cf3fff640 1 -- 172.21.15.83:0/1741923142 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246757 (secure 0 0 0) 0x7f1cfc074f90 con 0x7f1cdc066f60 2024-04-17T10:43:38.457 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:43:38.459 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:43:38.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.461+0000 7f1d014c2640 1 -- 172.21.15.83:0/1741923142 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1cdc066f60 msgr2=0x7f1cdc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:38.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.461+0000 7f1d014c2640 1 --2- 172.21.15.83:0/1741923142 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1cdc066f60 0x7f1cdc069420 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1cec0ad000 tx=0x7f1cec0a6000 comp rx=0 tx=0).stop 2024-04-17T10:43:38.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.461+0000 7f1d014c2640 1 -- 172.21.15.83:0/1741923142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cfc15dfa0 msgr2=0x7f1cfc160190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:38.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.461+0000 7f1d014c2640 1 --2- 172.21.15.83:0/1741923142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cfc15dfa0 0x7f1cfc160190 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4007d70 tx=0x7f1cf400e4b0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.461+0000 7f1d014c2640 1 -- 172.21.15.83:0/1741923142 shutdown_connections 2024-04-17T10:43:38.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.461+0000 7f1d014c2640 1 --2- 172.21.15.83:0/1741923142 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1cdc066f60 0x7f1cdc069420 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.461+0000 7f1d014c2640 1 --2- 172.21.15.83:0/1741923142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cfc15dfa0 0x7f1cfc160190 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.461+0000 7f1d014c2640 1 --2- 172.21.15.83:0/1741923142 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1cfc0ce980 0x7f1cfc15fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.461+0000 7f1d014c2640 1 --2- 172.21.15.83:0/1741923142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cfc0b5b40 0x7f1cfc159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.461+0000 7f1d014c2640 1 -- 172.21.15.83:0/1741923142 >> 172.21.15.83:0/1741923142 conn(0x7f1cfc0d0600 msgr2=0x7f1cfc160ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:38.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.461+0000 7f1d014c2640 1 -- 172.21.15.83:0/1741923142 shutdown_connections 2024-04-17T10:43:38.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.461+0000 7f1d014c2640 1 -- 172.21.15.83:0/1741923142 wait complete. 2024-04-17T10:43:38.473 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2024-04-17T10:43:38.278527+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":38,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":344896,"kb_used_data":8008,"kb_used_omap":12,"kb_used_meta":336755,"kb_avail":754629824,"statfs":{"total":773094113280,"available":772740939776,"internally_reserved":0,"allocated":8200192,"data_stored":4156128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":344837680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.822026"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.475786+0000","last_change":"2024-04-17T10:43:33.475786+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.475786+0000","last_undegraded":"2024-04-17T10:43:33.475786+0000","last_fullsized":"2024-04-17T10:43:33.475786+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.488388+0000","last_change":"2024-04-17T10:43:33.488388+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.488388+0000","last_undegraded":"2024-04-17T10:43:33.488388+0000","last_fullsized":"2024-04-17T10:43:33.488388+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.474093+0000","last_change":"2024-04-17T10:43:33.474093+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.474093+0000","last_undegraded":"2024-04-17T10:43:33.474093+0000","last_fullsized":"2024-04-17T10:43:33.474093+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.475007+0000","last_change":"2024-04-17T10:43:33.475007+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.475007+0000","last_undegraded":"2024-04-17T10:43:33.475007+0000","last_fullsized":"2024-04-17T10:43:33.475007+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.469492+0000","last_change":"2024-04-17T10:43:33.469492+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.469492+0000","last_undegraded":"2024-04-17T10:43:33.469492+0000","last_fullsized":"2024-04-17T10:43:33.469492+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.492867+0000","last_change":"2024-04-17T10:43:33.492867+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.492867+0000","last_undegraded":"2024-04-17T10:43:33.492867+0000","last_fullsized":"2024-04-17T10:43:33.492867+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.475679+0000","last_change":"2024-04-17T10:43:33.475679+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.475679+0000","last_undegraded":"2024-04-17T10:43:33.475679+0000","last_fullsized":"2024-04-17T10:43:33.475679+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.478045+0000","last_change":"2024-04-17T10:43:33.478045+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.478045+0000","last_undegraded":"2024-04-17T10:43:33.478045+0000","last_fullsized":"2024-04-17T10:43:33.478045+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.470779+0000","last_change":"2024-04-17T10:43:33.470779+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.470779+0000","last_undegraded":"2024-04-17T10:43:33.470779+0000","last_fullsized":"2024-04-17T10:43:33.470779+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.480695+0000","last_change":"2024-04-17T10:43:33.480695+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.480695+0000","last_undegraded":"2024-04-17T10:43:33.480695+0000","last_fullsized":"2024-04-17T10:43:33.480695+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.469761+0000","last_change":"2024-04-17T10:43:33.469761+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.469761+0000","last_undegraded":"2024-04-17T10:43:33.469761+0000","last_fullsized":"2024-04-17T10:43:33.469761+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.471328+0000","last_change":"2024-04-17T10:43:33.471328+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.471328+0000","last_undegraded":"2024-04-17T10:43:33.471328+0000","last_fullsized":"2024-04-17T10:43:33.471328+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.468149+0000","last_change":"2024-04-17T10:43:33.468149+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.468149+0000","last_undegraded":"2024-04-17T10:43:33.468149+0000","last_fullsized":"2024-04-17T10:43:33.468149+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.470994+0000","last_change":"2024-04-17T10:43:33.470994+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.470994+0000","last_undegraded":"2024-04-17T10:43:33.470994+0000","last_fullsized":"2024-04-17T10:43:33.470994+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.472752+0000","last_change":"2024-04-17T10:43:33.472752+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.472752+0000","last_undegraded":"2024-04-17T10:43:33.472752+0000","last_fullsized":"2024-04-17T10:43:33.472752+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.479475+0000","last_change":"2024-04-17T10:43:33.479475+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.479475+0000","last_undegraded":"2024-04-17T10:43:33.479475+0000","last_fullsized":"2024-04-17T10:43:33.479475+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.470329+0000","last_change":"2024-04-17T10:43:33.470329+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.470329+0000","last_undegraded":"2024-04-17T10:43:33.470329+0000","last_fullsized":"2024-04-17T10:43:33.470329+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.470663+0000","last_change":"2024-04-17T10:43:33.470663+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.470663+0000","last_undegraded":"2024-04-17T10:43:33.470663+0000","last_fullsized":"2024-04-17T10:43:33.470663+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.491227+0000","last_change":"2024-04-17T10:43:33.491227+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.491227+0000","last_undegraded":"2024-04-17T10:43:33.491227+0000","last_fullsized":"2024-04-17T10:43:33.491227+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.469792+0000","last_change":"2024-04-17T10:43:33.469792+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.469792+0000","last_undegraded":"2024-04-17T10:43:33.469792+0000","last_fullsized":"2024-04-17T10:43:33.469792+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.469177+0000","last_change":"2024-04-17T10:43:33.469177+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.469177+0000","last_undegraded":"2024-04-17T10:43:33.469177+0000","last_fullsized":"2024-04-17T10:43:33.469177+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.477573+0000","last_change":"2024-04-17T10:43:33.477573+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.477573+0000","last_undegraded":"2024-04-17T10:43:33.477573+0000","last_fullsized":"2024-04-17T10:43:33.477573+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.466522+0000","last_change":"2024-04-17T10:43:33.466522+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.466522+0000","last_undegraded":"2024-04-17T10:43:33.466522+0000","last_fullsized":"2024-04-17T10:43:33.466522+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.470617+0000","last_change":"2024-04-17T10:43:33.470617+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.470617+0000","last_undegraded":"2024-04-17T10:43:33.470617+0000","last_fullsized":"2024-04-17T10:43:33.470617+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.466969+0000","last_change":"2024-04-17T10:43:33.466969+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.466969+0000","last_undegraded":"2024-04-17T10:43:33.466969+0000","last_fullsized":"2024-04-17T10:43:33.466969+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.474514+0000","last_change":"2024-04-17T10:43:33.474514+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.474514+0000","last_undegraded":"2024-04-17T10:43:33.474514+0000","last_fullsized":"2024-04-17T10:43:33.474514+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.473429+0000","last_change":"2024-04-17T10:43:33.473429+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.473429+0000","last_undegraded":"2024-04-17T10:43:33.473429+0000","last_fullsized":"2024-04-17T10:43:33.473429+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.467411+0000","last_change":"2024-04-17T10:43:33.467411+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.467411+0000","last_undegraded":"2024-04-17T10:43:33.467411+0000","last_fullsized":"2024-04-17T10:43:33.467411+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.473115+0000","last_change":"2024-04-17T10:43:33.473115+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.473115+0000","last_undegraded":"2024-04-17T10:43:33.473115+0000","last_fullsized":"2024-04-17T10:43:33.473115+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.472797+0000","last_change":"2024-04-17T10:43:33.472797+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.472797+0000","last_undegraded":"2024-04-17T10:43:33.472797+0000","last_fullsized":"2024-04-17T10:43:33.472797+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.473897+0000","last_change":"2024-04-17T10:43:33.473897+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.473897+0000","last_undegraded":"2024-04-17T10:43:33.473897+0000","last_fullsized":"2024-04-17T10:43:33.473897+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.476712+0000","last_change":"2024-04-17T10:43:33.476712+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.476712+0000","last_undegraded":"2024-04-17T10:43:33.476712+0000","last_fullsized":"2024-04-17T10:43:33.476712+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.469503+0000","last_change":"2024-04-17T10:43:33.469503+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.469503+0000","last_undegraded":"2024-04-17T10:43:33.469503+0000","last_fullsized":"2024-04-17T10:43:33.469503+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.497359+0000","last_change":"2024-04-17T10:43:33.488363+0000","last_active":"2024-04-17T10:43:33.497359+0000","last_peered":"2024-04-17T10:43:33.497359+0000","last_clean":"2024-04-17T10:43:33.497359+0000","last_became_active":"2024-04-17T10:43:33.487677+0000","last_became_peered":"2024-04-17T10:43:33.487677+0000","last_unstale":"2024-04-17T10:43:33.497359+0000","last_undegraded":"2024-04-17T10:43:33.497359+0000","last_fullsized":"2024-04-17T10:43:33.497359+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:43:20.629428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.468080+0000","last_change":"2024-04-17T10:43:33.468080+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.468080+0000","last_undegraded":"2024-04-17T10:43:33.468080+0000","last_fullsized":"2024-04-17T10:43:33.468080+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.479669+0000","last_change":"2024-04-17T10:43:33.478164+0000","last_active":"2024-04-17T10:43:33.479669+0000","last_peered":"2024-04-17T10:43:33.479669+0000","last_clean":"2024-04-17T10:43:33.479669+0000","last_became_active":"2024-04-17T10:43:33.477765+0000","last_became_peered":"2024-04-17T10:43:33.477765+0000","last_unstale":"2024-04-17T10:43:33.479669+0000","last_undegraded":"2024-04-17T10:43:33.479669+0000","last_fullsized":"2024-04-17T10:43:33.479669+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:40:08.909983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.486984+0000","last_change":"2024-04-17T10:43:33.486984+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.486984+0000","last_undegraded":"2024-04-17T10:43:33.486984+0000","last_fullsized":"2024-04-17T10:43:33.486984+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.510659+0000","last_change":"2024-04-17T10:43:33.504903+0000","last_active":"2024-04-17T10:43:33.510659+0000","last_peered":"2024-04-17T10:43:33.510659+0000","last_clean":"2024-04-17T10:43:33.510659+0000","last_became_active":"2024-04-17T10:43:33.503926+0000","last_became_peered":"2024-04-17T10:43:33.503926+0000","last_unstale":"2024-04-17T10:43:33.510659+0000","last_undegraded":"2024-04-17T10:43:33.510659+0000","last_fullsized":"2024-04-17T10:43:33.510659+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:33:06.929106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.468718+0000","last_change":"2024-04-17T10:43:33.468718+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.468718+0000","last_undegraded":"2024-04-17T10:43:33.468718+0000","last_fullsized":"2024-04-17T10:43:33.468718+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.496406+0000","last_change":"2024-04-17T10:43:33.491909+0000","last_active":"2024-04-17T10:43:33.496406+0000","last_peered":"2024-04-17T10:43:33.496406+0000","last_clean":"2024-04-17T10:43:33.496406+0000","last_became_active":"2024-04-17T10:43:33.490806+0000","last_became_peered":"2024-04-17T10:43:33.490806+0000","last_unstale":"2024-04-17T10:43:33.496406+0000","last_undegraded":"2024-04-17T10:43:33.496406+0000","last_fullsized":"2024-04-17T10:43:33.496406+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:33:38.078826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.471659+0000","last_change":"2024-04-17T10:43:33.471659+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.471659+0000","last_undegraded":"2024-04-17T10:43:33.471659+0000","last_fullsized":"2024-04-17T10:43:33.471659+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.504597+0000","last_change":"2024-04-17T10:43:33.504035+0000","last_active":"2024-04-17T10:43:33.504597+0000","last_peered":"2024-04-17T10:43:33.504597+0000","last_clean":"2024-04-17T10:43:33.504597+0000","last_became_active":"2024-04-17T10:43:33.503454+0000","last_became_peered":"2024-04-17T10:43:33.503454+0000","last_unstale":"2024-04-17T10:43:33.504597+0000","last_undegraded":"2024-04-17T10:43:33.504597+0000","last_fullsized":"2024-04-17T10:43:33.504597+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:53:36.645485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.467972+0000","last_change":"2024-04-17T10:43:33.467972+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.467972+0000","last_undegraded":"2024-04-17T10:43:33.467972+0000","last_fullsized":"2024-04-17T10:43:33.467972+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.505433+0000","last_change":"2024-04-17T10:43:33.502776+0000","last_active":"2024-04-17T10:43:33.505433+0000","last_peered":"2024-04-17T10:43:33.505433+0000","last_clean":"2024-04-17T10:43:33.505433+0000","last_became_active":"2024-04-17T10:43:33.498687+0000","last_became_peered":"2024-04-17T10:43:33.498687+0000","last_unstale":"2024-04-17T10:43:33.505433+0000","last_undegraded":"2024-04-17T10:43:33.505433+0000","last_fullsized":"2024-04-17T10:43:33.505433+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:45:05.760753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.468424+0000","last_change":"2024-04-17T10:43:33.468424+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.468424+0000","last_undegraded":"2024-04-17T10:43:33.468424+0000","last_fullsized":"2024-04-17T10:43:33.468424+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.506648+0000","last_change":"2024-04-17T10:43:33.506146+0000","last_active":"2024-04-17T10:43:33.506648+0000","last_peered":"2024-04-17T10:43:33.506648+0000","last_clean":"2024-04-17T10:43:33.506648+0000","last_became_active":"2024-04-17T10:43:33.505651+0000","last_became_peered":"2024-04-17T10:43:33.505651+0000","last_unstale":"2024-04-17T10:43:33.506648+0000","last_undegraded":"2024-04-17T10:43:33.506648+0000","last_fullsized":"2024-04-17T10:43:33.506648+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:44:26.965507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.469679+0000","last_change":"2024-04-17T10:43:33.469679+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.469679+0000","last_undegraded":"2024-04-17T10:43:33.469679+0000","last_fullsized":"2024-04-17T10:43:33.469679+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.480916+0000","last_change":"2024-04-17T10:43:33.478569+0000","last_active":"2024-04-17T10:43:33.480916+0000","last_peered":"2024-04-17T10:43:33.480916+0000","last_clean":"2024-04-17T10:43:33.480916+0000","last_became_active":"2024-04-17T10:43:33.477453+0000","last_became_peered":"2024-04-17T10:43:33.477453+0000","last_unstale":"2024-04-17T10:43:33.480916+0000","last_undegraded":"2024-04-17T10:43:33.480916+0000","last_fullsized":"2024-04-17T10:43:33.480916+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:59:17.699632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.473432+0000","last_change":"2024-04-17T10:43:33.473432+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.473432+0000","last_undegraded":"2024-04-17T10:43:33.473432+0000","last_fullsized":"2024-04-17T10:43:33.473432+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.509346+0000","last_change":"2024-04-17T10:43:33.499957+0000","last_active":"2024-04-17T10:43:33.509346+0000","last_peered":"2024-04-17T10:43:33.509346+0000","last_clean":"2024-04-17T10:43:33.509346+0000","last_became_active":"2024-04-17T10:43:33.481936+0000","last_became_peered":"2024-04-17T10:43:33.481936+0000","last_unstale":"2024-04-17T10:43:33.509346+0000","last_undegraded":"2024-04-17T10:43:33.509346+0000","last_fullsized":"2024-04-17T10:43:33.509346+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:34:00.759922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.474388+0000","last_change":"2024-04-17T10:43:33.474388+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.474388+0000","last_undegraded":"2024-04-17T10:43:33.474388+0000","last_fullsized":"2024-04-17T10:43:33.474388+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.497306+0000","last_change":"2024-04-17T10:43:33.494714+0000","last_active":"2024-04-17T10:43:33.497306+0000","last_peered":"2024-04-17T10:43:33.497306+0000","last_clean":"2024-04-17T10:43:33.497306+0000","last_became_active":"2024-04-17T10:43:33.493921+0000","last_became_peered":"2024-04-17T10:43:33.493921+0000","last_unstale":"2024-04-17T10:43:33.497306+0000","last_undegraded":"2024-04-17T10:43:33.497306+0000","last_fullsized":"2024-04-17T10:43:33.497306+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:34:40.985936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.483201+0000","last_change":"2024-04-17T10:43:33.483201+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.483201+0000","last_undegraded":"2024-04-17T10:43:33.483201+0000","last_fullsized":"2024-04-17T10:43:33.483201+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.493416+0000","last_change":"2024-04-17T10:43:33.492746+0000","last_active":"2024-04-17T10:43:33.493416+0000","last_peered":"2024-04-17T10:43:33.493416+0000","last_clean":"2024-04-17T10:43:33.493416+0000","last_became_active":"2024-04-17T10:43:33.492349+0000","last_became_peered":"2024-04-17T10:43:33.492349+0000","last_unstale":"2024-04-17T10:43:33.493416+0000","last_undegraded":"2024-04-17T10:43:33.493416+0000","last_fullsized":"2024-04-17T10:43:33.493416+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:37:09.710132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.471311+0000","last_change":"2024-04-17T10:43:33.471311+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.471311+0000","last_undegraded":"2024-04-17T10:43:33.471311+0000","last_fullsized":"2024-04-17T10:43:33.471311+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.480245+0000","last_change":"2024-04-17T10:43:33.476458+0000","last_active":"2024-04-17T10:43:33.480245+0000","last_peered":"2024-04-17T10:43:33.480245+0000","last_clean":"2024-04-17T10:43:33.480245+0000","last_became_active":"2024-04-17T10:43:33.475608+0000","last_became_peered":"2024-04-17T10:43:33.475608+0000","last_unstale":"2024-04-17T10:43:33.480245+0000","last_undegraded":"2024-04-17T10:43:33.480245+0000","last_fullsized":"2024-04-17T10:43:33.480245+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:45:49.643198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.469551+0000","last_change":"2024-04-17T10:43:33.469551+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.469551+0000","last_undegraded":"2024-04-17T10:43:33.469551+0000","last_fullsized":"2024-04-17T10:43:33.469551+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.481177+0000","last_change":"2024-04-17T10:43:33.479258+0000","last_active":"2024-04-17T10:43:33.481177+0000","last_peered":"2024-04-17T10:43:33.481177+0000","last_clean":"2024-04-17T10:43:33.481177+0000","last_became_active":"2024-04-17T10:43:33.478798+0000","last_became_peered":"2024-04-17T10:43:33.478798+0000","last_unstale":"2024-04-17T10:43:33.481177+0000","last_undegraded":"2024-04-17T10:43:33.481177+0000","last_fullsized":"2024-04-17T10:43:33.481177+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:45:05.760753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.471845+0000","last_change":"2024-04-17T10:43:33.471845+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.471845+0000","last_undegraded":"2024-04-17T10:43:33.471845+0000","last_fullsized":"2024-04-17T10:43:33.471845+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.504738+0000","last_change":"2024-04-17T10:43:33.499759+0000","last_active":"2024-04-17T10:43:33.504738+0000","last_peered":"2024-04-17T10:43:33.504738+0000","last_clean":"2024-04-17T10:43:33.504738+0000","last_became_active":"2024-04-17T10:43:33.498138+0000","last_became_peered":"2024-04-17T10:43:33.498138+0000","last_unstale":"2024-04-17T10:43:33.504738+0000","last_undegraded":"2024-04-17T10:43:33.504738+0000","last_fullsized":"2024-04-17T10:43:33.504738+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:26:58.859542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.469189+0000","last_change":"2024-04-17T10:43:33.469189+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.469189+0000","last_undegraded":"2024-04-17T10:43:33.469189+0000","last_fullsized":"2024-04-17T10:43:33.469189+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.493295+0000","last_change":"2024-04-17T10:43:33.488497+0000","last_active":"2024-04-17T10:43:33.493295+0000","last_peered":"2024-04-17T10:43:33.493295+0000","last_clean":"2024-04-17T10:43:33.493295+0000","last_became_active":"2024-04-17T10:43:33.487777+0000","last_became_peered":"2024-04-17T10:43:33.487777+0000","last_unstale":"2024-04-17T10:43:33.493295+0000","last_undegraded":"2024-04-17T10:43:33.493295+0000","last_fullsized":"2024-04-17T10:43:33.493295+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:42:26.108551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.468055+0000","last_change":"2024-04-17T10:43:33.468055+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.468055+0000","last_undegraded":"2024-04-17T10:43:33.468055+0000","last_fullsized":"2024-04-17T10:43:33.468055+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.510713+0000","last_change":"2024-04-17T10:43:33.505100+0000","last_active":"2024-04-17T10:43:33.510713+0000","last_peered":"2024-04-17T10:43:33.510713+0000","last_clean":"2024-04-17T10:43:33.510713+0000","last_became_active":"2024-04-17T10:43:33.504443+0000","last_became_peered":"2024-04-17T10:43:33.504443+0000","last_unstale":"2024-04-17T10:43:33.510713+0000","last_undegraded":"2024-04-17T10:43:33.510713+0000","last_fullsized":"2024-04-17T10:43:33.510713+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:38:49.951085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.479579+0000","last_change":"2024-04-17T10:43:33.479579+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.479579+0000","last_undegraded":"2024-04-17T10:43:33.479579+0000","last_fullsized":"2024-04-17T10:43:33.479579+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.504571+0000","last_change":"2024-04-17T10:43:33.497414+0000","last_active":"2024-04-17T10:43:33.504571+0000","last_peered":"2024-04-17T10:43:33.504571+0000","last_clean":"2024-04-17T10:43:33.504571+0000","last_became_active":"2024-04-17T10:43:33.495764+0000","last_became_peered":"2024-04-17T10:43:33.495764+0000","last_unstale":"2024-04-17T10:43:33.504571+0000","last_undegraded":"2024-04-17T10:43:33.504571+0000","last_fullsized":"2024-04-17T10:43:33.504571+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:02:33.405494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.470619+0000","last_change":"2024-04-17T10:43:33.470619+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.470619+0000","last_undegraded":"2024-04-17T10:43:33.470619+0000","last_fullsized":"2024-04-17T10:43:33.470619+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.511133+0000","last_change":"2024-04-17T10:43:33.509688+0000","last_active":"2024-04-17T10:43:33.511133+0000","last_peered":"2024-04-17T10:43:33.511133+0000","last_clean":"2024-04-17T10:43:33.511133+0000","last_became_active":"2024-04-17T10:43:33.508865+0000","last_became_peered":"2024-04-17T10:43:33.508865+0000","last_unstale":"2024-04-17T10:43:33.511133+0000","last_undegraded":"2024-04-17T10:43:33.511133+0000","last_fullsized":"2024-04-17T10:43:33.511133+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:11:20.457976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.470650+0000","last_change":"2024-04-17T10:43:33.470650+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.470650+0000","last_undegraded":"2024-04-17T10:43:33.470650+0000","last_fullsized":"2024-04-17T10:43:33.470650+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.509053+0000","last_change":"2024-04-17T10:43:33.507008+0000","last_active":"2024-04-17T10:43:33.509053+0000","last_peered":"2024-04-17T10:43:33.509053+0000","last_clean":"2024-04-17T10:43:33.509053+0000","last_became_active":"2024-04-17T10:43:33.506297+0000","last_became_peered":"2024-04-17T10:43:33.506297+0000","last_unstale":"2024-04-17T10:43:33.509053+0000","last_undegraded":"2024-04-17T10:43:33.509053+0000","last_fullsized":"2024-04-17T10:43:33.509053+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:36:45.162219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.468349+0000","last_change":"2024-04-17T10:43:33.468349+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.468349+0000","last_undegraded":"2024-04-17T10:43:33.468349+0000","last_fullsized":"2024-04-17T10:43:33.468349+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.509663+0000","last_change":"2024-04-17T10:43:33.508340+0000","last_active":"2024-04-17T10:43:33.509663+0000","last_peered":"2024-04-17T10:43:33.509663+0000","last_clean":"2024-04-17T10:43:33.509663+0000","last_became_active":"2024-04-17T10:43:33.507561+0000","last_became_peered":"2024-04-17T10:43:33.507561+0000","last_unstale":"2024-04-17T10:43:33.509663+0000","last_undegraded":"2024-04-17T10:43:33.509663+0000","last_fullsized":"2024-04-17T10:43:33.509663+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:17.570356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.472725+0000","last_change":"2024-04-17T10:43:33.472725+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.472725+0000","last_undegraded":"2024-04-17T10:43:33.472725+0000","last_fullsized":"2024-04-17T10:43:33.472725+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.504785+0000","last_change":"2024-04-17T10:43:33.500518+0000","last_active":"2024-04-17T10:43:33.504785+0000","last_peered":"2024-04-17T10:43:33.504785+0000","last_clean":"2024-04-17T10:43:33.504785+0000","last_became_active":"2024-04-17T10:43:33.499044+0000","last_became_peered":"2024-04-17T10:43:33.499044+0000","last_unstale":"2024-04-17T10:43:33.504785+0000","last_undegraded":"2024-04-17T10:43:33.504785+0000","last_fullsized":"2024-04-17T10:43:33.504785+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:45:49.643198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.472216+0000","last_change":"2024-04-17T10:43:33.472216+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.472216+0000","last_undegraded":"2024-04-17T10:43:33.472216+0000","last_fullsized":"2024-04-17T10:43:33.472216+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.494691+0000","last_change":"2024-04-17T10:43:33.492968+0000","last_active":"2024-04-17T10:43:33.494691+0000","last_peered":"2024-04-17T10:43:33.494691+0000","last_clean":"2024-04-17T10:43:33.494691+0000","last_became_active":"2024-04-17T10:43:33.492286+0000","last_became_peered":"2024-04-17T10:43:33.492286+0000","last_unstale":"2024-04-17T10:43:33.494691+0000","last_undegraded":"2024-04-17T10:43:33.494691+0000","last_fullsized":"2024-04-17T10:43:33.494691+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:07:18.944354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.472216+0000","last_change":"2024-04-17T10:43:33.472216+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.472216+0000","last_undegraded":"2024-04-17T10:43:33.472216+0000","last_fullsized":"2024-04-17T10:43:33.472216+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.483373+0000","last_change":"2024-04-17T10:43:33.482758+0000","last_active":"2024-04-17T10:43:33.483373+0000","last_peered":"2024-04-17T10:43:33.483373+0000","last_clean":"2024-04-17T10:43:33.483373+0000","last_became_active":"2024-04-17T10:43:33.482200+0000","last_became_peered":"2024-04-17T10:43:33.482200+0000","last_unstale":"2024-04-17T10:43:33.483373+0000","last_undegraded":"2024-04-17T10:43:33.483373+0000","last_fullsized":"2024-04-17T10:43:33.483373+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:05:45.628231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.468640+0000","last_change":"2024-04-17T10:43:33.468640+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.468640+0000","last_undegraded":"2024-04-17T10:43:33.468640+0000","last_fullsized":"2024-04-17T10:43:33.468640+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.503257+0000","last_change":"2024-04-17T10:43:33.496516+0000","last_active":"2024-04-17T10:43:33.503257+0000","last_peered":"2024-04-17T10:43:33.503257+0000","last_clean":"2024-04-17T10:43:33.503257+0000","last_became_active":"2024-04-17T10:43:33.494818+0000","last_became_peered":"2024-04-17T10:43:33.494818+0000","last_unstale":"2024-04-17T10:43:33.503257+0000","last_undegraded":"2024-04-17T10:43:33.503257+0000","last_fullsized":"2024-04-17T10:43:33.503257+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:51.195275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.472880+0000","last_change":"2024-04-17T10:43:33.472880+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.472880+0000","last_undegraded":"2024-04-17T10:43:33.472880+0000","last_fullsized":"2024-04-17T10:43:33.472880+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.492457+0000","last_change":"2024-04-17T10:43:33.488378+0000","last_active":"2024-04-17T10:43:33.492457+0000","last_peered":"2024-04-17T10:43:33.492457+0000","last_clean":"2024-04-17T10:43:33.492457+0000","last_became_active":"2024-04-17T10:43:33.487622+0000","last_became_peered":"2024-04-17T10:43:33.487622+0000","last_unstale":"2024-04-17T10:43:33.492457+0000","last_undegraded":"2024-04-17T10:43:33.492457+0000","last_fullsized":"2024-04-17T10:43:33.492457+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:54:54.413725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.466584+0000","last_change":"2024-04-17T10:43:33.466584+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.466584+0000","last_undegraded":"2024-04-17T10:43:33.466584+0000","last_fullsized":"2024-04-17T10:43:33.466584+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.482794+0000","last_change":"2024-04-17T10:43:33.480817+0000","last_active":"2024-04-17T10:43:33.482794+0000","last_peered":"2024-04-17T10:43:33.482794+0000","last_clean":"2024-04-17T10:43:33.482794+0000","last_became_active":"2024-04-17T10:43:33.480442+0000","last_became_peered":"2024-04-17T10:43:33.480442+0000","last_unstale":"2024-04-17T10:43:33.482794+0000","last_undegraded":"2024-04-17T10:43:33.482794+0000","last_fullsized":"2024-04-17T10:43:33.482794+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:45:30.015394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.467765+0000","last_change":"2024-04-17T10:43:33.467765+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.467765+0000","last_undegraded":"2024-04-17T10:43:33.467765+0000","last_fullsized":"2024-04-17T10:43:33.467765+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.510755+0000","last_change":"2024-04-17T10:43:33.510303+0000","last_active":"2024-04-17T10:43:33.510755+0000","last_peered":"2024-04-17T10:43:33.510755+0000","last_clean":"2024-04-17T10:43:33.510755+0000","last_became_active":"2024-04-17T10:43:33.509966+0000","last_became_peered":"2024-04-17T10:43:33.509966+0000","last_unstale":"2024-04-17T10:43:33.510755+0000","last_undegraded":"2024-04-17T10:43:33.510755+0000","last_fullsized":"2024-04-17T10:43:33.510755+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:54:54.413725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.468841+0000","last_change":"2024-04-17T10:43:33.468841+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.468841+0000","last_undegraded":"2024-04-17T10:43:33.468841+0000","last_fullsized":"2024-04-17T10:43:33.468841+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.490422+0000","last_change":"2024-04-17T10:43:33.489836+0000","last_active":"2024-04-17T10:43:33.490422+0000","last_peered":"2024-04-17T10:43:33.490422+0000","last_clean":"2024-04-17T10:43:33.490422+0000","last_became_active":"2024-04-17T10:43:33.489438+0000","last_became_peered":"2024-04-17T10:43:33.489438+0000","last_unstale":"2024-04-17T10:43:33.490422+0000","last_undegraded":"2024-04-17T10:43:33.490422+0000","last_fullsized":"2024-04-17T10:43:33.490422+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:35:50.957989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.488610+0000","last_change":"2024-04-17T10:43:33.488610+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.488610+0000","last_undegraded":"2024-04-17T10:43:33.488610+0000","last_fullsized":"2024-04-17T10:43:33.488610+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.504941+0000","last_change":"2024-04-17T10:43:33.503755+0000","last_active":"2024-04-17T10:43:33.504941+0000","last_peered":"2024-04-17T10:43:33.504941+0000","last_clean":"2024-04-17T10:43:33.504941+0000","last_became_active":"2024-04-17T10:43:33.503377+0000","last_became_peered":"2024-04-17T10:43:33.503377+0000","last_unstale":"2024-04-17T10:43:33.504941+0000","last_undegraded":"2024-04-17T10:43:33.504941+0000","last_fullsized":"2024-04-17T10:43:33.504941+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:36:45.162219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.466736+0000","last_change":"2024-04-17T10:43:33.466736+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.466736+0000","last_undegraded":"2024-04-17T10:43:33.466736+0000","last_fullsized":"2024-04-17T10:43:33.466736+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.510946+0000","last_change":"2024-04-17T10:43:33.506065+0000","last_active":"2024-04-17T10:43:33.510946+0000","last_peered":"2024-04-17T10:43:33.510946+0000","last_clean":"2024-04-17T10:43:33.510946+0000","last_became_active":"2024-04-17T10:43:33.504808+0000","last_became_peered":"2024-04-17T10:43:33.504808+0000","last_unstale":"2024-04-17T10:43:33.510946+0000","last_undegraded":"2024-04-17T10:43:33.510946+0000","last_fullsized":"2024-04-17T10:43:33.510946+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:01:00.790513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.476069+0000","last_change":"2024-04-17T10:43:33.476069+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.476069+0000","last_undegraded":"2024-04-17T10:43:33.476069+0000","last_fullsized":"2024-04-17T10:43:33.476069+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.493140+0000","last_change":"2024-04-17T10:43:33.487447+0000","last_active":"2024-04-17T10:43:33.493140+0000","last_peered":"2024-04-17T10:43:33.493140+0000","last_clean":"2024-04-17T10:43:33.493140+0000","last_became_active":"2024-04-17T10:43:33.486690+0000","last_became_peered":"2024-04-17T10:43:33.486690+0000","last_unstale":"2024-04-17T10:43:33.493140+0000","last_undegraded":"2024-04-17T10:43:33.493140+0000","last_fullsized":"2024-04-17T10:43:33.493140+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:44:20.188944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.472418+0000","last_change":"2024-04-17T10:43:33.472418+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.472418+0000","last_undegraded":"2024-04-17T10:43:33.472418+0000","last_fullsized":"2024-04-17T10:43:33.472418+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.497416+0000","last_change":"2024-04-17T10:43:33.495411+0000","last_active":"2024-04-17T10:43:33.497416+0000","last_peered":"2024-04-17T10:43:33.497416+0000","last_clean":"2024-04-17T10:43:33.497416+0000","last_became_active":"2024-04-17T10:43:33.494671+0000","last_became_peered":"2024-04-17T10:43:33.494671+0000","last_unstale":"2024-04-17T10:43:33.497416+0000","last_undegraded":"2024-04-17T10:43:33.497416+0000","last_fullsized":"2024-04-17T10:43:33.497416+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:42:52.241171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":276,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.476774+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:43:33.476774+0000","last_peered":"2024-04-17T10:43:33.476774+0000","last_clean":"2024-04-17T10:43:33.476774+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:43:33.476774+0000","last_undegraded":"2024-04-17T10:43:33.476774+0000","last_fullsized":"2024-04-17T10:43:33.476774+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574919,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28460,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343380,"statfs":{"total":96636764160,"available":96607621120,"internally_reserved":0,"allocated":1470464,"data_stored":962292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40360,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331480,"statfs":{"total":96636764160,"available":96595435520,"internally_reserved":0,"allocated":876544,"data_stored":371924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574919,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40360,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331480,"statfs":{"total":96636764160,"available":96595435520,"internally_reserved":0,"allocated":876544,"data_stored":371924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50084,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321756,"statfs":{"total":96636764160,"available":96585478144,"internally_reserved":0,"allocated":876544,"data_stored":371924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574920,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54248,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317592,"statfs":{"total":96636764160,"available":96581214208,"internally_reserved":0,"allocated":876544,"data_stored":371924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574918,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36200,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335640,"statfs":{"total":96636764160,"available":96599695360,"internally_reserved":0,"allocated":876544,"data_stored":371924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50088,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321752,"statfs":{"total":96636764160,"available":96585474048,"internally_reserved":0,"allocated":876544,"data_stored":371924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574918,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45096,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326744,"statfs":{"total":96636764160,"available":96590585856,"internally_reserved":0,"allocated":1470464,"data_stored":962292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:43:38.479 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:43:38.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.633+0000 7f530d81d640 1 Processor -- start 2024-04-17T10:43:38.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.633+0000 7f530d81d640 1 -- start start 2024-04-17T10:43:38.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f530d81d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5308072330 0x7f53080cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:38.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f530d81d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53080cfea0 con 0x7f53080cfa10 2024-04-17T10:43:38.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f530d81d640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f530806e320 con 0x7f530815ba30 2024-04-17T10:43:38.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f530d81d640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5308071ea0 con 0x7f5308072330 2024-04-17T10:43:38.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f53077fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f53080cfa10 0x7f530806c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55080/0 (socket says 172.21.15.83:55080) 2024-04-17T10:43:38.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f53077fe640 1 -- 172.21.15.83:0/3238987349 learned_addr learned my addr 172.21.15.83:0/3238987349 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:43:38.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f53067fc640 1 --2- 172.21.15.83:0/3238987349 >> v2:172.21.15.167:3300/0 conn(0x7f5308072330 0x7f53080cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:38.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f5305ffb640 1 -- 172.21.15.83:0/3238987349 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3642022656 0 0) 0x7f53080cfea0 con 0x7f53080cfa10 2024-04-17T10:43:38.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f5305ffb640 1 -- 172.21.15.83:0/3238987349 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52f4003530 con 0x7f53080cfa10 2024-04-17T10:43:38.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f5305ffb640 1 -- 172.21.15.83:0/3238987349 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2762610919 0 0) 0x7f52f4003530 con 0x7f53080cfa10 2024-04-17T10:43:38.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f5305ffb640 1 -- 172.21.15.83:0/3238987349 >> v2:172.21.15.167:3300/0 conn(0x7f5308072330 msgr2=0x7f53080cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:38.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f5305ffb640 1 --2- 172.21.15.83:0/3238987349 >> v2:172.21.15.167:3300/0 conn(0x7f5308072330 0x7f53080cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f5305ffb640 1 -- 172.21.15.83:0/3238987349 >> v1:172.21.15.167:6789/0 conn(0x7f530815ba30 legacy=0x7f530815c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:38.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f5305ffb640 1 -- 172.21.15.83:0/3238987349 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530806e040 con 0x7f53080cfa10 2024-04-17T10:43:38.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f5305ffb640 1 -- 172.21.15.83:0/3238987349 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f5300001a00 con 0x7f53080cfa10 2024-04-17T10:43:38.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f5305ffb640 1 -- 172.21.15.83:0/3238987349 >> v1:172.21.15.83:6789/0 conn(0x7f53080cfa10 legacy=0x7f530806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:38.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f5305ffb640 1 --2- 172.21.15.83:0/3238987349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52f4003a10 0x7f52f4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:38.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f5305ffb640 1 --2- 172.21.15.83:0/3238987349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52f4004810 0x7f52f4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:38.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f5305ffb640 1 --2- 172.21.15.83:0/3238987349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52f4005220 0x7f52f400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:38.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f5305ffb640 1 -- 172.21.15.83:0/3238987349 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5308071ea0 con 0x7f52f4005220 2024-04-17T10:43:38.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f5305ffb640 1 -- 172.21.15.83:0/3238987349 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f400c170 con 0x7f52f4004810 2024-04-17T10:43:38.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f5305ffb640 1 -- 172.21.15.83:0/3238987349 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f400c370 con 0x7f52f4003a10 2024-04-17T10:43:38.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f53077fe640 1 --2- 172.21.15.83:0/3238987349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52f4005220 0x7f52f400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:38.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f53067fc640 1 --2- 172.21.15.83:0/3238987349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52f4004810 0x7f52f4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:38.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f53077fe640 1 -- 172.21.15.83:0/3238987349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52f4003a10 msgr2=0x7f52f4003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:38.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f53077fe640 1 --2- 172.21.15.83:0/3238987349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52f4003a10 0x7f52f4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f53077fe640 1 -- 172.21.15.83:0/3238987349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52f4004810 msgr2=0x7f52f4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:38.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f53077fe640 1 --2- 172.21.15.83:0/3238987349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52f4004810 0x7f52f4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f53077fe640 1 -- 172.21.15.83:0/3238987349 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f4003530 con 0x7f52f4005220 2024-04-17T10:43:38.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f53077fe640 1 -- 172.21.15.83:0/3238987349 reap_dead start 2024-04-17T10:43:38.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f53077fe640 1 --2- 172.21.15.83:0/3238987349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52f4005220 0x7f52f400b9d0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f530000a760 tx=0x7f530001f9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ee4248dacd47ce server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:38.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f5305ffb640 1 -- 172.21.15.83:0/3238987349 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530000a3b0 con 0x7f52f4005220 2024-04-17T10:43:38.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f5305ffb640 1 -- 172.21.15.83:0/3238987349 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53000aa610 con 0x7f52f4005220 2024-04-17T10:43:38.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.637+0000 7f5305ffb640 1 -- 172.21.15.83:0/3238987349 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53000aaa00 con 0x7f52f4005220 2024-04-17T10:43:38.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.641+0000 7f530d81d640 1 -- 172.21.15.83:0/3238987349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52f4005220 msgr2=0x7f52f400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:38.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.641+0000 7f530d81d640 1 --2- 172.21.15.83:0/3238987349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52f4005220 0x7f52f400b9d0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f530000a760 tx=0x7f530001f9e0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.641+0000 7f530d81d640 1 -- 172.21.15.83:0/3238987349 shutdown_connections 2024-04-17T10:43:38.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.641+0000 7f530d81d640 1 --2- 172.21.15.83:0/3238987349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52f4005220 0x7f52f400b9d0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.641+0000 7f530d81d640 1 -- 172.21.15.83:0/3238987349 >> 172.21.15.83:0/3238987349 conn(0x7f530806abe0 msgr2=0x7f530806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:38.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.641+0000 7f530d81d640 1 -- 172.21.15.83:0/3238987349 shutdown_connections 2024-04-17T10:43:38.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.641+0000 7f530d81d640 1 -- 172.21.15.83:0/3238987349 wait complete. 2024-04-17T10:43:38.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.641+0000 7f530d81d640 1 Processor -- start 2024-04-17T10:43:38.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.641+0000 7f530d81d640 1 -- start start 2024-04-17T10:43:38.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.641+0000 7f530d81d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52f4005220 0x7f530815d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:38.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.641+0000 7f530d81d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f530815be10 0x7f5308150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:38.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.641+0000 7f530d81d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5308142120 0x7f5308151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:38.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.641+0000 7f530d81d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53080aeae0 con 0x7f530815be10 2024-04-17T10:43:38.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.641+0000 7f530d81d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5308071b30 con 0x7f5308142120 2024-04-17T10:43:38.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.641+0000 7f530d81d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f530806dca0 con 0x7f52f4005220 2024-04-17T10:43:38.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.641+0000 7f53067fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f530815be10 0x7f5308150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:38.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.641+0000 7f53067fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f530815be10 0x7f5308150580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52348/0 (socket says 172.21.15.83:52348) 2024-04-17T10:43:38.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.641+0000 7f53067fc640 1 -- 172.21.15.83:0/820207665 learned_addr learned my addr 172.21.15.83:0/820207665 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:38.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.641+0000 7f5306ffd640 1 --2- 172.21.15.83:0/820207665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52f4005220 0x7f530815d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:38.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.641+0000 7f53077fe640 1 --2- 172.21.15.83:0/820207665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5308142120 0x7f5308151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:38.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.641+0000 7f53067fc640 1 -- 172.21.15.83:0/820207665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52f4005220 msgr2=0x7f530815d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:38.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.641+0000 7f53067fc640 1 --2- 172.21.15.83:0/820207665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52f4005220 0x7f530815d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.641+0000 7f53067fc640 1 -- 172.21.15.83:0/820207665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5308142120 msgr2=0x7f5308151090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:38.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.641+0000 7f53067fc640 1 --2- 172.21.15.83:0/820207665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5308142120 0x7f5308151090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.641+0000 7f53067fc640 1 -- 172.21.15.83:0/820207665 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530815c4d0 con 0x7f530815be10 2024-04-17T10:43:38.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.641+0000 7f53067fc640 1 --2- 172.21.15.83:0/820207665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f530815be10 0x7f5308150580 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f52f0011560 tx=0x7f52f0011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:38.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.641+0000 7f52fffff640 1 -- 172.21.15.83:0/820207665 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f0015930 con 0x7f530815be10 2024-04-17T10:43:38.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.641+0000 7f52fffff640 1 -- 172.21.15.83:0/820207665 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f0015ad0 con 0x7f530815be10 2024-04-17T10:43:38.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.641+0000 7f52fffff640 1 -- 172.21.15.83:0/820207665 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f000f460 con 0x7f530815be10 2024-04-17T10:43:38.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.641+0000 7f530d81d640 1 -- 172.21.15.83:0/820207665 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f530815c7b0 con 0x7f530815be10 2024-04-17T10:43:38.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.641+0000 7f530d81d640 1 -- 172.21.15.83:0/820207665 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f530815cd70 con 0x7f530815be10 2024-04-17T10:43:38.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.645+0000 7f530d81d640 1 -- 172.21.15.83:0/820207665 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5308074f90 con 0x7f530815be10 2024-04-17T10:43:38.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.645+0000 7f52fffff640 1 -- 172.21.15.83:0/820207665 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f52f000f600 con 0x7f530815be10 2024-04-17T10:43:38.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.645+0000 7f52fffff640 1 --2- 172.21.15.83:0/820207665 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f52e8066f60 0x7f52e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:38.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.645+0000 7f52fffff640 1 -- 172.21.15.83:0/820207665 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f52f00858f0 con 0x7f530815be10 2024-04-17T10:43:38.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.645+0000 7f5306ffd640 1 --2- 172.21.15.83:0/820207665 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f52e8066f60 0x7f52e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:38.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.645+0000 7f5306ffd640 1 --2- 172.21.15.83:0/820207665 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f52e8066f60 0x7f52e8069420 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f52f80015b0 tx=0x7f52f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:38.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.649+0000 7f52fffff640 1 -- 172.21.15.83:0/820207665 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f52f0019030 con 0x7f530815be10 2024-04-17T10:43:38.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.801+0000 7f530d81d640 1 -- 172.21.15.83:0/820207665 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f53080cf4a0 con 0x7f52e8066f60 2024-04-17T10:43:38.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.809+0000 7f52fffff640 1 -- 172.21.15.83:0/820207665 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246757 (secure 0 0 0) 0x7f53080cf4a0 con 0x7f52e8066f60 2024-04-17T10:43:38.807 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:43:38.808 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:43:38.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.813+0000 7f530d81d640 1 -- 172.21.15.83:0/820207665 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f52e8066f60 msgr2=0x7f52e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:38.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.813+0000 7f530d81d640 1 --2- 172.21.15.83:0/820207665 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f52e8066f60 0x7f52e8069420 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f52f80015b0 tx=0x7f52f8009040 comp rx=0 tx=0).stop 2024-04-17T10:43:38.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.813+0000 7f530d81d640 1 -- 172.21.15.83:0/820207665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f530815be10 msgr2=0x7f5308150580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:38.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.813+0000 7f530d81d640 1 --2- 172.21.15.83:0/820207665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f530815be10 0x7f5308150580 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f52f0011560 tx=0x7f52f0011a20 comp rx=0 tx=0).stop 2024-04-17T10:43:38.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.813+0000 7f530d81d640 1 -- 172.21.15.83:0/820207665 shutdown_connections 2024-04-17T10:43:38.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.813+0000 7f530d81d640 1 --2- 172.21.15.83:0/820207665 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f52e8066f60 0x7f52e8069420 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.813+0000 7f530d81d640 1 --2- 172.21.15.83:0/820207665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5308142120 0x7f5308151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.813+0000 7f530d81d640 1 --2- 172.21.15.83:0/820207665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f530815be10 0x7f5308150580 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.813+0000 7f530d81d640 1 --2- 172.21.15.83:0/820207665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52f4005220 0x7f530815d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.813+0000 7f530d81d640 1 -- 172.21.15.83:0/820207665 >> 172.21.15.83:0/820207665 conn(0x7f530806abe0 msgr2=0x7f530806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:38.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.813+0000 7f530d81d640 1 -- 172.21.15.83:0/820207665 shutdown_connections 2024-04-17T10:43:38.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.813+0000 7f530d81d640 1 -- 172.21.15.83:0/820207665 wait complete. 2024-04-17T10:43:38.824 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2024-04-17T10:43:38.278527+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":38,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":344896,"kb_used_data":8008,"kb_used_omap":12,"kb_used_meta":336755,"kb_avail":754629824,"statfs":{"total":773094113280,"available":772740939776,"internally_reserved":0,"allocated":8200192,"data_stored":4156128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":344837680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.822026"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.475786+0000","last_change":"2024-04-17T10:43:33.475786+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.475786+0000","last_undegraded":"2024-04-17T10:43:33.475786+0000","last_fullsized":"2024-04-17T10:43:33.475786+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.488388+0000","last_change":"2024-04-17T10:43:33.488388+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.488388+0000","last_undegraded":"2024-04-17T10:43:33.488388+0000","last_fullsized":"2024-04-17T10:43:33.488388+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.474093+0000","last_change":"2024-04-17T10:43:33.474093+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.474093+0000","last_undegraded":"2024-04-17T10:43:33.474093+0000","last_fullsized":"2024-04-17T10:43:33.474093+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.475007+0000","last_change":"2024-04-17T10:43:33.475007+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.475007+0000","last_undegraded":"2024-04-17T10:43:33.475007+0000","last_fullsized":"2024-04-17T10:43:33.475007+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.469492+0000","last_change":"2024-04-17T10:43:33.469492+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.469492+0000","last_undegraded":"2024-04-17T10:43:33.469492+0000","last_fullsized":"2024-04-17T10:43:33.469492+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.492867+0000","last_change":"2024-04-17T10:43:33.492867+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.492867+0000","last_undegraded":"2024-04-17T10:43:33.492867+0000","last_fullsized":"2024-04-17T10:43:33.492867+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.475679+0000","last_change":"2024-04-17T10:43:33.475679+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.475679+0000","last_undegraded":"2024-04-17T10:43:33.475679+0000","last_fullsized":"2024-04-17T10:43:33.475679+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.478045+0000","last_change":"2024-04-17T10:43:33.478045+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.478045+0000","last_undegraded":"2024-04-17T10:43:33.478045+0000","last_fullsized":"2024-04-17T10:43:33.478045+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.470779+0000","last_change":"2024-04-17T10:43:33.470779+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.470779+0000","last_undegraded":"2024-04-17T10:43:33.470779+0000","last_fullsized":"2024-04-17T10:43:33.470779+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.480695+0000","last_change":"2024-04-17T10:43:33.480695+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.480695+0000","last_undegraded":"2024-04-17T10:43:33.480695+0000","last_fullsized":"2024-04-17T10:43:33.480695+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.469761+0000","last_change":"2024-04-17T10:43:33.469761+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.469761+0000","last_undegraded":"2024-04-17T10:43:33.469761+0000","last_fullsized":"2024-04-17T10:43:33.469761+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.471328+0000","last_change":"2024-04-17T10:43:33.471328+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.471328+0000","last_undegraded":"2024-04-17T10:43:33.471328+0000","last_fullsized":"2024-04-17T10:43:33.471328+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.468149+0000","last_change":"2024-04-17T10:43:33.468149+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.468149+0000","last_undegraded":"2024-04-17T10:43:33.468149+0000","last_fullsized":"2024-04-17T10:43:33.468149+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.470994+0000","last_change":"2024-04-17T10:43:33.470994+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.470994+0000","last_undegraded":"2024-04-17T10:43:33.470994+0000","last_fullsized":"2024-04-17T10:43:33.470994+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.472752+0000","last_change":"2024-04-17T10:43:33.472752+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.472752+0000","last_undegraded":"2024-04-17T10:43:33.472752+0000","last_fullsized":"2024-04-17T10:43:33.472752+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.479475+0000","last_change":"2024-04-17T10:43:33.479475+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.479475+0000","last_undegraded":"2024-04-17T10:43:33.479475+0000","last_fullsized":"2024-04-17T10:43:33.479475+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.470329+0000","last_change":"2024-04-17T10:43:33.470329+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.470329+0000","last_undegraded":"2024-04-17T10:43:33.470329+0000","last_fullsized":"2024-04-17T10:43:33.470329+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.470663+0000","last_change":"2024-04-17T10:43:33.470663+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.470663+0000","last_undegraded":"2024-04-17T10:43:33.470663+0000","last_fullsized":"2024-04-17T10:43:33.470663+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.491227+0000","last_change":"2024-04-17T10:43:33.491227+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.491227+0000","last_undegraded":"2024-04-17T10:43:33.491227+0000","last_fullsized":"2024-04-17T10:43:33.491227+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.469792+0000","last_change":"2024-04-17T10:43:33.469792+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.469792+0000","last_undegraded":"2024-04-17T10:43:33.469792+0000","last_fullsized":"2024-04-17T10:43:33.469792+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.469177+0000","last_change":"2024-04-17T10:43:33.469177+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.469177+0000","last_undegraded":"2024-04-17T10:43:33.469177+0000","last_fullsized":"2024-04-17T10:43:33.469177+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.477573+0000","last_change":"2024-04-17T10:43:33.477573+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.477573+0000","last_undegraded":"2024-04-17T10:43:33.477573+0000","last_fullsized":"2024-04-17T10:43:33.477573+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.466522+0000","last_change":"2024-04-17T10:43:33.466522+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.466522+0000","last_undegraded":"2024-04-17T10:43:33.466522+0000","last_fullsized":"2024-04-17T10:43:33.466522+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.470617+0000","last_change":"2024-04-17T10:43:33.470617+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.470617+0000","last_undegraded":"2024-04-17T10:43:33.470617+0000","last_fullsized":"2024-04-17T10:43:33.470617+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.466969+0000","last_change":"2024-04-17T10:43:33.466969+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.466969+0000","last_undegraded":"2024-04-17T10:43:33.466969+0000","last_fullsized":"2024-04-17T10:43:33.466969+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.474514+0000","last_change":"2024-04-17T10:43:33.474514+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.474514+0000","last_undegraded":"2024-04-17T10:43:33.474514+0000","last_fullsized":"2024-04-17T10:43:33.474514+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.473429+0000","last_change":"2024-04-17T10:43:33.473429+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.473429+0000","last_undegraded":"2024-04-17T10:43:33.473429+0000","last_fullsized":"2024-04-17T10:43:33.473429+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.467411+0000","last_change":"2024-04-17T10:43:33.467411+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.467411+0000","last_undegraded":"2024-04-17T10:43:33.467411+0000","last_fullsized":"2024-04-17T10:43:33.467411+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.473115+0000","last_change":"2024-04-17T10:43:33.473115+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.473115+0000","last_undegraded":"2024-04-17T10:43:33.473115+0000","last_fullsized":"2024-04-17T10:43:33.473115+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.472797+0000","last_change":"2024-04-17T10:43:33.472797+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.472797+0000","last_undegraded":"2024-04-17T10:43:33.472797+0000","last_fullsized":"2024-04-17T10:43:33.472797+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.473897+0000","last_change":"2024-04-17T10:43:33.473897+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.473897+0000","last_undegraded":"2024-04-17T10:43:33.473897+0000","last_fullsized":"2024-04-17T10:43:33.473897+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.476712+0000","last_change":"2024-04-17T10:43:33.476712+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.476712+0000","last_undegraded":"2024-04-17T10:43:33.476712+0000","last_fullsized":"2024-04-17T10:43:33.476712+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.469503+0000","last_change":"2024-04-17T10:43:33.469503+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.469503+0000","last_undegraded":"2024-04-17T10:43:33.469503+0000","last_fullsized":"2024-04-17T10:43:33.469503+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.497359+0000","last_change":"2024-04-17T10:43:33.488363+0000","last_active":"2024-04-17T10:43:33.497359+0000","last_peered":"2024-04-17T10:43:33.497359+0000","last_clean":"2024-04-17T10:43:33.497359+0000","last_became_active":"2024-04-17T10:43:33.487677+0000","last_became_peered":"2024-04-17T10:43:33.487677+0000","last_unstale":"2024-04-17T10:43:33.497359+0000","last_undegraded":"2024-04-17T10:43:33.497359+0000","last_fullsized":"2024-04-17T10:43:33.497359+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:43:20.629428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.468080+0000","last_change":"2024-04-17T10:43:33.468080+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.468080+0000","last_undegraded":"2024-04-17T10:43:33.468080+0000","last_fullsized":"2024-04-17T10:43:33.468080+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.479669+0000","last_change":"2024-04-17T10:43:33.478164+0000","last_active":"2024-04-17T10:43:33.479669+0000","last_peered":"2024-04-17T10:43:33.479669+0000","last_clean":"2024-04-17T10:43:33.479669+0000","last_became_active":"2024-04-17T10:43:33.477765+0000","last_became_peered":"2024-04-17T10:43:33.477765+0000","last_unstale":"2024-04-17T10:43:33.479669+0000","last_undegraded":"2024-04-17T10:43:33.479669+0000","last_fullsized":"2024-04-17T10:43:33.479669+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:40:08.909983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.486984+0000","last_change":"2024-04-17T10:43:33.486984+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.486984+0000","last_undegraded":"2024-04-17T10:43:33.486984+0000","last_fullsized":"2024-04-17T10:43:33.486984+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.510659+0000","last_change":"2024-04-17T10:43:33.504903+0000","last_active":"2024-04-17T10:43:33.510659+0000","last_peered":"2024-04-17T10:43:33.510659+0000","last_clean":"2024-04-17T10:43:33.510659+0000","last_became_active":"2024-04-17T10:43:33.503926+0000","last_became_peered":"2024-04-17T10:43:33.503926+0000","last_unstale":"2024-04-17T10:43:33.510659+0000","last_undegraded":"2024-04-17T10:43:33.510659+0000","last_fullsized":"2024-04-17T10:43:33.510659+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:33:06.929106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.468718+0000","last_change":"2024-04-17T10:43:33.468718+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.468718+0000","last_undegraded":"2024-04-17T10:43:33.468718+0000","last_fullsized":"2024-04-17T10:43:33.468718+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.496406+0000","last_change":"2024-04-17T10:43:33.491909+0000","last_active":"2024-04-17T10:43:33.496406+0000","last_peered":"2024-04-17T10:43:33.496406+0000","last_clean":"2024-04-17T10:43:33.496406+0000","last_became_active":"2024-04-17T10:43:33.490806+0000","last_became_peered":"2024-04-17T10:43:33.490806+0000","last_unstale":"2024-04-17T10:43:33.496406+0000","last_undegraded":"2024-04-17T10:43:33.496406+0000","last_fullsized":"2024-04-17T10:43:33.496406+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:33:38.078826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.471659+0000","last_change":"2024-04-17T10:43:33.471659+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.471659+0000","last_undegraded":"2024-04-17T10:43:33.471659+0000","last_fullsized":"2024-04-17T10:43:33.471659+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.504597+0000","last_change":"2024-04-17T10:43:33.504035+0000","last_active":"2024-04-17T10:43:33.504597+0000","last_peered":"2024-04-17T10:43:33.504597+0000","last_clean":"2024-04-17T10:43:33.504597+0000","last_became_active":"2024-04-17T10:43:33.503454+0000","last_became_peered":"2024-04-17T10:43:33.503454+0000","last_unstale":"2024-04-17T10:43:33.504597+0000","last_undegraded":"2024-04-17T10:43:33.504597+0000","last_fullsized":"2024-04-17T10:43:33.504597+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:53:36.645485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.467972+0000","last_change":"2024-04-17T10:43:33.467972+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.467972+0000","last_undegraded":"2024-04-17T10:43:33.467972+0000","last_fullsized":"2024-04-17T10:43:33.467972+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.505433+0000","last_change":"2024-04-17T10:43:33.502776+0000","last_active":"2024-04-17T10:43:33.505433+0000","last_peered":"2024-04-17T10:43:33.505433+0000","last_clean":"2024-04-17T10:43:33.505433+0000","last_became_active":"2024-04-17T10:43:33.498687+0000","last_became_peered":"2024-04-17T10:43:33.498687+0000","last_unstale":"2024-04-17T10:43:33.505433+0000","last_undegraded":"2024-04-17T10:43:33.505433+0000","last_fullsized":"2024-04-17T10:43:33.505433+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:45:05.760753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.468424+0000","last_change":"2024-04-17T10:43:33.468424+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.468424+0000","last_undegraded":"2024-04-17T10:43:33.468424+0000","last_fullsized":"2024-04-17T10:43:33.468424+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.506648+0000","last_change":"2024-04-17T10:43:33.506146+0000","last_active":"2024-04-17T10:43:33.506648+0000","last_peered":"2024-04-17T10:43:33.506648+0000","last_clean":"2024-04-17T10:43:33.506648+0000","last_became_active":"2024-04-17T10:43:33.505651+0000","last_became_peered":"2024-04-17T10:43:33.505651+0000","last_unstale":"2024-04-17T10:43:33.506648+0000","last_undegraded":"2024-04-17T10:43:33.506648+0000","last_fullsized":"2024-04-17T10:43:33.506648+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:44:26.965507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.469679+0000","last_change":"2024-04-17T10:43:33.469679+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.469679+0000","last_undegraded":"2024-04-17T10:43:33.469679+0000","last_fullsized":"2024-04-17T10:43:33.469679+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.480916+0000","last_change":"2024-04-17T10:43:33.478569+0000","last_active":"2024-04-17T10:43:33.480916+0000","last_peered":"2024-04-17T10:43:33.480916+0000","last_clean":"2024-04-17T10:43:33.480916+0000","last_became_active":"2024-04-17T10:43:33.477453+0000","last_became_peered":"2024-04-17T10:43:33.477453+0000","last_unstale":"2024-04-17T10:43:33.480916+0000","last_undegraded":"2024-04-17T10:43:33.480916+0000","last_fullsized":"2024-04-17T10:43:33.480916+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:59:17.699632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.473432+0000","last_change":"2024-04-17T10:43:33.473432+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.473432+0000","last_undegraded":"2024-04-17T10:43:33.473432+0000","last_fullsized":"2024-04-17T10:43:33.473432+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.509346+0000","last_change":"2024-04-17T10:43:33.499957+0000","last_active":"2024-04-17T10:43:33.509346+0000","last_peered":"2024-04-17T10:43:33.509346+0000","last_clean":"2024-04-17T10:43:33.509346+0000","last_became_active":"2024-04-17T10:43:33.481936+0000","last_became_peered":"2024-04-17T10:43:33.481936+0000","last_unstale":"2024-04-17T10:43:33.509346+0000","last_undegraded":"2024-04-17T10:43:33.509346+0000","last_fullsized":"2024-04-17T10:43:33.509346+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:34:00.759922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.474388+0000","last_change":"2024-04-17T10:43:33.474388+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.474388+0000","last_undegraded":"2024-04-17T10:43:33.474388+0000","last_fullsized":"2024-04-17T10:43:33.474388+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.497306+0000","last_change":"2024-04-17T10:43:33.494714+0000","last_active":"2024-04-17T10:43:33.497306+0000","last_peered":"2024-04-17T10:43:33.497306+0000","last_clean":"2024-04-17T10:43:33.497306+0000","last_became_active":"2024-04-17T10:43:33.493921+0000","last_became_peered":"2024-04-17T10:43:33.493921+0000","last_unstale":"2024-04-17T10:43:33.497306+0000","last_undegraded":"2024-04-17T10:43:33.497306+0000","last_fullsized":"2024-04-17T10:43:33.497306+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:34:40.985936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.483201+0000","last_change":"2024-04-17T10:43:33.483201+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.483201+0000","last_undegraded":"2024-04-17T10:43:33.483201+0000","last_fullsized":"2024-04-17T10:43:33.483201+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.493416+0000","last_change":"2024-04-17T10:43:33.492746+0000","last_active":"2024-04-17T10:43:33.493416+0000","last_peered":"2024-04-17T10:43:33.493416+0000","last_clean":"2024-04-17T10:43:33.493416+0000","last_became_active":"2024-04-17T10:43:33.492349+0000","last_became_peered":"2024-04-17T10:43:33.492349+0000","last_unstale":"2024-04-17T10:43:33.493416+0000","last_undegraded":"2024-04-17T10:43:33.493416+0000","last_fullsized":"2024-04-17T10:43:33.493416+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:37:09.710132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.471311+0000","last_change":"2024-04-17T10:43:33.471311+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.471311+0000","last_undegraded":"2024-04-17T10:43:33.471311+0000","last_fullsized":"2024-04-17T10:43:33.471311+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.480245+0000","last_change":"2024-04-17T10:43:33.476458+0000","last_active":"2024-04-17T10:43:33.480245+0000","last_peered":"2024-04-17T10:43:33.480245+0000","last_clean":"2024-04-17T10:43:33.480245+0000","last_became_active":"2024-04-17T10:43:33.475608+0000","last_became_peered":"2024-04-17T10:43:33.475608+0000","last_unstale":"2024-04-17T10:43:33.480245+0000","last_undegraded":"2024-04-17T10:43:33.480245+0000","last_fullsized":"2024-04-17T10:43:33.480245+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:45:49.643198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.469551+0000","last_change":"2024-04-17T10:43:33.469551+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.469551+0000","last_undegraded":"2024-04-17T10:43:33.469551+0000","last_fullsized":"2024-04-17T10:43:33.469551+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.481177+0000","last_change":"2024-04-17T10:43:33.479258+0000","last_active":"2024-04-17T10:43:33.481177+0000","last_peered":"2024-04-17T10:43:33.481177+0000","last_clean":"2024-04-17T10:43:33.481177+0000","last_became_active":"2024-04-17T10:43:33.478798+0000","last_became_peered":"2024-04-17T10:43:33.478798+0000","last_unstale":"2024-04-17T10:43:33.481177+0000","last_undegraded":"2024-04-17T10:43:33.481177+0000","last_fullsized":"2024-04-17T10:43:33.481177+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:45:05.760753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.471845+0000","last_change":"2024-04-17T10:43:33.471845+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.471845+0000","last_undegraded":"2024-04-17T10:43:33.471845+0000","last_fullsized":"2024-04-17T10:43:33.471845+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.504738+0000","last_change":"2024-04-17T10:43:33.499759+0000","last_active":"2024-04-17T10:43:33.504738+0000","last_peered":"2024-04-17T10:43:33.504738+0000","last_clean":"2024-04-17T10:43:33.504738+0000","last_became_active":"2024-04-17T10:43:33.498138+0000","last_became_peered":"2024-04-17T10:43:33.498138+0000","last_unstale":"2024-04-17T10:43:33.504738+0000","last_undegraded":"2024-04-17T10:43:33.504738+0000","last_fullsized":"2024-04-17T10:43:33.504738+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:26:58.859542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.469189+0000","last_change":"2024-04-17T10:43:33.469189+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.469189+0000","last_undegraded":"2024-04-17T10:43:33.469189+0000","last_fullsized":"2024-04-17T10:43:33.469189+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.493295+0000","last_change":"2024-04-17T10:43:33.488497+0000","last_active":"2024-04-17T10:43:33.493295+0000","last_peered":"2024-04-17T10:43:33.493295+0000","last_clean":"2024-04-17T10:43:33.493295+0000","last_became_active":"2024-04-17T10:43:33.487777+0000","last_became_peered":"2024-04-17T10:43:33.487777+0000","last_unstale":"2024-04-17T10:43:33.493295+0000","last_undegraded":"2024-04-17T10:43:33.493295+0000","last_fullsized":"2024-04-17T10:43:33.493295+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:42:26.108551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.468055+0000","last_change":"2024-04-17T10:43:33.468055+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.468055+0000","last_undegraded":"2024-04-17T10:43:33.468055+0000","last_fullsized":"2024-04-17T10:43:33.468055+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.510713+0000","last_change":"2024-04-17T10:43:33.505100+0000","last_active":"2024-04-17T10:43:33.510713+0000","last_peered":"2024-04-17T10:43:33.510713+0000","last_clean":"2024-04-17T10:43:33.510713+0000","last_became_active":"2024-04-17T10:43:33.504443+0000","last_became_peered":"2024-04-17T10:43:33.504443+0000","last_unstale":"2024-04-17T10:43:33.510713+0000","last_undegraded":"2024-04-17T10:43:33.510713+0000","last_fullsized":"2024-04-17T10:43:33.510713+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:38:49.951085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.479579+0000","last_change":"2024-04-17T10:43:33.479579+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.479579+0000","last_undegraded":"2024-04-17T10:43:33.479579+0000","last_fullsized":"2024-04-17T10:43:33.479579+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.504571+0000","last_change":"2024-04-17T10:43:33.497414+0000","last_active":"2024-04-17T10:43:33.504571+0000","last_peered":"2024-04-17T10:43:33.504571+0000","last_clean":"2024-04-17T10:43:33.504571+0000","last_became_active":"2024-04-17T10:43:33.495764+0000","last_became_peered":"2024-04-17T10:43:33.495764+0000","last_unstale":"2024-04-17T10:43:33.504571+0000","last_undegraded":"2024-04-17T10:43:33.504571+0000","last_fullsized":"2024-04-17T10:43:33.504571+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:02:33.405494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.470619+0000","last_change":"2024-04-17T10:43:33.470619+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.470619+0000","last_undegraded":"2024-04-17T10:43:33.470619+0000","last_fullsized":"2024-04-17T10:43:33.470619+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.511133+0000","last_change":"2024-04-17T10:43:33.509688+0000","last_active":"2024-04-17T10:43:33.511133+0000","last_peered":"2024-04-17T10:43:33.511133+0000","last_clean":"2024-04-17T10:43:33.511133+0000","last_became_active":"2024-04-17T10:43:33.508865+0000","last_became_peered":"2024-04-17T10:43:33.508865+0000","last_unstale":"2024-04-17T10:43:33.511133+0000","last_undegraded":"2024-04-17T10:43:33.511133+0000","last_fullsized":"2024-04-17T10:43:33.511133+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:11:20.457976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.470650+0000","last_change":"2024-04-17T10:43:33.470650+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.470650+0000","last_undegraded":"2024-04-17T10:43:33.470650+0000","last_fullsized":"2024-04-17T10:43:33.470650+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.509053+0000","last_change":"2024-04-17T10:43:33.507008+0000","last_active":"2024-04-17T10:43:33.509053+0000","last_peered":"2024-04-17T10:43:33.509053+0000","last_clean":"2024-04-17T10:43:33.509053+0000","last_became_active":"2024-04-17T10:43:33.506297+0000","last_became_peered":"2024-04-17T10:43:33.506297+0000","last_unstale":"2024-04-17T10:43:33.509053+0000","last_undegraded":"2024-04-17T10:43:33.509053+0000","last_fullsized":"2024-04-17T10:43:33.509053+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:36:45.162219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.468349+0000","last_change":"2024-04-17T10:43:33.468349+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.468349+0000","last_undegraded":"2024-04-17T10:43:33.468349+0000","last_fullsized":"2024-04-17T10:43:33.468349+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.509663+0000","last_change":"2024-04-17T10:43:33.508340+0000","last_active":"2024-04-17T10:43:33.509663+0000","last_peered":"2024-04-17T10:43:33.509663+0000","last_clean":"2024-04-17T10:43:33.509663+0000","last_became_active":"2024-04-17T10:43:33.507561+0000","last_became_peered":"2024-04-17T10:43:33.507561+0000","last_unstale":"2024-04-17T10:43:33.509663+0000","last_undegraded":"2024-04-17T10:43:33.509663+0000","last_fullsized":"2024-04-17T10:43:33.509663+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:17.570356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.472725+0000","last_change":"2024-04-17T10:43:33.472725+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.472725+0000","last_undegraded":"2024-04-17T10:43:33.472725+0000","last_fullsized":"2024-04-17T10:43:33.472725+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.504785+0000","last_change":"2024-04-17T10:43:33.500518+0000","last_active":"2024-04-17T10:43:33.504785+0000","last_peered":"2024-04-17T10:43:33.504785+0000","last_clean":"2024-04-17T10:43:33.504785+0000","last_became_active":"2024-04-17T10:43:33.499044+0000","last_became_peered":"2024-04-17T10:43:33.499044+0000","last_unstale":"2024-04-17T10:43:33.504785+0000","last_undegraded":"2024-04-17T10:43:33.504785+0000","last_fullsized":"2024-04-17T10:43:33.504785+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:45:49.643198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.472216+0000","last_change":"2024-04-17T10:43:33.472216+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.472216+0000","last_undegraded":"2024-04-17T10:43:33.472216+0000","last_fullsized":"2024-04-17T10:43:33.472216+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.494691+0000","last_change":"2024-04-17T10:43:33.492968+0000","last_active":"2024-04-17T10:43:33.494691+0000","last_peered":"2024-04-17T10:43:33.494691+0000","last_clean":"2024-04-17T10:43:33.494691+0000","last_became_active":"2024-04-17T10:43:33.492286+0000","last_became_peered":"2024-04-17T10:43:33.492286+0000","last_unstale":"2024-04-17T10:43:33.494691+0000","last_undegraded":"2024-04-17T10:43:33.494691+0000","last_fullsized":"2024-04-17T10:43:33.494691+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:07:18.944354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.472216+0000","last_change":"2024-04-17T10:43:33.472216+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.472216+0000","last_undegraded":"2024-04-17T10:43:33.472216+0000","last_fullsized":"2024-04-17T10:43:33.472216+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.483373+0000","last_change":"2024-04-17T10:43:33.482758+0000","last_active":"2024-04-17T10:43:33.483373+0000","last_peered":"2024-04-17T10:43:33.483373+0000","last_clean":"2024-04-17T10:43:33.483373+0000","last_became_active":"2024-04-17T10:43:33.482200+0000","last_became_peered":"2024-04-17T10:43:33.482200+0000","last_unstale":"2024-04-17T10:43:33.483373+0000","last_undegraded":"2024-04-17T10:43:33.483373+0000","last_fullsized":"2024-04-17T10:43:33.483373+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:05:45.628231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.468640+0000","last_change":"2024-04-17T10:43:33.468640+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.468640+0000","last_undegraded":"2024-04-17T10:43:33.468640+0000","last_fullsized":"2024-04-17T10:43:33.468640+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.503257+0000","last_change":"2024-04-17T10:43:33.496516+0000","last_active":"2024-04-17T10:43:33.503257+0000","last_peered":"2024-04-17T10:43:33.503257+0000","last_clean":"2024-04-17T10:43:33.503257+0000","last_became_active":"2024-04-17T10:43:33.494818+0000","last_became_peered":"2024-04-17T10:43:33.494818+0000","last_unstale":"2024-04-17T10:43:33.503257+0000","last_undegraded":"2024-04-17T10:43:33.503257+0000","last_fullsized":"2024-04-17T10:43:33.503257+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:51.195275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.472880+0000","last_change":"2024-04-17T10:43:33.472880+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.472880+0000","last_undegraded":"2024-04-17T10:43:33.472880+0000","last_fullsized":"2024-04-17T10:43:33.472880+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.492457+0000","last_change":"2024-04-17T10:43:33.488378+0000","last_active":"2024-04-17T10:43:33.492457+0000","last_peered":"2024-04-17T10:43:33.492457+0000","last_clean":"2024-04-17T10:43:33.492457+0000","last_became_active":"2024-04-17T10:43:33.487622+0000","last_became_peered":"2024-04-17T10:43:33.487622+0000","last_unstale":"2024-04-17T10:43:33.492457+0000","last_undegraded":"2024-04-17T10:43:33.492457+0000","last_fullsized":"2024-04-17T10:43:33.492457+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:54:54.413725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.466584+0000","last_change":"2024-04-17T10:43:33.466584+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.466584+0000","last_undegraded":"2024-04-17T10:43:33.466584+0000","last_fullsized":"2024-04-17T10:43:33.466584+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.482794+0000","last_change":"2024-04-17T10:43:33.480817+0000","last_active":"2024-04-17T10:43:33.482794+0000","last_peered":"2024-04-17T10:43:33.482794+0000","last_clean":"2024-04-17T10:43:33.482794+0000","last_became_active":"2024-04-17T10:43:33.480442+0000","last_became_peered":"2024-04-17T10:43:33.480442+0000","last_unstale":"2024-04-17T10:43:33.482794+0000","last_undegraded":"2024-04-17T10:43:33.482794+0000","last_fullsized":"2024-04-17T10:43:33.482794+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:45:30.015394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.467765+0000","last_change":"2024-04-17T10:43:33.467765+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.467765+0000","last_undegraded":"2024-04-17T10:43:33.467765+0000","last_fullsized":"2024-04-17T10:43:33.467765+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.510755+0000","last_change":"2024-04-17T10:43:33.510303+0000","last_active":"2024-04-17T10:43:33.510755+0000","last_peered":"2024-04-17T10:43:33.510755+0000","last_clean":"2024-04-17T10:43:33.510755+0000","last_became_active":"2024-04-17T10:43:33.509966+0000","last_became_peered":"2024-04-17T10:43:33.509966+0000","last_unstale":"2024-04-17T10:43:33.510755+0000","last_undegraded":"2024-04-17T10:43:33.510755+0000","last_fullsized":"2024-04-17T10:43:33.510755+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:54:54.413725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.468841+0000","last_change":"2024-04-17T10:43:33.468841+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.468841+0000","last_undegraded":"2024-04-17T10:43:33.468841+0000","last_fullsized":"2024-04-17T10:43:33.468841+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.490422+0000","last_change":"2024-04-17T10:43:33.489836+0000","last_active":"2024-04-17T10:43:33.490422+0000","last_peered":"2024-04-17T10:43:33.490422+0000","last_clean":"2024-04-17T10:43:33.490422+0000","last_became_active":"2024-04-17T10:43:33.489438+0000","last_became_peered":"2024-04-17T10:43:33.489438+0000","last_unstale":"2024-04-17T10:43:33.490422+0000","last_undegraded":"2024-04-17T10:43:33.490422+0000","last_fullsized":"2024-04-17T10:43:33.490422+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:35:50.957989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.488610+0000","last_change":"2024-04-17T10:43:33.488610+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.488610+0000","last_undegraded":"2024-04-17T10:43:33.488610+0000","last_fullsized":"2024-04-17T10:43:33.488610+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.504941+0000","last_change":"2024-04-17T10:43:33.503755+0000","last_active":"2024-04-17T10:43:33.504941+0000","last_peered":"2024-04-17T10:43:33.504941+0000","last_clean":"2024-04-17T10:43:33.504941+0000","last_became_active":"2024-04-17T10:43:33.503377+0000","last_became_peered":"2024-04-17T10:43:33.503377+0000","last_unstale":"2024-04-17T10:43:33.504941+0000","last_undegraded":"2024-04-17T10:43:33.504941+0000","last_fullsized":"2024-04-17T10:43:33.504941+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:36:45.162219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.466736+0000","last_change":"2024-04-17T10:43:33.466736+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.466736+0000","last_undegraded":"2024-04-17T10:43:33.466736+0000","last_fullsized":"2024-04-17T10:43:33.466736+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.510946+0000","last_change":"2024-04-17T10:43:33.506065+0000","last_active":"2024-04-17T10:43:33.510946+0000","last_peered":"2024-04-17T10:43:33.510946+0000","last_clean":"2024-04-17T10:43:33.510946+0000","last_became_active":"2024-04-17T10:43:33.504808+0000","last_became_peered":"2024-04-17T10:43:33.504808+0000","last_unstale":"2024-04-17T10:43:33.510946+0000","last_undegraded":"2024-04-17T10:43:33.510946+0000","last_fullsized":"2024-04-17T10:43:33.510946+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:01:00.790513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.476069+0000","last_change":"2024-04-17T10:43:33.476069+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.476069+0000","last_undegraded":"2024-04-17T10:43:33.476069+0000","last_fullsized":"2024-04-17T10:43:33.476069+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.493140+0000","last_change":"2024-04-17T10:43:33.487447+0000","last_active":"2024-04-17T10:43:33.493140+0000","last_peered":"2024-04-17T10:43:33.493140+0000","last_clean":"2024-04-17T10:43:33.493140+0000","last_became_active":"2024-04-17T10:43:33.486690+0000","last_became_peered":"2024-04-17T10:43:33.486690+0000","last_unstale":"2024-04-17T10:43:33.493140+0000","last_undegraded":"2024-04-17T10:43:33.493140+0000","last_fullsized":"2024-04-17T10:43:33.493140+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:44:20.188944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.472418+0000","last_change":"2024-04-17T10:43:33.472418+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.472418+0000","last_undegraded":"2024-04-17T10:43:33.472418+0000","last_fullsized":"2024-04-17T10:43:33.472418+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.497416+0000","last_change":"2024-04-17T10:43:33.495411+0000","last_active":"2024-04-17T10:43:33.497416+0000","last_peered":"2024-04-17T10:43:33.497416+0000","last_clean":"2024-04-17T10:43:33.497416+0000","last_became_active":"2024-04-17T10:43:33.494671+0000","last_became_peered":"2024-04-17T10:43:33.494671+0000","last_unstale":"2024-04-17T10:43:33.497416+0000","last_undegraded":"2024-04-17T10:43:33.497416+0000","last_fullsized":"2024-04-17T10:43:33.497416+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:42:52.241171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":276,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.476774+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:43:33.476774+0000","last_peered":"2024-04-17T10:43:33.476774+0000","last_clean":"2024-04-17T10:43:33.476774+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:43:33.476774+0000","last_undegraded":"2024-04-17T10:43:33.476774+0000","last_fullsized":"2024-04-17T10:43:33.476774+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574919,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28460,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343380,"statfs":{"total":96636764160,"available":96607621120,"internally_reserved":0,"allocated":1470464,"data_stored":962292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40360,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331480,"statfs":{"total":96636764160,"available":96595435520,"internally_reserved":0,"allocated":876544,"data_stored":371924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574919,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40360,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331480,"statfs":{"total":96636764160,"available":96595435520,"internally_reserved":0,"allocated":876544,"data_stored":371924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50084,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321756,"statfs":{"total":96636764160,"available":96585478144,"internally_reserved":0,"allocated":876544,"data_stored":371924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574920,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54248,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317592,"statfs":{"total":96636764160,"available":96581214208,"internally_reserved":0,"allocated":876544,"data_stored":371924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574918,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36200,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335640,"statfs":{"total":96636764160,"available":96599695360,"internally_reserved":0,"allocated":876544,"data_stored":371924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50088,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321752,"statfs":{"total":96636764160,"available":96585474048,"internally_reserved":0,"allocated":876544,"data_stored":371924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574918,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45096,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326744,"statfs":{"total":96636764160,"available":96590585856,"internally_reserved":0,"allocated":1470464,"data_stored":962292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:43:38.830 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:43:38.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.989+0000 7f246c2ec640 1 Processor -- start 2024-04-17T10:43:38.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.989+0000 7f246c2ec640 1 -- start start 2024-04-17T10:43:38.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.993+0000 7f246c2ec640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24640d2730 0x7f24640ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:38.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.993+0000 7f246c2ec640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24640d18c0 con 0x7f246406c750 2024-04-17T10:43:38.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.993+0000 7f246c2ec640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24640d1b60 con 0x7f246415ddd0 2024-04-17T10:43:38.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.993+0000 7f246c2ec640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24640d1e40 con 0x7f24640d2730 2024-04-17T10:43:38.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.993+0000 7f246a889640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f246406c750 0x7f24640ceb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55084/0 (socket says 172.21.15.83:55084) 2024-04-17T10:43:38.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.993+0000 7f246a889640 1 -- 172.21.15.83:0/279790176 learned_addr learned my addr 172.21.15.83:0/279790176 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:43:38.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.993+0000 7f2469887640 1 --2- 172.21.15.83:0/279790176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24640d2730 0x7f24640ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:38.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.993+0000 7f2469086640 1 -- 172.21.15.83:0/279790176 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 500206190 0 0) 0x7f24640d18c0 con 0x7f246406c750 2024-04-17T10:43:38.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.993+0000 7f2469086640 1 -- 172.21.15.83:0/279790176 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2450003530 con 0x7f246406c750 2024-04-17T10:43:38.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.993+0000 7f2469086640 1 -- 172.21.15.83:0/279790176 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 968841626 0 0) 0x7f2450003530 con 0x7f246406c750 2024-04-17T10:43:38.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.993+0000 7f2469086640 1 -- 172.21.15.83:0/279790176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24640d2730 msgr2=0x7f24640ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:38.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.993+0000 7f2469086640 1 --2- 172.21.15.83:0/279790176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24640d2730 0x7f24640ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.993+0000 7f2469086640 1 -- 172.21.15.83:0/279790176 >> v1:172.21.15.167:6789/0 conn(0x7f246415ddd0 legacy=0x7f24640d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:38.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.993+0000 7f2469086640 1 -- 172.21.15.83:0/279790176 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f246406b220 con 0x7f246406c750 2024-04-17T10:43:38.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.993+0000 7f2469086640 1 -- 172.21.15.83:0/279790176 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f245c001a00 con 0x7f246406c750 2024-04-17T10:43:38.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.993+0000 7f2469086640 1 -- 172.21.15.83:0/279790176 >> v1:172.21.15.83:6789/0 conn(0x7f246406c750 legacy=0x7f24640ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:38.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.993+0000 7f2469086640 1 --2- 172.21.15.83:0/279790176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24500039f0 0x7f2450003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:38.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.993+0000 7f2469086640 1 --2- 172.21.15.83:0/279790176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24640d2730 0x7f2450004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:38.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.993+0000 7f2469086640 1 --2- 172.21.15.83:0/279790176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2450004d50 0x7f24500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:38.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.993+0000 7f2469086640 1 -- 172.21.15.83:0/279790176 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24640d1e40 con 0x7f24500039f0 2024-04-17T10:43:38.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.993+0000 7f2469086640 1 -- 172.21.15.83:0/279790176 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245000bb90 con 0x7f2450004d50 2024-04-17T10:43:38.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.993+0000 7f2469086640 1 -- 172.21.15.83:0/279790176 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f245000bd90 con 0x7f24640d2730 2024-04-17T10:43:38.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.993+0000 7f246a088640 1 --2- 172.21.15.83:0/279790176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24500039f0 0x7f2450003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:38.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.993+0000 7f246a889640 1 --2- 172.21.15.83:0/279790176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2450004d50 0x7f24500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:38.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.993+0000 7f2469887640 1 --2- 172.21.15.83:0/279790176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24640d2730 0x7f2450004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:38.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.993+0000 7f246a088640 1 -- 172.21.15.83:0/279790176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24640d2730 msgr2=0x7f2450004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:38.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.993+0000 7f246a088640 1 --2- 172.21.15.83:0/279790176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24640d2730 0x7f2450004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.993+0000 7f246a088640 1 -- 172.21.15.83:0/279790176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2450004d50 msgr2=0x7f24500051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:38.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.993+0000 7f246a088640 1 --2- 172.21.15.83:0/279790176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2450004d50 0x7f24500051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.993+0000 7f246a088640 1 -- 172.21.15.83:0/279790176 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2450003530 con 0x7f24500039f0 2024-04-17T10:43:38.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f246a088640 1 --2- 172.21.15.83:0/279790176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24500039f0 0x7f2450003e10 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f2460001780 tx=0x7f246001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2032c743caa5c856 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:38.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f2469086640 1 -- 172.21.15.83:0/279790176 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24600013a0 con 0x7f24500039f0 2024-04-17T10:43:38.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f2469086640 1 -- 172.21.15.83:0/279790176 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24600052f0 con 0x7f24500039f0 2024-04-17T10:43:38.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f2469086640 1 -- 172.21.15.83:0/279790176 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2460003840 con 0x7f24500039f0 2024-04-17T10:43:38.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f246c2ec640 1 -- 172.21.15.83:0/279790176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24500039f0 msgr2=0x7f2450003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:38.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f246c2ec640 1 --2- 172.21.15.83:0/279790176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24500039f0 0x7f2450003e10 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f2460001780 tx=0x7f246001c3c0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f246a889640 1 -- 172.21.15.83:0/279790176 reap_dead start 2024-04-17T10:43:38.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f246c2ec640 1 -- 172.21.15.83:0/279790176 shutdown_connections 2024-04-17T10:43:38.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f246c2ec640 1 -- 172.21.15.83:0/279790176 >> 172.21.15.83:0/279790176 conn(0x7f24640d3a40 msgr2=0x7f24640d3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:38.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f246c2ec640 1 -- 172.21.15.83:0/279790176 shutdown_connections 2024-04-17T10:43:38.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f246c2ec640 1 -- 172.21.15.83:0/279790176 wait complete. 2024-04-17T10:43:38.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f246c2ec640 1 Processor -- start 2024-04-17T10:43:38.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f246c2ec640 1 -- start start 2024-04-17T10:43:38.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f246c2ec640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24500039f0 0x7f246414cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:38.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f246c2ec640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f246414c7e0 0x7f24640b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:38.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f246c2ec640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f246415aa20 0x7f246415aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:38.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f246c2ec640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2464169bd0 con 0x7f246414c7e0 2024-04-17T10:43:38.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f246c2ec640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2464169a50 con 0x7f246415aa20 2024-04-17T10:43:38.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f246c2ec640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2464169d50 con 0x7f24500039f0 2024-04-17T10:43:38.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f2469887640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f246414c7e0 0x7f24640b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:38.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f2469887640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f246414c7e0 0x7f24640b2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52378/0 (socket says 172.21.15.83:52378) 2024-04-17T10:43:38.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f2469887640 1 -- 172.21.15.83:0/2851600019 learned_addr learned my addr 172.21.15.83:0/2851600019 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:38.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f246a889640 1 --2- 172.21.15.83:0/2851600019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f246415aa20 0x7f246415aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:38.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f246a088640 1 --2- 172.21.15.83:0/2851600019 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24500039f0 0x7f246414cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:38.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f2469887640 1 -- 172.21.15.83:0/2851600019 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24500039f0 msgr2=0x7f246414cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:38.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f2469887640 1 --2- 172.21.15.83:0/2851600019 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24500039f0 0x7f246414cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f2469887640 1 -- 172.21.15.83:0/2851600019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f246415aa20 msgr2=0x7f246415aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:38.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f2469887640 1 --2- 172.21.15.83:0/2851600019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f246415aa20 0x7f246415aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:38.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f2469887640 1 -- 172.21.15.83:0/2851600019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2464150f90 con 0x7f246414c7e0 2024-04-17T10:43:38.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f2469887640 1 --2- 172.21.15.83:0/2851600019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f246414c7e0 0x7f24640b2200 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f24540108f0 tx=0x7f2454010db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:39.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f245affd640 1 -- 172.21.15.83:0/2851600019 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245400f040 con 0x7f246414c7e0 2024-04-17T10:43:39.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f245affd640 1 -- 172.21.15.83:0/2851600019 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f245400eb00 con 0x7f246414c7e0 2024-04-17T10:43:39.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f246c2ec640 1 -- 172.21.15.83:0/2851600019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2464158ec0 con 0x7f246414c7e0 2024-04-17T10:43:39.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f245affd640 1 -- 172.21.15.83:0/2851600019 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245401c400 con 0x7f246414c7e0 2024-04-17T10:43:39.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:38.997+0000 7f246c2ec640 1 -- 172.21.15.83:0/2851600019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2464151230 con 0x7f246414c7e0 2024-04-17T10:43:39.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.001+0000 7f246c2ec640 1 -- 172.21.15.83:0/2851600019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2464074f90 con 0x7f246414c7e0 2024-04-17T10:43:39.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.001+0000 7f245affd640 1 -- 172.21.15.83:0/2851600019 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f245401c5a0 con 0x7f246414c7e0 2024-04-17T10:43:39.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.001+0000 7f245affd640 1 --2- 172.21.15.83:0/2851600019 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2448066f90 0x7f2448069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:39.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.001+0000 7f245affd640 1 -- 172.21.15.83:0/2851600019 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24540855c0 con 0x7f246414c7e0 2024-04-17T10:43:39.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.001+0000 7f246a088640 1 --2- 172.21.15.83:0/2851600019 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2448066f90 0x7f2448069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:39.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.001+0000 7f246a088640 1 --2- 172.21.15.83:0/2851600019 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2448066f90 0x7f2448069450 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2460001750 tx=0x7f24600011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:39.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.005+0000 7f245affd640 1 -- 172.21.15.83:0/2851600019 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2454018030 con 0x7f246414c7e0 2024-04-17T10:43:39.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.145+0000 7f246c2ec640 1 -- 172.21.15.83:0/2851600019 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f246415bbb0 con 0x7f2448066f90 2024-04-17T10:43:39.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.153+0000 7f245affd640 1 -- 172.21.15.83:0/2851600019 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246757 (secure 0 0 0) 0x7f246415bbb0 con 0x7f2448066f90 2024-04-17T10:43:39.151 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:43:39.152 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:43:39.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.157+0000 7f246c2ec640 1 -- 172.21.15.83:0/2851600019 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2448066f90 msgr2=0x7f2448069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:39.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.157+0000 7f246c2ec640 1 --2- 172.21.15.83:0/2851600019 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2448066f90 0x7f2448069450 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2460001750 tx=0x7f24600011a0 comp rx=0 tx=0).stop 2024-04-17T10:43:39.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.157+0000 7f246c2ec640 1 -- 172.21.15.83:0/2851600019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f246414c7e0 msgr2=0x7f24640b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:39.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.157+0000 7f246c2ec640 1 --2- 172.21.15.83:0/2851600019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f246414c7e0 0x7f24640b2200 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f24540108f0 tx=0x7f2454010db0 comp rx=0 tx=0).stop 2024-04-17T10:43:39.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.157+0000 7f246c2ec640 1 -- 172.21.15.83:0/2851600019 shutdown_connections 2024-04-17T10:43:39.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.157+0000 7f246c2ec640 1 --2- 172.21.15.83:0/2851600019 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2448066f90 0x7f2448069450 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:39.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.157+0000 7f246c2ec640 1 --2- 172.21.15.83:0/2851600019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f246415aa20 0x7f246415aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:39.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.157+0000 7f246c2ec640 1 --2- 172.21.15.83:0/2851600019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f246414c7e0 0x7f24640b2200 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:39.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.157+0000 7f246c2ec640 1 --2- 172.21.15.83:0/2851600019 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24500039f0 0x7f246414cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:39.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.157+0000 7f246c2ec640 1 -- 172.21.15.83:0/2851600019 >> 172.21.15.83:0/2851600019 conn(0x7f24640d3a40 msgr2=0x7f2464168610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:39.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.157+0000 7f246c2ec640 1 -- 172.21.15.83:0/2851600019 shutdown_connections 2024-04-17T10:43:39.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.157+0000 7f246c2ec640 1 -- 172.21.15.83:0/2851600019 wait complete. 2024-04-17T10:43:39.167 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2024-04-17T10:43:38.278527+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":38,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":344896,"kb_used_data":8008,"kb_used_omap":12,"kb_used_meta":336755,"kb_avail":754629824,"statfs":{"total":773094113280,"available":772740939776,"internally_reserved":0,"allocated":8200192,"data_stored":4156128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":344837680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.822026"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.475786+0000","last_change":"2024-04-17T10:43:33.475786+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.475786+0000","last_undegraded":"2024-04-17T10:43:33.475786+0000","last_fullsized":"2024-04-17T10:43:33.475786+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.488388+0000","last_change":"2024-04-17T10:43:33.488388+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.488388+0000","last_undegraded":"2024-04-17T10:43:33.488388+0000","last_fullsized":"2024-04-17T10:43:33.488388+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.474093+0000","last_change":"2024-04-17T10:43:33.474093+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.474093+0000","last_undegraded":"2024-04-17T10:43:33.474093+0000","last_fullsized":"2024-04-17T10:43:33.474093+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.475007+0000","last_change":"2024-04-17T10:43:33.475007+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.475007+0000","last_undegraded":"2024-04-17T10:43:33.475007+0000","last_fullsized":"2024-04-17T10:43:33.475007+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.469492+0000","last_change":"2024-04-17T10:43:33.469492+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.469492+0000","last_undegraded":"2024-04-17T10:43:33.469492+0000","last_fullsized":"2024-04-17T10:43:33.469492+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.492867+0000","last_change":"2024-04-17T10:43:33.492867+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.492867+0000","last_undegraded":"2024-04-17T10:43:33.492867+0000","last_fullsized":"2024-04-17T10:43:33.492867+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.475679+0000","last_change":"2024-04-17T10:43:33.475679+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.475679+0000","last_undegraded":"2024-04-17T10:43:33.475679+0000","last_fullsized":"2024-04-17T10:43:33.475679+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.478045+0000","last_change":"2024-04-17T10:43:33.478045+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.478045+0000","last_undegraded":"2024-04-17T10:43:33.478045+0000","last_fullsized":"2024-04-17T10:43:33.478045+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.470779+0000","last_change":"2024-04-17T10:43:33.470779+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.470779+0000","last_undegraded":"2024-04-17T10:43:33.470779+0000","last_fullsized":"2024-04-17T10:43:33.470779+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.480695+0000","last_change":"2024-04-17T10:43:33.480695+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.480695+0000","last_undegraded":"2024-04-17T10:43:33.480695+0000","last_fullsized":"2024-04-17T10:43:33.480695+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.469761+0000","last_change":"2024-04-17T10:43:33.469761+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.469761+0000","last_undegraded":"2024-04-17T10:43:33.469761+0000","last_fullsized":"2024-04-17T10:43:33.469761+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.471328+0000","last_change":"2024-04-17T10:43:33.471328+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.471328+0000","last_undegraded":"2024-04-17T10:43:33.471328+0000","last_fullsized":"2024-04-17T10:43:33.471328+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.468149+0000","last_change":"2024-04-17T10:43:33.468149+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.468149+0000","last_undegraded":"2024-04-17T10:43:33.468149+0000","last_fullsized":"2024-04-17T10:43:33.468149+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.470994+0000","last_change":"2024-04-17T10:43:33.470994+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.470994+0000","last_undegraded":"2024-04-17T10:43:33.470994+0000","last_fullsized":"2024-04-17T10:43:33.470994+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.472752+0000","last_change":"2024-04-17T10:43:33.472752+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.472752+0000","last_undegraded":"2024-04-17T10:43:33.472752+0000","last_fullsized":"2024-04-17T10:43:33.472752+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.479475+0000","last_change":"2024-04-17T10:43:33.479475+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.479475+0000","last_undegraded":"2024-04-17T10:43:33.479475+0000","last_fullsized":"2024-04-17T10:43:33.479475+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.470329+0000","last_change":"2024-04-17T10:43:33.470329+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.470329+0000","last_undegraded":"2024-04-17T10:43:33.470329+0000","last_fullsized":"2024-04-17T10:43:33.470329+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.470663+0000","last_change":"2024-04-17T10:43:33.470663+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.470663+0000","last_undegraded":"2024-04-17T10:43:33.470663+0000","last_fullsized":"2024-04-17T10:43:33.470663+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.491227+0000","last_change":"2024-04-17T10:43:33.491227+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.491227+0000","last_undegraded":"2024-04-17T10:43:33.491227+0000","last_fullsized":"2024-04-17T10:43:33.491227+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.469792+0000","last_change":"2024-04-17T10:43:33.469792+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.469792+0000","last_undegraded":"2024-04-17T10:43:33.469792+0000","last_fullsized":"2024-04-17T10:43:33.469792+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.469177+0000","last_change":"2024-04-17T10:43:33.469177+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.469177+0000","last_undegraded":"2024-04-17T10:43:33.469177+0000","last_fullsized":"2024-04-17T10:43:33.469177+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.477573+0000","last_change":"2024-04-17T10:43:33.477573+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.477573+0000","last_undegraded":"2024-04-17T10:43:33.477573+0000","last_fullsized":"2024-04-17T10:43:33.477573+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.466522+0000","last_change":"2024-04-17T10:43:33.466522+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.466522+0000","last_undegraded":"2024-04-17T10:43:33.466522+0000","last_fullsized":"2024-04-17T10:43:33.466522+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.470617+0000","last_change":"2024-04-17T10:43:33.470617+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.470617+0000","last_undegraded":"2024-04-17T10:43:33.470617+0000","last_fullsized":"2024-04-17T10:43:33.470617+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.466969+0000","last_change":"2024-04-17T10:43:33.466969+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.466969+0000","last_undegraded":"2024-04-17T10:43:33.466969+0000","last_fullsized":"2024-04-17T10:43:33.466969+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.474514+0000","last_change":"2024-04-17T10:43:33.474514+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.474514+0000","last_undegraded":"2024-04-17T10:43:33.474514+0000","last_fullsized":"2024-04-17T10:43:33.474514+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.473429+0000","last_change":"2024-04-17T10:43:33.473429+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.473429+0000","last_undegraded":"2024-04-17T10:43:33.473429+0000","last_fullsized":"2024-04-17T10:43:33.473429+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.467411+0000","last_change":"2024-04-17T10:43:33.467411+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.467411+0000","last_undegraded":"2024-04-17T10:43:33.467411+0000","last_fullsized":"2024-04-17T10:43:33.467411+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.473115+0000","last_change":"2024-04-17T10:43:33.473115+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.473115+0000","last_undegraded":"2024-04-17T10:43:33.473115+0000","last_fullsized":"2024-04-17T10:43:33.473115+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.472797+0000","last_change":"2024-04-17T10:43:33.472797+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.472797+0000","last_undegraded":"2024-04-17T10:43:33.472797+0000","last_fullsized":"2024-04-17T10:43:33.472797+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.473897+0000","last_change":"2024-04-17T10:43:33.473897+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.473897+0000","last_undegraded":"2024-04-17T10:43:33.473897+0000","last_fullsized":"2024-04-17T10:43:33.473897+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.476712+0000","last_change":"2024-04-17T10:43:33.476712+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.476712+0000","last_undegraded":"2024-04-17T10:43:33.476712+0000","last_fullsized":"2024-04-17T10:43:33.476712+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.469503+0000","last_change":"2024-04-17T10:43:33.469503+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.469503+0000","last_undegraded":"2024-04-17T10:43:33.469503+0000","last_fullsized":"2024-04-17T10:43:33.469503+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.497359+0000","last_change":"2024-04-17T10:43:33.488363+0000","last_active":"2024-04-17T10:43:33.497359+0000","last_peered":"2024-04-17T10:43:33.497359+0000","last_clean":"2024-04-17T10:43:33.497359+0000","last_became_active":"2024-04-17T10:43:33.487677+0000","last_became_peered":"2024-04-17T10:43:33.487677+0000","last_unstale":"2024-04-17T10:43:33.497359+0000","last_undegraded":"2024-04-17T10:43:33.497359+0000","last_fullsized":"2024-04-17T10:43:33.497359+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:43:20.629428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.468080+0000","last_change":"2024-04-17T10:43:33.468080+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.468080+0000","last_undegraded":"2024-04-17T10:43:33.468080+0000","last_fullsized":"2024-04-17T10:43:33.468080+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.479669+0000","last_change":"2024-04-17T10:43:33.478164+0000","last_active":"2024-04-17T10:43:33.479669+0000","last_peered":"2024-04-17T10:43:33.479669+0000","last_clean":"2024-04-17T10:43:33.479669+0000","last_became_active":"2024-04-17T10:43:33.477765+0000","last_became_peered":"2024-04-17T10:43:33.477765+0000","last_unstale":"2024-04-17T10:43:33.479669+0000","last_undegraded":"2024-04-17T10:43:33.479669+0000","last_fullsized":"2024-04-17T10:43:33.479669+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:40:08.909983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.486984+0000","last_change":"2024-04-17T10:43:33.486984+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.486984+0000","last_undegraded":"2024-04-17T10:43:33.486984+0000","last_fullsized":"2024-04-17T10:43:33.486984+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.510659+0000","last_change":"2024-04-17T10:43:33.504903+0000","last_active":"2024-04-17T10:43:33.510659+0000","last_peered":"2024-04-17T10:43:33.510659+0000","last_clean":"2024-04-17T10:43:33.510659+0000","last_became_active":"2024-04-17T10:43:33.503926+0000","last_became_peered":"2024-04-17T10:43:33.503926+0000","last_unstale":"2024-04-17T10:43:33.510659+0000","last_undegraded":"2024-04-17T10:43:33.510659+0000","last_fullsized":"2024-04-17T10:43:33.510659+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:33:06.929106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.468718+0000","last_change":"2024-04-17T10:43:33.468718+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.468718+0000","last_undegraded":"2024-04-17T10:43:33.468718+0000","last_fullsized":"2024-04-17T10:43:33.468718+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.496406+0000","last_change":"2024-04-17T10:43:33.491909+0000","last_active":"2024-04-17T10:43:33.496406+0000","last_peered":"2024-04-17T10:43:33.496406+0000","last_clean":"2024-04-17T10:43:33.496406+0000","last_became_active":"2024-04-17T10:43:33.490806+0000","last_became_peered":"2024-04-17T10:43:33.490806+0000","last_unstale":"2024-04-17T10:43:33.496406+0000","last_undegraded":"2024-04-17T10:43:33.496406+0000","last_fullsized":"2024-04-17T10:43:33.496406+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:33:38.078826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.471659+0000","last_change":"2024-04-17T10:43:33.471659+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.471659+0000","last_undegraded":"2024-04-17T10:43:33.471659+0000","last_fullsized":"2024-04-17T10:43:33.471659+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.504597+0000","last_change":"2024-04-17T10:43:33.504035+0000","last_active":"2024-04-17T10:43:33.504597+0000","last_peered":"2024-04-17T10:43:33.504597+0000","last_clean":"2024-04-17T10:43:33.504597+0000","last_became_active":"2024-04-17T10:43:33.503454+0000","last_became_peered":"2024-04-17T10:43:33.503454+0000","last_unstale":"2024-04-17T10:43:33.504597+0000","last_undegraded":"2024-04-17T10:43:33.504597+0000","last_fullsized":"2024-04-17T10:43:33.504597+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:53:36.645485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.467972+0000","last_change":"2024-04-17T10:43:33.467972+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.467972+0000","last_undegraded":"2024-04-17T10:43:33.467972+0000","last_fullsized":"2024-04-17T10:43:33.467972+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.505433+0000","last_change":"2024-04-17T10:43:33.502776+0000","last_active":"2024-04-17T10:43:33.505433+0000","last_peered":"2024-04-17T10:43:33.505433+0000","last_clean":"2024-04-17T10:43:33.505433+0000","last_became_active":"2024-04-17T10:43:33.498687+0000","last_became_peered":"2024-04-17T10:43:33.498687+0000","last_unstale":"2024-04-17T10:43:33.505433+0000","last_undegraded":"2024-04-17T10:43:33.505433+0000","last_fullsized":"2024-04-17T10:43:33.505433+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:45:05.760753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.468424+0000","last_change":"2024-04-17T10:43:33.468424+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.468424+0000","last_undegraded":"2024-04-17T10:43:33.468424+0000","last_fullsized":"2024-04-17T10:43:33.468424+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.506648+0000","last_change":"2024-04-17T10:43:33.506146+0000","last_active":"2024-04-17T10:43:33.506648+0000","last_peered":"2024-04-17T10:43:33.506648+0000","last_clean":"2024-04-17T10:43:33.506648+0000","last_became_active":"2024-04-17T10:43:33.505651+0000","last_became_peered":"2024-04-17T10:43:33.505651+0000","last_unstale":"2024-04-17T10:43:33.506648+0000","last_undegraded":"2024-04-17T10:43:33.506648+0000","last_fullsized":"2024-04-17T10:43:33.506648+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:44:26.965507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.469679+0000","last_change":"2024-04-17T10:43:33.469679+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.469679+0000","last_undegraded":"2024-04-17T10:43:33.469679+0000","last_fullsized":"2024-04-17T10:43:33.469679+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.480916+0000","last_change":"2024-04-17T10:43:33.478569+0000","last_active":"2024-04-17T10:43:33.480916+0000","last_peered":"2024-04-17T10:43:33.480916+0000","last_clean":"2024-04-17T10:43:33.480916+0000","last_became_active":"2024-04-17T10:43:33.477453+0000","last_became_peered":"2024-04-17T10:43:33.477453+0000","last_unstale":"2024-04-17T10:43:33.480916+0000","last_undegraded":"2024-04-17T10:43:33.480916+0000","last_fullsized":"2024-04-17T10:43:33.480916+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:59:17.699632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.473432+0000","last_change":"2024-04-17T10:43:33.473432+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.473432+0000","last_undegraded":"2024-04-17T10:43:33.473432+0000","last_fullsized":"2024-04-17T10:43:33.473432+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.509346+0000","last_change":"2024-04-17T10:43:33.499957+0000","last_active":"2024-04-17T10:43:33.509346+0000","last_peered":"2024-04-17T10:43:33.509346+0000","last_clean":"2024-04-17T10:43:33.509346+0000","last_became_active":"2024-04-17T10:43:33.481936+0000","last_became_peered":"2024-04-17T10:43:33.481936+0000","last_unstale":"2024-04-17T10:43:33.509346+0000","last_undegraded":"2024-04-17T10:43:33.509346+0000","last_fullsized":"2024-04-17T10:43:33.509346+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:34:00.759922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.474388+0000","last_change":"2024-04-17T10:43:33.474388+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.474388+0000","last_undegraded":"2024-04-17T10:43:33.474388+0000","last_fullsized":"2024-04-17T10:43:33.474388+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.497306+0000","last_change":"2024-04-17T10:43:33.494714+0000","last_active":"2024-04-17T10:43:33.497306+0000","last_peered":"2024-04-17T10:43:33.497306+0000","last_clean":"2024-04-17T10:43:33.497306+0000","last_became_active":"2024-04-17T10:43:33.493921+0000","last_became_peered":"2024-04-17T10:43:33.493921+0000","last_unstale":"2024-04-17T10:43:33.497306+0000","last_undegraded":"2024-04-17T10:43:33.497306+0000","last_fullsized":"2024-04-17T10:43:33.497306+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:34:40.985936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.483201+0000","last_change":"2024-04-17T10:43:33.483201+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.483201+0000","last_undegraded":"2024-04-17T10:43:33.483201+0000","last_fullsized":"2024-04-17T10:43:33.483201+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.493416+0000","last_change":"2024-04-17T10:43:33.492746+0000","last_active":"2024-04-17T10:43:33.493416+0000","last_peered":"2024-04-17T10:43:33.493416+0000","last_clean":"2024-04-17T10:43:33.493416+0000","last_became_active":"2024-04-17T10:43:33.492349+0000","last_became_peered":"2024-04-17T10:43:33.492349+0000","last_unstale":"2024-04-17T10:43:33.493416+0000","last_undegraded":"2024-04-17T10:43:33.493416+0000","last_fullsized":"2024-04-17T10:43:33.493416+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:37:09.710132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.471311+0000","last_change":"2024-04-17T10:43:33.471311+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.471311+0000","last_undegraded":"2024-04-17T10:43:33.471311+0000","last_fullsized":"2024-04-17T10:43:33.471311+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.480245+0000","last_change":"2024-04-17T10:43:33.476458+0000","last_active":"2024-04-17T10:43:33.480245+0000","last_peered":"2024-04-17T10:43:33.480245+0000","last_clean":"2024-04-17T10:43:33.480245+0000","last_became_active":"2024-04-17T10:43:33.475608+0000","last_became_peered":"2024-04-17T10:43:33.475608+0000","last_unstale":"2024-04-17T10:43:33.480245+0000","last_undegraded":"2024-04-17T10:43:33.480245+0000","last_fullsized":"2024-04-17T10:43:33.480245+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:45:49.643198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.469551+0000","last_change":"2024-04-17T10:43:33.469551+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.469551+0000","last_undegraded":"2024-04-17T10:43:33.469551+0000","last_fullsized":"2024-04-17T10:43:33.469551+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.481177+0000","last_change":"2024-04-17T10:43:33.479258+0000","last_active":"2024-04-17T10:43:33.481177+0000","last_peered":"2024-04-17T10:43:33.481177+0000","last_clean":"2024-04-17T10:43:33.481177+0000","last_became_active":"2024-04-17T10:43:33.478798+0000","last_became_peered":"2024-04-17T10:43:33.478798+0000","last_unstale":"2024-04-17T10:43:33.481177+0000","last_undegraded":"2024-04-17T10:43:33.481177+0000","last_fullsized":"2024-04-17T10:43:33.481177+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:45:05.760753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.471845+0000","last_change":"2024-04-17T10:43:33.471845+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.471845+0000","last_undegraded":"2024-04-17T10:43:33.471845+0000","last_fullsized":"2024-04-17T10:43:33.471845+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.504738+0000","last_change":"2024-04-17T10:43:33.499759+0000","last_active":"2024-04-17T10:43:33.504738+0000","last_peered":"2024-04-17T10:43:33.504738+0000","last_clean":"2024-04-17T10:43:33.504738+0000","last_became_active":"2024-04-17T10:43:33.498138+0000","last_became_peered":"2024-04-17T10:43:33.498138+0000","last_unstale":"2024-04-17T10:43:33.504738+0000","last_undegraded":"2024-04-17T10:43:33.504738+0000","last_fullsized":"2024-04-17T10:43:33.504738+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:26:58.859542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.469189+0000","last_change":"2024-04-17T10:43:33.469189+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.469189+0000","last_undegraded":"2024-04-17T10:43:33.469189+0000","last_fullsized":"2024-04-17T10:43:33.469189+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.493295+0000","last_change":"2024-04-17T10:43:33.488497+0000","last_active":"2024-04-17T10:43:33.493295+0000","last_peered":"2024-04-17T10:43:33.493295+0000","last_clean":"2024-04-17T10:43:33.493295+0000","last_became_active":"2024-04-17T10:43:33.487777+0000","last_became_peered":"2024-04-17T10:43:33.487777+0000","last_unstale":"2024-04-17T10:43:33.493295+0000","last_undegraded":"2024-04-17T10:43:33.493295+0000","last_fullsized":"2024-04-17T10:43:33.493295+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:42:26.108551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.468055+0000","last_change":"2024-04-17T10:43:33.468055+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.468055+0000","last_undegraded":"2024-04-17T10:43:33.468055+0000","last_fullsized":"2024-04-17T10:43:33.468055+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.510713+0000","last_change":"2024-04-17T10:43:33.505100+0000","last_active":"2024-04-17T10:43:33.510713+0000","last_peered":"2024-04-17T10:43:33.510713+0000","last_clean":"2024-04-17T10:43:33.510713+0000","last_became_active":"2024-04-17T10:43:33.504443+0000","last_became_peered":"2024-04-17T10:43:33.504443+0000","last_unstale":"2024-04-17T10:43:33.510713+0000","last_undegraded":"2024-04-17T10:43:33.510713+0000","last_fullsized":"2024-04-17T10:43:33.510713+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:38:49.951085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.479579+0000","last_change":"2024-04-17T10:43:33.479579+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.479579+0000","last_undegraded":"2024-04-17T10:43:33.479579+0000","last_fullsized":"2024-04-17T10:43:33.479579+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.504571+0000","last_change":"2024-04-17T10:43:33.497414+0000","last_active":"2024-04-17T10:43:33.504571+0000","last_peered":"2024-04-17T10:43:33.504571+0000","last_clean":"2024-04-17T10:43:33.504571+0000","last_became_active":"2024-04-17T10:43:33.495764+0000","last_became_peered":"2024-04-17T10:43:33.495764+0000","last_unstale":"2024-04-17T10:43:33.504571+0000","last_undegraded":"2024-04-17T10:43:33.504571+0000","last_fullsized":"2024-04-17T10:43:33.504571+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:02:33.405494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.470619+0000","last_change":"2024-04-17T10:43:33.470619+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.470619+0000","last_undegraded":"2024-04-17T10:43:33.470619+0000","last_fullsized":"2024-04-17T10:43:33.470619+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.511133+0000","last_change":"2024-04-17T10:43:33.509688+0000","last_active":"2024-04-17T10:43:33.511133+0000","last_peered":"2024-04-17T10:43:33.511133+0000","last_clean":"2024-04-17T10:43:33.511133+0000","last_became_active":"2024-04-17T10:43:33.508865+0000","last_became_peered":"2024-04-17T10:43:33.508865+0000","last_unstale":"2024-04-17T10:43:33.511133+0000","last_undegraded":"2024-04-17T10:43:33.511133+0000","last_fullsized":"2024-04-17T10:43:33.511133+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:11:20.457976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.470650+0000","last_change":"2024-04-17T10:43:33.470650+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.470650+0000","last_undegraded":"2024-04-17T10:43:33.470650+0000","last_fullsized":"2024-04-17T10:43:33.470650+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.509053+0000","last_change":"2024-04-17T10:43:33.507008+0000","last_active":"2024-04-17T10:43:33.509053+0000","last_peered":"2024-04-17T10:43:33.509053+0000","last_clean":"2024-04-17T10:43:33.509053+0000","last_became_active":"2024-04-17T10:43:33.506297+0000","last_became_peered":"2024-04-17T10:43:33.506297+0000","last_unstale":"2024-04-17T10:43:33.509053+0000","last_undegraded":"2024-04-17T10:43:33.509053+0000","last_fullsized":"2024-04-17T10:43:33.509053+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:36:45.162219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.468349+0000","last_change":"2024-04-17T10:43:33.468349+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.468349+0000","last_undegraded":"2024-04-17T10:43:33.468349+0000","last_fullsized":"2024-04-17T10:43:33.468349+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.509663+0000","last_change":"2024-04-17T10:43:33.508340+0000","last_active":"2024-04-17T10:43:33.509663+0000","last_peered":"2024-04-17T10:43:33.509663+0000","last_clean":"2024-04-17T10:43:33.509663+0000","last_became_active":"2024-04-17T10:43:33.507561+0000","last_became_peered":"2024-04-17T10:43:33.507561+0000","last_unstale":"2024-04-17T10:43:33.509663+0000","last_undegraded":"2024-04-17T10:43:33.509663+0000","last_fullsized":"2024-04-17T10:43:33.509663+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:17.570356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.472725+0000","last_change":"2024-04-17T10:43:33.472725+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.472725+0000","last_undegraded":"2024-04-17T10:43:33.472725+0000","last_fullsized":"2024-04-17T10:43:33.472725+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.504785+0000","last_change":"2024-04-17T10:43:33.500518+0000","last_active":"2024-04-17T10:43:33.504785+0000","last_peered":"2024-04-17T10:43:33.504785+0000","last_clean":"2024-04-17T10:43:33.504785+0000","last_became_active":"2024-04-17T10:43:33.499044+0000","last_became_peered":"2024-04-17T10:43:33.499044+0000","last_unstale":"2024-04-17T10:43:33.504785+0000","last_undegraded":"2024-04-17T10:43:33.504785+0000","last_fullsized":"2024-04-17T10:43:33.504785+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:45:49.643198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.472216+0000","last_change":"2024-04-17T10:43:33.472216+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.472216+0000","last_undegraded":"2024-04-17T10:43:33.472216+0000","last_fullsized":"2024-04-17T10:43:33.472216+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.494691+0000","last_change":"2024-04-17T10:43:33.492968+0000","last_active":"2024-04-17T10:43:33.494691+0000","last_peered":"2024-04-17T10:43:33.494691+0000","last_clean":"2024-04-17T10:43:33.494691+0000","last_became_active":"2024-04-17T10:43:33.492286+0000","last_became_peered":"2024-04-17T10:43:33.492286+0000","last_unstale":"2024-04-17T10:43:33.494691+0000","last_undegraded":"2024-04-17T10:43:33.494691+0000","last_fullsized":"2024-04-17T10:43:33.494691+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:07:18.944354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.472216+0000","last_change":"2024-04-17T10:43:33.472216+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.472216+0000","last_undegraded":"2024-04-17T10:43:33.472216+0000","last_fullsized":"2024-04-17T10:43:33.472216+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.483373+0000","last_change":"2024-04-17T10:43:33.482758+0000","last_active":"2024-04-17T10:43:33.483373+0000","last_peered":"2024-04-17T10:43:33.483373+0000","last_clean":"2024-04-17T10:43:33.483373+0000","last_became_active":"2024-04-17T10:43:33.482200+0000","last_became_peered":"2024-04-17T10:43:33.482200+0000","last_unstale":"2024-04-17T10:43:33.483373+0000","last_undegraded":"2024-04-17T10:43:33.483373+0000","last_fullsized":"2024-04-17T10:43:33.483373+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:05:45.628231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.468640+0000","last_change":"2024-04-17T10:43:33.468640+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.468640+0000","last_undegraded":"2024-04-17T10:43:33.468640+0000","last_fullsized":"2024-04-17T10:43:33.468640+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.503257+0000","last_change":"2024-04-17T10:43:33.496516+0000","last_active":"2024-04-17T10:43:33.503257+0000","last_peered":"2024-04-17T10:43:33.503257+0000","last_clean":"2024-04-17T10:43:33.503257+0000","last_became_active":"2024-04-17T10:43:33.494818+0000","last_became_peered":"2024-04-17T10:43:33.494818+0000","last_unstale":"2024-04-17T10:43:33.503257+0000","last_undegraded":"2024-04-17T10:43:33.503257+0000","last_fullsized":"2024-04-17T10:43:33.503257+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:51.195275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.472880+0000","last_change":"2024-04-17T10:43:33.472880+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.472880+0000","last_undegraded":"2024-04-17T10:43:33.472880+0000","last_fullsized":"2024-04-17T10:43:33.472880+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.492457+0000","last_change":"2024-04-17T10:43:33.488378+0000","last_active":"2024-04-17T10:43:33.492457+0000","last_peered":"2024-04-17T10:43:33.492457+0000","last_clean":"2024-04-17T10:43:33.492457+0000","last_became_active":"2024-04-17T10:43:33.487622+0000","last_became_peered":"2024-04-17T10:43:33.487622+0000","last_unstale":"2024-04-17T10:43:33.492457+0000","last_undegraded":"2024-04-17T10:43:33.492457+0000","last_fullsized":"2024-04-17T10:43:33.492457+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:54:54.413725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.466584+0000","last_change":"2024-04-17T10:43:33.466584+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.466584+0000","last_undegraded":"2024-04-17T10:43:33.466584+0000","last_fullsized":"2024-04-17T10:43:33.466584+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.482794+0000","last_change":"2024-04-17T10:43:33.480817+0000","last_active":"2024-04-17T10:43:33.482794+0000","last_peered":"2024-04-17T10:43:33.482794+0000","last_clean":"2024-04-17T10:43:33.482794+0000","last_became_active":"2024-04-17T10:43:33.480442+0000","last_became_peered":"2024-04-17T10:43:33.480442+0000","last_unstale":"2024-04-17T10:43:33.482794+0000","last_undegraded":"2024-04-17T10:43:33.482794+0000","last_fullsized":"2024-04-17T10:43:33.482794+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:45:30.015394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.467765+0000","last_change":"2024-04-17T10:43:33.467765+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.467765+0000","last_undegraded":"2024-04-17T10:43:33.467765+0000","last_fullsized":"2024-04-17T10:43:33.467765+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.510755+0000","last_change":"2024-04-17T10:43:33.510303+0000","last_active":"2024-04-17T10:43:33.510755+0000","last_peered":"2024-04-17T10:43:33.510755+0000","last_clean":"2024-04-17T10:43:33.510755+0000","last_became_active":"2024-04-17T10:43:33.509966+0000","last_became_peered":"2024-04-17T10:43:33.509966+0000","last_unstale":"2024-04-17T10:43:33.510755+0000","last_undegraded":"2024-04-17T10:43:33.510755+0000","last_fullsized":"2024-04-17T10:43:33.510755+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:54:54.413725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.468841+0000","last_change":"2024-04-17T10:43:33.468841+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.468841+0000","last_undegraded":"2024-04-17T10:43:33.468841+0000","last_fullsized":"2024-04-17T10:43:33.468841+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.490422+0000","last_change":"2024-04-17T10:43:33.489836+0000","last_active":"2024-04-17T10:43:33.490422+0000","last_peered":"2024-04-17T10:43:33.490422+0000","last_clean":"2024-04-17T10:43:33.490422+0000","last_became_active":"2024-04-17T10:43:33.489438+0000","last_became_peered":"2024-04-17T10:43:33.489438+0000","last_unstale":"2024-04-17T10:43:33.490422+0000","last_undegraded":"2024-04-17T10:43:33.490422+0000","last_fullsized":"2024-04-17T10:43:33.490422+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:35:50.957989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.488610+0000","last_change":"2024-04-17T10:43:33.488610+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.488610+0000","last_undegraded":"2024-04-17T10:43:33.488610+0000","last_fullsized":"2024-04-17T10:43:33.488610+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.504941+0000","last_change":"2024-04-17T10:43:33.503755+0000","last_active":"2024-04-17T10:43:33.504941+0000","last_peered":"2024-04-17T10:43:33.504941+0000","last_clean":"2024-04-17T10:43:33.504941+0000","last_became_active":"2024-04-17T10:43:33.503377+0000","last_became_peered":"2024-04-17T10:43:33.503377+0000","last_unstale":"2024-04-17T10:43:33.504941+0000","last_undegraded":"2024-04-17T10:43:33.504941+0000","last_fullsized":"2024-04-17T10:43:33.504941+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:36:45.162219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.466736+0000","last_change":"2024-04-17T10:43:33.466736+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.466736+0000","last_undegraded":"2024-04-17T10:43:33.466736+0000","last_fullsized":"2024-04-17T10:43:33.466736+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.510946+0000","last_change":"2024-04-17T10:43:33.506065+0000","last_active":"2024-04-17T10:43:33.510946+0000","last_peered":"2024-04-17T10:43:33.510946+0000","last_clean":"2024-04-17T10:43:33.510946+0000","last_became_active":"2024-04-17T10:43:33.504808+0000","last_became_peered":"2024-04-17T10:43:33.504808+0000","last_unstale":"2024-04-17T10:43:33.510946+0000","last_undegraded":"2024-04-17T10:43:33.510946+0000","last_fullsized":"2024-04-17T10:43:33.510946+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:01:00.790513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.476069+0000","last_change":"2024-04-17T10:43:33.476069+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.476069+0000","last_undegraded":"2024-04-17T10:43:33.476069+0000","last_fullsized":"2024-04-17T10:43:33.476069+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.493140+0000","last_change":"2024-04-17T10:43:33.487447+0000","last_active":"2024-04-17T10:43:33.493140+0000","last_peered":"2024-04-17T10:43:33.493140+0000","last_clean":"2024-04-17T10:43:33.493140+0000","last_became_active":"2024-04-17T10:43:33.486690+0000","last_became_peered":"2024-04-17T10:43:33.486690+0000","last_unstale":"2024-04-17T10:43:33.493140+0000","last_undegraded":"2024-04-17T10:43:33.493140+0000","last_fullsized":"2024-04-17T10:43:33.493140+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:44:20.188944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.472418+0000","last_change":"2024-04-17T10:43:33.472418+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.472418+0000","last_undegraded":"2024-04-17T10:43:33.472418+0000","last_fullsized":"2024-04-17T10:43:33.472418+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.497416+0000","last_change":"2024-04-17T10:43:33.495411+0000","last_active":"2024-04-17T10:43:33.497416+0000","last_peered":"2024-04-17T10:43:33.497416+0000","last_clean":"2024-04-17T10:43:33.497416+0000","last_became_active":"2024-04-17T10:43:33.494671+0000","last_became_peered":"2024-04-17T10:43:33.494671+0000","last_unstale":"2024-04-17T10:43:33.497416+0000","last_undegraded":"2024-04-17T10:43:33.497416+0000","last_fullsized":"2024-04-17T10:43:33.497416+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:42:52.241171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":276,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.476774+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:43:33.476774+0000","last_peered":"2024-04-17T10:43:33.476774+0000","last_clean":"2024-04-17T10:43:33.476774+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:43:33.476774+0000","last_undegraded":"2024-04-17T10:43:33.476774+0000","last_fullsized":"2024-04-17T10:43:33.476774+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574919,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28460,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343380,"statfs":{"total":96636764160,"available":96607621120,"internally_reserved":0,"allocated":1470464,"data_stored":962292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40360,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331480,"statfs":{"total":96636764160,"available":96595435520,"internally_reserved":0,"allocated":876544,"data_stored":371924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574919,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40360,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331480,"statfs":{"total":96636764160,"available":96595435520,"internally_reserved":0,"allocated":876544,"data_stored":371924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50084,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321756,"statfs":{"total":96636764160,"available":96585478144,"internally_reserved":0,"allocated":876544,"data_stored":371924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574920,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54248,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317592,"statfs":{"total":96636764160,"available":96581214208,"internally_reserved":0,"allocated":876544,"data_stored":371924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574918,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36200,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335640,"statfs":{"total":96636764160,"available":96599695360,"internally_reserved":0,"allocated":876544,"data_stored":371924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50088,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321752,"statfs":{"total":96636764160,"available":96585474048,"internally_reserved":0,"allocated":876544,"data_stored":371924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574918,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45096,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326744,"statfs":{"total":96636764160,"available":96590585856,"internally_reserved":0,"allocated":1470464,"data_stored":962292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:43:39.171 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3f is not active+clean 2024-04-17T10:43:39.171 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.475786+0000', 'last_change': '2024-04-17T10:43:33.475786+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.475786+0000', 'last_undegraded': '2024-04-17T10:43:33.475786+0000', 'last_fullsized': '2024-04-17T10:43:33.475786+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:43:39.171 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3e is not active+clean 2024-04-17T10:43:39.171 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.488388+0000', 'last_change': '2024-04-17T10:43:33.488388+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.488388+0000', 'last_undegraded': '2024-04-17T10:43:33.488388+0000', 'last_fullsized': '2024-04-17T10:43:33.488388+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:43:39.172 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3d is not active+clean 2024-04-17T10:43:39.172 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.474093+0000', 'last_change': '2024-04-17T10:43:33.474093+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.474093+0000', 'last_undegraded': '2024-04-17T10:43:33.474093+0000', 'last_fullsized': '2024-04-17T10:43:33.474093+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:43:39.172 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3c is not active+clean 2024-04-17T10:43:39.172 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.475007+0000', 'last_change': '2024-04-17T10:43:33.475007+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.475007+0000', 'last_undegraded': '2024-04-17T10:43:33.475007+0000', 'last_fullsized': '2024-04-17T10:43:33.475007+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:43:39.172 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3b is not active+clean 2024-04-17T10:43:39.172 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.469492+0000', 'last_change': '2024-04-17T10:43:33.469492+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.469492+0000', 'last_undegraded': '2024-04-17T10:43:33.469492+0000', 'last_fullsized': '2024-04-17T10:43:33.469492+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:43:39.172 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3a is not active+clean 2024-04-17T10:43:39.172 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.492867+0000', 'last_change': '2024-04-17T10:43:33.492867+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.492867+0000', 'last_undegraded': '2024-04-17T10:43:33.492867+0000', 'last_fullsized': '2024-04-17T10:43:33.492867+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:43:39.172 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.39 is not active+clean 2024-04-17T10:43:39.172 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.475679+0000', 'last_change': '2024-04-17T10:43:33.475679+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.475679+0000', 'last_undegraded': '2024-04-17T10:43:33.475679+0000', 'last_fullsized': '2024-04-17T10:43:33.475679+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:43:39.173 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.38 is not active+clean 2024-04-17T10:43:39.173 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.478045+0000', 'last_change': '2024-04-17T10:43:33.478045+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.478045+0000', 'last_undegraded': '2024-04-17T10:43:33.478045+0000', 'last_fullsized': '2024-04-17T10:43:33.478045+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:43:39.173 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.37 is not active+clean 2024-04-17T10:43:39.173 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.470779+0000', 'last_change': '2024-04-17T10:43:33.470779+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.470779+0000', 'last_undegraded': '2024-04-17T10:43:33.470779+0000', 'last_fullsized': '2024-04-17T10:43:33.470779+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:43:39.173 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.36 is not active+clean 2024-04-17T10:43:39.173 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.480695+0000', 'last_change': '2024-04-17T10:43:33.480695+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.480695+0000', 'last_undegraded': '2024-04-17T10:43:33.480695+0000', 'last_fullsized': '2024-04-17T10:43:33.480695+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:43:39.173 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.35 is not active+clean 2024-04-17T10:43:39.173 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.469761+0000', 'last_change': '2024-04-17T10:43:33.469761+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.469761+0000', 'last_undegraded': '2024-04-17T10:43:33.469761+0000', 'last_fullsized': '2024-04-17T10:43:33.469761+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:43:39.173 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.34 is not active+clean 2024-04-17T10:43:39.173 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.471328+0000', 'last_change': '2024-04-17T10:43:33.471328+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.471328+0000', 'last_undegraded': '2024-04-17T10:43:33.471328+0000', 'last_fullsized': '2024-04-17T10:43:33.471328+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:43:39.173 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.33 is not active+clean 2024-04-17T10:43:39.174 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.468149+0000', 'last_change': '2024-04-17T10:43:33.468149+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.468149+0000', 'last_undegraded': '2024-04-17T10:43:33.468149+0000', 'last_fullsized': '2024-04-17T10:43:33.468149+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:43:39.174 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.32 is not active+clean 2024-04-17T10:43:39.174 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.470994+0000', 'last_change': '2024-04-17T10:43:33.470994+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.470994+0000', 'last_undegraded': '2024-04-17T10:43:33.470994+0000', 'last_fullsized': '2024-04-17T10:43:33.470994+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:43:39.174 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.31 is not active+clean 2024-04-17T10:43:39.174 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.472752+0000', 'last_change': '2024-04-17T10:43:33.472752+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.472752+0000', 'last_undegraded': '2024-04-17T10:43:33.472752+0000', 'last_fullsized': '2024-04-17T10:43:33.472752+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:43:39.174 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.30 is not active+clean 2024-04-17T10:43:39.174 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.479475+0000', 'last_change': '2024-04-17T10:43:33.479475+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.479475+0000', 'last_undegraded': '2024-04-17T10:43:33.479475+0000', 'last_fullsized': '2024-04-17T10:43:33.479475+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:43:39.174 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2f is not active+clean 2024-04-17T10:43:39.174 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.470329+0000', 'last_change': '2024-04-17T10:43:33.470329+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.470329+0000', 'last_undegraded': '2024-04-17T10:43:33.470329+0000', 'last_fullsized': '2024-04-17T10:43:33.470329+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:43:39.175 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2e is not active+clean 2024-04-17T10:43:39.175 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.470663+0000', 'last_change': '2024-04-17T10:43:33.470663+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.470663+0000', 'last_undegraded': '2024-04-17T10:43:33.470663+0000', 'last_fullsized': '2024-04-17T10:43:33.470663+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:43:39.175 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2d is not active+clean 2024-04-17T10:43:39.175 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.491227+0000', 'last_change': '2024-04-17T10:43:33.491227+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.491227+0000', 'last_undegraded': '2024-04-17T10:43:33.491227+0000', 'last_fullsized': '2024-04-17T10:43:33.491227+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:43:39.175 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2c is not active+clean 2024-04-17T10:43:39.175 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.469792+0000', 'last_change': '2024-04-17T10:43:33.469792+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.469792+0000', 'last_undegraded': '2024-04-17T10:43:33.469792+0000', 'last_fullsized': '2024-04-17T10:43:33.469792+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:43:39.175 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2b is not active+clean 2024-04-17T10:43:39.175 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.469177+0000', 'last_change': '2024-04-17T10:43:33.469177+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.469177+0000', 'last_undegraded': '2024-04-17T10:43:33.469177+0000', 'last_fullsized': '2024-04-17T10:43:33.469177+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:43:39.175 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2a is not active+clean 2024-04-17T10:43:39.175 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.477573+0000', 'last_change': '2024-04-17T10:43:33.477573+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.477573+0000', 'last_undegraded': '2024-04-17T10:43:33.477573+0000', 'last_fullsized': '2024-04-17T10:43:33.477573+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:43:39.175 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.29 is not active+clean 2024-04-17T10:43:39.176 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.466522+0000', 'last_change': '2024-04-17T10:43:33.466522+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.466522+0000', 'last_undegraded': '2024-04-17T10:43:33.466522+0000', 'last_fullsized': '2024-04-17T10:43:33.466522+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:43:39.176 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.28 is not active+clean 2024-04-17T10:43:39.176 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.470617+0000', 'last_change': '2024-04-17T10:43:33.470617+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.470617+0000', 'last_undegraded': '2024-04-17T10:43:33.470617+0000', 'last_fullsized': '2024-04-17T10:43:33.470617+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:43:39.176 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.27 is not active+clean 2024-04-17T10:43:39.176 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.466969+0000', 'last_change': '2024-04-17T10:43:33.466969+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.466969+0000', 'last_undegraded': '2024-04-17T10:43:33.466969+0000', 'last_fullsized': '2024-04-17T10:43:33.466969+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:43:39.176 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.26 is not active+clean 2024-04-17T10:43:39.176 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.474514+0000', 'last_change': '2024-04-17T10:43:33.474514+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.474514+0000', 'last_undegraded': '2024-04-17T10:43:33.474514+0000', 'last_fullsized': '2024-04-17T10:43:33.474514+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:43:39.176 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.25 is not active+clean 2024-04-17T10:43:39.176 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.473429+0000', 'last_change': '2024-04-17T10:43:33.473429+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.473429+0000', 'last_undegraded': '2024-04-17T10:43:33.473429+0000', 'last_fullsized': '2024-04-17T10:43:33.473429+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:43:39.176 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.24 is not active+clean 2024-04-17T10:43:39.177 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.467411+0000', 'last_change': '2024-04-17T10:43:33.467411+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.467411+0000', 'last_undegraded': '2024-04-17T10:43:33.467411+0000', 'last_fullsized': '2024-04-17T10:43:33.467411+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:43:39.177 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.23 is not active+clean 2024-04-17T10:43:39.177 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.473115+0000', 'last_change': '2024-04-17T10:43:33.473115+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.473115+0000', 'last_undegraded': '2024-04-17T10:43:33.473115+0000', 'last_fullsized': '2024-04-17T10:43:33.473115+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:43:39.177 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.22 is not active+clean 2024-04-17T10:43:39.177 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.472797+0000', 'last_change': '2024-04-17T10:43:33.472797+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.472797+0000', 'last_undegraded': '2024-04-17T10:43:33.472797+0000', 'last_fullsized': '2024-04-17T10:43:33.472797+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:43:39.177 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.21 is not active+clean 2024-04-17T10:43:39.177 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.473897+0000', 'last_change': '2024-04-17T10:43:33.473897+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.473897+0000', 'last_undegraded': '2024-04-17T10:43:33.473897+0000', 'last_fullsized': '2024-04-17T10:43:33.473897+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:43:39.177 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.20 is not active+clean 2024-04-17T10:43:39.177 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.476712+0000', 'last_change': '2024-04-17T10:43:33.476712+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.476712+0000', 'last_undegraded': '2024-04-17T10:43:33.476712+0000', 'last_fullsized': '2024-04-17T10:43:33.476712+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:43:39.177 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1e is not active+clean 2024-04-17T10:43:39.177 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.469503+0000', 'last_change': '2024-04-17T10:43:33.469503+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.469503+0000', 'last_undegraded': '2024-04-17T10:43:33.469503+0000', 'last_fullsized': '2024-04-17T10:43:33.469503+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:43:39.178 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1f is not active+clean 2024-04-17T10:43:39.178 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.468080+0000', 'last_change': '2024-04-17T10:43:33.468080+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.468080+0000', 'last_undegraded': '2024-04-17T10:43:33.468080+0000', 'last_fullsized': '2024-04-17T10:43:33.468080+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:43:39.178 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1c is not active+clean 2024-04-17T10:43:39.178 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.486984+0000', 'last_change': '2024-04-17T10:43:33.486984+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.486984+0000', 'last_undegraded': '2024-04-17T10:43:33.486984+0000', 'last_fullsized': '2024-04-17T10:43:33.486984+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:43:39.178 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1d is not active+clean 2024-04-17T10:43:39.178 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.468718+0000', 'last_change': '2024-04-17T10:43:33.468718+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.468718+0000', 'last_undegraded': '2024-04-17T10:43:33.468718+0000', 'last_fullsized': '2024-04-17T10:43:33.468718+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:43:39.178 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1a is not active+clean 2024-04-17T10:43:39.178 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.471659+0000', 'last_change': '2024-04-17T10:43:33.471659+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.471659+0000', 'last_undegraded': '2024-04-17T10:43:33.471659+0000', 'last_fullsized': '2024-04-17T10:43:33.471659+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:43:39.178 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1b is not active+clean 2024-04-17T10:43:39.178 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.467972+0000', 'last_change': '2024-04-17T10:43:33.467972+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.467972+0000', 'last_undegraded': '2024-04-17T10:43:33.467972+0000', 'last_fullsized': '2024-04-17T10:43:33.467972+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:43:39.179 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.18 is not active+clean 2024-04-17T10:43:39.179 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.468424+0000', 'last_change': '2024-04-17T10:43:33.468424+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.468424+0000', 'last_undegraded': '2024-04-17T10:43:33.468424+0000', 'last_fullsized': '2024-04-17T10:43:33.468424+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:43:39.179 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.19 is not active+clean 2024-04-17T10:43:39.179 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.469679+0000', 'last_change': '2024-04-17T10:43:33.469679+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.469679+0000', 'last_undegraded': '2024-04-17T10:43:33.469679+0000', 'last_fullsized': '2024-04-17T10:43:33.469679+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:43:39.179 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.16 is not active+clean 2024-04-17T10:43:39.179 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.473432+0000', 'last_change': '2024-04-17T10:43:33.473432+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.473432+0000', 'last_undegraded': '2024-04-17T10:43:33.473432+0000', 'last_fullsized': '2024-04-17T10:43:33.473432+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:43:39.179 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.17 is not active+clean 2024-04-17T10:43:39.179 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.474388+0000', 'last_change': '2024-04-17T10:43:33.474388+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.474388+0000', 'last_undegraded': '2024-04-17T10:43:33.474388+0000', 'last_fullsized': '2024-04-17T10:43:33.474388+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:43:39.179 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.14 is not active+clean 2024-04-17T10:43:39.179 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.483201+0000', 'last_change': '2024-04-17T10:43:33.483201+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.483201+0000', 'last_undegraded': '2024-04-17T10:43:33.483201+0000', 'last_fullsized': '2024-04-17T10:43:33.483201+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:43:39.180 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.15 is not active+clean 2024-04-17T10:43:39.180 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.471311+0000', 'last_change': '2024-04-17T10:43:33.471311+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.471311+0000', 'last_undegraded': '2024-04-17T10:43:33.471311+0000', 'last_fullsized': '2024-04-17T10:43:33.471311+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:43:39.180 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.12 is not active+clean 2024-04-17T10:43:39.180 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.469551+0000', 'last_change': '2024-04-17T10:43:33.469551+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.469551+0000', 'last_undegraded': '2024-04-17T10:43:33.469551+0000', 'last_fullsized': '2024-04-17T10:43:33.469551+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:43:39.180 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.13 is not active+clean 2024-04-17T10:43:39.180 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.471845+0000', 'last_change': '2024-04-17T10:43:33.471845+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.471845+0000', 'last_undegraded': '2024-04-17T10:43:33.471845+0000', 'last_fullsized': '2024-04-17T10:43:33.471845+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:43:39.180 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.10 is not active+clean 2024-04-17T10:43:39.180 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.469189+0000', 'last_change': '2024-04-17T10:43:33.469189+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.469189+0000', 'last_undegraded': '2024-04-17T10:43:33.469189+0000', 'last_fullsized': '2024-04-17T10:43:33.469189+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:43:39.180 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.11 is not active+clean 2024-04-17T10:43:39.180 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.468055+0000', 'last_change': '2024-04-17T10:43:33.468055+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.468055+0000', 'last_undegraded': '2024-04-17T10:43:33.468055+0000', 'last_fullsized': '2024-04-17T10:43:33.468055+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:43:39.181 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.e is not active+clean 2024-04-17T10:43:39.181 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.479579+0000', 'last_change': '2024-04-17T10:43:33.479579+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.479579+0000', 'last_undegraded': '2024-04-17T10:43:33.479579+0000', 'last_fullsized': '2024-04-17T10:43:33.479579+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:43:39.181 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.f is not active+clean 2024-04-17T10:43:39.181 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.470619+0000', 'last_change': '2024-04-17T10:43:33.470619+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.470619+0000', 'last_undegraded': '2024-04-17T10:43:33.470619+0000', 'last_fullsized': '2024-04-17T10:43:33.470619+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:43:39.181 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.c is not active+clean 2024-04-17T10:43:39.181 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.470650+0000', 'last_change': '2024-04-17T10:43:33.470650+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.470650+0000', 'last_undegraded': '2024-04-17T10:43:33.470650+0000', 'last_fullsized': '2024-04-17T10:43:33.470650+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:43:39.181 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.d is not active+clean 2024-04-17T10:43:39.181 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.468349+0000', 'last_change': '2024-04-17T10:43:33.468349+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.468349+0000', 'last_undegraded': '2024-04-17T10:43:33.468349+0000', 'last_fullsized': '2024-04-17T10:43:33.468349+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:43:39.181 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.b is not active+clean 2024-04-17T10:43:39.181 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.472725+0000', 'last_change': '2024-04-17T10:43:33.472725+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.472725+0000', 'last_undegraded': '2024-04-17T10:43:33.472725+0000', 'last_fullsized': '2024-04-17T10:43:33.472725+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:43:39.181 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.8 is not active+clean 2024-04-17T10:43:39.182 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.472216+0000', 'last_change': '2024-04-17T10:43:33.472216+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.472216+0000', 'last_undegraded': '2024-04-17T10:43:33.472216+0000', 'last_fullsized': '2024-04-17T10:43:33.472216+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:43:39.182 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.9 is not active+clean 2024-04-17T10:43:39.182 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.472216+0000', 'last_change': '2024-04-17T10:43:33.472216+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.472216+0000', 'last_undegraded': '2024-04-17T10:43:33.472216+0000', 'last_fullsized': '2024-04-17T10:43:33.472216+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:43:39.182 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.6 is not active+clean 2024-04-17T10:43:39.182 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.468640+0000', 'last_change': '2024-04-17T10:43:33.468640+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.468640+0000', 'last_undegraded': '2024-04-17T10:43:33.468640+0000', 'last_fullsized': '2024-04-17T10:43:33.468640+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:43:39.182 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.7 is not active+clean 2024-04-17T10:43:39.182 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.472880+0000', 'last_change': '2024-04-17T10:43:33.472880+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.472880+0000', 'last_undegraded': '2024-04-17T10:43:33.472880+0000', 'last_fullsized': '2024-04-17T10:43:33.472880+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:43:39.182 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.4 is not active+clean 2024-04-17T10:43:39.182 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.466584+0000', 'last_change': '2024-04-17T10:43:33.466584+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.466584+0000', 'last_undegraded': '2024-04-17T10:43:33.466584+0000', 'last_fullsized': '2024-04-17T10:43:33.466584+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:43:39.183 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.5 is not active+clean 2024-04-17T10:43:39.183 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.467765+0000', 'last_change': '2024-04-17T10:43:33.467765+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.467765+0000', 'last_undegraded': '2024-04-17T10:43:33.467765+0000', 'last_fullsized': '2024-04-17T10:43:33.467765+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:43:39.183 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2 is not active+clean 2024-04-17T10:43:39.183 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.468841+0000', 'last_change': '2024-04-17T10:43:33.468841+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.468841+0000', 'last_undegraded': '2024-04-17T10:43:33.468841+0000', 'last_fullsized': '2024-04-17T10:43:33.468841+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:43:39.183 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3 is not active+clean 2024-04-17T10:43:39.183 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.488610+0000', 'last_change': '2024-04-17T10:43:33.488610+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.488610+0000', 'last_undegraded': '2024-04-17T10:43:33.488610+0000', 'last_fullsized': '2024-04-17T10:43:33.488610+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:43:39.183 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.0 is not active+clean 2024-04-17T10:43:39.183 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.466736+0000', 'last_change': '2024-04-17T10:43:33.466736+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.466736+0000', 'last_undegraded': '2024-04-17T10:43:33.466736+0000', 'last_fullsized': '2024-04-17T10:43:33.466736+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:43:39.183 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1 is not active+clean 2024-04-17T10:43:39.183 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.476069+0000', 'last_change': '2024-04-17T10:43:33.476069+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.476069+0000', 'last_undegraded': '2024-04-17T10:43:33.476069+0000', 'last_fullsized': '2024-04-17T10:43:33.476069+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:43:39.183 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.a is not active+clean 2024-04-17T10:43:39.184 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:43:33.472418+0000', 'last_change': '2024-04-17T10:43:33.472418+0000', 'last_active': '2024-04-17T10:43:33.454791+0000', 'last_peered': '2024-04-17T10:43:33.454791+0000', 'last_clean': '2024-04-17T10:43:33.454791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:43:33.472418+0000', 'last_undegraded': '2024-04-17T10:43:33.472418+0000', 'last_fullsized': '2024-04-17T10:43:33.472418+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'last_clean_scrub_stamp': '2024-04-17T10:43:33.454791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:43:39.184 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-17T10:43:39.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.329+0000 7f4699a24640 1 Processor -- start 2024-04-17T10:43:39.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.329+0000 7f4699a24640 1 -- start start 2024-04-17T10:43:39.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.329+0000 7f4699a24640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f469415ba30 0x7f4694072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:39.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.329+0000 7f4699a24640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f46940cfbe0 0x7f469406c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:39.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.329+0000 7f4699a24640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f469406e320 con 0x7f469415c960 2024-04-17T10:43:39.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.329+0000 7f4699a24640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4694071ea0 con 0x7f46940cfbe0 2024-04-17T10:43:39.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.329+0000 7f4699a24640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f469406e040 con 0x7f469415ba30 2024-04-17T10:43:39.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.329+0000 7f4692ffd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f469415ba30 0x7f4694072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:39.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.329+0000 7f46937fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f46940cfbe0 0x7f469406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:39.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.329+0000 7f4692ffd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f469415ba30 0x7f4694072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:36372/0 (socket says 172.21.15.83:36372) 2024-04-17T10:43:39.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.329+0000 7f4692ffd640 1 -- 172.21.15.83:0/309030634 learned_addr learned my addr 172.21.15.83:0/309030634 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:39.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.333+0000 7f4692ffd640 1 -- 172.21.15.83:0/309030634 >> v2:172.21.15.167:3300/0 conn(0x7f46940cfbe0 msgr2=0x7f469406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:39.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.333+0000 7f4692ffd640 1 --2- 172.21.15.83:0/309030634 >> v2:172.21.15.167:3300/0 conn(0x7f46940cfbe0 0x7f469406c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:39.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.333+0000 7f4692ffd640 1 -- 172.21.15.83:0/309030634 >> v1:172.21.15.167:6789/0 conn(0x7f469415c960 legacy=0x7f46940cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:39.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.333+0000 7f4692ffd640 1 -- 172.21.15.83:0/309030634 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4694071850 con 0x7f469415ba30 2024-04-17T10:43:39.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.333+0000 7f4692ffd640 1 --2- 172.21.15.83:0/309030634 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f469415ba30 0x7f4694072330 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4684009870 tx=0x7f468401c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78ea2580c4e19e90 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:39.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.333+0000 7f4691ffb640 1 -- 172.21.15.83:0/309030634 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46840b0070 con 0x7f469415ba30 2024-04-17T10:43:39.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.333+0000 7f4691ffb640 1 -- 172.21.15.83:0/309030634 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46840ab600 con 0x7f469415ba30 2024-04-17T10:43:39.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.333+0000 7f4691ffb640 1 -- 172.21.15.83:0/309030634 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46840ab9d0 con 0x7f469415ba30 2024-04-17T10:43:39.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.333+0000 7f4699a24640 1 -- 172.21.15.83:0/309030634 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f469415ba30 msgr2=0x7f4694072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:39.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.333+0000 7f4699a24640 1 --2- 172.21.15.83:0/309030634 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f469415ba30 0x7f4694072330 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4684009870 tx=0x7f468401c4d0 comp rx=0 tx=0).stop 2024-04-17T10:43:39.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.333+0000 7f4699a24640 1 -- 172.21.15.83:0/309030634 shutdown_connections 2024-04-17T10:43:39.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.333+0000 7f4699a24640 1 --2- 172.21.15.83:0/309030634 >> v2:172.21.15.167:3300/0 conn(0x7f46940cfbe0 0x7f469406c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:39.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.333+0000 7f4699a24640 1 --2- 172.21.15.83:0/309030634 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f469415ba30 0x7f4694072330 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:39.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.333+0000 7f4699a24640 1 -- 172.21.15.83:0/309030634 >> 172.21.15.83:0/309030634 conn(0x7f469406abe0 msgr2=0x7f469406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:39.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.333+0000 7f4699a24640 1 -- 172.21.15.83:0/309030634 shutdown_connections 2024-04-17T10:43:39.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.333+0000 7f4699a24640 1 -- 172.21.15.83:0/309030634 wait complete. 2024-04-17T10:43:39.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.337+0000 7f4699a24640 1 Processor -- start 2024-04-17T10:43:39.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.337+0000 7f4699a24640 1 -- start start 2024-04-17T10:43:39.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.337+0000 7f4699a24640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46940cfbe0 0x7f4694148a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:39.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.337+0000 7f4699a24640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f469415ba30 0x7f4694149280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:39.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.337+0000 7f4699a24640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f469415c960 0x7f46941497c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:39.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.337+0000 7f4699a24640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469415cee0 con 0x7f469415c960 2024-04-17T10:43:39.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.337+0000 7f4699a24640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469415cd60 con 0x7f469415ba30 2024-04-17T10:43:39.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.337+0000 7f4699a24640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4694168520 con 0x7f46940cfbe0 2024-04-17T10:43:39.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.337+0000 7f46937fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f469415c960 0x7f46941497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:39.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.337+0000 7f46937fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f469415c960 0x7f46941497c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52380/0 (socket says 172.21.15.83:52380) 2024-04-17T10:43:39.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.337+0000 7f46937fe640 1 -- 172.21.15.83:0/4233243236 learned_addr learned my addr 172.21.15.83:0/4233243236 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:39.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.337+0000 7f4692ffd640 1 --2- 172.21.15.83:0/4233243236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46940cfbe0 0x7f4694148a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:39.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.337+0000 7f46927fc640 1 --2- 172.21.15.83:0/4233243236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f469415ba30 0x7f4694149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:39.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.337+0000 7f46937fe640 1 -- 172.21.15.83:0/4233243236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46940cfbe0 msgr2=0x7f4694148a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:39.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.337+0000 7f46937fe640 1 --2- 172.21.15.83:0/4233243236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46940cfbe0 0x7f4694148a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:39.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.337+0000 7f46937fe640 1 -- 172.21.15.83:0/4233243236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f469415ba30 msgr2=0x7f4694149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:39.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.337+0000 7f46937fe640 1 --2- 172.21.15.83:0/4233243236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f469415ba30 0x7f4694149280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:39.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.337+0000 7f46937fe640 1 -- 172.21.15.83:0/4233243236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4694149ff0 con 0x7f469415c960 2024-04-17T10:43:39.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.337+0000 7f46927fc640 1 --2- 172.21.15.83:0/4233243236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f469415ba30 0x7f4694149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:39.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.337+0000 7f4692ffd640 1 --2- 172.21.15.83:0/4233243236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46940cfbe0 0x7f4694148a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:43:39.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.337+0000 7f46937fe640 1 --2- 172.21.15.83:0/4233243236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f469415c960 0x7f46941497c0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f468c00efc0 tx=0x7f468c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:39.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.337+0000 7f467bfff640 1 -- 172.21.15.83:0/4233243236 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c018020 con 0x7f469415c960 2024-04-17T10:43:39.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.337+0000 7f467bfff640 1 -- 172.21.15.83:0/4233243236 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468c014480 con 0x7f469415c960 2024-04-17T10:43:39.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.337+0000 7f4699a24640 1 -- 172.21.15.83:0/4233243236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f469414a2d0 con 0x7f469415c960 2024-04-17T10:43:39.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.337+0000 7f467bfff640 1 -- 172.21.15.83:0/4233243236 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c014870 con 0x7f469415c960 2024-04-17T10:43:39.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.337+0000 7f4699a24640 1 -- 172.21.15.83:0/4233243236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f469414a890 con 0x7f469415c960 2024-04-17T10:43:39.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.337+0000 7f4699a24640 1 -- 172.21.15.83:0/4233243236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4694074f90 con 0x7f469415c960 2024-04-17T10:43:39.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.337+0000 7f467bfff640 1 -- 172.21.15.83:0/4233243236 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f468c01b020 con 0x7f469415c960 2024-04-17T10:43:39.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.341+0000 7f467bfff640 1 --2- 172.21.15.83:0/4233243236 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4680066f60 0x7f4680069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:39.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.341+0000 7f4692ffd640 1 --2- 172.21.15.83:0/4233243236 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4680066f60 0x7f4680069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:39.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.341+0000 7f467bfff640 1 -- 172.21.15.83:0/4233243236 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f468c085e40 con 0x7f469415c960 2024-04-17T10:43:39.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.341+0000 7f4692ffd640 1 --2- 172.21.15.83:0/4233243236 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4680066f60 0x7f4680069420 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f46840ad000 tx=0x7f46840a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:39.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.345+0000 7f467bfff640 1 -- 172.21.15.83:0/4233243236 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f468c022030 con 0x7f469415c960 2024-04-17T10:43:39.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.545+0000 7f4699a24640 1 -- 172.21.15.83:0/4233243236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f46941562d0 con 0x7f469415c960 2024-04-17T10:43:39.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.545+0000 7f467bfff640 1 -- 172.21.15.83:0/4233243236 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1454 (secure 0 0 0) 0x7f468c010040 con 0x7f469415c960 2024-04-17T10:43:39.543 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:43:39.543 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":352,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":74,"num_osds":8,"num_up_osds":8,"osd_up_since":1713350279,"num_in_osds":8,"osd_in_since":1713350270,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":353173504,"bytes_avail":772740939776,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":48,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"d","status":"up:active","gid":7104}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-04-17T10:43:24.274662+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-17T10:43:39.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.549+0000 7f4699a24640 1 -- 172.21.15.83:0/4233243236 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4680066f60 msgr2=0x7f4680069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:39.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.549+0000 7f4699a24640 1 --2- 172.21.15.83:0/4233243236 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4680066f60 0x7f4680069420 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f46840ad000 tx=0x7f46840a6000 comp rx=0 tx=0).stop 2024-04-17T10:43:39.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.549+0000 7f4699a24640 1 -- 172.21.15.83:0/4233243236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f469415c960 msgr2=0x7f46941497c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:39.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.549+0000 7f4699a24640 1 --2- 172.21.15.83:0/4233243236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f469415c960 0x7f46941497c0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f468c00efc0 tx=0x7f468c00c490 comp rx=0 tx=0).stop 2024-04-17T10:43:39.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.549+0000 7f4699a24640 1 -- 172.21.15.83:0/4233243236 shutdown_connections 2024-04-17T10:43:39.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.549+0000 7f4699a24640 1 --2- 172.21.15.83:0/4233243236 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4680066f60 0x7f4680069420 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:39.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.549+0000 7f4699a24640 1 --2- 172.21.15.83:0/4233243236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f469415c960 0x7f46941497c0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:39.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.549+0000 7f4699a24640 1 --2- 172.21.15.83:0/4233243236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f469415ba30 0x7f4694149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:39.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.549+0000 7f4699a24640 1 --2- 172.21.15.83:0/4233243236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46940cfbe0 0x7f4694148a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:39.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.549+0000 7f4699a24640 1 -- 172.21.15.83:0/4233243236 >> 172.21.15.83:0/4233243236 conn(0x7f469406abe0 msgr2=0x7f469415e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:39.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.549+0000 7f4699a24640 1 -- 172.21.15.83:0/4233243236 shutdown_connections 2024-04-17T10:43:39.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.549+0000 7f4699a24640 1 -- 172.21.15.83:0/4233243236 wait complete. 2024-04-17T10:43:39.558 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-17T10:43:39.559 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:43:39.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.669+0000 7fc7b80b1640 1 Processor -- start 2024-04-17T10:43:39.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.669+0000 7fc7b80b1640 1 -- start start 2024-04-17T10:43:39.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.669+0000 7fc7b80b1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7b015f1d0 0x7fc7b015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:39.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.669+0000 7fc7b80b1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc7b00b2de0 0x7fc7b015fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:39.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.669+0000 7fc7b80b1640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc7b00d22d0 0x7fc7b01602b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:39.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.669+0000 7fc7b80b1640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7b015f5b0 con 0x7fc7b00b2de0 2024-04-17T10:43:39.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.669+0000 7fc7b80b1640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7b00b2620 con 0x7fc7b00d22d0 2024-04-17T10:43:39.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.669+0000 7fc7b80b1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b00b2340 con 0x7fc7b015f1d0 2024-04-17T10:43:39.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.669+0000 7fc7b564c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc7b00b2de0 0x7fc7b015fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:39.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.669+0000 7fc7b564c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc7b00b2de0 0x7fc7b015fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52392/0 (socket says 172.21.15.83:52392) 2024-04-17T10:43:39.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.669+0000 7fc7b564c640 1 -- 172.21.15.83:0/69450012 learned_addr learned my addr 172.21.15.83:0/69450012 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:39.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.669+0000 7fc7b664e640 1 --2- 172.21.15.83:0/69450012 >> v2:172.21.15.167:3300/0 conn(0x7fc7b00d22d0 0x7fc7b01602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:39.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.669+0000 7fc7b564c640 1 -- 172.21.15.83:0/69450012 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7b015f1d0 msgr2=0x7fc7b015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:39.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.669+0000 7fc7b5e4d640 1 --2- 172.21.15.83:0/69450012 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7b015f1d0 0x7fc7b015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:39.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.669+0000 7fc7b564c640 1 --2- 172.21.15.83:0/69450012 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7b015f1d0 0x7fc7b015f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:39.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.669+0000 7fc7b564c640 1 -- 172.21.15.83:0/69450012 >> v2:172.21.15.167:3300/0 conn(0x7fc7b00d22d0 msgr2=0x7fc7b01602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:39.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.669+0000 7fc7b564c640 1 --2- 172.21.15.83:0/69450012 >> v2:172.21.15.167:3300/0 conn(0x7fc7b00d22d0 0x7fc7b01602b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:39.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.669+0000 7fc7b564c640 1 -- 172.21.15.83:0/69450012 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b00b1760 con 0x7fc7b00b2de0 2024-04-17T10:43:39.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.669+0000 7fc7b664e640 1 --2- 172.21.15.83:0/69450012 >> v2:172.21.15.167:3300/0 conn(0x7fc7b00d22d0 0x7fc7b01602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:39.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b564c640 1 --2- 172.21.15.83:0/69450012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7b00b2de0 0x7fc7b015fd70 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0072150 tx=0x7fc7ac01c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=267489c9c0329b7d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:39.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b4e4b640 1 -- 172.21.15.83:0/69450012 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac0ab5f0 con 0x7fc7b00b2de0 2024-04-17T10:43:39.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b4e4b640 1 -- 172.21.15.83:0/69450012 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7ac0b0070 con 0x7fc7b00b2de0 2024-04-17T10:43:39.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b4e4b640 1 -- 172.21.15.83:0/69450012 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac0ab9b0 con 0x7fc7b00b2de0 2024-04-17T10:43:39.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b80b1640 1 -- 172.21.15.83:0/69450012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7b00b2de0 msgr2=0x7fc7b015fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:39.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b80b1640 1 --2- 172.21.15.83:0/69450012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7b00b2de0 0x7fc7b015fd70 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0072150 tx=0x7fc7ac01c4c0 comp rx=0 tx=0).stop 2024-04-17T10:43:39.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b80b1640 1 -- 172.21.15.83:0/69450012 shutdown_connections 2024-04-17T10:43:39.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b80b1640 1 --2- 172.21.15.83:0/69450012 >> v2:172.21.15.167:3300/0 conn(0x7fc7b00d22d0 0x7fc7b01602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:39.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b80b1640 1 --2- 172.21.15.83:0/69450012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7b00b2de0 0x7fc7b015fd70 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:39.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b80b1640 1 --2- 172.21.15.83:0/69450012 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7b015f1d0 0x7fc7b015f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:39.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b80b1640 1 -- 172.21.15.83:0/69450012 >> 172.21.15.83:0/69450012 conn(0x7fc7b00d17b0 msgr2=0x7fc7b00d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:39.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b80b1640 1 -- 172.21.15.83:0/69450012 shutdown_connections 2024-04-17T10:43:39.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b80b1640 1 -- 172.21.15.83:0/69450012 wait complete. 2024-04-17T10:43:39.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b80b1640 1 Processor -- start 2024-04-17T10:43:39.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b80b1640 1 -- start start 2024-04-17T10:43:39.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b80b1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7b00d22d0 0x7fc7b0151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:39.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b80b1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7b015f1d0 0x7fc7b0150490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:39.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b80b1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7b0152230 0x7fc7b00c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:39.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b80b1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b00d0400 con 0x7fc7b015f1d0 2024-04-17T10:43:39.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b80b1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b00d0280 con 0x7fc7b0152230 2024-04-17T10:43:39.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b80b1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b00d0580 con 0x7fc7b00d22d0 2024-04-17T10:43:39.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b564c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7b015f1d0 0x7fc7b0150490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:39.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b564c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7b015f1d0 0x7fc7b0150490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52404/0 (socket says 172.21.15.83:52404) 2024-04-17T10:43:39.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b564c640 1 -- 172.21.15.83:0/3377434893 learned_addr learned my addr 172.21.15.83:0/3377434893 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:39.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b664e640 1 --2- 172.21.15.83:0/3377434893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7b0152230 0x7fc7b00c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:39.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b564c640 1 -- 172.21.15.83:0/3377434893 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7b00d22d0 msgr2=0x7fc7b0151b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:39.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b564c640 1 --2- 172.21.15.83:0/3377434893 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7b00d22d0 0x7fc7b0151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:39.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b564c640 1 -- 172.21.15.83:0/3377434893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7b0152230 msgr2=0x7fc7b00c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:39.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b564c640 1 --2- 172.21.15.83:0/3377434893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7b0152230 0x7fc7b00c7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:39.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b564c640 1 -- 172.21.15.83:0/3377434893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b00c7e80 con 0x7fc7b015f1d0 2024-04-17T10:43:39.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b664e640 1 --2- 172.21.15.83:0/3377434893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7b0152230 0x7fc7b00c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:39.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b564c640 1 --2- 172.21.15.83:0/3377434893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7b015f1d0 0x7fc7b0150490 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac01ab70 tx=0x7fc7ac01ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:39.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7a67fc640 1 -- 172.21.15.83:0/3377434893 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac0be5f0 con 0x7fc7b015f1d0 2024-04-17T10:43:39.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7a67fc640 1 -- 172.21.15.83:0/3377434893 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7ac0be7b0 con 0x7fc7b015f1d0 2024-04-17T10:43:39.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b80b1640 1 -- 172.21.15.83:0/3377434893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b0150d90 con 0x7fc7b015f1d0 2024-04-17T10:43:39.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7a67fc640 1 -- 172.21.15.83:0/3377434893 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac0beba0 con 0x7fc7b015f1d0 2024-04-17T10:43:39.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.673+0000 7fc7b80b1640 1 -- 172.21.15.83:0/3377434893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b0151310 con 0x7fc7b015f1d0 2024-04-17T10:43:39.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.677+0000 7fc7b80b1640 1 -- 172.21.15.83:0/3377434893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7b0074f90 con 0x7fc7b015f1d0 2024-04-17T10:43:39.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.677+0000 7fc7a67fc640 1 -- 172.21.15.83:0/3377434893 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc7ac0b43f0 con 0x7fc7b015f1d0 2024-04-17T10:43:39.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.677+0000 7fc7a67fc640 1 --2- 172.21.15.83:0/3377434893 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc794066f90 0x7fc794069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:39.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.677+0000 7fc7a67fc640 1 -- 172.21.15.83:0/3377434893 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc7ac11f480 con 0x7fc7b015f1d0 2024-04-17T10:43:39.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.677+0000 7fc7b5e4d640 1 --2- 172.21.15.83:0/3377434893 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc794066f90 0x7fc794069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:39.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.677+0000 7fc7b5e4d640 1 --2- 172.21.15.83:0/3377434893 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc794066f90 0x7fc794069450 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00ce1b0 tx=0x7fc7a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:39.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.681+0000 7fc7a67fc640 1 -- 172.21.15.83:0/3377434893 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc7ac0c4060 con 0x7fc7b015f1d0 2024-04-17T10:43:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.833+0000 7fc7b80b1640 1 -- 172.21.15.83:0/3377434893 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc7b0072690 con 0x7fc794066f90 2024-04-17T10:43:39.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.853+0000 7fc7a67fc640 1 -- 172.21.15.83:0/3377434893 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246757 (secure 0 0 0) 0x7fc7b0072690 con 0x7fc794066f90 2024-04-17T10:43:39.850 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:43:39.852 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:43:39.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.857+0000 7fc7b80b1640 1 -- 172.21.15.83:0/3377434893 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc794066f90 msgr2=0x7fc794069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:39.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.857+0000 7fc7b80b1640 1 --2- 172.21.15.83:0/3377434893 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc794066f90 0x7fc794069450 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00ce1b0 tx=0x7fc7a0008040 comp rx=0 tx=0).stop 2024-04-17T10:43:39.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.857+0000 7fc7b80b1640 1 -- 172.21.15.83:0/3377434893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7b015f1d0 msgr2=0x7fc7b0150490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:39.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.857+0000 7fc7b80b1640 1 --2- 172.21.15.83:0/3377434893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7b015f1d0 0x7fc7b0150490 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac01ab70 tx=0x7fc7ac01ad00 comp rx=0 tx=0).stop 2024-04-17T10:43:39.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.857+0000 7fc7b80b1640 1 -- 172.21.15.83:0/3377434893 shutdown_connections 2024-04-17T10:43:39.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.857+0000 7fc7b80b1640 1 --2- 172.21.15.83:0/3377434893 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc794066f90 0x7fc794069450 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:39.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.857+0000 7fc7b80b1640 1 --2- 172.21.15.83:0/3377434893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7b0152230 0x7fc7b00c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:39.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.857+0000 7fc7b80b1640 1 --2- 172.21.15.83:0/3377434893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7b015f1d0 0x7fc7b0150490 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:39.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.857+0000 7fc7b80b1640 1 --2- 172.21.15.83:0/3377434893 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7b00d22d0 0x7fc7b0151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:39.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.857+0000 7fc7b80b1640 1 -- 172.21.15.83:0/3377434893 >> 172.21.15.83:0/3377434893 conn(0x7fc7b00d17b0 msgr2=0x7fc7b00ce5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:39.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.857+0000 7fc7b80b1640 1 -- 172.21.15.83:0/3377434893 shutdown_connections 2024-04-17T10:43:39.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:39.857+0000 7fc7b80b1640 1 -- 172.21.15.83:0/3377434893 wait complete. 2024-04-17T10:43:39.868 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2024-04-17T10:43:38.278527+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":38,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":344896,"kb_used_data":8008,"kb_used_omap":12,"kb_used_meta":336755,"kb_avail":754629824,"statfs":{"total":773094113280,"available":772740939776,"internally_reserved":0,"allocated":8200192,"data_stored":4156128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":344837680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.822026"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.475786+0000","last_change":"2024-04-17T10:43:33.475786+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.475786+0000","last_undegraded":"2024-04-17T10:43:33.475786+0000","last_fullsized":"2024-04-17T10:43:33.475786+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.488388+0000","last_change":"2024-04-17T10:43:33.488388+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.488388+0000","last_undegraded":"2024-04-17T10:43:33.488388+0000","last_fullsized":"2024-04-17T10:43:33.488388+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.474093+0000","last_change":"2024-04-17T10:43:33.474093+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.474093+0000","last_undegraded":"2024-04-17T10:43:33.474093+0000","last_fullsized":"2024-04-17T10:43:33.474093+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.475007+0000","last_change":"2024-04-17T10:43:33.475007+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.475007+0000","last_undegraded":"2024-04-17T10:43:33.475007+0000","last_fullsized":"2024-04-17T10:43:33.475007+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.469492+0000","last_change":"2024-04-17T10:43:33.469492+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.469492+0000","last_undegraded":"2024-04-17T10:43:33.469492+0000","last_fullsized":"2024-04-17T10:43:33.469492+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.492867+0000","last_change":"2024-04-17T10:43:33.492867+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.492867+0000","last_undegraded":"2024-04-17T10:43:33.492867+0000","last_fullsized":"2024-04-17T10:43:33.492867+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.475679+0000","last_change":"2024-04-17T10:43:33.475679+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.475679+0000","last_undegraded":"2024-04-17T10:43:33.475679+0000","last_fullsized":"2024-04-17T10:43:33.475679+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.478045+0000","last_change":"2024-04-17T10:43:33.478045+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.478045+0000","last_undegraded":"2024-04-17T10:43:33.478045+0000","last_fullsized":"2024-04-17T10:43:33.478045+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.470779+0000","last_change":"2024-04-17T10:43:33.470779+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.470779+0000","last_undegraded":"2024-04-17T10:43:33.470779+0000","last_fullsized":"2024-04-17T10:43:33.470779+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.480695+0000","last_change":"2024-04-17T10:43:33.480695+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.480695+0000","last_undegraded":"2024-04-17T10:43:33.480695+0000","last_fullsized":"2024-04-17T10:43:33.480695+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.469761+0000","last_change":"2024-04-17T10:43:33.469761+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.469761+0000","last_undegraded":"2024-04-17T10:43:33.469761+0000","last_fullsized":"2024-04-17T10:43:33.469761+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.471328+0000","last_change":"2024-04-17T10:43:33.471328+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.471328+0000","last_undegraded":"2024-04-17T10:43:33.471328+0000","last_fullsized":"2024-04-17T10:43:33.471328+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.468149+0000","last_change":"2024-04-17T10:43:33.468149+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.468149+0000","last_undegraded":"2024-04-17T10:43:33.468149+0000","last_fullsized":"2024-04-17T10:43:33.468149+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.470994+0000","last_change":"2024-04-17T10:43:33.470994+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.470994+0000","last_undegraded":"2024-04-17T10:43:33.470994+0000","last_fullsized":"2024-04-17T10:43:33.470994+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.472752+0000","last_change":"2024-04-17T10:43:33.472752+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.472752+0000","last_undegraded":"2024-04-17T10:43:33.472752+0000","last_fullsized":"2024-04-17T10:43:33.472752+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.479475+0000","last_change":"2024-04-17T10:43:33.479475+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.479475+0000","last_undegraded":"2024-04-17T10:43:33.479475+0000","last_fullsized":"2024-04-17T10:43:33.479475+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.470329+0000","last_change":"2024-04-17T10:43:33.470329+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.470329+0000","last_undegraded":"2024-04-17T10:43:33.470329+0000","last_fullsized":"2024-04-17T10:43:33.470329+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.470663+0000","last_change":"2024-04-17T10:43:33.470663+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.470663+0000","last_undegraded":"2024-04-17T10:43:33.470663+0000","last_fullsized":"2024-04-17T10:43:33.470663+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.491227+0000","last_change":"2024-04-17T10:43:33.491227+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.491227+0000","last_undegraded":"2024-04-17T10:43:33.491227+0000","last_fullsized":"2024-04-17T10:43:33.491227+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.469792+0000","last_change":"2024-04-17T10:43:33.469792+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.469792+0000","last_undegraded":"2024-04-17T10:43:33.469792+0000","last_fullsized":"2024-04-17T10:43:33.469792+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.469177+0000","last_change":"2024-04-17T10:43:33.469177+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.469177+0000","last_undegraded":"2024-04-17T10:43:33.469177+0000","last_fullsized":"2024-04-17T10:43:33.469177+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.477573+0000","last_change":"2024-04-17T10:43:33.477573+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.477573+0000","last_undegraded":"2024-04-17T10:43:33.477573+0000","last_fullsized":"2024-04-17T10:43:33.477573+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.466522+0000","last_change":"2024-04-17T10:43:33.466522+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.466522+0000","last_undegraded":"2024-04-17T10:43:33.466522+0000","last_fullsized":"2024-04-17T10:43:33.466522+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.470617+0000","last_change":"2024-04-17T10:43:33.470617+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.470617+0000","last_undegraded":"2024-04-17T10:43:33.470617+0000","last_fullsized":"2024-04-17T10:43:33.470617+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.466969+0000","last_change":"2024-04-17T10:43:33.466969+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.466969+0000","last_undegraded":"2024-04-17T10:43:33.466969+0000","last_fullsized":"2024-04-17T10:43:33.466969+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.474514+0000","last_change":"2024-04-17T10:43:33.474514+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.474514+0000","last_undegraded":"2024-04-17T10:43:33.474514+0000","last_fullsized":"2024-04-17T10:43:33.474514+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.473429+0000","last_change":"2024-04-17T10:43:33.473429+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.473429+0000","last_undegraded":"2024-04-17T10:43:33.473429+0000","last_fullsized":"2024-04-17T10:43:33.473429+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.467411+0000","last_change":"2024-04-17T10:43:33.467411+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.467411+0000","last_undegraded":"2024-04-17T10:43:33.467411+0000","last_fullsized":"2024-04-17T10:43:33.467411+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.473115+0000","last_change":"2024-04-17T10:43:33.473115+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.473115+0000","last_undegraded":"2024-04-17T10:43:33.473115+0000","last_fullsized":"2024-04-17T10:43:33.473115+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.472797+0000","last_change":"2024-04-17T10:43:33.472797+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.472797+0000","last_undegraded":"2024-04-17T10:43:33.472797+0000","last_fullsized":"2024-04-17T10:43:33.472797+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.473897+0000","last_change":"2024-04-17T10:43:33.473897+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.473897+0000","last_undegraded":"2024-04-17T10:43:33.473897+0000","last_fullsized":"2024-04-17T10:43:33.473897+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.476712+0000","last_change":"2024-04-17T10:43:33.476712+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.476712+0000","last_undegraded":"2024-04-17T10:43:33.476712+0000","last_fullsized":"2024-04-17T10:43:33.476712+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.469503+0000","last_change":"2024-04-17T10:43:33.469503+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.469503+0000","last_undegraded":"2024-04-17T10:43:33.469503+0000","last_fullsized":"2024-04-17T10:43:33.469503+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.497359+0000","last_change":"2024-04-17T10:43:33.488363+0000","last_active":"2024-04-17T10:43:33.497359+0000","last_peered":"2024-04-17T10:43:33.497359+0000","last_clean":"2024-04-17T10:43:33.497359+0000","last_became_active":"2024-04-17T10:43:33.487677+0000","last_became_peered":"2024-04-17T10:43:33.487677+0000","last_unstale":"2024-04-17T10:43:33.497359+0000","last_undegraded":"2024-04-17T10:43:33.497359+0000","last_fullsized":"2024-04-17T10:43:33.497359+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:43:20.629428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.468080+0000","last_change":"2024-04-17T10:43:33.468080+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.468080+0000","last_undegraded":"2024-04-17T10:43:33.468080+0000","last_fullsized":"2024-04-17T10:43:33.468080+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.479669+0000","last_change":"2024-04-17T10:43:33.478164+0000","last_active":"2024-04-17T10:43:33.479669+0000","last_peered":"2024-04-17T10:43:33.479669+0000","last_clean":"2024-04-17T10:43:33.479669+0000","last_became_active":"2024-04-17T10:43:33.477765+0000","last_became_peered":"2024-04-17T10:43:33.477765+0000","last_unstale":"2024-04-17T10:43:33.479669+0000","last_undegraded":"2024-04-17T10:43:33.479669+0000","last_fullsized":"2024-04-17T10:43:33.479669+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:40:08.909983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.486984+0000","last_change":"2024-04-17T10:43:33.486984+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.486984+0000","last_undegraded":"2024-04-17T10:43:33.486984+0000","last_fullsized":"2024-04-17T10:43:33.486984+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.510659+0000","last_change":"2024-04-17T10:43:33.504903+0000","last_active":"2024-04-17T10:43:33.510659+0000","last_peered":"2024-04-17T10:43:33.510659+0000","last_clean":"2024-04-17T10:43:33.510659+0000","last_became_active":"2024-04-17T10:43:33.503926+0000","last_became_peered":"2024-04-17T10:43:33.503926+0000","last_unstale":"2024-04-17T10:43:33.510659+0000","last_undegraded":"2024-04-17T10:43:33.510659+0000","last_fullsized":"2024-04-17T10:43:33.510659+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:33:06.929106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.468718+0000","last_change":"2024-04-17T10:43:33.468718+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.468718+0000","last_undegraded":"2024-04-17T10:43:33.468718+0000","last_fullsized":"2024-04-17T10:43:33.468718+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.496406+0000","last_change":"2024-04-17T10:43:33.491909+0000","last_active":"2024-04-17T10:43:33.496406+0000","last_peered":"2024-04-17T10:43:33.496406+0000","last_clean":"2024-04-17T10:43:33.496406+0000","last_became_active":"2024-04-17T10:43:33.490806+0000","last_became_peered":"2024-04-17T10:43:33.490806+0000","last_unstale":"2024-04-17T10:43:33.496406+0000","last_undegraded":"2024-04-17T10:43:33.496406+0000","last_fullsized":"2024-04-17T10:43:33.496406+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:33:38.078826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.471659+0000","last_change":"2024-04-17T10:43:33.471659+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.471659+0000","last_undegraded":"2024-04-17T10:43:33.471659+0000","last_fullsized":"2024-04-17T10:43:33.471659+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.504597+0000","last_change":"2024-04-17T10:43:33.504035+0000","last_active":"2024-04-17T10:43:33.504597+0000","last_peered":"2024-04-17T10:43:33.504597+0000","last_clean":"2024-04-17T10:43:33.504597+0000","last_became_active":"2024-04-17T10:43:33.503454+0000","last_became_peered":"2024-04-17T10:43:33.503454+0000","last_unstale":"2024-04-17T10:43:33.504597+0000","last_undegraded":"2024-04-17T10:43:33.504597+0000","last_fullsized":"2024-04-17T10:43:33.504597+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:53:36.645485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.467972+0000","last_change":"2024-04-17T10:43:33.467972+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.467972+0000","last_undegraded":"2024-04-17T10:43:33.467972+0000","last_fullsized":"2024-04-17T10:43:33.467972+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.505433+0000","last_change":"2024-04-17T10:43:33.502776+0000","last_active":"2024-04-17T10:43:33.505433+0000","last_peered":"2024-04-17T10:43:33.505433+0000","last_clean":"2024-04-17T10:43:33.505433+0000","last_became_active":"2024-04-17T10:43:33.498687+0000","last_became_peered":"2024-04-17T10:43:33.498687+0000","last_unstale":"2024-04-17T10:43:33.505433+0000","last_undegraded":"2024-04-17T10:43:33.505433+0000","last_fullsized":"2024-04-17T10:43:33.505433+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:45:05.760753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.468424+0000","last_change":"2024-04-17T10:43:33.468424+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.468424+0000","last_undegraded":"2024-04-17T10:43:33.468424+0000","last_fullsized":"2024-04-17T10:43:33.468424+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.506648+0000","last_change":"2024-04-17T10:43:33.506146+0000","last_active":"2024-04-17T10:43:33.506648+0000","last_peered":"2024-04-17T10:43:33.506648+0000","last_clean":"2024-04-17T10:43:33.506648+0000","last_became_active":"2024-04-17T10:43:33.505651+0000","last_became_peered":"2024-04-17T10:43:33.505651+0000","last_unstale":"2024-04-17T10:43:33.506648+0000","last_undegraded":"2024-04-17T10:43:33.506648+0000","last_fullsized":"2024-04-17T10:43:33.506648+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:44:26.965507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.469679+0000","last_change":"2024-04-17T10:43:33.469679+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.469679+0000","last_undegraded":"2024-04-17T10:43:33.469679+0000","last_fullsized":"2024-04-17T10:43:33.469679+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.480916+0000","last_change":"2024-04-17T10:43:33.478569+0000","last_active":"2024-04-17T10:43:33.480916+0000","last_peered":"2024-04-17T10:43:33.480916+0000","last_clean":"2024-04-17T10:43:33.480916+0000","last_became_active":"2024-04-17T10:43:33.477453+0000","last_became_peered":"2024-04-17T10:43:33.477453+0000","last_unstale":"2024-04-17T10:43:33.480916+0000","last_undegraded":"2024-04-17T10:43:33.480916+0000","last_fullsized":"2024-04-17T10:43:33.480916+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:59:17.699632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.473432+0000","last_change":"2024-04-17T10:43:33.473432+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.473432+0000","last_undegraded":"2024-04-17T10:43:33.473432+0000","last_fullsized":"2024-04-17T10:43:33.473432+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.509346+0000","last_change":"2024-04-17T10:43:33.499957+0000","last_active":"2024-04-17T10:43:33.509346+0000","last_peered":"2024-04-17T10:43:33.509346+0000","last_clean":"2024-04-17T10:43:33.509346+0000","last_became_active":"2024-04-17T10:43:33.481936+0000","last_became_peered":"2024-04-17T10:43:33.481936+0000","last_unstale":"2024-04-17T10:43:33.509346+0000","last_undegraded":"2024-04-17T10:43:33.509346+0000","last_fullsized":"2024-04-17T10:43:33.509346+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:34:00.759922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.474388+0000","last_change":"2024-04-17T10:43:33.474388+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.474388+0000","last_undegraded":"2024-04-17T10:43:33.474388+0000","last_fullsized":"2024-04-17T10:43:33.474388+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.497306+0000","last_change":"2024-04-17T10:43:33.494714+0000","last_active":"2024-04-17T10:43:33.497306+0000","last_peered":"2024-04-17T10:43:33.497306+0000","last_clean":"2024-04-17T10:43:33.497306+0000","last_became_active":"2024-04-17T10:43:33.493921+0000","last_became_peered":"2024-04-17T10:43:33.493921+0000","last_unstale":"2024-04-17T10:43:33.497306+0000","last_undegraded":"2024-04-17T10:43:33.497306+0000","last_fullsized":"2024-04-17T10:43:33.497306+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:34:40.985936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.483201+0000","last_change":"2024-04-17T10:43:33.483201+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.483201+0000","last_undegraded":"2024-04-17T10:43:33.483201+0000","last_fullsized":"2024-04-17T10:43:33.483201+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.493416+0000","last_change":"2024-04-17T10:43:33.492746+0000","last_active":"2024-04-17T10:43:33.493416+0000","last_peered":"2024-04-17T10:43:33.493416+0000","last_clean":"2024-04-17T10:43:33.493416+0000","last_became_active":"2024-04-17T10:43:33.492349+0000","last_became_peered":"2024-04-17T10:43:33.492349+0000","last_unstale":"2024-04-17T10:43:33.493416+0000","last_undegraded":"2024-04-17T10:43:33.493416+0000","last_fullsized":"2024-04-17T10:43:33.493416+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:37:09.710132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.471311+0000","last_change":"2024-04-17T10:43:33.471311+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.471311+0000","last_undegraded":"2024-04-17T10:43:33.471311+0000","last_fullsized":"2024-04-17T10:43:33.471311+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.480245+0000","last_change":"2024-04-17T10:43:33.476458+0000","last_active":"2024-04-17T10:43:33.480245+0000","last_peered":"2024-04-17T10:43:33.480245+0000","last_clean":"2024-04-17T10:43:33.480245+0000","last_became_active":"2024-04-17T10:43:33.475608+0000","last_became_peered":"2024-04-17T10:43:33.475608+0000","last_unstale":"2024-04-17T10:43:33.480245+0000","last_undegraded":"2024-04-17T10:43:33.480245+0000","last_fullsized":"2024-04-17T10:43:33.480245+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:45:49.643198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.469551+0000","last_change":"2024-04-17T10:43:33.469551+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.469551+0000","last_undegraded":"2024-04-17T10:43:33.469551+0000","last_fullsized":"2024-04-17T10:43:33.469551+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.481177+0000","last_change":"2024-04-17T10:43:33.479258+0000","last_active":"2024-04-17T10:43:33.481177+0000","last_peered":"2024-04-17T10:43:33.481177+0000","last_clean":"2024-04-17T10:43:33.481177+0000","last_became_active":"2024-04-17T10:43:33.478798+0000","last_became_peered":"2024-04-17T10:43:33.478798+0000","last_unstale":"2024-04-17T10:43:33.481177+0000","last_undegraded":"2024-04-17T10:43:33.481177+0000","last_fullsized":"2024-04-17T10:43:33.481177+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:45:05.760753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.471845+0000","last_change":"2024-04-17T10:43:33.471845+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.471845+0000","last_undegraded":"2024-04-17T10:43:33.471845+0000","last_fullsized":"2024-04-17T10:43:33.471845+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.504738+0000","last_change":"2024-04-17T10:43:33.499759+0000","last_active":"2024-04-17T10:43:33.504738+0000","last_peered":"2024-04-17T10:43:33.504738+0000","last_clean":"2024-04-17T10:43:33.504738+0000","last_became_active":"2024-04-17T10:43:33.498138+0000","last_became_peered":"2024-04-17T10:43:33.498138+0000","last_unstale":"2024-04-17T10:43:33.504738+0000","last_undegraded":"2024-04-17T10:43:33.504738+0000","last_fullsized":"2024-04-17T10:43:33.504738+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:26:58.859542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.469189+0000","last_change":"2024-04-17T10:43:33.469189+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.469189+0000","last_undegraded":"2024-04-17T10:43:33.469189+0000","last_fullsized":"2024-04-17T10:43:33.469189+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.493295+0000","last_change":"2024-04-17T10:43:33.488497+0000","last_active":"2024-04-17T10:43:33.493295+0000","last_peered":"2024-04-17T10:43:33.493295+0000","last_clean":"2024-04-17T10:43:33.493295+0000","last_became_active":"2024-04-17T10:43:33.487777+0000","last_became_peered":"2024-04-17T10:43:33.487777+0000","last_unstale":"2024-04-17T10:43:33.493295+0000","last_undegraded":"2024-04-17T10:43:33.493295+0000","last_fullsized":"2024-04-17T10:43:33.493295+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:42:26.108551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.468055+0000","last_change":"2024-04-17T10:43:33.468055+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.468055+0000","last_undegraded":"2024-04-17T10:43:33.468055+0000","last_fullsized":"2024-04-17T10:43:33.468055+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.510713+0000","last_change":"2024-04-17T10:43:33.505100+0000","last_active":"2024-04-17T10:43:33.510713+0000","last_peered":"2024-04-17T10:43:33.510713+0000","last_clean":"2024-04-17T10:43:33.510713+0000","last_became_active":"2024-04-17T10:43:33.504443+0000","last_became_peered":"2024-04-17T10:43:33.504443+0000","last_unstale":"2024-04-17T10:43:33.510713+0000","last_undegraded":"2024-04-17T10:43:33.510713+0000","last_fullsized":"2024-04-17T10:43:33.510713+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:38:49.951085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.479579+0000","last_change":"2024-04-17T10:43:33.479579+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.479579+0000","last_undegraded":"2024-04-17T10:43:33.479579+0000","last_fullsized":"2024-04-17T10:43:33.479579+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.504571+0000","last_change":"2024-04-17T10:43:33.497414+0000","last_active":"2024-04-17T10:43:33.504571+0000","last_peered":"2024-04-17T10:43:33.504571+0000","last_clean":"2024-04-17T10:43:33.504571+0000","last_became_active":"2024-04-17T10:43:33.495764+0000","last_became_peered":"2024-04-17T10:43:33.495764+0000","last_unstale":"2024-04-17T10:43:33.504571+0000","last_undegraded":"2024-04-17T10:43:33.504571+0000","last_fullsized":"2024-04-17T10:43:33.504571+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:02:33.405494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.470619+0000","last_change":"2024-04-17T10:43:33.470619+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.470619+0000","last_undegraded":"2024-04-17T10:43:33.470619+0000","last_fullsized":"2024-04-17T10:43:33.470619+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.511133+0000","last_change":"2024-04-17T10:43:33.509688+0000","last_active":"2024-04-17T10:43:33.511133+0000","last_peered":"2024-04-17T10:43:33.511133+0000","last_clean":"2024-04-17T10:43:33.511133+0000","last_became_active":"2024-04-17T10:43:33.508865+0000","last_became_peered":"2024-04-17T10:43:33.508865+0000","last_unstale":"2024-04-17T10:43:33.511133+0000","last_undegraded":"2024-04-17T10:43:33.511133+0000","last_fullsized":"2024-04-17T10:43:33.511133+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:11:20.457976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.470650+0000","last_change":"2024-04-17T10:43:33.470650+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.470650+0000","last_undegraded":"2024-04-17T10:43:33.470650+0000","last_fullsized":"2024-04-17T10:43:33.470650+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.509053+0000","last_change":"2024-04-17T10:43:33.507008+0000","last_active":"2024-04-17T10:43:33.509053+0000","last_peered":"2024-04-17T10:43:33.509053+0000","last_clean":"2024-04-17T10:43:33.509053+0000","last_became_active":"2024-04-17T10:43:33.506297+0000","last_became_peered":"2024-04-17T10:43:33.506297+0000","last_unstale":"2024-04-17T10:43:33.509053+0000","last_undegraded":"2024-04-17T10:43:33.509053+0000","last_fullsized":"2024-04-17T10:43:33.509053+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:36:45.162219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.468349+0000","last_change":"2024-04-17T10:43:33.468349+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.468349+0000","last_undegraded":"2024-04-17T10:43:33.468349+0000","last_fullsized":"2024-04-17T10:43:33.468349+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.509663+0000","last_change":"2024-04-17T10:43:33.508340+0000","last_active":"2024-04-17T10:43:33.509663+0000","last_peered":"2024-04-17T10:43:33.509663+0000","last_clean":"2024-04-17T10:43:33.509663+0000","last_became_active":"2024-04-17T10:43:33.507561+0000","last_became_peered":"2024-04-17T10:43:33.507561+0000","last_unstale":"2024-04-17T10:43:33.509663+0000","last_undegraded":"2024-04-17T10:43:33.509663+0000","last_fullsized":"2024-04-17T10:43:33.509663+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:17.570356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.472725+0000","last_change":"2024-04-17T10:43:33.472725+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.472725+0000","last_undegraded":"2024-04-17T10:43:33.472725+0000","last_fullsized":"2024-04-17T10:43:33.472725+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.504785+0000","last_change":"2024-04-17T10:43:33.500518+0000","last_active":"2024-04-17T10:43:33.504785+0000","last_peered":"2024-04-17T10:43:33.504785+0000","last_clean":"2024-04-17T10:43:33.504785+0000","last_became_active":"2024-04-17T10:43:33.499044+0000","last_became_peered":"2024-04-17T10:43:33.499044+0000","last_unstale":"2024-04-17T10:43:33.504785+0000","last_undegraded":"2024-04-17T10:43:33.504785+0000","last_fullsized":"2024-04-17T10:43:33.504785+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:45:49.643198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.472216+0000","last_change":"2024-04-17T10:43:33.472216+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.472216+0000","last_undegraded":"2024-04-17T10:43:33.472216+0000","last_fullsized":"2024-04-17T10:43:33.472216+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.494691+0000","last_change":"2024-04-17T10:43:33.492968+0000","last_active":"2024-04-17T10:43:33.494691+0000","last_peered":"2024-04-17T10:43:33.494691+0000","last_clean":"2024-04-17T10:43:33.494691+0000","last_became_active":"2024-04-17T10:43:33.492286+0000","last_became_peered":"2024-04-17T10:43:33.492286+0000","last_unstale":"2024-04-17T10:43:33.494691+0000","last_undegraded":"2024-04-17T10:43:33.494691+0000","last_fullsized":"2024-04-17T10:43:33.494691+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:07:18.944354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.472216+0000","last_change":"2024-04-17T10:43:33.472216+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.472216+0000","last_undegraded":"2024-04-17T10:43:33.472216+0000","last_fullsized":"2024-04-17T10:43:33.472216+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.483373+0000","last_change":"2024-04-17T10:43:33.482758+0000","last_active":"2024-04-17T10:43:33.483373+0000","last_peered":"2024-04-17T10:43:33.483373+0000","last_clean":"2024-04-17T10:43:33.483373+0000","last_became_active":"2024-04-17T10:43:33.482200+0000","last_became_peered":"2024-04-17T10:43:33.482200+0000","last_unstale":"2024-04-17T10:43:33.483373+0000","last_undegraded":"2024-04-17T10:43:33.483373+0000","last_fullsized":"2024-04-17T10:43:33.483373+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:05:45.628231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.468640+0000","last_change":"2024-04-17T10:43:33.468640+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.468640+0000","last_undegraded":"2024-04-17T10:43:33.468640+0000","last_fullsized":"2024-04-17T10:43:33.468640+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.503257+0000","last_change":"2024-04-17T10:43:33.496516+0000","last_active":"2024-04-17T10:43:33.503257+0000","last_peered":"2024-04-17T10:43:33.503257+0000","last_clean":"2024-04-17T10:43:33.503257+0000","last_became_active":"2024-04-17T10:43:33.494818+0000","last_became_peered":"2024-04-17T10:43:33.494818+0000","last_unstale":"2024-04-17T10:43:33.503257+0000","last_undegraded":"2024-04-17T10:43:33.503257+0000","last_fullsized":"2024-04-17T10:43:33.503257+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:51.195275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.472880+0000","last_change":"2024-04-17T10:43:33.472880+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.472880+0000","last_undegraded":"2024-04-17T10:43:33.472880+0000","last_fullsized":"2024-04-17T10:43:33.472880+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.492457+0000","last_change":"2024-04-17T10:43:33.488378+0000","last_active":"2024-04-17T10:43:33.492457+0000","last_peered":"2024-04-17T10:43:33.492457+0000","last_clean":"2024-04-17T10:43:33.492457+0000","last_became_active":"2024-04-17T10:43:33.487622+0000","last_became_peered":"2024-04-17T10:43:33.487622+0000","last_unstale":"2024-04-17T10:43:33.492457+0000","last_undegraded":"2024-04-17T10:43:33.492457+0000","last_fullsized":"2024-04-17T10:43:33.492457+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:54:54.413725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.466584+0000","last_change":"2024-04-17T10:43:33.466584+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.466584+0000","last_undegraded":"2024-04-17T10:43:33.466584+0000","last_fullsized":"2024-04-17T10:43:33.466584+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.482794+0000","last_change":"2024-04-17T10:43:33.480817+0000","last_active":"2024-04-17T10:43:33.482794+0000","last_peered":"2024-04-17T10:43:33.482794+0000","last_clean":"2024-04-17T10:43:33.482794+0000","last_became_active":"2024-04-17T10:43:33.480442+0000","last_became_peered":"2024-04-17T10:43:33.480442+0000","last_unstale":"2024-04-17T10:43:33.482794+0000","last_undegraded":"2024-04-17T10:43:33.482794+0000","last_fullsized":"2024-04-17T10:43:33.482794+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:45:30.015394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.467765+0000","last_change":"2024-04-17T10:43:33.467765+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.467765+0000","last_undegraded":"2024-04-17T10:43:33.467765+0000","last_fullsized":"2024-04-17T10:43:33.467765+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.510755+0000","last_change":"2024-04-17T10:43:33.510303+0000","last_active":"2024-04-17T10:43:33.510755+0000","last_peered":"2024-04-17T10:43:33.510755+0000","last_clean":"2024-04-17T10:43:33.510755+0000","last_became_active":"2024-04-17T10:43:33.509966+0000","last_became_peered":"2024-04-17T10:43:33.509966+0000","last_unstale":"2024-04-17T10:43:33.510755+0000","last_undegraded":"2024-04-17T10:43:33.510755+0000","last_fullsized":"2024-04-17T10:43:33.510755+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:54:54.413725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.468841+0000","last_change":"2024-04-17T10:43:33.468841+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.468841+0000","last_undegraded":"2024-04-17T10:43:33.468841+0000","last_fullsized":"2024-04-17T10:43:33.468841+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.490422+0000","last_change":"2024-04-17T10:43:33.489836+0000","last_active":"2024-04-17T10:43:33.490422+0000","last_peered":"2024-04-17T10:43:33.490422+0000","last_clean":"2024-04-17T10:43:33.490422+0000","last_became_active":"2024-04-17T10:43:33.489438+0000","last_became_peered":"2024-04-17T10:43:33.489438+0000","last_unstale":"2024-04-17T10:43:33.490422+0000","last_undegraded":"2024-04-17T10:43:33.490422+0000","last_fullsized":"2024-04-17T10:43:33.490422+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:35:50.957989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.488610+0000","last_change":"2024-04-17T10:43:33.488610+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.488610+0000","last_undegraded":"2024-04-17T10:43:33.488610+0000","last_fullsized":"2024-04-17T10:43:33.488610+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.504941+0000","last_change":"2024-04-17T10:43:33.503755+0000","last_active":"2024-04-17T10:43:33.504941+0000","last_peered":"2024-04-17T10:43:33.504941+0000","last_clean":"2024-04-17T10:43:33.504941+0000","last_became_active":"2024-04-17T10:43:33.503377+0000","last_became_peered":"2024-04-17T10:43:33.503377+0000","last_unstale":"2024-04-17T10:43:33.504941+0000","last_undegraded":"2024-04-17T10:43:33.504941+0000","last_fullsized":"2024-04-17T10:43:33.504941+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:36:45.162219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.466736+0000","last_change":"2024-04-17T10:43:33.466736+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.466736+0000","last_undegraded":"2024-04-17T10:43:33.466736+0000","last_fullsized":"2024-04-17T10:43:33.466736+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.510946+0000","last_change":"2024-04-17T10:43:33.506065+0000","last_active":"2024-04-17T10:43:33.510946+0000","last_peered":"2024-04-17T10:43:33.510946+0000","last_clean":"2024-04-17T10:43:33.510946+0000","last_became_active":"2024-04-17T10:43:33.504808+0000","last_became_peered":"2024-04-17T10:43:33.504808+0000","last_unstale":"2024-04-17T10:43:33.510946+0000","last_undegraded":"2024-04-17T10:43:33.510946+0000","last_fullsized":"2024-04-17T10:43:33.510946+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:01:00.790513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.476069+0000","last_change":"2024-04-17T10:43:33.476069+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.476069+0000","last_undegraded":"2024-04-17T10:43:33.476069+0000","last_fullsized":"2024-04-17T10:43:33.476069+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.493140+0000","last_change":"2024-04-17T10:43:33.487447+0000","last_active":"2024-04-17T10:43:33.493140+0000","last_peered":"2024-04-17T10:43:33.493140+0000","last_clean":"2024-04-17T10:43:33.493140+0000","last_became_active":"2024-04-17T10:43:33.486690+0000","last_became_peered":"2024-04-17T10:43:33.486690+0000","last_unstale":"2024-04-17T10:43:33.493140+0000","last_undegraded":"2024-04-17T10:43:33.493140+0000","last_fullsized":"2024-04-17T10:43:33.493140+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:44:20.188944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-17T10:43:33.472418+0000","last_change":"2024-04-17T10:43:33.472418+0000","last_active":"2024-04-17T10:43:33.454791+0000","last_peered":"2024-04-17T10:43:33.454791+0000","last_clean":"2024-04-17T10:43:33.454791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:43:33.472418+0000","last_undegraded":"2024-04-17T10:43:33.472418+0000","last_fullsized":"2024-04-17T10:43:33.472418+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.497416+0000","last_change":"2024-04-17T10:43:33.495411+0000","last_active":"2024-04-17T10:43:33.497416+0000","last_peered":"2024-04-17T10:43:33.497416+0000","last_clean":"2024-04-17T10:43:33.497416+0000","last_became_active":"2024-04-17T10:43:33.494671+0000","last_became_peered":"2024-04-17T10:43:33.494671+0000","last_unstale":"2024-04-17T10:43:33.497416+0000","last_undegraded":"2024-04-17T10:43:33.497416+0000","last_fullsized":"2024-04-17T10:43:33.497416+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:42:52.241171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":276,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-17T10:43:33.476774+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:43:33.476774+0000","last_peered":"2024-04-17T10:43:33.476774+0000","last_clean":"2024-04-17T10:43:33.476774+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:43:33.476774+0000","last_undegraded":"2024-04-17T10:43:33.476774+0000","last_fullsized":"2024-04-17T10:43:33.476774+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574919,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28460,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343380,"statfs":{"total":96636764160,"available":96607621120,"internally_reserved":0,"allocated":1470464,"data_stored":962292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40360,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331480,"statfs":{"total":96636764160,"available":96595435520,"internally_reserved":0,"allocated":876544,"data_stored":371924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574919,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40360,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331480,"statfs":{"total":96636764160,"available":96595435520,"internally_reserved":0,"allocated":876544,"data_stored":371924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50084,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321756,"statfs":{"total":96636764160,"available":96585478144,"internally_reserved":0,"allocated":876544,"data_stored":371924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574920,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54248,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317592,"statfs":{"total":96636764160,"available":96581214208,"internally_reserved":0,"allocated":876544,"data_stored":371924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574918,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36200,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335640,"statfs":{"total":96636764160,"available":96599695360,"internally_reserved":0,"allocated":876544,"data_stored":371924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50088,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321752,"statfs":{"total":96636764160,"available":96585474048,"internally_reserved":0,"allocated":876544,"data_stored":371924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574918,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45096,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326744,"statfs":{"total":96636764160,"available":96590585856,"internally_reserved":0,"allocated":1470464,"data_stored":962292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:43:42.873 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:43:43.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.009+0000 7f3fb9df1640 1 Processor -- start 2024-04-17T10:43:43.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.009+0000 7f3fb9df1640 1 -- start start 2024-04-17T10:43:43.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.009+0000 7f3fb9df1640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3fb415db50 0x7f3fb4070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:43.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.009+0000 7f3fb9df1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fb40aedd0 0x7f3fb406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:43.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.009+0000 7f3fb9df1640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fb40709b0 con 0x7f3fb406dae0 2024-04-17T10:43:43.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.009+0000 7f3fb9df1640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fb415e440 con 0x7f3fb415db50 2024-04-17T10:43:43.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.009+0000 7f3fb9df1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb415e720 con 0x7f3fb40aedd0 2024-04-17T10:43:43.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.009+0000 7f3fb2ffd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fb40aedd0 0x7f3fb406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:43.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.009+0000 7f3fb37fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3fb415db50 0x7f3fb4070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:43.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.009+0000 7f3fb37fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3fb415db50 0x7f3fb4070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:54136/0 (socket says 172.21.15.83:54136) 2024-04-17T10:43:43.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.009+0000 7f3fb37fe640 1 -- 172.21.15.83:0/3403773813 learned_addr learned my addr 172.21.15.83:0/3403773813 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:43.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.009+0000 7f3fb3fff640 1 --1- 172.21.15.83:0/3403773813 >> v1:172.21.15.167:6789/0 conn(0x7f3fb406dae0 0x7f3fb40cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:36004/0 (socket says 172.21.15.83:36004) 2024-04-17T10:43:43.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.013+0000 7f3fb37fe640 1 -- 172.21.15.83:0/3403773813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fb40aedd0 msgr2=0x7f3fb406d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:43.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.013+0000 7f3fb37fe640 1 --2- 172.21.15.83:0/3403773813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fb40aedd0 0x7f3fb406d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:43.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.013+0000 7f3fb37fe640 1 -- 172.21.15.83:0/3403773813 >> v1:172.21.15.167:6789/0 conn(0x7f3fb406dae0 legacy=0x7f3fb40cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:43.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.013+0000 7f3fb37fe640 1 -- 172.21.15.83:0/3403773813 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb415ea00 con 0x7f3fb415db50 2024-04-17T10:43:43.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.013+0000 7f3fb37fe640 1 --2- 172.21.15.83:0/3403773813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fb415db50 0x7f3fb4070470 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4009870 tx=0x7f3fa401c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec45ebe313ef1d1c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:43.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.013+0000 7f3fb27fc640 1 -- 172.21.15.83:0/3403773813 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa40ab650 con 0x7f3fb415db50 2024-04-17T10:43:43.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.013+0000 7f3fb27fc640 1 -- 172.21.15.83:0/3403773813 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa40b0070 con 0x7f3fb415db50 2024-04-17T10:43:43.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.013+0000 7f3fb27fc640 1 -- 172.21.15.83:0/3403773813 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa40aba10 con 0x7f3fb415db50 2024-04-17T10:43:43.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.013+0000 7f3fb9df1640 1 -- 172.21.15.83:0/3403773813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fb415db50 msgr2=0x7f3fb4070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:43.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.013+0000 7f3fb9df1640 1 --2- 172.21.15.83:0/3403773813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fb415db50 0x7f3fb4070470 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4009870 tx=0x7f3fa401c560 comp rx=0 tx=0).stop 2024-04-17T10:43:43.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.013+0000 7f3fb9df1640 1 -- 172.21.15.83:0/3403773813 shutdown_connections 2024-04-17T10:43:43.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.013+0000 7f3fb9df1640 1 --2- 172.21.15.83:0/3403773813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fb40aedd0 0x7f3fb406d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:43.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.013+0000 7f3fb9df1640 1 --2- 172.21.15.83:0/3403773813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fb415db50 0x7f3fb4070470 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:43.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.013+0000 7f3fb9df1640 1 -- 172.21.15.83:0/3403773813 >> 172.21.15.83:0/3403773813 conn(0x7f3fb406abe0 msgr2=0x7f3fb406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:43.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.013+0000 7f3fb9df1640 1 -- 172.21.15.83:0/3403773813 shutdown_connections 2024-04-17T10:43:43.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.013+0000 7f3fb9df1640 1 -- 172.21.15.83:0/3403773813 wait complete. 2024-04-17T10:43:43.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.017+0000 7f3fb9df1640 1 Processor -- start 2024-04-17T10:43:43.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.017+0000 7f3fb9df1640 1 -- start start 2024-04-17T10:43:43.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.017+0000 7f3fb9df1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb406dae0 0x7f3fb4164e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:43.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.017+0000 7f3fb9df1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fb40aedd0 0x7f3fb41515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:43.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.017+0000 7f3fb9df1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fb415db50 0x7f3fb40ade20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:43.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.017+0000 7f3fb9df1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb406c610 con 0x7f3fb406dae0 2024-04-17T10:43:43.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.017+0000 7f3fb9df1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb406c490 con 0x7f3fb40aedd0 2024-04-17T10:43:43.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.017+0000 7f3fb9df1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb406c790 con 0x7f3fb415db50 2024-04-17T10:43:43.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.017+0000 7f3fb37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb406dae0 0x7f3fb4164e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:43.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.017+0000 7f3fb37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb406dae0 0x7f3fb4164e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52406/0 (socket says 172.21.15.83:52406) 2024-04-17T10:43:43.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.017+0000 7f3fb37fe640 1 -- 172.21.15.83:0/824775053 learned_addr learned my addr 172.21.15.83:0/824775053 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:43.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.017+0000 7f3fb2ffd640 1 --2- 172.21.15.83:0/824775053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fb40aedd0 0x7f3fb41515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:43.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.017+0000 7f3fb3fff640 1 --2- 172.21.15.83:0/824775053 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fb415db50 0x7f3fb40ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:43.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.017+0000 7f3fb37fe640 1 -- 172.21.15.83:0/824775053 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fb415db50 msgr2=0x7f3fb40ade20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:43.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.017+0000 7f3fb37fe640 1 --2- 172.21.15.83:0/824775053 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fb415db50 0x7f3fb40ade20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:43.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.017+0000 7f3fb37fe640 1 -- 172.21.15.83:0/824775053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fb40aedd0 msgr2=0x7f3fb41515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:43.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.017+0000 7f3fb37fe640 1 --2- 172.21.15.83:0/824775053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fb40aedd0 0x7f3fb41515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:43.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.017+0000 7f3fb37fe640 1 -- 172.21.15.83:0/824775053 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb41551b0 con 0x7f3fb406dae0 2024-04-17T10:43:43.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.017+0000 7f3fb2ffd640 1 --2- 172.21.15.83:0/824775053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fb40aedd0 0x7f3fb41515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:43:43.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.017+0000 7f3fb37fe640 1 --2- 172.21.15.83:0/824775053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb406dae0 0x7f3fb4164e10 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f3fa40ad040 tx=0x7f3fa401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:43.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.017+0000 7f3f9bfff640 1 -- 172.21.15.83:0/824775053 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa40be5f0 con 0x7f3fb406dae0 2024-04-17T10:43:43.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.017+0000 7f3f9bfff640 1 -- 172.21.15.83:0/824775053 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa40be790 con 0x7f3fb406dae0 2024-04-17T10:43:43.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.017+0000 7f3fb9df1640 1 -- 172.21.15.83:0/824775053 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fb4154940 con 0x7f3fb406dae0 2024-04-17T10:43:43.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.017+0000 7f3f9bfff640 1 -- 172.21.15.83:0/824775053 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa40beb60 con 0x7f3fb406dae0 2024-04-17T10:43:43.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.017+0000 7f3fb9df1640 1 -- 172.21.15.83:0/824775053 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fb415b280 con 0x7f3fb406dae0 2024-04-17T10:43:43.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.017+0000 7f3fb9df1640 1 -- 172.21.15.83:0/824775053 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fb4074f90 con 0x7f3fb406dae0 2024-04-17T10:43:43.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.021+0000 7f3f9bfff640 1 -- 172.21.15.83:0/824775053 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3fa40b43f0 con 0x7f3fb406dae0 2024-04-17T10:43:43.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.021+0000 7f3f9bfff640 1 --2- 172.21.15.83:0/824775053 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3fa0066f60 0x7f3fa0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:43.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.021+0000 7f3f9bfff640 1 -- 172.21.15.83:0/824775053 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3fa411fdd0 con 0x7f3fb406dae0 2024-04-17T10:43:43.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.021+0000 7f3fb2ffd640 1 --2- 172.21.15.83:0/824775053 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3fa0066f60 0x7f3fa0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:43.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.021+0000 7f3fb2ffd640 1 --2- 172.21.15.83:0/824775053 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3fa0066f60 0x7f3fa0069420 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3fb40b3580 tx=0x7f3fa8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:43.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.025+0000 7f3f9bfff640 1 -- 172.21.15.83:0/824775053 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3fb4154940 con 0x7f3fb406dae0 2024-04-17T10:43:43.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.177+0000 7f3fb9df1640 1 -- 172.21.15.83:0/824775053 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3fb40b2af0 con 0x7f3fa0066f60 2024-04-17T10:43:43.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.189+0000 7f3f9bfff640 1 -- 172.21.15.83:0/824775053 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252069 (secure 0 0 0) 0x7f3fb40b2af0 con 0x7f3fa0066f60 2024-04-17T10:43:43.189 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:43:43.190 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:43:43.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.193+0000 7f3fb9df1640 1 -- 172.21.15.83:0/824775053 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3fa0066f60 msgr2=0x7f3fa0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:43.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.193+0000 7f3fb9df1640 1 --2- 172.21.15.83:0/824775053 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3fa0066f60 0x7f3fa0069420 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3fb40b3580 tx=0x7f3fa8008040 comp rx=0 tx=0).stop 2024-04-17T10:43:43.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.193+0000 7f3fb9df1640 1 -- 172.21.15.83:0/824775053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb406dae0 msgr2=0x7f3fb4164e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:43.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.193+0000 7f3fb9df1640 1 --2- 172.21.15.83:0/824775053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb406dae0 0x7f3fb4164e10 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f3fa40ad040 tx=0x7f3fa401ada0 comp rx=0 tx=0).stop 2024-04-17T10:43:43.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.193+0000 7f3fb9df1640 1 -- 172.21.15.83:0/824775053 shutdown_connections 2024-04-17T10:43:43.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.193+0000 7f3fb9df1640 1 --2- 172.21.15.83:0/824775053 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3fa0066f60 0x7f3fa0069420 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:43.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.193+0000 7f3fb9df1640 1 --2- 172.21.15.83:0/824775053 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fb415db50 0x7f3fb40ade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:43.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.193+0000 7f3fb9df1640 1 --2- 172.21.15.83:0/824775053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fb40aedd0 0x7f3fb41515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:43.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.193+0000 7f3fb9df1640 1 --2- 172.21.15.83:0/824775053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb406dae0 0x7f3fb4164e10 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:43.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.193+0000 7f3fb9df1640 1 -- 172.21.15.83:0/824775053 >> 172.21.15.83:0/824775053 conn(0x7f3fb406abe0 msgr2=0x7f3fb40b00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:43.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.193+0000 7f3fb9df1640 1 -- 172.21.15.83:0/824775053 shutdown_connections 2024-04-17T10:43:43.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.193+0000 7f3fb9df1640 1 -- 172.21.15.83:0/824775053 wait complete. 2024-04-17T10:43:43.207 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":239,"stamp":"2024-04-17T10:43:42.280086+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":363204,"kb_used_data":8264,"kb_used_omap":12,"kb_used_meta":354803,"kb_avail":754611516,"statfs":{"total":773094113280,"available":772722192384,"internally_reserved":0,"allocated":8462336,"data_stored":4265548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":363318832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002721"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.511427+0000","last_change":"2024-04-17T10:43:34.506782+0000","last_active":"2024-04-17T10:43:35.511427+0000","last_peered":"2024-04-17T10:43:35.511427+0000","last_clean":"2024-04-17T10:43:35.511427+0000","last_became_active":"2024-04-17T10:43:34.506443+0000","last_became_peered":"2024-04-17T10:43:34.506443+0000","last_unstale":"2024-04-17T10:43:35.511427+0000","last_undegraded":"2024-04-17T10:43:35.511427+0000","last_fullsized":"2024-04-17T10:43:35.511427+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:28:55.370355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.670944+0000","last_change":"2024-04-17T10:43:34.519933+0000","last_active":"2024-04-17T10:43:35.670944+0000","last_peered":"2024-04-17T10:43:35.670944+0000","last_clean":"2024-04-17T10:43:35.670944+0000","last_became_active":"2024-04-17T10:43:34.519515+0000","last_became_peered":"2024-04-17T10:43:34.519515+0000","last_unstale":"2024-04-17T10:43:35.670944+0000","last_undegraded":"2024-04-17T10:43:35.670944+0000","last_fullsized":"2024-04-17T10:43:35.670944+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:54:55.423886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.925895+0000","last_change":"2024-04-17T10:43:34.516906+0000","last_active":"2024-04-17T10:43:35.925895+0000","last_peered":"2024-04-17T10:43:35.925895+0000","last_clean":"2024-04-17T10:43:35.925895+0000","last_became_active":"2024-04-17T10:43:34.514886+0000","last_became_peered":"2024-04-17T10:43:34.514886+0000","last_unstale":"2024-04-17T10:43:35.925895+0000","last_undegraded":"2024-04-17T10:43:35.925895+0000","last_fullsized":"2024-04-17T10:43:35.925895+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:26:22.342709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.923904+0000","last_change":"2024-04-17T10:43:34.498070+0000","last_active":"2024-04-17T10:43:35.923904+0000","last_peered":"2024-04-17T10:43:35.923904+0000","last_clean":"2024-04-17T10:43:35.923904+0000","last_became_active":"2024-04-17T10:43:34.497431+0000","last_became_peered":"2024-04-17T10:43:34.497431+0000","last_unstale":"2024-04-17T10:43:35.923904+0000","last_undegraded":"2024-04-17T10:43:35.923904+0000","last_fullsized":"2024-04-17T10:43:35.923904+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:54:20.089633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.923072+0000","last_change":"2024-04-17T10:43:34.515886+0000","last_active":"2024-04-17T10:43:35.923072+0000","last_peered":"2024-04-17T10:43:35.923072+0000","last_clean":"2024-04-17T10:43:35.923072+0000","last_became_active":"2024-04-17T10:43:34.515497+0000","last_became_peered":"2024-04-17T10:43:34.515497+0000","last_unstale":"2024-04-17T10:43:35.923072+0000","last_undegraded":"2024-04-17T10:43:35.923072+0000","last_fullsized":"2024-04-17T10:43:35.923072+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:36:46.172381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.670742+0000","last_change":"2024-04-17T10:43:34.512751+0000","last_active":"2024-04-17T10:43:35.670742+0000","last_peered":"2024-04-17T10:43:35.670742+0000","last_clean":"2024-04-17T10:43:35.670742+0000","last_became_active":"2024-04-17T10:43:34.510446+0000","last_became_peered":"2024-04-17T10:43:34.510446+0000","last_unstale":"2024-04-17T10:43:35.670742+0000","last_undegraded":"2024-04-17T10:43:35.670742+0000","last_fullsized":"2024-04-17T10:43:35.670742+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:05:46.638392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.515879+0000","last_change":"2024-04-17T10:43:34.517409+0000","last_active":"2024-04-17T10:43:35.515879+0000","last_peered":"2024-04-17T10:43:35.515879+0000","last_clean":"2024-04-17T10:43:35.515879+0000","last_became_active":"2024-04-17T10:43:34.517060+0000","last_became_peered":"2024-04-17T10:43:34.517060+0000","last_unstale":"2024-04-17T10:43:35.515879+0000","last_undegraded":"2024-04-17T10:43:35.515879+0000","last_fullsized":"2024-04-17T10:43:35.515879+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:17:46.702817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.669872+0000","last_change":"2024-04-17T10:43:34.512715+0000","last_active":"2024-04-17T10:43:35.669872+0000","last_peered":"2024-04-17T10:43:35.669872+0000","last_clean":"2024-04-17T10:43:35.669872+0000","last_became_active":"2024-04-17T10:43:34.512375+0000","last_became_peered":"2024-04-17T10:43:34.512375+0000","last_unstale":"2024-04-17T10:43:35.669872+0000","last_undegraded":"2024-04-17T10:43:35.669872+0000","last_fullsized":"2024-04-17T10:43:35.669872+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:12:09.212432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.922993+0000","last_change":"2024-04-17T10:43:34.507042+0000","last_active":"2024-04-17T10:43:35.922993+0000","last_peered":"2024-04-17T10:43:35.922993+0000","last_clean":"2024-04-17T10:43:35.922993+0000","last_became_active":"2024-04-17T10:43:34.506198+0000","last_became_peered":"2024-04-17T10:43:34.506198+0000","last_unstale":"2024-04-17T10:43:35.922993+0000","last_undegraded":"2024-04-17T10:43:35.922993+0000","last_fullsized":"2024-04-17T10:43:35.922993+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:45:50.653360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.514053+0000","last_change":"2024-04-17T10:43:34.518669+0000","last_active":"2024-04-17T10:43:35.514053+0000","last_peered":"2024-04-17T10:43:35.514053+0000","last_clean":"2024-04-17T10:43:35.514053+0000","last_became_active":"2024-04-17T10:43:34.518338+0000","last_became_peered":"2024-04-17T10:43:34.518338+0000","last_unstale":"2024-04-17T10:43:35.514053+0000","last_undegraded":"2024-04-17T10:43:35.514053+0000","last_fullsized":"2024-04-17T10:43:35.514053+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:29:04.316734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.510963+0000","last_change":"2024-04-17T10:43:34.505118+0000","last_active":"2024-04-17T10:43:35.510963+0000","last_peered":"2024-04-17T10:43:35.510963+0000","last_clean":"2024-04-17T10:43:35.510963+0000","last_became_active":"2024-04-17T10:43:34.504491+0000","last_became_peered":"2024-04-17T10:43:34.504491+0000","last_unstale":"2024-04-17T10:43:35.510963+0000","last_undegraded":"2024-04-17T10:43:35.510963+0000","last_fullsized":"2024-04-17T10:43:35.510963+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:37:10.720293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.924301+0000","last_change":"2024-04-17T10:43:34.504598+0000","last_active":"2024-04-17T10:43:35.924301+0000","last_peered":"2024-04-17T10:43:35.924301+0000","last_clean":"2024-04-17T10:43:35.924301+0000","last_became_active":"2024-04-17T10:43:34.503719+0000","last_became_peered":"2024-04-17T10:43:34.503719+0000","last_unstale":"2024-04-17T10:43:35.924301+0000","last_undegraded":"2024-04-17T10:43:35.924301+0000","last_fullsized":"2024-04-17T10:43:35.924301+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:05:04.664339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.508965+0000","last_change":"2024-04-17T10:43:34.505547+0000","last_active":"2024-04-17T10:43:35.508965+0000","last_peered":"2024-04-17T10:43:35.508965+0000","last_clean":"2024-04-17T10:43:35.508965+0000","last_became_active":"2024-04-17T10:43:34.505223+0000","last_became_peered":"2024-04-17T10:43:34.505223+0000","last_unstale":"2024-04-17T10:43:35.508965+0000","last_undegraded":"2024-04-17T10:43:35.508965+0000","last_fullsized":"2024-04-17T10:43:35.508965+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:58:17.012943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.925014+0000","last_change":"2024-04-17T10:43:34.516937+0000","last_active":"2024-04-17T10:43:35.925014+0000","last_peered":"2024-04-17T10:43:35.925014+0000","last_clean":"2024-04-17T10:43:35.925014+0000","last_became_active":"2024-04-17T10:43:34.516312+0000","last_became_peered":"2024-04-17T10:43:34.516312+0000","last_unstale":"2024-04-17T10:43:35.925014+0000","last_undegraded":"2024-04-17T10:43:35.925014+0000","last_fullsized":"2024-04-17T10:43:35.925014+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:33:23.255777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.508820+0000","last_change":"2024-04-17T10:43:34.505268+0000","last_active":"2024-04-17T10:43:35.508820+0000","last_peered":"2024-04-17T10:43:35.508820+0000","last_clean":"2024-04-17T10:43:35.508820+0000","last_became_active":"2024-04-17T10:43:34.504761+0000","last_became_peered":"2024-04-17T10:43:34.504761+0000","last_unstale":"2024-04-17T10:43:35.508820+0000","last_undegraded":"2024-04-17T10:43:35.508820+0000","last_fullsized":"2024-04-17T10:43:35.508820+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:07:09.761324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.512867+0000","last_change":"2024-04-17T10:43:34.515366+0000","last_active":"2024-04-17T10:43:35.512867+0000","last_peered":"2024-04-17T10:43:35.512867+0000","last_clean":"2024-04-17T10:43:35.512867+0000","last_became_active":"2024-04-17T10:43:34.514987+0000","last_became_peered":"2024-04-17T10:43:34.514987+0000","last_unstale":"2024-04-17T10:43:35.512867+0000","last_undegraded":"2024-04-17T10:43:35.512867+0000","last_fullsized":"2024-04-17T10:43:35.512867+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:18.580518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.924044+0000","last_change":"2024-04-17T10:43:34.507130+0000","last_active":"2024-04-17T10:43:35.924044+0000","last_peered":"2024-04-17T10:43:35.924044+0000","last_clean":"2024-04-17T10:43:35.924044+0000","last_became_active":"2024-04-17T10:43:34.506033+0000","last_became_peered":"2024-04-17T10:43:34.506033+0000","last_unstale":"2024-04-17T10:43:35.924044+0000","last_undegraded":"2024-04-17T10:43:35.924044+0000","last_fullsized":"2024-04-17T10:43:35.924044+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:41:28.110368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.926258+0000","last_change":"2024-04-17T10:43:34.498059+0000","last_active":"2024-04-17T10:43:35.926258+0000","last_peered":"2024-04-17T10:43:35.926258+0000","last_clean":"2024-04-17T10:43:35.926258+0000","last_became_active":"2024-04-17T10:43:34.497327+0000","last_became_peered":"2024-04-17T10:43:34.497327+0000","last_unstale":"2024-04-17T10:43:35.926258+0000","last_undegraded":"2024-04-17T10:43:35.926258+0000","last_fullsized":"2024-04-17T10:43:35.926258+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:34:01.770084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.670024+0000","last_change":"2024-04-17T10:43:34.511951+0000","last_active":"2024-04-17T10:43:35.670024+0000","last_peered":"2024-04-17T10:43:35.670024+0000","last_clean":"2024-04-17T10:43:35.670024+0000","last_became_active":"2024-04-17T10:43:34.511615+0000","last_became_peered":"2024-04-17T10:43:34.511615+0000","last_unstale":"2024-04-17T10:43:35.670024+0000","last_undegraded":"2024-04-17T10:43:35.670024+0000","last_fullsized":"2024-04-17T10:43:35.670024+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:18:51.169499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.508927+0000","last_change":"2024-04-17T10:43:34.516225+0000","last_active":"2024-04-17T10:43:35.508927+0000","last_peered":"2024-04-17T10:43:35.508927+0000","last_clean":"2024-04-17T10:43:35.508927+0000","last_became_active":"2024-04-17T10:43:34.515797+0000","last_became_peered":"2024-04-17T10:43:34.515797+0000","last_unstale":"2024-04-17T10:43:35.508927+0000","last_undegraded":"2024-04-17T10:43:35.508927+0000","last_fullsized":"2024-04-17T10:43:35.508927+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:20:42.718901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.513369+0000","last_change":"2024-04-17T10:43:34.518748+0000","last_active":"2024-04-17T10:43:35.513369+0000","last_peered":"2024-04-17T10:43:35.513369+0000","last_clean":"2024-04-17T10:43:35.513369+0000","last_became_active":"2024-04-17T10:43:34.518388+0000","last_became_peered":"2024-04-17T10:43:34.518388+0000","last_unstale":"2024-04-17T10:43:35.513369+0000","last_undegraded":"2024-04-17T10:43:35.513369+0000","last_fullsized":"2024-04-17T10:43:35.513369+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:31:32.961244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.513012+0000","last_change":"2024-04-17T10:43:34.514394+0000","last_active":"2024-04-17T10:43:35.513012+0000","last_peered":"2024-04-17T10:43:35.513012+0000","last_clean":"2024-04-17T10:43:35.513012+0000","last_became_active":"2024-04-17T10:43:34.514080+0000","last_became_peered":"2024-04-17T10:43:34.514080+0000","last_unstale":"2024-04-17T10:43:35.513012+0000","last_undegraded":"2024-04-17T10:43:35.513012+0000","last_fullsized":"2024-04-17T10:43:35.513012+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:46:02.797033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.510950+0000","last_change":"2024-04-17T10:43:34.515738+0000","last_active":"2024-04-17T10:43:35.510950+0000","last_peered":"2024-04-17T10:43:35.510950+0000","last_clean":"2024-04-17T10:43:35.510950+0000","last_became_active":"2024-04-17T10:43:34.515329+0000","last_became_peered":"2024-04-17T10:43:34.515329+0000","last_unstale":"2024-04-17T10:43:35.510950+0000","last_undegraded":"2024-04-17T10:43:35.510950+0000","last_fullsized":"2024-04-17T10:43:35.510950+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:53:37.655647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.510501+0000","last_change":"2024-04-17T10:43:34.499682+0000","last_active":"2024-04-17T10:43:35.510501+0000","last_peered":"2024-04-17T10:43:35.510501+0000","last_clean":"2024-04-17T10:43:35.510501+0000","last_became_active":"2024-04-17T10:43:34.498881+0000","last_became_peered":"2024-04-17T10:43:34.498881+0000","last_unstale":"2024-04-17T10:43:35.510501+0000","last_undegraded":"2024-04-17T10:43:35.510501+0000","last_fullsized":"2024-04-17T10:43:35.510501+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:08:25.575135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.514214+0000","last_change":"2024-04-17T10:43:34.524664+0000","last_active":"2024-04-17T10:43:35.514214+0000","last_peered":"2024-04-17T10:43:35.514214+0000","last_clean":"2024-04-17T10:43:35.514214+0000","last_became_active":"2024-04-17T10:43:34.524231+0000","last_became_peered":"2024-04-17T10:43:34.524231+0000","last_unstale":"2024-04-17T10:43:35.514214+0000","last_undegraded":"2024-04-17T10:43:35.514214+0000","last_fullsized":"2024-04-17T10:43:35.514214+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:20:42.718901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.515363+0000","last_change":"2024-04-17T10:43:34.514899+0000","last_active":"2024-04-17T10:43:35.515363+0000","last_peered":"2024-04-17T10:43:35.515363+0000","last_clean":"2024-04-17T10:43:35.515363+0000","last_became_active":"2024-04-17T10:43:34.514626+0000","last_became_peered":"2024-04-17T10:43:34.514626+0000","last_unstale":"2024-04-17T10:43:35.515363+0000","last_undegraded":"2024-04-17T10:43:35.515363+0000","last_fullsized":"2024-04-17T10:43:35.515363+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:48.222470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.926177+0000","last_change":"2024-04-17T10:43:34.521068+0000","last_active":"2024-04-17T10:43:35.926177+0000","last_peered":"2024-04-17T10:43:35.926177+0000","last_clean":"2024-04-17T10:43:35.926177+0000","last_became_active":"2024-04-17T10:43:34.520082+0000","last_became_peered":"2024-04-17T10:43:34.520082+0000","last_unstale":"2024-04-17T10:43:35.926177+0000","last_undegraded":"2024-04-17T10:43:35.926177+0000","last_fullsized":"2024-04-17T10:43:35.926177+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:42:07.542193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.513761+0000","last_change":"2024-04-17T10:43:34.513805+0000","last_active":"2024-04-17T10:43:35.513761+0000","last_peered":"2024-04-17T10:43:35.513761+0000","last_clean":"2024-04-17T10:43:35.513761+0000","last_became_active":"2024-04-17T10:43:34.513443+0000","last_became_peered":"2024-04-17T10:43:34.513443+0000","last_unstale":"2024-04-17T10:43:35.513761+0000","last_undegraded":"2024-04-17T10:43:35.513761+0000","last_fullsized":"2024-04-17T10:43:35.513761+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:33:32.966906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.925695+0000","last_change":"2024-04-17T10:43:34.521143+0000","last_active":"2024-04-17T10:43:35.925695+0000","last_peered":"2024-04-17T10:43:35.925695+0000","last_clean":"2024-04-17T10:43:35.925695+0000","last_became_active":"2024-04-17T10:43:34.520327+0000","last_became_peered":"2024-04-17T10:43:34.520327+0000","last_unstale":"2024-04-17T10:43:35.925695+0000","last_undegraded":"2024-04-17T10:43:35.925695+0000","last_fullsized":"2024-04-17T10:43:35.925695+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:57:47.416037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.515264+0000","last_change":"2024-04-17T10:43:34.515966+0000","last_active":"2024-04-17T10:43:35.515264+0000","last_peered":"2024-04-17T10:43:35.515264+0000","last_clean":"2024-04-17T10:43:35.515264+0000","last_became_active":"2024-04-17T10:43:34.515587+0000","last_became_peered":"2024-04-17T10:43:34.515587+0000","last_unstale":"2024-04-17T10:43:35.515264+0000","last_undegraded":"2024-04-17T10:43:35.515264+0000","last_fullsized":"2024-04-17T10:43:35.515264+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:09:50.041661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.515354+0000","last_change":"2024-04-17T10:43:34.513504+0000","last_active":"2024-04-17T10:43:35.515354+0000","last_peered":"2024-04-17T10:43:35.515354+0000","last_clean":"2024-04-17T10:43:35.515354+0000","last_became_active":"2024-04-17T10:43:34.513168+0000","last_became_peered":"2024-04-17T10:43:34.513168+0000","last_unstale":"2024-04-17T10:43:35.515354+0000","last_undegraded":"2024-04-17T10:43:35.515354+0000","last_fullsized":"2024-04-17T10:43:35.515354+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:13:08.068679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.670280+0000","last_change":"2024-04-17T10:43:34.522092+0000","last_active":"2024-04-17T10:43:35.670280+0000","last_peered":"2024-04-17T10:43:35.670280+0000","last_clean":"2024-04-17T10:43:35.670280+0000","last_became_active":"2024-04-17T10:43:34.521779+0000","last_became_peered":"2024-04-17T10:43:34.521779+0000","last_unstale":"2024-04-17T10:43:35.670280+0000","last_undegraded":"2024-04-17T10:43:35.670280+0000","last_fullsized":"2024-04-17T10:43:35.670280+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:46:45.677528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.511207+0000","last_change":"2024-04-17T10:43:34.501264+0000","last_active":"2024-04-17T10:43:35.511207+0000","last_peered":"2024-04-17T10:43:35.511207+0000","last_clean":"2024-04-17T10:43:35.511207+0000","last_became_active":"2024-04-17T10:43:34.500514+0000","last_became_peered":"2024-04-17T10:43:34.500514+0000","last_unstale":"2024-04-17T10:43:35.511207+0000","last_undegraded":"2024-04-17T10:43:35.511207+0000","last_fullsized":"2024-04-17T10:43:35.511207+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:05:46.477914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"73'4","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.512078+0000","last_change":"2024-04-17T10:43:33.488363+0000","last_active":"2024-04-17T10:43:35.512078+0000","last_peered":"2024-04-17T10:43:35.512078+0000","last_clean":"2024-04-17T10:43:35.512078+0000","last_became_active":"2024-04-17T10:43:33.487677+0000","last_became_peered":"2024-04-17T10:43:33.487677+0000","last_unstale":"2024-04-17T10:43:35.512078+0000","last_undegraded":"2024-04-17T10:43:35.512078+0000","last_fullsized":"2024-04-17T10:43:35.512078+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:31:31.951082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.924073+0000","last_change":"2024-04-17T10:43:34.506567+0000","last_active":"2024-04-17T10:43:35.924073+0000","last_peered":"2024-04-17T10:43:35.924073+0000","last_clean":"2024-04-17T10:43:35.924073+0000","last_became_active":"2024-04-17T10:43:34.505125+0000","last_became_peered":"2024-04-17T10:43:34.505125+0000","last_unstale":"2024-04-17T10:43:35.924073+0000","last_undegraded":"2024-04-17T10:43:35.924073+0000","last_fullsized":"2024-04-17T10:43:35.924073+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:31:44.776465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.509177+0000","last_change":"2024-04-17T10:43:33.478164+0000","last_active":"2024-04-17T10:43:35.509177+0000","last_peered":"2024-04-17T10:43:35.509177+0000","last_clean":"2024-04-17T10:43:35.509177+0000","last_became_active":"2024-04-17T10:43:33.477765+0000","last_became_peered":"2024-04-17T10:43:33.477765+0000","last_unstale":"2024-04-17T10:43:35.509177+0000","last_undegraded":"2024-04-17T10:43:35.509177+0000","last_fullsized":"2024-04-17T10:43:35.509177+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:48:07.131314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.670443+0000","last_change":"2024-04-17T10:43:34.500531+0000","last_active":"2024-04-17T10:43:35.670443+0000","last_peered":"2024-04-17T10:43:35.670443+0000","last_clean":"2024-04-17T10:43:35.670443+0000","last_became_active":"2024-04-17T10:43:34.499671+0000","last_became_peered":"2024-04-17T10:43:34.499671+0000","last_unstale":"2024-04-17T10:43:35.670443+0000","last_undegraded":"2024-04-17T10:43:35.670443+0000","last_fullsized":"2024-04-17T10:43:35.670443+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:11:29.008974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.514002+0000","last_change":"2024-04-17T10:43:33.504903+0000","last_active":"2024-04-17T10:43:35.514002+0000","last_peered":"2024-04-17T10:43:35.514002+0000","last_clean":"2024-04-17T10:43:35.514002+0000","last_became_active":"2024-04-17T10:43:33.503926+0000","last_became_peered":"2024-04-17T10:43:33.503926+0000","last_unstale":"2024-04-17T10:43:35.514002+0000","last_undegraded":"2024-04-17T10:43:35.514002+0000","last_fullsized":"2024-04-17T10:43:35.514002+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:18:50.159338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.511189+0000","last_change":"2024-04-17T10:43:34.500040+0000","last_active":"2024-04-17T10:43:35.511189+0000","last_peered":"2024-04-17T10:43:35.511189+0000","last_clean":"2024-04-17T10:43:35.511189+0000","last_became_active":"2024-04-17T10:43:34.499300+0000","last_became_peered":"2024-04-17T10:43:34.499300+0000","last_unstale":"2024-04-17T10:43:35.511189+0000","last_undegraded":"2024-04-17T10:43:35.511189+0000","last_fullsized":"2024-04-17T10:43:35.511189+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:29:50.384267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.670789+0000","last_change":"2024-04-17T10:43:33.491909+0000","last_active":"2024-04-17T10:43:35.670789+0000","last_peered":"2024-04-17T10:43:35.670789+0000","last_clean":"2024-04-17T10:43:35.670789+0000","last_became_active":"2024-04-17T10:43:33.490806+0000","last_became_peered":"2024-04-17T10:43:33.490806+0000","last_unstale":"2024-04-17T10:43:35.670789+0000","last_undegraded":"2024-04-17T10:43:35.670789+0000","last_fullsized":"2024-04-17T10:43:35.670789+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:41:03.800140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.515426+0000","last_change":"2024-04-17T10:43:34.512331+0000","last_active":"2024-04-17T10:43:35.515426+0000","last_peered":"2024-04-17T10:43:35.515426+0000","last_clean":"2024-04-17T10:43:35.515426+0000","last_became_active":"2024-04-17T10:43:34.511952+0000","last_became_peered":"2024-04-17T10:43:34.511952+0000","last_unstale":"2024-04-17T10:43:35.515426+0000","last_undegraded":"2024-04-17T10:43:35.515426+0000","last_fullsized":"2024-04-17T10:43:35.515426+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:00:18.356933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.926074+0000","last_change":"2024-04-17T10:43:33.504035+0000","last_active":"2024-04-17T10:43:35.926074+0000","last_peered":"2024-04-17T10:43:35.926074+0000","last_clean":"2024-04-17T10:43:35.926074+0000","last_became_active":"2024-04-17T10:43:33.503454+0000","last_became_peered":"2024-04-17T10:43:33.503454+0000","last_unstale":"2024-04-17T10:43:35.926074+0000","last_undegraded":"2024-04-17T10:43:35.926074+0000","last_fullsized":"2024-04-17T10:43:35.926074+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:05:45.467752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.513369+0000","last_change":"2024-04-17T10:43:34.524580+0000","last_active":"2024-04-17T10:43:35.513369+0000","last_peered":"2024-04-17T10:43:35.513369+0000","last_clean":"2024-04-17T10:43:35.513369+0000","last_became_active":"2024-04-17T10:43:34.524259+0000","last_became_peered":"2024-04-17T10:43:34.524259+0000","last_unstale":"2024-04-17T10:43:35.513369+0000","last_undegraded":"2024-04-17T10:43:35.513369+0000","last_fullsized":"2024-04-17T10:43:35.513369+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:12:08.381484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.669849+0000","last_change":"2024-04-17T10:43:33.502776+0000","last_active":"2024-04-17T10:43:35.669849+0000","last_peered":"2024-04-17T10:43:35.669849+0000","last_clean":"2024-04-17T10:43:35.669849+0000","last_became_active":"2024-04-17T10:43:33.498687+0000","last_became_peered":"2024-04-17T10:43:33.498687+0000","last_unstale":"2024-04-17T10:43:35.669849+0000","last_undegraded":"2024-04-17T10:43:35.669849+0000","last_fullsized":"2024-04-17T10:43:35.669849+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:38:33.824469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.922301+0000","last_change":"2024-04-17T10:43:34.509989+0000","last_active":"2024-04-17T10:43:35.922301+0000","last_peered":"2024-04-17T10:43:35.922301+0000","last_clean":"2024-04-17T10:43:35.922301+0000","last_became_active":"2024-04-17T10:43:34.509472+0000","last_became_peered":"2024-04-17T10:43:34.509472+0000","last_unstale":"2024-04-17T10:43:35.922301+0000","last_undegraded":"2024-04-17T10:43:35.922301+0000","last_fullsized":"2024-04-17T10:43:35.922301+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:26:59.869704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.923662+0000","last_change":"2024-04-17T10:43:33.506146+0000","last_active":"2024-04-17T10:43:35.923662+0000","last_peered":"2024-04-17T10:43:35.923662+0000","last_clean":"2024-04-17T10:43:35.923662+0000","last_became_active":"2024-04-17T10:43:33.505651+0000","last_became_peered":"2024-04-17T10:43:33.505651+0000","last_unstale":"2024-04-17T10:43:35.923662+0000","last_undegraded":"2024-04-17T10:43:35.923662+0000","last_fullsized":"2024-04-17T10:43:35.923662+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:31.070611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.508957+0000","last_change":"2024-04-17T10:43:34.515010+0000","last_active":"2024-04-17T10:43:35.508957+0000","last_peered":"2024-04-17T10:43:35.508957+0000","last_clean":"2024-04-17T10:43:35.508957+0000","last_became_active":"2024-04-17T10:43:34.514506+0000","last_became_peered":"2024-04-17T10:43:34.514506+0000","last_unstale":"2024-04-17T10:43:35.508957+0000","last_undegraded":"2024-04-17T10:43:35.508957+0000","last_fullsized":"2024-04-17T10:43:35.508957+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:17:56.396146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.922582+0000","last_change":"2024-04-17T10:43:33.478569+0000","last_active":"2024-04-17T10:43:35.922582+0000","last_peered":"2024-04-17T10:43:35.922582+0000","last_clean":"2024-04-17T10:43:35.922582+0000","last_became_active":"2024-04-17T10:43:33.477453+0000","last_became_peered":"2024-04-17T10:43:33.477453+0000","last_unstale":"2024-04-17T10:43:35.922582+0000","last_undegraded":"2024-04-17T10:43:35.922582+0000","last_fullsized":"2024-04-17T10:43:35.922582+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:44:53.437163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.515632+0000","last_change":"2024-04-17T10:43:34.518775+0000","last_active":"2024-04-17T10:43:35.515632+0000","last_peered":"2024-04-17T10:43:35.515632+0000","last_clean":"2024-04-17T10:43:35.515632+0000","last_became_active":"2024-04-17T10:43:34.517570+0000","last_became_peered":"2024-04-17T10:43:34.517570+0000","last_unstale":"2024-04-17T10:43:35.515632+0000","last_undegraded":"2024-04-17T10:43:35.515632+0000","last_fullsized":"2024-04-17T10:43:35.515632+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:09:22.706967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.514013+0000","last_change":"2024-04-17T10:43:33.499957+0000","last_active":"2024-04-17T10:43:35.514013+0000","last_peered":"2024-04-17T10:43:35.514013+0000","last_clean":"2024-04-17T10:43:35.514013+0000","last_became_active":"2024-04-17T10:43:33.481936+0000","last_became_peered":"2024-04-17T10:43:33.481936+0000","last_unstale":"2024-04-17T10:43:35.514013+0000","last_undegraded":"2024-04-17T10:43:35.514013+0000","last_fullsized":"2024-04-17T10:43:35.514013+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:12:08.202271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.511911+0000","last_change":"2024-04-17T10:43:34.501278+0000","last_active":"2024-04-17T10:43:35.511911+0000","last_peered":"2024-04-17T10:43:35.511911+0000","last_clean":"2024-04-17T10:43:35.511911+0000","last_became_active":"2024-04-17T10:43:34.500576+0000","last_became_peered":"2024-04-17T10:43:34.500576+0000","last_unstale":"2024-04-17T10:43:35.511911+0000","last_undegraded":"2024-04-17T10:43:35.511911+0000","last_fullsized":"2024-04-17T10:43:35.511911+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:25:31.835708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.670901+0000","last_change":"2024-04-17T10:43:33.494714+0000","last_active":"2024-04-17T10:43:35.670901+0000","last_peered":"2024-04-17T10:43:35.670901+0000","last_clean":"2024-04-17T10:43:35.670901+0000","last_became_active":"2024-04-17T10:43:33.493921+0000","last_became_peered":"2024-04-17T10:43:33.493921+0000","last_unstale":"2024-04-17T10:43:35.670901+0000","last_undegraded":"2024-04-17T10:43:35.670901+0000","last_fullsized":"2024-04-17T10:43:35.670901+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:33:22.245616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.670683+0000","last_change":"2024-04-17T10:43:34.519706+0000","last_active":"2024-04-17T10:43:35.670683+0000","last_peered":"2024-04-17T10:43:35.670683+0000","last_clean":"2024-04-17T10:43:35.670683+0000","last_became_active":"2024-04-17T10:43:34.519385+0000","last_became_peered":"2024-04-17T10:43:34.519385+0000","last_unstale":"2024-04-17T10:43:35.670683+0000","last_undegraded":"2024-04-17T10:43:35.670683+0000","last_fullsized":"2024-04-17T10:43:35.670683+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:40:09.920144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"73'3","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.926171+0000","last_change":"2024-04-17T10:43:33.492746+0000","last_active":"2024-04-17T10:43:35.926171+0000","last_peered":"2024-04-17T10:43:35.926171+0000","last_clean":"2024-04-17T10:43:35.926171+0000","last_became_active":"2024-04-17T10:43:33.492349+0000","last_became_peered":"2024-04-17T10:43:33.492349+0000","last_unstale":"2024-04-17T10:43:35.926171+0000","last_undegraded":"2024-04-17T10:43:35.926171+0000","last_fullsized":"2024-04-17T10:43:35.926171+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:28:54.360193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.923350+0000","last_change":"2024-04-17T10:43:34.509531+0000","last_active":"2024-04-17T10:43:35.923350+0000","last_peered":"2024-04-17T10:43:35.923350+0000","last_clean":"2024-04-17T10:43:35.923350+0000","last_became_active":"2024-04-17T10:43:34.509136+0000","last_became_peered":"2024-04-17T10:43:34.509136+0000","last_unstale":"2024-04-17T10:43:35.923350+0000","last_undegraded":"2024-04-17T10:43:35.923350+0000","last_fullsized":"2024-04-17T10:43:35.923350+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:44:27.975668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"73'6","reported_seq":20,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.924809+0000","last_change":"2024-04-17T10:43:33.476458+0000","last_active":"2024-04-17T10:43:35.924809+0000","last_peered":"2024-04-17T10:43:35.924809+0000","last_clean":"2024-04-17T10:43:35.924809+0000","last_became_active":"2024-04-17T10:43:33.475608+0000","last_became_peered":"2024-04-17T10:43:33.475608+0000","last_unstale":"2024-04-17T10:43:35.924809+0000","last_undegraded":"2024-04-17T10:43:35.924809+0000","last_fullsized":"2024-04-17T10:43:35.924809+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:33:22.245616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.510611+0000","last_change":"2024-04-17T10:43:34.507122+0000","last_active":"2024-04-17T10:43:35.510611+0000","last_peered":"2024-04-17T10:43:35.510611+0000","last_clean":"2024-04-17T10:43:35.510611+0000","last_became_active":"2024-04-17T10:43:34.506660+0000","last_became_peered":"2024-04-17T10:43:34.506660+0000","last_unstale":"2024-04-17T10:43:35.510611+0000","last_undegraded":"2024-04-17T10:43:35.510611+0000","last_fullsized":"2024-04-17T10:43:35.510611+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:17:46.702817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.923403+0000","last_change":"2024-04-17T10:43:33.479258+0000","last_active":"2024-04-17T10:43:35.923403+0000","last_peered":"2024-04-17T10:43:35.923403+0000","last_clean":"2024-04-17T10:43:35.923403+0000","last_became_active":"2024-04-17T10:43:33.478798+0000","last_became_peered":"2024-04-17T10:43:33.478798+0000","last_unstale":"2024-04-17T10:43:35.923403+0000","last_undegraded":"2024-04-17T10:43:35.923403+0000","last_fullsized":"2024-04-17T10:43:35.923403+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:31:43.766303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.515603+0000","last_change":"2024-04-17T10:43:34.519858+0000","last_active":"2024-04-17T10:43:35.515603+0000","last_peered":"2024-04-17T10:43:35.515603+0000","last_clean":"2024-04-17T10:43:35.515603+0000","last_became_active":"2024-04-17T10:43:34.519572+0000","last_became_peered":"2024-04-17T10:43:34.519572+0000","last_unstale":"2024-04-17T10:43:35.515603+0000","last_undegraded":"2024-04-17T10:43:35.515603+0000","last_fullsized":"2024-04-17T10:43:35.515603+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:08:25.575135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.670205+0000","last_change":"2024-04-17T10:43:33.499759+0000","last_active":"2024-04-17T10:43:35.670205+0000","last_peered":"2024-04-17T10:43:35.670205+0000","last_clean":"2024-04-17T10:43:35.670205+0000","last_became_active":"2024-04-17T10:43:33.498138+0000","last_became_peered":"2024-04-17T10:43:33.498138+0000","last_unstale":"2024-04-17T10:43:35.670205+0000","last_undegraded":"2024-04-17T10:43:35.670205+0000","last_fullsized":"2024-04-17T10:43:35.670205+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:30:42.196624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.508652+0000","last_change":"2024-04-17T10:43:34.507753+0000","last_active":"2024-04-17T10:43:35.508652+0000","last_peered":"2024-04-17T10:43:35.508652+0000","last_clean":"2024-04-17T10:43:35.508652+0000","last_became_active":"2024-04-17T10:43:34.507058+0000","last_became_peered":"2024-04-17T10:43:34.507058+0000","last_unstale":"2024-04-17T10:43:35.508652+0000","last_undegraded":"2024-04-17T10:43:35.508652+0000","last_fullsized":"2024-04-17T10:43:35.508652+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:29:04.316734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.515050+0000","last_change":"2024-04-17T10:43:33.488497+0000","last_active":"2024-04-17T10:43:35.515050+0000","last_peered":"2024-04-17T10:43:35.515050+0000","last_clean":"2024-04-17T10:43:35.515050+0000","last_became_active":"2024-04-17T10:43:33.487777+0000","last_became_peered":"2024-04-17T10:43:33.487777+0000","last_unstale":"2024-04-17T10:43:35.515050+0000","last_undegraded":"2024-04-17T10:43:35.515050+0000","last_fullsized":"2024-04-17T10:43:35.515050+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:17.570356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.923167+0000","last_change":"2024-04-17T10:43:34.504596+0000","last_active":"2024-04-17T10:43:35.923167+0000","last_peered":"2024-04-17T10:43:35.923167+0000","last_clean":"2024-04-17T10:43:35.923167+0000","last_became_active":"2024-04-17T10:43:34.504219+0000","last_became_peered":"2024-04-17T10:43:34.504219+0000","last_unstale":"2024-04-17T10:43:35.923167+0000","last_undegraded":"2024-04-17T10:43:35.923167+0000","last_fullsized":"2024-04-17T10:43:35.923167+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:45:06.770915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"73'8","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.512886+0000","last_change":"2024-04-17T10:43:33.505100+0000","last_active":"2024-04-17T10:43:35.512886+0000","last_peered":"2024-04-17T10:43:35.512886+0000","last_clean":"2024-04-17T10:43:35.512886+0000","last_became_active":"2024-04-17T10:43:33.504443+0000","last_became_peered":"2024-04-17T10:43:33.504443+0000","last_unstale":"2024-04-17T10:43:35.512886+0000","last_undegraded":"2024-04-17T10:43:35.512886+0000","last_fullsized":"2024-04-17T10:43:35.512886+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:28:44.525141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.514205+0000","last_change":"2024-04-17T10:43:34.519154+0000","last_active":"2024-04-17T10:43:35.514205+0000","last_peered":"2024-04-17T10:43:35.514205+0000","last_clean":"2024-04-17T10:43:35.514205+0000","last_became_active":"2024-04-17T10:43:34.518916+0000","last_became_peered":"2024-04-17T10:43:34.518916+0000","last_unstale":"2024-04-17T10:43:35.514205+0000","last_undegraded":"2024-04-17T10:43:35.514205+0000","last_fullsized":"2024-04-17T10:43:35.514205+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:07:09.761324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.670161+0000","last_change":"2024-04-17T10:43:33.497414+0000","last_active":"2024-04-17T10:43:35.670161+0000","last_peered":"2024-04-17T10:43:35.670161+0000","last_clean":"2024-04-17T10:43:35.670161+0000","last_became_active":"2024-04-17T10:43:33.495764+0000","last_became_peered":"2024-04-17T10:43:33.495764+0000","last_unstale":"2024-04-17T10:43:35.670161+0000","last_undegraded":"2024-04-17T10:43:35.670161+0000","last_fullsized":"2024-04-17T10:43:35.670161+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:44:26.965507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.512161+0000","last_change":"2024-04-17T10:43:34.504035+0000","last_active":"2024-04-17T10:43:35.512161+0000","last_peered":"2024-04-17T10:43:35.512161+0000","last_clean":"2024-04-17T10:43:35.512161+0000","last_became_active":"2024-04-17T10:43:34.503679+0000","last_became_peered":"2024-04-17T10:43:34.503679+0000","last_unstale":"2024-04-17T10:43:35.512161+0000","last_undegraded":"2024-04-17T10:43:35.512161+0000","last_fullsized":"2024-04-17T10:43:35.512161+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:55:15.790585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.514338+0000","last_change":"2024-04-17T10:43:33.509688+0000","last_active":"2024-04-17T10:43:35.514338+0000","last_peered":"2024-04-17T10:43:35.514338+0000","last_clean":"2024-04-17T10:43:35.514338+0000","last_became_active":"2024-04-17T10:43:33.508865+0000","last_became_peered":"2024-04-17T10:43:33.508865+0000","last_unstale":"2024-04-17T10:43:35.514338+0000","last_undegraded":"2024-04-17T10:43:35.514338+0000","last_fullsized":"2024-04-17T10:43:35.514338+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:46:44.667367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.924569+0000","last_change":"2024-04-17T10:43:34.503841+0000","last_active":"2024-04-17T10:43:35.924569+0000","last_peered":"2024-04-17T10:43:35.924569+0000","last_clean":"2024-04-17T10:43:35.924569+0000","last_became_active":"2024-04-17T10:43:34.503325+0000","last_became_peered":"2024-04-17T10:43:34.503325+0000","last_unstale":"2024-04-17T10:43:35.924569+0000","last_undegraded":"2024-04-17T10:43:35.924569+0000","last_fullsized":"2024-04-17T10:43:35.924569+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:38:45.127325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.670573+0000","last_change":"2024-04-17T10:43:33.507008+0000","last_active":"2024-04-17T10:43:35.670573+0000","last_peered":"2024-04-17T10:43:35.670573+0000","last_clean":"2024-04-17T10:43:35.670573+0000","last_became_active":"2024-04-17T10:43:33.506297+0000","last_became_peered":"2024-04-17T10:43:33.506297+0000","last_unstale":"2024-04-17T10:43:35.670573+0000","last_undegraded":"2024-04-17T10:43:35.670573+0000","last_fullsized":"2024-04-17T10:43:35.670573+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:41:27.100206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.923466+0000","last_change":"2024-04-17T10:43:34.518602+0000","last_active":"2024-04-17T10:43:35.923466+0000","last_peered":"2024-04-17T10:43:35.923466+0000","last_clean":"2024-04-17T10:43:35.923466+0000","last_became_active":"2024-04-17T10:43:34.518300+0000","last_became_peered":"2024-04-17T10:43:34.518300+0000","last_unstale":"2024-04-17T10:43:35.923466+0000","last_undegraded":"2024-04-17T10:43:35.923466+0000","last_fullsized":"2024-04-17T10:43:35.923466+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:32.080773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.511831+0000","last_change":"2024-04-17T10:43:33.508340+0000","last_active":"2024-04-17T10:43:35.511831+0000","last_peered":"2024-04-17T10:43:35.511831+0000","last_clean":"2024-04-17T10:43:35.511831+0000","last_became_active":"2024-04-17T10:43:33.507561+0000","last_became_peered":"2024-04-17T10:43:33.507561+0000","last_unstale":"2024-04-17T10:43:35.511831+0000","last_undegraded":"2024-04-17T10:43:35.511831+0000","last_fullsized":"2024-04-17T10:43:35.511831+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:46:01.786871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.925830+0000","last_change":"2024-04-17T10:43:34.517177+0000","last_active":"2024-04-17T10:43:35.925830+0000","last_peered":"2024-04-17T10:43:35.925830+0000","last_clean":"2024-04-17T10:43:35.925830+0000","last_became_active":"2024-04-17T10:43:34.514409+0000","last_became_peered":"2024-04-17T10:43:34.514409+0000","last_unstale":"2024-04-17T10:43:35.925830+0000","last_undegraded":"2024-04-17T10:43:35.925830+0000","last_fullsized":"2024-04-17T10:43:35.925830+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:12:13.190861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.670479+0000","last_change":"2024-04-17T10:43:33.500518+0000","last_active":"2024-04-17T10:43:35.670479+0000","last_peered":"2024-04-17T10:43:35.670479+0000","last_clean":"2024-04-17T10:43:35.670479+0000","last_became_active":"2024-04-17T10:43:33.499044+0000","last_became_peered":"2024-04-17T10:43:33.499044+0000","last_unstale":"2024-04-17T10:43:35.670479+0000","last_undegraded":"2024-04-17T10:43:35.670479+0000","last_fullsized":"2024-04-17T10:43:35.670479+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:12:45.140818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.922691+0000","last_change":"2024-04-17T10:43:34.516761+0000","last_active":"2024-04-17T10:43:35.922691+0000","last_peered":"2024-04-17T10:43:35.922691+0000","last_clean":"2024-04-17T10:43:35.922691+0000","last_became_active":"2024-04-17T10:43:34.515020+0000","last_became_peered":"2024-04-17T10:43:34.515020+0000","last_unstale":"2024-04-17T10:43:35.922691+0000","last_undegraded":"2024-04-17T10:43:35.922691+0000","last_fullsized":"2024-04-17T10:43:35.922691+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:49:38.687857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.515542+0000","last_change":"2024-04-17T10:43:33.492968+0000","last_active":"2024-04-17T10:43:35.515542+0000","last_peered":"2024-04-17T10:43:35.515542+0000","last_clean":"2024-04-17T10:43:35.515542+0000","last_became_active":"2024-04-17T10:43:33.492286+0000","last_became_peered":"2024-04-17T10:43:33.492286+0000","last_unstale":"2024-04-17T10:43:35.515542+0000","last_undegraded":"2024-04-17T10:43:35.515542+0000","last_fullsized":"2024-04-17T10:43:35.515542+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:46:01.786871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.924737+0000","last_change":"2024-04-17T10:43:34.518232+0000","last_active":"2024-04-17T10:43:35.924737+0000","last_peered":"2024-04-17T10:43:35.924737+0000","last_clean":"2024-04-17T10:43:35.924737+0000","last_became_active":"2024-04-17T10:43:34.517626+0000","last_became_peered":"2024-04-17T10:43:34.517626+0000","last_unstale":"2024-04-17T10:43:35.924737+0000","last_undegraded":"2024-04-17T10:43:35.924737+0000","last_fullsized":"2024-04-17T10:43:35.924737+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:24:13.684428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.508516+0000","last_change":"2024-04-17T10:43:33.482758+0000","last_active":"2024-04-17T10:43:35.508516+0000","last_peered":"2024-04-17T10:43:35.508516+0000","last_clean":"2024-04-17T10:43:35.508516+0000","last_became_active":"2024-04-17T10:43:33.482200+0000","last_became_peered":"2024-04-17T10:43:33.482200+0000","last_unstale":"2024-04-17T10:43:35.508516+0000","last_undegraded":"2024-04-17T10:43:35.508516+0000","last_fullsized":"2024-04-17T10:43:35.508516+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:42:26.108551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.509205+0000","last_change":"2024-04-17T10:43:34.502496+0000","last_active":"2024-04-17T10:43:35.509205+0000","last_peered":"2024-04-17T10:43:35.509205+0000","last_clean":"2024-04-17T10:43:35.509205+0000","last_became_active":"2024-04-17T10:43:34.501993+0000","last_became_peered":"2024-04-17T10:43:34.501993+0000","last_unstale":"2024-04-17T10:43:35.509205+0000","last_undegraded":"2024-04-17T10:43:35.509205+0000","last_fullsized":"2024-04-17T10:43:35.509205+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:09:50.041661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.670264+0000","last_change":"2024-04-17T10:43:33.496516+0000","last_active":"2024-04-17T10:43:35.670264+0000","last_peered":"2024-04-17T10:43:35.670264+0000","last_clean":"2024-04-17T10:43:35.670264+0000","last_became_active":"2024-04-17T10:43:33.494818+0000","last_became_peered":"2024-04-17T10:43:33.494818+0000","last_unstale":"2024-04-17T10:43:35.670264+0000","last_undegraded":"2024-04-17T10:43:35.670264+0000","last_fullsized":"2024-04-17T10:43:35.670264+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:31.070611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.515296+0000","last_change":"2024-04-17T10:43:34.517453+0000","last_active":"2024-04-17T10:43:35.515296+0000","last_peered":"2024-04-17T10:43:35.515296+0000","last_clean":"2024-04-17T10:43:35.515296+0000","last_became_active":"2024-04-17T10:43:34.517104+0000","last_became_peered":"2024-04-17T10:43:34.517104+0000","last_unstale":"2024-04-17T10:43:35.515296+0000","last_undegraded":"2024-04-17T10:43:35.515296+0000","last_fullsized":"2024-04-17T10:43:35.515296+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:17:56.396146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"74'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.515629+0000","last_change":"2024-04-17T10:43:33.488378+0000","last_active":"2024-04-17T10:43:35.515629+0000","last_peered":"2024-04-17T10:43:35.515629+0000","last_clean":"2024-04-17T10:43:35.515629+0000","last_became_active":"2024-04-17T10:43:33.487622+0000","last_became_peered":"2024-04-17T10:43:33.487622+0000","last_unstale":"2024-04-17T10:43:35.515629+0000","last_undegraded":"2024-04-17T10:43:35.515629+0000","last_fullsized":"2024-04-17T10:43:35.515629+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:48:07.131314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.511613+0000","last_change":"2024-04-17T10:43:34.501692+0000","last_active":"2024-04-17T10:43:35.511613+0000","last_peered":"2024-04-17T10:43:35.511613+0000","last_clean":"2024-04-17T10:43:35.511613+0000","last_became_active":"2024-04-17T10:43:34.498972+0000","last_became_peered":"2024-04-17T10:43:34.498972+0000","last_unstale":"2024-04-17T10:43:35.511613+0000","last_undegraded":"2024-04-17T10:43:35.511613+0000","last_fullsized":"2024-04-17T10:43:35.511613+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:33:42.032077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.923748+0000","last_change":"2024-04-17T10:43:33.480817+0000","last_active":"2024-04-17T10:43:35.923748+0000","last_peered":"2024-04-17T10:43:35.923748+0000","last_clean":"2024-04-17T10:43:35.923748+0000","last_became_active":"2024-04-17T10:43:33.480442+0000","last_became_peered":"2024-04-17T10:43:33.480442+0000","last_unstale":"2024-04-17T10:43:35.923748+0000","last_undegraded":"2024-04-17T10:43:35.923748+0000","last_fullsized":"2024-04-17T10:43:35.923748+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:43:38.278551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.511453+0000","last_change":"2024-04-17T10:43:34.517715+0000","last_active":"2024-04-17T10:43:35.511453+0000","last_peered":"2024-04-17T10:43:35.511453+0000","last_clean":"2024-04-17T10:43:35.511453+0000","last_became_active":"2024-04-17T10:43:34.516696+0000","last_became_peered":"2024-04-17T10:43:34.516696+0000","last_unstale":"2024-04-17T10:43:35.511453+0000","last_undegraded":"2024-04-17T10:43:35.511453+0000","last_fullsized":"2024-04-17T10:43:35.511453+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:33:02.606984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.509085+0000","last_change":"2024-04-17T10:43:33.510303+0000","last_active":"2024-04-17T10:43:35.509085+0000","last_peered":"2024-04-17T10:43:35.509085+0000","last_clean":"2024-04-17T10:43:35.509085+0000","last_became_active":"2024-04-17T10:43:33.509966+0000","last_became_peered":"2024-04-17T10:43:33.509966+0000","last_unstale":"2024-04-17T10:43:35.509085+0000","last_undegraded":"2024-04-17T10:43:35.509085+0000","last_fullsized":"2024-04-17T10:43:35.509085+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:07:18.944354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.925217+0000","last_change":"2024-04-17T10:43:34.506852+0000","last_active":"2024-04-17T10:43:35.925217+0000","last_peered":"2024-04-17T10:43:35.925217+0000","last_clean":"2024-04-17T10:43:35.925217+0000","last_became_active":"2024-04-17T10:43:34.506406+0000","last_became_peered":"2024-04-17T10:43:34.506406+0000","last_unstale":"2024-04-17T10:43:35.925217+0000","last_undegraded":"2024-04-17T10:43:35.925217+0000","last_fullsized":"2024-04-17T10:43:35.925217+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:31:03.624665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.923576+0000","last_change":"2024-04-17T10:43:33.489836+0000","last_active":"2024-04-17T10:43:35.923576+0000","last_peered":"2024-04-17T10:43:35.923576+0000","last_clean":"2024-04-17T10:43:35.923576+0000","last_became_active":"2024-04-17T10:43:33.489438+0000","last_became_peered":"2024-04-17T10:43:33.489438+0000","last_unstale":"2024-04-17T10:43:35.923576+0000","last_undegraded":"2024-04-17T10:43:35.923576+0000","last_fullsized":"2024-04-17T10:43:35.923576+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:11:35.719578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.671129+0000","last_change":"2024-04-17T10:43:34.502069+0000","last_active":"2024-04-17T10:43:35.671129+0000","last_peered":"2024-04-17T10:43:35.671129+0000","last_clean":"2024-04-17T10:43:35.671129+0000","last_became_active":"2024-04-17T10:43:34.501346+0000","last_became_peered":"2024-04-17T10:43:34.501346+0000","last_unstale":"2024-04-17T10:43:35.671129+0000","last_undegraded":"2024-04-17T10:43:35.671129+0000","last_fullsized":"2024-04-17T10:43:35.671129+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:42:27.118713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.922574+0000","last_change":"2024-04-17T10:43:33.503755+0000","last_active":"2024-04-17T10:43:35.922574+0000","last_peered":"2024-04-17T10:43:35.922574+0000","last_clean":"2024-04-17T10:43:35.922574+0000","last_became_active":"2024-04-17T10:43:33.503377+0000","last_became_peered":"2024-04-17T10:43:33.503377+0000","last_unstale":"2024-04-17T10:43:35.922574+0000","last_undegraded":"2024-04-17T10:43:35.922574+0000","last_fullsized":"2024-04-17T10:43:35.922574+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:41:27.100206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.924185+0000","last_change":"2024-04-17T10:43:34.505600+0000","last_active":"2024-04-17T10:43:35.924185+0000","last_peered":"2024-04-17T10:43:35.924185+0000","last_clean":"2024-04-17T10:43:35.924185+0000","last_became_active":"2024-04-17T10:43:34.505081+0000","last_became_peered":"2024-04-17T10:43:34.505081+0000","last_unstale":"2024-04-17T10:43:35.924185+0000","last_undegraded":"2024-04-17T10:43:35.924185+0000","last_fullsized":"2024-04-17T10:43:35.924185+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:32:40.777551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.513101+0000","last_change":"2024-04-17T10:43:33.506065+0000","last_active":"2024-04-17T10:43:35.513101+0000","last_peered":"2024-04-17T10:43:35.513101+0000","last_clean":"2024-04-17T10:43:35.513101+0000","last_became_active":"2024-04-17T10:43:33.504808+0000","last_became_peered":"2024-04-17T10:43:33.504808+0000","last_unstale":"2024-04-17T10:43:35.513101+0000","last_undegraded":"2024-04-17T10:43:35.513101+0000","last_fullsized":"2024-04-17T10:43:35.513101+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:55:36.126060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.669702+0000","last_change":"2024-04-17T10:43:34.508021+0000","last_active":"2024-04-17T10:43:35.669702+0000","last_peered":"2024-04-17T10:43:35.669702+0000","last_clean":"2024-04-17T10:43:35.669702+0000","last_became_active":"2024-04-17T10:43:34.507660+0000","last_became_peered":"2024-04-17T10:43:34.507660+0000","last_unstale":"2024-04-17T10:43:35.669702+0000","last_undegraded":"2024-04-17T10:43:35.669702+0000","last_fullsized":"2024-04-17T10:43:35.669702+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:55:37.136221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.670979+0000","last_change":"2024-04-17T10:43:33.487447+0000","last_active":"2024-04-17T10:43:35.670979+0000","last_peered":"2024-04-17T10:43:35.670979+0000","last_clean":"2024-04-17T10:43:35.670979+0000","last_became_active":"2024-04-17T10:43:33.486690+0000","last_became_peered":"2024-04-17T10:43:33.486690+0000","last_unstale":"2024-04-17T10:43:35.670979+0000","last_undegraded":"2024-04-17T10:43:35.670979+0000","last_fullsized":"2024-04-17T10:43:35.670979+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:31:02.614503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.925709+0000","last_change":"2024-04-17T10:43:34.520583+0000","last_active":"2024-04-17T10:43:35.925709+0000","last_peered":"2024-04-17T10:43:35.925709+0000","last_clean":"2024-04-17T10:43:35.925709+0000","last_became_active":"2024-04-17T10:43:34.520187+0000","last_became_peered":"2024-04-17T10:43:34.520187+0000","last_unstale":"2024-04-17T10:43:35.925709+0000","last_undegraded":"2024-04-17T10:43:35.925709+0000","last_fullsized":"2024-04-17T10:43:35.925709+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:33.454791+0000","last_clean_scrub_stamp":"2024-04-17T10:43:33.454791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:52:08.030078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.670897+0000","last_change":"2024-04-17T10:43:33.495411+0000","last_active":"2024-04-17T10:43:35.670897+0000","last_peered":"2024-04-17T10:43:35.670897+0000","last_clean":"2024-04-17T10:43:35.670897+0000","last_became_active":"2024-04-17T10:43:33.494671+0000","last_became_peered":"2024-04-17T10:43:33.494671+0000","last_unstale":"2024-04-17T10:43:35.670897+0000","last_undegraded":"2024-04-17T10:43:35.670897+0000","last_fullsized":"2024-04-17T10:43:35.670897+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:43:32.444630+0000","last_clean_scrub_stamp":"2024-04-17T10:43:32.444630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:31:43.766303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":280,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-17T10:43:35.510672+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:43:35.510672+0000","last_peered":"2024-04-17T10:43:35.510672+0000","last_clean":"2024-04-17T10:43:35.510672+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:43:35.510672+0000","last_undegraded":"2024-04-17T10:43:35.510672+0000","last_fullsized":"2024-04-17T10:43:35.510672+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574920,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46540,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325300,"statfs":{"total":96636764160,"available":96589107200,"internally_reserved":0,"allocated":1503232,"data_stored":975983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574919,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40392,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331448,"statfs":{"total":96636764160,"available":96595402752,"internally_reserved":0,"allocated":909312,"data_stored":385091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574920,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40388,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331452,"statfs":{"total":96636764160,"available":96595406848,"internally_reserved":0,"allocated":905216,"data_stored":385089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574919,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50120,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321720,"statfs":{"total":96636764160,"available":96585441280,"internally_reserved":0,"allocated":905216,"data_stored":385069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574921,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54276,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317564,"statfs":{"total":96636764160,"available":96581185536,"internally_reserved":0,"allocated":909312,"data_stored":385111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574919,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36232,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335608,"statfs":{"total":96636764160,"available":96599662592,"internally_reserved":0,"allocated":913408,"data_stored":386611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50128,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321712,"statfs":{"total":96636764160,"available":96585433088,"internally_reserved":0,"allocated":921600,"data_stored":387227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574919,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45128,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326712,"statfs":{"total":96636764160,"available":96590553088,"internally_reserved":0,"allocated":1495040,"data_stored":975367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:43:43.210 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-17T10:43:43.211 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-17T10:43:43.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.389+0000 7f38fcdb5640 1 Processor -- start 2024-04-17T10:43:43.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.389+0000 7f38fcdb5640 1 -- start start 2024-04-17T10:43:43.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.389+0000 7f38fcdb5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f38f80b5b60 0x7f38f80b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:43.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.389+0000 7f38fcdb5640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f815e4d0 con 0x7f38f815e0f0 2024-04-17T10:43:43.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.389+0000 7f38fcdb5640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f8162580 con 0x7f38f80ce400 2024-04-17T10:43:43.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.389+0000 7f38fcdb5640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38f8162b40 con 0x7f38f80b5b60 2024-04-17T10:43:43.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.389+0000 7f38f6d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f38f80b5b60 0x7f38f80b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:43.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.389+0000 7f38f6d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f38f80b5b60 0x7f38f80b6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52422/0 (socket says 172.21.15.83:52422) 2024-04-17T10:43:43.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.389+0000 7f38f6d9d640 1 -- 172.21.15.83:0/1666304768 learned_addr learned my addr 172.21.15.83:0/1666304768 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:43.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.389+0000 7f38f659c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f38f815e0f0 0x7f38f8161c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55100/0 (socket says 172.21.15.83:55100) 2024-04-17T10:43:43.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.389+0000 7f38f6d9d640 1 -- 172.21.15.83:0/1666304768 >> v1:172.21.15.167:6789/0 conn(0x7f38f80ce400 legacy=0x7f38f80b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:43.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.389+0000 7f38f6d9d640 1 -- 172.21.15.83:0/1666304768 >> v1:172.21.15.83:6789/0 conn(0x7f38f815e0f0 legacy=0x7f38f8161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:43.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.389+0000 7f38f6d9d640 1 -- 172.21.15.83:0/1666304768 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f8162860 con 0x7f38f80b5b60 2024-04-17T10:43:43.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.393+0000 7f38f6d9d640 1 --2- 172.21.15.83:0/1666304768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38f80b5b60 0x7f38f80b6d00 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f38f0009e50 tx=0x7f38f001c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24d5c8260b72f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:43.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.393+0000 7f38f559a640 1 -- 172.21.15.83:0/1666304768 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f00ac020 con 0x7f38f80b5b60 2024-04-17T10:43:43.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.393+0000 7f38f559a640 1 -- 172.21.15.83:0/1666304768 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f00a6480 con 0x7f38f80b5b60 2024-04-17T10:43:43.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.393+0000 7f38f559a640 1 -- 172.21.15.83:0/1666304768 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f00a6850 con 0x7f38f80b5b60 2024-04-17T10:43:43.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.393+0000 7f38fcdb5640 1 -- 172.21.15.83:0/1666304768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38f80b5b60 msgr2=0x7f38f80b6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:43.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.393+0000 7f38fcdb5640 1 --2- 172.21.15.83:0/1666304768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38f80b5b60 0x7f38f80b6d00 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f38f0009e50 tx=0x7f38f001c7f0 comp rx=0 tx=0).stop 2024-04-17T10:43:43.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.393+0000 7f38fcdb5640 1 -- 172.21.15.83:0/1666304768 shutdown_connections 2024-04-17T10:43:43.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.393+0000 7f38fcdb5640 1 --2- 172.21.15.83:0/1666304768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38f80b5b60 0x7f38f80b6d00 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:43.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.393+0000 7f38fcdb5640 1 -- 172.21.15.83:0/1666304768 >> 172.21.15.83:0/1666304768 conn(0x7f38f80d0450 msgr2=0x7f38f80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:43.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.393+0000 7f38fcdb5640 1 -- 172.21.15.83:0/1666304768 shutdown_connections 2024-04-17T10:43:43.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.393+0000 7f38fcdb5640 1 -- 172.21.15.83:0/1666304768 wait complete. 2024-04-17T10:43:43.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.393+0000 7f38fcdb5640 1 Processor -- start 2024-04-17T10:43:43.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.393+0000 7f38fcdb5640 1 -- start start 2024-04-17T10:43:43.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.393+0000 7f38fcdb5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38f80b5b60 0x7f38f81502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:43.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.393+0000 7f38fcdb5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38f80ce400 0x7f38f8158850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:43.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.393+0000 7f38fcdb5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38f815e0f0 0x7f38f8158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:43.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.393+0000 7f38fcdb5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f80d30e0 con 0x7f38f80ce400 2024-04-17T10:43:43.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.393+0000 7f38fcdb5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f80d2f60 con 0x7f38f80b5b60 2024-04-17T10:43:43.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.393+0000 7f38fcdb5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f80d3260 con 0x7f38f815e0f0 2024-04-17T10:43:43.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.397+0000 7f38f5d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38f80ce400 0x7f38f8158850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:43.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.397+0000 7f38f5d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38f80ce400 0x7f38f8158850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52432/0 (socket says 172.21.15.83:52432) 2024-04-17T10:43:43.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.397+0000 7f38f5d9b640 1 -- 172.21.15.83:0/690917393 learned_addr learned my addr 172.21.15.83:0/690917393 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:43.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.397+0000 7f38f659c640 1 --2- 172.21.15.83:0/690917393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38f80b5b60 0x7f38f81502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:43.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.397+0000 7f38f6d9d640 1 --2- 172.21.15.83:0/690917393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38f815e0f0 0x7f38f8158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:43.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.397+0000 7f38f5d9b640 1 -- 172.21.15.83:0/690917393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38f815e0f0 msgr2=0x7f38f8158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:43.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.397+0000 7f38f5d9b640 1 --2- 172.21.15.83:0/690917393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38f815e0f0 0x7f38f8158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:43.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.397+0000 7f38f5d9b640 1 -- 172.21.15.83:0/690917393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38f80b5b60 msgr2=0x7f38f81502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:43.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.397+0000 7f38f5d9b640 1 --2- 172.21.15.83:0/690917393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38f80b5b60 0x7f38f81502c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:43.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.397+0000 7f38f5d9b640 1 -- 172.21.15.83:0/690917393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f8155d20 con 0x7f38f80ce400 2024-04-17T10:43:43.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.397+0000 7f38f5d9b640 1 --2- 172.21.15.83:0/690917393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38f80ce400 0x7f38f8158850 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f38e800c410 tx=0x7f38e800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:43.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.397+0000 7f38e6ffd640 1 -- 172.21.15.83:0/690917393 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e8011790 con 0x7f38f80ce400 2024-04-17T10:43:43.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.397+0000 7f38e6ffd640 1 -- 172.21.15.83:0/690917393 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e8017070 con 0x7f38f80ce400 2024-04-17T10:43:43.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.397+0000 7f38fcdb5640 1 -- 172.21.15.83:0/690917393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f8156000 con 0x7f38f80ce400 2024-04-17T10:43:43.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.397+0000 7f38e6ffd640 1 -- 172.21.15.83:0/690917393 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e8011b70 con 0x7f38f80ce400 2024-04-17T10:43:43.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.397+0000 7f38fcdb5640 1 -- 172.21.15.83:0/690917393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f8150f60 con 0x7f38f80ce400 2024-04-17T10:43:43.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.397+0000 7f38fcdb5640 1 -- 172.21.15.83:0/690917393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f80681a0 con 0x7f38f80ce400 2024-04-17T10:43:43.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.397+0000 7f38e6ffd640 1 -- 172.21.15.83:0/690917393 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f38e8011d10 con 0x7f38f80ce400 2024-04-17T10:43:43.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.397+0000 7f38e6ffd640 1 --2- 172.21.15.83:0/690917393 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f38d8066f60 0x7f38d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:43.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.397+0000 7f38e6ffd640 1 -- 172.21.15.83:0/690917393 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38e8084e80 con 0x7f38f80ce400 2024-04-17T10:43:43.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.397+0000 7f38f659c640 1 --2- 172.21.15.83:0/690917393 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f38d8066f60 0x7f38d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:43.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.397+0000 7f38f659c640 1 --2- 172.21.15.83:0/690917393 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f38d8066f60 0x7f38d8069420 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f38ec000f70 tx=0x7f38ec007c70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:43.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.401+0000 7f38e6ffd640 1 -- 172.21.15.83:0/690917393 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f38e8055e90 con 0x7f38f80ce400 2024-04-17T10:43:43.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.613+0000 7f38fcdb5640 1 -- 172.21.15.83:0/690917393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f38f8074f90 con 0x7f38f80ce400 2024-04-17T10:43:43.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.617+0000 7f38e6ffd640 1 -- 172.21.15.83:0/690917393 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7f38e8055790 con 0x7f38f80ce400 2024-04-17T10:43:43.612 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-04-17T10:43:43.612 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBUph9mHGmXGBAAZF+JiAG9YrFtbW+dCy2IEQ== 2024-04-17T10:43:43.612 INFO:teuthology.orchestra.run.smithi083.stdout: caps mds = "allow" 2024-04-17T10:43:43.612 INFO:teuthology.orchestra.run.smithi083.stdout: caps mon = "allow r" 2024-04-17T10:43:43.613 INFO:teuthology.orchestra.run.smithi083.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-17T10:43:43.613 INFO:teuthology.orchestra.run.smithi083.stderr:updated caps for client.0 2024-04-17T10:43:43.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.621+0000 7f38fcdb5640 1 -- 172.21.15.83:0/690917393 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f38d8066f60 msgr2=0x7f38d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:43.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.621+0000 7f38fcdb5640 1 --2- 172.21.15.83:0/690917393 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f38d8066f60 0x7f38d8069420 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f38ec000f70 tx=0x7f38ec007c70 comp rx=0 tx=0).stop 2024-04-17T10:43:43.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.621+0000 7f38fcdb5640 1 -- 172.21.15.83:0/690917393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38f80ce400 msgr2=0x7f38f8158850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:43.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.621+0000 7f38fcdb5640 1 --2- 172.21.15.83:0/690917393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38f80ce400 0x7f38f8158850 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f38e800c410 tx=0x7f38e800c8d0 comp rx=0 tx=0).stop 2024-04-17T10:43:43.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.621+0000 7f38fcdb5640 1 -- 172.21.15.83:0/690917393 shutdown_connections 2024-04-17T10:43:43.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.621+0000 7f38fcdb5640 1 --2- 172.21.15.83:0/690917393 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f38d8066f60 0x7f38d8069420 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:43.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.621+0000 7f38fcdb5640 1 --2- 172.21.15.83:0/690917393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38f815e0f0 0x7f38f8158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:43.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.621+0000 7f38fcdb5640 1 --2- 172.21.15.83:0/690917393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38f80ce400 0x7f38f8158850 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:43.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.621+0000 7f38fcdb5640 1 --2- 172.21.15.83:0/690917393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38f80b5b60 0x7f38f81502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:43.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.621+0000 7f38fcdb5640 1 -- 172.21.15.83:0/690917393 >> 172.21.15.83:0/690917393 conn(0x7f38f80d0450 msgr2=0x7f38f80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:43.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.621+0000 7f38fcdb5640 1 -- 172.21.15.83:0/690917393 shutdown_connections 2024-04-17T10:43:43.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.621+0000 7f38fcdb5640 1 -- 172.21.15.83:0/690917393 wait complete. 2024-04-17T10:43:43.634 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:43:43.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.745+0000 7fe1b4323640 1 Processor -- start 2024-04-17T10:43:43.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.745+0000 7fe1b4323640 1 -- start start 2024-04-17T10:43:43.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.749+0000 7fe1b4323640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1ac0cf060 0x7fe1ac0b4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:43.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.749+0000 7fe1b4323640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe1ac070b80 0x7fe1ac06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:43.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.749+0000 7fe1b4323640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1ac0b2bd0 con 0x7fe1ac0b3990 2024-04-17T10:43:43.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.749+0000 7fe1b4323640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1ac0b2980 con 0x7fe1ac070b80 2024-04-17T10:43:43.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.749+0000 7fe1b4323640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ac0b2eb0 con 0x7fe1ac0cf060 2024-04-17T10:43:43.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.749+0000 7fe1b28c0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe1ac070b80 0x7fe1ac06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:43.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.749+0000 7fe1b18be640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe1ac0b3990 0x7fe1ac070470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55112/0 (socket says 172.21.15.83:55112) 2024-04-17T10:43:43.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.749+0000 7fe1b28c0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe1ac070b80 0x7fe1ac06c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52444/0 (socket says 172.21.15.83:52444) 2024-04-17T10:43:43.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.749+0000 7fe1b18be640 1 -- 172.21.15.83:0/3662776441 learned_addr learned my addr 172.21.15.83:0/3662776441 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:43:43.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.749+0000 7fe1b20bf640 1 --2- 172.21.15.83:0/3662776441 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1ac0cf060 0x7fe1ac0b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:43.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.749+0000 7fe1b28c0640 1 -- 172.21.15.83:0/3662776441 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1ac0cf060 msgr2=0x7fe1ac0b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:43.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.749+0000 7fe1b28c0640 1 --2- 172.21.15.83:0/3662776441 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1ac0cf060 0x7fe1ac0b4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:43.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.749+0000 7fe1b28c0640 1 -- 172.21.15.83:0/3662776441 >> v1:172.21.15.83:6789/0 conn(0x7fe1ac0b3990 legacy=0x7fe1ac070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:43.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.749+0000 7fe1b28c0640 1 -- 172.21.15.83:0/3662776441 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1ac0b3190 con 0x7fe1ac070b80 2024-04-17T10:43:43.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.749+0000 7fe1b28c0640 1 --2- 172.21.15.83:0/3662776441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1ac070b80 0x7fe1ac06c490 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4009d90 tx=0x7fe1a401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf64267192a24477 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:43.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.749+0000 7fe1b10bd640 1 -- 172.21.15.83:0/3662776441 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a40b2070 con 0x7fe1ac070b80 2024-04-17T10:43:43.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.749+0000 7fe1b10bd640 1 -- 172.21.15.83:0/3662776441 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a40a58b0 con 0x7fe1ac070b80 2024-04-17T10:43:43.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.749+0000 7fe1b10bd640 1 -- 172.21.15.83:0/3662776441 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a40a5cc0 con 0x7fe1ac070b80 2024-04-17T10:43:43.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1b4323640 1 -- 172.21.15.83:0/3662776441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1ac070b80 msgr2=0x7fe1ac06c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:43.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1b4323640 1 --2- 172.21.15.83:0/3662776441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1ac070b80 0x7fe1ac06c490 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4009d90 tx=0x7fe1a401c960 comp rx=0 tx=0).stop 2024-04-17T10:43:43.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1b4323640 1 -- 172.21.15.83:0/3662776441 shutdown_connections 2024-04-17T10:43:43.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1b4323640 1 --2- 172.21.15.83:0/3662776441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1ac070b80 0x7fe1ac06c490 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:43.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1b4323640 1 --2- 172.21.15.83:0/3662776441 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1ac0cf060 0x7fe1ac0b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:43.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1b4323640 1 -- 172.21.15.83:0/3662776441 >> 172.21.15.83:0/3662776441 conn(0x7fe1ac06abe0 msgr2=0x7fe1ac06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:43.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1b4323640 1 -- 172.21.15.83:0/3662776441 shutdown_connections 2024-04-17T10:43:43.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1b4323640 1 -- 172.21.15.83:0/3662776441 wait complete. 2024-04-17T10:43:43.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1b4323640 1 Processor -- start 2024-04-17T10:43:43.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1b4323640 1 -- start start 2024-04-17T10:43:43.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1b4323640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1ac070b80 0x7fe1ac152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:43.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1b4323640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1ac0b3990 0x7fe1ac06d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:43.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1b4323640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1ac0cf060 0x7fe1ac154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:43.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1b4323640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ac06cd60 con 0x7fe1ac0cf060 2024-04-17T10:43:43.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1b4323640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ac06cbe0 con 0x7fe1ac0b3990 2024-04-17T10:43:43.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1b4323640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ac06cee0 con 0x7fe1ac070b80 2024-04-17T10:43:43.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1b28c0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1ac0cf060 0x7fe1ac154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:43.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1b28c0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1ac0cf060 0x7fe1ac154ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52456/0 (socket says 172.21.15.83:52456) 2024-04-17T10:43:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1b28c0640 1 -- 172.21.15.83:0/3357303238 learned_addr learned my addr 172.21.15.83:0/3357303238 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1b20bf640 1 --2- 172.21.15.83:0/3357303238 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1ac070b80 0x7fe1ac152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1b28c0640 1 -- 172.21.15.83:0/3357303238 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1ac070b80 msgr2=0x7fe1ac152380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1b28c0640 1 --2- 172.21.15.83:0/3357303238 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1ac070b80 0x7fe1ac152380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1b18be640 1 --2- 172.21.15.83:0/3357303238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1ac0b3990 0x7fe1ac06d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1b28c0640 1 -- 172.21.15.83:0/3357303238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1ac0b3990 msgr2=0x7fe1ac06d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1b28c0640 1 --2- 172.21.15.83:0/3357303238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1ac0b3990 0x7fe1ac06d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1b28c0640 1 -- 172.21.15.83:0/3357303238 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1ac1529d0 con 0x7fe1ac0cf060 2024-04-17T10:43:43.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1b28c0640 1 --2- 172.21.15.83:0/3357303238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1ac0cf060 0x7fe1ac154ac0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fe1a40b0040 tx=0x7fe1a4019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:43.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1a2ffd640 1 -- 172.21.15.83:0/3357303238 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a40bd020 con 0x7fe1ac0cf060 2024-04-17T10:43:43.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1a2ffd640 1 -- 172.21.15.83:0/3357303238 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a40b8670 con 0x7fe1ac0cf060 2024-04-17T10:43:43.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1b4323640 1 -- 172.21.15.83:0/3357303238 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1ac14c410 con 0x7fe1ac0cf060 2024-04-17T10:43:43.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1a2ffd640 1 -- 172.21.15.83:0/3357303238 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a40b2040 con 0x7fe1ac0cf060 2024-04-17T10:43:43.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1b4323640 1 -- 172.21.15.83:0/3357303238 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1ac14c9d0 con 0x7fe1ac0cf060 2024-04-17T10:43:43.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.753+0000 7fe1b4323640 1 -- 172.21.15.83:0/3357303238 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1ac074f90 con 0x7fe1ac0cf060 2024-04-17T10:43:43.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.757+0000 7fe1a2ffd640 1 -- 172.21.15.83:0/3357303238 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe1a40c0020 con 0x7fe1ac0cf060 2024-04-17T10:43:43.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.757+0000 7fe1a2ffd640 1 --2- 172.21.15.83:0/3357303238 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe190066f60 0x7fe190069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:43.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.757+0000 7fe1a2ffd640 1 -- 172.21.15.83:0/3357303238 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1a4120250 con 0x7fe1ac0cf060 2024-04-17T10:43:43.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.757+0000 7fe1b20bf640 1 --2- 172.21.15.83:0/3357303238 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe190066f60 0x7fe190069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:43.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.757+0000 7fe1b20bf640 1 --2- 172.21.15.83:0/3357303238 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe190066f60 0x7fe190069420 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac158a00 tx=0x7fe19c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:43.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.761+0000 7fe1a2ffd640 1 -- 172.21.15.83:0/3357303238 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe1a40c7030 con 0x7fe1ac0cf060 2024-04-17T10:43:43.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.937+0000 7fe1b4323640 1 -- 172.21.15.83:0/3357303238 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1ac159fb0 con 0x7fe1ac0cf060 2024-04-17T10:43:43.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.937+0000 7fe1a2ffd640 1 -- 172.21.15.83:0/3357303238 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fe1a4016580 con 0x7fe1ac0cf060 2024-04-17T10:43:43.935 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:43:43.935 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":48,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5258,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2259892875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2259892875},{"type":"v1","addr":"172.21.15.167:6835","nonce":2259892875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":7095,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/3769102527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3769102527},{"type":"v1","addr":"172.21.15.83:6836","nonce":3769102527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":7098,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/395594830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":395594830},{"type":"v1","addr":"172.21.15.83:6837","nonce":395594830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:43:33.830720+0000","modified":"2024-04-17T10:43:36.867500+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7104},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7104":{"gid":7104,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6833/2796648400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2796648400},{"type":"v1","addr":"172.21.15.167:6833","nonce":2796648400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-17T10:43:43.935 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 48 2024-04-17T10:43:43.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.941+0000 7fe1b4323640 1 -- 172.21.15.83:0/3357303238 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe190066f60 msgr2=0x7fe190069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:43.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.941+0000 7fe1b4323640 1 --2- 172.21.15.83:0/3357303238 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe190066f60 0x7fe190069420 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac158a00 tx=0x7fe19c008040 comp rx=0 tx=0).stop 2024-04-17T10:43:43.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.941+0000 7fe1b4323640 1 -- 172.21.15.83:0/3357303238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1ac0cf060 msgr2=0x7fe1ac154ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:43.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.941+0000 7fe1b4323640 1 --2- 172.21.15.83:0/3357303238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1ac0cf060 0x7fe1ac154ac0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fe1a40b0040 tx=0x7fe1a4019700 comp rx=0 tx=0).stop 2024-04-17T10:43:43.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.941+0000 7fe1b4323640 1 -- 172.21.15.83:0/3357303238 shutdown_connections 2024-04-17T10:43:43.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.941+0000 7fe1b4323640 1 --2- 172.21.15.83:0/3357303238 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe190066f60 0x7fe190069420 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:43.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.941+0000 7fe1b4323640 1 --2- 172.21.15.83:0/3357303238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1ac0cf060 0x7fe1ac154ac0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:43.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.941+0000 7fe1b4323640 1 --2- 172.21.15.83:0/3357303238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1ac0b3990 0x7fe1ac06d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:43.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.941+0000 7fe1b4323640 1 --2- 172.21.15.83:0/3357303238 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1ac070b80 0x7fe1ac152380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:43.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.941+0000 7fe1b4323640 1 -- 172.21.15.83:0/3357303238 >> 172.21.15.83:0/3357303238 conn(0x7fe1ac06abe0 msgr2=0x7fe1ac1570c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:43.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.941+0000 7fe1b4323640 1 -- 172.21.15.83:0/3357303238 shutdown_connections 2024-04-17T10:43:43.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:43.941+0000 7fe1b4323640 1 -- 172.21.15.83:0/3357303238 wait complete. 2024-04-17T10:43:44.355 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 48, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T10:43:33.830720+0000', 'modified': '2024-04-17T10:43:36.867500+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7104}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7104': {'gid': 7104, 'name': 'd', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.167:6833/2796648400', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6832', 'nonce': 2796648400}, {'type': 'v1', 'addr': '172.21.15.167:6833', 'nonce': 2796648400}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:43:44.355 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T10:43:44.355 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-17T10:43:44.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.493+0000 7f01161a1640 1 Processor -- start 2024-04-17T10:43:44.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.493+0000 7f01161a1640 1 -- start start 2024-04-17T10:43:44.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.493+0000 7f01161a1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f011015bac0 0x7f0110071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:44.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.493+0000 7f01161a1640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f011006c1f0 con 0x7f011015b6e0 2024-04-17T10:43:44.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.493+0000 7f01161a1640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0110070a10 con 0x7f0110070470 2024-04-17T10:43:44.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.493+0000 7f01161a1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0110070cf0 con 0x7f011015bac0 2024-04-17T10:43:44.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.493+0000 7f010f7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f011015b6e0 0x7f01100cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55124/0 (socket says 172.21.15.83:55124) 2024-04-17T10:43:44.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.493+0000 7f010f7fe640 1 -- 172.21.15.83:0/861426140 learned_addr learned my addr 172.21.15.83:0/861426140 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:43:44.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.493+0000 7f010e7fc640 1 -- 172.21.15.83:0/861426140 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 792650713 0 0) 0x7f011006c1f0 con 0x7f011015b6e0 2024-04-17T10:43:44.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.493+0000 7f010e7fc640 1 -- 172.21.15.83:0/861426140 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00fc003530 con 0x7f011015b6e0 2024-04-17T10:43:44.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.493+0000 7f010effd640 1 --2- 172.21.15.83:0/861426140 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f011015bac0 0x7f0110071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:44.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.493+0000 7f010e7fc640 1 -- 172.21.15.83:0/861426140 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4122280538 0 0) 0x7f00fc003530 con 0x7f011015b6e0 2024-04-17T10:43:44.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.493+0000 7f010e7fc640 1 -- 172.21.15.83:0/861426140 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f011015bac0 msgr2=0x7f0110071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:44.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.493+0000 7f010e7fc640 1 --2- 172.21.15.83:0/861426140 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f011015bac0 0x7f0110071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.493+0000 7f010e7fc640 1 -- 172.21.15.83:0/861426140 >> v1:172.21.15.167:6789/0 conn(0x7f0110070470 legacy=0x7f011006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:44.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.493+0000 7f010e7fc640 1 -- 172.21.15.83:0/861426140 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0110070fd0 con 0x7f011015b6e0 2024-04-17T10:43:44.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.493+0000 7f010e7fc640 1 -- 172.21.15.83:0/861426140 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f0100001680 con 0x7f011015b6e0 2024-04-17T10:43:44.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.493+0000 7f010e7fc640 1 -- 172.21.15.83:0/861426140 >> v1:172.21.15.83:6789/0 conn(0x7f011015b6e0 legacy=0x7f01100cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:44.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.493+0000 7f010e7fc640 1 --2- 172.21.15.83:0/861426140 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f011015bac0 0x7f00fc003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:44.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.493+0000 7f010e7fc640 1 --2- 172.21.15.83:0/861426140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00fc004090 0x7f00fc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:44.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.493+0000 7f010e7fc640 1 --2- 172.21.15.83:0/861426140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00fc004da0 0x7f00fc005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:44.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.493+0000 7f010e7fc640 1 -- 172.21.15.83:0/861426140 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0110070cf0 con 0x7f00fc004da0 2024-04-17T10:43:44.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.493+0000 7f010e7fc640 1 -- 172.21.15.83:0/861426140 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc00bbe0 con 0x7f00fc004090 2024-04-17T10:43:44.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.493+0000 7f010e7fc640 1 -- 172.21.15.83:0/861426140 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc00bde0 con 0x7f011015bac0 2024-04-17T10:43:44.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.493+0000 7f010ffff640 1 --2- 172.21.15.83:0/861426140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00fc004da0 0x7f00fc005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:44.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.493+0000 7f010f7fe640 1 --2- 172.21.15.83:0/861426140 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f011015bac0 0x7f00fc003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:44.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.493+0000 7f010effd640 1 --2- 172.21.15.83:0/861426140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00fc004090 0x7f00fc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:44.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.493+0000 7f010ffff640 1 -- 172.21.15.83:0/861426140 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f011015bac0 msgr2=0x7f00fc003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:44.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.493+0000 7f010ffff640 1 --2- 172.21.15.83:0/861426140 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f011015bac0 0x7f00fc003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.493+0000 7f010ffff640 1 -- 172.21.15.83:0/861426140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00fc004090 msgr2=0x7f00fc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:44.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.493+0000 7f010ffff640 1 --2- 172.21.15.83:0/861426140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00fc004090 0x7f00fc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.493+0000 7f010ffff640 1 -- 172.21.15.83:0/861426140 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00fc003530 con 0x7f00fc004da0 2024-04-17T10:43:44.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.497+0000 7f010ffff640 1 --2- 172.21.15.83:0/861426140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00fc004da0 0x7f00fc005220 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f0108001e70 tx=0x7f010801c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=face4b7910448f60 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:44.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.497+0000 7f010e7fc640 1 -- 172.21.15.83:0/861426140 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0108001a90 con 0x7f00fc004da0 2024-04-17T10:43:44.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.497+0000 7f010e7fc640 1 -- 172.21.15.83:0/861426140 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0108003b80 con 0x7f00fc004da0 2024-04-17T10:43:44.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.497+0000 7f010e7fc640 1 -- 172.21.15.83:0/861426140 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0108005080 con 0x7f00fc004da0 2024-04-17T10:43:44.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.497+0000 7f01161a1640 1 -- 172.21.15.83:0/861426140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00fc004da0 msgr2=0x7f00fc005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:44.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.497+0000 7f01161a1640 1 --2- 172.21.15.83:0/861426140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00fc004da0 0x7f00fc005220 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f0108001e70 tx=0x7f010801c780 comp rx=0 tx=0).stop 2024-04-17T10:43:44.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.497+0000 7f010ffff640 1 -- 172.21.15.83:0/861426140 reap_dead start 2024-04-17T10:43:44.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.497+0000 7f01161a1640 1 -- 172.21.15.83:0/861426140 shutdown_connections 2024-04-17T10:43:44.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.497+0000 7f01161a1640 1 -- 172.21.15.83:0/861426140 >> 172.21.15.83:0/861426140 conn(0x7f01100b15b0 msgr2=0x7f01100b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:44.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.497+0000 7f01161a1640 1 -- 172.21.15.83:0/861426140 shutdown_connections 2024-04-17T10:43:44.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.497+0000 7f01161a1640 1 -- 172.21.15.83:0/861426140 wait complete. 2024-04-17T10:43:44.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.497+0000 7f01161a1640 1 Processor -- start 2024-04-17T10:43:44.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.497+0000 7f01161a1640 1 -- start start 2024-04-17T10:43:44.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.501+0000 7f01161a1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00fc004da0 0x7f0110146f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:44.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.501+0000 7f01161a1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f011015e320 0x7f011015e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:44.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.501+0000 7f01161a1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01100ad1c0 0x7f01100ad640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:44.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.501+0000 7f01161a1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011006b6a0 con 0x7f011015e320 2024-04-17T10:43:44.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.501+0000 7f01161a1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011006b520 con 0x7f00fc004da0 2024-04-17T10:43:44.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.501+0000 7f01161a1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0110071cc0 con 0x7f01100ad1c0 2024-04-17T10:43:44.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.501+0000 7f010effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f011015e320 0x7f011015e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:44.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.501+0000 7f010effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f011015e320 0x7f011015e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52472/0 (socket says 172.21.15.83:52472) 2024-04-17T10:43:44.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.501+0000 7f010effd640 1 -- 172.21.15.83:0/3252914069 learned_addr learned my addr 172.21.15.83:0/3252914069 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:44.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.501+0000 7f010f7fe640 1 --2- 172.21.15.83:0/3252914069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00fc004da0 0x7f0110146f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:44.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.501+0000 7f010ffff640 1 --2- 172.21.15.83:0/3252914069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01100ad1c0 0x7f01100ad640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:44.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.501+0000 7f010effd640 1 -- 172.21.15.83:0/3252914069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01100ad1c0 msgr2=0x7f01100ad640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:44.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.501+0000 7f010effd640 1 --2- 172.21.15.83:0/3252914069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01100ad1c0 0x7f01100ad640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.501+0000 7f010effd640 1 -- 172.21.15.83:0/3252914069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00fc004da0 msgr2=0x7f0110146f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:44.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.501+0000 7f010effd640 1 --2- 172.21.15.83:0/3252914069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00fc004da0 0x7f0110146f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.501+0000 7f010effd640 1 -- 172.21.15.83:0/3252914069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011006d010 con 0x7f011015e320 2024-04-17T10:43:44.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.501+0000 7f010ffff640 1 --2- 172.21.15.83:0/3252914069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01100ad1c0 0x7f01100ad640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:44.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.501+0000 7f010f7fe640 1 --2- 172.21.15.83:0/3252914069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00fc004da0 0x7f0110146f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:43:44.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.501+0000 7f010effd640 1 --2- 172.21.15.83:0/3252914069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f011015e320 0x7f011015e7a0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f00f800f580 tx=0x7f00f800fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:44.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.501+0000 7f0107fff640 1 -- 172.21.15.83:0/3252914069 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f8004230 con 0x7f011015e320 2024-04-17T10:43:44.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.501+0000 7f0107fff640 1 -- 172.21.15.83:0/3252914069 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f80043d0 con 0x7f011015e320 2024-04-17T10:43:44.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.501+0000 7f0107fff640 1 -- 172.21.15.83:0/3252914069 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f8014890 con 0x7f011015e320 2024-04-17T10:43:44.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.501+0000 7f01161a1640 1 -- 172.21.15.83:0/3252914069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01100adea0 con 0x7f011015e320 2024-04-17T10:43:44.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.501+0000 7f01161a1640 1 -- 172.21.15.83:0/3252914069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01100ae460 con 0x7f011015e320 2024-04-17T10:43:44.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.501+0000 7f0107fff640 1 -- 172.21.15.83:0/3252914069 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f00f8012070 con 0x7f011015e320 2024-04-17T10:43:44.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.501+0000 7f0107fff640 1 --2- 172.21.15.83:0/3252914069 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f00f0066f60 0x7f00f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:44.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f0107fff640 1 -- 172.21.15.83:0/3252914069 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00f8085740 con 0x7f011015e320 2024-04-17T10:43:44.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f010f7fe640 1 --2- 172.21.15.83:0/3252914069 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f00f0066f60 0x7f00f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:44.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f010f7fe640 1 --2- 172.21.15.83:0/3252914069 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f00f0066f60 0x7f00f0069420 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0100001ec0 tx=0x7f0100009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:44.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f011930e000 1 Processor -- start 2024-04-17T10:43:44.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f011930e000 1 -- start start 2024-04-17T10:43:44.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f011930e000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f268d64b70 0x55f268d64f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:44.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f011930e000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f268d65490 0x55f268d1ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:44.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f011930e000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f268d1b340 0x55f268d1b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:44.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f011930e000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f268bfe5f0 con 0x55f268d64b70 2024-04-17T10:43:44.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f011930e000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f268b65da0 con 0x55f268d65490 2024-04-17T10:43:44.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f011930e000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f268aee150 con 0x55f268d1b340 2024-04-17T10:43:44.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f010f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f268d64b70 0x55f268d64f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:44.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f010f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f268d64b70 0x55f268d64f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52486/0 (socket says 172.21.15.83:52486) 2024-04-17T10:43:44.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f010f7fe640 1 -- 172.21.15.83:0/2466893088 learned_addr learned my addr 172.21.15.83:0/2466893088 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:44.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f010effd640 1 --2- 172.21.15.83:0/2466893088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f268d65490 0x55f268d1ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:44.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f010f7fe640 1 -- 172.21.15.83:0/2466893088 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f268d1b340 msgr2=0x55f268d1b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:44.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f010ffff640 1 --2- 172.21.15.83:0/2466893088 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f268d1b340 0x55f268d1b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:44.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f010f7fe640 1 --2- 172.21.15.83:0/2466893088 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f268d1b340 0x55f268d1b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f010f7fe640 1 -- 172.21.15.83:0/2466893088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f268d65490 msgr2=0x55f268d1ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:44.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f010f7fe640 1 --2- 172.21.15.83:0/2466893088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f268d65490 0x55f268d1ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f010f7fe640 1 -- 172.21.15.83:0/2466893088 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f268b07ca0 con 0x55f268d64b70 2024-04-17T10:43:44.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f010effd640 1 --2- 172.21.15.83:0/2466893088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f268d65490 0x55f268d1ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:44.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f010ffff640 1 --2- 172.21.15.83:0/2466893088 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f268d1b340 0x55f268d1b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:44.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f010f7fe640 1 --2- 172.21.15.83:0/2466893088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f268d64b70 0x55f268d64f50 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f0100010c10 tx=0x7f010000c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6fbc8c11cbf77d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:44.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f01057fa640 1 -- 172.21.15.83:0/2466893088 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0100027070 con 0x55f268d64b70 2024-04-17T10:43:44.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f01057fa640 1 -- 172.21.15.83:0/2466893088 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0100014c10 con 0x55f268d64b70 2024-04-17T10:43:44.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f01057fa640 1 -- 172.21.15.83:0/2466893088 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0100013360 con 0x55f268d64b70 2024-04-17T10:43:44.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f011930e000 1 -- 172.21.15.83:0/2466893088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f268d64b70 msgr2=0x55f268d64f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:44.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f011930e000 1 --2- 172.21.15.83:0/2466893088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f268d64b70 0x55f268d64f50 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f0100010c10 tx=0x7f010000c000 comp rx=0 tx=0).stop 2024-04-17T10:43:44.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f011930e000 1 -- 172.21.15.83:0/2466893088 shutdown_connections 2024-04-17T10:43:44.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f011930e000 1 --2- 172.21.15.83:0/2466893088 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f268d1b340 0x55f268d1b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f011930e000 1 --2- 172.21.15.83:0/2466893088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f268d65490 0x55f268d1ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f011930e000 1 --2- 172.21.15.83:0/2466893088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f268d64b70 0x55f268d64f50 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f011930e000 1 -- 172.21.15.83:0/2466893088 >> 172.21.15.83:0/2466893088 conn(0x55f268d1d2a0 msgr2=0x55f268d152d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:44.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f011930e000 1 -- 172.21.15.83:0/2466893088 shutdown_connections 2024-04-17T10:43:44.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f011930e000 1 -- 172.21.15.83:0/2466893088 wait complete. 2024-04-17T10:43:44.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f011930e000 1 -- start start 2024-04-17T10:43:44.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.505+0000 7f011930e000 1 Processor -- start 2024-04-17T10:43:44.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f011930e000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f268d1b340 0x55f268d4e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:44.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f011930e000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f268d64b70 0x55f268d4ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:44.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f011930e000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f268d65490 0x55f268d4f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:44.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f011930e000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f268b20990 con 0x55f268d65490 2024-04-17T10:43:44.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f011930e000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f268b61e40 con 0x55f268d1b340 2024-04-17T10:43:44.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f011930e000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f268af1900 con 0x55f268d64b70 2024-04-17T10:43:44.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f011930e000 10 client.0 _ll_register_callbacks cb 0x55f268c191c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:43:44.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f010ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f268d65490 0x55f268d4f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:44.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f010ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f268d65490 0x55f268d4f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52494/0 (socket says 172.21.15.83:52494) 2024-04-17T10:43:44.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f010ffff640 1 -- 172.21.15.83:0/2292932325 learned_addr learned my addr 172.21.15.83:0/2292932325 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:44.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f010f7fe640 1 --2- 172.21.15.83:0/2292932325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f268d1b340 0x55f268d4e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:44.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f010ffff640 1 -- 172.21.15.83:0/2292932325 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f268d64b70 msgr2=0x55f268d4ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:44.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f010ffff640 1 --2- 172.21.15.83:0/2292932325 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f268d64b70 0x55f268d4ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f010ffff640 1 -- 172.21.15.83:0/2292932325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f268d1b340 msgr2=0x55f268d4e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:44.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f010ffff640 1 --2- 172.21.15.83:0/2292932325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f268d1b340 0x55f268d4e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f010ffff640 1 -- 172.21.15.83:0/2292932325 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f268b1e630 con 0x55f268d65490 2024-04-17T10:43:44.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f010f7fe640 1 --2- 172.21.15.83:0/2292932325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f268d1b340 0x55f268d4e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:44.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f010ffff640 1 --2- 172.21.15.83:0/2292932325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f268d65490 0x55f268d4f480 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f010801c5b0 tx=0x7f0108019520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac23d6401903798 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:44.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f00ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:43:44.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f00ee7fc640 1 -- 172.21.15.83:0/2292932325 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0108003980 con 0x55f268d65490 2024-04-17T10:43:44.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f00ee7fc640 1 -- 172.21.15.83:0/2292932325 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0108003b20 con 0x55f268d65490 2024-04-17T10:43:44.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f011930e000 1 -- 172.21.15.83:0/2292932325 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0100014c10 con 0x55f268d65490 2024-04-17T10:43:44.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f00ee7fc640 1 -- 172.21.15.83:0/2292932325 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0108003d20 con 0x55f268d65490 2024-04-17T10:43:44.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f011930e000 10 client.7263 fetch_fsmap learned FSMap version 48 2024-04-17T10:43:44.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f011930e000 1 -- 172.21.15.83:0/2292932325 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x55f268aee3a0 con 0x55f268d65490 2024-04-17T10:43:44.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f00ee7fc640 1 -- 172.21.15.83:0/2292932325 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0100014c10 con 0x55f268d65490 2024-04-17T10:43:44.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f00ee7fc640 1 -- 172.21.15.83:0/2292932325 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 48) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0108005030 con 0x55f268d65490 2024-04-17T10:43:44.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f011930e000 10 client.7263 fetch_fsmap finished waiting for FSMap version 48 2024-04-17T10:43:44.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f011930e000 10 client.7263 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-17T10:43:44.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f011930e000 20 client.7263 populate_metadata read hostname 'smithi083' 2024-04-17T10:43:44.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f011930e000 1 --2- 172.21.15.83:0/2292932325 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x55f268d4fe90 0x55f268d50270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:44.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f011930e000 4 client.7263 mds_command: new command op to 7104 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:43:44.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f011930e000 1 -- 172.21.15.83:0/2292932325 --> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f268b01a50 con 0x55f268d4fe90 2024-04-17T10:43:44.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f010f7fe640 1 --2- 172.21.15.83:0/2292932325 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x55f268d4fe90 0x55f268d50270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:44.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f00ee7fc640 10 client.7263 ms_handle_connect on v2:172.21.15.167:6832/2796648400 2024-04-17T10:43:44.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.509+0000 7f010f7fe640 1 --2- 172.21.15.83:0/2292932325 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x55f268d4fe90 0x55f268d50270 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4a644aacd372784 server_cookie=1f9b8bf83b970168 in_seq=0 out_seq=0 2024-04-17T10:43:44.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.513+0000 7f00ee7fc640 1 -- 172.21.15.83:0/2292932325 <== mds.0 v2:172.21.15.167:6832/2796648400 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f268b01a50 con 0x55f268d4fe90 2024-04-17T10:43:44.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.513+0000 7f00ee7fc640 10 client.7263 handle_command_reply: tid=0 2024-04-17T10:43:44.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.513+0000 7f011930e000 1 client.7263 shutdown 2024-04-17T10:43:44.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.513+0000 7f011930e000 1 -- 172.21.15.83:0/2292932325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f268d65490 msgr2=0x55f268d4f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:44.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.513+0000 7f011930e000 1 --2- 172.21.15.83:0/2292932325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f268d65490 0x55f268d4f480 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f010801c5b0 tx=0x7f0108019520 comp rx=0 tx=0).stop 2024-04-17T10:43:44.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.513+0000 7f011930e000 1 -- 172.21.15.83:0/2292932325 shutdown_connections 2024-04-17T10:43:44.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.513+0000 7f011930e000 1 --2- 172.21.15.83:0/2292932325 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x55f268d4fe90 0x55f268d50270 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.513+0000 7f011930e000 1 --2- 172.21.15.83:0/2292932325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f268d65490 0x55f268d4f480 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.513+0000 7f011930e000 1 --2- 172.21.15.83:0/2292932325 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f268d64b70 0x55f268d4ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.513+0000 7f00ee7fc640 0 client.7263 ms_handle_reset on v2:172.21.15.167:6832/2796648400 2024-04-17T10:43:44.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.513+0000 7f011930e000 1 --2- 172.21.15.83:0/2292932325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f268d1b340 0x55f268d4e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.513+0000 7f011930e000 1 -- 172.21.15.83:0/2292932325 >> 172.21.15.83:0/2292932325 conn(0x55f268d1d2a0 msgr2=0x55f268d1c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:44.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.513+0000 7f011930e000 1 -- 172.21.15.83:0/2292932325 shutdown_connections 2024-04-17T10:43:44.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.513+0000 7f011930e000 1 -- 172.21.15.83:0/2292932325 wait complete. 2024-04-17T10:43:44.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.513+0000 7f011930e000 20 client.7263 trim_cache size 0 max 16384 2024-04-17T10:43:44.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f011930e000 1 Processor -- start 2024-04-17T10:43:44.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f011930e000 1 -- start start 2024-04-17T10:43:44.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f011930e000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f268d1b340 0x55f268d1c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:44.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f011930e000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f268d64b70 0x55f268d1cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:44.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f011930e000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f268d65490 0x55f268d770b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:44.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f011930e000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01100b4250 con 0x55f268d65490 2024-04-17T10:43:44.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f011930e000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01100b0960 con 0x55f268d1b340 2024-04-17T10:43:44.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f011930e000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01100b3c50 con 0x55f268d64b70 2024-04-17T10:43:44.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f010ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f268d65490 0x55f268d770b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:44.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f010ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f268d65490 0x55f268d770b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52506/0 (socket says 172.21.15.83:52506) 2024-04-17T10:43:44.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f010ffff640 1 -- 172.21.15.83:0/2945925794 learned_addr learned my addr 172.21.15.83:0/2945925794 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:44.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f010ffff640 1 -- 172.21.15.83:0/2945925794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f268d64b70 msgr2=0x55f268d1cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:44.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f010f7fe640 1 --2- 172.21.15.83:0/2945925794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f268d1b340 0x55f268d1c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:44.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f010effd640 1 --2- 172.21.15.83:0/2945925794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f268d64b70 0x55f268d1cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:44.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f010ffff640 1 --2- 172.21.15.83:0/2945925794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f268d64b70 0x55f268d1cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f010ffff640 1 -- 172.21.15.83:0/2945925794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f268d1b340 msgr2=0x55f268d1c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:44.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f010ffff640 1 --2- 172.21.15.83:0/2945925794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f268d1b340 0x55f268d1c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f010ffff640 1 -- 172.21.15.83:0/2945925794 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f268d1b170 con 0x55f268d65490 2024-04-17T10:43:44.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f010f7fe640 1 --2- 172.21.15.83:0/2945925794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f268d1b340 0x55f268d1c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:44.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f010ffff640 1 --2- 172.21.15.83:0/2945925794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f268d65490 0x55f268d770b0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f010801d020 tx=0x7f01080b0000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b615b3c9bd491e5b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:44.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f0104ff9640 1 -- 172.21.15.83:0/2945925794 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0108003980 con 0x55f268d65490 2024-04-17T10:43:44.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f0104ff9640 1 -- 172.21.15.83:0/2945925794 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0108003b20 con 0x55f268d65490 2024-04-17T10:43:44.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f0104ff9640 1 -- 172.21.15.83:0/2945925794 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01080a6680 con 0x55f268d65490 2024-04-17T10:43:44.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f011930e000 1 -- 172.21.15.83:0/2945925794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f268d65490 msgr2=0x55f268d770b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:44.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f011930e000 1 --2- 172.21.15.83:0/2945925794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f268d65490 0x55f268d770b0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f010801d020 tx=0x7f01080b0000 comp rx=0 tx=0).stop 2024-04-17T10:43:44.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f011930e000 1 -- 172.21.15.83:0/2945925794 shutdown_connections 2024-04-17T10:43:44.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f011930e000 1 --2- 172.21.15.83:0/2945925794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f268d65490 0x55f268d770b0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f011930e000 1 --2- 172.21.15.83:0/2945925794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f268d64b70 0x55f268d1cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f011930e000 1 --2- 172.21.15.83:0/2945925794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f268d1b340 0x55f268d1c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f011930e000 1 -- 172.21.15.83:0/2945925794 >> 172.21.15.83:0/2945925794 conn(0x55f268d1d2a0 msgr2=0x55f268d15940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:44.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f011930e000 1 -- 172.21.15.83:0/2945925794 shutdown_connections 2024-04-17T10:43:44.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f011930e000 1 -- 172.21.15.83:0/2945925794 wait complete. 2024-04-17T10:43:44.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f011930e000 1 -- start start 2024-04-17T10:43:44.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.529+0000 7f011930e000 1 Processor -- start 2024-04-17T10:43:44.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f011930e000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f268d1b340 0x55f268d7a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:44.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f011930e000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f268d64b70 0x55f268dc1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:44.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f011930e000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f268d65490 0x55f268dc17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:44.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f011930e000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f268b6a3d0 con 0x55f268d65490 2024-04-17T10:43:44.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f011930e000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f268c21d50 con 0x55f268d1b340 2024-04-17T10:43:44.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f011930e000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f268b00980 con 0x55f268d64b70 2024-04-17T10:43:44.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f011930e000 10 client.0 _ll_register_callbacks cb 0x7f0100005860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:43:44.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f010ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f268d65490 0x55f268dc17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:44.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f010ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f268d65490 0x55f268dc17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52514/0 (socket says 172.21.15.83:52514) 2024-04-17T10:43:44.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f010ffff640 1 -- 172.21.15.83:0/915623764 learned_addr learned my addr 172.21.15.83:0/915623764 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:44.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f010f7fe640 1 --2- 172.21.15.83:0/915623764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f268d1b340 0x55f268d7a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:44.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f010effd640 1 --2- 172.21.15.83:0/915623764 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f268d64b70 0x55f268dc1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:44.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f010ffff640 1 -- 172.21.15.83:0/915623764 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f268d64b70 msgr2=0x55f268dc1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:44.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f010ffff640 1 --2- 172.21.15.83:0/915623764 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f268d64b70 0x55f268dc1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f010ffff640 1 -- 172.21.15.83:0/915623764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f268d1b340 msgr2=0x55f268d7a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:44.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f010ffff640 1 --2- 172.21.15.83:0/915623764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f268d1b340 0x55f268d7a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f010ffff640 1 -- 172.21.15.83:0/915623764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f268d652c0 con 0x55f268d65490 2024-04-17T10:43:44.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f010f7fe640 1 --2- 172.21.15.83:0/915623764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f268d1b340 0x55f268d7a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:44.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f010ffff640 1 --2- 172.21.15.83:0/915623764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f268d65490 0x55f268dc17d0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f010801ccf0 tx=0x7f0108007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33bd0a3cb0e2a35b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:44.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f00ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:43:44.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f00ee7fc640 1 -- 172.21.15.83:0/915623764 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01080a5150 con 0x55f268d65490 2024-04-17T10:43:44.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f00ee7fc640 1 -- 172.21.15.83:0/915623764 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01080a6a40 con 0x55f268d65490 2024-04-17T10:43:44.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f011930e000 1 -- 172.21.15.83:0/915623764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0108003b20 con 0x55f268d65490 2024-04-17T10:43:44.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f00ee7fc640 1 -- 172.21.15.83:0/915623764 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01080d1050 con 0x55f268d65490 2024-04-17T10:43:44.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f011930e000 10 client.7269 fetch_fsmap learned FSMap version 48 2024-04-17T10:43:44.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f011930e000 1 -- 172.21.15.83:0/915623764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x55f268be7600 con 0x55f268d65490 2024-04-17T10:43:44.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f00ee7fc640 1 -- 172.21.15.83:0/915623764 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0108003b20 con 0x55f268d65490 2024-04-17T10:43:44.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f00ee7fc640 1 -- 172.21.15.83:0/915623764 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 48) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f01080a6c40 con 0x55f268d65490 2024-04-17T10:43:44.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f011930e000 10 client.7269 fetch_fsmap finished waiting for FSMap version 48 2024-04-17T10:43:44.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f011930e000 10 client.7269 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-17T10:43:44.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f011930e000 20 client.7269 populate_metadata read hostname 'smithi083' 2024-04-17T10:43:44.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f011930e000 1 --2- 172.21.15.83:0/915623764 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x55f268dc1d10 0x55f268dc20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:44.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f011930e000 4 client.7269 mds_command: new command op to 7104 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T10:43:44.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f011930e000 1 -- 172.21.15.83:0/915623764 --> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f268b732e0 con 0x55f268dc1d10 2024-04-17T10:43:44.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f010f7fe640 1 --2- 172.21.15.83:0/915623764 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x55f268dc1d10 0x55f268dc20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:44.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f010f7fe640 1 --2- 172.21.15.83:0/915623764 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x55f268dc1d10 0x55f268dc20f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a1e1333f0868c11 server_cookie=53c5f61a0429ce0b in_seq=0 out_seq=0 2024-04-17T10:43:44.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f00ee7fc640 10 client.7269 ms_handle_connect on v2:172.21.15.167:6832/2796648400 2024-04-17T10:43:44.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f00ee7fc640 1 -- 172.21.15.83:0/915623764 <== mds.0 v2:172.21.15.167:6832/2796648400 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55f268b732e0 con 0x55f268dc1d10 2024-04-17T10:43:44.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f00ee7fc640 10 client.7269 handle_command_reply: tid=0 2024-04-17T10:43:44.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f011930e000 1 client.7269 shutdown 2024-04-17T10:43:44.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f011930e000 1 -- 172.21.15.83:0/915623764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f268d65490 msgr2=0x55f268dc17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:44.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f011930e000 1 --2- 172.21.15.83:0/915623764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f268d65490 0x55f268dc17d0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f010801ccf0 tx=0x7f0108007f90 comp rx=0 tx=0).stop 2024-04-17T10:43:44.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f011930e000 1 -- 172.21.15.83:0/915623764 shutdown_connections 2024-04-17T10:43:44.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f011930e000 1 --2- 172.21.15.83:0/915623764 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x55f268dc1d10 0x55f268dc20f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f011930e000 1 --2- 172.21.15.83:0/915623764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f268d65490 0x55f268dc17d0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f00ee7fc640 0 client.7269 ms_handle_reset on v2:172.21.15.167:6832/2796648400 2024-04-17T10:43:44.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f011930e000 1 --2- 172.21.15.83:0/915623764 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f268d64b70 0x55f268dc1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f011930e000 1 --2- 172.21.15.83:0/915623764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f268d1b340 0x55f268d7a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f011930e000 1 -- 172.21.15.83:0/915623764 >> 172.21.15.83:0/915623764 conn(0x55f268d1d2a0 msgr2=0x55f268d1b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:44.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f011930e000 1 -- 172.21.15.83:0/915623764 shutdown_connections 2024-04-17T10:43:44.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f011930e000 1 -- 172.21.15.83:0/915623764 wait complete. 2024-04-17T10:43:44.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f011930e000 20 client.7269 trim_cache size 0 max 16384 2024-04-17T10:43:44.537 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:43:44.537 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "3de9b7be-af7f-488a-969e-2b6b8db63bcd", 2024-04-17T10:43:44.537 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-04-17T10:43:44.537 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 7104, 2024-04-17T10:43:44.537 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-17T10:43:44.537 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-17T10:43:44.537 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-17T10:43:44.537 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 10.034175146000001, 2024-04-17T10:43:44.537 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 48, 2024-04-17T10:43:44.538 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 74, 2024-04-17T10:43:44.538 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 74, 2024-04-17T10:43:44.538 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 15.098732963 2024-04-17T10:43:44.538 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:43:44.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f01161a1640 1 -- 172.21.15.83:0/3252914069 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f00f0066f60 msgr2=0x7f00f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:44.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f01161a1640 1 --2- 172.21.15.83:0/3252914069 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f00f0066f60 0x7f00f0069420 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0100001ec0 tx=0x7f0100009040 comp rx=0 tx=0).stop 2024-04-17T10:43:44.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f01161a1640 1 -- 172.21.15.83:0/3252914069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f011015e320 msgr2=0x7f011015e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:44.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f01161a1640 1 --2- 172.21.15.83:0/3252914069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f011015e320 0x7f011015e7a0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f00f800f580 tx=0x7f00f800fa40 comp rx=0 tx=0).stop 2024-04-17T10:43:44.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f01161a1640 1 -- 172.21.15.83:0/3252914069 shutdown_connections 2024-04-17T10:43:44.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f01161a1640 1 --2- 172.21.15.83:0/3252914069 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f00f0066f60 0x7f00f0069420 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f01161a1640 1 --2- 172.21.15.83:0/3252914069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01100ad1c0 0x7f01100ad640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f01161a1640 1 --2- 172.21.15.83:0/3252914069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f011015e320 0x7f011015e7a0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f01161a1640 1 --2- 172.21.15.83:0/3252914069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00fc004da0 0x7f0110146f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f01161a1640 1 -- 172.21.15.83:0/3252914069 >> 172.21.15.83:0/3252914069 conn(0x7f01100b15b0 msgr2=0x7f01100cdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:44.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f01161a1640 1 -- 172.21.15.83:0/3252914069 shutdown_connections 2024-04-17T10:43:44.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.533+0000 7f01161a1640 1 -- 172.21.15.83:0/3252914069 wait complete. 2024-04-17T10:43:44.547 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T10:43:44.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.661+0000 7f679bd28640 1 Processor -- start 2024-04-17T10:43:44.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.661+0000 7f679bd28640 1 -- start start 2024-04-17T10:43:44.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.665+0000 7f679bd28640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f679415db50 0x7f6794070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:44.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.665+0000 7f679bd28640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67940709b0 con 0x7f679406dcb0 2024-04-17T10:43:44.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.665+0000 7f679bd28640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f679415e440 con 0x7f67940aedd0 2024-04-17T10:43:44.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.665+0000 7f679bd28640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f679415e720 con 0x7f679415db50 2024-04-17T10:43:44.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.665+0000 7f679a2c5640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f679406dcb0 0x7f67940b2e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55132/0 (socket says 172.21.15.83:55132) 2024-04-17T10:43:44.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.665+0000 7f679a2c5640 1 -- 172.21.15.83:0/552506188 learned_addr learned my addr 172.21.15.83:0/552506188 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:43:44.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.665+0000 7f6798ac2640 1 -- 172.21.15.83:0/552506188 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1876315444 0 0) 0x7f678c0014a0 con 0x7f679406dcb0 2024-04-17T10:43:44.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.665+0000 7f6799ac4640 1 --2- 172.21.15.83:0/552506188 >> v2:172.21.15.167:3300/0 conn(0x7f679415db50 0x7f6794070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.665+0000 7f6798ac2640 1 -- 172.21.15.83:0/552506188 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6780003530 con 0x7f679406dcb0 2024-04-17T10:43:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.665+0000 7f6798ac2640 1 -- 172.21.15.83:0/552506188 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2548107333 0 0) 0x7f6780003530 con 0x7f679406dcb0 2024-04-17T10:43:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.665+0000 7f6798ac2640 1 -- 172.21.15.83:0/552506188 >> v2:172.21.15.167:3300/0 conn(0x7f679415db50 msgr2=0x7f6794070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.665+0000 7f6798ac2640 1 --2- 172.21.15.83:0/552506188 >> v2:172.21.15.167:3300/0 conn(0x7f679415db50 0x7f6794070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.665+0000 7f6798ac2640 1 -- 172.21.15.83:0/552506188 >> v1:172.21.15.167:6789/0 conn(0x7f67940aedd0 legacy=0x7f679406d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.665+0000 7f6798ac2640 1 -- 172.21.15.83:0/552506188 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679415ea00 con 0x7f679406dcb0 2024-04-17T10:43:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.665+0000 7f6798ac2640 1 -- 172.21.15.83:0/552506188 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f678c001bd0 con 0x7f679406dcb0 2024-04-17T10:43:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.665+0000 7f6798ac2640 1 -- 172.21.15.83:0/552506188 >> v1:172.21.15.83:6789/0 conn(0x7f679406dcb0 legacy=0x7f67940b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.665+0000 7f6798ac2640 1 --2- 172.21.15.83:0/552506188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67800039c0 0x7f6780003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.665+0000 7f6798ac2640 1 --2- 172.21.15.83:0/552506188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6780004810 0x7f6780004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.665+0000 7f6798ac2640 1 --2- 172.21.15.83:0/552506188 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67800051d0 0x7f678000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.665+0000 7f6798ac2640 1 -- 172.21.15.83:0/552506188 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679415e720 con 0x7f6780004810 2024-04-17T10:43:44.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.665+0000 7f6798ac2640 1 -- 172.21.15.83:0/552506188 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678000c070 con 0x7f67800039c0 2024-04-17T10:43:44.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.665+0000 7f6798ac2640 1 -- 172.21.15.83:0/552506188 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678000c270 con 0x7f67800051d0 2024-04-17T10:43:44.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.665+0000 7f67992c3640 1 --2- 172.21.15.83:0/552506188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6780004810 0x7f6780004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:44.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.665+0000 7f679a2c5640 1 --2- 172.21.15.83:0/552506188 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67800051d0 0x7f678000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:44.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.665+0000 7f67992c3640 1 -- 172.21.15.83:0/552506188 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67800051d0 msgr2=0x7f678000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:44.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.665+0000 7f6799ac4640 1 --2- 172.21.15.83:0/552506188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67800039c0 0x7f6780003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:44.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.665+0000 7f67992c3640 1 --2- 172.21.15.83:0/552506188 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67800051d0 0x7f678000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.665+0000 7f67992c3640 1 -- 172.21.15.83:0/552506188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67800039c0 msgr2=0x7f6780003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:44.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.665+0000 7f67992c3640 1 --2- 172.21.15.83:0/552506188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67800039c0 0x7f6780003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.665+0000 7f679a2c5640 1 -- 172.21.15.83:0/552506188 reap_dead start 2024-04-17T10:43:44.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.665+0000 7f67992c3640 1 -- 172.21.15.83:0/552506188 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6780003530 con 0x7f6780004810 2024-04-17T10:43:44.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f67992c3640 1 --2- 172.21.15.83:0/552506188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6780004810 0x7f6780004c90 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f6790001780 tx=0x7f679001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330495f718117607 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:44.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f6798ac2640 1 -- 172.21.15.83:0/552506188 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67900013a0 con 0x7f6780004810 2024-04-17T10:43:44.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f6798ac2640 1 -- 172.21.15.83:0/552506188 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67900052f0 con 0x7f6780004810 2024-04-17T10:43:44.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f6798ac2640 1 -- 172.21.15.83:0/552506188 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6790003840 con 0x7f6780004810 2024-04-17T10:43:44.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f679bd28640 1 -- 172.21.15.83:0/552506188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6780004810 msgr2=0x7f6780004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:44.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f679bd28640 1 --2- 172.21.15.83:0/552506188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6780004810 0x7f6780004c90 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f6790001780 tx=0x7f679001c400 comp rx=0 tx=0).stop 2024-04-17T10:43:44.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f679bd28640 1 -- 172.21.15.83:0/552506188 shutdown_connections 2024-04-17T10:43:44.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f679bd28640 1 --2- 172.21.15.83:0/552506188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6780004810 0x7f6780004c90 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f679bd28640 1 -- 172.21.15.83:0/552506188 >> 172.21.15.83:0/552506188 conn(0x7f679406abe0 msgr2=0x7f679406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:44.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f679bd28640 1 -- 172.21.15.83:0/552506188 shutdown_connections 2024-04-17T10:43:44.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f679bd28640 1 -- 172.21.15.83:0/552506188 wait complete. 2024-04-17T10:43:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f679bd28640 1 Processor -- start 2024-04-17T10:43:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f679bd28640 1 -- start start 2024-04-17T10:43:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f679bd28640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6780004810 0x7f67941f3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f679bd28640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f679415b7e0 0x7f67941f6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f679bd28640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f679414c4d0 0x7f67940ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f679bd28640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679415df30 con 0x7f6780004810 2024-04-17T10:43:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f679bd28640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679415efc0 con 0x7f679414c4d0 2024-04-17T10:43:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f679bd28640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f679415e0b0 con 0x7f679415b7e0 2024-04-17T10:43:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f6799ac4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6780004810 0x7f67941f3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f6799ac4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6780004810 0x7f67941f3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52528/0 (socket says 172.21.15.83:52528) 2024-04-17T10:43:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f6799ac4640 1 -- 172.21.15.83:0/3238221763 learned_addr learned my addr 172.21.15.83:0/3238221763 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f679a2c5640 1 --2- 172.21.15.83:0/3238221763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f679414c4d0 0x7f67940ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f67992c3640 1 --2- 172.21.15.83:0/3238221763 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f679415b7e0 0x7f67941f6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f6799ac4640 1 -- 172.21.15.83:0/3238221763 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f679415b7e0 msgr2=0x7f67941f6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:44.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f6799ac4640 1 --2- 172.21.15.83:0/3238221763 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f679415b7e0 0x7f67941f6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f6799ac4640 1 -- 172.21.15.83:0/3238221763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f679414c4d0 msgr2=0x7f67940ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:44.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f6799ac4640 1 --2- 172.21.15.83:0/3238221763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f679414c4d0 0x7f67940ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f6799ac4640 1 -- 172.21.15.83:0/3238221763 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67941f6fa0 con 0x7f6780004810 2024-04-17T10:43:44.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f6799ac4640 1 --2- 172.21.15.83:0/3238221763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6780004810 0x7f67941f3cc0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f6784010830 tx=0x7f6784010cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:44.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f678a7fc640 1 -- 172.21.15.83:0/3238221763 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678400f040 con 0x7f6780004810 2024-04-17T10:43:44.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f678a7fc640 1 -- 172.21.15.83:0/3238221763 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678400eb00 con 0x7f6780004810 2024-04-17T10:43:44.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f679bd28640 1 -- 172.21.15.83:0/3238221763 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f679414cd50 con 0x7f6780004810 2024-04-17T10:43:44.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f678a7fc640 1 -- 172.21.15.83:0/3238221763 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678401c400 con 0x7f6780004810 2024-04-17T10:43:44.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.669+0000 7f679bd28640 1 -- 172.21.15.83:0/3238221763 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f679414d310 con 0x7f6780004810 2024-04-17T10:43:44.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.673+0000 7f679bd28640 1 -- 172.21.15.83:0/3238221763 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6794074f90 con 0x7f6780004810 2024-04-17T10:43:44.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.673+0000 7f678a7fc640 1 -- 172.21.15.83:0/3238221763 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f678401c5a0 con 0x7f6780004810 2024-04-17T10:43:44.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.673+0000 7f678a7fc640 1 --2- 172.21.15.83:0/3238221763 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6778066f90 0x7f6778069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:44.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.673+0000 7f67992c3640 1 --2- 172.21.15.83:0/3238221763 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6778066f90 0x7f6778069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:44.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.673+0000 7f678a7fc640 1 -- 172.21.15.83:0/3238221763 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6784085600 con 0x7f6780004810 2024-04-17T10:43:44.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.673+0000 7f67992c3640 1 --2- 172.21.15.83:0/3238221763 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6778066f90 0x7f6778069450 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6790004000 tx=0x7f67900011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:44.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.677+0000 7f678a7fc640 1 -- 172.21.15.83:0/3238221763 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6784018030 con 0x7f6780004810 2024-04-17T10:43:44.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.865+0000 7f679bd28640 1 -- 172.21.15.83:0/3238221763 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f67940b2af0 con 0x7f6780004810 2024-04-17T10:43:44.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.865+0000 7f678a7fc640 1 -- 172.21.15.83:0/3238221763 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v48) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6784056610 con 0x7f6780004810 2024-04-17T10:43:44.860 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T10:43:44.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.865+0000 7f679bd28640 1 -- 172.21.15.83:0/3238221763 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6778066f90 msgr2=0x7f6778069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:44.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.865+0000 7f679bd28640 1 --2- 172.21.15.83:0/3238221763 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6778066f90 0x7f6778069450 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6790004000 tx=0x7f67900011a0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.865+0000 7f679bd28640 1 -- 172.21.15.83:0/3238221763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6780004810 msgr2=0x7f67941f3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:44.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.865+0000 7f679bd28640 1 --2- 172.21.15.83:0/3238221763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6780004810 0x7f67941f3cc0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f6784010830 tx=0x7f6784010cf0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.865+0000 7f679bd28640 1 -- 172.21.15.83:0/3238221763 shutdown_connections 2024-04-17T10:43:44.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.865+0000 7f679bd28640 1 --2- 172.21.15.83:0/3238221763 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6778066f90 0x7f6778069450 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.865+0000 7f679bd28640 1 --2- 172.21.15.83:0/3238221763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f679414c4d0 0x7f67940ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.865+0000 7f679bd28640 1 --2- 172.21.15.83:0/3238221763 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f679415b7e0 0x7f67941f6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.865+0000 7f679bd28640 1 --2- 172.21.15.83:0/3238221763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6780004810 0x7f67941f3cc0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:44.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.865+0000 7f679bd28640 1 -- 172.21.15.83:0/3238221763 >> 172.21.15.83:0/3238221763 conn(0x7f679406abe0 msgr2=0x7f679406c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:44.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.865+0000 7f679bd28640 1 -- 172.21.15.83:0/3238221763 shutdown_connections 2024-04-17T10:43:44.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:44.865+0000 7f679bd28640 1 -- 172.21.15.83:0/3238221763 wait complete. 2024-04-17T10:43:44.878 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T10:43:44.878 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T10:43:44.878 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-04-17T10:43:44.878 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T10:43:44.878 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T10:43:44.878 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T10:43:44.878 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T10:43:44.878 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T10:43:44.878 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:43:44.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:43:44.879 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-04-17T10:43:44.885 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T10:43:44.886 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T10:43:44.886 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T10:43:44.886 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:43:44.886 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-17T10:43:44.886 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:43:44.886 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:43:44.886 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-04-17T10:43:44.886 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-04-17T10:43:44.886 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:43:44.887 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-04-17T10:43:44.887 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-04-17T10:43:44.887 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:43:44.887 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:43:44.887 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T10:43:44.887 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:43:44.887 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-17T10:43:44.887 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42159sec preferred_lft 42159sec 2024-04-17T10:43:44.887 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-04-17T10:43:44.887 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:43:44.888 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T10:43:44.888 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:37:0e:23:b2 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:43:44.888 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T10:43:44.888 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:43:44.888 INFO:teuthology.orchestra.run.smithi083.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T10:43:44.888 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-17T10:43:44.888 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T10:43:44.888 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:43:44.888 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-17T10:43:44.889 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:43:44.889 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:43:44.889 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:43:44.889 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-17T10:43:44.889 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:43:44.889 INFO:teuthology.orchestra.run.smithi083.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:43:44.889 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:43:44.889 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-17T10:43:44.889 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:43:44.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:43:44.890 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T10:43:44.937 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-04-17T10:43:44.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:43:44.939 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-04-17T10:43:44.991 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-04-17T10:43:44.991 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T10:43:44.991 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-17T10:43:44.991 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-17T10:43:44.992 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-17T10:43:44.992 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T10:43:44.992 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-17T10:43:44.992 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:43:44.992 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:43:44.992 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:43:44.992 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T10:43:44.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:43:44.992 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2024-04-17T10:43:44.992 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:43:44.993 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-17T10:43:44.993 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-17T10:43:44.993 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-17T10:43:44.993 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:43:45.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:43:45.067 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-17T10:43:45.116 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-17T10:43:45.116 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T10:43:45.117 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:43:45.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:43:45.117 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-04-17T10:43:45.171 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-17T10:43:45.171 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:43:45.172 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T10:43:45.220 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:43:45.220 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:43:45.220 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:43:45.268 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:43:45.325 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:43:45.374 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:43:45.374 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:43:45.374 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:43:45.374 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:43:45.374 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:43:45.374 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:43:45.374 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:43:45.375 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:43:45.375 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:43:45.375 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:43:45.375 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:43:45.375 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:43:45.375 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:43:45.375 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:43:45.375 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:43:45.375 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:43:45.375 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:43:45.376 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:43:45.376 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:43:45.376 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:43:45.376 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:43:45.376 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:43:45.376 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:43:45.376 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:43:45.376 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:43:45.376 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:43:45.376 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:43:45.376 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:43:45.376 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:43:45.376 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:43:45.376 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:43:45.377 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:43:45.377 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:43:45.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:43:45.378 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:43:45.428 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T10:43:45.428 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-17T10:43:45.472 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:43:45.488 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:43:45.536 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:43:45.536 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:43:45.536 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:43:45.536 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:43:45.536 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:43:45.536 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:43:45.536 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:43:45.536 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:43:45.536 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:43:45.536 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:43:45.537 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:43:45.537 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:43:45.537 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:43:45.537 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:43:45.537 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:43:45.537 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:43:45.537 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:43:45.537 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:43:45.537 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:43:45.537 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:43:45.537 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:43:45.537 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:43:45.537 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:43:45.537 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:43:45.538 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:43:45.538 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:43:45.538 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:43:45.538 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:43:45.538 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:43:45.538 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:43:45.538 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:43:45.538 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:43:45.538 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:43:45.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:43:45.539 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:43:45.554 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:2024-04-17T10:43:45.557+0000 7f0ad4ded580 -1 init, newargv = 0x558fbfc94360 newargc=15 2024-04-17T10:43:45.554 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[56841]: starting ceph client 2024-04-17T10:43:45.560 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[56841]: starting fuse 2024-04-17T10:43:46.659 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:43:46.677 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:43:46.726 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:43:46.726 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:43:46.727 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:43:46.727 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:43:46.727 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:43:46.727 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:43:46.727 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:43:46.727 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:43:46.727 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:43:46.727 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:43:46.727 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:43:46.727 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:43:46.727 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:43:46.727 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:43:46.727 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:43:46.727 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:43:46.728 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:43:46.728 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:43:46.728 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:43:46.728 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:43:46.728 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:43:46.728 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:43:46.728 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:43:46.728 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:43:46.728 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:43:46.728 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:43:46.728 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:43:46.728 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:43:46.728 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:43:46.729 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:43:46.729 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:43:46.729 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:43:46.729 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:43:46.729 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-17T10:43:46.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:43:46.730 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:43:46.775 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-04-17T10:43:46.776 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-17T10:43:46.776 DEBUG:teuthology.orchestra.run.smithi083:> sudo stdin-killer -- python3 -c ' 2024-04-17T10:43:46.776 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2024-04-17T10:43:46.776 DEBUG:teuthology.orchestra.run.smithi083:> import re 2024-04-17T10:43:46.776 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-04-17T10:43:46.776 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2024-04-17T10:43:46.776 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:43:46.776 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2024-04-17T10:43:46.776 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-17T10:43:46.777 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2024-04-17T10:43:46.777 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-17T10:43:46.777 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:43:46.777 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2024-04-17T10:43:46.777 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2024-04-17T10:43:46.777 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2024-04-17T10:43:46.777 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2024-04-17T10:43:46.777 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:43:46.777 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2024-04-17T10:43:46.777 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T10:43:46.777 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T10:43:46.777 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T10:43:46.777 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2024-04-17T10:43:46.777 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2024-04-17T10:43:46.777 DEBUG:teuthology.orchestra.run.smithi083:> return f 2024-04-17T10:43:46.778 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T10:43:46.778 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:43:46.778 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.0")) 2024-04-17T10:43:46.778 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-04-17T10:43:46.900 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.0.56841.asok 2024-04-17T10:43:46.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:43:46.912 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.56841.asok 2024-04-17T10:43:46.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:43:46.912 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.56841.asok status 2024-04-17T10:43:47.070 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:43:47.070 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2024-04-17T10:43:47.070 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-17T10:43:47.070 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-17T10:43:47.070 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "0", 2024-04-17T10:43:47.070 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-04-17T10:43:47.070 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-17T10:43:47.071 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "56841", 2024-04-17T10:43:47.071 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-04-17T10:43:47.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:43:47.071 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2024-04-17T10:43:47.071 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2024-04-17T10:43:47.071 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 7281, 2024-04-17T10:43:47.071 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2024-04-17T10:43:47.071 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-04-17T10:43:47.071 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-04-17T10:43:47.071 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 7281 2024-04-17T10:43:47.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:43:47.071 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-04-17T10:43:47.071 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-17T10:43:47.072 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:43:47.072 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 39057851 2024-04-17T10:43:47.072 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:43:47.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:43:47.072 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-04-17T10:43:47.072 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-17T10:43:47.072 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:43:47.072 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 39057851 2024-04-17T10:43:47.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:43:47.072 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.7281 192.168.0.1:0/39057851", 2024-04-17T10:43:47.072 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.1:0/39057851", 2024-04-17T10:43:47.072 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 1, 2024-04-17T10:43:47.072 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 48, 2024-04-17T10:43:47.073 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 74, 2024-04-17T10:43:47.073 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2024-04-17T10:43:47.073 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2024-04-17T10:43:47.073 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2024-04-17T10:43:47.073 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:43:47.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:43:47.084 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2024-04-17T10:43:47.084 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-17T10:43:47.090 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2024-04-17T10:43:47.091 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-17T10:43:47.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:43:47.091 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:43:47.149 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-04-17T10:43:47.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.325+0000 7f8a23d1b640 1 Processor -- start 2024-04-17T10:43:47.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.325+0000 7f8a23d1b640 1 -- start start 2024-04-17T10:43:47.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.325+0000 7f8a23d1b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8a1c06ebf0 0x7f8a1c071580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:47.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.325+0000 7f8a23d1b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a1c071ac0 0x7f8a1c15c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:47.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.325+0000 7f8a23d1b640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a1c0cfee0 con 0x7f8a1c15a890 2024-04-17T10:43:47.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.325+0000 7f8a23d1b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a1c06e760 con 0x7f8a1c06ebf0 2024-04-17T10:43:47.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.325+0000 7f8a23d1b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c0d0180 con 0x7f8a1c071ac0 2024-04-17T10:43:47.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.325+0000 7f8a212b6640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8a1c06ebf0 0x7f8a1c071580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:47.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.325+0000 7f8a212b6640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8a1c06ebf0 0x7f8a1c071580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51972/0 (socket says 172.21.15.83:51972) 2024-04-17T10:43:47.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.325+0000 7f8a212b6640 1 -- 172.21.15.83:0/3672700819 learned_addr learned my addr 172.21.15.83:0/3672700819 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:47.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.325+0000 7f8a222b8640 1 --2- 172.21.15.83:0/3672700819 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a1c071ac0 0x7f8a1c15c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:47.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.325+0000 7f8a212b6640 1 -- 172.21.15.83:0/3672700819 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a1c071ac0 msgr2=0x7f8a1c15c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:43:47.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.325+0000 7f8a212b6640 1 --2- 172.21.15.83:0/3672700819 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a1c071ac0 0x7f8a1c15c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:47.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.325+0000 7f8a212b6640 1 -- 172.21.15.83:0/3672700819 >> v1:172.21.15.167:6789/0 conn(0x7f8a1c15a890 legacy=0x7f8a1c15ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:47.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.325+0000 7f8a212b6640 1 -- 172.21.15.83:0/3672700819 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a1c0d0460 con 0x7f8a1c06ebf0 2024-04-17T10:43:47.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.325+0000 7f8a222b8640 1 --2- 172.21.15.83:0/3672700819 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a1c071ac0 0x7f8a1c15c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:43:47.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.329+0000 7f8a212b6640 1 --2- 172.21.15.83:0/3672700819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a1c06ebf0 0x7f8a1c071580 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f8a10009870 tx=0x7f8a1001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96706d48046dd4ab server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:47.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.329+0000 7f8a20ab5640 1 -- 172.21.15.83:0/3672700819 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a100ab650 con 0x7f8a1c06ebf0 2024-04-17T10:43:47.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.329+0000 7f8a20ab5640 1 -- 172.21.15.83:0/3672700819 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a100b0070 con 0x7f8a1c06ebf0 2024-04-17T10:43:47.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.329+0000 7f8a20ab5640 1 -- 172.21.15.83:0/3672700819 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a100aba10 con 0x7f8a1c06ebf0 2024-04-17T10:43:47.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.329+0000 7f8a23d1b640 1 -- 172.21.15.83:0/3672700819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a1c06ebf0 msgr2=0x7f8a1c071580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:47.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.329+0000 7f8a23d1b640 1 --2- 172.21.15.83:0/3672700819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a1c06ebf0 0x7f8a1c071580 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f8a10009870 tx=0x7f8a1001c560 comp rx=0 tx=0).stop 2024-04-17T10:43:47.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.329+0000 7f8a23d1b640 1 -- 172.21.15.83:0/3672700819 shutdown_connections 2024-04-17T10:43:47.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.329+0000 7f8a23d1b640 1 --2- 172.21.15.83:0/3672700819 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a1c071ac0 0x7f8a1c15c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:47.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.329+0000 7f8a23d1b640 1 --2- 172.21.15.83:0/3672700819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a1c06ebf0 0x7f8a1c071580 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:47.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.329+0000 7f8a23d1b640 1 -- 172.21.15.83:0/3672700819 >> 172.21.15.83:0/3672700819 conn(0x7f8a1c0b2410 msgr2=0x7f8a1c0b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:47.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.329+0000 7f8a23d1b640 1 -- 172.21.15.83:0/3672700819 shutdown_connections 2024-04-17T10:43:47.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.329+0000 7f8a23d1b640 1 -- 172.21.15.83:0/3672700819 wait complete. 2024-04-17T10:43:47.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.329+0000 7f8a23d1b640 1 Processor -- start 2024-04-17T10:43:47.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.329+0000 7f8a23d1b640 1 -- start start 2024-04-17T10:43:47.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.333+0000 7f8a23d1b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a1c06ebf0 0x7f8a1c147340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:47.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.333+0000 7f8a23d1b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a1c071ac0 0x7f8a1c151030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:47.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.333+0000 7f8a23d1b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a1c15a890 0x7f8a1c151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.333+0000 7f8a23d1b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c06de60 con 0x7f8a1c06ebf0 2024-04-17T10:43:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.333+0000 7f8a23d1b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c06dce0 con 0x7f8a1c071ac0 2024-04-17T10:43:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.333+0000 7f8a23d1b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c15cb40 con 0x7f8a1c15a890 2024-04-17T10:43:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.333+0000 7f8a21ab7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a1c06ebf0 0x7f8a1c147340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.333+0000 7f8a21ab7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a1c06ebf0 0x7f8a1c147340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51980/0 (socket says 172.21.15.83:51980) 2024-04-17T10:43:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.333+0000 7f8a21ab7640 1 -- 172.21.15.83:0/1408245776 learned_addr learned my addr 172.21.15.83:0/1408245776 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:43:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.333+0000 7f8a212b6640 1 --2- 172.21.15.83:0/1408245776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a1c071ac0 0x7f8a1c151030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.333+0000 7f8a222b8640 1 --2- 172.21.15.83:0/1408245776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a1c15a890 0x7f8a1c151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.333+0000 7f8a21ab7640 1 -- 172.21.15.83:0/1408245776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a1c15a890 msgr2=0x7f8a1c151570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.333+0000 7f8a21ab7640 1 --2- 172.21.15.83:0/1408245776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a1c15a890 0x7f8a1c151570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.333+0000 7f8a21ab7640 1 -- 172.21.15.83:0/1408245776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a1c071ac0 msgr2=0x7f8a1c151030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.333+0000 7f8a21ab7640 1 --2- 172.21.15.83:0/1408245776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a1c071ac0 0x7f8a1c151030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:47.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.333+0000 7f8a21ab7640 1 -- 172.21.15.83:0/1408245776 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a1c14a080 con 0x7f8a1c06ebf0 2024-04-17T10:43:47.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.333+0000 7f8a21ab7640 1 --2- 172.21.15.83:0/1408245776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a1c06ebf0 0x7f8a1c147340 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400c730 tx=0x7f8a1400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:47.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.333+0000 7f8a1a7fc640 1 -- 172.21.15.83:0/1408245776 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a14016020 con 0x7f8a1c06ebf0 2024-04-17T10:43:47.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.333+0000 7f8a1a7fc640 1 -- 172.21.15.83:0/1408245776 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a14006d60 con 0x7f8a1c06ebf0 2024-04-17T10:43:47.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.333+0000 7f8a23d1b640 1 -- 172.21.15.83:0/1408245776 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a1c14a360 con 0x7f8a1c06ebf0 2024-04-17T10:43:47.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.333+0000 7f8a1a7fc640 1 -- 172.21.15.83:0/1408245776 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a14003bc0 con 0x7f8a1c06ebf0 2024-04-17T10:43:47.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.333+0000 7f8a23d1b640 1 -- 172.21.15.83:0/1408245776 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a1c14a920 con 0x7f8a1c06ebf0 2024-04-17T10:43:47.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.333+0000 7f8a23d1b640 1 -- 172.21.15.83:0/1408245776 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a1c075000 con 0x7f8a1c06ebf0 2024-04-17T10:43:47.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.333+0000 7f8a1a7fc640 1 -- 172.21.15.83:0/1408245776 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8a1400a850 con 0x7f8a1c06ebf0 2024-04-17T10:43:47.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.333+0000 7f8a1a7fc640 1 --2- 172.21.15.83:0/1408245776 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8a08066f60 0x7f8a08069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:43:47.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.333+0000 7f8a212b6640 1 --2- 172.21.15.83:0/1408245776 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8a08066f60 0x7f8a08069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:43:47.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.333+0000 7f8a1a7fc640 1 -- 172.21.15.83:0/1408245776 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a14085ad0 con 0x7f8a1c06ebf0 2024-04-17T10:43:47.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.333+0000 7f8a212b6640 1 --2- 172.21.15.83:0/1408245776 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8a08066f60 0x7f8a08069420 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8a100af000 tx=0x7f8a100a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:43:47.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.337+0000 7f8a1a7fc640 1 -- 172.21.15.83:0/1408245776 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8a14020020 con 0x7f8a1c06ebf0 2024-04-17T10:43:47.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:47.537+0000 7f8a23d1b640 1 -- 172.21.15.83:0/1408245776 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8a1c0afd10 con 0x7f8a1c06ebf0 2024-04-17T10:43:48.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:48.353+0000 7f8a1a7fc640 1 -- 172.21.15.83:0/1408245776 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v49) v1 ==== 167+0+0 (secure 0 0 0) 0x7f8a14023e40 con 0x7f8a1c06ebf0 2024-04-17T10:43:48.347 INFO:teuthology.orchestra.run.smithi083.stderr:enabled new snapshots 2024-04-17T10:43:48.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:48.357+0000 7f8a23d1b640 1 -- 172.21.15.83:0/1408245776 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8a08066f60 msgr2=0x7f8a08069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:48.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:48.357+0000 7f8a23d1b640 1 --2- 172.21.15.83:0/1408245776 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8a08066f60 0x7f8a08069420 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8a100af000 tx=0x7f8a100a7000 comp rx=0 tx=0).stop 2024-04-17T10:43:48.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:48.357+0000 7f8a23d1b640 1 -- 172.21.15.83:0/1408245776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a1c06ebf0 msgr2=0x7f8a1c147340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:43:48.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:48.357+0000 7f8a23d1b640 1 --2- 172.21.15.83:0/1408245776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a1c06ebf0 0x7f8a1c147340 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400c730 tx=0x7f8a1400cbf0 comp rx=0 tx=0).stop 2024-04-17T10:43:48.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:48.357+0000 7f8a23d1b640 1 -- 172.21.15.83:0/1408245776 shutdown_connections 2024-04-17T10:43:48.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:48.357+0000 7f8a23d1b640 1 --2- 172.21.15.83:0/1408245776 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8a08066f60 0x7f8a08069420 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:48.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:48.357+0000 7f8a23d1b640 1 --2- 172.21.15.83:0/1408245776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a1c15a890 0x7f8a1c151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:48.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:48.357+0000 7f8a23d1b640 1 --2- 172.21.15.83:0/1408245776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a1c071ac0 0x7f8a1c151030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:48.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:48.357+0000 7f8a23d1b640 1 --2- 172.21.15.83:0/1408245776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a1c06ebf0 0x7f8a1c147340 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:43:48.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:48.357+0000 7f8a23d1b640 1 -- 172.21.15.83:0/1408245776 >> 172.21.15.83:0/1408245776 conn(0x7f8a1c0b2410 msgr2=0x7f8a1c154e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:43:48.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:48.357+0000 7f8a23d1b640 1 -- 172.21.15.83:0/1408245776 shutdown_connections 2024-04-17T10:43:48.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:43:48.357+0000 7f8a23d1b640 1 -- 172.21.15.83:0/1408245776 wait complete. 2024-04-17T10:43:48.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:43:48.374 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stray_eval_dir) 2024-04-17T10:43:48.391 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:43:48.392 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:43:48.441 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-04-17T10:43:48.441 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-04-17T10:43:48.441 DEBUG:teuthology.orchestra.run.smithi083:> import uuid 2024-04-17T10:43:48.441 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:43:48.442 DEBUG:teuthology.orchestra.run.smithi083:> n = 2000 2024-04-17T10:43:48.442 DEBUG:teuthology.orchestra.run.smithi083:> create_hard_links = False 2024-04-17T10:43:48.442 DEBUG:teuthology.orchestra.run.smithi083:> if 3 > 0: 2024-04-17T10:43:48.442 DEBUG:teuthology.orchestra.run.smithi083:> create_hard_links = True 2024-04-17T10:43:48.442 DEBUG:teuthology.orchestra.run.smithi083:> path = "/home/ubuntu/cephtest/mnt.0/stray_eval_dir/file" 2024-04-17T10:43:48.442 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:43:48.442 DEBUG:teuthology.orchestra.run.smithi083:> dpath = os.path.dirname(path) 2024-04-17T10:43:48.442 DEBUG:teuthology.orchestra.run.smithi083:> fnameprefix = os.path.basename(path) 2024-04-17T10:43:48.442 DEBUG:teuthology.orchestra.run.smithi083:> os.makedirs(dpath, exist_ok=True) 2024-04-17T10:43:48.442 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:43:48.442 DEBUG:teuthology.orchestra.run.smithi083:> try: 2024-04-17T10:43:48.442 DEBUG:teuthology.orchestra.run.smithi083:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-17T10:43:48.442 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:43:48.442 DEBUG:teuthology.orchestra.run.smithi083:> for i in range(n): 2024-04-17T10:43:48.443 DEBUG:teuthology.orchestra.run.smithi083:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-17T10:43:48.443 DEBUG:teuthology.orchestra.run.smithi083:> with open(fpath, '"'"'w'"'"') as f: 2024-04-17T10:43:48.443 DEBUG:teuthology.orchestra.run.smithi083:> f.write(f"{i}") 2024-04-17T10:43:48.443 DEBUG:teuthology.orchestra.run.smithi083:> if False: 2024-04-17T10:43:48.443 DEBUG:teuthology.orchestra.run.smithi083:> f.flush() 2024-04-17T10:43:48.443 DEBUG:teuthology.orchestra.run.smithi083:> os.fsync(f.fileno()) 2024-04-17T10:43:48.443 DEBUG:teuthology.orchestra.run.smithi083:> if False: 2024-04-17T10:43:48.443 DEBUG:teuthology.orchestra.run.smithi083:> os.unlink(fpath) 2024-04-17T10:43:48.443 DEBUG:teuthology.orchestra.run.smithi083:> if False: 2024-04-17T10:43:48.443 DEBUG:teuthology.orchestra.run.smithi083:> os.fsync(dirfd) 2024-04-17T10:43:48.443 DEBUG:teuthology.orchestra.run.smithi083:> if create_hard_links: 2024-04-17T10:43:48.443 DEBUG:teuthology.orchestra.run.smithi083:> for j in range(3): 2024-04-17T10:43:48.443 DEBUG:teuthology.orchestra.run.smithi083:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-17T10:43:48.443 DEBUG:teuthology.orchestra.run.smithi083:> if False: 2024-04-17T10:43:48.443 DEBUG:teuthology.orchestra.run.smithi083:> os.fsync(dirfd) 2024-04-17T10:43:48.444 DEBUG:teuthology.orchestra.run.smithi083:> finally: 2024-04-17T10:43:48.444 DEBUG:teuthology.orchestra.run.smithi083:> os.close(dirfd) 2024-04-17T10:43:48.444 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-04-17T10:44:07.215 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:44:07.217 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:44:28.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:44:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:44:28.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:44:28.439 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-17T10:44:28.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:44:28.480 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec find /home/ubuntu/cephtest/mnt.0/stray_eval_dir/ -type f -delete) 2024-04-17T10:44:37.469 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:44:37.472 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:45:07.824 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:45:07.827 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:45:37.880 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:45:37.883 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:45:48.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:45:48.332 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-17T10:45:48.517 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-17T10:45:48.517 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 perf dump 2024-04-17T10:45:48.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.651+0000 7fd2449b8640 1 Processor -- start 2024-04-17T10:45:48.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.651+0000 7fd2449b8640 1 -- start start 2024-04-17T10:45:48.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.651+0000 7fd2449b8640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd24015bac0 0x7fd240071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.651+0000 7fd2449b8640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd24006c1f0 con 0x7fd240070470 2024-04-17T10:45:48.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.651+0000 7fd2449b8640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd240070a10 con 0x7fd24015b6e0 2024-04-17T10:45:48.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.651+0000 7fd2449b8640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd240070cf0 con 0x7fd24015bac0 2024-04-17T10:45:48.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.651+0000 7fd23ed9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd240070470 0x7fd24006ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50904/0 (socket says 172.21.15.83:50904) 2024-04-17T10:45:48.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.651+0000 7fd23ed9d640 1 -- 172.21.15.83:0/2581489219 learned_addr learned my addr 172.21.15.83:0/2581489219 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:45:48.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.655+0000 7fd23dd9b640 1 --2- 172.21.15.83:0/2581489219 >> v2:172.21.15.167:3300/0 conn(0x7fd24015bac0 0x7fd240071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.655+0000 7fd23d59a640 1 -- 172.21.15.83:0/2581489219 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3750498966 0 0) 0x7fd24006c1f0 con 0x7fd240070470 2024-04-17T10:45:48.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.655+0000 7fd23d59a640 1 -- 172.21.15.83:0/2581489219 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd228003530 con 0x7fd240070470 2024-04-17T10:45:48.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.655+0000 7fd23d59a640 1 -- 172.21.15.83:0/2581489219 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2414912438 0 0) 0x7fd228003530 con 0x7fd240070470 2024-04-17T10:45:48.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.655+0000 7fd23d59a640 1 -- 172.21.15.83:0/2581489219 >> v2:172.21.15.167:3300/0 conn(0x7fd24015bac0 msgr2=0x7fd240071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:45:48.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.655+0000 7fd23d59a640 1 --2- 172.21.15.83:0/2581489219 >> v2:172.21.15.167:3300/0 conn(0x7fd24015bac0 0x7fd240071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.655+0000 7fd23d59a640 1 -- 172.21.15.83:0/2581489219 >> v1:172.21.15.167:6789/0 conn(0x7fd24015b6e0 legacy=0x7fd2400cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:45:48.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.655+0000 7fd23d59a640 1 -- 172.21.15.83:0/2581489219 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd240070fd0 con 0x7fd240070470 2024-04-17T10:45:48.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.655+0000 7fd23d59a640 1 -- 172.21.15.83:0/2581489219 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fd238001a20 con 0x7fd240070470 2024-04-17T10:45:48.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.655+0000 7fd23d59a640 1 -- 172.21.15.83:0/2581489219 >> v1:172.21.15.83:6789/0 conn(0x7fd240070470 legacy=0x7fd24006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:45:48.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.655+0000 7fd23d59a640 1 --2- 172.21.15.83:0/2581489219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd228003a30 0x7fd228003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.655+0000 7fd23d59a640 1 --2- 172.21.15.83:0/2581489219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd228004810 0x7fd228004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.655+0000 7fd23d59a640 1 --2- 172.21.15.83:0/2581489219 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2280051d0 0x7fd22800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.655+0000 7fd23d59a640 1 -- 172.21.15.83:0/2581489219 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd240070cf0 con 0x7fd228004810 2024-04-17T10:45:48.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.655+0000 7fd23d59a640 1 -- 172.21.15.83:0/2581489219 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22800c100 con 0x7fd228003a30 2024-04-17T10:45:48.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.655+0000 7fd23d59a640 1 -- 172.21.15.83:0/2581489219 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd22800c300 con 0x7fd2280051d0 2024-04-17T10:45:48.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.655+0000 7fd23dd9b640 1 --2- 172.21.15.83:0/2581489219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd228004810 0x7fd228004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.655+0000 7fd23dd9b640 1 -- 172.21.15.83:0/2581489219 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2280051d0 msgr2=0x7fd22800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:45:48.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.655+0000 7fd23dd9b640 1 --2- 172.21.15.83:0/2581489219 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2280051d0 0x7fd22800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.655+0000 7fd23dd9b640 1 -- 172.21.15.83:0/2581489219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd228003a30 msgr2=0x7fd228003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:45:48.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.655+0000 7fd23dd9b640 1 --2- 172.21.15.83:0/2581489219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd228003a30 0x7fd228003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.655+0000 7fd23ed9d640 1 -- 172.21.15.83:0/2581489219 reap_dead start 2024-04-17T10:45:48.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.655+0000 7fd23dd9b640 1 -- 172.21.15.83:0/2581489219 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd228003530 con 0x7fd228004810 2024-04-17T10:45:48.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.655+0000 7fd23dd9b640 1 --2- 172.21.15.83:0/2581489219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd228004810 0x7fd228004c90 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fd234009ae0 tx=0x7fd23401ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f0a3ebf174f39a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:45:48.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.655+0000 7fd23d59a640 1 -- 172.21.15.83:0/2581489219 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2340b2070 con 0x7fd228004810 2024-04-17T10:45:48.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.655+0000 7fd23d59a640 1 -- 172.21.15.83:0/2581489219 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2340a5680 con 0x7fd228004810 2024-04-17T10:45:48.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.655+0000 7fd23d59a640 1 -- 172.21.15.83:0/2581489219 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2340096b0 con 0x7fd228004810 2024-04-17T10:45:48.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.659+0000 7fd2449b8640 1 -- 172.21.15.83:0/2581489219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd228004810 msgr2=0x7fd228004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:45:48.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.659+0000 7fd2449b8640 1 --2- 172.21.15.83:0/2581489219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd228004810 0x7fd228004c90 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fd234009ae0 tx=0x7fd23401ca90 comp rx=0 tx=0).stop 2024-04-17T10:45:48.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.659+0000 7fd2449b8640 1 -- 172.21.15.83:0/2581489219 shutdown_connections 2024-04-17T10:45:48.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.659+0000 7fd2449b8640 1 --2- 172.21.15.83:0/2581489219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd228004810 0x7fd228004c90 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.659+0000 7fd2449b8640 1 -- 172.21.15.83:0/2581489219 >> 172.21.15.83:0/2581489219 conn(0x7fd2400b15b0 msgr2=0x7fd2400b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:45:48.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.659+0000 7fd2449b8640 1 -- 172.21.15.83:0/2581489219 shutdown_connections 2024-04-17T10:45:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.659+0000 7fd2449b8640 1 -- 172.21.15.83:0/2581489219 wait complete. 2024-04-17T10:45:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.659+0000 7fd2449b8640 1 Processor -- start 2024-04-17T10:45:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.659+0000 7fd2449b8640 1 -- start start 2024-04-17T10:45:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.659+0000 7fd2449b8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd228004810 0x7fd240141f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.659+0000 7fd2449b8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd24014c350 0x7fd240150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.659+0000 7fd2449b8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2401529c0 0x7fd240152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.659+0000 7fd2449b8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd240071e40 con 0x7fd24014c350 2024-04-17T10:45:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.659+0000 7fd2449b8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd240071cc0 con 0x7fd228004810 2024-04-17T10:45:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.659+0000 7fd2449b8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd240071fc0 con 0x7fd2401529c0 2024-04-17T10:45:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.659+0000 7fd23dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd24014c350 0x7fd240150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.659+0000 7fd23dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd24014c350 0x7fd240150e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56730/0 (socket says 172.21.15.83:56730) 2024-04-17T10:45:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.659+0000 7fd23dd9b640 1 -- 172.21.15.83:0/3188711185 learned_addr learned my addr 172.21.15.83:0/3188711185 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:45:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.659+0000 7fd23e59c640 1 --2- 172.21.15.83:0/3188711185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd228004810 0x7fd240141f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.659+0000 7fd23ed9d640 1 --2- 172.21.15.83:0/3188711185 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2401529c0 0x7fd240152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.659+0000 7fd23dd9b640 1 -- 172.21.15.83:0/3188711185 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2401529c0 msgr2=0x7fd240152e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:45:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.659+0000 7fd23dd9b640 1 --2- 172.21.15.83:0/3188711185 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2401529c0 0x7fd240152e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.659+0000 7fd23dd9b640 1 -- 172.21.15.83:0/3188711185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd228004810 msgr2=0x7fd240141f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:45:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.659+0000 7fd23dd9b640 1 --2- 172.21.15.83:0/3188711185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd228004810 0x7fd240141f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.659+0000 7fd23dd9b640 1 -- 172.21.15.83:0/3188711185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2401f1a70 con 0x7fd24014c350 2024-04-17T10:45:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.659+0000 7fd23e59c640 1 --2- 172.21.15.83:0/3188711185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd228004810 0x7fd240141f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:45:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.659+0000 7fd23dd9b640 1 --2- 172.21.15.83:0/3188711185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd24014c350 0x7fd240150e60 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fd23401c8c0 tx=0x7fd234019830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:45:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.659+0000 7fd22effd640 1 -- 172.21.15.83:0/3188711185 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2340b2040 con 0x7fd24014c350 2024-04-17T10:45:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.659+0000 7fd22effd640 1 -- 172.21.15.83:0/3188711185 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23400f000 con 0x7fd24014c350 2024-04-17T10:45:48.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.659+0000 7fd2449b8640 1 -- 172.21.15.83:0/3188711185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd24014dd30 con 0x7fd24014c350 2024-04-17T10:45:48.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.659+0000 7fd22effd640 1 -- 172.21.15.83:0/3188711185 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2340096b0 con 0x7fd24014c350 2024-04-17T10:45:48.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.659+0000 7fd2449b8640 1 -- 172.21.15.83:0/3188711185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd24014e2f0 con 0x7fd24014c350 2024-04-17T10:45:48.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.663+0000 7fd22effd640 1 -- 172.21.15.83:0/3188711185 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd2340bf030 con 0x7fd24014c350 2024-04-17T10:45:48.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.663+0000 7fd22effd640 1 --2- 172.21.15.83:0/3188711185 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd220066f60 0x7fd220069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.663+0000 7fd22effd640 1 -- 172.21.15.83:0/3188711185 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd23411ed50 con 0x7fd24014c350 2024-04-17T10:45:48.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.663+0000 7fd23e59c640 1 --2- 172.21.15.83:0/3188711185 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd220066f60 0x7fd220069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.663+0000 7fd23e59c640 1 --2- 172.21.15.83:0/3188711185 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd220066f60 0x7fd220069420 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd24006a390 tx=0x7fd230009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:45:48.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.663+0000 7fd247b25000 1 Processor -- start 2024-04-17T10:45:48.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.663+0000 7fd247b25000 1 -- start start 2024-04-17T10:45:48.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.663+0000 7fd247b25000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588988a3a40 0x5588988a3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.663+0000 7fd247b25000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5588988a4360 0x55889885e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.663+0000 7fd247b25000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55889885ed90 0x55889885f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.663+0000 7fd247b25000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588987411f0 con 0x5588988a3a40 2024-04-17T10:45:48.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.663+0000 7fd247b25000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588986a7e90 con 0x55889885ed90 2024-04-17T10:45:48.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.663+0000 7fd247b25000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55889862ff50 con 0x5588988a4360 2024-04-17T10:45:48.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.663+0000 7fd23e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588988a3a40 0x5588988a3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.663+0000 7fd23e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588988a3a40 0x5588988a3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56738/0 (socket says 172.21.15.83:56738) 2024-04-17T10:45:48.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.663+0000 7fd23e59c640 1 -- 172.21.15.83:0/3450201820 learned_addr learned my addr 172.21.15.83:0/3450201820 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:45:48.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.663+0000 7fd23dd9b640 1 --2- 172.21.15.83:0/3450201820 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5588988a4360 0x55889885e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.663+0000 7fd23ed9d640 1 --2- 172.21.15.83:0/3450201820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55889885ed90 0x55889885f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.663+0000 7fd23e59c640 1 -- 172.21.15.83:0/3450201820 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5588988a4360 msgr2=0x55889885e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:45:48.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.663+0000 7fd23e59c640 1 --2- 172.21.15.83:0/3450201820 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5588988a4360 0x55889885e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.663+0000 7fd23e59c640 1 -- 172.21.15.83:0/3450201820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55889885ed90 msgr2=0x55889885f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:45:48.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.663+0000 7fd23e59c640 1 --2- 172.21.15.83:0/3450201820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55889885ed90 0x55889885f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.663+0000 7fd23e59c640 1 -- 172.21.15.83:0/3450201820 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55889867f8b0 con 0x5588988a3a40 2024-04-17T10:45:48.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.663+0000 7fd23e59c640 1 --2- 172.21.15.83:0/3450201820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588988a3a40 0x5588988a3e20 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fd230005120 tx=0x7fd230011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bced7e382458917 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:45:48.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.663+0000 7fd21ffff640 1 -- 172.21.15.83:0/3450201820 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd230003600 con 0x5588988a3a40 2024-04-17T10:45:48.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.663+0000 7fd21ffff640 1 -- 172.21.15.83:0/3450201820 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2300037a0 con 0x5588988a3a40 2024-04-17T10:45:48.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.663+0000 7fd21ffff640 1 -- 172.21.15.83:0/3450201820 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd230003b50 con 0x5588988a3a40 2024-04-17T10:45:48.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd247b25000 1 -- 172.21.15.83:0/3450201820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588988a3a40 msgr2=0x5588988a3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:45:48.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd247b25000 1 --2- 172.21.15.83:0/3450201820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588988a3a40 0x5588988a3e20 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fd230005120 tx=0x7fd230011000 comp rx=0 tx=0).stop 2024-04-17T10:45:48.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd247b25000 1 -- 172.21.15.83:0/3450201820 shutdown_connections 2024-04-17T10:45:48.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd247b25000 1 --2- 172.21.15.83:0/3450201820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55889885ed90 0x55889885f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd247b25000 1 --2- 172.21.15.83:0/3450201820 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5588988a4360 0x55889885e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd247b25000 1 --2- 172.21.15.83:0/3450201820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588988a3a40 0x5588988a3e20 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd247b25000 1 -- 172.21.15.83:0/3450201820 >> 172.21.15.83:0/3450201820 conn(0x55889885ad40 msgr2=0x55889885d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:45:48.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd247b25000 1 -- 172.21.15.83:0/3450201820 shutdown_connections 2024-04-17T10:45:48.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd247b25000 1 -- 172.21.15.83:0/3450201820 wait complete. 2024-04-17T10:45:48.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd247b25000 1 -- start start 2024-04-17T10:45:48.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd247b25000 1 Processor -- start 2024-04-17T10:45:48.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd247b25000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889885ed90 0x5588988ab1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd247b25000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5588988a3a40 0x5588988ab700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd247b25000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588988a4360 0x558898864de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd247b25000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x558898662540 con 0x55889885ed90 2024-04-17T10:45:48.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd247b25000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588986a3f30 con 0x5588988a4360 2024-04-17T10:45:48.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd247b25000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588986747c0 con 0x5588988a3a40 2024-04-17T10:45:48.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd247b25000 10 client.0 _ll_register_callbacks cb 0x55889875c180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:45:48.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd23e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889885ed90 0x5588988ab1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd23e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889885ed90 0x5588988ab1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56746/0 (socket says 172.21.15.83:56746) 2024-04-17T10:45:48.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd23e59c640 1 -- 172.21.15.83:0/3804752986 learned_addr learned my addr 172.21.15.83:0/3804752986 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:45:48.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd23ed9d640 1 --2- 172.21.15.83:0/3804752986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588988a4360 0x558898864de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd23dd9b640 1 --2- 172.21.15.83:0/3804752986 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5588988a3a40 0x5588988ab700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd23e59c640 1 -- 172.21.15.83:0/3804752986 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5588988a3a40 msgr2=0x5588988ab700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:45:48.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd23e59c640 1 --2- 172.21.15.83:0/3804752986 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5588988a3a40 0x5588988ab700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd23e59c640 1 -- 172.21.15.83:0/3804752986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588988a4360 msgr2=0x558898864de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:45:48.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd23e59c640 1 --2- 172.21.15.83:0/3804752986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588988a4360 0x558898864de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd23e59c640 1 -- 172.21.15.83:0/3804752986 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588986301a0 con 0x55889885ed90 2024-04-17T10:45:48.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd23ed9d640 1 --2- 172.21.15.83:0/3804752986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588988a4360 0x558898864de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:45:48.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd23dd9b640 1 --2- 172.21.15.83:0/3804752986 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5588988a3a40 0x5588988ab700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:45:48.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd21d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:45:48.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd23e59c640 1 --2- 172.21.15.83:0/3804752986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889885ed90 0x5588988ab1c0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fd23000ead0 tx=0x7fd230013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6abf2db8afaa5692 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:45:48.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd21d7fa640 1 -- 172.21.15.83:0/3804752986 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd230021450 con 0x55889885ed90 2024-04-17T10:45:48.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd21d7fa640 1 -- 172.21.15.83:0/3804752986 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(76..76 src has 1..76) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd230010050 con 0x55889885ed90 2024-04-17T10:45:48.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd247b25000 1 -- 172.21.15.83:0/3804752986 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2300037a0 con 0x55889885ed90 2024-04-17T10:45:48.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd21d7fa640 1 -- 172.21.15.83:0/3804752986 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd230031050 con 0x55889885ed90 2024-04-17T10:45:48.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd247b25000 10 client.7299 fetch_fsmap learned FSMap version 49 2024-04-17T10:45:48.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd247b25000 1 -- 172.21.15.83:0/3804752986 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x55889872a840 con 0x55889885ed90 2024-04-17T10:45:48.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd21d7fa640 1 -- 172.21.15.83:0/3804752986 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2300037a0 con 0x55889885ed90 2024-04-17T10:45:48.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd21d7fa640 1 -- 172.21.15.83:0/3804752986 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd230034020 con 0x55889885ed90 2024-04-17T10:45:48.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd247b25000 10 client.7299 fetch_fsmap finished waiting for FSMap version 49 2024-04-17T10:45:48.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd247b25000 10 client.7299 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-17T10:45:48.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd247b25000 20 client.7299 populate_metadata read hostname 'smithi083' 2024-04-17T10:45:48.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd247b25000 1 --2- 172.21.15.83:0/3804752986 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x558898865b80 0x558898865f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd247b25000 4 client.7299 mds_command: new command op to 7104 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:45:48.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd247b25000 1 -- 172.21.15.83:0/3804752986 --> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558898643be0 con 0x558898865b80 2024-04-17T10:45:48.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd23dd9b640 1 --2- 172.21.15.83:0/3804752986 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x558898865b80 0x558898865f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd21d7fa640 10 client.7299 ms_handle_connect on v2:172.21.15.167:6832/2796648400 2024-04-17T10:45:48.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.667+0000 7fd23dd9b640 1 --2- 172.21.15.83:0/3804752986 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x558898865b80 0x558898865f60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fee67d71a0d5708e server_cookie=bb552222371652d2 in_seq=0 out_seq=0 2024-04-17T10:45:48.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.671+0000 7fd21d7fa640 1 -- 172.21.15.83:0/3804752986 <== mds.0 v2:172.21.15.167:6832/2796648400 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558898643be0 con 0x558898865b80 2024-04-17T10:45:48.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.671+0000 7fd21d7fa640 10 client.7299 handle_command_reply: tid=0 2024-04-17T10:45:48.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.671+0000 7fd247b25000 1 client.7299 shutdown 2024-04-17T10:45:48.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.671+0000 7fd247b25000 1 -- 172.21.15.83:0/3804752986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889885ed90 msgr2=0x5588988ab1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:45:48.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.671+0000 7fd247b25000 1 --2- 172.21.15.83:0/3804752986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889885ed90 0x5588988ab1c0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fd23000ead0 tx=0x7fd230013000 comp rx=0 tx=0).stop 2024-04-17T10:45:48.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.671+0000 7fd247b25000 1 -- 172.21.15.83:0/3804752986 shutdown_connections 2024-04-17T10:45:48.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.671+0000 7fd247b25000 1 --2- 172.21.15.83:0/3804752986 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x558898865b80 0x558898865f60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.671+0000 7fd247b25000 1 --2- 172.21.15.83:0/3804752986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588988a4360 0x558898864de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.671+0000 7fd247b25000 1 --2- 172.21.15.83:0/3804752986 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5588988a3a40 0x5588988ab700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.671+0000 7fd21d7fa640 0 client.7299 ms_handle_reset on v2:172.21.15.167:6832/2796648400 2024-04-17T10:45:48.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.671+0000 7fd247b25000 1 --2- 172.21.15.83:0/3804752986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889885ed90 0x5588988ab1c0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.671+0000 7fd247b25000 1 -- 172.21.15.83:0/3804752986 >> 172.21.15.83:0/3804752986 conn(0x55889885ad40 msgr2=0x55889885d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:45:48.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.671+0000 7fd247b25000 1 -- 172.21.15.83:0/3804752986 shutdown_connections 2024-04-17T10:45:48.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.671+0000 7fd247b25000 1 -- 172.21.15.83:0/3804752986 wait complete. 2024-04-17T10:45:48.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.671+0000 7fd247b25000 20 client.7299 trim_cache size 0 max 16384 2024-04-17T10:45:48.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.683+0000 7fd247b25000 1 Processor -- start 2024-04-17T10:45:48.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.683+0000 7fd247b25000 1 -- start start 2024-04-17T10:45:48.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd247b25000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55889885ed90 0x558898908aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd247b25000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588988a3a40 0x558898908fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd247b25000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588988a4360 0x558898909520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd247b25000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2401fe650 con 0x5588988a3a40 2024-04-17T10:45:48.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd247b25000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2401fd800 con 0x5588988a4360 2024-04-17T10:45:48.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd247b25000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd24014fb40 con 0x55889885ed90 2024-04-17T10:45:48.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd23dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588988a3a40 0x558898908fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd23dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588988a3a40 0x558898908fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56754/0 (socket says 172.21.15.83:56754) 2024-04-17T10:45:48.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd23dd9b640 1 -- 172.21.15.83:0/1175502922 learned_addr learned my addr 172.21.15.83:0/1175502922 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:45:48.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd23e59c640 1 --2- 172.21.15.83:0/1175502922 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55889885ed90 0x558898908aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd23dd9b640 1 -- 172.21.15.83:0/1175502922 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55889885ed90 msgr2=0x558898908aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:45:48.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd23dd9b640 1 --2- 172.21.15.83:0/1175502922 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55889885ed90 0x558898908aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd23dd9b640 1 -- 172.21.15.83:0/1175502922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588988a4360 msgr2=0x558898909520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:45:48.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd23ed9d640 1 --2- 172.21.15.83:0/1175502922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588988a4360 0x558898909520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd23dd9b640 1 --2- 172.21.15.83:0/1175502922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588988a4360 0x558898909520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd23dd9b640 1 -- 172.21.15.83:0/1175502922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558898864d20 con 0x5588988a3a40 2024-04-17T10:45:48.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd23ed9d640 1 --2- 172.21.15.83:0/1175502922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588988a4360 0x558898909520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:45:48.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd23dd9b640 1 --2- 172.21.15.83:0/1175502922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588988a3a40 0x558898908fe0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fd23401d000 tx=0x7fd23411d1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0988372f75d4f1c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:45:48.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd21f7fe640 1 -- 172.21.15.83:0/1175502922 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2340f7dc0 con 0x5588988a3a40 2024-04-17T10:45:48.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd21f7fe640 1 -- 172.21.15.83:0/1175502922 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2340ed750 con 0x5588988a3a40 2024-04-17T10:45:48.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd21f7fe640 1 -- 172.21.15.83:0/1175502922 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd234107100 con 0x5588988a3a40 2024-04-17T10:45:48.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd247b25000 1 -- 172.21.15.83:0/1175502922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588988a3a40 msgr2=0x558898908fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:45:48.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd247b25000 1 --2- 172.21.15.83:0/1175502922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588988a3a40 0x558898908fe0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fd23401d000 tx=0x7fd23411d1a0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd247b25000 1 -- 172.21.15.83:0/1175502922 shutdown_connections 2024-04-17T10:45:48.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd247b25000 1 --2- 172.21.15.83:0/1175502922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588988a4360 0x558898909520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd247b25000 1 --2- 172.21.15.83:0/1175502922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588988a3a40 0x558898908fe0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd247b25000 1 --2- 172.21.15.83:0/1175502922 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55889885ed90 0x558898908aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd247b25000 1 -- 172.21.15.83:0/1175502922 >> 172.21.15.83:0/1175502922 conn(0x55889885ad40 msgr2=0x5588988a2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:45:48.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd247b25000 1 -- 172.21.15.83:0/1175502922 shutdown_connections 2024-04-17T10:45:48.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd247b25000 1 -- 172.21.15.83:0/1175502922 wait complete. 2024-04-17T10:45:48.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd247b25000 1 -- start start 2024-04-17T10:45:48.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd247b25000 1 Processor -- start 2024-04-17T10:45:48.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd247b25000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889885ed90 0x55889890d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd247b25000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5588988a3a40 0x55889890d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd247b25000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588988a4360 0x5588988a8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd247b25000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55889864ec00 con 0x55889885ed90 2024-04-17T10:45:48.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd247b25000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2401fd230 con 0x5588988a4360 2024-04-17T10:45:48.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd247b25000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588986a3f30 con 0x5588988a3a40 2024-04-17T10:45:48.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd247b25000 10 client.0 _ll_register_callbacks cb 0x55889866aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:45:48.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd23e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889885ed90 0x55889890d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd23e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889885ed90 0x55889890d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56758/0 (socket says 172.21.15.83:56758) 2024-04-17T10:45:48.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd23e59c640 1 -- 172.21.15.83:0/1257096305 learned_addr learned my addr 172.21.15.83:0/1257096305 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:45:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd23dd9b640 1 --2- 172.21.15.83:0/1257096305 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5588988a3a40 0x55889890d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd23e59c640 1 -- 172.21.15.83:0/1257096305 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5588988a3a40 msgr2=0x55889890d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:45:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd23ed9d640 1 --2- 172.21.15.83:0/1257096305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588988a4360 0x5588988a8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd23e59c640 1 --2- 172.21.15.83:0/1257096305 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5588988a3a40 0x55889890d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd23e59c640 1 -- 172.21.15.83:0/1257096305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588988a4360 msgr2=0x5588988a8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:45:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd23e59c640 1 --2- 172.21.15.83:0/1257096305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588988a4360 0x5588988a8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd23e59c640 1 -- 172.21.15.83:0/1257096305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558898701a10 con 0x55889885ed90 2024-04-17T10:45:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd23e59c640 1 --2- 172.21.15.83:0/1257096305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889885ed90 0x55889890d0e0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fd2300149f0 tx=0x7fd230001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100e81831077cafd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:45:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd21d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:45:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.687+0000 7fd21d7fa640 1 -- 172.21.15.83:0/1257096305 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd230025020 con 0x55889885ed90 2024-04-17T10:45:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd21d7fa640 1 -- 172.21.15.83:0/1257096305 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(76..76 src has 1..76) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd230029040 con 0x55889885ed90 2024-04-17T10:45:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd247b25000 1 -- 172.21.15.83:0/1257096305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2340ed750 con 0x55889885ed90 2024-04-17T10:45:48.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd21d7fa640 1 -- 172.21.15.83:0/1257096305 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd230003b50 con 0x55889885ed90 2024-04-17T10:45:48.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd247b25000 10 client.7305 fetch_fsmap learned FSMap version 49 2024-04-17T10:45:48.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd247b25000 1 -- 172.21.15.83:0/1257096305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x55889863cb00 con 0x55889885ed90 2024-04-17T10:45:48.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd21d7fa640 1 -- 172.21.15.83:0/1257096305 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2340ed750 con 0x55889885ed90 2024-04-17T10:45:48.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd21d7fa640 1 -- 172.21.15.83:0/1257096305 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd230034040 con 0x55889885ed90 2024-04-17T10:45:48.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd247b25000 10 client.7305 fetch_fsmap finished waiting for FSMap version 49 2024-04-17T10:45:48.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd247b25000 10 client.7305 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-17T10:45:48.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd247b25000 20 client.7305 populate_metadata read hostname 'smithi083' 2024-04-17T10:45:48.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd247b25000 1 --2- 172.21.15.83:0/1257096305 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x5588988a9890 0x5588988a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd247b25000 4 client.7305 mds_command: new command op to 7104 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-17T10:45:48.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd247b25000 1 -- 172.21.15.83:0/1257096305 --> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x5588986b53d0 con 0x5588988a9890 2024-04-17T10:45:48.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd23dd9b640 1 --2- 172.21.15.83:0/1257096305 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x5588988a9890 0x5588988a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd21d7fa640 10 client.7305 ms_handle_connect on v2:172.21.15.167:6832/2796648400 2024-04-17T10:45:48.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd23dd9b640 1 --2- 172.21.15.83:0/1257096305 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x5588988a9890 0x5588988a9c70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e18bca71bb101ea server_cookie=e52dfae846b2fc47 in_seq=0 out_seq=0 2024-04-17T10:45:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd21d7fa640 1 -- 172.21.15.83:0/1257096305 <== mds.0 v2:172.21.15.167:6832/2796648400 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17932 (crc 0 0 0) 0x5588986b53d0 con 0x5588988a9890 2024-04-17T10:45:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd21d7fa640 10 client.7305 handle_command_reply: tid=0 2024-04-17T10:45:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd247b25000 1 client.7305 shutdown 2024-04-17T10:45:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd247b25000 1 -- 172.21.15.83:0/1257096305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889885ed90 msgr2=0x55889890d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:45:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd247b25000 1 --2- 172.21.15.83:0/1257096305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889885ed90 0x55889890d0e0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fd2300149f0 tx=0x7fd230001130 comp rx=0 tx=0).stop 2024-04-17T10:45:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd247b25000 1 -- 172.21.15.83:0/1257096305 shutdown_connections 2024-04-17T10:45:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd247b25000 1 --2- 172.21.15.83:0/1257096305 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x5588988a9890 0x5588988a9c70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd247b25000 1 --2- 172.21.15.83:0/1257096305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588988a4360 0x5588988a8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd247b25000 1 --2- 172.21.15.83:0/1257096305 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5588988a3a40 0x55889890d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd247b25000 1 --2- 172.21.15.83:0/1257096305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889885ed90 0x55889890d0e0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd247b25000 1 -- 172.21.15.83:0/1257096305 >> 172.21.15.83:0/1257096305 conn(0x55889885ad40 msgr2=0x5588988a29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:45:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd21d7fa640 0 client.7305 ms_handle_reset on v2:172.21.15.167:6832/2796648400 2024-04-17T10:45:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd247b25000 1 -- 172.21.15.83:0/1257096305 shutdown_connections 2024-04-17T10:45:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd247b25000 1 -- 172.21.15.83:0/1257096305 wait complete. 2024-04-17T10:45:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd247b25000 20 client.7305 trim_cache size 0 max 16384 2024-04-17T10:45:48.697 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:45:48.697 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T10:45:48.697 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 3322, 2024-04-17T10:45:48.697 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 3323, 2024-04-17T10:45:48.697 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1026872, 2024-04-17T10:45:48.697 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 33685194, 2024-04-17T10:45:48.697 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 9, 2024-04-17T10:45:48.697 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 3, 2024-04-17T10:45:48.697 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.591383313, 2024-04-17T10:45:48.698 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.265573969, 2024-04-17T10:45:48.698 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.111621819, 2024-04-17T10:45:48.698 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.173514716, 2024-04-17T10:45:48.698 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T10:45:48.698 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3322, 2024-04-17T10:45:48.698 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.459395786, 2024-04-17T10:45:48.698 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000138288 2024-04-17T10:45:48.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.698 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-17T10:45:48.698 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2, 2024-04-17T10:45:48.698 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000389, 2024-04-17T10:45:48.698 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000194 2024-04-17T10:45:48.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.699 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-17T10:45:48.699 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 0 2024-04-17T10:45:48.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.699 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T10:45:48.699 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 35748, 2024-04-17T10:45:48.699 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 62180, 2024-04-17T10:45:48.699 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 12648026, 2024-04-17T10:45:48.699 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 59997974, 2024-04-17T10:45:48.699 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 7, 2024-04-17T10:45:48.699 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 4, 2024-04-17T10:45:48.699 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 6.901844928, 2024-04-17T10:45:48.700 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 3.321932350, 2024-04-17T10:45:48.700 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 3.930538358, 2024-04-17T10:45:48.700 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.188472885, 2024-04-17T10:45:48.700 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T10:45:48.700 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 62180, 2024-04-17T10:45:48.700 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 3.346430869, 2024-04-17T10:45:48.700 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000053818 2024-04-17T10:45:48.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.700 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-17T10:45:48.700 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 88189, 2024-04-17T10:45:48.700 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.034275046, 2024-04-17T10:45:48.700 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000388 2024-04-17T10:45:48.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.701 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 121408, 2024-04-17T10:45:48.701 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 31136 2024-04-17T10:45:48.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.701 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T10:45:48.701 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 2875, 2024-04-17T10:45:48.701 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 2904, 2024-04-17T10:45:48.701 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 713423, 2024-04-17T10:45:48.701 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 21207706, 2024-04-17T10:45:48.701 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 6, 2024-04-17T10:45:48.701 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 4, 2024-04-17T10:45:48.702 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.487395313, 2024-04-17T10:45:48.702 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.215350832, 2024-04-17T10:45:48.702 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.095951879, 2024-04-17T10:45:48.702 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.142633761, 2024-04-17T10:45:48.702 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T10:45:48.702 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2903, 2024-04-17T10:45:48.702 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.527414424, 2024-04-17T10:45:48.702 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000181679 2024-04-17T10:45:48.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.702 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-17T10:45:48.702 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:45:48.702 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:45:48.703 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:45:48.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.703 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-17T10:45:48.703 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 135200 2024-04-17T10:45:48.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.703 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-04-17T10:45:48.703 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 1, 2024-04-17T10:45:48.703 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-04-17T10:45:48.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.703 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-MDSRank": { 2024-04-17T10:45:48.703 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-04-17T10:45:48.703 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-04-17T10:45:48.704 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 10204, 2024-04-17T10:45:48.704 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 35.512306091, 2024-04-17T10:45:48.704 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.003480233 2024-04-17T10:45:48.704 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:45:48.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.704 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-PurgeQueue": { 2024-04-17T10:45:48.704 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-04-17T10:45:48.704 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-04-17T10:45:48.704 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1918, 2024-04-17T10:45:48.704 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 1.815895789, 2024-04-17T10:45:48.704 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000946765 2024-04-17T10:45:48.704 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:45:48.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.705 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": { 2024-04-17T10:45:48.705 INFO:teuthology.orchestra.run.smithi083.stdout: "request": 27238, 2024-04-17T10:45:48.705 INFO:teuthology.orchestra.run.smithi083.stdout: "reply": 27238, 2024-04-17T10:45:48.705 INFO:teuthology.orchestra.run.smithi083.stdout: "reply_latency": { 2024-04-17T10:45:48.705 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 27238, 2024-04-17T10:45:48.705 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 506.689763276, 2024-04-17T10:45:48.705 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.018602311 2024-04-17T10:45:48.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.705 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_reply": 0, 2024-04-17T10:45:48.705 INFO:teuthology.orchestra.run.smithi083.stdout: "forward": 0, 2024-04-17T10:45:48.705 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_complete": 0, 2024-04-17T10:45:48.706 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_keys": 0, 2024-04-17T10:45:48.706 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_commit": 12, 2024-04-17T10:45:48.706 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_split": 0, 2024-04-17T10:45:48.706 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_merge": 0, 2024-04-17T10:45:48.706 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes": 19867, 2024-04-17T10:45:48.706 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_top": 0, 2024-04-17T10:45:48.706 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_bottom": 19867, 2024-04-17T10:45:48.706 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pin_tail": 0, 2024-04-17T10:45:48.706 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pinned": 19867, 2024-04-17T10:45:48.706 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_expired": 0, 2024-04-17T10:45:48.706 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_with_caps": 202, 2024-04-17T10:45:48.706 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": 202, 2024-04-17T10:45:48.707 INFO:teuthology.orchestra.run.smithi083.stdout: "subtrees": 2, 2024-04-17T10:45:48.707 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse": 45319, 2024-04-17T10:45:48.707 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_hit": 44824, 2024-04-17T10:45:48.707 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_forward": 0, 2024-04-17T10:45:48.707 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_discover": 0, 2024-04-17T10:45:48.707 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_dir_fetch": 0, 2024-04-17T10:45:48.707 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_remote_ino": 0, 2024-04-17T10:45:48.707 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_lock": 156, 2024-04-17T10:45:48.707 INFO:teuthology.orchestra.run.smithi083.stdout: "load_cent": 0, 2024-04-17T10:45:48.707 INFO:teuthology.orchestra.run.smithi083.stdout: "q": 0, 2024-04-17T10:45:48.707 INFO:teuthology.orchestra.run.smithi083.stdout: "exported": 0, 2024-04-17T10:45:48.707 INFO:teuthology.orchestra.run.smithi083.stdout: "exported_inodes": 0, 2024-04-17T10:45:48.708 INFO:teuthology.orchestra.run.smithi083.stdout: "imported": 0, 2024-04-17T10:45:48.708 INFO:teuthology.orchestra.run.smithi083.stdout: "imported_inodes": 0, 2024-04-17T10:45:48.708 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_dir_fetch": 0, 2024-04-17T10:45:48.708 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_backtrace_fetch": 0, 2024-04-17T10:45:48.708 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_peer_discover": 0, 2024-04-17T10:45:48.708 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rfiles": 0, 2024-04-17T10:45:48.708 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rbytes": 0, 2024-04-17T10:45:48.708 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rsnaps": 0, 2024-04-17T10:45:48.708 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_fetch": 0, 2024-04-17T10:45:48.708 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_set_tag": 0, 2024-04-17T10:45:48.708 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_repaired": 0, 2024-04-17T10:45:48.708 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_inotable_repaired": 0, 2024-04-17T10:45:48.708 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_inodes": 0, 2024-04-17T10:45:48.709 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_base_inodes": 0, 2024-04-17T10:45:48.709 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dirfrag_rstats": 0, 2024-04-17T10:45:48.709 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_file_inodes": 0, 2024-04-17T10:45:48.709 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_inode_file_caps": 0, 2024-04-17T10:45:48.709 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_revoke": 7085, 2024-04-17T10:45:48.709 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_grant": 6618, 2024-04-17T10:45:48.709 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_trunc": 0, 2024-04-17T10:45:48.709 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-17T10:45:48.709 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-17T10:45:48.709 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps": 7085, 2024-04-17T10:45:48.709 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps_dirty": 2000, 2024-04-17T10:45:48.709 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_cap_release": 61, 2024-04-17T10:45:48.710 INFO:teuthology.orchestra.run.smithi083.stdout: "process_request_cap_release": 17937 2024-04-17T10:45:48.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.710 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache": { 2024-04-17T10:45:48.710 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update": 0, 2024-04-17T10:45:48.710 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update_receipt": 0, 2024-04-17T10:45:48.710 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_try_discover": 0, 2024-04-17T10:45:48.710 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_send_discover": 0, 2024-04-17T10:45:48.710 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_handle_discover": 0, 2024-04-17T10:45:48.710 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays": 2000, 2024-04-17T10:45:48.710 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_delayed": 0, 2024-04-17T10:45:48.710 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_enqueuing": 0, 2024-04-17T10:45:48.710 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_created": 4152, 2024-04-17T10:45:48.711 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_enqueued": 1800, 2024-04-17T10:45:48.711 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_reintegrated": 2253, 2024-04-17T10:45:48.711 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_migrated": 0, 2024-04-17T10:45:48.711 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_processing": 0, 2024-04-17T10:45:48.711 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_enqueued": 0, 2024-04-17T10:45:48.711 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_prioritized": 0, 2024-04-17T10:45:48.711 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_started": 0, 2024-04-17T10:45:48.711 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_completed": 0, 2024-04-17T10:45:48.711 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_enqueue_scrub": 0, 2024-04-17T10:45:48.711 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_exportdir": 0, 2024-04-17T10:45:48.711 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_flush": 0, 2024-04-17T10:45:48.711 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragmentdir": 0, 2024-04-17T10:45:48.712 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragstats": 0, 2024-04-17T10:45:48.712 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_inodestats": 0 2024-04-17T10:45:48.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.712 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log": { 2024-04-17T10:45:48.712 INFO:teuthology.orchestra.run.smithi083.stdout: "evlrg": 0, 2024-04-17T10:45:48.712 INFO:teuthology.orchestra.run.smithi083.stdout: "evadd": 23531, 2024-04-17T10:45:48.712 INFO:teuthology.orchestra.run.smithi083.stdout: "evex": 0, 2024-04-17T10:45:48.712 INFO:teuthology.orchestra.run.smithi083.stdout: "evtrm": 0, 2024-04-17T10:45:48.712 INFO:teuthology.orchestra.run.smithi083.stdout: "ev": 23531, 2024-04-17T10:45:48.712 INFO:teuthology.orchestra.run.smithi083.stdout: "evexg": 0, 2024-04-17T10:45:48.712 INFO:teuthology.orchestra.run.smithi083.stdout: "evexd": 0, 2024-04-17T10:45:48.712 INFO:teuthology.orchestra.run.smithi083.stdout: "segadd": 31, 2024-04-17T10:45:48.713 INFO:teuthology.orchestra.run.smithi083.stdout: "segex": 0, 2024-04-17T10:45:48.713 INFO:teuthology.orchestra.run.smithi083.stdout: "segtrm": 0, 2024-04-17T10:45:48.713 INFO:teuthology.orchestra.run.smithi083.stdout: "seg": 31, 2024-04-17T10:45:48.713 INFO:teuthology.orchestra.run.smithi083.stdout: "segmjr": 3, 2024-04-17T10:45:48.713 INFO:teuthology.orchestra.run.smithi083.stdout: "segexg": 0, 2024-04-17T10:45:48.713 INFO:teuthology.orchestra.run.smithi083.stdout: "segexd": 0, 2024-04-17T10:45:48.713 INFO:teuthology.orchestra.run.smithi083.stdout: "expos": 4194304, 2024-04-17T10:45:48.713 INFO:teuthology.orchestra.run.smithi083.stdout: "wrpos": 85309310, 2024-04-17T10:45:48.713 INFO:teuthology.orchestra.run.smithi083.stdout: "rdpos": 0, 2024-04-17T10:45:48.713 INFO:teuthology.orchestra.run.smithi083.stdout: "jlat": { 2024-04-17T10:45:48.713 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 7219, 2024-04-17T10:45:48.713 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 129.197065773, 2024-04-17T10:45:48.714 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.017896809 2024-04-17T10:45:48.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.714 INFO:teuthology.orchestra.run.smithi083.stdout: "replayed": 0 2024-04-17T10:45:48.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.714 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_mem": { 2024-04-17T10:45:48.714 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 2014, 2024-04-17T10:45:48.714 INFO:teuthology.orchestra.run.smithi083.stdout: "ino+": 2014, 2024-04-17T10:45:48.714 INFO:teuthology.orchestra.run.smithi083.stdout: "ino-": 0, 2024-04-17T10:45:48.714 INFO:teuthology.orchestra.run.smithi083.stdout: "dir": 13, 2024-04-17T10:45:48.714 INFO:teuthology.orchestra.run.smithi083.stdout: "dir+": 13, 2024-04-17T10:45:48.714 INFO:teuthology.orchestra.run.smithi083.stdout: "dir-": 0, 2024-04-17T10:45:48.714 INFO:teuthology.orchestra.run.smithi083.stdout: "dn": 19867, 2024-04-17T10:45:48.715 INFO:teuthology.orchestra.run.smithi083.stdout: "dn+": 19867, 2024-04-17T10:45:48.715 INFO:teuthology.orchestra.run.smithi083.stdout: "dn-": 0, 2024-04-17T10:45:48.715 INFO:teuthology.orchestra.run.smithi083.stdout: "cap": 202, 2024-04-17T10:45:48.715 INFO:teuthology.orchestra.run.smithi083.stdout: "cap+": 2002, 2024-04-17T10:45:48.715 INFO:teuthology.orchestra.run.smithi083.stdout: "cap-": 1800, 2024-04-17T10:45:48.715 INFO:teuthology.orchestra.run.smithi083.stdout: "rss": 101212, 2024-04-17T10:45:48.715 INFO:teuthology.orchestra.run.smithi083.stdout: "heap": 215340 2024-04-17T10:45:48.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.715 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_server": { 2024-04-17T10:45:48.715 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_client_request": 36904, 2024-04-17T10:45:48.715 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_server_request": 0, 2024-04-17T10:45:48.715 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_request": 27238, 2024-04-17T10:45:48.716 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_session": 9, 2024-04-17T10:45:48.716 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_peer_request": 0, 2024-04-17T10:45:48.716 INFO:teuthology.orchestra.run.smithi083.stdout: "req_create_latency": { 2024-04-17T10:45:48.716 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2000, 2024-04-17T10:45:48.716 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 3.400930560, 2024-04-17T10:45:48.716 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.001700465 2024-04-17T10:45:48.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.716 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getattr_latency": { 2024-04-17T10:45:48.716 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 934, 2024-04-17T10:45:48.716 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.805390969, 2024-04-17T10:45:48.716 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000862302 2024-04-17T10:45:48.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.717 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getfilelock_latency": { 2024-04-17T10:45:48.717 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:45:48.717 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:45:48.717 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:45:48.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.717 INFO:teuthology.orchestra.run.smithi083.stdout: "req_link_latency": { 2024-04-17T10:45:48.717 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 6000, 2024-04-17T10:45:48.717 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 22.435763586, 2024-04-17T10:45:48.717 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.003739293 2024-04-17T10:45:48.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.717 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookup_latency": { 2024-04-17T10:45:48.717 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 8038, 2024-04-17T10:45:48.717 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 19.302557117, 2024-04-17T10:45:48.718 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.002401412 2024-04-17T10:45:48.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.718 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookuphash_latency": { 2024-04-17T10:45:48.718 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:45:48.718 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:45:48.718 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:45:48.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.718 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupino_latency": { 2024-04-17T10:45:48.718 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:45:48.718 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:45:48.718 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:45:48.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.719 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupname_latency": { 2024-04-17T10:45:48.719 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:45:48.719 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:45:48.719 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:45:48.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.719 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupparent_latency": { 2024-04-17T10:45:48.719 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:45:48.719 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:45:48.719 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:45:48.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.719 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupsnap_latency": { 2024-04-17T10:45:48.719 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3, 2024-04-17T10:45:48.720 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.106284228, 2024-04-17T10:45:48.720 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.035428076 2024-04-17T10:45:48.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.720 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lssnap_latency": { 2024-04-17T10:45:48.720 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:45:48.720 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:45:48.720 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:45:48.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.720 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mkdir_latency": { 2024-04-17T10:45:48.720 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1, 2024-04-17T10:45:48.720 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.004477250, 2024-04-17T10:45:48.721 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.004477250 2024-04-17T10:45:48.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.721 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mknod_latency": { 2024-04-17T10:45:48.721 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:45:48.721 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:45:48.721 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:45:48.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.721 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mksnap_latency": { 2024-04-17T10:45:48.721 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1, 2024-04-17T10:45:48.721 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.006911550, 2024-04-17T10:45:48.721 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.006911550 2024-04-17T10:45:48.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.722 INFO:teuthology.orchestra.run.smithi083.stdout: "req_open_latency": { 2024-04-17T10:45:48.722 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:45:48.722 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:45:48.722 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:45:48.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.722 INFO:teuthology.orchestra.run.smithi083.stdout: "req_readdir_latency": { 2024-04-17T10:45:48.722 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 6, 2024-04-17T10:45:48.722 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.245347336, 2024-04-17T10:45:48.722 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.040891222 2024-04-17T10:45:48.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.722 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rename_latency": { 2024-04-17T10:45:48.723 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2253, 2024-04-17T10:45:48.723 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 423.681745573, 2024-04-17T10:45:48.723 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.188052261 2024-04-17T10:45:48.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.723 INFO:teuthology.orchestra.run.smithi083.stdout: "req_renamesnap_latency": { 2024-04-17T10:45:48.723 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:45:48.723 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:45:48.723 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:45:48.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.723 INFO:teuthology.orchestra.run.smithi083.stdout: "req_snapdiff_latency": { 2024-04-17T10:45:48.723 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:45:48.724 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:45:48.724 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:45:48.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.724 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmdir_latency": { 2024-04-17T10:45:48.724 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:45:48.724 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:45:48.724 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:45:48.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.724 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmsnap_latency": { 2024-04-17T10:45:48.724 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1, 2024-04-17T10:45:48.724 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.036436369, 2024-04-17T10:45:48.724 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.036436369 2024-04-17T10:45:48.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.725 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmxattr_latency": { 2024-04-17T10:45:48.725 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:45:48.725 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:45:48.725 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:45:48.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.725 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setattr_latency": { 2024-04-17T10:45:48.725 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1, 2024-04-17T10:45:48.725 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000554538, 2024-04-17T10:45:48.725 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000554538 2024-04-17T10:45:48.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.725 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setdirlayout_latency": { 2024-04-17T10:45:48.726 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:45:48.726 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:45:48.726 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:45:48.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.726 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setfilelock_latency": { 2024-04-17T10:45:48.726 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:45:48.726 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:45:48.726 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:45:48.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.726 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setlayout_latency": { 2024-04-17T10:45:48.726 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:45:48.726 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:45:48.727 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:45:48.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.727 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setxattr_latency": { 2024-04-17T10:45:48.727 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:45:48.727 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:45:48.727 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:45:48.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.727 INFO:teuthology.orchestra.run.smithi083.stdout: "req_symlink_latency": { 2024-04-17T10:45:48.727 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:45:48.727 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:45:48.727 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:45:48.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.728 INFO:teuthology.orchestra.run.smithi083.stdout: "req_unlink_latency": { 2024-04-17T10:45:48.728 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 8000, 2024-04-17T10:45:48.728 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 36.663364200, 2024-04-17T10:45:48.728 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.004582920 2024-04-17T10:45:48.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.728 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_revoke_eviction": 0, 2024-04-17T10:45:48.728 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_acquisition_throttle": 0, 2024-04-17T10:45:48.728 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getvxattr_latency": { 2024-04-17T10:45:48.728 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:45:48.728 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:45:48.728 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:45:48.728 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:45:48.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.729 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_sessions": { 2024-04-17T10:45:48.729 INFO:teuthology.orchestra.run.smithi083.stdout: "session_count": 1, 2024-04-17T10:45:48.729 INFO:teuthology.orchestra.run.smithi083.stdout: "session_add": 1, 2024-04-17T10:45:48.729 INFO:teuthology.orchestra.run.smithi083.stdout: "session_remove": 0, 2024-04-17T10:45:48.729 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_open": 1, 2024-04-17T10:45:48.729 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_stale": 0, 2024-04-17T10:45:48.729 INFO:teuthology.orchestra.run.smithi083.stdout: "total_load": 13344, 2024-04-17T10:45:48.729 INFO:teuthology.orchestra.run.smithi083.stdout: "average_load": 13344, 2024-04-17T10:45:48.729 INFO:teuthology.orchestra.run.smithi083.stdout: "avg_session_uptime": 118, 2024-04-17T10:45:48.729 INFO:teuthology.orchestra.run.smithi083.stdout: "mdthresh_evicted": 0 2024-04-17T10:45:48.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.730 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-04-17T10:45:48.730 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-04-17T10:45:48.730 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-04-17T10:45:48.730 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T10:45:48.730 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-04-17T10:45:48.730 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T10:45:48.730 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-04-17T10:45:48.730 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T10:45:48.730 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T10:45:48.730 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T10:45:48.730 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T10:45:48.731 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T10:45:48.731 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-04-17T10:45:48.731 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T10:45:48.731 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T10:45:48.731 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-04-17T10:45:48.731 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-04-17T10:45:48.731 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-04-17T10:45:48.731 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-04-17T10:45:48.731 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T10:45:48.731 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T10:45:48.731 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T10:45:48.731 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T10:45:48.731 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T10:45:48.732 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-04-17T10:45:48.732 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-04-17T10:45:48.732 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-04-17T10:45:48.732 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T10:45:48.732 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T10:45:48.732 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-04-17T10:45:48.732 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-04-17T10:45:48.732 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-04-17T10:45:48.732 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-04-17T10:45:48.732 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T10:45:48.732 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-04-17T10:45:48.732 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T10:45:48.733 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-04-17T10:45:48.733 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 16422265, 2024-04-17T10:45:48.733 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 4174, 2024-04-17T10:45:48.733 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-04-17T10:45:48.733 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-04-17T10:45:48.733 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-04-17T10:45:48.733 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-04-17T10:45:48.733 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-04-17T10:45:48.733 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-04-17T10:45:48.733 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-04-17T10:45:48.733 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-04-17T10:45:48.733 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 5504, 2024-04-17T10:45:48.734 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 79, 2024-04-17T10:45:48.734 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T10:45:48.734 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-04-17T10:45:48.734 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-04-17T10:45:48.734 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-04-17T10:45:48.734 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 20912235, 2024-04-17T10:45:48.734 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 813409, 2024-04-17T10:45:48.734 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-04-17T10:45:48.734 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-04-17T10:45:48.734 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-04-17T10:45:48.734 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-04-17T10:45:48.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.735 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter": { 2024-04-17T10:45:48.735 INFO:teuthology.orchestra.run.smithi083.stdout: "op_active": 0, 2024-04-17T10:45:48.735 INFO:teuthology.orchestra.run.smithi083.stdout: "op_laggy": 0, 2024-04-17T10:45:48.735 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send": 9534, 2024-04-17T10:45:48.735 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send_bytes": 81495902, 2024-04-17T10:45:48.735 INFO:teuthology.orchestra.run.smithi083.stdout: "op_resend": 37, 2024-04-17T10:45:48.735 INFO:teuthology.orchestra.run.smithi083.stdout: "op_reply": 9534, 2024-04-17T10:45:48.735 INFO:teuthology.orchestra.run.smithi083.stdout: "op_latency": { 2024-04-17T10:45:48.735 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 9534, 2024-04-17T10:45:48.735 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 42.899552625, 2024-04-17T10:45:48.735 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.004499638 2024-04-17T10:45:48.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.736 INFO:teuthology.orchestra.run.smithi083.stdout: "op_inflight": 0, 2024-04-17T10:45:48.736 INFO:teuthology.orchestra.run.smithi083.stdout: "oplen_avg": { 2024-04-17T10:45:48.736 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 9534, 2024-04-17T10:45:48.736 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 9741 2024-04-17T10:45:48.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.736 INFO:teuthology.orchestra.run.smithi083.stdout: "op": 9534, 2024-04-17T10:45:48.736 INFO:teuthology.orchestra.run.smithi083.stdout: "op_r": 131, 2024-04-17T10:45:48.736 INFO:teuthology.orchestra.run.smithi083.stdout: "op_w": 9403, 2024-04-17T10:45:48.736 INFO:teuthology.orchestra.run.smithi083.stdout: "op_rmw": 0, 2024-04-17T10:45:48.736 INFO:teuthology.orchestra.run.smithi083.stdout: "op_pg": 0, 2024-04-17T10:45:48.736 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_stat": 0, 2024-04-17T10:45:48.736 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_create": 11, 2024-04-17T10:45:48.737 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_read": 131, 2024-04-17T10:45:48.737 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_write": 7350, 2024-04-17T10:45:48.737 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writefull": 77, 2024-04-17T10:45:48.737 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writesame": 0, 2024-04-17T10:45:48.737 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_append": 0, 2024-04-17T10:45:48.737 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_zero": 1800, 2024-04-17T10:45:48.737 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_truncate": 0, 2024-04-17T10:45:48.737 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_delete": 31, 2024-04-17T10:45:48.737 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_mapext": 0, 2024-04-17T10:45:48.737 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_sparse_read": 0, 2024-04-17T10:45:48.737 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_clonerange": 0, 2024-04-17T10:45:48.737 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_getxattr": 0, 2024-04-17T10:45:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_setxattr": 22, 2024-04-17T10:45:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_cmpxattr": 0, 2024-04-17T10:45:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_rmxattr": 0, 2024-04-17T10:45:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_resetxattrs": 0, 2024-04-17T10:45:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_call": 0, 2024-04-17T10:45:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_watch": 0, 2024-04-17T10:45:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_notify": 0, 2024-04-17T10:45:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_src_cmpxattr": 0, 2024-04-17T10:45:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls": 0, 2024-04-17T10:45:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls_filter": 0, 2024-04-17T10:45:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_other": 0, 2024-04-17T10:45:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_active": 0, 2024-04-17T10:45:48.739 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_send": 0, 2024-04-17T10:45:48.739 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_resend": 0, 2024-04-17T10:45:48.739 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_ping": 0, 2024-04-17T10:45:48.739 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_active": 0, 2024-04-17T10:45:48.739 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_send": 0, 2024-04-17T10:45:48.739 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_resend": 0, 2024-04-17T10:45:48.739 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_active": 0, 2024-04-17T10:45:48.739 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_send": 0, 2024-04-17T10:45:48.739 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_resend": 0, 2024-04-17T10:45:48.739 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_active": 0, 2024-04-17T10:45:48.739 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_send": 0, 2024-04-17T10:45:48.739 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_resend": 0, 2024-04-17T10:45:48.740 INFO:teuthology.orchestra.run.smithi083.stdout: "command_active": 0, 2024-04-17T10:45:48.740 INFO:teuthology.orchestra.run.smithi083.stdout: "command_send": 0, 2024-04-17T10:45:48.740 INFO:teuthology.orchestra.run.smithi083.stdout: "command_resend": 0, 2024-04-17T10:45:48.740 INFO:teuthology.orchestra.run.smithi083.stdout: "map_epoch": 76, 2024-04-17T10:45:48.740 INFO:teuthology.orchestra.run.smithi083.stdout: "map_full": 0, 2024-04-17T10:45:48.740 INFO:teuthology.orchestra.run.smithi083.stdout: "map_inc": 3, 2024-04-17T10:45:48.740 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_sessions": 8, 2024-04-17T10:45:48.740 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_open": 8, 2024-04-17T10:45:48.740 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_close": 0, 2024-04-17T10:45:48.740 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_laggy": 0, 2024-04-17T10:45:48.740 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_wr": 255, 2024-04-17T10:45:48.740 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_rd": 0, 2024-04-17T10:45:48.741 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_del": 64 2024-04-17T10:45:48.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.741 INFO:teuthology.orchestra.run.smithi083.stdout: "oft": { 2024-04-17T10:45:48.741 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_objs": 1, 2024-04-17T10:45:48.741 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_kv_pairs": 0, 2024-04-17T10:45:48.741 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_updates": 0, 2024-04-17T10:45:48.741 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_removes": 0 2024-04-17T10:45:48.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.741 INFO:teuthology.orchestra.run.smithi083.stdout: "osd-slow-ops": { 2024-04-17T10:45:48.741 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_ops_count": 0 2024-04-17T10:45:48.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.741 INFO:teuthology.orchestra.run.smithi083.stdout: "purge_queue": { 2024-04-17T10:45:48.741 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops": 0, 2024-04-17T10:45:48.742 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops_high_water": 32, 2024-04-17T10:45:48.742 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing": 0, 2024-04-17T10:45:48.742 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_high_water": 32, 2024-04-17T10:45:48.742 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed_ops": 1800, 2024-04-17T10:45:48.742 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed": 1800, 2024-04-17T10:45:48.742 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_item_in_journal": 0 2024-04-17T10:45:48.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.742 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T10:45:48.742 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-17T10:45:48.742 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-17T10:45:48.742 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-17T10:45:48.743 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 41940, 2024-04-17T10:45:48.743 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 10607650, 2024-04-17T10:45:48.743 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-17T10:45:48.743 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 41940, 2024-04-17T10:45:48.743 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-17T10:45:48.743 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-17T10:45:48.743 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 41940, 2024-04-17T10:45:48.743 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 10607650, 2024-04-17T10:45:48.743 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-17T10:45:48.743 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:45:48.743 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:45:48.743 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:45:48.744 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:45:48.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.744 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_bytes": { 2024-04-17T10:45:48.744 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-17T10:45:48.744 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-17T10:45:48.744 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-17T10:45:48.744 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-17T10:45:48.744 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-17T10:45:48.744 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-17T10:45:48.744 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-17T10:45:48.745 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 9534, 2024-04-17T10:45:48.745 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 81692102, 2024-04-17T10:45:48.745 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 9534, 2024-04-17T10:45:48.745 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 81692102, 2024-04-17T10:45:48.745 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-17T10:45:48.745 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:45:48.745 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:45:48.745 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:45:48.745 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:45:48.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.745 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_ops": { 2024-04-17T10:45:48.745 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-17T10:45:48.746 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 1024, 2024-04-17T10:45:48.746 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-17T10:45:48.746 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-17T10:45:48.746 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-17T10:45:48.746 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-17T10:45:48.746 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-17T10:45:48.746 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 9534, 2024-04-17T10:45:48.746 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 9534, 2024-04-17T10:45:48.746 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 9534, 2024-04-17T10:45:48.746 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 9534, 2024-04-17T10:45:48.747 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-17T10:45:48.747 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:45:48.747 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:45:48.747 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:45:48.747 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:45:48.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.747 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle": { 2024-04-17T10:45:48.747 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-17T10:45:48.747 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-04-17T10:45:48.747 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-17T10:45:48.747 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 1800, 2024-04-17T10:45:48.748 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 196200, 2024-04-17T10:45:48.748 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-17T10:45:48.748 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 1800, 2024-04-17T10:45:48.748 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-17T10:45:48.748 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-17T10:45:48.748 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 131, 2024-04-17T10:45:48.748 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 196200, 2024-04-17T10:45:48.748 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-17T10:45:48.748 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:45:48.748 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:45:48.748 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:45:48.748 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:45:48.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:45:48.749 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle-0x557bcc38ea00": { 2024-04-17T10:45:48.749 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 84, 2024-04-17T10:45:48.749 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-04-17T10:45:48.749 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-17T10:45:48.749 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 23531, 2024-04-17T10:45:48.749 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 81115006, 2024-04-17T10:45:48.749 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-17T10:45:48.749 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 23531, 2024-04-17T10:45:48.749 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-17T10:45:48.749 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-17T10:45:48.749 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 7219, 2024-04-17T10:45:48.750 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 81114922, 2024-04-17T10:45:48.750 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-17T10:45:48.750 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:45:48.750 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:45:48.750 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:45:48.750 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:45:48.750 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:45:48.750 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:45:48.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd2449b8640 1 -- 172.21.15.83:0/3188711185 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd220066f60 msgr2=0x7fd220069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:45:48.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd2449b8640 1 --2- 172.21.15.83:0/3188711185 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd220066f60 0x7fd220069420 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd24006a390 tx=0x7fd230009040 comp rx=0 tx=0).stop 2024-04-17T10:45:48.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd2449b8640 1 -- 172.21.15.83:0/3188711185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd24014c350 msgr2=0x7fd240150e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:45:48.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd2449b8640 1 --2- 172.21.15.83:0/3188711185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd24014c350 0x7fd240150e60 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fd23401c8c0 tx=0x7fd234019830 comp rx=0 tx=0).stop 2024-04-17T10:45:48.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd2449b8640 1 -- 172.21.15.83:0/3188711185 shutdown_connections 2024-04-17T10:45:48.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd2449b8640 1 --2- 172.21.15.83:0/3188711185 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd220066f60 0x7fd220069420 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd2449b8640 1 --2- 172.21.15.83:0/3188711185 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2401529c0 0x7fd240152e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd2449b8640 1 --2- 172.21.15.83:0/3188711185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd24014c350 0x7fd240150e60 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd2449b8640 1 --2- 172.21.15.83:0/3188711185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd228004810 0x7fd240141f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd2449b8640 1 -- 172.21.15.83:0/3188711185 >> 172.21.15.83:0/3188711185 conn(0x7fd2400b15b0 msgr2=0x7fd2400afa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:45:48.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd2449b8640 1 -- 172.21.15.83:0/3188711185 shutdown_connections 2024-04-17T10:45:48.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.691+0000 7fd2449b8640 1 -- 172.21.15.83:0/3188711185 wait complete. 2024-04-17T10:45:48.754 INFO:tasks.cephfs.test_scrub_checks:num of strays: 2000 2024-04-17T10:45:48.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:45:48.754 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start '~mdsdir' recursive 2024-04-17T10:45:48.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.875+0000 7efc8d2b5640 1 Processor -- start 2024-04-17T10:45:48.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.875+0000 7efc8d2b5640 1 -- start start 2024-04-17T10:45:48.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.879+0000 7efc8d2b5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efc880aebd0 0x7efc880ced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.879+0000 7efc8d2b5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7efc8815b990 0x7efc880cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.879+0000 7efc8d2b5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc880cd6c0 0x7efc8806abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.879+0000 7efc8d2b5640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc8815e5c0 con 0x7efc880aebd0 2024-04-17T10:45:48.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.879+0000 7efc8d2b5640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc880cf3e0 con 0x7efc8815b990 2024-04-17T10:45:48.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.879+0000 7efc8d2b5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc880b4230 con 0x7efc880cd6c0 2024-04-17T10:45:48.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.879+0000 7efc86ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efc880aebd0 0x7efc880ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.879+0000 7efc86ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efc880aebd0 0x7efc880ced50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56760/0 (socket says 172.21.15.83:56760) 2024-04-17T10:45:48.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.879+0000 7efc86ffd640 1 -- 172.21.15.83:0/3254646817 learned_addr learned my addr 172.21.15.83:0/3254646817 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:45:48.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.879+0000 7efc867fc640 1 --2- 172.21.15.83:0/3254646817 >> v2:172.21.15.167:3300/0 conn(0x7efc8815b990 0x7efc880cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.879+0000 7efc877fe640 1 --2- 172.21.15.83:0/3254646817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc880cd6c0 0x7efc8806abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.879+0000 7efc86ffd640 1 -- 172.21.15.83:0/3254646817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc880cd6c0 msgr2=0x7efc8806abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:45:48.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.879+0000 7efc86ffd640 1 --2- 172.21.15.83:0/3254646817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc880cd6c0 0x7efc8806abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.879+0000 7efc86ffd640 1 -- 172.21.15.83:0/3254646817 >> v2:172.21.15.167:3300/0 conn(0x7efc8815b990 msgr2=0x7efc880cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:45:48.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.879+0000 7efc86ffd640 1 --2- 172.21.15.83:0/3254646817 >> v2:172.21.15.167:3300/0 conn(0x7efc8815b990 0x7efc880cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.879+0000 7efc86ffd640 1 -- 172.21.15.83:0/3254646817 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc880b4510 con 0x7efc880aebd0 2024-04-17T10:45:48.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.879+0000 7efc86ffd640 1 --2- 172.21.15.83:0/3254646817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc880aebd0 0x7efc880ced50 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7efc880cd130 tx=0x7efc7801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721201032fba911e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:45:48.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.879+0000 7efc85ffb640 1 -- 172.21.15.83:0/3254646817 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc780a5670 con 0x7efc880aebd0 2024-04-17T10:45:48.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc85ffb640 1 -- 172.21.15.83:0/3254646817 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc780a5810 con 0x7efc880aebd0 2024-04-17T10:45:48.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc85ffb640 1 -- 172.21.15.83:0/3254646817 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc780a5be0 con 0x7efc880aebd0 2024-04-17T10:45:48.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc8d2b5640 1 -- 172.21.15.83:0/3254646817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc880aebd0 msgr2=0x7efc880ced50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:45:48.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc8d2b5640 1 --2- 172.21.15.83:0/3254646817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc880aebd0 0x7efc880ced50 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7efc880cd130 tx=0x7efc7801c9f0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc8d2b5640 1 -- 172.21.15.83:0/3254646817 shutdown_connections 2024-04-17T10:45:48.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc8d2b5640 1 --2- 172.21.15.83:0/3254646817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc880cd6c0 0x7efc8806abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc8d2b5640 1 --2- 172.21.15.83:0/3254646817 >> v2:172.21.15.167:3300/0 conn(0x7efc8815b990 0x7efc880cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc8d2b5640 1 --2- 172.21.15.83:0/3254646817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc880aebd0 0x7efc880ced50 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc8d2b5640 1 -- 172.21.15.83:0/3254646817 >> 172.21.15.83:0/3254646817 conn(0x7efc880cc050 msgr2=0x7efc880cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:45:48.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc8d2b5640 1 -- 172.21.15.83:0/3254646817 shutdown_connections 2024-04-17T10:45:48.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc8d2b5640 1 -- 172.21.15.83:0/3254646817 wait complete. 2024-04-17T10:45:48.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc8d2b5640 1 Processor -- start 2024-04-17T10:45:48.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc8d2b5640 1 -- start start 2024-04-17T10:45:48.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc8d2b5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc880aebd0 0x7efc8815bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc8d2b5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc8815b990 0x7efc8813e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc8d2b5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc88157390 0x7efc8814eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc8d2b5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8806d3f0 con 0x7efc8815b990 2024-04-17T10:45:48.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc8d2b5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8806d270 con 0x7efc88157390 2024-04-17T10:45:48.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc8d2b5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc8806d570 con 0x7efc880aebd0 2024-04-17T10:45:48.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc867fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc8815b990 0x7efc8813e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc867fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc8815b990 0x7efc8813e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56766/0 (socket says 172.21.15.83:56766) 2024-04-17T10:45:48.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc867fc640 1 -- 172.21.15.83:0/2822504162 learned_addr learned my addr 172.21.15.83:0/2822504162 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:45:48.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc86ffd640 1 --2- 172.21.15.83:0/2822504162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc880aebd0 0x7efc8815bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc877fe640 1 --2- 172.21.15.83:0/2822504162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc88157390 0x7efc8814eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc867fc640 1 -- 172.21.15.83:0/2822504162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc880aebd0 msgr2=0x7efc8815bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:45:48.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc867fc640 1 --2- 172.21.15.83:0/2822504162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc880aebd0 0x7efc8815bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc867fc640 1 -- 172.21.15.83:0/2822504162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc88157390 msgr2=0x7efc8814eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:45:48.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc867fc640 1 --2- 172.21.15.83:0/2822504162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc88157390 0x7efc8814eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc867fc640 1 -- 172.21.15.83:0/2822504162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc88157e10 con 0x7efc8815b990 2024-04-17T10:45:48.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc867fc640 1 --2- 172.21.15.83:0/2822504162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc8815b990 0x7efc8813e260 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00d5f0 tx=0x7efc7c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:45:48.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc6f7fe640 1 -- 172.21.15.83:0/2822504162 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c0125f0 con 0x7efc8815b990 2024-04-17T10:45:48.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc6f7fe640 1 -- 172.21.15.83:0/2822504162 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc7c019070 con 0x7efc8815b990 2024-04-17T10:45:48.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.883+0000 7efc6f7fe640 1 -- 172.21.15.83:0/2822504162 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c0129b0 con 0x7efc8815b990 2024-04-17T10:45:48.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.887+0000 7efc8d2b5640 1 -- 172.21.15.83:0/2822504162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc881580f0 con 0x7efc8815b990 2024-04-17T10:45:48.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.887+0000 7efc8d2b5640 1 -- 172.21.15.83:0/2822504162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc881586b0 con 0x7efc8815b990 2024-04-17T10:45:48.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.887+0000 7efc6f7fe640 1 -- 172.21.15.83:0/2822504162 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efc7c012b50 con 0x7efc8815b990 2024-04-17T10:45:48.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.887+0000 7efc6f7fe640 1 --2- 172.21.15.83:0/2822504162 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efc74066f90 0x7efc74069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.887+0000 7efc86ffd640 1 --2- 172.21.15.83:0/2822504162 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efc74066f90 0x7efc74069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.887+0000 7efc6f7fe640 1 -- 172.21.15.83:0/2822504162 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5767+0+0 (secure 0 0 0) 0x7efc7c085420 con 0x7efc8815b990 2024-04-17T10:45:48.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.887+0000 7efc86ffd640 1 --2- 172.21.15.83:0/2822504162 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efc74066f90 0x7efc74069450 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7efc780ae000 tx=0x7efc780a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:45:48.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.887+0000 7efc90422000 1 Processor -- start 2024-04-17T10:45:48.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.887+0000 7efc90422000 1 -- start start 2024-04-17T10:45:48.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.887+0000 7efc90422000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cbf928c0 0x5569cbf92ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.887+0000 7efc90422000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5569cbf931e0 0x5569cbf48110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.887+0000 7efc90422000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569cbf48650 0x5569cbf48a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.887+0000 7efc90422000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569cbd96dd0 con 0x5569cbf928c0 2024-04-17T10:45:48.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.887+0000 7efc90422000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569cbd1f010 con 0x5569cbf48650 2024-04-17T10:45:48.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.887+0000 7efc90422000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5569cbd92e70 con 0x5569cbf931e0 2024-04-17T10:45:48.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.887+0000 7efc86ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cbf928c0 0x5569cbf92ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc86ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cbf928c0 0x5569cbf92ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56778/0 (socket says 172.21.15.83:56778) 2024-04-17T10:45:48.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc86ffd640 1 -- 172.21.15.83:0/3598803452 learned_addr learned my addr 172.21.15.83:0/3598803452 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:45:48.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc867fc640 1 --2- 172.21.15.83:0/3598803452 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5569cbf931e0 0x5569cbf48110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc877fe640 1 --2- 172.21.15.83:0/3598803452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569cbf48650 0x5569cbf48a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc86ffd640 1 -- 172.21.15.83:0/3598803452 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5569cbf931e0 msgr2=0x5569cbf48110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:45:48.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc86ffd640 1 --2- 172.21.15.83:0/3598803452 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5569cbf931e0 0x5569cbf48110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc86ffd640 1 -- 172.21.15.83:0/3598803452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569cbf48650 msgr2=0x5569cbf48a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:45:48.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc86ffd640 1 --2- 172.21.15.83:0/3598803452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569cbf48650 0x5569cbf48a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc86ffd640 1 -- 172.21.15.83:0/3598803452 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569cbd4f9c0 con 0x5569cbf928c0 2024-04-17T10:45:48.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc86ffd640 1 --2- 172.21.15.83:0/3598803452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cbf928c0 0x5569cbf92ca0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7efc7801ca30 tx=0x7efc78004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38134c8095ba9b76 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:45:48.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc6cff9640 1 -- 172.21.15.83:0/3598803452 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc780c7070 con 0x5569cbf928c0 2024-04-17T10:45:48.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc6cff9640 1 -- 172.21.15.83:0/3598803452 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc780a5dc0 con 0x5569cbf928c0 2024-04-17T10:45:48.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc6cff9640 1 -- 172.21.15.83:0/3598803452 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc780b6620 con 0x5569cbf928c0 2024-04-17T10:45:48.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc90422000 1 -- 172.21.15.83:0/3598803452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cbf928c0 msgr2=0x5569cbf92ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:45:48.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc90422000 1 --2- 172.21.15.83:0/3598803452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cbf928c0 0x5569cbf92ca0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7efc7801ca30 tx=0x7efc78004ac0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc90422000 1 -- 172.21.15.83:0/3598803452 shutdown_connections 2024-04-17T10:45:48.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc90422000 1 --2- 172.21.15.83:0/3598803452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569cbf48650 0x5569cbf48a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc90422000 1 --2- 172.21.15.83:0/3598803452 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5569cbf931e0 0x5569cbf48110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc90422000 1 --2- 172.21.15.83:0/3598803452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cbf928c0 0x5569cbf92ca0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc90422000 1 -- 172.21.15.83:0/3598803452 >> 172.21.15.83:0/3598803452 conn(0x5569cbe30870 msgr2=0x5569cbf469f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:45:48.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc90422000 1 -- 172.21.15.83:0/3598803452 shutdown_connections 2024-04-17T10:45:48.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc90422000 1 -- 172.21.15.83:0/3598803452 wait complete. 2024-04-17T10:45:48.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc90422000 1 -- start start 2024-04-17T10:45:48.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc90422000 1 Processor -- start 2024-04-17T10:45:48.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc90422000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cbf48650 0x5569cbf4cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc90422000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5569cbf928c0 0x5569cbf4d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc90422000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569cbf931e0 0x5569cbf4d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc90422000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569cbd51420 con 0x5569cbf48650 2024-04-17T10:45:48.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc90422000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569cbd960e0 con 0x5569cbf931e0 2024-04-17T10:45:48.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc90422000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5569cbf51c20 con 0x5569cbf928c0 2024-04-17T10:45:48.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc90422000 10 client.0 _ll_register_callbacks cb 0x5569cbf33f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:45:48.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc86ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cbf48650 0x5569cbf4cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc86ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cbf48650 0x5569cbf4cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56786/0 (socket says 172.21.15.83:56786) 2024-04-17T10:45:48.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc86ffd640 1 -- 172.21.15.83:0/3335899505 learned_addr learned my addr 172.21.15.83:0/3335899505 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:45:48.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc867fc640 1 --2- 172.21.15.83:0/3335899505 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5569cbf928c0 0x5569cbf4d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc877fe640 1 --2- 172.21.15.83:0/3335899505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569cbf931e0 0x5569cbf4d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc86ffd640 1 -- 172.21.15.83:0/3335899505 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5569cbf928c0 msgr2=0x5569cbf4d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:45:48.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc86ffd640 1 --2- 172.21.15.83:0/3335899505 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5569cbf928c0 0x5569cbf4d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc86ffd640 1 -- 172.21.15.83:0/3335899505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569cbf931e0 msgr2=0x5569cbf4d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:45:48.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc86ffd640 1 --2- 172.21.15.83:0/3335899505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569cbf931e0 0x5569cbf4d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc86ffd640 1 -- 172.21.15.83:0/3335899505 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569cbd1f260 con 0x5569cbf48650 2024-04-17T10:45:48.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc86ffd640 1 --2- 172.21.15.83:0/3335899505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cbf48650 0x5569cbf4cd90 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7efc7801ced0 tx=0x7efc7801b230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67bf4448dd3c891e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:45:48.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:45:48.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc65ffb640 1 -- 172.21.15.83:0/3335899505 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc780c7070 con 0x5569cbf48650 2024-04-17T10:45:48.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc65ffb640 1 -- 172.21.15.83:0/3335899505 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(76..76 src has 1..76) v4 ==== 5767+0+0 (secure 0 0 0) 0x7efc780b6b10 con 0x5569cbf48650 2024-04-17T10:45:48.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc90422000 1 -- 172.21.15.83:0/3335899505 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc780a5dc0 con 0x5569cbf48650 2024-04-17T10:45:48.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc65ffb640 1 -- 172.21.15.83:0/3335899505 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc780b6d10 con 0x5569cbf48650 2024-04-17T10:45:48.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc90422000 10 client.7317 fetch_fsmap learned FSMap version 49 2024-04-17T10:45:48.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.891+0000 7efc90422000 1 -- 172.21.15.83:0/3335899505 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x5569cbd51b30 con 0x5569cbf48650 2024-04-17T10:45:48.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.895+0000 7efc65ffb640 1 -- 172.21.15.83:0/3335899505 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc780a5dc0 con 0x5569cbf48650 2024-04-17T10:45:48.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.895+0000 7efc65ffb640 1 -- 172.21.15.83:0/3335899505 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efc78008030 con 0x5569cbf48650 2024-04-17T10:45:48.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.895+0000 7efc90422000 10 client.7317 fetch_fsmap finished waiting for FSMap version 49 2024-04-17T10:45:48.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.895+0000 7efc90422000 10 client.7317 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-17T10:45:48.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.895+0000 7efc90422000 20 client.7317 populate_metadata read hostname 'smithi083' 2024-04-17T10:45:48.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.895+0000 7efc90422000 1 --2- 172.21.15.83:0/3335899505 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x5569cbfa1950 0x5569cbfa3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.895+0000 7efc90422000 4 client.7317 mds_command: new command op to 7104 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:45:48.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.895+0000 7efc90422000 1 -- 172.21.15.83:0/3335899505 --> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5569cbd32c80 con 0x5569cbfa1950 2024-04-17T10:45:48.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.895+0000 7efc867fc640 1 --2- 172.21.15.83:0/3335899505 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x5569cbfa1950 0x5569cbfa3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.895+0000 7efc65ffb640 10 client.7317 ms_handle_connect on v2:172.21.15.167:6832/2796648400 2024-04-17T10:45:48.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.895+0000 7efc867fc640 1 --2- 172.21.15.83:0/3335899505 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x5569cbfa1950 0x5569cbfa3d40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6d2c444b35fda43 server_cookie=85831eb7b0a11c09 in_seq=0 out_seq=0 2024-04-17T10:45:48.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.895+0000 7efc65ffb640 1 -- 172.21.15.83:0/3335899505 <== mds.0 v2:172.21.15.167:6832/2796648400 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5569cbd32c80 con 0x5569cbfa1950 2024-04-17T10:45:48.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.895+0000 7efc65ffb640 10 client.7317 handle_command_reply: tid=0 2024-04-17T10:45:48.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.895+0000 7efc90422000 1 client.7317 shutdown 2024-04-17T10:45:48.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.895+0000 7efc90422000 1 -- 172.21.15.83:0/3335899505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cbf48650 msgr2=0x5569cbf4cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:45:48.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.895+0000 7efc90422000 1 --2- 172.21.15.83:0/3335899505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cbf48650 0x5569cbf4cd90 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7efc7801ced0 tx=0x7efc7801b230 comp rx=0 tx=0).stop 2024-04-17T10:45:48.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.895+0000 7efc90422000 1 -- 172.21.15.83:0/3335899505 shutdown_connections 2024-04-17T10:45:48.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.895+0000 7efc90422000 1 --2- 172.21.15.83:0/3335899505 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x5569cbfa1950 0x5569cbfa3d40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.895+0000 7efc90422000 1 --2- 172.21.15.83:0/3335899505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569cbf931e0 0x5569cbf4d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.895+0000 7efc90422000 1 --2- 172.21.15.83:0/3335899505 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5569cbf928c0 0x5569cbf4d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.895+0000 7efc65ffb640 0 client.7317 ms_handle_reset on v2:172.21.15.167:6832/2796648400 2024-04-17T10:45:48.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.895+0000 7efc90422000 1 --2- 172.21.15.83:0/3335899505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cbf48650 0x5569cbf4cd90 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.895+0000 7efc90422000 1 -- 172.21.15.83:0/3335899505 >> 172.21.15.83:0/3335899505 conn(0x5569cbe30870 msgr2=0x5569cbf469f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:45:48.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.895+0000 7efc90422000 1 -- 172.21.15.83:0/3335899505 shutdown_connections 2024-04-17T10:45:48.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.895+0000 7efc90422000 1 -- 172.21.15.83:0/3335899505 wait complete. 2024-04-17T10:45:48.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.895+0000 7efc90422000 20 client.7317 trim_cache size 0 max 16384 2024-04-17T10:45:48.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.911+0000 7efc90422000 1 Processor -- start 2024-04-17T10:45:48.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.911+0000 7efc90422000 1 -- start start 2024-04-17T10:45:48.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.911+0000 7efc90422000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569cbf48650 0x5569cbff67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.911+0000 7efc90422000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5569cbf928c0 0x5569cbff6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.911+0000 7efc90422000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cbf931e0 0x5569cbff7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.911+0000 7efc90422000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8806cf80 con 0x5569cbf931e0 2024-04-17T10:45:48.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.911+0000 7efc90422000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8806c160 con 0x5569cbf48650 2024-04-17T10:45:48.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.911+0000 7efc90422000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc8814fd30 con 0x5569cbf928c0 2024-04-17T10:45:48.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.911+0000 7efc877fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cbf931e0 0x5569cbff7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.911+0000 7efc877fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cbf931e0 0x5569cbff7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56794/0 (socket says 172.21.15.83:56794) 2024-04-17T10:45:48.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.911+0000 7efc877fe640 1 -- 172.21.15.83:0/2231658546 learned_addr learned my addr 172.21.15.83:0/2231658546 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:45:48.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.911+0000 7efc86ffd640 1 --2- 172.21.15.83:0/2231658546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569cbf48650 0x5569cbff67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.911+0000 7efc877fe640 1 -- 172.21.15.83:0/2231658546 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5569cbf928c0 msgr2=0x5569cbff6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:45:48.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.911+0000 7efc877fe640 1 --2- 172.21.15.83:0/2231658546 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5569cbf928c0 0x5569cbff6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.911+0000 7efc877fe640 1 -- 172.21.15.83:0/2231658546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569cbf48650 msgr2=0x5569cbff67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:45:48.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.911+0000 7efc877fe640 1 --2- 172.21.15.83:0/2231658546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569cbf48650 0x5569cbff67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.911+0000 7efc877fe640 1 -- 172.21.15.83:0/2231658546 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569cbe1c6d0 con 0x5569cbf931e0 2024-04-17T10:45:48.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.911+0000 7efc86ffd640 1 --2- 172.21.15.83:0/2231658546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569cbf48650 0x5569cbff67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:45:48.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.911+0000 7efc877fe640 1 --2- 172.21.15.83:0/2231658546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cbf931e0 0x5569cbff7270 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7efc80010970 tx=0x7efc80010e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31cda8a922c1f23c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:45:48.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.911+0000 7efc67fff640 1 -- 172.21.15.83:0/2231658546 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc80019030 con 0x5569cbf931e0 2024-04-17T10:45:48.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.911+0000 7efc67fff640 1 -- 172.21.15.83:0/2231658546 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc80012b00 con 0x5569cbf931e0 2024-04-17T10:45:48.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.911+0000 7efc67fff640 1 -- 172.21.15.83:0/2231658546 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc80012cf0 con 0x5569cbf931e0 2024-04-17T10:45:48.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.911+0000 7efc90422000 1 -- 172.21.15.83:0/2231658546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cbf931e0 msgr2=0x5569cbff7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:45:48.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.911+0000 7efc90422000 1 --2- 172.21.15.83:0/2231658546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cbf931e0 0x5569cbff7270 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7efc80010970 tx=0x7efc80010e30 comp rx=0 tx=0).stop 2024-04-17T10:45:48.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.911+0000 7efc90422000 1 -- 172.21.15.83:0/2231658546 shutdown_connections 2024-04-17T10:45:48.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.911+0000 7efc90422000 1 --2- 172.21.15.83:0/2231658546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cbf931e0 0x5569cbff7270 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.911+0000 7efc90422000 1 --2- 172.21.15.83:0/2231658546 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5569cbf928c0 0x5569cbff6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.911+0000 7efc90422000 1 --2- 172.21.15.83:0/2231658546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569cbf48650 0x5569cbff67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.911+0000 7efc90422000 1 -- 172.21.15.83:0/2231658546 >> 172.21.15.83:0/2231658546 conn(0x5569cbe30870 msgr2=0x5569cbf48a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:45:48.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.911+0000 7efc90422000 1 -- 172.21.15.83:0/2231658546 shutdown_connections 2024-04-17T10:45:48.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.911+0000 7efc90422000 1 -- 172.21.15.83:0/2231658546 wait complete. 2024-04-17T10:45:48.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc90422000 1 -- start start 2024-04-17T10:45:48.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc90422000 1 Processor -- start 2024-04-17T10:45:48.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc90422000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cbf48650 0x5569cbffaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc90422000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569cbf928c0 0x5569cbffb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc90422000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5569cbf931e0 0x5569cbff7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc90422000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569cbe19470 con 0x5569cbf48650 2024-04-17T10:45:48.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc90422000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc88150330 con 0x5569cbf928c0 2024-04-17T10:45:48.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc90422000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5569cbd9b400 con 0x5569cbf931e0 2024-04-17T10:45:48.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc90422000 10 client.0 _ll_register_callbacks cb 0x5569cbd33200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:45:48.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc86ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cbf48650 0x5569cbffaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc86ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cbf48650 0x5569cbffaf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56798/0 (socket says 172.21.15.83:56798) 2024-04-17T10:45:48.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc86ffd640 1 -- 172.21.15.83:0/2364712082 learned_addr learned my addr 172.21.15.83:0/2364712082 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:45:48.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc867fc640 1 --2- 172.21.15.83:0/2364712082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569cbf928c0 0x5569cbffb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc86ffd640 1 -- 172.21.15.83:0/2364712082 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5569cbf931e0 msgr2=0x5569cbff7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:45:48.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc86ffd640 1 --2- 172.21.15.83:0/2364712082 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5569cbf931e0 0x5569cbff7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc86ffd640 1 -- 172.21.15.83:0/2364712082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569cbf928c0 msgr2=0x5569cbffb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:45:48.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc86ffd640 1 --2- 172.21.15.83:0/2364712082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569cbf928c0 0x5569cbffb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc86ffd640 1 -- 172.21.15.83:0/2364712082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569cbe19900 con 0x5569cbf48650 2024-04-17T10:45:48.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc867fc640 1 --2- 172.21.15.83:0/2364712082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569cbf928c0 0x5569cbffb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:45:48.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc86ffd640 1 --2- 172.21.15.83:0/2364712082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cbf48650 0x5569cbffaf40 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7efc780a6fd0 tx=0x7efc780a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=920e9dcf32fcec32 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:45:48.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:45:48.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc65ffb640 1 -- 172.21.15.83:0/2364712082 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc780ca9a0 con 0x5569cbf48650 2024-04-17T10:45:48.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc65ffb640 1 -- 172.21.15.83:0/2364712082 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(76..76 src has 1..76) v4 ==== 5767+0+0 (secure 0 0 0) 0x7efc780cab40 con 0x5569cbf48650 2024-04-17T10:45:48.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc90422000 1 -- 172.21.15.83:0/2364712082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc80012b00 con 0x5569cbf48650 2024-04-17T10:45:48.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc65ffb640 1 -- 172.21.15.83:0/2364712082 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc780ca3f0 con 0x5569cbf48650 2024-04-17T10:45:48.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc90422000 10 client.7323 fetch_fsmap learned FSMap version 49 2024-04-17T10:45:48.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc90422000 1 -- 172.21.15.83:0/2364712082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x5569cbd3df20 con 0x5569cbf48650 2024-04-17T10:45:48.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc65ffb640 1 -- 172.21.15.83:0/2364712082 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc80012b00 con 0x5569cbf48650 2024-04-17T10:45:48.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc65ffb640 1 -- 172.21.15.83:0/2364712082 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efc78008040 con 0x5569cbf48650 2024-04-17T10:45:48.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc90422000 10 client.7323 fetch_fsmap finished waiting for FSMap version 49 2024-04-17T10:45:48.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc90422000 10 client.7323 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-17T10:45:48.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc90422000 20 client.7323 populate_metadata read hostname 'smithi083' 2024-04-17T10:45:48.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc90422000 1 --2- 172.21.15.83:0/2364712082 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x5569cbff7f10 0x5569cbff8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:48.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc90422000 4 client.7323 mds_command: new command op to 7104 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}] 2024-04-17T10:45:48.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc90422000 1 -- 172.21.15.83:0/2364712082 --> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] -- command(tid 0: {"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}) v1 -- 0x5569cbda4310 con 0x5569cbff7f10 2024-04-17T10:45:48.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc867fc640 1 --2- 172.21.15.83:0/2364712082 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x5569cbff7f10 0x5569cbff8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:48.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc65ffb640 10 client.7323 ms_handle_connect on v2:172.21.15.167:6832/2796648400 2024-04-17T10:45:48.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.915+0000 7efc867fc640 1 --2- 172.21.15.83:0/2364712082 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x5569cbff7f10 0x5569cbff8320 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac80abcfb521e58 server_cookie=41599d4e3f67ac2d in_seq=0 out_seq=0 2024-04-17T10:45:48.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.939+0000 7efc65ffb640 1 -- 172.21.15.83:0/2364712082 <== mds.0 v2:172.21.15.167:6832/2796648400 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5569cbda4310 con 0x5569cbff7f10 2024-04-17T10:45:48.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.939+0000 7efc65ffb640 10 client.7323 handle_command_reply: tid=0 2024-04-17T10:45:48.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.939+0000 7efc90422000 1 client.7323 shutdown 2024-04-17T10:45:48.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.939+0000 7efc90422000 1 -- 172.21.15.83:0/2364712082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cbf48650 msgr2=0x5569cbffaf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:45:48.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.939+0000 7efc90422000 1 --2- 172.21.15.83:0/2364712082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cbf48650 0x5569cbffaf40 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7efc780a6fd0 tx=0x7efc780a7000 comp rx=0 tx=0).stop 2024-04-17T10:45:48.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.939+0000 7efc90422000 1 -- 172.21.15.83:0/2364712082 shutdown_connections 2024-04-17T10:45:48.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.939+0000 7efc90422000 1 --2- 172.21.15.83:0/2364712082 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x5569cbff7f10 0x5569cbff8320 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.939+0000 7efc90422000 1 --2- 172.21.15.83:0/2364712082 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5569cbf931e0 0x5569cbff7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.939+0000 7efc90422000 1 --2- 172.21.15.83:0/2364712082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569cbf928c0 0x5569cbffb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.939+0000 7efc65ffb640 0 client.7323 ms_handle_reset on v2:172.21.15.167:6832/2796648400 2024-04-17T10:45:48.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.939+0000 7efc90422000 1 --2- 172.21.15.83:0/2364712082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cbf48650 0x5569cbffaf40 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.939+0000 7efc90422000 1 -- 172.21.15.83:0/2364712082 >> 172.21.15.83:0/2364712082 conn(0x5569cbe30870 msgr2=0x5569cbf492c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:45:48.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.939+0000 7efc90422000 1 -- 172.21.15.83:0/2364712082 shutdown_connections 2024-04-17T10:45:48.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.939+0000 7efc90422000 1 -- 172.21.15.83:0/2364712082 wait complete. 2024-04-17T10:45:48.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.939+0000 7efc90422000 20 client.7323 trim_cache size 0 max 16384 2024-04-17T10:45:48.939 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:45:48.940 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-17T10:45:48.940 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "cb3dbd05-49de-4f5f-b21f-5e73796791cc", 2024-04-17T10:45:48.940 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-17T10:45:48.940 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:45:48.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.939+0000 7efc8d2b5640 1 -- 172.21.15.83:0/2822504162 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efc74066f90 msgr2=0x7efc74069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:45:48.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.939+0000 7efc8d2b5640 1 --2- 172.21.15.83:0/2822504162 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efc74066f90 0x7efc74069450 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7efc780ae000 tx=0x7efc780a7070 comp rx=0 tx=0).stop 2024-04-17T10:45:48.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.939+0000 7efc8d2b5640 1 -- 172.21.15.83:0/2822504162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc8815b990 msgr2=0x7efc8813e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:45:48.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.939+0000 7efc8d2b5640 1 --2- 172.21.15.83:0/2822504162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc8815b990 0x7efc8813e260 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00d5f0 tx=0x7efc7c00dab0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.939+0000 7efc8d2b5640 1 -- 172.21.15.83:0/2822504162 shutdown_connections 2024-04-17T10:45:48.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.939+0000 7efc8d2b5640 1 --2- 172.21.15.83:0/2822504162 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efc74066f90 0x7efc74069450 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.939+0000 7efc8d2b5640 1 --2- 172.21.15.83:0/2822504162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc88157390 0x7efc8814eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.939+0000 7efc8d2b5640 1 --2- 172.21.15.83:0/2822504162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc8815b990 0x7efc8813e260 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.939+0000 7efc8d2b5640 1 --2- 172.21.15.83:0/2822504162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc880aebd0 0x7efc8815bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:48.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.939+0000 7efc8d2b5640 1 -- 172.21.15.83:0/2822504162 >> 172.21.15.83:0/2822504162 conn(0x7efc880cc050 msgr2=0x7efc8815ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:45:48.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.939+0000 7efc8d2b5640 1 -- 172.21.15.83:0/2822504162 shutdown_connections 2024-04-17T10:45:48.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:48.939+0000 7efc8d2b5640 1 -- 172.21.15.83:0/2822504162 wait complete. 2024-04-17T10:45:48.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:45:48.954 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-17T10:45:49.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.083+0000 7f1e22f6e640 1 Processor -- start 2024-04-17T10:45:49.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.083+0000 7f1e22f6e640 1 -- start start 2024-04-17T10:45:49.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.083+0000 7f1e22f6e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1e1c0b4210 0x7f1e1c15e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:49.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.083+0000 7f1e22f6e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1c15bd70 con 0x7f1e1c06b520 2024-04-17T10:45:49.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.083+0000 7f1e22f6e640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1c15bf60 con 0x7f1e1c154de0 2024-04-17T10:45:49.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.083+0000 7f1e22f6e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e1c0b4870 con 0x7f1e1c0b4210 2024-04-17T10:45:49.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.083+0000 7f1e1bfff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1e1c06b520 0x7f1e1c0b3b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50914/0 (socket says 172.21.15.83:50914) 2024-04-17T10:45:49.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.083+0000 7f1e1bfff640 1 -- 172.21.15.83:0/1962234357 learned_addr learned my addr 172.21.15.83:0/1962234357 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:45:49.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.083+0000 7f1e2150b640 1 --2- 172.21.15.83:0/1962234357 >> v2:172.21.15.83:3300/0 conn(0x7f1e1c0b4210 0x7f1e1c15e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:49.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.083+0000 7f1e1b7fe640 1 -- 172.21.15.83:0/1962234357 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2854644028 0 0) 0x7f1e1c15bd70 con 0x7f1e1c06b520 2024-04-17T10:45:49.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.083+0000 7f1e1b7fe640 1 -- 172.21.15.83:0/1962234357 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e0c005740 con 0x7f1e1c06b520 2024-04-17T10:45:49.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.083+0000 7f1e2150b640 1 -- 172.21.15.83:0/1962234357 >> v1:172.21.15.167:6789/0 conn(0x7f1e1c154de0 legacy=0x7f1e1c06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:45:49.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.083+0000 7f1e2150b640 1 -- 172.21.15.83:0/1962234357 >> v1:172.21.15.83:6789/0 conn(0x7f1e1c06b520 legacy=0x7f1e1c0b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:45:49.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.083+0000 7f1e2150b640 1 -- 172.21.15.83:0/1962234357 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1c15afb0 con 0x7f1e1c0b4210 2024-04-17T10:45:49.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.087+0000 7f1e2150b640 1 --2- 172.21.15.83:0/1962234357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e1c0b4210 0x7f1e1c15e390 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f1e14009d90 tx=0x7f1e1401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdcb3408586033d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:45:49.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.087+0000 7f1e1b7fe640 1 -- 172.21.15.83:0/1962234357 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e140b2070 con 0x7f1e1c0b4210 2024-04-17T10:45:49.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.087+0000 7f1e1b7fe640 1 -- 172.21.15.83:0/1962234357 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e140a5830 con 0x7f1e1c0b4210 2024-04-17T10:45:49.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.087+0000 7f1e1b7fe640 1 -- 172.21.15.83:0/1962234357 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e140a5be0 con 0x7f1e1c0b4210 2024-04-17T10:45:49.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.087+0000 7f1e22f6e640 1 -- 172.21.15.83:0/1962234357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e1c0b4210 msgr2=0x7f1e1c15e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:45:49.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.087+0000 7f1e22f6e640 1 --2- 172.21.15.83:0/1962234357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e1c0b4210 0x7f1e1c15e390 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f1e14009d90 tx=0x7f1e1401c970 comp rx=0 tx=0).stop 2024-04-17T10:45:49.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.087+0000 7f1e22f6e640 1 -- 172.21.15.83:0/1962234357 shutdown_connections 2024-04-17T10:45:49.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.087+0000 7f1e22f6e640 1 --2- 172.21.15.83:0/1962234357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e1c0b4210 0x7f1e1c15e390 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:49.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.087+0000 7f1e22f6e640 1 -- 172.21.15.83:0/1962234357 >> 172.21.15.83:0/1962234357 conn(0x7f1e1c0b15b0 msgr2=0x7f1e1c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:45:49.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.087+0000 7f1e22f6e640 1 -- 172.21.15.83:0/1962234357 shutdown_connections 2024-04-17T10:45:49.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.087+0000 7f1e22f6e640 1 -- 172.21.15.83:0/1962234357 wait complete. 2024-04-17T10:45:49.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.087+0000 7f1e22f6e640 1 Processor -- start 2024-04-17T10:45:49.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.087+0000 7f1e22f6e640 1 -- start start 2024-04-17T10:45:49.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.087+0000 7f1e22f6e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e1c06b520 0x7f1e1c0a9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:49.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.091+0000 7f1e22f6e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e1c0b4210 0x7f1e1c143010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:49.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.091+0000 7f1e22f6e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e1c154de0 0x7f1e1c0cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:49.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.091+0000 7f1e22f6e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c070470 con 0x7f1e1c06b520 2024-04-17T10:45:49.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.091+0000 7f1e22f6e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c1551c0 con 0x7f1e1c154de0 2024-04-17T10:45:49.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.091+0000 7f1e22f6e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c0705f0 con 0x7f1e1c0b4210 2024-04-17T10:45:49.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.091+0000 7f1e20d0a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e1c06b520 0x7f1e1c0a9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:49.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.091+0000 7f1e20d0a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e1c06b520 0x7f1e1c0a9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56808/0 (socket says 172.21.15.83:56808) 2024-04-17T10:45:49.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.091+0000 7f1e20d0a640 1 -- 172.21.15.83:0/68586791 learned_addr learned my addr 172.21.15.83:0/68586791 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:45:49.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.091+0000 7f1e1bfff640 1 --2- 172.21.15.83:0/68586791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e1c0b4210 0x7f1e1c143010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:49.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.091+0000 7f1e20d0a640 1 -- 172.21.15.83:0/68586791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e1c0b4210 msgr2=0x7f1e1c143010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:45:49.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.091+0000 7f1e20d0a640 1 --2- 172.21.15.83:0/68586791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e1c0b4210 0x7f1e1c143010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:49.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.091+0000 7f1e20d0a640 1 -- 172.21.15.83:0/68586791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e1c154de0 msgr2=0x7f1e1c0cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:45:49.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.091+0000 7f1e20d0a640 1 --2- 172.21.15.83:0/68586791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e1c154de0 0x7f1e1c0cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:49.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.091+0000 7f1e20d0a640 1 -- 172.21.15.83:0/68586791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1c156b30 con 0x7f1e1c06b520 2024-04-17T10:45:49.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.091+0000 7f1e20d0a640 1 --2- 172.21.15.83:0/68586791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e1c06b520 0x7f1e1c0a9e10 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000c370 tx=0x7f1e1000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:45:49.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.091+0000 7f1e197fa640 1 -- 172.21.15.83:0/68586791 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e100052d0 con 0x7f1e1c06b520 2024-04-17T10:45:49.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.091+0000 7f1e197fa640 1 -- 172.21.15.83:0/68586791 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e10005470 con 0x7f1e1c06b520 2024-04-17T10:45:49.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.091+0000 7f1e22f6e640 1 -- 172.21.15.83:0/68586791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e1c156e10 con 0x7f1e1c06b520 2024-04-17T10:45:49.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.091+0000 7f1e197fa640 1 -- 172.21.15.83:0/68586791 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e10007260 con 0x7f1e1c06b520 2024-04-17T10:45:49.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.091+0000 7f1e22f6e640 1 -- 172.21.15.83:0/68586791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e1c1573d0 con 0x7f1e1c06b520 2024-04-17T10:45:49.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.091+0000 7f1e197fa640 1 -- 172.21.15.83:0/68586791 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1e10007400 con 0x7f1e1c06b520 2024-04-17T10:45:49.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.091+0000 7f1e197fa640 1 --2- 172.21.15.83:0/68586791 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1e00066f60 0x7f1e00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:49.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.091+0000 7f1e197fa640 1 -- 172.21.15.83:0/68586791 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1e100843c0 con 0x7f1e1c06b520 2024-04-17T10:45:49.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.091+0000 7f1e1bfff640 1 --2- 172.21.15.83:0/68586791 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1e00066f60 0x7f1e00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:49.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e1bfff640 1 --2- 172.21.15.83:0/68586791 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1e00066f60 0x7f1e00069420 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c1581e0 tx=0x7f1e08007d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:45:49.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e260db000 1 Processor -- start 2024-04-17T10:45:49.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e260db000 1 -- start start 2024-04-17T10:45:49.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e260db000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb52b35e30 0x55cb52b36210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:49.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e260db000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb52b36750 0x55cb52aebf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:49.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e260db000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cb52aec4b0 0x55cb52aec890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:49.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e260db000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb52934ff0 con 0x55cb52b36750 2024-04-17T10:45:49.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e260db000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb528bd150 con 0x55cb52b35e30 2024-04-17T10:45:49.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e260db000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb52931090 con 0x55cb52aec4b0 2024-04-17T10:45:49.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e1bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb52b36750 0x55cb52aebf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:49.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e1bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb52b36750 0x55cb52aebf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56822/0 (socket says 172.21.15.83:56822) 2024-04-17T10:45:49.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e1bfff640 1 -- 172.21.15.83:0/3370032622 learned_addr learned my addr 172.21.15.83:0/3370032622 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:45:49.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e20d0a640 1 --2- 172.21.15.83:0/3370032622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb52b35e30 0x55cb52b36210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:49.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e2150b640 1 --2- 172.21.15.83:0/3370032622 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cb52aec4b0 0x55cb52aec890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:49.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e1bfff640 1 -- 172.21.15.83:0/3370032622 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cb52aec4b0 msgr2=0x55cb52aec890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:45:49.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e1bfff640 1 --2- 172.21.15.83:0/3370032622 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cb52aec4b0 0x55cb52aec890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:49.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e1bfff640 1 -- 172.21.15.83:0/3370032622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb52b35e30 msgr2=0x55cb52b36210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:45:49.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e1bfff640 1 --2- 172.21.15.83:0/3370032622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb52b35e30 0x55cb52b36210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:49.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e1bfff640 1 -- 172.21.15.83:0/3370032622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb52acb0d0 con 0x55cb52b36750 2024-04-17T10:45:49.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e20d0a640 1 --2- 172.21.15.83:0/3370032622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb52b35e30 0x55cb52b36210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:45:49.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e1bfff640 1 --2- 172.21.15.83:0/3370032622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb52b36750 0x55cb52aebf70 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f1e08005be0 tx=0x7f1e08012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea5ef170553b8fd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:45:49.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e067fc640 1 -- 172.21.15.83:0/3370032622 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08026070 con 0x55cb52b36750 2024-04-17T10:45:49.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e067fc640 1 -- 172.21.15.83:0/3370032622 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e080014f0 con 0x55cb52b36750 2024-04-17T10:45:49.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e067fc640 1 -- 172.21.15.83:0/3370032622 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e080018a0 con 0x55cb52b36750 2024-04-17T10:45:49.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e260db000 1 -- 172.21.15.83:0/3370032622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb52b36750 msgr2=0x55cb52aebf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:45:49.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e260db000 1 --2- 172.21.15.83:0/3370032622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb52b36750 0x55cb52aebf70 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f1e08005be0 tx=0x7f1e08012000 comp rx=0 tx=0).stop 2024-04-17T10:45:49.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e260db000 1 -- 172.21.15.83:0/3370032622 shutdown_connections 2024-04-17T10:45:49.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e260db000 1 --2- 172.21.15.83:0/3370032622 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cb52aec4b0 0x55cb52aec890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:49.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e260db000 1 --2- 172.21.15.83:0/3370032622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb52b36750 0x55cb52aebf70 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:49.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e260db000 1 --2- 172.21.15.83:0/3370032622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb52b35e30 0x55cb52b36210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:49.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e260db000 1 -- 172.21.15.83:0/3370032622 >> 172.21.15.83:0/3370032622 conn(0x55cb52ae4020 msgr2=0x55cb52ae6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:45:49.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e260db000 1 -- 172.21.15.83:0/3370032622 shutdown_connections 2024-04-17T10:45:49.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.095+0000 7f1e260db000 1 -- 172.21.15.83:0/3370032622 wait complete. 2024-04-17T10:45:49.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1e260db000 1 -- start start 2024-04-17T10:45:49.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1e260db000 1 Processor -- start 2024-04-17T10:45:49.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1e260db000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb52aec4b0 0x55cb52b35810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:49.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1e260db000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cb52b35e30 0x55cb52af2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:49.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1e260db000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb52b36750 0x55cb52af28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:49.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1e260db000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb52939620 con 0x55cb52b36750 2024-04-17T10:45:49.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1e260db000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb528ef690 con 0x55cb52aec4b0 2024-04-17T10:45:49.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1e260db000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb528bda40 con 0x55cb52b35e30 2024-04-17T10:45:49.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1e260db000 10 client.0 _ll_register_callbacks cb 0x55cb52ad2e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:45:49.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1e2150b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb52b36750 0x55cb52af28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:49.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1e2150b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb52b36750 0x55cb52af28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56824/0 (socket says 172.21.15.83:56824) 2024-04-17T10:45:49.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1e2150b640 1 -- 172.21.15.83:0/1809091946 learned_addr learned my addr 172.21.15.83:0/1809091946 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:45:49.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1e20d0a640 1 --2- 172.21.15.83:0/1809091946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb52aec4b0 0x55cb52b35810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:49.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1e1bfff640 1 --2- 172.21.15.83:0/1809091946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cb52b35e30 0x55cb52af2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:49.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1e2150b640 1 -- 172.21.15.83:0/1809091946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cb52b35e30 msgr2=0x55cb52af2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:45:49.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1e2150b640 1 --2- 172.21.15.83:0/1809091946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cb52b35e30 0x55cb52af2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:49.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1e2150b640 1 -- 172.21.15.83:0/1809091946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb52aec4b0 msgr2=0x55cb52b35810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:45:49.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1e2150b640 1 --2- 172.21.15.83:0/1809091946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb52aec4b0 0x55cb52b35810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:49.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1e2150b640 1 -- 172.21.15.83:0/1809091946 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb529e21c0 con 0x55cb52b36750 2024-04-17T10:45:49.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1e2150b640 1 --2- 172.21.15.83:0/1809091946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb52b36750 0x55cb52af28a0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f1e140b0040 tx=0x7f1e14019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdea9e8374f1ec82 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:45:49.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1dfb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:45:49.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1dfb7fe640 1 -- 172.21.15.83:0/1809091946 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e140bc030 con 0x55cb52b36750 2024-04-17T10:45:49.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1dfb7fe640 1 -- 172.21.15.83:0/1809091946 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(76..76 src has 1..76) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1e140a5630 con 0x55cb52b36750 2024-04-17T10:45:49.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1e260db000 1 -- 172.21.15.83:0/1809091946 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e080014f0 con 0x55cb52b36750 2024-04-17T10:45:49.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1dfb7fe640 1 -- 172.21.15.83:0/1809091946 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e140c8050 con 0x55cb52b36750 2024-04-17T10:45:49.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1e260db000 10 client.7338 fetch_fsmap learned FSMap version 49 2024-04-17T10:45:49.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1e260db000 1 -- 172.21.15.83:0/1809091946 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x55cb529de170 con 0x55cb52b36750 2024-04-17T10:45:49.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1dfb7fe640 1 -- 172.21.15.83:0/1809091946 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e080014f0 con 0x55cb52b36750 2024-04-17T10:45:49.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1dfb7fe640 1 -- 172.21.15.83:0/1809091946 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1e140cb020 con 0x55cb52b36750 2024-04-17T10:45:49.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1e260db000 10 client.7338 fetch_fsmap finished waiting for FSMap version 49 2024-04-17T10:45:49.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1e260db000 10 client.7338 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-17T10:45:49.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1e260db000 20 client.7338 populate_metadata read hostname 'smithi083' 2024-04-17T10:45:49.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1e260db000 1 --2- 172.21.15.83:0/1809091946 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x55cb52af3640 0x55cb52b480f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:49.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1e260db000 4 client.7338 mds_command: new command op to 7104 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:45:49.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1e260db000 1 -- 172.21.15.83:0/1809091946 --> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb528d0c60 con 0x55cb52af3640 2024-04-17T10:45:49.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.099+0000 7f1e20d0a640 1 --2- 172.21.15.83:0/1809091946 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x55cb52af3640 0x55cb52b480f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:49.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.103+0000 7f1e20d0a640 1 --2- 172.21.15.83:0/1809091946 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x55cb52af3640 0x55cb52b480f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c6c69c351764de1 server_cookie=803915264654de0d in_seq=0 out_seq=0 2024-04-17T10:45:49.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.103+0000 7f1dfb7fe640 10 client.7338 ms_handle_connect on v2:172.21.15.167:6832/2796648400 2024-04-17T10:45:49.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.103+0000 7f1dfb7fe640 1 -- 172.21.15.83:0/1809091946 <== mds.0 v2:172.21.15.167:6832/2796648400 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cb528d0c60 con 0x55cb52af3640 2024-04-17T10:45:49.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.103+0000 7f1dfb7fe640 10 client.7338 handle_command_reply: tid=0 2024-04-17T10:45:49.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.103+0000 7f1e260db000 1 client.7338 shutdown 2024-04-17T10:45:49.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.103+0000 7f1e260db000 1 -- 172.21.15.83:0/1809091946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb52b36750 msgr2=0x55cb52af28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:45:49.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.103+0000 7f1e260db000 1 --2- 172.21.15.83:0/1809091946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb52b36750 0x55cb52af28a0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f1e140b0040 tx=0x7f1e14019710 comp rx=0 tx=0).stop 2024-04-17T10:45:49.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.103+0000 7f1e260db000 1 -- 172.21.15.83:0/1809091946 shutdown_connections 2024-04-17T10:45:49.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.103+0000 7f1e260db000 1 --2- 172.21.15.83:0/1809091946 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x55cb52af3640 0x55cb52b480f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:49.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.103+0000 7f1e260db000 1 --2- 172.21.15.83:0/1809091946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb52b36750 0x55cb52af28a0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:49.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.103+0000 7f1e260db000 1 --2- 172.21.15.83:0/1809091946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cb52b35e30 0x55cb52af2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:49.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.103+0000 7f1e260db000 1 --2- 172.21.15.83:0/1809091946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb52aec4b0 0x55cb52b35810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:49.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.103+0000 7f1dfb7fe640 0 client.7338 ms_handle_reset on v2:172.21.15.167:6832/2796648400 2024-04-17T10:45:49.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.103+0000 7f1e260db000 1 -- 172.21.15.83:0/1809091946 >> 172.21.15.83:0/1809091946 conn(0x55cb52ae4020 msgr2=0x55cb52aed120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:45:49.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.103+0000 7f1e260db000 1 -- 172.21.15.83:0/1809091946 shutdown_connections 2024-04-17T10:45:49.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.103+0000 7f1e260db000 1 -- 172.21.15.83:0/1809091946 wait complete. 2024-04-17T10:45:49.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.103+0000 7f1e260db000 20 client.7338 trim_cache size 0 max 16384 2024-04-17T10:45:49.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.119+0000 7f1e260db000 1 Processor -- start 2024-04-17T10:45:49.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.119+0000 7f1e260db000 1 -- start start 2024-04-17T10:45:49.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.123+0000 7f1e260db000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb52aec4b0 0x55cb52b4a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:49.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.123+0000 7f1e260db000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cb52b35e30 0x55cb52b4a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:49.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.123+0000 7f1e260db000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb52b36750 0x55cb52b432a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:49.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.123+0000 7f1e260db000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c1feaf0 con 0x55cb52aec4b0 2024-04-17T10:45:49.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.123+0000 7f1e260db000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c156850 con 0x55cb52b36750 2024-04-17T10:45:49.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.123+0000 7f1e260db000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c155c50 con 0x55cb52b35e30 2024-04-17T10:45:49.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.123+0000 7f1e20d0a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb52aec4b0 0x55cb52b4a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:49.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.123+0000 7f1e20d0a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb52aec4b0 0x55cb52b4a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56830/0 (socket says 172.21.15.83:56830) 2024-04-17T10:45:49.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.123+0000 7f1e20d0a640 1 -- 172.21.15.83:0/3014400948 learned_addr learned my addr 172.21.15.83:0/3014400948 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:45:49.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.123+0000 7f1e2150b640 1 --2- 172.21.15.83:0/3014400948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb52b36750 0x55cb52b432a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:49.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.123+0000 7f1e1bfff640 1 --2- 172.21.15.83:0/3014400948 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cb52b35e30 0x55cb52b4a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:49.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.123+0000 7f1e20d0a640 1 -- 172.21.15.83:0/3014400948 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cb52b35e30 msgr2=0x55cb52b4a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:45:49.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.123+0000 7f1e20d0a640 1 --2- 172.21.15.83:0/3014400948 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cb52b35e30 0x55cb52b4a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:49.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.123+0000 7f1e20d0a640 1 -- 172.21.15.83:0/3014400948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb52b36750 msgr2=0x55cb52b432a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:45:49.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.123+0000 7f1e20d0a640 1 --2- 172.21.15.83:0/3014400948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb52b36750 0x55cb52b432a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:49.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.123+0000 7f1e20d0a640 1 -- 172.21.15.83:0/3014400948 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb5297bde0 con 0x55cb52aec4b0 2024-04-17T10:45:49.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.123+0000 7f1e20d0a640 1 --2- 172.21.15.83:0/3014400948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb52aec4b0 0x55cb52b4a110 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f1e10082940 tx=0x7f1e1005b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49190ea773bf447f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:45:49.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.123+0000 7f1e05ffb640 1 -- 172.21.15.83:0/3014400948 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e100530d0 con 0x55cb52aec4b0 2024-04-17T10:45:49.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.123+0000 7f1e05ffb640 1 -- 172.21.15.83:0/3014400948 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e100840c0 con 0x55cb52aec4b0 2024-04-17T10:45:49.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.123+0000 7f1e05ffb640 1 -- 172.21.15.83:0/3014400948 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e10054220 con 0x55cb52aec4b0 2024-04-17T10:45:49.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.123+0000 7f1e260db000 1 -- 172.21.15.83:0/3014400948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb52aec4b0 msgr2=0x55cb52b4a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:45:49.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.123+0000 7f1e260db000 1 --2- 172.21.15.83:0/3014400948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb52aec4b0 0x55cb52b4a110 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f1e10082940 tx=0x7f1e1005b040 comp rx=0 tx=0).stop 2024-04-17T10:45:49.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.123+0000 7f1e260db000 1 -- 172.21.15.83:0/3014400948 shutdown_connections 2024-04-17T10:45:49.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.123+0000 7f1e260db000 1 --2- 172.21.15.83:0/3014400948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb52b36750 0x55cb52b432a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:49.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.123+0000 7f1e260db000 1 --2- 172.21.15.83:0/3014400948 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cb52b35e30 0x55cb52b4a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:49.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.123+0000 7f1e260db000 1 --2- 172.21.15.83:0/3014400948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb52aec4b0 0x55cb52b4a110 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:49.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.123+0000 7f1e260db000 1 -- 172.21.15.83:0/3014400948 >> 172.21.15.83:0/3014400948 conn(0x55cb52ae4020 msgr2=0x55cb52b42b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:45:49.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.123+0000 7f1e260db000 1 -- 172.21.15.83:0/3014400948 shutdown_connections 2024-04-17T10:45:49.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.123+0000 7f1e260db000 1 -- 172.21.15.83:0/3014400948 wait complete. 2024-04-17T10:45:49.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.123+0000 7f1e260db000 1 -- start start 2024-04-17T10:45:49.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.123+0000 7f1e260db000 1 Processor -- start 2024-04-17T10:45:49.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e260db000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb52aec4b0 0x55cb52b482d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:49.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e260db000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb52b35e30 0x55cb52b48810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:49.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e260db000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cb52b36750 0x55cb52b48d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:49.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e260db000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb528cfe00 con 0x55cb52b35e30 2024-04-17T10:45:49.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e260db000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c156250 con 0x55cb52aec4b0 2024-04-17T10:45:49.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e260db000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb52934300 con 0x55cb52b36750 2024-04-17T10:45:49.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e260db000 10 client.0 _ll_register_callbacks cb 0x55cb52ab3040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:45:49.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e1bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb52b35e30 0x55cb52b48810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:49.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e1bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb52b35e30 0x55cb52b48810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56838/0 (socket says 172.21.15.83:56838) 2024-04-17T10:45:49.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e1bfff640 1 -- 172.21.15.83:0/439104075 learned_addr learned my addr 172.21.15.83:0/439104075 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:45:49.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e20d0a640 1 --2- 172.21.15.83:0/439104075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb52aec4b0 0x55cb52b482d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:49.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e2150b640 1 --2- 172.21.15.83:0/439104075 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cb52b36750 0x55cb52b48d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:49.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e1bfff640 1 -- 172.21.15.83:0/439104075 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cb52b36750 msgr2=0x55cb52b48d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:45:49.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e1bfff640 1 --2- 172.21.15.83:0/439104075 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cb52b36750 0x55cb52b48d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:49.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e1bfff640 1 -- 172.21.15.83:0/439104075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb52aec4b0 msgr2=0x55cb52b482d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:45:49.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e1bfff640 1 --2- 172.21.15.83:0/439104075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb52aec4b0 0x55cb52b482d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:49.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e1bfff640 1 -- 172.21.15.83:0/439104075 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb528d68e0 con 0x55cb52b35e30 2024-04-17T10:45:49.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e1bfff640 1 --2- 172.21.15.83:0/439104075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb52b35e30 0x55cb52b48810 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f1e08024040 tx=0x7f1e08000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=945c99a34565777d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:45:49.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1dfb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:45:49.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1dfb7fe640 1 -- 172.21.15.83:0/439104075 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08026040 con 0x55cb52b35e30 2024-04-17T10:45:49.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1dfb7fe640 1 -- 172.21.15.83:0/439104075 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(76..76 src has 1..76) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1e08020630 con 0x55cb52b35e30 2024-04-17T10:45:49.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e260db000 1 -- 172.21.15.83:0/439104075 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e100840c0 con 0x55cb52b35e30 2024-04-17T10:45:49.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1dfb7fe640 1 -- 172.21.15.83:0/439104075 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e0803c050 con 0x55cb52b35e30 2024-04-17T10:45:49.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e260db000 10 client.7344 fetch_fsmap learned FSMap version 49 2024-04-17T10:45:49.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e260db000 1 -- 172.21.15.83:0/439104075 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x55cb5298e8f0 con 0x55cb52b35e30 2024-04-17T10:45:49.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1dfb7fe640 1 -- 172.21.15.83:0/439104075 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e08027040 con 0x55cb52b35e30 2024-04-17T10:45:49.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1dfb7fe640 1 -- 172.21.15.83:0/439104075 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1e0803f020 con 0x55cb52b35e30 2024-04-17T10:45:49.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e260db000 10 client.7344 fetch_fsmap finished waiting for FSMap version 49 2024-04-17T10:45:49.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e260db000 10 client.7344 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-17T10:45:49.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e260db000 20 client.7344 populate_metadata read hostname 'smithi083' 2024-04-17T10:45:49.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e260db000 1 --2- 172.21.15.83:0/439104075 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x55cb52b49850 0x55cb52b9dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:45:49.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e260db000 4 client.7344 mds_command: new command op to 7104 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T10:45:49.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e260db000 1 -- 172.21.15.83:0/439104075 --> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f1e100840c0 con 0x55cb52b49850 2024-04-17T10:45:49.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e20d0a640 1 --2- 172.21.15.83:0/439104075 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x55cb52b49850 0x55cb52b9dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:45:49.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1dfb7fe640 10 client.7344 ms_handle_connect on v2:172.21.15.167:6832/2796648400 2024-04-17T10:45:49.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e20d0a640 1 --2- 172.21.15.83:0/439104075 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x55cb52b49850 0x55cb52b9dfe0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=240a30bda1e3b78b server_cookie=425f0a7d51d849cf in_seq=0 out_seq=0 2024-04-17T10:45:49.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1dfb7fe640 1 -- 172.21.15.83:0/439104075 <== mds.0 v2:172.21.15.167:6832/2796648400 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x7f1e100840c0 con 0x55cb52b49850 2024-04-17T10:45:49.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1dfb7fe640 10 client.7344 handle_command_reply: tid=0 2024-04-17T10:45:49.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e260db000 1 client.7344 shutdown 2024-04-17T10:45:49.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e260db000 1 -- 172.21.15.83:0/439104075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb52b35e30 msgr2=0x55cb52b48810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:45:49.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e260db000 1 --2- 172.21.15.83:0/439104075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb52b35e30 0x55cb52b48810 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f1e08024040 tx=0x7f1e08000ee0 comp rx=0 tx=0).stop 2024-04-17T10:45:49.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e260db000 1 -- 172.21.15.83:0/439104075 shutdown_connections 2024-04-17T10:45:49.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e260db000 1 --2- 172.21.15.83:0/439104075 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x55cb52b49850 0x55cb52b9dfe0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:49.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e260db000 1 --2- 172.21.15.83:0/439104075 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55cb52b36750 0x55cb52b48d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:49.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e260db000 1 --2- 172.21.15.83:0/439104075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb52b35e30 0x55cb52b48810 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:49.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e260db000 1 --2- 172.21.15.83:0/439104075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb52aec4b0 0x55cb52b482d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:49.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e260db000 1 -- 172.21.15.83:0/439104075 >> 172.21.15.83:0/439104075 conn(0x55cb52ae4020 msgr2=0x55cb52b43420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:45:49.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1dfb7fe640 0 client.7344 ms_handle_reset on v2:172.21.15.167:6832/2796648400 2024-04-17T10:45:49.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e260db000 1 -- 172.21.15.83:0/439104075 shutdown_connections 2024-04-17T10:45:49.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e260db000 1 -- 172.21.15.83:0/439104075 wait complete. 2024-04-17T10:45:49.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.127+0000 7f1e260db000 20 client.7344 trim_cache size 0 max 16384 2024-04-17T10:45:49.130 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:45:49.131 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "scrub active (1004 inodes in the stack)", 2024-04-17T10:45:49.131 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-17T10:45:49.131 INFO:teuthology.orchestra.run.smithi083.stdout: "cb3dbd05-49de-4f5f-b21f-5e73796791cc": { 2024-04-17T10:45:49.131 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "~mds0", 2024-04-17T10:45:49.131 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "cb3dbd05-49de-4f5f-b21f-5e73796791cc", 2024-04-17T10:45:49.131 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive" 2024-04-17T10:45:49.131 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:45:49.131 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:45:49.131 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:45:49.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.131+0000 7f1e22f6e640 1 -- 172.21.15.83:0/68586791 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1e00066f60 msgr2=0x7f1e00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:45:49.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.131+0000 7f1e22f6e640 1 --2- 172.21.15.83:0/68586791 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1e00066f60 0x7f1e00069420 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c1581e0 tx=0x7f1e08007d40 comp rx=0 tx=0).stop 2024-04-17T10:45:49.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.131+0000 7f1e22f6e640 1 -- 172.21.15.83:0/68586791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e1c06b520 msgr2=0x7f1e1c0a9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:45:49.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.131+0000 7f1e22f6e640 1 --2- 172.21.15.83:0/68586791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e1c06b520 0x7f1e1c0a9e10 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000c370 tx=0x7f1e1000c830 comp rx=0 tx=0).stop 2024-04-17T10:45:49.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.131+0000 7f1e22f6e640 1 -- 172.21.15.83:0/68586791 shutdown_connections 2024-04-17T10:45:49.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.131+0000 7f1e22f6e640 1 --2- 172.21.15.83:0/68586791 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1e00066f60 0x7f1e00069420 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:49.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.131+0000 7f1e22f6e640 1 --2- 172.21.15.83:0/68586791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e1c154de0 0x7f1e1c0cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:49.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.131+0000 7f1e22f6e640 1 --2- 172.21.15.83:0/68586791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e1c0b4210 0x7f1e1c143010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:49.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.131+0000 7f1e22f6e640 1 --2- 172.21.15.83:0/68586791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e1c06b520 0x7f1e1c0a9e10 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:45:49.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.131+0000 7f1e22f6e640 1 -- 172.21.15.83:0/68586791 >> 172.21.15.83:0/68586791 conn(0x7f1e1c0b15b0 msgr2=0x7f1e1c0b05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:45:49.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.131+0000 7f1e22f6e640 1 -- 172.21.15.83:0/68586791 shutdown_connections 2024-04-17T10:45:49.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:45:49.131+0000 7f1e22f6e640 1 -- 172.21.15.83:0/68586791 wait complete. 2024-04-17T10:45:49.144 INFO:tasks.cephfs.filesystem:scrub status for tag:cb3dbd05-49de-4f5f-b21f-5e73796791cc - {'path': '~mds0', 'tag': 'cb3dbd05-49de-4f5f-b21f-5e73796791cc', 'options': 'recursive'} 2024-04-17T10:46:07.919 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:46:07.923 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:46:19.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:19.146 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-17T10:46:19.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.283+0000 7fcacf09c640 1 Processor -- start 2024-04-17T10:46:19.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.283+0000 7fcacf09c640 1 -- start start 2024-04-17T10:46:19.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.283+0000 7fcacf09c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcac815b6e0 0x7fcac8071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.283+0000 7fcacf09c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcac815bac0 0x7fcac80cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.283+0000 7fcacf09c640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcac806c1f0 con 0x7fcac8070470 2024-04-17T10:46:19.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.283+0000 7fcacf09c640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcac80cdd30 con 0x7fcac815b6e0 2024-04-17T10:46:19.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.283+0000 7fcacf09c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac8070a10 con 0x7fcac815bac0 2024-04-17T10:46:19.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.283+0000 7fcacd639640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcac8070470 0x7fcac806ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:44084/0 (socket says 172.21.15.83:44084) 2024-04-17T10:46:19.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.283+0000 7fcacd639640 1 -- 172.21.15.83:0/2409304695 learned_addr learned my addr 172.21.15.83:0/2409304695 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:46:19.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.283+0000 7fcacce38640 1 --2- 172.21.15.83:0/2409304695 >> v2:172.21.15.83:3300/0 conn(0x7fcac815b6e0 0x7fcac8071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.287+0000 7fcac77fe640 1 -- 172.21.15.83:0/2409304695 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1176793019 0 0) 0x7fcac806c1f0 con 0x7fcac8070470 2024-04-17T10:46:19.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.287+0000 7fcac77fe640 1 -- 172.21.15.83:0/2409304695 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcab0003530 con 0x7fcac8070470 2024-04-17T10:46:19.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.287+0000 7fcac7fff640 1 --2- 172.21.15.83:0/2409304695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcac815bac0 0x7fcac80cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.287+0000 7fcacce38640 1 -- 172.21.15.83:0/2409304695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcac815bac0 msgr2=0x7fcac80cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:19.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.287+0000 7fcacce38640 1 --2- 172.21.15.83:0/2409304695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcac815bac0 0x7fcac80cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.287+0000 7fcacce38640 1 -- 172.21.15.83:0/2409304695 >> v1:172.21.15.83:6789/0 conn(0x7fcac8070470 legacy=0x7fcac806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:19.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.287+0000 7fcacce38640 1 -- 172.21.15.83:0/2409304695 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac8070cf0 con 0x7fcac815b6e0 2024-04-17T10:46:19.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.287+0000 7fcacce38640 1 --2- 172.21.15.83:0/2409304695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcac815b6e0 0x7fcac8071cc0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fcab80097c0 tx=0x7fcab801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6cb68a6cd5c2be8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:19.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.287+0000 7fcac77fe640 1 -- 172.21.15.83:0/2409304695 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab80b0070 con 0x7fcac815b6e0 2024-04-17T10:46:19.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.287+0000 7fcac77fe640 1 -- 172.21.15.83:0/2409304695 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab80ab600 con 0x7fcac815b6e0 2024-04-17T10:46:19.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.287+0000 7fcac77fe640 1 -- 172.21.15.83:0/2409304695 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab80ab9d0 con 0x7fcac815b6e0 2024-04-17T10:46:19.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.287+0000 7fcacf09c640 1 -- 172.21.15.83:0/2409304695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcac815b6e0 msgr2=0x7fcac8071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:19.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.287+0000 7fcacf09c640 1 --2- 172.21.15.83:0/2409304695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcac815b6e0 0x7fcac8071cc0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fcab80097c0 tx=0x7fcab801c4a0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.287+0000 7fcacf09c640 1 -- 172.21.15.83:0/2409304695 shutdown_connections 2024-04-17T10:46:19.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.287+0000 7fcacf09c640 1 --2- 172.21.15.83:0/2409304695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcac815bac0 0x7fcac80cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.287+0000 7fcacf09c640 1 --2- 172.21.15.83:0/2409304695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcac815b6e0 0x7fcac8071cc0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.287+0000 7fcacf09c640 1 -- 172.21.15.83:0/2409304695 >> 172.21.15.83:0/2409304695 conn(0x7fcac80b15b0 msgr2=0x7fcac80b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:19.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.287+0000 7fcacf09c640 1 -- 172.21.15.83:0/2409304695 shutdown_connections 2024-04-17T10:46:19.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.287+0000 7fcacf09c640 1 -- 172.21.15.83:0/2409304695 wait complete. 2024-04-17T10:46:19.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.291+0000 7fcacf09c640 1 Processor -- start 2024-04-17T10:46:19.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.291+0000 7fcacf09c640 1 -- start start 2024-04-17T10:46:19.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.291+0000 7fcacf09c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcac8070470 0x7fcac8148a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.291+0000 7fcacf09c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcac815b6e0 0x7fcac815dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.291+0000 7fcacf09c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcac815bac0 0x7fcac8146020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.291+0000 7fcacf09c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac806cee0 con 0x7fcac815b6e0 2024-04-17T10:46:19.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.291+0000 7fcacf09c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac806b730 con 0x7fcac815bac0 2024-04-17T10:46:19.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.291+0000 7fcacf09c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac806d060 con 0x7fcac8070470 2024-04-17T10:46:19.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.291+0000 7fcac7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcac815b6e0 0x7fcac815dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.291+0000 7fcac7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcac815b6e0 0x7fcac815dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52174/0 (socket says 172.21.15.83:52174) 2024-04-17T10:46:19.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.291+0000 7fcac7fff640 1 -- 172.21.15.83:0/2582443092 learned_addr learned my addr 172.21.15.83:0/2582443092 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:19.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.291+0000 7fcac7fff640 1 -- 172.21.15.83:0/2582443092 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcac8070470 msgr2=0x7fcac8148a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:19.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.291+0000 7fcac7fff640 1 --2- 172.21.15.83:0/2582443092 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcac8070470 0x7fcac8148a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.291+0000 7fcac7fff640 1 -- 172.21.15.83:0/2582443092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcac815bac0 msgr2=0x7fcac8146020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:19.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.291+0000 7fcac7fff640 1 --2- 172.21.15.83:0/2582443092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcac815bac0 0x7fcac8146020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.291+0000 7fcac7fff640 1 -- 172.21.15.83:0/2582443092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac8149540 con 0x7fcac815b6e0 2024-04-17T10:46:19.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.291+0000 7fcac7fff640 1 --2- 172.21.15.83:0/2582443092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcac815b6e0 0x7fcac815dd30 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fcabc00d600 tx=0x7fcabc00dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:19.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.291+0000 7fcac57fa640 1 -- 172.21.15.83:0/2582443092 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcabc004240 con 0x7fcac815b6e0 2024-04-17T10:46:19.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.291+0000 7fcac57fa640 1 -- 172.21.15.83:0/2582443092 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcabc0043e0 con 0x7fcac815b6e0 2024-04-17T10:46:19.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.291+0000 7fcacf09c640 1 -- 172.21.15.83:0/2582443092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac8149820 con 0x7fcac815b6e0 2024-04-17T10:46:19.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.291+0000 7fcac57fa640 1 -- 172.21.15.83:0/2582443092 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcabc003840 con 0x7fcac815b6e0 2024-04-17T10:46:19.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.291+0000 7fcacf09c640 1 -- 172.21.15.83:0/2582443092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac815d1b0 con 0x7fcac815b6e0 2024-04-17T10:46:19.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.291+0000 7fcac57fa640 1 -- 172.21.15.83:0/2582443092 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcabc0039e0 con 0x7fcac815b6e0 2024-04-17T10:46:19.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.291+0000 7fcac57fa640 1 --2- 172.21.15.83:0/2582443092 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcab4066f60 0x7fcab4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcac57fa640 1 -- 172.21.15.83:0/2582443092 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fcabc085350 con 0x7fcac815b6e0 2024-04-17T10:46:19.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcacce38640 1 --2- 172.21.15.83:0/2582443092 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcab4066f60 0x7fcab4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcacce38640 1 --2- 172.21.15.83:0/2582443092 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcab4066f60 0x7fcab4069420 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcab80ad000 tx=0x7fcab8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:19.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcad2209000 1 Processor -- start 2024-04-17T10:46:19.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcad2209000 1 -- start start 2024-04-17T10:46:19.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcad2209000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5632d5e159d0 0x5632d5e15db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcad2209000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d5e162f0 0x5632d5dcbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcad2209000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632d5dcc0a0 0x5632d5dcc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcad2209000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d5c18ff0 con 0x5632d5e162f0 2024-04-17T10:46:19.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcad2209000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d5ba1150 con 0x5632d5dcc0a0 2024-04-17T10:46:19.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcad2209000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632d5c15090 con 0x5632d5e159d0 2024-04-17T10:46:19.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcac7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d5e162f0 0x5632d5dcbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcac7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d5e162f0 0x5632d5dcbb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52182/0 (socket says 172.21.15.83:52182) 2024-04-17T10:46:19.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcac7fff640 1 -- 172.21.15.83:0/4128131089 learned_addr learned my addr 172.21.15.83:0/4128131089 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:19.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcac7fff640 1 -- 172.21.15.83:0/4128131089 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5632d5e159d0 msgr2=0x5632d5e15db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:19.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcac7fff640 1 --2- 172.21.15.83:0/4128131089 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5632d5e159d0 0x5632d5e15db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcac7fff640 1 -- 172.21.15.83:0/4128131089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632d5dcc0a0 msgr2=0x5632d5dcc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:19.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcac7fff640 1 --2- 172.21.15.83:0/4128131089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632d5dcc0a0 0x5632d5dcc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcac7fff640 1 -- 172.21.15.83:0/4128131089 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632d5daf0d0 con 0x5632d5e162f0 2024-04-17T10:46:19.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcac7fff640 1 --2- 172.21.15.83:0/4128131089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d5e162f0 0x5632d5dcbb60 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fcabc083820 tx=0x7fcabc055620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a9d0625cb89f868 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:19.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcaae7fc640 1 -- 172.21.15.83:0/4128131089 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcabc055c00 con 0x5632d5e162f0 2024-04-17T10:46:19.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcaae7fc640 1 -- 172.21.15.83:0/4128131089 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcabc07e020 con 0x5632d5e162f0 2024-04-17T10:46:19.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcaae7fc640 1 -- 172.21.15.83:0/4128131089 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcabc059d50 con 0x5632d5e162f0 2024-04-17T10:46:19.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcad2209000 1 -- 172.21.15.83:0/4128131089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d5e162f0 msgr2=0x5632d5dcbb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:19.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcad2209000 1 --2- 172.21.15.83:0/4128131089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d5e162f0 0x5632d5dcbb60 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fcabc083820 tx=0x7fcabc055620 comp rx=0 tx=0).stop 2024-04-17T10:46:19.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcad2209000 1 -- 172.21.15.83:0/4128131089 shutdown_connections 2024-04-17T10:46:19.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcad2209000 1 --2- 172.21.15.83:0/4128131089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632d5dcc0a0 0x5632d5dcc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcad2209000 1 --2- 172.21.15.83:0/4128131089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d5e162f0 0x5632d5dcbb60 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcad2209000 1 --2- 172.21.15.83:0/4128131089 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5632d5e159d0 0x5632d5e15db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcad2209000 1 -- 172.21.15.83:0/4128131089 >> 172.21.15.83:0/4128131089 conn(0x5632d5dc8020 msgr2=0x5632d5dca440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:19.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcad2209000 1 -- 172.21.15.83:0/4128131089 shutdown_connections 2024-04-17T10:46:19.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcad2209000 1 -- 172.21.15.83:0/4128131089 wait complete. 2024-04-17T10:46:19.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcad2209000 1 -- start start 2024-04-17T10:46:19.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.295+0000 7fcad2209000 1 Processor -- start 2024-04-17T10:46:19.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcad2209000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632d5dcc0a0 0x5632d5dfa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcad2209000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d5e159d0 0x5632d5dfad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcad2209000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5632d5e162f0 0x5632d5dfb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcad2209000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d5bdd250 con 0x5632d5e159d0 2024-04-17T10:46:19.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcad2209000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d5bd3690 con 0x5632d5dcc0a0 2024-04-17T10:46:19.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcad2209000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632d5ba1a40 con 0x5632d5e162f0 2024-04-17T10:46:19.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcad2209000 10 client.0 _ll_register_callbacks cb 0x5632d5db6e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:46:19.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcac7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d5e159d0 0x5632d5dfad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcac7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d5e159d0 0x5632d5dfad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52186/0 (socket says 172.21.15.83:52186) 2024-04-17T10:46:19.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcac7fff640 1 -- 172.21.15.83:0/3269191262 learned_addr learned my addr 172.21.15.83:0/3269191262 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:19.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcacce38640 1 --2- 172.21.15.83:0/3269191262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632d5dcc0a0 0x5632d5dfa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcacd639640 1 --2- 172.21.15.83:0/3269191262 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5632d5e162f0 0x5632d5dfb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcac7fff640 1 -- 172.21.15.83:0/3269191262 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5632d5e162f0 msgr2=0x5632d5dfb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:19.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcac7fff640 1 --2- 172.21.15.83:0/3269191262 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5632d5e162f0 0x5632d5dfb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcac7fff640 1 -- 172.21.15.83:0/3269191262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632d5dcc0a0 msgr2=0x5632d5dfa7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:19.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcac7fff640 1 --2- 172.21.15.83:0/3269191262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632d5dcc0a0 0x5632d5dfa7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcac7fff640 1 -- 172.21.15.83:0/3269191262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632d5cc61c0 con 0x5632d5e159d0 2024-04-17T10:46:19.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcacd639640 1 --2- 172.21.15.83:0/3269191262 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5632d5e162f0 0x5632d5dfb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:19.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcac7fff640 1 --2- 172.21.15.83:0/3269191262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d5e159d0 0x5632d5dfad30 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fcabc055b90 tx=0x7fcabc000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a981545431701a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:19.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcaa77fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:46:19.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcaa77fe640 1 -- 172.21.15.83:0/3269191262 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcabc073070 con 0x5632d5e159d0 2024-04-17T10:46:19.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcad2209000 1 -- 172.21.15.83:0/3269191262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcabc07e020 con 0x5632d5e159d0 2024-04-17T10:46:19.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcaa77fe640 1 -- 172.21.15.83:0/3269191262 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fcabc0850b0 con 0x5632d5e159d0 2024-04-17T10:46:19.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcaa77fe640 1 -- 172.21.15.83:0/3269191262 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcabc055c00 con 0x5632d5e159d0 2024-04-17T10:46:19.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcad2209000 10 client.7359 fetch_fsmap learned FSMap version 49 2024-04-17T10:46:19.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcad2209000 1 -- 172.21.15.83:0/3269191262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x5632d5cc2170 con 0x5632d5e159d0 2024-04-17T10:46:19.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcaa77fe640 1 -- 172.21.15.83:0/3269191262 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcabc07e020 con 0x5632d5e159d0 2024-04-17T10:46:19.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcaa77fe640 1 -- 172.21.15.83:0/3269191262 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcabc013070 con 0x5632d5e159d0 2024-04-17T10:46:19.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcad2209000 10 client.7359 fetch_fsmap finished waiting for FSMap version 49 2024-04-17T10:46:19.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcad2209000 10 client.7359 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-17T10:46:19.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcad2209000 20 client.7359 populate_metadata read hostname 'smithi083' 2024-04-17T10:46:19.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcad2209000 1 --2- 172.21.15.83:0/3269191262 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x5632d5e25c80 0x5632d5e28070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcad2209000 4 client.7359 mds_command: new command op to 7104 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:46:19.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcad2209000 1 -- 172.21.15.83:0/3269191262 --> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5632d5bb4c60 con 0x5632d5e25c80 2024-04-17T10:46:19.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcacce38640 1 --2- 172.21.15.83:0/3269191262 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x5632d5e25c80 0x5632d5e28070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcaa77fe640 10 client.7359 ms_handle_connect on v2:172.21.15.167:6832/2796648400 2024-04-17T10:46:19.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.299+0000 7fcacce38640 1 --2- 172.21.15.83:0/3269191262 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x5632d5e25c80 0x5632d5e28070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ca7ad107608cd7f server_cookie=308cbf068e70407a in_seq=0 out_seq=0 2024-04-17T10:46:19.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.303+0000 7fcaa77fe640 1 -- 172.21.15.83:0/3269191262 <== mds.0 v2:172.21.15.167:6832/2796648400 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5632d5bb4c60 con 0x5632d5e25c80 2024-04-17T10:46:19.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.303+0000 7fcaa77fe640 10 client.7359 handle_command_reply: tid=0 2024-04-17T10:46:19.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.303+0000 7fcad2209000 1 client.7359 shutdown 2024-04-17T10:46:19.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.303+0000 7fcad2209000 1 -- 172.21.15.83:0/3269191262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d5e159d0 msgr2=0x5632d5dfad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:19.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.303+0000 7fcad2209000 1 --2- 172.21.15.83:0/3269191262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d5e159d0 0x5632d5dfad30 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fcabc055b90 tx=0x7fcabc000bd0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.303+0000 7fcad2209000 1 -- 172.21.15.83:0/3269191262 shutdown_connections 2024-04-17T10:46:19.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.303+0000 7fcad2209000 1 --2- 172.21.15.83:0/3269191262 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x5632d5e25c80 0x5632d5e28070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.303+0000 7fcad2209000 1 --2- 172.21.15.83:0/3269191262 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5632d5e162f0 0x5632d5dfb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.303+0000 7fcad2209000 1 --2- 172.21.15.83:0/3269191262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d5e159d0 0x5632d5dfad30 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.303+0000 7fcaa77fe640 0 client.7359 ms_handle_reset on v2:172.21.15.167:6832/2796648400 2024-04-17T10:46:19.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.303+0000 7fcad2209000 1 --2- 172.21.15.83:0/3269191262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632d5dcc0a0 0x5632d5dfa7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.303+0000 7fcad2209000 1 -- 172.21.15.83:0/3269191262 >> 172.21.15.83:0/3269191262 conn(0x5632d5dc8020 msgr2=0x5632d5dcaca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:19.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.303+0000 7fcad2209000 1 -- 172.21.15.83:0/3269191262 shutdown_connections 2024-04-17T10:46:19.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.303+0000 7fcad2209000 1 -- 172.21.15.83:0/3269191262 wait complete. 2024-04-17T10:46:19.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.303+0000 7fcad2209000 20 client.7359 trim_cache size 0 max 16384 2024-04-17T10:46:19.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.315+0000 7fcad2209000 1 Processor -- start 2024-04-17T10:46:19.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.315+0000 7fcad2209000 1 -- start start 2024-04-17T10:46:19.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcad2209000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5632d5dcc0a0 0x5632d5dca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcad2209000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d5e159d0 0x5632d5dca950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcad2209000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632d5e162f0 0x5632d5dcae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcad2209000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac82009c0 con 0x5632d5e159d0 2024-04-17T10:46:19.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcad2209000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac81ffbc0 con 0x5632d5e162f0 2024-04-17T10:46:19.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcad2209000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac8147160 con 0x5632d5dcc0a0 2024-04-17T10:46:19.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcac7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d5e159d0 0x5632d5dca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcac7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d5e159d0 0x5632d5dca950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52200/0 (socket says 172.21.15.83:52200) 2024-04-17T10:46:19.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcac7fff640 1 -- 172.21.15.83:0/1184251401 learned_addr learned my addr 172.21.15.83:0/1184251401 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:19.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcac7fff640 1 -- 172.21.15.83:0/1184251401 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5632d5dcc0a0 msgr2=0x5632d5dca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:19.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcacce38640 1 --2- 172.21.15.83:0/1184251401 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5632d5dcc0a0 0x5632d5dca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcac7fff640 1 --2- 172.21.15.83:0/1184251401 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5632d5dcc0a0 0x5632d5dca410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcacd639640 1 --2- 172.21.15.83:0/1184251401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632d5e162f0 0x5632d5dcae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcac7fff640 1 -- 172.21.15.83:0/1184251401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632d5e162f0 msgr2=0x5632d5dcae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:19.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcac7fff640 1 --2- 172.21.15.83:0/1184251401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632d5e162f0 0x5632d5dcae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcac7fff640 1 -- 172.21.15.83:0/1184251401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632d5c5fde0 con 0x5632d5e159d0 2024-04-17T10:46:19.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcacd639640 1 --2- 172.21.15.83:0/1184251401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632d5e162f0 0x5632d5dcae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:19.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcac7fff640 1 --2- 172.21.15.83:0/1184251401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d5e159d0 0x5632d5dca950 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fcabc0564b0 tx=0x7fcabc06f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaea09d8aab35e96 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:19.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcaadffb640 1 -- 172.21.15.83:0/1184251401 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcabc083430 con 0x5632d5e159d0 2024-04-17T10:46:19.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcaadffb640 1 -- 172.21.15.83:0/1184251401 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcabc003cd0 con 0x5632d5e159d0 2024-04-17T10:46:19.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcaadffb640 1 -- 172.21.15.83:0/1184251401 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcabc069bc0 con 0x5632d5e159d0 2024-04-17T10:46:19.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcad2209000 1 -- 172.21.15.83:0/1184251401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d5e159d0 msgr2=0x5632d5dca950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:19.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcad2209000 1 --2- 172.21.15.83:0/1184251401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d5e159d0 0x5632d5dca950 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fcabc0564b0 tx=0x7fcabc06f000 comp rx=0 tx=0).stop 2024-04-17T10:46:19.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcad2209000 1 -- 172.21.15.83:0/1184251401 shutdown_connections 2024-04-17T10:46:19.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcad2209000 1 --2- 172.21.15.83:0/1184251401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632d5e162f0 0x5632d5dcae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcad2209000 1 --2- 172.21.15.83:0/1184251401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d5e159d0 0x5632d5dca950 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcad2209000 1 --2- 172.21.15.83:0/1184251401 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5632d5dcc0a0 0x5632d5dca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcad2209000 1 -- 172.21.15.83:0/1184251401 >> 172.21.15.83:0/1184251401 conn(0x5632d5dc8020 msgr2=0x5632d5e22fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:19.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcad2209000 1 -- 172.21.15.83:0/1184251401 shutdown_connections 2024-04-17T10:46:19.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcad2209000 1 -- 172.21.15.83:0/1184251401 wait complete. 2024-04-17T10:46:19.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcad2209000 1 -- start start 2024-04-17T10:46:19.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcad2209000 1 Processor -- start 2024-04-17T10:46:19.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcad2209000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5632d5dcc0a0 0x5632d5e7e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcad2209000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d5e159d0 0x5632d5e7e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcad2209000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632d5e162f0 0x5632d5e27fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcad2209000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d5bb3e00 con 0x5632d5e159d0 2024-04-17T10:46:19.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcad2209000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac8147760 con 0x5632d5e162f0 2024-04-17T10:46:19.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcad2209000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632d5c18300 con 0x5632d5dcc0a0 2024-04-17T10:46:19.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcad2209000 10 client.0 _ll_register_callbacks cb 0x5632d5d97040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:46:19.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcac7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d5e159d0 0x5632d5e7e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcac7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d5e159d0 0x5632d5e7e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52214/0 (socket says 172.21.15.83:52214) 2024-04-17T10:46:19.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcac7fff640 1 -- 172.21.15.83:0/312906694 learned_addr learned my addr 172.21.15.83:0/312906694 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:19.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcacce38640 1 --2- 172.21.15.83:0/312906694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5632d5dcc0a0 0x5632d5e7e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcacd639640 1 --2- 172.21.15.83:0/312906694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632d5e162f0 0x5632d5e27fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcac7fff640 1 -- 172.21.15.83:0/312906694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5632d5dcc0a0 msgr2=0x5632d5e7e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:19.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcac7fff640 1 --2- 172.21.15.83:0/312906694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5632d5dcc0a0 0x5632d5e7e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcac7fff640 1 -- 172.21.15.83:0/312906694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632d5e162f0 msgr2=0x5632d5e27fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:19.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcac7fff640 1 --2- 172.21.15.83:0/312906694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632d5e162f0 0x5632d5e27fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcac7fff640 1 -- 172.21.15.83:0/312906694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632d5bba8e0 con 0x5632d5e159d0 2024-04-17T10:46:19.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcac7fff640 1 --2- 172.21.15.83:0/312906694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d5e159d0 0x5632d5e7e890 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fcabc055900 tx=0x7fcabc06d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d4d2800f3fc0b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:19.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcaa77fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:46:19.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcaa77fe640 1 -- 172.21.15.83:0/312906694 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcabc083430 con 0x5632d5e159d0 2024-04-17T10:46:19.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcaa77fe640 1 -- 172.21.15.83:0/312906694 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fcabc069bc0 con 0x5632d5e159d0 2024-04-17T10:46:19.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.319+0000 7fcad2209000 1 -- 172.21.15.83:0/312906694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcabc003cd0 con 0x5632d5e159d0 2024-04-17T10:46:19.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcaa77fe640 1 -- 172.21.15.83:0/312906694 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcabc069de0 con 0x5632d5e159d0 2024-04-17T10:46:19.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcad2209000 10 client.7365 fetch_fsmap learned FSMap version 49 2024-04-17T10:46:19.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcad2209000 1 -- 172.21.15.83:0/312906694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x5632d5c728f0 con 0x5632d5e159d0 2024-04-17T10:46:19.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcaa77fe640 1 -- 172.21.15.83:0/312906694 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcabc003cd0 con 0x5632d5e159d0 2024-04-17T10:46:19.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcaa77fe640 1 -- 172.21.15.83:0/312906694 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcabc066060 con 0x5632d5e159d0 2024-04-17T10:46:19.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcad2209000 10 client.7365 fetch_fsmap finished waiting for FSMap version 49 2024-04-17T10:46:19.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcad2209000 10 client.7365 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-17T10:46:19.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcad2209000 20 client.7365 populate_metadata read hostname 'smithi083' 2024-04-17T10:46:19.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcad2209000 1 --2- 172.21.15.83:0/312906694 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x5632d5e288f0 0x5632d5e28d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcad2209000 4 client.7365 mds_command: new command op to 7104 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T10:46:19.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcad2209000 1 -- 172.21.15.83:0/312906694 --> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5632d5c26530 con 0x5632d5e288f0 2024-04-17T10:46:19.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcacce38640 1 --2- 172.21.15.83:0/312906694 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x5632d5e288f0 0x5632d5e28d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcaa77fe640 10 client.7365 ms_handle_connect on v2:172.21.15.167:6832/2796648400 2024-04-17T10:46:19.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcacce38640 1 --2- 172.21.15.83:0/312906694 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x5632d5e288f0 0x5632d5e28d00 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48f63b9317ac4ce server_cookie=3ab2b16868708137 in_seq=0 out_seq=0 2024-04-17T10:46:19.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcaa77fe640 1 -- 172.21.15.83:0/312906694 <== mds.0 v2:172.21.15.167:6832/2796648400 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5632d5c26530 con 0x5632d5e288f0 2024-04-17T10:46:19.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcaa77fe640 10 client.7365 handle_command_reply: tid=0 2024-04-17T10:46:19.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcad2209000 1 client.7365 shutdown 2024-04-17T10:46:19.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcad2209000 1 -- 172.21.15.83:0/312906694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d5e159d0 msgr2=0x5632d5e7e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:19.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcad2209000 1 --2- 172.21.15.83:0/312906694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d5e159d0 0x5632d5e7e890 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fcabc055900 tx=0x7fcabc06d000 comp rx=0 tx=0).stop 2024-04-17T10:46:19.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcad2209000 1 -- 172.21.15.83:0/312906694 shutdown_connections 2024-04-17T10:46:19.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcad2209000 1 --2- 172.21.15.83:0/312906694 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x5632d5e288f0 0x5632d5e28d00 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcad2209000 1 --2- 172.21.15.83:0/312906694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632d5e162f0 0x5632d5e27fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcad2209000 1 --2- 172.21.15.83:0/312906694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d5e159d0 0x5632d5e7e890 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcaa77fe640 0 client.7365 ms_handle_reset on v2:172.21.15.167:6832/2796648400 2024-04-17T10:46:19.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcad2209000 1 --2- 172.21.15.83:0/312906694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5632d5dcc0a0 0x5632d5e7e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcad2209000 1 -- 172.21.15.83:0/312906694 >> 172.21.15.83:0/312906694 conn(0x5632d5dc8020 msgr2=0x5632d5e14070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:19.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcad2209000 1 -- 172.21.15.83:0/312906694 shutdown_connections 2024-04-17T10:46:19.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcad2209000 1 -- 172.21.15.83:0/312906694 wait complete. 2024-04-17T10:46:19.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcad2209000 20 client.7365 trim_cache size 0 max 16384 2024-04-17T10:46:19.326 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:46:19.326 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-17T10:46:19.326 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-17T10:46:19.327 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:46:19.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcacf09c640 1 -- 172.21.15.83:0/2582443092 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcab4066f60 msgr2=0x7fcab4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:19.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcacf09c640 1 --2- 172.21.15.83:0/2582443092 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcab4066f60 0x7fcab4069420 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcab80ad000 tx=0x7fcab8008040 comp rx=0 tx=0).stop 2024-04-17T10:46:19.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcacf09c640 1 -- 172.21.15.83:0/2582443092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcac815b6e0 msgr2=0x7fcac815dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:19.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcacf09c640 1 --2- 172.21.15.83:0/2582443092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcac815b6e0 0x7fcac815dd30 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fcabc00d600 tx=0x7fcabc00dac0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcacf09c640 1 -- 172.21.15.83:0/2582443092 shutdown_connections 2024-04-17T10:46:19.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcacf09c640 1 --2- 172.21.15.83:0/2582443092 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcab4066f60 0x7fcab4069420 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcacf09c640 1 --2- 172.21.15.83:0/2582443092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcac815bac0 0x7fcac8146020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcacf09c640 1 --2- 172.21.15.83:0/2582443092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcac815b6e0 0x7fcac815dd30 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcacf09c640 1 --2- 172.21.15.83:0/2582443092 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcac8070470 0x7fcac8148a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcacf09c640 1 -- 172.21.15.83:0/2582443092 >> 172.21.15.83:0/2582443092 conn(0x7fcac80b15b0 msgr2=0x7fcac80b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:19.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcacf09c640 1 -- 172.21.15.83:0/2582443092 shutdown_connections 2024-04-17T10:46:19.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.323+0000 7fcacf09c640 1 -- 172.21.15.83:0/2582443092 wait complete. 2024-04-17T10:46:19.336 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T10:46:19.337 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-17T10:46:19.337 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 perf dump 2024-04-17T10:46:19.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.455+0000 7fe01d1f6640 1 Processor -- start 2024-04-17T10:46:19.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.455+0000 7fe01d1f6640 1 -- start start 2024-04-17T10:46:19.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.455+0000 7fe01d1f6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe01815e330 0x7fe01815e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.455+0000 7fe01d1f6640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0181625d0 con 0x7fe018070aa0 2024-04-17T10:46:19.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.455+0000 7fe01d1f6640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe018161810 con 0x7fe01806ae10 2024-04-17T10:46:19.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.455+0000 7fe01d1f6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe018161af0 con 0x7fe01815e330 2024-04-17T10:46:19.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.455+0000 7fe01759e640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe018070aa0 0x7fe018070f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:44100/0 (socket says 172.21.15.83:44100) 2024-04-17T10:46:19.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.455+0000 7fe01759e640 1 -- 172.21.15.83:0/325544395 learned_addr learned my addr 172.21.15.83:0/325544395 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:46:19.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe01659c640 1 --2- 172.21.15.83:0/325544395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe01815e330 0x7fe01815e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe015d9b640 1 -- 172.21.15.83:0/325544395 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 853929116 0 0) 0x7fe0181625d0 con 0x7fe018070aa0 2024-04-17T10:46:19.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe015d9b640 1 -- 172.21.15.83:0/325544395 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe000003530 con 0x7fe018070aa0 2024-04-17T10:46:19.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe015d9b640 1 -- 172.21.15.83:0/325544395 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 190132222 0 0) 0x7fe000003530 con 0x7fe018070aa0 2024-04-17T10:46:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe015d9b640 1 -- 172.21.15.83:0/325544395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe01815e330 msgr2=0x7fe01815e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe015d9b640 1 --2- 172.21.15.83:0/325544395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe01815e330 0x7fe01815e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe015d9b640 1 -- 172.21.15.83:0/325544395 >> v1:172.21.15.167:6789/0 conn(0x7fe01806ae10 legacy=0x7fe01806b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe015d9b640 1 -- 172.21.15.83:0/325544395 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe018161dd0 con 0x7fe018070aa0 2024-04-17T10:46:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe015d9b640 1 -- 172.21.15.83:0/325544395 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fe010001a00 con 0x7fe018070aa0 2024-04-17T10:46:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe015d9b640 1 -- 172.21.15.83:0/325544395 >> v1:172.21.15.83:6789/0 conn(0x7fe018070aa0 legacy=0x7fe018070f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe015d9b640 1 --2- 172.21.15.83:0/325544395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe000003a60 0x7fe000003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe015d9b640 1 --2- 172.21.15.83:0/325544395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe000004810 0x7fe000004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe015d9b640 1 --2- 172.21.15.83:0/325544395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0000052f0 0x7fe00000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe015d9b640 1 -- 172.21.15.83:0/325544395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe018161af0 con 0x7fe0000052f0 2024-04-17T10:46:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe015d9b640 1 -- 172.21.15.83:0/325544395 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00000c1a0 con 0x7fe000004810 2024-04-17T10:46:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe015d9b640 1 -- 172.21.15.83:0/325544395 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe00000c3a0 con 0x7fe000003a60 2024-04-17T10:46:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe01659c640 1 --2- 172.21.15.83:0/325544395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe01815e330 0x7fe01815e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:46:19.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe01759e640 1 --2- 172.21.15.83:0/325544395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0000052f0 0x7fe00000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe016d9d640 1 --2- 172.21.15.83:0/325544395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe000003a60 0x7fe000003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe01759e640 1 -- 172.21.15.83:0/325544395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe000003a60 msgr2=0x7fe000003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:19.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe01759e640 1 --2- 172.21.15.83:0/325544395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe000003a60 0x7fe000003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe01659c640 1 --2- 172.21.15.83:0/325544395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe000004810 0x7fe000004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe01759e640 1 -- 172.21.15.83:0/325544395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe000004810 msgr2=0x7fe000004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:19.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe01759e640 1 --2- 172.21.15.83:0/325544395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe000004810 0x7fe000004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe01759e640 1 -- 172.21.15.83:0/325544395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe000003530 con 0x7fe0000052f0 2024-04-17T10:46:19.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe016d9d640 1 --2- 172.21.15.83:0/325544395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe000003a60 0x7fe000003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:19.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe01759e640 1 --2- 172.21.15.83:0/325544395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0000052f0 0x7fe00000b9d0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fe01000a830 tx=0x7fe01001fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b026b13ebb7ee0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:19.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe015d9b640 1 -- 172.21.15.83:0/325544395 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01000a480 con 0x7fe0000052f0 2024-04-17T10:46:19.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe015d9b640 1 -- 172.21.15.83:0/325544395 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0100aa6f0 con 0x7fe0000052f0 2024-04-17T10:46:19.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe015d9b640 1 -- 172.21.15.83:0/325544395 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0100aab40 con 0x7fe0000052f0 2024-04-17T10:46:19.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe01d1f6640 1 -- 172.21.15.83:0/325544395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0000052f0 msgr2=0x7fe00000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:19.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe01d1f6640 1 --2- 172.21.15.83:0/325544395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0000052f0 0x7fe00000b9d0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fe01000a830 tx=0x7fe01001fac0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe01759e640 1 -- 172.21.15.83:0/325544395 reap_dead start 2024-04-17T10:46:19.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe01d1f6640 1 -- 172.21.15.83:0/325544395 shutdown_connections 2024-04-17T10:46:19.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe01d1f6640 1 -- 172.21.15.83:0/325544395 >> 172.21.15.83:0/325544395 conn(0x7fe0180b15b0 msgr2=0x7fe0180b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:19.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe01d1f6640 1 -- 172.21.15.83:0/325544395 shutdown_connections 2024-04-17T10:46:19.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.459+0000 7fe01d1f6640 1 -- 172.21.15.83:0/325544395 wait complete. 2024-04-17T10:46:19.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.463+0000 7fe01d1f6640 1 Processor -- start 2024-04-17T10:46:19.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.463+0000 7fe01d1f6640 1 -- start start 2024-04-17T10:46:19.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.463+0000 7fe01d1f6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0000052f0 0x7fe0180ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.463+0000 7fe01d1f6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe01815d130 0x7fe01815d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.463+0000 7fe01d1f6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0180c4650 0x7fe0180c4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.463+0000 7fe01d1f6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0180cf470 con 0x7fe0000052f0 2024-04-17T10:46:19.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.463+0000 7fe01d1f6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0180cf2f0 con 0x7fe0180c4650 2024-04-17T10:46:19.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.463+0000 7fe01d1f6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe018071640 con 0x7fe01815d130 2024-04-17T10:46:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.463+0000 7fe016d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0000052f0 0x7fe0180ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.463+0000 7fe016d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0000052f0 0x7fe0180ce1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52230/0 (socket says 172.21.15.83:52230) 2024-04-17T10:46:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.463+0000 7fe016d9d640 1 -- 172.21.15.83:0/1540082596 learned_addr learned my addr 172.21.15.83:0/1540082596 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.463+0000 7fe01759e640 1 --2- 172.21.15.83:0/1540082596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0180c4650 0x7fe0180c4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.463+0000 7fe01659c640 1 --2- 172.21.15.83:0/1540082596 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe01815d130 0x7fe01815d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.463+0000 7fe016d9d640 1 -- 172.21.15.83:0/1540082596 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe01815d130 msgr2=0x7fe01815d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.463+0000 7fe016d9d640 1 --2- 172.21.15.83:0/1540082596 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe01815d130 0x7fe01815d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.463+0000 7fe016d9d640 1 -- 172.21.15.83:0/1540082596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0180c4650 msgr2=0x7fe0180c4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.463+0000 7fe016d9d640 1 --2- 172.21.15.83:0/1540082596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0180c4650 0x7fe0180c4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.463+0000 7fe016d9d640 1 -- 172.21.15.83:0/1540082596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0180c6400 con 0x7fe0000052f0 2024-04-17T10:46:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.463+0000 7fe016d9d640 1 --2- 172.21.15.83:0/1540082596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0000052f0 0x7fe0180ce1d0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fe00800d7a0 tx=0x7fe00800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.463+0000 7fdfff7fe640 1 -- 172.21.15.83:0/1540082596 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe008005270 con 0x7fe0000052f0 2024-04-17T10:46:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.463+0000 7fdfff7fe640 1 -- 172.21.15.83:0/1540082596 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe008017070 con 0x7fe0000052f0 2024-04-17T10:46:19.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.463+0000 7fdfff7fe640 1 -- 172.21.15.83:0/1540082596 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00800b6b0 con 0x7fe0000052f0 2024-04-17T10:46:19.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.463+0000 7fe01d1f6640 1 -- 172.21.15.83:0/1540082596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe018135c20 con 0x7fe0000052f0 2024-04-17T10:46:19.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.463+0000 7fe01d1f6640 1 -- 172.21.15.83:0/1540082596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0180c5640 con 0x7fe0000052f0 2024-04-17T10:46:19.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fdfff7fe640 1 -- 172.21.15.83:0/1540082596 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe00800b850 con 0x7fe0000052f0 2024-04-17T10:46:19.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fdfff7fe640 1 --2- 172.21.15.83:0/1540082596 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe004066f60 0x7fe004069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fdfff7fe640 1 -- 172.21.15.83:0/1540082596 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe008084fc0 con 0x7fe0000052f0 2024-04-17T10:46:19.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fe01659c640 1 --2- 172.21.15.83:0/1540082596 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe004066f60 0x7fe004069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fe01659c640 1 --2- 172.21.15.83:0/1540082596 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe004066f60 0x7fe004069420 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe01815e270 tx=0x7fe00c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:19.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fe020363000 1 Processor -- start 2024-04-17T10:46:19.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fe020363000 1 -- start start 2024-04-17T10:46:19.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fe020363000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c304afa40 0x564c304afe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fe020363000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c304b0360 0x564c3046a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fe020363000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564c3046ad90 0x564c3046b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fe020363000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c3034d1f0 con 0x564c304afa40 2024-04-17T10:46:19.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fe020363000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c302b3e90 con 0x564c304b0360 2024-04-17T10:46:19.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fe020363000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c3023bf50 con 0x564c3046ad90 2024-04-17T10:46:19.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fe016d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c304afa40 0x564c304afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fe016d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c304afa40 0x564c304afe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52240/0 (socket says 172.21.15.83:52240) 2024-04-17T10:46:19.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fe016d9d640 1 -- 172.21.15.83:0/4004146034 learned_addr learned my addr 172.21.15.83:0/4004146034 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:19.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fe016d9d640 1 -- 172.21.15.83:0/4004146034 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564c3046ad90 msgr2=0x564c3046b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:19.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fe016d9d640 1 --2- 172.21.15.83:0/4004146034 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564c3046ad90 0x564c3046b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fe016d9d640 1 -- 172.21.15.83:0/4004146034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c304b0360 msgr2=0x564c3046a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:19.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fe016d9d640 1 --2- 172.21.15.83:0/4004146034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c304b0360 0x564c3046a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fe016d9d640 1 -- 172.21.15.83:0/4004146034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c3028b8b0 con 0x564c304afa40 2024-04-17T10:46:19.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fe016d9d640 1 --2- 172.21.15.83:0/4004146034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c304afa40 0x564c304afe20 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fe008051cc0 tx=0x7fe0080554d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926f0aaa269074e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:19.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fdffcff9640 1 -- 172.21.15.83:0/4004146034 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00805df70 con 0x564c304afa40 2024-04-17T10:46:19.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fdffcff9640 1 -- 172.21.15.83:0/4004146034 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe008053b60 con 0x564c304afa40 2024-04-17T10:46:19.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fdffcff9640 1 -- 172.21.15.83:0/4004146034 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe008051ed0 con 0x564c304afa40 2024-04-17T10:46:19.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fe020363000 1 -- 172.21.15.83:0/4004146034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c304afa40 msgr2=0x564c304afe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:19.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fe020363000 1 --2- 172.21.15.83:0/4004146034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c304afa40 0x564c304afe20 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fe008051cc0 tx=0x7fe0080554d0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fe020363000 1 -- 172.21.15.83:0/4004146034 shutdown_connections 2024-04-17T10:46:19.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fe020363000 1 --2- 172.21.15.83:0/4004146034 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564c3046ad90 0x564c3046b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fe020363000 1 --2- 172.21.15.83:0/4004146034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c304b0360 0x564c3046a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fe020363000 1 --2- 172.21.15.83:0/4004146034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c304afa40 0x564c304afe20 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fe020363000 1 -- 172.21.15.83:0/4004146034 >> 172.21.15.83:0/4004146034 conn(0x564c30466d40 msgr2=0x564c30469130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:19.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fe020363000 1 -- 172.21.15.83:0/4004146034 shutdown_connections 2024-04-17T10:46:19.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.467+0000 7fe020363000 1 -- 172.21.15.83:0/4004146034 wait complete. 2024-04-17T10:46:19.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fe020363000 1 -- start start 2024-04-17T10:46:19.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fe020363000 1 Processor -- start 2024-04-17T10:46:19.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fe020363000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c3046ad90 0x564c304b71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fe020363000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c304afa40 0x564c304b7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fe020363000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564c304b0360 0x564c30470dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fe020363000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c3026e540 con 0x564c304afa40 2024-04-17T10:46:19.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fe020363000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c302aff30 con 0x564c3046ad90 2024-04-17T10:46:19.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fe020363000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c30292010 con 0x564c304b0360 2024-04-17T10:46:19.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fe020363000 10 client.0 _ll_register_callbacks cb 0x564c30368180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:46:19.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fe01659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c304afa40 0x564c304b7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fe01659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c304afa40 0x564c304b7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52254/0 (socket says 172.21.15.83:52254) 2024-04-17T10:46:19.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fe01659c640 1 -- 172.21.15.83:0/3431689305 learned_addr learned my addr 172.21.15.83:0/3431689305 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:19.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fe01659c640 1 -- 172.21.15.83:0/3431689305 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564c304b0360 msgr2=0x564c30470dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:19.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fe01659c640 1 --2- 172.21.15.83:0/3431689305 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564c304b0360 0x564c30470dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fe01659c640 1 -- 172.21.15.83:0/3431689305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c3046ad90 msgr2=0x564c304b71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:19.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fe01659c640 1 --2- 172.21.15.83:0/3431689305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c3046ad90 0x564c304b71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fe01659c640 1 -- 172.21.15.83:0/3431689305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c3023c1a0 con 0x564c304afa40 2024-04-17T10:46:19.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fe01659c640 1 --2- 172.21.15.83:0/3431689305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c304afa40 0x564c304b7700 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fe00c016700 tx=0x7fe00c016bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36a4c56f9ca9dc14 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:19.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fdff5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:46:19.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fdff5ffb640 1 -- 172.21.15.83:0/3431689305 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00c004020 con 0x564c304afa40 2024-04-17T10:46:19.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fdff5ffb640 1 -- 172.21.15.83:0/3431689305 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe00c0041c0 con 0x564c304afa40 2024-04-17T10:46:19.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fe020363000 1 -- 172.21.15.83:0/3431689305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe008053b60 con 0x564c304afa40 2024-04-17T10:46:19.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fdff5ffb640 1 -- 172.21.15.83:0/3431689305 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe00c02b050 con 0x564c304afa40 2024-04-17T10:46:19.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fe020363000 10 client.7377 fetch_fsmap learned FSMap version 49 2024-04-17T10:46:19.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fe020363000 1 -- 172.21.15.83:0/3431689305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x564c30336840 con 0x564c304afa40 2024-04-17T10:46:19.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fdff5ffb640 1 -- 172.21.15.83:0/3431689305 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe008053b60 con 0x564c304afa40 2024-04-17T10:46:19.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fdff5ffb640 1 -- 172.21.15.83:0/3431689305 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe00c02e020 con 0x564c304afa40 2024-04-17T10:46:19.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fe020363000 10 client.7377 fetch_fsmap finished waiting for FSMap version 49 2024-04-17T10:46:19.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fe020363000 10 client.7377 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-17T10:46:19.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fe020363000 20 client.7377 populate_metadata read hostname 'smithi083' 2024-04-17T10:46:19.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fe020363000 1 --2- 172.21.15.83:0/3431689305 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x564c30471b60 0x564c30471f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fe020363000 4 client.7377 mds_command: new command op to 7104 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:46:19.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fe020363000 1 -- 172.21.15.83:0/3431689305 --> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c3024fbe0 con 0x564c30471b60 2024-04-17T10:46:19.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fe016d9d640 1 --2- 172.21.15.83:0/3431689305 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x564c30471b60 0x564c30471f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fdff5ffb640 10 client.7377 ms_handle_connect on v2:172.21.15.167:6832/2796648400 2024-04-17T10:46:19.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.471+0000 7fe016d9d640 1 --2- 172.21.15.83:0/3431689305 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x564c30471b60 0x564c30471f70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bf8878e43378639 server_cookie=df700433688b0711 in_seq=0 out_seq=0 2024-04-17T10:46:19.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.475+0000 7fdff5ffb640 1 -- 172.21.15.83:0/3431689305 <== mds.0 v2:172.21.15.167:6832/2796648400 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564c3024fbe0 con 0x564c30471b60 2024-04-17T10:46:19.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.475+0000 7fdff5ffb640 10 client.7377 handle_command_reply: tid=0 2024-04-17T10:46:19.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.475+0000 7fe020363000 1 client.7377 shutdown 2024-04-17T10:46:19.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.475+0000 7fe020363000 1 -- 172.21.15.83:0/3431689305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c304afa40 msgr2=0x564c304b7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:19.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.475+0000 7fe020363000 1 --2- 172.21.15.83:0/3431689305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c304afa40 0x564c304b7700 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fe00c016700 tx=0x7fe00c016bc0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.475+0000 7fe020363000 1 -- 172.21.15.83:0/3431689305 shutdown_connections 2024-04-17T10:46:19.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.475+0000 7fe020363000 1 --2- 172.21.15.83:0/3431689305 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x564c30471b60 0x564c30471f70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.475+0000 7fe020363000 1 --2- 172.21.15.83:0/3431689305 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564c304b0360 0x564c30470dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.475+0000 7fe020363000 1 --2- 172.21.15.83:0/3431689305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c304afa40 0x564c304b7700 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.475+0000 7fe020363000 1 --2- 172.21.15.83:0/3431689305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c3046ad90 0x564c304b71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.475+0000 7fdff5ffb640 0 client.7377 ms_handle_reset on v2:172.21.15.167:6832/2796648400 2024-04-17T10:46:19.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.475+0000 7fe020363000 1 -- 172.21.15.83:0/3431689305 >> 172.21.15.83:0/3431689305 conn(0x564c30466d40 msgr2=0x564c304699c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:19.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.475+0000 7fe020363000 1 -- 172.21.15.83:0/3431689305 shutdown_connections 2024-04-17T10:46:19.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.475+0000 7fe020363000 1 -- 172.21.15.83:0/3431689305 wait complete. 2024-04-17T10:46:19.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.475+0000 7fe020363000 20 client.7377 trim_cache size 0 max 16384 2024-04-17T10:46:19.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.487+0000 7fe020363000 1 Processor -- start 2024-04-17T10:46:19.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.487+0000 7fe020363000 1 -- start start 2024-04-17T10:46:19.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c3046ad90 0x564c3050be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564c304afa40 0x564c3050c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c304b0360 0x564c3050c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0182005a0 con 0x564c304b0360 2024-04-17T10:46:19.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0181ff760 con 0x564c3046ad90 2024-04-17T10:46:19.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0181358b0 con 0x564c304afa40 2024-04-17T10:46:19.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe01759e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c304b0360 0x564c3050c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe01759e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c304b0360 0x564c3050c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52260/0 (socket says 172.21.15.83:52260) 2024-04-17T10:46:19.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe01759e640 1 -- 172.21.15.83:0/4199089383 learned_addr learned my addr 172.21.15.83:0/4199089383 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:19.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe016d9d640 1 --2- 172.21.15.83:0/4199089383 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c3046ad90 0x564c3050be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe01659c640 1 --2- 172.21.15.83:0/4199089383 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564c304afa40 0x564c3050c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe01759e640 1 -- 172.21.15.83:0/4199089383 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564c304afa40 msgr2=0x564c3050c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:19.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe01759e640 1 --2- 172.21.15.83:0/4199089383 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564c304afa40 0x564c3050c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe01759e640 1 -- 172.21.15.83:0/4199089383 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c3046ad90 msgr2=0x564c3050be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:19.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe01759e640 1 --2- 172.21.15.83:0/4199089383 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c3046ad90 0x564c3050be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe01759e640 1 -- 172.21.15.83:0/4199089383 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c3030da10 con 0x564c304b0360 2024-04-17T10:46:19.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe016d9d640 1 --2- 172.21.15.83:0/4199089383 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c3046ad90 0x564c3050be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:19.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe01759e640 1 --2- 172.21.15.83:0/4199089383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c304b0360 0x564c3050c8f0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fe01001f8f0 tx=0x7fe01001c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faef6f280f1b6d18 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:19.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fdff7fff640 1 -- 172.21.15.83:0/4199089383 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe010004b70 con 0x564c304b0360 2024-04-17T10:46:19.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fdff7fff640 1 -- 172.21.15.83:0/4199089383 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe010004d10 con 0x564c304b0360 2024-04-17T10:46:19.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fdff7fff640 1 -- 172.21.15.83:0/4199089383 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe010001a00 con 0x564c304b0360 2024-04-17T10:46:19.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 1 -- 172.21.15.83:0/4199089383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c304b0360 msgr2=0x564c3050c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:19.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 1 --2- 172.21.15.83:0/4199089383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c304b0360 0x564c3050c8f0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fe01001f8f0 tx=0x7fe01001c860 comp rx=0 tx=0).stop 2024-04-17T10:46:19.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 1 -- 172.21.15.83:0/4199089383 shutdown_connections 2024-04-17T10:46:19.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 1 --2- 172.21.15.83:0/4199089383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c304b0360 0x564c3050c8f0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 1 --2- 172.21.15.83:0/4199089383 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564c304afa40 0x564c3050c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 1 --2- 172.21.15.83:0/4199089383 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c3046ad90 0x564c3050be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 1 -- 172.21.15.83:0/4199089383 >> 172.21.15.83:0/4199089383 conn(0x564c30466d40 msgr2=0x564c304bd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:19.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 1 -- 172.21.15.83:0/4199089383 shutdown_connections 2024-04-17T10:46:19.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 1 -- 172.21.15.83:0/4199089383 wait complete. 2024-04-17T10:46:19.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 1 -- start start 2024-04-17T10:46:19.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 1 Processor -- start 2024-04-17T10:46:19.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c3046ad90 0x564c30510090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c304afa40 0x564c305105d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564c304b0360 0x564c304b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c302aff30 con 0x564c3046ad90 2024-04-17T10:46:19.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0181ff160 con 0x564c304afa40 2024-04-17T10:46:19.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c3025ac00 con 0x564c304b0360 2024-04-17T10:46:19.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 10 client.0 _ll_register_callbacks cb 0x564c30276a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:46:19.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe016d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c3046ad90 0x564c30510090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe016d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c3046ad90 0x564c30510090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52276/0 (socket says 172.21.15.83:52276) 2024-04-17T10:46:19.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe016d9d640 1 -- 172.21.15.83:0/2314500168 learned_addr learned my addr 172.21.15.83:0/2314500168 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:19.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe016d9d640 1 -- 172.21.15.83:0/2314500168 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564c304b0360 msgr2=0x564c304b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:19.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe01759e640 1 --2- 172.21.15.83:0/2314500168 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564c304b0360 0x564c304b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe016d9d640 1 --2- 172.21.15.83:0/2314500168 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564c304b0360 0x564c304b46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe016d9d640 1 -- 172.21.15.83:0/2314500168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c304afa40 msgr2=0x564c305105d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:19.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe016d9d640 1 --2- 172.21.15.83:0/2314500168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c304afa40 0x564c305105d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe016d9d640 1 -- 172.21.15.83:0/2314500168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c30248b00 con 0x564c3046ad90 2024-04-17T10:46:19.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe016d9d640 1 --2- 172.21.15.83:0/2314500168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c3046ad90 0x564c30510090 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fe008005630 tx=0x7fe008001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c52ef5ef076fbed6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:19.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fdff5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:46:19.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fdff5ffb640 1 -- 172.21.15.83:0/2314500168 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe008051b00 con 0x564c3046ad90 2024-04-17T10:46:19.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fdff5ffb640 1 -- 172.21.15.83:0/2314500168 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe008083420 con 0x564c3046ad90 2024-04-17T10:46:19.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 1 -- 172.21.15.83:0/2314500168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe010004d10 con 0x564c3046ad90 2024-04-17T10:46:19.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fdff5ffb640 1 -- 172.21.15.83:0/2314500168 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe008053b60 con 0x564c3046ad90 2024-04-17T10:46:19.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 10 client.7383 fetch_fsmap learned FSMap version 49 2024-04-17T10:46:19.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 1 -- 172.21.15.83:0/2314500168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x564c3025af80 con 0x564c3046ad90 2024-04-17T10:46:19.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fdff5ffb640 1 -- 172.21.15.83:0/2314500168 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe010004d10 con 0x564c3046ad90 2024-04-17T10:46:19.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fdff5ffb640 1 -- 172.21.15.83:0/2314500168 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe00806d200 con 0x564c3046ad90 2024-04-17T10:46:19.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 10 client.7383 fetch_fsmap finished waiting for FSMap version 49 2024-04-17T10:46:19.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 10 client.7383 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-17T10:46:19.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 20 client.7383 populate_metadata read hostname 'smithi083' 2024-04-17T10:46:19.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 1 --2- 172.21.15.83:0/2314500168 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x564c304b5040 0x564c304b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 4 client.7383 mds_command: new command op to 7104 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-17T10:46:19.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.491+0000 7fe020363000 1 -- 172.21.15.83:0/2314500168 --> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x564c302c13d0 con 0x564c304b5040 2024-04-17T10:46:19.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.495+0000 7fe01659c640 1 --2- 172.21.15.83:0/2314500168 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x564c304b5040 0x564c304b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.495+0000 7fdff5ffb640 10 client.7383 ms_handle_connect on v2:172.21.15.167:6832/2796648400 2024-04-17T10:46:19.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.495+0000 7fe01659c640 1 --2- 172.21.15.83:0/2314500168 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x564c304b5040 0x564c304b5450 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83086fc931a67d0d server_cookie=8601aceb17fa91ee in_seq=0 out_seq=0 2024-04-17T10:46:19.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.495+0000 7fdff5ffb640 1 -- 172.21.15.83:0/2314500168 <== mds.0 v2:172.21.15.167:6832/2796648400 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17955 (crc 0 0 0) 0x564c302c13d0 con 0x564c304b5040 2024-04-17T10:46:19.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.495+0000 7fdff5ffb640 10 client.7383 handle_command_reply: tid=0 2024-04-17T10:46:19.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.495+0000 7fe020363000 1 client.7383 shutdown 2024-04-17T10:46:19.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.495+0000 7fe020363000 1 -- 172.21.15.83:0/2314500168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c3046ad90 msgr2=0x564c30510090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:19.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.495+0000 7fe020363000 1 --2- 172.21.15.83:0/2314500168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c3046ad90 0x564c30510090 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fe008005630 tx=0x7fe008001130 comp rx=0 tx=0).stop 2024-04-17T10:46:19.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.495+0000 7fe020363000 1 -- 172.21.15.83:0/2314500168 shutdown_connections 2024-04-17T10:46:19.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.495+0000 7fe020363000 1 --2- 172.21.15.83:0/2314500168 >> [v2:172.21.15.167:6832/2796648400,v1:172.21.15.167:6833/2796648400] conn(0x564c304b5040 0x564c304b5450 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.495+0000 7fe020363000 1 --2- 172.21.15.83:0/2314500168 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564c304b0360 0x564c304b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.495+0000 7fdff5ffb640 0 client.7383 ms_handle_reset on v2:172.21.15.167:6832/2796648400 2024-04-17T10:46:19.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.495+0000 7fe020363000 1 --2- 172.21.15.83:0/2314500168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c304afa40 0x564c305105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.495+0000 7fe020363000 1 --2- 172.21.15.83:0/2314500168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c3046ad90 0x564c30510090 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.495+0000 7fe020363000 1 -- 172.21.15.83:0/2314500168 >> 172.21.15.83:0/2314500168 conn(0x564c30466d40 msgr2=0x564c304ae120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:19.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.495+0000 7fe020363000 1 -- 172.21.15.83:0/2314500168 shutdown_connections 2024-04-17T10:46:19.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.495+0000 7fe020363000 1 -- 172.21.15.83:0/2314500168 wait complete. 2024-04-17T10:46:19.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.495+0000 7fe020363000 20 client.7383 trim_cache size 0 max 16384 2024-04-17T10:46:19.498 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:46:19.498 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T10:46:19.499 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 4689, 2024-04-17T10:46:19.499 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 4690, 2024-04-17T10:46:19.499 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1575724, 2024-04-17T10:46:19.499 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 34432695, 2024-04-17T10:46:19.499 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 17, 2024-04-17T10:46:19.499 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 6, 2024-04-17T10:46:19.499 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.755673622, 2024-04-17T10:46:19.499 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.345456664, 2024-04-17T10:46:19.499 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.140157119, 2024-04-17T10:46:19.499 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.212387600, 2024-04-17T10:46:19.499 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T10:46:19.499 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 4689, 2024-04-17T10:46:19.499 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.607573698, 2024-04-17T10:46:19.500 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000129574 2024-04-17T10:46:19.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.500 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-17T10:46:19.500 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 10, 2024-04-17T10:46:19.500 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000001976, 2024-04-17T10:46:19.500 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000197 2024-04-17T10:46:19.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.500 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-17T10:46:19.500 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 0 2024-04-17T10:46:19.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.500 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T10:46:19.500 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 36717, 2024-04-17T10:46:19.500 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 63116, 2024-04-17T10:46:19.501 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 12904364, 2024-04-17T10:46:19.501 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 61520740, 2024-04-17T10:46:19.501 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 7, 2024-04-17T10:46:19.501 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 4, 2024-04-17T10:46:19.501 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 6.988050140, 2024-04-17T10:46:19.501 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 3.359582031, 2024-04-17T10:46:19.501 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 3.947834818, 2024-04-17T10:46:19.501 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.215128633, 2024-04-17T10:46:19.501 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T10:46:19.501 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 63116, 2024-04-17T10:46:19.501 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 3.425062433, 2024-04-17T10:46:19.501 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000054266 2024-04-17T10:46:19.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.502 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-17T10:46:19.502 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 88224, 2024-04-17T10:46:19.502 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.034282885, 2024-04-17T10:46:19.502 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000388 2024-04-17T10:46:19.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.502 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 125888, 2024-04-17T10:46:19.502 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 38160 2024-04-17T10:46:19.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.502 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T10:46:19.502 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 4632, 2024-04-17T10:46:19.502 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 4667, 2024-04-17T10:46:19.502 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1160281, 2024-04-17T10:46:19.502 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 21796506, 2024-04-17T10:46:19.503 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 6, 2024-04-17T10:46:19.503 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 4, 2024-04-17T10:46:19.503 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.613311308, 2024-04-17T10:46:19.503 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.256397097, 2024-04-17T10:46:19.503 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.124173591, 2024-04-17T10:46:19.503 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.188590148, 2024-04-17T10:46:19.503 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T10:46:19.503 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 4666, 2024-04-17T10:46:19.503 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.699595316, 2024-04-17T10:46:19.503 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000149934 2024-04-17T10:46:19.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.503 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-17T10:46:19.503 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:46:19.504 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:46:19.504 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:46:19.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.504 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-17T10:46:19.504 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 145952 2024-04-17T10:46:19.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.504 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-04-17T10:46:19.504 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 1, 2024-04-17T10:46:19.504 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-04-17T10:46:19.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.504 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-MDSRank": { 2024-04-17T10:46:19.504 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-04-17T10:46:19.504 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-04-17T10:46:19.504 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 11678, 2024-04-17T10:46:19.505 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 36.010490263, 2024-04-17T10:46:19.505 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.003083617 2024-04-17T10:46:19.505 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:46:19.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.505 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-PurgeQueue": { 2024-04-17T10:46:19.505 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-04-17T10:46:19.505 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-04-17T10:46:19.505 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3335, 2024-04-17T10:46:19.505 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 2.311619729, 2024-04-17T10:46:19.505 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000693139 2024-04-17T10:46:19.505 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:46:19.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.505 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": { 2024-04-17T10:46:19.506 INFO:teuthology.orchestra.run.smithi083.stdout: "request": 27238, 2024-04-17T10:46:19.506 INFO:teuthology.orchestra.run.smithi083.stdout: "reply": 27238, 2024-04-17T10:46:19.506 INFO:teuthology.orchestra.run.smithi083.stdout: "reply_latency": { 2024-04-17T10:46:19.506 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 27238, 2024-04-17T10:46:19.506 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 506.689763276, 2024-04-17T10:46:19.506 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.018602311 2024-04-17T10:46:19.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.506 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_reply": 0, 2024-04-17T10:46:19.506 INFO:teuthology.orchestra.run.smithi083.stdout: "forward": 0, 2024-04-17T10:46:19.506 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_complete": 0, 2024-04-17T10:46:19.506 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_keys": 0, 2024-04-17T10:46:19.506 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_commit": 12, 2024-04-17T10:46:19.506 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_split": 0, 2024-04-17T10:46:19.507 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_merge": 0, 2024-04-17T10:46:19.507 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes": 17867, 2024-04-17T10:46:19.507 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_top": 0, 2024-04-17T10:46:19.507 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_bottom": 17867, 2024-04-17T10:46:19.507 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pin_tail": 0, 2024-04-17T10:46:19.507 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pinned": 17867, 2024-04-17T10:46:19.507 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_expired": 0, 2024-04-17T10:46:19.507 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_with_caps": 2, 2024-04-17T10:46:19.507 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": 2, 2024-04-17T10:46:19.507 INFO:teuthology.orchestra.run.smithi083.stdout: "subtrees": 2, 2024-04-17T10:46:19.507 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse": 45320, 2024-04-17T10:46:19.507 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_hit": 44825, 2024-04-17T10:46:19.507 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_forward": 0, 2024-04-17T10:46:19.507 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_discover": 0, 2024-04-17T10:46:19.508 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_dir_fetch": 0, 2024-04-17T10:46:19.508 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_remote_ino": 0, 2024-04-17T10:46:19.508 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_lock": 156, 2024-04-17T10:46:19.508 INFO:teuthology.orchestra.run.smithi083.stdout: "load_cent": 0, 2024-04-17T10:46:19.508 INFO:teuthology.orchestra.run.smithi083.stdout: "q": 0, 2024-04-17T10:46:19.508 INFO:teuthology.orchestra.run.smithi083.stdout: "exported": 0, 2024-04-17T10:46:19.508 INFO:teuthology.orchestra.run.smithi083.stdout: "exported_inodes": 0, 2024-04-17T10:46:19.508 INFO:teuthology.orchestra.run.smithi083.stdout: "imported": 0, 2024-04-17T10:46:19.508 INFO:teuthology.orchestra.run.smithi083.stdout: "imported_inodes": 0, 2024-04-17T10:46:19.508 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_dir_fetch": 0, 2024-04-17T10:46:19.508 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_backtrace_fetch": 2011, 2024-04-17T10:46:19.508 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_peer_discover": 0, 2024-04-17T10:46:19.508 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rfiles": 0, 2024-04-17T10:46:19.509 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rbytes": 0, 2024-04-17T10:46:19.509 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rsnaps": 0, 2024-04-17T10:46:19.509 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_fetch": 2011, 2024-04-17T10:46:19.509 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_set_tag": 0, 2024-04-17T10:46:19.509 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_repaired": 0, 2024-04-17T10:46:19.509 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_inotable_repaired": 0, 2024-04-17T10:46:19.509 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_inodes": 11, 2024-04-17T10:46:19.509 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_base_inodes": 1, 2024-04-17T10:46:19.509 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dirfrag_rstats": 10, 2024-04-17T10:46:19.509 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_file_inodes": 2000, 2024-04-17T10:46:19.509 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_inode_file_caps": 0, 2024-04-17T10:46:19.509 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_revoke": 7085, 2024-04-17T10:46:19.509 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_grant": 6618, 2024-04-17T10:46:19.509 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_trunc": 0, 2024-04-17T10:46:19.510 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-17T10:46:19.510 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-17T10:46:19.510 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps": 7085, 2024-04-17T10:46:19.510 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps_dirty": 2000, 2024-04-17T10:46:19.510 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_cap_release": 63, 2024-04-17T10:46:19.510 INFO:teuthology.orchestra.run.smithi083.stdout: "process_request_cap_release": 17937 2024-04-17T10:46:19.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.510 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache": { 2024-04-17T10:46:19.510 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update": 0, 2024-04-17T10:46:19.510 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update_receipt": 0, 2024-04-17T10:46:19.510 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_try_discover": 0, 2024-04-17T10:46:19.510 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_send_discover": 0, 2024-04-17T10:46:19.510 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_handle_discover": 0, 2024-04-17T10:46:19.510 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays": 0, 2024-04-17T10:46:19.511 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_delayed": 0, 2024-04-17T10:46:19.511 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_enqueuing": 0, 2024-04-17T10:46:19.511 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_created": 4152, 2024-04-17T10:46:19.511 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_enqueued": 3800, 2024-04-17T10:46:19.511 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_reintegrated": 2253, 2024-04-17T10:46:19.511 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_migrated": 0, 2024-04-17T10:46:19.511 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_processing": 0, 2024-04-17T10:46:19.511 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_enqueued": 0, 2024-04-17T10:46:19.511 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_prioritized": 0, 2024-04-17T10:46:19.511 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_started": 0, 2024-04-17T10:46:19.511 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_completed": 0, 2024-04-17T10:46:19.511 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_enqueue_scrub": 1, 2024-04-17T10:46:19.511 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_exportdir": 0, 2024-04-17T10:46:19.512 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_flush": 0, 2024-04-17T10:46:19.512 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragmentdir": 0, 2024-04-17T10:46:19.512 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragstats": 0, 2024-04-17T10:46:19.512 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_inodestats": 0 2024-04-17T10:46:19.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.512 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log": { 2024-04-17T10:46:19.512 INFO:teuthology.orchestra.run.smithi083.stdout: "evlrg": 0, 2024-04-17T10:46:19.512 INFO:teuthology.orchestra.run.smithi083.stdout: "evadd": 25538, 2024-04-17T10:46:19.512 INFO:teuthology.orchestra.run.smithi083.stdout: "evex": 0, 2024-04-17T10:46:19.512 INFO:teuthology.orchestra.run.smithi083.stdout: "evtrm": 0, 2024-04-17T10:46:19.512 INFO:teuthology.orchestra.run.smithi083.stdout: "ev": 25538, 2024-04-17T10:46:19.512 INFO:teuthology.orchestra.run.smithi083.stdout: "evexg": 0, 2024-04-17T10:46:19.512 INFO:teuthology.orchestra.run.smithi083.stdout: "evexd": 0, 2024-04-17T10:46:19.512 INFO:teuthology.orchestra.run.smithi083.stdout: "segadd": 33, 2024-04-17T10:46:19.513 INFO:teuthology.orchestra.run.smithi083.stdout: "segex": 0, 2024-04-17T10:46:19.513 INFO:teuthology.orchestra.run.smithi083.stdout: "segtrm": 0, 2024-04-17T10:46:19.513 INFO:teuthology.orchestra.run.smithi083.stdout: "seg": 33, 2024-04-17T10:46:19.513 INFO:teuthology.orchestra.run.smithi083.stdout: "segmjr": 4, 2024-04-17T10:46:19.513 INFO:teuthology.orchestra.run.smithi083.stdout: "segexg": 0, 2024-04-17T10:46:19.513 INFO:teuthology.orchestra.run.smithi083.stdout: "segexd": 0, 2024-04-17T10:46:19.513 INFO:teuthology.orchestra.run.smithi083.stdout: "expos": 4194304, 2024-04-17T10:46:19.513 INFO:teuthology.orchestra.run.smithi083.stdout: "wrpos": 86521992, 2024-04-17T10:46:19.513 INFO:teuthology.orchestra.run.smithi083.stdout: "rdpos": 0, 2024-04-17T10:46:19.513 INFO:teuthology.orchestra.run.smithi083.stdout: "jlat": { 2024-04-17T10:46:19.513 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 7223, 2024-04-17T10:46:19.513 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 129.250565849, 2024-04-17T10:46:19.513 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.017894305 2024-04-17T10:46:19.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.514 INFO:teuthology.orchestra.run.smithi083.stdout: "replayed": 0 2024-04-17T10:46:19.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.514 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_mem": { 2024-04-17T10:46:19.514 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 14, 2024-04-17T10:46:19.514 INFO:teuthology.orchestra.run.smithi083.stdout: "ino+": 2015, 2024-04-17T10:46:19.514 INFO:teuthology.orchestra.run.smithi083.stdout: "ino-": 2001, 2024-04-17T10:46:19.514 INFO:teuthology.orchestra.run.smithi083.stdout: "dir": 13, 2024-04-17T10:46:19.514 INFO:teuthology.orchestra.run.smithi083.stdout: "dir+": 13, 2024-04-17T10:46:19.514 INFO:teuthology.orchestra.run.smithi083.stdout: "dir-": 0, 2024-04-17T10:46:19.514 INFO:teuthology.orchestra.run.smithi083.stdout: "dn": 17867, 2024-04-17T10:46:19.514 INFO:teuthology.orchestra.run.smithi083.stdout: "dn+": 19867, 2024-04-17T10:46:19.514 INFO:teuthology.orchestra.run.smithi083.stdout: "dn-": 2000, 2024-04-17T10:46:19.514 INFO:teuthology.orchestra.run.smithi083.stdout: "cap": 2, 2024-04-17T10:46:19.515 INFO:teuthology.orchestra.run.smithi083.stdout: "cap+": 2002, 2024-04-17T10:46:19.515 INFO:teuthology.orchestra.run.smithi083.stdout: "cap-": 2000, 2024-04-17T10:46:19.515 INFO:teuthology.orchestra.run.smithi083.stdout: "rss": 92616, 2024-04-17T10:46:19.515 INFO:teuthology.orchestra.run.smithi083.stdout: "heap": 215340 2024-04-17T10:46:19.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.515 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_server": { 2024-04-17T10:46:19.515 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_client_request": 36904, 2024-04-17T10:46:19.515 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_server_request": 0, 2024-04-17T10:46:19.515 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_request": 27238, 2024-04-17T10:46:19.515 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_session": 10, 2024-04-17T10:46:19.515 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_peer_request": 0, 2024-04-17T10:46:19.515 INFO:teuthology.orchestra.run.smithi083.stdout: "req_create_latency": { 2024-04-17T10:46:19.515 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2000, 2024-04-17T10:46:19.515 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 3.400930560, 2024-04-17T10:46:19.516 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.001700465 2024-04-17T10:46:19.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.516 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getattr_latency": { 2024-04-17T10:46:19.516 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 934, 2024-04-17T10:46:19.516 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.805390969, 2024-04-17T10:46:19.516 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000862302 2024-04-17T10:46:19.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.516 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getfilelock_latency": { 2024-04-17T10:46:19.516 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:46:19.516 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:46:19.516 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:46:19.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.516 INFO:teuthology.orchestra.run.smithi083.stdout: "req_link_latency": { 2024-04-17T10:46:19.516 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 6000, 2024-04-17T10:46:19.517 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 22.435763586, 2024-04-17T10:46:19.517 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.003739293 2024-04-17T10:46:19.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.517 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookup_latency": { 2024-04-17T10:46:19.517 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 8038, 2024-04-17T10:46:19.517 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 19.302557117, 2024-04-17T10:46:19.517 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.002401412 2024-04-17T10:46:19.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.517 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookuphash_latency": { 2024-04-17T10:46:19.517 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:46:19.517 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:46:19.517 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:46:19.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.518 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupino_latency": { 2024-04-17T10:46:19.518 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:46:19.518 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:46:19.518 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:46:19.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.518 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupname_latency": { 2024-04-17T10:46:19.518 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:46:19.518 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:46:19.518 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:46:19.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.518 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupparent_latency": { 2024-04-17T10:46:19.518 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:46:19.518 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:46:19.519 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:46:19.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.519 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupsnap_latency": { 2024-04-17T10:46:19.519 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3, 2024-04-17T10:46:19.519 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.106284228, 2024-04-17T10:46:19.519 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.035428076 2024-04-17T10:46:19.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.519 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lssnap_latency": { 2024-04-17T10:46:19.519 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:46:19.519 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:46:19.519 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:46:19.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.520 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mkdir_latency": { 2024-04-17T10:46:19.520 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1, 2024-04-17T10:46:19.520 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.004477250, 2024-04-17T10:46:19.520 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.004477250 2024-04-17T10:46:19.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.520 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mknod_latency": { 2024-04-17T10:46:19.520 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:46:19.520 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:46:19.520 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:46:19.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.520 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mksnap_latency": { 2024-04-17T10:46:19.520 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1, 2024-04-17T10:46:19.520 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.006911550, 2024-04-17T10:46:19.520 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.006911550 2024-04-17T10:46:19.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.521 INFO:teuthology.orchestra.run.smithi083.stdout: "req_open_latency": { 2024-04-17T10:46:19.521 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:46:19.521 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:46:19.521 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:46:19.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.521 INFO:teuthology.orchestra.run.smithi083.stdout: "req_readdir_latency": { 2024-04-17T10:46:19.521 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 6, 2024-04-17T10:46:19.521 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.245347336, 2024-04-17T10:46:19.521 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.040891222 2024-04-17T10:46:19.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.521 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rename_latency": { 2024-04-17T10:46:19.521 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2253, 2024-04-17T10:46:19.521 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 423.681745573, 2024-04-17T10:46:19.522 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.188052261 2024-04-17T10:46:19.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.522 INFO:teuthology.orchestra.run.smithi083.stdout: "req_renamesnap_latency": { 2024-04-17T10:46:19.522 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:46:19.522 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:46:19.522 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:46:19.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.522 INFO:teuthology.orchestra.run.smithi083.stdout: "req_snapdiff_latency": { 2024-04-17T10:46:19.522 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:46:19.522 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:46:19.522 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:46:19.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.522 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmdir_latency": { 2024-04-17T10:46:19.523 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:46:19.523 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:46:19.523 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:46:19.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.523 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmsnap_latency": { 2024-04-17T10:46:19.523 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1, 2024-04-17T10:46:19.523 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.036436369, 2024-04-17T10:46:19.523 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.036436369 2024-04-17T10:46:19.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.523 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmxattr_latency": { 2024-04-17T10:46:19.523 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:46:19.523 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:46:19.524 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:46:19.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.524 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setattr_latency": { 2024-04-17T10:46:19.524 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1, 2024-04-17T10:46:19.524 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000554538, 2024-04-17T10:46:19.524 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000554538 2024-04-17T10:46:19.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.524 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setdirlayout_latency": { 2024-04-17T10:46:19.524 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:46:19.524 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:46:19.524 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:46:19.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.525 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setfilelock_latency": { 2024-04-17T10:46:19.525 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:46:19.525 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:46:19.525 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:46:19.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.525 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setlayout_latency": { 2024-04-17T10:46:19.525 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:46:19.525 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:46:19.525 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:46:19.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.525 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setxattr_latency": { 2024-04-17T10:46:19.525 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:46:19.526 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:46:19.526 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:46:19.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.526 INFO:teuthology.orchestra.run.smithi083.stdout: "req_symlink_latency": { 2024-04-17T10:46:19.526 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:46:19.526 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:46:19.526 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:46:19.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.526 INFO:teuthology.orchestra.run.smithi083.stdout: "req_unlink_latency": { 2024-04-17T10:46:19.526 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 8000, 2024-04-17T10:46:19.526 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 36.663364200, 2024-04-17T10:46:19.526 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.004582920 2024-04-17T10:46:19.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.527 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_revoke_eviction": 0, 2024-04-17T10:46:19.527 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_acquisition_throttle": 0, 2024-04-17T10:46:19.527 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getvxattr_latency": { 2024-04-17T10:46:19.527 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:46:19.527 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:46:19.527 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:46:19.527 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:46:19.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.527 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_sessions": { 2024-04-17T10:46:19.527 INFO:teuthology.orchestra.run.smithi083.stdout: "session_count": 1, 2024-04-17T10:46:19.527 INFO:teuthology.orchestra.run.smithi083.stdout: "session_add": 1, 2024-04-17T10:46:19.528 INFO:teuthology.orchestra.run.smithi083.stdout: "session_remove": 0, 2024-04-17T10:46:19.528 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_open": 1, 2024-04-17T10:46:19.528 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_stale": 0, 2024-04-17T10:46:19.528 INFO:teuthology.orchestra.run.smithi083.stdout: "total_load": 13344, 2024-04-17T10:46:19.528 INFO:teuthology.orchestra.run.smithi083.stdout: "average_load": 13344, 2024-04-17T10:46:19.528 INFO:teuthology.orchestra.run.smithi083.stdout: "avg_session_uptime": 148, 2024-04-17T10:46:19.528 INFO:teuthology.orchestra.run.smithi083.stdout: "mdthresh_evicted": 0 2024-04-17T10:46:19.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.528 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-04-17T10:46:19.528 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-04-17T10:46:19.528 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-04-17T10:46:19.528 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T10:46:19.529 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-04-17T10:46:19.529 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T10:46:19.529 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-04-17T10:46:19.529 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T10:46:19.529 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T10:46:19.529 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T10:46:19.529 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T10:46:19.529 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T10:46:19.529 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-04-17T10:46:19.529 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T10:46:19.529 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T10:46:19.529 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-04-17T10:46:19.529 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-04-17T10:46:19.529 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-04-17T10:46:19.530 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-04-17T10:46:19.530 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T10:46:19.530 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T10:46:19.530 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T10:46:19.530 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T10:46:19.530 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T10:46:19.530 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-04-17T10:46:19.530 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-04-17T10:46:19.530 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-04-17T10:46:19.530 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T10:46:19.530 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T10:46:19.530 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-04-17T10:46:19.530 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-04-17T10:46:19.531 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-04-17T10:46:19.531 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-04-17T10:46:19.531 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T10:46:19.531 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-04-17T10:46:19.531 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T10:46:19.531 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-04-17T10:46:19.531 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 251802, 2024-04-17T10:46:19.531 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 90, 2024-04-17T10:46:19.531 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-04-17T10:46:19.531 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-04-17T10:46:19.531 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-04-17T10:46:19.531 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-04-17T10:46:19.531 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-04-17T10:46:19.531 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-04-17T10:46:19.532 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-04-17T10:46:19.532 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-04-17T10:46:19.532 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 5608, 2024-04-17T10:46:19.532 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 81, 2024-04-17T10:46:19.532 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T10:46:19.532 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-04-17T10:46:19.532 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-04-17T10:46:19.532 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-04-17T10:46:19.532 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 13453940, 2024-04-17T10:46:19.532 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 661255, 2024-04-17T10:46:19.532 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-04-17T10:46:19.532 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-04-17T10:46:19.532 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-04-17T10:46:19.533 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-04-17T10:46:19.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.533 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter": { 2024-04-17T10:46:19.533 INFO:teuthology.orchestra.run.smithi083.stdout: "op_active": 0, 2024-04-17T10:46:19.533 INFO:teuthology.orchestra.run.smithi083.stdout: "op_laggy": 0, 2024-04-17T10:46:19.533 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send": 13570, 2024-04-17T10:46:19.533 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send_bytes": 82925972, 2024-04-17T10:46:19.533 INFO:teuthology.orchestra.run.smithi083.stdout: "op_resend": 37, 2024-04-17T10:46:19.533 INFO:teuthology.orchestra.run.smithi083.stdout: "op_reply": 13570, 2024-04-17T10:46:19.533 INFO:teuthology.orchestra.run.smithi083.stdout: "op_latency": { 2024-04-17T10:46:19.533 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 13570, 2024-04-17T10:46:19.533 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 54.143437678, 2024-04-17T10:46:19.533 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.003989936 2024-04-17T10:46:19.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.534 INFO:teuthology.orchestra.run.smithi083.stdout: "op_inflight": 0, 2024-04-17T10:46:19.534 INFO:teuthology.orchestra.run.smithi083.stdout: "oplen_avg": { 2024-04-17T10:46:19.534 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 13570, 2024-04-17T10:46:19.534 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 13782 2024-04-17T10:46:19.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.534 INFO:teuthology.orchestra.run.smithi083.stdout: "op": 13570, 2024-04-17T10:46:19.534 INFO:teuthology.orchestra.run.smithi083.stdout: "op_r": 2150, 2024-04-17T10:46:19.534 INFO:teuthology.orchestra.run.smithi083.stdout: "op_w": 11420, 2024-04-17T10:46:19.534 INFO:teuthology.orchestra.run.smithi083.stdout: "op_rmw": 0, 2024-04-17T10:46:19.534 INFO:teuthology.orchestra.run.smithi083.stdout: "op_pg": 0, 2024-04-17T10:46:19.534 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_stat": 0, 2024-04-17T10:46:19.534 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_create": 11, 2024-04-17T10:46:19.535 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_read": 138, 2024-04-17T10:46:19.535 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_write": 7360, 2024-04-17T10:46:19.535 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writefull": 81, 2024-04-17T10:46:19.535 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writesame": 0, 2024-04-17T10:46:19.535 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_append": 0, 2024-04-17T10:46:19.535 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_zero": 1800, 2024-04-17T10:46:19.535 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_truncate": 0, 2024-04-17T10:46:19.535 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_delete": 2031, 2024-04-17T10:46:19.535 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_mapext": 0, 2024-04-17T10:46:19.535 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_sparse_read": 0, 2024-04-17T10:46:19.535 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_clonerange": 0, 2024-04-17T10:46:19.536 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_getxattr": 2012, 2024-04-17T10:46:19.536 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_setxattr": 22, 2024-04-17T10:46:19.536 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_cmpxattr": 0, 2024-04-17T10:46:19.536 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_rmxattr": 0, 2024-04-17T10:46:19.536 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_resetxattrs": 0, 2024-04-17T10:46:19.536 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_call": 0, 2024-04-17T10:46:19.536 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_watch": 0, 2024-04-17T10:46:19.536 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_notify": 0, 2024-04-17T10:46:19.536 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_src_cmpxattr": 0, 2024-04-17T10:46:19.536 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls": 0, 2024-04-17T10:46:19.536 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls_filter": 0, 2024-04-17T10:46:19.536 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_other": 0, 2024-04-17T10:46:19.536 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_active": 0, 2024-04-17T10:46:19.537 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_send": 0, 2024-04-17T10:46:19.537 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_resend": 0, 2024-04-17T10:46:19.537 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_ping": 0, 2024-04-17T10:46:19.537 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_active": 0, 2024-04-17T10:46:19.537 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_send": 0, 2024-04-17T10:46:19.537 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_resend": 0, 2024-04-17T10:46:19.537 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_active": 0, 2024-04-17T10:46:19.537 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_send": 0, 2024-04-17T10:46:19.537 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_resend": 0, 2024-04-17T10:46:19.537 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_active": 0, 2024-04-17T10:46:19.537 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_send": 0, 2024-04-17T10:46:19.537 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_resend": 0, 2024-04-17T10:46:19.537 INFO:teuthology.orchestra.run.smithi083.stdout: "command_active": 0, 2024-04-17T10:46:19.538 INFO:teuthology.orchestra.run.smithi083.stdout: "command_send": 0, 2024-04-17T10:46:19.538 INFO:teuthology.orchestra.run.smithi083.stdout: "command_resend": 0, 2024-04-17T10:46:19.538 INFO:teuthology.orchestra.run.smithi083.stdout: "map_epoch": 78, 2024-04-17T10:46:19.538 INFO:teuthology.orchestra.run.smithi083.stdout: "map_full": 0, 2024-04-17T10:46:19.538 INFO:teuthology.orchestra.run.smithi083.stdout: "map_inc": 5, 2024-04-17T10:46:19.538 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_sessions": 8, 2024-04-17T10:46:19.538 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_open": 8, 2024-04-17T10:46:19.538 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_close": 0, 2024-04-17T10:46:19.538 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_laggy": 0, 2024-04-17T10:46:19.538 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_wr": 260, 2024-04-17T10:46:19.538 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_rd": 0, 2024-04-17T10:46:19.538 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_del": 67 2024-04-17T10:46:19.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.539 INFO:teuthology.orchestra.run.smithi083.stdout: "oft": { 2024-04-17T10:46:19.539 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_objs": 1, 2024-04-17T10:46:19.539 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_kv_pairs": 0, 2024-04-17T10:46:19.539 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_updates": 0, 2024-04-17T10:46:19.539 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_removes": 0 2024-04-17T10:46:19.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.539 INFO:teuthology.orchestra.run.smithi083.stdout: "osd-slow-ops": { 2024-04-17T10:46:19.539 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_ops_count": 0 2024-04-17T10:46:19.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.539 INFO:teuthology.orchestra.run.smithi083.stdout: "purge_queue": { 2024-04-17T10:46:19.539 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops": 0, 2024-04-17T10:46:19.539 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops_high_water": 32, 2024-04-17T10:46:19.539 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing": 0, 2024-04-17T10:46:19.540 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_high_water": 32, 2024-04-17T10:46:19.540 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed_ops": 3800, 2024-04-17T10:46:19.540 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed": 3800, 2024-04-17T10:46:19.540 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_item_in_journal": 0 2024-04-17T10:46:19.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.540 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T10:46:19.540 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-17T10:46:19.540 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-17T10:46:19.540 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-17T10:46:19.540 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 46033, 2024-04-17T10:46:19.540 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 11490603, 2024-04-17T10:46:19.540 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-17T10:46:19.540 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 46033, 2024-04-17T10:46:19.540 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-17T10:46:19.541 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-17T10:46:19.541 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 46033, 2024-04-17T10:46:19.541 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 11490603, 2024-04-17T10:46:19.541 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-17T10:46:19.541 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:46:19.541 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:46:19.541 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:46:19.541 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:46:19.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.541 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_bytes": { 2024-04-17T10:46:19.541 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-17T10:46:19.542 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-17T10:46:19.542 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-17T10:46:19.542 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-17T10:46:19.542 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-17T10:46:19.542 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-17T10:46:19.542 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-17T10:46:19.542 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 13570, 2024-04-17T10:46:19.542 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 83324172, 2024-04-17T10:46:19.542 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 13570, 2024-04-17T10:46:19.542 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 83324172, 2024-04-17T10:46:19.542 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-17T10:46:19.542 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:46:19.543 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:46:19.543 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:46:19.543 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:46:19.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.543 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_ops": { 2024-04-17T10:46:19.543 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-17T10:46:19.543 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 1024, 2024-04-17T10:46:19.543 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-17T10:46:19.543 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-17T10:46:19.543 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-17T10:46:19.543 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-17T10:46:19.543 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-17T10:46:19.543 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 13570, 2024-04-17T10:46:19.543 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 13570, 2024-04-17T10:46:19.544 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 13570, 2024-04-17T10:46:19.544 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 13570, 2024-04-17T10:46:19.544 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-17T10:46:19.544 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:46:19.544 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:46:19.544 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:46:19.544 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:46:19.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.544 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle": { 2024-04-17T10:46:19.544 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-17T10:46:19.544 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-04-17T10:46:19.544 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-17T10:46:19.544 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 3800, 2024-04-17T10:46:19.544 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 398200, 2024-04-17T10:46:19.545 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-17T10:46:19.545 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 3800, 2024-04-17T10:46:19.545 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-17T10:46:19.545 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-17T10:46:19.545 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 137, 2024-04-17T10:46:19.545 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 398200, 2024-04-17T10:46:19.545 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-17T10:46:19.545 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:46:19.545 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:46:19.545 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:46:19.545 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:46:19.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:19.546 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle-0x557bcc38ea00": { 2024-04-17T10:46:19.546 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-17T10:46:19.546 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-04-17T10:46:19.546 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-17T10:46:19.546 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 25538, 2024-04-17T10:46:19.546 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 82327688, 2024-04-17T10:46:19.546 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-17T10:46:19.546 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 25538, 2024-04-17T10:46:19.546 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-17T10:46:19.546 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-17T10:46:19.546 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 7223, 2024-04-17T10:46:19.546 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 82327688, 2024-04-17T10:46:19.546 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-17T10:46:19.547 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-17T10:46:19.547 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-17T10:46:19.547 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-17T10:46:19.547 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:46:19.547 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:46:19.547 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:46:19.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.495+0000 7fe01d1f6640 1 -- 172.21.15.83:0/1540082596 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe004066f60 msgr2=0x7fe004069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:19.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.495+0000 7fe01d1f6640 1 --2- 172.21.15.83:0/1540082596 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe004066f60 0x7fe004069420 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe01815e270 tx=0x7fe00c008040 comp rx=0 tx=0).stop 2024-04-17T10:46:19.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.495+0000 7fe01d1f6640 1 -- 172.21.15.83:0/1540082596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0000052f0 msgr2=0x7fe0180ce1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:19.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.495+0000 7fe01d1f6640 1 --2- 172.21.15.83:0/1540082596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0000052f0 0x7fe0180ce1d0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fe00800d7a0 tx=0x7fe00800dc60 comp rx=0 tx=0).stop 2024-04-17T10:46:19.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.495+0000 7fe01d1f6640 1 -- 172.21.15.83:0/1540082596 shutdown_connections 2024-04-17T10:46:19.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.495+0000 7fe01d1f6640 1 --2- 172.21.15.83:0/1540082596 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe004066f60 0x7fe004069420 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.495+0000 7fe01d1f6640 1 --2- 172.21.15.83:0/1540082596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0180c4650 0x7fe0180c4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.495+0000 7fe01d1f6640 1 --2- 172.21.15.83:0/1540082596 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe01815d130 0x7fe01815d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.495+0000 7fe01d1f6640 1 --2- 172.21.15.83:0/1540082596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0000052f0 0x7fe0180ce1d0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.495+0000 7fe01d1f6640 1 -- 172.21.15.83:0/1540082596 >> 172.21.15.83:0/1540082596 conn(0x7fe0180b15b0 msgr2=0x7fe0180b06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:19.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.495+0000 7fe01d1f6640 1 -- 172.21.15.83:0/1540082596 shutdown_connections 2024-04-17T10:46:19.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.495+0000 7fe01d1f6640 1 -- 172.21.15.83:0/1540082596 wait complete. 2024-04-17T10:46:19.550 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T10:46:19.550 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:46:19.599 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:46:19.659 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T10:46:19.660 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:46:19.660 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:46:19.666 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:46:19.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.835+0000 7f0d0dfff640 1 Processor -- start 2024-04-17T10:46:19.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.835+0000 7f0d0dfff640 1 -- start start 2024-04-17T10:46:19.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.835+0000 7f0d0dfff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0d0815ba30 0x7f0d08072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.835+0000 7f0d0dfff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d0815c960 0x7f0d080cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.835+0000 7f0d0dfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0d080cfa10 0x7f0d0806c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.835+0000 7f0d0dfff640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d08071ea0 con 0x7f0d080cfa10 2024-04-17T10:46:19.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.835+0000 7f0d0dfff640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d080cfea0 con 0x7f0d0815ba30 2024-04-17T10:46:19.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.835+0000 7f0d0dfff640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0806e320 con 0x7f0d0815c960 2024-04-17T10:46:19.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.835+0000 7f0d07fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0d080cfa10 0x7f0d0806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.835+0000 7f0d07fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0d080cfa10 0x7f0d0806c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52292/0 (socket says 172.21.15.83:52292) 2024-04-17T10:46:19.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.835+0000 7f0d07fff640 1 -- 172.21.15.83:0/3274032874 learned_addr learned my addr 172.21.15.83:0/3274032874 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:19.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.835+0000 7f0d077fe640 1 --2- 172.21.15.83:0/3274032874 >> v2:172.21.15.167:3300/0 conn(0x7f0d0815ba30 0x7f0d08072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.835+0000 7f0d06ffd640 1 --2- 172.21.15.83:0/3274032874 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d0815c960 0x7f0d080cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.835+0000 7f0d07fff640 1 -- 172.21.15.83:0/3274032874 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d0815c960 msgr2=0x7f0d080cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:19.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.835+0000 7f0d07fff640 1 --2- 172.21.15.83:0/3274032874 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d0815c960 0x7f0d080cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.835+0000 7f0d07fff640 1 -- 172.21.15.83:0/3274032874 >> v2:172.21.15.167:3300/0 conn(0x7f0d0815ba30 msgr2=0x7f0d08072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:19.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.835+0000 7f0d07fff640 1 --2- 172.21.15.83:0/3274032874 >> v2:172.21.15.167:3300/0 conn(0x7f0d0815ba30 0x7f0d08072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.835+0000 7f0d07fff640 1 -- 172.21.15.83:0/3274032874 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d0806e040 con 0x7f0d080cfa10 2024-04-17T10:46:19.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.835+0000 7f0d077fe640 1 --2- 172.21.15.83:0/3274032874 >> v2:172.21.15.167:3300/0 conn(0x7f0d0815ba30 0x7f0d08072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:19.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.835+0000 7f0d07fff640 1 --2- 172.21.15.83:0/3274032874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d080cfa10 0x7f0d0806c490 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f0d080b2410 tx=0x7f0d0001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eaad0636604e106 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:19.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d067fc640 1 -- 172.21.15.83:0/3274032874 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d000a56d0 con 0x7f0d080cfa10 2024-04-17T10:46:19.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d067fc640 1 -- 172.21.15.83:0/3274032874 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d000a5870 con 0x7f0d080cfa10 2024-04-17T10:46:19.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d067fc640 1 -- 172.21.15.83:0/3274032874 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d000a5c40 con 0x7f0d080cfa10 2024-04-17T10:46:19.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d0dfff640 1 -- 172.21.15.83:0/3274032874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d080cfa10 msgr2=0x7f0d0806c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:19.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d0dfff640 1 --2- 172.21.15.83:0/3274032874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d080cfa10 0x7f0d0806c490 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f0d080b2410 tx=0x7f0d0001c960 comp rx=0 tx=0).stop 2024-04-17T10:46:19.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d0dfff640 1 -- 172.21.15.83:0/3274032874 shutdown_connections 2024-04-17T10:46:19.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d0dfff640 1 --2- 172.21.15.83:0/3274032874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d080cfa10 0x7f0d0806c490 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d0dfff640 1 --2- 172.21.15.83:0/3274032874 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d0815c960 0x7f0d080cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d0dfff640 1 --2- 172.21.15.83:0/3274032874 >> v2:172.21.15.167:3300/0 conn(0x7f0d0815ba30 0x7f0d08072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d0dfff640 1 -- 172.21.15.83:0/3274032874 >> 172.21.15.83:0/3274032874 conn(0x7f0d0806abe0 msgr2=0x7f0d0806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:19.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d0dfff640 1 -- 172.21.15.83:0/3274032874 shutdown_connections 2024-04-17T10:46:19.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d0dfff640 1 -- 172.21.15.83:0/3274032874 wait complete. 2024-04-17T10:46:19.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d0dfff640 1 Processor -- start 2024-04-17T10:46:19.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d0dfff640 1 -- start start 2024-04-17T10:46:19.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d0dfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d080cfa10 0x7f0d08150d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d0dfff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d0815ba30 0x7f0d0814b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d0dfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d0815c960 0x7f0d08151430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d0dfff640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0806de20 con 0x7f0d080cfa10 2024-04-17T10:46:19.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d0dfff640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0806dca0 con 0x7f0d0815c960 2024-04-17T10:46:19.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d0dfff640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0815cd40 con 0x7f0d0815ba30 2024-04-17T10:46:19.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d077fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d080cfa10 0x7f0d08150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d077fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d080cfa10 0x7f0d08150d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52302/0 (socket says 172.21.15.83:52302) 2024-04-17T10:46:19.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d077fe640 1 -- 172.21.15.83:0/267769701 learned_addr learned my addr 172.21.15.83:0/267769701 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:19.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d07fff640 1 --2- 172.21.15.83:0/267769701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d0815c960 0x7f0d08151430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d06ffd640 1 --2- 172.21.15.83:0/267769701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d0815ba30 0x7f0d0814b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d077fe640 1 -- 172.21.15.83:0/267769701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d0815ba30 msgr2=0x7f0d0814b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:19.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d077fe640 1 --2- 172.21.15.83:0/267769701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d0815ba30 0x7f0d0814b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d077fe640 1 -- 172.21.15.83:0/267769701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d0815c960 msgr2=0x7f0d08151430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:19.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d077fe640 1 --2- 172.21.15.83:0/267769701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d0815c960 0x7f0d08151430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:19.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d077fe640 1 -- 172.21.15.83:0/267769701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d0814c500 con 0x7f0d080cfa10 2024-04-17T10:46:19.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d07fff640 1 --2- 172.21.15.83:0/267769701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d0815c960 0x7f0d08151430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:19.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d077fe640 1 --2- 172.21.15.83:0/267769701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d080cfa10 0x7f0d08150d20 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800c7f0 tx=0x7f0cf800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:19.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0cfffff640 1 -- 172.21.15.83:0/267769701 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8015070 con 0x7f0d080cfa10 2024-04-17T10:46:19.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0cfffff640 1 -- 172.21.15.83:0/267769701 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf8007820 con 0x7f0d080cfa10 2024-04-17T10:46:19.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d0dfff640 1 -- 172.21.15.83:0/267769701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d0814c7e0 con 0x7f0d080cfa10 2024-04-17T10:46:19.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0cfffff640 1 -- 172.21.15.83:0/267769701 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8007c10 con 0x7f0d080cfa10 2024-04-17T10:46:19.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.839+0000 7f0d0dfff640 1 -- 172.21.15.83:0/267769701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d0814cda0 con 0x7f0d080cfa10 2024-04-17T10:46:19.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.843+0000 7f0d0dfff640 1 -- 172.21.15.83:0/267769701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d08151260 con 0x7f0d080cfa10 2024-04-17T10:46:19.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.843+0000 7f0cfffff640 1 -- 172.21.15.83:0/267769701 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0cf8018070 con 0x7f0d080cfa10 2024-04-17T10:46:19.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.843+0000 7f0cfffff640 1 --2- 172.21.15.83:0/267769701 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0ce8066f60 0x7f0ce8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:19.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.843+0000 7f0cfffff640 1 -- 172.21.15.83:0/267769701 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0cf8085530 con 0x7f0d080cfa10 2024-04-17T10:46:19.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.843+0000 7f0d06ffd640 1 --2- 172.21.15.83:0/267769701 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0ce8066f60 0x7f0ce8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:19.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.843+0000 7f0d06ffd640 1 --2- 172.21.15.83:0/267769701 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0ce8066f60 0x7f0ce8069420 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0d0806b100 tx=0x7f0cf0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:19.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.847+0000 7f0cfffff640 1 -- 172.21.15.83:0/267769701 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0cf80565c0 con 0x7f0d080cfa10 2024-04-17T10:46:19.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:19.999+0000 7f0d0dfff640 1 -- 172.21.15.83:0/267769701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d0815de00 con 0x7f0d080cfa10 2024-04-17T10:46:19.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.003+0000 7f0cfffff640 1 -- 172.21.15.83:0/267769701 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16759 (secure 0 0 0) 0x7f0cf8055ec0 con 0x7f0d080cfa10 2024-04-17T10:46:19.999 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:46:19.999 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":78,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:45:57.116959+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:43:32.090869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-17T10:43:32.815921+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":77,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.4","mappings":[{"from":6,"to":0}]},{"pgid":"11.2d","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:46:20.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.003+0000 7f0d0dfff640 1 -- 172.21.15.83:0/267769701 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0ce8066f60 msgr2=0x7f0ce8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:20.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.003+0000 7f0d0dfff640 1 --2- 172.21.15.83:0/267769701 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0ce8066f60 0x7f0ce8069420 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0d0806b100 tx=0x7f0cf0008040 comp rx=0 tx=0).stop 2024-04-17T10:46:20.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.003+0000 7f0d0dfff640 1 -- 172.21.15.83:0/267769701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d080cfa10 msgr2=0x7f0d08150d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:20.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.003+0000 7f0d0dfff640 1 --2- 172.21.15.83:0/267769701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d080cfa10 0x7f0d08150d20 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800c7f0 tx=0x7f0cf800ccb0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.003+0000 7f0d0dfff640 1 -- 172.21.15.83:0/267769701 shutdown_connections 2024-04-17T10:46:20.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.003+0000 7f0d0dfff640 1 --2- 172.21.15.83:0/267769701 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0ce8066f60 0x7f0ce8069420 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.003+0000 7f0d0dfff640 1 --2- 172.21.15.83:0/267769701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d0815c960 0x7f0d08151430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.003+0000 7f0d0dfff640 1 --2- 172.21.15.83:0/267769701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d0815ba30 0x7f0d0814b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.003+0000 7f0d0dfff640 1 --2- 172.21.15.83:0/267769701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d080cfa10 0x7f0d08150d20 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.003+0000 7f0d0dfff640 1 -- 172.21.15.83:0/267769701 >> 172.21.15.83:0/267769701 conn(0x7f0d0806abe0 msgr2=0x7f0d0815e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:20.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.003+0000 7f0d0dfff640 1 -- 172.21.15.83:0/267769701 shutdown_connections 2024-04-17T10:46:20.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.003+0000 7f0d0dfff640 1 -- 172.21.15.83:0/267769701 wait complete. 2024-04-17T10:46:20.017 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:43:32.090869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:43:32.815921+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 77, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:46:20.017 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:46:20.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.139+0000 7f2904ae2640 1 Processor -- start 2024-04-17T10:46:20.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.139+0000 7f2904ae2640 1 -- start start 2024-04-17T10:46:20.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.139+0000 7f2904ae2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f290015b6e0 0x7f2900071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:20.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.139+0000 7f2904ae2640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f290015bac0 0x7f29000cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:20.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.139+0000 7f2904ae2640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2900070470 0x7f290006ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:20.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.139+0000 7f2904ae2640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f290006c1f0 con 0x7f290015b6e0 2024-04-17T10:46:20.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.139+0000 7f2904ae2640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29000cdd30 con 0x7f290015bac0 2024-04-17T10:46:20.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.139+0000 7f2904ae2640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2900070a10 con 0x7f2900070470 2024-04-17T10:46:20.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.139+0000 7f28fe59c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f290015b6e0 0x7f2900071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:20.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.139+0000 7f28fe59c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f290015b6e0 0x7f2900071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52308/0 (socket says 172.21.15.83:52308) 2024-04-17T10:46:20.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.139+0000 7f28fe59c640 1 -- 172.21.15.83:0/1159411561 learned_addr learned my addr 172.21.15.83:0/1159411561 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:20.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.139+0000 7f28fed9d640 1 --2- 172.21.15.83:0/1159411561 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2900070470 0x7f290006ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:20.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.139+0000 7f28f5d9b640 1 --2- 172.21.15.83:0/1159411561 >> v2:172.21.15.167:3300/0 conn(0x7f290015bac0 0x7f29000cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:20.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.139+0000 7f28fe59c640 1 -- 172.21.15.83:0/1159411561 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2900070470 msgr2=0x7f290006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:20.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.139+0000 7f28fe59c640 1 --2- 172.21.15.83:0/1159411561 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2900070470 0x7f290006ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.139+0000 7f28fe59c640 1 -- 172.21.15.83:0/1159411561 >> v2:172.21.15.167:3300/0 conn(0x7f290015bac0 msgr2=0x7f29000cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:20.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.139+0000 7f28fe59c640 1 --2- 172.21.15.83:0/1159411561 >> v2:172.21.15.167:3300/0 conn(0x7f290015bac0 0x7f29000cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.139+0000 7f28fe59c640 1 -- 172.21.15.83:0/1159411561 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2900070cf0 con 0x7f290015b6e0 2024-04-17T10:46:20.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.139+0000 7f28fe59c640 1 --2- 172.21.15.83:0/1159411561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f290015b6e0 0x7f2900071cc0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f2900071c50 tx=0x7f28f001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=151022f958901fe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:20.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.143+0000 7f28fdd9b640 1 -- 172.21.15.83:0/1159411561 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f00a5780 con 0x7f290015b6e0 2024-04-17T10:46:20.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.143+0000 7f28fdd9b640 1 -- 172.21.15.83:0/1159411561 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f00a5920 con 0x7f290015b6e0 2024-04-17T10:46:20.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.143+0000 7f28fdd9b640 1 -- 172.21.15.83:0/1159411561 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f00a5cf0 con 0x7f290015b6e0 2024-04-17T10:46:20.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.143+0000 7f2904ae2640 1 -- 172.21.15.83:0/1159411561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f290015b6e0 msgr2=0x7f2900071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:20.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.143+0000 7f2904ae2640 1 --2- 172.21.15.83:0/1159411561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f290015b6e0 0x7f2900071cc0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f2900071c50 tx=0x7f28f001c9f0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.143+0000 7f2904ae2640 1 -- 172.21.15.83:0/1159411561 shutdown_connections 2024-04-17T10:46:20.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.143+0000 7f2904ae2640 1 --2- 172.21.15.83:0/1159411561 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2900070470 0x7f290006ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.143+0000 7f2904ae2640 1 --2- 172.21.15.83:0/1159411561 >> v2:172.21.15.167:3300/0 conn(0x7f290015bac0 0x7f29000cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.143+0000 7f2904ae2640 1 --2- 172.21.15.83:0/1159411561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f290015b6e0 0x7f2900071cc0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.143+0000 7f2904ae2640 1 -- 172.21.15.83:0/1159411561 >> 172.21.15.83:0/1159411561 conn(0x7f29000b15b0 msgr2=0x7f29000b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:20.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.143+0000 7f2904ae2640 1 -- 172.21.15.83:0/1159411561 shutdown_connections 2024-04-17T10:46:20.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.143+0000 7f2904ae2640 1 -- 172.21.15.83:0/1159411561 wait complete. 2024-04-17T10:46:20.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.143+0000 7f2904ae2640 1 Processor -- start 2024-04-17T10:46:20.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.143+0000 7f2904ae2640 1 -- start start 2024-04-17T10:46:20.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.143+0000 7f2904ae2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2900070470 0x7f29000aa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:20.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.143+0000 7f2904ae2640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f290015b6e0 0x7f2900157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:20.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.143+0000 7f2904ae2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f290015bac0 0x7f29001427a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:20.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.143+0000 7f2904ae2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290006d270 con 0x7f2900070470 2024-04-17T10:46:20.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.143+0000 7f2904ae2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290006d0f0 con 0x7f290015bac0 2024-04-17T10:46:20.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.143+0000 7f2904ae2640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f290006b760 con 0x7f290015b6e0 2024-04-17T10:46:20.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.143+0000 7f28fe59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2900070470 0x7f29000aa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:20.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.143+0000 7f28fe59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2900070470 0x7f29000aa9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52314/0 (socket says 172.21.15.83:52314) 2024-04-17T10:46:20.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.143+0000 7f28fe59c640 1 -- 172.21.15.83:0/3949728072 learned_addr learned my addr 172.21.15.83:0/3949728072 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:20.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.143+0000 7f28fe59c640 1 -- 172.21.15.83:0/3949728072 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f290015b6e0 msgr2=0x7f2900157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:20.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.143+0000 7f28fe59c640 1 --2- 172.21.15.83:0/3949728072 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f290015b6e0 0x7f2900157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.143+0000 7f28fe59c640 1 -- 172.21.15.83:0/3949728072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f290015bac0 msgr2=0x7f29001427a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:20.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.143+0000 7f28fe59c640 1 --2- 172.21.15.83:0/3949728072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f290015bac0 0x7f29001427a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.143+0000 7f28fe59c640 1 -- 172.21.15.83:0/3949728072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29000a9930 con 0x7f2900070470 2024-04-17T10:46:20.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.143+0000 7f28fe59c640 1 --2- 172.21.15.83:0/3949728072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2900070470 0x7f29000aa9d0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f28f00af040 tx=0x7f28f0019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:20.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.143+0000 7f28f77fe640 1 -- 172.21.15.83:0/3949728072 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f00bc020 con 0x7f2900070470 2024-04-17T10:46:20.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.147+0000 7f28f77fe640 1 -- 172.21.15.83:0/3949728072 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f00b86b0 con 0x7f2900070470 2024-04-17T10:46:20.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.147+0000 7f2904ae2640 1 -- 172.21.15.83:0/3949728072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29000a9c10 con 0x7f2900070470 2024-04-17T10:46:20.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.147+0000 7f28f77fe640 1 -- 172.21.15.83:0/3949728072 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f00b8aa0 con 0x7f2900070470 2024-04-17T10:46:20.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.147+0000 7f2904ae2640 1 -- 172.21.15.83:0/3949728072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29000aa1d0 con 0x7f2900070470 2024-04-17T10:46:20.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.147+0000 7f2904ae2640 1 -- 172.21.15.83:0/3949728072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2900074f90 con 0x7f2900070470 2024-04-17T10:46:20.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.147+0000 7f28f77fe640 1 -- 172.21.15.83:0/3949728072 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f28f00bf020 con 0x7f2900070470 2024-04-17T10:46:20.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.147+0000 7f28f77fe640 1 --2- 172.21.15.83:0/3949728072 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f28e0066f90 0x7f28e0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:20.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.147+0000 7f28f77fe640 1 -- 172.21.15.83:0/3949728072 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f28f011fe50 con 0x7f2900070470 2024-04-17T10:46:20.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.147+0000 7f28f5d9b640 1 --2- 172.21.15.83:0/3949728072 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f28e0066f90 0x7f28e0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:20.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.147+0000 7f28f5d9b640 1 --2- 172.21.15.83:0/3949728072 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f28e0066f90 0x7f28e0069450 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f29001f7b60 tx=0x7f28e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:20.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.151+0000 7f28f77fe640 1 -- 172.21.15.83:0/3949728072 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f28f00165e0 con 0x7f2900070470 2024-04-17T10:46:20.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.295+0000 7f2904ae2640 1 -- 172.21.15.83:0/3949728072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f290015e860 con 0x7f2900070470 2024-04-17T10:46:20.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.295+0000 7f28f77fe640 1 -- 172.21.15.83:0/3949728072 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f28f00f0e90 con 0x7f2900070470 2024-04-17T10:46:20.292 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:46:20.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.299+0000 7f2904ae2640 1 -- 172.21.15.83:0/3949728072 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f28e0066f90 msgr2=0x7f28e0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:20.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.299+0000 7f2904ae2640 1 --2- 172.21.15.83:0/3949728072 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f28e0066f90 0x7f28e0069450 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f29001f7b60 tx=0x7f28e8008040 comp rx=0 tx=0).stop 2024-04-17T10:46:20.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.299+0000 7f2904ae2640 1 -- 172.21.15.83:0/3949728072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2900070470 msgr2=0x7f29000aa9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:20.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.299+0000 7f2904ae2640 1 --2- 172.21.15.83:0/3949728072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2900070470 0x7f29000aa9d0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f28f00af040 tx=0x7f28f0019790 comp rx=0 tx=0).stop 2024-04-17T10:46:20.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.299+0000 7f2904ae2640 1 -- 172.21.15.83:0/3949728072 shutdown_connections 2024-04-17T10:46:20.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.299+0000 7f2904ae2640 1 --2- 172.21.15.83:0/3949728072 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f28e0066f90 0x7f28e0069450 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.299+0000 7f2904ae2640 1 --2- 172.21.15.83:0/3949728072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f290015bac0 0x7f29001427a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.299+0000 7f2904ae2640 1 --2- 172.21.15.83:0/3949728072 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f290015b6e0 0x7f2900157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.299+0000 7f2904ae2640 1 --2- 172.21.15.83:0/3949728072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2900070470 0x7f29000aa9d0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.299+0000 7f2904ae2640 1 -- 172.21.15.83:0/3949728072 >> 172.21.15.83:0/3949728072 conn(0x7f29000b15b0 msgr2=0x7f29000b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:20.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.299+0000 7f2904ae2640 1 -- 172.21.15.83:0/3949728072 shutdown_connections 2024-04-17T10:46:20.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.299+0000 7f2904ae2640 1 -- 172.21.15.83:0/3949728072 wait complete. 2024-04-17T10:46:20.309 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:46:20.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.435+0000 7f076fe8e640 1 Processor -- start 2024-04-17T10:46:20.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.435+0000 7f076fe8e640 1 -- start start 2024-04-17T10:46:20.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.435+0000 7f076fe8e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f07680d0970 0x7f07680d17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:20.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.435+0000 7f076fe8e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f07680ce4e0 0x7f07680d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:20.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.435+0000 7f076fe8e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07680d24f0 0x7f076815d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:20.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.435+0000 7f076fe8e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07680d2940 con 0x7f07680ce4e0 2024-04-17T10:46:20.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.435+0000 7f076fe8e640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07680cf7d0 con 0x7f07680d0970 2024-04-17T10:46:20.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.435+0000 7f076fe8e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07680ce940 con 0x7f07680d24f0 2024-04-17T10:46:20.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.435+0000 7f076d429640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f07680ce4e0 0x7f07680d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:20.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.435+0000 7f076d429640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f07680ce4e0 0x7f07680d1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52330/0 (socket says 172.21.15.83:52330) 2024-04-17T10:46:20.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.435+0000 7f076d429640 1 -- 172.21.15.83:0/1651052449 learned_addr learned my addr 172.21.15.83:0/1651052449 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:20.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.435+0000 7f076dc2a640 1 --2- 172.21.15.83:0/1651052449 >> v2:172.21.15.167:3300/0 conn(0x7f07680d0970 0x7f07680d17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:20.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.435+0000 7f076e42b640 1 --2- 172.21.15.83:0/1651052449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07680d24f0 0x7f076815d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:20.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.435+0000 7f076d429640 1 -- 172.21.15.83:0/1651052449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07680d24f0 msgr2=0x7f076815d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:20.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.435+0000 7f076d429640 1 --2- 172.21.15.83:0/1651052449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07680d24f0 0x7f076815d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.435+0000 7f076d429640 1 -- 172.21.15.83:0/1651052449 >> v2:172.21.15.167:3300/0 conn(0x7f07680d0970 msgr2=0x7f07680d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:20.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.435+0000 7f076d429640 1 --2- 172.21.15.83:0/1651052449 >> v2:172.21.15.167:3300/0 conn(0x7f07680d0970 0x7f07680d17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.435+0000 7f076d429640 1 -- 172.21.15.83:0/1651052449 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07680d4890 con 0x7f07680ce4e0 2024-04-17T10:46:20.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.439+0000 7f076d429640 1 --2- 172.21.15.83:0/1651052449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07680ce4e0 0x7f07680d1fb0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f076806af50 tx=0x7f076401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba47dc349e8da06 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:20.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.439+0000 7f076cc28640 1 -- 172.21.15.83:0/1651052449 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07640aa610 con 0x7f07680ce4e0 2024-04-17T10:46:20.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.439+0000 7f076cc28640 1 -- 172.21.15.83:0/1651052449 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07640af070 con 0x7f07680ce4e0 2024-04-17T10:46:20.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.439+0000 7f076cc28640 1 -- 172.21.15.83:0/1651052449 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07640aa9d0 con 0x7f07680ce4e0 2024-04-17T10:46:20.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.439+0000 7f076fe8e640 1 -- 172.21.15.83:0/1651052449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07680ce4e0 msgr2=0x7f07680d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:20.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.439+0000 7f076fe8e640 1 --2- 172.21.15.83:0/1651052449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07680ce4e0 0x7f07680d1fb0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f076806af50 tx=0x7f076401c4f0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.439+0000 7f076fe8e640 1 -- 172.21.15.83:0/1651052449 shutdown_connections 2024-04-17T10:46:20.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.439+0000 7f076fe8e640 1 --2- 172.21.15.83:0/1651052449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07680d24f0 0x7f076815d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.439+0000 7f076fe8e640 1 --2- 172.21.15.83:0/1651052449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07680ce4e0 0x7f07680d1fb0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.439+0000 7f076fe8e640 1 --2- 172.21.15.83:0/1651052449 >> v2:172.21.15.167:3300/0 conn(0x7f07680d0970 0x7f07680d17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.439+0000 7f076fe8e640 1 -- 172.21.15.83:0/1651052449 >> 172.21.15.83:0/1651052449 conn(0x7f076809cd50 msgr2=0x7f07680d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:20.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.439+0000 7f076fe8e640 1 -- 172.21.15.83:0/1651052449 shutdown_connections 2024-04-17T10:46:20.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.439+0000 7f076fe8e640 1 -- 172.21.15.83:0/1651052449 wait complete. 2024-04-17T10:46:20.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.443+0000 7f076fe8e640 1 Processor -- start 2024-04-17T10:46:20.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.443+0000 7f076fe8e640 1 -- start start 2024-04-17T10:46:20.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.443+0000 7f076fe8e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07680ce4e0 0x7f07680c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:20.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.443+0000 7f076fe8e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07680d0970 0x7f0768160830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:20.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.443+0000 7f076fe8e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07680d24f0 0x7f0768153960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:20.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.443+0000 7f076fe8e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07680d0d50 con 0x7f07680d24f0 2024-04-17T10:46:20.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.443+0000 7f076fe8e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07680cf460 con 0x7f07680d0970 2024-04-17T10:46:20.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.443+0000 7f076fe8e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0768161990 con 0x7f07680ce4e0 2024-04-17T10:46:20.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.443+0000 7f076e42b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07680d24f0 0x7f0768153960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:20.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.443+0000 7f076e42b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07680d24f0 0x7f0768153960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52344/0 (socket says 172.21.15.83:52344) 2024-04-17T10:46:20.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.443+0000 7f076e42b640 1 -- 172.21.15.83:0/807207164 learned_addr learned my addr 172.21.15.83:0/807207164 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:20.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.443+0000 7f076dc2a640 1 --2- 172.21.15.83:0/807207164 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07680ce4e0 0x7f07680c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:20.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.443+0000 7f076e42b640 1 -- 172.21.15.83:0/807207164 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07680ce4e0 msgr2=0x7f07680c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:20.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.443+0000 7f076e42b640 1 --2- 172.21.15.83:0/807207164 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07680ce4e0 0x7f07680c9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.443+0000 7f076d429640 1 --2- 172.21.15.83:0/807207164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07680d0970 0x7f0768160830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:20.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.443+0000 7f076e42b640 1 -- 172.21.15.83:0/807207164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07680d0970 msgr2=0x7f0768160830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:20.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.443+0000 7f076e42b640 1 --2- 172.21.15.83:0/807207164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07680d0970 0x7f0768160830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.443+0000 7f076e42b640 1 -- 172.21.15.83:0/807207164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07680b73b0 con 0x7f07680d24f0 2024-04-17T10:46:20.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.443+0000 7f076e42b640 1 --2- 172.21.15.83:0/807207164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07680d24f0 0x7f0768153960 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f0760007e60 tx=0x7f076000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:20.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.443+0000 7f075e7fc640 1 -- 172.21.15.83:0/807207164 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0760014420 con 0x7f07680d24f0 2024-04-17T10:46:20.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.443+0000 7f075e7fc640 1 -- 172.21.15.83:0/807207164 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0760019070 con 0x7f07680d24f0 2024-04-17T10:46:20.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.443+0000 7f076fe8e640 1 -- 172.21.15.83:0/807207164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07680b8b50 con 0x7f07680d24f0 2024-04-17T10:46:20.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.443+0000 7f075e7fc640 1 -- 172.21.15.83:0/807207164 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0760014820 con 0x7f07680d24f0 2024-04-17T10:46:20.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.443+0000 7f076fe8e640 1 -- 172.21.15.83:0/807207164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0768160030 con 0x7f07680d24f0 2024-04-17T10:46:20.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.443+0000 7f076fe8e640 1 -- 172.21.15.83:0/807207164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07680681a0 con 0x7f07680d24f0 2024-04-17T10:46:20.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.443+0000 7f075e7fc640 1 -- 172.21.15.83:0/807207164 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f07600149c0 con 0x7f07680d24f0 2024-04-17T10:46:20.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.443+0000 7f075e7fc640 1 --2- 172.21.15.83:0/807207164 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f074c066f60 0x7f074c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:20.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.443+0000 7f075e7fc640 1 -- 172.21.15.83:0/807207164 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0760085980 con 0x7f07680d24f0 2024-04-17T10:46:20.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.443+0000 7f076dc2a640 1 --2- 172.21.15.83:0/807207164 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f074c066f60 0x7f074c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:20.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.447+0000 7f076dc2a640 1 --2- 172.21.15.83:0/807207164 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f074c066f60 0x7f074c069420 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f076806ce70 tx=0x7f0758008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:20.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.451+0000 7f075e7fc640 1 -- 172.21.15.83:0/807207164 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0760056890 con 0x7f07680d24f0 2024-04-17T10:46:20.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.611+0000 7f076fe8e640 1 -- 172.21.15.83:0/807207164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0768074f90 con 0x7f07680d24f0 2024-04-17T10:46:20.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.611+0000 7f075e7fc640 1 -- 172.21.15.83:0/807207164 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0760056190 con 0x7f07680d24f0 2024-04-17T10:46:20.608 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:46:20.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.615+0000 7f076fe8e640 1 -- 172.21.15.83:0/807207164 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f074c066f60 msgr2=0x7f074c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:20.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.615+0000 7f076fe8e640 1 --2- 172.21.15.83:0/807207164 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f074c066f60 0x7f074c069420 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f076806ce70 tx=0x7f0758008040 comp rx=0 tx=0).stop 2024-04-17T10:46:20.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.615+0000 7f076fe8e640 1 -- 172.21.15.83:0/807207164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07680d24f0 msgr2=0x7f0768153960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:20.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.615+0000 7f076fe8e640 1 --2- 172.21.15.83:0/807207164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07680d24f0 0x7f0768153960 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f0760007e60 tx=0x7f076000e4b0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.615+0000 7f076fe8e640 1 -- 172.21.15.83:0/807207164 shutdown_connections 2024-04-17T10:46:20.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.615+0000 7f076fe8e640 1 --2- 172.21.15.83:0/807207164 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f074c066f60 0x7f074c069420 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.615+0000 7f076fe8e640 1 --2- 172.21.15.83:0/807207164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07680d24f0 0x7f0768153960 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.615+0000 7f076fe8e640 1 --2- 172.21.15.83:0/807207164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07680d0970 0x7f0768160830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.615+0000 7f076fe8e640 1 --2- 172.21.15.83:0/807207164 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07680ce4e0 0x7f07680c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.615+0000 7f076fe8e640 1 -- 172.21.15.83:0/807207164 >> 172.21.15.83:0/807207164 conn(0x7f076809cd50 msgr2=0x7f07680b3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:20.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.615+0000 7f076fe8e640 1 -- 172.21.15.83:0/807207164 shutdown_connections 2024-04-17T10:46:20.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.615+0000 7f076fe8e640 1 -- 172.21.15.83:0/807207164 wait complete. 2024-04-17T10:46:20.627 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:46:20.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.755+0000 7f548c5fe640 1 Processor -- start 2024-04-17T10:46:20.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.755+0000 7f548c5fe640 1 -- start start 2024-04-17T10:46:20.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.755+0000 7f548c5fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f54840d25b0 0x7f5484168b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:20.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.755+0000 7f548c5fe640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54840ce4e0 con 0x7f548415dab0 2024-04-17T10:46:20.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.755+0000 7f548c5fe640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5484162660 con 0x7f548415de90 2024-04-17T10:46:20.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.755+0000 7f548c5fe640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54840cf370 con 0x7f54840d25b0 2024-04-17T10:46:20.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.755+0000 7f548a39a640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f548415dab0 0x7f5484161840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:44104/0 (socket says 172.21.15.83:44104) 2024-04-17T10:46:20.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.755+0000 7f548a39a640 1 -- 172.21.15.83:0/3364312179 learned_addr learned my addr 172.21.15.83:0/3364312179 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:46:20.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.755+0000 7f548ab9b640 1 --2- 172.21.15.83:0/3364312179 >> v2:172.21.15.167:3300/0 conn(0x7f54840d25b0 0x7f5484168b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:20.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.755+0000 7f5489398640 1 -- 172.21.15.83:0/3364312179 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 262084130 0 0) 0x7f5480001120 con 0x7f548415dab0 2024-04-17T10:46:20.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.755+0000 7f5489398640 1 -- 172.21.15.83:0/3364312179 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5470003530 con 0x7f548415dab0 2024-04-17T10:46:20.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.755+0000 7f5489398640 1 -- 172.21.15.83:0/3364312179 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4255646363 0 0) 0x7f5470003530 con 0x7f548415dab0 2024-04-17T10:46:20.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.755+0000 7f5489398640 1 -- 172.21.15.83:0/3364312179 >> v2:172.21.15.167:3300/0 conn(0x7f54840d25b0 msgr2=0x7f5484168b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:20.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.755+0000 7f5489398640 1 --2- 172.21.15.83:0/3364312179 >> v2:172.21.15.167:3300/0 conn(0x7f54840d25b0 0x7f5484168b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.755+0000 7f5489398640 1 -- 172.21.15.83:0/3364312179 >> v1:172.21.15.167:6789/0 conn(0x7f548415de90 legacy=0x7f5484161f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:20.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.755+0000 7f5489398640 1 -- 172.21.15.83:0/3364312179 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54840ce760 con 0x7f548415dab0 2024-04-17T10:46:20.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.755+0000 7f5489398640 1 -- 172.21.15.83:0/3364312179 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f5480001850 con 0x7f548415dab0 2024-04-17T10:46:20.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.755+0000 7f5489398640 1 -- 172.21.15.83:0/3364312179 >> v1:172.21.15.83:6789/0 conn(0x7f548415dab0 legacy=0x7f5484161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:20.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.755+0000 7f5489398640 1 --2- 172.21.15.83:0/3364312179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5470003a30 0x7f5470003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:20.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.755+0000 7f5489398640 1 --2- 172.21.15.83:0/3364312179 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5470004810 0x7f5470004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:20.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.755+0000 7f5489398640 1 --2- 172.21.15.83:0/3364312179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54700051d0 0x7f547000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:20.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.755+0000 7f5489398640 1 -- 172.21.15.83:0/3364312179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547000c0d0 con 0x7f5470003a30 2024-04-17T10:46:20.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.755+0000 7f5489398640 1 -- 172.21.15.83:0/3364312179 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547000c280 con 0x7f54700051d0 2024-04-17T10:46:20.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.755+0000 7f5489398640 1 -- 172.21.15.83:0/3364312179 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f547000c480 con 0x7f5470004810 2024-04-17T10:46:20.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.755+0000 7f548a39a640 1 --2- 172.21.15.83:0/3364312179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5470003a30 0x7f5470003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:20.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.755+0000 7f5489b99640 1 --2- 172.21.15.83:0/3364312179 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5470004810 0x7f5470004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:20.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.755+0000 7f548a39a640 1 -- 172.21.15.83:0/3364312179 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5470004810 msgr2=0x7f5470004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:20.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.755+0000 7f548a39a640 1 --2- 172.21.15.83:0/3364312179 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5470004810 0x7f5470004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.755+0000 7f548a39a640 1 -- 172.21.15.83:0/3364312179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54700051d0 msgr2=0x7f547000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:20.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.755+0000 7f548a39a640 1 --2- 172.21.15.83:0/3364312179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54700051d0 0x7f547000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.755+0000 7f548ab9b640 1 -- 172.21.15.83:0/3364312179 reap_dead start 2024-04-17T10:46:20.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.755+0000 7f548a39a640 1 -- 172.21.15.83:0/3364312179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5470003530 con 0x7f5470003a30 2024-04-17T10:46:20.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.759+0000 7f548a39a640 1 --2- 172.21.15.83:0/3364312179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5470003a30 0x7f5470003e50 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f548000c4c0 tx=0x7f548001cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda312a4e3b1361e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:20.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.759+0000 7f5489398640 1 -- 172.21.15.83:0/3364312179 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548000c0e0 con 0x7f5470003a30 2024-04-17T10:46:20.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.759+0000 7f5489398640 1 -- 172.21.15.83:0/3364312179 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54800a9070 con 0x7f5470003a30 2024-04-17T10:46:20.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.759+0000 7f5489398640 1 -- 172.21.15.83:0/3364312179 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5480005d80 con 0x7f5470003a30 2024-04-17T10:46:20.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.759+0000 7f548c5fe640 1 -- 172.21.15.83:0/3364312179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5470003a30 msgr2=0x7f5470003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:20.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.759+0000 7f548c5fe640 1 --2- 172.21.15.83:0/3364312179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5470003a30 0x7f5470003e50 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f548000c4c0 tx=0x7f548001cca0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.759+0000 7f548c5fe640 1 -- 172.21.15.83:0/3364312179 shutdown_connections 2024-04-17T10:46:20.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.759+0000 7f548c5fe640 1 --2- 172.21.15.83:0/3364312179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5470003a30 0x7f5470003e50 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.759+0000 7f548c5fe640 1 -- 172.21.15.83:0/3364312179 >> 172.21.15.83:0/3364312179 conn(0x7f548409cd70 msgr2=0x7f54840d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:20.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.759+0000 7f548c5fe640 1 -- 172.21.15.83:0/3364312179 shutdown_connections 2024-04-17T10:46:20.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.759+0000 7f548c5fe640 1 -- 172.21.15.83:0/3364312179 wait complete. 2024-04-17T10:46:20.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.763+0000 7f548c5fe640 1 Processor -- start 2024-04-17T10:46:20.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.763+0000 7f548c5fe640 1 -- start start 2024-04-17T10:46:20.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.763+0000 7f548c5fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5470003a30 0x7f5484151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:20.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.763+0000 7f548c5fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54841527f0 0x7f548409ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:20.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.763+0000 7f548c5fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54840a04c0 0x7f548409ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:20.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.763+0000 7f548c5fe640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54840cf800 con 0x7f54841527f0 2024-04-17T10:46:20.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.763+0000 7f548c5fe640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54840d4910 con 0x7f5470003a30 2024-04-17T10:46:20.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.763+0000 7f548c5fe640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54840a0960 con 0x7f54840a04c0 2024-04-17T10:46:20.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.763+0000 7f5489b99640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54841527f0 0x7f548409ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:20.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.763+0000 7f5489b99640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54841527f0 0x7f548409ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52358/0 (socket says 172.21.15.83:52358) 2024-04-17T10:46:20.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.763+0000 7f5489b99640 1 -- 172.21.15.83:0/3674673811 learned_addr learned my addr 172.21.15.83:0/3674673811 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:20.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.763+0000 7f548a39a640 1 --2- 172.21.15.83:0/3674673811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5470003a30 0x7f5484151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:20.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.763+0000 7f548ab9b640 1 --2- 172.21.15.83:0/3674673811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54840a04c0 0x7f548409ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:20.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.763+0000 7f5489b99640 1 -- 172.21.15.83:0/3674673811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54840a04c0 msgr2=0x7f548409ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:20.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.763+0000 7f5489b99640 1 --2- 172.21.15.83:0/3674673811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54840a04c0 0x7f548409ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.763+0000 7f5489b99640 1 -- 172.21.15.83:0/3674673811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5470003a30 msgr2=0x7f5484151f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:20.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.763+0000 7f5489b99640 1 --2- 172.21.15.83:0/3674673811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5470003a30 0x7f5484151f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.763+0000 7f5489b99640 1 -- 172.21.15.83:0/3674673811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54840a0c40 con 0x7f54841527f0 2024-04-17T10:46:20.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.763+0000 7f5489b99640 1 --2- 172.21.15.83:0/3674673811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54841527f0 0x7f548409ff80 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f547400e4e0 tx=0x7f547400e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:20.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.763+0000 7f547affd640 1 -- 172.21.15.83:0/3674673811 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54740052e0 con 0x7f54841527f0 2024-04-17T10:46:20.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.763+0000 7f547affd640 1 -- 172.21.15.83:0/3674673811 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5474018070 con 0x7f54841527f0 2024-04-17T10:46:20.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.763+0000 7f547affd640 1 -- 172.21.15.83:0/3674673811 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5474005480 con 0x7f54841527f0 2024-04-17T10:46:20.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.763+0000 7f548c5fe640 1 -- 172.21.15.83:0/3674673811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54840a0f20 con 0x7f54841527f0 2024-04-17T10:46:20.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.763+0000 7f548c5fe640 1 -- 172.21.15.83:0/3674673811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54840a14e0 con 0x7f54841527f0 2024-04-17T10:46:20.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.763+0000 7f548c5fe640 1 -- 172.21.15.83:0/3674673811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54840681a0 con 0x7f54841527f0 2024-04-17T10:46:20.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.763+0000 7f547affd640 1 -- 172.21.15.83:0/3674673811 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5474003840 con 0x7f54841527f0 2024-04-17T10:46:20.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.763+0000 7f547affd640 1 --2- 172.21.15.83:0/3674673811 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5468066f60 0x7f5468069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:20.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.763+0000 7f547affd640 1 -- 172.21.15.83:0/3674673811 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f5474084c40 con 0x7f54841527f0 2024-04-17T10:46:20.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.763+0000 7f548a39a640 1 --2- 172.21.15.83:0/3674673811 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5468066f60 0x7f5468069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:20.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.767+0000 7f548a39a640 1 --2- 172.21.15.83:0/3674673811 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5468066f60 0x7f5468069420 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f548000c490 tx=0x7f54800a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:20.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.767+0000 7f547affd640 1 -- 172.21.15.83:0/3674673811 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5474055c80 con 0x7f54841527f0 2024-04-17T10:46:20.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.915+0000 7f548c5fe640 1 -- 172.21.15.83:0/3674673811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5484074f90 con 0x7f54841527f0 2024-04-17T10:46:20.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.915+0000 7f547affd640 1 -- 172.21.15.83:0/3674673811 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5474055580 con 0x7f54841527f0 2024-04-17T10:46:20.911 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:46:20.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.915+0000 7f548c5fe640 1 -- 172.21.15.83:0/3674673811 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5468066f60 msgr2=0x7f5468069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:20.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.915+0000 7f548c5fe640 1 --2- 172.21.15.83:0/3674673811 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5468066f60 0x7f5468069420 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f548000c490 tx=0x7f54800a7070 comp rx=0 tx=0).stop 2024-04-17T10:46:20.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.915+0000 7f548c5fe640 1 -- 172.21.15.83:0/3674673811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54841527f0 msgr2=0x7f548409ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:20.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.915+0000 7f548c5fe640 1 --2- 172.21.15.83:0/3674673811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54841527f0 0x7f548409ff80 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f547400e4e0 tx=0x7f547400e9a0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.915+0000 7f548c5fe640 1 -- 172.21.15.83:0/3674673811 shutdown_connections 2024-04-17T10:46:20.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.915+0000 7f548c5fe640 1 --2- 172.21.15.83:0/3674673811 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5468066f60 0x7f5468069420 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.915+0000 7f548c5fe640 1 --2- 172.21.15.83:0/3674673811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54840a04c0 0x7f548409ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.915+0000 7f548c5fe640 1 --2- 172.21.15.83:0/3674673811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54841527f0 0x7f548409ff80 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.915+0000 7f548c5fe640 1 --2- 172.21.15.83:0/3674673811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5470003a30 0x7f5484151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:20.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.915+0000 7f548c5fe640 1 -- 172.21.15.83:0/3674673811 >> 172.21.15.83:0/3674673811 conn(0x7f548409cd70 msgr2=0x7f548415ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:20.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.915+0000 7f548c5fe640 1 -- 172.21.15.83:0/3674673811 shutdown_connections 2024-04-17T10:46:20.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:20.915+0000 7f548c5fe640 1 -- 172.21.15.83:0/3674673811 wait complete. 2024-04-17T10:46:20.929 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:46:21.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.051+0000 7f06e9c80640 1 Processor -- start 2024-04-17T10:46:21.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.051+0000 7f06e9c80640 1 -- start start 2024-04-17T10:46:21.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.051+0000 7f06e9c80640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f06e406b390 0x7f06e40cfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:21.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.051+0000 7f06e9c80640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f06e40cff90 0x7f06e40cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:21.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.051+0000 7f06e9c80640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f06e40cdfe0 0x7f06e415f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:21.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.051+0000 7f06e9c80640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06e406b770 con 0x7f06e40cff90 2024-04-17T10:46:21.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.051+0000 7f06e9c80640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06e4072280 con 0x7f06e406b390 2024-04-17T10:46:21.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.051+0000 7f06e9c80640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e40716b0 con 0x7f06e40cdfe0 2024-04-17T10:46:21.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.051+0000 7f06dbfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f06e40cff90 0x7f06e40cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:21.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.051+0000 7f06dbfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f06e40cff90 0x7f06e40cdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52372/0 (socket says 172.21.15.83:52372) 2024-04-17T10:46:21.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.051+0000 7f06dbfff640 1 -- 172.21.15.83:0/1268139406 learned_addr learned my addr 172.21.15.83:0/1268139406 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:21.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.055+0000 7f06e37fe640 1 --2- 172.21.15.83:0/1268139406 >> v2:172.21.15.167:3300/0 conn(0x7f06e406b390 0x7f06e40cfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:21.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.055+0000 7f06e3fff640 1 --2- 172.21.15.83:0/1268139406 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f06e40cdfe0 0x7f06e415f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:21.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.055+0000 7f06dbfff640 1 -- 172.21.15.83:0/1268139406 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f06e40cdfe0 msgr2=0x7f06e415f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:21.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.055+0000 7f06dbfff640 1 --2- 172.21.15.83:0/1268139406 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f06e40cdfe0 0x7f06e415f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:21.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.055+0000 7f06dbfff640 1 -- 172.21.15.83:0/1268139406 >> v2:172.21.15.167:3300/0 conn(0x7f06e406b390 msgr2=0x7f06e40cfa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.055+0000 7f06dbfff640 1 --2- 172.21.15.83:0/1268139406 >> v2:172.21.15.167:3300/0 conn(0x7f06e406b390 0x7f06e40cfa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.055+0000 7f06dbfff640 1 -- 172.21.15.83:0/1268139406 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e4071950 con 0x7f06e40cff90 2024-04-17T10:46:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.055+0000 7f06dbfff640 1 --2- 172.21.15.83:0/1268139406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06e40cff90 0x7f06e40cdaa0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f06e4067770 tx=0x7f06cc01c1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c869fe042df338 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:21.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.055+0000 7f06e2ffd640 1 -- 172.21.15.83:0/1268139406 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06cc0b0070 con 0x7f06e40cff90 2024-04-17T10:46:21.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.055+0000 7f06e2ffd640 1 -- 172.21.15.83:0/1268139406 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06cc0ab600 con 0x7f06e40cff90 2024-04-17T10:46:21.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.055+0000 7f06e2ffd640 1 -- 172.21.15.83:0/1268139406 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06cc0ab9d0 con 0x7f06e40cff90 2024-04-17T10:46:21.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.055+0000 7f06e9c80640 1 -- 172.21.15.83:0/1268139406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06e40cff90 msgr2=0x7f06e40cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:21.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.055+0000 7f06e9c80640 1 --2- 172.21.15.83:0/1268139406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06e40cff90 0x7f06e40cdaa0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f06e4067770 tx=0x7f06cc01c1a0 comp rx=0 tx=0).stop 2024-04-17T10:46:21.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.055+0000 7f06e9c80640 1 -- 172.21.15.83:0/1268139406 shutdown_connections 2024-04-17T10:46:21.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.055+0000 7f06e9c80640 1 --2- 172.21.15.83:0/1268139406 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f06e40cdfe0 0x7f06e415f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:21.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.055+0000 7f06e9c80640 1 --2- 172.21.15.83:0/1268139406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06e40cff90 0x7f06e40cdaa0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:21.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.055+0000 7f06e9c80640 1 --2- 172.21.15.83:0/1268139406 >> v2:172.21.15.167:3300/0 conn(0x7f06e406b390 0x7f06e40cfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:21.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.055+0000 7f06e9c80640 1 -- 172.21.15.83:0/1268139406 >> 172.21.15.83:0/1268139406 conn(0x7f06e40b24f0 msgr2=0x7f06e40b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:21.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.055+0000 7f06e9c80640 1 -- 172.21.15.83:0/1268139406 shutdown_connections 2024-04-17T10:46:21.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.055+0000 7f06e9c80640 1 -- 172.21.15.83:0/1268139406 wait complete. 2024-04-17T10:46:21.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.059+0000 7f06e9c80640 1 Processor -- start 2024-04-17T10:46:21.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.059+0000 7f06e9c80640 1 -- start start 2024-04-17T10:46:21.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.059+0000 7f06e9c80640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06e406b390 0x7f06e415dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:21.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.059+0000 7f06e9c80640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06e40cdfe0 0x7f06e415d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:21.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.059+0000 7f06e9c80640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f06e40cff90 0x7f06e40aae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:21.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.059+0000 7f06e9c80640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e40ce570 con 0x7f06e406b390 2024-04-17T10:46:21.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.059+0000 7f06e9c80640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e40ce3f0 con 0x7f06e40cdfe0 2024-04-17T10:46:21.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.059+0000 7f06e9c80640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e40b3e90 con 0x7f06e40cff90 2024-04-17T10:46:21.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.059+0000 7f06e37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06e406b390 0x7f06e415dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:21.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.059+0000 7f06e37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06e406b390 0x7f06e415dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52384/0 (socket says 172.21.15.83:52384) 2024-04-17T10:46:21.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.059+0000 7f06e37fe640 1 -- 172.21.15.83:0/1890587800 learned_addr learned my addr 172.21.15.83:0/1890587800 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:21.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.059+0000 7f06e3fff640 1 --2- 172.21.15.83:0/1890587800 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f06e40cff90 0x7f06e40aae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:21.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.059+0000 7f06dbfff640 1 --2- 172.21.15.83:0/1890587800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06e40cdfe0 0x7f06e415d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:21.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.059+0000 7f06e37fe640 1 -- 172.21.15.83:0/1890587800 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f06e40cff90 msgr2=0x7f06e40aae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:21.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.059+0000 7f06e37fe640 1 --2- 172.21.15.83:0/1890587800 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f06e40cff90 0x7f06e40aae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:21.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.059+0000 7f06e37fe640 1 -- 172.21.15.83:0/1890587800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06e40cdfe0 msgr2=0x7f06e415d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:21.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.059+0000 7f06e37fe640 1 --2- 172.21.15.83:0/1890587800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06e40cdfe0 0x7f06e415d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:21.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.059+0000 7f06e37fe640 1 -- 172.21.15.83:0/1890587800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e40aa090 con 0x7f06e406b390 2024-04-17T10:46:21.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.059+0000 7f06e37fe640 1 --2- 172.21.15.83:0/1890587800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06e406b390 0x7f06e415dbe0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f06d400d800 tx=0x7f06d400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:21.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.059+0000 7f06e0ff9640 1 -- 172.21.15.83:0/1890587800 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d400b6c0 con 0x7f06e406b390 2024-04-17T10:46:21.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.059+0000 7f06e0ff9640 1 -- 172.21.15.83:0/1890587800 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d4019070 con 0x7f06e406b390 2024-04-17T10:46:21.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.059+0000 7f06e9c80640 1 -- 172.21.15.83:0/1890587800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06e40aa370 con 0x7f06e406b390 2024-04-17T10:46:21.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.059+0000 7f06e0ff9640 1 -- 172.21.15.83:0/1890587800 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d400ba60 con 0x7f06e406b390 2024-04-17T10:46:21.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.059+0000 7f06e9c80640 1 -- 172.21.15.83:0/1890587800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06e40aa930 con 0x7f06e406b390 2024-04-17T10:46:21.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.059+0000 7f06e9c80640 1 -- 172.21.15.83:0/1890587800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06e40683d0 con 0x7f06e406b390 2024-04-17T10:46:21.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.059+0000 7f06e0ff9640 1 -- 172.21.15.83:0/1890587800 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f06d400bc00 con 0x7f06e406b390 2024-04-17T10:46:21.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.059+0000 7f06e0ff9640 1 --2- 172.21.15.83:0/1890587800 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f06c4066f60 0x7f06c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:21.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.059+0000 7f06dbfff640 1 --2- 172.21.15.83:0/1890587800 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f06c4066f60 0x7f06c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:21.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.059+0000 7f06e0ff9640 1 -- 172.21.15.83:0/1890587800 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f06d40855d0 con 0x7f06e406b390 2024-04-17T10:46:21.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.063+0000 7f06dbfff640 1 --2- 172.21.15.83:0/1890587800 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f06c4066f60 0x7f06c4069420 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f06cc0ad000 tx=0x7f06cc0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.067+0000 7f06e0ff9640 1 -- 172.21.15.83:0/1890587800 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f06d4056580 con 0x7f06e406b390 2024-04-17T10:46:21.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.223+0000 7f06e9c80640 1 -- 172.21.15.83:0/1890587800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f06e4140160 con 0x7f06e406b390 2024-04-17T10:46:21.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.223+0000 7f06e0ff9640 1 -- 172.21.15.83:0/1890587800 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v78) v1 ==== 80+0+0 (secure 0 0 0) 0x7f06d4055e80 con 0x7f06e406b390 2024-04-17T10:46:21.221 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-17T10:46:21.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.227+0000 7f06e9c80640 1 -- 172.21.15.83:0/1890587800 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f06c4066f60 msgr2=0x7f06c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:21.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.227+0000 7f06e9c80640 1 --2- 172.21.15.83:0/1890587800 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f06c4066f60 0x7f06c4069420 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f06cc0ad000 tx=0x7f06cc0a6000 comp rx=0 tx=0).stop 2024-04-17T10:46:21.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.227+0000 7f06e9c80640 1 -- 172.21.15.83:0/1890587800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06e406b390 msgr2=0x7f06e415dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:21.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.227+0000 7f06e9c80640 1 --2- 172.21.15.83:0/1890587800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06e406b390 0x7f06e415dbe0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f06d400d800 tx=0x7f06d400dcc0 comp rx=0 tx=0).stop 2024-04-17T10:46:21.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.227+0000 7f06e9c80640 1 -- 172.21.15.83:0/1890587800 shutdown_connections 2024-04-17T10:46:21.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.227+0000 7f06e9c80640 1 --2- 172.21.15.83:0/1890587800 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f06c4066f60 0x7f06c4069420 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:21.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.227+0000 7f06e9c80640 1 --2- 172.21.15.83:0/1890587800 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f06e40cff90 0x7f06e40aae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:21.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.227+0000 7f06e9c80640 1 --2- 172.21.15.83:0/1890587800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06e40cdfe0 0x7f06e415d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:21.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.227+0000 7f06e9c80640 1 --2- 172.21.15.83:0/1890587800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06e406b390 0x7f06e415dbe0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:21.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.227+0000 7f06e9c80640 1 -- 172.21.15.83:0/1890587800 >> 172.21.15.83:0/1890587800 conn(0x7f06e40b24f0 msgr2=0x7f06e4144910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:21.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.227+0000 7f06e9c80640 1 -- 172.21.15.83:0/1890587800 shutdown_connections 2024-04-17T10:46:21.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.227+0000 7f06e9c80640 1 -- 172.21.15.83:0/1890587800 wait complete. 2024-04-17T10:46:21.238 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2024-04-17T10:46:21.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:21.238 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-17T10:46:21.250 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[56841]: fuse finished with error 0 and tester_r 0 2024-04-17T10:46:21.253 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-17T10:46:21.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:21.253 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T10:46:21.423 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:46:21.423 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:46:21.431 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:46:21.431 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-04-17T10:46:21.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:21.432 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-17T10:46:21.479 INFO:teuthology.orchestra.run.smithi167.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-17T10:46:21.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:46:21.480 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:46:21.480 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:46:21.486 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16063 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:46:21.487 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-17T10:46:21.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:21.487 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-17T10:46:21.534 INFO:teuthology.orchestra.run.smithi083.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-17T10:46:21.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:46:21.535 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:46:21.535 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:46:21.542 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:46:21.542 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-04-17T10:46:21.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:21.542 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-17T10:46:21.590 INFO:teuthology.orchestra.run.smithi167.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-17T10:46:21.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:46:21.592 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:46:21.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.727+0000 7ff486cec640 1 Processor -- start 2024-04-17T10:46:21.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.727+0000 7ff486cec640 1 -- start start 2024-04-17T10:46:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.727+0000 7ff486cec640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4800ce980 0x7ff4800b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.727+0000 7ff486cec640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff480162a00 con 0x7ff4800cee00 2024-04-17T10:46:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.727+0000 7ff486cec640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff480161e80 con 0x7ff48015dfa0 2024-04-17T10:46:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.727+0000 7ff486cec640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff480162160 con 0x7ff4800ce980 2024-04-17T10:46:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.727+0000 7ff485289640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff4800cee00 0x7ff4800d1d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:44114/0 (socket says 172.21.15.83:44114) 2024-04-17T10:46:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.727+0000 7ff485289640 1 -- 172.21.15.83:0/2515640471 learned_addr learned my addr 172.21.15.83:0/2515640471 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:46:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.727+0000 7ff47ffff640 1 --2- 172.21.15.83:0/2515640471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4800ce980 0x7ff4800b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.727+0000 7ff47f7fe640 1 -- 172.21.15.83:0/2515640471 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2679198172 0 0) 0x7ff480162a00 con 0x7ff4800cee00 2024-04-17T10:46:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.727+0000 7ff47f7fe640 1 -- 172.21.15.83:0/2515640471 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff470003530 con 0x7ff4800cee00 2024-04-17T10:46:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.727+0000 7ff47f7fe640 1 -- 172.21.15.83:0/2515640471 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2563364868 0 0) 0x7ff470003530 con 0x7ff4800cee00 2024-04-17T10:46:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.727+0000 7ff47f7fe640 1 -- 172.21.15.83:0/2515640471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4800ce980 msgr2=0x7ff4800b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:21.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.727+0000 7ff47f7fe640 1 --2- 172.21.15.83:0/2515640471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4800ce980 0x7ff4800b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.727+0000 7ff47f7fe640 1 -- 172.21.15.83:0/2515640471 >> v1:172.21.15.167:6789/0 conn(0x7ff48015dfa0 legacy=0x7ff4800b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.727+0000 7ff47f7fe640 1 -- 172.21.15.83:0/2515640471 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff480162440 con 0x7ff4800cee00 2024-04-17T10:46:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.727+0000 7ff47f7fe640 1 -- 172.21.15.83:0/2515640471 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7ff478001a00 con 0x7ff4800cee00 2024-04-17T10:46:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.727+0000 7ff47f7fe640 1 -- 172.21.15.83:0/2515640471 >> v1:172.21.15.83:6789/0 conn(0x7ff4800cee00 legacy=0x7ff4800d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.727+0000 7ff47f7fe640 1 --2- 172.21.15.83:0/2515640471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4800ce980 0x7ff4700039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.727+0000 7ff47f7fe640 1 --2- 172.21.15.83:0/2515640471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff470003f70 0x7ff470004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.727+0000 7ff47f7fe640 1 --2- 172.21.15.83:0/2515640471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff470004d50 0x7ff4700051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.727+0000 7ff47f7fe640 1 -- 172.21.15.83:0/2515640471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff480162160 con 0x7ff470004d50 2024-04-17T10:46:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.727+0000 7ff47f7fe640 1 -- 172.21.15.83:0/2515640471 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47000bb60 con 0x7ff470003f70 2024-04-17T10:46:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.727+0000 7ff47f7fe640 1 -- 172.21.15.83:0/2515640471 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47000bd60 con 0x7ff4800ce980 2024-04-17T10:46:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.727+0000 7ff485289640 1 --2- 172.21.15.83:0/2515640471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff470004d50 0x7ff4700051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.727+0000 7ff47ffff640 1 --2- 172.21.15.83:0/2515640471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff470003f70 0x7ff470004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:21.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.727+0000 7ff484a88640 1 --2- 172.21.15.83:0/2515640471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4800ce980 0x7ff4700039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:21.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.727+0000 7ff485289640 1 -- 172.21.15.83:0/2515640471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4800ce980 msgr2=0x7ff4700039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:21.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.727+0000 7ff485289640 1 --2- 172.21.15.83:0/2515640471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4800ce980 0x7ff4700039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:21.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.727+0000 7ff485289640 1 -- 172.21.15.83:0/2515640471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff470003f70 msgr2=0x7ff470004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:21.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.727+0000 7ff485289640 1 --2- 172.21.15.83:0/2515640471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff470003f70 0x7ff470004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:21.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.727+0000 7ff485289640 1 -- 172.21.15.83:0/2515640471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff470003530 con 0x7ff470004d50 2024-04-17T10:46:21.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.731+0000 7ff485289640 1 --2- 172.21.15.83:0/2515640471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff470004d50 0x7ff4700051d0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7ff47800a850 tx=0x7ff47801fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a8078fd67195dc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:21.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.731+0000 7ff47f7fe640 1 -- 172.21.15.83:0/2515640471 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47800a4a0 con 0x7ff470004d50 2024-04-17T10:46:21.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.731+0000 7ff47f7fe640 1 -- 172.21.15.83:0/2515640471 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4780ad070 con 0x7ff470004d50 2024-04-17T10:46:21.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.731+0000 7ff47f7fe640 1 -- 172.21.15.83:0/2515640471 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4780a9960 con 0x7ff470004d50 2024-04-17T10:46:21.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.731+0000 7ff486cec640 1 -- 172.21.15.83:0/2515640471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff470004d50 msgr2=0x7ff4700051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:21.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.731+0000 7ff486cec640 1 --2- 172.21.15.83:0/2515640471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff470004d50 0x7ff4700051d0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7ff47800a850 tx=0x7ff47801fb00 comp rx=0 tx=0).stop 2024-04-17T10:46:21.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.731+0000 7ff485289640 1 -- 172.21.15.83:0/2515640471 reap_dead start 2024-04-17T10:46:21.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.731+0000 7ff486cec640 1 -- 172.21.15.83:0/2515640471 shutdown_connections 2024-04-17T10:46:21.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.731+0000 7ff486cec640 1 -- 172.21.15.83:0/2515640471 >> 172.21.15.83:0/2515640471 conn(0x7ff4800d0600 msgr2=0x7ff4800d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:21.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.731+0000 7ff486cec640 1 -- 172.21.15.83:0/2515640471 shutdown_connections 2024-04-17T10:46:21.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.731+0000 7ff486cec640 1 -- 172.21.15.83:0/2515640471 wait complete. 2024-04-17T10:46:21.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.731+0000 7ff486cec640 1 Processor -- start 2024-04-17T10:46:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.731+0000 7ff486cec640 1 -- start start 2024-04-17T10:46:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.735+0000 7ff486cec640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff470004d50 0x7ff480159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.735+0000 7ff486cec640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4800ccb00 0x7ff48014e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.735+0000 7ff486cec640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4800caa80 0x7ff4800caf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.735+0000 7ff486cec640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff480161b10 con 0x7ff470004d50 2024-04-17T10:46:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.735+0000 7ff486cec640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48015e380 con 0x7ff4800caa80 2024-04-17T10:46:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.735+0000 7ff486cec640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4800d2440 con 0x7ff4800ccb00 2024-04-17T10:46:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.735+0000 7ff484a88640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff470004d50 0x7ff480159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.735+0000 7ff484a88640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff470004d50 0x7ff480159d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52408/0 (socket says 172.21.15.83:52408) 2024-04-17T10:46:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.735+0000 7ff484a88640 1 -- 172.21.15.83:0/1789282282 learned_addr learned my addr 172.21.15.83:0/1789282282 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:21.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.735+0000 7ff485289640 1 --2- 172.21.15.83:0/1789282282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4800caa80 0x7ff4800caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:21.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.735+0000 7ff47ffff640 1 --2- 172.21.15.83:0/1789282282 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4800ccb00 0x7ff48014e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:21.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.735+0000 7ff484a88640 1 -- 172.21.15.83:0/1789282282 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4800ccb00 msgr2=0x7ff48014e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:21.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.735+0000 7ff484a88640 1 --2- 172.21.15.83:0/1789282282 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4800ccb00 0x7ff48014e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:21.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.735+0000 7ff484a88640 1 -- 172.21.15.83:0/1789282282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4800caa80 msgr2=0x7ff4800caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:21.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.735+0000 7ff484a88640 1 --2- 172.21.15.83:0/1789282282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4800caa80 0x7ff4800caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:21.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.735+0000 7ff484a88640 1 -- 172.21.15.83:0/1789282282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48015a2d0 con 0x7ff470004d50 2024-04-17T10:46:21.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.735+0000 7ff484a88640 1 --2- 172.21.15.83:0/1789282282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff470004d50 0x7ff480159d90 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7ff47400e550 tx=0x7ff47400ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:21.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.735+0000 7ff47d7fa640 1 -- 172.21.15.83:0/1789282282 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4740052e0 con 0x7ff470004d50 2024-04-17T10:46:21.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.735+0000 7ff47d7fa640 1 -- 172.21.15.83:0/1789282282 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff474018070 con 0x7ff470004d50 2024-04-17T10:46:21.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.735+0000 7ff47d7fa640 1 -- 172.21.15.83:0/1789282282 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff474003820 con 0x7ff470004d50 2024-04-17T10:46:21.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.735+0000 7ff486cec640 1 -- 172.21.15.83:0/1789282282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff48015a5a0 con 0x7ff470004d50 2024-04-17T10:46:21.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.735+0000 7ff486cec640 1 -- 172.21.15.83:0/1789282282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff48015ab60 con 0x7ff470004d50 2024-04-17T10:46:21.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.735+0000 7ff486cec640 1 -- 172.21.15.83:0/1789282282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4800681a0 con 0x7ff470004d50 2024-04-17T10:46:21.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.735+0000 7ff47d7fa640 1 -- 172.21.15.83:0/1789282282 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff4740039c0 con 0x7ff470004d50 2024-04-17T10:46:21.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.735+0000 7ff47d7fa640 1 --2- 172.21.15.83:0/1789282282 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff464066f60 0x7ff464069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:21.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.735+0000 7ff47d7fa640 1 -- 172.21.15.83:0/1789282282 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff474084d20 con 0x7ff470004d50 2024-04-17T10:46:21.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.735+0000 7ff47ffff640 1 --2- 172.21.15.83:0/1789282282 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff464066f60 0x7ff464069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:21.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.735+0000 7ff47ffff640 1 --2- 172.21.15.83:0/1789282282 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff464066f60 0x7ff464069420 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff480153f70 tx=0x7ff46c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.739+0000 7ff47d7fa640 1 -- 172.21.15.83:0/1789282282 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff474055cd0 con 0x7ff470004d50 2024-04-17T10:46:21.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.931+0000 7ff486cec640 1 -- 172.21.15.83:0/1789282282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4800cb440 con 0x7ff470004d50 2024-04-17T10:46:21.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.931+0000 7ff47d7fa640 1 -- 172.21.15.83:0/1789282282 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4943 (secure 0 0 0) 0x7ff474005480 con 0x7ff470004d50 2024-04-17T10:46:21.928 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:46:21.928 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5258,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2259892875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2259892875},{"type":"v1","addr":"172.21.15.167:6835","nonce":2259892875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":7095,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/3769102527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3769102527},{"type":"v1","addr":"172.21.15.83:6836","nonce":3769102527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":7098,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/395594830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":395594830},{"type":"v1","addr":"172.21.15.83:6837","nonce":395594830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-17T10:43:33.830720+0000","modified":"2024-04-17T10:43:47.538354+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7104},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7104":{"gid":7104,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6833/2796648400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2796648400},{"type":"v1","addr":"172.21.15.167:6833","nonce":2796648400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-17T10:46:21.928 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 49 2024-04-17T10:46:21.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.931+0000 7ff486cec640 1 -- 172.21.15.83:0/1789282282 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff464066f60 msgr2=0x7ff464069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:21.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.931+0000 7ff486cec640 1 --2- 172.21.15.83:0/1789282282 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff464066f60 0x7ff464069420 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff480153f70 tx=0x7ff46c008040 comp rx=0 tx=0).stop 2024-04-17T10:46:21.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.931+0000 7ff486cec640 1 -- 172.21.15.83:0/1789282282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff470004d50 msgr2=0x7ff480159d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:21.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.931+0000 7ff486cec640 1 --2- 172.21.15.83:0/1789282282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff470004d50 0x7ff480159d90 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7ff47400e550 tx=0x7ff47400ea10 comp rx=0 tx=0).stop 2024-04-17T10:46:21.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.935+0000 7ff486cec640 1 -- 172.21.15.83:0/1789282282 shutdown_connections 2024-04-17T10:46:21.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.935+0000 7ff486cec640 1 --2- 172.21.15.83:0/1789282282 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff464066f60 0x7ff464069420 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:21.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.935+0000 7ff486cec640 1 --2- 172.21.15.83:0/1789282282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4800caa80 0x7ff4800caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:21.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.935+0000 7ff486cec640 1 --2- 172.21.15.83:0/1789282282 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4800ccb00 0x7ff48014e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:21.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.935+0000 7ff486cec640 1 --2- 172.21.15.83:0/1789282282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff470004d50 0x7ff480159d90 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:21.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.935+0000 7ff486cec640 1 -- 172.21.15.83:0/1789282282 >> 172.21.15.83:0/1789282282 conn(0x7ff4800d0600 msgr2=0x7ff4800d45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:21.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.935+0000 7ff486cec640 1 -- 172.21.15.83:0/1789282282 shutdown_connections 2024-04-17T10:46:21.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:21.935+0000 7ff486cec640 1 -- 172.21.15.83:0/1789282282 wait complete. 2024-04-17T10:46:21.945 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:46:22.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.055+0000 7f939e5c4640 1 Processor -- start 2024-04-17T10:46:22.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.055+0000 7f939e5c4640 1 -- start start 2024-04-17T10:46:22.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.059+0000 7f939e5c4640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f93981540d0 0x7f9398070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:22.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.059+0000 7f939e5c4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f939815ddb0 0x7f93980b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:22.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.059+0000 7f939e5c4640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f939806ce70 con 0x7f939815b270 2024-04-17T10:46:22.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.059+0000 7f939e5c4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9398072730 con 0x7f939815ddb0 2024-04-17T10:46:22.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.059+0000 7f939e5c4640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f939806d140 con 0x7f93981540d0 2024-04-17T10:46:22.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.059+0000 7f939cb61640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f939815ddb0 0x7f93980b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:22.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.059+0000 7f939cb61640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f939815ddb0 0x7f93980b28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52416/0 (socket says 172.21.15.83:52416) 2024-04-17T10:46:22.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.059+0000 7f939cb61640 1 -- 172.21.15.83:0/3801872198 learned_addr learned my addr 172.21.15.83:0/3801872198 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:22.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.059+0000 7f9397fff640 1 --2- 172.21.15.83:0/3801872198 >> v2:172.21.15.167:3300/0 conn(0x7f93981540d0 0x7f9398070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:22.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.059+0000 7f939cb61640 1 -- 172.21.15.83:0/3801872198 >> v2:172.21.15.167:3300/0 conn(0x7f93981540d0 msgr2=0x7f9398070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:22.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.059+0000 7f939cb61640 1 --2- 172.21.15.83:0/3801872198 >> v2:172.21.15.167:3300/0 conn(0x7f93981540d0 0x7f9398070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.059+0000 7f939cb61640 1 -- 172.21.15.83:0/3801872198 >> v1:172.21.15.167:6789/0 conn(0x7f939815b270 legacy=0x7f939815d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:22.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.059+0000 7f939cb61640 1 -- 172.21.15.83:0/3801872198 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93980720e0 con 0x7f939815ddb0 2024-04-17T10:46:22.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.059+0000 7f939cb61640 1 --2- 172.21.15.83:0/3801872198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f939815ddb0 0x7f93980b28e0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f9398067a50 tx=0x7f938801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fabaffa22af870e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:22.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.059+0000 7f9396ffd640 1 -- 172.21.15.83:0/3801872198 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93880ab640 con 0x7f939815ddb0 2024-04-17T10:46:22.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.059+0000 7f9396ffd640 1 -- 172.21.15.83:0/3801872198 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93880b0070 con 0x7f939815ddb0 2024-04-17T10:46:22.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.059+0000 7f9396ffd640 1 -- 172.21.15.83:0/3801872198 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93880aba00 con 0x7f939815ddb0 2024-04-17T10:46:22.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.059+0000 7f939e5c4640 1 -- 172.21.15.83:0/3801872198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f939815ddb0 msgr2=0x7f93980b28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:22.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.059+0000 7f939e5c4640 1 --2- 172.21.15.83:0/3801872198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f939815ddb0 0x7f93980b28e0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f9398067a50 tx=0x7f938801c4b0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.059+0000 7f939e5c4640 1 -- 172.21.15.83:0/3801872198 shutdown_connections 2024-04-17T10:46:22.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.063+0000 7f939e5c4640 1 --2- 172.21.15.83:0/3801872198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f939815ddb0 0x7f93980b28e0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.063+0000 7f939e5c4640 1 --2- 172.21.15.83:0/3801872198 >> v2:172.21.15.167:3300/0 conn(0x7f93981540d0 0x7f9398070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.063+0000 7f939e5c4640 1 -- 172.21.15.83:0/3801872198 >> 172.21.15.83:0/3801872198 conn(0x7f939806abe0 msgr2=0x7f939806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:22.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.063+0000 7f939e5c4640 1 -- 172.21.15.83:0/3801872198 shutdown_connections 2024-04-17T10:46:22.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.063+0000 7f939e5c4640 1 -- 172.21.15.83:0/3801872198 wait complete. 2024-04-17T10:46:22.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.063+0000 7f939e5c4640 1 Processor -- start 2024-04-17T10:46:22.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.063+0000 7f939e5c4640 1 -- start start 2024-04-17T10:46:22.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.063+0000 7f939e5c4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93981540d0 0x7f939814ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:22.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.063+0000 7f939e5c4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f939815b270 0x7f939814e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:22.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.063+0000 7f939e5c4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f939815ddb0 0x7f9398137cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:22.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.063+0000 7f939e5c4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93980709b0 con 0x7f939815ddb0 2024-04-17T10:46:22.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.063+0000 7f939e5c4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939815e380 con 0x7f939815b270 2024-04-17T10:46:22.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.063+0000 7f939e5c4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f939814b0c0 con 0x7f93981540d0 2024-04-17T10:46:22.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.063+0000 7f939cb61640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f939815ddb0 0x7f9398137cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:22.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.063+0000 7f939cb61640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f939815ddb0 0x7f9398137cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52422/0 (socket says 172.21.15.83:52422) 2024-04-17T10:46:22.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.063+0000 7f939cb61640 1 -- 172.21.15.83:0/1294757663 learned_addr learned my addr 172.21.15.83:0/1294757663 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:22.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.063+0000 7f93977fe640 1 --2- 172.21.15.83:0/1294757663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f939815b270 0x7f939814e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:22.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.063+0000 7f9397fff640 1 --2- 172.21.15.83:0/1294757663 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93981540d0 0x7f939814ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:22.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.063+0000 7f939cb61640 1 -- 172.21.15.83:0/1294757663 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93981540d0 msgr2=0x7f939814ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:22.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.063+0000 7f939cb61640 1 --2- 172.21.15.83:0/1294757663 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93981540d0 0x7f939814ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.063+0000 7f939cb61640 1 -- 172.21.15.83:0/1294757663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f939815b270 msgr2=0x7f939814e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:22.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.063+0000 7f939cb61640 1 --2- 172.21.15.83:0/1294757663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f939815b270 0x7f939814e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.063+0000 7f939cb61640 1 -- 172.21.15.83:0/1294757663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939814f270 con 0x7f939815ddb0 2024-04-17T10:46:22.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.063+0000 7f9397fff640 1 --2- 172.21.15.83:0/1294757663 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93981540d0 0x7f939814ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:22.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.063+0000 7f939cb61640 1 --2- 172.21.15.83:0/1294757663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f939815ddb0 0x7f9398137cd0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f93880ad040 tx=0x7f938801acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.063+0000 7f9394ff9640 1 -- 172.21.15.83:0/1294757663 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93880be3f0 con 0x7f939815ddb0 2024-04-17T10:46:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.063+0000 7f9394ff9640 1 -- 172.21.15.83:0/1294757663 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93880be5b0 con 0x7f939815ddb0 2024-04-17T10:46:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.063+0000 7f939e5c4640 1 -- 172.21.15.83:0/1294757663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f939814f550 con 0x7f939815ddb0 2024-04-17T10:46:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.063+0000 7f9394ff9640 1 -- 172.21.15.83:0/1294757663 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93880be9a0 con 0x7f939815ddb0 2024-04-17T10:46:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.063+0000 7f939e5c4640 1 -- 172.21.15.83:0/1294757663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9398135e10 con 0x7f939815ddb0 2024-04-17T10:46:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.063+0000 7f939e5c4640 1 -- 172.21.15.83:0/1294757663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9398074f90 con 0x7f939815ddb0 2024-04-17T10:46:22.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.067+0000 7f9394ff9640 1 -- 172.21.15.83:0/1294757663 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f93880c7430 con 0x7f939815ddb0 2024-04-17T10:46:22.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.067+0000 7f9394ff9640 1 --2- 172.21.15.83:0/1294757663 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9384066f60 0x7f9384069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:22.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.067+0000 7f9397fff640 1 --2- 172.21.15.83:0/1294757663 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9384066f60 0x7f9384069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:22.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.067+0000 7f9394ff9640 1 -- 172.21.15.83:0/1294757663 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f938811f920 con 0x7f939815ddb0 2024-04-17T10:46:22.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.067+0000 7f9397fff640 1 --2- 172.21.15.83:0/1294757663 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9384066f60 0x7f9384069420 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9398166910 tx=0x7f938c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:22.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.071+0000 7f9394ff9640 1 -- 172.21.15.83:0/1294757663 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f93880160d0 con 0x7f939815ddb0 2024-04-17T10:46:22.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.219+0000 7f939e5c4640 1 -- 172.21.15.83:0/1294757663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9398074f90 con 0x7f939815ddb0 2024-04-17T10:46:22.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.219+0000 7f9394ff9640 1 -- 172.21.15.83:0/1294757663 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16759 (secure 0 0 0) 0x7f93880f0960 con 0x7f939815ddb0 2024-04-17T10:46:22.216 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:46:22.217 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":78,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:45:57.116959+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:43:32.090869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-17T10:43:32.815921+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":77,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.4","mappings":[{"from":6,"to":0}]},{"pgid":"11.2d","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:46:22.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.223+0000 7f939e5c4640 1 -- 172.21.15.83:0/1294757663 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9384066f60 msgr2=0x7f9384069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.223+0000 7f939e5c4640 1 --2- 172.21.15.83:0/1294757663 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9384066f60 0x7f9384069420 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9398166910 tx=0x7f938c008040 comp rx=0 tx=0).stop 2024-04-17T10:46:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.223+0000 7f939e5c4640 1 -- 172.21.15.83:0/1294757663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f939815ddb0 msgr2=0x7f9398137cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.223+0000 7f939e5c4640 1 --2- 172.21.15.83:0/1294757663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f939815ddb0 0x7f9398137cd0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f93880ad040 tx=0x7f938801acf0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.223+0000 7f939e5c4640 1 -- 172.21.15.83:0/1294757663 shutdown_connections 2024-04-17T10:46:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.223+0000 7f939e5c4640 1 --2- 172.21.15.83:0/1294757663 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9384066f60 0x7f9384069420 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.223+0000 7f939e5c4640 1 --2- 172.21.15.83:0/1294757663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f939815ddb0 0x7f9398137cd0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.223+0000 7f939e5c4640 1 --2- 172.21.15.83:0/1294757663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f939815b270 0x7f939814e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.223+0000 7f939e5c4640 1 --2- 172.21.15.83:0/1294757663 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93981540d0 0x7f939814ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.223+0000 7f939e5c4640 1 -- 172.21.15.83:0/1294757663 >> 172.21.15.83:0/1294757663 conn(0x7f939806abe0 msgr2=0x7f939814bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.223+0000 7f939e5c4640 1 -- 172.21.15.83:0/1294757663 shutdown_connections 2024-04-17T10:46:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.223+0000 7f939e5c4640 1 -- 172.21.15.83:0/1294757663 wait complete. 2024-04-17T10:46:22.235 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:43:32.090869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:43:32.815921+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 77, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:46:22.235 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:46:22.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.351+0000 7f96195b8640 1 Processor -- start 2024-04-17T10:46:22.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.351+0000 7f96195b8640 1 -- start start 2024-04-17T10:46:22.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.351+0000 7f96195b8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f961406ae10 0x7f961406b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:22.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.351+0000 7f96195b8640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9614070aa0 0x7f9614070f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.351+0000 7f96195b8640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96141625d0 con 0x7f961415e330 2024-04-17T10:46:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.351+0000 7f96195b8640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f961415ee80 con 0x7f9614070aa0 2024-04-17T10:46:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.351+0000 7f96195b8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f961406b730 con 0x7f961406ae10 2024-04-17T10:46:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.351+0000 7f96137fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9614070aa0 0x7f9614070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.351+0000 7f96137fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9614070aa0 0x7f9614070f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52428/0 (socket says 172.21.15.83:52428) 2024-04-17T10:46:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.351+0000 7f96137fe640 1 -- 172.21.15.83:0/1842930640 learned_addr learned my addr 172.21.15.83:0/1842930640 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.351+0000 7f960bfff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f961415e330 0x7f961415e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:44128/0 (socket says 172.21.15.83:44128) 2024-04-17T10:46:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.351+0000 7f9612ffd640 1 --2- 172.21.15.83:0/1842930640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f961406ae10 0x7f961406b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.351+0000 7f96137fe640 1 -- 172.21.15.83:0/1842930640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f961406ae10 msgr2=0x7f961406b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.351+0000 7f96137fe640 1 --2- 172.21.15.83:0/1842930640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f961406ae10 0x7f961406b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.351+0000 7f96137fe640 1 -- 172.21.15.83:0/1842930640 >> v1:172.21.15.83:6789/0 conn(0x7f961415e330 legacy=0x7f961415e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.351+0000 7f96137fe640 1 -- 172.21.15.83:0/1842930640 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9614161810 con 0x7f9614070aa0 2024-04-17T10:46:22.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.351+0000 7f96127fc640 1 -- 172.21.15.83:0/1842930640 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2888380908 0 0) 0x7f96141625d0 con 0x7f961415e330 2024-04-17T10:46:22.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f96137fe640 1 --2- 172.21.15.83:0/1842930640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9614070aa0 0x7f9614070f30 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f960c009e50 tx=0x7f960c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d887bb860c38cc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:22.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f96127fc640 1 -- 172.21.15.83:0/1842930640 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c0a5690 con 0x7f9614070aa0 2024-04-17T10:46:22.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f96127fc640 1 -- 172.21.15.83:0/1842930640 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960c0a5830 con 0x7f9614070aa0 2024-04-17T10:46:22.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f96127fc640 1 -- 172.21.15.83:0/1842930640 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c0a5c00 con 0x7f9614070aa0 2024-04-17T10:46:22.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f96195b8640 1 -- 172.21.15.83:0/1842930640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9614070aa0 msgr2=0x7f9614070f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:22.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f96195b8640 1 --2- 172.21.15.83:0/1842930640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9614070aa0 0x7f9614070f30 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f960c009e50 tx=0x7f960c01c8e0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f96195b8640 1 -- 172.21.15.83:0/1842930640 shutdown_connections 2024-04-17T10:46:22.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f96195b8640 1 --2- 172.21.15.83:0/1842930640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9614070aa0 0x7f9614070f30 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f96195b8640 1 --2- 172.21.15.83:0/1842930640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f961406ae10 0x7f961406b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f96195b8640 1 -- 172.21.15.83:0/1842930640 >> 172.21.15.83:0/1842930640 conn(0x7f96140b15b0 msgr2=0x7f96140b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:22.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f96195b8640 1 -- 172.21.15.83:0/1842930640 shutdown_connections 2024-04-17T10:46:22.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f96195b8640 1 -- 172.21.15.83:0/1842930640 wait complete. 2024-04-17T10:46:22.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f96195b8640 1 Processor -- start 2024-04-17T10:46:22.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f96195b8640 1 -- start start 2024-04-17T10:46:22.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f96195b8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f961406ae10 0x7f9614137c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:22.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f96195b8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9614070aa0 0x7f961413e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:22.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f96195b8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f961415e330 0x7f9614156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:22.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f96195b8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96140cf470 con 0x7f961406ae10 2024-04-17T10:46:22.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f96195b8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96140cf2f0 con 0x7f961415e330 2024-04-17T10:46:22.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f96195b8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f961416aa10 con 0x7f9614070aa0 2024-04-17T10:46:22.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f9612ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f961406ae10 0x7f9614137c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:22.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f9612ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f961406ae10 0x7f9614137c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52440/0 (socket says 172.21.15.83:52440) 2024-04-17T10:46:22.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f9612ffd640 1 -- 172.21.15.83:0/1093258471 learned_addr learned my addr 172.21.15.83:0/1093258471 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:22.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f9612ffd640 1 -- 172.21.15.83:0/1093258471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9614070aa0 msgr2=0x7f961413e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:22.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f9612ffd640 1 --2- 172.21.15.83:0/1093258471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9614070aa0 0x7f961413e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f9612ffd640 1 -- 172.21.15.83:0/1093258471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f961415e330 msgr2=0x7f9614156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:22.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f9612ffd640 1 --2- 172.21.15.83:0/1093258471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f961415e330 0x7f9614156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f9612ffd640 1 -- 172.21.15.83:0/1093258471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f961413cdb0 con 0x7f961406ae10 2024-04-17T10:46:22.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f9612ffd640 1 --2- 172.21.15.83:0/1093258471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f961406ae10 0x7f9614137c20 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00c820 tx=0x7f95fc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:22.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f960b7fe640 1 -- 172.21.15.83:0/1093258471 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc015070 con 0x7f961406ae10 2024-04-17T10:46:22.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f960b7fe640 1 -- 172.21.15.83:0/1093258471 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95fc007a30 con 0x7f961406ae10 2024-04-17T10:46:22.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f96195b8640 1 -- 172.21.15.83:0/1093258471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f961413d090 con 0x7f961406ae10 2024-04-17T10:46:22.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f960b7fe640 1 -- 172.21.15.83:0/1093258471 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc007e20 con 0x7f961406ae10 2024-04-17T10:46:22.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.355+0000 7f96195b8640 1 -- 172.21.15.83:0/1093258471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f961415be90 con 0x7f961406ae10 2024-04-17T10:46:22.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.359+0000 7f96195b8640 1 -- 172.21.15.83:0/1093258471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9614074f90 con 0x7f961406ae10 2024-04-17T10:46:22.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.359+0000 7f960b7fe640 1 -- 172.21.15.83:0/1093258471 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f95fc018070 con 0x7f961406ae10 2024-04-17T10:46:22.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.359+0000 7f960b7fe640 1 --2- 172.21.15.83:0/1093258471 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f95f4066f60 0x7f95f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:22.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.359+0000 7f960b7fe640 1 -- 172.21.15.83:0/1093258471 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f95fc084f50 con 0x7f961406ae10 2024-04-17T10:46:22.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.359+0000 7f960bfff640 1 --2- 172.21.15.83:0/1093258471 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f95f4066f60 0x7f95f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:22.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.359+0000 7f960bfff640 1 --2- 172.21.15.83:0/1093258471 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f95f4066f60 0x7f95f4069420 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f96040011f0 tx=0x7f9604007930 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:22.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.363+0000 7f960b7fe640 1 -- 172.21.15.83:0/1093258471 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f95fc055ef0 con 0x7f961406ae10 2024-04-17T10:46:22.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.523+0000 7f96195b8640 1 -- 172.21.15.83:0/1093258471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9614072740 con 0x7f961406ae10 2024-04-17T10:46:22.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.523+0000 7f960b7fe640 1 -- 172.21.15.83:0/1093258471 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f95fc0557f0 con 0x7f961406ae10 2024-04-17T10:46:22.520 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:46:22.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.527+0000 7f96195b8640 1 -- 172.21.15.83:0/1093258471 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f95f4066f60 msgr2=0x7f95f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:22.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.527+0000 7f96195b8640 1 --2- 172.21.15.83:0/1093258471 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f95f4066f60 0x7f95f4069420 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f96040011f0 tx=0x7f9604007930 comp rx=0 tx=0).stop 2024-04-17T10:46:22.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.527+0000 7f96195b8640 1 -- 172.21.15.83:0/1093258471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f961406ae10 msgr2=0x7f9614137c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:22.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.527+0000 7f96195b8640 1 --2- 172.21.15.83:0/1093258471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f961406ae10 0x7f9614137c20 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00c820 tx=0x7f95fc00cce0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.527+0000 7f96195b8640 1 -- 172.21.15.83:0/1093258471 shutdown_connections 2024-04-17T10:46:22.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.527+0000 7f96195b8640 1 --2- 172.21.15.83:0/1093258471 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f95f4066f60 0x7f95f4069420 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.527+0000 7f96195b8640 1 --2- 172.21.15.83:0/1093258471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f961415e330 0x7f9614156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.527+0000 7f96195b8640 1 --2- 172.21.15.83:0/1093258471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9614070aa0 0x7f961413e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.527+0000 7f96195b8640 1 --2- 172.21.15.83:0/1093258471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f961406ae10 0x7f9614137c20 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.527+0000 7f96195b8640 1 -- 172.21.15.83:0/1093258471 >> 172.21.15.83:0/1093258471 conn(0x7f96140b15b0 msgr2=0x7f96140b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:22.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.527+0000 7f96195b8640 1 -- 172.21.15.83:0/1093258471 shutdown_connections 2024-04-17T10:46:22.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.527+0000 7f96195b8640 1 -- 172.21.15.83:0/1093258471 wait complete. 2024-04-17T10:46:22.538 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:46:22.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.651+0000 7f77c92f7640 1 Processor -- start 2024-04-17T10:46:22.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.651+0000 7f77c92f7640 1 -- start start 2024-04-17T10:46:22.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.651+0000 7f77c92f7640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f77c415b6e0 0x7f77c4071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:22.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.651+0000 7f77c92f7640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f77c4070470 0x7f77c406ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:22.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.651+0000 7f77c92f7640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77c406c1f0 con 0x7f77c415bac0 2024-04-17T10:46:22.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.651+0000 7f77c92f7640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77c4070a10 con 0x7f77c415b6e0 2024-04-17T10:46:22.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.651+0000 7f77c92f7640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77c4070cf0 con 0x7f77c4070470 2024-04-17T10:46:22.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.651+0000 7f77c2ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f77c415b6e0 0x7f77c4071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:22.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.651+0000 7f77c2ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f77c415b6e0 0x7f77c4071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52452/0 (socket says 172.21.15.83:52452) 2024-04-17T10:46:22.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.651+0000 7f77c2ffd640 1 -- 172.21.15.83:0/1148344113 learned_addr learned my addr 172.21.15.83:0/1148344113 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:22.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.651+0000 7f77c2ffd640 1 -- 172.21.15.83:0/1148344113 >> v2:172.21.15.167:3300/0 conn(0x7f77c4070470 msgr2=0x7f77c406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:22.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.651+0000 7f77c2ffd640 1 --2- 172.21.15.83:0/1148344113 >> v2:172.21.15.167:3300/0 conn(0x7f77c4070470 0x7f77c406ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.651+0000 7f77c2ffd640 1 -- 172.21.15.83:0/1148344113 >> v1:172.21.15.167:6789/0 conn(0x7f77c415bac0 legacy=0x7f77c40cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:22.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.651+0000 7f77c2ffd640 1 -- 172.21.15.83:0/1148344113 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c4070fd0 con 0x7f77c415b6e0 2024-04-17T10:46:22.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.655+0000 7f77c2ffd640 1 --2- 172.21.15.83:0/1148344113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77c415b6e0 0x7f77c4071cc0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f77c40728a0 tx=0x7f77b401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40c13b29994be8f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:22.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.655+0000 7f77c1ffb640 1 -- 172.21.15.83:0/1148344113 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b40b0070 con 0x7f77c415b6e0 2024-04-17T10:46:22.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.655+0000 7f77c1ffb640 1 -- 172.21.15.83:0/1148344113 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b40ab600 con 0x7f77c415b6e0 2024-04-17T10:46:22.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.655+0000 7f77c1ffb640 1 -- 172.21.15.83:0/1148344113 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b40ab9d0 con 0x7f77c415b6e0 2024-04-17T10:46:22.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.655+0000 7f77c92f7640 1 -- 172.21.15.83:0/1148344113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77c415b6e0 msgr2=0x7f77c4071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:22.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.655+0000 7f77c92f7640 1 --2- 172.21.15.83:0/1148344113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77c415b6e0 0x7f77c4071cc0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f77c40728a0 tx=0x7f77b401c4e0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.655+0000 7f77c92f7640 1 -- 172.21.15.83:0/1148344113 shutdown_connections 2024-04-17T10:46:22.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.655+0000 7f77c92f7640 1 --2- 172.21.15.83:0/1148344113 >> v2:172.21.15.167:3300/0 conn(0x7f77c4070470 0x7f77c406ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.655+0000 7f77c92f7640 1 --2- 172.21.15.83:0/1148344113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77c415b6e0 0x7f77c4071cc0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.655+0000 7f77c92f7640 1 -- 172.21.15.83:0/1148344113 >> 172.21.15.83:0/1148344113 conn(0x7f77c40b15b0 msgr2=0x7f77c40b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:22.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.655+0000 7f77c92f7640 1 -- 172.21.15.83:0/1148344113 shutdown_connections 2024-04-17T10:46:22.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.655+0000 7f77c92f7640 1 -- 172.21.15.83:0/1148344113 wait complete. 2024-04-17T10:46:22.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.655+0000 7f77c92f7640 1 Processor -- start 2024-04-17T10:46:22.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.655+0000 7f77c92f7640 1 -- start start 2024-04-17T10:46:22.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.655+0000 7f77c92f7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77c4070470 0x7f77c40b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:22.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.655+0000 7f77c92f7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77c415b6e0 0x7f77c40b0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:22.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.655+0000 7f77c92f7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77c415bac0 0x7f77c40adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:22.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.655+0000 7f77c92f7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c406d2f0 con 0x7f77c4070470 2024-04-17T10:46:22.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.655+0000 7f77c92f7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c406d170 con 0x7f77c415bac0 2024-04-17T10:46:22.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.655+0000 7f77c92f7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c406b760 con 0x7f77c415b6e0 2024-04-17T10:46:22.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.655+0000 7f77c2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77c4070470 0x7f77c40b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:22.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.655+0000 7f77c2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77c4070470 0x7f77c40b42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52456/0 (socket says 172.21.15.83:52456) 2024-04-17T10:46:22.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.655+0000 7f77c2ffd640 1 -- 172.21.15.83:0/2110813226 learned_addr learned my addr 172.21.15.83:0/2110813226 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:22.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.655+0000 7f77c27fc640 1 --2- 172.21.15.83:0/2110813226 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77c415b6e0 0x7f77c40b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:22.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.655+0000 7f77c2ffd640 1 -- 172.21.15.83:0/2110813226 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77c415b6e0 msgr2=0x7f77c40b0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:22.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.655+0000 7f77c37fe640 1 --2- 172.21.15.83:0/2110813226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77c415bac0 0x7f77c40adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:22.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.659+0000 7f77c2ffd640 1 --2- 172.21.15.83:0/2110813226 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77c415b6e0 0x7f77c40b0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.659+0000 7f77c2ffd640 1 -- 172.21.15.83:0/2110813226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77c415bac0 msgr2=0x7f77c40adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:22.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.659+0000 7f77c2ffd640 1 --2- 172.21.15.83:0/2110813226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77c415bac0 0x7f77c40adaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.659+0000 7f77c2ffd640 1 -- 172.21.15.83:0/2110813226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c40b3290 con 0x7f77c4070470 2024-04-17T10:46:22.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.659+0000 7f77c2ffd640 1 --2- 172.21.15.83:0/2110813226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77c4070470 0x7f77c40b42f0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f77b40ae040 tx=0x7f77b401ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:22.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.659+0000 7f77ab7fe640 1 -- 172.21.15.83:0/2110813226 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b40b0040 con 0x7f77c4070470 2024-04-17T10:46:22.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.659+0000 7f77ab7fe640 1 -- 172.21.15.83:0/2110813226 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b40bc070 con 0x7f77c4070470 2024-04-17T10:46:22.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.659+0000 7f77c92f7640 1 -- 172.21.15.83:0/2110813226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77c40b3530 con 0x7f77c4070470 2024-04-17T10:46:22.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.659+0000 7f77ab7fe640 1 -- 172.21.15.83:0/2110813226 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b40aa8e0 con 0x7f77c4070470 2024-04-17T10:46:22.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.659+0000 7f77c92f7640 1 -- 172.21.15.83:0/2110813226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77c40b3af0 con 0x7f77c4070470 2024-04-17T10:46:22.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.659+0000 7f77c92f7640 1 -- 172.21.15.83:0/2110813226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77c4074f90 con 0x7f77c4070470 2024-04-17T10:46:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.659+0000 7f77ab7fe640 1 -- 172.21.15.83:0/2110813226 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f77b40aaa80 con 0x7f77c4070470 2024-04-17T10:46:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.659+0000 7f77ab7fe640 1 --2- 172.21.15.83:0/2110813226 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f77b0066f60 0x7f77b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.659+0000 7f77ab7fe640 1 -- 172.21.15.83:0/2110813226 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f77b411f6c0 con 0x7f77c4070470 2024-04-17T10:46:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.659+0000 7f77c27fc640 1 --2- 172.21.15.83:0/2110813226 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f77b0066f60 0x7f77b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:22.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.659+0000 7f77c27fc640 1 --2- 172.21.15.83:0/2110813226 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f77b0066f60 0x7f77b0069420 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f77c4155c40 tx=0x7f77b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:22.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.663+0000 7f77ab7fe640 1 -- 172.21.15.83:0/2110813226 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f77b40160d0 con 0x7f77c4070470 2024-04-17T10:46:22.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.807+0000 7f77c92f7640 1 -- 172.21.15.83:0/2110813226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f77c40aee90 con 0x7f77c4070470 2024-04-17T10:46:22.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.807+0000 7f77ab7fe640 1 -- 172.21.15.83:0/2110813226 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) v1 ==== 104+0+11 (secure 0 0 0) 0x7f77b40f0670 con 0x7f77c4070470 2024-04-17T10:46:22.805 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:46:22.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.811+0000 7f77c92f7640 1 -- 172.21.15.83:0/2110813226 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f77b0066f60 msgr2=0x7f77b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:22.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.811+0000 7f77c92f7640 1 --2- 172.21.15.83:0/2110813226 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f77b0066f60 0x7f77b0069420 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f77c4155c40 tx=0x7f77b8008040 comp rx=0 tx=0).stop 2024-04-17T10:46:22.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.811+0000 7f77c92f7640 1 -- 172.21.15.83:0/2110813226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77c4070470 msgr2=0x7f77c40b42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:22.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.811+0000 7f77c92f7640 1 --2- 172.21.15.83:0/2110813226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77c4070470 0x7f77c40b42f0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f77b40ae040 tx=0x7f77b401ad20 comp rx=0 tx=0).stop 2024-04-17T10:46:22.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.811+0000 7f77c92f7640 1 -- 172.21.15.83:0/2110813226 shutdown_connections 2024-04-17T10:46:22.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.811+0000 7f77c92f7640 1 --2- 172.21.15.83:0/2110813226 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f77b0066f60 0x7f77b0069420 secure :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f77c4155c40 tx=0x7f77b8008040 comp rx=0 tx=0).stop 2024-04-17T10:46:22.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.811+0000 7f77c92f7640 1 --2- 172.21.15.83:0/2110813226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77c415bac0 0x7f77c40adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.811+0000 7f77c92f7640 1 --2- 172.21.15.83:0/2110813226 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77c415b6e0 0x7f77c40b0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.811+0000 7f77c92f7640 1 --2- 172.21.15.83:0/2110813226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77c4070470 0x7f77c40b42f0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.811+0000 7f77c92f7640 1 -- 172.21.15.83:0/2110813226 >> 172.21.15.83:0/2110813226 conn(0x7f77c40b15b0 msgr2=0x7f77c40b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:22.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.811+0000 7f77c92f7640 1 -- 172.21.15.83:0/2110813226 shutdown_connections 2024-04-17T10:46:22.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.811+0000 7f77c92f7640 1 -- 172.21.15.83:0/2110813226 wait complete. 2024-04-17T10:46:22.823 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:46:22.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.943+0000 7fea43dd5640 1 Processor -- start 2024-04-17T10:46:22.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.943+0000 7fea43dd5640 1 -- start start 2024-04-17T10:46:22.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.943+0000 7fea43dd5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fea3c154de0 0x7fea3c06ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:22.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.943+0000 7fea43dd5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea3c06b350 0x7fea3c0b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:22.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.943+0000 7fea43dd5640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea3c15bd70 con 0x7fea3c0b4040 2024-04-17T10:46:22.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.943+0000 7fea43dd5640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea3c0b4870 con 0x7fea3c154de0 2024-04-17T10:46:22.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.943+0000 7fea43dd5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c15bf60 con 0x7fea3c06b350 2024-04-17T10:46:22.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.943+0000 7fea41b71640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fea3c154de0 0x7fea3c06ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:22.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.943+0000 7fea42372640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fea3c0b4040 0x7fea3c071580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:44144/0 (socket says 172.21.15.83:44144) 2024-04-17T10:46:22.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.943+0000 7fea42372640 1 -- 172.21.15.83:0/116957306 learned_addr learned my addr 172.21.15.83:0/116957306 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:46:22.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.943+0000 7fea40b6f640 1 -- 172.21.15.83:0/116957306 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2364717271 0 0) 0x7fea3c15bd70 con 0x7fea3c0b4040 2024-04-17T10:46:22.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.943+0000 7fea40b6f640 1 -- 172.21.15.83:0/116957306 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea28005740 con 0x7fea3c0b4040 2024-04-17T10:46:22.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.943+0000 7fea41370640 1 --2- 172.21.15.83:0/116957306 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea3c06b350 0x7fea3c0b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:22.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.943+0000 7fea41b71640 1 -- 172.21.15.83:0/116957306 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea3c06b350 msgr2=0x7fea3c0b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:22.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.943+0000 7fea41b71640 1 --2- 172.21.15.83:0/116957306 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea3c06b350 0x7fea3c0b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.943+0000 7fea41b71640 1 -- 172.21.15.83:0/116957306 >> v1:172.21.15.83:6789/0 conn(0x7fea3c0b4040 legacy=0x7fea3c071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:22.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.943+0000 7fea41b71640 1 -- 172.21.15.83:0/116957306 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3c15afb0 con 0x7fea3c154de0 2024-04-17T10:46:22.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.943+0000 7fea41370640 1 --2- 172.21.15.83:0/116957306 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea3c06b350 0x7fea3c0b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:22.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea41b71640 1 --2- 172.21.15.83:0/116957306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea3c154de0 0x7fea3c06ae10 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fea380097c0 tx=0x7fea3801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607fbf6dc3e40961 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:22.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea40b6f640 1 -- 172.21.15.83:0/116957306 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea380b0070 con 0x7fea3c154de0 2024-04-17T10:46:22.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea40b6f640 1 -- 172.21.15.83:0/116957306 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea380ab600 con 0x7fea3c154de0 2024-04-17T10:46:22.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea40b6f640 1 -- 172.21.15.83:0/116957306 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea380ab9d0 con 0x7fea3c154de0 2024-04-17T10:46:22.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea43dd5640 1 -- 172.21.15.83:0/116957306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea3c154de0 msgr2=0x7fea3c06ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:22.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea43dd5640 1 --2- 172.21.15.83:0/116957306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea3c154de0 0x7fea3c06ae10 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fea380097c0 tx=0x7fea3801c4a0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea43dd5640 1 -- 172.21.15.83:0/116957306 shutdown_connections 2024-04-17T10:46:22.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea43dd5640 1 --2- 172.21.15.83:0/116957306 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea3c06b350 0x7fea3c0b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea43dd5640 1 --2- 172.21.15.83:0/116957306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea3c154de0 0x7fea3c06ae10 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea43dd5640 1 -- 172.21.15.83:0/116957306 >> 172.21.15.83:0/116957306 conn(0x7fea3c0b15b0 msgr2=0x7fea3c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:22.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea43dd5640 1 -- 172.21.15.83:0/116957306 shutdown_connections 2024-04-17T10:46:22.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea43dd5640 1 -- 172.21.15.83:0/116957306 wait complete. 2024-04-17T10:46:22.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea43dd5640 1 Processor -- start 2024-04-17T10:46:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea43dd5640 1 -- start start 2024-04-17T10:46:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea43dd5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea3c06b350 0x7fea3c1579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea43dd5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea3c0b4040 0x7fea3c14f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea43dd5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea3c154de0 0x7fea3c147de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea43dd5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c1551c0 con 0x7fea3c154de0 2024-04-17T10:46:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea43dd5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c15b850 con 0x7fea3c0b4040 2024-04-17T10:46:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea43dd5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c0b4450 con 0x7fea3c06b350 2024-04-17T10:46:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea42372640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea3c154de0 0x7fea3c147de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea42372640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea3c154de0 0x7fea3c147de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52476/0 (socket says 172.21.15.83:52476) 2024-04-17T10:46:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea42372640 1 -- 172.21.15.83:0/750676879 learned_addr learned my addr 172.21.15.83:0/750676879 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea41b71640 1 --2- 172.21.15.83:0/750676879 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea3c06b350 0x7fea3c1579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea41370640 1 --2- 172.21.15.83:0/750676879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea3c0b4040 0x7fea3c14f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea42372640 1 -- 172.21.15.83:0/750676879 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea3c06b350 msgr2=0x7fea3c1579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:22.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea42372640 1 --2- 172.21.15.83:0/750676879 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea3c06b350 0x7fea3c1579c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea42372640 1 -- 172.21.15.83:0/750676879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea3c0b4040 msgr2=0x7fea3c14f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea42372640 1 --2- 172.21.15.83:0/750676879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea3c0b4040 0x7fea3c14f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea42372640 1 -- 172.21.15.83:0/750676879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3c14ea80 con 0x7fea3c154de0 2024-04-17T10:46:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea41370640 1 --2- 172.21.15.83:0/750676879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea3c0b4040 0x7fea3c14f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea42372640 1 --2- 172.21.15.83:0/750676879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea3c154de0 0x7fea3c147de0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fea3400caa0 tx=0x7fea3400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea327fc640 1 -- 172.21.15.83:0/750676879 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea34017070 con 0x7fea3c154de0 2024-04-17T10:46:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea327fc640 1 -- 172.21.15.83:0/750676879 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3400ac00 con 0x7fea3c154de0 2024-04-17T10:46:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea43dd5640 1 -- 172.21.15.83:0/750676879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea3c156ee0 con 0x7fea3c154de0 2024-04-17T10:46:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea327fc640 1 -- 172.21.15.83:0/750676879 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea34012400 con 0x7fea3c154de0 2024-04-17T10:46:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.947+0000 7fea43dd5640 1 -- 172.21.15.83:0/750676879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea3c1574a0 con 0x7fea3c154de0 2024-04-17T10:46:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.951+0000 7fea43dd5640 1 -- 172.21.15.83:0/750676879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea3c074f90 con 0x7fea3c154de0 2024-04-17T10:46:22.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.951+0000 7fea327fc640 1 -- 172.21.15.83:0/750676879 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fea3401a070 con 0x7fea3c154de0 2024-04-17T10:46:22.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.951+0000 7fea327fc640 1 --2- 172.21.15.83:0/750676879 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fea20066f90 0x7fea20069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:22.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.951+0000 7fea327fc640 1 -- 172.21.15.83:0/750676879 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fea34084db0 con 0x7fea3c154de0 2024-04-17T10:46:22.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.951+0000 7fea41b71640 1 --2- 172.21.15.83:0/750676879 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fea20066f90 0x7fea20069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:22.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.951+0000 7fea41b71640 1 --2- 172.21.15.83:0/750676879 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fea20066f90 0x7fea20069450 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fea380ad000 tx=0x7fea380a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:22.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:22.955+0000 7fea327fc640 1 -- 172.21.15.83:0/750676879 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fea34055d60 con 0x7fea3c154de0 2024-04-17T10:46:23.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.103+0000 7fea43dd5640 1 -- 172.21.15.83:0/750676879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fea3c157670 con 0x7fea3c154de0 2024-04-17T10:46:23.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.103+0000 7fea327fc640 1 -- 172.21.15.83:0/750676879 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) v1 ==== 100+0+11 (secure 0 0 0) 0x7fea34055660 con 0x7fea3c154de0 2024-04-17T10:46:23.099 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:46:23.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.103+0000 7fea43dd5640 1 -- 172.21.15.83:0/750676879 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fea20066f90 msgr2=0x7fea20069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:23.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.103+0000 7fea43dd5640 1 --2- 172.21.15.83:0/750676879 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fea20066f90 0x7fea20069450 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fea380ad000 tx=0x7fea380a6000 comp rx=0 tx=0).stop 2024-04-17T10:46:23.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.103+0000 7fea43dd5640 1 -- 172.21.15.83:0/750676879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea3c154de0 msgr2=0x7fea3c147de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:23.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.103+0000 7fea43dd5640 1 --2- 172.21.15.83:0/750676879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea3c154de0 0x7fea3c147de0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fea3400caa0 tx=0x7fea3400cf60 comp rx=0 tx=0).stop 2024-04-17T10:46:23.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.103+0000 7fea43dd5640 1 -- 172.21.15.83:0/750676879 shutdown_connections 2024-04-17T10:46:23.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.103+0000 7fea43dd5640 1 --2- 172.21.15.83:0/750676879 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fea20066f90 0x7fea20069450 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:23.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.103+0000 7fea43dd5640 1 --2- 172.21.15.83:0/750676879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea3c154de0 0x7fea3c147de0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:23.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.103+0000 7fea43dd5640 1 --2- 172.21.15.83:0/750676879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea3c0b4040 0x7fea3c14f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:23.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.103+0000 7fea43dd5640 1 --2- 172.21.15.83:0/750676879 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea3c06b350 0x7fea3c1579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:23.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.103+0000 7fea43dd5640 1 -- 172.21.15.83:0/750676879 >> 172.21.15.83:0/750676879 conn(0x7fea3c0b15b0 msgr2=0x7fea3c0b05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:23.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.103+0000 7fea43dd5640 1 -- 172.21.15.83:0/750676879 shutdown_connections 2024-04-17T10:46:23.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.103+0000 7fea43dd5640 1 -- 172.21.15.83:0/750676879 wait complete. 2024-04-17T10:46:23.117 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:46:23.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.239+0000 7f38f270b640 1 Processor -- start 2024-04-17T10:46:23.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.239+0000 7f38f270b640 1 -- start start 2024-04-17T10:46:23.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.239+0000 7f38f270b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38ec146a00 0x7f38ec146e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:23.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.239+0000 7f38f270b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f38ec06d690 0x7f38ec1620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:23.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.239+0000 7f38f270b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38ec06daf0 con 0x7f38ec0b2670 2024-04-17T10:46:23.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.239+0000 7f38f270b640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38ec162610 con 0x7f38ec06d690 2024-04-17T10:46:23.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.239+0000 7f38f270b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec06c960 con 0x7f38ec146a00 2024-04-17T10:46:23.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.239+0000 7f38f0ca8640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f38ec0b2670 0x7f38ec070470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:44150/0 (socket says 172.21.15.83:44150) 2024-04-17T10:46:23.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.239+0000 7f38f0ca8640 1 -- 172.21.15.83:0/3764238511 learned_addr learned my addr 172.21.15.83:0/3764238511 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:46:23.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.239+0000 7f38eb7fe640 1 --2- 172.21.15.83:0/3764238511 >> v2:172.21.15.167:3300/0 conn(0x7f38ec06d690 0x7f38ec1620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:23.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.239+0000 7f38ebfff640 1 --2- 172.21.15.83:0/3764238511 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38ec146a00 0x7f38ec146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:23.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.239+0000 7f38eaffd640 1 -- 172.21.15.83:0/3764238511 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 397888513 0 0) 0x7f38ec06daf0 con 0x7f38ec0b2670 2024-04-17T10:46:23.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.239+0000 7f38eaffd640 1 -- 172.21.15.83:0/3764238511 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38d8003530 con 0x7f38ec0b2670 2024-04-17T10:46:23.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.239+0000 7f38eb7fe640 1 -- 172.21.15.83:0/3764238511 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38ec146a00 msgr2=0x7f38ec146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:23.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.239+0000 7f38eb7fe640 1 --2- 172.21.15.83:0/3764238511 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38ec146a00 0x7f38ec146e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:23.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.239+0000 7f38eb7fe640 1 -- 172.21.15.83:0/3764238511 >> v1:172.21.15.83:6789/0 conn(0x7f38ec0b2670 legacy=0x7f38ec070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:23.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.239+0000 7f38eb7fe640 1 -- 172.21.15.83:0/3764238511 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ec147570 con 0x7f38ec06d690 2024-04-17T10:46:23.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.243+0000 7f38eb7fe640 1 --2- 172.21.15.83:0/3764238511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38ec06d690 0x7f38ec1620d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f38d40097c0 tx=0x7f38d401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87ec7ef04f53d857 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:23.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.243+0000 7f38eaffd640 1 -- 172.21.15.83:0/3764238511 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d40b1070 con 0x7f38ec06d690 2024-04-17T10:46:23.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.243+0000 7f38eaffd640 1 -- 172.21.15.83:0/3764238511 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d40ac410 con 0x7f38ec06d690 2024-04-17T10:46:23.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.243+0000 7f38eaffd640 1 -- 172.21.15.83:0/3764238511 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d40ac7c0 con 0x7f38ec06d690 2024-04-17T10:46:23.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.243+0000 7f38f270b640 1 -- 172.21.15.83:0/3764238511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38ec06d690 msgr2=0x7f38ec1620d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:23.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.243+0000 7f38f270b640 1 --2- 172.21.15.83:0/3764238511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38ec06d690 0x7f38ec1620d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f38d40097c0 tx=0x7f38d401c4a0 comp rx=0 tx=0).stop 2024-04-17T10:46:23.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.243+0000 7f38f270b640 1 -- 172.21.15.83:0/3764238511 shutdown_connections 2024-04-17T10:46:23.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.243+0000 7f38f270b640 1 --2- 172.21.15.83:0/3764238511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38ec06d690 0x7f38ec1620d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:23.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.243+0000 7f38f270b640 1 --2- 172.21.15.83:0/3764238511 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38ec146a00 0x7f38ec146e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:23.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.243+0000 7f38f270b640 1 -- 172.21.15.83:0/3764238511 >> 172.21.15.83:0/3764238511 conn(0x7f38ec06abe0 msgr2=0x7f38ec06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:23.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.243+0000 7f38f270b640 1 -- 172.21.15.83:0/3764238511 shutdown_connections 2024-04-17T10:46:23.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.243+0000 7f38f270b640 1 -- 172.21.15.83:0/3764238511 wait complete. 2024-04-17T10:46:23.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.247+0000 7f38f270b640 1 Processor -- start 2024-04-17T10:46:23.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.247+0000 7f38f270b640 1 -- start start 2024-04-17T10:46:23.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.247+0000 7f38f270b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38ec06d690 0x7f38ec1598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:23.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.247+0000 7f38f270b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38ec0b2670 0x7f38ec15b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:23.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.247+0000 7f38f270b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38ec146a00 0x7f38ec13aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:23.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.247+0000 7f38f270b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec071110 con 0x7f38ec0b2670 2024-04-17T10:46:23.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.247+0000 7f38f270b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec070f90 con 0x7f38ec146a00 2024-04-17T10:46:23.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.247+0000 7f38f270b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec16b200 con 0x7f38ec06d690 2024-04-17T10:46:23.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.247+0000 7f38eb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38ec0b2670 0x7f38ec15b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:23.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.247+0000 7f38eb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38ec0b2670 0x7f38ec15b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52482/0 (socket says 172.21.15.83:52482) 2024-04-17T10:46:23.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.247+0000 7f38eb7fe640 1 -- 172.21.15.83:0/3937243709 learned_addr learned my addr 172.21.15.83:0/3937243709 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:23.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.247+0000 7f38f0ca8640 1 --2- 172.21.15.83:0/3937243709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38ec146a00 0x7f38ec13aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:23.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.247+0000 7f38ebfff640 1 --2- 172.21.15.83:0/3937243709 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38ec06d690 0x7f38ec1598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:23.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.247+0000 7f38eb7fe640 1 -- 172.21.15.83:0/3937243709 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38ec06d690 msgr2=0x7f38ec1598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:23.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.247+0000 7f38eb7fe640 1 --2- 172.21.15.83:0/3937243709 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38ec06d690 0x7f38ec1598a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:23.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.247+0000 7f38eb7fe640 1 -- 172.21.15.83:0/3937243709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38ec146a00 msgr2=0x7f38ec13aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:23.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.247+0000 7f38eb7fe640 1 --2- 172.21.15.83:0/3937243709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38ec146a00 0x7f38ec13aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:23.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.247+0000 7f38eb7fe640 1 -- 172.21.15.83:0/3937243709 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ec15a010 con 0x7f38ec0b2670 2024-04-17T10:46:23.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.247+0000 7f38f0ca8640 1 --2- 172.21.15.83:0/3937243709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38ec146a00 0x7f38ec13aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:23.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.247+0000 7f38eb7fe640 1 --2- 172.21.15.83:0/3937243709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38ec0b2670 0x7f38ec15b0b0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f38d4000c10 tx=0x7f38d4019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:23.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.247+0000 7f38e8ff9640 1 -- 172.21.15.83:0/3937243709 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d40bb070 con 0x7f38ec0b2670 2024-04-17T10:46:23.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.247+0000 7f38e8ff9640 1 -- 172.21.15.83:0/3937243709 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d4003620 con 0x7f38ec0b2670 2024-04-17T10:46:23.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.247+0000 7f38f270b640 1 -- 172.21.15.83:0/3937243709 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38ec15a2f0 con 0x7f38ec0b2670 2024-04-17T10:46:23.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.247+0000 7f38e8ff9640 1 -- 172.21.15.83:0/3937243709 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d40b1040 con 0x7f38ec0b2670 2024-04-17T10:46:23.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.247+0000 7f38f270b640 1 -- 172.21.15.83:0/3937243709 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38ec15a8b0 con 0x7f38ec0b2670 2024-04-17T10:46:23.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.247+0000 7f38f270b640 1 -- 172.21.15.83:0/3937243709 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38ec074f90 con 0x7f38ec0b2670 2024-04-17T10:46:23.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.247+0000 7f38e8ff9640 1 -- 172.21.15.83:0/3937243709 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f38d40039d0 con 0x7f38ec0b2670 2024-04-17T10:46:23.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.247+0000 7f38e8ff9640 1 --2- 172.21.15.83:0/3937243709 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f38cc066f60 0x7f38cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:23.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.247+0000 7f38e8ff9640 1 -- 172.21.15.83:0/3937243709 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f38d411dde0 con 0x7f38ec0b2670 2024-04-17T10:46:23.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.247+0000 7f38ebfff640 1 --2- 172.21.15.83:0/3937243709 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f38cc066f60 0x7f38cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:23.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.251+0000 7f38ebfff640 1 --2- 172.21.15.83:0/3937243709 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f38cc066f60 0x7f38cc069420 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009fd0 tx=0x7f38dc00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:23.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.251+0000 7f38e8ff9640 1 -- 172.21.15.83:0/3937243709 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f38d4016090 con 0x7f38ec0b2670 2024-04-17T10:46:23.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.431+0000 7f38f270b640 1 -- 172.21.15.83:0/3937243709 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38ec06b260 con 0x7f38ec0b2670 2024-04-17T10:46:23.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.431+0000 7f38e8ff9640 1 -- 172.21.15.83:0/3937243709 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f38d40ee6c0 con 0x7f38ec0b2670 2024-04-17T10:46:23.427 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:46:23.427 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5258,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2259892875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2259892875},{"type":"v1","addr":"172.21.15.167:6835","nonce":2259892875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":7095,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/3769102527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3769102527},{"type":"v1","addr":"172.21.15.83:6836","nonce":3769102527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":7098,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/395594830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":395594830},{"type":"v1","addr":"172.21.15.83:6837","nonce":395594830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-17T10:43:33.830720+0000","modified":"2024-04-17T10:43:47.538354+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7104},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7104":{"gid":7104,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6833/2796648400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2796648400},{"type":"v1","addr":"172.21.15.167:6833","nonce":2796648400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-17T10:46:23.427 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 49 2024-04-17T10:46:23.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.431+0000 7f38f270b640 1 -- 172.21.15.83:0/3937243709 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f38cc066f60 msgr2=0x7f38cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:23.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.431+0000 7f38f270b640 1 --2- 172.21.15.83:0/3937243709 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f38cc066f60 0x7f38cc069420 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009fd0 tx=0x7f38dc00b040 comp rx=0 tx=0).stop 2024-04-17T10:46:23.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.431+0000 7f38f270b640 1 -- 172.21.15.83:0/3937243709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38ec0b2670 msgr2=0x7f38ec15b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:23.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.431+0000 7f38f270b640 1 --2- 172.21.15.83:0/3937243709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38ec0b2670 0x7f38ec15b0b0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f38d4000c10 tx=0x7f38d4019240 comp rx=0 tx=0).stop 2024-04-17T10:46:23.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.431+0000 7f38f270b640 1 -- 172.21.15.83:0/3937243709 shutdown_connections 2024-04-17T10:46:23.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.431+0000 7f38f270b640 1 --2- 172.21.15.83:0/3937243709 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f38cc066f60 0x7f38cc069420 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:23.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.431+0000 7f38f270b640 1 --2- 172.21.15.83:0/3937243709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38ec146a00 0x7f38ec13aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:23.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.431+0000 7f38f270b640 1 --2- 172.21.15.83:0/3937243709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38ec0b2670 0x7f38ec15b0b0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:23.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.431+0000 7f38f270b640 1 --2- 172.21.15.83:0/3937243709 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38ec06d690 0x7f38ec1598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:23.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.431+0000 7f38f270b640 1 -- 172.21.15.83:0/3937243709 >> 172.21.15.83:0/3937243709 conn(0x7f38ec06abe0 msgr2=0x7f38ec0ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:23.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.431+0000 7f38f270b640 1 -- 172.21.15.83:0/3937243709 shutdown_connections 2024-04-17T10:46:23.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.431+0000 7f38f270b640 1 -- 172.21.15.83:0/3937243709 wait complete. 2024-04-17T10:46:23.445 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:46:23.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd8658a2640 1 Processor -- start 2024-04-17T10:46:23.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd8658a2640 1 -- start start 2024-04-17T10:46:23.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd8658a2640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd86015ba30 0x7fd860072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:23.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd8658a2640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd86006e320 con 0x7fd8600cfbe0 2024-04-17T10:46:23.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd8658a2640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd86006e040 con 0x7fd86015c960 2024-04-17T10:46:23.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd8658a2640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd860071ea0 con 0x7fd86015ba30 2024-04-17T10:46:23.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd85f7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd8600cfbe0 0x7fd86006c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:44156/0 (socket says 172.21.15.83:44156) 2024-04-17T10:46:23.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd85f7fe640 1 -- 172.21.15.83:0/3467622653 learned_addr learned my addr 172.21.15.83:0/3467622653 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:46:23.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd85effd640 1 --2- 172.21.15.83:0/3467622653 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd86015ba30 0x7fd860072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:23.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd85e7fc640 1 -- 172.21.15.83:0/3467622653 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2660911961 0 0) 0x7fd86006e320 con 0x7fd8600cfbe0 2024-04-17T10:46:23.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd85e7fc640 1 -- 172.21.15.83:0/3467622653 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd84c003530 con 0x7fd8600cfbe0 2024-04-17T10:46:23.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd85e7fc640 1 -- 172.21.15.83:0/3467622653 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3436909270 0 0) 0x7fd84c003530 con 0x7fd8600cfbe0 2024-04-17T10:46:23.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd85e7fc640 1 -- 172.21.15.83:0/3467622653 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd86015ba30 msgr2=0x7fd860072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:23.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd85e7fc640 1 --2- 172.21.15.83:0/3467622653 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd86015ba30 0x7fd860072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:23.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd85e7fc640 1 -- 172.21.15.83:0/3467622653 >> v1:172.21.15.167:6789/0 conn(0x7fd86015c960 legacy=0x7fd8600cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:23.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd85e7fc640 1 -- 172.21.15.83:0/3467622653 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd860071850 con 0x7fd8600cfbe0 2024-04-17T10:46:23.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd85e7fc640 1 -- 172.21.15.83:0/3467622653 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fd858001a20 con 0x7fd8600cfbe0 2024-04-17T10:46:23.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd85e7fc640 1 -- 172.21.15.83:0/3467622653 >> v1:172.21.15.83:6789/0 conn(0x7fd8600cfbe0 legacy=0x7fd86006c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:23.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd85e7fc640 1 --2- 172.21.15.83:0/3467622653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd84c0039f0 0x7fd84c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:23.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd85e7fc640 1 --2- 172.21.15.83:0/3467622653 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd86015ba30 0x7fd84c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:23.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd85e7fc640 1 --2- 172.21.15.83:0/3467622653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd84c004d50 0x7fd84c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:23.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd85e7fc640 1 -- 172.21.15.83:0/3467622653 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd860071ea0 con 0x7fd84c0039f0 2024-04-17T10:46:23.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd85e7fc640 1 -- 172.21.15.83:0/3467622653 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c00bb00 con 0x7fd84c004d50 2024-04-17T10:46:23.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd85e7fc640 1 -- 172.21.15.83:0/3467622653 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c00bd00 con 0x7fd86015ba30 2024-04-17T10:46:23.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd85effd640 1 --2- 172.21.15.83:0/3467622653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd84c0039f0 0x7fd84c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:23.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd857fff640 1 --2- 172.21.15.83:0/3467622653 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd86015ba30 0x7fd84c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:23.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd85effd640 1 -- 172.21.15.83:0/3467622653 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd86015ba30 msgr2=0x7fd84c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:23.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd85f7fe640 1 --2- 172.21.15.83:0/3467622653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd84c004d50 0x7fd84c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:23.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd85effd640 1 --2- 172.21.15.83:0/3467622653 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd86015ba30 0x7fd84c004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:23.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd85effd640 1 -- 172.21.15.83:0/3467622653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd84c004d50 msgr2=0x7fd84c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:23.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd85effd640 1 --2- 172.21.15.83:0/3467622653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd84c004d50 0x7fd84c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:23.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd85effd640 1 -- 172.21.15.83:0/3467622653 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd84c003530 con 0x7fd84c0039f0 2024-04-17T10:46:23.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.563+0000 7fd85f7fe640 1 --2- 172.21.15.83:0/3467622653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd84c004d50 0x7fd84c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:23.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.567+0000 7fd85effd640 1 --2- 172.21.15.83:0/3467622653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd84c0039f0 0x7fd84c003e10 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fd850009ae0 tx=0x7fd86006d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8974349832835479 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:23.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.567+0000 7fd85e7fc640 1 -- 172.21.15.83:0/3467622653 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8500096b0 con 0x7fd84c0039f0 2024-04-17T10:46:23.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.567+0000 7fd85e7fc640 1 -- 172.21.15.83:0/3467622653 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8500a83f0 con 0x7fd84c0039f0 2024-04-17T10:46:23.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.567+0000 7fd85e7fc640 1 -- 172.21.15.83:0/3467622653 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8500a87e0 con 0x7fd84c0039f0 2024-04-17T10:46:23.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.567+0000 7fd8658a2640 1 -- 172.21.15.83:0/3467622653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd84c0039f0 msgr2=0x7fd84c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:23.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.567+0000 7fd8658a2640 1 --2- 172.21.15.83:0/3467622653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd84c0039f0 0x7fd84c003e10 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fd850009ae0 tx=0x7fd86006d3b0 comp rx=0 tx=0).stop 2024-04-17T10:46:23.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.567+0000 7fd85f7fe640 1 -- 172.21.15.83:0/3467622653 reap_dead start 2024-04-17T10:46:23.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.567+0000 7fd8658a2640 1 -- 172.21.15.83:0/3467622653 shutdown_connections 2024-04-17T10:46:23.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.567+0000 7fd8658a2640 1 -- 172.21.15.83:0/3467622653 >> 172.21.15.83:0/3467622653 conn(0x7fd86006abe0 msgr2=0x7fd86006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:23.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.567+0000 7fd8658a2640 1 -- 172.21.15.83:0/3467622653 shutdown_connections 2024-04-17T10:46:23.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.567+0000 7fd8658a2640 1 -- 172.21.15.83:0/3467622653 wait complete. 2024-04-17T10:46:23.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.567+0000 7fd8658a2640 1 Processor -- start 2024-04-17T10:46:23.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.567+0000 7fd8658a2640 1 -- start start 2024-04-17T10:46:23.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.567+0000 7fd8658a2640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd84c0039f0 0x7fd860155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:23.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.567+0000 7fd8658a2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd860138d70 0x7fd860154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:23.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.567+0000 7fd8658a2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8601545c0 0x7fd860151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:23.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.567+0000 7fd8658a2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86015cec0 con 0x7fd8601545c0 2024-04-17T10:46:23.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.567+0000 7fd8658a2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86015cd40 con 0x7fd860138d70 2024-04-17T10:46:23.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.567+0000 7fd8658a2640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd860167520 con 0x7fd84c0039f0 2024-04-17T10:46:23.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.567+0000 7fd85f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8601545c0 0x7fd860151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:23.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.567+0000 7fd85f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8601545c0 0x7fd860151590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52490/0 (socket says 172.21.15.83:52490) 2024-04-17T10:46:23.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.567+0000 7fd85f7fe640 1 -- 172.21.15.83:0/894635322 learned_addr learned my addr 172.21.15.83:0/894635322 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:23.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.571+0000 7fd857fff640 1 --2- 172.21.15.83:0/894635322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd860138d70 0x7fd860154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:23.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.571+0000 7fd85effd640 1 --2- 172.21.15.83:0/894635322 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd84c0039f0 0x7fd860155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:23.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.571+0000 7fd85f7fe640 1 -- 172.21.15.83:0/894635322 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd84c0039f0 msgr2=0x7fd860155f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:23.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.571+0000 7fd85f7fe640 1 --2- 172.21.15.83:0/894635322 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd84c0039f0 0x7fd860155f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:23.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.571+0000 7fd85f7fe640 1 -- 172.21.15.83:0/894635322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd860138d70 msgr2=0x7fd860154080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:23.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.571+0000 7fd85f7fe640 1 --2- 172.21.15.83:0/894635322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd860138d70 0x7fd860154080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:23.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.571+0000 7fd85f7fe640 1 -- 172.21.15.83:0/894635322 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd860154ef0 con 0x7fd8601545c0 2024-04-17T10:46:23.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.571+0000 7fd85f7fe640 1 --2- 172.21.15.83:0/894635322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8601545c0 0x7fd860151590 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fd860070100 tx=0x7fd85800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:23.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.571+0000 7fd85cff9640 1 -- 172.21.15.83:0/894635322 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85800bb80 con 0x7fd8601545c0 2024-04-17T10:46:23.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.571+0000 7fd85cff9640 1 -- 172.21.15.83:0/894635322 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85800bd40 con 0x7fd8601545c0 2024-04-17T10:46:23.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.571+0000 7fd8658a2640 1 -- 172.21.15.83:0/894635322 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8601551d0 con 0x7fd8601545c0 2024-04-17T10:46:23.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.571+0000 7fd85cff9640 1 -- 172.21.15.83:0/894635322 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8580072b0 con 0x7fd8601545c0 2024-04-17T10:46:23.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.571+0000 7fd8658a2640 1 -- 172.21.15.83:0/894635322 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd860155790 con 0x7fd8601545c0 2024-04-17T10:46:23.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.571+0000 7fd8658a2640 1 -- 172.21.15.83:0/894635322 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd860074f90 con 0x7fd8601545c0 2024-04-17T10:46:23.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.571+0000 7fd85cff9640 1 -- 172.21.15.83:0/894635322 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd85801b020 con 0x7fd8601545c0 2024-04-17T10:46:23.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.571+0000 7fd85cff9640 1 --2- 172.21.15.83:0/894635322 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd840066f60 0x7fd840069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:23.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.571+0000 7fd85cff9640 1 -- 172.21.15.83:0/894635322 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd858085250 con 0x7fd8601545c0 2024-04-17T10:46:23.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.571+0000 7fd85effd640 1 --2- 172.21.15.83:0/894635322 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd840066f60 0x7fd840069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:23.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.571+0000 7fd85effd640 1 --2- 172.21.15.83:0/894635322 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd840066f60 0x7fd840069420 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd8500ad000 tx=0x7fd8500b2040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:23.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.575+0000 7fd85cff9640 1 -- 172.21.15.83:0/894635322 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd858056200 con 0x7fd8601545c0 2024-04-17T10:46:23.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.723+0000 7fd8658a2640 1 -- 172.21.15.83:0/894635322 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8600b0740 con 0x7fd8601545c0 2024-04-17T10:46:23.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.723+0000 7fd85cff9640 1 -- 172.21.15.83:0/894635322 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16759 (secure 0 0 0) 0x7fd858055b00 con 0x7fd8601545c0 2024-04-17T10:46:23.721 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:46:23.721 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":78,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:45:57.116959+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:43:32.090869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-17T10:43:32.815921+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":77,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.4","mappings":[{"from":6,"to":0}]},{"pgid":"11.2d","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:46:23.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.727+0000 7fd8658a2640 1 -- 172.21.15.83:0/894635322 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd840066f60 msgr2=0x7fd840069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:23.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.727+0000 7fd8658a2640 1 --2- 172.21.15.83:0/894635322 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd840066f60 0x7fd840069420 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd8500ad000 tx=0x7fd8500b2040 comp rx=0 tx=0).stop 2024-04-17T10:46:23.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.727+0000 7fd8658a2640 1 -- 172.21.15.83:0/894635322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8601545c0 msgr2=0x7fd860151590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:23.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.727+0000 7fd8658a2640 1 --2- 172.21.15.83:0/894635322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8601545c0 0x7fd860151590 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fd860070100 tx=0x7fd85800b4a0 comp rx=0 tx=0).stop 2024-04-17T10:46:23.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.727+0000 7fd8658a2640 1 -- 172.21.15.83:0/894635322 shutdown_connections 2024-04-17T10:46:23.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.727+0000 7fd8658a2640 1 --2- 172.21.15.83:0/894635322 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd840066f60 0x7fd840069420 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:23.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.727+0000 7fd8658a2640 1 --2- 172.21.15.83:0/894635322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8601545c0 0x7fd860151590 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:23.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.727+0000 7fd8658a2640 1 --2- 172.21.15.83:0/894635322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd860138d70 0x7fd860154080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:23.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.727+0000 7fd8658a2640 1 --2- 172.21.15.83:0/894635322 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd84c0039f0 0x7fd860155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:23.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.727+0000 7fd8658a2640 1 -- 172.21.15.83:0/894635322 >> 172.21.15.83:0/894635322 conn(0x7fd86006abe0 msgr2=0x7fd86006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:23.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.727+0000 7fd8658a2640 1 -- 172.21.15.83:0/894635322 shutdown_connections 2024-04-17T10:46:23.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.727+0000 7fd8658a2640 1 -- 172.21.15.83:0/894635322 wait complete. 2024-04-17T10:46:23.738 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-17T10:46:23.738 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:46:23.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.855+0000 7f59d08e4640 1 Processor -- start 2024-04-17T10:46:23.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.855+0000 7f59d08e4640 1 -- start start 2024-04-17T10:46:23.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.859+0000 7f59d08e4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59cc15ba30 0x7f59cc072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:23.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.859+0000 7f59d08e4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f59cc0cfbe0 0x7f59cc06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:23.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.859+0000 7f59d08e4640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59cc06e320 con 0x7f59cc15c960 2024-04-17T10:46:23.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.859+0000 7f59d08e4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59cc071ea0 con 0x7f59cc0cfbe0 2024-04-17T10:46:23.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.859+0000 7f59d08e4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc06e040 con 0x7f59cc15ba30 2024-04-17T10:46:23.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.859+0000 7f59cad9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f59cc0cfbe0 0x7f59cc06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:23.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.859+0000 7f59cad9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f59cc0cfbe0 0x7f59cc06c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52496/0 (socket says 172.21.15.83:52496) 2024-04-17T10:46:23.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.859+0000 7f59c9d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f59cc15c960 0x7f59cc0cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:44170/0 (socket says 172.21.15.83:44170) 2024-04-17T10:46:23.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.859+0000 7f59cad9d640 1 -- 172.21.15.83:0/3092097897 learned_addr learned my addr 172.21.15.83:0/3092097897 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:23.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.859+0000 7f59ca59c640 1 --2- 172.21.15.83:0/3092097897 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59cc15ba30 0x7f59cc072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:23.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.859+0000 7f59cad9d640 1 -- 172.21.15.83:0/3092097897 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59cc15ba30 msgr2=0x7f59cc072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:23.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.859+0000 7f59cad9d640 1 --2- 172.21.15.83:0/3092097897 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59cc15ba30 0x7f59cc072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:23.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.859+0000 7f59cad9d640 1 -- 172.21.15.83:0/3092097897 >> v1:172.21.15.83:6789/0 conn(0x7f59cc15c960 legacy=0x7f59cc0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:23.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.859+0000 7f59cad9d640 1 -- 172.21.15.83:0/3092097897 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59cc071850 con 0x7f59cc0cfbe0 2024-04-17T10:46:23.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.859+0000 7f59c959a640 1 -- 172.21.15.83:0/3092097897 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1182186092 0 0) 0x7f59cc06e320 con 0x7f59cc15c960 2024-04-17T10:46:23.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.859+0000 7f59cad9d640 1 --2- 172.21.15.83:0/3092097897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59cc0cfbe0 0x7f59cc06c490 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f59bc009e50 tx=0x7f59bc01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6332eb453b1dfef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:23.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.859+0000 7f59c959a640 1 -- 172.21.15.83:0/3092097897 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59bc0a56b0 con 0x7f59cc0cfbe0 2024-04-17T10:46:23.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.859+0000 7f59c959a640 1 -- 172.21.15.83:0/3092097897 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59bc0a5850 con 0x7f59cc0cfbe0 2024-04-17T10:46:23.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.859+0000 7f59c959a640 1 -- 172.21.15.83:0/3092097897 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59bc0a5c20 con 0x7f59cc0cfbe0 2024-04-17T10:46:23.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59d08e4640 1 -- 172.21.15.83:0/3092097897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59cc0cfbe0 msgr2=0x7f59cc06c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:23.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59d08e4640 1 --2- 172.21.15.83:0/3092097897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59cc0cfbe0 0x7f59cc06c490 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f59bc009e50 tx=0x7f59bc01c9a0 comp rx=0 tx=0).stop 2024-04-17T10:46:23.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59d08e4640 1 -- 172.21.15.83:0/3092097897 shutdown_connections 2024-04-17T10:46:23.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59d08e4640 1 --2- 172.21.15.83:0/3092097897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59cc0cfbe0 0x7f59cc06c490 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:23.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59d08e4640 1 --2- 172.21.15.83:0/3092097897 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59cc15ba30 0x7f59cc072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:23.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59d08e4640 1 -- 172.21.15.83:0/3092097897 >> 172.21.15.83:0/3092097897 conn(0x7f59cc06abe0 msgr2=0x7f59cc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:23.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59d08e4640 1 -- 172.21.15.83:0/3092097897 shutdown_connections 2024-04-17T10:46:23.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59d08e4640 1 -- 172.21.15.83:0/3092097897 wait complete. 2024-04-17T10:46:23.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59d08e4640 1 Processor -- start 2024-04-17T10:46:23.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59d08e4640 1 -- start start 2024-04-17T10:46:23.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59d08e4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59cc0cfbe0 0x7f59cc1f6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:23.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59d08e4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59cc15ba30 0x7f59cc15abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:23.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59d08e4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59cc15c960 0x7f59cc1555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:23.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59d08e4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc06de20 con 0x7f59cc15c960 2024-04-17T10:46:23.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59d08e4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc06dca0 con 0x7f59cc0cfbe0 2024-04-17T10:46:23.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59d08e4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc168520 con 0x7f59cc15ba30 2024-04-17T10:46:23.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59cad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59cc15c960 0x7f59cc1555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:23.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59cad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59cc15c960 0x7f59cc1555f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52510/0 (socket says 172.21.15.83:52510) 2024-04-17T10:46:23.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59cad9d640 1 -- 172.21.15.83:0/975931890 learned_addr learned my addr 172.21.15.83:0/975931890 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:23.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59ca59c640 1 --2- 172.21.15.83:0/975931890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59cc0cfbe0 0x7f59cc1f6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:23.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59cad9d640 1 -- 172.21.15.83:0/975931890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59cc15ba30 msgr2=0x7f59cc15abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:23.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59c9d9b640 1 --2- 172.21.15.83:0/975931890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59cc15ba30 0x7f59cc15abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59cad9d640 1 --2- 172.21.15.83:0/975931890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59cc15ba30 0x7f59cc15abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59cad9d640 1 -- 172.21.15.83:0/975931890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59cc0cfbe0 msgr2=0x7f59cc1f6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59cad9d640 1 --2- 172.21.15.83:0/975931890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59cc0cfbe0 0x7f59cc1f6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59cad9d640 1 -- 172.21.15.83:0/975931890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59cc15ece0 con 0x7f59cc15c960 2024-04-17T10:46:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59ca59c640 1 --2- 172.21.15.83:0/975931890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59cc0cfbe0 0x7f59cc1f6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59cad9d640 1 --2- 172.21.15.83:0/975931890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59cc15c960 0x7f59cc1555f0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f59bc009e50 tx=0x7f59bc019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:23.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59baffd640 1 -- 172.21.15.83:0/975931890 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59bc0bc030 con 0x7f59cc15c960 2024-04-17T10:46:23.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59baffd640 1 -- 172.21.15.83:0/975931890 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59bc0b8470 con 0x7f59cc15c960 2024-04-17T10:46:23.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59d08e4640 1 -- 172.21.15.83:0/975931890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59cc0ce6b0 con 0x7f59cc15c960 2024-04-17T10:46:23.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59baffd640 1 -- 172.21.15.83:0/975931890 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59bc0b8820 con 0x7f59cc15c960 2024-04-17T10:46:23.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59d08e4640 1 -- 172.21.15.83:0/975931890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59cc15e940 con 0x7f59cc15c960 2024-04-17T10:46:23.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.863+0000 7f59d08e4640 1 -- 172.21.15.83:0/975931890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59cc074f90 con 0x7f59cc15c960 2024-04-17T10:46:23.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.867+0000 7f59baffd640 1 -- 172.21.15.83:0/975931890 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f59bc0bf070 con 0x7f59cc15c960 2024-04-17T10:46:23.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.867+0000 7f59baffd640 1 --2- 172.21.15.83:0/975931890 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f59ac066f60 0x7f59ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:23.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.867+0000 7f59baffd640 1 -- 172.21.15.83:0/975931890 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f59bc120020 con 0x7f59cc15c960 2024-04-17T10:46:23.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.867+0000 7f59ca59c640 1 --2- 172.21.15.83:0/975931890 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f59ac066f60 0x7f59ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:23.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.867+0000 7f59ca59c640 1 --2- 172.21.15.83:0/975931890 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f59ac066f60 0x7f59ac069420 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f59c4006fd0 tx=0x7f59c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:23.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:23.871+0000 7f59baffd640 1 -- 172.21.15.83:0/975931890 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f59bc016620 con 0x7f59cc15c960 2024-04-17T10:46:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.047+0000 7f59d08e4640 1 -- 172.21.15.83:0/975931890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59cc06b0f0 con 0x7f59cc15c960 2024-04-17T10:46:24.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.047+0000 7f59baffd640 1 -- 172.21.15.83:0/975931890 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f59bc0f1180 con 0x7f59cc15c960 2024-04-17T10:46:24.044 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:46:24.044 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5258,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2259892875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2259892875},{"type":"v1","addr":"172.21.15.167:6835","nonce":2259892875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":7095,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/3769102527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3769102527},{"type":"v1","addr":"172.21.15.83:6836","nonce":3769102527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":7098,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/395594830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":395594830},{"type":"v1","addr":"172.21.15.83:6837","nonce":395594830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-17T10:43:33.830720+0000","modified":"2024-04-17T10:43:47.538354+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7104},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7104":{"gid":7104,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6833/2796648400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2796648400},{"type":"v1","addr":"172.21.15.167:6833","nonce":2796648400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-17T10:46:24.044 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 49 2024-04-17T10:46:24.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.051+0000 7f59d08e4640 1 -- 172.21.15.83:0/975931890 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f59ac066f60 msgr2=0x7f59ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:24.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.051+0000 7f59d08e4640 1 --2- 172.21.15.83:0/975931890 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f59ac066f60 0x7f59ac069420 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f59c4006fd0 tx=0x7f59c4008040 comp rx=0 tx=0).stop 2024-04-17T10:46:24.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.051+0000 7f59d08e4640 1 -- 172.21.15.83:0/975931890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59cc15c960 msgr2=0x7f59cc1555f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:24.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.051+0000 7f59d08e4640 1 --2- 172.21.15.83:0/975931890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59cc15c960 0x7f59cc1555f0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f59bc009e50 tx=0x7f59bc019740 comp rx=0 tx=0).stop 2024-04-17T10:46:24.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.051+0000 7f59d08e4640 1 -- 172.21.15.83:0/975931890 shutdown_connections 2024-04-17T10:46:24.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.051+0000 7f59d08e4640 1 --2- 172.21.15.83:0/975931890 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f59ac066f60 0x7f59ac069420 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.051+0000 7f59d08e4640 1 --2- 172.21.15.83:0/975931890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59cc15c960 0x7f59cc1555f0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.051+0000 7f59d08e4640 1 --2- 172.21.15.83:0/975931890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59cc15ba30 0x7f59cc15abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.051+0000 7f59d08e4640 1 --2- 172.21.15.83:0/975931890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59cc0cfbe0 0x7f59cc1f6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.051+0000 7f59d08e4640 1 -- 172.21.15.83:0/975931890 >> 172.21.15.83:0/975931890 conn(0x7f59cc06abe0 msgr2=0x7f59cc0af2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:24.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.051+0000 7f59d08e4640 1 -- 172.21.15.83:0/975931890 shutdown_connections 2024-04-17T10:46:24.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.051+0000 7f59d08e4640 1 -- 172.21.15.83:0/975931890 wait complete. 2024-04-17T10:46:24.061 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:46:24.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.175+0000 7f8530549640 1 Processor -- start 2024-04-17T10:46:24.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.175+0000 7f8530549640 1 -- start start 2024-04-17T10:46:24.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.175+0000 7f8530549640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f852815dfa0 0x7f85280b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:24.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.175+0000 7f8530549640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f85280ce980 0x7f85280b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:24.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.175+0000 7f8530549640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85280cee30 0x7f85280b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:24.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.175+0000 7f8530549640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8528162a00 con 0x7f85280ce980 2024-04-17T10:46:24.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.175+0000 7f8530549640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8528161e80 con 0x7f852815dfa0 2024-04-17T10:46:24.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.175+0000 7f8530549640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8528162160 con 0x7f85280cee30 2024-04-17T10:46:24.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.175+0000 7f852dae4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f85280ce980 0x7f85280b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:24.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.175+0000 7f852dae4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f85280ce980 0x7f85280b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52512/0 (socket says 172.21.15.83:52512) 2024-04-17T10:46:24.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.175+0000 7f852dae4640 1 -- 172.21.15.83:0/3075479773 learned_addr learned my addr 172.21.15.83:0/3075479773 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:24.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.175+0000 7f852e2e5640 1 --2- 172.21.15.83:0/3075479773 >> v2:172.21.15.167:3300/0 conn(0x7f852815dfa0 0x7f85280b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:24.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.175+0000 7f852eae6640 1 --2- 172.21.15.83:0/3075479773 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85280cee30 0x7f85280b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:24.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.175+0000 7f852dae4640 1 -- 172.21.15.83:0/3075479773 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85280cee30 msgr2=0x7f85280b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:24.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.175+0000 7f852dae4640 1 --2- 172.21.15.83:0/3075479773 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85280cee30 0x7f85280b5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.175+0000 7f852dae4640 1 -- 172.21.15.83:0/3075479773 >> v2:172.21.15.167:3300/0 conn(0x7f852815dfa0 msgr2=0x7f85280b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:24.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.175+0000 7f852dae4640 1 --2- 172.21.15.83:0/3075479773 >> v2:172.21.15.167:3300/0 conn(0x7f852815dfa0 0x7f85280b6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.175+0000 7f852dae4640 1 -- 172.21.15.83:0/3075479773 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8528162440 con 0x7f85280ce980 2024-04-17T10:46:24.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f852dae4640 1 --2- 172.21.15.83:0/3075479773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85280ce980 0x7f85280b5430 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f852808c3f0 tx=0x7f852401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5694806b84a580fb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:24.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f852d2e3640 1 -- 172.21.15.83:0/3075479773 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85240b1070 con 0x7f85280ce980 2024-04-17T10:46:24.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f852d2e3640 1 -- 172.21.15.83:0/3075479773 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85240ac620 con 0x7f85280ce980 2024-04-17T10:46:24.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f852d2e3640 1 -- 172.21.15.83:0/3075479773 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85240ac9f0 con 0x7f85280ce980 2024-04-17T10:46:24.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f8530549640 1 -- 172.21.15.83:0/3075479773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85280ce980 msgr2=0x7f85280b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:24.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f8530549640 1 --2- 172.21.15.83:0/3075479773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85280ce980 0x7f85280b5430 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f852808c3f0 tx=0x7f852401c460 comp rx=0 tx=0).stop 2024-04-17T10:46:24.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f8530549640 1 -- 172.21.15.83:0/3075479773 shutdown_connections 2024-04-17T10:46:24.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f8530549640 1 --2- 172.21.15.83:0/3075479773 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85280cee30 0x7f85280b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f8530549640 1 --2- 172.21.15.83:0/3075479773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85280ce980 0x7f85280b5430 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f8530549640 1 --2- 172.21.15.83:0/3075479773 >> v2:172.21.15.167:3300/0 conn(0x7f852815dfa0 0x7f85280b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f8530549640 1 -- 172.21.15.83:0/3075479773 >> 172.21.15.83:0/3075479773 conn(0x7f85280d0600 msgr2=0x7f85280d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:24.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f8530549640 1 -- 172.21.15.83:0/3075479773 shutdown_connections 2024-04-17T10:46:24.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f8530549640 1 -- 172.21.15.83:0/3075479773 wait complete. 2024-04-17T10:46:24.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f8530549640 1 Processor -- start 2024-04-17T10:46:24.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f8530549640 1 -- start start 2024-04-17T10:46:24.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f8530549640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85280ce980 0x7f852815af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:24.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f8530549640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85280cee30 0x7f8528159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:24.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f8530549640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f852815dfa0 0x7f852814fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:24.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f8530549640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85280d2540 con 0x7f85280ce980 2024-04-17T10:46:24.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f8530549640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8528161b10 con 0x7f85280cee30 2024-04-17T10:46:24.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f8530549640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85280d26c0 con 0x7f852815dfa0 2024-04-17T10:46:24.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f852e2e5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85280ce980 0x7f852815af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:24.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f852e2e5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85280ce980 0x7f852815af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52520/0 (socket says 172.21.15.83:52520) 2024-04-17T10:46:24.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f852e2e5640 1 -- 172.21.15.83:0/2316510222 learned_addr learned my addr 172.21.15.83:0/2316510222 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:24.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f852e2e5640 1 -- 172.21.15.83:0/2316510222 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f852815dfa0 msgr2=0x7f852814fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:24.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f852e2e5640 1 --2- 172.21.15.83:0/2316510222 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f852815dfa0 0x7f852814fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f852e2e5640 1 -- 172.21.15.83:0/2316510222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85280cee30 msgr2=0x7f8528159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:24.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f852e2e5640 1 --2- 172.21.15.83:0/2316510222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85280cee30 0x7f8528159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f852e2e5640 1 -- 172.21.15.83:0/2316510222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8528159ca0 con 0x7f85280ce980 2024-04-17T10:46:24.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f852e2e5640 1 --2- 172.21.15.83:0/2316510222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85280ce980 0x7f852815af80 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f851c00d620 tx=0x7f851c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:24.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f851affd640 1 -- 172.21.15.83:0/2316510222 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c00b620 con 0x7f85280ce980 2024-04-17T10:46:24.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f851affd640 1 -- 172.21.15.83:0/2316510222 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851c019070 con 0x7f85280ce980 2024-04-17T10:46:24.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f851affd640 1 -- 172.21.15.83:0/2316510222 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c00b9e0 con 0x7f85280ce980 2024-04-17T10:46:24.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f8530549640 1 -- 172.21.15.83:0/2316510222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85281538f0 con 0x7f85280ce980 2024-04-17T10:46:24.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.179+0000 7f8530549640 1 -- 172.21.15.83:0/2316510222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f852815a4a0 con 0x7f85280ce980 2024-04-17T10:46:24.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.183+0000 7f8530549640 1 -- 172.21.15.83:0/2316510222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8528074f90 con 0x7f85280ce980 2024-04-17T10:46:24.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.183+0000 7f851affd640 1 -- 172.21.15.83:0/2316510222 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f851c00bb80 con 0x7f85280ce980 2024-04-17T10:46:24.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.183+0000 7f851affd640 1 --2- 172.21.15.83:0/2316510222 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f850c066f60 0x7f850c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:24.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.183+0000 7f852dae4640 1 --2- 172.21.15.83:0/2316510222 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f850c066f60 0x7f850c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:24.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.183+0000 7f851affd640 1 -- 172.21.15.83:0/2316510222 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f851c085410 con 0x7f85280ce980 2024-04-17T10:46:24.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.183+0000 7f852dae4640 1 --2- 172.21.15.83:0/2316510222 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f850c066f60 0x7f850c069420 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f85240ae000 tx=0x7f8524004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:24.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.187+0000 7f851affd640 1 -- 172.21.15.83:0/2316510222 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f851c056450 con 0x7f85280ce980 2024-04-17T10:46:24.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.367+0000 7f8530549640 1 -- 172.21.15.83:0/2316510222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f852806c7d0 con 0x7f85280ce980 2024-04-17T10:46:24.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.367+0000 7f851affd640 1 -- 172.21.15.83:0/2316510222 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f851c055d50 con 0x7f85280ce980 2024-04-17T10:46:24.365 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:46:24.365 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5258,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2259892875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2259892875},{"type":"v1","addr":"172.21.15.167:6835","nonce":2259892875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":7095,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/3769102527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3769102527},{"type":"v1","addr":"172.21.15.83:6836","nonce":3769102527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":7098,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/395594830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":395594830},{"type":"v1","addr":"172.21.15.83:6837","nonce":395594830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-17T10:43:33.830720+0000","modified":"2024-04-17T10:43:47.538354+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7104},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7104":{"gid":7104,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6833/2796648400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2796648400},{"type":"v1","addr":"172.21.15.167:6833","nonce":2796648400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-17T10:46:24.365 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 49 2024-04-17T10:46:24.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.371+0000 7f8530549640 1 -- 172.21.15.83:0/2316510222 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f850c066f60 msgr2=0x7f850c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:24.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.371+0000 7f8530549640 1 --2- 172.21.15.83:0/2316510222 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f850c066f60 0x7f850c069420 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f85240ae000 tx=0x7f8524004740 comp rx=0 tx=0).stop 2024-04-17T10:46:24.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.371+0000 7f8530549640 1 -- 172.21.15.83:0/2316510222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85280ce980 msgr2=0x7f852815af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:24.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.371+0000 7f8530549640 1 --2- 172.21.15.83:0/2316510222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85280ce980 0x7f852815af80 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f851c00d620 tx=0x7f851c00dae0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.371+0000 7f8530549640 1 -- 172.21.15.83:0/2316510222 shutdown_connections 2024-04-17T10:46:24.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.371+0000 7f8530549640 1 --2- 172.21.15.83:0/2316510222 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f850c066f60 0x7f850c069420 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.371+0000 7f8530549640 1 --2- 172.21.15.83:0/2316510222 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f852815dfa0 0x7f852814fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.371+0000 7f8530549640 1 --2- 172.21.15.83:0/2316510222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85280cee30 0x7f8528159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.371+0000 7f8530549640 1 --2- 172.21.15.83:0/2316510222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85280ce980 0x7f852815af80 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.371+0000 7f8530549640 1 -- 172.21.15.83:0/2316510222 >> 172.21.15.83:0/2316510222 conn(0x7f85280d0600 msgr2=0x7f8528154280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:24.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.371+0000 7f8530549640 1 -- 172.21.15.83:0/2316510222 shutdown_connections 2024-04-17T10:46:24.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.371+0000 7f8530549640 1 -- 172.21.15.83:0/2316510222 wait complete. 2024-04-17T10:46:24.382 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:46:24.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.495+0000 7fa8c5454640 1 Processor -- start 2024-04-17T10:46:24.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.495+0000 7fa8c5454640 1 -- start start 2024-04-17T10:46:24.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.495+0000 7fa8c5454640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa8c015b770 0x7fa8c006d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:24.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.495+0000 7fa8c5454640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa8c0072220 0x7fa8c00cd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:24.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.495+0000 7fa8c5454640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8c006e540 con 0x7fa8c00cbde0 2024-04-17T10:46:24.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.495+0000 7fa8c5454640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c006d160 con 0x7fa8c0072220 2024-04-17T10:46:24.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.495+0000 7fa8c5454640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c006c270 con 0x7fa8c015b770 2024-04-17T10:46:24.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.495+0000 7fa8c4c53640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa8c0072220 0x7fa8c00cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:24.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.495+0000 7fa8c4c53640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa8c0072220 0x7fa8c00cd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52524/0 (socket says 172.21.15.83:52524) 2024-04-17T10:46:24.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.495+0000 7fa8c4c53640 1 -- 172.21.15.83:0/13036758 learned_addr learned my addr 172.21.15.83:0/13036758 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:24.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.495+0000 7fa8bffff640 1 --2- 172.21.15.83:0/13036758 >> v2:172.21.15.167:3300/0 conn(0x7fa8c015b770 0x7fa8c006d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:24.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.495+0000 7fa8c4c53640 1 -- 172.21.15.83:0/13036758 >> v2:172.21.15.167:3300/0 conn(0x7fa8c015b770 msgr2=0x7fa8c006d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:24.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.495+0000 7fa8c4c53640 1 --2- 172.21.15.83:0/13036758 >> v2:172.21.15.167:3300/0 conn(0x7fa8c015b770 0x7fa8c006d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.495+0000 7fa8c4c53640 1 -- 172.21.15.83:0/13036758 >> v1:172.21.15.167:6789/0 conn(0x7fa8c00cbde0 legacy=0x7fa8c0071b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:24.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.495+0000 7fa8c4c53640 1 -- 172.21.15.83:0/13036758 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c006c550 con 0x7fa8c0072220 2024-04-17T10:46:24.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.499+0000 7fa8c4c53640 1 --2- 172.21.15.83:0/13036758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8c0072220 0x7fa8c00cd490 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0067510 tx=0x7fa8b801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15dcf91a6582745 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:24.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.499+0000 7fa8bf7fe640 1 -- 172.21.15.83:0/13036758 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b80a5730 con 0x7fa8c0072220 2024-04-17T10:46:24.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.499+0000 7fa8bf7fe640 1 -- 172.21.15.83:0/13036758 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b80a58d0 con 0x7fa8c0072220 2024-04-17T10:46:24.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.499+0000 7fa8bf7fe640 1 -- 172.21.15.83:0/13036758 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b80a5ca0 con 0x7fa8c0072220 2024-04-17T10:46:24.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.499+0000 7fa8c5454640 1 -- 172.21.15.83:0/13036758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8c0072220 msgr2=0x7fa8c00cd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:24.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.499+0000 7fa8c5454640 1 --2- 172.21.15.83:0/13036758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8c0072220 0x7fa8c00cd490 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0067510 tx=0x7fa8b801c930 comp rx=0 tx=0).stop 2024-04-17T10:46:24.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.499+0000 7fa8c5454640 1 -- 172.21.15.83:0/13036758 shutdown_connections 2024-04-17T10:46:24.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.499+0000 7fa8c5454640 1 --2- 172.21.15.83:0/13036758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8c0072220 0x7fa8c00cd490 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.499+0000 7fa8c5454640 1 --2- 172.21.15.83:0/13036758 >> v2:172.21.15.167:3300/0 conn(0x7fa8c015b770 0x7fa8c006d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.499+0000 7fa8c5454640 1 -- 172.21.15.83:0/13036758 >> 172.21.15.83:0/13036758 conn(0x7fa8c00b1160 msgr2=0x7fa8c00b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:24.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.499+0000 7fa8c5454640 1 -- 172.21.15.83:0/13036758 shutdown_connections 2024-04-17T10:46:24.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.499+0000 7fa8c5454640 1 -- 172.21.15.83:0/13036758 wait complete. 2024-04-17T10:46:24.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.499+0000 7fa8c5454640 1 Processor -- start 2024-04-17T10:46:24.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.499+0000 7fa8c5454640 1 -- start start 2024-04-17T10:46:24.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.499+0000 7fa8c5454640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8c0072220 0x7fa8c00c4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:24.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.499+0000 7fa8c5454640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8c00cbde0 0x7fa8c01638b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:24.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.499+0000 7fa8c5454640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8c015b770 0x7fa8c0158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:24.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.499+0000 7fa8c5454640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c015bb50 con 0x7fa8c015b770 2024-04-17T10:46:24.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.499+0000 7fa8c5454640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c006db30 con 0x7fa8c00cbde0 2024-04-17T10:46:24.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.499+0000 7fa8c5454640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c015bcd0 con 0x7fa8c0072220 2024-04-17T10:46:24.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.499+0000 7fa8c4c53640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8c015b770 0x7fa8c0158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:24.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.499+0000 7fa8c4c53640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8c015b770 0x7fa8c0158040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52536/0 (socket says 172.21.15.83:52536) 2024-04-17T10:46:24.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.499+0000 7fa8c4c53640 1 -- 172.21.15.83:0/2090281894 learned_addr learned my addr 172.21.15.83:0/2090281894 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:24.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.499+0000 7fa8bffff640 1 --2- 172.21.15.83:0/2090281894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8c0072220 0x7fa8c00c4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:24.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.499+0000 7fa8b7fff640 1 --2- 172.21.15.83:0/2090281894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8c00cbde0 0x7fa8c01638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:24.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.499+0000 7fa8c4c53640 1 -- 172.21.15.83:0/2090281894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8c0072220 msgr2=0x7fa8c00c4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:24.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.499+0000 7fa8c4c53640 1 --2- 172.21.15.83:0/2090281894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8c0072220 0x7fa8c00c4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.499+0000 7fa8c4c53640 1 -- 172.21.15.83:0/2090281894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8c00cbde0 msgr2=0x7fa8c01638b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:24.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.499+0000 7fa8c4c53640 1 --2- 172.21.15.83:0/2090281894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8c00cbde0 0x7fa8c01638b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.499+0000 7fa8c4c53640 1 -- 172.21.15.83:0/2090281894 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c00b22f0 con 0x7fa8c015b770 2024-04-17T10:46:24.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.503+0000 7fa8c4c53640 1 --2- 172.21.15.83:0/2090281894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8c015b770 0x7fa8c0158040 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8019540 tx=0x7fa8b80196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:24.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.503+0000 7fa8bd7fa640 1 -- 172.21.15.83:0/2090281894 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b80bc020 con 0x7fa8c015b770 2024-04-17T10:46:24.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.503+0000 7fa8bd7fa640 1 -- 172.21.15.83:0/2090281894 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b80b8670 con 0x7fa8c015b770 2024-04-17T10:46:24.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.503+0000 7fa8c5454640 1 -- 172.21.15.83:0/2090281894 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c0158580 con 0x7fa8c015b770 2024-04-17T10:46:24.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.503+0000 7fa8bd7fa640 1 -- 172.21.15.83:0/2090281894 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b80b8a60 con 0x7fa8c015b770 2024-04-17T10:46:24.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.503+0000 7fa8c5454640 1 -- 172.21.15.83:0/2090281894 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c00c57c0 con 0x7fa8c015b770 2024-04-17T10:46:24.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.503+0000 7fa8c5454640 1 -- 172.21.15.83:0/2090281894 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c004efa0 con 0x7fa8c015b770 2024-04-17T10:46:24.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.503+0000 7fa8bd7fa640 1 -- 172.21.15.83:0/2090281894 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa8b80bf020 con 0x7fa8c015b770 2024-04-17T10:46:24.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.503+0000 7fa8bd7fa640 1 --2- 172.21.15.83:0/2090281894 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa8a0066f60 0x7fa8a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:24.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.503+0000 7fa8bd7fa640 1 -- 172.21.15.83:0/2090281894 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa8b8120180 con 0x7fa8c015b770 2024-04-17T10:46:24.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.503+0000 7fa8bffff640 1 --2- 172.21.15.83:0/2090281894 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa8a0066f60 0x7fa8a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:24.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.503+0000 7fa8bffff640 1 --2- 172.21.15.83:0/2090281894 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa8a0066f60 0x7fa8a0069420 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa8b000b930 tx=0x7fa8b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:24.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.507+0000 7fa8bd7fa640 1 -- 172.21.15.83:0/2090281894 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa8b80c6030 con 0x7fa8c015b770 2024-04-17T10:46:24.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.655+0000 7fa8c5454640 1 -- 172.21.15.83:0/2090281894 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8c00716e0 con 0x7fa8c015b770 2024-04-17T10:46:24.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.659+0000 7fa8bd7fa640 1 -- 172.21.15.83:0/2090281894 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16759 (secure 0 0 0) 0x7fa8b80165e0 con 0x7fa8c015b770 2024-04-17T10:46:24.656 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:46:24.656 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":78,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:45:57.116959+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:43:32.090869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-17T10:43:32.815921+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":77,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.4","mappings":[{"from":6,"to":0}]},{"pgid":"11.2d","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:46:24.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.663+0000 7fa8c5454640 1 -- 172.21.15.83:0/2090281894 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa8a0066f60 msgr2=0x7fa8a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:24.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.663+0000 7fa8c5454640 1 --2- 172.21.15.83:0/2090281894 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa8a0066f60 0x7fa8a0069420 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa8b000b930 tx=0x7fa8b0008040 comp rx=0 tx=0).stop 2024-04-17T10:46:24.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.663+0000 7fa8c5454640 1 -- 172.21.15.83:0/2090281894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8c015b770 msgr2=0x7fa8c0158040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:24.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.663+0000 7fa8c5454640 1 --2- 172.21.15.83:0/2090281894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8c015b770 0x7fa8c0158040 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8019540 tx=0x7fa8b80196d0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.663+0000 7fa8c5454640 1 -- 172.21.15.83:0/2090281894 shutdown_connections 2024-04-17T10:46:24.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.663+0000 7fa8c5454640 1 --2- 172.21.15.83:0/2090281894 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa8a0066f60 0x7fa8a0069420 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.663+0000 7fa8c5454640 1 --2- 172.21.15.83:0/2090281894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8c015b770 0x7fa8c0158040 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.663+0000 7fa8c5454640 1 --2- 172.21.15.83:0/2090281894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8c00cbde0 0x7fa8c01638b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.663+0000 7fa8c5454640 1 --2- 172.21.15.83:0/2090281894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8c0072220 0x7fa8c00c4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.663+0000 7fa8c5454640 1 -- 172.21.15.83:0/2090281894 >> 172.21.15.83:0/2090281894 conn(0x7fa8c00b1160 msgr2=0x7fa8c015e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:24.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.663+0000 7fa8c5454640 1 -- 172.21.15.83:0/2090281894 shutdown_connections 2024-04-17T10:46:24.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.663+0000 7fa8c5454640 1 -- 172.21.15.83:0/2090281894 wait complete. 2024-04-17T10:46:24.675 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-17T10:46:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.795+0000 7f413bd55640 1 Processor -- start 2024-04-17T10:46:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.795+0000 7f413bd55640 1 -- start start 2024-04-17T10:46:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.795+0000 7f413bd55640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f413406b520 0x7f41340b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.795+0000 7f413bd55640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f41340b4040 0x7f413406cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.795+0000 7f413bd55640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f413415bd70 con 0x7f4134154de0 2024-04-17T10:46:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.795+0000 7f413bd55640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41340b4870 con 0x7f413406b520 2024-04-17T10:46:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.795+0000 7f413bd55640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f413415bf60 con 0x7f41340b4040 2024-04-17T10:46:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.795+0000 7f41392f0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f413406b520 0x7f41340b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.795+0000 7f41392f0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f413406b520 0x7f41340b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52552/0 (socket says 172.21.15.83:52552) 2024-04-17T10:46:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.795+0000 7f41392f0640 1 -- 172.21.15.83:0/2384768549 learned_addr learned my addr 172.21.15.83:0/2384768549 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.795+0000 7f413a2f2640 1 --2- 172.21.15.83:0/2384768549 >> v2:172.21.15.167:3300/0 conn(0x7f41340b4040 0x7f413406cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.795+0000 7f41392f0640 1 -- 172.21.15.83:0/2384768549 >> v2:172.21.15.167:3300/0 conn(0x7f41340b4040 msgr2=0x7f413406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.795+0000 7f41392f0640 1 --2- 172.21.15.83:0/2384768549 >> v2:172.21.15.167:3300/0 conn(0x7f41340b4040 0x7f413406cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.795+0000 7f41392f0640 1 -- 172.21.15.83:0/2384768549 >> v1:172.21.15.83:6789/0 conn(0x7f4134154de0 legacy=0x7f413406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.795+0000 7f41392f0640 1 -- 172.21.15.83:0/2384768549 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413415afb0 con 0x7f413406b520 2024-04-17T10:46:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.795+0000 7f4138aef640 1 -- 172.21.15.83:0/2384768549 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1064089444 0 0) 0x7f413415bd70 con 0x7f4134154de0 2024-04-17T10:46:24.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.799+0000 7f41392f0640 1 --2- 172.21.15.83:0/2384768549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f413406b520 0x7f41340b3b00 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f413406e8d0 tx=0x7f412401ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc15e71193ce3a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:24.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.799+0000 7f4138aef640 1 -- 172.21.15.83:0/2384768549 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41240b4070 con 0x7f413406b520 2024-04-17T10:46:24.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.799+0000 7f4138aef640 1 -- 172.21.15.83:0/2384768549 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41240a7bb0 con 0x7f413406b520 2024-04-17T10:46:24.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.799+0000 7f4138aef640 1 -- 172.21.15.83:0/2384768549 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41240af440 con 0x7f413406b520 2024-04-17T10:46:24.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.799+0000 7f413bd55640 1 -- 172.21.15.83:0/2384768549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f413406b520 msgr2=0x7f41340b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:24.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.799+0000 7f413bd55640 1 --2- 172.21.15.83:0/2384768549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f413406b520 0x7f41340b3b00 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f413406e8d0 tx=0x7f412401ebf0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.799+0000 7f413bd55640 1 -- 172.21.15.83:0/2384768549 shutdown_connections 2024-04-17T10:46:24.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.799+0000 7f413bd55640 1 --2- 172.21.15.83:0/2384768549 >> v2:172.21.15.167:3300/0 conn(0x7f41340b4040 0x7f413406cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.799+0000 7f413bd55640 1 --2- 172.21.15.83:0/2384768549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f413406b520 0x7f41340b3b00 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.799+0000 7f413bd55640 1 -- 172.21.15.83:0/2384768549 >> 172.21.15.83:0/2384768549 conn(0x7f41340b15b0 msgr2=0x7f41340b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:24.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.799+0000 7f413bd55640 1 -- 172.21.15.83:0/2384768549 shutdown_connections 2024-04-17T10:46:24.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.799+0000 7f413bd55640 1 -- 172.21.15.83:0/2384768549 wait complete. 2024-04-17T10:46:24.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.799+0000 7f413bd55640 1 Processor -- start 2024-04-17T10:46:24.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.799+0000 7f413bd55640 1 -- start start 2024-04-17T10:46:24.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.803+0000 7f413bd55640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f413406b520 0x7f413414c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:24.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.803+0000 7f413bd55640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41340b4040 0x7f41341553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:24.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.803+0000 7f413bd55640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4134154de0 0x7f413413aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:24.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.803+0000 7f413bd55640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41340b4420 con 0x7f413406b520 2024-04-17T10:46:24.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.803+0000 7f413bd55640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41341551c0 con 0x7f4134154de0 2024-04-17T10:46:24.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.803+0000 7f413bd55640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41340b45a0 con 0x7f41340b4040 2024-04-17T10:46:24.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.803+0000 7f4139af1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f413406b520 0x7f413414c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:24.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.803+0000 7f4139af1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f413406b520 0x7f413414c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52554/0 (socket says 172.21.15.83:52554) 2024-04-17T10:46:24.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.803+0000 7f4139af1640 1 -- 172.21.15.83:0/3352030682 learned_addr learned my addr 172.21.15.83:0/3352030682 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:24.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.803+0000 7f4139af1640 1 -- 172.21.15.83:0/3352030682 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41340b4040 msgr2=0x7f41341553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:24.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.803+0000 7f4139af1640 1 --2- 172.21.15.83:0/3352030682 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41340b4040 0x7f41341553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.803+0000 7f4139af1640 1 -- 172.21.15.83:0/3352030682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4134154de0 msgr2=0x7f413413aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:24.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.803+0000 7f4139af1640 1 --2- 172.21.15.83:0/3352030682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4134154de0 0x7f413413aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.803+0000 7f4139af1640 1 -- 172.21.15.83:0/3352030682 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4134155b40 con 0x7f413406b520 2024-04-17T10:46:24.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.803+0000 7f4139af1640 1 --2- 172.21.15.83:0/3352030682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f413406b520 0x7f413414c660 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f413000c7a0 tx=0x7f413000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:24.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.803+0000 7f412a7fc640 1 -- 172.21.15.83:0/3352030682 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4130017070 con 0x7f413406b520 2024-04-17T10:46:24.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.803+0000 7f412a7fc640 1 -- 172.21.15.83:0/3352030682 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4130003db0 con 0x7f413406b520 2024-04-17T10:46:24.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.803+0000 7f413bd55640 1 -- 172.21.15.83:0/3352030682 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4134155e20 con 0x7f413406b520 2024-04-17T10:46:24.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.803+0000 7f412a7fc640 1 -- 172.21.15.83:0/3352030682 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4130005270 con 0x7f413406b520 2024-04-17T10:46:24.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.803+0000 7f413bd55640 1 -- 172.21.15.83:0/3352030682 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41341563e0 con 0x7f413406b520 2024-04-17T10:46:24.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.803+0000 7f413bd55640 1 -- 172.21.15.83:0/3352030682 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4134074f90 con 0x7f413406b520 2024-04-17T10:46:24.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.803+0000 7f412a7fc640 1 -- 172.21.15.83:0/3352030682 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f413001a070 con 0x7f413406b520 2024-04-17T10:46:24.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.803+0000 7f412a7fc640 1 --2- 172.21.15.83:0/3352030682 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4118066f60 0x7f4118069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:24.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.803+0000 7f412a7fc640 1 -- 172.21.15.83:0/3352030682 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f41300845f0 con 0x7f413406b520 2024-04-17T10:46:24.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.803+0000 7f41392f0640 1 --2- 172.21.15.83:0/3352030682 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4118066f60 0x7f4118069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:24.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.803+0000 7f41392f0640 1 --2- 172.21.15.83:0/3352030682 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4118066f60 0x7f4118069420 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f41240b1000 tx=0x7f4124004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:24.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.807+0000 7f412a7fc640 1 -- 172.21.15.83:0/3352030682 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4130054fe0 con 0x7f413406b520 2024-04-17T10:46:24.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.983+0000 7f413bd55640 1 -- 172.21.15.83:0/3352030682 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f41340b3280 con 0x7f413406b520 2024-04-17T10:46:24.981 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-04-17T10:46:24.983+0000 7f35c3b81640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T10:46:24.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.987+0000 7f412a7fc640 1 -- 172.21.15.83:0/3352030682 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v50) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4130059080 con 0x7f413406b520 2024-04-17T10:46:24.983 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-17T10:46:24.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.991+0000 7f413bd55640 1 -- 172.21.15.83:0/3352030682 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4118066f60 msgr2=0x7f4118069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:24.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.991+0000 7f413bd55640 1 --2- 172.21.15.83:0/3352030682 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4118066f60 0x7f4118069420 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f41240b1000 tx=0x7f4124004740 comp rx=0 tx=0).stop 2024-04-17T10:46:24.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.991+0000 7f413bd55640 1 -- 172.21.15.83:0/3352030682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f413406b520 msgr2=0x7f413414c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:24.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.991+0000 7f413bd55640 1 --2- 172.21.15.83:0/3352030682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f413406b520 0x7f413414c660 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f413000c7a0 tx=0x7f413000cc60 comp rx=0 tx=0).stop 2024-04-17T10:46:24.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.991+0000 7f413bd55640 1 -- 172.21.15.83:0/3352030682 shutdown_connections 2024-04-17T10:46:24.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.991+0000 7f413bd55640 1 --2- 172.21.15.83:0/3352030682 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4118066f60 0x7f4118069420 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.991+0000 7f413bd55640 1 --2- 172.21.15.83:0/3352030682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4134154de0 0x7f413413aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.991+0000 7f413bd55640 1 --2- 172.21.15.83:0/3352030682 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41340b4040 0x7f41341553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:24.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.991+0000 7f413bd55640 1 --2- 172.21.15.83:0/3352030682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f413406b520 0x7f413414c660 secure :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f413000c7a0 tx=0x7f413000cc60 comp rx=0 tx=0).stop 2024-04-17T10:46:24.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.991+0000 7f413bd55640 1 -- 172.21.15.83:0/3352030682 >> 172.21.15.83:0/3352030682 conn(0x7f41340b15b0 msgr2=0x7f4134159a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:24.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.991+0000 7f413bd55640 1 -- 172.21.15.83:0/3352030682 shutdown_connections 2024-04-17T10:46:24.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:24.991+0000 7f413bd55640 1 -- 172.21.15.83:0/3352030682 wait complete. 2024-04-17T10:46:25.005 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-17T10:46:25.080 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2024-04-17T10:46:25.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.119+0000 7fac66d07640 1 Processor -- start 2024-04-17T10:46:25.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.119+0000 7fac66d07640 1 -- start start 2024-04-17T10:46:25.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.119+0000 7fac66d07640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fac6015b6e0 0x7fac60071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:25.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.123+0000 7fac66d07640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac6006c1f0 con 0x7fac6015bac0 2024-04-17T10:46:25.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.123+0000 7fac66d07640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac60070a10 con 0x7fac60070470 2024-04-17T10:46:25.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.123+0000 7fac66d07640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac60070cf0 con 0x7fac6015b6e0 2024-04-17T10:46:25.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.123+0000 7fac5ffff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fac6015bac0 0x7fac600cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:44188/0 (socket says 172.21.15.83:44188) 2024-04-17T10:46:25.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.123+0000 7fac5ffff640 1 -- 172.21.15.83:0/823527665 learned_addr learned my addr 172.21.15.83:0/823527665 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:46:25.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.123+0000 7fac64aa3640 1 --2- 172.21.15.83:0/823527665 >> v2:172.21.15.167:3300/0 conn(0x7fac6015b6e0 0x7fac60071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:25.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.123+0000 7fac5f7fe640 1 -- 172.21.15.83:0/823527665 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3333314827 0 0) 0x7fac6006c1f0 con 0x7fac6015bac0 2024-04-17T10:46:25.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.123+0000 7fac5f7fe640 1 -- 172.21.15.83:0/823527665 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac48003530 con 0x7fac6015bac0 2024-04-17T10:46:25.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.123+0000 7fac64aa3640 1 -- 172.21.15.83:0/823527665 >> v1:172.21.15.167:6789/0 conn(0x7fac60070470 legacy=0x7fac6006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:25.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.123+0000 7fac64aa3640 1 -- 172.21.15.83:0/823527665 >> v1:172.21.15.83:6789/0 conn(0x7fac6015bac0 legacy=0x7fac600cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:25.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.123+0000 7fac64aa3640 1 -- 172.21.15.83:0/823527665 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac60070fd0 con 0x7fac6015b6e0 2024-04-17T10:46:25.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.123+0000 7fac64aa3640 1 --2- 172.21.15.83:0/823527665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac6015b6e0 0x7fac60071cc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fac500097c0 tx=0x7fac5001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fdab28ade9581fa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:25.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.123+0000 7fac5f7fe640 1 -- 172.21.15.83:0/823527665 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac500b0070 con 0x7fac6015b6e0 2024-04-17T10:46:25.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.123+0000 7fac5f7fe640 1 -- 172.21.15.83:0/823527665 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac500ab600 con 0x7fac6015b6e0 2024-04-17T10:46:25.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.123+0000 7fac5f7fe640 1 -- 172.21.15.83:0/823527665 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac500ab9b0 con 0x7fac6015b6e0 2024-04-17T10:46:25.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.123+0000 7fac66d07640 1 -- 172.21.15.83:0/823527665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac6015b6e0 msgr2=0x7fac60071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:25.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.123+0000 7fac66d07640 1 --2- 172.21.15.83:0/823527665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac6015b6e0 0x7fac60071cc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fac500097c0 tx=0x7fac5001c4a0 comp rx=0 tx=0).stop 2024-04-17T10:46:25.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.123+0000 7fac66d07640 1 -- 172.21.15.83:0/823527665 shutdown_connections 2024-04-17T10:46:25.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.123+0000 7fac66d07640 1 --2- 172.21.15.83:0/823527665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac6015b6e0 0x7fac60071cc0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:25.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.123+0000 7fac66d07640 1 -- 172.21.15.83:0/823527665 >> 172.21.15.83:0/823527665 conn(0x7fac600b15b0 msgr2=0x7fac600b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:25.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.123+0000 7fac66d07640 1 -- 172.21.15.83:0/823527665 shutdown_connections 2024-04-17T10:46:25.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.123+0000 7fac66d07640 1 -- 172.21.15.83:0/823527665 wait complete. 2024-04-17T10:46:25.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.127+0000 7fac66d07640 1 Processor -- start 2024-04-17T10:46:25.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.127+0000 7fac66d07640 1 -- start start 2024-04-17T10:46:25.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.127+0000 7fac66d07640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac60070470 0x7fac601511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:25.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.127+0000 7fac66d07640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac6015b6e0 0x7fac6015bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:25.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.127+0000 7fac66d07640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac6015bac0 0x7fac60159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:25.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.127+0000 7fac66d07640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6006cee0 con 0x7fac6015bac0 2024-04-17T10:46:25.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.127+0000 7fac66d07640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6006d060 con 0x7fac60070470 2024-04-17T10:46:25.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.127+0000 7fac66d07640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac6006d1e0 con 0x7fac6015b6e0 2024-04-17T10:46:25.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.127+0000 7fac652a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac6015bac0 0x7fac60159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:25.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.127+0000 7fac652a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac6015bac0 0x7fac60159eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52564/0 (socket says 172.21.15.83:52564) 2024-04-17T10:46:25.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.127+0000 7fac652a4640 1 -- 172.21.15.83:0/1475046362 learned_addr learned my addr 172.21.15.83:0/1475046362 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:25.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.127+0000 7fac64aa3640 1 --2- 172.21.15.83:0/1475046362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac60070470 0x7fac601511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:25.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.127+0000 7fac5ffff640 1 --2- 172.21.15.83:0/1475046362 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac6015b6e0 0x7fac6015bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:25.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.127+0000 7fac652a4640 1 -- 172.21.15.83:0/1475046362 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac6015b6e0 msgr2=0x7fac6015bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:25.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.127+0000 7fac652a4640 1 --2- 172.21.15.83:0/1475046362 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac6015b6e0 0x7fac6015bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:25.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.127+0000 7fac652a4640 1 -- 172.21.15.83:0/1475046362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac60070470 msgr2=0x7fac601511d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:25.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.127+0000 7fac652a4640 1 --2- 172.21.15.83:0/1475046362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac60070470 0x7fac601511d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:25.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.127+0000 7fac652a4640 1 -- 172.21.15.83:0/1475046362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac60158e50 con 0x7fac6015bac0 2024-04-17T10:46:25.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.127+0000 7fac64aa3640 1 --2- 172.21.15.83:0/1475046362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac60070470 0x7fac601511d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:46:25.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.127+0000 7fac652a4640 1 --2- 172.21.15.83:0/1475046362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac6015bac0 0x7fac60159eb0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fac5800c760 tx=0x7fac5800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:25.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.127+0000 7fac5d7fa640 1 -- 172.21.15.83:0/1475046362 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac58003b80 con 0x7fac6015bac0 2024-04-17T10:46:25.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.127+0000 7fac5d7fa640 1 -- 172.21.15.83:0/1475046362 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac58003d20 con 0x7fac6015bac0 2024-04-17T10:46:25.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.127+0000 7fac66d07640 1 -- 172.21.15.83:0/1475046362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac601590f0 con 0x7fac6015bac0 2024-04-17T10:46:25.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.127+0000 7fac5d7fa640 1 -- 172.21.15.83:0/1475046362 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5800aa30 con 0x7fac6015bac0 2024-04-17T10:46:25.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.127+0000 7fac66d07640 1 -- 172.21.15.83:0/1475046362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac601596b0 con 0x7fac6015bac0 2024-04-17T10:46:25.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.127+0000 7fac66d07640 1 -- 172.21.15.83:0/1475046362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac60074f90 con 0x7fac6015bac0 2024-04-17T10:46:25.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.131+0000 7fac5d7fa640 1 -- 172.21.15.83:0/1475046362 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fac5800abd0 con 0x7fac6015bac0 2024-04-17T10:46:25.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.131+0000 7fac5d7fa640 1 --2- 172.21.15.83:0/1475046362 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fac4c066f60 0x7fac4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:25.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.131+0000 7fac64aa3640 1 --2- 172.21.15.83:0/1475046362 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fac4c066f60 0x7fac4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:25.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.131+0000 7fac5d7fa640 1 -- 172.21.15.83:0/1475046362 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fac58085a00 con 0x7fac6015bac0 2024-04-17T10:46:25.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.131+0000 7fac64aa3640 1 --2- 172.21.15.83:0/1475046362 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fac4c066f60 0x7fac4c069420 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fac500ad000 tx=0x7fac500a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:25.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.135+0000 7fac5d7fa640 1 -- 172.21.15.83:0/1475046362 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fac580569f0 con 0x7fac6015bac0 2024-04-17T10:46:25.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.311+0000 7fac66d07640 1 -- 172.21.15.83:0/1475046362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fac600cdbc0 con 0x7fac6015bac0 2024-04-17T10:46:25.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.991+0000 7fac5d7fa640 1 -- 172.21.15.83:0/1475046362 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v51) v1 ==== 104+0+0 (secure 0 0 0) 0x7fac580562f0 con 0x7fac6015bac0 2024-04-17T10:46:25.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.995+0000 7fac66d07640 1 -- 172.21.15.83:0/1475046362 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fac4c066f60 msgr2=0x7fac4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:25.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.995+0000 7fac66d07640 1 --2- 172.21.15.83:0/1475046362 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fac4c066f60 0x7fac4c069420 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fac500ad000 tx=0x7fac500a6000 comp rx=0 tx=0).stop 2024-04-17T10:46:25.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.995+0000 7fac66d07640 1 -- 172.21.15.83:0/1475046362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac6015bac0 msgr2=0x7fac60159eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:25.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.995+0000 7fac66d07640 1 --2- 172.21.15.83:0/1475046362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac6015bac0 0x7fac60159eb0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fac5800c760 tx=0x7fac5800cc20 comp rx=0 tx=0).stop 2024-04-17T10:46:25.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.995+0000 7fac66d07640 1 -- 172.21.15.83:0/1475046362 shutdown_connections 2024-04-17T10:46:25.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.995+0000 7fac66d07640 1 --2- 172.21.15.83:0/1475046362 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fac4c066f60 0x7fac4c069420 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:25.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.995+0000 7fac66d07640 1 --2- 172.21.15.83:0/1475046362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac6015bac0 0x7fac60159eb0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:25.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.995+0000 7fac66d07640 1 --2- 172.21.15.83:0/1475046362 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac6015b6e0 0x7fac6015bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:25.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.995+0000 7fac66d07640 1 --2- 172.21.15.83:0/1475046362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac60070470 0x7fac601511d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:25.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.995+0000 7fac66d07640 1 -- 172.21.15.83:0/1475046362 >> 172.21.15.83:0/1475046362 conn(0x7fac600b15b0 msgr2=0x7fac600b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:25.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.995+0000 7fac66d07640 1 -- 172.21.15.83:0/1475046362 shutdown_connections 2024-04-17T10:46:25.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:25.995+0000 7fac66d07640 1 -- 172.21.15.83:0/1475046362 wait complete. 2024-04-17T10:46:26.012 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-17T10:46:26.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.147+0000 7faab59a5640 1 Processor -- start 2024-04-17T10:46:26.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.147+0000 7faab59a5640 1 -- start start 2024-04-17T10:46:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.147+0000 7faab59a5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faab00aed60 0x7faab0162530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.147+0000 7faab59a5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faab00ce320 0x7faab00b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.147+0000 7faab59a5640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faab0162ea0 con 0x7faab006ac50 2024-04-17T10:46:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.147+0000 7faab59a5640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faab00b3470 con 0x7faab00ce320 2024-04-17T10:46:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.147+0000 7faab59a5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab0163180 con 0x7faab00aed60 2024-04-17T10:46:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.147+0000 7faaaeffd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faab00aed60 0x7faab0162530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.147+0000 7faaaf7fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faab00ce320 0x7faab00b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.147+0000 7faaaf7fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faab00ce320 0x7faab00b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:53288/0 (socket says 172.21.15.83:53288) 2024-04-17T10:46:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.147+0000 7faaae7fc640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7faab006ac50 0x7faab00cdc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:46110/0 (socket says 172.21.15.83:46110) 2024-04-17T10:46:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.147+0000 7faaaf7fe640 1 -- 172.21.15.83:0/1889597201 learned_addr learned my addr 172.21.15.83:0/1889597201 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.147+0000 7faaaf7fe640 1 -- 172.21.15.83:0/1889597201 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faab00aed60 msgr2=0x7faab0162530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:26.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.147+0000 7faaaf7fe640 1 --2- 172.21.15.83:0/1889597201 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faab00aed60 0x7faab0162530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:26.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.147+0000 7faaaf7fe640 1 -- 172.21.15.83:0/1889597201 >> v1:172.21.15.167:6789/0 conn(0x7faab006ac50 legacy=0x7faab00cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:26.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.147+0000 7faaaf7fe640 1 -- 172.21.15.83:0/1889597201 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab0163460 con 0x7faab00ce320 2024-04-17T10:46:26.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.147+0000 7faaaeffd640 1 --2- 172.21.15.83:0/1889597201 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faab00aed60 0x7faab0162530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:46:26.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.151+0000 7faaaf7fe640 1 --2- 172.21.15.83:0/1889597201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faab00ce320 0x7faab00b3900 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7faaa0009870 tx=0x7faaa001c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a233584529104bc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:26.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.151+0000 7faaadffb640 1 -- 172.21.15.83:0/1889597201 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa00a7070 con 0x7faab00ce320 2024-04-17T10:46:26.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.151+0000 7faaadffb640 1 -- 172.21.15.83:0/1889597201 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa00b0070 con 0x7faab00ce320 2024-04-17T10:46:26.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.151+0000 7faaadffb640 1 -- 172.21.15.83:0/1889597201 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa00ab890 con 0x7faab00ce320 2024-04-17T10:46:26.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.151+0000 7faab59a5640 1 -- 172.21.15.83:0/1889597201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faab00ce320 msgr2=0x7faab00b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:26.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.151+0000 7faab59a5640 1 --2- 172.21.15.83:0/1889597201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faab00ce320 0x7faab00b3900 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7faaa0009870 tx=0x7faaa001c560 comp rx=0 tx=0).stop 2024-04-17T10:46:26.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.151+0000 7faab59a5640 1 -- 172.21.15.83:0/1889597201 shutdown_connections 2024-04-17T10:46:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.151+0000 7faab59a5640 1 --2- 172.21.15.83:0/1889597201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faab00ce320 0x7faab00b3900 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.151+0000 7faab59a5640 1 --2- 172.21.15.83:0/1889597201 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faab00aed60 0x7faab0162530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.151+0000 7faab59a5640 1 -- 172.21.15.83:0/1889597201 >> 172.21.15.83:0/1889597201 conn(0x7faab00cc050 msgr2=0x7faab00cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.151+0000 7faab59a5640 1 -- 172.21.15.83:0/1889597201 shutdown_connections 2024-04-17T10:46:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.151+0000 7faab59a5640 1 -- 172.21.15.83:0/1889597201 wait complete. 2024-04-17T10:46:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.155+0000 7faab59a5640 1 Processor -- start 2024-04-17T10:46:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.155+0000 7faab59a5640 1 -- start start 2024-04-17T10:46:26.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.155+0000 7faab59a5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faab006ac50 0x7faab0165090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:26.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.155+0000 7faab59a5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faab00aed60 0x7faab0165710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:26.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.155+0000 7faab59a5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faab00ce320 0x7faab0165c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:26.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.155+0000 7faab59a5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab016ec90 con 0x7faab006ac50 2024-04-17T10:46:26.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.155+0000 7faab59a5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab016ee10 con 0x7faab00ce320 2024-04-17T10:46:26.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.155+0000 7faab59a5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab016ef90 con 0x7faab00aed60 2024-04-17T10:46:26.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.155+0000 7faaaeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faab006ac50 0x7faab0165090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:26.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.155+0000 7faaaeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faab006ac50 0x7faab0165090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52578/0 (socket says 172.21.15.83:52578) 2024-04-17T10:46:26.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.155+0000 7faaaeffd640 1 -- 172.21.15.83:0/3732536680 learned_addr learned my addr 172.21.15.83:0/3732536680 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:26.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.155+0000 7faaae7fc640 1 --2- 172.21.15.83:0/3732536680 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faab00aed60 0x7faab0165710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:26.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.155+0000 7faaaf7fe640 1 --2- 172.21.15.83:0/3732536680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faab00ce320 0x7faab0165c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:26.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.155+0000 7faaaeffd640 1 -- 172.21.15.83:0/3732536680 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faab00aed60 msgr2=0x7faab0165710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:26.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.155+0000 7faaaeffd640 1 --2- 172.21.15.83:0/3732536680 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faab00aed60 0x7faab0165710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:26.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.155+0000 7faaaeffd640 1 -- 172.21.15.83:0/3732536680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faab00ce320 msgr2=0x7faab0165c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:26.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.155+0000 7faaaeffd640 1 --2- 172.21.15.83:0/3732536680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faab00ce320 0x7faab0165c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:26.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.155+0000 7faaaeffd640 1 -- 172.21.15.83:0/3732536680 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab0164720 con 0x7faab006ac50 2024-04-17T10:46:26.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.155+0000 7faaaeffd640 1 --2- 172.21.15.83:0/3732536680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faab006ac50 0x7faab0165090 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7faa9800efc0 tx=0x7faa9800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:26.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.155+0000 7faaa7fff640 1 -- 172.21.15.83:0/3732536680 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa98018020 con 0x7faab006ac50 2024-04-17T10:46:26.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.155+0000 7faaa7fff640 1 -- 172.21.15.83:0/3732536680 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa98014680 con 0x7faab006ac50 2024-04-17T10:46:26.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.155+0000 7faab59a5640 1 -- 172.21.15.83:0/3732536680 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faab0164a00 con 0x7faab006ac50 2024-04-17T10:46:26.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.155+0000 7faaa7fff640 1 -- 172.21.15.83:0/3732536680 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa98014a50 con 0x7faab006ac50 2024-04-17T10:46:26.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.155+0000 7faab59a5640 1 -- 172.21.15.83:0/3732536680 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faab0158dc0 con 0x7faab006ac50 2024-04-17T10:46:26.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.159+0000 7faab59a5640 1 -- 172.21.15.83:0/3732536680 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faab0075000 con 0x7faab006ac50 2024-04-17T10:46:26.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.159+0000 7faaa7fff640 1 -- 172.21.15.83:0/3732536680 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faa9801b020 con 0x7faab006ac50 2024-04-17T10:46:26.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.159+0000 7faaa7fff640 1 --2- 172.21.15.83:0/3732536680 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faa90066f60 0x7faa90069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:26.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.159+0000 7faaa7fff640 1 -- 172.21.15.83:0/3732536680 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5853+0+0 (secure 0 0 0) 0x7faa98085ea0 con 0x7faab006ac50 2024-04-17T10:46:26.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.159+0000 7faaae7fc640 1 --2- 172.21.15.83:0/3732536680 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faa90066f60 0x7faa90069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:26.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.159+0000 7faaae7fc640 1 --2- 172.21.15.83:0/3732536680 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faa90066f60 0x7faa90069420 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7faab01663e0 tx=0x7faaa8007cd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:26.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.163+0000 7faaa7fff640 1 -- 172.21.15.83:0/3732536680 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faa98022030 con 0x7faab006ac50 2024-04-17T10:46:26.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.323+0000 7faab59a5640 1 -- 172.21.15.83:0/3732536680 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7faab006c6e0 con 0x7faab006ac50 2024-04-17T10:46:26.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.991+0000 7faaa7fff640 1 -- 172.21.15.83:0/3732536680 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v80) v1 ==== 181+0+0 (secure 0 0 0) 0x7faa98010040 con 0x7faab006ac50 2024-04-17T10:46:26.988 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2024-04-17T10:46:26.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.995+0000 7faab59a5640 1 -- 172.21.15.83:0/3732536680 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faa90066f60 msgr2=0x7faa90069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:26.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.995+0000 7faab59a5640 1 --2- 172.21.15.83:0/3732536680 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faa90066f60 0x7faa90069420 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7faab01663e0 tx=0x7faaa8007cd0 comp rx=0 tx=0).stop 2024-04-17T10:46:26.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.995+0000 7faab59a5640 1 -- 172.21.15.83:0/3732536680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faab006ac50 msgr2=0x7faab0165090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:26.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.995+0000 7faab59a5640 1 --2- 172.21.15.83:0/3732536680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faab006ac50 0x7faab0165090 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7faa9800efc0 tx=0x7faa9800c490 comp rx=0 tx=0).stop 2024-04-17T10:46:26.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.999+0000 7faab59a5640 1 -- 172.21.15.83:0/3732536680 shutdown_connections 2024-04-17T10:46:26.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.999+0000 7faab59a5640 1 --2- 172.21.15.83:0/3732536680 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faa90066f60 0x7faa90069420 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:26.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.999+0000 7faab59a5640 1 --2- 172.21.15.83:0/3732536680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faab00ce320 0x7faab0165c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:26.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.999+0000 7faab59a5640 1 --2- 172.21.15.83:0/3732536680 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faab00aed60 0x7faab0165710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:26.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.999+0000 7faab59a5640 1 --2- 172.21.15.83:0/3732536680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faab006ac50 0x7faab0165090 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:26.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.999+0000 7faab59a5640 1 -- 172.21.15.83:0/3732536680 >> 172.21.15.83:0/3732536680 conn(0x7faab00cc050 msgr2=0x7faab015c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:26.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.999+0000 7faab59a5640 1 -- 172.21.15.83:0/3732536680 shutdown_connections 2024-04-17T10:46:26.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:26.999+0000 7faab59a5640 1 -- 172.21.15.83:0/3732536680 wait complete. 2024-04-17T10:46:27.022 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-17T10:46:27.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.151+0000 7ff90dd04640 1 Processor -- start 2024-04-17T10:46:27.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.151+0000 7ff90dd04640 1 -- start start 2024-04-17T10:46:27.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.151+0000 7ff90dd04640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff90806b2f0 0x7ff908168ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:27.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.151+0000 7ff90dd04640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff908071dd0 con 0x7ff90806d580 2024-04-17T10:46:27.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.151+0000 7ff90dd04640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9080720b0 con 0x7ff908147340 2024-04-17T10:46:27.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.151+0000 7ff90dd04640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9080b3c10 con 0x7ff90806b2f0 2024-04-17T10:46:27.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.151+0000 7ff8fffff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff90806d580 0x7ff90806abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50652/0 (socket says 172.21.15.83:50652) 2024-04-17T10:46:27.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.151+0000 7ff8fffff640 1 -- 172.21.15.83:0/200358661 learned_addr learned my addr 172.21.15.83:0/200358661 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:46:27.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.151+0000 7ff907fff640 1 --2- 172.21.15.83:0/200358661 >> v2:172.21.15.167:3300/0 conn(0x7ff90806b2f0 0x7ff908168ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:27.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.151+0000 7ff906ffd640 1 -- 172.21.15.83:0/200358661 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 334577559 0 0) 0x7ff908071dd0 con 0x7ff90806d580 2024-04-17T10:46:27.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.151+0000 7ff906ffd640 1 -- 172.21.15.83:0/200358661 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8f4003530 con 0x7ff90806d580 2024-04-17T10:46:27.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.151+0000 7ff906ffd640 1 -- 172.21.15.83:0/200358661 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4197792582 0 0) 0x7ff8f4003530 con 0x7ff90806d580 2024-04-17T10:46:27.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.151+0000 7ff906ffd640 1 -- 172.21.15.83:0/200358661 >> v2:172.21.15.167:3300/0 conn(0x7ff90806b2f0 msgr2=0x7ff908168ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:27.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.151+0000 7ff906ffd640 1 --2- 172.21.15.83:0/200358661 >> v2:172.21.15.167:3300/0 conn(0x7ff90806b2f0 0x7ff908168ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:27.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.151+0000 7ff906ffd640 1 -- 172.21.15.83:0/200358661 >> v1:172.21.15.167:6789/0 conn(0x7ff908147340 legacy=0x7ff908071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:27.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.151+0000 7ff906ffd640 1 -- 172.21.15.83:0/200358661 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff908072390 con 0x7ff90806d580 2024-04-17T10:46:27.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.151+0000 7ff906ffd640 1 -- 172.21.15.83:0/200358661 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7ff8f0001680 con 0x7ff90806d580 2024-04-17T10:46:27.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.151+0000 7ff906ffd640 1 -- 172.21.15.83:0/200358661 >> v1:172.21.15.83:6789/0 conn(0x7ff90806d580 legacy=0x7ff90806abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:27.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.151+0000 7ff906ffd640 1 --2- 172.21.15.83:0/200358661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8f4003a10 0x7ff8f4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:27.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.151+0000 7ff906ffd640 1 --2- 172.21.15.83:0/200358661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8f4004810 0x7ff8f4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:27.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.151+0000 7ff906ffd640 1 --2- 172.21.15.83:0/200358661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8f40051d0 0x7ff8f400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:27.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.151+0000 7ff906ffd640 1 -- 172.21.15.83:0/200358661 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9080b3c10 con 0x7ff8f4003a10 2024-04-17T10:46:27.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.151+0000 7ff906ffd640 1 -- 172.21.15.83:0/200358661 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f400c0d0 con 0x7ff8f40051d0 2024-04-17T10:46:27.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.151+0000 7ff906ffd640 1 -- 172.21.15.83:0/200358661 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f400c2d0 con 0x7ff8f4004810 2024-04-17T10:46:27.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.151+0000 7ff9077fe640 1 --2- 172.21.15.83:0/200358661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8f4003a10 0x7ff8f4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:27.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.151+0000 7ff8fffff640 1 --2- 172.21.15.83:0/200358661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8f4004810 0x7ff8f4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:27.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.151+0000 7ff9077fe640 1 -- 172.21.15.83:0/200358661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8f4004810 msgr2=0x7ff8f4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:27.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.151+0000 7ff9077fe640 1 --2- 172.21.15.83:0/200358661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8f4004810 0x7ff8f4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:27.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.151+0000 7ff9077fe640 1 -- 172.21.15.83:0/200358661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8f40051d0 msgr2=0x7ff8f400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:27.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.151+0000 7ff9077fe640 1 --2- 172.21.15.83:0/200358661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8f40051d0 0x7ff8f400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:27.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.151+0000 7ff907fff640 1 -- 172.21.15.83:0/200358661 reap_dead start 2024-04-17T10:46:27.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.151+0000 7ff9077fe640 1 -- 172.21.15.83:0/200358661 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f4003530 con 0x7ff8f4003a10 2024-04-17T10:46:27.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.155+0000 7ff9077fe640 1 --2- 172.21.15.83:0/200358661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8f4003a10 0x7ff8f4003e30 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8001780 tx=0x7ff8f801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ff6a51ad3d6f78 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:27.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.155+0000 7ff906ffd640 1 -- 172.21.15.83:0/200358661 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f80013a0 con 0x7ff8f4003a10 2024-04-17T10:46:27.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.155+0000 7ff906ffd640 1 -- 172.21.15.83:0/200358661 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f80052f0 con 0x7ff8f4003a10 2024-04-17T10:46:27.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.155+0000 7ff906ffd640 1 -- 172.21.15.83:0/200358661 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f8003840 con 0x7ff8f4003a10 2024-04-17T10:46:27.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.155+0000 7ff90dd04640 1 -- 172.21.15.83:0/200358661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8f4003a10 msgr2=0x7ff8f4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:27.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.155+0000 7ff90dd04640 1 --2- 172.21.15.83:0/200358661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8f4003a10 0x7ff8f4003e30 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8001780 tx=0x7ff8f801c3f0 comp rx=0 tx=0).stop 2024-04-17T10:46:27.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.155+0000 7ff90dd04640 1 -- 172.21.15.83:0/200358661 shutdown_connections 2024-04-17T10:46:27.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.155+0000 7ff90dd04640 1 --2- 172.21.15.83:0/200358661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8f4003a10 0x7ff8f4003e30 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:27.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.155+0000 7ff90dd04640 1 -- 172.21.15.83:0/200358661 >> 172.21.15.83:0/200358661 conn(0x7ff9080b2410 msgr2=0x7ff9080b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:27.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.155+0000 7ff90dd04640 1 -- 172.21.15.83:0/200358661 shutdown_connections 2024-04-17T10:46:27.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.155+0000 7ff90dd04640 1 -- 172.21.15.83:0/200358661 wait complete. 2024-04-17T10:46:27.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.155+0000 7ff90dd04640 1 Processor -- start 2024-04-17T10:46:27.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.155+0000 7ff90dd04640 1 -- start start 2024-04-17T10:46:27.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.159+0000 7ff90dd04640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8f4003a10 0x7ff9081e9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:27.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.159+0000 7ff90dd04640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff90815ce30 0x7ff90815d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:27.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.159+0000 7ff90dd04640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9081605d0 0x7ff908160a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:27.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.159+0000 7ff90dd04640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff908071580 con 0x7ff90815ce30 2024-04-17T10:46:27.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.159+0000 7ff90dd04640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff908147930 con 0x7ff8f4003a10 2024-04-17T10:46:27.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.159+0000 7ff90dd04640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff908071700 con 0x7ff9081605d0 2024-04-17T10:46:27.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.159+0000 7ff8fffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff90815ce30 0x7ff90815d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:27.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.159+0000 7ff8fffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff90815ce30 0x7ff90815d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60920/0 (socket says 172.21.15.83:60920) 2024-04-17T10:46:27.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.159+0000 7ff8fffff640 1 -- 172.21.15.83:0/1812186551 learned_addr learned my addr 172.21.15.83:0/1812186551 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:27.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.159+0000 7ff907fff640 1 --2- 172.21.15.83:0/1812186551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9081605d0 0x7ff908160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:27.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.159+0000 7ff9077fe640 1 --2- 172.21.15.83:0/1812186551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8f4003a10 0x7ff9081e9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:27.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.159+0000 7ff8fffff640 1 -- 172.21.15.83:0/1812186551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9081605d0 msgr2=0x7ff908160a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:27.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.159+0000 7ff8fffff640 1 --2- 172.21.15.83:0/1812186551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9081605d0 0x7ff908160a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:27.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.159+0000 7ff8fffff640 1 -- 172.21.15.83:0/1812186551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8f4003a10 msgr2=0x7ff9081e9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:27.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.159+0000 7ff8fffff640 1 --2- 172.21.15.83:0/1812186551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8f4003a10 0x7ff9081e9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:27.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.159+0000 7ff8fffff640 1 -- 172.21.15.83:0/1812186551 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff90815d7f0 con 0x7ff90815ce30 2024-04-17T10:46:27.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.159+0000 7ff9077fe640 1 --2- 172.21.15.83:0/1812186551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8f4003a10 0x7ff9081e9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:27.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.159+0000 7ff8fffff640 1 --2- 172.21.15.83:0/1812186551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff90815ce30 0x7ff90815d2b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0001060 tx=0x7ff8f000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:27.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.159+0000 7ff904ff9640 1 -- 172.21.15.83:0/1812186551 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f000c6c0 con 0x7ff90815ce30 2024-04-17T10:46:27.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.159+0000 7ff904ff9640 1 -- 172.21.15.83:0/1812186551 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f0018070 con 0x7ff90815ce30 2024-04-17T10:46:27.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.159+0000 7ff90dd04640 1 -- 172.21.15.83:0/1812186551 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff908160fc0 con 0x7ff90815ce30 2024-04-17T10:46:27.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.159+0000 7ff904ff9640 1 -- 172.21.15.83:0/1812186551 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f000cac0 con 0x7ff90815ce30 2024-04-17T10:46:27.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.159+0000 7ff90dd04640 1 -- 172.21.15.83:0/1812186551 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff908141c60 con 0x7ff90815ce30 2024-04-17T10:46:27.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.159+0000 7ff90dd04640 1 -- 172.21.15.83:0/1812186551 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff908075000 con 0x7ff90815ce30 2024-04-17T10:46:27.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.159+0000 7ff904ff9640 1 -- 172.21.15.83:0/1812186551 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff8f001d460 con 0x7ff90815ce30 2024-04-17T10:46:27.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.159+0000 7ff904ff9640 1 --2- 172.21.15.83:0/1812186551 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff8e8066f60 0x7ff8e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:27.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.159+0000 7ff904ff9640 1 -- 172.21.15.83:0/1812186551 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5415+0+0 (secure 0 0 0) 0x7ff8f0084f90 con 0x7ff90815ce30 2024-04-17T10:46:27.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.159+0000 7ff9077fe640 1 --2- 172.21.15.83:0/1812186551 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff8e8066f60 0x7ff8e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:27.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.159+0000 7ff9077fe640 1 --2- 172.21.15.83:0/1812186551 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff8e8066f60 0x7ff8e8069420 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8001750 tx=0x7ff8f80011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:27.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.163+0000 7ff904ff9640 1 -- 172.21.15.83:0/1812186551 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff8f00560c0 con 0x7ff90815ce30 2024-04-17T10:46:27.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:27.323+0000 7ff90dd04640 1 -- 172.21.15.83:0/1812186551 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff9080adea0 con 0x7ff90815ce30 2024-04-17T10:46:28.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.131+0000 7ff904ff9640 1 -- 172.21.15.83:0/1812186551 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v81) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff8f00559c0 con 0x7ff90815ce30 2024-04-17T10:46:28.127 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2024-04-17T10:46:28.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.135+0000 7ff90dd04640 1 -- 172.21.15.83:0/1812186551 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff8e8066f60 msgr2=0x7ff8e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:28.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.135+0000 7ff90dd04640 1 --2- 172.21.15.83:0/1812186551 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff8e8066f60 0x7ff8e8069420 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8001750 tx=0x7ff8f80011a0 comp rx=0 tx=0).stop 2024-04-17T10:46:28.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.135+0000 7ff90dd04640 1 -- 172.21.15.83:0/1812186551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff90815ce30 msgr2=0x7ff90815d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.135+0000 7ff90dd04640 1 --2- 172.21.15.83:0/1812186551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff90815ce30 0x7ff90815d2b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0001060 tx=0x7ff8f000eb90 comp rx=0 tx=0).stop 2024-04-17T10:46:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.135+0000 7ff90dd04640 1 -- 172.21.15.83:0/1812186551 shutdown_connections 2024-04-17T10:46:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.135+0000 7ff90dd04640 1 --2- 172.21.15.83:0/1812186551 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff8e8066f60 0x7ff8e8069420 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.135+0000 7ff90dd04640 1 --2- 172.21.15.83:0/1812186551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9081605d0 0x7ff908160a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.135+0000 7ff90dd04640 1 --2- 172.21.15.83:0/1812186551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff90815ce30 0x7ff90815d2b0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.135+0000 7ff90dd04640 1 --2- 172.21.15.83:0/1812186551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8f4003a10 0x7ff9081e9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.135+0000 7ff90dd04640 1 -- 172.21.15.83:0/1812186551 >> 172.21.15.83:0/1812186551 conn(0x7ff9080b2410 msgr2=0x7ff9080b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.135+0000 7ff90dd04640 1 -- 172.21.15.83:0/1812186551 shutdown_connections 2024-04-17T10:46:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.135+0000 7ff90dd04640 1 -- 172.21.15.83:0/1812186551 wait complete. 2024-04-17T10:46:28.158 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-17T10:46:28.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.287+0000 7fd8d2b24640 1 Processor -- start 2024-04-17T10:46:28.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.287+0000 7fd8d2b24640 1 -- start start 2024-04-17T10:46:28.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.291+0000 7fd8d2b24640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd8cc154de0 0x7fd8cc06ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:28.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.291+0000 7fd8d2b24640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8cc15bd70 con 0x7fd8cc06b350 2024-04-17T10:46:28.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.291+0000 7fd8d2b24640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8cc15bf60 con 0x7fd8cc0b4210 2024-04-17T10:46:28.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.291+0000 7fd8d2b24640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8cc0b4870 con 0x7fd8cc154de0 2024-04-17T10:46:28.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.291+0000 7fd8d08c0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd8cc154de0 0x7fd8cc06ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:28.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.291+0000 7fd8c3fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd8cc06b350 0x7fd8cc0b3b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50656/0 (socket says 172.21.15.83:50656) 2024-04-17T10:46:28.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.291+0000 7fd8c3fff640 1 -- 172.21.15.83:0/20464796 learned_addr learned my addr 172.21.15.83:0/20464796 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:46:28.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.291+0000 7fd8d08c0640 1 -- 172.21.15.83:0/20464796 >> v1:172.21.15.167:6789/0 conn(0x7fd8cc0b4210 legacy=0x7fd8cc06cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:28.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.291+0000 7fd8d08c0640 1 -- 172.21.15.83:0/20464796 >> v1:172.21.15.83:6789/0 conn(0x7fd8cc06b350 legacy=0x7fd8cc0b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:28.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.291+0000 7fd8d08c0640 1 -- 172.21.15.83:0/20464796 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8cc15afb0 con 0x7fd8cc154de0 2024-04-17T10:46:28.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.291+0000 7fd8c37fe640 1 -- 172.21.15.83:0/20464796 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 204788207 0 0) 0x7fd8cc15bd70 con 0x7fd8cc06b350 2024-04-17T10:46:28.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.291+0000 7fd8d08c0640 1 --2- 172.21.15.83:0/20464796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8cc154de0 0x7fd8cc06ae10 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80097c0 tx=0x7fd8c801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d0a56758f3d7aae server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:28.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.291+0000 7fd8c37fe640 1 -- 172.21.15.83:0/20464796 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c80b0070 con 0x7fd8cc154de0 2024-04-17T10:46:28.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.291+0000 7fd8c37fe640 1 -- 172.21.15.83:0/20464796 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c80ab600 con 0x7fd8cc154de0 2024-04-17T10:46:28.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.291+0000 7fd8c37fe640 1 -- 172.21.15.83:0/20464796 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c80ab9d0 con 0x7fd8cc154de0 2024-04-17T10:46:28.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.295+0000 7fd8d2b24640 1 -- 172.21.15.83:0/20464796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8cc154de0 msgr2=0x7fd8cc06ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:28.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.295+0000 7fd8d2b24640 1 --2- 172.21.15.83:0/20464796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8cc154de0 0x7fd8cc06ae10 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80097c0 tx=0x7fd8c801c460 comp rx=0 tx=0).stop 2024-04-17T10:46:28.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.295+0000 7fd8d2b24640 1 -- 172.21.15.83:0/20464796 shutdown_connections 2024-04-17T10:46:28.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.295+0000 7fd8d2b24640 1 --2- 172.21.15.83:0/20464796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8cc154de0 0x7fd8cc06ae10 secure :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80097c0 tx=0x7fd8c801c460 comp rx=0 tx=0).stop 2024-04-17T10:46:28.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.295+0000 7fd8d2b24640 1 -- 172.21.15.83:0/20464796 >> 172.21.15.83:0/20464796 conn(0x7fd8cc0b15b0 msgr2=0x7fd8cc0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:28.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.295+0000 7fd8d2b24640 1 -- 172.21.15.83:0/20464796 shutdown_connections 2024-04-17T10:46:28.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.295+0000 7fd8d2b24640 1 -- 172.21.15.83:0/20464796 wait complete. 2024-04-17T10:46:28.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.295+0000 7fd8d2b24640 1 Processor -- start 2024-04-17T10:46:28.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.295+0000 7fd8d2b24640 1 -- start start 2024-04-17T10:46:28.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.295+0000 7fd8d2b24640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8cc06b350 0x7fd8cc0aaa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:28.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.295+0000 7fd8d2b24640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8cc0b4210 0x7fd8cc156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:28.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.295+0000 7fd8d2b24640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8cc159a00 0x7fd8cc153700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:28.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.295+0000 7fd8d2b24640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8cc1551c0 con 0x7fd8cc0b4210 2024-04-17T10:46:28.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.295+0000 7fd8d2b24640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8cc06b7c0 con 0x7fd8cc159a00 2024-04-17T10:46:28.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.295+0000 7fd8d2b24640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8cc070470 con 0x7fd8cc06b350 2024-04-17T10:46:28.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.295+0000 7fd8c3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8cc0b4210 0x7fd8cc156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:28.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.295+0000 7fd8c3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8cc0b4210 0x7fd8cc156940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60940/0 (socket says 172.21.15.83:60940) 2024-04-17T10:46:28.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.295+0000 7fd8c3fff640 1 -- 172.21.15.83:0/1639795188 learned_addr learned my addr 172.21.15.83:0/1639795188 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:28.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.295+0000 7fd8d08c0640 1 --2- 172.21.15.83:0/1639795188 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8cc06b350 0x7fd8cc0aaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:28.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.295+0000 7fd8d10c1640 1 --2- 172.21.15.83:0/1639795188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8cc159a00 0x7fd8cc153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:28.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.295+0000 7fd8c3fff640 1 -- 172.21.15.83:0/1639795188 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8cc06b350 msgr2=0x7fd8cc0aaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:28.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.295+0000 7fd8c3fff640 1 --2- 172.21.15.83:0/1639795188 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8cc06b350 0x7fd8cc0aaa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:28.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.295+0000 7fd8c3fff640 1 -- 172.21.15.83:0/1639795188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8cc159a00 msgr2=0x7fd8cc153700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:28.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.295+0000 7fd8c3fff640 1 --2- 172.21.15.83:0/1639795188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8cc159a00 0x7fd8cc153700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:28.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.295+0000 7fd8c3fff640 1 -- 172.21.15.83:0/1639795188 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8cc157190 con 0x7fd8cc0b4210 2024-04-17T10:46:28.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.295+0000 7fd8c3fff640 1 --2- 172.21.15.83:0/1639795188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8cc0b4210 0x7fd8cc156940 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00c4a0 tx=0x7fd8bc00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:28.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.295+0000 7fd8c17fa640 1 -- 172.21.15.83:0/1639795188 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc018070 con 0x7fd8cc0b4210 2024-04-17T10:46:28.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.295+0000 7fd8c17fa640 1 -- 172.21.15.83:0/1639795188 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8bc0074e0 con 0x7fd8cc0b4210 2024-04-17T10:46:28.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.295+0000 7fd8c17fa640 1 -- 172.21.15.83:0/1639795188 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc004a30 con 0x7fd8cc0b4210 2024-04-17T10:46:28.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.295+0000 7fd8d2b24640 1 -- 172.21.15.83:0/1639795188 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8cc157470 con 0x7fd8cc0b4210 2024-04-17T10:46:28.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.295+0000 7fd8d2b24640 1 -- 172.21.15.83:0/1639795188 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8cc157a30 con 0x7fd8cc0b4210 2024-04-17T10:46:28.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.299+0000 7fd8d2b24640 1 -- 172.21.15.83:0/1639795188 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8cc157f20 con 0x7fd8cc0b4210 2024-04-17T10:46:28.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.299+0000 7fd8c17fa640 1 -- 172.21.15.83:0/1639795188 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd8bc01b070 con 0x7fd8cc0b4210 2024-04-17T10:46:28.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.299+0000 7fd8c17fa640 1 --2- 172.21.15.83:0/1639795188 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd8b8066f60 0x7fd8b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:28.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.299+0000 7fd8d08c0640 1 --2- 172.21.15.83:0/1639795188 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd8b8066f60 0x7fd8b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:28.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.299+0000 7fd8c17fa640 1 -- 172.21.15.83:0/1639795188 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd8bc083fb0 con 0x7fd8cc0b4210 2024-04-17T10:46:28.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.299+0000 7fd8d08c0640 1 --2- 172.21.15.83:0/1639795188 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd8b8066f60 0x7fd8b8069420 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80ad000 tx=0x7fd8c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:28.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.303+0000 7fd8c17fa640 1 -- 172.21.15.83:0/1639795188 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd8bc055350 con 0x7fd8cc0b4210 2024-04-17T10:46:28.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.475+0000 7fd8d2b24640 1 -- 172.21.15.83:0/1639795188 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd8cc074f90 con 0x7fd8cc0b4210 2024-04-17T10:46:28.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.479+0000 7fd8c17fa640 1 -- 172.21.15.83:0/1639795188 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd8bc054c50 con 0x7fd8cc0b4210 2024-04-17T10:46:28.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.483+0000 7fd8d2b24640 1 -- 172.21.15.83:0/1639795188 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd8b8066f60 msgr2=0x7fd8b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:28.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.483+0000 7fd8d2b24640 1 --2- 172.21.15.83:0/1639795188 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd8b8066f60 0x7fd8b8069420 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80ad000 tx=0x7fd8c8008040 comp rx=0 tx=0).stop 2024-04-17T10:46:28.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.483+0000 7fd8d2b24640 1 -- 172.21.15.83:0/1639795188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8cc0b4210 msgr2=0x7fd8cc156940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:28.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.483+0000 7fd8d2b24640 1 --2- 172.21.15.83:0/1639795188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8cc0b4210 0x7fd8cc156940 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00c4a0 tx=0x7fd8bc00c960 comp rx=0 tx=0).stop 2024-04-17T10:46:28.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.483+0000 7fd8d2b24640 1 -- 172.21.15.83:0/1639795188 shutdown_connections 2024-04-17T10:46:28.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.483+0000 7fd8d2b24640 1 --2- 172.21.15.83:0/1639795188 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd8b8066f60 0x7fd8b8069420 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:28.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.483+0000 7fd8d2b24640 1 --2- 172.21.15.83:0/1639795188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8cc159a00 0x7fd8cc153700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:28.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.483+0000 7fd8d2b24640 1 --2- 172.21.15.83:0/1639795188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8cc0b4210 0x7fd8cc156940 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:28.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.483+0000 7fd8d2b24640 1 --2- 172.21.15.83:0/1639795188 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8cc06b350 0x7fd8cc0aaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:28.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.483+0000 7fd8d2b24640 1 -- 172.21.15.83:0/1639795188 >> 172.21.15.83:0/1639795188 conn(0x7fd8cc0b15b0 msgr2=0x7fd8cc0b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:28.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.483+0000 7fd8d2b24640 1 -- 172.21.15.83:0/1639795188 shutdown_connections 2024-04-17T10:46:28.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.483+0000 7fd8d2b24640 1 -- 172.21.15.83:0/1639795188 wait complete. 2024-04-17T10:46:28.497 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub' 2024-04-17T10:46:28.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.615+0000 7f71aba4b640 1 Processor -- start 2024-04-17T10:46:28.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.615+0000 7f71aba4b640 1 -- start start 2024-04-17T10:46:28.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f71aba4b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71a40b2de0 0x7f71a415ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:28.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f71aba4b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f71a40d22d0 0x7f71a40ae520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:28.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f71aba4b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71a415f5b0 con 0x7f71a415f1d0 2024-04-17T10:46:28.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f71aba4b640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71a40b2620 con 0x7f71a40d22d0 2024-04-17T10:46:28.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f71aba4b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a40b2340 con 0x7f71a40b2de0 2024-04-17T10:46:28.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f71a97e7640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f71a415f1d0 0x7f71a415f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50672/0 (socket says 172.21.15.83:50672) 2024-04-17T10:46:28.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f71a97e7640 1 -- 172.21.15.83:0/545317405 learned_addr learned my addr 172.21.15.83:0/545317405 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:46:28.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f719bfff640 1 -- 172.21.15.83:0/545317405 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3534442151 0 0) 0x7f71a415f5b0 con 0x7f71a415f1d0 2024-04-17T10:46:28.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f719bfff640 1 -- 172.21.15.83:0/545317405 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7190003530 con 0x7f71a415f1d0 2024-04-17T10:46:28.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f71a9fe8640 1 --2- 172.21.15.83:0/545317405 >> v2:172.21.15.167:3300/0 conn(0x7f71a40d22d0 0x7f71a40ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:28.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f71a8fe6640 1 --2- 172.21.15.83:0/545317405 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71a40b2de0 0x7f71a415ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:28.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f719bfff640 1 -- 172.21.15.83:0/545317405 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3895809190 0 0) 0x7f7190003530 con 0x7f71a415f1d0 2024-04-17T10:46:28.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f719bfff640 1 -- 172.21.15.83:0/545317405 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71a40b2de0 msgr2=0x7f71a415ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:28.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f719bfff640 1 --2- 172.21.15.83:0/545317405 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71a40b2de0 0x7f71a415ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:28.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f719bfff640 1 -- 172.21.15.83:0/545317405 >> v2:172.21.15.167:3300/0 conn(0x7f71a40d22d0 msgr2=0x7f71a40ae520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:28.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f719bfff640 1 --2- 172.21.15.83:0/545317405 >> v2:172.21.15.167:3300/0 conn(0x7f71a40d22d0 0x7f71a40ae520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:28.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f719bfff640 1 -- 172.21.15.83:0/545317405 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a40b1760 con 0x7f71a415f1d0 2024-04-17T10:46:28.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f719bfff640 1 -- 172.21.15.83:0/545317405 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f71a0001a00 con 0x7f71a415f1d0 2024-04-17T10:46:28.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f71a8fe6640 1 --2- 172.21.15.83:0/545317405 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71a40b2de0 0x7f71a415ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:46:28.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f719bfff640 1 -- 172.21.15.83:0/545317405 >> v1:172.21.15.83:6789/0 conn(0x7f71a415f1d0 legacy=0x7f71a415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:28.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f719bfff640 1 --2- 172.21.15.83:0/545317405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71900039c0 0x7f7190003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:28.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f719bfff640 1 --2- 172.21.15.83:0/545317405 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7190004810 0x7f7190004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:28.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f719bfff640 1 --2- 172.21.15.83:0/545317405 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71900051d0 0x7f719000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:28.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f719bfff640 1 -- 172.21.15.83:0/545317405 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a40b2620 con 0x7f71900039c0 2024-04-17T10:46:28.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f719bfff640 1 -- 172.21.15.83:0/545317405 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a40b2340 con 0x7f7190004810 2024-04-17T10:46:28.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f719bfff640 1 -- 172.21.15.83:0/545317405 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7190005680 con 0x7f71900051d0 2024-04-17T10:46:28.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f71a97e7640 1 --2- 172.21.15.83:0/545317405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71900039c0 0x7f7190003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:28.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f71a97e7640 1 -- 172.21.15.83:0/545317405 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71900051d0 msgr2=0x7f719000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:28.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f71a9fe8640 1 --2- 172.21.15.83:0/545317405 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71900051d0 0x7f719000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:28.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f71a97e7640 1 --2- 172.21.15.83:0/545317405 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71900051d0 0x7f719000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:28.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f71a97e7640 1 -- 172.21.15.83:0/545317405 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7190004810 msgr2=0x7f7190004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:28.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f71a97e7640 1 --2- 172.21.15.83:0/545317405 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7190004810 0x7f7190004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:28.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.619+0000 7f71a97e7640 1 -- 172.21.15.83:0/545317405 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7190003530 con 0x7f71900039c0 2024-04-17T10:46:28.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.623+0000 7f71a97e7640 1 --2- 172.21.15.83:0/545317405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71900039c0 0x7f7190003de0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f71a000a870 tx=0x7f71a001d350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7801eed1adc6420c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:28.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.623+0000 7f719bfff640 1 -- 172.21.15.83:0/545317405 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a000a4c0 con 0x7f71900039c0 2024-04-17T10:46:28.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.623+0000 7f719bfff640 1 -- 172.21.15.83:0/545317405 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a00ad650 con 0x7f71900039c0 2024-04-17T10:46:28.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.623+0000 7f719bfff640 1 -- 172.21.15.83:0/545317405 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a00ada40 con 0x7f71900039c0 2024-04-17T10:46:28.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.623+0000 7f71aba4b640 1 -- 172.21.15.83:0/545317405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71900039c0 msgr2=0x7f7190003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:28.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.623+0000 7f71aba4b640 1 --2- 172.21.15.83:0/545317405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71900039c0 0x7f7190003de0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f71a000a870 tx=0x7f71a001d350 comp rx=0 tx=0).stop 2024-04-17T10:46:28.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.623+0000 7f71a9fe8640 1 -- 172.21.15.83:0/545317405 reap_dead start 2024-04-17T10:46:28.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.623+0000 7f71aba4b640 1 -- 172.21.15.83:0/545317405 shutdown_connections 2024-04-17T10:46:28.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.623+0000 7f71aba4b640 1 -- 172.21.15.83:0/545317405 >> 172.21.15.83:0/545317405 conn(0x7f71a40d17b0 msgr2=0x7f71a40d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:28.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.623+0000 7f71aba4b640 1 -- 172.21.15.83:0/545317405 shutdown_connections 2024-04-17T10:46:28.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.623+0000 7f71aba4b640 1 -- 172.21.15.83:0/545317405 wait complete. 2024-04-17T10:46:28.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.623+0000 7f71aba4b640 1 Processor -- start 2024-04-17T10:46:28.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.623+0000 7f71aba4b640 1 -- start start 2024-04-17T10:46:28.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.623+0000 7f71aba4b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71900039c0 0x7f71a40b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:28.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.623+0000 7f71aba4b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71a40c75d0 0x7f71a415a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:28.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.623+0000 7f71aba4b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71a40af330 0x7f71a40af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:28.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.623+0000 7f71aba4b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a40aedf0 con 0x7f71900039c0 2024-04-17T10:46:28.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.623+0000 7f71aba4b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a40aec70 con 0x7f71a40af330 2024-04-17T10:46:28.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.623+0000 7f71aba4b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a40aef70 con 0x7f71a40c75d0 2024-04-17T10:46:28.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.623+0000 7f71a97e7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71900039c0 0x7f71a40b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:28.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.623+0000 7f71a97e7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71900039c0 0x7f71a40b7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60960/0 (socket says 172.21.15.83:60960) 2024-04-17T10:46:28.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.623+0000 7f71a97e7640 1 -- 172.21.15.83:0/3657457848 learned_addr learned my addr 172.21.15.83:0/3657457848 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:28.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.623+0000 7f71a9fe8640 1 --2- 172.21.15.83:0/3657457848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71a40af330 0x7f71a40af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:28.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.623+0000 7f71a97e7640 1 -- 172.21.15.83:0/3657457848 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71a40c75d0 msgr2=0x7f71a415a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:28.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.623+0000 7f71a97e7640 1 --2- 172.21.15.83:0/3657457848 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71a40c75d0 0x7f71a415a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:28.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.623+0000 7f71a97e7640 1 -- 172.21.15.83:0/3657457848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71a40af330 msgr2=0x7f71a40af7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:28.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.623+0000 7f71a97e7640 1 --2- 172.21.15.83:0/3657457848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71a40af330 0x7f71a40af7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:28.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.623+0000 7f71a97e7640 1 -- 172.21.15.83:0/3657457848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a40c7ba0 con 0x7f71900039c0 2024-04-17T10:46:28.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.623+0000 7f71a97e7640 1 --2- 172.21.15.83:0/3657457848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71900039c0 0x7f71a40b7250 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f71a0000f90 tx=0x7f7190005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:28.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.627+0000 7f7199ffb640 1 -- 172.21.15.83:0/3657457848 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a00ad430 con 0x7f71900039c0 2024-04-17T10:46:28.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.627+0000 7f7199ffb640 1 -- 172.21.15.83:0/3657457848 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a0004b70 con 0x7f71900039c0 2024-04-17T10:46:28.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.627+0000 7f7199ffb640 1 -- 172.21.15.83:0/3657457848 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a00017c0 con 0x7f71900039c0 2024-04-17T10:46:28.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.627+0000 7f71aba4b640 1 -- 172.21.15.83:0/3657457848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a40c7e80 con 0x7f71900039c0 2024-04-17T10:46:28.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.627+0000 7f71aba4b640 1 -- 172.21.15.83:0/3657457848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a40c8db0 con 0x7f71900039c0 2024-04-17T10:46:28.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.627+0000 7f71aba4b640 1 -- 172.21.15.83:0/3657457848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71a4074f90 con 0x7f71900039c0 2024-04-17T10:46:28.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.627+0000 7f7199ffb640 1 -- 172.21.15.83:0/3657457848 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f71a00b5070 con 0x7f71900039c0 2024-04-17T10:46:28.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.627+0000 7f7199ffb640 1 --2- 172.21.15.83:0/3657457848 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7188066f90 0x7f7188069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:28.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.627+0000 7f71a8fe6640 1 --2- 172.21.15.83:0/3657457848 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7188066f90 0x7f7188069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:28.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.627+0000 7f7199ffb640 1 -- 172.21.15.83:0/3657457848 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f71a011f330 con 0x7f71900039c0 2024-04-17T10:46:28.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.627+0000 7f71a8fe6640 1 --2- 172.21.15.83:0/3657457848 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7188066f90 0x7f7188069450 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7194006fd0 tx=0x7f7194008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:28.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.631+0000 7f7199ffb640 1 -- 172.21.15.83:0/3657457848 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f71a00bd060 con 0x7f71900039c0 2024-04-17T10:46:28.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:28.851+0000 7f71aba4b640 1 -- 172.21.15.83:0/3657457848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]} v 0) v1 -- 0x7f71a40c5020 con 0x7f71900039c0 2024-04-17T10:46:29.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.139+0000 7f7199ffb640 1 -- 172.21.15.83:0/3657457848 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]}]=0 v360) v1 ==== 158+0+0 (secure 0 0 0) 0x7f71a0017090 con 0x7f71900039c0 2024-04-17T10:46:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.143+0000 7f71aba4b640 1 -- 172.21.15.83:0/3657457848 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7188066f90 msgr2=0x7f7188069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.143+0000 7f71aba4b640 1 --2- 172.21.15.83:0/3657457848 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7188066f90 0x7f7188069450 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7194006fd0 tx=0x7f7194008040 comp rx=0 tx=0).stop 2024-04-17T10:46:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.143+0000 7f71aba4b640 1 -- 172.21.15.83:0/3657457848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71900039c0 msgr2=0x7f71a40b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.143+0000 7f71aba4b640 1 --2- 172.21.15.83:0/3657457848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71900039c0 0x7f71a40b7250 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f71a0000f90 tx=0x7f7190005840 comp rx=0 tx=0).stop 2024-04-17T10:46:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.143+0000 7f71aba4b640 1 -- 172.21.15.83:0/3657457848 shutdown_connections 2024-04-17T10:46:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.143+0000 7f71aba4b640 1 --2- 172.21.15.83:0/3657457848 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7188066f90 0x7f7188069450 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.143+0000 7f71aba4b640 1 --2- 172.21.15.83:0/3657457848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71a40af330 0x7f71a40af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.143+0000 7f71aba4b640 1 --2- 172.21.15.83:0/3657457848 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71a40c75d0 0x7f71a415a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.143+0000 7f71aba4b640 1 --2- 172.21.15.83:0/3657457848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71900039c0 0x7f71a40b7250 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.143+0000 7f71aba4b640 1 -- 172.21.15.83:0/3657457848 >> 172.21.15.83:0/3657457848 conn(0x7f71a40d17b0 msgr2=0x7f71a40d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.143+0000 7f71aba4b640 1 -- 172.21.15.83:0/3657457848 shutdown_connections 2024-04-17T10:46:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.143+0000 7f71aba4b640 1 -- 172.21.15.83:0/3657457848 wait complete. 2024-04-17T10:46:29.164 INFO:tasks.cephfs_test_runner:test that scrub can iterate over ~mdsdir and evaluate strays ... ok 2024-04-17T10:46:29.164 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-17T10:46:29.164 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:46:29.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.295+0000 7f4926dab640 1 Processor -- start 2024-04-17T10:46:29.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.295+0000 7f4926dab640 1 -- start start 2024-04-17T10:46:29.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.295+0000 7f4926dab640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f49200b1ce0 0x7f492006dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:29.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.295+0000 7f4926dab640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f492006c1b0 0x7f4920164660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:29.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.295+0000 7f4926dab640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f492015b8c0 con 0x7f492015ae80 2024-04-17T10:46:29.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.295+0000 7f4926dab640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f492015bba0 con 0x7f49200b1ce0 2024-04-17T10:46:29.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.295+0000 7f4926dab640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492015be80 con 0x7f492006c1b0 2024-04-17T10:46:29.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.295+0000 7f4924b47640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f49200b1ce0 0x7f492006dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:29.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.295+0000 7f491ffff640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f492015ae80 0x7f49200b20c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:34676/0 (socket says 172.21.15.83:34676) 2024-04-17T10:46:29.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.295+0000 7f491ffff640 1 -- 172.21.15.83:0/1612693751 learned_addr learned my addr 172.21.15.83:0/1612693751 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:46:29.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.295+0000 7f4925348640 1 --2- 172.21.15.83:0/1612693751 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f492006c1b0 0x7f4920164660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:29.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.295+0000 7f4924b47640 1 -- 172.21.15.83:0/1612693751 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f492006c1b0 msgr2=0x7f4920164660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:29.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.295+0000 7f4924b47640 1 --2- 172.21.15.83:0/1612693751 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f492006c1b0 0x7f4920164660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:29.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.295+0000 7f4924b47640 1 -- 172.21.15.83:0/1612693751 >> v1:172.21.15.167:6789/0 conn(0x7f492015ae80 legacy=0x7f49200b20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:29.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.295+0000 7f4924b47640 1 -- 172.21.15.83:0/1612693751 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492015c160 con 0x7f49200b1ce0 2024-04-17T10:46:29.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.299+0000 7f4924b47640 1 --2- 172.21.15.83:0/1612693751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49200b1ce0 0x7f492006dc40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f49100097c0 tx=0x7f491001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df22d82254af96d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:29.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.299+0000 7f491f7fe640 1 -- 172.21.15.83:0/1612693751 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49100b2070 con 0x7f49200b1ce0 2024-04-17T10:46:29.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.299+0000 7f491f7fe640 1 -- 172.21.15.83:0/1612693751 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49100ad680 con 0x7f49200b1ce0 2024-04-17T10:46:29.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.299+0000 7f491f7fe640 1 -- 172.21.15.83:0/1612693751 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49100ada50 con 0x7f49200b1ce0 2024-04-17T10:46:29.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.299+0000 7f4926dab640 1 -- 172.21.15.83:0/1612693751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49200b1ce0 msgr2=0x7f492006dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:29.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.299+0000 7f4926dab640 1 --2- 172.21.15.83:0/1612693751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49200b1ce0 0x7f492006dc40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f49100097c0 tx=0x7f491001c470 comp rx=0 tx=0).stop 2024-04-17T10:46:29.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.299+0000 7f4926dab640 1 -- 172.21.15.83:0/1612693751 shutdown_connections 2024-04-17T10:46:29.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.299+0000 7f4926dab640 1 --2- 172.21.15.83:0/1612693751 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f492006c1b0 0x7f4920164660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:29.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.299+0000 7f4926dab640 1 --2- 172.21.15.83:0/1612693751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49200b1ce0 0x7f492006dc40 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:29.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.299+0000 7f4926dab640 1 -- 172.21.15.83:0/1612693751 >> 172.21.15.83:0/1612693751 conn(0x7f49200b4eb0 msgr2=0x7f49200b5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:29.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.299+0000 7f4926dab640 1 -- 172.21.15.83:0/1612693751 shutdown_connections 2024-04-17T10:46:29.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.299+0000 7f4926dab640 1 -- 172.21.15.83:0/1612693751 wait complete. 2024-04-17T10:46:29.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.303+0000 7f4926dab640 1 Processor -- start 2024-04-17T10:46:29.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.303+0000 7f4926dab640 1 -- start start 2024-04-17T10:46:29.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.303+0000 7f4926dab640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f492006c1b0 0x7f492009d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:29.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.303+0000 7f4926dab640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49200b1ce0 0x7f49201fb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:29.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.303+0000 7f4926dab640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f492015ae80 0x7f49200d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:29.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.303+0000 7f4926dab640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4920167750 con 0x7f492015ae80 2024-04-17T10:46:29.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.303+0000 7f4926dab640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49201675d0 con 0x7f49200b1ce0 2024-04-17T10:46:29.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.303+0000 7f4926dab640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49201678d0 con 0x7f492006c1b0 2024-04-17T10:46:29.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.303+0000 7f4925348640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f492015ae80 0x7f49200d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:29.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.303+0000 7f4925348640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f492015ae80 0x7f49200d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60974/0 (socket says 172.21.15.83:60974) 2024-04-17T10:46:29.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.303+0000 7f4925348640 1 -- 172.21.15.83:0/1750330270 learned_addr learned my addr 172.21.15.83:0/1750330270 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:29.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.303+0000 7f4924b47640 1 --2- 172.21.15.83:0/1750330270 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f492006c1b0 0x7f492009d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:29.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.303+0000 7f491ffff640 1 --2- 172.21.15.83:0/1750330270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49200b1ce0 0x7f49201fb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:29.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.303+0000 7f4925348640 1 -- 172.21.15.83:0/1750330270 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f492006c1b0 msgr2=0x7f492009d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:29.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.303+0000 7f4925348640 1 --2- 172.21.15.83:0/1750330270 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f492006c1b0 0x7f492009d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:29.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.303+0000 7f4925348640 1 -- 172.21.15.83:0/1750330270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49200b1ce0 msgr2=0x7f49201fb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:29.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.303+0000 7f4925348640 1 --2- 172.21.15.83:0/1750330270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49200b1ce0 0x7f49201fb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:29.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.303+0000 7f4925348640 1 -- 172.21.15.83:0/1750330270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49201594e0 con 0x7f492015ae80 2024-04-17T10:46:29.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.303+0000 7f4925348640 1 --2- 172.21.15.83:0/1750330270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f492015ae80 0x7f49200d00e0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f491800e930 tx=0x7f491800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:29.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.303+0000 7f491d7fa640 1 -- 172.21.15.83:0/1750330270 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4918017070 con 0x7f492015ae80 2024-04-17T10:46:29.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.303+0000 7f491d7fa640 1 -- 172.21.15.83:0/1750330270 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491800c9a0 con 0x7f492015ae80 2024-04-17T10:46:29.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.303+0000 7f4926dab640 1 -- 172.21.15.83:0/1750330270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49201597c0 con 0x7f492015ae80 2024-04-17T10:46:29.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.303+0000 7f491d7fa640 1 -- 172.21.15.83:0/1750330270 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491800cd70 con 0x7f492015ae80 2024-04-17T10:46:29.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.303+0000 7f4926dab640 1 -- 172.21.15.83:0/1750330270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4920159d80 con 0x7f492015ae80 2024-04-17T10:46:29.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.303+0000 7f4926dab640 1 -- 172.21.15.83:0/1750330270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4920074f90 con 0x7f492015ae80 2024-04-17T10:46:29.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.303+0000 7f491d7fa640 1 -- 172.21.15.83:0/1750330270 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f491801a070 con 0x7f492015ae80 2024-04-17T10:46:29.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.307+0000 7f491d7fa640 1 --2- 172.21.15.83:0/1750330270 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f490c066f60 0x7f490c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:29.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.307+0000 7f491d7fa640 1 -- 172.21.15.83:0/1750330270 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4918084d90 con 0x7f492015ae80 2024-04-17T10:46:29.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.307+0000 7f4924b47640 1 --2- 172.21.15.83:0/1750330270 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f490c066f60 0x7f490c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:29.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.307+0000 7f4924b47640 1 --2- 172.21.15.83:0/1750330270 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f490c066f60 0x7f490c069420 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f49100af000 tx=0x7f49100a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:29.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.311+0000 7f491d7fa640 1 -- 172.21.15.83:0/1750330270 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f49180560e0 con 0x7f492015ae80 2024-04-17T10:46:29.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.475+0000 7f4926dab640 1 -- 172.21.15.83:0/1750330270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49200b2cf0 con 0x7f492015ae80 2024-04-17T10:46:29.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.475+0000 7f491d7fa640 1 -- 172.21.15.83:0/1750330270 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+12951 (secure 0 0 0) 0x7f49180559e0 con 0x7f492015ae80 2024-04-17T10:46:29.473 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:46:29.473 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":81,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:46:28.130946+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.167:6833/2796648400":"2024-04-18T10:46:24.988757+0000","172.21.15.167:6832/2796648400":"2024-04-18T10:46:24.988757+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:46:29.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.479+0000 7f4926dab640 1 -- 172.21.15.83:0/1750330270 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f490c066f60 msgr2=0x7f490c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:29.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.479+0000 7f4926dab640 1 --2- 172.21.15.83:0/1750330270 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f490c066f60 0x7f490c069420 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f49100af000 tx=0x7f49100a7000 comp rx=0 tx=0).stop 2024-04-17T10:46:29.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.479+0000 7f4926dab640 1 -- 172.21.15.83:0/1750330270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f492015ae80 msgr2=0x7f49200d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:29.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.479+0000 7f4926dab640 1 --2- 172.21.15.83:0/1750330270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f492015ae80 0x7f49200d00e0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f491800e930 tx=0x7f491800edf0 comp rx=0 tx=0).stop 2024-04-17T10:46:29.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.479+0000 7f4926dab640 1 -- 172.21.15.83:0/1750330270 shutdown_connections 2024-04-17T10:46:29.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.479+0000 7f4926dab640 1 --2- 172.21.15.83:0/1750330270 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f490c066f60 0x7f490c069420 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:29.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.479+0000 7f4926dab640 1 --2- 172.21.15.83:0/1750330270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f492015ae80 0x7f49200d00e0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:29.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.479+0000 7f4926dab640 1 --2- 172.21.15.83:0/1750330270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49200b1ce0 0x7f49201fb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:29.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.479+0000 7f4926dab640 1 --2- 172.21.15.83:0/1750330270 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f492006c1b0 0x7f492009d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:29.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.479+0000 7f4926dab640 1 -- 172.21.15.83:0/1750330270 >> 172.21.15.83:0/1750330270 conn(0x7f49200b4eb0 msgr2=0x7f492006cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:29.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.479+0000 7f4926dab640 1 -- 172.21.15.83:0/1750330270 shutdown_connections 2024-04-17T10:46:29.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.479+0000 7f4926dab640 1 -- 172.21.15.83:0/1750330270 wait complete. 2024-04-17T10:46:29.492 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:46:29.493 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:46:29.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.615+0000 7ffb964a4640 1 Processor -- start 2024-04-17T10:46:29.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.615+0000 7ffb964a4640 1 -- start start 2024-04-17T10:46:29.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb964a4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb9015bac0 0x7ffb90071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:29.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb964a4640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ffb90070470 0x7ffb9006ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:29.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb964a4640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb9006c1f0 con 0x7ffb9015b6e0 2024-04-17T10:46:29.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb964a4640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb90070a10 con 0x7ffb90070470 2024-04-17T10:46:29.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb964a4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb90070cf0 con 0x7ffb9015bac0 2024-04-17T10:46:29.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb8ffff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ffb9015b6e0 0x7ffb900cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50678/0 (socket says 172.21.15.83:50678) 2024-04-17T10:46:29.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb8ffff640 1 -- 172.21.15.83:0/3140198314 learned_addr learned my addr 172.21.15.83:0/3140198314 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:46:29.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb94a41640 1 --2- 172.21.15.83:0/3140198314 >> v2:172.21.15.167:3300/0 conn(0x7ffb90070470 0x7ffb9006ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:29.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb8f7fe640 1 --2- 172.21.15.83:0/3140198314 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb9015bac0 0x7ffb90071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:29.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb8effd640 1 -- 172.21.15.83:0/3140198314 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1975713074 0 0) 0x7ffb9006c1f0 con 0x7ffb9015b6e0 2024-04-17T10:46:29.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb8effd640 1 -- 172.21.15.83:0/3140198314 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb78003530 con 0x7ffb9015b6e0 2024-04-17T10:46:29.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb8effd640 1 -- 172.21.15.83:0/3140198314 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4042631843 0 0) 0x7ffb78003530 con 0x7ffb9015b6e0 2024-04-17T10:46:29.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb8effd640 1 -- 172.21.15.83:0/3140198314 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb9015bac0 msgr2=0x7ffb90071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:29.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb8effd640 1 --2- 172.21.15.83:0/3140198314 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb9015bac0 0x7ffb90071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:29.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb8effd640 1 -- 172.21.15.83:0/3140198314 >> v2:172.21.15.167:3300/0 conn(0x7ffb90070470 msgr2=0x7ffb9006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:29.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb8effd640 1 --2- 172.21.15.83:0/3140198314 >> v2:172.21.15.167:3300/0 conn(0x7ffb90070470 0x7ffb9006ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:29.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb8effd640 1 -- 172.21.15.83:0/3140198314 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb90070fd0 con 0x7ffb9015b6e0 2024-04-17T10:46:29.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb8effd640 1 -- 172.21.15.83:0/3140198314 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7ffb80001680 con 0x7ffb9015b6e0 2024-04-17T10:46:29.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb8effd640 1 -- 172.21.15.83:0/3140198314 >> v1:172.21.15.83:6789/0 conn(0x7ffb9015b6e0 legacy=0x7ffb900cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:29.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb8effd640 1 --2- 172.21.15.83:0/3140198314 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb9015bac0 0x7ffb78003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:29.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb8effd640 1 --2- 172.21.15.83:0/3140198314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb78003fb0 0x7ffb78004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:29.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb8effd640 1 --2- 172.21.15.83:0/3140198314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb78004d50 0x7ffb780051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:29.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb8effd640 1 -- 172.21.15.83:0/3140198314 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb90070a10 con 0x7ffb78003fb0 2024-04-17T10:46:29.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb8effd640 1 -- 172.21.15.83:0/3140198314 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb90070cf0 con 0x7ffb78004d50 2024-04-17T10:46:29.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb8effd640 1 -- 172.21.15.83:0/3140198314 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7800ba00 con 0x7ffb9015bac0 2024-04-17T10:46:29.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb8f7fe640 1 --2- 172.21.15.83:0/3140198314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb78003fb0 0x7ffb78004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:29.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb8ffff640 1 --2- 172.21.15.83:0/3140198314 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb9015bac0 0x7ffb78003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:29.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb94a41640 1 --2- 172.21.15.83:0/3140198314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb78004d50 0x7ffb780051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:29.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb8f7fe640 1 -- 172.21.15.83:0/3140198314 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb9015bac0 msgr2=0x7ffb78003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:29.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb8f7fe640 1 --2- 172.21.15.83:0/3140198314 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb9015bac0 0x7ffb78003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:29.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb8f7fe640 1 -- 172.21.15.83:0/3140198314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb78004d50 msgr2=0x7ffb780051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:29.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb8f7fe640 1 --2- 172.21.15.83:0/3140198314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb78004d50 0x7ffb780051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:29.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb8f7fe640 1 -- 172.21.15.83:0/3140198314 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb78003530 con 0x7ffb78003fb0 2024-04-17T10:46:29.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.619+0000 7ffb94a41640 1 --2- 172.21.15.83:0/3140198314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb78004d50 0x7ffb780051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:29.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.623+0000 7ffb8f7fe640 1 --2- 172.21.15.83:0/3140198314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb78003fb0 0x7ffb78004810 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7ffb84009ab0 tx=0x7ffb8401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b4e384d6b72b5d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:29.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.623+0000 7ffb8effd640 1 -- 172.21.15.83:0/3140198314 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb840b2070 con 0x7ffb78003fb0 2024-04-17T10:46:29.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.623+0000 7ffb8effd640 1 -- 172.21.15.83:0/3140198314 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb840a5530 con 0x7ffb78003fb0 2024-04-17T10:46:29.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.623+0000 7ffb8effd640 1 -- 172.21.15.83:0/3140198314 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb840096b0 con 0x7ffb78003fb0 2024-04-17T10:46:29.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.623+0000 7ffb964a4640 1 -- 172.21.15.83:0/3140198314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb78003fb0 msgr2=0x7ffb78004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:29.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.623+0000 7ffb964a4640 1 --2- 172.21.15.83:0/3140198314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb78003fb0 0x7ffb78004810 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7ffb84009ab0 tx=0x7ffb8401ca50 comp rx=0 tx=0).stop 2024-04-17T10:46:29.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.623+0000 7ffb94a41640 1 -- 172.21.15.83:0/3140198314 reap_dead start 2024-04-17T10:46:29.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.623+0000 7ffb964a4640 1 -- 172.21.15.83:0/3140198314 shutdown_connections 2024-04-17T10:46:29.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.623+0000 7ffb964a4640 1 -- 172.21.15.83:0/3140198314 >> 172.21.15.83:0/3140198314 conn(0x7ffb900b15b0 msgr2=0x7ffb900b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:29.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.623+0000 7ffb964a4640 1 -- 172.21.15.83:0/3140198314 shutdown_connections 2024-04-17T10:46:29.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.623+0000 7ffb964a4640 1 -- 172.21.15.83:0/3140198314 wait complete. 2024-04-17T10:46:29.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.623+0000 7ffb964a4640 1 Processor -- start 2024-04-17T10:46:29.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.623+0000 7ffb964a4640 1 -- start start 2024-04-17T10:46:29.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.623+0000 7ffb964a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb78003fb0 0x7ffb90142200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:29.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.623+0000 7ffb964a4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb9014fbf0 0x7ffb90150050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:29.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.623+0000 7ffb964a4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb90151cd0 0x7ffb90152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:29.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.623+0000 7ffb964a4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9006b4d0 con 0x7ffb78003fb0 2024-04-17T10:46:29.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.623+0000 7ffb964a4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9006b350 con 0x7ffb9014fbf0 2024-04-17T10:46:29.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.623+0000 7ffb964a4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9006b650 con 0x7ffb90151cd0 2024-04-17T10:46:29.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.623+0000 7ffb8ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb78003fb0 0x7ffb90142200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:29.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.623+0000 7ffb8ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb78003fb0 0x7ffb90142200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60988/0 (socket says 172.21.15.83:60988) 2024-04-17T10:46:29.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.623+0000 7ffb8ffff640 1 -- 172.21.15.83:0/1635153204 learned_addr learned my addr 172.21.15.83:0/1635153204 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:29.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.623+0000 7ffb8f7fe640 1 --2- 172.21.15.83:0/1635153204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb9014fbf0 0x7ffb90150050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:29.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.623+0000 7ffb94a41640 1 --2- 172.21.15.83:0/1635153204 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb90151cd0 0x7ffb90152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:29.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.623+0000 7ffb8ffff640 1 -- 172.21.15.83:0/1635153204 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb90151cd0 msgr2=0x7ffb90152180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:29.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.623+0000 7ffb8ffff640 1 --2- 172.21.15.83:0/1635153204 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb90151cd0 0x7ffb90152180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:29.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.623+0000 7ffb8ffff640 1 -- 172.21.15.83:0/1635153204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb9014fbf0 msgr2=0x7ffb90150050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:29.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.623+0000 7ffb8ffff640 1 --2- 172.21.15.83:0/1635153204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb9014fbf0 0x7ffb90150050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:29.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.623+0000 7ffb8ffff640 1 -- 172.21.15.83:0/1635153204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb90150990 con 0x7ffb78003fb0 2024-04-17T10:46:29.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.623+0000 7ffb8ffff640 1 --2- 172.21.15.83:0/1635153204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb78003fb0 0x7ffb90142200 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7ffb80011810 tx=0x7ffb80011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:29.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.627+0000 7ffb8cff9640 1 -- 172.21.15.83:0/1635153204 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb80010040 con 0x7ffb78003fb0 2024-04-17T10:46:29.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.627+0000 7ffb8cff9640 1 -- 172.21.15.83:0/1635153204 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb8001a070 con 0x7ffb78003fb0 2024-04-17T10:46:29.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.627+0000 7ffb964a4640 1 -- 172.21.15.83:0/1635153204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb90150c70 con 0x7ffb78003fb0 2024-04-17T10:46:29.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.627+0000 7ffb8cff9640 1 -- 172.21.15.83:0/1635153204 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8000fb60 con 0x7ffb78003fb0 2024-04-17T10:46:29.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.627+0000 7ffb964a4640 1 -- 172.21.15.83:0/1635153204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb90151230 con 0x7ffb78003fb0 2024-04-17T10:46:29.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.627+0000 7ffb964a4640 1 -- 172.21.15.83:0/1635153204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb90074f90 con 0x7ffb78003fb0 2024-04-17T10:46:29.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.627+0000 7ffb8cff9640 1 -- 172.21.15.83:0/1635153204 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffb8000fd00 con 0x7ffb78003fb0 2024-04-17T10:46:29.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.627+0000 7ffb8cff9640 1 --2- 172.21.15.83:0/1635153204 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffb7c066f90 0x7ffb7c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:29.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.627+0000 7ffb8cff9640 1 -- 172.21.15.83:0/1635153204 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ffb80087dc0 con 0x7ffb78003fb0 2024-04-17T10:46:29.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.627+0000 7ffb8f7fe640 1 --2- 172.21.15.83:0/1635153204 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffb7c066f90 0x7ffb7c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:29.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.627+0000 7ffb8f7fe640 1 --2- 172.21.15.83:0/1635153204 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffb7c066f90 0x7ffb7c069450 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ffb840af000 tx=0x7ffb840046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:29.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.631+0000 7ffb8cff9640 1 -- 172.21.15.83:0/1635153204 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffb800590d0 con 0x7ffb78003fb0 2024-04-17T10:46:29.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.795+0000 7ffb964a4640 1 -- 172.21.15.83:0/1635153204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffb90151400 con 0x7ffb78003fb0 2024-04-17T10:46:29.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.795+0000 7ffb8cff9640 1 -- 172.21.15.83:0/1635153204 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffb800589d0 con 0x7ffb78003fb0 2024-04-17T10:46:29.790 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:46:29.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.795+0000 7ffb964a4640 1 -- 172.21.15.83:0/1635153204 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffb7c066f90 msgr2=0x7ffb7c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:29.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.795+0000 7ffb964a4640 1 --2- 172.21.15.83:0/1635153204 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffb7c066f90 0x7ffb7c069450 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ffb840af000 tx=0x7ffb840046d0 comp rx=0 tx=0).stop 2024-04-17T10:46:29.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.795+0000 7ffb964a4640 1 -- 172.21.15.83:0/1635153204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb78003fb0 msgr2=0x7ffb90142200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:29.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.795+0000 7ffb964a4640 1 --2- 172.21.15.83:0/1635153204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb78003fb0 0x7ffb90142200 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7ffb80011810 tx=0x7ffb80011cd0 comp rx=0 tx=0).stop 2024-04-17T10:46:29.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.795+0000 7ffb964a4640 1 -- 172.21.15.83:0/1635153204 shutdown_connections 2024-04-17T10:46:29.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.795+0000 7ffb964a4640 1 --2- 172.21.15.83:0/1635153204 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffb7c066f90 0x7ffb7c069450 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:29.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.795+0000 7ffb964a4640 1 --2- 172.21.15.83:0/1635153204 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb90151cd0 0x7ffb90152180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:29.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.795+0000 7ffb964a4640 1 --2- 172.21.15.83:0/1635153204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb9014fbf0 0x7ffb90150050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:29.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.795+0000 7ffb964a4640 1 --2- 172.21.15.83:0/1635153204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb78003fb0 0x7ffb90142200 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:29.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.795+0000 7ffb964a4640 1 -- 172.21.15.83:0/1635153204 >> 172.21.15.83:0/1635153204 conn(0x7ffb900b15b0 msgr2=0x7ffb900b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:29.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.795+0000 7ffb964a4640 1 -- 172.21.15.83:0/1635153204 shutdown_connections 2024-04-17T10:46:29.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.795+0000 7ffb964a4640 1 -- 172.21.15.83:0/1635153204 wait complete. 2024-04-17T10:46:29.809 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort' 2024-04-17T10:46:29.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.931+0000 7f54b6e07640 1 Processor -- start 2024-04-17T10:46:29.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.931+0000 7f54b6e07640 1 -- start start 2024-04-17T10:46:29.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.931+0000 7f54b6e07640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54b015ba30 0x7f54b0072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:29.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.931+0000 7f54b6e07640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f54b00cfbe0 0x7f54b006c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:29.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.931+0000 7f54b6e07640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54b006e320 con 0x7f54b015c960 2024-04-17T10:46:29.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.931+0000 7f54b6e07640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b0071ea0 con 0x7f54b00cfbe0 2024-04-17T10:46:29.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.931+0000 7f54b6e07640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b006e040 con 0x7f54b015ba30 2024-04-17T10:46:29.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.931+0000 7f54affff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f54b015c960 0x7f54b00cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50690/0 (socket says 172.21.15.83:50690) 2024-04-17T10:46:29.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.931+0000 7f54affff640 1 -- 172.21.15.83:0/813365919 learned_addr learned my addr 172.21.15.83:0/813365919 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:46:29.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.931+0000 7f54b53a4640 1 --2- 172.21.15.83:0/813365919 >> v2:172.21.15.83:3300/0 conn(0x7f54b00cfbe0 0x7f54b006c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:29.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.931+0000 7f54af7fe640 1 -- 172.21.15.83:0/813365919 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 613504248 0 0) 0x7f54b006e320 con 0x7f54b015c960 2024-04-17T10:46:29.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.931+0000 7f54af7fe640 1 -- 172.21.15.83:0/813365919 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5498003530 con 0x7f54b015c960 2024-04-17T10:46:29.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.931+0000 7f54b4ba3640 1 --2- 172.21.15.83:0/813365919 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54b015ba30 0x7f54b0072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:29.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.931+0000 7f54b53a4640 1 -- 172.21.15.83:0/813365919 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54b015ba30 msgr2=0x7f54b0072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:29.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.931+0000 7f54b53a4640 1 --2- 172.21.15.83:0/813365919 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54b015ba30 0x7f54b0072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:29.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.931+0000 7f54b53a4640 1 -- 172.21.15.83:0/813365919 >> v1:172.21.15.83:6789/0 conn(0x7f54b015c960 legacy=0x7f54b00cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:29.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.931+0000 7f54b53a4640 1 -- 172.21.15.83:0/813365919 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b0071850 con 0x7f54b00cfbe0 2024-04-17T10:46:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.935+0000 7f54b53a4640 1 --2- 172.21.15.83:0/813365919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54b00cfbe0 0x7f54b006c490 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009d90 tx=0x7f54a801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e68b3b21e46e57 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.935+0000 7f54af7fe640 1 -- 172.21.15.83:0/813365919 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a80b2070 con 0x7f54b00cfbe0 2024-04-17T10:46:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.935+0000 7f54af7fe640 1 -- 172.21.15.83:0/813365919 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a80a57f0 con 0x7f54b00cfbe0 2024-04-17T10:46:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.935+0000 7f54af7fe640 1 -- 172.21.15.83:0/813365919 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a80a5bc0 con 0x7f54b00cfbe0 2024-04-17T10:46:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.935+0000 7f54b6e07640 1 -- 172.21.15.83:0/813365919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54b00cfbe0 msgr2=0x7f54b006c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.935+0000 7f54b6e07640 1 --2- 172.21.15.83:0/813365919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54b00cfbe0 0x7f54b006c490 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009d90 tx=0x7f54a801c8e0 comp rx=0 tx=0).stop 2024-04-17T10:46:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.935+0000 7f54b6e07640 1 -- 172.21.15.83:0/813365919 shutdown_connections 2024-04-17T10:46:29.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.935+0000 7f54b6e07640 1 --2- 172.21.15.83:0/813365919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54b00cfbe0 0x7f54b006c490 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:29.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.935+0000 7f54b6e07640 1 --2- 172.21.15.83:0/813365919 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54b015ba30 0x7f54b0072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:29.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.935+0000 7f54b6e07640 1 -- 172.21.15.83:0/813365919 >> 172.21.15.83:0/813365919 conn(0x7f54b006abe0 msgr2=0x7f54b006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:29.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.935+0000 7f54b6e07640 1 -- 172.21.15.83:0/813365919 shutdown_connections 2024-04-17T10:46:29.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.935+0000 7f54b6e07640 1 -- 172.21.15.83:0/813365919 wait complete. 2024-04-17T10:46:29.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.939+0000 7f54b6e07640 1 Processor -- start 2024-04-17T10:46:29.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.939+0000 7f54b6e07640 1 -- start start 2024-04-17T10:46:29.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.939+0000 7f54b6e07640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54b00cfbe0 0x7f54b0149590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:29.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.939+0000 7f54b6e07640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54b015ba30 0x7f54b0143e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:29.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.939+0000 7f54b6e07640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54b015c960 0x7f54b00ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:29.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.939+0000 7f54b6e07640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b015cee0 con 0x7f54b015ba30 2024-04-17T10:46:29.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.939+0000 7f54b6e07640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b015cd60 con 0x7f54b015c960 2024-04-17T10:46:29.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.939+0000 7f54b6e07640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b0168520 con 0x7f54b00cfbe0 2024-04-17T10:46:29.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.939+0000 7f54affff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54b015ba30 0x7f54b0143e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:29.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.939+0000 7f54affff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54b015ba30 0x7f54b0143e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32768/0 (socket says 172.21.15.83:32768) 2024-04-17T10:46:29.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.939+0000 7f54affff640 1 -- 172.21.15.83:0/1524669997 learned_addr learned my addr 172.21.15.83:0/1524669997 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:29.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.939+0000 7f54b53a4640 1 --2- 172.21.15.83:0/1524669997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54b015c960 0x7f54b00ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:29.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.939+0000 7f54b4ba3640 1 --2- 172.21.15.83:0/1524669997 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54b00cfbe0 0x7f54b0149590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:29.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.939+0000 7f54affff640 1 -- 172.21.15.83:0/1524669997 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54b00cfbe0 msgr2=0x7f54b0149590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:29.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.939+0000 7f54affff640 1 --2- 172.21.15.83:0/1524669997 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54b00cfbe0 0x7f54b0149590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:29.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.939+0000 7f54affff640 1 -- 172.21.15.83:0/1524669997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54b015c960 msgr2=0x7f54b00ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:29.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.939+0000 7f54affff640 1 --2- 172.21.15.83:0/1524669997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54b015c960 0x7f54b00ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:29.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.939+0000 7f54affff640 1 -- 172.21.15.83:0/1524669997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b0148530 con 0x7f54b015ba30 2024-04-17T10:46:29.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.939+0000 7f54b4ba3640 1 --2- 172.21.15.83:0/1524669997 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54b00cfbe0 0x7f54b0149590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:29.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.939+0000 7f54affff640 1 --2- 172.21.15.83:0/1524669997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54b015ba30 0x7f54b0143e20 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f54a40010f0 tx=0x7f54a400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:29.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.939+0000 7f54ad7fa640 1 -- 172.21.15.83:0/1524669997 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a4017070 con 0x7f54b015ba30 2024-04-17T10:46:29.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.939+0000 7f54ad7fa640 1 -- 172.21.15.83:0/1524669997 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a4005d10 con 0x7f54b015ba30 2024-04-17T10:46:29.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.939+0000 7f54ad7fa640 1 -- 172.21.15.83:0/1524669997 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a4007310 con 0x7f54b015ba30 2024-04-17T10:46:29.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.939+0000 7f54b6e07640 1 -- 172.21.15.83:0/1524669997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b01487d0 con 0x7f54b015ba30 2024-04-17T10:46:29.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.939+0000 7f54b6e07640 1 -- 172.21.15.83:0/1524669997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b0148d90 con 0x7f54b015ba30 2024-04-17T10:46:29.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.939+0000 7f54b6e07640 1 -- 172.21.15.83:0/1524669997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54b0074f90 con 0x7f54b015ba30 2024-04-17T10:46:29.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.939+0000 7f54ad7fa640 1 -- 172.21.15.83:0/1524669997 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f54a401a070 con 0x7f54b015ba30 2024-04-17T10:46:29.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.939+0000 7f54ad7fa640 1 --2- 172.21.15.83:0/1524669997 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f549c066f60 0x7f549c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:29.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.939+0000 7f54ad7fa640 1 -- 172.21.15.83:0/1524669997 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f54a4084070 con 0x7f54b015ba30 2024-04-17T10:46:29.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.939+0000 7f54b4ba3640 1 --2- 172.21.15.83:0/1524669997 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f549c066f60 0x7f549c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:29.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.943+0000 7f54b4ba3640 1 --2- 172.21.15.83:0/1524669997 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f549c066f60 0x7f549c069420 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f54a0009840 tx=0x7f54a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:29.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:29.943+0000 7f54ad7fa640 1 -- 172.21.15.83:0/1524669997 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f54a4055410 con 0x7f54b015ba30 2024-04-17T10:46:30.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:30.159+0000 7f54b6e07640 1 -- 172.21.15.83:0/1524669997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]} v 0) v1 -- 0x7f54b015ace0 con 0x7f54b015ba30 2024-04-17T10:46:31.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.151+0000 7f54ad7fa640 1 -- 172.21.15.83:0/1524669997 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]}]=0 v362) v1 ==== 147+0+0 (secure 0 0 0) 0x7f54a4054d10 con 0x7f54b015ba30 2024-04-17T10:46:31.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.155+0000 7f54b6e07640 1 -- 172.21.15.83:0/1524669997 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f549c066f60 msgr2=0x7f549c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:31.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.155+0000 7f54b6e07640 1 --2- 172.21.15.83:0/1524669997 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f549c066f60 0x7f549c069420 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f54a0009840 tx=0x7f54a0008040 comp rx=0 tx=0).stop 2024-04-17T10:46:31.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.155+0000 7f54b6e07640 1 -- 172.21.15.83:0/1524669997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54b015ba30 msgr2=0x7f54b0143e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:31.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.155+0000 7f54b6e07640 1 --2- 172.21.15.83:0/1524669997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54b015ba30 0x7f54b0143e20 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f54a40010f0 tx=0x7f54a400c890 comp rx=0 tx=0).stop 2024-04-17T10:46:31.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.155+0000 7f54b6e07640 1 -- 172.21.15.83:0/1524669997 shutdown_connections 2024-04-17T10:46:31.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.155+0000 7f54b6e07640 1 --2- 172.21.15.83:0/1524669997 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f549c066f60 0x7f549c069420 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:31.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.155+0000 7f54b6e07640 1 --2- 172.21.15.83:0/1524669997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54b015c960 0x7f54b00ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:31.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.155+0000 7f54b6e07640 1 --2- 172.21.15.83:0/1524669997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54b015ba30 0x7f54b0143e20 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:31.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.155+0000 7f54b6e07640 1 --2- 172.21.15.83:0/1524669997 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54b00cfbe0 0x7f54b0149590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:31.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.155+0000 7f54b6e07640 1 -- 172.21.15.83:0/1524669997 >> 172.21.15.83:0/1524669997 conn(0x7f54b006abe0 msgr2=0x7f54b0152730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:31.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.155+0000 7f54b6e07640 1 -- 172.21.15.83:0/1524669997 shutdown_connections 2024-04-17T10:46:31.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.155+0000 7f54b6e07640 1 -- 172.21.15.83:0/1524669997 wait complete. 2024-04-17T10:46:31.172 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-17T10:46:31.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a99a9a640 1 Processor -- start 2024-04-17T10:46:31.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a99a9a640 1 -- start start 2024-04-17T10:46:31.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a99a9a640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0a94154de0 0x7f0a9406ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:31.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a99a9a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a9406b350 0x7f0a940b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:31.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a99a9a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a9415bd70 con 0x7f0a940b4040 2024-04-17T10:46:31.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a99a9a640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a940b4870 con 0x7f0a94154de0 2024-04-17T10:46:31.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a99a9a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9415bf60 con 0x7f0a9406b350 2024-04-17T10:46:31.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a93fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0a940b4040 0x7f0a94071580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50692/0 (socket says 172.21.15.83:50692) 2024-04-17T10:46:31.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a93fff640 1 -- 172.21.15.83:0/1441130215 learned_addr learned my addr 172.21.15.83:0/1441130215 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:46:31.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a937fe640 1 --2- 172.21.15.83:0/1441130215 >> v2:172.21.15.167:3300/0 conn(0x7f0a94154de0 0x7f0a9406ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:31.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a927fc640 1 -- 172.21.15.83:0/1441130215 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3784222288 0 0) 0x7f0a9415bd70 con 0x7f0a940b4040 2024-04-17T10:46:31.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a92ffd640 1 --2- 172.21.15.83:0/1441130215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a9406b350 0x7f0a940b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:31.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a927fc640 1 -- 172.21.15.83:0/1441130215 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a7c003530 con 0x7f0a940b4040 2024-04-17T10:46:31.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a927fc640 1 -- 172.21.15.83:0/1441130215 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2699488050 0 0) 0x7f0a7c003530 con 0x7f0a940b4040 2024-04-17T10:46:31.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a927fc640 1 -- 172.21.15.83:0/1441130215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a9406b350 msgr2=0x7f0a940b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:31.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a927fc640 1 --2- 172.21.15.83:0/1441130215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a9406b350 0x7f0a940b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:31.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a927fc640 1 -- 172.21.15.83:0/1441130215 >> v2:172.21.15.167:3300/0 conn(0x7f0a94154de0 msgr2=0x7f0a9406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:31.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a927fc640 1 --2- 172.21.15.83:0/1441130215 >> v2:172.21.15.167:3300/0 conn(0x7f0a94154de0 0x7f0a9406ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:31.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a927fc640 1 -- 172.21.15.83:0/1441130215 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9415afb0 con 0x7f0a940b4040 2024-04-17T10:46:31.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a927fc640 1 -- 172.21.15.83:0/1441130215 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f0a8c001680 con 0x7f0a940b4040 2024-04-17T10:46:31.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a927fc640 1 -- 172.21.15.83:0/1441130215 >> v1:172.21.15.83:6789/0 conn(0x7f0a940b4040 legacy=0x7f0a94071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:31.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a927fc640 1 --2- 172.21.15.83:0/1441130215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a7c003a30 0x7f0a7c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:31.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a927fc640 1 --2- 172.21.15.83:0/1441130215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a7c004810 0x7f0a7c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:31.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a927fc640 1 --2- 172.21.15.83:0/1441130215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a9406b350 0x7f0a7c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:31.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a927fc640 1 -- 172.21.15.83:0/1441130215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9415bf60 con 0x7f0a7c003a30 2024-04-17T10:46:31.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a927fc640 1 -- 172.21.15.83:0/1441130215 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c00b9d0 con 0x7f0a7c004810 2024-04-17T10:46:31.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a927fc640 1 -- 172.21.15.83:0/1441130215 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c00bb80 con 0x7f0a9406b350 2024-04-17T10:46:31.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a937fe640 1 --2- 172.21.15.83:0/1441130215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a7c003a30 0x7f0a7c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:31.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a93fff640 1 --2- 172.21.15.83:0/1441130215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a9406b350 0x7f0a7c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:31.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a937fe640 1 -- 172.21.15.83:0/1441130215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a9406b350 msgr2=0x7f0a7c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:31.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a937fe640 1 --2- 172.21.15.83:0/1441130215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a9406b350 0x7f0a7c0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:31.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a937fe640 1 -- 172.21.15.83:0/1441130215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a7c004810 msgr2=0x7f0a7c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:31.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a937fe640 1 --2- 172.21.15.83:0/1441130215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a7c004810 0x7f0a7c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:31.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.303+0000 7f0a937fe640 1 -- 172.21.15.83:0/1441130215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7c003530 con 0x7f0a7c003a30 2024-04-17T10:46:31.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.307+0000 7f0a937fe640 1 --2- 172.21.15.83:0/1441130215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a7c003a30 0x7f0a7c003e50 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009fc0 tx=0x7f0a8401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fdac18a26473a75 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:31.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.307+0000 7f0a927fc640 1 -- 172.21.15.83:0/1441130215 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a840b2070 con 0x7f0a7c003a30 2024-04-17T10:46:31.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.307+0000 7f0a927fc640 1 -- 172.21.15.83:0/1441130215 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a840a59d0 con 0x7f0a7c003a30 2024-04-17T10:46:31.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.307+0000 7f0a927fc640 1 -- 172.21.15.83:0/1441130215 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a84009c10 con 0x7f0a7c003a30 2024-04-17T10:46:31.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.307+0000 7f0a99a9a640 1 -- 172.21.15.83:0/1441130215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a7c003a30 msgr2=0x7f0a7c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:31.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.307+0000 7f0a99a9a640 1 --2- 172.21.15.83:0/1441130215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a7c003a30 0x7f0a7c003e50 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009fc0 tx=0x7f0a8401ccc0 comp rx=0 tx=0).stop 2024-04-17T10:46:31.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.307+0000 7f0a93fff640 1 -- 172.21.15.83:0/1441130215 reap_dead start 2024-04-17T10:46:31.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.307+0000 7f0a99a9a640 1 -- 172.21.15.83:0/1441130215 shutdown_connections 2024-04-17T10:46:31.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.307+0000 7f0a99a9a640 1 -- 172.21.15.83:0/1441130215 >> 172.21.15.83:0/1441130215 conn(0x7f0a940b15b0 msgr2=0x7f0a940b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:31.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.307+0000 7f0a99a9a640 1 -- 172.21.15.83:0/1441130215 shutdown_connections 2024-04-17T10:46:31.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.307+0000 7f0a99a9a640 1 -- 172.21.15.83:0/1441130215 wait complete. 2024-04-17T10:46:31.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.307+0000 7f0a99a9a640 1 Processor -- start 2024-04-17T10:46:31.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.307+0000 7f0a99a9a640 1 -- start start 2024-04-17T10:46:31.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.307+0000 7f0a99a9a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a7c003a30 0x7f0a9414ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:31.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.307+0000 7f0a99a9a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a940a9a20 0x7f0a94155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:31.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.307+0000 7f0a99a9a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a94158950 0x7f0a94158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:31.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.307+0000 7f0a99a9a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a941551c0 con 0x7f0a7c003a30 2024-04-17T10:46:31.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.307+0000 7f0a99a9a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9415b850 con 0x7f0a940a9a20 2024-04-17T10:46:31.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.307+0000 7f0a99a9a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a940b4470 con 0x7f0a94158950 2024-04-17T10:46:31.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.307+0000 7f0a937fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a7c003a30 0x7f0a9414ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:31.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.307+0000 7f0a937fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a7c003a30 0x7f0a9414ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32784/0 (socket says 172.21.15.83:32784) 2024-04-17T10:46:31.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.307+0000 7f0a937fe640 1 -- 172.21.15.83:0/1876942132 learned_addr learned my addr 172.21.15.83:0/1876942132 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:31.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.311+0000 7f0a92ffd640 1 --2- 172.21.15.83:0/1876942132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a940a9a20 0x7f0a94155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:31.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.311+0000 7f0a93fff640 1 --2- 172.21.15.83:0/1876942132 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a94158950 0x7f0a94158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:31.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.311+0000 7f0a937fe640 1 -- 172.21.15.83:0/1876942132 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a94158950 msgr2=0x7f0a94158dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:31.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.311+0000 7f0a937fe640 1 --2- 172.21.15.83:0/1876942132 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a94158950 0x7f0a94158dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:31.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.311+0000 7f0a937fe640 1 -- 172.21.15.83:0/1876942132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a940a9a20 msgr2=0x7f0a94155d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:31.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.311+0000 7f0a937fe640 1 --2- 172.21.15.83:0/1876942132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a940a9a20 0x7f0a94155d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:31.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.311+0000 7f0a937fe640 1 -- 172.21.15.83:0/1876942132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a940a9f50 con 0x7f0a7c003a30 2024-04-17T10:46:31.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.311+0000 7f0a92ffd640 1 --2- 172.21.15.83:0/1876942132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a940a9a20 0x7f0a94155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:31.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.311+0000 7f0a937fe640 1 --2- 172.21.15.83:0/1876942132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a7c003a30 0x7f0a9414ff90 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f0a8401c730 tx=0x7f0a84009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:31.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.311+0000 7f0a7bfff640 1 -- 172.21.15.83:0/1876942132 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a840ba070 con 0x7f0a7c003a30 2024-04-17T10:46:31.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.311+0000 7f0a7bfff640 1 -- 172.21.15.83:0/1876942132 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a84003ba0 con 0x7f0a7c003a30 2024-04-17T10:46:31.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.311+0000 7f0a7bfff640 1 -- 172.21.15.83:0/1876942132 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a840b2040 con 0x7f0a7c003a30 2024-04-17T10:46:31.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.311+0000 7f0a99a9a640 1 -- 172.21.15.83:0/1876942132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a940aa230 con 0x7f0a7c003a30 2024-04-17T10:46:31.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.311+0000 7f0a99a9a640 1 -- 172.21.15.83:0/1876942132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a940aa7f0 con 0x7f0a7c003a30 2024-04-17T10:46:31.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.311+0000 7f0a99a9a640 1 -- 172.21.15.83:0/1876942132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a94159310 con 0x7f0a7c003a30 2024-04-17T10:46:31.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.311+0000 7f0a7bfff640 1 -- 172.21.15.83:0/1876942132 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0a840a5890 con 0x7f0a7c003a30 2024-04-17T10:46:31.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.311+0000 7f0a7bfff640 1 --2- 172.21.15.83:0/1876942132 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0a80066f90 0x7f0a80069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:31.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.311+0000 7f0a7bfff640 1 -- 172.21.15.83:0/1876942132 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0a8411f1e0 con 0x7f0a7c003a30 2024-04-17T10:46:31.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.311+0000 7f0a92ffd640 1 --2- 172.21.15.83:0/1876942132 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0a80066f90 0x7f0a80069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:31.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.311+0000 7f0a92ffd640 1 --2- 172.21.15.83:0/1876942132 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0a80066f90 0x7f0a80069450 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0a94156940 tx=0x7f0a88008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:31.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.315+0000 7f0a7bfff640 1 -- 172.21.15.83:0/1876942132 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0a840169d0 con 0x7f0a7c003a30 2024-04-17T10:46:31.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.475+0000 7f0a99a9a640 1 -- 172.21.15.83:0/1876942132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0a94074f90 con 0x7f0a7c003a30 2024-04-17T10:46:31.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.479+0000 7f0a7bfff640 1 -- 172.21.15.83:0/1876942132 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0a840effc0 con 0x7f0a7c003a30 2024-04-17T10:46:31.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.483+0000 7f0a99a9a640 1 -- 172.21.15.83:0/1876942132 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0a80066f90 msgr2=0x7f0a80069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:31.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.483+0000 7f0a99a9a640 1 --2- 172.21.15.83:0/1876942132 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0a80066f90 0x7f0a80069450 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0a94156940 tx=0x7f0a88008040 comp rx=0 tx=0).stop 2024-04-17T10:46:31.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.483+0000 7f0a99a9a640 1 -- 172.21.15.83:0/1876942132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a7c003a30 msgr2=0x7f0a9414ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:31.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.483+0000 7f0a99a9a640 1 --2- 172.21.15.83:0/1876942132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a7c003a30 0x7f0a9414ff90 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f0a8401c730 tx=0x7f0a84009750 comp rx=0 tx=0).stop 2024-04-17T10:46:31.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.483+0000 7f0a99a9a640 1 -- 172.21.15.83:0/1876942132 shutdown_connections 2024-04-17T10:46:31.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.483+0000 7f0a99a9a640 1 --2- 172.21.15.83:0/1876942132 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0a80066f90 0x7f0a80069450 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:31.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.483+0000 7f0a99a9a640 1 --2- 172.21.15.83:0/1876942132 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a94158950 0x7f0a94158dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:31.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.483+0000 7f0a99a9a640 1 --2- 172.21.15.83:0/1876942132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a940a9a20 0x7f0a94155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:31.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.483+0000 7f0a99a9a640 1 --2- 172.21.15.83:0/1876942132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a7c003a30 0x7f0a9414ff90 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:31.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.483+0000 7f0a99a9a640 1 -- 172.21.15.83:0/1876942132 >> 172.21.15.83:0/1876942132 conn(0x7f0a940b15b0 msgr2=0x7f0a940ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:31.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.483+0000 7f0a99a9a640 1 -- 172.21.15.83:0/1876942132 shutdown_connections 2024-04-17T10:46:31.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.483+0000 7f0a99a9a640 1 -- 172.21.15.83:0/1876942132 wait complete. 2024-04-17T10:46:31.495 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T10:46:31.495 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:46:31.543 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:46:31.587 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T10:46:31.587 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:46:31.588 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:46:31.594 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16063 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:46:31.595 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:46:31.595 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:46:31.634 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:46:31.634 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:46:31.634 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:46:31.642 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16063 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:46:31.642 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:46:31.642 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:46:31.683 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:46:31.683 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-17T10:46:31.683 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:46:31.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.811+0000 7fa427d24640 1 Processor -- start 2024-04-17T10:46:31.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.811+0000 7fa427d24640 1 -- start start 2024-04-17T10:46:31.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa427d24640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa42015ba30 0x7fa420072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa427d24640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa42015c960 0x7fa4200cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa427d24640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4200cfea0 con 0x7fa4200cfa10 2024-04-17T10:46:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa427d24640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa420071ea0 con 0x7fa42015ba30 2024-04-17T10:46:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa427d24640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa42006e320 con 0x7fa42015c960 2024-04-17T10:46:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa4262c1640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa4200cfa10 0x7fa42006c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50698/0 (socket says 172.21.15.83:50698) 2024-04-17T10:46:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa4262c1640 1 -- 172.21.15.83:0/3929029838 learned_addr learned my addr 172.21.15.83:0/3929029838 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:46:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa424abe640 1 -- 172.21.15.83:0/3929029838 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2329265168 0 0) 0x7fa4200cfea0 con 0x7fa4200cfa10 2024-04-17T10:46:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa425ac0640 1 --2- 172.21.15.83:0/3929029838 >> v2:172.21.15.167:3300/0 conn(0x7fa42015ba30 0x7fa420072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa4252bf640 1 --2- 172.21.15.83:0/3929029838 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa42015c960 0x7fa4200cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa424abe640 1 -- 172.21.15.83:0/3929029838 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa40c003530 con 0x7fa4200cfa10 2024-04-17T10:46:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa424abe640 1 -- 172.21.15.83:0/3929029838 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2616149209 0 0) 0x7fa40c003530 con 0x7fa4200cfa10 2024-04-17T10:46:31.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa424abe640 1 -- 172.21.15.83:0/3929029838 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa42015c960 msgr2=0x7fa4200cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:31.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa424abe640 1 --2- 172.21.15.83:0/3929029838 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa42015c960 0x7fa4200cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:31.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa424abe640 1 -- 172.21.15.83:0/3929029838 >> v2:172.21.15.167:3300/0 conn(0x7fa42015ba30 msgr2=0x7fa420072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:31.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa424abe640 1 --2- 172.21.15.83:0/3929029838 >> v2:172.21.15.167:3300/0 conn(0x7fa42015ba30 0x7fa420072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:31.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa424abe640 1 -- 172.21.15.83:0/3929029838 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa42006e040 con 0x7fa4200cfa10 2024-04-17T10:46:31.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa424abe640 1 -- 172.21.15.83:0/3929029838 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fa418001680 con 0x7fa4200cfa10 2024-04-17T10:46:31.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa424abe640 1 -- 172.21.15.83:0/3929029838 >> v1:172.21.15.83:6789/0 conn(0x7fa4200cfa10 legacy=0x7fa42006c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:31.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa424abe640 1 --2- 172.21.15.83:0/3929029838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa40c0039c0 0x7fa40c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:31.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa424abe640 1 --2- 172.21.15.83:0/3929029838 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa42015c960 0x7fa40c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:31.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa424abe640 1 --2- 172.21.15.83:0/3929029838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa40c004d50 0x7fa40c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:31.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa424abe640 1 -- 172.21.15.83:0/3929029838 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa420071ea0 con 0x7fa40c0039c0 2024-04-17T10:46:31.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa424abe640 1 -- 172.21.15.83:0/3929029838 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42006e320 con 0x7fa40c004d50 2024-04-17T10:46:31.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa424abe640 1 -- 172.21.15.83:0/3929029838 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c00b9d0 con 0x7fa42015c960 2024-04-17T10:46:31.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa425ac0640 1 --2- 172.21.15.83:0/3929029838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa40c0039c0 0x7fa40c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:31.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa4252bf640 1 --2- 172.21.15.83:0/3929029838 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa42015c960 0x7fa40c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:31.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa425ac0640 1 -- 172.21.15.83:0/3929029838 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa42015c960 msgr2=0x7fa40c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:31.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa425ac0640 1 --2- 172.21.15.83:0/3929029838 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa42015c960 0x7fa40c004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:31.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa4262c1640 1 --2- 172.21.15.83:0/3929029838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa40c004d50 0x7fa40c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:31.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa425ac0640 1 -- 172.21.15.83:0/3929029838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa40c004d50 msgr2=0x7fa40c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:31.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa425ac0640 1 --2- 172.21.15.83:0/3929029838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa40c004d50 0x7fa40c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:31.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa425ac0640 1 -- 172.21.15.83:0/3929029838 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40c003530 con 0x7fa40c0039c0 2024-04-17T10:46:31.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.815+0000 7fa4262c1640 1 --2- 172.21.15.83:0/3929029838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa40c004d50 0x7fa40c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:31.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.819+0000 7fa425ac0640 1 --2- 172.21.15.83:0/3929029838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa40c0039c0 0x7fa40c003de0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fa410009ae0 tx=0x7fa42006d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf301e5e43a0b70 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:31.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.819+0000 7fa424abe640 1 -- 172.21.15.83:0/3929029838 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4100096b0 con 0x7fa40c0039c0 2024-04-17T10:46:31.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.819+0000 7fa424abe640 1 -- 172.21.15.83:0/3929029838 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4100a6490 con 0x7fa40c0039c0 2024-04-17T10:46:31.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.819+0000 7fa424abe640 1 -- 172.21.15.83:0/3929029838 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4100a6880 con 0x7fa40c0039c0 2024-04-17T10:46:31.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.819+0000 7fa427d24640 1 -- 172.21.15.83:0/3929029838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa40c0039c0 msgr2=0x7fa40c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:31.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.819+0000 7fa427d24640 1 --2- 172.21.15.83:0/3929029838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa40c0039c0 0x7fa40c003de0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fa410009ae0 tx=0x7fa42006d3b0 comp rx=0 tx=0).stop 2024-04-17T10:46:31.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.819+0000 7fa4262c1640 1 -- 172.21.15.83:0/3929029838 reap_dead start 2024-04-17T10:46:31.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.819+0000 7fa427d24640 1 -- 172.21.15.83:0/3929029838 shutdown_connections 2024-04-17T10:46:31.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.819+0000 7fa427d24640 1 -- 172.21.15.83:0/3929029838 >> 172.21.15.83:0/3929029838 conn(0x7fa42006abe0 msgr2=0x7fa42006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:31.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.819+0000 7fa427d24640 1 -- 172.21.15.83:0/3929029838 shutdown_connections 2024-04-17T10:46:31.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.819+0000 7fa427d24640 1 -- 172.21.15.83:0/3929029838 wait complete. 2024-04-17T10:46:31.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.819+0000 7fa427d24640 1 Processor -- start 2024-04-17T10:46:31.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.819+0000 7fa427d24640 1 -- start start 2024-04-17T10:46:31.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.819+0000 7fa427d24640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa40c0039c0 0x7fa420161e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:31.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.819+0000 7fa427d24640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa42014fdc0 0x7fa420160a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:31.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.819+0000 7fa427d24640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa420150220 0x7fa420160f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:31.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.819+0000 7fa427d24640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42006de20 con 0x7fa42014fdc0 2024-04-17T10:46:31.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.819+0000 7fa427d24640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42006dca0 con 0x7fa40c0039c0 2024-04-17T10:46:31.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.819+0000 7fa427d24640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa42015cd40 con 0x7fa420150220 2024-04-17T10:46:31.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.819+0000 7fa4252bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa42014fdc0 0x7fa420160a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:31.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.819+0000 7fa4252bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa42014fdc0 0x7fa420160a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32802/0 (socket says 172.21.15.83:32802) 2024-04-17T10:46:31.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.819+0000 7fa4252bf640 1 -- 172.21.15.83:0/262545505 learned_addr learned my addr 172.21.15.83:0/262545505 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:31.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.819+0000 7fa425ac0640 1 --2- 172.21.15.83:0/262545505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa40c0039c0 0x7fa420161e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:31.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.819+0000 7fa4262c1640 1 --2- 172.21.15.83:0/262545505 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa420150220 0x7fa420160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:31.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.823+0000 7fa4252bf640 1 -- 172.21.15.83:0/262545505 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa420150220 msgr2=0x7fa420160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:31.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.823+0000 7fa4252bf640 1 --2- 172.21.15.83:0/262545505 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa420150220 0x7fa420160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:31.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.823+0000 7fa4252bf640 1 -- 172.21.15.83:0/262545505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa40c0039c0 msgr2=0x7fa420161e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:31.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.823+0000 7fa4252bf640 1 --2- 172.21.15.83:0/262545505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa40c0039c0 0x7fa420161e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:31.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.823+0000 7fa4252bf640 1 -- 172.21.15.83:0/262545505 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa42015aad0 con 0x7fa42014fdc0 2024-04-17T10:46:31.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.823+0000 7fa4252bf640 1 --2- 172.21.15.83:0/262545505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa42014fdc0 0x7fa420160a20 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fa41c00f380 tx=0x7fa41c00f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:31.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.823+0000 7fa4167fc640 1 -- 172.21.15.83:0/262545505 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c01c070 con 0x7fa42014fdc0 2024-04-17T10:46:31.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.823+0000 7fa4167fc640 1 -- 172.21.15.83:0/262545505 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41c017620 con 0x7fa42014fdc0 2024-04-17T10:46:31.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.823+0000 7fa427d24640 1 -- 172.21.15.83:0/262545505 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa42015adb0 con 0x7fa42014fdc0 2024-04-17T10:46:31.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.823+0000 7fa4167fc640 1 -- 172.21.15.83:0/262545505 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c017a10 con 0x7fa42014fdc0 2024-04-17T10:46:31.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.823+0000 7fa427d24640 1 -- 172.21.15.83:0/262545505 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa420159d80 con 0x7fa42014fdc0 2024-04-17T10:46:31.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.823+0000 7fa427d24640 1 -- 172.21.15.83:0/262545505 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa420074f90 con 0x7fa42014fdc0 2024-04-17T10:46:31.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.823+0000 7fa4167fc640 1 -- 172.21.15.83:0/262545505 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa41c01f070 con 0x7fa42014fdc0 2024-04-17T10:46:31.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.823+0000 7fa4167fc640 1 --2- 172.21.15.83:0/262545505 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa404066f60 0x7fa404069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:31.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.823+0000 7fa4167fc640 1 -- 172.21.15.83:0/262545505 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa41c084fd0 con 0x7fa42014fdc0 2024-04-17T10:46:31.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.823+0000 7fa425ac0640 1 --2- 172.21.15.83:0/262545505 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa404066f60 0x7fa404069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:31.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.823+0000 7fa425ac0640 1 --2- 172.21.15.83:0/262545505 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa404066f60 0x7fa404069420 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa4100ac000 tx=0x7fa4100b1040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:31.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:31.827+0000 7fa4167fc640 1 -- 172.21.15.83:0/262545505 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa41c012070 con 0x7fa42014fdc0 2024-04-17T10:46:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.023+0000 7fa427d24640 1 -- 172.21.15.83:0/262545505 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa42006afc0 con 0x7fa42014fdc0 2024-04-17T10:46:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.023+0000 7fa4167fc640 1 -- 172.21.15.83:0/262545505 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+3702 (secure 0 0 0) 0x7fa41c055d50 con 0x7fa42014fdc0 2024-04-17T10:46:32.020 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:46:32.020 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":51,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5258,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2259892875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2259892875},{"type":"v1","addr":"172.21.15.167:6835","nonce":2259892875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5378,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2006521035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2006521035},{"type":"v1","addr":"172.21.15.167:6833","nonce":2006521035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":7095,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/3769102527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3769102527},{"type":"v1","addr":"172.21.15.83:6836","nonce":3769102527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":7098,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/395594830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":395594830},{"type":"v1","addr":"172.21.15.83:6837","nonce":395594830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42}],"filesystems":[]} 2024-04-17T10:46:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 51 2024-04-17T10:46:32.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.027+0000 7fa427d24640 1 -- 172.21.15.83:0/262545505 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa404066f60 msgr2=0x7fa404069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:32.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.027+0000 7fa427d24640 1 --2- 172.21.15.83:0/262545505 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa404066f60 0x7fa404069420 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa4100ac000 tx=0x7fa4100b1040 comp rx=0 tx=0).stop 2024-04-17T10:46:32.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.027+0000 7fa427d24640 1 -- 172.21.15.83:0/262545505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa42014fdc0 msgr2=0x7fa420160a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:32.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.027+0000 7fa427d24640 1 --2- 172.21.15.83:0/262545505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa42014fdc0 0x7fa420160a20 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fa41c00f380 tx=0x7fa41c00f840 comp rx=0 tx=0).stop 2024-04-17T10:46:32.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.027+0000 7fa427d24640 1 -- 172.21.15.83:0/262545505 shutdown_connections 2024-04-17T10:46:32.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.027+0000 7fa427d24640 1 --2- 172.21.15.83:0/262545505 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa404066f60 0x7fa404069420 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:32.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.027+0000 7fa427d24640 1 --2- 172.21.15.83:0/262545505 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa420150220 0x7fa420160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:32.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.027+0000 7fa427d24640 1 --2- 172.21.15.83:0/262545505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa42014fdc0 0x7fa420160a20 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:32.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.027+0000 7fa427d24640 1 --2- 172.21.15.83:0/262545505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa40c0039c0 0x7fa420161e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:32.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.027+0000 7fa427d24640 1 -- 172.21.15.83:0/262545505 >> 172.21.15.83:0/262545505 conn(0x7fa42006abe0 msgr2=0x7fa4201575f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:32.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.027+0000 7fa427d24640 1 -- 172.21.15.83:0/262545505 shutdown_connections 2024-04-17T10:46:32.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.027+0000 7fa427d24640 1 -- 172.21.15.83:0/262545505 wait complete. 2024-04-17T10:46:32.039 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-17T10:46:32.040 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-17T10:46:32.040 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-17T10:46:32.040 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:46:32.040 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-17T10:46:32.040 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-17T10:46:32.040 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-17T10:46:32.041 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:46:32.041 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-17T10:46:32.041 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-17T10:46:32.041 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-17T10:46:32.041 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:46:32.041 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-17T10:46:32.041 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-17T10:46:32.042 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-17T10:46:32.042 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:46:32.051 INFO:tasks.ceph.mds.a:Stopped 2024-04-17T10:46:32.051 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-17T10:46:32.054 INFO:tasks.ceph.mds.c:Stopped 2024-04-17T10:46:32.054 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-17T10:46:32.057 INFO:tasks.ceph.mds.b:Stopped 2024-04-17T10:46:32.057 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-17T10:46:32.058 INFO:tasks.ceph.mds.d:Stopped 2024-04-17T10:46:32.058 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-17T10:46:32.098 INFO:tasks.ceph.mds.a:Started 2024-04-17T10:46:32.099 DEBUG:teuthology.parallel:result is None 2024-04-17T10:46:32.099 INFO:tasks.ceph.mds.c:Started 2024-04-17T10:46:32.099 DEBUG:teuthology.parallel:result is None 2024-04-17T10:46:32.106 INFO:tasks.ceph.mds.b:Started 2024-04-17T10:46:32.106 DEBUG:teuthology.parallel:result is None 2024-04-17T10:46:32.107 INFO:tasks.ceph.mds.d:Started 2024-04-17T10:46:32.108 DEBUG:teuthology.parallel:result is None 2024-04-17T10:46:32.108 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-17T10:46:32.162 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-04-17T10:46:32.162 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-04-17T10:46:32.182 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2024-04-17T10:46:32.183 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2024-04-17T10:46:32.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.231+0000 7fe0eec09640 1 Processor -- start 2024-04-17T10:46:32.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.231+0000 7fe0eec09640 1 -- start start 2024-04-17T10:46:32.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.235+0000 7fe0eec09640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe0e80b3860 0x7fe0e806abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:32.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.235+0000 7fe0eec09640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0e815e180 con 0x7fe0e806b120 2024-04-17T10:46:32.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.235+0000 7fe0eec09640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0e806d340 con 0x7fe0e8147570 2024-04-17T10:46:32.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.235+0000 7fe0eec09640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0e815e420 con 0x7fe0e80b3860 2024-04-17T10:46:32.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.235+0000 7fe0ed1a6640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe0e806b120 0x7fe0e80d0480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50730/0 (socket says 172.21.15.83:50730) 2024-04-17T10:46:32.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.235+0000 7fe0ed1a6640 1 -- 172.21.15.83:0/1150726994 learned_addr learned my addr 172.21.15.83:0/1150726994 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:46:32.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.235+0000 7fe0e7fff640 1 --2- 172.21.15.83:0/1150726994 >> v2:172.21.15.167:3300/0 conn(0x7fe0e80b3860 0x7fe0e806abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:32.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.235+0000 7fe0e77fe640 1 -- 172.21.15.83:0/1150726994 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3831899723 0 0) 0x7fe0e815e180 con 0x7fe0e806b120 2024-04-17T10:46:32.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.235+0000 7fe0e77fe640 1 -- 172.21.15.83:0/1150726994 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0d8003530 con 0x7fe0e806b120 2024-04-17T10:46:32.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.235+0000 7fe0e77fe640 1 -- 172.21.15.83:0/1150726994 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 321587810 0 0) 0x7fe0e806d340 con 0x7fe0e8147570 2024-04-17T10:46:32.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.235+0000 7fe0e77fe640 1 -- 172.21.15.83:0/1150726994 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0e815e180 con 0x7fe0e8147570 2024-04-17T10:46:32.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.235+0000 7fe0e7fff640 1 -- 172.21.15.83:0/1150726994 >> v1:172.21.15.167:6789/0 conn(0x7fe0e8147570 legacy=0x7fe0e806ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:32.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.235+0000 7fe0e77fe640 1 -- 172.21.15.83:0/1150726994 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2051112300 0 0) 0x7fe0d8003530 con 0x7fe0e806b120 2024-04-17T10:46:32.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.235+0000 7fe0e7fff640 1 -- 172.21.15.83:0/1150726994 >> v1:172.21.15.83:6789/0 conn(0x7fe0e806b120 legacy=0x7fe0e80d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:32.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.235+0000 7fe0e7fff640 1 -- 172.21.15.83:0/1150726994 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e815e700 con 0x7fe0e80b3860 2024-04-17T10:46:32.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.235+0000 7fe0e7fff640 1 --2- 172.21.15.83:0/1150726994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0e80b3860 0x7fe0e806abe0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe0d40097c0 tx=0x7fe0d401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ce0af39f29e0619 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:32.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.235+0000 7fe0e77fe640 1 -- 172.21.15.83:0/1150726994 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d40b0070 con 0x7fe0e80b3860 2024-04-17T10:46:32.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.235+0000 7fe0e77fe640 1 -- 172.21.15.83:0/1150726994 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d40ab400 con 0x7fe0e80b3860 2024-04-17T10:46:32.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.235+0000 7fe0e77fe640 1 -- 172.21.15.83:0/1150726994 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0001a00 con 0x7fe0e80b3860 2024-04-17T10:46:32.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.239+0000 7fe0eec09640 1 -- 172.21.15.83:0/1150726994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0e80b3860 msgr2=0x7fe0e806abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:32.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.239+0000 7fe0eec09640 1 --2- 172.21.15.83:0/1150726994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0e80b3860 0x7fe0e806abe0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe0d40097c0 tx=0x7fe0d401c3c0 comp rx=0 tx=0).stop 2024-04-17T10:46:32.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.239+0000 7fe0eec09640 1 -- 172.21.15.83:0/1150726994 shutdown_connections 2024-04-17T10:46:32.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.239+0000 7fe0eec09640 1 --2- 172.21.15.83:0/1150726994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0e80b3860 0x7fe0e806abe0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:32.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.239+0000 7fe0eec09640 1 -- 172.21.15.83:0/1150726994 >> 172.21.15.83:0/1150726994 conn(0x7fe0e806bcf0 msgr2=0x7fe0e806c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:32.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.239+0000 7fe0eec09640 1 -- 172.21.15.83:0/1150726994 shutdown_connections 2024-04-17T10:46:32.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.239+0000 7fe0eec09640 1 -- 172.21.15.83:0/1150726994 wait complete. 2024-04-17T10:46:32.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.239+0000 7fe0eec09640 1 Processor -- start 2024-04-17T10:46:32.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.239+0000 7fe0eec09640 1 -- start start 2024-04-17T10:46:32.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.239+0000 7fe0eec09640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e806b120 0x7fe0e813e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:32.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.239+0000 7fe0eec09640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0e80b3860 0x7fe0e813f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:32.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.239+0000 7fe0eec09640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0e8147570 0x7fe0e81fa400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:32.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.239+0000 7fe0eec09640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e816bda0 con 0x7fe0e806b120 2024-04-17T10:46:32.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.239+0000 7fe0eec09640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e816bc20 con 0x7fe0e80b3860 2024-04-17T10:46:32.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.239+0000 7fe0eec09640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e816bf20 con 0x7fe0e8147570 2024-04-17T10:46:32.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.239+0000 7fe0ec9a5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e806b120 0x7fe0e813e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:32.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.239+0000 7fe0ec9a5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e806b120 0x7fe0e813e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32828/0 (socket says 172.21.15.83:32828) 2024-04-17T10:46:32.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.239+0000 7fe0ec9a5640 1 -- 172.21.15.83:0/3033076705 learned_addr learned my addr 172.21.15.83:0/3033076705 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:32.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.239+0000 7fe0e7fff640 1 --2- 172.21.15.83:0/3033076705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0e80b3860 0x7fe0e813f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:32.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.239+0000 7fe0ed1a6640 1 --2- 172.21.15.83:0/3033076705 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0e8147570 0x7fe0e81fa400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:32.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.239+0000 7fe0ec9a5640 1 -- 172.21.15.83:0/3033076705 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0e8147570 msgr2=0x7fe0e81fa400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:32.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.239+0000 7fe0ec9a5640 1 --2- 172.21.15.83:0/3033076705 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0e8147570 0x7fe0e81fa400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:32.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.239+0000 7fe0ec9a5640 1 -- 172.21.15.83:0/3033076705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0e80b3860 msgr2=0x7fe0e813f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:32.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.239+0000 7fe0ec9a5640 1 --2- 172.21.15.83:0/3033076705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0e80b3860 0x7fe0e813f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:32.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.239+0000 7fe0ec9a5640 1 -- 172.21.15.83:0/3033076705 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e81402c0 con 0x7fe0e806b120 2024-04-17T10:46:32.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.239+0000 7fe0ec9a5640 1 --2- 172.21.15.83:0/3033076705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e806b120 0x7fe0e813e5b0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc00c490 tx=0x7fe0dc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:32.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.239+0000 7fe0e57fa640 1 -- 172.21.15.83:0/3033076705 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc0052d0 con 0x7fe0e806b120 2024-04-17T10:46:32.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.239+0000 7fe0e57fa640 1 -- 172.21.15.83:0/3033076705 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0dc005470 con 0x7fe0e806b120 2024-04-17T10:46:32.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.239+0000 7fe0eec09640 1 -- 172.21.15.83:0/3033076705 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0e81405a0 con 0x7fe0e806b120 2024-04-17T10:46:32.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.239+0000 7fe0e57fa640 1 -- 172.21.15.83:0/3033076705 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc012870 con 0x7fe0e806b120 2024-04-17T10:46:32.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.239+0000 7fe0eec09640 1 -- 172.21.15.83:0/3033076705 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0e813e090 con 0x7fe0e806b120 2024-04-17T10:46:32.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.243+0000 7fe0eec09640 1 -- 172.21.15.83:0/3033076705 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0e8074f90 con 0x7fe0e806b120 2024-04-17T10:46:32.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.243+0000 7fe0e57fa640 1 -- 172.21.15.83:0/3033076705 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe0dc012a10 con 0x7fe0e806b120 2024-04-17T10:46:32.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.243+0000 7fe0e57fa640 1 --2- 172.21.15.83:0/3033076705 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe0cc066f60 0x7fe0cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:32.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.243+0000 7fe0e7fff640 1 --2- 172.21.15.83:0/3033076705 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe0cc066f60 0x7fe0cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:32.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.243+0000 7fe0e57fa640 1 -- 172.21.15.83:0/3033076705 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe0dc0849d0 con 0x7fe0e806b120 2024-04-17T10:46:32.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.243+0000 7fe0e7fff640 1 --2- 172.21.15.83:0/3033076705 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe0cc066f60 0x7fe0cc069420 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe0d40ad000 tx=0x7fe0d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:32.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.247+0000 7fe0e57fa640 1 -- 172.21.15.83:0/3033076705 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe0dc055ce0 con 0x7fe0e806b120 2024-04-17T10:46:32.260 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-04-17T10:46:32.260 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-17T10:46:32.260 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-17T10:46:32.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:32.415+0000 7fe0eec09640 1 -- 172.21.15.83:0/3033076705 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe0e806ea60 con 0x7fe0e806b120 2024-04-17T10:46:33.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.175+0000 7fe0e57fa640 1 -- 172.21.15.83:0/3033076705 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v83) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe0dc0555e0 con 0x7fe0e806b120 2024-04-17T10:46:33.172 INFO:teuthology.orchestra.run.smithi083.stderr: removed all blocklist entries 2024-04-17T10:46:33.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.179+0000 7fe0eec09640 1 -- 172.21.15.83:0/3033076705 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe0cc066f60 msgr2=0x7fe0cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:33.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.179+0000 7fe0eec09640 1 --2- 172.21.15.83:0/3033076705 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe0cc066f60 0x7fe0cc069420 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe0d40ad000 tx=0x7fe0d4008040 comp rx=0 tx=0).stop 2024-04-17T10:46:33.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.179+0000 7fe0eec09640 1 -- 172.21.15.83:0/3033076705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e806b120 msgr2=0x7fe0e813e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:33.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.179+0000 7fe0eec09640 1 --2- 172.21.15.83:0/3033076705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e806b120 0x7fe0e813e5b0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc00c490 tx=0x7fe0dc00c950 comp rx=0 tx=0).stop 2024-04-17T10:46:33.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.179+0000 7fe0eec09640 1 -- 172.21.15.83:0/3033076705 shutdown_connections 2024-04-17T10:46:33.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.179+0000 7fe0eec09640 1 --2- 172.21.15.83:0/3033076705 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe0cc066f60 0x7fe0cc069420 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:33.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.179+0000 7fe0eec09640 1 --2- 172.21.15.83:0/3033076705 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0e8147570 0x7fe0e81fa400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:33.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.179+0000 7fe0eec09640 1 --2- 172.21.15.83:0/3033076705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0e80b3860 0x7fe0e813f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:33.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.179+0000 7fe0eec09640 1 --2- 172.21.15.83:0/3033076705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e806b120 0x7fe0e813e5b0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:33.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.179+0000 7fe0eec09640 1 -- 172.21.15.83:0/3033076705 >> 172.21.15.83:0/3033076705 conn(0x7fe0e806bcf0 msgr2=0x7fe0e806e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:33.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.179+0000 7fe0eec09640 1 -- 172.21.15.83:0/3033076705 shutdown_connections 2024-04-17T10:46:33.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.179+0000 7fe0eec09640 1 -- 172.21.15.83:0/3033076705 wait complete. 2024-04-17T10:46:33.198 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-17T10:46:33.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.315+0000 7f0e6d43f640 1 Processor -- start 2024-04-17T10:46:33.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.315+0000 7f0e6d43f640 1 -- start start 2024-04-17T10:46:33.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.319+0000 7f0e6d43f640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0e680ce980 0x7f0e680b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:33.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.319+0000 7f0e6d43f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0e680cee00 0x7f0e680b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:33.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.319+0000 7f0e6d43f640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e68162a00 con 0x7f0e6815dfa0 2024-04-17T10:46:33.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.319+0000 7f0e6d43f640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e68161e80 con 0x7f0e680cee00 2024-04-17T10:46:33.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.319+0000 7f0e6d43f640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e68162160 con 0x7f0e680ce980 2024-04-17T10:46:33.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.319+0000 7f0e677fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0e680cee00 0x7f0e680b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:33.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.319+0000 7f0e677fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0e680cee00 0x7f0e680b5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32834/0 (socket says 172.21.15.83:32834) 2024-04-17T10:46:33.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.319+0000 7f0e677fe640 1 -- 172.21.15.83:0/3521730370 learned_addr learned my addr 172.21.15.83:0/3521730370 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:33.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.319+0000 7f0e667fc640 1 --2- 172.21.15.83:0/3521730370 >> v2:172.21.15.167:3300/0 conn(0x7f0e680ce980 0x7f0e680b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:33.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.319+0000 7f0e677fe640 1 -- 172.21.15.83:0/3521730370 >> v2:172.21.15.167:3300/0 conn(0x7f0e680ce980 msgr2=0x7f0e680b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:33.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.319+0000 7f0e677fe640 1 --2- 172.21.15.83:0/3521730370 >> v2:172.21.15.167:3300/0 conn(0x7f0e680ce980 0x7f0e680b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:33.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.319+0000 7f0e677fe640 1 -- 172.21.15.83:0/3521730370 >> v1:172.21.15.167:6789/0 conn(0x7f0e6815dfa0 legacy=0x7f0e680b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:33.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.319+0000 7f0e677fe640 1 -- 172.21.15.83:0/3521730370 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e68162440 con 0x7f0e680cee00 2024-04-17T10:46:33.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e677fe640 1 --2- 172.21.15.83:0/3521730370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e680cee00 0x7f0e680b5970 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f0e6808c8b0 tx=0x7f0e6001c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f288dd96250010b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:33.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e65ffb640 1 -- 172.21.15.83:0/3521730370 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e600a5750 con 0x7f0e680cee00 2024-04-17T10:46:33.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e65ffb640 1 -- 172.21.15.83:0/3521730370 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e600a58f0 con 0x7f0e680cee00 2024-04-17T10:46:33.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e65ffb640 1 -- 172.21.15.83:0/3521730370 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e600a5cc0 con 0x7f0e680cee00 2024-04-17T10:46:33.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e6d43f640 1 -- 172.21.15.83:0/3521730370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e680cee00 msgr2=0x7f0e680b5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:33.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e6d43f640 1 --2- 172.21.15.83:0/3521730370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e680cee00 0x7f0e680b5970 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f0e6808c8b0 tx=0x7f0e6001c8f0 comp rx=0 tx=0).stop 2024-04-17T10:46:33.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e6d43f640 1 -- 172.21.15.83:0/3521730370 shutdown_connections 2024-04-17T10:46:33.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e6d43f640 1 --2- 172.21.15.83:0/3521730370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e680cee00 0x7f0e680b5970 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:33.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e6d43f640 1 --2- 172.21.15.83:0/3521730370 >> v2:172.21.15.167:3300/0 conn(0x7f0e680ce980 0x7f0e680b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:33.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e6d43f640 1 -- 172.21.15.83:0/3521730370 >> 172.21.15.83:0/3521730370 conn(0x7f0e680d0600 msgr2=0x7f0e680d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:33.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e6d43f640 1 -- 172.21.15.83:0/3521730370 shutdown_connections 2024-04-17T10:46:33.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e6d43f640 1 -- 172.21.15.83:0/3521730370 wait complete. 2024-04-17T10:46:33.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e6d43f640 1 Processor -- start 2024-04-17T10:46:33.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e6d43f640 1 -- start start 2024-04-17T10:46:33.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e6d43f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e680ce980 0x7f0e680c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:33.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e6d43f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e680cee00 0x7f0e680c5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:33.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e6d43f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e6815dfa0 0x7f0e6814c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:33.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e6d43f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e680d26c0 con 0x7f0e680ce980 2024-04-17T10:46:33.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e6d43f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e680d2540 con 0x7f0e680cee00 2024-04-17T10:46:33.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e6d43f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e680d2840 con 0x7f0e6815dfa0 2024-04-17T10:46:33.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e66ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e680ce980 0x7f0e680c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:33.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e66ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e680ce980 0x7f0e680c9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32840/0 (socket says 172.21.15.83:32840) 2024-04-17T10:46:33.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e66ffd640 1 -- 172.21.15.83:0/1376682221 learned_addr learned my addr 172.21.15.83:0/1376682221 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:33.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e667fc640 1 --2- 172.21.15.83:0/1376682221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e680cee00 0x7f0e680c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:33.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e66ffd640 1 -- 172.21.15.83:0/1376682221 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e6815dfa0 msgr2=0x7f0e6814c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:33.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e66ffd640 1 --2- 172.21.15.83:0/1376682221 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e6815dfa0 0x7f0e6814c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:33.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e66ffd640 1 -- 172.21.15.83:0/1376682221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e680cee00 msgr2=0x7f0e680c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:33.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e66ffd640 1 --2- 172.21.15.83:0/1376682221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e680cee00 0x7f0e680c5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:33.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e66ffd640 1 -- 172.21.15.83:0/1376682221 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e680c8440 con 0x7f0e680ce980 2024-04-17T10:46:33.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e667fc640 1 --2- 172.21.15.83:0/1376682221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e680cee00 0x7f0e680c5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:33.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e66ffd640 1 --2- 172.21.15.83:0/1376682221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e680ce980 0x7f0e680c9ef0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800b440 tx=0x7f0e5800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:33.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e4f7fe640 1 -- 172.21.15.83:0/1376682221 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e58015020 con 0x7f0e680ce980 2024-04-17T10:46:33.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e4f7fe640 1 -- 172.21.15.83:0/1376682221 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e58011890 con 0x7f0e680ce980 2024-04-17T10:46:33.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e6d43f640 1 -- 172.21.15.83:0/1376682221 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e680c8720 con 0x7f0e680ce980 2024-04-17T10:46:33.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e4f7fe640 1 -- 172.21.15.83:0/1376682221 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e58011c80 con 0x7f0e680ce980 2024-04-17T10:46:33.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.323+0000 7f0e6d43f640 1 -- 172.21.15.83:0/1376682221 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e680c6ad0 con 0x7f0e680ce980 2024-04-17T10:46:33.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.327+0000 7f0e6d43f640 1 -- 172.21.15.83:0/1376682221 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e68074f90 con 0x7f0e680ce980 2024-04-17T10:46:33.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.327+0000 7f0e4f7fe640 1 -- 172.21.15.83:0/1376682221 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0e58018020 con 0x7f0e680ce980 2024-04-17T10:46:33.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.327+0000 7f0e4f7fe640 1 --2- 172.21.15.83:0/1376682221 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0e54066f90 0x7f0e54069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:33.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.327+0000 7f0e667fc640 1 --2- 172.21.15.83:0/1376682221 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0e54066f90 0x7f0e54069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:33.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.327+0000 7f0e4f7fe640 1 -- 172.21.15.83:0/1376682221 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0e58084a70 con 0x7f0e680ce980 2024-04-17T10:46:33.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.327+0000 7f0e667fc640 1 --2- 172.21.15.83:0/1376682221 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0e54066f90 0x7f0e54069450 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0e68147b60 tx=0x7f0e5c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:33.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.331+0000 7f0e4f7fe640 1 -- 172.21.15.83:0/1376682221 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0e5801f030 con 0x7f0e680ce980 2024-04-17T10:46:33.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.543+0000 7f0e6d43f640 1 -- 172.21.15.83:0/1376682221 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0e68159930 con 0x7f0e680ce980 2024-04-17T10:46:33.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.543+0000 7f0e4f7fe640 1 -- 172.21.15.83:0/1376682221 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f0e5800d040 con 0x7f0e680ce980 2024-04-17T10:46:33.540 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:46:33.540 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:46:33.540 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_dump": [ 2024-04-17T10:46:33.540 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:46:33.540 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.a", 2024-04-17T10:46:33.540 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mfpxsNxAANpL7rcWevXwd2KMftSQnEQ==", 2024-04-17T10:46:33.540 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:46:33.540 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:46:33.540 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:46:33.540 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:46:33.540 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:46:33.540 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:46:33.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:33.541 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:46:33.541 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.b", 2024-04-17T10:46:33.541 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mXlI9ChAAYdcrGVmtYdtWWgCLVmpTTg==", 2024-04-17T10:46:33.541 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:46:33.541 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:46:33.541 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:46:33.541 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:46:33.541 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:46:33.541 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:46:33.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:33.541 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:46:33.541 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.c", 2024-04-17T10:46:33.542 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mupo3BBAAyKH801vpFLXYs9mfzSlT0w==", 2024-04-17T10:46:33.542 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:46:33.542 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:46:33.542 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:46:33.542 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:46:33.542 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:46:33.542 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:46:33.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:33.542 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:46:33.542 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.d", 2024-04-17T10:46:33.542 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mbm++EhAAIBMOtQoD+kPeMqR6gK5ctw==", 2024-04-17T10:46:33.542 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:46:33.542 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:46:33.542 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:46:33.543 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:46:33.543 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:46:33.543 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:46:33.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:33.543 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:46:33.543 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.0", 2024-04-17T10:46:33.543 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBXph9m1HWhIRAAOq+9nGq7OGonUCE2aeiAzQ==", 2024-04-17T10:46:33.543 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:46:33.543 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:46:33.543 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:46:33.543 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:46:33.543 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:46:33.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:33.543 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:46:33.544 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.1", 2024-04-17T10:46:33.544 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBaph9mlMmtJRAAXHLMjXfm/+WjIFURLkbAfA==", 2024-04-17T10:46:33.544 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:46:33.544 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:46:33.544 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:46:33.544 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:46:33.544 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:46:33.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:33.544 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:46:33.544 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.2", 2024-04-17T10:46:33.544 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBdph9mBsNPLRAAtR5hwGdDkjLDLS19QUDDYQ==", 2024-04-17T10:46:33.544 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:46:33.544 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:46:33.545 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:46:33.545 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:46:33.545 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:46:33.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:33.545 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:46:33.545 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.3", 2024-04-17T10:46:33.545 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBgph9mZ4EwNRAAIyL7e0EK1523dPXmbOOkzw==", 2024-04-17T10:46:33.545 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:46:33.545 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:46:33.545 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:46:33.545 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:46:33.545 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:46:33.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:33.545 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:46:33.546 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.4", 2024-04-17T10:46:33.546 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBmph9mphAsMhAAtF4uV28Nb66PNjuugB12Hw==", 2024-04-17T10:46:33.546 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:46:33.546 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:46:33.546 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:46:33.546 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:46:33.546 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:46:33.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:33.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:46:33.546 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.5", 2024-04-17T10:46:33.546 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBpph9mHJQNNxAA72eOFYFbhakQac1JKRNjzw==", 2024-04-17T10:46:33.547 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:46:33.547 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:46:33.548 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:46:33.548 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:46:33.548 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:46:33.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:33.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:46:33.548 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.6", 2024-04-17T10:46:33.548 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBtph9mb6RKAxAAoVX6Cp7l7j3Lm1ES2Sq+fQ==", 2024-04-17T10:46:33.548 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:46:33.548 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:46:33.548 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:46:33.548 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:46:33.548 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:46:33.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:33.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:46:33.549 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.7", 2024-04-17T10:46:33.549 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBwph9mdrdrBxAApDwjml2ahsLbRol4uL/BxA==", 2024-04-17T10:46:33.549 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:46:33.549 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:46:33.549 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:46:33.549 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:46:33.549 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:46:33.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:33.549 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:46:33.549 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.0", 2024-04-17T10:46:33.549 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mHGmXGBAAZF+JiAG9YrFtbW+dCy2IEQ==", 2024-04-17T10:46:33.549 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:46:33.549 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:46:33.549 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow r", 2024-04-17T10:46:33.550 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-17T10:46:33.550 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:46:33.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:33.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:46:33.550 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.1", 2024-04-17T10:46:33.550 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mB7wgIBAABdavAo/uzVuSjfkkcYtqJg==", 2024-04-17T10:46:33.550 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:46:33.550 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:46:33.550 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-17T10:46:33.550 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-17T10:46:33.550 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-17T10:46:33.550 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:46:33.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:33.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:46:33.551 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.2", 2024-04-17T10:46:33.551 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9m+wKgHRAAe8IUFx34ItLpsaDqSYyEgA==", 2024-04-17T10:46:33.551 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:46:33.551 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:46:33.551 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-17T10:46:33.551 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-17T10:46:33.551 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-17T10:46:33.551 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:46:33.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:33.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:46:33.551 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.3", 2024-04-17T10:46:33.551 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mfJ0yJRAAHH7rRh/BUIujTT04AF2l0w==", 2024-04-17T10:46:33.551 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:46:33.552 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:46:33.552 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-17T10:46:33.552 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-17T10:46:33.552 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-17T10:46:33.552 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:46:33.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:33.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:46:33.552 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.admin", 2024-04-17T10:46:33.552 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mjJ3GIBAA/2qaCbRiLJ5X1cv3ZGdfoA==", 2024-04-17T10:46:33.552 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:46:33.552 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-17T10:46:33.553 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:46:33.553 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:46:33.553 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:46:33.553 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:46:33.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:33.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:46:33.553 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.x", 2024-04-17T10:46:33.553 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mMFptNBAA/G0zEfMZTCy9Iox+MwHG3Q==", 2024-04-17T10:46:33.553 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:46:33.553 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-17T10:46:33.553 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-04-17T10:46:33.553 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:46:33.553 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:46:33.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:33.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:46:33.554 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.y", 2024-04-17T10:46:33.554 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mjFbRMRAAjY2d8POBzp8hEp9GSO0ylQ==", 2024-04-17T10:46:33.554 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:46:33.554 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-17T10:46:33.554 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-04-17T10:46:33.554 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:46:33.554 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:46:33.554 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:46:33.554 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-17T10:46:33.554 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:46:33.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.547+0000 7f0e6d43f640 1 -- 172.21.15.83:0/1376682221 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0e54066f90 msgr2=0x7f0e54069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:33.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.547+0000 7f0e6d43f640 1 --2- 172.21.15.83:0/1376682221 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0e54066f90 0x7f0e54069450 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0e68147b60 tx=0x7f0e5c008040 comp rx=0 tx=0).stop 2024-04-17T10:46:33.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.547+0000 7f0e6d43f640 1 -- 172.21.15.83:0/1376682221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e680ce980 msgr2=0x7f0e680c9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:33.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.547+0000 7f0e6d43f640 1 --2- 172.21.15.83:0/1376682221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e680ce980 0x7f0e680c9ef0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800b440 tx=0x7f0e5800b900 comp rx=0 tx=0).stop 2024-04-17T10:46:33.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.547+0000 7f0e6d43f640 1 -- 172.21.15.83:0/1376682221 shutdown_connections 2024-04-17T10:46:33.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.547+0000 7f0e6d43f640 1 --2- 172.21.15.83:0/1376682221 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0e54066f90 0x7f0e54069450 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:33.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.547+0000 7f0e6d43f640 1 --2- 172.21.15.83:0/1376682221 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e6815dfa0 0x7f0e6814c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:33.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.547+0000 7f0e6d43f640 1 --2- 172.21.15.83:0/1376682221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e680cee00 0x7f0e680c5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:33.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.547+0000 7f0e6d43f640 1 --2- 172.21.15.83:0/1376682221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e680ce980 0x7f0e680c9ef0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:33.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.547+0000 7f0e6d43f640 1 -- 172.21.15.83:0/1376682221 >> 172.21.15.83:0/1376682221 conn(0x7f0e680d0600 msgr2=0x7f0e681582c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:33.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.547+0000 7f0e6d43f640 1 -- 172.21.15.83:0/1376682221 shutdown_connections 2024-04-17T10:46:33.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.547+0000 7f0e6d43f640 1 -- 172.21.15.83:0/1376682221 wait complete. 2024-04-17T10:46:33.558 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:46:33.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.667+0000 7ff628078640 1 Processor -- start 2024-04-17T10:46:33.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.667+0000 7ff628078640 1 -- start start 2024-04-17T10:46:33.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.667+0000 7ff628078640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff62015df70 0x7ff6200b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:33.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.667+0000 7ff628078640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff6200cf850 0x7ff6200cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:33.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.667+0000 7ff628078640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6200b2ca0 con 0x7ff6200b3e50 2024-04-17T10:46:33.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.667+0000 7ff628078640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff62015e350 con 0x7ff6200cf850 2024-04-17T10:46:33.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.667+0000 7ff628078640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff62006c9a0 con 0x7ff62015df70 2024-04-17T10:46:33.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.667+0000 7ff626615640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff6200cf850 0x7ff6200cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:33.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.667+0000 7ff626615640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff6200cf850 0x7ff6200cfcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32844/0 (socket says 172.21.15.83:32844) 2024-04-17T10:46:33.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.667+0000 7ff626615640 1 -- 172.21.15.83:0/20085132 learned_addr learned my addr 172.21.15.83:0/20085132 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:33.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.667+0000 7ff625e14640 1 --2- 172.21.15.83:0/20085132 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff62015df70 0x7ff6200b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:33.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.667+0000 7ff626615640 1 -- 172.21.15.83:0/20085132 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff62015df70 msgr2=0x7ff6200b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:33.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.667+0000 7ff626615640 1 --2- 172.21.15.83:0/20085132 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff62015df70 0x7ff6200b11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:33.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.667+0000 7ff626615640 1 -- 172.21.15.83:0/20085132 >> v1:172.21.15.167:6789/0 conn(0x7ff6200b3e50 legacy=0x7ff6200b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:33.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.667+0000 7ff626615640 1 -- 172.21.15.83:0/20085132 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6200b2f80 con 0x7ff6200cf850 2024-04-17T10:46:33.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.667+0000 7ff626615640 1 --2- 172.21.15.83:0/20085132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6200cf850 0x7ff6200cfcb0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7ff618009e50 tx=0x7ff61801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9133a1fb650acd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:33.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff624e12640 1 -- 172.21.15.83:0/20085132 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6180a5670 con 0x7ff6200cf850 2024-04-17T10:46:33.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff624e12640 1 -- 172.21.15.83:0/20085132 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6180a5810 con 0x7ff6200cf850 2024-04-17T10:46:33.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff624e12640 1 -- 172.21.15.83:0/20085132 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6180a5be0 con 0x7ff6200cf850 2024-04-17T10:46:33.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff628078640 1 -- 172.21.15.83:0/20085132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6200cf850 msgr2=0x7ff6200cfcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:33.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff628078640 1 --2- 172.21.15.83:0/20085132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6200cf850 0x7ff6200cfcb0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7ff618009e50 tx=0x7ff61801c9f0 comp rx=0 tx=0).stop 2024-04-17T10:46:33.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff628078640 1 -- 172.21.15.83:0/20085132 shutdown_connections 2024-04-17T10:46:33.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff628078640 1 --2- 172.21.15.83:0/20085132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6200cf850 0x7ff6200cfcb0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:33.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff628078640 1 --2- 172.21.15.83:0/20085132 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff62015df70 0x7ff6200b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:33.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff628078640 1 -- 172.21.15.83:0/20085132 >> 172.21.15.83:0/20085132 conn(0x7ff6200b3330 msgr2=0x7ff6200b3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:33.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff628078640 1 -- 172.21.15.83:0/20085132 shutdown_connections 2024-04-17T10:46:33.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff628078640 1 -- 172.21.15.83:0/20085132 wait complete. 2024-04-17T10:46:33.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff628078640 1 Processor -- start 2024-04-17T10:46:33.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff628078640 1 -- start start 2024-04-17T10:46:33.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff628078640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6200b3e50 0x7ff6201fa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:33.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff628078640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6200cf850 0x7ff6200c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:33.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff628078640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff62015df70 0x7ff6200c6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:33.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff628078640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62016ad30 con 0x7ff62015df70 2024-04-17T10:46:33.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff628078640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6200b2000 con 0x7ff6200cf850 2024-04-17T10:46:33.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff628078640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff62016aeb0 con 0x7ff6200b3e50 2024-04-17T10:46:33.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff626615640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff62015df70 0x7ff6200c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:33.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff626615640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff62015df70 0x7ff6200c6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32854/0 (socket says 172.21.15.83:32854) 2024-04-17T10:46:33.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff626615640 1 -- 172.21.15.83:0/268390643 learned_addr learned my addr 172.21.15.83:0/268390643 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:33.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff626615640 1 -- 172.21.15.83:0/268390643 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6200b3e50 msgr2=0x7ff6201fa9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:33.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff625e14640 1 --2- 172.21.15.83:0/268390643 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6200b3e50 0x7ff6201fa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:33.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff625613640 1 --2- 172.21.15.83:0/268390643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6200cf850 0x7ff6200c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:33.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff626615640 1 --2- 172.21.15.83:0/268390643 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6200b3e50 0x7ff6201fa9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:33.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff626615640 1 -- 172.21.15.83:0/268390643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6200cf850 msgr2=0x7ff6200c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:33.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff626615640 1 --2- 172.21.15.83:0/268390643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6200cf850 0x7ff6200c5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:33.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff626615640 1 -- 172.21.15.83:0/268390643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6200c67c0 con 0x7ff62015df70 2024-04-17T10:46:33.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff625613640 1 --2- 172.21.15.83:0/268390643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6200cf850 0x7ff6200c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:33.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff626615640 1 --2- 172.21.15.83:0/268390643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff62015df70 0x7ff6200c6280 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7ff6180af040 tx=0x7ff618019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:33.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff6167fc640 1 -- 172.21.15.83:0/268390643 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6180b8670 con 0x7ff62015df70 2024-04-17T10:46:33.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff6167fc640 1 -- 172.21.15.83:0/268390643 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6180be070 con 0x7ff62015df70 2024-04-17T10:46:33.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff628078640 1 -- 172.21.15.83:0/268390643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6200c72d0 con 0x7ff62015df70 2024-04-17T10:46:33.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff6167fc640 1 -- 172.21.15.83:0/268390643 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6180b8a30 con 0x7ff62015df70 2024-04-17T10:46:33.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.671+0000 7ff628078640 1 -- 172.21.15.83:0/268390643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6200c6a30 con 0x7ff62015df70 2024-04-17T10:46:33.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.675+0000 7ff628078640 1 -- 172.21.15.83:0/268390643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff620074f90 con 0x7ff62015df70 2024-04-17T10:46:33.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.675+0000 7ff6167fc640 1 -- 172.21.15.83:0/268390643 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff6180b8bd0 con 0x7ff62015df70 2024-04-17T10:46:33.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.675+0000 7ff6167fc640 1 --2- 172.21.15.83:0/268390643 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff604066f60 0x7ff604069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:33.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.675+0000 7ff6167fc640 1 -- 172.21.15.83:0/268390643 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff61811ffb0 con 0x7ff62015df70 2024-04-17T10:46:33.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.675+0000 7ff625e14640 1 --2- 172.21.15.83:0/268390643 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff604066f60 0x7ff604069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:33.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.675+0000 7ff625e14640 1 --2- 172.21.15.83:0/268390643 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff604066f60 0x7ff604069420 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00b7c0 tx=0x7ff61c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:33.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.679+0000 7ff6167fc640 1 -- 172.21.15.83:0/268390643 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff6180165e0 con 0x7ff62015df70 2024-04-17T10:46:33.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.823+0000 7ff628078640 1 -- 172.21.15.83:0/268390643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6200d1570 con 0x7ff62015df70 2024-04-17T10:46:33.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.827+0000 7ff6167fc640 1 -- 172.21.15.83:0/268390643 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+12820 (secure 0 0 0) 0x7ff6180f1390 con 0x7ff62015df70 2024-04-17T10:46:33.822 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:46:33.823 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":85,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:46:33.188967+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:46:33.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.827+0000 7ff628078640 1 -- 172.21.15.83:0/268390643 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff604066f60 msgr2=0x7ff604069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:33.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.827+0000 7ff628078640 1 --2- 172.21.15.83:0/268390643 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff604066f60 0x7ff604069420 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00b7c0 tx=0x7ff61c008040 comp rx=0 tx=0).stop 2024-04-17T10:46:33.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.827+0000 7ff628078640 1 -- 172.21.15.83:0/268390643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff62015df70 msgr2=0x7ff6200c6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:33.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.827+0000 7ff628078640 1 --2- 172.21.15.83:0/268390643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff62015df70 0x7ff6200c6280 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7ff6180af040 tx=0x7ff618019790 comp rx=0 tx=0).stop 2024-04-17T10:46:33.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.827+0000 7ff628078640 1 -- 172.21.15.83:0/268390643 shutdown_connections 2024-04-17T10:46:33.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.827+0000 7ff628078640 1 --2- 172.21.15.83:0/268390643 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff604066f60 0x7ff604069420 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:33.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.827+0000 7ff628078640 1 --2- 172.21.15.83:0/268390643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff62015df70 0x7ff6200c6280 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:33.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.827+0000 7ff628078640 1 --2- 172.21.15.83:0/268390643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6200cf850 0x7ff6200c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:33.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.827+0000 7ff628078640 1 --2- 172.21.15.83:0/268390643 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6200b3e50 0x7ff6201fa9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:33.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.827+0000 7ff628078640 1 -- 172.21.15.83:0/268390643 >> 172.21.15.83:0/268390643 conn(0x7ff6200b3330 msgr2=0x7ff6200ae610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:33.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.827+0000 7ff628078640 1 -- 172.21.15.83:0/268390643 shutdown_connections 2024-04-17T10:46:33.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.827+0000 7ff628078640 1 -- 172.21.15.83:0/268390643 wait complete. 2024-04-17T10:46:33.840 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:46:33.841 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:46:33.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.955+0000 7f8d046f7640 1 Processor -- start 2024-04-17T10:46:33.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.955+0000 7f8d046f7640 1 -- start start 2024-04-17T10:46:33.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d046f7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cfc15bac0 0x7f8cfc071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:33.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d046f7640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cfc06c1f0 con 0x7f8cfc15b6e0 2024-04-17T10:46:33.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d046f7640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cfc070a10 con 0x7f8cfc070470 2024-04-17T10:46:33.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d046f7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc070cf0 con 0x7f8cfc15bac0 2024-04-17T10:46:33.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d02493640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8cfc15b6e0 0x7f8cfc0cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50742/0 (socket says 172.21.15.83:50742) 2024-04-17T10:46:33.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d02493640 1 -- 172.21.15.83:0/2987502690 learned_addr learned my addr 172.21.15.83:0/2987502690 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:46:33.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d01c92640 1 --2- 172.21.15.83:0/2987502690 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cfc15bac0 0x7f8cfc071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:33.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d01491640 1 -- 172.21.15.83:0/2987502690 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3571062625 0 0) 0x7f8cfc06c1f0 con 0x7f8cfc15b6e0 2024-04-17T10:46:33.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d01491640 1 -- 172.21.15.83:0/2987502690 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ce8003530 con 0x7f8cfc15b6e0 2024-04-17T10:46:33.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d01491640 1 -- 172.21.15.83:0/2987502690 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3648470049 0 0) 0x7f8ce8003530 con 0x7f8cfc15b6e0 2024-04-17T10:46:33.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d01491640 1 -- 172.21.15.83:0/2987502690 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cfc15bac0 msgr2=0x7f8cfc071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:33.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d01491640 1 --2- 172.21.15.83:0/2987502690 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cfc15bac0 0x7f8cfc071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:33.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d01491640 1 -- 172.21.15.83:0/2987502690 >> v1:172.21.15.167:6789/0 conn(0x7f8cfc070470 legacy=0x7f8cfc06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:33.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d01491640 1 -- 172.21.15.83:0/2987502690 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cfc070fd0 con 0x7f8cfc15b6e0 2024-04-17T10:46:33.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d01491640 1 -- 172.21.15.83:0/2987502690 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f8cf8001680 con 0x7f8cfc15b6e0 2024-04-17T10:46:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d01491640 1 -- 172.21.15.83:0/2987502690 >> v1:172.21.15.83:6789/0 conn(0x7f8cfc15b6e0 legacy=0x7f8cfc0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d01491640 1 --2- 172.21.15.83:0/2987502690 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cfc15bac0 0x7f8ce8003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d01491640 1 --2- 172.21.15.83:0/2987502690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ce8003fe0 0x7f8ce8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d01491640 1 --2- 172.21.15.83:0/2987502690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ce8004d50 0x7f8ce80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d01491640 1 -- 172.21.15.83:0/2987502690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc070cf0 con 0x7f8ce8004d50 2024-04-17T10:46:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d01491640 1 -- 172.21.15.83:0/2987502690 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce800bb30 con 0x7f8ce8003fe0 2024-04-17T10:46:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d01491640 1 -- 172.21.15.83:0/2987502690 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce800bd30 con 0x7f8cfc15bac0 2024-04-17T10:46:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d02c94640 1 --2- 172.21.15.83:0/2987502690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ce8004d50 0x7f8ce80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d01c92640 1 --2- 172.21.15.83:0/2987502690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ce8003fe0 0x7f8ce8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d02c94640 1 -- 172.21.15.83:0/2987502690 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cfc15bac0 msgr2=0x7f8ce8003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d02c94640 1 --2- 172.21.15.83:0/2987502690 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cfc15bac0 0x7f8ce8003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d02c94640 1 -- 172.21.15.83:0/2987502690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ce8003fe0 msgr2=0x7f8ce8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d02c94640 1 --2- 172.21.15.83:0/2987502690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ce8003fe0 0x7f8ce8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:33.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d02c94640 1 -- 172.21.15.83:0/2987502690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce8003530 con 0x7f8ce8004d50 2024-04-17T10:46:33.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d02c94640 1 --2- 172.21.15.83:0/2987502690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ce8004d50 0x7f8ce80051d0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4001e70 tx=0x7f8cf401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d794a5e961e1df9e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:33.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d01491640 1 -- 172.21.15.83:0/2987502690 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf4001a90 con 0x7f8ce8004d50 2024-04-17T10:46:33.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d01491640 1 -- 172.21.15.83:0/2987502690 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf4003b80 con 0x7f8ce8004d50 2024-04-17T10:46:33.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.959+0000 7f8d01491640 1 -- 172.21.15.83:0/2987502690 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf4005080 con 0x7f8ce8004d50 2024-04-17T10:46:33.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.963+0000 7f8d046f7640 1 -- 172.21.15.83:0/2987502690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ce8004d50 msgr2=0x7f8ce80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:33.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.963+0000 7f8d046f7640 1 --2- 172.21.15.83:0/2987502690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ce8004d50 0x7f8ce80051d0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4001e70 tx=0x7f8cf401c780 comp rx=0 tx=0).stop 2024-04-17T10:46:33.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.963+0000 7f8d02c94640 1 -- 172.21.15.83:0/2987502690 reap_dead start 2024-04-17T10:46:33.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.963+0000 7f8d046f7640 1 -- 172.21.15.83:0/2987502690 shutdown_connections 2024-04-17T10:46:33.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.963+0000 7f8d046f7640 1 -- 172.21.15.83:0/2987502690 >> 172.21.15.83:0/2987502690 conn(0x7f8cfc0b15b0 msgr2=0x7f8cfc0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:33.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.963+0000 7f8d046f7640 1 -- 172.21.15.83:0/2987502690 shutdown_connections 2024-04-17T10:46:33.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.963+0000 7f8d046f7640 1 -- 172.21.15.83:0/2987502690 wait complete. 2024-04-17T10:46:33.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.963+0000 7f8d046f7640 1 Processor -- start 2024-04-17T10:46:33.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.963+0000 7f8d046f7640 1 -- start start 2024-04-17T10:46:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.963+0000 7f8d046f7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ce8004d50 0x7f8cfc154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.963+0000 7f8d046f7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cfc154cc0 0x7f8cfc15dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.963+0000 7f8d046f7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cfc1f3980 0x7f8cfc0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.963+0000 7f8d046f7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc06b6a0 con 0x7f8cfc154cc0 2024-04-17T10:46:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.963+0000 7f8d046f7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc06b520 con 0x7f8ce8004d50 2024-04-17T10:46:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.963+0000 7f8d046f7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc071cc0 con 0x7f8cfc1f3980 2024-04-17T10:46:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.963+0000 7f8d01c92640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cfc154cc0 0x7f8cfc15dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.963+0000 7f8d01c92640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cfc154cc0 0x7f8cfc15dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32866/0 (socket says 172.21.15.83:32866) 2024-04-17T10:46:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.963+0000 7f8d01c92640 1 -- 172.21.15.83:0/245753718 learned_addr learned my addr 172.21.15.83:0/245753718 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:33.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.963+0000 7f8d02493640 1 --2- 172.21.15.83:0/245753718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ce8004d50 0x7f8cfc154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:33.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.963+0000 7f8d01c92640 1 -- 172.21.15.83:0/245753718 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cfc1f3980 msgr2=0x7f8cfc0ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:33.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.963+0000 7f8d01c92640 1 --2- 172.21.15.83:0/245753718 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cfc1f3980 0x7f8cfc0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:33.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.963+0000 7f8d01c92640 1 -- 172.21.15.83:0/245753718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ce8004d50 msgr2=0x7f8cfc154440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:33.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.963+0000 7f8d01c92640 1 --2- 172.21.15.83:0/245753718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ce8004d50 0x7f8cfc154440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:33.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.963+0000 7f8d01c92640 1 -- 172.21.15.83:0/245753718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cfc15e5b0 con 0x7f8cfc154cc0 2024-04-17T10:46:33.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.963+0000 7f8d02493640 1 --2- 172.21.15.83:0/245753718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ce8004d50 0x7f8cfc154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:33.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.963+0000 7f8d01c92640 1 --2- 172.21.15.83:0/245753718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cfc154cc0 0x7f8cfc15dcd0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f8cec011550 tx=0x7f8cec011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:33.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.963+0000 7f8cf2ffd640 1 -- 172.21.15.83:0/245753718 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cec015870 con 0x7f8cfc154cc0 2024-04-17T10:46:33.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.963+0000 7f8cf2ffd640 1 -- 172.21.15.83:0/245753718 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cec015a10 con 0x7f8cfc154cc0 2024-04-17T10:46:33.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.963+0000 7f8cf2ffd640 1 -- 172.21.15.83:0/245753718 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cec015e00 con 0x7f8cfc154cc0 2024-04-17T10:46:33.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.963+0000 7f8d046f7640 1 -- 172.21.15.83:0/245753718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cfc15c130 con 0x7f8cfc154cc0 2024-04-17T10:46:33.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.963+0000 7f8d046f7640 1 -- 172.21.15.83:0/245753718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cfc154980 con 0x7f8cfc154cc0 2024-04-17T10:46:33.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.967+0000 7f8d046f7640 1 -- 172.21.15.83:0/245753718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cfc15bf50 con 0x7f8cfc154cc0 2024-04-17T10:46:33.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.967+0000 7f8cf2ffd640 1 -- 172.21.15.83:0/245753718 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8cec00f460 con 0x7f8cfc154cc0 2024-04-17T10:46:33.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.967+0000 7f8cf2ffd640 1 --2- 172.21.15.83:0/245753718 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8ce0066f90 0x7f8ce0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:33.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.967+0000 7f8cf2ffd640 1 -- 172.21.15.83:0/245753718 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8cec0854e0 con 0x7f8cfc154cc0 2024-04-17T10:46:33.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.967+0000 7f8d02493640 1 --2- 172.21.15.83:0/245753718 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8ce0066f90 0x7f8ce0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:33.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.967+0000 7f8d02493640 1 --2- 172.21.15.83:0/245753718 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8ce0066f90 0x7f8ce0069450 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8001ec0 tx=0x7f8cf8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:33.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:33.971+0000 7f8cf2ffd640 1 -- 172.21.15.83:0/245753718 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8cfc15bf50 con 0x7f8cfc154cc0 2024-04-17T10:46:34.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.115+0000 7f8d046f7640 1 -- 172.21.15.83:0/245753718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8cfc06d910 con 0x7f8cfc154cc0 2024-04-17T10:46:34.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.119+0000 7f8cf2ffd640 1 -- 172.21.15.83:0/245753718 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8cec019030 con 0x7f8cfc154cc0 2024-04-17T10:46:34.114 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:46:34.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.119+0000 7f8d046f7640 1 -- 172.21.15.83:0/245753718 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8ce0066f90 msgr2=0x7f8ce0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:34.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.119+0000 7f8d046f7640 1 --2- 172.21.15.83:0/245753718 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8ce0066f90 0x7f8ce0069450 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8001ec0 tx=0x7f8cf8009040 comp rx=0 tx=0).stop 2024-04-17T10:46:34.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.119+0000 7f8d046f7640 1 -- 172.21.15.83:0/245753718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cfc154cc0 msgr2=0x7f8cfc15dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:34.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.119+0000 7f8d046f7640 1 --2- 172.21.15.83:0/245753718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cfc154cc0 0x7f8cfc15dcd0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f8cec011550 tx=0x7f8cec011a10 comp rx=0 tx=0).stop 2024-04-17T10:46:34.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.119+0000 7f8d046f7640 1 -- 172.21.15.83:0/245753718 shutdown_connections 2024-04-17T10:46:34.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.119+0000 7f8d046f7640 1 --2- 172.21.15.83:0/245753718 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8ce0066f90 0x7f8ce0069450 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:34.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.119+0000 7f8d046f7640 1 --2- 172.21.15.83:0/245753718 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cfc1f3980 0x7f8cfc0ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:34.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.119+0000 7f8d046f7640 1 --2- 172.21.15.83:0/245753718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cfc154cc0 0x7f8cfc15dcd0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:34.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.119+0000 7f8d046f7640 1 --2- 172.21.15.83:0/245753718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ce8004d50 0x7f8cfc154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:34.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.119+0000 7f8d046f7640 1 -- 172.21.15.83:0/245753718 >> 172.21.15.83:0/245753718 conn(0x7f8cfc0b15b0 msgr2=0x7f8cfc06ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:34.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.119+0000 7f8d046f7640 1 -- 172.21.15.83:0/245753718 shutdown_connections 2024-04-17T10:46:34.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.119+0000 7f8d046f7640 1 -- 172.21.15.83:0/245753718 wait complete. 2024-04-17T10:46:34.131 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-17T10:46:34.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.255+0000 7f33e6138640 1 Processor -- start 2024-04-17T10:46:34.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.255+0000 7f33e6138640 1 -- start start 2024-04-17T10:46:34.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.259+0000 7f33e6138640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f33e015db50 0x7f33e0070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:34.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.259+0000 7f33e6138640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33e006dcb0 0x7f33e00b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:34.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.259+0000 7f33e6138640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e00709b0 con 0x7f33e00aedd0 2024-04-17T10:46:34.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.259+0000 7f33e6138640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33e015e440 con 0x7f33e015db50 2024-04-17T10:46:34.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.259+0000 7f33e6138640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e015e720 con 0x7f33e006dcb0 2024-04-17T10:46:34.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.259+0000 7f33df7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f33e015db50 0x7f33e0070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:34.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.259+0000 7f33df7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f33e015db50 0x7f33e0070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32882/0 (socket says 172.21.15.83:32882) 2024-04-17T10:46:34.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.259+0000 7f33df7fe640 1 -- 172.21.15.83:0/516856303 learned_addr learned my addr 172.21.15.83:0/516856303 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:34.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.259+0000 7f33df7fe640 1 -- 172.21.15.83:0/516856303 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33e006dcb0 msgr2=0x7f33e00b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:34.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.259+0000 7f33df7fe640 1 --2- 172.21.15.83:0/516856303 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33e006dcb0 0x7f33e00b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:34.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.259+0000 7f33df7fe640 1 -- 172.21.15.83:0/516856303 >> v1:172.21.15.167:6789/0 conn(0x7f33e00aedd0 legacy=0x7f33e006d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:34.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.259+0000 7f33df7fe640 1 -- 172.21.15.83:0/516856303 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e015ea00 con 0x7f33e015db50 2024-04-17T10:46:34.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.263+0000 7f33df7fe640 1 --2- 172.21.15.83:0/516856303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33e015db50 0x7f33e0070470 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f33c8009870 tx=0x7f33c801c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac510cad4928f0b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:34.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.263+0000 7f33deffd640 1 -- 172.21.15.83:0/516856303 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c80b0070 con 0x7f33e015db50 2024-04-17T10:46:34.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.263+0000 7f33deffd640 1 -- 172.21.15.83:0/516856303 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c80ab600 con 0x7f33e015db50 2024-04-17T10:46:34.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.263+0000 7f33deffd640 1 -- 172.21.15.83:0/516856303 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c80ab9d0 con 0x7f33e015db50 2024-04-17T10:46:34.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.263+0000 7f33e6138640 1 -- 172.21.15.83:0/516856303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33e015db50 msgr2=0x7f33e0070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:34.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.263+0000 7f33e6138640 1 --2- 172.21.15.83:0/516856303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33e015db50 0x7f33e0070470 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f33c8009870 tx=0x7f33c801c330 comp rx=0 tx=0).stop 2024-04-17T10:46:34.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.263+0000 7f33e6138640 1 -- 172.21.15.83:0/516856303 shutdown_connections 2024-04-17T10:46:34.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.263+0000 7f33e6138640 1 --2- 172.21.15.83:0/516856303 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33e006dcb0 0x7f33e00b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:34.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.263+0000 7f33e6138640 1 --2- 172.21.15.83:0/516856303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33e015db50 0x7f33e0070470 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:34.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.263+0000 7f33e6138640 1 -- 172.21.15.83:0/516856303 >> 172.21.15.83:0/516856303 conn(0x7f33e006abe0 msgr2=0x7f33e006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:34.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.263+0000 7f33e6138640 1 -- 172.21.15.83:0/516856303 shutdown_connections 2024-04-17T10:46:34.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.263+0000 7f33e6138640 1 -- 172.21.15.83:0/516856303 wait complete. 2024-04-17T10:46:34.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.263+0000 7f33e6138640 1 Processor -- start 2024-04-17T10:46:34.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.263+0000 7f33e6138640 1 -- start start 2024-04-17T10:46:34.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.263+0000 7f33e6138640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33e006dcb0 0x7f33e0154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:34.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.263+0000 7f33e6138640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33e00aedd0 0x7f33e01502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:34.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.263+0000 7f33e6138640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33e015db50 0x7f33e0151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:34.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.263+0000 7f33e6138640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e00b3ce0 con 0x7f33e00aedd0 2024-04-17T10:46:34.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.263+0000 7f33e6138640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e00b3b60 con 0x7f33e015db50 2024-04-17T10:46:34.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.263+0000 7f33e6138640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e00b3e60 con 0x7f33e006dcb0 2024-04-17T10:46:34.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.263+0000 7f33d7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33e00aedd0 0x7f33e01502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:34.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.263+0000 7f33d7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33e00aedd0 0x7f33e01502c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32884/0 (socket says 172.21.15.83:32884) 2024-04-17T10:46:34.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.263+0000 7f33d7fff640 1 -- 172.21.15.83:0/3819029538 learned_addr learned my addr 172.21.15.83:0/3819029538 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:34.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.263+0000 7f33dffff640 1 --2- 172.21.15.83:0/3819029538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33e015db50 0x7f33e0151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:34.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.263+0000 7f33d7fff640 1 -- 172.21.15.83:0/3819029538 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33e006dcb0 msgr2=0x7f33e0154ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:34.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.263+0000 7f33d7fff640 1 --2- 172.21.15.83:0/3819029538 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33e006dcb0 0x7f33e0154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:34.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.263+0000 7f33d7fff640 1 -- 172.21.15.83:0/3819029538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33e015db50 msgr2=0x7f33e0151f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:34.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.263+0000 7f33d7fff640 1 --2- 172.21.15.83:0/3819029538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33e015db50 0x7f33e0151f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:34.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.263+0000 7f33d7fff640 1 -- 172.21.15.83:0/3819029538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e015afc0 con 0x7f33e00aedd0 2024-04-17T10:46:34.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.263+0000 7f33dffff640 1 --2- 172.21.15.83:0/3819029538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33e015db50 0x7f33e0151f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:34.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.267+0000 7f33d7fff640 1 --2- 172.21.15.83:0/3819029538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33e00aedd0 0x7f33e01502c0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f33d000c450 tx=0x7f33d000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:34.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.267+0000 7f33dcff9640 1 -- 172.21.15.83:0/3819029538 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d0012830 con 0x7f33e00aedd0 2024-04-17T10:46:34.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.267+0000 7f33dcff9640 1 -- 172.21.15.83:0/3819029538 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d0017070 con 0x7f33e00aedd0 2024-04-17T10:46:34.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.267+0000 7f33e6138640 1 -- 172.21.15.83:0/3819029538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e0152f70 con 0x7f33e00aedd0 2024-04-17T10:46:34.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.267+0000 7f33dcff9640 1 -- 172.21.15.83:0/3819029538 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d0012c30 con 0x7f33e00aedd0 2024-04-17T10:46:34.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.267+0000 7f33e6138640 1 -- 172.21.15.83:0/3819029538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e0153530 con 0x7f33e00aedd0 2024-04-17T10:46:34.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.267+0000 7f33e6138640 1 -- 172.21.15.83:0/3819029538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e0074f90 con 0x7f33e00aedd0 2024-04-17T10:46:34.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.267+0000 7f33dcff9640 1 -- 172.21.15.83:0/3819029538 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f33d001c460 con 0x7f33e00aedd0 2024-04-17T10:46:34.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.267+0000 7f33dcff9640 1 --2- 172.21.15.83:0/3819029538 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f33c0066f60 0x7f33c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:34.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.267+0000 7f33dcff9640 1 -- 172.21.15.83:0/3819029538 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f33d0084a50 con 0x7f33e00aedd0 2024-04-17T10:46:34.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.267+0000 7f33df7fe640 1 --2- 172.21.15.83:0/3819029538 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f33c0066f60 0x7f33c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:34.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.267+0000 7f33df7fe640 1 --2- 172.21.15.83:0/3819029538 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f33c0066f60 0x7f33c0069420 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f33c80ad000 tx=0x7f33c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:34.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.271+0000 7f33dcff9640 1 -- 172.21.15.83:0/3819029538 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f33d0055db0 con 0x7f33e00aedd0 2024-04-17T10:46:34.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.431+0000 7f33e6138640 1 -- 172.21.15.83:0/3819029538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f33e00b2af0 con 0x7f33e00aedd0 2024-04-17T10:46:34.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.435+0000 7f33dcff9640 1 -- 172.21.15.83:0/3819029538 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v85) v1 ==== 84+0+65 (secure 0 0 0) 0x7f33d00556b0 con 0x7f33e00aedd0 2024-04-17T10:46:34.430 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:46:34.430 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-04-17T10:46:34.430 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:46:34.430 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2024-04-17T10:46:34.430 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2024-04-17T10:46:34.430 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:46:34.430 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-04-17T10:46:34.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.435+0000 7f33e6138640 1 -- 172.21.15.83:0/3819029538 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f33c0066f60 msgr2=0x7f33c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:34.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.435+0000 7f33e6138640 1 --2- 172.21.15.83:0/3819029538 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f33c0066f60 0x7f33c0069420 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f33c80ad000 tx=0x7f33c8008040 comp rx=0 tx=0).stop 2024-04-17T10:46:34.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.435+0000 7f33e6138640 1 -- 172.21.15.83:0/3819029538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33e00aedd0 msgr2=0x7f33e01502c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:34.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.435+0000 7f33e6138640 1 --2- 172.21.15.83:0/3819029538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33e00aedd0 0x7f33e01502c0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f33d000c450 tx=0x7f33d000c910 comp rx=0 tx=0).stop 2024-04-17T10:46:34.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.435+0000 7f33e6138640 1 -- 172.21.15.83:0/3819029538 shutdown_connections 2024-04-17T10:46:34.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.435+0000 7f33e6138640 1 --2- 172.21.15.83:0/3819029538 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f33c0066f60 0x7f33c0069420 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:34.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.435+0000 7f33e6138640 1 --2- 172.21.15.83:0/3819029538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33e015db50 0x7f33e0151f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:34.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.435+0000 7f33e6138640 1 --2- 172.21.15.83:0/3819029538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33e00aedd0 0x7f33e01502c0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:34.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.435+0000 7f33e6138640 1 --2- 172.21.15.83:0/3819029538 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33e006dcb0 0x7f33e0154ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:34.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.435+0000 7f33e6138640 1 -- 172.21.15.83:0/3819029538 >> 172.21.15.83:0/3819029538 conn(0x7f33e006abe0 msgr2=0x7f33e006cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:34.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.435+0000 7f33e6138640 1 -- 172.21.15.83:0/3819029538 shutdown_connections 2024-04-17T10:46:34.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.435+0000 7f33e6138640 1 -- 172.21.15.83:0/3819029538 wait complete. 2024-04-17T10:46:34.447 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-17T10:46:34.447 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-17T10:46:34.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.575+0000 7f321366a640 1 Processor -- start 2024-04-17T10:46:34.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.575+0000 7f321366a640 1 -- start start 2024-04-17T10:46:34.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.575+0000 7f321366a640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f320c0aedf0 0x7f320c0b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:34.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.575+0000 7f321366a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f320c15cb40 0x7f320c06abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:34.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.575+0000 7f321366a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f320c06b120 0x7f320c15f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:34.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.575+0000 7f321366a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f320c071290 con 0x7f320c15cb40 2024-04-17T10:46:34.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.575+0000 7f321366a640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f320c0b39d0 con 0x7f320c0aedf0 2024-04-17T10:46:34.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.575+0000 7f321366a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f320c070c40 con 0x7f320c06b120 2024-04-17T10:46:34.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.575+0000 7f3210c05640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f320c15cb40 0x7f320c06abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:34.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.575+0000 7f3210c05640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f320c15cb40 0x7f320c06abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32900/0 (socket says 172.21.15.83:32900) 2024-04-17T10:46:34.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.575+0000 7f3210c05640 1 -- 172.21.15.83:0/2051056846 learned_addr learned my addr 172.21.15.83:0/2051056846 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:34.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.575+0000 7f3211406640 1 --2- 172.21.15.83:0/2051056846 >> v2:172.21.15.167:3300/0 conn(0x7f320c0aedf0 0x7f320c0b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:34.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.575+0000 7f3211c07640 1 --2- 172.21.15.83:0/2051056846 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f320c06b120 0x7f320c15f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:34.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.575+0000 7f3210c05640 1 -- 172.21.15.83:0/2051056846 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f320c06b120 msgr2=0x7f320c15f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:34.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.575+0000 7f3210c05640 1 --2- 172.21.15.83:0/2051056846 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f320c06b120 0x7f320c15f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:34.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.575+0000 7f3210c05640 1 -- 172.21.15.83:0/2051056846 >> v2:172.21.15.167:3300/0 conn(0x7f320c0aedf0 msgr2=0x7f320c0b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:34.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.575+0000 7f3210c05640 1 --2- 172.21.15.83:0/2051056846 >> v2:172.21.15.167:3300/0 conn(0x7f320c0aedf0 0x7f320c0b30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:34.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.575+0000 7f3210c05640 1 -- 172.21.15.83:0/2051056846 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f320c070f20 con 0x7f320c15cb40 2024-04-17T10:46:34.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.575+0000 7f3211c07640 1 --2- 172.21.15.83:0/2051056846 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f320c06b120 0x7f320c15f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:34.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.579+0000 7f3210c05640 1 --2- 172.21.15.83:0/2051056846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f320c15cb40 0x7f320c06abe0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f320c0cf4a0 tx=0x7f31fc01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b11bb4a461f92 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:34.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.579+0000 7f3203fff640 1 -- 172.21.15.83:0/2051056846 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc0b0070 con 0x7f320c15cb40 2024-04-17T10:46:34.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.579+0000 7f3203fff640 1 -- 172.21.15.83:0/2051056846 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31fc0ab600 con 0x7f320c15cb40 2024-04-17T10:46:34.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.579+0000 7f3203fff640 1 -- 172.21.15.83:0/2051056846 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc0ab9d0 con 0x7f320c15cb40 2024-04-17T10:46:34.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.579+0000 7f321366a640 1 -- 172.21.15.83:0/2051056846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f320c15cb40 msgr2=0x7f320c06abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:34.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.579+0000 7f321366a640 1 --2- 172.21.15.83:0/2051056846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f320c15cb40 0x7f320c06abe0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f320c0cf4a0 tx=0x7f31fc01c260 comp rx=0 tx=0).stop 2024-04-17T10:46:34.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.579+0000 7f321366a640 1 -- 172.21.15.83:0/2051056846 shutdown_connections 2024-04-17T10:46:34.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.579+0000 7f321366a640 1 --2- 172.21.15.83:0/2051056846 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f320c06b120 0x7f320c15f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:34.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.579+0000 7f321366a640 1 --2- 172.21.15.83:0/2051056846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f320c15cb40 0x7f320c06abe0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:34.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.579+0000 7f321366a640 1 --2- 172.21.15.83:0/2051056846 >> v2:172.21.15.167:3300/0 conn(0x7f320c0aedf0 0x7f320c0b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:34.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.579+0000 7f321366a640 1 -- 172.21.15.83:0/2051056846 >> 172.21.15.83:0/2051056846 conn(0x7f320c0cc050 msgr2=0x7f320c0cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:34.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.579+0000 7f321366a640 1 -- 172.21.15.83:0/2051056846 shutdown_connections 2024-04-17T10:46:34.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.579+0000 7f321366a640 1 -- 172.21.15.83:0/2051056846 wait complete. 2024-04-17T10:46:34.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.579+0000 7f321366a640 1 Processor -- start 2024-04-17T10:46:34.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.579+0000 7f321366a640 1 -- start start 2024-04-17T10:46:34.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.579+0000 7f321366a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f320c06b120 0x7f320c1f4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:34.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.579+0000 7f321366a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f320c0aedf0 0x7f320c151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:34.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.579+0000 7f321366a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f320c15cb40 0x7f320c14d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:34.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.579+0000 7f321366a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c071880 con 0x7f320c0aedf0 2024-04-17T10:46:34.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.579+0000 7f321366a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c071700 con 0x7f320c06b120 2024-04-17T10:46:34.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.579+0000 7f321366a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f320c071a00 con 0x7f320c15cb40 2024-04-17T10:46:34.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.579+0000 7f3210c05640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f320c0aedf0 0x7f320c151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:34.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.579+0000 7f3210c05640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f320c0aedf0 0x7f320c151600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32914/0 (socket says 172.21.15.83:32914) 2024-04-17T10:46:34.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.579+0000 7f3210c05640 1 -- 172.21.15.83:0/3496272633 learned_addr learned my addr 172.21.15.83:0/3496272633 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:34.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.579+0000 7f3211406640 1 --2- 172.21.15.83:0/3496272633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f320c06b120 0x7f320c1f4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:34.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.579+0000 7f3211c07640 1 --2- 172.21.15.83:0/3496272633 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f320c15cb40 0x7f320c14d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:34.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.579+0000 7f3210c05640 1 -- 172.21.15.83:0/3496272633 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f320c15cb40 msgr2=0x7f320c14d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:34.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.579+0000 7f3210c05640 1 --2- 172.21.15.83:0/3496272633 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f320c15cb40 0x7f320c14d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:34.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.579+0000 7f3210c05640 1 -- 172.21.15.83:0/3496272633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f320c06b120 msgr2=0x7f320c1f4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:34.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.583+0000 7f3210c05640 1 --2- 172.21.15.83:0/3496272633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f320c06b120 0x7f320c1f4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:34.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.583+0000 7f3210c05640 1 -- 172.21.15.83:0/3496272633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f320c1f4fe0 con 0x7f320c0aedf0 2024-04-17T10:46:34.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.583+0000 7f3210c05640 1 --2- 172.21.15.83:0/3496272633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f320c0aedf0 0x7f320c151600 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f31fc0ae040 tx=0x7f31fc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:34.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.583+0000 7f3201ffb640 1 -- 172.21.15.83:0/3496272633 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc0ba020 con 0x7f320c0aedf0 2024-04-17T10:46:34.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.583+0000 7f3201ffb640 1 -- 172.21.15.83:0/3496272633 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31fc0b6620 con 0x7f320c0aedf0 2024-04-17T10:46:34.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.583+0000 7f321366a640 1 -- 172.21.15.83:0/3496272633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f320c151b40 con 0x7f320c0aedf0 2024-04-17T10:46:34.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.583+0000 7f3201ffb640 1 -- 172.21.15.83:0/3496272633 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc0b0040 con 0x7f320c0aedf0 2024-04-17T10:46:34.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.583+0000 7f321366a640 1 -- 172.21.15.83:0/3496272633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320c1f5550 con 0x7f320c0aedf0 2024-04-17T10:46:34.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.583+0000 7f321366a640 1 -- 172.21.15.83:0/3496272633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f320c075000 con 0x7f320c0aedf0 2024-04-17T10:46:34.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.583+0000 7f3201ffb640 1 -- 172.21.15.83:0/3496272633 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f31fc0bd020 con 0x7f320c0aedf0 2024-04-17T10:46:34.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.583+0000 7f3201ffb640 1 --2- 172.21.15.83:0/3496272633 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f31f0066f60 0x7f31f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:34.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.583+0000 7f3201ffb640 1 -- 172.21.15.83:0/3496272633 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f31fc11ea30 con 0x7f320c0aedf0 2024-04-17T10:46:34.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.583+0000 7f3211406640 1 --2- 172.21.15.83:0/3496272633 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f31f0066f60 0x7f31f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:34.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.583+0000 7f3211406640 1 --2- 172.21.15.83:0/3496272633 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f31f0066f60 0x7f31f0069420 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f320c154780 tx=0x7f3208008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:34.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.587+0000 7f3201ffb640 1 -- 172.21.15.83:0/3496272633 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f31fc016110 con 0x7f320c0aedf0 2024-04-17T10:46:34.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:34.735+0000 7f321366a640 1 -- 172.21.15.83:0/3496272633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f320c0ccd90 con 0x7f320c0aedf0 2024-04-17T10:46:35.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.191+0000 7f3201ffb640 1 -- 172.21.15.83:0/3496272633 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v86) v1 ==== 138+0+0 (secure 0 0 0) 0x7f31fc0efe20 con 0x7f320c0aedf0 2024-04-17T10:46:35.188 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2024-04-17T10:46:35.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.195+0000 7f321366a640 1 -- 172.21.15.83:0/3496272633 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f31f0066f60 msgr2=0x7f31f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:35.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.195+0000 7f321366a640 1 --2- 172.21.15.83:0/3496272633 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f31f0066f60 0x7f31f0069420 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f320c154780 tx=0x7f3208008040 comp rx=0 tx=0).stop 2024-04-17T10:46:35.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.195+0000 7f321366a640 1 -- 172.21.15.83:0/3496272633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f320c0aedf0 msgr2=0x7f320c151600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:35.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.195+0000 7f321366a640 1 --2- 172.21.15.83:0/3496272633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f320c0aedf0 0x7f320c151600 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f31fc0ae040 tx=0x7f31fc019000 comp rx=0 tx=0).stop 2024-04-17T10:46:35.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.195+0000 7f321366a640 1 -- 172.21.15.83:0/3496272633 shutdown_connections 2024-04-17T10:46:35.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.195+0000 7f321366a640 1 --2- 172.21.15.83:0/3496272633 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f31f0066f60 0x7f31f0069420 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:35.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.195+0000 7f321366a640 1 --2- 172.21.15.83:0/3496272633 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f320c15cb40 0x7f320c14d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:35.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.195+0000 7f321366a640 1 --2- 172.21.15.83:0/3496272633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f320c0aedf0 0x7f320c151600 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:35.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.195+0000 7f321366a640 1 --2- 172.21.15.83:0/3496272633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f320c06b120 0x7f320c1f4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:35.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.195+0000 7f321366a640 1 -- 172.21.15.83:0/3496272633 >> 172.21.15.83:0/3496272633 conn(0x7f320c0cc050 msgr2=0x7f320c15d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:35.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.195+0000 7f321366a640 1 -- 172.21.15.83:0/3496272633 shutdown_connections 2024-04-17T10:46:35.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.195+0000 7f321366a640 1 -- 172.21.15.83:0/3496272633 wait complete. 2024-04-17T10:46:35.216 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-17T10:46:35.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe62edb8640 1 Processor -- start 2024-04-17T10:46:35.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe62edb8640 1 -- start start 2024-04-17T10:46:35.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe62edb8640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe62815ba00 0x7fe628072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:35.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe62edb8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6280ca970 0x7fe6280cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:35.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe62edb8640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6280cb7b0 con 0x7fe6280cb360 2024-04-17T10:46:35.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe62edb8640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe62806cbd0 con 0x7fe62815ba00 2024-04-17T10:46:35.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe62edb8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe62806f5e0 con 0x7fe6280ca970 2024-04-17T10:46:35.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe62d355640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe6280cb360 0x7fe62815f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50752/0 (socket says 172.21.15.83:50752) 2024-04-17T10:46:35.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe62d355640 1 -- 172.21.15.83:0/2435702998 learned_addr learned my addr 172.21.15.83:0/2435702998 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:46:35.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe62cb54640 1 --2- 172.21.15.83:0/2435702998 >> v2:172.21.15.167:3300/0 conn(0x7fe62815ba00 0x7fe628072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:35.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe627fff640 1 --2- 172.21.15.83:0/2435702998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6280ca970 0x7fe6280cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:35.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe6277fe640 1 -- 172.21.15.83:0/2435702998 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2975002835 0 0) 0x7fe6280cb7b0 con 0x7fe6280cb360 2024-04-17T10:46:35.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe6277fe640 1 -- 172.21.15.83:0/2435702998 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe610003530 con 0x7fe6280cb360 2024-04-17T10:46:35.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe6277fe640 1 -- 172.21.15.83:0/2435702998 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3311908795 0 0) 0x7fe610003530 con 0x7fe6280cb360 2024-04-17T10:46:35.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe6277fe640 1 -- 172.21.15.83:0/2435702998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6280ca970 msgr2=0x7fe6280cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:35.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe6277fe640 1 --2- 172.21.15.83:0/2435702998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6280ca970 0x7fe6280cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:35.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe6277fe640 1 -- 172.21.15.83:0/2435702998 >> v2:172.21.15.167:3300/0 conn(0x7fe62815ba00 msgr2=0x7fe628072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:35.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe6277fe640 1 --2- 172.21.15.83:0/2435702998 >> v2:172.21.15.167:3300/0 conn(0x7fe62815ba00 0x7fe628072960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:35.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe6277fe640 1 -- 172.21.15.83:0/2435702998 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6280c4ea0 con 0x7fe6280cb360 2024-04-17T10:46:35.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe6277fe640 1 -- 172.21.15.83:0/2435702998 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fe61c001680 con 0x7fe6280cb360 2024-04-17T10:46:35.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe6277fe640 1 -- 172.21.15.83:0/2435702998 >> v1:172.21.15.83:6789/0 conn(0x7fe6280cb360 legacy=0x7fe62815f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:35.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe6277fe640 1 --2- 172.21.15.83:0/2435702998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe610003a30 0x7fe610003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:35.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe6277fe640 1 --2- 172.21.15.83:0/2435702998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe610004810 0x7fe610004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:35.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe6277fe640 1 --2- 172.21.15.83:0/2435702998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6280ca970 0x7fe6100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:35.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe6277fe640 1 -- 172.21.15.83:0/2435702998 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62806f5e0 con 0x7fe610003a30 2024-04-17T10:46:35.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe6277fe640 1 -- 172.21.15.83:0/2435702998 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61000ba00 con 0x7fe610004810 2024-04-17T10:46:35.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe6277fe640 1 -- 172.21.15.83:0/2435702998 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe61000bc00 con 0x7fe6280ca970 2024-04-17T10:46:35.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe62cb54640 1 --2- 172.21.15.83:0/2435702998 >> v2:172.21.15.167:3300/0 conn(0x7fe62815ba00 0x7fe628072960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:46:35.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe62cb54640 1 --2- 172.21.15.83:0/2435702998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe610003a30 0x7fe610003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:35.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe627fff640 1 --2- 172.21.15.83:0/2435702998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe610004810 0x7fe610004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:35.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe62d355640 1 --2- 172.21.15.83:0/2435702998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6280ca970 0x7fe6100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:35.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe62cb54640 1 -- 172.21.15.83:0/2435702998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6280ca970 msgr2=0x7fe6100051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:35.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe62cb54640 1 --2- 172.21.15.83:0/2435702998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6280ca970 0x7fe6100051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:35.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe62cb54640 1 -- 172.21.15.83:0/2435702998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe610004810 msgr2=0x7fe610004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:35.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe62cb54640 1 --2- 172.21.15.83:0/2435702998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe610004810 0x7fe610004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:35.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.347+0000 7fe62cb54640 1 -- 172.21.15.83:0/2435702998 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe610003530 con 0x7fe610003a30 2024-04-17T10:46:35.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.351+0000 7fe62cb54640 1 --2- 172.21.15.83:0/2435702998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe610003a30 0x7fe610003e70 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fe62000e350 tx=0x7fe62001ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bdd682f5a55683 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:35.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.351+0000 7fe6277fe640 1 -- 172.21.15.83:0/2435702998 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe620009dc0 con 0x7fe610003a30 2024-04-17T10:46:35.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.351+0000 7fe6277fe640 1 -- 172.21.15.83:0/2435702998 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6200a5d30 con 0x7fe610003a30 2024-04-17T10:46:35.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.351+0000 7fe6277fe640 1 -- 172.21.15.83:0/2435702998 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6281605b0 con 0x7fe610003a30 2024-04-17T10:46:35.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.351+0000 7fe62edb8640 1 -- 172.21.15.83:0/2435702998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe610003a30 msgr2=0x7fe610003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:35.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.351+0000 7fe62edb8640 1 --2- 172.21.15.83:0/2435702998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe610003a30 0x7fe610003e70 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fe62000e350 tx=0x7fe62001ce30 comp rx=0 tx=0).stop 2024-04-17T10:46:35.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.351+0000 7fe62d355640 1 -- 172.21.15.83:0/2435702998 reap_dead start 2024-04-17T10:46:35.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.351+0000 7fe62edb8640 1 -- 172.21.15.83:0/2435702998 shutdown_connections 2024-04-17T10:46:35.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.351+0000 7fe62edb8640 1 -- 172.21.15.83:0/2435702998 >> 172.21.15.83:0/2435702998 conn(0x7fe62806abe0 msgr2=0x7fe62806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:35.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.351+0000 7fe62edb8640 1 -- 172.21.15.83:0/2435702998 shutdown_connections 2024-04-17T10:46:35.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.351+0000 7fe62edb8640 1 -- 172.21.15.83:0/2435702998 wait complete. 2024-04-17T10:46:35.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.351+0000 7fe62edb8640 1 Processor -- start 2024-04-17T10:46:35.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.351+0000 7fe62edb8640 1 -- start start 2024-04-17T10:46:35.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.355+0000 7fe62edb8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe610003a30 0x7fe628156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:35.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.355+0000 7fe62edb8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6280b3600 0x7fe628158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:35.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.355+0000 7fe62edb8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe628161d00 0x7fe6281621b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:35.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.355+0000 7fe62edb8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6280c6670 con 0x7fe6280b3600 2024-04-17T10:46:35.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.355+0000 7fe62edb8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6280c5140 con 0x7fe610003a30 2024-04-17T10:46:35.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.355+0000 7fe62edb8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6280c67f0 con 0x7fe628161d00 2024-04-17T10:46:35.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.355+0000 7fe627fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6280b3600 0x7fe628158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:35.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.355+0000 7fe627fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6280b3600 0x7fe628158770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32932/0 (socket says 172.21.15.83:32932) 2024-04-17T10:46:35.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.355+0000 7fe627fff640 1 -- 172.21.15.83:0/1442309441 learned_addr learned my addr 172.21.15.83:0/1442309441 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:35.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.355+0000 7fe62d355640 1 --2- 172.21.15.83:0/1442309441 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe628161d00 0x7fe6281621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:35.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.355+0000 7fe627fff640 1 -- 172.21.15.83:0/1442309441 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe628161d00 msgr2=0x7fe6281621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:35.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.355+0000 7fe62cb54640 1 --2- 172.21.15.83:0/1442309441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe610003a30 0x7fe628156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:35.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.355+0000 7fe627fff640 1 --2- 172.21.15.83:0/1442309441 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe628161d00 0x7fe6281621b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:35.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.355+0000 7fe627fff640 1 -- 172.21.15.83:0/1442309441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe610003a30 msgr2=0x7fe628156660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:35.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.355+0000 7fe627fff640 1 --2- 172.21.15.83:0/1442309441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe610003a30 0x7fe628156660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:35.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.355+0000 7fe627fff640 1 -- 172.21.15.83:0/1442309441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6280729c0 con 0x7fe6280b3600 2024-04-17T10:46:35.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.355+0000 7fe62d355640 1 --2- 172.21.15.83:0/1442309441 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe628161d00 0x7fe6281621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:35.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.355+0000 7fe627fff640 1 --2- 172.21.15.83:0/1442309441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6280b3600 0x7fe628158770 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fe6180115e0 tx=0x7fe618011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:35.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.355+0000 7fe6257fa640 1 -- 172.21.15.83:0/1442309441 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61800f8f0 con 0x7fe6280b3600 2024-04-17T10:46:35.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.355+0000 7fe6257fa640 1 -- 172.21.15.83:0/1442309441 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe61800fa90 con 0x7fe6280b3600 2024-04-17T10:46:35.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.355+0000 7fe62edb8640 1 -- 172.21.15.83:0/1442309441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe628156ba0 con 0x7fe6280b3600 2024-04-17T10:46:35.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.355+0000 7fe6257fa640 1 -- 172.21.15.83:0/1442309441 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61800fe40 con 0x7fe6280b3600 2024-04-17T10:46:35.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.355+0000 7fe62edb8640 1 -- 172.21.15.83:0/1442309441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6280b3ba0 con 0x7fe6280b3600 2024-04-17T10:46:35.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.355+0000 7fe62edb8640 1 -- 172.21.15.83:0/1442309441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe628074f90 con 0x7fe6280b3600 2024-04-17T10:46:35.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.355+0000 7fe6257fa640 1 -- 172.21.15.83:0/1442309441 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe618015460 con 0x7fe6280b3600 2024-04-17T10:46:35.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.355+0000 7fe6257fa640 1 --2- 172.21.15.83:0/1442309441 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe614066f60 0x7fe614069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:35.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.355+0000 7fe6257fa640 1 -- 172.21.15.83:0/1442309441 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fe618085940 con 0x7fe6280b3600 2024-04-17T10:46:35.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.355+0000 7fe62cb54640 1 --2- 172.21.15.83:0/1442309441 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe614066f60 0x7fe614069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:35.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.359+0000 7fe62cb54640 1 --2- 172.21.15.83:0/1442309441 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe614066f60 0x7fe614069420 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe6200ae000 tx=0x7fe620004ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:35.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.359+0000 7fe6257fa640 1 -- 172.21.15.83:0/1442309441 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe628156ba0 con 0x7fe6280b3600 2024-04-17T10:46:35.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:35.523+0000 7fe62edb8640 1 -- 172.21.15.83:0/1442309441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe628066ae0 con 0x7fe6280b3600 2024-04-17T10:46:36.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.203+0000 7fe6257fa640 1 -- 172.21.15.83:0/1442309441 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v87) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe618019030 con 0x7fe6280b3600 2024-04-17T10:46:36.200 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2024-04-17T10:46:36.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.207+0000 7fe62edb8640 1 -- 172.21.15.83:0/1442309441 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe614066f60 msgr2=0x7fe614069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:36.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.207+0000 7fe62edb8640 1 --2- 172.21.15.83:0/1442309441 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe614066f60 0x7fe614069420 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe6200ae000 tx=0x7fe620004ac0 comp rx=0 tx=0).stop 2024-04-17T10:46:36.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.207+0000 7fe62edb8640 1 -- 172.21.15.83:0/1442309441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6280b3600 msgr2=0x7fe628158770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:36.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.207+0000 7fe62edb8640 1 --2- 172.21.15.83:0/1442309441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6280b3600 0x7fe628158770 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fe6180115e0 tx=0x7fe618011aa0 comp rx=0 tx=0).stop 2024-04-17T10:46:36.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.207+0000 7fe62edb8640 1 -- 172.21.15.83:0/1442309441 shutdown_connections 2024-04-17T10:46:36.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.207+0000 7fe62edb8640 1 --2- 172.21.15.83:0/1442309441 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe614066f60 0x7fe614069420 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:36.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.207+0000 7fe62edb8640 1 --2- 172.21.15.83:0/1442309441 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe628161d00 0x7fe6281621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:36.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.207+0000 7fe62edb8640 1 --2- 172.21.15.83:0/1442309441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6280b3600 0x7fe628158770 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:36.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.207+0000 7fe62edb8640 1 --2- 172.21.15.83:0/1442309441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe610003a30 0x7fe628156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:36.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.207+0000 7fe62edb8640 1 -- 172.21.15.83:0/1442309441 >> 172.21.15.83:0/1442309441 conn(0x7fe62806abe0 msgr2=0x7fe6281605b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:36.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.207+0000 7fe62edb8640 1 -- 172.21.15.83:0/1442309441 shutdown_connections 2024-04-17T10:46:36.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.207+0000 7fe62edb8640 1 -- 172.21.15.83:0/1442309441 wait complete. 2024-04-17T10:46:36.238 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-17T10:46:36.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.363+0000 7f47115d7640 1 Processor -- start 2024-04-17T10:46:36.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.363+0000 7f47115d7640 1 -- start start 2024-04-17T10:46:36.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.367+0000 7f47115d7640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f470c15b6e0 0x7f470c071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:36.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.367+0000 7f47115d7640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f470c06c1f0 con 0x7f470c070470 2024-04-17T10:46:36.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.367+0000 7f47115d7640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f470c070a10 con 0x7f470c15bac0 2024-04-17T10:46:36.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.367+0000 7f47115d7640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f470c070cf0 con 0x7f470c15b6e0 2024-04-17T10:46:36.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.367+0000 7f470affd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f470c15b6e0 0x7f470c071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:36.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.367+0000 7f470affd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f470c15b6e0 0x7f470c071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32938/0 (socket says 172.21.15.83:32938) 2024-04-17T10:46:36.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.367+0000 7f470affd640 1 -- 172.21.15.83:0/1224379082 learned_addr learned my addr 172.21.15.83:0/1224379082 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:36.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.367+0000 7f470affd640 1 -- 172.21.15.83:0/1224379082 >> v1:172.21.15.167:6789/0 conn(0x7f470c15bac0 legacy=0x7f470c0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:36.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.367+0000 7f470affd640 1 -- 172.21.15.83:0/1224379082 >> v1:172.21.15.83:6789/0 conn(0x7f470c070470 legacy=0x7f470c06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:36.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.367+0000 7f470a7fc640 1 -- 172.21.15.83:0/1224379082 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2513665756 0 0) 0x7f470c06c1f0 con 0x7f470c070470 2024-04-17T10:46:36.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.367+0000 7f470affd640 1 -- 172.21.15.83:0/1224379082 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f470c070fd0 con 0x7f470c15b6e0 2024-04-17T10:46:36.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.367+0000 7f470affd640 1 --2- 172.21.15.83:0/1224379082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c15b6e0 0x7f470c071cc0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f46fc009870 tx=0x7f46fc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a958454b1f01c7e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:36.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.371+0000 7f470a7fc640 1 -- 172.21.15.83:0/1224379082 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46fc0b2070 con 0x7f470c15b6e0 2024-04-17T10:46:36.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.371+0000 7f470a7fc640 1 -- 172.21.15.83:0/1224379082 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46fc0a9070 con 0x7f470c15b6e0 2024-04-17T10:46:36.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.371+0000 7f470a7fc640 1 -- 172.21.15.83:0/1224379082 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46fc0ad880 con 0x7f470c15b6e0 2024-04-17T10:46:36.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.371+0000 7f47115d7640 1 -- 172.21.15.83:0/1224379082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c15b6e0 msgr2=0x7f470c071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:36.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.371+0000 7f47115d7640 1 --2- 172.21.15.83:0/1224379082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c15b6e0 0x7f470c071cc0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f46fc009870 tx=0x7f46fc01c560 comp rx=0 tx=0).stop 2024-04-17T10:46:36.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.371+0000 7f47115d7640 1 -- 172.21.15.83:0/1224379082 shutdown_connections 2024-04-17T10:46:36.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.371+0000 7f47115d7640 1 --2- 172.21.15.83:0/1224379082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c15b6e0 0x7f470c071cc0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:36.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.371+0000 7f47115d7640 1 -- 172.21.15.83:0/1224379082 >> 172.21.15.83:0/1224379082 conn(0x7f470c0b15b0 msgr2=0x7f470c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:36.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.371+0000 7f47115d7640 1 -- 172.21.15.83:0/1224379082 shutdown_connections 2024-04-17T10:46:36.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.371+0000 7f47115d7640 1 -- 172.21.15.83:0/1224379082 wait complete. 2024-04-17T10:46:36.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.371+0000 7f47115d7640 1 Processor -- start 2024-04-17T10:46:36.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.371+0000 7f47115d7640 1 -- start start 2024-04-17T10:46:36.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.371+0000 7f47115d7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c070470 0x7f470c13d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:36.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.371+0000 7f47115d7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f470c15b6e0 0x7f470c1f64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.371+0000 7f47115d7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f470c15bac0 0x7f470c0b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.371+0000 7f47115d7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c06cee0 con 0x7f470c070470 2024-04-17T10:46:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.371+0000 7f47115d7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c06b730 con 0x7f470c15bac0 2024-04-17T10:46:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.371+0000 7f47115d7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f470c06d060 con 0x7f470c15b6e0 2024-04-17T10:46:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.371+0000 7f470affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c070470 0x7f470c13d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.371+0000 7f470affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c070470 0x7f470c13d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32952/0 (socket says 172.21.15.83:32952) 2024-04-17T10:46:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.371+0000 7f470affd640 1 -- 172.21.15.83:0/2598105349 learned_addr learned my addr 172.21.15.83:0/2598105349 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.371+0000 7f470b7fe640 1 --2- 172.21.15.83:0/2598105349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f470c15bac0 0x7f470c0b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.371+0000 7f47027fc640 1 --2- 172.21.15.83:0/2598105349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f470c15b6e0 0x7f470c1f64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.371+0000 7f470affd640 1 -- 172.21.15.83:0/2598105349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f470c15b6e0 msgr2=0x7f470c1f64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.371+0000 7f470affd640 1 --2- 172.21.15.83:0/2598105349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f470c15b6e0 0x7f470c1f64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.371+0000 7f470affd640 1 -- 172.21.15.83:0/2598105349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f470c15bac0 msgr2=0x7f470c0b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.371+0000 7f470affd640 1 --2- 172.21.15.83:0/2598105349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f470c15bac0 0x7f470c0b2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:36.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.371+0000 7f470affd640 1 -- 172.21.15.83:0/2598105349 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f470c145f30 con 0x7f470c070470 2024-04-17T10:46:36.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.371+0000 7f47027fc640 1 --2- 172.21.15.83:0/2598105349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f470c15b6e0 0x7f470c1f64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:36.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.371+0000 7f470affd640 1 --2- 172.21.15.83:0/2598105349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c070470 0x7f470c13d120 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f46fc0b0040 tx=0x7f46fc01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:36.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.375+0000 7f4703fff640 1 -- 172.21.15.83:0/2598105349 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46fc0b2040 con 0x7f470c070470 2024-04-17T10:46:36.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.375+0000 7f4703fff640 1 -- 172.21.15.83:0/2598105349 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46fc0be070 con 0x7f470c070470 2024-04-17T10:46:36.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.375+0000 7f47115d7640 1 -- 172.21.15.83:0/2598105349 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f470c146210 con 0x7f470c070470 2024-04-17T10:46:36.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.375+0000 7f4703fff640 1 -- 172.21.15.83:0/2598105349 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46fc0b8880 con 0x7f470c070470 2024-04-17T10:46:36.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.375+0000 7f47115d7640 1 -- 172.21.15.83:0/2598105349 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f470c13d660 con 0x7f470c070470 2024-04-17T10:46:36.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.375+0000 7f47115d7640 1 -- 172.21.15.83:0/2598105349 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f470c074f90 con 0x7f470c070470 2024-04-17T10:46:36.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.375+0000 7f4703fff640 1 -- 172.21.15.83:0/2598105349 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f46fc0ab070 con 0x7f470c070470 2024-04-17T10:46:36.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.375+0000 7f4703fff640 1 --2- 172.21.15.83:0/2598105349 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f46ec066f60 0x7f46ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:36.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.375+0000 7f4703fff640 1 -- 172.21.15.83:0/2598105349 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f46fc11f370 con 0x7f470c070470 2024-04-17T10:46:36.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.375+0000 7f47027fc640 1 --2- 172.21.15.83:0/2598105349 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f46ec066f60 0x7f46ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:36.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.375+0000 7f47027fc640 1 --2- 172.21.15.83:0/2598105349 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f46ec066f60 0x7f46ec069420 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f470c0cdb40 tx=0x7f46f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:36.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.379+0000 7f4703fff640 1 -- 172.21.15.83:0/2598105349 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f46fc016150 con 0x7f470c070470 2024-04-17T10:46:36.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:36.575+0000 7f47115d7640 1 -- 172.21.15.83:0/2598105349 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f470c0cd890 con 0x7f470c070470 2024-04-17T10:46:37.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.207+0000 7f4703fff640 1 -- 172.21.15.83:0/2598105349 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v54) v1 ==== 174+0+0 (secure 0 0 0) 0x7f46fc0f04a0 con 0x7f470c070470 2024-04-17T10:46:37.202 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 12 and data pool 13 2024-04-17T10:46:37.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.211+0000 7f47115d7640 1 -- 172.21.15.83:0/2598105349 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f46ec066f60 msgr2=0x7f46ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:37.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.211+0000 7f47115d7640 1 --2- 172.21.15.83:0/2598105349 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f46ec066f60 0x7f46ec069420 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f470c0cdb40 tx=0x7f46f4009040 comp rx=0 tx=0).stop 2024-04-17T10:46:37.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.211+0000 7f47115d7640 1 -- 172.21.15.83:0/2598105349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c070470 msgr2=0x7f470c13d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:37.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.211+0000 7f47115d7640 1 --2- 172.21.15.83:0/2598105349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c070470 0x7f470c13d120 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f46fc0b0040 tx=0x7f46fc01ada0 comp rx=0 tx=0).stop 2024-04-17T10:46:37.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.211+0000 7f47115d7640 1 -- 172.21.15.83:0/2598105349 shutdown_connections 2024-04-17T10:46:37.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.211+0000 7f47115d7640 1 --2- 172.21.15.83:0/2598105349 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f46ec066f60 0x7f46ec069420 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:37.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.211+0000 7f47115d7640 1 --2- 172.21.15.83:0/2598105349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f470c15bac0 0x7f470c0b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:37.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.211+0000 7f47115d7640 1 --2- 172.21.15.83:0/2598105349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f470c15b6e0 0x7f470c1f64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:37.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.211+0000 7f47115d7640 1 --2- 172.21.15.83:0/2598105349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470c070470 0x7f470c13d120 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:37.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.211+0000 7f47115d7640 1 -- 172.21.15.83:0/2598105349 >> 172.21.15.83:0/2598105349 conn(0x7f470c0b15b0 msgr2=0x7f470c0b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:37.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.211+0000 7f47115d7640 1 -- 172.21.15.83:0/2598105349 shutdown_connections 2024-04-17T10:46:37.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.211+0000 7f47115d7640 1 -- 172.21.15.83:0/2598105349 wait complete. 2024-04-17T10:46:37.248 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:46:37.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.367+0000 7feded924640 1 Processor -- start 2024-04-17T10:46:37.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.367+0000 7feded924640 1 -- start start 2024-04-17T10:46:37.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.371+0000 7feded924640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fede80aedd0 0x7fede806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:37.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.371+0000 7feded924640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fede815e440 con 0x7fede815db50 2024-04-17T10:46:37.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.371+0000 7feded924640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fede815e720 con 0x7fede806dae0 2024-04-17T10:46:37.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.371+0000 7feded924640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fede815ea00 con 0x7fede80aedd0 2024-04-17T10:46:37.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.371+0000 7fede6ffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fede815db50 0x7fede8070470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:37356/0 (socket says 172.21.15.83:37356) 2024-04-17T10:46:37.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.371+0000 7fede6ffd640 1 -- 172.21.15.83:0/99908112 learned_addr learned my addr 172.21.15.83:0/99908112 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:46:37.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.371+0000 7fede5ffb640 1 -- 172.21.15.83:0/99908112 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2642622413 0 0) 0x7fede815e440 con 0x7fede815db50 2024-04-17T10:46:37.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.371+0000 7fede5ffb640 1 -- 172.21.15.83:0/99908112 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedd0003530 con 0x7fede815db50 2024-04-17T10:46:37.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.371+0000 7fede67fc640 1 --2- 172.21.15.83:0/99908112 >> v2:172.21.15.167:3300/0 conn(0x7fede80aedd0 0x7fede806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:37.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.371+0000 7fede5ffb640 1 -- 172.21.15.83:0/99908112 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 165515468 0 0) 0x7fedd0003530 con 0x7fede815db50 2024-04-17T10:46:37.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.371+0000 7fede5ffb640 1 -- 172.21.15.83:0/99908112 >> v2:172.21.15.167:3300/0 conn(0x7fede80aedd0 msgr2=0x7fede806d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:37.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.371+0000 7fede5ffb640 1 --2- 172.21.15.83:0/99908112 >> v2:172.21.15.167:3300/0 conn(0x7fede80aedd0 0x7fede806d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:37.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.371+0000 7fede5ffb640 1 -- 172.21.15.83:0/99908112 >> v1:172.21.15.167:6789/0 conn(0x7fede806dae0 legacy=0x7fede80b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:37.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.371+0000 7fede5ffb640 1 -- 172.21.15.83:0/99908112 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede815ece0 con 0x7fede815db50 2024-04-17T10:46:37.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.371+0000 7fede5ffb640 1 -- 172.21.15.83:0/99908112 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fede0001680 con 0x7fede815db50 2024-04-17T10:46:37.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.371+0000 7fede5ffb640 1 -- 172.21.15.83:0/99908112 >> v1:172.21.15.83:6789/0 conn(0x7fede815db50 legacy=0x7fede8070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:37.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.371+0000 7fede5ffb640 1 --2- 172.21.15.83:0/99908112 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fedd00039c0 0x7fedd0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:37.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.371+0000 7fede5ffb640 1 --2- 172.21.15.83:0/99908112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd0004810 0x7fedd0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:37.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.371+0000 7fede5ffb640 1 --2- 172.21.15.83:0/99908112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedd00051d0 0x7fedd000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:37.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.371+0000 7fede5ffb640 1 -- 172.21.15.83:0/99908112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede815ea00 con 0x7fedd0004810 2024-04-17T10:46:37.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.371+0000 7fede5ffb640 1 -- 172.21.15.83:0/99908112 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd000c100 con 0x7fedd00051d0 2024-04-17T10:46:37.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.371+0000 7fede5ffb640 1 -- 172.21.15.83:0/99908112 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd000c300 con 0x7fedd00039c0 2024-04-17T10:46:37.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.371+0000 7fede67fc640 1 --2- 172.21.15.83:0/99908112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd0004810 0x7fedd0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:37.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.371+0000 7fede77fe640 1 --2- 172.21.15.83:0/99908112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedd00051d0 0x7fedd000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:37.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.371+0000 7fede67fc640 1 -- 172.21.15.83:0/99908112 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fedd00039c0 msgr2=0x7fedd0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:37.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.371+0000 7fede67fc640 1 --2- 172.21.15.83:0/99908112 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fedd00039c0 0x7fedd0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:37.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.371+0000 7fede67fc640 1 -- 172.21.15.83:0/99908112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedd00051d0 msgr2=0x7fedd000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:37.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.371+0000 7fede67fc640 1 --2- 172.21.15.83:0/99908112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedd00051d0 0x7fedd000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:37.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.371+0000 7fede67fc640 1 -- 172.21.15.83:0/99908112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd0003530 con 0x7fedd0004810 2024-04-17T10:46:37.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.371+0000 7fede77fe640 1 --2- 172.21.15.83:0/99908112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedd00051d0 0x7fedd000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:37.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.371+0000 7fede77fe640 1 -- 172.21.15.83:0/99908112 reap_dead start 2024-04-17T10:46:37.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.371+0000 7fede67fc640 1 --2- 172.21.15.83:0/99908112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd0004810 0x7fedd0004c90 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7feddc007330 tx=0x7feddc01ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d167fa4cbd4b0f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:37.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7fede5ffb640 1 -- 172.21.15.83:0/99908112 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feddc0b2070 con 0x7fedd0004810 2024-04-17T10:46:37.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7fede5ffb640 1 -- 172.21.15.83:0/99908112 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feddc0a5d20 con 0x7fedd0004810 2024-04-17T10:46:37.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7fede5ffb640 1 -- 172.21.15.83:0/99908112 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feddc009be0 con 0x7fedd0004810 2024-04-17T10:46:37.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7feded924640 1 -- 172.21.15.83:0/99908112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd0004810 msgr2=0x7fedd0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:37.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7feded924640 1 --2- 172.21.15.83:0/99908112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd0004810 0x7fedd0004c90 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7feddc007330 tx=0x7feddc01ce20 comp rx=0 tx=0).stop 2024-04-17T10:46:37.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7feded924640 1 -- 172.21.15.83:0/99908112 shutdown_connections 2024-04-17T10:46:37.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7feded924640 1 --2- 172.21.15.83:0/99908112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd0004810 0x7fedd0004c90 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:37.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7feded924640 1 -- 172.21.15.83:0/99908112 >> 172.21.15.83:0/99908112 conn(0x7fede806abe0 msgr2=0x7fede806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:37.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7feded924640 1 -- 172.21.15.83:0/99908112 shutdown_connections 2024-04-17T10:46:37.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7feded924640 1 -- 172.21.15.83:0/99908112 wait complete. 2024-04-17T10:46:37.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7feded924640 1 Processor -- start 2024-04-17T10:46:37.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7feded924640 1 -- start start 2024-04-17T10:46:37.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7feded924640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd0004810 0x7fede8155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:37.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7feded924640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fede81e50c0 0x7fede81f44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:37.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7feded924640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fede80b2ec0 0x7fede80b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:37.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7feded924640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede815e0b0 con 0x7fedd0004810 2024-04-17T10:46:37.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7feded924640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede815df30 con 0x7fede80b2ec0 2024-04-17T10:46:37.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7feded924640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fede806d5a0 con 0x7fede81e50c0 2024-04-17T10:46:37.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7fede6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd0004810 0x7fede8155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:37.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7fede6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd0004810 0x7fede8155830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47508/0 (socket says 172.21.15.83:47508) 2024-04-17T10:46:37.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7fede6ffd640 1 -- 172.21.15.83:0/2775704996 learned_addr learned my addr 172.21.15.83:0/2775704996 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:37.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7fede67fc640 1 --2- 172.21.15.83:0/2775704996 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fede81e50c0 0x7fede81f44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:37.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7fede6ffd640 1 -- 172.21.15.83:0/2775704996 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fede81e50c0 msgr2=0x7fede81f44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:37.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7fede6ffd640 1 --2- 172.21.15.83:0/2775704996 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fede81e50c0 0x7fede81f44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:37.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7fede6ffd640 1 -- 172.21.15.83:0/2775704996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fede80b2ec0 msgr2=0x7fede80b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:37.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7fede77fe640 1 --2- 172.21.15.83:0/2775704996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fede80b2ec0 0x7fede80b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:37.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7fede6ffd640 1 --2- 172.21.15.83:0/2775704996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fede80b2ec0 0x7fede80b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:37.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7fede6ffd640 1 -- 172.21.15.83:0/2775704996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede8072700 con 0x7fedd0004810 2024-04-17T10:46:37.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7fede77fe640 1 --2- 172.21.15.83:0/2775704996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fede80b2ec0 0x7fede80b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:37.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7fede6ffd640 1 --2- 172.21.15.83:0/2775704996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd0004810 0x7fede8155830 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fede000f460 tx=0x7fede000f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:37.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7fedcffff640 1 -- 172.21.15.83:0/2775704996 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede001b070 con 0x7fedd0004810 2024-04-17T10:46:37.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7fedcffff640 1 -- 172.21.15.83:0/2775704996 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede0007d00 con 0x7fedd0004810 2024-04-17T10:46:37.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7feded924640 1 -- 172.21.15.83:0/2775704996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede8155ef0 con 0x7fedd0004810 2024-04-17T10:46:37.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7fedcffff640 1 -- 172.21.15.83:0/2775704996 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede000a2f0 con 0x7fedd0004810 2024-04-17T10:46:37.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.375+0000 7feded924640 1 -- 172.21.15.83:0/2775704996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede81564b0 con 0x7fedd0004810 2024-04-17T10:46:37.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.379+0000 7feded924640 1 -- 172.21.15.83:0/2775704996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fede8074f90 con 0x7fedd0004810 2024-04-17T10:46:37.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.379+0000 7fedcffff640 1 -- 172.21.15.83:0/2775704996 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fede001e070 con 0x7fedd0004810 2024-04-17T10:46:37.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.379+0000 7fedcffff640 1 --2- 172.21.15.83:0/2775704996 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fedd4066f90 0x7fedd4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:37.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.379+0000 7fedcffff640 1 -- 172.21.15.83:0/2775704996 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fede0087470 con 0x7fedd0004810 2024-04-17T10:46:37.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.379+0000 7fede67fc640 1 --2- 172.21.15.83:0/2775704996 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fedd4066f90 0x7fedd4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:37.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.379+0000 7fede67fc640 1 --2- 172.21.15.83:0/2775704996 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fedd4066f90 0x7fedd4069450 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7feddc0af000 tx=0x7feddc00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:37.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.383+0000 7fedcffff640 1 -- 172.21.15.83:0/2775704996 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fede00583e0 con 0x7fedd0004810 2024-04-17T10:46:37.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.543+0000 7feded924640 1 -- 172.21.15.83:0/2775704996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fede80b2af0 con 0x7fedd0004810 2024-04-17T10:46:37.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.543+0000 7fedcffff640 1 -- 172.21.15.83:0/2775704996 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16601 (secure 0 0 0) 0x7fede0057ce0 con 0x7fedd0004810 2024-04-17T10:46:37.540 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:46:37.540 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":88,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:46:36.578609+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:46:34.779174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-17T10:46:35.563972+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:46:37.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.547+0000 7feded924640 1 -- 172.21.15.83:0/2775704996 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fedd4066f90 msgr2=0x7fedd4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:37.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.547+0000 7feded924640 1 --2- 172.21.15.83:0/2775704996 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fedd4066f90 0x7fedd4069450 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7feddc0af000 tx=0x7feddc00b040 comp rx=0 tx=0).stop 2024-04-17T10:46:37.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.547+0000 7feded924640 1 -- 172.21.15.83:0/2775704996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd0004810 msgr2=0x7fede8155830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:37.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.547+0000 7feded924640 1 --2- 172.21.15.83:0/2775704996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd0004810 0x7fede8155830 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fede000f460 tx=0x7fede000f920 comp rx=0 tx=0).stop 2024-04-17T10:46:37.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.547+0000 7feded924640 1 -- 172.21.15.83:0/2775704996 shutdown_connections 2024-04-17T10:46:37.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.547+0000 7feded924640 1 --2- 172.21.15.83:0/2775704996 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fedd4066f90 0x7fedd4069450 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:37.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.547+0000 7feded924640 1 --2- 172.21.15.83:0/2775704996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fede80b2ec0 0x7fede80b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:37.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.547+0000 7feded924640 1 --2- 172.21.15.83:0/2775704996 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fede81e50c0 0x7fede81f44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:37.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.547+0000 7feded924640 1 --2- 172.21.15.83:0/2775704996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd0004810 0x7fede8155830 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:37.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.547+0000 7feded924640 1 -- 172.21.15.83:0/2775704996 >> 172.21.15.83:0/2775704996 conn(0x7fede806abe0 msgr2=0x7fede8070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:37.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.547+0000 7feded924640 1 -- 172.21.15.83:0/2775704996 shutdown_connections 2024-04-17T10:46:37.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.547+0000 7feded924640 1 -- 172.21.15.83:0/2775704996 wait complete. 2024-04-17T10:46:37.560 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:46:37.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.671+0000 7fce64ede640 1 Processor -- start 2024-04-17T10:46:37.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.671+0000 7fce64ede640 1 -- start start 2024-04-17T10:46:37.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.675+0000 7fce64ede640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce6015ba30 0x7fce60072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:37.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.675+0000 7fce64ede640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fce600cfbe0 0x7fce6006c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:37.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.675+0000 7fce64ede640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce6006e320 con 0x7fce6015c960 2024-04-17T10:46:37.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.675+0000 7fce64ede640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce60071ea0 con 0x7fce600cfbe0 2024-04-17T10:46:37.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.675+0000 7fce64ede640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6006e040 con 0x7fce6015ba30 2024-04-17T10:46:37.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.675+0000 7fce5ed9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fce600cfbe0 0x7fce6006c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:37.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.675+0000 7fce5ed9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fce600cfbe0 0x7fce6006c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47516/0 (socket says 172.21.15.83:47516) 2024-04-17T10:46:37.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.675+0000 7fce5ed9d640 1 -- 172.21.15.83:0/3300150483 learned_addr learned my addr 172.21.15.83:0/3300150483 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:37.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.675+0000 7fce5ed9d640 1 -- 172.21.15.83:0/3300150483 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce6015ba30 msgr2=0x7fce60072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:37.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.675+0000 7fce5e59c640 1 --2- 172.21.15.83:0/3300150483 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce6015ba30 0x7fce60072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:37.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.675+0000 7fce5ed9d640 1 --2- 172.21.15.83:0/3300150483 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce6015ba30 0x7fce60072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:37.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.675+0000 7fce5ed9d640 1 -- 172.21.15.83:0/3300150483 >> v1:172.21.15.167:6789/0 conn(0x7fce6015c960 legacy=0x7fce600cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:37.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.675+0000 7fce5ed9d640 1 -- 172.21.15.83:0/3300150483 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce60071850 con 0x7fce600cfbe0 2024-04-17T10:46:37.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.675+0000 7fce5ed9d640 1 --2- 172.21.15.83:0/3300150483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce600cfbe0 0x7fce6006c490 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fce58009e50 tx=0x7fce5801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d333aeac7f6bf8a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:37.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.675+0000 7fce5dd9b640 1 -- 172.21.15.83:0/3300150483 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce580a56b0 con 0x7fce600cfbe0 2024-04-17T10:46:37.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.675+0000 7fce5dd9b640 1 -- 172.21.15.83:0/3300150483 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce580a5850 con 0x7fce600cfbe0 2024-04-17T10:46:37.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.675+0000 7fce5dd9b640 1 -- 172.21.15.83:0/3300150483 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce580a5c20 con 0x7fce600cfbe0 2024-04-17T10:46:37.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.679+0000 7fce64ede640 1 -- 172.21.15.83:0/3300150483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce600cfbe0 msgr2=0x7fce6006c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:37.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.679+0000 7fce64ede640 1 --2- 172.21.15.83:0/3300150483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce600cfbe0 0x7fce6006c490 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fce58009e50 tx=0x7fce5801c9a0 comp rx=0 tx=0).stop 2024-04-17T10:46:37.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.679+0000 7fce64ede640 1 -- 172.21.15.83:0/3300150483 shutdown_connections 2024-04-17T10:46:37.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.679+0000 7fce64ede640 1 --2- 172.21.15.83:0/3300150483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce600cfbe0 0x7fce6006c490 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:37.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.679+0000 7fce64ede640 1 --2- 172.21.15.83:0/3300150483 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce6015ba30 0x7fce60072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:37.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.679+0000 7fce64ede640 1 -- 172.21.15.83:0/3300150483 >> 172.21.15.83:0/3300150483 conn(0x7fce6006abe0 msgr2=0x7fce6006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:37.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.679+0000 7fce64ede640 1 -- 172.21.15.83:0/3300150483 shutdown_connections 2024-04-17T10:46:37.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.679+0000 7fce64ede640 1 -- 172.21.15.83:0/3300150483 wait complete. 2024-04-17T10:46:37.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.679+0000 7fce64ede640 1 Processor -- start 2024-04-17T10:46:37.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.679+0000 7fce64ede640 1 -- start start 2024-04-17T10:46:37.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.679+0000 7fce64ede640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce600cfbe0 0x7fce601499a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:37.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.679+0000 7fce64ede640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce6015ba30 0x7fce6014cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:37.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.679+0000 7fce64ede640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce6015c960 0x7fce6014d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:37.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.679+0000 7fce64ede640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6015cee0 con 0x7fce6015c960 2024-04-17T10:46:37.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.679+0000 7fce64ede640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6015cd60 con 0x7fce6015ba30 2024-04-17T10:46:37.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.679+0000 7fce64ede640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce60168520 con 0x7fce600cfbe0 2024-04-17T10:46:37.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.679+0000 7fce5ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce6015c960 0x7fce6014d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:37.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.679+0000 7fce5ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce6015c960 0x7fce6014d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47526/0 (socket says 172.21.15.83:47526) 2024-04-17T10:46:37.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.679+0000 7fce5ed9d640 1 -- 172.21.15.83:0/1476719239 learned_addr learned my addr 172.21.15.83:0/1476719239 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:37.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.679+0000 7fce5e59c640 1 --2- 172.21.15.83:0/1476719239 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce600cfbe0 0x7fce601499a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:37.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.679+0000 7fce55d9b640 1 --2- 172.21.15.83:0/1476719239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce6015ba30 0x7fce6014cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:37.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.679+0000 7fce5ed9d640 1 -- 172.21.15.83:0/1476719239 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce600cfbe0 msgr2=0x7fce601499a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:37.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.679+0000 7fce5ed9d640 1 --2- 172.21.15.83:0/1476719239 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce600cfbe0 0x7fce601499a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:37.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.679+0000 7fce5ed9d640 1 -- 172.21.15.83:0/1476719239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce6015ba30 msgr2=0x7fce6014cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:37.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.679+0000 7fce5ed9d640 1 --2- 172.21.15.83:0/1476719239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce6015ba30 0x7fce6014cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:37.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.679+0000 7fce5ed9d640 1 -- 172.21.15.83:0/1476719239 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6014a0c0 con 0x7fce6015c960 2024-04-17T10:46:37.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.679+0000 7fce5ed9d640 1 --2- 172.21.15.83:0/1476719239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce6015c960 0x7fce6014d270 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fce5801a5d0 tx=0x7fce58019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:37.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.679+0000 7fce577fe640 1 -- 172.21.15.83:0/1476719239 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce580bc020 con 0x7fce6015c960 2024-04-17T10:46:37.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.679+0000 7fce577fe640 1 -- 172.21.15.83:0/1476719239 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce580b8670 con 0x7fce6015c960 2024-04-17T10:46:37.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.679+0000 7fce64ede640 1 -- 172.21.15.83:0/1476719239 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce6014a360 con 0x7fce6015c960 2024-04-17T10:46:37.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.679+0000 7fce577fe640 1 -- 172.21.15.83:0/1476719239 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce580b8a40 con 0x7fce6015c960 2024-04-17T10:46:37.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.679+0000 7fce64ede640 1 -- 172.21.15.83:0/1476719239 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce6014a920 con 0x7fce6015c960 2024-04-17T10:46:37.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.683+0000 7fce64ede640 1 -- 172.21.15.83:0/1476719239 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce60074f90 con 0x7fce6015c960 2024-04-17T10:46:37.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.683+0000 7fce577fe640 1 -- 172.21.15.83:0/1476719239 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fce580bf020 con 0x7fce6015c960 2024-04-17T10:46:37.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.683+0000 7fce577fe640 1 --2- 172.21.15.83:0/1476719239 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fce40066f60 0x7fce40069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:37.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.683+0000 7fce577fe640 1 -- 172.21.15.83:0/1476719239 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce581201e0 con 0x7fce6015c960 2024-04-17T10:46:37.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.683+0000 7fce5e59c640 1 --2- 172.21.15.83:0/1476719239 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fce40066f60 0x7fce40069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:37.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.683+0000 7fce5e59c640 1 --2- 172.21.15.83:0/1476719239 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fce40066f60 0x7fce40069420 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fce5000b800 tx=0x7fce50008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:37.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.687+0000 7fce577fe640 1 -- 172.21.15.83:0/1476719239 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fce580c6030 con 0x7fce6015c960 2024-04-17T10:46:37.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.843+0000 7fce64ede640 1 -- 172.21.15.83:0/1476719239 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce60152c00 con 0x7fce6015c960 2024-04-17T10:46:37.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.847+0000 7fce577fe640 1 -- 172.21.15.83:0/1476719239 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16601 (secure 0 0 0) 0x7fce58016620 con 0x7fce6015c960 2024-04-17T10:46:37.843 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:46:37.843 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":88,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:46:36.578609+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:46:34.779174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-17T10:46:35.563972+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:46:37.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.847+0000 7fce64ede640 1 -- 172.21.15.83:0/1476719239 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fce40066f60 msgr2=0x7fce40069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:37.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.847+0000 7fce64ede640 1 --2- 172.21.15.83:0/1476719239 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fce40066f60 0x7fce40069420 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fce5000b800 tx=0x7fce50008040 comp rx=0 tx=0).stop 2024-04-17T10:46:37.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.847+0000 7fce64ede640 1 -- 172.21.15.83:0/1476719239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce6015c960 msgr2=0x7fce6014d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:37.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.847+0000 7fce64ede640 1 --2- 172.21.15.83:0/1476719239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce6015c960 0x7fce6014d270 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fce5801a5d0 tx=0x7fce58019740 comp rx=0 tx=0).stop 2024-04-17T10:46:37.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.847+0000 7fce64ede640 1 -- 172.21.15.83:0/1476719239 shutdown_connections 2024-04-17T10:46:37.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.847+0000 7fce64ede640 1 --2- 172.21.15.83:0/1476719239 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fce40066f60 0x7fce40069420 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:37.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.847+0000 7fce64ede640 1 --2- 172.21.15.83:0/1476719239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce6015c960 0x7fce6014d270 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:37.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.847+0000 7fce64ede640 1 --2- 172.21.15.83:0/1476719239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce6015ba30 0x7fce6014cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:37.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.847+0000 7fce64ede640 1 --2- 172.21.15.83:0/1476719239 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce600cfbe0 0x7fce601499a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:37.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.847+0000 7fce64ede640 1 -- 172.21.15.83:0/1476719239 >> 172.21.15.83:0/1476719239 conn(0x7fce6006abe0 msgr2=0x7fce600b1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:37.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.847+0000 7fce64ede640 1 -- 172.21.15.83:0/1476719239 shutdown_connections 2024-04-17T10:46:37.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.847+0000 7fce64ede640 1 -- 172.21.15.83:0/1476719239 wait complete. 2024-04-17T10:46:37.864 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-17T10:46:37.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.979+0000 7f18f9dc6640 1 Processor -- start 2024-04-17T10:46:37.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.979+0000 7f18f9dc6640 1 -- start start 2024-04-17T10:46:37.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.979+0000 7f18f9dc6640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f18f4154670 0x7f18f40aea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:37.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.979+0000 7f18f9dc6640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f18f40b4090 0x7f18f406c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:37.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.979+0000 7f18f9dc6640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18f40b2ff0 con 0x7f18f40b4520 2024-04-17T10:46:37.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.979+0000 7f18f9dc6640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f4154a50 con 0x7f18f4154670 2024-04-17T10:46:37.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.979+0000 7f18f9dc6640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f40b32d0 con 0x7f18f40b4090 2024-04-17T10:46:37.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.983+0000 7f18f37fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f18f4154670 0x7f18f40aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:37.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.983+0000 7f18f3fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f18f40b4520 0x7f18f4070470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:37364/0 (socket says 172.21.15.83:37364) 2024-04-17T10:46:37.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.983+0000 7f18f3fff640 1 -- 172.21.15.83:0/2839896987 learned_addr learned my addr 172.21.15.83:0/2839896987 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:46:37.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.983+0000 7f18f37fe640 1 -- 172.21.15.83:0/2839896987 >> v2:172.21.15.167:3300/0 conn(0x7f18f40b4090 msgr2=0x7f18f406c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:37.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.983+0000 7f18eaffd640 1 --2- 172.21.15.83:0/2839896987 >> v2:172.21.15.167:3300/0 conn(0x7f18f40b4090 0x7f18f406c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:37.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.983+0000 7f18f37fe640 1 --2- 172.21.15.83:0/2839896987 >> v2:172.21.15.167:3300/0 conn(0x7f18f40b4090 0x7f18f406c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:37.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.983+0000 7f18f37fe640 1 -- 172.21.15.83:0/2839896987 >> v1:172.21.15.83:6789/0 conn(0x7f18f40b4520 legacy=0x7f18f4070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:37.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.983+0000 7f18f2ffd640 1 -- 172.21.15.83:0/2839896987 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4064557825 0 0) 0x7f18f40b2ff0 con 0x7f18f40b4520 2024-04-17T10:46:37.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.983+0000 7f18f37fe640 1 -- 172.21.15.83:0/2839896987 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f40b35b0 con 0x7f18f4154670 2024-04-17T10:46:37.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.983+0000 7f18f37fe640 1 --2- 172.21.15.83:0/2839896987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18f4154670 0x7f18f40aea50 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f18f40728a0 tx=0x7f18e401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa895d56b68f8ef server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:37.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.983+0000 7f18f2ffd640 1 -- 172.21.15.83:0/2839896987 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e40ab650 con 0x7f18f4154670 2024-04-17T10:46:37.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.983+0000 7f18f2ffd640 1 -- 172.21.15.83:0/2839896987 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e40b0070 con 0x7f18f4154670 2024-04-17T10:46:37.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.983+0000 7f18f2ffd640 1 -- 172.21.15.83:0/2839896987 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e40aba10 con 0x7f18f4154670 2024-04-17T10:46:37.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.983+0000 7f18f9dc6640 1 -- 172.21.15.83:0/2839896987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18f4154670 msgr2=0x7f18f40aea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:37.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.983+0000 7f18f9dc6640 1 --2- 172.21.15.83:0/2839896987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18f4154670 0x7f18f40aea50 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f18f40728a0 tx=0x7f18e401c450 comp rx=0 tx=0).stop 2024-04-17T10:46:37.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.983+0000 7f18f9dc6640 1 -- 172.21.15.83:0/2839896987 shutdown_connections 2024-04-17T10:46:37.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.983+0000 7f18f9dc6640 1 --2- 172.21.15.83:0/2839896987 >> v2:172.21.15.167:3300/0 conn(0x7f18f40b4090 0x7f18f406c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:37.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.983+0000 7f18f9dc6640 1 --2- 172.21.15.83:0/2839896987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18f4154670 0x7f18f40aea50 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:37.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.983+0000 7f18f9dc6640 1 -- 172.21.15.83:0/2839896987 >> 172.21.15.83:0/2839896987 conn(0x7f18f40b15b0 msgr2=0x7f18f40b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:37.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.983+0000 7f18f9dc6640 1 -- 172.21.15.83:0/2839896987 shutdown_connections 2024-04-17T10:46:37.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.983+0000 7f18f9dc6640 1 -- 172.21.15.83:0/2839896987 wait complete. 2024-04-17T10:46:37.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.987+0000 7f18f9dc6640 1 Processor -- start 2024-04-17T10:46:37.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.987+0000 7f18f9dc6640 1 -- start start 2024-04-17T10:46:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.987+0000 7f18f9dc6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18f40b4090 0x7f18f414a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.987+0000 7f18f9dc6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18f40b4520 0x7f18f4152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.987+0000 7f18f9dc6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18f4154670 0x7f18f414b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.987+0000 7f18f9dc6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f40aef90 con 0x7f18f40b4520 2024-04-17T10:46:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.987+0000 7f18f9dc6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f406cec0 con 0x7f18f40b4090 2024-04-17T10:46:37.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.987+0000 7f18f9dc6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f40af110 con 0x7f18f4154670 2024-04-17T10:46:37.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.987+0000 7f18eaffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18f40b4520 0x7f18f4152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:37.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.987+0000 7f18eaffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18f40b4520 0x7f18f4152620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47534/0 (socket says 172.21.15.83:47534) 2024-04-17T10:46:37.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.987+0000 7f18eaffd640 1 -- 172.21.15.83:0/1135501860 learned_addr learned my addr 172.21.15.83:0/1135501860 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:37.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.987+0000 7f18f37fe640 1 --2- 172.21.15.83:0/1135501860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18f40b4090 0x7f18f414a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:37.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.987+0000 7f18eaffd640 1 -- 172.21.15.83:0/1135501860 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18f4154670 msgr2=0x7f18f414b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:37.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.987+0000 7f18eaffd640 1 --2- 172.21.15.83:0/1135501860 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18f4154670 0x7f18f414b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:37.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.987+0000 7f18eaffd640 1 -- 172.21.15.83:0/1135501860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18f40b4090 msgr2=0x7f18f414a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:37.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.987+0000 7f18eaffd640 1 --2- 172.21.15.83:0/1135501860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18f40b4090 0x7f18f414a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:37.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.987+0000 7f18eaffd640 1 -- 172.21.15.83:0/1135501860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f40cf510 con 0x7f18f40b4520 2024-04-17T10:46:37.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.987+0000 7f18f37fe640 1 --2- 172.21.15.83:0/1135501860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18f40b4090 0x7f18f414a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:37.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.987+0000 7f18eaffd640 1 --2- 172.21.15.83:0/1135501860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18f40b4520 0x7f18f4152620 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00c820 tx=0x7f18dc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.987+0000 7f18f0ff9640 1 -- 172.21.15.83:0/1135501860 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc015070 con 0x7f18f40b4520 2024-04-17T10:46:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.987+0000 7f18f0ff9640 1 -- 172.21.15.83:0/1135501860 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18dc007a30 con 0x7f18f40b4520 2024-04-17T10:46:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.987+0000 7f18f9dc6640 1 -- 172.21.15.83:0/1135501860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18f4148b20 con 0x7f18f40b4520 2024-04-17T10:46:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.987+0000 7f18f0ff9640 1 -- 172.21.15.83:0/1135501860 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc007e20 con 0x7f18f40b4520 2024-04-17T10:46:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.987+0000 7f18f9dc6640 1 -- 172.21.15.83:0/1135501860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f41490e0 con 0x7f18f40b4520 2024-04-17T10:46:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.987+0000 7f18f9dc6640 1 -- 172.21.15.83:0/1135501860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f4074f90 con 0x7f18f40b4520 2024-04-17T10:46:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.987+0000 7f18f0ff9640 1 -- 172.21.15.83:0/1135501860 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f18dc018070 con 0x7f18f40b4520 2024-04-17T10:46:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.991+0000 7f18f0ff9640 1 --2- 172.21.15.83:0/1135501860 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f18d4066f60 0x7f18d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.991+0000 7f18f0ff9640 1 -- 172.21.15.83:0/1135501860 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f18dc084dd0 con 0x7f18f40b4520 2024-04-17T10:46:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.991+0000 7f18f37fe640 1 --2- 172.21.15.83:0/1135501860 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f18d4066f60 0x7f18d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.991+0000 7f18f37fe640 1 --2- 172.21.15.83:0/1135501860 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f18d4066f60 0x7f18d4069420 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f18e40af000 tx=0x7f18e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:37.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:37.995+0000 7f18f0ff9640 1 -- 172.21.15.83:0/1135501860 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f18dc055e70 con 0x7f18f40b4520 2024-04-17T10:46:38.034 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:46:38.037 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:46:38.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.179+0000 7f18f9dc6640 1 -- 172.21.15.83:0/1135501860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f18f40b0460 con 0x7f18f40b4520 2024-04-17T10:46:38.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.231+0000 7f18f0ff9640 1 -- 172.21.15.83:0/1135501860 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v55) v1 ==== 118+0+0 (secure 0 0 0) 0x7f18dc055770 con 0x7f18f40b4520 2024-04-17T10:46:38.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.235+0000 7f18f9dc6640 1 -- 172.21.15.83:0/1135501860 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f18d4066f60 msgr2=0x7f18d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:38.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.235+0000 7f18f9dc6640 1 --2- 172.21.15.83:0/1135501860 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f18d4066f60 0x7f18d4069420 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f18e40af000 tx=0x7f18e4008040 comp rx=0 tx=0).stop 2024-04-17T10:46:38.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.235+0000 7f18f9dc6640 1 -- 172.21.15.83:0/1135501860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18f40b4520 msgr2=0x7f18f4152620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:38.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.235+0000 7f18f9dc6640 1 --2- 172.21.15.83:0/1135501860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18f40b4520 0x7f18f4152620 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00c820 tx=0x7f18dc00cce0 comp rx=0 tx=0).stop 2024-04-17T10:46:38.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.235+0000 7f18f9dc6640 1 -- 172.21.15.83:0/1135501860 shutdown_connections 2024-04-17T10:46:38.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.235+0000 7f18f9dc6640 1 --2- 172.21.15.83:0/1135501860 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f18d4066f60 0x7f18d4069420 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:38.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.235+0000 7f18f9dc6640 1 --2- 172.21.15.83:0/1135501860 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18f4154670 0x7f18f414b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:38.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.235+0000 7f18f9dc6640 1 --2- 172.21.15.83:0/1135501860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18f40b4520 0x7f18f4152620 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:38.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.235+0000 7f18f9dc6640 1 --2- 172.21.15.83:0/1135501860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18f40b4090 0x7f18f414a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:38.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.235+0000 7f18f9dc6640 1 -- 172.21.15.83:0/1135501860 >> 172.21.15.83:0/1135501860 conn(0x7f18f40b15b0 msgr2=0x7f18f40cf740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:38.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.235+0000 7f18f9dc6640 1 -- 172.21.15.83:0/1135501860 shutdown_connections 2024-04-17T10:46:38.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.235+0000 7f18f9dc6640 1 -- 172.21.15.83:0/1135501860 wait complete. 2024-04-17T10:46:38.246 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-17T10:46:38.247 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-17T10:46:38.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.363+0000 7fcb758e7640 1 Processor -- start 2024-04-17T10:46:38.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.363+0000 7fcb758e7640 1 -- start start 2024-04-17T10:46:38.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.367+0000 7fcb758e7640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcb7015bc10 0x7fcb700d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:38.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.367+0000 7fcb758e7640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcb700d0490 0x7fcb700cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:38.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.367+0000 7fcb758e7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb700cd3d0 0x7fcb7006e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:38.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.367+0000 7fcb758e7640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb700ced00 con 0x7fcb700d0490 2024-04-17T10:46:38.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.367+0000 7fcb758e7640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb700cefe0 con 0x7fcb7015bc10 2024-04-17T10:46:38.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.367+0000 7fcb758e7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb700cc690 con 0x7fcb700cd3d0 2024-04-17T10:46:38.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.367+0000 7fcb667fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcb700d0490 0x7fcb700cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:38.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.367+0000 7fcb667fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcb700d0490 0x7fcb700cce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47540/0 (socket says 172.21.15.83:47540) 2024-04-17T10:46:38.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.367+0000 7fcb667fc640 1 -- 172.21.15.83:0/925729176 learned_addr learned my addr 172.21.15.83:0/925729176 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:38.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.367+0000 7fcb667fc640 1 -- 172.21.15.83:0/925729176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb700cd3d0 msgr2=0x7fcb7006e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:38.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.367+0000 7fcb667fc640 1 --2- 172.21.15.83:0/925729176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb700cd3d0 0x7fcb7006e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:38.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.367+0000 7fcb6effd640 1 --2- 172.21.15.83:0/925729176 >> v2:172.21.15.167:3300/0 conn(0x7fcb7015bc10 0x7fcb700d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:38.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.367+0000 7fcb667fc640 1 -- 172.21.15.83:0/925729176 >> v2:172.21.15.167:3300/0 conn(0x7fcb7015bc10 msgr2=0x7fcb700d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:38.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.367+0000 7fcb667fc640 1 --2- 172.21.15.83:0/925729176 >> v2:172.21.15.167:3300/0 conn(0x7fcb7015bc10 0x7fcb700d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:38.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.367+0000 7fcb667fc640 1 -- 172.21.15.83:0/925729176 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb700cc970 con 0x7fcb700d0490 2024-04-17T10:46:38.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.367+0000 7fcb6effd640 1 --2- 172.21.15.83:0/925729176 >> v2:172.21.15.167:3300/0 conn(0x7fcb7015bc10 0x7fcb700d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:38.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.367+0000 7fcb667fc640 1 --2- 172.21.15.83:0/925729176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb700d0490 0x7fcb700cce90 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fcb700678a0 tx=0x7fcb5801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73742595ced01c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:38.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.367+0000 7fcb6e7fc640 1 -- 172.21.15.83:0/925729176 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb580b0070 con 0x7fcb700d0490 2024-04-17T10:46:38.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.367+0000 7fcb6e7fc640 1 -- 172.21.15.83:0/925729176 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb580ab600 con 0x7fcb700d0490 2024-04-17T10:46:38.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.367+0000 7fcb6e7fc640 1 -- 172.21.15.83:0/925729176 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb580ab9d0 con 0x7fcb700d0490 2024-04-17T10:46:38.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb758e7640 1 -- 172.21.15.83:0/925729176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb700d0490 msgr2=0x7fcb700cce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:38.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb758e7640 1 --2- 172.21.15.83:0/925729176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb700d0490 0x7fcb700cce90 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fcb700678a0 tx=0x7fcb5801c490 comp rx=0 tx=0).stop 2024-04-17T10:46:38.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb758e7640 1 -- 172.21.15.83:0/925729176 shutdown_connections 2024-04-17T10:46:38.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb758e7640 1 --2- 172.21.15.83:0/925729176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb700cd3d0 0x7fcb7006e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:38.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb758e7640 1 --2- 172.21.15.83:0/925729176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb700d0490 0x7fcb700cce90 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:38.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb758e7640 1 --2- 172.21.15.83:0/925729176 >> v2:172.21.15.167:3300/0 conn(0x7fcb7015bc10 0x7fcb700d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:38.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb758e7640 1 -- 172.21.15.83:0/925729176 >> 172.21.15.83:0/925729176 conn(0x7fcb700d17e0 msgr2=0x7fcb700d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:38.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb758e7640 1 -- 172.21.15.83:0/925729176 shutdown_connections 2024-04-17T10:46:38.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb758e7640 1 -- 172.21.15.83:0/925729176 wait complete. 2024-04-17T10:46:38.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb758e7640 1 Processor -- start 2024-04-17T10:46:38.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb758e7640 1 -- start start 2024-04-17T10:46:38.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb758e7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb700cd3d0 0x7fcb700cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:38.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb758e7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb700d0490 0x7fcb700c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:38.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb758e7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb7015bc10 0x7fcb701555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:38.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb758e7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7015f9f0 con 0x7fcb7015bc10 2024-04-17T10:46:38.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb758e7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7015f870 con 0x7fcb700cd3d0 2024-04-17T10:46:38.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb758e7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7015fb70 con 0x7fcb700d0490 2024-04-17T10:46:38.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb6f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb7015bc10 0x7fcb701555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:38.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb6f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb7015bc10 0x7fcb701555f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47548/0 (socket says 172.21.15.83:47548) 2024-04-17T10:46:38.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb6f7fe640 1 -- 172.21.15.83:0/1542640821 learned_addr learned my addr 172.21.15.83:0/1542640821 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:38.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb6effd640 1 --2- 172.21.15.83:0/1542640821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb700cd3d0 0x7fcb700cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:38.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb667fc640 1 --2- 172.21.15.83:0/1542640821 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb700d0490 0x7fcb700c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:38.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb6f7fe640 1 -- 172.21.15.83:0/1542640821 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb700d0490 msgr2=0x7fcb700c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:38.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb6f7fe640 1 --2- 172.21.15.83:0/1542640821 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb700d0490 0x7fcb700c88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:38.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb6f7fe640 1 -- 172.21.15.83:0/1542640821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb700cd3d0 msgr2=0x7fcb700cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:38.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb6f7fe640 1 --2- 172.21.15.83:0/1542640821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb700cd3d0 0x7fcb700cb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:38.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb6f7fe640 1 -- 172.21.15.83:0/1542640821 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb700ca2c0 con 0x7fcb7015bc10 2024-04-17T10:46:38.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb6f7fe640 1 --2- 172.21.15.83:0/1542640821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb7015bc10 0x7fcb701555f0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800b9a0 tx=0x7fcb6800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:38.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb6cff9640 1 -- 172.21.15.83:0/1542640821 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb68019070 con 0x7fcb7015bc10 2024-04-17T10:46:38.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb6cff9640 1 -- 172.21.15.83:0/1542640821 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6800cb60 con 0x7fcb7015bc10 2024-04-17T10:46:38.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb758e7640 1 -- 172.21.15.83:0/1542640821 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb700c9220 con 0x7fcb7015bc10 2024-04-17T10:46:38.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb6cff9640 1 -- 172.21.15.83:0/1542640821 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb68014440 con 0x7fcb7015bc10 2024-04-17T10:46:38.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.371+0000 7fcb758e7640 1 -- 172.21.15.83:0/1542640821 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb700c97e0 con 0x7fcb7015bc10 2024-04-17T10:46:38.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.375+0000 7fcb758e7640 1 -- 172.21.15.83:0/1542640821 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb70074f90 con 0x7fcb7015bc10 2024-04-17T10:46:38.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.375+0000 7fcb6cff9640 1 -- 172.21.15.83:0/1542640821 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcb6801c070 con 0x7fcb7015bc10 2024-04-17T10:46:38.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.375+0000 7fcb6cff9640 1 --2- 172.21.15.83:0/1542640821 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcb50066f60 0x7fcb50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:38.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.375+0000 7fcb6cff9640 1 -- 172.21.15.83:0/1542640821 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb68084e40 con 0x7fcb7015bc10 2024-04-17T10:46:38.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.375+0000 7fcb6effd640 1 --2- 172.21.15.83:0/1542640821 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcb50066f60 0x7fcb50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:38.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.375+0000 7fcb6effd640 1 --2- 172.21.15.83:0/1542640821 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcb50066f60 0x7fcb50069420 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fcb60006fd0 tx=0x7fcb60008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:38.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.379+0000 7fcb6cff9640 1 -- 172.21.15.83:0/1542640821 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcb68055d20 con 0x7fcb7015bc10 2024-04-17T10:46:38.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:38.555+0000 7fcb758e7640 1 -- 172.21.15.83:0/1542640821 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fcb700b7760 con 0x7fcb7015bc10 2024-04-17T10:46:38.981 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-04-17T10:46:38.981 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-04-17T10:46:38.981 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-17T10:46:39.180 INFO:tasks.ceph.mds.c.smithi083.stderr:2024-04-17T10:46:39.183+0000 7f6c9539d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T10:46:39.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.243+0000 7fcb6cff9640 1 -- 172.21.15.83:0/1542640821 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v56) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcb68055620 con 0x7fcb7015bc10 2024-04-17T10:46:39.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.247+0000 7fcb758e7640 1 -- 172.21.15.83:0/1542640821 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcb50066f60 msgr2=0x7fcb50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:39.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.247+0000 7fcb758e7640 1 --2- 172.21.15.83:0/1542640821 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcb50066f60 0x7fcb50069420 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fcb60006fd0 tx=0x7fcb60008040 comp rx=0 tx=0).stop 2024-04-17T10:46:39.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.247+0000 7fcb758e7640 1 -- 172.21.15.83:0/1542640821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb7015bc10 msgr2=0x7fcb701555f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:39.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.247+0000 7fcb758e7640 1 --2- 172.21.15.83:0/1542640821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb7015bc10 0x7fcb701555f0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800b9a0 tx=0x7fcb6800be60 comp rx=0 tx=0).stop 2024-04-17T10:46:39.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.247+0000 7fcb758e7640 1 -- 172.21.15.83:0/1542640821 shutdown_connections 2024-04-17T10:46:39.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.247+0000 7fcb758e7640 1 --2- 172.21.15.83:0/1542640821 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcb50066f60 0x7fcb50069420 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:39.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.247+0000 7fcb758e7640 1 --2- 172.21.15.83:0/1542640821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb7015bc10 0x7fcb701555f0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:39.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.247+0000 7fcb758e7640 1 --2- 172.21.15.83:0/1542640821 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb700d0490 0x7fcb700c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:39.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.247+0000 7fcb758e7640 1 --2- 172.21.15.83:0/1542640821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb700cd3d0 0x7fcb700cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:39.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.247+0000 7fcb758e7640 1 -- 172.21.15.83:0/1542640821 >> 172.21.15.83:0/1542640821 conn(0x7fcb700d17e0 msgr2=0x7fcb700b1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:39.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.247+0000 7fcb758e7640 1 -- 172.21.15.83:0/1542640821 shutdown_connections 2024-04-17T10:46:39.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.247+0000 7fcb758e7640 1 -- 172.21.15.83:0/1542640821 wait complete. 2024-04-17T10:46:39.263 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-17T10:46:39.263 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:46:39.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.387+0000 7fcd90989640 1 Processor -- start 2024-04-17T10:46:39.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.387+0000 7fcd90989640 1 -- start start 2024-04-17T10:46:39.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.387+0000 7fcd90989640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd8c0aedd0 0x7fcd8c06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:39.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.387+0000 7fcd90989640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd8c15e440 con 0x7fcd8c06dae0 2024-04-17T10:46:39.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.387+0000 7fcd90989640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd8c15e720 con 0x7fcd8c15db50 2024-04-17T10:46:39.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.387+0000 7fcd90989640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c15ea00 con 0x7fcd8c0aedd0 2024-04-17T10:46:39.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.387+0000 7fcd8ad9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcd8c06dae0 0x7fcd8c0b2e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:37366/0 (socket says 172.21.15.83:37366) 2024-04-17T10:46:39.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.387+0000 7fcd8ad9d640 1 -- 172.21.15.83:0/592999232 learned_addr learned my addr 172.21.15.83:0/592999232 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:46:39.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.387+0000 7fcd81d9b640 1 --2- 172.21.15.83:0/592999232 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd8c0aedd0 0x7fcd8c06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:39.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.387+0000 7fcd89d9b640 1 -- 172.21.15.83:0/592999232 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 61216571 0 0) 0x7fcd8c15e440 con 0x7fcd8c06dae0 2024-04-17T10:46:39.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.387+0000 7fcd89d9b640 1 -- 172.21.15.83:0/592999232 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd78003530 con 0x7fcd8c06dae0 2024-04-17T10:46:39.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.387+0000 7fcd89d9b640 1 -- 172.21.15.83:0/592999232 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1740032046 0 0) 0x7fcd78003530 con 0x7fcd8c06dae0 2024-04-17T10:46:39.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.387+0000 7fcd89d9b640 1 -- 172.21.15.83:0/592999232 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd8c0aedd0 msgr2=0x7fcd8c06d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:39.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.387+0000 7fcd89d9b640 1 --2- 172.21.15.83:0/592999232 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd8c0aedd0 0x7fcd8c06d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:39.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.387+0000 7fcd89d9b640 1 -- 172.21.15.83:0/592999232 >> v1:172.21.15.167:6789/0 conn(0x7fcd8c15db50 legacy=0x7fcd8c070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:39.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.387+0000 7fcd89d9b640 1 -- 172.21.15.83:0/592999232 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8c15ece0 con 0x7fcd8c06dae0 2024-04-17T10:46:39.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.387+0000 7fcd89d9b640 1 -- 172.21.15.83:0/592999232 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fcd84001a00 con 0x7fcd8c06dae0 2024-04-17T10:46:39.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.387+0000 7fcd89d9b640 1 -- 172.21.15.83:0/592999232 >> v1:172.21.15.83:6789/0 conn(0x7fcd8c06dae0 legacy=0x7fcd8c0b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:39.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.387+0000 7fcd89d9b640 1 --2- 172.21.15.83:0/592999232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd780039f0 0x7fcd78003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:39.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.387+0000 7fcd89d9b640 1 --2- 172.21.15.83:0/592999232 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd8c0aedd0 0x7fcd78004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:39.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.387+0000 7fcd89d9b640 1 --2- 172.21.15.83:0/592999232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd78004d50 0x7fcd780051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:39.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.387+0000 7fcd89d9b640 1 -- 172.21.15.83:0/592999232 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c15ea00 con 0x7fcd780039f0 2024-04-17T10:46:39.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.387+0000 7fcd89d9b640 1 -- 172.21.15.83:0/592999232 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7800ba80 con 0x7fcd78004d50 2024-04-17T10:46:39.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.387+0000 7fcd89d9b640 1 -- 172.21.15.83:0/592999232 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7800bc80 con 0x7fcd8c0aedd0 2024-04-17T10:46:39.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.387+0000 7fcd8a59c640 1 --2- 172.21.15.83:0/592999232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd780039f0 0x7fcd78003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:39.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.387+0000 7fcd8ad9d640 1 --2- 172.21.15.83:0/592999232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd78004d50 0x7fcd780051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:39.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.387+0000 7fcd81d9b640 1 --2- 172.21.15.83:0/592999232 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd8c0aedd0 0x7fcd78004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:39.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.387+0000 7fcd8a59c640 1 -- 172.21.15.83:0/592999232 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd8c0aedd0 msgr2=0x7fcd78004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:39.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.387+0000 7fcd8a59c640 1 --2- 172.21.15.83:0/592999232 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd8c0aedd0 0x7fcd78004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:39.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.387+0000 7fcd8a59c640 1 -- 172.21.15.83:0/592999232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd78004d50 msgr2=0x7fcd780051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:39.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.387+0000 7fcd8a59c640 1 --2- 172.21.15.83:0/592999232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd78004d50 0x7fcd780051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:39.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.387+0000 7fcd8a59c640 1 -- 172.21.15.83:0/592999232 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd78003530 con 0x7fcd780039f0 2024-04-17T10:46:39.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.391+0000 7fcd8a59c640 1 --2- 172.21.15.83:0/592999232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd780039f0 0x7fcd78003e10 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c001750 tx=0x7fcd7c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a49a94ca2ed5166c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:39.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.391+0000 7fcd89d9b640 1 -- 172.21.15.83:0/592999232 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c0013a0 con 0x7fcd780039f0 2024-04-17T10:46:39.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.391+0000 7fcd89d9b640 1 -- 172.21.15.83:0/592999232 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd7c0052f0 con 0x7fcd780039f0 2024-04-17T10:46:39.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.391+0000 7fcd89d9b640 1 -- 172.21.15.83:0/592999232 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c003840 con 0x7fcd780039f0 2024-04-17T10:46:39.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.391+0000 7fcd90989640 1 -- 172.21.15.83:0/592999232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd780039f0 msgr2=0x7fcd78003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:39.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.391+0000 7fcd90989640 1 --2- 172.21.15.83:0/592999232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd780039f0 0x7fcd78003e10 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c001750 tx=0x7fcd7c01c3e0 comp rx=0 tx=0).stop 2024-04-17T10:46:39.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.391+0000 7fcd8ad9d640 1 -- 172.21.15.83:0/592999232 reap_dead start 2024-04-17T10:46:39.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.391+0000 7fcd90989640 1 -- 172.21.15.83:0/592999232 shutdown_connections 2024-04-17T10:46:39.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.391+0000 7fcd90989640 1 -- 172.21.15.83:0/592999232 >> 172.21.15.83:0/592999232 conn(0x7fcd8c06abe0 msgr2=0x7fcd8c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:39.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.391+0000 7fcd90989640 1 -- 172.21.15.83:0/592999232 shutdown_connections 2024-04-17T10:46:39.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.391+0000 7fcd90989640 1 -- 172.21.15.83:0/592999232 wait complete. 2024-04-17T10:46:39.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.391+0000 7fcd90989640 1 Processor -- start 2024-04-17T10:46:39.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.391+0000 7fcd90989640 1 -- start start 2024-04-17T10:46:39.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.395+0000 7fcd90989640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd780039f0 0x7fcd8c1567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:39.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.395+0000 7fcd90989640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd8c159690 0x7fcd8c156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:39.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.395+0000 7fcd90989640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd8c157400 0x7fcd8c0b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:39.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.395+0000 7fcd90989640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c0b4100 con 0x7fcd8c159690 2024-04-17T10:46:39.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.395+0000 7fcd90989640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c0b3f80 con 0x7fcd780039f0 2024-04-17T10:46:39.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.395+0000 7fcd90989640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c166660 con 0x7fcd8c157400 2024-04-17T10:46:39.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.395+0000 7fcd81d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd8c159690 0x7fcd8c156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:39.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.395+0000 7fcd81d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd8c159690 0x7fcd8c156ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47574/0 (socket says 172.21.15.83:47574) 2024-04-17T10:46:39.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.395+0000 7fcd81d9b640 1 -- 172.21.15.83:0/4153014486 learned_addr learned my addr 172.21.15.83:0/4153014486 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:39.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.395+0000 7fcd8ad9d640 1 --2- 172.21.15.83:0/4153014486 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd8c157400 0x7fcd8c0b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:39.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.395+0000 7fcd8a59c640 1 --2- 172.21.15.83:0/4153014486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd780039f0 0x7fcd8c1567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:39.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.395+0000 7fcd81d9b640 1 -- 172.21.15.83:0/4153014486 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd8c157400 msgr2=0x7fcd8c0b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:39.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.395+0000 7fcd81d9b640 1 --2- 172.21.15.83:0/4153014486 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd8c157400 0x7fcd8c0b14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:39.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.395+0000 7fcd81d9b640 1 -- 172.21.15.83:0/4153014486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd780039f0 msgr2=0x7fcd8c1567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:39.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.395+0000 7fcd81d9b640 1 --2- 172.21.15.83:0/4153014486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd780039f0 0x7fcd8c1567b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:39.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.395+0000 7fcd81d9b640 1 -- 172.21.15.83:0/4153014486 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8c1585f0 con 0x7fcd8c159690 2024-04-17T10:46:39.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.395+0000 7fcd81d9b640 1 --2- 172.21.15.83:0/4153014486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd8c159690 0x7fcd8c156ec0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fcd740119d0 tx=0x7fcd74011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:39.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.395+0000 7fcd837fe640 1 -- 172.21.15.83:0/4153014486 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7400fb50 con 0x7fcd8c159690 2024-04-17T10:46:39.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.395+0000 7fcd837fe640 1 -- 172.21.15.83:0/4153014486 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd7400fcf0 con 0x7fcd8c159690 2024-04-17T10:46:39.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.395+0000 7fcd90989640 1 -- 172.21.15.83:0/4153014486 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd8c1588d0 con 0x7fcd8c159690 2024-04-17T10:46:39.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.395+0000 7fcd837fe640 1 -- 172.21.15.83:0/4153014486 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7400fea0 con 0x7fcd8c159690 2024-04-17T10:46:39.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.395+0000 7fcd90989640 1 -- 172.21.15.83:0/4153014486 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8c158e90 con 0x7fcd8c159690 2024-04-17T10:46:39.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.395+0000 7fcd90989640 1 -- 172.21.15.83:0/4153014486 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd8c074f90 con 0x7fcd8c159690 2024-04-17T10:46:39.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.395+0000 7fcd837fe640 1 -- 172.21.15.83:0/4153014486 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcd74012040 con 0x7fcd8c159690 2024-04-17T10:46:39.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.395+0000 7fcd837fe640 1 --2- 172.21.15.83:0/4153014486 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd6c066f60 0x7fcd6c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:39.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.395+0000 7fcd837fe640 1 -- 172.21.15.83:0/4153014486 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd74085a50 con 0x7fcd8c159690 2024-04-17T10:46:39.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.395+0000 7fcd8a59c640 1 --2- 172.21.15.83:0/4153014486 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd6c066f60 0x7fcd6c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:39.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.395+0000 7fcd8a59c640 1 --2- 172.21.15.83:0/4153014486 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd6c066f60 0x7fcd6c069420 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcd780057f0 tx=0x7fcd7c0011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:39.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.399+0000 7fcd837fe640 1 -- 172.21.15.83:0/4153014486 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcd7402b020 con 0x7fcd8c159690 2024-04-17T10:46:39.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.591+0000 7fcd90989640 1 -- 172.21.15.83:0/4153014486 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd8c0b3300 con 0x7fcd8c159690 2024-04-17T10:46:39.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.591+0000 7fcd837fe640 1 -- 172.21.15.83:0/4153014486 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fcd74019030 con 0x7fcd8c159690 2024-04-17T10:46:39.586 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:46:39.587 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5429,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6834/3936640219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3936640219},{"type":"v1","addr":"172.21.15.167:6834","nonce":3936640219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":5998,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4152882285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6833","nonce":4152882285},{"type":"v1","addr":"172.21.15.167:6835","nonce":4152882285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1562645860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1562645860},{"type":"v1","addr":"172.21.15.83:6837","nonce":1562645860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:46:36.578918+0000","modified":"2024-04-17T10:46:38.561701+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7575":{"gid":7575,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/1056175765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1056175765},{"type":"v1","addr":"172.21.15.83:6836","nonce":1056175765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-17T10:46:39.587 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 56 2024-04-17T10:46:39.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.591+0000 7fcd90989640 1 -- 172.21.15.83:0/4153014486 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd6c066f60 msgr2=0x7fcd6c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:39.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.591+0000 7fcd90989640 1 --2- 172.21.15.83:0/4153014486 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd6c066f60 0x7fcd6c069420 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcd780057f0 tx=0x7fcd7c0011a0 comp rx=0 tx=0).stop 2024-04-17T10:46:39.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.591+0000 7fcd90989640 1 -- 172.21.15.83:0/4153014486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd8c159690 msgr2=0x7fcd8c156ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:39.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.591+0000 7fcd90989640 1 --2- 172.21.15.83:0/4153014486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd8c159690 0x7fcd8c156ec0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fcd740119d0 tx=0x7fcd74011e90 comp rx=0 tx=0).stop 2024-04-17T10:46:39.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.591+0000 7fcd90989640 1 -- 172.21.15.83:0/4153014486 shutdown_connections 2024-04-17T10:46:39.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.591+0000 7fcd90989640 1 --2- 172.21.15.83:0/4153014486 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd6c066f60 0x7fcd6c069420 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:39.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.591+0000 7fcd90989640 1 --2- 172.21.15.83:0/4153014486 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd8c157400 0x7fcd8c0b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:39.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.591+0000 7fcd90989640 1 --2- 172.21.15.83:0/4153014486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd8c159690 0x7fcd8c156ec0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:39.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.591+0000 7fcd90989640 1 --2- 172.21.15.83:0/4153014486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd780039f0 0x7fcd8c1567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:39.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.591+0000 7fcd90989640 1 -- 172.21.15.83:0/4153014486 >> 172.21.15.83:0/4153014486 conn(0x7fcd8c06abe0 msgr2=0x7fcd8c160d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:39.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.591+0000 7fcd90989640 1 -- 172.21.15.83:0/4153014486 shutdown_connections 2024-04-17T10:46:39.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.591+0000 7fcd90989640 1 -- 172.21.15.83:0/4153014486 wait complete. 2024-04-17T10:46:39.606 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:46:39.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.719+0000 7f48820ef640 1 Processor -- start 2024-04-17T10:46:39.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.719+0000 7f48820ef640 1 -- start start 2024-04-17T10:46:39.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.719+0000 7f48820ef640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f487c15ba30 0x7f487c072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:39.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.719+0000 7f48820ef640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f487c0cfbe0 0x7f487c06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:39.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.719+0000 7f48820ef640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f487c06e320 con 0x7f487c15c960 2024-04-17T10:46:39.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.719+0000 7f48820ef640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f487c071ea0 con 0x7f487c0cfbe0 2024-04-17T10:46:39.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.719+0000 7f48820ef640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487c06e040 con 0x7f487c15ba30 2024-04-17T10:46:39.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.719+0000 7f487bfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f487c0cfbe0 0x7f487c06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:39.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.719+0000 7f487bfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f487c0cfbe0 0x7f487c06c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47586/0 (socket says 172.21.15.83:47586) 2024-04-17T10:46:39.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.719+0000 7f487bfff640 1 -- 172.21.15.83:0/2710974844 learned_addr learned my addr 172.21.15.83:0/2710974844 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:39.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.719+0000 7f487b7fe640 1 --2- 172.21.15.83:0/2710974844 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f487c15ba30 0x7f487c072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:39.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.719+0000 7f487bfff640 1 -- 172.21.15.83:0/2710974844 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f487c15ba30 msgr2=0x7f487c072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:39.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.719+0000 7f487bfff640 1 --2- 172.21.15.83:0/2710974844 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f487c15ba30 0x7f487c072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:39.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.719+0000 7f487bfff640 1 -- 172.21.15.83:0/2710974844 >> v1:172.21.15.167:6789/0 conn(0x7f487c15c960 legacy=0x7f487c0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:39.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.719+0000 7f487bfff640 1 -- 172.21.15.83:0/2710974844 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487c071850 con 0x7f487c0cfbe0 2024-04-17T10:46:39.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f487bfff640 1 --2- 172.21.15.83:0/2710974844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f487c0cfbe0 0x7f487c06c490 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f487c06d7e0 tx=0x7f487401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf2496b239e6e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:39.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f487a7fc640 1 -- 172.21.15.83:0/2710974844 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48740a5620 con 0x7f487c0cfbe0 2024-04-17T10:46:39.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f487a7fc640 1 -- 172.21.15.83:0/2710974844 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48740a57c0 con 0x7f487c0cfbe0 2024-04-17T10:46:39.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f487a7fc640 1 -- 172.21.15.83:0/2710974844 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48740a5b90 con 0x7f487c0cfbe0 2024-04-17T10:46:39.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f48820ef640 1 -- 172.21.15.83:0/2710974844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f487c0cfbe0 msgr2=0x7f487c06c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:39.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f48820ef640 1 --2- 172.21.15.83:0/2710974844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f487c0cfbe0 0x7f487c06c490 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f487c06d7e0 tx=0x7f487401c920 comp rx=0 tx=0).stop 2024-04-17T10:46:39.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f48820ef640 1 -- 172.21.15.83:0/2710974844 shutdown_connections 2024-04-17T10:46:39.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f48820ef640 1 --2- 172.21.15.83:0/2710974844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f487c0cfbe0 0x7f487c06c490 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:39.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f48820ef640 1 --2- 172.21.15.83:0/2710974844 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f487c15ba30 0x7f487c072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:39.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f48820ef640 1 -- 172.21.15.83:0/2710974844 >> 172.21.15.83:0/2710974844 conn(0x7f487c06abe0 msgr2=0x7f487c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:39.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f48820ef640 1 -- 172.21.15.83:0/2710974844 shutdown_connections 2024-04-17T10:46:39.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f48820ef640 1 -- 172.21.15.83:0/2710974844 wait complete. 2024-04-17T10:46:39.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f48820ef640 1 Processor -- start 2024-04-17T10:46:39.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f48820ef640 1 -- start start 2024-04-17T10:46:39.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f48820ef640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f487c0cfbe0 0x7f487c14d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:39.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f48820ef640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f487c15ba30 0x7f487c0b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:39.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f48820ef640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f487c15c960 0x7f487c1598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:39.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f48820ef640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c06de20 con 0x7f487c15ba30 2024-04-17T10:46:39.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f48820ef640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c06dca0 con 0x7f487c0cfbe0 2024-04-17T10:46:39.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f48820ef640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487c168520 con 0x7f487c15c960 2024-04-17T10:46:39.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f487affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f487c15ba30 0x7f487c0b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:39.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f487affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f487c15ba30 0x7f487c0b30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47592/0 (socket says 172.21.15.83:47592) 2024-04-17T10:46:39.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f487affd640 1 -- 172.21.15.83:0/946672327 learned_addr learned my addr 172.21.15.83:0/946672327 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:39.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f487b7fe640 1 --2- 172.21.15.83:0/946672327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f487c0cfbe0 0x7f487c14d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:39.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f487bfff640 1 --2- 172.21.15.83:0/946672327 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f487c15c960 0x7f487c1598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:39.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f487affd640 1 -- 172.21.15.83:0/946672327 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f487c15c960 msgr2=0x7f487c1598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:39.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f487affd640 1 --2- 172.21.15.83:0/946672327 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f487c15c960 0x7f487c1598b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:39.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f487affd640 1 -- 172.21.15.83:0/946672327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f487c0cfbe0 msgr2=0x7f487c14d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:39.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f487affd640 1 --2- 172.21.15.83:0/946672327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f487c0cfbe0 0x7f487c14d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:39.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f487affd640 1 -- 172.21.15.83:0/946672327 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487c0b3600 con 0x7f487c15ba30 2024-04-17T10:46:39.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f487affd640 1 --2- 172.21.15.83:0/946672327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f487c15ba30 0x7f487c0b30c0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f486c00c450 tx=0x7f486c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:39.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f4863fff640 1 -- 172.21.15.83:0/946672327 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c012830 con 0x7f487c15ba30 2024-04-17T10:46:39.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f4863fff640 1 -- 172.21.15.83:0/946672327 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f486c017070 con 0x7f487c15ba30 2024-04-17T10:46:39.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f48820ef640 1 -- 172.21.15.83:0/946672327 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f487c15a7c0 con 0x7f487c15ba30 2024-04-17T10:46:39.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f4863fff640 1 -- 172.21.15.83:0/946672327 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c012c30 con 0x7f487c15ba30 2024-04-17T10:46:39.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.723+0000 7f48820ef640 1 -- 172.21.15.83:0/946672327 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f487c15ad80 con 0x7f487c15ba30 2024-04-17T10:46:39.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.727+0000 7f48820ef640 1 -- 172.21.15.83:0/946672327 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f487c0cd550 con 0x7f487c15ba30 2024-04-17T10:46:39.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.727+0000 7f4863fff640 1 -- 172.21.15.83:0/946672327 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f486c01c460 con 0x7f487c15ba30 2024-04-17T10:46:39.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.727+0000 7f4863fff640 1 --2- 172.21.15.83:0/946672327 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4868066f60 0x7f4868069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:39.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.727+0000 7f4863fff640 1 -- 172.21.15.83:0/946672327 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f486c084d50 con 0x7f487c15ba30 2024-04-17T10:46:39.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.727+0000 7f487b7fe640 1 --2- 172.21.15.83:0/946672327 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4868066f60 0x7f4868069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:39.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.727+0000 7f487b7fe640 1 --2- 172.21.15.83:0/946672327 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4868066f60 0x7f4868069420 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f487000a970 tx=0x7f4870008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:39.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.731+0000 7f4863fff640 1 -- 172.21.15.83:0/946672327 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f486c088030 con 0x7f487c15ba30 2024-04-17T10:46:39.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.887+0000 7f48820ef640 1 -- 172.21.15.83:0/946672327 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f487c06b0d0 con 0x7f487c15ba30 2024-04-17T10:46:39.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.891+0000 7f4863fff640 1 -- 172.21.15.83:0/946672327 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16588 (secure 0 0 0) 0x7f486c055df0 con 0x7f487c15ba30 2024-04-17T10:46:39.888 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:46:39.888 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":89,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:46:38.209473+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:46:34.779174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-17T10:46:35.563972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:46:39.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.895+0000 7f48820ef640 1 -- 172.21.15.83:0/946672327 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4868066f60 msgr2=0x7f4868069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:39.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.895+0000 7f48820ef640 1 --2- 172.21.15.83:0/946672327 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4868066f60 0x7f4868069420 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f487000a970 tx=0x7f4870008040 comp rx=0 tx=0).stop 2024-04-17T10:46:39.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.895+0000 7f48820ef640 1 -- 172.21.15.83:0/946672327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f487c15ba30 msgr2=0x7f487c0b30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:39.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.895+0000 7f48820ef640 1 --2- 172.21.15.83:0/946672327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f487c15ba30 0x7f487c0b30c0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f486c00c450 tx=0x7f486c00c910 comp rx=0 tx=0).stop 2024-04-17T10:46:39.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.895+0000 7f48820ef640 1 -- 172.21.15.83:0/946672327 shutdown_connections 2024-04-17T10:46:39.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.895+0000 7f48820ef640 1 --2- 172.21.15.83:0/946672327 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4868066f60 0x7f4868069420 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:39.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.895+0000 7f48820ef640 1 --2- 172.21.15.83:0/946672327 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f487c15c960 0x7f487c1598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:39.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.895+0000 7f48820ef640 1 --2- 172.21.15.83:0/946672327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f487c15ba30 0x7f487c0b30c0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:39.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.895+0000 7f48820ef640 1 --2- 172.21.15.83:0/946672327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f487c0cfbe0 0x7f487c14d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:39.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.895+0000 7f48820ef640 1 -- 172.21.15.83:0/946672327 >> 172.21.15.83:0/946672327 conn(0x7f487c06abe0 msgr2=0x7f487c15d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:39.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.895+0000 7f48820ef640 1 -- 172.21.15.83:0/946672327 shutdown_connections 2024-04-17T10:46:39.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:39.895+0000 7f48820ef640 1 -- 172.21.15.83:0/946672327 wait complete. 2024-04-17T10:46:39.906 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-17T10:46:39.906 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:46:40.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.019+0000 7f42dbb3f640 1 Processor -- start 2024-04-17T10:46:40.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.019+0000 7f42dbb3f640 1 -- start start 2024-04-17T10:46:40.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.019+0000 7f42dbb3f640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f42d415eeb0 0x7f42d406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:40.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.019+0000 7f42dbb3f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f42d40cf780 0x7f42d406bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:40.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.019+0000 7f42dbb3f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42d406e980 con 0x7f42d415f290 2024-04-17T10:46:40.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.019+0000 7f42dbb3f640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d40cf030 con 0x7f42d40cf780 2024-04-17T10:46:40.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.019+0000 7f42dbb3f640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d40cdb90 con 0x7f42d415eeb0 2024-04-17T10:46:40.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.019+0000 7f42d90da640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f42d40cf780 0x7f42d406bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:40.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.019+0000 7f42da0dc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f42d415f290 0x7f42d415f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:37368/0 (socket says 172.21.15.83:37368) 2024-04-17T10:46:40.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.019+0000 7f42da0dc640 1 -- 172.21.15.83:0/2129048859 learned_addr learned my addr 172.21.15.83:0/2129048859 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:46:40.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.019+0000 7f42d98db640 1 --2- 172.21.15.83:0/2129048859 >> v2:172.21.15.167:3300/0 conn(0x7f42d415eeb0 0x7f42d406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:40.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.019+0000 7f42d90da640 1 -- 172.21.15.83:0/2129048859 >> v2:172.21.15.167:3300/0 conn(0x7f42d415eeb0 msgr2=0x7f42d406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:40.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.019+0000 7f42d90da640 1 --2- 172.21.15.83:0/2129048859 >> v2:172.21.15.167:3300/0 conn(0x7f42d415eeb0 0x7f42d406d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:40.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.019+0000 7f42d90da640 1 -- 172.21.15.83:0/2129048859 >> v1:172.21.15.83:6789/0 conn(0x7f42d415f290 legacy=0x7f42d415f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:40.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.019+0000 7f42d90da640 1 -- 172.21.15.83:0/2129048859 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d406e0e0 con 0x7f42d40cf780 2024-04-17T10:46:40.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.023+0000 7f42d90da640 1 --2- 172.21.15.83:0/2129048859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d40cf780 0x7f42d406bbf0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f42d4068520 tx=0x7f42c401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a0f050b980ae65 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:40.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.023+0000 7f42d88d9640 1 -- 172.21.15.83:0/2129048859 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c40a7070 con 0x7f42d40cf780 2024-04-17T10:46:40.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.023+0000 7f42d88d9640 1 -- 172.21.15.83:0/2129048859 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c40b0070 con 0x7f42d40cf780 2024-04-17T10:46:40.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.023+0000 7f42d88d9640 1 -- 172.21.15.83:0/2129048859 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c40ab890 con 0x7f42d40cf780 2024-04-17T10:46:40.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.023+0000 7f42dbb3f640 1 -- 172.21.15.83:0/2129048859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d40cf780 msgr2=0x7f42d406bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:40.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.023+0000 7f42dbb3f640 1 --2- 172.21.15.83:0/2129048859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d40cf780 0x7f42d406bbf0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f42d4068520 tx=0x7f42c401c450 comp rx=0 tx=0).stop 2024-04-17T10:46:40.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.023+0000 7f42dbb3f640 1 -- 172.21.15.83:0/2129048859 shutdown_connections 2024-04-17T10:46:40.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.023+0000 7f42dbb3f640 1 --2- 172.21.15.83:0/2129048859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d40cf780 0x7f42d406bbf0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:40.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.023+0000 7f42dbb3f640 1 --2- 172.21.15.83:0/2129048859 >> v2:172.21.15.167:3300/0 conn(0x7f42d415eeb0 0x7f42d406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:40.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.023+0000 7f42dbb3f640 1 -- 172.21.15.83:0/2129048859 >> 172.21.15.83:0/2129048859 conn(0x7f42d40cc910 msgr2=0x7f42d40ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:40.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.023+0000 7f42dbb3f640 1 -- 172.21.15.83:0/2129048859 shutdown_connections 2024-04-17T10:46:40.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.023+0000 7f42dbb3f640 1 -- 172.21.15.83:0/2129048859 wait complete. 2024-04-17T10:46:40.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.023+0000 7f42dbb3f640 1 Processor -- start 2024-04-17T10:46:40.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.023+0000 7f42dbb3f640 1 -- start start 2024-04-17T10:46:40.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.023+0000 7f42dbb3f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d40cf780 0x7f42d415c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:40.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.023+0000 7f42dbb3f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42d415eeb0 0x7f42d4154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:40.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.023+0000 7f42dbb3f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42d415f290 0x7f42d40b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:40.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.023+0000 7f42dbb3f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d4168a80 con 0x7f42d40cf780 2024-04-17T10:46:40.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.023+0000 7f42dbb3f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d406c540 con 0x7f42d415f290 2024-04-17T10:46:40.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.023+0000 7f42dbb3f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d4168c00 con 0x7f42d415eeb0 2024-04-17T10:46:40.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.023+0000 7f42d98db640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d40cf780 0x7f42d415c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:40.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.023+0000 7f42d98db640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d40cf780 0x7f42d415c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47606/0 (socket says 172.21.15.83:47606) 2024-04-17T10:46:40.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.023+0000 7f42d98db640 1 -- 172.21.15.83:0/1684541186 learned_addr learned my addr 172.21.15.83:0/1684541186 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:40.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.023+0000 7f42d98db640 1 -- 172.21.15.83:0/1684541186 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42d415eeb0 msgr2=0x7f42d4154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:40.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.023+0000 7f42d98db640 1 --2- 172.21.15.83:0/1684541186 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42d415eeb0 0x7f42d4154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:40.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.023+0000 7f42d98db640 1 -- 172.21.15.83:0/1684541186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42d415f290 msgr2=0x7f42d40b0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:40.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.023+0000 7f42d98db640 1 --2- 172.21.15.83:0/1684541186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42d415f290 0x7f42d40b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:40.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.023+0000 7f42d98db640 1 -- 172.21.15.83:0/1684541186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d415c860 con 0x7f42d40cf780 2024-04-17T10:46:40.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.023+0000 7f42d98db640 1 --2- 172.21.15.83:0/1684541186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d40cf780 0x7f42d415c320 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f42d000d800 tx=0x7f42d000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:40.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.023+0000 7f42ca7fc640 1 -- 172.21.15.83:0/1684541186 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d000b6c0 con 0x7f42d40cf780 2024-04-17T10:46:40.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.027+0000 7f42ca7fc640 1 -- 172.21.15.83:0/1684541186 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d0019070 con 0x7f42d40cf780 2024-04-17T10:46:40.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.027+0000 7f42ca7fc640 1 -- 172.21.15.83:0/1684541186 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d000ba60 con 0x7f42d40cf780 2024-04-17T10:46:40.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.027+0000 7f42dbb3f640 1 -- 172.21.15.83:0/1684541186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d415cae0 con 0x7f42d40cf780 2024-04-17T10:46:40.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.027+0000 7f42dbb3f640 1 -- 172.21.15.83:0/1684541186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d415d0a0 con 0x7f42d40cf780 2024-04-17T10:46:40.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.027+0000 7f42dbb3f640 1 -- 172.21.15.83:0/1684541186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d4074f90 con 0x7f42d40cf780 2024-04-17T10:46:40.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.027+0000 7f42ca7fc640 1 -- 172.21.15.83:0/1684541186 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f42d000bc00 con 0x7f42d40cf780 2024-04-17T10:46:40.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.027+0000 7f42ca7fc640 1 --2- 172.21.15.83:0/1684541186 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f42b8066f90 0x7f42b8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:40.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.027+0000 7f42ca7fc640 1 -- 172.21.15.83:0/1684541186 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f42d00854f0 con 0x7f42d40cf780 2024-04-17T10:46:40.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.027+0000 7f42d90da640 1 --2- 172.21.15.83:0/1684541186 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f42b8066f90 0x7f42b8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:40.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.027+0000 7f42d90da640 1 --2- 172.21.15.83:0/1684541186 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f42b8066f90 0x7f42b8069450 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f42c40af000 tx=0x7f42c4004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:40.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.031+0000 7f42ca7fc640 1 -- 172.21.15.83:0/1684541186 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f42d415c860 con 0x7f42d40cf780 2024-04-17T10:46:40.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.175+0000 7f42dbb3f640 1 -- 172.21.15.83:0/1684541186 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f42d4068150 con 0x7f42b8066f90 2024-04-17T10:46:40.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.179+0000 7f42ca7fc640 1 -- 172.21.15.83:0/1684541186 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+237307 (secure 0 0 0) 0x7f42d4068150 con 0x7f42b8066f90 2024-04-17T10:46:40.180 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:46:40.182 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:46:40.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.183+0000 7f42dbb3f640 1 -- 172.21.15.83:0/1684541186 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f42b8066f90 msgr2=0x7f42b8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:40.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.183+0000 7f42dbb3f640 1 --2- 172.21.15.83:0/1684541186 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f42b8066f90 0x7f42b8069450 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f42c40af000 tx=0x7f42c4004740 comp rx=0 tx=0).stop 2024-04-17T10:46:40.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.183+0000 7f42dbb3f640 1 -- 172.21.15.83:0/1684541186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d40cf780 msgr2=0x7f42d415c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:40.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.183+0000 7f42dbb3f640 1 --2- 172.21.15.83:0/1684541186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d40cf780 0x7f42d415c320 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f42d000d800 tx=0x7f42d000dcc0 comp rx=0 tx=0).stop 2024-04-17T10:46:40.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.183+0000 7f42dbb3f640 1 -- 172.21.15.83:0/1684541186 shutdown_connections 2024-04-17T10:46:40.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.183+0000 7f42dbb3f640 1 --2- 172.21.15.83:0/1684541186 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f42b8066f90 0x7f42b8069450 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:40.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.183+0000 7f42dbb3f640 1 --2- 172.21.15.83:0/1684541186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42d415f290 0x7f42d40b0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:40.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.183+0000 7f42dbb3f640 1 --2- 172.21.15.83:0/1684541186 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42d415eeb0 0x7f42d4154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:40.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.183+0000 7f42dbb3f640 1 --2- 172.21.15.83:0/1684541186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d40cf780 0x7f42d415c320 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:40.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.183+0000 7f42dbb3f640 1 -- 172.21.15.83:0/1684541186 >> 172.21.15.83:0/1684541186 conn(0x7f42d40cc910 msgr2=0x7f42d409c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:40.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.183+0000 7f42dbb3f640 1 -- 172.21.15.83:0/1684541186 shutdown_connections 2024-04-17T10:46:40.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.183+0000 7f42dbb3f640 1 -- 172.21.15.83:0/1684541186 wait complete. 2024-04-17T10:46:40.195 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2024-04-17T10:46:38.334017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":487704,"kb_used_data":9188,"kb_used_omap":12,"kb_used_meta":478387,"kb_avail":754487016,"statfs":{"total":773094113280,"available":772594704384,"internally_reserved":0,"allocated":9408512,"data_stored":4706123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":489868848},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000959"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:35.183153+0000","last_change":"2024-04-17T10:46:35.183153+0000","last_active":"2024-04-17T10:46:35.183153+0000","last_peered":"2024-04-17T10:46:35.183153+0000","last_clean":"2024-04-17T10:46:35.183153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:35.183153+0000","last_undegraded":"2024-04-17T10:46:35.183153+0000","last_fullsized":"2024-04-17T10:46:35.183153+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:35.183153+0000","last_change":"2024-04-17T10:46:35.183153+0000","last_active":"2024-04-17T10:46:35.183153+0000","last_peered":"2024-04-17T10:46:35.183153+0000","last_clean":"2024-04-17T10:46:35.183153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:35.183153+0000","last_undegraded":"2024-04-17T10:46:35.183153+0000","last_fullsized":"2024-04-17T10:46:35.183153+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:35.183153+0000","last_change":"2024-04-17T10:46:35.183153+0000","last_active":"2024-04-17T10:46:35.183153+0000","last_peered":"2024-04-17T10:46:35.183153+0000","last_clean":"2024-04-17T10:46:35.183153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:35.183153+0000","last_undegraded":"2024-04-17T10:46:35.183153+0000","last_fullsized":"2024-04-17T10:46:35.183153+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:35.183153+0000","last_change":"2024-04-17T10:46:35.183153+0000","last_active":"2024-04-17T10:46:35.183153+0000","last_peered":"2024-04-17T10:46:35.183153+0000","last_clean":"2024-04-17T10:46:35.183153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:35.183153+0000","last_undegraded":"2024-04-17T10:46:35.183153+0000","last_fullsized":"2024-04-17T10:46:35.183153+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:35.183153+0000","last_change":"2024-04-17T10:46:35.183153+0000","last_active":"2024-04-17T10:46:35.183153+0000","last_peered":"2024-04-17T10:46:35.183153+0000","last_clean":"2024-04-17T10:46:35.183153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:35.183153+0000","last_undegraded":"2024-04-17T10:46:35.183153+0000","last_fullsized":"2024-04-17T10:46:35.183153+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:35.183153+0000","last_change":"2024-04-17T10:46:35.183153+0000","last_active":"2024-04-17T10:46:35.183153+0000","last_peered":"2024-04-17T10:46:35.183153+0000","last_clean":"2024-04-17T10:46:35.183153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:35.183153+0000","last_undegraded":"2024-04-17T10:46:35.183153+0000","last_fullsized":"2024-04-17T10:46:35.183153+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:35.183153+0000","last_change":"2024-04-17T10:46:35.183153+0000","last_active":"2024-04-17T10:46:35.183153+0000","last_peered":"2024-04-17T10:46:35.183153+0000","last_clean":"2024-04-17T10:46:35.183153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:35.183153+0000","last_undegraded":"2024-04-17T10:46:35.183153+0000","last_fullsized":"2024-04-17T10:46:35.183153+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:35.183153+0000","last_change":"2024-04-17T10:46:35.183153+0000","last_active":"2024-04-17T10:46:35.183153+0000","last_peered":"2024-04-17T10:46:35.183153+0000","last_clean":"2024-04-17T10:46:35.183153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:35.183153+0000","last_undegraded":"2024-04-17T10:46:35.183153+0000","last_fullsized":"2024-04-17T10:46:35.183153+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:35.183153+0000","last_change":"2024-04-17T10:46:35.183153+0000","last_active":"2024-04-17T10:46:35.183153+0000","last_peered":"2024-04-17T10:46:35.183153+0000","last_clean":"2024-04-17T10:46:35.183153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:35.183153+0000","last_undegraded":"2024-04-17T10:46:35.183153+0000","last_fullsized":"2024-04-17T10:46:35.183153+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:35.183153+0000","last_change":"2024-04-17T10:46:35.183153+0000","last_active":"2024-04-17T10:46:35.183153+0000","last_peered":"2024-04-17T10:46:35.183153+0000","last_clean":"2024-04-17T10:46:35.183153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:35.183153+0000","last_undegraded":"2024-04-17T10:46:35.183153+0000","last_fullsized":"2024-04-17T10:46:35.183153+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:35.183153+0000","last_change":"2024-04-17T10:46:35.183153+0000","last_active":"2024-04-17T10:46:35.183153+0000","last_peered":"2024-04-17T10:46:35.183153+0000","last_clean":"2024-04-17T10:46:35.183153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:35.183153+0000","last_undegraded":"2024-04-17T10:46:35.183153+0000","last_fullsized":"2024-04-17T10:46:35.183153+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:35.183153+0000","last_change":"2024-04-17T10:46:35.183153+0000","last_active":"2024-04-17T10:46:35.183153+0000","last_peered":"2024-04-17T10:46:35.183153+0000","last_clean":"2024-04-17T10:46:35.183153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:35.183153+0000","last_undegraded":"2024-04-17T10:46:35.183153+0000","last_fullsized":"2024-04-17T10:46:35.183153+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:35.183153+0000","last_change":"2024-04-17T10:46:35.183153+0000","last_active":"2024-04-17T10:46:35.183153+0000","last_peered":"2024-04-17T10:46:35.183153+0000","last_clean":"2024-04-17T10:46:35.183153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:35.183153+0000","last_undegraded":"2024-04-17T10:46:35.183153+0000","last_fullsized":"2024-04-17T10:46:35.183153+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:35.183153+0000","last_change":"2024-04-17T10:46:35.183153+0000","last_active":"2024-04-17T10:46:35.183153+0000","last_peered":"2024-04-17T10:46:35.183153+0000","last_clean":"2024-04-17T10:46:35.183153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:35.183153+0000","last_undegraded":"2024-04-17T10:46:35.183153+0000","last_fullsized":"2024-04-17T10:46:35.183153+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:35.183153+0000","last_change":"2024-04-17T10:46:35.183153+0000","last_active":"2024-04-17T10:46:35.183153+0000","last_peered":"2024-04-17T10:46:35.183153+0000","last_clean":"2024-04-17T10:46:35.183153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:35.183153+0000","last_undegraded":"2024-04-17T10:46:35.183153+0000","last_fullsized":"2024-04-17T10:46:35.183153+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:35.183153+0000","last_change":"2024-04-17T10:46:35.183153+0000","last_active":"2024-04-17T10:46:35.183153+0000","last_peered":"2024-04-17T10:46:35.183153+0000","last_clean":"2024-04-17T10:46:35.183153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:35.183153+0000","last_undegraded":"2024-04-17T10:46:35.183153+0000","last_fullsized":"2024-04-17T10:46:35.183153+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:35.183153+0000","last_change":"2024-04-17T10:46:35.183153+0000","last_active":"2024-04-17T10:46:35.183153+0000","last_peered":"2024-04-17T10:46:35.183153+0000","last_clean":"2024-04-17T10:46:35.183153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:35.183153+0000","last_undegraded":"2024-04-17T10:46:35.183153+0000","last_fullsized":"2024-04-17T10:46:35.183153+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:35.183153+0000","last_change":"2024-04-17T10:46:35.183153+0000","last_active":"2024-04-17T10:46:35.183153+0000","last_peered":"2024-04-17T10:46:35.183153+0000","last_clean":"2024-04-17T10:46:35.183153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:35.183153+0000","last_undegraded":"2024-04-17T10:46:35.183153+0000","last_fullsized":"2024-04-17T10:46:35.183153+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:35.183153+0000","last_change":"2024-04-17T10:46:35.183153+0000","last_active":"2024-04-17T10:46:35.183153+0000","last_peered":"2024-04-17T10:46:35.183153+0000","last_clean":"2024-04-17T10:46:35.183153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:35.183153+0000","last_undegraded":"2024-04-17T10:46:35.183153+0000","last_fullsized":"2024-04-17T10:46:35.183153+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:35.183153+0000","last_change":"2024-04-17T10:46:35.183153+0000","last_active":"2024-04-17T10:46:35.183153+0000","last_peered":"2024-04-17T10:46:35.183153+0000","last_clean":"2024-04-17T10:46:35.183153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:35.183153+0000","last_undegraded":"2024-04-17T10:46:35.183153+0000","last_fullsized":"2024-04-17T10:46:35.183153+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:35.183153+0000","last_change":"2024-04-17T10:46:35.183153+0000","last_active":"2024-04-17T10:46:35.183153+0000","last_peered":"2024-04-17T10:46:35.183153+0000","last_clean":"2024-04-17T10:46:35.183153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:35.183153+0000","last_undegraded":"2024-04-17T10:46:35.183153+0000","last_fullsized":"2024-04-17T10:46:35.183153+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:35.183153+0000","last_change":"2024-04-17T10:46:35.183153+0000","last_active":"2024-04-17T10:46:35.183153+0000","last_peered":"2024-04-17T10:46:35.183153+0000","last_clean":"2024-04-17T10:46:35.183153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:35.183153+0000","last_undegraded":"2024-04-17T10:46:35.183153+0000","last_fullsized":"2024-04-17T10:46:35.183153+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:35.183153+0000","last_change":"2024-04-17T10:46:35.183153+0000","last_active":"2024-04-17T10:46:35.183153+0000","last_peered":"2024-04-17T10:46:35.183153+0000","last_clean":"2024-04-17T10:46:35.183153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:35.183153+0000","last_undegraded":"2024-04-17T10:46:35.183153+0000","last_fullsized":"2024-04-17T10:46:35.183153+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:35.183153+0000","last_change":"2024-04-17T10:46:35.183153+0000","last_active":"2024-04-17T10:46:35.183153+0000","last_peered":"2024-04-17T10:46:35.183153+0000","last_clean":"2024-04-17T10:46:35.183153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:35.183153+0000","last_undegraded":"2024-04-17T10:46:35.183153+0000","last_fullsized":"2024-04-17T10:46:35.183153+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:35.183153+0000","last_change":"2024-04-17T10:46:35.183153+0000","last_active":"2024-04-17T10:46:35.183153+0000","last_peered":"2024-04-17T10:46:35.183153+0000","last_clean":"2024-04-17T10:46:35.183153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:35.183153+0000","last_undegraded":"2024-04-17T10:46:35.183153+0000","last_fullsized":"2024-04-17T10:46:35.183153+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:35.183153+0000","last_change":"2024-04-17T10:46:35.183153+0000","last_active":"2024-04-17T10:46:35.183153+0000","last_peered":"2024-04-17T10:46:35.183153+0000","last_clean":"2024-04-17T10:46:35.183153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:35.183153+0000","last_undegraded":"2024-04-17T10:46:35.183153+0000","last_fullsized":"2024-04-17T10:46:35.183153+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:35.183153+0000","last_change":"2024-04-17T10:46:35.183153+0000","last_active":"2024-04-17T10:46:35.183153+0000","last_peered":"2024-04-17T10:46:35.183153+0000","last_clean":"2024-04-17T10:46:35.183153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:35.183153+0000","last_undegraded":"2024-04-17T10:46:35.183153+0000","last_fullsized":"2024-04-17T10:46:35.183153+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:35.183153+0000","last_change":"2024-04-17T10:46:35.183153+0000","last_active":"2024-04-17T10:46:35.183153+0000","last_peered":"2024-04-17T10:46:35.183153+0000","last_clean":"2024-04-17T10:46:35.183153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:35.183153+0000","last_undegraded":"2024-04-17T10:46:35.183153+0000","last_fullsized":"2024-04-17T10:46:35.183153+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:35.183153+0000","last_change":"2024-04-17T10:46:35.183153+0000","last_active":"2024-04-17T10:46:35.183153+0000","last_peered":"2024-04-17T10:46:35.183153+0000","last_clean":"2024-04-17T10:46:35.183153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:35.183153+0000","last_undegraded":"2024-04-17T10:46:35.183153+0000","last_fullsized":"2024-04-17T10:46:35.183153+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:35.183153+0000","last_change":"2024-04-17T10:46:35.183153+0000","last_active":"2024-04-17T10:46:35.183153+0000","last_peered":"2024-04-17T10:46:35.183153+0000","last_clean":"2024-04-17T10:46:35.183153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:35.183153+0000","last_undegraded":"2024-04-17T10:46:35.183153+0000","last_fullsized":"2024-04-17T10:46:35.183153+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:35.183153+0000","last_change":"2024-04-17T10:46:35.183153+0000","last_active":"2024-04-17T10:46:35.183153+0000","last_peered":"2024-04-17T10:46:35.183153+0000","last_clean":"2024-04-17T10:46:35.183153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:35.183153+0000","last_undegraded":"2024-04-17T10:46:35.183153+0000","last_fullsized":"2024-04-17T10:46:35.183153+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:36.196540+0000","last_change":"2024-04-17T10:46:36.196540+0000","last_active":"2024-04-17T10:46:36.196540+0000","last_peered":"2024-04-17T10:46:36.196540+0000","last_clean":"2024-04-17T10:46:36.196540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:36.196540+0000","last_undegraded":"2024-04-17T10:46:36.196540+0000","last_fullsized":"2024-04-17T10:46:36.196540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:46:35.183153+0000","last_change":"2024-04-17T10:46:35.183153+0000","last_active":"2024-04-17T10:46:35.183153+0000","last_peered":"2024-04-17T10:46:35.183153+0000","last_clean":"2024-04-17T10:46:35.183153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:46:35.183153+0000","last_undegraded":"2024-04-17T10:46:35.183153+0000","last_fullsized":"2024-04-17T10:46:35.183153+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":304,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-17T10:46:33.196813+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:46:33.196813+0000","last_peered":"2024-04-17T10:46:33.196813+0000","last_clean":"2024-04-17T10:46:33.196813+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:46:33.196813+0000","last_undegraded":"2024-04-17T10:46:33.196813+0000","last_fullsized":"2024-04-17T10:46:33.196813+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574955,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59160,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312680,"statfs":{"total":96636764160,"available":96576184320,"internally_reserved":0,"allocated":1638400,"data_stored":1038023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":64088,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307752,"statfs":{"total":96636764160,"available":96571138048,"internally_reserved":0,"allocated":995328,"data_stored":427271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574955,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50236,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321604,"statfs":{"total":96636764160,"available":96585322496,"internally_reserved":0,"allocated":1032192,"data_stored":442598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":64144,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307696,"statfs":{"total":96636764160,"available":96571080704,"internally_reserved":0,"allocated":1044480,"data_stored":447655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574956,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":64112,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307728,"statfs":{"total":96636764160,"available":96571113472,"internally_reserved":0,"allocated":1019904,"data_stored":437541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":64616903},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54396,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317444,"statfs":{"total":96636764160,"available":96581062656,"internally_reserved":0,"allocated":1032192,"data_stored":442598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":72456,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94299384,"statfs":{"total":96636764160,"available":96562569216,"internally_reserved":0,"allocated":1044480,"data_stored":447655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574954,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59112,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312728,"statfs":{"total":96636764160,"available":96576233472,"internally_reserved":0,"allocated":1601536,"data_stored":1022782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:46:40.203 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:46:40.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.367+0000 7f5a9be64640 1 Processor -- start 2024-04-17T10:46:40.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.367+0000 7f5a9be64640 1 -- start start 2024-04-17T10:46:40.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.367+0000 7f5a9be64640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5a94156160 0x7f5a9415bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:40.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.367+0000 7f5a9be64640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a9415c490 con 0x7f5a9415d190 2024-04-17T10:46:40.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.367+0000 7f5a9be64640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a9415c740 con 0x7f5a9406d840 2024-04-17T10:46:40.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.367+0000 7f5a9be64640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a9415ca20 con 0x7f5a94156160 2024-04-17T10:46:40.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a9a401640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5a9415d190 0x7f5a9406abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:37380/0 (socket says 172.21.15.83:37380) 2024-04-17T10:46:40.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a9a401640 1 -- 172.21.15.83:0/2302145298 learned_addr learned my addr 172.21.15.83:0/2302145298 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:46:40.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a99c00640 1 --2- 172.21.15.83:0/2302145298 >> v2:172.21.15.167:3300/0 conn(0x7f5a94156160 0x7f5a9415bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:40.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a98bfe640 1 -- 172.21.15.83:0/2302145298 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1766307128 0 0) 0x7f5a9415c490 con 0x7f5a9415d190 2024-04-17T10:46:40.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a98bfe640 1 -- 172.21.15.83:0/2302145298 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a80003530 con 0x7f5a9415d190 2024-04-17T10:46:40.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a98bfe640 1 -- 172.21.15.83:0/2302145298 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1603063363 0 0) 0x7f5a80003530 con 0x7f5a9415d190 2024-04-17T10:46:40.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a98bfe640 1 -- 172.21.15.83:0/2302145298 >> v2:172.21.15.167:3300/0 conn(0x7f5a94156160 msgr2=0x7f5a9415bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:40.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a98bfe640 1 --2- 172.21.15.83:0/2302145298 >> v2:172.21.15.167:3300/0 conn(0x7f5a94156160 0x7f5a9415bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:40.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a98bfe640 1 -- 172.21.15.83:0/2302145298 >> v1:172.21.15.167:6789/0 conn(0x7f5a9406d840 legacy=0x7f5a9415fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:40.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a98bfe640 1 -- 172.21.15.83:0/2302145298 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9415cd00 con 0x7f5a9415d190 2024-04-17T10:46:40.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a98bfe640 1 -- 172.21.15.83:0/2302145298 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f5a8c001a00 con 0x7f5a9415d190 2024-04-17T10:46:40.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a98bfe640 1 -- 172.21.15.83:0/2302145298 >> v1:172.21.15.83:6789/0 conn(0x7f5a9415d190 legacy=0x7f5a9406abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:40.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a98bfe640 1 --2- 172.21.15.83:0/2302145298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a800039c0 0x7f5a80003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:40.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a98bfe640 1 --2- 172.21.15.83:0/2302145298 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a80004810 0x7f5a80004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:40.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a98bfe640 1 --2- 172.21.15.83:0/2302145298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a800051d0 0x7f5a8000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:40.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a98bfe640 1 -- 172.21.15.83:0/2302145298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9415ca20 con 0x7f5a800039c0 2024-04-17T10:46:40.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a98bfe640 1 -- 172.21.15.83:0/2302145298 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8000c070 con 0x7f5a800051d0 2024-04-17T10:46:40.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a98bfe640 1 -- 172.21.15.83:0/2302145298 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8000c270 con 0x7f5a80004810 2024-04-17T10:46:40.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a99c00640 1 --2- 172.21.15.83:0/2302145298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a800039c0 0x7f5a80003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:40.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a993ff640 1 --2- 172.21.15.83:0/2302145298 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a80004810 0x7f5a80004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:40.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a99c00640 1 -- 172.21.15.83:0/2302145298 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a80004810 msgr2=0x7f5a80004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:40.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a99c00640 1 --2- 172.21.15.83:0/2302145298 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a80004810 0x7f5a80004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:40.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a99c00640 1 -- 172.21.15.83:0/2302145298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a800051d0 msgr2=0x7f5a8000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:40.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a9a401640 1 --2- 172.21.15.83:0/2302145298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a800051d0 0x7f5a8000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:40.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a99c00640 1 --2- 172.21.15.83:0/2302145298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a800051d0 0x7f5a8000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:40.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a99c00640 1 -- 172.21.15.83:0/2302145298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a80003530 con 0x7f5a800039c0 2024-04-17T10:46:40.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a9a401640 1 --2- 172.21.15.83:0/2302145298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a800051d0 0x7f5a8000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:40.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a993ff640 1 --2- 172.21.15.83:0/2302145298 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a80004810 0x7f5a80004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:40.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a9a401640 1 -- 172.21.15.83:0/2302145298 reap_dead start 2024-04-17T10:46:40.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a99c00640 1 --2- 172.21.15.83:0/2302145298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a800039c0 0x7f5a80003de0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f5a90009ae0 tx=0x7f5a9001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa036650825a975b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:40.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a98bfe640 1 -- 172.21.15.83:0/2302145298 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a900096b0 con 0x7f5a800039c0 2024-04-17T10:46:40.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a98bfe640 1 -- 172.21.15.83:0/2302145298 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a900a6450 con 0x7f5a800039c0 2024-04-17T10:46:40.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.371+0000 7f5a98bfe640 1 -- 172.21.15.83:0/2302145298 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a900a6840 con 0x7f5a800039c0 2024-04-17T10:46:40.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.375+0000 7f5a9be64640 1 -- 172.21.15.83:0/2302145298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a800039c0 msgr2=0x7f5a80003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:40.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.375+0000 7f5a9be64640 1 --2- 172.21.15.83:0/2302145298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a800039c0 0x7f5a80003de0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f5a90009ae0 tx=0x7f5a9001ca70 comp rx=0 tx=0).stop 2024-04-17T10:46:40.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.375+0000 7f5a9be64640 1 -- 172.21.15.83:0/2302145298 shutdown_connections 2024-04-17T10:46:40.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.375+0000 7f5a9be64640 1 --2- 172.21.15.83:0/2302145298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a800039c0 0x7f5a80003de0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:40.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.375+0000 7f5a9be64640 1 -- 172.21.15.83:0/2302145298 >> 172.21.15.83:0/2302145298 conn(0x7f5a940d07b0 msgr2=0x7f5a940d0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:40.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.375+0000 7f5a9be64640 1 -- 172.21.15.83:0/2302145298 shutdown_connections 2024-04-17T10:46:40.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.375+0000 7f5a9be64640 1 -- 172.21.15.83:0/2302145298 wait complete. 2024-04-17T10:46:40.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.375+0000 7f5a9be64640 1 Processor -- start 2024-04-17T10:46:40.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.375+0000 7f5a9be64640 1 -- start start 2024-04-17T10:46:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.375+0000 7f5a9be64640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a800039c0 0x7f5a9414b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.375+0000 7f5a9be64640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a941529f0 0x7f5a9415dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.375+0000 7f5a9be64640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a9415e190 0x7f5a941f8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.375+0000 7f5a9be64640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9406ad60 con 0x7f5a9415e190 2024-04-17T10:46:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.375+0000 7f5a9be64640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9406abe0 con 0x7f5a941529f0 2024-04-17T10:46:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.375+0000 7f5a9be64640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9406aee0 con 0x7f5a800039c0 2024-04-17T10:46:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.375+0000 7f5a9a401640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a9415e190 0x7f5a941f8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.375+0000 7f5a9a401640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a9415e190 0x7f5a941f8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47624/0 (socket says 172.21.15.83:47624) 2024-04-17T10:46:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.375+0000 7f5a9a401640 1 -- 172.21.15.83:0/167790925 learned_addr learned my addr 172.21.15.83:0/167790925 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.375+0000 7f5a993ff640 1 --2- 172.21.15.83:0/167790925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a941529f0 0x7f5a9415dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.375+0000 7f5a99c00640 1 --2- 172.21.15.83:0/167790925 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a800039c0 0x7f5a9414b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:40.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.375+0000 7f5a9a401640 1 -- 172.21.15.83:0/167790925 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a800039c0 msgr2=0x7f5a9414b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:40.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.375+0000 7f5a9a401640 1 --2- 172.21.15.83:0/167790925 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a800039c0 0x7f5a9414b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:40.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.375+0000 7f5a9a401640 1 -- 172.21.15.83:0/167790925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a941529f0 msgr2=0x7f5a9415dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:40.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.375+0000 7f5a9a401640 1 --2- 172.21.15.83:0/167790925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a941529f0 0x7f5a9415dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:40.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.375+0000 7f5a9a401640 1 -- 172.21.15.83:0/167790925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a941f9600 con 0x7f5a9415e190 2024-04-17T10:46:40.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.375+0000 7f5a9a401640 1 --2- 172.21.15.83:0/167790925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a9415e190 0x7f5a941f8880 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f5a9406ccd0 tx=0x7f5a8c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:40.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.375+0000 7f5a8a7fc640 1 -- 172.21.15.83:0/167790925 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c004d70 con 0x7f5a9415e190 2024-04-17T10:46:40.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.375+0000 7f5a8a7fc640 1 -- 172.21.15.83:0/167790925 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8c00bb20 con 0x7f5a9415e190 2024-04-17T10:46:40.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.375+0000 7f5a9be64640 1 -- 172.21.15.83:0/167790925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a94151c30 con 0x7f5a9415e190 2024-04-17T10:46:40.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.375+0000 7f5a8a7fc640 1 -- 172.21.15.83:0/167790925 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c007110 con 0x7f5a9415e190 2024-04-17T10:46:40.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.375+0000 7f5a9be64640 1 -- 172.21.15.83:0/167790925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a941521f0 con 0x7f5a9415e190 2024-04-17T10:46:40.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.379+0000 7f5a9be64640 1 -- 172.21.15.83:0/167790925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a94074f90 con 0x7f5a9415e190 2024-04-17T10:46:40.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.379+0000 7f5a8a7fc640 1 -- 172.21.15.83:0/167790925 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5a8c01b020 con 0x7f5a9415e190 2024-04-17T10:46:40.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.379+0000 7f5a8a7fc640 1 --2- 172.21.15.83:0/167790925 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5a78066f60 0x7f5a78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:40.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.379+0000 7f5a99c00640 1 --2- 172.21.15.83:0/167790925 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5a78066f60 0x7f5a78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:40.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.379+0000 7f5a8a7fc640 1 -- 172.21.15.83:0/167790925 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a8c0851f0 con 0x7f5a9415e190 2024-04-17T10:46:40.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.379+0000 7f5a99c00640 1 --2- 172.21.15.83:0/167790925 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5a78066f60 0x7f5a78069420 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5a900ad000 tx=0x7f5a900a9040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:40.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.383+0000 7f5a8a7fc640 1 -- 172.21.15.83:0/167790925 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5a8c056200 con 0x7f5a9415e190 2024-04-17T10:46:40.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.527+0000 7f5a9be64640 1 -- 172.21.15.83:0/167790925 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5a94068140 con 0x7f5a78066f60 2024-04-17T10:46:40.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.535+0000 7f5a8a7fc640 1 -- 172.21.15.83:0/167790925 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252090 (secure 0 0 0) 0x7f5a94068140 con 0x7f5a78066f60 2024-04-17T10:46:40.533 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:46:40.535 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:46:40.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.539+0000 7f5a9be64640 1 -- 172.21.15.83:0/167790925 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5a78066f60 msgr2=0x7f5a78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:40.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.539+0000 7f5a9be64640 1 --2- 172.21.15.83:0/167790925 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5a78066f60 0x7f5a78069420 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5a900ad000 tx=0x7f5a900a9040 comp rx=0 tx=0).stop 2024-04-17T10:46:40.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.539+0000 7f5a9be64640 1 -- 172.21.15.83:0/167790925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a9415e190 msgr2=0x7f5a941f8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:40.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.539+0000 7f5a9be64640 1 --2- 172.21.15.83:0/167790925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a9415e190 0x7f5a941f8880 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f5a9406ccd0 tx=0x7f5a8c00b4a0 comp rx=0 tx=0).stop 2024-04-17T10:46:40.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.539+0000 7f5a9be64640 1 -- 172.21.15.83:0/167790925 shutdown_connections 2024-04-17T10:46:40.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.539+0000 7f5a9be64640 1 --2- 172.21.15.83:0/167790925 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5a78066f60 0x7f5a78069420 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:40.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.539+0000 7f5a9be64640 1 --2- 172.21.15.83:0/167790925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a9415e190 0x7f5a941f8880 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:40.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.539+0000 7f5a9be64640 1 --2- 172.21.15.83:0/167790925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a941529f0 0x7f5a9415dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:40.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.539+0000 7f5a9be64640 1 --2- 172.21.15.83:0/167790925 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a800039c0 0x7f5a9414b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:40.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.539+0000 7f5a9be64640 1 -- 172.21.15.83:0/167790925 >> 172.21.15.83:0/167790925 conn(0x7f5a940d07b0 msgr2=0x7f5a940b3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:40.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.539+0000 7f5a9be64640 1 -- 172.21.15.83:0/167790925 shutdown_connections 2024-04-17T10:46:40.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.539+0000 7f5a9be64640 1 -- 172.21.15.83:0/167790925 wait complete. 2024-04-17T10:46:40.550 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-04-17T10:46:40.334384+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":151,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":488296,"kb_used_data":9768,"kb_used_omap":12,"kb_used_meta":478387,"kb_avail":754486424,"statfs":{"total":773094113280,"available":772594098176,"internally_reserved":0,"allocated":10002432,"data_stored":4968132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":489868848},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.135826"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.307839+0000","last_change":"2024-04-17T10:46:37.269771+0000","last_active":"2024-04-17T10:46:38.307839+0000","last_peered":"2024-04-17T10:46:38.307839+0000","last_clean":"2024-04-17T10:46:38.307839+0000","last_became_active":"2024-04-17T10:46:37.269122+0000","last_became_peered":"2024-04-17T10:46:37.269122+0000","last_unstale":"2024-04-17T10:46:38.307839+0000","last_undegraded":"2024-04-17T10:46:38.307839+0000","last_fullsized":"2024-04-17T10:46:38.307839+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:15:12.428436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.305843+0000","last_change":"2024-04-17T10:46:37.269741+0000","last_active":"2024-04-17T10:46:38.305843+0000","last_peered":"2024-04-17T10:46:38.305843+0000","last_clean":"2024-04-17T10:46:38.305843+0000","last_became_active":"2024-04-17T10:46:37.269391+0000","last_became_peered":"2024-04-17T10:46:37.269391+0000","last_unstale":"2024-04-17T10:46:38.305843+0000","last_undegraded":"2024-04-17T10:46:38.305843+0000","last_fullsized":"2024-04-17T10:46:38.305843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:39:43.593162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.219475+0000","last_change":"2024-04-17T10:46:37.270809+0000","last_active":"2024-04-17T10:46:38.219475+0000","last_peered":"2024-04-17T10:46:38.219475+0000","last_clean":"2024-04-17T10:46:38.219475+0000","last_became_active":"2024-04-17T10:46:37.270059+0000","last_became_peered":"2024-04-17T10:46:37.270059+0000","last_unstale":"2024-04-17T10:46:38.219475+0000","last_undegraded":"2024-04-17T10:46:38.219475+0000","last_fullsized":"2024-04-17T10:46:38.219475+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:30:56.593271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.305540+0000","last_change":"2024-04-17T10:46:37.224130+0000","last_active":"2024-04-17T10:46:38.305540+0000","last_peered":"2024-04-17T10:46:38.305540+0000","last_clean":"2024-04-17T10:46:38.305540+0000","last_became_active":"2024-04-17T10:46:37.223867+0000","last_became_peered":"2024-04-17T10:46:37.223867+0000","last_unstale":"2024-04-17T10:46:38.305540+0000","last_undegraded":"2024-04-17T10:46:38.305540+0000","last_fullsized":"2024-04-17T10:46:38.305540+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:17:17.182976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.220518+0000","last_change":"2024-04-17T10:46:37.253805+0000","last_active":"2024-04-17T10:46:38.220518+0000","last_peered":"2024-04-17T10:46:38.220518+0000","last_clean":"2024-04-17T10:46:38.220518+0000","last_became_active":"2024-04-17T10:46:37.240039+0000","last_became_peered":"2024-04-17T10:46:37.240039+0000","last_unstale":"2024-04-17T10:46:38.220518+0000","last_undegraded":"2024-04-17T10:46:38.220518+0000","last_fullsized":"2024-04-17T10:46:38.220518+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:02:14.237783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.306270+0000","last_change":"2024-04-17T10:46:37.250882+0000","last_active":"2024-04-17T10:46:38.306270+0000","last_peered":"2024-04-17T10:46:38.306270+0000","last_clean":"2024-04-17T10:46:38.306270+0000","last_became_active":"2024-04-17T10:46:37.248606+0000","last_became_peered":"2024-04-17T10:46:37.248606+0000","last_unstale":"2024-04-17T10:46:38.306270+0000","last_undegraded":"2024-04-17T10:46:38.306270+0000","last_fullsized":"2024-04-17T10:46:38.306270+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:15:12.428436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.304757+0000","last_change":"2024-04-17T10:46:37.220677+0000","last_active":"2024-04-17T10:46:38.304757+0000","last_peered":"2024-04-17T10:46:38.304757+0000","last_clean":"2024-04-17T10:46:38.304757+0000","last_became_active":"2024-04-17T10:46:37.220386+0000","last_became_peered":"2024-04-17T10:46:37.220386+0000","last_unstale":"2024-04-17T10:46:38.304757+0000","last_undegraded":"2024-04-17T10:46:38.304757+0000","last_fullsized":"2024-04-17T10:46:38.304757+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:11:11.638040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.305833+0000","last_change":"2024-04-17T10:46:37.249152+0000","last_active":"2024-04-17T10:46:38.305833+0000","last_peered":"2024-04-17T10:46:38.305833+0000","last_clean":"2024-04-17T10:46:38.305833+0000","last_became_active":"2024-04-17T10:46:37.248294+0000","last_became_peered":"2024-04-17T10:46:37.248294+0000","last_unstale":"2024-04-17T10:46:38.305833+0000","last_undegraded":"2024-04-17T10:46:38.305833+0000","last_fullsized":"2024-04-17T10:46:38.305833+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:13:08.182711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.307672+0000","last_change":"2024-04-17T10:46:37.248937+0000","last_active":"2024-04-17T10:46:38.307672+0000","last_peered":"2024-04-17T10:46:38.307672+0000","last_clean":"2024-04-17T10:46:38.307672+0000","last_became_active":"2024-04-17T10:46:37.248198+0000","last_became_peered":"2024-04-17T10:46:37.248198+0000","last_unstale":"2024-04-17T10:46:38.307672+0000","last_undegraded":"2024-04-17T10:46:38.307672+0000","last_fullsized":"2024-04-17T10:46:38.307672+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:34:35.813540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.307270+0000","last_change":"2024-04-17T10:46:37.263752+0000","last_active":"2024-04-17T10:46:38.307270+0000","last_peered":"2024-04-17T10:46:38.307270+0000","last_clean":"2024-04-17T10:46:38.307270+0000","last_became_active":"2024-04-17T10:46:37.262968+0000","last_became_peered":"2024-04-17T10:46:37.262968+0000","last_unstale":"2024-04-17T10:46:38.307270+0000","last_undegraded":"2024-04-17T10:46:38.307270+0000","last_fullsized":"2024-04-17T10:46:38.307270+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:28:47.815781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.305776+0000","last_change":"2024-04-17T10:46:37.244114+0000","last_active":"2024-04-17T10:46:38.305776+0000","last_peered":"2024-04-17T10:46:38.305776+0000","last_clean":"2024-04-17T10:46:38.305776+0000","last_became_active":"2024-04-17T10:46:37.243892+0000","last_became_peered":"2024-04-17T10:46:37.243892+0000","last_unstale":"2024-04-17T10:46:38.305776+0000","last_undegraded":"2024-04-17T10:46:38.305776+0000","last_fullsized":"2024-04-17T10:46:38.305776+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:16:45.499275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.306881+0000","last_change":"2024-04-17T10:46:37.272995+0000","last_active":"2024-04-17T10:46:38.306881+0000","last_peered":"2024-04-17T10:46:38.306881+0000","last_clean":"2024-04-17T10:46:38.306881+0000","last_became_active":"2024-04-17T10:46:37.269853+0000","last_became_peered":"2024-04-17T10:46:37.269853+0000","last_unstale":"2024-04-17T10:46:38.306881+0000","last_undegraded":"2024-04-17T10:46:38.306881+0000","last_fullsized":"2024-04-17T10:46:38.306881+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:32:04.935048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.219782+0000","last_change":"2024-04-17T10:46:37.247407+0000","last_active":"2024-04-17T10:46:38.219782+0000","last_peered":"2024-04-17T10:46:38.219782+0000","last_clean":"2024-04-17T10:46:38.219782+0000","last_became_active":"2024-04-17T10:46:37.246549+0000","last_became_peered":"2024-04-17T10:46:37.246549+0000","last_unstale":"2024-04-17T10:46:38.219782+0000","last_undegraded":"2024-04-17T10:46:38.219782+0000","last_fullsized":"2024-04-17T10:46:38.219782+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:51:19.314122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.305123+0000","last_change":"2024-04-17T10:46:37.225452+0000","last_active":"2024-04-17T10:46:38.305123+0000","last_peered":"2024-04-17T10:46:38.305123+0000","last_clean":"2024-04-17T10:46:38.305123+0000","last_became_active":"2024-04-17T10:46:37.225182+0000","last_became_peered":"2024-04-17T10:46:37.225182+0000","last_unstale":"2024-04-17T10:46:38.305123+0000","last_undegraded":"2024-04-17T10:46:38.305123+0000","last_fullsized":"2024-04-17T10:46:38.305123+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:31:55.267433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.304746+0000","last_change":"2024-04-17T10:46:37.247878+0000","last_active":"2024-04-17T10:46:38.304746+0000","last_peered":"2024-04-17T10:46:38.304746+0000","last_clean":"2024-04-17T10:46:38.304746+0000","last_became_active":"2024-04-17T10:46:37.247570+0000","last_became_peered":"2024-04-17T10:46:37.247570+0000","last_unstale":"2024-04-17T10:46:38.304746+0000","last_undegraded":"2024-04-17T10:46:38.304746+0000","last_fullsized":"2024-04-17T10:46:38.304746+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:51:19.314122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.219036+0000","last_change":"2024-04-17T10:46:37.263215+0000","last_active":"2024-04-17T10:46:38.219036+0000","last_peered":"2024-04-17T10:46:38.219036+0000","last_clean":"2024-04-17T10:46:38.219036+0000","last_became_active":"2024-04-17T10:46:37.262477+0000","last_became_peered":"2024-04-17T10:46:37.262477+0000","last_unstale":"2024-04-17T10:46:38.219036+0000","last_undegraded":"2024-04-17T10:46:38.219036+0000","last_fullsized":"2024-04-17T10:46:38.219036+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:32:04.935048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.220902+0000","last_change":"2024-04-17T10:46:37.234286+0000","last_active":"2024-04-17T10:46:38.220902+0000","last_peered":"2024-04-17T10:46:38.220902+0000","last_clean":"2024-04-17T10:46:38.220902+0000","last_became_active":"2024-04-17T10:46:37.231008+0000","last_became_peered":"2024-04-17T10:46:37.231008+0000","last_unstale":"2024-04-17T10:46:38.220902+0000","last_undegraded":"2024-04-17T10:46:38.220902+0000","last_fullsized":"2024-04-17T10:46:38.220902+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:31:39.193659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.305989+0000","last_change":"2024-04-17T10:46:37.269744+0000","last_active":"2024-04-17T10:46:38.305989+0000","last_peered":"2024-04-17T10:46:38.305989+0000","last_clean":"2024-04-17T10:46:38.305989+0000","last_became_active":"2024-04-17T10:46:37.269379+0000","last_became_peered":"2024-04-17T10:46:37.269379+0000","last_unstale":"2024-04-17T10:46:38.305989+0000","last_undegraded":"2024-04-17T10:46:38.305989+0000","last_fullsized":"2024-04-17T10:46:38.305989+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:23:33.238449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.217823+0000","last_change":"2024-04-17T10:46:37.234266+0000","last_active":"2024-04-17T10:46:38.217823+0000","last_peered":"2024-04-17T10:46:38.217823+0000","last_clean":"2024-04-17T10:46:38.217823+0000","last_became_active":"2024-04-17T10:46:37.232175+0000","last_became_peered":"2024-04-17T10:46:37.232175+0000","last_unstale":"2024-04-17T10:46:38.217823+0000","last_undegraded":"2024-04-17T10:46:38.217823+0000","last_fullsized":"2024-04-17T10:46:38.217823+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:41.169548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.305543+0000","last_change":"2024-04-17T10:46:37.233146+0000","last_active":"2024-04-17T10:46:38.305543+0000","last_peered":"2024-04-17T10:46:38.305543+0000","last_clean":"2024-04-17T10:46:38.305543+0000","last_became_active":"2024-04-17T10:46:37.232728+0000","last_became_peered":"2024-04-17T10:46:37.232728+0000","last_unstale":"2024-04-17T10:46:38.305543+0000","last_undegraded":"2024-04-17T10:46:38.305543+0000","last_fullsized":"2024-04-17T10:46:38.305543+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:15:20.366737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.305380+0000","last_change":"2024-04-17T10:46:37.232721+0000","last_active":"2024-04-17T10:46:38.305380+0000","last_peered":"2024-04-17T10:46:38.305380+0000","last_clean":"2024-04-17T10:46:38.305380+0000","last_became_active":"2024-04-17T10:46:37.232415+0000","last_became_peered":"2024-04-17T10:46:37.232415+0000","last_unstale":"2024-04-17T10:46:38.305380+0000","last_undegraded":"2024-04-17T10:46:38.305380+0000","last_fullsized":"2024-04-17T10:46:38.305380+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:33:24.661163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.304751+0000","last_change":"2024-04-17T10:46:37.236607+0000","last_active":"2024-04-17T10:46:38.304751+0000","last_peered":"2024-04-17T10:46:38.304751+0000","last_clean":"2024-04-17T10:46:38.304751+0000","last_became_active":"2024-04-17T10:46:37.235960+0000","last_became_peered":"2024-04-17T10:46:37.235960+0000","last_unstale":"2024-04-17T10:46:38.304751+0000","last_undegraded":"2024-04-17T10:46:38.304751+0000","last_fullsized":"2024-04-17T10:46:38.304751+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:25:03.784829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.307295+0000","last_change":"2024-04-17T10:46:37.259129+0000","last_active":"2024-04-17T10:46:38.307295+0000","last_peered":"2024-04-17T10:46:38.307295+0000","last_clean":"2024-04-17T10:46:38.307295+0000","last_became_active":"2024-04-17T10:46:37.258329+0000","last_became_peered":"2024-04-17T10:46:37.258329+0000","last_unstale":"2024-04-17T10:46:38.307295+0000","last_undegraded":"2024-04-17T10:46:38.307295+0000","last_fullsized":"2024-04-17T10:46:38.307295+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:02:14.237783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.304470+0000","last_change":"2024-04-17T10:46:37.220518+0000","last_active":"2024-04-17T10:46:38.304470+0000","last_peered":"2024-04-17T10:46:38.304470+0000","last_clean":"2024-04-17T10:46:38.304470+0000","last_became_active":"2024-04-17T10:46:37.220219+0000","last_became_peered":"2024-04-17T10:46:37.220219+0000","last_unstale":"2024-04-17T10:46:38.304470+0000","last_undegraded":"2024-04-17T10:46:38.304470+0000","last_fullsized":"2024-04-17T10:46:38.304470+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:21:51.066578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.305948+0000","last_change":"2024-04-17T10:46:37.260407+0000","last_active":"2024-04-17T10:46:38.305948+0000","last_peered":"2024-04-17T10:46:38.305948+0000","last_clean":"2024-04-17T10:46:38.305948+0000","last_became_active":"2024-04-17T10:46:37.260200+0000","last_became_peered":"2024-04-17T10:46:37.260200+0000","last_unstale":"2024-04-17T10:46:38.305948+0000","last_undegraded":"2024-04-17T10:46:38.305948+0000","last_fullsized":"2024-04-17T10:46:38.305948+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:49:27.574418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.220177+0000","last_change":"2024-04-17T10:46:37.245419+0000","last_active":"2024-04-17T10:46:38.220177+0000","last_peered":"2024-04-17T10:46:38.220177+0000","last_clean":"2024-04-17T10:46:38.220177+0000","last_became_active":"2024-04-17T10:46:37.239167+0000","last_became_peered":"2024-04-17T10:46:37.239167+0000","last_unstale":"2024-04-17T10:46:38.220177+0000","last_undegraded":"2024-04-17T10:46:38.220177+0000","last_fullsized":"2024-04-17T10:46:38.220177+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:08.155094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.221207+0000","last_change":"2024-04-17T10:46:37.235367+0000","last_active":"2024-04-17T10:46:38.221207+0000","last_peered":"2024-04-17T10:46:38.221207+0000","last_clean":"2024-04-17T10:46:38.221207+0000","last_became_active":"2024-04-17T10:46:37.234597+0000","last_became_peered":"2024-04-17T10:46:37.234597+0000","last_unstale":"2024-04-17T10:46:38.221207+0000","last_undegraded":"2024-04-17T10:46:38.221207+0000","last_fullsized":"2024-04-17T10:46:38.221207+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:04:49.666714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.307116+0000","last_change":"2024-04-17T10:46:37.272970+0000","last_active":"2024-04-17T10:46:38.307116+0000","last_peered":"2024-04-17T10:46:38.307116+0000","last_clean":"2024-04-17T10:46:38.307116+0000","last_became_active":"2024-04-17T10:46:37.270259+0000","last_became_peered":"2024-04-17T10:46:37.270259+0000","last_unstale":"2024-04-17T10:46:38.307116+0000","last_undegraded":"2024-04-17T10:46:38.307116+0000","last_fullsized":"2024-04-17T10:46:38.307116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:20:00.786089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.305114+0000","last_change":"2024-04-17T10:46:37.236916+0000","last_active":"2024-04-17T10:46:38.305114+0000","last_peered":"2024-04-17T10:46:38.305114+0000","last_clean":"2024-04-17T10:46:38.305114+0000","last_became_active":"2024-04-17T10:46:37.236405+0000","last_became_peered":"2024-04-17T10:46:37.236405+0000","last_unstale":"2024-04-17T10:46:38.305114+0000","last_undegraded":"2024-04-17T10:46:38.305114+0000","last_fullsized":"2024-04-17T10:46:38.305114+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:31:09.017895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.304888+0000","last_change":"2024-04-17T10:46:37.234848+0000","last_active":"2024-04-17T10:46:38.304888+0000","last_peered":"2024-04-17T10:46:38.304888+0000","last_clean":"2024-04-17T10:46:38.304888+0000","last_became_active":"2024-04-17T10:46:37.234569+0000","last_became_peered":"2024-04-17T10:46:37.234569+0000","last_unstale":"2024-04-17T10:46:38.304888+0000","last_undegraded":"2024-04-17T10:46:38.304888+0000","last_fullsized":"2024-04-17T10:46:38.304888+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:07.608711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.219489+0000","last_change":"2024-04-17T10:46:37.256862+0000","last_active":"2024-04-17T10:46:38.219489+0000","last_peered":"2024-04-17T10:46:38.219489+0000","last_clean":"2024-04-17T10:46:38.219489+0000","last_became_active":"2024-04-17T10:46:37.256010+0000","last_became_peered":"2024-04-17T10:46:37.256010+0000","last_unstale":"2024-04-17T10:46:38.219489+0000","last_undegraded":"2024-04-17T10:46:38.219489+0000","last_fullsized":"2024-04-17T10:46:38.219489+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:20:00.786089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.306187+0000","last_change":"2024-04-17T10:46:37.257338+0000","last_active":"2024-04-17T10:46:38.306187+0000","last_peered":"2024-04-17T10:46:38.306187+0000","last_clean":"2024-04-17T10:46:38.306187+0000","last_became_active":"2024-04-17T10:46:37.245576+0000","last_became_peered":"2024-04-17T10:46:37.245576+0000","last_unstale":"2024-04-17T10:46:38.306187+0000","last_undegraded":"2024-04-17T10:46:38.306187+0000","last_fullsized":"2024-04-17T10:46:38.306187+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:35:27.964754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.220497+0000","last_change":"2024-04-17T10:46:37.274317+0000","last_active":"2024-04-17T10:46:38.220497+0000","last_peered":"2024-04-17T10:46:38.220497+0000","last_clean":"2024-04-17T10:46:38.220497+0000","last_became_active":"2024-04-17T10:46:37.274007+0000","last_became_peered":"2024-04-17T10:46:37.274007+0000","last_unstale":"2024-04-17T10:46:38.220497+0000","last_undegraded":"2024-04-17T10:46:38.220497+0000","last_fullsized":"2024-04-17T10:46:38.220497+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:28:47.815781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"88'4","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.308028+0000","last_change":"2024-04-17T10:46:36.244232+0000","last_active":"2024-04-17T10:46:38.308028+0000","last_peered":"2024-04-17T10:46:38.308028+0000","last_clean":"2024-04-17T10:46:38.308028+0000","last_became_active":"2024-04-17T10:46:36.242828+0000","last_became_peered":"2024-04-17T10:46:36.242828+0000","last_unstale":"2024-04-17T10:46:38.308028+0000","last_undegraded":"2024-04-17T10:46:38.308028+0000","last_fullsized":"2024-04-17T10:46:38.308028+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:34:44.797492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.307878+0000","last_change":"2024-04-17T10:46:37.268462+0000","last_active":"2024-04-17T10:46:38.307878+0000","last_peered":"2024-04-17T10:46:38.307878+0000","last_clean":"2024-04-17T10:46:38.307878+0000","last_became_active":"2024-04-17T10:46:37.267865+0000","last_became_peered":"2024-04-17T10:46:37.267865+0000","last_unstale":"2024-04-17T10:46:38.307878+0000","last_undegraded":"2024-04-17T10:46:38.307878+0000","last_fullsized":"2024-04-17T10:46:38.307878+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:04:49.666714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"88'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.220807+0000","last_change":"2024-04-17T10:46:36.227002+0000","last_active":"2024-04-17T10:46:38.220807+0000","last_peered":"2024-04-17T10:46:38.220807+0000","last_clean":"2024-04-17T10:46:38.220807+0000","last_became_active":"2024-04-17T10:46:36.224917+0000","last_became_peered":"2024-04-17T10:46:36.224917+0000","last_unstale":"2024-04-17T10:46:38.220807+0000","last_undegraded":"2024-04-17T10:46:38.220807+0000","last_fullsized":"2024-04-17T10:46:38.220807+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:06:57.904005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.307479+0000","last_change":"2024-04-17T10:46:37.264609+0000","last_active":"2024-04-17T10:46:38.307479+0000","last_peered":"2024-04-17T10:46:38.307479+0000","last_clean":"2024-04-17T10:46:38.307479+0000","last_became_active":"2024-04-17T10:46:37.263803+0000","last_became_peered":"2024-04-17T10:46:37.263803+0000","last_unstale":"2024-04-17T10:46:38.307479+0000","last_undegraded":"2024-04-17T10:46:38.307479+0000","last_fullsized":"2024-04-17T10:46:38.307479+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:13:08.182711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"88'2","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.217468+0000","last_change":"2024-04-17T10:46:36.224138+0000","last_active":"2024-04-17T10:46:38.217468+0000","last_peered":"2024-04-17T10:46:38.217468+0000","last_clean":"2024-04-17T10:46:38.217468+0000","last_became_active":"2024-04-17T10:46:36.223748+0000","last_became_peered":"2024-04-17T10:46:36.223748+0000","last_unstale":"2024-04-17T10:46:38.217468+0000","last_undegraded":"2024-04-17T10:46:38.217468+0000","last_fullsized":"2024-04-17T10:46:38.217468+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:56:59.363078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.219505+0000","last_change":"2024-04-17T10:46:37.246030+0000","last_active":"2024-04-17T10:46:38.219505+0000","last_peered":"2024-04-17T10:46:38.219505+0000","last_clean":"2024-04-17T10:46:38.219505+0000","last_became_active":"2024-04-17T10:46:37.239480+0000","last_became_peered":"2024-04-17T10:46:37.239480+0000","last_unstale":"2024-04-17T10:46:38.219505+0000","last_undegraded":"2024-04-17T10:46:38.219505+0000","last_fullsized":"2024-04-17T10:46:38.219505+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:34:24.262783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"88'2","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.217406+0000","last_change":"2024-04-17T10:46:36.225001+0000","last_active":"2024-04-17T10:46:38.217406+0000","last_peered":"2024-04-17T10:46:38.217406+0000","last_clean":"2024-04-17T10:46:38.217406+0000","last_became_active":"2024-04-17T10:46:36.224659+0000","last_became_peered":"2024-04-17T10:46:36.224659+0000","last_unstale":"2024-04-17T10:46:38.217406+0000","last_undegraded":"2024-04-17T10:46:38.217406+0000","last_fullsized":"2024-04-17T10:46:38.217406+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:01:03.748435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.305016+0000","last_change":"2024-04-17T10:46:37.228612+0000","last_active":"2024-04-17T10:46:38.305016+0000","last_peered":"2024-04-17T10:46:38.305016+0000","last_clean":"2024-04-17T10:46:38.305016+0000","last_became_active":"2024-04-17T10:46:37.228287+0000","last_became_peered":"2024-04-17T10:46:37.228287+0000","last_unstale":"2024-04-17T10:46:38.305016+0000","last_undegraded":"2024-04-17T10:46:38.305016+0000","last_fullsized":"2024-04-17T10:46:38.305016+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:02:31.538030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"88'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.221456+0000","last_change":"2024-04-17T10:46:36.227665+0000","last_active":"2024-04-17T10:46:38.221456+0000","last_peered":"2024-04-17T10:46:38.221456+0000","last_clean":"2024-04-17T10:46:38.221456+0000","last_became_active":"2024-04-17T10:46:36.226830+0000","last_became_peered":"2024-04-17T10:46:36.226830+0000","last_unstale":"2024-04-17T10:46:38.221456+0000","last_undegraded":"2024-04-17T10:46:38.221456+0000","last_fullsized":"2024-04-17T10:46:38.221456+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:08:38.415067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.220977+0000","last_change":"2024-04-17T10:46:37.234567+0000","last_active":"2024-04-17T10:46:38.220977+0000","last_peered":"2024-04-17T10:46:38.220977+0000","last_clean":"2024-04-17T10:46:38.220977+0000","last_became_active":"2024-04-17T10:46:37.231177+0000","last_became_peered":"2024-04-17T10:46:37.231177+0000","last_unstale":"2024-04-17T10:46:38.220977+0000","last_undegraded":"2024-04-17T10:46:38.220977+0000","last_fullsized":"2024-04-17T10:46:38.220977+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:15:12.428436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"88'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.219803+0000","last_change":"2024-04-17T10:46:36.229427+0000","last_active":"2024-04-17T10:46:38.219803+0000","last_peered":"2024-04-17T10:46:38.219803+0000","last_clean":"2024-04-17T10:46:38.219803+0000","last_became_active":"2024-04-17T10:46:36.228665+0000","last_became_peered":"2024-04-17T10:46:36.228665+0000","last_unstale":"2024-04-17T10:46:38.219803+0000","last_undegraded":"2024-04-17T10:46:38.219803+0000","last_fullsized":"2024-04-17T10:46:38.219803+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:57:09.453756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.218730+0000","last_change":"2024-04-17T10:46:37.252461+0000","last_active":"2024-04-17T10:46:38.218730+0000","last_peered":"2024-04-17T10:46:38.218730+0000","last_clean":"2024-04-17T10:46:38.218730+0000","last_became_active":"2024-04-17T10:46:37.238605+0000","last_became_peered":"2024-04-17T10:46:37.238605+0000","last_unstale":"2024-04-17T10:46:38.218730+0000","last_undegraded":"2024-04-17T10:46:38.218730+0000","last_fullsized":"2024-04-17T10:46:38.218730+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:47:45.589390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.217018+0000","last_change":"2024-04-17T10:46:36.237135+0000","last_active":"2024-04-17T10:46:38.217018+0000","last_peered":"2024-04-17T10:46:38.217018+0000","last_clean":"2024-04-17T10:46:38.217018+0000","last_became_active":"2024-04-17T10:46:36.236845+0000","last_became_peered":"2024-04-17T10:46:36.236845+0000","last_unstale":"2024-04-17T10:46:38.217018+0000","last_undegraded":"2024-04-17T10:46:38.217018+0000","last_fullsized":"2024-04-17T10:46:38.217018+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:48:24.381082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.304927+0000","last_change":"2024-04-17T10:46:37.239885+0000","last_active":"2024-04-17T10:46:38.304927+0000","last_peered":"2024-04-17T10:46:38.304927+0000","last_clean":"2024-04-17T10:46:38.304927+0000","last_became_active":"2024-04-17T10:46:37.239509+0000","last_became_peered":"2024-04-17T10:46:37.239509+0000","last_unstale":"2024-04-17T10:46:38.304927+0000","last_undegraded":"2024-04-17T10:46:38.304927+0000","last_fullsized":"2024-04-17T10:46:38.304927+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:07:51.400779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.304835+0000","last_change":"2024-04-17T10:46:36.237984+0000","last_active":"2024-04-17T10:46:38.304835+0000","last_peered":"2024-04-17T10:46:38.304835+0000","last_clean":"2024-04-17T10:46:38.304835+0000","last_became_active":"2024-04-17T10:46:36.237496+0000","last_became_peered":"2024-04-17T10:46:36.237496+0000","last_unstale":"2024-04-17T10:46:38.304835+0000","last_undegraded":"2024-04-17T10:46:38.304835+0000","last_fullsized":"2024-04-17T10:46:38.304835+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:51:44.106996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.305589+0000","last_change":"2024-04-17T10:46:37.229516+0000","last_active":"2024-04-17T10:46:38.305589+0000","last_peered":"2024-04-17T10:46:38.305589+0000","last_clean":"2024-04-17T10:46:38.305589+0000","last_became_active":"2024-04-17T10:46:37.228713+0000","last_became_peered":"2024-04-17T10:46:37.228713+0000","last_unstale":"2024-04-17T10:46:38.305589+0000","last_undegraded":"2024-04-17T10:46:38.305589+0000","last_fullsized":"2024-04-17T10:46:38.305589+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:08.155094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.308182+0000","last_change":"2024-04-17T10:46:36.240909+0000","last_active":"2024-04-17T10:46:38.308182+0000","last_peered":"2024-04-17T10:46:38.308182+0000","last_clean":"2024-04-17T10:46:38.308182+0000","last_became_active":"2024-04-17T10:46:36.240132+0000","last_became_peered":"2024-04-17T10:46:36.240132+0000","last_unstale":"2024-04-17T10:46:38.308182+0000","last_undegraded":"2024-04-17T10:46:38.308182+0000","last_fullsized":"2024-04-17T10:46:38.308182+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:08:38.415067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.307906+0000","last_change":"2024-04-17T10:46:37.238806+0000","last_active":"2024-04-17T10:46:38.307906+0000","last_peered":"2024-04-17T10:46:38.307906+0000","last_clean":"2024-04-17T10:46:38.307906+0000","last_became_active":"2024-04-17T10:46:37.237796+0000","last_became_peered":"2024-04-17T10:46:37.237796+0000","last_unstale":"2024-04-17T10:46:38.307906+0000","last_undegraded":"2024-04-17T10:46:38.307906+0000","last_fullsized":"2024-04-17T10:46:38.307906+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:32:14.502209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.307476+0000","last_change":"2024-04-17T10:46:36.243848+0000","last_active":"2024-04-17T10:46:38.307476+0000","last_peered":"2024-04-17T10:46:38.307476+0000","last_clean":"2024-04-17T10:46:38.307476+0000","last_became_active":"2024-04-17T10:46:36.243111+0000","last_became_peered":"2024-04-17T10:46:36.243111+0000","last_unstale":"2024-04-17T10:46:38.307476+0000","last_undegraded":"2024-04-17T10:46:38.307476+0000","last_fullsized":"2024-04-17T10:46:38.307476+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:15:59.957009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.217544+0000","last_change":"2024-04-17T10:46:37.274037+0000","last_active":"2024-04-17T10:46:38.217544+0000","last_peered":"2024-04-17T10:46:38.217544+0000","last_clean":"2024-04-17T10:46:38.217544+0000","last_became_active":"2024-04-17T10:46:37.273764+0000","last_became_peered":"2024-04-17T10:46:37.273764+0000","last_unstale":"2024-04-17T10:46:38.217544+0000","last_undegraded":"2024-04-17T10:46:38.217544+0000","last_fullsized":"2024-04-17T10:46:38.217544+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:31:39.193659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"88'3","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.306475+0000","last_change":"2024-04-17T10:46:36.243230+0000","last_active":"2024-04-17T10:46:38.306475+0000","last_peered":"2024-04-17T10:46:38.306475+0000","last_clean":"2024-04-17T10:46:38.306475+0000","last_became_active":"2024-04-17T10:46:36.242317+0000","last_became_peered":"2024-04-17T10:46:36.242317+0000","last_unstale":"2024-04-17T10:46:38.306475+0000","last_undegraded":"2024-04-17T10:46:38.306475+0000","last_fullsized":"2024-04-17T10:46:38.306475+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:32:31.315206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.221423+0000","last_change":"2024-04-17T10:46:37.256766+0000","last_active":"2024-04-17T10:46:38.221423+0000","last_peered":"2024-04-17T10:46:38.221423+0000","last_clean":"2024-04-17T10:46:38.221423+0000","last_became_active":"2024-04-17T10:46:37.254332+0000","last_became_peered":"2024-04-17T10:46:37.254332+0000","last_unstale":"2024-04-17T10:46:38.221423+0000","last_undegraded":"2024-04-17T10:46:38.221423+0000","last_fullsized":"2024-04-17T10:46:38.221423+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:41.169548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"88'6","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.306406+0000","last_change":"2024-04-17T10:46:36.246775+0000","last_active":"2024-04-17T10:46:38.306406+0000","last_peered":"2024-04-17T10:46:38.306406+0000","last_clean":"2024-04-17T10:46:38.306406+0000","last_became_active":"2024-04-17T10:46:36.246005+0000","last_became_peered":"2024-04-17T10:46:36.246005+0000","last_unstale":"2024-04-17T10:46:38.306406+0000","last_undegraded":"2024-04-17T10:46:38.306406+0000","last_fullsized":"2024-04-17T10:46:38.306406+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:56:59.363078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.217823+0000","last_change":"2024-04-17T10:46:37.240656+0000","last_active":"2024-04-17T10:46:38.217823+0000","last_peered":"2024-04-17T10:46:38.217823+0000","last_clean":"2024-04-17T10:46:38.217823+0000","last_became_active":"2024-04-17T10:46:37.240369+0000","last_became_peered":"2024-04-17T10:46:37.240369+0000","last_unstale":"2024-04-17T10:46:38.217823+0000","last_undegraded":"2024-04-17T10:46:38.217823+0000","last_fullsized":"2024-04-17T10:46:38.217823+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:04:49.666714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"88'2","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.305239+0000","last_change":"2024-04-17T10:46:36.222655+0000","last_active":"2024-04-17T10:46:38.305239+0000","last_peered":"2024-04-17T10:46:38.305239+0000","last_clean":"2024-04-17T10:46:38.305239+0000","last_became_active":"2024-04-17T10:46:36.222335+0000","last_became_peered":"2024-04-17T10:46:36.222335+0000","last_unstale":"2024-04-17T10:46:38.305239+0000","last_undegraded":"2024-04-17T10:46:38.305239+0000","last_fullsized":"2024-04-17T10:46:38.305239+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:48:35.121494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.220155+0000","last_change":"2024-04-17T10:46:37.238232+0000","last_active":"2024-04-17T10:46:38.220155+0000","last_peered":"2024-04-17T10:46:38.220155+0000","last_clean":"2024-04-17T10:46:38.220155+0000","last_became_active":"2024-04-17T10:46:37.237140+0000","last_became_peered":"2024-04-17T10:46:37.237140+0000","last_unstale":"2024-04-17T10:46:38.220155+0000","last_undegraded":"2024-04-17T10:46:38.220155+0000","last_fullsized":"2024-04-17T10:46:38.220155+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:07:51.400779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.305123+0000","last_change":"2024-04-17T10:46:36.220569+0000","last_active":"2024-04-17T10:46:38.305123+0000","last_peered":"2024-04-17T10:46:38.305123+0000","last_clean":"2024-04-17T10:46:38.305123+0000","last_became_active":"2024-04-17T10:46:36.220298+0000","last_became_peered":"2024-04-17T10:46:36.220298+0000","last_unstale":"2024-04-17T10:46:38.305123+0000","last_undegraded":"2024-04-17T10:46:38.305123+0000","last_fullsized":"2024-04-17T10:46:38.305123+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:47:44.576003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.304717+0000","last_change":"2024-04-17T10:46:37.228636+0000","last_active":"2024-04-17T10:46:38.304717+0000","last_peered":"2024-04-17T10:46:38.304717+0000","last_clean":"2024-04-17T10:46:38.304717+0000","last_became_active":"2024-04-17T10:46:37.228327+0000","last_became_peered":"2024-04-17T10:46:37.228327+0000","last_unstale":"2024-04-17T10:46:38.304717+0000","last_undegraded":"2024-04-17T10:46:38.304717+0000","last_fullsized":"2024-04-17T10:46:38.304717+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:27:02.702511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.308034+0000","last_change":"2024-04-17T10:46:36.247744+0000","last_active":"2024-04-17T10:46:38.308034+0000","last_peered":"2024-04-17T10:46:38.308034+0000","last_clean":"2024-04-17T10:46:38.308034+0000","last_became_active":"2024-04-17T10:46:36.247175+0000","last_became_peered":"2024-04-17T10:46:36.247175+0000","last_unstale":"2024-04-17T10:46:38.308034+0000","last_undegraded":"2024-04-17T10:46:38.308034+0000","last_fullsized":"2024-04-17T10:46:38.308034+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:35:42.074181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.304690+0000","last_change":"2024-04-17T10:46:37.269163+0000","last_active":"2024-04-17T10:46:38.304690+0000","last_peered":"2024-04-17T10:46:38.304690+0000","last_clean":"2024-04-17T10:46:38.304690+0000","last_became_active":"2024-04-17T10:46:37.268826+0000","last_became_peered":"2024-04-17T10:46:37.268826+0000","last_unstale":"2024-04-17T10:46:38.304690+0000","last_undegraded":"2024-04-17T10:46:38.304690+0000","last_fullsized":"2024-04-17T10:46:38.304690+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:59:15.563893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"88'8","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.216891+0000","last_change":"2024-04-17T10:46:36.235377+0000","last_active":"2024-04-17T10:46:38.216891+0000","last_peered":"2024-04-17T10:46:38.216891+0000","last_clean":"2024-04-17T10:46:38.216891+0000","last_became_active":"2024-04-17T10:46:36.235126+0000","last_became_peered":"2024-04-17T10:46:36.235126+0000","last_unstale":"2024-04-17T10:46:38.216891+0000","last_undegraded":"2024-04-17T10:46:38.216891+0000","last_fullsized":"2024-04-17T10:46:38.216891+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:16:08.580172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.219776+0000","last_change":"2024-04-17T10:46:37.253741+0000","last_active":"2024-04-17T10:46:38.219776+0000","last_peered":"2024-04-17T10:46:38.219776+0000","last_clean":"2024-04-17T10:46:38.219776+0000","last_became_active":"2024-04-17T10:46:37.251342+0000","last_became_peered":"2024-04-17T10:46:37.251342+0000","last_unstale":"2024-04-17T10:46:38.219776+0000","last_undegraded":"2024-04-17T10:46:38.219776+0000","last_fullsized":"2024-04-17T10:46:38.219776+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:02:18.957145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"88'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.308307+0000","last_change":"2024-04-17T10:46:36.248451+0000","last_active":"2024-04-17T10:46:38.308307+0000","last_peered":"2024-04-17T10:46:38.308307+0000","last_clean":"2024-04-17T10:46:38.308307+0000","last_became_active":"2024-04-17T10:46:36.247798+0000","last_became_peered":"2024-04-17T10:46:36.247798+0000","last_unstale":"2024-04-17T10:46:38.308307+0000","last_undegraded":"2024-04-17T10:46:38.308307+0000","last_fullsized":"2024-04-17T10:46:38.308307+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:32:24.323774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.308046+0000","last_change":"2024-04-17T10:46:37.260989+0000","last_active":"2024-04-17T10:46:38.308046+0000","last_peered":"2024-04-17T10:46:38.308046+0000","last_clean":"2024-04-17T10:46:38.308046+0000","last_became_active":"2024-04-17T10:46:37.260147+0000","last_became_peered":"2024-04-17T10:46:37.260147+0000","last_unstale":"2024-04-17T10:46:38.308046+0000","last_undegraded":"2024-04-17T10:46:38.308046+0000","last_fullsized":"2024-04-17T10:46:38.308046+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:41.169548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"88'2","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.217507+0000","last_change":"2024-04-17T10:46:36.239610+0000","last_active":"2024-04-17T10:46:38.217507+0000","last_peered":"2024-04-17T10:46:38.217507+0000","last_clean":"2024-04-17T10:46:38.217507+0000","last_became_active":"2024-04-17T10:46:36.239329+0000","last_became_peered":"2024-04-17T10:46:36.239329+0000","last_unstale":"2024-04-17T10:46:38.217507+0000","last_undegraded":"2024-04-17T10:46:38.217507+0000","last_fullsized":"2024-04-17T10:46:38.217507+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:35:42.074181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.221710+0000","last_change":"2024-04-17T10:46:37.255117+0000","last_active":"2024-04-17T10:46:38.221710+0000","last_peered":"2024-04-17T10:46:38.221710+0000","last_clean":"2024-04-17T10:46:38.221710+0000","last_became_active":"2024-04-17T10:46:37.240160+0000","last_became_peered":"2024-04-17T10:46:37.240160+0000","last_unstale":"2024-04-17T10:46:38.221710+0000","last_undegraded":"2024-04-17T10:46:38.221710+0000","last_fullsized":"2024-04-17T10:46:38.221710+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:34:35.813540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.217613+0000","last_change":"2024-04-17T10:46:36.237315+0000","last_active":"2024-04-17T10:46:38.217613+0000","last_peered":"2024-04-17T10:46:38.217613+0000","last_clean":"2024-04-17T10:46:38.217613+0000","last_became_active":"2024-04-17T10:46:36.236967+0000","last_became_peered":"2024-04-17T10:46:36.236967+0000","last_unstale":"2024-04-17T10:46:38.217613+0000","last_undegraded":"2024-04-17T10:46:38.217613+0000","last_fullsized":"2024-04-17T10:46:38.217613+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:35:26.951367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.305162+0000","last_change":"2024-04-17T10:46:37.225316+0000","last_active":"2024-04-17T10:46:38.305162+0000","last_peered":"2024-04-17T10:46:38.305162+0000","last_clean":"2024-04-17T10:46:38.305162+0000","last_became_active":"2024-04-17T10:46:37.225027+0000","last_became_peered":"2024-04-17T10:46:37.225027+0000","last_unstale":"2024-04-17T10:46:38.305162+0000","last_undegraded":"2024-04-17T10:46:38.305162+0000","last_fullsized":"2024-04-17T10:46:38.305162+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:25:51.214524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"88'2","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.305229+0000","last_change":"2024-04-17T10:46:36.222745+0000","last_active":"2024-04-17T10:46:38.305229+0000","last_peered":"2024-04-17T10:46:38.305229+0000","last_clean":"2024-04-17T10:46:38.305229+0000","last_became_active":"2024-04-17T10:46:36.222490+0000","last_became_peered":"2024-04-17T10:46:36.222490+0000","last_unstale":"2024-04-17T10:46:38.305229+0000","last_undegraded":"2024-04-17T10:46:38.305229+0000","last_fullsized":"2024-04-17T10:46:38.305229+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:31:09.732265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.306290+0000","last_change":"2024-04-17T10:46:37.238769+0000","last_active":"2024-04-17T10:46:38.306290+0000","last_peered":"2024-04-17T10:46:38.306290+0000","last_clean":"2024-04-17T10:46:38.306290+0000","last_became_active":"2024-04-17T10:46:37.237948+0000","last_became_peered":"2024-04-17T10:46:37.237948+0000","last_unstale":"2024-04-17T10:46:38.306290+0000","last_undegraded":"2024-04-17T10:46:38.306290+0000","last_fullsized":"2024-04-17T10:46:38.306290+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:04:49.666714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.305246+0000","last_change":"2024-04-17T10:46:36.225863+0000","last_active":"2024-04-17T10:46:38.305246+0000","last_peered":"2024-04-17T10:46:38.305246+0000","last_clean":"2024-04-17T10:46:38.305246+0000","last_became_active":"2024-04-17T10:46:36.225479+0000","last_became_peered":"2024-04-17T10:46:36.225479+0000","last_unstale":"2024-04-17T10:46:38.305246+0000","last_undegraded":"2024-04-17T10:46:38.305246+0000","last_fullsized":"2024-04-17T10:46:38.305246+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:00:45.781185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.304668+0000","last_change":"2024-04-17T10:46:37.248665+0000","last_active":"2024-04-17T10:46:38.304668+0000","last_peered":"2024-04-17T10:46:38.304668+0000","last_clean":"2024-04-17T10:46:38.304668+0000","last_became_active":"2024-04-17T10:46:37.248406+0000","last_became_peered":"2024-04-17T10:46:37.248406+0000","last_unstale":"2024-04-17T10:46:38.304668+0000","last_undegraded":"2024-04-17T10:46:38.304668+0000","last_fullsized":"2024-04-17T10:46:38.304668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:39:34.285069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.304878+0000","last_change":"2024-04-17T10:46:36.244068+0000","last_active":"2024-04-17T10:46:38.304878+0000","last_peered":"2024-04-17T10:46:38.304878+0000","last_clean":"2024-04-17T10:46:38.304878+0000","last_became_active":"2024-04-17T10:46:36.243788+0000","last_became_peered":"2024-04-17T10:46:36.243788+0000","last_unstale":"2024-04-17T10:46:38.304878+0000","last_undegraded":"2024-04-17T10:46:38.304878+0000","last_fullsized":"2024-04-17T10:46:38.304878+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:45:09.270555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.305844+0000","last_change":"2024-04-17T10:46:37.240304+0000","last_active":"2024-04-17T10:46:38.305844+0000","last_peered":"2024-04-17T10:46:38.305844+0000","last_clean":"2024-04-17T10:46:38.305844+0000","last_became_active":"2024-04-17T10:46:37.239544+0000","last_became_peered":"2024-04-17T10:46:37.239544+0000","last_unstale":"2024-04-17T10:46:38.305844+0000","last_undegraded":"2024-04-17T10:46:38.305844+0000","last_fullsized":"2024-04-17T10:46:38.305844+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:31:39.193659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.304449+0000","last_change":"2024-04-17T10:46:36.224525+0000","last_active":"2024-04-17T10:46:38.304449+0000","last_peered":"2024-04-17T10:46:38.304449+0000","last_clean":"2024-04-17T10:46:38.304449+0000","last_became_active":"2024-04-17T10:46:36.224247+0000","last_became_peered":"2024-04-17T10:46:36.224247+0000","last_unstale":"2024-04-17T10:46:38.304449+0000","last_undegraded":"2024-04-17T10:46:38.304449+0000","last_fullsized":"2024-04-17T10:46:38.304449+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:45:20.591665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.220569+0000","last_change":"2024-04-17T10:46:37.267155+0000","last_active":"2024-04-17T10:46:38.220569+0000","last_peered":"2024-04-17T10:46:38.220569+0000","last_clean":"2024-04-17T10:46:38.220569+0000","last_became_active":"2024-04-17T10:46:37.266516+0000","last_became_peered":"2024-04-17T10:46:37.266516+0000","last_unstale":"2024-04-17T10:46:38.220569+0000","last_undegraded":"2024-04-17T10:46:38.220569+0000","last_fullsized":"2024-04-17T10:46:38.220569+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:13:08.182711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"88'2","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.306019+0000","last_change":"2024-04-17T10:46:36.239059+0000","last_active":"2024-04-17T10:46:38.306019+0000","last_peered":"2024-04-17T10:46:38.306019+0000","last_clean":"2024-04-17T10:46:38.306019+0000","last_became_active":"2024-04-17T10:46:36.238283+0000","last_became_peered":"2024-04-17T10:46:36.238283+0000","last_unstale":"2024-04-17T10:46:38.306019+0000","last_undegraded":"2024-04-17T10:46:38.306019+0000","last_fullsized":"2024-04-17T10:46:38.306019+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:05:20.023656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.220499+0000","last_change":"2024-04-17T10:46:37.270602+0000","last_active":"2024-04-17T10:46:38.220499+0000","last_peered":"2024-04-17T10:46:38.220499+0000","last_clean":"2024-04-17T10:46:38.220499+0000","last_became_active":"2024-04-17T10:46:37.269895+0000","last_became_peered":"2024-04-17T10:46:37.269895+0000","last_unstale":"2024-04-17T10:46:38.220499+0000","last_undegraded":"2024-04-17T10:46:38.220499+0000","last_fullsized":"2024-04-17T10:46:38.220499+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:26.205130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"89'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:39.218232+0000","last_change":"2024-04-17T10:46:36.224372+0000","last_active":"2024-04-17T10:46:39.218232+0000","last_peered":"2024-04-17T10:46:39.218232+0000","last_clean":"2024-04-17T10:46:39.218232+0000","last_became_active":"2024-04-17T10:46:36.223738+0000","last_became_peered":"2024-04-17T10:46:36.223738+0000","last_unstale":"2024-04-17T10:46:39.218232+0000","last_undegraded":"2024-04-17T10:46:39.218232+0000","last_fullsized":"2024-04-17T10:46:39.218232+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:56:57.932847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.307552+0000","last_change":"2024-04-17T10:46:37.262664+0000","last_active":"2024-04-17T10:46:38.307552+0000","last_peered":"2024-04-17T10:46:38.307552+0000","last_clean":"2024-04-17T10:46:38.307552+0000","last_became_active":"2024-04-17T10:46:37.260094+0000","last_became_peered":"2024-04-17T10:46:37.260094+0000","last_unstale":"2024-04-17T10:46:38.307552+0000","last_undegraded":"2024-04-17T10:46:38.307552+0000","last_fullsized":"2024-04-17T10:46:38.307552+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:31:39.193659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.306946+0000","last_change":"2024-04-17T10:46:36.245344+0000","last_active":"2024-04-17T10:46:38.306946+0000","last_peered":"2024-04-17T10:46:38.306946+0000","last_clean":"2024-04-17T10:46:38.306946+0000","last_became_active":"2024-04-17T10:46:36.244704+0000","last_became_peered":"2024-04-17T10:46:36.244704+0000","last_unstale":"2024-04-17T10:46:38.306946+0000","last_undegraded":"2024-04-17T10:46:38.306946+0000","last_fullsized":"2024-04-17T10:46:38.306946+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:37:40.239049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.221922+0000","last_change":"2024-04-17T10:46:37.257240+0000","last_active":"2024-04-17T10:46:38.221922+0000","last_peered":"2024-04-17T10:46:38.221922+0000","last_clean":"2024-04-17T10:46:38.221922+0000","last_became_active":"2024-04-17T10:46:37.255922+0000","last_became_peered":"2024-04-17T10:46:37.255922+0000","last_unstale":"2024-04-17T10:46:38.221922+0000","last_undegraded":"2024-04-17T10:46:38.221922+0000","last_fullsized":"2024-04-17T10:46:38.221922+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:32:14.502209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.307722+0000","last_change":"2024-04-17T10:46:36.241171+0000","last_active":"2024-04-17T10:46:38.307722+0000","last_peered":"2024-04-17T10:46:38.307722+0000","last_clean":"2024-04-17T10:46:38.307722+0000","last_became_active":"2024-04-17T10:46:36.240425+0000","last_became_peered":"2024-04-17T10:46:36.240425+0000","last_unstale":"2024-04-17T10:46:38.307722+0000","last_undegraded":"2024-04-17T10:46:38.307722+0000","last_fullsized":"2024-04-17T10:46:38.307722+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:46:46.704062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.306779+0000","last_change":"2024-04-17T10:46:37.248498+0000","last_active":"2024-04-17T10:46:38.306779+0000","last_peered":"2024-04-17T10:46:38.306779+0000","last_clean":"2024-04-17T10:46:38.306779+0000","last_became_active":"2024-04-17T10:46:37.247660+0000","last_became_peered":"2024-04-17T10:46:37.247660+0000","last_unstale":"2024-04-17T10:46:38.306779+0000","last_undegraded":"2024-04-17T10:46:38.306779+0000","last_fullsized":"2024-04-17T10:46:38.306779+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:41.169548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.217842+0000","last_change":"2024-04-17T10:46:36.238450+0000","last_active":"2024-04-17T10:46:38.217842+0000","last_peered":"2024-04-17T10:46:38.217842+0000","last_clean":"2024-04-17T10:46:38.217842+0000","last_became_active":"2024-04-17T10:46:36.238101+0000","last_became_peered":"2024-04-17T10:46:36.238101+0000","last_unstale":"2024-04-17T10:46:38.217842+0000","last_undegraded":"2024-04-17T10:46:38.217842+0000","last_fullsized":"2024-04-17T10:46:38.217842+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:54:35.160626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.217733+0000","last_change":"2024-04-17T10:46:37.240715+0000","last_active":"2024-04-17T10:46:38.217733+0000","last_peered":"2024-04-17T10:46:38.217733+0000","last_clean":"2024-04-17T10:46:38.217733+0000","last_became_active":"2024-04-17T10:46:37.240423+0000","last_became_peered":"2024-04-17T10:46:37.240423+0000","last_unstale":"2024-04-17T10:46:38.217733+0000","last_undegraded":"2024-04-17T10:46:38.217733+0000","last_fullsized":"2024-04-17T10:46:38.217733+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:15:12.428436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.220061+0000","last_change":"2024-04-17T10:46:36.225956+0000","last_active":"2024-04-17T10:46:38.220061+0000","last_peered":"2024-04-17T10:46:38.220061+0000","last_clean":"2024-04-17T10:46:38.220061+0000","last_became_active":"2024-04-17T10:46:36.225130+0000","last_became_peered":"2024-04-17T10:46:36.225130+0000","last_unstale":"2024-04-17T10:46:38.220061+0000","last_undegraded":"2024-04-17T10:46:38.220061+0000","last_fullsized":"2024-04-17T10:46:38.220061+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:34:44.797492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.219090+0000","last_change":"2024-04-17T10:46:37.253081+0000","last_active":"2024-04-17T10:46:38.219090+0000","last_peered":"2024-04-17T10:46:38.219090+0000","last_clean":"2024-04-17T10:46:38.219090+0000","last_became_active":"2024-04-17T10:46:37.248053+0000","last_became_peered":"2024-04-17T10:46:37.248053+0000","last_unstale":"2024-04-17T10:46:38.219090+0000","last_undegraded":"2024-04-17T10:46:38.219090+0000","last_fullsized":"2024-04-17T10:46:38.219090+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:31:10.745652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.307084+0000","last_change":"2024-04-17T10:46:36.240674+0000","last_active":"2024-04-17T10:46:38.307084+0000","last_peered":"2024-04-17T10:46:38.307084+0000","last_clean":"2024-04-17T10:46:38.307084+0000","last_became_active":"2024-04-17T10:46:36.239924+0000","last_became_peered":"2024-04-17T10:46:36.239924+0000","last_unstale":"2024-04-17T10:46:38.307084+0000","last_undegraded":"2024-04-17T10:46:38.307084+0000","last_fullsized":"2024-04-17T10:46:38.307084+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:00.018668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.306829+0000","last_change":"2024-04-17T10:46:37.265121+0000","last_active":"2024-04-17T10:46:38.306829+0000","last_peered":"2024-04-17T10:46:38.306829+0000","last_clean":"2024-04-17T10:46:38.306829+0000","last_became_active":"2024-04-17T10:46:37.264396+0000","last_became_peered":"2024-04-17T10:46:37.264396+0000","last_unstale":"2024-04-17T10:46:38.306829+0000","last_undegraded":"2024-04-17T10:46:38.306829+0000","last_fullsized":"2024-04-17T10:46:38.306829+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:54:36.174013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.219311+0000","last_change":"2024-04-17T10:46:36.245702+0000","last_active":"2024-04-17T10:46:38.219311+0000","last_peered":"2024-04-17T10:46:38.219311+0000","last_clean":"2024-04-17T10:46:38.219311+0000","last_became_active":"2024-04-17T10:46:36.243715+0000","last_became_peered":"2024-04-17T10:46:36.243715+0000","last_unstale":"2024-04-17T10:46:38.219311+0000","last_undegraded":"2024-04-17T10:46:38.219311+0000","last_fullsized":"2024-04-17T10:46:38.219311+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:32:24.323774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.305418+0000","last_change":"2024-04-17T10:46:37.242755+0000","last_active":"2024-04-17T10:46:38.305418+0000","last_peered":"2024-04-17T10:46:38.305418+0000","last_clean":"2024-04-17T10:46:38.305418+0000","last_became_active":"2024-04-17T10:46:37.242446+0000","last_became_peered":"2024-04-17T10:46:37.242446+0000","last_unstale":"2024-04-17T10:46:38.305418+0000","last_undegraded":"2024-04-17T10:46:38.305418+0000","last_fullsized":"2024-04-17T10:46:38.305418+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:36.196540+0000","last_clean_scrub_stamp":"2024-04-17T10:46:36.196540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:20:47.838328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"88'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.219792+0000","last_change":"2024-04-17T10:46:36.241299+0000","last_active":"2024-04-17T10:46:38.219792+0000","last_peered":"2024-04-17T10:46:38.219792+0000","last_clean":"2024-04-17T10:46:38.219792+0000","last_became_active":"2024-04-17T10:46:36.240627+0000","last_became_peered":"2024-04-17T10:46:36.240627+0000","last_unstale":"2024-04-17T10:46:38.219792+0000","last_undegraded":"2024-04-17T10:46:38.219792+0000","last_fullsized":"2024-04-17T10:46:38.219792+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:46:35.183153+0000","last_clean_scrub_stamp":"2024-04-17T10:46:35.183153+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:04:48.653328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":312,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T10:46:38.219213+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:46:38.219213+0000","last_peered":"2024-04-17T10:46:38.219213+0000","last_clean":"2024-04-17T10:46:38.219213+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:46:38.219213+0000","last_undegraded":"2024-04-17T10:46:38.219213+0000","last_fullsized":"2024-04-17T10:46:38.219213+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574956,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59200,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312640,"statfs":{"total":96636764160,"available":96576143360,"internally_reserved":0,"allocated":1691648,"data_stored":1063260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574955,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64196,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307644,"statfs":{"total":96636764160,"available":96571027456,"internally_reserved":0,"allocated":1105920,"data_stored":473516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574956,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50312,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321528,"statfs":{"total":96636764160,"available":96585244672,"internally_reserved":0,"allocated":1110016,"data_stored":474456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574955,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64192,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307648,"statfs":{"total":96636764160,"available":96571031552,"internally_reserved":0,"allocated":1093632,"data_stored":472822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574957,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64196,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307644,"statfs":{"total":96636764160,"available":96571027456,"internally_reserved":0,"allocated":1105920,"data_stored":474434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":64616903},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574955,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54468,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317372,"statfs":{"total":96636764160,"available":96580988928,"internally_reserved":0,"allocated":1097728,"data_stored":473404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574955,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72524,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94299316,"statfs":{"total":96636764160,"available":96562499584,"internally_reserved":0,"allocated":1105920,"data_stored":473016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574955,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59208,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312632,"statfs":{"total":96636764160,"available":96576135168,"internally_reserved":0,"allocated":1691648,"data_stored":1063224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:46:40.554 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-17T10:46:40.554 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-17T10:46:40.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.727+0000 7faaf8e48640 1 Processor -- start 2024-04-17T10:46:40.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.727+0000 7faaf8e48640 1 -- start start 2024-04-17T10:46:40.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.727+0000 7faaf8e48640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faaf4154de0 0x7faaf406ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:40.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.727+0000 7faaf8e48640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faaf406b350 0x7faaf40b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:40.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.727+0000 7faaf8e48640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faaf40b4040 0x7faaf4071580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:40.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.727+0000 7faaf8e48640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaf40b4870 con 0x7faaf406b350 2024-04-17T10:46:40.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.727+0000 7faaf8e48640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaf415bd70 con 0x7faaf4154de0 2024-04-17T10:46:40.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.727+0000 7faaf8e48640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaf415bf60 con 0x7faaf40b4040 2024-04-17T10:46:40.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.727+0000 7faaf1d9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faaf406b350 0x7faaf40b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:40.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.727+0000 7faaf1d9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faaf406b350 0x7faaf40b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47634/0 (socket says 172.21.15.83:47634) 2024-04-17T10:46:40.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.727+0000 7faaf1d9b640 1 -- 172.21.15.83:0/983946173 learned_addr learned my addr 172.21.15.83:0/983946173 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:40.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.727+0000 7faaf259c640 1 --2- 172.21.15.83:0/983946173 >> v2:172.21.15.167:3300/0 conn(0x7faaf4154de0 0x7faaf406ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:40.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.727+0000 7faaf2d9d640 1 --2- 172.21.15.83:0/983946173 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faaf40b4040 0x7faaf4071580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:40.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.727+0000 7faaf1d9b640 1 -- 172.21.15.83:0/983946173 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faaf40b4040 msgr2=0x7faaf4071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:40.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.727+0000 7faaf1d9b640 1 --2- 172.21.15.83:0/983946173 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faaf40b4040 0x7faaf4071580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:40.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.727+0000 7faaf1d9b640 1 -- 172.21.15.83:0/983946173 >> v2:172.21.15.167:3300/0 conn(0x7faaf4154de0 msgr2=0x7faaf406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:40.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.727+0000 7faaf1d9b640 1 --2- 172.21.15.83:0/983946173 >> v2:172.21.15.167:3300/0 conn(0x7faaf4154de0 0x7faaf406ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:40.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.727+0000 7faaf1d9b640 1 -- 172.21.15.83:0/983946173 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf415afb0 con 0x7faaf406b350 2024-04-17T10:46:40.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.731+0000 7faaf1d9b640 1 --2- 172.21.15.83:0/983946173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf406b350 0x7faaf40b3b00 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7faaf40b2de0 tx=0x7faae801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afafbc073606b06c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:40.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.731+0000 7faaf159a640 1 -- 172.21.15.83:0/983946173 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae80b0070 con 0x7faaf406b350 2024-04-17T10:46:40.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.731+0000 7faaf159a640 1 -- 172.21.15.83:0/983946173 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faae80ab600 con 0x7faaf406b350 2024-04-17T10:46:40.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.731+0000 7faaf159a640 1 -- 172.21.15.83:0/983946173 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae80ab9d0 con 0x7faaf406b350 2024-04-17T10:46:40.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.731+0000 7faaf8e48640 1 -- 172.21.15.83:0/983946173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf406b350 msgr2=0x7faaf40b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:40.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.731+0000 7faaf8e48640 1 --2- 172.21.15.83:0/983946173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf406b350 0x7faaf40b3b00 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7faaf40b2de0 tx=0x7faae801c260 comp rx=0 tx=0).stop 2024-04-17T10:46:40.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.731+0000 7faaf8e48640 1 -- 172.21.15.83:0/983946173 shutdown_connections 2024-04-17T10:46:40.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.731+0000 7faaf8e48640 1 --2- 172.21.15.83:0/983946173 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faaf40b4040 0x7faaf4071580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:40.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.731+0000 7faaf8e48640 1 --2- 172.21.15.83:0/983946173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf406b350 0x7faaf40b3b00 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:40.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.731+0000 7faaf8e48640 1 --2- 172.21.15.83:0/983946173 >> v2:172.21.15.167:3300/0 conn(0x7faaf4154de0 0x7faaf406ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:40.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.731+0000 7faaf8e48640 1 -- 172.21.15.83:0/983946173 >> 172.21.15.83:0/983946173 conn(0x7faaf40b15b0 msgr2=0x7faaf40b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:40.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.731+0000 7faaf8e48640 1 -- 172.21.15.83:0/983946173 shutdown_connections 2024-04-17T10:46:40.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.731+0000 7faaf8e48640 1 -- 172.21.15.83:0/983946173 wait complete. 2024-04-17T10:46:40.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.735+0000 7faaf8e48640 1 Processor -- start 2024-04-17T10:46:40.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.735+0000 7faaf8e48640 1 -- start start 2024-04-17T10:46:40.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.735+0000 7faaf8e48640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faaf406b350 0x7faaf415c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:40.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.735+0000 7faaf8e48640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf40b4040 0x7faaf4157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:40.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.735+0000 7faaf8e48640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faaf4154de0 0x7faaf4070c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:40.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.735+0000 7faaf8e48640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf40b4450 con 0x7faaf40b4040 2024-04-17T10:46:40.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.735+0000 7faaf8e48640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf41551c0 con 0x7faaf406b350 2024-04-17T10:46:40.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.735+0000 7faaf8e48640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaf40b45d0 con 0x7faaf4154de0 2024-04-17T10:46:40.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.735+0000 7faaf1d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf40b4040 0x7faaf4157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:40.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.735+0000 7faaf1d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf40b4040 0x7faaf4157ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47650/0 (socket says 172.21.15.83:47650) 2024-04-17T10:46:40.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.735+0000 7faaf1d9b640 1 -- 172.21.15.83:0/679956862 learned_addr learned my addr 172.21.15.83:0/679956862 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:40.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.735+0000 7faaf1d9b640 1 -- 172.21.15.83:0/679956862 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faaf4154de0 msgr2=0x7faaf4070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:40.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.735+0000 7faaf1d9b640 1 --2- 172.21.15.83:0/679956862 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faaf4154de0 0x7faaf4070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:40.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.735+0000 7faaf1d9b640 1 -- 172.21.15.83:0/679956862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faaf406b350 msgr2=0x7faaf415c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:40.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.735+0000 7faaf1d9b640 1 --2- 172.21.15.83:0/679956862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faaf406b350 0x7faaf415c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:40.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.735+0000 7faaf1d9b640 1 -- 172.21.15.83:0/679956862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf415ccf0 con 0x7faaf40b4040 2024-04-17T10:46:40.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.735+0000 7faaf1d9b640 1 --2- 172.21.15.83:0/679956862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf40b4040 0x7faaf4157ab0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7faae801a910 tx=0x7faae8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:40.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.735+0000 7faae2ffd640 1 -- 172.21.15.83:0/679956862 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae80ba070 con 0x7faaf40b4040 2024-04-17T10:46:40.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.735+0000 7faae2ffd640 1 -- 172.21.15.83:0/679956862 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faae80b4620 con 0x7faaf40b4040 2024-04-17T10:46:40.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.735+0000 7faae2ffd640 1 -- 172.21.15.83:0/679956862 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae80b0040 con 0x7faaf40b4040 2024-04-17T10:46:40.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.735+0000 7faaf8e48640 1 -- 172.21.15.83:0/679956862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaf415cfd0 con 0x7faaf40b4040 2024-04-17T10:46:40.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.735+0000 7faaf8e48640 1 -- 172.21.15.83:0/679956862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaf415d590 con 0x7faaf40b4040 2024-04-17T10:46:40.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.735+0000 7faaf8e48640 1 -- 172.21.15.83:0/679956862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaf4074f90 con 0x7faaf40b4040 2024-04-17T10:46:40.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.735+0000 7faae2ffd640 1 -- 172.21.15.83:0/679956862 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faae80ab4c0 con 0x7faaf40b4040 2024-04-17T10:46:40.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.739+0000 7faae2ffd640 1 --2- 172.21.15.83:0/679956862 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faad4066f60 0x7faad4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:40.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.739+0000 7faae2ffd640 1 -- 172.21.15.83:0/679956862 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faae811e2d0 con 0x7faaf40b4040 2024-04-17T10:46:40.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.739+0000 7faaf259c640 1 --2- 172.21.15.83:0/679956862 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faad4066f60 0x7faad4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:40.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.739+0000 7faaf259c640 1 --2- 172.21.15.83:0/679956862 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faad4066f60 0x7faad4069420 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7faae4009870 tx=0x7faae4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:40.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.743+0000 7faae2ffd640 1 -- 172.21.15.83:0/679956862 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faae8016110 con 0x7faaf40b4040 2024-04-17T10:46:40.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.951+0000 7faaf8e48640 1 -- 172.21.15.83:0/679956862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7faaf415d7d0 con 0x7faaf40b4040 2024-04-17T10:46:40.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.955+0000 7faae2ffd640 1 -- 172.21.15.83:0/679956862 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7faae80ef2e0 con 0x7faaf40b4040 2024-04-17T10:46:40.954 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-04-17T10:46:40.954 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBUph9mHGmXGBAAZF+JiAG9YrFtbW+dCy2IEQ== 2024-04-17T10:46:40.954 INFO:teuthology.orchestra.run.smithi083.stdout: caps mds = "allow" 2024-04-17T10:46:40.954 INFO:teuthology.orchestra.run.smithi083.stdout: caps mon = "allow r" 2024-04-17T10:46:40.954 INFO:teuthology.orchestra.run.smithi083.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-17T10:46:40.956 INFO:teuthology.orchestra.run.smithi083.stderr:updated caps for client.0 2024-04-17T10:46:40.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.959+0000 7faaf8e48640 1 -- 172.21.15.83:0/679956862 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faad4066f60 msgr2=0x7faad4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:40.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.959+0000 7faaf8e48640 1 --2- 172.21.15.83:0/679956862 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faad4066f60 0x7faad4069420 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7faae4009870 tx=0x7faae4008040 comp rx=0 tx=0).stop 2024-04-17T10:46:40.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.959+0000 7faaf8e48640 1 -- 172.21.15.83:0/679956862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf40b4040 msgr2=0x7faaf4157ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:40.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.959+0000 7faaf8e48640 1 --2- 172.21.15.83:0/679956862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf40b4040 0x7faaf4157ab0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7faae801a910 tx=0x7faae8019000 comp rx=0 tx=0).stop 2024-04-17T10:46:40.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.959+0000 7faaf8e48640 1 -- 172.21.15.83:0/679956862 shutdown_connections 2024-04-17T10:46:40.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.959+0000 7faaf8e48640 1 --2- 172.21.15.83:0/679956862 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faad4066f60 0x7faad4069420 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:40.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.959+0000 7faaf8e48640 1 --2- 172.21.15.83:0/679956862 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faaf4154de0 0x7faaf4070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:40.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.959+0000 7faaf8e48640 1 --2- 172.21.15.83:0/679956862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf40b4040 0x7faaf4157ab0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:40.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.959+0000 7faaf8e48640 1 --2- 172.21.15.83:0/679956862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faaf406b350 0x7faaf415c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:40.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.959+0000 7faaf8e48640 1 -- 172.21.15.83:0/679956862 >> 172.21.15.83:0/679956862 conn(0x7faaf40b15b0 msgr2=0x7faaf4070470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:40.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.959+0000 7faaf8e48640 1 -- 172.21.15.83:0/679956862 shutdown_connections 2024-04-17T10:46:40.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:40.959+0000 7faaf8e48640 1 -- 172.21.15.83:0/679956862 wait complete. 2024-04-17T10:46:40.972 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:46:41.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.083+0000 7ff38a5b9640 1 Processor -- start 2024-04-17T10:46:41.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.083+0000 7ff38a5b9640 1 -- start start 2024-04-17T10:46:41.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.083+0000 7ff38a5b9640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff3840cf060 0x7ff3840b4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.083+0000 7ff38a5b9640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff384070b80 0x7ff38406c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.083+0000 7ff38a5b9640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3840b2bd0 con 0x7ff3840b3990 2024-04-17T10:46:41.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.083+0000 7ff38a5b9640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3840b2980 con 0x7ff3840cf060 2024-04-17T10:46:41.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.083+0000 7ff38a5b9640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3840cf440 con 0x7ff384070b80 2024-04-17T10:46:41.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.087+0000 7ff383fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff3840cf060 0x7ff3840b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:41.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.087+0000 7ff383fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff3840cf060 0x7ff3840b4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47660/0 (socket says 172.21.15.83:47660) 2024-04-17T10:46:41.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.087+0000 7ff383fff640 1 -- 172.21.15.83:0/1519515851 learned_addr learned my addr 172.21.15.83:0/1519515851 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:41.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.087+0000 7ff383fff640 1 -- 172.21.15.83:0/1519515851 >> v2:172.21.15.167:3300/0 conn(0x7ff384070b80 msgr2=0x7ff38406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:41.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.087+0000 7ff388b56640 1 --2- 172.21.15.83:0/1519515851 >> v2:172.21.15.167:3300/0 conn(0x7ff384070b80 0x7ff38406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:41.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.087+0000 7ff383fff640 1 --2- 172.21.15.83:0/1519515851 >> v2:172.21.15.167:3300/0 conn(0x7ff384070b80 0x7ff38406c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.087+0000 7ff383fff640 1 -- 172.21.15.83:0/1519515851 >> v1:172.21.15.167:6789/0 conn(0x7ff3840b3990 legacy=0x7ff384070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:41.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.087+0000 7ff383fff640 1 -- 172.21.15.83:0/1519515851 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3840b2eb0 con 0x7ff3840cf060 2024-04-17T10:46:41.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.087+0000 7ff383fff640 1 --2- 172.21.15.83:0/1519515851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3840cf060 0x7ff3840b4060 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7ff3840aebd0 tx=0x7ff37401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d1b1f464af42f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:41.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.087+0000 7ff3837fe640 1 -- 172.21.15.83:0/1519515851 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3740aa610 con 0x7ff3840cf060 2024-04-17T10:46:41.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.087+0000 7ff3837fe640 1 -- 172.21.15.83:0/1519515851 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3740af070 con 0x7ff3840cf060 2024-04-17T10:46:41.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.087+0000 7ff3837fe640 1 -- 172.21.15.83:0/1519515851 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3740aa9d0 con 0x7ff3840cf060 2024-04-17T10:46:41.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.087+0000 7ff38a5b9640 1 -- 172.21.15.83:0/1519515851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3840cf060 msgr2=0x7ff3840b4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:41.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.087+0000 7ff38a5b9640 1 --2- 172.21.15.83:0/1519515851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3840cf060 0x7ff3840b4060 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7ff3840aebd0 tx=0x7ff37401c4b0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.087+0000 7ff38a5b9640 1 -- 172.21.15.83:0/1519515851 shutdown_connections 2024-04-17T10:46:41.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.087+0000 7ff38a5b9640 1 --2- 172.21.15.83:0/1519515851 >> v2:172.21.15.167:3300/0 conn(0x7ff384070b80 0x7ff38406c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.087+0000 7ff38a5b9640 1 --2- 172.21.15.83:0/1519515851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3840cf060 0x7ff3840b4060 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.087+0000 7ff38a5b9640 1 -- 172.21.15.83:0/1519515851 >> 172.21.15.83:0/1519515851 conn(0x7ff38406abe0 msgr2=0x7ff38406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:41.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.087+0000 7ff38a5b9640 1 -- 172.21.15.83:0/1519515851 shutdown_connections 2024-04-17T10:46:41.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.087+0000 7ff38a5b9640 1 -- 172.21.15.83:0/1519515851 wait complete. 2024-04-17T10:46:41.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.091+0000 7ff38a5b9640 1 Processor -- start 2024-04-17T10:46:41.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.091+0000 7ff38a5b9640 1 -- start start 2024-04-17T10:46:41.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.091+0000 7ff38a5b9640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff384070b80 0x7ff38414c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.091+0000 7ff38a5b9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3840b3990 0x7ff3840ae490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.091+0000 7ff38a5b9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3840cf060 0x7ff38414bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.091+0000 7ff38a5b9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff384166f10 con 0x7ff3840b3990 2024-04-17T10:46:41.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.091+0000 7ff38a5b9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff384166d90 con 0x7ff3840cf060 2024-04-17T10:46:41.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.091+0000 7ff38a5b9640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff384167090 con 0x7ff384070b80 2024-04-17T10:46:41.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.091+0000 7ff37bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3840b3990 0x7ff3840ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:41.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.091+0000 7ff37bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3840b3990 0x7ff3840ae490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47668/0 (socket says 172.21.15.83:47668) 2024-04-17T10:46:41.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.091+0000 7ff37bfff640 1 -- 172.21.15.83:0/4002171986 learned_addr learned my addr 172.21.15.83:0/4002171986 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:41.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.091+0000 7ff388b56640 1 --2- 172.21.15.83:0/4002171986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3840cf060 0x7ff38414bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:41.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.091+0000 7ff383fff640 1 --2- 172.21.15.83:0/4002171986 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff384070b80 0x7ff38414c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:41.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.091+0000 7ff37bfff640 1 -- 172.21.15.83:0/4002171986 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff384070b80 msgr2=0x7ff38414c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:41.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.091+0000 7ff37bfff640 1 --2- 172.21.15.83:0/4002171986 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff384070b80 0x7ff38414c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.091+0000 7ff37bfff640 1 -- 172.21.15.83:0/4002171986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3840cf060 msgr2=0x7ff38414bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:41.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.091+0000 7ff37bfff640 1 --2- 172.21.15.83:0/4002171986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3840cf060 0x7ff38414bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.091+0000 7ff37bfff640 1 -- 172.21.15.83:0/4002171986 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3840ae9d0 con 0x7ff3840b3990 2024-04-17T10:46:41.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.091+0000 7ff388b56640 1 --2- 172.21.15.83:0/4002171986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3840cf060 0x7ff38414bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:41.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.091+0000 7ff37bfff640 1 --2- 172.21.15.83:0/4002171986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3840b3990 0x7ff3840ae490 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00c490 tx=0x7ff36c001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:41.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.091+0000 7ff3817fa640 1 -- 172.21.15.83:0/4002171986 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c017070 con 0x7ff3840b3990 2024-04-17T10:46:41.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.091+0000 7ff3817fa640 1 -- 172.21.15.83:0/4002171986 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff36c003600 con 0x7ff3840b3990 2024-04-17T10:46:41.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.091+0000 7ff38a5b9640 1 -- 172.21.15.83:0/4002171986 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff38414c120 con 0x7ff3840b3990 2024-04-17T10:46:41.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.091+0000 7ff3817fa640 1 -- 172.21.15.83:0/4002171986 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c003a30 con 0x7ff3840b3990 2024-04-17T10:46:41.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.091+0000 7ff38a5b9640 1 -- 172.21.15.83:0/4002171986 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3840afcd0 con 0x7ff3840b3990 2024-04-17T10:46:41.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.091+0000 7ff38a5b9640 1 -- 172.21.15.83:0/4002171986 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff384074f90 con 0x7ff3840b3990 2024-04-17T10:46:41.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.095+0000 7ff3817fa640 1 -- 172.21.15.83:0/4002171986 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff36c01a070 con 0x7ff3840b3990 2024-04-17T10:46:41.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.095+0000 7ff3817fa640 1 --2- 172.21.15.83:0/4002171986 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff364066f60 0x7ff364069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.095+0000 7ff383fff640 1 --2- 172.21.15.83:0/4002171986 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff364066f60 0x7ff364069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:41.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.095+0000 7ff3817fa640 1 -- 172.21.15.83:0/4002171986 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff36c084500 con 0x7ff3840b3990 2024-04-17T10:46:41.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.095+0000 7ff383fff640 1 --2- 172.21.15.83:0/4002171986 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff364066f60 0x7ff364069420 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff3740ae000 tx=0x7ff374008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:41.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.099+0000 7ff3817fa640 1 -- 172.21.15.83:0/4002171986 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff3840ae9d0 con 0x7ff3840b3990 2024-04-17T10:46:41.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.279+0000 7ff38a5b9640 1 -- 172.21.15.83:0/4002171986 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff384070930 con 0x7ff3840b3990 2024-04-17T10:46:41.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.279+0000 7ff3817fa640 1 -- 172.21.15.83:0/4002171986 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4946 (secure 0 0 0) 0x7ff36c055480 con 0x7ff3840b3990 2024-04-17T10:46:41.275 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:46:41.275 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5429,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6834/3936640219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3936640219},{"type":"v1","addr":"172.21.15.167:6834","nonce":3936640219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":5998,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4152882285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6833","nonce":4152882285},{"type":"v1","addr":"172.21.15.167:6835","nonce":4152882285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1562645860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1562645860},{"type":"v1","addr":"172.21.15.83:6837","nonce":1562645860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:46:36.578918+0000","modified":"2024-04-17T10:46:38.561701+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7575":{"gid":7575,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/1056175765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1056175765},{"type":"v1","addr":"172.21.15.83:6836","nonce":1056175765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-17T10:46:41.275 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 56 2024-04-17T10:46:41.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.279+0000 7ff38a5b9640 1 -- 172.21.15.83:0/4002171986 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff364066f60 msgr2=0x7ff364069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:41.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.279+0000 7ff38a5b9640 1 --2- 172.21.15.83:0/4002171986 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff364066f60 0x7ff364069420 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff3740ae000 tx=0x7ff374008040 comp rx=0 tx=0).stop 2024-04-17T10:46:41.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.279+0000 7ff38a5b9640 1 -- 172.21.15.83:0/4002171986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3840b3990 msgr2=0x7ff3840ae490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:41.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.279+0000 7ff38a5b9640 1 --2- 172.21.15.83:0/4002171986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3840b3990 0x7ff3840ae490 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00c490 tx=0x7ff36c001e80 comp rx=0 tx=0).stop 2024-04-17T10:46:41.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.279+0000 7ff38a5b9640 1 -- 172.21.15.83:0/4002171986 shutdown_connections 2024-04-17T10:46:41.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.279+0000 7ff38a5b9640 1 --2- 172.21.15.83:0/4002171986 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff364066f60 0x7ff364069420 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.279+0000 7ff38a5b9640 1 --2- 172.21.15.83:0/4002171986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3840cf060 0x7ff38414bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.279+0000 7ff38a5b9640 1 --2- 172.21.15.83:0/4002171986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3840b3990 0x7ff3840ae490 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.279+0000 7ff38a5b9640 1 --2- 172.21.15.83:0/4002171986 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff384070b80 0x7ff38414c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.279+0000 7ff38a5b9640 1 -- 172.21.15.83:0/4002171986 >> 172.21.15.83:0/4002171986 conn(0x7ff38406abe0 msgr2=0x7ff3840cf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:41.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.279+0000 7ff38a5b9640 1 -- 172.21.15.83:0/4002171986 shutdown_connections 2024-04-17T10:46:41.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.283+0000 7ff38a5b9640 1 -- 172.21.15.83:0/4002171986 wait complete. 2024-04-17T10:46:41.696 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T10:46:36.578918+0000', 'modified': '2024-04-17T10:46:38.561701+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7575}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7575': {'gid': 7575, 'name': 'c', 'rank': 0, 'incarnation': 54, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6836/1056175765', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6834', 'nonce': 1056175765}, {'type': 'v1', 'addr': '172.21.15.83:6836', 'nonce': 1056175765}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:46:41.696 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T10:46:41.697 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-17T10:46:41.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.831+0000 7f4272841640 1 Processor -- start 2024-04-17T10:46:41.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.831+0000 7f4272841640 1 -- start start 2024-04-17T10:46:41.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.831+0000 7f4272841640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f426c15bac0 0x7f426c071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.831+0000 7f4272841640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f426c070470 0x7f426c06ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.831+0000 7f4272841640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f426c06c1f0 con 0x7f426c15b6e0 2024-04-17T10:46:41.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.831+0000 7f4272841640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f426c070a10 con 0x7f426c070470 2024-04-17T10:46:41.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.831+0000 7f4272841640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f426c070cf0 con 0x7f426c15bac0 2024-04-17T10:46:41.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.831+0000 7f4270dde640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f426c070470 0x7f426c06ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:41.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.831+0000 7f4270dde640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f426c070470 0x7f426c06ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47670/0 (socket says 172.21.15.83:47670) 2024-04-17T10:46:41.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.831+0000 7f4270dde640 1 -- 172.21.15.83:0/1593120695 learned_addr learned my addr 172.21.15.83:0/1593120695 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:41.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.831+0000 7f42637fe640 1 --2- 172.21.15.83:0/1593120695 >> v2:172.21.15.167:3300/0 conn(0x7f426c15bac0 0x7f426c071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:41.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.831+0000 7f4270dde640 1 -- 172.21.15.83:0/1593120695 >> v2:172.21.15.167:3300/0 conn(0x7f426c15bac0 msgr2=0x7f426c071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:41.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.831+0000 7f4270dde640 1 --2- 172.21.15.83:0/1593120695 >> v2:172.21.15.167:3300/0 conn(0x7f426c15bac0 0x7f426c071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.831+0000 7f4270dde640 1 -- 172.21.15.83:0/1593120695 >> v1:172.21.15.167:6789/0 conn(0x7f426c15b6e0 legacy=0x7f426c0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:41.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.831+0000 7f4270dde640 1 -- 172.21.15.83:0/1593120695 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f426c070fd0 con 0x7f426c070470 2024-04-17T10:46:41.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.835+0000 7f4270dde640 1 --2- 172.21.15.83:0/1593120695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f426c070470 0x7f426c06ae10 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f426c0cd5e0 tx=0x7f426401c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de948aae59ed25a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:41.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.835+0000 7f426b7fe640 1 -- 172.21.15.83:0/1593120695 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42640a5650 con 0x7f426c070470 2024-04-17T10:46:41.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.835+0000 7f426b7fe640 1 -- 172.21.15.83:0/1593120695 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42640a57f0 con 0x7f426c070470 2024-04-17T10:46:41.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.835+0000 7f426b7fe640 1 -- 172.21.15.83:0/1593120695 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42640a5bc0 con 0x7f426c070470 2024-04-17T10:46:41.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.835+0000 7f4272841640 1 -- 172.21.15.83:0/1593120695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f426c070470 msgr2=0x7f426c06ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:41.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.835+0000 7f4272841640 1 --2- 172.21.15.83:0/1593120695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f426c070470 0x7f426c06ae10 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f426c0cd5e0 tx=0x7f426401c990 comp rx=0 tx=0).stop 2024-04-17T10:46:41.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.835+0000 7f4272841640 1 -- 172.21.15.83:0/1593120695 shutdown_connections 2024-04-17T10:46:41.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.835+0000 7f4272841640 1 --2- 172.21.15.83:0/1593120695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f426c070470 0x7f426c06ae10 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.835+0000 7f4272841640 1 --2- 172.21.15.83:0/1593120695 >> v2:172.21.15.167:3300/0 conn(0x7f426c15bac0 0x7f426c071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.835+0000 7f4272841640 1 -- 172.21.15.83:0/1593120695 >> 172.21.15.83:0/1593120695 conn(0x7f426c0b15b0 msgr2=0x7f426c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:41.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.835+0000 7f4272841640 1 -- 172.21.15.83:0/1593120695 shutdown_connections 2024-04-17T10:46:41.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.835+0000 7f4272841640 1 -- 172.21.15.83:0/1593120695 wait complete. 2024-04-17T10:46:41.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.839+0000 7f4272841640 1 Processor -- start 2024-04-17T10:46:41.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.839+0000 7f4272841640 1 -- start start 2024-04-17T10:46:41.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.839+0000 7f4272841640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f426c070470 0x7f426c13dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.839+0000 7f4272841640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f426c15b6e0 0x7f426c0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.839+0000 7f4272841640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f426c15bac0 0x7f426c0add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.839+0000 7f4272841640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426c06d2b0 con 0x7f426c15b6e0 2024-04-17T10:46:41.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.839+0000 7f4272841640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426c06d130 con 0x7f426c070470 2024-04-17T10:46:41.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.839+0000 7f4272841640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f426c06b760 con 0x7f426c15bac0 2024-04-17T10:46:41.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.839+0000 7f42637fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f426c15b6e0 0x7f426c0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:41.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.839+0000 7f42637fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f426c15b6e0 0x7f426c0ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47682/0 (socket says 172.21.15.83:47682) 2024-04-17T10:46:41.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.839+0000 7f42637fe640 1 -- 172.21.15.83:0/1614506147 learned_addr learned my addr 172.21.15.83:0/1614506147 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:41.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.839+0000 7f426bfff640 1 --2- 172.21.15.83:0/1614506147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f426c070470 0x7f426c13dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:41.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.839+0000 7f4270dde640 1 --2- 172.21.15.83:0/1614506147 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f426c15bac0 0x7f426c0add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:41.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.839+0000 7f42637fe640 1 -- 172.21.15.83:0/1614506147 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f426c15bac0 msgr2=0x7f426c0add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:41.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.839+0000 7f42637fe640 1 --2- 172.21.15.83:0/1614506147 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f426c15bac0 0x7f426c0add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.839+0000 7f42637fe640 1 -- 172.21.15.83:0/1614506147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f426c070470 msgr2=0x7f426c13dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:41.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.839+0000 7f42637fe640 1 --2- 172.21.15.83:0/1614506147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f426c070470 0x7f426c13dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.839+0000 7f42637fe640 1 -- 172.21.15.83:0/1614506147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f426c1545d0 con 0x7f426c15b6e0 2024-04-17T10:46:41.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.839+0000 7f42637fe640 1 --2- 172.21.15.83:0/1614506147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f426c15b6e0 0x7f426c0ad840 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f425400d5f0 tx=0x7f425400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:41.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.839+0000 7f42697fa640 1 -- 172.21.15.83:0/1614506147 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42540125f0 con 0x7f426c15b6e0 2024-04-17T10:46:41.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.839+0000 7f42697fa640 1 -- 172.21.15.83:0/1614506147 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4254019070 con 0x7f426c15b6e0 2024-04-17T10:46:41.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.839+0000 7f4272841640 1 -- 172.21.15.83:0/1614506147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f426c1548b0 con 0x7f426c15b6e0 2024-04-17T10:46:41.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.839+0000 7f42697fa640 1 -- 172.21.15.83:0/1614506147 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42540129b0 con 0x7f426c15b6e0 2024-04-17T10:46:41.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.839+0000 7f4272841640 1 -- 172.21.15.83:0/1614506147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f426c13e4e0 con 0x7f426c15b6e0 2024-04-17T10:46:41.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f42697fa640 1 -- 172.21.15.83:0/1614506147 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4254012b50 con 0x7f426c15b6e0 2024-04-17T10:46:41.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f42697fa640 1 --2- 172.21.15.83:0/1614506147 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f424c066f60 0x7f424c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f42697fa640 1 -- 172.21.15.83:0/1614506147 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4254085310 con 0x7f426c15b6e0 2024-04-17T10:46:41.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f426bfff640 1 --2- 172.21.15.83:0/1614506147 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f424c066f60 0x7f424c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:41.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f426bfff640 1 --2- 172.21.15.83:0/1614506147 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f424c066f60 0x7f424c069420 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f426c0b49c0 tx=0x7f425c007e40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:41.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f42759ae000 1 Processor -- start 2024-04-17T10:46:41.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f42759ae000 1 -- start start 2024-04-17T10:46:41.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f42759ae000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563683658920 0x563683658d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f42759ae000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563683659240 0x56368360e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f42759ae000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56368360ef30 0x56368360f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f42759ae000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636834f65f0 con 0x563683658920 2024-04-17T10:46:41.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f42759ae000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368345dda0 con 0x563683659240 2024-04-17T10:46:41.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f42759ae000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636833e6150 con 0x56368360ef30 2024-04-17T10:46:41.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f426bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563683658920 0x563683658d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:41.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f426bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563683658920 0x563683658d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47698/0 (socket says 172.21.15.83:47698) 2024-04-17T10:46:41.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f426bfff640 1 -- 172.21.15.83:0/2900390073 learned_addr learned my addr 172.21.15.83:0/2900390073 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:41.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f42637fe640 1 --2- 172.21.15.83:0/2900390073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563683659240 0x56368360e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:41.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f4270dde640 1 --2- 172.21.15.83:0/2900390073 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56368360ef30 0x56368360f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:41.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f426bfff640 1 -- 172.21.15.83:0/2900390073 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56368360ef30 msgr2=0x56368360f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:41.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f426bfff640 1 --2- 172.21.15.83:0/2900390073 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56368360ef30 0x56368360f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f426bfff640 1 -- 172.21.15.83:0/2900390073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563683659240 msgr2=0x56368360e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:41.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f426bfff640 1 --2- 172.21.15.83:0/2900390073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563683659240 0x56368360e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f426bfff640 1 -- 172.21.15.83:0/2900390073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636833ffca0 con 0x563683658920 2024-04-17T10:46:41.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f42637fe640 1 --2- 172.21.15.83:0/2900390073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563683659240 0x56368360e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:41.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f426bfff640 1 --2- 172.21.15.83:0/2900390073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563683658920 0x563683658d00 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f425c003d80 tx=0x7f425c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa65c5e955bce46 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:41.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f4261ffb640 1 -- 172.21.15.83:0/2900390073 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c01b050 con 0x563683658920 2024-04-17T10:46:41.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f4261ffb640 1 -- 172.21.15.83:0/2900390073 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425c025070 con 0x563683658920 2024-04-17T10:46:41.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f4261ffb640 1 -- 172.21.15.83:0/2900390073 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c020440 con 0x563683658920 2024-04-17T10:46:41.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f42759ae000 1 -- 172.21.15.83:0/2900390073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563683658920 msgr2=0x563683658d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:41.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f42759ae000 1 --2- 172.21.15.83:0/2900390073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563683658920 0x563683658d00 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f425c003d80 tx=0x7f425c012000 comp rx=0 tx=0).stop 2024-04-17T10:46:41.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f42759ae000 1 -- 172.21.15.83:0/2900390073 shutdown_connections 2024-04-17T10:46:41.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f42759ae000 1 --2- 172.21.15.83:0/2900390073 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56368360ef30 0x56368360f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f42759ae000 1 --2- 172.21.15.83:0/2900390073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563683659240 0x56368360e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f42759ae000 1 --2- 172.21.15.83:0/2900390073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563683658920 0x563683658d00 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f42759ae000 1 -- 172.21.15.83:0/2900390073 >> 172.21.15.83:0/2900390073 conn(0x5636836152a0 msgr2=0x56368360d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:41.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f42759ae000 1 -- 172.21.15.83:0/2900390073 shutdown_connections 2024-04-17T10:46:41.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.843+0000 7f42759ae000 1 -- 172.21.15.83:0/2900390073 wait complete. 2024-04-17T10:46:41.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42759ae000 1 -- start start 2024-04-17T10:46:41.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42759ae000 1 Processor -- start 2024-04-17T10:46:41.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42759ae000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56368360ef30 0x56368365fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42759ae000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563683658920 0x5636836601c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42759ae000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563683659240 0x563683612aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42759ae000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x563683418990 con 0x563683658920 2024-04-17T10:46:41.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42759ae000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563683459e40 con 0x563683659240 2024-04-17T10:46:41.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42759ae000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636833e9900 con 0x56368360ef30 2024-04-17T10:46:41.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42759ae000 10 client.0 _ll_register_callbacks cb 0x5636835111c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:46:41.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42637fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563683658920 0x5636836601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:41.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42637fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563683658920 0x5636836601c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47708/0 (socket says 172.21.15.83:47708) 2024-04-17T10:46:41.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42637fe640 1 -- 172.21.15.83:0/668304834 learned_addr learned my addr 172.21.15.83:0/668304834 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:41.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42637fe640 1 -- 172.21.15.83:0/668304834 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56368360ef30 msgr2=0x56368365fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:41.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42637fe640 1 --2- 172.21.15.83:0/668304834 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56368360ef30 0x56368365fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42637fe640 1 -- 172.21.15.83:0/668304834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563683659240 msgr2=0x563683612aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:41.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42637fe640 1 --2- 172.21.15.83:0/668304834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563683659240 0x563683612aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42637fe640 1 -- 172.21.15.83:0/668304834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563683416630 con 0x563683658920 2024-04-17T10:46:41.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42637fe640 1 --2- 172.21.15.83:0/668304834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563683658920 0x5636836601c0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f425405de90 tx=0x7f425405c0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6269bedc8f3dae6e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:41.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f424affd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:46:41.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f424affd640 1 -- 172.21.15.83:0/668304834 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4254059e00 con 0x563683658920 2024-04-17T10:46:41.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f424affd640 1 -- 172.21.15.83:0/668304834 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4254085070 con 0x563683658920 2024-04-17T10:46:41.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42759ae000 1 -- 172.21.15.83:0/668304834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f425c025070 con 0x563683658920 2024-04-17T10:46:41.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f424affd640 1 -- 172.21.15.83:0/668304834 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f425405e3c0 con 0x563683658920 2024-04-17T10:46:41.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42759ae000 10 client.7716 fetch_fsmap learned FSMap version 56 2024-04-17T10:46:41.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42759ae000 1 -- 172.21.15.83:0/668304834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x5636833e63a0 con 0x563683658920 2024-04-17T10:46:41.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f424affd640 1 -- 172.21.15.83:0/668304834 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425c025070 con 0x563683658920 2024-04-17T10:46:41.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f424affd640 1 -- 172.21.15.83:0/668304834 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 56) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4254013070 con 0x563683658920 2024-04-17T10:46:41.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42759ae000 10 client.7716 fetch_fsmap finished waiting for FSMap version 56 2024-04-17T10:46:41.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42759ae000 10 client.7716 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-17T10:46:41.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42759ae000 20 client.7716 populate_metadata read hostname 'smithi083' 2024-04-17T10:46:41.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42759ae000 1 --2- 172.21.15.83:0/668304834 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x563683614160 0x563683669ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42759ae000 4 client.7716 mds_command: new command op to 7575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:46:41.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42759ae000 1 -- 172.21.15.83:0/668304834 --> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5636833f9a50 con 0x563683614160 2024-04-17T10:46:41.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f426bfff640 1 --2- 172.21.15.83:0/668304834 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x563683614160 0x563683669ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:41.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f424affd640 10 client.7716 ms_handle_connect on v2:172.21.15.83:6834/1056175765 2024-04-17T10:46:41.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f426bfff640 1 --2- 172.21.15.83:0/668304834 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x563683614160 0x563683669ce0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81a9e5426b6f5b9f server_cookie=862a42f4e87d0efc in_seq=0 out_seq=0 2024-04-17T10:46:41.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f424affd640 1 -- 172.21.15.83:0/668304834 <== mds.0 v2:172.21.15.83:6834/1056175765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5636833f9a50 con 0x563683614160 2024-04-17T10:46:41.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f424affd640 10 client.7716 handle_command_reply: tid=0 2024-04-17T10:46:41.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42759ae000 1 client.7716 shutdown 2024-04-17T10:46:41.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42759ae000 1 -- 172.21.15.83:0/668304834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563683658920 msgr2=0x5636836601c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:41.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42759ae000 1 --2- 172.21.15.83:0/668304834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563683658920 0x5636836601c0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f425405de90 tx=0x7f425405c0b0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42759ae000 1 -- 172.21.15.83:0/668304834 shutdown_connections 2024-04-17T10:46:41.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42759ae000 1 --2- 172.21.15.83:0/668304834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563683659240 0x563683612aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42759ae000 1 --2- 172.21.15.83:0/668304834 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x563683614160 0x563683669ce0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42759ae000 1 --2- 172.21.15.83:0/668304834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563683658920 0x5636836601c0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42759ae000 1 --2- 172.21.15.83:0/668304834 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56368360ef30 0x56368365fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f424affd640 0 client.7716 ms_handle_reset on v2:172.21.15.83:6834/1056175765 2024-04-17T10:46:41.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42759ae000 1 -- 172.21.15.83:0/668304834 >> 172.21.15.83:0/668304834 conn(0x5636836152a0 msgr2=0x56368360d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:41.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42759ae000 1 -- 172.21.15.83:0/668304834 shutdown_connections 2024-04-17T10:46:41.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42759ae000 1 -- 172.21.15.83:0/668304834 wait complete. 2024-04-17T10:46:41.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.847+0000 7f42759ae000 20 client.7716 trim_cache size 0 max 16384 2024-04-17T10:46:41.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.863+0000 7f42759ae000 1 Processor -- start 2024-04-17T10:46:41.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.863+0000 7f42759ae000 1 -- start start 2024-04-17T10:46:41.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42759ae000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56368360ef30 0x5636836122c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42759ae000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563683658920 0x563683612800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42759ae000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x563683659240 0x563683661760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42759ae000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426c0b4350 con 0x563683658920 2024-04-17T10:46:41.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42759ae000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426c157e40 con 0x56368360ef30 2024-04-17T10:46:41.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42759ae000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f426c0b3d50 con 0x563683659240 2024-04-17T10:46:41.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42637fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563683658920 0x563683612800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:41.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42637fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563683658920 0x563683612800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47722/0 (socket says 172.21.15.83:47722) 2024-04-17T10:46:41.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42637fe640 1 -- 172.21.15.83:0/4215027047 learned_addr learned my addr 172.21.15.83:0/4215027047 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:41.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f426bfff640 1 --2- 172.21.15.83:0/4215027047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56368360ef30 0x5636836122c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:41.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f4270dde640 1 --2- 172.21.15.83:0/4215027047 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x563683659240 0x563683661760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:41.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42637fe640 1 -- 172.21.15.83:0/4215027047 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x563683659240 msgr2=0x563683661760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:41.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42637fe640 1 --2- 172.21.15.83:0/4215027047 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x563683659240 0x563683661760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42637fe640 1 -- 172.21.15.83:0/4215027047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56368360ef30 msgr2=0x5636836122c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:41.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42637fe640 1 --2- 172.21.15.83:0/4215027047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56368360ef30 0x5636836122c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42637fe640 1 -- 172.21.15.83:0/4215027047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563683659070 con 0x563683658920 2024-04-17T10:46:41.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f426bfff640 1 --2- 172.21.15.83:0/4215027047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56368360ef30 0x5636836122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:41.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42637fe640 1 --2- 172.21.15.83:0/4215027047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563683658920 0x563683612800 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f4254055bf0 tx=0x7f4254085000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ed6b22cc744592 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:41.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42617fa640 1 -- 172.21.15.83:0/4215027047 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42540524b0 con 0x563683658920 2024-04-17T10:46:41.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42617fa640 1 -- 172.21.15.83:0/4215027047 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4254059e00 con 0x563683658920 2024-04-17T10:46:41.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42617fa640 1 -- 172.21.15.83:0/4215027047 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4254055100 con 0x563683658920 2024-04-17T10:46:41.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42759ae000 1 -- 172.21.15.83:0/4215027047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563683658920 msgr2=0x563683612800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:41.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42759ae000 1 --2- 172.21.15.83:0/4215027047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563683658920 0x563683612800 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f4254055bf0 tx=0x7f4254085000 comp rx=0 tx=0).stop 2024-04-17T10:46:41.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42759ae000 1 -- 172.21.15.83:0/4215027047 shutdown_connections 2024-04-17T10:46:41.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42759ae000 1 --2- 172.21.15.83:0/4215027047 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x563683659240 0x563683661760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42759ae000 1 --2- 172.21.15.83:0/4215027047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563683658920 0x563683612800 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42759ae000 1 --2- 172.21.15.83:0/4215027047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56368360ef30 0x5636836122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42759ae000 1 -- 172.21.15.83:0/4215027047 >> 172.21.15.83:0/4215027047 conn(0x5636836152a0 msgr2=0x563683611bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:41.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42759ae000 1 -- 172.21.15.83:0/4215027047 shutdown_connections 2024-04-17T10:46:41.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42759ae000 1 -- 172.21.15.83:0/4215027047 wait complete. 2024-04-17T10:46:41.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42759ae000 1 -- start start 2024-04-17T10:46:41.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42759ae000 1 Processor -- start 2024-04-17T10:46:41.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42759ae000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56368360ef30 0x56368367c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42759ae000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563683658920 0x5636836c1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42759ae000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563683659240 0x5636836c2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42759ae000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636834623d0 con 0x563683658920 2024-04-17T10:46:41.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42759ae000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563683519d50 con 0x563683659240 2024-04-17T10:46:41.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42759ae000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636833f8980 con 0x56368360ef30 2024-04-17T10:46:41.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42759ae000 10 client.0 _ll_register_callbacks cb 0x7f425c003bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:46:41.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42637fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563683658920 0x5636836c1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:41.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42637fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563683658920 0x5636836c1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47726/0 (socket says 172.21.15.83:47726) 2024-04-17T10:46:41.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42637fe640 1 -- 172.21.15.83:0/117159637 learned_addr learned my addr 172.21.15.83:0/117159637 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:41.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f426bfff640 1 --2- 172.21.15.83:0/117159637 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56368360ef30 0x56368367c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:41.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f4270dde640 1 --2- 172.21.15.83:0/117159637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563683659240 0x5636836c2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:41.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42637fe640 1 -- 172.21.15.83:0/117159637 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56368360ef30 msgr2=0x56368367c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:41.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42637fe640 1 --2- 172.21.15.83:0/117159637 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56368360ef30 0x56368367c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42637fe640 1 -- 172.21.15.83:0/117159637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563683659240 msgr2=0x5636836c2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:41.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42637fe640 1 --2- 172.21.15.83:0/117159637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563683659240 0x5636836c2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42637fe640 1 -- 172.21.15.83:0/117159637 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636834df600 con 0x563683658920 2024-04-17T10:46:41.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f4270dde640 1 --2- 172.21.15.83:0/117159637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563683659240 0x5636836c2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:46:41.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42637fe640 1 --2- 172.21.15.83:0/117159637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563683658920 0x5636836c1ad0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f4254053f60 tx=0x7f4254020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d1e461be8a0f79d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:41.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f424affd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:46:41.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f424affd640 1 -- 172.21.15.83:0/117159637 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42540524b0 con 0x563683658920 2024-04-17T10:46:41.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f424affd640 1 -- 172.21.15.83:0/117159637 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f42540850b0 con 0x563683658920 2024-04-17T10:46:41.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42759ae000 1 -- 172.21.15.83:0/117159637 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4254059e00 con 0x563683658920 2024-04-17T10:46:41.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f424affd640 1 -- 172.21.15.83:0/117159637 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4254051a00 con 0x563683658920 2024-04-17T10:46:41.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42759ae000 10 client.7722 fetch_fsmap learned FSMap version 56 2024-04-17T10:46:41.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f42759ae000 1 -- 172.21.15.83:0/117159637 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x5636833f29b0 con 0x563683658920 2024-04-17T10:46:41.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f424affd640 1 -- 172.21.15.83:0/117159637 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4254059e00 con 0x563683658920 2024-04-17T10:46:41.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.867+0000 7f424affd640 1 -- 172.21.15.83:0/117159637 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 56) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4254051ba0 con 0x563683658920 2024-04-17T10:46:41.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f42759ae000 10 client.7722 fetch_fsmap finished waiting for FSMap version 56 2024-04-17T10:46:41.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f42759ae000 10 client.7722 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-17T10:46:41.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f42759ae000 20 client.7722 populate_metadata read hostname 'smithi083' 2024-04-17T10:46:41.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f42759ae000 1 --2- 172.21.15.83:0/117159637 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x5636836c2550 0x563683664530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f42759ae000 4 client.7722 mds_command: new command op to 7575 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T10:46:41.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f42759ae000 1 -- 172.21.15.83:0/117159637 --> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56368346b2e0 con 0x5636836c2550 2024-04-17T10:46:41.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f426bfff640 1 --2- 172.21.15.83:0/117159637 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x5636836c2550 0x563683664530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:41.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f424affd640 10 client.7722 ms_handle_connect on v2:172.21.15.83:6834/1056175765 2024-04-17T10:46:41.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f426bfff640 1 --2- 172.21.15.83:0/117159637 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x5636836c2550 0x563683664530 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5b605d70b9d6c51 server_cookie=b260b5c68cfbe30a in_seq=0 out_seq=0 2024-04-17T10:46:41.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f424affd640 1 -- 172.21.15.83:0/117159637 <== mds.0 v2:172.21.15.83:6834/1056175765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x56368346b2e0 con 0x5636836c2550 2024-04-17T10:46:41.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f424affd640 10 client.7722 handle_command_reply: tid=0 2024-04-17T10:46:41.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f42759ae000 1 client.7722 shutdown 2024-04-17T10:46:41.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f42759ae000 1 -- 172.21.15.83:0/117159637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563683658920 msgr2=0x5636836c1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:41.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f42759ae000 1 --2- 172.21.15.83:0/117159637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563683658920 0x5636836c1ad0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f4254053f60 tx=0x7f4254020000 comp rx=0 tx=0).stop 2024-04-17T10:46:41.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f42759ae000 1 -- 172.21.15.83:0/117159637 shutdown_connections 2024-04-17T10:46:41.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f42759ae000 1 --2- 172.21.15.83:0/117159637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563683659240 0x5636836c2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f42759ae000 1 --2- 172.21.15.83:0/117159637 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x5636836c2550 0x563683664530 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f42759ae000 1 --2- 172.21.15.83:0/117159637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563683658920 0x5636836c1ad0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f424affd640 0 client.7722 ms_handle_reset on v2:172.21.15.83:6834/1056175765 2024-04-17T10:46:41.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f42759ae000 1 --2- 172.21.15.83:0/117159637 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56368360ef30 0x56368367c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f42759ae000 1 -- 172.21.15.83:0/117159637 >> 172.21.15.83:0/117159637 conn(0x5636836152a0 msgr2=0x5636836122c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:41.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f42759ae000 1 -- 172.21.15.83:0/117159637 shutdown_connections 2024-04-17T10:46:41.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f42759ae000 1 -- 172.21.15.83:0/117159637 wait complete. 2024-04-17T10:46:41.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f42759ae000 20 client.7722 trim_cache size 0 max 16384 2024-04-17T10:46:41.874 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:46:41.875 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "3de9b7be-af7f-488a-969e-2b6b8db63bcd", 2024-04-17T10:46:41.875 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-04-17T10:46:41.875 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 7575, 2024-04-17T10:46:41.875 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-17T10:46:41.875 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-17T10:46:41.875 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-17T10:46:41.875 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 4.6634641029999999, 2024-04-17T10:46:41.875 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 56, 2024-04-17T10:46:41.875 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 89, 2024-04-17T10:46:41.875 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 89, 2024-04-17T10:46:41.875 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 9.7026764589999992 2024-04-17T10:46:41.875 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:46:41.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f4272841640 1 -- 172.21.15.83:0/1614506147 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f424c066f60 msgr2=0x7f424c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:41.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f4272841640 1 --2- 172.21.15.83:0/1614506147 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f424c066f60 0x7f424c069420 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f426c0b49c0 tx=0x7f425c007e40 comp rx=0 tx=0).stop 2024-04-17T10:46:41.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f4272841640 1 -- 172.21.15.83:0/1614506147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f426c15b6e0 msgr2=0x7f426c0ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:41.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f4272841640 1 --2- 172.21.15.83:0/1614506147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f426c15b6e0 0x7f426c0ad840 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f425400d5f0 tx=0x7f425400dab0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f4272841640 1 -- 172.21.15.83:0/1614506147 shutdown_connections 2024-04-17T10:46:41.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f4272841640 1 --2- 172.21.15.83:0/1614506147 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f424c066f60 0x7f424c069420 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f4272841640 1 --2- 172.21.15.83:0/1614506147 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f426c15bac0 0x7f426c0add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f4272841640 1 --2- 172.21.15.83:0/1614506147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f426c15b6e0 0x7f426c0ad840 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f4272841640 1 --2- 172.21.15.83:0/1614506147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f426c070470 0x7f426c13dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f4272841640 1 -- 172.21.15.83:0/1614506147 >> 172.21.15.83:0/1614506147 conn(0x7f426c0b15b0 msgr2=0x7f426c158ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:41.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f4272841640 1 -- 172.21.15.83:0/1614506147 shutdown_connections 2024-04-17T10:46:41.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.871+0000 7f4272841640 1 -- 172.21.15.83:0/1614506147 wait complete. 2024-04-17T10:46:41.882 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T10:46:41.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.991+0000 7fd16219d640 1 Processor -- start 2024-04-17T10:46:41.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.991+0000 7fd16219d640 1 -- start start 2024-04-17T10:46:41.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.991+0000 7fd16219d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd15c15ba30 0x7fd15c072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.991+0000 7fd16219d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd15c15c960 0x7fd15c0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.991+0000 7fd16219d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd15c0cfa10 0x7fd15c06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.991+0000 7fd16219d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd15c071ea0 con 0x7fd15c15ba30 2024-04-17T10:46:41.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.991+0000 7fd16219d640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd15c0cfea0 con 0x7fd15c15c960 2024-04-17T10:46:41.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.991+0000 7fd16219d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd15c06e320 con 0x7fd15c0cfa10 2024-04-17T10:46:41.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.991+0000 7fd15b7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd15c15ba30 0x7fd15c072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:41.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.991+0000 7fd15b7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd15c15ba30 0x7fd15c072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47728/0 (socket says 172.21.15.83:47728) 2024-04-17T10:46:41.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.991+0000 7fd15b7fe640 1 -- 172.21.15.83:0/1939346054 learned_addr learned my addr 172.21.15.83:0/1939346054 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:41.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.991+0000 7fd152ffd640 1 --2- 172.21.15.83:0/1939346054 >> v2:172.21.15.167:3300/0 conn(0x7fd15c15c960 0x7fd15c0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:41.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.991+0000 7fd15bfff640 1 --2- 172.21.15.83:0/1939346054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd15c0cfa10 0x7fd15c06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:41.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.991+0000 7fd15b7fe640 1 -- 172.21.15.83:0/1939346054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd15c0cfa10 msgr2=0x7fd15c06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:41.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.991+0000 7fd15b7fe640 1 --2- 172.21.15.83:0/1939346054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd15c0cfa10 0x7fd15c06c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.991+0000 7fd15b7fe640 1 -- 172.21.15.83:0/1939346054 >> v2:172.21.15.167:3300/0 conn(0x7fd15c15c960 msgr2=0x7fd15c0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:46:41.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.991+0000 7fd15b7fe640 1 --2- 172.21.15.83:0/1939346054 >> v2:172.21.15.167:3300/0 conn(0x7fd15c15c960 0x7fd15c0cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.991+0000 7fd15b7fe640 1 -- 172.21.15.83:0/1939346054 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd15c06e040 con 0x7fd15c15ba30 2024-04-17T10:46:41.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.995+0000 7fd15b7fe640 1 --2- 172.21.15.83:0/1939346054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd15c15ba30 0x7fd15c072330 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fd15c0af2b0 tx=0x7fd14c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3914d29e28bf11 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:41.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.995+0000 7fd15affd640 1 -- 172.21.15.83:0/1939346054 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14c0b0070 con 0x7fd15c15ba30 2024-04-17T10:46:41.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.995+0000 7fd15affd640 1 -- 172.21.15.83:0/1939346054 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd14c0ab600 con 0x7fd15c15ba30 2024-04-17T10:46:41.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.995+0000 7fd15affd640 1 -- 172.21.15.83:0/1939346054 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14c0ab9d0 con 0x7fd15c15ba30 2024-04-17T10:46:41.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.995+0000 7fd16219d640 1 -- 172.21.15.83:0/1939346054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd15c15ba30 msgr2=0x7fd15c072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:41.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.995+0000 7fd16219d640 1 --2- 172.21.15.83:0/1939346054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd15c15ba30 0x7fd15c072330 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fd15c0af2b0 tx=0x7fd14c01c490 comp rx=0 tx=0).stop 2024-04-17T10:46:41.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.995+0000 7fd16219d640 1 -- 172.21.15.83:0/1939346054 shutdown_connections 2024-04-17T10:46:41.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.995+0000 7fd16219d640 1 --2- 172.21.15.83:0/1939346054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd15c0cfa10 0x7fd15c06c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.995+0000 7fd16219d640 1 --2- 172.21.15.83:0/1939346054 >> v2:172.21.15.167:3300/0 conn(0x7fd15c15c960 0x7fd15c0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.995+0000 7fd16219d640 1 --2- 172.21.15.83:0/1939346054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd15c15ba30 0x7fd15c072330 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.995+0000 7fd16219d640 1 -- 172.21.15.83:0/1939346054 >> 172.21.15.83:0/1939346054 conn(0x7fd15c06abe0 msgr2=0x7fd15c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:41.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.995+0000 7fd16219d640 1 -- 172.21.15.83:0/1939346054 shutdown_connections 2024-04-17T10:46:41.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.995+0000 7fd16219d640 1 -- 172.21.15.83:0/1939346054 wait complete. 2024-04-17T10:46:41.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.995+0000 7fd16219d640 1 Processor -- start 2024-04-17T10:46:41.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.995+0000 7fd16219d640 1 -- start start 2024-04-17T10:46:41.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.999+0000 7fd16219d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd15c0cfa10 0x7fd15c1f45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.999+0000 7fd16219d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd15c15c960 0x7fd15c147c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.999+0000 7fd16219d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd15c0ce9c0 0x7fd15c0cee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.999+0000 7fd16219d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15c06de20 con 0x7fd15c15c960 2024-04-17T10:46:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.999+0000 7fd16219d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15c06dca0 con 0x7fd15c0ce9c0 2024-04-17T10:46:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.999+0000 7fd16219d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd15c15cd40 con 0x7fd15c0cfa10 2024-04-17T10:46:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.999+0000 7fd152ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd15c15c960 0x7fd15c147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.999+0000 7fd152ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd15c15c960 0x7fd15c147c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47730/0 (socket says 172.21.15.83:47730) 2024-04-17T10:46:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.999+0000 7fd152ffd640 1 -- 172.21.15.83:0/1007439699 learned_addr learned my addr 172.21.15.83:0/1007439699 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:46:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.999+0000 7fd15bfff640 1 --2- 172.21.15.83:0/1007439699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd15c0ce9c0 0x7fd15c0cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.999+0000 7fd15b7fe640 1 --2- 172.21.15.83:0/1007439699 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd15c0cfa10 0x7fd15c1f45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.999+0000 7fd152ffd640 1 -- 172.21.15.83:0/1007439699 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd15c0cfa10 msgr2=0x7fd15c1f45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.999+0000 7fd152ffd640 1 --2- 172.21.15.83:0/1007439699 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd15c0cfa10 0x7fd15c1f45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.999+0000 7fd152ffd640 1 -- 172.21.15.83:0/1007439699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd15c0ce9c0 msgr2=0x7fd15c0cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:41.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.999+0000 7fd152ffd640 1 --2- 172.21.15.83:0/1007439699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd15c0ce9c0 0x7fd15c0cee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:41.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.999+0000 7fd152ffd640 1 -- 172.21.15.83:0/1007439699 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd15c158100 con 0x7fd15c15c960 2024-04-17T10:46:41.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.999+0000 7fd15b7fe640 1 --2- 172.21.15.83:0/1007439699 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd15c0cfa10 0x7fd15c1f45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:46:41.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.999+0000 7fd15bfff640 1 --2- 172.21.15.83:0/1007439699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd15c0ce9c0 0x7fd15c0cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:46:41.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.999+0000 7fd152ffd640 1 --2- 172.21.15.83:0/1007439699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd15c15c960 0x7fd15c147c30 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fd14400d5f0 tx=0x7fd14400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:41.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.999+0000 7fd158ff9640 1 -- 172.21.15.83:0/1007439699 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1440125f0 con 0x7fd15c15c960 2024-04-17T10:46:41.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.999+0000 7fd158ff9640 1 -- 172.21.15.83:0/1007439699 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd144019070 con 0x7fd15c15c960 2024-04-17T10:46:41.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.999+0000 7fd16219d640 1 -- 172.21.15.83:0/1007439699 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd15c158380 con 0x7fd15c15c960 2024-04-17T10:46:41.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.999+0000 7fd158ff9640 1 -- 172.21.15.83:0/1007439699 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1440129b0 con 0x7fd15c15c960 2024-04-17T10:46:41.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.999+0000 7fd16219d640 1 -- 172.21.15.83:0/1007439699 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd15c158940 con 0x7fd15c15c960 2024-04-17T10:46:41.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.999+0000 7fd16219d640 1 -- 172.21.15.83:0/1007439699 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd15c074f90 con 0x7fd15c15c960 2024-04-17T10:46:41.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.999+0000 7fd158ff9640 1 -- 172.21.15.83:0/1007439699 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd144012b50 con 0x7fd15c15c960 2024-04-17T10:46:41.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.999+0000 7fd158ff9640 1 --2- 172.21.15.83:0/1007439699 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd13c066f60 0x7fd13c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:46:41.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.999+0000 7fd158ff9640 1 -- 172.21.15.83:0/1007439699 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1440853f0 con 0x7fd15c15c960 2024-04-17T10:46:41.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.999+0000 7fd15b7fe640 1 --2- 172.21.15.83:0/1007439699 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd13c066f60 0x7fd13c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:46:41.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:41.999+0000 7fd15b7fe640 1 --2- 172.21.15.83:0/1007439699 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd13c066f60 0x7fd13c069420 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd15c0b2410 tx=0x7fd14c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:46:42.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:42.003+0000 7fd158ff9640 1 -- 172.21.15.83:0/1007439699 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd15c158100 con 0x7fd15c15c960 2024-04-17T10:46:42.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:42.183+0000 7fd16219d640 1 -- 172.21.15.83:0/1007439699 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd15c1f4af0 con 0x7fd15c15c960 2024-04-17T10:46:42.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:42.183+0000 7fd158ff9640 1 -- 172.21.15.83:0/1007439699 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v56) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd144011340 con 0x7fd15c15c960 2024-04-17T10:46:42.180 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T10:46:42.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:42.187+0000 7fd16219d640 1 -- 172.21.15.83:0/1007439699 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd13c066f60 msgr2=0x7fd13c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:42.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:42.187+0000 7fd16219d640 1 --2- 172.21.15.83:0/1007439699 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd13c066f60 0x7fd13c069420 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd15c0b2410 tx=0x7fd14c009040 comp rx=0 tx=0).stop 2024-04-17T10:46:42.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:42.187+0000 7fd16219d640 1 -- 172.21.15.83:0/1007439699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd15c15c960 msgr2=0x7fd15c147c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:46:42.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:42.187+0000 7fd16219d640 1 --2- 172.21.15.83:0/1007439699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd15c15c960 0x7fd15c147c30 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fd14400d5f0 tx=0x7fd14400dab0 comp rx=0 tx=0).stop 2024-04-17T10:46:42.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:42.187+0000 7fd16219d640 1 -- 172.21.15.83:0/1007439699 shutdown_connections 2024-04-17T10:46:42.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:42.187+0000 7fd16219d640 1 --2- 172.21.15.83:0/1007439699 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd13c066f60 0x7fd13c069420 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:42.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:42.187+0000 7fd16219d640 1 --2- 172.21.15.83:0/1007439699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd15c0ce9c0 0x7fd15c0cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:42.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:42.187+0000 7fd16219d640 1 --2- 172.21.15.83:0/1007439699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd15c15c960 0x7fd15c147c30 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:42.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:42.187+0000 7fd16219d640 1 --2- 172.21.15.83:0/1007439699 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd15c0cfa10 0x7fd15c1f45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:46:42.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:42.187+0000 7fd16219d640 1 -- 172.21.15.83:0/1007439699 >> 172.21.15.83:0/1007439699 conn(0x7fd15c06abe0 msgr2=0x7fd15c148a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:46:42.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:42.187+0000 7fd16219d640 1 -- 172.21.15.83:0/1007439699 shutdown_connections 2024-04-17T10:46:42.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:42.187+0000 7fd16219d640 1 -- 172.21.15.83:0/1007439699 wait complete. 2024-04-17T10:46:42.198 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T10:46:42.198 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T10:46:42.198 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-04-17T10:46:42.199 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T10:46:42.199 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T10:46:42.199 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T10:46:42.199 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T10:46:42.199 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T10:46:42.199 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:46:42.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:42.199 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-04-17T10:46:42.205 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T10:46:42.205 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T10:46:42.206 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T10:46:42.206 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:46:42.206 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-17T10:46:42.206 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:46:42.206 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:46:42.206 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-04-17T10:46:42.206 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-04-17T10:46:42.206 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:46:42.206 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-04-17T10:46:42.206 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-04-17T10:46:42.206 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:46:42.206 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:46:42.206 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T10:46:42.207 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:46:42.207 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-17T10:46:42.207 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 41982sec preferred_lft 41982sec 2024-04-17T10:46:42.207 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-04-17T10:46:42.207 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:46:42.207 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T10:46:42.207 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:37:0e:23:b2 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:46:42.207 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T10:46:42.207 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:46:42.207 INFO:teuthology.orchestra.run.smithi083.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T10:46:42.207 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-17T10:46:42.208 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T10:46:42.208 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:46:42.208 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-17T10:46:42.208 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:46:42.208 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:46:42.208 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:46:42.208 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-17T10:46:42.208 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:46:42.208 INFO:teuthology.orchestra.run.smithi083.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:46:42.208 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:46:42.208 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-17T10:46:42.208 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:46:42.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:42.209 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T10:46:42.259 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-04-17T10:46:42.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:42.261 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-04-17T10:46:42.314 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-04-17T10:46:42.314 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T10:46:42.314 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-17T10:46:42.314 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-17T10:46:42.314 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-17T10:46:42.315 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T10:46:42.315 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-17T10:46:42.315 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:46:42.315 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:46:42.315 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:46:42.315 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T10:46:42.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:42.315 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2024-04-17T10:46:42.315 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:46:42.315 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-17T10:46:42.315 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-17T10:46:42.316 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-17T10:46:42.316 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:46:42.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:42.390 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-17T10:46:42.438 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-17T10:46:42.438 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T10:46:42.438 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:46:42.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:42.439 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-04-17T10:46:42.492 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-17T10:46:42.493 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:46:42.493 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T10:46:42.537 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:46:42.537 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:46:42.538 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:46:42.585 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:46:42.647 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:46:42.695 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:46:42.695 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:46:42.695 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:46:42.695 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:46:42.695 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:46:42.695 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:46:42.695 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:46:42.695 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:46:42.695 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:46:42.695 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:46:42.695 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:46:42.696 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:46:42.696 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:46:42.696 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:46:42.696 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:46:42.696 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:46:42.696 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:46:42.696 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:46:42.696 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:46:42.696 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:46:42.696 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:46:42.696 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:46:42.696 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:46:42.696 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:46:42.697 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:46:42.697 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:46:42.697 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:46:42.697 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:46:42.697 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:46:42.697 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:46:42.697 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:46:42.697 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:46:42.697 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:46:42.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:42.698 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:46:42.746 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T10:46:42.746 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-17T10:46:42.791 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:46:42.805 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:46:42.854 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:46:42.854 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:46:42.854 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:46:42.854 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:46:42.854 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:46:42.854 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:46:42.854 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:46:42.854 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:46:42.855 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:46:42.855 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:46:42.855 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:46:42.855 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:46:42.855 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:46:42.855 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:46:42.855 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:46:42.855 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:46:42.855 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:46:42.855 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:46:42.855 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:46:42.855 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:46:42.855 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:46:42.855 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:46:42.856 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:46:42.856 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:46:42.856 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:46:42.856 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:46:42.856 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:46:42.856 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:46:42.856 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:46:42.856 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:46:42.856 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:46:42.856 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:46:42.856 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:46:42.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:42.857 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:46:42.869 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:2024-04-17T10:46:42.871+0000 7f465ec7c580 -1 init, newargv = 0x564845f10400 newargc=15 2024-04-17T10:46:42.869 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[70535]: starting ceph client 2024-04-17T10:46:42.874 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[70535]: starting fuse 2024-04-17T10:46:43.975 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:46:43.995 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:46:44.043 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:46:44.043 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:46:44.043 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:46:44.043 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:46:44.043 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:46:44.043 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:46:44.043 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:46:44.043 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:46:44.043 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:46:44.044 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:46:44.044 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:46:44.044 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:46:44.044 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:46:44.044 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:46:44.044 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:46:44.044 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:46:44.044 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:46:44.044 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:46:44.044 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:46:44.044 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:46:44.044 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:46:44.044 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:46:44.045 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:46:44.045 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:46:44.045 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:46:44.045 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:46:44.045 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:46:44.045 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:46:44.045 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:46:44.045 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:46:44.045 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:46:44.045 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:46:44.045 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:46:44.045 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-17T10:46:44.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:44.046 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:46:44.094 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-04-17T10:46:44.094 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-17T10:46:44.095 DEBUG:teuthology.orchestra.run.smithi083:> sudo stdin-killer -- python3 -c ' 2024-04-17T10:46:44.095 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2024-04-17T10:46:44.095 DEBUG:teuthology.orchestra.run.smithi083:> import re 2024-04-17T10:46:44.095 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-04-17T10:46:44.095 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2024-04-17T10:46:44.095 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:46:44.095 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2024-04-17T10:46:44.095 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-17T10:46:44.095 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2024-04-17T10:46:44.095 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-17T10:46:44.095 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:46:44.095 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2024-04-17T10:46:44.095 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2024-04-17T10:46:44.095 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2024-04-17T10:46:44.096 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2024-04-17T10:46:44.096 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:46:44.096 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2024-04-17T10:46:44.096 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T10:46:44.096 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T10:46:44.096 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T10:46:44.096 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2024-04-17T10:46:44.096 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2024-04-17T10:46:44.096 DEBUG:teuthology.orchestra.run.smithi083:> return f 2024-04-17T10:46:44.096 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T10:46:44.096 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:46:44.096 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.0")) 2024-04-17T10:46:44.096 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-04-17T10:46:44.219 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.0.70535.asok 2024-04-17T10:46:44.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:46:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:46:44.231 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.70535.asok 2024-04-17T10:46:44.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:44.231 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.70535.asok status 2024-04-17T10:46:44.390 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:46:44.391 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2024-04-17T10:46:44.391 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-17T10:46:44.391 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-17T10:46:44.391 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "0", 2024-04-17T10:46:44.391 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-04-17T10:46:44.391 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-17T10:46:44.391 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "70535", 2024-04-17T10:46:44.391 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-04-17T10:46:44.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:44.391 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2024-04-17T10:46:44.391 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2024-04-17T10:46:44.391 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 7734, 2024-04-17T10:46:44.391 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2024-04-17T10:46:44.392 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-04-17T10:46:44.392 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-04-17T10:46:44.392 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 7734 2024-04-17T10:46:44.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:44.392 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-04-17T10:46:44.392 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-17T10:46:44.392 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:46:44.392 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1628505415 2024-04-17T10:46:44.392 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:46:44.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:44.392 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-04-17T10:46:44.392 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-17T10:46:44.392 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:46:44.392 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1628505415 2024-04-17T10:46:44.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:46:44.393 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.7734 192.168.0.1:0/1628505415", 2024-04-17T10:46:44.393 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.1:0/1628505415", 2024-04-17T10:46:44.393 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 1, 2024-04-17T10:46:44.393 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 56, 2024-04-17T10:46:44.393 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 89, 2024-04-17T10:46:44.393 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2024-04-17T10:46:44.393 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2024-04-17T10:46:44.393 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2024-04-17T10:46:44.393 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:46:44.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:44.402 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2024-04-17T10:46:44.402 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-17T10:46:44.408 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2024-04-17T10:46:44.408 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-17T10:46:44.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:44.409 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:46:44.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:44.463 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:46:44.463 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:46:44.463 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.0 2024-04-17T10:46:44.463 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:46:44.463 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:46:44.463 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:46:44.463 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:46:44.545 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:44.545 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:44.545 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160222 s, 65.4 MB/s 2024-04-17T10:46:44.563 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:44.564 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:44.564 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016228 s, 64.6 MB/s 2024-04-17T10:46:44.581 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:44.581 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:44.582 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150516 s, 69.7 MB/s 2024-04-17T10:46:44.599 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:44.599 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:44.599 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136918 s, 76.6 MB/s 2024-04-17T10:46:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:44.618 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:44.618 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166578 s, 62.9 MB/s 2024-04-17T10:46:44.635 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:44.635 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:44.635 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147294 s, 71.2 MB/s 2024-04-17T10:46:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165316 s, 63.4 MB/s 2024-04-17T10:46:44.669 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:44.669 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:44.669 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117633 s, 89.1 MB/s 2024-04-17T10:46:44.687 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:44.687 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:44.687 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152186 s, 68.9 MB/s 2024-04-17T10:46:44.700 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:44.701 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:44.701 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113282 s, 92.6 MB/s 2024-04-17T10:46:44.716 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:44.716 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:44.716 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129338 s, 81.1 MB/s 2024-04-17T10:46:44.728 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:44.729 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:44.729 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102387 s, 102 MB/s 2024-04-17T10:46:44.740 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:44.740 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:44.741 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00942448 s, 111 MB/s 2024-04-17T10:46:44.752 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:44.752 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:44.753 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00926737 s, 113 MB/s 2024-04-17T10:46:44.766 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:44.766 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:44.766 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118202 s, 88.7 MB/s 2024-04-17T10:46:44.782 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:44.782 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:44.782 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127387 s, 82.3 MB/s 2024-04-17T10:46:44.796 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:44.796 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:44.796 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113767 s, 92.2 MB/s 2024-04-17T10:46:44.810 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:44.811 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:44.811 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124254 s, 84.4 MB/s 2024-04-17T10:46:44.826 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:44.826 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:44.826 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127634 s, 82.2 MB/s 2024-04-17T10:46:44.839 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:44.840 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:44.840 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108683 s, 96.5 MB/s 2024-04-17T10:46:44.851 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:44.851 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:44.851 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00946731 s, 111 MB/s 2024-04-17T10:46:44.866 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:44.866 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:44.866 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121705 s, 86.2 MB/s 2024-04-17T10:46:44.882 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:44.882 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:44.882 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131795 s, 79.6 MB/s 2024-04-17T10:46:44.897 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:44.897 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:44.897 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125319 s, 83.7 MB/s 2024-04-17T10:46:44.912 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:44.912 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:44.912 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121167 s, 86.5 MB/s 2024-04-17T10:46:44.930 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:44.930 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:44.930 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151124 s, 69.4 MB/s 2024-04-17T10:46:44.944 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:44.944 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:44.944 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116591 s, 89.9 MB/s 2024-04-17T10:46:44.959 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:44.959 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:44.959 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124425 s, 84.3 MB/s 2024-04-17T10:46:44.974 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:44.974 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:44.974 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125601 s, 83.5 MB/s 2024-04-17T10:46:44.988 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:44.988 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:44.988 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102892 s, 102 MB/s 2024-04-17T10:46:45.002 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.002 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.002 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01174 s, 89.3 MB/s 2024-04-17T10:46:45.019 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.019 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.019 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140682 s, 74.5 MB/s 2024-04-17T10:46:45.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:45.020 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:46:45.020 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:46:45.020 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.1 2024-04-17T10:46:45.020 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:46:45.020 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:46:45.020 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:46:45.020 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:46:45.086 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.087 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.087 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122794 s, 85.4 MB/s 2024-04-17T10:46:45.100 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.101 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.101 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109825 s, 95.5 MB/s 2024-04-17T10:46:45.116 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.116 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.116 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013146 s, 79.8 MB/s 2024-04-17T10:46:45.131 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.131 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.131 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120371 s, 87.1 MB/s 2024-04-17T10:46:45.145 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.145 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.146 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115712 s, 90.6 MB/s 2024-04-17T10:46:45.161 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.161 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.162 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134157 s, 78.2 MB/s 2024-04-17T10:46:45.177 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.177 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.177 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013142 s, 79.8 MB/s 2024-04-17T10:46:45.193 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.193 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.193 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130713 s, 80.2 MB/s 2024-04-17T10:46:45.210 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.210 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.210 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132714 s, 79.0 MB/s 2024-04-17T10:46:45.225 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.226 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.226 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127893 s, 82.0 MB/s 2024-04-17T10:46:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142013 s, 73.8 MB/s 2024-04-17T10:46:45.259 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.259 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.259 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129885 s, 80.7 MB/s 2024-04-17T10:46:45.274 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.274 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.274 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112734 s, 93.0 MB/s 2024-04-17T10:46:45.291 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.291 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.291 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138819 s, 75.5 MB/s 2024-04-17T10:46:45.307 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.307 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.307 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134539 s, 77.9 MB/s 2024-04-17T10:46:45.320 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.320 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.320 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100287 s, 105 MB/s 2024-04-17T10:46:45.337 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.337 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.337 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142482 s, 73.6 MB/s 2024-04-17T10:46:45.353 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.353 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.353 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013479 s, 77.8 MB/s 2024-04-17T10:46:45.368 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.369 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.369 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126328 s, 83.0 MB/s 2024-04-17T10:46:45.382 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.382 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.383 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109517 s, 95.7 MB/s 2024-04-17T10:46:45.400 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.400 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.400 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014639 s, 71.6 MB/s 2024-04-17T10:46:45.417 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.417 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.417 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137912 s, 76.0 MB/s 2024-04-17T10:46:45.431 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.431 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.431 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111955 s, 93.7 MB/s 2024-04-17T10:46:45.448 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.448 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.448 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145723 s, 72.0 MB/s 2024-04-17T10:46:45.465 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.466 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.466 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143049 s, 73.3 MB/s 2024-04-17T10:46:45.482 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.482 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.482 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139173 s, 75.3 MB/s 2024-04-17T10:46:45.500 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.500 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.500 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147251 s, 71.2 MB/s 2024-04-17T10:46:45.516 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.516 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.516 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133949 s, 78.3 MB/s 2024-04-17T10:46:45.531 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.531 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.531 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115563 s, 90.7 MB/s 2024-04-17T10:46:45.547 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.547 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.547 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135025 s, 77.7 MB/s 2024-04-17T10:46:45.561 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.561 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.561 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112029 s, 93.6 MB/s 2024-04-17T10:46:45.575 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.575 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.575 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111542 s, 94.0 MB/s 2024-04-17T10:46:45.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:45.576 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:46:45.576 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:46:45.576 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.2 2024-04-17T10:46:45.576 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:46:45.576 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:46:45.577 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:46:45.577 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:46:45.646 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.646 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.646 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132093 s, 79.4 MB/s 2024-04-17T10:46:45.665 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.665 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.665 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152794 s, 68.6 MB/s 2024-04-17T10:46:45.680 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.680 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.680 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115627 s, 90.7 MB/s 2024-04-17T10:46:45.696 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.696 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.696 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124204 s, 84.4 MB/s 2024-04-17T10:46:45.715 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.715 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.715 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147504 s, 71.1 MB/s 2024-04-17T10:46:45.731 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.732 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.732 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125839 s, 83.3 MB/s 2024-04-17T10:46:45.747 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.747 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.747 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116797 s, 89.8 MB/s 2024-04-17T10:46:45.760 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.760 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.760 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010763 s, 97.4 MB/s 2024-04-17T10:46:45.776 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.776 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.776 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128082 s, 81.9 MB/s 2024-04-17T10:46:45.792 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.792 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.792 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131651 s, 79.6 MB/s 2024-04-17T10:46:45.807 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.807 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.807 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127478 s, 82.3 MB/s 2024-04-17T10:46:45.823 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.823 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.823 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128755 s, 81.4 MB/s 2024-04-17T10:46:45.841 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.841 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.841 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146744 s, 71.5 MB/s 2024-04-17T10:46:45.855 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.855 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.855 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115148 s, 91.1 MB/s 2024-04-17T10:46:45.870 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.870 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.870 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123974 s, 84.6 MB/s 2024-04-17T10:46:45.888 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.888 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.888 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013962 s, 75.1 MB/s 2024-04-17T10:46:45.905 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.905 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.905 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142718 s, 73.5 MB/s 2024-04-17T10:46:45.921 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.921 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.921 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139657 s, 75.1 MB/s 2024-04-17T10:46:45.939 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.939 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.939 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136016 s, 77.1 MB/s 2024-04-17T10:46:45.956 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.957 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.957 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142085 s, 73.8 MB/s 2024-04-17T10:46:45.972 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.972 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.972 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114415 s, 91.6 MB/s 2024-04-17T10:46:45.991 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:45.991 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:45.991 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148726 s, 70.5 MB/s 2024-04-17T10:46:46.009 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.010 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.010 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149276 s, 70.2 MB/s 2024-04-17T10:46:46.028 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.029 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.029 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148917 s, 70.4 MB/s 2024-04-17T10:46:46.047 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.047 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.047 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147692 s, 71.0 MB/s 2024-04-17T10:46:46.066 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.066 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.066 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149001 s, 70.4 MB/s 2024-04-17T10:46:46.079 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.080 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.080 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108833 s, 96.3 MB/s 2024-04-17T10:46:46.096 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.096 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.097 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139464 s, 75.2 MB/s 2024-04-17T10:46:46.113 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.113 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.113 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013964 s, 75.1 MB/s 2024-04-17T10:46:46.130 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.130 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.130 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134874 s, 77.7 MB/s 2024-04-17T10:46:46.147 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.147 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.147 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01367 s, 76.7 MB/s 2024-04-17T10:46:46.161 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.162 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.162 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112147 s, 93.5 MB/s 2024-04-17T10:46:46.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:46.163 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:46:46.163 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:46:46.163 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.3 2024-04-17T10:46:46.163 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:46:46.163 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:46:46.163 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:46:46.163 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:46:46.234 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.235 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.235 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124021 s, 84.5 MB/s 2024-04-17T10:46:46.251 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.252 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.252 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141941 s, 73.9 MB/s 2024-04-17T10:46:46.267 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.267 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.268 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132878 s, 78.9 MB/s 2024-04-17T10:46:46.283 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.283 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.284 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132016 s, 79.4 MB/s 2024-04-17T10:46:46.300 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.300 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.300 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137046 s, 76.5 MB/s 2024-04-17T10:46:46.315 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.316 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.316 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129576 s, 80.9 MB/s 2024-04-17T10:46:46.330 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.330 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.330 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121163 s, 86.5 MB/s 2024-04-17T10:46:46.346 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.346 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.346 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132338 s, 79.2 MB/s 2024-04-17T10:46:46.361 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.361 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.361 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012012 s, 87.3 MB/s 2024-04-17T10:46:46.377 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.377 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.377 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137982 s, 76.0 MB/s 2024-04-17T10:46:46.392 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.392 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.392 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120808 s, 86.8 MB/s 2024-04-17T10:46:46.409 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.409 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.409 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142417 s, 73.6 MB/s 2024-04-17T10:46:46.422 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.422 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.422 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100023 s, 105 MB/s 2024-04-17T10:46:46.438 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.438 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.438 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134713 s, 77.8 MB/s 2024-04-17T10:46:46.453 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.453 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.453 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120799 s, 86.8 MB/s 2024-04-17T10:46:46.471 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.472 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.472 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144249 s, 72.7 MB/s 2024-04-17T10:46:46.488 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.488 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.488 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131959 s, 79.5 MB/s 2024-04-17T10:46:46.508 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.509 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.509 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162343 s, 64.6 MB/s 2024-04-17T10:46:46.526 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.526 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.526 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013347 s, 78.6 MB/s 2024-04-17T10:46:46.544 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.544 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.544 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138695 s, 75.6 MB/s 2024-04-17T10:46:46.560 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.561 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.561 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132382 s, 79.2 MB/s 2024-04-17T10:46:46.577 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.577 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.577 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132444 s, 79.2 MB/s 2024-04-17T10:46:46.591 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.591 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.591 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011062 s, 94.8 MB/s 2024-04-17T10:46:46.609 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.609 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.609 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150247 s, 69.8 MB/s 2024-04-17T10:46:46.624 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.625 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.625 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126212 s, 83.1 MB/s 2024-04-17T10:46:46.639 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.639 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.639 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114677 s, 91.4 MB/s 2024-04-17T10:46:46.656 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.656 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.656 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135135 s, 77.6 MB/s 2024-04-17T10:46:46.670 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.670 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.670 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011718 s, 89.5 MB/s 2024-04-17T10:46:46.685 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.685 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.685 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114644 s, 91.5 MB/s 2024-04-17T10:46:46.704 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.704 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.704 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153165 s, 68.5 MB/s 2024-04-17T10:46:46.723 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.723 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.723 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155333 s, 67.5 MB/s 2024-04-17T10:46:46.742 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.742 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.742 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150858 s, 69.5 MB/s 2024-04-17T10:46:46.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:46.743 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:46:46.744 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:46:46.744 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.4 2024-04-17T10:46:46.744 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:46:46.744 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:46:46.744 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:46:46.744 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:46:46.813 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.814 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.814 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139493 s, 75.2 MB/s 2024-04-17T10:46:46.831 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.831 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.831 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146256 s, 71.7 MB/s 2024-04-17T10:46:46.845 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.845 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.846 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113704 s, 92.2 MB/s 2024-04-17T10:46:46.862 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.862 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.862 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136246 s, 77.0 MB/s 2024-04-17T10:46:46.879 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.879 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.879 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135573 s, 77.3 MB/s 2024-04-17T10:46:46.894 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.894 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.894 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131764 s, 79.6 MB/s 2024-04-17T10:46:46.911 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.911 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.911 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133961 s, 78.3 MB/s 2024-04-17T10:46:46.926 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.926 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.926 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119688 s, 87.6 MB/s 2024-04-17T10:46:46.941 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.941 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.941 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012409 s, 84.5 MB/s 2024-04-17T10:46:46.957 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.957 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.957 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123546 s, 84.9 MB/s 2024-04-17T10:46:46.973 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.973 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.973 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131893 s, 79.5 MB/s 2024-04-17T10:46:46.987 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:46.987 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:46.987 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109501 s, 95.8 MB/s 2024-04-17T10:46:47.003 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.004 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.004 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137588 s, 76.2 MB/s 2024-04-17T10:46:47.018 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.018 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.019 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120751 s, 86.8 MB/s 2024-04-17T10:46:47.033 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.033 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.034 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124364 s, 84.3 MB/s 2024-04-17T10:46:47.049 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.049 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.049 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127385 s, 82.3 MB/s 2024-04-17T10:46:47.063 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.063 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.063 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111839 s, 93.8 MB/s 2024-04-17T10:46:47.076 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.076 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.076 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100701 s, 104 MB/s 2024-04-17T10:46:47.092 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.092 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.092 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130289 s, 80.5 MB/s 2024-04-17T10:46:47.105 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.105 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.105 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107326 s, 97.7 MB/s 2024-04-17T10:46:47.119 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.119 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.119 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110753 s, 94.7 MB/s 2024-04-17T10:46:47.132 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.132 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.132 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00983647 s, 107 MB/s 2024-04-17T10:46:47.148 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.148 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.148 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136671 s, 76.7 MB/s 2024-04-17T10:46:47.164 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.164 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.164 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012782 s, 82.0 MB/s 2024-04-17T10:46:47.180 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.180 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.180 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136537 s, 76.8 MB/s 2024-04-17T10:46:47.195 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.195 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.195 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119297 s, 87.9 MB/s 2024-04-17T10:46:47.207 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.207 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.207 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00949341 s, 110 MB/s 2024-04-17T10:46:47.225 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.225 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.226 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150436 s, 69.7 MB/s 2024-04-17T10:46:47.239 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.239 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.240 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116577 s, 89.9 MB/s 2024-04-17T10:46:47.253 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.254 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.254 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115177 s, 91.0 MB/s 2024-04-17T10:46:47.266 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.266 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.266 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0093 s, 113 MB/s 2024-04-17T10:46:47.281 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.281 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.281 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117645 s, 89.1 MB/s 2024-04-17T10:46:47.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:47.282 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:46:47.282 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:46:47.282 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.5 2024-04-17T10:46:47.282 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:46:47.282 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:46:47.282 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:46:47.282 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:46:47.350 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.351 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.351 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120481 s, 87.0 MB/s 2024-04-17T10:46:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120472 s, 87.0 MB/s 2024-04-17T10:46:47.378 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.378 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.378 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106427 s, 98.5 MB/s 2024-04-17T10:46:47.390 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.390 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.390 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00913105 s, 115 MB/s 2024-04-17T10:46:47.404 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.404 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.404 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120648 s, 86.9 MB/s 2024-04-17T10:46:47.416 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.416 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.416 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00959456 s, 109 MB/s 2024-04-17T10:46:47.432 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.432 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.432 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132425 s, 79.2 MB/s 2024-04-17T10:46:47.448 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.448 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.448 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132301 s, 79.3 MB/s 2024-04-17T10:46:47.463 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.463 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.463 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122662 s, 85.5 MB/s 2024-04-17T10:46:47.476 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.476 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.476 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101591 s, 103 MB/s 2024-04-17T10:46:47.492 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.492 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.492 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130904 s, 80.1 MB/s 2024-04-17T10:46:47.507 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.507 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.507 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121439 s, 86.3 MB/s 2024-04-17T10:46:47.523 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.523 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.523 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134009 s, 78.2 MB/s 2024-04-17T10:46:47.536 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.536 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.536 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109838 s, 95.5 MB/s 2024-04-17T10:46:47.553 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.554 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.554 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144152 s, 72.7 MB/s 2024-04-17T10:46:47.566 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.567 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.567 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00992331 s, 106 MB/s 2024-04-17T10:46:47.581 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.581 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.582 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123775 s, 84.7 MB/s 2024-04-17T10:46:47.597 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.597 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.597 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122751 s, 85.4 MB/s 2024-04-17T10:46:47.612 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.612 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.613 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126705 s, 82.8 MB/s 2024-04-17T10:46:47.625 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.625 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.625 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101913 s, 103 MB/s 2024-04-17T10:46:47.639 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.640 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.640 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012009 s, 87.3 MB/s 2024-04-17T10:46:47.655 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.656 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.656 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128182 s, 81.8 MB/s 2024-04-17T10:46:47.671 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.671 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.671 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132094 s, 79.4 MB/s 2024-04-17T10:46:47.684 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.685 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.685 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102546 s, 102 MB/s 2024-04-17T10:46:47.698 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.699 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.699 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103353 s, 101 MB/s 2024-04-17T10:46:47.712 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.712 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.712 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127082 s, 82.5 MB/s 2024-04-17T10:46:47.729 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.729 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.730 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136423 s, 76.9 MB/s 2024-04-17T10:46:47.745 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.745 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.745 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126509 s, 82.9 MB/s 2024-04-17T10:46:47.762 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.762 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.762 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138274 s, 75.8 MB/s 2024-04-17T10:46:47.779 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.779 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.779 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140449 s, 74.7 MB/s 2024-04-17T10:46:47.797 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.798 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.798 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150251 s, 69.8 MB/s 2024-04-17T10:46:47.815 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.815 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.815 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153568 s, 68.3 MB/s 2024-04-17T10:46:47.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:47.816 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:46:47.817 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:46:47.817 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.6 2024-04-17T10:46:47.817 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:46:47.817 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:46:47.817 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:46:47.817 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:46:47.889 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.889 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.890 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131418 s, 79.8 MB/s 2024-04-17T10:46:47.905 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.905 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.906 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132416 s, 79.2 MB/s 2024-04-17T10:46:47.920 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.920 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.921 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120335 s, 87.1 MB/s 2024-04-17T10:46:47.936 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.936 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.936 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131656 s, 79.6 MB/s 2024-04-17T10:46:47.953 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.953 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.953 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134461 s, 78.0 MB/s 2024-04-17T10:46:47.970 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.970 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.970 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131118 s, 80.0 MB/s 2024-04-17T10:46:47.985 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:47.985 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:47.985 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127154 s, 82.5 MB/s 2024-04-17T10:46:48.001 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.001 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.001 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128781 s, 81.4 MB/s 2024-04-17T10:46:48.017 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.017 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.017 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130456 s, 80.4 MB/s 2024-04-17T10:46:48.033 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.033 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.033 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131204 s, 79.9 MB/s 2024-04-17T10:46:48.049 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.049 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.049 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133112 s, 78.8 MB/s 2024-04-17T10:46:48.065 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.065 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.065 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128483 s, 81.6 MB/s 2024-04-17T10:46:48.081 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.083 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.083 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124666 s, 84.1 MB/s 2024-04-17T10:46:48.097 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.097 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.097 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125238 s, 83.7 MB/s 2024-04-17T10:46:48.112 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.112 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.112 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128283 s, 81.7 MB/s 2024-04-17T10:46:48.129 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.130 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.130 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141459 s, 74.1 MB/s 2024-04-17T10:46:48.143 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.144 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.144 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108989 s, 96.2 MB/s 2024-04-17T10:46:48.160 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.160 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.160 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133382 s, 78.6 MB/s 2024-04-17T10:46:48.175 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.175 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.175 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123921 s, 84.6 MB/s 2024-04-17T10:46:48.192 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.192 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.192 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139602 s, 75.1 MB/s 2024-04-17T10:46:48.207 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.207 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.207 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124324 s, 84.3 MB/s 2024-04-17T10:46:48.224 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.224 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.224 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131067 s, 80.0 MB/s 2024-04-17T10:46:48.240 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.240 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.240 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126281 s, 83.0 MB/s 2024-04-17T10:46:48.256 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.257 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.257 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138397 s, 75.8 MB/s 2024-04-17T10:46:48.272 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.272 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.272 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128457 s, 81.6 MB/s 2024-04-17T10:46:48.287 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.288 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.288 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126389 s, 83.0 MB/s 2024-04-17T10:46:48.304 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.304 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.304 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135908 s, 77.2 MB/s 2024-04-17T10:46:48.319 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.319 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.319 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124343 s, 84.3 MB/s 2024-04-17T10:46:48.334 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.334 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.334 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122795 s, 85.4 MB/s 2024-04-17T10:46:48.349 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.350 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.350 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012169 s, 86.2 MB/s 2024-04-17T10:46:48.363 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.363 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.363 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107376 s, 97.7 MB/s 2024-04-17T10:46:48.377 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.377 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.377 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011553 s, 90.8 MB/s 2024-04-17T10:46:48.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:48.379 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:46:48.379 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:46:48.379 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.7 2024-04-17T10:46:48.379 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:46:48.379 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:46:48.379 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:46:48.379 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:46:48.448 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.448 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.448 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013507 s, 77.6 MB/s 2024-04-17T10:46:48.464 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.464 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.464 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123919 s, 84.6 MB/s 2024-04-17T10:46:48.480 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.481 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.481 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013909 s, 75.4 MB/s 2024-04-17T10:46:48.494 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.494 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.495 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111475 s, 94.1 MB/s 2024-04-17T10:46:48.508 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.508 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.509 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108865 s, 96.3 MB/s 2024-04-17T10:46:48.525 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.525 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.525 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136736 s, 76.7 MB/s 2024-04-17T10:46:48.539 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.539 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.539 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109237 s, 96.0 MB/s 2024-04-17T10:46:48.555 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.555 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.555 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127037 s, 82.5 MB/s 2024-04-17T10:46:48.567 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.568 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.568 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100623 s, 104 MB/s 2024-04-17T10:46:48.582 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.582 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.582 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115919 s, 90.5 MB/s 2024-04-17T10:46:48.598 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.598 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.598 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130417 s, 80.4 MB/s 2024-04-17T10:46:48.614 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.615 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.615 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013312 s, 78.8 MB/s 2024-04-17T10:46:48.633 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.633 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.633 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147785 s, 71.0 MB/s 2024-04-17T10:46:48.649 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.649 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.649 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133557 s, 78.5 MB/s 2024-04-17T10:46:48.664 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.664 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.664 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123373 s, 85.0 MB/s 2024-04-17T10:46:48.678 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.678 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.678 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106664 s, 98.3 MB/s 2024-04-17T10:46:48.694 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.694 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.694 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128221 s, 81.8 MB/s 2024-04-17T10:46:48.708 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.708 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.709 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112798 s, 93.0 MB/s 2024-04-17T10:46:48.724 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.724 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.724 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127512 s, 82.2 MB/s 2024-04-17T10:46:48.741 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.741 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.741 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140555 s, 74.6 MB/s 2024-04-17T10:46:48.757 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.757 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.757 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130851 s, 80.1 MB/s 2024-04-17T10:46:48.773 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.773 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.773 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128611 s, 81.5 MB/s 2024-04-17T10:46:48.787 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.788 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.788 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114833 s, 91.3 MB/s 2024-04-17T10:46:48.803 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.803 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.803 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126742 s, 82.7 MB/s 2024-04-17T10:46:48.816 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.816 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.817 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107662 s, 97.4 MB/s 2024-04-17T10:46:48.833 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.833 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.833 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133343 s, 78.6 MB/s 2024-04-17T10:46:48.846 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.846 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.846 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103524 s, 101 MB/s 2024-04-17T10:46:48.861 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.861 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.861 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119357 s, 87.9 MB/s 2024-04-17T10:46:48.875 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.875 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.875 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114802 s, 91.3 MB/s 2024-04-17T10:46:48.891 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.892 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.892 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130919 s, 80.1 MB/s 2024-04-17T10:46:48.904 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.905 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.905 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100686 s, 104 MB/s 2024-04-17T10:46:48.923 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.923 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.923 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123778 s, 84.7 MB/s 2024-04-17T10:46:48.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:48.923 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:46:48.923 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:46:48.924 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.8 2024-04-17T10:46:48.924 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:46:48.924 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:46:48.924 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:46:48.924 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:46:48.992 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:48.992 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:48.992 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133074 s, 78.8 MB/s 2024-04-17T10:46:49.009 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.009 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.009 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143232 s, 73.2 MB/s 2024-04-17T10:46:49.022 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.023 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.023 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104052 s, 101 MB/s 2024-04-17T10:46:49.036 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.037 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.037 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010708 s, 97.9 MB/s 2024-04-17T10:46:49.052 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.052 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.052 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125721 s, 83.4 MB/s 2024-04-17T10:46:49.068 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.068 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.069 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138495 s, 75.7 MB/s 2024-04-17T10:46:49.085 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.085 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.085 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012505 s, 83.9 MB/s 2024-04-17T10:46:49.101 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.102 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.102 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127882 s, 82.0 MB/s 2024-04-17T10:46:49.120 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.120 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.120 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148857 s, 70.4 MB/s 2024-04-17T10:46:49.135 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.135 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.136 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113237 s, 92.6 MB/s 2024-04-17T10:46:49.151 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.152 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.152 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134446 s, 78.0 MB/s 2024-04-17T10:46:49.168 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.168 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.168 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130093 s, 80.6 MB/s 2024-04-17T10:46:49.184 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.185 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.185 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137917 s, 76.0 MB/s 2024-04-17T10:46:49.199 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.199 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.199 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116316 s, 90.1 MB/s 2024-04-17T10:46:49.215 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.216 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.216 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132955 s, 78.9 MB/s 2024-04-17T10:46:49.233 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.233 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.233 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142445 s, 73.6 MB/s 2024-04-17T10:46:49.247 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.247 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.247 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109925 s, 95.4 MB/s 2024-04-17T10:46:49.263 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.263 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.263 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130202 s, 80.5 MB/s 2024-04-17T10:46:49.276 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.276 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.276 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105365 s, 99.5 MB/s 2024-04-17T10:46:49.290 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.290 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.290 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116234 s, 90.2 MB/s 2024-04-17T10:46:49.307 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.307 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.307 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013032 s, 80.5 MB/s 2024-04-17T10:46:49.323 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.323 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.323 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132662 s, 79.0 MB/s 2024-04-17T10:46:49.338 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.338 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.338 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125274 s, 83.7 MB/s 2024-04-17T10:46:49.354 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.354 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.354 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131406 s, 79.8 MB/s 2024-04-17T10:46:49.370 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.370 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.370 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127855 s, 82.0 MB/s 2024-04-17T10:46:49.384 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.384 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.385 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120484 s, 87.0 MB/s 2024-04-17T10:46:49.401 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.401 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.401 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139081 s, 75.4 MB/s 2024-04-17T10:46:49.416 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.417 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.417 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126093 s, 83.2 MB/s 2024-04-17T10:46:49.432 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.432 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.432 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124165 s, 84.5 MB/s 2024-04-17T10:46:49.450 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.450 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.450 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150695 s, 69.6 MB/s 2024-04-17T10:46:49.465 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.465 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.465 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012742 s, 82.3 MB/s 2024-04-17T10:46:49.482 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.482 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.482 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131123 s, 80.0 MB/s 2024-04-17T10:46:49.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:49.484 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:46:49.484 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:46:49.484 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.9 2024-04-17T10:46:49.484 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:46:49.484 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:46:49.484 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:46:49.484 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:46:49.549 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.550 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.550 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112623 s, 93.1 MB/s 2024-04-17T10:46:49.564 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.564 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.564 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116009 s, 90.4 MB/s 2024-04-17T10:46:49.578 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.578 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.578 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110176 s, 95.2 MB/s 2024-04-17T10:46:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121862 s, 86.0 MB/s 2024-04-17T10:46:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133015 s, 78.8 MB/s 2024-04-17T10:46:49.623 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.623 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.623 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114927 s, 91.2 MB/s 2024-04-17T10:46:49.636 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.636 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.636 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101737 s, 103 MB/s 2024-04-17T10:46:49.650 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.651 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.651 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113324 s, 92.5 MB/s 2024-04-17T10:46:49.665 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.665 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.665 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012298 s, 85.3 MB/s 2024-04-17T10:46:49.677 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.677 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.677 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00894479 s, 117 MB/s 2024-04-17T10:46:49.692 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.692 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.692 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123413 s, 85.0 MB/s 2024-04-17T10:46:49.706 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.706 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.706 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112397 s, 93.3 MB/s 2024-04-17T10:46:49.721 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.721 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.721 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121061 s, 86.6 MB/s 2024-04-17T10:46:49.735 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.735 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.735 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110792 s, 94.6 MB/s 2024-04-17T10:46:49.749 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.749 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.749 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116342 s, 90.1 MB/s 2024-04-17T10:46:49.764 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.764 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.764 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122038 s, 85.9 MB/s 2024-04-17T10:46:49.779 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.780 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.780 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123912 s, 84.6 MB/s 2024-04-17T10:46:49.794 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.795 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.795 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120823 s, 86.8 MB/s 2024-04-17T10:46:49.807 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.807 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.807 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00956028 s, 110 MB/s 2024-04-17T10:46:49.820 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.820 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.820 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107832 s, 97.2 MB/s 2024-04-17T10:46:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115786 s, 90.6 MB/s 2024-04-17T10:46:49.849 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.849 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.849 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011852 s, 88.5 MB/s 2024-04-17T10:46:49.865 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.865 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.865 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131448 s, 79.8 MB/s 2024-04-17T10:46:49.880 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.880 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.880 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124543 s, 84.2 MB/s 2024-04-17T10:46:49.900 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.900 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.900 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129493 s, 81.0 MB/s 2024-04-17T10:46:49.914 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.915 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.915 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119374 s, 87.8 MB/s 2024-04-17T10:46:49.929 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.929 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.929 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116261 s, 90.2 MB/s 2024-04-17T10:46:49.943 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.943 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.943 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114877 s, 91.3 MB/s 2024-04-17T10:46:49.957 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.957 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.957 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110714 s, 94.7 MB/s 2024-04-17T10:46:49.971 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.971 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.971 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115586 s, 90.7 MB/s 2024-04-17T10:46:49.988 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:49.988 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:49.988 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139522 s, 75.2 MB/s 2024-04-17T10:46:50.000 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.000 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.000 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00961353 s, 109 MB/s 2024-04-17T10:46:50.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:50.001 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:46:50.001 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:46:50.001 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.10 2024-04-17T10:46:50.001 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:46:50.001 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:46:50.002 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:46:50.002 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:46:50.071 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.071 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.071 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118334 s, 88.6 MB/s 2024-04-17T10:46:50.084 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.084 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.084 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102532 s, 102 MB/s 2024-04-17T10:46:50.096 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.096 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.096 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00921321 s, 114 MB/s 2024-04-17T10:46:50.112 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.113 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.113 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137007 s, 76.5 MB/s 2024-04-17T10:46:50.128 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.128 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.128 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122061 s, 85.9 MB/s 2024-04-17T10:46:50.143 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.143 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.143 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121408 s, 86.4 MB/s 2024-04-17T10:46:50.159 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.159 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.159 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129657 s, 80.9 MB/s 2024-04-17T10:46:50.173 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.174 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.174 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104373 s, 100 MB/s 2024-04-17T10:46:50.189 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.189 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.189 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012506 s, 83.8 MB/s 2024-04-17T10:46:50.203 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.203 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.203 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00977068 s, 107 MB/s 2024-04-17T10:46:50.216 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.216 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.216 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105698 s, 99.2 MB/s 2024-04-17T10:46:50.229 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.229 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.229 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100613 s, 104 MB/s 2024-04-17T10:46:50.244 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.245 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.245 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129841 s, 80.8 MB/s 2024-04-17T10:46:50.257 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.257 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.257 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00956975 s, 110 MB/s 2024-04-17T10:46:50.272 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.272 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.272 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120744 s, 86.8 MB/s 2024-04-17T10:46:50.287 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.287 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.287 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122907 s, 85.3 MB/s 2024-04-17T10:46:50.299 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.299 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.299 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00951078 s, 110 MB/s 2024-04-17T10:46:50.315 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.315 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.315 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132154 s, 79.3 MB/s 2024-04-17T10:46:50.332 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.332 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.332 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127769 s, 82.1 MB/s 2024-04-17T10:46:50.347 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.347 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.347 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126385 s, 83.0 MB/s 2024-04-17T10:46:50.362 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.362 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.362 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119503 s, 87.7 MB/s 2024-04-17T10:46:50.376 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.376 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.376 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117197 s, 89.5 MB/s 2024-04-17T10:46:50.389 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.389 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.389 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107042 s, 98.0 MB/s 2024-04-17T10:46:50.405 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.405 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.405 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012604 s, 83.2 MB/s 2024-04-17T10:46:50.419 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.419 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.419 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111979 s, 93.6 MB/s 2024-04-17T10:46:50.432 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.432 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.432 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00997916 s, 105 MB/s 2024-04-17T10:46:50.446 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.447 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.447 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119841 s, 87.5 MB/s 2024-04-17T10:46:50.461 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.461 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.461 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119051 s, 88.1 MB/s 2024-04-17T10:46:50.475 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.475 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.475 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113149 s, 92.7 MB/s 2024-04-17T10:46:50.491 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.492 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.492 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013478 s, 77.8 MB/s 2024-04-17T10:46:50.507 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.507 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.507 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129192 s, 81.2 MB/s 2024-04-17T10:46:50.522 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.522 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.523 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123142 s, 85.2 MB/s 2024-04-17T10:46:50.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:50.524 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:46:50.524 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:46:50.524 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.11 2024-04-17T10:46:50.524 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:46:50.524 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:46:50.524 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:46:50.524 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:46:50.595 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.595 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.595 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120986 s, 86.7 MB/s 2024-04-17T10:46:50.612 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.612 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.612 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142902 s, 73.4 MB/s 2024-04-17T10:46:50.624 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.624 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.624 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00936347 s, 112 MB/s 2024-04-17T10:46:50.639 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.639 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.639 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123835 s, 84.7 MB/s 2024-04-17T10:46:50.652 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.652 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.652 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00958034 s, 109 MB/s 2024-04-17T10:46:50.668 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.668 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.668 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134557 s, 77.9 MB/s 2024-04-17T10:46:50.682 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.683 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.683 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116067 s, 90.3 MB/s 2024-04-17T10:46:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127537 s, 82.2 MB/s 2024-04-17T10:46:50.712 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.712 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.713 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118516 s, 88.5 MB/s 2024-04-17T10:46:50.728 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.728 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.728 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126246 s, 83.1 MB/s 2024-04-17T10:46:50.743 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.743 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.743 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129558 s, 80.9 MB/s 2024-04-17T10:46:50.759 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.759 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.759 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127685 s, 82.1 MB/s 2024-04-17T10:46:50.774 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.774 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.774 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123214 s, 85.1 MB/s 2024-04-17T10:46:50.788 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.788 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.788 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011144 s, 94.1 MB/s 2024-04-17T10:46:50.803 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.803 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.803 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124221 s, 84.4 MB/s 2024-04-17T10:46:50.817 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.817 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.817 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112437 s, 93.3 MB/s 2024-04-17T10:46:50.831 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.831 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.831 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117785 s, 89.0 MB/s 2024-04-17T10:46:50.847 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.847 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.847 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131579 s, 79.7 MB/s 2024-04-17T10:46:50.861 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.862 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.862 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123184 s, 85.1 MB/s 2024-04-17T10:46:50.875 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.875 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.875 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00999434 s, 105 MB/s 2024-04-17T10:46:50.891 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.891 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.891 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011917 s, 88.0 MB/s 2024-04-17T10:46:50.914 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.914 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.914 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110182 s, 95.2 MB/s 2024-04-17T10:46:50.928 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.928 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.928 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107263 s, 97.8 MB/s 2024-04-17T10:46:50.941 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.941 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.941 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105742 s, 99.2 MB/s 2024-04-17T10:46:50.954 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.954 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.954 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106461 s, 98.5 MB/s 2024-04-17T10:46:50.968 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.968 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.968 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119563 s, 87.7 MB/s 2024-04-17T10:46:50.981 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.981 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.981 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107481 s, 97.6 MB/s 2024-04-17T10:46:50.995 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:50.995 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:50.995 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109846 s, 95.5 MB/s 2024-04-17T10:46:51.008 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.008 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.008 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110492 s, 94.9 MB/s 2024-04-17T10:46:51.021 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.022 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.022 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108721 s, 96.4 MB/s 2024-04-17T10:46:51.032 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.033 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.033 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851865 s, 123 MB/s 2024-04-17T10:46:51.046 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.046 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.046 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112086 s, 93.6 MB/s 2024-04-17T10:46:51.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:51.048 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:46:51.048 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:46:51.048 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.12 2024-04-17T10:46:51.048 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:46:51.048 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:46:51.048 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:46:51.048 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:46:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109129 s, 96.1 MB/s 2024-04-17T10:46:51.125 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.126 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.126 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117024 s, 89.6 MB/s 2024-04-17T10:46:51.137 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.137 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.137 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880707 s, 119 MB/s 2024-04-17T10:46:51.153 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.153 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.153 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139204 s, 75.3 MB/s 2024-04-17T10:46:51.168 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.168 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.168 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124553 s, 84.2 MB/s 2024-04-17T10:46:51.180 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.180 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.180 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00915518 s, 115 MB/s 2024-04-17T10:46:51.194 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.194 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.194 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121704 s, 86.2 MB/s 2024-04-17T10:46:51.208 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.208 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.208 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109077 s, 96.1 MB/s 2024-04-17T10:46:51.222 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.222 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.223 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012043 s, 87.1 MB/s 2024-04-17T10:46:51.237 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.237 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.237 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012216 s, 85.8 MB/s 2024-04-17T10:46:51.253 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.253 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.253 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013564 s, 77.3 MB/s 2024-04-17T10:46:51.268 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.268 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.268 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121738 s, 86.1 MB/s 2024-04-17T10:46:51.282 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.282 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.282 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114994 s, 91.2 MB/s 2024-04-17T10:46:51.298 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.298 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.298 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131356 s, 79.8 MB/s 2024-04-17T10:46:51.311 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.312 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.312 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109425 s, 95.8 MB/s 2024-04-17T10:46:51.324 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.324 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.324 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00981735 s, 107 MB/s 2024-04-17T10:46:51.338 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.338 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.338 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113365 s, 92.5 MB/s 2024-04-17T10:46:51.352 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.352 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.352 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011758 s, 89.2 MB/s 2024-04-17T10:46:51.366 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.366 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.366 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115759 s, 90.6 MB/s 2024-04-17T10:46:51.379 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.379 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.379 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00967722 s, 108 MB/s 2024-04-17T10:46:51.394 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.394 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.394 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126869 s, 82.7 MB/s 2024-04-17T10:46:51.408 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.408 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.408 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113065 s, 92.7 MB/s 2024-04-17T10:46:51.425 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.425 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.425 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138526 s, 75.7 MB/s 2024-04-17T10:46:51.438 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.438 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.438 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105148 s, 99.7 MB/s 2024-04-17T10:46:51.453 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.453 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.453 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127484 s, 82.3 MB/s 2024-04-17T10:46:51.468 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.469 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.469 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125978 s, 83.2 MB/s 2024-04-17T10:46:51.481 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.481 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.481 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00989131 s, 106 MB/s 2024-04-17T10:46:51.492 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.493 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.493 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00890344 s, 118 MB/s 2024-04-17T10:46:51.506 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.507 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.507 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114152 s, 91.9 MB/s 2024-04-17T10:46:51.521 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.521 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.521 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121642 s, 86.2 MB/s 2024-04-17T10:46:51.535 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.536 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.536 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120283 s, 87.2 MB/s 2024-04-17T10:46:51.551 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.551 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.551 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125119 s, 83.8 MB/s 2024-04-17T10:46:51.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:51.552 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:46:51.552 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:46:51.552 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.13 2024-04-17T10:46:51.552 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:46:51.552 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:46:51.552 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:46:51.552 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:46:51.622 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.622 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.623 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120498 s, 87.0 MB/s 2024-04-17T10:46:51.635 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.635 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.635 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00994923 s, 105 MB/s 2024-04-17T10:46:51.653 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.653 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.653 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156231 s, 67.1 MB/s 2024-04-17T10:46:51.670 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.670 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.670 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140659 s, 74.5 MB/s 2024-04-17T10:46:51.685 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.685 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.685 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127347 s, 82.3 MB/s 2024-04-17T10:46:51.705 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.706 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.706 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173947 s, 60.3 MB/s 2024-04-17T10:46:51.720 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.720 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.720 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117134 s, 89.5 MB/s 2024-04-17T10:46:51.733 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.733 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.733 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105698 s, 99.2 MB/s 2024-04-17T10:46:51.744 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.744 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.744 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850988 s, 123 MB/s 2024-04-17T10:46:51.757 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.757 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.758 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010835 s, 96.8 MB/s 2024-04-17T10:46:51.772 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.772 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.772 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011632 s, 90.1 MB/s 2024-04-17T10:46:51.782 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.782 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.782 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780649 s, 134 MB/s 2024-04-17T10:46:51.796 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.796 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.796 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116589 s, 89.9 MB/s 2024-04-17T10:46:51.808 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.808 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.808 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00903475 s, 116 MB/s 2024-04-17T10:46:51.826 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.826 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.826 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122501 s, 85.6 MB/s 2024-04-17T10:46:51.836 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.836 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.836 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111598 s, 94.0 MB/s 2024-04-17T10:46:51.852 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.852 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.852 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129192 s, 81.2 MB/s 2024-04-17T10:46:51.867 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.868 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.868 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116884 s, 89.7 MB/s 2024-04-17T10:46:51.882 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.882 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.882 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112325 s, 93.4 MB/s 2024-04-17T10:46:51.912 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.912 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.913 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833662 s, 126 MB/s 2024-04-17T10:46:51.926 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.926 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.926 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111611 s, 93.9 MB/s 2024-04-17T10:46:51.940 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.940 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.940 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113599 s, 92.3 MB/s 2024-04-17T10:46:51.954 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.955 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.955 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118211 s, 88.7 MB/s 2024-04-17T10:46:51.968 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.968 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.968 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104203 s, 101 MB/s 2024-04-17T10:46:51.979 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.980 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.980 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00913458 s, 115 MB/s 2024-04-17T10:46:51.993 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:51.993 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:51.993 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106562 s, 98.4 MB/s 2024-04-17T10:46:52.005 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.005 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.005 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00985423 s, 106 MB/s 2024-04-17T10:46:52.019 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.019 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.019 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113214 s, 92.6 MB/s 2024-04-17T10:46:52.032 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.033 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.033 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110576 s, 94.8 MB/s 2024-04-17T10:46:52.047 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124995 s, 83.9 MB/s 2024-04-17T10:46:52.061 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.061 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.061 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108136 s, 97.0 MB/s 2024-04-17T10:46:52.072 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.072 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.072 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848752 s, 124 MB/s 2024-04-17T10:46:52.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:52.073 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:46:52.073 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:46:52.073 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.14 2024-04-17T10:46:52.073 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:46:52.073 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:46:52.073 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:46:52.073 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:46:52.141 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.141 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.142 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126488 s, 82.9 MB/s 2024-04-17T10:46:52.153 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.153 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.153 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880463 s, 119 MB/s 2024-04-17T10:46:52.169 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.169 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.169 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132609 s, 79.1 MB/s 2024-04-17T10:46:52.181 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.181 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.181 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00966618 s, 108 MB/s 2024-04-17T10:46:52.195 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.195 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.195 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011581 s, 90.5 MB/s 2024-04-17T10:46:52.208 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.209 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.209 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108584 s, 96.6 MB/s 2024-04-17T10:46:52.224 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.225 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.225 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135078 s, 77.6 MB/s 2024-04-17T10:46:52.240 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.241 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.241 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012929 s, 81.1 MB/s 2024-04-17T10:46:52.256 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.256 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.256 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013092 s, 80.1 MB/s 2024-04-17T10:46:52.269 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.269 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.269 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110337 s, 95.0 MB/s 2024-04-17T10:46:52.282 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.283 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.283 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110779 s, 94.7 MB/s 2024-04-17T10:46:52.298 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.298 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.298 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131657 s, 79.6 MB/s 2024-04-17T10:46:52.311 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.311 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.311 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102991 s, 102 MB/s 2024-04-17T10:46:52.324 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.325 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.325 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116827 s, 89.8 MB/s 2024-04-17T10:46:52.334 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.335 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.335 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771108 s, 136 MB/s 2024-04-17T10:46:52.349 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.349 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.349 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126108 s, 83.1 MB/s 2024-04-17T10:46:52.359 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.360 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.360 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811867 s, 129 MB/s 2024-04-17T10:46:52.373 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.373 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.373 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115456 s, 90.8 MB/s 2024-04-17T10:46:52.388 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.388 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.388 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121466 s, 86.3 MB/s 2024-04-17T10:46:52.402 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.402 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.403 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123889 s, 84.6 MB/s 2024-04-17T10:46:52.414 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.414 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.414 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793675 s, 132 MB/s 2024-04-17T10:46:52.425 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.426 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.426 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104622 s, 100 MB/s 2024-04-17T10:46:52.439 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.439 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.439 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109299 s, 95.9 MB/s 2024-04-17T10:46:52.452 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.452 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.452 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109177 s, 96.0 MB/s 2024-04-17T10:46:52.462 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.462 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.463 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783134 s, 134 MB/s 2024-04-17T10:46:52.474 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.474 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.474 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00927748 s, 113 MB/s 2024-04-17T10:46:52.488 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.489 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.489 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122039 s, 85.9 MB/s 2024-04-17T10:46:52.499 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.499 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.499 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815055 s, 129 MB/s 2024-04-17T10:46:52.511 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.511 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.511 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101157 s, 104 MB/s 2024-04-17T10:46:52.521 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.521 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.521 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762824 s, 137 MB/s 2024-04-17T10:46:52.536 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.536 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.536 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130659 s, 80.3 MB/s 2024-04-17T10:46:52.549 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.549 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.549 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010129 s, 104 MB/s 2024-04-17T10:46:52.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:52.550 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:46:52.550 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:46:52.550 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.15 2024-04-17T10:46:52.550 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:46:52.550 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:46:52.550 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:46:52.550 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:46:52.621 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.621 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.621 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142172 s, 73.8 MB/s 2024-04-17T10:46:52.633 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.633 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.633 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00959011 s, 109 MB/s 2024-04-17T10:46:52.644 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.645 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.645 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106254 s, 98.7 MB/s 2024-04-17T10:46:52.658 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.658 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.658 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115821 s, 90.5 MB/s 2024-04-17T10:46:52.671 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.672 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.672 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114402 s, 91.7 MB/s 2024-04-17T10:46:52.683 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.684 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.684 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100179 s, 105 MB/s 2024-04-17T10:46:52.696 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.696 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.696 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107559 s, 97.5 MB/s 2024-04-17T10:46:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00992524 s, 106 MB/s 2024-04-17T10:46:52.721 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.721 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.721 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110393 s, 95.0 MB/s 2024-04-17T10:46:52.734 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.734 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.735 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110959 s, 94.5 MB/s 2024-04-17T10:46:52.748 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.748 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.748 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112178 s, 93.5 MB/s 2024-04-17T10:46:52.759 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.760 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.760 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00972397 s, 108 MB/s 2024-04-17T10:46:52.772 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.772 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.772 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106791 s, 98.2 MB/s 2024-04-17T10:46:52.781 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.781 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.781 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698799 s, 150 MB/s 2024-04-17T10:46:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113449 s, 92.4 MB/s 2024-04-17T10:46:52.810 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.810 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.810 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120258 s, 87.2 MB/s 2024-04-17T10:46:52.823 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.823 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.823 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107219 s, 97.8 MB/s 2024-04-17T10:46:52.834 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.834 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.834 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00928937 s, 113 MB/s 2024-04-17T10:46:52.846 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.847 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.847 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103134 s, 102 MB/s 2024-04-17T10:46:52.860 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.860 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.860 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111181 s, 94.3 MB/s 2024-04-17T10:46:52.872 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.872 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.873 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106788 s, 98.2 MB/s 2024-04-17T10:46:52.887 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.887 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.887 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126492 s, 82.9 MB/s 2024-04-17T10:46:52.922 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.922 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.923 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132572 s, 79.1 MB/s 2024-04-17T10:46:52.934 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.934 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.934 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00980982 s, 107 MB/s 2024-04-17T10:46:52.949 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.949 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.949 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124821 s, 84.0 MB/s 2024-04-17T10:46:52.959 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:52.959 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:52.959 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767727 s, 137 MB/s 2024-04-17T10:46:52.974 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.128 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.128 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124555 s, 84.2 MB/s 2024-04-17T10:46:53.130 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.130 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.130 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104913 s, 99.9 MB/s 2024-04-17T10:46:53.130 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.130 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.130 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112861 s, 92.9 MB/s 2024-04-17T10:46:53.130 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.130 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.131 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105705 s, 99.2 MB/s 2024-04-17T10:46:53.131 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.131 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.131 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114807 s, 91.3 MB/s 2024-04-17T10:46:53.131 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.131 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.131 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112948 s, 92.8 MB/s 2024-04-17T10:46:53.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:53.131 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:46:53.131 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:46:53.132 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.16 2024-04-17T10:46:53.132 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:46:53.132 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:46:53.132 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:46:53.132 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:46:53.203 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.203 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.203 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134902 s, 77.7 MB/s 2024-04-17T10:46:53.217 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.217 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.217 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116305 s, 90.2 MB/s 2024-04-17T10:46:53.232 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.232 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.232 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122851 s, 85.4 MB/s 2024-04-17T10:46:53.245 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.245 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.245 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110115 s, 95.2 MB/s 2024-04-17T10:46:53.260 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.260 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.260 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122749 s, 85.4 MB/s 2024-04-17T10:46:53.274 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.275 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.275 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122245 s, 85.8 MB/s 2024-04-17T10:46:53.288 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.288 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.288 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011007 s, 95.3 MB/s 2024-04-17T10:46:53.302 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.302 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.302 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116831 s, 89.8 MB/s 2024-04-17T10:46:53.315 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.315 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.316 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100237 s, 105 MB/s 2024-04-17T10:46:53.332 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.332 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.332 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140306 s, 74.7 MB/s 2024-04-17T10:46:53.347 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.347 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.347 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122532 s, 85.6 MB/s 2024-04-17T10:46:53.363 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.363 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.363 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128068 s, 81.9 MB/s 2024-04-17T10:46:53.378 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.378 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.378 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123887 s, 84.6 MB/s 2024-04-17T10:46:53.394 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.394 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.394 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130853 s, 80.1 MB/s 2024-04-17T10:46:53.410 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.410 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.410 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130901 s, 80.1 MB/s 2024-04-17T10:46:53.424 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.424 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.424 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112394 s, 93.3 MB/s 2024-04-17T10:46:53.440 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.440 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.440 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127366 s, 82.3 MB/s 2024-04-17T10:46:53.454 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.454 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.454 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112098 s, 93.5 MB/s 2024-04-17T10:46:53.471 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.471 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.471 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013703 s, 76.5 MB/s 2024-04-17T10:46:53.489 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.489 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.489 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143992 s, 72.8 MB/s 2024-04-17T10:46:53.503 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.503 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.503 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107407 s, 97.6 MB/s 2024-04-17T10:46:53.519 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.519 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.519 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128669 s, 81.5 MB/s 2024-04-17T10:46:53.534 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.535 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.535 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126543 s, 82.9 MB/s 2024-04-17T10:46:53.548 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.548 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.548 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010476 s, 100 MB/s 2024-04-17T10:46:53.563 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.563 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.563 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124939 s, 83.9 MB/s 2024-04-17T10:46:53.578 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.578 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.578 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120601 s, 86.9 MB/s 2024-04-17T10:46:53.591 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.591 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.591 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101686 s, 103 MB/s 2024-04-17T10:46:53.607 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.607 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.608 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135177 s, 77.6 MB/s 2024-04-17T10:46:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130907 s, 80.1 MB/s 2024-04-17T10:46:53.638 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.639 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.639 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00963916 s, 109 MB/s 2024-04-17T10:46:53.651 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.651 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.651 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124851 s, 84.0 MB/s 2024-04-17T10:46:53.667 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.667 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.667 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132973 s, 78.9 MB/s 2024-04-17T10:46:53.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:53.669 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:46:53.669 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:46:53.669 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.17 2024-04-17T10:46:53.669 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:46:53.669 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:46:53.669 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:46:53.669 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:46:53.740 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.741 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.741 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134113 s, 78.2 MB/s 2024-04-17T10:46:53.757 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.757 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.757 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129769 s, 80.8 MB/s 2024-04-17T10:46:53.774 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.774 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.775 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140713 s, 74.5 MB/s 2024-04-17T10:46:53.790 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.790 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.790 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125935 s, 83.3 MB/s 2024-04-17T10:46:53.807 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.807 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.807 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138749 s, 75.6 MB/s 2024-04-17T10:46:53.821 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.821 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.821 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108782 s, 96.4 MB/s 2024-04-17T10:46:53.837 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.837 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.837 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133539 s, 78.5 MB/s 2024-04-17T10:46:53.854 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.854 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.854 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134582 s, 77.9 MB/s 2024-04-17T10:46:53.869 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.870 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.870 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012735 s, 82.3 MB/s 2024-04-17T10:46:53.886 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.886 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.886 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01332 s, 78.7 MB/s 2024-04-17T10:46:53.931 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.932 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.932 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133351 s, 78.6 MB/s 2024-04-17T10:46:53.947 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.947 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.947 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126311 s, 83.0 MB/s 2024-04-17T10:46:53.963 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.963 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.963 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128701 s, 81.5 MB/s 2024-04-17T10:46:53.978 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.979 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.979 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130165 s, 80.6 MB/s 2024-04-17T10:46:53.993 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:53.993 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:53.994 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120193 s, 87.2 MB/s 2024-04-17T10:46:54.010 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.010 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.010 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013706 s, 76.5 MB/s 2024-04-17T10:46:54.025 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.025 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.026 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129113 s, 81.2 MB/s 2024-04-17T10:46:54.041 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.041 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.041 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131204 s, 79.9 MB/s 2024-04-17T10:46:54.058 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.059 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.059 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137382 s, 76.3 MB/s 2024-04-17T10:46:54.074 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.075 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.075 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132951 s, 78.9 MB/s 2024-04-17T10:46:54.092 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.092 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.092 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143831 s, 72.9 MB/s 2024-04-17T10:46:54.107 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.107 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.107 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119943 s, 87.4 MB/s 2024-04-17T10:46:54.126 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.126 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.126 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150593 s, 69.6 MB/s 2024-04-17T10:46:54.144 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.144 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.144 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014419 s, 72.7 MB/s 2024-04-17T10:46:54.162 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.162 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.162 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146899 s, 71.4 MB/s 2024-04-17T10:46:54.180 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.180 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.181 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145449 s, 72.1 MB/s 2024-04-17T10:46:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139614 s, 75.1 MB/s 2024-04-17T10:46:54.215 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.215 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.215 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140613 s, 74.6 MB/s 2024-04-17T10:46:54.230 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.230 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.230 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115377 s, 90.9 MB/s 2024-04-17T10:46:54.247 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.247 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.247 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136368 s, 76.9 MB/s 2024-04-17T10:46:54.264 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.265 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.265 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142997 s, 73.3 MB/s 2024-04-17T10:46:54.283 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.283 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.283 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156214 s, 67.1 MB/s 2024-04-17T10:46:54.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:54.284 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:46:54.284 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:46:54.284 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.18 2024-04-17T10:46:54.285 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:46:54.285 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:46:54.285 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:46:54.285 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:46:54.351 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.351 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.352 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122329 s, 85.7 MB/s 2024-04-17T10:46:54.364 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.364 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.364 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00950438 s, 110 MB/s 2024-04-17T10:46:54.380 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.380 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.380 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131127 s, 80.0 MB/s 2024-04-17T10:46:54.396 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.396 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.396 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133114 s, 78.8 MB/s 2024-04-17T10:46:54.411 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.411 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.411 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120342 s, 87.1 MB/s 2024-04-17T10:46:54.424 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.424 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.424 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108089 s, 97.0 MB/s 2024-04-17T10:46:54.436 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.436 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.436 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0097823 s, 107 MB/s 2024-04-17T10:46:54.450 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.450 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.450 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010681 s, 98.2 MB/s 2024-04-17T10:46:54.464 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.464 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.465 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011743 s, 89.3 MB/s 2024-04-17T10:46:54.479 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.479 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.479 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117633 s, 89.1 MB/s 2024-04-17T10:46:54.493 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.493 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.493 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121723 s, 86.1 MB/s 2024-04-17T10:46:54.507 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.507 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.507 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109467 s, 95.8 MB/s 2024-04-17T10:46:54.521 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.521 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.521 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120369 s, 87.1 MB/s 2024-04-17T10:46:54.535 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.535 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.536 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011777 s, 89.0 MB/s 2024-04-17T10:46:54.553 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.553 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157695 s, 66.5 MB/s 2024-04-17T10:46:54.567 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:54.567 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:54.567 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112042 s, 93.6 MB/s 2024-04-17T10:46:55.438 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:55.439 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:55.439 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.868683 s, 1.2 MB/s 2024-04-17T10:46:55.457 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:55.457 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:55.457 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133805 s, 78.4 MB/s 2024-04-17T10:46:55.479 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:55.479 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:55.479 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182761 s, 57.4 MB/s 2024-04-17T10:46:55.502 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:55.502 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:55.502 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198935 s, 52.7 MB/s 2024-04-17T10:46:55.519 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:55.519 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:55.519 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139514 s, 75.2 MB/s 2024-04-17T10:46:55.534 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:55.534 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:55.535 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127813 s, 82.0 MB/s 2024-04-17T10:46:55.551 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:55.551 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:55.552 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142803 s, 73.4 MB/s 2024-04-17T10:46:55.565 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:55.565 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:55.566 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110295 s, 95.1 MB/s 2024-04-17T10:46:55.581 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:55.581 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:55.581 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125437 s, 83.6 MB/s 2024-04-17T10:46:55.596 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:55.596 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:55.596 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123325 s, 85.0 MB/s 2024-04-17T10:46:55.612 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:55.612 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:55.612 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129563 s, 80.9 MB/s 2024-04-17T10:46:55.628 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:55.628 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:55.628 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134846 s, 77.8 MB/s 2024-04-17T10:46:55.643 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:55.643 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:55.643 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121566 s, 86.3 MB/s 2024-04-17T10:46:55.660 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:55.660 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:55.660 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137072 s, 76.5 MB/s 2024-04-17T10:46:55.678 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:55.678 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:55.678 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150267 s, 69.8 MB/s 2024-04-17T10:46:55.694 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:55.695 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:55.695 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129366 s, 81.1 MB/s 2024-04-17T10:46:55.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:55.697 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:46:55.697 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:46:55.697 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.19 2024-04-17T10:46:55.697 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:46:55.697 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:46:55.697 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:46:55.697 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:46:55.767 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:55.768 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:55.768 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110954 s, 94.5 MB/s 2024-04-17T10:46:55.785 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:55.786 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:55.786 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134046 s, 78.2 MB/s 2024-04-17T10:46:55.799 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:55.799 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:55.800 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104803 s, 100 MB/s 2024-04-17T10:46:55.816 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:55.817 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:55.817 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142909 s, 73.4 MB/s 2024-04-17T10:46:55.831 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:55.831 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:55.831 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106163 s, 98.8 MB/s 2024-04-17T10:46:55.847 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:55.847 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:55.847 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135193 s, 77.6 MB/s 2024-04-17T10:46:55.862 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:55.862 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:55.862 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012142 s, 86.4 MB/s 2024-04-17T10:46:55.877 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:55.877 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:55.877 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124281 s, 84.4 MB/s 2024-04-17T10:46:55.894 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:55.894 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:55.895 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112463 s, 93.2 MB/s 2024-04-17T10:46:55.910 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:55.910 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:55.911 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169182 s, 62.0 MB/s 2024-04-17T10:46:55.945 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:55.946 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:55.946 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139234 s, 75.3 MB/s 2024-04-17T10:46:55.962 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:55.962 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:55.963 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131361 s, 79.8 MB/s 2024-04-17T10:46:55.978 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:55.978 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:55.978 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119631 s, 87.7 MB/s 2024-04-17T10:46:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122974 s, 85.3 MB/s 2024-04-17T10:46:56.008 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.008 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.008 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112545 s, 93.2 MB/s 2024-04-17T10:46:56.023 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.023 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.023 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113937 s, 92.0 MB/s 2024-04-17T10:46:56.035 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.036 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.036 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00957356 s, 110 MB/s 2024-04-17T10:46:56.050 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.050 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.050 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111814 s, 93.8 MB/s 2024-04-17T10:46:56.064 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.064 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.064 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011021 s, 95.1 MB/s 2024-04-17T10:46:56.078 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.079 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.079 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011793 s, 88.9 MB/s 2024-04-17T10:46:56.093 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.093 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.093 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116517 s, 90.0 MB/s 2024-04-17T10:46:56.108 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.108 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.108 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121253 s, 86.5 MB/s 2024-04-17T10:46:56.122 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.122 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.122 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112282 s, 93.4 MB/s 2024-04-17T10:46:56.136 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.136 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.136 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112903 s, 92.9 MB/s 2024-04-17T10:46:56.152 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.152 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.153 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131903 s, 79.5 MB/s 2024-04-17T10:46:56.166 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.166 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.167 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111437 s, 94.1 MB/s 2024-04-17T10:46:56.179 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.180 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.180 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105611 s, 99.3 MB/s 2024-04-17T10:46:56.195 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.195 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.195 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126795 s, 82.7 MB/s 2024-04-17T10:46:56.209 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.209 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.209 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011519 s, 91.0 MB/s 2024-04-17T10:46:56.224 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.224 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.224 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114185 s, 91.8 MB/s 2024-04-17T10:46:56.241 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.241 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.241 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125227 s, 83.7 MB/s 2024-04-17T10:46:56.256 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.256 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.256 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118039 s, 88.8 MB/s 2024-04-17T10:46:56.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:56.257 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:46:56.257 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:46:56.257 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.20 2024-04-17T10:46:56.258 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:46:56.258 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:46:56.258 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:46:56.258 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:46:56.327 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.327 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.327 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116797 s, 89.8 MB/s 2024-04-17T10:46:56.344 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.345 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.345 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130819 s, 80.2 MB/s 2024-04-17T10:46:56.361 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.361 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.361 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149233 s, 70.3 MB/s 2024-04-17T10:46:56.376 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.376 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.376 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115383 s, 90.9 MB/s 2024-04-17T10:46:56.391 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.391 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.391 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127479 s, 82.3 MB/s 2024-04-17T10:46:56.405 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.405 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.405 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107689 s, 97.4 MB/s 2024-04-17T10:46:56.420 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.420 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.420 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126289 s, 83.0 MB/s 2024-04-17T10:46:56.432 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.432 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.432 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00915669 s, 115 MB/s 2024-04-17T10:46:56.444 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.444 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.444 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00948187 s, 111 MB/s 2024-04-17T10:46:56.456 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.456 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.456 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.009418 s, 111 MB/s 2024-04-17T10:46:56.469 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.469 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.469 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00949708 s, 110 MB/s 2024-04-17T10:46:56.483 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.483 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.483 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00903638 s, 116 MB/s 2024-04-17T10:46:56.497 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.497 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.497 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136441 s, 76.9 MB/s 2024-04-17T10:46:56.514 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.514 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.514 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140107 s, 74.8 MB/s 2024-04-17T10:46:56.527 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.527 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.527 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00999558 s, 105 MB/s 2024-04-17T10:46:56.543 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.543 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.543 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013504 s, 77.6 MB/s 2024-04-17T10:46:56.556 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.557 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.557 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103314 s, 101 MB/s 2024-04-17T10:46:56.571 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.571 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.572 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121976 s, 86.0 MB/s 2024-04-17T10:46:56.587 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.587 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.587 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117923 s, 88.9 MB/s 2024-04-17T10:46:56.602 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.602 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.602 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130265 s, 80.5 MB/s 2024-04-17T10:46:56.616 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.617 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.617 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119219 s, 88.0 MB/s 2024-04-17T10:46:56.633 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.633 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.633 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125185 s, 83.8 MB/s 2024-04-17T10:46:56.647 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.647 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.647 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109555 s, 95.7 MB/s 2024-04-17T10:46:56.664 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.664 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.665 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014666 s, 71.5 MB/s 2024-04-17T10:46:56.680 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.680 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.680 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124954 s, 83.9 MB/s 2024-04-17T10:46:56.696 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.696 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.696 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128428 s, 81.6 MB/s 2024-04-17T10:46:56.713 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.713 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.713 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012601 s, 83.2 MB/s 2024-04-17T10:46:56.729 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.729 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.729 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134922 s, 77.7 MB/s 2024-04-17T10:46:56.743 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.744 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.744 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105618 s, 99.3 MB/s 2024-04-17T10:46:56.759 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.759 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.759 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127851 s, 82.0 MB/s 2024-04-17T10:46:56.776 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.776 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.776 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136802 s, 76.6 MB/s 2024-04-17T10:46:56.792 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.792 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.792 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133451 s, 78.6 MB/s 2024-04-17T10:46:56.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:56.793 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:46:56.793 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:46:56.793 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.21 2024-04-17T10:46:56.794 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:46:56.794 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:46:56.794 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:46:56.794 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:46:56.863 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.864 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.864 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130492 s, 80.4 MB/s 2024-04-17T10:46:56.880 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.880 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.880 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129379 s, 81.0 MB/s 2024-04-17T10:46:56.895 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.895 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.895 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122822 s, 85.4 MB/s 2024-04-17T10:46:56.912 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.912 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.912 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138519 s, 75.7 MB/s 2024-04-17T10:46:56.947 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.947 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.947 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120072 s, 87.3 MB/s 2024-04-17T10:46:56.963 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.963 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.963 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121474 s, 86.3 MB/s 2024-04-17T10:46:56.981 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.981 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.981 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138411 s, 75.8 MB/s 2024-04-17T10:46:56.996 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:56.996 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:56.996 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011638 s, 90.1 MB/s 2024-04-17T10:46:57.012 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.012 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.012 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119014 s, 88.1 MB/s 2024-04-17T10:46:57.026 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.026 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.026 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104044 s, 101 MB/s 2024-04-17T10:46:57.042 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.042 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.042 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125861 s, 83.3 MB/s 2024-04-17T10:46:57.059 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.059 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.059 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138746 s, 75.6 MB/s 2024-04-17T10:46:57.075 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.075 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.075 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131342 s, 79.8 MB/s 2024-04-17T10:46:57.089 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.090 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.090 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117873 s, 89.0 MB/s 2024-04-17T10:46:57.105 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.105 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.105 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128262 s, 81.8 MB/s 2024-04-17T10:46:57.123 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.123 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.123 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125279 s, 83.7 MB/s 2024-04-17T10:46:57.135 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.135 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.135 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114862 s, 91.3 MB/s 2024-04-17T10:46:57.151 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.152 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.152 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136083 s, 77.1 MB/s 2024-04-17T10:46:57.167 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.167 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.167 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129007 s, 81.3 MB/s 2024-04-17T10:46:57.179 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.179 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.179 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00975793 s, 107 MB/s 2024-04-17T10:46:57.197 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.197 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.197 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148239 s, 70.7 MB/s 2024-04-17T10:46:57.213 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.213 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.213 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012893 s, 81.3 MB/s 2024-04-17T10:46:57.227 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.227 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.227 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011039 s, 95.0 MB/s 2024-04-17T10:46:57.241 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.241 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.241 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116347 s, 90.1 MB/s 2024-04-17T10:46:57.255 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.255 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.255 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117475 s, 89.3 MB/s 2024-04-17T10:46:57.270 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.270 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.270 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128165 s, 81.8 MB/s 2024-04-17T10:46:57.283 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.283 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.283 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109559 s, 95.7 MB/s 2024-04-17T10:46:57.297 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.298 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.298 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121611 s, 86.2 MB/s 2024-04-17T10:46:57.314 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.314 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.314 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139604 s, 75.1 MB/s 2024-04-17T10:46:57.324 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.324 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.324 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769856 s, 136 MB/s 2024-04-17T10:46:57.336 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.336 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.336 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102315 s, 102 MB/s 2024-04-17T10:46:57.349 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.349 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.349 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107246 s, 97.8 MB/s 2024-04-17T10:46:57.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:57.350 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:46:57.351 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:46:57.351 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.22 2024-04-17T10:46:57.351 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:46:57.351 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:46:57.351 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:46:57.351 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:46:57.413 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.413 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.413 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105564 s, 99.3 MB/s 2024-04-17T10:46:57.424 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.424 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.424 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846503 s, 124 MB/s 2024-04-17T10:46:57.439 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.439 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.439 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128658 s, 81.5 MB/s 2024-04-17T10:46:57.452 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.452 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.452 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010729 s, 97.7 MB/s 2024-04-17T10:46:57.464 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.465 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.465 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107798 s, 97.3 MB/s 2024-04-17T10:46:57.478 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.478 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.478 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109183 s, 96.0 MB/s 2024-04-17T10:46:57.487 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.487 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.487 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759175 s, 138 MB/s 2024-04-17T10:46:57.498 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.498 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.498 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837189 s, 125 MB/s 2024-04-17T10:46:57.508 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.508 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.508 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761094 s, 138 MB/s 2024-04-17T10:46:57.521 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.521 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.521 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109554 s, 95.7 MB/s 2024-04-17T10:46:57.534 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.534 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.534 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105425 s, 99.5 MB/s 2024-04-17T10:46:57.546 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.547 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.547 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108254 s, 96.9 MB/s 2024-04-17T10:46:57.560 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.560 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.560 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011233 s, 93.3 MB/s 2024-04-17T10:46:57.574 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.574 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.575 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119693 s, 87.6 MB/s 2024-04-17T10:46:57.587 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.587 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.587 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104955 s, 99.9 MB/s 2024-04-17T10:46:57.599 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.599 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.599 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010332 s, 101 MB/s 2024-04-17T10:46:57.613 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.613 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.613 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117011 s, 89.6 MB/s 2024-04-17T10:46:57.629 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.630 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.630 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144675 s, 72.5 MB/s 2024-04-17T10:46:57.645 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.645 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.645 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131749 s, 79.6 MB/s 2024-04-17T10:46:57.660 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.660 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.660 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122036 s, 85.9 MB/s 2024-04-17T10:46:57.675 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.676 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.676 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127027 s, 82.5 MB/s 2024-04-17T10:46:57.687 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.687 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.687 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845862 s, 124 MB/s 2024-04-17T10:46:57.701 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.701 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.701 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010813 s, 97.0 MB/s 2024-04-17T10:46:57.713 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.714 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.714 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00956435 s, 110 MB/s 2024-04-17T10:46:57.729 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.729 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.729 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135502 s, 77.4 MB/s 2024-04-17T10:46:57.742 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.742 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.743 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108544 s, 96.6 MB/s 2024-04-17T10:46:57.754 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.755 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.755 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00997276 s, 105 MB/s 2024-04-17T10:46:57.765 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.766 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.766 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00868962 s, 121 MB/s 2024-04-17T10:46:57.778 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.778 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.778 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104786 s, 100 MB/s 2024-04-17T10:46:57.792 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.792 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.792 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011711 s, 89.5 MB/s 2024-04-17T10:46:57.806 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.806 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.806 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112349 s, 93.3 MB/s 2024-04-17T10:46:57.820 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.820 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.820 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111306 s, 94.2 MB/s 2024-04-17T10:46:57.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:57.821 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:46:57.821 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:46:57.821 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.23 2024-04-17T10:46:57.821 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:46:57.821 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:46:57.821 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:46:57.821 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:46:57.887 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.888 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.888 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108276 s, 96.8 MB/s 2024-04-17T10:46:57.902 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.902 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.902 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120334 s, 87.1 MB/s 2024-04-17T10:46:57.919 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.919 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.919 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014677 s, 71.4 MB/s 2024-04-17T10:46:57.941 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.942 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.942 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011966 s, 87.6 MB/s 2024-04-17T10:46:57.955 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.955 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.955 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110523 s, 94.9 MB/s 2024-04-17T10:46:57.964 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.964 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.964 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756336 s, 139 MB/s 2024-04-17T10:46:57.976 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.976 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.976 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00888546 s, 118 MB/s 2024-04-17T10:46:57.991 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:57.991 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:57.991 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127157 s, 82.5 MB/s 2024-04-17T10:46:58.001 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.002 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.002 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008794 s, 119 MB/s 2024-04-17T10:46:58.014 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.014 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.014 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103623 s, 101 MB/s 2024-04-17T10:46:58.028 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.028 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.028 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116349 s, 90.1 MB/s 2024-04-17T10:46:58.042 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.043 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.043 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011992 s, 87.4 MB/s 2024-04-17T10:46:58.055 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.055 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.055 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103745 s, 101 MB/s 2024-04-17T10:46:58.067 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.068 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.068 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100865 s, 104 MB/s 2024-04-17T10:46:58.077 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.077 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.077 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074299 s, 141 MB/s 2024-04-17T10:46:58.091 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.091 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.091 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114084 s, 91.9 MB/s 2024-04-17T10:46:58.103 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.103 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.103 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00984056 s, 107 MB/s 2024-04-17T10:46:58.115 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.115 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.115 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101746 s, 103 MB/s 2024-04-17T10:46:58.124 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.125 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.125 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766245 s, 137 MB/s 2024-04-17T10:46:58.134 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.134 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.135 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764003 s, 137 MB/s 2024-04-17T10:46:58.149 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.149 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.149 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121687 s, 86.2 MB/s 2024-04-17T10:46:58.162 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.163 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.163 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115731 s, 90.6 MB/s 2024-04-17T10:46:58.175 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.175 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.175 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103258 s, 102 MB/s 2024-04-17T10:46:58.187 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.187 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.187 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00864081 s, 121 MB/s 2024-04-17T10:46:58.200 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.200 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.200 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120761 s, 86.8 MB/s 2024-04-17T10:46:58.213 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.214 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.214 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107754 s, 97.3 MB/s 2024-04-17T10:46:58.227 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.227 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.227 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111792 s, 93.8 MB/s 2024-04-17T10:46:58.240 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.240 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.240 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113172 s, 92.7 MB/s 2024-04-17T10:46:58.254 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.254 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.254 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115394 s, 90.9 MB/s 2024-04-17T10:46:58.266 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.266 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.266 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100569 s, 104 MB/s 2024-04-17T10:46:58.280 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.280 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.280 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115733 s, 90.6 MB/s 2024-04-17T10:46:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115022 s, 91.2 MB/s 2024-04-17T10:46:58.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:58.295 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:46:58.296 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:46:58.296 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.24 2024-04-17T10:46:58.296 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:46:58.296 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:46:58.296 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:46:58.296 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:46:58.366 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.367 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.367 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115406 s, 90.9 MB/s 2024-04-17T10:46:58.382 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.383 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.383 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012953 s, 81.0 MB/s 2024-04-17T10:46:58.397 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.397 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.397 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110431 s, 95.0 MB/s 2024-04-17T10:46:58.411 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.411 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.411 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111102 s, 94.4 MB/s 2024-04-17T10:46:58.425 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.425 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.425 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115899 s, 90.5 MB/s 2024-04-17T10:46:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00931008 s, 113 MB/s 2024-04-17T10:46:58.448 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.449 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.449 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0087594 s, 120 MB/s 2024-04-17T10:46:58.462 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.462 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.462 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011034 s, 95.0 MB/s 2024-04-17T10:46:58.474 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.474 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.474 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00892755 s, 117 MB/s 2024-04-17T10:46:58.485 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.485 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.485 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00888974 s, 118 MB/s 2024-04-17T10:46:58.499 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.499 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.499 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113101 s, 92.7 MB/s 2024-04-17T10:46:58.513 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.514 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.514 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117282 s, 89.4 MB/s 2024-04-17T10:46:58.525 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.525 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.525 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00903831 s, 116 MB/s 2024-04-17T10:46:58.539 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.540 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.540 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115325 s, 90.9 MB/s 2024-04-17T10:46:58.554 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.554 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.554 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116336 s, 90.1 MB/s 2024-04-17T10:46:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103526 s, 101 MB/s 2024-04-17T10:46:58.581 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.581 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.581 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116553 s, 90.0 MB/s 2024-04-17T10:46:58.596 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.596 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.596 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116728 s, 89.8 MB/s 2024-04-17T10:46:58.610 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.610 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.610 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114756 s, 91.4 MB/s 2024-04-17T10:46:58.624 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.624 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.624 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115199 s, 91.0 MB/s 2024-04-17T10:46:58.635 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.635 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.635 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00869487 s, 121 MB/s 2024-04-17T10:46:58.649 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.649 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.649 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111059 s, 94.4 MB/s 2024-04-17T10:46:58.660 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.660 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.661 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859545 s, 122 MB/s 2024-04-17T10:46:58.674 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.674 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.674 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010277 s, 102 MB/s 2024-04-17T10:46:58.687 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.687 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.687 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111784 s, 93.8 MB/s 2024-04-17T10:46:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110897 s, 94.6 MB/s 2024-04-17T10:46:58.712 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.712 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.712 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849103 s, 123 MB/s 2024-04-17T10:46:58.727 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.727 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.727 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118869 s, 88.2 MB/s 2024-04-17T10:46:58.740 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.740 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.740 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107191 s, 97.8 MB/s 2024-04-17T10:46:58.754 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.754 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.754 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113095 s, 92.7 MB/s 2024-04-17T10:46:58.768 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.768 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.768 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111808 s, 93.8 MB/s 2024-04-17T10:46:58.781 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.781 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.781 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010555 s, 99.3 MB/s 2024-04-17T10:46:58.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:58.782 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:46:58.782 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:46:58.782 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.25 2024-04-17T10:46:58.782 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:46:58.782 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:46:58.782 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:46:58.782 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:46:58.848 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.848 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.849 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115502 s, 90.8 MB/s 2024-04-17T10:46:58.863 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.863 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.863 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115397 s, 90.9 MB/s 2024-04-17T10:46:58.874 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.874 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.874 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854734 s, 123 MB/s 2024-04-17T10:46:58.888 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.888 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.888 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106815 s, 98.2 MB/s 2024-04-17T10:46:58.902 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.902 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.902 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110687 s, 94.7 MB/s 2024-04-17T10:46:58.916 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.916 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.916 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116178 s, 90.3 MB/s 2024-04-17T10:46:58.942 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.943 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.943 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108182 s, 96.9 MB/s 2024-04-17T10:46:58.953 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.953 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.953 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814463 s, 129 MB/s 2024-04-17T10:46:58.966 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.967 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.967 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108717 s, 96.5 MB/s 2024-04-17T10:46:58.980 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.980 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.980 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107341 s, 97.7 MB/s 2024-04-17T10:46:58.991 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:58.991 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:58.991 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825294 s, 127 MB/s 2024-04-17T10:46:59.004 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.004 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.004 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100379 s, 104 MB/s 2024-04-17T10:46:59.017 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.017 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.017 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109586 s, 95.7 MB/s 2024-04-17T10:46:59.028 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.028 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.028 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00868304 s, 121 MB/s 2024-04-17T10:46:59.038 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.039 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.039 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773316 s, 136 MB/s 2024-04-17T10:46:59.050 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.051 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.051 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100226 s, 105 MB/s 2024-04-17T10:46:59.060 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.060 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.060 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758709 s, 138 MB/s 2024-04-17T10:46:59.070 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.071 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.071 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798216 s, 131 MB/s 2024-04-17T10:46:59.084 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.084 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.084 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112551 s, 93.2 MB/s 2024-04-17T10:46:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124218 s, 84.4 MB/s 2024-04-17T10:46:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123046 s, 85.2 MB/s 2024-04-17T10:46:59.125 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.125 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.125 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00967618 s, 108 MB/s 2024-04-17T10:46:59.137 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.137 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.138 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00997687 s, 105 MB/s 2024-04-17T10:46:59.152 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.152 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.152 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011702 s, 89.6 MB/s 2024-04-17T10:46:59.163 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.164 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.164 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00948174 s, 111 MB/s 2024-04-17T10:46:59.173 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.173 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.173 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751453 s, 140 MB/s 2024-04-17T10:46:59.186 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.186 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.186 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107717 s, 97.3 MB/s 2024-04-17T10:46:59.199 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.199 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.199 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104682 s, 100 MB/s 2024-04-17T10:46:59.211 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.211 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.211 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0096859 s, 108 MB/s 2024-04-17T10:46:59.224 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.224 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.224 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108189 s, 96.9 MB/s 2024-04-17T10:46:59.237 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.237 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.237 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111567 s, 94.0 MB/s 2024-04-17T10:46:59.247 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.247 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.247 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740495 s, 142 MB/s 2024-04-17T10:46:59.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:59.248 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:46:59.248 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:46:59.248 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.26 2024-04-17T10:46:59.248 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:46:59.248 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:46:59.248 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:46:59.248 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:46:59.308 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.308 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.308 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799743 s, 131 MB/s 2024-04-17T10:46:59.318 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.318 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.319 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780654 s, 134 MB/s 2024-04-17T10:46:59.332 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.332 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.332 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113915 s, 92.0 MB/s 2024-04-17T10:46:59.344 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.344 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.344 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00975415 s, 108 MB/s 2024-04-17T10:46:59.353 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.354 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.354 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743669 s, 141 MB/s 2024-04-17T10:46:59.365 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.365 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.365 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00945081 s, 111 MB/s 2024-04-17T10:46:59.377 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.377 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.378 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101852 s, 103 MB/s 2024-04-17T10:46:59.390 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.390 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.390 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103069 s, 102 MB/s 2024-04-17T10:46:59.402 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.402 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.402 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0094922 s, 110 MB/s 2024-04-17T10:46:59.411 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.411 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.411 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745345 s, 141 MB/s 2024-04-17T10:46:59.427 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.427 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.427 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127285 s, 82.4 MB/s 2024-04-17T10:46:59.439 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103815 s, 101 MB/s 2024-04-17T10:46:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00993302 s, 106 MB/s 2024-04-17T10:46:59.464 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.464 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.464 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101381 s, 103 MB/s 2024-04-17T10:46:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105316 s, 99.6 MB/s 2024-04-17T10:46:59.489 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.489 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.489 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00972397 s, 108 MB/s 2024-04-17T10:46:59.500 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.500 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.500 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00937222 s, 112 MB/s 2024-04-17T10:46:59.510 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.510 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.510 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752966 s, 139 MB/s 2024-04-17T10:46:59.522 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.522 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.522 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102621 s, 102 MB/s 2024-04-17T10:46:59.534 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.534 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.534 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00968437 s, 108 MB/s 2024-04-17T10:46:59.547 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.547 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.548 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108329 s, 96.8 MB/s 2024-04-17T10:46:59.560 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.560 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.560 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105751 s, 99.2 MB/s 2024-04-17T10:46:59.572 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.572 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.572 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101422 s, 103 MB/s 2024-04-17T10:46:59.582 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.582 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.582 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798617 s, 131 MB/s 2024-04-17T10:46:59.594 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.594 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.594 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00940272 s, 112 MB/s 2024-04-17T10:46:59.606 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.606 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.606 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100822 s, 104 MB/s 2024-04-17T10:46:59.619 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.620 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.620 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110344 s, 95.0 MB/s 2024-04-17T10:46:59.632 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.632 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.632 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107328 s, 97.7 MB/s 2024-04-17T10:46:59.645 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.646 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.646 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110422 s, 95.0 MB/s 2024-04-17T10:46:59.658 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.659 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.659 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110159 s, 95.2 MB/s 2024-04-17T10:46:59.672 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.672 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.672 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116384 s, 90.1 MB/s 2024-04-17T10:46:59.686 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.686 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.686 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106081 s, 98.8 MB/s 2024-04-17T10:46:59.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:46:59.686 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:46:59.686 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:46:59.686 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.27 2024-04-17T10:46:59.687 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:46:59.687 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:46:59.687 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:46:59.687 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:46:59.752 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.752 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.752 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112344 s, 93.3 MB/s 2024-04-17T10:46:59.765 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.765 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.766 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112637 s, 93.1 MB/s 2024-04-17T10:46:59.783 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.783 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.784 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109282 s, 96.0 MB/s 2024-04-17T10:46:59.792 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.792 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.792 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113712 s, 92.2 MB/s 2024-04-17T10:46:59.805 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.805 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.805 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104689 s, 100 MB/s 2024-04-17T10:46:59.820 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.820 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.820 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127583 s, 82.2 MB/s 2024-04-17T10:46:59.829 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.829 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.829 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727955 s, 144 MB/s 2024-04-17T10:46:59.842 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.842 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.842 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106154 s, 98.8 MB/s 2024-04-17T10:46:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109893 s, 95.4 MB/s 2024-04-17T10:46:59.867 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.867 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.867 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.009885 s, 106 MB/s 2024-04-17T10:46:59.877 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.877 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.877 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800183 s, 131 MB/s 2024-04-17T10:46:59.891 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.891 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.891 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111791 s, 93.8 MB/s 2024-04-17T10:46:59.901 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.901 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.901 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793666 s, 132 MB/s 2024-04-17T10:46:59.915 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.915 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.915 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117802 s, 89.0 MB/s 2024-04-17T10:46:59.936 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.936 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.937 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100649 s, 104 MB/s 2024-04-17T10:46:59.947 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.947 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.947 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792388 s, 132 MB/s 2024-04-17T10:46:59.961 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.961 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.961 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123646 s, 84.8 MB/s 2024-04-17T10:46:59.972 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.972 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.972 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821731 s, 128 MB/s 2024-04-17T10:46:59.986 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:46:59.986 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:46:59.986 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115411 s, 90.9 MB/s 2024-04-17T10:47:00.000 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.000 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.000 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111784 s, 93.8 MB/s 2024-04-17T10:47:00.010 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.010 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.010 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786814 s, 133 MB/s 2024-04-17T10:47:00.023 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.023 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.023 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109503 s, 95.8 MB/s 2024-04-17T10:47:00.038 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.038 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.038 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125227 s, 83.7 MB/s 2024-04-17T10:47:00.049 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.049 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.049 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00922086 s, 114 MB/s 2024-04-17T10:47:00.065 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.065 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.065 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137579 s, 76.2 MB/s 2024-04-17T10:47:00.079 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.080 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.080 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120436 s, 87.1 MB/s 2024-04-17T10:47:00.092 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.092 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.092 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105712 s, 99.2 MB/s 2024-04-17T10:47:00.107 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.107 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.107 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124572 s, 84.2 MB/s 2024-04-17T10:47:00.122 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.122 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.122 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012639 s, 83.0 MB/s 2024-04-17T10:47:00.133 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.133 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.133 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850233 s, 123 MB/s 2024-04-17T10:47:00.146 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.147 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.147 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113077 s, 92.7 MB/s 2024-04-17T10:47:00.159 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.160 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.160 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109383 s, 95.9 MB/s 2024-04-17T10:47:00.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:00.161 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:00.161 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:00.161 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.28 2024-04-17T10:47:00.161 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:00.161 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:00.161 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:00.161 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:00.223 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.223 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.223 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738036 s, 142 MB/s 2024-04-17T10:47:00.235 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.236 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.236 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102266 s, 103 MB/s 2024-04-17T10:47:00.247 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.247 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.248 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00990179 s, 106 MB/s 2024-04-17T10:47:00.260 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.261 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.261 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110595 s, 94.8 MB/s 2024-04-17T10:47:00.272 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.272 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.272 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00955116 s, 110 MB/s 2024-04-17T10:47:00.285 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.285 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.285 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107027 s, 98.0 MB/s 2024-04-17T10:47:00.296 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.296 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.296 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00932213 s, 112 MB/s 2024-04-17T10:47:00.305 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.305 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.306 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738853 s, 142 MB/s 2024-04-17T10:47:00.316 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.316 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.316 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00903686 s, 116 MB/s 2024-04-17T10:47:00.329 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.330 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.330 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110905 s, 94.5 MB/s 2024-04-17T10:47:00.342 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.342 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.342 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108863 s, 96.3 MB/s 2024-04-17T10:47:00.353 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.353 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.354 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00946121 s, 111 MB/s 2024-04-17T10:47:00.365 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.365 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.366 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100506 s, 104 MB/s 2024-04-17T10:47:00.377 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.377 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.377 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00992357 s, 106 MB/s 2024-04-17T10:47:00.391 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.391 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.391 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114407 s, 91.7 MB/s 2024-04-17T10:47:00.402 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.402 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.403 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00973602 s, 108 MB/s 2024-04-17T10:47:00.412 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.412 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.412 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070277 s, 149 MB/s 2024-04-17T10:47:00.425 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.425 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.425 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011139 s, 94.1 MB/s 2024-04-17T10:47:00.436 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.436 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.436 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00909525 s, 115 MB/s 2024-04-17T10:47:00.450 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.450 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.450 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121935 s, 86.0 MB/s 2024-04-17T10:47:00.462 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.462 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.462 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00986648 s, 106 MB/s 2024-04-17T10:47:00.475 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.475 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.475 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011269 s, 93.0 MB/s 2024-04-17T10:47:00.488 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.488 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.488 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107075 s, 97.9 MB/s 2024-04-17T10:47:00.500 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.500 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.500 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0098865 s, 106 MB/s 2024-04-17T10:47:00.515 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.515 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.515 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00946921 s, 111 MB/s 2024-04-17T10:47:00.524 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.524 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.524 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010131 s, 104 MB/s 2024-04-17T10:47:00.535 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.535 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.536 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00966065 s, 109 MB/s 2024-04-17T10:47:00.551 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.551 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.551 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127488 s, 82.2 MB/s 2024-04-17T10:47:00.564 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.565 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.565 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112274 s, 93.4 MB/s 2024-04-17T10:47:00.578 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.578 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.578 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112916 s, 92.9 MB/s 2024-04-17T10:47:00.592 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.593 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.593 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120977 s, 86.7 MB/s 2024-04-17T10:47:00.606 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.606 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.606 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108271 s, 96.8 MB/s 2024-04-17T10:47:00.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:00.608 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:00.608 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:00.608 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.29 2024-04-17T10:47:00.609 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:00.609 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:00.609 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:00.609 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:00.675 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.675 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.675 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133271 s, 78.7 MB/s 2024-04-17T10:47:00.691 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.691 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.691 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132493 s, 79.1 MB/s 2024-04-17T10:47:00.705 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.705 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.705 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011905 s, 88.1 MB/s 2024-04-17T10:47:00.721 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.721 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.721 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137069 s, 76.5 MB/s 2024-04-17T10:47:00.735 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.735 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.735 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113934 s, 92.0 MB/s 2024-04-17T10:47:00.744 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.745 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.745 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757529 s, 138 MB/s 2024-04-17T10:47:00.754 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.754 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.754 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759281 s, 138 MB/s 2024-04-17T10:47:00.765 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.766 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.766 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00906732 s, 116 MB/s 2024-04-17T10:47:00.775 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.775 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.775 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772225 s, 136 MB/s 2024-04-17T10:47:00.791 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.792 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.792 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140833 s, 74.5 MB/s 2024-04-17T10:47:00.807 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.807 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.807 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135889 s, 77.2 MB/s 2024-04-17T10:47:00.818 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.818 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.818 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865143 s, 121 MB/s 2024-04-17T10:47:00.831 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.831 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.831 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109216 s, 96.0 MB/s 2024-04-17T10:47:00.845 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.845 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.845 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114894 s, 91.3 MB/s 2024-04-17T10:47:00.858 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.858 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.858 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011276 s, 93.0 MB/s 2024-04-17T10:47:00.873 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.873 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.873 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116837 s, 89.7 MB/s 2024-04-17T10:47:00.887 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.887 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.887 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113191 s, 92.6 MB/s 2024-04-17T10:47:00.903 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.903 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.903 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124421 s, 84.3 MB/s 2024-04-17T10:47:00.918 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.919 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.919 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134627 s, 77.9 MB/s 2024-04-17T10:47:00.940 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.940 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.940 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120547 s, 87.0 MB/s 2024-04-17T10:47:00.956 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.956 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.956 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132459 s, 79.2 MB/s 2024-04-17T10:47:00.970 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.970 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.970 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118963 s, 88.1 MB/s 2024-04-17T10:47:00.984 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.984 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:00.984 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115486 s, 90.8 MB/s 2024-04-17T10:47:00.999 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:00.999 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.000 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125125 s, 83.8 MB/s 2024-04-17T10:47:01.012 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.012 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.012 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100655 s, 104 MB/s 2024-04-17T10:47:01.027 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.027 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.027 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123654 s, 84.8 MB/s 2024-04-17T10:47:01.041 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.041 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.041 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107113 s, 97.9 MB/s 2024-04-17T10:47:01.056 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.056 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.056 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123511 s, 84.9 MB/s 2024-04-17T10:47:01.071 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.071 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.071 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121943 s, 86.0 MB/s 2024-04-17T10:47:01.088 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.088 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.088 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140855 s, 74.4 MB/s 2024-04-17T10:47:01.100 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.101 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.101 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.009983 s, 105 MB/s 2024-04-17T10:47:01.115 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.115 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.115 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116347 s, 90.1 MB/s 2024-04-17T10:47:01.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:01.116 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:01.116 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:01.116 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.30 2024-04-17T10:47:01.116 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:01.116 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:01.116 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:01.116 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.183 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.183 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116093 s, 90.3 MB/s 2024-04-17T10:47:01.197 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.197 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.197 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113701 s, 92.2 MB/s 2024-04-17T10:47:01.214 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.214 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.214 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132464 s, 79.2 MB/s 2024-04-17T10:47:01.229 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.229 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.229 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127698 s, 82.1 MB/s 2024-04-17T10:47:01.243 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.243 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.243 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108859 s, 96.3 MB/s 2024-04-17T10:47:01.258 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.258 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.259 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126209 s, 83.1 MB/s 2024-04-17T10:47:01.273 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.273 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.273 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108692 s, 96.5 MB/s 2024-04-17T10:47:01.287 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.287 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.287 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110983 s, 94.5 MB/s 2024-04-17T10:47:01.304 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.304 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.304 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146557 s, 71.5 MB/s 2024-04-17T10:47:01.320 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.320 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.320 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116885 s, 89.7 MB/s 2024-04-17T10:47:01.337 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.337 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.337 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140049 s, 74.9 MB/s 2024-04-17T10:47:01.353 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.353 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.353 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123861 s, 84.7 MB/s 2024-04-17T10:47:01.368 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.368 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.368 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123828 s, 84.7 MB/s 2024-04-17T10:47:01.385 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.385 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.385 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123886 s, 84.6 MB/s 2024-04-17T10:47:01.398 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.398 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.399 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111448 s, 94.1 MB/s 2024-04-17T10:47:01.419 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.419 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.419 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148655 s, 70.5 MB/s 2024-04-17T10:47:01.433 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.433 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.433 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135788 s, 77.2 MB/s 2024-04-17T10:47:01.446 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.446 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.446 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107506 s, 97.5 MB/s 2024-04-17T10:47:01.461 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.461 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.461 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121207 s, 86.5 MB/s 2024-04-17T10:47:01.477 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.477 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.477 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138254 s, 75.8 MB/s 2024-04-17T10:47:01.492 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.492 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.492 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012789 s, 82.0 MB/s 2024-04-17T10:47:01.509 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.509 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.509 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138039 s, 76.0 MB/s 2024-04-17T10:47:01.526 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.526 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.526 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141003 s, 74.4 MB/s 2024-04-17T10:47:01.540 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.540 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.540 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111924 s, 93.7 MB/s 2024-04-17T10:47:01.556 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.556 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.557 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013139 s, 79.8 MB/s 2024-04-17T10:47:01.572 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.572 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.572 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125758 s, 83.4 MB/s 2024-04-17T10:47:01.588 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.588 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.588 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131979 s, 79.5 MB/s 2024-04-17T10:47:01.604 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.605 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.605 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132482 s, 79.1 MB/s 2024-04-17T10:47:01.620 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.620 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.621 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130783 s, 80.2 MB/s 2024-04-17T10:47:01.637 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.637 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.637 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136142 s, 77.0 MB/s 2024-04-17T10:47:01.656 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.656 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.656 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146681 s, 71.5 MB/s 2024-04-17T10:47:01.674 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.674 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.674 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146776 s, 71.4 MB/s 2024-04-17T10:47:01.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:01.676 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:01.676 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:01.676 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.31 2024-04-17T10:47:01.676 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:01.676 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:01.676 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:01.676 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:01.741 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.741 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.741 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011945 s, 87.8 MB/s 2024-04-17T10:47:01.755 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.755 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.755 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110738 s, 94.7 MB/s 2024-04-17T10:47:01.768 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.769 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.769 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110852 s, 94.6 MB/s 2024-04-17T10:47:01.782 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.782 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.782 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101524 s, 103 MB/s 2024-04-17T10:47:01.796 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.796 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.796 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121317 s, 86.4 MB/s 2024-04-17T10:47:01.811 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.811 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.811 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124324 s, 84.3 MB/s 2024-04-17T10:47:01.824 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.824 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.824 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00994908 s, 105 MB/s 2024-04-17T10:47:01.839 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.839 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.839 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130573 s, 80.3 MB/s 2024-04-17T10:47:01.852 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.852 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.852 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103803 s, 101 MB/s 2024-04-17T10:47:01.868 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.868 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.868 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130767 s, 80.2 MB/s 2024-04-17T10:47:01.881 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.881 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.881 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00984818 s, 106 MB/s 2024-04-17T10:47:01.896 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.896 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.896 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121884 s, 86.0 MB/s 2024-04-17T10:47:01.912 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.912 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.912 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133982 s, 78.3 MB/s 2024-04-17T10:47:01.926 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.927 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.927 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114026 s, 92.0 MB/s 2024-04-17T10:47:01.951 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.952 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.952 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01185 s, 88.5 MB/s 2024-04-17T10:47:01.964 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.964 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.964 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101806 s, 103 MB/s 2024-04-17T10:47:01.979 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.979 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.979 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120489 s, 87.0 MB/s 2024-04-17T10:47:01.993 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:01.993 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:01.993 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113392 s, 92.5 MB/s 2024-04-17T10:47:02.007 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:02.007 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:02.008 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119375 s, 87.8 MB/s 2024-04-17T10:47:02.023 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:02.023 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:02.023 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128019 s, 81.9 MB/s 2024-04-17T10:47:02.038 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:02.038 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:02.038 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123783 s, 84.7 MB/s 2024-04-17T10:47:02.053 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:02.053 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:02.053 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125558 s, 83.5 MB/s 2024-04-17T10:47:02.068 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:02.068 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:02.068 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117285 s, 89.4 MB/s 2024-04-17T10:47:02.083 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:02.083 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:02.083 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126491 s, 82.9 MB/s 2024-04-17T10:47:02.099 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:02.099 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:02.099 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121466 s, 86.3 MB/s 2024-04-17T10:47:02.114 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:02.114 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:02.114 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128092 s, 81.9 MB/s 2024-04-17T10:47:02.129 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:02.129 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:02.129 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011981 s, 87.5 MB/s 2024-04-17T10:47:02.143 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:02.143 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:02.144 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120472 s, 87.0 MB/s 2024-04-17T10:47:02.159 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:02.159 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:02.159 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012612 s, 83.1 MB/s 2024-04-17T10:47:02.174 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:02.174 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:02.174 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128129 s, 81.8 MB/s 2024-04-17T10:47:02.190 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:02.190 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:02.190 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135567 s, 77.3 MB/s 2024-04-17T10:47:02.206 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:02.206 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:02.206 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130074 s, 80.6 MB/s 2024-04-17T10:47:02.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:02.208 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /scrub_control_test_path recursive 2024-04-17T10:47:02.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.379+0000 7fc0b5ac7640 1 Processor -- start 2024-04-17T10:47:02.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.379+0000 7fc0b5ac7640 1 -- start start 2024-04-17T10:47:02.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.379+0000 7fc0b5ac7640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc0b015e020 0x7fc0b00cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.379+0000 7fc0b5ac7640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc0b00d1fd0 0x7fc0b00d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.379+0000 7fc0b5ac7640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0b0074800 con 0x7fc0b00d4510 2024-04-17T10:47:02.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.379+0000 7fc0b5ac7640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0b00cf550 con 0x7fc0b00d1fd0 2024-04-17T10:47:02.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.379+0000 7fc0b5ac7640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0b00cff20 con 0x7fc0b015e020 2024-04-17T10:47:02.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.379+0000 7fc0affff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc0b00d1fd0 0x7fc0b00d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.379+0000 7fc0affff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc0b00d1fd0 0x7fc0b00d23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58178/0 (socket says 172.21.15.83:58178) 2024-04-17T10:47:02.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.379+0000 7fc0affff640 1 -- 172.21.15.83:0/2093897993 learned_addr learned my addr 172.21.15.83:0/2093897993 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:02.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.379+0000 7fc0affff640 1 -- 172.21.15.83:0/2093897993 >> v2:172.21.15.167:3300/0 conn(0x7fc0b015e020 msgr2=0x7fc0b00cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:02.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.379+0000 7fc0affff640 1 --2- 172.21.15.83:0/2093897993 >> v2:172.21.15.167:3300/0 conn(0x7fc0b015e020 0x7fc0b00cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.379+0000 7fc0affff640 1 -- 172.21.15.83:0/2093897993 >> v1:172.21.15.167:6789/0 conn(0x7fc0b00d4510 legacy=0x7fc0b00d17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:02.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.379+0000 7fc0affff640 1 -- 172.21.15.83:0/2093897993 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b00d2920 con 0x7fc0b00d1fd0 2024-04-17T10:47:02.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.383+0000 7fc0affff640 1 --2- 172.21.15.83:0/2093897993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0b00d1fd0 0x7fc0b00d23e0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fc0b00b65e0 tx=0x7fc0a801c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82386a6582c302f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:02.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.383+0000 7fc0ae7fc640 1 -- 172.21.15.83:0/2093897993 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a80a5760 con 0x7fc0b00d1fd0 2024-04-17T10:47:02.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.383+0000 7fc0ae7fc640 1 -- 172.21.15.83:0/2093897993 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a80a5900 con 0x7fc0b00d1fd0 2024-04-17T10:47:02.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.383+0000 7fc0ae7fc640 1 -- 172.21.15.83:0/2093897993 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a80a5cd0 con 0x7fc0b00d1fd0 2024-04-17T10:47:02.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.383+0000 7fc0b5ac7640 1 -- 172.21.15.83:0/2093897993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0b00d1fd0 msgr2=0x7fc0b00d23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:02.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.383+0000 7fc0b5ac7640 1 --2- 172.21.15.83:0/2093897993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0b00d1fd0 0x7fc0b00d23e0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fc0b00b65e0 tx=0x7fc0a801c9b0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.383+0000 7fc0b5ac7640 1 -- 172.21.15.83:0/2093897993 shutdown_connections 2024-04-17T10:47:02.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.383+0000 7fc0b5ac7640 1 --2- 172.21.15.83:0/2093897993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0b00d1fd0 0x7fc0b00d23e0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.383+0000 7fc0b5ac7640 1 --2- 172.21.15.83:0/2093897993 >> v2:172.21.15.167:3300/0 conn(0x7fc0b015e020 0x7fc0b00cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.383+0000 7fc0b5ac7640 1 -- 172.21.15.83:0/2093897993 >> 172.21.15.83:0/2093897993 conn(0x7fc0b00d39f0 msgr2=0x7fc0b00d3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:02.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.383+0000 7fc0b5ac7640 1 -- 172.21.15.83:0/2093897993 shutdown_connections 2024-04-17T10:47:02.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.383+0000 7fc0b5ac7640 1 -- 172.21.15.83:0/2093897993 wait complete. 2024-04-17T10:47:02.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.383+0000 7fc0b5ac7640 1 Processor -- start 2024-04-17T10:47:02.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.383+0000 7fc0b5ac7640 1 -- start start 2024-04-17T10:47:02.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.383+0000 7fc0b5ac7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0b00d1fd0 0x7fc0b00b8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.383+0000 7fc0b5ac7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0b00d4510 0x7fc0b00caac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.383+0000 7fc0b5ac7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0b015e020 0x7fc0b00b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.383+0000 7fc0b5ac7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b006ba80 con 0x7fc0b00d4510 2024-04-17T10:47:02.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.383+0000 7fc0b5ac7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b006b900 con 0x7fc0b015e020 2024-04-17T10:47:02.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.383+0000 7fc0b5ac7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b006bc00 con 0x7fc0b00d1fd0 2024-04-17T10:47:02.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.383+0000 7fc0aeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0b00d4510 0x7fc0b00caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.387+0000 7fc0aeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0b00d4510 0x7fc0b00caac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58188/0 (socket says 172.21.15.83:58188) 2024-04-17T10:47:02.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.387+0000 7fc0aeffd640 1 -- 172.21.15.83:0/3339753541 learned_addr learned my addr 172.21.15.83:0/3339753541 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:02.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.387+0000 7fc0affff640 1 --2- 172.21.15.83:0/3339753541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0b015e020 0x7fc0b00b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.387+0000 7fc0af7fe640 1 --2- 172.21.15.83:0/3339753541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0b00d1fd0 0x7fc0b00b8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.387+0000 7fc0aeffd640 1 -- 172.21.15.83:0/3339753541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0b00d1fd0 msgr2=0x7fc0b00b8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:02.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.387+0000 7fc0aeffd640 1 --2- 172.21.15.83:0/3339753541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0b00d1fd0 0x7fc0b00b8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.387+0000 7fc0aeffd640 1 -- 172.21.15.83:0/3339753541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0b015e020 msgr2=0x7fc0b00b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:02.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.387+0000 7fc0aeffd640 1 --2- 172.21.15.83:0/3339753541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0b015e020 0x7fc0b00b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.387+0000 7fc0aeffd640 1 -- 172.21.15.83:0/3339753541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b00cb210 con 0x7fc0b00d4510 2024-04-17T10:47:02.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.387+0000 7fc0affff640 1 --2- 172.21.15.83:0/3339753541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0b015e020 0x7fc0b00b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:02.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.387+0000 7fc0aeffd640 1 --2- 172.21.15.83:0/3339753541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0b00d4510 0x7fc0b00caac0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fc0a400b410 tx=0x7fc0a400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:02.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.387+0000 7fc097fff640 1 -- 172.21.15.83:0/3339753541 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4015020 con 0x7fc0b00d4510 2024-04-17T10:47:02.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.387+0000 7fc097fff640 1 -- 172.21.15.83:0/3339753541 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a4011830 con 0x7fc0b00d4510 2024-04-17T10:47:02.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.387+0000 7fc0b5ac7640 1 -- 172.21.15.83:0/3339753541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0b00cb4f0 con 0x7fc0b00d4510 2024-04-17T10:47:02.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.387+0000 7fc097fff640 1 -- 172.21.15.83:0/3339753541 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4011c20 con 0x7fc0b00d4510 2024-04-17T10:47:02.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.387+0000 7fc0b5ac7640 1 -- 172.21.15.83:0/3339753541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b00cbab0 con 0x7fc0b00d4510 2024-04-17T10:47:02.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.387+0000 7fc097fff640 1 -- 172.21.15.83:0/3339753541 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc0a4018020 con 0x7fc0b00d4510 2024-04-17T10:47:02.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.387+0000 7fc097fff640 1 --2- 172.21.15.83:0/3339753541 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc09c066f60 0x7fc09c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.387+0000 7fc097fff640 1 -- 172.21.15.83:0/3339753541 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fc0a4084f70 con 0x7fc0b00d4510 2024-04-17T10:47:02.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.387+0000 7fc0af7fe640 1 --2- 172.21.15.83:0/3339753541 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc09c066f60 0x7fc09c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.387+0000 7fc0af7fe640 1 --2- 172.21.15.83:0/3339753541 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc09c066f60 0x7fc09c069420 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc0a000a810 tx=0x7fc0a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:02.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.387+0000 7fc0b8c34000 1 Processor -- start 2024-04-17T10:47:02.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.387+0000 7fc0b8c34000 1 -- start start 2024-04-17T10:47:02.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0b8c34000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d3fd6703f0 0x55d3fd6707d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0b8c34000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fd670d10 0x55d3fd626fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0b8c34000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3fd627520 0x55d3fd627900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0b8c34000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3fd474d50 con 0x55d3fd627520 2024-04-17T10:47:02.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0b8c34000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3fd3fce40 con 0x55d3fd670d10 2024-04-17T10:47:02.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0b8c34000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d3fd470df0 con 0x55d3fd6703f0 2024-04-17T10:47:02.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0affff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3fd627520 0x55d3fd627900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0affff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3fd627520 0x55d3fd627900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58198/0 (socket says 172.21.15.83:58198) 2024-04-17T10:47:02.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0affff640 1 -- 172.21.15.83:0/1639163838 learned_addr learned my addr 172.21.15.83:0/1639163838 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:02.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0af7fe640 1 --2- 172.21.15.83:0/1639163838 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d3fd6703f0 0x55d3fd6707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0aeffd640 1 --2- 172.21.15.83:0/1639163838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fd670d10 0x55d3fd626fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0affff640 1 -- 172.21.15.83:0/1639163838 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d3fd6703f0 msgr2=0x55d3fd6707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:02.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0affff640 1 --2- 172.21.15.83:0/1639163838 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d3fd6703f0 0x55d3fd6707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0affff640 1 -- 172.21.15.83:0/1639163838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fd670d10 msgr2=0x55d3fd626fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:02.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0affff640 1 --2- 172.21.15.83:0/1639163838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fd670d10 0x55d3fd626fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0affff640 1 -- 172.21.15.83:0/1639163838 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3fd411d90 con 0x55d3fd627520 2024-04-17T10:47:02.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0aeffd640 1 --2- 172.21.15.83:0/1639163838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fd670d10 0x55d3fd626fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:02.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0affff640 1 --2- 172.21.15.83:0/1639163838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3fd627520 0x55d3fd627900 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fc0a80af040 tx=0x7fc0a8004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409cf5297ed4635f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:02.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0957fa640 1 -- 172.21.15.83:0/1639163838 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a80a5960 con 0x55d3fd627520 2024-04-17T10:47:02.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0957fa640 1 -- 172.21.15.83:0/1639163838 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a80a5b00 con 0x55d3fd627520 2024-04-17T10:47:02.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0957fa640 1 -- 172.21.15.83:0/1639163838 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a8009250 con 0x55d3fd627520 2024-04-17T10:47:02.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0b8c34000 1 -- 172.21.15.83:0/1639163838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3fd627520 msgr2=0x55d3fd627900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:02.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0b8c34000 1 --2- 172.21.15.83:0/1639163838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3fd627520 0x55d3fd627900 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fc0a80af040 tx=0x7fc0a8004ac0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0b8c34000 1 -- 172.21.15.83:0/1639163838 shutdown_connections 2024-04-17T10:47:02.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0b8c34000 1 --2- 172.21.15.83:0/1639163838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3fd627520 0x55d3fd627900 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0b8c34000 1 --2- 172.21.15.83:0/1639163838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fd670d10 0x55d3fd626fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0b8c34000 1 --2- 172.21.15.83:0/1639163838 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d3fd6703f0 0x55d3fd6707d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0b8c34000 1 -- 172.21.15.83:0/1639163838 >> 172.21.15.83:0/1639163838 conn(0x55d3fd6234d0 msgr2=0x55d3fd6258c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:02.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0b8c34000 1 -- 172.21.15.83:0/1639163838 shutdown_connections 2024-04-17T10:47:02.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0b8c34000 1 -- 172.21.15.83:0/1639163838 wait complete. 2024-04-17T10:47:02.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0b8c34000 1 -- start start 2024-04-17T10:47:02.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0b8c34000 1 Processor -- start 2024-04-17T10:47:02.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0b8c34000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fd627520 0x55d3fd677540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0b8c34000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3fd6703f0 0x55d3fd677a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0b8c34000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d3fd670d10 0x55d3fd631bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0b8c34000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3fd41a610 con 0x55d3fd6703f0 2024-04-17T10:47:02.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0b8c34000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3fd42f9a0 con 0x55d3fd627520 2024-04-17T10:47:02.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0b8c34000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d3fd479380 con 0x55d3fd670d10 2024-04-17T10:47:02.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0b8c34000 10 client.0 _ll_register_callbacks cb 0x55d3fd5290d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:47:02.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0aeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3fd6703f0 0x55d3fd677a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0aeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3fd6703f0 0x55d3fd677a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58204/0 (socket says 172.21.15.83:58204) 2024-04-17T10:47:02.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0aeffd640 1 -- 172.21.15.83:0/1227617654 learned_addr learned my addr 172.21.15.83:0/1227617654 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:02.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0af7fe640 1 --2- 172.21.15.83:0/1227617654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fd627520 0x55d3fd677540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0affff640 1 --2- 172.21.15.83:0/1227617654 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d3fd670d10 0x55d3fd631bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0aeffd640 1 -- 172.21.15.83:0/1227617654 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d3fd670d10 msgr2=0x55d3fd631bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:02.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0aeffd640 1 --2- 172.21.15.83:0/1227617654 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d3fd670d10 0x55d3fd631bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0aeffd640 1 -- 172.21.15.83:0/1227617654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fd627520 msgr2=0x55d3fd677540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:02.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.391+0000 7fc0aeffd640 1 --2- 172.21.15.83:0/1227617654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fd627520 0x55d3fd677540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc0aeffd640 1 -- 172.21.15.83:0/1227617654 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3fd46fd40 con 0x55d3fd6703f0 2024-04-17T10:47:02.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc0aeffd640 1 --2- 172.21.15.83:0/1227617654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3fd6703f0 0x55d3fd677a80 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4004700 tx=0x7fc0a4055a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22776a7511ff4b79 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:02.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc08e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:47:02.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc08e7fc640 1 -- 172.21.15.83:0/1227617654 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4051f60 con 0x55d3fd6703f0 2024-04-17T10:47:02.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc08e7fc640 1 -- 172.21.15.83:0/1227617654 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fc0a4054d40 con 0x55d3fd6703f0 2024-04-17T10:47:02.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc0b8c34000 1 -- 172.21.15.83:0/1227617654 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc0a80a5b00 con 0x55d3fd6703f0 2024-04-17T10:47:02.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc08e7fc640 1 -- 172.21.15.83:0/1227617654 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0a4052100 con 0x55d3fd6703f0 2024-04-17T10:47:02.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc0b8c34000 10 client.7746 fetch_fsmap learned FSMap version 56 2024-04-17T10:47:02.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc0b8c34000 1 -- 172.21.15.83:0/1227617654 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x55d3fd4f7f00 con 0x55d3fd6703f0 2024-04-17T10:47:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc08e7fc640 1 -- 172.21.15.83:0/1227617654 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a80a5b00 con 0x55d3fd6703f0 2024-04-17T10:47:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc08e7fc640 1 -- 172.21.15.83:0/1227617654 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 56) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc0a4051640 con 0x55d3fd6703f0 2024-04-17T10:47:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc0b8c34000 10 client.7746 fetch_fsmap finished waiting for FSMap version 56 2024-04-17T10:47:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc0b8c34000 10 client.7746 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-17T10:47:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc0b8c34000 20 client.7746 populate_metadata read hostname 'smithi083' 2024-04-17T10:47:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc0b8c34000 1 --2- 172.21.15.83:0/1227617654 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x55d3fd632960 0x55d3fd632d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc0b8c34000 4 client.7746 mds_command: new command op to 7575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:47:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc0b8c34000 1 -- 172.21.15.83:0/1227617654 --> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3fd410a20 con 0x55d3fd632960 2024-04-17T10:47:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc0af7fe640 1 --2- 172.21.15.83:0/1227617654 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x55d3fd632960 0x55d3fd632d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc0af7fe640 1 --2- 172.21.15.83:0/1227617654 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x55d3fd632960 0x55d3fd632d70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52229f12f56a0f57 server_cookie=fcd26279bd38babc in_seq=0 out_seq=0 2024-04-17T10:47:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc08e7fc640 10 client.7746 ms_handle_connect on v2:172.21.15.83:6834/1056175765 2024-04-17T10:47:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc08e7fc640 1 -- 172.21.15.83:0/1227617654 <== mds.0 v2:172.21.15.83:6834/1056175765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d3fd410a20 con 0x55d3fd632960 2024-04-17T10:47:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc08e7fc640 10 client.7746 handle_command_reply: tid=0 2024-04-17T10:47:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc0b8c34000 1 client.7746 shutdown 2024-04-17T10:47:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc0b8c34000 1 -- 172.21.15.83:0/1227617654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3fd6703f0 msgr2=0x55d3fd677a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc0b8c34000 1 --2- 172.21.15.83:0/1227617654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3fd6703f0 0x55d3fd677a80 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4004700 tx=0x7fc0a4055a30 comp rx=0 tx=0).stop 2024-04-17T10:47:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc0b8c34000 1 -- 172.21.15.83:0/1227617654 shutdown_connections 2024-04-17T10:47:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc0b8c34000 1 --2- 172.21.15.83:0/1227617654 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d3fd670d10 0x55d3fd631bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc0b8c34000 1 --2- 172.21.15.83:0/1227617654 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x55d3fd632960 0x55d3fd632d70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc0b8c34000 1 --2- 172.21.15.83:0/1227617654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3fd6703f0 0x55d3fd677a80 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc0b8c34000 1 --2- 172.21.15.83:0/1227617654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fd627520 0x55d3fd677540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc08e7fc640 0 client.7746 ms_handle_reset on v2:172.21.15.83:6834/1056175765 2024-04-17T10:47:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc0b8c34000 1 -- 172.21.15.83:0/1227617654 >> 172.21.15.83:0/1227617654 conn(0x55d3fd6234d0 msgr2=0x55d3fd626150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc0b8c34000 1 -- 172.21.15.83:0/1227617654 shutdown_connections 2024-04-17T10:47:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc0b8c34000 1 -- 172.21.15.83:0/1227617654 wait complete. 2024-04-17T10:47:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.395+0000 7fc0b8c34000 20 client.7746 trim_cache size 0 max 16384 2024-04-17T10:47:02.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.411+0000 7fc0b8c34000 1 Processor -- start 2024-04-17T10:47:02.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.411+0000 7fc0b8c34000 1 -- start start 2024-04-17T10:47:02.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.411+0000 7fc0b8c34000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fd6234d0 0x55d3fd62cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.411+0000 7fc0b8c34000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d3fd627520 0x55d3fd62d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.411+0000 7fc0b8c34000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3fd6703f0 0x55d3fd66f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.411+0000 7fc0b8c34000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b0202d10 con 0x55d3fd6703f0 2024-04-17T10:47:02.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.411+0000 7fc0b8c34000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b00cd3c0 con 0x55d3fd6234d0 2024-04-17T10:47:02.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.411+0000 7fc0b8c34000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b015eeb0 con 0x55d3fd627520 2024-04-17T10:47:02.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.411+0000 7fc0affff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3fd6703f0 0x55d3fd66f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.411+0000 7fc0affff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3fd6703f0 0x55d3fd66f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58212/0 (socket says 172.21.15.83:58212) 2024-04-17T10:47:02.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.411+0000 7fc0affff640 1 -- 172.21.15.83:0/3064303690 learned_addr learned my addr 172.21.15.83:0/3064303690 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:02.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.411+0000 7fc0af7fe640 1 --2- 172.21.15.83:0/3064303690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fd6234d0 0x55d3fd62cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.411+0000 7fc0aeffd640 1 --2- 172.21.15.83:0/3064303690 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d3fd627520 0x55d3fd62d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.411+0000 7fc0affff640 1 -- 172.21.15.83:0/3064303690 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d3fd627520 msgr2=0x55d3fd62d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:02.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.411+0000 7fc0affff640 1 --2- 172.21.15.83:0/3064303690 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d3fd627520 0x55d3fd62d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.411+0000 7fc0affff640 1 -- 172.21.15.83:0/3064303690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fd6234d0 msgr2=0x55d3fd62cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:02.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.411+0000 7fc0affff640 1 --2- 172.21.15.83:0/3064303690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fd6234d0 0x55d3fd62cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.411+0000 7fc0affff640 1 -- 172.21.15.83:0/3064303690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3fd4ce330 con 0x55d3fd6703f0 2024-04-17T10:47:02.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.411+0000 7fc0affff640 1 --2- 172.21.15.83:0/3064303690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3fd6703f0 0x55d3fd66f660 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fc0a801c9f0 tx=0x7fc0a80a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2916064025bcd43 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:02.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.411+0000 7fc094ff9640 1 -- 172.21.15.83:0/3064303690 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a80c7070 con 0x55d3fd6703f0 2024-04-17T10:47:02.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.411+0000 7fc094ff9640 1 -- 172.21.15.83:0/3064303690 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a8003c00 con 0x55d3fd6703f0 2024-04-17T10:47:02.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.411+0000 7fc094ff9640 1 -- 172.21.15.83:0/3064303690 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a80a5760 con 0x55d3fd6703f0 2024-04-17T10:47:02.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.411+0000 7fc0b8c34000 1 -- 172.21.15.83:0/3064303690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3fd6703f0 msgr2=0x55d3fd66f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:02.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.411+0000 7fc0b8c34000 1 --2- 172.21.15.83:0/3064303690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3fd6703f0 0x55d3fd66f660 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fc0a801c9f0 tx=0x7fc0a80a7000 comp rx=0 tx=0).stop 2024-04-17T10:47:02.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.411+0000 7fc0b8c34000 1 -- 172.21.15.83:0/3064303690 shutdown_connections 2024-04-17T10:47:02.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.411+0000 7fc0b8c34000 1 --2- 172.21.15.83:0/3064303690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3fd6703f0 0x55d3fd66f660 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.411+0000 7fc0b8c34000 1 --2- 172.21.15.83:0/3064303690 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d3fd627520 0x55d3fd62d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.411+0000 7fc0b8c34000 1 --2- 172.21.15.83:0/3064303690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fd6234d0 0x55d3fd62cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.411+0000 7fc0b8c34000 1 -- 172.21.15.83:0/3064303690 >> 172.21.15.83:0/3064303690 conn(0x55d3fd6767d0 msgr2=0x55d3fd66ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:02.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.411+0000 7fc0b8c34000 1 -- 172.21.15.83:0/3064303690 shutdown_connections 2024-04-17T10:47:02.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.411+0000 7fc0b8c34000 1 -- 172.21.15.83:0/3064303690 wait complete. 2024-04-17T10:47:02.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.411+0000 7fc0b8c34000 1 -- start start 2024-04-17T10:47:02.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc0b8c34000 1 Processor -- start 2024-04-17T10:47:02.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc0b8c34000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d3fd6234d0 0x55d3fd674540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc0b8c34000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3fd627520 0x55d3fd68dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc0b8c34000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fd6703f0 0x55d3fd68e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc0b8c34000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3fd42bdc0 con 0x55d3fd627520 2024-04-17T10:47:02.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc0b8c34000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b00ccdc0 con 0x55d3fd6703f0 2024-04-17T10:47:02.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc0b8c34000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d3fd474060 con 0x55d3fd6234d0 2024-04-17T10:47:02.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc0b8c34000 10 client.0 _ll_register_callbacks cb 0x7fc0a8009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:47:02.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc0aeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3fd627520 0x55d3fd68dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc0aeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3fd627520 0x55d3fd68dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58220/0 (socket says 172.21.15.83:58220) 2024-04-17T10:47:02.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc0aeffd640 1 -- 172.21.15.83:0/2827404398 learned_addr learned my addr 172.21.15.83:0/2827404398 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:02.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc0af7fe640 1 --2- 172.21.15.83:0/2827404398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d3fd6234d0 0x55d3fd674540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc0affff640 1 --2- 172.21.15.83:0/2827404398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fd6703f0 0x55d3fd68e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc0aeffd640 1 -- 172.21.15.83:0/2827404398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d3fd6234d0 msgr2=0x55d3fd674540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:02.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc0aeffd640 1 --2- 172.21.15.83:0/2827404398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d3fd6234d0 0x55d3fd674540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc0aeffd640 1 -- 172.21.15.83:0/2827404398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fd6703f0 msgr2=0x55d3fd68e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:02.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc0aeffd640 1 --2- 172.21.15.83:0/2827404398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fd6703f0 0x55d3fd68e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc0aeffd640 1 -- 172.21.15.83:0/2827404398 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3fd4415f0 con 0x55d3fd627520 2024-04-17T10:47:02.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc0affff640 1 --2- 172.21.15.83:0/2827404398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fd6703f0 0x55d3fd68e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:02.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc0aeffd640 1 --2- 172.21.15.83:0/2827404398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3fd627520 0x55d3fd68dc00 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fc0a40555f0 tx=0x7fc0a40833e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc439ff5a66a3f7a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:02.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc08e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:47:02.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc08e7fc640 1 -- 172.21.15.83:0/2827404398 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4078060 con 0x55d3fd627520 2024-04-17T10:47:02.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc08e7fc640 1 -- 172.21.15.83:0/2827404398 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fc0a4054f50 con 0x55d3fd627520 2024-04-17T10:47:02.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc0b8c34000 1 -- 172.21.15.83:0/2827404398 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc0a8003c00 con 0x55d3fd627520 2024-04-17T10:47:02.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc08e7fc640 1 -- 172.21.15.83:0/2827404398 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0a4071070 con 0x55d3fd627520 2024-04-17T10:47:02.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc0b8c34000 10 client.7752 fetch_fsmap learned FSMap version 56 2024-04-17T10:47:02.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc0b8c34000 1 -- 172.21.15.83:0/2827404398 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x55d3fd41bfe0 con 0x55d3fd627520 2024-04-17T10:47:02.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc08e7fc640 1 -- 172.21.15.83:0/2827404398 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a8003c00 con 0x55d3fd627520 2024-04-17T10:47:02.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc08e7fc640 1 -- 172.21.15.83:0/2827404398 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 56) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc0a406d0d0 con 0x55d3fd627520 2024-04-17T10:47:02.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc0b8c34000 10 client.7752 fetch_fsmap finished waiting for FSMap version 56 2024-04-17T10:47:02.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc0b8c34000 10 client.7752 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-17T10:47:02.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc0b8c34000 20 client.7752 populate_metadata read hostname 'smithi083' 2024-04-17T10:47:02.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc0b8c34000 1 --2- 172.21.15.83:0/2827404398 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x55d3fd670d10 0x55d3fd6d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc0b8c34000 4 client.7752 mds_command: new command op to 7575 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-17T10:47:02.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc0b8c34000 1 -- 172.21.15.83:0/2827404398 --> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x55d3fd482290 con 0x55d3fd670d10 2024-04-17T10:47:02.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc0af7fe640 1 --2- 172.21.15.83:0/2827404398 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x55d3fd670d10 0x55d3fd6d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc08e7fc640 10 client.7752 ms_handle_connect on v2:172.21.15.83:6834/1056175765 2024-04-17T10:47:02.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.415+0000 7fc0af7fe640 1 --2- 172.21.15.83:0/2827404398 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x55d3fd670d10 0x55d3fd6d1210 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=384dc8851418c27f server_cookie=a0b03c1fa1b8319f in_seq=0 out_seq=0 2024-04-17T10:47:02.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.419+0000 7fc08e7fc640 1 -- 172.21.15.83:0/2827404398 <== mds.0 v2:172.21.15.83:6834/1056175765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d3fd482290 con 0x55d3fd670d10 2024-04-17T10:47:02.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.419+0000 7fc08e7fc640 10 client.7752 handle_command_reply: tid=0 2024-04-17T10:47:02.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.419+0000 7fc0b8c34000 1 client.7752 shutdown 2024-04-17T10:47:02.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.419+0000 7fc0b8c34000 1 -- 172.21.15.83:0/2827404398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3fd627520 msgr2=0x55d3fd68dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:02.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.419+0000 7fc0b8c34000 1 --2- 172.21.15.83:0/2827404398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3fd627520 0x55d3fd68dc00 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fc0a40555f0 tx=0x7fc0a40833e0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.419+0000 7fc0b8c34000 1 -- 172.21.15.83:0/2827404398 shutdown_connections 2024-04-17T10:47:02.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.419+0000 7fc0b8c34000 1 --2- 172.21.15.83:0/2827404398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fd6703f0 0x55d3fd68e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.419+0000 7fc0b8c34000 1 --2- 172.21.15.83:0/2827404398 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x55d3fd670d10 0x55d3fd6d1210 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.419+0000 7fc0b8c34000 1 --2- 172.21.15.83:0/2827404398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3fd627520 0x55d3fd68dc00 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.419+0000 7fc0b8c34000 1 --2- 172.21.15.83:0/2827404398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d3fd6234d0 0x55d3fd674540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.419+0000 7fc0b8c34000 1 -- 172.21.15.83:0/2827404398 >> 172.21.15.83:0/2827404398 conn(0x55d3fd6767d0 msgr2=0x55d3fd66f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:02.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.419+0000 7fc08e7fc640 0 client.7752 ms_handle_reset on v2:172.21.15.83:6834/1056175765 2024-04-17T10:47:02.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.419+0000 7fc0b8c34000 1 -- 172.21.15.83:0/2827404398 shutdown_connections 2024-04-17T10:47:02.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.419+0000 7fc0b8c34000 1 -- 172.21.15.83:0/2827404398 wait complete. 2024-04-17T10:47:02.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.419+0000 7fc0b8c34000 20 client.7752 trim_cache size 0 max 16384 2024-04-17T10:47:02.420 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:47:02.420 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-17T10:47:02.420 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "873d1145-8664-4166-a41c-2540df29ca70", 2024-04-17T10:47:02.420 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-17T10:47:02.420 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:47:02.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.419+0000 7fc0b5ac7640 1 -- 172.21.15.83:0/3339753541 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc09c066f60 msgr2=0x7fc09c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:02.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.419+0000 7fc0b5ac7640 1 --2- 172.21.15.83:0/3339753541 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc09c066f60 0x7fc09c069420 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc0a000a810 tx=0x7fc0a0008040 comp rx=0 tx=0).stop 2024-04-17T10:47:02.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.419+0000 7fc0b5ac7640 1 -- 172.21.15.83:0/3339753541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0b00d4510 msgr2=0x7fc0b00caac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:02.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.419+0000 7fc0b5ac7640 1 --2- 172.21.15.83:0/3339753541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0b00d4510 0x7fc0b00caac0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fc0a400b410 tx=0x7fc0a400b8d0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.419+0000 7fc0b5ac7640 1 -- 172.21.15.83:0/3339753541 shutdown_connections 2024-04-17T10:47:02.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.419+0000 7fc0b5ac7640 1 --2- 172.21.15.83:0/3339753541 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc09c066f60 0x7fc09c069420 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.419+0000 7fc0b5ac7640 1 --2- 172.21.15.83:0/3339753541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0b015e020 0x7fc0b00b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.419+0000 7fc0b5ac7640 1 --2- 172.21.15.83:0/3339753541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0b00d4510 0x7fc0b00caac0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.419+0000 7fc0b5ac7640 1 --2- 172.21.15.83:0/3339753541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0b00d1fd0 0x7fc0b00b8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.419+0000 7fc0b5ac7640 1 -- 172.21.15.83:0/3339753541 >> 172.21.15.83:0/3339753541 conn(0x7fc0b00d39f0 msgr2=0x7fc0b00d3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:02.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.419+0000 7fc0b5ac7640 1 -- 172.21.15.83:0/3339753541 shutdown_connections 2024-04-17T10:47:02.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.419+0000 7fc0b5ac7640 1 -- 172.21.15.83:0/3339753541 wait complete. 2024-04-17T10:47:02.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:02.432 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub abort 2024-04-17T10:47:02.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.575+0000 7fdb7e403640 1 Processor -- start 2024-04-17T10:47:02.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.575+0000 7fdb7e403640 1 -- start start 2024-04-17T10:47:02.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.575+0000 7fdb7e403640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdb7815a8a0 0x7fdb7806e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.575+0000 7fdb7e403640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb780b36f0 0x7fdb7806bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.575+0000 7fdb7e403640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdb7806c230 0x7fdb780cd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.575+0000 7fdb7e403640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb780afbb0 con 0x7fdb7806c230 2024-04-17T10:47:02.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.575+0000 7fdb7e403640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb7806d760 con 0x7fdb7815a8a0 2024-04-17T10:47:02.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.575+0000 7fdb7e403640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7806da40 con 0x7fdb780b36f0 2024-04-17T10:47:02.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.575+0000 7fdb7c9a0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdb7806c230 0x7fdb780cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.575+0000 7fdb7c9a0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdb7806c230 0x7fdb780cd720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58222/0 (socket says 172.21.15.83:58222) 2024-04-17T10:47:02.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.575+0000 7fdb7c9a0640 1 -- 172.21.15.83:0/350469236 learned_addr learned my addr 172.21.15.83:0/350469236 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:02.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.575+0000 7fdb77fff640 1 --2- 172.21.15.83:0/350469236 >> v2:172.21.15.167:3300/0 conn(0x7fdb7815a8a0 0x7fdb7806e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.575+0000 7fdb777fe640 1 --2- 172.21.15.83:0/350469236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb780b36f0 0x7fdb7806bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.575+0000 7fdb7c9a0640 1 -- 172.21.15.83:0/350469236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb780b36f0 msgr2=0x7fdb7806bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:02.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.575+0000 7fdb7c9a0640 1 --2- 172.21.15.83:0/350469236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb780b36f0 0x7fdb7806bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.575+0000 7fdb7c9a0640 1 -- 172.21.15.83:0/350469236 >> v2:172.21.15.167:3300/0 conn(0x7fdb7815a8a0 msgr2=0x7fdb7806e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:02.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.575+0000 7fdb7c9a0640 1 --2- 172.21.15.83:0/350469236 >> v2:172.21.15.167:3300/0 conn(0x7fdb7815a8a0 0x7fdb7806e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.575+0000 7fdb7c9a0640 1 -- 172.21.15.83:0/350469236 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7806dd20 con 0x7fdb7806c230 2024-04-17T10:47:02.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.579+0000 7fdb7c9a0640 1 --2- 172.21.15.83:0/350469236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb7806c230 0x7fdb780cd720 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fdb780b5bc0 tx=0x7fdb7001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2449f45055a132d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:02.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.579+0000 7fdb76ffd640 1 -- 172.21.15.83:0/350469236 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb700a5780 con 0x7fdb7806c230 2024-04-17T10:47:02.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.579+0000 7fdb76ffd640 1 -- 172.21.15.83:0/350469236 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb700a5920 con 0x7fdb7806c230 2024-04-17T10:47:02.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.579+0000 7fdb76ffd640 1 -- 172.21.15.83:0/350469236 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb700a5cf0 con 0x7fdb7806c230 2024-04-17T10:47:02.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.579+0000 7fdb7e403640 1 -- 172.21.15.83:0/350469236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb7806c230 msgr2=0x7fdb780cd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:02.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.579+0000 7fdb7e403640 1 --2- 172.21.15.83:0/350469236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb7806c230 0x7fdb780cd720 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fdb780b5bc0 tx=0x7fdb7001c9f0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.579+0000 7fdb7e403640 1 -- 172.21.15.83:0/350469236 shutdown_connections 2024-04-17T10:47:02.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.579+0000 7fdb7e403640 1 --2- 172.21.15.83:0/350469236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb7806c230 0x7fdb780cd720 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.579+0000 7fdb7e403640 1 --2- 172.21.15.83:0/350469236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb780b36f0 0x7fdb7806bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.579+0000 7fdb7e403640 1 --2- 172.21.15.83:0/350469236 >> v2:172.21.15.167:3300/0 conn(0x7fdb7815a8a0 0x7fdb7806e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.579+0000 7fdb7e403640 1 -- 172.21.15.83:0/350469236 >> 172.21.15.83:0/350469236 conn(0x7fdb780b23f0 msgr2=0x7fdb780b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:02.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.579+0000 7fdb7e403640 1 -- 172.21.15.83:0/350469236 shutdown_connections 2024-04-17T10:47:02.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.579+0000 7fdb7e403640 1 -- 172.21.15.83:0/350469236 wait complete. 2024-04-17T10:47:02.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.579+0000 7fdb7e403640 1 Processor -- start 2024-04-17T10:47:02.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.579+0000 7fdb7e403640 1 -- start start 2024-04-17T10:47:02.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.583+0000 7fdb7e403640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb7806c230 0x7fdb780ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.583+0000 7fdb7e403640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb780b36f0 0x7fdb7815c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.583+0000 7fdb7e403640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb7815a8a0 0x7fdb7806abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.583+0000 7fdb7e403640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb780b3ad0 con 0x7fdb780b36f0 2024-04-17T10:47:02.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.583+0000 7fdb7e403640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7806d3f0 con 0x7fdb7815a8a0 2024-04-17T10:47:02.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.583+0000 7fdb7e403640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb780b3c50 con 0x7fdb7806c230 2024-04-17T10:47:02.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.583+0000 7fdb777fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb780b36f0 0x7fdb7815c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.583+0000 7fdb777fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb780b36f0 0x7fdb7815c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58228/0 (socket says 172.21.15.83:58228) 2024-04-17T10:47:02.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.583+0000 7fdb777fe640 1 -- 172.21.15.83:0/449254716 learned_addr learned my addr 172.21.15.83:0/449254716 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:02.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.583+0000 7fdb77fff640 1 --2- 172.21.15.83:0/449254716 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb7806c230 0x7fdb780ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.583+0000 7fdb7c9a0640 1 --2- 172.21.15.83:0/449254716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb7815a8a0 0x7fdb7806abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.583+0000 7fdb777fe640 1 -- 172.21.15.83:0/449254716 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb7806c230 msgr2=0x7fdb780ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:02.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.583+0000 7fdb777fe640 1 --2- 172.21.15.83:0/449254716 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb7806c230 0x7fdb780ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.583+0000 7fdb777fe640 1 -- 172.21.15.83:0/449254716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb7815a8a0 msgr2=0x7fdb7806abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:02.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.583+0000 7fdb777fe640 1 --2- 172.21.15.83:0/449254716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb7815a8a0 0x7fdb7806abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.583+0000 7fdb777fe640 1 -- 172.21.15.83:0/449254716 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb780ce8e0 con 0x7fdb780b36f0 2024-04-17T10:47:02.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.583+0000 7fdb7c9a0640 1 --2- 172.21.15.83:0/449254716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb7815a8a0 0x7fdb7806abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:02.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.583+0000 7fdb77fff640 1 --2- 172.21.15.83:0/449254716 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb7806c230 0x7fdb780ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:02.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.583+0000 7fdb777fe640 1 --2- 172.21.15.83:0/449254716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb780b36f0 0x7fdb7815c980 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fdb6800d830 tx=0x7fdb6800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:02.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.583+0000 7fdb74ff9640 1 -- 172.21.15.83:0/449254716 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb68018020 con 0x7fdb780b36f0 2024-04-17T10:47:02.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.583+0000 7fdb74ff9640 1 -- 172.21.15.83:0/449254716 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb680096a0 con 0x7fdb780b36f0 2024-04-17T10:47:02.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.583+0000 7fdb7e403640 1 -- 172.21.15.83:0/449254716 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb780cebc0 con 0x7fdb780b36f0 2024-04-17T10:47:02.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.583+0000 7fdb74ff9640 1 -- 172.21.15.83:0/449254716 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb68009a50 con 0x7fdb780b36f0 2024-04-17T10:47:02.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.583+0000 7fdb7e403640 1 -- 172.21.15.83:0/449254716 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb7814af70 con 0x7fdb780b36f0 2024-04-17T10:47:02.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.583+0000 7fdb74ff9640 1 -- 172.21.15.83:0/449254716 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdb6801b020 con 0x7fdb780b36f0 2024-04-17T10:47:02.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.583+0000 7fdb74ff9640 1 --2- 172.21.15.83:0/449254716 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdb58066f90 0x7fdb58069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.583+0000 7fdb74ff9640 1 -- 172.21.15.83:0/449254716 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fdb68085530 con 0x7fdb780b36f0 2024-04-17T10:47:02.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.583+0000 7fdb77fff640 1 --2- 172.21.15.83:0/449254716 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdb58066f90 0x7fdb58069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.583+0000 7fdb77fff640 1 --2- 172.21.15.83:0/449254716 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdb58066f90 0x7fdb58069450 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdb600097a0 tx=0x7fdb60008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:02.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb81570000 1 Processor -- start 2024-04-17T10:47:02.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb81570000 1 -- start start 2024-04-17T10:47:02.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb81570000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d441c96a40 0x55d441c96e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb81570000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d441c97360 0x55d441c51850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb81570000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d441c51d90 0x55d441c52170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb81570000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d441b341f0 con 0x55d441c96a40 2024-04-17T10:47:02.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb81570000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d441a9ae90 con 0x55d441c97360 2024-04-17T10:47:02.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb81570000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d441a22f50 con 0x55d441c51d90 2024-04-17T10:47:02.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb77fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d441c96a40 0x55d441c96e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb77fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d441c96a40 0x55d441c96e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58242/0 (socket says 172.21.15.83:58242) 2024-04-17T10:47:02.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb77fff640 1 -- 172.21.15.83:0/4076381865 learned_addr learned my addr 172.21.15.83:0/4076381865 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:02.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb7c9a0640 1 --2- 172.21.15.83:0/4076381865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d441c51d90 0x55d441c52170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb777fe640 1 --2- 172.21.15.83:0/4076381865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d441c97360 0x55d441c51850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb77fff640 1 -- 172.21.15.83:0/4076381865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d441c51d90 msgr2=0x55d441c52170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:02.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb77fff640 1 --2- 172.21.15.83:0/4076381865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d441c51d90 0x55d441c52170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb77fff640 1 -- 172.21.15.83:0/4076381865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d441c97360 msgr2=0x55d441c51850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:02.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb77fff640 1 --2- 172.21.15.83:0/4076381865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d441c97360 0x55d441c51850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb77fff640 1 -- 172.21.15.83:0/4076381865 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d441a728b0 con 0x55d441c96a40 2024-04-17T10:47:02.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb7c9a0640 1 --2- 172.21.15.83:0/4076381865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d441c51d90 0x55d441c52170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:02.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb77fff640 1 --2- 172.21.15.83:0/4076381865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d441c96a40 0x55d441c96e20 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fdb60004350 tx=0x7fdb60011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c935a8049e85a05 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:02.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb6dffb640 1 -- 172.21.15.83:0/4076381865 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6000f040 con 0x55d441c96a40 2024-04-17T10:47:02.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb6dffb640 1 -- 172.21.15.83:0/4076381865 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb60004380 con 0x55d441c96a40 2024-04-17T10:47:02.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb6dffb640 1 -- 172.21.15.83:0/4076381865 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6000ee80 con 0x55d441c96a40 2024-04-17T10:47:02.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb81570000 1 -- 172.21.15.83:0/4076381865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d441c96a40 msgr2=0x55d441c96e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:02.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb81570000 1 --2- 172.21.15.83:0/4076381865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d441c96a40 0x55d441c96e20 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fdb60004350 tx=0x7fdb60011000 comp rx=0 tx=0).stop 2024-04-17T10:47:02.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb81570000 1 -- 172.21.15.83:0/4076381865 shutdown_connections 2024-04-17T10:47:02.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb81570000 1 --2- 172.21.15.83:0/4076381865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d441c51d90 0x55d441c52170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb81570000 1 --2- 172.21.15.83:0/4076381865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d441c97360 0x55d441c51850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb81570000 1 --2- 172.21.15.83:0/4076381865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d441c96a40 0x55d441c96e20 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb81570000 1 -- 172.21.15.83:0/4076381865 >> 172.21.15.83:0/4076381865 conn(0x55d441c4dd40 msgr2=0x55d441c50130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:02.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb81570000 1 -- 172.21.15.83:0/4076381865 shutdown_connections 2024-04-17T10:47:02.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb81570000 1 -- 172.21.15.83:0/4076381865 wait complete. 2024-04-17T10:47:02.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb81570000 1 -- start start 2024-04-17T10:47:02.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb81570000 1 Processor -- start 2024-04-17T10:47:02.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb81570000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d441c51d90 0x55d441c9e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb81570000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d441c96a40 0x55d441c9e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb81570000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d441c97360 0x55d441c57d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb81570000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d441a55540 con 0x55d441c96a40 2024-04-17T10:47:02.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb81570000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d441a96f30 con 0x55d441c97360 2024-04-17T10:47:02.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb81570000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d441a79010 con 0x55d441c51d90 2024-04-17T10:47:02.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb81570000 10 client.0 _ll_register_callbacks cb 0x55d441b4f180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:47:02.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.587+0000 7fdb777fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d441c96a40 0x55d441c9e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb777fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d441c96a40 0x55d441c9e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58258/0 (socket says 172.21.15.83:58258) 2024-04-17T10:47:02.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb777fe640 1 -- 172.21.15.83:0/3500188617 learned_addr learned my addr 172.21.15.83:0/3500188617 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:02.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb7c9a0640 1 --2- 172.21.15.83:0/3500188617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d441c97360 0x55d441c57d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb77fff640 1 --2- 172.21.15.83:0/3500188617 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d441c51d90 0x55d441c9e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb777fe640 1 -- 172.21.15.83:0/3500188617 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d441c51d90 msgr2=0x55d441c9e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:02.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb777fe640 1 --2- 172.21.15.83:0/3500188617 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d441c51d90 0x55d441c9e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb777fe640 1 -- 172.21.15.83:0/3500188617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d441c97360 msgr2=0x55d441c57d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:02.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb777fe640 1 --2- 172.21.15.83:0/3500188617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d441c97360 0x55d441c57d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb777fe640 1 -- 172.21.15.83:0/3500188617 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d441a231a0 con 0x55d441c96a40 2024-04-17T10:47:02.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb77fff640 1 --2- 172.21.15.83:0/3500188617 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d441c51d90 0x55d441c9e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:02.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb777fe640 1 --2- 172.21.15.83:0/3500188617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d441c96a40 0x55d441c9e700 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fdb68009490 tx=0x7fdb68055eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0ab2551e5c1c51 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:02.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:47:02.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb56ffd640 1 -- 172.21.15.83:0/3500188617 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6806d020 con 0x55d441c96a40 2024-04-17T10:47:02.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb56ffd640 1 -- 172.21.15.83:0/3500188617 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fdb68052520 con 0x55d441c96a40 2024-04-17T10:47:02.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb81570000 1 -- 172.21.15.83:0/3500188617 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb60004380 con 0x55d441c96a40 2024-04-17T10:47:02.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb56ffd640 1 -- 172.21.15.83:0/3500188617 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb68083e60 con 0x55d441c96a40 2024-04-17T10:47:02.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb81570000 10 client.7764 fetch_fsmap learned FSMap version 56 2024-04-17T10:47:02.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb81570000 1 -- 172.21.15.83:0/3500188617 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x55d441b1d840 con 0x55d441c96a40 2024-04-17T10:47:02.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb56ffd640 1 -- 172.21.15.83:0/3500188617 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb60004380 con 0x55d441c96a40 2024-04-17T10:47:02.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb56ffd640 1 -- 172.21.15.83:0/3500188617 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 56) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdb68013050 con 0x55d441c96a40 2024-04-17T10:47:02.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb81570000 10 client.7764 fetch_fsmap finished waiting for FSMap version 56 2024-04-17T10:47:02.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb81570000 10 client.7764 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-17T10:47:02.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb81570000 20 client.7764 populate_metadata read hostname 'smithi083' 2024-04-17T10:47:02.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb81570000 1 --2- 172.21.15.83:0/3500188617 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x55d441c58b20 0x55d441c58f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb81570000 4 client.7764 mds_command: new command op to 7575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:47:02.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb81570000 1 -- 172.21.15.83:0/3500188617 --> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d441a36be0 con 0x55d441c58b20 2024-04-17T10:47:02.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb77fff640 1 --2- 172.21.15.83:0/3500188617 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x55d441c58b20 0x55d441c58f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb56ffd640 10 client.7764 ms_handle_connect on v2:172.21.15.83:6834/1056175765 2024-04-17T10:47:02.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb77fff640 1 --2- 172.21.15.83:0/3500188617 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x55d441c58b20 0x55d441c58f30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d9bde1311b3f53a server_cookie=f6d587f31e4e7491 in_seq=0 out_seq=0 2024-04-17T10:47:02.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb56ffd640 1 -- 172.21.15.83:0/3500188617 <== mds.0 v2:172.21.15.83:6834/1056175765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d441a36be0 con 0x55d441c58b20 2024-04-17T10:47:02.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb56ffd640 10 client.7764 handle_command_reply: tid=0 2024-04-17T10:47:02.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb81570000 1 client.7764 shutdown 2024-04-17T10:47:02.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb81570000 1 -- 172.21.15.83:0/3500188617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d441c96a40 msgr2=0x55d441c9e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:02.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb81570000 1 --2- 172.21.15.83:0/3500188617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d441c96a40 0x55d441c9e700 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fdb68009490 tx=0x7fdb68055eb0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb81570000 1 -- 172.21.15.83:0/3500188617 shutdown_connections 2024-04-17T10:47:02.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb81570000 1 --2- 172.21.15.83:0/3500188617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d441c97360 0x55d441c57d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb81570000 1 --2- 172.21.15.83:0/3500188617 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x55d441c58b20 0x55d441c58f30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb81570000 1 --2- 172.21.15.83:0/3500188617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d441c96a40 0x55d441c9e700 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb81570000 1 --2- 172.21.15.83:0/3500188617 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d441c51d90 0x55d441c9e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb56ffd640 0 client.7764 ms_handle_reset on v2:172.21.15.83:6834/1056175765 2024-04-17T10:47:02.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb81570000 1 -- 172.21.15.83:0/3500188617 >> 172.21.15.83:0/3500188617 conn(0x55d441c4dd40 msgr2=0x55d441c509c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:02.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb81570000 1 -- 172.21.15.83:0/3500188617 shutdown_connections 2024-04-17T10:47:02.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb81570000 1 -- 172.21.15.83:0/3500188617 wait complete. 2024-04-17T10:47:02.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.591+0000 7fdb81570000 20 client.7764 trim_cache size 0 max 16384 2024-04-17T10:47:02.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb81570000 1 Processor -- start 2024-04-17T10:47:02.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb81570000 1 -- start start 2024-04-17T10:47:02.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb81570000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d441c51d90 0x55d441cf2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb81570000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d441c97360 0x55d441cf33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb81570000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d441cf38f0 0x55d441c50130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb81570000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb78200b80 con 0x55d441cf38f0 2024-04-17T10:47:02.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb81570000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb781ffd30 con 0x55d441c97360 2024-04-17T10:47:02.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb81570000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb780cffb0 con 0x55d441c51d90 2024-04-17T10:47:02.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb7c9a0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d441cf38f0 0x55d441c50130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb7c9a0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d441cf38f0 0x55d441c50130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58262/0 (socket says 172.21.15.83:58262) 2024-04-17T10:47:02.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb7c9a0640 1 -- 172.21.15.83:0/864223659 learned_addr learned my addr 172.21.15.83:0/864223659 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:02.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb777fe640 1 --2- 172.21.15.83:0/864223659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d441c97360 0x55d441cf33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb77fff640 1 --2- 172.21.15.83:0/864223659 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d441c51d90 0x55d441cf2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb7c9a0640 1 -- 172.21.15.83:0/864223659 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d441c51d90 msgr2=0x55d441cf2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:02.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb7c9a0640 1 --2- 172.21.15.83:0/864223659 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d441c51d90 0x55d441cf2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb7c9a0640 1 -- 172.21.15.83:0/864223659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d441c97360 msgr2=0x55d441cf33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:02.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb7c9a0640 1 --2- 172.21.15.83:0/864223659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d441c97360 0x55d441cf33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb7c9a0640 1 -- 172.21.15.83:0/864223659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d441af4a10 con 0x55d441cf38f0 2024-04-17T10:47:02.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb7c9a0640 1 --2- 172.21.15.83:0/864223659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d441cf38f0 0x55d441c50130 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000c090 tx=0x7fdb70004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d8cf02ba9cd16d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:02.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb6d7fa640 1 -- 172.21.15.83:0/864223659 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb700a5780 con 0x55d441cf38f0 2024-04-17T10:47:02.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb6d7fa640 1 -- 172.21.15.83:0/864223659 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb700a5920 con 0x55d441cf38f0 2024-04-17T10:47:02.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb6d7fa640 1 -- 172.21.15.83:0/864223659 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb700a5d10 con 0x55d441cf38f0 2024-04-17T10:47:02.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb81570000 1 -- 172.21.15.83:0/864223659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d441cf38f0 msgr2=0x55d441c50130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:02.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb81570000 1 --2- 172.21.15.83:0/864223659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d441cf38f0 0x55d441c50130 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000c090 tx=0x7fdb70004ac0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb81570000 1 -- 172.21.15.83:0/864223659 shutdown_connections 2024-04-17T10:47:02.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb81570000 1 --2- 172.21.15.83:0/864223659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d441cf38f0 0x55d441c50130 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb81570000 1 --2- 172.21.15.83:0/864223659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d441c97360 0x55d441cf33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb81570000 1 --2- 172.21.15.83:0/864223659 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d441c51d90 0x55d441cf2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb81570000 1 -- 172.21.15.83:0/864223659 >> 172.21.15.83:0/864223659 conn(0x55d441c4dd40 msgr2=0x55d441c950e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:02.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb81570000 1 -- 172.21.15.83:0/864223659 shutdown_connections 2024-04-17T10:47:02.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb81570000 1 -- 172.21.15.83:0/864223659 wait complete. 2024-04-17T10:47:02.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb81570000 1 -- start start 2024-04-17T10:47:02.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.607+0000 7fdb81570000 1 Processor -- start 2024-04-17T10:47:02.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb81570000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d441c51d90 0x55d441cf7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb81570000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d441c97360 0x55d441cfa260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb81570000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d441cf38f0 0x55d441cfc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb81570000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d441a96f30 con 0x55d441c51d90 2024-04-17T10:47:02.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb81570000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb780d05b0 con 0x55d441cf38f0 2024-04-17T10:47:02.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb81570000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d441a41c00 con 0x55d441c97360 2024-04-17T10:47:02.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb81570000 10 client.0 _ll_register_callbacks cb 0x55d441a5da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:47:02.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb77fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d441c51d90 0x55d441cf7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb77fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d441c51d90 0x55d441cf7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58266/0 (socket says 172.21.15.83:58266) 2024-04-17T10:47:02.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb77fff640 1 -- 172.21.15.83:0/86864776 learned_addr learned my addr 172.21.15.83:0/86864776 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:02.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb7c9a0640 1 --2- 172.21.15.83:0/86864776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d441cf38f0 0x55d441cfc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb77fff640 1 -- 172.21.15.83:0/86864776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d441c97360 msgr2=0x55d441cfa260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:02.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb777fe640 1 --2- 172.21.15.83:0/86864776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d441c97360 0x55d441cfa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb77fff640 1 --2- 172.21.15.83:0/86864776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d441c97360 0x55d441cfa260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb77fff640 1 -- 172.21.15.83:0/86864776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d441cf38f0 msgr2=0x55d441cfc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb77fff640 1 --2- 172.21.15.83:0/86864776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d441cf38f0 0x55d441cfc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb77fff640 1 -- 172.21.15.83:0/86864776 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d441a2fb00 con 0x55d441c51d90 2024-04-17T10:47:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb7c9a0640 1 --2- 172.21.15.83:0/86864776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d441cf38f0 0x55d441cfc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb77fff640 1 --2- 172.21.15.83:0/86864776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d441c51d90 0x55d441cf7d10 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fdb6000e540 tx=0x7fdb60004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54e90fc24e70736b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:47:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb56ffd640 1 -- 172.21.15.83:0/86864776 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb60003db0 con 0x55d441c51d90 2024-04-17T10:47:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb56ffd640 1 -- 172.21.15.83:0/86864776 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fdb6002ba90 con 0x55d441c51d90 2024-04-17T10:47:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb81570000 1 -- 172.21.15.83:0/86864776 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb700a5920 con 0x55d441c51d90 2024-04-17T10:47:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb56ffd640 1 -- 172.21.15.83:0/86864776 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb6003a050 con 0x55d441c51d90 2024-04-17T10:47:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb81570000 10 client.7770 fetch_fsmap learned FSMap version 56 2024-04-17T10:47:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb81570000 1 -- 172.21.15.83:0/86864776 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x55d441a41f80 con 0x55d441c51d90 2024-04-17T10:47:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb56ffd640 1 -- 172.21.15.83:0/86864776 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb700a5920 con 0x55d441c51d90 2024-04-17T10:47:02.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb56ffd640 1 -- 172.21.15.83:0/86864776 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 56) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdb6003d020 con 0x55d441c51d90 2024-04-17T10:47:02.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb81570000 10 client.7770 fetch_fsmap finished waiting for FSMap version 56 2024-04-17T10:47:02.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb81570000 10 client.7770 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-17T10:47:02.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb81570000 20 client.7770 populate_metadata read hostname 'smithi083' 2024-04-17T10:47:02.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb81570000 1 --2- 172.21.15.83:0/86864776 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x55d441cfcf00 0x55d441cff2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb81570000 4 client.7770 mds_command: new command op to 7575 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-17T10:47:02.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb81570000 1 -- 172.21.15.83:0/86864776 --> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x55d441aa83d0 con 0x55d441cfcf00 2024-04-17T10:47:02.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb777fe640 1 --2- 172.21.15.83:0/86864776 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x55d441cfcf00 0x55d441cff2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb56ffd640 10 client.7770 ms_handle_connect on v2:172.21.15.83:6834/1056175765 2024-04-17T10:47:02.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb777fe640 1 --2- 172.21.15.83:0/86864776 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x55d441cfcf00 0x55d441cff2f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cab71a2d4b55f86e server_cookie=4f811037869c6eb1 in_seq=0 out_seq=0 2024-04-17T10:47:02.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb56ffd640 1 -- 172.21.15.83:0/86864776 <== mds.0 v2:172.21.15.83:6834/1056175765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55d441aa83d0 con 0x55d441cfcf00 2024-04-17T10:47:02.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb56ffd640 10 client.7770 handle_command_reply: tid=0 2024-04-17T10:47:02.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb81570000 1 client.7770 shutdown 2024-04-17T10:47:02.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb81570000 1 -- 172.21.15.83:0/86864776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d441c51d90 msgr2=0x55d441cf7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:02.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb81570000 1 --2- 172.21.15.83:0/86864776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d441c51d90 0x55d441cf7d10 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fdb6000e540 tx=0x7fdb60004740 comp rx=0 tx=0).stop 2024-04-17T10:47:02.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb81570000 1 -- 172.21.15.83:0/86864776 shutdown_connections 2024-04-17T10:47:02.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb81570000 1 --2- 172.21.15.83:0/86864776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d441cf38f0 0x55d441cfc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb81570000 1 --2- 172.21.15.83:0/86864776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d441c97360 0x55d441cfa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb81570000 1 --2- 172.21.15.83:0/86864776 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x55d441cfcf00 0x55d441cff2f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb81570000 1 --2- 172.21.15.83:0/86864776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d441c51d90 0x55d441cf7d10 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb56ffd640 0 client.7770 ms_handle_reset on v2:172.21.15.83:6834/1056175765 2024-04-17T10:47:02.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb81570000 1 -- 172.21.15.83:0/86864776 >> 172.21.15.83:0/86864776 conn(0x55d441c4dd40 msgr2=0x55d441c950e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:02.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb81570000 1 -- 172.21.15.83:0/86864776 shutdown_connections 2024-04-17T10:47:02.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb81570000 1 -- 172.21.15.83:0/86864776 wait complete. 2024-04-17T10:47:02.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb81570000 20 client.7770 trim_cache size 0 max 16384 2024-04-17T10:47:02.615 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:47:02.615 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0 2024-04-17T10:47:02.615 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:47:02.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb7e403640 1 -- 172.21.15.83:0/449254716 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdb58066f90 msgr2=0x7fdb58069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:02.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb7e403640 1 --2- 172.21.15.83:0/449254716 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdb58066f90 0x7fdb58069450 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdb600097a0 tx=0x7fdb60008040 comp rx=0 tx=0).stop 2024-04-17T10:47:02.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb7e403640 1 -- 172.21.15.83:0/449254716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb780b36f0 msgr2=0x7fdb7815c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:02.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb7e403640 1 --2- 172.21.15.83:0/449254716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb780b36f0 0x7fdb7815c980 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fdb6800d830 tx=0x7fdb6800dcf0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb7e403640 1 -- 172.21.15.83:0/449254716 shutdown_connections 2024-04-17T10:47:02.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb7e403640 1 --2- 172.21.15.83:0/449254716 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdb58066f90 0x7fdb58069450 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb7e403640 1 --2- 172.21.15.83:0/449254716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb7815a8a0 0x7fdb7806abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb7e403640 1 --2- 172.21.15.83:0/449254716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb780b36f0 0x7fdb7815c980 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb7e403640 1 --2- 172.21.15.83:0/449254716 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb7806c230 0x7fdb780ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb7e403640 1 -- 172.21.15.83:0/449254716 >> 172.21.15.83:0/449254716 conn(0x7fdb780b23f0 msgr2=0x7fdb7814c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:02.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb7e403640 1 -- 172.21.15.83:0/449254716 shutdown_connections 2024-04-17T10:47:02.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.611+0000 7fdb7e403640 1 -- 172.21.15.83:0/449254716 wait complete. 2024-04-17T10:47:02.622 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-17T10:47:02.622 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-17T10:47:02.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.743+0000 7f7632be9640 1 Processor -- start 2024-04-17T10:47:02.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.743+0000 7f7632be9640 1 -- start start 2024-04-17T10:47:02.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.743+0000 7f7632be9640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f762c06b520 0x7f762c071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.743+0000 7f7632be9640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f762c06c1f0 con 0x7f762c15bac0 2024-04-17T10:47:02.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.743+0000 7f7632be9640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f762c070a10 con 0x7f762c15b6e0 2024-04-17T10:47:02.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.743+0000 7f7632be9640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f762c070cf0 con 0x7f762c06b520 2024-04-17T10:47:02.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.743+0000 7f762bfff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f762c15bac0 0x7f762c06ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56770/0 (socket says 172.21.15.83:56770) 2024-04-17T10:47:02.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.743+0000 7f762bfff640 1 -- 172.21.15.83:0/2589257354 learned_addr learned my addr 172.21.15.83:0/2589257354 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:02.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.743+0000 7f7631186640 1 --2- 172.21.15.83:0/2589257354 >> v2:172.21.15.83:3300/0 conn(0x7f762c06b520 0x7f762c071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.743+0000 7f762b7fe640 1 -- 172.21.15.83:0/2589257354 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3213383424 0 0) 0x7f762c06c1f0 con 0x7f762c15bac0 2024-04-17T10:47:02.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.743+0000 7f762b7fe640 1 -- 172.21.15.83:0/2589257354 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f761c003530 con 0x7f762c15bac0 2024-04-17T10:47:02.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.743+0000 7f7631186640 1 -- 172.21.15.83:0/2589257354 >> v1:172.21.15.167:6789/0 conn(0x7f762c15b6e0 legacy=0x7f762c0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:02.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.743+0000 7f7631186640 1 -- 172.21.15.83:0/2589257354 >> v1:172.21.15.83:6789/0 conn(0x7f762c15bac0 legacy=0x7f762c06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:02.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.743+0000 7f7631186640 1 -- 172.21.15.83:0/2589257354 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f762c070fd0 con 0x7f762c06b520 2024-04-17T10:47:02.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.747+0000 7f7631186640 1 --2- 172.21.15.83:0/2589257354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f762c06b520 0x7f762c071cc0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f7624009d90 tx=0x7f762401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4711ab769847de server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:02.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.747+0000 7f762b7fe640 1 -- 172.21.15.83:0/2589257354 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76240b2070 con 0x7f762c06b520 2024-04-17T10:47:02.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.747+0000 7f762b7fe640 1 -- 172.21.15.83:0/2589257354 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76240a5830 con 0x7f762c06b520 2024-04-17T10:47:02.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.747+0000 7f762b7fe640 1 -- 172.21.15.83:0/2589257354 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76240a5c00 con 0x7f762c06b520 2024-04-17T10:47:02.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.747+0000 7f7632be9640 1 -- 172.21.15.83:0/2589257354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f762c06b520 msgr2=0x7f762c071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:02.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.747+0000 7f7632be9640 1 --2- 172.21.15.83:0/2589257354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f762c06b520 0x7f762c071cc0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f7624009d90 tx=0x7f762401c970 comp rx=0 tx=0).stop 2024-04-17T10:47:02.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.747+0000 7f7632be9640 1 -- 172.21.15.83:0/2589257354 shutdown_connections 2024-04-17T10:47:02.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.747+0000 7f7632be9640 1 --2- 172.21.15.83:0/2589257354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f762c06b520 0x7f762c071cc0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.747+0000 7f7632be9640 1 -- 172.21.15.83:0/2589257354 >> 172.21.15.83:0/2589257354 conn(0x7f762c0b15b0 msgr2=0x7f762c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:02.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.747+0000 7f7632be9640 1 -- 172.21.15.83:0/2589257354 shutdown_connections 2024-04-17T10:47:02.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.747+0000 7f7632be9640 1 -- 172.21.15.83:0/2589257354 wait complete. 2024-04-17T10:47:02.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.747+0000 7f7632be9640 1 Processor -- start 2024-04-17T10:47:02.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.747+0000 7f7632be9640 1 -- start start 2024-04-17T10:47:02.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.747+0000 7f7632be9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f762c06b520 0x7f762c13f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.747+0000 7f7632be9640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f762c15b6e0 0x7f762c15d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.747+0000 7f7632be9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f762c15bac0 0x7f762c159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.747+0000 7f7632be9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762c06d2b0 con 0x7f762c15bac0 2024-04-17T10:47:02.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.747+0000 7f7632be9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762c06d130 con 0x7f762c06b520 2024-04-17T10:47:02.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.747+0000 7f7632be9640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f762c169610 con 0x7f762c15b6e0 2024-04-17T10:47:02.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.747+0000 7f7631186640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f762c15bac0 0x7f762c159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.747+0000 7f7631186640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f762c15bac0 0x7f762c159d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58292/0 (socket says 172.21.15.83:58292) 2024-04-17T10:47:02.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.747+0000 7f7631186640 1 -- 172.21.15.83:0/2036272427 learned_addr learned my addr 172.21.15.83:0/2036272427 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:02.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.747+0000 7f7630985640 1 --2- 172.21.15.83:0/2036272427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f762c06b520 0x7f762c13f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.747+0000 7f762bfff640 1 --2- 172.21.15.83:0/2036272427 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f762c15b6e0 0x7f762c15d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.747+0000 7f7631186640 1 -- 172.21.15.83:0/2036272427 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f762c15b6e0 msgr2=0x7f762c15d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:02.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.747+0000 7f7631186640 1 --2- 172.21.15.83:0/2036272427 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f762c15b6e0 0x7f762c15d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.747+0000 7f7631186640 1 -- 172.21.15.83:0/2036272427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f762c06b520 msgr2=0x7f762c13f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:02.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.747+0000 7f7631186640 1 --2- 172.21.15.83:0/2036272427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f762c06b520 0x7f762c13f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.747+0000 7f7631186640 1 -- 172.21.15.83:0/2036272427 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f762c14f720 con 0x7f762c15bac0 2024-04-17T10:47:02.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.747+0000 7f7630985640 1 --2- 172.21.15.83:0/2036272427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f762c06b520 0x7f762c13f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:02.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.747+0000 7f7631186640 1 --2- 172.21.15.83:0/2036272427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f762c15bac0 0x7f762c159d30 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f76240b0040 tx=0x7f7624019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:02.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.751+0000 7f76297fa640 1 -- 172.21.15.83:0/2036272427 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76240b2040 con 0x7f762c15bac0 2024-04-17T10:47:02.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.751+0000 7f76297fa640 1 -- 172.21.15.83:0/2036272427 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76240a5630 con 0x7f762c15bac0 2024-04-17T10:47:02.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.751+0000 7f7632be9640 1 -- 172.21.15.83:0/2036272427 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f762c15e120 con 0x7f762c15bac0 2024-04-17T10:47:02.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.751+0000 7f76297fa640 1 -- 172.21.15.83:0/2036272427 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7624009450 con 0x7f762c15bac0 2024-04-17T10:47:02.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.751+0000 7f7632be9640 1 -- 172.21.15.83:0/2036272427 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f762c15e3e0 con 0x7f762c15bac0 2024-04-17T10:47:02.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.751+0000 7f76297fa640 1 -- 172.21.15.83:0/2036272427 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f76240c1020 con 0x7f762c15bac0 2024-04-17T10:47:02.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.751+0000 7f76297fa640 1 --2- 172.21.15.83:0/2036272427 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7610066f60 0x7f7610069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.751+0000 7f76297fa640 1 -- 172.21.15.83:0/2036272427 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f762411fa00 con 0x7f762c15bac0 2024-04-17T10:47:02.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.751+0000 7f7630985640 1 --2- 172.21.15.83:0/2036272427 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7610066f60 0x7f7610069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.751+0000 7f7630985640 1 --2- 172.21.15.83:0/2036272427 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7610066f60 0x7f7610069420 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7618000f70 tx=0x7f7618009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:02.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.751+0000 7f7635d56000 1 Processor -- start 2024-04-17T10:47:02.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.751+0000 7f7635d56000 1 -- start start 2024-04-17T10:47:02.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.751+0000 7f7635d56000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1012fb9d0 0x55f1012fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.751+0000 7f7635d56000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1012fc2f0 0x55f1012b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.751+0000 7f7635d56000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f1012b20a0 0x55f1012b2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.751+0000 7f7635d56000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1010feff0 con 0x55f1012fb9d0 2024-04-17T10:47:02.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.751+0000 7f7635d56000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f101087150 con 0x55f1012fc2f0 2024-04-17T10:47:02.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.751+0000 7f7635d56000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f1010fb090 con 0x55f1012b20a0 2024-04-17T10:47:02.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7630985640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1012fb9d0 0x55f1012fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7630985640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1012fb9d0 0x55f1012fbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58308/0 (socket says 172.21.15.83:58308) 2024-04-17T10:47:02.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7630985640 1 -- 172.21.15.83:0/3316205417 learned_addr learned my addr 172.21.15.83:0/3316205417 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:02.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7631186640 1 --2- 172.21.15.83:0/3316205417 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f1012b20a0 0x55f1012b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f762bfff640 1 --2- 172.21.15.83:0/3316205417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1012fc2f0 0x55f1012b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7630985640 1 -- 172.21.15.83:0/3316205417 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f1012b20a0 msgr2=0x55f1012b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:02.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7630985640 1 --2- 172.21.15.83:0/3316205417 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f1012b20a0 0x55f1012b2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7630985640 1 -- 172.21.15.83:0/3316205417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1012fc2f0 msgr2=0x55f1012b1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:02.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7630985640 1 --2- 172.21.15.83:0/3316205417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1012fc2f0 0x55f1012b1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7630985640 1 -- 172.21.15.83:0/3316205417 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1012950d0 con 0x55f1012fb9d0 2024-04-17T10:47:02.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7631186640 1 --2- 172.21.15.83:0/3316205417 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f1012b20a0 0x55f1012b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:02.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7630985640 1 --2- 172.21.15.83:0/3316205417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1012fb9d0 0x55f1012fbdb0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f761800da90 tx=0x7f7618010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a2d377154956f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:02.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f76167fc640 1 -- 172.21.15.83:0/3316205417 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7618024070 con 0x55f1012fb9d0 2024-04-17T10:47:02.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f76167fc640 1 -- 172.21.15.83:0/3316205417 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7618014ba0 con 0x55f1012fb9d0 2024-04-17T10:47:02.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f76167fc640 1 -- 172.21.15.83:0/3316205417 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761801f400 con 0x55f1012fb9d0 2024-04-17T10:47:02.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7635d56000 1 -- 172.21.15.83:0/3316205417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1012fb9d0 msgr2=0x55f1012fbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:02.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7635d56000 1 --2- 172.21.15.83:0/3316205417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1012fb9d0 0x55f1012fbdb0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f761800da90 tx=0x7f7618010000 comp rx=0 tx=0).stop 2024-04-17T10:47:02.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7635d56000 1 -- 172.21.15.83:0/3316205417 shutdown_connections 2024-04-17T10:47:02.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7635d56000 1 --2- 172.21.15.83:0/3316205417 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f1012b20a0 0x55f1012b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7635d56000 1 --2- 172.21.15.83:0/3316205417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1012fc2f0 0x55f1012b1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7635d56000 1 --2- 172.21.15.83:0/3316205417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1012fb9d0 0x55f1012fbdb0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7635d56000 1 -- 172.21.15.83:0/3316205417 >> 172.21.15.83:0/3316205417 conn(0x55f1012ae020 msgr2=0x55f1012b0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:02.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7635d56000 1 -- 172.21.15.83:0/3316205417 shutdown_connections 2024-04-17T10:47:02.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7635d56000 1 -- 172.21.15.83:0/3316205417 wait complete. 2024-04-17T10:47:02.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7635d56000 1 -- start start 2024-04-17T10:47:02.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7635d56000 1 Processor -- start 2024-04-17T10:47:02.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7635d56000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1012b20a0 0x55f1012e07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7635d56000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1012fb9d0 0x55f1012e0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7635d56000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f1012fc2f0 0x55f1012e1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7635d56000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1010c3250 con 0x55f1012fb9d0 2024-04-17T10:47:02.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7635d56000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1010b9690 con 0x55f1012b20a0 2024-04-17T10:47:02.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7635d56000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f101087a40 con 0x55f1012fc2f0 2024-04-17T10:47:02.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7635d56000 10 client.0 _ll_register_callbacks cb 0x55f10129ce90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:47:02.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f762bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1012fb9d0 0x55f1012e0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f762bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1012fb9d0 0x55f1012e0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58318/0 (socket says 172.21.15.83:58318) 2024-04-17T10:47:02.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f762bfff640 1 -- 172.21.15.83:0/1095078481 learned_addr learned my addr 172.21.15.83:0/1095078481 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:02.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7631186640 1 --2- 172.21.15.83:0/1095078481 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f1012fc2f0 0x55f1012e1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7630985640 1 --2- 172.21.15.83:0/1095078481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1012b20a0 0x55f1012e07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f762bfff640 1 -- 172.21.15.83:0/1095078481 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f1012fc2f0 msgr2=0x55f1012e1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:02.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f762bfff640 1 --2- 172.21.15.83:0/1095078481 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f1012fc2f0 0x55f1012e1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f762bfff640 1 -- 172.21.15.83:0/1095078481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1012b20a0 msgr2=0x55f1012e07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:02.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f762bfff640 1 --2- 172.21.15.83:0/1095078481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1012b20a0 0x55f1012e07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f762bfff640 1 -- 172.21.15.83:0/1095078481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f1011ac1c0 con 0x55f1012fb9d0 2024-04-17T10:47:02.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f762bfff640 1 --2- 172.21.15.83:0/1095078481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1012fb9d0 0x55f1012e0d30 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f7620001d50 tx=0x7f762000e590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51cad7fa712ac40 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:02.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f760b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:47:02.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f760b7fe640 1 -- 172.21.15.83:0/1095078481 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7620005a60 con 0x55f1012fb9d0 2024-04-17T10:47:02.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f760b7fe640 1 -- 172.21.15.83:0/1095078481 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f7620005c00 con 0x55f1012fb9d0 2024-04-17T10:47:02.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7635d56000 1 -- 172.21.15.83:0/1095078481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7618014ba0 con 0x55f1012fb9d0 2024-04-17T10:47:02.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f760b7fe640 1 -- 172.21.15.83:0/1095078481 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f762001f050 con 0x55f1012fb9d0 2024-04-17T10:47:02.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7635d56000 10 client.7785 fetch_fsmap learned FSMap version 56 2024-04-17T10:47:02.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.755+0000 7f7635d56000 1 -- 172.21.15.83:0/1095078481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x55f1011a8170 con 0x55f1012fb9d0 2024-04-17T10:47:02.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.759+0000 7f760b7fe640 1 -- 172.21.15.83:0/1095078481 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7618014ba0 con 0x55f1012fb9d0 2024-04-17T10:47:02.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.759+0000 7f760b7fe640 1 -- 172.21.15.83:0/1095078481 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 56) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7620022020 con 0x55f1012fb9d0 2024-04-17T10:47:02.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.759+0000 7f7635d56000 10 client.7785 fetch_fsmap finished waiting for FSMap version 56 2024-04-17T10:47:02.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.759+0000 7f7635d56000 10 client.7785 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-17T10:47:02.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.759+0000 7f7635d56000 20 client.7785 populate_metadata read hostname 'smithi083' 2024-04-17T10:47:02.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.759+0000 7f7635d56000 1 --2- 172.21.15.83:0/1095078481 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x55f10130bc80 0x55f10130e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.759+0000 7f7635d56000 4 client.7785 mds_command: new command op to 7575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:47:02.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.759+0000 7f7635d56000 1 -- 172.21.15.83:0/1095078481 --> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f10109ac60 con 0x55f10130bc80 2024-04-17T10:47:02.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.759+0000 7f7630985640 1 --2- 172.21.15.83:0/1095078481 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x55f10130bc80 0x55f10130e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.759+0000 7f7630985640 1 --2- 172.21.15.83:0/1095078481 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x55f10130bc80 0x55f10130e070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a162a4a0a0c46aa0 server_cookie=7ac4ca76308288 in_seq=0 out_seq=0 2024-04-17T10:47:02.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.759+0000 7f760b7fe640 10 client.7785 ms_handle_connect on v2:172.21.15.83:6834/1056175765 2024-04-17T10:47:02.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.759+0000 7f760b7fe640 1 -- 172.21.15.83:0/1095078481 <== mds.0 v2:172.21.15.83:6834/1056175765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f10109ac60 con 0x55f10130bc80 2024-04-17T10:47:02.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.759+0000 7f760b7fe640 10 client.7785 handle_command_reply: tid=0 2024-04-17T10:47:02.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.759+0000 7f7635d56000 1 client.7785 shutdown 2024-04-17T10:47:02.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.759+0000 7f7635d56000 1 -- 172.21.15.83:0/1095078481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1012fb9d0 msgr2=0x55f1012e0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:02.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.759+0000 7f7635d56000 1 --2- 172.21.15.83:0/1095078481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1012fb9d0 0x55f1012e0d30 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f7620001d50 tx=0x7f762000e590 comp rx=0 tx=0).stop 2024-04-17T10:47:02.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.759+0000 7f7635d56000 1 -- 172.21.15.83:0/1095078481 shutdown_connections 2024-04-17T10:47:02.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.759+0000 7f7635d56000 1 --2- 172.21.15.83:0/1095078481 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f1012fc2f0 0x55f1012e1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.759+0000 7f7635d56000 1 --2- 172.21.15.83:0/1095078481 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x55f10130bc80 0x55f10130e070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.759+0000 7f7635d56000 1 --2- 172.21.15.83:0/1095078481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1012fb9d0 0x55f1012e0d30 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.759+0000 7f7635d56000 1 --2- 172.21.15.83:0/1095078481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1012b20a0 0x55f1012e07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.759+0000 7f760b7fe640 0 client.7785 ms_handle_reset on v2:172.21.15.83:6834/1056175765 2024-04-17T10:47:02.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.759+0000 7f7635d56000 1 -- 172.21.15.83:0/1095078481 >> 172.21.15.83:0/1095078481 conn(0x55f1012ae020 msgr2=0x55f1012b0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:02.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.759+0000 7f7635d56000 1 -- 172.21.15.83:0/1095078481 shutdown_connections 2024-04-17T10:47:02.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.759+0000 7f7635d56000 1 -- 172.21.15.83:0/1095078481 wait complete. 2024-04-17T10:47:02.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.759+0000 7f7635d56000 20 client.7785 trim_cache size 0 max 16384 2024-04-17T10:47:02.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7635d56000 1 Processor -- start 2024-04-17T10:47:02.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7635d56000 1 -- start start 2024-04-17T10:47:02.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7635d56000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1012b20a0 0x55f101357ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7635d56000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1012fb9d0 0x55f101358420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7635d56000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f1012fc2f0 0x55f101358960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7635d56000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762c200a30 con 0x55f1012b20a0 2024-04-17T10:47:02.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7635d56000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762c157eb0 con 0x55f1012fb9d0 2024-04-17T10:47:02.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7635d56000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f762c15cc00 con 0x55f1012fc2f0 2024-04-17T10:47:02.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7630985640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1012b20a0 0x55f101357ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7630985640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1012b20a0 0x55f101357ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58328/0 (socket says 172.21.15.83:58328) 2024-04-17T10:47:02.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7630985640 1 -- 172.21.15.83:0/2524614683 learned_addr learned my addr 172.21.15.83:0/2524614683 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:02.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7630985640 1 -- 172.21.15.83:0/2524614683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f1012fc2f0 msgr2=0x55f101358960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:02.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7630985640 1 --2- 172.21.15.83:0/2524614683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f1012fc2f0 0x55f101358960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7630985640 1 -- 172.21.15.83:0/2524614683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1012fb9d0 msgr2=0x55f101358420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:02.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7630985640 1 --2- 172.21.15.83:0/2524614683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1012fb9d0 0x55f101358420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7630985640 1 -- 172.21.15.83:0/2524614683 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f101145de0 con 0x55f1012b20a0 2024-04-17T10:47:02.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7630985640 1 --2- 172.21.15.83:0/2524614683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1012b20a0 0x55f101357ee0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f761801f000 tx=0x7f7618012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1275edffaee0def1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:02.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7615ffb640 1 -- 172.21.15.83:0/2524614683 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7618035070 con 0x55f1012b20a0 2024-04-17T10:47:02.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7615ffb640 1 -- 172.21.15.83:0/2524614683 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7618030030 con 0x55f1012b20a0 2024-04-17T10:47:02.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7615ffb640 1 -- 172.21.15.83:0/2524614683 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7618024040 con 0x55f1012b20a0 2024-04-17T10:47:02.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7635d56000 1 -- 172.21.15.83:0/2524614683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1012b20a0 msgr2=0x55f101357ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:02.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7635d56000 1 --2- 172.21.15.83:0/2524614683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1012b20a0 0x55f101357ee0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f761801f000 tx=0x7f7618012000 comp rx=0 tx=0).stop 2024-04-17T10:47:02.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7635d56000 1 -- 172.21.15.83:0/2524614683 shutdown_connections 2024-04-17T10:47:02.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7635d56000 1 --2- 172.21.15.83:0/2524614683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f1012fc2f0 0x55f101358960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7635d56000 1 --2- 172.21.15.83:0/2524614683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1012fb9d0 0x55f101358420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7635d56000 1 --2- 172.21.15.83:0/2524614683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1012b20a0 0x55f101357ee0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7635d56000 1 -- 172.21.15.83:0/2524614683 >> 172.21.15.83:0/2524614683 conn(0x55f1012ae020 msgr2=0x55f101308fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:02.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7635d56000 1 -- 172.21.15.83:0/2524614683 shutdown_connections 2024-04-17T10:47:02.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7635d56000 1 -- 172.21.15.83:0/2524614683 wait complete. 2024-04-17T10:47:02.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7635d56000 1 -- start start 2024-04-17T10:47:02.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7635d56000 1 Processor -- start 2024-04-17T10:47:02.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7635d56000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f1012b20a0 0x55f10135bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7635d56000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1012fb9d0 0x55f10135c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7635d56000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1012fc2f0 0x55f10130dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7635d56000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f101099e00 con 0x55f1012fb9d0 2024-04-17T10:47:02.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7635d56000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762c1578b0 con 0x55f1012fc2f0 2024-04-17T10:47:02.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.775+0000 7f7635d56000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f1010fe300 con 0x55f1012b20a0 2024-04-17T10:47:02.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7635d56000 10 client.0 _ll_register_callbacks cb 0x55f10127d040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:47:02.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f762bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1012fb9d0 0x55f10135c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f762bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1012fb9d0 0x55f10135c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58336/0 (socket says 172.21.15.83:58336) 2024-04-17T10:47:02.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f762bfff640 1 -- 172.21.15.83:0/3505652904 learned_addr learned my addr 172.21.15.83:0/3505652904 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:02.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f762bfff640 1 -- 172.21.15.83:0/3505652904 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f1012b20a0 msgr2=0x55f10135bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:02.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f762bfff640 1 --2- 172.21.15.83:0/3505652904 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f1012b20a0 0x55f10135bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f762bfff640 1 -- 172.21.15.83:0/3505652904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1012fc2f0 msgr2=0x55f10130dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:02.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f762bfff640 1 --2- 172.21.15.83:0/3505652904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1012fc2f0 0x55f10130dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f762bfff640 1 -- 172.21.15.83:0/3505652904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f1010a08e0 con 0x55f1012fb9d0 2024-04-17T10:47:02.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f762bfff640 1 --2- 172.21.15.83:0/3505652904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1012fb9d0 0x55f10135c080 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f7620007fd0 tx=0x7f7620000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c451805474589ce server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:02.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f760b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:47:02.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f760b7fe640 1 -- 172.21.15.83:0/3505652904 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76200075b0 con 0x55f1012fb9d0 2024-04-17T10:47:02.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7635d56000 1 -- 172.21.15.83:0/3505652904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7618030030 con 0x55f1012fb9d0 2024-04-17T10:47:02.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f760b7fe640 1 -- 172.21.15.83:0/3505652904 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f7620013440 con 0x55f1012fb9d0 2024-04-17T10:47:02.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f760b7fe640 1 -- 172.21.15.83:0/3505652904 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7620032050 con 0x55f1012fb9d0 2024-04-17T10:47:02.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7635d56000 10 client.7791 fetch_fsmap learned FSMap version 56 2024-04-17T10:47:02.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7635d56000 1 -- 172.21.15.83:0/3505652904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x55f1011588f0 con 0x55f1012fb9d0 2024-04-17T10:47:02.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f760b7fe640 1 -- 172.21.15.83:0/3505652904 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7618030030 con 0x55f1012fb9d0 2024-04-17T10:47:02.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f760b7fe640 1 -- 172.21.15.83:0/3505652904 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 56) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7620035020 con 0x55f1012fb9d0 2024-04-17T10:47:02.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7635d56000 10 client.7791 fetch_fsmap finished waiting for FSMap version 56 2024-04-17T10:47:02.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7635d56000 10 client.7791 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-17T10:47:02.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7635d56000 20 client.7791 populate_metadata read hostname 'smithi083' 2024-04-17T10:47:02.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7635d56000 1 --2- 172.21.15.83:0/3505652904 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x55f10130ec20 0x55f10130f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:02.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7635d56000 4 client.7791 mds_command: new command op to 7575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T10:47:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7635d56000 1 -- 172.21.15.83:0/3505652904 --> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f10110c530 con 0x55f10130ec20 2024-04-17T10:47:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7630985640 1 --2- 172.21.15.83:0/3505652904 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x55f10130ec20 0x55f10130f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f760b7fe640 10 client.7791 ms_handle_connect on v2:172.21.15.83:6834/1056175765 2024-04-17T10:47:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7630985640 1 --2- 172.21.15.83:0/3505652904 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x55f10130ec20 0x55f10130f030 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51d1ae0572aa789d server_cookie=c16721632de8a67d in_seq=0 out_seq=0 2024-04-17T10:47:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f760b7fe640 1 -- 172.21.15.83:0/3505652904 <== mds.0 v2:172.21.15.83:6834/1056175765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x55f10110c530 con 0x55f10130ec20 2024-04-17T10:47:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f760b7fe640 10 client.7791 handle_command_reply: tid=0 2024-04-17T10:47:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7635d56000 1 client.7791 shutdown 2024-04-17T10:47:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7635d56000 1 -- 172.21.15.83:0/3505652904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1012fb9d0 msgr2=0x55f10135c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7635d56000 1 --2- 172.21.15.83:0/3505652904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1012fb9d0 0x55f10135c080 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f7620007fd0 tx=0x7f7620000ee0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7635d56000 1 -- 172.21.15.83:0/3505652904 shutdown_connections 2024-04-17T10:47:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7635d56000 1 --2- 172.21.15.83:0/3505652904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1012fc2f0 0x55f10130dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7635d56000 1 --2- 172.21.15.83:0/3505652904 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x55f10130ec20 0x55f10130f030 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7635d56000 1 --2- 172.21.15.83:0/3505652904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1012fb9d0 0x55f10135c080 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7635d56000 1 --2- 172.21.15.83:0/3505652904 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55f1012b20a0 0x55f10135bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7635d56000 1 -- 172.21.15.83:0/3505652904 >> 172.21.15.83:0/3505652904 conn(0x55f1012ae020 msgr2=0x55f1012fa070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:02.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f760b7fe640 0 client.7791 ms_handle_reset on v2:172.21.15.83:6834/1056175765 2024-04-17T10:47:02.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7635d56000 1 -- 172.21.15.83:0/3505652904 shutdown_connections 2024-04-17T10:47:02.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7635d56000 1 -- 172.21.15.83:0/3505652904 wait complete. 2024-04-17T10:47:02.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7635d56000 20 client.7791 trim_cache size 0 max 16384 2024-04-17T10:47:02.782 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:47:02.782 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T10:47:02.782 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-17T10:47:02.782 INFO:teuthology.orchestra.run.smithi083.stdout: "873d1145-8664-4166-a41c-2540df29ca70": { 2024-04-17T10:47:02.782 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/scrub_control_test_path", 2024-04-17T10:47:02.782 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "873d1145-8664-4166-a41c-2540df29ca70", 2024-04-17T10:47:02.783 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive" 2024-04-17T10:47:02.783 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:47:02.783 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:47:02.783 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:47:02.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7632be9640 1 -- 172.21.15.83:0/2036272427 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7610066f60 msgr2=0x7f7610069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:02.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7632be9640 1 --2- 172.21.15.83:0/2036272427 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7610066f60 0x7f7610069420 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7618000f70 tx=0x7f7618009040 comp rx=0 tx=0).stop 2024-04-17T10:47:02.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7632be9640 1 -- 172.21.15.83:0/2036272427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f762c15bac0 msgr2=0x7f762c159d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:02.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7632be9640 1 --2- 172.21.15.83:0/2036272427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f762c15bac0 0x7f762c159d30 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f76240b0040 tx=0x7f7624019710 comp rx=0 tx=0).stop 2024-04-17T10:47:02.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7632be9640 1 -- 172.21.15.83:0/2036272427 shutdown_connections 2024-04-17T10:47:02.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7632be9640 1 --2- 172.21.15.83:0/2036272427 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7610066f60 0x7f7610069420 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7632be9640 1 --2- 172.21.15.83:0/2036272427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f762c15bac0 0x7f762c159d30 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7632be9640 1 --2- 172.21.15.83:0/2036272427 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f762c15b6e0 0x7f762c15d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7632be9640 1 --2- 172.21.15.83:0/2036272427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f762c06b520 0x7f762c13f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:02.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7632be9640 1 -- 172.21.15.83:0/2036272427 >> 172.21.15.83:0/2036272427 conn(0x7f762c0b15b0 msgr2=0x7f762c14fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:02.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7632be9640 1 -- 172.21.15.83:0/2036272427 shutdown_connections 2024-04-17T10:47:02.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:02.779+0000 7f7632be9640 1 -- 172.21.15.83:0/2036272427 wait complete. 2024-04-17T10:47:07.792 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-17T10:47:07.792 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-17T10:47:07.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.935+0000 7fb62efb1640 1 Processor -- start 2024-04-17T10:47:07.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.935+0000 7fb62efb1640 1 -- start start 2024-04-17T10:47:07.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.935+0000 7fb62efb1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb62815b6e0 0x7fb628071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:07.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.935+0000 7fb62efb1640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb62806c1f0 con 0x7fb62815bac0 2024-04-17T10:47:07.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.935+0000 7fb62efb1640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb628070a10 con 0x7fb628070470 2024-04-17T10:47:07.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.935+0000 7fb62efb1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb628070cf0 con 0x7fb62815b6e0 2024-04-17T10:47:07.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.935+0000 7fb627fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb62815bac0 0x7fb6280cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40492/0 (socket says 172.21.15.83:40492) 2024-04-17T10:47:07.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.935+0000 7fb627fff640 1 -- 172.21.15.83:0/2046987593 learned_addr learned my addr 172.21.15.83:0/2046987593 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:07.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.935+0000 7fb62cd4d640 1 --2- 172.21.15.83:0/2046987593 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb62815b6e0 0x7fb628071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:07.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.935+0000 7fb6277fe640 1 -- 172.21.15.83:0/2046987593 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 142842157 0 0) 0x7fb62806c1f0 con 0x7fb62815bac0 2024-04-17T10:47:07.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.935+0000 7fb6277fe640 1 -- 172.21.15.83:0/2046987593 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb610003530 con 0x7fb62815bac0 2024-04-17T10:47:07.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.935+0000 7fb6277fe640 1 -- 172.21.15.83:0/2046987593 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3569210720 0 0) 0x7fb610003530 con 0x7fb62815bac0 2024-04-17T10:47:07.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.935+0000 7fb6277fe640 1 -- 172.21.15.83:0/2046987593 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb62815b6e0 msgr2=0x7fb628071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:07.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.935+0000 7fb6277fe640 1 --2- 172.21.15.83:0/2046987593 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb62815b6e0 0x7fb628071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:07.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.935+0000 7fb6277fe640 1 -- 172.21.15.83:0/2046987593 >> v1:172.21.15.167:6789/0 conn(0x7fb628070470 legacy=0x7fb62806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:07.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.935+0000 7fb6277fe640 1 -- 172.21.15.83:0/2046987593 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb628070fd0 con 0x7fb62815bac0 2024-04-17T10:47:07.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.935+0000 7fb6277fe640 1 -- 172.21.15.83:0/2046987593 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fb61c001680 con 0x7fb62815bac0 2024-04-17T10:47:07.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.935+0000 7fb6277fe640 1 -- 172.21.15.83:0/2046987593 >> v1:172.21.15.83:6789/0 conn(0x7fb62815bac0 legacy=0x7fb6280cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:07.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.935+0000 7fb6277fe640 1 --2- 172.21.15.83:0/2046987593 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb62815b6e0 0x7fb610003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:07.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.935+0000 7fb6277fe640 1 --2- 172.21.15.83:0/2046987593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb610003fe0 0x7fb610004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:07.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.935+0000 7fb6277fe640 1 --2- 172.21.15.83:0/2046987593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb610004d50 0x7fb6100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:07.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.935+0000 7fb6277fe640 1 -- 172.21.15.83:0/2046987593 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb628070cf0 con 0x7fb610004d50 2024-04-17T10:47:07.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.935+0000 7fb6277fe640 1 -- 172.21.15.83:0/2046987593 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61000bb30 con 0x7fb610003fe0 2024-04-17T10:47:07.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.935+0000 7fb6277fe640 1 -- 172.21.15.83:0/2046987593 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb61000bd30 con 0x7fb62815b6e0 2024-04-17T10:47:07.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.935+0000 7fb62d54e640 1 --2- 172.21.15.83:0/2046987593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb610004d50 0x7fb6100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:07.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.935+0000 7fb627fff640 1 --2- 172.21.15.83:0/2046987593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb610003fe0 0x7fb610004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:07.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.935+0000 7fb62d54e640 1 -- 172.21.15.83:0/2046987593 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb62815b6e0 msgr2=0x7fb610003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:07.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.935+0000 7fb62d54e640 1 --2- 172.21.15.83:0/2046987593 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb62815b6e0 0x7fb610003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:07.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.935+0000 7fb62d54e640 1 -- 172.21.15.83:0/2046987593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb610003fe0 msgr2=0x7fb610004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:07.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.935+0000 7fb62d54e640 1 --2- 172.21.15.83:0/2046987593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb610003fe0 0x7fb610004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:07.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.935+0000 7fb62d54e640 1 -- 172.21.15.83:0/2046987593 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb610003530 con 0x7fb610004d50 2024-04-17T10:47:07.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.935+0000 7fb627fff640 1 --2- 172.21.15.83:0/2046987593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb610003fe0 0x7fb610004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:47:07.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.939+0000 7fb62d54e640 1 --2- 172.21.15.83:0/2046987593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb610004d50 0x7fb6100051d0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fb620001e70 tx=0x7fb62001c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83a3def6e8b3613 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:07.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.939+0000 7fb6277fe640 1 -- 172.21.15.83:0/2046987593 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb620001a90 con 0x7fb610004d50 2024-04-17T10:47:07.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.939+0000 7fb6277fe640 1 -- 172.21.15.83:0/2046987593 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb620003b80 con 0x7fb610004d50 2024-04-17T10:47:07.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.939+0000 7fb6277fe640 1 -- 172.21.15.83:0/2046987593 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb620005080 con 0x7fb610004d50 2024-04-17T10:47:07.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.939+0000 7fb62efb1640 1 -- 172.21.15.83:0/2046987593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb610004d50 msgr2=0x7fb6100051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:07.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.939+0000 7fb62efb1640 1 --2- 172.21.15.83:0/2046987593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb610004d50 0x7fb6100051d0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fb620001e70 tx=0x7fb62001c780 comp rx=0 tx=0).stop 2024-04-17T10:47:07.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.939+0000 7fb62d54e640 1 -- 172.21.15.83:0/2046987593 reap_dead start 2024-04-17T10:47:07.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.939+0000 7fb62efb1640 1 -- 172.21.15.83:0/2046987593 shutdown_connections 2024-04-17T10:47:07.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.939+0000 7fb62efb1640 1 -- 172.21.15.83:0/2046987593 >> 172.21.15.83:0/2046987593 conn(0x7fb6280b15b0 msgr2=0x7fb6280b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:07.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.939+0000 7fb62efb1640 1 -- 172.21.15.83:0/2046987593 shutdown_connections 2024-04-17T10:47:07.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.939+0000 7fb62efb1640 1 -- 172.21.15.83:0/2046987593 wait complete. 2024-04-17T10:47:07.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.943+0000 7fb62efb1640 1 Processor -- start 2024-04-17T10:47:07.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.943+0000 7fb62efb1640 1 -- start start 2024-04-17T10:47:07.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.943+0000 7fb62efb1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb628150450 0x7fb62815e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:07.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.943+0000 7fb62efb1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb628150830 0x7fb6281593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:07.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.943+0000 7fb62efb1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6281552f0 0x7fb628159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:07.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.943+0000 7fb62efb1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62806b6a0 con 0x7fb6281552f0 2024-04-17T10:47:07.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.943+0000 7fb62efb1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62806b520 con 0x7fb628150830 2024-04-17T10:47:07.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.943+0000 7fb62efb1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb628071cc0 con 0x7fb628150450 2024-04-17T10:47:07.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.943+0000 7fb62d54e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6281552f0 0x7fb628159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:07.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.943+0000 7fb62d54e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6281552f0 0x7fb628159920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58838/0 (socket says 172.21.15.83:58838) 2024-04-17T10:47:07.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.943+0000 7fb62d54e640 1 -- 172.21.15.83:0/2586361908 learned_addr learned my addr 172.21.15.83:0/2586361908 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:07.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.943+0000 7fb627fff640 1 --2- 172.21.15.83:0/2586361908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb628150830 0x7fb6281593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:07.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.943+0000 7fb62cd4d640 1 --2- 172.21.15.83:0/2586361908 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb628150450 0x7fb62815e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:07.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.943+0000 7fb62d54e640 1 -- 172.21.15.83:0/2586361908 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb628150450 msgr2=0x7fb62815e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:07.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.943+0000 7fb62d54e640 1 --2- 172.21.15.83:0/2586361908 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb628150450 0x7fb62815e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:07.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.943+0000 7fb62d54e640 1 -- 172.21.15.83:0/2586361908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb628150830 msgr2=0x7fb6281593e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:07.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.943+0000 7fb62d54e640 1 --2- 172.21.15.83:0/2586361908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb628150830 0x7fb6281593e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:07.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.943+0000 7fb62d54e640 1 -- 172.21.15.83:0/2586361908 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb628154250 con 0x7fb6281552f0 2024-04-17T10:47:07.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.943+0000 7fb62cd4d640 1 --2- 172.21.15.83:0/2586361908 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb628150450 0x7fb62815e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:47:07.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.943+0000 7fb62d54e640 1 --2- 172.21.15.83:0/2586361908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6281552f0 0x7fb628159920 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fb62001cc60 tx=0x7fb610005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:07.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.943+0000 7fb6257fa640 1 -- 172.21.15.83:0/2586361908 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6200b8070 con 0x7fb6281552f0 2024-04-17T10:47:07.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.943+0000 7fb6257fa640 1 -- 172.21.15.83:0/2586361908 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb620005820 con 0x7fb6281552f0 2024-04-17T10:47:07.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.943+0000 7fb62efb1640 1 -- 172.21.15.83:0/2586361908 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb628154530 con 0x7fb6281552f0 2024-04-17T10:47:07.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.943+0000 7fb6257fa640 1 -- 172.21.15.83:0/2586361908 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb620007dc0 con 0x7fb6281552f0 2024-04-17T10:47:07.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.943+0000 7fb62efb1640 1 -- 172.21.15.83:0/2586361908 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb628154af0 con 0x7fb6281552f0 2024-04-17T10:47:07.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.943+0000 7fb6257fa640 1 -- 172.21.15.83:0/2586361908 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb6200bb070 con 0x7fb6281552f0 2024-04-17T10:47:07.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb6257fa640 1 --2- 172.21.15.83:0/2586361908 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb614066f90 0x7fb614069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:07.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb6257fa640 1 -- 172.21.15.83:0/2586361908 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fb62011e220 con 0x7fb6281552f0 2024-04-17T10:47:07.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb62cd4d640 1 --2- 172.21.15.83:0/2586361908 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb614066f90 0x7fb614069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:07.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb63211e000 1 Processor -- start 2024-04-17T10:47:07.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb62cd4d640 1 --2- 172.21.15.83:0/2586361908 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb614066f90 0x7fb614069450 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb6180047c0 tx=0x7fb618008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:07.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb63211e000 1 -- start start 2024-04-17T10:47:07.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb63211e000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e650f829d0 0x55e650f82db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:07.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb63211e000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e650f832f0 0x55e650f38b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:07.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb63211e000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e650f390a0 0x55e650f39480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:07.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb63211e000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e650d85ff0 con 0x55e650f832f0 2024-04-17T10:47:07.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb63211e000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e650d0e150 con 0x55e650f829d0 2024-04-17T10:47:07.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb63211e000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e650d82090 con 0x55e650f390a0 2024-04-17T10:47:07.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb627fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e650f832f0 0x55e650f38b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:07.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb627fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e650f832f0 0x55e650f38b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58840/0 (socket says 172.21.15.83:58840) 2024-04-17T10:47:07.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb627fff640 1 -- 172.21.15.83:0/634449831 learned_addr learned my addr 172.21.15.83:0/634449831 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:07.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb62cd4d640 1 --2- 172.21.15.83:0/634449831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e650f829d0 0x55e650f82db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:07.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb62d54e640 1 --2- 172.21.15.83:0/634449831 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e650f390a0 0x55e650f39480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:07.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb627fff640 1 -- 172.21.15.83:0/634449831 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e650f390a0 msgr2=0x55e650f39480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:07.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb627fff640 1 --2- 172.21.15.83:0/634449831 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e650f390a0 0x55e650f39480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:07.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb627fff640 1 -- 172.21.15.83:0/634449831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e650f829d0 msgr2=0x55e650f82db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:07.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb627fff640 1 --2- 172.21.15.83:0/634449831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e650f829d0 0x55e650f82db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:07.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb627fff640 1 -- 172.21.15.83:0/634449831 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e650f1c0d0 con 0x55e650f832f0 2024-04-17T10:47:07.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb62d54e640 1 --2- 172.21.15.83:0/634449831 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e650f390a0 0x55e650f39480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:07.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb627fff640 1 --2- 172.21.15.83:0/634449831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e650f832f0 0x55e650f38b60 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fb61c000c10 tx=0x7fb61c005ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d8566ad8c37f1b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:07.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb60e7fc640 1 -- 172.21.15.83:0/634449831 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c011bf0 con 0x55e650f832f0 2024-04-17T10:47:07.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb60e7fc640 1 -- 172.21.15.83:0/634449831 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61c011d90 con 0x55e650f832f0 2024-04-17T10:47:07.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb60e7fc640 1 -- 172.21.15.83:0/634449831 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c0048b0 con 0x55e650f832f0 2024-04-17T10:47:07.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb63211e000 1 -- 172.21.15.83:0/634449831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e650f832f0 msgr2=0x55e650f38b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:07.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb63211e000 1 --2- 172.21.15.83:0/634449831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e650f832f0 0x55e650f38b60 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fb61c000c10 tx=0x7fb61c005ed0 comp rx=0 tx=0).stop 2024-04-17T10:47:07.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb63211e000 1 -- 172.21.15.83:0/634449831 shutdown_connections 2024-04-17T10:47:07.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb63211e000 1 --2- 172.21.15.83:0/634449831 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e650f390a0 0x55e650f39480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:07.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb63211e000 1 --2- 172.21.15.83:0/634449831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e650f832f0 0x55e650f38b60 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:07.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb63211e000 1 --2- 172.21.15.83:0/634449831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e650f829d0 0x55e650f82db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:07.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb63211e000 1 -- 172.21.15.83:0/634449831 >> 172.21.15.83:0/634449831 conn(0x55e650f35020 msgr2=0x55e650f37440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:07.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb63211e000 1 -- 172.21.15.83:0/634449831 shutdown_connections 2024-04-17T10:47:07.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb63211e000 1 -- 172.21.15.83:0/634449831 wait complete. 2024-04-17T10:47:07.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb63211e000 1 -- start start 2024-04-17T10:47:07.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.947+0000 7fb63211e000 1 Processor -- start 2024-04-17T10:47:07.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb63211e000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e650f390a0 0x55e650f677f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:07.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb63211e000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e650f829d0 0x55e650f67d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:07.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb63211e000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e650f832f0 0x55e650f68270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:07.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb63211e000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e650d4a250 con 0x55e650f829d0 2024-04-17T10:47:07.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb63211e000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e650d40690 con 0x55e650f390a0 2024-04-17T10:47:07.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb63211e000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e650d0ea40 con 0x55e650f832f0 2024-04-17T10:47:07.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb63211e000 10 client.0 _ll_register_callbacks cb 0x55e650f23e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:47:07.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb627fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e650f829d0 0x55e650f67d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:07.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb627fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e650f829d0 0x55e650f67d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58850/0 (socket says 172.21.15.83:58850) 2024-04-17T10:47:07.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb627fff640 1 -- 172.21.15.83:0/3035008923 learned_addr learned my addr 172.21.15.83:0/3035008923 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:07.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb62cd4d640 1 --2- 172.21.15.83:0/3035008923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e650f390a0 0x55e650f677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:07.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb62d54e640 1 --2- 172.21.15.83:0/3035008923 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e650f832f0 0x55e650f68270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:07.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb627fff640 1 -- 172.21.15.83:0/3035008923 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e650f832f0 msgr2=0x55e650f68270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:07.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb627fff640 1 --2- 172.21.15.83:0/3035008923 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e650f832f0 0x55e650f68270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:07.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb627fff640 1 -- 172.21.15.83:0/3035008923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e650f390a0 msgr2=0x55e650f677f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:07.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb627fff640 1 --2- 172.21.15.83:0/3035008923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e650f390a0 0x55e650f677f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:07.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb627fff640 1 -- 172.21.15.83:0/3035008923 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e650e331c0 con 0x55e650f829d0 2024-04-17T10:47:07.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb627fff640 1 --2- 172.21.15.83:0/3035008923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e650f829d0 0x55e650f67d30 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fb61c005f10 tx=0x7fb61c001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfee697bbb0edf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:07.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb6077fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:47:07.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb6077fe640 1 -- 172.21.15.83:0/3035008923 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c023070 con 0x55e650f829d0 2024-04-17T10:47:07.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb6077fe640 1 -- 172.21.15.83:0/3035008923 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fb61c01a070 con 0x55e650f829d0 2024-04-17T10:47:07.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb63211e000 1 -- 172.21.15.83:0/3035008923 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb61c011d90 con 0x55e650f829d0 2024-04-17T10:47:07.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb6077fe640 1 -- 172.21.15.83:0/3035008923 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb61c02e050 con 0x55e650f829d0 2024-04-17T10:47:07.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb63211e000 10 client.7803 fetch_fsmap learned FSMap version 56 2024-04-17T10:47:07.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb63211e000 1 -- 172.21.15.83:0/3035008923 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x55e650e2f170 con 0x55e650f829d0 2024-04-17T10:47:07.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb6077fe640 1 -- 172.21.15.83:0/3035008923 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61c011d90 con 0x55e650f829d0 2024-04-17T10:47:07.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb6077fe640 1 -- 172.21.15.83:0/3035008923 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 56) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb61c031020 con 0x55e650f829d0 2024-04-17T10:47:07.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb63211e000 10 client.7803 fetch_fsmap finished waiting for FSMap version 56 2024-04-17T10:47:07.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb63211e000 10 client.7803 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-17T10:47:07.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb63211e000 20 client.7803 populate_metadata read hostname 'smithi083' 2024-04-17T10:47:07.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb63211e000 1 --2- 172.21.15.83:0/3035008923 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x55e650f92c80 0x55e650f95070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:07.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb63211e000 4 client.7803 mds_command: new command op to 7575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:47:07.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb63211e000 1 -- 172.21.15.83:0/3035008923 --> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e650d21c60 con 0x55e650f92c80 2024-04-17T10:47:07.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb62cd4d640 1 --2- 172.21.15.83:0/3035008923 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x55e650f92c80 0x55e650f95070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:07.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb62cd4d640 1 --2- 172.21.15.83:0/3035008923 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x55e650f92c80 0x55e650f95070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b54932b75c1f56b0 server_cookie=5b32374e0ed24ab0 in_seq=0 out_seq=0 2024-04-17T10:47:07.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb6077fe640 10 client.7803 ms_handle_connect on v2:172.21.15.83:6834/1056175765 2024-04-17T10:47:07.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb6077fe640 1 -- 172.21.15.83:0/3035008923 <== mds.0 v2:172.21.15.83:6834/1056175765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e650d21c60 con 0x55e650f92c80 2024-04-17T10:47:07.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb6077fe640 10 client.7803 handle_command_reply: tid=0 2024-04-17T10:47:07.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb63211e000 1 client.7803 shutdown 2024-04-17T10:47:07.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb63211e000 1 -- 172.21.15.83:0/3035008923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e650f829d0 msgr2=0x55e650f67d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:07.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb63211e000 1 --2- 172.21.15.83:0/3035008923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e650f829d0 0x55e650f67d30 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fb61c005f10 tx=0x7fb61c001e80 comp rx=0 tx=0).stop 2024-04-17T10:47:07.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb63211e000 1 -- 172.21.15.83:0/3035008923 shutdown_connections 2024-04-17T10:47:07.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb63211e000 1 --2- 172.21.15.83:0/3035008923 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e650f832f0 0x55e650f68270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:07.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb63211e000 1 --2- 172.21.15.83:0/3035008923 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x55e650f92c80 0x55e650f95070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:07.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb63211e000 1 --2- 172.21.15.83:0/3035008923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e650f829d0 0x55e650f67d30 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:07.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb63211e000 1 --2- 172.21.15.83:0/3035008923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e650f390a0 0x55e650f677f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:07.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb6077fe640 0 client.7803 ms_handle_reset on v2:172.21.15.83:6834/1056175765 2024-04-17T10:47:07.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb63211e000 1 -- 172.21.15.83:0/3035008923 >> 172.21.15.83:0/3035008923 conn(0x55e650f35020 msgr2=0x55e650f37ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:07.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb63211e000 1 -- 172.21.15.83:0/3035008923 shutdown_connections 2024-04-17T10:47:07.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb63211e000 1 -- 172.21.15.83:0/3035008923 wait complete. 2024-04-17T10:47:07.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.951+0000 7fb63211e000 20 client.7803 trim_cache size 0 max 16384 2024-04-17T10:47:07.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.971+0000 7fb63211e000 1 Processor -- start 2024-04-17T10:47:07.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.971+0000 7fb63211e000 1 -- start start 2024-04-17T10:47:07.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.971+0000 7fb63211e000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e650f390a0 0x55e650f37410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:07.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.971+0000 7fb63211e000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e650f829d0 0x55e650f37950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:07.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.971+0000 7fb63211e000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e650f832f0 0x55e650f37e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:07.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.971+0000 7fb63211e000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb628200760 con 0x55e650f832f0 2024-04-17T10:47:07.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.971+0000 7fb63211e000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6281ff920 con 0x55e650f829d0 2024-04-17T10:47:07.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.971+0000 7fb63211e000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb62815cd30 con 0x55e650f390a0 2024-04-17T10:47:07.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.971+0000 7fb62d54e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e650f832f0 0x55e650f37e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:07.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.971+0000 7fb62d54e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e650f832f0 0x55e650f37e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58864/0 (socket says 172.21.15.83:58864) 2024-04-17T10:47:07.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.971+0000 7fb62d54e640 1 -- 172.21.15.83:0/215436938 learned_addr learned my addr 172.21.15.83:0/215436938 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:07.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.971+0000 7fb627fff640 1 --2- 172.21.15.83:0/215436938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e650f829d0 0x55e650f37950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:07.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.971+0000 7fb62cd4d640 1 --2- 172.21.15.83:0/215436938 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e650f390a0 0x55e650f37410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:07.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.971+0000 7fb62d54e640 1 -- 172.21.15.83:0/215436938 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e650f390a0 msgr2=0x55e650f37410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:07.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.971+0000 7fb62d54e640 1 --2- 172.21.15.83:0/215436938 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e650f390a0 0x55e650f37410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:07.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.971+0000 7fb62d54e640 1 -- 172.21.15.83:0/215436938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e650f829d0 msgr2=0x55e650f37950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:07.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.971+0000 7fb62d54e640 1 --2- 172.21.15.83:0/215436938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e650f829d0 0x55e650f37950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:07.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.971+0000 7fb62d54e640 1 -- 172.21.15.83:0/215436938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e650dccde0 con 0x55e650f832f0 2024-04-17T10:47:07.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.971+0000 7fb62d54e640 1 --2- 172.21.15.83:0/215436938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e650f832f0 0x55e650f37e90 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fb6200eead0 tx=0x7fb6200eea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f3584ec887f2c2e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:07.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.971+0000 7fb60dffb640 1 -- 172.21.15.83:0/215436938 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6200eeb40 con 0x55e650f832f0 2024-04-17T10:47:07.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.971+0000 7fb60dffb640 1 -- 172.21.15.83:0/215436938 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6200f2c90 con 0x55e650f832f0 2024-04-17T10:47:07.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.971+0000 7fb60dffb640 1 -- 172.21.15.83:0/215436938 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6200f72e0 con 0x55e650f832f0 2024-04-17T10:47:07.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.971+0000 7fb63211e000 1 -- 172.21.15.83:0/215436938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e650f832f0 msgr2=0x55e650f37e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:07.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.971+0000 7fb63211e000 1 --2- 172.21.15.83:0/215436938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e650f832f0 0x55e650f37e90 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fb6200eead0 tx=0x7fb6200eea20 comp rx=0 tx=0).stop 2024-04-17T10:47:07.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.971+0000 7fb63211e000 1 -- 172.21.15.83:0/215436938 shutdown_connections 2024-04-17T10:47:07.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.971+0000 7fb63211e000 1 --2- 172.21.15.83:0/215436938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e650f832f0 0x55e650f37e90 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:07.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.971+0000 7fb63211e000 1 --2- 172.21.15.83:0/215436938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e650f829d0 0x55e650f37950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:07.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.971+0000 7fb63211e000 1 --2- 172.21.15.83:0/215436938 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e650f390a0 0x55e650f37410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:07.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.971+0000 7fb63211e000 1 -- 172.21.15.83:0/215436938 >> 172.21.15.83:0/215436938 conn(0x55e650f35020 msgr2=0x55e650f8ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:07.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.971+0000 7fb63211e000 1 -- 172.21.15.83:0/215436938 shutdown_connections 2024-04-17T10:47:07.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.971+0000 7fb63211e000 1 -- 172.21.15.83:0/215436938 wait complete. 2024-04-17T10:47:07.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb63211e000 1 -- start start 2024-04-17T10:47:07.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb63211e000 1 Processor -- start 2024-04-17T10:47:07.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb63211e000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e650f390a0 0x55e650f95380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:07.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb63211e000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e650f829d0 0x55e650f958c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:07.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb63211e000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e650f832f0 0x55e650f95e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:07.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb63211e000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e650d20e00 con 0x55e650f829d0 2024-04-17T10:47:07.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb63211e000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6281ff320 con 0x55e650f832f0 2024-04-17T10:47:07.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb63211e000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e650d85300 con 0x55e650f390a0 2024-04-17T10:47:07.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb63211e000 10 client.0 _ll_register_callbacks cb 0x55e650f04040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:47:07.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb627fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e650f829d0 0x55e650f958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:07.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb627fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e650f829d0 0x55e650f958c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58870/0 (socket says 172.21.15.83:58870) 2024-04-17T10:47:07.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb627fff640 1 -- 172.21.15.83:0/894576125 learned_addr learned my addr 172.21.15.83:0/894576125 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:07.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb62cd4d640 1 --2- 172.21.15.83:0/894576125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e650f390a0 0x55e650f95380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:07.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb62d54e640 1 --2- 172.21.15.83:0/894576125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e650f832f0 0x55e650f95e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:07.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb627fff640 1 -- 172.21.15.83:0/894576125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e650f390a0 msgr2=0x55e650f95380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:07.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb627fff640 1 --2- 172.21.15.83:0/894576125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e650f390a0 0x55e650f95380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:07.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb627fff640 1 -- 172.21.15.83:0/894576125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e650f832f0 msgr2=0x55e650f95e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:07.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb627fff640 1 --2- 172.21.15.83:0/894576125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e650f832f0 0x55e650f95e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:07.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb627fff640 1 -- 172.21.15.83:0/894576125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e650d278e0 con 0x55e650f829d0 2024-04-17T10:47:07.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb62cd4d640 1 --2- 172.21.15.83:0/894576125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e650f390a0 0x55e650f95380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:07.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb627fff640 1 --2- 172.21.15.83:0/894576125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e650f829d0 0x55e650f958c0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fb61c005f10 tx=0x7fb61c007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f42a84285610937 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:07.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb6077fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:47:07.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb6077fe640 1 -- 172.21.15.83:0/894576125 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c023040 con 0x55e650f829d0 2024-04-17T10:47:07.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb6077fe640 1 -- 172.21.15.83:0/894576125 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fb61c015670 con 0x55e650f829d0 2024-04-17T10:47:07.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb63211e000 1 -- 172.21.15.83:0/894576125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb6200f2c90 con 0x55e650f829d0 2024-04-17T10:47:07.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb6077fe640 1 -- 172.21.15.83:0/894576125 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb61c03e050 con 0x55e650f829d0 2024-04-17T10:47:07.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb63211e000 10 client.7809 fetch_fsmap learned FSMap version 56 2024-04-17T10:47:07.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb63211e000 1 -- 172.21.15.83:0/894576125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x55e650ddf8f0 con 0x55e650f829d0 2024-04-17T10:47:07.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb6077fe640 1 -- 172.21.15.83:0/894576125 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6200f2c90 con 0x55e650f829d0 2024-04-17T10:47:07.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb6077fe640 1 -- 172.21.15.83:0/894576125 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 56) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb61c041020 con 0x55e650f829d0 2024-04-17T10:47:07.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb63211e000 10 client.7809 fetch_fsmap finished waiting for FSMap version 56 2024-04-17T10:47:07.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb63211e000 10 client.7809 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-17T10:47:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb63211e000 20 client.7809 populate_metadata read hostname 'smithi083' 2024-04-17T10:47:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb63211e000 1 --2- 172.21.15.83:0/894576125 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x55e650f96340 0x55e650f96750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb63211e000 4 client.7809 mds_command: new command op to 7575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T10:47:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb63211e000 1 -- 172.21.15.83:0/894576125 --> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e650d93530 con 0x55e650f96340 2024-04-17T10:47:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb62cd4d640 1 --2- 172.21.15.83:0/894576125 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x55e650f96340 0x55e650f96750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb6077fe640 10 client.7809 ms_handle_connect on v2:172.21.15.83:6834/1056175765 2024-04-17T10:47:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb62cd4d640 1 --2- 172.21.15.83:0/894576125 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x55e650f96340 0x55e650f96750 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46e898950551a967 server_cookie=19d3293d05247ec6 in_seq=0 out_seq=0 2024-04-17T10:47:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb6077fe640 1 -- 172.21.15.83:0/894576125 <== mds.0 v2:172.21.15.83:6834/1056175765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e650d93530 con 0x55e650f96340 2024-04-17T10:47:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb6077fe640 10 client.7809 handle_command_reply: tid=0 2024-04-17T10:47:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb63211e000 1 client.7809 shutdown 2024-04-17T10:47:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb63211e000 1 -- 172.21.15.83:0/894576125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e650f829d0 msgr2=0x55e650f958c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb63211e000 1 --2- 172.21.15.83:0/894576125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e650f829d0 0x55e650f958c0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fb61c005f10 tx=0x7fb61c007040 comp rx=0 tx=0).stop 2024-04-17T10:47:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb63211e000 1 -- 172.21.15.83:0/894576125 shutdown_connections 2024-04-17T10:47:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb63211e000 1 --2- 172.21.15.83:0/894576125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e650f832f0 0x55e650f95e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:07.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb63211e000 1 --2- 172.21.15.83:0/894576125 >> [v2:172.21.15.83:6834/1056175765,v1:172.21.15.83:6836/1056175765] conn(0x55e650f96340 0x55e650f96750 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:07.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb63211e000 1 --2- 172.21.15.83:0/894576125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e650f829d0 0x55e650f958c0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:07.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb63211e000 1 --2- 172.21.15.83:0/894576125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e650f390a0 0x55e650f95380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:07.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb6077fe640 0 client.7809 ms_handle_reset on v2:172.21.15.83:6834/1056175765 2024-04-17T10:47:07.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb63211e000 1 -- 172.21.15.83:0/894576125 >> 172.21.15.83:0/894576125 conn(0x55e650f35020 msgr2=0x55e650f81070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:07.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb63211e000 1 -- 172.21.15.83:0/894576125 shutdown_connections 2024-04-17T10:47:07.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb63211e000 1 -- 172.21.15.83:0/894576125 wait complete. 2024-04-17T10:47:07.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb63211e000 20 client.7809 trim_cache size 0 max 16384 2024-04-17T10:47:07.981 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:47:07.981 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-17T10:47:07.982 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-17T10:47:07.982 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:47:07.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb62efb1640 1 -- 172.21.15.83:0/2586361908 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb614066f90 msgr2=0x7fb614069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:07.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb62efb1640 1 --2- 172.21.15.83:0/2586361908 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb614066f90 0x7fb614069450 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb6180047c0 tx=0x7fb618008040 comp rx=0 tx=0).stop 2024-04-17T10:47:07.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb62efb1640 1 -- 172.21.15.83:0/2586361908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6281552f0 msgr2=0x7fb628159920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:07.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb62efb1640 1 --2- 172.21.15.83:0/2586361908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6281552f0 0x7fb628159920 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fb62001cc60 tx=0x7fb610005840 comp rx=0 tx=0).stop 2024-04-17T10:47:07.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb62efb1640 1 -- 172.21.15.83:0/2586361908 shutdown_connections 2024-04-17T10:47:07.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb62efb1640 1 --2- 172.21.15.83:0/2586361908 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb614066f90 0x7fb614069450 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:07.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb62efb1640 1 --2- 172.21.15.83:0/2586361908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6281552f0 0x7fb628159920 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:07.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb62efb1640 1 --2- 172.21.15.83:0/2586361908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb628150830 0x7fb6281593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:07.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb62efb1640 1 --2- 172.21.15.83:0/2586361908 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb628150450 0x7fb62815e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:07.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.975+0000 7fb62efb1640 1 -- 172.21.15.83:0/2586361908 >> 172.21.15.83:0/2586361908 conn(0x7fb6280b15b0 msgr2=0x7fb62815f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:07.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.979+0000 7fb62efb1640 1 -- 172.21.15.83:0/2586361908 shutdown_connections 2024-04-17T10:47:07.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:07.979+0000 7fb62efb1640 1 -- 172.21.15.83:0/2586361908 wait complete. 2024-04-17T10:47:07.989 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T10:47:07.990 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:47:08.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.103+0000 7f7fd9169640 1 Processor -- start 2024-04-17T10:47:08.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.103+0000 7f7fd9169640 1 -- start start 2024-04-17T10:47:08.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.107+0000 7f7fd9169640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7fd4072330 0x7f7fd40cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:08.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.107+0000 7f7fd9169640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7fd40cfa10 0x7f7fd406c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:08.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.107+0000 7f7fd9169640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fd40cfea0 con 0x7f7fd415ba30 2024-04-17T10:47:08.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.107+0000 7f7fd9169640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fd4071ea0 con 0x7f7fd40cfa10 2024-04-17T10:47:08.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.107+0000 7f7fd9169640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fd406e320 con 0x7f7fd4072330 2024-04-17T10:47:08.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.107+0000 7f7fd359e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7fd40cfa10 0x7f7fd406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:08.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.107+0000 7f7fd2d9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7fd415ba30 0x7f7fd415c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40506/0 (socket says 172.21.15.83:40506) 2024-04-17T10:47:08.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.107+0000 7f7fd2d9d640 1 -- 172.21.15.83:0/2596776594 learned_addr learned my addr 172.21.15.83:0/2596776594 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:08.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.107+0000 7f7fd259c640 1 --2- 172.21.15.83:0/2596776594 >> v2:172.21.15.167:3300/0 conn(0x7f7fd4072330 0x7f7fd40cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:08.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.107+0000 7f7fd359e640 1 -- 172.21.15.83:0/2596776594 >> v2:172.21.15.167:3300/0 conn(0x7f7fd4072330 msgr2=0x7f7fd40cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:08.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.107+0000 7f7fd359e640 1 --2- 172.21.15.83:0/2596776594 >> v2:172.21.15.167:3300/0 conn(0x7f7fd4072330 0x7f7fd40cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:08.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.107+0000 7f7fd359e640 1 -- 172.21.15.83:0/2596776594 >> v1:172.21.15.83:6789/0 conn(0x7f7fd415ba30 legacy=0x7f7fd415c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:08.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.107+0000 7f7fd359e640 1 -- 172.21.15.83:0/2596776594 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd406e040 con 0x7f7fd40cfa10 2024-04-17T10:47:08.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.107+0000 7f7fd1d9b640 1 -- 172.21.15.83:0/2596776594 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3439071765 0 0) 0x7f7fd40cfea0 con 0x7f7fd415ba30 2024-04-17T10:47:08.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.111+0000 7f7fd359e640 1 --2- 172.21.15.83:0/2596776594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fd40cfa10 0x7f7fd406c490 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40af2b0 tx=0x7f7fcc01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8456c99aaa7b142 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:08.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.111+0000 7f7fd1d9b640 1 -- 172.21.15.83:0/2596776594 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc0b2070 con 0x7f7fd40cfa10 2024-04-17T10:47:08.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.111+0000 7f7fd1d9b640 1 -- 172.21.15.83:0/2596776594 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fcc0a5870 con 0x7f7fd40cfa10 2024-04-17T10:47:08.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.111+0000 7f7fd1d9b640 1 -- 172.21.15.83:0/2596776594 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc0a5c40 con 0x7f7fd40cfa10 2024-04-17T10:47:08.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.111+0000 7f7fd9169640 1 -- 172.21.15.83:0/2596776594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fd40cfa10 msgr2=0x7f7fd406c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:08.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.111+0000 7f7fd9169640 1 --2- 172.21.15.83:0/2596776594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fd40cfa10 0x7f7fd406c490 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40af2b0 tx=0x7f7fcc01c8a0 comp rx=0 tx=0).stop 2024-04-17T10:47:08.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.111+0000 7f7fd9169640 1 -- 172.21.15.83:0/2596776594 shutdown_connections 2024-04-17T10:47:08.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.111+0000 7f7fd9169640 1 --2- 172.21.15.83:0/2596776594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fd40cfa10 0x7f7fd406c490 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:08.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.111+0000 7f7fd9169640 1 --2- 172.21.15.83:0/2596776594 >> v2:172.21.15.167:3300/0 conn(0x7f7fd4072330 0x7f7fd40cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:08.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.111+0000 7f7fd9169640 1 -- 172.21.15.83:0/2596776594 >> 172.21.15.83:0/2596776594 conn(0x7f7fd406abe0 msgr2=0x7f7fd406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:08.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.111+0000 7f7fd9169640 1 -- 172.21.15.83:0/2596776594 shutdown_connections 2024-04-17T10:47:08.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.111+0000 7f7fd9169640 1 -- 172.21.15.83:0/2596776594 wait complete. 2024-04-17T10:47:08.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.111+0000 7f7fd9169640 1 Processor -- start 2024-04-17T10:47:08.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.111+0000 7f7fd9169640 1 -- start start 2024-04-17T10:47:08.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.111+0000 7f7fd9169640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fd4072330 0x7f7fd414b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:08.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.111+0000 7f7fd9169640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fd40cfa10 0x7f7fd4152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:08.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.111+0000 7f7fd9169640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fd415ba30 0x7f7fd40ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:08.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.111+0000 7f7fd9169640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd406de20 con 0x7f7fd415ba30 2024-04-17T10:47:08.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.111+0000 7f7fd9169640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd406dca0 con 0x7f7fd40cfa10 2024-04-17T10:47:08.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.111+0000 7f7fd9169640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd4168520 con 0x7f7fd4072330 2024-04-17T10:47:08.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.111+0000 7f7fd359e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fd415ba30 0x7f7fd40ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:08.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.111+0000 7f7fd359e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fd415ba30 0x7f7fd40ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58894/0 (socket says 172.21.15.83:58894) 2024-04-17T10:47:08.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.111+0000 7f7fd359e640 1 -- 172.21.15.83:0/3005224002 learned_addr learned my addr 172.21.15.83:0/3005224002 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:08.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.111+0000 7f7fd2d9d640 1 --2- 172.21.15.83:0/3005224002 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fd4072330 0x7f7fd414b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:08.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.111+0000 7f7fd259c640 1 --2- 172.21.15.83:0/3005224002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fd40cfa10 0x7f7fd4152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:08.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.111+0000 7f7fd359e640 1 -- 172.21.15.83:0/3005224002 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fd4072330 msgr2=0x7f7fd414b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:08.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.111+0000 7f7fd359e640 1 --2- 172.21.15.83:0/3005224002 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fd4072330 0x7f7fd414b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:08.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.111+0000 7f7fd359e640 1 -- 172.21.15.83:0/3005224002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fd40cfa10 msgr2=0x7f7fd4152260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:08.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.111+0000 7f7fd359e640 1 --2- 172.21.15.83:0/3005224002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fd40cfa10 0x7f7fd4152260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:08.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.111+0000 7f7fd359e640 1 -- 172.21.15.83:0/3005224002 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd40b2960 con 0x7f7fd415ba30 2024-04-17T10:47:08.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.115+0000 7f7fd359e640 1 --2- 172.21.15.83:0/3005224002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fd415ba30 0x7f7fd40ad840 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc0194b0 tx=0x7f7fcc019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:08.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.115+0000 7f7fcb7fe640 1 -- 172.21.15.83:0/3005224002 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc0bd020 con 0x7f7fd415ba30 2024-04-17T10:47:08.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.115+0000 7f7fcb7fe640 1 -- 172.21.15.83:0/3005224002 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fcc0b8670 con 0x7f7fd415ba30 2024-04-17T10:47:08.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.115+0000 7f7fd9169640 1 -- 172.21.15.83:0/3005224002 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fd4152b40 con 0x7f7fd415ba30 2024-04-17T10:47:08.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.115+0000 7f7fcb7fe640 1 -- 172.21.15.83:0/3005224002 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc0b2040 con 0x7f7fd415ba30 2024-04-17T10:47:08.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.115+0000 7f7fd9169640 1 -- 172.21.15.83:0/3005224002 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd40b2ee0 con 0x7f7fd415ba30 2024-04-17T10:47:08.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.115+0000 7f7fd9169640 1 -- 172.21.15.83:0/3005224002 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fd4074f90 con 0x7f7fd415ba30 2024-04-17T10:47:08.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.115+0000 7f7fcb7fe640 1 -- 172.21.15.83:0/3005224002 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7fcc0c0020 con 0x7f7fd415ba30 2024-04-17T10:47:08.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.115+0000 7f7fcb7fe640 1 --2- 172.21.15.83:0/3005224002 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7fb4066f60 0x7f7fb4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:08.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.115+0000 7f7fcb7fe640 1 -- 172.21.15.83:0/3005224002 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f7fcc1202c0 con 0x7f7fd415ba30 2024-04-17T10:47:08.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.115+0000 7f7fd2d9d640 1 --2- 172.21.15.83:0/3005224002 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7fb4066f60 0x7f7fb4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:08.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.115+0000 7f7fd2d9d640 1 --2- 172.21.15.83:0/3005224002 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7fb4066f60 0x7f7fb4069420 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4000f70 tx=0x7f7fc4007b80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:08.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.119+0000 7f7fcb7fe640 1 -- 172.21.15.83:0/3005224002 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7fcc0c7030 con 0x7f7fd415ba30 2024-04-17T10:47:08.131 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:47:08.133 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:47:08.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.315+0000 7f7fd9169640 1 -- 172.21.15.83:0/3005224002 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fd4159c50 con 0x7f7fd415ba30 2024-04-17T10:47:08.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.315+0000 7f7fcb7fe640 1 -- 172.21.15.83:0/3005224002 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f7fcc016520 con 0x7f7fd415ba30 2024-04-17T10:47:08.312 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:47:08.312 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5429,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6834/3936640219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3936640219},{"type":"v1","addr":"172.21.15.167:6834","nonce":3936640219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":5998,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4152882285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6833","nonce":4152882285},{"type":"v1","addr":"172.21.15.167:6835","nonce":4152882285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1562645860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1562645860},{"type":"v1","addr":"172.21.15.83:6837","nonce":1562645860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:46:36.578918+0000","modified":"2024-04-17T10:46:38.561701+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7575":{"gid":7575,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/1056175765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1056175765},{"type":"v1","addr":"172.21.15.83:6836","nonce":1056175765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-17T10:47:08.312 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 56 2024-04-17T10:47:08.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.319+0000 7f7fd9169640 1 -- 172.21.15.83:0/3005224002 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7fb4066f60 msgr2=0x7f7fb4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:08.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.319+0000 7f7fd9169640 1 --2- 172.21.15.83:0/3005224002 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7fb4066f60 0x7f7fb4069420 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4000f70 tx=0x7f7fc4007b80 comp rx=0 tx=0).stop 2024-04-17T10:47:08.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.319+0000 7f7fd9169640 1 -- 172.21.15.83:0/3005224002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fd415ba30 msgr2=0x7f7fd40ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:08.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.319+0000 7f7fd9169640 1 --2- 172.21.15.83:0/3005224002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fd415ba30 0x7f7fd40ad840 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc0194b0 tx=0x7f7fcc019640 comp rx=0 tx=0).stop 2024-04-17T10:47:08.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.319+0000 7f7fd9169640 1 -- 172.21.15.83:0/3005224002 shutdown_connections 2024-04-17T10:47:08.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.319+0000 7f7fd9169640 1 --2- 172.21.15.83:0/3005224002 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7fb4066f60 0x7f7fb4069420 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:08.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.319+0000 7f7fd9169640 1 --2- 172.21.15.83:0/3005224002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fd415ba30 0x7f7fd40ad840 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:08.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.319+0000 7f7fd9169640 1 --2- 172.21.15.83:0/3005224002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fd40cfa10 0x7f7fd4152260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:08.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.319+0000 7f7fd9169640 1 --2- 172.21.15.83:0/3005224002 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fd4072330 0x7f7fd414b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:08.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.319+0000 7f7fd9169640 1 -- 172.21.15.83:0/3005224002 >> 172.21.15.83:0/3005224002 conn(0x7f7fd406abe0 msgr2=0x7f7fd4159e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:08.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.319+0000 7f7fd9169640 1 -- 172.21.15.83:0/3005224002 shutdown_connections 2024-04-17T10:47:08.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.319+0000 7f7fd9169640 1 -- 172.21.15.83:0/3005224002 wait complete. 2024-04-17T10:47:08.331 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-17T10:47:08.331 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-17T10:47:08.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.451+0000 7f269aaea640 1 Processor -- start 2024-04-17T10:47:08.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.451+0000 7f269aaea640 1 -- start start 2024-04-17T10:47:08.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.451+0000 7f269aaea640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f269415dfa0 0x7f26940b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:08.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.451+0000 7f269aaea640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f26940b5b40 0x7f26940d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:08.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.451+0000 7f269aaea640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2694162a00 con 0x7f26940ce980 2024-04-17T10:47:08.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.451+0000 7f269aaea640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2694161e80 con 0x7f269415dfa0 2024-04-17T10:47:08.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.451+0000 7f269aaea640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2694162160 con 0x7f26940b5b40 2024-04-17T10:47:08.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.451+0000 7f2698886640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f269415dfa0 0x7f26940b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:08.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.451+0000 7f2693fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f26940ce980 0x7f26940b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40516/0 (socket says 172.21.15.83:40516) 2024-04-17T10:47:08.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.451+0000 7f2698886640 1 --2- 172.21.15.83:0/3206540582 >> v2:172.21.15.83:3300/0 conn(0x7f269415dfa0 0x7f26940b6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58898/0 (socket says 172.21.15.83:58898) 2024-04-17T10:47:08.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.451+0000 7f2693fff640 1 -- 172.21.15.83:0/3206540582 learned_addr learned my addr 172.21.15.83:0/3206540582 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:08.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.451+0000 7f2699087640 1 --2- 172.21.15.83:0/3206540582 >> v2:172.21.15.167:3300/0 conn(0x7f26940b5b40 0x7f26940d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:08.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.455+0000 7f2698886640 1 -- 172.21.15.83:0/3206540582 >> v2:172.21.15.167:3300/0 conn(0x7f26940b5b40 msgr2=0x7f26940d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:08.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.455+0000 7f2698886640 1 --2- 172.21.15.83:0/3206540582 >> v2:172.21.15.167:3300/0 conn(0x7f26940b5b40 0x7f26940d1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:08.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.455+0000 7f2698886640 1 -- 172.21.15.83:0/3206540582 >> v1:172.21.15.83:6789/0 conn(0x7f26940ce980 legacy=0x7f26940b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:08.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.455+0000 7f2698886640 1 -- 172.21.15.83:0/3206540582 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2694162440 con 0x7f269415dfa0 2024-04-17T10:47:08.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.455+0000 7f26937fe640 1 -- 172.21.15.83:0/3206540582 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 542327 0 0) 0x7f2694162a00 con 0x7f26940ce980 2024-04-17T10:47:08.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.455+0000 7f2698886640 1 --2- 172.21.15.83:0/3206540582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f269415dfa0 0x7f26940b6c20 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f26940ca980 tx=0x7f268401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b29041e572c40 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:08.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.455+0000 7f26937fe640 1 -- 172.21.15.83:0/3206540582 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26840b0070 con 0x7f269415dfa0 2024-04-17T10:47:08.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.455+0000 7f26937fe640 1 -- 172.21.15.83:0/3206540582 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26840ab600 con 0x7f269415dfa0 2024-04-17T10:47:08.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.455+0000 7f26937fe640 1 -- 172.21.15.83:0/3206540582 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26840ab9d0 con 0x7f269415dfa0 2024-04-17T10:47:08.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.455+0000 7f269aaea640 1 -- 172.21.15.83:0/3206540582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f269415dfa0 msgr2=0x7f26940b6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:08.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.455+0000 7f269aaea640 1 --2- 172.21.15.83:0/3206540582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f269415dfa0 0x7f26940b6c20 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f26940ca980 tx=0x7f268401c3c0 comp rx=0 tx=0).stop 2024-04-17T10:47:08.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.455+0000 7f269aaea640 1 -- 172.21.15.83:0/3206540582 shutdown_connections 2024-04-17T10:47:08.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.455+0000 7f269aaea640 1 --2- 172.21.15.83:0/3206540582 >> v2:172.21.15.167:3300/0 conn(0x7f26940b5b40 0x7f26940d1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:08.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.455+0000 7f269aaea640 1 --2- 172.21.15.83:0/3206540582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f269415dfa0 0x7f26940b6c20 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:08.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.455+0000 7f269aaea640 1 -- 172.21.15.83:0/3206540582 >> 172.21.15.83:0/3206540582 conn(0x7f26940d0600 msgr2=0x7f26940d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:08.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.455+0000 7f269aaea640 1 -- 172.21.15.83:0/3206540582 shutdown_connections 2024-04-17T10:47:08.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.455+0000 7f269aaea640 1 -- 172.21.15.83:0/3206540582 wait complete. 2024-04-17T10:47:08.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.459+0000 7f269aaea640 1 Processor -- start 2024-04-17T10:47:08.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.459+0000 7f269aaea640 1 -- start start 2024-04-17T10:47:08.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.459+0000 7f269aaea640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26940b5b40 0x7f26941f7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:08.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.459+0000 7f269aaea640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26940ce980 0x7f2694153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:08.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.459+0000 7f269aaea640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f269415dfa0 0x7f26940c54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:08.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.459+0000 7f269aaea640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269416ae50 con 0x7f26940b5b40 2024-04-17T10:47:08.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.459+0000 7f269aaea640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2694161b10 con 0x7f269415dfa0 2024-04-17T10:47:08.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.459+0000 7f269aaea640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f269416afd0 con 0x7f26940ce980 2024-04-17T10:47:08.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.459+0000 7f2698886640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26940b5b40 0x7f26941f7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:08.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.459+0000 7f2698886640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26940b5b40 0x7f26941f7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58908/0 (socket says 172.21.15.83:58908) 2024-04-17T10:47:08.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.459+0000 7f2698886640 1 -- 172.21.15.83:0/3373993329 learned_addr learned my addr 172.21.15.83:0/3373993329 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:08.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.459+0000 7f2699087640 1 --2- 172.21.15.83:0/3373993329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f269415dfa0 0x7f26940c54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:08.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.459+0000 7f2693fff640 1 --2- 172.21.15.83:0/3373993329 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26940ce980 0x7f2694153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:08.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.459+0000 7f2698886640 1 -- 172.21.15.83:0/3373993329 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26940ce980 msgr2=0x7f2694153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:08.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.459+0000 7f2698886640 1 --2- 172.21.15.83:0/3373993329 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26940ce980 0x7f2694153320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:08.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.459+0000 7f2698886640 1 -- 172.21.15.83:0/3373993329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f269415dfa0 msgr2=0x7f26940c54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:08.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.459+0000 7f2698886640 1 --2- 172.21.15.83:0/3373993329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f269415dfa0 0x7f26940c54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:08.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.459+0000 7f2698886640 1 -- 172.21.15.83:0/3373993329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26941f7bf0 con 0x7f26940b5b40 2024-04-17T10:47:08.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.459+0000 7f2693fff640 1 --2- 172.21.15.83:0/3373993329 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26940ce980 0x7f2694153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:08.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.459+0000 7f2698886640 1 --2- 172.21.15.83:0/3373993329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26940b5b40 0x7f26941f7330 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f26840ae040 tx=0x7f268401ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:08.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.459+0000 7f26917fa640 1 -- 172.21.15.83:0/3373993329 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26840b0050 con 0x7f26940b5b40 2024-04-17T10:47:08.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.459+0000 7f26917fa640 1 -- 172.21.15.83:0/3373993329 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26840bc070 con 0x7f26940b5b40 2024-04-17T10:47:08.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.459+0000 7f269aaea640 1 -- 172.21.15.83:0/3373993329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26941f7ed0 con 0x7f26940b5b40 2024-04-17T10:47:08.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.459+0000 7f26917fa640 1 -- 172.21.15.83:0/3373993329 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26840b5870 con 0x7f26940b5b40 2024-04-17T10:47:08.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.459+0000 7f269aaea640 1 -- 172.21.15.83:0/3373993329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26941f8490 con 0x7f26940b5b40 2024-04-17T10:47:08.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.459+0000 7f269aaea640 1 -- 172.21.15.83:0/3373993329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2694074f90 con 0x7f26940b5b40 2024-04-17T10:47:08.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.459+0000 7f26917fa640 1 -- 172.21.15.83:0/3373993329 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f26840b5a10 con 0x7f26940b5b40 2024-04-17T10:47:08.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.459+0000 7f26917fa640 1 --2- 172.21.15.83:0/3373993329 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2680066f60 0x7f2680069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:08.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.463+0000 7f26917fa640 1 -- 172.21.15.83:0/3373993329 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f268411fa20 con 0x7f26940b5b40 2024-04-17T10:47:08.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.463+0000 7f2693fff640 1 --2- 172.21.15.83:0/3373993329 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2680066f60 0x7f2680069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:08.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.463+0000 7f2693fff640 1 --2- 172.21.15.83:0/3373993329 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2680066f60 0x7f2680069420 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f26940cb920 tx=0x7f2688009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:08.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.467+0000 7f26917fa640 1 -- 172.21.15.83:0/3373993329 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2684016070 con 0x7f26940b5b40 2024-04-17T10:47:08.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.679+0000 7f269aaea640 1 -- 172.21.15.83:0/3373993329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f26940cb950 con 0x7f26940b5b40 2024-04-17T10:47:08.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.679+0000 7f26917fa640 1 -- 172.21.15.83:0/3373993329 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1810 (secure 0 0 0) 0x7f26840f0980 con 0x7f26940b5b40 2024-04-17T10:47:08.677 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:47:08.677 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":561,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":91,"num_osds":8,"num_up_osds":8,"osd_up_since":1713350279,"num_in_osds":8,"osd_in_since":1713350270,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077750389,"bytes_used":2689982464,"bytes_avail":770404130816,"bytes_total":773094113280,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":873555,"recovering_keys_per_sec":0,"num_objects_recovered":10,"num_bytes_recovered":10485760,"num_keys_recovered":0,"write_bytes_sec":38800890,"read_op_per_sec":0,"write_op_per_sec":39},"fsmap":{"epoch":56,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7575}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-04-17T10:47:04.340508+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/scrub_control_test_path]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-17T10:47:08.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.683+0000 7f269aaea640 1 -- 172.21.15.83:0/3373993329 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2680066f60 msgr2=0x7f2680069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:08.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.683+0000 7f269aaea640 1 --2- 172.21.15.83:0/3373993329 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2680066f60 0x7f2680069420 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f26940cb920 tx=0x7f2688009040 comp rx=0 tx=0).stop 2024-04-17T10:47:08.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.683+0000 7f269aaea640 1 -- 172.21.15.83:0/3373993329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26940b5b40 msgr2=0x7f26941f7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:08.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.683+0000 7f269aaea640 1 --2- 172.21.15.83:0/3373993329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26940b5b40 0x7f26941f7330 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f26840ae040 tx=0x7f268401ac00 comp rx=0 tx=0).stop 2024-04-17T10:47:08.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.683+0000 7f269aaea640 1 -- 172.21.15.83:0/3373993329 shutdown_connections 2024-04-17T10:47:08.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.683+0000 7f269aaea640 1 --2- 172.21.15.83:0/3373993329 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2680066f60 0x7f2680069420 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:08.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.683+0000 7f269aaea640 1 --2- 172.21.15.83:0/3373993329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f269415dfa0 0x7f26940c54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:08.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.683+0000 7f269aaea640 1 --2- 172.21.15.83:0/3373993329 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26940ce980 0x7f2694153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:08.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.683+0000 7f269aaea640 1 --2- 172.21.15.83:0/3373993329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26940b5b40 0x7f26941f7330 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:08.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.683+0000 7f269aaea640 1 -- 172.21.15.83:0/3373993329 >> 172.21.15.83:0/3373993329 conn(0x7f26940d0600 msgr2=0x7f26940d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:08.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.683+0000 7f269aaea640 1 -- 172.21.15.83:0/3373993329 shutdown_connections 2024-04-17T10:47:08.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:08.683+0000 7f269aaea640 1 -- 172.21.15.83:0/3373993329 wait complete. 2024-04-17T10:47:08.695 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'idle+waiting paths [/scrub_control_test_path]'} 2024-04-17T10:47:09.697 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:47:09.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.831+0000 7f3332aaf640 1 Processor -- start 2024-04-17T10:47:09.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.831+0000 7f3332aaf640 1 -- start start 2024-04-17T10:47:09.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.831+0000 7f3332aaf640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f332c147530 0x7f332c06ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:09.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.831+0000 7f3332aaf640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f332c0b5650 0x7f332c06d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:09.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.831+0000 7f3332aaf640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f332c0d0760 con 0x7f332c06b3c0 2024-04-17T10:47:09.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.831+0000 7f3332aaf640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f332c147910 con 0x7f332c147530 2024-04-17T10:47:09.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.831+0000 7f3332aaf640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f332c0d0a40 con 0x7f332c0b5650 2024-04-17T10:47:09.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.831+0000 7f333084b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f332c147530 0x7f332c06ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:09.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.831+0000 7f333084b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f332c147530 0x7f332c06ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:35608/0 (socket says 172.21.15.83:35608) 2024-04-17T10:47:09.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.831+0000 7f333084b640 1 -- 172.21.15.83:0/2401996208 learned_addr learned my addr 172.21.15.83:0/2401996208 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:09.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.831+0000 7f333104c640 1 --2- 172.21.15.83:0/2401996208 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f332c0b5650 0x7f332c06d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:09.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.831+0000 7f333084b640 1 -- 172.21.15.83:0/2401996208 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f332c0b5650 msgr2=0x7f332c06d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:09.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.831+0000 7f333084b640 1 --2- 172.21.15.83:0/2401996208 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f332c0b5650 0x7f332c06d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:09.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.831+0000 7f333084b640 1 -- 172.21.15.83:0/2401996208 >> v1:172.21.15.167:6789/0 conn(0x7f332c06b3c0 legacy=0x7f332c0b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:09.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.831+0000 7f333084b640 1 -- 172.21.15.83:0/2401996208 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f332c0d0d20 con 0x7f332c147530 2024-04-17T10:47:09.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.831+0000 7f333104c640 1 --2- 172.21.15.83:0/2401996208 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f332c0b5650 0x7f332c06d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:47:09.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.835+0000 7f333084b640 1 --2- 172.21.15.83:0/2401996208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f332c147530 0x7f332c06ae80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f331c009870 tx=0x7f331c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdc6d067b82097fb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:09.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.835+0000 7f332b7fe640 1 -- 172.21.15.83:0/2401996208 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331c0aa600 con 0x7f332c147530 2024-04-17T10:47:09.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.835+0000 7f332b7fe640 1 -- 172.21.15.83:0/2401996208 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f331c0af070 con 0x7f332c147530 2024-04-17T10:47:09.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.835+0000 7f332b7fe640 1 -- 172.21.15.83:0/2401996208 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331c0aa9c0 con 0x7f332c147530 2024-04-17T10:47:09.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.835+0000 7f3332aaf640 1 -- 172.21.15.83:0/2401996208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f332c147530 msgr2=0x7f332c06ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:09.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.835+0000 7f3332aaf640 1 --2- 172.21.15.83:0/2401996208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f332c147530 0x7f332c06ae80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f331c009870 tx=0x7f331c01c4a0 comp rx=0 tx=0).stop 2024-04-17T10:47:09.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.835+0000 7f3332aaf640 1 -- 172.21.15.83:0/2401996208 shutdown_connections 2024-04-17T10:47:09.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.835+0000 7f3332aaf640 1 --2- 172.21.15.83:0/2401996208 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f332c0b5650 0x7f332c06d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:09.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.835+0000 7f3332aaf640 1 --2- 172.21.15.83:0/2401996208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f332c147530 0x7f332c06ae80 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:09.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.835+0000 7f3332aaf640 1 -- 172.21.15.83:0/2401996208 >> 172.21.15.83:0/2401996208 conn(0x7f332c06bcf0 msgr2=0x7f332c06c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:09.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.835+0000 7f3332aaf640 1 -- 172.21.15.83:0/2401996208 shutdown_connections 2024-04-17T10:47:09.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.835+0000 7f3332aaf640 1 -- 172.21.15.83:0/2401996208 wait complete. 2024-04-17T10:47:09.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.835+0000 7f3332aaf640 1 Processor -- start 2024-04-17T10:47:09.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.835+0000 7f3332aaf640 1 -- start start 2024-04-17T10:47:09.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.839+0000 7f3332aaf640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f332c06b3c0 0x7f332c1376e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:09.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.839+0000 7f3332aaf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f332c0b5650 0x7f332c1397b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:09.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.839+0000 7f3332aaf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f332c147530 0x7f332c13a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:09.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.839+0000 7f3332aaf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332c06e500 con 0x7f332c147530 2024-04-17T10:47:09.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.839+0000 7f3332aaf640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332c06e380 con 0x7f332c0b5650 2024-04-17T10:47:09.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.839+0000 7f3332aaf640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f332c06e680 con 0x7f332c06b3c0 2024-04-17T10:47:09.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.839+0000 7f333104c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f332c147530 0x7f332c13a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:09.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.839+0000 7f333104c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f332c147530 0x7f332c13a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58920/0 (socket says 172.21.15.83:58920) 2024-04-17T10:47:09.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.839+0000 7f333104c640 1 -- 172.21.15.83:0/741804217 learned_addr learned my addr 172.21.15.83:0/741804217 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:09.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.839+0000 7f333084b640 1 --2- 172.21.15.83:0/741804217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f332c06b3c0 0x7f332c1376e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:09.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.839+0000 7f332bfff640 1 --2- 172.21.15.83:0/741804217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f332c0b5650 0x7f332c1397b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:09.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.839+0000 7f333104c640 1 -- 172.21.15.83:0/741804217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f332c06b3c0 msgr2=0x7f332c1376e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:09.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.839+0000 7f333104c640 1 --2- 172.21.15.83:0/741804217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f332c06b3c0 0x7f332c1376e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:09.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.839+0000 7f333104c640 1 -- 172.21.15.83:0/741804217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f332c0b5650 msgr2=0x7f332c1397b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:09.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.839+0000 7f333104c640 1 --2- 172.21.15.83:0/741804217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f332c0b5650 0x7f332c1397b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:09.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.839+0000 7f333104c640 1 -- 172.21.15.83:0/741804217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f332c137da0 con 0x7f332c147530 2024-04-17T10:47:09.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.839+0000 7f333084b640 1 --2- 172.21.15.83:0/741804217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f332c06b3c0 0x7f332c1376e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:47:09.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.839+0000 7f333104c640 1 --2- 172.21.15.83:0/741804217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f332c147530 0x7f332c13a400 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f332400efc0 tx=0x7f3324009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:09.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.839+0000 7f33297fa640 1 -- 172.21.15.83:0/741804217 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3324018030 con 0x7f332c147530 2024-04-17T10:47:09.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.839+0000 7f33297fa640 1 -- 172.21.15.83:0/741804217 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332400cde0 con 0x7f332c147530 2024-04-17T10:47:09.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.839+0000 7f3332aaf640 1 -- 172.21.15.83:0/741804217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f332c141c50 con 0x7f332c147530 2024-04-17T10:47:09.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.839+0000 7f33297fa640 1 -- 172.21.15.83:0/741804217 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3324003bc0 con 0x7f332c147530 2024-04-17T10:47:09.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.839+0000 7f3332aaf640 1 -- 172.21.15.83:0/741804217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f332c142210 con 0x7f332c147530 2024-04-17T10:47:09.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.839+0000 7f3332aaf640 1 -- 172.21.15.83:0/741804217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f332c074f90 con 0x7f332c147530 2024-04-17T10:47:09.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.839+0000 7f33297fa640 1 -- 172.21.15.83:0/741804217 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f332401b070 con 0x7f332c147530 2024-04-17T10:47:09.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.839+0000 7f33297fa640 1 --2- 172.21.15.83:0/741804217 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3318066f60 0x7f3318069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:09.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.839+0000 7f33297fa640 1 -- 172.21.15.83:0/741804217 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f3324085aa0 con 0x7f332c147530 2024-04-17T10:47:09.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.839+0000 7f333084b640 1 --2- 172.21.15.83:0/741804217 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3318066f60 0x7f3318069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:09.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.839+0000 7f333084b640 1 --2- 172.21.15.83:0/741804217 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3318066f60 0x7f3318069420 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f331c0ae000 tx=0x7f331c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:09.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:09.843+0000 7f33297fa640 1 -- 172.21.15.83:0/741804217 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3324010040 con 0x7f332c147530 2024-04-17T10:47:10.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.031+0000 7f3332aaf640 1 -- 172.21.15.83:0/741804217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f332c071890 con 0x7f332c147530 2024-04-17T10:47:10.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.031+0000 7f33297fa640 1 -- 172.21.15.83:0/741804217 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f3324056b00 con 0x7f332c147530 2024-04-17T10:47:10.029 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:47:10.029 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5429,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6834/3936640219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3936640219},{"type":"v1","addr":"172.21.15.167:6834","nonce":3936640219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":5998,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4152882285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6833","nonce":4152882285},{"type":"v1","addr":"172.21.15.167:6835","nonce":4152882285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1562645860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1562645860},{"type":"v1","addr":"172.21.15.83:6837","nonce":1562645860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:46:36.578918+0000","modified":"2024-04-17T10:46:38.561701+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7575":{"gid":7575,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/1056175765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1056175765},{"type":"v1","addr":"172.21.15.83:6836","nonce":1056175765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-17T10:47:10.029 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 56 2024-04-17T10:47:10.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.035+0000 7f3332aaf640 1 -- 172.21.15.83:0/741804217 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3318066f60 msgr2=0x7f3318069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:10.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.035+0000 7f3332aaf640 1 --2- 172.21.15.83:0/741804217 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3318066f60 0x7f3318069420 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f331c0ae000 tx=0x7f331c0a7000 comp rx=0 tx=0).stop 2024-04-17T10:47:10.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.035+0000 7f3332aaf640 1 -- 172.21.15.83:0/741804217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f332c147530 msgr2=0x7f332c13a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:10.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.035+0000 7f3332aaf640 1 --2- 172.21.15.83:0/741804217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f332c147530 0x7f332c13a400 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f332400efc0 tx=0x7f3324009f90 comp rx=0 tx=0).stop 2024-04-17T10:47:10.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.035+0000 7f3332aaf640 1 -- 172.21.15.83:0/741804217 shutdown_connections 2024-04-17T10:47:10.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.035+0000 7f3332aaf640 1 --2- 172.21.15.83:0/741804217 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3318066f60 0x7f3318069420 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:10.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.035+0000 7f3332aaf640 1 --2- 172.21.15.83:0/741804217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f332c147530 0x7f332c13a400 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:10.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.035+0000 7f3332aaf640 1 --2- 172.21.15.83:0/741804217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f332c0b5650 0x7f332c1397b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:10.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.035+0000 7f3332aaf640 1 --2- 172.21.15.83:0/741804217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f332c06b3c0 0x7f332c1376e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:10.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.035+0000 7f3332aaf640 1 -- 172.21.15.83:0/741804217 >> 172.21.15.83:0/741804217 conn(0x7f332c06bcf0 msgr2=0x7f332c0cfa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:10.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.035+0000 7f3332aaf640 1 -- 172.21.15.83:0/741804217 shutdown_connections 2024-04-17T10:47:10.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.035+0000 7f3332aaf640 1 -- 172.21.15.83:0/741804217 wait complete. 2024-04-17T10:47:10.046 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-17T10:47:10.047 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-17T10:47:10.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.155+0000 7fe8c4831640 1 Processor -- start 2024-04-17T10:47:10.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.155+0000 7fe8c4831640 1 -- start start 2024-04-17T10:47:10.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.159+0000 7fe8c4831640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8c00af530 0x7fe8c0166b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:10.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.159+0000 7fe8c4831640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8c0160650 con 0x7fe8c00b2de0 2024-04-17T10:47:10.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.159+0000 7fe8c4831640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8c015f5b0 con 0x7fe8c015f1d0 2024-04-17T10:47:10.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.159+0000 7fe8c4831640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c00b2620 con 0x7fe8c00af530 2024-04-17T10:47:10.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.159+0000 7fe8b5d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe8c00b2de0 0x7fe8c015ff40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40522/0 (socket says 172.21.15.83:40522) 2024-04-17T10:47:10.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.159+0000 7fe8b5d9b640 1 -- 172.21.15.83:0/2461894312 learned_addr learned my addr 172.21.15.83:0/2461894312 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:10.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.159+0000 7fe8bdd9b640 1 -- 172.21.15.83:0/2461894312 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2241276091 0 0) 0x7fe8c0160650 con 0x7fe8c00b2de0 2024-04-17T10:47:10.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.159+0000 7fe8bdd9b640 1 -- 172.21.15.83:0/2461894312 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8ac003530 con 0x7fe8c00b2de0 2024-04-17T10:47:10.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.159+0000 7fe8bed9d640 1 --2- 172.21.15.83:0/2461894312 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8c00af530 0x7fe8c0166b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:10.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.159+0000 7fe8bdd9b640 1 -- 172.21.15.83:0/2461894312 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1092803182 0 0) 0x7fe8ac003530 con 0x7fe8c00b2de0 2024-04-17T10:47:10.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.159+0000 7fe8bdd9b640 1 -- 172.21.15.83:0/2461894312 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8c00af530 msgr2=0x7fe8c0166b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:10.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.159+0000 7fe8bdd9b640 1 --2- 172.21.15.83:0/2461894312 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8c00af530 0x7fe8c0166b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:10.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.159+0000 7fe8bdd9b640 1 -- 172.21.15.83:0/2461894312 >> v1:172.21.15.167:6789/0 conn(0x7fe8c015f1d0 legacy=0x7fe8c015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:10.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.159+0000 7fe8bdd9b640 1 -- 172.21.15.83:0/2461894312 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c00b2340 con 0x7fe8c00b2de0 2024-04-17T10:47:10.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.159+0000 7fe8bdd9b640 1 -- 172.21.15.83:0/2461894312 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fe8b0001680 con 0x7fe8c00b2de0 2024-04-17T10:47:10.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.159+0000 7fe8bdd9b640 1 -- 172.21.15.83:0/2461894312 >> v1:172.21.15.83:6789/0 conn(0x7fe8c00b2de0 legacy=0x7fe8c015ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:10.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.159+0000 7fe8bdd9b640 1 --2- 172.21.15.83:0/2461894312 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8c00af530 0x7fe8ac0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:10.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.159+0000 7fe8bdd9b640 1 --2- 172.21.15.83:0/2461894312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8ac004020 0x7fe8ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:10.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.159+0000 7fe8bdd9b640 1 --2- 172.21.15.83:0/2461894312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ac004d50 0x7fe8ac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:10.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.159+0000 7fe8bdd9b640 1 -- 172.21.15.83:0/2461894312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c00b2620 con 0x7fe8ac004d50 2024-04-17T10:47:10.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.159+0000 7fe8bdd9b640 1 -- 172.21.15.83:0/2461894312 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac00bb90 con 0x7fe8ac004020 2024-04-17T10:47:10.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.159+0000 7fe8bdd9b640 1 -- 172.21.15.83:0/2461894312 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac00bd90 con 0x7fe8c00af530 2024-04-17T10:47:10.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.159+0000 7fe8bed9d640 1 --2- 172.21.15.83:0/2461894312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ac004d50 0x7fe8ac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:10.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.159+0000 7fe8b5d9b640 1 --2- 172.21.15.83:0/2461894312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8ac004020 0x7fe8ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:10.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.159+0000 7fe8be59c640 1 --2- 172.21.15.83:0/2461894312 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8c00af530 0x7fe8ac0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:10.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.159+0000 7fe8bed9d640 1 -- 172.21.15.83:0/2461894312 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8c00af530 msgr2=0x7fe8ac0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:10.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.159+0000 7fe8bed9d640 1 --2- 172.21.15.83:0/2461894312 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8c00af530 0x7fe8ac0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:10.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.159+0000 7fe8bed9d640 1 -- 172.21.15.83:0/2461894312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8ac004020 msgr2=0x7fe8ac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:10.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.159+0000 7fe8bed9d640 1 --2- 172.21.15.83:0/2461894312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8ac004020 0x7fe8ac004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:10.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.159+0000 7fe8bed9d640 1 -- 172.21.15.83:0/2461894312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ac003530 con 0x7fe8ac004d50 2024-04-17T10:47:10.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8bed9d640 1 --2- 172.21.15.83:0/2461894312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ac004d50 0x7fe8ac0051d0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8007360 tx=0x7fe8b801ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f99784472e80a553 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:10.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8bdd9b640 1 -- 172.21.15.83:0/2461894312 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b80b2070 con 0x7fe8ac004d50 2024-04-17T10:47:10.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8bdd9b640 1 -- 172.21.15.83:0/2461894312 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b80a5d40 con 0x7fe8ac004d50 2024-04-17T10:47:10.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8bdd9b640 1 -- 172.21.15.83:0/2461894312 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8009be0 con 0x7fe8ac004d50 2024-04-17T10:47:10.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8c4831640 1 -- 172.21.15.83:0/2461894312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ac004d50 msgr2=0x7fe8ac0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:10.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8c4831640 1 --2- 172.21.15.83:0/2461894312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ac004d50 0x7fe8ac0051d0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8007360 tx=0x7fe8b801ce40 comp rx=0 tx=0).stop 2024-04-17T10:47:10.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8bed9d640 1 -- 172.21.15.83:0/2461894312 reap_dead start 2024-04-17T10:47:10.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8c4831640 1 -- 172.21.15.83:0/2461894312 shutdown_connections 2024-04-17T10:47:10.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8c4831640 1 -- 172.21.15.83:0/2461894312 >> 172.21.15.83:0/2461894312 conn(0x7fe8c00d17b0 msgr2=0x7fe8c00d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:10.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8c4831640 1 -- 172.21.15.83:0/2461894312 shutdown_connections 2024-04-17T10:47:10.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8c4831640 1 -- 172.21.15.83:0/2461894312 wait complete. 2024-04-17T10:47:10.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8c4831640 1 Processor -- start 2024-04-17T10:47:10.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8c4831640 1 -- start start 2024-04-17T10:47:10.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8c4831640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8ac004d50 0x7fe8c00cb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:10.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8c4831640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8c00b6f90 0x7fe8c00b5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:10.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8c4831640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c0158d00 0x7fe8c01595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:10.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8c4831640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c00b1d20 con 0x7fe8c0158d00 2024-04-17T10:47:10.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8c4831640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c00b1a40 con 0x7fe8ac004d50 2024-04-17T10:47:10.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8c4831640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c00d22d0 con 0x7fe8c00b6f90 2024-04-17T10:47:10.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8bed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c0158d00 0x7fe8c01595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:10.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8bed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c0158d00 0x7fe8c01595e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58930/0 (socket says 172.21.15.83:58930) 2024-04-17T10:47:10.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8bed9d640 1 -- 172.21.15.83:0/2733373957 learned_addr learned my addr 172.21.15.83:0/2733373957 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:10.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8be59c640 1 --2- 172.21.15.83:0/2733373957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8ac004d50 0x7fe8c00cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:10.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8b5d9b640 1 --2- 172.21.15.83:0/2733373957 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8c00b6f90 0x7fe8c00b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:10.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8bed9d640 1 -- 172.21.15.83:0/2733373957 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8c00b6f90 msgr2=0x7fe8c00b5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:10.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8bed9d640 1 --2- 172.21.15.83:0/2733373957 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8c00b6f90 0x7fe8c00b5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:10.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8bed9d640 1 -- 172.21.15.83:0/2733373957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8ac004d50 msgr2=0x7fe8c00cb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:10.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8bed9d640 1 --2- 172.21.15.83:0/2733373957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8ac004d50 0x7fe8c00cb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:10.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8bed9d640 1 -- 172.21.15.83:0/2733373957 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c006f7b0 con 0x7fe8c0158d00 2024-04-17T10:47:10.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8b5d9b640 1 --2- 172.21.15.83:0/2733373957 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8c00b6f90 0x7fe8c00b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:10.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8bed9d640 1 --2- 172.21.15.83:0/2733373957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c0158d00 0x7fe8c01595e0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fe8b80b0040 tx=0x7fe8ac005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:10.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8b77fe640 1 -- 172.21.15.83:0/2733373957 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b80b2040 con 0x7fe8c0158d00 2024-04-17T10:47:10.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.163+0000 7fe8b77fe640 1 -- 172.21.15.83:0/2733373957 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b80a5b40 con 0x7fe8c0158d00 2024-04-17T10:47:10.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.167+0000 7fe8c4831640 1 -- 172.21.15.83:0/2733373957 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c00b5ef0 con 0x7fe8c0158d00 2024-04-17T10:47:10.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.167+0000 7fe8b77fe640 1 -- 172.21.15.83:0/2733373957 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8009be0 con 0x7fe8c0158d00 2024-04-17T10:47:10.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.167+0000 7fe8c4831640 1 -- 172.21.15.83:0/2733373957 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c00b64b0 con 0x7fe8c0158d00 2024-04-17T10:47:10.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.167+0000 7fe8c4831640 1 -- 172.21.15.83:0/2733373957 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c0074f90 con 0x7fe8c0158d00 2024-04-17T10:47:10.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.167+0000 7fe8b77fe640 1 -- 172.21.15.83:0/2733373957 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe8b80bf060 con 0x7fe8c0158d00 2024-04-17T10:47:10.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.167+0000 7fe8b77fe640 1 --2- 172.21.15.83:0/2733373957 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe8a0066f90 0x7fe8a0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:10.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.167+0000 7fe8be59c640 1 --2- 172.21.15.83:0/2733373957 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe8a0066f90 0x7fe8a0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:10.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.167+0000 7fe8b77fe640 1 -- 172.21.15.83:0/2733373957 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fe8b811f1b0 con 0x7fe8c0158d00 2024-04-17T10:47:10.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.167+0000 7fe8be59c640 1 --2- 172.21.15.83:0/2733373957 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe8a0066f90 0x7fe8a0069450 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8000c10 tx=0x7fe8a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:10.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.171+0000 7fe8b77fe640 1 -- 172.21.15.83:0/2733373957 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe8b8016b50 con 0x7fe8c0158d00 2024-04-17T10:47:10.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.379+0000 7fe8c4831640 1 -- 172.21.15.83:0/2733373957 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe8c0072690 con 0x7fe8c0158d00 2024-04-17T10:47:10.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.379+0000 7fe8b77fe640 1 -- 172.21.15.83:0/2733373957 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1810 (secure 0 0 0) 0x7fe8b80f0110 con 0x7fe8c0158d00 2024-04-17T10:47:10.376 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:47:10.376 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":563,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":91,"num_osds":8,"num_up_osds":8,"osd_up_since":1713350279,"num_in_osds":8,"osd_in_since":1713350270,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077750389,"bytes_used":2689982464,"bytes_avail":770404130816,"bytes_total":773094113280,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":873555,"recovering_keys_per_sec":0,"num_objects_recovered":10,"num_bytes_recovered":10485760,"num_keys_recovered":0,"write_bytes_sec":38800890,"read_op_per_sec":0,"write_op_per_sec":39},"fsmap":{"epoch":56,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7575}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-04-17T10:47:04.340508+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/scrub_control_test_path]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-17T10:47:10.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.383+0000 7fe8c4831640 1 -- 172.21.15.83:0/2733373957 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe8a0066f90 msgr2=0x7fe8a0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:10.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.383+0000 7fe8c4831640 1 --2- 172.21.15.83:0/2733373957 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe8a0066f90 0x7fe8a0069450 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8000c10 tx=0x7fe8a8009040 comp rx=0 tx=0).stop 2024-04-17T10:47:10.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.383+0000 7fe8c4831640 1 -- 172.21.15.83:0/2733373957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c0158d00 msgr2=0x7fe8c01595e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:10.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.383+0000 7fe8c4831640 1 --2- 172.21.15.83:0/2733373957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c0158d00 0x7fe8c01595e0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fe8b80b0040 tx=0x7fe8ac005840 comp rx=0 tx=0).stop 2024-04-17T10:47:10.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.383+0000 7fe8c4831640 1 -- 172.21.15.83:0/2733373957 shutdown_connections 2024-04-17T10:47:10.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.383+0000 7fe8c4831640 1 --2- 172.21.15.83:0/2733373957 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe8a0066f90 0x7fe8a0069450 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:10.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.383+0000 7fe8c4831640 1 --2- 172.21.15.83:0/2733373957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c0158d00 0x7fe8c01595e0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:10.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.383+0000 7fe8c4831640 1 --2- 172.21.15.83:0/2733373957 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8c00b6f90 0x7fe8c00b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:10.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.383+0000 7fe8c4831640 1 --2- 172.21.15.83:0/2733373957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8ac004d50 0x7fe8c00cb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:10.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.383+0000 7fe8c4831640 1 -- 172.21.15.83:0/2733373957 >> 172.21.15.83:0/2733373957 conn(0x7fe8c00d17b0 msgr2=0x7fe8c006e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:10.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.383+0000 7fe8c4831640 1 -- 172.21.15.83:0/2733373957 shutdown_connections 2024-04-17T10:47:10.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:10.383+0000 7fe8c4831640 1 -- 172.21.15.83:0/2733373957 wait complete. 2024-04-17T10:47:10.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'idle+waiting paths [/scrub_control_test_path]'} 2024-04-17T10:47:11.394 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:47:11.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.523+0000 7f0d9d3d2640 1 Processor -- start 2024-04-17T10:47:11.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.527+0000 7f0d9d3d2640 1 -- start start 2024-04-17T10:47:11.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.527+0000 7f0d9d3d2640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d9815dfa0 0x7f0d980b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:11.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.527+0000 7f0d9d3d2640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0d980ce980 0x7f0d980b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:11.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.527+0000 7f0d9d3d2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0d980cee00 0x7f0d980b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:11.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.527+0000 7f0d9d3d2640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d98162a00 con 0x7f0d980cee00 2024-04-17T10:47:11.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.527+0000 7f0d9d3d2640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d98161e80 con 0x7f0d980ce980 2024-04-17T10:47:11.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.527+0000 7f0d9d3d2640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d98162160 con 0x7f0d9815dfa0 2024-04-17T10:47:11.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.527+0000 7f0d977fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0d980cee00 0x7f0d980b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:11.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.527+0000 7f0d977fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0d980cee00 0x7f0d980b5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58940/0 (socket says 172.21.15.83:58940) 2024-04-17T10:47:11.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.527+0000 7f0d977fe640 1 -- 172.21.15.83:0/184985985 learned_addr learned my addr 172.21.15.83:0/184985985 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:11.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.527+0000 7f0d967fc640 1 --2- 172.21.15.83:0/184985985 >> v2:172.21.15.167:3300/0 conn(0x7f0d980ce980 0x7f0d980b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:11.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.527+0000 7f0d96ffd640 1 --2- 172.21.15.83:0/184985985 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d9815dfa0 0x7f0d980b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:11.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.527+0000 7f0d977fe640 1 -- 172.21.15.83:0/184985985 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d9815dfa0 msgr2=0x7f0d980b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:11.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.527+0000 7f0d977fe640 1 --2- 172.21.15.83:0/184985985 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d9815dfa0 0x7f0d980b6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:11.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.527+0000 7f0d977fe640 1 -- 172.21.15.83:0/184985985 >> v2:172.21.15.167:3300/0 conn(0x7f0d980ce980 msgr2=0x7f0d980b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:11.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.527+0000 7f0d977fe640 1 --2- 172.21.15.83:0/184985985 >> v2:172.21.15.167:3300/0 conn(0x7f0d980ce980 0x7f0d980b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:11.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.527+0000 7f0d977fe640 1 -- 172.21.15.83:0/184985985 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d98162440 con 0x7f0d980cee00 2024-04-17T10:47:11.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.527+0000 7f0d967fc640 1 --2- 172.21.15.83:0/184985985 >> v2:172.21.15.167:3300/0 conn(0x7f0d980ce980 0x7f0d980b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:47:11.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.531+0000 7f0d977fe640 1 --2- 172.21.15.83:0/184985985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d980cee00 0x7f0d980b5970 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f0d9806db90 tx=0x7f0d9001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e0de4fd2c43aa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:11.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.531+0000 7f0d95ffb640 1 -- 172.21.15.83:0/184985985 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d900a5750 con 0x7f0d980cee00 2024-04-17T10:47:11.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.531+0000 7f0d95ffb640 1 -- 172.21.15.83:0/184985985 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d900a58f0 con 0x7f0d980cee00 2024-04-17T10:47:11.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.531+0000 7f0d95ffb640 1 -- 172.21.15.83:0/184985985 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d900a5cc0 con 0x7f0d980cee00 2024-04-17T10:47:11.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.531+0000 7f0d9d3d2640 1 -- 172.21.15.83:0/184985985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d980cee00 msgr2=0x7f0d980b5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:11.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.531+0000 7f0d9d3d2640 1 --2- 172.21.15.83:0/184985985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d980cee00 0x7f0d980b5970 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f0d9806db90 tx=0x7f0d9001c930 comp rx=0 tx=0).stop 2024-04-17T10:47:11.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.531+0000 7f0d9d3d2640 1 -- 172.21.15.83:0/184985985 shutdown_connections 2024-04-17T10:47:11.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.531+0000 7f0d9d3d2640 1 --2- 172.21.15.83:0/184985985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d980cee00 0x7f0d980b5970 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:11.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.531+0000 7f0d9d3d2640 1 --2- 172.21.15.83:0/184985985 >> v2:172.21.15.167:3300/0 conn(0x7f0d980ce980 0x7f0d980b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:11.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.531+0000 7f0d9d3d2640 1 --2- 172.21.15.83:0/184985985 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d9815dfa0 0x7f0d980b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:11.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.531+0000 7f0d9d3d2640 1 -- 172.21.15.83:0/184985985 >> 172.21.15.83:0/184985985 conn(0x7f0d980d0600 msgr2=0x7f0d980d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:11.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.531+0000 7f0d9d3d2640 1 -- 172.21.15.83:0/184985985 shutdown_connections 2024-04-17T10:47:11.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.531+0000 7f0d9d3d2640 1 -- 172.21.15.83:0/184985985 wait complete. 2024-04-17T10:47:11.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.531+0000 7f0d9d3d2640 1 Processor -- start 2024-04-17T10:47:11.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.531+0000 7f0d9d3d2640 1 -- start start 2024-04-17T10:47:11.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.531+0000 7f0d9d3d2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d980ce980 0x7f0d980c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:11.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.531+0000 7f0d9d3d2640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d980cee00 0x7f0d980c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:11.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.531+0000 7f0d9d3d2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d9815dfa0 0x7f0d980c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:11.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.531+0000 7f0d9d3d2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d980d2540 con 0x7f0d980ce980 2024-04-17T10:47:11.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.531+0000 7f0d9d3d2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d98161b10 con 0x7f0d9815dfa0 2024-04-17T10:47:11.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.531+0000 7f0d9d3d2640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d980d26c0 con 0x7f0d980cee00 2024-04-17T10:47:11.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.531+0000 7f0d96ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d980ce980 0x7f0d980c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:11.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.531+0000 7f0d96ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d980ce980 0x7f0d980c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58956/0 (socket says 172.21.15.83:58956) 2024-04-17T10:47:11.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.531+0000 7f0d96ffd640 1 -- 172.21.15.83:0/2129907677 learned_addr learned my addr 172.21.15.83:0/2129907677 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:11.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.531+0000 7f0d967fc640 1 --2- 172.21.15.83:0/2129907677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d980cee00 0x7f0d980c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:11.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.535+0000 7f0d977fe640 1 --2- 172.21.15.83:0/2129907677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d9815dfa0 0x7f0d980c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:11.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.535+0000 7f0d96ffd640 1 -- 172.21.15.83:0/2129907677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d980cee00 msgr2=0x7f0d980c76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:11.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.535+0000 7f0d96ffd640 1 --2- 172.21.15.83:0/2129907677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d980cee00 0x7f0d980c76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:11.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.535+0000 7f0d96ffd640 1 -- 172.21.15.83:0/2129907677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d9815dfa0 msgr2=0x7f0d980c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:11.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.535+0000 7f0d96ffd640 1 --2- 172.21.15.83:0/2129907677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d9815dfa0 0x7f0d980c7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:11.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.535+0000 7f0d96ffd640 1 -- 172.21.15.83:0/2129907677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d981576e0 con 0x7f0d980ce980 2024-04-17T10:47:11.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.535+0000 7f0d977fe640 1 --2- 172.21.15.83:0/2129907677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d9815dfa0 0x7f0d980c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:11.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.535+0000 7f0d96ffd640 1 --2- 172.21.15.83:0/2129907677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d980ce980 0x7f0d980c9d20 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f0d8800d620 tx=0x7f0d8800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:11.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.535+0000 7f0d7f7fe640 1 -- 172.21.15.83:0/2129907677 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8800b620 con 0x7f0d980ce980 2024-04-17T10:47:11.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.535+0000 7f0d7f7fe640 1 -- 172.21.15.83:0/2129907677 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d88019070 con 0x7f0d980ce980 2024-04-17T10:47:11.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.535+0000 7f0d9d3d2640 1 -- 172.21.15.83:0/2129907677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d980cb870 con 0x7f0d980ce980 2024-04-17T10:47:11.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.535+0000 7f0d7f7fe640 1 -- 172.21.15.83:0/2129907677 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8800b9e0 con 0x7f0d980ce980 2024-04-17T10:47:11.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.535+0000 7f0d9d3d2640 1 -- 172.21.15.83:0/2129907677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d980cbe30 con 0x7f0d980ce980 2024-04-17T10:47:11.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.535+0000 7f0d9d3d2640 1 -- 172.21.15.83:0/2129907677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d98074f90 con 0x7f0d980ce980 2024-04-17T10:47:11.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.535+0000 7f0d7f7fe640 1 -- 172.21.15.83:0/2129907677 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0d8800bb80 con 0x7f0d980ce980 2024-04-17T10:47:11.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.535+0000 7f0d7f7fe640 1 --2- 172.21.15.83:0/2129907677 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0d84066f60 0x7f0d84069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:11.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.535+0000 7f0d7f7fe640 1 -- 172.21.15.83:0/2129907677 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f0d88085e30 con 0x7f0d980ce980 2024-04-17T10:47:11.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.535+0000 7f0d967fc640 1 --2- 172.21.15.83:0/2129907677 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0d84066f60 0x7f0d84069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:11.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.535+0000 7f0d967fc640 1 --2- 172.21.15.83:0/2129907677 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0d84066f60 0x7f0d84069420 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0d980c9880 tx=0x7f0d8c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:11.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.539+0000 7f0d7f7fe640 1 -- 172.21.15.83:0/2129907677 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0d88056f90 con 0x7f0d980ce980 2024-04-17T10:47:11.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.715+0000 7f0d9d3d2640 1 -- 172.21.15.83:0/2129907677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d980c7220 con 0x7f0d980ce980 2024-04-17T10:47:11.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.715+0000 7f0d7f7fe640 1 -- 172.21.15.83:0/2129907677 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f0d88056890 con 0x7f0d980ce980 2024-04-17T10:47:11.713 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:47:11.713 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5429,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6834/3936640219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3936640219},{"type":"v1","addr":"172.21.15.167:6834","nonce":3936640219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":5998,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4152882285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6833","nonce":4152882285},{"type":"v1","addr":"172.21.15.167:6835","nonce":4152882285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1562645860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1562645860},{"type":"v1","addr":"172.21.15.83:6837","nonce":1562645860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:46:36.578918+0000","modified":"2024-04-17T10:46:38.561701+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7575":{"gid":7575,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/1056175765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1056175765},{"type":"v1","addr":"172.21.15.83:6836","nonce":1056175765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-17T10:47:11.713 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 56 2024-04-17T10:47:11.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.719+0000 7f0d9d3d2640 1 -- 172.21.15.83:0/2129907677 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0d84066f60 msgr2=0x7f0d84069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:11.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.719+0000 7f0d9d3d2640 1 --2- 172.21.15.83:0/2129907677 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0d84066f60 0x7f0d84069420 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0d980c9880 tx=0x7f0d8c008040 comp rx=0 tx=0).stop 2024-04-17T10:47:11.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.719+0000 7f0d9d3d2640 1 -- 172.21.15.83:0/2129907677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d980ce980 msgr2=0x7f0d980c9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:11.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.719+0000 7f0d9d3d2640 1 --2- 172.21.15.83:0/2129907677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d980ce980 0x7f0d980c9d20 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f0d8800d620 tx=0x7f0d8800dae0 comp rx=0 tx=0).stop 2024-04-17T10:47:11.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.719+0000 7f0d9d3d2640 1 -- 172.21.15.83:0/2129907677 shutdown_connections 2024-04-17T10:47:11.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.719+0000 7f0d9d3d2640 1 --2- 172.21.15.83:0/2129907677 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0d84066f60 0x7f0d84069420 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:11.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.719+0000 7f0d9d3d2640 1 --2- 172.21.15.83:0/2129907677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d9815dfa0 0x7f0d980c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:11.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.719+0000 7f0d9d3d2640 1 --2- 172.21.15.83:0/2129907677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d980cee00 0x7f0d980c76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:11.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.719+0000 7f0d9d3d2640 1 --2- 172.21.15.83:0/2129907677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d980ce980 0x7f0d980c9d20 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:11.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.719+0000 7f0d9d3d2640 1 -- 172.21.15.83:0/2129907677 >> 172.21.15.83:0/2129907677 conn(0x7f0d980d0600 msgr2=0x7f0d9806ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:11.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.719+0000 7f0d9d3d2640 1 -- 172.21.15.83:0/2129907677 shutdown_connections 2024-04-17T10:47:11.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.719+0000 7f0d9d3d2640 1 -- 172.21.15.83:0/2129907677 wait complete. 2024-04-17T10:47:11.730 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-17T10:47:11.731 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-17T10:47:11.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.847+0000 7f38e3c96640 1 Processor -- start 2024-04-17T10:47:11.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.847+0000 7f38e3c96640 1 -- start start 2024-04-17T10:47:11.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.847+0000 7f38e3c96640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f38dc072330 0x7f38dc0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:11.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.847+0000 7f38e3c96640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38dc0cfea0 con 0x7f38dc0cfa10 2024-04-17T10:47:11.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.847+0000 7f38e3c96640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38dc06e320 con 0x7f38dc15ba30 2024-04-17T10:47:11.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.847+0000 7f38e3c96640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38dc071ea0 con 0x7f38dc072330 2024-04-17T10:47:11.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.847+0000 7f38e1231640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f38dc072330 0x7f38dc0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:11.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.847+0000 7f38e1231640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f38dc072330 0x7f38dc0cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58958/0 (socket says 172.21.15.83:58958) 2024-04-17T10:47:11.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.847+0000 7f38e1231640 1 -- 172.21.15.83:0/2037546835 learned_addr learned my addr 172.21.15.83:0/2037546835 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:11.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.851+0000 7f38e1231640 1 -- 172.21.15.83:0/2037546835 >> v1:172.21.15.167:6789/0 conn(0x7f38dc15ba30 legacy=0x7f38dc15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:11.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.851+0000 7f38e1231640 1 -- 172.21.15.83:0/2037546835 >> v1:172.21.15.83:6789/0 conn(0x7f38dc0cfa10 legacy=0x7f38dc06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:11.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.851+0000 7f38e0a30640 1 -- 172.21.15.83:0/2037546835 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1684049453 0 0) 0x7f38dc0cfea0 con 0x7f38dc0cfa10 2024-04-17T10:47:11.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.851+0000 7f38e1231640 1 -- 172.21.15.83:0/2037546835 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38dc06e040 con 0x7f38dc072330 2024-04-17T10:47:11.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.851+0000 7f38e1231640 1 --2- 172.21.15.83:0/2037546835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38dc072330 0x7f38dc0cf4d0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f38cc009870 tx=0x7f38cc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5bd137789b46e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:11.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.851+0000 7f38e0a30640 1 -- 172.21.15.83:0/2037546835 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc0b1070 con 0x7f38dc072330 2024-04-17T10:47:11.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.851+0000 7f38e0a30640 1 -- 172.21.15.83:0/2037546835 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38cc0ac600 con 0x7f38dc072330 2024-04-17T10:47:11.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.851+0000 7f38e0a30640 1 -- 172.21.15.83:0/2037546835 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc0ac9d0 con 0x7f38dc072330 2024-04-17T10:47:11.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.851+0000 7f38e3c96640 1 -- 172.21.15.83:0/2037546835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38dc072330 msgr2=0x7f38dc0cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:11.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.851+0000 7f38e3c96640 1 --2- 172.21.15.83:0/2037546835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38dc072330 0x7f38dc0cf4d0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f38cc009870 tx=0x7f38cc01c490 comp rx=0 tx=0).stop 2024-04-17T10:47:11.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.851+0000 7f38e3c96640 1 -- 172.21.15.83:0/2037546835 shutdown_connections 2024-04-17T10:47:11.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.851+0000 7f38e3c96640 1 --2- 172.21.15.83:0/2037546835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38dc072330 0x7f38dc0cf4d0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:11.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.851+0000 7f38e3c96640 1 -- 172.21.15.83:0/2037546835 >> 172.21.15.83:0/2037546835 conn(0x7f38dc06abe0 msgr2=0x7f38dc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:11.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.851+0000 7f38e3c96640 1 -- 172.21.15.83:0/2037546835 shutdown_connections 2024-04-17T10:47:11.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.851+0000 7f38e3c96640 1 -- 172.21.15.83:0/2037546835 wait complete. 2024-04-17T10:47:11.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.855+0000 7f38e3c96640 1 Processor -- start 2024-04-17T10:47:11.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.855+0000 7f38e3c96640 1 -- start start 2024-04-17T10:47:11.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.855+0000 7f38e3c96640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38dc072330 0x7f38dc150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:11.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.855+0000 7f38e3c96640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38dc0cfa10 0x7f38dc1581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:11.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.855+0000 7f38e3c96640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38dc15ba30 0x7f38dc158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:11.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.855+0000 7f38e3c96640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc06de20 con 0x7f38dc072330 2024-04-17T10:47:11.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.855+0000 7f38e3c96640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc06dca0 con 0x7f38dc0cfa10 2024-04-17T10:47:11.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.855+0000 7f38e3c96640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc169530 con 0x7f38dc15ba30 2024-04-17T10:47:11.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.855+0000 7f38e1a32640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38dc072330 0x7f38dc150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:11.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.855+0000 7f38e1a32640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38dc072330 0x7f38dc150e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58966/0 (socket says 172.21.15.83:58966) 2024-04-17T10:47:11.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.855+0000 7f38e1a32640 1 -- 172.21.15.83:0/3035608522 learned_addr learned my addr 172.21.15.83:0/3035608522 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:11.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.855+0000 7f38e1a32640 1 -- 172.21.15.83:0/3035608522 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38dc15ba30 msgr2=0x7f38dc158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:11.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.855+0000 7f38e1a32640 1 --2- 172.21.15.83:0/3035608522 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38dc15ba30 0x7f38dc158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:11.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.855+0000 7f38e1a32640 1 -- 172.21.15.83:0/3035608522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38dc0cfa10 msgr2=0x7f38dc1581c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:11.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.855+0000 7f38e1231640 1 --2- 172.21.15.83:0/3035608522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38dc0cfa10 0x7f38dc1581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:11.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.855+0000 7f38e1a32640 1 --2- 172.21.15.83:0/3035608522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38dc0cfa10 0x7f38dc1581c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:11.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.855+0000 7f38e1a32640 1 -- 172.21.15.83:0/3035608522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38dc158c40 con 0x7f38dc072330 2024-04-17T10:47:11.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.855+0000 7f38e1a32640 1 --2- 172.21.15.83:0/3035608522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38dc072330 0x7f38dc150e90 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f38d800b3e0 tx=0x7f38d800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:11.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.855+0000 7f38d27fc640 1 -- 172.21.15.83:0/3035608522 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d8015020 con 0x7f38dc072330 2024-04-17T10:47:11.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.855+0000 7f38d27fc640 1 -- 172.21.15.83:0/3035608522 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d8011850 con 0x7f38dc072330 2024-04-17T10:47:11.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.855+0000 7f38e3c96640 1 -- 172.21.15.83:0/3035608522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38dc157120 con 0x7f38dc072330 2024-04-17T10:47:11.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.855+0000 7f38d27fc640 1 -- 172.21.15.83:0/3035608522 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d8011c40 con 0x7f38dc072330 2024-04-17T10:47:11.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.855+0000 7f38e3c96640 1 -- 172.21.15.83:0/3035608522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38dc1576e0 con 0x7f38dc072330 2024-04-17T10:47:11.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.855+0000 7f38e3c96640 1 -- 172.21.15.83:0/3035608522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38dc074f90 con 0x7f38dc072330 2024-04-17T10:47:11.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.855+0000 7f38d27fc640 1 -- 172.21.15.83:0/3035608522 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f38d8018020 con 0x7f38dc072330 2024-04-17T10:47:11.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.859+0000 7f38d27fc640 1 --2- 172.21.15.83:0/3035608522 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f38c0066f90 0x7f38c0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:11.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.859+0000 7f38d27fc640 1 -- 172.21.15.83:0/3035608522 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f38d8084f50 con 0x7f38dc072330 2024-04-17T10:47:11.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.859+0000 7f38e1231640 1 --2- 172.21.15.83:0/3035608522 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f38c0066f90 0x7f38c0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:11.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.859+0000 7f38e1231640 1 --2- 172.21.15.83:0/3035608522 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f38c0066f90 0x7f38c0069450 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f38cc0ae000 tx=0x7f38cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:11.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:11.863+0000 7f38d27fc640 1 -- 172.21.15.83:0/3035608522 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f38d801f020 con 0x7f38dc072330 2024-04-17T10:47:12.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.059+0000 7f38e3c96640 1 -- 172.21.15.83:0/3035608522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f38dc155150 con 0x7f38dc072330 2024-04-17T10:47:12.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.059+0000 7f38d27fc640 1 -- 172.21.15.83:0/3035608522 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1748 (secure 0 0 0) 0x7f38d80172c0 con 0x7f38dc072330 2024-04-17T10:47:12.055 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:47:12.055 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":564,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":91,"num_osds":8,"num_up_osds":8,"osd_up_since":1713350279,"num_in_osds":8,"osd_in_since":1713350270,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1078397078,"bytes_used":2691276800,"bytes_avail":770402836480,"bytes_total":773094113280,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":873551,"recovering_keys_per_sec":0,"num_objects_recovered":10,"num_bytes_recovered":10485760,"num_keys_recovered":0,"write_bytes_sec":38854622,"read_op_per_sec":0,"write_op_per_sec":40},"fsmap":{"epoch":56,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7575}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":18,"modified":"2024-04-17T10:47:10.343010+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-17T10:47:12.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.059+0000 7f38e3c96640 1 -- 172.21.15.83:0/3035608522 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f38c0066f90 msgr2=0x7f38c0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:12.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.059+0000 7f38e3c96640 1 --2- 172.21.15.83:0/3035608522 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f38c0066f90 0x7f38c0069450 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f38cc0ae000 tx=0x7f38cc008040 comp rx=0 tx=0).stop 2024-04-17T10:47:12.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.059+0000 7f38e3c96640 1 -- 172.21.15.83:0/3035608522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38dc072330 msgr2=0x7f38dc150e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:12.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.059+0000 7f38e3c96640 1 --2- 172.21.15.83:0/3035608522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38dc072330 0x7f38dc150e90 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f38d800b3e0 tx=0x7f38d800b8a0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.063+0000 7f38e3c96640 1 -- 172.21.15.83:0/3035608522 shutdown_connections 2024-04-17T10:47:12.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.063+0000 7f38e3c96640 1 --2- 172.21.15.83:0/3035608522 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f38c0066f90 0x7f38c0069450 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.063+0000 7f38e3c96640 1 --2- 172.21.15.83:0/3035608522 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38dc15ba30 0x7f38dc158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.063+0000 7f38e3c96640 1 --2- 172.21.15.83:0/3035608522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38dc0cfa10 0x7f38dc1581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.063+0000 7f38e3c96640 1 --2- 172.21.15.83:0/3035608522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38dc072330 0x7f38dc150e90 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.063+0000 7f38e3c96640 1 -- 172.21.15.83:0/3035608522 >> 172.21.15.83:0/3035608522 conn(0x7f38dc06abe0 msgr2=0x7f38dc06c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:12.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.063+0000 7f38e3c96640 1 -- 172.21.15.83:0/3035608522 shutdown_connections 2024-04-17T10:47:12.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.063+0000 7f38e3c96640 1 -- 172.21.15.83:0/3035608522 wait complete. 2024-04-17T10:47:12.072 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-17T10:47:12.073 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T10:47:12.073 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:47:12.118 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:47:12.167 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T10:47:12.167 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:47:12.167 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:47:12.172 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:47:12.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.339+0000 7f4b1bf1d640 1 Processor -- start 2024-04-17T10:47:12.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.339+0000 7f4b1bf1d640 1 -- start start 2024-04-17T10:47:12.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b1bf1d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4b14146b40 0x7f4b1406d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:12.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b1bf1d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b14146f20 0x7f4b140720d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:12.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b1bf1d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b14161ae0 con 0x7f4b1406abe0 2024-04-17T10:47:12.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b1bf1d640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b14161dc0 con 0x7f4b14146b40 2024-04-17T10:47:12.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b1bf1d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b141620a0 con 0x7f4b14146f20 2024-04-17T10:47:12.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b1a4ba640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f4b1406abe0 0x7f4b140d07d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40550/0 (socket says 172.21.15.83:40550) 2024-04-17T10:47:12.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b1a4ba640 1 -- 172.21.15.83:0/2125542556 learned_addr learned my addr 172.21.15.83:0/2125542556 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:12.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b18cb7640 1 -- 172.21.15.83:0/2125542556 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3587453066 0 0) 0x7f4b14161ae0 con 0x7f4b1406abe0 2024-04-17T10:47:12.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b19cb9640 1 --2- 172.21.15.83:0/2125542556 >> v2:172.21.15.167:3300/0 conn(0x7f4b14146b40 0x7f4b1406d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:12.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b18cb7640 1 -- 172.21.15.83:0/2125542556 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b00003530 con 0x7f4b1406abe0 2024-04-17T10:47:12.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b194b8640 1 --2- 172.21.15.83:0/2125542556 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b14146f20 0x7f4b140720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:12.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b18cb7640 1 -- 172.21.15.83:0/2125542556 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 867116588 0 0) 0x7f4b00003530 con 0x7f4b1406abe0 2024-04-17T10:47:12.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b18cb7640 1 -- 172.21.15.83:0/2125542556 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b14146f20 msgr2=0x7f4b140720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:12.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b18cb7640 1 --2- 172.21.15.83:0/2125542556 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b14146f20 0x7f4b140720d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b18cb7640 1 -- 172.21.15.83:0/2125542556 >> v2:172.21.15.167:3300/0 conn(0x7f4b14146b40 msgr2=0x7f4b1406d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:12.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b18cb7640 1 --2- 172.21.15.83:0/2125542556 >> v2:172.21.15.167:3300/0 conn(0x7f4b14146b40 0x7f4b1406d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b18cb7640 1 -- 172.21.15.83:0/2125542556 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b14162380 con 0x7f4b1406abe0 2024-04-17T10:47:12.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b18cb7640 1 -- 172.21.15.83:0/2125542556 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f4b10001680 con 0x7f4b1406abe0 2024-04-17T10:47:12.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b18cb7640 1 -- 172.21.15.83:0/2125542556 >> v1:172.21.15.83:6789/0 conn(0x7f4b1406abe0 legacy=0x7f4b140d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:12.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b18cb7640 1 --2- 172.21.15.83:0/2125542556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b000039c0 0x7f4b00003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:12.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b18cb7640 1 --2- 172.21.15.83:0/2125542556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b00004810 0x7f4b00004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:12.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b18cb7640 1 --2- 172.21.15.83:0/2125542556 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b14146f20 0x7f4b000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:12.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b18cb7640 1 -- 172.21.15.83:0/2125542556 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b14161dc0 con 0x7f4b000039c0 2024-04-17T10:47:12.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b18cb7640 1 -- 172.21.15.83:0/2125542556 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b141620a0 con 0x7f4b00004810 2024-04-17T10:47:12.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b18cb7640 1 -- 172.21.15.83:0/2125542556 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0000ba00 con 0x7f4b14146f20 2024-04-17T10:47:12.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b19cb9640 1 --2- 172.21.15.83:0/2125542556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b000039c0 0x7f4b00003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:12.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b194b8640 1 --2- 172.21.15.83:0/2125542556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b00004810 0x7f4b00004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:12.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b1a4ba640 1 --2- 172.21.15.83:0/2125542556 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b14146f20 0x7f4b000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:12.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b19cb9640 1 -- 172.21.15.83:0/2125542556 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b14146f20 msgr2=0x7f4b000051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:12.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b19cb9640 1 --2- 172.21.15.83:0/2125542556 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b14146f20 0x7f4b000051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b19cb9640 1 -- 172.21.15.83:0/2125542556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b00004810 msgr2=0x7f4b00004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:12.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b19cb9640 1 --2- 172.21.15.83:0/2125542556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b00004810 0x7f4b00004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b19cb9640 1 -- 172.21.15.83:0/2125542556 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b00003530 con 0x7f4b000039c0 2024-04-17T10:47:12.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.343+0000 7f4b194b8640 1 --2- 172.21.15.83:0/2125542556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b00004810 0x7f4b00004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:12.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b19cb9640 1 --2- 172.21.15.83:0/2125542556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b000039c0 0x7f4b00003de0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f4b04009ae0 tx=0x7f4b0401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=761ae24665913d45 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:12.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b18cb7640 1 -- 172.21.15.83:0/2125542556 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b040096b0 con 0x7f4b000039c0 2024-04-17T10:47:12.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b18cb7640 1 -- 172.21.15.83:0/2125542556 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b040a9480 con 0x7f4b000039c0 2024-04-17T10:47:12.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b18cb7640 1 -- 172.21.15.83:0/2125542556 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b040a9870 con 0x7f4b000039c0 2024-04-17T10:47:12.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b1bf1d640 1 -- 172.21.15.83:0/2125542556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b000039c0 msgr2=0x7f4b00003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:12.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b1bf1d640 1 --2- 172.21.15.83:0/2125542556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b000039c0 0x7f4b00003de0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f4b04009ae0 tx=0x7f4b0401cab0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b1a4ba640 1 -- 172.21.15.83:0/2125542556 reap_dead start 2024-04-17T10:47:12.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b1bf1d640 1 -- 172.21.15.83:0/2125542556 shutdown_connections 2024-04-17T10:47:12.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b1bf1d640 1 -- 172.21.15.83:0/2125542556 >> 172.21.15.83:0/2125542556 conn(0x7f4b1406bcf0 msgr2=0x7f4b1406c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:12.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b1bf1d640 1 -- 172.21.15.83:0/2125542556 shutdown_connections 2024-04-17T10:47:12.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b1bf1d640 1 -- 172.21.15.83:0/2125542556 wait complete. 2024-04-17T10:47:12.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b1bf1d640 1 Processor -- start 2024-04-17T10:47:12.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b1bf1d640 1 -- start start 2024-04-17T10:47:12.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b1bf1d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b000039c0 0x7f4b1413e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:12.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b1bf1d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b1413e0e0 0x7f4b1415ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:12.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b1bf1d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b141602e0 0x7f4b14160790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:12.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b1bf1d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b140724a0 con 0x7f4b1413e0e0 2024-04-17T10:47:12.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b1bf1d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b14072320 con 0x7f4b000039c0 2024-04-17T10:47:12.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b1bf1d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b14072620 con 0x7f4b141602e0 2024-04-17T10:47:12.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b194b8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b1413e0e0 0x7f4b1415ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:12.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b194b8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b1413e0e0 0x7f4b1415ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58986/0 (socket says 172.21.15.83:58986) 2024-04-17T10:47:12.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b194b8640 1 -- 172.21.15.83:0/3654332945 learned_addr learned my addr 172.21.15.83:0/3654332945 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:12.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b19cb9640 1 --2- 172.21.15.83:0/3654332945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b000039c0 0x7f4b1413e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:12.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b1a4ba640 1 --2- 172.21.15.83:0/3654332945 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b141602e0 0x7f4b14160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:12.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b194b8640 1 -- 172.21.15.83:0/3654332945 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b141602e0 msgr2=0x7f4b14160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:12.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b194b8640 1 --2- 172.21.15.83:0/3654332945 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b141602e0 0x7f4b14160790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b194b8640 1 -- 172.21.15.83:0/3654332945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b000039c0 msgr2=0x7f4b1413e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:12.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b194b8640 1 --2- 172.21.15.83:0/3654332945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b000039c0 0x7f4b1413e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b194b8640 1 -- 172.21.15.83:0/3654332945 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b1413efc0 con 0x7f4b1413e0e0 2024-04-17T10:47:12.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b1a4ba640 1 --2- 172.21.15.83:0/3654332945 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b141602e0 0x7f4b14160790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:12.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b194b8640 1 --2- 172.21.15.83:0/3654332945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b1413e0e0 0x7f4b1415ce50 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c009f80 tx=0x7f4b0c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:12.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b0a7fc640 1 -- 172.21.15.83:0/3654332945 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c018020 con 0x7f4b1413e0e0 2024-04-17T10:47:12.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b0a7fc640 1 -- 172.21.15.83:0/3654332945 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0c0145f0 con 0x7f4b1413e0e0 2024-04-17T10:47:12.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b1bf1d640 1 -- 172.21.15.83:0/3654332945 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b1413f2a0 con 0x7f4b1413e0e0 2024-04-17T10:47:12.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b0a7fc640 1 -- 172.21.15.83:0/3654332945 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c0149c0 con 0x7f4b1413e0e0 2024-04-17T10:47:12.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.347+0000 7f4b1bf1d640 1 -- 172.21.15.83:0/3654332945 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b1415d390 con 0x7f4b1413e0e0 2024-04-17T10:47:12.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.351+0000 7f4b1bf1d640 1 -- 172.21.15.83:0/3654332945 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b1413c110 con 0x7f4b1413e0e0 2024-04-17T10:47:12.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.351+0000 7f4b0a7fc640 1 -- 172.21.15.83:0/3654332945 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4b0c01b020 con 0x7f4b1413e0e0 2024-04-17T10:47:12.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.351+0000 7f4b0a7fc640 1 --2- 172.21.15.83:0/3654332945 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4af8066f90 0x7f4af8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:12.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.351+0000 7f4b0a7fc640 1 -- 172.21.15.83:0/3654332945 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f4b0c085cf0 con 0x7f4b1413e0e0 2024-04-17T10:47:12.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.351+0000 7f4b19cb9640 1 --2- 172.21.15.83:0/3654332945 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4af8066f90 0x7f4af8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:12.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.351+0000 7f4b19cb9640 1 --2- 172.21.15.83:0/3654332945 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4af8066f90 0x7f4af8069450 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4b040ae000 tx=0x7f4b040046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:12.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.355+0000 7f4b0a7fc640 1 -- 172.21.15.83:0/3654332945 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4b0c08a070 con 0x7f4b1413e0e0 2024-04-17T10:47:12.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.515+0000 7f4b1bf1d640 1 -- 172.21.15.83:0/3654332945 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b14074f90 con 0x7f4b1413e0e0 2024-04-17T10:47:12.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.519+0000 7f4b0a7fc640 1 -- 172.21.15.83:0/3654332945 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+16638 (secure 0 0 0) 0x7f4b0c022020 con 0x7f4b1413e0e0 2024-04-17T10:47:12.513 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:47:12.513 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":91,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:46:55.414653+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:46:34.779174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-17T10:46:35.563972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.10","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:47:12.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.519+0000 7f4b1bf1d640 1 -- 172.21.15.83:0/3654332945 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4af8066f90 msgr2=0x7f4af8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:12.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.519+0000 7f4b1bf1d640 1 --2- 172.21.15.83:0/3654332945 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4af8066f90 0x7f4af8069450 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4b040ae000 tx=0x7f4b040046d0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.519+0000 7f4b1bf1d640 1 -- 172.21.15.83:0/3654332945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b1413e0e0 msgr2=0x7f4b1415ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:12.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.519+0000 7f4b1bf1d640 1 --2- 172.21.15.83:0/3654332945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b1413e0e0 0x7f4b1415ce50 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c009f80 tx=0x7f4b0c007f90 comp rx=0 tx=0).stop 2024-04-17T10:47:12.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.519+0000 7f4b1bf1d640 1 -- 172.21.15.83:0/3654332945 shutdown_connections 2024-04-17T10:47:12.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.519+0000 7f4b1bf1d640 1 --2- 172.21.15.83:0/3654332945 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4af8066f90 0x7f4af8069450 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.519+0000 7f4b1bf1d640 1 --2- 172.21.15.83:0/3654332945 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b141602e0 0x7f4b14160790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.519+0000 7f4b1bf1d640 1 --2- 172.21.15.83:0/3654332945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b1413e0e0 0x7f4b1415ce50 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.519+0000 7f4b1bf1d640 1 --2- 172.21.15.83:0/3654332945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b000039c0 0x7f4b1413e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.519+0000 7f4b1bf1d640 1 -- 172.21.15.83:0/3654332945 >> 172.21.15.83:0/3654332945 conn(0x7f4b1406bcf0 msgr2=0x7f4b141460b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:12.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.519+0000 7f4b1bf1d640 1 -- 172.21.15.83:0/3654332945 shutdown_connections 2024-04-17T10:47:12.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.519+0000 7f4b1bf1d640 1 -- 172.21.15.83:0/3654332945 wait complete. 2024-04-17T10:47:12.532 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:46:34.779174+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:46:35.563972+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:47:12.532 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:47:12.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.659+0000 7fa670e69640 1 Processor -- start 2024-04-17T10:47:12.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.659+0000 7fa670e69640 1 -- start start 2024-04-17T10:47:12.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.659+0000 7fa670e69640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa66c154de0 0x7fa66c06ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:12.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.659+0000 7fa670e69640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa66c0b4210 0x7fa66c06cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:12.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.659+0000 7fa670e69640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa66c15bd70 con 0x7fa66c06b350 2024-04-17T10:47:12.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.659+0000 7fa670e69640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa66c0b4870 con 0x7fa66c0b4210 2024-04-17T10:47:12.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.659+0000 7fa670e69640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa66c15bf60 con 0x7fa66c154de0 2024-04-17T10:47:12.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.659+0000 7fa66ad9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa66c0b4210 0x7fa66c06cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:12.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.659+0000 7fa66ad9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa66c0b4210 0x7fa66c06cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58992/0 (socket says 172.21.15.83:58992) 2024-04-17T10:47:12.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.659+0000 7fa66ad9d640 1 -- 172.21.15.83:0/45297231 learned_addr learned my addr 172.21.15.83:0/45297231 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:12.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.659+0000 7fa66a59c640 1 --2- 172.21.15.83:0/45297231 >> v2:172.21.15.167:3300/0 conn(0x7fa66c154de0 0x7fa66c06ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:12.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.659+0000 7fa66ad9d640 1 -- 172.21.15.83:0/45297231 >> v2:172.21.15.167:3300/0 conn(0x7fa66c154de0 msgr2=0x7fa66c06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:12.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.659+0000 7fa66ad9d640 1 --2- 172.21.15.83:0/45297231 >> v2:172.21.15.167:3300/0 conn(0x7fa66c154de0 0x7fa66c06ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.659+0000 7fa66ad9d640 1 -- 172.21.15.83:0/45297231 >> v1:172.21.15.167:6789/0 conn(0x7fa66c06b350 legacy=0x7fa66c0b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:12.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.659+0000 7fa66ad9d640 1 -- 172.21.15.83:0/45297231 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa66c15afb0 con 0x7fa66c0b4210 2024-04-17T10:47:12.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.663+0000 7fa66ad9d640 1 --2- 172.21.15.83:0/45297231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa66c0b4210 0x7fa66c06cbe0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fa66c0b2de0 tx=0x7fa66401ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36dda0f1c56d1788 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:12.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.663+0000 7fa66959a640 1 -- 172.21.15.83:0/45297231 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66400b2d0 con 0x7fa66c0b4210 2024-04-17T10:47:12.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.663+0000 7fa66959a640 1 -- 172.21.15.83:0/45297231 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa66400b470 con 0x7fa66c0b4210 2024-04-17T10:47:12.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.663+0000 7fa66959a640 1 -- 172.21.15.83:0/45297231 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6640a7aa0 con 0x7fa66c0b4210 2024-04-17T10:47:12.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.663+0000 7fa670e69640 1 -- 172.21.15.83:0/45297231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa66c0b4210 msgr2=0x7fa66c06cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:12.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.663+0000 7fa670e69640 1 --2- 172.21.15.83:0/45297231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa66c0b4210 0x7fa66c06cbe0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fa66c0b2de0 tx=0x7fa66401ebb0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.663+0000 7fa670e69640 1 -- 172.21.15.83:0/45297231 shutdown_connections 2024-04-17T10:47:12.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.663+0000 7fa670e69640 1 --2- 172.21.15.83:0/45297231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa66c0b4210 0x7fa66c06cbe0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.663+0000 7fa670e69640 1 --2- 172.21.15.83:0/45297231 >> v2:172.21.15.167:3300/0 conn(0x7fa66c154de0 0x7fa66c06ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.663+0000 7fa670e69640 1 -- 172.21.15.83:0/45297231 >> 172.21.15.83:0/45297231 conn(0x7fa66c0b15b0 msgr2=0x7fa66c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:12.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.663+0000 7fa670e69640 1 -- 172.21.15.83:0/45297231 shutdown_connections 2024-04-17T10:47:12.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.663+0000 7fa670e69640 1 -- 172.21.15.83:0/45297231 wait complete. 2024-04-17T10:47:12.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.663+0000 7fa670e69640 1 Processor -- start 2024-04-17T10:47:12.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.663+0000 7fa670e69640 1 -- start start 2024-04-17T10:47:12.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.663+0000 7fa670e69640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa66c06b350 0x7fa66c151970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:12.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.663+0000 7fa670e69640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa66c0b4210 0x7fa66c153870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:12.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.663+0000 7fa670e69640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa66c154de0 0x7fa66c14d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:12.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.663+0000 7fa670e69640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c070470 con 0x7fa66c0b4210 2024-04-17T10:47:12.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.663+0000 7fa670e69640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c1551c0 con 0x7fa66c06b350 2024-04-17T10:47:12.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.663+0000 7fa670e69640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c0705f0 con 0x7fa66c154de0 2024-04-17T10:47:12.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.663+0000 7fa669d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa66c0b4210 0x7fa66c153870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:12.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.663+0000 7fa669d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa66c0b4210 0x7fa66c153870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59006/0 (socket says 172.21.15.83:59006) 2024-04-17T10:47:12.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.663+0000 7fa669d9b640 1 -- 172.21.15.83:0/672349657 learned_addr learned my addr 172.21.15.83:0/672349657 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:12.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.663+0000 7fa66a59c640 1 --2- 172.21.15.83:0/672349657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa66c06b350 0x7fa66c151970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:12.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.663+0000 7fa66ad9d640 1 --2- 172.21.15.83:0/672349657 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa66c154de0 0x7fa66c14d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:12.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.663+0000 7fa669d9b640 1 -- 172.21.15.83:0/672349657 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa66c154de0 msgr2=0x7fa66c14d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:12.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.663+0000 7fa669d9b640 1 --2- 172.21.15.83:0/672349657 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa66c154de0 0x7fa66c14d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.663+0000 7fa669d9b640 1 -- 172.21.15.83:0/672349657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa66c06b350 msgr2=0x7fa66c151970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:12.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.663+0000 7fa669d9b640 1 --2- 172.21.15.83:0/672349657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa66c06b350 0x7fa66c151970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.663+0000 7fa669d9b640 1 -- 172.21.15.83:0/672349657 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa66c151eb0 con 0x7fa66c0b4210 2024-04-17T10:47:12.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.667+0000 7fa669d9b640 1 --2- 172.21.15.83:0/672349657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa66c0b4210 0x7fa66c153870 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fa66000c450 tx=0x7fa66000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:12.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.667+0000 7fa65affd640 1 -- 172.21.15.83:0/672349657 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa660012830 con 0x7fa66c0b4210 2024-04-17T10:47:12.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.667+0000 7fa65affd640 1 -- 172.21.15.83:0/672349657 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa660017070 con 0x7fa66c0b4210 2024-04-17T10:47:12.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.667+0000 7fa65affd640 1 -- 172.21.15.83:0/672349657 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa660012c30 con 0x7fa66c0b4210 2024-04-17T10:47:12.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.667+0000 7fa670e69640 1 -- 172.21.15.83:0/672349657 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa66c153db0 con 0x7fa66c0b4210 2024-04-17T10:47:12.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.667+0000 7fa670e69640 1 -- 172.21.15.83:0/672349657 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa66c14b710 con 0x7fa66c0b4210 2024-04-17T10:47:12.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.667+0000 7fa670e69640 1 -- 172.21.15.83:0/672349657 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa66c074f90 con 0x7fa66c0b4210 2024-04-17T10:47:12.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.667+0000 7fa65affd640 1 -- 172.21.15.83:0/672349657 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa66001c460 con 0x7fa66c0b4210 2024-04-17T10:47:12.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.667+0000 7fa65affd640 1 --2- 172.21.15.83:0/672349657 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa64c066f60 0x7fa64c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:12.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.667+0000 7fa65affd640 1 -- 172.21.15.83:0/672349657 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fa660084db0 con 0x7fa66c0b4210 2024-04-17T10:47:12.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.667+0000 7fa66a59c640 1 --2- 172.21.15.83:0/672349657 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa64c066f60 0x7fa64c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:12.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.667+0000 7fa66a59c640 1 --2- 172.21.15.83:0/672349657 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa64c066f60 0x7fa64c069420 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa65c00b930 tx=0x7fa65c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:12.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.671+0000 7fa65affd640 1 -- 172.21.15.83:0/672349657 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa660055db0 con 0x7fa66c0b4210 2024-04-17T10:47:12.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.823+0000 7fa670e69640 1 -- 172.21.15.83:0/672349657 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa66c073bf0 con 0x7fa66c0b4210 2024-04-17T10:47:12.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.827+0000 7fa65affd640 1 -- 172.21.15.83:0/672349657 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa6600556b0 con 0x7fa66c0b4210 2024-04-17T10:47:12.821 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:47:12.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.827+0000 7fa670e69640 1 -- 172.21.15.83:0/672349657 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa64c066f60 msgr2=0x7fa64c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:12.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.827+0000 7fa670e69640 1 --2- 172.21.15.83:0/672349657 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa64c066f60 0x7fa64c069420 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa65c00b930 tx=0x7fa65c008040 comp rx=0 tx=0).stop 2024-04-17T10:47:12.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.827+0000 7fa670e69640 1 -- 172.21.15.83:0/672349657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa66c0b4210 msgr2=0x7fa66c153870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:12.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.827+0000 7fa670e69640 1 --2- 172.21.15.83:0/672349657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa66c0b4210 0x7fa66c153870 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fa66000c450 tx=0x7fa66000c910 comp rx=0 tx=0).stop 2024-04-17T10:47:12.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.827+0000 7fa670e69640 1 -- 172.21.15.83:0/672349657 shutdown_connections 2024-04-17T10:47:12.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.827+0000 7fa670e69640 1 --2- 172.21.15.83:0/672349657 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa64c066f60 0x7fa64c069420 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.827+0000 7fa670e69640 1 --2- 172.21.15.83:0/672349657 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa66c154de0 0x7fa66c14d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.827+0000 7fa670e69640 1 --2- 172.21.15.83:0/672349657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa66c0b4210 0x7fa66c153870 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.827+0000 7fa670e69640 1 --2- 172.21.15.83:0/672349657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa66c06b350 0x7fa66c151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.827+0000 7fa670e69640 1 -- 172.21.15.83:0/672349657 >> 172.21.15.83:0/672349657 conn(0x7fa66c0b15b0 msgr2=0x7fa66c0b39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:12.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.827+0000 7fa670e69640 1 -- 172.21.15.83:0/672349657 shutdown_connections 2024-04-17T10:47:12.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.827+0000 7fa670e69640 1 -- 172.21.15.83:0/672349657 wait complete. 2024-04-17T10:47:12.839 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:47:12.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.951+0000 7f9f23154640 1 Processor -- start 2024-04-17T10:47:12.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.951+0000 7f9f23154640 1 -- start start 2024-04-17T10:47:12.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.951+0000 7f9f23154640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9f1c15b6e0 0x7f9f1c071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:12.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.951+0000 7f9f23154640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f1c15bac0 0x7f9f1c0cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:12.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.951+0000 7f9f23154640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f1c06c1f0 con 0x7f9f1c070470 2024-04-17T10:47:12.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.951+0000 7f9f23154640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f1c0cdd30 con 0x7f9f1c15b6e0 2024-04-17T10:47:12.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.951+0000 7f9f23154640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c070a10 con 0x7f9f1c15bac0 2024-04-17T10:47:12.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.951+0000 7f9f20ef0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9f1c15b6e0 0x7f9f1c071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:12.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.951+0000 7f9f20ef0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9f1c15b6e0 0x7f9f1c071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59022/0 (socket says 172.21.15.83:59022) 2024-04-17T10:47:12.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.951+0000 7f9f20ef0640 1 -- 172.21.15.83:0/3492209702 learned_addr learned my addr 172.21.15.83:0/3492209702 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:12.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.951+0000 7f9f20ef0640 1 -- 172.21.15.83:0/3492209702 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f1c15bac0 msgr2=0x7f9f1c0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:12.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.951+0000 7f9f20ef0640 1 --2- 172.21.15.83:0/3492209702 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f1c15bac0 0x7f9f1c0cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.951+0000 7f9f20ef0640 1 -- 172.21.15.83:0/3492209702 >> v1:172.21.15.167:6789/0 conn(0x7f9f1c070470 legacy=0x7f9f1c06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:12.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.951+0000 7f9f20ef0640 1 -- 172.21.15.83:0/3492209702 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f1c070cf0 con 0x7f9f1c15b6e0 2024-04-17T10:47:12.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.955+0000 7f9f20ef0640 1 --2- 172.21.15.83:0/3492209702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f1c15b6e0 0x7f9f1c071cc0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c0728a0 tx=0x7f9f1401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a02b96ee9afd09b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:12.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.955+0000 7f9f1b7fe640 1 -- 172.21.15.83:0/3492209702 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f140b0070 con 0x7f9f1c15b6e0 2024-04-17T10:47:12.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.955+0000 7f9f1b7fe640 1 -- 172.21.15.83:0/3492209702 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f140ab600 con 0x7f9f1c15b6e0 2024-04-17T10:47:12.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.955+0000 7f9f1b7fe640 1 -- 172.21.15.83:0/3492209702 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f140ab9d0 con 0x7f9f1c15b6e0 2024-04-17T10:47:12.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.955+0000 7f9f23154640 1 -- 172.21.15.83:0/3492209702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f1c15b6e0 msgr2=0x7f9f1c071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:12.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.955+0000 7f9f23154640 1 --2- 172.21.15.83:0/3492209702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f1c15b6e0 0x7f9f1c071cc0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c0728a0 tx=0x7f9f1401c4e0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.955+0000 7f9f23154640 1 -- 172.21.15.83:0/3492209702 shutdown_connections 2024-04-17T10:47:12.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.955+0000 7f9f23154640 1 --2- 172.21.15.83:0/3492209702 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f1c15bac0 0x7f9f1c0cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.955+0000 7f9f23154640 1 --2- 172.21.15.83:0/3492209702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f1c15b6e0 0x7f9f1c071cc0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.955+0000 7f9f23154640 1 -- 172.21.15.83:0/3492209702 >> 172.21.15.83:0/3492209702 conn(0x7f9f1c0b15b0 msgr2=0x7f9f1c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:12.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.955+0000 7f9f23154640 1 -- 172.21.15.83:0/3492209702 shutdown_connections 2024-04-17T10:47:12.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.955+0000 7f9f23154640 1 -- 172.21.15.83:0/3492209702 wait complete. 2024-04-17T10:47:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.955+0000 7f9f23154640 1 Processor -- start 2024-04-17T10:47:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.955+0000 7f9f23154640 1 -- start start 2024-04-17T10:47:12.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.955+0000 7f9f23154640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f1c070470 0x7f9f1c15d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.955+0000 7f9f23154640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f1c15b6e0 0x7f9f1c1500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.955+0000 7f9f23154640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f1c15bac0 0x7f9f1c159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.955+0000 7f9f23154640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c06cee0 con 0x7f9f1c15bac0 2024-04-17T10:47:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.955+0000 7f9f23154640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c06b730 con 0x7f9f1c15b6e0 2024-04-17T10:47:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.955+0000 7f9f23154640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c06d060 con 0x7f9f1c070470 2024-04-17T10:47:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.955+0000 7f9f216f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f1c15bac0 0x7f9f1c159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.955+0000 7f9f216f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f1c15bac0 0x7f9f1c159890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59026/0 (socket says 172.21.15.83:59026) 2024-04-17T10:47:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.955+0000 7f9f216f1640 1 -- 172.21.15.83:0/3786214933 learned_addr learned my addr 172.21.15.83:0/3786214933 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.955+0000 7f9f216f1640 1 -- 172.21.15.83:0/3786214933 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f1c070470 msgr2=0x7f9f1c15d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.955+0000 7f9f216f1640 1 --2- 172.21.15.83:0/3786214933 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f1c070470 0x7f9f1c15d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.955+0000 7f9f216f1640 1 -- 172.21.15.83:0/3786214933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f1c15b6e0 msgr2=0x7f9f1c1500d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.955+0000 7f9f1bfff640 1 --2- 172.21.15.83:0/3786214933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f1c15b6e0 0x7f9f1c1500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.955+0000 7f9f216f1640 1 --2- 172.21.15.83:0/3786214933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f1c15b6e0 0x7f9f1c1500d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:12.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.955+0000 7f9f216f1640 1 -- 172.21.15.83:0/3786214933 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f1c0b3990 con 0x7f9f1c15bac0 2024-04-17T10:47:12.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.959+0000 7f9f216f1640 1 --2- 172.21.15.83:0/3786214933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f1c15bac0 0x7f9f1c159890 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c00c780 tx=0x7f9f0c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:12.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.959+0000 7f9f197fa640 1 -- 172.21.15.83:0/3786214933 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c004b70 con 0x7f9f1c15bac0 2024-04-17T10:47:12.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.959+0000 7f9f197fa640 1 -- 172.21.15.83:0/3786214933 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f0c004d10 con 0x7f9f1c15bac0 2024-04-17T10:47:12.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.959+0000 7f9f23154640 1 -- 172.21.15.83:0/3786214933 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f1c0b3c70 con 0x7f9f1c15bac0 2024-04-17T10:47:12.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.959+0000 7f9f197fa640 1 -- 172.21.15.83:0/3786214933 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c00aa90 con 0x7f9f1c15bac0 2024-04-17T10:47:12.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.959+0000 7f9f23154640 1 -- 172.21.15.83:0/3786214933 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f1c0b4230 con 0x7f9f1c15bac0 2024-04-17T10:47:12.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.959+0000 7f9f23154640 1 -- 172.21.15.83:0/3786214933 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f1c074f90 con 0x7f9f1c15bac0 2024-04-17T10:47:12.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.959+0000 7f9f197fa640 1 -- 172.21.15.83:0/3786214933 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9f0c00ac30 con 0x7f9f1c15bac0 2024-04-17T10:47:12.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.959+0000 7f9f197fa640 1 --2- 172.21.15.83:0/3786214933 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9f08066f60 0x7f9f08069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:12.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.959+0000 7f9f197fa640 1 -- 172.21.15.83:0/3786214933 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f9f0c085680 con 0x7f9f1c15bac0 2024-04-17T10:47:12.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.959+0000 7f9f20ef0640 1 --2- 172.21.15.83:0/3786214933 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9f08066f60 0x7f9f08069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:12.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.959+0000 7f9f20ef0640 1 --2- 172.21.15.83:0/3786214933 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9f08066f60 0x7f9f08069420 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9f140ad000 tx=0x7f9f14008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:12.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:12.963+0000 7f9f197fa640 1 -- 172.21.15.83:0/3786214933 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9f0c056670 con 0x7f9f1c15bac0 2024-04-17T10:47:13.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.107+0000 7f9f23154640 1 -- 172.21.15.83:0/3786214933 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9f1c06af80 con 0x7f9f1c15bac0 2024-04-17T10:47:13.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.107+0000 7f9f197fa640 1 -- 172.21.15.83:0/3786214933 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9f0c055f70 con 0x7f9f1c15bac0 2024-04-17T10:47:13.104 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:47:13.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.111+0000 7f9f23154640 1 -- 172.21.15.83:0/3786214933 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9f08066f60 msgr2=0x7f9f08069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:13.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.111+0000 7f9f23154640 1 --2- 172.21.15.83:0/3786214933 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9f08066f60 0x7f9f08069420 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9f140ad000 tx=0x7f9f14008040 comp rx=0 tx=0).stop 2024-04-17T10:47:13.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.111+0000 7f9f23154640 1 -- 172.21.15.83:0/3786214933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f1c15bac0 msgr2=0x7f9f1c159890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:13.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.111+0000 7f9f23154640 1 --2- 172.21.15.83:0/3786214933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f1c15bac0 0x7f9f1c159890 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c00c780 tx=0x7f9f0c00cc40 comp rx=0 tx=0).stop 2024-04-17T10:47:13.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.111+0000 7f9f23154640 1 -- 172.21.15.83:0/3786214933 shutdown_connections 2024-04-17T10:47:13.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.111+0000 7f9f23154640 1 --2- 172.21.15.83:0/3786214933 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9f08066f60 0x7f9f08069420 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:13.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.111+0000 7f9f23154640 1 --2- 172.21.15.83:0/3786214933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f1c15bac0 0x7f9f1c159890 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:13.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.111+0000 7f9f23154640 1 --2- 172.21.15.83:0/3786214933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f1c15b6e0 0x7f9f1c1500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:13.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.111+0000 7f9f23154640 1 --2- 172.21.15.83:0/3786214933 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f1c070470 0x7f9f1c15d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:13.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.111+0000 7f9f23154640 1 -- 172.21.15.83:0/3786214933 >> 172.21.15.83:0/3786214933 conn(0x7f9f1c0b15b0 msgr2=0x7f9f1c1512b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:13.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.111+0000 7f9f23154640 1 -- 172.21.15.83:0/3786214933 shutdown_connections 2024-04-17T10:47:13.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.111+0000 7f9f23154640 1 -- 172.21.15.83:0/3786214933 wait complete. 2024-04-17T10:47:13.121 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:47:13.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.231+0000 7fbbea0e5640 1 Processor -- start 2024-04-17T10:47:13.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.231+0000 7fbbea0e5640 1 -- start start 2024-04-17T10:47:13.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.235+0000 7fbbea0e5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbbe406b520 0x7fbbe40b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:13.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.235+0000 7fbbea0e5640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe415bd70 con 0x7fbbe4154de0 2024-04-17T10:47:13.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.235+0000 7fbbea0e5640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe415bf60 con 0x7fbbe40b4040 2024-04-17T10:47:13.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.235+0000 7fbbea0e5640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe40b4870 con 0x7fbbe406b520 2024-04-17T10:47:13.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.235+0000 7fbbe37fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbbe4154de0 0x7fbbe406ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40564/0 (socket says 172.21.15.83:40564) 2024-04-17T10:47:13.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.235+0000 7fbbe37fe640 1 -- 172.21.15.83:0/1143088521 learned_addr learned my addr 172.21.15.83:0/1143088521 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:13.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.235+0000 7fbbdaffd640 1 --2- 172.21.15.83:0/1143088521 >> v2:172.21.15.167:3300/0 conn(0x7fbbe406b520 0x7fbbe40b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:13.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.235+0000 7fbbe2ffd640 1 -- 172.21.15.83:0/1143088521 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 951159418 0 0) 0x7fbbe415bd70 con 0x7fbbe4154de0 2024-04-17T10:47:13.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.235+0000 7fbbe2ffd640 1 -- 172.21.15.83:0/1143088521 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbd0003530 con 0x7fbbe4154de0 2024-04-17T10:47:13.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.235+0000 7fbbe2ffd640 1 -- 172.21.15.83:0/1143088521 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 345719865 0 0) 0x7fbbd0003530 con 0x7fbbe4154de0 2024-04-17T10:47:13.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.235+0000 7fbbe2ffd640 1 -- 172.21.15.83:0/1143088521 >> v2:172.21.15.167:3300/0 conn(0x7fbbe406b520 msgr2=0x7fbbe40b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:13.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.235+0000 7fbbe2ffd640 1 --2- 172.21.15.83:0/1143088521 >> v2:172.21.15.167:3300/0 conn(0x7fbbe406b520 0x7fbbe40b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:13.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.235+0000 7fbbe2ffd640 1 -- 172.21.15.83:0/1143088521 >> v1:172.21.15.167:6789/0 conn(0x7fbbe40b4040 legacy=0x7fbbe406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:13.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.235+0000 7fbbe2ffd640 1 -- 172.21.15.83:0/1143088521 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe415afb0 con 0x7fbbe4154de0 2024-04-17T10:47:13.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.235+0000 7fbbe2ffd640 1 -- 172.21.15.83:0/1143088521 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fbbd4001680 con 0x7fbbe4154de0 2024-04-17T10:47:13.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.235+0000 7fbbe2ffd640 1 -- 172.21.15.83:0/1143088521 >> v1:172.21.15.83:6789/0 conn(0x7fbbe4154de0 legacy=0x7fbbe406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:13.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.235+0000 7fbbe2ffd640 1 --2- 172.21.15.83:0/1143088521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbd0003a30 0x7fbbd0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:13.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.235+0000 7fbbe2ffd640 1 --2- 172.21.15.83:0/1143088521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbd0004810 0x7fbbd0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:13.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.235+0000 7fbbe2ffd640 1 --2- 172.21.15.83:0/1143088521 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbd00051d0 0x7fbbd000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:13.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.235+0000 7fbbe2ffd640 1 -- 172.21.15.83:0/1143088521 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd000c0d0 con 0x7fbbd0004810 2024-04-17T10:47:13.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.235+0000 7fbbe2ffd640 1 -- 172.21.15.83:0/1143088521 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd000c280 con 0x7fbbd0003a30 2024-04-17T10:47:13.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.235+0000 7fbbe2ffd640 1 -- 172.21.15.83:0/1143088521 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd000c480 con 0x7fbbd00051d0 2024-04-17T10:47:13.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.235+0000 7fbbdaffd640 1 --2- 172.21.15.83:0/1143088521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbd0004810 0x7fbbd0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:13.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.235+0000 7fbbdaffd640 1 -- 172.21.15.83:0/1143088521 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbd00051d0 msgr2=0x7fbbd000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:13.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.235+0000 7fbbdaffd640 1 --2- 172.21.15.83:0/1143088521 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbd00051d0 0x7fbbd000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:13.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.235+0000 7fbbdaffd640 1 -- 172.21.15.83:0/1143088521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbd0003a30 msgr2=0x7fbbd0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:13.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.235+0000 7fbbe37fe640 1 --2- 172.21.15.83:0/1143088521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbd0003a30 0x7fbbd0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:13.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.235+0000 7fbbdaffd640 1 --2- 172.21.15.83:0/1143088521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbd0003a30 0x7fbbd0003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:13.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.235+0000 7fbbe3fff640 1 -- 172.21.15.83:0/1143088521 reap_dead start 2024-04-17T10:47:13.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.235+0000 7fbbdaffd640 1 -- 172.21.15.83:0/1143088521 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd0003530 con 0x7fbbd0004810 2024-04-17T10:47:13.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.235+0000 7fbbe37fe640 1 --2- 172.21.15.83:0/1143088521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbd0003a30 0x7fbbd0003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:13.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbdaffd640 1 --2- 172.21.15.83:0/1143088521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbd0004810 0x7fbbd0004c90 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc009ae0 tx=0x7fbbcc01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ecfd43a41aeb043 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:13.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbe2ffd640 1 -- 172.21.15.83:0/1143088521 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc0b2070 con 0x7fbbd0004810 2024-04-17T10:47:13.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbe2ffd640 1 -- 172.21.15.83:0/1143088521 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbcc0a5660 con 0x7fbbd0004810 2024-04-17T10:47:13.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbe2ffd640 1 -- 172.21.15.83:0/1143088521 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc0096b0 con 0x7fbbd0004810 2024-04-17T10:47:13.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbea0e5640 1 -- 172.21.15.83:0/1143088521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbd0004810 msgr2=0x7fbbd0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:13.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbea0e5640 1 --2- 172.21.15.83:0/1143088521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbd0004810 0x7fbbd0004c90 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc009ae0 tx=0x7fbbcc01ca70 comp rx=0 tx=0).stop 2024-04-17T10:47:13.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbea0e5640 1 -- 172.21.15.83:0/1143088521 shutdown_connections 2024-04-17T10:47:13.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbea0e5640 1 --2- 172.21.15.83:0/1143088521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbd0004810 0x7fbbd0004c90 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:13.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbea0e5640 1 -- 172.21.15.83:0/1143088521 >> 172.21.15.83:0/1143088521 conn(0x7fbbe40b15b0 msgr2=0x7fbbe40b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:13.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbea0e5640 1 -- 172.21.15.83:0/1143088521 shutdown_connections 2024-04-17T10:47:13.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbea0e5640 1 -- 172.21.15.83:0/1143088521 wait complete. 2024-04-17T10:47:13.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbea0e5640 1 Processor -- start 2024-04-17T10:47:13.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbea0e5640 1 -- start start 2024-04-17T10:47:13.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbea0e5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbd0004810 0x7fbbe4150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:13.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbea0e5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbe4153240 0x7fbbe41516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:13.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbea0e5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe4151c20 0x7fbbe40aa100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:13.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbea0e5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe415b850 con 0x7fbbd0004810 2024-04-17T10:47:13.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbea0e5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe415b570 con 0x7fbbe4151c20 2024-04-17T10:47:13.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbea0e5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe41551c0 con 0x7fbbe4153240 2024-04-17T10:47:13.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbe37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbd0004810 0x7fbbe4150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:13.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbe37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbd0004810 0x7fbbe4150370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59034/0 (socket says 172.21.15.83:59034) 2024-04-17T10:47:13.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbe37fe640 1 -- 172.21.15.83:0/1972274225 learned_addr learned my addr 172.21.15.83:0/1972274225 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:13.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbdaffd640 1 --2- 172.21.15.83:0/1972274225 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbe4153240 0x7fbbe41516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:13.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbe3fff640 1 --2- 172.21.15.83:0/1972274225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe4151c20 0x7fbbe40aa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:13.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbe37fe640 1 -- 172.21.15.83:0/1972274225 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbe4153240 msgr2=0x7fbbe41516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:13.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbe37fe640 1 --2- 172.21.15.83:0/1972274225 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbe4153240 0x7fbbe41516e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:13.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbe37fe640 1 -- 172.21.15.83:0/1972274225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe4151c20 msgr2=0x7fbbe40aa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:13.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbe37fe640 1 --2- 172.21.15.83:0/1972274225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe4151c20 0x7fbbe40aa100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:13.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbe37fe640 1 -- 172.21.15.83:0/1972274225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe41521a0 con 0x7fbbd0004810 2024-04-17T10:47:13.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbe37fe640 1 --2- 172.21.15.83:0/1972274225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbd0004810 0x7fbbe4150370 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4010b70 tx=0x7fbbd400e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:13.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbe0ff9640 1 -- 172.21.15.83:0/1972274225 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd400eb80 con 0x7fbbd0004810 2024-04-17T10:47:13.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbe0ff9640 1 -- 172.21.15.83:0/1972274225 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd4011040 con 0x7fbbd0004810 2024-04-17T10:47:13.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.239+0000 7fbbea0e5640 1 -- 172.21.15.83:0/1972274225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe4152480 con 0x7fbbd0004810 2024-04-17T10:47:13.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.243+0000 7fbbe0ff9640 1 -- 172.21.15.83:0/1972274225 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4015400 con 0x7fbbd0004810 2024-04-17T10:47:13.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.243+0000 7fbbea0e5640 1 -- 172.21.15.83:0/1972274225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe4152a40 con 0x7fbbd0004810 2024-04-17T10:47:13.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.243+0000 7fbbea0e5640 1 -- 172.21.15.83:0/1972274225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe4074f90 con 0x7fbbd0004810 2024-04-17T10:47:13.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.243+0000 7fbbe0ff9640 1 -- 172.21.15.83:0/1972274225 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbbd401d020 con 0x7fbbd0004810 2024-04-17T10:47:13.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.243+0000 7fbbe0ff9640 1 --2- 172.21.15.83:0/1972274225 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbbc4066f60 0x7fbbc4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:13.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.243+0000 7fbbdaffd640 1 --2- 172.21.15.83:0/1972274225 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbbc4066f60 0x7fbbc4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:13.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.243+0000 7fbbe0ff9640 1 -- 172.21.15.83:0/1972274225 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fbbd4087e40 con 0x7fbbd0004810 2024-04-17T10:47:13.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.243+0000 7fbbdaffd640 1 --2- 172.21.15.83:0/1972274225 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbbc4066f60 0x7fbbc4069420 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc0af000 tx=0x7fbbcc0046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:13.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.247+0000 7fbbe0ff9640 1 -- 172.21.15.83:0/1972274225 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbbd4058e30 con 0x7fbbd0004810 2024-04-17T10:47:13.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.399+0000 7fbbea0e5640 1 -- 172.21.15.83:0/1972274225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbbe40b3280 con 0x7fbbd0004810 2024-04-17T10:47:13.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.399+0000 7fbbe0ff9640 1 -- 172.21.15.83:0/1972274225 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbbd4058730 con 0x7fbbd0004810 2024-04-17T10:47:13.394 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:47:13.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.399+0000 7fbbea0e5640 1 -- 172.21.15.83:0/1972274225 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbbc4066f60 msgr2=0x7fbbc4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:13.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.399+0000 7fbbea0e5640 1 --2- 172.21.15.83:0/1972274225 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbbc4066f60 0x7fbbc4069420 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc0af000 tx=0x7fbbcc0046d0 comp rx=0 tx=0).stop 2024-04-17T10:47:13.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.399+0000 7fbbea0e5640 1 -- 172.21.15.83:0/1972274225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbd0004810 msgr2=0x7fbbe4150370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:13.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.399+0000 7fbbea0e5640 1 --2- 172.21.15.83:0/1972274225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbd0004810 0x7fbbe4150370 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4010b70 tx=0x7fbbd400e4d0 comp rx=0 tx=0).stop 2024-04-17T10:47:13.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.399+0000 7fbbea0e5640 1 -- 172.21.15.83:0/1972274225 shutdown_connections 2024-04-17T10:47:13.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.399+0000 7fbbea0e5640 1 --2- 172.21.15.83:0/1972274225 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbbc4066f60 0x7fbbc4069420 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:13.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.399+0000 7fbbea0e5640 1 --2- 172.21.15.83:0/1972274225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe4151c20 0x7fbbe40aa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:13.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.399+0000 7fbbea0e5640 1 --2- 172.21.15.83:0/1972274225 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbe4153240 0x7fbbe41516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:13.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.399+0000 7fbbea0e5640 1 --2- 172.21.15.83:0/1972274225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbd0004810 0x7fbbe4150370 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:13.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.399+0000 7fbbea0e5640 1 -- 172.21.15.83:0/1972274225 >> 172.21.15.83:0/1972274225 conn(0x7fbbe40b15b0 msgr2=0x7fbbe4071cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:13.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.399+0000 7fbbea0e5640 1 -- 172.21.15.83:0/1972274225 shutdown_connections 2024-04-17T10:47:13.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.399+0000 7fbbea0e5640 1 -- 172.21.15.83:0/1972274225 wait complete. 2024-04-17T10:47:13.412 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:47:13.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.527+0000 7f7ab4462640 1 Processor -- start 2024-04-17T10:47:13.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.527+0000 7f7ab4462640 1 -- start start 2024-04-17T10:47:13.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.527+0000 7f7ab4462640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7aac15b6e0 0x7f7aac071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:13.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.527+0000 7f7ab4462640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aac06c1f0 con 0x7f7aac15bac0 2024-04-17T10:47:13.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.527+0000 7f7ab4462640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aac070a10 con 0x7f7aac070470 2024-04-17T10:47:13.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.527+0000 7f7ab4462640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aac070cf0 con 0x7f7aac15b6e0 2024-04-17T10:47:13.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.527+0000 7f7ab21fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7aac15b6e0 0x7f7aac071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:13.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.527+0000 7f7ab19fd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7aac15bac0 0x7f7aac0cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40576/0 (socket says 172.21.15.83:40576) 2024-04-17T10:47:13.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.527+0000 7f7ab19fd640 1 -- 172.21.15.83:0/2691926279 learned_addr learned my addr 172.21.15.83:0/2691926279 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:13.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.527+0000 7f7ab21fe640 1 -- 172.21.15.83:0/2691926279 >> v1:172.21.15.167:6789/0 conn(0x7f7aac070470 legacy=0x7f7aac06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:13.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.527+0000 7f7ab21fe640 1 -- 172.21.15.83:0/2691926279 >> v1:172.21.15.83:6789/0 conn(0x7f7aac15bac0 legacy=0x7f7aac0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:13.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.527+0000 7f7ab21fe640 1 -- 172.21.15.83:0/2691926279 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aac070fd0 con 0x7f7aac15b6e0 2024-04-17T10:47:13.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.527+0000 7f7ab11fc640 1 -- 172.21.15.83:0/2691926279 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3217605675 0 0) 0x7f7aac06c1f0 con 0x7f7aac15bac0 2024-04-17T10:47:13.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.531+0000 7f7ab21fe640 1 --2- 172.21.15.83:0/2691926279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7aac15b6e0 0x7f7aac071cc0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009d90 tx=0x7f7a9c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299cf42e5852cd47 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:13.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.531+0000 7f7ab11fc640 1 -- 172.21.15.83:0/2691926279 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c0b2070 con 0x7f7aac15b6e0 2024-04-17T10:47:13.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.531+0000 7f7ab11fc640 1 -- 172.21.15.83:0/2691926279 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9c0a57f0 con 0x7f7aac15b6e0 2024-04-17T10:47:13.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.531+0000 7f7ab11fc640 1 -- 172.21.15.83:0/2691926279 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c0a5bc0 con 0x7f7aac15b6e0 2024-04-17T10:47:13.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.531+0000 7f7ab4462640 1 -- 172.21.15.83:0/2691926279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7aac15b6e0 msgr2=0x7f7aac071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:13.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.531+0000 7f7ab4462640 1 --2- 172.21.15.83:0/2691926279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7aac15b6e0 0x7f7aac071cc0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009d90 tx=0x7f7a9c01c930 comp rx=0 tx=0).stop 2024-04-17T10:47:13.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.531+0000 7f7ab4462640 1 -- 172.21.15.83:0/2691926279 shutdown_connections 2024-04-17T10:47:13.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.531+0000 7f7ab4462640 1 --2- 172.21.15.83:0/2691926279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7aac15b6e0 0x7f7aac071cc0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:13.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.531+0000 7f7ab4462640 1 -- 172.21.15.83:0/2691926279 >> 172.21.15.83:0/2691926279 conn(0x7f7aac0b15b0 msgr2=0x7f7aac0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:13.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.531+0000 7f7ab4462640 1 -- 172.21.15.83:0/2691926279 shutdown_connections 2024-04-17T10:47:13.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.531+0000 7f7ab4462640 1 -- 172.21.15.83:0/2691926279 wait complete. 2024-04-17T10:47:13.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.531+0000 7f7ab4462640 1 Processor -- start 2024-04-17T10:47:13.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.531+0000 7f7ab4462640 1 -- start start 2024-04-17T10:47:13.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.531+0000 7f7ab4462640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7aac070470 0x7f7aac145d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:13.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.531+0000 7f7ab4462640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7aac15b6e0 0x7f7aac146f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:13.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.531+0000 7f7ab4462640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7aac15bac0 0x7f7aac1478b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:13.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.531+0000 7f7ab4462640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac06cee0 con 0x7f7aac070470 2024-04-17T10:47:13.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.531+0000 7f7ab4462640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac06d060 con 0x7f7aac15b6e0 2024-04-17T10:47:13.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.531+0000 7f7ab4462640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac06d1e0 con 0x7f7aac15bac0 2024-04-17T10:47:13.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.531+0000 7f7ab21fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7aac070470 0x7f7aac145d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:13.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.531+0000 7f7ab21fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7aac070470 0x7f7aac145d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59040/0 (socket says 172.21.15.83:59040) 2024-04-17T10:47:13.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.531+0000 7f7ab21fe640 1 -- 172.21.15.83:0/3548454534 learned_addr learned my addr 172.21.15.83:0/3548454534 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:13.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.531+0000 7f7ab19fd640 1 --2- 172.21.15.83:0/3548454534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7aac15b6e0 0x7f7aac146f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:13.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.531+0000 7f7ab21fe640 1 -- 172.21.15.83:0/3548454534 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7aac15bac0 msgr2=0x7f7aac1478b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:13.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.531+0000 7f7ab29ff640 1 --2- 172.21.15.83:0/3548454534 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7aac15bac0 0x7f7aac1478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.531+0000 7f7ab21fe640 1 --2- 172.21.15.83:0/3548454534 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7aac15bac0 0x7f7aac1478b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.531+0000 7f7ab21fe640 1 -- 172.21.15.83:0/3548454534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7aac15b6e0 msgr2=0x7f7aac146f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.531+0000 7f7ab21fe640 1 --2- 172.21.15.83:0/3548454534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7aac15b6e0 0x7f7aac146f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.531+0000 7f7ab21fe640 1 -- 172.21.15.83:0/3548454534 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aac149a40 con 0x7f7aac070470 2024-04-17T10:47:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.531+0000 7f7ab19fd640 1 --2- 172.21.15.83:0/3548454534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7aac15b6e0 0x7f7aac146f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.535+0000 7f7ab21fe640 1 --2- 172.21.15.83:0/3548454534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7aac070470 0x7f7aac145d60 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c019540 tx=0x7f7a9c0196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.535+0000 7f7aaaffd640 1 -- 172.21.15.83:0/3548454534 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c0b2040 con 0x7f7aac070470 2024-04-17T10:47:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.535+0000 7f7aaaffd640 1 -- 172.21.15.83:0/3548454534 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9c0bf070 con 0x7f7aac070470 2024-04-17T10:47:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.535+0000 7f7ab4462640 1 -- 172.21.15.83:0/3548454534 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aac149d20 con 0x7f7aac070470 2024-04-17T10:47:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.535+0000 7f7aaaffd640 1 -- 172.21.15.83:0/3548454534 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c0ada10 con 0x7f7aac070470 2024-04-17T10:47:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.535+0000 7f7ab4462640 1 -- 172.21.15.83:0/3548454534 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aac146550 con 0x7f7aac070470 2024-04-17T10:47:13.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.535+0000 7f7ab4462640 1 -- 172.21.15.83:0/3548454534 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aac06b7b0 con 0x7f7aac070470 2024-04-17T10:47:13.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.535+0000 7f7aaaffd640 1 -- 172.21.15.83:0/3548454534 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7a9c0adbb0 con 0x7f7aac070470 2024-04-17T10:47:13.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.535+0000 7f7aaaffd640 1 --2- 172.21.15.83:0/3548454534 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7a98066f60 0x7f7a98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:13.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.535+0000 7f7ab19fd640 1 --2- 172.21.15.83:0/3548454534 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7a98066f60 0x7f7a98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:13.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.535+0000 7f7aaaffd640 1 -- 172.21.15.83:0/3548454534 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f7a9c120480 con 0x7f7aac070470 2024-04-17T10:47:13.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.535+0000 7f7ab19fd640 1 --2- 172.21.15.83:0/3548454534 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7a98066f60 0x7f7a98069420 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0ae630 tx=0x7f7aa0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:13.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.539+0000 7f7aaaffd640 1 -- 172.21.15.83:0/3548454534 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7a9c016520 con 0x7f7aac070470 2024-04-17T10:47:13.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.695+0000 7f7ab4462640 1 -- 172.21.15.83:0/3548454534 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7aac074f90 con 0x7f7aac070470 2024-04-17T10:47:13.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.695+0000 7f7aaaffd640 1 -- 172.21.15.83:0/3548454534 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v91) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7a9c0ab340 con 0x7f7aac070470 2024-04-17T10:47:13.693 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-17T10:47:13.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.699+0000 7f7ab4462640 1 -- 172.21.15.83:0/3548454534 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7a98066f60 msgr2=0x7f7a98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:13.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.699+0000 7f7ab4462640 1 --2- 172.21.15.83:0/3548454534 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7a98066f60 0x7f7a98069420 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0ae630 tx=0x7f7aa0009040 comp rx=0 tx=0).stop 2024-04-17T10:47:13.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.699+0000 7f7ab4462640 1 -- 172.21.15.83:0/3548454534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7aac070470 msgr2=0x7f7aac145d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:13.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.699+0000 7f7ab4462640 1 --2- 172.21.15.83:0/3548454534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7aac070470 0x7f7aac145d60 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c019540 tx=0x7f7a9c0196d0 comp rx=0 tx=0).stop 2024-04-17T10:47:13.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.699+0000 7f7ab4462640 1 -- 172.21.15.83:0/3548454534 shutdown_connections 2024-04-17T10:47:13.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.699+0000 7f7ab4462640 1 --2- 172.21.15.83:0/3548454534 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7a98066f60 0x7f7a98069420 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:13.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.699+0000 7f7ab4462640 1 --2- 172.21.15.83:0/3548454534 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7aac15bac0 0x7f7aac1478b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:13.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.699+0000 7f7ab4462640 1 --2- 172.21.15.83:0/3548454534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7aac15b6e0 0x7f7aac146f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:13.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.699+0000 7f7ab4462640 1 --2- 172.21.15.83:0/3548454534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7aac070470 0x7f7aac145d60 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:13.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.699+0000 7f7ab4462640 1 -- 172.21.15.83:0/3548454534 >> 172.21.15.83:0/3548454534 conn(0x7f7aac0b15b0 msgr2=0x7f7aac0cd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:13.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.699+0000 7f7ab4462640 1 -- 172.21.15.83:0/3548454534 shutdown_connections 2024-04-17T10:47:13.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:13.699+0000 7f7ab4462640 1 -- 172.21.15.83:0/3548454534 wait complete. 2024-04-17T10:47:13.710 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2024-04-17T10:47:13.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:13.710 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-17T10:47:13.721 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[70535]: fuse finished with error 0 and tester_r 0 2024-04-17T10:47:13.825 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-17T10:47:13.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:13.825 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T10:47:13.891 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:47:13.891 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:47:13.897 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:47:13.898 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-04-17T10:47:13.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:13.898 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-17T10:47:13.946 INFO:teuthology.orchestra.run.smithi167.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-17T10:47:13.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:47:13.947 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:47:13.947 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:47:13.951 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16063 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:47:13.952 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-17T10:47:13.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:13.952 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-17T10:47:13.997 INFO:teuthology.orchestra.run.smithi083.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-17T10:47:13.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:47:13.998 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:47:13.999 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:47:14.005 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:47:14.005 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-04-17T10:47:14.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:14.005 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-17T10:47:14.054 INFO:teuthology.orchestra.run.smithi167.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-17T10:47:14.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:47:14.055 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:47:14.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.179+0000 7f26f96b2640 1 Processor -- start 2024-04-17T10:47:14.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.179+0000 7f26f96b2640 1 -- start start 2024-04-17T10:47:14.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.183+0000 7f26f96b2640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f26f415db50 0x7f26f4070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:14.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.183+0000 7f26f96b2640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26f40709b0 con 0x7f26f406dcb0 2024-04-17T10:47:14.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.183+0000 7f26f96b2640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26f415e440 con 0x7f26f40aedd0 2024-04-17T10:47:14.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.183+0000 7f26f96b2640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f415e720 con 0x7f26f415db50 2024-04-17T10:47:14.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.183+0000 7f26f37fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f26f406dcb0 0x7f26f40b2e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40586/0 (socket says 172.21.15.83:40586) 2024-04-17T10:47:14.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.183+0000 7f26f37fe640 1 -- 172.21.15.83:0/1651147967 learned_addr learned my addr 172.21.15.83:0/1651147967 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:14.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.183+0000 7f26f27fc640 1 -- 172.21.15.83:0/1651147967 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 669855316 0 0) 0x7f26ec0014a0 con 0x7f26f406dcb0 2024-04-17T10:47:14.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.183+0000 7f26f2ffd640 1 --2- 172.21.15.83:0/1651147967 >> v2:172.21.15.167:3300/0 conn(0x7f26f415db50 0x7f26f4070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:14.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.183+0000 7f26f27fc640 1 -- 172.21.15.83:0/1651147967 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26e0003530 con 0x7f26f406dcb0 2024-04-17T10:47:14.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.183+0000 7f26f27fc640 1 -- 172.21.15.83:0/1651147967 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3938567744 0 0) 0x7f26e0003530 con 0x7f26f406dcb0 2024-04-17T10:47:14.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.183+0000 7f26f27fc640 1 -- 172.21.15.83:0/1651147967 >> v2:172.21.15.167:3300/0 conn(0x7f26f415db50 msgr2=0x7f26f4070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:14.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.183+0000 7f26f27fc640 1 --2- 172.21.15.83:0/1651147967 >> v2:172.21.15.167:3300/0 conn(0x7f26f415db50 0x7f26f4070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:14.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.183+0000 7f26f27fc640 1 -- 172.21.15.83:0/1651147967 >> v1:172.21.15.167:6789/0 conn(0x7f26f40aedd0 legacy=0x7f26f406d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:14.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.183+0000 7f26f27fc640 1 -- 172.21.15.83:0/1651147967 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f415ea00 con 0x7f26f406dcb0 2024-04-17T10:47:14.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.183+0000 7f26f27fc640 1 -- 172.21.15.83:0/1651147967 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f26ec001bd0 con 0x7f26f406dcb0 2024-04-17T10:47:14.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.183+0000 7f26f27fc640 1 -- 172.21.15.83:0/1651147967 >> v1:172.21.15.83:6789/0 conn(0x7f26f406dcb0 legacy=0x7f26f40b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:14.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.183+0000 7f26f27fc640 1 --2- 172.21.15.83:0/1651147967 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26e00039c0 0x7f26e0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:14.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.183+0000 7f26f27fc640 1 --2- 172.21.15.83:0/1651147967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26e0004810 0x7f26e0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:14.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.183+0000 7f26f27fc640 1 --2- 172.21.15.83:0/1651147967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26e00051d0 0x7f26e000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:14.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.183+0000 7f26f27fc640 1 -- 172.21.15.83:0/1651147967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f415e720 con 0x7f26e00051d0 2024-04-17T10:47:14.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.183+0000 7f26f27fc640 1 -- 172.21.15.83:0/1651147967 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e000c070 con 0x7f26e0004810 2024-04-17T10:47:14.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.183+0000 7f26f27fc640 1 -- 172.21.15.83:0/1651147967 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e000c270 con 0x7f26e00039c0 2024-04-17T10:47:14.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.183+0000 7f26f37fe640 1 --2- 172.21.15.83:0/1651147967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26e00051d0 0x7f26e000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:14.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.183+0000 7f26f2ffd640 1 --2- 172.21.15.83:0/1651147967 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26e00039c0 0x7f26e0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:14.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.183+0000 7f26f37fe640 1 -- 172.21.15.83:0/1651147967 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26e00039c0 msgr2=0x7f26e0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:14.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.183+0000 7f26f37fe640 1 --2- 172.21.15.83:0/1651147967 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26e00039c0 0x7f26e0003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:14.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.183+0000 7f26f37fe640 1 -- 172.21.15.83:0/1651147967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26e0004810 msgr2=0x7f26e0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:14.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.183+0000 7f26ebfff640 1 --2- 172.21.15.83:0/1651147967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26e0004810 0x7f26e0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:14.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.183+0000 7f26f37fe640 1 --2- 172.21.15.83:0/1651147967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26e0004810 0x7f26e0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:14.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.183+0000 7f26f37fe640 1 -- 172.21.15.83:0/1651147967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e0003530 con 0x7f26e00051d0 2024-04-17T10:47:14.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.183+0000 7f26ebfff640 1 --2- 172.21.15.83:0/1651147967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26e0004810 0x7f26e0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:14.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26f37fe640 1 -- 172.21.15.83:0/1651147967 reap_dead start 2024-04-17T10:47:14.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26f37fe640 1 --2- 172.21.15.83:0/1651147967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26e00051d0 0x7f26e000b9d0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f26f406e5a0 tx=0x7f26ec01d300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e2bc9d68107798 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:14.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26f27fc640 1 -- 172.21.15.83:0/1651147967 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec00a620 con 0x7f26e00051d0 2024-04-17T10:47:14.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26f27fc640 1 -- 172.21.15.83:0/1651147967 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ec0ad650 con 0x7f26e00051d0 2024-04-17T10:47:14.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26f27fc640 1 -- 172.21.15.83:0/1651147967 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec0ada40 con 0x7f26e00051d0 2024-04-17T10:47:14.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26f96b2640 1 -- 172.21.15.83:0/1651147967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26e00051d0 msgr2=0x7f26e000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:14.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26f96b2640 1 --2- 172.21.15.83:0/1651147967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26e00051d0 0x7f26e000b9d0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f26f406e5a0 tx=0x7f26ec01d300 comp rx=0 tx=0).stop 2024-04-17T10:47:14.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26f96b2640 1 -- 172.21.15.83:0/1651147967 shutdown_connections 2024-04-17T10:47:14.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26f96b2640 1 --2- 172.21.15.83:0/1651147967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26e00051d0 0x7f26e000b9d0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:14.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26f96b2640 1 -- 172.21.15.83:0/1651147967 >> 172.21.15.83:0/1651147967 conn(0x7f26f406abe0 msgr2=0x7f26f406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:14.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26f96b2640 1 -- 172.21.15.83:0/1651147967 shutdown_connections 2024-04-17T10:47:14.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26f96b2640 1 -- 172.21.15.83:0/1651147967 wait complete. 2024-04-17T10:47:14.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26f96b2640 1 Processor -- start 2024-04-17T10:47:14.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26f96b2640 1 -- start start 2024-04-17T10:47:14.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26f96b2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26e00051d0 0x7f26f41f3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:14.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26f96b2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26f4157050 0x7f26f41574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:14.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26f96b2640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26f4144e60 0x7f26f41452e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:14.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26f96b2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f415efc0 con 0x7f26e00051d0 2024-04-17T10:47:14.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26f96b2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f415e2c0 con 0x7f26f4157050 2024-04-17T10:47:14.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26f96b2640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f415df30 con 0x7f26f4144e60 2024-04-17T10:47:14.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26f2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26e00051d0 0x7f26f41f3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:14.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26f2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26e00051d0 0x7f26f41f3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59056/0 (socket says 172.21.15.83:59056) 2024-04-17T10:47:14.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26f2ffd640 1 -- 172.21.15.83:0/2711475686 learned_addr learned my addr 172.21.15.83:0/2711475686 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:14.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26ebfff640 1 --2- 172.21.15.83:0/2711475686 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26f4157050 0x7f26f41574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:14.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26f37fe640 1 --2- 172.21.15.83:0/2711475686 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26f4144e60 0x7f26f41452e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:14.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26f2ffd640 1 -- 172.21.15.83:0/2711475686 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26f4144e60 msgr2=0x7f26f41452e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:14.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26f2ffd640 1 --2- 172.21.15.83:0/2711475686 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26f4144e60 0x7f26f41452e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:14.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26f2ffd640 1 -- 172.21.15.83:0/2711475686 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26f4157050 msgr2=0x7f26f41574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:14.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26f2ffd640 1 --2- 172.21.15.83:0/2711475686 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26f4157050 0x7f26f41574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:14.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26f2ffd640 1 -- 172.21.15.83:0/2711475686 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f4157aa0 con 0x7f26e00051d0 2024-04-17T10:47:14.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26ebfff640 1 --2- 172.21.15.83:0/2711475686 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26f4157050 0x7f26f41574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:14.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26f2ffd640 1 --2- 172.21.15.83:0/2711475686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26e00051d0 0x7f26f41f3d80 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f26e40108c0 tx=0x7f26e4010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:14.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26eb7fe640 1 -- 172.21.15.83:0/2711475686 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e400f040 con 0x7f26e00051d0 2024-04-17T10:47:14.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26eb7fe640 1 -- 172.21.15.83:0/2711475686 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e400eb60 con 0x7f26e00051d0 2024-04-17T10:47:14.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.187+0000 7f26eb7fe640 1 -- 172.21.15.83:0/2711475686 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e401c400 con 0x7f26e00051d0 2024-04-17T10:47:14.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.191+0000 7f26f96b2640 1 -- 172.21.15.83:0/2711475686 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f4157d80 con 0x7f26e00051d0 2024-04-17T10:47:14.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.191+0000 7f26f96b2640 1 -- 172.21.15.83:0/2711475686 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f4158340 con 0x7f26e00051d0 2024-04-17T10:47:14.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.191+0000 7f26f96b2640 1 -- 172.21.15.83:0/2711475686 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f4074f90 con 0x7f26e00051d0 2024-04-17T10:47:14.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.191+0000 7f26eb7fe640 1 -- 172.21.15.83:0/2711475686 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f26e401c5a0 con 0x7f26e00051d0 2024-04-17T10:47:14.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.191+0000 7f26eb7fe640 1 --2- 172.21.15.83:0/2711475686 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f26d4066f60 0x7f26d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:14.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.191+0000 7f26eb7fe640 1 -- 172.21.15.83:0/2711475686 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f26e40856a0 con 0x7f26e00051d0 2024-04-17T10:47:14.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.191+0000 7f26ebfff640 1 --2- 172.21.15.83:0/2711475686 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f26d4066f60 0x7f26d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:14.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.191+0000 7f26ebfff640 1 --2- 172.21.15.83:0/2711475686 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f26d4066f60 0x7f26d4069420 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f26f4155bf0 tx=0x7f26dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:14.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.195+0000 7f26eb7fe640 1 -- 172.21.15.83:0/2711475686 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f26e4018030 con 0x7f26e00051d0 2024-04-17T10:47:14.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.387+0000 7f26f96b2640 1 -- 172.21.15.83:0/2711475686 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26f40b32b0 con 0x7f26e00051d0 2024-04-17T10:47:14.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.387+0000 7f26eb7fe640 1 -- 172.21.15.83:0/2711475686 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f26e4056040 con 0x7f26e00051d0 2024-04-17T10:47:14.385 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:47:14.385 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5429,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6834/3936640219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3936640219},{"type":"v1","addr":"172.21.15.167:6834","nonce":3936640219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":5998,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4152882285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6833","nonce":4152882285},{"type":"v1","addr":"172.21.15.167:6835","nonce":4152882285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1562645860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1562645860},{"type":"v1","addr":"172.21.15.83:6837","nonce":1562645860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:46:36.578918+0000","modified":"2024-04-17T10:46:38.561701+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7575":{"gid":7575,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/1056175765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1056175765},{"type":"v1","addr":"172.21.15.83:6836","nonce":1056175765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-17T10:47:14.385 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 56 2024-04-17T10:47:14.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.391+0000 7f26f96b2640 1 -- 172.21.15.83:0/2711475686 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f26d4066f60 msgr2=0x7f26d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:14.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.391+0000 7f26f96b2640 1 --2- 172.21.15.83:0/2711475686 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f26d4066f60 0x7f26d4069420 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f26f4155bf0 tx=0x7f26dc009040 comp rx=0 tx=0).stop 2024-04-17T10:47:14.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.391+0000 7f26f96b2640 1 -- 172.21.15.83:0/2711475686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26e00051d0 msgr2=0x7f26f41f3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:14.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.391+0000 7f26f96b2640 1 --2- 172.21.15.83:0/2711475686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26e00051d0 0x7f26f41f3d80 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f26e40108c0 tx=0x7f26e4010d80 comp rx=0 tx=0).stop 2024-04-17T10:47:14.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.391+0000 7f26f96b2640 1 -- 172.21.15.83:0/2711475686 shutdown_connections 2024-04-17T10:47:14.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.391+0000 7f26f96b2640 1 --2- 172.21.15.83:0/2711475686 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f26d4066f60 0x7f26d4069420 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:14.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.391+0000 7f26f96b2640 1 --2- 172.21.15.83:0/2711475686 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26f4144e60 0x7f26f41452e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:14.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.391+0000 7f26f96b2640 1 --2- 172.21.15.83:0/2711475686 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26f4157050 0x7f26f41574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:14.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.391+0000 7f26f96b2640 1 --2- 172.21.15.83:0/2711475686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26e00051d0 0x7f26f41f3d80 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:14.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.391+0000 7f26f96b2640 1 -- 172.21.15.83:0/2711475686 >> 172.21.15.83:0/2711475686 conn(0x7f26f406abe0 msgr2=0x7f26f406caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:14.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.391+0000 7f26f96b2640 1 -- 172.21.15.83:0/2711475686 shutdown_connections 2024-04-17T10:47:14.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.391+0000 7f26f96b2640 1 -- 172.21.15.83:0/2711475686 wait complete. 2024-04-17T10:47:14.404 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:47:14.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.523+0000 7fccaeb34640 1 Processor -- start 2024-04-17T10:47:14.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.523+0000 7fccaeb34640 1 -- start start 2024-04-17T10:47:14.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.527+0000 7fccaeb34640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcca80ce750 0x7fcca80d2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:14.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.527+0000 7fccaeb34640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcca8162150 0x7fcca806c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:14.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.527+0000 7fccaeb34640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcca806e980 con 0x7fcca80b0a70 2024-04-17T10:47:14.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.527+0000 7fccaeb34640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcca81625e0 con 0x7fcca8162150 2024-04-17T10:47:14.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.527+0000 7fccaeb34640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcca806f290 con 0x7fcca80ce750 2024-04-17T10:47:14.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.527+0000 7fccad0d1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcca8162150 0x7fcca806c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:14.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.527+0000 7fcca7fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcca80b0a70 0x7fcca8161a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40596/0 (socket says 172.21.15.83:40596) 2024-04-17T10:47:14.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.527+0000 7fcca7fff640 1 -- 172.21.15.83:0/1014754385 learned_addr learned my addr 172.21.15.83:0/1014754385 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:14.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.527+0000 7fccac8d0640 1 --2- 172.21.15.83:0/1014754385 >> v2:172.21.15.167:3300/0 conn(0x7fcca80ce750 0x7fcca80d2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:14.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.527+0000 7fccad0d1640 1 -- 172.21.15.83:0/1014754385 >> v2:172.21.15.167:3300/0 conn(0x7fcca80ce750 msgr2=0x7fcca80d2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:14.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.527+0000 7fccad0d1640 1 --2- 172.21.15.83:0/1014754385 >> v2:172.21.15.167:3300/0 conn(0x7fcca80ce750 0x7fcca80d2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:14.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.527+0000 7fccad0d1640 1 -- 172.21.15.83:0/1014754385 >> v1:172.21.15.83:6789/0 conn(0x7fcca80b0a70 legacy=0x7fcca8161a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:14.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.527+0000 7fccad0d1640 1 -- 172.21.15.83:0/1014754385 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca8162810 con 0x7fcca8162150 2024-04-17T10:47:14.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.527+0000 7fcca77fe640 1 -- 172.21.15.83:0/1014754385 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1056528099 0 0) 0x7fcca806e980 con 0x7fcca80b0a70 2024-04-17T10:47:14.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.527+0000 7fccad0d1640 1 --2- 172.21.15.83:0/1014754385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcca8162150 0x7fcca806c700 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fcca80b42e0 tx=0x7fcca001cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaee922ffdc894fe server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:14.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.527+0000 7fcca77fe640 1 -- 172.21.15.83:0/1014754385 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca00b1070 con 0x7fcca8162150 2024-04-17T10:47:14.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.527+0000 7fcca77fe640 1 -- 172.21.15.83:0/1014754385 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca00ac5f0 con 0x7fcca8162150 2024-04-17T10:47:14.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.527+0000 7fcca77fe640 1 -- 172.21.15.83:0/1014754385 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca00ac9c0 con 0x7fcca8162150 2024-04-17T10:47:14.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fccaeb34640 1 -- 172.21.15.83:0/1014754385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcca8162150 msgr2=0x7fcca806c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:14.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fccaeb34640 1 --2- 172.21.15.83:0/1014754385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcca8162150 0x7fcca806c700 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fcca80b42e0 tx=0x7fcca001cee0 comp rx=0 tx=0).stop 2024-04-17T10:47:14.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fccaeb34640 1 -- 172.21.15.83:0/1014754385 shutdown_connections 2024-04-17T10:47:14.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fccaeb34640 1 --2- 172.21.15.83:0/1014754385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcca8162150 0x7fcca806c700 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:14.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fccaeb34640 1 --2- 172.21.15.83:0/1014754385 >> v2:172.21.15.167:3300/0 conn(0x7fcca80ce750 0x7fcca80d2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:14.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fccaeb34640 1 -- 172.21.15.83:0/1014754385 >> 172.21.15.83:0/1014754385 conn(0x7fcca80d1770 msgr2=0x7fcca80d1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:14.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fccaeb34640 1 -- 172.21.15.83:0/1014754385 shutdown_connections 2024-04-17T10:47:14.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fccaeb34640 1 -- 172.21.15.83:0/1014754385 wait complete. 2024-04-17T10:47:14.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fccaeb34640 1 Processor -- start 2024-04-17T10:47:14.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fccaeb34640 1 -- start start 2024-04-17T10:47:14.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fccaeb34640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcca80b0a70 0x7fcca814f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:14.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fccaeb34640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcca80ce750 0x7fcca8158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:14.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fccaeb34640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcca8162150 0x7fcca80c7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:14.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fccaeb34640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca806d450 con 0x7fcca8162150 2024-04-17T10:47:14.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fccaeb34640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca806d5d0 con 0x7fcca80ce750 2024-04-17T10:47:14.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fccaeb34640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcca806d750 con 0x7fcca80b0a70 2024-04-17T10:47:14.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fccad0d1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcca8162150 0x7fcca80c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:14.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fccad0d1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcca8162150 0x7fcca80c7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59070/0 (socket says 172.21.15.83:59070) 2024-04-17T10:47:14.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fccad0d1640 1 -- 172.21.15.83:0/1631941180 learned_addr learned my addr 172.21.15.83:0/1631941180 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:14.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fcca7fff640 1 --2- 172.21.15.83:0/1631941180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcca80ce750 0x7fcca8158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:14.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fccac8d0640 1 --2- 172.21.15.83:0/1631941180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcca80b0a70 0x7fcca814f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:14.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fccad0d1640 1 -- 172.21.15.83:0/1631941180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcca80b0a70 msgr2=0x7fcca814f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:14.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fccad0d1640 1 --2- 172.21.15.83:0/1631941180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcca80b0a70 0x7fcca814f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:14.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fccad0d1640 1 -- 172.21.15.83:0/1631941180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcca80ce750 msgr2=0x7fcca8158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:14.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fccad0d1640 1 --2- 172.21.15.83:0/1631941180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcca80ce750 0x7fcca8158300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:14.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fccad0d1640 1 -- 172.21.15.83:0/1631941180 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca815ab50 con 0x7fcca8162150 2024-04-17T10:47:14.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fcca7fff640 1 --2- 172.21.15.83:0/1631941180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcca80ce750 0x7fcca8158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:14.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fccad0d1640 1 --2- 172.21.15.83:0/1631941180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcca8162150 0x7fcca80c7270 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fcca001ab30 tx=0x7fcca0019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:14.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fcca57fa640 1 -- 172.21.15.83:0/1631941180 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca00b1040 con 0x7fcca8162150 2024-04-17T10:47:14.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fcca57fa640 1 -- 172.21.15.83:0/1631941180 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca00bd070 con 0x7fcca8162150 2024-04-17T10:47:14.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fccaeb34640 1 -- 172.21.15.83:0/1631941180 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcca815ae30 con 0x7fcca8162150 2024-04-17T10:47:14.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fcca57fa640 1 -- 172.21.15.83:0/1631941180 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca0006e20 con 0x7fcca8162150 2024-04-17T10:47:14.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.531+0000 7fccaeb34640 1 -- 172.21.15.83:0/1631941180 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcca815b3f0 con 0x7fcca8162150 2024-04-17T10:47:14.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.535+0000 7fccaeb34640 1 -- 172.21.15.83:0/1631941180 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcca80c5450 con 0x7fcca8162150 2024-04-17T10:47:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.535+0000 7fcca57fa640 1 -- 172.21.15.83:0/1631941180 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcca00b7660 con 0x7fcca8162150 2024-04-17T10:47:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.535+0000 7fcca57fa640 1 --2- 172.21.15.83:0/1631941180 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcc94066f90 0x7fcc94069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.535+0000 7fcca57fa640 1 -- 172.21.15.83:0/1631941180 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fcca011f2c0 con 0x7fcca8162150 2024-04-17T10:47:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.535+0000 7fccac8d0640 1 --2- 172.21.15.83:0/1631941180 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcc94066f90 0x7fcc94069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.535+0000 7fccac8d0640 1 --2- 172.21.15.83:0/1631941180 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcc94066f90 0x7fcc94069450 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fcc98009f30 tx=0x7fcc9800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:14.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.539+0000 7fcca57fa640 1 -- 172.21.15.83:0/1631941180 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcca00a9070 con 0x7fcca8162150 2024-04-17T10:47:14.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.691+0000 7fccaeb34640 1 -- 172.21.15.83:0/1631941180 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcca8074f90 con 0x7fcca8162150 2024-04-17T10:47:14.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.695+0000 7fcca57fa640 1 -- 172.21.15.83:0/1631941180 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+16638 (secure 0 0 0) 0x7fcca0016070 con 0x7fcca8162150 2024-04-17T10:47:14.690 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:47:14.690 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":91,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:46:55.414653+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:46:34.779174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-17T10:46:35.563972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.10","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:47:14.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.695+0000 7fccaeb34640 1 -- 172.21.15.83:0/1631941180 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcc94066f90 msgr2=0x7fcc94069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:14.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.695+0000 7fccaeb34640 1 --2- 172.21.15.83:0/1631941180 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcc94066f90 0x7fcc94069450 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fcc98009f30 tx=0x7fcc9800b040 comp rx=0 tx=0).stop 2024-04-17T10:47:14.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.695+0000 7fccaeb34640 1 -- 172.21.15.83:0/1631941180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcca8162150 msgr2=0x7fcca80c7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:14.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.695+0000 7fccaeb34640 1 --2- 172.21.15.83:0/1631941180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcca8162150 0x7fcca80c7270 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fcca001ab30 tx=0x7fcca0019220 comp rx=0 tx=0).stop 2024-04-17T10:47:14.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.695+0000 7fccaeb34640 1 -- 172.21.15.83:0/1631941180 shutdown_connections 2024-04-17T10:47:14.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.695+0000 7fccaeb34640 1 --2- 172.21.15.83:0/1631941180 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcc94066f90 0x7fcc94069450 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:14.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.695+0000 7fccaeb34640 1 --2- 172.21.15.83:0/1631941180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcca8162150 0x7fcca80c7270 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:14.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.695+0000 7fccaeb34640 1 --2- 172.21.15.83:0/1631941180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcca80ce750 0x7fcca8158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:14.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.695+0000 7fccaeb34640 1 --2- 172.21.15.83:0/1631941180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcca80b0a70 0x7fcca814f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:14.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.695+0000 7fccaeb34640 1 -- 172.21.15.83:0/1631941180 >> 172.21.15.83:0/1631941180 conn(0x7fcca80d1770 msgr2=0x7fcca80b3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:14.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.695+0000 7fccaeb34640 1 -- 172.21.15.83:0/1631941180 shutdown_connections 2024-04-17T10:47:14.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.695+0000 7fccaeb34640 1 -- 172.21.15.83:0/1631941180 wait complete. 2024-04-17T10:47:14.708 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:46:34.779174+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:46:35.563972+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:47:14.708 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:47:14.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.831+0000 7fe3870b8640 1 Processor -- start 2024-04-17T10:47:14.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.831+0000 7fe3870b8640 1 -- start start 2024-04-17T10:47:14.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.831+0000 7fe3870b8640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe38015e330 0x7fe38015e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:14.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.831+0000 7fe3870b8640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe380070aa0 0x7fe380070f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:14.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.831+0000 7fe3870b8640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3801625d0 con 0x7fe38006ae10 2024-04-17T10:47:14.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.831+0000 7fe3870b8640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe380161810 con 0x7fe380070aa0 2024-04-17T10:47:14.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.831+0000 7fe3870b8640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe380161af0 con 0x7fe38015e330 2024-04-17T10:47:14.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.831+0000 7fe384e54640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe38006ae10 0x7fe38006b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40612/0 (socket says 172.21.15.83:40612) 2024-04-17T10:47:14.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.831+0000 7fe384e54640 1 -- 172.21.15.83:0/49756070 learned_addr learned my addr 172.21.15.83:0/49756070 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:14.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.831+0000 7fe385655640 1 --2- 172.21.15.83:0/49756070 >> v2:172.21.15.83:3300/0 conn(0x7fe380070aa0 0x7fe380070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:14.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.831+0000 7fe37ffff640 1 --2- 172.21.15.83:0/49756070 >> v2:172.21.15.167:3300/0 conn(0x7fe38015e330 0x7fe38015e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:14.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.831+0000 7fe37f7fe640 1 -- 172.21.15.83:0/49756070 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3292107718 0 0) 0x7fe3801625d0 con 0x7fe38006ae10 2024-04-17T10:47:14.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.831+0000 7fe37f7fe640 1 -- 172.21.15.83:0/49756070 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe368003530 con 0x7fe38006ae10 2024-04-17T10:47:14.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.831+0000 7fe385655640 1 -- 172.21.15.83:0/49756070 >> v2:172.21.15.167:3300/0 conn(0x7fe38015e330 msgr2=0x7fe38015e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:14.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.831+0000 7fe385655640 1 --2- 172.21.15.83:0/49756070 >> v2:172.21.15.167:3300/0 conn(0x7fe38015e330 0x7fe38015e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:14.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.831+0000 7fe385655640 1 -- 172.21.15.83:0/49756070 >> v1:172.21.15.83:6789/0 conn(0x7fe38006ae10 legacy=0x7fe38006b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:14.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.831+0000 7fe385655640 1 -- 172.21.15.83:0/49756070 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe380161dd0 con 0x7fe380070aa0 2024-04-17T10:47:14.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.835+0000 7fe385655640 1 --2- 172.21.15.83:0/49756070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe380070aa0 0x7fe380070f30 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fe3800b2480 tx=0x7fe37801c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf9f0099c14efa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:14.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.835+0000 7fe37f7fe640 1 -- 172.21.15.83:0/49756070 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3780b2070 con 0x7fe380070aa0 2024-04-17T10:47:14.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.835+0000 7fe37f7fe640 1 -- 172.21.15.83:0/49756070 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3780a57b0 con 0x7fe380070aa0 2024-04-17T10:47:14.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.835+0000 7fe37f7fe640 1 -- 172.21.15.83:0/49756070 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3780a5b80 con 0x7fe380070aa0 2024-04-17T10:47:14.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.835+0000 7fe3870b8640 1 -- 172.21.15.83:0/49756070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe380070aa0 msgr2=0x7fe380070f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:14.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.835+0000 7fe3870b8640 1 --2- 172.21.15.83:0/49756070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe380070aa0 0x7fe380070f30 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fe3800b2480 tx=0x7fe37801c7e0 comp rx=0 tx=0).stop 2024-04-17T10:47:14.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.835+0000 7fe3870b8640 1 -- 172.21.15.83:0/49756070 shutdown_connections 2024-04-17T10:47:14.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.835+0000 7fe3870b8640 1 --2- 172.21.15.83:0/49756070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe380070aa0 0x7fe380070f30 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:14.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.835+0000 7fe3870b8640 1 --2- 172.21.15.83:0/49756070 >> v2:172.21.15.167:3300/0 conn(0x7fe38015e330 0x7fe38015e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:14.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.835+0000 7fe3870b8640 1 -- 172.21.15.83:0/49756070 >> 172.21.15.83:0/49756070 conn(0x7fe3800b15b0 msgr2=0x7fe3800b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:14.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.835+0000 7fe3870b8640 1 -- 172.21.15.83:0/49756070 shutdown_connections 2024-04-17T10:47:14.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.835+0000 7fe3870b8640 1 -- 172.21.15.83:0/49756070 wait complete. 2024-04-17T10:47:14.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.835+0000 7fe3870b8640 1 Processor -- start 2024-04-17T10:47:14.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.835+0000 7fe3870b8640 1 -- start start 2024-04-17T10:47:14.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.835+0000 7fe3870b8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe38006ae10 0x7fe38015da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:14.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.835+0000 7fe3870b8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe380070aa0 0x7fe380156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:14.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.835+0000 7fe3870b8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe38015e330 0x7fe38015bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:14.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.835+0000 7fe3870b8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38016ab90 con 0x7fe380070aa0 2024-04-17T10:47:14.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.835+0000 7fe3870b8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38016aa10 con 0x7fe38006ae10 2024-04-17T10:47:14.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.835+0000 7fe3870b8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe38016ad10 con 0x7fe38015e330 2024-04-17T10:47:14.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.835+0000 7fe37ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe380070aa0 0x7fe380156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:14.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.835+0000 7fe37ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe380070aa0 0x7fe380156af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59094/0 (socket says 172.21.15.83:59094) 2024-04-17T10:47:14.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.835+0000 7fe37ffff640 1 -- 172.21.15.83:0/2412487166 learned_addr learned my addr 172.21.15.83:0/2412487166 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:14.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.835+0000 7fe384e54640 1 --2- 172.21.15.83:0/2412487166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe38006ae10 0x7fe38015da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:14.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.835+0000 7fe385655640 1 --2- 172.21.15.83:0/2412487166 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe38015e330 0x7fe38015bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:14.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.835+0000 7fe37ffff640 1 -- 172.21.15.83:0/2412487166 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe38015e330 msgr2=0x7fe38015bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:14.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.835+0000 7fe37ffff640 1 --2- 172.21.15.83:0/2412487166 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe38015e330 0x7fe38015bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:14.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.835+0000 7fe37ffff640 1 -- 172.21.15.83:0/2412487166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe38006ae10 msgr2=0x7fe38015da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:14.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.835+0000 7fe37ffff640 1 --2- 172.21.15.83:0/2412487166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe38006ae10 0x7fe38015da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:14.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.835+0000 7fe37ffff640 1 -- 172.21.15.83:0/2412487166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3801507c0 con 0x7fe380070aa0 2024-04-17T10:47:14.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.835+0000 7fe37ffff640 1 --2- 172.21.15.83:0/2412487166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe380070aa0 0x7fe380156af0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fe37400d800 tx=0x7fe37400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:14.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.839+0000 7fe37d7fa640 1 -- 172.21.15.83:0/2412487166 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37400b6c0 con 0x7fe380070aa0 2024-04-17T10:47:14.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.839+0000 7fe37d7fa640 1 -- 172.21.15.83:0/2412487166 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe374019070 con 0x7fe380070aa0 2024-04-17T10:47:14.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.839+0000 7fe3870b8640 1 -- 172.21.15.83:0/2412487166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe38014f2f0 con 0x7fe380070aa0 2024-04-17T10:47:14.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.839+0000 7fe37d7fa640 1 -- 172.21.15.83:0/2412487166 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37400ba60 con 0x7fe380070aa0 2024-04-17T10:47:14.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.839+0000 7fe3870b8640 1 -- 172.21.15.83:0/2412487166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe38014f8b0 con 0x7fe380070aa0 2024-04-17T10:47:14.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.839+0000 7fe3870b8640 1 -- 172.21.15.83:0/2412487166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe380074f90 con 0x7fe380070aa0 2024-04-17T10:47:14.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.839+0000 7fe37d7fa640 1 -- 172.21.15.83:0/2412487166 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe37400bc00 con 0x7fe380070aa0 2024-04-17T10:47:14.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.839+0000 7fe37d7fa640 1 --2- 172.21.15.83:0/2412487166 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe36c066f60 0x7fe36c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:14.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.839+0000 7fe384e54640 1 --2- 172.21.15.83:0/2412487166 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe36c066f60 0x7fe36c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:14.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.839+0000 7fe37d7fa640 1 -- 172.21.15.83:0/2412487166 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fe3740854c0 con 0x7fe380070aa0 2024-04-17T10:47:14.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.839+0000 7fe384e54640 1 --2- 172.21.15.83:0/2412487166 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe36c066f60 0x7fe36c069420 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe370005890 tx=0x7fe370009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:14.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.843+0000 7fe37d7fa640 1 -- 172.21.15.83:0/2412487166 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe3801507c0 con 0x7fe380070aa0 2024-04-17T10:47:14.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.991+0000 7fe3870b8640 1 -- 172.21.15.83:0/2412487166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe380072740 con 0x7fe380070aa0 2024-04-17T10:47:14.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.991+0000 7fe37d7fa640 1 -- 172.21.15.83:0/2412487166 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe3740564a0 con 0x7fe380070aa0 2024-04-17T10:47:14.987 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:47:14.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.991+0000 7fe3870b8640 1 -- 172.21.15.83:0/2412487166 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe36c066f60 msgr2=0x7fe36c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:14.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.991+0000 7fe3870b8640 1 --2- 172.21.15.83:0/2412487166 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe36c066f60 0x7fe36c069420 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe370005890 tx=0x7fe370009040 comp rx=0 tx=0).stop 2024-04-17T10:47:14.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.991+0000 7fe3870b8640 1 -- 172.21.15.83:0/2412487166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe380070aa0 msgr2=0x7fe380156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:14.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.991+0000 7fe3870b8640 1 --2- 172.21.15.83:0/2412487166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe380070aa0 0x7fe380156af0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fe37400d800 tx=0x7fe37400dcc0 comp rx=0 tx=0).stop 2024-04-17T10:47:14.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.991+0000 7fe3870b8640 1 -- 172.21.15.83:0/2412487166 shutdown_connections 2024-04-17T10:47:14.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.991+0000 7fe3870b8640 1 --2- 172.21.15.83:0/2412487166 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe36c066f60 0x7fe36c069420 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:14.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.991+0000 7fe3870b8640 1 --2- 172.21.15.83:0/2412487166 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe38015e330 0x7fe38015bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:14.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.991+0000 7fe3870b8640 1 --2- 172.21.15.83:0/2412487166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe380070aa0 0x7fe380156af0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:14.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.991+0000 7fe3870b8640 1 --2- 172.21.15.83:0/2412487166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe38006ae10 0x7fe38015da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:14.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.991+0000 7fe3870b8640 1 -- 172.21.15.83:0/2412487166 >> 172.21.15.83:0/2412487166 conn(0x7fe3800b15b0 msgr2=0x7fe380154fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:14.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.995+0000 7fe3870b8640 1 -- 172.21.15.83:0/2412487166 shutdown_connections 2024-04-17T10:47:14.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:14.995+0000 7fe3870b8640 1 -- 172.21.15.83:0/2412487166 wait complete. 2024-04-17T10:47:15.004 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:47:15.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.123+0000 7f6f3f7c7640 1 Processor -- start 2024-04-17T10:47:15.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.123+0000 7f6f3f7c7640 1 -- start start 2024-04-17T10:47:15.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.127+0000 7f6f3f7c7640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6f3815e0f0 0x7f6f380b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:15.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.127+0000 7f6f3f7c7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f380ce400 0x7f6f38161c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:15.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.127+0000 7f6f3f7c7640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f3815e4d0 con 0x7f6f380b5450 2024-04-17T10:47:15.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.127+0000 7f6f3f7c7640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f38162b40 con 0x7f6f3815e0f0 2024-04-17T10:47:15.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.127+0000 7f6f3f7c7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f38162580 con 0x7f6f380ce400 2024-04-17T10:47:15.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.127+0000 7f6f3d563640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6f3815e0f0 0x7f6f380b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:15.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.127+0000 7f6f3cd62640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f380ce400 0x7f6f38161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:15.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.127+0000 7f6f3d563640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6f3815e0f0 0x7f6f380b6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:35818/0 (socket says 172.21.15.83:35818) 2024-04-17T10:47:15.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.127+0000 7f6f3d563640 1 -- 172.21.15.83:0/3016456024 learned_addr learned my addr 172.21.15.83:0/3016456024 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:15.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.127+0000 7f6f3d563640 1 -- 172.21.15.83:0/3016456024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f380ce400 msgr2=0x7f6f38161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:15.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.127+0000 7f6f3d563640 1 --2- 172.21.15.83:0/3016456024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f380ce400 0x7f6f38161c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:15.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.127+0000 7f6f3d563640 1 -- 172.21.15.83:0/3016456024 >> v1:172.21.15.167:6789/0 conn(0x7f6f380b5450 legacy=0x7f6f380d2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:15.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.127+0000 7f6f3d563640 1 -- 172.21.15.83:0/3016456024 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f38162860 con 0x7f6f3815e0f0 2024-04-17T10:47:15.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.127+0000 7f6f3cd62640 1 --2- 172.21.15.83:0/3016456024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f380ce400 0x7f6f38161c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:47:15.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.131+0000 7f6f3d563640 1 --2- 172.21.15.83:0/3016456024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f3815e0f0 0x7f6f380b6d00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6f34009870 tx=0x7f6f3401c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bece1ad11f4fc5f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:15.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.131+0000 7f6f2ffff640 1 -- 172.21.15.83:0/3016456024 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f340b2070 con 0x7f6f3815e0f0 2024-04-17T10:47:15.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.131+0000 7f6f2ffff640 1 -- 172.21.15.83:0/3016456024 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f340ad640 con 0x7f6f3815e0f0 2024-04-17T10:47:15.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.131+0000 7f6f2ffff640 1 -- 172.21.15.83:0/3016456024 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f340ada10 con 0x7f6f3815e0f0 2024-04-17T10:47:15.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.131+0000 7f6f3f7c7640 1 -- 172.21.15.83:0/3016456024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f3815e0f0 msgr2=0x7f6f380b6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:15.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.131+0000 7f6f3f7c7640 1 --2- 172.21.15.83:0/3016456024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f3815e0f0 0x7f6f380b6d00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6f34009870 tx=0x7f6f3401c4d0 comp rx=0 tx=0).stop 2024-04-17T10:47:15.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.131+0000 7f6f3f7c7640 1 -- 172.21.15.83:0/3016456024 shutdown_connections 2024-04-17T10:47:15.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.131+0000 7f6f3f7c7640 1 --2- 172.21.15.83:0/3016456024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f380ce400 0x7f6f38161c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:15.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.131+0000 7f6f3f7c7640 1 --2- 172.21.15.83:0/3016456024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f3815e0f0 0x7f6f380b6d00 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:15.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.131+0000 7f6f3f7c7640 1 -- 172.21.15.83:0/3016456024 >> 172.21.15.83:0/3016456024 conn(0x7f6f380d0450 msgr2=0x7f6f380d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:15.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.131+0000 7f6f3f7c7640 1 -- 172.21.15.83:0/3016456024 shutdown_connections 2024-04-17T10:47:15.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.131+0000 7f6f3f7c7640 1 -- 172.21.15.83:0/3016456024 wait complete. 2024-04-17T10:47:15.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.131+0000 7f6f3f7c7640 1 Processor -- start 2024-04-17T10:47:15.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.131+0000 7f6f3f7c7640 1 -- start start 2024-04-17T10:47:15.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.135+0000 7f6f3f7c7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f380b5450 0x7f6f380c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:15.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.135+0000 7f6f3f7c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f380ce400 0x7f6f3814e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:15.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.135+0000 7f6f3f7c7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f3815e0f0 0x7f6f3815bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:15.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.135+0000 7f6f3f7c7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f381622d0 con 0x7f6f380ce400 2024-04-17T10:47:15.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.135+0000 7f6f3f7c7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f38162150 con 0x7f6f3815e0f0 2024-04-17T10:47:15.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.135+0000 7f6f3f7c7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3816af50 con 0x7f6f380b5450 2024-04-17T10:47:15.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.135+0000 7f6f3cd62640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f380ce400 0x7f6f3814e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:15.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.135+0000 7f6f3cd62640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f380ce400 0x7f6f3814e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59108/0 (socket says 172.21.15.83:59108) 2024-04-17T10:47:15.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.135+0000 7f6f3cd62640 1 -- 172.21.15.83:0/1390801544 learned_addr learned my addr 172.21.15.83:0/1390801544 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:15.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.135+0000 7f6f3d563640 1 --2- 172.21.15.83:0/1390801544 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f380b5450 0x7f6f380c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:15.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.135+0000 7f6f3dd64640 1 --2- 172.21.15.83:0/1390801544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f3815e0f0 0x7f6f3815bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:15.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.135+0000 7f6f3cd62640 1 -- 172.21.15.83:0/1390801544 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f380b5450 msgr2=0x7f6f380c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:15.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.135+0000 7f6f3cd62640 1 --2- 172.21.15.83:0/1390801544 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f380b5450 0x7f6f380c9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:15.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.135+0000 7f6f3cd62640 1 -- 172.21.15.83:0/1390801544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f3815e0f0 msgr2=0x7f6f3815bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:15.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.135+0000 7f6f3cd62640 1 --2- 172.21.15.83:0/1390801544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f3815e0f0 0x7f6f3815bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:15.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.135+0000 7f6f3cd62640 1 -- 172.21.15.83:0/1390801544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3815aff0 con 0x7f6f380ce400 2024-04-17T10:47:15.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.135+0000 7f6f3cd62640 1 --2- 172.21.15.83:0/1390801544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f380ce400 0x7f6f3814e5e0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f6f2800e9d0 tx=0x7f6f2800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:15.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.135+0000 7f6f2dffb640 1 -- 172.21.15.83:0/1390801544 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f28017070 con 0x7f6f380ce400 2024-04-17T10:47:15.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.135+0000 7f6f2dffb640 1 -- 172.21.15.83:0/1390801544 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2800ca60 con 0x7f6f380ce400 2024-04-17T10:47:15.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.135+0000 7f6f3f7c7640 1 -- 172.21.15.83:0/1390801544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f3815b2d0 con 0x7f6f380ce400 2024-04-17T10:47:15.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.135+0000 7f6f2dffb640 1 -- 172.21.15.83:0/1390801544 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2800ce50 con 0x7f6f380ce400 2024-04-17T10:47:15.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.135+0000 7f6f3f7c7640 1 -- 172.21.15.83:0/1390801544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f3815b890 con 0x7f6f380ce400 2024-04-17T10:47:15.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.135+0000 7f6f3f7c7640 1 -- 172.21.15.83:0/1390801544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f380681a0 con 0x7f6f380ce400 2024-04-17T10:47:15.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.135+0000 7f6f2dffb640 1 -- 172.21.15.83:0/1390801544 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6f2801a070 con 0x7f6f380ce400 2024-04-17T10:47:15.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.135+0000 7f6f2dffb640 1 --2- 172.21.15.83:0/1390801544 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6f1c067100 0x7f6f1c0695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:15.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.135+0000 7f6f2dffb640 1 -- 172.21.15.83:0/1390801544 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f6f28085000 con 0x7f6f380ce400 2024-04-17T10:47:15.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.135+0000 7f6f3d563640 1 --2- 172.21.15.83:0/1390801544 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6f1c067100 0x7f6f1c0695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:15.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.135+0000 7f6f3d563640 1 --2- 172.21.15.83:0/1390801544 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6f1c067100 0x7f6f1c0695c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6f340af000 tx=0x7f6f34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:15.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.139+0000 7f6f2dffb640 1 -- 172.21.15.83:0/1390801544 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6f28055ff0 con 0x7f6f380ce400 2024-04-17T10:47:15.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.295+0000 7f6f3f7c7640 1 -- 172.21.15.83:0/1390801544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6f38074f90 con 0x7f6f380ce400 2024-04-17T10:47:15.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.299+0000 7f6f2dffb640 1 -- 172.21.15.83:0/1390801544 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6f280558f0 con 0x7f6f380ce400 2024-04-17T10:47:15.294 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:47:15.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.299+0000 7f6f3f7c7640 1 -- 172.21.15.83:0/1390801544 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6f1c067100 msgr2=0x7f6f1c0695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:15.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.299+0000 7f6f3f7c7640 1 --2- 172.21.15.83:0/1390801544 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6f1c067100 0x7f6f1c0695c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6f340af000 tx=0x7f6f34008040 comp rx=0 tx=0).stop 2024-04-17T10:47:15.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.299+0000 7f6f3f7c7640 1 -- 172.21.15.83:0/1390801544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f380ce400 msgr2=0x7f6f3814e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:15.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.299+0000 7f6f3f7c7640 1 --2- 172.21.15.83:0/1390801544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f380ce400 0x7f6f3814e5e0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f6f2800e9d0 tx=0x7f6f2800ee90 comp rx=0 tx=0).stop 2024-04-17T10:47:15.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.299+0000 7f6f3f7c7640 1 -- 172.21.15.83:0/1390801544 shutdown_connections 2024-04-17T10:47:15.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.299+0000 7f6f3f7c7640 1 --2- 172.21.15.83:0/1390801544 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6f1c067100 0x7f6f1c0695c0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:15.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.299+0000 7f6f3f7c7640 1 --2- 172.21.15.83:0/1390801544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f3815e0f0 0x7f6f3815bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:15.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.299+0000 7f6f3f7c7640 1 --2- 172.21.15.83:0/1390801544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f380ce400 0x7f6f3814e5e0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:15.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.299+0000 7f6f3f7c7640 1 --2- 172.21.15.83:0/1390801544 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f380b5450 0x7f6f380c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:15.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.299+0000 7f6f3f7c7640 1 -- 172.21.15.83:0/1390801544 >> 172.21.15.83:0/1390801544 conn(0x7f6f380d0450 msgr2=0x7f6f3815e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:15.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.299+0000 7f6f3f7c7640 1 -- 172.21.15.83:0/1390801544 shutdown_connections 2024-04-17T10:47:15.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.299+0000 7f6f3f7c7640 1 -- 172.21.15.83:0/1390801544 wait complete. 2024-04-17T10:47:15.313 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:47:15.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.439+0000 7fde936bb640 1 Processor -- start 2024-04-17T10:47:15.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.439+0000 7fde936bb640 1 -- start start 2024-04-17T10:47:15.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.443+0000 7fde936bb640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde8c15bac0 0x7fde8c071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:15.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.443+0000 7fde936bb640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde8c06c1f0 con 0x7fde8c070470 2024-04-17T10:47:15.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.443+0000 7fde936bb640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde8c070a10 con 0x7fde8c15b6e0 2024-04-17T10:47:15.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.443+0000 7fde936bb640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c070cf0 con 0x7fde8c15bac0 2024-04-17T10:47:15.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.443+0000 7fde91c58640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fde8c070470 0x7fde8c06ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40620/0 (socket says 172.21.15.83:40620) 2024-04-17T10:47:15.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.443+0000 7fde91c58640 1 -- 172.21.15.83:0/1828145478 learned_addr learned my addr 172.21.15.83:0/1828145478 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:15.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.443+0000 7fde90c56640 1 --2- 172.21.15.83:0/1828145478 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde8c15bac0 0x7fde8c071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:15.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.443+0000 7fde90c56640 1 -- 172.21.15.83:0/1828145478 >> v1:172.21.15.167:6789/0 conn(0x7fde8c15b6e0 legacy=0x7fde8c0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:15.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.443+0000 7fde90c56640 1 -- 172.21.15.83:0/1828145478 >> v1:172.21.15.83:6789/0 conn(0x7fde8c070470 legacy=0x7fde8c06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:15.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.443+0000 7fde90c56640 1 -- 172.21.15.83:0/1828145478 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde8c070fd0 con 0x7fde8c15bac0 2024-04-17T10:47:15.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.443+0000 7fde90c56640 1 --2- 172.21.15.83:0/1828145478 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde8c15bac0 0x7fde8c071cc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fde7c0097f0 tx=0x7fde7c01c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bdd69451ae0c5e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:15.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.447+0000 7fde83fff640 1 -- 172.21.15.83:0/1828145478 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c0b0070 con 0x7fde8c15bac0 2024-04-17T10:47:15.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.447+0000 7fde83fff640 1 -- 172.21.15.83:0/1828145478 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7c0ab600 con 0x7fde8c15bac0 2024-04-17T10:47:15.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.447+0000 7fde83fff640 1 -- 172.21.15.83:0/1828145478 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c0ab9d0 con 0x7fde8c15bac0 2024-04-17T10:47:15.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.447+0000 7fde936bb640 1 -- 172.21.15.83:0/1828145478 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde8c15bac0 msgr2=0x7fde8c071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:15.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.447+0000 7fde936bb640 1 --2- 172.21.15.83:0/1828145478 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde8c15bac0 0x7fde8c071cc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fde7c0097f0 tx=0x7fde7c01c4e0 comp rx=0 tx=0).stop 2024-04-17T10:47:15.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.447+0000 7fde936bb640 1 -- 172.21.15.83:0/1828145478 shutdown_connections 2024-04-17T10:47:15.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.447+0000 7fde936bb640 1 --2- 172.21.15.83:0/1828145478 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde8c15bac0 0x7fde8c071cc0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:15.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.447+0000 7fde936bb640 1 -- 172.21.15.83:0/1828145478 >> 172.21.15.83:0/1828145478 conn(0x7fde8c0b15b0 msgr2=0x7fde8c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:15.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.447+0000 7fde936bb640 1 -- 172.21.15.83:0/1828145478 shutdown_connections 2024-04-17T10:47:15.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.447+0000 7fde936bb640 1 -- 172.21.15.83:0/1828145478 wait complete. 2024-04-17T10:47:15.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.447+0000 7fde936bb640 1 Processor -- start 2024-04-17T10:47:15.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.447+0000 7fde936bb640 1 -- start start 2024-04-17T10:47:15.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.447+0000 7fde936bb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde8c070470 0x7fde8c1593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:15.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.447+0000 7fde936bb640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde8c15b6e0 0x7fde8c1537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:15.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.447+0000 7fde936bb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde8c15bac0 0x7fde8c153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:15.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.451+0000 7fde936bb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c06cee0 con 0x7fde8c070470 2024-04-17T10:47:15.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.451+0000 7fde936bb640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c06b770 con 0x7fde8c15bac0 2024-04-17T10:47:15.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.451+0000 7fde936bb640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c06d060 con 0x7fde8c15b6e0 2024-04-17T10:47:15.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.451+0000 7fde91457640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde8c070470 0x7fde8c1593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:15.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.451+0000 7fde91457640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde8c070470 0x7fde8c1593a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59112/0 (socket says 172.21.15.83:59112) 2024-04-17T10:47:15.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.451+0000 7fde91457640 1 -- 172.21.15.83:0/3461581485 learned_addr learned my addr 172.21.15.83:0/3461581485 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:15.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.451+0000 7fde90c56640 1 --2- 172.21.15.83:0/3461581485 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde8c15b6e0 0x7fde8c1537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:15.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.451+0000 7fde91c58640 1 --2- 172.21.15.83:0/3461581485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde8c15bac0 0x7fde8c153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:15.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.451+0000 7fde91457640 1 -- 172.21.15.83:0/3461581485 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde8c15b6e0 msgr2=0x7fde8c1537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:15.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.451+0000 7fde91457640 1 --2- 172.21.15.83:0/3461581485 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde8c15b6e0 0x7fde8c1537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:15.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.451+0000 7fde91457640 1 -- 172.21.15.83:0/3461581485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde8c15bac0 msgr2=0x7fde8c153d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:15.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.451+0000 7fde91457640 1 --2- 172.21.15.83:0/3461581485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde8c15bac0 0x7fde8c153d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:15.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.451+0000 7fde91457640 1 -- 172.21.15.83:0/3461581485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde8c1598e0 con 0x7fde8c070470 2024-04-17T10:47:15.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.451+0000 7fde90c56640 1 --2- 172.21.15.83:0/3461581485 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde8c15b6e0 0x7fde8c1537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:47:15.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.451+0000 7fde91c58640 1 --2- 172.21.15.83:0/3461581485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde8c15bac0 0x7fde8c153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:47:15.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.451+0000 7fde91457640 1 --2- 172.21.15.83:0/3461581485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde8c070470 0x7fde8c1593a0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fde8800c450 tx=0x7fde8800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:15.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.451+0000 7fde81ffb640 1 -- 172.21.15.83:0/3461581485 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde88016020 con 0x7fde8c070470 2024-04-17T10:47:15.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.451+0000 7fde81ffb640 1 -- 172.21.15.83:0/3461581485 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde880052d0 con 0x7fde8c070470 2024-04-17T10:47:15.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.451+0000 7fde936bb640 1 -- 172.21.15.83:0/3461581485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde8c159b60 con 0x7fde8c070470 2024-04-17T10:47:15.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.451+0000 7fde81ffb640 1 -- 172.21.15.83:0/3461581485 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde88007260 con 0x7fde8c070470 2024-04-17T10:47:15.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.451+0000 7fde936bb640 1 -- 172.21.15.83:0/3461581485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde8c15a120 con 0x7fde8c070470 2024-04-17T10:47:15.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.451+0000 7fde936bb640 1 -- 172.21.15.83:0/3461581485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde8c074f90 con 0x7fde8c070470 2024-04-17T10:47:15.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.451+0000 7fde81ffb640 1 -- 172.21.15.83:0/3461581485 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fde88007400 con 0x7fde8c070470 2024-04-17T10:47:15.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.451+0000 7fde81ffb640 1 --2- 172.21.15.83:0/3461581485 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fde70066f90 0x7fde70069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:15.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.451+0000 7fde81ffb640 1 -- 172.21.15.83:0/3461581485 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fde88084a20 con 0x7fde8c070470 2024-04-17T10:47:15.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.451+0000 7fde90c56640 1 --2- 172.21.15.83:0/3461581485 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fde70066f90 0x7fde70069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:15.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.451+0000 7fde90c56640 1 --2- 172.21.15.83:0/3461581485 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fde70066f90 0x7fde70069450 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fde7c0ad000 tx=0x7fde7c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:15.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.455+0000 7fde81ffb640 1 -- 172.21.15.83:0/3461581485 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fde88020020 con 0x7fde8c070470 2024-04-17T10:47:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.615+0000 7fde936bb640 1 -- 172.21.15.83:0/3461581485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fde8c0b2a60 con 0x7fde8c070470 2024-04-17T10:47:15.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.615+0000 7fde81ffb640 1 -- 172.21.15.83:0/3461581485 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7fde8c1598e0 con 0x7fde8c070470 2024-04-17T10:47:15.612 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:47:15.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.619+0000 7fde936bb640 1 -- 172.21.15.83:0/3461581485 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fde70066f90 msgr2=0x7fde70069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:15.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.619+0000 7fde936bb640 1 --2- 172.21.15.83:0/3461581485 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fde70066f90 0x7fde70069450 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fde7c0ad000 tx=0x7fde7c008040 comp rx=0 tx=0).stop 2024-04-17T10:47:15.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.619+0000 7fde936bb640 1 -- 172.21.15.83:0/3461581485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde8c070470 msgr2=0x7fde8c1593a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:15.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.619+0000 7fde936bb640 1 --2- 172.21.15.83:0/3461581485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde8c070470 0x7fde8c1593a0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fde8800c450 tx=0x7fde8800c910 comp rx=0 tx=0).stop 2024-04-17T10:47:15.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.619+0000 7fde936bb640 1 -- 172.21.15.83:0/3461581485 shutdown_connections 2024-04-17T10:47:15.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.619+0000 7fde936bb640 1 --2- 172.21.15.83:0/3461581485 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fde70066f90 0x7fde70069450 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:15.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.619+0000 7fde936bb640 1 --2- 172.21.15.83:0/3461581485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde8c15bac0 0x7fde8c153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:15.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.619+0000 7fde936bb640 1 --2- 172.21.15.83:0/3461581485 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde8c15b6e0 0x7fde8c1537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:15.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.619+0000 7fde936bb640 1 --2- 172.21.15.83:0/3461581485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde8c070470 0x7fde8c1593a0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:15.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.619+0000 7fde936bb640 1 -- 172.21.15.83:0/3461581485 >> 172.21.15.83:0/3461581485 conn(0x7fde8c0b15b0 msgr2=0x7fde8c148060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:15.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.619+0000 7fde936bb640 1 -- 172.21.15.83:0/3461581485 shutdown_connections 2024-04-17T10:47:15.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.619+0000 7fde936bb640 1 -- 172.21.15.83:0/3461581485 wait complete. 2024-04-17T10:47:15.632 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:47:15.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.755+0000 7f0655704640 1 Processor -- start 2024-04-17T10:47:15.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.755+0000 7f0655704640 1 -- start start 2024-04-17T10:47:15.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.759+0000 7f0655704640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f065015ba30 0x7f0650072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:15.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.759+0000 7f0655704640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f065015c960 0x7f06500cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:15.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.759+0000 7f0655704640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f06500cfa10 0x7f065006c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:15.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.759+0000 7f0655704640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0650071ea0 con 0x7f06500cfa10 2024-04-17T10:47:15.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.759+0000 7f0655704640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06500cfea0 con 0x7f065015ba30 2024-04-17T10:47:15.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.759+0000 7f0655704640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065006e320 con 0x7f065015c960 2024-04-17T10:47:15.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.759+0000 7f064f7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f06500cfa10 0x7f065006c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:15.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.759+0000 7f064f7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f06500cfa10 0x7f065006c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59122/0 (socket says 172.21.15.83:59122) 2024-04-17T10:47:15.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.759+0000 7f064f7fe640 1 -- 172.21.15.83:0/573620281 learned_addr learned my addr 172.21.15.83:0/573620281 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:15.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.759+0000 7f064effd640 1 --2- 172.21.15.83:0/573620281 >> v2:172.21.15.167:3300/0 conn(0x7f065015ba30 0x7f0650072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:15.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.759+0000 7f064e7fc640 1 --2- 172.21.15.83:0/573620281 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f065015c960 0x7f06500cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:15.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.759+0000 7f064f7fe640 1 -- 172.21.15.83:0/573620281 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f065015c960 msgr2=0x7f06500cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:15.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.759+0000 7f064f7fe640 1 --2- 172.21.15.83:0/573620281 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f065015c960 0x7f06500cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:15.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.759+0000 7f064f7fe640 1 -- 172.21.15.83:0/573620281 >> v2:172.21.15.167:3300/0 conn(0x7f065015ba30 msgr2=0x7f0650072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:15.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.759+0000 7f064f7fe640 1 --2- 172.21.15.83:0/573620281 >> v2:172.21.15.167:3300/0 conn(0x7f065015ba30 0x7f0650072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:15.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.759+0000 7f064f7fe640 1 -- 172.21.15.83:0/573620281 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065006e040 con 0x7f06500cfa10 2024-04-17T10:47:15.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.759+0000 7f064f7fe640 1 --2- 172.21.15.83:0/573620281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06500cfa10 0x7f065006c490 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f06500b2410 tx=0x7f064801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb638d621c44ab6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:15.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.759+0000 7f064dffb640 1 -- 172.21.15.83:0/573620281 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06480a56d0 con 0x7f06500cfa10 2024-04-17T10:47:15.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.759+0000 7f064dffb640 1 -- 172.21.15.83:0/573620281 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06480a5870 con 0x7f06500cfa10 2024-04-17T10:47:15.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.759+0000 7f064dffb640 1 -- 172.21.15.83:0/573620281 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06480a5c40 con 0x7f06500cfa10 2024-04-17T10:47:15.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.763+0000 7f0655704640 1 -- 172.21.15.83:0/573620281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06500cfa10 msgr2=0x7f065006c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:15.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.763+0000 7f0655704640 1 --2- 172.21.15.83:0/573620281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06500cfa10 0x7f065006c490 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f06500b2410 tx=0x7f064801c960 comp rx=0 tx=0).stop 2024-04-17T10:47:15.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.763+0000 7f0655704640 1 -- 172.21.15.83:0/573620281 shutdown_connections 2024-04-17T10:47:15.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.763+0000 7f0655704640 1 --2- 172.21.15.83:0/573620281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06500cfa10 0x7f065006c490 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:15.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.763+0000 7f0655704640 1 --2- 172.21.15.83:0/573620281 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f065015c960 0x7f06500cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:15.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.763+0000 7f0655704640 1 --2- 172.21.15.83:0/573620281 >> v2:172.21.15.167:3300/0 conn(0x7f065015ba30 0x7f0650072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:15.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.763+0000 7f0655704640 1 -- 172.21.15.83:0/573620281 >> 172.21.15.83:0/573620281 conn(0x7f065006abe0 msgr2=0x7f065006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:15.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.763+0000 7f0655704640 1 -- 172.21.15.83:0/573620281 shutdown_connections 2024-04-17T10:47:15.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.763+0000 7f0655704640 1 -- 172.21.15.83:0/573620281 wait complete. 2024-04-17T10:47:15.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.763+0000 7f0655704640 1 Processor -- start 2024-04-17T10:47:15.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.763+0000 7f0655704640 1 -- start start 2024-04-17T10:47:15.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.763+0000 7f0655704640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06500cfa10 0x7f0650142e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:15.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.763+0000 7f0655704640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f065015ba30 0x7f06501495b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:15.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.763+0000 7f0655704640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f065015c960 0x7f065014fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:15.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.763+0000 7f0655704640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065006de20 con 0x7f06500cfa10 2024-04-17T10:47:15.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.763+0000 7f0655704640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065006dca0 con 0x7f065015c960 2024-04-17T10:47:15.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.763+0000 7f0655704640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065015cd40 con 0x7f065015ba30 2024-04-17T10:47:15.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.763+0000 7f064effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06500cfa10 0x7f0650142e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:15.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.763+0000 7f064effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06500cfa10 0x7f0650142e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59132/0 (socket says 172.21.15.83:59132) 2024-04-17T10:47:15.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.763+0000 7f064effd640 1 -- 172.21.15.83:0/674022234 learned_addr learned my addr 172.21.15.83:0/674022234 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:15.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.763+0000 7f064f7fe640 1 --2- 172.21.15.83:0/674022234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f065015c960 0x7f065014fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:15.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.763+0000 7f064e7fc640 1 --2- 172.21.15.83:0/674022234 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f065015ba30 0x7f06501495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:15.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.767+0000 7f064effd640 1 -- 172.21.15.83:0/674022234 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f065015ba30 msgr2=0x7f06501495b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:15.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.767+0000 7f064effd640 1 --2- 172.21.15.83:0/674022234 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f065015ba30 0x7f06501495b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:15.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.767+0000 7f064effd640 1 -- 172.21.15.83:0/674022234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f065015c960 msgr2=0x7f065014fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:15.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.767+0000 7f064effd640 1 --2- 172.21.15.83:0/674022234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f065015c960 0x7f065014fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:15.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.767+0000 7f064effd640 1 -- 172.21.15.83:0/674022234 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06501437e0 con 0x7f06500cfa10 2024-04-17T10:47:15.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.767+0000 7f064e7fc640 1 --2- 172.21.15.83:0/674022234 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f065015ba30 0x7f06501495b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:15.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.767+0000 7f064effd640 1 --2- 172.21.15.83:0/674022234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06500cfa10 0x7f0650142e10 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f064400d800 tx=0x7f064400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:15.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.767+0000 7f06377fe640 1 -- 172.21.15.83:0/674022234 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0644018030 con 0x7f06500cfa10 2024-04-17T10:47:15.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.767+0000 7f06377fe640 1 -- 172.21.15.83:0/674022234 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0644009640 con 0x7f06500cfa10 2024-04-17T10:47:15.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.767+0000 7f0655704640 1 -- 172.21.15.83:0/674022234 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0650149ee0 con 0x7f06500cfa10 2024-04-17T10:47:15.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.767+0000 7f06377fe640 1 -- 172.21.15.83:0/674022234 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0644009810 con 0x7f06500cfa10 2024-04-17T10:47:15.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.767+0000 7f0655704640 1 -- 172.21.15.83:0/674022234 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0650143a80 con 0x7f06500cfa10 2024-04-17T10:47:15.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.767+0000 7f0655704640 1 -- 172.21.15.83:0/674022234 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0650074f90 con 0x7f06500cfa10 2024-04-17T10:47:15.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.767+0000 7f06377fe640 1 -- 172.21.15.83:0/674022234 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f064401b070 con 0x7f06500cfa10 2024-04-17T10:47:15.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.767+0000 7f06377fe640 1 --2- 172.21.15.83:0/674022234 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f063c066f90 0x7f063c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:15.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.767+0000 7f06377fe640 1 -- 172.21.15.83:0/674022234 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f0644085190 con 0x7f06500cfa10 2024-04-17T10:47:15.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.767+0000 7f064e7fc640 1 --2- 172.21.15.83:0/674022234 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f063c066f90 0x7f063c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:15.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.767+0000 7f064e7fc640 1 --2- 172.21.15.83:0/674022234 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f063c066f90 0x7f063c069450 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f06501492d0 tx=0x7f0640008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:15.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.771+0000 7f06377fe640 1 -- 172.21.15.83:0/674022234 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0644056210 con 0x7f06500cfa10 2024-04-17T10:47:15.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.955+0000 7f0655704640 1 -- 172.21.15.83:0/674022234 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06500cf4b0 con 0x7f06500cfa10 2024-04-17T10:47:15.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.955+0000 7f06377fe640 1 -- 172.21.15.83:0/674022234 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f0644055b10 con 0x7f06500cfa10 2024-04-17T10:47:15.951 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:47:15.951 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5429,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6834/3936640219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3936640219},{"type":"v1","addr":"172.21.15.167:6834","nonce":3936640219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":5998,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4152882285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6833","nonce":4152882285},{"type":"v1","addr":"172.21.15.167:6835","nonce":4152882285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1562645860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1562645860},{"type":"v1","addr":"172.21.15.83:6837","nonce":1562645860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:46:36.578918+0000","modified":"2024-04-17T10:46:38.561701+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7575":{"gid":7575,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/1056175765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1056175765},{"type":"v1","addr":"172.21.15.83:6836","nonce":1056175765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-17T10:47:15.951 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 56 2024-04-17T10:47:15.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.955+0000 7f0655704640 1 -- 172.21.15.83:0/674022234 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f063c066f90 msgr2=0x7f063c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:15.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.955+0000 7f0655704640 1 --2- 172.21.15.83:0/674022234 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f063c066f90 0x7f063c069450 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f06501492d0 tx=0x7f0640008040 comp rx=0 tx=0).stop 2024-04-17T10:47:15.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.955+0000 7f0655704640 1 -- 172.21.15.83:0/674022234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06500cfa10 msgr2=0x7f0650142e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:15.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.955+0000 7f0655704640 1 --2- 172.21.15.83:0/674022234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06500cfa10 0x7f0650142e10 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f064400d800 tx=0x7f064400dcc0 comp rx=0 tx=0).stop 2024-04-17T10:47:15.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.959+0000 7f0655704640 1 -- 172.21.15.83:0/674022234 shutdown_connections 2024-04-17T10:47:15.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.959+0000 7f0655704640 1 --2- 172.21.15.83:0/674022234 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f063c066f90 0x7f063c069450 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:15.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.959+0000 7f0655704640 1 --2- 172.21.15.83:0/674022234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f065015c960 0x7f065014fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:15.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.959+0000 7f0655704640 1 --2- 172.21.15.83:0/674022234 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f065015ba30 0x7f06501495b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:15.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.959+0000 7f0655704640 1 --2- 172.21.15.83:0/674022234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06500cfa10 0x7f0650142e10 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:15.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.959+0000 7f0655704640 1 -- 172.21.15.83:0/674022234 >> 172.21.15.83:0/674022234 conn(0x7f065006abe0 msgr2=0x7f0650152280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:15.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.959+0000 7f0655704640 1 -- 172.21.15.83:0/674022234 shutdown_connections 2024-04-17T10:47:15.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:15.959+0000 7f0655704640 1 -- 172.21.15.83:0/674022234 wait complete. 2024-04-17T10:47:15.969 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:47:16.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.091+0000 7f35143d8640 1 Processor -- start 2024-04-17T10:47:16.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.091+0000 7f35143d8640 1 -- start start 2024-04-17T10:47:16.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.091+0000 7f35143d8640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f350c072330 0x7f350c0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:16.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.091+0000 7f35143d8640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f350c0cfa10 0x7f350c06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:16.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.091+0000 7f35143d8640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f350c0cfea0 con 0x7f350c15ba30 2024-04-17T10:47:16.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.091+0000 7f35143d8640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f350c071ea0 con 0x7f350c072330 2024-04-17T10:47:16.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.091+0000 7f35143d8640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f350c06e320 con 0x7f350c0cfa10 2024-04-17T10:47:16.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.091+0000 7f3511973640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f350c072330 0x7f350c0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:16.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.091+0000 7f3512174640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f350c15ba30 0x7f350c15c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40622/0 (socket says 172.21.15.83:40622) 2024-04-17T10:47:16.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.091+0000 7f3512174640 1 -- 172.21.15.83:0/1644442823 learned_addr learned my addr 172.21.15.83:0/1644442823 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:16.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.091+0000 7f3511172640 1 -- 172.21.15.83:0/1644442823 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 458558065 0 0) 0x7f350c0cfea0 con 0x7f350c15ba30 2024-04-17T10:47:16.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.091+0000 7f3511172640 1 -- 172.21.15.83:0/1644442823 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34f8003530 con 0x7f350c15ba30 2024-04-17T10:47:16.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.091+0000 7f3512975640 1 --2- 172.21.15.83:0/1644442823 >> v2:172.21.15.167:3300/0 conn(0x7f350c0cfa10 0x7f350c06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:16.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.091+0000 7f3511973640 1 -- 172.21.15.83:0/1644442823 >> v2:172.21.15.167:3300/0 conn(0x7f350c0cfa10 msgr2=0x7f350c06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:16.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.091+0000 7f3511973640 1 --2- 172.21.15.83:0/1644442823 >> v2:172.21.15.167:3300/0 conn(0x7f350c0cfa10 0x7f350c06c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:16.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.091+0000 7f3511973640 1 -- 172.21.15.83:0/1644442823 >> v1:172.21.15.83:6789/0 conn(0x7f350c15ba30 legacy=0x7f350c15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:16.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.091+0000 7f3511973640 1 -- 172.21.15.83:0/1644442823 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f350c06e040 con 0x7f350c072330 2024-04-17T10:47:16.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.095+0000 7f3511973640 1 --2- 172.21.15.83:0/1644442823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f350c072330 0x7f350c0cf4d0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f350c0676a0 tx=0x7f34fc01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c71a713c17d7b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:16.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.095+0000 7f3511172640 1 -- 172.21.15.83:0/1644442823 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc0b0070 con 0x7f350c072330 2024-04-17T10:47:16.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.095+0000 7f3511172640 1 -- 172.21.15.83:0/1644442823 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34fc0ab600 con 0x7f350c072330 2024-04-17T10:47:16.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.095+0000 7f3511172640 1 -- 172.21.15.83:0/1644442823 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc0ab9d0 con 0x7f350c072330 2024-04-17T10:47:16.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.095+0000 7f35143d8640 1 -- 172.21.15.83:0/1644442823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f350c072330 msgr2=0x7f350c0cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:16.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.095+0000 7f35143d8640 1 --2- 172.21.15.83:0/1644442823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f350c072330 0x7f350c0cf4d0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f350c0676a0 tx=0x7f34fc01c3f0 comp rx=0 tx=0).stop 2024-04-17T10:47:16.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.095+0000 7f35143d8640 1 -- 172.21.15.83:0/1644442823 shutdown_connections 2024-04-17T10:47:16.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.095+0000 7f35143d8640 1 --2- 172.21.15.83:0/1644442823 >> v2:172.21.15.167:3300/0 conn(0x7f350c0cfa10 0x7f350c06c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:16.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.095+0000 7f35143d8640 1 --2- 172.21.15.83:0/1644442823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f350c072330 0x7f350c0cf4d0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:16.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.095+0000 7f35143d8640 1 -- 172.21.15.83:0/1644442823 >> 172.21.15.83:0/1644442823 conn(0x7f350c06abe0 msgr2=0x7f350c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:16.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.095+0000 7f35143d8640 1 -- 172.21.15.83:0/1644442823 shutdown_connections 2024-04-17T10:47:16.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.095+0000 7f35143d8640 1 -- 172.21.15.83:0/1644442823 wait complete. 2024-04-17T10:47:16.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.095+0000 7f35143d8640 1 Processor -- start 2024-04-17T10:47:16.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.095+0000 7f35143d8640 1 -- start start 2024-04-17T10:47:16.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.099+0000 7f35143d8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f350c072330 0x7f350c0cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:16.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.099+0000 7f35143d8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f350c0cfa10 0x7f350c1577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:16.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.099+0000 7f35143d8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f350c15ba30 0x7f350c14f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:16.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.099+0000 7f35143d8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c06de20 con 0x7f350c072330 2024-04-17T10:47:16.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.099+0000 7f35143d8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c06dca0 con 0x7f350c0cfa10 2024-04-17T10:47:16.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.099+0000 7f35143d8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f350c168520 con 0x7f350c15ba30 2024-04-17T10:47:16.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.099+0000 7f3512174640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f350c072330 0x7f350c0cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:16.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.099+0000 7f3512174640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f350c072330 0x7f350c0cdce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59154/0 (socket says 172.21.15.83:59154) 2024-04-17T10:47:16.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.099+0000 7f3512174640 1 -- 172.21.15.83:0/3324673575 learned_addr learned my addr 172.21.15.83:0/3324673575 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:16.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.099+0000 7f3512975640 1 --2- 172.21.15.83:0/3324673575 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f350c15ba30 0x7f350c14f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:16.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.099+0000 7f3511973640 1 --2- 172.21.15.83:0/3324673575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f350c0cfa10 0x7f350c1577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:16.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.099+0000 7f3512174640 1 -- 172.21.15.83:0/3324673575 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f350c15ba30 msgr2=0x7f350c14f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:16.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.099+0000 7f3512174640 1 --2- 172.21.15.83:0/3324673575 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f350c15ba30 0x7f350c14f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:16.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.099+0000 7f3512174640 1 -- 172.21.15.83:0/3324673575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f350c0cfa10 msgr2=0x7f350c1577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:16.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.099+0000 7f3512174640 1 --2- 172.21.15.83:0/3324673575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f350c0cfa10 0x7f350c1577f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:16.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.099+0000 7f3512174640 1 -- 172.21.15.83:0/3324673575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f350c156a30 con 0x7f350c072330 2024-04-17T10:47:16.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.099+0000 7f3512174640 1 --2- 172.21.15.83:0/3324673575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f350c072330 0x7f350c0cdce0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f350800c480 tx=0x7f350800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:16.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.099+0000 7f3502ffd640 1 -- 172.21.15.83:0/3324673575 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508017070 con 0x7f350c072330 2024-04-17T10:47:16.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.099+0000 7f3502ffd640 1 -- 172.21.15.83:0/3324673575 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3508005cf0 con 0x7f350c072330 2024-04-17T10:47:16.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.099+0000 7f3502ffd640 1 -- 172.21.15.83:0/3324673575 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508007290 con 0x7f350c072330 2024-04-17T10:47:16.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.099+0000 7f35143d8640 1 -- 172.21.15.83:0/3324673575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f350c156d10 con 0x7f350c072330 2024-04-17T10:47:16.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.099+0000 7f35143d8640 1 -- 172.21.15.83:0/3324673575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f350c1572d0 con 0x7f350c072330 2024-04-17T10:47:16.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.099+0000 7f35143d8640 1 -- 172.21.15.83:0/3324673575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f350c074f90 con 0x7f350c072330 2024-04-17T10:47:16.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.099+0000 7f3502ffd640 1 -- 172.21.15.83:0/3324673575 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f350801a070 con 0x7f350c072330 2024-04-17T10:47:16.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.099+0000 7f3502ffd640 1 --2- 172.21.15.83:0/3324673575 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f34f0066f60 0x7f34f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:16.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.099+0000 7f3502ffd640 1 -- 172.21.15.83:0/3324673575 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f3508084400 con 0x7f350c072330 2024-04-17T10:47:16.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.099+0000 7f3511973640 1 --2- 172.21.15.83:0/3324673575 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f34f0066f60 0x7f34f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:16.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.103+0000 7f3511973640 1 --2- 172.21.15.83:0/3324673575 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f34f0066f60 0x7f34f0069420 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f34fc0ad000 tx=0x7f34fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:16.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.103+0000 7f3502ffd640 1 -- 172.21.15.83:0/3324673575 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3508055480 con 0x7f350c072330 2024-04-17T10:47:16.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.263+0000 7f35143d8640 1 -- 172.21.15.83:0/3324673575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f350c070da0 con 0x7f350c072330 2024-04-17T10:47:16.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.267+0000 7f3502ffd640 1 -- 172.21.15.83:0/3324673575 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+16638 (secure 0 0 0) 0x7f3508054d80 con 0x7f350c072330 2024-04-17T10:47:16.262 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:47:16.263 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":91,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:46:55.414653+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:46:34.779174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-17T10:46:35.563972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.10","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:47:16.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.267+0000 7f35143d8640 1 -- 172.21.15.83:0/3324673575 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f34f0066f60 msgr2=0x7f34f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:16.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.267+0000 7f35143d8640 1 --2- 172.21.15.83:0/3324673575 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f34f0066f60 0x7f34f0069420 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f34fc0ad000 tx=0x7f34fc008040 comp rx=0 tx=0).stop 2024-04-17T10:47:16.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.267+0000 7f35143d8640 1 -- 172.21.15.83:0/3324673575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f350c072330 msgr2=0x7f350c0cdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:16.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.267+0000 7f35143d8640 1 --2- 172.21.15.83:0/3324673575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f350c072330 0x7f350c0cdce0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f350800c480 tx=0x7f350800c940 comp rx=0 tx=0).stop 2024-04-17T10:47:16.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.267+0000 7f35143d8640 1 -- 172.21.15.83:0/3324673575 shutdown_connections 2024-04-17T10:47:16.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.267+0000 7f35143d8640 1 --2- 172.21.15.83:0/3324673575 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f34f0066f60 0x7f34f0069420 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:16.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.267+0000 7f35143d8640 1 --2- 172.21.15.83:0/3324673575 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f350c15ba30 0x7f350c14f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:16.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.267+0000 7f35143d8640 1 --2- 172.21.15.83:0/3324673575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f350c0cfa10 0x7f350c1577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:16.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.267+0000 7f35143d8640 1 --2- 172.21.15.83:0/3324673575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f350c072330 0x7f350c0cdce0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:16.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.267+0000 7f35143d8640 1 -- 172.21.15.83:0/3324673575 >> 172.21.15.83:0/3324673575 conn(0x7f350c06abe0 msgr2=0x7f350c15e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:16.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.267+0000 7f35143d8640 1 -- 172.21.15.83:0/3324673575 shutdown_connections 2024-04-17T10:47:16.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.267+0000 7f35143d8640 1 -- 172.21.15.83:0/3324673575 wait complete. 2024-04-17T10:47:16.283 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-17T10:47:16.283 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:47:16.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.403+0000 7f1e1e1e9640 1 Processor -- start 2024-04-17T10:47:16.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.403+0000 7f1e1e1e9640 1 -- start start 2024-04-17T10:47:16.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.407+0000 7f1e1e1e9640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1e180aedd0 0x7f1e1806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:16.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.407+0000 7f1e1e1e9640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1e1806dae0 0x7f1e180b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:16.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.407+0000 7f1e1e1e9640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1815e440 con 0x7f1e1815db50 2024-04-17T10:47:16.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.407+0000 7f1e1e1e9640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e1815e720 con 0x7f1e180aedd0 2024-04-17T10:47:16.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.407+0000 7f1e1e1e9640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e1815ea00 con 0x7f1e1806dae0 2024-04-17T10:47:16.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.407+0000 7f1e16ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1e180aedd0 0x7f1e1806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:16.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.407+0000 7f1e16ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1e180aedd0 0x7f1e1806d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59164/0 (socket says 172.21.15.83:59164) 2024-04-17T10:47:16.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.407+0000 7f1e16ffd640 1 -- 172.21.15.83:0/1174850200 learned_addr learned my addr 172.21.15.83:0/1174850200 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:16.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.407+0000 7f1e17fff640 1 --2- 172.21.15.83:0/1174850200 >> v2:172.21.15.167:3300/0 conn(0x7f1e1806dae0 0x7f1e180b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:16.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.407+0000 7f1e16ffd640 1 -- 172.21.15.83:0/1174850200 >> v2:172.21.15.167:3300/0 conn(0x7f1e1806dae0 msgr2=0x7f1e180b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:16.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.407+0000 7f1e16ffd640 1 --2- 172.21.15.83:0/1174850200 >> v2:172.21.15.167:3300/0 conn(0x7f1e1806dae0 0x7f1e180b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:16.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.407+0000 7f1e16ffd640 1 -- 172.21.15.83:0/1174850200 >> v1:172.21.15.167:6789/0 conn(0x7f1e1815db50 legacy=0x7f1e18070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:16.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.407+0000 7f1e16ffd640 1 -- 172.21.15.83:0/1174850200 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1815ece0 con 0x7f1e180aedd0 2024-04-17T10:47:16.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.407+0000 7f1e17fff640 1 --2- 172.21.15.83:0/1174850200 >> v2:172.21.15.167:3300/0 conn(0x7f1e1806dae0 0x7f1e180b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:16.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.411+0000 7f1e16ffd640 1 --2- 172.21.15.83:0/1174850200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e180aedd0 0x7f1e1806d5a0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c009870 tx=0x7f1e0c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5eaaad9ab46bf67 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:16.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.411+0000 7f1e167fc640 1 -- 172.21.15.83:0/1174850200 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c0b0070 con 0x7f1e180aedd0 2024-04-17T10:47:16.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.411+0000 7f1e167fc640 1 -- 172.21.15.83:0/1174850200 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0c0ab600 con 0x7f1e180aedd0 2024-04-17T10:47:16.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.411+0000 7f1e167fc640 1 -- 172.21.15.83:0/1174850200 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c0ab9d0 con 0x7f1e180aedd0 2024-04-17T10:47:16.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.411+0000 7f1e1e1e9640 1 -- 172.21.15.83:0/1174850200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e180aedd0 msgr2=0x7f1e1806d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:16.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.411+0000 7f1e1e1e9640 1 --2- 172.21.15.83:0/1174850200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e180aedd0 0x7f1e1806d5a0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c009870 tx=0x7f1e0c01c4e0 comp rx=0 tx=0).stop 2024-04-17T10:47:16.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.411+0000 7f1e1e1e9640 1 -- 172.21.15.83:0/1174850200 shutdown_connections 2024-04-17T10:47:16.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.411+0000 7f1e1e1e9640 1 --2- 172.21.15.83:0/1174850200 >> v2:172.21.15.167:3300/0 conn(0x7f1e1806dae0 0x7f1e180b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:16.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.411+0000 7f1e1e1e9640 1 --2- 172.21.15.83:0/1174850200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e180aedd0 0x7f1e1806d5a0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:16.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.411+0000 7f1e1e1e9640 1 -- 172.21.15.83:0/1174850200 >> 172.21.15.83:0/1174850200 conn(0x7f1e1806abe0 msgr2=0x7f1e1806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:16.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.411+0000 7f1e1e1e9640 1 -- 172.21.15.83:0/1174850200 shutdown_connections 2024-04-17T10:47:16.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.411+0000 7f1e1e1e9640 1 -- 172.21.15.83:0/1174850200 wait complete. 2024-04-17T10:47:16.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.415+0000 7f1e1e1e9640 1 Processor -- start 2024-04-17T10:47:16.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.415+0000 7f1e1e1e9640 1 -- start start 2024-04-17T10:47:16.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.415+0000 7f1e1e1e9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e1806dae0 0x7f1e1814f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:16.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.415+0000 7f1e1e1e9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e180aedd0 0x7f1e1815ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:16.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.415+0000 7f1e1e1e9640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e1815db50 0x7f1e1806c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:16.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.415+0000 7f1e1e1e9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e180b3ef0 con 0x7f1e1806dae0 2024-04-17T10:47:16.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.415+0000 7f1e1e1e9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e180b3d70 con 0x7f1e180aedd0 2024-04-17T10:47:16.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.415+0000 7f1e1e1e9640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e180b4070 con 0x7f1e1815db50 2024-04-17T10:47:16.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.415+0000 7f1e177fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e1806dae0 0x7f1e1814f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:16.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.415+0000 7f1e177fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e1806dae0 0x7f1e1814f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59172/0 (socket says 172.21.15.83:59172) 2024-04-17T10:47:16.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.415+0000 7f1e177fe640 1 -- 172.21.15.83:0/742698645 learned_addr learned my addr 172.21.15.83:0/742698645 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:16.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.415+0000 7f1e177fe640 1 -- 172.21.15.83:0/742698645 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e1815db50 msgr2=0x7f1e1806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:16.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.415+0000 7f1e177fe640 1 --2- 172.21.15.83:0/742698645 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e1815db50 0x7f1e1806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:16.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.415+0000 7f1e177fe640 1 -- 172.21.15.83:0/742698645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e180aedd0 msgr2=0x7f1e1815ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:16.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.415+0000 7f1e177fe640 1 --2- 172.21.15.83:0/742698645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e180aedd0 0x7f1e1815ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:16.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.415+0000 7f1e177fe640 1 -- 172.21.15.83:0/742698645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e180b2620 con 0x7f1e1806dae0 2024-04-17T10:47:16.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.415+0000 7f1e177fe640 1 --2- 172.21.15.83:0/742698645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e1806dae0 0x7f1e1814f390 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800b3e0 tx=0x7f1e0800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:16.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.415+0000 7f1dfffff640 1 -- 172.21.15.83:0/742698645 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08015020 con 0x7f1e1806dae0 2024-04-17T10:47:16.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.415+0000 7f1dfffff640 1 -- 172.21.15.83:0/742698645 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e08011850 con 0x7f1e1806dae0 2024-04-17T10:47:16.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.415+0000 7f1dfffff640 1 -- 172.21.15.83:0/742698645 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08011c40 con 0x7f1e1806dae0 2024-04-17T10:47:16.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.415+0000 7f1e1e1e9640 1 -- 172.21.15.83:0/742698645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e18158ae0 con 0x7f1e1806dae0 2024-04-17T10:47:16.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.415+0000 7f1e1e1e9640 1 -- 172.21.15.83:0/742698645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e18159380 con 0x7f1e1806dae0 2024-04-17T10:47:16.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.419+0000 7f1e1e1e9640 1 -- 172.21.15.83:0/742698645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e18074f90 con 0x7f1e1806dae0 2024-04-17T10:47:16.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.419+0000 7f1dfffff640 1 -- 172.21.15.83:0/742698645 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1e08018020 con 0x7f1e1806dae0 2024-04-17T10:47:16.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.419+0000 7f1dfffff640 1 --2- 172.21.15.83:0/742698645 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1e04066f60 0x7f1e04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:16.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.419+0000 7f1e16ffd640 1 --2- 172.21.15.83:0/742698645 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1e04066f60 0x7f1e04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:16.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.419+0000 7f1dfffff640 1 -- 172.21.15.83:0/742698645 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f1e08085030 con 0x7f1e1806dae0 2024-04-17T10:47:16.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.419+0000 7f1e16ffd640 1 --2- 172.21.15.83:0/742698645 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1e04066f60 0x7f1e04069420 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c0ad000 tx=0x7f1e0c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:16.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.423+0000 7f1dfffff640 1 -- 172.21.15.83:0/742698645 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1e0801f020 con 0x7f1e1806dae0 2024-04-17T10:47:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.603+0000 7f1e1e1e9640 1 -- 172.21.15.83:0/742698645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e180b2af0 con 0x7f1e1806dae0 2024-04-17T10:47:16.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.603+0000 7f1dfffff640 1 -- 172.21.15.83:0/742698645 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f1e080172c0 con 0x7f1e1806dae0 2024-04-17T10:47:16.600 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:47:16.600 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5429,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6834/3936640219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3936640219},{"type":"v1","addr":"172.21.15.167:6834","nonce":3936640219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":5998,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4152882285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6833","nonce":4152882285},{"type":"v1","addr":"172.21.15.167:6835","nonce":4152882285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1562645860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1562645860},{"type":"v1","addr":"172.21.15.83:6837","nonce":1562645860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:46:36.578918+0000","modified":"2024-04-17T10:46:38.561701+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7575":{"gid":7575,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/1056175765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1056175765},{"type":"v1","addr":"172.21.15.83:6836","nonce":1056175765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-17T10:47:16.600 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 56 2024-04-17T10:47:16.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.607+0000 7f1e1e1e9640 1 -- 172.21.15.83:0/742698645 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1e04066f60 msgr2=0x7f1e04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:16.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.607+0000 7f1e1e1e9640 1 --2- 172.21.15.83:0/742698645 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1e04066f60 0x7f1e04069420 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c0ad000 tx=0x7f1e0c008040 comp rx=0 tx=0).stop 2024-04-17T10:47:16.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.607+0000 7f1e1e1e9640 1 -- 172.21.15.83:0/742698645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e1806dae0 msgr2=0x7f1e1814f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:16.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.607+0000 7f1e1e1e9640 1 --2- 172.21.15.83:0/742698645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e1806dae0 0x7f1e1814f390 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800b3e0 tx=0x7f1e0800b8a0 comp rx=0 tx=0).stop 2024-04-17T10:47:16.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.607+0000 7f1e1e1e9640 1 -- 172.21.15.83:0/742698645 shutdown_connections 2024-04-17T10:47:16.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.607+0000 7f1e1e1e9640 1 --2- 172.21.15.83:0/742698645 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1e04066f60 0x7f1e04069420 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:16.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.607+0000 7f1e1e1e9640 1 --2- 172.21.15.83:0/742698645 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e1815db50 0x7f1e1806c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:16.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.607+0000 7f1e1e1e9640 1 --2- 172.21.15.83:0/742698645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e180aedd0 0x7f1e1815ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:16.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.607+0000 7f1e1e1e9640 1 --2- 172.21.15.83:0/742698645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e1806dae0 0x7f1e1814f390 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:16.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.607+0000 7f1e1e1e9640 1 -- 172.21.15.83:0/742698645 >> 172.21.15.83:0/742698645 conn(0x7f1e1806abe0 msgr2=0x7f1e18070470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:16.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.607+0000 7f1e1e1e9640 1 -- 172.21.15.83:0/742698645 shutdown_connections 2024-04-17T10:47:16.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.607+0000 7f1e1e1e9640 1 -- 172.21.15.83:0/742698645 wait complete. 2024-04-17T10:47:16.617 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:47:16.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.743+0000 7f847562d640 1 Processor -- start 2024-04-17T10:47:16.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.743+0000 7f847562d640 1 -- start start 2024-04-17T10:47:16.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.747+0000 7f847562d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f84700ce980 0x7f84700b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:16.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.747+0000 7f847562d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f84700cee00 0x7f84700b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:16.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.747+0000 7f847562d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8470162a00 con 0x7f847015dfa0 2024-04-17T10:47:16.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.747+0000 7f847562d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8470161e80 con 0x7f84700ce980 2024-04-17T10:47:16.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.747+0000 7f847562d640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8470162160 con 0x7f84700cee00 2024-04-17T10:47:16.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.747+0000 7f846e7fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f84700ce980 0x7f84700b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:16.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.747+0000 7f846effd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f847015dfa0 0x7f84700b6c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35020/0 (socket says 172.21.15.83:35020) 2024-04-17T10:47:16.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.747+0000 7f846effd640 1 -- 172.21.15.83:0/469811480 learned_addr learned my addr 172.21.15.83:0/469811480 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:16.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.747+0000 7f846f7fe640 1 --2- 172.21.15.83:0/469811480 >> v2:172.21.15.167:3300/0 conn(0x7f84700cee00 0x7f84700b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:16.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.747+0000 7f846e7fc640 1 -- 172.21.15.83:0/469811480 >> v2:172.21.15.167:3300/0 conn(0x7f84700cee00 msgr2=0x7f84700b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:16.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.747+0000 7f846e7fc640 1 --2- 172.21.15.83:0/469811480 >> v2:172.21.15.167:3300/0 conn(0x7f84700cee00 0x7f84700b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:16.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.747+0000 7f846e7fc640 1 -- 172.21.15.83:0/469811480 >> v1:172.21.15.83:6789/0 conn(0x7f847015dfa0 legacy=0x7f84700b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:16.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.747+0000 7f846e7fc640 1 -- 172.21.15.83:0/469811480 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8470162440 con 0x7f84700ce980 2024-04-17T10:47:16.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.747+0000 7f846dffb640 1 -- 172.21.15.83:0/469811480 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1109491879 0 0) 0x7f8470162a00 con 0x7f847015dfa0 2024-04-17T10:47:16.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.747+0000 7f846f7fe640 1 --2- 172.21.15.83:0/469811480 >> v2:172.21.15.167:3300/0 conn(0x7f84700cee00 0x7f84700b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:16.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.747+0000 7f846e7fc640 1 --2- 172.21.15.83:0/469811480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84700ce980 0x7f84700b5430 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f8470067820 tx=0x7f846001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2760a637d6ff1e3b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:16.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.747+0000 7f846dffb640 1 -- 172.21.15.83:0/469811480 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84600b2070 con 0x7f84700ce980 2024-04-17T10:47:16.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.747+0000 7f846dffb640 1 -- 172.21.15.83:0/469811480 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84600a9070 con 0x7f84700ce980 2024-04-17T10:47:16.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.747+0000 7f846dffb640 1 -- 172.21.15.83:0/469811480 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84600ad880 con 0x7f84700ce980 2024-04-17T10:47:16.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.751+0000 7f847562d640 1 -- 172.21.15.83:0/469811480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84700ce980 msgr2=0x7f84700b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:16.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.751+0000 7f847562d640 1 --2- 172.21.15.83:0/469811480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84700ce980 0x7f84700b5430 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f8470067820 tx=0x7f846001c3c0 comp rx=0 tx=0).stop 2024-04-17T10:47:16.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.751+0000 7f847562d640 1 -- 172.21.15.83:0/469811480 shutdown_connections 2024-04-17T10:47:16.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.751+0000 7f847562d640 1 --2- 172.21.15.83:0/469811480 >> v2:172.21.15.167:3300/0 conn(0x7f84700cee00 0x7f84700b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:16.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.751+0000 7f847562d640 1 --2- 172.21.15.83:0/469811480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84700ce980 0x7f84700b5430 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:16.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.751+0000 7f847562d640 1 -- 172.21.15.83:0/469811480 >> 172.21.15.83:0/469811480 conn(0x7f84700d0600 msgr2=0x7f84700d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:16.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.751+0000 7f847562d640 1 -- 172.21.15.83:0/469811480 shutdown_connections 2024-04-17T10:47:16.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.751+0000 7f847562d640 1 -- 172.21.15.83:0/469811480 wait complete. 2024-04-17T10:47:16.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.751+0000 7f847562d640 1 Processor -- start 2024-04-17T10:47:16.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.751+0000 7f847562d640 1 -- start start 2024-04-17T10:47:16.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.751+0000 7f847562d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84700ce980 0x7f84700cd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:16.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.751+0000 7f847562d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84700cee00 0x7f847014f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:16.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.751+0000 7f847562d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f847015dfa0 0x7f84700c5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:16.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.751+0000 7f847562d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84700d26c0 con 0x7f84700ce980 2024-04-17T10:47:16.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.751+0000 7f847562d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84700d2540 con 0x7f84700cee00 2024-04-17T10:47:16.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.751+0000 7f847562d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84700d2840 con 0x7f847015dfa0 2024-04-17T10:47:16.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.751+0000 7f846effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84700ce980 0x7f84700cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:16.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.751+0000 7f846effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84700ce980 0x7f84700cd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41886/0 (socket says 172.21.15.83:41886) 2024-04-17T10:47:16.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.751+0000 7f846effd640 1 -- 172.21.15.83:0/3962926098 learned_addr learned my addr 172.21.15.83:0/3962926098 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:16.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.751+0000 7f846e7fc640 1 --2- 172.21.15.83:0/3962926098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84700cee00 0x7f847014f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:16.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.751+0000 7f846effd640 1 -- 172.21.15.83:0/3962926098 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f847015dfa0 msgr2=0x7f84700c5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:16.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.751+0000 7f846effd640 1 --2- 172.21.15.83:0/3962926098 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f847015dfa0 0x7f84700c5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:16.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.751+0000 7f846effd640 1 -- 172.21.15.83:0/3962926098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84700cee00 msgr2=0x7f847014f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:16.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.751+0000 7f846effd640 1 --2- 172.21.15.83:0/3962926098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84700cee00 0x7f847014f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:16.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.751+0000 7f846effd640 1 -- 172.21.15.83:0/3962926098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847014b460 con 0x7f84700ce980 2024-04-17T10:47:16.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.751+0000 7f846e7fc640 1 --2- 172.21.15.83:0/3962926098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84700cee00 0x7f847014f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:16.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.751+0000 7f846effd640 1 --2- 172.21.15.83:0/3962926098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84700ce980 0x7f84700cd120 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f84640011f0 tx=0x7f846400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:16.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.751+0000 7f84577fe640 1 -- 172.21.15.83:0/3962926098 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8464018070 con 0x7f84700ce980 2024-04-17T10:47:16.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.751+0000 7f84577fe640 1 -- 172.21.15.83:0/3962926098 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84640074e0 con 0x7f84700ce980 2024-04-17T10:47:16.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.751+0000 7f847562d640 1 -- 172.21.15.83:0/3962926098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f847014b740 con 0x7f84700ce980 2024-04-17T10:47:16.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.751+0000 7f84577fe640 1 -- 172.21.15.83:0/3962926098 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8464004a10 con 0x7f84700ce980 2024-04-17T10:47:16.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.751+0000 7f847562d640 1 -- 172.21.15.83:0/3962926098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84700c6f10 con 0x7f84700ce980 2024-04-17T10:47:16.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.755+0000 7f847562d640 1 -- 172.21.15.83:0/3962926098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8470074f90 con 0x7f84700ce980 2024-04-17T10:47:16.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.755+0000 7f84577fe640 1 -- 172.21.15.83:0/3962926098 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f846401b070 con 0x7f84700ce980 2024-04-17T10:47:16.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.755+0000 7f84577fe640 1 --2- 172.21.15.83:0/3962926098 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f845c066f60 0x7f845c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:16.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.755+0000 7f846e7fc640 1 --2- 172.21.15.83:0/3962926098 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f845c066f60 0x7f845c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:16.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.755+0000 7f84577fe640 1 -- 172.21.15.83:0/3962926098 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f8464084320 con 0x7f84700ce980 2024-04-17T10:47:16.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.755+0000 7f846e7fc640 1 --2- 172.21.15.83:0/3962926098 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f845c066f60 0x7f845c069420 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f84600af000 tx=0x7f8460008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:16.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.759+0000 7f84577fe640 1 -- 172.21.15.83:0/3962926098 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f84640553a0 con 0x7f84700ce980 2024-04-17T10:47:16.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.951+0000 7f847562d640 1 -- 172.21.15.83:0/3962926098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f847014da50 con 0x7f84700ce980 2024-04-17T10:47:16.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.951+0000 7f84577fe640 1 -- 172.21.15.83:0/3962926098 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f8464054ca0 con 0x7f84700ce980 2024-04-17T10:47:16.947 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:47:16.947 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5429,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6834/3936640219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3936640219},{"type":"v1","addr":"172.21.15.167:6834","nonce":3936640219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":5998,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4152882285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6833","nonce":4152882285},{"type":"v1","addr":"172.21.15.167:6835","nonce":4152882285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1562645860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1562645860},{"type":"v1","addr":"172.21.15.83:6837","nonce":1562645860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:46:36.578918+0000","modified":"2024-04-17T10:46:38.561701+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7575":{"gid":7575,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/1056175765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1056175765},{"type":"v1","addr":"172.21.15.83:6836","nonce":1056175765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-17T10:47:16.947 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 56 2024-04-17T10:47:16.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.951+0000 7f847562d640 1 -- 172.21.15.83:0/3962926098 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f845c066f60 msgr2=0x7f845c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:16.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.951+0000 7f847562d640 1 --2- 172.21.15.83:0/3962926098 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f845c066f60 0x7f845c069420 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f84600af000 tx=0x7f8460008040 comp rx=0 tx=0).stop 2024-04-17T10:47:16.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.951+0000 7f847562d640 1 -- 172.21.15.83:0/3962926098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84700ce980 msgr2=0x7f84700cd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:16.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.955+0000 7f847562d640 1 --2- 172.21.15.83:0/3962926098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84700ce980 0x7f84700cd120 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f84640011f0 tx=0x7f846400c9b0 comp rx=0 tx=0).stop 2024-04-17T10:47:16.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.955+0000 7f847562d640 1 -- 172.21.15.83:0/3962926098 shutdown_connections 2024-04-17T10:47:16.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.955+0000 7f847562d640 1 --2- 172.21.15.83:0/3962926098 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f845c066f60 0x7f845c069420 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:16.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.955+0000 7f847562d640 1 --2- 172.21.15.83:0/3962926098 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f847015dfa0 0x7f84700c5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:16.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.955+0000 7f847562d640 1 --2- 172.21.15.83:0/3962926098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84700cee00 0x7f847014f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:16.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.955+0000 7f847562d640 1 --2- 172.21.15.83:0/3962926098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84700ce980 0x7f84700cd120 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:16.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.955+0000 7f847562d640 1 -- 172.21.15.83:0/3962926098 >> 172.21.15.83:0/3962926098 conn(0x7f84700d0600 msgr2=0x7f8470158020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:16.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.955+0000 7f847562d640 1 -- 172.21.15.83:0/3962926098 shutdown_connections 2024-04-17T10:47:16.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:16.955+0000 7f847562d640 1 -- 172.21.15.83:0/3962926098 wait complete. 2024-04-17T10:47:16.965 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:47:17.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.079+0000 7fa754a83640 1 Processor -- start 2024-04-17T10:47:17.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.079+0000 7fa754a83640 1 -- start start 2024-04-17T10:47:17.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.079+0000 7fa754a83640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa7500cf060 0x7fa7500b4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:17.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.079+0000 7fa754a83640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa7500b3990 0x7fa750070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:17.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.079+0000 7fa754a83640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7500b2bd0 con 0x7fa7500b45a0 2024-04-17T10:47:17.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.079+0000 7fa754a83640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7500b2980 con 0x7fa7500b3990 2024-04-17T10:47:17.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.079+0000 7fa754a83640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7500cf440 con 0x7fa7500cf060 2024-04-17T10:47:17.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.079+0000 7fa747fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa7500b3990 0x7fa750070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:17.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.079+0000 7fa74ed9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa7500b45a0 0x7fa75015d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35034/0 (socket says 172.21.15.83:35034) 2024-04-17T10:47:17.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.079+0000 7fa74ed9d640 1 -- 172.21.15.83:0/1204051460 learned_addr learned my addr 172.21.15.83:0/1204051460 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:17.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.079+0000 7fa74dd9b640 1 -- 172.21.15.83:0/1204051460 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1859709548 0 0) 0x7fa7500b2bd0 con 0x7fa7500b45a0 2024-04-17T10:47:17.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.079+0000 7fa74e59c640 1 --2- 172.21.15.83:0/1204051460 >> v2:172.21.15.167:3300/0 conn(0x7fa7500cf060 0x7fa7500b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:17.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.079+0000 7fa74dd9b640 1 -- 172.21.15.83:0/1204051460 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa73c003530 con 0x7fa7500b45a0 2024-04-17T10:47:17.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.079+0000 7fa747fff640 1 -- 172.21.15.83:0/1204051460 >> v2:172.21.15.167:3300/0 conn(0x7fa7500cf060 msgr2=0x7fa7500b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:17.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.079+0000 7fa747fff640 1 --2- 172.21.15.83:0/1204051460 >> v2:172.21.15.167:3300/0 conn(0x7fa7500cf060 0x7fa7500b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:17.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.079+0000 7fa747fff640 1 -- 172.21.15.83:0/1204051460 >> v1:172.21.15.83:6789/0 conn(0x7fa7500b45a0 legacy=0x7fa75015d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:17.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.079+0000 7fa74dd9b640 1 -- 172.21.15.83:0/1204051460 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 46941750 0 0) 0x7fa73c003530 con 0x7fa7500b45a0 2024-04-17T10:47:17.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.079+0000 7fa747fff640 1 -- 172.21.15.83:0/1204051460 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7500cf5c0 con 0x7fa7500b3990 2024-04-17T10:47:17.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.079+0000 7fa747fff640 1 --2- 172.21.15.83:0/1204051460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7500b3990 0x7fa750070470 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fa75006e6c0 tx=0x7fa73801c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4956c32c98f6473 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:17.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa74dd9b640 1 -- 172.21.15.83:0/1204051460 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7380b0070 con 0x7fa7500b3990 2024-04-17T10:47:17.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa74dd9b640 1 -- 172.21.15.83:0/1204051460 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7380ab400 con 0x7fa7500b3990 2024-04-17T10:47:17.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa74dd9b640 1 -- 172.21.15.83:0/1204051460 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa748001a00 con 0x7fa7500b3990 2024-04-17T10:47:17.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa754a83640 1 -- 172.21.15.83:0/1204051460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7500b3990 msgr2=0x7fa750070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:17.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa754a83640 1 --2- 172.21.15.83:0/1204051460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7500b3990 0x7fa750070470 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fa75006e6c0 tx=0x7fa73801c370 comp rx=0 tx=0).stop 2024-04-17T10:47:17.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa754a83640 1 -- 172.21.15.83:0/1204051460 shutdown_connections 2024-04-17T10:47:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa754a83640 1 --2- 172.21.15.83:0/1204051460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7500b3990 0x7fa750070470 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa754a83640 1 --2- 172.21.15.83:0/1204051460 >> v2:172.21.15.167:3300/0 conn(0x7fa7500cf060 0x7fa7500b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa754a83640 1 -- 172.21.15.83:0/1204051460 >> 172.21.15.83:0/1204051460 conn(0x7fa75006abe0 msgr2=0x7fa75006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa754a83640 1 -- 172.21.15.83:0/1204051460 shutdown_connections 2024-04-17T10:47:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa754a83640 1 -- 172.21.15.83:0/1204051460 wait complete. 2024-04-17T10:47:17.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa754a83640 1 Processor -- start 2024-04-17T10:47:17.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa754a83640 1 -- start start 2024-04-17T10:47:17.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa754a83640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7500b3990 0x7fa7501554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:17.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa754a83640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7500b45a0 0x7fa7501508b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa754a83640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7500cf060 0x7fa7500b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa754a83640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75015b310 con 0x7fa7500b45a0 2024-04-17T10:47:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa754a83640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7500b3470 con 0x7fa7500b3990 2024-04-17T10:47:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa754a83640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa750164d80 con 0x7fa7500cf060 2024-04-17T10:47:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa747fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7500b45a0 0x7fa7501508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa747fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7500b45a0 0x7fa7501508b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41902/0 (socket says 172.21.15.83:41902) 2024-04-17T10:47:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa747fff640 1 -- 172.21.15.83:0/3949368950 learned_addr learned my addr 172.21.15.83:0/3949368950 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa74e59c640 1 --2- 172.21.15.83:0/3949368950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7500b3990 0x7fa7501554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa747fff640 1 -- 172.21.15.83:0/3949368950 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7500cf060 msgr2=0x7fa7500b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa747fff640 1 --2- 172.21.15.83:0/3949368950 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7500cf060 0x7fa7500b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa747fff640 1 -- 172.21.15.83:0/3949368950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7500b3990 msgr2=0x7fa7501554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa747fff640 1 --2- 172.21.15.83:0/3949368950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7500b3990 0x7fa7501554b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa747fff640 1 -- 172.21.15.83:0/3949368950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa750155b40 con 0x7fa7500b45a0 2024-04-17T10:47:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa747fff640 1 --2- 172.21.15.83:0/3949368950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7500b45a0 0x7fa7501508b0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fa7380ae040 tx=0x7fa738019110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa746ffd640 1 -- 172.21.15.83:0/3949368950 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7380b8020 con 0x7fa7500b45a0 2024-04-17T10:47:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa746ffd640 1 -- 172.21.15.83:0/3949368950 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7380b4410 con 0x7fa7500b45a0 2024-04-17T10:47:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa754a83640 1 -- 172.21.15.83:0/3949368950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa750155e20 con 0x7fa7500b45a0 2024-04-17T10:47:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa746ffd640 1 -- 172.21.15.83:0/3949368950 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7380b0040 con 0x7fa7500b45a0 2024-04-17T10:47:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.083+0000 7fa754a83640 1 -- 172.21.15.83:0/3949368950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7501563e0 con 0x7fa7500b45a0 2024-04-17T10:47:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.087+0000 7fa754a83640 1 -- 172.21.15.83:0/3949368950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa750074f90 con 0x7fa7500b45a0 2024-04-17T10:47:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.087+0000 7fa746ffd640 1 -- 172.21.15.83:0/3949368950 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa7380bb020 con 0x7fa7500b45a0 2024-04-17T10:47:17.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.087+0000 7fa746ffd640 1 --2- 172.21.15.83:0/3949368950 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa730066f60 0x7fa730069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:17.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.087+0000 7fa746ffd640 1 -- 172.21.15.83:0/3949368950 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fa73811ebf0 con 0x7fa7500b45a0 2024-04-17T10:47:17.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.087+0000 7fa74e59c640 1 --2- 172.21.15.83:0/3949368950 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa730066f60 0x7fa730069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:17.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.087+0000 7fa74e59c640 1 --2- 172.21.15.83:0/3949368950 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa730066f60 0x7fa730069420 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa7400096f0 tx=0x7fa740008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:17.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.091+0000 7fa746ffd640 1 -- 172.21.15.83:0/3949368950 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa7380c2030 con 0x7fa7500b45a0 2024-04-17T10:47:17.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.239+0000 7fa754a83640 1 -- 172.21.15.83:0/3949368950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa750165390 con 0x7fa7500b45a0 2024-04-17T10:47:17.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.243+0000 7fa746ffd640 1 -- 172.21.15.83:0/3949368950 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+16638 (secure 0 0 0) 0x7fa738015f90 con 0x7fa7500b45a0 2024-04-17T10:47:17.239 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:47:17.239 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":91,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:46:55.414653+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:46:34.779174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-17T10:46:35.563972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.10","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:47:17.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.247+0000 7fa754a83640 1 -- 172.21.15.83:0/3949368950 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa730066f60 msgr2=0x7fa730069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:17.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.247+0000 7fa754a83640 1 --2- 172.21.15.83:0/3949368950 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa730066f60 0x7fa730069420 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa7400096f0 tx=0x7fa740008040 comp rx=0 tx=0).stop 2024-04-17T10:47:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.247+0000 7fa754a83640 1 -- 172.21.15.83:0/3949368950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7500b45a0 msgr2=0x7fa7501508b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.247+0000 7fa754a83640 1 --2- 172.21.15.83:0/3949368950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7500b45a0 0x7fa7501508b0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fa7380ae040 tx=0x7fa738019110 comp rx=0 tx=0).stop 2024-04-17T10:47:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.247+0000 7fa754a83640 1 -- 172.21.15.83:0/3949368950 shutdown_connections 2024-04-17T10:47:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.247+0000 7fa754a83640 1 --2- 172.21.15.83:0/3949368950 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa730066f60 0x7fa730069420 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.247+0000 7fa754a83640 1 --2- 172.21.15.83:0/3949368950 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7500cf060 0x7fa7500b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.247+0000 7fa754a83640 1 --2- 172.21.15.83:0/3949368950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7500b45a0 0x7fa7501508b0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.247+0000 7fa754a83640 1 --2- 172.21.15.83:0/3949368950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7500b3990 0x7fa7501554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.247+0000 7fa754a83640 1 -- 172.21.15.83:0/3949368950 >> 172.21.15.83:0/3949368950 conn(0x7fa75006abe0 msgr2=0x7fa75006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.247+0000 7fa754a83640 1 -- 172.21.15.83:0/3949368950 shutdown_connections 2024-04-17T10:47:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.247+0000 7fa754a83640 1 -- 172.21.15.83:0/3949368950 wait complete. 2024-04-17T10:47:17.258 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-17T10:47:17.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.379+0000 7f7e9fe5d640 1 Processor -- start 2024-04-17T10:47:17.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.379+0000 7f7e9fe5d640 1 -- start start 2024-04-17T10:47:17.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.379+0000 7f7e9fe5d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7e9815ed70 0x7f7e9806bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:17.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.379+0000 7f7e9fe5d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e98161d20 0x7f7e981621d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:17.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.379+0000 7f7e9fe5d640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e980d2430 con 0x7f7e98161940 2024-04-17T10:47:17.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.379+0000 7f7e9fe5d640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e980d2710 con 0x7f7e9815ed70 2024-04-17T10:47:17.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.379+0000 7f7e9fe5d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e980ced00 con 0x7f7e98161d20 2024-04-17T10:47:17.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.379+0000 7f7e9e3fa640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e98161d20 0x7f7e981621d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:17.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.383+0000 7f7e9e3fa640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e98161d20 0x7f7e981621d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:33676/0 (socket says 172.21.15.83:33676) 2024-04-17T10:47:17.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.383+0000 7f7e9e3fa640 1 -- 172.21.15.83:0/1959100209 learned_addr learned my addr 172.21.15.83:0/1959100209 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:17.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.383+0000 7f7e9d3f8640 1 --2- 172.21.15.83:0/1959100209 >> v2:172.21.15.167:3300/0 conn(0x7f7e9815ed70 0x7f7e9806bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:17.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.383+0000 7f7e9e3fa640 1 -- 172.21.15.83:0/1959100209 >> v2:172.21.15.167:3300/0 conn(0x7f7e9815ed70 msgr2=0x7f7e9806bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:17.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.383+0000 7f7e9e3fa640 1 --2- 172.21.15.83:0/1959100209 >> v2:172.21.15.167:3300/0 conn(0x7f7e9815ed70 0x7f7e9806bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:17.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.383+0000 7f7e9e3fa640 1 -- 172.21.15.83:0/1959100209 >> v1:172.21.15.167:6789/0 conn(0x7f7e98161940 legacy=0x7f7e980cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:17.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.383+0000 7f7e9e3fa640 1 -- 172.21.15.83:0/1959100209 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e980cefe0 con 0x7f7e98161d20 2024-04-17T10:47:17.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.383+0000 7f7e9e3fa640 1 --2- 172.21.15.83:0/1959100209 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e98161d20 0x7f7e981621d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7e90009e50 tx=0x7f7e9001c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d561c484b5d648d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:17.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.383+0000 7f7e9cbf7640 1 -- 172.21.15.83:0/1959100209 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e900a55d0 con 0x7f7e98161d20 2024-04-17T10:47:17.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.383+0000 7f7e9cbf7640 1 -- 172.21.15.83:0/1959100209 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e900a5770 con 0x7f7e98161d20 2024-04-17T10:47:17.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.383+0000 7f7e9cbf7640 1 -- 172.21.15.83:0/1959100209 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e900a5b40 con 0x7f7e98161d20 2024-04-17T10:47:17.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.383+0000 7f7e9fe5d640 1 -- 172.21.15.83:0/1959100209 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e98161d20 msgr2=0x7f7e981621d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:17.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.383+0000 7f7e9fe5d640 1 --2- 172.21.15.83:0/1959100209 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e98161d20 0x7f7e981621d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7e90009e50 tx=0x7f7e9001c910 comp rx=0 tx=0).stop 2024-04-17T10:47:17.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.383+0000 7f7e9fe5d640 1 -- 172.21.15.83:0/1959100209 shutdown_connections 2024-04-17T10:47:17.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.383+0000 7f7e9fe5d640 1 --2- 172.21.15.83:0/1959100209 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e98161d20 0x7f7e981621d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:17.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.383+0000 7f7e9fe5d640 1 --2- 172.21.15.83:0/1959100209 >> v2:172.21.15.167:3300/0 conn(0x7f7e9815ed70 0x7f7e9806bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:17.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.383+0000 7f7e9fe5d640 1 -- 172.21.15.83:0/1959100209 >> 172.21.15.83:0/1959100209 conn(0x7f7e980d17e0 msgr2=0x7f7e980d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:17.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.387+0000 7f7e9fe5d640 1 -- 172.21.15.83:0/1959100209 shutdown_connections 2024-04-17T10:47:17.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.387+0000 7f7e9fe5d640 1 -- 172.21.15.83:0/1959100209 wait complete. 2024-04-17T10:47:17.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.387+0000 7f7e9fe5d640 1 Processor -- start 2024-04-17T10:47:17.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.387+0000 7f7e9fe5d640 1 -- start start 2024-04-17T10:47:17.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.387+0000 7f7e9fe5d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e9815ed70 0x7f7e980af190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:17.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.387+0000 7f7e9fe5d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e98161940 0x7f7e980b6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:17.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.387+0000 7f7e9fe5d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e98161d20 0x7f7e98154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:17.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.387+0000 7f7e9fe5d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9806d130 con 0x7f7e9815ed70 2024-04-17T10:47:17.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.387+0000 7f7e9fe5d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9806cfb0 con 0x7f7e98161d20 2024-04-17T10:47:17.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.387+0000 7f7e9fe5d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9816aa20 con 0x7f7e98161940 2024-04-17T10:47:17.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.387+0000 7f7e9dbf9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e9815ed70 0x7f7e980af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:17.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.387+0000 7f7e9dbf9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e9815ed70 0x7f7e980af190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41906/0 (socket says 172.21.15.83:41906) 2024-04-17T10:47:17.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.387+0000 7f7e9dbf9640 1 -- 172.21.15.83:0/263262795 learned_addr learned my addr 172.21.15.83:0/263262795 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:17.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.387+0000 7f7e9e3fa640 1 --2- 172.21.15.83:0/263262795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e98161d20 0x7f7e98154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:17.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.387+0000 7f7e9d3f8640 1 --2- 172.21.15.83:0/263262795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e98161940 0x7f7e980b6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:17.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.387+0000 7f7e9dbf9640 1 -- 172.21.15.83:0/263262795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e98161940 msgr2=0x7f7e980b6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:17.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.387+0000 7f7e9dbf9640 1 --2- 172.21.15.83:0/263262795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e98161940 0x7f7e980b6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:17.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.387+0000 7f7e9dbf9640 1 -- 172.21.15.83:0/263262795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e98161d20 msgr2=0x7f7e98154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:17.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.387+0000 7f7e9dbf9640 1 --2- 172.21.15.83:0/263262795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e98161d20 0x7f7e98154450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:17.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.387+0000 7f7e9dbf9640 1 -- 172.21.15.83:0/263262795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e98159280 con 0x7f7e9815ed70 2024-04-17T10:47:17.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.387+0000 7f7e9dbf9640 1 --2- 172.21.15.83:0/263262795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e9815ed70 0x7f7e980af190 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400c3a0 tx=0x7f7e9400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:17.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.387+0000 7f7e8a7fc640 1 -- 172.21.15.83:0/263262795 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e940052d0 con 0x7f7e9815ed70 2024-04-17T10:47:17.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.387+0000 7f7e8a7fc640 1 -- 172.21.15.83:0/263262795 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e94005470 con 0x7f7e9815ed70 2024-04-17T10:47:17.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.387+0000 7f7e8a7fc640 1 -- 172.21.15.83:0/263262795 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e94007260 con 0x7f7e9815ed70 2024-04-17T10:47:17.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.387+0000 7f7e9fe5d640 1 -- 172.21.15.83:0/263262795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e98159560 con 0x7f7e9815ed70 2024-04-17T10:47:17.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.387+0000 7f7e9fe5d640 1 -- 172.21.15.83:0/263262795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e98159b20 con 0x7f7e9815ed70 2024-04-17T10:47:17.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.391+0000 7f7e9fe5d640 1 -- 172.21.15.83:0/263262795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e98074f90 con 0x7f7e9815ed70 2024-04-17T10:47:17.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.391+0000 7f7e8a7fc640 1 -- 172.21.15.83:0/263262795 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7e94007400 con 0x7f7e9815ed70 2024-04-17T10:47:17.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.391+0000 7f7e8a7fc640 1 --2- 172.21.15.83:0/263262795 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7e7c066f60 0x7f7e7c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:17.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.391+0000 7f7e8a7fc640 1 -- 172.21.15.83:0/263262795 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f7e94084450 con 0x7f7e9815ed70 2024-04-17T10:47:17.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.391+0000 7f7e9d3f8640 1 --2- 172.21.15.83:0/263262795 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7e7c066f60 0x7f7e7c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:17.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.391+0000 7f7e9d3f8640 1 --2- 172.21.15.83:0/263262795 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7e7c066f60 0x7f7e7c069420 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7e980d15a0 tx=0x7f7e8c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:17.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.395+0000 7f7e8a7fc640 1 -- 172.21.15.83:0/263262795 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7e940554d0 con 0x7f7e9815ed70 2024-04-17T10:47:17.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.587+0000 7f7e9fe5d640 1 -- 172.21.15.83:0/263262795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7e98072740 con 0x7f7e9815ed70 2024-04-17T10:47:17.585 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-04-17T10:47:17.591+0000 7f35c3b81640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T10:47:17.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.591+0000 7f7e8a7fc640 1 -- 172.21.15.83:0/263262795 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v57) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7e94054dd0 con 0x7f7e9815ed70 2024-04-17T10:47:17.588 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-17T10:47:17.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.595+0000 7f7e9fe5d640 1 -- 172.21.15.83:0/263262795 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7e7c066f60 msgr2=0x7f7e7c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:17.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.595+0000 7f7e9fe5d640 1 --2- 172.21.15.83:0/263262795 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7e7c066f60 0x7f7e7c069420 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7e980d15a0 tx=0x7f7e8c008040 comp rx=0 tx=0).stop 2024-04-17T10:47:17.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.595+0000 7f7e9fe5d640 1 -- 172.21.15.83:0/263262795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e9815ed70 msgr2=0x7f7e980af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:17.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.595+0000 7f7e9fe5d640 1 --2- 172.21.15.83:0/263262795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e9815ed70 0x7f7e980af190 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400c3a0 tx=0x7f7e9400c860 comp rx=0 tx=0).stop 2024-04-17T10:47:17.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.595+0000 7f7e9fe5d640 1 -- 172.21.15.83:0/263262795 shutdown_connections 2024-04-17T10:47:17.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.595+0000 7f7e9fe5d640 1 --2- 172.21.15.83:0/263262795 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7e7c066f60 0x7f7e7c069420 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:17.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.595+0000 7f7e9fe5d640 1 --2- 172.21.15.83:0/263262795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e98161d20 0x7f7e98154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:17.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.595+0000 7f7e9fe5d640 1 --2- 172.21.15.83:0/263262795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e98161940 0x7f7e980b6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:17.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.595+0000 7f7e9fe5d640 1 --2- 172.21.15.83:0/263262795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e9815ed70 0x7f7e980af190 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:17.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.595+0000 7f7e9fe5d640 1 -- 172.21.15.83:0/263262795 >> 172.21.15.83:0/263262795 conn(0x7f7e980d17e0 msgr2=0x7f7e980d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:17.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.595+0000 7f7e9fe5d640 1 -- 172.21.15.83:0/263262795 shutdown_connections 2024-04-17T10:47:17.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.595+0000 7f7e9fe5d640 1 -- 172.21.15.83:0/263262795 wait complete. 2024-04-17T10:47:17.610 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-17T10:47:17.666 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-04-17T10:47:17.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.731+0000 7f563fe7c640 1 Processor -- start 2024-04-17T10:47:17.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.731+0000 7f563fe7c640 1 -- start start 2024-04-17T10:47:17.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.731+0000 7f563fe7c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f563815bc10 0x7f56380d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:17.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.731+0000 7f563fe7c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f563806e500 0x7f563806e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:17.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.731+0000 7f563fe7c640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56380ced00 con 0x7f56380d0490 2024-04-17T10:47:17.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.731+0000 7f563fe7c640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56380cefe0 con 0x7f563806e500 2024-04-17T10:47:17.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.731+0000 7f563fe7c640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56380cc690 con 0x7f563815bc10 2024-04-17T10:47:17.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.731+0000 7f563e419640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f563806e500 0x7f563806e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:17.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.731+0000 7f563e419640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f563806e500 0x7f563806e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41926/0 (socket says 172.21.15.83:41926) 2024-04-17T10:47:17.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.731+0000 7f563e419640 1 -- 172.21.15.83:0/33865372 learned_addr learned my addr 172.21.15.83:0/33865372 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:17.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.731+0000 7f563dc18640 1 --2- 172.21.15.83:0/33865372 >> v2:172.21.15.167:3300/0 conn(0x7f563815bc10 0x7f56380d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:17.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.731+0000 7f563e419640 1 -- 172.21.15.83:0/33865372 >> v2:172.21.15.167:3300/0 conn(0x7f563815bc10 msgr2=0x7f56380d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:17.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.731+0000 7f563e419640 1 --2- 172.21.15.83:0/33865372 >> v2:172.21.15.167:3300/0 conn(0x7f563815bc10 0x7f56380d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:17.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.731+0000 7f563e419640 1 -- 172.21.15.83:0/33865372 >> v1:172.21.15.167:6789/0 conn(0x7f56380d0490 legacy=0x7f56380cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:17.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.731+0000 7f563e419640 1 -- 172.21.15.83:0/33865372 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56380cc970 con 0x7f563806e500 2024-04-17T10:47:17.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563e419640 1 --2- 172.21.15.83:0/33865372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f563806e500 0x7f563806e950 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f56380678a0 tx=0x7f562801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4635e5fde0468264 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:17.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563cc16640 1 -- 172.21.15.83:0/33865372 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56280b0070 con 0x7f563806e500 2024-04-17T10:47:17.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563cc16640 1 -- 172.21.15.83:0/33865372 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56280ab600 con 0x7f563806e500 2024-04-17T10:47:17.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563cc16640 1 -- 172.21.15.83:0/33865372 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56280ab9d0 con 0x7f563806e500 2024-04-17T10:47:17.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563fe7c640 1 -- 172.21.15.83:0/33865372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f563806e500 msgr2=0x7f563806e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:17.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563fe7c640 1 --2- 172.21.15.83:0/33865372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f563806e500 0x7f563806e950 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f56380678a0 tx=0x7f562801c450 comp rx=0 tx=0).stop 2024-04-17T10:47:17.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563fe7c640 1 -- 172.21.15.83:0/33865372 shutdown_connections 2024-04-17T10:47:17.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563fe7c640 1 --2- 172.21.15.83:0/33865372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f563806e500 0x7f563806e950 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:17.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563fe7c640 1 --2- 172.21.15.83:0/33865372 >> v2:172.21.15.167:3300/0 conn(0x7f563815bc10 0x7f56380d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:17.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563fe7c640 1 -- 172.21.15.83:0/33865372 >> 172.21.15.83:0/33865372 conn(0x7f56380d17e0 msgr2=0x7f56380d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:17.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563fe7c640 1 -- 172.21.15.83:0/33865372 shutdown_connections 2024-04-17T10:47:17.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563fe7c640 1 -- 172.21.15.83:0/33865372 wait complete. 2024-04-17T10:47:17.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563fe7c640 1 Processor -- start 2024-04-17T10:47:17.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563fe7c640 1 -- start start 2024-04-17T10:47:17.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563fe7c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f563806e500 0x7f56380c5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:17.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563fe7c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56380d0490 0x7f5638153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:17.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563fe7c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f563815bc10 0x7f5638151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:17.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563fe7c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5638168b10 con 0x7f563806e500 2024-04-17T10:47:17.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563fe7c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5638168990 con 0x7f563815bc10 2024-04-17T10:47:17.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563fe7c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5638168c90 con 0x7f56380d0490 2024-04-17T10:47:17.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563dc18640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f563806e500 0x7f56380c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:17.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563dc18640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f563806e500 0x7f56380c5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41938/0 (socket says 172.21.15.83:41938) 2024-04-17T10:47:17.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563dc18640 1 -- 172.21.15.83:0/790815562 learned_addr learned my addr 172.21.15.83:0/790815562 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:17.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563d417640 1 --2- 172.21.15.83:0/790815562 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56380d0490 0x7f5638153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:17.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563e419640 1 --2- 172.21.15.83:0/790815562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f563815bc10 0x7f5638151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:17.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563dc18640 1 -- 172.21.15.83:0/790815562 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56380d0490 msgr2=0x7f5638153ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:17.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563dc18640 1 --2- 172.21.15.83:0/790815562 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56380d0490 0x7f5638153ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:17.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563dc18640 1 -- 172.21.15.83:0/790815562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f563815bc10 msgr2=0x7f5638151870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:17.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563dc18640 1 --2- 172.21.15.83:0/790815562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f563815bc10 0x7f5638151870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:17.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563dc18640 1 -- 172.21.15.83:0/790815562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5638151db0 con 0x7f563806e500 2024-04-17T10:47:17.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563d417640 1 --2- 172.21.15.83:0/790815562 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56380d0490 0x7f5638153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:17.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563dc18640 1 --2- 172.21.15.83:0/790815562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f563806e500 0x7f56380c5240 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f5634007f20 tx=0x7f563400e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:17.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f562e7fc640 1 -- 172.21.15.83:0/790815562 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5634014620 con 0x7f563806e500 2024-04-17T10:47:17.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f562e7fc640 1 -- 172.21.15.83:0/790815562 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5634019070 con 0x7f563806e500 2024-04-17T10:47:17.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.735+0000 7f563fe7c640 1 -- 172.21.15.83:0/790815562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5638152060 con 0x7f563806e500 2024-04-17T10:47:17.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.739+0000 7f562e7fc640 1 -- 172.21.15.83:0/790815562 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5634014a00 con 0x7f563806e500 2024-04-17T10:47:17.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.739+0000 7f563fe7c640 1 -- 172.21.15.83:0/790815562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56380b2690 con 0x7f563806e500 2024-04-17T10:47:17.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.739+0000 7f563fe7c640 1 -- 172.21.15.83:0/790815562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5638074f90 con 0x7f563806e500 2024-04-17T10:47:17.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.739+0000 7f562e7fc640 1 -- 172.21.15.83:0/790815562 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5634014ba0 con 0x7f563806e500 2024-04-17T10:47:17.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.739+0000 7f562e7fc640 1 --2- 172.21.15.83:0/790815562 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5624066f60 0x7f5624069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:17.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.739+0000 7f562e7fc640 1 -- 172.21.15.83:0/790815562 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f56340859f0 con 0x7f563806e500 2024-04-17T10:47:17.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.739+0000 7f563d417640 1 --2- 172.21.15.83:0/790815562 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5624066f60 0x7f5624069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:17.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.739+0000 7f563d417640 1 --2- 172.21.15.83:0/790815562 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5624066f60 0x7f5624069420 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f56380cdac0 tx=0x7f5630009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:17.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.743+0000 7f562e7fc640 1 -- 172.21.15.83:0/790815562 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5634056990 con 0x7f563806e500 2024-04-17T10:47:17.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:17.931+0000 7f563fe7c640 1 -- 172.21.15.83:0/790815562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f56380b7760 con 0x7f563806e500 2024-04-17T10:47:18.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.603+0000 7f562e7fc640 1 -- 172.21.15.83:0/790815562 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v58) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5634056290 con 0x7f563806e500 2024-04-17T10:47:18.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.607+0000 7f563fe7c640 1 -- 172.21.15.83:0/790815562 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5624066f60 msgr2=0x7f5624069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:18.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.607+0000 7f563fe7c640 1 --2- 172.21.15.83:0/790815562 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5624066f60 0x7f5624069420 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f56380cdac0 tx=0x7f5630009040 comp rx=0 tx=0).stop 2024-04-17T10:47:18.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.607+0000 7f563fe7c640 1 -- 172.21.15.83:0/790815562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f563806e500 msgr2=0x7f56380c5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:18.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.607+0000 7f563fe7c640 1 --2- 172.21.15.83:0/790815562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f563806e500 0x7f56380c5240 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f5634007f20 tx=0x7f563400e570 comp rx=0 tx=0).stop 2024-04-17T10:47:18.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.607+0000 7f563fe7c640 1 -- 172.21.15.83:0/790815562 shutdown_connections 2024-04-17T10:47:18.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.607+0000 7f563fe7c640 1 --2- 172.21.15.83:0/790815562 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5624066f60 0x7f5624069420 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:18.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.607+0000 7f563fe7c640 1 --2- 172.21.15.83:0/790815562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f563815bc10 0x7f5638151870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:18.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.607+0000 7f563fe7c640 1 --2- 172.21.15.83:0/790815562 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56380d0490 0x7f5638153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:18.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.607+0000 7f563fe7c640 1 --2- 172.21.15.83:0/790815562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f563806e500 0x7f56380c5240 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:18.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.607+0000 7f563fe7c640 1 -- 172.21.15.83:0/790815562 >> 172.21.15.83:0/790815562 conn(0x7f56380d17e0 msgr2=0x7f56380d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:18.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.607+0000 7f563fe7c640 1 -- 172.21.15.83:0/790815562 shutdown_connections 2024-04-17T10:47:18.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.607+0000 7f563fe7c640 1 -- 172.21.15.83:0/790815562 wait complete. 2024-04-17T10:47:18.623 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-17T10:47:18.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.751+0000 7f67bccaa640 1 Processor -- start 2024-04-17T10:47:18.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.751+0000 7f67bccaa640 1 -- start start 2024-04-17T10:47:18.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.751+0000 7f67bccaa640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67b806b2f0 0x7f67b8166880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:18.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.751+0000 7f67bccaa640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67b80b39d0 con 0x7f67b815cb40 2024-04-17T10:47:18.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.751+0000 7f67bccaa640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67b8070c40 con 0x7f67b80aedf0 2024-04-17T10:47:18.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.751+0000 7f67bccaa640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b8072730 con 0x7f67b806b2f0 2024-04-17T10:47:18.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.751+0000 7f67b5d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f67b815cb40 0x7f67b806abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35046/0 (socket says 172.21.15.83:35046) 2024-04-17T10:47:18.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.751+0000 7f67b5d9b640 1 -- 172.21.15.83:0/3334416100 learned_addr learned my addr 172.21.15.83:0/3334416100 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:18.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.751+0000 7f67b6d9d640 1 --2- 172.21.15.83:0/3334416100 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67b806b2f0 0x7f67b8166880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:18.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67b559a640 1 -- 172.21.15.83:0/3334416100 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1875385122 0 0) 0x7f67b80b39d0 con 0x7f67b815cb40 2024-04-17T10:47:18.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67b559a640 1 -- 172.21.15.83:0/3334416100 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67a4003530 con 0x7f67b815cb40 2024-04-17T10:47:18.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67b559a640 1 -- 172.21.15.83:0/3334416100 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2771732334 0 0) 0x7f67a4003530 con 0x7f67b815cb40 2024-04-17T10:47:18.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67b559a640 1 -- 172.21.15.83:0/3334416100 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67b806b2f0 msgr2=0x7f67b8166880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:18.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67b559a640 1 --2- 172.21.15.83:0/3334416100 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67b806b2f0 0x7f67b8166880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:18.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67b559a640 1 -- 172.21.15.83:0/3334416100 >> v1:172.21.15.167:6789/0 conn(0x7f67b80aedf0 legacy=0x7f67b80b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:18.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67b559a640 1 -- 172.21.15.83:0/3334416100 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b8070f20 con 0x7f67b815cb40 2024-04-17T10:47:18.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67b559a640 1 -- 172.21.15.83:0/3334416100 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f67a0001680 con 0x7f67b815cb40 2024-04-17T10:47:18.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67b559a640 1 -- 172.21.15.83:0/3334416100 >> v1:172.21.15.83:6789/0 conn(0x7f67b815cb40 legacy=0x7f67b806abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:18.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67b559a640 1 --2- 172.21.15.83:0/3334416100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67a40039f0 0x7f67a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:18.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67b559a640 1 --2- 172.21.15.83:0/3334416100 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67b806b2f0 0x7f67a4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:18.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67b559a640 1 --2- 172.21.15.83:0/3334416100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67a4004d50 0x7f67a40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:18.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67b559a640 1 -- 172.21.15.83:0/3334416100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a400b9d0 con 0x7f67a4004d50 2024-04-17T10:47:18.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67b559a640 1 -- 172.21.15.83:0/3334416100 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a400bb80 con 0x7f67a40039f0 2024-04-17T10:47:18.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67b559a640 1 -- 172.21.15.83:0/3334416100 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a400bd80 con 0x7f67b806b2f0 2024-04-17T10:47:18.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67b6d9d640 1 --2- 172.21.15.83:0/3334416100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67a4004d50 0x7f67a40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:18.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67b659c640 1 --2- 172.21.15.83:0/3334416100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67a40039f0 0x7f67a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:18.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67b5d9b640 1 --2- 172.21.15.83:0/3334416100 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67b806b2f0 0x7f67a4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:18.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67b6d9d640 1 -- 172.21.15.83:0/3334416100 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67b806b2f0 msgr2=0x7f67a4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:18.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67b6d9d640 1 --2- 172.21.15.83:0/3334416100 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67b806b2f0 0x7f67a4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:18.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67b6d9d640 1 -- 172.21.15.83:0/3334416100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67a40039f0 msgr2=0x7f67a4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:18.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67b6d9d640 1 --2- 172.21.15.83:0/3334416100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67a40039f0 0x7f67a4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:18.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67b6d9d640 1 -- 172.21.15.83:0/3334416100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a4003530 con 0x7f67a4004d50 2024-04-17T10:47:18.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67b659c640 1 --2- 172.21.15.83:0/3334416100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67a40039f0 0x7f67a4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:47:18.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67b6d9d640 1 --2- 172.21.15.83:0/3334416100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67a4004d50 0x7f67a40051d0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f67b0009ac0 tx=0x7f67b001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265b806ae92a8a93 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:18.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67b559a640 1 -- 172.21.15.83:0/3334416100 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b00b2070 con 0x7f67a4004d50 2024-04-17T10:47:18.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67b559a640 1 -- 172.21.15.83:0/3334416100 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b00a5530 con 0x7f67a4004d50 2024-04-17T10:47:18.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67b559a640 1 -- 172.21.15.83:0/3334416100 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0009690 con 0x7f67a4004d50 2024-04-17T10:47:18.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67bccaa640 1 -- 172.21.15.83:0/3334416100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67a4004d50 msgr2=0x7f67a40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:18.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67bccaa640 1 --2- 172.21.15.83:0/3334416100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67a4004d50 0x7f67a40051d0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f67b0009ac0 tx=0x7f67b001ca50 comp rx=0 tx=0).stop 2024-04-17T10:47:18.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67b6d9d640 1 -- 172.21.15.83:0/3334416100 reap_dead start 2024-04-17T10:47:18.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67bccaa640 1 -- 172.21.15.83:0/3334416100 shutdown_connections 2024-04-17T10:47:18.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67bccaa640 1 -- 172.21.15.83:0/3334416100 >> 172.21.15.83:0/3334416100 conn(0x7f67b80cc050 msgr2=0x7f67b80cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:18.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67bccaa640 1 -- 172.21.15.83:0/3334416100 shutdown_connections 2024-04-17T10:47:18.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.755+0000 7f67bccaa640 1 -- 172.21.15.83:0/3334416100 wait complete. 2024-04-17T10:47:18.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.759+0000 7f67bccaa640 1 Processor -- start 2024-04-17T10:47:18.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.759+0000 7f67bccaa640 1 -- start start 2024-04-17T10:47:18.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.759+0000 7f67bccaa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67a4004d50 0x7f67b814d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:18.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.759+0000 7f67bccaa640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67b814d8f0 0x7f67b81f4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:18.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.759+0000 7f67bccaa640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67b81f5300 0x7f67b81f5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:18.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.759+0000 7f67bccaa640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b806c160 con 0x7f67a4004d50 2024-04-17T10:47:18.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.759+0000 7f67bccaa640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b8071290 con 0x7f67b814d8f0 2024-04-17T10:47:18.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.759+0000 7f67bccaa640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b806c2e0 con 0x7f67b81f5300 2024-04-17T10:47:18.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.759+0000 7f67b659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67a4004d50 0x7f67b814d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:18.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.759+0000 7f67b659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67a4004d50 0x7f67b814d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41968/0 (socket says 172.21.15.83:41968) 2024-04-17T10:47:18.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.759+0000 7f67b659c640 1 -- 172.21.15.83:0/657108885 learned_addr learned my addr 172.21.15.83:0/657108885 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:18.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.759+0000 7f67b5d9b640 1 --2- 172.21.15.83:0/657108885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67b814d8f0 0x7f67b81f4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:18.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.759+0000 7f67b6d9d640 1 --2- 172.21.15.83:0/657108885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67b81f5300 0x7f67b81f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:18.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.759+0000 7f67b659c640 1 -- 172.21.15.83:0/657108885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67b81f5300 msgr2=0x7f67b81f5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:18.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.759+0000 7f67b659c640 1 --2- 172.21.15.83:0/657108885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67b81f5300 0x7f67b81f5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:18.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.759+0000 7f67b659c640 1 -- 172.21.15.83:0/657108885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67b814d8f0 msgr2=0x7f67b81f4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:18.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.759+0000 7f67b659c640 1 --2- 172.21.15.83:0/657108885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67b814d8f0 0x7f67b81f4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:18.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.759+0000 7f67b659c640 1 -- 172.21.15.83:0/657108885 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b815ab20 con 0x7f67a4004d50 2024-04-17T10:47:18.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.759+0000 7f67b6d9d640 1 --2- 172.21.15.83:0/657108885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67b81f5300 0x7f67b81f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:18.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.759+0000 7f67b659c640 1 --2- 172.21.15.83:0/657108885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67a4004d50 0x7f67b814d2c0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f67a800e970 tx=0x7f67a800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:18.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.759+0000 7f67aeffd640 1 -- 172.21.15.83:0/657108885 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a8003b90 con 0x7f67a4004d50 2024-04-17T10:47:18.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.759+0000 7f67aeffd640 1 -- 172.21.15.83:0/657108885 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a8018070 con 0x7f67a4004d50 2024-04-17T10:47:18.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.759+0000 7f67bccaa640 1 -- 172.21.15.83:0/657108885 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b815ae00 con 0x7f67a4004d50 2024-04-17T10:47:18.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.759+0000 7f67aeffd640 1 -- 172.21.15.83:0/657108885 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a80050d0 con 0x7f67a4004d50 2024-04-17T10:47:18.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.759+0000 7f67bccaa640 1 -- 172.21.15.83:0/657108885 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b814dfd0 con 0x7f67a4004d50 2024-04-17T10:47:18.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.759+0000 7f67bccaa640 1 -- 172.21.15.83:0/657108885 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67b8075000 con 0x7f67a4004d50 2024-04-17T10:47:18.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.763+0000 7f67aeffd640 1 -- 172.21.15.83:0/657108885 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f67a8005270 con 0x7f67a4004d50 2024-04-17T10:47:18.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.763+0000 7f67aeffd640 1 --2- 172.21.15.83:0/657108885 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6798066f90 0x7f6798069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:18.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.763+0000 7f67aeffd640 1 -- 172.21.15.83:0/657108885 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f67a8084ff0 con 0x7f67a4004d50 2024-04-17T10:47:18.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.763+0000 7f67b5d9b640 1 --2- 172.21.15.83:0/657108885 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6798066f90 0x7f6798069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:18.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.763+0000 7f67b5d9b640 1 --2- 172.21.15.83:0/657108885 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6798066f90 0x7f6798069450 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f67a0001d00 tx=0x7f67a0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:18.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.767+0000 7f67aeffd640 1 -- 172.21.15.83:0/657108885 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f67a8056020 con 0x7f67a4004d50 2024-04-17T10:47:18.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:18.927+0000 7f67bccaa640 1 -- 172.21.15.83:0/657108885 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f67b80b2cc0 con 0x7f67a4004d50 2024-04-17T10:47:19.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.603+0000 7f67aeffd640 1 -- 172.21.15.83:0/657108885 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v93) v1 ==== 181+0+0 (secure 0 0 0) 0x7f67a801be40 con 0x7f67a4004d50 2024-04-17T10:47:19.601 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2024-04-17T10:47:19.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.611+0000 7f67bccaa640 1 -- 172.21.15.83:0/657108885 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6798066f90 msgr2=0x7f6798069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:19.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.611+0000 7f67bccaa640 1 --2- 172.21.15.83:0/657108885 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6798066f90 0x7f6798069450 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f67a0001d00 tx=0x7f67a0009040 comp rx=0 tx=0).stop 2024-04-17T10:47:19.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.611+0000 7f67bccaa640 1 -- 172.21.15.83:0/657108885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67a4004d50 msgr2=0x7f67b814d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:19.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.611+0000 7f67bccaa640 1 --2- 172.21.15.83:0/657108885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67a4004d50 0x7f67b814d2c0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f67a800e970 tx=0x7f67a800ee30 comp rx=0 tx=0).stop 2024-04-17T10:47:19.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.611+0000 7f67bccaa640 1 -- 172.21.15.83:0/657108885 shutdown_connections 2024-04-17T10:47:19.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.611+0000 7f67bccaa640 1 --2- 172.21.15.83:0/657108885 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6798066f90 0x7f6798069450 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:19.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.611+0000 7f67bccaa640 1 --2- 172.21.15.83:0/657108885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67b81f5300 0x7f67b81f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:19.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.611+0000 7f67bccaa640 1 --2- 172.21.15.83:0/657108885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67b814d8f0 0x7f67b81f4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:19.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.611+0000 7f67bccaa640 1 --2- 172.21.15.83:0/657108885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67a4004d50 0x7f67b814d2c0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:19.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.611+0000 7f67bccaa640 1 -- 172.21.15.83:0/657108885 >> 172.21.15.83:0/657108885 conn(0x7f67b80cc050 msgr2=0x7f67b806c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:19.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.611+0000 7f67bccaa640 1 -- 172.21.15.83:0/657108885 shutdown_connections 2024-04-17T10:47:19.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.611+0000 7f67bccaa640 1 -- 172.21.15.83:0/657108885 wait complete. 2024-04-17T10:47:19.633 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-17T10:47:19.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.759+0000 7fadf9a41640 1 Processor -- start 2024-04-17T10:47:19.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.759+0000 7fadf9a41640 1 -- start start 2024-04-17T10:47:19.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.759+0000 7fadf9a41640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fadf41612c0 0x7fadf40d1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:19.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.759+0000 7fadf9a41640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadf40d0990 0x7fadf40b4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:19.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.759+0000 7fadf9a41640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf40b34b0 con 0x7fadf40d2510 2024-04-17T10:47:19.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.759+0000 7fadf9a41640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadf40b0600 con 0x7fadf41612c0 2024-04-17T10:47:19.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.759+0000 7fadf9a41640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf40b3750 con 0x7fadf40d0990 2024-04-17T10:47:19.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.759+0000 7fadf2ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fadf41612c0 0x7fadf40d1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:19.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.759+0000 7fadf27fc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fadf40d2510 0x7fadf40d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35060/0 (socket says 172.21.15.83:35060) 2024-04-17T10:47:19.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.759+0000 7fadf27fc640 1 -- 172.21.15.83:0/3063773864 learned_addr learned my addr 172.21.15.83:0/3063773864 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:19.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.759+0000 7fadf2ffd640 1 -- 172.21.15.83:0/3063773864 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadf40d0990 msgr2=0x7fadf40b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:19.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.759+0000 7fadf2ffd640 1 --2- 172.21.15.83:0/3063773864 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadf40d0990 0x7fadf40b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:19.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.759+0000 7fadf2ffd640 1 -- 172.21.15.83:0/3063773864 >> v1:172.21.15.83:6789/0 conn(0x7fadf40d2510 legacy=0x7fadf40d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:19.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.759+0000 7fadf2ffd640 1 -- 172.21.15.83:0/3063773864 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf40b3a30 con 0x7fadf41612c0 2024-04-17T10:47:19.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.759+0000 7fadf1ffb640 1 -- 172.21.15.83:0/3063773864 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2769209597 0 0) 0x7fadf40b34b0 con 0x7fadf40d2510 2024-04-17T10:47:19.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.763+0000 7fadf2ffd640 1 --2- 172.21.15.83:0/3063773864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf41612c0 0x7fadf40d1fd0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fade40097c0 tx=0x7fade401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9752abfb8be9cff6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:19.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.763+0000 7fadf1ffb640 1 -- 172.21.15.83:0/3063773864 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade40ab610 con 0x7fadf41612c0 2024-04-17T10:47:19.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.763+0000 7fadf1ffb640 1 -- 172.21.15.83:0/3063773864 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade40b0070 con 0x7fadf41612c0 2024-04-17T10:47:19.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.763+0000 7fadf1ffb640 1 -- 172.21.15.83:0/3063773864 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade40ab9d0 con 0x7fadf41612c0 2024-04-17T10:47:19.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.763+0000 7fadf9a41640 1 -- 172.21.15.83:0/3063773864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf41612c0 msgr2=0x7fadf40d1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:19.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.763+0000 7fadf9a41640 1 --2- 172.21.15.83:0/3063773864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf41612c0 0x7fadf40d1fd0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fade40097c0 tx=0x7fade401c470 comp rx=0 tx=0).stop 2024-04-17T10:47:19.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.763+0000 7fadf9a41640 1 -- 172.21.15.83:0/3063773864 shutdown_connections 2024-04-17T10:47:19.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.763+0000 7fadf9a41640 1 --2- 172.21.15.83:0/3063773864 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadf40d0990 0x7fadf40b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:19.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.763+0000 7fadf9a41640 1 --2- 172.21.15.83:0/3063773864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf41612c0 0x7fadf40d1fd0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:19.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.763+0000 7fadf9a41640 1 -- 172.21.15.83:0/3063773864 >> 172.21.15.83:0/3063773864 conn(0x7fadf409cd10 msgr2=0x7fadf40d3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:19.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.763+0000 7fadf9a41640 1 -- 172.21.15.83:0/3063773864 shutdown_connections 2024-04-17T10:47:19.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.763+0000 7fadf9a41640 1 -- 172.21.15.83:0/3063773864 wait complete. 2024-04-17T10:47:19.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.763+0000 7fadf9a41640 1 Processor -- start 2024-04-17T10:47:19.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.763+0000 7fadf9a41640 1 -- start start 2024-04-17T10:47:19.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.767+0000 7fadf9a41640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadf40d0990 0x7fadf40b6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:19.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.767+0000 7fadf9a41640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf40d2510 0x7fadf4159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:19.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.767+0000 7fadf9a41640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadf41612c0 0x7fadf40c7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:19.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.767+0000 7fadf9a41640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf40b5100 con 0x7fadf40d2510 2024-04-17T10:47:19.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.767+0000 7fadf9a41640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf40b4f80 con 0x7fadf40d0990 2024-04-17T10:47:19.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.767+0000 7fadf9a41640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf40b5280 con 0x7fadf41612c0 2024-04-17T10:47:19.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.767+0000 7fadf27fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf40d2510 0x7fadf4159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:19.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.767+0000 7fadf27fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf40d2510 0x7fadf4159b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41986/0 (socket says 172.21.15.83:41986) 2024-04-17T10:47:19.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.767+0000 7fadf27fc640 1 -- 172.21.15.83:0/2805391210 learned_addr learned my addr 172.21.15.83:0/2805391210 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:19.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.767+0000 7fadf37fe640 1 --2- 172.21.15.83:0/2805391210 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadf41612c0 0x7fadf40c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:19.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.767+0000 7fadf27fc640 1 -- 172.21.15.83:0/2805391210 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadf41612c0 msgr2=0x7fadf40c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:19.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.767+0000 7fadf27fc640 1 --2- 172.21.15.83:0/2805391210 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadf41612c0 0x7fadf40c7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:19.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.767+0000 7fadf27fc640 1 -- 172.21.15.83:0/2805391210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadf40d0990 msgr2=0x7fadf40b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:19.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.767+0000 7fadf27fc640 1 --2- 172.21.15.83:0/2805391210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadf40d0990 0x7fadf40b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:19.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.767+0000 7fadf27fc640 1 -- 172.21.15.83:0/2805391210 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf4158ad0 con 0x7fadf40d2510 2024-04-17T10:47:19.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.767+0000 7fadf37fe640 1 --2- 172.21.15.83:0/2805391210 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadf41612c0 0x7fadf40c7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:19.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.767+0000 7fadf27fc640 1 --2- 172.21.15.83:0/2805391210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf40d2510 0x7fadf4159b40 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fade800c480 tx=0x7fade800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:19.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.767+0000 7faddbfff640 1 -- 172.21.15.83:0/2805391210 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade8018070 con 0x7fadf40d2510 2024-04-17T10:47:19.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.767+0000 7faddbfff640 1 -- 172.21.15.83:0/2805391210 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade80074e0 con 0x7fadf40d2510 2024-04-17T10:47:19.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.767+0000 7fadf9a41640 1 -- 172.21.15.83:0/2805391210 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf4159060 con 0x7fadf40d2510 2024-04-17T10:47:19.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.767+0000 7faddbfff640 1 -- 172.21.15.83:0/2805391210 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade8004a30 con 0x7fadf40d2510 2024-04-17T10:47:19.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.767+0000 7fadf9a41640 1 -- 172.21.15.83:0/2805391210 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf4159620 con 0x7fadf40d2510 2024-04-17T10:47:19.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.767+0000 7fadf9a41640 1 -- 172.21.15.83:0/2805391210 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf4075000 con 0x7fadf40d2510 2024-04-17T10:47:19.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.767+0000 7faddbfff640 1 -- 172.21.15.83:0/2805391210 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fade801b070 con 0x7fadf40d2510 2024-04-17T10:47:19.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.771+0000 7faddbfff640 1 --2- 172.21.15.83:0/2805391210 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fade0066f60 0x7fade0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:19.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.771+0000 7faddbfff640 1 -- 172.21.15.83:0/2805391210 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5386+0+0 (secure 0 0 0) 0x7fade8084420 con 0x7fadf40d2510 2024-04-17T10:47:19.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.771+0000 7fadf2ffd640 1 --2- 172.21.15.83:0/2805391210 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fade0066f60 0x7fade0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:19.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.771+0000 7fadf2ffd640 1 --2- 172.21.15.83:0/2805391210 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fade0066f60 0x7fade0069420 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fade40af000 tx=0x7fade4004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:19.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.771+0000 7faddbfff640 1 -- 172.21.15.83:0/2805391210 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fade80556e0 con 0x7fadf40d2510 2024-04-17T10:47:19.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:19.923+0000 7fadf9a41640 1 -- 172.21.15.83:0/2805391210 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fadf40b1c80 con 0x7fadf40d2510 2024-04-17T10:47:20.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.615+0000 7faddbfff640 1 -- 172.21.15.83:0/2805391210 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v94) v1 ==== 169+0+0 (secure 0 0 0) 0x7fade8054fe0 con 0x7fadf40d2510 2024-04-17T10:47:20.609 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2024-04-17T10:47:20.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.615+0000 7fadf9a41640 1 -- 172.21.15.83:0/2805391210 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fade0066f60 msgr2=0x7fade0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:20.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.615+0000 7fadf9a41640 1 --2- 172.21.15.83:0/2805391210 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fade0066f60 0x7fade0069420 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fade40af000 tx=0x7fade4004740 comp rx=0 tx=0).stop 2024-04-17T10:47:20.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.615+0000 7fadf9a41640 1 -- 172.21.15.83:0/2805391210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf40d2510 msgr2=0x7fadf4159b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:20.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.615+0000 7fadf9a41640 1 --2- 172.21.15.83:0/2805391210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf40d2510 0x7fadf4159b40 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fade800c480 tx=0x7fade800c940 comp rx=0 tx=0).stop 2024-04-17T10:47:20.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.615+0000 7fadf9a41640 1 -- 172.21.15.83:0/2805391210 shutdown_connections 2024-04-17T10:47:20.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.615+0000 7fadf9a41640 1 --2- 172.21.15.83:0/2805391210 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fade0066f60 0x7fade0069420 secure :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fade40af000 tx=0x7fade4004740 comp rx=0 tx=0).stop 2024-04-17T10:47:20.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.615+0000 7fadf9a41640 1 --2- 172.21.15.83:0/2805391210 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadf41612c0 0x7fadf40c7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:20.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.615+0000 7fadf9a41640 1 --2- 172.21.15.83:0/2805391210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf40d2510 0x7fadf4159b40 secure :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fade800c480 tx=0x7fade800c940 comp rx=0 tx=0).stop 2024-04-17T10:47:20.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.615+0000 7fadf9a41640 1 --2- 172.21.15.83:0/2805391210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadf40d0990 0x7fadf40b6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:20.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.615+0000 7fadf9a41640 1 -- 172.21.15.83:0/2805391210 >> 172.21.15.83:0/2805391210 conn(0x7fadf409cd10 msgr2=0x7fadf40cf990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:20.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.619+0000 7fadf9a41640 1 -- 172.21.15.83:0/2805391210 shutdown_connections 2024-04-17T10:47:20.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.619+0000 7fadf9a41640 1 -- 172.21.15.83:0/2805391210 wait complete. 2024-04-17T10:47:20.646 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-17T10:47:20.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.771+0000 7eff93c78640 1 Processor -- start 2024-04-17T10:47:20.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.771+0000 7eff93c78640 1 -- start start 2024-04-17T10:47:20.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.775+0000 7eff93c78640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7eff8c154de0 0x7eff8c06ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:20.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.775+0000 7eff93c78640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7eff8c0b4210 0x7eff8c06cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:20.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.775+0000 7eff93c78640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff8c15bd70 con 0x7eff8c06b350 2024-04-17T10:47:20.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.775+0000 7eff93c78640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff8c0b4870 con 0x7eff8c154de0 2024-04-17T10:47:20.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.775+0000 7eff93c78640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff8c15bf60 con 0x7eff8c0b4210 2024-04-17T10:47:20.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.775+0000 7eff91a14640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7eff8c154de0 0x7eff8c06ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:20.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.775+0000 7eff91a14640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7eff8c154de0 0x7eff8c06ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41998/0 (socket says 172.21.15.83:41998) 2024-04-17T10:47:20.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.775+0000 7eff91a14640 1 -- 172.21.15.83:0/2948605431 learned_addr learned my addr 172.21.15.83:0/2948605431 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:20.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.775+0000 7eff92215640 1 --2- 172.21.15.83:0/2948605431 >> v2:172.21.15.167:3300/0 conn(0x7eff8c0b4210 0x7eff8c06cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:20.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.775+0000 7eff91a14640 1 -- 172.21.15.83:0/2948605431 >> v2:172.21.15.167:3300/0 conn(0x7eff8c0b4210 msgr2=0x7eff8c06cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:20.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.775+0000 7eff91a14640 1 --2- 172.21.15.83:0/2948605431 >> v2:172.21.15.167:3300/0 conn(0x7eff8c0b4210 0x7eff8c06cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:20.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.775+0000 7eff91a14640 1 -- 172.21.15.83:0/2948605431 >> v1:172.21.15.167:6789/0 conn(0x7eff8c06b350 legacy=0x7eff8c0b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:20.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.775+0000 7eff91a14640 1 -- 172.21.15.83:0/2948605431 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff8c15afb0 con 0x7eff8c154de0 2024-04-17T10:47:20.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.775+0000 7eff92215640 1 --2- 172.21.15.83:0/2948605431 >> v2:172.21.15.167:3300/0 conn(0x7eff8c0b4210 0x7eff8c06cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:47:20.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.779+0000 7eff91a14640 1 --2- 172.21.15.83:0/2948605431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff8c154de0 0x7eff8c06ae10 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7eff8c15e2c0 tx=0x7eff8801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775afe87908eac57 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:20.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.779+0000 7eff90a12640 1 -- 172.21.15.83:0/2948605431 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff880b0070 con 0x7eff8c154de0 2024-04-17T10:47:20.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.779+0000 7eff90a12640 1 -- 172.21.15.83:0/2948605431 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff880ab600 con 0x7eff8c154de0 2024-04-17T10:47:20.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.779+0000 7eff90a12640 1 -- 172.21.15.83:0/2948605431 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff880ab9d0 con 0x7eff8c154de0 2024-04-17T10:47:20.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.779+0000 7eff93c78640 1 -- 172.21.15.83:0/2948605431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff8c154de0 msgr2=0x7eff8c06ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:20.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.779+0000 7eff93c78640 1 --2- 172.21.15.83:0/2948605431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff8c154de0 0x7eff8c06ae10 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7eff8c15e2c0 tx=0x7eff8801c520 comp rx=0 tx=0).stop 2024-04-17T10:47:20.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.779+0000 7eff93c78640 1 -- 172.21.15.83:0/2948605431 shutdown_connections 2024-04-17T10:47:20.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.779+0000 7eff93c78640 1 --2- 172.21.15.83:0/2948605431 >> v2:172.21.15.167:3300/0 conn(0x7eff8c0b4210 0x7eff8c06cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:20.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.779+0000 7eff93c78640 1 --2- 172.21.15.83:0/2948605431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff8c154de0 0x7eff8c06ae10 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:20.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.779+0000 7eff93c78640 1 -- 172.21.15.83:0/2948605431 >> 172.21.15.83:0/2948605431 conn(0x7eff8c0b15b0 msgr2=0x7eff8c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:20.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.779+0000 7eff93c78640 1 -- 172.21.15.83:0/2948605431 shutdown_connections 2024-04-17T10:47:20.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.779+0000 7eff93c78640 1 -- 172.21.15.83:0/2948605431 wait complete. 2024-04-17T10:47:20.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.779+0000 7eff93c78640 1 Processor -- start 2024-04-17T10:47:20.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.779+0000 7eff93c78640 1 -- start start 2024-04-17T10:47:20.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.779+0000 7eff93c78640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff8c06b350 0x7eff8c14c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:20.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.779+0000 7eff93c78640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff8c0b4210 0x7eff8c14cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:20.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.779+0000 7eff93c78640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff8c154de0 0x7eff8c14d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:20.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.779+0000 7eff93c78640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c1551c0 con 0x7eff8c0b4210 2024-04-17T10:47:20.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.779+0000 7eff93c78640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c06b780 con 0x7eff8c06b350 2024-04-17T10:47:20.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.779+0000 7eff93c78640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c070470 con 0x7eff8c154de0 2024-04-17T10:47:20.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.779+0000 7eff91213640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff8c0b4210 0x7eff8c14cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:20.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.779+0000 7eff91213640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff8c0b4210 0x7eff8c14cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42014/0 (socket says 172.21.15.83:42014) 2024-04-17T10:47:20.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.779+0000 7eff91213640 1 -- 172.21.15.83:0/2540790732 learned_addr learned my addr 172.21.15.83:0/2540790732 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.779+0000 7eff91a14640 1 --2- 172.21.15.83:0/2540790732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff8c06b350 0x7eff8c14c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.779+0000 7eff92215640 1 --2- 172.21.15.83:0/2540790732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff8c154de0 0x7eff8c14d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.779+0000 7eff91213640 1 -- 172.21.15.83:0/2540790732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff8c154de0 msgr2=0x7eff8c14d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.779+0000 7eff91213640 1 --2- 172.21.15.83:0/2540790732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff8c154de0 0x7eff8c14d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.779+0000 7eff91213640 1 -- 172.21.15.83:0/2540790732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff8c06b350 msgr2=0x7eff8c14c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.779+0000 7eff91213640 1 --2- 172.21.15.83:0/2540790732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff8c06b350 0x7eff8c14c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.779+0000 7eff91213640 1 -- 172.21.15.83:0/2540790732 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff8c1569e0 con 0x7eff8c0b4210 2024-04-17T10:47:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.783+0000 7eff91213640 1 --2- 172.21.15.83:0/2540790732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff8c0b4210 0x7eff8c14cf80 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00c450 tx=0x7eff7c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.783+0000 7eff827fc640 1 -- 172.21.15.83:0/2540790732 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c016020 con 0x7eff8c0b4210 2024-04-17T10:47:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.783+0000 7eff827fc640 1 -- 172.21.15.83:0/2540790732 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff7c0052d0 con 0x7eff8c0b4210 2024-04-17T10:47:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.783+0000 7eff93c78640 1 -- 172.21.15.83:0/2540790732 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff8c156cc0 con 0x7eff8c0b4210 2024-04-17T10:47:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.783+0000 7eff827fc640 1 -- 172.21.15.83:0/2540790732 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c007260 con 0x7eff8c0b4210 2024-04-17T10:47:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.783+0000 7eff93c78640 1 -- 172.21.15.83:0/2540790732 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff8c157280 con 0x7eff8c0b4210 2024-04-17T10:47:20.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.783+0000 7eff93c78640 1 -- 172.21.15.83:0/2540790732 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff8c074f90 con 0x7eff8c0b4210 2024-04-17T10:47:20.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.783+0000 7eff827fc640 1 -- 172.21.15.83:0/2540790732 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7eff7c007400 con 0x7eff8c0b4210 2024-04-17T10:47:20.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.783+0000 7eff827fc640 1 --2- 172.21.15.83:0/2540790732 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7eff70066f60 0x7eff70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:20.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.783+0000 7eff827fc640 1 -- 172.21.15.83:0/2540790732 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7eff7c0844e0 con 0x7eff8c0b4210 2024-04-17T10:47:20.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.783+0000 7eff91a14640 1 --2- 172.21.15.83:0/2540790732 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7eff70066f60 0x7eff70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:20.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.783+0000 7eff91a14640 1 --2- 172.21.15.83:0/2540790732 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7eff70066f60 0x7eff70069420 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7eff880ad000 tx=0x7eff88008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:20.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.787+0000 7eff827fc640 1 -- 172.21.15.83:0/2540790732 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7eff7c020020 con 0x7eff8c0b4210 2024-04-17T10:47:20.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.951+0000 7eff93c78640 1 -- 172.21.15.83:0/2540790732 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7eff8c158750 con 0x7eff8c0b4210 2024-04-17T10:47:20.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.955+0000 7eff827fc640 1 -- 172.21.15.83:0/2540790732 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7eff7c005480 con 0x7eff8c0b4210 2024-04-17T10:47:20.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.959+0000 7eff93c78640 1 -- 172.21.15.83:0/2540790732 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7eff70066f60 msgr2=0x7eff70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:20.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.959+0000 7eff93c78640 1 --2- 172.21.15.83:0/2540790732 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7eff70066f60 0x7eff70069420 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7eff880ad000 tx=0x7eff88008040 comp rx=0 tx=0).stop 2024-04-17T10:47:20.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.959+0000 7eff93c78640 1 -- 172.21.15.83:0/2540790732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff8c0b4210 msgr2=0x7eff8c14cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:20.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.959+0000 7eff93c78640 1 --2- 172.21.15.83:0/2540790732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff8c0b4210 0x7eff8c14cf80 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00c450 tx=0x7eff7c00c910 comp rx=0 tx=0).stop 2024-04-17T10:47:20.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.959+0000 7eff93c78640 1 -- 172.21.15.83:0/2540790732 shutdown_connections 2024-04-17T10:47:20.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.959+0000 7eff93c78640 1 --2- 172.21.15.83:0/2540790732 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7eff70066f60 0x7eff70069420 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:20.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.959+0000 7eff93c78640 1 --2- 172.21.15.83:0/2540790732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff8c154de0 0x7eff8c14d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:20.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.959+0000 7eff93c78640 1 --2- 172.21.15.83:0/2540790732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff8c0b4210 0x7eff8c14cf80 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:20.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.959+0000 7eff93c78640 1 --2- 172.21.15.83:0/2540790732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff8c06b350 0x7eff8c14c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:20.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.959+0000 7eff93c78640 1 -- 172.21.15.83:0/2540790732 >> 172.21.15.83:0/2540790732 conn(0x7eff8c0b15b0 msgr2=0x7eff8c15a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:20.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.959+0000 7eff93c78640 1 -- 172.21.15.83:0/2540790732 shutdown_connections 2024-04-17T10:47:20.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:20.959+0000 7eff93c78640 1 -- 172.21.15.83:0/2540790732 wait complete. 2024-04-17T10:47:20.973 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort' 2024-04-17T10:47:21.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.091+0000 7f87dabdd640 1 Processor -- start 2024-04-17T10:47:21.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.091+0000 7f87dabdd640 1 -- start start 2024-04-17T10:47:21.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.091+0000 7f87dabdd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f87d415dfa0 0x7f87d40b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:21.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.091+0000 7f87dabdd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f87d40ce980 0x7f87d40b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:21.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.091+0000 7f87dabdd640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87d4162a00 con 0x7f87d40cee30 2024-04-17T10:47:21.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.091+0000 7f87dabdd640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87d4161e80 con 0x7f87d40ce980 2024-04-17T10:47:21.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.091+0000 7f87dabdd640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87d4162160 con 0x7f87d415dfa0 2024-04-17T10:47:21.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.091+0000 7f87d3fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f87d40ce980 0x7f87d40b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:21.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.091+0000 7f87d3fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f87d40ce980 0x7f87d40b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42016/0 (socket says 172.21.15.83:42016) 2024-04-17T10:47:21.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.091+0000 7f87d3fff640 1 -- 172.21.15.83:0/3506389022 learned_addr learned my addr 172.21.15.83:0/3506389022 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:21.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.091+0000 7f87d8979640 1 --2- 172.21.15.83:0/3506389022 >> v2:172.21.15.167:3300/0 conn(0x7f87d415dfa0 0x7f87d40b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:21.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.091+0000 7f87d3fff640 1 -- 172.21.15.83:0/3506389022 >> v2:172.21.15.167:3300/0 conn(0x7f87d415dfa0 msgr2=0x7f87d40b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:21.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.091+0000 7f87d3fff640 1 --2- 172.21.15.83:0/3506389022 >> v2:172.21.15.167:3300/0 conn(0x7f87d415dfa0 0x7f87d40b6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:21.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.091+0000 7f87d3fff640 1 -- 172.21.15.83:0/3506389022 >> v1:172.21.15.167:6789/0 conn(0x7f87d40cee30 legacy=0x7f87d40d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:21.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.091+0000 7f87d3fff640 1 -- 172.21.15.83:0/3506389022 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d4162440 con 0x7f87d40ce980 2024-04-17T10:47:21.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.095+0000 7f87d3fff640 1 --2- 172.21.15.83:0/3506389022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87d40ce980 0x7f87d40b5430 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f87d408c3f0 tx=0x7f87c801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d7f9556b04d830 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:21.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.095+0000 7f87d37fe640 1 -- 172.21.15.83:0/3506389022 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c80b0070 con 0x7f87d40ce980 2024-04-17T10:47:21.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.095+0000 7f87d37fe640 1 -- 172.21.15.83:0/3506389022 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c80ab600 con 0x7f87d40ce980 2024-04-17T10:47:21.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.095+0000 7f87d37fe640 1 -- 172.21.15.83:0/3506389022 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c80ab9d0 con 0x7f87d40ce980 2024-04-17T10:47:21.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.095+0000 7f87dabdd640 1 -- 172.21.15.83:0/3506389022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87d40ce980 msgr2=0x7f87d40b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:21.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.095+0000 7f87dabdd640 1 --2- 172.21.15.83:0/3506389022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87d40ce980 0x7f87d40b5430 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f87d408c3f0 tx=0x7f87c801c420 comp rx=0 tx=0).stop 2024-04-17T10:47:21.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.095+0000 7f87dabdd640 1 -- 172.21.15.83:0/3506389022 shutdown_connections 2024-04-17T10:47:21.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.095+0000 7f87dabdd640 1 --2- 172.21.15.83:0/3506389022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87d40ce980 0x7f87d40b5430 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:21.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.095+0000 7f87dabdd640 1 --2- 172.21.15.83:0/3506389022 >> v2:172.21.15.167:3300/0 conn(0x7f87d415dfa0 0x7f87d40b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:21.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.095+0000 7f87dabdd640 1 -- 172.21.15.83:0/3506389022 >> 172.21.15.83:0/3506389022 conn(0x7f87d40d0600 msgr2=0x7f87d40d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:21.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.095+0000 7f87dabdd640 1 -- 172.21.15.83:0/3506389022 shutdown_connections 2024-04-17T10:47:21.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.095+0000 7f87dabdd640 1 -- 172.21.15.83:0/3506389022 wait complete. 2024-04-17T10:47:21.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.095+0000 7f87dabdd640 1 Processor -- start 2024-04-17T10:47:21.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.095+0000 7f87dabdd640 1 -- start start 2024-04-17T10:47:21.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.095+0000 7f87dabdd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87d40ce980 0x7f87d40cce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:21.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.095+0000 7f87dabdd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87d40cee30 0x7f87d414e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:21.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.095+0000 7f87dabdd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87d415dfa0 0x7f87d40cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:21.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.095+0000 7f87dabdd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d4161b10 con 0x7f87d40ce980 2024-04-17T10:47:21.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.095+0000 7f87dabdd640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d40d2850 con 0x7f87d415dfa0 2024-04-17T10:47:21.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.095+0000 7f87dabdd640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d416b660 con 0x7f87d40cee30 2024-04-17T10:47:21.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.095+0000 7f87d8979640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87d40ce980 0x7f87d40cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:21.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.095+0000 7f87d8979640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87d40ce980 0x7f87d40cce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42032/0 (socket says 172.21.15.83:42032) 2024-04-17T10:47:21.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.095+0000 7f87d8979640 1 -- 172.21.15.83:0/3049856706 learned_addr learned my addr 172.21.15.83:0/3049856706 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:21.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.095+0000 7f87d8979640 1 -- 172.21.15.83:0/3049856706 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87d40cee30 msgr2=0x7f87d414e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:21.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.095+0000 7f87d3fff640 1 --2- 172.21.15.83:0/3049856706 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87d40cee30 0x7f87d414e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:21.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.095+0000 7f87d917a640 1 --2- 172.21.15.83:0/3049856706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87d415dfa0 0x7f87d40cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:21.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.095+0000 7f87d8979640 1 --2- 172.21.15.83:0/3049856706 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87d40cee30 0x7f87d414e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:21.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.095+0000 7f87d8979640 1 -- 172.21.15.83:0/3049856706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87d415dfa0 msgr2=0x7f87d40cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:21.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.095+0000 7f87d8979640 1 --2- 172.21.15.83:0/3049856706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87d415dfa0 0x7f87d40cb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:21.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.095+0000 7f87d8979640 1 -- 172.21.15.83:0/3049856706 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d40cbd70 con 0x7f87d40ce980 2024-04-17T10:47:21.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.095+0000 7f87d8979640 1 --2- 172.21.15.83:0/3049856706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87d40ce980 0x7f87d40cce10 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f87c400c820 tx=0x7f87c400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:21.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.099+0000 7f87d17fa640 1 -- 172.21.15.83:0/3049856706 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c4015070 con 0x7f87d40ce980 2024-04-17T10:47:21.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.099+0000 7f87d17fa640 1 -- 172.21.15.83:0/3049856706 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c4007a30 con 0x7f87d40ce980 2024-04-17T10:47:21.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.099+0000 7f87dabdd640 1 -- 172.21.15.83:0/3049856706 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d40cc050 con 0x7f87d40ce980 2024-04-17T10:47:21.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.099+0000 7f87d17fa640 1 -- 172.21.15.83:0/3049856706 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c4007e20 con 0x7f87d40ce980 2024-04-17T10:47:21.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.099+0000 7f87dabdd640 1 -- 172.21.15.83:0/3049856706 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d40cc610 con 0x7f87d40ce980 2024-04-17T10:47:21.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.099+0000 7f87dabdd640 1 -- 172.21.15.83:0/3049856706 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87d4074f90 con 0x7f87d40ce980 2024-04-17T10:47:21.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.099+0000 7f87d17fa640 1 -- 172.21.15.83:0/3049856706 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f87c4018070 con 0x7f87d40ce980 2024-04-17T10:47:21.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.099+0000 7f87d17fa640 1 --2- 172.21.15.83:0/3049856706 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f87c0066f60 0x7f87c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:21.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.099+0000 7f87d17fa640 1 -- 172.21.15.83:0/3049856706 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f87c4084d10 con 0x7f87d40ce980 2024-04-17T10:47:21.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.099+0000 7f87d3fff640 1 --2- 172.21.15.83:0/3049856706 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f87c0066f60 0x7f87c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:21.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.099+0000 7f87d3fff640 1 --2- 172.21.15.83:0/3049856706 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f87c0066f60 0x7f87c0069420 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f87c80ad000 tx=0x7f87c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:21.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.103+0000 7f87d17fa640 1 -- 172.21.15.83:0/3049856706 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f87c4055f90 con 0x7f87d40ce980 2024-04-17T10:47:21.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.347+0000 7f87dabdd640 1 -- 172.21.15.83:0/3049856706 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]} v 0) v1 -- 0x7f87d4159190 con 0x7f87d40ce980 2024-04-17T10:47:21.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.615+0000 7f87d17fa640 1 -- 172.21.15.83:0/3049856706 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]}]=0 v402) v1 ==== 144+0+0 (secure 0 0 0) 0x7f87c4055890 con 0x7f87d40ce980 2024-04-17T10:47:21.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.619+0000 7f87dabdd640 1 -- 172.21.15.83:0/3049856706 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f87c0066f60 msgr2=0x7f87c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:21.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.619+0000 7f87dabdd640 1 --2- 172.21.15.83:0/3049856706 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f87c0066f60 0x7f87c0069420 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f87c80ad000 tx=0x7f87c8008040 comp rx=0 tx=0).stop 2024-04-17T10:47:21.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.619+0000 7f87dabdd640 1 -- 172.21.15.83:0/3049856706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87d40ce980 msgr2=0x7f87d40cce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:21.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.619+0000 7f87dabdd640 1 --2- 172.21.15.83:0/3049856706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87d40ce980 0x7f87d40cce10 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f87c400c820 tx=0x7f87c400cce0 comp rx=0 tx=0).stop 2024-04-17T10:47:21.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.619+0000 7f87dabdd640 1 -- 172.21.15.83:0/3049856706 shutdown_connections 2024-04-17T10:47:21.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.619+0000 7f87dabdd640 1 --2- 172.21.15.83:0/3049856706 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f87c0066f60 0x7f87c0069420 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:21.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.619+0000 7f87dabdd640 1 --2- 172.21.15.83:0/3049856706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87d415dfa0 0x7f87d40cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:21.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.619+0000 7f87dabdd640 1 --2- 172.21.15.83:0/3049856706 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87d40cee30 0x7f87d414e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:21.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.619+0000 7f87dabdd640 1 --2- 172.21.15.83:0/3049856706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87d40ce980 0x7f87d40cce10 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:21.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.619+0000 7f87dabdd640 1 -- 172.21.15.83:0/3049856706 >> 172.21.15.83:0/3049856706 conn(0x7f87d40d0600 msgr2=0x7f87d4158020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:21.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.619+0000 7f87dabdd640 1 -- 172.21.15.83:0/3049856706 shutdown_connections 2024-04-17T10:47:21.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.619+0000 7f87dabdd640 1 -- 172.21.15.83:0/3049856706 wait complete. 2024-04-17T10:47:21.635 INFO:tasks.cephfs_test_runner:test_scrub_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-17T10:47:21.635 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-17T10:47:21.636 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:47:21.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.763+0000 7f0a427c8640 1 Processor -- start 2024-04-17T10:47:21.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.763+0000 7f0a427c8640 1 -- start start 2024-04-17T10:47:21.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.763+0000 7f0a427c8640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0a3c072330 0x7f0a3c0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:21.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.763+0000 7f0a427c8640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0a3c0cfa10 0x7f0a3c06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.763+0000 7f0a427c8640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a3c0cfea0 con 0x7f0a3c15ba30 2024-04-17T10:47:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.763+0000 7f0a427c8640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a3c071ea0 con 0x7f0a3c0cfa10 2024-04-17T10:47:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.763+0000 7f0a427c8640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a3c06e320 con 0x7f0a3c072330 2024-04-17T10:47:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.763+0000 7f0a40d65640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0a3c0cfa10 0x7f0a3c06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.763+0000 7f0a3bfff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0a3c15ba30 0x7f0a3c15c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35074/0 (socket says 172.21.15.83:35074) 2024-04-17T10:47:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.763+0000 7f0a3bfff640 1 -- 172.21.15.83:0/3468885361 learned_addr learned my addr 172.21.15.83:0/3468885361 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.763+0000 7f0a3b7fe640 1 --2- 172.21.15.83:0/3468885361 >> v2:172.21.15.167:3300/0 conn(0x7f0a3c072330 0x7f0a3c0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.763+0000 7f0a40d65640 1 -- 172.21.15.83:0/3468885361 >> v2:172.21.15.167:3300/0 conn(0x7f0a3c072330 msgr2=0x7f0a3c0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.763+0000 7f0a40d65640 1 --2- 172.21.15.83:0/3468885361 >> v2:172.21.15.167:3300/0 conn(0x7f0a3c072330 0x7f0a3c0cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.763+0000 7f0a40d65640 1 -- 172.21.15.83:0/3468885361 >> v1:172.21.15.83:6789/0 conn(0x7f0a3c15ba30 legacy=0x7f0a3c15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.763+0000 7f0a3affd640 1 -- 172.21.15.83:0/3468885361 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2379841164 0 0) 0x7f0a3c0cfea0 con 0x7f0a3c15ba30 2024-04-17T10:47:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.763+0000 7f0a40d65640 1 -- 172.21.15.83:0/3468885361 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3c06e040 con 0x7f0a3c0cfa10 2024-04-17T10:47:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.763+0000 7f0a3b7fe640 1 --2- 172.21.15.83:0/3468885361 >> v2:172.21.15.167:3300/0 conn(0x7f0a3c072330 0x7f0a3c0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.767+0000 7f0a40d65640 1 --2- 172.21.15.83:0/3468885361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a3c0cfa10 0x7f0a3c06c490 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c0af2b0 tx=0x7f0a2c01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997170196729dccf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.767+0000 7f0a3affd640 1 -- 172.21.15.83:0/3468885361 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c0b2070 con 0x7f0a3c0cfa10 2024-04-17T10:47:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.767+0000 7f0a3affd640 1 -- 172.21.15.83:0/3468885361 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2c0a5870 con 0x7f0a3c0cfa10 2024-04-17T10:47:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.767+0000 7f0a3affd640 1 -- 172.21.15.83:0/3468885361 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c0a5c40 con 0x7f0a3c0cfa10 2024-04-17T10:47:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.767+0000 7f0a427c8640 1 -- 172.21.15.83:0/3468885361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a3c0cfa10 msgr2=0x7f0a3c06c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.767+0000 7f0a427c8640 1 --2- 172.21.15.83:0/3468885361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a3c0cfa10 0x7f0a3c06c490 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c0af2b0 tx=0x7f0a2c01c8a0 comp rx=0 tx=0).stop 2024-04-17T10:47:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.767+0000 7f0a427c8640 1 -- 172.21.15.83:0/3468885361 shutdown_connections 2024-04-17T10:47:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.767+0000 7f0a427c8640 1 --2- 172.21.15.83:0/3468885361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a3c0cfa10 0x7f0a3c06c490 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:21.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.767+0000 7f0a427c8640 1 --2- 172.21.15.83:0/3468885361 >> v2:172.21.15.167:3300/0 conn(0x7f0a3c072330 0x7f0a3c0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:21.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.767+0000 7f0a427c8640 1 -- 172.21.15.83:0/3468885361 >> 172.21.15.83:0/3468885361 conn(0x7f0a3c06abe0 msgr2=0x7f0a3c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:21.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.767+0000 7f0a427c8640 1 -- 172.21.15.83:0/3468885361 shutdown_connections 2024-04-17T10:47:21.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.767+0000 7f0a427c8640 1 -- 172.21.15.83:0/3468885361 wait complete. 2024-04-17T10:47:21.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.767+0000 7f0a427c8640 1 Processor -- start 2024-04-17T10:47:21.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.767+0000 7f0a427c8640 1 -- start start 2024-04-17T10:47:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.767+0000 7f0a427c8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a3c072330 0x7f0a3c15de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.767+0000 7f0a427c8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a3c0cfa10 0x7f0a3c15c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.767+0000 7f0a427c8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a3c15ba30 0x7f0a3c0adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.767+0000 7f0a427c8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c06de20 con 0x7f0a3c15ba30 2024-04-17T10:47:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.767+0000 7f0a427c8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c06dca0 con 0x7f0a3c072330 2024-04-17T10:47:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.767+0000 7f0a427c8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c168520 con 0x7f0a3c0cfa10 2024-04-17T10:47:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.767+0000 7f0a40d65640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a3c15ba30 0x7f0a3c0adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.767+0000 7f0a40d65640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a3c15ba30 0x7f0a3c0adaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42044/0 (socket says 172.21.15.83:42044) 2024-04-17T10:47:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.767+0000 7f0a40d65640 1 -- 172.21.15.83:0/692364236 learned_addr learned my addr 172.21.15.83:0/692364236 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.767+0000 7f0a3bfff640 1 --2- 172.21.15.83:0/692364236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a3c072330 0x7f0a3c15de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.767+0000 7f0a3b7fe640 1 --2- 172.21.15.83:0/692364236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a3c0cfa10 0x7f0a3c15c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.767+0000 7f0a40d65640 1 -- 172.21.15.83:0/692364236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a3c0cfa10 msgr2=0x7f0a3c15c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.767+0000 7f0a40d65640 1 --2- 172.21.15.83:0/692364236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a3c0cfa10 0x7f0a3c15c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.767+0000 7f0a40d65640 1 -- 172.21.15.83:0/692364236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a3c072330 msgr2=0x7f0a3c15de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.767+0000 7f0a40d65640 1 --2- 172.21.15.83:0/692364236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a3c072330 0x7f0a3c15de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.767+0000 7f0a40d65640 1 -- 172.21.15.83:0/692364236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3c15cda0 con 0x7f0a3c15ba30 2024-04-17T10:47:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.767+0000 7f0a3b7fe640 1 --2- 172.21.15.83:0/692364236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a3c0cfa10 0x7f0a3c15c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.771+0000 7f0a40d65640 1 --2- 172.21.15.83:0/692364236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a3c15ba30 0x7f0a3c0adaa0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c01cd80 tx=0x7f0a2c019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.771+0000 7f0a38ff9640 1 -- 172.21.15.83:0/692364236 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c0bd030 con 0x7f0a3c15ba30 2024-04-17T10:47:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.771+0000 7f0a38ff9640 1 -- 172.21.15.83:0/692364236 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2c0039a0 con 0x7f0a3c15ba30 2024-04-17T10:47:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.771+0000 7f0a427c8640 1 -- 172.21.15.83:0/692364236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a3c15d080 con 0x7f0a3c15ba30 2024-04-17T10:47:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.771+0000 7f0a38ff9640 1 -- 172.21.15.83:0/692364236 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c0b2040 con 0x7f0a3c15ba30 2024-04-17T10:47:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.771+0000 7f0a427c8640 1 -- 172.21.15.83:0/692364236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a3c15d920 con 0x7f0a3c15ba30 2024-04-17T10:47:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.771+0000 7f0a427c8640 1 -- 172.21.15.83:0/692364236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a3c15d250 con 0x7f0a3c15ba30 2024-04-17T10:47:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.771+0000 7f0a38ff9640 1 -- 172.21.15.83:0/692364236 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0a2c0b8470 con 0x7f0a3c15ba30 2024-04-17T10:47:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.771+0000 7f0a38ff9640 1 --2- 172.21.15.83:0/692364236 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0a28066fb0 0x7f0a28069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.771+0000 7f0a38ff9640 1 -- 172.21.15.83:0/692364236 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0a2c11f6a0 con 0x7f0a3c15ba30 2024-04-17T10:47:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.771+0000 7f0a3bfff640 1 --2- 172.21.15.83:0/692364236 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0a28066fb0 0x7f0a28069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.771+0000 7f0a3bfff640 1 --2- 172.21.15.83:0/692364236 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0a28066fb0 0x7f0a28069470 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0a34000f70 tx=0x7f0a34007b60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.775+0000 7f0a38ff9640 1 -- 172.21.15.83:0/692364236 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0a2c0c7020 con 0x7f0a3c15ba30 2024-04-17T10:47:21.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.938+0000 7f0a427c8640 1 -- 172.21.15.83:0/692364236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a3c074f90 con 0x7f0a3c15ba30 2024-04-17T10:47:21.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.938+0000 7f0a38ff9640 1 -- 172.21.15.83:0/692364236 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+12949 (secure 0 0 0) 0x7f0a2c016520 con 0x7f0a3c15ba30 2024-04-17T10:47:21.936 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:47:21.936 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":94,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:47:20.609946+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6836/1056175765":"2024-04-18T10:47:17.593078+0000","172.21.15.83:6834/1056175765":"2024-04-18T10:47:17.593078+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:47:21.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.942+0000 7f0a427c8640 1 -- 172.21.15.83:0/692364236 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0a28066fb0 msgr2=0x7f0a28069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:21.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.942+0000 7f0a427c8640 1 --2- 172.21.15.83:0/692364236 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0a28066fb0 0x7f0a28069470 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0a34000f70 tx=0x7f0a34007b60 comp rx=0 tx=0).stop 2024-04-17T10:47:21.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.942+0000 7f0a427c8640 1 -- 172.21.15.83:0/692364236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a3c15ba30 msgr2=0x7f0a3c0adaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:21.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.942+0000 7f0a427c8640 1 --2- 172.21.15.83:0/692364236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a3c15ba30 0x7f0a3c0adaa0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c01cd80 tx=0x7f0a2c019640 comp rx=0 tx=0).stop 2024-04-17T10:47:21.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.942+0000 7f0a427c8640 1 -- 172.21.15.83:0/692364236 shutdown_connections 2024-04-17T10:47:21.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.942+0000 7f0a427c8640 1 --2- 172.21.15.83:0/692364236 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0a28066fb0 0x7f0a28069470 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:21.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.942+0000 7f0a427c8640 1 --2- 172.21.15.83:0/692364236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a3c15ba30 0x7f0a3c0adaa0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:21.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.942+0000 7f0a427c8640 1 --2- 172.21.15.83:0/692364236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a3c0cfa10 0x7f0a3c15c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:21.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.942+0000 7f0a427c8640 1 --2- 172.21.15.83:0/692364236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a3c072330 0x7f0a3c15de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:21.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.942+0000 7f0a427c8640 1 -- 172.21.15.83:0/692364236 >> 172.21.15.83:0/692364236 conn(0x7f0a3c06abe0 msgr2=0x7f0a3c15ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:21.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.942+0000 7f0a427c8640 1 -- 172.21.15.83:0/692364236 shutdown_connections 2024-04-17T10:47:21.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:21.942+0000 7f0a427c8640 1 -- 172.21.15.83:0/692364236 wait complete. 2024-04-17T10:47:21.955 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:47:21.955 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:47:22.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.090+0000 7f5a51b5c640 1 Processor -- start 2024-04-17T10:47:22.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.090+0000 7f5a51b5c640 1 -- start start 2024-04-17T10:47:22.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.094+0000 7f5a51b5c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5a4c06b520 0x7f5a4c0b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:22.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.094+0000 7f5a51b5c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a4c0b4040 0x7f5a4c06cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:22.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.094+0000 7f5a51b5c640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a4c15bd70 con 0x7f5a4c154de0 2024-04-17T10:47:22.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.094+0000 7f5a51b5c640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a4c0b4870 con 0x7f5a4c06b520 2024-04-17T10:47:22.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.094+0000 7f5a51b5c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c15bf60 con 0x7f5a4c0b4040 2024-04-17T10:47:22.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.094+0000 7f5a4affd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5a4c06b520 0x7f5a4c0b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:22.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.094+0000 7f5a4affd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5a4c06b520 0x7f5a4c0b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42048/0 (socket says 172.21.15.83:42048) 2024-04-17T10:47:22.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.094+0000 7f5a4affd640 1 -- 172.21.15.83:0/496948882 learned_addr learned my addr 172.21.15.83:0/496948882 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:22.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.094+0000 7f5a4bfff640 1 --2- 172.21.15.83:0/496948882 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a4c0b4040 0x7f5a4c06cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:22.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.094+0000 7f5a4affd640 1 -- 172.21.15.83:0/496948882 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a4c0b4040 msgr2=0x7f5a4c06cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:22.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.094+0000 7f5a4affd640 1 --2- 172.21.15.83:0/496948882 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a4c0b4040 0x7f5a4c06cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:22.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.094+0000 7f5a4affd640 1 -- 172.21.15.83:0/496948882 >> v1:172.21.15.83:6789/0 conn(0x7f5a4c154de0 legacy=0x7f5a4c06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:22.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.094+0000 7f5a4affd640 1 -- 172.21.15.83:0/496948882 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a4c15afb0 con 0x7f5a4c06b520 2024-04-17T10:47:22.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.094+0000 7f5a4bfff640 1 --2- 172.21.15.83:0/496948882 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a4c0b4040 0x7f5a4c06cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:22.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.094+0000 7f5a4affd640 1 --2- 172.21.15.83:0/496948882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a4c06b520 0x7f5a4c0b3b00 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009870 tx=0x7f5a3401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de57f196460f4d90 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:22.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.094+0000 7f5a4a7fc640 1 -- 172.21.15.83:0/496948882 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a340b0070 con 0x7f5a4c06b520 2024-04-17T10:47:22.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.094+0000 7f5a4a7fc640 1 -- 172.21.15.83:0/496948882 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a340ab600 con 0x7f5a4c06b520 2024-04-17T10:47:22.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.094+0000 7f5a4a7fc640 1 -- 172.21.15.83:0/496948882 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a340ab9d0 con 0x7f5a4c06b520 2024-04-17T10:47:22.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a51b5c640 1 -- 172.21.15.83:0/496948882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a4c06b520 msgr2=0x7f5a4c0b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:22.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a51b5c640 1 --2- 172.21.15.83:0/496948882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a4c06b520 0x7f5a4c0b3b00 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009870 tx=0x7f5a3401c3f0 comp rx=0 tx=0).stop 2024-04-17T10:47:22.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a51b5c640 1 -- 172.21.15.83:0/496948882 shutdown_connections 2024-04-17T10:47:22.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a51b5c640 1 --2- 172.21.15.83:0/496948882 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a4c0b4040 0x7f5a4c06cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:22.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a51b5c640 1 --2- 172.21.15.83:0/496948882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a4c06b520 0x7f5a4c0b3b00 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:22.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a51b5c640 1 -- 172.21.15.83:0/496948882 >> 172.21.15.83:0/496948882 conn(0x7f5a4c0b15b0 msgr2=0x7f5a4c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:22.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a51b5c640 1 -- 172.21.15.83:0/496948882 shutdown_connections 2024-04-17T10:47:22.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a51b5c640 1 -- 172.21.15.83:0/496948882 wait complete. 2024-04-17T10:47:22.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a51b5c640 1 Processor -- start 2024-04-17T10:47:22.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a51b5c640 1 -- start start 2024-04-17T10:47:22.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a51b5c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a4c06b520 0x7f5a4c0af170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:22.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a51b5c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a4c0b4040 0x7f5a4c1494e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:22.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a51b5c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a4c154de0 0x7f5a4c1565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:22.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a51b5c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c0b4450 con 0x7f5a4c0b4040 2024-04-17T10:47:22.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a51b5c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c1551c0 con 0x7f5a4c06b520 2024-04-17T10:47:22.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a51b5c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c0b45d0 con 0x7f5a4c154de0 2024-04-17T10:47:22.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a4affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a4c0b4040 0x7f5a4c1494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:22.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a4affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a4c0b4040 0x7f5a4c1494e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42052/0 (socket says 172.21.15.83:42052) 2024-04-17T10:47:22.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a4affd640 1 -- 172.21.15.83:0/3359608502 learned_addr learned my addr 172.21.15.83:0/3359608502 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:22.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a4bfff640 1 --2- 172.21.15.83:0/3359608502 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a4c154de0 0x7f5a4c1565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:22.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a4b7fe640 1 --2- 172.21.15.83:0/3359608502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a4c06b520 0x7f5a4c0af170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:22.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a4affd640 1 -- 172.21.15.83:0/3359608502 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a4c154de0 msgr2=0x7f5a4c1565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:22.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a4affd640 1 --2- 172.21.15.83:0/3359608502 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a4c154de0 0x7f5a4c1565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:22.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a4affd640 1 -- 172.21.15.83:0/3359608502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a4c06b520 msgr2=0x7f5a4c0af170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:22.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a4affd640 1 --2- 172.21.15.83:0/3359608502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a4c06b520 0x7f5a4c0af170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:22.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a4affd640 1 -- 172.21.15.83:0/3359608502 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a4c157140 con 0x7f5a4c0b4040 2024-04-17T10:47:22.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a4b7fe640 1 --2- 172.21.15.83:0/3359608502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a4c06b520 0x7f5a4c0af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:22.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a4affd640 1 --2- 172.21.15.83:0/3359608502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a4c0b4040 0x7f5a4c1494e0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f5a340ae040 tx=0x7f5a3401ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:22.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a43fff640 1 -- 172.21.15.83:0/3359608502 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a340ba020 con 0x7f5a4c0b4040 2024-04-17T10:47:22.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a43fff640 1 -- 172.21.15.83:0/3359608502 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a340b6620 con 0x7f5a4c0b4040 2024-04-17T10:47:22.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a51b5c640 1 -- 172.21.15.83:0/3359608502 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a4c157420 con 0x7f5a4c0b4040 2024-04-17T10:47:22.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a43fff640 1 -- 172.21.15.83:0/3359608502 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a340b0040 con 0x7f5a4c0b4040 2024-04-17T10:47:22.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.098+0000 7f5a51b5c640 1 -- 172.21.15.83:0/3359608502 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a4c14d280 con 0x7f5a4c0b4040 2024-04-17T10:47:22.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.102+0000 7f5a51b5c640 1 -- 172.21.15.83:0/3359608502 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a4c074f90 con 0x7f5a4c0b4040 2024-04-17T10:47:22.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.102+0000 7f5a43fff640 1 -- 172.21.15.83:0/3359608502 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5a340bd020 con 0x7f5a4c0b4040 2024-04-17T10:47:22.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.102+0000 7f5a43fff640 1 --2- 172.21.15.83:0/3359608502 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5a2c066f60 0x7f5a2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:22.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.102+0000 7f5a4b7fe640 1 --2- 172.21.15.83:0/3359608502 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5a2c066f60 0x7f5a2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:22.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.102+0000 7f5a43fff640 1 -- 172.21.15.83:0/3359608502 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5a3411eff0 con 0x7f5a4c0b4040 2024-04-17T10:47:22.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.102+0000 7f5a4b7fe640 1 --2- 172.21.15.83:0/3359608502 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5a2c066f60 0x7f5a2c069420 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c06e900 tx=0x7f5a3c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:22.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.106+0000 7f5a43fff640 1 -- 172.21.15.83:0/3359608502 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5a340c4030 con 0x7f5a4c0b4040 2024-04-17T10:47:22.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.262+0000 7f5a51b5c640 1 -- 172.21.15.83:0/3359608502 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5a4c0b3280 con 0x7f5a4c0b4040 2024-04-17T10:47:22.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.266+0000 7f5a43fff640 1 -- 172.21.15.83:0/3359608502 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5a34016070 con 0x7f5a4c0b4040 2024-04-17T10:47:22.261 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:47:22.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.266+0000 7f5a51b5c640 1 -- 172.21.15.83:0/3359608502 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5a2c066f60 msgr2=0x7f5a2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:22.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.266+0000 7f5a51b5c640 1 --2- 172.21.15.83:0/3359608502 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5a2c066f60 0x7f5a2c069420 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c06e900 tx=0x7f5a3c009040 comp rx=0 tx=0).stop 2024-04-17T10:47:22.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.266+0000 7f5a51b5c640 1 -- 172.21.15.83:0/3359608502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a4c0b4040 msgr2=0x7f5a4c1494e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:22.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.266+0000 7f5a51b5c640 1 --2- 172.21.15.83:0/3359608502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a4c0b4040 0x7f5a4c1494e0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f5a340ae040 tx=0x7f5a3401ac30 comp rx=0 tx=0).stop 2024-04-17T10:47:22.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.266+0000 7f5a51b5c640 1 -- 172.21.15.83:0/3359608502 shutdown_connections 2024-04-17T10:47:22.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.266+0000 7f5a51b5c640 1 --2- 172.21.15.83:0/3359608502 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5a2c066f60 0x7f5a2c069420 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:22.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.266+0000 7f5a51b5c640 1 --2- 172.21.15.83:0/3359608502 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a4c154de0 0x7f5a4c1565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:22.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.266+0000 7f5a51b5c640 1 --2- 172.21.15.83:0/3359608502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a4c0b4040 0x7f5a4c1494e0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:22.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.266+0000 7f5a51b5c640 1 --2- 172.21.15.83:0/3359608502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a4c06b520 0x7f5a4c0af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:22.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.266+0000 7f5a51b5c640 1 -- 172.21.15.83:0/3359608502 >> 172.21.15.83:0/3359608502 conn(0x7f5a4c0b15b0 msgr2=0x7f5a4c070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:22.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.266+0000 7f5a51b5c640 1 -- 172.21.15.83:0/3359608502 shutdown_connections 2024-04-17T10:47:22.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.266+0000 7f5a51b5c640 1 -- 172.21.15.83:0/3359608502 wait complete. 2024-04-17T10:47:22.281 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume' 2024-04-17T10:47:22.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.410+0000 7fac14d40640 1 Processor -- start 2024-04-17T10:47:22.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.410+0000 7fac14d40640 1 -- start start 2024-04-17T10:47:22.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.410+0000 7fac14d40640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac10159670 0x7fac100af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:22.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.410+0000 7fac14d40640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fac100b37f0 0x7fac100afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:22.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.410+0000 7fac14d40640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac1015c0b0 con 0x7fac1006abe0 2024-04-17T10:47:22.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.410+0000 7fac14d40640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac100cd0c0 con 0x7fac100b37f0 2024-04-17T10:47:22.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.410+0000 7fac14d40640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac1015c390 con 0x7fac10159670 2024-04-17T10:47:22.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.410+0000 7fac07fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fac100b37f0 0x7fac100afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:22.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.410+0000 7fac07fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fac100b37f0 0x7fac100afbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42056/0 (socket says 172.21.15.83:42056) 2024-04-17T10:47:22.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.410+0000 7fac07fff640 1 -- 172.21.15.83:0/2217792119 learned_addr learned my addr 172.21.15.83:0/2217792119 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:22.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.410+0000 7fac0e59c640 1 --2- 172.21.15.83:0/2217792119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac10159670 0x7fac100af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:22.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.410+0000 7fac07fff640 1 -- 172.21.15.83:0/2217792119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac10159670 msgr2=0x7fac100af6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:22.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.410+0000 7fac07fff640 1 --2- 172.21.15.83:0/2217792119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac10159670 0x7fac100af6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:22.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.410+0000 7fac07fff640 1 -- 172.21.15.83:0/2217792119 >> v1:172.21.15.167:6789/0 conn(0x7fac1006abe0 legacy=0x7fac1015f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:22.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.410+0000 7fac07fff640 1 -- 172.21.15.83:0/2217792119 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1015c670 con 0x7fac100b37f0 2024-04-17T10:47:22.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.414+0000 7fac07fff640 1 --2- 172.21.15.83:0/2217792119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac100b37f0 0x7fac100afbe0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fabf8009870 tx=0x7fabf801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44cb5d78f39f3d24 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:22.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.414+0000 7fac0dd9b640 1 -- 172.21.15.83:0/2217792119 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf80b0070 con 0x7fac100b37f0 2024-04-17T10:47:22.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.414+0000 7fac0dd9b640 1 -- 172.21.15.83:0/2217792119 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf80ab600 con 0x7fac100b37f0 2024-04-17T10:47:22.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.414+0000 7fac0dd9b640 1 -- 172.21.15.83:0/2217792119 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf80ab9d0 con 0x7fac100b37f0 2024-04-17T10:47:22.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.414+0000 7fac14d40640 1 -- 172.21.15.83:0/2217792119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac100b37f0 msgr2=0x7fac100afbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:22.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.414+0000 7fac14d40640 1 --2- 172.21.15.83:0/2217792119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac100b37f0 0x7fac100afbe0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fabf8009870 tx=0x7fabf801c490 comp rx=0 tx=0).stop 2024-04-17T10:47:22.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.414+0000 7fac14d40640 1 -- 172.21.15.83:0/2217792119 shutdown_connections 2024-04-17T10:47:22.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.414+0000 7fac14d40640 1 --2- 172.21.15.83:0/2217792119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac100b37f0 0x7fac100afbe0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:22.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.414+0000 7fac14d40640 1 --2- 172.21.15.83:0/2217792119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac10159670 0x7fac100af6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:22.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.414+0000 7fac14d40640 1 -- 172.21.15.83:0/2217792119 >> 172.21.15.83:0/2217792119 conn(0x7fac1006bcf0 msgr2=0x7fac1006c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:22.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.414+0000 7fac14d40640 1 -- 172.21.15.83:0/2217792119 shutdown_connections 2024-04-17T10:47:22.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.414+0000 7fac14d40640 1 -- 172.21.15.83:0/2217792119 wait complete. 2024-04-17T10:47:22.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.414+0000 7fac14d40640 1 Processor -- start 2024-04-17T10:47:22.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.414+0000 7fac14d40640 1 -- start start 2024-04-17T10:47:22.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.414+0000 7fac14d40640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac1006abe0 0x7fac101572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:22.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.414+0000 7fac14d40640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac100b37f0 0x7fac1014f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:22.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.414+0000 7fac14d40640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac10159670 0x7fac1014fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:22.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.414+0000 7fac14d40640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac10168c30 con 0x7fac1006abe0 2024-04-17T10:47:22.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.414+0000 7fac14d40640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac10168ab0 con 0x7fac10159670 2024-04-17T10:47:22.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.414+0000 7fac14d40640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac10168db0 con 0x7fac100b37f0 2024-04-17T10:47:22.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.414+0000 7fac0e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac1006abe0 0x7fac101572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:22.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.414+0000 7fac0e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac1006abe0 0x7fac101572e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42070/0 (socket says 172.21.15.83:42070) 2024-04-17T10:47:22.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.414+0000 7fac0e59c640 1 -- 172.21.15.83:0/273567577 learned_addr learned my addr 172.21.15.83:0/273567577 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:22.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.414+0000 7fac07fff640 1 --2- 172.21.15.83:0/273567577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac100b37f0 0x7fac1014f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:22.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.414+0000 7fac0e59c640 1 -- 172.21.15.83:0/273567577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac100b37f0 msgr2=0x7fac1014f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:22.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.414+0000 7fac0e59c640 1 --2- 172.21.15.83:0/273567577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac100b37f0 0x7fac1014f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:22.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.414+0000 7fac0e59c640 1 -- 172.21.15.83:0/273567577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac10159670 msgr2=0x7fac1014fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:22.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.414+0000 7fac0ed9d640 1 --2- 172.21.15.83:0/273567577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac10159670 0x7fac1014fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:22.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.414+0000 7fac0e59c640 1 --2- 172.21.15.83:0/273567577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac10159670 0x7fac1014fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:22.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.414+0000 7fac0e59c640 1 -- 172.21.15.83:0/273567577 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac101507a0 con 0x7fac1006abe0 2024-04-17T10:47:22.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.414+0000 7fac0ed9d640 1 --2- 172.21.15.83:0/273567577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac10159670 0x7fac1014fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:22.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.414+0000 7fac0e59c640 1 --2- 172.21.15.83:0/273567577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac1006abe0 0x7fac101572e0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fac0000d800 tx=0x7fac0000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:22.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.418+0000 7fac06ffd640 1 -- 172.21.15.83:0/273567577 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0000b6c0 con 0x7fac1006abe0 2024-04-17T10:47:22.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.418+0000 7fac06ffd640 1 -- 172.21.15.83:0/273567577 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac00019070 con 0x7fac1006abe0 2024-04-17T10:47:22.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.418+0000 7fac14d40640 1 -- 172.21.15.83:0/273567577 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac101504c0 con 0x7fac1006abe0 2024-04-17T10:47:22.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.418+0000 7fac06ffd640 1 -- 172.21.15.83:0/273567577 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0000ba60 con 0x7fac1006abe0 2024-04-17T10:47:22.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.418+0000 7fac14d40640 1 -- 172.21.15.83:0/273567577 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac101526b0 con 0x7fac1006abe0 2024-04-17T10:47:22.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.418+0000 7fac14d40640 1 -- 172.21.15.83:0/273567577 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac10074f90 con 0x7fac1006abe0 2024-04-17T10:47:22.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.418+0000 7fac06ffd640 1 -- 172.21.15.83:0/273567577 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fac0000bc00 con 0x7fac1006abe0 2024-04-17T10:47:22.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.418+0000 7fac06ffd640 1 --2- 172.21.15.83:0/273567577 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fabf0066f60 0x7fabf0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:22.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.418+0000 7fac07fff640 1 --2- 172.21.15.83:0/273567577 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fabf0066f60 0x7fabf0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:22.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.418+0000 7fac06ffd640 1 -- 172.21.15.83:0/273567577 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fac000852a0 con 0x7fac1006abe0 2024-04-17T10:47:22.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.418+0000 7fac07fff640 1 --2- 172.21.15.83:0/273567577 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fabf0066f60 0x7fabf0069420 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fabf80ad000 tx=0x7fabf8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:22.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.422+0000 7fac06ffd640 1 -- 172.21.15.83:0/273567577 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fac00056520 con 0x7fac1006abe0 2024-04-17T10:47:22.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:22.638+0000 7fac14d40640 1 -- 172.21.15.83:0/273567577 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]} v 0) v1 -- 0x7fac10166670 con 0x7fac1006abe0 2024-04-17T10:47:23.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.626+0000 7fac06ffd640 1 -- 172.21.15.83:0/273567577 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]}]=0 v404) v1 ==== 158+0+0 (secure 0 0 0) 0x7fac00055e20 con 0x7fac1006abe0 2024-04-17T10:47:23.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.630+0000 7fac14d40640 1 -- 172.21.15.83:0/273567577 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fabf0066f60 msgr2=0x7fabf0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:23.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.630+0000 7fac14d40640 1 --2- 172.21.15.83:0/273567577 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fabf0066f60 0x7fabf0069420 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fabf80ad000 tx=0x7fabf8008040 comp rx=0 tx=0).stop 2024-04-17T10:47:23.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.630+0000 7fac14d40640 1 -- 172.21.15.83:0/273567577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac1006abe0 msgr2=0x7fac101572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:23.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.630+0000 7fac14d40640 1 --2- 172.21.15.83:0/273567577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac1006abe0 0x7fac101572e0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fac0000d800 tx=0x7fac0000dcc0 comp rx=0 tx=0).stop 2024-04-17T10:47:23.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.630+0000 7fac14d40640 1 -- 172.21.15.83:0/273567577 shutdown_connections 2024-04-17T10:47:23.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.630+0000 7fac14d40640 1 --2- 172.21.15.83:0/273567577 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fabf0066f60 0x7fabf0069420 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:23.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.630+0000 7fac14d40640 1 --2- 172.21.15.83:0/273567577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac10159670 0x7fac1014fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:23.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.630+0000 7fac14d40640 1 --2- 172.21.15.83:0/273567577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac100b37f0 0x7fac1014f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:23.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.630+0000 7fac14d40640 1 --2- 172.21.15.83:0/273567577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac1006abe0 0x7fac101572e0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:23.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.630+0000 7fac14d40640 1 -- 172.21.15.83:0/273567577 >> 172.21.15.83:0/273567577 conn(0x7fac1006bcf0 msgr2=0x7fac1015d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:23.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.630+0000 7fac14d40640 1 -- 172.21.15.83:0/273567577 shutdown_connections 2024-04-17T10:47:23.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.630+0000 7fac14d40640 1 -- 172.21.15.83:0/273567577 wait complete. 2024-04-17T10:47:23.647 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-17T10:47:23.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.782+0000 7f4cbdad8640 1 Processor -- start 2024-04-17T10:47:23.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.782+0000 7f4cbdad8640 1 -- start start 2024-04-17T10:47:23.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.782+0000 7f4cbdad8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4cb815e0f0 0x7f4cb80b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:23.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.782+0000 7f4cbdad8640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4cb80ce400 0x7f4cb8161c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:23.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.782+0000 7f4cbdad8640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cb815e4d0 con 0x7f4cb80b5450 2024-04-17T10:47:23.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.782+0000 7f4cbdad8640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cb8162b40 con 0x7f4cb80ce400 2024-04-17T10:47:23.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.782+0000 7f4cbdad8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb8162580 con 0x7f4cb815e0f0 2024-04-17T10:47:23.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.782+0000 7f4cb77fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4cb815e0f0 0x7f4cb80b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:23.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.782+0000 7f4cb6ffd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4cb80ce400 0x7f4cb8161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:23.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.782+0000 7f4cb77fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4cb815e0f0 0x7f4cb80b6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:33816/0 (socket says 172.21.15.83:33816) 2024-04-17T10:47:23.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.782+0000 7f4cb77fe640 1 -- 172.21.15.83:0/1088732078 learned_addr learned my addr 172.21.15.83:0/1088732078 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:23.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.782+0000 7f4cb77fe640 1 -- 172.21.15.83:0/1088732078 >> v2:172.21.15.167:3300/0 conn(0x7f4cb80ce400 msgr2=0x7f4cb8161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:23.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.782+0000 7f4cb77fe640 1 --2- 172.21.15.83:0/1088732078 >> v2:172.21.15.167:3300/0 conn(0x7f4cb80ce400 0x7f4cb8161c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:23.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.782+0000 7f4cb77fe640 1 -- 172.21.15.83:0/1088732078 >> v1:172.21.15.167:6789/0 conn(0x7f4cb80b5450 legacy=0x7f4cb80d2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:23.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.782+0000 7f4cb77fe640 1 -- 172.21.15.83:0/1088732078 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb8162860 con 0x7f4cb815e0f0 2024-04-17T10:47:23.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.786+0000 7f4cb77fe640 1 --2- 172.21.15.83:0/1088732078 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4cb815e0f0 0x7f4cb80b6d00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4ca8009870 tx=0x7f4ca801c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffaed18a5dc649f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:23.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.786+0000 7f4cb67fc640 1 -- 172.21.15.83:0/1088732078 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca80b2070 con 0x7f4cb815e0f0 2024-04-17T10:47:23.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.786+0000 7f4cb67fc640 1 -- 172.21.15.83:0/1088732078 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ca80ad640 con 0x7f4cb815e0f0 2024-04-17T10:47:23.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.786+0000 7f4cb67fc640 1 -- 172.21.15.83:0/1088732078 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca80ada10 con 0x7f4cb815e0f0 2024-04-17T10:47:23.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.786+0000 7f4cbdad8640 1 -- 172.21.15.83:0/1088732078 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4cb815e0f0 msgr2=0x7f4cb80b6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:23.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.786+0000 7f4cbdad8640 1 --2- 172.21.15.83:0/1088732078 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4cb815e0f0 0x7f4cb80b6d00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4ca8009870 tx=0x7f4ca801c4d0 comp rx=0 tx=0).stop 2024-04-17T10:47:23.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.786+0000 7f4cbdad8640 1 -- 172.21.15.83:0/1088732078 shutdown_connections 2024-04-17T10:47:23.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.786+0000 7f4cbdad8640 1 --2- 172.21.15.83:0/1088732078 >> v2:172.21.15.167:3300/0 conn(0x7f4cb80ce400 0x7f4cb8161c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:23.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.786+0000 7f4cbdad8640 1 --2- 172.21.15.83:0/1088732078 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4cb815e0f0 0x7f4cb80b6d00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:23.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.786+0000 7f4cbdad8640 1 -- 172.21.15.83:0/1088732078 >> 172.21.15.83:0/1088732078 conn(0x7f4cb80d0450 msgr2=0x7f4cb80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:23.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.786+0000 7f4cbdad8640 1 -- 172.21.15.83:0/1088732078 shutdown_connections 2024-04-17T10:47:23.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.786+0000 7f4cbdad8640 1 -- 172.21.15.83:0/1088732078 wait complete. 2024-04-17T10:47:23.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.786+0000 7f4cbdad8640 1 Processor -- start 2024-04-17T10:47:23.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.786+0000 7f4cbdad8640 1 -- start start 2024-04-17T10:47:23.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.790+0000 7f4cbdad8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cb80b5450 0x7f4cb8153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:23.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.790+0000 7f4cbdad8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4cb80ce400 0x7f4cb8071470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:23.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.790+0000 7f4cbdad8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cb815e0f0 0x7f4cb80719b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:23.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.790+0000 7f4cbdad8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb81622d0 con 0x7f4cb815e0f0 2024-04-17T10:47:23.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.790+0000 7f4cbdad8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb8162150 con 0x7f4cb80b5450 2024-04-17T10:47:23.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.790+0000 7f4cbdad8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb816af50 con 0x7f4cb80ce400 2024-04-17T10:47:23.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.790+0000 7f4cb7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cb815e0f0 0x7f4cb80719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:23.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.790+0000 7f4cb7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cb815e0f0 0x7f4cb80719b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42078/0 (socket says 172.21.15.83:42078) 2024-04-17T10:47:23.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.790+0000 7f4cb7fff640 1 -- 172.21.15.83:0/2188992400 learned_addr learned my addr 172.21.15.83:0/2188992400 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:23.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.790+0000 7f4cb6ffd640 1 --2- 172.21.15.83:0/2188992400 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4cb80ce400 0x7f4cb8071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:23.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.790+0000 7f4cb77fe640 1 --2- 172.21.15.83:0/2188992400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cb80b5450 0x7f4cb8153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:23.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.790+0000 7f4cb7fff640 1 -- 172.21.15.83:0/2188992400 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4cb80ce400 msgr2=0x7f4cb8071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:23.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.790+0000 7f4cb7fff640 1 --2- 172.21.15.83:0/2188992400 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4cb80ce400 0x7f4cb8071470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:23.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.790+0000 7f4cb7fff640 1 -- 172.21.15.83:0/2188992400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cb80b5450 msgr2=0x7f4cb8153160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:23.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.790+0000 7f4cb7fff640 1 --2- 172.21.15.83:0/2188992400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cb80b5450 0x7f4cb8153160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:23.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.790+0000 7f4cb7fff640 1 -- 172.21.15.83:0/2188992400 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb81536a0 con 0x7f4cb815e0f0 2024-04-17T10:47:23.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.790+0000 7f4cb77fe640 1 --2- 172.21.15.83:0/2188992400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cb80b5450 0x7f4cb8153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:23.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.790+0000 7f4cb7fff640 1 --2- 172.21.15.83:0/2188992400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cb815e0f0 0x7f4cb80719b0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f4cb000c730 tx=0x7f4cb000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:23.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.790+0000 7f4c9ffff640 1 -- 172.21.15.83:0/2188992400 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0003b80 con 0x7f4cb815e0f0 2024-04-17T10:47:23.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.790+0000 7f4c9ffff640 1 -- 172.21.15.83:0/2188992400 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb0003d20 con 0x7f4cb815e0f0 2024-04-17T10:47:23.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.790+0000 7f4cbdad8640 1 -- 172.21.15.83:0/2188992400 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cb80cabd0 con 0x7f4cb815e0f0 2024-04-17T10:47:23.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.790+0000 7f4c9ffff640 1 -- 172.21.15.83:0/2188992400 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb000aa30 con 0x7f4cb815e0f0 2024-04-17T10:47:23.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.790+0000 7f4cbdad8640 1 -- 172.21.15.83:0/2188992400 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cb80cb190 con 0x7f4cb815e0f0 2024-04-17T10:47:23.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.790+0000 7f4cbdad8640 1 -- 172.21.15.83:0/2188992400 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cb80681a0 con 0x7f4cb815e0f0 2024-04-17T10:47:23.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.790+0000 7f4c9ffff640 1 -- 172.21.15.83:0/2188992400 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4cb000abd0 con 0x7f4cb815e0f0 2024-04-17T10:47:23.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.790+0000 7f4c9ffff640 1 --2- 172.21.15.83:0/2188992400 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4ca4066f60 0x7f4ca4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:23.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.790+0000 7f4c9ffff640 1 -- 172.21.15.83:0/2188992400 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4cb0084bd0 con 0x7f4cb815e0f0 2024-04-17T10:47:23.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.790+0000 7f4cb77fe640 1 --2- 172.21.15.83:0/2188992400 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4ca4066f60 0x7f4ca4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:23.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.790+0000 7f4cb77fe640 1 --2- 172.21.15.83:0/2188992400 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4ca4066f60 0x7f4ca4069420 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4ca80af000 tx=0x7f4ca8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:23.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.794+0000 7f4c9ffff640 1 -- 172.21.15.83:0/2188992400 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4cb0055ee0 con 0x7f4cb815e0f0 2024-04-17T10:47:23.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.954+0000 7f4cbdad8640 1 -- 172.21.15.83:0/2188992400 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f4cb8074f90 con 0x7f4cb815e0f0 2024-04-17T10:47:23.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.958+0000 7f4c9ffff640 1 -- 172.21.15.83:0/2188992400 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4cb00557e0 con 0x7f4cb815e0f0 2024-04-17T10:47:23.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.962+0000 7f4cbdad8640 1 -- 172.21.15.83:0/2188992400 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4ca4066f60 msgr2=0x7f4ca4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:23.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.962+0000 7f4cbdad8640 1 --2- 172.21.15.83:0/2188992400 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4ca4066f60 0x7f4ca4069420 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4ca80af000 tx=0x7f4ca8008040 comp rx=0 tx=0).stop 2024-04-17T10:47:23.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.962+0000 7f4cbdad8640 1 -- 172.21.15.83:0/2188992400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cb815e0f0 msgr2=0x7f4cb80719b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:23.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.962+0000 7f4cbdad8640 1 --2- 172.21.15.83:0/2188992400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cb815e0f0 0x7f4cb80719b0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f4cb000c730 tx=0x7f4cb000cbf0 comp rx=0 tx=0).stop 2024-04-17T10:47:23.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.962+0000 7f4cbdad8640 1 -- 172.21.15.83:0/2188992400 shutdown_connections 2024-04-17T10:47:23.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.962+0000 7f4cbdad8640 1 --2- 172.21.15.83:0/2188992400 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4ca4066f60 0x7f4ca4069420 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:23.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.962+0000 7f4cbdad8640 1 --2- 172.21.15.83:0/2188992400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cb815e0f0 0x7f4cb80719b0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:23.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.962+0000 7f4cbdad8640 1 --2- 172.21.15.83:0/2188992400 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4cb80ce400 0x7f4cb8071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:23.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.962+0000 7f4cbdad8640 1 --2- 172.21.15.83:0/2188992400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cb80b5450 0x7f4cb8153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:23.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.962+0000 7f4cbdad8640 1 -- 172.21.15.83:0/2188992400 >> 172.21.15.83:0/2188992400 conn(0x7f4cb80d0450 msgr2=0x7f4cb80d2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:23.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.962+0000 7f4cbdad8640 1 -- 172.21.15.83:0/2188992400 shutdown_connections 2024-04-17T10:47:23.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:23.962+0000 7f4cbdad8640 1 -- 172.21.15.83:0/2188992400 wait complete. 2024-04-17T10:47:23.987 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T10:47:23.987 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:47:24.038 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:47:24.083 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T10:47:24.083 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:47:24.083 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:47:24.089 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16063 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:47:24.089 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:47:24.089 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:47:24.134 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:47:24.134 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:47:24.134 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:47:24.139 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16063 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:47:24.139 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:47:24.139 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:47:24.183 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:47:24.183 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-17T10:47:24.183 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:47:24.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.318+0000 7f026fbde640 1 Processor -- start 2024-04-17T10:47:24.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.318+0000 7f026fbde640 1 -- start start 2024-04-17T10:47:24.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.318+0000 7f026fbde640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f026815ba30 0x7f0268072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:24.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.318+0000 7f026fbde640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f026815c960 0x7f02680cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:24.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.318+0000 7f026fbde640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02680cfea0 con 0x7f02680cfa10 2024-04-17T10:47:24.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.318+0000 7f026fbde640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0268071ea0 con 0x7f026815c960 2024-04-17T10:47:24.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.318+0000 7f026fbde640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f026806e320 con 0x7f026815ba30 2024-04-17T10:47:24.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.318+0000 7f026e17b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f02680cfa10 0x7f026806c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35082/0 (socket says 172.21.15.83:35082) 2024-04-17T10:47:24.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.318+0000 7f026e17b640 1 -- 172.21.15.83:0/1198125083 learned_addr learned my addr 172.21.15.83:0/1198125083 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:24.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.318+0000 7f026d179640 1 --2- 172.21.15.83:0/1198125083 >> v2:172.21.15.83:3300/0 conn(0x7f026815c960 0x7f02680cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:24.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.318+0000 7f026d97a640 1 --2- 172.21.15.83:0/1198125083 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f026815ba30 0x7f0268072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:24.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.318+0000 7f026c978640 1 -- 172.21.15.83:0/1198125083 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 774764362 0 0) 0x7f02680cfea0 con 0x7f02680cfa10 2024-04-17T10:47:24.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.318+0000 7f026c978640 1 -- 172.21.15.83:0/1198125083 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0254003530 con 0x7f02680cfa10 2024-04-17T10:47:24.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.318+0000 7f026d179640 1 -- 172.21.15.83:0/1198125083 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f026815ba30 msgr2=0x7f0268072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:24.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.318+0000 7f026d179640 1 --2- 172.21.15.83:0/1198125083 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f026815ba30 0x7f0268072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:24.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.318+0000 7f026d179640 1 -- 172.21.15.83:0/1198125083 >> v1:172.21.15.83:6789/0 conn(0x7f02680cfa10 legacy=0x7f026806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:24.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.318+0000 7f026d179640 1 -- 172.21.15.83:0/1198125083 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f026806e040 con 0x7f026815c960 2024-04-17T10:47:24.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f026d179640 1 --2- 172.21.15.83:0/1198125083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f026815c960 0x7f02680cf4d0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f02580097c0 tx=0x7f025801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab392037ba25954 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:24.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f026c978640 1 -- 172.21.15.83:0/1198125083 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02580b0070 con 0x7f026815c960 2024-04-17T10:47:24.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f026c978640 1 -- 172.21.15.83:0/1198125083 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02580ab600 con 0x7f026815c960 2024-04-17T10:47:24.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f026c978640 1 -- 172.21.15.83:0/1198125083 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02580ab9b0 con 0x7f026815c960 2024-04-17T10:47:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f026fbde640 1 -- 172.21.15.83:0/1198125083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f026815c960 msgr2=0x7f02680cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f026fbde640 1 --2- 172.21.15.83:0/1198125083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f026815c960 0x7f02680cf4d0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f02580097c0 tx=0x7f025801c410 comp rx=0 tx=0).stop 2024-04-17T10:47:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f026fbde640 1 -- 172.21.15.83:0/1198125083 shutdown_connections 2024-04-17T10:47:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f026fbde640 1 --2- 172.21.15.83:0/1198125083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f026815c960 0x7f02680cf4d0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f026fbde640 1 --2- 172.21.15.83:0/1198125083 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f026815ba30 0x7f0268072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f026fbde640 1 -- 172.21.15.83:0/1198125083 >> 172.21.15.83:0/1198125083 conn(0x7f026806abe0 msgr2=0x7f026806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f026fbde640 1 -- 172.21.15.83:0/1198125083 shutdown_connections 2024-04-17T10:47:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f026fbde640 1 -- 172.21.15.83:0/1198125083 wait complete. 2024-04-17T10:47:24.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f026fbde640 1 Processor -- start 2024-04-17T10:47:24.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f026fbde640 1 -- start start 2024-04-17T10:47:24.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f026fbde640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02680cfa10 0x7f026814a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:24.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f026fbde640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f026815ba30 0x7f026814dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:24.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f026fbde640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f026815c960 0x7f026815ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:24.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f026fbde640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026806dca0 con 0x7f02680cfa10 2024-04-17T10:47:24.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f026fbde640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02680aeae0 con 0x7f026815c960 2024-04-17T10:47:24.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f026fbde640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f026806de20 con 0x7f026815ba30 2024-04-17T10:47:24.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f026d97a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02680cfa10 0x7f026814a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:24.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f026d97a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02680cfa10 0x7f026814a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42090/0 (socket says 172.21.15.83:42090) 2024-04-17T10:47:24.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f026d97a640 1 -- 172.21.15.83:0/1319758857 learned_addr learned my addr 172.21.15.83:0/1319758857 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:24.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f026d97a640 1 -- 172.21.15.83:0/1319758857 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f026815ba30 msgr2=0x7f026814dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:24.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f026d97a640 1 --2- 172.21.15.83:0/1319758857 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f026815ba30 0x7f026814dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:24.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f026d97a640 1 -- 172.21.15.83:0/1319758857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f026815c960 msgr2=0x7f026815ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:24.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f026d97a640 1 --2- 172.21.15.83:0/1319758857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f026815c960 0x7f026815ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:24.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f026d97a640 1 -- 172.21.15.83:0/1319758857 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0268155b30 con 0x7f02680cfa10 2024-04-17T10:47:24.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f026d97a640 1 --2- 172.21.15.83:0/1319758857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02680cfa10 0x7f026814a4d0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f026400d620 tx=0x7f026400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:24.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f025e7fc640 1 -- 172.21.15.83:0/1319758857 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026400b620 con 0x7f02680cfa10 2024-04-17T10:47:24.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f025e7fc640 1 -- 172.21.15.83:0/1319758857 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0264019070 con 0x7f02680cfa10 2024-04-17T10:47:24.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f026fbde640 1 -- 172.21.15.83:0/1319758857 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f026814d0d0 con 0x7f02680cfa10 2024-04-17T10:47:24.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.322+0000 7f025e7fc640 1 -- 172.21.15.83:0/1319758857 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026400b9e0 con 0x7f02680cfa10 2024-04-17T10:47:24.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.326+0000 7f026fbde640 1 -- 172.21.15.83:0/1319758857 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f026814bcc0 con 0x7f02680cfa10 2024-04-17T10:47:24.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.326+0000 7f026fbde640 1 -- 172.21.15.83:0/1319758857 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0268074f90 con 0x7f02680cfa10 2024-04-17T10:47:24.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.326+0000 7f025e7fc640 1 -- 172.21.15.83:0/1319758857 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f026400bb80 con 0x7f02680cfa10 2024-04-17T10:47:24.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.326+0000 7f025e7fc640 1 --2- 172.21.15.83:0/1319758857 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f024c066f60 0x7f024c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:24.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.326+0000 7f025e7fc640 1 -- 172.21.15.83:0/1319758857 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f02640850b0 con 0x7f02680cfa10 2024-04-17T10:47:24.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.326+0000 7f026d179640 1 --2- 172.21.15.83:0/1319758857 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f024c066f60 0x7f024c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:24.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.326+0000 7f026d179640 1 --2- 172.21.15.83:0/1319758857 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f024c066f60 0x7f024c069420 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f02580ad000 tx=0x7f02580046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:24.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.330+0000 7f025e7fc640 1 -- 172.21.15.83:0/1319758857 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0264056450 con 0x7f02680cfa10 2024-04-17T10:47:24.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.526+0000 7f026fbde640 1 -- 172.21.15.83:0/1319758857 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02680712f0 con 0x7f02680cfa10 2024-04-17T10:47:24.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.526+0000 7f025e7fc640 1 -- 172.21.15.83:0/1319758857 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f0264055d50 con 0x7f02680cfa10 2024-04-17T10:47:24.522 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:47:24.522 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":58,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5429,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6834/3936640219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3936640219},{"type":"v1","addr":"172.21.15.167:6834","nonce":3936640219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":5998,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4152882285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6833","nonce":4152882285},{"type":"v1","addr":"172.21.15.167:6835","nonce":4152882285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1562645860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1562645860},{"type":"v1","addr":"172.21.15.83:6837","nonce":1562645860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":7962,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/1293949657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1293949657},{"type":"v1","addr":"172.21.15.83:6836","nonce":1293949657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[]} 2024-04-17T10:47:24.522 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 58 2024-04-17T10:47:24.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.526+0000 7f026fbde640 1 -- 172.21.15.83:0/1319758857 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f024c066f60 msgr2=0x7f024c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:24.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.526+0000 7f026fbde640 1 --2- 172.21.15.83:0/1319758857 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f024c066f60 0x7f024c069420 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f02580ad000 tx=0x7f02580046d0 comp rx=0 tx=0).stop 2024-04-17T10:47:24.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.526+0000 7f026fbde640 1 -- 172.21.15.83:0/1319758857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02680cfa10 msgr2=0x7f026814a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:24.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.526+0000 7f026fbde640 1 --2- 172.21.15.83:0/1319758857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02680cfa10 0x7f026814a4d0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f026400d620 tx=0x7f026400dae0 comp rx=0 tx=0).stop 2024-04-17T10:47:24.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.526+0000 7f026fbde640 1 -- 172.21.15.83:0/1319758857 shutdown_connections 2024-04-17T10:47:24.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.526+0000 7f026fbde640 1 --2- 172.21.15.83:0/1319758857 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f024c066f60 0x7f024c069420 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:24.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.526+0000 7f026fbde640 1 --2- 172.21.15.83:0/1319758857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f026815c960 0x7f026815ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:24.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.526+0000 7f026fbde640 1 --2- 172.21.15.83:0/1319758857 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f026815ba30 0x7f026814dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:24.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.526+0000 7f026fbde640 1 --2- 172.21.15.83:0/1319758857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02680cfa10 0x7f026814a4d0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:24.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.526+0000 7f026fbde640 1 -- 172.21.15.83:0/1319758857 >> 172.21.15.83:0/1319758857 conn(0x7f026806abe0 msgr2=0x7f026806c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:24.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.526+0000 7f026fbde640 1 -- 172.21.15.83:0/1319758857 shutdown_connections 2024-04-17T10:47:24.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.526+0000 7f026fbde640 1 -- 172.21.15.83:0/1319758857 wait complete. 2024-04-17T10:47:24.542 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-17T10:47:24.542 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-17T10:47:24.542 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-17T10:47:24.542 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:47:24.542 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-17T10:47:24.542 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-17T10:47:24.542 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-17T10:47:24.543 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:47:24.543 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-17T10:47:24.543 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-17T10:47:24.543 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-17T10:47:24.543 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:47:24.543 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-17T10:47:24.543 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-17T10:47:24.544 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-17T10:47:24.544 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:47:24.553 INFO:tasks.ceph.mds.a:Stopped 2024-04-17T10:47:24.554 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-17T10:47:24.555 INFO:tasks.ceph.mds.c:Stopped 2024-04-17T10:47:24.555 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-17T10:47:24.556 INFO:tasks.ceph.mds.b:Stopped 2024-04-17T10:47:24.556 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-17T10:47:24.557 INFO:tasks.ceph.mds.d:Stopped 2024-04-17T10:47:24.557 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-17T10:47:24.599 INFO:tasks.ceph.mds.b:Started 2024-04-17T10:47:24.599 DEBUG:teuthology.parallel:result is None 2024-04-17T10:47:24.600 INFO:tasks.ceph.mds.d:Started 2024-04-17T10:47:24.600 DEBUG:teuthology.parallel:result is None 2024-04-17T10:47:24.603 INFO:tasks.ceph.mds.a:Started 2024-04-17T10:47:24.604 DEBUG:teuthology.parallel:result is None 2024-04-17T10:47:24.604 INFO:tasks.ceph.mds.c:Started 2024-04-17T10:47:24.605 DEBUG:teuthology.parallel:result is None 2024-04-17T10:47:24.605 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-17T10:47:24.667 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-04-17T10:47:24.667 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-04-17T10:47:24.672 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2024-04-17T10:47:24.674 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2024-04-17T10:47:24.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.726+0000 7f38e70ab640 1 Processor -- start 2024-04-17T10:47:24.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.726+0000 7f38e70ab640 1 -- start start 2024-04-17T10:47:24.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.726+0000 7f38e70ab640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f38e015e0f0 0x7f38e00b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:24.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.726+0000 7f38e70ab640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38e00b5450 0x7f38e00b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:24.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.726+0000 7f38e70ab640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e015e4d0 con 0x7f38e00ce400 2024-04-17T10:47:24.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.726+0000 7f38e70ab640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38e0162b40 con 0x7f38e015e0f0 2024-04-17T10:47:24.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.726+0000 7f38e70ab640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e0162320 con 0x7f38e00b5450 2024-04-17T10:47:24.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.726+0000 7f38e4e47640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f38e015e0f0 0x7f38e00b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:24.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.730+0000 7f38e4e47640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f38e015e0f0 0x7f38e00b6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42108/0 (socket says 172.21.15.83:42108) 2024-04-17T10:47:24.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.730+0000 7f38e4e47640 1 -- 172.21.15.83:0/2844367102 learned_addr learned my addr 172.21.15.83:0/2844367102 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:24.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.730+0000 7f38e5648640 1 --2- 172.21.15.83:0/2844367102 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38e00b5450 0x7f38e00b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:24.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.730+0000 7f38e4e47640 1 -- 172.21.15.83:0/2844367102 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38e00b5450 msgr2=0x7f38e00b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:24.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.730+0000 7f38e4e47640 1 --2- 172.21.15.83:0/2844367102 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38e00b5450 0x7f38e00b58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:24.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.730+0000 7f38e4e47640 1 -- 172.21.15.83:0/2844367102 >> v1:172.21.15.167:6789/0 conn(0x7f38e00ce400 legacy=0x7f38e0161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:24.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.730+0000 7f38e4e47640 1 -- 172.21.15.83:0/2844367102 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e0162580 con 0x7f38e015e0f0 2024-04-17T10:47:24.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.730+0000 7f38e5648640 1 --2- 172.21.15.83:0/2844367102 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38e00b5450 0x7f38e00b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:24.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.730+0000 7f38e4e47640 1 --2- 172.21.15.83:0/2844367102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e015e0f0 0x7f38e00b6d00 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f38d00098a0 tx=0x7f38d001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd0213ed32d8d8e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:24.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.730+0000 7f38df7fe640 1 -- 172.21.15.83:0/2844367102 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d00b0070 con 0x7f38e015e0f0 2024-04-17T10:47:24.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.730+0000 7f38df7fe640 1 -- 172.21.15.83:0/2844367102 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d00ab600 con 0x7f38e015e0f0 2024-04-17T10:47:24.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.730+0000 7f38df7fe640 1 -- 172.21.15.83:0/2844367102 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d00ab9d0 con 0x7f38e015e0f0 2024-04-17T10:47:24.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.730+0000 7f38e70ab640 1 -- 172.21.15.83:0/2844367102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e015e0f0 msgr2=0x7f38e00b6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:24.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.730+0000 7f38e70ab640 1 --2- 172.21.15.83:0/2844367102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e015e0f0 0x7f38e00b6d00 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f38d00098a0 tx=0x7f38d001c470 comp rx=0 tx=0).stop 2024-04-17T10:47:24.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.730+0000 7f38e70ab640 1 -- 172.21.15.83:0/2844367102 shutdown_connections 2024-04-17T10:47:24.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.730+0000 7f38e70ab640 1 --2- 172.21.15.83:0/2844367102 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38e00b5450 0x7f38e00b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:24.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.730+0000 7f38e70ab640 1 --2- 172.21.15.83:0/2844367102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e015e0f0 0x7f38e00b6d00 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:24.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.730+0000 7f38e70ab640 1 -- 172.21.15.83:0/2844367102 >> 172.21.15.83:0/2844367102 conn(0x7f38e00d0450 msgr2=0x7f38e00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:24.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.730+0000 7f38e70ab640 1 -- 172.21.15.83:0/2844367102 shutdown_connections 2024-04-17T10:47:24.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.730+0000 7f38e70ab640 1 -- 172.21.15.83:0/2844367102 wait complete. 2024-04-17T10:47:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.734+0000 7f38e70ab640 1 Processor -- start 2024-04-17T10:47:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.734+0000 7f38e70ab640 1 -- start start 2024-04-17T10:47:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.734+0000 7f38e70ab640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38e00b5450 0x7f38e014e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.734+0000 7f38e70ab640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38e00ce400 0x7f38e0154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.734+0000 7f38e70ab640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e015e0f0 0x7f38e014c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.734+0000 7f38e70ab640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e00d2ed0 con 0x7f38e015e0f0 2024-04-17T10:47:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.734+0000 7f38e70ab640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e00d2d50 con 0x7f38e00b5450 2024-04-17T10:47:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.734+0000 7f38e70ab640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e00d3050 con 0x7f38e00ce400 2024-04-17T10:47:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.734+0000 7f38e5648640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e015e0f0 0x7f38e014c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.734+0000 7f38e5648640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e015e0f0 0x7f38e014c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42118/0 (socket says 172.21.15.83:42118) 2024-04-17T10:47:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.734+0000 7f38e5648640 1 -- 172.21.15.83:0/2654299237 learned_addr learned my addr 172.21.15.83:0/2654299237 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.734+0000 7f38e4e47640 1 --2- 172.21.15.83:0/2654299237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38e00b5450 0x7f38e014e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.734+0000 7f38dffff640 1 --2- 172.21.15.83:0/2654299237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38e00ce400 0x7f38e0154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.734+0000 7f38e5648640 1 -- 172.21.15.83:0/2654299237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38e00ce400 msgr2=0x7f38e0154260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.734+0000 7f38e5648640 1 --2- 172.21.15.83:0/2654299237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38e00ce400 0x7f38e0154260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.734+0000 7f38e5648640 1 -- 172.21.15.83:0/2654299237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38e00b5450 msgr2=0x7f38e014e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.734+0000 7f38e5648640 1 --2- 172.21.15.83:0/2654299237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38e00b5450 0x7f38e014e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.734+0000 7f38e5648640 1 -- 172.21.15.83:0/2654299237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e01547a0 con 0x7f38e015e0f0 2024-04-17T10:47:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.734+0000 7f38e4e47640 1 --2- 172.21.15.83:0/2654299237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38e00b5450 0x7f38e014e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:47:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.734+0000 7f38e5648640 1 --2- 172.21.15.83:0/2654299237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e015e0f0 0x7f38e014c640 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f38d800cef0 tx=0x7f38d800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:24.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.734+0000 7f38dd7fa640 1 -- 172.21.15.83:0/2654299237 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d801a070 con 0x7f38e015e0f0 2024-04-17T10:47:24.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.734+0000 7f38dd7fa640 1 -- 172.21.15.83:0/2654299237 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d800d860 con 0x7f38e015e0f0 2024-04-17T10:47:24.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.734+0000 7f38e70ab640 1 -- 172.21.15.83:0/2654299237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e00cb5e0 con 0x7f38e015e0f0 2024-04-17T10:47:24.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.734+0000 7f38dd7fa640 1 -- 172.21.15.83:0/2654299237 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d800dc50 con 0x7f38e015e0f0 2024-04-17T10:47:24.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.734+0000 7f38e70ab640 1 -- 172.21.15.83:0/2654299237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e0163180 con 0x7f38e015e0f0 2024-04-17T10:47:24.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.734+0000 7f38e70ab640 1 -- 172.21.15.83:0/2654299237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e00681a0 con 0x7f38e015e0f0 2024-04-17T10:47:24.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.734+0000 7f38dd7fa640 1 -- 172.21.15.83:0/2654299237 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f38d801d070 con 0x7f38e015e0f0 2024-04-17T10:47:24.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.738+0000 7f38dd7fa640 1 --2- 172.21.15.83:0/2654299237 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f38cc066f60 0x7f38cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:24.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.738+0000 7f38e4e47640 1 --2- 172.21.15.83:0/2654299237 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f38cc066f60 0x7f38cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:24.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.738+0000 7f38dd7fa640 1 -- 172.21.15.83:0/2654299237 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f38d8085550 con 0x7f38e015e0f0 2024-04-17T10:47:24.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.738+0000 7f38e4e47640 1 --2- 172.21.15.83:0/2654299237 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f38cc066f60 0x7f38cc069420 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f38d00ad000 tx=0x7f38d00a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:24.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.742+0000 7f38dd7fa640 1 -- 172.21.15.83:0/2654299237 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f38e01547a0 con 0x7f38e015e0f0 2024-04-17T10:47:24.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:24.914+0000 7f38e70ab640 1 -- 172.21.15.83:0/2654299237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f38e0074f90 con 0x7f38e015e0f0 2024-04-17T10:47:25.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:25.994+0000 7f38dd7fa640 1 -- 172.21.15.83:0/2654299237 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v99) v1 ==== 97+0+0 (secure 0 0 0) 0x7f38d80569b0 con 0x7f38e015e0f0 2024-04-17T10:47:25.990 INFO:teuthology.orchestra.run.smithi083.stderr: removed all blocklist entries 2024-04-17T10:47:25.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:25.998+0000 7f38e70ab640 1 -- 172.21.15.83:0/2654299237 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f38cc066f60 msgr2=0x7f38cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:25.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:25.998+0000 7f38e70ab640 1 --2- 172.21.15.83:0/2654299237 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f38cc066f60 0x7f38cc069420 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f38d00ad000 tx=0x7f38d00a6000 comp rx=0 tx=0).stop 2024-04-17T10:47:25.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:25.998+0000 7f38e70ab640 1 -- 172.21.15.83:0/2654299237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e015e0f0 msgr2=0x7f38e014c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:25.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:25.998+0000 7f38e70ab640 1 --2- 172.21.15.83:0/2654299237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e015e0f0 0x7f38e014c640 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f38d800cef0 tx=0x7f38d800a710 comp rx=0 tx=0).stop 2024-04-17T10:47:25.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:25.998+0000 7f38e70ab640 1 -- 172.21.15.83:0/2654299237 shutdown_connections 2024-04-17T10:47:25.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:25.998+0000 7f38e70ab640 1 --2- 172.21.15.83:0/2654299237 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f38cc066f60 0x7f38cc069420 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:25.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:25.998+0000 7f38e70ab640 1 --2- 172.21.15.83:0/2654299237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38e015e0f0 0x7f38e014c640 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:25.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:25.998+0000 7f38e70ab640 1 --2- 172.21.15.83:0/2654299237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38e00ce400 0x7f38e0154260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:25.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:25.998+0000 7f38e70ab640 1 --2- 172.21.15.83:0/2654299237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38e00b5450 0x7f38e014e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:25.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:25.998+0000 7f38e70ab640 1 -- 172.21.15.83:0/2654299237 >> 172.21.15.83:0/2654299237 conn(0x7f38e00d0450 msgr2=0x7f38e00ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:25.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:25.998+0000 7f38e70ab640 1 -- 172.21.15.83:0/2654299237 shutdown_connections 2024-04-17T10:47:25.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:25.998+0000 7f38e70ab640 1 -- 172.21.15.83:0/2654299237 wait complete. 2024-04-17T10:47:26.015 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-17T10:47:26.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.134+0000 7f385a05b640 1 Processor -- start 2024-04-17T10:47:26.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.134+0000 7f385a05b640 1 -- start start 2024-04-17T10:47:26.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.134+0000 7f385a05b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f385415dfa0 0x7f38540b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:26.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.134+0000 7f385a05b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f38540ce980 0x7f38540b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:26.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.134+0000 7f385a05b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3854162a00 con 0x7f38540cee00 2024-04-17T10:47:26.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.134+0000 7f385a05b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3854161e80 con 0x7f38540ce980 2024-04-17T10:47:26.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.134+0000 7f385a05b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3854162160 con 0x7f385415dfa0 2024-04-17T10:47:26.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.134+0000 7f3852ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f38540ce980 0x7f38540b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:26.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.134+0000 7f3853fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f38540cee00 0x7f38540d1d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35112/0 (socket says 172.21.15.83:35112) 2024-04-17T10:47:26.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.134+0000 7f3852ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f38540ce980 0x7f38540b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42126/0 (socket says 172.21.15.83:42126) 2024-04-17T10:47:26.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.134+0000 7f3853fff640 1 -- 172.21.15.83:0/2604143790 learned_addr learned my addr 172.21.15.83:0/2604143790 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:26.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.134+0000 7f38537fe640 1 --2- 172.21.15.83:0/2604143790 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f385415dfa0 0x7f38540b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:26.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.134+0000 7f3852ffd640 1 -- 172.21.15.83:0/2604143790 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f385415dfa0 msgr2=0x7f38540b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:26.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.134+0000 7f3852ffd640 1 --2- 172.21.15.83:0/2604143790 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f385415dfa0 0x7f38540b6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:26.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.134+0000 7f3852ffd640 1 -- 172.21.15.83:0/2604143790 >> v1:172.21.15.83:6789/0 conn(0x7f38540cee00 legacy=0x7f38540d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:26.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.134+0000 7f3852ffd640 1 -- 172.21.15.83:0/2604143790 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3854162440 con 0x7f38540ce980 2024-04-17T10:47:26.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.138+0000 7f3852ffd640 1 --2- 172.21.15.83:0/2604143790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38540ce980 0x7f38540b5430 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f38480097c0 tx=0x7f384801c2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1ff7cea90d4dc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:26.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.138+0000 7f38527fc640 1 -- 172.21.15.83:0/2604143790 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38480b0070 con 0x7f38540ce980 2024-04-17T10:47:26.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.138+0000 7f38527fc640 1 -- 172.21.15.83:0/2604143790 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38480ab600 con 0x7f38540ce980 2024-04-17T10:47:26.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.138+0000 7f38527fc640 1 -- 172.21.15.83:0/2604143790 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38480ab9d0 con 0x7f38540ce980 2024-04-17T10:47:26.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.138+0000 7f385a05b640 1 -- 172.21.15.83:0/2604143790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38540ce980 msgr2=0x7f38540b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:26.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.138+0000 7f385a05b640 1 --2- 172.21.15.83:0/2604143790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38540ce980 0x7f38540b5430 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f38480097c0 tx=0x7f384801c2a0 comp rx=0 tx=0).stop 2024-04-17T10:47:26.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.138+0000 7f385a05b640 1 -- 172.21.15.83:0/2604143790 shutdown_connections 2024-04-17T10:47:26.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.138+0000 7f385a05b640 1 --2- 172.21.15.83:0/2604143790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38540ce980 0x7f38540b5430 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:26.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.138+0000 7f385a05b640 1 --2- 172.21.15.83:0/2604143790 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f385415dfa0 0x7f38540b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:26.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.138+0000 7f385a05b640 1 -- 172.21.15.83:0/2604143790 >> 172.21.15.83:0/2604143790 conn(0x7f38540d0600 msgr2=0x7f38540d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:26.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.138+0000 7f385a05b640 1 -- 172.21.15.83:0/2604143790 shutdown_connections 2024-04-17T10:47:26.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.138+0000 7f385a05b640 1 -- 172.21.15.83:0/2604143790 wait complete. 2024-04-17T10:47:26.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.138+0000 7f385a05b640 1 Processor -- start 2024-04-17T10:47:26.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.138+0000 7f385a05b640 1 -- start start 2024-04-17T10:47:26.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.138+0000 7f385a05b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38540ce980 0x7f3854160d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:26.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.138+0000 7f385a05b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38540cee00 0x7f38540cc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:26.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.138+0000 7f385a05b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f385415dfa0 0x7f38540c9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:26.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.138+0000 7f385a05b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38540b7160 con 0x7f38540cee00 2024-04-17T10:47:26.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.138+0000 7f385a05b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385415e380 con 0x7f385415dfa0 2024-04-17T10:47:26.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.138+0000 7f385a05b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3854161b10 con 0x7f38540ce980 2024-04-17T10:47:26.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.138+0000 7f3852ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38540cee00 0x7f38540cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:26.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.138+0000 7f3852ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38540cee00 0x7f38540cc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42128/0 (socket says 172.21.15.83:42128) 2024-04-17T10:47:26.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.138+0000 7f3852ffd640 1 -- 172.21.15.83:0/849243439 learned_addr learned my addr 172.21.15.83:0/849243439 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:26.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.138+0000 7f3853fff640 1 --2- 172.21.15.83:0/849243439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f385415dfa0 0x7f38540c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:26.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.138+0000 7f38537fe640 1 --2- 172.21.15.83:0/849243439 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38540ce980 0x7f3854160d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:26.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.138+0000 7f3852ffd640 1 -- 172.21.15.83:0/849243439 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38540ce980 msgr2=0x7f3854160d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:26.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.138+0000 7f3852ffd640 1 --2- 172.21.15.83:0/849243439 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38540ce980 0x7f3854160d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:26.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.138+0000 7f3852ffd640 1 -- 172.21.15.83:0/849243439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f385415dfa0 msgr2=0x7f38540c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:26.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.138+0000 7f3852ffd640 1 --2- 172.21.15.83:0/849243439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f385415dfa0 0x7f38540c9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:26.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.138+0000 7f3852ffd640 1 -- 172.21.15.83:0/849243439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38540b64f0 con 0x7f38540cee00 2024-04-17T10:47:26.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.138+0000 7f3853fff640 1 --2- 172.21.15.83:0/849243439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f385415dfa0 0x7f38540c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:26.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.138+0000 7f3852ffd640 1 --2- 172.21.15.83:0/849243439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38540cee00 0x7f38540cc170 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f38480ae040 tx=0x7f3848019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:26.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.142+0000 7f383bfff640 1 -- 172.21.15.83:0/849243439 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38480b8020 con 0x7f38540cee00 2024-04-17T10:47:26.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.142+0000 7f383bfff640 1 -- 172.21.15.83:0/849243439 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38480aa7d0 con 0x7f38540cee00 2024-04-17T10:47:26.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.142+0000 7f385a05b640 1 -- 172.21.15.83:0/849243439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3854161370 con 0x7f38540cee00 2024-04-17T10:47:26.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.142+0000 7f383bfff640 1 -- 172.21.15.83:0/849243439 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38480b0040 con 0x7f38540cee00 2024-04-17T10:47:26.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.142+0000 7f385a05b640 1 -- 172.21.15.83:0/849243439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38540c9140 con 0x7f38540cee00 2024-04-17T10:47:26.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.142+0000 7f385a05b640 1 -- 172.21.15.83:0/849243439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3854074f90 con 0x7f38540cee00 2024-04-17T10:47:26.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.142+0000 7f383bfff640 1 -- 172.21.15.83:0/849243439 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f38480bb020 con 0x7f38540cee00 2024-04-17T10:47:26.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.142+0000 7f383bfff640 1 --2- 172.21.15.83:0/849243439 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3840066f60 0x7f3840069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:26.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.142+0000 7f383bfff640 1 -- 172.21.15.83:0/849243439 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f384811eda0 con 0x7f38540cee00 2024-04-17T10:47:26.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.142+0000 7f38537fe640 1 --2- 172.21.15.83:0/849243439 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3840066f60 0x7f3840069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:26.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.142+0000 7f38537fe640 1 --2- 172.21.15.83:0/849243439 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3840066f60 0x7f3840069420 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f3844009790 tx=0x7f3844008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:26.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.146+0000 7f383bfff640 1 -- 172.21.15.83:0/849243439 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f38480c2030 con 0x7f38540cee00 2024-04-17T10:47:26.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.346+0000 7f385a05b640 1 -- 172.21.15.83:0/849243439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f385415c930 con 0x7f38540cee00 2024-04-17T10:47:26.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.346+0000 7f383bfff640 1 -- 172.21.15.83:0/849243439 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3848015fb0 con 0x7f38540cee00 2024-04-17T10:47:26.343 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:47:26.343 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:47:26.343 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_dump": [ 2024-04-17T10:47:26.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:47:26.344 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.a", 2024-04-17T10:47:26.344 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mfpxsNxAANpL7rcWevXwd2KMftSQnEQ==", 2024-04-17T10:47:26.344 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:47:26.344 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:47:26.344 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:47:26.344 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:47:26.345 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:47:26.345 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:47:26.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:47:26.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:47:26.345 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.b", 2024-04-17T10:47:26.345 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mXlI9ChAAYdcrGVmtYdtWWgCLVmpTTg==", 2024-04-17T10:47:26.345 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:47:26.346 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:47:26.346 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:47:26.346 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:47:26.346 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:47:26.346 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:47:26.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:47:26.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:47:26.347 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.c", 2024-04-17T10:47:26.347 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mupo3BBAAyKH801vpFLXYs9mfzSlT0w==", 2024-04-17T10:47:26.347 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:47:26.347 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:47:26.347 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:47:26.348 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:47:26.348 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:47:26.348 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:47:26.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:47:26.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:47:26.348 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.d", 2024-04-17T10:47:26.348 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mbm++EhAAIBMOtQoD+kPeMqR6gK5ctw==", 2024-04-17T10:47:26.348 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:47:26.348 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:47:26.348 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:47:26.348 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:47:26.348 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:47:26.349 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:47:26.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:47:26.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:47:26.349 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.0", 2024-04-17T10:47:26.349 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBXph9m1HWhIRAAOq+9nGq7OGonUCE2aeiAzQ==", 2024-04-17T10:47:26.349 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:47:26.349 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:47:26.349 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:47:26.349 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:47:26.349 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:47:26.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:47:26.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:47:26.350 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.1", 2024-04-17T10:47:26.350 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBaph9mlMmtJRAAXHLMjXfm/+WjIFURLkbAfA==", 2024-04-17T10:47:26.350 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:47:26.350 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:47:26.350 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:47:26.350 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:47:26.350 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:47:26.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:47:26.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:47:26.350 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.2", 2024-04-17T10:47:26.350 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBdph9mBsNPLRAAtR5hwGdDkjLDLS19QUDDYQ==", 2024-04-17T10:47:26.351 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:47:26.351 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:47:26.351 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:47:26.351 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:47:26.351 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:47:26.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:47:26.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:47:26.351 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.3", 2024-04-17T10:47:26.351 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBgph9mZ4EwNRAAIyL7e0EK1523dPXmbOOkzw==", 2024-04-17T10:47:26.351 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:47:26.351 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:47:26.351 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:47:26.352 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:47:26.352 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:47:26.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:47:26.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:47:26.352 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.4", 2024-04-17T10:47:26.352 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBmph9mphAsMhAAtF4uV28Nb66PNjuugB12Hw==", 2024-04-17T10:47:26.352 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:47:26.352 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:47:26.352 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:47:26.352 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:47:26.352 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:47:26.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:47:26.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:47:26.353 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.5", 2024-04-17T10:47:26.353 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBpph9mHJQNNxAA72eOFYFbhakQac1JKRNjzw==", 2024-04-17T10:47:26.353 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:47:26.353 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:47:26.353 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:47:26.353 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:47:26.353 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:47:26.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:47:26.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:47:26.353 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.6", 2024-04-17T10:47:26.353 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBtph9mb6RKAxAAoVX6Cp7l7j3Lm1ES2Sq+fQ==", 2024-04-17T10:47:26.353 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:47:26.353 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:47:26.353 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:47:26.354 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:47:26.354 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:47:26.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:47:26.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:47:26.354 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.7", 2024-04-17T10:47:26.354 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBwph9mdrdrBxAApDwjml2ahsLbRol4uL/BxA==", 2024-04-17T10:47:26.354 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:47:26.354 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:47:26.354 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:47:26.354 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:47:26.354 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:47:26.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:47:26.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:47:26.355 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.0", 2024-04-17T10:47:26.355 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mHGmXGBAAZF+JiAG9YrFtbW+dCy2IEQ==", 2024-04-17T10:47:26.355 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:47:26.355 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:47:26.355 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow r", 2024-04-17T10:47:26.355 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-17T10:47:26.355 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:47:26.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:47:26.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:47:26.355 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.1", 2024-04-17T10:47:26.355 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mB7wgIBAABdavAo/uzVuSjfkkcYtqJg==", 2024-04-17T10:47:26.355 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:47:26.355 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:47:26.356 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-17T10:47:26.356 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-17T10:47:26.356 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-17T10:47:26.356 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:47:26.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:47:26.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:47:26.356 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.2", 2024-04-17T10:47:26.357 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9m+wKgHRAAe8IUFx34ItLpsaDqSYyEgA==", 2024-04-17T10:47:26.357 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:47:26.357 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:47:26.357 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-17T10:47:26.357 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-17T10:47:26.357 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-17T10:47:26.358 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:47:26.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:47:26.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:47:26.358 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.3", 2024-04-17T10:47:26.358 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mfJ0yJRAAHH7rRh/BUIujTT04AF2l0w==", 2024-04-17T10:47:26.358 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:47:26.359 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:47:26.359 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-17T10:47:26.359 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-17T10:47:26.359 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-17T10:47:26.359 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:47:26.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:47:26.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:47:26.360 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.admin", 2024-04-17T10:47:26.360 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mjJ3GIBAA/2qaCbRiLJ5X1cv3ZGdfoA==", 2024-04-17T10:47:26.360 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:47:26.360 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-17T10:47:26.360 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:47:26.360 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:47:26.360 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:47:26.361 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:47:26.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:47:26.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:47:26.361 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.x", 2024-04-17T10:47:26.361 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mMFptNBAA/G0zEfMZTCy9Iox+MwHG3Q==", 2024-04-17T10:47:26.361 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:47:26.361 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-17T10:47:26.361 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-04-17T10:47:26.361 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:47:26.362 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:47:26.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:47:26.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:47:26.362 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.y", 2024-04-17T10:47:26.362 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mjFbRMRAAjY2d8POBzp8hEp9GSO0ylQ==", 2024-04-17T10:47:26.362 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:47:26.362 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-17T10:47:26.362 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-04-17T10:47:26.363 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:47:26.363 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:47:26.363 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:47:26.363 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-17T10:47:26.363 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:47:26.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.346+0000 7f385a05b640 1 -- 172.21.15.83:0/849243439 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3840066f60 msgr2=0x7f3840069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:26.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.346+0000 7f385a05b640 1 --2- 172.21.15.83:0/849243439 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3840066f60 0x7f3840069420 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f3844009790 tx=0x7f3844008040 comp rx=0 tx=0).stop 2024-04-17T10:47:26.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.346+0000 7f385a05b640 1 -- 172.21.15.83:0/849243439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38540cee00 msgr2=0x7f38540cc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:26.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.346+0000 7f385a05b640 1 --2- 172.21.15.83:0/849243439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38540cee00 0x7f38540cc170 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f38480ae040 tx=0x7f3848019040 comp rx=0 tx=0).stop 2024-04-17T10:47:26.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.346+0000 7f385a05b640 1 -- 172.21.15.83:0/849243439 shutdown_connections 2024-04-17T10:47:26.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.346+0000 7f385a05b640 1 --2- 172.21.15.83:0/849243439 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3840066f60 0x7f3840069420 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:26.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.346+0000 7f385a05b640 1 --2- 172.21.15.83:0/849243439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f385415dfa0 0x7f38540c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:26.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.346+0000 7f385a05b640 1 --2- 172.21.15.83:0/849243439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38540cee00 0x7f38540cc170 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:26.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.346+0000 7f385a05b640 1 --2- 172.21.15.83:0/849243439 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38540ce980 0x7f3854160d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:26.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.346+0000 7f385a05b640 1 -- 172.21.15.83:0/849243439 >> 172.21.15.83:0/849243439 conn(0x7f38540d0600 msgr2=0x7f38540b5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:26.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.350+0000 7f385a05b640 1 -- 172.21.15.83:0/849243439 shutdown_connections 2024-04-17T10:47:26.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.350+0000 7f385a05b640 1 -- 172.21.15.83:0/849243439 wait complete. 2024-04-17T10:47:26.367 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:47:26.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.486+0000 7f6c6cb72640 1 Processor -- start 2024-04-17T10:47:26.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.486+0000 7f6c6cb72640 1 -- start start 2024-04-17T10:47:26.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.486+0000 7f6c6cb72640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c6815ba30 0x7f6c68072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:26.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.486+0000 7f6c6cb72640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6c6815c960 0x7f6c680cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:26.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.486+0000 7f6c6cb72640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6c680cfa10 0x7f6c6806c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:26.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.486+0000 7f6c6cb72640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c68071ea0 con 0x7f6c6815c960 2024-04-17T10:47:26.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.486+0000 7f6c6cb72640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c680cfea0 con 0x7f6c680cfa10 2024-04-17T10:47:26.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.486+0000 7f6c6cb72640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6806e320 con 0x7f6c6815ba30 2024-04-17T10:47:26.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.486+0000 7f6c5ffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6c6815c960 0x7f6c680cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:26.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.486+0000 7f6c5ffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6c6815c960 0x7f6c680cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42142/0 (socket says 172.21.15.83:42142) 2024-04-17T10:47:26.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.486+0000 7f6c5ffff640 1 -- 172.21.15.83:0/136667983 learned_addr learned my addr 172.21.15.83:0/136667983 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:26.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.486+0000 7f6c6659c640 1 --2- 172.21.15.83:0/136667983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c6815ba30 0x7f6c68072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:26.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.486+0000 7f6c5ffff640 1 -- 172.21.15.83:0/136667983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c6815ba30 msgr2=0x7f6c68072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:26.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.486+0000 7f6c5ffff640 1 --2- 172.21.15.83:0/136667983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c6815ba30 0x7f6c68072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:26.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.486+0000 7f6c66d9d640 1 --2- 172.21.15.83:0/136667983 >> v2:172.21.15.167:3300/0 conn(0x7f6c680cfa10 0x7f6c6806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:26.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.486+0000 7f6c5ffff640 1 -- 172.21.15.83:0/136667983 >> v2:172.21.15.167:3300/0 conn(0x7f6c680cfa10 msgr2=0x7f6c6806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:26.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.486+0000 7f6c5ffff640 1 --2- 172.21.15.83:0/136667983 >> v2:172.21.15.167:3300/0 conn(0x7f6c680cfa10 0x7f6c6806c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:26.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.486+0000 7f6c5ffff640 1 -- 172.21.15.83:0/136667983 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6806e040 con 0x7f6c6815c960 2024-04-17T10:47:26.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.486+0000 7f6c6659c640 1 --2- 172.21.15.83:0/136667983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c6815ba30 0x7f6c68072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:26.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.490+0000 7f6c5ffff640 1 --2- 172.21.15.83:0/136667983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c6815c960 0x7f6c680cf4d0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f6c6806d810 tx=0x7f6c5001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bcf2fb16f87ed5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:26.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.490+0000 7f6c65d9b640 1 -- 172.21.15.83:0/136667983 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c500b0070 con 0x7f6c6815c960 2024-04-17T10:47:26.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.490+0000 7f6c65d9b640 1 -- 172.21.15.83:0/136667983 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c500ab600 con 0x7f6c6815c960 2024-04-17T10:47:26.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.490+0000 7f6c65d9b640 1 -- 172.21.15.83:0/136667983 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c500ab9d0 con 0x7f6c6815c960 2024-04-17T10:47:26.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.490+0000 7f6c6cb72640 1 -- 172.21.15.83:0/136667983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c6815c960 msgr2=0x7f6c680cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:26.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.490+0000 7f6c6cb72640 1 --2- 172.21.15.83:0/136667983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c6815c960 0x7f6c680cf4d0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f6c6806d810 tx=0x7f6c5001c490 comp rx=0 tx=0).stop 2024-04-17T10:47:26.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.490+0000 7f6c6cb72640 1 -- 172.21.15.83:0/136667983 shutdown_connections 2024-04-17T10:47:26.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.490+0000 7f6c6cb72640 1 --2- 172.21.15.83:0/136667983 >> v2:172.21.15.167:3300/0 conn(0x7f6c680cfa10 0x7f6c6806c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:26.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.490+0000 7f6c6cb72640 1 --2- 172.21.15.83:0/136667983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c6815c960 0x7f6c680cf4d0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:26.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.490+0000 7f6c6cb72640 1 --2- 172.21.15.83:0/136667983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c6815ba30 0x7f6c68072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:26.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.490+0000 7f6c6cb72640 1 -- 172.21.15.83:0/136667983 >> 172.21.15.83:0/136667983 conn(0x7f6c6806abe0 msgr2=0x7f6c6806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:26.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.490+0000 7f6c6cb72640 1 -- 172.21.15.83:0/136667983 shutdown_connections 2024-04-17T10:47:26.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.490+0000 7f6c6cb72640 1 -- 172.21.15.83:0/136667983 wait complete. 2024-04-17T10:47:26.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.494+0000 7f6c6cb72640 1 Processor -- start 2024-04-17T10:47:26.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.494+0000 7f6c6cb72640 1 -- start start 2024-04-17T10:47:26.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.494+0000 7f6c6cb72640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c680cfa10 0x7f6c681378d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:26.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.494+0000 7f6c6cb72640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c6815ba30 0x7f6c680af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:26.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.494+0000 7f6c6cb72640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c6815c960 0x7f6c680afc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:26.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.494+0000 7f6c6cb72640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6806dca0 con 0x7f6c680cfa10 2024-04-17T10:47:26.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.494+0000 7f6c6cb72640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c680aeae0 con 0x7f6c6815c960 2024-04-17T10:47:26.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.494+0000 7f6c6cb72640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6806de20 con 0x7f6c6815ba30 2024-04-17T10:47:26.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.494+0000 7f6c6659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c680cfa10 0x7f6c681378d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:26.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.494+0000 7f6c6659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c680cfa10 0x7f6c681378d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42144/0 (socket says 172.21.15.83:42144) 2024-04-17T10:47:26.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.494+0000 7f6c6659c640 1 -- 172.21.15.83:0/1658731626 learned_addr learned my addr 172.21.15.83:0/1658731626 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:26.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.494+0000 7f6c5ffff640 1 --2- 172.21.15.83:0/1658731626 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c6815ba30 0x7f6c680af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:26.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.494+0000 7f6c66d9d640 1 --2- 172.21.15.83:0/1658731626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c6815c960 0x7f6c680afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:26.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.494+0000 7f6c6659c640 1 -- 172.21.15.83:0/1658731626 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c6815ba30 msgr2=0x7f6c680af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:26.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.494+0000 7f6c6659c640 1 --2- 172.21.15.83:0/1658731626 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c6815ba30 0x7f6c680af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:26.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.494+0000 7f6c6659c640 1 -- 172.21.15.83:0/1658731626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c6815c960 msgr2=0x7f6c680afc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:26.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.494+0000 7f6c6659c640 1 --2- 172.21.15.83:0/1658731626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c6815c960 0x7f6c680afc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:26.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.494+0000 7f6c6659c640 1 -- 172.21.15.83:0/1658731626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c68139e40 con 0x7f6c680cfa10 2024-04-17T10:47:26.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.494+0000 7f6c66d9d640 1 --2- 172.21.15.83:0/1658731626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c6815c960 0x7f6c680afc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:26.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.494+0000 7f6c6659c640 1 --2- 172.21.15.83:0/1658731626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c680cfa10 0x7f6c681378d0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f6c5800c7f0 tx=0x7f6c5800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:26.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.494+0000 7f6c5effd640 1 -- 172.21.15.83:0/1658731626 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c58015070 con 0x7f6c680cfa10 2024-04-17T10:47:26.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.494+0000 7f6c5effd640 1 -- 172.21.15.83:0/1658731626 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c58007820 con 0x7f6c680cfa10 2024-04-17T10:47:26.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.494+0000 7f6c6cb72640 1 -- 172.21.15.83:0/1658731626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c68138da0 con 0x7f6c680cfa10 2024-04-17T10:47:26.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.494+0000 7f6c5effd640 1 -- 172.21.15.83:0/1658731626 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c58007c10 con 0x7f6c680cfa10 2024-04-17T10:47:26.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.494+0000 7f6c6cb72640 1 -- 172.21.15.83:0/1658731626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c68139360 con 0x7f6c680cfa10 2024-04-17T10:47:26.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.494+0000 7f6c6cb72640 1 -- 172.21.15.83:0/1658731626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c68074f90 con 0x7f6c680cfa10 2024-04-17T10:47:26.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.494+0000 7f6c5effd640 1 -- 172.21.15.83:0/1658731626 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6c58018070 con 0x7f6c680cfa10 2024-04-17T10:47:26.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.498+0000 7f6c5effd640 1 --2- 172.21.15.83:0/1658731626 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6c48066f60 0x7f6c48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:26.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.498+0000 7f6c5effd640 1 -- 172.21.15.83:0/1658731626 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6c58084e50 con 0x7f6c680cfa10 2024-04-17T10:47:26.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.498+0000 7f6c5ffff640 1 --2- 172.21.15.83:0/1658731626 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6c48066f60 0x7f6c48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:26.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.498+0000 7f6c5ffff640 1 --2- 172.21.15.83:0/1658731626 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6c48066f60 0x7f6c48069420 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6c68070060 tx=0x7f6c50008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:26.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.502+0000 7f6c5effd640 1 -- 172.21.15.83:0/1658731626 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6c580561b0 con 0x7f6c680cfa10 2024-04-17T10:47:26.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.654+0000 7f6c6cb72640 1 -- 172.21.15.83:0/1658731626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c680aee90 con 0x7f6c680cfa10 2024-04-17T10:47:26.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.654+0000 7f6c5effd640 1 -- 172.21.15.83:0/1658731626 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+12820 (secure 0 0 0) 0x7f6c58055ab0 con 0x7f6c680cfa10 2024-04-17T10:47:26.651 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:47:26.651 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":99,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:47:25.995847+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:47:26.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.654+0000 7f6c6cb72640 1 -- 172.21.15.83:0/1658731626 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6c48066f60 msgr2=0x7f6c48069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:26.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.654+0000 7f6c6cb72640 1 --2- 172.21.15.83:0/1658731626 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6c48066f60 0x7f6c48069420 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6c68070060 tx=0x7f6c50008040 comp rx=0 tx=0).stop 2024-04-17T10:47:26.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.654+0000 7f6c6cb72640 1 -- 172.21.15.83:0/1658731626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c680cfa10 msgr2=0x7f6c681378d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:26.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.654+0000 7f6c6cb72640 1 --2- 172.21.15.83:0/1658731626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c680cfa10 0x7f6c681378d0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f6c5800c7f0 tx=0x7f6c5800ccb0 comp rx=0 tx=0).stop 2024-04-17T10:47:26.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.658+0000 7f6c6cb72640 1 -- 172.21.15.83:0/1658731626 shutdown_connections 2024-04-17T10:47:26.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.658+0000 7f6c6cb72640 1 --2- 172.21.15.83:0/1658731626 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6c48066f60 0x7f6c48069420 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:26.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.658+0000 7f6c6cb72640 1 --2- 172.21.15.83:0/1658731626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c6815c960 0x7f6c680afc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:26.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.658+0000 7f6c6cb72640 1 --2- 172.21.15.83:0/1658731626 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c6815ba30 0x7f6c680af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:26.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.658+0000 7f6c6cb72640 1 --2- 172.21.15.83:0/1658731626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c680cfa10 0x7f6c681378d0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:26.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.658+0000 7f6c6cb72640 1 -- 172.21.15.83:0/1658731626 >> 172.21.15.83:0/1658731626 conn(0x7f6c6806abe0 msgr2=0x7f6c6814ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:26.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.658+0000 7f6c6cb72640 1 -- 172.21.15.83:0/1658731626 shutdown_connections 2024-04-17T10:47:26.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.658+0000 7f6c6cb72640 1 -- 172.21.15.83:0/1658731626 wait complete. 2024-04-17T10:47:26.669 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:47:26.669 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:47:26.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.794+0000 7f0d4d641640 1 Processor -- start 2024-04-17T10:47:26.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.794+0000 7f0d4d641640 1 -- start start 2024-04-17T10:47:26.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.798+0000 7f0d4d641640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d48162020 0x7f0d480b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:26.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.798+0000 7f0d4d641640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0d4806b520 0x7f0d48162400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:26.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.798+0000 7f0d4d641640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d4806c1f0 con 0x7f0d480b3440 2024-04-17T10:47:26.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.798+0000 7f0d4d641640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d480b29f0 con 0x7f0d4806b520 2024-04-17T10:47:26.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.798+0000 7f0d4d641640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d480b2fb0 con 0x7f0d48162020 2024-04-17T10:47:26.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.798+0000 7f0d3e7fc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0d480b3440 0x7f0d4806ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41556/0 (socket says 172.21.15.83:41556) 2024-04-17T10:47:26.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.798+0000 7f0d3e7fc640 1 -- 172.21.15.83:0/3484825878 learned_addr learned my addr 172.21.15.83:0/3484825878 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:26.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.798+0000 7f0d477fe640 1 --2- 172.21.15.83:0/3484825878 >> v2:172.21.15.83:3300/0 conn(0x7f0d4806b520 0x7f0d48162400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:26.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.798+0000 7f0d46ffd640 1 --2- 172.21.15.83:0/3484825878 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d48162020 0x7f0d480b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:26.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.798+0000 7f0d467fc640 1 -- 172.21.15.83:0/3484825878 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2295639908 0 0) 0x7f0d4806c1f0 con 0x7f0d480b3440 2024-04-17T10:47:26.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.798+0000 7f0d477fe640 1 -- 172.21.15.83:0/3484825878 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d48162020 msgr2=0x7f0d480b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:26.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.798+0000 7f0d477fe640 1 --2- 172.21.15.83:0/3484825878 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d48162020 0x7f0d480b45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:26.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.798+0000 7f0d477fe640 1 -- 172.21.15.83:0/3484825878 >> v1:172.21.15.83:6789/0 conn(0x7f0d480b3440 legacy=0x7f0d4806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:26.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.798+0000 7f0d477fe640 1 -- 172.21.15.83:0/3484825878 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d4806c490 con 0x7f0d4806b520 2024-04-17T10:47:26.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.798+0000 7f0d477fe640 1 --2- 172.21.15.83:0/3484825878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d4806b520 0x7f0d48162400 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f0d480b2b70 tx=0x7f0d4001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d00c4a146d7da96 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:26.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.798+0000 7f0d467fc640 1 -- 172.21.15.83:0/3484825878 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d400b2070 con 0x7f0d4806b520 2024-04-17T10:47:26.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.798+0000 7f0d467fc640 1 -- 172.21.15.83:0/3484825878 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d400a5a10 con 0x7f0d4806b520 2024-04-17T10:47:26.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.798+0000 7f0d467fc640 1 -- 172.21.15.83:0/3484825878 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d400a5de0 con 0x7f0d4806b520 2024-04-17T10:47:26.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.798+0000 7f0d4d641640 1 -- 172.21.15.83:0/3484825878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d4806b520 msgr2=0x7f0d48162400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:26.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.798+0000 7f0d4d641640 1 --2- 172.21.15.83:0/3484825878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d4806b520 0x7f0d48162400 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f0d480b2b70 tx=0x7f0d4001c950 comp rx=0 tx=0).stop 2024-04-17T10:47:26.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.798+0000 7f0d4d641640 1 -- 172.21.15.83:0/3484825878 shutdown_connections 2024-04-17T10:47:26.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.798+0000 7f0d4d641640 1 --2- 172.21.15.83:0/3484825878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d4806b520 0x7f0d48162400 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:26.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.798+0000 7f0d4d641640 1 --2- 172.21.15.83:0/3484825878 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d48162020 0x7f0d480b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:26.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.798+0000 7f0d4d641640 1 -- 172.21.15.83:0/3484825878 >> 172.21.15.83:0/3484825878 conn(0x7f0d480b15b0 msgr2=0x7f0d480b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:26.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.798+0000 7f0d4d641640 1 -- 172.21.15.83:0/3484825878 shutdown_connections 2024-04-17T10:47:26.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.798+0000 7f0d4d641640 1 -- 172.21.15.83:0/3484825878 wait complete. 2024-04-17T10:47:26.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.802+0000 7f0d4d641640 1 Processor -- start 2024-04-17T10:47:26.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.802+0000 7f0d4d641640 1 -- start start 2024-04-17T10:47:26.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.802+0000 7f0d4d641640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d4806b520 0x7f0d480adb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:26.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.802+0000 7f0d4d641640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d480b3440 0x7f0d48148730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:26.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.802+0000 7f0d4d641640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d48162020 0x7f0d481fb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:26.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.802+0000 7f0d4d641640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d480aebe0 con 0x7f0d48162020 2024-04-17T10:47:26.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.802+0000 7f0d4d641640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d480aea60 con 0x7f0d4806b520 2024-04-17T10:47:26.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.802+0000 7f0d4d641640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d48162d50 con 0x7f0d480b3440 2024-04-17T10:47:26.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.802+0000 7f0d477fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d48162020 0x7f0d481fb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:26.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.802+0000 7f0d477fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d48162020 0x7f0d481fb3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60246/0 (socket says 172.21.15.83:60246) 2024-04-17T10:47:26.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.802+0000 7f0d477fe640 1 -- 172.21.15.83:0/3830523131 learned_addr learned my addr 172.21.15.83:0/3830523131 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:26.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.802+0000 7f0d46ffd640 1 --2- 172.21.15.83:0/3830523131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d4806b520 0x7f0d480adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:26.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.802+0000 7f0d3e7fc640 1 --2- 172.21.15.83:0/3830523131 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d480b3440 0x7f0d48148730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:26.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.802+0000 7f0d477fe640 1 -- 172.21.15.83:0/3830523131 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d480b3440 msgr2=0x7f0d48148730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:26.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.802+0000 7f0d477fe640 1 --2- 172.21.15.83:0/3830523131 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d480b3440 0x7f0d48148730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:26.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.802+0000 7f0d477fe640 1 -- 172.21.15.83:0/3830523131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d4806b520 msgr2=0x7f0d480adb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:26.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.802+0000 7f0d477fe640 1 --2- 172.21.15.83:0/3830523131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d4806b520 0x7f0d480adb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:26.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.802+0000 7f0d477fe640 1 -- 172.21.15.83:0/3830523131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d480a97b0 con 0x7f0d48162020 2024-04-17T10:47:26.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.802+0000 7f0d46ffd640 1 --2- 172.21.15.83:0/3830523131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d4806b520 0x7f0d480adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:26.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.802+0000 7f0d3e7fc640 1 --2- 172.21.15.83:0/3830523131 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d480b3440 0x7f0d48148730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:26.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.802+0000 7f0d477fe640 1 --2- 172.21.15.83:0/3830523131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d48162020 0x7f0d481fb3b0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f0d4001b000 tx=0x7f0d400196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:26.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.802+0000 7f0d3ffff640 1 -- 172.21.15.83:0/3830523131 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d400b2050 con 0x7f0d48162020 2024-04-17T10:47:26.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.802+0000 7f0d3ffff640 1 -- 172.21.15.83:0/3830523131 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d400be070 con 0x7f0d48162020 2024-04-17T10:47:26.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.802+0000 7f0d3ffff640 1 -- 172.21.15.83:0/3830523131 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d400adb10 con 0x7f0d48162020 2024-04-17T10:47:26.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.802+0000 7f0d4d641640 1 -- 172.21.15.83:0/3830523131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d480a9a90 con 0x7f0d48162020 2024-04-17T10:47:26.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.802+0000 7f0d4d641640 1 -- 172.21.15.83:0/3830523131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d480aa050 con 0x7f0d48162020 2024-04-17T10:47:26.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.802+0000 7f0d4d641640 1 -- 172.21.15.83:0/3830523131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d48074f90 con 0x7f0d48162020 2024-04-17T10:47:26.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.806+0000 7f0d3ffff640 1 -- 172.21.15.83:0/3830523131 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0d400adcb0 con 0x7f0d48162020 2024-04-17T10:47:26.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.806+0000 7f0d3ffff640 1 --2- 172.21.15.83:0/3830523131 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0d28066f60 0x7f0d28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:26.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.806+0000 7f0d3ffff640 1 -- 172.21.15.83:0/3830523131 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0d40120070 con 0x7f0d48162020 2024-04-17T10:47:26.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.806+0000 7f0d46ffd640 1 --2- 172.21.15.83:0/3830523131 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0d28066f60 0x7f0d28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:26.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.806+0000 7f0d46ffd640 1 --2- 172.21.15.83:0/3830523131 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0d28066f60 0x7f0d28069420 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0d300097a0 tx=0x7f0d30008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:26.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.810+0000 7f0d3ffff640 1 -- 172.21.15.83:0/3830523131 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0d40016540 con 0x7f0d48162020 2024-04-17T10:47:26.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.966+0000 7f0d4d641640 1 -- 172.21.15.83:0/3830523131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0d48148c70 con 0x7f0d48162020 2024-04-17T10:47:26.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.966+0000 7f0d3ffff640 1 -- 172.21.15.83:0/3830523131 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0d400ab340 con 0x7f0d48162020 2024-04-17T10:47:26.965 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:47:26.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.970+0000 7f0d4d641640 1 -- 172.21.15.83:0/3830523131 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0d28066f60 msgr2=0x7f0d28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:26.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.970+0000 7f0d4d641640 1 --2- 172.21.15.83:0/3830523131 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0d28066f60 0x7f0d28069420 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0d300097a0 tx=0x7f0d30008040 comp rx=0 tx=0).stop 2024-04-17T10:47:26.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.970+0000 7f0d4d641640 1 -- 172.21.15.83:0/3830523131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d48162020 msgr2=0x7f0d481fb3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:26.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.970+0000 7f0d4d641640 1 --2- 172.21.15.83:0/3830523131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d48162020 0x7f0d481fb3b0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f0d4001b000 tx=0x7f0d400196f0 comp rx=0 tx=0).stop 2024-04-17T10:47:26.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.970+0000 7f0d4d641640 1 -- 172.21.15.83:0/3830523131 shutdown_connections 2024-04-17T10:47:26.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.970+0000 7f0d4d641640 1 --2- 172.21.15.83:0/3830523131 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0d28066f60 0x7f0d28069420 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:26.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.970+0000 7f0d4d641640 1 --2- 172.21.15.83:0/3830523131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d48162020 0x7f0d481fb3b0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:26.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.970+0000 7f0d4d641640 1 --2- 172.21.15.83:0/3830523131 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d480b3440 0x7f0d48148730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:26.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.970+0000 7f0d4d641640 1 --2- 172.21.15.83:0/3830523131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d4806b520 0x7f0d480adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:26.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.970+0000 7f0d4d641640 1 -- 172.21.15.83:0/3830523131 >> 172.21.15.83:0/3830523131 conn(0x7f0d480b15b0 msgr2=0x7f0d4815ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:26.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.970+0000 7f0d4d641640 1 -- 172.21.15.83:0/3830523131 shutdown_connections 2024-04-17T10:47:26.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:26.970+0000 7f0d4d641640 1 -- 172.21.15.83:0/3830523131 wait complete. 2024-04-17T10:47:26.985 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-17T10:47:27.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.110+0000 7f3002af3640 1 Processor -- start 2024-04-17T10:47:27.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.110+0000 7f3002af3640 1 -- start start 2024-04-17T10:47:27.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.110+0000 7f3002af3640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2ffc15ba30 0x7f2ffc072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:27.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.110+0000 7f3002af3640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2ffc15c960 0x7f2ffc0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:27.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.110+0000 7f3002af3640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ffc0cfea0 con 0x7f2ffc0cfa10 2024-04-17T10:47:27.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.110+0000 7f3002af3640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ffc071ea0 con 0x7f2ffc15c960 2024-04-17T10:47:27.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.110+0000 7f3002af3640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ffc06e320 con 0x7f2ffc15ba30 2024-04-17T10:47:27.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.110+0000 7f2ffbfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2ffc15c960 0x7f2ffc0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:27.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.110+0000 7f2ffbfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2ffc15c960 0x7f2ffc0cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60260/0 (socket says 172.21.15.83:60260) 2024-04-17T10:47:27.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.110+0000 7f2ffbfff640 1 -- 172.21.15.83:0/2561704856 learned_addr learned my addr 172.21.15.83:0/2561704856 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:27.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.110+0000 7f300088f640 1 --2- 172.21.15.83:0/2561704856 >> v2:172.21.15.167:3300/0 conn(0x7f2ffc15ba30 0x7f2ffc072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:27.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.110+0000 7f2ffbfff640 1 -- 172.21.15.83:0/2561704856 >> v2:172.21.15.167:3300/0 conn(0x7f2ffc15ba30 msgr2=0x7f2ffc072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:27.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.110+0000 7f2ffbfff640 1 --2- 172.21.15.83:0/2561704856 >> v2:172.21.15.167:3300/0 conn(0x7f2ffc15ba30 0x7f2ffc072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:27.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.110+0000 7f2ffbfff640 1 -- 172.21.15.83:0/2561704856 >> v1:172.21.15.167:6789/0 conn(0x7f2ffc0cfa10 legacy=0x7f2ffc06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:27.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.110+0000 7f2ffbfff640 1 -- 172.21.15.83:0/2561704856 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ffc06e040 con 0x7f2ffc15c960 2024-04-17T10:47:27.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.114+0000 7f2ffbfff640 1 --2- 172.21.15.83:0/2561704856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ffc15c960 0x7f2ffc0cf4d0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc0b2410 tx=0x7f2ff001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac1524a28fe4622 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:27.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.114+0000 7f2ffb7fe640 1 -- 172.21.15.83:0/2561704856 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff00b0070 con 0x7f2ffc15c960 2024-04-17T10:47:27.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.114+0000 7f2ffb7fe640 1 -- 172.21.15.83:0/2561704856 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff00ab600 con 0x7f2ffc15c960 2024-04-17T10:47:27.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.114+0000 7f2ffb7fe640 1 -- 172.21.15.83:0/2561704856 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff00ab9d0 con 0x7f2ffc15c960 2024-04-17T10:47:27.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.114+0000 7f3002af3640 1 -- 172.21.15.83:0/2561704856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ffc15c960 msgr2=0x7f2ffc0cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:27.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.114+0000 7f3002af3640 1 --2- 172.21.15.83:0/2561704856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ffc15c960 0x7f2ffc0cf4d0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc0b2410 tx=0x7f2ff001c450 comp rx=0 tx=0).stop 2024-04-17T10:47:27.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.114+0000 7f3002af3640 1 -- 172.21.15.83:0/2561704856 shutdown_connections 2024-04-17T10:47:27.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.114+0000 7f3002af3640 1 --2- 172.21.15.83:0/2561704856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ffc15c960 0x7f2ffc0cf4d0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:27.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.114+0000 7f3002af3640 1 --2- 172.21.15.83:0/2561704856 >> v2:172.21.15.167:3300/0 conn(0x7f2ffc15ba30 0x7f2ffc072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:27.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.114+0000 7f3002af3640 1 -- 172.21.15.83:0/2561704856 >> 172.21.15.83:0/2561704856 conn(0x7f2ffc06abe0 msgr2=0x7f2ffc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:27.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.114+0000 7f3002af3640 1 -- 172.21.15.83:0/2561704856 shutdown_connections 2024-04-17T10:47:27.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.114+0000 7f3002af3640 1 -- 172.21.15.83:0/2561704856 wait complete. 2024-04-17T10:47:27.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.118+0000 7f3002af3640 1 Processor -- start 2024-04-17T10:47:27.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.118+0000 7f3002af3640 1 -- start start 2024-04-17T10:47:27.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.118+0000 7f3002af3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ffc0cfa10 0x7f2ffc144600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:27.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.118+0000 7f3002af3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ffc15ba30 0x7f2ffc157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:27.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.118+0000 7f3002af3640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ffc15c960 0x7f2ffc157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:27.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.118+0000 7f3002af3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc06dca0 con 0x7f2ffc15ba30 2024-04-17T10:47:27.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.118+0000 7f3002af3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc0aeae0 con 0x7f2ffc0cfa10 2024-04-17T10:47:27.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.118+0000 7f3002af3640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc06de20 con 0x7f2ffc15c960 2024-04-17T10:47:27.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.118+0000 7f2ffbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ffc15ba30 0x7f2ffc157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:27.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.118+0000 7f2ffbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ffc15ba30 0x7f2ffc157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60270/0 (socket says 172.21.15.83:60270) 2024-04-17T10:47:27.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.118+0000 7f2ffbfff640 1 -- 172.21.15.83:0/1663876042 learned_addr learned my addr 172.21.15.83:0/1663876042 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:27.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.118+0000 7f3001090640 1 --2- 172.21.15.83:0/1663876042 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ffc15c960 0x7f2ffc157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:27.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.118+0000 7f300088f640 1 --2- 172.21.15.83:0/1663876042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ffc0cfa10 0x7f2ffc144600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:27.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.118+0000 7f2ffbfff640 1 -- 172.21.15.83:0/1663876042 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ffc15c960 msgr2=0x7f2ffc157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:27.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.118+0000 7f2ffbfff640 1 --2- 172.21.15.83:0/1663876042 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ffc15c960 0x7f2ffc157cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:27.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.118+0000 7f2ffbfff640 1 -- 172.21.15.83:0/1663876042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ffc0cfa10 msgr2=0x7f2ffc144600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:27.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.118+0000 7f2ffbfff640 1 --2- 172.21.15.83:0/1663876042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ffc0cfa10 0x7f2ffc144600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:27.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.118+0000 7f2ffbfff640 1 -- 172.21.15.83:0/1663876042 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ffc0afec0 con 0x7f2ffc15ba30 2024-04-17T10:47:27.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.118+0000 7f300088f640 1 --2- 172.21.15.83:0/1663876042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ffc0cfa10 0x7f2ffc144600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:27.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.118+0000 7f2ffbfff640 1 --2- 172.21.15.83:0/1663876042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ffc15ba30 0x7f2ffc157770 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f2ff001a080 tx=0x7f2ffc06d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:27.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.118+0000 7f2ff97fa640 1 -- 172.21.15.83:0/1663876042 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff00b9070 con 0x7f2ffc15ba30 2024-04-17T10:47:27.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.118+0000 7f2ff97fa640 1 -- 172.21.15.83:0/1663876042 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff00b4420 con 0x7f2ffc15ba30 2024-04-17T10:47:27.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.118+0000 7f3002af3640 1 -- 172.21.15.83:0/1663876042 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ffc0b01a0 con 0x7f2ffc15ba30 2024-04-17T10:47:27.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.118+0000 7f2ff97fa640 1 -- 172.21.15.83:0/1663876042 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff00b0040 con 0x7f2ffc15ba30 2024-04-17T10:47:27.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.118+0000 7f3002af3640 1 -- 172.21.15.83:0/1663876042 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ffc0b0760 con 0x7f2ffc15ba30 2024-04-17T10:47:27.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.118+0000 7f3002af3640 1 -- 172.21.15.83:0/1663876042 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ffc074f90 con 0x7f2ffc15ba30 2024-04-17T10:47:27.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.118+0000 7f2ff97fa640 1 -- 172.21.15.83:0/1663876042 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2ff00bc070 con 0x7f2ffc15ba30 2024-04-17T10:47:27.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.118+0000 7f2ff97fa640 1 --2- 172.21.15.83:0/1663876042 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2fe8066f60 0x7f2fe8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:27.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.118+0000 7f2ff97fa640 1 -- 172.21.15.83:0/1663876042 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2ff011de80 con 0x7f2ffc15ba30 2024-04-17T10:47:27.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.118+0000 7f300088f640 1 --2- 172.21.15.83:0/1663876042 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2fe8066f60 0x7f2fe8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:27.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.122+0000 7f300088f640 1 --2- 172.21.15.83:0/1663876042 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2fe8066f60 0x7f2fe8069420 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f2fec0097a0 tx=0x7f2fec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:27.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.122+0000 7f2ff97fa640 1 -- 172.21.15.83:0/1663876042 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2ff00160d0 con 0x7f2ffc15ba30 2024-04-17T10:47:27.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.270+0000 7f3002af3640 1 -- 172.21.15.83:0/1663876042 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2ffc15e2a0 con 0x7f2ffc15ba30 2024-04-17T10:47:27.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.270+0000 7f2ff97fa640 1 -- 172.21.15.83:0/1663876042 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v99) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2ff00ef1e0 con 0x7f2ffc15ba30 2024-04-17T10:47:27.269 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:47:27.269 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-04-17T10:47:27.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:47:27.269 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2024-04-17T10:47:27.269 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2024-04-17T10:47:27.269 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:47:27.269 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-04-17T10:47:27.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.274+0000 7f3002af3640 1 -- 172.21.15.83:0/1663876042 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2fe8066f60 msgr2=0x7f2fe8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:27.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.274+0000 7f3002af3640 1 --2- 172.21.15.83:0/1663876042 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2fe8066f60 0x7f2fe8069420 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f2fec0097a0 tx=0x7f2fec008040 comp rx=0 tx=0).stop 2024-04-17T10:47:27.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.274+0000 7f3002af3640 1 -- 172.21.15.83:0/1663876042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ffc15ba30 msgr2=0x7f2ffc157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:27.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.274+0000 7f3002af3640 1 --2- 172.21.15.83:0/1663876042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ffc15ba30 0x7f2ffc157770 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f2ff001a080 tx=0x7f2ffc06d3b0 comp rx=0 tx=0).stop 2024-04-17T10:47:27.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.274+0000 7f3002af3640 1 -- 172.21.15.83:0/1663876042 shutdown_connections 2024-04-17T10:47:27.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.274+0000 7f3002af3640 1 --2- 172.21.15.83:0/1663876042 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2fe8066f60 0x7f2fe8069420 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:27.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.274+0000 7f3002af3640 1 --2- 172.21.15.83:0/1663876042 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ffc15c960 0x7f2ffc157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:27.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.274+0000 7f3002af3640 1 --2- 172.21.15.83:0/1663876042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ffc15ba30 0x7f2ffc157770 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:27.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.274+0000 7f3002af3640 1 --2- 172.21.15.83:0/1663876042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ffc0cfa10 0x7f2ffc144600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:27.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.274+0000 7f3002af3640 1 -- 172.21.15.83:0/1663876042 >> 172.21.15.83:0/1663876042 conn(0x7f2ffc06abe0 msgr2=0x7f2ffc06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:27.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.274+0000 7f3002af3640 1 -- 172.21.15.83:0/1663876042 shutdown_connections 2024-04-17T10:47:27.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.274+0000 7f3002af3640 1 -- 172.21.15.83:0/1663876042 wait complete. 2024-04-17T10:47:27.287 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-17T10:47:27.287 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-17T10:47:27.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.414+0000 7f11ada7a640 1 Processor -- start 2024-04-17T10:47:27.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.414+0000 7f11ada7a640 1 -- start start 2024-04-17T10:47:27.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.414+0000 7f11ada7a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11a80d1fd0 0x7f11a80d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:27.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.414+0000 7f11ada7a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11a8074800 con 0x7f11a80d4510 2024-04-17T10:47:27.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.414+0000 7f11ada7a640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11a80d2920 con 0x7f11a815e020 2024-04-17T10:47:27.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.414+0000 7f11ada7a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a80cf550 con 0x7f11a80d1fd0 2024-04-17T10:47:27.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.414+0000 7f11a6ffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f11a80d4510 0x7f11a80d17d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41566/0 (socket says 172.21.15.83:41566) 2024-04-17T10:47:27.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.414+0000 7f11a6ffd640 1 -- 172.21.15.83:0/309888713 learned_addr learned my addr 172.21.15.83:0/309888713 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:27.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.414+0000 7f11a7fff640 1 --2- 172.21.15.83:0/309888713 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11a80d1fd0 0x7f11a80d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:27.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.414+0000 7f11a67fc640 1 -- 172.21.15.83:0/309888713 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1830042017 0 0) 0x7f11a8074800 con 0x7f11a80d4510 2024-04-17T10:47:27.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.414+0000 7f11a67fc640 1 -- 172.21.15.83:0/309888713 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1194003530 con 0x7f11a80d4510 2024-04-17T10:47:27.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.418+0000 7f11a67fc640 1 -- 172.21.15.83:0/309888713 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1171828206 0 0) 0x7f1194003530 con 0x7f11a80d4510 2024-04-17T10:47:27.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.418+0000 7f11a67fc640 1 -- 172.21.15.83:0/309888713 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11a80d1fd0 msgr2=0x7f11a80d23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:27.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.418+0000 7f11a67fc640 1 --2- 172.21.15.83:0/309888713 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11a80d1fd0 0x7f11a80d23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:27.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.418+0000 7f11a67fc640 1 -- 172.21.15.83:0/309888713 >> v1:172.21.15.167:6789/0 conn(0x7f11a815e020 legacy=0x7f11a80cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:27.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.418+0000 7f11a67fc640 1 -- 172.21.15.83:0/309888713 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a80ce980 con 0x7f11a80d4510 2024-04-17T10:47:27.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.418+0000 7f11a67fc640 1 -- 172.21.15.83:0/309888713 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f1190001680 con 0x7f11a80d4510 2024-04-17T10:47:27.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.418+0000 7f11a67fc640 1 -- 172.21.15.83:0/309888713 >> v1:172.21.15.83:6789/0 conn(0x7f11a80d4510 legacy=0x7f11a80d17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:27.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.418+0000 7f11a67fc640 1 --2- 172.21.15.83:0/309888713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11940039f0 0x7f1194003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:27.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.418+0000 7f11a67fc640 1 --2- 172.21.15.83:0/309888713 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11a80d1fd0 0x7f1194004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:27.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.418+0000 7f11a67fc640 1 --2- 172.21.15.83:0/309888713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1194004d50 0x7f11940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:27.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.418+0000 7f11a67fc640 1 -- 172.21.15.83:0/309888713 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a80cf550 con 0x7f11940039f0 2024-04-17T10:47:27.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.418+0000 7f11a67fc640 1 -- 172.21.15.83:0/309888713 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119400ba80 con 0x7f1194004d50 2024-04-17T10:47:27.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.418+0000 7f11a67fc640 1 -- 172.21.15.83:0/309888713 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f119400bc80 con 0x7f11a80d1fd0 2024-04-17T10:47:27.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.418+0000 7f11a77fe640 1 --2- 172.21.15.83:0/309888713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11940039f0 0x7f1194003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:27.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.418+0000 7f11a7fff640 1 --2- 172.21.15.83:0/309888713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1194004d50 0x7f11940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:27.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.418+0000 7f11a6ffd640 1 --2- 172.21.15.83:0/309888713 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11a80d1fd0 0x7f1194004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:27.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.418+0000 7f11a77fe640 1 -- 172.21.15.83:0/309888713 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11a80d1fd0 msgr2=0x7f1194004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:27.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.418+0000 7f11a77fe640 1 --2- 172.21.15.83:0/309888713 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11a80d1fd0 0x7f1194004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:27.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.418+0000 7f11a77fe640 1 -- 172.21.15.83:0/309888713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1194004d50 msgr2=0x7f11940051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:27.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.418+0000 7f11a77fe640 1 --2- 172.21.15.83:0/309888713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1194004d50 0x7f11940051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:27.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.418+0000 7f11a77fe640 1 -- 172.21.15.83:0/309888713 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1194003530 con 0x7f11940039f0 2024-04-17T10:47:27.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.418+0000 7f11a77fe640 1 --2- 172.21.15.83:0/309888713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11940039f0 0x7f1194003e30 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f1198001750 tx=0x7f119801c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c172437163a5404 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:27.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.418+0000 7f11a67fc640 1 -- 172.21.15.83:0/309888713 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11980013a0 con 0x7f11940039f0 2024-04-17T10:47:27.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.418+0000 7f11a67fc640 1 -- 172.21.15.83:0/309888713 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11980052f0 con 0x7f11940039f0 2024-04-17T10:47:27.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.418+0000 7f11a67fc640 1 -- 172.21.15.83:0/309888713 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1198003840 con 0x7f11940039f0 2024-04-17T10:47:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.418+0000 7f11ada7a640 1 -- 172.21.15.83:0/309888713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11940039f0 msgr2=0x7f1194003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.418+0000 7f11ada7a640 1 --2- 172.21.15.83:0/309888713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11940039f0 0x7f1194003e30 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f1198001750 tx=0x7f119801c1d0 comp rx=0 tx=0).stop 2024-04-17T10:47:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.418+0000 7f11a7fff640 1 -- 172.21.15.83:0/309888713 reap_dead start 2024-04-17T10:47:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.418+0000 7f11ada7a640 1 -- 172.21.15.83:0/309888713 shutdown_connections 2024-04-17T10:47:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.418+0000 7f11ada7a640 1 -- 172.21.15.83:0/309888713 >> 172.21.15.83:0/309888713 conn(0x7f11a80d39f0 msgr2=0x7f11a80d3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.418+0000 7f11ada7a640 1 -- 172.21.15.83:0/309888713 shutdown_connections 2024-04-17T10:47:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.418+0000 7f11ada7a640 1 -- 172.21.15.83:0/309888713 wait complete. 2024-04-17T10:47:27.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.422+0000 7f11ada7a640 1 Processor -- start 2024-04-17T10:47:27.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.422+0000 7f11ada7a640 1 -- start start 2024-04-17T10:47:27.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.422+0000 7f11ada7a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11940039f0 0x7f11a80b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:27.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.422+0000 7f11ada7a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11a80b1950 0x7f11a80b0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:27.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.422+0000 7f11ada7a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11a80b0c40 0x7f11a81428e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:27.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.422+0000 7f11ada7a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a806ba80 con 0x7f11a80b1950 2024-04-17T10:47:27.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.422+0000 7f11ada7a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a806b900 con 0x7f11a80b0c40 2024-04-17T10:47:27.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.422+0000 7f11ada7a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a806bc00 con 0x7f11940039f0 2024-04-17T10:47:27.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.422+0000 7f11a6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11a80b1950 0x7f11a80b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:27.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.422+0000 7f11a6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11a80b1950 0x7f11a80b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60280/0 (socket says 172.21.15.83:60280) 2024-04-17T10:47:27.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.422+0000 7f11a6ffd640 1 -- 172.21.15.83:0/4289502667 learned_addr learned my addr 172.21.15.83:0/4289502667 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:27.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.422+0000 7f11a77fe640 1 --2- 172.21.15.83:0/4289502667 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11940039f0 0x7f11a80b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:27.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.422+0000 7f11a6ffd640 1 -- 172.21.15.83:0/4289502667 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11940039f0 msgr2=0x7f11a80b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:27.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.422+0000 7f11a7fff640 1 --2- 172.21.15.83:0/4289502667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11a80b0c40 0x7f11a81428e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:27.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.422+0000 7f11a6ffd640 1 --2- 172.21.15.83:0/4289502667 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11940039f0 0x7f11a80b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:27.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.422+0000 7f11a6ffd640 1 -- 172.21.15.83:0/4289502667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11a80b0c40 msgr2=0x7f11a81428e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:27.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.422+0000 7f11a6ffd640 1 --2- 172.21.15.83:0/4289502667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11a80b0c40 0x7f11a81428e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:27.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.422+0000 7f11a6ffd640 1 -- 172.21.15.83:0/4289502667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a8157250 con 0x7f11a80b1950 2024-04-17T10:47:27.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.422+0000 7f11a77fe640 1 --2- 172.21.15.83:0/4289502667 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11940039f0 0x7f11a80b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:27.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.422+0000 7f11a7fff640 1 --2- 172.21.15.83:0/4289502667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11a80b0c40 0x7f11a81428e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:27.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.422+0000 7f11a6ffd640 1 --2- 172.21.15.83:0/4289502667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11a80b1950 0x7f11a80b0700 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f119000e830 tx=0x7f119000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:27.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.422+0000 7f119ffff640 1 -- 172.21.15.83:0/4289502667 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119000ee80 con 0x7f11a80b1950 2024-04-17T10:47:27.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.422+0000 7f119ffff640 1 -- 172.21.15.83:0/4289502667 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1190018070 con 0x7f11a80b1950 2024-04-17T10:47:27.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.422+0000 7f11ada7a640 1 -- 172.21.15.83:0/4289502667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a80b51e0 con 0x7f11a80b1950 2024-04-17T10:47:27.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.422+0000 7f119ffff640 1 -- 172.21.15.83:0/4289502667 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119000caf0 con 0x7f11a80b1950 2024-04-17T10:47:27.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.422+0000 7f11ada7a640 1 -- 172.21.15.83:0/4289502667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a80b5760 con 0x7f11a80b1950 2024-04-17T10:47:27.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.422+0000 7f11ada7a640 1 -- 172.21.15.83:0/4289502667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a80b1db0 con 0x7f11a80b1950 2024-04-17T10:47:27.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.426+0000 7f119ffff640 1 -- 172.21.15.83:0/4289502667 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f119000cc90 con 0x7f11a80b1950 2024-04-17T10:47:27.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.426+0000 7f119ffff640 1 --2- 172.21.15.83:0/4289502667 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1188066f90 0x7f1188069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:27.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.426+0000 7f119ffff640 1 -- 172.21.15.83:0/4289502667 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1190084930 con 0x7f11a80b1950 2024-04-17T10:47:27.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.426+0000 7f11a77fe640 1 --2- 172.21.15.83:0/4289502667 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1188066f90 0x7f1188069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:27.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.426+0000 7f11a77fe640 1 --2- 172.21.15.83:0/4289502667 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1188066f90 0x7f1188069450 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1194004790 tx=0x7f11980011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:27.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.430+0000 7f119ffff640 1 -- 172.21.15.83:0/4289502667 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f11a80b1db0 con 0x7f11a80b1950 2024-04-17T10:47:27.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:27.574+0000 7f11ada7a640 1 -- 172.21.15.83:0/4289502667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f11a8075000 con 0x7f11a80b1950 2024-04-17T10:47:27.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.002+0000 7f119ffff640 1 -- 172.21.15.83:0/4289502667 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v100) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1190055c00 con 0x7f11a80b1950 2024-04-17T10:47:27.998 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2024-04-17T10:47:28.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.006+0000 7f11ada7a640 1 -- 172.21.15.83:0/4289502667 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1188066f90 msgr2=0x7f1188069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:28.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.006+0000 7f11ada7a640 1 --2- 172.21.15.83:0/4289502667 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1188066f90 0x7f1188069450 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1194004790 tx=0x7f11980011a0 comp rx=0 tx=0).stop 2024-04-17T10:47:28.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.006+0000 7f11ada7a640 1 -- 172.21.15.83:0/4289502667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11a80b1950 msgr2=0x7f11a80b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:28.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.006+0000 7f11ada7a640 1 --2- 172.21.15.83:0/4289502667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11a80b1950 0x7f11a80b0700 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f119000e830 tx=0x7f119000ec30 comp rx=0 tx=0).stop 2024-04-17T10:47:28.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.006+0000 7f11ada7a640 1 -- 172.21.15.83:0/4289502667 shutdown_connections 2024-04-17T10:47:28.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.006+0000 7f11ada7a640 1 --2- 172.21.15.83:0/4289502667 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1188066f90 0x7f1188069450 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:28.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.006+0000 7f11ada7a640 1 --2- 172.21.15.83:0/4289502667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11a80b0c40 0x7f11a81428e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:28.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.006+0000 7f11ada7a640 1 --2- 172.21.15.83:0/4289502667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11a80b1950 0x7f11a80b0700 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:28.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.006+0000 7f11ada7a640 1 --2- 172.21.15.83:0/4289502667 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11940039f0 0x7f11a80b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:28.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.006+0000 7f11ada7a640 1 -- 172.21.15.83:0/4289502667 >> 172.21.15.83:0/4289502667 conn(0x7f11a80d39f0 msgr2=0x7f11a8162090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:28.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.006+0000 7f11ada7a640 1 -- 172.21.15.83:0/4289502667 shutdown_connections 2024-04-17T10:47:28.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.006+0000 7f11ada7a640 1 -- 172.21.15.83:0/4289502667 wait complete. 2024-04-17T10:47:28.029 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-17T10:47:28.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.162+0000 7f72f7774640 1 Processor -- start 2024-04-17T10:47:28.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.162+0000 7f72f7774640 1 -- start start 2024-04-17T10:47:28.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.162+0000 7f72f7774640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f72f015e330 0x7f72f015e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:28.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.162+0000 7f72f7774640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72f0070aa0 0x7f72f0070f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:28.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.162+0000 7f72f7774640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72f01625d0 con 0x7f72f006ae10 2024-04-17T10:47:28.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.162+0000 7f72f7774640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72f0161810 con 0x7f72f015e330 2024-04-17T10:47:28.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.162+0000 7f72f7774640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f0161af0 con 0x7f72f0070aa0 2024-04-17T10:47:28.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.162+0000 7f72f5d11640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72f0070aa0 0x7f72f0070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.162+0000 7f72f4d0f640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f72f015e330 0x7f72f015e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.162+0000 7f72f5d11640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72f0070aa0 0x7f72f0070f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:52070/0 (socket says 172.21.15.83:52070) 2024-04-17T10:47:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.162+0000 7f72f5d11640 1 -- 172.21.15.83:0/1225720526 learned_addr learned my addr 172.21.15.83:0/1225720526 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.162+0000 7f72f5d11640 1 -- 172.21.15.83:0/1225720526 >> v2:172.21.15.167:3300/0 conn(0x7f72f015e330 msgr2=0x7f72f015e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.162+0000 7f72f5d11640 1 --2- 172.21.15.83:0/1225720526 >> v2:172.21.15.167:3300/0 conn(0x7f72f015e330 0x7f72f015e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.162+0000 7f72f5d11640 1 -- 172.21.15.83:0/1225720526 >> v1:172.21.15.167:6789/0 conn(0x7f72f006ae10 legacy=0x7f72f006b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.162+0000 7f72f5d11640 1 -- 172.21.15.83:0/1225720526 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f0161dd0 con 0x7f72f0070aa0 2024-04-17T10:47:28.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.166+0000 7f72f5d11640 1 --2- 172.21.15.83:0/1225720526 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72f0070aa0 0x7f72f0070f30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f72e8009870 tx=0x7f72e801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=529c1242be76ebb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:28.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.166+0000 7f72effff640 1 -- 172.21.15.83:0/1225720526 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e80b0070 con 0x7f72f0070aa0 2024-04-17T10:47:28.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.166+0000 7f72effff640 1 -- 172.21.15.83:0/1225720526 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e80ab600 con 0x7f72f0070aa0 2024-04-17T10:47:28.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.166+0000 7f72effff640 1 -- 172.21.15.83:0/1225720526 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e80ab9d0 con 0x7f72f0070aa0 2024-04-17T10:47:28.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.166+0000 7f72f7774640 1 -- 172.21.15.83:0/1225720526 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72f0070aa0 msgr2=0x7f72f0070f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:28.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.166+0000 7f72f7774640 1 --2- 172.21.15.83:0/1225720526 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72f0070aa0 0x7f72f0070f30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f72e8009870 tx=0x7f72e801c410 comp rx=0 tx=0).stop 2024-04-17T10:47:28.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.166+0000 7f72f7774640 1 -- 172.21.15.83:0/1225720526 shutdown_connections 2024-04-17T10:47:28.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.166+0000 7f72f7774640 1 --2- 172.21.15.83:0/1225720526 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72f0070aa0 0x7f72f0070f30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:28.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.166+0000 7f72f7774640 1 --2- 172.21.15.83:0/1225720526 >> v2:172.21.15.167:3300/0 conn(0x7f72f015e330 0x7f72f015e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:28.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.166+0000 7f72f7774640 1 -- 172.21.15.83:0/1225720526 >> 172.21.15.83:0/1225720526 conn(0x7f72f00b15b0 msgr2=0x7f72f00b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:28.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.166+0000 7f72f7774640 1 -- 172.21.15.83:0/1225720526 shutdown_connections 2024-04-17T10:47:28.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.166+0000 7f72f7774640 1 -- 172.21.15.83:0/1225720526 wait complete. 2024-04-17T10:47:28.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.170+0000 7f72f7774640 1 Processor -- start 2024-04-17T10:47:28.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.170+0000 7f72f7774640 1 -- start start 2024-04-17T10:47:28.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.170+0000 7f72f7774640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72f006ae10 0x7f72f015d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:28.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.170+0000 7f72f7774640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72f0070aa0 0x7f72f013f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:28.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.170+0000 7f72f7774640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72f015e330 0x7f72f015b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:28.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.170+0000 7f72f7774640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f016ab90 con 0x7f72f015e330 2024-04-17T10:47:28.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.170+0000 7f72f7774640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f016aa10 con 0x7f72f0070aa0 2024-04-17T10:47:28.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.170+0000 7f72f7774640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f016ad10 con 0x7f72f006ae10 2024-04-17T10:47:28.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.170+0000 7f72f5d11640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72f015e330 0x7f72f015b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:28.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.170+0000 7f72f5d11640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72f015e330 0x7f72f015b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60282/0 (socket says 172.21.15.83:60282) 2024-04-17T10:47:28.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.170+0000 7f72f5d11640 1 -- 172.21.15.83:0/3083064259 learned_addr learned my addr 172.21.15.83:0/3083064259 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:28.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.170+0000 7f72f5510640 1 --2- 172.21.15.83:0/3083064259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72f006ae10 0x7f72f015d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:28.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.170+0000 7f72f4d0f640 1 --2- 172.21.15.83:0/3083064259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72f0070aa0 0x7f72f013f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:28.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.170+0000 7f72f5d11640 1 -- 172.21.15.83:0/3083064259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72f006ae10 msgr2=0x7f72f015d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:28.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.170+0000 7f72f5d11640 1 --2- 172.21.15.83:0/3083064259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72f006ae10 0x7f72f015d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:28.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.170+0000 7f72f5d11640 1 -- 172.21.15.83:0/3083064259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72f0070aa0 msgr2=0x7f72f013f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:28.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.170+0000 7f72f5d11640 1 --2- 172.21.15.83:0/3083064259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72f0070aa0 0x7f72f013f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:28.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.170+0000 7f72f5d11640 1 -- 172.21.15.83:0/3083064259 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f015de30 con 0x7f72f015e330 2024-04-17T10:47:28.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.170+0000 7f72f5d11640 1 --2- 172.21.15.83:0/3083064259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72f015e330 0x7f72f015b1d0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f72e801a040 tx=0x7f72e80191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:28.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.170+0000 7f72edffb640 1 -- 172.21.15.83:0/3083064259 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e80bb070 con 0x7f72f015e330 2024-04-17T10:47:28.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.170+0000 7f72edffb640 1 -- 172.21.15.83:0/3083064259 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e80abe40 con 0x7f72f015e330 2024-04-17T10:47:28.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.170+0000 7f72f7774640 1 -- 172.21.15.83:0/3083064259 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f0159640 con 0x7f72f015e330 2024-04-17T10:47:28.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.170+0000 7f72edffb640 1 -- 172.21.15.83:0/3083064259 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e80b0040 con 0x7f72f015e330 2024-04-17T10:47:28.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.170+0000 7f72f7774640 1 -- 172.21.15.83:0/3083064259 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f0159c00 con 0x7f72f015e330 2024-04-17T10:47:28.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.170+0000 7f72f7774640 1 -- 172.21.15.83:0/3083064259 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72f0074f90 con 0x7f72f015e330 2024-04-17T10:47:28.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.170+0000 7f72edffb640 1 -- 172.21.15.83:0/3083064259 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f72e80b53d0 con 0x7f72f015e330 2024-04-17T10:47:28.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.174+0000 7f72edffb640 1 --2- 172.21.15.83:0/3083064259 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f72dc066f90 0x7f72dc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:28.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.174+0000 7f72edffb640 1 -- 172.21.15.83:0/3083064259 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f72e811e460 con 0x7f72f015e330 2024-04-17T10:47:28.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.174+0000 7f72f5510640 1 --2- 172.21.15.83:0/3083064259 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f72dc066f90 0x7f72dc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:28.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.174+0000 7f72f5510640 1 --2- 172.21.15.83:0/3083064259 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f72dc066f90 0x7f72dc069450 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f72e00015b0 tx=0x7f72e0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:28.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.178+0000 7f72edffb640 1 -- 172.21.15.83:0/3083064259 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f72e8016150 con 0x7f72f015e330 2024-04-17T10:47:28.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:28.334+0000 7f72f7774640 1 -- 172.21.15.83:0/3083064259 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f72f0070880 con 0x7f72f015e330 2024-04-17T10:47:29.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.010+0000 7f72edffb640 1 -- 172.21.15.83:0/3083064259 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v101) v1 ==== 170+0+0 (secure 0 0 0) 0x7f72e80eef80 con 0x7f72f015e330 2024-04-17T10:47:29.008 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2024-04-17T10:47:29.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.014+0000 7f72f7774640 1 -- 172.21.15.83:0/3083064259 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f72dc066f90 msgr2=0x7f72dc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:29.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.014+0000 7f72f7774640 1 --2- 172.21.15.83:0/3083064259 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f72dc066f90 0x7f72dc069450 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f72e00015b0 tx=0x7f72e0009040 comp rx=0 tx=0).stop 2024-04-17T10:47:29.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.014+0000 7f72f7774640 1 -- 172.21.15.83:0/3083064259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72f015e330 msgr2=0x7f72f015b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:29.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.014+0000 7f72f7774640 1 --2- 172.21.15.83:0/3083064259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72f015e330 0x7f72f015b1d0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f72e801a040 tx=0x7f72e80191b0 comp rx=0 tx=0).stop 2024-04-17T10:47:29.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.014+0000 7f72f7774640 1 -- 172.21.15.83:0/3083064259 shutdown_connections 2024-04-17T10:47:29.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.014+0000 7f72f7774640 1 --2- 172.21.15.83:0/3083064259 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f72dc066f90 0x7f72dc069450 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:29.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.014+0000 7f72f7774640 1 --2- 172.21.15.83:0/3083064259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72f015e330 0x7f72f015b1d0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:29.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.014+0000 7f72f7774640 1 --2- 172.21.15.83:0/3083064259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72f0070aa0 0x7f72f013f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:29.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.014+0000 7f72f7774640 1 --2- 172.21.15.83:0/3083064259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72f006ae10 0x7f72f015d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:29.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.014+0000 7f72f7774640 1 -- 172.21.15.83:0/3083064259 >> 172.21.15.83:0/3083064259 conn(0x7f72f00b15b0 msgr2=0x7f72f006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:29.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.014+0000 7f72f7774640 1 -- 172.21.15.83:0/3083064259 shutdown_connections 2024-04-17T10:47:29.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.014+0000 7f72f7774640 1 -- 172.21.15.83:0/3083064259 wait complete. 2024-04-17T10:47:29.045 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-17T10:47:29.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.170+0000 7f1b23b1d640 1 Processor -- start 2024-04-17T10:47:29.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.170+0000 7f1b23b1d640 1 -- start start 2024-04-17T10:47:29.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.170+0000 7f1b23b1d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1b1c0d2300 0x7f1b1c0b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:29.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.170+0000 7f1b23b1d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b1c0cfaa0 0x7f1b1c0cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:29.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.170+0000 7f1b23b1d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b1c0b0a30 con 0x7f1b1c15bba0 2024-04-17T10:47:29.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.170+0000 7f1b23b1d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b1c0d0430 con 0x7f1b1c0d2300 2024-04-17T10:47:29.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.170+0000 7f1b23b1d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c0cf3e0 con 0x7f1b1c0cfaa0 2024-04-17T10:47:29.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.170+0000 7f1b210b8640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1b1c0d2300 0x7f1b1c0b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:29.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.170+0000 7f1b218b9640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1b1c15bba0 0x7f1b1c0cc280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41578/0 (socket says 172.21.15.83:41578) 2024-04-17T10:47:29.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.170+0000 7f1b218b9640 1 -- 172.21.15.83:0/4020504339 learned_addr learned my addr 172.21.15.83:0/4020504339 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:29.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.170+0000 7f1b220ba640 1 --2- 172.21.15.83:0/4020504339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b1c0cfaa0 0x7f1b1c0cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:29.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.170+0000 7f1b210b8640 1 -- 172.21.15.83:0/4020504339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b1c0cfaa0 msgr2=0x7f1b1c0cfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:29.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.170+0000 7f1b210b8640 1 --2- 172.21.15.83:0/4020504339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b1c0cfaa0 0x7f1b1c0cfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:29.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.170+0000 7f1b210b8640 1 -- 172.21.15.83:0/4020504339 >> v1:172.21.15.83:6789/0 conn(0x7f1b1c15bba0 legacy=0x7f1b1c0cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:29.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.170+0000 7f1b210b8640 1 -- 172.21.15.83:0/4020504339 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b1c0cf630 con 0x7f1b1c0d2300 2024-04-17T10:47:29.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.170+0000 7f1b208b7640 1 -- 172.21.15.83:0/4020504339 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4114857454 0 0) 0x7f1b1c0b0a30 con 0x7f1b1c15bba0 2024-04-17T10:47:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.174+0000 7f1b210b8640 1 --2- 172.21.15.83:0/4020504339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b1c0d2300 0x7f1b1c0b0c70 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f1b18009d90 tx=0x7f1b1801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=847a6b5af054ecbd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.174+0000 7f1b208b7640 1 -- 172.21.15.83:0/4020504339 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b180b2070 con 0x7f1b1c0d2300 2024-04-17T10:47:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.174+0000 7f1b208b7640 1 -- 172.21.15.83:0/4020504339 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b180a58b0 con 0x7f1b1c0d2300 2024-04-17T10:47:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.174+0000 7f1b208b7640 1 -- 172.21.15.83:0/4020504339 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b180a5c80 con 0x7f1b1c0d2300 2024-04-17T10:47:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.174+0000 7f1b23b1d640 1 -- 172.21.15.83:0/4020504339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b1c0d2300 msgr2=0x7f1b1c0b0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.174+0000 7f1b23b1d640 1 --2- 172.21.15.83:0/4020504339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b1c0d2300 0x7f1b1c0b0c70 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f1b18009d90 tx=0x7f1b1801c8e0 comp rx=0 tx=0).stop 2024-04-17T10:47:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.174+0000 7f1b23b1d640 1 -- 172.21.15.83:0/4020504339 shutdown_connections 2024-04-17T10:47:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.174+0000 7f1b23b1d640 1 --2- 172.21.15.83:0/4020504339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b1c0cfaa0 0x7f1b1c0cfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.174+0000 7f1b23b1d640 1 --2- 172.21.15.83:0/4020504339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b1c0d2300 0x7f1b1c0b0c70 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.174+0000 7f1b23b1d640 1 -- 172.21.15.83:0/4020504339 >> 172.21.15.83:0/4020504339 conn(0x7f1b1c0d1810 msgr2=0x7f1b1c0d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.174+0000 7f1b23b1d640 1 -- 172.21.15.83:0/4020504339 shutdown_connections 2024-04-17T10:47:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.174+0000 7f1b23b1d640 1 -- 172.21.15.83:0/4020504339 wait complete. 2024-04-17T10:47:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.174+0000 7f1b23b1d640 1 Processor -- start 2024-04-17T10:47:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.174+0000 7f1b23b1d640 1 -- start start 2024-04-17T10:47:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.174+0000 7f1b23b1d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b1c0cfaa0 0x7f1b1c0b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.174+0000 7f1b23b1d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b1c0d2300 0x7f1b1c1549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.174+0000 7f1b23b1d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b1c15bba0 0x7f1b1c154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.174+0000 7f1b23b1d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c0cee00 con 0x7f1b1c0d2300 2024-04-17T10:47:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.174+0000 7f1b23b1d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c0cec80 con 0x7f1b1c15bba0 2024-04-17T10:47:29.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.174+0000 7f1b23b1d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c0cef80 con 0x7f1b1c0cfaa0 2024-04-17T10:47:29.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.178+0000 7f1b210b8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b1c0d2300 0x7f1b1c1549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:29.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.178+0000 7f1b210b8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b1c0d2300 0x7f1b1c1549d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60312/0 (socket says 172.21.15.83:60312) 2024-04-17T10:47:29.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.178+0000 7f1b210b8640 1 -- 172.21.15.83:0/2068668467 learned_addr learned my addr 172.21.15.83:0/2068668467 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:29.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.178+0000 7f1b220ba640 1 --2- 172.21.15.83:0/2068668467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b1c15bba0 0x7f1b1c154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:29.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.178+0000 7f1b210b8640 1 -- 172.21.15.83:0/2068668467 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b1c0cfaa0 msgr2=0x7f1b1c0b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:29.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.178+0000 7f1b218b9640 1 --2- 172.21.15.83:0/2068668467 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b1c0cfaa0 0x7f1b1c0b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:29.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.178+0000 7f1b210b8640 1 --2- 172.21.15.83:0/2068668467 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b1c0cfaa0 0x7f1b1c0b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:29.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.178+0000 7f1b210b8640 1 -- 172.21.15.83:0/2068668467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b1c15bba0 msgr2=0x7f1b1c154f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:29.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.178+0000 7f1b210b8640 1 --2- 172.21.15.83:0/2068668467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b1c15bba0 0x7f1b1c154f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:29.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.178+0000 7f1b210b8640 1 -- 172.21.15.83:0/2068668467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b1c150f20 con 0x7f1b1c0d2300 2024-04-17T10:47:29.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.178+0000 7f1b220ba640 1 --2- 172.21.15.83:0/2068668467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b1c15bba0 0x7f1b1c154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:29.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.178+0000 7f1b218b9640 1 --2- 172.21.15.83:0/2068668467 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b1c0cfaa0 0x7f1b1c0b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:29.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.178+0000 7f1b210b8640 1 --2- 172.21.15.83:0/2068668467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b1c0d2300 0x7f1b1c1549d0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f1b180b0040 tx=0x7f1b18019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:29.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.178+0000 7f1b127fc640 1 -- 172.21.15.83:0/2068668467 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b180b2050 con 0x7f1b1c0d2300 2024-04-17T10:47:29.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.178+0000 7f1b127fc640 1 -- 172.21.15.83:0/2068668467 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b180be070 con 0x7f1b1c0d2300 2024-04-17T10:47:29.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.178+0000 7f1b23b1d640 1 -- 172.21.15.83:0/2068668467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b1c151200 con 0x7f1b1c0d2300 2024-04-17T10:47:29.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.178+0000 7f1b127fc640 1 -- 172.21.15.83:0/2068668467 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b180ada50 con 0x7f1b1c0d2300 2024-04-17T10:47:29.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.178+0000 7f1b23b1d640 1 -- 172.21.15.83:0/2068668467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b1c0b3f60 con 0x7f1b1c0d2300 2024-04-17T10:47:29.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.178+0000 7f1b23b1d640 1 -- 172.21.15.83:0/2068668467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b1c074f90 con 0x7f1b1c0d2300 2024-04-17T10:47:29.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.178+0000 7f1b127fc640 1 -- 172.21.15.83:0/2068668467 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1b180adbf0 con 0x7f1b1c0d2300 2024-04-17T10:47:29.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.178+0000 7f1b127fc640 1 --2- 172.21.15.83:0/2068668467 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1b00066f60 0x7f1b00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:29.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.178+0000 7f1b127fc640 1 -- 172.21.15.83:0/2068668467 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f1b18120af0 con 0x7f1b1c0d2300 2024-04-17T10:47:29.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.178+0000 7f1b218b9640 1 --2- 172.21.15.83:0/2068668467 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1b00066f60 0x7f1b00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:29.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.178+0000 7f1b218b9640 1 --2- 172.21.15.83:0/2068668467 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1b00066f60 0x7f1b00069420 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c000f70 tx=0x7f1b0c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:29.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.182+0000 7f1b127fc640 1 -- 172.21.15.83:0/2068668467 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1b18016560 con 0x7f1b1c0d2300 2024-04-17T10:47:29.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:29.378+0000 7f1b23b1d640 1 -- 172.21.15.83:0/2068668467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1b1c0cdda0 con 0x7f1b1c0d2300 2024-04-17T10:47:30.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.014+0000 7f1b127fc640 1 -- 172.21.15.83:0/2068668467 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v60) v1 ==== 174+0+0 (secure 0 0 0) 0x7f1b180f1cd0 con 0x7f1b1c0d2300 2024-04-17T10:47:30.011 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 14 and data pool 15 2024-04-17T10:47:30.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.018+0000 7f1b23b1d640 1 -- 172.21.15.83:0/2068668467 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1b00066f60 msgr2=0x7f1b00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:30.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.018+0000 7f1b23b1d640 1 --2- 172.21.15.83:0/2068668467 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1b00066f60 0x7f1b00069420 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c000f70 tx=0x7f1b0c009040 comp rx=0 tx=0).stop 2024-04-17T10:47:30.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.018+0000 7f1b23b1d640 1 -- 172.21.15.83:0/2068668467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b1c0d2300 msgr2=0x7f1b1c1549d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:30.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.018+0000 7f1b23b1d640 1 --2- 172.21.15.83:0/2068668467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b1c0d2300 0x7f1b1c1549d0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f1b180b0040 tx=0x7f1b18019680 comp rx=0 tx=0).stop 2024-04-17T10:47:30.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.018+0000 7f1b23b1d640 1 -- 172.21.15.83:0/2068668467 shutdown_connections 2024-04-17T10:47:30.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.018+0000 7f1b23b1d640 1 --2- 172.21.15.83:0/2068668467 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1b00066f60 0x7f1b00069420 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:30.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.018+0000 7f1b23b1d640 1 --2- 172.21.15.83:0/2068668467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b1c15bba0 0x7f1b1c154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:30.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.018+0000 7f1b23b1d640 1 --2- 172.21.15.83:0/2068668467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b1c0d2300 0x7f1b1c1549d0 secure :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f1b180b0040 tx=0x7f1b18019680 comp rx=0 tx=0).stop 2024-04-17T10:47:30.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.018+0000 7f1b23b1d640 1 --2- 172.21.15.83:0/2068668467 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b1c0cfaa0 0x7f1b1c0b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:30.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.018+0000 7f1b23b1d640 1 -- 172.21.15.83:0/2068668467 >> 172.21.15.83:0/2068668467 conn(0x7f1b1c0d1810 msgr2=0x7f1b1c0d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:30.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.026+0000 7f1b23b1d640 1 -- 172.21.15.83:0/2068668467 shutdown_connections 2024-04-17T10:47:30.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.026+0000 7f1b23b1d640 1 -- 172.21.15.83:0/2068668467 wait complete. 2024-04-17T10:47:30.054 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:47:30.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f581a32b640 1 Processor -- start 2024-04-17T10:47:30.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f581a32b640 1 -- start start 2024-04-17T10:47:30.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f581a32b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f581415bac0 0x7f58140afe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:30.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f581a32b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58140d0b40 con 0x7f58140d0610 2024-04-17T10:47:30.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f581a32b640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58140d0e20 con 0x7f581406abe0 2024-04-17T10:47:30.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f581a32b640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58140d1100 con 0x7f581415bac0 2024-04-17T10:47:30.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f58188c8640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f58140d0610 0x7f58140cec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41588/0 (socket says 172.21.15.83:41588) 2024-04-17T10:47:30.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f58188c8640 1 -- 172.21.15.83:0/4235191916 learned_addr learned my addr 172.21.15.83:0/4235191916 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:30.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f58137fe640 1 -- 172.21.15.83:0/4235191916 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1507791026 0 0) 0x7f58140d0b40 con 0x7f58140d0610 2024-04-17T10:47:30.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f58137fe640 1 -- 172.21.15.83:0/4235191916 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5800003530 con 0x7f58140d0610 2024-04-17T10:47:30.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f580b7fe640 1 --2- 172.21.15.83:0/4235191916 >> v2:172.21.15.167:3300/0 conn(0x7f581415bac0 0x7f58140afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:30.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f58137fe640 1 -- 172.21.15.83:0/4235191916 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1773961762 0 0) 0x7f5800003530 con 0x7f58140d0610 2024-04-17T10:47:30.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f58137fe640 1 -- 172.21.15.83:0/4235191916 >> v2:172.21.15.167:3300/0 conn(0x7f581415bac0 msgr2=0x7f58140afe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:30.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f58137fe640 1 --2- 172.21.15.83:0/4235191916 >> v2:172.21.15.167:3300/0 conn(0x7f581415bac0 0x7f58140afe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:30.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f58137fe640 1 -- 172.21.15.83:0/4235191916 >> v1:172.21.15.167:6789/0 conn(0x7f581406abe0 legacy=0x7f581406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:30.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f58137fe640 1 -- 172.21.15.83:0/4235191916 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58140d13e0 con 0x7f58140d0610 2024-04-17T10:47:30.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f58137fe640 1 -- 172.21.15.83:0/4235191916 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f580c001a00 con 0x7f58140d0610 2024-04-17T10:47:30.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f58137fe640 1 -- 172.21.15.83:0/4235191916 >> v1:172.21.15.83:6789/0 conn(0x7f58140d0610 legacy=0x7f58140cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:30.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f58137fe640 1 --2- 172.21.15.83:0/4235191916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5800003a10 0x7f5800003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:30.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f58137fe640 1 --2- 172.21.15.83:0/4235191916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5800004810 0x7f5800004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:30.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f58137fe640 1 --2- 172.21.15.83:0/4235191916 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5800005220 0x7f580000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:30.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f58137fe640 1 -- 172.21.15.83:0/4235191916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58140d1100 con 0x7f5800003a10 2024-04-17T10:47:30.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f58137fe640 1 -- 172.21.15.83:0/4235191916 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580000c170 con 0x7f5800004810 2024-04-17T10:47:30.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f58137fe640 1 -- 172.21.15.83:0/4235191916 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580000c370 con 0x7f5800005220 2024-04-17T10:47:30.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f5813fff640 1 --2- 172.21.15.83:0/4235191916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5800003a10 0x7f5800003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:30.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f580b7fe640 1 --2- 172.21.15.83:0/4235191916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5800004810 0x7f5800004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:30.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f58188c8640 1 --2- 172.21.15.83:0/4235191916 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5800005220 0x7f580000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:30.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f5813fff640 1 -- 172.21.15.83:0/4235191916 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5800005220 msgr2=0x7f580000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:30.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f5813fff640 1 --2- 172.21.15.83:0/4235191916 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5800005220 0x7f580000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:30.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f5813fff640 1 -- 172.21.15.83:0/4235191916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5800004810 msgr2=0x7f5800004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:30.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f5813fff640 1 --2- 172.21.15.83:0/4235191916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5800004810 0x7f5800004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:30.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f5813fff640 1 -- 172.21.15.83:0/4235191916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5800003530 con 0x7f5800003a10 2024-04-17T10:47:30.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f58188c8640 1 -- 172.21.15.83:0/4235191916 reap_dead start 2024-04-17T10:47:30.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.174+0000 7f580b7fe640 1 --2- 172.21.15.83:0/4235191916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5800004810 0x7f5800004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:30.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.178+0000 7f5813fff640 1 --2- 172.21.15.83:0/4235191916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5800003a10 0x7f5800003e30 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f5804001780 tx=0x7f580401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33443df3bd6e7c6b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:30.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.178+0000 7f58137fe640 1 -- 172.21.15.83:0/4235191916 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58040013a0 con 0x7f5800003a10 2024-04-17T10:47:30.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.178+0000 7f58137fe640 1 -- 172.21.15.83:0/4235191916 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58040052f0 con 0x7f5800003a10 2024-04-17T10:47:30.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.178+0000 7f58137fe640 1 -- 172.21.15.83:0/4235191916 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5804003840 con 0x7f5800003a10 2024-04-17T10:47:30.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.178+0000 7f581a32b640 1 -- 172.21.15.83:0/4235191916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5800003a10 msgr2=0x7f5800003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:30.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.178+0000 7f581a32b640 1 --2- 172.21.15.83:0/4235191916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5800003a10 0x7f5800003e30 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f5804001780 tx=0x7f580401c400 comp rx=0 tx=0).stop 2024-04-17T10:47:30.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.178+0000 7f581a32b640 1 -- 172.21.15.83:0/4235191916 shutdown_connections 2024-04-17T10:47:30.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.178+0000 7f581a32b640 1 --2- 172.21.15.83:0/4235191916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5800003a10 0x7f5800003e30 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:30.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.178+0000 7f581a32b640 1 -- 172.21.15.83:0/4235191916 >> 172.21.15.83:0/4235191916 conn(0x7f581406bcf0 msgr2=0x7f581406c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:30.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.178+0000 7f581a32b640 1 -- 172.21.15.83:0/4235191916 shutdown_connections 2024-04-17T10:47:30.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.178+0000 7f581a32b640 1 -- 172.21.15.83:0/4235191916 wait complete. 2024-04-17T10:47:30.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.178+0000 7f581a32b640 1 Processor -- start 2024-04-17T10:47:30.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.178+0000 7f581a32b640 1 -- start start 2024-04-17T10:47:30.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.182+0000 7f581a32b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5800003a10 0x7f58140b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:30.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.182+0000 7f581a32b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f581415c6c0 0x7f581415cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:30.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.182+0000 7f581a32b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5814145f00 0x7f5814146380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:30.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.182+0000 7f581a32b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581406df30 con 0x7f5814145f00 2024-04-17T10:47:30.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.182+0000 7f581a32b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581406ddb0 con 0x7f581415c6c0 2024-04-17T10:47:30.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.182+0000 7f581a32b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581406e0b0 con 0x7f5800003a10 2024-04-17T10:47:30.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.182+0000 7f58188c8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5814145f00 0x7f5814146380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:30.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.182+0000 7f58188c8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5814145f00 0x7f5814146380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60324/0 (socket says 172.21.15.83:60324) 2024-04-17T10:47:30.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.182+0000 7f58188c8640 1 -- 172.21.15.83:0/2104886449 learned_addr learned my addr 172.21.15.83:0/2104886449 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:30.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.182+0000 7f580b7fe640 1 --2- 172.21.15.83:0/2104886449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f581415c6c0 0x7f581415cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:30.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.182+0000 7f5813fff640 1 --2- 172.21.15.83:0/2104886449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5800003a10 0x7f58140b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:30.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.182+0000 7f58188c8640 1 -- 172.21.15.83:0/2104886449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5800003a10 msgr2=0x7f58140b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:30.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.182+0000 7f58188c8640 1 --2- 172.21.15.83:0/2104886449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5800003a10 0x7f58140b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:30.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.182+0000 7f58188c8640 1 -- 172.21.15.83:0/2104886449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f581415c6c0 msgr2=0x7f581415cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:30.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.182+0000 7f58188c8640 1 --2- 172.21.15.83:0/2104886449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f581415c6c0 0x7f581415cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:30.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.182+0000 7f58188c8640 1 -- 172.21.15.83:0/2104886449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58141468c0 con 0x7f5814145f00 2024-04-17T10:47:30.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.182+0000 7f58188c8640 1 --2- 172.21.15.83:0/2104886449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5814145f00 0x7f5814146380 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f581406ba50 tx=0x7f580c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:30.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.182+0000 7f58117fa640 1 -- 172.21.15.83:0/2104886449 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c004d70 con 0x7f5814145f00 2024-04-17T10:47:30.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.182+0000 7f58117fa640 1 -- 172.21.15.83:0/2104886449 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580c00bb80 con 0x7f5814145f00 2024-04-17T10:47:30.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.182+0000 7f581a32b640 1 -- 172.21.15.83:0/2104886449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f581415d060 con 0x7f5814145f00 2024-04-17T10:47:30.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.182+0000 7f58117fa640 1 -- 172.21.15.83:0/2104886449 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c007070 con 0x7f5814145f00 2024-04-17T10:47:30.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.182+0000 7f581a32b640 1 -- 172.21.15.83:0/2104886449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f581415f500 con 0x7f5814145f00 2024-04-17T10:47:30.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.182+0000 7f581a32b640 1 -- 172.21.15.83:0/2104886449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5814074f90 con 0x7f5814145f00 2024-04-17T10:47:30.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.182+0000 7f58117fa640 1 -- 172.21.15.83:0/2104886449 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f580c01b020 con 0x7f5814145f00 2024-04-17T10:47:30.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.182+0000 7f58117fa640 1 --2- 172.21.15.83:0/2104886449 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f57f4066f90 0x7f57f4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:30.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.182+0000 7f58117fa640 1 -- 172.21.15.83:0/2104886449 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f580c0851f0 con 0x7f5814145f00 2024-04-17T10:47:30.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.182+0000 7f5813fff640 1 --2- 172.21.15.83:0/2104886449 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f57f4066f90 0x7f57f4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:30.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.182+0000 7f5813fff640 1 --2- 172.21.15.83:0/2104886449 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f57f4066f90 0x7f57f4069450 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5804001750 tx=0x7f58040011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:30.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.186+0000 7f58117fa640 1 -- 172.21.15.83:0/2104886449 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f580c056170 con 0x7f5814145f00 2024-04-17T10:47:30.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.350+0000 7f581a32b640 1 -- 172.21.15.83:0/2104886449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5814159790 con 0x7f5814145f00 2024-04-17T10:47:30.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.350+0000 7f58117fa640 1 -- 172.21.15.83:0/2104886449 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16560 (secure 0 0 0) 0x7f580c055a70 con 0x7f5814145f00 2024-04-17T10:47:30.348 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:47:30.348 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":102,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:47:29.384360+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:47:27.619432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-17T10:47:28.379472+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:47:30.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.354+0000 7f581a32b640 1 -- 172.21.15.83:0/2104886449 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f57f4066f90 msgr2=0x7f57f4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:30.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.354+0000 7f581a32b640 1 --2- 172.21.15.83:0/2104886449 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f57f4066f90 0x7f57f4069450 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5804001750 tx=0x7f58040011a0 comp rx=0 tx=0).stop 2024-04-17T10:47:30.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.354+0000 7f581a32b640 1 -- 172.21.15.83:0/2104886449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5814145f00 msgr2=0x7f5814146380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:30.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.354+0000 7f581a32b640 1 --2- 172.21.15.83:0/2104886449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5814145f00 0x7f5814146380 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f581406ba50 tx=0x7f580c00b4a0 comp rx=0 tx=0).stop 2024-04-17T10:47:30.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.354+0000 7f581a32b640 1 -- 172.21.15.83:0/2104886449 shutdown_connections 2024-04-17T10:47:30.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.354+0000 7f581a32b640 1 --2- 172.21.15.83:0/2104886449 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f57f4066f90 0x7f57f4069450 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:30.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.354+0000 7f581a32b640 1 --2- 172.21.15.83:0/2104886449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5814145f00 0x7f5814146380 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:30.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.354+0000 7f581a32b640 1 --2- 172.21.15.83:0/2104886449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f581415c6c0 0x7f581415cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:30.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.354+0000 7f581a32b640 1 --2- 172.21.15.83:0/2104886449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5800003a10 0x7f58140b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:30.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.354+0000 7f581a32b640 1 -- 172.21.15.83:0/2104886449 >> 172.21.15.83:0/2104886449 conn(0x7f581406bcf0 msgr2=0x7f5814159cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:30.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.354+0000 7f581a32b640 1 -- 172.21.15.83:0/2104886449 shutdown_connections 2024-04-17T10:47:30.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.354+0000 7f581a32b640 1 -- 172.21.15.83:0/2104886449 wait complete. 2024-04-17T10:47:30.368 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:47:30.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.494+0000 7fd678ad1640 1 Processor -- start 2024-04-17T10:47:30.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.494+0000 7fd678ad1640 1 -- start start 2024-04-17T10:47:30.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.494+0000 7fd678ad1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd67415dac0 0x7fd67406e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:30.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.494+0000 7fd678ad1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd674161e20 0x7fd6740cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:30.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.494+0000 7fd678ad1640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd67406bd90 con 0x7fd67406eb20 2024-04-17T10:47:30.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.494+0000 7fd678ad1640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd67406c9a0 con 0x7fd674161e20 2024-04-17T10:47:30.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.494+0000 7fd678ad1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd67406c070 con 0x7fd67415dac0 2024-04-17T10:47:30.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.494+0000 7fd672d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd674161e20 0x7fd6740cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:30.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.494+0000 7fd671d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd67406eb20 0x7fd674161710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41592/0 (socket says 172.21.15.83:41592) 2024-04-17T10:47:30.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.494+0000 7fd671d9b640 1 -- 172.21.15.83:0/1371397946 learned_addr learned my addr 172.21.15.83:0/1371397946 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:30.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.494+0000 7fd67259c640 1 --2- 172.21.15.83:0/1371397946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd67415dac0 0x7fd67406e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:30.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.494+0000 7fd672d9d640 1 -- 172.21.15.83:0/1371397946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd67415dac0 msgr2=0x7fd67406e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:30.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.494+0000 7fd672d9d640 1 --2- 172.21.15.83:0/1371397946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd67415dac0 0x7fd67406e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:30.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.494+0000 7fd672d9d640 1 -- 172.21.15.83:0/1371397946 >> v1:172.21.15.83:6789/0 conn(0x7fd67406eb20 legacy=0x7fd674161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:30.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.494+0000 7fd672d9d640 1 -- 172.21.15.83:0/1371397946 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd67406c350 con 0x7fd674161e20 2024-04-17T10:47:30.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.494+0000 7fd67159a640 1 -- 172.21.15.83:0/1371397946 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 366339760 0 0) 0x7fd67406bd90 con 0x7fd67406eb20 2024-04-17T10:47:30.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.498+0000 7fd672d9d640 1 --2- 172.21.15.83:0/1371397946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd674161e20 0x7fd6740cf8a0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fd66c009d90 tx=0x7fd66c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=140fa3164cf9fe3c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:30.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.498+0000 7fd67159a640 1 -- 172.21.15.83:0/1371397946 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c0b2070 con 0x7fd674161e20 2024-04-17T10:47:30.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.498+0000 7fd67159a640 1 -- 172.21.15.83:0/1371397946 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66c0a5a20 con 0x7fd674161e20 2024-04-17T10:47:30.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.498+0000 7fd67159a640 1 -- 172.21.15.83:0/1371397946 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c0a5df0 con 0x7fd674161e20 2024-04-17T10:47:30.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.498+0000 7fd678ad1640 1 -- 172.21.15.83:0/1371397946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd674161e20 msgr2=0x7fd6740cf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:30.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.498+0000 7fd678ad1640 1 --2- 172.21.15.83:0/1371397946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd674161e20 0x7fd6740cf8a0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fd66c009d90 tx=0x7fd66c01c940 comp rx=0 tx=0).stop 2024-04-17T10:47:30.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.498+0000 7fd678ad1640 1 -- 172.21.15.83:0/1371397946 shutdown_connections 2024-04-17T10:47:30.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.498+0000 7fd678ad1640 1 --2- 172.21.15.83:0/1371397946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd674161e20 0x7fd6740cf8a0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:30.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.498+0000 7fd678ad1640 1 --2- 172.21.15.83:0/1371397946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd67415dac0 0x7fd67406e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:30.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.498+0000 7fd678ad1640 1 -- 172.21.15.83:0/1371397946 >> 172.21.15.83:0/1371397946 conn(0x7fd6740b3220 msgr2=0x7fd6740b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:30.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.498+0000 7fd678ad1640 1 -- 172.21.15.83:0/1371397946 shutdown_connections 2024-04-17T10:47:30.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.498+0000 7fd678ad1640 1 -- 172.21.15.83:0/1371397946 wait complete. 2024-04-17T10:47:30.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.498+0000 7fd678ad1640 1 Processor -- start 2024-04-17T10:47:30.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.498+0000 7fd678ad1640 1 -- start start 2024-04-17T10:47:30.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.498+0000 7fd678ad1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd67406eb20 0x7fd674155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:30.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.498+0000 7fd678ad1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd67415dac0 0x7fd67415ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:30.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.498+0000 7fd678ad1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd674161e20 0x7fd674160480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:30.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.498+0000 7fd678ad1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd674162510 con 0x7fd674161e20 2024-04-17T10:47:30.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.498+0000 7fd678ad1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd674162390 con 0x7fd67406eb20 2024-04-17T10:47:30.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.498+0000 7fd678ad1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd67416aab0 con 0x7fd67415dac0 2024-04-17T10:47:30.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.498+0000 7fd672d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd674161e20 0x7fd674160480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:30.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.498+0000 7fd672d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd674161e20 0x7fd674160480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60342/0 (socket says 172.21.15.83:60342) 2024-04-17T10:47:30.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.498+0000 7fd672d9d640 1 -- 172.21.15.83:0/1193355655 learned_addr learned my addr 172.21.15.83:0/1193355655 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:30.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.498+0000 7fd671d9b640 1 --2- 172.21.15.83:0/1193355655 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd67415dac0 0x7fd67415ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:30.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.498+0000 7fd67259c640 1 --2- 172.21.15.83:0/1193355655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd67406eb20 0x7fd674155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:30.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.498+0000 7fd672d9d640 1 -- 172.21.15.83:0/1193355655 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd67415dac0 msgr2=0x7fd67415ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:30.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.498+0000 7fd672d9d640 1 --2- 172.21.15.83:0/1193355655 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd67415dac0 0x7fd67415ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:30.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.498+0000 7fd672d9d640 1 -- 172.21.15.83:0/1193355655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd67406eb20 msgr2=0x7fd674155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:30.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.498+0000 7fd672d9d640 1 --2- 172.21.15.83:0/1193355655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd67406eb20 0x7fd674155d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:30.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.498+0000 7fd672d9d640 1 -- 172.21.15.83:0/1193355655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd674156250 con 0x7fd674161e20 2024-04-17T10:47:30.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.502+0000 7fd672d9d640 1 --2- 172.21.15.83:0/1193355655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd674161e20 0x7fd674160480 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fd66c0b0040 tx=0x7fd66c0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:30.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.502+0000 7fd662ffd640 1 -- 172.21.15.83:0/1193355655 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c0b2050 con 0x7fd674161e20 2024-04-17T10:47:30.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.502+0000 7fd662ffd640 1 -- 172.21.15.83:0/1193355655 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66c0be070 con 0x7fd674161e20 2024-04-17T10:47:30.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.502+0000 7fd678ad1640 1 -- 172.21.15.83:0/1193355655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd674158000 con 0x7fd674161e20 2024-04-17T10:47:30.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.502+0000 7fd662ffd640 1 -- 172.21.15.83:0/1193355655 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c0adaf0 con 0x7fd674161e20 2024-04-17T10:47:30.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.502+0000 7fd678ad1640 1 -- 172.21.15.83:0/1193355655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd67415e570 con 0x7fd674161e20 2024-04-17T10:47:30.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.502+0000 7fd678ad1640 1 -- 172.21.15.83:0/1193355655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd674074f90 con 0x7fd674161e20 2024-04-17T10:47:30.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.502+0000 7fd662ffd640 1 -- 172.21.15.83:0/1193355655 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd66c0adc90 con 0x7fd674161e20 2024-04-17T10:47:30.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.502+0000 7fd662ffd640 1 --2- 172.21.15.83:0/1193355655 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd654066f90 0x7fd654069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:30.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.502+0000 7fd662ffd640 1 -- 172.21.15.83:0/1193355655 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd66c120360 con 0x7fd674161e20 2024-04-17T10:47:30.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.502+0000 7fd67259c640 1 --2- 172.21.15.83:0/1193355655 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd654066f90 0x7fd654069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:30.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.502+0000 7fd67259c640 1 --2- 172.21.15.83:0/1193355655 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd654066f90 0x7fd654069450 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd664009790 tx=0x7fd664008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:30.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.506+0000 7fd662ffd640 1 -- 172.21.15.83:0/1193355655 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd674158000 con 0x7fd674161e20 2024-04-17T10:47:30.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.666+0000 7fd678ad1640 1 -- 172.21.15.83:0/1193355655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6740730a0 con 0x7fd674161e20 2024-04-17T10:47:30.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.670+0000 7fd662ffd640 1 -- 172.21.15.83:0/1193355655 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16560 (secure 0 0 0) 0x7fd66c016560 con 0x7fd674161e20 2024-04-17T10:47:30.666 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:47:30.666 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":102,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:47:29.384360+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:47:27.619432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-17T10:47:28.379472+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:47:30.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.670+0000 7fd678ad1640 1 -- 172.21.15.83:0/1193355655 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd654066f90 msgr2=0x7fd654069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:30.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.670+0000 7fd678ad1640 1 --2- 172.21.15.83:0/1193355655 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd654066f90 0x7fd654069450 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd664009790 tx=0x7fd664008040 comp rx=0 tx=0).stop 2024-04-17T10:47:30.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.670+0000 7fd678ad1640 1 -- 172.21.15.83:0/1193355655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd674161e20 msgr2=0x7fd674160480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:30.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.670+0000 7fd678ad1640 1 --2- 172.21.15.83:0/1193355655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd674161e20 0x7fd674160480 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fd66c0b0040 tx=0x7fd66c0196e0 comp rx=0 tx=0).stop 2024-04-17T10:47:30.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.670+0000 7fd678ad1640 1 -- 172.21.15.83:0/1193355655 shutdown_connections 2024-04-17T10:47:30.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.670+0000 7fd678ad1640 1 --2- 172.21.15.83:0/1193355655 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd654066f90 0x7fd654069450 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:30.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.670+0000 7fd678ad1640 1 --2- 172.21.15.83:0/1193355655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd674161e20 0x7fd674160480 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:30.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.670+0000 7fd678ad1640 1 --2- 172.21.15.83:0/1193355655 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd67415dac0 0x7fd67415ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:30.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.670+0000 7fd678ad1640 1 --2- 172.21.15.83:0/1193355655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd67406eb20 0x7fd674155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:30.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.670+0000 7fd678ad1640 1 -- 172.21.15.83:0/1193355655 >> 172.21.15.83:0/1193355655 conn(0x7fd6740b3220 msgr2=0x7fd6740b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:30.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.670+0000 7fd678ad1640 1 -- 172.21.15.83:0/1193355655 shutdown_connections 2024-04-17T10:47:30.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.670+0000 7fd678ad1640 1 -- 172.21.15.83:0/1193355655 wait complete. 2024-04-17T10:47:30.685 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-17T10:47:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.802+0000 7fa622d23640 1 Processor -- start 2024-04-17T10:47:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.802+0000 7fa622d23640 1 -- start start 2024-04-17T10:47:30.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.802+0000 7fa622d23640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa61c06c7d0 0x7fa61c15af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:30.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.802+0000 7fa622d23640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa61c06cbb0 0x7fa61c0b4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:30.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.802+0000 7fa622d23640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa61c06ae50 0x7fa61c150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:30.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.802+0000 7fa622d23640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa61c15b4d0 con 0x7fa61c06cbb0 2024-04-17T10:47:30.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.802+0000 7fa622d23640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa61c0b47d0 con 0x7fa61c06ae50 2024-04-17T10:47:30.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.802+0000 7fa622d23640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c0aece0 con 0x7fa61c06c7d0 2024-04-17T10:47:30.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.802+0000 7fa61bfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa61c06cbb0 0x7fa61c0b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:30.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.802+0000 7fa61bfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa61c06cbb0 0x7fa61c0b4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60356/0 (socket says 172.21.15.83:60356) 2024-04-17T10:47:30.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.802+0000 7fa61bfff640 1 -- 172.21.15.83:0/68746369 learned_addr learned my addr 172.21.15.83:0/68746369 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:30.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.802+0000 7fa6212c0640 1 --2- 172.21.15.83:0/68746369 >> v2:172.21.15.167:3300/0 conn(0x7fa61c06ae50 0x7fa61c150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:30.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.802+0000 7fa620abf640 1 --2- 172.21.15.83:0/68746369 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa61c06c7d0 0x7fa61c15af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:30.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.802+0000 7fa61bfff640 1 -- 172.21.15.83:0/68746369 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa61c06c7d0 msgr2=0x7fa61c15af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:30.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.802+0000 7fa61bfff640 1 --2- 172.21.15.83:0/68746369 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa61c06c7d0 0x7fa61c15af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:30.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.802+0000 7fa61bfff640 1 -- 172.21.15.83:0/68746369 >> v2:172.21.15.167:3300/0 conn(0x7fa61c06ae50 msgr2=0x7fa61c150ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:30.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.802+0000 7fa61bfff640 1 --2- 172.21.15.83:0/68746369 >> v2:172.21.15.167:3300/0 conn(0x7fa61c06ae50 0x7fa61c150ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:30.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.802+0000 7fa61bfff640 1 -- 172.21.15.83:0/68746369 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa61c0b31f0 con 0x7fa61c06cbb0 2024-04-17T10:47:30.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.802+0000 7fa6212c0640 1 --2- 172.21.15.83:0/68746369 >> v2:172.21.15.167:3300/0 conn(0x7fa61c06ae50 0x7fa61c150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:30.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.806+0000 7fa61bfff640 1 --2- 172.21.15.83:0/68746369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa61c06cbb0 0x7fa61c0b4290 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fa61c0cf5c0 tx=0x7fa60801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf55d036d8920a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:30.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.806+0000 7fa61b7fe640 1 -- 172.21.15.83:0/68746369 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6080ab620 con 0x7fa61c06cbb0 2024-04-17T10:47:30.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.806+0000 7fa61b7fe640 1 -- 172.21.15.83:0/68746369 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6080b0070 con 0x7fa61c06cbb0 2024-04-17T10:47:30.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.806+0000 7fa61b7fe640 1 -- 172.21.15.83:0/68746369 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6080ab9e0 con 0x7fa61c06cbb0 2024-04-17T10:47:30.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.806+0000 7fa622d23640 1 -- 172.21.15.83:0/68746369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa61c06cbb0 msgr2=0x7fa61c0b4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:30.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.806+0000 7fa622d23640 1 --2- 172.21.15.83:0/68746369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa61c06cbb0 0x7fa61c0b4290 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fa61c0cf5c0 tx=0x7fa60801c460 comp rx=0 tx=0).stop 2024-04-17T10:47:30.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.806+0000 7fa622d23640 1 -- 172.21.15.83:0/68746369 shutdown_connections 2024-04-17T10:47:30.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.806+0000 7fa622d23640 1 --2- 172.21.15.83:0/68746369 >> v2:172.21.15.167:3300/0 conn(0x7fa61c06ae50 0x7fa61c150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:30.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.806+0000 7fa622d23640 1 --2- 172.21.15.83:0/68746369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa61c06cbb0 0x7fa61c0b4290 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:30.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.806+0000 7fa622d23640 1 --2- 172.21.15.83:0/68746369 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa61c06c7d0 0x7fa61c15af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:30.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.806+0000 7fa622d23640 1 -- 172.21.15.83:0/68746369 >> 172.21.15.83:0/68746369 conn(0x7fa61c0b15b0 msgr2=0x7fa61c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:30.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.806+0000 7fa622d23640 1 -- 172.21.15.83:0/68746369 shutdown_connections 2024-04-17T10:47:30.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.806+0000 7fa622d23640 1 -- 172.21.15.83:0/68746369 wait complete. 2024-04-17T10:47:30.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.806+0000 7fa622d23640 1 Processor -- start 2024-04-17T10:47:30.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.806+0000 7fa622d23640 1 -- start start 2024-04-17T10:47:30.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.806+0000 7fa622d23640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa61c06ae50 0x7fa61c14a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:30.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.806+0000 7fa622d23640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa61c06c7d0 0x7fa61c1527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:30.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.806+0000 7fa622d23640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa61c06cbb0 0x7fa61c153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:30.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.806+0000 7fa622d23640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c0b3d70 con 0x7fa61c06c7d0 2024-04-17T10:47:30.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.806+0000 7fa622d23640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c0b3a90 con 0x7fa61c06cbb0 2024-04-17T10:47:30.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.806+0000 7fa622d23640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c0b2e50 con 0x7fa61c06ae50 2024-04-17T10:47:30.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.806+0000 7fa61bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa61c06c7d0 0x7fa61c1527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:30.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.806+0000 7fa61bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa61c06c7d0 0x7fa61c1527c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60368/0 (socket says 172.21.15.83:60368) 2024-04-17T10:47:30.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.806+0000 7fa61bfff640 1 -- 172.21.15.83:0/1927676172 learned_addr learned my addr 172.21.15.83:0/1927676172 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:30.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.806+0000 7fa61bfff640 1 -- 172.21.15.83:0/1927676172 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa61c06ae50 msgr2=0x7fa61c14a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:30.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.806+0000 7fa61bfff640 1 --2- 172.21.15.83:0/1927676172 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa61c06ae50 0x7fa61c14a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:30.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.806+0000 7fa61bfff640 1 -- 172.21.15.83:0/1927676172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa61c06cbb0 msgr2=0x7fa61c153fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:30.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.806+0000 7fa61bfff640 1 --2- 172.21.15.83:0/1927676172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa61c06cbb0 0x7fa61c153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:30.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.806+0000 7fa61bfff640 1 -- 172.21.15.83:0/1927676172 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa61c152f30 con 0x7fa61c06c7d0 2024-04-17T10:47:30.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.810+0000 7fa61bfff640 1 --2- 172.21.15.83:0/1927676172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa61c06c7d0 0x7fa61c1527c0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fa60801ab10 tx=0x7fa60801aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:30.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.810+0000 7fa6197fa640 1 -- 172.21.15.83:0/1927676172 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6080abdb0 con 0x7fa61c06c7d0 2024-04-17T10:47:30.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.810+0000 7fa6197fa640 1 -- 172.21.15.83:0/1927676172 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6080be3f0 con 0x7fa61c06c7d0 2024-04-17T10:47:30.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.810+0000 7fa622d23640 1 -- 172.21.15.83:0/1927676172 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa61c153210 con 0x7fa61c06c7d0 2024-04-17T10:47:30.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.810+0000 7fa6197fa640 1 -- 172.21.15.83:0/1927676172 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6080be7a0 con 0x7fa61c06c7d0 2024-04-17T10:47:30.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.810+0000 7fa622d23640 1 -- 172.21.15.83:0/1927676172 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa61c1537d0 con 0x7fa61c06c7d0 2024-04-17T10:47:30.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.810+0000 7fa622d23640 1 -- 172.21.15.83:0/1927676172 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa61c074f90 con 0x7fa61c06c7d0 2024-04-17T10:47:30.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.810+0000 7fa6197fa640 1 -- 172.21.15.83:0/1927676172 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa6080b43f0 con 0x7fa61c06c7d0 2024-04-17T10:47:30.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.810+0000 7fa6197fa640 1 --2- 172.21.15.83:0/1927676172 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa600066f60 0x7fa600069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:30.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.810+0000 7fa620abf640 1 --2- 172.21.15.83:0/1927676172 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa600066f60 0x7fa600069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:30.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.810+0000 7fa6197fa640 1 -- 172.21.15.83:0/1927676172 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa60811f5b0 con 0x7fa61c06c7d0 2024-04-17T10:47:30.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.810+0000 7fa620abf640 1 --2- 172.21.15.83:0/1927676172 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa600066f60 0x7fa600069420 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa610006b20 tx=0x7fa610008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:30.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.814+0000 7fa6197fa640 1 -- 172.21.15.83:0/1927676172 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa6080c4060 con 0x7fa61c06c7d0 2024-04-17T10:47:30.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:30.990+0000 7fa622d23640 1 -- 172.21.15.83:0/1927676172 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa61c0721b0 con 0x7fa61c06c7d0 2024-04-17T10:47:31.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.034+0000 7fa6197fa640 1 -- 172.21.15.83:0/1927676172 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v61) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa608016110 con 0x7fa61c06c7d0 2024-04-17T10:47:31.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.038+0000 7fa622d23640 1 -- 172.21.15.83:0/1927676172 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa600066f60 msgr2=0x7fa600069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:31.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.038+0000 7fa622d23640 1 --2- 172.21.15.83:0/1927676172 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa600066f60 0x7fa600069420 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa610006b20 tx=0x7fa610008040 comp rx=0 tx=0).stop 2024-04-17T10:47:31.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.038+0000 7fa622d23640 1 -- 172.21.15.83:0/1927676172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa61c06c7d0 msgr2=0x7fa61c1527c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:31.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.038+0000 7fa622d23640 1 --2- 172.21.15.83:0/1927676172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa61c06c7d0 0x7fa61c1527c0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fa60801ab10 tx=0x7fa60801aca0 comp rx=0 tx=0).stop 2024-04-17T10:47:31.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.038+0000 7fa622d23640 1 -- 172.21.15.83:0/1927676172 shutdown_connections 2024-04-17T10:47:31.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.038+0000 7fa622d23640 1 --2- 172.21.15.83:0/1927676172 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa600066f60 0x7fa600069420 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:31.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.038+0000 7fa622d23640 1 --2- 172.21.15.83:0/1927676172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa61c06cbb0 0x7fa61c153fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:31.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.038+0000 7fa622d23640 1 --2- 172.21.15.83:0/1927676172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa61c06c7d0 0x7fa61c1527c0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:31.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.038+0000 7fa622d23640 1 --2- 172.21.15.83:0/1927676172 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa61c06ae50 0x7fa61c14a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:31.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.038+0000 7fa622d23640 1 -- 172.21.15.83:0/1927676172 >> 172.21.15.83:0/1927676172 conn(0x7fa61c0b15b0 msgr2=0x7fa61c151eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:31.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.038+0000 7fa622d23640 1 -- 172.21.15.83:0/1927676172 shutdown_connections 2024-04-17T10:47:31.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.038+0000 7fa622d23640 1 -- 172.21.15.83:0/1927676172 wait complete. 2024-04-17T10:47:31.052 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-17T10:47:31.052 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-17T10:47:31.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f7661fb6640 1 Processor -- start 2024-04-17T10:47:31.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f7661fb6640 1 -- start start 2024-04-17T10:47:31.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f7661fb6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f765c15baf0 0x7f765c0cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:31.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f7661fb6640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f765c0b3cf0 con 0x7f765c06b2f0 2024-04-17T10:47:31.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f7661fb6640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f765c0b3fd0 con 0x7f765c15b710 2024-04-17T10:47:31.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f7661fb6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f765c0b4900 con 0x7f765c15baf0 2024-04-17T10:47:31.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f765affd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f765c06b2f0 0x7f765c0ceb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41602/0 (socket says 172.21.15.83:41602) 2024-04-17T10:47:31.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f765affd640 1 -- 172.21.15.83:0/2643647903 learned_addr learned my addr 172.21.15.83:0/2643647903 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:31.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f765bfff640 1 --2- 172.21.15.83:0/2643647903 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f765c15baf0 0x7f765c0cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:31.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f765a7fc640 1 -- 172.21.15.83:0/2643647903 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3428130192 0 0) 0x7f765c0b3cf0 con 0x7f765c06b2f0 2024-04-17T10:47:31.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f765a7fc640 1 -- 172.21.15.83:0/2643647903 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7648003530 con 0x7f765c06b2f0 2024-04-17T10:47:31.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f765a7fc640 1 -- 172.21.15.83:0/2643647903 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 93586777 0 0) 0x7f7648003530 con 0x7f765c06b2f0 2024-04-17T10:47:31.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f765a7fc640 1 -- 172.21.15.83:0/2643647903 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f765c15baf0 msgr2=0x7f765c0cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:31.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f765a7fc640 1 --2- 172.21.15.83:0/2643647903 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f765c15baf0 0x7f765c0cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:31.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f765a7fc640 1 -- 172.21.15.83:0/2643647903 >> v1:172.21.15.167:6789/0 conn(0x7f765c15b710 legacy=0x7f765c06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:31.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f765a7fc640 1 -- 172.21.15.83:0/2643647903 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765c0b42b0 con 0x7f765c06b2f0 2024-04-17T10:47:31.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f765a7fc640 1 -- 172.21.15.83:0/2643647903 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f7644001680 con 0x7f765c06b2f0 2024-04-17T10:47:31.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f765bfff640 1 --2- 172.21.15.83:0/2643647903 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f765c15baf0 0x7f765c0cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:47:31.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f765a7fc640 1 -- 172.21.15.83:0/2643647903 >> v1:172.21.15.83:6789/0 conn(0x7f765c06b2f0 legacy=0x7f765c0ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:31.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f765a7fc640 1 --2- 172.21.15.83:0/2643647903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7648003a60 0x7f7648003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:31.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f765a7fc640 1 --2- 172.21.15.83:0/2643647903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7648004810 0x7f7648004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:31.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f765a7fc640 1 --2- 172.21.15.83:0/2643647903 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f765c15baf0 0x7f7648005240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:31.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f765a7fc640 1 -- 172.21.15.83:0/2643647903 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765c0b4900 con 0x7f7648004810 2024-04-17T10:47:31.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f765a7fc640 1 -- 172.21.15.83:0/2643647903 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764800bb60 con 0x7f7648003a60 2024-04-17T10:47:31.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f765a7fc640 1 -- 172.21.15.83:0/2643647903 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764800bd30 con 0x7f765c15baf0 2024-04-17T10:47:31.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f765affd640 1 --2- 172.21.15.83:0/2643647903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7648004810 0x7f7648004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:31.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f765affd640 1 -- 172.21.15.83:0/2643647903 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f765c15baf0 msgr2=0x7f7648005240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:31.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f765bfff640 1 --2- 172.21.15.83:0/2643647903 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f765c15baf0 0x7f7648005240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:31.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f765b7fe640 1 --2- 172.21.15.83:0/2643647903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7648003a60 0x7f7648003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:31.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f765affd640 1 --2- 172.21.15.83:0/2643647903 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f765c15baf0 0x7f7648005240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:31.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f765affd640 1 -- 172.21.15.83:0/2643647903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7648003a60 msgr2=0x7f7648003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:31.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f765affd640 1 --2- 172.21.15.83:0/2643647903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7648003a60 0x7f7648003e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:31.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.174+0000 7f765affd640 1 -- 172.21.15.83:0/2643647903 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7648003530 con 0x7f7648004810 2024-04-17T10:47:31.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.178+0000 7f765affd640 1 --2- 172.21.15.83:0/2643647903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7648004810 0x7f7648004c90 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f764400c240 tx=0x7f764401f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9d3a288637e2ef server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:31.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.178+0000 7f765a7fc640 1 -- 172.21.15.83:0/2643647903 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7644007d20 con 0x7f7648004810 2024-04-17T10:47:31.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.178+0000 7f765a7fc640 1 -- 172.21.15.83:0/2643647903 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7644005ae0 con 0x7f7648004810 2024-04-17T10:47:31.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.178+0000 7f765a7fc640 1 -- 172.21.15.83:0/2643647903 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7644007070 con 0x7f7648004810 2024-04-17T10:47:31.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.178+0000 7f7661fb6640 1 -- 172.21.15.83:0/2643647903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7648004810 msgr2=0x7f7648004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:31.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.178+0000 7f7661fb6640 1 --2- 172.21.15.83:0/2643647903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7648004810 0x7f7648004c90 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f764400c240 tx=0x7f764401f3f0 comp rx=0 tx=0).stop 2024-04-17T10:47:31.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.178+0000 7f765bfff640 1 -- 172.21.15.83:0/2643647903 reap_dead start 2024-04-17T10:47:31.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.178+0000 7f7661fb6640 1 -- 172.21.15.83:0/2643647903 shutdown_connections 2024-04-17T10:47:31.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.178+0000 7f7661fb6640 1 -- 172.21.15.83:0/2643647903 >> 172.21.15.83:0/2643647903 conn(0x7f765c06bcf0 msgr2=0x7f765c06c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:31.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.178+0000 7f7661fb6640 1 -- 172.21.15.83:0/2643647903 shutdown_connections 2024-04-17T10:47:31.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.178+0000 7f7661fb6640 1 -- 172.21.15.83:0/2643647903 wait complete. 2024-04-17T10:47:31.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.178+0000 7f7661fb6640 1 Processor -- start 2024-04-17T10:47:31.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.178+0000 7f7661fb6640 1 -- start start 2024-04-17T10:47:31.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.182+0000 7f7661fb6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7648004810 0x7f765c13c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:31.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.182+0000 7f7661fb6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f765c13ee10 0x7f765c14d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:31.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.182+0000 7f7661fb6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f765c13b8b0 0x7f765c13e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:31.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.182+0000 7f7661fb6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765c071e40 con 0x7f765c13b8b0 2024-04-17T10:47:31.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.182+0000 7f7661fb6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765c071cc0 con 0x7f7648004810 2024-04-17T10:47:31.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.182+0000 7f7661fb6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f765c0b37f0 con 0x7f765c13ee10 2024-04-17T10:47:31.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.182+0000 7f765bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f765c13b8b0 0x7f765c13e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:31.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.182+0000 7f765bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f765c13b8b0 0x7f765c13e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60374/0 (socket says 172.21.15.83:60374) 2024-04-17T10:47:31.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.182+0000 7f765bfff640 1 -- 172.21.15.83:0/2683759672 learned_addr learned my addr 172.21.15.83:0/2683759672 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:31.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.182+0000 7f765b7fe640 1 --2- 172.21.15.83:0/2683759672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7648004810 0x7f765c13c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:31.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.182+0000 7f765bfff640 1 -- 172.21.15.83:0/2683759672 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f765c13ee10 msgr2=0x7f765c14d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:31.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.182+0000 7f765affd640 1 --2- 172.21.15.83:0/2683759672 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f765c13ee10 0x7f765c14d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:31.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.182+0000 7f765bfff640 1 --2- 172.21.15.83:0/2683759672 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f765c13ee10 0x7f765c14d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:31.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.182+0000 7f765bfff640 1 -- 172.21.15.83:0/2683759672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7648004810 msgr2=0x7f765c13c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:31.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.182+0000 7f765bfff640 1 --2- 172.21.15.83:0/2683759672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7648004810 0x7f765c13c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:31.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.182+0000 7f765bfff640 1 -- 172.21.15.83:0/2683759672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765c153140 con 0x7f765c13b8b0 2024-04-17T10:47:31.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.182+0000 7f765b7fe640 1 --2- 172.21.15.83:0/2683759672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7648004810 0x7f765c13c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:31.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.182+0000 7f765bfff640 1 --2- 172.21.15.83:0/2683759672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f765c13b8b0 0x7f765c13e080 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f765c0b5ee0 tx=0x7f765400f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:31.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.182+0000 7f7653fff640 1 -- 172.21.15.83:0/2683759672 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7654004440 con 0x7f765c13b8b0 2024-04-17T10:47:31.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.182+0000 7f7653fff640 1 -- 172.21.15.83:0/2683759672 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f765401a070 con 0x7f765c13b8b0 2024-04-17T10:47:31.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.182+0000 7f7661fb6640 1 -- 172.21.15.83:0/2683759672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f765c150200 con 0x7f765c13b8b0 2024-04-17T10:47:31.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.182+0000 7f7653fff640 1 -- 172.21.15.83:0/2683759672 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7654003860 con 0x7f765c13b8b0 2024-04-17T10:47:31.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.182+0000 7f7661fb6640 1 -- 172.21.15.83:0/2683759672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f765c13bd30 con 0x7f765c13b8b0 2024-04-17T10:47:31.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.182+0000 7f7661fb6640 1 -- 172.21.15.83:0/2683759672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f765c0b1cb0 con 0x7f765c13b8b0 2024-04-17T10:47:31.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.182+0000 7f7653fff640 1 -- 172.21.15.83:0/2683759672 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7654012070 con 0x7f765c13b8b0 2024-04-17T10:47:31.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.182+0000 7f7653fff640 1 --2- 172.21.15.83:0/2683759672 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f763c066f90 0x7f763c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:31.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.182+0000 7f765b7fe640 1 --2- 172.21.15.83:0/2683759672 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f763c066f90 0x7f763c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:31.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.182+0000 7f7653fff640 1 -- 172.21.15.83:0/2683759672 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7654084af0 con 0x7f765c13b8b0 2024-04-17T10:47:31.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.182+0000 7f765b7fe640 1 --2- 172.21.15.83:0/2683759672 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f763c066f90 0x7f763c069450 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f764c007fd0 tx=0x7f764c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:31.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.186+0000 7f7653fff640 1 -- 172.21.15.83:0/2683759672 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7654055b00 con 0x7f765c13b8b0 2024-04-17T10:47:31.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:31.382+0000 7f7661fb6640 1 -- 172.21.15.83:0/2683759672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f765c074f90 con 0x7f765c13b8b0 2024-04-17T10:47:32.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.054+0000 7f7653fff640 1 -- 172.21.15.83:0/2683759672 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v62) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7654055400 con 0x7f765c13b8b0 2024-04-17T10:47:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.058+0000 7f7661fb6640 1 -- 172.21.15.83:0/2683759672 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f763c066f90 msgr2=0x7f763c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:32.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.058+0000 7f7661fb6640 1 --2- 172.21.15.83:0/2683759672 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f763c066f90 0x7f763c069450 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f764c007fd0 tx=0x7f764c009040 comp rx=0 tx=0).stop 2024-04-17T10:47:32.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.058+0000 7f7661fb6640 1 -- 172.21.15.83:0/2683759672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f765c13b8b0 msgr2=0x7f765c13e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:32.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.058+0000 7f7661fb6640 1 --2- 172.21.15.83:0/2683759672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f765c13b8b0 0x7f765c13e080 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f765c0b5ee0 tx=0x7f765400f6f0 comp rx=0 tx=0).stop 2024-04-17T10:47:32.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.058+0000 7f7661fb6640 1 -- 172.21.15.83:0/2683759672 shutdown_connections 2024-04-17T10:47:32.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.058+0000 7f7661fb6640 1 --2- 172.21.15.83:0/2683759672 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f763c066f90 0x7f763c069450 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:32.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.058+0000 7f7661fb6640 1 --2- 172.21.15.83:0/2683759672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f765c13b8b0 0x7f765c13e080 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:32.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.058+0000 7f7661fb6640 1 --2- 172.21.15.83:0/2683759672 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f765c13ee10 0x7f765c14d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:32.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.058+0000 7f7661fb6640 1 --2- 172.21.15.83:0/2683759672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7648004810 0x7f765c13c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:32.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.058+0000 7f7661fb6640 1 -- 172.21.15.83:0/2683759672 >> 172.21.15.83:0/2683759672 conn(0x7f765c06bcf0 msgr2=0x7f765c15c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:32.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.058+0000 7f7661fb6640 1 -- 172.21.15.83:0/2683759672 shutdown_connections 2024-04-17T10:47:32.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.058+0000 7f7661fb6640 1 -- 172.21.15.83:0/2683759672 wait complete. 2024-04-17T10:47:32.076 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-17T10:47:32.076 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:47:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.198+0000 7f6a14bd1640 1 Processor -- start 2024-04-17T10:47:32.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.198+0000 7f6a14bd1640 1 -- start start 2024-04-17T10:47:32.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.202+0000 7f6a14bd1640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6a10146a00 0x7f6a10146e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:32.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.202+0000 7f6a14bd1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6a100b2d80 0x7f6a101620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:32.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.202+0000 7f6a14bd1640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a1006daf0 con 0x7f6a1006d690 2024-04-17T10:47:32.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.202+0000 7f6a14bd1640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a10162610 con 0x7f6a100b2d80 2024-04-17T10:47:32.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.202+0000 7f6a14bd1640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a1006c960 con 0x7f6a10146a00 2024-04-17T10:47:32.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.202+0000 7f6a0ed9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6a100b2d80 0x7f6a101620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:32.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.202+0000 7f6a0ed9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6a100b2d80 0x7f6a101620d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60388/0 (socket says 172.21.15.83:60388) 2024-04-17T10:47:32.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.202+0000 7f6a0ed9d640 1 -- 172.21.15.83:0/3247025752 learned_addr learned my addr 172.21.15.83:0/3247025752 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:32.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.202+0000 7f6a0e59c640 1 --2- 172.21.15.83:0/3247025752 >> v2:172.21.15.167:3300/0 conn(0x7f6a10146a00 0x7f6a10146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:32.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.202+0000 7f6a0ed9d640 1 -- 172.21.15.83:0/3247025752 >> v2:172.21.15.167:3300/0 conn(0x7f6a10146a00 msgr2=0x7f6a10146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:32.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.202+0000 7f6a0ed9d640 1 --2- 172.21.15.83:0/3247025752 >> v2:172.21.15.167:3300/0 conn(0x7f6a10146a00 0x7f6a10146e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:32.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.202+0000 7f6a0ed9d640 1 -- 172.21.15.83:0/3247025752 >> v1:172.21.15.167:6789/0 conn(0x7f6a1006d690 legacy=0x7f6a100b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:32.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.202+0000 7f6a0ed9d640 1 -- 172.21.15.83:0/3247025752 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a10147570 con 0x7f6a100b2d80 2024-04-17T10:47:32.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.202+0000 7f6a0ed9d640 1 --2- 172.21.15.83:0/3247025752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a100b2d80 0x7f6a101620d0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f6a1006ffe0 tx=0x7f6a0801c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a56eb076f64e9e48 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:32.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.202+0000 7f6a0d59a640 1 -- 172.21.15.83:0/3247025752 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a080a5780 con 0x7f6a100b2d80 2024-04-17T10:47:32.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.202+0000 7f6a0d59a640 1 -- 172.21.15.83:0/3247025752 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a080a5920 con 0x7f6a100b2d80 2024-04-17T10:47:32.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.202+0000 7f6a0d59a640 1 -- 172.21.15.83:0/3247025752 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a080a5cf0 con 0x7f6a100b2d80 2024-04-17T10:47:32.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a14bd1640 1 -- 172.21.15.83:0/3247025752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a100b2d80 msgr2=0x7f6a101620d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:32.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a14bd1640 1 --2- 172.21.15.83:0/3247025752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a100b2d80 0x7f6a101620d0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f6a1006ffe0 tx=0x7f6a0801c9b0 comp rx=0 tx=0).stop 2024-04-17T10:47:32.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a14bd1640 1 -- 172.21.15.83:0/3247025752 shutdown_connections 2024-04-17T10:47:32.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a14bd1640 1 --2- 172.21.15.83:0/3247025752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a100b2d80 0x7f6a101620d0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:32.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a14bd1640 1 --2- 172.21.15.83:0/3247025752 >> v2:172.21.15.167:3300/0 conn(0x7f6a10146a00 0x7f6a10146e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:32.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a14bd1640 1 -- 172.21.15.83:0/3247025752 >> 172.21.15.83:0/3247025752 conn(0x7f6a1006abe0 msgr2=0x7f6a1006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:32.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a14bd1640 1 -- 172.21.15.83:0/3247025752 shutdown_connections 2024-04-17T10:47:32.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a14bd1640 1 -- 172.21.15.83:0/3247025752 wait complete. 2024-04-17T10:47:32.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a14bd1640 1 Processor -- start 2024-04-17T10:47:32.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a14bd1640 1 -- start start 2024-04-17T10:47:32.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a14bd1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a1006d690 0x7f6a1015eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:32.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a14bd1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a100b2d80 0x7f6a1015bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:32.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a14bd1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a10146a00 0x7f6a1013ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:32.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a14bd1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a10071050 con 0x7f6a1006d690 2024-04-17T10:47:32.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a14bd1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a10070ed0 con 0x7f6a100b2d80 2024-04-17T10:47:32.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a14bd1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1016a9f0 con 0x7f6a10146a00 2024-04-17T10:47:32.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a0e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a1006d690 0x7f6a1015eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:32.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a0e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a1006d690 0x7f6a1015eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60400/0 (socket says 172.21.15.83:60400) 2024-04-17T10:47:32.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a0e59c640 1 -- 172.21.15.83:0/389661387 learned_addr learned my addr 172.21.15.83:0/389661387 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:32.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a0ed9d640 1 --2- 172.21.15.83:0/389661387 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a10146a00 0x7f6a1013ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:32.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a0dd9b640 1 --2- 172.21.15.83:0/389661387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a100b2d80 0x7f6a1015bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:32.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a0e59c640 1 -- 172.21.15.83:0/389661387 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a10146a00 msgr2=0x7f6a1013ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:32.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a0e59c640 1 --2- 172.21.15.83:0/389661387 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a10146a00 0x7f6a1013ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:32.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a0e59c640 1 -- 172.21.15.83:0/389661387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a100b2d80 msgr2=0x7f6a1015bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:32.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a0e59c640 1 --2- 172.21.15.83:0/389661387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a100b2d80 0x7f6a1015bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:32.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a0e59c640 1 -- 172.21.15.83:0/389661387 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a10163400 con 0x7f6a1006d690 2024-04-17T10:47:32.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a0ed9d640 1 --2- 172.21.15.83:0/389661387 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a10146a00 0x7f6a1013ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:32.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a0dd9b640 1 --2- 172.21.15.83:0/389661387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a100b2d80 0x7f6a1015bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:32.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a0e59c640 1 --2- 172.21.15.83:0/389661387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a1006d690 0x7f6a1015eb10 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000d5f0 tx=0x7f6a0000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:32.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a06ffd640 1 -- 172.21.15.83:0/389661387 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a000125f0 con 0x7f6a1006d690 2024-04-17T10:47:32.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a06ffd640 1 -- 172.21.15.83:0/389661387 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a00019070 con 0x7f6a1006d690 2024-04-17T10:47:32.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a14bd1640 1 -- 172.21.15.83:0/389661387 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a101636e0 con 0x7f6a1006d690 2024-04-17T10:47:32.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a06ffd640 1 -- 172.21.15.83:0/389661387 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a000129b0 con 0x7f6a1006d690 2024-04-17T10:47:32.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.206+0000 7f6a14bd1640 1 -- 172.21.15.83:0/389661387 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a10163ca0 con 0x7f6a1006d690 2024-04-17T10:47:32.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.210+0000 7f6a14bd1640 1 -- 172.21.15.83:0/389661387 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a10074f90 con 0x7f6a1006d690 2024-04-17T10:47:32.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.210+0000 7f6a06ffd640 1 -- 172.21.15.83:0/389661387 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6a00012b50 con 0x7f6a1006d690 2024-04-17T10:47:32.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.210+0000 7f6a06ffd640 1 --2- 172.21.15.83:0/389661387 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f69f0066f60 0x7f69f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:32.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.210+0000 7f6a06ffd640 1 -- 172.21.15.83:0/389661387 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a00085440 con 0x7f6a1006d690 2024-04-17T10:47:32.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.210+0000 7f6a0dd9b640 1 --2- 172.21.15.83:0/389661387 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f69f0066f60 0x7f69f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:32.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.210+0000 7f6a0dd9b640 1 --2- 172.21.15.83:0/389661387 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f69f0066f60 0x7f69f0069420 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6a100b0920 tx=0x7f69f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:32.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.214+0000 7f6a06ffd640 1 -- 172.21.15.83:0/389661387 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6a00056450 con 0x7f6a1006d690 2024-04-17T10:47:32.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.402+0000 7f6a14bd1640 1 -- 172.21.15.83:0/389661387 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a100ae5d0 con 0x7f6a1006d690 2024-04-17T10:47:32.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.402+0000 7f6a06ffd640 1 -- 172.21.15.83:0/389661387 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f6a00055d50 con 0x7f6a1006d690 2024-04-17T10:47:32.400 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:47:32.400 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/437268957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":437268957},{"type":"v1","addr":"172.21.15.167:6833","nonce":437268957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":6253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3214640674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3214640674},{"type":"v1","addr":"172.21.15.167:6835","nonce":3214640674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":8046,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/715498323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":715498323},{"type":"v1","addr":"172.21.15.83:6835","nonce":715498323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:47:29.385103+0000","modified":"2024-04-17T10:47:31.389088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8049":{"gid":8049,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2353406669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":2353406669},{"type":"v1","addr":"172.21.15.83:6837","nonce":2353406669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-17T10:47:32.400 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 62 2024-04-17T10:47:32.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.406+0000 7f6a14bd1640 1 -- 172.21.15.83:0/389661387 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f69f0066f60 msgr2=0x7f69f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:32.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.406+0000 7f6a14bd1640 1 --2- 172.21.15.83:0/389661387 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f69f0066f60 0x7f69f0069420 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6a100b0920 tx=0x7f69f8009040 comp rx=0 tx=0).stop 2024-04-17T10:47:32.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.406+0000 7f6a14bd1640 1 -- 172.21.15.83:0/389661387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a1006d690 msgr2=0x7f6a1015eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:32.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.406+0000 7f6a14bd1640 1 --2- 172.21.15.83:0/389661387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a1006d690 0x7f6a1015eb10 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000d5f0 tx=0x7f6a0000dab0 comp rx=0 tx=0).stop 2024-04-17T10:47:32.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.406+0000 7f6a14bd1640 1 -- 172.21.15.83:0/389661387 shutdown_connections 2024-04-17T10:47:32.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.406+0000 7f6a14bd1640 1 --2- 172.21.15.83:0/389661387 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f69f0066f60 0x7f69f0069420 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:32.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.406+0000 7f6a14bd1640 1 --2- 172.21.15.83:0/389661387 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a10146a00 0x7f6a1013ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:32.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.406+0000 7f6a14bd1640 1 --2- 172.21.15.83:0/389661387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a100b2d80 0x7f6a1015bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:32.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.406+0000 7f6a14bd1640 1 --2- 172.21.15.83:0/389661387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a1006d690 0x7f6a1015eb10 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:32.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.406+0000 7f6a14bd1640 1 -- 172.21.15.83:0/389661387 >> 172.21.15.83:0/389661387 conn(0x7f6a1006abe0 msgr2=0x7f6a100ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:32.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.406+0000 7f6a14bd1640 1 -- 172.21.15.83:0/389661387 shutdown_connections 2024-04-17T10:47:32.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.406+0000 7f6a14bd1640 1 -- 172.21.15.83:0/389661387 wait complete. 2024-04-17T10:47:32.417 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:47:32.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.542+0000 7f1986f8c640 1 Processor -- start 2024-04-17T10:47:32.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.542+0000 7f1986f8c640 1 -- start start 2024-04-17T10:47:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.546+0000 7f1986f8c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1980146b40 0x7f198006d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.546+0000 7f1986f8c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f198006abe0 0x7f198006b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.546+0000 7f1986f8c640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1980161ae0 con 0x7f1980146f20 2024-04-17T10:47:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.546+0000 7f1986f8c640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f198006b570 con 0x7f198006abe0 2024-04-17T10:47:32.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.546+0000 7f1986f8c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19800727e0 con 0x7f1980146b40 2024-04-17T10:47:32.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.546+0000 7f1985529640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f198006abe0 0x7f198006b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:32.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.546+0000 7f1985529640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f198006abe0 0x7f198006b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60404/0 (socket says 172.21.15.83:60404) 2024-04-17T10:47:32.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.546+0000 7f1985529640 1 -- 172.21.15.83:0/1266092665 learned_addr learned my addr 172.21.15.83:0/1266092665 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:32.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.546+0000 7f1984d28640 1 --2- 172.21.15.83:0/1266092665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1980146b40 0x7f198006d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:32.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.546+0000 7f1985529640 1 -- 172.21.15.83:0/1266092665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1980146b40 msgr2=0x7f198006d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:32.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.546+0000 7f1985529640 1 --2- 172.21.15.83:0/1266092665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1980146b40 0x7f198006d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:32.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.546+0000 7f1985529640 1 -- 172.21.15.83:0/1266092665 >> v1:172.21.15.167:6789/0 conn(0x7f1980146f20 legacy=0x7f19800720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:32.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.546+0000 7f1985529640 1 -- 172.21.15.83:0/1266092665 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1980161dc0 con 0x7f198006abe0 2024-04-17T10:47:32.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.546+0000 7f1985529640 1 --2- 172.21.15.83:0/1266092665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f198006abe0 0x7f198006b030 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f19700098a0 tx=0x7f197001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=524a3c5ec65b6b78 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:32.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.546+0000 7f197f7fe640 1 -- 172.21.15.83:0/1266092665 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19700b1070 con 0x7f198006abe0 2024-04-17T10:47:32.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f197f7fe640 1 -- 172.21.15.83:0/1266092665 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19700ac410 con 0x7f198006abe0 2024-04-17T10:47:32.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f197f7fe640 1 -- 172.21.15.83:0/1266092665 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19700ac7e0 con 0x7f198006abe0 2024-04-17T10:47:32.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f1986f8c640 1 -- 172.21.15.83:0/1266092665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f198006abe0 msgr2=0x7f198006b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:32.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f1986f8c640 1 --2- 172.21.15.83:0/1266092665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f198006abe0 0x7f198006b030 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f19700098a0 tx=0x7f197001c500 comp rx=0 tx=0).stop 2024-04-17T10:47:32.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f1986f8c640 1 -- 172.21.15.83:0/1266092665 shutdown_connections 2024-04-17T10:47:32.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f1986f8c640 1 --2- 172.21.15.83:0/1266092665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f198006abe0 0x7f198006b030 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:32.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f1986f8c640 1 --2- 172.21.15.83:0/1266092665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1980146b40 0x7f198006d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:32.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f1986f8c640 1 -- 172.21.15.83:0/1266092665 >> 172.21.15.83:0/1266092665 conn(0x7f198006bcf0 msgr2=0x7f198006c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:32.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f1986f8c640 1 -- 172.21.15.83:0/1266092665 shutdown_connections 2024-04-17T10:47:32.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f1986f8c640 1 -- 172.21.15.83:0/1266092665 wait complete. 2024-04-17T10:47:32.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f1986f8c640 1 Processor -- start 2024-04-17T10:47:32.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f1986f8c640 1 -- start start 2024-04-17T10:47:32.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f1986f8c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f198006abe0 0x7f1980144640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:32.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f1986f8c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1980146b40 0x7f19800cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:32.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f1986f8c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1980146f20 0x7f19801f9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:32.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f1986f8c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19800d1140 con 0x7f1980146b40 2024-04-17T10:47:32.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f1986f8c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19800d0fc0 con 0x7f1980146f20 2024-04-17T10:47:32.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f1986f8c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19800d12c0 con 0x7f198006abe0 2024-04-17T10:47:32.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f197ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1980146b40 0x7f19800cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:32.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f197ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1980146b40 0x7f19800cfc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60418/0 (socket says 172.21.15.83:60418) 2024-04-17T10:47:32.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f197ffff640 1 -- 172.21.15.83:0/3719470536 learned_addr learned my addr 172.21.15.83:0/3719470536 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:32.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f1984d28640 1 --2- 172.21.15.83:0/3719470536 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f198006abe0 0x7f1980144640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:32.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f1985529640 1 --2- 172.21.15.83:0/3719470536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1980146f20 0x7f19801f9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:32.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f197ffff640 1 -- 172.21.15.83:0/3719470536 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f198006abe0 msgr2=0x7f1980144640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:32.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f197ffff640 1 --2- 172.21.15.83:0/3719470536 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f198006abe0 0x7f1980144640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:32.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f197ffff640 1 -- 172.21.15.83:0/3719470536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1980146f20 msgr2=0x7f19801f9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:32.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f197ffff640 1 --2- 172.21.15.83:0/3719470536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1980146f20 0x7f19801f9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:32.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f197ffff640 1 -- 172.21.15.83:0/3719470536 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1980160e80 con 0x7f1980146b40 2024-04-17T10:47:32.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f197ffff640 1 --2- 172.21.15.83:0/3719470536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1980146b40 0x7f19800cfc90 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f197800ba50 tx=0x7f197800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:32.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f197d7fa640 1 -- 172.21.15.83:0/3719470536 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1978019070 con 0x7f1980146b40 2024-04-17T10:47:32.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f197d7fa640 1 -- 172.21.15.83:0/3719470536 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197800cc20 con 0x7f1980146b40 2024-04-17T10:47:32.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f1986f8c640 1 -- 172.21.15.83:0/3719470536 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f198015ca80 con 0x7f1980146b40 2024-04-17T10:47:32.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f197d7fa640 1 -- 172.21.15.83:0/3719470536 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1978014440 con 0x7f1980146b40 2024-04-17T10:47:32.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.550+0000 7f1986f8c640 1 -- 172.21.15.83:0/3719470536 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f198015cd20 con 0x7f1980146b40 2024-04-17T10:47:32.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.554+0000 7f1986f8c640 1 -- 172.21.15.83:0/3719470536 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1980074f90 con 0x7f1980146b40 2024-04-17T10:47:32.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.554+0000 7f197d7fa640 1 -- 172.21.15.83:0/3719470536 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f197801c070 con 0x7f1980146b40 2024-04-17T10:47:32.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.554+0000 7f197d7fa640 1 --2- 172.21.15.83:0/3719470536 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f196c066f60 0x7f196c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:32.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.554+0000 7f197d7fa640 1 -- 172.21.15.83:0/3719470536 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1978084c50 con 0x7f1980146b40 2024-04-17T10:47:32.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.554+0000 7f1984d28640 1 --2- 172.21.15.83:0/3719470536 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f196c066f60 0x7f196c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:32.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.554+0000 7f1984d28640 1 --2- 172.21.15.83:0/3719470536 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f196c066f60 0x7f196c069420 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f197400a970 tx=0x7f1974008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:32.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.558+0000 7f197d7fa640 1 -- 172.21.15.83:0/3719470536 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f197808a020 con 0x7f1980146b40 2024-04-17T10:47:32.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.714+0000 7f1986f8c640 1 -- 172.21.15.83:0/3719470536 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19800b2a00 con 0x7f1980146b40 2024-04-17T10:47:32.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.714+0000 7f197d7fa640 1 -- 172.21.15.83:0/3719470536 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16547 (secure 0 0 0) 0x7f1978055610 con 0x7f1980146b40 2024-04-17T10:47:32.711 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:47:32.711 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":103,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:47:31.018926+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:47:27.619432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-17T10:47:28.379472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:47:32.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.714+0000 7f1986f8c640 1 -- 172.21.15.83:0/3719470536 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f196c066f60 msgr2=0x7f196c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:32.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.714+0000 7f1986f8c640 1 --2- 172.21.15.83:0/3719470536 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f196c066f60 0x7f196c069420 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f197400a970 tx=0x7f1974008040 comp rx=0 tx=0).stop 2024-04-17T10:47:32.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.714+0000 7f1986f8c640 1 -- 172.21.15.83:0/3719470536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1980146b40 msgr2=0x7f19800cfc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:32.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.714+0000 7f1986f8c640 1 --2- 172.21.15.83:0/3719470536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1980146b40 0x7f19800cfc90 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f197800ba50 tx=0x7f197800bf10 comp rx=0 tx=0).stop 2024-04-17T10:47:32.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.714+0000 7f1986f8c640 1 -- 172.21.15.83:0/3719470536 shutdown_connections 2024-04-17T10:47:32.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.714+0000 7f1986f8c640 1 --2- 172.21.15.83:0/3719470536 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f196c066f60 0x7f196c069420 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:32.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.714+0000 7f1986f8c640 1 --2- 172.21.15.83:0/3719470536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1980146f20 0x7f19801f9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:32.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.714+0000 7f1986f8c640 1 --2- 172.21.15.83:0/3719470536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1980146b40 0x7f19800cfc90 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:32.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.714+0000 7f1986f8c640 1 --2- 172.21.15.83:0/3719470536 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f198006abe0 0x7f1980144640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:32.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.714+0000 7f1986f8c640 1 -- 172.21.15.83:0/3719470536 >> 172.21.15.83:0/3719470536 conn(0x7f198006bcf0 msgr2=0x7f19800b1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:32.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.714+0000 7f1986f8c640 1 -- 172.21.15.83:0/3719470536 shutdown_connections 2024-04-17T10:47:32.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.714+0000 7f1986f8c640 1 -- 172.21.15.83:0/3719470536 wait complete. 2024-04-17T10:47:32.728 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-17T10:47:32.729 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:47:32.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a3bdf1640 1 Processor -- start 2024-04-17T10:47:32.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a3bdf1640 1 -- start start 2024-04-17T10:47:32.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a3bdf1640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2a34070470 0x7f2a34070880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:32.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a3bdf1640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a34070dc0 con 0x7f2a340b3990 2024-04-17T10:47:32.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a3bdf1640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a340b2bd0 con 0x7f2a340cf060 2024-04-17T10:47:32.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a3bdf1640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a340b2980 con 0x7f2a34070470 2024-04-17T10:47:32.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a3938c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2a340b3990 0x7f2a340cf440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41606/0 (socket says 172.21.15.83:41606) 2024-04-17T10:47:32.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a3938c640 1 -- 172.21.15.83:0/1964139479 learned_addr learned my addr 172.21.15.83:0/1964139479 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:32.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a38b8b640 1 -- 172.21.15.83:0/1964139479 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1068163182 0 0) 0x7f2a30001120 con 0x7f2a340b3990 2024-04-17T10:47:32.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a38b8b640 1 -- 172.21.15.83:0/1964139479 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a20003530 con 0x7f2a340b3990 2024-04-17T10:47:32.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a3a38e640 1 --2- 172.21.15.83:0/1964139479 >> v2:172.21.15.167:3300/0 conn(0x7f2a34070470 0x7f2a34070880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:32.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a38b8b640 1 -- 172.21.15.83:0/1964139479 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4047082881 0 0) 0x7f2a20003530 con 0x7f2a340b3990 2024-04-17T10:47:32.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a38b8b640 1 -- 172.21.15.83:0/1964139479 >> v2:172.21.15.167:3300/0 conn(0x7f2a34070470 msgr2=0x7f2a34070880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:32.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a38b8b640 1 --2- 172.21.15.83:0/1964139479 >> v2:172.21.15.167:3300/0 conn(0x7f2a34070470 0x7f2a34070880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:32.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a38b8b640 1 -- 172.21.15.83:0/1964139479 >> v1:172.21.15.167:6789/0 conn(0x7f2a340cf060 legacy=0x7f2a340b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:32.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a38b8b640 1 -- 172.21.15.83:0/1964139479 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a340b2eb0 con 0x7f2a340b3990 2024-04-17T10:47:32.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a38b8b640 1 -- 172.21.15.83:0/1964139479 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f2a30001850 con 0x7f2a340b3990 2024-04-17T10:47:32.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a38b8b640 1 -- 172.21.15.83:0/1964139479 >> v1:172.21.15.83:6789/0 conn(0x7f2a340b3990 legacy=0x7f2a340cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:32.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a38b8b640 1 --2- 172.21.15.83:0/1964139479 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a20003a10 0x7f2a20003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:32.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a38b8b640 1 --2- 172.21.15.83:0/1964139479 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a20004810 0x7f2a20004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:32.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a38b8b640 1 --2- 172.21.15.83:0/1964139479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a20005240 0x7f2a2000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:32.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a38b8b640 1 -- 172.21.15.83:0/1964139479 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a340b2980 con 0x7f2a20005240 2024-04-17T10:47:32.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a38b8b640 1 -- 172.21.15.83:0/1964139479 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2000c1a0 con 0x7f2a20004810 2024-04-17T10:47:32.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a38b8b640 1 -- 172.21.15.83:0/1964139479 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2000c3a0 con 0x7f2a20003a10 2024-04-17T10:47:32.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a3a38e640 1 --2- 172.21.15.83:0/1964139479 >> v2:172.21.15.167:3300/0 conn(0x7f2a34070470 0x7f2a34070880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:47:32.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a3a38e640 1 --2- 172.21.15.83:0/1964139479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a20005240 0x7f2a2000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:32.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a39b8d640 1 --2- 172.21.15.83:0/1964139479 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a20003a10 0x7f2a20003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:32.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a3938c640 1 --2- 172.21.15.83:0/1964139479 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a20004810 0x7f2a20004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:32.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a3a38e640 1 -- 172.21.15.83:0/1964139479 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a20003a10 msgr2=0x7f2a20003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:32.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a3a38e640 1 --2- 172.21.15.83:0/1964139479 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a20003a10 0x7f2a20003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:32.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a3a38e640 1 -- 172.21.15.83:0/1964139479 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a20004810 msgr2=0x7f2a20004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:32.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a3a38e640 1 --2- 172.21.15.83:0/1964139479 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a20004810 0x7f2a20004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:32.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.846+0000 7f2a3a38e640 1 -- 172.21.15.83:0/1964139479 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a20003530 con 0x7f2a20005240 2024-04-17T10:47:32.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.850+0000 7f2a3a38e640 1 -- 172.21.15.83:0/1964139479 reap_dead start 2024-04-17T10:47:32.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.850+0000 7f2a3a38e640 1 --2- 172.21.15.83:0/1964139479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a20005240 0x7f2a2000b9d0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0073e0 tx=0x7f2a2c01cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf416c6e4368b4d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:32.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.850+0000 7f2a38b8b640 1 -- 172.21.15.83:0/1964139479 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c009c70 con 0x7f2a20005240 2024-04-17T10:47:32.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.850+0000 7f2a38b8b640 1 -- 172.21.15.83:0/1964139479 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2c0ad440 con 0x7f2a20005240 2024-04-17T10:47:32.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.850+0000 7f2a38b8b640 1 -- 172.21.15.83:0/1964139479 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c0a5e60 con 0x7f2a20005240 2024-04-17T10:47:32.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.850+0000 7f2a3bdf1640 1 -- 172.21.15.83:0/1964139479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a20005240 msgr2=0x7f2a2000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:32.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.850+0000 7f2a3bdf1640 1 --2- 172.21.15.83:0/1964139479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a20005240 0x7f2a2000b9d0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0073e0 tx=0x7f2a2c01cf40 comp rx=0 tx=0).stop 2024-04-17T10:47:32.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.850+0000 7f2a3bdf1640 1 -- 172.21.15.83:0/1964139479 shutdown_connections 2024-04-17T10:47:32.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.850+0000 7f2a3bdf1640 1 --2- 172.21.15.83:0/1964139479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a20005240 0x7f2a2000b9d0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:32.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.850+0000 7f2a3bdf1640 1 -- 172.21.15.83:0/1964139479 >> 172.21.15.83:0/1964139479 conn(0x7f2a3406abe0 msgr2=0x7f2a3406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:32.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.850+0000 7f2a3bdf1640 1 -- 172.21.15.83:0/1964139479 shutdown_connections 2024-04-17T10:47:32.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.850+0000 7f2a3bdf1640 1 -- 172.21.15.83:0/1964139479 wait complete. 2024-04-17T10:47:32.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.850+0000 7f2a3bdf1640 1 Processor -- start 2024-04-17T10:47:32.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.850+0000 7f2a3bdf1640 1 -- start start 2024-04-17T10:47:32.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.854+0000 7f2a3bdf1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a20005240 0x7f2a3415d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:32.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.854+0000 7f2a3bdf1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a34155cb0 0x7f2a341567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:32.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.854+0000 7f2a3bdf1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a34148150 0x7f2a34154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:32.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.854+0000 7f2a3bdf1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3415b310 con 0x7f2a34148150 2024-04-17T10:47:32.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.854+0000 7f2a3bdf1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a340b3470 con 0x7f2a34155cb0 2024-04-17T10:47:32.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.854+0000 7f2a3bdf1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a34070880 con 0x7f2a20005240 2024-04-17T10:47:32.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.854+0000 7f2a3a38e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a34148150 0x7f2a34154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:32.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.854+0000 7f2a3a38e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a34148150 0x7f2a34154c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60448/0 (socket says 172.21.15.83:60448) 2024-04-17T10:47:32.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.854+0000 7f2a3a38e640 1 -- 172.21.15.83:0/3289300429 learned_addr learned my addr 172.21.15.83:0/3289300429 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:32.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.854+0000 7f2a3938c640 1 --2- 172.21.15.83:0/3289300429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a34155cb0 0x7f2a341567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:32.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.854+0000 7f2a39b8d640 1 --2- 172.21.15.83:0/3289300429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a20005240 0x7f2a3415d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:32.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.854+0000 7f2a3a38e640 1 -- 172.21.15.83:0/3289300429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a20005240 msgr2=0x7f2a3415d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:32.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.854+0000 7f2a3a38e640 1 --2- 172.21.15.83:0/3289300429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a20005240 0x7f2a3415d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:32.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.854+0000 7f2a3a38e640 1 -- 172.21.15.83:0/3289300429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a34155cb0 msgr2=0x7f2a341567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:32.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.854+0000 7f2a3a38e640 1 --2- 172.21.15.83:0/3289300429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a34155cb0 0x7f2a341567c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:32.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.854+0000 7f2a3a38e640 1 -- 172.21.15.83:0/3289300429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3415bfd0 con 0x7f2a34148150 2024-04-17T10:47:32.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.854+0000 7f2a39b8d640 1 --2- 172.21.15.83:0/3289300429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a20005240 0x7f2a3415d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:32.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.854+0000 7f2a3a38e640 1 --2- 172.21.15.83:0/3289300429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a34148150 0x7f2a34154c30 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c01d7e0 tx=0x7f2a20005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:32.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.854+0000 7f2a2a7fc640 1 -- 172.21.15.83:0/3289300429 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c0b85f0 con 0x7f2a34148150 2024-04-17T10:47:32.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.854+0000 7f2a2a7fc640 1 -- 172.21.15.83:0/3289300429 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2c0b8790 con 0x7f2a34148150 2024-04-17T10:47:32.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.854+0000 7f2a3bdf1640 1 -- 172.21.15.83:0/3289300429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a3415c2b0 con 0x7f2a34148150 2024-04-17T10:47:32.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.854+0000 7f2a2a7fc640 1 -- 172.21.15.83:0/3289300429 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c0b8b60 con 0x7f2a34148150 2024-04-17T10:47:32.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.854+0000 7f2a3bdf1640 1 -- 172.21.15.83:0/3289300429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3415c870 con 0x7f2a34148150 2024-04-17T10:47:32.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.854+0000 7f2a3bdf1640 1 -- 172.21.15.83:0/3289300429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a34074f90 con 0x7f2a34148150 2024-04-17T10:47:32.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.854+0000 7f2a2a7fc640 1 -- 172.21.15.83:0/3289300429 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2a2c0ac3e0 con 0x7f2a34148150 2024-04-17T10:47:32.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.854+0000 7f2a2a7fc640 1 --2- 172.21.15.83:0/3289300429 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2a18066f90 0x7f2a18069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:32.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.854+0000 7f2a2a7fc640 1 -- 172.21.15.83:0/3289300429 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a2c11fb10 con 0x7f2a34148150 2024-04-17T10:47:32.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.854+0000 7f2a39b8d640 1 --2- 172.21.15.83:0/3289300429 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2a18066f90 0x7f2a18069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:32.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.854+0000 7f2a39b8d640 1 --2- 172.21.15.83:0/3289300429 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2a18066f90 0x7f2a18069450 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2a24000c10 tx=0x7f2a24009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:32.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:32.858+0000 7f2a2a7fc640 1 -- 172.21.15.83:0/3289300429 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2a2c0b5070 con 0x7f2a34148150 2024-04-17T10:47:32.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.002+0000 7f2a3bdf1640 1 -- 172.21.15.83:0/3289300429 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2a3414d680 con 0x7f2a18066f90 2024-04-17T10:47:33.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.010+0000 7f2a2a7fc640 1 -- 172.21.15.83:0/3289300429 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247088 (secure 0 0 0) 0x7f2a3414d680 con 0x7f2a18066f90 2024-04-17T10:47:33.010 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:47:33.012 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:47:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.014+0000 7f2a3bdf1640 1 -- 172.21.15.83:0/3289300429 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2a18066f90 msgr2=0x7f2a18069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.014+0000 7f2a3bdf1640 1 --2- 172.21.15.83:0/3289300429 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2a18066f90 0x7f2a18069450 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2a24000c10 tx=0x7f2a24009040 comp rx=0 tx=0).stop 2024-04-17T10:47:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.014+0000 7f2a3bdf1640 1 -- 172.21.15.83:0/3289300429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a34148150 msgr2=0x7f2a34154c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.014+0000 7f2a3bdf1640 1 --2- 172.21.15.83:0/3289300429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a34148150 0x7f2a34154c30 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c01d7e0 tx=0x7f2a20005840 comp rx=0 tx=0).stop 2024-04-17T10:47:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.014+0000 7f2a3bdf1640 1 -- 172.21.15.83:0/3289300429 shutdown_connections 2024-04-17T10:47:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.014+0000 7f2a3bdf1640 1 --2- 172.21.15.83:0/3289300429 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2a18066f90 0x7f2a18069450 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.014+0000 7f2a3bdf1640 1 --2- 172.21.15.83:0/3289300429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a34148150 0x7f2a34154c30 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.014+0000 7f2a3bdf1640 1 --2- 172.21.15.83:0/3289300429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a34155cb0 0x7f2a341567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.014+0000 7f2a3bdf1640 1 --2- 172.21.15.83:0/3289300429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a20005240 0x7f2a3415d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.014+0000 7f2a3bdf1640 1 -- 172.21.15.83:0/3289300429 >> 172.21.15.83:0/3289300429 conn(0x7f2a3406abe0 msgr2=0x7f2a34159320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.014+0000 7f2a3bdf1640 1 -- 172.21.15.83:0/3289300429 shutdown_connections 2024-04-17T10:47:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.014+0000 7f2a3bdf1640 1 -- 172.21.15.83:0/3289300429 wait complete. 2024-04-17T10:47:33.027 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2024-04-17T10:47:32.348528+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":524064,"kb_used_data":10860,"kb_used_omap":12,"kb_used_meta":513075,"kb_avail":754450656,"statfs":{"total":773094113280,"available":772557471744,"internally_reserved":0,"allocated":11120640,"data_stored":5471361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":525389360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.340819"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.031507+0000","last_change":"2024-04-17T10:47:29.031507+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.031507+0000","last_undegraded":"2024-04-17T10:47:29.031507+0000","last_fullsized":"2024-04-17T10:47:29.031507+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.055397+0000","last_change":"2024-04-17T10:47:29.055397+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.055397+0000","last_undegraded":"2024-04-17T10:47:29.055397+0000","last_fullsized":"2024-04-17T10:47:29.055397+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.027558+0000","last_change":"2024-04-17T10:47:29.027558+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.027558+0000","last_undegraded":"2024-04-17T10:47:29.027558+0000","last_fullsized":"2024-04-17T10:47:29.027558+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.044961+0000","last_change":"2024-04-17T10:47:29.044961+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.044961+0000","last_undegraded":"2024-04-17T10:47:29.044961+0000","last_fullsized":"2024-04-17T10:47:29.044961+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.038581+0000","last_change":"2024-04-17T10:47:29.038581+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.038581+0000","last_undegraded":"2024-04-17T10:47:29.038581+0000","last_fullsized":"2024-04-17T10:47:29.038581+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.021494+0000","last_change":"2024-04-17T10:47:29.021494+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.021494+0000","last_undegraded":"2024-04-17T10:47:29.021494+0000","last_fullsized":"2024-04-17T10:47:29.021494+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.025999+0000","last_change":"2024-04-17T10:47:29.025999+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.025999+0000","last_undegraded":"2024-04-17T10:47:29.025999+0000","last_fullsized":"2024-04-17T10:47:29.025999+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.049465+0000","last_change":"2024-04-17T10:47:29.049465+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.049465+0000","last_undegraded":"2024-04-17T10:47:29.049465+0000","last_fullsized":"2024-04-17T10:47:29.049465+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.024602+0000","last_change":"2024-04-17T10:47:29.024602+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.024602+0000","last_undegraded":"2024-04-17T10:47:29.024602+0000","last_fullsized":"2024-04-17T10:47:29.024602+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.030562+0000","last_change":"2024-04-17T10:47:29.030562+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.030562+0000","last_undegraded":"2024-04-17T10:47:29.030562+0000","last_fullsized":"2024-04-17T10:47:29.030562+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.032399+0000","last_change":"2024-04-17T10:47:29.032399+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.032399+0000","last_undegraded":"2024-04-17T10:47:29.032399+0000","last_fullsized":"2024-04-17T10:47:29.032399+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.029188+0000","last_change":"2024-04-17T10:47:29.029188+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.029188+0000","last_undegraded":"2024-04-17T10:47:29.029188+0000","last_fullsized":"2024-04-17T10:47:29.029188+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.046072+0000","last_change":"2024-04-17T10:47:29.046072+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.046072+0000","last_undegraded":"2024-04-17T10:47:29.046072+0000","last_fullsized":"2024-04-17T10:47:29.046072+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.037835+0000","last_change":"2024-04-17T10:47:29.037835+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.037835+0000","last_undegraded":"2024-04-17T10:47:29.037835+0000","last_fullsized":"2024-04-17T10:47:29.037835+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.045596+0000","last_change":"2024-04-17T10:47:29.045596+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.045596+0000","last_undegraded":"2024-04-17T10:47:29.045596+0000","last_fullsized":"2024-04-17T10:47:29.045596+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.028858+0000","last_change":"2024-04-17T10:47:29.028858+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.028858+0000","last_undegraded":"2024-04-17T10:47:29.028858+0000","last_fullsized":"2024-04-17T10:47:29.028858+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.044488+0000","last_change":"2024-04-17T10:47:29.044488+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.044488+0000","last_undegraded":"2024-04-17T10:47:29.044488+0000","last_fullsized":"2024-04-17T10:47:29.044488+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.044801+0000","last_change":"2024-04-17T10:47:29.044801+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.044801+0000","last_undegraded":"2024-04-17T10:47:29.044801+0000","last_fullsized":"2024-04-17T10:47:29.044801+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.034394+0000","last_change":"2024-04-17T10:47:29.034394+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.034394+0000","last_undegraded":"2024-04-17T10:47:29.034394+0000","last_fullsized":"2024-04-17T10:47:29.034394+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.023972+0000","last_change":"2024-04-17T10:47:29.023972+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.023972+0000","last_undegraded":"2024-04-17T10:47:29.023972+0000","last_fullsized":"2024-04-17T10:47:29.023972+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.034527+0000","last_change":"2024-04-17T10:47:29.034527+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.034527+0000","last_undegraded":"2024-04-17T10:47:29.034527+0000","last_fullsized":"2024-04-17T10:47:29.034527+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.045648+0000","last_change":"2024-04-17T10:47:29.045648+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.045648+0000","last_undegraded":"2024-04-17T10:47:29.045648+0000","last_fullsized":"2024-04-17T10:47:29.045648+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.045251+0000","last_change":"2024-04-17T10:47:29.045251+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.045251+0000","last_undegraded":"2024-04-17T10:47:29.045251+0000","last_fullsized":"2024-04-17T10:47:29.045251+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.027697+0000","last_change":"2024-04-17T10:47:29.027697+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.027697+0000","last_undegraded":"2024-04-17T10:47:29.027697+0000","last_fullsized":"2024-04-17T10:47:29.027697+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.037694+0000","last_change":"2024-04-17T10:47:29.037694+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.037694+0000","last_undegraded":"2024-04-17T10:47:29.037694+0000","last_fullsized":"2024-04-17T10:47:29.037694+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.050922+0000","last_change":"2024-04-17T10:47:29.050922+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.050922+0000","last_undegraded":"2024-04-17T10:47:29.050922+0000","last_fullsized":"2024-04-17T10:47:29.050922+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.024737+0000","last_change":"2024-04-17T10:47:29.024737+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.024737+0000","last_undegraded":"2024-04-17T10:47:29.024737+0000","last_fullsized":"2024-04-17T10:47:29.024737+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.043747+0000","last_change":"2024-04-17T10:47:29.043747+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.043747+0000","last_undegraded":"2024-04-17T10:47:29.043747+0000","last_fullsized":"2024-04-17T10:47:29.043747+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.047296+0000","last_change":"2024-04-17T10:47:29.047296+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.047296+0000","last_undegraded":"2024-04-17T10:47:29.047296+0000","last_fullsized":"2024-04-17T10:47:29.047296+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.035574+0000","last_change":"2024-04-17T10:47:29.035574+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.035574+0000","last_undegraded":"2024-04-17T10:47:29.035574+0000","last_fullsized":"2024-04-17T10:47:29.035574+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.045037+0000","last_change":"2024-04-17T10:47:29.045037+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.045037+0000","last_undegraded":"2024-04-17T10:47:29.045037+0000","last_fullsized":"2024-04-17T10:47:29.045037+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.028571+0000","last_change":"2024-04-17T10:47:29.028571+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.028571+0000","last_undegraded":"2024-04-17T10:47:29.028571+0000","last_fullsized":"2024-04-17T10:47:29.028571+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.034477+0000","last_change":"2024-04-17T10:47:29.034477+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.034477+0000","last_undegraded":"2024-04-17T10:47:29.034477+0000","last_fullsized":"2024-04-17T10:47:29.034477+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.058518+0000","last_change":"2024-04-17T10:47:29.058101+0000","last_active":"2024-04-17T10:47:29.058518+0000","last_peered":"2024-04-17T10:47:29.058518+0000","last_clean":"2024-04-17T10:47:29.058518+0000","last_became_active":"2024-04-17T10:47:29.056266+0000","last_became_peered":"2024-04-17T10:47:29.056266+0000","last_unstale":"2024-04-17T10:47:29.058518+0000","last_undegraded":"2024-04-17T10:47:29.058518+0000","last_fullsized":"2024-04-17T10:47:29.058518+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:10:51.407123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.047078+0000","last_change":"2024-04-17T10:47:29.047078+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.047078+0000","last_undegraded":"2024-04-17T10:47:29.047078+0000","last_fullsized":"2024-04-17T10:47:29.047078+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.044854+0000","last_change":"2024-04-17T10:47:29.037268+0000","last_active":"2024-04-17T10:47:29.044854+0000","last_peered":"2024-04-17T10:47:29.044854+0000","last_clean":"2024-04-17T10:47:29.044854+0000","last_became_active":"2024-04-17T10:47:29.035743+0000","last_became_peered":"2024-04-17T10:47:29.035743+0000","last_unstale":"2024-04-17T10:47:29.044854+0000","last_undegraded":"2024-04-17T10:47:29.044854+0000","last_fullsized":"2024-04-17T10:47:29.044854+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:16:45.173638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.030657+0000","last_change":"2024-04-17T10:47:29.030657+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.030657+0000","last_undegraded":"2024-04-17T10:47:29.030657+0000","last_fullsized":"2024-04-17T10:47:29.030657+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.051205+0000","last_change":"2024-04-17T10:47:29.046225+0000","last_active":"2024-04-17T10:47:29.051205+0000","last_peered":"2024-04-17T10:47:29.051205+0000","last_clean":"2024-04-17T10:47:29.051205+0000","last_became_active":"2024-04-17T10:47:29.043738+0000","last_became_peered":"2024-04-17T10:47:29.043738+0000","last_unstale":"2024-04-17T10:47:29.051205+0000","last_undegraded":"2024-04-17T10:47:29.051205+0000","last_fullsized":"2024-04-17T10:47:29.051205+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:43.205753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.023044+0000","last_change":"2024-04-17T10:47:29.023044+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.023044+0000","last_undegraded":"2024-04-17T10:47:29.023044+0000","last_fullsized":"2024-04-17T10:47:29.023044+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.042042+0000","last_change":"2024-04-17T10:47:29.041206+0000","last_active":"2024-04-17T10:47:29.042042+0000","last_peered":"2024-04-17T10:47:29.042042+0000","last_clean":"2024-04-17T10:47:29.042042+0000","last_became_active":"2024-04-17T10:47:29.040924+0000","last_became_peered":"2024-04-17T10:47:29.040924+0000","last_unstale":"2024-04-17T10:47:29.042042+0000","last_undegraded":"2024-04-17T10:47:29.042042+0000","last_fullsized":"2024-04-17T10:47:29.042042+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:35:16.067757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.028743+0000","last_change":"2024-04-17T10:47:29.028743+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.028743+0000","last_undegraded":"2024-04-17T10:47:29.028743+0000","last_fullsized":"2024-04-17T10:47:29.028743+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.054287+0000","last_change":"2024-04-17T10:47:29.053852+0000","last_active":"2024-04-17T10:47:29.054287+0000","last_peered":"2024-04-17T10:47:29.054287+0000","last_clean":"2024-04-17T10:47:29.054287+0000","last_became_active":"2024-04-17T10:47:29.053434+0000","last_became_peered":"2024-04-17T10:47:29.053434+0000","last_unstale":"2024-04-17T10:47:29.054287+0000","last_undegraded":"2024-04-17T10:47:29.054287+0000","last_fullsized":"2024-04-17T10:47:29.054287+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:36:34.892543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.030058+0000","last_change":"2024-04-17T10:47:29.030058+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.030058+0000","last_undegraded":"2024-04-17T10:47:29.030058+0000","last_fullsized":"2024-04-17T10:47:29.030058+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.041154+0000","last_change":"2024-04-17T10:47:29.040335+0000","last_active":"2024-04-17T10:47:29.041154+0000","last_peered":"2024-04-17T10:47:29.041154+0000","last_clean":"2024-04-17T10:47:29.041154+0000","last_became_active":"2024-04-17T10:47:29.039941+0000","last_became_peered":"2024-04-17T10:47:29.039941+0000","last_unstale":"2024-04-17T10:47:29.041154+0000","last_undegraded":"2024-04-17T10:47:29.041154+0000","last_fullsized":"2024-04-17T10:47:29.041154+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:31:48.398246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.044793+0000","last_change":"2024-04-17T10:47:29.044793+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.044793+0000","last_undegraded":"2024-04-17T10:47:29.044793+0000","last_fullsized":"2024-04-17T10:47:29.044793+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.052502+0000","last_change":"2024-04-17T10:47:29.045408+0000","last_active":"2024-04-17T10:47:29.052502+0000","last_peered":"2024-04-17T10:47:29.052502+0000","last_clean":"2024-04-17T10:47:29.052502+0000","last_became_active":"2024-04-17T10:47:29.044801+0000","last_became_peered":"2024-04-17T10:47:29.044801+0000","last_unstale":"2024-04-17T10:47:29.052502+0000","last_undegraded":"2024-04-17T10:47:29.052502+0000","last_fullsized":"2024-04-17T10:47:29.052502+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:52:11.119097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.024952+0000","last_change":"2024-04-17T10:47:29.024952+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.024952+0000","last_undegraded":"2024-04-17T10:47:29.024952+0000","last_fullsized":"2024-04-17T10:47:29.024952+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.046171+0000","last_change":"2024-04-17T10:47:29.038936+0000","last_active":"2024-04-17T10:47:29.046171+0000","last_peered":"2024-04-17T10:47:29.046171+0000","last_clean":"2024-04-17T10:47:29.046171+0000","last_became_active":"2024-04-17T10:47:29.038436+0000","last_became_peered":"2024-04-17T10:47:29.038436+0000","last_unstale":"2024-04-17T10:47:29.046171+0000","last_undegraded":"2024-04-17T10:47:29.046171+0000","last_fullsized":"2024-04-17T10:47:29.046171+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:20:55.139531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.049911+0000","last_change":"2024-04-17T10:47:29.049911+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.049911+0000","last_undegraded":"2024-04-17T10:47:29.049911+0000","last_fullsized":"2024-04-17T10:47:29.049911+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.042156+0000","last_change":"2024-04-17T10:47:29.040517+0000","last_active":"2024-04-17T10:47:29.042156+0000","last_peered":"2024-04-17T10:47:29.042156+0000","last_clean":"2024-04-17T10:47:29.042156+0000","last_became_active":"2024-04-17T10:47:29.039818+0000","last_became_peered":"2024-04-17T10:47:29.039818+0000","last_unstale":"2024-04-17T10:47:29.042156+0000","last_undegraded":"2024-04-17T10:47:29.042156+0000","last_fullsized":"2024-04-17T10:47:29.042156+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:49:17.199444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.023180+0000","last_change":"2024-04-17T10:47:29.023180+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.023180+0000","last_undegraded":"2024-04-17T10:47:29.023180+0000","last_fullsized":"2024-04-17T10:47:29.023180+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.042609+0000","last_change":"2024-04-17T10:47:29.037265+0000","last_active":"2024-04-17T10:47:29.042609+0000","last_peered":"2024-04-17T10:47:29.042609+0000","last_clean":"2024-04-17T10:47:29.042609+0000","last_became_active":"2024-04-17T10:47:29.036735+0000","last_became_peered":"2024-04-17T10:47:29.036735+0000","last_unstale":"2024-04-17T10:47:29.042609+0000","last_undegraded":"2024-04-17T10:47:29.042609+0000","last_fullsized":"2024-04-17T10:47:29.042609+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:49.197211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.042244+0000","last_change":"2024-04-17T10:47:29.042244+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.042244+0000","last_undegraded":"2024-04-17T10:47:29.042244+0000","last_fullsized":"2024-04-17T10:47:29.042244+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.055631+0000","last_change":"2024-04-17T10:47:29.049241+0000","last_active":"2024-04-17T10:47:29.055631+0000","last_peered":"2024-04-17T10:47:29.055631+0000","last_clean":"2024-04-17T10:47:29.055631+0000","last_became_active":"2024-04-17T10:47:29.048527+0000","last_became_peered":"2024-04-17T10:47:29.048527+0000","last_unstale":"2024-04-17T10:47:29.055631+0000","last_undegraded":"2024-04-17T10:47:29.055631+0000","last_fullsized":"2024-04-17T10:47:29.055631+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:26:50.584899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.023708+0000","last_change":"2024-04-17T10:47:29.023708+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.023708+0000","last_undegraded":"2024-04-17T10:47:29.023708+0000","last_fullsized":"2024-04-17T10:47:29.023708+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.050875+0000","last_change":"2024-04-17T10:47:29.047222+0000","last_active":"2024-04-17T10:47:29.050875+0000","last_peered":"2024-04-17T10:47:29.050875+0000","last_clean":"2024-04-17T10:47:29.050875+0000","last_became_active":"2024-04-17T10:47:29.046540+0000","last_became_peered":"2024-04-17T10:47:29.046540+0000","last_unstale":"2024-04-17T10:47:29.050875+0000","last_undegraded":"2024-04-17T10:47:29.050875+0000","last_fullsized":"2024-04-17T10:47:29.050875+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:41:18.010105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.045109+0000","last_change":"2024-04-17T10:47:29.045109+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.045109+0000","last_undegraded":"2024-04-17T10:47:29.045109+0000","last_fullsized":"2024-04-17T10:47:29.045109+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.051787+0000","last_change":"2024-04-17T10:47:29.051191+0000","last_active":"2024-04-17T10:47:29.051787+0000","last_peered":"2024-04-17T10:47:29.051787+0000","last_clean":"2024-04-17T10:47:29.051787+0000","last_became_active":"2024-04-17T10:47:29.050924+0000","last_became_peered":"2024-04-17T10:47:29.050924+0000","last_unstale":"2024-04-17T10:47:29.051787+0000","last_undegraded":"2024-04-17T10:47:29.051787+0000","last_fullsized":"2024-04-17T10:47:29.051787+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:47:39.522424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.027964+0000","last_change":"2024-04-17T10:47:29.027964+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.027964+0000","last_undegraded":"2024-04-17T10:47:29.027964+0000","last_fullsized":"2024-04-17T10:47:29.027964+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.043366+0000","last_change":"2024-04-17T10:47:29.041596+0000","last_active":"2024-04-17T10:47:29.043366+0000","last_peered":"2024-04-17T10:47:29.043366+0000","last_clean":"2024-04-17T10:47:29.043366+0000","last_became_active":"2024-04-17T10:47:29.041198+0000","last_became_peered":"2024-04-17T10:47:29.041198+0000","last_unstale":"2024-04-17T10:47:29.043366+0000","last_undegraded":"2024-04-17T10:47:29.043366+0000","last_fullsized":"2024-04-17T10:47:29.043366+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:33:24.133568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.024813+0000","last_change":"2024-04-17T10:47:29.024813+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.024813+0000","last_undegraded":"2024-04-17T10:47:29.024813+0000","last_fullsized":"2024-04-17T10:47:29.024813+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.050803+0000","last_change":"2024-04-17T10:47:29.047014+0000","last_active":"2024-04-17T10:47:29.050803+0000","last_peered":"2024-04-17T10:47:29.050803+0000","last_clean":"2024-04-17T10:47:29.050803+0000","last_became_active":"2024-04-17T10:47:29.046269+0000","last_became_peered":"2024-04-17T10:47:29.046269+0000","last_unstale":"2024-04-17T10:47:29.050803+0000","last_undegraded":"2024-04-17T10:47:29.050803+0000","last_fullsized":"2024-04-17T10:47:29.050803+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:59.960069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.034425+0000","last_change":"2024-04-17T10:47:29.034425+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.034425+0000","last_undegraded":"2024-04-17T10:47:29.034425+0000","last_fullsized":"2024-04-17T10:47:29.034425+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.040239+0000","last_change":"2024-04-17T10:47:29.037711+0000","last_active":"2024-04-17T10:47:29.040239+0000","last_peered":"2024-04-17T10:47:29.040239+0000","last_clean":"2024-04-17T10:47:29.040239+0000","last_became_active":"2024-04-17T10:47:29.037183+0000","last_became_peered":"2024-04-17T10:47:29.037183+0000","last_unstale":"2024-04-17T10:47:29.040239+0000","last_undegraded":"2024-04-17T10:47:29.040239+0000","last_fullsized":"2024-04-17T10:47:29.040239+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:07:50.722366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.030365+0000","last_change":"2024-04-17T10:47:29.030365+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.030365+0000","last_undegraded":"2024-04-17T10:47:29.030365+0000","last_fullsized":"2024-04-17T10:47:29.030365+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.051165+0000","last_change":"2024-04-17T10:47:29.046483+0000","last_active":"2024-04-17T10:47:29.051165+0000","last_peered":"2024-04-17T10:47:29.051165+0000","last_clean":"2024-04-17T10:47:29.051165+0000","last_became_active":"2024-04-17T10:47:29.042701+0000","last_became_peered":"2024-04-17T10:47:29.042701+0000","last_unstale":"2024-04-17T10:47:29.051165+0000","last_undegraded":"2024-04-17T10:47:29.051165+0000","last_fullsized":"2024-04-17T10:47:29.051165+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:43.205753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.026502+0000","last_change":"2024-04-17T10:47:29.026502+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.026502+0000","last_undegraded":"2024-04-17T10:47:29.026502+0000","last_fullsized":"2024-04-17T10:47:29.026502+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.052476+0000","last_change":"2024-04-17T10:47:29.044936+0000","last_active":"2024-04-17T10:47:29.052476+0000","last_peered":"2024-04-17T10:47:29.052476+0000","last_clean":"2024-04-17T10:47:29.052476+0000","last_became_active":"2024-04-17T10:47:29.044091+0000","last_became_peered":"2024-04-17T10:47:29.044091+0000","last_unstale":"2024-04-17T10:47:29.052476+0000","last_undegraded":"2024-04-17T10:47:29.052476+0000","last_fullsized":"2024-04-17T10:47:29.052476+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:03:10.762120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.022345+0000","last_change":"2024-04-17T10:47:29.022345+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.022345+0000","last_undegraded":"2024-04-17T10:47:29.022345+0000","last_fullsized":"2024-04-17T10:47:29.022345+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.055071+0000","last_change":"2024-04-17T10:47:29.043948+0000","last_active":"2024-04-17T10:47:29.055071+0000","last_peered":"2024-04-17T10:47:29.055071+0000","last_clean":"2024-04-17T10:47:29.055071+0000","last_became_active":"2024-04-17T10:47:29.043517+0000","last_became_peered":"2024-04-17T10:47:29.043517+0000","last_unstale":"2024-04-17T10:47:29.055071+0000","last_undegraded":"2024-04-17T10:47:29.055071+0000","last_fullsized":"2024-04-17T10:47:29.055071+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:52:11.119097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.035024+0000","last_change":"2024-04-17T10:47:29.035024+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.035024+0000","last_undegraded":"2024-04-17T10:47:29.035024+0000","last_fullsized":"2024-04-17T10:47:29.035024+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.055246+0000","last_change":"2024-04-17T10:47:29.049650+0000","last_active":"2024-04-17T10:47:29.055246+0000","last_peered":"2024-04-17T10:47:29.055246+0000","last_clean":"2024-04-17T10:47:29.055246+0000","last_became_active":"2024-04-17T10:47:29.049263+0000","last_became_peered":"2024-04-17T10:47:29.049263+0000","last_unstale":"2024-04-17T10:47:29.055246+0000","last_undegraded":"2024-04-17T10:47:29.055246+0000","last_fullsized":"2024-04-17T10:47:29.055246+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:41:18.010105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.033707+0000","last_change":"2024-04-17T10:47:29.033707+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.033707+0000","last_undegraded":"2024-04-17T10:47:29.033707+0000","last_fullsized":"2024-04-17T10:47:29.033707+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.052049+0000","last_change":"2024-04-17T10:47:29.041858+0000","last_active":"2024-04-17T10:47:29.052049+0000","last_peered":"2024-04-17T10:47:29.052049+0000","last_clean":"2024-04-17T10:47:29.052049+0000","last_became_active":"2024-04-17T10:47:29.041099+0000","last_became_peered":"2024-04-17T10:47:29.041099+0000","last_unstale":"2024-04-17T10:47:29.052049+0000","last_undegraded":"2024-04-17T10:47:29.052049+0000","last_fullsized":"2024-04-17T10:47:29.052049+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:41:18.010105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.035881+0000","last_change":"2024-04-17T10:47:29.035881+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.035881+0000","last_undegraded":"2024-04-17T10:47:29.035881+0000","last_fullsized":"2024-04-17T10:47:29.035881+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.050068+0000","last_change":"2024-04-17T10:47:29.039405+0000","last_active":"2024-04-17T10:47:29.050068+0000","last_peered":"2024-04-17T10:47:29.050068+0000","last_clean":"2024-04-17T10:47:29.050068+0000","last_became_active":"2024-04-17T10:47:29.038596+0000","last_became_peered":"2024-04-17T10:47:29.038596+0000","last_unstale":"2024-04-17T10:47:29.050068+0000","last_undegraded":"2024-04-17T10:47:29.050068+0000","last_fullsized":"2024-04-17T10:47:29.050068+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:43.205753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.039260+0000","last_change":"2024-04-17T10:47:29.039260+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.039260+0000","last_undegraded":"2024-04-17T10:47:29.039260+0000","last_fullsized":"2024-04-17T10:47:29.039260+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.049867+0000","last_change":"2024-04-17T10:47:29.043402+0000","last_active":"2024-04-17T10:47:29.049867+0000","last_peered":"2024-04-17T10:47:29.049867+0000","last_clean":"2024-04-17T10:47:29.049867+0000","last_became_active":"2024-04-17T10:47:29.042994+0000","last_became_peered":"2024-04-17T10:47:29.042994+0000","last_unstale":"2024-04-17T10:47:29.049867+0000","last_undegraded":"2024-04-17T10:47:29.049867+0000","last_fullsized":"2024-04-17T10:47:29.049867+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:03:10.762120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.048879+0000","last_change":"2024-04-17T10:47:29.048879+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.048879+0000","last_undegraded":"2024-04-17T10:47:29.048879+0000","last_fullsized":"2024-04-17T10:47:29.048879+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.051965+0000","last_change":"2024-04-17T10:47:29.041380+0000","last_active":"2024-04-17T10:47:29.051965+0000","last_peered":"2024-04-17T10:47:29.051965+0000","last_clean":"2024-04-17T10:47:29.051965+0000","last_became_active":"2024-04-17T10:47:29.040614+0000","last_became_peered":"2024-04-17T10:47:29.040614+0000","last_unstale":"2024-04-17T10:47:29.051965+0000","last_undegraded":"2024-04-17T10:47:29.051965+0000","last_fullsized":"2024-04-17T10:47:29.051965+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:59.960069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.030002+0000","last_change":"2024-04-17T10:47:29.030002+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.030002+0000","last_undegraded":"2024-04-17T10:47:29.030002+0000","last_fullsized":"2024-04-17T10:47:29.030002+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.058438+0000","last_change":"2024-04-17T10:47:29.056214+0000","last_active":"2024-04-17T10:47:29.058438+0000","last_peered":"2024-04-17T10:47:29.058438+0000","last_clean":"2024-04-17T10:47:29.058438+0000","last_became_active":"2024-04-17T10:47:29.055718+0000","last_became_peered":"2024-04-17T10:47:29.055718+0000","last_unstale":"2024-04-17T10:47:29.058438+0000","last_undegraded":"2024-04-17T10:47:29.058438+0000","last_fullsized":"2024-04-17T10:47:29.058438+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:28.525331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.028102+0000","last_change":"2024-04-17T10:47:29.028102+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.028102+0000","last_undegraded":"2024-04-17T10:47:29.028102+0000","last_fullsized":"2024-04-17T10:47:29.028102+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.055096+0000","last_change":"2024-04-17T10:47:29.046779+0000","last_active":"2024-04-17T10:47:29.055096+0000","last_peered":"2024-04-17T10:47:29.055096+0000","last_clean":"2024-04-17T10:47:29.055096+0000","last_became_active":"2024-04-17T10:47:29.045869+0000","last_became_peered":"2024-04-17T10:47:29.045869+0000","last_unstale":"2024-04-17T10:47:29.055096+0000","last_undegraded":"2024-04-17T10:47:29.055096+0000","last_fullsized":"2024-04-17T10:47:29.055096+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:59.960069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.037035+0000","last_change":"2024-04-17T10:47:29.037035+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.037035+0000","last_undegraded":"2024-04-17T10:47:29.037035+0000","last_fullsized":"2024-04-17T10:47:29.037035+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.054192+0000","last_change":"2024-04-17T10:47:29.050848+0000","last_active":"2024-04-17T10:47:29.054192+0000","last_peered":"2024-04-17T10:47:29.054192+0000","last_clean":"2024-04-17T10:47:29.054192+0000","last_became_active":"2024-04-17T10:47:29.048692+0000","last_became_peered":"2024-04-17T10:47:29.048692+0000","last_unstale":"2024-04-17T10:47:29.054192+0000","last_undegraded":"2024-04-17T10:47:29.054192+0000","last_fullsized":"2024-04-17T10:47:29.054192+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:43.205753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.030935+0000","last_change":"2024-04-17T10:47:29.030935+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.030935+0000","last_undegraded":"2024-04-17T10:47:29.030935+0000","last_fullsized":"2024-04-17T10:47:29.030935+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.055330+0000","last_change":"2024-04-17T10:47:29.050890+0000","last_active":"2024-04-17T10:47:29.055330+0000","last_peered":"2024-04-17T10:47:29.055330+0000","last_clean":"2024-04-17T10:47:29.055330+0000","last_became_active":"2024-04-17T10:47:29.050540+0000","last_became_peered":"2024-04-17T10:47:29.050540+0000","last_unstale":"2024-04-17T10:47:29.055330+0000","last_undegraded":"2024-04-17T10:47:29.055330+0000","last_fullsized":"2024-04-17T10:47:29.055330+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:19:07.385658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.022875+0000","last_change":"2024-04-17T10:47:29.022875+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.022875+0000","last_undegraded":"2024-04-17T10:47:29.022875+0000","last_fullsized":"2024-04-17T10:47:29.022875+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.053684+0000","last_change":"2024-04-17T10:47:29.051971+0000","last_active":"2024-04-17T10:47:29.053684+0000","last_peered":"2024-04-17T10:47:29.053684+0000","last_clean":"2024-04-17T10:47:29.053684+0000","last_became_active":"2024-04-17T10:47:29.051054+0000","last_became_peered":"2024-04-17T10:47:29.051054+0000","last_unstale":"2024-04-17T10:47:29.053684+0000","last_undegraded":"2024-04-17T10:47:29.053684+0000","last_fullsized":"2024-04-17T10:47:29.053684+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:59.960069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.029219+0000","last_change":"2024-04-17T10:47:29.029219+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.029219+0000","last_undegraded":"2024-04-17T10:47:29.029219+0000","last_fullsized":"2024-04-17T10:47:29.029219+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.037159+0000","last_change":"2024-04-17T10:47:29.032908+0000","last_active":"2024-04-17T10:47:29.037159+0000","last_peered":"2024-04-17T10:47:29.037159+0000","last_clean":"2024-04-17T10:47:29.037159+0000","last_became_active":"2024-04-17T10:47:29.032182+0000","last_became_peered":"2024-04-17T10:47:29.032182+0000","last_unstale":"2024-04-17T10:47:29.037159+0000","last_undegraded":"2024-04-17T10:47:29.037159+0000","last_fullsized":"2024-04-17T10:47:29.037159+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:11:47.505394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.038374+0000","last_change":"2024-04-17T10:47:29.038374+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.038374+0000","last_undegraded":"2024-04-17T10:47:29.038374+0000","last_fullsized":"2024-04-17T10:47:29.038374+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.047599+0000","last_change":"2024-04-17T10:47:29.039563+0000","last_active":"2024-04-17T10:47:29.047599+0000","last_peered":"2024-04-17T10:47:29.047599+0000","last_clean":"2024-04-17T10:47:29.047599+0000","last_became_active":"2024-04-17T10:47:29.039021+0000","last_became_peered":"2024-04-17T10:47:29.039021+0000","last_unstale":"2024-04-17T10:47:29.047599+0000","last_undegraded":"2024-04-17T10:47:29.047599+0000","last_fullsized":"2024-04-17T10:47:29.047599+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:52.837030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.033845+0000","last_change":"2024-04-17T10:47:29.033845+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.033845+0000","last_undegraded":"2024-04-17T10:47:29.033845+0000","last_fullsized":"2024-04-17T10:47:29.033845+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.035392+0000","last_change":"2024-04-17T10:47:29.031551+0000","last_active":"2024-04-17T10:47:29.035392+0000","last_peered":"2024-04-17T10:47:29.035392+0000","last_clean":"2024-04-17T10:47:29.035392+0000","last_became_active":"2024-04-17T10:47:29.030845+0000","last_became_peered":"2024-04-17T10:47:29.030845+0000","last_unstale":"2024-04-17T10:47:29.035392+0000","last_undegraded":"2024-04-17T10:47:29.035392+0000","last_fullsized":"2024-04-17T10:47:29.035392+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:49.197211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":336,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.029346+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:47:29.029346+0000","last_peered":"2024-04-17T10:47:29.029346+0000","last_clean":"2024-04-17T10:47:29.029346+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:47:29.029346+0000","last_undegraded":"2024-04-17T10:47:29.029346+0000","last_fullsized":"2024-04-17T10:47:29.029346+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574966,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59340,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312500,"statfs":{"total":96636764160,"available":96576000000,"internally_reserved":0,"allocated":1830912,"data_stored":1124343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574965,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64328,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307512,"statfs":{"total":96636764160,"available":96570892288,"internally_reserved":0,"allocated":1236992,"data_stored":533975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574966,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50440,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321400,"statfs":{"total":96636764160,"available":96585113600,"internally_reserved":0,"allocated":1236992,"data_stored":533975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574965,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68500,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303340,"statfs":{"total":96636764160,"available":96566620160,"internally_reserved":0,"allocated":1249280,"data_stored":540250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574967,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68496,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303344,"statfs":{"total":96636764160,"available":96566624256,"internally_reserved":0,"allocated":1249280,"data_stored":540250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574965,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58756,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313084,"statfs":{"total":96636764160,"available":96576598016,"internally_reserved":0,"allocated":1236992,"data_stored":533975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94852,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94276988,"statfs":{"total":96636764160,"available":96539635712,"internally_reserved":0,"allocated":1236992,"data_stored":533975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574965,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59352,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312488,"statfs":{"total":96636764160,"available":96575987712,"internally_reserved":0,"allocated":1843200,"data_stored":1130618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:47:33.032 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:47:33.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.190+0000 7fe58cfc5640 1 Processor -- start 2024-04-17T10:47:33.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.190+0000 7fe58cfc5640 1 -- start start 2024-04-17T10:47:33.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.190+0000 7fe58cfc5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe588072330 0x7fe5880cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:33.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.190+0000 7fe58cfc5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe5880cfa10 0x7fe58806c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:33.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.190+0000 7fe58cfc5640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5880cfea0 con 0x7fe58815ba30 2024-04-17T10:47:33.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.190+0000 7fe58cfc5640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe588071ea0 con 0x7fe588072330 2024-04-17T10:47:33.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.190+0000 7fe58cfc5640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe58806e320 con 0x7fe5880cfa10 2024-04-17T10:47:33.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.190+0000 7fe585d9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe588072330 0x7fe5880cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:33.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.190+0000 7fe585d9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe588072330 0x7fe5880cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60456/0 (socket says 172.21.15.83:60456) 2024-04-17T10:47:33.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.190+0000 7fe585d9b640 1 -- 172.21.15.83:0/3074988771 learned_addr learned my addr 172.21.15.83:0/3074988771 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:33.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.190+0000 7fe586d9d640 1 --2- 172.21.15.83:0/3074988771 >> v2:172.21.15.167:3300/0 conn(0x7fe5880cfa10 0x7fe58806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:33.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.190+0000 7fe585d9b640 1 -- 172.21.15.83:0/3074988771 >> v2:172.21.15.167:3300/0 conn(0x7fe5880cfa10 msgr2=0x7fe58806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:33.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.190+0000 7fe585d9b640 1 --2- 172.21.15.83:0/3074988771 >> v2:172.21.15.167:3300/0 conn(0x7fe5880cfa10 0x7fe58806c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:33.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.190+0000 7fe585d9b640 1 -- 172.21.15.83:0/3074988771 >> v1:172.21.15.167:6789/0 conn(0x7fe58815ba30 legacy=0x7fe58815c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:33.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.190+0000 7fe585d9b640 1 -- 172.21.15.83:0/3074988771 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe58806e040 con 0x7fe588072330 2024-04-17T10:47:33.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.194+0000 7fe585d9b640 1 --2- 172.21.15.83:0/3074988771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe588072330 0x7fe5880cf4d0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fe5880676a0 tx=0x7fe57c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54efe1d6cde9aadf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:33.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.194+0000 7fe58559a640 1 -- 172.21.15.83:0/3074988771 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c0b2070 con 0x7fe588072330 2024-04-17T10:47:33.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.194+0000 7fe58559a640 1 -- 172.21.15.83:0/3074988771 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57c0a9070 con 0x7fe588072330 2024-04-17T10:47:33.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.194+0000 7fe58559a640 1 -- 172.21.15.83:0/3074988771 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c0ad880 con 0x7fe588072330 2024-04-17T10:47:33.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.194+0000 7fe58cfc5640 1 -- 172.21.15.83:0/3074988771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe588072330 msgr2=0x7fe5880cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:33.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.194+0000 7fe58cfc5640 1 --2- 172.21.15.83:0/3074988771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe588072330 0x7fe5880cf4d0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fe5880676a0 tx=0x7fe57c01c450 comp rx=0 tx=0).stop 2024-04-17T10:47:33.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.194+0000 7fe58cfc5640 1 -- 172.21.15.83:0/3074988771 shutdown_connections 2024-04-17T10:47:33.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.194+0000 7fe58cfc5640 1 --2- 172.21.15.83:0/3074988771 >> v2:172.21.15.167:3300/0 conn(0x7fe5880cfa10 0x7fe58806c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:33.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.194+0000 7fe58cfc5640 1 --2- 172.21.15.83:0/3074988771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe588072330 0x7fe5880cf4d0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:33.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.194+0000 7fe58cfc5640 1 -- 172.21.15.83:0/3074988771 >> 172.21.15.83:0/3074988771 conn(0x7fe58806abe0 msgr2=0x7fe58806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:33.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.194+0000 7fe58cfc5640 1 -- 172.21.15.83:0/3074988771 shutdown_connections 2024-04-17T10:47:33.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.194+0000 7fe58cfc5640 1 -- 172.21.15.83:0/3074988771 wait complete. 2024-04-17T10:47:33.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.198+0000 7fe58cfc5640 1 Processor -- start 2024-04-17T10:47:33.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.198+0000 7fe58cfc5640 1 -- start start 2024-04-17T10:47:33.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.198+0000 7fe58cfc5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe588072330 0x7fe58814d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:33.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.198+0000 7fe58cfc5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5880cfa10 0x7fe58814e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:33.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.198+0000 7fe58cfc5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe58815ba30 0x7fe588155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:33.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.198+0000 7fe58cfc5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58806de20 con 0x7fe588072330 2024-04-17T10:47:33.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.198+0000 7fe58cfc5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58806dca0 con 0x7fe5880cfa10 2024-04-17T10:47:33.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.198+0000 7fe58cfc5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe588168520 con 0x7fe58815ba30 2024-04-17T10:47:33.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.198+0000 7fe58659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe588072330 0x7fe58814d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:33.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.198+0000 7fe58659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe588072330 0x7fe58814d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60472/0 (socket says 172.21.15.83:60472) 2024-04-17T10:47:33.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.198+0000 7fe58659c640 1 -- 172.21.15.83:0/27319575 learned_addr learned my addr 172.21.15.83:0/27319575 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:33.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.198+0000 7fe585d9b640 1 --2- 172.21.15.83:0/27319575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5880cfa10 0x7fe58814e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:33.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.198+0000 7fe586d9d640 1 --2- 172.21.15.83:0/27319575 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe58815ba30 0x7fe588155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:33.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.198+0000 7fe58659c640 1 -- 172.21.15.83:0/27319575 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe58815ba30 msgr2=0x7fe588155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:33.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.198+0000 7fe58659c640 1 --2- 172.21.15.83:0/27319575 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe58815ba30 0x7fe588155730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:33.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.198+0000 7fe58659c640 1 -- 172.21.15.83:0/27319575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5880cfa10 msgr2=0x7fe58814e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:33.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.198+0000 7fe58659c640 1 --2- 172.21.15.83:0/27319575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5880cfa10 0x7fe58814e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:33.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.198+0000 7fe58659c640 1 -- 172.21.15.83:0/27319575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe588156df0 con 0x7fe588072330 2024-04-17T10:47:33.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.198+0000 7fe585d9b640 1 --2- 172.21.15.83:0/27319575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5880cfa10 0x7fe58814e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:33.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.198+0000 7fe586d9d640 1 --2- 172.21.15.83:0/27319575 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe58815ba30 0x7fe588155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:33.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.198+0000 7fe58659c640 1 --2- 172.21.15.83:0/27319575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe588072330 0x7fe58814d470 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fe57800c490 tx=0x7fe578001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:33.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.198+0000 7fe576ffd640 1 -- 172.21.15.83:0/27319575 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe578017070 con 0x7fe588072330 2024-04-17T10:47:33.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.198+0000 7fe576ffd640 1 -- 172.21.15.83:0/27319575 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe578003600 con 0x7fe588072330 2024-04-17T10:47:33.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.198+0000 7fe576ffd640 1 -- 172.21.15.83:0/27319575 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5780039f0 con 0x7fe588072330 2024-04-17T10:47:33.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.198+0000 7fe58cfc5640 1 -- 172.21.15.83:0/27319575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe58814d9b0 con 0x7fe588072330 2024-04-17T10:47:33.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.198+0000 7fe58cfc5640 1 -- 172.21.15.83:0/27319575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe58814f450 con 0x7fe588072330 2024-04-17T10:47:33.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.198+0000 7fe58cfc5640 1 -- 172.21.15.83:0/27319575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe588074f90 con 0x7fe588072330 2024-04-17T10:47:33.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.202+0000 7fe576ffd640 1 -- 172.21.15.83:0/27319575 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe578003b90 con 0x7fe588072330 2024-04-17T10:47:33.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.202+0000 7fe576ffd640 1 --2- 172.21.15.83:0/27319575 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe568066f60 0x7fe568069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:33.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.202+0000 7fe576ffd640 1 -- 172.21.15.83:0/27319575 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe578084380 con 0x7fe588072330 2024-04-17T10:47:33.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.202+0000 7fe585d9b640 1 --2- 172.21.15.83:0/27319575 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe568066f60 0x7fe568069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:33.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.202+0000 7fe585d9b640 1 --2- 172.21.15.83:0/27319575 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe568066f60 0x7fe568069420 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe57c0af000 tx=0x7fe57c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:33.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.206+0000 7fe576ffd640 1 -- 172.21.15.83:0/27319575 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe578055390 con 0x7fe588072330 2024-04-17T10:47:33.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.350+0000 7fe58cfc5640 1 -- 172.21.15.83:0/27319575 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe58806b160 con 0x7fe568066f60 2024-04-17T10:47:33.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.366+0000 7fe576ffd640 1 -- 172.21.15.83:0/27319575 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247088 (secure 0 0 0) 0x7fe58806b160 con 0x7fe568066f60 2024-04-17T10:47:33.365 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:47:33.366 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:47:33.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.370+0000 7fe58cfc5640 1 -- 172.21.15.83:0/27319575 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe568066f60 msgr2=0x7fe568069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:33.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.370+0000 7fe58cfc5640 1 --2- 172.21.15.83:0/27319575 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe568066f60 0x7fe568069420 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe57c0af000 tx=0x7fe57c008040 comp rx=0 tx=0).stop 2024-04-17T10:47:33.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.370+0000 7fe58cfc5640 1 -- 172.21.15.83:0/27319575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe588072330 msgr2=0x7fe58814d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:33.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.370+0000 7fe58cfc5640 1 --2- 172.21.15.83:0/27319575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe588072330 0x7fe58814d470 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fe57800c490 tx=0x7fe578001e80 comp rx=0 tx=0).stop 2024-04-17T10:47:33.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.370+0000 7fe58cfc5640 1 -- 172.21.15.83:0/27319575 shutdown_connections 2024-04-17T10:47:33.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.370+0000 7fe58cfc5640 1 --2- 172.21.15.83:0/27319575 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe568066f60 0x7fe568069420 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:33.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.370+0000 7fe58cfc5640 1 --2- 172.21.15.83:0/27319575 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe58815ba30 0x7fe588155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:33.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.370+0000 7fe58cfc5640 1 --2- 172.21.15.83:0/27319575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5880cfa10 0x7fe58814e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:33.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.370+0000 7fe58cfc5640 1 --2- 172.21.15.83:0/27319575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe588072330 0x7fe58814d470 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:33.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.370+0000 7fe58cfc5640 1 -- 172.21.15.83:0/27319575 >> 172.21.15.83:0/27319575 conn(0x7fe58806abe0 msgr2=0x7fe588159b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:33.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.370+0000 7fe58cfc5640 1 -- 172.21.15.83:0/27319575 shutdown_connections 2024-04-17T10:47:33.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.370+0000 7fe58cfc5640 1 -- 172.21.15.83:0/27319575 wait complete. 2024-04-17T10:47:33.383 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2024-04-17T10:47:32.348528+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":524064,"kb_used_data":10860,"kb_used_omap":12,"kb_used_meta":513075,"kb_avail":754450656,"statfs":{"total":773094113280,"available":772557471744,"internally_reserved":0,"allocated":11120640,"data_stored":5471361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":525389360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.340819"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.031507+0000","last_change":"2024-04-17T10:47:29.031507+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.031507+0000","last_undegraded":"2024-04-17T10:47:29.031507+0000","last_fullsized":"2024-04-17T10:47:29.031507+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.055397+0000","last_change":"2024-04-17T10:47:29.055397+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.055397+0000","last_undegraded":"2024-04-17T10:47:29.055397+0000","last_fullsized":"2024-04-17T10:47:29.055397+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.027558+0000","last_change":"2024-04-17T10:47:29.027558+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.027558+0000","last_undegraded":"2024-04-17T10:47:29.027558+0000","last_fullsized":"2024-04-17T10:47:29.027558+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.044961+0000","last_change":"2024-04-17T10:47:29.044961+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.044961+0000","last_undegraded":"2024-04-17T10:47:29.044961+0000","last_fullsized":"2024-04-17T10:47:29.044961+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.038581+0000","last_change":"2024-04-17T10:47:29.038581+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.038581+0000","last_undegraded":"2024-04-17T10:47:29.038581+0000","last_fullsized":"2024-04-17T10:47:29.038581+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.021494+0000","last_change":"2024-04-17T10:47:29.021494+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.021494+0000","last_undegraded":"2024-04-17T10:47:29.021494+0000","last_fullsized":"2024-04-17T10:47:29.021494+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.025999+0000","last_change":"2024-04-17T10:47:29.025999+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.025999+0000","last_undegraded":"2024-04-17T10:47:29.025999+0000","last_fullsized":"2024-04-17T10:47:29.025999+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.049465+0000","last_change":"2024-04-17T10:47:29.049465+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.049465+0000","last_undegraded":"2024-04-17T10:47:29.049465+0000","last_fullsized":"2024-04-17T10:47:29.049465+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.024602+0000","last_change":"2024-04-17T10:47:29.024602+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.024602+0000","last_undegraded":"2024-04-17T10:47:29.024602+0000","last_fullsized":"2024-04-17T10:47:29.024602+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.030562+0000","last_change":"2024-04-17T10:47:29.030562+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.030562+0000","last_undegraded":"2024-04-17T10:47:29.030562+0000","last_fullsized":"2024-04-17T10:47:29.030562+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.032399+0000","last_change":"2024-04-17T10:47:29.032399+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.032399+0000","last_undegraded":"2024-04-17T10:47:29.032399+0000","last_fullsized":"2024-04-17T10:47:29.032399+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.029188+0000","last_change":"2024-04-17T10:47:29.029188+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.029188+0000","last_undegraded":"2024-04-17T10:47:29.029188+0000","last_fullsized":"2024-04-17T10:47:29.029188+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.046072+0000","last_change":"2024-04-17T10:47:29.046072+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.046072+0000","last_undegraded":"2024-04-17T10:47:29.046072+0000","last_fullsized":"2024-04-17T10:47:29.046072+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.037835+0000","last_change":"2024-04-17T10:47:29.037835+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.037835+0000","last_undegraded":"2024-04-17T10:47:29.037835+0000","last_fullsized":"2024-04-17T10:47:29.037835+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.045596+0000","last_change":"2024-04-17T10:47:29.045596+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.045596+0000","last_undegraded":"2024-04-17T10:47:29.045596+0000","last_fullsized":"2024-04-17T10:47:29.045596+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.028858+0000","last_change":"2024-04-17T10:47:29.028858+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.028858+0000","last_undegraded":"2024-04-17T10:47:29.028858+0000","last_fullsized":"2024-04-17T10:47:29.028858+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.044488+0000","last_change":"2024-04-17T10:47:29.044488+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.044488+0000","last_undegraded":"2024-04-17T10:47:29.044488+0000","last_fullsized":"2024-04-17T10:47:29.044488+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.044801+0000","last_change":"2024-04-17T10:47:29.044801+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.044801+0000","last_undegraded":"2024-04-17T10:47:29.044801+0000","last_fullsized":"2024-04-17T10:47:29.044801+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.034394+0000","last_change":"2024-04-17T10:47:29.034394+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.034394+0000","last_undegraded":"2024-04-17T10:47:29.034394+0000","last_fullsized":"2024-04-17T10:47:29.034394+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.023972+0000","last_change":"2024-04-17T10:47:29.023972+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.023972+0000","last_undegraded":"2024-04-17T10:47:29.023972+0000","last_fullsized":"2024-04-17T10:47:29.023972+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.034527+0000","last_change":"2024-04-17T10:47:29.034527+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.034527+0000","last_undegraded":"2024-04-17T10:47:29.034527+0000","last_fullsized":"2024-04-17T10:47:29.034527+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.045648+0000","last_change":"2024-04-17T10:47:29.045648+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.045648+0000","last_undegraded":"2024-04-17T10:47:29.045648+0000","last_fullsized":"2024-04-17T10:47:29.045648+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.045251+0000","last_change":"2024-04-17T10:47:29.045251+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.045251+0000","last_undegraded":"2024-04-17T10:47:29.045251+0000","last_fullsized":"2024-04-17T10:47:29.045251+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.027697+0000","last_change":"2024-04-17T10:47:29.027697+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.027697+0000","last_undegraded":"2024-04-17T10:47:29.027697+0000","last_fullsized":"2024-04-17T10:47:29.027697+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.037694+0000","last_change":"2024-04-17T10:47:29.037694+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.037694+0000","last_undegraded":"2024-04-17T10:47:29.037694+0000","last_fullsized":"2024-04-17T10:47:29.037694+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.050922+0000","last_change":"2024-04-17T10:47:29.050922+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.050922+0000","last_undegraded":"2024-04-17T10:47:29.050922+0000","last_fullsized":"2024-04-17T10:47:29.050922+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.024737+0000","last_change":"2024-04-17T10:47:29.024737+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.024737+0000","last_undegraded":"2024-04-17T10:47:29.024737+0000","last_fullsized":"2024-04-17T10:47:29.024737+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.043747+0000","last_change":"2024-04-17T10:47:29.043747+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.043747+0000","last_undegraded":"2024-04-17T10:47:29.043747+0000","last_fullsized":"2024-04-17T10:47:29.043747+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.047296+0000","last_change":"2024-04-17T10:47:29.047296+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.047296+0000","last_undegraded":"2024-04-17T10:47:29.047296+0000","last_fullsized":"2024-04-17T10:47:29.047296+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.035574+0000","last_change":"2024-04-17T10:47:29.035574+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.035574+0000","last_undegraded":"2024-04-17T10:47:29.035574+0000","last_fullsized":"2024-04-17T10:47:29.035574+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.045037+0000","last_change":"2024-04-17T10:47:29.045037+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.045037+0000","last_undegraded":"2024-04-17T10:47:29.045037+0000","last_fullsized":"2024-04-17T10:47:29.045037+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.028571+0000","last_change":"2024-04-17T10:47:29.028571+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.028571+0000","last_undegraded":"2024-04-17T10:47:29.028571+0000","last_fullsized":"2024-04-17T10:47:29.028571+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.034477+0000","last_change":"2024-04-17T10:47:29.034477+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.034477+0000","last_undegraded":"2024-04-17T10:47:29.034477+0000","last_fullsized":"2024-04-17T10:47:29.034477+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.058518+0000","last_change":"2024-04-17T10:47:29.058101+0000","last_active":"2024-04-17T10:47:29.058518+0000","last_peered":"2024-04-17T10:47:29.058518+0000","last_clean":"2024-04-17T10:47:29.058518+0000","last_became_active":"2024-04-17T10:47:29.056266+0000","last_became_peered":"2024-04-17T10:47:29.056266+0000","last_unstale":"2024-04-17T10:47:29.058518+0000","last_undegraded":"2024-04-17T10:47:29.058518+0000","last_fullsized":"2024-04-17T10:47:29.058518+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:10:51.407123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.047078+0000","last_change":"2024-04-17T10:47:29.047078+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.047078+0000","last_undegraded":"2024-04-17T10:47:29.047078+0000","last_fullsized":"2024-04-17T10:47:29.047078+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.044854+0000","last_change":"2024-04-17T10:47:29.037268+0000","last_active":"2024-04-17T10:47:29.044854+0000","last_peered":"2024-04-17T10:47:29.044854+0000","last_clean":"2024-04-17T10:47:29.044854+0000","last_became_active":"2024-04-17T10:47:29.035743+0000","last_became_peered":"2024-04-17T10:47:29.035743+0000","last_unstale":"2024-04-17T10:47:29.044854+0000","last_undegraded":"2024-04-17T10:47:29.044854+0000","last_fullsized":"2024-04-17T10:47:29.044854+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:16:45.173638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.030657+0000","last_change":"2024-04-17T10:47:29.030657+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.030657+0000","last_undegraded":"2024-04-17T10:47:29.030657+0000","last_fullsized":"2024-04-17T10:47:29.030657+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.051205+0000","last_change":"2024-04-17T10:47:29.046225+0000","last_active":"2024-04-17T10:47:29.051205+0000","last_peered":"2024-04-17T10:47:29.051205+0000","last_clean":"2024-04-17T10:47:29.051205+0000","last_became_active":"2024-04-17T10:47:29.043738+0000","last_became_peered":"2024-04-17T10:47:29.043738+0000","last_unstale":"2024-04-17T10:47:29.051205+0000","last_undegraded":"2024-04-17T10:47:29.051205+0000","last_fullsized":"2024-04-17T10:47:29.051205+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:43.205753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.023044+0000","last_change":"2024-04-17T10:47:29.023044+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.023044+0000","last_undegraded":"2024-04-17T10:47:29.023044+0000","last_fullsized":"2024-04-17T10:47:29.023044+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.042042+0000","last_change":"2024-04-17T10:47:29.041206+0000","last_active":"2024-04-17T10:47:29.042042+0000","last_peered":"2024-04-17T10:47:29.042042+0000","last_clean":"2024-04-17T10:47:29.042042+0000","last_became_active":"2024-04-17T10:47:29.040924+0000","last_became_peered":"2024-04-17T10:47:29.040924+0000","last_unstale":"2024-04-17T10:47:29.042042+0000","last_undegraded":"2024-04-17T10:47:29.042042+0000","last_fullsized":"2024-04-17T10:47:29.042042+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:35:16.067757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.028743+0000","last_change":"2024-04-17T10:47:29.028743+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.028743+0000","last_undegraded":"2024-04-17T10:47:29.028743+0000","last_fullsized":"2024-04-17T10:47:29.028743+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.054287+0000","last_change":"2024-04-17T10:47:29.053852+0000","last_active":"2024-04-17T10:47:29.054287+0000","last_peered":"2024-04-17T10:47:29.054287+0000","last_clean":"2024-04-17T10:47:29.054287+0000","last_became_active":"2024-04-17T10:47:29.053434+0000","last_became_peered":"2024-04-17T10:47:29.053434+0000","last_unstale":"2024-04-17T10:47:29.054287+0000","last_undegraded":"2024-04-17T10:47:29.054287+0000","last_fullsized":"2024-04-17T10:47:29.054287+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:36:34.892543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.030058+0000","last_change":"2024-04-17T10:47:29.030058+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.030058+0000","last_undegraded":"2024-04-17T10:47:29.030058+0000","last_fullsized":"2024-04-17T10:47:29.030058+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.041154+0000","last_change":"2024-04-17T10:47:29.040335+0000","last_active":"2024-04-17T10:47:29.041154+0000","last_peered":"2024-04-17T10:47:29.041154+0000","last_clean":"2024-04-17T10:47:29.041154+0000","last_became_active":"2024-04-17T10:47:29.039941+0000","last_became_peered":"2024-04-17T10:47:29.039941+0000","last_unstale":"2024-04-17T10:47:29.041154+0000","last_undegraded":"2024-04-17T10:47:29.041154+0000","last_fullsized":"2024-04-17T10:47:29.041154+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:31:48.398246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.044793+0000","last_change":"2024-04-17T10:47:29.044793+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.044793+0000","last_undegraded":"2024-04-17T10:47:29.044793+0000","last_fullsized":"2024-04-17T10:47:29.044793+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.052502+0000","last_change":"2024-04-17T10:47:29.045408+0000","last_active":"2024-04-17T10:47:29.052502+0000","last_peered":"2024-04-17T10:47:29.052502+0000","last_clean":"2024-04-17T10:47:29.052502+0000","last_became_active":"2024-04-17T10:47:29.044801+0000","last_became_peered":"2024-04-17T10:47:29.044801+0000","last_unstale":"2024-04-17T10:47:29.052502+0000","last_undegraded":"2024-04-17T10:47:29.052502+0000","last_fullsized":"2024-04-17T10:47:29.052502+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:52:11.119097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.024952+0000","last_change":"2024-04-17T10:47:29.024952+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.024952+0000","last_undegraded":"2024-04-17T10:47:29.024952+0000","last_fullsized":"2024-04-17T10:47:29.024952+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.046171+0000","last_change":"2024-04-17T10:47:29.038936+0000","last_active":"2024-04-17T10:47:29.046171+0000","last_peered":"2024-04-17T10:47:29.046171+0000","last_clean":"2024-04-17T10:47:29.046171+0000","last_became_active":"2024-04-17T10:47:29.038436+0000","last_became_peered":"2024-04-17T10:47:29.038436+0000","last_unstale":"2024-04-17T10:47:29.046171+0000","last_undegraded":"2024-04-17T10:47:29.046171+0000","last_fullsized":"2024-04-17T10:47:29.046171+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:20:55.139531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.049911+0000","last_change":"2024-04-17T10:47:29.049911+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.049911+0000","last_undegraded":"2024-04-17T10:47:29.049911+0000","last_fullsized":"2024-04-17T10:47:29.049911+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.042156+0000","last_change":"2024-04-17T10:47:29.040517+0000","last_active":"2024-04-17T10:47:29.042156+0000","last_peered":"2024-04-17T10:47:29.042156+0000","last_clean":"2024-04-17T10:47:29.042156+0000","last_became_active":"2024-04-17T10:47:29.039818+0000","last_became_peered":"2024-04-17T10:47:29.039818+0000","last_unstale":"2024-04-17T10:47:29.042156+0000","last_undegraded":"2024-04-17T10:47:29.042156+0000","last_fullsized":"2024-04-17T10:47:29.042156+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:49:17.199444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.023180+0000","last_change":"2024-04-17T10:47:29.023180+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.023180+0000","last_undegraded":"2024-04-17T10:47:29.023180+0000","last_fullsized":"2024-04-17T10:47:29.023180+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.042609+0000","last_change":"2024-04-17T10:47:29.037265+0000","last_active":"2024-04-17T10:47:29.042609+0000","last_peered":"2024-04-17T10:47:29.042609+0000","last_clean":"2024-04-17T10:47:29.042609+0000","last_became_active":"2024-04-17T10:47:29.036735+0000","last_became_peered":"2024-04-17T10:47:29.036735+0000","last_unstale":"2024-04-17T10:47:29.042609+0000","last_undegraded":"2024-04-17T10:47:29.042609+0000","last_fullsized":"2024-04-17T10:47:29.042609+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:49.197211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.042244+0000","last_change":"2024-04-17T10:47:29.042244+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.042244+0000","last_undegraded":"2024-04-17T10:47:29.042244+0000","last_fullsized":"2024-04-17T10:47:29.042244+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.055631+0000","last_change":"2024-04-17T10:47:29.049241+0000","last_active":"2024-04-17T10:47:29.055631+0000","last_peered":"2024-04-17T10:47:29.055631+0000","last_clean":"2024-04-17T10:47:29.055631+0000","last_became_active":"2024-04-17T10:47:29.048527+0000","last_became_peered":"2024-04-17T10:47:29.048527+0000","last_unstale":"2024-04-17T10:47:29.055631+0000","last_undegraded":"2024-04-17T10:47:29.055631+0000","last_fullsized":"2024-04-17T10:47:29.055631+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:26:50.584899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.023708+0000","last_change":"2024-04-17T10:47:29.023708+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.023708+0000","last_undegraded":"2024-04-17T10:47:29.023708+0000","last_fullsized":"2024-04-17T10:47:29.023708+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.050875+0000","last_change":"2024-04-17T10:47:29.047222+0000","last_active":"2024-04-17T10:47:29.050875+0000","last_peered":"2024-04-17T10:47:29.050875+0000","last_clean":"2024-04-17T10:47:29.050875+0000","last_became_active":"2024-04-17T10:47:29.046540+0000","last_became_peered":"2024-04-17T10:47:29.046540+0000","last_unstale":"2024-04-17T10:47:29.050875+0000","last_undegraded":"2024-04-17T10:47:29.050875+0000","last_fullsized":"2024-04-17T10:47:29.050875+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:41:18.010105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.045109+0000","last_change":"2024-04-17T10:47:29.045109+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.045109+0000","last_undegraded":"2024-04-17T10:47:29.045109+0000","last_fullsized":"2024-04-17T10:47:29.045109+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.051787+0000","last_change":"2024-04-17T10:47:29.051191+0000","last_active":"2024-04-17T10:47:29.051787+0000","last_peered":"2024-04-17T10:47:29.051787+0000","last_clean":"2024-04-17T10:47:29.051787+0000","last_became_active":"2024-04-17T10:47:29.050924+0000","last_became_peered":"2024-04-17T10:47:29.050924+0000","last_unstale":"2024-04-17T10:47:29.051787+0000","last_undegraded":"2024-04-17T10:47:29.051787+0000","last_fullsized":"2024-04-17T10:47:29.051787+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:47:39.522424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.027964+0000","last_change":"2024-04-17T10:47:29.027964+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.027964+0000","last_undegraded":"2024-04-17T10:47:29.027964+0000","last_fullsized":"2024-04-17T10:47:29.027964+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.043366+0000","last_change":"2024-04-17T10:47:29.041596+0000","last_active":"2024-04-17T10:47:29.043366+0000","last_peered":"2024-04-17T10:47:29.043366+0000","last_clean":"2024-04-17T10:47:29.043366+0000","last_became_active":"2024-04-17T10:47:29.041198+0000","last_became_peered":"2024-04-17T10:47:29.041198+0000","last_unstale":"2024-04-17T10:47:29.043366+0000","last_undegraded":"2024-04-17T10:47:29.043366+0000","last_fullsized":"2024-04-17T10:47:29.043366+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:33:24.133568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.024813+0000","last_change":"2024-04-17T10:47:29.024813+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.024813+0000","last_undegraded":"2024-04-17T10:47:29.024813+0000","last_fullsized":"2024-04-17T10:47:29.024813+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.050803+0000","last_change":"2024-04-17T10:47:29.047014+0000","last_active":"2024-04-17T10:47:29.050803+0000","last_peered":"2024-04-17T10:47:29.050803+0000","last_clean":"2024-04-17T10:47:29.050803+0000","last_became_active":"2024-04-17T10:47:29.046269+0000","last_became_peered":"2024-04-17T10:47:29.046269+0000","last_unstale":"2024-04-17T10:47:29.050803+0000","last_undegraded":"2024-04-17T10:47:29.050803+0000","last_fullsized":"2024-04-17T10:47:29.050803+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:59.960069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.034425+0000","last_change":"2024-04-17T10:47:29.034425+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.034425+0000","last_undegraded":"2024-04-17T10:47:29.034425+0000","last_fullsized":"2024-04-17T10:47:29.034425+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.040239+0000","last_change":"2024-04-17T10:47:29.037711+0000","last_active":"2024-04-17T10:47:29.040239+0000","last_peered":"2024-04-17T10:47:29.040239+0000","last_clean":"2024-04-17T10:47:29.040239+0000","last_became_active":"2024-04-17T10:47:29.037183+0000","last_became_peered":"2024-04-17T10:47:29.037183+0000","last_unstale":"2024-04-17T10:47:29.040239+0000","last_undegraded":"2024-04-17T10:47:29.040239+0000","last_fullsized":"2024-04-17T10:47:29.040239+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:07:50.722366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.030365+0000","last_change":"2024-04-17T10:47:29.030365+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.030365+0000","last_undegraded":"2024-04-17T10:47:29.030365+0000","last_fullsized":"2024-04-17T10:47:29.030365+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.051165+0000","last_change":"2024-04-17T10:47:29.046483+0000","last_active":"2024-04-17T10:47:29.051165+0000","last_peered":"2024-04-17T10:47:29.051165+0000","last_clean":"2024-04-17T10:47:29.051165+0000","last_became_active":"2024-04-17T10:47:29.042701+0000","last_became_peered":"2024-04-17T10:47:29.042701+0000","last_unstale":"2024-04-17T10:47:29.051165+0000","last_undegraded":"2024-04-17T10:47:29.051165+0000","last_fullsized":"2024-04-17T10:47:29.051165+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:43.205753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.026502+0000","last_change":"2024-04-17T10:47:29.026502+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.026502+0000","last_undegraded":"2024-04-17T10:47:29.026502+0000","last_fullsized":"2024-04-17T10:47:29.026502+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.052476+0000","last_change":"2024-04-17T10:47:29.044936+0000","last_active":"2024-04-17T10:47:29.052476+0000","last_peered":"2024-04-17T10:47:29.052476+0000","last_clean":"2024-04-17T10:47:29.052476+0000","last_became_active":"2024-04-17T10:47:29.044091+0000","last_became_peered":"2024-04-17T10:47:29.044091+0000","last_unstale":"2024-04-17T10:47:29.052476+0000","last_undegraded":"2024-04-17T10:47:29.052476+0000","last_fullsized":"2024-04-17T10:47:29.052476+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:03:10.762120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.022345+0000","last_change":"2024-04-17T10:47:29.022345+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.022345+0000","last_undegraded":"2024-04-17T10:47:29.022345+0000","last_fullsized":"2024-04-17T10:47:29.022345+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.055071+0000","last_change":"2024-04-17T10:47:29.043948+0000","last_active":"2024-04-17T10:47:29.055071+0000","last_peered":"2024-04-17T10:47:29.055071+0000","last_clean":"2024-04-17T10:47:29.055071+0000","last_became_active":"2024-04-17T10:47:29.043517+0000","last_became_peered":"2024-04-17T10:47:29.043517+0000","last_unstale":"2024-04-17T10:47:29.055071+0000","last_undegraded":"2024-04-17T10:47:29.055071+0000","last_fullsized":"2024-04-17T10:47:29.055071+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:52:11.119097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.035024+0000","last_change":"2024-04-17T10:47:29.035024+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.035024+0000","last_undegraded":"2024-04-17T10:47:29.035024+0000","last_fullsized":"2024-04-17T10:47:29.035024+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.055246+0000","last_change":"2024-04-17T10:47:29.049650+0000","last_active":"2024-04-17T10:47:29.055246+0000","last_peered":"2024-04-17T10:47:29.055246+0000","last_clean":"2024-04-17T10:47:29.055246+0000","last_became_active":"2024-04-17T10:47:29.049263+0000","last_became_peered":"2024-04-17T10:47:29.049263+0000","last_unstale":"2024-04-17T10:47:29.055246+0000","last_undegraded":"2024-04-17T10:47:29.055246+0000","last_fullsized":"2024-04-17T10:47:29.055246+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:41:18.010105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.033707+0000","last_change":"2024-04-17T10:47:29.033707+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.033707+0000","last_undegraded":"2024-04-17T10:47:29.033707+0000","last_fullsized":"2024-04-17T10:47:29.033707+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.052049+0000","last_change":"2024-04-17T10:47:29.041858+0000","last_active":"2024-04-17T10:47:29.052049+0000","last_peered":"2024-04-17T10:47:29.052049+0000","last_clean":"2024-04-17T10:47:29.052049+0000","last_became_active":"2024-04-17T10:47:29.041099+0000","last_became_peered":"2024-04-17T10:47:29.041099+0000","last_unstale":"2024-04-17T10:47:29.052049+0000","last_undegraded":"2024-04-17T10:47:29.052049+0000","last_fullsized":"2024-04-17T10:47:29.052049+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:41:18.010105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.035881+0000","last_change":"2024-04-17T10:47:29.035881+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.035881+0000","last_undegraded":"2024-04-17T10:47:29.035881+0000","last_fullsized":"2024-04-17T10:47:29.035881+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.050068+0000","last_change":"2024-04-17T10:47:29.039405+0000","last_active":"2024-04-17T10:47:29.050068+0000","last_peered":"2024-04-17T10:47:29.050068+0000","last_clean":"2024-04-17T10:47:29.050068+0000","last_became_active":"2024-04-17T10:47:29.038596+0000","last_became_peered":"2024-04-17T10:47:29.038596+0000","last_unstale":"2024-04-17T10:47:29.050068+0000","last_undegraded":"2024-04-17T10:47:29.050068+0000","last_fullsized":"2024-04-17T10:47:29.050068+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:43.205753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.039260+0000","last_change":"2024-04-17T10:47:29.039260+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.039260+0000","last_undegraded":"2024-04-17T10:47:29.039260+0000","last_fullsized":"2024-04-17T10:47:29.039260+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.049867+0000","last_change":"2024-04-17T10:47:29.043402+0000","last_active":"2024-04-17T10:47:29.049867+0000","last_peered":"2024-04-17T10:47:29.049867+0000","last_clean":"2024-04-17T10:47:29.049867+0000","last_became_active":"2024-04-17T10:47:29.042994+0000","last_became_peered":"2024-04-17T10:47:29.042994+0000","last_unstale":"2024-04-17T10:47:29.049867+0000","last_undegraded":"2024-04-17T10:47:29.049867+0000","last_fullsized":"2024-04-17T10:47:29.049867+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:03:10.762120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.048879+0000","last_change":"2024-04-17T10:47:29.048879+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.048879+0000","last_undegraded":"2024-04-17T10:47:29.048879+0000","last_fullsized":"2024-04-17T10:47:29.048879+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.051965+0000","last_change":"2024-04-17T10:47:29.041380+0000","last_active":"2024-04-17T10:47:29.051965+0000","last_peered":"2024-04-17T10:47:29.051965+0000","last_clean":"2024-04-17T10:47:29.051965+0000","last_became_active":"2024-04-17T10:47:29.040614+0000","last_became_peered":"2024-04-17T10:47:29.040614+0000","last_unstale":"2024-04-17T10:47:29.051965+0000","last_undegraded":"2024-04-17T10:47:29.051965+0000","last_fullsized":"2024-04-17T10:47:29.051965+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:59.960069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.030002+0000","last_change":"2024-04-17T10:47:29.030002+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.030002+0000","last_undegraded":"2024-04-17T10:47:29.030002+0000","last_fullsized":"2024-04-17T10:47:29.030002+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.058438+0000","last_change":"2024-04-17T10:47:29.056214+0000","last_active":"2024-04-17T10:47:29.058438+0000","last_peered":"2024-04-17T10:47:29.058438+0000","last_clean":"2024-04-17T10:47:29.058438+0000","last_became_active":"2024-04-17T10:47:29.055718+0000","last_became_peered":"2024-04-17T10:47:29.055718+0000","last_unstale":"2024-04-17T10:47:29.058438+0000","last_undegraded":"2024-04-17T10:47:29.058438+0000","last_fullsized":"2024-04-17T10:47:29.058438+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:28.525331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.028102+0000","last_change":"2024-04-17T10:47:29.028102+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.028102+0000","last_undegraded":"2024-04-17T10:47:29.028102+0000","last_fullsized":"2024-04-17T10:47:29.028102+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.055096+0000","last_change":"2024-04-17T10:47:29.046779+0000","last_active":"2024-04-17T10:47:29.055096+0000","last_peered":"2024-04-17T10:47:29.055096+0000","last_clean":"2024-04-17T10:47:29.055096+0000","last_became_active":"2024-04-17T10:47:29.045869+0000","last_became_peered":"2024-04-17T10:47:29.045869+0000","last_unstale":"2024-04-17T10:47:29.055096+0000","last_undegraded":"2024-04-17T10:47:29.055096+0000","last_fullsized":"2024-04-17T10:47:29.055096+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:59.960069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.037035+0000","last_change":"2024-04-17T10:47:29.037035+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.037035+0000","last_undegraded":"2024-04-17T10:47:29.037035+0000","last_fullsized":"2024-04-17T10:47:29.037035+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.054192+0000","last_change":"2024-04-17T10:47:29.050848+0000","last_active":"2024-04-17T10:47:29.054192+0000","last_peered":"2024-04-17T10:47:29.054192+0000","last_clean":"2024-04-17T10:47:29.054192+0000","last_became_active":"2024-04-17T10:47:29.048692+0000","last_became_peered":"2024-04-17T10:47:29.048692+0000","last_unstale":"2024-04-17T10:47:29.054192+0000","last_undegraded":"2024-04-17T10:47:29.054192+0000","last_fullsized":"2024-04-17T10:47:29.054192+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:43.205753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.030935+0000","last_change":"2024-04-17T10:47:29.030935+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.030935+0000","last_undegraded":"2024-04-17T10:47:29.030935+0000","last_fullsized":"2024-04-17T10:47:29.030935+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.055330+0000","last_change":"2024-04-17T10:47:29.050890+0000","last_active":"2024-04-17T10:47:29.055330+0000","last_peered":"2024-04-17T10:47:29.055330+0000","last_clean":"2024-04-17T10:47:29.055330+0000","last_became_active":"2024-04-17T10:47:29.050540+0000","last_became_peered":"2024-04-17T10:47:29.050540+0000","last_unstale":"2024-04-17T10:47:29.055330+0000","last_undegraded":"2024-04-17T10:47:29.055330+0000","last_fullsized":"2024-04-17T10:47:29.055330+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:19:07.385658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.022875+0000","last_change":"2024-04-17T10:47:29.022875+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.022875+0000","last_undegraded":"2024-04-17T10:47:29.022875+0000","last_fullsized":"2024-04-17T10:47:29.022875+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.053684+0000","last_change":"2024-04-17T10:47:29.051971+0000","last_active":"2024-04-17T10:47:29.053684+0000","last_peered":"2024-04-17T10:47:29.053684+0000","last_clean":"2024-04-17T10:47:29.053684+0000","last_became_active":"2024-04-17T10:47:29.051054+0000","last_became_peered":"2024-04-17T10:47:29.051054+0000","last_unstale":"2024-04-17T10:47:29.053684+0000","last_undegraded":"2024-04-17T10:47:29.053684+0000","last_fullsized":"2024-04-17T10:47:29.053684+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:59.960069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.029219+0000","last_change":"2024-04-17T10:47:29.029219+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.029219+0000","last_undegraded":"2024-04-17T10:47:29.029219+0000","last_fullsized":"2024-04-17T10:47:29.029219+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.037159+0000","last_change":"2024-04-17T10:47:29.032908+0000","last_active":"2024-04-17T10:47:29.037159+0000","last_peered":"2024-04-17T10:47:29.037159+0000","last_clean":"2024-04-17T10:47:29.037159+0000","last_became_active":"2024-04-17T10:47:29.032182+0000","last_became_peered":"2024-04-17T10:47:29.032182+0000","last_unstale":"2024-04-17T10:47:29.037159+0000","last_undegraded":"2024-04-17T10:47:29.037159+0000","last_fullsized":"2024-04-17T10:47:29.037159+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:11:47.505394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.038374+0000","last_change":"2024-04-17T10:47:29.038374+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.038374+0000","last_undegraded":"2024-04-17T10:47:29.038374+0000","last_fullsized":"2024-04-17T10:47:29.038374+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.047599+0000","last_change":"2024-04-17T10:47:29.039563+0000","last_active":"2024-04-17T10:47:29.047599+0000","last_peered":"2024-04-17T10:47:29.047599+0000","last_clean":"2024-04-17T10:47:29.047599+0000","last_became_active":"2024-04-17T10:47:29.039021+0000","last_became_peered":"2024-04-17T10:47:29.039021+0000","last_unstale":"2024-04-17T10:47:29.047599+0000","last_undegraded":"2024-04-17T10:47:29.047599+0000","last_fullsized":"2024-04-17T10:47:29.047599+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:52.837030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.033845+0000","last_change":"2024-04-17T10:47:29.033845+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.033845+0000","last_undegraded":"2024-04-17T10:47:29.033845+0000","last_fullsized":"2024-04-17T10:47:29.033845+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.035392+0000","last_change":"2024-04-17T10:47:29.031551+0000","last_active":"2024-04-17T10:47:29.035392+0000","last_peered":"2024-04-17T10:47:29.035392+0000","last_clean":"2024-04-17T10:47:29.035392+0000","last_became_active":"2024-04-17T10:47:29.030845+0000","last_became_peered":"2024-04-17T10:47:29.030845+0000","last_unstale":"2024-04-17T10:47:29.035392+0000","last_undegraded":"2024-04-17T10:47:29.035392+0000","last_fullsized":"2024-04-17T10:47:29.035392+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:49.197211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":336,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.029346+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:47:29.029346+0000","last_peered":"2024-04-17T10:47:29.029346+0000","last_clean":"2024-04-17T10:47:29.029346+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:47:29.029346+0000","last_undegraded":"2024-04-17T10:47:29.029346+0000","last_fullsized":"2024-04-17T10:47:29.029346+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574966,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59340,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312500,"statfs":{"total":96636764160,"available":96576000000,"internally_reserved":0,"allocated":1830912,"data_stored":1124343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574965,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64328,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307512,"statfs":{"total":96636764160,"available":96570892288,"internally_reserved":0,"allocated":1236992,"data_stored":533975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574966,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50440,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321400,"statfs":{"total":96636764160,"available":96585113600,"internally_reserved":0,"allocated":1236992,"data_stored":533975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574965,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68500,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303340,"statfs":{"total":96636764160,"available":96566620160,"internally_reserved":0,"allocated":1249280,"data_stored":540250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574967,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68496,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303344,"statfs":{"total":96636764160,"available":96566624256,"internally_reserved":0,"allocated":1249280,"data_stored":540250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574965,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58756,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313084,"statfs":{"total":96636764160,"available":96576598016,"internally_reserved":0,"allocated":1236992,"data_stored":533975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94852,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94276988,"statfs":{"total":96636764160,"available":96539635712,"internally_reserved":0,"allocated":1236992,"data_stored":533975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574965,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59352,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312488,"statfs":{"total":96636764160,"available":96575987712,"internally_reserved":0,"allocated":1843200,"data_stored":1130618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:47:33.387 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:47:33.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.546+0000 7f0bc532e640 1 Processor -- start 2024-04-17T10:47:33.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.546+0000 7f0bc532e640 1 -- start start 2024-04-17T10:47:33.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.546+0000 7f0bc532e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bc015db50 0x7f0bc0070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:33.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.546+0000 7f0bc532e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0bc006dcb0 0x7f0bc00b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:33.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.546+0000 7f0bc532e640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bc00709b0 con 0x7f0bc00aedd0 2024-04-17T10:47:33.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.546+0000 7f0bc532e640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bc015e440 con 0x7f0bc006dcb0 2024-04-17T10:47:33.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.546+0000 7f0bc532e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc015e720 con 0x7f0bc015db50 2024-04-17T10:47:33.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.546+0000 7f0bbf7fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0bc006dcb0 0x7f0bc00b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:33.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.546+0000 7f0bbeffd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bc015db50 0x7f0bc0070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:33.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.546+0000 7f0bb7fff640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f0bc00aedd0 0x7f0bc006d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:56052/0 (socket says 172.21.15.83:56052) 2024-04-17T10:47:33.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.546+0000 7f0bb7fff640 1 -- 172.21.15.83:0/267614818 learned_addr learned my addr 172.21.15.83:0/267614818 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:33.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.546+0000 7f0bbeffd640 1 -- 172.21.15.83:0/267614818 >> v2:172.21.15.167:3300/0 conn(0x7f0bc006dcb0 msgr2=0x7f0bc00b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:33.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.546+0000 7f0bbeffd640 1 --2- 172.21.15.83:0/267614818 >> v2:172.21.15.167:3300/0 conn(0x7f0bc006dcb0 0x7f0bc00b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:33.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.546+0000 7f0bbeffd640 1 -- 172.21.15.83:0/267614818 >> v1:172.21.15.167:6789/0 conn(0x7f0bc00aedd0 legacy=0x7f0bc006d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:33.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.546+0000 7f0bbeffd640 1 -- 172.21.15.83:0/267614818 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc015ea00 con 0x7f0bc015db50 2024-04-17T10:47:33.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.550+0000 7f0bbeffd640 1 --2- 172.21.15.83:0/267614818 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bc015db50 0x7f0bc0070470 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0bb00097c0 tx=0x7f0bb001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=934b3e8014bdf3a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:33.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.550+0000 7f0bbe7fc640 1 -- 172.21.15.83:0/267614818 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb00a7070 con 0x7f0bc015db50 2024-04-17T10:47:33.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.550+0000 7f0bbe7fc640 1 -- 172.21.15.83:0/267614818 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb00b0070 con 0x7f0bc015db50 2024-04-17T10:47:33.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.550+0000 7f0bbe7fc640 1 -- 172.21.15.83:0/267614818 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb00ab890 con 0x7f0bc015db50 2024-04-17T10:47:33.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.550+0000 7f0bc532e640 1 -- 172.21.15.83:0/267614818 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bc015db50 msgr2=0x7f0bc0070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:33.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.550+0000 7f0bc532e640 1 --2- 172.21.15.83:0/267614818 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bc015db50 0x7f0bc0070470 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0bb00097c0 tx=0x7f0bb001c470 comp rx=0 tx=0).stop 2024-04-17T10:47:33.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.550+0000 7f0bc532e640 1 -- 172.21.15.83:0/267614818 shutdown_connections 2024-04-17T10:47:33.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.550+0000 7f0bc532e640 1 --2- 172.21.15.83:0/267614818 >> v2:172.21.15.167:3300/0 conn(0x7f0bc006dcb0 0x7f0bc00b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:33.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.550+0000 7f0bc532e640 1 --2- 172.21.15.83:0/267614818 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bc015db50 0x7f0bc0070470 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:33.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.550+0000 7f0bc532e640 1 -- 172.21.15.83:0/267614818 >> 172.21.15.83:0/267614818 conn(0x7f0bc006abe0 msgr2=0x7f0bc006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:33.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.550+0000 7f0bc532e640 1 -- 172.21.15.83:0/267614818 shutdown_connections 2024-04-17T10:47:33.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.550+0000 7f0bc532e640 1 -- 172.21.15.83:0/267614818 wait complete. 2024-04-17T10:47:33.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.550+0000 7f0bc532e640 1 Processor -- start 2024-04-17T10:47:33.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.550+0000 7f0bc532e640 1 -- start start 2024-04-17T10:47:33.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.554+0000 7f0bc532e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bc006dcb0 0x7f0bc0150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:33.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.554+0000 7f0bc532e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bc00aedd0 0x7f0bc00b0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:33.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.554+0000 7f0bc532e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc015db50 0x7f0bc0157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:33.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.554+0000 7f0bc532e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc00b3ce0 con 0x7f0bc006dcb0 2024-04-17T10:47:33.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.554+0000 7f0bc532e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc00b3b60 con 0x7f0bc015db50 2024-04-17T10:47:33.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.554+0000 7f0bc532e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc00b3e60 con 0x7f0bc00aedd0 2024-04-17T10:47:33.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.554+0000 7f0bbeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bc006dcb0 0x7f0bc0150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:33.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.554+0000 7f0bbeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bc006dcb0 0x7f0bc0150810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60474/0 (socket says 172.21.15.83:60474) 2024-04-17T10:47:33.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.554+0000 7f0bbeffd640 1 -- 172.21.15.83:0/2578227352 learned_addr learned my addr 172.21.15.83:0/2578227352 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:33.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.554+0000 7f0bb7fff640 1 --2- 172.21.15.83:0/2578227352 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bc00aedd0 0x7f0bc00b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:33.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.554+0000 7f0bbf7fe640 1 --2- 172.21.15.83:0/2578227352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc015db50 0x7f0bc0157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:33.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.554+0000 7f0bbeffd640 1 -- 172.21.15.83:0/2578227352 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bc00aedd0 msgr2=0x7f0bc00b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:33.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.554+0000 7f0bbeffd640 1 --2- 172.21.15.83:0/2578227352 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bc00aedd0 0x7f0bc00b0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:33.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.554+0000 7f0bbeffd640 1 -- 172.21.15.83:0/2578227352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc015db50 msgr2=0x7f0bc0157880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:33.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.554+0000 7f0bbeffd640 1 --2- 172.21.15.83:0/2578227352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc015db50 0x7f0bc0157880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:33.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.554+0000 7f0bbeffd640 1 -- 172.21.15.83:0/2578227352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc0158c50 con 0x7f0bc006dcb0 2024-04-17T10:47:33.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.554+0000 7f0bbeffd640 1 --2- 172.21.15.83:0/2578227352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bc006dcb0 0x7f0bc0150810 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f0bb001ab20 tx=0x7f0bb0019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:33.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.554+0000 7f0bb77fe640 1 -- 172.21.15.83:0/2578227352 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb00abb90 con 0x7f0bc006dcb0 2024-04-17T10:47:33.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.554+0000 7f0bb77fe640 1 -- 172.21.15.83:0/2578227352 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb00abd30 con 0x7f0bc006dcb0 2024-04-17T10:47:33.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.554+0000 7f0bc532e640 1 -- 172.21.15.83:0/2578227352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc00af4a0 con 0x7f0bc006dcb0 2024-04-17T10:47:33.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.554+0000 7f0bb77fe640 1 -- 172.21.15.83:0/2578227352 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb00bf690 con 0x7f0bc006dcb0 2024-04-17T10:47:33.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.554+0000 7f0bc532e640 1 -- 172.21.15.83:0/2578227352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc00afa60 con 0x7f0bc006dcb0 2024-04-17T10:47:33.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.554+0000 7f0bc532e640 1 -- 172.21.15.83:0/2578227352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bc0074f90 con 0x7f0bc006dcb0 2024-04-17T10:47:33.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.554+0000 7f0bb77fe640 1 -- 172.21.15.83:0/2578227352 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0bb00bf830 con 0x7f0bc006dcb0 2024-04-17T10:47:33.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.554+0000 7f0bb77fe640 1 --2- 172.21.15.83:0/2578227352 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0ba0066f60 0x7f0ba0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:33.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.554+0000 7f0bb77fe640 1 -- 172.21.15.83:0/2578227352 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0bb011f9b0 con 0x7f0bc006dcb0 2024-04-17T10:47:33.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.554+0000 7f0bb7fff640 1 --2- 172.21.15.83:0/2578227352 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0ba0066f60 0x7f0ba0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:33.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.554+0000 7f0bb7fff640 1 --2- 172.21.15.83:0/2578227352 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0ba0066f60 0x7f0ba0069420 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0bc00aead0 tx=0x7f0ba8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:33.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.558+0000 7f0bb77fe640 1 -- 172.21.15.83:0/2578227352 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0bb0016090 con 0x7f0bc006dcb0 2024-04-17T10:47:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.714+0000 7f0bc532e640 1 -- 172.21.15.83:0/2578227352 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0bc00b2af0 con 0x7f0ba0066f60 2024-04-17T10:47:33.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.726+0000 7f0bb77fe640 1 -- 172.21.15.83:0/2578227352 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247088 (secure 0 0 0) 0x7f0bc00b2af0 con 0x7f0ba0066f60 2024-04-17T10:47:33.725 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:47:33.726 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:47:33.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.730+0000 7f0bc532e640 1 -- 172.21.15.83:0/2578227352 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0ba0066f60 msgr2=0x7f0ba0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:33.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.730+0000 7f0bc532e640 1 --2- 172.21.15.83:0/2578227352 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0ba0066f60 0x7f0ba0069420 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0bc00aead0 tx=0x7f0ba8009040 comp rx=0 tx=0).stop 2024-04-17T10:47:33.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.730+0000 7f0bc532e640 1 -- 172.21.15.83:0/2578227352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bc006dcb0 msgr2=0x7f0bc0150810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:33.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.730+0000 7f0bc532e640 1 --2- 172.21.15.83:0/2578227352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bc006dcb0 0x7f0bc0150810 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f0bb001ab20 tx=0x7f0bb0019210 comp rx=0 tx=0).stop 2024-04-17T10:47:33.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.730+0000 7f0bc532e640 1 -- 172.21.15.83:0/2578227352 shutdown_connections 2024-04-17T10:47:33.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.730+0000 7f0bc532e640 1 --2- 172.21.15.83:0/2578227352 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0ba0066f60 0x7f0ba0069420 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:33.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.730+0000 7f0bc532e640 1 --2- 172.21.15.83:0/2578227352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc015db50 0x7f0bc0157880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:33.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.730+0000 7f0bc532e640 1 --2- 172.21.15.83:0/2578227352 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bc00aedd0 0x7f0bc00b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:33.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.730+0000 7f0bc532e640 1 --2- 172.21.15.83:0/2578227352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bc006dcb0 0x7f0bc0150810 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:33.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.730+0000 7f0bc532e640 1 -- 172.21.15.83:0/2578227352 >> 172.21.15.83:0/2578227352 conn(0x7f0bc006abe0 msgr2=0x7f0bc00b13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:33.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.730+0000 7f0bc532e640 1 -- 172.21.15.83:0/2578227352 shutdown_connections 2024-04-17T10:47:33.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.730+0000 7f0bc532e640 1 -- 172.21.15.83:0/2578227352 wait complete. 2024-04-17T10:47:33.744 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2024-04-17T10:47:32.348528+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":524064,"kb_used_data":10860,"kb_used_omap":12,"kb_used_meta":513075,"kb_avail":754450656,"statfs":{"total":773094113280,"available":772557471744,"internally_reserved":0,"allocated":11120640,"data_stored":5471361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":525389360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.340819"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.031507+0000","last_change":"2024-04-17T10:47:29.031507+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.031507+0000","last_undegraded":"2024-04-17T10:47:29.031507+0000","last_fullsized":"2024-04-17T10:47:29.031507+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.055397+0000","last_change":"2024-04-17T10:47:29.055397+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.055397+0000","last_undegraded":"2024-04-17T10:47:29.055397+0000","last_fullsized":"2024-04-17T10:47:29.055397+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.027558+0000","last_change":"2024-04-17T10:47:29.027558+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.027558+0000","last_undegraded":"2024-04-17T10:47:29.027558+0000","last_fullsized":"2024-04-17T10:47:29.027558+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.044961+0000","last_change":"2024-04-17T10:47:29.044961+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.044961+0000","last_undegraded":"2024-04-17T10:47:29.044961+0000","last_fullsized":"2024-04-17T10:47:29.044961+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.038581+0000","last_change":"2024-04-17T10:47:29.038581+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.038581+0000","last_undegraded":"2024-04-17T10:47:29.038581+0000","last_fullsized":"2024-04-17T10:47:29.038581+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.021494+0000","last_change":"2024-04-17T10:47:29.021494+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.021494+0000","last_undegraded":"2024-04-17T10:47:29.021494+0000","last_fullsized":"2024-04-17T10:47:29.021494+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.025999+0000","last_change":"2024-04-17T10:47:29.025999+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.025999+0000","last_undegraded":"2024-04-17T10:47:29.025999+0000","last_fullsized":"2024-04-17T10:47:29.025999+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.049465+0000","last_change":"2024-04-17T10:47:29.049465+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.049465+0000","last_undegraded":"2024-04-17T10:47:29.049465+0000","last_fullsized":"2024-04-17T10:47:29.049465+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.024602+0000","last_change":"2024-04-17T10:47:29.024602+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.024602+0000","last_undegraded":"2024-04-17T10:47:29.024602+0000","last_fullsized":"2024-04-17T10:47:29.024602+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.030562+0000","last_change":"2024-04-17T10:47:29.030562+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.030562+0000","last_undegraded":"2024-04-17T10:47:29.030562+0000","last_fullsized":"2024-04-17T10:47:29.030562+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.032399+0000","last_change":"2024-04-17T10:47:29.032399+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.032399+0000","last_undegraded":"2024-04-17T10:47:29.032399+0000","last_fullsized":"2024-04-17T10:47:29.032399+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.029188+0000","last_change":"2024-04-17T10:47:29.029188+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.029188+0000","last_undegraded":"2024-04-17T10:47:29.029188+0000","last_fullsized":"2024-04-17T10:47:29.029188+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.046072+0000","last_change":"2024-04-17T10:47:29.046072+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.046072+0000","last_undegraded":"2024-04-17T10:47:29.046072+0000","last_fullsized":"2024-04-17T10:47:29.046072+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.037835+0000","last_change":"2024-04-17T10:47:29.037835+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.037835+0000","last_undegraded":"2024-04-17T10:47:29.037835+0000","last_fullsized":"2024-04-17T10:47:29.037835+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.045596+0000","last_change":"2024-04-17T10:47:29.045596+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.045596+0000","last_undegraded":"2024-04-17T10:47:29.045596+0000","last_fullsized":"2024-04-17T10:47:29.045596+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.028858+0000","last_change":"2024-04-17T10:47:29.028858+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.028858+0000","last_undegraded":"2024-04-17T10:47:29.028858+0000","last_fullsized":"2024-04-17T10:47:29.028858+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.044488+0000","last_change":"2024-04-17T10:47:29.044488+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.044488+0000","last_undegraded":"2024-04-17T10:47:29.044488+0000","last_fullsized":"2024-04-17T10:47:29.044488+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.044801+0000","last_change":"2024-04-17T10:47:29.044801+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.044801+0000","last_undegraded":"2024-04-17T10:47:29.044801+0000","last_fullsized":"2024-04-17T10:47:29.044801+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.034394+0000","last_change":"2024-04-17T10:47:29.034394+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.034394+0000","last_undegraded":"2024-04-17T10:47:29.034394+0000","last_fullsized":"2024-04-17T10:47:29.034394+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.023972+0000","last_change":"2024-04-17T10:47:29.023972+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.023972+0000","last_undegraded":"2024-04-17T10:47:29.023972+0000","last_fullsized":"2024-04-17T10:47:29.023972+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.034527+0000","last_change":"2024-04-17T10:47:29.034527+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.034527+0000","last_undegraded":"2024-04-17T10:47:29.034527+0000","last_fullsized":"2024-04-17T10:47:29.034527+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.045648+0000","last_change":"2024-04-17T10:47:29.045648+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.045648+0000","last_undegraded":"2024-04-17T10:47:29.045648+0000","last_fullsized":"2024-04-17T10:47:29.045648+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.045251+0000","last_change":"2024-04-17T10:47:29.045251+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.045251+0000","last_undegraded":"2024-04-17T10:47:29.045251+0000","last_fullsized":"2024-04-17T10:47:29.045251+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.027697+0000","last_change":"2024-04-17T10:47:29.027697+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.027697+0000","last_undegraded":"2024-04-17T10:47:29.027697+0000","last_fullsized":"2024-04-17T10:47:29.027697+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.037694+0000","last_change":"2024-04-17T10:47:29.037694+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.037694+0000","last_undegraded":"2024-04-17T10:47:29.037694+0000","last_fullsized":"2024-04-17T10:47:29.037694+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.050922+0000","last_change":"2024-04-17T10:47:29.050922+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.050922+0000","last_undegraded":"2024-04-17T10:47:29.050922+0000","last_fullsized":"2024-04-17T10:47:29.050922+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.024737+0000","last_change":"2024-04-17T10:47:29.024737+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.024737+0000","last_undegraded":"2024-04-17T10:47:29.024737+0000","last_fullsized":"2024-04-17T10:47:29.024737+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.043747+0000","last_change":"2024-04-17T10:47:29.043747+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.043747+0000","last_undegraded":"2024-04-17T10:47:29.043747+0000","last_fullsized":"2024-04-17T10:47:29.043747+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.047296+0000","last_change":"2024-04-17T10:47:29.047296+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.047296+0000","last_undegraded":"2024-04-17T10:47:29.047296+0000","last_fullsized":"2024-04-17T10:47:29.047296+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.035574+0000","last_change":"2024-04-17T10:47:29.035574+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.035574+0000","last_undegraded":"2024-04-17T10:47:29.035574+0000","last_fullsized":"2024-04-17T10:47:29.035574+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.045037+0000","last_change":"2024-04-17T10:47:29.045037+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.045037+0000","last_undegraded":"2024-04-17T10:47:29.045037+0000","last_fullsized":"2024-04-17T10:47:29.045037+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.028571+0000","last_change":"2024-04-17T10:47:29.028571+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.028571+0000","last_undegraded":"2024-04-17T10:47:29.028571+0000","last_fullsized":"2024-04-17T10:47:29.028571+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.034477+0000","last_change":"2024-04-17T10:47:29.034477+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.034477+0000","last_undegraded":"2024-04-17T10:47:29.034477+0000","last_fullsized":"2024-04-17T10:47:29.034477+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.058518+0000","last_change":"2024-04-17T10:47:29.058101+0000","last_active":"2024-04-17T10:47:29.058518+0000","last_peered":"2024-04-17T10:47:29.058518+0000","last_clean":"2024-04-17T10:47:29.058518+0000","last_became_active":"2024-04-17T10:47:29.056266+0000","last_became_peered":"2024-04-17T10:47:29.056266+0000","last_unstale":"2024-04-17T10:47:29.058518+0000","last_undegraded":"2024-04-17T10:47:29.058518+0000","last_fullsized":"2024-04-17T10:47:29.058518+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:10:51.407123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.047078+0000","last_change":"2024-04-17T10:47:29.047078+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.047078+0000","last_undegraded":"2024-04-17T10:47:29.047078+0000","last_fullsized":"2024-04-17T10:47:29.047078+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.044854+0000","last_change":"2024-04-17T10:47:29.037268+0000","last_active":"2024-04-17T10:47:29.044854+0000","last_peered":"2024-04-17T10:47:29.044854+0000","last_clean":"2024-04-17T10:47:29.044854+0000","last_became_active":"2024-04-17T10:47:29.035743+0000","last_became_peered":"2024-04-17T10:47:29.035743+0000","last_unstale":"2024-04-17T10:47:29.044854+0000","last_undegraded":"2024-04-17T10:47:29.044854+0000","last_fullsized":"2024-04-17T10:47:29.044854+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:16:45.173638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.030657+0000","last_change":"2024-04-17T10:47:29.030657+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.030657+0000","last_undegraded":"2024-04-17T10:47:29.030657+0000","last_fullsized":"2024-04-17T10:47:29.030657+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.051205+0000","last_change":"2024-04-17T10:47:29.046225+0000","last_active":"2024-04-17T10:47:29.051205+0000","last_peered":"2024-04-17T10:47:29.051205+0000","last_clean":"2024-04-17T10:47:29.051205+0000","last_became_active":"2024-04-17T10:47:29.043738+0000","last_became_peered":"2024-04-17T10:47:29.043738+0000","last_unstale":"2024-04-17T10:47:29.051205+0000","last_undegraded":"2024-04-17T10:47:29.051205+0000","last_fullsized":"2024-04-17T10:47:29.051205+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:43.205753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.023044+0000","last_change":"2024-04-17T10:47:29.023044+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.023044+0000","last_undegraded":"2024-04-17T10:47:29.023044+0000","last_fullsized":"2024-04-17T10:47:29.023044+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.042042+0000","last_change":"2024-04-17T10:47:29.041206+0000","last_active":"2024-04-17T10:47:29.042042+0000","last_peered":"2024-04-17T10:47:29.042042+0000","last_clean":"2024-04-17T10:47:29.042042+0000","last_became_active":"2024-04-17T10:47:29.040924+0000","last_became_peered":"2024-04-17T10:47:29.040924+0000","last_unstale":"2024-04-17T10:47:29.042042+0000","last_undegraded":"2024-04-17T10:47:29.042042+0000","last_fullsized":"2024-04-17T10:47:29.042042+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:35:16.067757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.028743+0000","last_change":"2024-04-17T10:47:29.028743+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.028743+0000","last_undegraded":"2024-04-17T10:47:29.028743+0000","last_fullsized":"2024-04-17T10:47:29.028743+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.054287+0000","last_change":"2024-04-17T10:47:29.053852+0000","last_active":"2024-04-17T10:47:29.054287+0000","last_peered":"2024-04-17T10:47:29.054287+0000","last_clean":"2024-04-17T10:47:29.054287+0000","last_became_active":"2024-04-17T10:47:29.053434+0000","last_became_peered":"2024-04-17T10:47:29.053434+0000","last_unstale":"2024-04-17T10:47:29.054287+0000","last_undegraded":"2024-04-17T10:47:29.054287+0000","last_fullsized":"2024-04-17T10:47:29.054287+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:36:34.892543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.030058+0000","last_change":"2024-04-17T10:47:29.030058+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.030058+0000","last_undegraded":"2024-04-17T10:47:29.030058+0000","last_fullsized":"2024-04-17T10:47:29.030058+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.041154+0000","last_change":"2024-04-17T10:47:29.040335+0000","last_active":"2024-04-17T10:47:29.041154+0000","last_peered":"2024-04-17T10:47:29.041154+0000","last_clean":"2024-04-17T10:47:29.041154+0000","last_became_active":"2024-04-17T10:47:29.039941+0000","last_became_peered":"2024-04-17T10:47:29.039941+0000","last_unstale":"2024-04-17T10:47:29.041154+0000","last_undegraded":"2024-04-17T10:47:29.041154+0000","last_fullsized":"2024-04-17T10:47:29.041154+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:31:48.398246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.044793+0000","last_change":"2024-04-17T10:47:29.044793+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.044793+0000","last_undegraded":"2024-04-17T10:47:29.044793+0000","last_fullsized":"2024-04-17T10:47:29.044793+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.052502+0000","last_change":"2024-04-17T10:47:29.045408+0000","last_active":"2024-04-17T10:47:29.052502+0000","last_peered":"2024-04-17T10:47:29.052502+0000","last_clean":"2024-04-17T10:47:29.052502+0000","last_became_active":"2024-04-17T10:47:29.044801+0000","last_became_peered":"2024-04-17T10:47:29.044801+0000","last_unstale":"2024-04-17T10:47:29.052502+0000","last_undegraded":"2024-04-17T10:47:29.052502+0000","last_fullsized":"2024-04-17T10:47:29.052502+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:52:11.119097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.024952+0000","last_change":"2024-04-17T10:47:29.024952+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.024952+0000","last_undegraded":"2024-04-17T10:47:29.024952+0000","last_fullsized":"2024-04-17T10:47:29.024952+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.046171+0000","last_change":"2024-04-17T10:47:29.038936+0000","last_active":"2024-04-17T10:47:29.046171+0000","last_peered":"2024-04-17T10:47:29.046171+0000","last_clean":"2024-04-17T10:47:29.046171+0000","last_became_active":"2024-04-17T10:47:29.038436+0000","last_became_peered":"2024-04-17T10:47:29.038436+0000","last_unstale":"2024-04-17T10:47:29.046171+0000","last_undegraded":"2024-04-17T10:47:29.046171+0000","last_fullsized":"2024-04-17T10:47:29.046171+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:20:55.139531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.049911+0000","last_change":"2024-04-17T10:47:29.049911+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.049911+0000","last_undegraded":"2024-04-17T10:47:29.049911+0000","last_fullsized":"2024-04-17T10:47:29.049911+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.042156+0000","last_change":"2024-04-17T10:47:29.040517+0000","last_active":"2024-04-17T10:47:29.042156+0000","last_peered":"2024-04-17T10:47:29.042156+0000","last_clean":"2024-04-17T10:47:29.042156+0000","last_became_active":"2024-04-17T10:47:29.039818+0000","last_became_peered":"2024-04-17T10:47:29.039818+0000","last_unstale":"2024-04-17T10:47:29.042156+0000","last_undegraded":"2024-04-17T10:47:29.042156+0000","last_fullsized":"2024-04-17T10:47:29.042156+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:49:17.199444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.023180+0000","last_change":"2024-04-17T10:47:29.023180+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.023180+0000","last_undegraded":"2024-04-17T10:47:29.023180+0000","last_fullsized":"2024-04-17T10:47:29.023180+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.042609+0000","last_change":"2024-04-17T10:47:29.037265+0000","last_active":"2024-04-17T10:47:29.042609+0000","last_peered":"2024-04-17T10:47:29.042609+0000","last_clean":"2024-04-17T10:47:29.042609+0000","last_became_active":"2024-04-17T10:47:29.036735+0000","last_became_peered":"2024-04-17T10:47:29.036735+0000","last_unstale":"2024-04-17T10:47:29.042609+0000","last_undegraded":"2024-04-17T10:47:29.042609+0000","last_fullsized":"2024-04-17T10:47:29.042609+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:49.197211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.042244+0000","last_change":"2024-04-17T10:47:29.042244+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.042244+0000","last_undegraded":"2024-04-17T10:47:29.042244+0000","last_fullsized":"2024-04-17T10:47:29.042244+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.055631+0000","last_change":"2024-04-17T10:47:29.049241+0000","last_active":"2024-04-17T10:47:29.055631+0000","last_peered":"2024-04-17T10:47:29.055631+0000","last_clean":"2024-04-17T10:47:29.055631+0000","last_became_active":"2024-04-17T10:47:29.048527+0000","last_became_peered":"2024-04-17T10:47:29.048527+0000","last_unstale":"2024-04-17T10:47:29.055631+0000","last_undegraded":"2024-04-17T10:47:29.055631+0000","last_fullsized":"2024-04-17T10:47:29.055631+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:26:50.584899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.023708+0000","last_change":"2024-04-17T10:47:29.023708+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.023708+0000","last_undegraded":"2024-04-17T10:47:29.023708+0000","last_fullsized":"2024-04-17T10:47:29.023708+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.050875+0000","last_change":"2024-04-17T10:47:29.047222+0000","last_active":"2024-04-17T10:47:29.050875+0000","last_peered":"2024-04-17T10:47:29.050875+0000","last_clean":"2024-04-17T10:47:29.050875+0000","last_became_active":"2024-04-17T10:47:29.046540+0000","last_became_peered":"2024-04-17T10:47:29.046540+0000","last_unstale":"2024-04-17T10:47:29.050875+0000","last_undegraded":"2024-04-17T10:47:29.050875+0000","last_fullsized":"2024-04-17T10:47:29.050875+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:41:18.010105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.045109+0000","last_change":"2024-04-17T10:47:29.045109+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.045109+0000","last_undegraded":"2024-04-17T10:47:29.045109+0000","last_fullsized":"2024-04-17T10:47:29.045109+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.051787+0000","last_change":"2024-04-17T10:47:29.051191+0000","last_active":"2024-04-17T10:47:29.051787+0000","last_peered":"2024-04-17T10:47:29.051787+0000","last_clean":"2024-04-17T10:47:29.051787+0000","last_became_active":"2024-04-17T10:47:29.050924+0000","last_became_peered":"2024-04-17T10:47:29.050924+0000","last_unstale":"2024-04-17T10:47:29.051787+0000","last_undegraded":"2024-04-17T10:47:29.051787+0000","last_fullsized":"2024-04-17T10:47:29.051787+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:47:39.522424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.027964+0000","last_change":"2024-04-17T10:47:29.027964+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.027964+0000","last_undegraded":"2024-04-17T10:47:29.027964+0000","last_fullsized":"2024-04-17T10:47:29.027964+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.043366+0000","last_change":"2024-04-17T10:47:29.041596+0000","last_active":"2024-04-17T10:47:29.043366+0000","last_peered":"2024-04-17T10:47:29.043366+0000","last_clean":"2024-04-17T10:47:29.043366+0000","last_became_active":"2024-04-17T10:47:29.041198+0000","last_became_peered":"2024-04-17T10:47:29.041198+0000","last_unstale":"2024-04-17T10:47:29.043366+0000","last_undegraded":"2024-04-17T10:47:29.043366+0000","last_fullsized":"2024-04-17T10:47:29.043366+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:33:24.133568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.024813+0000","last_change":"2024-04-17T10:47:29.024813+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.024813+0000","last_undegraded":"2024-04-17T10:47:29.024813+0000","last_fullsized":"2024-04-17T10:47:29.024813+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.050803+0000","last_change":"2024-04-17T10:47:29.047014+0000","last_active":"2024-04-17T10:47:29.050803+0000","last_peered":"2024-04-17T10:47:29.050803+0000","last_clean":"2024-04-17T10:47:29.050803+0000","last_became_active":"2024-04-17T10:47:29.046269+0000","last_became_peered":"2024-04-17T10:47:29.046269+0000","last_unstale":"2024-04-17T10:47:29.050803+0000","last_undegraded":"2024-04-17T10:47:29.050803+0000","last_fullsized":"2024-04-17T10:47:29.050803+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:59.960069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.034425+0000","last_change":"2024-04-17T10:47:29.034425+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.034425+0000","last_undegraded":"2024-04-17T10:47:29.034425+0000","last_fullsized":"2024-04-17T10:47:29.034425+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.040239+0000","last_change":"2024-04-17T10:47:29.037711+0000","last_active":"2024-04-17T10:47:29.040239+0000","last_peered":"2024-04-17T10:47:29.040239+0000","last_clean":"2024-04-17T10:47:29.040239+0000","last_became_active":"2024-04-17T10:47:29.037183+0000","last_became_peered":"2024-04-17T10:47:29.037183+0000","last_unstale":"2024-04-17T10:47:29.040239+0000","last_undegraded":"2024-04-17T10:47:29.040239+0000","last_fullsized":"2024-04-17T10:47:29.040239+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:07:50.722366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.030365+0000","last_change":"2024-04-17T10:47:29.030365+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.030365+0000","last_undegraded":"2024-04-17T10:47:29.030365+0000","last_fullsized":"2024-04-17T10:47:29.030365+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.051165+0000","last_change":"2024-04-17T10:47:29.046483+0000","last_active":"2024-04-17T10:47:29.051165+0000","last_peered":"2024-04-17T10:47:29.051165+0000","last_clean":"2024-04-17T10:47:29.051165+0000","last_became_active":"2024-04-17T10:47:29.042701+0000","last_became_peered":"2024-04-17T10:47:29.042701+0000","last_unstale":"2024-04-17T10:47:29.051165+0000","last_undegraded":"2024-04-17T10:47:29.051165+0000","last_fullsized":"2024-04-17T10:47:29.051165+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:43.205753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.026502+0000","last_change":"2024-04-17T10:47:29.026502+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.026502+0000","last_undegraded":"2024-04-17T10:47:29.026502+0000","last_fullsized":"2024-04-17T10:47:29.026502+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.052476+0000","last_change":"2024-04-17T10:47:29.044936+0000","last_active":"2024-04-17T10:47:29.052476+0000","last_peered":"2024-04-17T10:47:29.052476+0000","last_clean":"2024-04-17T10:47:29.052476+0000","last_became_active":"2024-04-17T10:47:29.044091+0000","last_became_peered":"2024-04-17T10:47:29.044091+0000","last_unstale":"2024-04-17T10:47:29.052476+0000","last_undegraded":"2024-04-17T10:47:29.052476+0000","last_fullsized":"2024-04-17T10:47:29.052476+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:03:10.762120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.022345+0000","last_change":"2024-04-17T10:47:29.022345+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.022345+0000","last_undegraded":"2024-04-17T10:47:29.022345+0000","last_fullsized":"2024-04-17T10:47:29.022345+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.055071+0000","last_change":"2024-04-17T10:47:29.043948+0000","last_active":"2024-04-17T10:47:29.055071+0000","last_peered":"2024-04-17T10:47:29.055071+0000","last_clean":"2024-04-17T10:47:29.055071+0000","last_became_active":"2024-04-17T10:47:29.043517+0000","last_became_peered":"2024-04-17T10:47:29.043517+0000","last_unstale":"2024-04-17T10:47:29.055071+0000","last_undegraded":"2024-04-17T10:47:29.055071+0000","last_fullsized":"2024-04-17T10:47:29.055071+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:52:11.119097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.035024+0000","last_change":"2024-04-17T10:47:29.035024+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.035024+0000","last_undegraded":"2024-04-17T10:47:29.035024+0000","last_fullsized":"2024-04-17T10:47:29.035024+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.055246+0000","last_change":"2024-04-17T10:47:29.049650+0000","last_active":"2024-04-17T10:47:29.055246+0000","last_peered":"2024-04-17T10:47:29.055246+0000","last_clean":"2024-04-17T10:47:29.055246+0000","last_became_active":"2024-04-17T10:47:29.049263+0000","last_became_peered":"2024-04-17T10:47:29.049263+0000","last_unstale":"2024-04-17T10:47:29.055246+0000","last_undegraded":"2024-04-17T10:47:29.055246+0000","last_fullsized":"2024-04-17T10:47:29.055246+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:41:18.010105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.033707+0000","last_change":"2024-04-17T10:47:29.033707+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.033707+0000","last_undegraded":"2024-04-17T10:47:29.033707+0000","last_fullsized":"2024-04-17T10:47:29.033707+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.052049+0000","last_change":"2024-04-17T10:47:29.041858+0000","last_active":"2024-04-17T10:47:29.052049+0000","last_peered":"2024-04-17T10:47:29.052049+0000","last_clean":"2024-04-17T10:47:29.052049+0000","last_became_active":"2024-04-17T10:47:29.041099+0000","last_became_peered":"2024-04-17T10:47:29.041099+0000","last_unstale":"2024-04-17T10:47:29.052049+0000","last_undegraded":"2024-04-17T10:47:29.052049+0000","last_fullsized":"2024-04-17T10:47:29.052049+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:41:18.010105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.035881+0000","last_change":"2024-04-17T10:47:29.035881+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.035881+0000","last_undegraded":"2024-04-17T10:47:29.035881+0000","last_fullsized":"2024-04-17T10:47:29.035881+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.050068+0000","last_change":"2024-04-17T10:47:29.039405+0000","last_active":"2024-04-17T10:47:29.050068+0000","last_peered":"2024-04-17T10:47:29.050068+0000","last_clean":"2024-04-17T10:47:29.050068+0000","last_became_active":"2024-04-17T10:47:29.038596+0000","last_became_peered":"2024-04-17T10:47:29.038596+0000","last_unstale":"2024-04-17T10:47:29.050068+0000","last_undegraded":"2024-04-17T10:47:29.050068+0000","last_fullsized":"2024-04-17T10:47:29.050068+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:43.205753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.039260+0000","last_change":"2024-04-17T10:47:29.039260+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.039260+0000","last_undegraded":"2024-04-17T10:47:29.039260+0000","last_fullsized":"2024-04-17T10:47:29.039260+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.049867+0000","last_change":"2024-04-17T10:47:29.043402+0000","last_active":"2024-04-17T10:47:29.049867+0000","last_peered":"2024-04-17T10:47:29.049867+0000","last_clean":"2024-04-17T10:47:29.049867+0000","last_became_active":"2024-04-17T10:47:29.042994+0000","last_became_peered":"2024-04-17T10:47:29.042994+0000","last_unstale":"2024-04-17T10:47:29.049867+0000","last_undegraded":"2024-04-17T10:47:29.049867+0000","last_fullsized":"2024-04-17T10:47:29.049867+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:03:10.762120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.048879+0000","last_change":"2024-04-17T10:47:29.048879+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.048879+0000","last_undegraded":"2024-04-17T10:47:29.048879+0000","last_fullsized":"2024-04-17T10:47:29.048879+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.051965+0000","last_change":"2024-04-17T10:47:29.041380+0000","last_active":"2024-04-17T10:47:29.051965+0000","last_peered":"2024-04-17T10:47:29.051965+0000","last_clean":"2024-04-17T10:47:29.051965+0000","last_became_active":"2024-04-17T10:47:29.040614+0000","last_became_peered":"2024-04-17T10:47:29.040614+0000","last_unstale":"2024-04-17T10:47:29.051965+0000","last_undegraded":"2024-04-17T10:47:29.051965+0000","last_fullsized":"2024-04-17T10:47:29.051965+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:59.960069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.030002+0000","last_change":"2024-04-17T10:47:29.030002+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.030002+0000","last_undegraded":"2024-04-17T10:47:29.030002+0000","last_fullsized":"2024-04-17T10:47:29.030002+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.058438+0000","last_change":"2024-04-17T10:47:29.056214+0000","last_active":"2024-04-17T10:47:29.058438+0000","last_peered":"2024-04-17T10:47:29.058438+0000","last_clean":"2024-04-17T10:47:29.058438+0000","last_became_active":"2024-04-17T10:47:29.055718+0000","last_became_peered":"2024-04-17T10:47:29.055718+0000","last_unstale":"2024-04-17T10:47:29.058438+0000","last_undegraded":"2024-04-17T10:47:29.058438+0000","last_fullsized":"2024-04-17T10:47:29.058438+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:28.525331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.028102+0000","last_change":"2024-04-17T10:47:29.028102+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.028102+0000","last_undegraded":"2024-04-17T10:47:29.028102+0000","last_fullsized":"2024-04-17T10:47:29.028102+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.055096+0000","last_change":"2024-04-17T10:47:29.046779+0000","last_active":"2024-04-17T10:47:29.055096+0000","last_peered":"2024-04-17T10:47:29.055096+0000","last_clean":"2024-04-17T10:47:29.055096+0000","last_became_active":"2024-04-17T10:47:29.045869+0000","last_became_peered":"2024-04-17T10:47:29.045869+0000","last_unstale":"2024-04-17T10:47:29.055096+0000","last_undegraded":"2024-04-17T10:47:29.055096+0000","last_fullsized":"2024-04-17T10:47:29.055096+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:59.960069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.037035+0000","last_change":"2024-04-17T10:47:29.037035+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.037035+0000","last_undegraded":"2024-04-17T10:47:29.037035+0000","last_fullsized":"2024-04-17T10:47:29.037035+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.054192+0000","last_change":"2024-04-17T10:47:29.050848+0000","last_active":"2024-04-17T10:47:29.054192+0000","last_peered":"2024-04-17T10:47:29.054192+0000","last_clean":"2024-04-17T10:47:29.054192+0000","last_became_active":"2024-04-17T10:47:29.048692+0000","last_became_peered":"2024-04-17T10:47:29.048692+0000","last_unstale":"2024-04-17T10:47:29.054192+0000","last_undegraded":"2024-04-17T10:47:29.054192+0000","last_fullsized":"2024-04-17T10:47:29.054192+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:43.205753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.030935+0000","last_change":"2024-04-17T10:47:29.030935+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.030935+0000","last_undegraded":"2024-04-17T10:47:29.030935+0000","last_fullsized":"2024-04-17T10:47:29.030935+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.055330+0000","last_change":"2024-04-17T10:47:29.050890+0000","last_active":"2024-04-17T10:47:29.055330+0000","last_peered":"2024-04-17T10:47:29.055330+0000","last_clean":"2024-04-17T10:47:29.055330+0000","last_became_active":"2024-04-17T10:47:29.050540+0000","last_became_peered":"2024-04-17T10:47:29.050540+0000","last_unstale":"2024-04-17T10:47:29.055330+0000","last_undegraded":"2024-04-17T10:47:29.055330+0000","last_fullsized":"2024-04-17T10:47:29.055330+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:19:07.385658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.022875+0000","last_change":"2024-04-17T10:47:29.022875+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.022875+0000","last_undegraded":"2024-04-17T10:47:29.022875+0000","last_fullsized":"2024-04-17T10:47:29.022875+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.053684+0000","last_change":"2024-04-17T10:47:29.051971+0000","last_active":"2024-04-17T10:47:29.053684+0000","last_peered":"2024-04-17T10:47:29.053684+0000","last_clean":"2024-04-17T10:47:29.053684+0000","last_became_active":"2024-04-17T10:47:29.051054+0000","last_became_peered":"2024-04-17T10:47:29.051054+0000","last_unstale":"2024-04-17T10:47:29.053684+0000","last_undegraded":"2024-04-17T10:47:29.053684+0000","last_fullsized":"2024-04-17T10:47:29.053684+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:59.960069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.029219+0000","last_change":"2024-04-17T10:47:29.029219+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.029219+0000","last_undegraded":"2024-04-17T10:47:29.029219+0000","last_fullsized":"2024-04-17T10:47:29.029219+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.037159+0000","last_change":"2024-04-17T10:47:29.032908+0000","last_active":"2024-04-17T10:47:29.037159+0000","last_peered":"2024-04-17T10:47:29.037159+0000","last_clean":"2024-04-17T10:47:29.037159+0000","last_became_active":"2024-04-17T10:47:29.032182+0000","last_became_peered":"2024-04-17T10:47:29.032182+0000","last_unstale":"2024-04-17T10:47:29.037159+0000","last_undegraded":"2024-04-17T10:47:29.037159+0000","last_fullsized":"2024-04-17T10:47:29.037159+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:11:47.505394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.038374+0000","last_change":"2024-04-17T10:47:29.038374+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.038374+0000","last_undegraded":"2024-04-17T10:47:29.038374+0000","last_fullsized":"2024-04-17T10:47:29.038374+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.047599+0000","last_change":"2024-04-17T10:47:29.039563+0000","last_active":"2024-04-17T10:47:29.047599+0000","last_peered":"2024-04-17T10:47:29.047599+0000","last_clean":"2024-04-17T10:47:29.047599+0000","last_became_active":"2024-04-17T10:47:29.039021+0000","last_became_peered":"2024-04-17T10:47:29.039021+0000","last_unstale":"2024-04-17T10:47:29.047599+0000","last_undegraded":"2024-04-17T10:47:29.047599+0000","last_fullsized":"2024-04-17T10:47:29.047599+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:52.837030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.033845+0000","last_change":"2024-04-17T10:47:29.033845+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.033845+0000","last_undegraded":"2024-04-17T10:47:29.033845+0000","last_fullsized":"2024-04-17T10:47:29.033845+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.035392+0000","last_change":"2024-04-17T10:47:29.031551+0000","last_active":"2024-04-17T10:47:29.035392+0000","last_peered":"2024-04-17T10:47:29.035392+0000","last_clean":"2024-04-17T10:47:29.035392+0000","last_became_active":"2024-04-17T10:47:29.030845+0000","last_became_peered":"2024-04-17T10:47:29.030845+0000","last_unstale":"2024-04-17T10:47:29.035392+0000","last_undegraded":"2024-04-17T10:47:29.035392+0000","last_fullsized":"2024-04-17T10:47:29.035392+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:49.197211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":336,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.029346+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:47:29.029346+0000","last_peered":"2024-04-17T10:47:29.029346+0000","last_clean":"2024-04-17T10:47:29.029346+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:47:29.029346+0000","last_undegraded":"2024-04-17T10:47:29.029346+0000","last_fullsized":"2024-04-17T10:47:29.029346+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574966,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59340,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312500,"statfs":{"total":96636764160,"available":96576000000,"internally_reserved":0,"allocated":1830912,"data_stored":1124343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574965,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64328,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307512,"statfs":{"total":96636764160,"available":96570892288,"internally_reserved":0,"allocated":1236992,"data_stored":533975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574966,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50440,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321400,"statfs":{"total":96636764160,"available":96585113600,"internally_reserved":0,"allocated":1236992,"data_stored":533975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574965,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68500,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303340,"statfs":{"total":96636764160,"available":96566620160,"internally_reserved":0,"allocated":1249280,"data_stored":540250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574967,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68496,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303344,"statfs":{"total":96636764160,"available":96566624256,"internally_reserved":0,"allocated":1249280,"data_stored":540250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574965,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58756,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313084,"statfs":{"total":96636764160,"available":96576598016,"internally_reserved":0,"allocated":1236992,"data_stored":533975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94852,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94276988,"statfs":{"total":96636764160,"available":96539635712,"internally_reserved":0,"allocated":1236992,"data_stored":533975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574965,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59352,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312488,"statfs":{"total":96636764160,"available":96575987712,"internally_reserved":0,"allocated":1843200,"data_stored":1130618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:47:33.748 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3f is not active+clean 2024-04-17T10:47:33.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.031507+0000', 'last_change': '2024-04-17T10:47:29.031507+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.031507+0000', 'last_undegraded': '2024-04-17T10:47:29.031507+0000', 'last_fullsized': '2024-04-17T10:47:29.031507+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:47:33.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3e is not active+clean 2024-04-17T10:47:33.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.055397+0000', 'last_change': '2024-04-17T10:47:29.055397+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.055397+0000', 'last_undegraded': '2024-04-17T10:47:29.055397+0000', 'last_fullsized': '2024-04-17T10:47:29.055397+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:47:33.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3d is not active+clean 2024-04-17T10:47:33.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.027558+0000', 'last_change': '2024-04-17T10:47:29.027558+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.027558+0000', 'last_undegraded': '2024-04-17T10:47:29.027558+0000', 'last_fullsized': '2024-04-17T10:47:29.027558+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:47:33.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3c is not active+clean 2024-04-17T10:47:33.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.044961+0000', 'last_change': '2024-04-17T10:47:29.044961+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.044961+0000', 'last_undegraded': '2024-04-17T10:47:29.044961+0000', 'last_fullsized': '2024-04-17T10:47:29.044961+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:47:33.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3b is not active+clean 2024-04-17T10:47:33.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.038581+0000', 'last_change': '2024-04-17T10:47:29.038581+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.038581+0000', 'last_undegraded': '2024-04-17T10:47:29.038581+0000', 'last_fullsized': '2024-04-17T10:47:29.038581+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:47:33.750 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3a is not active+clean 2024-04-17T10:47:33.750 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.021494+0000', 'last_change': '2024-04-17T10:47:29.021494+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.021494+0000', 'last_undegraded': '2024-04-17T10:47:29.021494+0000', 'last_fullsized': '2024-04-17T10:47:29.021494+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:47:33.750 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.39 is not active+clean 2024-04-17T10:47:33.750 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.025999+0000', 'last_change': '2024-04-17T10:47:29.025999+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.025999+0000', 'last_undegraded': '2024-04-17T10:47:29.025999+0000', 'last_fullsized': '2024-04-17T10:47:29.025999+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:47:33.750 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.38 is not active+clean 2024-04-17T10:47:33.750 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.049465+0000', 'last_change': '2024-04-17T10:47:29.049465+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.049465+0000', 'last_undegraded': '2024-04-17T10:47:29.049465+0000', 'last_fullsized': '2024-04-17T10:47:29.049465+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:47:33.750 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.37 is not active+clean 2024-04-17T10:47:33.750 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.024602+0000', 'last_change': '2024-04-17T10:47:29.024602+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.024602+0000', 'last_undegraded': '2024-04-17T10:47:29.024602+0000', 'last_fullsized': '2024-04-17T10:47:29.024602+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:47:33.750 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.36 is not active+clean 2024-04-17T10:47:33.750 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.030562+0000', 'last_change': '2024-04-17T10:47:29.030562+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.030562+0000', 'last_undegraded': '2024-04-17T10:47:29.030562+0000', 'last_fullsized': '2024-04-17T10:47:29.030562+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:47:33.751 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.35 is not active+clean 2024-04-17T10:47:33.751 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.032399+0000', 'last_change': '2024-04-17T10:47:29.032399+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.032399+0000', 'last_undegraded': '2024-04-17T10:47:29.032399+0000', 'last_fullsized': '2024-04-17T10:47:29.032399+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:47:33.751 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.34 is not active+clean 2024-04-17T10:47:33.751 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.029188+0000', 'last_change': '2024-04-17T10:47:29.029188+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.029188+0000', 'last_undegraded': '2024-04-17T10:47:29.029188+0000', 'last_fullsized': '2024-04-17T10:47:29.029188+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:47:33.751 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.33 is not active+clean 2024-04-17T10:47:33.751 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.046072+0000', 'last_change': '2024-04-17T10:47:29.046072+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.046072+0000', 'last_undegraded': '2024-04-17T10:47:29.046072+0000', 'last_fullsized': '2024-04-17T10:47:29.046072+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:47:33.751 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.32 is not active+clean 2024-04-17T10:47:33.751 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.037835+0000', 'last_change': '2024-04-17T10:47:29.037835+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.037835+0000', 'last_undegraded': '2024-04-17T10:47:29.037835+0000', 'last_fullsized': '2024-04-17T10:47:29.037835+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:47:33.751 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.31 is not active+clean 2024-04-17T10:47:33.751 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.045596+0000', 'last_change': '2024-04-17T10:47:29.045596+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.045596+0000', 'last_undegraded': '2024-04-17T10:47:29.045596+0000', 'last_fullsized': '2024-04-17T10:47:29.045596+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:47:33.752 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.30 is not active+clean 2024-04-17T10:47:33.752 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.028858+0000', 'last_change': '2024-04-17T10:47:29.028858+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.028858+0000', 'last_undegraded': '2024-04-17T10:47:29.028858+0000', 'last_fullsized': '2024-04-17T10:47:29.028858+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:47:33.752 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2f is not active+clean 2024-04-17T10:47:33.752 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.044488+0000', 'last_change': '2024-04-17T10:47:29.044488+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.044488+0000', 'last_undegraded': '2024-04-17T10:47:29.044488+0000', 'last_fullsized': '2024-04-17T10:47:29.044488+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:47:33.752 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2e is not active+clean 2024-04-17T10:47:33.752 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.044801+0000', 'last_change': '2024-04-17T10:47:29.044801+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.044801+0000', 'last_undegraded': '2024-04-17T10:47:29.044801+0000', 'last_fullsized': '2024-04-17T10:47:29.044801+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:47:33.752 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2d is not active+clean 2024-04-17T10:47:33.752 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.034394+0000', 'last_change': '2024-04-17T10:47:29.034394+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.034394+0000', 'last_undegraded': '2024-04-17T10:47:29.034394+0000', 'last_fullsized': '2024-04-17T10:47:29.034394+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:47:33.752 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2c is not active+clean 2024-04-17T10:47:33.752 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.023972+0000', 'last_change': '2024-04-17T10:47:29.023972+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.023972+0000', 'last_undegraded': '2024-04-17T10:47:29.023972+0000', 'last_fullsized': '2024-04-17T10:47:29.023972+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:47:33.753 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2b is not active+clean 2024-04-17T10:47:33.753 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.034527+0000', 'last_change': '2024-04-17T10:47:29.034527+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.034527+0000', 'last_undegraded': '2024-04-17T10:47:29.034527+0000', 'last_fullsized': '2024-04-17T10:47:29.034527+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:47:33.753 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2a is not active+clean 2024-04-17T10:47:33.753 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.045648+0000', 'last_change': '2024-04-17T10:47:29.045648+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.045648+0000', 'last_undegraded': '2024-04-17T10:47:29.045648+0000', 'last_fullsized': '2024-04-17T10:47:29.045648+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:47:33.753 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.29 is not active+clean 2024-04-17T10:47:33.753 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.045251+0000', 'last_change': '2024-04-17T10:47:29.045251+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.045251+0000', 'last_undegraded': '2024-04-17T10:47:29.045251+0000', 'last_fullsized': '2024-04-17T10:47:29.045251+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:47:33.753 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.28 is not active+clean 2024-04-17T10:47:33.753 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.027697+0000', 'last_change': '2024-04-17T10:47:29.027697+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.027697+0000', 'last_undegraded': '2024-04-17T10:47:29.027697+0000', 'last_fullsized': '2024-04-17T10:47:29.027697+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:47:33.753 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.27 is not active+clean 2024-04-17T10:47:33.753 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.037694+0000', 'last_change': '2024-04-17T10:47:29.037694+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.037694+0000', 'last_undegraded': '2024-04-17T10:47:29.037694+0000', 'last_fullsized': '2024-04-17T10:47:29.037694+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:47:33.754 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.26 is not active+clean 2024-04-17T10:47:33.754 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.050922+0000', 'last_change': '2024-04-17T10:47:29.050922+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.050922+0000', 'last_undegraded': '2024-04-17T10:47:29.050922+0000', 'last_fullsized': '2024-04-17T10:47:29.050922+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:47:33.754 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.25 is not active+clean 2024-04-17T10:47:33.754 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.024737+0000', 'last_change': '2024-04-17T10:47:29.024737+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.024737+0000', 'last_undegraded': '2024-04-17T10:47:29.024737+0000', 'last_fullsized': '2024-04-17T10:47:29.024737+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:47:33.754 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.24 is not active+clean 2024-04-17T10:47:33.754 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.043747+0000', 'last_change': '2024-04-17T10:47:29.043747+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.043747+0000', 'last_undegraded': '2024-04-17T10:47:29.043747+0000', 'last_fullsized': '2024-04-17T10:47:29.043747+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:47:33.754 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.23 is not active+clean 2024-04-17T10:47:33.754 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.047296+0000', 'last_change': '2024-04-17T10:47:29.047296+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.047296+0000', 'last_undegraded': '2024-04-17T10:47:29.047296+0000', 'last_fullsized': '2024-04-17T10:47:29.047296+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:47:33.754 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.22 is not active+clean 2024-04-17T10:47:33.754 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.035574+0000', 'last_change': '2024-04-17T10:47:29.035574+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.035574+0000', 'last_undegraded': '2024-04-17T10:47:29.035574+0000', 'last_fullsized': '2024-04-17T10:47:29.035574+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:47:33.754 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.21 is not active+clean 2024-04-17T10:47:33.755 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.045037+0000', 'last_change': '2024-04-17T10:47:29.045037+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.045037+0000', 'last_undegraded': '2024-04-17T10:47:29.045037+0000', 'last_fullsized': '2024-04-17T10:47:29.045037+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:47:33.755 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.20 is not active+clean 2024-04-17T10:47:33.755 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.028571+0000', 'last_change': '2024-04-17T10:47:29.028571+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.028571+0000', 'last_undegraded': '2024-04-17T10:47:29.028571+0000', 'last_fullsized': '2024-04-17T10:47:29.028571+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:47:33.755 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1e is not active+clean 2024-04-17T10:47:33.755 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.034477+0000', 'last_change': '2024-04-17T10:47:29.034477+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.034477+0000', 'last_undegraded': '2024-04-17T10:47:29.034477+0000', 'last_fullsized': '2024-04-17T10:47:29.034477+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:47:33.755 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1f is not active+clean 2024-04-17T10:47:33.756 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.047078+0000', 'last_change': '2024-04-17T10:47:29.047078+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.047078+0000', 'last_undegraded': '2024-04-17T10:47:29.047078+0000', 'last_fullsized': '2024-04-17T10:47:29.047078+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:47:33.756 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1c is not active+clean 2024-04-17T10:47:33.756 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.030657+0000', 'last_change': '2024-04-17T10:47:29.030657+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.030657+0000', 'last_undegraded': '2024-04-17T10:47:29.030657+0000', 'last_fullsized': '2024-04-17T10:47:29.030657+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:47:33.756 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1d is not active+clean 2024-04-17T10:47:33.756 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.023044+0000', 'last_change': '2024-04-17T10:47:29.023044+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.023044+0000', 'last_undegraded': '2024-04-17T10:47:29.023044+0000', 'last_fullsized': '2024-04-17T10:47:29.023044+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:47:33.756 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1a is not active+clean 2024-04-17T10:47:33.756 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.028743+0000', 'last_change': '2024-04-17T10:47:29.028743+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.028743+0000', 'last_undegraded': '2024-04-17T10:47:29.028743+0000', 'last_fullsized': '2024-04-17T10:47:29.028743+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:47:33.756 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1b is not active+clean 2024-04-17T10:47:33.756 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.030058+0000', 'last_change': '2024-04-17T10:47:29.030058+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.030058+0000', 'last_undegraded': '2024-04-17T10:47:29.030058+0000', 'last_fullsized': '2024-04-17T10:47:29.030058+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:47:33.757 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.18 is not active+clean 2024-04-17T10:47:33.757 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.044793+0000', 'last_change': '2024-04-17T10:47:29.044793+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.044793+0000', 'last_undegraded': '2024-04-17T10:47:29.044793+0000', 'last_fullsized': '2024-04-17T10:47:29.044793+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:47:33.757 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.19 is not active+clean 2024-04-17T10:47:33.757 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.024952+0000', 'last_change': '2024-04-17T10:47:29.024952+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.024952+0000', 'last_undegraded': '2024-04-17T10:47:29.024952+0000', 'last_fullsized': '2024-04-17T10:47:29.024952+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:47:33.757 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.16 is not active+clean 2024-04-17T10:47:33.757 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.049911+0000', 'last_change': '2024-04-17T10:47:29.049911+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.049911+0000', 'last_undegraded': '2024-04-17T10:47:29.049911+0000', 'last_fullsized': '2024-04-17T10:47:29.049911+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:47:33.757 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.17 is not active+clean 2024-04-17T10:47:33.757 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.023180+0000', 'last_change': '2024-04-17T10:47:29.023180+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.023180+0000', 'last_undegraded': '2024-04-17T10:47:29.023180+0000', 'last_fullsized': '2024-04-17T10:47:29.023180+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:47:33.757 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.14 is not active+clean 2024-04-17T10:47:33.757 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.042244+0000', 'last_change': '2024-04-17T10:47:29.042244+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.042244+0000', 'last_undegraded': '2024-04-17T10:47:29.042244+0000', 'last_fullsized': '2024-04-17T10:47:29.042244+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:47:33.757 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.15 is not active+clean 2024-04-17T10:47:33.758 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.023708+0000', 'last_change': '2024-04-17T10:47:29.023708+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.023708+0000', 'last_undegraded': '2024-04-17T10:47:29.023708+0000', 'last_fullsized': '2024-04-17T10:47:29.023708+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:47:33.758 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.12 is not active+clean 2024-04-17T10:47:33.758 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.045109+0000', 'last_change': '2024-04-17T10:47:29.045109+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.045109+0000', 'last_undegraded': '2024-04-17T10:47:29.045109+0000', 'last_fullsized': '2024-04-17T10:47:29.045109+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:47:33.758 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.13 is not active+clean 2024-04-17T10:47:33.758 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.027964+0000', 'last_change': '2024-04-17T10:47:29.027964+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.027964+0000', 'last_undegraded': '2024-04-17T10:47:29.027964+0000', 'last_fullsized': '2024-04-17T10:47:29.027964+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:47:33.758 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.10 is not active+clean 2024-04-17T10:47:33.758 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.024813+0000', 'last_change': '2024-04-17T10:47:29.024813+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.024813+0000', 'last_undegraded': '2024-04-17T10:47:29.024813+0000', 'last_fullsized': '2024-04-17T10:47:29.024813+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:47:33.758 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.11 is not active+clean 2024-04-17T10:47:33.758 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.034425+0000', 'last_change': '2024-04-17T10:47:29.034425+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.034425+0000', 'last_undegraded': '2024-04-17T10:47:29.034425+0000', 'last_fullsized': '2024-04-17T10:47:29.034425+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:47:33.758 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.f is not active+clean 2024-04-17T10:47:33.759 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.030365+0000', 'last_change': '2024-04-17T10:47:29.030365+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.030365+0000', 'last_undegraded': '2024-04-17T10:47:29.030365+0000', 'last_fullsized': '2024-04-17T10:47:29.030365+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:47:33.759 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.c is not active+clean 2024-04-17T10:47:33.759 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.026502+0000', 'last_change': '2024-04-17T10:47:29.026502+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.026502+0000', 'last_undegraded': '2024-04-17T10:47:29.026502+0000', 'last_fullsized': '2024-04-17T10:47:29.026502+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:47:33.759 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.d is not active+clean 2024-04-17T10:47:33.759 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.022345+0000', 'last_change': '2024-04-17T10:47:29.022345+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.022345+0000', 'last_undegraded': '2024-04-17T10:47:29.022345+0000', 'last_fullsized': '2024-04-17T10:47:29.022345+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:47:33.759 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.a is not active+clean 2024-04-17T10:47:33.759 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.035024+0000', 'last_change': '2024-04-17T10:47:29.035024+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.035024+0000', 'last_undegraded': '2024-04-17T10:47:29.035024+0000', 'last_fullsized': '2024-04-17T10:47:29.035024+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:47:33.759 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.b is not active+clean 2024-04-17T10:47:33.759 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.033707+0000', 'last_change': '2024-04-17T10:47:29.033707+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.033707+0000', 'last_undegraded': '2024-04-17T10:47:29.033707+0000', 'last_fullsized': '2024-04-17T10:47:29.033707+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:47:33.759 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.8 is not active+clean 2024-04-17T10:47:33.760 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.035881+0000', 'last_change': '2024-04-17T10:47:29.035881+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.035881+0000', 'last_undegraded': '2024-04-17T10:47:29.035881+0000', 'last_fullsized': '2024-04-17T10:47:29.035881+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:47:33.760 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.9 is not active+clean 2024-04-17T10:47:33.760 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.039260+0000', 'last_change': '2024-04-17T10:47:29.039260+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.039260+0000', 'last_undegraded': '2024-04-17T10:47:29.039260+0000', 'last_fullsized': '2024-04-17T10:47:29.039260+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:47:33.760 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.6 is not active+clean 2024-04-17T10:47:33.760 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.048879+0000', 'last_change': '2024-04-17T10:47:29.048879+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.048879+0000', 'last_undegraded': '2024-04-17T10:47:29.048879+0000', 'last_fullsized': '2024-04-17T10:47:29.048879+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:47:33.760 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.7 is not active+clean 2024-04-17T10:47:33.760 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.030002+0000', 'last_change': '2024-04-17T10:47:29.030002+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.030002+0000', 'last_undegraded': '2024-04-17T10:47:29.030002+0000', 'last_fullsized': '2024-04-17T10:47:29.030002+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:47:33.760 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.4 is not active+clean 2024-04-17T10:47:33.760 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.028102+0000', 'last_change': '2024-04-17T10:47:29.028102+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.028102+0000', 'last_undegraded': '2024-04-17T10:47:29.028102+0000', 'last_fullsized': '2024-04-17T10:47:29.028102+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:47:33.761 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.5 is not active+clean 2024-04-17T10:47:33.761 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.037035+0000', 'last_change': '2024-04-17T10:47:29.037035+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.037035+0000', 'last_undegraded': '2024-04-17T10:47:29.037035+0000', 'last_fullsized': '2024-04-17T10:47:29.037035+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:47:33.761 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2 is not active+clean 2024-04-17T10:47:33.761 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.030935+0000', 'last_change': '2024-04-17T10:47:29.030935+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.030935+0000', 'last_undegraded': '2024-04-17T10:47:29.030935+0000', 'last_fullsized': '2024-04-17T10:47:29.030935+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:47:33.761 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3 is not active+clean 2024-04-17T10:47:33.762 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.022875+0000', 'last_change': '2024-04-17T10:47:29.022875+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.022875+0000', 'last_undegraded': '2024-04-17T10:47:29.022875+0000', 'last_fullsized': '2024-04-17T10:47:29.022875+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:47:33.762 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.0 is not active+clean 2024-04-17T10:47:33.762 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.029219+0000', 'last_change': '2024-04-17T10:47:29.029219+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.029219+0000', 'last_undegraded': '2024-04-17T10:47:29.029219+0000', 'last_fullsized': '2024-04-17T10:47:29.029219+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:47:33.762 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1 is not active+clean 2024-04-17T10:47:33.762 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.038374+0000', 'last_change': '2024-04-17T10:47:29.038374+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.038374+0000', 'last_undegraded': '2024-04-17T10:47:29.038374+0000', 'last_fullsized': '2024-04-17T10:47:29.038374+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:47:33.762 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.e is not active+clean 2024-04-17T10:47:33.762 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:47:29.033845+0000', 'last_change': '2024-04-17T10:47:29.033845+0000', 'last_active': '2024-04-17T10:47:29.006354+0000', 'last_peered': '2024-04-17T10:47:29.006354+0000', 'last_clean': '2024-04-17T10:47:29.006354+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:47:29.033845+0000', 'last_undegraded': '2024-04-17T10:47:29.033845+0000', 'last_fullsized': '2024-04-17T10:47:29.033845+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'last_clean_scrub_stamp': '2024-04-17T10:47:29.006354+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:47:33.763 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-17T10:47:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.910+0000 7fca2483f640 1 Processor -- start 2024-04-17T10:47:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.910+0000 7fca2483f640 1 -- start start 2024-04-17T10:47:33.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.910+0000 7fca2483f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca2015ba30 0x7fca20072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:33.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca2483f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca2006e320 con 0x7fca2015c960 2024-04-17T10:47:33.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca2483f640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca2006e040 con 0x7fca200cfbe0 2024-04-17T10:47:33.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca2483f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca20071ea0 con 0x7fca2015ba30 2024-04-17T10:47:33.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca1dd9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fca2015c960 0x7fca200cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41610/0 (socket says 172.21.15.83:41610) 2024-04-17T10:47:33.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca1dd9b640 1 -- 172.21.15.83:0/619487194 learned_addr learned my addr 172.21.15.83:0/619487194 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:33.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca1e59c640 1 --2- 172.21.15.83:0/619487194 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca2015ba30 0x7fca20072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:33.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca1d59a640 1 -- 172.21.15.83:0/619487194 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3962463638 0 0) 0x7fca2006e320 con 0x7fca2015c960 2024-04-17T10:47:33.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca1d59a640 1 -- 172.21.15.83:0/619487194 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca08003530 con 0x7fca2015c960 2024-04-17T10:47:33.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca1d59a640 1 -- 172.21.15.83:0/619487194 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 441847985 0 0) 0x7fca08003530 con 0x7fca2015c960 2024-04-17T10:47:33.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca1d59a640 1 -- 172.21.15.83:0/619487194 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca2015ba30 msgr2=0x7fca20072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:33.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca1d59a640 1 --2- 172.21.15.83:0/619487194 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca2015ba30 0x7fca20072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:33.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca1d59a640 1 -- 172.21.15.83:0/619487194 >> v1:172.21.15.167:6789/0 conn(0x7fca200cfbe0 legacy=0x7fca2006c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:33.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca1d59a640 1 -- 172.21.15.83:0/619487194 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca20071850 con 0x7fca2015c960 2024-04-17T10:47:33.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca1d59a640 1 -- 172.21.15.83:0/619487194 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fca140016a0 con 0x7fca2015c960 2024-04-17T10:47:33.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca1d59a640 1 -- 172.21.15.83:0/619487194 >> v1:172.21.15.83:6789/0 conn(0x7fca2015c960 legacy=0x7fca200cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:33.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca1d59a640 1 --2- 172.21.15.83:0/619487194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca080039f0 0x7fca08003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:33.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca1d59a640 1 --2- 172.21.15.83:0/619487194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca08004810 0x7fca08004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:33.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca1d59a640 1 --2- 172.21.15.83:0/619487194 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca2015ba30 0x7fca08005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:33.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca1d59a640 1 -- 172.21.15.83:0/619487194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca20071ea0 con 0x7fca08004810 2024-04-17T10:47:33.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca1d59a640 1 -- 172.21.15.83:0/619487194 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0800bb00 con 0x7fca080039f0 2024-04-17T10:47:33.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca1d59a640 1 -- 172.21.15.83:0/619487194 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca0800bd00 con 0x7fca2015ba30 2024-04-17T10:47:33.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca1dd9b640 1 --2- 172.21.15.83:0/619487194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca08004810 0x7fca08004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:33.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca1e59c640 1 --2- 172.21.15.83:0/619487194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca080039f0 0x7fca08003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:33.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca1ed9d640 1 --2- 172.21.15.83:0/619487194 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca2015ba30 0x7fca08005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:33.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca1dd9b640 1 -- 172.21.15.83:0/619487194 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca2015ba30 msgr2=0x7fca08005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:33.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca1dd9b640 1 --2- 172.21.15.83:0/619487194 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca2015ba30 0x7fca08005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:33.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca1dd9b640 1 -- 172.21.15.83:0/619487194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca080039f0 msgr2=0x7fca08003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:33.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca1dd9b640 1 --2- 172.21.15.83:0/619487194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca080039f0 0x7fca08003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:33.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca1dd9b640 1 -- 172.21.15.83:0/619487194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca08003530 con 0x7fca08004810 2024-04-17T10:47:33.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca1e59c640 1 --2- 172.21.15.83:0/619487194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca080039f0 0x7fca08003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:33.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca1ed9d640 1 --2- 172.21.15.83:0/619487194 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca2015ba30 0x7fca08005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:33.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca1dd9b640 1 --2- 172.21.15.83:0/619487194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca08004810 0x7fca08004c90 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fca1400c290 tx=0x7fca2006d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6cf807966bc07eb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:33.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca1d59a640 1 -- 172.21.15.83:0/619487194 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca14007d60 con 0x7fca08004810 2024-04-17T10:47:33.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca1d59a640 1 -- 172.21.15.83:0/619487194 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca140a9020 con 0x7fca08004810 2024-04-17T10:47:33.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.914+0000 7fca1d59a640 1 -- 172.21.15.83:0/619487194 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca14005b00 con 0x7fca08004810 2024-04-17T10:47:33.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.918+0000 7fca2483f640 1 -- 172.21.15.83:0/619487194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca08004810 msgr2=0x7fca08004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:33.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.918+0000 7fca2483f640 1 --2- 172.21.15.83:0/619487194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca08004810 0x7fca08004c90 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fca1400c290 tx=0x7fca2006d3b0 comp rx=0 tx=0).stop 2024-04-17T10:47:33.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.918+0000 7fca1ed9d640 1 -- 172.21.15.83:0/619487194 reap_dead start 2024-04-17T10:47:33.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.918+0000 7fca2483f640 1 -- 172.21.15.83:0/619487194 shutdown_connections 2024-04-17T10:47:33.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.918+0000 7fca2483f640 1 -- 172.21.15.83:0/619487194 >> 172.21.15.83:0/619487194 conn(0x7fca2006abe0 msgr2=0x7fca2006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:33.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.918+0000 7fca2483f640 1 -- 172.21.15.83:0/619487194 shutdown_connections 2024-04-17T10:47:33.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.918+0000 7fca2483f640 1 -- 172.21.15.83:0/619487194 wait complete. 2024-04-17T10:47:33.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.918+0000 7fca2483f640 1 Processor -- start 2024-04-17T10:47:33.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.918+0000 7fca2483f640 1 -- start start 2024-04-17T10:47:33.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.918+0000 7fca2483f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca08004810 0x7fca2014ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:33.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.918+0000 7fca2483f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca200b40c0 0x7fca20158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:33.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.918+0000 7fca2483f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca200b4520 0x7fca2014c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:33.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.918+0000 7fca2483f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2006de20 con 0x7fca200b4520 2024-04-17T10:47:33.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.918+0000 7fca2483f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2006dca0 con 0x7fca08004810 2024-04-17T10:47:33.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.918+0000 7fca2483f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca20167520 con 0x7fca200b40c0 2024-04-17T10:47:33.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.918+0000 7fca1ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca200b4520 0x7fca2014c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:33.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.918+0000 7fca1ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca200b4520 0x7fca2014c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60488/0 (socket says 172.21.15.83:60488) 2024-04-17T10:47:33.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.918+0000 7fca1ed9d640 1 -- 172.21.15.83:0/2806452784 learned_addr learned my addr 172.21.15.83:0/2806452784 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:33.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.918+0000 7fca1dd9b640 1 --2- 172.21.15.83:0/2806452784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca200b40c0 0x7fca20158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:33.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.918+0000 7fca1e59c640 1 --2- 172.21.15.83:0/2806452784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca08004810 0x7fca2014ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:33.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.918+0000 7fca1ed9d640 1 -- 172.21.15.83:0/2806452784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca200b40c0 msgr2=0x7fca20158a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:33.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.918+0000 7fca1ed9d640 1 --2- 172.21.15.83:0/2806452784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca200b40c0 0x7fca20158a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:33.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.918+0000 7fca1ed9d640 1 -- 172.21.15.83:0/2806452784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca08004810 msgr2=0x7fca2014ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:33.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.918+0000 7fca1ed9d640 1 --2- 172.21.15.83:0/2806452784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca08004810 0x7fca2014ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:33.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.918+0000 7fca1ed9d640 1 -- 172.21.15.83:0/2806452784 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2014db10 con 0x7fca200b4520 2024-04-17T10:47:33.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.918+0000 7fca1ed9d640 1 --2- 172.21.15.83:0/2806452784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca200b4520 0x7fca2014c280 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fca200676a0 tx=0x7fca1800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:33.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.918+0000 7fca0effd640 1 -- 172.21.15.83:0/2806452784 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca18006d60 con 0x7fca200b4520 2024-04-17T10:47:33.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.918+0000 7fca2483f640 1 -- 172.21.15.83:0/2806452784 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca20159dd0 con 0x7fca200b4520 2024-04-17T10:47:33.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.918+0000 7fca0effd640 1 -- 172.21.15.83:0/2806452784 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca18003980 con 0x7fca200b4520 2024-04-17T10:47:33.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.918+0000 7fca0effd640 1 -- 172.21.15.83:0/2806452784 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca18003db0 con 0x7fca200b4520 2024-04-17T10:47:33.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.918+0000 7fca2483f640 1 -- 172.21.15.83:0/2806452784 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca20157f70 con 0x7fca200b4520 2024-04-17T10:47:33.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.922+0000 7fca2483f640 1 -- 172.21.15.83:0/2806452784 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca20074f90 con 0x7fca200b4520 2024-04-17T10:47:33.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.922+0000 7fca0effd640 1 -- 172.21.15.83:0/2806452784 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fca1801b020 con 0x7fca200b4520 2024-04-17T10:47:33.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.922+0000 7fca0effd640 1 --2- 172.21.15.83:0/2806452784 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fca00066f60 0x7fca00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:33.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.922+0000 7fca0effd640 1 -- 172.21.15.83:0/2806452784 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca18084c10 con 0x7fca200b4520 2024-04-17T10:47:33.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.922+0000 7fca1e59c640 1 --2- 172.21.15.83:0/2806452784 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fca00066f60 0x7fca00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:33.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.922+0000 7fca1e59c640 1 --2- 172.21.15.83:0/2806452784 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fca00066f60 0x7fca00069420 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fca10009ac0 tx=0x7fca10008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:33.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:33.926+0000 7fca0effd640 1 -- 172.21.15.83:0/2806452784 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fca2014db10 con 0x7fca200b4520 2024-04-17T10:47:34.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.142+0000 7fca2483f640 1 -- 172.21.15.83:0/2806452784 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fca200b0560 con 0x7fca200b4520 2024-04-17T10:47:34.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.142+0000 7fca0effd640 1 -- 172.21.15.83:0/2806452784 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1455 (secure 0 0 0) 0x7fca18088080 con 0x7fca200b4520 2024-04-17T10:47:34.140 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:47:34.140 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":586,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":103,"num_osds":8,"num_up_osds":8,"osd_up_since":1713350279,"num_in_osds":8,"osd_in_since":1713350270,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":536641536,"bytes_avail":772557471744,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":62,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8049}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-04-17T10:47:18.345526+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-17T10:47:34.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.146+0000 7fca2483f640 1 -- 172.21.15.83:0/2806452784 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fca00066f60 msgr2=0x7fca00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:34.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.146+0000 7fca2483f640 1 --2- 172.21.15.83:0/2806452784 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fca00066f60 0x7fca00069420 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fca10009ac0 tx=0x7fca10008040 comp rx=0 tx=0).stop 2024-04-17T10:47:34.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.146+0000 7fca2483f640 1 -- 172.21.15.83:0/2806452784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca200b4520 msgr2=0x7fca2014c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:34.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.146+0000 7fca2483f640 1 --2- 172.21.15.83:0/2806452784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca200b4520 0x7fca2014c280 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fca200676a0 tx=0x7fca1800dc00 comp rx=0 tx=0).stop 2024-04-17T10:47:34.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.146+0000 7fca2483f640 1 -- 172.21.15.83:0/2806452784 shutdown_connections 2024-04-17T10:47:34.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.146+0000 7fca2483f640 1 --2- 172.21.15.83:0/2806452784 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fca00066f60 0x7fca00069420 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:34.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.146+0000 7fca2483f640 1 --2- 172.21.15.83:0/2806452784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca200b4520 0x7fca2014c280 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:34.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.146+0000 7fca2483f640 1 --2- 172.21.15.83:0/2806452784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca200b40c0 0x7fca20158a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:34.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.146+0000 7fca2483f640 1 --2- 172.21.15.83:0/2806452784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca08004810 0x7fca2014ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:34.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.146+0000 7fca2483f640 1 -- 172.21.15.83:0/2806452784 >> 172.21.15.83:0/2806452784 conn(0x7fca2006abe0 msgr2=0x7fca2006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:34.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.146+0000 7fca2483f640 1 -- 172.21.15.83:0/2806452784 shutdown_connections 2024-04-17T10:47:34.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.146+0000 7fca2483f640 1 -- 172.21.15.83:0/2806452784 wait complete. 2024-04-17T10:47:34.159 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-17T10:47:34.160 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:47:34.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8e38e1640 1 Processor -- start 2024-04-17T10:47:34.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8e38e1640 1 -- start start 2024-04-17T10:47:34.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8e38e1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8dc15ba30 0x7ff8dc072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:34.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8e38e1640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8dc06e320 con 0x7ff8dc0cfbe0 2024-04-17T10:47:34.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8e38e1640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8dc06e040 con 0x7ff8dc15c960 2024-04-17T10:47:34.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8e38e1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8dc071ea0 con 0x7ff8dc15ba30 2024-04-17T10:47:34.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8e1e7e640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff8dc0cfbe0 0x7ff8dc06c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41626/0 (socket says 172.21.15.83:41626) 2024-04-17T10:47:34.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8e1e7e640 1 -- 172.21.15.83:0/250160157 learned_addr learned my addr 172.21.15.83:0/250160157 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:34.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8e167d640 1 --2- 172.21.15.83:0/250160157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8dc15ba30 0x7ff8dc072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:34.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8dbfff640 1 -- 172.21.15.83:0/250160157 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4279266711 0 0) 0x7ff8dc06e320 con 0x7ff8dc0cfbe0 2024-04-17T10:47:34.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8dbfff640 1 -- 172.21.15.83:0/250160157 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8c4003530 con 0x7ff8dc0cfbe0 2024-04-17T10:47:34.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8dbfff640 1 -- 172.21.15.83:0/250160157 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2136036038 0 0) 0x7ff8c4003530 con 0x7ff8dc0cfbe0 2024-04-17T10:47:34.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8dbfff640 1 -- 172.21.15.83:0/250160157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8dc15ba30 msgr2=0x7ff8dc072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:34.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8dbfff640 1 --2- 172.21.15.83:0/250160157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8dc15ba30 0x7ff8dc072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:34.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8dbfff640 1 -- 172.21.15.83:0/250160157 >> v1:172.21.15.167:6789/0 conn(0x7ff8dc15c960 legacy=0x7ff8dc0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:34.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8dbfff640 1 -- 172.21.15.83:0/250160157 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8dc071850 con 0x7ff8dc0cfbe0 2024-04-17T10:47:34.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8dbfff640 1 -- 172.21.15.83:0/250160157 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7ff8d40016a0 con 0x7ff8dc0cfbe0 2024-04-17T10:47:34.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8dbfff640 1 -- 172.21.15.83:0/250160157 >> v1:172.21.15.83:6789/0 conn(0x7ff8dc0cfbe0 legacy=0x7ff8dc06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:34.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8dbfff640 1 --2- 172.21.15.83:0/250160157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8c40039f0 0x7ff8c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:34.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8dbfff640 1 --2- 172.21.15.83:0/250160157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8c40043a0 0x7ff8c4004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:34.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8dbfff640 1 --2- 172.21.15.83:0/250160157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8dc15ba30 0x7ff8c4005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:34.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8dbfff640 1 -- 172.21.15.83:0/250160157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8dc071ea0 con 0x7ff8c40039f0 2024-04-17T10:47:34.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8dbfff640 1 -- 172.21.15.83:0/250160157 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c400ba80 con 0x7ff8c40043a0 2024-04-17T10:47:34.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8dbfff640 1 -- 172.21.15.83:0/250160157 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c400bc80 con 0x7ff8dc15ba30 2024-04-17T10:47:34.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8e167d640 1 --2- 172.21.15.83:0/250160157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8c40039f0 0x7ff8c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:34.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8e0e7c640 1 --2- 172.21.15.83:0/250160157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8c40043a0 0x7ff8c4004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:34.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8e1e7e640 1 --2- 172.21.15.83:0/250160157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8dc15ba30 0x7ff8c4005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:34.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8e167d640 1 -- 172.21.15.83:0/250160157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8dc15ba30 msgr2=0x7ff8c4005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:34.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8e167d640 1 --2- 172.21.15.83:0/250160157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8dc15ba30 0x7ff8c4005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:34.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8e167d640 1 -- 172.21.15.83:0/250160157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8c40043a0 msgr2=0x7ff8c4004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:34.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8e167d640 1 --2- 172.21.15.83:0/250160157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8c40043a0 0x7ff8c4004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:34.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8e167d640 1 -- 172.21.15.83:0/250160157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c4003530 con 0x7ff8c40039f0 2024-04-17T10:47:34.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.286+0000 7ff8e1e7e640 1 --2- 172.21.15.83:0/250160157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8dc15ba30 0x7ff8c4005170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.290+0000 7ff8e167d640 1 --2- 172.21.15.83:0/250160157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8c40039f0 0x7ff8c4003dd0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc009fc0 tx=0x7ff8dc06d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6e9c4c45f56904 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.290+0000 7ff8dbfff640 1 -- 172.21.15.83:0/250160157 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc009c10 con 0x7ff8c40039f0 2024-04-17T10:47:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.290+0000 7ff8dbfff640 1 -- 172.21.15.83:0/250160157 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8cc0a5a70 con 0x7ff8c40039f0 2024-04-17T10:47:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.290+0000 7ff8dbfff640 1 -- 172.21.15.83:0/250160157 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc0a5e60 con 0x7ff8c40039f0 2024-04-17T10:47:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.290+0000 7ff8e38e1640 1 -- 172.21.15.83:0/250160157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8c40039f0 msgr2=0x7ff8c4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.290+0000 7ff8e38e1640 1 --2- 172.21.15.83:0/250160157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8c40039f0 0x7ff8c4003dd0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc009fc0 tx=0x7ff8dc06d3b0 comp rx=0 tx=0).stop 2024-04-17T10:47:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.290+0000 7ff8e1e7e640 1 -- 172.21.15.83:0/250160157 reap_dead start 2024-04-17T10:47:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.290+0000 7ff8e38e1640 1 -- 172.21.15.83:0/250160157 shutdown_connections 2024-04-17T10:47:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.290+0000 7ff8e38e1640 1 -- 172.21.15.83:0/250160157 >> 172.21.15.83:0/250160157 conn(0x7ff8dc06abe0 msgr2=0x7ff8dc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.290+0000 7ff8e38e1640 1 -- 172.21.15.83:0/250160157 shutdown_connections 2024-04-17T10:47:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.290+0000 7ff8e38e1640 1 -- 172.21.15.83:0/250160157 wait complete. 2024-04-17T10:47:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.290+0000 7ff8e38e1640 1 Processor -- start 2024-04-17T10:47:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.290+0000 7ff8e38e1640 1 -- start start 2024-04-17T10:47:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.290+0000 7ff8e38e1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8c40039f0 0x7ff8dc1512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.294+0000 7ff8e38e1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8dc156260 0x7ff8dc1596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.294+0000 7ff8e38e1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8dc14c870 0x7ff8dc14ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.294+0000 7ff8e38e1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8dc15cf00 con 0x7ff8dc156260 2024-04-17T10:47:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.294+0000 7ff8e38e1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8dc15cd80 con 0x7ff8dc14c870 2024-04-17T10:47:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.294+0000 7ff8e38e1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8dc167520 con 0x7ff8c40039f0 2024-04-17T10:47:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.294+0000 7ff8e0e7c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8dc156260 0x7ff8dc1596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.294+0000 7ff8e0e7c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8dc156260 0x7ff8dc1596a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60506/0 (socket says 172.21.15.83:60506) 2024-04-17T10:47:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.294+0000 7ff8e0e7c640 1 -- 172.21.15.83:0/3214167058 learned_addr learned my addr 172.21.15.83:0/3214167058 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.294+0000 7ff8e167d640 1 --2- 172.21.15.83:0/3214167058 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8c40039f0 0x7ff8dc1512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.294+0000 7ff8e1e7e640 1 --2- 172.21.15.83:0/3214167058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8dc14c870 0x7ff8dc14ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.294+0000 7ff8e0e7c640 1 -- 172.21.15.83:0/3214167058 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8c40039f0 msgr2=0x7ff8dc1512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.294+0000 7ff8e0e7c640 1 --2- 172.21.15.83:0/3214167058 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8c40039f0 0x7ff8dc1512b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.294+0000 7ff8e0e7c640 1 -- 172.21.15.83:0/3214167058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8dc14c870 msgr2=0x7ff8dc14ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.294+0000 7ff8e0e7c640 1 --2- 172.21.15.83:0/3214167058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8dc14c870 0x7ff8dc14ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.294+0000 7ff8e0e7c640 1 -- 172.21.15.83:0/3214167058 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8dc1548d0 con 0x7ff8dc156260 2024-04-17T10:47:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.294+0000 7ff8e0e7c640 1 --2- 172.21.15.83:0/3214167058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8dc156260 0x7ff8dc1596a0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7ff8d000e7d0 tx=0x7ff8d000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.294+0000 7ff8d9ffb640 1 -- 172.21.15.83:0/3214167058 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d00050d0 con 0x7ff8dc156260 2024-04-17T10:47:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.294+0000 7ff8d9ffb640 1 -- 172.21.15.83:0/3214167058 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d0018070 con 0x7ff8dc156260 2024-04-17T10:47:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.294+0000 7ff8e38e1640 1 -- 172.21.15.83:0/3214167058 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8dc154bb0 con 0x7ff8dc156260 2024-04-17T10:47:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.294+0000 7ff8d9ffb640 1 -- 172.21.15.83:0/3214167058 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d00054b0 con 0x7ff8dc156260 2024-04-17T10:47:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.294+0000 7ff8e38e1640 1 -- 172.21.15.83:0/3214167058 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8dc1566e0 con 0x7ff8dc156260 2024-04-17T10:47:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.294+0000 7ff8e38e1640 1 -- 172.21.15.83:0/3214167058 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8dc074f90 con 0x7ff8dc156260 2024-04-17T10:47:34.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.294+0000 7ff8d9ffb640 1 -- 172.21.15.83:0/3214167058 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff8d000c6c0 con 0x7ff8dc156260 2024-04-17T10:47:34.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.294+0000 7ff8d9ffb640 1 --2- 172.21.15.83:0/3214167058 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff8c8067130 0x7ff8c80695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:34.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.294+0000 7ff8d9ffb640 1 -- 172.21.15.83:0/3214167058 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8d0084a20 con 0x7ff8dc156260 2024-04-17T10:47:34.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.294+0000 7ff8e167d640 1 --2- 172.21.15.83:0/3214167058 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff8c8067130 0x7ff8c80695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:34.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.294+0000 7ff8e167d640 1 --2- 172.21.15.83:0/3214167058 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff8c8067130 0x7ff8c80695f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc0ae000 tx=0x7ff8cc01cc80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:34.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.298+0000 7ff8d9ffb640 1 -- 172.21.15.83:0/3214167058 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff8d0055a30 con 0x7ff8dc156260 2024-04-17T10:47:34.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.450+0000 7ff8e38e1640 1 -- 172.21.15.83:0/3214167058 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff8dc14d230 con 0x7ff8c8067130 2024-04-17T10:47:34.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.458+0000 7ff8d9ffb640 1 -- 172.21.15.83:0/3214167058 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247088 (secure 0 0 0) 0x7ff8cc0a6980 con 0x7ff8c8067130 2024-04-17T10:47:34.457 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:47:34.458 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:47:34.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.462+0000 7ff8e38e1640 1 -- 172.21.15.83:0/3214167058 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff8c8067130 msgr2=0x7ff8c80695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:34.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.462+0000 7ff8e38e1640 1 --2- 172.21.15.83:0/3214167058 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff8c8067130 0x7ff8c80695f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc0ae000 tx=0x7ff8cc01cc80 comp rx=0 tx=0).stop 2024-04-17T10:47:34.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.462+0000 7ff8e38e1640 1 -- 172.21.15.83:0/3214167058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8dc156260 msgr2=0x7ff8dc1596a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:34.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.462+0000 7ff8e38e1640 1 --2- 172.21.15.83:0/3214167058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8dc156260 0x7ff8dc1596a0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7ff8d000e7d0 tx=0x7ff8d000ec90 comp rx=0 tx=0).stop 2024-04-17T10:47:34.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.462+0000 7ff8e38e1640 1 -- 172.21.15.83:0/3214167058 shutdown_connections 2024-04-17T10:47:34.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.462+0000 7ff8e38e1640 1 --2- 172.21.15.83:0/3214167058 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff8c8067130 0x7ff8c80695f0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:34.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.462+0000 7ff8e38e1640 1 --2- 172.21.15.83:0/3214167058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8dc14c870 0x7ff8dc14ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:34.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.462+0000 7ff8e38e1640 1 --2- 172.21.15.83:0/3214167058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8dc156260 0x7ff8dc1596a0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:34.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.462+0000 7ff8e38e1640 1 --2- 172.21.15.83:0/3214167058 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8c40039f0 0x7ff8dc1512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:34.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.462+0000 7ff8e38e1640 1 -- 172.21.15.83:0/3214167058 >> 172.21.15.83:0/3214167058 conn(0x7ff8dc06abe0 msgr2=0x7ff8dc06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:34.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.462+0000 7ff8e38e1640 1 -- 172.21.15.83:0/3214167058 shutdown_connections 2024-04-17T10:47:34.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:34.462+0000 7ff8e38e1640 1 -- 172.21.15.83:0/3214167058 wait complete. 2024-04-17T10:47:34.474 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-04-17T10:47:34.349037+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":524064,"kb_used_data":10860,"kb_used_omap":12,"kb_used_meta":513075,"kb_avail":754450656,"statfs":{"total":773094113280,"available":772557471744,"internally_reserved":0,"allocated":11120640,"data_stored":5471361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":525389360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001431"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.031507+0000","last_change":"2024-04-17T10:47:29.031507+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.031507+0000","last_undegraded":"2024-04-17T10:47:29.031507+0000","last_fullsized":"2024-04-17T10:47:29.031507+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.055397+0000","last_change":"2024-04-17T10:47:29.055397+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.055397+0000","last_undegraded":"2024-04-17T10:47:29.055397+0000","last_fullsized":"2024-04-17T10:47:29.055397+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.027558+0000","last_change":"2024-04-17T10:47:29.027558+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.027558+0000","last_undegraded":"2024-04-17T10:47:29.027558+0000","last_fullsized":"2024-04-17T10:47:29.027558+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.044961+0000","last_change":"2024-04-17T10:47:29.044961+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.044961+0000","last_undegraded":"2024-04-17T10:47:29.044961+0000","last_fullsized":"2024-04-17T10:47:29.044961+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.038581+0000","last_change":"2024-04-17T10:47:29.038581+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.038581+0000","last_undegraded":"2024-04-17T10:47:29.038581+0000","last_fullsized":"2024-04-17T10:47:29.038581+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.021494+0000","last_change":"2024-04-17T10:47:29.021494+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.021494+0000","last_undegraded":"2024-04-17T10:47:29.021494+0000","last_fullsized":"2024-04-17T10:47:29.021494+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.025999+0000","last_change":"2024-04-17T10:47:29.025999+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.025999+0000","last_undegraded":"2024-04-17T10:47:29.025999+0000","last_fullsized":"2024-04-17T10:47:29.025999+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.049465+0000","last_change":"2024-04-17T10:47:29.049465+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.049465+0000","last_undegraded":"2024-04-17T10:47:29.049465+0000","last_fullsized":"2024-04-17T10:47:29.049465+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.024602+0000","last_change":"2024-04-17T10:47:29.024602+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.024602+0000","last_undegraded":"2024-04-17T10:47:29.024602+0000","last_fullsized":"2024-04-17T10:47:29.024602+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.030562+0000","last_change":"2024-04-17T10:47:29.030562+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.030562+0000","last_undegraded":"2024-04-17T10:47:29.030562+0000","last_fullsized":"2024-04-17T10:47:29.030562+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.032399+0000","last_change":"2024-04-17T10:47:29.032399+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.032399+0000","last_undegraded":"2024-04-17T10:47:29.032399+0000","last_fullsized":"2024-04-17T10:47:29.032399+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.029188+0000","last_change":"2024-04-17T10:47:29.029188+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.029188+0000","last_undegraded":"2024-04-17T10:47:29.029188+0000","last_fullsized":"2024-04-17T10:47:29.029188+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.046072+0000","last_change":"2024-04-17T10:47:29.046072+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.046072+0000","last_undegraded":"2024-04-17T10:47:29.046072+0000","last_fullsized":"2024-04-17T10:47:29.046072+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.037835+0000","last_change":"2024-04-17T10:47:29.037835+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.037835+0000","last_undegraded":"2024-04-17T10:47:29.037835+0000","last_fullsized":"2024-04-17T10:47:29.037835+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.045596+0000","last_change":"2024-04-17T10:47:29.045596+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.045596+0000","last_undegraded":"2024-04-17T10:47:29.045596+0000","last_fullsized":"2024-04-17T10:47:29.045596+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.028858+0000","last_change":"2024-04-17T10:47:29.028858+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.028858+0000","last_undegraded":"2024-04-17T10:47:29.028858+0000","last_fullsized":"2024-04-17T10:47:29.028858+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.044488+0000","last_change":"2024-04-17T10:47:29.044488+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.044488+0000","last_undegraded":"2024-04-17T10:47:29.044488+0000","last_fullsized":"2024-04-17T10:47:29.044488+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.044801+0000","last_change":"2024-04-17T10:47:29.044801+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.044801+0000","last_undegraded":"2024-04-17T10:47:29.044801+0000","last_fullsized":"2024-04-17T10:47:29.044801+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.034394+0000","last_change":"2024-04-17T10:47:29.034394+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.034394+0000","last_undegraded":"2024-04-17T10:47:29.034394+0000","last_fullsized":"2024-04-17T10:47:29.034394+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.023972+0000","last_change":"2024-04-17T10:47:29.023972+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.023972+0000","last_undegraded":"2024-04-17T10:47:29.023972+0000","last_fullsized":"2024-04-17T10:47:29.023972+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.034527+0000","last_change":"2024-04-17T10:47:29.034527+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.034527+0000","last_undegraded":"2024-04-17T10:47:29.034527+0000","last_fullsized":"2024-04-17T10:47:29.034527+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.045648+0000","last_change":"2024-04-17T10:47:29.045648+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.045648+0000","last_undegraded":"2024-04-17T10:47:29.045648+0000","last_fullsized":"2024-04-17T10:47:29.045648+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.045251+0000","last_change":"2024-04-17T10:47:29.045251+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.045251+0000","last_undegraded":"2024-04-17T10:47:29.045251+0000","last_fullsized":"2024-04-17T10:47:29.045251+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.027697+0000","last_change":"2024-04-17T10:47:29.027697+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.027697+0000","last_undegraded":"2024-04-17T10:47:29.027697+0000","last_fullsized":"2024-04-17T10:47:29.027697+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.037694+0000","last_change":"2024-04-17T10:47:29.037694+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.037694+0000","last_undegraded":"2024-04-17T10:47:29.037694+0000","last_fullsized":"2024-04-17T10:47:29.037694+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.050922+0000","last_change":"2024-04-17T10:47:29.050922+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.050922+0000","last_undegraded":"2024-04-17T10:47:29.050922+0000","last_fullsized":"2024-04-17T10:47:29.050922+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.024737+0000","last_change":"2024-04-17T10:47:29.024737+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.024737+0000","last_undegraded":"2024-04-17T10:47:29.024737+0000","last_fullsized":"2024-04-17T10:47:29.024737+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.043747+0000","last_change":"2024-04-17T10:47:29.043747+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.043747+0000","last_undegraded":"2024-04-17T10:47:29.043747+0000","last_fullsized":"2024-04-17T10:47:29.043747+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.047296+0000","last_change":"2024-04-17T10:47:29.047296+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.047296+0000","last_undegraded":"2024-04-17T10:47:29.047296+0000","last_fullsized":"2024-04-17T10:47:29.047296+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.035574+0000","last_change":"2024-04-17T10:47:29.035574+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.035574+0000","last_undegraded":"2024-04-17T10:47:29.035574+0000","last_fullsized":"2024-04-17T10:47:29.035574+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.045037+0000","last_change":"2024-04-17T10:47:29.045037+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.045037+0000","last_undegraded":"2024-04-17T10:47:29.045037+0000","last_fullsized":"2024-04-17T10:47:29.045037+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.028571+0000","last_change":"2024-04-17T10:47:29.028571+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.028571+0000","last_undegraded":"2024-04-17T10:47:29.028571+0000","last_fullsized":"2024-04-17T10:47:29.028571+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.034477+0000","last_change":"2024-04-17T10:47:29.034477+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.034477+0000","last_undegraded":"2024-04-17T10:47:29.034477+0000","last_fullsized":"2024-04-17T10:47:29.034477+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.058518+0000","last_change":"2024-04-17T10:47:29.058101+0000","last_active":"2024-04-17T10:47:29.058518+0000","last_peered":"2024-04-17T10:47:29.058518+0000","last_clean":"2024-04-17T10:47:29.058518+0000","last_became_active":"2024-04-17T10:47:29.056266+0000","last_became_peered":"2024-04-17T10:47:29.056266+0000","last_unstale":"2024-04-17T10:47:29.058518+0000","last_undegraded":"2024-04-17T10:47:29.058518+0000","last_fullsized":"2024-04-17T10:47:29.058518+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:10:51.407123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.047078+0000","last_change":"2024-04-17T10:47:29.047078+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.047078+0000","last_undegraded":"2024-04-17T10:47:29.047078+0000","last_fullsized":"2024-04-17T10:47:29.047078+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.044854+0000","last_change":"2024-04-17T10:47:29.037268+0000","last_active":"2024-04-17T10:47:29.044854+0000","last_peered":"2024-04-17T10:47:29.044854+0000","last_clean":"2024-04-17T10:47:29.044854+0000","last_became_active":"2024-04-17T10:47:29.035743+0000","last_became_peered":"2024-04-17T10:47:29.035743+0000","last_unstale":"2024-04-17T10:47:29.044854+0000","last_undegraded":"2024-04-17T10:47:29.044854+0000","last_fullsized":"2024-04-17T10:47:29.044854+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:16:45.173638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.030657+0000","last_change":"2024-04-17T10:47:29.030657+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.030657+0000","last_undegraded":"2024-04-17T10:47:29.030657+0000","last_fullsized":"2024-04-17T10:47:29.030657+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.051205+0000","last_change":"2024-04-17T10:47:29.046225+0000","last_active":"2024-04-17T10:47:29.051205+0000","last_peered":"2024-04-17T10:47:29.051205+0000","last_clean":"2024-04-17T10:47:29.051205+0000","last_became_active":"2024-04-17T10:47:29.043738+0000","last_became_peered":"2024-04-17T10:47:29.043738+0000","last_unstale":"2024-04-17T10:47:29.051205+0000","last_undegraded":"2024-04-17T10:47:29.051205+0000","last_fullsized":"2024-04-17T10:47:29.051205+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:43.205753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.023044+0000","last_change":"2024-04-17T10:47:29.023044+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.023044+0000","last_undegraded":"2024-04-17T10:47:29.023044+0000","last_fullsized":"2024-04-17T10:47:29.023044+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.042042+0000","last_change":"2024-04-17T10:47:29.041206+0000","last_active":"2024-04-17T10:47:29.042042+0000","last_peered":"2024-04-17T10:47:29.042042+0000","last_clean":"2024-04-17T10:47:29.042042+0000","last_became_active":"2024-04-17T10:47:29.040924+0000","last_became_peered":"2024-04-17T10:47:29.040924+0000","last_unstale":"2024-04-17T10:47:29.042042+0000","last_undegraded":"2024-04-17T10:47:29.042042+0000","last_fullsized":"2024-04-17T10:47:29.042042+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:35:16.067757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.028743+0000","last_change":"2024-04-17T10:47:29.028743+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.028743+0000","last_undegraded":"2024-04-17T10:47:29.028743+0000","last_fullsized":"2024-04-17T10:47:29.028743+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.054287+0000","last_change":"2024-04-17T10:47:29.053852+0000","last_active":"2024-04-17T10:47:29.054287+0000","last_peered":"2024-04-17T10:47:29.054287+0000","last_clean":"2024-04-17T10:47:29.054287+0000","last_became_active":"2024-04-17T10:47:29.053434+0000","last_became_peered":"2024-04-17T10:47:29.053434+0000","last_unstale":"2024-04-17T10:47:29.054287+0000","last_undegraded":"2024-04-17T10:47:29.054287+0000","last_fullsized":"2024-04-17T10:47:29.054287+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:36:34.892543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.030058+0000","last_change":"2024-04-17T10:47:29.030058+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.030058+0000","last_undegraded":"2024-04-17T10:47:29.030058+0000","last_fullsized":"2024-04-17T10:47:29.030058+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.041154+0000","last_change":"2024-04-17T10:47:29.040335+0000","last_active":"2024-04-17T10:47:29.041154+0000","last_peered":"2024-04-17T10:47:29.041154+0000","last_clean":"2024-04-17T10:47:29.041154+0000","last_became_active":"2024-04-17T10:47:29.039941+0000","last_became_peered":"2024-04-17T10:47:29.039941+0000","last_unstale":"2024-04-17T10:47:29.041154+0000","last_undegraded":"2024-04-17T10:47:29.041154+0000","last_fullsized":"2024-04-17T10:47:29.041154+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:31:48.398246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.044793+0000","last_change":"2024-04-17T10:47:29.044793+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.044793+0000","last_undegraded":"2024-04-17T10:47:29.044793+0000","last_fullsized":"2024-04-17T10:47:29.044793+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.052502+0000","last_change":"2024-04-17T10:47:29.045408+0000","last_active":"2024-04-17T10:47:29.052502+0000","last_peered":"2024-04-17T10:47:29.052502+0000","last_clean":"2024-04-17T10:47:29.052502+0000","last_became_active":"2024-04-17T10:47:29.044801+0000","last_became_peered":"2024-04-17T10:47:29.044801+0000","last_unstale":"2024-04-17T10:47:29.052502+0000","last_undegraded":"2024-04-17T10:47:29.052502+0000","last_fullsized":"2024-04-17T10:47:29.052502+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:52:11.119097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.024952+0000","last_change":"2024-04-17T10:47:29.024952+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.024952+0000","last_undegraded":"2024-04-17T10:47:29.024952+0000","last_fullsized":"2024-04-17T10:47:29.024952+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.046171+0000","last_change":"2024-04-17T10:47:29.038936+0000","last_active":"2024-04-17T10:47:29.046171+0000","last_peered":"2024-04-17T10:47:29.046171+0000","last_clean":"2024-04-17T10:47:29.046171+0000","last_became_active":"2024-04-17T10:47:29.038436+0000","last_became_peered":"2024-04-17T10:47:29.038436+0000","last_unstale":"2024-04-17T10:47:29.046171+0000","last_undegraded":"2024-04-17T10:47:29.046171+0000","last_fullsized":"2024-04-17T10:47:29.046171+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:20:55.139531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.049911+0000","last_change":"2024-04-17T10:47:29.049911+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.049911+0000","last_undegraded":"2024-04-17T10:47:29.049911+0000","last_fullsized":"2024-04-17T10:47:29.049911+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.042156+0000","last_change":"2024-04-17T10:47:29.040517+0000","last_active":"2024-04-17T10:47:29.042156+0000","last_peered":"2024-04-17T10:47:29.042156+0000","last_clean":"2024-04-17T10:47:29.042156+0000","last_became_active":"2024-04-17T10:47:29.039818+0000","last_became_peered":"2024-04-17T10:47:29.039818+0000","last_unstale":"2024-04-17T10:47:29.042156+0000","last_undegraded":"2024-04-17T10:47:29.042156+0000","last_fullsized":"2024-04-17T10:47:29.042156+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:49:17.199444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.023180+0000","last_change":"2024-04-17T10:47:29.023180+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.023180+0000","last_undegraded":"2024-04-17T10:47:29.023180+0000","last_fullsized":"2024-04-17T10:47:29.023180+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.042609+0000","last_change":"2024-04-17T10:47:29.037265+0000","last_active":"2024-04-17T10:47:29.042609+0000","last_peered":"2024-04-17T10:47:29.042609+0000","last_clean":"2024-04-17T10:47:29.042609+0000","last_became_active":"2024-04-17T10:47:29.036735+0000","last_became_peered":"2024-04-17T10:47:29.036735+0000","last_unstale":"2024-04-17T10:47:29.042609+0000","last_undegraded":"2024-04-17T10:47:29.042609+0000","last_fullsized":"2024-04-17T10:47:29.042609+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:49.197211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.042244+0000","last_change":"2024-04-17T10:47:29.042244+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.042244+0000","last_undegraded":"2024-04-17T10:47:29.042244+0000","last_fullsized":"2024-04-17T10:47:29.042244+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.055631+0000","last_change":"2024-04-17T10:47:29.049241+0000","last_active":"2024-04-17T10:47:29.055631+0000","last_peered":"2024-04-17T10:47:29.055631+0000","last_clean":"2024-04-17T10:47:29.055631+0000","last_became_active":"2024-04-17T10:47:29.048527+0000","last_became_peered":"2024-04-17T10:47:29.048527+0000","last_unstale":"2024-04-17T10:47:29.055631+0000","last_undegraded":"2024-04-17T10:47:29.055631+0000","last_fullsized":"2024-04-17T10:47:29.055631+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:26:50.584899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.023708+0000","last_change":"2024-04-17T10:47:29.023708+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.023708+0000","last_undegraded":"2024-04-17T10:47:29.023708+0000","last_fullsized":"2024-04-17T10:47:29.023708+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.050875+0000","last_change":"2024-04-17T10:47:29.047222+0000","last_active":"2024-04-17T10:47:29.050875+0000","last_peered":"2024-04-17T10:47:29.050875+0000","last_clean":"2024-04-17T10:47:29.050875+0000","last_became_active":"2024-04-17T10:47:29.046540+0000","last_became_peered":"2024-04-17T10:47:29.046540+0000","last_unstale":"2024-04-17T10:47:29.050875+0000","last_undegraded":"2024-04-17T10:47:29.050875+0000","last_fullsized":"2024-04-17T10:47:29.050875+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:41:18.010105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.045109+0000","last_change":"2024-04-17T10:47:29.045109+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.045109+0000","last_undegraded":"2024-04-17T10:47:29.045109+0000","last_fullsized":"2024-04-17T10:47:29.045109+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.051787+0000","last_change":"2024-04-17T10:47:29.051191+0000","last_active":"2024-04-17T10:47:29.051787+0000","last_peered":"2024-04-17T10:47:29.051787+0000","last_clean":"2024-04-17T10:47:29.051787+0000","last_became_active":"2024-04-17T10:47:29.050924+0000","last_became_peered":"2024-04-17T10:47:29.050924+0000","last_unstale":"2024-04-17T10:47:29.051787+0000","last_undegraded":"2024-04-17T10:47:29.051787+0000","last_fullsized":"2024-04-17T10:47:29.051787+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:47:39.522424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.027964+0000","last_change":"2024-04-17T10:47:29.027964+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.027964+0000","last_undegraded":"2024-04-17T10:47:29.027964+0000","last_fullsized":"2024-04-17T10:47:29.027964+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.043366+0000","last_change":"2024-04-17T10:47:29.041596+0000","last_active":"2024-04-17T10:47:29.043366+0000","last_peered":"2024-04-17T10:47:29.043366+0000","last_clean":"2024-04-17T10:47:29.043366+0000","last_became_active":"2024-04-17T10:47:29.041198+0000","last_became_peered":"2024-04-17T10:47:29.041198+0000","last_unstale":"2024-04-17T10:47:29.043366+0000","last_undegraded":"2024-04-17T10:47:29.043366+0000","last_fullsized":"2024-04-17T10:47:29.043366+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:33:24.133568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.024813+0000","last_change":"2024-04-17T10:47:29.024813+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.024813+0000","last_undegraded":"2024-04-17T10:47:29.024813+0000","last_fullsized":"2024-04-17T10:47:29.024813+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.050803+0000","last_change":"2024-04-17T10:47:29.047014+0000","last_active":"2024-04-17T10:47:29.050803+0000","last_peered":"2024-04-17T10:47:29.050803+0000","last_clean":"2024-04-17T10:47:29.050803+0000","last_became_active":"2024-04-17T10:47:29.046269+0000","last_became_peered":"2024-04-17T10:47:29.046269+0000","last_unstale":"2024-04-17T10:47:29.050803+0000","last_undegraded":"2024-04-17T10:47:29.050803+0000","last_fullsized":"2024-04-17T10:47:29.050803+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:59.960069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.034425+0000","last_change":"2024-04-17T10:47:29.034425+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.034425+0000","last_undegraded":"2024-04-17T10:47:29.034425+0000","last_fullsized":"2024-04-17T10:47:29.034425+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.040239+0000","last_change":"2024-04-17T10:47:29.037711+0000","last_active":"2024-04-17T10:47:29.040239+0000","last_peered":"2024-04-17T10:47:29.040239+0000","last_clean":"2024-04-17T10:47:29.040239+0000","last_became_active":"2024-04-17T10:47:29.037183+0000","last_became_peered":"2024-04-17T10:47:29.037183+0000","last_unstale":"2024-04-17T10:47:29.040239+0000","last_undegraded":"2024-04-17T10:47:29.040239+0000","last_fullsized":"2024-04-17T10:47:29.040239+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:07:50.722366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.030365+0000","last_change":"2024-04-17T10:47:29.030365+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.030365+0000","last_undegraded":"2024-04-17T10:47:29.030365+0000","last_fullsized":"2024-04-17T10:47:29.030365+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.051165+0000","last_change":"2024-04-17T10:47:29.046483+0000","last_active":"2024-04-17T10:47:29.051165+0000","last_peered":"2024-04-17T10:47:29.051165+0000","last_clean":"2024-04-17T10:47:29.051165+0000","last_became_active":"2024-04-17T10:47:29.042701+0000","last_became_peered":"2024-04-17T10:47:29.042701+0000","last_unstale":"2024-04-17T10:47:29.051165+0000","last_undegraded":"2024-04-17T10:47:29.051165+0000","last_fullsized":"2024-04-17T10:47:29.051165+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:43.205753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.026502+0000","last_change":"2024-04-17T10:47:29.026502+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.026502+0000","last_undegraded":"2024-04-17T10:47:29.026502+0000","last_fullsized":"2024-04-17T10:47:29.026502+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.052476+0000","last_change":"2024-04-17T10:47:29.044936+0000","last_active":"2024-04-17T10:47:29.052476+0000","last_peered":"2024-04-17T10:47:29.052476+0000","last_clean":"2024-04-17T10:47:29.052476+0000","last_became_active":"2024-04-17T10:47:29.044091+0000","last_became_peered":"2024-04-17T10:47:29.044091+0000","last_unstale":"2024-04-17T10:47:29.052476+0000","last_undegraded":"2024-04-17T10:47:29.052476+0000","last_fullsized":"2024-04-17T10:47:29.052476+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:03:10.762120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.022345+0000","last_change":"2024-04-17T10:47:29.022345+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.022345+0000","last_undegraded":"2024-04-17T10:47:29.022345+0000","last_fullsized":"2024-04-17T10:47:29.022345+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.055071+0000","last_change":"2024-04-17T10:47:29.043948+0000","last_active":"2024-04-17T10:47:29.055071+0000","last_peered":"2024-04-17T10:47:29.055071+0000","last_clean":"2024-04-17T10:47:29.055071+0000","last_became_active":"2024-04-17T10:47:29.043517+0000","last_became_peered":"2024-04-17T10:47:29.043517+0000","last_unstale":"2024-04-17T10:47:29.055071+0000","last_undegraded":"2024-04-17T10:47:29.055071+0000","last_fullsized":"2024-04-17T10:47:29.055071+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:52:11.119097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.035024+0000","last_change":"2024-04-17T10:47:29.035024+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.035024+0000","last_undegraded":"2024-04-17T10:47:29.035024+0000","last_fullsized":"2024-04-17T10:47:29.035024+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.055246+0000","last_change":"2024-04-17T10:47:29.049650+0000","last_active":"2024-04-17T10:47:29.055246+0000","last_peered":"2024-04-17T10:47:29.055246+0000","last_clean":"2024-04-17T10:47:29.055246+0000","last_became_active":"2024-04-17T10:47:29.049263+0000","last_became_peered":"2024-04-17T10:47:29.049263+0000","last_unstale":"2024-04-17T10:47:29.055246+0000","last_undegraded":"2024-04-17T10:47:29.055246+0000","last_fullsized":"2024-04-17T10:47:29.055246+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:41:18.010105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.033707+0000","last_change":"2024-04-17T10:47:29.033707+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.033707+0000","last_undegraded":"2024-04-17T10:47:29.033707+0000","last_fullsized":"2024-04-17T10:47:29.033707+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.052049+0000","last_change":"2024-04-17T10:47:29.041858+0000","last_active":"2024-04-17T10:47:29.052049+0000","last_peered":"2024-04-17T10:47:29.052049+0000","last_clean":"2024-04-17T10:47:29.052049+0000","last_became_active":"2024-04-17T10:47:29.041099+0000","last_became_peered":"2024-04-17T10:47:29.041099+0000","last_unstale":"2024-04-17T10:47:29.052049+0000","last_undegraded":"2024-04-17T10:47:29.052049+0000","last_fullsized":"2024-04-17T10:47:29.052049+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:41:18.010105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.035881+0000","last_change":"2024-04-17T10:47:29.035881+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.035881+0000","last_undegraded":"2024-04-17T10:47:29.035881+0000","last_fullsized":"2024-04-17T10:47:29.035881+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.050068+0000","last_change":"2024-04-17T10:47:29.039405+0000","last_active":"2024-04-17T10:47:29.050068+0000","last_peered":"2024-04-17T10:47:29.050068+0000","last_clean":"2024-04-17T10:47:29.050068+0000","last_became_active":"2024-04-17T10:47:29.038596+0000","last_became_peered":"2024-04-17T10:47:29.038596+0000","last_unstale":"2024-04-17T10:47:29.050068+0000","last_undegraded":"2024-04-17T10:47:29.050068+0000","last_fullsized":"2024-04-17T10:47:29.050068+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:43.205753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.039260+0000","last_change":"2024-04-17T10:47:29.039260+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.039260+0000","last_undegraded":"2024-04-17T10:47:29.039260+0000","last_fullsized":"2024-04-17T10:47:29.039260+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.049867+0000","last_change":"2024-04-17T10:47:29.043402+0000","last_active":"2024-04-17T10:47:29.049867+0000","last_peered":"2024-04-17T10:47:29.049867+0000","last_clean":"2024-04-17T10:47:29.049867+0000","last_became_active":"2024-04-17T10:47:29.042994+0000","last_became_peered":"2024-04-17T10:47:29.042994+0000","last_unstale":"2024-04-17T10:47:29.049867+0000","last_undegraded":"2024-04-17T10:47:29.049867+0000","last_fullsized":"2024-04-17T10:47:29.049867+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:03:10.762120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.048879+0000","last_change":"2024-04-17T10:47:29.048879+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.048879+0000","last_undegraded":"2024-04-17T10:47:29.048879+0000","last_fullsized":"2024-04-17T10:47:29.048879+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.051965+0000","last_change":"2024-04-17T10:47:29.041380+0000","last_active":"2024-04-17T10:47:29.051965+0000","last_peered":"2024-04-17T10:47:29.051965+0000","last_clean":"2024-04-17T10:47:29.051965+0000","last_became_active":"2024-04-17T10:47:29.040614+0000","last_became_peered":"2024-04-17T10:47:29.040614+0000","last_unstale":"2024-04-17T10:47:29.051965+0000","last_undegraded":"2024-04-17T10:47:29.051965+0000","last_fullsized":"2024-04-17T10:47:29.051965+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:59.960069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.030002+0000","last_change":"2024-04-17T10:47:29.030002+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.030002+0000","last_undegraded":"2024-04-17T10:47:29.030002+0000","last_fullsized":"2024-04-17T10:47:29.030002+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.058438+0000","last_change":"2024-04-17T10:47:29.056214+0000","last_active":"2024-04-17T10:47:29.058438+0000","last_peered":"2024-04-17T10:47:29.058438+0000","last_clean":"2024-04-17T10:47:29.058438+0000","last_became_active":"2024-04-17T10:47:29.055718+0000","last_became_peered":"2024-04-17T10:47:29.055718+0000","last_unstale":"2024-04-17T10:47:29.058438+0000","last_undegraded":"2024-04-17T10:47:29.058438+0000","last_fullsized":"2024-04-17T10:47:29.058438+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:28.525331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.028102+0000","last_change":"2024-04-17T10:47:29.028102+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.028102+0000","last_undegraded":"2024-04-17T10:47:29.028102+0000","last_fullsized":"2024-04-17T10:47:29.028102+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.055096+0000","last_change":"2024-04-17T10:47:29.046779+0000","last_active":"2024-04-17T10:47:29.055096+0000","last_peered":"2024-04-17T10:47:29.055096+0000","last_clean":"2024-04-17T10:47:29.055096+0000","last_became_active":"2024-04-17T10:47:29.045869+0000","last_became_peered":"2024-04-17T10:47:29.045869+0000","last_unstale":"2024-04-17T10:47:29.055096+0000","last_undegraded":"2024-04-17T10:47:29.055096+0000","last_fullsized":"2024-04-17T10:47:29.055096+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:59.960069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.037035+0000","last_change":"2024-04-17T10:47:29.037035+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.037035+0000","last_undegraded":"2024-04-17T10:47:29.037035+0000","last_fullsized":"2024-04-17T10:47:29.037035+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.054192+0000","last_change":"2024-04-17T10:47:29.050848+0000","last_active":"2024-04-17T10:47:29.054192+0000","last_peered":"2024-04-17T10:47:29.054192+0000","last_clean":"2024-04-17T10:47:29.054192+0000","last_became_active":"2024-04-17T10:47:29.048692+0000","last_became_peered":"2024-04-17T10:47:29.048692+0000","last_unstale":"2024-04-17T10:47:29.054192+0000","last_undegraded":"2024-04-17T10:47:29.054192+0000","last_fullsized":"2024-04-17T10:47:29.054192+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:43.205753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.030935+0000","last_change":"2024-04-17T10:47:29.030935+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.030935+0000","last_undegraded":"2024-04-17T10:47:29.030935+0000","last_fullsized":"2024-04-17T10:47:29.030935+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.055330+0000","last_change":"2024-04-17T10:47:29.050890+0000","last_active":"2024-04-17T10:47:29.055330+0000","last_peered":"2024-04-17T10:47:29.055330+0000","last_clean":"2024-04-17T10:47:29.055330+0000","last_became_active":"2024-04-17T10:47:29.050540+0000","last_became_peered":"2024-04-17T10:47:29.050540+0000","last_unstale":"2024-04-17T10:47:29.055330+0000","last_undegraded":"2024-04-17T10:47:29.055330+0000","last_fullsized":"2024-04-17T10:47:29.055330+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:19:07.385658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.022875+0000","last_change":"2024-04-17T10:47:29.022875+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.022875+0000","last_undegraded":"2024-04-17T10:47:29.022875+0000","last_fullsized":"2024-04-17T10:47:29.022875+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.053684+0000","last_change":"2024-04-17T10:47:29.051971+0000","last_active":"2024-04-17T10:47:29.053684+0000","last_peered":"2024-04-17T10:47:29.053684+0000","last_clean":"2024-04-17T10:47:29.053684+0000","last_became_active":"2024-04-17T10:47:29.051054+0000","last_became_peered":"2024-04-17T10:47:29.051054+0000","last_unstale":"2024-04-17T10:47:29.053684+0000","last_undegraded":"2024-04-17T10:47:29.053684+0000","last_fullsized":"2024-04-17T10:47:29.053684+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:59.960069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.029219+0000","last_change":"2024-04-17T10:47:29.029219+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.029219+0000","last_undegraded":"2024-04-17T10:47:29.029219+0000","last_fullsized":"2024-04-17T10:47:29.029219+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.037159+0000","last_change":"2024-04-17T10:47:29.032908+0000","last_active":"2024-04-17T10:47:29.037159+0000","last_peered":"2024-04-17T10:47:29.037159+0000","last_clean":"2024-04-17T10:47:29.037159+0000","last_became_active":"2024-04-17T10:47:29.032182+0000","last_became_peered":"2024-04-17T10:47:29.032182+0000","last_unstale":"2024-04-17T10:47:29.037159+0000","last_undegraded":"2024-04-17T10:47:29.037159+0000","last_fullsized":"2024-04-17T10:47:29.037159+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:11:47.505394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.038374+0000","last_change":"2024-04-17T10:47:29.038374+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.038374+0000","last_undegraded":"2024-04-17T10:47:29.038374+0000","last_fullsized":"2024-04-17T10:47:29.038374+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.047599+0000","last_change":"2024-04-17T10:47:29.039563+0000","last_active":"2024-04-17T10:47:29.047599+0000","last_peered":"2024-04-17T10:47:29.047599+0000","last_clean":"2024-04-17T10:47:29.047599+0000","last_became_active":"2024-04-17T10:47:29.039021+0000","last_became_peered":"2024-04-17T10:47:29.039021+0000","last_unstale":"2024-04-17T10:47:29.047599+0000","last_undegraded":"2024-04-17T10:47:29.047599+0000","last_fullsized":"2024-04-17T10:47:29.047599+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:52.837030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-04-17T10:47:29.033845+0000","last_change":"2024-04-17T10:47:29.033845+0000","last_active":"2024-04-17T10:47:29.006354+0000","last_peered":"2024-04-17T10:47:29.006354+0000","last_clean":"2024-04-17T10:47:29.006354+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:47:29.033845+0000","last_undegraded":"2024-04-17T10:47:29.033845+0000","last_fullsized":"2024-04-17T10:47:29.033845+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.035392+0000","last_change":"2024-04-17T10:47:29.031551+0000","last_active":"2024-04-17T10:47:29.035392+0000","last_peered":"2024-04-17T10:47:29.035392+0000","last_clean":"2024-04-17T10:47:29.035392+0000","last_became_active":"2024-04-17T10:47:29.030845+0000","last_became_peered":"2024-04-17T10:47:29.030845+0000","last_unstale":"2024-04-17T10:47:29.035392+0000","last_undegraded":"2024-04-17T10:47:29.035392+0000","last_fullsized":"2024-04-17T10:47:29.035392+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:49.197211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":336,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-17T10:47:29.029346+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:47:29.029346+0000","last_peered":"2024-04-17T10:47:29.029346+0000","last_clean":"2024-04-17T10:47:29.029346+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:47:29.029346+0000","last_undegraded":"2024-04-17T10:47:29.029346+0000","last_fullsized":"2024-04-17T10:47:29.029346+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574966,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59340,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312500,"statfs":{"total":96636764160,"available":96576000000,"internally_reserved":0,"allocated":1830912,"data_stored":1124343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574965,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64328,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307512,"statfs":{"total":96636764160,"available":96570892288,"internally_reserved":0,"allocated":1236992,"data_stored":533975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574966,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50440,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321400,"statfs":{"total":96636764160,"available":96585113600,"internally_reserved":0,"allocated":1236992,"data_stored":533975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574965,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68500,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303340,"statfs":{"total":96636764160,"available":96566620160,"internally_reserved":0,"allocated":1249280,"data_stored":540250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574967,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68496,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303344,"statfs":{"total":96636764160,"available":96566624256,"internally_reserved":0,"allocated":1249280,"data_stored":540250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574965,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58756,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313084,"statfs":{"total":96636764160,"available":96576598016,"internally_reserved":0,"allocated":1236992,"data_stored":533975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94852,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94276988,"statfs":{"total":96636764160,"available":96539635712,"internally_reserved":0,"allocated":1236992,"data_stored":533975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574965,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59352,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312488,"statfs":{"total":96636764160,"available":96575987712,"internally_reserved":0,"allocated":1843200,"data_stored":1130618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:47:35.646 INFO:tasks.ceph.mds.a.smithi083.stderr:2024-04-17T10:47:35.650+0000 7fbfab2ed640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T10:47:37.480 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:47:37.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.614+0000 7f93f7150640 1 Processor -- start 2024-04-17T10:47:37.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.614+0000 7f93f7150640 1 -- start start 2024-04-17T10:47:37.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.618+0000 7f93f7150640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93f015ba30 0x7f93f0072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:37.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.618+0000 7f93f7150640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f93f015c960 0x7f93f00cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:37.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.618+0000 7f93f7150640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93f00cfea0 con 0x7f93f00cfa10 2024-04-17T10:47:37.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.618+0000 7f93f7150640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93f0071ea0 con 0x7f93f015c960 2024-04-17T10:47:37.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.618+0000 7f93f7150640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93f006e320 con 0x7f93f015ba30 2024-04-17T10:47:37.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.618+0000 7f93effff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f93f015c960 0x7f93f00cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:37.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.618+0000 7f93f56ed640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f93f00cfa10 0x7f93f006c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60154/0 (socket says 172.21.15.83:60154) 2024-04-17T10:47:37.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.618+0000 7f93f56ed640 1 -- 172.21.15.83:0/2459395367 learned_addr learned my addr 172.21.15.83:0/2459395367 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:37.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.618+0000 7f93f4eec640 1 --2- 172.21.15.83:0/2459395367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93f015ba30 0x7f93f0072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:37.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.618+0000 7f93effff640 1 -- 172.21.15.83:0/2459395367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93f015ba30 msgr2=0x7f93f0072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:37.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.618+0000 7f93effff640 1 --2- 172.21.15.83:0/2459395367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93f015ba30 0x7f93f0072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:37.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.618+0000 7f93effff640 1 -- 172.21.15.83:0/2459395367 >> v1:172.21.15.83:6789/0 conn(0x7f93f00cfa10 legacy=0x7f93f006c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:37.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.618+0000 7f93effff640 1 -- 172.21.15.83:0/2459395367 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f006e040 con 0x7f93f015c960 2024-04-17T10:47:37.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.618+0000 7f93effff640 1 --2- 172.21.15.83:0/2459395367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93f015c960 0x7f93f00cf4d0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f93e00097c0 tx=0x7f93e001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a61fb5fe3f67f1e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:37.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.618+0000 7f93ef7fe640 1 -- 172.21.15.83:0/2459395367 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e00b0070 con 0x7f93f015c960 2024-04-17T10:47:37.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.618+0000 7f93ef7fe640 1 -- 172.21.15.83:0/2459395367 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e00ab600 con 0x7f93f015c960 2024-04-17T10:47:37.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.618+0000 7f93ef7fe640 1 -- 172.21.15.83:0/2459395367 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e00ab9d0 con 0x7f93f015c960 2024-04-17T10:47:37.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93f7150640 1 -- 172.21.15.83:0/2459395367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93f015c960 msgr2=0x7f93f00cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:37.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93f7150640 1 --2- 172.21.15.83:0/2459395367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93f015c960 0x7f93f00cf4d0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f93e00097c0 tx=0x7f93e001c410 comp rx=0 tx=0).stop 2024-04-17T10:47:37.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93f7150640 1 -- 172.21.15.83:0/2459395367 shutdown_connections 2024-04-17T10:47:37.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93f7150640 1 --2- 172.21.15.83:0/2459395367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93f015c960 0x7f93f00cf4d0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:37.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93f7150640 1 --2- 172.21.15.83:0/2459395367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93f015ba30 0x7f93f0072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:37.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93f7150640 1 -- 172.21.15.83:0/2459395367 >> 172.21.15.83:0/2459395367 conn(0x7f93f006abe0 msgr2=0x7f93f006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:37.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93f7150640 1 -- 172.21.15.83:0/2459395367 shutdown_connections 2024-04-17T10:47:37.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93f7150640 1 -- 172.21.15.83:0/2459395367 wait complete. 2024-04-17T10:47:37.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93f7150640 1 Processor -- start 2024-04-17T10:47:37.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93f7150640 1 -- start start 2024-04-17T10:47:37.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93f7150640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93f00cfa10 0x7f93f0147ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:37.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93f7150640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93f015ba30 0x7f93f015ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:37.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93f7150640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93f015c960 0x7f93f014a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:37.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93f7150640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f006dca0 con 0x7f93f015c960 2024-04-17T10:47:37.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93f7150640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f00aeae0 con 0x7f93f015ba30 2024-04-17T10:47:37.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93f7150640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93f006de20 con 0x7f93f00cfa10 2024-04-17T10:47:37.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93f56ed640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93f015c960 0x7f93f014a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:37.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93f56ed640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93f015c960 0x7f93f014a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34806/0 (socket says 172.21.15.83:34806) 2024-04-17T10:47:37.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93f56ed640 1 -- 172.21.15.83:0/3478371881 learned_addr learned my addr 172.21.15.83:0/3478371881 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:37.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93effff640 1 --2- 172.21.15.83:0/3478371881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93f015ba30 0x7f93f015ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:37.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93f4eec640 1 --2- 172.21.15.83:0/3478371881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93f00cfa10 0x7f93f0147ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:37.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93f56ed640 1 -- 172.21.15.83:0/3478371881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93f00cfa10 msgr2=0x7f93f0147ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:37.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93f56ed640 1 --2- 172.21.15.83:0/3478371881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93f00cfa10 0x7f93f0147ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:37.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93f56ed640 1 -- 172.21.15.83:0/3478371881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93f015ba30 msgr2=0x7f93f015ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:37.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93f56ed640 1 --2- 172.21.15.83:0/3478371881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93f015ba30 0x7f93f015ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:37.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93f56ed640 1 -- 172.21.15.83:0/3478371881 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f0148830 con 0x7f93f015c960 2024-04-17T10:47:37.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93f4eec640 1 --2- 172.21.15.83:0/3478371881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93f00cfa10 0x7f93f0147ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:37.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93effff640 1 --2- 172.21.15.83:0/3478371881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93f015ba30 0x7f93f015ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:47:37.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93f56ed640 1 --2- 172.21.15.83:0/3478371881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93f015c960 0x7f93f014a750 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f93e800ca20 tx=0x7f93e800cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:37.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93ed7fa640 1 -- 172.21.15.83:0/3478371881 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e8017070 con 0x7f93f015c960 2024-04-17T10:47:37.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93ed7fa640 1 -- 172.21.15.83:0/3478371881 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e800abd0 con 0x7f93f015c960 2024-04-17T10:47:37.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93f7150640 1 -- 172.21.15.83:0/3478371881 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f0148b10 con 0x7f93f015c960 2024-04-17T10:47:37.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93ed7fa640 1 -- 172.21.15.83:0/3478371881 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e8012400 con 0x7f93f015c960 2024-04-17T10:47:37.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.622+0000 7f93f7150640 1 -- 172.21.15.83:0/3478371881 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f01490d0 con 0x7f93f015c960 2024-04-17T10:47:37.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.626+0000 7f93f7150640 1 -- 172.21.15.83:0/3478371881 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f0074f90 con 0x7f93f015c960 2024-04-17T10:47:37.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.626+0000 7f93ed7fa640 1 -- 172.21.15.83:0/3478371881 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f93e801a070 con 0x7f93f015c960 2024-04-17T10:47:37.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.626+0000 7f93ed7fa640 1 --2- 172.21.15.83:0/3478371881 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f93dc066f90 0x7f93dc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:37.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.626+0000 7f93ed7fa640 1 -- 172.21.15.83:0/3478371881 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f93e8084d50 con 0x7f93f015c960 2024-04-17T10:47:37.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.626+0000 7f93f4eec640 1 --2- 172.21.15.83:0/3478371881 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f93dc066f90 0x7f93dc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:37.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.626+0000 7f93f4eec640 1 --2- 172.21.15.83:0/3478371881 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f93dc066f90 0x7f93dc069450 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f93e4009840 tx=0x7f93e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:37.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.630+0000 7f93ed7fa640 1 -- 172.21.15.83:0/3478371881 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f93e8055cc0 con 0x7f93f015c960 2024-04-17T10:47:37.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.786+0000 7f93f7150640 1 -- 172.21.15.83:0/3478371881 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f93f0074480 con 0x7f93dc066f90 2024-04-17T10:47:37.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.790+0000 7f93ed7fa640 1 -- 172.21.15.83:0/3478371881 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252487 (secure 0 0 0) 0x7f93f0074480 con 0x7f93dc066f90 2024-04-17T10:47:37.791 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:47:37.793 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:47:37.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.794+0000 7f93f7150640 1 -- 172.21.15.83:0/3478371881 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f93dc066f90 msgr2=0x7f93dc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:37.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.794+0000 7f93f7150640 1 --2- 172.21.15.83:0/3478371881 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f93dc066f90 0x7f93dc069450 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f93e4009840 tx=0x7f93e4008040 comp rx=0 tx=0).stop 2024-04-17T10:47:37.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.794+0000 7f93f7150640 1 -- 172.21.15.83:0/3478371881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93f015c960 msgr2=0x7f93f014a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:37.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.794+0000 7f93f7150640 1 --2- 172.21.15.83:0/3478371881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93f015c960 0x7f93f014a750 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f93e800ca20 tx=0x7f93e800cee0 comp rx=0 tx=0).stop 2024-04-17T10:47:37.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.794+0000 7f93f7150640 1 -- 172.21.15.83:0/3478371881 shutdown_connections 2024-04-17T10:47:37.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.794+0000 7f93f7150640 1 --2- 172.21.15.83:0/3478371881 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f93dc066f90 0x7f93dc069450 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:37.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.794+0000 7f93f7150640 1 --2- 172.21.15.83:0/3478371881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93f015c960 0x7f93f014a750 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:37.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.794+0000 7f93f7150640 1 --2- 172.21.15.83:0/3478371881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93f015ba30 0x7f93f015ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:37.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.794+0000 7f93f7150640 1 --2- 172.21.15.83:0/3478371881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93f00cfa10 0x7f93f0147ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:37.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.794+0000 7f93f7150640 1 -- 172.21.15.83:0/3478371881 >> 172.21.15.83:0/3478371881 conn(0x7f93f006abe0 msgr2=0x7f93f006c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:37.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.794+0000 7f93f7150640 1 -- 172.21.15.83:0/3478371881 shutdown_connections 2024-04-17T10:47:37.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.794+0000 7f93f7150640 1 -- 172.21.15.83:0/3478371881 wait complete. 2024-04-17T10:47:37.809 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-04-17T10:47:36.349634+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":524400,"kb_used_data":11176,"kb_used_omap":12,"kb_used_meta":513075,"kb_avail":754450320,"statfs":{"total":773094113280,"available":772557127680,"internally_reserved":0,"allocated":11444224,"data_stored":5612156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":525389360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.331380"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.873741+0000","last_change":"2024-04-17T10:47:30.048705+0000","last_active":"2024-04-17T10:47:31.873741+0000","last_peered":"2024-04-17T10:47:31.873741+0000","last_clean":"2024-04-17T10:47:31.873741+0000","last_became_active":"2024-04-17T10:47:30.048456+0000","last_became_peered":"2024-04-17T10:47:30.048456+0000","last_unstale":"2024-04-17T10:47:31.873741+0000","last_undegraded":"2024-04-17T10:47:31.873741+0000","last_fullsized":"2024-04-17T10:47:31.873741+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:49:33.979362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.024826+0000","last_change":"2024-04-17T10:47:30.042844+0000","last_active":"2024-04-17T10:47:31.024826+0000","last_peered":"2024-04-17T10:47:31.024826+0000","last_clean":"2024-04-17T10:47:31.024826+0000","last_became_active":"2024-04-17T10:47:30.042549+0000","last_became_peered":"2024-04-17T10:47:30.042549+0000","last_unstale":"2024-04-17T10:47:31.024826+0000","last_undegraded":"2024-04-17T10:47:31.024826+0000","last_fullsized":"2024-04-17T10:47:31.024826+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:12:53.822098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.873684+0000","last_change":"2024-04-17T10:47:30.046005+0000","last_active":"2024-04-17T10:47:31.873684+0000","last_peered":"2024-04-17T10:47:31.873684+0000","last_clean":"2024-04-17T10:47:31.873684+0000","last_became_active":"2024-04-17T10:47:30.045770+0000","last_became_peered":"2024-04-17T10:47:30.045770+0000","last_unstale":"2024-04-17T10:47:31.873684+0000","last_undegraded":"2024-04-17T10:47:31.873684+0000","last_fullsized":"2024-04-17T10:47:31.873684+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:05:42.476528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.089350+0000","last_change":"2024-04-17T10:47:30.035472+0000","last_active":"2024-04-17T10:47:31.089350+0000","last_peered":"2024-04-17T10:47:31.089350+0000","last_clean":"2024-04-17T10:47:31.089350+0000","last_became_active":"2024-04-17T10:47:30.035179+0000","last_became_peered":"2024-04-17T10:47:30.035179+0000","last_unstale":"2024-04-17T10:47:31.089350+0000","last_undegraded":"2024-04-17T10:47:31.089350+0000","last_fullsized":"2024-04-17T10:47:31.089350+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:45:21.880364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.114345+0000","last_change":"2024-04-17T10:47:30.055700+0000","last_active":"2024-04-17T10:47:31.114345+0000","last_peered":"2024-04-17T10:47:31.114345+0000","last_clean":"2024-04-17T10:47:31.114345+0000","last_became_active":"2024-04-17T10:47:30.055070+0000","last_became_peered":"2024-04-17T10:47:30.055070+0000","last_unstale":"2024-04-17T10:47:31.114345+0000","last_undegraded":"2024-04-17T10:47:31.114345+0000","last_fullsized":"2024-04-17T10:47:31.114345+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:15:31.586989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.026803+0000","last_change":"2024-04-17T10:47:30.040930+0000","last_active":"2024-04-17T10:47:31.026803+0000","last_peered":"2024-04-17T10:47:31.026803+0000","last_clean":"2024-04-17T10:47:31.026803+0000","last_became_active":"2024-04-17T10:47:30.033726+0000","last_became_peered":"2024-04-17T10:47:30.033726+0000","last_unstale":"2024-04-17T10:47:31.026803+0000","last_undegraded":"2024-04-17T10:47:31.026803+0000","last_fullsized":"2024-04-17T10:47:31.026803+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:45:13.356210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.026411+0000","last_change":"2024-04-17T10:47:30.045198+0000","last_active":"2024-04-17T10:47:31.026411+0000","last_peered":"2024-04-17T10:47:31.026411+0000","last_clean":"2024-04-17T10:47:31.026411+0000","last_became_active":"2024-04-17T10:47:30.044444+0000","last_became_peered":"2024-04-17T10:47:30.044444+0000","last_unstale":"2024-04-17T10:47:31.026411+0000","last_undegraded":"2024-04-17T10:47:31.026411+0000","last_fullsized":"2024-04-17T10:47:31.026411+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:36:21.212093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.024540+0000","last_change":"2024-04-17T10:47:30.045423+0000","last_active":"2024-04-17T10:47:31.024540+0000","last_peered":"2024-04-17T10:47:31.024540+0000","last_clean":"2024-04-17T10:47:31.024540+0000","last_became_active":"2024-04-17T10:47:30.034506+0000","last_became_peered":"2024-04-17T10:47:30.034506+0000","last_unstale":"2024-04-17T10:47:31.024540+0000","last_undegraded":"2024-04-17T10:47:31.024540+0000","last_fullsized":"2024-04-17T10:47:31.024540+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:06.136101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.029148+0000","last_change":"2024-04-17T10:47:30.069996+0000","last_active":"2024-04-17T10:47:31.029148+0000","last_peered":"2024-04-17T10:47:31.029148+0000","last_clean":"2024-04-17T10:47:31.029148+0000","last_became_active":"2024-04-17T10:47:30.069110+0000","last_became_peered":"2024-04-17T10:47:30.069110+0000","last_unstale":"2024-04-17T10:47:31.029148+0000","last_undegraded":"2024-04-17T10:47:31.029148+0000","last_fullsized":"2024-04-17T10:47:31.029148+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:16:46.178477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.114008+0000","last_change":"2024-04-17T10:47:30.065917+0000","last_active":"2024-04-17T10:47:31.114008+0000","last_peered":"2024-04-17T10:47:31.114008+0000","last_clean":"2024-04-17T10:47:31.114008+0000","last_became_active":"2024-04-17T10:47:30.065240+0000","last_became_peered":"2024-04-17T10:47:30.065240+0000","last_unstale":"2024-04-17T10:47:31.114008+0000","last_undegraded":"2024-04-17T10:47:31.114008+0000","last_fullsized":"2024-04-17T10:47:31.114008+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:16:46.178477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.026497+0000","last_change":"2024-04-17T10:47:30.075423+0000","last_active":"2024-04-17T10:47:31.026497+0000","last_peered":"2024-04-17T10:47:31.026497+0000","last_clean":"2024-04-17T10:47:31.026497+0000","last_became_active":"2024-04-17T10:47:30.074751+0000","last_became_peered":"2024-04-17T10:47:30.074751+0000","last_unstale":"2024-04-17T10:47:31.026497+0000","last_undegraded":"2024-04-17T10:47:31.026497+0000","last_fullsized":"2024-04-17T10:47:31.026497+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:26:19.562816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.027645+0000","last_change":"2024-04-17T10:47:30.073468+0000","last_active":"2024-04-17T10:47:31.027645+0000","last_peered":"2024-04-17T10:47:31.027645+0000","last_clean":"2024-04-17T10:47:31.027645+0000","last_became_active":"2024-04-17T10:47:30.072588+0000","last_became_peered":"2024-04-17T10:47:30.072588+0000","last_unstale":"2024-04-17T10:47:31.027645+0000","last_undegraded":"2024-04-17T10:47:31.027645+0000","last_fullsized":"2024-04-17T10:47:31.027645+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:28:35.040589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.089086+0000","last_change":"2024-04-17T10:47:30.072317+0000","last_active":"2024-04-17T10:47:31.089086+0000","last_peered":"2024-04-17T10:47:31.089086+0000","last_clean":"2024-04-17T10:47:31.089086+0000","last_became_active":"2024-04-17T10:47:30.071999+0000","last_became_peered":"2024-04-17T10:47:30.071999+0000","last_unstale":"2024-04-17T10:47:31.089086+0000","last_undegraded":"2024-04-17T10:47:31.089086+0000","last_fullsized":"2024-04-17T10:47:31.089086+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:21.379075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.057743+0000","last_change":"2024-04-17T10:47:30.052004+0000","last_active":"2024-04-17T10:47:31.057743+0000","last_peered":"2024-04-17T10:47:31.057743+0000","last_clean":"2024-04-17T10:47:31.057743+0000","last_became_active":"2024-04-17T10:47:30.051673+0000","last_became_peered":"2024-04-17T10:47:30.051673+0000","last_unstale":"2024-04-17T10:47:31.057743+0000","last_undegraded":"2024-04-17T10:47:31.057743+0000","last_fullsized":"2024-04-17T10:47:31.057743+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:20:53.595903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.088970+0000","last_change":"2024-04-17T10:47:30.039168+0000","last_active":"2024-04-17T10:47:31.088970+0000","last_peered":"2024-04-17T10:47:31.088970+0000","last_clean":"2024-04-17T10:47:31.088970+0000","last_became_active":"2024-04-17T10:47:30.038841+0000","last_became_peered":"2024-04-17T10:47:30.038841+0000","last_unstale":"2024-04-17T10:47:31.088970+0000","last_undegraded":"2024-04-17T10:47:31.088970+0000","last_fullsized":"2024-04-17T10:47:31.088970+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:12:57.806505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.026458+0000","last_change":"2024-04-17T10:47:30.051035+0000","last_active":"2024-04-17T10:47:31.026458+0000","last_peered":"2024-04-17T10:47:31.026458+0000","last_clean":"2024-04-17T10:47:31.026458+0000","last_became_active":"2024-04-17T10:47:30.050240+0000","last_became_peered":"2024-04-17T10:47:30.050240+0000","last_unstale":"2024-04-17T10:47:31.026458+0000","last_undegraded":"2024-04-17T10:47:31.026458+0000","last_fullsized":"2024-04-17T10:47:31.026458+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:21.379075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.025662+0000","last_change":"2024-04-17T10:47:30.042165+0000","last_active":"2024-04-17T10:47:31.025662+0000","last_peered":"2024-04-17T10:47:31.025662+0000","last_clean":"2024-04-17T10:47:31.025662+0000","last_became_active":"2024-04-17T10:47:30.041937+0000","last_became_peered":"2024-04-17T10:47:30.041937+0000","last_unstale":"2024-04-17T10:47:31.025662+0000","last_undegraded":"2024-04-17T10:47:31.025662+0000","last_fullsized":"2024-04-17T10:47:31.025662+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:48.410480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.089317+0000","last_change":"2024-04-17T10:47:30.036450+0000","last_active":"2024-04-17T10:47:31.089317+0000","last_peered":"2024-04-17T10:47:31.089317+0000","last_clean":"2024-04-17T10:47:31.089317+0000","last_became_active":"2024-04-17T10:47:30.036178+0000","last_became_peered":"2024-04-17T10:47:30.036178+0000","last_unstale":"2024-04-17T10:47:31.089317+0000","last_undegraded":"2024-04-17T10:47:31.089317+0000","last_fullsized":"2024-04-17T10:47:31.089317+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:26:19.562816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.058155+0000","last_change":"2024-04-17T10:47:30.044242+0000","last_active":"2024-04-17T10:47:31.058155+0000","last_peered":"2024-04-17T10:47:31.058155+0000","last_clean":"2024-04-17T10:47:31.058155+0000","last_became_active":"2024-04-17T10:47:30.041309+0000","last_became_peered":"2024-04-17T10:47:30.041309+0000","last_unstale":"2024-04-17T10:47:31.058155+0000","last_undegraded":"2024-04-17T10:47:31.058155+0000","last_fullsized":"2024-04-17T10:47:31.058155+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:49:33.979362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.028016+0000","last_change":"2024-04-17T10:47:30.073517+0000","last_active":"2024-04-17T10:47:31.028016+0000","last_peered":"2024-04-17T10:47:31.028016+0000","last_clean":"2024-04-17T10:47:31.028016+0000","last_became_active":"2024-04-17T10:47:30.072996+0000","last_became_peered":"2024-04-17T10:47:30.072996+0000","last_unstale":"2024-04-17T10:47:31.028016+0000","last_undegraded":"2024-04-17T10:47:31.028016+0000","last_fullsized":"2024-04-17T10:47:31.028016+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:11:48.510233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.057956+0000","last_change":"2024-04-17T10:47:30.034715+0000","last_active":"2024-04-17T10:47:31.057956+0000","last_peered":"2024-04-17T10:47:31.057956+0000","last_clean":"2024-04-17T10:47:31.057956+0000","last_became_active":"2024-04-17T10:47:30.034412+0000","last_became_peered":"2024-04-17T10:47:30.034412+0000","last_unstale":"2024-04-17T10:47:31.057956+0000","last_undegraded":"2024-04-17T10:47:31.057956+0000","last_fullsized":"2024-04-17T10:47:31.057956+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:03:07.047597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.024621+0000","last_change":"2024-04-17T10:47:30.034397+0000","last_active":"2024-04-17T10:47:31.024621+0000","last_peered":"2024-04-17T10:47:31.024621+0000","last_clean":"2024-04-17T10:47:31.024621+0000","last_became_active":"2024-04-17T10:47:30.034079+0000","last_became_peered":"2024-04-17T10:47:30.034079+0000","last_unstale":"2024-04-17T10:47:31.024621+0000","last_undegraded":"2024-04-17T10:47:31.024621+0000","last_fullsized":"2024-04-17T10:47:31.024621+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:14:24.680333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.024859+0000","last_change":"2024-04-17T10:47:30.033757+0000","last_active":"2024-04-17T10:47:31.024859+0000","last_peered":"2024-04-17T10:47:31.024859+0000","last_clean":"2024-04-17T10:47:31.024859+0000","last_became_active":"2024-04-17T10:47:30.033478+0000","last_became_peered":"2024-04-17T10:47:30.033478+0000","last_unstale":"2024-04-17T10:47:31.024859+0000","last_undegraded":"2024-04-17T10:47:31.024859+0000","last_fullsized":"2024-04-17T10:47:31.024859+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:15:48.896725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.027806+0000","last_change":"2024-04-17T10:47:30.043252+0000","last_active":"2024-04-17T10:47:31.027806+0000","last_peered":"2024-04-17T10:47:31.027806+0000","last_clean":"2024-04-17T10:47:31.027806+0000","last_became_active":"2024-04-17T10:47:30.042514+0000","last_became_peered":"2024-04-17T10:47:30.042514+0000","last_unstale":"2024-04-17T10:47:31.027806+0000","last_undegraded":"2024-04-17T10:47:31.027806+0000","last_fullsized":"2024-04-17T10:47:31.027806+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:40:43.032738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.114290+0000","last_change":"2024-04-17T10:47:30.063945+0000","last_active":"2024-04-17T10:47:31.114290+0000","last_peered":"2024-04-17T10:47:31.114290+0000","last_clean":"2024-04-17T10:47:31.114290+0000","last_became_active":"2024-04-17T10:47:30.063269+0000","last_became_peered":"2024-04-17T10:47:30.063269+0000","last_unstale":"2024-04-17T10:47:31.114290+0000","last_undegraded":"2024-04-17T10:47:31.114290+0000","last_fullsized":"2024-04-17T10:47:31.114290+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:30:04.283622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.025446+0000","last_change":"2024-04-17T10:47:30.042831+0000","last_active":"2024-04-17T10:47:31.025446+0000","last_peered":"2024-04-17T10:47:31.025446+0000","last_clean":"2024-04-17T10:47:31.025446+0000","last_became_active":"2024-04-17T10:47:30.042595+0000","last_became_peered":"2024-04-17T10:47:30.042595+0000","last_unstale":"2024-04-17T10:47:31.025446+0000","last_undegraded":"2024-04-17T10:47:31.025446+0000","last_fullsized":"2024-04-17T10:47:31.025446+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:34:09.026317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.027003+0000","last_change":"2024-04-17T10:47:30.035616+0000","last_active":"2024-04-17T10:47:31.027003+0000","last_peered":"2024-04-17T10:47:31.027003+0000","last_clean":"2024-04-17T10:47:31.027003+0000","last_became_active":"2024-04-17T10:47:30.033557+0000","last_became_peered":"2024-04-17T10:47:30.033557+0000","last_unstale":"2024-04-17T10:47:31.027003+0000","last_undegraded":"2024-04-17T10:47:31.027003+0000","last_fullsized":"2024-04-17T10:47:31.027003+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:20:15.304702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.113995+0000","last_change":"2024-04-17T10:47:30.051179+0000","last_active":"2024-04-17T10:47:31.113995+0000","last_peered":"2024-04-17T10:47:31.113995+0000","last_clean":"2024-04-17T10:47:31.113995+0000","last_became_active":"2024-04-17T10:47:30.050074+0000","last_became_peered":"2024-04-17T10:47:30.050074+0000","last_unstale":"2024-04-17T10:47:31.113995+0000","last_undegraded":"2024-04-17T10:47:31.113995+0000","last_fullsized":"2024-04-17T10:47:31.113995+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:50.202050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.024946+0000","last_change":"2024-04-17T10:47:30.045238+0000","last_active":"2024-04-17T10:47:31.024946+0000","last_peered":"2024-04-17T10:47:31.024946+0000","last_clean":"2024-04-17T10:47:31.024946+0000","last_became_active":"2024-04-17T10:47:30.044952+0000","last_became_peered":"2024-04-17T10:47:30.044952+0000","last_unstale":"2024-04-17T10:47:31.024946+0000","last_undegraded":"2024-04-17T10:47:31.024946+0000","last_fullsized":"2024-04-17T10:47:31.024946+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:19:07.797895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.057931+0000","last_change":"2024-04-17T10:47:30.041997+0000","last_active":"2024-04-17T10:47:31.057931+0000","last_peered":"2024-04-17T10:47:31.057931+0000","last_clean":"2024-04-17T10:47:31.057931+0000","last_became_active":"2024-04-17T10:47:30.041687+0000","last_became_peered":"2024-04-17T10:47:30.041687+0000","last_unstale":"2024-04-17T10:47:31.057931+0000","last_undegraded":"2024-04-17T10:47:31.057931+0000","last_fullsized":"2024-04-17T10:47:31.057931+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:29:40.625595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.089043+0000","last_change":"2024-04-17T10:47:30.052178+0000","last_active":"2024-04-17T10:47:31.089043+0000","last_peered":"2024-04-17T10:47:31.089043+0000","last_clean":"2024-04-17T10:47:31.089043+0000","last_became_active":"2024-04-17T10:47:30.051790+0000","last_became_peered":"2024-04-17T10:47:30.051790+0000","last_unstale":"2024-04-17T10:47:31.089043+0000","last_undegraded":"2024-04-17T10:47:31.089043+0000","last_fullsized":"2024-04-17T10:47:31.089043+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:48.410480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.873178+0000","last_change":"2024-04-17T10:47:30.051787+0000","last_active":"2024-04-17T10:47:31.873178+0000","last_peered":"2024-04-17T10:47:31.873178+0000","last_clean":"2024-04-17T10:47:31.873178+0000","last_became_active":"2024-04-17T10:47:30.051478+0000","last_became_peered":"2024-04-17T10:47:30.051478+0000","last_unstale":"2024-04-17T10:47:31.873178+0000","last_undegraded":"2024-04-17T10:47:31.873178+0000","last_fullsized":"2024-04-17T10:47:31.873178+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:16:05.238250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.057981+0000","last_change":"2024-04-17T10:47:30.036287+0000","last_active":"2024-04-17T10:47:31.057981+0000","last_peered":"2024-04-17T10:47:31.057981+0000","last_clean":"2024-04-17T10:47:31.057981+0000","last_became_active":"2024-04-17T10:47:30.035982+0000","last_became_peered":"2024-04-17T10:47:30.035982+0000","last_unstale":"2024-04-17T10:47:31.057981+0000","last_undegraded":"2024-04-17T10:47:31.057981+0000","last_fullsized":"2024-04-17T10:47:31.057981+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:14:00.992525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"102'4","reported_seq":18,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.025617+0000","last_change":"2024-04-17T10:47:29.058101+0000","last_active":"2024-04-17T10:47:31.025617+0000","last_peered":"2024-04-17T10:47:31.025617+0000","last_clean":"2024-04-17T10:47:31.025617+0000","last_became_active":"2024-04-17T10:47:29.056266+0000","last_became_peered":"2024-04-17T10:47:29.056266+0000","last_unstale":"2024-04-17T10:47:31.025617+0000","last_undegraded":"2024-04-17T10:47:31.025617+0000","last_fullsized":"2024-04-17T10:47:31.025617+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:11:58.186874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.089404+0000","last_change":"2024-04-17T10:47:30.060361+0000","last_active":"2024-04-17T10:47:31.089404+0000","last_peered":"2024-04-17T10:47:31.089404+0000","last_clean":"2024-04-17T10:47:31.089404+0000","last_became_active":"2024-04-17T10:47:30.060134+0000","last_became_peered":"2024-04-17T10:47:30.060134+0000","last_unstale":"2024-04-17T10:47:31.089404+0000","last_undegraded":"2024-04-17T10:47:31.089404+0000","last_fullsized":"2024-04-17T10:47:31.089404+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:45:13.356210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.027291+0000","last_change":"2024-04-17T10:47:29.037268+0000","last_active":"2024-04-17T10:47:31.027291+0000","last_peered":"2024-04-17T10:47:31.027291+0000","last_clean":"2024-04-17T10:47:31.027291+0000","last_became_active":"2024-04-17T10:47:29.035743+0000","last_became_peered":"2024-04-17T10:47:29.035743+0000","last_unstale":"2024-04-17T10:47:31.027291+0000","last_undegraded":"2024-04-17T10:47:31.027291+0000","last_fullsized":"2024-04-17T10:47:31.027291+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:30:03.278783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.874039+0000","last_change":"2024-04-17T10:47:30.074378+0000","last_active":"2024-04-17T10:47:31.874039+0000","last_peered":"2024-04-17T10:47:31.874039+0000","last_clean":"2024-04-17T10:47:31.874039+0000","last_became_active":"2024-04-17T10:47:30.074086+0000","last_became_peered":"2024-04-17T10:47:30.074086+0000","last_unstale":"2024-04-17T10:47:31.874039+0000","last_undegraded":"2024-04-17T10:47:31.874039+0000","last_fullsized":"2024-04-17T10:47:31.874039+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:33:07.312023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.025031+0000","last_change":"2024-04-17T10:47:29.046225+0000","last_active":"2024-04-17T10:47:31.025031+0000","last_peered":"2024-04-17T10:47:31.025031+0000","last_clean":"2024-04-17T10:47:31.025031+0000","last_became_active":"2024-04-17T10:47:29.043738+0000","last_became_peered":"2024-04-17T10:47:29.043738+0000","last_unstale":"2024-04-17T10:47:31.025031+0000","last_undegraded":"2024-04-17T10:47:31.025031+0000","last_fullsized":"2024-04-17T10:47:31.025031+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:09:40.122044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.028975+0000","last_change":"2024-04-17T10:47:30.071971+0000","last_active":"2024-04-17T10:47:31.028975+0000","last_peered":"2024-04-17T10:47:31.028975+0000","last_clean":"2024-04-17T10:47:31.028975+0000","last_became_active":"2024-04-17T10:47:30.071447+0000","last_became_peered":"2024-04-17T10:47:30.071447+0000","last_unstale":"2024-04-17T10:47:31.028975+0000","last_undegraded":"2024-04-17T10:47:31.028975+0000","last_fullsized":"2024-04-17T10:47:31.028975+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:50.202050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.024988+0000","last_change":"2024-04-17T10:47:29.041206+0000","last_active":"2024-04-17T10:47:31.024988+0000","last_peered":"2024-04-17T10:47:31.024988+0000","last_clean":"2024-04-17T10:47:31.024988+0000","last_became_active":"2024-04-17T10:47:29.040924+0000","last_became_peered":"2024-04-17T10:47:29.040924+0000","last_unstale":"2024-04-17T10:47:31.024988+0000","last_undegraded":"2024-04-17T10:47:31.024988+0000","last_fullsized":"2024-04-17T10:47:31.024988+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:56:20.145631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.873679+0000","last_change":"2024-04-17T10:47:30.069069+0000","last_active":"2024-04-17T10:47:31.873679+0000","last_peered":"2024-04-17T10:47:31.873679+0000","last_clean":"2024-04-17T10:47:31.873679+0000","last_became_active":"2024-04-17T10:47:30.068686+0000","last_became_peered":"2024-04-17T10:47:30.068686+0000","last_unstale":"2024-04-17T10:47:31.873679+0000","last_undegraded":"2024-04-17T10:47:31.873679+0000","last_fullsized":"2024-04-17T10:47:31.873679+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:56:21.150470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.873457+0000","last_change":"2024-04-17T10:47:29.053852+0000","last_active":"2024-04-17T10:47:31.873457+0000","last_peered":"2024-04-17T10:47:31.873457+0000","last_clean":"2024-04-17T10:47:31.873457+0000","last_became_active":"2024-04-17T10:47:29.053434+0000","last_became_peered":"2024-04-17T10:47:29.053434+0000","last_unstale":"2024-04-17T10:47:31.873457+0000","last_undegraded":"2024-04-17T10:47:31.873457+0000","last_fullsized":"2024-04-17T10:47:31.873457+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:57:52.181439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.026641+0000","last_change":"2024-04-17T10:47:30.061612+0000","last_active":"2024-04-17T10:47:31.026641+0000","last_peered":"2024-04-17T10:47:31.026641+0000","last_clean":"2024-04-17T10:47:31.026641+0000","last_became_active":"2024-04-17T10:47:30.060921+0000","last_became_peered":"2024-04-17T10:47:30.060921+0000","last_unstale":"2024-04-17T10:47:31.026641+0000","last_undegraded":"2024-04-17T10:47:31.026641+0000","last_fullsized":"2024-04-17T10:47:31.026641+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:45:21.880364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.024635+0000","last_change":"2024-04-17T10:47:29.040335+0000","last_active":"2024-04-17T10:47:31.024635+0000","last_peered":"2024-04-17T10:47:31.024635+0000","last_clean":"2024-04-17T10:47:31.024635+0000","last_became_active":"2024-04-17T10:47:29.039941+0000","last_became_peered":"2024-04-17T10:47:29.039941+0000","last_unstale":"2024-04-17T10:47:31.024635+0000","last_undegraded":"2024-04-17T10:47:31.024635+0000","last_fullsized":"2024-04-17T10:47:31.024635+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:54:43.119163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.088937+0000","last_change":"2024-04-17T10:47:30.042107+0000","last_active":"2024-04-17T10:47:31.088937+0000","last_peered":"2024-04-17T10:47:31.088937+0000","last_clean":"2024-04-17T10:47:31.088937+0000","last_became_active":"2024-04-17T10:47:30.041858+0000","last_became_peered":"2024-04-17T10:47:30.041858+0000","last_unstale":"2024-04-17T10:47:31.088937+0000","last_undegraded":"2024-04-17T10:47:31.088937+0000","last_fullsized":"2024-04-17T10:47:31.088937+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:33:20.855259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.113941+0000","last_change":"2024-04-17T10:47:29.045408+0000","last_active":"2024-04-17T10:47:31.113941+0000","last_peered":"2024-04-17T10:47:31.113941+0000","last_clean":"2024-04-17T10:47:31.113941+0000","last_became_active":"2024-04-17T10:47:29.044801+0000","last_became_peered":"2024-04-17T10:47:29.044801+0000","last_unstale":"2024-04-17T10:47:31.113941+0000","last_undegraded":"2024-04-17T10:47:31.113941+0000","last_fullsized":"2024-04-17T10:47:31.113941+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:41:18.010105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.026140+0000","last_change":"2024-04-17T10:47:30.070162+0000","last_active":"2024-04-17T10:47:31.026140+0000","last_peered":"2024-04-17T10:47:31.026140+0000","last_clean":"2024-04-17T10:47:31.026140+0000","last_became_active":"2024-04-17T10:47:30.068646+0000","last_became_peered":"2024-04-17T10:47:30.068646+0000","last_unstale":"2024-04-17T10:47:31.026140+0000","last_undegraded":"2024-04-17T10:47:31.026140+0000","last_fullsized":"2024-04-17T10:47:31.026140+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:35.423589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.873107+0000","last_change":"2024-04-17T10:47:29.038936+0000","last_active":"2024-04-17T10:47:31.873107+0000","last_peered":"2024-04-17T10:47:31.873107+0000","last_clean":"2024-04-17T10:47:31.873107+0000","last_became_active":"2024-04-17T10:47:29.038436+0000","last_became_peered":"2024-04-17T10:47:29.038436+0000","last_unstale":"2024-04-17T10:47:31.873107+0000","last_undegraded":"2024-04-17T10:47:31.873107+0000","last_fullsized":"2024-04-17T10:47:31.873107+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:01:56.566797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.025453+0000","last_change":"2024-04-17T10:47:30.070206+0000","last_active":"2024-04-17T10:47:31.025453+0000","last_peered":"2024-04-17T10:47:31.025453+0000","last_clean":"2024-04-17T10:47:31.025453+0000","last_became_active":"2024-04-17T10:47:30.069901+0000","last_became_peered":"2024-04-17T10:47:30.069901+0000","last_unstale":"2024-04-17T10:47:31.025453+0000","last_undegraded":"2024-04-17T10:47:31.025453+0000","last_fullsized":"2024-04-17T10:47:31.025453+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:33:20.855259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.874131+0000","last_change":"2024-04-17T10:47:29.040517+0000","last_active":"2024-04-17T10:47:31.874131+0000","last_peered":"2024-04-17T10:47:31.874131+0000","last_clean":"2024-04-17T10:47:31.874131+0000","last_became_active":"2024-04-17T10:47:29.039818+0000","last_became_peered":"2024-04-17T10:47:29.039818+0000","last_unstale":"2024-04-17T10:47:31.874131+0000","last_undegraded":"2024-04-17T10:47:31.874131+0000","last_fullsized":"2024-04-17T10:47:31.874131+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:36:19.769729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.029198+0000","last_change":"2024-04-17T10:47:30.070518+0000","last_active":"2024-04-17T10:47:31.029198+0000","last_peered":"2024-04-17T10:47:31.029198+0000","last_clean":"2024-04-17T10:47:31.029198+0000","last_became_active":"2024-04-17T10:47:30.069180+0000","last_became_peered":"2024-04-17T10:47:30.069180+0000","last_unstale":"2024-04-17T10:47:31.029198+0000","last_undegraded":"2024-04-17T10:47:31.029198+0000","last_fullsized":"2024-04-17T10:47:31.029198+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:26:51.589738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.089274+0000","last_change":"2024-04-17T10:47:29.037265+0000","last_active":"2024-04-17T10:47:31.089274+0000","last_peered":"2024-04-17T10:47:31.089274+0000","last_clean":"2024-04-17T10:47:31.089274+0000","last_became_active":"2024-04-17T10:47:29.036735+0000","last_became_peered":"2024-04-17T10:47:29.036735+0000","last_unstale":"2024-04-17T10:47:31.089274+0000","last_undegraded":"2024-04-17T10:47:31.089274+0000","last_fullsized":"2024-04-17T10:47:31.089274+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:26:50.584899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.114154+0000","last_change":"2024-04-17T10:47:30.060770+0000","last_active":"2024-04-17T10:47:31.114154+0000","last_peered":"2024-04-17T10:47:31.114154+0000","last_clean":"2024-04-17T10:47:31.114154+0000","last_became_active":"2024-04-17T10:47:30.059971+0000","last_became_peered":"2024-04-17T10:47:30.059971+0000","last_unstale":"2024-04-17T10:47:31.114154+0000","last_undegraded":"2024-04-17T10:47:31.114154+0000","last_fullsized":"2024-04-17T10:47:31.114154+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:26:51.589738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"102'3","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.028002+0000","last_change":"2024-04-17T10:47:29.049241+0000","last_active":"2024-04-17T10:47:31.028002+0000","last_peered":"2024-04-17T10:47:31.028002+0000","last_clean":"2024-04-17T10:47:31.028002+0000","last_became_active":"2024-04-17T10:47:29.048527+0000","last_became_peered":"2024-04-17T10:47:29.048527+0000","last_unstale":"2024-04-17T10:47:31.028002+0000","last_undegraded":"2024-04-17T10:47:31.028002+0000","last_fullsized":"2024-04-17T10:47:31.028002+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:15:30.582150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.027400+0000","last_change":"2024-04-17T10:47:30.052542+0000","last_active":"2024-04-17T10:47:31.027400+0000","last_peered":"2024-04-17T10:47:31.027400+0000","last_clean":"2024-04-17T10:47:31.027400+0000","last_became_active":"2024-04-17T10:47:30.051856+0000","last_became_peered":"2024-04-17T10:47:30.051856+0000","last_unstale":"2024-04-17T10:47:31.027400+0000","last_undegraded":"2024-04-17T10:47:31.027400+0000","last_fullsized":"2024-04-17T10:47:31.027400+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:07:34.000873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"102'6","reported_seq":20,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.028219+0000","last_change":"2024-04-17T10:47:29.047222+0000","last_active":"2024-04-17T10:47:31.028219+0000","last_peered":"2024-04-17T10:47:31.028219+0000","last_clean":"2024-04-17T10:47:31.028219+0000","last_became_active":"2024-04-17T10:47:29.046540+0000","last_became_peered":"2024-04-17T10:47:29.046540+0000","last_unstale":"2024-04-17T10:47:31.028219+0000","last_undegraded":"2024-04-17T10:47:31.028219+0000","last_fullsized":"2024-04-17T10:47:31.028219+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:19:07.385658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.089134+0000","last_change":"2024-04-17T10:47:30.035439+0000","last_active":"2024-04-17T10:47:31.089134+0000","last_peered":"2024-04-17T10:47:31.089134+0000","last_clean":"2024-04-17T10:47:31.089134+0000","last_became_active":"2024-04-17T10:47:30.035134+0000","last_became_peered":"2024-04-17T10:47:30.035134+0000","last_unstale":"2024-04-17T10:47:31.089134+0000","last_undegraded":"2024-04-17T10:47:31.089134+0000","last_fullsized":"2024-04-17T10:47:31.089134+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:36:21.212093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.058083+0000","last_change":"2024-04-17T10:47:29.051191+0000","last_active":"2024-04-17T10:47:31.058083+0000","last_peered":"2024-04-17T10:47:31.058083+0000","last_clean":"2024-04-17T10:47:31.058083+0000","last_became_active":"2024-04-17T10:47:29.050924+0000","last_became_peered":"2024-04-17T10:47:29.050924+0000","last_unstale":"2024-04-17T10:47:31.058083+0000","last_undegraded":"2024-04-17T10:47:31.058083+0000","last_fullsized":"2024-04-17T10:47:31.058083+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:38:33.057410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.873540+0000","last_change":"2024-04-17T10:47:30.046665+0000","last_active":"2024-04-17T10:47:31.873540+0000","last_peered":"2024-04-17T10:47:31.873540+0000","last_clean":"2024-04-17T10:47:31.873540+0000","last_became_active":"2024-04-17T10:47:30.046185+0000","last_became_peered":"2024-04-17T10:47:30.046185+0000","last_unstale":"2024-04-17T10:47:31.873540+0000","last_undegraded":"2024-04-17T10:47:31.873540+0000","last_fullsized":"2024-04-17T10:47:31.873540+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:35:30.111415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.058185+0000","last_change":"2024-04-17T10:47:29.041596+0000","last_active":"2024-04-17T10:47:31.058185+0000","last_peered":"2024-04-17T10:47:31.058185+0000","last_clean":"2024-04-17T10:47:31.058185+0000","last_became_active":"2024-04-17T10:47:29.041198+0000","last_became_peered":"2024-04-17T10:47:29.041198+0000","last_unstale":"2024-04-17T10:47:31.058185+0000","last_undegraded":"2024-04-17T10:47:31.058185+0000","last_fullsized":"2024-04-17T10:47:31.058185+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:16:52.775371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.026211+0000","last_change":"2024-04-17T10:47:30.068477+0000","last_active":"2024-04-17T10:47:31.026211+0000","last_peered":"2024-04-17T10:47:31.026211+0000","last_clean":"2024-04-17T10:47:31.026211+0000","last_became_active":"2024-04-17T10:47:30.067910+0000","last_became_peered":"2024-04-17T10:47:30.067910+0000","last_unstale":"2024-04-17T10:47:31.026211+0000","last_undegraded":"2024-04-17T10:47:31.026211+0000","last_fullsized":"2024-04-17T10:47:31.026211+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:26:10.204564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.026093+0000","last_change":"2024-04-17T10:47:29.047014+0000","last_active":"2024-04-17T10:47:31.026093+0000","last_peered":"2024-04-17T10:47:31.026093+0000","last_clean":"2024-04-17T10:47:31.026093+0000","last_became_active":"2024-04-17T10:47:29.046269+0000","last_became_peered":"2024-04-17T10:47:29.046269+0000","last_unstale":"2024-04-17T10:47:31.026093+0000","last_undegraded":"2024-04-17T10:47:31.026093+0000","last_fullsized":"2024-04-17T10:47:31.026093+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:28.525331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.057745+0000","last_change":"2024-04-17T10:47:30.072060+0000","last_active":"2024-04-17T10:47:31.057745+0000","last_peered":"2024-04-17T10:47:31.057745+0000","last_clean":"2024-04-17T10:47:31.057745+0000","last_became_active":"2024-04-17T10:47:30.071792+0000","last_became_peered":"2024-04-17T10:47:30.071792+0000","last_unstale":"2024-04-17T10:47:31.057745+0000","last_undegraded":"2024-04-17T10:47:31.057745+0000","last_fullsized":"2024-04-17T10:47:31.057745+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:32:57.744862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"102'8","reported_seq":22,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.873506+0000","last_change":"2024-04-17T10:47:29.037711+0000","last_active":"2024-04-17T10:47:31.873506+0000","last_peered":"2024-04-17T10:47:31.873506+0000","last_clean":"2024-04-17T10:47:31.873506+0000","last_became_active":"2024-04-17T10:47:29.037183+0000","last_became_peered":"2024-04-17T10:47:29.037183+0000","last_unstale":"2024-04-17T10:47:31.873506+0000","last_undegraded":"2024-04-17T10:47:31.873506+0000","last_fullsized":"2024-04-17T10:47:31.873506+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:17:01.398534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.114321+0000","last_change":"2024-04-17T10:47:30.071168+0000","last_active":"2024-04-17T10:47:31.114321+0000","last_peered":"2024-04-17T10:47:31.114321+0000","last_clean":"2024-04-17T10:47:31.114321+0000","last_became_active":"2024-04-17T10:47:30.070634+0000","last_became_peered":"2024-04-17T10:47:30.070634+0000","last_unstale":"2024-04-17T10:47:31.114321+0000","last_undegraded":"2024-04-17T10:47:31.114321+0000","last_fullsized":"2024-04-17T10:47:31.114321+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:23:56.363006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.028770+0000","last_change":"2024-04-17T10:47:29.046483+0000","last_active":"2024-04-17T10:47:31.028770+0000","last_peered":"2024-04-17T10:47:31.028770+0000","last_clean":"2024-04-17T10:47:31.028770+0000","last_became_active":"2024-04-17T10:47:29.042701+0000","last_became_peered":"2024-04-17T10:47:29.042701+0000","last_unstale":"2024-04-17T10:47:31.028770+0000","last_undegraded":"2024-04-17T10:47:31.028770+0000","last_fullsized":"2024-04-17T10:47:31.028770+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:10:51.407123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.873701+0000","last_change":"2024-04-17T10:47:30.052297+0000","last_active":"2024-04-17T10:47:31.873701+0000","last_peered":"2024-04-17T10:47:31.873701+0000","last_clean":"2024-04-17T10:47:31.873701+0000","last_became_active":"2024-04-17T10:47:30.051891+0000","last_became_peered":"2024-04-17T10:47:30.051891+0000","last_unstale":"2024-04-17T10:47:31.873701+0000","last_undegraded":"2024-04-17T10:47:31.873701+0000","last_fullsized":"2024-04-17T10:47:31.873701+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:41:24.748239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.113959+0000","last_change":"2024-04-17T10:47:29.044936+0000","last_active":"2024-04-17T10:47:31.113959+0000","last_peered":"2024-04-17T10:47:31.113959+0000","last_clean":"2024-04-17T10:47:31.113959+0000","last_became_active":"2024-04-17T10:47:29.044091+0000","last_became_peered":"2024-04-17T10:47:29.044091+0000","last_unstale":"2024-04-17T10:47:31.113959+0000","last_undegraded":"2024-04-17T10:47:31.113959+0000","last_fullsized":"2024-04-17T10:47:31.113959+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:28.525331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.027776+0000","last_change":"2024-04-17T10:47:30.075198+0000","last_active":"2024-04-17T10:47:31.027776+0000","last_peered":"2024-04-17T10:47:31.027776+0000","last_clean":"2024-04-17T10:47:31.027776+0000","last_became_active":"2024-04-17T10:47:30.074236+0000","last_became_peered":"2024-04-17T10:47:30.074236+0000","last_unstale":"2024-04-17T10:47:31.027776+0000","last_undegraded":"2024-04-17T10:47:31.027776+0000","last_fullsized":"2024-04-17T10:47:31.027776+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:09:43.541549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.025173+0000","last_change":"2024-04-17T10:47:29.043948+0000","last_active":"2024-04-17T10:47:31.025173+0000","last_peered":"2024-04-17T10:47:31.025173+0000","last_clean":"2024-04-17T10:47:31.025173+0000","last_became_active":"2024-04-17T10:47:29.043517+0000","last_became_peered":"2024-04-17T10:47:29.043517+0000","last_unstale":"2024-04-17T10:47:31.025173+0000","last_undegraded":"2024-04-17T10:47:31.025173+0000","last_fullsized":"2024-04-17T10:47:31.025173+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:07:55.729551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.058092+0000","last_change":"2024-04-17T10:47:30.041127+0000","last_active":"2024-04-17T10:47:31.058092+0000","last_peered":"2024-04-17T10:47:31.058092+0000","last_clean":"2024-04-17T10:47:31.058092+0000","last_became_active":"2024-04-17T10:47:30.038400+0000","last_became_peered":"2024-04-17T10:47:30.038400+0000","last_unstale":"2024-04-17T10:47:31.058092+0000","last_undegraded":"2024-04-17T10:47:31.058092+0000","last_fullsized":"2024-04-17T10:47:31.058092+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:05:42.476528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.025135+0000","last_change":"2024-04-17T10:47:29.049650+0000","last_active":"2024-04-17T10:47:31.025135+0000","last_peered":"2024-04-17T10:47:31.025135+0000","last_clean":"2024-04-17T10:47:31.025135+0000","last_became_active":"2024-04-17T10:47:29.049263+0000","last_became_peered":"2024-04-17T10:47:29.049263+0000","last_unstale":"2024-04-17T10:47:31.025135+0000","last_undegraded":"2024-04-17T10:47:31.025135+0000","last_fullsized":"2024-04-17T10:47:31.025135+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:27:28.502783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.058018+0000","last_change":"2024-04-17T10:47:30.049676+0000","last_active":"2024-04-17T10:47:31.058018+0000","last_peered":"2024-04-17T10:47:31.058018+0000","last_clean":"2024-04-17T10:47:31.058018+0000","last_became_active":"2024-04-17T10:47:30.049294+0000","last_became_peered":"2024-04-17T10:47:30.049294+0000","last_unstale":"2024-04-17T10:47:31.058018+0000","last_undegraded":"2024-04-17T10:47:31.058018+0000","last_fullsized":"2024-04-17T10:47:31.058018+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:32:32.003472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.873654+0000","last_change":"2024-04-17T10:47:29.041858+0000","last_active":"2024-04-17T10:47:31.873654+0000","last_peered":"2024-04-17T10:47:31.873654+0000","last_clean":"2024-04-17T10:47:31.873654+0000","last_became_active":"2024-04-17T10:47:29.041099+0000","last_became_peered":"2024-04-17T10:47:29.041099+0000","last_unstale":"2024-04-17T10:47:31.873654+0000","last_undegraded":"2024-04-17T10:47:31.873654+0000","last_fullsized":"2024-04-17T10:47:31.873654+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:28.525331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.057647+0000","last_change":"2024-04-17T10:47:30.037138+0000","last_active":"2024-04-17T10:47:31.057647+0000","last_peered":"2024-04-17T10:47:31.057647+0000","last_clean":"2024-04-17T10:47:31.057647+0000","last_became_active":"2024-04-17T10:47:30.036819+0000","last_became_peered":"2024-04-17T10:47:30.036819+0000","last_unstale":"2024-04-17T10:47:31.057647+0000","last_undegraded":"2024-04-17T10:47:31.057647+0000","last_fullsized":"2024-04-17T10:47:31.057647+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:36:20.774568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.873688+0000","last_change":"2024-04-17T10:47:29.039405+0000","last_active":"2024-04-17T10:47:31.873688+0000","last_peered":"2024-04-17T10:47:31.873688+0000","last_clean":"2024-04-17T10:47:31.873688+0000","last_became_active":"2024-04-17T10:47:29.038596+0000","last_became_peered":"2024-04-17T10:47:29.038596+0000","last_unstale":"2024-04-17T10:47:31.873688+0000","last_undegraded":"2024-04-17T10:47:31.873688+0000","last_fullsized":"2024-04-17T10:47:31.873688+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:19:07.385658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.113467+0000","last_change":"2024-04-17T10:47:30.065547+0000","last_active":"2024-04-17T10:47:31.113467+0000","last_peered":"2024-04-17T10:47:31.113467+0000","last_clean":"2024-04-17T10:47:31.113467+0000","last_became_active":"2024-04-17T10:47:30.064989+0000","last_became_peered":"2024-04-17T10:47:30.064989+0000","last_unstale":"2024-04-17T10:47:31.113467+0000","last_undegraded":"2024-04-17T10:47:31.113467+0000","last_fullsized":"2024-04-17T10:47:31.113467+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:28:35.040589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.024695+0000","last_change":"2024-04-17T10:47:29.043402+0000","last_active":"2024-04-17T10:47:31.024695+0000","last_peered":"2024-04-17T10:47:31.024695+0000","last_clean":"2024-04-17T10:47:31.024695+0000","last_became_active":"2024-04-17T10:47:29.042994+0000","last_became_peered":"2024-04-17T10:47:29.042994+0000","last_unstale":"2024-04-17T10:47:31.024695+0000","last_undegraded":"2024-04-17T10:47:31.024695+0000","last_fullsized":"2024-04-17T10:47:31.024695+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:07:59.153872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.025635+0000","last_change":"2024-04-17T10:47:30.046649+0000","last_active":"2024-04-17T10:47:31.025635+0000","last_peered":"2024-04-17T10:47:31.025635+0000","last_clean":"2024-04-17T10:47:31.025635+0000","last_became_active":"2024-04-17T10:47:30.046330+0000","last_became_peered":"2024-04-17T10:47:30.046330+0000","last_unstale":"2024-04-17T10:47:31.025635+0000","last_undegraded":"2024-04-17T10:47:31.025635+0000","last_fullsized":"2024-04-17T10:47:31.025635+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:12:57.806505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.874090+0000","last_change":"2024-04-17T10:47:29.041380+0000","last_active":"2024-04-17T10:47:31.874090+0000","last_peered":"2024-04-17T10:47:31.874090+0000","last_clean":"2024-04-17T10:47:31.874090+0000","last_became_active":"2024-04-17T10:47:29.040614+0000","last_became_peered":"2024-04-17T10:47:29.040614+0000","last_unstale":"2024-04-17T10:47:31.874090+0000","last_undegraded":"2024-04-17T10:47:31.874090+0000","last_fullsized":"2024-04-17T10:47:31.874090+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:10:51.407123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.874243+0000","last_change":"2024-04-17T10:47:30.045456+0000","last_active":"2024-04-17T10:47:31.874243+0000","last_peered":"2024-04-17T10:47:31.874243+0000","last_clean":"2024-04-17T10:47:31.874243+0000","last_became_active":"2024-04-17T10:47:30.045187+0000","last_became_peered":"2024-04-17T10:47:30.045187+0000","last_unstale":"2024-04-17T10:47:31.874243+0000","last_undegraded":"2024-04-17T10:47:31.874243+0000","last_fullsized":"2024-04-17T10:47:31.874243+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:35:40.144263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"103'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.025693+0000","last_change":"2024-04-17T10:47:29.056214+0000","last_active":"2024-04-17T10:47:31.025693+0000","last_peered":"2024-04-17T10:47:31.025693+0000","last_clean":"2024-04-17T10:47:31.025693+0000","last_became_active":"2024-04-17T10:47:29.055718+0000","last_became_peered":"2024-04-17T10:47:29.055718+0000","last_unstale":"2024-04-17T10:47:31.025693+0000","last_undegraded":"2024-04-17T10:47:31.025693+0000","last_fullsized":"2024-04-17T10:47:31.025693+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:26:09.199725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.873891+0000","last_change":"2024-04-17T10:47:30.065389+0000","last_active":"2024-04-17T10:47:31.873891+0000","last_peered":"2024-04-17T10:47:31.873891+0000","last_clean":"2024-04-17T10:47:31.873891+0000","last_became_active":"2024-04-17T10:47:30.065035+0000","last_became_peered":"2024-04-17T10:47:30.065035+0000","last_unstale":"2024-04-17T10:47:31.873891+0000","last_undegraded":"2024-04-17T10:47:31.873891+0000","last_fullsized":"2024-04-17T10:47:31.873891+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:35:28.623354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.025276+0000","last_change":"2024-04-17T10:47:29.046779+0000","last_active":"2024-04-17T10:47:31.025276+0000","last_peered":"2024-04-17T10:47:31.025276+0000","last_clean":"2024-04-17T10:47:31.025276+0000","last_became_active":"2024-04-17T10:47:29.045869+0000","last_became_peered":"2024-04-17T10:47:29.045869+0000","last_unstale":"2024-04-17T10:47:31.025276+0000","last_undegraded":"2024-04-17T10:47:31.025276+0000","last_fullsized":"2024-04-17T10:47:31.025276+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:34.418750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.114545+0000","last_change":"2024-04-17T10:47:30.057840+0000","last_active":"2024-04-17T10:47:31.114545+0000","last_peered":"2024-04-17T10:47:31.114545+0000","last_clean":"2024-04-17T10:47:31.114545+0000","last_became_active":"2024-04-17T10:47:30.057074+0000","last_became_peered":"2024-04-17T10:47:30.057074+0000","last_unstale":"2024-04-17T10:47:31.114545+0000","last_undegraded":"2024-04-17T10:47:31.114545+0000","last_fullsized":"2024-04-17T10:47:31.114545+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:25:05.755856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.114274+0000","last_change":"2024-04-17T10:47:29.050848+0000","last_active":"2024-04-17T10:47:31.114274+0000","last_peered":"2024-04-17T10:47:31.114274+0000","last_clean":"2024-04-17T10:47:31.114274+0000","last_became_active":"2024-04-17T10:47:29.048692+0000","last_became_peered":"2024-04-17T10:47:29.048692+0000","last_unstale":"2024-04-17T10:47:31.114274+0000","last_undegraded":"2024-04-17T10:47:31.114274+0000","last_fullsized":"2024-04-17T10:47:31.114274+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:19:07.385658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.058196+0000","last_change":"2024-04-17T10:47:30.072398+0000","last_active":"2024-04-17T10:47:31.058196+0000","last_peered":"2024-04-17T10:47:31.058196+0000","last_clean":"2024-04-17T10:47:31.058196+0000","last_became_active":"2024-04-17T10:47:30.071695+0000","last_became_peered":"2024-04-17T10:47:30.071695+0000","last_unstale":"2024-04-17T10:47:31.058196+0000","last_undegraded":"2024-04-17T10:47:31.058196+0000","last_fullsized":"2024-04-17T10:47:31.058196+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:35:28.623354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.025183+0000","last_change":"2024-04-17T10:47:29.050890+0000","last_active":"2024-04-17T10:47:31.025183+0000","last_peered":"2024-04-17T10:47:31.025183+0000","last_clean":"2024-04-17T10:47:31.025183+0000","last_became_active":"2024-04-17T10:47:29.050540+0000","last_became_peered":"2024-04-17T10:47:29.050540+0000","last_unstale":"2024-04-17T10:47:31.025183+0000","last_undegraded":"2024-04-17T10:47:31.025183+0000","last_fullsized":"2024-04-17T10:47:31.025183+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:41:23.743400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.027328+0000","last_change":"2024-04-17T10:47:30.072143+0000","last_active":"2024-04-17T10:47:31.027328+0000","last_peered":"2024-04-17T10:47:31.027328+0000","last_clean":"2024-04-17T10:47:31.027328+0000","last_became_active":"2024-04-17T10:47:30.071370+0000","last_became_peered":"2024-04-17T10:47:30.071370+0000","last_unstale":"2024-04-17T10:47:31.027328+0000","last_undegraded":"2024-04-17T10:47:31.027328+0000","last_fullsized":"2024-04-17T10:47:31.027328+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:56:49.097010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.113408+0000","last_change":"2024-04-17T10:47:29.051971+0000","last_active":"2024-04-17T10:47:31.113408+0000","last_peered":"2024-04-17T10:47:31.113408+0000","last_clean":"2024-04-17T10:47:31.113408+0000","last_became_active":"2024-04-17T10:47:29.051054+0000","last_became_peered":"2024-04-17T10:47:29.051054+0000","last_unstale":"2024-04-17T10:47:31.113408+0000","last_undegraded":"2024-04-17T10:47:31.113408+0000","last_fullsized":"2024-04-17T10:47:31.113408+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:10:51.407123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.057658+0000","last_change":"2024-04-17T10:47:30.072148+0000","last_active":"2024-04-17T10:47:31.057658+0000","last_peered":"2024-04-17T10:47:31.057658+0000","last_clean":"2024-04-17T10:47:31.057658+0000","last_became_active":"2024-04-17T10:47:30.071855+0000","last_became_peered":"2024-04-17T10:47:30.071855+0000","last_unstale":"2024-04-17T10:47:31.057658+0000","last_undegraded":"2024-04-17T10:47:31.057658+0000","last_fullsized":"2024-04-17T10:47:31.057658+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:55:28.983827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.089135+0000","last_change":"2024-04-17T10:47:29.032908+0000","last_active":"2024-04-17T10:47:31.089135+0000","last_peered":"2024-04-17T10:47:31.089135+0000","last_clean":"2024-04-17T10:47:31.089135+0000","last_became_active":"2024-04-17T10:47:29.032182+0000","last_became_peered":"2024-04-17T10:47:29.032182+0000","last_unstale":"2024-04-17T10:47:31.089135+0000","last_undegraded":"2024-04-17T10:47:31.089135+0000","last_fullsized":"2024-04-17T10:47:31.089135+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:16:45.173638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.113667+0000","last_change":"2024-04-17T10:47:30.050911+0000","last_active":"2024-04-17T10:47:31.113667+0000","last_peered":"2024-04-17T10:47:31.113667+0000","last_clean":"2024-04-17T10:47:31.113667+0000","last_became_active":"2024-04-17T10:47:30.049884+0000","last_became_peered":"2024-04-17T10:47:30.049884+0000","last_unstale":"2024-04-17T10:47:31.113667+0000","last_undegraded":"2024-04-17T10:47:31.113667+0000","last_fullsized":"2024-04-17T10:47:31.113667+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:11:48.510233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.057753+0000","last_change":"2024-04-17T10:47:29.039563+0000","last_active":"2024-04-17T10:47:31.057753+0000","last_peered":"2024-04-17T10:47:31.057753+0000","last_clean":"2024-04-17T10:47:31.057753+0000","last_became_active":"2024-04-17T10:47:29.039021+0000","last_became_peered":"2024-04-17T10:47:29.039021+0000","last_unstale":"2024-04-17T10:47:31.057753+0000","last_undegraded":"2024-04-17T10:47:31.057753+0000","last_fullsized":"2024-04-17T10:47:31.057753+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:58:02.272118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.058051+0000","last_change":"2024-04-17T10:47:30.036957+0000","last_active":"2024-04-17T10:47:31.058051+0000","last_peered":"2024-04-17T10:47:31.058051+0000","last_clean":"2024-04-17T10:47:31.058051+0000","last_became_active":"2024-04-17T10:47:30.036612+0000","last_became_peered":"2024-04-17T10:47:30.036612+0000","last_unstale":"2024-04-17T10:47:31.058051+0000","last_undegraded":"2024-04-17T10:47:31.058051+0000","last_fullsized":"2024-04-17T10:47:31.058051+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:29.006354+0000","last_clean_scrub_stamp":"2024-04-17T10:47:29.006354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:16:05.238250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.027029+0000","last_change":"2024-04-17T10:47:29.031551+0000","last_active":"2024-04-17T10:47:31.027029+0000","last_peered":"2024-04-17T10:47:31.027029+0000","last_clean":"2024-04-17T10:47:31.027029+0000","last_became_active":"2024-04-17T10:47:29.030845+0000","last_became_peered":"2024-04-17T10:47:29.030845+0000","last_unstale":"2024-04-17T10:47:31.027029+0000","last_undegraded":"2024-04-17T10:47:31.027029+0000","last_fullsized":"2024-04-17T10:47:31.027029+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:47:28.001515+0000","last_clean_scrub_stamp":"2024-04-17T10:47:28.001515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:23:55.358167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":340,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-17T10:47:31.027662+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:47:31.027662+0000","last_peered":"2024-04-17T10:47:31.027662+0000","last_clean":"2024-04-17T10:47:31.027662+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:47:31.027662+0000","last_undegraded":"2024-04-17T10:47:31.027662+0000","last_fullsized":"2024-04-17T10:47:31.027662+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574967,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59388,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312452,"statfs":{"total":96636764160,"available":96575950848,"internally_reserved":0,"allocated":1875968,"data_stored":1143805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574966,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64376,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307464,"statfs":{"total":96636764160,"available":96570843136,"internally_reserved":0,"allocated":1282048,"data_stored":553977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574967,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50476,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321364,"statfs":{"total":96636764160,"available":96585076736,"internally_reserved":0,"allocated":1277952,"data_stored":553395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574966,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68544,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303296,"statfs":{"total":96636764160,"available":96566575104,"internally_reserved":0,"allocated":1294336,"data_stored":555061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574968,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68544,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303296,"statfs":{"total":96636764160,"available":96566575104,"internally_reserved":0,"allocated":1290240,"data_stored":554959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574966,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58800,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313040,"statfs":{"total":96636764160,"available":96576552960,"internally_reserved":0,"allocated":1277952,"data_stored":553359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574966,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94892,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94276948,"statfs":{"total":96636764160,"available":96539594752,"internally_reserved":0,"allocated":1273856,"data_stored":553325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574966,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59380,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312460,"statfs":{"total":96636764160,"available":96575959040,"internally_reserved":0,"allocated":1871872,"data_stored":1144275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:47:37.860 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-17T10:47:37.860 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-17T10:47:37.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.990+0000 7f5928e90640 1 Processor -- start 2024-04-17T10:47:37.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.990+0000 7f5928e90640 1 -- start start 2024-04-17T10:47:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.990+0000 7f5928e90640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f59240b5b60 0x7f59240b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.990+0000 7f5928e90640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f592415e4d0 con 0x7f592415e0f0 2024-04-17T10:47:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.990+0000 7f5928e90640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5924162580 con 0x7f59240ce400 2024-04-17T10:47:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.990+0000 7f5928e90640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5924162b40 con 0x7f59240b5b60 2024-04-17T10:47:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.990+0000 7f592259c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f592415e0f0 0x7f5924161c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60156/0 (socket says 172.21.15.83:60156) 2024-04-17T10:47:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.990+0000 7f592259c640 1 -- 172.21.15.83:0/83397741 learned_addr learned my addr 172.21.15.83:0/83397741 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.990+0000 7f592159a640 1 -- 172.21.15.83:0/83397741 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2190093199 0 0) 0x7f592415e4d0 con 0x7f592415e0f0 2024-04-17T10:47:37.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.990+0000 7f592159a640 1 -- 172.21.15.83:0/83397741 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5910003530 con 0x7f592415e0f0 2024-04-17T10:47:37.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.990+0000 7f5922d9d640 1 --2- 172.21.15.83:0/83397741 >> v2:172.21.15.167:3300/0 conn(0x7f59240b5b60 0x7f59240b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.990+0000 7f592159a640 1 -- 172.21.15.83:0/83397741 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2644742377 0 0) 0x7f5910003530 con 0x7f592415e0f0 2024-04-17T10:47:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.990+0000 7f592159a640 1 -- 172.21.15.83:0/83397741 >> v2:172.21.15.167:3300/0 conn(0x7f59240b5b60 msgr2=0x7f59240b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.990+0000 7f592159a640 1 --2- 172.21.15.83:0/83397741 >> v2:172.21.15.167:3300/0 conn(0x7f59240b5b60 0x7f59240b6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.990+0000 7f592159a640 1 -- 172.21.15.83:0/83397741 >> v1:172.21.15.167:6789/0 conn(0x7f59240ce400 legacy=0x7f59240b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.990+0000 7f592159a640 1 -- 172.21.15.83:0/83397741 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5924162860 con 0x7f592415e0f0 2024-04-17T10:47:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.990+0000 7f592159a640 1 -- 172.21.15.83:0/83397741 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f5914001a00 con 0x7f592415e0f0 2024-04-17T10:47:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.990+0000 7f592159a640 1 -- 172.21.15.83:0/83397741 >> v1:172.21.15.83:6789/0 conn(0x7f592415e0f0 legacy=0x7f5924161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.990+0000 7f592159a640 1 --2- 172.21.15.83:0/83397741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5910003a30 0x7f5910003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.990+0000 7f592159a640 1 --2- 172.21.15.83:0/83397741 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5910004810 0x7f5910004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.990+0000 7f592159a640 1 --2- 172.21.15.83:0/83397741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5910005240 0x7f591000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.990+0000 7f592159a640 1 -- 172.21.15.83:0/83397741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5924162b40 con 0x7f5910005240 2024-04-17T10:47:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.990+0000 7f592159a640 1 -- 172.21.15.83:0/83397741 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591000c1a0 con 0x7f5910003a30 2024-04-17T10:47:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.990+0000 7f592159a640 1 -- 172.21.15.83:0/83397741 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591000c3a0 con 0x7f5910004810 2024-04-17T10:47:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.990+0000 7f5922d9d640 1 --2- 172.21.15.83:0/83397741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5910005240 0x7f591000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.990+0000 7f592259c640 1 --2- 172.21.15.83:0/83397741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5910003a30 0x7f5910003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.990+0000 7f5921d9b640 1 --2- 172.21.15.83:0/83397741 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5910004810 0x7f5910004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.990+0000 7f5922d9d640 1 -- 172.21.15.83:0/83397741 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5910004810 msgr2=0x7f5910004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.990+0000 7f5922d9d640 1 --2- 172.21.15.83:0/83397741 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5910004810 0x7f5910004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:37.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.990+0000 7f5922d9d640 1 -- 172.21.15.83:0/83397741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5910003a30 msgr2=0x7f5910003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:37.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.990+0000 7f5922d9d640 1 --2- 172.21.15.83:0/83397741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5910003a30 0x7f5910003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:37.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.990+0000 7f5922d9d640 1 -- 172.21.15.83:0/83397741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5910003530 con 0x7f5910005240 2024-04-17T10:47:37.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.994+0000 7f5922d9d640 1 -- 172.21.15.83:0/83397741 reap_dead start 2024-04-17T10:47:37.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.994+0000 7f5922d9d640 1 --2- 172.21.15.83:0/83397741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5910005240 0x7f591000b9d0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f59240b6430 tx=0x7f591c01c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c487212575091f6b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:37.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.994+0000 7f592159a640 1 -- 172.21.15.83:0/83397741 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c0096b0 con 0x7f5910005240 2024-04-17T10:47:37.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.994+0000 7f592159a640 1 -- 172.21.15.83:0/83397741 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f591c004020 con 0x7f5910005240 2024-04-17T10:47:37.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.994+0000 7f592159a640 1 -- 172.21.15.83:0/83397741 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c004410 con 0x7f5910005240 2024-04-17T10:47:37.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.994+0000 7f5928e90640 1 -- 172.21.15.83:0/83397741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5910005240 msgr2=0x7f591000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:37.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.994+0000 7f5928e90640 1 --2- 172.21.15.83:0/83397741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5910005240 0x7f591000b9d0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f59240b6430 tx=0x7f591c01c990 comp rx=0 tx=0).stop 2024-04-17T10:47:37.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.994+0000 7f5928e90640 1 -- 172.21.15.83:0/83397741 shutdown_connections 2024-04-17T10:47:37.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.994+0000 7f5928e90640 1 --2- 172.21.15.83:0/83397741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5910005240 0x7f591000b9d0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:37.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.994+0000 7f5928e90640 1 -- 172.21.15.83:0/83397741 >> 172.21.15.83:0/83397741 conn(0x7f59240d0450 msgr2=0x7f59240d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:37.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.994+0000 7f5928e90640 1 -- 172.21.15.83:0/83397741 shutdown_connections 2024-04-17T10:47:37.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.994+0000 7f5928e90640 1 -- 172.21.15.83:0/83397741 wait complete. 2024-04-17T10:47:37.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.998+0000 7f5928e90640 1 Processor -- start 2024-04-17T10:47:37.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.998+0000 7f5928e90640 1 -- start start 2024-04-17T10:47:37.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.998+0000 7f5928e90640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5910005240 0x7f59240cd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:37.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.998+0000 7f5928e90640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59240b85f0 0x7f59240c9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.998+0000 7f5928e90640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59240b8a70 0x7f59240c9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.998+0000 7f5928e90640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59240b6d00 con 0x7f59240b8a70 2024-04-17T10:47:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.998+0000 7f5928e90640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59240d2020 con 0x7f5910005240 2024-04-17T10:47:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.998+0000 7f5928e90640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59240b6e80 con 0x7f59240b85f0 2024-04-17T10:47:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.998+0000 7f5922d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59240b8a70 0x7f59240c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.998+0000 7f5922d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59240b8a70 0x7f59240c9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34822/0 (socket says 172.21.15.83:34822) 2024-04-17T10:47:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.998+0000 7f5922d9d640 1 -- 172.21.15.83:0/3060187829 learned_addr learned my addr 172.21.15.83:0/3060187829 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.998+0000 7f592259c640 1 --2- 172.21.15.83:0/3060187829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5910005240 0x7f59240cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.998+0000 7f5921d9b640 1 --2- 172.21.15.83:0/3060187829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59240b85f0 0x7f59240c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.998+0000 7f5922d9d640 1 -- 172.21.15.83:0/3060187829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59240b85f0 msgr2=0x7f59240c9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.998+0000 7f5922d9d640 1 --2- 172.21.15.83:0/3060187829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59240b85f0 0x7f59240c9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.998+0000 7f5922d9d640 1 -- 172.21.15.83:0/3060187829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5910005240 msgr2=0x7f59240cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.998+0000 7f5922d9d640 1 --2- 172.21.15.83:0/3060187829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5910005240 0x7f59240cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.998+0000 7f5922d9d640 1 -- 172.21.15.83:0/3060187829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592415eb30 con 0x7f59240b8a70 2024-04-17T10:47:37.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.998+0000 7f5922d9d640 1 --2- 172.21.15.83:0/3060187829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59240b8a70 0x7f59240c9680 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f591c01d230 tx=0x7f591c019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:37.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.998+0000 7f591affd640 1 -- 172.21.15.83:0/3060187829 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c0b8070 con 0x7f59240b8a70 2024-04-17T10:47:37.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.998+0000 7f591affd640 1 -- 172.21.15.83:0/3060187829 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f591c0af070 con 0x7f59240b8a70 2024-04-17T10:47:37.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.998+0000 7f5928e90640 1 -- 172.21.15.83:0/3060187829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592415ee10 con 0x7f59240b8a70 2024-04-17T10:47:37.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.998+0000 7f591affd640 1 -- 172.21.15.83:0/3060187829 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c003d10 con 0x7f59240b8a70 2024-04-17T10:47:37.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.998+0000 7f5928e90640 1 -- 172.21.15.83:0/3060187829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592415f3d0 con 0x7f59240b8a70 2024-04-17T10:47:37.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.998+0000 7f5928e90640 1 -- 172.21.15.83:0/3060187829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59240681a0 con 0x7f59240b8a70 2024-04-17T10:47:37.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:37.998+0000 7f591affd640 1 -- 172.21.15.83:0/3060187829 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f591c0b1070 con 0x7f59240b8a70 2024-04-17T10:47:37.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.002+0000 7f591affd640 1 --2- 172.21.15.83:0/3060187829 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5904066f60 0x7f5904069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:37.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.002+0000 7f591affd640 1 -- 172.21.15.83:0/3060187829 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f591c11de50 con 0x7f59240b8a70 2024-04-17T10:47:37.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.002+0000 7f592259c640 1 --2- 172.21.15.83:0/3060187829 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5904066f60 0x7f5904069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:37.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.002+0000 7f592259c640 1 --2- 172.21.15.83:0/3060187829 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5904066f60 0x7f5904069420 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5914001340 tx=0x7f5914009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:38.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.006+0000 7f591affd640 1 -- 172.21.15.83:0/3060187829 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f591c0166a0 con 0x7f59240b8a70 2024-04-17T10:47:38.179 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:47:38.181 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:47:38.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.214+0000 7f5928e90640 1 -- 172.21.15.83:0/3060187829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f5924074f90 con 0x7f59240b8a70 2024-04-17T10:47:38.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.218+0000 7f591affd640 1 -- 172.21.15.83:0/3060187829 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7f591c0eee70 con 0x7f59240b8a70 2024-04-17T10:47:38.214 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-04-17T10:47:38.214 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBUph9mHGmXGBAAZF+JiAG9YrFtbW+dCy2IEQ== 2024-04-17T10:47:38.214 INFO:teuthology.orchestra.run.smithi083.stdout: caps mds = "allow" 2024-04-17T10:47:38.214 INFO:teuthology.orchestra.run.smithi083.stdout: caps mon = "allow r" 2024-04-17T10:47:38.214 INFO:teuthology.orchestra.run.smithi083.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-17T10:47:38.215 INFO:teuthology.orchestra.run.smithi083.stderr:updated caps for client.0 2024-04-17T10:47:38.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.218+0000 7f5928e90640 1 -- 172.21.15.83:0/3060187829 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5904066f60 msgr2=0x7f5904069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:38.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.218+0000 7f5928e90640 1 --2- 172.21.15.83:0/3060187829 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5904066f60 0x7f5904069420 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5914001340 tx=0x7f5914009040 comp rx=0 tx=0).stop 2024-04-17T10:47:38.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.218+0000 7f5928e90640 1 -- 172.21.15.83:0/3060187829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59240b8a70 msgr2=0x7f59240c9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:38.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.218+0000 7f5928e90640 1 --2- 172.21.15.83:0/3060187829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59240b8a70 0x7f59240c9680 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f591c01d230 tx=0x7f591c019730 comp rx=0 tx=0).stop 2024-04-17T10:47:38.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.218+0000 7f5928e90640 1 -- 172.21.15.83:0/3060187829 shutdown_connections 2024-04-17T10:47:38.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.218+0000 7f5928e90640 1 --2- 172.21.15.83:0/3060187829 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5904066f60 0x7f5904069420 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:38.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.218+0000 7f5928e90640 1 --2- 172.21.15.83:0/3060187829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59240b8a70 0x7f59240c9680 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:38.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.218+0000 7f5928e90640 1 --2- 172.21.15.83:0/3060187829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59240b85f0 0x7f59240c9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:38.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.218+0000 7f5928e90640 1 --2- 172.21.15.83:0/3060187829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5910005240 0x7f59240cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:38.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.218+0000 7f5928e90640 1 -- 172.21.15.83:0/3060187829 >> 172.21.15.83:0/3060187829 conn(0x7f59240d0450 msgr2=0x7f592415c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:38.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.218+0000 7f5928e90640 1 -- 172.21.15.83:0/3060187829 shutdown_connections 2024-04-17T10:47:38.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.218+0000 7f5928e90640 1 -- 172.21.15.83:0/3060187829 wait complete. 2024-04-17T10:47:38.232 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:47:38.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.438+0000 7f6f479a0640 1 Processor -- start 2024-04-17T10:47:38.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.438+0000 7f6f479a0640 1 -- start start 2024-04-17T10:47:38.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.438+0000 7f6f479a0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6f40072330 0x7f6f400cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:38.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.438+0000 7f6f479a0640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f400cfea0 con 0x7f6f4015ba30 2024-04-17T10:47:38.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.438+0000 7f6f479a0640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f4006e320 con 0x7f6f400cfa10 2024-04-17T10:47:38.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.438+0000 7f6f479a0640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f40071ea0 con 0x7f6f40072330 2024-04-17T10:47:38.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.438+0000 7f6f4573c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6f4015ba30 0x7f6f4015c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60162/0 (socket says 172.21.15.83:60162) 2024-04-17T10:47:38.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.438+0000 7f6f4573c640 1 -- 172.21.15.83:0/1527671647 learned_addr learned my addr 172.21.15.83:0/1527671647 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:38.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.438+0000 7f6f44f3b640 1 --2- 172.21.15.83:0/1527671647 >> v2:172.21.15.83:3300/0 conn(0x7f6f40072330 0x7f6f400cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:38.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.438+0000 7f6f37fff640 1 -- 172.21.15.83:0/1527671647 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 910709334 0 0) 0x7f6f400cfea0 con 0x7f6f4015ba30 2024-04-17T10:47:38.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.438+0000 7f6f37fff640 1 -- 172.21.15.83:0/1527671647 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f2c003530 con 0x7f6f4015ba30 2024-04-17T10:47:38.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.438+0000 7f6f44f3b640 1 -- 172.21.15.83:0/1527671647 >> v1:172.21.15.167:6789/0 conn(0x7f6f400cfa10 legacy=0x7f6f4006c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:38.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.438+0000 7f6f37fff640 1 -- 172.21.15.83:0/1527671647 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 588704967 0 0) 0x7f6f2c003530 con 0x7f6f4015ba30 2024-04-17T10:47:38.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.438+0000 7f6f44f3b640 1 -- 172.21.15.83:0/1527671647 >> v1:172.21.15.83:6789/0 conn(0x7f6f4015ba30 legacy=0x7f6f4015c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:38.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.438+0000 7f6f44f3b640 1 -- 172.21.15.83:0/1527671647 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f4006e040 con 0x7f6f40072330 2024-04-17T10:47:38.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f44f3b640 1 --2- 172.21.15.83:0/1527671647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f40072330 0x7f6f400cf4d0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f6f300097c0 tx=0x7f6f3001c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d987db7fe69b629f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:38.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f37fff640 1 -- 172.21.15.83:0/1527671647 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f300b0070 con 0x7f6f40072330 2024-04-17T10:47:38.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f37fff640 1 -- 172.21.15.83:0/1527671647 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f300a4e50 con 0x7f6f40072330 2024-04-17T10:47:38.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f37fff640 1 -- 172.21.15.83:0/1527671647 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c001680 con 0x7f6f40072330 2024-04-17T10:47:38.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f479a0640 1 -- 172.21.15.83:0/1527671647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f40072330 msgr2=0x7f6f400cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:38.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f479a0640 1 --2- 172.21.15.83:0/1527671647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f40072330 0x7f6f400cf4d0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f6f300097c0 tx=0x7f6f3001c330 comp rx=0 tx=0).stop 2024-04-17T10:47:38.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f479a0640 1 -- 172.21.15.83:0/1527671647 shutdown_connections 2024-04-17T10:47:38.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f479a0640 1 --2- 172.21.15.83:0/1527671647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f40072330 0x7f6f400cf4d0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:38.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f479a0640 1 -- 172.21.15.83:0/1527671647 >> 172.21.15.83:0/1527671647 conn(0x7f6f4006abe0 msgr2=0x7f6f4006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:38.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f479a0640 1 -- 172.21.15.83:0/1527671647 shutdown_connections 2024-04-17T10:47:38.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f479a0640 1 -- 172.21.15.83:0/1527671647 wait complete. 2024-04-17T10:47:38.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f479a0640 1 Processor -- start 2024-04-17T10:47:38.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f479a0640 1 -- start start 2024-04-17T10:47:38.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f479a0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f40072330 0x7f6f400b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:38.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f479a0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f400cfa10 0x7f6f40142100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:38.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f479a0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f4015ba30 0x7f6f40150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:38.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f479a0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4006de20 con 0x7f6f4015ba30 2024-04-17T10:47:38.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f479a0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4006dca0 con 0x7f6f40072330 2024-04-17T10:47:38.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f479a0640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f40169530 con 0x7f6f400cfa10 2024-04-17T10:47:38.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f45f3d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f4015ba30 0x7f6f40150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:38.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f45f3d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f4015ba30 0x7f6f40150190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34838/0 (socket says 172.21.15.83:34838) 2024-04-17T10:47:38.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f45f3d640 1 -- 172.21.15.83:0/1822107990 learned_addr learned my addr 172.21.15.83:0/1822107990 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:38.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f4573c640 1 --2- 172.21.15.83:0/1822107990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f40072330 0x7f6f400b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:38.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f44f3b640 1 --2- 172.21.15.83:0/1822107990 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f400cfa10 0x7f6f40142100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:38.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f45f3d640 1 -- 172.21.15.83:0/1822107990 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f400cfa10 msgr2=0x7f6f40142100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:38.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f45f3d640 1 --2- 172.21.15.83:0/1822107990 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f400cfa10 0x7f6f40142100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:38.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f45f3d640 1 -- 172.21.15.83:0/1822107990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f40072330 msgr2=0x7f6f400b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:38.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f45f3d640 1 --2- 172.21.15.83:0/1822107990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f40072330 0x7f6f400b37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:38.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f45f3d640 1 -- 172.21.15.83:0/1822107990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f400b2710 con 0x7f6f4015ba30 2024-04-17T10:47:38.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f4573c640 1 --2- 172.21.15.83:0/1822107990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f40072330 0x7f6f400b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:38.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f45f3d640 1 --2- 172.21.15.83:0/1822107990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f4015ba30 0x7f6f40150190 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800c770 tx=0x7f6f3800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:38.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f35ffb640 1 -- 172.21.15.83:0/1822107990 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f38003b80 con 0x7f6f4015ba30 2024-04-17T10:47:38.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f35ffb640 1 -- 172.21.15.83:0/1822107990 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f38003d20 con 0x7f6f4015ba30 2024-04-17T10:47:38.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f479a0640 1 -- 172.21.15.83:0/1822107990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f400b29f0 con 0x7f6f4015ba30 2024-04-17T10:47:38.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.442+0000 7f6f35ffb640 1 -- 172.21.15.83:0/1822107990 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3800aa90 con 0x7f6f4015ba30 2024-04-17T10:47:38.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.446+0000 7f6f479a0640 1 -- 172.21.15.83:0/1822107990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f400b2fb0 con 0x7f6f4015ba30 2024-04-17T10:47:38.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.446+0000 7f6f479a0640 1 -- 172.21.15.83:0/1822107990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f40074f90 con 0x7f6f4015ba30 2024-04-17T10:47:38.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.446+0000 7f6f35ffb640 1 -- 172.21.15.83:0/1822107990 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6f3800ac30 con 0x7f6f4015ba30 2024-04-17T10:47:38.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.446+0000 7f6f35ffb640 1 --2- 172.21.15.83:0/1822107990 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6f24066f60 0x7f6f24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:38.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.446+0000 7f6f4573c640 1 --2- 172.21.15.83:0/1822107990 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6f24066f60 0x7f6f24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:38.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.446+0000 7f6f35ffb640 1 -- 172.21.15.83:0/1822107990 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f38084fb0 con 0x7f6f4015ba30 2024-04-17T10:47:38.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.446+0000 7f6f4573c640 1 --2- 172.21.15.83:0/1822107990 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6f24066f60 0x7f6f24069420 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c001ec0 tx=0x7f6f3c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:38.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.450+0000 7f6f35ffb640 1 -- 172.21.15.83:0/1822107990 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6f38055fc0 con 0x7f6f4015ba30 2024-04-17T10:47:38.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.646+0000 7f6f479a0640 1 -- 172.21.15.83:0/1822107990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f400cbb60 con 0x7f6f4015ba30 2024-04-17T10:47:38.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.646+0000 7f6f35ffb640 1 -- 172.21.15.83:0/1822107990 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f6f380558c0 con 0x7f6f4015ba30 2024-04-17T10:47:38.644 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:47:38.644 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/437268957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":437268957},{"type":"v1","addr":"172.21.15.167:6833","nonce":437268957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":6253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3214640674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3214640674},{"type":"v1","addr":"172.21.15.167:6835","nonce":3214640674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":8046,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/715498323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":715498323},{"type":"v1","addr":"172.21.15.83:6835","nonce":715498323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:47:29.385103+0000","modified":"2024-04-17T10:47:31.389088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8049":{"gid":8049,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2353406669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":2353406669},{"type":"v1","addr":"172.21.15.83:6837","nonce":2353406669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-17T10:47:38.644 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 62 2024-04-17T10:47:38.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.650+0000 7f6f479a0640 1 -- 172.21.15.83:0/1822107990 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6f24066f60 msgr2=0x7f6f24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:38.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.650+0000 7f6f479a0640 1 --2- 172.21.15.83:0/1822107990 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6f24066f60 0x7f6f24069420 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c001ec0 tx=0x7f6f3c009040 comp rx=0 tx=0).stop 2024-04-17T10:47:38.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.650+0000 7f6f479a0640 1 -- 172.21.15.83:0/1822107990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f4015ba30 msgr2=0x7f6f40150190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:38.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.650+0000 7f6f479a0640 1 --2- 172.21.15.83:0/1822107990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f4015ba30 0x7f6f40150190 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800c770 tx=0x7f6f3800cc30 comp rx=0 tx=0).stop 2024-04-17T10:47:38.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.650+0000 7f6f479a0640 1 -- 172.21.15.83:0/1822107990 shutdown_connections 2024-04-17T10:47:38.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.650+0000 7f6f479a0640 1 --2- 172.21.15.83:0/1822107990 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6f24066f60 0x7f6f24069420 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:38.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.650+0000 7f6f479a0640 1 --2- 172.21.15.83:0/1822107990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f4015ba30 0x7f6f40150190 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:38.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.650+0000 7f6f479a0640 1 --2- 172.21.15.83:0/1822107990 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f400cfa10 0x7f6f40142100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:38.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.650+0000 7f6f479a0640 1 --2- 172.21.15.83:0/1822107990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f40072330 0x7f6f400b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:38.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.650+0000 7f6f479a0640 1 -- 172.21.15.83:0/1822107990 >> 172.21.15.83:0/1822107990 conn(0x7f6f4006abe0 msgr2=0x7f6f40070470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:38.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.650+0000 7f6f479a0640 1 -- 172.21.15.83:0/1822107990 shutdown_connections 2024-04-17T10:47:38.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:38.650+0000 7f6f479a0640 1 -- 172.21.15.83:0/1822107990 wait complete. 2024-04-17T10:47:39.066 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T10:47:29.385103+0000', 'modified': '2024-04-17T10:47:31.389088+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8049}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8049': {'gid': 8049, 'name': 'a', 'rank': 0, 'incarnation': 60, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6837/2353406669', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6836', 'nonce': 2353406669}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 2353406669}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:47:39.067 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T10:47:39.067 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-17T10:47:39.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.202+0000 7fd56acfd640 1 Processor -- start 2024-04-17T10:47:39.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.202+0000 7fd56acfd640 1 -- start start 2024-04-17T10:47:39.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.202+0000 7fd56acfd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd564154670 0x7fd5640aea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:39.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.202+0000 7fd56acfd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd56406cc80 0x7fd564070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:39.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.202+0000 7fd56acfd640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5640b2ff0 con 0x7fd5640b4090 2024-04-17T10:47:39.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.202+0000 7fd56acfd640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd564154a50 con 0x7fd564154670 2024-04-17T10:47:39.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.202+0000 7fd56acfd640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5640b32d0 con 0x7fd56406cc80 2024-04-17T10:47:39.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.202+0000 7fd568a99640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd564154670 0x7fd5640aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:39.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.202+0000 7fd563fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd5640b4090 0x7fd56406c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60172/0 (socket says 172.21.15.83:60172) 2024-04-17T10:47:39.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.202+0000 7fd563fff640 1 -- 172.21.15.83:0/3912692583 learned_addr learned my addr 172.21.15.83:0/3912692583 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:39.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.202+0000 7fd56929a640 1 --2- 172.21.15.83:0/3912692583 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd56406cc80 0x7fd564070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:39.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.202+0000 7fd568a99640 1 -- 172.21.15.83:0/3912692583 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd56406cc80 msgr2=0x7fd564070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:39.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.202+0000 7fd5637fe640 1 -- 172.21.15.83:0/3912692583 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 486220063 0 0) 0x7fd5640b2ff0 con 0x7fd5640b4090 2024-04-17T10:47:39.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.202+0000 7fd568a99640 1 --2- 172.21.15.83:0/3912692583 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd56406cc80 0x7fd564070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.202+0000 7fd568a99640 1 -- 172.21.15.83:0/3912692583 >> v1:172.21.15.83:6789/0 conn(0x7fd5640b4090 legacy=0x7fd56406c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:39.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.202+0000 7fd568a99640 1 -- 172.21.15.83:0/3912692583 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5640b35b0 con 0x7fd564154670 2024-04-17T10:47:39.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.202+0000 7fd56929a640 1 --2- 172.21.15.83:0/3912692583 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd56406cc80 0x7fd564070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:39.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.206+0000 7fd568a99640 1 --2- 172.21.15.83:0/3912692583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd564154670 0x7fd5640aea50 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fd5580097c0 tx=0x7fd55801d580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8831c0d0d6807c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:39.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.206+0000 7fd5637fe640 1 -- 172.21.15.83:0/3912692583 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5580aa630 con 0x7fd564154670 2024-04-17T10:47:39.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.206+0000 7fd5637fe640 1 -- 172.21.15.83:0/3912692583 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5580aa7d0 con 0x7fd564154670 2024-04-17T10:47:39.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.206+0000 7fd5637fe640 1 -- 172.21.15.83:0/3912692583 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5580aaba0 con 0x7fd564154670 2024-04-17T10:47:39.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.206+0000 7fd56acfd640 1 -- 172.21.15.83:0/3912692583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd564154670 msgr2=0x7fd5640aea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:39.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.206+0000 7fd56acfd640 1 --2- 172.21.15.83:0/3912692583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd564154670 0x7fd5640aea50 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fd5580097c0 tx=0x7fd55801d580 comp rx=0 tx=0).stop 2024-04-17T10:47:39.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.206+0000 7fd56acfd640 1 -- 172.21.15.83:0/3912692583 shutdown_connections 2024-04-17T10:47:39.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.206+0000 7fd56acfd640 1 --2- 172.21.15.83:0/3912692583 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd56406cc80 0x7fd564070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.206+0000 7fd56acfd640 1 --2- 172.21.15.83:0/3912692583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd564154670 0x7fd5640aea50 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.206+0000 7fd56acfd640 1 -- 172.21.15.83:0/3912692583 >> 172.21.15.83:0/3912692583 conn(0x7fd5640b15b0 msgr2=0x7fd5640b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.206+0000 7fd56acfd640 1 -- 172.21.15.83:0/3912692583 shutdown_connections 2024-04-17T10:47:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.206+0000 7fd56acfd640 1 -- 172.21.15.83:0/3912692583 wait complete. 2024-04-17T10:47:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.206+0000 7fd56acfd640 1 Processor -- start 2024-04-17T10:47:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.206+0000 7fd56acfd640 1 -- start start 2024-04-17T10:47:39.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.210+0000 7fd56acfd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd56406cc80 0x7fd56415c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:39.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.210+0000 7fd56acfd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5640b4090 0x7fd564146200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:39.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.210+0000 7fd56acfd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd564154670 0x7fd564146740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:39.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.210+0000 7fd56acfd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5640aef90 con 0x7fd56406cc80 2024-04-17T10:47:39.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.210+0000 7fd56acfd640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56415b3c0 con 0x7fd564154670 2024-04-17T10:47:39.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.210+0000 7fd56acfd640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5640af110 con 0x7fd5640b4090 2024-04-17T10:47:39.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.210+0000 7fd568a99640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd56406cc80 0x7fd56415c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:39.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.210+0000 7fd568a99640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd56406cc80 0x7fd56415c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34862/0 (socket says 172.21.15.83:34862) 2024-04-17T10:47:39.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.210+0000 7fd568a99640 1 -- 172.21.15.83:0/1108473601 learned_addr learned my addr 172.21.15.83:0/1108473601 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:39.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.210+0000 7fd563fff640 1 --2- 172.21.15.83:0/1108473601 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5640b4090 0x7fd564146200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:39.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.210+0000 7fd568a99640 1 -- 172.21.15.83:0/1108473601 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5640b4090 msgr2=0x7fd564146200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:39.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.210+0000 7fd568a99640 1 --2- 172.21.15.83:0/1108473601 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5640b4090 0x7fd564146200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.210+0000 7fd568a99640 1 -- 172.21.15.83:0/1108473601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd564154670 msgr2=0x7fd564146740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:39.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.210+0000 7fd56929a640 1 --2- 172.21.15.83:0/1108473601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd564154670 0x7fd564146740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:39.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.210+0000 7fd568a99640 1 --2- 172.21.15.83:0/1108473601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd564154670 0x7fd564146740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.210+0000 7fd568a99640 1 -- 172.21.15.83:0/1108473601 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5641f4a60 con 0x7fd56406cc80 2024-04-17T10:47:39.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.210+0000 7fd563fff640 1 --2- 172.21.15.83:0/1108473601 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5640b4090 0x7fd564146200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:39.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.210+0000 7fd568a99640 1 --2- 172.21.15.83:0/1108473601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd56406cc80 0x7fd56415c640 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fd55801b1b0 tx=0x7fd55801bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:39.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.210+0000 7fd5617fa640 1 -- 172.21.15.83:0/1108473601 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5580bf070 con 0x7fd56406cc80 2024-04-17T10:47:39.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.210+0000 7fd5617fa640 1 -- 172.21.15.83:0/1108473601 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5580b2820 con 0x7fd56406cc80 2024-04-17T10:47:39.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.210+0000 7fd56acfd640 1 -- 172.21.15.83:0/1108473601 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5641f4d40 con 0x7fd56406cc80 2024-04-17T10:47:39.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.210+0000 7fd5617fa640 1 -- 172.21.15.83:0/1108473601 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5580b2c10 con 0x7fd56406cc80 2024-04-17T10:47:39.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.210+0000 7fd56acfd640 1 -- 172.21.15.83:0/1108473601 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5641f5300 con 0x7fd56406cc80 2024-04-17T10:47:39.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.210+0000 7fd5617fa640 1 -- 172.21.15.83:0/1108473601 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd5580c2070 con 0x7fd56406cc80 2024-04-17T10:47:39.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.210+0000 7fd5617fa640 1 --2- 172.21.15.83:0/1108473601 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd548066f60 0x7fd548069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:39.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.210+0000 7fd5617fa640 1 -- 172.21.15.83:0/1108473601 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd55811f620 con 0x7fd56406cc80 2024-04-17T10:47:39.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.210+0000 7fd563fff640 1 --2- 172.21.15.83:0/1108473601 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd548066f60 0x7fd548069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:39.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.214+0000 7fd563fff640 1 --2- 172.21.15.83:0/1108473601 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd548066f60 0x7fd548069420 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd56415d6c0 tx=0x7fd550009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:39.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.214+0000 7fd56de6a000 1 Processor -- start 2024-04-17T10:47:39.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.214+0000 7fd56de6a000 1 -- start start 2024-04-17T10:47:39.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.214+0000 7fd56de6a000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f0503a920 0x561f0503ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:39.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.214+0000 7fd56de6a000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561f0503b240 0x561f04ff09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:39.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.214+0000 7fd56de6a000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f04ff0f30 0x561f04ff1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:39.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.214+0000 7fd56de6a000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f04ed85f0 con 0x561f0503a920 2024-04-17T10:47:39.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.214+0000 7fd56de6a000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f04e3fda0 con 0x561f04ff0f30 2024-04-17T10:47:39.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.214+0000 7fd56de6a000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x561f04dc8150 con 0x561f0503b240 2024-04-17T10:47:39.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.214+0000 7fd568a99640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f0503a920 0x561f0503ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:39.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.214+0000 7fd568a99640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f0503a920 0x561f0503ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34870/0 (socket says 172.21.15.83:34870) 2024-04-17T10:47:39.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.214+0000 7fd568a99640 1 -- 172.21.15.83:0/3229867863 learned_addr learned my addr 172.21.15.83:0/3229867863 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:39.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.214+0000 7fd568a99640 1 -- 172.21.15.83:0/3229867863 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561f0503b240 msgr2=0x561f04ff09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:39.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.214+0000 7fd568a99640 1 --2- 172.21.15.83:0/3229867863 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561f0503b240 0x561f04ff09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.214+0000 7fd568a99640 1 -- 172.21.15.83:0/3229867863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f04ff0f30 msgr2=0x561f04ff1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:39.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.214+0000 7fd568a99640 1 --2- 172.21.15.83:0/3229867863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f04ff0f30 0x561f04ff1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.214+0000 7fd568a99640 1 -- 172.21.15.83:0/3229867863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f04de1ca0 con 0x561f0503a920 2024-04-17T10:47:39.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.214+0000 7fd568a99640 1 --2- 172.21.15.83:0/3229867863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f0503a920 0x561f0503ad00 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fd5580f81a0 tx=0x7fd5580ef920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942480fa008f99c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:39.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.214+0000 7fd54e7fc640 1 -- 172.21.15.83:0/3229867863 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5580f4110 con 0x561f0503a920 2024-04-17T10:47:39.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.214+0000 7fd54e7fc640 1 -- 172.21.15.83:0/3229867863 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5580f86d0 con 0x561f0503a920 2024-04-17T10:47:39.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.214+0000 7fd54e7fc640 1 -- 172.21.15.83:0/3229867863 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5580ee2c0 con 0x561f0503a920 2024-04-17T10:47:39.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.214+0000 7fd56de6a000 1 -- 172.21.15.83:0/3229867863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f0503a920 msgr2=0x561f0503ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:39.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.214+0000 7fd56de6a000 1 --2- 172.21.15.83:0/3229867863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f0503a920 0x561f0503ad00 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fd5580f81a0 tx=0x7fd5580ef920 comp rx=0 tx=0).stop 2024-04-17T10:47:39.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.214+0000 7fd56de6a000 1 -- 172.21.15.83:0/3229867863 shutdown_connections 2024-04-17T10:47:39.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.214+0000 7fd56de6a000 1 --2- 172.21.15.83:0/3229867863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f04ff0f30 0x561f04ff1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.214+0000 7fd56de6a000 1 --2- 172.21.15.83:0/3229867863 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561f0503b240 0x561f04ff09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.214+0000 7fd56de6a000 1 --2- 172.21.15.83:0/3229867863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f0503a920 0x561f0503ad00 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.214+0000 7fd56de6a000 1 -- 172.21.15.83:0/3229867863 >> 172.21.15.83:0/3229867863 conn(0x561f04ff72a0 msgr2=0x561f04fef2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:39.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.214+0000 7fd56de6a000 1 -- 172.21.15.83:0/3229867863 shutdown_connections 2024-04-17T10:47:39.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.214+0000 7fd56de6a000 1 -- 172.21.15.83:0/3229867863 wait complete. 2024-04-17T10:47:39.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.214+0000 7fd56de6a000 1 -- start start 2024-04-17T10:47:39.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.214+0000 7fd56de6a000 1 Processor -- start 2024-04-17T10:47:39.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd56de6a000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f04ff0f30 0x561f04ff4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:39.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd56de6a000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561f0503a920 0x561f04ff53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:39.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd56de6a000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f0503b240 0x561f04ff5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:39.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd56de6a000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f04dfa990 con 0x561f04ff0f30 2024-04-17T10:47:39.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd56de6a000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f04e3be40 con 0x561f0503b240 2024-04-17T10:47:39.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd56de6a000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x561f04dcb900 con 0x561f0503a920 2024-04-17T10:47:39.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd56de6a000 10 client.0 _ll_register_callbacks cb 0x561f04ef31c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:47:39.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd568a99640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f04ff0f30 0x561f04ff4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:39.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd568a99640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f04ff0f30 0x561f04ff4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34884/0 (socket says 172.21.15.83:34884) 2024-04-17T10:47:39.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd568a99640 1 -- 172.21.15.83:0/2008796173 learned_addr learned my addr 172.21.15.83:0/2008796173 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:39.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd563fff640 1 --2- 172.21.15.83:0/2008796173 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561f0503a920 0x561f04ff53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:39.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd56929a640 1 --2- 172.21.15.83:0/2008796173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f0503b240 0x561f04ff5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:39.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd568a99640 1 -- 172.21.15.83:0/2008796173 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561f0503a920 msgr2=0x561f04ff53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:39.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd568a99640 1 --2- 172.21.15.83:0/2008796173 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561f0503a920 0x561f04ff53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd568a99640 1 -- 172.21.15.83:0/2008796173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f0503b240 msgr2=0x561f04ff5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:39.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd568a99640 1 --2- 172.21.15.83:0/2008796173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f0503b240 0x561f04ff5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd568a99640 1 -- 172.21.15.83:0/2008796173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f04df8630 con 0x561f04ff0f30 2024-04-17T10:47:39.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd563fff640 1 --2- 172.21.15.83:0/2008796173 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561f0503a920 0x561f04ff53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:39.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd56929a640 1 --2- 172.21.15.83:0/2008796173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f0503b240 0x561f04ff5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:39.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd568a99640 1 --2- 172.21.15.83:0/2008796173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f04ff0f30 0x561f04ff4e90 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fd55801eb90 tx=0x7fd55801a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a3dd9699c4cb2d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:39.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd5437fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:47:39.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd5437fe640 1 -- 172.21.15.83:0/2008796173 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd558107040 con 0x561f04ff0f30 2024-04-17T10:47:39.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd5437fe640 1 -- 172.21.15.83:0/2008796173 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd55811d760 con 0x561f04ff0f30 2024-04-17T10:47:39.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd56de6a000 1 -- 172.21.15.83:0/2008796173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5580f86d0 con 0x561f04ff0f30 2024-04-17T10:47:39.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd5437fe640 1 -- 172.21.15.83:0/2008796173 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5580f8240 con 0x561f04ff0f30 2024-04-17T10:47:39.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd56de6a000 10 client.8217 fetch_fsmap learned FSMap version 62 2024-04-17T10:47:39.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd56de6a000 1 -- 172.21.15.83:0/2008796173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x561f04dc83a0 con 0x561f04ff0f30 2024-04-17T10:47:39.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd5437fe640 1 -- 172.21.15.83:0/2008796173 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5580f86d0 con 0x561f04ff0f30 2024-04-17T10:47:39.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd5437fe640 1 -- 172.21.15.83:0/2008796173 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd5580f2070 con 0x561f04ff0f30 2024-04-17T10:47:39.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd56de6a000 10 client.8217 fetch_fsmap finished waiting for FSMap version 62 2024-04-17T10:47:39.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd56de6a000 10 client.8217 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T10:47:39.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd56de6a000 20 client.8217 populate_metadata read hostname 'smithi083' 2024-04-17T10:47:39.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd56de6a000 1 --2- 172.21.15.83:0/2008796173 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x561f05049bc0 0x561f0504bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:39.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd56de6a000 4 client.8217 mds_command: new command op to 8049 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:47:39.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd56de6a000 1 -- 172.21.15.83:0/2008796173 --> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f04ddba50 con 0x561f05049bc0 2024-04-17T10:47:39.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd563fff640 1 --2- 172.21.15.83:0/2008796173 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x561f05049bc0 0x561f0504bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:39.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd563fff640 1 --2- 172.21.15.83:0/2008796173 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x561f05049bc0 0x561f0504bfb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f1c14e51cd03ee0 server_cookie=94b2ea6189aa08c1 in_seq=0 out_seq=0 2024-04-17T10:47:39.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd5437fe640 10 client.8217 ms_handle_connect on v2:172.21.15.83:6836/2353406669 2024-04-17T10:47:39.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd5437fe640 1 -- 172.21.15.83:0/2008796173 <== mds.0 v2:172.21.15.83:6836/2353406669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561f04ddba50 con 0x561f05049bc0 2024-04-17T10:47:39.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd5437fe640 10 client.8217 handle_command_reply: tid=0 2024-04-17T10:47:39.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd56de6a000 1 client.8217 shutdown 2024-04-17T10:47:39.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd56de6a000 1 -- 172.21.15.83:0/2008796173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f04ff0f30 msgr2=0x561f04ff4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:39.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd56de6a000 1 --2- 172.21.15.83:0/2008796173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f04ff0f30 0x561f04ff4e90 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fd55801eb90 tx=0x7fd55801a320 comp rx=0 tx=0).stop 2024-04-17T10:47:39.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd56de6a000 1 -- 172.21.15.83:0/2008796173 shutdown_connections 2024-04-17T10:47:39.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd56de6a000 1 --2- 172.21.15.83:0/2008796173 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x561f05049bc0 0x561f0504bfb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd56de6a000 1 --2- 172.21.15.83:0/2008796173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f0503b240 0x561f04ff5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd56de6a000 1 --2- 172.21.15.83:0/2008796173 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561f0503a920 0x561f04ff53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd5437fe640 0 client.8217 ms_handle_reset on v2:172.21.15.83:6836/2353406669 2024-04-17T10:47:39.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd56de6a000 1 --2- 172.21.15.83:0/2008796173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f04ff0f30 0x561f04ff4e90 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd56de6a000 1 -- 172.21.15.83:0/2008796173 >> 172.21.15.83:0/2008796173 conn(0x561f04ff72a0 msgr2=0x561f04fef9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:39.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd56de6a000 1 -- 172.21.15.83:0/2008796173 shutdown_connections 2024-04-17T10:47:39.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd56de6a000 1 -- 172.21.15.83:0/2008796173 wait complete. 2024-04-17T10:47:39.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.218+0000 7fd56de6a000 20 client.8217 trim_cache size 0 max 16384 2024-04-17T10:47:39.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.234+0000 7fd56de6a000 1 Processor -- start 2024-04-17T10:47:39.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.234+0000 7fd56de6a000 1 -- start start 2024-04-17T10:47:39.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.234+0000 7fd56de6a000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f04ff0f30 0x561f04ff1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:39.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.234+0000 7fd56de6a000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561f0503a920 0x561f04ff1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:39.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.234+0000 7fd56de6a000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f0503b240 0x561f04ff1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:39.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.234+0000 7fd56de6a000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd564157890 con 0x561f04ff0f30 2024-04-17T10:47:39.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.234+0000 7fd56de6a000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd564155950 con 0x561f0503b240 2024-04-17T10:47:39.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.234+0000 7fd56de6a000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd564157290 con 0x561f0503a920 2024-04-17T10:47:39.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.234+0000 7fd568a99640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f04ff0f30 0x561f04ff1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:39.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.234+0000 7fd568a99640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f04ff0f30 0x561f04ff1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34886/0 (socket says 172.21.15.83:34886) 2024-04-17T10:47:39.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.234+0000 7fd568a99640 1 -- 172.21.15.83:0/4177586262 learned_addr learned my addr 172.21.15.83:0/4177586262 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:39.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.234+0000 7fd563fff640 1 --2- 172.21.15.83:0/4177586262 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561f0503a920 0x561f04ff1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:39.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.234+0000 7fd568a99640 1 -- 172.21.15.83:0/4177586262 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561f0503a920 msgr2=0x561f04ff1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:39.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.234+0000 7fd568a99640 1 --2- 172.21.15.83:0/4177586262 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561f0503a920 0x561f04ff1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.234+0000 7fd568a99640 1 -- 172.21.15.83:0/4177586262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f0503b240 msgr2=0x561f04ff1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:39.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.234+0000 7fd568a99640 1 --2- 172.21.15.83:0/4177586262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f0503b240 0x561f04ff1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.234+0000 7fd568a99640 1 -- 172.21.15.83:0/4177586262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f0503b070 con 0x561f04ff0f30 2024-04-17T10:47:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.234+0000 7fd568a99640 1 --2- 172.21.15.83:0/4177586262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f04ff0f30 0x561f04ff1310 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fd558000c10 tx=0x7fd5580c5000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47469218b72a144a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.234+0000 7fd54dffb640 1 -- 172.21.15.83:0/4177586262 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5580f4110 con 0x561f04ff0f30 2024-04-17T10:47:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.234+0000 7fd54dffb640 1 -- 172.21.15.83:0/4177586262 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5580ee060 con 0x561f04ff0f30 2024-04-17T10:47:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.234+0000 7fd54dffb640 1 -- 172.21.15.83:0/4177586262 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5580ef650 con 0x561f04ff0f30 2024-04-17T10:47:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.234+0000 7fd56de6a000 1 -- 172.21.15.83:0/4177586262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f04ff0f30 msgr2=0x561f04ff1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.234+0000 7fd56de6a000 1 --2- 172.21.15.83:0/4177586262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f04ff0f30 0x561f04ff1310 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fd558000c10 tx=0x7fd5580c5000 comp rx=0 tx=0).stop 2024-04-17T10:47:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.234+0000 7fd56de6a000 1 -- 172.21.15.83:0/4177586262 shutdown_connections 2024-04-17T10:47:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.234+0000 7fd56de6a000 1 --2- 172.21.15.83:0/4177586262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f0503b240 0x561f04ff1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.234+0000 7fd56de6a000 1 --2- 172.21.15.83:0/4177586262 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561f0503a920 0x561f04ff1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.234+0000 7fd56de6a000 1 --2- 172.21.15.83:0/4177586262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f04ff0f30 0x561f04ff1310 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.234+0000 7fd56de6a000 1 -- 172.21.15.83:0/4177586262 >> 172.21.15.83:0/4177586262 conn(0x561f04ff72a0 msgr2=0x561f05025c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:39.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.234+0000 7fd56de6a000 1 -- 172.21.15.83:0/4177586262 shutdown_connections 2024-04-17T10:47:39.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.234+0000 7fd56de6a000 1 -- 172.21.15.83:0/4177586262 wait complete. 2024-04-17T10:47:39.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56de6a000 1 -- start start 2024-04-17T10:47:39.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56de6a000 1 Processor -- start 2024-04-17T10:47:39.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56de6a000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f04ff0f30 0x561f04ff4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:39.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56de6a000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f0503a920 0x561f050998d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:39.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56de6a000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561f0503b240 0x561f05099e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:39.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56de6a000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f04e443d0 con 0x561f0503a920 2024-04-17T10:47:39.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56de6a000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f04efbd50 con 0x561f04ff0f30 2024-04-17T10:47:39.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56de6a000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x561f04dda980 con 0x561f0503b240 2024-04-17T10:47:39.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56de6a000 10 client.0 _ll_register_callbacks cb 0x7fd558009420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:47:39.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd563fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f0503a920 0x561f050998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:39.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd563fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f0503a920 0x561f050998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34902/0 (socket says 172.21.15.83:34902) 2024-04-17T10:47:39.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd563fff640 1 -- 172.21.15.83:0/3705916012 learned_addr learned my addr 172.21.15.83:0/3705916012 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:39.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56929a640 1 --2- 172.21.15.83:0/3705916012 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561f0503b240 0x561f05099e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:39.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd568a99640 1 --2- 172.21.15.83:0/3705916012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f04ff0f30 0x561f04ff4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:39.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd563fff640 1 -- 172.21.15.83:0/3705916012 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561f0503b240 msgr2=0x561f05099e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:39.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd563fff640 1 --2- 172.21.15.83:0/3705916012 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561f0503b240 0x561f05099e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd563fff640 1 -- 172.21.15.83:0/3705916012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f04ff0f30 msgr2=0x561f04ff4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:39.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd563fff640 1 --2- 172.21.15.83:0/3705916012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f04ff0f30 0x561f04ff4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd563fff640 1 -- 172.21.15.83:0/3705916012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f04ec1600 con 0x561f0503a920 2024-04-17T10:47:39.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56929a640 1 --2- 172.21.15.83:0/3705916012 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561f0503b240 0x561f05099e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:39.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd563fff640 1 --2- 172.21.15.83:0/3705916012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f0503a920 0x561f050998d0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fd56415d070 tx=0x7fd550000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a122639f2f935f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:39.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd5437fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:47:39.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd5437fe640 1 -- 172.21.15.83:0/3705916012 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd550005cd0 con 0x561f0503a920 2024-04-17T10:47:39.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd5437fe640 1 -- 172.21.15.83:0/3705916012 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd550026890 con 0x561f0503a920 2024-04-17T10:47:39.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56de6a000 1 -- 172.21.15.83:0/3705916012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5580ee060 con 0x561f0503a920 2024-04-17T10:47:39.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd5437fe640 1 -- 172.21.15.83:0/3705916012 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd550031050 con 0x561f0503a920 2024-04-17T10:47:39.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56de6a000 10 client.8223 fetch_fsmap learned FSMap version 62 2024-04-17T10:47:39.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56de6a000 1 -- 172.21.15.83:0/3705916012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x561f04dd49b0 con 0x561f0503a920 2024-04-17T10:47:39.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd5437fe640 1 -- 172.21.15.83:0/3705916012 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5580ee060 con 0x561f0503a920 2024-04-17T10:47:39.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd5437fe640 1 -- 172.21.15.83:0/3705916012 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd550026ad0 con 0x561f0503a920 2024-04-17T10:47:39.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56de6a000 10 client.8223 fetch_fsmap finished waiting for FSMap version 62 2024-04-17T10:47:39.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56de6a000 10 client.8223 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T10:47:39.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56de6a000 20 client.8223 populate_metadata read hostname 'smithi083' 2024-04-17T10:47:39.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56de6a000 1 --2- 172.21.15.83:0/3705916012 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x561f05055a30 0x561f05055e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:39.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56de6a000 4 client.8223 mds_command: new command op to 8049 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T10:47:39.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56de6a000 1 -- 172.21.15.83:0/3705916012 --> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561f04e4d2e0 con 0x561f05055a30 2024-04-17T10:47:39.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd568a99640 1 --2- 172.21.15.83:0/3705916012 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x561f05055a30 0x561f05055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:39.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd5437fe640 10 client.8223 ms_handle_connect on v2:172.21.15.83:6836/2353406669 2024-04-17T10:47:39.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd568a99640 1 --2- 172.21.15.83:0/3705916012 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x561f05055a30 0x561f05055e10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebc85b282e93958d server_cookie=5fb31bfe511abbd4 in_seq=0 out_seq=0 2024-04-17T10:47:39.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd5437fe640 1 -- 172.21.15.83:0/3705916012 <== mds.0 v2:172.21.15.83:6836/2353406669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x561f04e4d2e0 con 0x561f05055a30 2024-04-17T10:47:39.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd5437fe640 10 client.8223 handle_command_reply: tid=0 2024-04-17T10:47:39.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56de6a000 1 client.8223 shutdown 2024-04-17T10:47:39.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56de6a000 1 -- 172.21.15.83:0/3705916012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f0503a920 msgr2=0x561f050998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:39.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56de6a000 1 --2- 172.21.15.83:0/3705916012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f0503a920 0x561f050998d0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fd56415d070 tx=0x7fd550000ee0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56de6a000 1 -- 172.21.15.83:0/3705916012 shutdown_connections 2024-04-17T10:47:39.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56de6a000 1 --2- 172.21.15.83:0/3705916012 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x561f05055a30 0x561f05055e10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56de6a000 1 --2- 172.21.15.83:0/3705916012 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561f0503b240 0x561f05099e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd5437fe640 0 client.8223 ms_handle_reset on v2:172.21.15.83:6836/2353406669 2024-04-17T10:47:39.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56de6a000 1 --2- 172.21.15.83:0/3705916012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561f0503a920 0x561f050998d0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56de6a000 1 --2- 172.21.15.83:0/3705916012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f04ff0f30 0x561f04ff4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56de6a000 1 -- 172.21.15.83:0/3705916012 >> 172.21.15.83:0/3705916012 conn(0x561f04ff72a0 msgr2=0x561f05025c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:39.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56de6a000 1 -- 172.21.15.83:0/3705916012 shutdown_connections 2024-04-17T10:47:39.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56de6a000 1 -- 172.21.15.83:0/3705916012 wait complete. 2024-04-17T10:47:39.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56de6a000 20 client.8223 trim_cache size 0 max 16384 2024-04-17T10:47:39.245 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:47:39.245 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "3de9b7be-af7f-488a-969e-2b6b8db63bcd", 2024-04-17T10:47:39.245 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-04-17T10:47:39.245 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 8049, 2024-04-17T10:47:39.245 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-17T10:47:39.246 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-17T10:47:39.246 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-17T10:47:39.246 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 9.2244449320000008, 2024-04-17T10:47:39.246 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 62, 2024-04-17T10:47:39.246 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 103, 2024-04-17T10:47:39.246 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 103, 2024-04-17T10:47:39.246 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 14.566760219000001 2024-04-17T10:47:39.246 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:47:39.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56acfd640 1 -- 172.21.15.83:0/1108473601 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd548066f60 msgr2=0x7fd548069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:39.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.238+0000 7fd56acfd640 1 --2- 172.21.15.83:0/1108473601 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd548066f60 0x7fd548069420 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd56415d6c0 tx=0x7fd550009040 comp rx=0 tx=0).stop 2024-04-17T10:47:39.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.242+0000 7fd56acfd640 1 -- 172.21.15.83:0/1108473601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd56406cc80 msgr2=0x7fd56415c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:39.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.242+0000 7fd56acfd640 1 --2- 172.21.15.83:0/1108473601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd56406cc80 0x7fd56415c640 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fd55801b1b0 tx=0x7fd55801bdc0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.242+0000 7fd56acfd640 1 -- 172.21.15.83:0/1108473601 shutdown_connections 2024-04-17T10:47:39.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.242+0000 7fd56acfd640 1 --2- 172.21.15.83:0/1108473601 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd548066f60 0x7fd548069420 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.242+0000 7fd56acfd640 1 --2- 172.21.15.83:0/1108473601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd564154670 0x7fd564146740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.242+0000 7fd56acfd640 1 --2- 172.21.15.83:0/1108473601 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5640b4090 0x7fd564146200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.242+0000 7fd56acfd640 1 --2- 172.21.15.83:0/1108473601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd56406cc80 0x7fd56415c640 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.242+0000 7fd56acfd640 1 -- 172.21.15.83:0/1108473601 >> 172.21.15.83:0/1108473601 conn(0x7fd5640b15b0 msgr2=0x7fd5640b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:39.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.242+0000 7fd56acfd640 1 -- 172.21.15.83:0/1108473601 shutdown_connections 2024-04-17T10:47:39.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.242+0000 7fd56acfd640 1 -- 172.21.15.83:0/1108473601 wait complete. 2024-04-17T10:47:39.253 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T10:47:39.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.370+0000 7f8dad4ca640 1 Processor -- start 2024-04-17T10:47:39.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.370+0000 7f8dad4ca640 1 -- start start 2024-04-17T10:47:39.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.370+0000 7f8dad4ca640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8da80b2de0 0x7f8da815ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:39.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.370+0000 7f8dad4ca640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8da815f5b0 con 0x7f8da80d22d0 2024-04-17T10:47:39.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.370+0000 7f8dad4ca640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8da80b2620 con 0x7f8da815f1d0 2024-04-17T10:47:39.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.370+0000 7f8dad4ca640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8da80b2340 con 0x7f8da80b2de0 2024-04-17T10:47:39.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.370+0000 7f8da77fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8da80d22d0 0x7f8da80ae520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60178/0 (socket says 172.21.15.83:60178) 2024-04-17T10:47:39.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.370+0000 7f8da67fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8da80b2de0 0x7f8da815ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:39.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.370+0000 7f8da77fe640 1 -- 172.21.15.83:0/219023585 learned_addr learned my addr 172.21.15.83:0/219023585 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:39.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.370+0000 7f8da67fc640 1 -- 172.21.15.83:0/219023585 >> v1:172.21.15.167:6789/0 conn(0x7f8da815f1d0 legacy=0x7f8da815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:39.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.370+0000 7f8da5ffb640 1 -- 172.21.15.83:0/219023585 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848837097 0 0) 0x7f8da815f5b0 con 0x7f8da80d22d0 2024-04-17T10:47:39.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.370+0000 7f8da67fc640 1 -- 172.21.15.83:0/219023585 >> v1:172.21.15.83:6789/0 conn(0x7f8da80d22d0 legacy=0x7f8da80ae520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:39.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.370+0000 7f8da67fc640 1 -- 172.21.15.83:0/219023585 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da80b1760 con 0x7f8da80b2de0 2024-04-17T10:47:39.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.374+0000 7f8da67fc640 1 --2- 172.21.15.83:0/219023585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8da80b2de0 0x7f8da815ff40 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c0097c0 tx=0x7f8d9c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919c23c67d734bc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:39.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.374+0000 7f8da5ffb640 1 -- 172.21.15.83:0/219023585 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c0a7070 con 0x7f8da80b2de0 2024-04-17T10:47:39.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.374+0000 7f8da5ffb640 1 -- 172.21.15.83:0/219023585 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d9c0b0070 con 0x7f8da80b2de0 2024-04-17T10:47:39.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.374+0000 7f8da5ffb640 1 -- 172.21.15.83:0/219023585 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c0ab890 con 0x7f8da80b2de0 2024-04-17T10:47:39.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.374+0000 7f8dad4ca640 1 -- 172.21.15.83:0/219023585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8da80b2de0 msgr2=0x7f8da815ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:39.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.374+0000 7f8dad4ca640 1 --2- 172.21.15.83:0/219023585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8da80b2de0 0x7f8da815ff40 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c0097c0 tx=0x7f8d9c01c470 comp rx=0 tx=0).stop 2024-04-17T10:47:39.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.374+0000 7f8dad4ca640 1 -- 172.21.15.83:0/219023585 shutdown_connections 2024-04-17T10:47:39.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.374+0000 7f8dad4ca640 1 --2- 172.21.15.83:0/219023585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8da80b2de0 0x7f8da815ff40 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.374+0000 7f8dad4ca640 1 -- 172.21.15.83:0/219023585 >> 172.21.15.83:0/219023585 conn(0x7f8da80d17b0 msgr2=0x7f8da80d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:39.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.374+0000 7f8dad4ca640 1 -- 172.21.15.83:0/219023585 shutdown_connections 2024-04-17T10:47:39.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.374+0000 7f8dad4ca640 1 -- 172.21.15.83:0/219023585 wait complete. 2024-04-17T10:47:39.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.374+0000 7f8dad4ca640 1 Processor -- start 2024-04-17T10:47:39.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.374+0000 7f8dad4ca640 1 -- start start 2024-04-17T10:47:39.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.374+0000 7f8dad4ca640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8da80d22d0 0x7f8da814c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:39.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.374+0000 7f8dad4ca640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8da815f1d0 0x7f8da80c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:39.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.374+0000 7f8dad4ca640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8da814cde0 0x7f8da80ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:39.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.374+0000 7f8dad4ca640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da80af440 con 0x7f8da815f1d0 2024-04-17T10:47:39.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.374+0000 7f8dad4ca640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da8160690 con 0x7f8da80d22d0 2024-04-17T10:47:39.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.374+0000 7f8dad4ca640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da80af5c0 con 0x7f8da814cde0 2024-04-17T10:47:39.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.374+0000 7f8da67fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8da815f1d0 0x7f8da80c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:39.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.374+0000 7f8da67fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8da815f1d0 0x7f8da80c76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34922/0 (socket says 172.21.15.83:34922) 2024-04-17T10:47:39.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.374+0000 7f8da67fc640 1 -- 172.21.15.83:0/1528553846 learned_addr learned my addr 172.21.15.83:0/1528553846 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:39.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.374+0000 7f8da77fe640 1 --2- 172.21.15.83:0/1528553846 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8da814cde0 0x7f8da80ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:39.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.374+0000 7f8da6ffd640 1 --2- 172.21.15.83:0/1528553846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8da80d22d0 0x7f8da814c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:39.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.374+0000 7f8da67fc640 1 -- 172.21.15.83:0/1528553846 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8da814cde0 msgr2=0x7f8da80ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:39.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.374+0000 7f8da67fc640 1 --2- 172.21.15.83:0/1528553846 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8da814cde0 0x7f8da80ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.374+0000 7f8da67fc640 1 -- 172.21.15.83:0/1528553846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8da80d22d0 msgr2=0x7f8da814c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:39.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.374+0000 7f8da67fc640 1 --2- 172.21.15.83:0/1528553846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8da80d22d0 0x7f8da814c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.374+0000 7f8da67fc640 1 -- 172.21.15.83:0/1528553846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da80c6060 con 0x7f8da815f1d0 2024-04-17T10:47:39.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.374+0000 7f8da77fe640 1 --2- 172.21.15.83:0/1528553846 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8da814cde0 0x7f8da80ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:39.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.378+0000 7f8da67fc640 1 --2- 172.21.15.83:0/1528553846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8da815f1d0 0x7f8da80c76f0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c0ad040 tx=0x7f8d9c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:39.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.378+0000 7f8d8f7fe640 1 -- 172.21.15.83:0/1528553846 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c0b5610 con 0x7f8da815f1d0 2024-04-17T10:47:39.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.378+0000 7f8d8f7fe640 1 -- 172.21.15.83:0/1528553846 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d9c0b57f0 con 0x7f8da815f1d0 2024-04-17T10:47:39.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.378+0000 7f8dad4ca640 1 -- 172.21.15.83:0/1528553846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da80c6340 con 0x7f8da815f1d0 2024-04-17T10:47:39.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.378+0000 7f8d8f7fe640 1 -- 172.21.15.83:0/1528553846 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c0b5be0 con 0x7f8da815f1d0 2024-04-17T10:47:39.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.378+0000 7f8dad4ca640 1 -- 172.21.15.83:0/1528553846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da80c6900 con 0x7f8da815f1d0 2024-04-17T10:47:39.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.378+0000 7f8dad4ca640 1 -- 172.21.15.83:0/1528553846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da8074f90 con 0x7f8da815f1d0 2024-04-17T10:47:39.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.378+0000 7f8d8f7fe640 1 -- 172.21.15.83:0/1528553846 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8d9c0bd020 con 0x7f8da815f1d0 2024-04-17T10:47:39.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.378+0000 7f8d8f7fe640 1 --2- 172.21.15.83:0/1528553846 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8d94066f60 0x7f8d94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:39.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.378+0000 7f8da6ffd640 1 --2- 172.21.15.83:0/1528553846 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8d94066f60 0x7f8d94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:39.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.378+0000 7f8d8f7fe640 1 -- 172.21.15.83:0/1528553846 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d9c11fa60 con 0x7f8da815f1d0 2024-04-17T10:47:39.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.378+0000 7f8da6ffd640 1 --2- 172.21.15.83:0/1528553846 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8d94066f60 0x7f8d94069420 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8d98007fd0 tx=0x7f8d98009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:39.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.382+0000 7f8d8f7fe640 1 -- 172.21.15.83:0/1528553846 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8d9c016090 con 0x7f8da815f1d0 2024-04-17T10:47:39.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.566+0000 7f8dad4ca640 1 -- 172.21.15.83:0/1528553846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8da8153ac0 con 0x7f8da815f1d0 2024-04-17T10:47:39.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.566+0000 7f8d8f7fe640 1 -- 172.21.15.83:0/1528553846 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v62) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8d9c0f0be0 con 0x7f8da815f1d0 2024-04-17T10:47:39.564 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T10:47:39.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.570+0000 7f8dad4ca640 1 -- 172.21.15.83:0/1528553846 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8d94066f60 msgr2=0x7f8d94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:39.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.570+0000 7f8dad4ca640 1 --2- 172.21.15.83:0/1528553846 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8d94066f60 0x7f8d94069420 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8d98007fd0 tx=0x7f8d98009040 comp rx=0 tx=0).stop 2024-04-17T10:47:39.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.570+0000 7f8dad4ca640 1 -- 172.21.15.83:0/1528553846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8da815f1d0 msgr2=0x7f8da80c76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:39.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.570+0000 7f8dad4ca640 1 --2- 172.21.15.83:0/1528553846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8da815f1d0 0x7f8da80c76f0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c0ad040 tx=0x7f8d9c01acb0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.570+0000 7f8dad4ca640 1 -- 172.21.15.83:0/1528553846 shutdown_connections 2024-04-17T10:47:39.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.570+0000 7f8dad4ca640 1 --2- 172.21.15.83:0/1528553846 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8d94066f60 0x7f8d94069420 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.570+0000 7f8dad4ca640 1 --2- 172.21.15.83:0/1528553846 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8da814cde0 0x7f8da80ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.570+0000 7f8dad4ca640 1 --2- 172.21.15.83:0/1528553846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8da815f1d0 0x7f8da80c76f0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.570+0000 7f8dad4ca640 1 --2- 172.21.15.83:0/1528553846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8da80d22d0 0x7f8da814c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:39.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.570+0000 7f8dad4ca640 1 -- 172.21.15.83:0/1528553846 >> 172.21.15.83:0/1528553846 conn(0x7f8da80d17b0 msgr2=0x7f8da806fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:39.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.570+0000 7f8dad4ca640 1 -- 172.21.15.83:0/1528553846 shutdown_connections 2024-04-17T10:47:39.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:39.570+0000 7f8dad4ca640 1 -- 172.21.15.83:0/1528553846 wait complete. 2024-04-17T10:47:39.581 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T10:47:39.581 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T10:47:39.581 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-04-17T10:47:39.581 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T10:47:39.581 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T10:47:39.581 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T10:47:39.581 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T10:47:39.581 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T10:47:39.581 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:47:39.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:39.582 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-04-17T10:47:39.587 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T10:47:39.588 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T10:47:39.588 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T10:47:39.588 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:47:39.588 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-17T10:47:39.588 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:47:39.588 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:47:39.588 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-04-17T10:47:39.588 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-04-17T10:47:39.588 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:47:39.588 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-04-17T10:47:39.588 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-04-17T10:47:39.588 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:47:39.588 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:47:39.589 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T10:47:39.589 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:47:39.589 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-17T10:47:39.589 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 41924sec preferred_lft 41924sec 2024-04-17T10:47:39.589 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-04-17T10:47:39.589 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:47:39.589 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T10:47:39.589 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:37:0e:23:b2 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:47:39.589 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T10:47:39.589 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:47:39.589 INFO:teuthology.orchestra.run.smithi083.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T10:47:39.589 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-17T10:47:39.590 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T10:47:39.590 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:47:39.590 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-17T10:47:39.590 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:47:39.590 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:47:39.590 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:47:39.590 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-17T10:47:39.590 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:47:39.590 INFO:teuthology.orchestra.run.smithi083.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:47:39.590 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:47:39.590 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-17T10:47:39.590 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:47:39.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:39.591 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T10:47:39.638 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-04-17T10:47:39.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:39.640 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-04-17T10:47:39.688 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-04-17T10:47:39.688 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T10:47:39.688 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-17T10:47:39.688 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-17T10:47:39.688 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-17T10:47:39.689 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T10:47:39.689 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-17T10:47:39.689 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:47:39.689 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:47:39.689 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:47:39.689 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T10:47:39.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:39.689 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2024-04-17T10:47:39.689 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:39.689 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-17T10:47:39.689 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-17T10:47:39.690 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-17T10:47:39.690 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:39.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:39.767 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-17T10:47:39.813 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-17T10:47:39.813 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T10:47:39.814 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:47:39.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:39.814 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-04-17T10:47:39.870 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-17T10:47:39.870 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:47:39.870 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T10:47:39.918 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:47:39.918 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:47:39.918 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:47:39.965 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:47:40.023 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:47:40.071 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:47:40.071 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:47:40.071 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:47:40.071 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:47:40.071 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:47:40.071 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:47:40.071 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:47:40.071 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:47:40.071 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:47:40.071 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:47:40.072 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:47:40.072 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:47:40.072 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:47:40.072 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:47:40.072 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:47:40.072 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:47:40.072 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:47:40.072 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:47:40.072 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:47:40.072 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:47:40.072 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:47:40.072 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:47:40.072 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:47:40.073 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:47:40.073 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:47:40.073 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:47:40.073 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:47:40.073 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:47:40.073 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:47:40.073 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:47:40.073 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:47:40.073 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:47:40.073 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:47:40.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:40.074 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:47:40.121 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T10:47:40.122 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-17T10:47:40.166 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:47:40.181 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:47:40.229 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:47:40.230 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:47:40.230 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:47:40.230 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:47:40.230 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:47:40.230 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:47:40.230 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:47:40.230 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:47:40.230 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:47:40.230 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:47:40.230 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:47:40.230 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:47:40.230 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:47:40.231 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:47:40.231 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:47:40.231 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:47:40.231 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:47:40.231 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:47:40.231 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:47:40.231 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:47:40.231 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:47:40.231 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:47:40.231 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:47:40.231 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:47:40.231 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:47:40.231 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:47:40.231 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:47:40.232 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:47:40.232 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:47:40.232 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:47:40.232 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:47:40.232 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:47:40.232 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:47:40.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:40.233 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:47:40.242 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:2024-04-17T10:47:40.246+0000 7f8271eab580 -1 init, newargv = 0x5597070f13d0 newargc=15 2024-04-17T10:47:40.242 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[73527]: starting ceph client 2024-04-17T10:47:40.248 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[73527]: starting fuse 2024-04-17T10:47:41.348 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:47:41.363 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:47:41.412 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:47:41.412 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:47:41.412 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:47:41.412 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:47:41.412 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:47:41.412 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:47:41.412 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:47:41.412 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:47:41.413 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:47:41.413 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:47:41.413 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:47:41.413 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:47:41.413 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:47:41.413 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:47:41.413 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:47:41.413 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:47:41.413 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:47:41.413 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:47:41.413 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:47:41.413 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:47:41.413 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:47:41.413 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:47:41.414 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:47:41.414 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:47:41.414 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:47:41.414 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:47:41.414 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:47:41.414 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:47:41.414 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:47:41.414 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:47:41.414 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:47:41.414 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:47:41.414 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:47:41.414 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-17T10:47:41.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:41.415 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:47:41.462 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-04-17T10:47:41.462 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-17T10:47:41.463 DEBUG:teuthology.orchestra.run.smithi083:> sudo stdin-killer -- python3 -c ' 2024-04-17T10:47:41.463 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2024-04-17T10:47:41.463 DEBUG:teuthology.orchestra.run.smithi083:> import re 2024-04-17T10:47:41.463 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-04-17T10:47:41.463 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2024-04-17T10:47:41.463 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:47:41.463 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2024-04-17T10:47:41.463 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-17T10:47:41.463 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2024-04-17T10:47:41.463 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-17T10:47:41.464 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:47:41.464 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2024-04-17T10:47:41.464 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2024-04-17T10:47:41.464 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2024-04-17T10:47:41.464 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2024-04-17T10:47:41.464 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:47:41.464 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2024-04-17T10:47:41.464 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T10:47:41.464 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T10:47:41.464 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T10:47:41.464 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2024-04-17T10:47:41.464 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2024-04-17T10:47:41.464 DEBUG:teuthology.orchestra.run.smithi083:> return f 2024-04-17T10:47:41.464 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T10:47:41.464 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:47:41.465 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.0")) 2024-04-17T10:47:41.465 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-04-17T10:47:41.586 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.0.73527.asok 2024-04-17T10:47:41.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:47:41.599 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.73527.asok 2024-04-17T10:47:41.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:41.600 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.73527.asok status 2024-04-17T10:47:41.754 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:47:41.754 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2024-04-17T10:47:41.754 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-17T10:47:41.754 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-17T10:47:41.754 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "0", 2024-04-17T10:47:41.755 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-04-17T10:47:41.755 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-17T10:47:41.755 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "73527", 2024-04-17T10:47:41.755 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-04-17T10:47:41.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:47:41.755 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2024-04-17T10:47:41.755 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2024-04-17T10:47:41.755 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 8235, 2024-04-17T10:47:41.755 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2024-04-17T10:47:41.755 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-04-17T10:47:41.755 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-04-17T10:47:41.755 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 8235 2024-04-17T10:47:41.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:47:41.756 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-04-17T10:47:41.756 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-17T10:47:41.756 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:47:41.756 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2102785463 2024-04-17T10:47:41.756 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:47:41.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:47:41.756 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-04-17T10:47:41.756 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-17T10:47:41.756 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:47:41.756 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2102785463 2024-04-17T10:47:41.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:47:41.757 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.8235 192.168.0.1:0/2102785463", 2024-04-17T10:47:41.757 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.1:0/2102785463", 2024-04-17T10:47:41.757 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 1, 2024-04-17T10:47:41.757 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 62, 2024-04-17T10:47:41.757 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 103, 2024-04-17T10:47:41.757 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2024-04-17T10:47:41.757 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2024-04-17T10:47:41.757 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2024-04-17T10:47:41.757 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:47:41.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:41.768 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2024-04-17T10:47:41.768 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-17T10:47:41.775 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2024-04-17T10:47:41.775 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-17T10:47:41.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:41.775 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:47:41.830 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-17T10:47:41.830 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_control_test_path 2024-04-17T10:47:41.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:41.830 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:41.830 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:41.830 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.0 2024-04-17T10:47:41.831 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:41.831 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:41.831 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:41.831 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:41.919 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:41.919 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:41.919 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181475 s, 57.8 MB/s 2024-04-17T10:47:41.938 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:41.939 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:41.939 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157431 s, 66.6 MB/s 2024-04-17T10:47:41.959 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:41.959 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:41.959 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172605 s, 60.8 MB/s 2024-04-17T10:47:41.976 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:41.976 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:41.976 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147462 s, 71.1 MB/s 2024-04-17T10:47:41.993 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:41.993 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:41.993 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137461 s, 76.3 MB/s 2024-04-17T10:47:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.008 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117443 s, 89.3 MB/s 2024-04-17T10:47:42.025 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.025 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.025 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014137 s, 74.2 MB/s 2024-04-17T10:47:42.040 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.041 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.041 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131281 s, 79.9 MB/s 2024-04-17T10:47:42.055 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.055 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.055 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122102 s, 85.9 MB/s 2024-04-17T10:47:42.071 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.071 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.071 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129583 s, 80.9 MB/s 2024-04-17T10:47:42.087 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.087 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.087 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130467 s, 80.4 MB/s 2024-04-17T10:47:42.105 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.105 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.105 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155736 s, 67.3 MB/s 2024-04-17T10:47:42.121 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.121 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.122 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134149 s, 78.2 MB/s 2024-04-17T10:47:42.139 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.139 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.139 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142771 s, 73.4 MB/s 2024-04-17T10:47:42.155 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.155 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.155 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132835 s, 78.9 MB/s 2024-04-17T10:47:42.169 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.169 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.169 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113875 s, 92.1 MB/s 2024-04-17T10:47:42.186 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.186 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.186 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013594 s, 77.1 MB/s 2024-04-17T10:47:42.201 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.202 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.202 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102987 s, 102 MB/s 2024-04-17T10:47:42.218 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.218 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.219 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015163 s, 69.2 MB/s 2024-04-17T10:47:42.236 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.236 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.236 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144795 s, 72.4 MB/s 2024-04-17T10:47:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138739 s, 75.6 MB/s 2024-04-17T10:47:42.268 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.268 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.268 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120579 s, 87.0 MB/s 2024-04-17T10:47:42.285 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.285 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.285 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014182 s, 73.9 MB/s 2024-04-17T10:47:42.303 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.303 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.303 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147422 s, 71.1 MB/s 2024-04-17T10:47:42.323 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.323 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.323 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151268 s, 69.3 MB/s 2024-04-17T10:47:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117417 s, 89.3 MB/s 2024-04-17T10:47:42.359 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.359 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.359 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148931 s, 70.4 MB/s 2024-04-17T10:47:42.377 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.377 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.377 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141365 s, 74.2 MB/s 2024-04-17T10:47:42.391 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.392 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.392 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114197 s, 91.8 MB/s 2024-04-17T10:47:42.408 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.408 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.408 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132062 s, 79.4 MB/s 2024-04-17T10:47:42.426 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.426 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.426 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146193 s, 71.7 MB/s 2024-04-17T10:47:42.445 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.445 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.445 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156014 s, 67.2 MB/s 2024-04-17T10:47:42.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:42.447 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:42.447 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:42.447 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.1 2024-04-17T10:47:42.447 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:42.447 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:42.447 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:42.447 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:42.473 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.474 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.474 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121197 s, 86.5 MB/s 2024-04-17T10:47:42.487 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.487 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.488 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112619 s, 93.1 MB/s 2024-04-17T10:47:42.503 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.503 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.503 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124474 s, 84.2 MB/s 2024-04-17T10:47:42.519 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.519 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.519 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127265 s, 82.4 MB/s 2024-04-17T10:47:42.535 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.535 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.535 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134083 s, 78.2 MB/s 2024-04-17T10:47:42.549 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.549 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.549 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113183 s, 92.6 MB/s 2024-04-17T10:47:42.566 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.566 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.566 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135781 s, 77.2 MB/s 2024-04-17T10:47:42.580 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.580 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.580 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110207 s, 95.1 MB/s 2024-04-17T10:47:42.594 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.594 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.594 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113749 s, 92.2 MB/s 2024-04-17T10:47:42.610 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.610 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.610 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127206 s, 82.4 MB/s 2024-04-17T10:47:42.625 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.626 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.626 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125085 s, 83.8 MB/s 2024-04-17T10:47:42.642 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.642 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.642 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140282 s, 74.7 MB/s 2024-04-17T10:47:42.659 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.659 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.659 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139811 s, 75.0 MB/s 2024-04-17T10:47:42.675 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.675 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.675 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131722 s, 79.6 MB/s 2024-04-17T10:47:42.692 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.692 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.693 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142477 s, 73.6 MB/s 2024-04-17T10:47:42.710 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.710 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.710 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142217 s, 73.7 MB/s 2024-04-17T10:47:42.725 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.725 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.725 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129109 s, 81.2 MB/s 2024-04-17T10:47:42.741 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.741 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.741 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129924 s, 80.7 MB/s 2024-04-17T10:47:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119642 s, 87.6 MB/s 2024-04-17T10:47:42.773 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.773 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.773 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141225 s, 74.2 MB/s 2024-04-17T10:47:42.788 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.788 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.788 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120891 s, 86.7 MB/s 2024-04-17T10:47:42.805 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.805 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.805 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140579 s, 74.6 MB/s 2024-04-17T10:47:42.820 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.821 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.821 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012192 s, 86.0 MB/s 2024-04-17T10:47:42.838 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.838 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.839 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147776 s, 71.0 MB/s 2024-04-17T10:47:42.854 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.854 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.854 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120087 s, 87.3 MB/s 2024-04-17T10:47:42.872 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.873 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.873 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150925 s, 69.5 MB/s 2024-04-17T10:47:42.892 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.893 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.893 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155067 s, 67.6 MB/s 2024-04-17T10:47:42.912 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.912 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.912 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149914 s, 69.9 MB/s 2024-04-17T10:47:42.930 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.930 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.930 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142988 s, 73.3 MB/s 2024-04-17T10:47:42.948 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.948 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.948 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136031 s, 77.1 MB/s 2024-04-17T10:47:42.961 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.961 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.961 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103917 s, 101 MB/s 2024-04-17T10:47:42.978 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:42.978 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:42.978 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137496 s, 76.3 MB/s 2024-04-17T10:47:42.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:42.979 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:42.979 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:42.979 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.2 2024-04-17T10:47:42.979 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:42.979 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:42.980 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:42.980 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:43.048 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.049 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.049 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134899 s, 77.7 MB/s 2024-04-17T10:47:43.062 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.063 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.063 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113111 s, 92.7 MB/s 2024-04-17T10:47:43.079 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.080 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.080 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013945 s, 75.2 MB/s 2024-04-17T10:47:43.094 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.094 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.094 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111195 s, 94.3 MB/s 2024-04-17T10:47:43.106 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.107 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.107 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00994534 s, 105 MB/s 2024-04-17T10:47:43.119 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.119 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.119 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00980591 s, 107 MB/s 2024-04-17T10:47:43.135 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.135 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.135 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127845 s, 82.0 MB/s 2024-04-17T10:47:43.147 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.147 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.147 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00947544 s, 111 MB/s 2024-04-17T10:47:43.160 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.160 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.161 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104084 s, 101 MB/s 2024-04-17T10:47:43.173 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.173 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.173 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00993193 s, 106 MB/s 2024-04-17T10:47:43.186 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.186 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.186 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00999746 s, 105 MB/s 2024-04-17T10:47:43.203 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.203 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.203 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141445 s, 74.1 MB/s 2024-04-17T10:47:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.220 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136579 s, 76.8 MB/s 2024-04-17T10:47:43.236 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.236 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.236 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132115 s, 79.4 MB/s 2024-04-17T10:47:43.251 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.252 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.252 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131097 s, 80.0 MB/s 2024-04-17T10:47:43.268 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.268 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.268 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127607 s, 82.2 MB/s 2024-04-17T10:47:43.283 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.283 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.283 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124202 s, 84.4 MB/s 2024-04-17T10:47:43.299 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.299 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.299 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132003 s, 79.4 MB/s 2024-04-17T10:47:43.313 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.313 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.313 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113577 s, 92.3 MB/s 2024-04-17T10:47:43.329 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.330 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.330 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134102 s, 78.2 MB/s 2024-04-17T10:47:43.342 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.342 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.342 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00989061 s, 106 MB/s 2024-04-17T10:47:43.357 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.357 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.357 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126381 s, 83.0 MB/s 2024-04-17T10:47:43.372 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.372 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.372 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114834 s, 91.3 MB/s 2024-04-17T10:47:43.388 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.388 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.388 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137455 s, 76.3 MB/s 2024-04-17T10:47:43.403 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.403 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.403 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01207 s, 86.9 MB/s 2024-04-17T10:47:43.416 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.416 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.416 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010156 s, 103 MB/s 2024-04-17T10:47:43.431 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.431 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.431 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126324 s, 83.0 MB/s 2024-04-17T10:47:43.447 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.447 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.447 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135127 s, 77.6 MB/s 2024-04-17T10:47:43.461 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.461 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.461 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112858 s, 92.9 MB/s 2024-04-17T10:47:43.476 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.476 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.476 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124874 s, 84.0 MB/s 2024-04-17T10:47:43.491 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.491 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.491 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121564 s, 86.3 MB/s 2024-04-17T10:47:43.507 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.507 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.507 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013152 s, 79.7 MB/s 2024-04-17T10:47:43.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:43.508 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:43.509 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:43.509 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.3 2024-04-17T10:47:43.509 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:43.509 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:43.509 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:43.509 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:43.577 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.577 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.577 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112138 s, 93.5 MB/s 2024-04-17T10:47:43.591 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.592 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.592 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114862 s, 91.3 MB/s 2024-04-17T10:47:43.608 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.608 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.608 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136794 s, 76.7 MB/s 2024-04-17T10:47:43.623 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.623 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.624 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122889 s, 85.3 MB/s 2024-04-17T10:47:43.641 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.641 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.641 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141959 s, 73.9 MB/s 2024-04-17T10:47:43.655 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.655 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.656 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114865 s, 91.3 MB/s 2024-04-17T10:47:43.672 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.672 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.672 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140794 s, 74.5 MB/s 2024-04-17T10:47:43.688 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.688 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.688 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126638 s, 82.8 MB/s 2024-04-17T10:47:43.701 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.701 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.702 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010607 s, 98.9 MB/s 2024-04-17T10:47:43.714 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.715 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.715 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104014 s, 101 MB/s 2024-04-17T10:47:43.730 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.730 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.730 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127376 s, 82.3 MB/s 2024-04-17T10:47:43.746 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.746 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.746 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130325 s, 80.5 MB/s 2024-04-17T10:47:43.762 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.762 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.762 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013444 s, 78.0 MB/s 2024-04-17T10:47:43.779 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.779 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.779 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138421 s, 75.8 MB/s 2024-04-17T10:47:43.795 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.795 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.796 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139962 s, 74.9 MB/s 2024-04-17T10:47:43.811 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.812 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.812 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134223 s, 78.1 MB/s 2024-04-17T10:47:43.827 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.827 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.827 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129466 s, 81.0 MB/s 2024-04-17T10:47:43.845 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.845 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.845 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154397 s, 67.9 MB/s 2024-04-17T10:47:43.861 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.861 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.861 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128628 s, 81.5 MB/s 2024-04-17T10:47:43.876 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.876 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.876 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121173 s, 86.5 MB/s 2024-04-17T10:47:43.890 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.890 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.891 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113011 s, 92.8 MB/s 2024-04-17T10:47:43.903 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.903 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.903 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00960247 s, 109 MB/s 2024-04-17T10:47:43.919 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.920 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.920 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140256 s, 74.8 MB/s 2024-04-17T10:47:43.935 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.935 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.935 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123279 s, 85.1 MB/s 2024-04-17T10:47:43.951 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.951 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.951 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012165 s, 86.2 MB/s 2024-04-17T10:47:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136419 s, 76.9 MB/s 2024-04-17T10:47:43.982 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.983 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.983 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126662 s, 82.8 MB/s 2024-04-17T10:47:43.999 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:43.999 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:43.999 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140775 s, 74.5 MB/s 2024-04-17T10:47:44.012 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.012 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.012 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100925 s, 104 MB/s 2024-04-17T10:47:44.029 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.029 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.029 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014423 s, 72.7 MB/s 2024-04-17T10:47:44.045 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.045 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.045 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013633 s, 76.9 MB/s 2024-04-17T10:47:44.058 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.058 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.058 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010229 s, 103 MB/s 2024-04-17T10:47:44.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:44.059 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:44.059 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:44.059 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.4 2024-04-17T10:47:44.060 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:44.060 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:44.060 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:44.060 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:44.124 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.125 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.125 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121373 s, 86.4 MB/s 2024-04-17T10:47:44.138 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.139 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.139 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112827 s, 92.9 MB/s 2024-04-17T10:47:44.149 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.150 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.150 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0087893 s, 119 MB/s 2024-04-17T10:47:44.161 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.161 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.161 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00882304 s, 119 MB/s 2024-04-17T10:47:44.174 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.174 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.174 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112835 s, 92.9 MB/s 2024-04-17T10:47:44.187 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.187 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.187 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107533 s, 97.5 MB/s 2024-04-17T10:47:44.202 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.202 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.202 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126825 s, 82.7 MB/s 2024-04-17T10:47:44.216 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.216 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.216 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117285 s, 89.4 MB/s 2024-04-17T10:47:44.226 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.227 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.227 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813848 s, 129 MB/s 2024-04-17T10:47:44.237 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.237 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.237 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830536 s, 126 MB/s 2024-04-17T10:47:44.250 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.251 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.251 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114626 s, 91.5 MB/s 2024-04-17T10:47:44.260 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.260 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.260 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776226 s, 135 MB/s 2024-04-17T10:47:44.273 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.273 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.273 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104499 s, 100 MB/s 2024-04-17T10:47:44.284 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.284 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.284 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00874805 s, 120 MB/s 2024-04-17T10:47:44.297 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.297 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.297 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106316 s, 98.6 MB/s 2024-04-17T10:47:44.306 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.307 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.307 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786495 s, 133 MB/s 2024-04-17T10:47:44.320 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.320 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.320 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011398 s, 92.0 MB/s 2024-04-17T10:47:44.334 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.334 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.334 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112826 s, 92.9 MB/s 2024-04-17T10:47:44.348 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.348 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.349 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124413 s, 84.3 MB/s 2024-04-17T10:47:44.361 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.362 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.362 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110733 s, 94.7 MB/s 2024-04-17T10:47:44.371 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.371 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.372 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786799 s, 133 MB/s 2024-04-17T10:47:44.382 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.382 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.382 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844039 s, 124 MB/s 2024-04-17T10:47:44.392 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.392 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.392 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775294 s, 135 MB/s 2024-04-17T10:47:44.405 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.405 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.405 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109312 s, 95.9 MB/s 2024-04-17T10:47:44.414 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.414 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.415 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775503 s, 135 MB/s 2024-04-17T10:47:44.428 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.428 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.428 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110959 s, 94.5 MB/s 2024-04-17T10:47:44.442 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.442 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.442 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121735 s, 86.1 MB/s 2024-04-17T10:47:44.456 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.456 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.457 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124354 s, 84.3 MB/s 2024-04-17T10:47:44.471 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.472 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.472 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126216 s, 83.1 MB/s 2024-04-17T10:47:44.481 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.481 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.481 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770476 s, 136 MB/s 2024-04-17T10:47:44.494 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.494 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.494 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106017 s, 98.9 MB/s 2024-04-17T10:47:44.508 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.508 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.508 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117718 s, 89.1 MB/s 2024-04-17T10:47:44.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:44.509 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:44.509 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:44.510 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.5 2024-04-17T10:47:44.510 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:44.510 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:44.510 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:44.510 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:44.576 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.577 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.577 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013725 s, 76.4 MB/s 2024-04-17T10:47:44.592 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.593 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.593 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136367 s, 76.9 MB/s 2024-04-17T10:47:44.606 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.606 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.606 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111311 s, 94.2 MB/s 2024-04-17T10:47:44.618 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.618 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.618 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00907097 s, 116 MB/s 2024-04-17T10:47:44.634 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.634 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.634 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139968 s, 74.9 MB/s 2024-04-17T10:47:44.648 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.648 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.648 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116086 s, 90.3 MB/s 2024-04-17T10:47:44.661 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.661 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103433 s, 101 MB/s 2024-04-17T10:47:44.677 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.677 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.677 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129072 s, 81.2 MB/s 2024-04-17T10:47:44.692 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.692 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.692 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131747 s, 79.6 MB/s 2024-04-17T10:47:44.705 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.706 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.706 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102147 s, 103 MB/s 2024-04-17T10:47:44.722 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.723 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.723 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142722 s, 73.5 MB/s 2024-04-17T10:47:44.735 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.735 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.735 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00991395 s, 106 MB/s 2024-04-17T10:47:44.748 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.748 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.748 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111204 s, 94.3 MB/s 2024-04-17T10:47:44.758 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.758 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.758 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079591 s, 132 MB/s 2024-04-17T10:47:44.774 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.774 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.774 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013061 s, 80.3 MB/s 2024-04-17T10:47:44.788 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.788 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.788 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125498 s, 83.6 MB/s 2024-04-17T10:47:44.804 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.804 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.804 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129779 s, 80.8 MB/s 2024-04-17T10:47:44.819 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.819 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.820 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134139 s, 78.2 MB/s 2024-04-17T10:47:44.830 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.830 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.830 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854322 s, 123 MB/s 2024-04-17T10:47:44.844 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.844 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.844 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104693 s, 100 MB/s 2024-04-17T10:47:44.854 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.854 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.854 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828295 s, 127 MB/s 2024-04-17T10:47:44.864 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.865 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.865 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839947 s, 125 MB/s 2024-04-17T10:47:44.879 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.879 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.879 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01182 s, 88.7 MB/s 2024-04-17T10:47:44.889 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.889 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.889 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828686 s, 127 MB/s 2024-04-17T10:47:44.902 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.902 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.902 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105464 s, 99.4 MB/s 2024-04-17T10:47:44.917 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.917 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.917 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120129 s, 87.3 MB/s 2024-04-17T10:47:44.929 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.930 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.930 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105987 s, 98.9 MB/s 2024-04-17T10:47:44.943 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.944 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.944 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116356 s, 90.1 MB/s 2024-04-17T10:47:44.957 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.957 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.957 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109729 s, 95.6 MB/s 2024-04-17T10:47:44.971 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.971 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.972 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121161 s, 86.5 MB/s 2024-04-17T10:47:44.982 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.982 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.982 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083346 s, 126 MB/s 2024-04-17T10:47:44.997 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:44.997 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:44.997 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127747 s, 82.1 MB/s 2024-04-17T10:47:44.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:44.999 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:44.999 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:44.999 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.6 2024-04-17T10:47:44.999 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:44.999 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:44.999 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:44.999 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:45.063 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.063 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.064 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134641 s, 77.9 MB/s 2024-04-17T10:47:45.077 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.077 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.078 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113407 s, 92.5 MB/s 2024-04-17T10:47:45.090 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.091 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.091 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108876 s, 96.3 MB/s 2024-04-17T10:47:45.104 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.104 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.104 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107546 s, 97.5 MB/s 2024-04-17T10:47:45.117 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.117 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.117 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109415 s, 95.8 MB/s 2024-04-17T10:47:45.128 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.128 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.128 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00889205 s, 118 MB/s 2024-04-17T10:47:45.143 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.143 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.144 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128434 s, 81.6 MB/s 2024-04-17T10:47:45.156 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.157 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.157 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107616 s, 97.4 MB/s 2024-04-17T10:47:45.174 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.174 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.174 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151125 s, 69.4 MB/s 2024-04-17T10:47:45.187 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.187 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.187 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102278 s, 103 MB/s 2024-04-17T10:47:45.201 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.201 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.201 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113291 s, 92.6 MB/s 2024-04-17T10:47:45.214 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.214 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.214 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117227 s, 89.4 MB/s 2024-04-17T10:47:45.229 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.229 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.229 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113738 s, 92.2 MB/s 2024-04-17T10:47:45.241 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.241 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.241 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108951 s, 96.2 MB/s 2024-04-17T10:47:45.256 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.256 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.256 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126497 s, 82.9 MB/s 2024-04-17T10:47:45.268 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.268 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.269 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100565 s, 104 MB/s 2024-04-17T10:47:45.281 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.281 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.282 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108173 s, 96.9 MB/s 2024-04-17T10:47:45.293 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.293 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.293 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867287 s, 121 MB/s 2024-04-17T10:47:45.307 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.307 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.307 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012491 s, 83.9 MB/s 2024-04-17T10:47:45.317 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.318 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.318 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819192 s, 128 MB/s 2024-04-17T10:47:45.330 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.330 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.330 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010301 s, 102 MB/s 2024-04-17T10:47:45.341 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.341 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.341 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00884296 s, 119 MB/s 2024-04-17T10:47:45.356 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.356 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.356 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122712 s, 85.4 MB/s 2024-04-17T10:47:45.370 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.370 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.371 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120819 s, 86.8 MB/s 2024-04-17T10:47:45.383 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.383 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.383 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105374 s, 99.5 MB/s 2024-04-17T10:47:45.397 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.397 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.397 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117543 s, 89.2 MB/s 2024-04-17T10:47:45.412 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.412 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.412 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123381 s, 85.0 MB/s 2024-04-17T10:47:45.429 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.429 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.429 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132876 s, 78.9 MB/s 2024-04-17T10:47:45.442 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.442 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.442 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101232 s, 104 MB/s 2024-04-17T10:47:45.456 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.456 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.456 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116619 s, 89.9 MB/s 2024-04-17T10:47:45.471 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.472 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.472 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121548 s, 86.3 MB/s 2024-04-17T10:47:45.487 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.487 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.487 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120634 s, 86.9 MB/s 2024-04-17T10:47:45.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:45.488 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:45.488 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:45.488 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.7 2024-04-17T10:47:45.489 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:45.489 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:45.489 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:45.489 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:45.556 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.556 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.556 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124228 s, 84.4 MB/s 2024-04-17T10:47:45.571 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.571 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.571 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121648 s, 86.2 MB/s 2024-04-17T10:47:45.586 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.586 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.587 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126055 s, 83.2 MB/s 2024-04-17T10:47:45.601 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.601 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.601 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117937 s, 88.9 MB/s 2024-04-17T10:47:45.615 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.615 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.615 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108804 s, 96.4 MB/s 2024-04-17T10:47:45.631 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.631 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.631 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013357 s, 78.5 MB/s 2024-04-17T10:47:45.645 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.645 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.646 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116132 s, 90.3 MB/s 2024-04-17T10:47:45.660 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.660 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.660 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117028 s, 89.6 MB/s 2024-04-17T10:47:45.676 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.676 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.677 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137725 s, 76.1 MB/s 2024-04-17T10:47:45.691 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.691 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.691 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121836 s, 86.1 MB/s 2024-04-17T10:47:45.707 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.707 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.707 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126056 s, 83.2 MB/s 2024-04-17T10:47:45.718 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.718 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.719 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00906439 s, 116 MB/s 2024-04-17T10:47:45.731 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.731 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.731 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00951856 s, 110 MB/s 2024-04-17T10:47:45.746 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.746 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.746 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125254 s, 83.7 MB/s 2024-04-17T10:47:45.763 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.763 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.763 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142617 s, 73.5 MB/s 2024-04-17T10:47:45.779 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.780 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.780 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132234 s, 79.3 MB/s 2024-04-17T10:47:45.795 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.795 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.795 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125356 s, 83.6 MB/s 2024-04-17T10:47:45.811 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.812 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.812 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140256 s, 74.8 MB/s 2024-04-17T10:47:45.828 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.828 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.828 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130102 s, 80.6 MB/s 2024-04-17T10:47:45.844 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.844 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.844 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013349 s, 78.6 MB/s 2024-04-17T10:47:45.861 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.861 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.861 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137419 s, 76.3 MB/s 2024-04-17T10:47:45.878 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.879 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.879 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142244 s, 73.7 MB/s 2024-04-17T10:47:45.893 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.894 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.894 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011646 s, 90.0 MB/s 2024-04-17T10:47:45.910 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.910 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.910 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133496 s, 78.5 MB/s 2024-04-17T10:47:45.923 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.924 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.924 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00998972 s, 105 MB/s 2024-04-17T10:47:45.936 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.936 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.936 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00964721 s, 109 MB/s 2024-04-17T10:47:45.951 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.951 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.951 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122375 s, 85.7 MB/s 2024-04-17T10:47:45.967 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.967 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.967 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126321 s, 83.0 MB/s 2024-04-17T10:47:45.982 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.982 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.982 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119691 s, 87.6 MB/s 2024-04-17T10:47:45.999 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:45.999 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:45.999 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137243 s, 76.4 MB/s 2024-04-17T10:47:46.014 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.014 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.014 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127768 s, 82.1 MB/s 2024-04-17T10:47:46.029 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.029 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.029 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118658 s, 88.4 MB/s 2024-04-17T10:47:46.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:46.030 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:46.030 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:46.030 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.8 2024-04-17T10:47:46.030 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:46.030 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:46.031 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:46.031 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:46.095 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.095 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.095 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108077 s, 97.0 MB/s 2024-04-17T10:47:46.110 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.110 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.110 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128675 s, 81.5 MB/s 2024-04-17T10:47:46.125 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.125 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.125 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116677 s, 89.9 MB/s 2024-04-17T10:47:46.135 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.135 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.136 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083642 s, 125 MB/s 2024-04-17T10:47:46.150 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.150 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.150 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117095 s, 89.5 MB/s 2024-04-17T10:47:46.161 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.161 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.161 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00926226 s, 113 MB/s 2024-04-17T10:47:46.174 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.175 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.175 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107816 s, 97.3 MB/s 2024-04-17T10:47:46.189 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.189 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.189 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117317 s, 89.4 MB/s 2024-04-17T10:47:46.202 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.202 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.202 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108827 s, 96.4 MB/s 2024-04-17T10:47:46.215 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.215 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.215 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110416 s, 95.0 MB/s 2024-04-17T10:47:46.228 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.228 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.228 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103546 s, 101 MB/s 2024-04-17T10:47:46.242 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.242 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.242 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116748 s, 89.8 MB/s 2024-04-17T10:47:46.256 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.256 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.256 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116121 s, 90.3 MB/s 2024-04-17T10:47:46.271 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.271 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.271 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011747 s, 89.3 MB/s 2024-04-17T10:47:46.286 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.286 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.286 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119818 s, 87.5 MB/s 2024-04-17T10:47:46.295 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.295 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.295 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830646 s, 126 MB/s 2024-04-17T10:47:46.309 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.309 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.309 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117878 s, 89.0 MB/s 2024-04-17T10:47:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.324 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.324 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125324 s, 83.7 MB/s 2024-04-17T10:47:46.334 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.334 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.334 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00858441 s, 122 MB/s 2024-04-17T10:47:46.348 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.349 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.349 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116209 s, 90.2 MB/s 2024-04-17T10:47:46.363 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.363 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.364 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127484 s, 82.3 MB/s 2024-04-17T10:47:46.375 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.375 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.375 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862935 s, 122 MB/s 2024-04-17T10:47:46.391 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.391 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.391 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123898 s, 84.6 MB/s 2024-04-17T10:47:46.408 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.408 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.408 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130894 s, 80.1 MB/s 2024-04-17T10:47:46.422 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.423 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.423 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119248 s, 87.9 MB/s 2024-04-17T10:47:46.436 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.436 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.436 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109215 s, 96.0 MB/s 2024-04-17T10:47:46.451 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.451 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.451 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125435 s, 83.6 MB/s 2024-04-17T10:47:46.465 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.465 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.465 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109413 s, 95.8 MB/s 2024-04-17T10:47:46.478 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.478 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.478 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101901 s, 103 MB/s 2024-04-17T10:47:46.492 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.492 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.492 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114678 s, 91.4 MB/s 2024-04-17T10:47:46.506 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.506 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.507 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117936 s, 88.9 MB/s 2024-04-17T10:47:46.520 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.520 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.520 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106091 s, 98.8 MB/s 2024-04-17T10:47:46.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:46.521 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:46.521 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:46.521 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.9 2024-04-17T10:47:46.521 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:46.521 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:46.521 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:46.521 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:46.590 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.591 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.591 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131556 s, 79.7 MB/s 2024-04-17T10:47:46.605 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.605 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.605 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119363 s, 87.8 MB/s 2024-04-17T10:47:46.617 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.617 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.618 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00963511 s, 109 MB/s 2024-04-17T10:47:46.632 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.632 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.632 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112808 s, 93.0 MB/s 2024-04-17T10:47:46.645 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.645 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.645 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111036 s, 94.4 MB/s 2024-04-17T10:47:46.660 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.660 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.660 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121076 s, 86.6 MB/s 2024-04-17T10:47:46.672 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.672 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.672 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00925091 s, 113 MB/s 2024-04-17T10:47:46.688 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.688 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.688 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129196 s, 81.2 MB/s 2024-04-17T10:47:46.705 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.705 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.705 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137555 s, 76.2 MB/s 2024-04-17T10:47:46.719 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.719 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.719 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114823 s, 91.3 MB/s 2024-04-17T10:47:46.733 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.734 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.734 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117384 s, 89.3 MB/s 2024-04-17T10:47:46.750 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.750 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.750 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130039 s, 80.6 MB/s 2024-04-17T10:47:46.764 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.764 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.764 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115429 s, 90.8 MB/s 2024-04-17T10:47:46.782 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.782 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.782 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139631 s, 75.1 MB/s 2024-04-17T10:47:46.796 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.796 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.796 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114837 s, 91.3 MB/s 2024-04-17T10:47:46.810 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.810 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.810 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105554 s, 99.3 MB/s 2024-04-17T10:47:46.828 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.829 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.829 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134541 s, 77.9 MB/s 2024-04-17T10:47:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012561 s, 83.5 MB/s 2024-04-17T10:47:46.855 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.855 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.855 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105706 s, 99.2 MB/s 2024-04-17T10:47:46.870 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.870 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.870 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121395 s, 86.4 MB/s 2024-04-17T10:47:46.886 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.887 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.887 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107651 s, 97.4 MB/s 2024-04-17T10:47:46.899 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.899 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.899 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126733 s, 82.7 MB/s 2024-04-17T10:47:46.910 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.910 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.911 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0091509 s, 115 MB/s 2024-04-17T10:47:46.926 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.926 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.926 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127132 s, 82.5 MB/s 2024-04-17T10:47:46.941 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.941 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.941 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124768 s, 84.0 MB/s 2024-04-17T10:47:46.956 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.956 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.956 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120447 s, 87.1 MB/s 2024-04-17T10:47:46.970 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.971 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.971 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120752 s, 86.8 MB/s 2024-04-17T10:47:46.987 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.987 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.987 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132166 s, 79.3 MB/s 2024-04-17T10:47:46.999 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:46.999 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:46.999 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00966375 s, 109 MB/s 2024-04-17T10:47:47.013 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.013 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.013 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011309 s, 92.7 MB/s 2024-04-17T10:47:47.028 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.028 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.028 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122523 s, 85.6 MB/s 2024-04-17T10:47:47.045 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.045 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.045 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131683 s, 79.6 MB/s 2024-04-17T10:47:47.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:47.046 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:47.046 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:47.046 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.10 2024-04-17T10:47:47.046 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:47.046 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:47.046 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:47.046 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:47.112 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.112 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.112 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113354 s, 92.5 MB/s 2024-04-17T10:47:47.127 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.127 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.127 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124002 s, 84.6 MB/s 2024-04-17T10:47:47.139 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.139 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.139 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00921484 s, 114 MB/s 2024-04-17T10:47:47.150 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.150 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.150 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850594 s, 123 MB/s 2024-04-17T10:47:47.163 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.163 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.164 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113003 s, 92.8 MB/s 2024-04-17T10:47:47.178 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.178 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.178 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122811 s, 85.4 MB/s 2024-04-17T10:47:47.193 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.193 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.193 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125121 s, 83.8 MB/s 2024-04-17T10:47:47.207 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.207 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.207 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01091 s, 96.1 MB/s 2024-04-17T10:47:47.220 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.220 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.220 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104961 s, 99.9 MB/s 2024-04-17T10:47:47.234 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.234 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.234 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112363 s, 93.3 MB/s 2024-04-17T10:47:47.248 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.248 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.248 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116629 s, 89.9 MB/s 2024-04-17T10:47:47.261 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.261 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.261 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104791 s, 100 MB/s 2024-04-17T10:47:47.276 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.276 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.276 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124364 s, 84.3 MB/s 2024-04-17T10:47:47.289 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.289 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.289 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104885 s, 100 MB/s 2024-04-17T10:47:47.303 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.303 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.303 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118481 s, 88.5 MB/s 2024-04-17T10:47:47.316 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.316 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.316 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107242 s, 97.8 MB/s 2024-04-17T10:47:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.331 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123389 s, 85.0 MB/s 2024-04-17T10:47:47.340 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.340 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.341 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00790381 s, 133 MB/s 2024-04-17T10:47:47.354 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.355 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.355 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114124 s, 91.9 MB/s 2024-04-17T10:47:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845513 s, 124 MB/s 2024-04-17T10:47:47.380 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.380 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.380 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123493 s, 84.9 MB/s 2024-04-17T10:47:47.394 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.394 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.394 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113944 s, 92.0 MB/s 2024-04-17T10:47:47.408 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.408 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.408 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126207 s, 83.1 MB/s 2024-04-17T10:47:47.418 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.418 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.418 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073605 s, 142 MB/s 2024-04-17T10:47:47.432 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.432 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.432 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118208 s, 88.7 MB/s 2024-04-17T10:47:47.442 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.443 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.443 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00874078 s, 120 MB/s 2024-04-17T10:47:47.453 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.453 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.453 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830118 s, 126 MB/s 2024-04-17T10:47:47.467 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.467 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.467 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116427 s, 90.1 MB/s 2024-04-17T10:47:47.480 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.480 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.480 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105988 s, 98.9 MB/s 2024-04-17T10:47:47.492 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.492 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.492 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100218 s, 105 MB/s 2024-04-17T10:47:47.502 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.502 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.502 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850991 s, 123 MB/s 2024-04-17T10:47:47.516 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.516 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.516 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115383 s, 90.9 MB/s 2024-04-17T10:47:47.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:47.517 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:47.517 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:47.517 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.11 2024-04-17T10:47:47.517 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:47.518 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:47.518 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:47.518 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:47.588 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.588 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.588 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130758 s, 80.2 MB/s 2024-04-17T10:47:47.602 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.602 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.603 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119649 s, 87.6 MB/s 2024-04-17T10:47:47.615 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.615 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.615 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105557 s, 99.3 MB/s 2024-04-17T10:47:47.628 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.629 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.629 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115764 s, 90.6 MB/s 2024-04-17T10:47:47.641 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.641 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.641 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103849 s, 101 MB/s 2024-04-17T10:47:47.652 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.652 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.652 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805331 s, 130 MB/s 2024-04-17T10:47:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.663 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.663 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00876407 s, 120 MB/s 2024-04-17T10:47:47.674 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.674 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.674 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0096286 s, 109 MB/s 2024-04-17T10:47:47.684 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.684 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.684 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748811 s, 140 MB/s 2024-04-17T10:47:47.698 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.698 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.698 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119232 s, 87.9 MB/s 2024-04-17T10:47:47.711 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.711 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.711 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105408 s, 99.5 MB/s 2024-04-17T10:47:47.722 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.723 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.723 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00969895 s, 108 MB/s 2024-04-17T10:47:47.734 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.734 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.734 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100216 s, 105 MB/s 2024-04-17T10:47:47.744 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.744 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.744 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716966 s, 146 MB/s 2024-04-17T10:47:47.756 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.756 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.756 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00989693 s, 106 MB/s 2024-04-17T10:47:47.765 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.765 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.765 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739279 s, 142 MB/s 2024-04-17T10:47:47.779 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.779 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.779 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118982 s, 88.1 MB/s 2024-04-17T10:47:47.791 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.791 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.791 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00991936 s, 106 MB/s 2024-04-17T10:47:47.804 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.804 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.804 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114184 s, 91.8 MB/s 2024-04-17T10:47:47.814 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.814 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.814 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764755 s, 137 MB/s 2024-04-17T10:47:47.828 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.828 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.828 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112646 s, 93.1 MB/s 2024-04-17T10:47:47.840 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.840 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.840 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00972586 s, 108 MB/s 2024-04-17T10:47:47.851 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.851 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.851 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00917867 s, 114 MB/s 2024-04-17T10:47:47.861 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.861 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.861 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746747 s, 140 MB/s 2024-04-17T10:47:47.875 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.875 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.875 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114932 s, 91.2 MB/s 2024-04-17T10:47:47.887 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.888 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.888 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104674 s, 100 MB/s 2024-04-17T10:47:47.901 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.901 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.901 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113992 s, 92.0 MB/s 2024-04-17T10:47:47.914 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.914 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.914 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107965 s, 97.1 MB/s 2024-04-17T10:47:47.929 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.929 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.930 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119919 s, 87.4 MB/s 2024-04-17T10:47:47.942 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.942 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.942 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118069 s, 88.8 MB/s 2024-04-17T10:47:47.958 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.958 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.958 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133128 s, 78.8 MB/s 2024-04-17T10:47:47.972 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:47.972 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:47.972 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119338 s, 87.9 MB/s 2024-04-17T10:47:47.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:47.973 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:47.973 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:47.973 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.12 2024-04-17T10:47:47.973 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:47.973 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:47.974 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:47.974 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:48.039 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.040 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.040 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845024 s, 124 MB/s 2024-04-17T10:47:48.053 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.054 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.054 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116878 s, 89.7 MB/s 2024-04-17T10:47:48.067 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.067 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.067 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114813 s, 91.3 MB/s 2024-04-17T10:47:48.080 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.080 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.081 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109989 s, 95.3 MB/s 2024-04-17T10:47:48.092 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.092 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.092 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00948042 s, 111 MB/s 2024-04-17T10:47:48.106 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.106 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.106 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108768 s, 96.4 MB/s 2024-04-17T10:47:48.119 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.119 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.119 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109219 s, 96.0 MB/s 2024-04-17T10:47:48.134 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.135 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.135 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134817 s, 77.8 MB/s 2024-04-17T10:47:48.144 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.144 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.145 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735861 s, 142 MB/s 2024-04-17T10:47:48.157 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.157 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.157 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00993532 s, 106 MB/s 2024-04-17T10:47:48.170 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.170 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.170 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107718 s, 97.3 MB/s 2024-04-17T10:47:48.182 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.182 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.182 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100506 s, 104 MB/s 2024-04-17T10:47:48.195 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.195 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.195 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112338 s, 93.3 MB/s 2024-04-17T10:47:48.209 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.209 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.210 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119449 s, 87.8 MB/s 2024-04-17T10:47:48.219 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.219 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.220 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803257 s, 131 MB/s 2024-04-17T10:47:48.228 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.229 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.229 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710325 s, 148 MB/s 2024-04-17T10:47:48.243 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.243 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.243 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121351 s, 86.4 MB/s 2024-04-17T10:47:48.256 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.256 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.256 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103893 s, 101 MB/s 2024-04-17T10:47:48.303 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.303 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.303 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114141 s, 91.9 MB/s 2024-04-17T10:47:48.320 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.320 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.320 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128801 s, 81.4 MB/s 2024-04-17T10:47:48.336 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.336 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.336 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119199 s, 88.0 MB/s 2024-04-17T10:47:48.352 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.352 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.352 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122345 s, 85.7 MB/s 2024-04-17T10:47:48.370 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.371 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.371 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140184 s, 74.8 MB/s 2024-04-17T10:47:48.388 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.388 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.388 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013104 s, 80.0 MB/s 2024-04-17T10:47:48.405 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.405 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.405 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125227 s, 83.7 MB/s 2024-04-17T10:47:48.423 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.423 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.423 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013361 s, 78.5 MB/s 2024-04-17T10:47:48.438 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.439 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.439 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010907 s, 96.1 MB/s 2024-04-17T10:47:48.453 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.453 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.453 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107751 s, 97.3 MB/s 2024-04-17T10:47:48.470 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.471 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.471 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145893 s, 71.9 MB/s 2024-04-17T10:47:48.486 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.486 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.486 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122719 s, 85.4 MB/s 2024-04-17T10:47:48.500 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.500 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.500 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109113 s, 96.1 MB/s 2024-04-17T10:47:48.513 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.513 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.513 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00988666 s, 106 MB/s 2024-04-17T10:47:48.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:48.514 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:48.514 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:48.514 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.13 2024-04-17T10:47:48.514 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:48.514 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:48.514 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:48.514 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:48.583 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.584 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.584 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132106 s, 79.4 MB/s 2024-04-17T10:47:48.597 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.598 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.598 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103857 s, 101 MB/s 2024-04-17T10:47:48.614 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.614 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.614 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131898 s, 79.5 MB/s 2024-04-17T10:47:48.630 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.630 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.630 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132182 s, 79.3 MB/s 2024-04-17T10:47:48.646 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.646 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.646 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125707 s, 83.4 MB/s 2024-04-17T10:47:48.662 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.662 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.662 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125715 s, 83.4 MB/s 2024-04-17T10:47:48.678 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.678 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.678 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127422 s, 82.3 MB/s 2024-04-17T10:47:48.691 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00997506 s, 105 MB/s 2024-04-17T10:47:48.704 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.704 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.704 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00946458 s, 111 MB/s 2024-04-17T10:47:48.721 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.721 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.721 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140153 s, 74.8 MB/s 2024-04-17T10:47:48.736 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.737 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.737 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124316 s, 84.3 MB/s 2024-04-17T10:47:48.752 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.752 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.752 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012237 s, 85.7 MB/s 2024-04-17T10:47:48.767 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.767 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.768 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126309 s, 83.0 MB/s 2024-04-17T10:47:48.782 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.782 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.783 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119878 s, 87.5 MB/s 2024-04-17T10:47:48.796 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.796 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.796 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100643 s, 104 MB/s 2024-04-17T10:47:48.812 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.812 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.812 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134197 s, 78.1 MB/s 2024-04-17T10:47:48.828 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.829 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.829 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138763 s, 75.6 MB/s 2024-04-17T10:47:48.840 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.840 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.840 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00911983 s, 115 MB/s 2024-04-17T10:47:48.853 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.853 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.854 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101409 s, 103 MB/s 2024-04-17T10:47:48.868 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.869 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.869 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119604 s, 87.7 MB/s 2024-04-17T10:47:48.880 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.880 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.880 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00892178 s, 118 MB/s 2024-04-17T10:47:48.892 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.893 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.893 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00974097 s, 108 MB/s 2024-04-17T10:47:48.906 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.906 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.906 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00964416 s, 109 MB/s 2024-04-17T10:47:48.922 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.923 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.923 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142816 s, 73.4 MB/s 2024-04-17T10:47:48.939 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.939 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.939 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128867 s, 81.4 MB/s 2024-04-17T10:47:48.954 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.954 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.954 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123471 s, 84.9 MB/s 2024-04-17T10:47:48.971 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.971 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.971 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014481 s, 72.4 MB/s 2024-04-17T10:47:48.987 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:48.987 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:48.987 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121934 s, 86.0 MB/s 2024-04-17T10:47:49.002 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.002 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.002 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116876 s, 89.7 MB/s 2024-04-17T10:47:49.018 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.018 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.018 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129052 s, 81.3 MB/s 2024-04-17T10:47:49.031 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.031 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.032 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104937 s, 99.9 MB/s 2024-04-17T10:47:49.046 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.046 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.046 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116273 s, 90.2 MB/s 2024-04-17T10:47:49.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:49.047 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:49.047 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:49.048 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.14 2024-04-17T10:47:49.048 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:49.048 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:49.048 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:49.048 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:49.112 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.112 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.112 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010113 s, 104 MB/s 2024-04-17T10:47:49.126 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.126 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.126 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109305 s, 95.9 MB/s 2024-04-17T10:47:49.140 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.140 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.140 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107961 s, 97.1 MB/s 2024-04-17T10:47:49.156 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.156 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.156 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121708 s, 86.2 MB/s 2024-04-17T10:47:49.171 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.171 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.171 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011485 s, 91.3 MB/s 2024-04-17T10:47:49.188 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.188 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.188 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140221 s, 74.8 MB/s 2024-04-17T10:47:49.206 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.206 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.206 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145104 s, 72.3 MB/s 2024-04-17T10:47:49.223 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.223 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.223 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136799 s, 76.7 MB/s 2024-04-17T10:47:49.239 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.240 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.240 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128981 s, 81.3 MB/s 2024-04-17T10:47:49.258 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.258 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.258 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135545 s, 77.4 MB/s 2024-04-17T10:47:49.308 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.308 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.308 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117043 s, 89.6 MB/s 2024-04-17T10:47:49.320 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.320 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.320 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00941349 s, 111 MB/s 2024-04-17T10:47:49.335 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.336 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.336 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122543 s, 85.6 MB/s 2024-04-17T10:47:49.350 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.350 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.350 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107012 s, 98.0 MB/s 2024-04-17T10:47:49.366 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.366 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.366 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132284 s, 79.3 MB/s 2024-04-17T10:47:49.379 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.379 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.379 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00942792 s, 111 MB/s 2024-04-17T10:47:49.394 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.394 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.394 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123583 s, 84.8 MB/s 2024-04-17T10:47:49.408 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.408 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.408 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115236 s, 91.0 MB/s 2024-04-17T10:47:49.422 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.422 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.422 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107278 s, 97.7 MB/s 2024-04-17T10:47:49.436 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.436 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.436 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117283 s, 89.4 MB/s 2024-04-17T10:47:49.451 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.451 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.451 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116718 s, 89.8 MB/s 2024-04-17T10:47:49.467 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.468 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.468 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136936 s, 76.6 MB/s 2024-04-17T10:47:49.480 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.480 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.480 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00975712 s, 107 MB/s 2024-04-17T10:47:49.495 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.495 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.495 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116535 s, 90.0 MB/s 2024-04-17T10:47:49.512 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.512 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.512 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129465 s, 81.0 MB/s 2024-04-17T10:47:49.527 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.527 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.527 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010611 s, 98.8 MB/s 2024-04-17T10:47:49.543 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.588 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.588 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012588 s, 83.3 MB/s 2024-04-17T10:47:49.589 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.590 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.590 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010644 s, 98.5 MB/s 2024-04-17T10:47:49.590 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.590 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.590 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124057 s, 84.5 MB/s 2024-04-17T10:47:49.590 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.590 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.590 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113174 s, 92.7 MB/s 2024-04-17T10:47:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116778 s, 89.8 MB/s 2024-04-17T10:47:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.618 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.618 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135243 s, 77.5 MB/s 2024-04-17T10:47:49.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:49.619 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:49.619 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:49.619 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.15 2024-04-17T10:47:49.619 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:49.619 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:49.619 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:49.619 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:49.688 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.688 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.689 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120939 s, 86.7 MB/s 2024-04-17T10:47:49.703 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.703 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.703 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011232 s, 93.4 MB/s 2024-04-17T10:47:49.717 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.717 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.717 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107205 s, 97.8 MB/s 2024-04-17T10:47:49.731 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.731 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.732 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111167 s, 94.3 MB/s 2024-04-17T10:47:49.743 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.743 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.743 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00915059 s, 115 MB/s 2024-04-17T10:47:49.756 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.756 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.756 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107781 s, 97.3 MB/s 2024-04-17T10:47:49.767 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.767 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.767 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811971 s, 129 MB/s 2024-04-17T10:47:49.778 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.778 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.778 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00883533 s, 119 MB/s 2024-04-17T10:47:49.793 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.793 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.793 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128808 s, 81.4 MB/s 2024-04-17T10:47:49.807 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.807 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.807 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114291 s, 91.7 MB/s 2024-04-17T10:47:49.820 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.820 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.820 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104066 s, 101 MB/s 2024-04-17T10:47:49.833 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.833 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.834 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104635 s, 100 MB/s 2024-04-17T10:47:49.847 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.847 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.847 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010903 s, 96.2 MB/s 2024-04-17T10:47:49.858 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.858 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.858 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792613 s, 132 MB/s 2024-04-17T10:47:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112305 s, 93.4 MB/s 2024-04-17T10:47:49.885 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.885 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.885 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103354 s, 101 MB/s 2024-04-17T10:47:49.899 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.899 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.900 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119492 s, 87.8 MB/s 2024-04-17T10:47:49.912 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.912 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.912 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101514 s, 103 MB/s 2024-04-17T10:47:49.923 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.923 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.923 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083559 s, 125 MB/s 2024-04-17T10:47:49.937 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.937 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.937 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110293 s, 95.1 MB/s 2024-04-17T10:47:49.951 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.951 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.951 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113044 s, 92.8 MB/s 2024-04-17T10:47:49.962 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.963 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.963 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00858146 s, 122 MB/s 2024-04-17T10:47:49.976 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.976 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.976 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106362 s, 98.6 MB/s 2024-04-17T10:47:49.991 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:49.991 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:49.991 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127636 s, 82.2 MB/s 2024-04-17T10:47:50.005 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.005 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.005 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105933 s, 99.0 MB/s 2024-04-17T10:47:50.019 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.019 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.019 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111372 s, 94.2 MB/s 2024-04-17T10:47:50.030 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.030 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.030 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831461 s, 126 MB/s 2024-04-17T10:47:50.045 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.045 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.045 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124535 s, 84.2 MB/s 2024-04-17T10:47:50.058 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.058 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.059 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116194 s, 90.2 MB/s 2024-04-17T10:47:50.071 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105973 s, 98.9 MB/s 2024-04-17T10:47:50.085 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.085 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.085 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108785 s, 96.4 MB/s 2024-04-17T10:47:50.099 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.099 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.099 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111418 s, 94.1 MB/s 2024-04-17T10:47:50.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:50.100 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:50.101 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:50.101 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.16 2024-04-17T10:47:50.101 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:50.101 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:50.101 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:50.101 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:50.172 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.172 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.172 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113098 s, 92.7 MB/s 2024-04-17T10:47:50.184 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.185 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.185 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0089675 s, 117 MB/s 2024-04-17T10:47:50.200 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.200 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.200 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011661 s, 89.9 MB/s 2024-04-17T10:47:50.214 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.214 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.214 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111019 s, 94.4 MB/s 2024-04-17T10:47:50.229 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.229 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.230 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126588 s, 82.8 MB/s 2024-04-17T10:47:50.242 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.242 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.242 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00885376 s, 118 MB/s 2024-04-17T10:47:50.257 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.257 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.257 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124659 s, 84.1 MB/s 2024-04-17T10:47:50.304 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.304 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.304 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103727 s, 101 MB/s 2024-04-17T10:47:50.315 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.315 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.316 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00869881 s, 121 MB/s 2024-04-17T10:47:50.332 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.332 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.332 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013699 s, 76.5 MB/s 2024-04-17T10:47:50.348 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.348 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.348 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135111 s, 77.6 MB/s 2024-04-17T10:47:50.363 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.363 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.363 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123598 s, 84.8 MB/s 2024-04-17T10:47:50.377 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.377 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.377 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108979 s, 96.2 MB/s 2024-04-17T10:47:50.394 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.394 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.394 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012059 s, 87.0 MB/s 2024-04-17T10:47:50.406 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.406 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.406 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112904 s, 92.9 MB/s 2024-04-17T10:47:50.420 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.420 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.421 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118879 s, 88.2 MB/s 2024-04-17T10:47:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00984613 s, 106 MB/s 2024-04-17T10:47:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00896802 s, 117 MB/s 2024-04-17T10:47:50.461 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.461 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.461 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129365 s, 81.1 MB/s 2024-04-17T10:47:50.472 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.473 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.473 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00920556 s, 114 MB/s 2024-04-17T10:47:50.486 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.486 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.487 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109114 s, 96.1 MB/s 2024-04-17T10:47:50.503 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.503 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.503 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01397 s, 75.1 MB/s 2024-04-17T10:47:50.518 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.518 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.519 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125356 s, 83.6 MB/s 2024-04-17T10:47:50.535 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.535 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.535 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129826 s, 80.8 MB/s 2024-04-17T10:47:50.547 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.548 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.548 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100431 s, 104 MB/s 2024-04-17T10:47:50.563 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.563 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.563 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127572 s, 82.2 MB/s 2024-04-17T10:47:50.579 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.579 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.579 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134534 s, 77.9 MB/s 2024-04-17T10:47:50.592 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.593 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.593 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102192 s, 103 MB/s 2024-04-17T10:47:50.608 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.608 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.608 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121289 s, 86.5 MB/s 2024-04-17T10:47:50.624 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.624 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.625 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142596 s, 73.5 MB/s 2024-04-17T10:47:50.637 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.638 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.638 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104007 s, 101 MB/s 2024-04-17T10:47:50.652 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.652 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.652 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119652 s, 87.6 MB/s 2024-04-17T10:47:50.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:50.654 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:50.654 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:50.654 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.17 2024-04-17T10:47:50.654 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:50.654 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:50.654 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:50.654 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:50.721 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.721 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.721 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101516 s, 103 MB/s 2024-04-17T10:47:50.734 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.734 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.734 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101483 s, 103 MB/s 2024-04-17T10:47:50.749 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.750 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.750 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126162 s, 83.1 MB/s 2024-04-17T10:47:50.763 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.763 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.763 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107641 s, 97.4 MB/s 2024-04-17T10:47:50.778 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.778 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.778 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119247 s, 87.9 MB/s 2024-04-17T10:47:50.790 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.790 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.791 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101412 s, 103 MB/s 2024-04-17T10:47:50.806 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.806 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.806 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116187 s, 90.2 MB/s 2024-04-17T10:47:50.817 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.817 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.817 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00972128 s, 108 MB/s 2024-04-17T10:47:50.833 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.833 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.833 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129512 s, 81.0 MB/s 2024-04-17T10:47:50.843 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.844 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.844 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008187 s, 128 MB/s 2024-04-17T10:47:50.859 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.860 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.860 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128218 s, 81.8 MB/s 2024-04-17T10:47:50.872 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.872 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.872 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00893913 s, 117 MB/s 2024-04-17T10:47:50.887 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.887 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.888 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123372 s, 85.0 MB/s 2024-04-17T10:47:50.901 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.901 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.901 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103434 s, 101 MB/s 2024-04-17T10:47:50.919 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.919 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.919 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014592 s, 71.9 MB/s 2024-04-17T10:47:50.933 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.933 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.933 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114808 s, 91.3 MB/s 2024-04-17T10:47:50.946 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.946 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.946 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00990987 s, 106 MB/s 2024-04-17T10:47:50.961 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.961 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.961 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116894 s, 89.7 MB/s 2024-04-17T10:47:50.976 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.977 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.977 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012086 s, 86.8 MB/s 2024-04-17T10:47:50.995 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:50.995 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:50.995 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155749 s, 67.3 MB/s 2024-04-17T10:47:51.011 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.011 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.011 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122326 s, 85.7 MB/s 2024-04-17T10:47:51.026 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.026 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.026 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116334 s, 90.1 MB/s 2024-04-17T10:47:51.040 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.040 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.041 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118222 s, 88.7 MB/s 2024-04-17T10:47:51.054 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.054 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.054 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108703 s, 96.5 MB/s 2024-04-17T10:47:51.065 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.065 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.065 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838502 s, 125 MB/s 2024-04-17T10:47:51.079 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.080 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.080 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011809 s, 88.8 MB/s 2024-04-17T10:47:51.090 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.091 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.091 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815628 s, 129 MB/s 2024-04-17T10:47:51.106 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.106 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.106 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125998 s, 83.2 MB/s 2024-04-17T10:47:51.118 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.119 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.119 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104151 s, 101 MB/s 2024-04-17T10:47:51.133 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.133 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.133 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120821 s, 86.8 MB/s 2024-04-17T10:47:51.144 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.144 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.144 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779158 s, 135 MB/s 2024-04-17T10:47:51.157 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.157 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.158 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010977 s, 95.5 MB/s 2024-04-17T10:47:51.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:51.159 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:51.159 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:51.159 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.18 2024-04-17T10:47:51.159 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:51.159 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:51.159 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:51.159 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:51.225 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.226 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.226 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102048 s, 103 MB/s 2024-04-17T10:47:51.237 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.238 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.238 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00885348 s, 118 MB/s 2024-04-17T10:47:51.250 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.251 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.251 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101562 s, 103 MB/s 2024-04-17T10:47:51.265 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.265 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.265 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104142 s, 101 MB/s 2024-04-17T10:47:51.290 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.290 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.290 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00913571 s, 115 MB/s 2024-04-17T10:47:51.363 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.363 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.363 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122985 s, 85.3 MB/s 2024-04-17T10:47:51.380 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.380 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.381 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133854 s, 78.3 MB/s 2024-04-17T10:47:51.398 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.399 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.399 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144255 s, 72.7 MB/s 2024-04-17T10:47:51.422 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.422 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.422 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167711 s, 62.5 MB/s 2024-04-17T10:47:51.437 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.437 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.437 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134745 s, 77.8 MB/s 2024-04-17T10:47:51.454 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.454 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.454 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134916 s, 77.7 MB/s 2024-04-17T10:47:51.472 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.472 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.472 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141649 s, 74.0 MB/s 2024-04-17T10:47:51.488 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.488 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.488 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130931 s, 80.1 MB/s 2024-04-17T10:47:51.506 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.506 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.506 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148292 s, 70.7 MB/s 2024-04-17T10:47:51.522 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.522 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.522 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129909 s, 80.7 MB/s 2024-04-17T10:47:51.541 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.541 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.541 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151709 s, 69.1 MB/s 2024-04-17T10:47:51.559 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.559 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.559 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139794 s, 75.0 MB/s 2024-04-17T10:47:51.577 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.577 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.578 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139005 s, 75.4 MB/s 2024-04-17T10:47:51.597 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.597 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.597 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143455 s, 73.1 MB/s 2024-04-17T10:47:51.615 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.615 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.615 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144964 s, 72.3 MB/s 2024-04-17T10:47:51.630 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.630 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.630 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112011 s, 93.6 MB/s 2024-04-17T10:47:51.647 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.647 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.647 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132015 s, 79.4 MB/s 2024-04-17T10:47:51.660 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.661 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.661 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108998 s, 96.2 MB/s 2024-04-17T10:47:51.676 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.676 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.676 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122669 s, 85.5 MB/s 2024-04-17T10:47:51.691 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.691 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.691 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012477 s, 84.0 MB/s 2024-04-17T10:47:51.707 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.707 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.707 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132136 s, 79.4 MB/s 2024-04-17T10:47:51.722 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.722 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.722 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121792 s, 86.1 MB/s 2024-04-17T10:47:51.737 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.737 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.738 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125849 s, 83.3 MB/s 2024-04-17T10:47:51.751 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.751 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.751 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107187 s, 97.8 MB/s 2024-04-17T10:47:51.767 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.767 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.767 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122641 s, 85.5 MB/s 2024-04-17T10:47:51.782 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.783 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.783 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012842 s, 81.7 MB/s 2024-04-17T10:47:51.797 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.797 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.797 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109735 s, 95.6 MB/s 2024-04-17T10:47:51.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:51.799 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:51.799 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:51.799 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.19 2024-04-17T10:47:51.799 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:51.799 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:51.799 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:51.799 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:51.870 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.870 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.870 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138935 s, 75.5 MB/s 2024-04-17T10:47:51.883 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.883 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.884 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108472 s, 96.7 MB/s 2024-04-17T10:47:51.899 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.899 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.899 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123173 s, 85.1 MB/s 2024-04-17T10:47:51.914 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.915 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.915 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123645 s, 84.8 MB/s 2024-04-17T10:47:51.929 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.929 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.930 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113613 s, 92.3 MB/s 2024-04-17T10:47:51.944 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.944 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.944 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115823 s, 90.5 MB/s 2024-04-17T10:47:51.961 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.961 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.961 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131793 s, 79.6 MB/s 2024-04-17T10:47:51.976 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.976 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.977 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118452 s, 88.5 MB/s 2024-04-17T10:47:51.990 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:51.991 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:51.991 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110961 s, 94.5 MB/s 2024-04-17T10:47:52.005 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.006 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.006 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123737 s, 84.7 MB/s 2024-04-17T10:47:52.021 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.022 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.022 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132764 s, 79.0 MB/s 2024-04-17T10:47:52.036 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.036 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.036 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010802 s, 97.1 MB/s 2024-04-17T10:47:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130767 s, 80.2 MB/s 2024-04-17T10:47:52.065 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.065 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.065 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00989634 s, 106 MB/s 2024-04-17T10:47:52.083 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.083 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.083 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148934 s, 70.4 MB/s 2024-04-17T10:47:52.097 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.097 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.097 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116461 s, 90.0 MB/s 2024-04-17T10:47:52.110 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.110 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.111 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103862 s, 101 MB/s 2024-04-17T10:47:52.125 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.125 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.125 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117781 s, 89.0 MB/s 2024-04-17T10:47:52.141 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.141 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.141 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131494 s, 79.7 MB/s 2024-04-17T10:47:52.156 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.156 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.156 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121275 s, 86.5 MB/s 2024-04-17T10:47:52.171 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.172 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.172 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130855 s, 80.1 MB/s 2024-04-17T10:47:52.185 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.186 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.186 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107361 s, 97.7 MB/s 2024-04-17T10:47:52.201 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.201 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.202 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122583 s, 85.5 MB/s 2024-04-17T10:47:52.216 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.216 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.216 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108147 s, 97.0 MB/s 2024-04-17T10:47:52.231 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.231 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.231 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113041 s, 92.8 MB/s 2024-04-17T10:47:52.247 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.247 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.247 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129155 s, 81.2 MB/s 2024-04-17T10:47:52.262 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.262 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.262 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105808 s, 99.1 MB/s 2024-04-17T10:47:52.279 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.279 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.279 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134995 s, 77.7 MB/s 2024-04-17T10:47:52.327 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.327 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.327 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140541 s, 74.6 MB/s 2024-04-17T10:47:52.343 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.343 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.343 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124766 s, 84.0 MB/s 2024-04-17T10:47:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117741 s, 89.1 MB/s 2024-04-17T10:47:52.375 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.375 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.375 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133548 s, 78.5 MB/s 2024-04-17T10:47:52.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:52.376 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:52.376 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:52.376 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.20 2024-04-17T10:47:52.376 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:52.376 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:52.376 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:52.377 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:52.445 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.445 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.445 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120514 s, 87.0 MB/s 2024-04-17T10:47:52.459 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.459 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.459 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114799 s, 91.3 MB/s 2024-04-17T10:47:52.474 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.474 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.474 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123266 s, 85.1 MB/s 2024-04-17T10:47:52.488 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.489 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.489 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110141 s, 95.2 MB/s 2024-04-17T10:47:52.506 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.506 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.506 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141184 s, 74.3 MB/s 2024-04-17T10:47:52.520 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.520 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.520 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117941 s, 88.9 MB/s 2024-04-17T10:47:52.535 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.535 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.535 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122538 s, 85.6 MB/s 2024-04-17T10:47:52.549 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.550 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.550 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115614 s, 90.7 MB/s 2024-04-17T10:47:52.561 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.561 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.561 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00924045 s, 113 MB/s 2024-04-17T10:47:52.575 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.575 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.575 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114464 s, 91.6 MB/s 2024-04-17T10:47:52.591 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.592 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.592 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135951 s, 77.1 MB/s 2024-04-17T10:47:52.603 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.604 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.604 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00933204 s, 112 MB/s 2024-04-17T10:47:52.618 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.618 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.618 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120706 s, 86.9 MB/s 2024-04-17T10:47:52.630 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.630 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.630 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00908299 s, 115 MB/s 2024-04-17T10:47:52.641 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.641 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.641 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00858201 s, 122 MB/s 2024-04-17T10:47:52.656 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.656 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.656 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121 s, 86.7 MB/s 2024-04-17T10:47:52.671 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.671 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.671 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124851 s, 84.0 MB/s 2024-04-17T10:47:52.685 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.685 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.685 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118119 s, 88.8 MB/s 2024-04-17T10:47:52.700 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.700 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.700 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012495 s, 83.9 MB/s 2024-04-17T10:47:52.712 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.712 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.713 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00915724 s, 115 MB/s 2024-04-17T10:47:52.728 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.729 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.729 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130914 s, 80.1 MB/s 2024-04-17T10:47:52.741 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.741 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.741 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0095697 s, 110 MB/s 2024-04-17T10:47:52.755 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.755 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.755 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118876 s, 88.2 MB/s 2024-04-17T10:47:52.771 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.771 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.771 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126503 s, 82.9 MB/s 2024-04-17T10:47:52.786 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.786 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.786 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012151 s, 86.3 MB/s 2024-04-17T10:47:52.800 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.800 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.801 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117684 s, 89.1 MB/s 2024-04-17T10:47:52.814 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.815 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.815 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115757 s, 90.6 MB/s 2024-04-17T10:47:52.829 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.829 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.829 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115753 s, 90.6 MB/s 2024-04-17T10:47:52.843 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.843 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.843 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114477 s, 91.6 MB/s 2024-04-17T10:47:52.858 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.858 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.859 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123645 s, 84.8 MB/s 2024-04-17T10:47:52.873 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.874 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.874 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124394 s, 84.3 MB/s 2024-04-17T10:47:52.887 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.887 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.887 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102601 s, 102 MB/s 2024-04-17T10:47:52.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:52.888 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:52.888 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:52.889 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.21 2024-04-17T10:47:52.889 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:52.889 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:52.889 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:52.889 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:52.960 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.960 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.960 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012336 s, 85.0 MB/s 2024-04-17T10:47:52.972 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.973 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.973 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00978516 s, 107 MB/s 2024-04-17T10:47:52.985 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.985 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.985 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0097473 s, 108 MB/s 2024-04-17T10:47:52.996 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:52.996 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:52.996 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00874099 s, 120 MB/s 2024-04-17T10:47:53.013 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.013 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.013 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135565 s, 77.3 MB/s 2024-04-17T10:47:53.026 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.027 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.027 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111364 s, 94.2 MB/s 2024-04-17T10:47:53.041 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.041 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.041 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114425 s, 91.6 MB/s 2024-04-17T10:47:53.057 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.058 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.058 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122222 s, 85.8 MB/s 2024-04-17T10:47:53.070 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.070 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.070 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109004 s, 96.2 MB/s 2024-04-17T10:47:53.084 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.084 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.084 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110094 s, 95.2 MB/s 2024-04-17T10:47:53.098 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.098 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.098 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108575 s, 96.6 MB/s 2024-04-17T10:47:53.113 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.113 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.113 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114317 s, 91.7 MB/s 2024-04-17T10:47:53.126 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.127 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.127 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107996 s, 97.1 MB/s 2024-04-17T10:47:53.142 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.142 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.142 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121354 s, 86.4 MB/s 2024-04-17T10:47:53.156 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.156 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.156 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011335 s, 92.5 MB/s 2024-04-17T10:47:53.172 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.172 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.172 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012496 s, 83.9 MB/s 2024-04-17T10:47:53.184 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.184 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.185 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00950689 s, 110 MB/s 2024-04-17T10:47:53.200 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.200 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.201 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122789 s, 85.4 MB/s 2024-04-17T10:47:53.214 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.214 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.214 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107009 s, 98.0 MB/s 2024-04-17T10:47:53.226 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.226 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.226 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00967357 s, 108 MB/s 2024-04-17T10:47:53.242 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.242 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.243 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129339 s, 81.1 MB/s 2024-04-17T10:47:53.257 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.258 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.258 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120941 s, 86.7 MB/s 2024-04-17T10:47:53.273 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.274 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.274 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128123 s, 81.8 MB/s 2024-04-17T10:47:53.289 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.290 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.290 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013076 s, 80.2 MB/s 2024-04-17T10:47:53.331 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.331 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.331 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105559 s, 99.3 MB/s 2024-04-17T10:47:53.346 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.346 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.346 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123479 s, 84.9 MB/s 2024-04-17T10:47:53.359 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.359 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.360 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107816 s, 97.3 MB/s 2024-04-17T10:47:53.374 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.374 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.374 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011697 s, 89.6 MB/s 2024-04-17T10:47:53.389 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.389 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.389 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119222 s, 88.0 MB/s 2024-04-17T10:47:53.404 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.404 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.404 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123069 s, 85.2 MB/s 2024-04-17T10:47:53.421 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.421 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.421 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134162 s, 78.2 MB/s 2024-04-17T10:47:53.435 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.435 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.435 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103677 s, 101 MB/s 2024-04-17T10:47:53.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:53.437 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:53.437 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:53.437 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.22 2024-04-17T10:47:53.437 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:53.437 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:53.437 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:53.438 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:53.509 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.509 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.509 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131904 s, 79.5 MB/s 2024-04-17T10:47:53.525 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.525 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.525 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120276 s, 87.2 MB/s 2024-04-17T10:47:53.540 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.540 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.540 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123177 s, 85.1 MB/s 2024-04-17T10:47:53.555 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.556 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.556 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125971 s, 83.2 MB/s 2024-04-17T10:47:53.573 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.573 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.573 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144834 s, 72.4 MB/s 2024-04-17T10:47:53.589 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.589 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.590 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134496 s, 78.0 MB/s 2024-04-17T10:47:53.605 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.606 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.606 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132271 s, 79.3 MB/s 2024-04-17T10:47:53.622 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.622 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.622 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135951 s, 77.1 MB/s 2024-04-17T10:47:53.638 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.638 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.638 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129454 s, 81.0 MB/s 2024-04-17T10:47:53.652 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.652 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.653 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119544 s, 87.7 MB/s 2024-04-17T10:47:53.667 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.667 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.667 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120372 s, 87.1 MB/s 2024-04-17T10:47:53.682 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.682 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.682 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121249 s, 86.5 MB/s 2024-04-17T10:47:53.697 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.697 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.698 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131098 s, 80.0 MB/s 2024-04-17T10:47:53.712 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.712 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.712 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113637 s, 92.3 MB/s 2024-04-17T10:47:53.729 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.729 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.729 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134194 s, 78.1 MB/s 2024-04-17T10:47:53.743 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.743 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.743 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105677 s, 99.2 MB/s 2024-04-17T10:47:53.756 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.756 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.756 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107527 s, 97.5 MB/s 2024-04-17T10:47:53.772 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.772 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.772 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126116 s, 83.1 MB/s 2024-04-17T10:47:53.786 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.786 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.786 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112207 s, 93.5 MB/s 2024-04-17T10:47:53.802 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.802 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.802 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132015 s, 79.4 MB/s 2024-04-17T10:47:53.813 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.813 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.814 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00925357 s, 113 MB/s 2024-04-17T10:47:53.829 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.829 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.829 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126063 s, 83.2 MB/s 2024-04-17T10:47:53.843 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.843 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.843 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120815 s, 86.8 MB/s 2024-04-17T10:47:53.859 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.859 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.859 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124519 s, 84.2 MB/s 2024-04-17T10:47:53.874 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.874 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.875 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129893 s, 80.7 MB/s 2024-04-17T10:47:53.887 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.887 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.887 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101276 s, 104 MB/s 2024-04-17T10:47:53.902 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.902 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.902 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123714 s, 84.8 MB/s 2024-04-17T10:47:53.917 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.917 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.918 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124274 s, 84.4 MB/s 2024-04-17T10:47:53.930 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.930 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.930 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00985948 s, 106 MB/s 2024-04-17T10:47:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.945 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.945 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120438 s, 87.1 MB/s 2024-04-17T10:47:53.957 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.957 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.957 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.009585 s, 109 MB/s 2024-04-17T10:47:53.970 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:53.971 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:53.971 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100013 s, 105 MB/s 2024-04-17T10:47:53.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:53.972 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:53.972 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:53.972 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.23 2024-04-17T10:47:53.972 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:53.972 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:53.972 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:53.972 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:54.042 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.042 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.042 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013854 s, 75.7 MB/s 2024-04-17T10:47:54.057 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.057 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.057 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124372 s, 84.3 MB/s 2024-04-17T10:47:54.072 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.072 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.072 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116156 s, 90.3 MB/s 2024-04-17T10:47:54.084 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.084 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.084 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00945939 s, 111 MB/s 2024-04-17T10:47:54.099 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.099 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.099 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114699 s, 91.4 MB/s 2024-04-17T10:47:54.114 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.114 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.114 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115385 s, 90.9 MB/s 2024-04-17T10:47:54.128 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.128 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.128 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110827 s, 94.6 MB/s 2024-04-17T10:47:54.140 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.140 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.140 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00896332 s, 117 MB/s 2024-04-17T10:47:54.153 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.154 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.154 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104741 s, 100 MB/s 2024-04-17T10:47:54.169 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.169 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.169 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122238 s, 85.8 MB/s 2024-04-17T10:47:54.182 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.182 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.182 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010499 s, 99.9 MB/s 2024-04-17T10:47:54.197 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122779 s, 85.4 MB/s 2024-04-17T10:47:54.211 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.211 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.211 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010986 s, 95.4 MB/s 2024-04-17T10:47:54.225 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.225 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.225 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115159 s, 91.1 MB/s 2024-04-17T10:47:54.239 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.239 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.239 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110807 s, 94.6 MB/s 2024-04-17T10:47:54.254 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.254 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.255 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130177 s, 80.6 MB/s 2024-04-17T10:47:54.265 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.265 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.265 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844419 s, 124 MB/s 2024-04-17T10:47:54.282 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.282 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.282 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137864 s, 76.1 MB/s 2024-04-17T10:47:54.296 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.296 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.296 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119337 s, 87.9 MB/s 2024-04-17T10:47:54.328 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.328 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.328 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871306 s, 120 MB/s 2024-04-17T10:47:54.342 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.342 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.342 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117728 s, 89.1 MB/s 2024-04-17T10:47:54.355 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.355 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.355 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00959968 s, 109 MB/s 2024-04-17T10:47:54.372 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.372 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.372 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141724 s, 74.0 MB/s 2024-04-17T10:47:54.384 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.384 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.385 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103239 s, 102 MB/s 2024-04-17T10:47:54.398 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.398 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.398 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106845 s, 98.1 MB/s 2024-04-17T10:47:54.413 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.413 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.414 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122619 s, 85.5 MB/s 2024-04-17T10:47:54.429 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.429 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.429 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119695 s, 87.6 MB/s 2024-04-17T10:47:54.444 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.444 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.444 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118259 s, 88.7 MB/s 2024-04-17T10:47:54.458 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.458 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.458 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011594 s, 90.4 MB/s 2024-04-17T10:47:54.471 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.471 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.471 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00985462 s, 106 MB/s 2024-04-17T10:47:54.487 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.487 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.487 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137886 s, 76.0 MB/s 2024-04-17T10:47:54.501 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.501 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.501 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105472 s, 99.4 MB/s 2024-04-17T10:47:54.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:54.502 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:54.502 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:54.502 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.24 2024-04-17T10:47:54.503 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:54.503 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:54.503 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:54.503 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:54.572 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.572 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.572 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130073 s, 80.6 MB/s 2024-04-17T10:47:54.586 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.586 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.586 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113582 s, 92.3 MB/s 2024-04-17T10:47:54.602 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.602 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.602 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126836 s, 82.7 MB/s 2024-04-17T10:47:54.615 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.615 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.615 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00983844 s, 107 MB/s 2024-04-17T10:47:54.632 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.632 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.632 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141981 s, 73.9 MB/s 2024-04-17T10:47:54.646 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.647 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.647 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118998 s, 88.1 MB/s 2024-04-17T10:47:54.663 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.663 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.663 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132086 s, 79.4 MB/s 2024-04-17T10:47:54.680 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.680 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139172 s, 75.3 MB/s 2024-04-17T10:47:54.694 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110546 s, 94.9 MB/s 2024-04-17T10:47:54.709 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.710 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.710 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117797 s, 89.0 MB/s 2024-04-17T10:47:54.725 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.725 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.725 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125397 s, 83.6 MB/s 2024-04-17T10:47:54.740 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.741 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.741 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129538 s, 80.9 MB/s 2024-04-17T10:47:54.752 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.752 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.753 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00913509 s, 115 MB/s 2024-04-17T10:47:54.768 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.768 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.768 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013037 s, 80.4 MB/s 2024-04-17T10:47:54.782 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.782 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.782 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112247 s, 93.4 MB/s 2024-04-17T10:47:54.796 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.796 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.796 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118381 s, 88.6 MB/s 2024-04-17T10:47:54.811 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.811 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.812 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129677 s, 80.9 MB/s 2024-04-17T10:47:54.824 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.824 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.825 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010701 s, 98.0 MB/s 2024-04-17T10:47:54.840 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.840 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.841 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013518 s, 77.6 MB/s 2024-04-17T10:47:54.850 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.850 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.850 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779642 s, 134 MB/s 2024-04-17T10:47:54.860 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.860 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.860 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793125 s, 132 MB/s 2024-04-17T10:47:54.874 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.874 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.874 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111752 s, 93.8 MB/s 2024-04-17T10:47:54.887 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.887 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.887 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109464 s, 95.8 MB/s 2024-04-17T10:47:54.900 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.900 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.900 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103938 s, 101 MB/s 2024-04-17T10:47:54.912 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.912 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.913 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010475 s, 100 MB/s 2024-04-17T10:47:54.926 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.927 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.927 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116886 s, 89.7 MB/s 2024-04-17T10:47:54.942 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.942 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.942 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130078 s, 80.6 MB/s 2024-04-17T10:47:54.954 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.954 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.955 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101829 s, 103 MB/s 2024-04-17T10:47:54.965 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.965 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.965 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079743 s, 131 MB/s 2024-04-17T10:47:54.975 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.975 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.975 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816464 s, 128 MB/s 2024-04-17T10:47:54.988 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:54.988 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:54.988 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104588 s, 100 MB/s 2024-04-17T10:47:55.001 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.001 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.001 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010625 s, 98.7 MB/s 2024-04-17T10:47:55.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:55.002 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:55.002 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:55.003 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.25 2024-04-17T10:47:55.003 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:55.003 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:55.003 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:55.003 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:55.066 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.067 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.067 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113677 s, 92.2 MB/s 2024-04-17T10:47:55.082 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.082 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.082 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127224 s, 82.4 MB/s 2024-04-17T10:47:55.093 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.093 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.093 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00874824 s, 120 MB/s 2024-04-17T10:47:55.109 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.109 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.109 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132104 s, 79.4 MB/s 2024-04-17T10:47:55.120 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.121 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.121 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00907296 s, 116 MB/s 2024-04-17T10:47:55.135 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.135 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.135 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118345 s, 88.6 MB/s 2024-04-17T10:47:55.149 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.149 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.149 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117311 s, 89.4 MB/s 2024-04-17T10:47:55.160 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.161 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.161 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00887499 s, 118 MB/s 2024-04-17T10:47:55.174 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.174 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.174 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107612 s, 97.4 MB/s 2024-04-17T10:47:55.185 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.185 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.185 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877976 s, 119 MB/s 2024-04-17T10:47:55.199 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.199 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.199 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011204 s, 93.6 MB/s 2024-04-17T10:47:55.212 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.213 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.213 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108843 s, 96.3 MB/s 2024-04-17T10:47:55.226 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.226 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.226 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111194 s, 94.3 MB/s 2024-04-17T10:47:55.237 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.237 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.237 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00907717 s, 116 MB/s 2024-04-17T10:47:55.252 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.252 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.253 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127017 s, 82.6 MB/s 2024-04-17T10:47:55.265 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.265 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.265 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104003 s, 101 MB/s 2024-04-17T10:47:55.277 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.277 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.277 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00902868 s, 116 MB/s 2024-04-17T10:47:55.290 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.290 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.290 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100403 s, 104 MB/s 2024-04-17T10:47:55.311 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.311 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.312 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116368 s, 90.1 MB/s 2024-04-17T10:47:55.328 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.328 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.328 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104348 s, 100 MB/s 2024-04-17T10:47:55.339 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.339 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.339 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880899 s, 119 MB/s 2024-04-17T10:47:55.350 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.350 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.350 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814496 s, 129 MB/s 2024-04-17T10:47:55.363 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.363 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.363 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111181 s, 94.3 MB/s 2024-04-17T10:47:55.376 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.377 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.377 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105034 s, 99.8 MB/s 2024-04-17T10:47:55.389 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.390 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.390 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109279 s, 96.0 MB/s 2024-04-17T10:47:55.400 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.400 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.400 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807667 s, 130 MB/s 2024-04-17T10:47:55.414 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.414 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.414 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012191 s, 86.0 MB/s 2024-04-17T10:47:55.430 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.431 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.431 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133719 s, 78.4 MB/s 2024-04-17T10:47:55.445 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.445 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.445 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115946 s, 90.4 MB/s 2024-04-17T10:47:55.459 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.459 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.459 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114338 s, 91.7 MB/s 2024-04-17T10:47:55.470 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.471 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.471 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00913582 s, 115 MB/s 2024-04-17T10:47:55.485 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.486 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.486 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01122 s, 93.5 MB/s 2024-04-17T10:47:55.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:55.487 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:55.487 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:55.487 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.26 2024-04-17T10:47:55.487 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:55.487 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:55.487 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:55.487 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:55.550 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.550 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.550 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00873622 s, 120 MB/s 2024-04-17T10:47:55.561 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.561 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.561 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849826 s, 123 MB/s 2024-04-17T10:47:55.576 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.576 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.576 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012519 s, 83.8 MB/s 2024-04-17T10:47:55.591 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.591 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.591 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121107 s, 86.6 MB/s 2024-04-17T10:47:55.605 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.605 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.605 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117689 s, 89.1 MB/s 2024-04-17T10:47:55.620 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.621 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.621 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121668 s, 86.2 MB/s 2024-04-17T10:47:55.636 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.637 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.637 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129583 s, 80.9 MB/s 2024-04-17T10:47:55.651 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.651 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.651 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118683 s, 88.4 MB/s 2024-04-17T10:47:55.666 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.666 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.667 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125621 s, 83.5 MB/s 2024-04-17T10:47:55.681 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.682 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.682 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124738 s, 84.1 MB/s 2024-04-17T10:47:55.695 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.695 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.695 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101517 s, 103 MB/s 2024-04-17T10:47:55.709 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.709 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.710 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116478 s, 90.0 MB/s 2024-04-17T10:47:55.724 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.724 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.724 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119573 s, 87.7 MB/s 2024-04-17T10:47:55.738 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.738 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.738 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114098 s, 91.9 MB/s 2024-04-17T10:47:55.752 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.752 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.752 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112189 s, 93.5 MB/s 2024-04-17T10:47:55.766 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.766 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.766 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111464 s, 94.1 MB/s 2024-04-17T10:47:55.779 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.779 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.779 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100562 s, 104 MB/s 2024-04-17T10:47:55.793 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.793 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.793 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109638 s, 95.6 MB/s 2024-04-17T10:47:55.808 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.808 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.808 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125626 s, 83.5 MB/s 2024-04-17T10:47:55.824 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.824 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.824 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125875 s, 83.3 MB/s 2024-04-17T10:47:55.842 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.842 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.842 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150287 s, 69.8 MB/s 2024-04-17T10:47:55.857 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.858 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.858 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130164 s, 80.6 MB/s 2024-04-17T10:47:55.870 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.870 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.870 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00956585 s, 110 MB/s 2024-04-17T10:47:55.885 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.885 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.885 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122955 s, 85.3 MB/s 2024-04-17T10:47:55.901 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.901 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.901 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127962 s, 81.9 MB/s 2024-04-17T10:47:55.914 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.915 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.915 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101737 s, 103 MB/s 2024-04-17T10:47:55.927 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.927 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.928 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.009988 s, 105 MB/s 2024-04-17T10:47:55.942 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.943 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.943 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011936 s, 87.8 MB/s 2024-04-17T10:47:55.960 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.960 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.960 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143131 s, 73.3 MB/s 2024-04-17T10:47:55.974 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.974 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.974 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114362 s, 91.7 MB/s 2024-04-17T10:47:55.988 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:55.988 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:55.989 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119156 s, 88.0 MB/s 2024-04-17T10:47:56.004 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.004 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.004 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131664 s, 79.6 MB/s 2024-04-17T10:47:56.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:56.006 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:56.006 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:56.006 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.27 2024-04-17T10:47:56.006 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:56.006 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:56.006 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:56.006 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:56.074 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.074 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.074 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124314 s, 84.3 MB/s 2024-04-17T10:47:56.090 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.090 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.090 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128087 s, 81.9 MB/s 2024-04-17T10:47:56.106 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.106 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.106 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131289 s, 79.9 MB/s 2024-04-17T10:47:56.121 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.121 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.121 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124353 s, 84.3 MB/s 2024-04-17T10:47:56.137 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.137 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.137 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120564 s, 87.0 MB/s 2024-04-17T10:47:56.153 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.154 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.154 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129463 s, 81.0 MB/s 2024-04-17T10:47:56.166 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.166 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.167 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105029 s, 99.8 MB/s 2024-04-17T10:47:56.178 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.179 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.179 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00948845 s, 111 MB/s 2024-04-17T10:47:56.195 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.195 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.195 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013322 s, 78.7 MB/s 2024-04-17T10:47:56.206 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.206 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.207 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00904158 s, 116 MB/s 2024-04-17T10:47:56.222 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.222 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.222 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131652 s, 79.6 MB/s 2024-04-17T10:47:56.236 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.236 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.236 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102867 s, 102 MB/s 2024-04-17T10:47:56.252 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.252 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.252 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130365 s, 80.4 MB/s 2024-04-17T10:47:56.264 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.264 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.264 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00927851 s, 113 MB/s 2024-04-17T10:47:56.276 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.277 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.277 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010153 s, 103 MB/s 2024-04-17T10:47:56.289 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.290 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.290 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100957 s, 104 MB/s 2024-04-17T10:47:56.305 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.305 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.305 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125989 s, 83.2 MB/s 2024-04-17T10:47:56.342 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.342 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.342 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107914 s, 97.2 MB/s 2024-04-17T10:47:56.357 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.357 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.357 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125923 s, 83.3 MB/s 2024-04-17T10:47:56.368 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.368 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.368 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870415 s, 120 MB/s 2024-04-17T10:47:56.385 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.386 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.386 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144152 s, 72.7 MB/s 2024-04-17T10:47:56.401 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.402 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.402 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132331 s, 79.2 MB/s 2024-04-17T10:47:56.414 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.414 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.414 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102613 s, 102 MB/s 2024-04-17T10:47:56.426 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.426 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.426 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00926466 s, 113 MB/s 2024-04-17T10:47:56.443 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.443 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.443 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146385 s, 71.6 MB/s 2024-04-17T10:47:56.459 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.459 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.459 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01286 s, 81.5 MB/s 2024-04-17T10:47:56.472 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.473 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.473 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01109 s, 94.6 MB/s 2024-04-17T10:47:56.487 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.487 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.487 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119372 s, 87.8 MB/s 2024-04-17T10:47:56.501 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.501 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.501 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108396 s, 96.7 MB/s 2024-04-17T10:47:56.514 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.514 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.514 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110288 s, 95.1 MB/s 2024-04-17T10:47:56.529 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.529 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.529 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121513 s, 86.3 MB/s 2024-04-17T10:47:56.544 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.544 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.544 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117647 s, 89.1 MB/s 2024-04-17T10:47:56.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:56.545 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:56.545 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:56.545 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.28 2024-04-17T10:47:56.545 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:56.545 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:56.545 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:56.545 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:56.609 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.609 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.609 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844325 s, 124 MB/s 2024-04-17T10:47:56.625 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.625 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.625 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134962 s, 77.7 MB/s 2024-04-17T10:47:56.638 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.638 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.638 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107092 s, 97.9 MB/s 2024-04-17T10:47:56.652 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.653 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.653 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122795 s, 85.4 MB/s 2024-04-17T10:47:56.664 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.664 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.664 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00890705 s, 118 MB/s 2024-04-17T10:47:56.677 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.678 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.678 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108325 s, 96.8 MB/s 2024-04-17T10:47:56.691 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.691 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.691 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105483 s, 99.4 MB/s 2024-04-17T10:47:56.705 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.705 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.706 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121749 s, 86.1 MB/s 2024-04-17T10:47:56.720 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.720 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.720 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119757 s, 87.6 MB/s 2024-04-17T10:47:56.733 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.733 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.733 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105733 s, 99.2 MB/s 2024-04-17T10:47:56.748 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.748 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.748 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122816 s, 85.4 MB/s 2024-04-17T10:47:56.762 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.762 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.762 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119441 s, 87.8 MB/s 2024-04-17T10:47:56.774 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.774 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.774 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00904989 s, 116 MB/s 2024-04-17T10:47:56.789 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.789 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.789 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012514 s, 83.8 MB/s 2024-04-17T10:47:56.800 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.800 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.800 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00911818 s, 115 MB/s 2024-04-17T10:47:56.814 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.815 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.815 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116889 s, 89.7 MB/s 2024-04-17T10:47:56.830 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.830 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.830 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123921 s, 84.6 MB/s 2024-04-17T10:47:56.844 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.845 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.845 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124559 s, 84.2 MB/s 2024-04-17T10:47:56.859 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.859 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.859 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121723 s, 86.1 MB/s 2024-04-17T10:47:56.872 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.872 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.872 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102959 s, 102 MB/s 2024-04-17T10:47:56.888 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.888 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.888 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130087 s, 80.6 MB/s 2024-04-17T10:47:56.901 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.901 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.901 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0099408 s, 105 MB/s 2024-04-17T10:47:56.915 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.915 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.916 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117035 s, 89.6 MB/s 2024-04-17T10:47:56.929 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.929 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.929 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115863 s, 90.5 MB/s 2024-04-17T10:47:56.944 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.944 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.944 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125513 s, 83.5 MB/s 2024-04-17T10:47:56.959 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.959 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.959 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121327 s, 86.4 MB/s 2024-04-17T10:47:56.974 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.975 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.975 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129219 s, 81.1 MB/s 2024-04-17T10:47:56.991 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:56.991 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:56.991 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140006 s, 74.9 MB/s 2024-04-17T10:47:57.004 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.004 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.004 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106916 s, 98.1 MB/s 2024-04-17T10:47:57.018 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.019 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.019 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114992 s, 91.2 MB/s 2024-04-17T10:47:57.034 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.034 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.034 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130209 s, 80.5 MB/s 2024-04-17T10:47:57.049 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.049 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.049 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123073 s, 85.2 MB/s 2024-04-17T10:47:57.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:57.051 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:57.051 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:57.051 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.29 2024-04-17T10:47:57.051 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:57.051 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:57.051 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:57.051 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:57.114 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.114 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.114 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00937706 s, 112 MB/s 2024-04-17T10:47:57.128 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.128 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.128 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011285 s, 92.9 MB/s 2024-04-17T10:47:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122519 s, 85.6 MB/s 2024-04-17T10:47:57.158 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.158 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.158 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129664 s, 80.9 MB/s 2024-04-17T10:47:57.173 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.173 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.173 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123231 s, 85.1 MB/s 2024-04-17T10:47:57.187 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.187 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.187 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112418 s, 93.3 MB/s 2024-04-17T10:47:57.200 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.200 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.200 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108986 s, 96.2 MB/s 2024-04-17T10:47:57.214 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.214 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.214 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112462 s, 93.2 MB/s 2024-04-17T10:47:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.226 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.226 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0089873 s, 117 MB/s 2024-04-17T10:47:57.240 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.240 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.240 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118435 s, 88.5 MB/s 2024-04-17T10:47:57.251 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.251 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.251 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00879059 s, 119 MB/s 2024-04-17T10:47:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00926466 s, 113 MB/s 2024-04-17T10:47:57.277 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.278 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.278 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114924 s, 91.2 MB/s 2024-04-17T10:47:57.291 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.292 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.292 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115905 s, 90.5 MB/s 2024-04-17T10:47:57.306 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.307 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.307 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124294 s, 84.4 MB/s 2024-04-17T10:47:57.336 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.336 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.336 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101485 s, 103 MB/s 2024-04-17T10:47:57.350 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.350 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.350 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111153 s, 94.3 MB/s 2024-04-17T10:47:57.362 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.363 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.363 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100554 s, 104 MB/s 2024-04-17T10:47:57.377 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.377 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.377 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108522 s, 96.6 MB/s 2024-04-17T10:47:57.392 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.392 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.392 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120295 s, 87.2 MB/s 2024-04-17T10:47:57.406 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.406 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.407 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115618 s, 90.7 MB/s 2024-04-17T10:47:57.422 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.422 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.422 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123992 s, 84.6 MB/s 2024-04-17T10:47:57.435 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.435 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.436 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010032 s, 105 MB/s 2024-04-17T10:47:57.449 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.450 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.450 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105703 s, 99.2 MB/s 2024-04-17T10:47:57.465 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.465 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.465 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011851 s, 88.5 MB/s 2024-04-17T10:47:57.479 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.479 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.480 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117161 s, 89.5 MB/s 2024-04-17T10:47:57.495 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.495 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.495 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121174 s, 86.5 MB/s 2024-04-17T10:47:57.507 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.507 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.507 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00967566 s, 108 MB/s 2024-04-17T10:47:57.521 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.521 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.521 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011541 s, 90.9 MB/s 2024-04-17T10:47:57.536 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.536 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.536 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121069 s, 86.6 MB/s 2024-04-17T10:47:57.552 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.552 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.552 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130056 s, 80.6 MB/s 2024-04-17T10:47:57.565 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.566 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.566 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109267 s, 96.0 MB/s 2024-04-17T10:47:57.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:57.567 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:57.567 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:57.567 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.30 2024-04-17T10:47:57.567 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:57.567 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:57.567 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:57.567 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:57.637 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.637 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.637 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137706 s, 76.1 MB/s 2024-04-17T10:47:57.652 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.652 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.652 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128972 s, 81.3 MB/s 2024-04-17T10:47:57.665 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.665 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.665 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105106 s, 99.8 MB/s 2024-04-17T10:47:57.678 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.678 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.678 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00959927 s, 109 MB/s 2024-04-17T10:47:57.691 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.691 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.691 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109128 s, 96.1 MB/s 2024-04-17T10:47:57.706 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.706 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.706 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123308 s, 85.0 MB/s 2024-04-17T10:47:57.718 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.718 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.718 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00941197 s, 111 MB/s 2024-04-17T10:47:57.735 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.735 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.735 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013663 s, 76.7 MB/s 2024-04-17T10:47:57.747 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.747 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.747 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00934849 s, 112 MB/s 2024-04-17T10:47:57.766 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.766 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.766 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129416 s, 81.0 MB/s 2024-04-17T10:47:57.779 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.779 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.779 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129735 s, 80.8 MB/s 2024-04-17T10:47:57.792 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.792 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.792 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102297 s, 103 MB/s 2024-04-17T10:47:57.805 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.805 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.805 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010789 s, 97.2 MB/s 2024-04-17T10:47:57.817 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.817 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.817 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00909282 s, 115 MB/s 2024-04-17T10:47:57.832 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.832 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.832 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118999 s, 88.1 MB/s 2024-04-17T10:47:57.847 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.847 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.847 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012713 s, 82.5 MB/s 2024-04-17T10:47:57.862 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.862 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.863 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123639 s, 84.8 MB/s 2024-04-17T10:47:57.879 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.879 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.879 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136384 s, 76.9 MB/s 2024-04-17T10:47:57.893 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.893 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.893 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108432 s, 96.7 MB/s 2024-04-17T10:47:57.907 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.907 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.907 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116787 s, 89.8 MB/s 2024-04-17T10:47:57.920 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.921 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.921 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105977 s, 98.9 MB/s 2024-04-17T10:47:57.934 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.934 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.934 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110931 s, 94.5 MB/s 2024-04-17T10:47:57.948 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.948 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.948 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103185 s, 102 MB/s 2024-04-17T10:47:57.960 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.960 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.960 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00934525 s, 112 MB/s 2024-04-17T10:47:57.974 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.974 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.975 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118209 s, 88.7 MB/s 2024-04-17T10:47:57.986 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:57.986 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:57.986 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00922651 s, 114 MB/s 2024-04-17T10:47:58.000 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.000 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.000 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110326 s, 95.0 MB/s 2024-04-17T10:47:58.011 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.011 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.011 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862833 s, 122 MB/s 2024-04-17T10:47:58.026 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.026 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.026 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118956 s, 88.1 MB/s 2024-04-17T10:47:58.041 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.042 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.042 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128557 s, 81.6 MB/s 2024-04-17T10:47:58.054 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.055 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.055 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102493 s, 102 MB/s 2024-04-17T10:47:58.068 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.068 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.068 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108353 s, 96.8 MB/s 2024-04-17T10:47:58.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:58.069 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:47:58.069 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:47:58.069 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.31 2024-04-17T10:47:58.069 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:47:58.069 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:47:58.069 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:47:58.069 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:47:58.138 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.138 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.138 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124122 s, 84.5 MB/s 2024-04-17T10:47:58.151 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.151 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.151 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010494 s, 99.9 MB/s 2024-04-17T10:47:58.163 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.163 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.163 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00887197 s, 118 MB/s 2024-04-17T10:47:58.177 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.177 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.177 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113713 s, 92.2 MB/s 2024-04-17T10:47:58.187 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.188 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.188 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815314 s, 129 MB/s 2024-04-17T10:47:58.201 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.201 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.201 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108319 s, 96.8 MB/s 2024-04-17T10:47:58.211 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.211 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.211 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832853 s, 126 MB/s 2024-04-17T10:47:58.227 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.227 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.228 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134218 s, 78.1 MB/s 2024-04-17T10:47:58.240 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.241 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.241 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108863 s, 96.3 MB/s 2024-04-17T10:47:58.254 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.255 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.255 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115666 s, 90.7 MB/s 2024-04-17T10:47:58.268 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.268 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.268 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109456 s, 95.8 MB/s 2024-04-17T10:47:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110842 s, 94.6 MB/s 2024-04-17T10:47:58.293 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.293 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.293 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00953418 s, 110 MB/s 2024-04-17T10:47:58.303 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.303 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.304 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835572 s, 125 MB/s 2024-04-17T10:47:58.318 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.318 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.318 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123973 s, 84.6 MB/s 2024-04-17T10:47:58.338 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.338 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.339 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107658 s, 97.4 MB/s 2024-04-17T10:47:58.351 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.351 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.351 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109069 s, 96.1 MB/s 2024-04-17T10:47:58.364 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.364 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.364 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108201 s, 96.9 MB/s 2024-04-17T10:47:58.379 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.379 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.379 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128955 s, 81.3 MB/s 2024-04-17T10:47:58.393 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.393 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.393 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118032 s, 88.8 MB/s 2024-04-17T10:47:58.403 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.403 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.403 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775181 s, 135 MB/s 2024-04-17T10:47:58.414 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.414 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.415 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877906 s, 119 MB/s 2024-04-17T10:47:58.429 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.429 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.429 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124275 s, 84.4 MB/s 2024-04-17T10:47:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120448 s, 87.1 MB/s 2024-04-17T10:47:58.458 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.458 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.458 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124968 s, 83.9 MB/s 2024-04-17T10:47:58.473 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.474 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.474 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140327 s, 74.7 MB/s 2024-04-17T10:47:58.484 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.484 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.484 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00876829 s, 120 MB/s 2024-04-17T10:47:58.499 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.499 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.499 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122986 s, 85.3 MB/s 2024-04-17T10:47:58.513 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.513 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.513 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011929 s, 87.9 MB/s 2024-04-17T10:47:58.526 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.526 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.527 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110953 s, 94.5 MB/s 2024-04-17T10:47:58.539 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.539 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.540 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109832 s, 95.5 MB/s 2024-04-17T10:47:58.549 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:47:58.549 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:47:58.549 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758839 s, 138 MB/s 2024-04-17T10:47:58.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:58.551 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start /scrub_control_test_path recursive 2024-04-17T10:47:58.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.718+0000 7fe1055ab640 1 Processor -- start 2024-04-17T10:47:58.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.718+0000 7fe1055ab640 1 -- start start 2024-04-17T10:47:58.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.718+0000 7fe1055ab640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe1000ced50 0x7fe10015b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.718+0000 7fe1055ab640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe10006abe0 0x7fe10006b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.718+0000 7fe1055ab640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1000cf3e0 con 0x7fe1000aebd0 2024-04-17T10:47:58.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.718+0000 7fe1055ab640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe10015bed0 con 0x7fe1000ced50 2024-04-17T10:47:58.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.718+0000 7fe1055ab640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe10015e5c0 con 0x7fe10006abe0 2024-04-17T10:47:58.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.718+0000 7fe0fe7fc640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe1000ced50 0x7fe10015b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.718+0000 7fe0fe7fc640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe1000ced50 0x7fe10015b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:49886/0 (socket says 172.21.15.83:49886) 2024-04-17T10:47:58.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.718+0000 7fe0feffd640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fe1000aebd0 0x7fe1000cd180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:42280/0 (socket says 172.21.15.83:42280) 2024-04-17T10:47:58.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.718+0000 7fe0fe7fc640 1 -- 172.21.15.83:0/2959608205 learned_addr learned my addr 172.21.15.83:0/2959608205 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:58.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.718+0000 7fe0ff7fe640 1 --2- 172.21.15.83:0/2959608205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe10006abe0 0x7fe10006b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.718+0000 7fe0fe7fc640 1 -- 172.21.15.83:0/2959608205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe10006abe0 msgr2=0x7fe10006b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:58.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.718+0000 7fe0fe7fc640 1 --2- 172.21.15.83:0/2959608205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe10006abe0 0x7fe10006b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.718+0000 7fe0fe7fc640 1 -- 172.21.15.83:0/2959608205 >> v1:172.21.15.167:6789/0 conn(0x7fe1000aebd0 legacy=0x7fe1000cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:58.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.718+0000 7fe0fe7fc640 1 -- 172.21.15.83:0/2959608205 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1000cd890 con 0x7fe1000ced50 2024-04-17T10:47:58.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.722+0000 7fe0fe7fc640 1 --2- 172.21.15.83:0/2959608205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1000ced50 0x7fe10015b990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8009870 tx=0x7fe0e801c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f19d100ab69fcc81 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:58.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.722+0000 7fe0fdffb640 1 -- 172.21.15.83:0/2959608205 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e80b2070 con 0x7fe1000ced50 2024-04-17T10:47:58.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.722+0000 7fe0fdffb640 1 -- 172.21.15.83:0/2959608205 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e80a9070 con 0x7fe1000ced50 2024-04-17T10:47:58.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.722+0000 7fe0fdffb640 1 -- 172.21.15.83:0/2959608205 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e80ad880 con 0x7fe1000ced50 2024-04-17T10:47:58.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.722+0000 7fe1055ab640 1 -- 172.21.15.83:0/2959608205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1000ced50 msgr2=0x7fe10015b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:58.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.722+0000 7fe1055ab640 1 --2- 172.21.15.83:0/2959608205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1000ced50 0x7fe10015b990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8009870 tx=0x7fe0e801c560 comp rx=0 tx=0).stop 2024-04-17T10:47:58.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.722+0000 7fe1055ab640 1 -- 172.21.15.83:0/2959608205 shutdown_connections 2024-04-17T10:47:58.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.722+0000 7fe1055ab640 1 --2- 172.21.15.83:0/2959608205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe10006abe0 0x7fe10006b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.722+0000 7fe1055ab640 1 --2- 172.21.15.83:0/2959608205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1000ced50 0x7fe10015b990 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.722+0000 7fe1055ab640 1 -- 172.21.15.83:0/2959608205 >> 172.21.15.83:0/2959608205 conn(0x7fe1000cc050 msgr2=0x7fe1000cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:58.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.722+0000 7fe1055ab640 1 -- 172.21.15.83:0/2959608205 shutdown_connections 2024-04-17T10:47:58.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.722+0000 7fe1055ab640 1 -- 172.21.15.83:0/2959608205 wait complete. 2024-04-17T10:47:58.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.722+0000 7fe1055ab640 1 Processor -- start 2024-04-17T10:47:58.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.722+0000 7fe1055ab640 1 -- start start 2024-04-17T10:47:58.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.726+0000 7fe1055ab640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe10006abe0 0x7fe1001510e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.726+0000 7fe1055ab640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1000aebd0 0x7fe100153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.726+0000 7fe1055ab640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1000ced50 0x7fe1001540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.726+0000 7fe1055ab640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10006c160 con 0x7fe1000ced50 2024-04-17T10:47:58.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.726+0000 7fe1055ab640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1000aefb0 con 0x7fe10006abe0 2024-04-17T10:47:58.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.726+0000 7fe1055ab640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe10006c2e0 con 0x7fe1000aebd0 2024-04-17T10:47:58.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.726+0000 7fe0ff7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1000ced50 0x7fe1001540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.726+0000 7fe0ff7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1000ced50 0x7fe1001540f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46104/0 (socket says 172.21.15.83:46104) 2024-04-17T10:47:58.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.726+0000 7fe0ff7fe640 1 -- 172.21.15.83:0/44778678 learned_addr learned my addr 172.21.15.83:0/44778678 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:58.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.726+0000 7fe0feffd640 1 --2- 172.21.15.83:0/44778678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe10006abe0 0x7fe1001510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.726+0000 7fe0fe7fc640 1 --2- 172.21.15.83:0/44778678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1000aebd0 0x7fe100153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.726+0000 7fe0ff7fe640 1 -- 172.21.15.83:0/44778678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1000aebd0 msgr2=0x7fe100153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:58.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.726+0000 7fe0ff7fe640 1 --2- 172.21.15.83:0/44778678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1000aebd0 0x7fe100153bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.726+0000 7fe0ff7fe640 1 -- 172.21.15.83:0/44778678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe10006abe0 msgr2=0x7fe1001510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:58.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.726+0000 7fe0ff7fe640 1 --2- 172.21.15.83:0/44778678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe10006abe0 0x7fe1001510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.726+0000 7fe0ff7fe640 1 -- 172.21.15.83:0/44778678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10015d100 con 0x7fe1000ced50 2024-04-17T10:47:58.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.726+0000 7fe0ff7fe640 1 --2- 172.21.15.83:0/44778678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1000ced50 0x7fe1001540f0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009d20 tx=0x7fe0f800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:58.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.726+0000 7fe0f77fe640 1 -- 172.21.15.83:0/44778678 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8018020 con 0x7fe1000ced50 2024-04-17T10:47:58.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.726+0000 7fe0f77fe640 1 -- 172.21.15.83:0/44778678 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f8009b10 con 0x7fe1000ced50 2024-04-17T10:47:58.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.726+0000 7fe0f77fe640 1 -- 172.21.15.83:0/44778678 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f80146c0 con 0x7fe1000ced50 2024-04-17T10:47:58.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.726+0000 7fe1055ab640 1 -- 172.21.15.83:0/44778678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe100157b90 con 0x7fe1000ced50 2024-04-17T10:47:58.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.726+0000 7fe1055ab640 1 -- 172.21.15.83:0/44778678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe100154a00 con 0x7fe1000ced50 2024-04-17T10:47:58.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.726+0000 7fe0f77fe640 1 -- 172.21.15.83:0/44778678 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe0f801b020 con 0x7fe1000ced50 2024-04-17T10:47:58.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.726+0000 7fe0f77fe640 1 --2- 172.21.15.83:0/44778678 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe0e0066f60 0x7fe0e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.726+0000 7fe0f77fe640 1 -- 172.21.15.83:0/44778678 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0f8085aa0 con 0x7fe1000ced50 2024-04-17T10:47:58.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.726+0000 7fe0feffd640 1 --2- 172.21.15.83:0/44778678 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe0e0066f60 0x7fe0e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.726+0000 7fe0feffd640 1 --2- 172.21.15.83:0/44778678 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe0e0066f60 0x7fe0e0069420 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0000f70 tx=0x7fe0f0007cd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:58.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.726+0000 7fe108718000 1 Processor -- start 2024-04-17T10:47:58.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe108718000 1 -- start start 2024-04-17T10:47:58.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe108718000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ff151b33f0 0x55ff151b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe108718000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ff151b3d10 0x55ff15169fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe108718000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff1516a520 0x55ff1516a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe108718000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff14fb7d50 con 0x55ff1516a520 2024-04-17T10:47:58.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe108718000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff14f3fe40 con 0x55ff151b33f0 2024-04-17T10:47:58.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe108718000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff14fb3df0 con 0x55ff151b3d10 2024-04-17T10:47:58.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe0ff7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff1516a520 0x55ff1516a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe0ff7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff1516a520 0x55ff1516a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46116/0 (socket says 172.21.15.83:46116) 2024-04-17T10:47:58.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe0ff7fe640 1 -- 172.21.15.83:0/3796409876 learned_addr learned my addr 172.21.15.83:0/3796409876 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:58.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe0feffd640 1 --2- 172.21.15.83:0/3796409876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ff151b33f0 0x55ff151b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe0fe7fc640 1 --2- 172.21.15.83:0/3796409876 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ff151b3d10 0x55ff15169fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe0ff7fe640 1 -- 172.21.15.83:0/3796409876 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ff151b3d10 msgr2=0x55ff15169fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:58.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe0ff7fe640 1 --2- 172.21.15.83:0/3796409876 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ff151b3d10 0x55ff15169fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe0ff7fe640 1 -- 172.21.15.83:0/3796409876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ff151b33f0 msgr2=0x55ff151b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:58.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe0ff7fe640 1 --2- 172.21.15.83:0/3796409876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ff151b33f0 0x55ff151b37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe0ff7fe640 1 -- 172.21.15.83:0/3796409876 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff14f54d90 con 0x55ff1516a520 2024-04-17T10:47:58.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe0feffd640 1 --2- 172.21.15.83:0/3796409876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ff151b33f0 0x55ff151b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:47:58.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe0fe7fc640 1 --2- 172.21.15.83:0/3796409876 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ff151b3d10 0x55ff15169fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:58.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe0ff7fe640 1 --2- 172.21.15.83:0/3796409876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff1516a520 0x55ff1516a900 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fe0f805e620 tx=0x7fe0f8056500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d31ef3cfe04011 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:58.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe0f4ff9640 1 -- 172.21.15.83:0/3796409876 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8082070 con 0x55ff1516a520 2024-04-17T10:47:58.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe0f4ff9640 1 -- 172.21.15.83:0/3796409876 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f8054740 con 0x55ff1516a520 2024-04-17T10:47:58.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe0f4ff9640 1 -- 172.21.15.83:0/3796409876 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f805a160 con 0x55ff1516a520 2024-04-17T10:47:58.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe108718000 1 -- 172.21.15.83:0/3796409876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff1516a520 msgr2=0x55ff1516a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:58.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe108718000 1 --2- 172.21.15.83:0/3796409876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff1516a520 0x55ff1516a900 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fe0f805e620 tx=0x7fe0f8056500 comp rx=0 tx=0).stop 2024-04-17T10:47:58.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe108718000 1 -- 172.21.15.83:0/3796409876 shutdown_connections 2024-04-17T10:47:58.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe108718000 1 --2- 172.21.15.83:0/3796409876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff1516a520 0x55ff1516a900 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe108718000 1 --2- 172.21.15.83:0/3796409876 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ff151b3d10 0x55ff15169fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe108718000 1 --2- 172.21.15.83:0/3796409876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ff151b33f0 0x55ff151b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe108718000 1 -- 172.21.15.83:0/3796409876 >> 172.21.15.83:0/3796409876 conn(0x55ff151664d0 msgr2=0x55ff151688c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:58.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe108718000 1 -- 172.21.15.83:0/3796409876 shutdown_connections 2024-04-17T10:47:58.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe108718000 1 -- 172.21.15.83:0/3796409876 wait complete. 2024-04-17T10:47:58.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe108718000 1 -- start start 2024-04-17T10:47:58.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe108718000 1 Processor -- start 2024-04-17T10:47:58.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe108718000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff1516a520 0x55ff1516ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe108718000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ff151b33f0 0x55ff151a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe108718000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ff151b3d10 0x55ff151a1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe108718000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff14f5d610 con 0x55ff1516a520 2024-04-17T10:47:58.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe108718000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff14f729a0 con 0x55ff151b3d10 2024-04-17T10:47:58.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe108718000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff14fbc380 con 0x55ff151b33f0 2024-04-17T10:47:58.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe108718000 10 client.0 _ll_register_callbacks cb 0x55ff1506c0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:47:58.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe0feffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff1516a520 0x55ff1516ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe0feffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff1516a520 0x55ff1516ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46118/0 (socket says 172.21.15.83:46118) 2024-04-17T10:47:58.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.730+0000 7fe0feffd640 1 -- 172.21.15.83:0/3209507871 learned_addr learned my addr 172.21.15.83:0/3209507871 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:58.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe0fe7fc640 1 --2- 172.21.15.83:0/3209507871 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ff151b33f0 0x55ff151a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe0ff7fe640 1 --2- 172.21.15.83:0/3209507871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ff151b3d10 0x55ff151a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe0feffd640 1 -- 172.21.15.83:0/3209507871 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ff151b33f0 msgr2=0x55ff151a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:58.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe0feffd640 1 --2- 172.21.15.83:0/3209507871 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ff151b33f0 0x55ff151a10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe0feffd640 1 -- 172.21.15.83:0/3209507871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ff151b3d10 msgr2=0x55ff151a1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:58.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe0feffd640 1 --2- 172.21.15.83:0/3209507871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ff151b3d10 0x55ff151a1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe0feffd640 1 -- 172.21.15.83:0/3209507871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff14fb2d40 con 0x55ff1516a520 2024-04-17T10:47:58.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe0ff7fe640 1 --2- 172.21.15.83:0/3209507871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ff151b3d10 0x55ff151a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:58.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe0feffd640 1 --2- 172.21.15.83:0/3209507871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff1516a520 0x55ff1516ffc0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0005950 tx=0x7fe0f0000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb62abc2a86d3cc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:58.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe0ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:47:58.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe0ddffb640 1 -- 172.21.15.83:0/3209507871 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0027060 con 0x55ff1516a520 2024-04-17T10:47:58.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe0ddffb640 1 -- 172.21.15.83:0/3209507871 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0f0010040 con 0x55ff1516a520 2024-04-17T10:47:58.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe108718000 1 -- 172.21.15.83:0/3209507871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0f8054740 con 0x55ff1516a520 2024-04-17T10:47:58.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe0ddffb640 1 -- 172.21.15.83:0/3209507871 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0f0005690 con 0x55ff1516a520 2024-04-17T10:47:58.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe108718000 10 client.8244 fetch_fsmap learned FSMap version 62 2024-04-17T10:47:58.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe108718000 1 -- 172.21.15.83:0/3209507871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55ff1503af00 con 0x55ff1516a520 2024-04-17T10:47:58.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe0ddffb640 1 -- 172.21.15.83:0/3209507871 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f8054740 con 0x55ff1516a520 2024-04-17T10:47:58.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe0ddffb640 1 -- 172.21.15.83:0/3209507871 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe0f0004c80 con 0x55ff1516a520 2024-04-17T10:47:58.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe108718000 10 client.8244 fetch_fsmap finished waiting for FSMap version 62 2024-04-17T10:47:58.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe108718000 10 client.8244 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-17T10:47:58.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe108718000 20 client.8244 populate_metadata read hostname 'smithi083' 2024-04-17T10:47:58.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe108718000 1 --2- 172.21.15.83:0/3209507871 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55ff151ba570 0x55ff151ba980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe108718000 4 client.8244 mds_command: new command op to 8049 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:47:58.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe108718000 1 -- 172.21.15.83:0/3209507871 --> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff14f53a20 con 0x55ff151ba570 2024-04-17T10:47:58.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe0fe7fc640 1 --2- 172.21.15.83:0/3209507871 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55ff151ba570 0x55ff151ba980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe0ddffb640 10 client.8244 ms_handle_connect on v2:172.21.15.83:6836/2353406669 2024-04-17T10:47:58.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe0fe7fc640 1 --2- 172.21.15.83:0/3209507871 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55ff151ba570 0x55ff151ba980 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46edde6cb9952037 server_cookie=16c54aaf8a3fbe3d in_seq=0 out_seq=0 2024-04-17T10:47:58.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe0ddffb640 1 -- 172.21.15.83:0/3209507871 <== mds.0 v2:172.21.15.83:6836/2353406669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ff14f53a20 con 0x55ff151ba570 2024-04-17T10:47:58.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe0ddffb640 10 client.8244 handle_command_reply: tid=0 2024-04-17T10:47:58.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe108718000 1 client.8244 shutdown 2024-04-17T10:47:58.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe108718000 1 -- 172.21.15.83:0/3209507871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff1516a520 msgr2=0x55ff1516ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:58.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe108718000 1 --2- 172.21.15.83:0/3209507871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff1516a520 0x55ff1516ffc0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0005950 tx=0x7fe0f0000bd0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe108718000 1 -- 172.21.15.83:0/3209507871 shutdown_connections 2024-04-17T10:47:58.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe108718000 1 --2- 172.21.15.83:0/3209507871 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55ff151ba570 0x55ff151ba980 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe108718000 1 --2- 172.21.15.83:0/3209507871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ff151b3d10 0x55ff151a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe108718000 1 --2- 172.21.15.83:0/3209507871 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ff151b33f0 0x55ff151a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe0ddffb640 0 client.8244 ms_handle_reset on v2:172.21.15.83:6836/2353406669 2024-04-17T10:47:58.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe108718000 1 --2- 172.21.15.83:0/3209507871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff1516a520 0x55ff1516ffc0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe108718000 1 -- 172.21.15.83:0/3209507871 >> 172.21.15.83:0/3209507871 conn(0x55ff151664d0 msgr2=0x55ff15169150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:58.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe108718000 1 -- 172.21.15.83:0/3209507871 shutdown_connections 2024-04-17T10:47:58.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe108718000 1 -- 172.21.15.83:0/3209507871 wait complete. 2024-04-17T10:47:58.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.734+0000 7fe108718000 20 client.8244 trim_cache size 0 max 16384 2024-04-17T10:47:58.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.750+0000 7fe108718000 1 Processor -- start 2024-04-17T10:47:58.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.750+0000 7fe108718000 1 -- start start 2024-04-17T10:47:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.750+0000 7fe108718000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ff1516a520 0x55ff151b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.750+0000 7fe108718000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ff151b33f0 0x55ff151b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.750+0000 7fe108718000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff151b3d10 0x55ff151b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.750+0000 7fe108718000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1002030b0 con 0x55ff151b3d10 2024-04-17T10:47:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.750+0000 7fe108718000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe100202250 con 0x55ff1516a520 2024-04-17T10:47:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.750+0000 7fe108718000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe100201710 con 0x55ff151b33f0 2024-04-17T10:47:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.750+0000 7fe0ff7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff151b3d10 0x55ff151b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.750+0000 7fe0ff7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff151b3d10 0x55ff151b6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46134/0 (socket says 172.21.15.83:46134) 2024-04-17T10:47:58.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.750+0000 7fe0ff7fe640 1 -- 172.21.15.83:0/1428533794 learned_addr learned my addr 172.21.15.83:0/1428533794 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:58.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.750+0000 7fe0ff7fe640 1 -- 172.21.15.83:0/1428533794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ff151b33f0 msgr2=0x55ff151b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:58.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.750+0000 7fe0ff7fe640 1 --2- 172.21.15.83:0/1428533794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ff151b33f0 0x55ff151b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.750+0000 7fe0ff7fe640 1 -- 172.21.15.83:0/1428533794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ff1516a520 msgr2=0x55ff151b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:58.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.750+0000 7fe0feffd640 1 --2- 172.21.15.83:0/1428533794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ff1516a520 0x55ff151b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.750+0000 7fe0ff7fe640 1 --2- 172.21.15.83:0/1428533794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ff1516a520 0x55ff151b27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.750+0000 7fe0ff7fe640 1 -- 172.21.15.83:0/1428533794 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff15011330 con 0x55ff151b3d10 2024-04-17T10:47:58.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.750+0000 7fe0ff7fe640 1 --2- 172.21.15.83:0/1428533794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff151b3d10 0x55ff151b6100 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fe0f805e620 tx=0x7fe0f8059000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a48acd9b175875c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:58.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.750+0000 7fe0dffff640 1 -- 172.21.15.83:0/1428533794 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8082070 con 0x55ff151b3d10 2024-04-17T10:47:58.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.750+0000 7fe0dffff640 1 -- 172.21.15.83:0/1428533794 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f8054740 con 0x55ff151b3d10 2024-04-17T10:47:58.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.750+0000 7fe0dffff640 1 -- 172.21.15.83:0/1428533794 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8056210 con 0x55ff151b3d10 2024-04-17T10:47:58.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.750+0000 7fe108718000 1 -- 172.21.15.83:0/1428533794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff151b3d10 msgr2=0x55ff151b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:58.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.750+0000 7fe108718000 1 --2- 172.21.15.83:0/1428533794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff151b3d10 0x55ff151b6100 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fe0f805e620 tx=0x7fe0f8059000 comp rx=0 tx=0).stop 2024-04-17T10:47:58.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.750+0000 7fe108718000 1 -- 172.21.15.83:0/1428533794 shutdown_connections 2024-04-17T10:47:58.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.750+0000 7fe108718000 1 --2- 172.21.15.83:0/1428533794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff151b3d10 0x55ff151b6100 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.750+0000 7fe108718000 1 --2- 172.21.15.83:0/1428533794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ff151b33f0 0x55ff151b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.750+0000 7fe108718000 1 --2- 172.21.15.83:0/1428533794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ff1516a520 0x55ff151b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.750+0000 7fe108718000 1 -- 172.21.15.83:0/1428533794 >> 172.21.15.83:0/1428533794 conn(0x55ff151664d0 msgr2=0x55ff151b2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:58.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.750+0000 7fe108718000 1 -- 172.21.15.83:0/1428533794 shutdown_connections 2024-04-17T10:47:58.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.750+0000 7fe108718000 1 -- 172.21.15.83:0/1428533794 wait complete. 2024-04-17T10:47:58.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.750+0000 7fe108718000 1 -- start start 2024-04-17T10:47:58.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.750+0000 7fe108718000 1 Processor -- start 2024-04-17T10:47:58.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe108718000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ff1516a520 0x55ff151b92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe108718000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff151b33f0 0x55ff151d1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe108718000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ff151b3d10 0x55ff151d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe108718000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff14f6edc0 con 0x55ff151b33f0 2024-04-17T10:47:58.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe108718000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe100201c80 con 0x55ff1516a520 2024-04-17T10:47:58.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe108718000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff14fb7060 con 0x55ff151b3d10 2024-04-17T10:47:58.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe108718000 10 client.0 _ll_register_callbacks cb 0x7fe0f8052760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:47:58.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe0fe7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff151b33f0 0x55ff151d1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe0fe7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff151b33f0 0x55ff151d1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46148/0 (socket says 172.21.15.83:46148) 2024-04-17T10:47:58.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe0fe7fc640 1 -- 172.21.15.83:0/818197989 learned_addr learned my addr 172.21.15.83:0/818197989 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:58.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe0fe7fc640 1 -- 172.21.15.83:0/818197989 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ff151b3d10 msgr2=0x55ff151d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:58.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe0fe7fc640 1 --2- 172.21.15.83:0/818197989 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ff151b3d10 0x55ff151d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe0fe7fc640 1 -- 172.21.15.83:0/818197989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ff1516a520 msgr2=0x55ff151b92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:58.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe0fe7fc640 1 --2- 172.21.15.83:0/818197989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ff1516a520 0x55ff151b92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe0fe7fc640 1 -- 172.21.15.83:0/818197989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff14f845f0 con 0x55ff151b33f0 2024-04-17T10:47:58.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe0fe7fc640 1 --2- 172.21.15.83:0/818197989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff151b33f0 0x55ff151d1020 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8009630 tx=0x7fe0e8019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955eb45a23ac94c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:58.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe0ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:47:58.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe0ddffb640 1 -- 172.21.15.83:0/818197989 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e80c3060 con 0x55ff151b33f0 2024-04-17T10:47:58.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe108718000 1 -- 172.21.15.83:0/818197989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0f8054740 con 0x55ff151b33f0 2024-04-17T10:47:58.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe0ddffb640 1 -- 172.21.15.83:0/818197989 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0e80be230 con 0x55ff151b33f0 2024-04-17T10:47:58.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe0ddffb640 1 -- 172.21.15.83:0/818197989 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0e80c9050 con 0x55ff151b33f0 2024-04-17T10:47:58.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe108718000 10 client.8250 fetch_fsmap learned FSMap version 62 2024-04-17T10:47:58.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe108718000 1 -- 172.21.15.83:0/818197989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55ff14f5efe0 con 0x55ff151b33f0 2024-04-17T10:47:58.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe0ddffb640 1 -- 172.21.15.83:0/818197989 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f8054740 con 0x55ff151b33f0 2024-04-17T10:47:58.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe0ddffb640 1 -- 172.21.15.83:0/818197989 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe0e80cc020 con 0x55ff151b33f0 2024-04-17T10:47:58.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe108718000 10 client.8250 fetch_fsmap finished waiting for FSMap version 62 2024-04-17T10:47:58.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe108718000 10 client.8250 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-17T10:47:58.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe108718000 20 client.8250 populate_metadata read hostname 'smithi083' 2024-04-17T10:47:58.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe108718000 1 --2- 172.21.15.83:0/818197989 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55ff15214630 0x55ff15214a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe108718000 4 client.8250 mds_command: new command op to 8049 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-17T10:47:58.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe108718000 1 -- 172.21.15.83:0/818197989 --> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x55ff14fc5290 con 0x55ff15214630 2024-04-17T10:47:58.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe0feffd640 1 --2- 172.21.15.83:0/818197989 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55ff15214630 0x55ff15214a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe0ddffb640 10 client.8250 ms_handle_connect on v2:172.21.15.83:6836/2353406669 2024-04-17T10:47:58.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.754+0000 7fe0feffd640 1 --2- 172.21.15.83:0/818197989 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55ff15214630 0x55ff15214a10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ae5349c902fb52a server_cookie=506f340227c367ae in_seq=0 out_seq=0 2024-04-17T10:47:58.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.758+0000 7fe0ddffb640 1 -- 172.21.15.83:0/818197989 <== mds.0 v2:172.21.15.83:6836/2353406669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ff14fc5290 con 0x55ff15214630 2024-04-17T10:47:58.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.758+0000 7fe0ddffb640 10 client.8250 handle_command_reply: tid=0 2024-04-17T10:47:58.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.758+0000 7fe108718000 1 client.8250 shutdown 2024-04-17T10:47:58.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.758+0000 7fe108718000 1 -- 172.21.15.83:0/818197989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff151b33f0 msgr2=0x55ff151d1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:58.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.758+0000 7fe108718000 1 --2- 172.21.15.83:0/818197989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff151b33f0 0x55ff151d1020 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8009630 tx=0x7fe0e8019300 comp rx=0 tx=0).stop 2024-04-17T10:47:58.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.758+0000 7fe108718000 1 -- 172.21.15.83:0/818197989 shutdown_connections 2024-04-17T10:47:58.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.758+0000 7fe108718000 1 --2- 172.21.15.83:0/818197989 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55ff15214630 0x55ff15214a10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.758+0000 7fe108718000 1 --2- 172.21.15.83:0/818197989 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ff151b3d10 0x55ff151d1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.758+0000 7fe108718000 1 --2- 172.21.15.83:0/818197989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff151b33f0 0x55ff151d1020 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.758+0000 7fe108718000 1 --2- 172.21.15.83:0/818197989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ff1516a520 0x55ff151b92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.758+0000 7fe108718000 1 -- 172.21.15.83:0/818197989 >> 172.21.15.83:0/818197989 conn(0x55ff151664d0 msgr2=0x55ff151b2920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:58.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.758+0000 7fe0ddffb640 0 client.8250 ms_handle_reset on v2:172.21.15.83:6836/2353406669 2024-04-17T10:47:58.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.758+0000 7fe108718000 1 -- 172.21.15.83:0/818197989 shutdown_connections 2024-04-17T10:47:58.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.758+0000 7fe108718000 1 -- 172.21.15.83:0/818197989 wait complete. 2024-04-17T10:47:58.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.758+0000 7fe108718000 20 client.8250 trim_cache size 0 max 16384 2024-04-17T10:47:58.757 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:47:58.758 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-17T10:47:58.758 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "1c3efbd8-8bfe-4dd7-aebe-ead10d1ddb8e", 2024-04-17T10:47:58.758 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-17T10:47:58.758 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:47:58.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.758+0000 7fe1055ab640 1 -- 172.21.15.83:0/44778678 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe0e0066f60 msgr2=0x7fe0e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:58.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.758+0000 7fe1055ab640 1 --2- 172.21.15.83:0/44778678 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe0e0066f60 0x7fe0e0069420 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0000f70 tx=0x7fe0f0007cd0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.758+0000 7fe1055ab640 1 -- 172.21.15.83:0/44778678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1000ced50 msgr2=0x7fe1001540f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:58.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.758+0000 7fe1055ab640 1 --2- 172.21.15.83:0/44778678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1000ced50 0x7fe1001540f0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009d20 tx=0x7fe0f800c490 comp rx=0 tx=0).stop 2024-04-17T10:47:58.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.758+0000 7fe1055ab640 1 -- 172.21.15.83:0/44778678 shutdown_connections 2024-04-17T10:47:58.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.758+0000 7fe1055ab640 1 --2- 172.21.15.83:0/44778678 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe0e0066f60 0x7fe0e0069420 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.758+0000 7fe1055ab640 1 --2- 172.21.15.83:0/44778678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1000ced50 0x7fe1001540f0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.758+0000 7fe1055ab640 1 --2- 172.21.15.83:0/44778678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1000aebd0 0x7fe100153bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.758+0000 7fe1055ab640 1 --2- 172.21.15.83:0/44778678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe10006abe0 0x7fe1001510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.758+0000 7fe1055ab640 1 -- 172.21.15.83:0/44778678 >> 172.21.15.83:0/44778678 conn(0x7fe1000cc050 msgr2=0x7fe10015c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:58.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.758+0000 7fe1055ab640 1 -- 172.21.15.83:0/44778678 shutdown_connections 2024-04-17T10:47:58.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.758+0000 7fe1055ab640 1 -- 172.21.15.83:0/44778678 wait complete. 2024-04-17T10:47:58.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:58.773 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub pause 2024-04-17T10:47:58.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.910+0000 7f2beafdb640 1 Processor -- start 2024-04-17T10:47:58.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.910+0000 7f2beafdb640 1 -- start start 2024-04-17T10:47:58.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.910+0000 7f2beafdb640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2be4154de0 0x7f2be406ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.910+0000 7f2beafdb640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2be406b350 0x7f2be40b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.910+0000 7f2beafdb640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2be40b4040 0x7f2be4071580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.910+0000 7f2beafdb640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be40b4870 con 0x7f2be40b4040 2024-04-17T10:47:58.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.910+0000 7f2beafdb640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be415bd70 con 0x7f2be406b350 2024-04-17T10:47:58.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.910+0000 7f2beafdb640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be415bf60 con 0x7f2be4154de0 2024-04-17T10:47:58.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.910+0000 7f2be9578640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2be40b4040 0x7f2be4071580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.910+0000 7f2be9578640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2be40b4040 0x7f2be4071580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46154/0 (socket says 172.21.15.83:46154) 2024-04-17T10:47:58.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.910+0000 7f2be9578640 1 -- 172.21.15.83:0/2671605070 learned_addr learned my addr 172.21.15.83:0/2671605070 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:58.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.910+0000 7f2be8d77640 1 --2- 172.21.15.83:0/2671605070 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2be4154de0 0x7f2be406ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.910+0000 7f2be3fff640 1 --2- 172.21.15.83:0/2671605070 >> v2:172.21.15.167:3300/0 conn(0x7f2be406b350 0x7f2be40b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.910+0000 7f2be9578640 1 -- 172.21.15.83:0/2671605070 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2be4154de0 msgr2=0x7f2be406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:58.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.910+0000 7f2be9578640 1 --2- 172.21.15.83:0/2671605070 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2be4154de0 0x7f2be406ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.910+0000 7f2be9578640 1 -- 172.21.15.83:0/2671605070 >> v2:172.21.15.167:3300/0 conn(0x7f2be406b350 msgr2=0x7f2be40b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:58.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.910+0000 7f2be9578640 1 --2- 172.21.15.83:0/2671605070 >> v2:172.21.15.167:3300/0 conn(0x7f2be406b350 0x7f2be40b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.910+0000 7f2be9578640 1 -- 172.21.15.83:0/2671605070 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be415afb0 con 0x7f2be40b4040 2024-04-17T10:47:58.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.910+0000 7f2be3fff640 1 --2- 172.21.15.83:0/2671605070 >> v2:172.21.15.167:3300/0 conn(0x7f2be406b350 0x7f2be40b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:58.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.910+0000 7f2be9578640 1 --2- 172.21.15.83:0/2671605070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2be40b4040 0x7f2be4071580 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f2be4072590 tx=0x7f2bdc01c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f60674be6f28c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:58.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.910+0000 7f2be37fe640 1 -- 172.21.15.83:0/2671605070 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc0a64d0 con 0x7f2be40b4040 2024-04-17T10:47:58.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.910+0000 7f2be37fe640 1 -- 172.21.15.83:0/2671605070 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bdc0a6670 con 0x7f2be40b4040 2024-04-17T10:47:58.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.910+0000 7f2be37fe640 1 -- 172.21.15.83:0/2671605070 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc0a6a40 con 0x7f2be40b4040 2024-04-17T10:47:58.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2beafdb640 1 -- 172.21.15.83:0/2671605070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2be40b4040 msgr2=0x7f2be4071580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:58.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2beafdb640 1 --2- 172.21.15.83:0/2671605070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2be40b4040 0x7f2be4071580 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f2be4072590 tx=0x7f2bdc01c730 comp rx=0 tx=0).stop 2024-04-17T10:47:58.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2beafdb640 1 -- 172.21.15.83:0/2671605070 shutdown_connections 2024-04-17T10:47:58.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2beafdb640 1 --2- 172.21.15.83:0/2671605070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2be40b4040 0x7f2be4071580 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2beafdb640 1 --2- 172.21.15.83:0/2671605070 >> v2:172.21.15.167:3300/0 conn(0x7f2be406b350 0x7f2be40b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2beafdb640 1 --2- 172.21.15.83:0/2671605070 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2be4154de0 0x7f2be406ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2beafdb640 1 -- 172.21.15.83:0/2671605070 >> 172.21.15.83:0/2671605070 conn(0x7f2be40b15b0 msgr2=0x7f2be40b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:58.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2beafdb640 1 -- 172.21.15.83:0/2671605070 shutdown_connections 2024-04-17T10:47:58.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2beafdb640 1 -- 172.21.15.83:0/2671605070 wait complete. 2024-04-17T10:47:58.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2beafdb640 1 Processor -- start 2024-04-17T10:47:58.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2beafdb640 1 -- start start 2024-04-17T10:47:58.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2beafdb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2be406b350 0x7f2be4146f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2beafdb640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2be40b4040 0x7f2be40ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2beafdb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2be4154de0 0x7f2be4151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2beafdb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be41551c0 con 0x7f2be4154de0 2024-04-17T10:47:58.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2beafdb640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be415b850 con 0x7f2be406b350 2024-04-17T10:47:58.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2beafdb640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be40b44d0 con 0x7f2be40b4040 2024-04-17T10:47:58.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2be9578640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2be4154de0 0x7f2be4151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2be9578640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2be4154de0 0x7f2be4151b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46156/0 (socket says 172.21.15.83:46156) 2024-04-17T10:47:58.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2be9578640 1 -- 172.21.15.83:0/2262785256 learned_addr learned my addr 172.21.15.83:0/2262785256 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:58.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2be8d77640 1 --2- 172.21.15.83:0/2262785256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2be406b350 0x7f2be4146f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2be3fff640 1 --2- 172.21.15.83:0/2262785256 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2be40b4040 0x7f2be40ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2be9578640 1 -- 172.21.15.83:0/2262785256 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2be40b4040 msgr2=0x7f2be40ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:58.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2be9578640 1 --2- 172.21.15.83:0/2262785256 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2be40b4040 0x7f2be40ccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2be9578640 1 -- 172.21.15.83:0/2262785256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2be406b350 msgr2=0x7f2be4146f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:58.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2be9578640 1 --2- 172.21.15.83:0/2262785256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2be406b350 0x7f2be4146f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2be9578640 1 -- 172.21.15.83:0/2262785256 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be414bca0 con 0x7f2be4154de0 2024-04-17T10:47:58.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2be3fff640 1 --2- 172.21.15.83:0/2262785256 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2be40b4040 0x7f2be40ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:58.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2be9578640 1 --2- 172.21.15.83:0/2262785256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2be4154de0 0x7f2be4151b40 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc01cc10 tx=0x7f2bdc0194d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:58.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2be17fa640 1 -- 172.21.15.83:0/2262785256 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc0a6c30 con 0x7f2be4154de0 2024-04-17T10:47:58.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2be17fa640 1 -- 172.21.15.83:0/2262785256 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bdc0a6df0 con 0x7f2be4154de0 2024-04-17T10:47:58.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2beafdb640 1 -- 172.21.15.83:0/2262785256 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be414bf80 con 0x7f2be4154de0 2024-04-17T10:47:58.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2be17fa640 1 -- 172.21.15.83:0/2262785256 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc0af6c0 con 0x7f2be4154de0 2024-04-17T10:47:58.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.914+0000 7f2beafdb640 1 -- 172.21.15.83:0/2262785256 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be414c540 con 0x7f2be4154de0 2024-04-17T10:47:58.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.918+0000 7f2be17fa640 1 -- 172.21.15.83:0/2262785256 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2bdc0af860 con 0x7f2be4154de0 2024-04-17T10:47:58.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.918+0000 7f2be17fa640 1 --2- 172.21.15.83:0/2262785256 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2bc8066f60 0x7f2bc8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.918+0000 7f2be17fa640 1 -- 172.21.15.83:0/2262785256 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2bdc11f650 con 0x7f2be4154de0 2024-04-17T10:47:58.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.918+0000 7f2be8d77640 1 --2- 172.21.15.83:0/2262785256 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2bc8066f60 0x7f2bc8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.918+0000 7f2be8d77640 1 --2- 172.21.15.83:0/2262785256 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2bc8066f60 0x7f2bc8069420 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800a8f0 tx=0x7f2bd8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:58.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.918+0000 7f2bee148000 1 Processor -- start 2024-04-17T10:47:58.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.918+0000 7f2bee148000 1 -- start start 2024-04-17T10:47:58.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.918+0000 7f2bee148000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7444e3a40 0x55d7444e3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.918+0000 7f2bee148000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d7444e4360 0x55d74449e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.918+0000 7f2bee148000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d74449ed90 0x55d74449f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.918+0000 7f2bee148000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7443811f0 con 0x55d7444e3a40 2024-04-17T10:47:58.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.918+0000 7f2bee148000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7442e7e90 con 0x55d74449ed90 2024-04-17T10:47:58.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.918+0000 7f2bee148000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d74426ff50 con 0x55d7444e4360 2024-04-17T10:47:58.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.918+0000 7f2be8d77640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7444e3a40 0x55d7444e3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.918+0000 7f2be8d77640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7444e3a40 0x55d7444e3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46158/0 (socket says 172.21.15.83:46158) 2024-04-17T10:47:58.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.918+0000 7f2be8d77640 1 -- 172.21.15.83:0/1965794222 learned_addr learned my addr 172.21.15.83:0/1965794222 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:58.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.918+0000 7f2be3fff640 1 --2- 172.21.15.83:0/1965794222 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d7444e4360 0x55d74449e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.918+0000 7f2be8d77640 1 -- 172.21.15.83:0/1965794222 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d7444e4360 msgr2=0x55d74449e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:58.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.918+0000 7f2be8d77640 1 --2- 172.21.15.83:0/1965794222 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d7444e4360 0x55d74449e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.918+0000 7f2be8d77640 1 -- 172.21.15.83:0/1965794222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d74449ed90 msgr2=0x55d74449f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:58.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.918+0000 7f2be9578640 1 --2- 172.21.15.83:0/1965794222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d74449ed90 0x55d74449f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.918+0000 7f2be8d77640 1 --2- 172.21.15.83:0/1965794222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d74449ed90 0x55d74449f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.918+0000 7f2be8d77640 1 -- 172.21.15.83:0/1965794222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7442bf8b0 con 0x55d7444e3a40 2024-04-17T10:47:58.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.918+0000 7f2be8d77640 1 --2- 172.21.15.83:0/1965794222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7444e3a40 0x55d7444e3e20 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800a2e0 tx=0x7f2bd8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a3be7115ccaeca server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:58.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.918+0000 7f2bce7fc640 1 -- 172.21.15.83:0/1965794222 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd801a050 con 0x55d7444e3a40 2024-04-17T10:47:58.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.918+0000 7f2bce7fc640 1 -- 172.21.15.83:0/1965794222 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd8024070 con 0x55d7444e3a40 2024-04-17T10:47:58.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.918+0000 7f2bce7fc640 1 -- 172.21.15.83:0/1965794222 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd801f440 con 0x55d7444e3a40 2024-04-17T10:47:58.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 1 -- 172.21.15.83:0/1965794222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7444e3a40 msgr2=0x55d7444e3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:58.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 1 --2- 172.21.15.83:0/1965794222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7444e3a40 0x55d7444e3e20 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800a2e0 tx=0x7f2bd8011000 comp rx=0 tx=0).stop 2024-04-17T10:47:58.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 1 -- 172.21.15.83:0/1965794222 shutdown_connections 2024-04-17T10:47:58.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 1 --2- 172.21.15.83:0/1965794222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d74449ed90 0x55d74449f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 1 --2- 172.21.15.83:0/1965794222 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d7444e4360 0x55d74449e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 1 --2- 172.21.15.83:0/1965794222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7444e3a40 0x55d7444e3e20 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 1 -- 172.21.15.83:0/1965794222 >> 172.21.15.83:0/1965794222 conn(0x55d74449ad40 msgr2=0x55d74449d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:58.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 1 -- 172.21.15.83:0/1965794222 shutdown_connections 2024-04-17T10:47:58.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 1 -- 172.21.15.83:0/1965794222 wait complete. 2024-04-17T10:47:58.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 1 -- start start 2024-04-17T10:47:58.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 1 Processor -- start 2024-04-17T10:47:58.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d74449ed90 0x55d7444eb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7444e3a40 0x55d7444eb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7444e4360 0x55d7444a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7442a2540 con 0x55d7444e4360 2024-04-17T10:47:58.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7442e3f30 con 0x55d7444e3a40 2024-04-17T10:47:58.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d744293260 con 0x55d74449ed90 2024-04-17T10:47:58.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 10 client.0 _ll_register_callbacks cb 0x55d74439c180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:47:58.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2be9578640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7444e4360 0x55d7444a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2be9578640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7444e4360 0x55d7444a4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46168/0 (socket says 172.21.15.83:46168) 2024-04-17T10:47:58.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2be9578640 1 -- 172.21.15.83:0/2612690267 learned_addr learned my addr 172.21.15.83:0/2612690267 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:58.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2be8d77640 1 --2- 172.21.15.83:0/2612690267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d74449ed90 0x55d7444eb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2be3fff640 1 --2- 172.21.15.83:0/2612690267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7444e3a40 0x55d7444eb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2be9578640 1 -- 172.21.15.83:0/2612690267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d74449ed90 msgr2=0x55d7444eb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:58.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2be9578640 1 --2- 172.21.15.83:0/2612690267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d74449ed90 0x55d7444eb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2be9578640 1 -- 172.21.15.83:0/2612690267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7444e3a40 msgr2=0x55d7444eb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:58.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2be9578640 1 --2- 172.21.15.83:0/2612690267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7444e3a40 0x55d7444eb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2be9578640 1 -- 172.21.15.83:0/2612690267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d7442701a0 con 0x55d7444e4360 2024-04-17T10:47:58.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2be9578640 1 --2- 172.21.15.83:0/2612690267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7444e4360 0x55d7444a4dc0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc01cca0 tx=0x7f2bdc0f00d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db9e458b795ecdf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:58.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bc37fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:47:58.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bc37fe640 1 -- 172.21.15.83:0/2612690267 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc0f81e0 con 0x55d7444e4360 2024-04-17T10:47:58.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bc37fe640 1 -- 172.21.15.83:0/2612690267 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2bdc0f8380 con 0x55d7444e4360 2024-04-17T10:47:58.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 1 -- 172.21.15.83:0/2612690267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2bd8024070 con 0x55d7444e4360 2024-04-17T10:47:58.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bc37fe640 1 -- 172.21.15.83:0/2612690267 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bdc0f60b0 con 0x55d7444e4360 2024-04-17T10:47:58.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 10 client.8262 fetch_fsmap learned FSMap version 62 2024-04-17T10:47:58.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 1 -- 172.21.15.83:0/2612690267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55d74436a840 con 0x55d7444e4360 2024-04-17T10:47:58.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bc37fe640 1 -- 172.21.15.83:0/2612690267 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd8024070 con 0x55d7444e4360 2024-04-17T10:47:58.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bc37fe640 1 -- 172.21.15.83:0/2612690267 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2bdc0c3030 con 0x55d7444e4360 2024-04-17T10:47:58.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 10 client.8262 fetch_fsmap finished waiting for FSMap version 62 2024-04-17T10:47:58.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 10 client.8262 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-17T10:47:58.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 20 client.8262 populate_metadata read hostname 'smithi083' 2024-04-17T10:47:58.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 1 --2- 172.21.15.83:0/2612690267 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55d7444a5b60 0x55d7444a5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 4 client.8262 mds_command: new command op to 8049 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:47:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 1 -- 172.21.15.83:0/2612690267 --> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d744283be0 con 0x55d7444a5b60 2024-04-17T10:47:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2be8d77640 1 --2- 172.21.15.83:0/2612690267 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55d7444a5b60 0x55d7444a5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bc37fe640 10 client.8262 ms_handle_connect on v2:172.21.15.83:6836/2353406669 2024-04-17T10:47:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2be8d77640 1 --2- 172.21.15.83:0/2612690267 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55d7444a5b60 0x55d7444a5f40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25d64a74180b8ba4 server_cookie=d0d899fb356b5a44 in_seq=0 out_seq=0 2024-04-17T10:47:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bc37fe640 1 -- 172.21.15.83:0/2612690267 <== mds.0 v2:172.21.15.83:6836/2353406669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d744283be0 con 0x55d7444a5b60 2024-04-17T10:47:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bc37fe640 10 client.8262 handle_command_reply: tid=0 2024-04-17T10:47:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 1 client.8262 shutdown 2024-04-17T10:47:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 1 -- 172.21.15.83:0/2612690267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7444e4360 msgr2=0x55d7444a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 1 --2- 172.21.15.83:0/2612690267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7444e4360 0x55d7444a4dc0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc01cca0 tx=0x7f2bdc0f00d0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 1 -- 172.21.15.83:0/2612690267 shutdown_connections 2024-04-17T10:47:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 1 --2- 172.21.15.83:0/2612690267 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55d7444a5b60 0x55d7444a5f40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.922+0000 7f2bee148000 1 --2- 172.21.15.83:0/2612690267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7444e4360 0x55d7444a4dc0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.926+0000 7f2bc37fe640 0 client.8262 ms_handle_reset on v2:172.21.15.83:6836/2353406669 2024-04-17T10:47:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.926+0000 7f2bee148000 1 --2- 172.21.15.83:0/2612690267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7444e3a40 0x55d7444eb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.926+0000 7f2bee148000 1 --2- 172.21.15.83:0/2612690267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d74449ed90 0x55d7444eb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.926+0000 7f2bee148000 1 -- 172.21.15.83:0/2612690267 >> 172.21.15.83:0/2612690267 conn(0x55d74449ad40 msgr2=0x55d74449d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.926+0000 7f2bee148000 1 -- 172.21.15.83:0/2612690267 shutdown_connections 2024-04-17T10:47:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.926+0000 7f2bee148000 1 -- 172.21.15.83:0/2612690267 wait complete. 2024-04-17T10:47:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.926+0000 7f2bee148000 20 client.8262 trim_cache size 0 max 16384 2024-04-17T10:47:58.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.938+0000 7f2bee148000 1 Processor -- start 2024-04-17T10:47:58.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.938+0000 7f2bee148000 1 -- start start 2024-04-17T10:47:58.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.938+0000 7f2bee148000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d74449ed90 0x55d74449e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.938+0000 7f2bee148000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7444a5b60 0x55d74449d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.938+0000 7f2bee148000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7444e3a40 0x55d74449d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.938+0000 7f2bee148000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be4200a00 con 0x55d7444e3a40 2024-04-17T10:47:58.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.938+0000 7f2bee148000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be41ffbe0 con 0x55d7444a5b60 2024-04-17T10:47:58.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.938+0000 7f2bee148000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be41ff130 con 0x55d74449ed90 2024-04-17T10:47:58.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.938+0000 7f2be9578640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7444e3a40 0x55d74449d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.938+0000 7f2be9578640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7444e3a40 0x55d74449d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46174/0 (socket says 172.21.15.83:46174) 2024-04-17T10:47:58.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.938+0000 7f2be9578640 1 -- 172.21.15.83:0/4086260192 learned_addr learned my addr 172.21.15.83:0/4086260192 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:58.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.938+0000 7f2be9578640 1 -- 172.21.15.83:0/4086260192 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d74449ed90 msgr2=0x55d74449e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.938+0000 7f2be9578640 1 --2- 172.21.15.83:0/4086260192 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d74449ed90 0x55d74449e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.938+0000 7f2be9578640 1 -- 172.21.15.83:0/4086260192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7444a5b60 msgr2=0x55d74449d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.938+0000 7f2be9578640 1 --2- 172.21.15.83:0/4086260192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7444a5b60 0x55d74449d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.938+0000 7f2be9578640 1 -- 172.21.15.83:0/4086260192 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7444e4190 con 0x55d7444e3a40 2024-04-17T10:47:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.938+0000 7f2be9578640 1 --2- 172.21.15.83:0/4086260192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7444e3a40 0x55d74449d670 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc0981e0 tx=0x7f2bdc0f3000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3049de63c22829bd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.938+0000 7f2bcdffb640 1 -- 172.21.15.83:0/4086260192 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc0efb50 con 0x55d7444e3a40 2024-04-17T10:47:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.938+0000 7f2bcdffb640 1 -- 172.21.15.83:0/4086260192 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bdc0f81e0 con 0x55d7444e3a40 2024-04-17T10:47:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.938+0000 7f2bcdffb640 1 -- 172.21.15.83:0/4086260192 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc0ec2b0 con 0x55d7444e3a40 2024-04-17T10:47:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.938+0000 7f2bee148000 1 -- 172.21.15.83:0/4086260192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7444e3a40 msgr2=0x55d74449d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.938+0000 7f2bee148000 1 --2- 172.21.15.83:0/4086260192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7444e3a40 0x55d74449d670 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc0981e0 tx=0x7f2bdc0f3000 comp rx=0 tx=0).stop 2024-04-17T10:47:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.938+0000 7f2bee148000 1 -- 172.21.15.83:0/4086260192 shutdown_connections 2024-04-17T10:47:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.938+0000 7f2bee148000 1 --2- 172.21.15.83:0/4086260192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7444e3a40 0x55d74449d670 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.938+0000 7f2bee148000 1 --2- 172.21.15.83:0/4086260192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7444a5b60 0x55d74449d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.938+0000 7f2bee148000 1 --2- 172.21.15.83:0/4086260192 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d74449ed90 0x55d74449e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.938+0000 7f2bee148000 1 -- 172.21.15.83:0/4086260192 >> 172.21.15.83:0/4086260192 conn(0x55d74449ad40 msgr2=0x55d7444e2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:58.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.938+0000 7f2bee148000 1 -- 172.21.15.83:0/4086260192 shutdown_connections 2024-04-17T10:47:58.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bee148000 1 -- 172.21.15.83:0/4086260192 wait complete. 2024-04-17T10:47:58.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bee148000 1 -- start start 2024-04-17T10:47:58.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bee148000 1 Processor -- start 2024-04-17T10:47:58.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bee148000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d74449ed90 0x55d7445444b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bee148000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7444a5b60 0x55d7445449f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bee148000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7444e3a40 0x55d7444e8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bee148000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7442e71a0 con 0x55d7444a5b60 2024-04-17T10:47:58.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bee148000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be41ff5e0 con 0x55d7444e3a40 2024-04-17T10:47:58.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bee148000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d7442c6010 con 0x55d74449ed90 2024-04-17T10:47:58.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bee148000 10 client.0 _ll_register_callbacks cb 0x55d7442aaa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:47:58.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2be3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7444a5b60 0x55d7445449f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2be3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7444a5b60 0x55d7445449f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46182/0 (socket says 172.21.15.83:46182) 2024-04-17T10:47:58.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2be3fff640 1 -- 172.21.15.83:0/4088928612 learned_addr learned my addr 172.21.15.83:0/4088928612 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:58.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2be8d77640 1 --2- 172.21.15.83:0/4088928612 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d74449ed90 0x55d7445444b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2be3fff640 1 -- 172.21.15.83:0/4088928612 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d74449ed90 msgr2=0x55d7445444b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2be3fff640 1 --2- 172.21.15.83:0/4088928612 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d74449ed90 0x55d7445444b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2be9578640 1 --2- 172.21.15.83:0/4088928612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7444e3a40 0x55d7444e8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2be3fff640 1 -- 172.21.15.83:0/4088928612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7444e3a40 msgr2=0x55d7444e8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2be3fff640 1 --2- 172.21.15.83:0/4088928612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7444e3a40 0x55d7444e8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2be3fff640 1 -- 172.21.15.83:0/4088928612 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d744341a10 con 0x55d7444a5b60 2024-04-17T10:47:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2be3fff640 1 --2- 172.21.15.83:0/4088928612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7444a5b60 0x55d7445449f0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000f2a0 tx=0x7f2bd000f760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c650b972a507b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bc37fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:47:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bc37fe640 1 -- 172.21.15.83:0/4088928612 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0012830 con 0x55d7444a5b60 2024-04-17T10:47:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bc37fe640 1 -- 172.21.15.83:0/4088928612 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2bd00129d0 con 0x55d7444a5b60 2024-04-17T10:47:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bee148000 1 -- 172.21.15.83:0/4088928612 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2bdc0f81e0 con 0x55d7444a5b60 2024-04-17T10:47:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bc37fe640 1 -- 172.21.15.83:0/4088928612 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bd0023050 con 0x55d7444a5b60 2024-04-17T10:47:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bee148000 10 client.8268 fetch_fsmap learned FSMap version 62 2024-04-17T10:47:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bee148000 1 -- 172.21.15.83:0/4088928612 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55d74427cb00 con 0x55d7444a5b60 2024-04-17T10:47:58.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bc37fe640 1 -- 172.21.15.83:0/4088928612 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bdc0f81e0 con 0x55d7444a5b60 2024-04-17T10:47:58.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bc37fe640 1 -- 172.21.15.83:0/4088928612 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2bd0026020 con 0x55d7444a5b60 2024-04-17T10:47:58.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bee148000 10 client.8268 fetch_fsmap finished waiting for FSMap version 62 2024-04-17T10:47:58.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bee148000 10 client.8268 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-17T10:47:58.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bee148000 20 client.8268 populate_metadata read hostname 'smithi083' 2024-04-17T10:47:58.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bee148000 1 --2- 172.21.15.83:0/4088928612 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55d7444e4360 0x55d7444e9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:58.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bee148000 4 client.8268 mds_command: new command op to 8049 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-17T10:47:58.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bee148000 1 -- 172.21.15.83:0/4088928612 --> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x55d7442f53d0 con 0x55d7444e4360 2024-04-17T10:47:58.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2be8d77640 1 --2- 172.21.15.83:0/4088928612 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55d7444e4360 0x55d7444e9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:58.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bc37fe640 10 client.8268 ms_handle_connect on v2:172.21.15.83:6836/2353406669 2024-04-17T10:47:58.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2be8d77640 1 --2- 172.21.15.83:0/4088928612 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55d7444e4360 0x55d7444e9c10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be2c68c83277b3b8 server_cookie=2e0ad324d92eb0e in_seq=0 out_seq=0 2024-04-17T10:47:58.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bc37fe640 1 -- 172.21.15.83:0/4088928612 <== mds.0 v2:172.21.15.83:6836/2353406669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55d7442f53d0 con 0x55d7444e4360 2024-04-17T10:47:58.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bc37fe640 10 client.8268 handle_command_reply: tid=0 2024-04-17T10:47:58.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bee148000 1 client.8268 shutdown 2024-04-17T10:47:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bee148000 1 -- 172.21.15.83:0/4088928612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7444a5b60 msgr2=0x55d7445449f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bee148000 1 --2- 172.21.15.83:0/4088928612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7444a5b60 0x55d7445449f0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000f2a0 tx=0x7f2bd000f760 comp rx=0 tx=0).stop 2024-04-17T10:47:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bee148000 1 -- 172.21.15.83:0/4088928612 shutdown_connections 2024-04-17T10:47:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bee148000 1 --2- 172.21.15.83:0/4088928612 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55d7444e4360 0x55d7444e9c10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bee148000 1 --2- 172.21.15.83:0/4088928612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7444e3a40 0x55d7444e8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bee148000 1 --2- 172.21.15.83:0/4088928612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7444a5b60 0x55d7445449f0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bee148000 1 --2- 172.21.15.83:0/4088928612 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d74449ed90 0x55d7445444b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bc37fe640 0 client.8268 ms_handle_reset on v2:172.21.15.83:6836/2353406669 2024-04-17T10:47:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bee148000 1 -- 172.21.15.83:0/4088928612 >> 172.21.15.83:0/4088928612 conn(0x55d74449ad40 msgr2=0x55d7444e29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:58.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bee148000 1 -- 172.21.15.83:0/4088928612 shutdown_connections 2024-04-17T10:47:58.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bee148000 1 -- 172.21.15.83:0/4088928612 wait complete. 2024-04-17T10:47:58.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.942+0000 7f2bee148000 20 client.8268 trim_cache size 0 max 16384 2024-04-17T10:47:58.947 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:47:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0 2024-04-17T10:47:58.947 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:47:58.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.946+0000 7f2beafdb640 1 -- 172.21.15.83:0/2262785256 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2bc8066f60 msgr2=0x7f2bc8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:58.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.946+0000 7f2beafdb640 1 --2- 172.21.15.83:0/2262785256 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2bc8066f60 0x7f2bc8069420 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800a8f0 tx=0x7f2bd8008040 comp rx=0 tx=0).stop 2024-04-17T10:47:58.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.946+0000 7f2beafdb640 1 -- 172.21.15.83:0/2262785256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2be4154de0 msgr2=0x7f2be4151b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:58.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.946+0000 7f2beafdb640 1 --2- 172.21.15.83:0/2262785256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2be4154de0 0x7f2be4151b40 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc01cc10 tx=0x7f2bdc0194d0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.946+0000 7f2beafdb640 1 -- 172.21.15.83:0/2262785256 shutdown_connections 2024-04-17T10:47:58.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.946+0000 7f2beafdb640 1 --2- 172.21.15.83:0/2262785256 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2bc8066f60 0x7f2bc8069420 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.946+0000 7f2beafdb640 1 --2- 172.21.15.83:0/2262785256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2be4154de0 0x7f2be4151b40 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.946+0000 7f2beafdb640 1 --2- 172.21.15.83:0/2262785256 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2be40b4040 0x7f2be40ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.946+0000 7f2beafdb640 1 --2- 172.21.15.83:0/2262785256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2be406b350 0x7f2be4146f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:58.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.946+0000 7f2beafdb640 1 -- 172.21.15.83:0/2262785256 >> 172.21.15.83:0/2262785256 conn(0x7f2be40b15b0 msgr2=0x7f2be41507d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:58.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.946+0000 7f2beafdb640 1 -- 172.21.15.83:0/2262785256 shutdown_connections 2024-04-17T10:47:58.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:58.946+0000 7f2beafdb640 1 -- 172.21.15.83:0/2262785256 wait complete. 2024-04-17T10:47:58.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:47:58.955 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-17T10:47:59.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.074+0000 7f84bc789640 1 Processor -- start 2024-04-17T10:47:59.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.074+0000 7f84bc789640 1 -- start start 2024-04-17T10:47:59.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.074+0000 7f84bc789640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f84b415b6e0 0x7f84b4071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:59.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.074+0000 7f84bc789640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f84b415bac0 0x7f84b40cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:59.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.074+0000 7f84bc789640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84b406c1f0 con 0x7f84b4070470 2024-04-17T10:47:59.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.074+0000 7f84bc789640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84b40cdd30 con 0x7f84b415bac0 2024-04-17T10:47:59.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.074+0000 7f84bc789640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b4070a10 con 0x7f84b415b6e0 2024-04-17T10:47:59.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.074+0000 7f84b9d24640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f84b415bac0 0x7f84b40cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:59.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.074+0000 7f84b9d24640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f84b415bac0 0x7f84b40cd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46188/0 (socket says 172.21.15.83:46188) 2024-04-17T10:47:59.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.074+0000 7f84b9d24640 1 -- 172.21.15.83:0/700346555 learned_addr learned my addr 172.21.15.83:0/700346555 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:59.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.074+0000 7f84ba525640 1 --2- 172.21.15.83:0/700346555 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f84b415b6e0 0x7f84b4071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:59.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.074+0000 7f84b9d24640 1 -- 172.21.15.83:0/700346555 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f84b415b6e0 msgr2=0x7f84b4071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:59.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.074+0000 7f84b9d24640 1 --2- 172.21.15.83:0/700346555 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f84b415b6e0 0x7f84b4071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.074+0000 7f84b9d24640 1 -- 172.21.15.83:0/700346555 >> v1:172.21.15.167:6789/0 conn(0x7f84b4070470 legacy=0x7f84b406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:59.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.074+0000 7f84b9d24640 1 -- 172.21.15.83:0/700346555 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b4070cf0 con 0x7f84b415bac0 2024-04-17T10:47:59.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.078+0000 7f84b9d24640 1 --2- 172.21.15.83:0/700346555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b415bac0 0x7f84b40cd7f0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f84b40b49f0 tx=0x7f84a401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c992e2226e569a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:59.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.078+0000 7f84b9523640 1 -- 172.21.15.83:0/700346555 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a40b0070 con 0x7f84b415bac0 2024-04-17T10:47:59.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.078+0000 7f84b9523640 1 -- 172.21.15.83:0/700346555 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a40ab600 con 0x7f84b415bac0 2024-04-17T10:47:59.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.078+0000 7f84b9523640 1 -- 172.21.15.83:0/700346555 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a40ab9d0 con 0x7f84b415bac0 2024-04-17T10:47:59.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.078+0000 7f84bc789640 1 -- 172.21.15.83:0/700346555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b415bac0 msgr2=0x7f84b40cd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:59.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.078+0000 7f84bc789640 1 --2- 172.21.15.83:0/700346555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b415bac0 0x7f84b40cd7f0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f84b40b49f0 tx=0x7f84a401c4e0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.078+0000 7f84bc789640 1 -- 172.21.15.83:0/700346555 shutdown_connections 2024-04-17T10:47:59.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.078+0000 7f84bc789640 1 --2- 172.21.15.83:0/700346555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b415bac0 0x7f84b40cd7f0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.078+0000 7f84bc789640 1 --2- 172.21.15.83:0/700346555 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f84b415b6e0 0x7f84b4071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.078+0000 7f84bc789640 1 -- 172.21.15.83:0/700346555 >> 172.21.15.83:0/700346555 conn(0x7f84b40b15b0 msgr2=0x7f84b40b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:59.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.078+0000 7f84bc789640 1 -- 172.21.15.83:0/700346555 shutdown_connections 2024-04-17T10:47:59.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.078+0000 7f84bc789640 1 -- 172.21.15.83:0/700346555 wait complete. 2024-04-17T10:47:59.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.078+0000 7f84bc789640 1 Processor -- start 2024-04-17T10:47:59.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.078+0000 7f84bc789640 1 -- start start 2024-04-17T10:47:59.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.082+0000 7f84bc789640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f84b4070470 0x7f84b4150470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:59.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.082+0000 7f84bc789640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b415b6e0 0x7f84b4144800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:59.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.082+0000 7f84bc789640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84b415bac0 0x7f84b415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:59.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.082+0000 7f84bc789640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b406cee0 con 0x7f84b415b6e0 2024-04-17T10:47:59.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.082+0000 7f84bc789640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b406b730 con 0x7f84b415bac0 2024-04-17T10:47:59.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.082+0000 7f84bc789640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b406d060 con 0x7f84b4070470 2024-04-17T10:47:59.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.082+0000 7f84b9d24640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b415b6e0 0x7f84b4144800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:59.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.082+0000 7f84b9d24640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b415b6e0 0x7f84b4144800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46190/0 (socket says 172.21.15.83:46190) 2024-04-17T10:47:59.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.082+0000 7f84b9d24640 1 -- 172.21.15.83:0/2206105694 learned_addr learned my addr 172.21.15.83:0/2206105694 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:59.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.082+0000 7f84bad26640 1 --2- 172.21.15.83:0/2206105694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84b415bac0 0x7f84b415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:59.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.082+0000 7f84ba525640 1 --2- 172.21.15.83:0/2206105694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f84b4070470 0x7f84b4150470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:59.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.082+0000 7f84b9d24640 1 -- 172.21.15.83:0/2206105694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f84b4070470 msgr2=0x7f84b4150470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:59.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.082+0000 7f84b9d24640 1 --2- 172.21.15.83:0/2206105694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f84b4070470 0x7f84b4150470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.082+0000 7f84b9d24640 1 -- 172.21.15.83:0/2206105694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84b415bac0 msgr2=0x7f84b415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:59.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.082+0000 7f84b9d24640 1 --2- 172.21.15.83:0/2206105694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84b415bac0 0x7f84b415bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.082+0000 7f84b9d24640 1 -- 172.21.15.83:0/2206105694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b4159ab0 con 0x7f84b415b6e0 2024-04-17T10:47:59.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.082+0000 7f84ba525640 1 --2- 172.21.15.83:0/2206105694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f84b4070470 0x7f84b4150470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:59.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.082+0000 7f84b9d24640 1 --2- 172.21.15.83:0/2206105694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b415b6e0 0x7f84b4144800 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f84a401cd80 tx=0x7f84a4019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:59.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.082+0000 7f84aaffd640 1 -- 172.21.15.83:0/2206105694 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a40b0040 con 0x7f84b415b6e0 2024-04-17T10:47:59.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.082+0000 7f84aaffd640 1 -- 172.21.15.83:0/2206105694 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a40b4640 con 0x7f84b415b6e0 2024-04-17T10:47:59.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.082+0000 7f84aaffd640 1 -- 172.21.15.83:0/2206105694 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a40b4a30 con 0x7f84b415b6e0 2024-04-17T10:47:59.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.082+0000 7f84bc789640 1 -- 172.21.15.83:0/2206105694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b4159d90 con 0x7f84b415b6e0 2024-04-17T10:47:59.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.082+0000 7f84bc789640 1 -- 172.21.15.83:0/2206105694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b41596f0 con 0x7f84b415b6e0 2024-04-17T10:47:59.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.082+0000 7f84aaffd640 1 -- 172.21.15.83:0/2206105694 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f84a40ab4c0 con 0x7f84b415b6e0 2024-04-17T10:47:59.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.082+0000 7f84aaffd640 1 --2- 172.21.15.83:0/2206105694 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8498066f60 0x7f8498069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:59.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.082+0000 7f84aaffd640 1 -- 172.21.15.83:0/2206105694 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f84a411ec70 con 0x7f84b415b6e0 2024-04-17T10:47:59.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.082+0000 7f84ba525640 1 --2- 172.21.15.83:0/2206105694 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8498066f60 0x7f8498069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:59.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f84ba525640 1 --2- 172.21.15.83:0/2206105694 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8498066f60 0x7f8498069420 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f84b40cc780 tx=0x7f84b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:59.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f84bf8f6000 1 Processor -- start 2024-04-17T10:47:59.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f84bf8f6000 1 -- start start 2024-04-17T10:47:59.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f84bf8f6000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7228909d0 0x55c722890db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:59.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f84bf8f6000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c7228912f0 0x55c722846b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:59.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f84bf8f6000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7228470a0 0x55c722847480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:59.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f84bf8f6000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c722693ff0 con 0x55c7228470a0 2024-04-17T10:47:59.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f84bf8f6000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c72261c150 con 0x55c7228909d0 2024-04-17T10:47:59.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f84bf8f6000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c722690090 con 0x55c7228912f0 2024-04-17T10:47:59.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f84bad26640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7228470a0 0x55c722847480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:59.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f84bad26640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7228470a0 0x55c722847480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46206/0 (socket says 172.21.15.83:46206) 2024-04-17T10:47:59.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f84bad26640 1 -- 172.21.15.83:0/158548780 learned_addr learned my addr 172.21.15.83:0/158548780 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:59.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f84b9d24640 1 --2- 172.21.15.83:0/158548780 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c7228912f0 0x55c722846b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:59.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f84ba525640 1 --2- 172.21.15.83:0/158548780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7228909d0 0x55c722890db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:59.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f84bad26640 1 -- 172.21.15.83:0/158548780 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c7228912f0 msgr2=0x55c722846b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:59.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f84bad26640 1 --2- 172.21.15.83:0/158548780 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c7228912f0 0x55c722846b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f84bad26640 1 -- 172.21.15.83:0/158548780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7228909d0 msgr2=0x55c722890db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:59.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f84bad26640 1 --2- 172.21.15.83:0/158548780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7228909d0 0x55c722890db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f84bad26640 1 -- 172.21.15.83:0/158548780 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c72282a0d0 con 0x55c7228470a0 2024-04-17T10:47:59.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f84ba525640 1 --2- 172.21.15.83:0/158548780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7228909d0 0x55c722890db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:59.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f84bad26640 1 --2- 172.21.15.83:0/158548780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7228470a0 0x55c722847480 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f84ac0053f0 tx=0x7f84ac0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb242cf31b796ca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:59.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f8497fff640 1 -- 172.21.15.83:0/158548780 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac006d70 con 0x55c7228470a0 2024-04-17T10:47:59.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f8497fff640 1 -- 172.21.15.83:0/158548780 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84ac018070 con 0x55c7228470a0 2024-04-17T10:47:59.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f8497fff640 1 -- 172.21.15.83:0/158548780 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac00eaa0 con 0x55c7228470a0 2024-04-17T10:47:59.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f84bf8f6000 1 -- 172.21.15.83:0/158548780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7228470a0 msgr2=0x55c722847480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:59.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f84bf8f6000 1 --2- 172.21.15.83:0/158548780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7228470a0 0x55c722847480 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f84ac0053f0 tx=0x7f84ac0058b0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f84bf8f6000 1 -- 172.21.15.83:0/158548780 shutdown_connections 2024-04-17T10:47:59.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f84bf8f6000 1 --2- 172.21.15.83:0/158548780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7228470a0 0x55c722847480 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f84bf8f6000 1 --2- 172.21.15.83:0/158548780 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c7228912f0 0x55c722846b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f84bf8f6000 1 --2- 172.21.15.83:0/158548780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7228909d0 0x55c722890db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f84bf8f6000 1 -- 172.21.15.83:0/158548780 >> 172.21.15.83:0/158548780 conn(0x55c722843020 msgr2=0x55c722845440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:59.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f84bf8f6000 1 -- 172.21.15.83:0/158548780 shutdown_connections 2024-04-17T10:47:59.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f84bf8f6000 1 -- 172.21.15.83:0/158548780 wait complete. 2024-04-17T10:47:59.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f84bf8f6000 1 -- start start 2024-04-17T10:47:59.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.086+0000 7f84bf8f6000 1 Processor -- start 2024-04-17T10:47:59.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bf8f6000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c7228470a0 0x55c7228757f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:59.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bf8f6000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7228909d0 0x55c722875d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:59.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bf8f6000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7228912f0 0x55c722876270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:59.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bf8f6000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c722658250 con 0x55c7228912f0 2024-04-17T10:47:59.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bf8f6000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c72264e690 con 0x55c7228909d0 2024-04-17T10:47:59.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bf8f6000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c72261ca40 con 0x55c7228470a0 2024-04-17T10:47:59.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bf8f6000 10 client.0 _ll_register_callbacks cb 0x55c722831e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:47:59.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bad26640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7228912f0 0x55c722876270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:59.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bad26640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7228912f0 0x55c722876270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46222/0 (socket says 172.21.15.83:46222) 2024-04-17T10:47:59.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bad26640 1 -- 172.21.15.83:0/3187550344 learned_addr learned my addr 172.21.15.83:0/3187550344 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:59.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84b9d24640 1 --2- 172.21.15.83:0/3187550344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7228909d0 0x55c722875d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:59.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84ba525640 1 --2- 172.21.15.83:0/3187550344 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c7228470a0 0x55c7228757f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:59.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bad26640 1 -- 172.21.15.83:0/3187550344 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c7228470a0 msgr2=0x55c7228757f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:59.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bad26640 1 --2- 172.21.15.83:0/3187550344 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c7228470a0 0x55c7228757f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bad26640 1 -- 172.21.15.83:0/3187550344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7228909d0 msgr2=0x55c722875d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:59.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bad26640 1 --2- 172.21.15.83:0/3187550344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7228909d0 0x55c722875d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bad26640 1 -- 172.21.15.83:0/3187550344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c7227411c0 con 0x55c7228912f0 2024-04-17T10:47:59.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bad26640 1 --2- 172.21.15.83:0/3187550344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7228912f0 0x55c722876270 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f84ac0058f0 tx=0x7f84ac003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c362894ce6c12e84 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:59.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84957fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:47:59.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84957fa640 1 -- 172.21.15.83:0/3187550344 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac00ee20 con 0x55c7228912f0 2024-04-17T10:47:59.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84957fa640 1 -- 172.21.15.83:0/3187550344 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f84ac013420 con 0x55c7228912f0 2024-04-17T10:47:59.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bf8f6000 1 -- 172.21.15.83:0/3187550344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84ac018070 con 0x55c7228912f0 2024-04-17T10:47:59.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84957fa640 1 -- 172.21.15.83:0/3187550344 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84ac02f050 con 0x55c7228912f0 2024-04-17T10:47:59.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bf8f6000 10 client.8280 fetch_fsmap learned FSMap version 62 2024-04-17T10:47:59.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bf8f6000 1 -- 172.21.15.83:0/3187550344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55c72273d170 con 0x55c7228912f0 2024-04-17T10:47:59.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84957fa640 1 -- 172.21.15.83:0/3187550344 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84ac018070 con 0x55c7228912f0 2024-04-17T10:47:59.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84957fa640 1 -- 172.21.15.83:0/3187550344 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f84ac032020 con 0x55c7228912f0 2024-04-17T10:47:59.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bf8f6000 10 client.8280 fetch_fsmap finished waiting for FSMap version 62 2024-04-17T10:47:59.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bf8f6000 10 client.8280 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-17T10:47:59.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bf8f6000 20 client.8280 populate_metadata read hostname 'smithi083' 2024-04-17T10:47:59.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bf8f6000 1 --2- 172.21.15.83:0/3187550344 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55c7228a0c80 0x55c7228a3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:59.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bf8f6000 4 client.8280 mds_command: new command op to 8049 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:47:59.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bf8f6000 1 -- 172.21.15.83:0/3187550344 --> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c72262fc60 con 0x55c7228a0c80 2024-04-17T10:47:59.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84ba525640 1 --2- 172.21.15.83:0/3187550344 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55c7228a0c80 0x55c7228a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:59.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84ba525640 1 --2- 172.21.15.83:0/3187550344 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55c7228a0c80 0x55c7228a3070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b9b2d8f832d509a server_cookie=6616e0cbb7a6ce49 in_seq=0 out_seq=0 2024-04-17T10:47:59.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84957fa640 10 client.8280 ms_handle_connect on v2:172.21.15.83:6836/2353406669 2024-04-17T10:47:59.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84957fa640 1 -- 172.21.15.83:0/3187550344 <== mds.0 v2:172.21.15.83:6836/2353406669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c72262fc60 con 0x55c7228a0c80 2024-04-17T10:47:59.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84957fa640 10 client.8280 handle_command_reply: tid=0 2024-04-17T10:47:59.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bf8f6000 1 client.8280 shutdown 2024-04-17T10:47:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bf8f6000 1 -- 172.21.15.83:0/3187550344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7228912f0 msgr2=0x55c722876270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bf8f6000 1 --2- 172.21.15.83:0/3187550344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7228912f0 0x55c722876270 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f84ac0058f0 tx=0x7f84ac003a80 comp rx=0 tx=0).stop 2024-04-17T10:47:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bf8f6000 1 -- 172.21.15.83:0/3187550344 shutdown_connections 2024-04-17T10:47:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bf8f6000 1 --2- 172.21.15.83:0/3187550344 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55c7228a0c80 0x55c7228a3070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bf8f6000 1 --2- 172.21.15.83:0/3187550344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7228912f0 0x55c722876270 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bf8f6000 1 --2- 172.21.15.83:0/3187550344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7228909d0 0x55c722875d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84957fa640 0 client.8280 ms_handle_reset on v2:172.21.15.83:6836/2353406669 2024-04-17T10:47:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bf8f6000 1 --2- 172.21.15.83:0/3187550344 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c7228470a0 0x55c7228757f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bf8f6000 1 -- 172.21.15.83:0/3187550344 >> 172.21.15.83:0/3187550344 conn(0x55c722843020 msgr2=0x55c722845ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bf8f6000 1 -- 172.21.15.83:0/3187550344 shutdown_connections 2024-04-17T10:47:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bf8f6000 1 -- 172.21.15.83:0/3187550344 wait complete. 2024-04-17T10:47:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.090+0000 7f84bf8f6000 20 client.8280 trim_cache size 0 max 16384 2024-04-17T10:47:59.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.106+0000 7f84bf8f6000 1 Processor -- start 2024-04-17T10:47:59.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.106+0000 7f84bf8f6000 1 -- start start 2024-04-17T10:47:59.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.106+0000 7f84bf8f6000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c7228470a0 0x55c72288fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:59.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.106+0000 7f84bf8f6000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7228909d0 0x55c722890140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:59.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.106+0000 7f84bf8f6000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7228912f0 0x55c722846420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:59.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.106+0000 7f84bf8f6000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b4200900 con 0x55c7228912f0 2024-04-17T10:47:59.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.106+0000 7f84bf8f6000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b4150190 con 0x55c7228909d0 2024-04-17T10:47:59.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.106+0000 7f84bf8f6000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b415eb90 con 0x55c7228470a0 2024-04-17T10:47:59.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.106+0000 7f84bad26640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7228912f0 0x55c722846420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:59.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.106+0000 7f84bad26640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7228912f0 0x55c722846420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46236/0 (socket says 172.21.15.83:46236) 2024-04-17T10:47:59.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.106+0000 7f84bad26640 1 -- 172.21.15.83:0/910947614 learned_addr learned my addr 172.21.15.83:0/910947614 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:59.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.106+0000 7f84b9d24640 1 --2- 172.21.15.83:0/910947614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7228909d0 0x55c722890140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:59.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.106+0000 7f84ba525640 1 --2- 172.21.15.83:0/910947614 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c7228470a0 0x55c72288fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:59.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.106+0000 7f84bad26640 1 -- 172.21.15.83:0/910947614 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c7228470a0 msgr2=0x55c72288fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:59.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.106+0000 7f84bad26640 1 --2- 172.21.15.83:0/910947614 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c7228470a0 0x55c72288fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.106+0000 7f84bad26640 1 -- 172.21.15.83:0/910947614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7228909d0 msgr2=0x55c722890140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:59.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.106+0000 7f84bad26640 1 --2- 172.21.15.83:0/910947614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7228909d0 0x55c722890140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.106+0000 7f84bad26640 1 -- 172.21.15.83:0/910947614 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7226dade0 con 0x55c7228912f0 2024-04-17T10:47:59.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.106+0000 7f84b9d24640 1 --2- 172.21.15.83:0/910947614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7228909d0 0x55c722890140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:59.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.106+0000 7f84bad26640 1 --2- 172.21.15.83:0/910947614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7228912f0 0x55c722846420 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f84ac0051b0 tx=0x7f84ac01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e236e26b79de7f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:59.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.106+0000 7f84977fe640 1 -- 172.21.15.83:0/910947614 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac00ea70 con 0x55c7228912f0 2024-04-17T10:47:59.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.106+0000 7f84977fe640 1 -- 172.21.15.83:0/910947614 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84ac018070 con 0x55c7228912f0 2024-04-17T10:47:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.106+0000 7f84977fe640 1 -- 172.21.15.83:0/910947614 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac005270 con 0x55c7228912f0 2024-04-17T10:47:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.106+0000 7f84bf8f6000 1 -- 172.21.15.83:0/910947614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7228912f0 msgr2=0x55c722846420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.106+0000 7f84bf8f6000 1 --2- 172.21.15.83:0/910947614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7228912f0 0x55c722846420 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f84ac0051b0 tx=0x7f84ac01e000 comp rx=0 tx=0).stop 2024-04-17T10:47:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.106+0000 7f84bf8f6000 1 -- 172.21.15.83:0/910947614 shutdown_connections 2024-04-17T10:47:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.106+0000 7f84bf8f6000 1 --2- 172.21.15.83:0/910947614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7228912f0 0x55c722846420 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.106+0000 7f84bf8f6000 1 --2- 172.21.15.83:0/910947614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7228909d0 0x55c722890140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.106+0000 7f84bf8f6000 1 --2- 172.21.15.83:0/910947614 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c7228470a0 0x55c72288fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.106+0000 7f84bf8f6000 1 -- 172.21.15.83:0/910947614 >> 172.21.15.83:0/910947614 conn(0x55c722843020 msgr2=0x55c72288f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.106+0000 7f84bf8f6000 1 -- 172.21.15.83:0/910947614 shutdown_connections 2024-04-17T10:47:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.106+0000 7f84bf8f6000 1 -- 172.21.15.83:0/910947614 wait complete. 2024-04-17T10:47:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bf8f6000 1 -- start start 2024-04-17T10:47:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bf8f6000 1 Processor -- start 2024-04-17T10:47:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bf8f6000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7228470a0 0x55c7228f1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bf8f6000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7228909d0 0x55c7228f1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bf8f6000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c7228912f0 0x55c7228a3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bf8f6000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c72262ee00 con 0x55c7228909d0 2024-04-17T10:47:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bf8f6000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b414fb90 con 0x55c7228470a0 2024-04-17T10:47:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bf8f6000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c722693300 con 0x55c7228912f0 2024-04-17T10:47:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bf8f6000 10 client.0 _ll_register_callbacks cb 0x55c722812040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:47:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84b9d24640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7228909d0 0x55c7228f1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84b9d24640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7228909d0 0x55c7228f1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46244/0 (socket says 172.21.15.83:46244) 2024-04-17T10:47:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84b9d24640 1 -- 172.21.15.83:0/97038203 learned_addr learned my addr 172.21.15.83:0/97038203 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84ba525640 1 --2- 172.21.15.83:0/97038203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7228470a0 0x55c7228f1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bad26640 1 --2- 172.21.15.83:0/97038203 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c7228912f0 0x55c7228a3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84b9d24640 1 -- 172.21.15.83:0/97038203 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c7228912f0 msgr2=0x55c7228a3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84b9d24640 1 --2- 172.21.15.83:0/97038203 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c7228912f0 0x55c7228a3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84b9d24640 1 -- 172.21.15.83:0/97038203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7228470a0 msgr2=0x55c7228f1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84b9d24640 1 --2- 172.21.15.83:0/97038203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7228470a0 0x55c7228f1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84b9d24640 1 -- 172.21.15.83:0/97038203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c7226358e0 con 0x55c7228909d0 2024-04-17T10:47:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bad26640 1 --2- 172.21.15.83:0/97038203 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c7228912f0 0x55c7228a3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84b9d24640 1 --2- 172.21.15.83:0/97038203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7228909d0 0x55c7228f1850 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f84a40eba70 tx=0x7f84a40ba3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ac3f3d9c276c45 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84957fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:47:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84957fa640 1 -- 172.21.15.83:0/97038203 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a40f7d20 con 0x55c7228909d0 2024-04-17T10:47:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84957fa640 1 -- 172.21.15.83:0/97038203 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f84a40ef870 con 0x55c7228909d0 2024-04-17T10:47:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bf8f6000 1 -- 172.21.15.83:0/97038203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84ac018070 con 0x55c7228909d0 2024-04-17T10:47:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84957fa640 1 -- 172.21.15.83:0/97038203 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84a40efab0 con 0x55c7228909d0 2024-04-17T10:47:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bf8f6000 10 client.8286 fetch_fsmap learned FSMap version 62 2024-04-17T10:47:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bf8f6000 1 -- 172.21.15.83:0/97038203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55c7226ed8f0 con 0x55c7228909d0 2024-04-17T10:47:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84957fa640 1 -- 172.21.15.83:0/97038203 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84ac018070 con 0x55c7228909d0 2024-04-17T10:47:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84957fa640 1 -- 172.21.15.83:0/97038203 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f84a40f7890 con 0x55c7228909d0 2024-04-17T10:47:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bf8f6000 10 client.8286 fetch_fsmap finished waiting for FSMap version 62 2024-04-17T10:47:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bf8f6000 10 client.8286 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-17T10:47:59.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bf8f6000 20 client.8286 populate_metadata read hostname 'smithi083' 2024-04-17T10:47:59.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bf8f6000 1 --2- 172.21.15.83:0/97038203 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55c7228a3d10 0x55c7228a4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:59.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bf8f6000 4 client.8286 mds_command: new command op to 8049 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T10:47:59.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bf8f6000 1 -- 172.21.15.83:0/97038203 --> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c7226a1530 con 0x55c7228a3d10 2024-04-17T10:47:59.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84ba525640 1 --2- 172.21.15.83:0/97038203 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55c7228a3d10 0x55c7228a4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:59.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84ba525640 1 --2- 172.21.15.83:0/97038203 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55c7228a3d10 0x55c7228a4120 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cca6bdfffb704e69 server_cookie=a8d49dc51a342450 in_seq=0 out_seq=0 2024-04-17T10:47:59.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84957fa640 10 client.8286 ms_handle_connect on v2:172.21.15.83:6836/2353406669 2024-04-17T10:47:59.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84957fa640 1 -- 172.21.15.83:0/97038203 <== mds.0 v2:172.21.15.83:6836/2353406669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x55c7226a1530 con 0x55c7228a3d10 2024-04-17T10:47:59.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84957fa640 10 client.8286 handle_command_reply: tid=0 2024-04-17T10:47:59.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bf8f6000 1 client.8286 shutdown 2024-04-17T10:47:59.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bf8f6000 1 -- 172.21.15.83:0/97038203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7228909d0 msgr2=0x55c7228f1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:59.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bf8f6000 1 --2- 172.21.15.83:0/97038203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7228909d0 0x55c7228f1850 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f84a40eba70 tx=0x7f84a40ba3f0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bf8f6000 1 -- 172.21.15.83:0/97038203 shutdown_connections 2024-04-17T10:47:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bf8f6000 1 --2- 172.21.15.83:0/97038203 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55c7228a3d10 0x55c7228a4120 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bf8f6000 1 --2- 172.21.15.83:0/97038203 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c7228912f0 0x55c7228a3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bf8f6000 1 --2- 172.21.15.83:0/97038203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7228909d0 0x55c7228f1850 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84957fa640 0 client.8286 ms_handle_reset on v2:172.21.15.83:6836/2353406669 2024-04-17T10:47:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bf8f6000 1 --2- 172.21.15.83:0/97038203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7228470a0 0x55c7228f1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bf8f6000 1 -- 172.21.15.83:0/97038203 >> 172.21.15.83:0/97038203 conn(0x55c722843020 msgr2=0x55c72288f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bf8f6000 1 -- 172.21.15.83:0/97038203 shutdown_connections 2024-04-17T10:47:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bf8f6000 1 -- 172.21.15.83:0/97038203 wait complete. 2024-04-17T10:47:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bf8f6000 20 client.8286 trim_cache size 0 max 16384 2024-04-17T10:47:59.116 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:47:59.116 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-17T10:47:59.116 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-17T10:47:59.116 INFO:teuthology.orchestra.run.smithi083.stdout: "1c3efbd8-8bfe-4dd7-aebe-ead10d1ddb8e": { 2024-04-17T10:47:59.116 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/scrub_control_test_path", 2024-04-17T10:47:59.116 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "1c3efbd8-8bfe-4dd7-aebe-ead10d1ddb8e", 2024-04-17T10:47:59.116 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive" 2024-04-17T10:47:59.116 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:47:59.116 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:47:59.116 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:47:59.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bc789640 1 -- 172.21.15.83:0/2206105694 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8498066f60 msgr2=0x7f8498069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:59.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bc789640 1 --2- 172.21.15.83:0/2206105694 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8498066f60 0x7f8498069420 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f84b40cc780 tx=0x7f84b0008040 comp rx=0 tx=0).stop 2024-04-17T10:47:59.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bc789640 1 -- 172.21.15.83:0/2206105694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b415b6e0 msgr2=0x7f84b4144800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:59.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bc789640 1 --2- 172.21.15.83:0/2206105694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b415b6e0 0x7f84b4144800 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f84a401cd80 tx=0x7f84a4019280 comp rx=0 tx=0).stop 2024-04-17T10:47:59.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bc789640 1 -- 172.21.15.83:0/2206105694 shutdown_connections 2024-04-17T10:47:59.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bc789640 1 --2- 172.21.15.83:0/2206105694 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8498066f60 0x7f8498069420 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bc789640 1 --2- 172.21.15.83:0/2206105694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84b415bac0 0x7f84b415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bc789640 1 --2- 172.21.15.83:0/2206105694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b415b6e0 0x7f84b4144800 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bc789640 1 --2- 172.21.15.83:0/2206105694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f84b4070470 0x7f84b4150470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bc789640 1 -- 172.21.15.83:0/2206105694 >> 172.21.15.83:0/2206105694 conn(0x7f84b40b15b0 msgr2=0x7f84b40cfd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:59.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bc789640 1 -- 172.21.15.83:0/2206105694 shutdown_connections 2024-04-17T10:47:59.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.110+0000 7f84bc789640 1 -- 172.21.15.83:0/2206105694 wait complete. 2024-04-17T10:47:59.124 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:47:59.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.230+0000 7f83b64bc640 1 Processor -- start 2024-04-17T10:47:59.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.234+0000 7f83b64bc640 1 -- start start 2024-04-17T10:47:59.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.234+0000 7f83b64bc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f83b01540d0 0x7f83b0070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:59.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.234+0000 7f83b64bc640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83b006ce70 con 0x7f83b015ddb0 2024-04-17T10:47:59.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.234+0000 7f83b64bc640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83b006d140 con 0x7f83b015b270 2024-04-17T10:47:59.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.234+0000 7f83b64bc640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83b0072730 con 0x7f83b01540d0 2024-04-17T10:47:59.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.234+0000 7f83affff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f83b01540d0 0x7f83b0070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:59.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.234+0000 7f83b4a59640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f83b015ddb0 0x7f83b00b28e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60670/0 (socket says 172.21.15.83:60670) 2024-04-17T10:47:59.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.234+0000 7f83b4a59640 1 -- 172.21.15.83:0/430916206 learned_addr learned my addr 172.21.15.83:0/430916206 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:47:59.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.234+0000 7f83affff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f83b01540d0 0x7f83b0070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46256/0 (socket says 172.21.15.83:46256) 2024-04-17T10:47:59.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.234+0000 7f83affff640 1 -- 172.21.15.83:0/430916206 >> v1:172.21.15.167:6789/0 conn(0x7f83b015b270 legacy=0x7f83b015d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:59.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.234+0000 7f83affff640 1 -- 172.21.15.83:0/430916206 >> v1:172.21.15.83:6789/0 conn(0x7f83b015ddb0 legacy=0x7f83b00b28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:59.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.234+0000 7f83affff640 1 -- 172.21.15.83:0/430916206 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b00720e0 con 0x7f83b01540d0 2024-04-17T10:47:59.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.234+0000 7f83aeffd640 1 -- 172.21.15.83:0/430916206 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2557018949 0 0) 0x7f83b006ce70 con 0x7f83b015ddb0 2024-04-17T10:47:59.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.234+0000 7f83affff640 1 --2- 172.21.15.83:0/430916206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83b01540d0 0x7f83b0070470 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f83a00097c0 tx=0x7f83a001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94fec66df1320a2c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:59.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.234+0000 7f83aeffd640 1 -- 172.21.15.83:0/430916206 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a00ab600 con 0x7f83b01540d0 2024-04-17T10:47:59.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.234+0000 7f83aeffd640 1 -- 172.21.15.83:0/430916206 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a00b0070 con 0x7f83b01540d0 2024-04-17T10:47:59.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.234+0000 7f83aeffd640 1 -- 172.21.15.83:0/430916206 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a00ab9c0 con 0x7f83b01540d0 2024-04-17T10:47:59.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.238+0000 7f83b64bc640 1 -- 172.21.15.83:0/430916206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83b01540d0 msgr2=0x7f83b0070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:59.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.238+0000 7f83b64bc640 1 --2- 172.21.15.83:0/430916206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83b01540d0 0x7f83b0070470 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f83a00097c0 tx=0x7f83a001c470 comp rx=0 tx=0).stop 2024-04-17T10:47:59.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.238+0000 7f83b64bc640 1 -- 172.21.15.83:0/430916206 shutdown_connections 2024-04-17T10:47:59.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.238+0000 7f83b64bc640 1 --2- 172.21.15.83:0/430916206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83b01540d0 0x7f83b0070470 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.238+0000 7f83b64bc640 1 -- 172.21.15.83:0/430916206 >> 172.21.15.83:0/430916206 conn(0x7f83b006abe0 msgr2=0x7f83b006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:59.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.238+0000 7f83b64bc640 1 -- 172.21.15.83:0/430916206 shutdown_connections 2024-04-17T10:47:59.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.238+0000 7f83b64bc640 1 -- 172.21.15.83:0/430916206 wait complete. 2024-04-17T10:47:59.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.238+0000 7f83b64bc640 1 Processor -- start 2024-04-17T10:47:59.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.238+0000 7f83b64bc640 1 -- start start 2024-04-17T10:47:59.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.238+0000 7f83b64bc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83b01540d0 0x7f83b01413c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:59.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.238+0000 7f83b64bc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83b015b270 0x7f83b0147700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:59.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.238+0000 7f83b64bc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83b015ddb0 0x7f83b01450e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:59.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.238+0000 7f83b64bc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b00709b0 con 0x7f83b01540d0 2024-04-17T10:47:59.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.238+0000 7f83b64bc640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b015e380 con 0x7f83b015ddb0 2024-04-17T10:47:59.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.238+0000 7f83b64bc640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b0155690 con 0x7f83b015b270 2024-04-17T10:47:59.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.238+0000 7f83affff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83b01540d0 0x7f83b01413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:59.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.238+0000 7f83affff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83b01540d0 0x7f83b01413c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46260/0 (socket says 172.21.15.83:46260) 2024-04-17T10:47:59.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.238+0000 7f83affff640 1 -- 172.21.15.83:0/300775235 learned_addr learned my addr 172.21.15.83:0/300775235 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:59.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.238+0000 7f83affff640 1 -- 172.21.15.83:0/300775235 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83b015b270 msgr2=0x7f83b0147700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:59.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.238+0000 7f83affff640 1 --2- 172.21.15.83:0/300775235 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83b015b270 0x7f83b0147700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.238+0000 7f83b4a59640 1 --2- 172.21.15.83:0/300775235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83b015ddb0 0x7f83b01450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:59.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.238+0000 7f83affff640 1 -- 172.21.15.83:0/300775235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83b015ddb0 msgr2=0x7f83b01450e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:59.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.238+0000 7f83affff640 1 --2- 172.21.15.83:0/300775235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83b015ddb0 0x7f83b01450e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.238+0000 7f83affff640 1 -- 172.21.15.83:0/300775235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b0154a80 con 0x7f83b01540d0 2024-04-17T10:47:59.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.238+0000 7f83affff640 1 --2- 172.21.15.83:0/300775235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83b01540d0 0x7f83b01413c0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f83a00097c0 tx=0x7f83a001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:59.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.238+0000 7f83acff9640 1 -- 172.21.15.83:0/300775235 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a00bf610 con 0x7f83b01540d0 2024-04-17T10:47:59.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.238+0000 7f83acff9640 1 -- 172.21.15.83:0/300775235 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b00b3800 con 0x7f83b01540d0 2024-04-17T10:47:59.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.238+0000 7f83b64bc640 1 -- 172.21.15.83:0/300775235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b0154d60 con 0x7f83b01540d0 2024-04-17T10:47:59.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.238+0000 7f83acff9640 1 -- 172.21.15.83:0/300775235 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a00bfa10 con 0x7f83b01540d0 2024-04-17T10:47:59.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.238+0000 7f83b64bc640 1 -- 172.21.15.83:0/300775235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b0155320 con 0x7f83b01540d0 2024-04-17T10:47:59.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.242+0000 7f83b64bc640 1 -- 172.21.15.83:0/300775235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83b0074f90 con 0x7f83b01540d0 2024-04-17T10:47:59.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.242+0000 7f83acff9640 1 -- 172.21.15.83:0/300775235 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f83a00bd020 con 0x7f83b01540d0 2024-04-17T10:47:59.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.242+0000 7f83acff9640 1 --2- 172.21.15.83:0/300775235 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8390066f60 0x7f8390069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:59.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.242+0000 7f83acff9640 1 -- 172.21.15.83:0/300775235 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f83a011f510 con 0x7f83b01540d0 2024-04-17T10:47:59.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.242+0000 7f83af7fe640 1 --2- 172.21.15.83:0/300775235 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8390066f60 0x7f8390069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:59.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.242+0000 7f83af7fe640 1 --2- 172.21.15.83:0/300775235 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8390066f60 0x7f8390069420 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8398000f70 tx=0x7f83980079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:59.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.246+0000 7f83acff9640 1 -- 172.21.15.83:0/300775235 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f83a0016090 con 0x7f83b01540d0 2024-04-17T10:47:59.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.438+0000 7f83b64bc640 1 -- 172.21.15.83:0/300775235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83b006dbb0 con 0x7f83b01540d0 2024-04-17T10:47:59.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.438+0000 7f83acff9640 1 -- 172.21.15.83:0/300775235 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f83a00f0520 con 0x7f83b01540d0 2024-04-17T10:47:59.435 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:47:59.435 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/437268957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":437268957},{"type":"v1","addr":"172.21.15.167:6833","nonce":437268957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":6253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3214640674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3214640674},{"type":"v1","addr":"172.21.15.167:6835","nonce":3214640674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":8046,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/715498323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":715498323},{"type":"v1","addr":"172.21.15.83:6835","nonce":715498323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:47:29.385103+0000","modified":"2024-04-17T10:47:31.389088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8049":{"gid":8049,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2353406669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":2353406669},{"type":"v1","addr":"172.21.15.83:6837","nonce":2353406669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-17T10:47:59.435 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 62 2024-04-17T10:47:59.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.442+0000 7f83b64bc640 1 -- 172.21.15.83:0/300775235 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8390066f60 msgr2=0x7f8390069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:59.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.442+0000 7f83b64bc640 1 --2- 172.21.15.83:0/300775235 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8390066f60 0x7f8390069420 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8398000f70 tx=0x7f83980079c0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.442+0000 7f83b64bc640 1 -- 172.21.15.83:0/300775235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83b01540d0 msgr2=0x7f83b01413c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:59.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.442+0000 7f83b64bc640 1 --2- 172.21.15.83:0/300775235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83b01540d0 0x7f83b01413c0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f83a00097c0 tx=0x7f83a001acb0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.442+0000 7f83b64bc640 1 -- 172.21.15.83:0/300775235 shutdown_connections 2024-04-17T10:47:59.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.442+0000 7f83b64bc640 1 --2- 172.21.15.83:0/300775235 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8390066f60 0x7f8390069420 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.442+0000 7f83b64bc640 1 --2- 172.21.15.83:0/300775235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83b015ddb0 0x7f83b01450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.442+0000 7f83b64bc640 1 --2- 172.21.15.83:0/300775235 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83b015b270 0x7f83b0147700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.442+0000 7f83b64bc640 1 --2- 172.21.15.83:0/300775235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83b01540d0 0x7f83b01413c0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.442+0000 7f83b64bc640 1 -- 172.21.15.83:0/300775235 >> 172.21.15.83:0/300775235 conn(0x7f83b006abe0 msgr2=0x7f83b015bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:59.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.442+0000 7f83b64bc640 1 -- 172.21.15.83:0/300775235 shutdown_connections 2024-04-17T10:47:59.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.442+0000 7f83b64bc640 1 -- 172.21.15.83:0/300775235 wait complete. 2024-04-17T10:47:59.453 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-17T10:47:59.453 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-17T10:47:59.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.570+0000 7f8f2f88d640 1 Processor -- start 2024-04-17T10:47:59.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.570+0000 7f8f2f88d640 1 -- start start 2024-04-17T10:47:59.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.570+0000 7f8f2f88d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f28072330 0x7f8f280cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:59.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.570+0000 7f8f2f88d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8f280cfa10 0x7f8f2806c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:59.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.570+0000 7f8f2f88d640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f280cfea0 con 0x7f8f2815ba30 2024-04-17T10:47:59.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.570+0000 7f8f2f88d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f28071ea0 con 0x7f8f280cfa10 2024-04-17T10:47:59.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.570+0000 7f8f2f88d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2806e320 con 0x7f8f28072330 2024-04-17T10:47:59.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.570+0000 7f8f2de2a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8f280cfa10 0x7f8f2806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:59.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.570+0000 7f8f2de2a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8f280cfa10 0x7f8f2806c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46274/0 (socket says 172.21.15.83:46274) 2024-04-17T10:47:59.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.570+0000 7f8f2de2a640 1 -- 172.21.15.83:0/4246085129 learned_addr learned my addr 172.21.15.83:0/4246085129 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:59.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.570+0000 7f8f2ce28640 1 --2- 172.21.15.83:0/4246085129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f28072330 0x7f8f280cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:59.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.574+0000 7f8f2de2a640 1 -- 172.21.15.83:0/4246085129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f28072330 msgr2=0x7f8f280cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:47:59.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.574+0000 7f8f2de2a640 1 --2- 172.21.15.83:0/4246085129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f28072330 0x7f8f280cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.574+0000 7f8f2de2a640 1 -- 172.21.15.83:0/4246085129 >> v1:172.21.15.167:6789/0 conn(0x7f8f2815ba30 legacy=0x7f8f2815c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:59.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.574+0000 7f8f2de2a640 1 -- 172.21.15.83:0/4246085129 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f2806e040 con 0x7f8f280cfa10 2024-04-17T10:47:59.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.574+0000 7f8f2de2a640 1 --2- 172.21.15.83:0/4246085129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f280cfa10 0x7f8f2806c490 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009e50 tx=0x7f8f2001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28038e696311a01 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:59.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.574+0000 7f8f1ffff640 1 -- 172.21.15.83:0/4246085129 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f200a5770 con 0x7f8f280cfa10 2024-04-17T10:47:59.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.574+0000 7f8f1ffff640 1 -- 172.21.15.83:0/4246085129 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f200a5910 con 0x7f8f280cfa10 2024-04-17T10:47:59.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.574+0000 7f8f1ffff640 1 -- 172.21.15.83:0/4246085129 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f200a5ce0 con 0x7f8f280cfa10 2024-04-17T10:47:59.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.574+0000 7f8f2f88d640 1 -- 172.21.15.83:0/4246085129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f280cfa10 msgr2=0x7f8f2806c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:59.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.574+0000 7f8f2f88d640 1 --2- 172.21.15.83:0/4246085129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f280cfa10 0x7f8f2806c490 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009e50 tx=0x7f8f2001c9a0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.574+0000 7f8f2f88d640 1 -- 172.21.15.83:0/4246085129 shutdown_connections 2024-04-17T10:47:59.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.574+0000 7f8f2f88d640 1 --2- 172.21.15.83:0/4246085129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f280cfa10 0x7f8f2806c490 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.574+0000 7f8f2f88d640 1 --2- 172.21.15.83:0/4246085129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f28072330 0x7f8f280cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.574+0000 7f8f2f88d640 1 -- 172.21.15.83:0/4246085129 >> 172.21.15.83:0/4246085129 conn(0x7f8f2806abe0 msgr2=0x7f8f2806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:59.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.574+0000 7f8f2f88d640 1 -- 172.21.15.83:0/4246085129 shutdown_connections 2024-04-17T10:47:59.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.574+0000 7f8f2f88d640 1 -- 172.21.15.83:0/4246085129 wait complete. 2024-04-17T10:47:59.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.578+0000 7f8f2f88d640 1 Processor -- start 2024-04-17T10:47:59.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.578+0000 7f8f2f88d640 1 -- start start 2024-04-17T10:47:59.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.578+0000 7f8f2f88d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f28072330 0x7f8f2814a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:59.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.578+0000 7f8f2f88d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f280cfa10 0x7f8f280ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:59.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.578+0000 7f8f2f88d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f2815ba30 0x7f8f280add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:59.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.578+0000 7f8f2f88d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2806de20 con 0x7f8f2815ba30 2024-04-17T10:47:59.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.578+0000 7f8f2f88d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2806dca0 con 0x7f8f28072330 2024-04-17T10:47:59.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.578+0000 7f8f2f88d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f28168520 con 0x7f8f280cfa10 2024-04-17T10:47:59.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.578+0000 7f8f2de2a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f2815ba30 0x7f8f280add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:59.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.578+0000 7f8f2de2a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f2815ba30 0x7f8f280add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46278/0 (socket says 172.21.15.83:46278) 2024-04-17T10:47:59.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.578+0000 7f8f2de2a640 1 -- 172.21.15.83:0/4124761287 learned_addr learned my addr 172.21.15.83:0/4124761287 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:47:59.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.578+0000 7f8f2d629640 1 --2- 172.21.15.83:0/4124761287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f28072330 0x7f8f2814a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:59.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.578+0000 7f8f2ce28640 1 --2- 172.21.15.83:0/4124761287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f280cfa10 0x7f8f280ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:59.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.578+0000 7f8f2de2a640 1 -- 172.21.15.83:0/4124761287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f280cfa10 msgr2=0x7f8f280ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:59.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.578+0000 7f8f2de2a640 1 --2- 172.21.15.83:0/4124761287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f280cfa10 0x7f8f280ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.578+0000 7f8f2de2a640 1 -- 172.21.15.83:0/4124761287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f28072330 msgr2=0x7f8f2814a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:59.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.578+0000 7f8f2de2a640 1 --2- 172.21.15.83:0/4124761287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f28072330 0x7f8f2814a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.578+0000 7f8f2de2a640 1 -- 172.21.15.83:0/4124761287 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f2814cd70 con 0x7f8f2815ba30 2024-04-17T10:47:59.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.578+0000 7f8f2ce28640 1 --2- 172.21.15.83:0/4124761287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f280cfa10 0x7f8f280ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:47:59.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.578+0000 7f8f2de2a640 1 --2- 172.21.15.83:0/4124761287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f2815ba30 0x7f8f280add80 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f8f200af040 tx=0x7f8f20019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:59.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.578+0000 7f8f1dffb640 1 -- 172.21.15.83:0/4124761287 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f200bc020 con 0x7f8f2815ba30 2024-04-17T10:47:59.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.578+0000 7f8f1dffb640 1 -- 172.21.15.83:0/4124761287 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f200b8670 con 0x7f8f2815ba30 2024-04-17T10:47:59.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.578+0000 7f8f2f88d640 1 -- 172.21.15.83:0/4124761287 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f2814feb0 con 0x7f8f2815ba30 2024-04-17T10:47:59.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.578+0000 7f8f1dffb640 1 -- 172.21.15.83:0/4124761287 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f200b8a40 con 0x7f8f2815ba30 2024-04-17T10:47:59.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.578+0000 7f8f2f88d640 1 -- 172.21.15.83:0/4124761287 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f2814b680 con 0x7f8f2815ba30 2024-04-17T10:47:59.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.578+0000 7f8f2f88d640 1 -- 172.21.15.83:0/4124761287 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f28074f90 con 0x7f8f2815ba30 2024-04-17T10:47:59.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.578+0000 7f8f1dffb640 1 -- 172.21.15.83:0/4124761287 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8f200bf020 con 0x7f8f2815ba30 2024-04-17T10:47:59.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.578+0000 7f8f1dffb640 1 --2- 172.21.15.83:0/4124761287 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8f0c066f90 0x7f8f0c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:47:59.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.582+0000 7f8f1dffb640 1 -- 172.21.15.83:0/4124761287 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f2011faa0 con 0x7f8f2815ba30 2024-04-17T10:47:59.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.582+0000 7f8f2d629640 1 --2- 172.21.15.83:0/4124761287 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8f0c066f90 0x7f8f0c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:47:59.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.582+0000 7f8f2d629640 1 --2- 172.21.15.83:0/4124761287 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8f0c066f90 0x7f8f0c069450 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8f18001650 tx=0x7f8f18009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:47:59.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.586+0000 7f8f1dffb640 1 -- 172.21.15.83:0/4124761287 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8f20016620 con 0x7f8f2815ba30 2024-04-17T10:47:59.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.866+0000 7f8f2f88d640 1 -- 172.21.15.83:0/4124761287 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f8f28152ec0 con 0x7f8f2815ba30 2024-04-17T10:47:59.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.866+0000 7f8f1dffb640 1 -- 172.21.15.83:0/4124761287 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1444 (secure 0 0 0) 0x7f8f200f0ab0 con 0x7f8f2815ba30 2024-04-17T10:47:59.862 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:47:59.863 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":612,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":103,"num_osds":8,"num_up_osds":8,"osd_up_since":1713350279,"num_in_osds":8,"osd_in_since":1713350270,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":786,"data_bytes":800888372,"bytes_used":2106171392,"bytes_avail":770987941888,"bytes_total":773094113280,"write_bytes_sec":53052687,"read_op_per_sec":0,"write_op_per_sec":53},"fsmap":{"epoch":62,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8049}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-04-17T10:47:18.345526+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-17T10:47:59.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.866+0000 7f8f2f88d640 1 -- 172.21.15.83:0/4124761287 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8f0c066f90 msgr2=0x7f8f0c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:59.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.866+0000 7f8f2f88d640 1 --2- 172.21.15.83:0/4124761287 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8f0c066f90 0x7f8f0c069450 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8f18001650 tx=0x7f8f18009040 comp rx=0 tx=0).stop 2024-04-17T10:47:59.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.866+0000 7f8f2f88d640 1 -- 172.21.15.83:0/4124761287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f2815ba30 msgr2=0x7f8f280add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:47:59.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.866+0000 7f8f2f88d640 1 --2- 172.21.15.83:0/4124761287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f2815ba30 0x7f8f280add80 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f8f200af040 tx=0x7f8f20019740 comp rx=0 tx=0).stop 2024-04-17T10:47:59.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.866+0000 7f8f2f88d640 1 -- 172.21.15.83:0/4124761287 shutdown_connections 2024-04-17T10:47:59.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.866+0000 7f8f2f88d640 1 --2- 172.21.15.83:0/4124761287 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8f0c066f90 0x7f8f0c069450 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.866+0000 7f8f2f88d640 1 --2- 172.21.15.83:0/4124761287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f2815ba30 0x7f8f280add80 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.866+0000 7f8f2f88d640 1 --2- 172.21.15.83:0/4124761287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f280cfa10 0x7f8f280ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.866+0000 7f8f2f88d640 1 --2- 172.21.15.83:0/4124761287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f28072330 0x7f8f2814a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:47:59.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.866+0000 7f8f2f88d640 1 -- 172.21.15.83:0/4124761287 >> 172.21.15.83:0/4124761287 conn(0x7f8f2806abe0 msgr2=0x7f8f28152260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:47:59.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.866+0000 7f8f2f88d640 1 -- 172.21.15.83:0/4124761287 shutdown_connections 2024-04-17T10:47:59.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:47:59.866+0000 7f8f2f88d640 1 -- 172.21.15.83:0/4124761287 wait complete. 2024-04-17T10:47:59.879 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-17T10:48:00.880 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:48:01.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143c2e4640 1 Processor -- start 2024-04-17T10:48:01.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143c2e4640 1 -- start start 2024-04-17T10:48:01.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143c2e4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1434146a00 0x7f1434146e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:01.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143c2e4640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f14340b2d80 0x7f14341620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:01.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143c2e4640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f143406daf0 con 0x7f143406d690 2024-04-17T10:48:01.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143c2e4640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1434162610 con 0x7f14340b2d80 2024-04-17T10:48:01.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143c2e4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f143406c960 con 0x7f1434146a00 2024-04-17T10:48:01.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143987f640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f143406d690 0x7f14340b2670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60676/0 (socket says 172.21.15.83:60676) 2024-04-17T10:48:01.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143987f640 1 -- 172.21.15.83:0/2156555680 learned_addr learned my addr 172.21.15.83:0/2156555680 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:01.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143a080640 1 --2- 172.21.15.83:0/2156555680 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1434146a00 0x7f1434146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:01.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143a881640 1 --2- 172.21.15.83:0/2156555680 >> v2:172.21.15.167:3300/0 conn(0x7f14340b2d80 0x7f14341620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:01.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143907e640 1 -- 172.21.15.83:0/2156555680 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4291392731 0 0) 0x7f143406daf0 con 0x7f143406d690 2024-04-17T10:48:01.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143907e640 1 -- 172.21.15.83:0/2156555680 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1420003530 con 0x7f143406d690 2024-04-17T10:48:01.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143907e640 1 -- 172.21.15.83:0/2156555680 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3002209568 0 0) 0x7f1420003530 con 0x7f143406d690 2024-04-17T10:48:01.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143907e640 1 -- 172.21.15.83:0/2156555680 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1434146a00 msgr2=0x7f1434146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:01.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143907e640 1 --2- 172.21.15.83:0/2156555680 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1434146a00 0x7f1434146e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:01.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143907e640 1 -- 172.21.15.83:0/2156555680 >> v2:172.21.15.167:3300/0 conn(0x7f14340b2d80 msgr2=0x7f14341620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:01.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143907e640 1 --2- 172.21.15.83:0/2156555680 >> v2:172.21.15.167:3300/0 conn(0x7f14340b2d80 0x7f14341620d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:01.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143907e640 1 -- 172.21.15.83:0/2156555680 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1434147570 con 0x7f143406d690 2024-04-17T10:48:01.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143907e640 1 -- 172.21.15.83:0/2156555680 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f1424001680 con 0x7f143406d690 2024-04-17T10:48:01.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143907e640 1 -- 172.21.15.83:0/2156555680 >> v1:172.21.15.83:6789/0 conn(0x7f143406d690 legacy=0x7f14340b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:01.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143907e640 1 --2- 172.21.15.83:0/2156555680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14200039c0 0x7f1420003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:01.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143907e640 1 --2- 172.21.15.83:0/2156555680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1420004810 0x7f1420004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:01.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143907e640 1 --2- 172.21.15.83:0/2156555680 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1434146a00 0x7f14200051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:01.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143907e640 1 -- 172.21.15.83:0/2156555680 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143406c960 con 0x7f1420004810 2024-04-17T10:48:01.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143907e640 1 -- 172.21.15.83:0/2156555680 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142000b9d0 con 0x7f14200039c0 2024-04-17T10:48:01.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143907e640 1 -- 172.21.15.83:0/2156555680 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f142000bb50 con 0x7f1434146a00 2024-04-17T10:48:01.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143987f640 1 --2- 172.21.15.83:0/2156555680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1420004810 0x7f1420004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:01.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143a881640 1 --2- 172.21.15.83:0/2156555680 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1434146a00 0x7f14200051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:01.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143a080640 1 --2- 172.21.15.83:0/2156555680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14200039c0 0x7f1420003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:01.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143987f640 1 -- 172.21.15.83:0/2156555680 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1434146a00 msgr2=0x7f14200051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:01.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143987f640 1 --2- 172.21.15.83:0/2156555680 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1434146a00 0x7f14200051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:01.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143987f640 1 -- 172.21.15.83:0/2156555680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14200039c0 msgr2=0x7f1420003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:01.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143987f640 1 --2- 172.21.15.83:0/2156555680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14200039c0 0x7f1420003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:01.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143987f640 1 -- 172.21.15.83:0/2156555680 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1420003530 con 0x7f1420004810 2024-04-17T10:48:01.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143a881640 1 --2- 172.21.15.83:0/2156555680 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1434146a00 0x7f14200051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:01.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.014+0000 7f143a080640 1 --2- 172.21.15.83:0/2156555680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14200039c0 0x7f1420003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:01.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.018+0000 7f143987f640 1 --2- 172.21.15.83:0/2156555680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1420004810 0x7f1420004c90 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f142400c210 tx=0x7f142401f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a01f90e6706c2cc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:01.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.018+0000 7f143907e640 1 -- 172.21.15.83:0/2156555680 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1424007d40 con 0x7f1420004810 2024-04-17T10:48:01.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.018+0000 7f143907e640 1 -- 172.21.15.83:0/2156555680 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14240ac020 con 0x7f1420004810 2024-04-17T10:48:01.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.018+0000 7f143907e640 1 -- 172.21.15.83:0/2156555680 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14240058e0 con 0x7f1420004810 2024-04-17T10:48:01.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.018+0000 7f143c2e4640 1 -- 172.21.15.83:0/2156555680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1420004810 msgr2=0x7f1420004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:01.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.018+0000 7f143c2e4640 1 --2- 172.21.15.83:0/2156555680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1420004810 0x7f1420004c90 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f142400c210 tx=0x7f142401f3d0 comp rx=0 tx=0).stop 2024-04-17T10:48:01.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.018+0000 7f143a881640 1 -- 172.21.15.83:0/2156555680 reap_dead start 2024-04-17T10:48:01.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.018+0000 7f143c2e4640 1 -- 172.21.15.83:0/2156555680 shutdown_connections 2024-04-17T10:48:01.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.018+0000 7f143c2e4640 1 -- 172.21.15.83:0/2156555680 >> 172.21.15.83:0/2156555680 conn(0x7f143406abe0 msgr2=0x7f143406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:01.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.018+0000 7f143c2e4640 1 -- 172.21.15.83:0/2156555680 shutdown_connections 2024-04-17T10:48:01.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.018+0000 7f143c2e4640 1 -- 172.21.15.83:0/2156555680 wait complete. 2024-04-17T10:48:01.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.018+0000 7f143c2e4640 1 Processor -- start 2024-04-17T10:48:01.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.018+0000 7f143c2e4640 1 -- start start 2024-04-17T10:48:01.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.018+0000 7f143c2e4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1420004810 0x7f14341f3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:01.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.018+0000 7f143c2e4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1434145300 0x7f143413ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:01.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.018+0000 7f143c2e4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1434145760 0x7f143413d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:01.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.018+0000 7f143c2e4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143406ce90 con 0x7f1434145760 2024-04-17T10:48:01.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.018+0000 7f143c2e4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143406d200 con 0x7f1420004810 2024-04-17T10:48:01.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.018+0000 7f143c2e4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1434146e70 con 0x7f1434145300 2024-04-17T10:48:01.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.022+0000 7f143a881640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1434145760 0x7f143413d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:01.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.022+0000 7f143a881640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1434145760 0x7f143413d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46296/0 (socket says 172.21.15.83:46296) 2024-04-17T10:48:01.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.022+0000 7f143a881640 1 -- 172.21.15.83:0/211918894 learned_addr learned my addr 172.21.15.83:0/211918894 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:01.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.022+0000 7f143987f640 1 --2- 172.21.15.83:0/211918894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1434145300 0x7f143413ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:01.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.022+0000 7f143a080640 1 --2- 172.21.15.83:0/211918894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1420004810 0x7f14341f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:01.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.022+0000 7f143a881640 1 -- 172.21.15.83:0/211918894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1434145300 msgr2=0x7f143413ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:01.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.022+0000 7f143a881640 1 --2- 172.21.15.83:0/211918894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1434145300 0x7f143413ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:01.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.022+0000 7f143a881640 1 -- 172.21.15.83:0/211918894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1420004810 msgr2=0x7f14341f3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:01.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.022+0000 7f143a881640 1 --2- 172.21.15.83:0/211918894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1420004810 0x7f14341f3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:01.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.022+0000 7f143a881640 1 -- 172.21.15.83:0/211918894 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1434142c40 con 0x7f1434145760 2024-04-17T10:48:01.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.022+0000 7f143a881640 1 --2- 172.21.15.83:0/211918894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1434145760 0x7f143413d3b0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f1434067670 tx=0x7f142c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:01.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.022+0000 7f142affd640 1 -- 172.21.15.83:0/211918894 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c01c070 con 0x7f1434145760 2024-04-17T10:48:01.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.022+0000 7f142affd640 1 -- 172.21.15.83:0/211918894 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f142c017400 con 0x7f1434145760 2024-04-17T10:48:01.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.022+0000 7f143c2e4640 1 -- 172.21.15.83:0/211918894 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1434142f20 con 0x7f1434145760 2024-04-17T10:48:01.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.022+0000 7f142affd640 1 -- 172.21.15.83:0/211918894 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c0177d0 con 0x7f1434145760 2024-04-17T10:48:01.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.022+0000 7f143c2e4640 1 -- 172.21.15.83:0/211918894 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14341434e0 con 0x7f1434145760 2024-04-17T10:48:01.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.022+0000 7f143c2e4640 1 -- 172.21.15.83:0/211918894 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1434074f90 con 0x7f1434145760 2024-04-17T10:48:01.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.022+0000 7f142affd640 1 -- 172.21.15.83:0/211918894 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f142c01f070 con 0x7f1434145760 2024-04-17T10:48:01.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.022+0000 7f142affd640 1 --2- 172.21.15.83:0/211918894 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1418066f90 0x7f1418069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:01.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.022+0000 7f142affd640 1 -- 172.21.15.83:0/211918894 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f142c085060 con 0x7f1434145760 2024-04-17T10:48:01.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.022+0000 7f143a080640 1 --2- 172.21.15.83:0/211918894 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1418066f90 0x7f1418069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:01.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.022+0000 7f143a080640 1 --2- 172.21.15.83:0/211918894 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1418066f90 0x7f1418069450 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1430009910 tx=0x7f1430008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:01.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.026+0000 7f142affd640 1 -- 172.21.15.83:0/211918894 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f142c012070 con 0x7f1434145760 2024-04-17T10:48:01.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.206+0000 7f143c2e4640 1 -- 172.21.15.83:0/211918894 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14340b0f60 con 0x7f1434145760 2024-04-17T10:48:01.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.210+0000 7f142affd640 1 -- 172.21.15.83:0/211918894 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f142c056100 con 0x7f1434145760 2024-04-17T10:48:01.205 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:01.205 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/437268957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":437268957},{"type":"v1","addr":"172.21.15.167:6833","nonce":437268957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":6253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3214640674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3214640674},{"type":"v1","addr":"172.21.15.167:6835","nonce":3214640674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":8046,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/715498323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":715498323},{"type":"v1","addr":"172.21.15.83:6835","nonce":715498323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:47:29.385103+0000","modified":"2024-04-17T10:47:31.389088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8049":{"gid":8049,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2353406669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":2353406669},{"type":"v1","addr":"172.21.15.83:6837","nonce":2353406669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-17T10:48:01.206 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 62 2024-04-17T10:48:01.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.210+0000 7f143c2e4640 1 -- 172.21.15.83:0/211918894 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1418066f90 msgr2=0x7f1418069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:01.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.210+0000 7f143c2e4640 1 --2- 172.21.15.83:0/211918894 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1418066f90 0x7f1418069450 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1430009910 tx=0x7f1430008040 comp rx=0 tx=0).stop 2024-04-17T10:48:01.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.210+0000 7f143c2e4640 1 -- 172.21.15.83:0/211918894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1434145760 msgr2=0x7f143413d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:01.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.210+0000 7f143c2e4640 1 --2- 172.21.15.83:0/211918894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1434145760 0x7f143413d3b0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f1434067670 tx=0x7f142c00f4a0 comp rx=0 tx=0).stop 2024-04-17T10:48:01.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.210+0000 7f143c2e4640 1 -- 172.21.15.83:0/211918894 shutdown_connections 2024-04-17T10:48:01.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.210+0000 7f143c2e4640 1 --2- 172.21.15.83:0/211918894 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1418066f90 0x7f1418069450 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:01.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.210+0000 7f143c2e4640 1 --2- 172.21.15.83:0/211918894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1434145760 0x7f143413d3b0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:01.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.210+0000 7f143c2e4640 1 --2- 172.21.15.83:0/211918894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1434145300 0x7f143413ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:01.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.210+0000 7f143c2e4640 1 --2- 172.21.15.83:0/211918894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1420004810 0x7f14341f3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:01.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.210+0000 7f143c2e4640 1 -- 172.21.15.83:0/211918894 >> 172.21.15.83:0/211918894 conn(0x7f143406abe0 msgr2=0x7f143415ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:01.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.210+0000 7f143c2e4640 1 -- 172.21.15.83:0/211918894 shutdown_connections 2024-04-17T10:48:01.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.210+0000 7f143c2e4640 1 -- 172.21.15.83:0/211918894 wait complete. 2024-04-17T10:48:01.223 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-17T10:48:01.223 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-17T10:48:01.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.330+0000 7fb02eb47640 1 Processor -- start 2024-04-17T10:48:01.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.330+0000 7fb02eb47640 1 -- start start 2024-04-17T10:48:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.330+0000 7fb02eb47640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb0280aedd0 0x7fb02806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.330+0000 7fb02eb47640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb02815e440 con 0x7fb02815db50 2024-04-17T10:48:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.330+0000 7fb02eb47640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb02815e720 con 0x7fb02806dae0 2024-04-17T10:48:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.330+0000 7fb02eb47640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb02815ea00 con 0x7fb0280aedd0 2024-04-17T10:48:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.330+0000 7fb02c8e3640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb02815db50 0x7fb028070470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60688/0 (socket says 172.21.15.83:60688) 2024-04-17T10:48:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.330+0000 7fb02c8e3640 1 -- 172.21.15.83:0/2980994633 learned_addr learned my addr 172.21.15.83:0/2980994633 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.330+0000 7fb027fff640 1 --2- 172.21.15.83:0/2980994633 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb0280aedd0 0x7fb02806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.330+0000 7fb0277fe640 1 -- 172.21.15.83:0/2980994633 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1893060072 0 0) 0x7fb02815e440 con 0x7fb02815db50 2024-04-17T10:48:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.330+0000 7fb0277fe640 1 -- 172.21.15.83:0/2980994633 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb014003530 con 0x7fb02815db50 2024-04-17T10:48:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.330+0000 7fb0277fe640 1 -- 172.21.15.83:0/2980994633 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2395208657 0 0) 0x7fb014003530 con 0x7fb02815db50 2024-04-17T10:48:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.330+0000 7fb0277fe640 1 -- 172.21.15.83:0/2980994633 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb0280aedd0 msgr2=0x7fb02806d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.330+0000 7fb0277fe640 1 --2- 172.21.15.83:0/2980994633 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb0280aedd0 0x7fb02806d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.330+0000 7fb0277fe640 1 -- 172.21.15.83:0/2980994633 >> v1:172.21.15.167:6789/0 conn(0x7fb02806dae0 legacy=0x7fb0280b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.330+0000 7fb0277fe640 1 -- 172.21.15.83:0/2980994633 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb02815ece0 con 0x7fb02815db50 2024-04-17T10:48:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.330+0000 7fb0277fe640 1 -- 172.21.15.83:0/2980994633 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fb018001680 con 0x7fb02815db50 2024-04-17T10:48:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.334+0000 7fb0277fe640 1 -- 172.21.15.83:0/2980994633 >> v1:172.21.15.83:6789/0 conn(0x7fb02815db50 legacy=0x7fb028070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.334+0000 7fb0277fe640 1 --2- 172.21.15.83:0/2980994633 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb0280aedd0 0x7fb0140039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.334+0000 7fb0277fe640 1 --2- 172.21.15.83:0/2980994633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb014003f70 0x7fb014004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.334+0000 7fb0277fe640 1 --2- 172.21.15.83:0/2980994633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb014004d50 0x7fb0140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.334+0000 7fb0277fe640 1 -- 172.21.15.83:0/2980994633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02815ea00 con 0x7fb014004d50 2024-04-17T10:48:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.334+0000 7fb0277fe640 1 -- 172.21.15.83:0/2980994633 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01400bab0 con 0x7fb014003f70 2024-04-17T10:48:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.334+0000 7fb0277fe640 1 -- 172.21.15.83:0/2980994633 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb01400bcb0 con 0x7fb0280aedd0 2024-04-17T10:48:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.334+0000 7fb02d0e4640 1 --2- 172.21.15.83:0/2980994633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb014004d50 0x7fb0140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:01.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.334+0000 7fb027fff640 1 --2- 172.21.15.83:0/2980994633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb014003f70 0x7fb014004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:01.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.334+0000 7fb02d0e4640 1 -- 172.21.15.83:0/2980994633 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb0280aedd0 msgr2=0x7fb0140039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:01.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.334+0000 7fb02d0e4640 1 --2- 172.21.15.83:0/2980994633 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb0280aedd0 0x7fb0140039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:01.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.334+0000 7fb02d0e4640 1 -- 172.21.15.83:0/2980994633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb014003f70 msgr2=0x7fb014004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:01.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.334+0000 7fb02d0e4640 1 --2- 172.21.15.83:0/2980994633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb014003f70 0x7fb014004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:01.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.334+0000 7fb02d0e4640 1 -- 172.21.15.83:0/2980994633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb014003530 con 0x7fb014004d50 2024-04-17T10:48:01.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.334+0000 7fb027fff640 1 --2- 172.21.15.83:0/2980994633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb014003f70 0x7fb014004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:01.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.334+0000 7fb02d0e4640 1 --2- 172.21.15.83:0/2980994633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb014004d50 0x7fb0140051d0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fb020001780 tx=0x7fb02001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b695dd9ef7bd3570 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:01.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.334+0000 7fb0277fe640 1 -- 172.21.15.83:0/2980994633 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0200013a0 con 0x7fb014004d50 2024-04-17T10:48:01.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.334+0000 7fb0277fe640 1 -- 172.21.15.83:0/2980994633 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0200a9070 con 0x7fb014004d50 2024-04-17T10:48:01.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.334+0000 7fb0277fe640 1 -- 172.21.15.83:0/2980994633 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb020003600 con 0x7fb014004d50 2024-04-17T10:48:01.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.334+0000 7fb02eb47640 1 -- 172.21.15.83:0/2980994633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb014004d50 msgr2=0x7fb0140051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:01.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.334+0000 7fb02eb47640 1 --2- 172.21.15.83:0/2980994633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb014004d50 0x7fb0140051d0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fb020001780 tx=0x7fb02001c420 comp rx=0 tx=0).stop 2024-04-17T10:48:01.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.334+0000 7fb02d0e4640 1 -- 172.21.15.83:0/2980994633 reap_dead start 2024-04-17T10:48:01.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.334+0000 7fb02eb47640 1 -- 172.21.15.83:0/2980994633 shutdown_connections 2024-04-17T10:48:01.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.334+0000 7fb02eb47640 1 -- 172.21.15.83:0/2980994633 >> 172.21.15.83:0/2980994633 conn(0x7fb02806abe0 msgr2=0x7fb02806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:01.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.334+0000 7fb02eb47640 1 -- 172.21.15.83:0/2980994633 shutdown_connections 2024-04-17T10:48:01.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.334+0000 7fb02eb47640 1 -- 172.21.15.83:0/2980994633 wait complete. 2024-04-17T10:48:01.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.338+0000 7fb02eb47640 1 Processor -- start 2024-04-17T10:48:01.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.338+0000 7fb02eb47640 1 -- start start 2024-04-17T10:48:01.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.338+0000 7fb02eb47640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb014004d50 0x7fb0280b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:01.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.338+0000 7fb02eb47640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb0280b1a70 0x7fb0280c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:01.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.338+0000 7fb02eb47640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0280c77b0 0x7fb0280c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:01.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.338+0000 7fb02eb47640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02815e0b0 con 0x7fb0280c77b0 2024-04-17T10:48:01.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.338+0000 7fb02eb47640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02815df30 con 0x7fb014004d50 2024-04-17T10:48:01.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.338+0000 7fb02eb47640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb02806d5a0 con 0x7fb0280b1a70 2024-04-17T10:48:01.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.338+0000 7fb02d0e4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0280c77b0 0x7fb0280c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:01.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.338+0000 7fb02d0e4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0280c77b0 0x7fb0280c7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46316/0 (socket says 172.21.15.83:46316) 2024-04-17T10:48:01.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.338+0000 7fb02d0e4640 1 -- 172.21.15.83:0/3855914266 learned_addr learned my addr 172.21.15.83:0/3855914266 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:01.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.338+0000 7fb02c8e3640 1 --2- 172.21.15.83:0/3855914266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb014004d50 0x7fb0280b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:01.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.338+0000 7fb027fff640 1 --2- 172.21.15.83:0/3855914266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb0280b1a70 0x7fb0280c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:01.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.338+0000 7fb02d0e4640 1 -- 172.21.15.83:0/3855914266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb0280b1a70 msgr2=0x7fb0280c67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:01.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.338+0000 7fb02d0e4640 1 --2- 172.21.15.83:0/3855914266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb0280b1a70 0x7fb0280c67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:01.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.338+0000 7fb02d0e4640 1 -- 172.21.15.83:0/3855914266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb014004d50 msgr2=0x7fb0280b0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:01.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.338+0000 7fb02d0e4640 1 --2- 172.21.15.83:0/3855914266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb014004d50 0x7fb0280b0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:01.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.338+0000 7fb02d0e4640 1 -- 172.21.15.83:0/3855914266 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0280b09d0 con 0x7fb0280c77b0 2024-04-17T10:48:01.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.338+0000 7fb02d0e4640 1 --2- 172.21.15.83:0/3855914266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0280c77b0 0x7fb0280c7c60 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fb020005040 tx=0x7fb014005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:01.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.338+0000 7fb0257fa640 1 -- 172.21.15.83:0/3855914266 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb020007a40 con 0x7fb0280c77b0 2024-04-17T10:48:01.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.338+0000 7fb0257fa640 1 -- 172.21.15.83:0/3855914266 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0200bc070 con 0x7fb0280c77b0 2024-04-17T10:48:01.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.338+0000 7fb02eb47640 1 -- 172.21.15.83:0/3855914266 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0280b0cb0 con 0x7fb0280c77b0 2024-04-17T10:48:01.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.338+0000 7fb0257fa640 1 -- 172.21.15.83:0/3855914266 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb020007450 con 0x7fb0280c77b0 2024-04-17T10:48:01.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.338+0000 7fb02eb47640 1 -- 172.21.15.83:0/3855914266 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0280b1270 con 0x7fb0280c77b0 2024-04-17T10:48:01.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.338+0000 7fb02eb47640 1 -- 172.21.15.83:0/3855914266 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb028074f90 con 0x7fb0280c77b0 2024-04-17T10:48:01.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.338+0000 7fb0257fa640 1 -- 172.21.15.83:0/3855914266 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb0200b5070 con 0x7fb0280c77b0 2024-04-17T10:48:01.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.342+0000 7fb0257fa640 1 --2- 172.21.15.83:0/3855914266 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb008066f90 0x7fb008069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:01.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.342+0000 7fb0257fa640 1 -- 172.21.15.83:0/3855914266 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb02011e850 con 0x7fb0280c77b0 2024-04-17T10:48:01.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.342+0000 7fb02c8e3640 1 --2- 172.21.15.83:0/3855914266 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb008066f90 0x7fb008069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:01.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.342+0000 7fb02c8e3640 1 --2- 172.21.15.83:0/3855914266 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb008066f90 0x7fb008069450 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb0180011f0 tx=0x7fb018001ad0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:01.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.346+0000 7fb0257fa640 1 -- 172.21.15.83:0/3855914266 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb020016130 con 0x7fb0280c77b0 2024-04-17T10:48:01.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.550+0000 7fb02eb47640 1 -- 172.21.15.83:0/3855914266 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb0280c5a20 con 0x7fb0280c77b0 2024-04-17T10:48:01.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.550+0000 7fb0257fa640 1 -- 172.21.15.83:0/3855914266 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1446 (secure 0 0 0) 0x7fb0200ef980 con 0x7fb0280c77b0 2024-04-17T10:48:01.546 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:01.546 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":614,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":103,"num_osds":8,"num_up_osds":8,"osd_up_since":1713350279,"num_in_osds":8,"osd_in_since":1713350270,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076816697,"bytes_used":2697879552,"bytes_avail":770396233728,"bytes_total":773094113280,"write_bytes_sec":76046202,"read_op_per_sec":0,"write_op_per_sec":76},"fsmap":{"epoch":62,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8049}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-04-17T10:47:18.345526+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-17T10:48:01.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.550+0000 7fb02eb47640 1 -- 172.21.15.83:0/3855914266 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb008066f90 msgr2=0x7fb008069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:01.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.550+0000 7fb02eb47640 1 --2- 172.21.15.83:0/3855914266 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb008066f90 0x7fb008069450 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb0180011f0 tx=0x7fb018001ad0 comp rx=0 tx=0).stop 2024-04-17T10:48:01.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.550+0000 7fb02eb47640 1 -- 172.21.15.83:0/3855914266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0280c77b0 msgr2=0x7fb0280c7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:01.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.550+0000 7fb02eb47640 1 --2- 172.21.15.83:0/3855914266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0280c77b0 0x7fb0280c7c60 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fb020005040 tx=0x7fb014005840 comp rx=0 tx=0).stop 2024-04-17T10:48:01.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.550+0000 7fb02eb47640 1 -- 172.21.15.83:0/3855914266 shutdown_connections 2024-04-17T10:48:01.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.550+0000 7fb02eb47640 1 --2- 172.21.15.83:0/3855914266 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb008066f90 0x7fb008069450 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:01.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.550+0000 7fb02eb47640 1 --2- 172.21.15.83:0/3855914266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0280c77b0 0x7fb0280c7c60 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:01.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.550+0000 7fb02eb47640 1 --2- 172.21.15.83:0/3855914266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb0280b1a70 0x7fb0280c67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:01.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.550+0000 7fb02eb47640 1 --2- 172.21.15.83:0/3855914266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb014004d50 0x7fb0280b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:01.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.550+0000 7fb02eb47640 1 -- 172.21.15.83:0/3855914266 >> 172.21.15.83:0/3855914266 conn(0x7fb02806abe0 msgr2=0x7fb0280cc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:01.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.550+0000 7fb02eb47640 1 -- 172.21.15.83:0/3855914266 shutdown_connections 2024-04-17T10:48:01.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:01.550+0000 7fb02eb47640 1 -- 172.21.15.83:0/3855914266 wait complete. 2024-04-17T10:48:01.563 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-17T10:48:02.564 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:48:02.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.698+0000 7fee47295640 1 Processor -- start 2024-04-17T10:48:02.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.698+0000 7fee47295640 1 -- start start 2024-04-17T10:48:02.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.702+0000 7fee47295640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee4015ba30 0x7fee40072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:02.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.702+0000 7fee47295640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fee4015c960 0x7fee400cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:02.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.702+0000 7fee47295640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fee400cfa10 0x7fee4006c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:02.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.702+0000 7fee47295640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee40071ea0 con 0x7fee400cfa10 2024-04-17T10:48:02.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.702+0000 7fee47295640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee400cfea0 con 0x7fee4015c960 2024-04-17T10:48:02.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.702+0000 7fee47295640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4006e320 con 0x7fee4015ba30 2024-04-17T10:48:02.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.702+0000 7fee45832640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fee400cfa10 0x7fee4006c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:02.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.702+0000 7fee45832640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fee400cfa10 0x7fee4006c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46324/0 (socket says 172.21.15.83:46324) 2024-04-17T10:48:02.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.702+0000 7fee45832640 1 -- 172.21.15.83:0/1298124081 learned_addr learned my addr 172.21.15.83:0/1298124081 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:02.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.702+0000 7fee45031640 1 --2- 172.21.15.83:0/1298124081 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee4015ba30 0x7fee40072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:02.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.702+0000 7fee44830640 1 --2- 172.21.15.83:0/1298124081 >> v2:172.21.15.167:3300/0 conn(0x7fee4015c960 0x7fee400cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:02.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.702+0000 7fee45832640 1 -- 172.21.15.83:0/1298124081 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee4015ba30 msgr2=0x7fee40072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:02.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.702+0000 7fee45832640 1 --2- 172.21.15.83:0/1298124081 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee4015ba30 0x7fee40072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:02.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.702+0000 7fee45832640 1 -- 172.21.15.83:0/1298124081 >> v2:172.21.15.167:3300/0 conn(0x7fee4015c960 msgr2=0x7fee400cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:02.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.702+0000 7fee45832640 1 --2- 172.21.15.83:0/1298124081 >> v2:172.21.15.167:3300/0 conn(0x7fee4015c960 0x7fee400cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:02.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.702+0000 7fee45832640 1 -- 172.21.15.83:0/1298124081 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4006e040 con 0x7fee400cfa10 2024-04-17T10:48:02.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.702+0000 7fee45832640 1 --2- 172.21.15.83:0/1298124081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee400cfa10 0x7fee4006c490 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fee40070060 tx=0x7fee3801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9e23d4680d7d80 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:02.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.702+0000 7fee37fff640 1 -- 172.21.15.83:0/1298124081 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee380a56d0 con 0x7fee400cfa10 2024-04-17T10:48:02.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.702+0000 7fee37fff640 1 -- 172.21.15.83:0/1298124081 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee380a5870 con 0x7fee400cfa10 2024-04-17T10:48:02.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.702+0000 7fee37fff640 1 -- 172.21.15.83:0/1298124081 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee380a5c40 con 0x7fee400cfa10 2024-04-17T10:48:02.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.706+0000 7fee47295640 1 -- 172.21.15.83:0/1298124081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee400cfa10 msgr2=0x7fee4006c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:02.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.706+0000 7fee47295640 1 --2- 172.21.15.83:0/1298124081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee400cfa10 0x7fee4006c490 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fee40070060 tx=0x7fee3801c960 comp rx=0 tx=0).stop 2024-04-17T10:48:02.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.706+0000 7fee47295640 1 -- 172.21.15.83:0/1298124081 shutdown_connections 2024-04-17T10:48:02.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.706+0000 7fee47295640 1 --2- 172.21.15.83:0/1298124081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee400cfa10 0x7fee4006c490 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:02.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.706+0000 7fee47295640 1 --2- 172.21.15.83:0/1298124081 >> v2:172.21.15.167:3300/0 conn(0x7fee4015c960 0x7fee400cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:02.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.706+0000 7fee47295640 1 --2- 172.21.15.83:0/1298124081 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee4015ba30 0x7fee40072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:02.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.706+0000 7fee47295640 1 -- 172.21.15.83:0/1298124081 >> 172.21.15.83:0/1298124081 conn(0x7fee4006abe0 msgr2=0x7fee4006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:02.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.706+0000 7fee47295640 1 -- 172.21.15.83:0/1298124081 shutdown_connections 2024-04-17T10:48:02.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.706+0000 7fee47295640 1 -- 172.21.15.83:0/1298124081 wait complete. 2024-04-17T10:48:02.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.706+0000 7fee47295640 1 Processor -- start 2024-04-17T10:48:02.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.706+0000 7fee47295640 1 -- start start 2024-04-17T10:48:02.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.706+0000 7fee47295640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee400cfa10 0x7fee4014c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:02.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.706+0000 7fee47295640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee4015ba30 0x7fee4014a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:02.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.706+0000 7fee47295640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee4015c960 0x7fee4013cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:02.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.706+0000 7fee47295640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4006de20 con 0x7fee4015c960 2024-04-17T10:48:02.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.706+0000 7fee47295640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4006dca0 con 0x7fee4015ba30 2024-04-17T10:48:02.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.706+0000 7fee47295640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4015cd40 con 0x7fee400cfa10 2024-04-17T10:48:02.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.706+0000 7fee45832640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee4015c960 0x7fee4013cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:02.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.706+0000 7fee45832640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee4015c960 0x7fee4013cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46334/0 (socket says 172.21.15.83:46334) 2024-04-17T10:48:02.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.706+0000 7fee45832640 1 -- 172.21.15.83:0/4213203503 learned_addr learned my addr 172.21.15.83:0/4213203503 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:02.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.706+0000 7fee44830640 1 --2- 172.21.15.83:0/4213203503 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee4015ba30 0x7fee4014a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:02.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.706+0000 7fee45031640 1 --2- 172.21.15.83:0/4213203503 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee400cfa10 0x7fee4014c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:02.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.706+0000 7fee45832640 1 -- 172.21.15.83:0/4213203503 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee400cfa10 msgr2=0x7fee4014c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:02.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.706+0000 7fee45832640 1 --2- 172.21.15.83:0/4213203503 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee400cfa10 0x7fee4014c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:02.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.706+0000 7fee45832640 1 -- 172.21.15.83:0/4213203503 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee4015ba30 msgr2=0x7fee4014a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:02.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.706+0000 7fee45832640 1 --2- 172.21.15.83:0/4213203503 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee4015ba30 0x7fee4014a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:02.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.706+0000 7fee45832640 1 -- 172.21.15.83:0/4213203503 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4014d8a0 con 0x7fee4015c960 2024-04-17T10:48:02.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.706+0000 7fee45832640 1 --2- 172.21.15.83:0/4213203503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee4015c960 0x7fee4013cb80 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fee3801b010 tx=0x7fee38019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:02.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.706+0000 7fee35ffb640 1 -- 172.21.15.83:0/4213203503 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee380bc020 con 0x7fee4015c960 2024-04-17T10:48:02.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.706+0000 7fee35ffb640 1 -- 172.21.15.83:0/4213203503 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee380b8670 con 0x7fee4015c960 2024-04-17T10:48:02.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.706+0000 7fee47295640 1 -- 172.21.15.83:0/4213203503 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee4014b930 con 0x7fee4015c960 2024-04-17T10:48:02.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.710+0000 7fee35ffb640 1 -- 172.21.15.83:0/4213203503 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee38009510 con 0x7fee4015c960 2024-04-17T10:48:02.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.710+0000 7fee47295640 1 -- 172.21.15.83:0/4213203503 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee4014bef0 con 0x7fee4015c960 2024-04-17T10:48:02.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.710+0000 7fee47295640 1 -- 172.21.15.83:0/4213203503 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee40074f90 con 0x7fee4015c960 2024-04-17T10:48:02.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.710+0000 7fee35ffb640 1 -- 172.21.15.83:0/4213203503 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fee380bf020 con 0x7fee4015c960 2024-04-17T10:48:02.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.710+0000 7fee35ffb640 1 --2- 172.21.15.83:0/4213203503 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fee24066f60 0x7fee24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:02.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.710+0000 7fee35ffb640 1 -- 172.21.15.83:0/4213203503 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fee3811ff50 con 0x7fee4015c960 2024-04-17T10:48:02.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.710+0000 7fee45031640 1 --2- 172.21.15.83:0/4213203503 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fee24066f60 0x7fee24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:02.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.710+0000 7fee45031640 1 --2- 172.21.15.83:0/4213203503 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fee24066f60 0x7fee24069420 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fee3c0045f0 tx=0x7fee3c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:02.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.714+0000 7fee35ffb640 1 -- 172.21.15.83:0/4213203503 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fee380c6030 con 0x7fee4015c960 2024-04-17T10:48:02.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.902+0000 7fee47295640 1 -- 172.21.15.83:0/4213203503 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee4013d0c0 con 0x7fee4015c960 2024-04-17T10:48:02.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.902+0000 7fee35ffb640 1 -- 172.21.15.83:0/4213203503 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fee380165e0 con 0x7fee4015c960 2024-04-17T10:48:02.900 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:02.900 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/437268957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":437268957},{"type":"v1","addr":"172.21.15.167:6833","nonce":437268957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":6253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3214640674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3214640674},{"type":"v1","addr":"172.21.15.167:6835","nonce":3214640674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":8046,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/715498323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":715498323},{"type":"v1","addr":"172.21.15.83:6835","nonce":715498323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:47:29.385103+0000","modified":"2024-04-17T10:47:31.389088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8049":{"gid":8049,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2353406669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":2353406669},{"type":"v1","addr":"172.21.15.83:6837","nonce":2353406669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-17T10:48:02.900 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 62 2024-04-17T10:48:02.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.906+0000 7fee47295640 1 -- 172.21.15.83:0/4213203503 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fee24066f60 msgr2=0x7fee24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:02.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.906+0000 7fee47295640 1 --2- 172.21.15.83:0/4213203503 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fee24066f60 0x7fee24069420 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fee3c0045f0 tx=0x7fee3c008040 comp rx=0 tx=0).stop 2024-04-17T10:48:02.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.906+0000 7fee47295640 1 -- 172.21.15.83:0/4213203503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee4015c960 msgr2=0x7fee4013cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:02.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.906+0000 7fee47295640 1 --2- 172.21.15.83:0/4213203503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee4015c960 0x7fee4013cb80 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fee3801b010 tx=0x7fee38019700 comp rx=0 tx=0).stop 2024-04-17T10:48:02.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.906+0000 7fee47295640 1 -- 172.21.15.83:0/4213203503 shutdown_connections 2024-04-17T10:48:02.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.906+0000 7fee47295640 1 --2- 172.21.15.83:0/4213203503 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fee24066f60 0x7fee24069420 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:02.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.906+0000 7fee47295640 1 --2- 172.21.15.83:0/4213203503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee4015c960 0x7fee4013cb80 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:02.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.906+0000 7fee47295640 1 --2- 172.21.15.83:0/4213203503 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee4015ba30 0x7fee4014a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:02.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.906+0000 7fee47295640 1 --2- 172.21.15.83:0/4213203503 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee400cfa10 0x7fee4014c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:02.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.906+0000 7fee47295640 1 -- 172.21.15.83:0/4213203503 >> 172.21.15.83:0/4213203503 conn(0x7fee4006abe0 msgr2=0x7fee40073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:02.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.906+0000 7fee47295640 1 -- 172.21.15.83:0/4213203503 shutdown_connections 2024-04-17T10:48:02.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:02.906+0000 7fee47295640 1 -- 172.21.15.83:0/4213203503 wait complete. 2024-04-17T10:48:02.917 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-17T10:48:02.917 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-17T10:48:03.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.042+0000 7f69e32aa640 1 Processor -- start 2024-04-17T10:48:03.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.042+0000 7f69e32aa640 1 -- start start 2024-04-17T10:48:03.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.042+0000 7f69e32aa640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f69dc06bbf0 0x7f69dc06c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.042+0000 7f69e32aa640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69dc071750 0x7f69dc071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.042+0000 7f69e32aa640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69dc161900 con 0x7f69dc161300 2024-04-17T10:48:03.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.042+0000 7f69e32aa640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69dc161ba0 con 0x7f69dc06bbf0 2024-04-17T10:48:03.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.042+0000 7f69e32aa640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69dc161e80 con 0x7f69dc071750 2024-04-17T10:48:03.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.042+0000 7f69e0845640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f69dc06bbf0 0x7f69dc06c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.042+0000 7f69e0845640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f69dc06bbf0 0x7f69dc06c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46342/0 (socket says 172.21.15.83:46342) 2024-04-17T10:48:03.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.042+0000 7f69e0845640 1 -- 172.21.15.83:0/3424610780 learned_addr learned my addr 172.21.15.83:0/3424610780 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:03.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.042+0000 7f69e1847640 1 --2- 172.21.15.83:0/3424610780 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69dc071750 0x7f69dc071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.042+0000 7f69e0845640 1 -- 172.21.15.83:0/3424610780 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69dc071750 msgr2=0x7f69dc071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:03.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.042+0000 7f69e0845640 1 --2- 172.21.15.83:0/3424610780 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69dc071750 0x7f69dc071be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.042+0000 7f69e0845640 1 -- 172.21.15.83:0/3424610780 >> v1:172.21.15.167:6789/0 conn(0x7f69dc161300 legacy=0x7f69dc06ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:03.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.042+0000 7f69e0845640 1 -- 172.21.15.83:0/3424610780 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69dc162160 con 0x7f69dc06bbf0 2024-04-17T10:48:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.046+0000 7f69e0845640 1 --2- 172.21.15.83:0/3424610780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69dc06bbf0 0x7f69dc06c070 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009870 tx=0x7f69d801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef81d9272dab8215 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.046+0000 7f69d3fff640 1 -- 172.21.15.83:0/3424610780 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d80b0070 con 0x7f69dc06bbf0 2024-04-17T10:48:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.046+0000 7f69d3fff640 1 -- 172.21.15.83:0/3424610780 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69d80ab600 con 0x7f69dc06bbf0 2024-04-17T10:48:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.046+0000 7f69d3fff640 1 -- 172.21.15.83:0/3424610780 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d80ab9d0 con 0x7f69dc06bbf0 2024-04-17T10:48:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.046+0000 7f69e32aa640 1 -- 172.21.15.83:0/3424610780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69dc06bbf0 msgr2=0x7f69dc06c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.046+0000 7f69e32aa640 1 --2- 172.21.15.83:0/3424610780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69dc06bbf0 0x7f69dc06c070 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009870 tx=0x7f69d801c4d0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.046+0000 7f69e32aa640 1 -- 172.21.15.83:0/3424610780 shutdown_connections 2024-04-17T10:48:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.046+0000 7f69e32aa640 1 --2- 172.21.15.83:0/3424610780 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69dc071750 0x7f69dc071be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.046+0000 7f69e32aa640 1 --2- 172.21.15.83:0/3424610780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69dc06bbf0 0x7f69dc06c070 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.046+0000 7f69e32aa640 1 -- 172.21.15.83:0/3424610780 >> 172.21.15.83:0/3424610780 conn(0x7f69dc0cc910 msgr2=0x7f69dc0ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.046+0000 7f69e32aa640 1 -- 172.21.15.83:0/3424610780 shutdown_connections 2024-04-17T10:48:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.046+0000 7f69e32aa640 1 -- 172.21.15.83:0/3424610780 wait complete. 2024-04-17T10:48:03.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.050+0000 7f69e32aa640 1 Processor -- start 2024-04-17T10:48:03.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.050+0000 7f69e32aa640 1 -- start start 2024-04-17T10:48:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.050+0000 7f69e32aa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69dc06bbf0 0x7f69dc159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.050+0000 7f69e32aa640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69dc071750 0x7f69dc155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.050+0000 7f69e32aa640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69dc161300 0x7f69dc09a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.050+0000 7f69e32aa640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69dc16af30 con 0x7f69dc06bbf0 2024-04-17T10:48:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.050+0000 7f69e32aa640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69dc16adb0 con 0x7f69dc071750 2024-04-17T10:48:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.050+0000 7f69e32aa640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69dc16b0b0 con 0x7f69dc161300 2024-04-17T10:48:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.050+0000 7f69e1046640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69dc06bbf0 0x7f69dc159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.050+0000 7f69e1046640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69dc06bbf0 0x7f69dc159290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46346/0 (socket says 172.21.15.83:46346) 2024-04-17T10:48:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.050+0000 7f69e1046640 1 -- 172.21.15.83:0/1595931161 learned_addr learned my addr 172.21.15.83:0/1595931161 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.050+0000 7f69e0845640 1 --2- 172.21.15.83:0/1595931161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69dc071750 0x7f69dc155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.050+0000 7f69e1847640 1 --2- 172.21.15.83:0/1595931161 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69dc161300 0x7f69dc09a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.050+0000 7f69e1046640 1 -- 172.21.15.83:0/1595931161 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69dc161300 msgr2=0x7f69dc09a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.050+0000 7f69e1046640 1 --2- 172.21.15.83:0/1595931161 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69dc161300 0x7f69dc09a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.050+0000 7f69e1046640 1 -- 172.21.15.83:0/1595931161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69dc071750 msgr2=0x7f69dc155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.050+0000 7f69e1046640 1 --2- 172.21.15.83:0/1595931161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69dc071750 0x7f69dc155e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.050+0000 7f69e1046640 1 -- 172.21.15.83:0/1595931161 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69dc159870 con 0x7f69dc06bbf0 2024-04-17T10:48:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.050+0000 7f69e0845640 1 --2- 172.21.15.83:0/1595931161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69dc071750 0x7f69dc155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.050+0000 7f69e1046640 1 --2- 172.21.15.83:0/1595931161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69dc06bbf0 0x7f69dc159290 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f69cc00c370 tx=0x7f69cc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.050+0000 7f69d1ffb640 1 -- 172.21.15.83:0/1595931161 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69cc0052d0 con 0x7f69dc06bbf0 2024-04-17T10:48:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.050+0000 7f69d1ffb640 1 -- 172.21.15.83:0/1595931161 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69cc005470 con 0x7f69dc06bbf0 2024-04-17T10:48:03.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.050+0000 7f69e32aa640 1 -- 172.21.15.83:0/1595931161 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69dc0b2b60 con 0x7f69dc06bbf0 2024-04-17T10:48:03.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.050+0000 7f69d1ffb640 1 -- 172.21.15.83:0/1595931161 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69cc007260 con 0x7f69dc06bbf0 2024-04-17T10:48:03.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.050+0000 7f69e32aa640 1 -- 172.21.15.83:0/1595931161 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69dc159b50 con 0x7f69dc06bbf0 2024-04-17T10:48:03.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.050+0000 7f69e32aa640 1 -- 172.21.15.83:0/1595931161 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69dc074f90 con 0x7f69dc06bbf0 2024-04-17T10:48:03.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.050+0000 7f69d1ffb640 1 -- 172.21.15.83:0/1595931161 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f69cc007400 con 0x7f69dc06bbf0 2024-04-17T10:48:03.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.050+0000 7f69d1ffb640 1 --2- 172.21.15.83:0/1595931161 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f69c0066f60 0x7f69c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.050+0000 7f69d1ffb640 1 -- 172.21.15.83:0/1595931161 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69cc084400 con 0x7f69dc06bbf0 2024-04-17T10:48:03.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.050+0000 7f69e0845640 1 --2- 172.21.15.83:0/1595931161 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f69c0066f60 0x7f69c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.054+0000 7f69e0845640 1 --2- 172.21.15.83:0/1595931161 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f69c0066f60 0x7f69c0069420 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f69d80ad000 tx=0x7f69d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:03.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.058+0000 7f69d1ffb640 1 -- 172.21.15.83:0/1595931161 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f69cc0554a0 con 0x7f69dc06bbf0 2024-04-17T10:48:03.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.274+0000 7f69e32aa640 1 -- 172.21.15.83:0/1595931161 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f69dc066d20 con 0x7f69dc06bbf0 2024-04-17T10:48:03.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.274+0000 7f69d1ffb640 1 -- 172.21.15.83:0/1595931161 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f69cc054da0 con 0x7f69dc06bbf0 2024-04-17T10:48:03.270 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:03.270 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":615,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":103,"num_osds":8,"num_up_osds":8,"osd_up_since":1713350279,"num_in_osds":8,"osd_in_since":1713350270,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076816697,"bytes_used":2697879552,"bytes_avail":770396233728,"bytes_total":773094113280,"write_bytes_sec":48843159,"read_op_per_sec":0,"write_op_per_sec":49},"fsmap":{"epoch":62,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8049}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-04-17T10:48:02.357277+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-17T10:48:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.274+0000 7f69e32aa640 1 -- 172.21.15.83:0/1595931161 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f69c0066f60 msgr2=0x7f69c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.274+0000 7f69e32aa640 1 --2- 172.21.15.83:0/1595931161 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f69c0066f60 0x7f69c0069420 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f69d80ad000 tx=0x7f69d8008040 comp rx=0 tx=0).stop 2024-04-17T10:48:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.274+0000 7f69e32aa640 1 -- 172.21.15.83:0/1595931161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69dc06bbf0 msgr2=0x7f69dc159290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.274+0000 7f69e32aa640 1 --2- 172.21.15.83:0/1595931161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69dc06bbf0 0x7f69dc159290 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f69cc00c370 tx=0x7f69cc00c830 comp rx=0 tx=0).stop 2024-04-17T10:48:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.274+0000 7f69e32aa640 1 -- 172.21.15.83:0/1595931161 shutdown_connections 2024-04-17T10:48:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.274+0000 7f69e32aa640 1 --2- 172.21.15.83:0/1595931161 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f69c0066f60 0x7f69c0069420 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.274+0000 7f69e32aa640 1 --2- 172.21.15.83:0/1595931161 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69dc161300 0x7f69dc09a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.274+0000 7f69e32aa640 1 --2- 172.21.15.83:0/1595931161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69dc071750 0x7f69dc155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.274+0000 7f69e32aa640 1 --2- 172.21.15.83:0/1595931161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69dc06bbf0 0x7f69dc159290 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.274+0000 7f69e32aa640 1 -- 172.21.15.83:0/1595931161 >> 172.21.15.83:0/1595931161 conn(0x7f69dc0cc910 msgr2=0x7f69dc15e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.274+0000 7f69e32aa640 1 -- 172.21.15.83:0/1595931161 shutdown_connections 2024-04-17T10:48:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.274+0000 7f69e32aa640 1 -- 172.21.15.83:0/1595931161 wait complete. 2024-04-17T10:48:03.288 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused'} 2024-04-17T10:48:03.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:03.288 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub resume 2024-04-17T10:48:03.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.410+0000 7f7653fa0640 1 Processor -- start 2024-04-17T10:48:03.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.410+0000 7f7653fa0640 1 -- start start 2024-04-17T10:48:03.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.410+0000 7f7653fa0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f764c0b4090 0x7f764c06c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.410+0000 7f7653fa0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f764c0b4520 0x7f764c06cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.410+0000 7f7653fa0640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f764c0af160 con 0x7f764c154670 2024-04-17T10:48:03.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.414+0000 7f7653fa0640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f764c154a50 con 0x7f764c0b4090 2024-04-17T10:48:03.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.414+0000 7f7653fa0640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764c0b2ff0 con 0x7f764c0b4520 2024-04-17T10:48:03.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.414+0000 7f765153b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f764c0b4090 0x7f764c06c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.414+0000 7f7651d3c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f764c154670 0x7f764c0aea50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60690/0 (socket says 172.21.15.83:60690) 2024-04-17T10:48:03.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.414+0000 7f7651d3c640 1 -- 172.21.15.83:0/4284082823 learned_addr learned my addr 172.21.15.83:0/4284082823 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:03.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.414+0000 7f765253d640 1 --2- 172.21.15.83:0/4284082823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f764c0b4520 0x7f764c06cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.414+0000 7f765153b640 1 -- 172.21.15.83:0/4284082823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f764c0b4520 msgr2=0x7f764c06cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:03.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.414+0000 7f765153b640 1 --2- 172.21.15.83:0/4284082823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f764c0b4520 0x7f764c06cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.414+0000 7f765153b640 1 -- 172.21.15.83:0/4284082823 >> v1:172.21.15.83:6789/0 conn(0x7f764c154670 legacy=0x7f764c0aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:03.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.414+0000 7f7650d3a640 1 -- 172.21.15.83:0/4284082823 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2080911501 0 0) 0x7f764c0af160 con 0x7f764c154670 2024-04-17T10:48:03.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.414+0000 7f765153b640 1 -- 172.21.15.83:0/4284082823 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764c0b32d0 con 0x7f764c0b4090 2024-04-17T10:48:03.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.414+0000 7f765253d640 1 --2- 172.21.15.83:0/4284082823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f764c0b4520 0x7f764c06cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:03.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.414+0000 7f765153b640 1 --2- 172.21.15.83:0/4284082823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f764c0b4090 0x7f764c06c570 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f763c0097c0 tx=0x7f763c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daaf0ca6ca41a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:03.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.414+0000 7f7650d3a640 1 -- 172.21.15.83:0/4284082823 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c0b0070 con 0x7f764c0b4090 2024-04-17T10:48:03.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.414+0000 7f7650d3a640 1 -- 172.21.15.83:0/4284082823 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763c0ab600 con 0x7f764c0b4090 2024-04-17T10:48:03.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.414+0000 7f7650d3a640 1 -- 172.21.15.83:0/4284082823 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c0ab9d0 con 0x7f764c0b4090 2024-04-17T10:48:03.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.414+0000 7f7653fa0640 1 -- 172.21.15.83:0/4284082823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f764c0b4090 msgr2=0x7f764c06c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:03.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.414+0000 7f7653fa0640 1 --2- 172.21.15.83:0/4284082823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f764c0b4090 0x7f764c06c570 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f763c0097c0 tx=0x7f763c01c470 comp rx=0 tx=0).stop 2024-04-17T10:48:03.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.414+0000 7f7653fa0640 1 -- 172.21.15.83:0/4284082823 shutdown_connections 2024-04-17T10:48:03.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.414+0000 7f7653fa0640 1 --2- 172.21.15.83:0/4284082823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f764c0b4520 0x7f764c06cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.414+0000 7f7653fa0640 1 --2- 172.21.15.83:0/4284082823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f764c0b4090 0x7f764c06c570 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.414+0000 7f7653fa0640 1 -- 172.21.15.83:0/4284082823 >> 172.21.15.83:0/4284082823 conn(0x7f764c0b15b0 msgr2=0x7f764c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:03.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.414+0000 7f7653fa0640 1 -- 172.21.15.83:0/4284082823 shutdown_connections 2024-04-17T10:48:03.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.414+0000 7f7653fa0640 1 -- 172.21.15.83:0/4284082823 wait complete. 2024-04-17T10:48:03.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.418+0000 7f7653fa0640 1 Processor -- start 2024-04-17T10:48:03.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.418+0000 7f7653fa0640 1 -- start start 2024-04-17T10:48:03.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.418+0000 7f7653fa0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f764c0b4090 0x7f764c153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.418+0000 7f7653fa0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f764c0b4520 0x7f764c14f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.418+0000 7f7653fa0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f764c154670 0x7f764c070c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.418+0000 7f7653fa0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c070470 con 0x7f764c0b4090 2024-04-17T10:48:03.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.418+0000 7f7653fa0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c15b3c0 con 0x7f764c154670 2024-04-17T10:48:03.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.418+0000 7f7653fa0640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764c0705f0 con 0x7f764c0b4520 2024-04-17T10:48:03.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.418+0000 7f7651d3c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f764c0b4090 0x7f764c153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.418+0000 7f7651d3c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f764c0b4090 0x7f764c153b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46370/0 (socket says 172.21.15.83:46370) 2024-04-17T10:48:03.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.418+0000 7f7651d3c640 1 -- 172.21.15.83:0/3684778374 learned_addr learned my addr 172.21.15.83:0/3684778374 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:03.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.418+0000 7f765153b640 1 --2- 172.21.15.83:0/3684778374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f764c0b4520 0x7f764c14f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.418+0000 7f765253d640 1 --2- 172.21.15.83:0/3684778374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f764c154670 0x7f764c070c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.418+0000 7f7651d3c640 1 -- 172.21.15.83:0/3684778374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f764c0b4520 msgr2=0x7f764c14f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:03.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.418+0000 7f7651d3c640 1 --2- 172.21.15.83:0/3684778374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f764c0b4520 0x7f764c14f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.418+0000 7f7651d3c640 1 -- 172.21.15.83:0/3684778374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f764c154670 msgr2=0x7f764c070c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:03.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.418+0000 7f7651d3c640 1 --2- 172.21.15.83:0/3684778374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f764c154670 0x7f764c070c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.418+0000 7f7651d3c640 1 -- 172.21.15.83:0/3684778374 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764c14fa40 con 0x7f764c0b4090 2024-04-17T10:48:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.418+0000 7f765153b640 1 --2- 172.21.15.83:0/3684778374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f764c0b4520 0x7f764c14f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.418+0000 7f7651d3c640 1 --2- 172.21.15.83:0/3684778374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f764c0b4090 0x7f764c153b40 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f764800c480 tx=0x7f764800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.418+0000 7f76427fc640 1 -- 172.21.15.83:0/3684778374 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7648018070 con 0x7f764c0b4090 2024-04-17T10:48:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.418+0000 7f76427fc640 1 -- 172.21.15.83:0/3684778374 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7648007500 con 0x7f764c0b4090 2024-04-17T10:48:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.418+0000 7f7653fa0640 1 -- 172.21.15.83:0/3684778374 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f764c14fd20 con 0x7f764c0b4090 2024-04-17T10:48:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.418+0000 7f76427fc640 1 -- 172.21.15.83:0/3684778374 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7648004a30 con 0x7f764c0b4090 2024-04-17T10:48:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.418+0000 7f7653fa0640 1 -- 172.21.15.83:0/3684778374 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764c1502e0 con 0x7f764c0b4090 2024-04-17T10:48:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.418+0000 7f76427fc640 1 -- 172.21.15.83:0/3684778374 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f764801b070 con 0x7f764c0b4090 2024-04-17T10:48:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f76427fc640 1 --2- 172.21.15.83:0/3684778374 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7630066f60 0x7f7630069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f765153b640 1 --2- 172.21.15.83:0/3684778374 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7630066f60 0x7f7630069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f76427fc640 1 -- 172.21.15.83:0/3684778374 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7648084360 con 0x7f764c0b4090 2024-04-17T10:48:03.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f765153b640 1 --2- 172.21.15.83:0/3684778374 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7630066f60 0x7f7630069420 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f763c0ad000 tx=0x7f763c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:03.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f765710d000 1 Processor -- start 2024-04-17T10:48:03.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f765710d000 1 -- start start 2024-04-17T10:48:03.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f765710d000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eda92f99d0 0x55eda92f9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f765710d000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eda92fa2f0 0x55eda92afb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f765710d000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eda92b00a0 0x55eda92b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f765710d000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eda90fcff0 con 0x55eda92f99d0 2024-04-17T10:48:03.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f765710d000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eda9085150 con 0x55eda92fa2f0 2024-04-17T10:48:03.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f765710d000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eda90f9090 con 0x55eda92b00a0 2024-04-17T10:48:03.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f7651d3c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eda92f99d0 0x55eda92f9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f7651d3c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eda92f99d0 0x55eda92f9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46372/0 (socket says 172.21.15.83:46372) 2024-04-17T10:48:03.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f7651d3c640 1 -- 172.21.15.83:0/2173381401 learned_addr learned my addr 172.21.15.83:0/2173381401 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:03.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f765253d640 1 --2- 172.21.15.83:0/2173381401 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eda92b00a0 0x55eda92b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f765153b640 1 --2- 172.21.15.83:0/2173381401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eda92fa2f0 0x55eda92afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f7651d3c640 1 -- 172.21.15.83:0/2173381401 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eda92b00a0 msgr2=0x55eda92b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:03.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f7651d3c640 1 --2- 172.21.15.83:0/2173381401 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eda92b00a0 0x55eda92b0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f7651d3c640 1 -- 172.21.15.83:0/2173381401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eda92fa2f0 msgr2=0x55eda92afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:03.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f7651d3c640 1 --2- 172.21.15.83:0/2173381401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eda92fa2f0 0x55eda92afb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f7651d3c640 1 -- 172.21.15.83:0/2173381401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eda92930d0 con 0x55eda92f99d0 2024-04-17T10:48:03.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f765253d640 1 --2- 172.21.15.83:0/2173381401 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eda92b00a0 0x55eda92b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:03.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f7651d3c640 1 --2- 172.21.15.83:0/2173381401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eda92f99d0 0x55eda92f9db0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f7648082940 tx=0x7f7648054790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fbd4ade0a08e89f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:03.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f762f7fe640 1 -- 172.21.15.83:0/2173381401 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7648001090 con 0x55eda92f99d0 2024-04-17T10:48:03.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f762f7fe640 1 -- 172.21.15.83:0/2173381401 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7648053000 con 0x55eda92f99d0 2024-04-17T10:48:03.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f762f7fe640 1 -- 172.21.15.83:0/2173381401 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7648051370 con 0x55eda92f99d0 2024-04-17T10:48:03.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f765710d000 1 -- 172.21.15.83:0/2173381401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eda92f99d0 msgr2=0x55eda92f9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:03.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f765710d000 1 --2- 172.21.15.83:0/2173381401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eda92f99d0 0x55eda92f9db0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f7648082940 tx=0x7f7648054790 comp rx=0 tx=0).stop 2024-04-17T10:48:03.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f765710d000 1 -- 172.21.15.83:0/2173381401 shutdown_connections 2024-04-17T10:48:03.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f765710d000 1 --2- 172.21.15.83:0/2173381401 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eda92b00a0 0x55eda92b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f765710d000 1 --2- 172.21.15.83:0/2173381401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eda92fa2f0 0x55eda92afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f765710d000 1 --2- 172.21.15.83:0/2173381401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eda92f99d0 0x55eda92f9db0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f765710d000 1 -- 172.21.15.83:0/2173381401 >> 172.21.15.83:0/2173381401 conn(0x55eda92ac020 msgr2=0x55eda92ae440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:03.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f765710d000 1 -- 172.21.15.83:0/2173381401 shutdown_connections 2024-04-17T10:48:03.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f765710d000 1 -- 172.21.15.83:0/2173381401 wait complete. 2024-04-17T10:48:03.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f765710d000 1 -- start start 2024-04-17T10:48:03.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.422+0000 7f765710d000 1 Processor -- start 2024-04-17T10:48:03.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765710d000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eda92b00a0 0x55eda92de7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765710d000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eda92f99d0 0x55eda92ded30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765710d000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eda92fa2f0 0x55eda92df270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765710d000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eda90c1250 con 0x55eda92b00a0 2024-04-17T10:48:03.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765710d000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eda90b7690 con 0x55eda92f99d0 2024-04-17T10:48:03.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765710d000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eda9085a40 con 0x55eda92fa2f0 2024-04-17T10:48:03.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765710d000 10 client.0 _ll_register_callbacks cb 0x55eda929ae90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:48:03.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f7651d3c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eda92b00a0 0x55eda92de7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f7651d3c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eda92b00a0 0x55eda92de7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46386/0 (socket says 172.21.15.83:46386) 2024-04-17T10:48:03.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f7651d3c640 1 -- 172.21.15.83:0/1129764487 learned_addr learned my addr 172.21.15.83:0/1129764487 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:03.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765253d640 1 --2- 172.21.15.83:0/1129764487 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eda92fa2f0 0x55eda92df270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765153b640 1 --2- 172.21.15.83:0/1129764487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eda92f99d0 0x55eda92ded30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f7651d3c640 1 -- 172.21.15.83:0/1129764487 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eda92fa2f0 msgr2=0x55eda92df270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:03.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f7651d3c640 1 --2- 172.21.15.83:0/1129764487 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eda92fa2f0 0x55eda92df270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f7651d3c640 1 -- 172.21.15.83:0/1129764487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eda92f99d0 msgr2=0x55eda92ded30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:03.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f7651d3c640 1 --2- 172.21.15.83:0/1129764487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eda92f99d0 0x55eda92ded30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f7651d3c640 1 -- 172.21.15.83:0/1129764487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eda91aa1c0 con 0x55eda92b00a0 2024-04-17T10:48:03.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765253d640 1 --2- 172.21.15.83:0/1129764487 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eda92fa2f0 0x55eda92df270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:03.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f7651d3c640 1 --2- 172.21.15.83:0/1129764487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eda92b00a0 0x55eda92de7f0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f76480828f0 tx=0x7f7648058000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61df354582c9a33a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:03.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f762cff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:48:03.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f762cff9640 1 -- 172.21.15.83:0/1129764487 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764805cf80 con 0x55eda92b00a0 2024-04-17T10:48:03.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f762cff9640 1 -- 172.21.15.83:0/1129764487 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f764805d120 con 0x55eda92b00a0 2024-04-17T10:48:03.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765710d000 1 -- 172.21.15.83:0/1129764487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7648053000 con 0x55eda92b00a0 2024-04-17T10:48:03.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f762cff9640 1 -- 172.21.15.83:0/1129764487 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7648054150 con 0x55eda92b00a0 2024-04-17T10:48:03.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765710d000 10 client.8340 fetch_fsmap learned FSMap version 62 2024-04-17T10:48:03.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765710d000 1 -- 172.21.15.83:0/1129764487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55eda91a6170 con 0x55eda92b00a0 2024-04-17T10:48:03.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f762cff9640 1 -- 172.21.15.83:0/1129764487 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7648053000 con 0x55eda92b00a0 2024-04-17T10:48:03.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f762cff9640 1 -- 172.21.15.83:0/1129764487 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f76480542f0 con 0x55eda92b00a0 2024-04-17T10:48:03.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765710d000 10 client.8340 fetch_fsmap finished waiting for FSMap version 62 2024-04-17T10:48:03.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765710d000 10 client.8340 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-17T10:48:03.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765710d000 20 client.8340 populate_metadata read hostname 'smithi083' 2024-04-17T10:48:03.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765710d000 1 --2- 172.21.15.83:0/1129764487 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55eda92dff50 0x55eda930bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765710d000 4 client.8340 mds_command: new command op to 8049 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:48:03.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765710d000 1 -- 172.21.15.83:0/1129764487 --> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eda9098c60 con 0x55eda92dff50 2024-04-17T10:48:03.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765153b640 1 --2- 172.21.15.83:0/1129764487 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55eda92dff50 0x55eda930bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765153b640 1 --2- 172.21.15.83:0/1129764487 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55eda92dff50 0x55eda930bc90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b820b183f44ca637 server_cookie=e07d85a71fb0cfae in_seq=0 out_seq=0 2024-04-17T10:48:03.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f762cff9640 10 client.8340 ms_handle_connect on v2:172.21.15.83:6836/2353406669 2024-04-17T10:48:03.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f762cff9640 1 -- 172.21.15.83:0/1129764487 <== mds.0 v2:172.21.15.83:6836/2353406669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eda9098c60 con 0x55eda92dff50 2024-04-17T10:48:03.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f762cff9640 10 client.8340 handle_command_reply: tid=0 2024-04-17T10:48:03.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765710d000 1 client.8340 shutdown 2024-04-17T10:48:03.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765710d000 1 -- 172.21.15.83:0/1129764487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eda92b00a0 msgr2=0x55eda92de7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:03.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765710d000 1 --2- 172.21.15.83:0/1129764487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eda92b00a0 0x55eda92de7f0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f76480828f0 tx=0x7f7648058000 comp rx=0 tx=0).stop 2024-04-17T10:48:03.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765710d000 1 -- 172.21.15.83:0/1129764487 shutdown_connections 2024-04-17T10:48:03.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765710d000 1 --2- 172.21.15.83:0/1129764487 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55eda92dff50 0x55eda930bc90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765710d000 1 --2- 172.21.15.83:0/1129764487 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eda92fa2f0 0x55eda92df270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765710d000 1 --2- 172.21.15.83:0/1129764487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eda92f99d0 0x55eda92ded30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765710d000 1 --2- 172.21.15.83:0/1129764487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eda92b00a0 0x55eda92de7f0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f762cff9640 0 client.8340 ms_handle_reset on v2:172.21.15.83:6836/2353406669 2024-04-17T10:48:03.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765710d000 1 -- 172.21.15.83:0/1129764487 >> 172.21.15.83:0/1129764487 conn(0x55eda92ac020 msgr2=0x55eda92aeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:03.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765710d000 1 -- 172.21.15.83:0/1129764487 shutdown_connections 2024-04-17T10:48:03.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765710d000 1 -- 172.21.15.83:0/1129764487 wait complete. 2024-04-17T10:48:03.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.426+0000 7f765710d000 20 client.8340 trim_cache size 0 max 16384 2024-04-17T10:48:03.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.442+0000 7f765710d000 1 Processor -- start 2024-04-17T10:48:03.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.442+0000 7f765710d000 1 -- start start 2024-04-17T10:48:03.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.442+0000 7f765710d000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eda92b00a0 0x55eda9355f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.442+0000 7f765710d000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eda92f99d0 0x55eda9356460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.442+0000 7f765710d000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eda92fa2f0 0x55eda93569a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.442+0000 7f765710d000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c2006d0 con 0x55eda92b00a0 2024-04-17T10:48:03.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.442+0000 7f765710d000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c14b3c0 con 0x55eda92fa2f0 2024-04-17T10:48:03.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.442+0000 7f765710d000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764c152720 con 0x55eda92f99d0 2024-04-17T10:48:03.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.442+0000 7f7651d3c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eda92b00a0 0x55eda9355f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.442+0000 7f7651d3c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eda92b00a0 0x55eda9355f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46402/0 (socket says 172.21.15.83:46402) 2024-04-17T10:48:03.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.442+0000 7f7651d3c640 1 -- 172.21.15.83:0/2872676412 learned_addr learned my addr 172.21.15.83:0/2872676412 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:03.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.442+0000 7f7651d3c640 1 -- 172.21.15.83:0/2872676412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eda92f99d0 msgr2=0x55eda9356460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:03.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.442+0000 7f7651d3c640 1 --2- 172.21.15.83:0/2872676412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eda92f99d0 0x55eda9356460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.442+0000 7f7651d3c640 1 -- 172.21.15.83:0/2872676412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eda92fa2f0 msgr2=0x55eda93569a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:03.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.442+0000 7f7651d3c640 1 --2- 172.21.15.83:0/2872676412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eda92fa2f0 0x55eda93569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.442+0000 7f7651d3c640 1 -- 172.21.15.83:0/2872676412 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eda9143de0 con 0x55eda92b00a0 2024-04-17T10:48:03.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.442+0000 7f7651d3c640 1 --2- 172.21.15.83:0/2872676412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eda92b00a0 0x55eda9355f20 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f7648050e40 tx=0x7f7648054c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3edf01a5ca4f2fc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:03.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.442+0000 7f762effd640 1 -- 172.21.15.83:0/2872676412 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7648082970 con 0x55eda92b00a0 2024-04-17T10:48:03.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.442+0000 7f762effd640 1 -- 172.21.15.83:0/2872676412 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7648053000 con 0x55eda92b00a0 2024-04-17T10:48:03.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.442+0000 7f762effd640 1 -- 172.21.15.83:0/2872676412 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764805cf80 con 0x55eda92b00a0 2024-04-17T10:48:03.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.442+0000 7f765710d000 1 -- 172.21.15.83:0/2872676412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eda92b00a0 msgr2=0x55eda9355f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:03.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.442+0000 7f765710d000 1 --2- 172.21.15.83:0/2872676412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eda92b00a0 0x55eda9355f20 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f7648050e40 tx=0x7f7648054c00 comp rx=0 tx=0).stop 2024-04-17T10:48:03.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.442+0000 7f765710d000 1 -- 172.21.15.83:0/2872676412 shutdown_connections 2024-04-17T10:48:03.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.442+0000 7f765710d000 1 --2- 172.21.15.83:0/2872676412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eda92fa2f0 0x55eda93569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.442+0000 7f765710d000 1 --2- 172.21.15.83:0/2872676412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eda92f99d0 0x55eda9356460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.442+0000 7f765710d000 1 --2- 172.21.15.83:0/2872676412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eda92b00a0 0x55eda9355f20 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.442+0000 7f765710d000 1 -- 172.21.15.83:0/2872676412 >> 172.21.15.83:0/2872676412 conn(0x55eda92ac020 msgr2=0x55eda9306fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:03.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.442+0000 7f765710d000 1 -- 172.21.15.83:0/2872676412 shutdown_connections 2024-04-17T10:48:03.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.442+0000 7f765710d000 1 -- 172.21.15.83:0/2872676412 wait complete. 2024-04-17T10:48:03.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.442+0000 7f765710d000 1 -- start start 2024-04-17T10:48:03.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.442+0000 7f765710d000 1 Processor -- start 2024-04-17T10:48:03.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765710d000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eda92b00a0 0x55eda930c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765710d000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eda92f99d0 0x55eda930c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765710d000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eda92fa2f0 0x55eda930ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765710d000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eda9097e00 con 0x55eda92f99d0 2024-04-17T10:48:03.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765710d000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c14adf0 con 0x55eda92fa2f0 2024-04-17T10:48:03.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765710d000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eda90fc300 con 0x55eda92b00a0 2024-04-17T10:48:03.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765710d000 10 client.0 _ll_register_callbacks cb 0x55eda927b040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:48:03.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765153b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eda92f99d0 0x55eda930c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765153b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eda92f99d0 0x55eda930c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46410/0 (socket says 172.21.15.83:46410) 2024-04-17T10:48:03.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765153b640 1 -- 172.21.15.83:0/1585282193 learned_addr learned my addr 172.21.15.83:0/1585282193 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:03.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f7651d3c640 1 --2- 172.21.15.83:0/1585282193 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eda92b00a0 0x55eda930c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765153b640 1 -- 172.21.15.83:0/1585282193 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eda92b00a0 msgr2=0x55eda930c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:03.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765153b640 1 --2- 172.21.15.83:0/1585282193 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eda92b00a0 0x55eda930c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765253d640 1 --2- 172.21.15.83:0/1585282193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eda92fa2f0 0x55eda930ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765153b640 1 -- 172.21.15.83:0/1585282193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eda92fa2f0 msgr2=0x55eda930ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:03.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765153b640 1 --2- 172.21.15.83:0/1585282193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eda92fa2f0 0x55eda930ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765153b640 1 -- 172.21.15.83:0/1585282193 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eda909e8e0 con 0x55eda92f99d0 2024-04-17T10:48:03.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765153b640 1 --2- 172.21.15.83:0/1585282193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eda92f99d0 0x55eda930c8d0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f763c000c10 tx=0x7f763c019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2555d0efb4efb93 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:03.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f762cff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:48:03.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f762cff9640 1 -- 172.21.15.83:0/1585282193 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c0b0040 con 0x55eda92f99d0 2024-04-17T10:48:03.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f762cff9640 1 -- 172.21.15.83:0/1585282193 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f763c0abb80 con 0x55eda92f99d0 2024-04-17T10:48:03.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765710d000 1 -- 172.21.15.83:0/1585282193 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7648053000 con 0x55eda92f99d0 2024-04-17T10:48:03.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f762cff9640 1 -- 172.21.15.83:0/1585282193 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f763c0abd80 con 0x55eda92f99d0 2024-04-17T10:48:03.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765710d000 10 client.8346 fetch_fsmap learned FSMap version 62 2024-04-17T10:48:03.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765710d000 1 -- 172.21.15.83:0/1585282193 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55eda91568f0 con 0x55eda92f99d0 2024-04-17T10:48:03.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f762cff9640 1 -- 172.21.15.83:0/1585282193 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7648053000 con 0x55eda92f99d0 2024-04-17T10:48:03.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f762cff9640 1 -- 172.21.15.83:0/1585282193 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f763c0c0040 con 0x55eda92f99d0 2024-04-17T10:48:03.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765710d000 10 client.8346 fetch_fsmap finished waiting for FSMap version 62 2024-04-17T10:48:03.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765710d000 10 client.8346 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-17T10:48:03.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765710d000 20 client.8346 populate_metadata read hostname 'smithi083' 2024-04-17T10:48:03.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765710d000 1 --2- 172.21.15.83:0/1585282193 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55eda935fb70 0x55eda9361f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765710d000 4 client.8346 mds_command: new command op to 8049 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-17T10:48:03.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765710d000 1 -- 172.21.15.83:0/1585282193 --> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x55eda910a530 con 0x55eda935fb70 2024-04-17T10:48:03.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f7651d3c640 1 --2- 172.21.15.83:0/1585282193 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55eda935fb70 0x55eda9361f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f762cff9640 10 client.8346 ms_handle_connect on v2:172.21.15.83:6836/2353406669 2024-04-17T10:48:03.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f7651d3c640 1 --2- 172.21.15.83:0/1585282193 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55eda935fb70 0x55eda9361f60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e0713844d09d8d9 server_cookie=c825fc0f4af9e720 in_seq=0 out_seq=0 2024-04-17T10:48:03.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f762cff9640 1 -- 172.21.15.83:0/1585282193 <== mds.0 v2:172.21.15.83:6836/2353406669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55eda910a530 con 0x55eda935fb70 2024-04-17T10:48:03.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f762cff9640 10 client.8346 handle_command_reply: tid=0 2024-04-17T10:48:03.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765710d000 1 client.8346 shutdown 2024-04-17T10:48:03.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765710d000 1 -- 172.21.15.83:0/1585282193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eda92f99d0 msgr2=0x55eda930c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:03.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765710d000 1 --2- 172.21.15.83:0/1585282193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eda92f99d0 0x55eda930c8d0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f763c000c10 tx=0x7f763c019210 comp rx=0 tx=0).stop 2024-04-17T10:48:03.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765710d000 1 -- 172.21.15.83:0/1585282193 shutdown_connections 2024-04-17T10:48:03.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765710d000 1 --2- 172.21.15.83:0/1585282193 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55eda935fb70 0x55eda9361f60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765710d000 1 --2- 172.21.15.83:0/1585282193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eda92fa2f0 0x55eda930ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765710d000 1 --2- 172.21.15.83:0/1585282193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eda92f99d0 0x55eda930c8d0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f762cff9640 0 client.8346 ms_handle_reset on v2:172.21.15.83:6836/2353406669 2024-04-17T10:48:03.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765710d000 1 --2- 172.21.15.83:0/1585282193 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eda92b00a0 0x55eda930c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765710d000 1 -- 172.21.15.83:0/1585282193 >> 172.21.15.83:0/1585282193 conn(0x55eda92ac020 msgr2=0x55eda92f83f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:03.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765710d000 1 -- 172.21.15.83:0/1585282193 shutdown_connections 2024-04-17T10:48:03.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765710d000 1 -- 172.21.15.83:0/1585282193 wait complete. 2024-04-17T10:48:03.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f765710d000 20 client.8346 trim_cache size 0 max 16384 2024-04-17T10:48:03.451 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:48:03.451 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0 2024-04-17T10:48:03.451 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:48:03.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f7653fa0640 1 -- 172.21.15.83:0/3684778374 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7630066f60 msgr2=0x7f7630069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:03.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f7653fa0640 1 --2- 172.21.15.83:0/3684778374 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7630066f60 0x7f7630069420 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f763c0ad000 tx=0x7f763c008040 comp rx=0 tx=0).stop 2024-04-17T10:48:03.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f7653fa0640 1 -- 172.21.15.83:0/3684778374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f764c0b4090 msgr2=0x7f764c153b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:03.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f7653fa0640 1 --2- 172.21.15.83:0/3684778374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f764c0b4090 0x7f764c153b40 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f764800c480 tx=0x7f764800c940 comp rx=0 tx=0).stop 2024-04-17T10:48:03.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f7653fa0640 1 -- 172.21.15.83:0/3684778374 shutdown_connections 2024-04-17T10:48:03.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f7653fa0640 1 --2- 172.21.15.83:0/3684778374 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7630066f60 0x7f7630069420 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f7653fa0640 1 --2- 172.21.15.83:0/3684778374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f764c154670 0x7f764c070c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f7653fa0640 1 --2- 172.21.15.83:0/3684778374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f764c0b4520 0x7f764c14f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f7653fa0640 1 --2- 172.21.15.83:0/3684778374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f764c0b4090 0x7f764c153b40 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f7653fa0640 1 -- 172.21.15.83:0/3684778374 >> 172.21.15.83:0/3684778374 conn(0x7f764c0b15b0 msgr2=0x7f764c0b0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:03.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f7653fa0640 1 -- 172.21.15.83:0/3684778374 shutdown_connections 2024-04-17T10:48:03.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.446+0000 7f7653fa0640 1 -- 172.21.15.83:0/3684778374 wait complete. 2024-04-17T10:48:03.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:03.459 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-17T10:48:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.566+0000 7f6ccf96c640 1 Processor -- start 2024-04-17T10:48:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.566+0000 7f6ccf96c640 1 -- start start 2024-04-17T10:48:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.566+0000 7f6ccf96c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6cc815e0f0 0x7f6cc80b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.566+0000 7f6ccf96c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc80ce400 0x7f6cc8161c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.566+0000 7f6ccf96c640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cc815e4d0 con 0x7f6cc80b5450 2024-04-17T10:48:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.566+0000 7f6ccf96c640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cc8162b40 con 0x7f6cc815e0f0 2024-04-17T10:48:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.566+0000 7f6ccf96c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc8162580 con 0x7f6cc80ce400 2024-04-17T10:48:03.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.570+0000 7f6ccd708640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6cc815e0f0 0x7f6cc80b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.570+0000 7f6cccf07640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc80ce400 0x7f6cc8161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.570+0000 7f6ccdf09640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f6cc80b5450 0x7f6cc80d2540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:42328/0 (socket says 172.21.15.83:42328) 2024-04-17T10:48:03.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.570+0000 7f6ccdf09640 1 -- 172.21.15.83:0/505171719 learned_addr learned my addr 172.21.15.83:0/505171719 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:03.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.570+0000 7f6cccf07640 1 -- 172.21.15.83:0/505171719 >> v2:172.21.15.167:3300/0 conn(0x7f6cc815e0f0 msgr2=0x7f6cc80b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:03.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.570+0000 7f6cccf07640 1 --2- 172.21.15.83:0/505171719 >> v2:172.21.15.167:3300/0 conn(0x7f6cc815e0f0 0x7f6cc80b6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.570+0000 7f6cccf07640 1 -- 172.21.15.83:0/505171719 >> v1:172.21.15.167:6789/0 conn(0x7f6cc80b5450 legacy=0x7f6cc80d2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:03.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.570+0000 7f6cccf07640 1 -- 172.21.15.83:0/505171719 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc8162860 con 0x7f6cc80ce400 2024-04-17T10:48:03.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.570+0000 7f6ccd708640 1 --2- 172.21.15.83:0/505171719 >> v2:172.21.15.167:3300/0 conn(0x7f6cc815e0f0 0x7f6cc80b6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:48:03.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.570+0000 7f6cccf07640 1 --2- 172.21.15.83:0/505171719 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc80ce400 0x7f6cc8161c10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80097c0 tx=0x7f6cb801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3181e3fdc7379781 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:03.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.570+0000 7f6cbffff640 1 -- 172.21.15.83:0/505171719 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb80b0070 con 0x7f6cc80ce400 2024-04-17T10:48:03.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.570+0000 7f6cbffff640 1 -- 172.21.15.83:0/505171719 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb80ab600 con 0x7f6cc80ce400 2024-04-17T10:48:03.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.570+0000 7f6cbffff640 1 -- 172.21.15.83:0/505171719 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb80ab9d0 con 0x7f6cc80ce400 2024-04-17T10:48:03.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.570+0000 7f6ccf96c640 1 -- 172.21.15.83:0/505171719 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc80ce400 msgr2=0x7f6cc8161c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:03.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.570+0000 7f6ccf96c640 1 --2- 172.21.15.83:0/505171719 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc80ce400 0x7f6cc8161c10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80097c0 tx=0x7f6cb801c410 comp rx=0 tx=0).stop 2024-04-17T10:48:03.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.570+0000 7f6ccf96c640 1 -- 172.21.15.83:0/505171719 shutdown_connections 2024-04-17T10:48:03.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.570+0000 7f6ccf96c640 1 --2- 172.21.15.83:0/505171719 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc80ce400 0x7f6cc8161c10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.570+0000 7f6ccf96c640 1 --2- 172.21.15.83:0/505171719 >> v2:172.21.15.167:3300/0 conn(0x7f6cc815e0f0 0x7f6cc80b6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.570+0000 7f6ccf96c640 1 -- 172.21.15.83:0/505171719 >> 172.21.15.83:0/505171719 conn(0x7f6cc80d0450 msgr2=0x7f6cc80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:03.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.570+0000 7f6ccf96c640 1 -- 172.21.15.83:0/505171719 shutdown_connections 2024-04-17T10:48:03.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.570+0000 7f6ccf96c640 1 -- 172.21.15.83:0/505171719 wait complete. 2024-04-17T10:48:03.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.574+0000 7f6ccf96c640 1 Processor -- start 2024-04-17T10:48:03.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.574+0000 7f6ccf96c640 1 -- start start 2024-04-17T10:48:03.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.574+0000 7f6ccf96c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cc80b5450 0x7f6cc815ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.574+0000 7f6ccf96c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc80ce400 0x7f6cc815b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.574+0000 7f6ccf96c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc815e0f0 0x7f6cc80c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.574+0000 7f6ccf96c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc81622d0 con 0x7f6cc815e0f0 2024-04-17T10:48:03.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.574+0000 7f6ccf96c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc8162150 con 0x7f6cc80b5450 2024-04-17T10:48:03.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.574+0000 7f6ccf96c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc816af50 con 0x7f6cc80ce400 2024-04-17T10:48:03.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.574+0000 7f6ccdf09640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc815e0f0 0x7f6cc80c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.574+0000 7f6ccdf09640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc815e0f0 0x7f6cc80c8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46416/0 (socket says 172.21.15.83:46416) 2024-04-17T10:48:03.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.574+0000 7f6ccdf09640 1 -- 172.21.15.83:0/182218635 learned_addr learned my addr 172.21.15.83:0/182218635 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:03.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.574+0000 7f6ccd708640 1 --2- 172.21.15.83:0/182218635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cc80b5450 0x7f6cc815ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.574+0000 7f6cccf07640 1 --2- 172.21.15.83:0/182218635 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc80ce400 0x7f6cc815b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.574+0000 7f6ccdf09640 1 -- 172.21.15.83:0/182218635 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc80ce400 msgr2=0x7f6cc815b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:03.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.574+0000 7f6ccdf09640 1 --2- 172.21.15.83:0/182218635 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc80ce400 0x7f6cc815b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.574+0000 7f6ccdf09640 1 -- 172.21.15.83:0/182218635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cc80b5450 msgr2=0x7f6cc815ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:03.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.574+0000 7f6ccdf09640 1 --2- 172.21.15.83:0/182218635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cc80b5450 0x7f6cc815ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.574+0000 7f6ccdf09640 1 -- 172.21.15.83:0/182218635 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc80c9460 con 0x7f6cc815e0f0 2024-04-17T10:48:03.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.574+0000 7f6ccd708640 1 --2- 172.21.15.83:0/182218635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cc80b5450 0x7f6cc815ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:48:03.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.574+0000 7f6ccdf09640 1 --2- 172.21.15.83:0/182218635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc815e0f0 0x7f6cc80c8810 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000ca80 tx=0x7f6cc000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:03.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.574+0000 7f6cbdffb640 1 -- 172.21.15.83:0/182218635 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0017070 con 0x7f6cc815e0f0 2024-04-17T10:48:03.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.574+0000 7f6cbdffb640 1 -- 172.21.15.83:0/182218635 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc000ac30 con 0x7f6cc815e0f0 2024-04-17T10:48:03.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.574+0000 7f6ccf96c640 1 -- 172.21.15.83:0/182218635 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cc80c9740 con 0x7f6cc815e0f0 2024-04-17T10:48:03.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.574+0000 7f6cbdffb640 1 -- 172.21.15.83:0/182218635 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0012600 con 0x7f6cc815e0f0 2024-04-17T10:48:03.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.574+0000 7f6ccf96c640 1 -- 172.21.15.83:0/182218635 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc8159740 con 0x7f6cc815e0f0 2024-04-17T10:48:03.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6cbdffb640 1 -- 172.21.15.83:0/182218635 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6cc001a070 con 0x7f6cc815e0f0 2024-04-17T10:48:03.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6cbdffb640 1 --2- 172.21.15.83:0/182218635 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6cac066f60 0x7f6cac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6cbdffb640 1 -- 172.21.15.83:0/182218635 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6cc0084c10 con 0x7f6cc815e0f0 2024-04-17T10:48:03.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6ccd708640 1 --2- 172.21.15.83:0/182218635 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6cac066f60 0x7f6cac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6ccd708640 1 --2- 172.21.15.83:0/182218635 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6cac066f60 0x7f6cac069420 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009bc0 tx=0x7f6cc4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:03.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6cd2ad9000 1 Processor -- start 2024-04-17T10:48:03.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6cd2ad9000 1 -- start start 2024-04-17T10:48:03.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6cd2ad9000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd693929d0 0x55bd69392db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6cd2ad9000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55bd693932f0 0x55bd69348b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6cd2ad9000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd693490a0 0x55bd69349480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6cd2ad9000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd69195ff0 con 0x55bd693490a0 2024-04-17T10:48:03.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6cd2ad9000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd6911e150 con 0x55bd693929d0 2024-04-17T10:48:03.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6cd2ad9000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd69192090 con 0x55bd693932f0 2024-04-17T10:48:03.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6ccdf09640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd693490a0 0x55bd69349480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6ccdf09640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd693490a0 0x55bd69349480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46422/0 (socket says 172.21.15.83:46422) 2024-04-17T10:48:03.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6ccdf09640 1 -- 172.21.15.83:0/3617498396 learned_addr learned my addr 172.21.15.83:0/3617498396 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:03.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6cccf07640 1 --2- 172.21.15.83:0/3617498396 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55bd693932f0 0x55bd69348b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6ccdf09640 1 -- 172.21.15.83:0/3617498396 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55bd693932f0 msgr2=0x55bd69348b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:03.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6ccdf09640 1 --2- 172.21.15.83:0/3617498396 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55bd693932f0 0x55bd69348b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6ccdf09640 1 -- 172.21.15.83:0/3617498396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd693929d0 msgr2=0x55bd69392db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:03.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6ccd708640 1 --2- 172.21.15.83:0/3617498396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd693929d0 0x55bd69392db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6ccdf09640 1 --2- 172.21.15.83:0/3617498396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd693929d0 0x55bd69392db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6ccdf09640 1 -- 172.21.15.83:0/3617498396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd6932c0d0 con 0x55bd693490a0 2024-04-17T10:48:03.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6ccdf09640 1 --2- 172.21.15.83:0/3617498396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd693490a0 0x55bd69349480 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0083160 tx=0x7f6cc0055690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563596fdd7c2fe73 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:03.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6cb2ffd640 1 -- 172.21.15.83:0/3617498396 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc00538d0 con 0x55bd693490a0 2024-04-17T10:48:03.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6cb2ffd640 1 -- 172.21.15.83:0/3617498396 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc0051c40 con 0x55bd693490a0 2024-04-17T10:48:03.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6cb2ffd640 1 -- 172.21.15.83:0/3617498396 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc005d9c0 con 0x55bd693490a0 2024-04-17T10:48:03.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6cd2ad9000 1 -- 172.21.15.83:0/3617498396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd693490a0 msgr2=0x55bd69349480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:03.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6cd2ad9000 1 --2- 172.21.15.83:0/3617498396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd693490a0 0x55bd69349480 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0083160 tx=0x7f6cc0055690 comp rx=0 tx=0).stop 2024-04-17T10:48:03.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6cd2ad9000 1 -- 172.21.15.83:0/3617498396 shutdown_connections 2024-04-17T10:48:03.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6cd2ad9000 1 --2- 172.21.15.83:0/3617498396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd693490a0 0x55bd69349480 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6cd2ad9000 1 --2- 172.21.15.83:0/3617498396 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55bd693932f0 0x55bd69348b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6cd2ad9000 1 --2- 172.21.15.83:0/3617498396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd693929d0 0x55bd69392db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6cd2ad9000 1 -- 172.21.15.83:0/3617498396 >> 172.21.15.83:0/3617498396 conn(0x55bd69345020 msgr2=0x55bd69347440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:03.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6cd2ad9000 1 -- 172.21.15.83:0/3617498396 shutdown_connections 2024-04-17T10:48:03.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6cd2ad9000 1 -- 172.21.15.83:0/3617498396 wait complete. 2024-04-17T10:48:03.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.578+0000 7f6cd2ad9000 1 -- start start 2024-04-17T10:48:03.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6cd2ad9000 1 Processor -- start 2024-04-17T10:48:03.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6cd2ad9000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd693490a0 0x55bd693777f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6cd2ad9000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd693929d0 0x55bd69377d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6cd2ad9000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55bd693932f0 0x55bd69378270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6cd2ad9000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd6915a250 con 0x55bd693490a0 2024-04-17T10:48:03.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6cd2ad9000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd69150690 con 0x55bd693929d0 2024-04-17T10:48:03.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6cd2ad9000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd6911ea40 con 0x55bd693932f0 2024-04-17T10:48:03.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6cd2ad9000 10 client.0 _ll_register_callbacks cb 0x55bd69333e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:48:03.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6ccd708640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd693490a0 0x55bd693777f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6ccd708640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd693490a0 0x55bd693777f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46424/0 (socket says 172.21.15.83:46424) 2024-04-17T10:48:03.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6ccd708640 1 -- 172.21.15.83:0/1536559055 learned_addr learned my addr 172.21.15.83:0/1536559055 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:03.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6ccdf09640 1 --2- 172.21.15.83:0/1536559055 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55bd693932f0 0x55bd69378270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6ccd708640 1 -- 172.21.15.83:0/1536559055 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55bd693932f0 msgr2=0x55bd69378270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:03.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6ccd708640 1 --2- 172.21.15.83:0/1536559055 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55bd693932f0 0x55bd69378270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6cccf07640 1 --2- 172.21.15.83:0/1536559055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd693929d0 0x55bd69377d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6ccd708640 1 -- 172.21.15.83:0/1536559055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd693929d0 msgr2=0x55bd69377d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:03.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6ccd708640 1 --2- 172.21.15.83:0/1536559055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd693929d0 0x55bd69377d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6ccd708640 1 -- 172.21.15.83:0/1536559055 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd692431c0 con 0x55bd693490a0 2024-04-17T10:48:03.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6ccd708640 1 --2- 172.21.15.83:0/1536559055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd693490a0 0x55bd693777f0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f6cc40156c0 tx=0x7f6cc40046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1200e1afa7e559f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:03.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6ca7fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:48:03.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6ca7fff640 1 -- 172.21.15.83:0/1536559055 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4025070 con 0x55bd693490a0 2024-04-17T10:48:03.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6cd2ad9000 1 -- 172.21.15.83:0/1536559055 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6cc0051c40 con 0x55bd693490a0 2024-04-17T10:48:03.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6ca7fff640 1 -- 172.21.15.83:0/1536559055 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6cc40147b0 con 0x55bd693490a0 2024-04-17T10:48:03.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6ca7fff640 1 -- 172.21.15.83:0/1536559055 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cc4030050 con 0x55bd693490a0 2024-04-17T10:48:03.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6cd2ad9000 10 client.8355 fetch_fsmap learned FSMap version 62 2024-04-17T10:48:03.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6cd2ad9000 1 -- 172.21.15.83:0/1536559055 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55bd6923f170 con 0x55bd693490a0 2024-04-17T10:48:03.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6ca7fff640 1 -- 172.21.15.83:0/1536559055 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc0051c40 con 0x55bd693490a0 2024-04-17T10:48:03.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6ca7fff640 1 -- 172.21.15.83:0/1536559055 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6cc4033020 con 0x55bd693490a0 2024-04-17T10:48:03.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6cd2ad9000 10 client.8355 fetch_fsmap finished waiting for FSMap version 62 2024-04-17T10:48:03.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6cd2ad9000 10 client.8355 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-17T10:48:03.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6cd2ad9000 20 client.8355 populate_metadata read hostname 'smithi083' 2024-04-17T10:48:03.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6cd2ad9000 1 --2- 172.21.15.83:0/1536559055 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55bd693a2c80 0x55bd693a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6cd2ad9000 4 client.8355 mds_command: new command op to 8049 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:48:03.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6cd2ad9000 1 -- 172.21.15.83:0/1536559055 --> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bd69131c60 con 0x55bd693a2c80 2024-04-17T10:48:03.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6cccf07640 1 --2- 172.21.15.83:0/1536559055 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55bd693a2c80 0x55bd693a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6cccf07640 1 --2- 172.21.15.83:0/1536559055 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55bd693a2c80 0x55bd693a5070 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cfb0cb118124841 server_cookie=b717ce939b99fa22 in_seq=0 out_seq=0 2024-04-17T10:48:03.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6ca7fff640 10 client.8355 ms_handle_connect on v2:172.21.15.83:6836/2353406669 2024-04-17T10:48:03.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6ca7fff640 1 -- 172.21.15.83:0/1536559055 <== mds.0 v2:172.21.15.83:6836/2353406669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bd69131c60 con 0x55bd693a2c80 2024-04-17T10:48:03.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6ca7fff640 10 client.8355 handle_command_reply: tid=0 2024-04-17T10:48:03.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6cd2ad9000 1 client.8355 shutdown 2024-04-17T10:48:03.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6cd2ad9000 1 -- 172.21.15.83:0/1536559055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd693490a0 msgr2=0x55bd693777f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:03.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6cd2ad9000 1 --2- 172.21.15.83:0/1536559055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd693490a0 0x55bd693777f0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f6cc40156c0 tx=0x7f6cc40046d0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6cd2ad9000 1 -- 172.21.15.83:0/1536559055 shutdown_connections 2024-04-17T10:48:03.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6cd2ad9000 1 --2- 172.21.15.83:0/1536559055 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55bd693a2c80 0x55bd693a5070 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6cd2ad9000 1 --2- 172.21.15.83:0/1536559055 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55bd693932f0 0x55bd69378270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6cd2ad9000 1 --2- 172.21.15.83:0/1536559055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd693929d0 0x55bd69377d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6cd2ad9000 1 --2- 172.21.15.83:0/1536559055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd693490a0 0x55bd693777f0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6ca7fff640 0 client.8355 ms_handle_reset on v2:172.21.15.83:6836/2353406669 2024-04-17T10:48:03.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6cd2ad9000 1 -- 172.21.15.83:0/1536559055 >> 172.21.15.83:0/1536559055 conn(0x55bd69345020 msgr2=0x55bd69347ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:03.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6cd2ad9000 1 -- 172.21.15.83:0/1536559055 shutdown_connections 2024-04-17T10:48:03.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6cd2ad9000 1 -- 172.21.15.83:0/1536559055 wait complete. 2024-04-17T10:48:03.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.582+0000 7f6cd2ad9000 20 client.8355 trim_cache size 0 max 16384 2024-04-17T10:48:03.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.598+0000 7f6cd2ad9000 1 Processor -- start 2024-04-17T10:48:03.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.598+0000 7f6cd2ad9000 1 -- start start 2024-04-17T10:48:03.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.598+0000 7f6cd2ad9000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55bd693490a0 0x55bd693eeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.598+0000 7f6cd2ad9000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd693929d0 0x55bd693ef420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.598+0000 7f6cd2ad9000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd693932f0 0x55bd693ef960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.598+0000 7f6cd2ad9000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc8203110 con 0x55bd693932f0 2024-04-17T10:48:03.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.598+0000 7f6cd2ad9000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc814dae0 con 0x55bd693929d0 2024-04-17T10:48:03.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.598+0000 7f6cd2ad9000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc814cee0 con 0x55bd693490a0 2024-04-17T10:48:03.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.598+0000 7f6ccdf09640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd693932f0 0x55bd693ef960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.598+0000 7f6ccdf09640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd693932f0 0x55bd693ef960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46426/0 (socket says 172.21.15.83:46426) 2024-04-17T10:48:03.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.598+0000 7f6ccdf09640 1 -- 172.21.15.83:0/2248295496 learned_addr learned my addr 172.21.15.83:0/2248295496 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:03.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.598+0000 7f6ccdf09640 1 -- 172.21.15.83:0/2248295496 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55bd693490a0 msgr2=0x55bd693eeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:03.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.598+0000 7f6ccdf09640 1 --2- 172.21.15.83:0/2248295496 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55bd693490a0 0x55bd693eeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.598+0000 7f6ccdf09640 1 -- 172.21.15.83:0/2248295496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd693929d0 msgr2=0x55bd693ef420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:03.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.598+0000 7f6ccdf09640 1 --2- 172.21.15.83:0/2248295496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd693929d0 0x55bd693ef420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.598+0000 7f6ccdf09640 1 -- 172.21.15.83:0/2248295496 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd691dcde0 con 0x55bd693932f0 2024-04-17T10:48:03.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.598+0000 7f6ccdf09640 1 --2- 172.21.15.83:0/2248295496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd693932f0 0x55bd693ef960 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f6cc00012f0 tx=0x7f6cc001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9544ede76be173b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.598+0000 7f6cb27fc640 1 -- 172.21.15.83:0/2248295496 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0051c40 con 0x55bd693932f0 2024-04-17T10:48:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.598+0000 7f6cb27fc640 1 -- 172.21.15.83:0/2248295496 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc005d7a0 con 0x55bd693932f0 2024-04-17T10:48:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.598+0000 7f6cb27fc640 1 -- 172.21.15.83:0/2248295496 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0051de0 con 0x55bd693932f0 2024-04-17T10:48:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.598+0000 7f6cd2ad9000 1 -- 172.21.15.83:0/2248295496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd693932f0 msgr2=0x55bd693ef960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.598+0000 7f6cd2ad9000 1 --2- 172.21.15.83:0/2248295496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd693932f0 0x55bd693ef960 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f6cc00012f0 tx=0x7f6cc001e000 comp rx=0 tx=0).stop 2024-04-17T10:48:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.598+0000 7f6cd2ad9000 1 -- 172.21.15.83:0/2248295496 shutdown_connections 2024-04-17T10:48:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.598+0000 7f6cd2ad9000 1 --2- 172.21.15.83:0/2248295496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd693932f0 0x55bd693ef960 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.598+0000 7f6cd2ad9000 1 --2- 172.21.15.83:0/2248295496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd693929d0 0x55bd693ef420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.598+0000 7f6cd2ad9000 1 --2- 172.21.15.83:0/2248295496 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55bd693490a0 0x55bd693eeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.598+0000 7f6cd2ad9000 1 -- 172.21.15.83:0/2248295496 >> 172.21.15.83:0/2248295496 conn(0x55bd69345020 msgr2=0x55bd6939ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.598+0000 7f6cd2ad9000 1 -- 172.21.15.83:0/2248295496 shutdown_connections 2024-04-17T10:48:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.598+0000 7f6cd2ad9000 1 -- 172.21.15.83:0/2248295496 wait complete. 2024-04-17T10:48:03.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.598+0000 7f6cd2ad9000 1 -- start start 2024-04-17T10:48:03.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.598+0000 7f6cd2ad9000 1 Processor -- start 2024-04-17T10:48:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6cd2ad9000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd693490a0 0x55bd693f2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6cd2ad9000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55bd693929d0 0x55bd693f3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6cd2ad9000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd693932f0 0x55bd693a4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6cd2ad9000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd69130e00 con 0x55bd693932f0 2024-04-17T10:48:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6cd2ad9000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc814d4e0 con 0x55bd693490a0 2024-04-17T10:48:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6cd2ad9000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd69195300 con 0x55bd693929d0 2024-04-17T10:48:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6cd2ad9000 10 client.0 _ll_register_callbacks cb 0x55bd69314040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:48:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ccdf09640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd693932f0 0x55bd693a4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ccdf09640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd693932f0 0x55bd693a4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46440/0 (socket says 172.21.15.83:46440) 2024-04-17T10:48:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ccdf09640 1 -- 172.21.15.83:0/715248409 learned_addr learned my addr 172.21.15.83:0/715248409 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ccd708640 1 --2- 172.21.15.83:0/715248409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd693490a0 0x55bd693f2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ccdf09640 1 -- 172.21.15.83:0/715248409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55bd693929d0 msgr2=0x55bd693f3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ccdf09640 1 --2- 172.21.15.83:0/715248409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55bd693929d0 0x55bd693f3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ccdf09640 1 -- 172.21.15.83:0/715248409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd693490a0 msgr2=0x55bd693f2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ccdf09640 1 --2- 172.21.15.83:0/715248409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd693490a0 0x55bd693f2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ccdf09640 1 -- 172.21.15.83:0/715248409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd691378e0 con 0x55bd693932f0 2024-04-17T10:48:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ccdf09640 1 --2- 172.21.15.83:0/715248409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd693932f0 0x55bd693a4fc0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f6cc005ddf0 tx=0x7f6cc0001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a774c065263ba97 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ca7fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:48:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ca7fff640 1 -- 172.21.15.83:0/715248409 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0004d70 con 0x55bd693932f0 2024-04-17T10:48:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ca7fff640 1 -- 172.21.15.83:0/715248409 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6cc0059170 con 0x55bd693932f0 2024-04-17T10:48:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6cd2ad9000 1 -- 172.21.15.83:0/715248409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6cc005d7a0 con 0x55bd693932f0 2024-04-17T10:48:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ca7fff640 1 -- 172.21.15.83:0/715248409 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cc0078050 con 0x55bd693932f0 2024-04-17T10:48:03.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6cd2ad9000 10 client.8361 fetch_fsmap learned FSMap version 62 2024-04-17T10:48:03.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6cd2ad9000 1 -- 172.21.15.83:0/715248409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55bd691ef8f0 con 0x55bd693932f0 2024-04-17T10:48:03.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ca7fff640 1 -- 172.21.15.83:0/715248409 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc005d7a0 con 0x55bd693932f0 2024-04-17T10:48:03.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ca7fff640 1 -- 172.21.15.83:0/715248409 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6cc007b020 con 0x55bd693932f0 2024-04-17T10:48:03.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6cd2ad9000 10 client.8361 fetch_fsmap finished waiting for FSMap version 62 2024-04-17T10:48:03.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6cd2ad9000 10 client.8361 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-17T10:48:03.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6cd2ad9000 20 client.8361 populate_metadata read hostname 'smithi083' 2024-04-17T10:48:03.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6cd2ad9000 1 --2- 172.21.15.83:0/715248409 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55bd693a5920 0x55bd693a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6cd2ad9000 4 client.8361 mds_command: new command op to 8049 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T10:48:03.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6cd2ad9000 1 -- 172.21.15.83:0/715248409 --> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bd691a3530 con 0x55bd693a5920 2024-04-17T10:48:03.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ccd708640 1 --2- 172.21.15.83:0/715248409 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55bd693a5920 0x55bd693a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ccd708640 1 --2- 172.21.15.83:0/715248409 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55bd693a5920 0x55bd693a5d30 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a00d15bdf5eaf2a3 server_cookie=3aa0fea454d8887d in_seq=0 out_seq=0 2024-04-17T10:48:03.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ca7fff640 10 client.8361 ms_handle_connect on v2:172.21.15.83:6836/2353406669 2024-04-17T10:48:03.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ca7fff640 1 -- 172.21.15.83:0/715248409 <== mds.0 v2:172.21.15.83:6836/2353406669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55bd691a3530 con 0x55bd693a5920 2024-04-17T10:48:03.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ca7fff640 10 client.8361 handle_command_reply: tid=0 2024-04-17T10:48:03.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6cd2ad9000 1 client.8361 shutdown 2024-04-17T10:48:03.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6cd2ad9000 1 -- 172.21.15.83:0/715248409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd693932f0 msgr2=0x55bd693a4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:03.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6cd2ad9000 1 --2- 172.21.15.83:0/715248409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd693932f0 0x55bd693a4fc0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f6cc005ddf0 tx=0x7f6cc0001260 comp rx=0 tx=0).stop 2024-04-17T10:48:03.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6cd2ad9000 1 -- 172.21.15.83:0/715248409 shutdown_connections 2024-04-17T10:48:03.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6cd2ad9000 1 --2- 172.21.15.83:0/715248409 >> [v2:172.21.15.83:6836/2353406669,v1:172.21.15.83:6837/2353406669] conn(0x55bd693a5920 0x55bd693a5d30 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6cd2ad9000 1 --2- 172.21.15.83:0/715248409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd693932f0 0x55bd693a4fc0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6cd2ad9000 1 --2- 172.21.15.83:0/715248409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55bd693929d0 0x55bd693f3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ca7fff640 0 client.8361 ms_handle_reset on v2:172.21.15.83:6836/2353406669 2024-04-17T10:48:03.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6cd2ad9000 1 --2- 172.21.15.83:0/715248409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd693490a0 0x55bd693f2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6cd2ad9000 1 -- 172.21.15.83:0/715248409 >> 172.21.15.83:0/715248409 conn(0x55bd69345020 msgr2=0x55bd693910a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:03.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6cd2ad9000 1 -- 172.21.15.83:0/715248409 shutdown_connections 2024-04-17T10:48:03.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6cd2ad9000 1 -- 172.21.15.83:0/715248409 wait complete. 2024-04-17T10:48:03.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6cd2ad9000 20 client.8361 trim_cache size 0 max 16384 2024-04-17T10:48:03.607 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:48:03.608 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-17T10:48:03.608 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-17T10:48:03.608 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:48:03.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ccf96c640 1 -- 172.21.15.83:0/182218635 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6cac066f60 msgr2=0x7f6cac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:03.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ccf96c640 1 --2- 172.21.15.83:0/182218635 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6cac066f60 0x7f6cac069420 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009bc0 tx=0x7f6cc4008040 comp rx=0 tx=0).stop 2024-04-17T10:48:03.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ccf96c640 1 -- 172.21.15.83:0/182218635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc815e0f0 msgr2=0x7f6cc80c8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:03.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ccf96c640 1 --2- 172.21.15.83:0/182218635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc815e0f0 0x7f6cc80c8810 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000ca80 tx=0x7f6cc000cf40 comp rx=0 tx=0).stop 2024-04-17T10:48:03.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ccf96c640 1 -- 172.21.15.83:0/182218635 shutdown_connections 2024-04-17T10:48:03.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ccf96c640 1 --2- 172.21.15.83:0/182218635 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6cac066f60 0x7f6cac069420 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ccf96c640 1 --2- 172.21.15.83:0/182218635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc815e0f0 0x7f6cc80c8810 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ccf96c640 1 --2- 172.21.15.83:0/182218635 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc80ce400 0x7f6cc815b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ccf96c640 1 --2- 172.21.15.83:0/182218635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cc80b5450 0x7f6cc815ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ccf96c640 1 -- 172.21.15.83:0/182218635 >> 172.21.15.83:0/182218635 conn(0x7f6cc80d0450 msgr2=0x7f6cc80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:03.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ccf96c640 1 -- 172.21.15.83:0/182218635 shutdown_connections 2024-04-17T10:48:03.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.602+0000 7f6ccf96c640 1 -- 172.21.15.83:0/182218635 wait complete. 2024-04-17T10:48:03.614 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:48:03.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.730+0000 7f5fb176e640 1 Processor -- start 2024-04-17T10:48:03.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.730+0000 7f5fb176e640 1 -- start start 2024-04-17T10:48:03.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.730+0000 7f5fb176e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5fac15ba30 0x7f5fac072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.730+0000 7f5fb176e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5fac0cfbe0 0x7f5fac06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.730+0000 7f5fb176e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fac06e320 con 0x7f5fac15c960 2024-04-17T10:48:03.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.730+0000 7f5fb176e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fac071ea0 con 0x7f5fac0cfbe0 2024-04-17T10:48:03.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.730+0000 7f5fb176e640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fac06e040 con 0x7f5fac15ba30 2024-04-17T10:48:03.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.734+0000 7f5fab7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5fac0cfbe0 0x7f5fac06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.734+0000 7f5fa3fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5fac15c960 0x7f5fac0cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60702/0 (socket says 172.21.15.83:60702) 2024-04-17T10:48:03.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.734+0000 7f5fa3fff640 1 -- 172.21.15.83:0/3768853862 learned_addr learned my addr 172.21.15.83:0/3768853862 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:03.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.734+0000 7f5faaffd640 1 --2- 172.21.15.83:0/3768853862 >> v2:172.21.15.167:3300/0 conn(0x7f5fac15ba30 0x7f5fac072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.734+0000 7f5fab7fe640 1 -- 172.21.15.83:0/3768853862 >> v2:172.21.15.167:3300/0 conn(0x7f5fac15ba30 msgr2=0x7f5fac072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:03.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.734+0000 7f5fab7fe640 1 --2- 172.21.15.83:0/3768853862 >> v2:172.21.15.167:3300/0 conn(0x7f5fac15ba30 0x7f5fac072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.734+0000 7f5fab7fe640 1 -- 172.21.15.83:0/3768853862 >> v1:172.21.15.83:6789/0 conn(0x7f5fac15c960 legacy=0x7f5fac0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:03.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.734+0000 7f5fab7fe640 1 -- 172.21.15.83:0/3768853862 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fac071850 con 0x7f5fac0cfbe0 2024-04-17T10:48:03.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.734+0000 7f5faa7fc640 1 -- 172.21.15.83:0/3768853862 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2237013385 0 0) 0x7f5fac06e320 con 0x7f5fac15c960 2024-04-17T10:48:03.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.734+0000 7f5faaffd640 1 --2- 172.21.15.83:0/3768853862 >> v2:172.21.15.167:3300/0 conn(0x7f5fac15ba30 0x7f5fac072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:48:03.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.734+0000 7f5fab7fe640 1 --2- 172.21.15.83:0/3768853862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fac0cfbe0 0x7f5fac06c490 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f5fac0b2410 tx=0x7f5fa401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dbfbc0af6a8cc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:03.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.734+0000 7f5faa7fc640 1 -- 172.21.15.83:0/3768853862 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa40b2070 con 0x7f5fac0cfbe0 2024-04-17T10:48:03.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.734+0000 7f5faa7fc640 1 -- 172.21.15.83:0/3768853862 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa40a57b0 con 0x7f5fac0cfbe0 2024-04-17T10:48:03.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.734+0000 7f5faa7fc640 1 -- 172.21.15.83:0/3768853862 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa40a5b80 con 0x7f5fac0cfbe0 2024-04-17T10:48:03.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.734+0000 7f5fb176e640 1 -- 172.21.15.83:0/3768853862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fac0cfbe0 msgr2=0x7f5fac06c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:03.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.734+0000 7f5fb176e640 1 --2- 172.21.15.83:0/3768853862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fac0cfbe0 0x7f5fac06c490 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f5fac0b2410 tx=0x7f5fa401c8a0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.734+0000 7f5fb176e640 1 -- 172.21.15.83:0/3768853862 shutdown_connections 2024-04-17T10:48:03.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.734+0000 7f5fb176e640 1 --2- 172.21.15.83:0/3768853862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fac0cfbe0 0x7f5fac06c490 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.734+0000 7f5fb176e640 1 --2- 172.21.15.83:0/3768853862 >> v2:172.21.15.167:3300/0 conn(0x7f5fac15ba30 0x7f5fac072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.734+0000 7f5fb176e640 1 -- 172.21.15.83:0/3768853862 >> 172.21.15.83:0/3768853862 conn(0x7f5fac06abe0 msgr2=0x7f5fac06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:03.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.734+0000 7f5fb176e640 1 -- 172.21.15.83:0/3768853862 shutdown_connections 2024-04-17T10:48:03.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.734+0000 7f5fb176e640 1 -- 172.21.15.83:0/3768853862 wait complete. 2024-04-17T10:48:03.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.738+0000 7f5fb176e640 1 Processor -- start 2024-04-17T10:48:03.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.738+0000 7f5fb176e640 1 -- start start 2024-04-17T10:48:03.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.738+0000 7f5fb176e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fac0cfbe0 0x7f5fac0b0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.738+0000 7f5fb176e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fac15ba30 0x7f5fac0b0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.738+0000 7f5fb176e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fac15c960 0x7f5fac149090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.738+0000 7f5fb176e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac06de20 con 0x7f5fac15c960 2024-04-17T10:48:03.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.738+0000 7f5fb176e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac06dca0 con 0x7f5fac15ba30 2024-04-17T10:48:03.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.738+0000 7f5fb176e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac168520 con 0x7f5fac0cfbe0 2024-04-17T10:48:03.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.738+0000 7f5fab7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fac15c960 0x7f5fac149090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.738+0000 7f5fab7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fac15c960 0x7f5fac149090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46456/0 (socket says 172.21.15.83:46456) 2024-04-17T10:48:03.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.738+0000 7f5fab7fe640 1 -- 172.21.15.83:0/3511654238 learned_addr learned my addr 172.21.15.83:0/3511654238 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:03.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.738+0000 7f5faaffd640 1 --2- 172.21.15.83:0/3511654238 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fac0cfbe0 0x7f5fac0b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.738+0000 7f5fa3fff640 1 --2- 172.21.15.83:0/3511654238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fac15ba30 0x7f5fac0b0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.738+0000 7f5fab7fe640 1 -- 172.21.15.83:0/3511654238 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fac0cfbe0 msgr2=0x7f5fac0b0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:03.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.738+0000 7f5fab7fe640 1 --2- 172.21.15.83:0/3511654238 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fac0cfbe0 0x7f5fac0b0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.738+0000 7f5fab7fe640 1 -- 172.21.15.83:0/3511654238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fac15ba30 msgr2=0x7f5fac0b0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:03.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.738+0000 7f5fab7fe640 1 --2- 172.21.15.83:0/3511654238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fac15ba30 0x7f5fac0b0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.738+0000 7f5fab7fe640 1 -- 172.21.15.83:0/3511654238 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fac0b2810 con 0x7f5fac15c960 2024-04-17T10:48:03.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.738+0000 7f5fa3fff640 1 --2- 172.21.15.83:0/3511654238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fac15ba30 0x7f5fac0b0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:03.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.738+0000 7f5fab7fe640 1 --2- 172.21.15.83:0/3511654238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fac15c960 0x7f5fac149090 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40194b0 tx=0x7f5fa4019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:03.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.738+0000 7f5fa37fe640 1 -- 172.21.15.83:0/3511654238 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa40b2050 con 0x7f5fac15c960 2024-04-17T10:48:03.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.738+0000 7f5fa37fe640 1 -- 172.21.15.83:0/3511654238 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa40b3040 con 0x7f5fac15c960 2024-04-17T10:48:03.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.738+0000 7f5fb176e640 1 -- 172.21.15.83:0/3511654238 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fac149ed0 con 0x7f5fac15c960 2024-04-17T10:48:03.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.738+0000 7f5fa37fe640 1 -- 172.21.15.83:0/3511654238 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa40ad650 con 0x7f5fac15c960 2024-04-17T10:48:03.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.738+0000 7f5fb176e640 1 -- 172.21.15.83:0/3511654238 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fac0b43f0 con 0x7f5fac15c960 2024-04-17T10:48:03.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.738+0000 7f5fb176e640 1 -- 172.21.15.83:0/3511654238 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fac0b2600 con 0x7f5fac15c960 2024-04-17T10:48:03.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.738+0000 7f5fa37fe640 1 -- 172.21.15.83:0/3511654238 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5fa40c0020 con 0x7f5fac15c960 2024-04-17T10:48:03.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.742+0000 7f5fa37fe640 1 --2- 172.21.15.83:0/3511654238 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5f8c066f60 0x7f5f8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:03.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.742+0000 7f5fa37fe640 1 -- 172.21.15.83:0/3511654238 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5fa411f910 con 0x7f5fac15c960 2024-04-17T10:48:03.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.742+0000 7f5faaffd640 1 --2- 172.21.15.83:0/3511654238 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5f8c066f60 0x7f5f8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:03.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.742+0000 7f5faaffd640 1 --2- 172.21.15.83:0/3511654238 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5f8c066f60 0x7f5f8c069420 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c00c900 tx=0x7f5f9c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:03.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.746+0000 7f5fa37fe640 1 -- 172.21.15.83:0/3511654238 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5fa4016520 con 0x7f5fac15c960 2024-04-17T10:48:03.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.930+0000 7f5fb176e640 1 -- 172.21.15.83:0/3511654238 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5fac074f90 con 0x7f5fac15c960 2024-04-17T10:48:03.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.930+0000 7f5fa37fe640 1 -- 172.21.15.83:0/3511654238 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f5fa40f0920 con 0x7f5fac15c960 2024-04-17T10:48:03.926 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:03.926 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/437268957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":437268957},{"type":"v1","addr":"172.21.15.167:6833","nonce":437268957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":6253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3214640674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3214640674},{"type":"v1","addr":"172.21.15.167:6835","nonce":3214640674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":8046,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/715498323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":715498323},{"type":"v1","addr":"172.21.15.83:6835","nonce":715498323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:47:29.385103+0000","modified":"2024-04-17T10:47:31.389088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8049":{"gid":8049,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2353406669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":2353406669},{"type":"v1","addr":"172.21.15.83:6837","nonce":2353406669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-17T10:48:03.926 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 62 2024-04-17T10:48:03.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.930+0000 7f5fb176e640 1 -- 172.21.15.83:0/3511654238 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5f8c066f60 msgr2=0x7f5f8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:03.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.930+0000 7f5fb176e640 1 --2- 172.21.15.83:0/3511654238 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5f8c066f60 0x7f5f8c069420 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c00c900 tx=0x7f5f9c008040 comp rx=0 tx=0).stop 2024-04-17T10:48:03.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.930+0000 7f5fb176e640 1 -- 172.21.15.83:0/3511654238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fac15c960 msgr2=0x7f5fac149090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:03.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.930+0000 7f5fb176e640 1 --2- 172.21.15.83:0/3511654238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fac15c960 0x7f5fac149090 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40194b0 tx=0x7f5fa4019640 comp rx=0 tx=0).stop 2024-04-17T10:48:03.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.930+0000 7f5fb176e640 1 -- 172.21.15.83:0/3511654238 shutdown_connections 2024-04-17T10:48:03.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.930+0000 7f5fb176e640 1 --2- 172.21.15.83:0/3511654238 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5f8c066f60 0x7f5f8c069420 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.930+0000 7f5fb176e640 1 --2- 172.21.15.83:0/3511654238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fac15c960 0x7f5fac149090 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.930+0000 7f5fb176e640 1 --2- 172.21.15.83:0/3511654238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fac15ba30 0x7f5fac0b0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.930+0000 7f5fb176e640 1 --2- 172.21.15.83:0/3511654238 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fac0cfbe0 0x7f5fac0b0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:03.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.930+0000 7f5fb176e640 1 -- 172.21.15.83:0/3511654238 >> 172.21.15.83:0/3511654238 conn(0x7f5fac06abe0 msgr2=0x7f5fac1526b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:03.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.930+0000 7f5fb176e640 1 -- 172.21.15.83:0/3511654238 shutdown_connections 2024-04-17T10:48:03.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:03.930+0000 7f5fb176e640 1 -- 172.21.15.83:0/3511654238 wait complete. 2024-04-17T10:48:03.943 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-17T10:48:03.943 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-17T10:48:04.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.054+0000 7fc31db7b640 1 Processor -- start 2024-04-17T10:48:04.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.054+0000 7fc31db7b640 1 -- start start 2024-04-17T10:48:04.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.058+0000 7fc31db7b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc3180b46e0 0x7fc31815ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:04.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.058+0000 7fc31db7b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc3180b3570 0x7fc31806daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:04.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.058+0000 7fc31db7b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc31815aef0 con 0x7fc318154f40 2024-04-17T10:48:04.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.058+0000 7fc31db7b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc31806f2f0 con 0x7fc3180b3570 2024-04-17T10:48:04.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.058+0000 7fc31db7b640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc31815f2d0 con 0x7fc3180b46e0 2024-04-17T10:48:04.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.058+0000 7fc3177fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc318154f40 0x7fc3180b3fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60710/0 (socket says 172.21.15.83:60710) 2024-04-17T10:48:04.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.058+0000 7fc3177fe640 1 -- 172.21.15.83:0/3893835165 learned_addr learned my addr 172.21.15.83:0/3893835165 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:04.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.058+0000 7fc317fff640 1 --2- 172.21.15.83:0/3893835165 >> v2:172.21.15.83:3300/0 conn(0x7fc3180b3570 0x7fc31806daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:04.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.058+0000 7fc316ffd640 1 --2- 172.21.15.83:0/3893835165 >> v2:172.21.15.167:3300/0 conn(0x7fc3180b46e0 0x7fc31815ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:04.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.058+0000 7fc317fff640 1 -- 172.21.15.83:0/3893835165 >> v2:172.21.15.167:3300/0 conn(0x7fc3180b46e0 msgr2=0x7fc31815ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:04.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.058+0000 7fc317fff640 1 --2- 172.21.15.83:0/3893835165 >> v2:172.21.15.167:3300/0 conn(0x7fc3180b46e0 0x7fc31815ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:04.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.058+0000 7fc317fff640 1 -- 172.21.15.83:0/3893835165 >> v1:172.21.15.83:6789/0 conn(0x7fc318154f40 legacy=0x7fc3180b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:04.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.058+0000 7fc3167fc640 1 -- 172.21.15.83:0/3893835165 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3783541214 0 0) 0x7fc31815aef0 con 0x7fc318154f40 2024-04-17T10:48:04.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.058+0000 7fc317fff640 1 -- 172.21.15.83:0/3893835165 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc31815b1d0 con 0x7fc3180b3570 2024-04-17T10:48:04.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.058+0000 7fc317fff640 1 --2- 172.21.15.83:0/3893835165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3180b3570 0x7fc31806daf0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fc3180724d0 tx=0x7fc31001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38dac48c61d173bb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:04.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.058+0000 7fc3167fc640 1 -- 172.21.15.83:0/3893835165 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3100b2070 con 0x7fc3180b3570 2024-04-17T10:48:04.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.058+0000 7fc3167fc640 1 -- 172.21.15.83:0/3893835165 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3100a59c0 con 0x7fc3180b3570 2024-04-17T10:48:04.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.058+0000 7fc3167fc640 1 -- 172.21.15.83:0/3893835165 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3100a5d90 con 0x7fc3180b3570 2024-04-17T10:48:04.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.058+0000 7fc31db7b640 1 -- 172.21.15.83:0/3893835165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3180b3570 msgr2=0x7fc31806daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:04.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.058+0000 7fc31db7b640 1 --2- 172.21.15.83:0/3893835165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3180b3570 0x7fc31806daf0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fc3180724d0 tx=0x7fc31001c900 comp rx=0 tx=0).stop 2024-04-17T10:48:04.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.058+0000 7fc31db7b640 1 -- 172.21.15.83:0/3893835165 shutdown_connections 2024-04-17T10:48:04.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.058+0000 7fc31db7b640 1 --2- 172.21.15.83:0/3893835165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3180b3570 0x7fc31806daf0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:04.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.058+0000 7fc31db7b640 1 --2- 172.21.15.83:0/3893835165 >> v2:172.21.15.167:3300/0 conn(0x7fc3180b46e0 0x7fc31815ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:04.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.058+0000 7fc31db7b640 1 -- 172.21.15.83:0/3893835165 >> 172.21.15.83:0/3893835165 conn(0x7fc31806bcf0 msgr2=0x7fc31806c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:04.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.058+0000 7fc31db7b640 1 -- 172.21.15.83:0/3893835165 shutdown_connections 2024-04-17T10:48:04.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.058+0000 7fc31db7b640 1 -- 172.21.15.83:0/3893835165 wait complete. 2024-04-17T10:48:04.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.062+0000 7fc31db7b640 1 Processor -- start 2024-04-17T10:48:04.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.062+0000 7fc31db7b640 1 -- start start 2024-04-17T10:48:04.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.062+0000 7fc31db7b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3180b3570 0x7fc3181f5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:04.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.062+0000 7fc31db7b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3180b46e0 0x7fc31815c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:04.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.062+0000 7fc31db7b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc318154f40 0x7fc31815cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:04.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.062+0000 7fc31db7b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31806e840 con 0x7fc3180b3570 2024-04-17T10:48:04.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.062+0000 7fc31db7b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31815be50 con 0x7fc3180b46e0 2024-04-17T10:48:04.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.062+0000 7fc31db7b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc31806e9c0 con 0x7fc318154f40 2024-04-17T10:48:04.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.062+0000 7fc3177fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3180b3570 0x7fc3181f5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:04.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.062+0000 7fc3177fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3180b3570 0x7fc3181f5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46462/0 (socket says 172.21.15.83:46462) 2024-04-17T10:48:04.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.062+0000 7fc3177fe640 1 -- 172.21.15.83:0/3010243979 learned_addr learned my addr 172.21.15.83:0/3010243979 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:04.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.062+0000 7fc316ffd640 1 --2- 172.21.15.83:0/3010243979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3180b46e0 0x7fc31815c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:04.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.062+0000 7fc3177fe640 1 -- 172.21.15.83:0/3010243979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc318154f40 msgr2=0x7fc31815cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:04.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.062+0000 7fc3177fe640 1 --2- 172.21.15.83:0/3010243979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc318154f40 0x7fc31815cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:04.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.062+0000 7fc3177fe640 1 -- 172.21.15.83:0/3010243979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3180b46e0 msgr2=0x7fc31815c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:04.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.062+0000 7fc3177fe640 1 --2- 172.21.15.83:0/3010243979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3180b46e0 0x7fc31815c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:04.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.062+0000 7fc3177fe640 1 -- 172.21.15.83:0/3010243979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc31815d550 con 0x7fc3180b3570 2024-04-17T10:48:04.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.062+0000 7fc3177fe640 1 --2- 172.21.15.83:0/3010243979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3180b3570 0x7fc3181f5790 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fc3080011f0 tx=0x7fc30800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:04.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.062+0000 7fc2fffff640 1 -- 172.21.15.83:0/3010243979 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc308018070 con 0x7fc3180b3570 2024-04-17T10:48:04.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.062+0000 7fc2fffff640 1 -- 172.21.15.83:0/3010243979 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3080074e0 con 0x7fc3180b3570 2024-04-17T10:48:04.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.062+0000 7fc2fffff640 1 -- 172.21.15.83:0/3010243979 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc308004a10 con 0x7fc3180b3570 2024-04-17T10:48:04.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.062+0000 7fc31db7b640 1 -- 172.21.15.83:0/3010243979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc31815d830 con 0x7fc3180b3570 2024-04-17T10:48:04.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.062+0000 7fc31db7b640 1 -- 172.21.15.83:0/3010243979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc31815ddf0 con 0x7fc3180b3570 2024-04-17T10:48:04.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.062+0000 7fc31db7b640 1 -- 172.21.15.83:0/3010243979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc318074f90 con 0x7fc3180b3570 2024-04-17T10:48:04.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.066+0000 7fc2fffff640 1 -- 172.21.15.83:0/3010243979 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc308004bb0 con 0x7fc3180b3570 2024-04-17T10:48:04.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.066+0000 7fc2fffff640 1 --2- 172.21.15.83:0/3010243979 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc304066f60 0x7fc304069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:04.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.066+0000 7fc2fffff640 1 -- 172.21.15.83:0/3010243979 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc308084430 con 0x7fc3180b3570 2024-04-17T10:48:04.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.066+0000 7fc316ffd640 1 --2- 172.21.15.83:0/3010243979 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc304066f60 0x7fc304069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:04.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.066+0000 7fc316ffd640 1 --2- 172.21.15.83:0/3010243979 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc304066f60 0x7fc304069420 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc3180ccb70 tx=0x7fc30c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:04.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.070+0000 7fc2fffff640 1 -- 172.21.15.83:0/3010243979 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc308055440 con 0x7fc3180b3570 2024-04-17T10:48:04.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.278+0000 7fc31db7b640 1 -- 172.21.15.83:0/3010243979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc3180b05a0 con 0x7fc3180b3570 2024-04-17T10:48:04.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.282+0000 7fc2fffff640 1 -- 172.21.15.83:0/3010243979 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7fc308054d40 con 0x7fc3180b3570 2024-04-17T10:48:04.277 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:04.277 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":616,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":103,"num_osds":8,"num_up_osds":8,"osd_up_since":1713350279,"num_in_osds":8,"osd_in_since":1713350270,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076816697,"bytes_used":2697879552,"bytes_avail":770396233728,"bytes_total":773094113280,"write_bytes_sec":48843159,"read_op_per_sec":0,"write_op_per_sec":49},"fsmap":{"epoch":62,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8049}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-04-17T10:48:02.357277+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-17T10:48:04.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.282+0000 7fc31db7b640 1 -- 172.21.15.83:0/3010243979 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc304066f60 msgr2=0x7fc304069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:04.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.282+0000 7fc31db7b640 1 --2- 172.21.15.83:0/3010243979 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc304066f60 0x7fc304069420 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc3180ccb70 tx=0x7fc30c008040 comp rx=0 tx=0).stop 2024-04-17T10:48:04.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.282+0000 7fc31db7b640 1 -- 172.21.15.83:0/3010243979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3180b3570 msgr2=0x7fc3181f5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:04.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.282+0000 7fc31db7b640 1 --2- 172.21.15.83:0/3010243979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3180b3570 0x7fc3181f5790 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fc3080011f0 tx=0x7fc30800c9b0 comp rx=0 tx=0).stop 2024-04-17T10:48:04.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.282+0000 7fc31db7b640 1 -- 172.21.15.83:0/3010243979 shutdown_connections 2024-04-17T10:48:04.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.282+0000 7fc31db7b640 1 --2- 172.21.15.83:0/3010243979 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc304066f60 0x7fc304069420 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:04.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.282+0000 7fc31db7b640 1 --2- 172.21.15.83:0/3010243979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc318154f40 0x7fc31815cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:04.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.282+0000 7fc31db7b640 1 --2- 172.21.15.83:0/3010243979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3180b46e0 0x7fc31815c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:04.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.282+0000 7fc31db7b640 1 --2- 172.21.15.83:0/3010243979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3180b3570 0x7fc3181f5790 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:04.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.282+0000 7fc31db7b640 1 -- 172.21.15.83:0/3010243979 >> 172.21.15.83:0/3010243979 conn(0x7fc31806bcf0 msgr2=0x7fc3180b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:04.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.282+0000 7fc31db7b640 1 -- 172.21.15.83:0/3010243979 shutdown_connections 2024-04-17T10:48:04.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:04.282+0000 7fc31db7b640 1 -- 172.21.15.83:0/3010243979 wait complete. 2024-04-17T10:48:04.294 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused'} 2024-04-17T10:48:05.295 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:48:05.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.430+0000 7fc166051640 1 Processor -- start 2024-04-17T10:48:05.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.430+0000 7fc166051640 1 -- start start 2024-04-17T10:48:05.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.434+0000 7fc166051640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1600aedd0 0x7fc16006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:05.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.434+0000 7fc166051640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc16006dae0 0x7fc1600b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:05.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.434+0000 7fc166051640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc16015e440 con 0x7fc16015db50 2024-04-17T10:48:05.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.434+0000 7fc166051640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc16015e720 con 0x7fc16006dae0 2024-04-17T10:48:05.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.434+0000 7fc166051640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16015ea00 con 0x7fc1600aedd0 2024-04-17T10:48:05.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.434+0000 7fc15ffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc16006dae0 0x7fc1600b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:05.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.434+0000 7fc15ffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc16006dae0 0x7fc1600b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46468/0 (socket says 172.21.15.83:46468) 2024-04-17T10:48:05.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.434+0000 7fc15ffff640 1 -- 172.21.15.83:0/1474167846 learned_addr learned my addr 172.21.15.83:0/1474167846 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:05.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.434+0000 7fc15effd640 1 --2- 172.21.15.83:0/1474167846 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1600aedd0 0x7fc16006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:05.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.434+0000 7fc15ffff640 1 -- 172.21.15.83:0/1474167846 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1600aedd0 msgr2=0x7fc16006d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:05.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.434+0000 7fc15ffff640 1 --2- 172.21.15.83:0/1474167846 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1600aedd0 0x7fc16006d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:05.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.434+0000 7fc15ffff640 1 -- 172.21.15.83:0/1474167846 >> v1:172.21.15.167:6789/0 conn(0x7fc16015db50 legacy=0x7fc160070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:05.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.434+0000 7fc15ffff640 1 -- 172.21.15.83:0/1474167846 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc16015ece0 con 0x7fc16006dae0 2024-04-17T10:48:05.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.434+0000 7fc15effd640 1 --2- 172.21.15.83:0/1474167846 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1600aedd0 0x7fc16006d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:05.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.434+0000 7fc15ffff640 1 --2- 172.21.15.83:0/1474167846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc16006dae0 0x7fc1600b2e10 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fc158009870 tx=0x7fc15801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb24761da139fb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:05.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.434+0000 7fc15e7fc640 1 -- 172.21.15.83:0/1474167846 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1580b0070 con 0x7fc16006dae0 2024-04-17T10:48:05.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.434+0000 7fc15e7fc640 1 -- 172.21.15.83:0/1474167846 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1580ab600 con 0x7fc16006dae0 2024-04-17T10:48:05.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.434+0000 7fc15e7fc640 1 -- 172.21.15.83:0/1474167846 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1580ab9d0 con 0x7fc16006dae0 2024-04-17T10:48:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.438+0000 7fc166051640 1 -- 172.21.15.83:0/1474167846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc16006dae0 msgr2=0x7fc1600b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.438+0000 7fc166051640 1 --2- 172.21.15.83:0/1474167846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc16006dae0 0x7fc1600b2e10 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fc158009870 tx=0x7fc15801c520 comp rx=0 tx=0).stop 2024-04-17T10:48:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.438+0000 7fc166051640 1 -- 172.21.15.83:0/1474167846 shutdown_connections 2024-04-17T10:48:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.438+0000 7fc166051640 1 --2- 172.21.15.83:0/1474167846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc16006dae0 0x7fc1600b2e10 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:05.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.438+0000 7fc166051640 1 --2- 172.21.15.83:0/1474167846 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1600aedd0 0x7fc16006d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:05.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.438+0000 7fc166051640 1 -- 172.21.15.83:0/1474167846 >> 172.21.15.83:0/1474167846 conn(0x7fc16006abe0 msgr2=0x7fc16006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:05.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.438+0000 7fc166051640 1 -- 172.21.15.83:0/1474167846 shutdown_connections 2024-04-17T10:48:05.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.438+0000 7fc166051640 1 -- 172.21.15.83:0/1474167846 wait complete. 2024-04-17T10:48:05.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.438+0000 7fc166051640 1 Processor -- start 2024-04-17T10:48:05.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.438+0000 7fc166051640 1 -- start start 2024-04-17T10:48:05.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.438+0000 7fc166051640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc16006dae0 0x7fc160156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:05.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.438+0000 7fc166051640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1600aedd0 0x7fc1600cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:05.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.438+0000 7fc166051640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc16015db50 0x7fc1600ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:05.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.438+0000 7fc166051640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1600b3ef0 con 0x7fc1600aedd0 2024-04-17T10:48:05.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.438+0000 7fc166051640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1600b3d70 con 0x7fc16006dae0 2024-04-17T10:48:05.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.438+0000 7fc166051640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1600b4070 con 0x7fc16015db50 2024-04-17T10:48:05.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.438+0000 7fc15effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1600aedd0 0x7fc1600cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:05.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.438+0000 7fc15effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1600aedd0 0x7fc1600cc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46474/0 (socket says 172.21.15.83:46474) 2024-04-17T10:48:05.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.438+0000 7fc15effd640 1 -- 172.21.15.83:0/3678115922 learned_addr learned my addr 172.21.15.83:0/3678115922 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:05.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.438+0000 7fc15ffff640 1 --2- 172.21.15.83:0/3678115922 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc16015db50 0x7fc1600ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:05.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.438+0000 7fc15f7fe640 1 --2- 172.21.15.83:0/3678115922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc16006dae0 0x7fc160156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:05.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.438+0000 7fc15effd640 1 -- 172.21.15.83:0/3678115922 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc16015db50 msgr2=0x7fc1600ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:05.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.438+0000 7fc15effd640 1 --2- 172.21.15.83:0/3678115922 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc16015db50 0x7fc1600ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:05.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.438+0000 7fc15effd640 1 -- 172.21.15.83:0/3678115922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc16006dae0 msgr2=0x7fc160156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:05.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.438+0000 7fc15effd640 1 --2- 172.21.15.83:0/3678115922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc16006dae0 0x7fc160156c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:05.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.438+0000 7fc15effd640 1 -- 172.21.15.83:0/3678115922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1600ad590 con 0x7fc1600aedd0 2024-04-17T10:48:05.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.438+0000 7fc15effd640 1 --2- 172.21.15.83:0/3678115922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1600aedd0 0x7fc1600cc610 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fc14800c7c0 tx=0x7fc14800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:05.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.438+0000 7fc157fff640 1 -- 172.21.15.83:0/3678115922 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc148007a30 con 0x7fc1600aedd0 2024-04-17T10:48:05.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.438+0000 7fc157fff640 1 -- 172.21.15.83:0/3678115922 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc148007bd0 con 0x7fc1600aedd0 2024-04-17T10:48:05.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.438+0000 7fc166051640 1 -- 172.21.15.83:0/3678115922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1600ad870 con 0x7fc1600aedd0 2024-04-17T10:48:05.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.438+0000 7fc157fff640 1 -- 172.21.15.83:0/3678115922 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1480113f0 con 0x7fc1600aedd0 2024-04-17T10:48:05.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.438+0000 7fc166051640 1 -- 172.21.15.83:0/3678115922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1600ade30 con 0x7fc1600aedd0 2024-04-17T10:48:05.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.442+0000 7fc166051640 1 -- 172.21.15.83:0/3678115922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc160074f90 con 0x7fc1600aedd0 2024-04-17T10:48:05.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.442+0000 7fc157fff640 1 -- 172.21.15.83:0/3678115922 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc148011590 con 0x7fc1600aedd0 2024-04-17T10:48:05.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.442+0000 7fc157fff640 1 --2- 172.21.15.83:0/3678115922 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc140066f60 0x7fc140069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:05.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.442+0000 7fc157fff640 1 -- 172.21.15.83:0/3678115922 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc148085270 con 0x7fc1600aedd0 2024-04-17T10:48:05.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.442+0000 7fc15f7fe640 1 --2- 172.21.15.83:0/3678115922 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc140066f60 0x7fc140069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:05.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.442+0000 7fc15f7fe640 1 --2- 172.21.15.83:0/3678115922 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc140066f60 0x7fc140069420 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc150007bf0 tx=0x7fc150007240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:05.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.446+0000 7fc157fff640 1 -- 172.21.15.83:0/3678115922 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc148015020 con 0x7fc1600aedd0 2024-04-17T10:48:05.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.634+0000 7fc166051640 1 -- 172.21.15.83:0/3678115922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1600b2af0 con 0x7fc1600aedd0 2024-04-17T10:48:05.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.634+0000 7fc157fff640 1 -- 172.21.15.83:0/3678115922 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fc148056280 con 0x7fc1600aedd0 2024-04-17T10:48:05.630 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:05.630 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/437268957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":437268957},{"type":"v1","addr":"172.21.15.167:6833","nonce":437268957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":6253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3214640674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3214640674},{"type":"v1","addr":"172.21.15.167:6835","nonce":3214640674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":8046,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/715498323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":715498323},{"type":"v1","addr":"172.21.15.83:6835","nonce":715498323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:47:29.385103+0000","modified":"2024-04-17T10:47:31.389088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8049":{"gid":8049,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2353406669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":2353406669},{"type":"v1","addr":"172.21.15.83:6837","nonce":2353406669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-17T10:48:05.630 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 62 2024-04-17T10:48:05.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.634+0000 7fc166051640 1 -- 172.21.15.83:0/3678115922 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc140066f60 msgr2=0x7fc140069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:05.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.634+0000 7fc166051640 1 --2- 172.21.15.83:0/3678115922 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc140066f60 0x7fc140069420 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc150007bf0 tx=0x7fc150007240 comp rx=0 tx=0).stop 2024-04-17T10:48:05.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.634+0000 7fc166051640 1 -- 172.21.15.83:0/3678115922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1600aedd0 msgr2=0x7fc1600cc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:05.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.634+0000 7fc166051640 1 --2- 172.21.15.83:0/3678115922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1600aedd0 0x7fc1600cc610 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fc14800c7c0 tx=0x7fc14800cc80 comp rx=0 tx=0).stop 2024-04-17T10:48:05.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.634+0000 7fc166051640 1 -- 172.21.15.83:0/3678115922 shutdown_connections 2024-04-17T10:48:05.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.634+0000 7fc166051640 1 --2- 172.21.15.83:0/3678115922 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc140066f60 0x7fc140069420 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:05.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.634+0000 7fc166051640 1 --2- 172.21.15.83:0/3678115922 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc16015db50 0x7fc1600ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:05.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.634+0000 7fc166051640 1 --2- 172.21.15.83:0/3678115922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1600aedd0 0x7fc1600cc610 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:05.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.634+0000 7fc166051640 1 --2- 172.21.15.83:0/3678115922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc16006dae0 0x7fc160156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:05.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.634+0000 7fc166051640 1 -- 172.21.15.83:0/3678115922 >> 172.21.15.83:0/3678115922 conn(0x7fc16006abe0 msgr2=0x7fc160070470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:05.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.634+0000 7fc166051640 1 -- 172.21.15.83:0/3678115922 shutdown_connections 2024-04-17T10:48:05.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.634+0000 7fc166051640 1 -- 172.21.15.83:0/3678115922 wait complete. 2024-04-17T10:48:05.647 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-17T10:48:05.648 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-17T10:48:05.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.758+0000 7fe8bb643640 1 Processor -- start 2024-04-17T10:48:05.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.758+0000 7fe8bb643640 1 -- start start 2024-04-17T10:48:05.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.758+0000 7fe8bb643640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe8b415db50 0x7fe8b4070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:05.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.758+0000 7fe8bb643640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe8b406dcb0 0x7fe8b40b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:05.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.758+0000 7fe8bb643640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8b40709b0 con 0x7fe8b40aedd0 2024-04-17T10:48:05.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.758+0000 7fe8bb643640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8b415e440 con 0x7fe8b415db50 2024-04-17T10:48:05.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.758+0000 7fe8bb643640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8b415e720 con 0x7fe8b406dcb0 2024-04-17T10:48:05.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.758+0000 7fe8b93df640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe8b415db50 0x7fe8b4070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:05.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.758+0000 7fe8b8bde640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe8b40aedd0 0x7fe8b406d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60724/0 (socket says 172.21.15.83:60724) 2024-04-17T10:48:05.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.758+0000 7fe8b8bde640 1 -- 172.21.15.83:0/421096845 learned_addr learned my addr 172.21.15.83:0/421096845 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:05.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.758+0000 7fe8b9be0640 1 --2- 172.21.15.83:0/421096845 >> v2:172.21.15.167:3300/0 conn(0x7fe8b406dcb0 0x7fe8b40b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:05.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.758+0000 7fe8b93df640 1 -- 172.21.15.83:0/421096845 >> v2:172.21.15.167:3300/0 conn(0x7fe8b406dcb0 msgr2=0x7fe8b40b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:05.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.758+0000 7fe8b93df640 1 --2- 172.21.15.83:0/421096845 >> v2:172.21.15.167:3300/0 conn(0x7fe8b406dcb0 0x7fe8b40b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:05.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.758+0000 7fe8b93df640 1 -- 172.21.15.83:0/421096845 >> v1:172.21.15.83:6789/0 conn(0x7fe8b40aedd0 legacy=0x7fe8b406d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:05.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.758+0000 7fe8b93df640 1 -- 172.21.15.83:0/421096845 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b415ea00 con 0x7fe8b415db50 2024-04-17T10:48:05.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.758+0000 7fe8abfff640 1 -- 172.21.15.83:0/421096845 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1535210175 0 0) 0x7fe8a4001120 con 0x7fe8b40aedd0 2024-04-17T10:48:05.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.762+0000 7fe8b93df640 1 --2- 172.21.15.83:0/421096845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8b415db50 0x7fe8b4070470 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00097c0 tx=0x7fe8b001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9167d0bb27d467bf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:05.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.762+0000 7fe8abfff640 1 -- 172.21.15.83:0/421096845 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b00ab430 con 0x7fe8b415db50 2024-04-17T10:48:05.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.762+0000 7fe8abfff640 1 -- 172.21.15.83:0/421096845 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b00b0070 con 0x7fe8b415db50 2024-04-17T10:48:05.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.762+0000 7fe8abfff640 1 -- 172.21.15.83:0/421096845 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b00ab7f0 con 0x7fe8b415db50 2024-04-17T10:48:05.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.762+0000 7fe8bb643640 1 -- 172.21.15.83:0/421096845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8b415db50 msgr2=0x7fe8b4070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:05.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.762+0000 7fe8bb643640 1 --2- 172.21.15.83:0/421096845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8b415db50 0x7fe8b4070470 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00097c0 tx=0x7fe8b001c430 comp rx=0 tx=0).stop 2024-04-17T10:48:05.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.762+0000 7fe8bb643640 1 -- 172.21.15.83:0/421096845 shutdown_connections 2024-04-17T10:48:05.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.762+0000 7fe8bb643640 1 --2- 172.21.15.83:0/421096845 >> v2:172.21.15.167:3300/0 conn(0x7fe8b406dcb0 0x7fe8b40b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:05.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.762+0000 7fe8bb643640 1 --2- 172.21.15.83:0/421096845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8b415db50 0x7fe8b4070470 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:05.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.762+0000 7fe8bb643640 1 -- 172.21.15.83:0/421096845 >> 172.21.15.83:0/421096845 conn(0x7fe8b406abe0 msgr2=0x7fe8b406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:05.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.762+0000 7fe8bb643640 1 -- 172.21.15.83:0/421096845 shutdown_connections 2024-04-17T10:48:05.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.762+0000 7fe8bb643640 1 -- 172.21.15.83:0/421096845 wait complete. 2024-04-17T10:48:05.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.762+0000 7fe8bb643640 1 Processor -- start 2024-04-17T10:48:05.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.762+0000 7fe8bb643640 1 -- start start 2024-04-17T10:48:05.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.766+0000 7fe8bb643640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8b406dcb0 0x7fe8b40c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:05.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.766+0000 7fe8bb643640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8b40aedd0 0x7fe8b41596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:05.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.766+0000 7fe8bb643640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8b415db50 0x7fe8b41578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:05.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.766+0000 7fe8bb643640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b40b3ce0 con 0x7fe8b40aedd0 2024-04-17T10:48:05.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.766+0000 7fe8bb643640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b40b3b60 con 0x7fe8b415db50 2024-04-17T10:48:05.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.766+0000 7fe8bb643640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b40b3e60 con 0x7fe8b406dcb0 2024-04-17T10:48:05.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.766+0000 7fe8b8bde640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8b40aedd0 0x7fe8b41596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:05.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.766+0000 7fe8b8bde640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8b40aedd0 0x7fe8b41596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46478/0 (socket says 172.21.15.83:46478) 2024-04-17T10:48:05.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.766+0000 7fe8b8bde640 1 -- 172.21.15.83:0/995380910 learned_addr learned my addr 172.21.15.83:0/995380910 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:05.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.766+0000 7fe8b9be0640 1 --2- 172.21.15.83:0/995380910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8b415db50 0x7fe8b41578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:05.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.766+0000 7fe8b8bde640 1 -- 172.21.15.83:0/995380910 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8b406dcb0 msgr2=0x7fe8b40c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:05.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.766+0000 7fe8b8bde640 1 --2- 172.21.15.83:0/995380910 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8b406dcb0 0x7fe8b40c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:05.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.766+0000 7fe8b8bde640 1 -- 172.21.15.83:0/995380910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8b415db50 msgr2=0x7fe8b41578d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:05.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.766+0000 7fe8b8bde640 1 --2- 172.21.15.83:0/995380910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8b415db50 0x7fe8b41578d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:05.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.766+0000 7fe8b8bde640 1 -- 172.21.15.83:0/995380910 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b40c56e0 con 0x7fe8b40aedd0 2024-04-17T10:48:05.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.766+0000 7fe8b8bde640 1 --2- 172.21.15.83:0/995380910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8b40aedd0 0x7fe8b41596f0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fe8a400c780 tx=0x7fe8a400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:05.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.766+0000 7fe8a9ffb640 1 -- 172.21.15.83:0/995380910 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4018070 con 0x7fe8b40aedd0 2024-04-17T10:48:05.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.766+0000 7fe8a9ffb640 1 -- 172.21.15.83:0/995380910 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a40072f0 con 0x7fe8b40aedd0 2024-04-17T10:48:05.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.766+0000 7fe8bb643640 1 -- 172.21.15.83:0/995380910 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b40c5980 con 0x7fe8b40aedd0 2024-04-17T10:48:05.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.766+0000 7fe8a9ffb640 1 -- 172.21.15.83:0/995380910 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a40047f0 con 0x7fe8b40aedd0 2024-04-17T10:48:05.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.766+0000 7fe8bb643640 1 -- 172.21.15.83:0/995380910 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b41570d0 con 0x7fe8b40aedd0 2024-04-17T10:48:05.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.766+0000 7fe8bb643640 1 -- 172.21.15.83:0/995380910 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b4074f90 con 0x7fe8b40aedd0 2024-04-17T10:48:05.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.766+0000 7fe8a9ffb640 1 -- 172.21.15.83:0/995380910 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe8a401b070 con 0x7fe8b40aedd0 2024-04-17T10:48:05.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.766+0000 7fe8a9ffb640 1 --2- 172.21.15.83:0/995380910 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe898066f60 0x7fe898069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:05.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.766+0000 7fe8a9ffb640 1 -- 172.21.15.83:0/995380910 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8a4084340 con 0x7fe8b40aedd0 2024-04-17T10:48:05.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.766+0000 7fe8b93df640 1 --2- 172.21.15.83:0/995380910 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe898066f60 0x7fe898069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:05.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.766+0000 7fe8b93df640 1 --2- 172.21.15.83:0/995380910 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe898066f60 0x7fe898069420 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00af000 tx=0x7fe8b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:05.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.770+0000 7fe8a9ffb640 1 -- 172.21.15.83:0/995380910 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe8b40709b0 con 0x7fe8b40aedd0 2024-04-17T10:48:05.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.990+0000 7fe8bb643640 1 -- 172.21.15.83:0/995380910 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe8b4070610 con 0x7fe8b40aedd0 2024-04-17T10:48:05.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.990+0000 7fe8a9ffb640 1 -- 172.21.15.83:0/995380910 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7fe8a40553e0 con 0x7fe8b40aedd0 2024-04-17T10:48:05.988 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:05.988 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":618,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":103,"num_osds":8,"num_up_osds":8,"osd_up_since":1713350279,"num_in_osds":8,"osd_in_since":1713350270,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076816697,"bytes_used":2697879552,"bytes_avail":770396233728,"bytes_total":773094113280,"write_bytes_sec":48842312,"read_op_per_sec":0,"write_op_per_sec":49},"fsmap":{"epoch":62,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8049}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-04-17T10:48:02.357277+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-17T10:48:05.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.994+0000 7fe8bb643640 1 -- 172.21.15.83:0/995380910 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe898066f60 msgr2=0x7fe898069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:05.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.994+0000 7fe8bb643640 1 --2- 172.21.15.83:0/995380910 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe898066f60 0x7fe898069420 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00af000 tx=0x7fe8b0008040 comp rx=0 tx=0).stop 2024-04-17T10:48:05.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.994+0000 7fe8bb643640 1 -- 172.21.15.83:0/995380910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8b40aedd0 msgr2=0x7fe8b41596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:05.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.994+0000 7fe8bb643640 1 --2- 172.21.15.83:0/995380910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8b40aedd0 0x7fe8b41596f0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fe8a400c780 tx=0x7fe8a400cc40 comp rx=0 tx=0).stop 2024-04-17T10:48:05.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.994+0000 7fe8bb643640 1 -- 172.21.15.83:0/995380910 shutdown_connections 2024-04-17T10:48:05.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.994+0000 7fe8bb643640 1 --2- 172.21.15.83:0/995380910 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe898066f60 0x7fe898069420 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:05.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.994+0000 7fe8bb643640 1 --2- 172.21.15.83:0/995380910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8b415db50 0x7fe8b41578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:05.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.994+0000 7fe8bb643640 1 --2- 172.21.15.83:0/995380910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8b40aedd0 0x7fe8b41596f0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:05.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.994+0000 7fe8bb643640 1 --2- 172.21.15.83:0/995380910 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8b406dcb0 0x7fe8b40c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:05.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.994+0000 7fe8bb643640 1 -- 172.21.15.83:0/995380910 >> 172.21.15.83:0/995380910 conn(0x7fe8b406abe0 msgr2=0x7fe8b40afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:05.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.994+0000 7fe8bb643640 1 -- 172.21.15.83:0/995380910 shutdown_connections 2024-04-17T10:48:05.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:05.994+0000 7fe8bb643640 1 -- 172.21.15.83:0/995380910 wait complete. 2024-04-17T10:48:06.006 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused'} 2024-04-17T10:48:07.007 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:48:07.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.138+0000 7fad42a91640 1 Processor -- start 2024-04-17T10:48:07.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.138+0000 7fad42a91640 1 -- start start 2024-04-17T10:48:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.138+0000 7fad42a91640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad3c15ba30 0x7fad3c072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.138+0000 7fad42a91640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fad3c0cfbe0 0x7fad3c06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.138+0000 7fad42a91640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad3c06e320 con 0x7fad3c15c960 2024-04-17T10:48:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.138+0000 7fad42a91640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad3c071ea0 con 0x7fad3c0cfbe0 2024-04-17T10:48:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.138+0000 7fad42a91640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c06e040 con 0x7fad3c15ba30 2024-04-17T10:48:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.138+0000 7fad4102e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fad3c0cfbe0 0x7fad3c06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.138+0000 7fad3bfff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fad3c15c960 0x7fad3c0cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38144/0 (socket says 172.21.15.83:38144) 2024-04-17T10:48:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.138+0000 7fad3bfff640 1 -- 172.21.15.83:0/2258033789 learned_addr learned my addr 172.21.15.83:0/2258033789 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.138+0000 7fad4082d640 1 --2- 172.21.15.83:0/2258033789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad3c15ba30 0x7fad3c072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.138+0000 7fad4102e640 1 -- 172.21.15.83:0/2258033789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad3c15ba30 msgr2=0x7fad3c072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.138+0000 7fad4102e640 1 --2- 172.21.15.83:0/2258033789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad3c15ba30 0x7fad3c072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.138+0000 7fad4102e640 1 -- 172.21.15.83:0/2258033789 >> v1:172.21.15.83:6789/0 conn(0x7fad3c15c960 legacy=0x7fad3c0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.138+0000 7fad3b7fe640 1 -- 172.21.15.83:0/2258033789 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2277770016 0 0) 0x7fad3c06e320 con 0x7fad3c15c960 2024-04-17T10:48:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.138+0000 7fad4102e640 1 -- 172.21.15.83:0/2258033789 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad3c071850 con 0x7fad3c0cfbe0 2024-04-17T10:48:07.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.142+0000 7fad4102e640 1 --2- 172.21.15.83:0/2258033789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad3c0cfbe0 0x7fad3c06c490 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fad34009d90 tx=0x7fad3401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6756c727c7007a29 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:07.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.142+0000 7fad3b7fe640 1 -- 172.21.15.83:0/2258033789 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad340b2070 con 0x7fad3c0cfbe0 2024-04-17T10:48:07.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.142+0000 7fad3b7fe640 1 -- 172.21.15.83:0/2258033789 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad340a57f0 con 0x7fad3c0cfbe0 2024-04-17T10:48:07.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.142+0000 7fad3b7fe640 1 -- 172.21.15.83:0/2258033789 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad340a5bc0 con 0x7fad3c0cfbe0 2024-04-17T10:48:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.142+0000 7fad42a91640 1 -- 172.21.15.83:0/2258033789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad3c0cfbe0 msgr2=0x7fad3c06c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.142+0000 7fad42a91640 1 --2- 172.21.15.83:0/2258033789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad3c0cfbe0 0x7fad3c06c490 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fad34009d90 tx=0x7fad3401c8e0 comp rx=0 tx=0).stop 2024-04-17T10:48:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.142+0000 7fad42a91640 1 -- 172.21.15.83:0/2258033789 shutdown_connections 2024-04-17T10:48:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.142+0000 7fad42a91640 1 --2- 172.21.15.83:0/2258033789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad3c0cfbe0 0x7fad3c06c490 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.142+0000 7fad42a91640 1 --2- 172.21.15.83:0/2258033789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad3c15ba30 0x7fad3c072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.142+0000 7fad42a91640 1 -- 172.21.15.83:0/2258033789 >> 172.21.15.83:0/2258033789 conn(0x7fad3c06abe0 msgr2=0x7fad3c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.142+0000 7fad42a91640 1 -- 172.21.15.83:0/2258033789 shutdown_connections 2024-04-17T10:48:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.142+0000 7fad42a91640 1 -- 172.21.15.83:0/2258033789 wait complete. 2024-04-17T10:48:07.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.142+0000 7fad42a91640 1 Processor -- start 2024-04-17T10:48:07.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.142+0000 7fad42a91640 1 -- start start 2024-04-17T10:48:07.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.142+0000 7fad42a91640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad3c0cfbe0 0x7fad3c151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:07.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.142+0000 7fad42a91640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad3c15ba30 0x7fad3c13a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:07.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.142+0000 7fad42a91640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad3c15c960 0x7fad3c0ce800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:07.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.142+0000 7fad42a91640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c06de20 con 0x7fad3c15ba30 2024-04-17T10:48:07.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.142+0000 7fad42a91640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c06dca0 con 0x7fad3c0cfbe0 2024-04-17T10:48:07.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.142+0000 7fad42a91640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c168520 con 0x7fad3c15c960 2024-04-17T10:48:07.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.142+0000 7fad3bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad3c15ba30 0x7fad3c13a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:07.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.142+0000 7fad3bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad3c15ba30 0x7fad3c13a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39602/0 (socket says 172.21.15.83:39602) 2024-04-17T10:48:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.142+0000 7fad3bfff640 1 -- 172.21.15.83:0/2905455472 learned_addr learned my addr 172.21.15.83:0/2905455472 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.142+0000 7fad4102e640 1 --2- 172.21.15.83:0/2905455472 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad3c15c960 0x7fad3c0ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.142+0000 7fad3bfff640 1 -- 172.21.15.83:0/2905455472 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad3c15c960 msgr2=0x7fad3c0ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.142+0000 7fad4082d640 1 --2- 172.21.15.83:0/2905455472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad3c0cfbe0 0x7fad3c151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.142+0000 7fad3bfff640 1 --2- 172.21.15.83:0/2905455472 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad3c15c960 0x7fad3c0ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.142+0000 7fad3bfff640 1 -- 172.21.15.83:0/2905455472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad3c0cfbe0 msgr2=0x7fad3c151320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.142+0000 7fad3bfff640 1 --2- 172.21.15.83:0/2905455472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad3c0cfbe0 0x7fad3c151320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.142+0000 7fad3bfff640 1 -- 172.21.15.83:0/2905455472 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad3c150e20 con 0x7fad3c15ba30 2024-04-17T10:48:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.142+0000 7fad4102e640 1 --2- 172.21.15.83:0/2905455472 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad3c15c960 0x7fad3c0ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.146+0000 7fad3bfff640 1 --2- 172.21.15.83:0/2905455472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad3c15ba30 0x7fad3c13a2d0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fad30007fd0 tx=0x7fad3000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.146+0000 7fad397fa640 1 -- 172.21.15.83:0/2905455472 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad30017070 con 0x7fad3c15ba30 2024-04-17T10:48:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.146+0000 7fad397fa640 1 -- 172.21.15.83:0/2905455472 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad30005d10 con 0x7fad3c15ba30 2024-04-17T10:48:07.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.146+0000 7fad42a91640 1 -- 172.21.15.83:0/2905455472 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad3c0afcf0 con 0x7fad3c15ba30 2024-04-17T10:48:07.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.146+0000 7fad397fa640 1 -- 172.21.15.83:0/2905455472 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad30007310 con 0x7fad3c15ba30 2024-04-17T10:48:07.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.146+0000 7fad42a91640 1 -- 172.21.15.83:0/2905455472 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad3c0b02b0 con 0x7fad3c15ba30 2024-04-17T10:48:07.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.146+0000 7fad42a91640 1 -- 172.21.15.83:0/2905455472 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad3c074f90 con 0x7fad3c15ba30 2024-04-17T10:48:07.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.146+0000 7fad397fa640 1 -- 172.21.15.83:0/2905455472 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fad3001a070 con 0x7fad3c15ba30 2024-04-17T10:48:07.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.146+0000 7fad397fa640 1 --2- 172.21.15.83:0/2905455472 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fad28066f60 0x7fad28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:07.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.146+0000 7fad397fa640 1 -- 172.21.15.83:0/2905455472 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad30084450 con 0x7fad3c15ba30 2024-04-17T10:48:07.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.146+0000 7fad4082d640 1 --2- 172.21.15.83:0/2905455472 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fad28066f60 0x7fad28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:07.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.146+0000 7fad4082d640 1 --2- 172.21.15.83:0/2905455472 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fad28066f60 0x7fad28069420 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fad3c0b0f00 tx=0x7fad2c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:07.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.150+0000 7fad397fa640 1 -- 172.21.15.83:0/2905455472 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fad30055460 con 0x7fad3c15ba30 2024-04-17T10:48:07.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.326+0000 7fad42a91640 1 -- 172.21.15.83:0/2905455472 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad3c15acb0 con 0x7fad3c15ba30 2024-04-17T10:48:07.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.330+0000 7fad397fa640 1 -- 172.21.15.83:0/2905455472 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fad30054d60 con 0x7fad3c15ba30 2024-04-17T10:48:07.325 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:07.325 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/437268957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":437268957},{"type":"v1","addr":"172.21.15.167:6833","nonce":437268957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":6253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3214640674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3214640674},{"type":"v1","addr":"172.21.15.167:6835","nonce":3214640674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":8046,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/715498323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":715498323},{"type":"v1","addr":"172.21.15.83:6835","nonce":715498323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:47:29.385103+0000","modified":"2024-04-17T10:47:31.389088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8049":{"gid":8049,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2353406669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":2353406669},{"type":"v1","addr":"172.21.15.83:6837","nonce":2353406669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-17T10:48:07.325 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 62 2024-04-17T10:48:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.330+0000 7fad42a91640 1 -- 172.21.15.83:0/2905455472 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fad28066f60 msgr2=0x7fad28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.330+0000 7fad42a91640 1 --2- 172.21.15.83:0/2905455472 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fad28066f60 0x7fad28069420 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fad3c0b0f00 tx=0x7fad2c008040 comp rx=0 tx=0).stop 2024-04-17T10:48:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.330+0000 7fad42a91640 1 -- 172.21.15.83:0/2905455472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad3c15ba30 msgr2=0x7fad3c13a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.330+0000 7fad42a91640 1 --2- 172.21.15.83:0/2905455472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad3c15ba30 0x7fad3c13a2d0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fad30007fd0 tx=0x7fad3000c8e0 comp rx=0 tx=0).stop 2024-04-17T10:48:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.330+0000 7fad42a91640 1 -- 172.21.15.83:0/2905455472 shutdown_connections 2024-04-17T10:48:07.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.330+0000 7fad42a91640 1 --2- 172.21.15.83:0/2905455472 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fad28066f60 0x7fad28069420 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:07.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.330+0000 7fad42a91640 1 --2- 172.21.15.83:0/2905455472 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad3c15c960 0x7fad3c0ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:07.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.330+0000 7fad42a91640 1 --2- 172.21.15.83:0/2905455472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad3c15ba30 0x7fad3c13a2d0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:07.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.330+0000 7fad42a91640 1 --2- 172.21.15.83:0/2905455472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad3c0cfbe0 0x7fad3c151320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:07.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.330+0000 7fad42a91640 1 -- 172.21.15.83:0/2905455472 >> 172.21.15.83:0/2905455472 conn(0x7fad3c06abe0 msgr2=0x7fad3c15e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:07.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.330+0000 7fad42a91640 1 -- 172.21.15.83:0/2905455472 shutdown_connections 2024-04-17T10:48:07.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.330+0000 7fad42a91640 1 -- 172.21.15.83:0/2905455472 wait complete. 2024-04-17T10:48:07.343 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-17T10:48:07.343 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-17T10:48:07.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.462+0000 7f529aaf5640 1 Processor -- start 2024-04-17T10:48:07.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.462+0000 7f529aaf5640 1 -- start start 2024-04-17T10:48:07.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.462+0000 7f529aaf5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f529415ba30 0x7f5294072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:07.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.462+0000 7f529aaf5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f529415c960 0x7f52940cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:07.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.462+0000 7f529aaf5640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52940cfea0 con 0x7f52940cfa10 2024-04-17T10:48:07.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.462+0000 7f529aaf5640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5294071ea0 con 0x7f529415c960 2024-04-17T10:48:07.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.462+0000 7f529aaf5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f529406e320 con 0x7f529415ba30 2024-04-17T10:48:07.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.462+0000 7f5293fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f529415c960 0x7f52940cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:07.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.462+0000 7f5293fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f529415c960 0x7f52940cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39614/0 (socket says 172.21.15.83:39614) 2024-04-17T10:48:07.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.462+0000 7f5293fff640 1 -- 172.21.15.83:0/2183925373 learned_addr learned my addr 172.21.15.83:0/2183925373 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:07.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.462+0000 7f5293fff640 1 -- 172.21.15.83:0/2183925373 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f529415ba30 msgr2=0x7f5294072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:07.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.462+0000 7f5293fff640 1 --2- 172.21.15.83:0/2183925373 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f529415ba30 0x7f5294072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:07.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.462+0000 7f5293fff640 1 -- 172.21.15.83:0/2183925373 >> v1:172.21.15.167:6789/0 conn(0x7f52940cfa10 legacy=0x7f529406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:07.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.462+0000 7f5293fff640 1 -- 172.21.15.83:0/2183925373 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529406e040 con 0x7f529415c960 2024-04-17T10:48:07.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.466+0000 7f5293fff640 1 --2- 172.21.15.83:0/2183925373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529415c960 0x7f52940cf4d0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f5288009870 tx=0x7f528801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be816c6a4be6a6e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:07.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.466+0000 7f52937fe640 1 -- 172.21.15.83:0/2183925373 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52880b0070 con 0x7f529415c960 2024-04-17T10:48:07.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.466+0000 7f52937fe640 1 -- 172.21.15.83:0/2183925373 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52880ab600 con 0x7f529415c960 2024-04-17T10:48:07.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.466+0000 7f52937fe640 1 -- 172.21.15.83:0/2183925373 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52880ab9d0 con 0x7f529415c960 2024-04-17T10:48:07.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.466+0000 7f529aaf5640 1 -- 172.21.15.83:0/2183925373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529415c960 msgr2=0x7f52940cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:07.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.466+0000 7f529aaf5640 1 --2- 172.21.15.83:0/2183925373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529415c960 0x7f52940cf4d0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f5288009870 tx=0x7f528801c490 comp rx=0 tx=0).stop 2024-04-17T10:48:07.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.466+0000 7f529aaf5640 1 -- 172.21.15.83:0/2183925373 shutdown_connections 2024-04-17T10:48:07.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.466+0000 7f529aaf5640 1 --2- 172.21.15.83:0/2183925373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529415c960 0x7f52940cf4d0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:07.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.466+0000 7f529aaf5640 1 --2- 172.21.15.83:0/2183925373 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f529415ba30 0x7f5294072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:07.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.466+0000 7f529aaf5640 1 -- 172.21.15.83:0/2183925373 >> 172.21.15.83:0/2183925373 conn(0x7f529406abe0 msgr2=0x7f529406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:07.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.466+0000 7f529aaf5640 1 -- 172.21.15.83:0/2183925373 shutdown_connections 2024-04-17T10:48:07.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.466+0000 7f529aaf5640 1 -- 172.21.15.83:0/2183925373 wait complete. 2024-04-17T10:48:07.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.466+0000 7f529aaf5640 1 Processor -- start 2024-04-17T10:48:07.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.466+0000 7f529aaf5640 1 -- start start 2024-04-17T10:48:07.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.470+0000 7f529aaf5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52940cfa10 0x7f529414c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:07.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.470+0000 7f529aaf5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529415ba30 0x7f5294150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:07.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.470+0000 7f529aaf5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f529415c960 0x7f52941513a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:07.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.470+0000 7f529aaf5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529406dca0 con 0x7f529415ba30 2024-04-17T10:48:07.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.470+0000 7f529aaf5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52940aeae0 con 0x7f529415c960 2024-04-17T10:48:07.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.470+0000 7f529aaf5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f529406de20 con 0x7f52940cfa10 2024-04-17T10:48:07.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.470+0000 7f5293fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529415ba30 0x7f5294150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:07.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.470+0000 7f5293fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529415ba30 0x7f5294150e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39616/0 (socket says 172.21.15.83:39616) 2024-04-17T10:48:07.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.470+0000 7f5293fff640 1 -- 172.21.15.83:0/3642223890 learned_addr learned my addr 172.21.15.83:0/3642223890 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:07.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.470+0000 7f5298891640 1 --2- 172.21.15.83:0/3642223890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52940cfa10 0x7f529414c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:07.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.470+0000 7f5299092640 1 --2- 172.21.15.83:0/3642223890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f529415c960 0x7f52941513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:07.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.470+0000 7f5293fff640 1 -- 172.21.15.83:0/3642223890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52940cfa10 msgr2=0x7f529414c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:07.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.470+0000 7f5293fff640 1 --2- 172.21.15.83:0/3642223890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52940cfa10 0x7f529414c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:07.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.470+0000 7f5293fff640 1 -- 172.21.15.83:0/3642223890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f529415c960 msgr2=0x7f52941513a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:07.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.470+0000 7f5293fff640 1 --2- 172.21.15.83:0/3642223890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f529415c960 0x7f52941513a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:07.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.470+0000 7f5293fff640 1 -- 172.21.15.83:0/3642223890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529414e000 con 0x7f529415ba30 2024-04-17T10:48:07.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.470+0000 7f5293fff640 1 --2- 172.21.15.83:0/3642223890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529415ba30 0x7f5294150e60 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f528801cd30 tx=0x7f529406d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:07.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.470+0000 7f52917fa640 1 -- 172.21.15.83:0/3642223890 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52880bb070 con 0x7f529415ba30 2024-04-17T10:48:07.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.470+0000 7f52917fa640 1 -- 172.21.15.83:0/3642223890 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52880b6660 con 0x7f529415ba30 2024-04-17T10:48:07.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.470+0000 7f529aaf5640 1 -- 172.21.15.83:0/3642223890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f529414e2e0 con 0x7f529415ba30 2024-04-17T10:48:07.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.470+0000 7f52917fa640 1 -- 172.21.15.83:0/3642223890 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52880b0040 con 0x7f529415ba30 2024-04-17T10:48:07.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.470+0000 7f529aaf5640 1 -- 172.21.15.83:0/3642223890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52941594a0 con 0x7f529415ba30 2024-04-17T10:48:07.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.470+0000 7f529aaf5640 1 -- 172.21.15.83:0/3642223890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5294074f90 con 0x7f529415ba30 2024-04-17T10:48:07.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.470+0000 7f52917fa640 1 -- 172.21.15.83:0/3642223890 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f52880be070 con 0x7f529415ba30 2024-04-17T10:48:07.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.470+0000 7f52917fa640 1 --2- 172.21.15.83:0/3642223890 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5280066f60 0x7f5280069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:07.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.470+0000 7f52917fa640 1 -- 172.21.15.83:0/3642223890 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f528811ed00 con 0x7f529415ba30 2024-04-17T10:48:07.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.470+0000 7f5298891640 1 --2- 172.21.15.83:0/3642223890 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5280066f60 0x7f5280069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:07.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.470+0000 7f5298891640 1 --2- 172.21.15.83:0/3642223890 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5280066f60 0x7f5280069420 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5284006fd0 tx=0x7f5284008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:07.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.474+0000 7f52917fa640 1 -- 172.21.15.83:0/3642223890 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5288016110 con 0x7f529415ba30 2024-04-17T10:48:07.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.674+0000 7f529aaf5640 1 -- 172.21.15.83:0/3642223890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f529406b1d0 con 0x7f529415ba30 2024-04-17T10:48:07.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.674+0000 7f52917fa640 1 -- 172.21.15.83:0/3642223890 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1575 (secure 0 0 0) 0x7f52880efe80 con 0x7f529415ba30 2024-04-17T10:48:07.671 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:07.671 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":620,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":103,"num_osds":8,"num_up_osds":8,"osd_up_since":1713350279,"num_in_osds":8,"osd_in_since":1713350270,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077901059,"bytes_used":2700042240,"bytes_avail":770394071040,"bytes_total":773094113280,"write_bytes_sec":48934107,"read_op_per_sec":0,"write_op_per_sec":50},"fsmap":{"epoch":62,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8049}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2024-04-17T10:48:06.358835+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-17T10:48:07.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.678+0000 7f529aaf5640 1 -- 172.21.15.83:0/3642223890 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5280066f60 msgr2=0x7f5280069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:07.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.678+0000 7f529aaf5640 1 --2- 172.21.15.83:0/3642223890 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5280066f60 0x7f5280069420 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5284006fd0 tx=0x7f5284008040 comp rx=0 tx=0).stop 2024-04-17T10:48:07.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.678+0000 7f529aaf5640 1 -- 172.21.15.83:0/3642223890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529415ba30 msgr2=0x7f5294150e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:07.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.678+0000 7f529aaf5640 1 --2- 172.21.15.83:0/3642223890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529415ba30 0x7f5294150e60 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f528801cd30 tx=0x7f529406d3b0 comp rx=0 tx=0).stop 2024-04-17T10:48:07.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.678+0000 7f529aaf5640 1 -- 172.21.15.83:0/3642223890 shutdown_connections 2024-04-17T10:48:07.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.678+0000 7f529aaf5640 1 --2- 172.21.15.83:0/3642223890 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5280066f60 0x7f5280069420 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:07.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.678+0000 7f529aaf5640 1 --2- 172.21.15.83:0/3642223890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f529415c960 0x7f52941513a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:07.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.678+0000 7f529aaf5640 1 --2- 172.21.15.83:0/3642223890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529415ba30 0x7f5294150e60 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:07.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.678+0000 7f529aaf5640 1 --2- 172.21.15.83:0/3642223890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52940cfa10 0x7f529414c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:07.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.678+0000 7f529aaf5640 1 -- 172.21.15.83:0/3642223890 >> 172.21.15.83:0/3642223890 conn(0x7f529406abe0 msgr2=0x7f5294152260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:07.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.678+0000 7f529aaf5640 1 -- 172.21.15.83:0/3642223890 shutdown_connections 2024-04-17T10:48:07.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.678+0000 7f529aaf5640 1 -- 172.21.15.83:0/3642223890 wait complete. 2024-04-17T10:48:07.688 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-17T10:48:07.688 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T10:48:07.688 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:48:07.738 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:48:07.791 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T10:48:07.791 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:48:07.791 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:48:07.797 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:48:07.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.966+0000 7f7a945ef640 1 Processor -- start 2024-04-17T10:48:07.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.966+0000 7f7a945ef640 1 -- start start 2024-04-17T10:48:07.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.966+0000 7f7a945ef640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7a8c0ce980 0x7f7a8c0b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:07.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.966+0000 7f7a945ef640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7a8c0cee00 0x7f7a8c0b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:07.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.966+0000 7f7a945ef640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a8c162a00 con 0x7f7a8c15dfa0 2024-04-17T10:48:07.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.966+0000 7f7a945ef640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a8c161e80 con 0x7f7a8c0ce980 2024-04-17T10:48:07.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.966+0000 7f7a945ef640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a8c162160 con 0x7f7a8c0cee00 2024-04-17T10:48:07.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.966+0000 7f7a91b8a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7a8c0ce980 0x7f7a8c0b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:07.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.966+0000 7f7a91b8a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7a8c0ce980 0x7f7a8c0b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39630/0 (socket says 172.21.15.83:39630) 2024-04-17T10:48:07.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.966+0000 7f7a91b8a640 1 -- 172.21.15.83:0/57518537 learned_addr learned my addr 172.21.15.83:0/57518537 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:07.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.966+0000 7f7a91b8a640 1 -- 172.21.15.83:0/57518537 >> v2:172.21.15.167:3300/0 conn(0x7f7a8c0cee00 msgr2=0x7f7a8c0b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:07.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.966+0000 7f7a92b8c640 1 --2- 172.21.15.83:0/57518537 >> v2:172.21.15.167:3300/0 conn(0x7f7a8c0cee00 0x7f7a8c0b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:07.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.970+0000 7f7a91b8a640 1 --2- 172.21.15.83:0/57518537 >> v2:172.21.15.167:3300/0 conn(0x7f7a8c0cee00 0x7f7a8c0b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:07.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.970+0000 7f7a91b8a640 1 -- 172.21.15.83:0/57518537 >> v1:172.21.15.167:6789/0 conn(0x7f7a8c15dfa0 legacy=0x7f7a8c0b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:07.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.970+0000 7f7a91b8a640 1 -- 172.21.15.83:0/57518537 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a8c162440 con 0x7f7a8c0ce980 2024-04-17T10:48:07.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.970+0000 7f7a92b8c640 1 --2- 172.21.15.83:0/57518537 >> v2:172.21.15.167:3300/0 conn(0x7f7a8c0cee00 0x7f7a8c0b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:07.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.970+0000 7f7a91b8a640 1 --2- 172.21.15.83:0/57518537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a8c0ce980 0x7f7a8c0b5430 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c067820 tx=0x7f7a7c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420496c9ee7dfd0f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:07.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.970+0000 7f7a91389640 1 -- 172.21.15.83:0/57518537 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c0b1070 con 0x7f7a8c0ce980 2024-04-17T10:48:07.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.970+0000 7f7a91389640 1 -- 172.21.15.83:0/57518537 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7c0ac620 con 0x7f7a8c0ce980 2024-04-17T10:48:07.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.970+0000 7f7a91389640 1 -- 172.21.15.83:0/57518537 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c0ac9f0 con 0x7f7a8c0ce980 2024-04-17T10:48:07.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.970+0000 7f7a945ef640 1 -- 172.21.15.83:0/57518537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a8c0ce980 msgr2=0x7f7a8c0b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:07.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.970+0000 7f7a945ef640 1 --2- 172.21.15.83:0/57518537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a8c0ce980 0x7f7a8c0b5430 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c067820 tx=0x7f7a7c01c420 comp rx=0 tx=0).stop 2024-04-17T10:48:07.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.970+0000 7f7a945ef640 1 -- 172.21.15.83:0/57518537 shutdown_connections 2024-04-17T10:48:07.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.970+0000 7f7a945ef640 1 --2- 172.21.15.83:0/57518537 >> v2:172.21.15.167:3300/0 conn(0x7f7a8c0cee00 0x7f7a8c0b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:07.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.970+0000 7f7a945ef640 1 --2- 172.21.15.83:0/57518537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a8c0ce980 0x7f7a8c0b5430 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:07.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.970+0000 7f7a945ef640 1 -- 172.21.15.83:0/57518537 >> 172.21.15.83:0/57518537 conn(0x7f7a8c0d0600 msgr2=0x7f7a8c0d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:07.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.970+0000 7f7a945ef640 1 -- 172.21.15.83:0/57518537 shutdown_connections 2024-04-17T10:48:07.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.970+0000 7f7a945ef640 1 -- 172.21.15.83:0/57518537 wait complete. 2024-04-17T10:48:07.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.974+0000 7f7a945ef640 1 Processor -- start 2024-04-17T10:48:07.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.974+0000 7f7a945ef640 1 -- start start 2024-04-17T10:48:07.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.974+0000 7f7a945ef640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a8c0ce980 0x7f7a8c15bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:07.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.974+0000 7f7a945ef640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a8c0cee00 0x7f7a8c15c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:07.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.974+0000 7f7a945ef640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a8c15dfa0 0x7f7a8c0cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:07.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.974+0000 7f7a945ef640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c0d26c0 con 0x7f7a8c0cee00 2024-04-17T10:48:07.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.974+0000 7f7a945ef640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c0d2540 con 0x7f7a8c15dfa0 2024-04-17T10:48:07.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.974+0000 7f7a945ef640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c0d2840 con 0x7f7a8c0ce980 2024-04-17T10:48:07.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.974+0000 7f7a91b8a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a8c0cee00 0x7f7a8c15c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:07.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.974+0000 7f7a91b8a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a8c0cee00 0x7f7a8c15c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39638/0 (socket says 172.21.15.83:39638) 2024-04-17T10:48:07.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.974+0000 7f7a91b8a640 1 -- 172.21.15.83:0/1845123861 learned_addr learned my addr 172.21.15.83:0/1845123861 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:07.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.974+0000 7f7a92b8c640 1 --2- 172.21.15.83:0/1845123861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a8c15dfa0 0x7f7a8c0cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:07.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.974+0000 7f7a91b8a640 1 -- 172.21.15.83:0/1845123861 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a8c0ce980 msgr2=0x7f7a8c15bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:07.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.974+0000 7f7a91b8a640 1 --2- 172.21.15.83:0/1845123861 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a8c0ce980 0x7f7a8c15bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:07.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.974+0000 7f7a91b8a640 1 -- 172.21.15.83:0/1845123861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a8c15dfa0 msgr2=0x7f7a8c0cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:07.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.974+0000 7f7a91b8a640 1 --2- 172.21.15.83:0/1845123861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a8c15dfa0 0x7f7a8c0cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:07.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.974+0000 7f7a91b8a640 1 -- 172.21.15.83:0/1845123861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a8c15ce20 con 0x7f7a8c0cee00 2024-04-17T10:48:07.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.974+0000 7f7a91b8a640 1 --2- 172.21.15.83:0/1845123861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a8c0cee00 0x7f7a8c15c2e0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0af040 tx=0x7f7a7c01ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:07.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.974+0000 7f7a82ffd640 1 -- 172.21.15.83:0/1845123861 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c0bb020 con 0x7f7a8c0cee00 2024-04-17T10:48:07.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.974+0000 7f7a82ffd640 1 -- 172.21.15.83:0/1845123861 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7c0b7640 con 0x7f7a8c0cee00 2024-04-17T10:48:07.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.974+0000 7f7a82ffd640 1 -- 172.21.15.83:0/1845123861 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c0b1040 con 0x7f7a8c0cee00 2024-04-17T10:48:07.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.974+0000 7f7a945ef640 1 -- 172.21.15.83:0/1845123861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a8c15d130 con 0x7f7a8c0cee00 2024-04-17T10:48:07.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.974+0000 7f7a945ef640 1 -- 172.21.15.83:0/1845123861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a8c15b880 con 0x7f7a8c0cee00 2024-04-17T10:48:07.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.974+0000 7f7a945ef640 1 -- 172.21.15.83:0/1845123861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a8c074f90 con 0x7f7a8c0cee00 2024-04-17T10:48:07.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.974+0000 7f7a82ffd640 1 -- 172.21.15.83:0/1845123861 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7a7c0be020 con 0x7f7a8c0cee00 2024-04-17T10:48:07.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.974+0000 7f7a82ffd640 1 --2- 172.21.15.83:0/1845123861 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7a70066f60 0x7f7a70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:07.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.974+0000 7f7a82ffd640 1 -- 172.21.15.83:0/1845123861 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a7c11f440 con 0x7f7a8c0cee00 2024-04-17T10:48:07.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.974+0000 7f7a9238b640 1 --2- 172.21.15.83:0/1845123861 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7a70066f60 0x7f7a70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:07.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.978+0000 7f7a9238b640 1 --2- 172.21.15.83:0/1845123861 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7a70066f60 0x7f7a70069420 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f7a84007fd0 tx=0x7f7a84007d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:07.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:07.982+0000 7f7a82ffd640 1 -- 172.21.15.83:0/1845123861 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7a7c0c5030 con 0x7f7a8c0cee00 2024-04-17T10:48:08.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.138+0000 7f7a945ef640 1 -- 172.21.15.83:0/1845123861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a8c15d460 con 0x7f7a8c0cee00 2024-04-17T10:48:08.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.142+0000 7f7a82ffd640 1 -- 172.21.15.83:0/1845123861 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16547 (secure 0 0 0) 0x7f7a8c15d130 con 0x7f7a8c0cee00 2024-04-17T10:48:08.139 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:08.139 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":103,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:47:31.018926+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:47:27.619432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-17T10:47:28.379472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:48:08.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.142+0000 7f7a945ef640 1 -- 172.21.15.83:0/1845123861 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7a70066f60 msgr2=0x7f7a70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:08.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.142+0000 7f7a945ef640 1 --2- 172.21.15.83:0/1845123861 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7a70066f60 0x7f7a70069420 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f7a84007fd0 tx=0x7f7a84007d40 comp rx=0 tx=0).stop 2024-04-17T10:48:08.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.142+0000 7f7a945ef640 1 -- 172.21.15.83:0/1845123861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a8c0cee00 msgr2=0x7f7a8c15c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:08.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.142+0000 7f7a945ef640 1 --2- 172.21.15.83:0/1845123861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a8c0cee00 0x7f7a8c15c2e0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0af040 tx=0x7f7a7c01ac60 comp rx=0 tx=0).stop 2024-04-17T10:48:08.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.146+0000 7f7a945ef640 1 -- 172.21.15.83:0/1845123861 shutdown_connections 2024-04-17T10:48:08.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.146+0000 7f7a945ef640 1 --2- 172.21.15.83:0/1845123861 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7a70066f60 0x7f7a70069420 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:08.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.146+0000 7f7a945ef640 1 --2- 172.21.15.83:0/1845123861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a8c15dfa0 0x7f7a8c0cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:08.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.146+0000 7f7a945ef640 1 --2- 172.21.15.83:0/1845123861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a8c0cee00 0x7f7a8c15c2e0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:08.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.146+0000 7f7a945ef640 1 --2- 172.21.15.83:0/1845123861 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a8c0ce980 0x7f7a8c15bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:08.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.146+0000 7f7a945ef640 1 -- 172.21.15.83:0/1845123861 >> 172.21.15.83:0/1845123861 conn(0x7f7a8c0d0600 msgr2=0x7f7a8c150c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:08.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.146+0000 7f7a945ef640 1 -- 172.21.15.83:0/1845123861 shutdown_connections 2024-04-17T10:48:08.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.146+0000 7f7a945ef640 1 -- 172.21.15.83:0/1845123861 wait complete. 2024-04-17T10:48:08.158 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:47:27.619432+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:47:28.379472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:48:08.159 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:48:08.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7facd2d03640 1 Processor -- start 2024-04-17T10:48:08.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7facd2d03640 1 -- start start 2024-04-17T10:48:08.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7facd2d03640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faccc06b520 0x7faccc0b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:08.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7facd2d03640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faccc0b4040 0x7faccc06cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:08.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7facd2d03640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faccc15bd70 con 0x7faccc154de0 2024-04-17T10:48:08.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7facd2d03640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faccc0b4870 con 0x7faccc06b520 2024-04-17T10:48:08.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7facd2d03640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faccc15bf60 con 0x7faccc0b4040 2024-04-17T10:48:08.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7facd0a9f640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7faccc154de0 0x7faccc06ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38154/0 (socket says 172.21.15.83:38154) 2024-04-17T10:48:08.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7facd0a9f640 1 -- 172.21.15.83:0/1627301713 learned_addr learned my addr 172.21.15.83:0/1627301713 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:08.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7facd12a0640 1 --2- 172.21.15.83:0/1627301713 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faccc0b4040 0x7faccc06cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:08.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7faccbfff640 1 --2- 172.21.15.83:0/1627301713 >> v2:172.21.15.167:3300/0 conn(0x7faccc06b520 0x7faccc0b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:08.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7faccb7fe640 1 -- 172.21.15.83:0/1627301713 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 124496836 0 0) 0x7faccc15bd70 con 0x7faccc154de0 2024-04-17T10:48:08.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7faccb7fe640 1 -- 172.21.15.83:0/1627301713 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facbc005740 con 0x7faccc154de0 2024-04-17T10:48:08.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7faccb7fe640 1 -- 172.21.15.83:0/1627301713 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1038324871 0 0) 0x7facbc005740 con 0x7faccc154de0 2024-04-17T10:48:08.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7faccb7fe640 1 -- 172.21.15.83:0/1627301713 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faccc0b4040 msgr2=0x7faccc06cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:08.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7faccb7fe640 1 --2- 172.21.15.83:0/1627301713 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faccc0b4040 0x7faccc06cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:08.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7faccb7fe640 1 -- 172.21.15.83:0/1627301713 >> v2:172.21.15.167:3300/0 conn(0x7faccc06b520 msgr2=0x7faccc0b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:08.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7faccb7fe640 1 --2- 172.21.15.83:0/1627301713 >> v2:172.21.15.167:3300/0 conn(0x7faccc06b520 0x7faccc0b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:08.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7faccb7fe640 1 -- 172.21.15.83:0/1627301713 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faccc15afb0 con 0x7faccc154de0 2024-04-17T10:48:08.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7faccb7fe640 1 -- 172.21.15.83:0/1627301713 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7facc00016a0 con 0x7faccc154de0 2024-04-17T10:48:08.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7faccb7fe640 1 -- 172.21.15.83:0/1627301713 >> v1:172.21.15.83:6789/0 conn(0x7faccc154de0 legacy=0x7faccc06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:08.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7faccb7fe640 1 --2- 172.21.15.83:0/1627301713 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faccc0b4040 0x7facbc005c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:08.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7faccb7fe640 1 --2- 172.21.15.83:0/1627301713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facbc0061c0 0x7facbc006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:08.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7faccb7fe640 1 --2- 172.21.15.83:0/1627301713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facbc006f60 0x7facbc0073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:08.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7faccb7fe640 1 -- 172.21.15.83:0/1627301713 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc15bf60 con 0x7facbc0061c0 2024-04-17T10:48:08.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7faccb7fe640 1 -- 172.21.15.83:0/1627301713 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facbc00dbe0 con 0x7facbc006f60 2024-04-17T10:48:08.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7faccb7fe640 1 -- 172.21.15.83:0/1627301713 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facbc00dd90 con 0x7faccc0b4040 2024-04-17T10:48:08.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7faccbfff640 1 --2- 172.21.15.83:0/1627301713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facbc0061c0 0x7facbc006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:08.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7faccbfff640 1 -- 172.21.15.83:0/1627301713 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faccc0b4040 msgr2=0x7facbc005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:08.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7faccbfff640 1 --2- 172.21.15.83:0/1627301713 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faccc0b4040 0x7facbc005c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:08.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7faccbfff640 1 -- 172.21.15.83:0/1627301713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facbc006f60 msgr2=0x7facbc0073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:08.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7faccbfff640 1 --2- 172.21.15.83:0/1627301713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facbc006f60 0x7facbc0073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:08.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.290+0000 7faccbfff640 1 -- 172.21.15.83:0/1627301713 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facbc005740 con 0x7facbc0061c0 2024-04-17T10:48:08.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.294+0000 7faccbfff640 1 --2- 172.21.15.83:0/1627301713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facbc0061c0 0x7facbc006a20 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7facb8009ae0 tx=0x7facb801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2540231a9683ad9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:08.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.294+0000 7faccb7fe640 1 -- 172.21.15.83:0/1627301713 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb80b2070 con 0x7facbc0061c0 2024-04-17T10:48:08.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.294+0000 7faccb7fe640 1 -- 172.21.15.83:0/1627301713 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb80a56a0 con 0x7facbc0061c0 2024-04-17T10:48:08.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.294+0000 7faccb7fe640 1 -- 172.21.15.83:0/1627301713 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb80096b0 con 0x7facbc0061c0 2024-04-17T10:48:08.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.294+0000 7facd2d03640 1 -- 172.21.15.83:0/1627301713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facbc0061c0 msgr2=0x7facbc006a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:08.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.294+0000 7facd2d03640 1 --2- 172.21.15.83:0/1627301713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facbc0061c0 0x7facbc006a20 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7facb8009ae0 tx=0x7facb801cab0 comp rx=0 tx=0).stop 2024-04-17T10:48:08.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.294+0000 7facd12a0640 1 -- 172.21.15.83:0/1627301713 reap_dead start 2024-04-17T10:48:08.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.294+0000 7facd2d03640 1 -- 172.21.15.83:0/1627301713 shutdown_connections 2024-04-17T10:48:08.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.294+0000 7facd2d03640 1 -- 172.21.15.83:0/1627301713 >> 172.21.15.83:0/1627301713 conn(0x7faccc0b15b0 msgr2=0x7faccc0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:08.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.294+0000 7facd2d03640 1 -- 172.21.15.83:0/1627301713 shutdown_connections 2024-04-17T10:48:08.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.294+0000 7facd2d03640 1 -- 172.21.15.83:0/1627301713 wait complete. 2024-04-17T10:48:08.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.294+0000 7facd2d03640 1 Processor -- start 2024-04-17T10:48:08.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.294+0000 7facd2d03640 1 -- start start 2024-04-17T10:48:08.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.294+0000 7facd2d03640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facbc0061c0 0x7faccc14e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:08.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.294+0000 7facd2d03640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faccc071e00 0x7faccc0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:08.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.294+0000 7facd2d03640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faccc0add80 0x7faccc1f6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:08.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.294+0000 7facd2d03640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc15b850 con 0x7facbc0061c0 2024-04-17T10:48:08.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.294+0000 7facd2d03640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc15b570 con 0x7faccc071e00 2024-04-17T10:48:08.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.294+0000 7facd2d03640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faccc1551c0 con 0x7faccc0add80 2024-04-17T10:48:08.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.294+0000 7facd0a9f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facbc0061c0 0x7faccc14e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:08.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.294+0000 7facd0a9f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facbc0061c0 0x7faccc14e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39658/0 (socket says 172.21.15.83:39658) 2024-04-17T10:48:08.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.294+0000 7facd0a9f640 1 -- 172.21.15.83:0/3920907132 learned_addr learned my addr 172.21.15.83:0/3920907132 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:08.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.294+0000 7faccbfff640 1 --2- 172.21.15.83:0/3920907132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faccc071e00 0x7faccc0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:08.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.298+0000 7facd12a0640 1 --2- 172.21.15.83:0/3920907132 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faccc0add80 0x7faccc1f6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:08.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.298+0000 7facd0a9f640 1 -- 172.21.15.83:0/3920907132 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faccc0add80 msgr2=0x7faccc1f6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:08.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.298+0000 7facd0a9f640 1 --2- 172.21.15.83:0/3920907132 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faccc0add80 0x7faccc1f6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:08.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.298+0000 7facd0a9f640 1 -- 172.21.15.83:0/3920907132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faccc071e00 msgr2=0x7faccc0ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:08.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.298+0000 7facd0a9f640 1 --2- 172.21.15.83:0/3920907132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faccc071e00 0x7faccc0ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:08.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.298+0000 7facd0a9f640 1 -- 172.21.15.83:0/3920907132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faccc14e670 con 0x7facbc0061c0 2024-04-17T10:48:08.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.298+0000 7faccbfff640 1 --2- 172.21.15.83:0/3920907132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faccc071e00 0x7faccc0ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:48:08.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.298+0000 7facd0a9f640 1 --2- 172.21.15.83:0/3920907132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facbc0061c0 0x7faccc14e130 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7facc000f3c0 tx=0x7facc000f880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:08.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.298+0000 7facc97fa640 1 -- 172.21.15.83:0/3920907132 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc0007dc0 con 0x7facbc0061c0 2024-04-17T10:48:08.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.298+0000 7facc97fa640 1 -- 172.21.15.83:0/3920907132 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc000a0d0 con 0x7facbc0061c0 2024-04-17T10:48:08.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.298+0000 7facd2d03640 1 -- 172.21.15.83:0/3920907132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faccc0cebf0 con 0x7facbc0061c0 2024-04-17T10:48:08.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.298+0000 7facc97fa640 1 -- 172.21.15.83:0/3920907132 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc000a4c0 con 0x7facbc0061c0 2024-04-17T10:48:08.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.298+0000 7facd2d03640 1 -- 172.21.15.83:0/3920907132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faccc14f9b0 con 0x7facbc0061c0 2024-04-17T10:48:08.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.298+0000 7facd2d03640 1 -- 172.21.15.83:0/3920907132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faccc14fbf0 con 0x7facbc0061c0 2024-04-17T10:48:08.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.298+0000 7facc97fa640 1 -- 172.21.15.83:0/3920907132 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7facc0014070 con 0x7facbc0061c0 2024-04-17T10:48:08.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.298+0000 7facc97fa640 1 --2- 172.21.15.83:0/3920907132 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7facb0066f60 0x7facb0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:08.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.298+0000 7faccbfff640 1 --2- 172.21.15.83:0/3920907132 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7facb0066f60 0x7facb0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:08.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.298+0000 7facc97fa640 1 -- 172.21.15.83:0/3920907132 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7facc0087be0 con 0x7facbc0061c0 2024-04-17T10:48:08.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.298+0000 7faccbfff640 1 --2- 172.21.15.83:0/3920907132 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7facb0066f60 0x7facb0069420 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7facb80af000 tx=0x7facb80046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:08.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.302+0000 7facc97fa640 1 -- 172.21.15.83:0/3920907132 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faccc14fbf0 con 0x7facbc0061c0 2024-04-17T10:48:08.302 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:48:08.303 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:48:08.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.458+0000 7facd2d03640 1 -- 172.21.15.83:0/3920907132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faccc0b3280 con 0x7facbc0061c0 2024-04-17T10:48:08.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.458+0000 7facc97fa640 1 -- 172.21.15.83:0/3920907132 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) v1 ==== 93+0+10 (secure 0 0 0) 0x7facc0058b60 con 0x7facbc0061c0 2024-04-17T10:48:08.456 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:48:08.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.462+0000 7facd2d03640 1 -- 172.21.15.83:0/3920907132 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7facb0066f60 msgr2=0x7facb0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:08.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.462+0000 7facd2d03640 1 --2- 172.21.15.83:0/3920907132 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7facb0066f60 0x7facb0069420 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7facb80af000 tx=0x7facb80046d0 comp rx=0 tx=0).stop 2024-04-17T10:48:08.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.462+0000 7facd2d03640 1 -- 172.21.15.83:0/3920907132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facbc0061c0 msgr2=0x7faccc14e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:08.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.462+0000 7facd2d03640 1 --2- 172.21.15.83:0/3920907132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facbc0061c0 0x7faccc14e130 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7facc000f3c0 tx=0x7facc000f880 comp rx=0 tx=0).stop 2024-04-17T10:48:08.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.462+0000 7facd2d03640 1 -- 172.21.15.83:0/3920907132 shutdown_connections 2024-04-17T10:48:08.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.462+0000 7facd2d03640 1 --2- 172.21.15.83:0/3920907132 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7facb0066f60 0x7facb0069420 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:08.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.462+0000 7facd2d03640 1 --2- 172.21.15.83:0/3920907132 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faccc0add80 0x7faccc1f6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:08.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.462+0000 7facd2d03640 1 --2- 172.21.15.83:0/3920907132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faccc071e00 0x7faccc0ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:08.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.462+0000 7facd2d03640 1 --2- 172.21.15.83:0/3920907132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facbc0061c0 0x7faccc14e130 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:08.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.462+0000 7facd2d03640 1 -- 172.21.15.83:0/3920907132 >> 172.21.15.83:0/3920907132 conn(0x7faccc0b15b0 msgr2=0x7faccc0706c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:08.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.462+0000 7facd2d03640 1 -- 172.21.15.83:0/3920907132 shutdown_connections 2024-04-17T10:48:08.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.462+0000 7facd2d03640 1 -- 172.21.15.83:0/3920907132 wait complete. 2024-04-17T10:48:08.473 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:48:08.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.586+0000 7fc66fe9c640 1 Processor -- start 2024-04-17T10:48:08.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.586+0000 7fc66fe9c640 1 -- start start 2024-04-17T10:48:08.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.590+0000 7fc66fe9c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc66815e0f0 0x7fc6680b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:08.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.590+0000 7fc66fe9c640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc66815e4d0 con 0x7fc6680b5450 2024-04-17T10:48:08.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.590+0000 7fc66fe9c640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc668162580 con 0x7fc6680ce400 2024-04-17T10:48:08.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.590+0000 7fc66fe9c640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc668162b40 con 0x7fc66815e0f0 2024-04-17T10:48:08.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.590+0000 7fc66dc38640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc66815e0f0 0x7fc6680b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:08.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.590+0000 7fc66e439640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc6680b5450 0x7fc6680d2540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38160/0 (socket says 172.21.15.83:38160) 2024-04-17T10:48:08.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.590+0000 7fc66e439640 1 -- 172.21.15.83:0/2986522371 learned_addr learned my addr 172.21.15.83:0/2986522371 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:08.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.590+0000 7fc66dc38640 1 -- 172.21.15.83:0/2986522371 >> v1:172.21.15.167:6789/0 conn(0x7fc6680ce400 legacy=0x7fc668161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:08.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.590+0000 7fc66dc38640 1 -- 172.21.15.83:0/2986522371 >> v1:172.21.15.83:6789/0 conn(0x7fc6680b5450 legacy=0x7fc6680d2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:08.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.590+0000 7fc66dc38640 1 -- 172.21.15.83:0/2986522371 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc668162860 con 0x7fc66815e0f0 2024-04-17T10:48:08.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.590+0000 7fc66dc38640 1 --2- 172.21.15.83:0/2986522371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc66815e0f0 0x7fc6680b6d00 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fc6640097c0 tx=0x7fc66401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d6842f1fa804ba server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:08.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.590+0000 7fc66cc36640 1 -- 172.21.15.83:0/2986522371 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6640b0070 con 0x7fc66815e0f0 2024-04-17T10:48:08.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.590+0000 7fc66cc36640 1 -- 172.21.15.83:0/2986522371 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6640ab600 con 0x7fc66815e0f0 2024-04-17T10:48:08.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.590+0000 7fc66cc36640 1 -- 172.21.15.83:0/2986522371 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6640ab9d0 con 0x7fc66815e0f0 2024-04-17T10:48:08.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.590+0000 7fc66fe9c640 1 -- 172.21.15.83:0/2986522371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc66815e0f0 msgr2=0x7fc6680b6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:08.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.590+0000 7fc66fe9c640 1 --2- 172.21.15.83:0/2986522371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc66815e0f0 0x7fc6680b6d00 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fc6640097c0 tx=0x7fc66401c3d0 comp rx=0 tx=0).stop 2024-04-17T10:48:08.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.590+0000 7fc66fe9c640 1 -- 172.21.15.83:0/2986522371 shutdown_connections 2024-04-17T10:48:08.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.590+0000 7fc66fe9c640 1 --2- 172.21.15.83:0/2986522371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc66815e0f0 0x7fc6680b6d00 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:08.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.590+0000 7fc66fe9c640 1 -- 172.21.15.83:0/2986522371 >> 172.21.15.83:0/2986522371 conn(0x7fc6680d0450 msgr2=0x7fc6680d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:08.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.590+0000 7fc66fe9c640 1 -- 172.21.15.83:0/2986522371 shutdown_connections 2024-04-17T10:48:08.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.590+0000 7fc66fe9c640 1 -- 172.21.15.83:0/2986522371 wait complete. 2024-04-17T10:48:08.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.594+0000 7fc66fe9c640 1 Processor -- start 2024-04-17T10:48:08.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.594+0000 7fc66fe9c640 1 -- start start 2024-04-17T10:48:08.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.594+0000 7fc66fe9c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6680b5450 0x7fc668150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:08.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.594+0000 7fc66fe9c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6680ce400 0x7fc66814b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:08.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.594+0000 7fc66fe9c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc66815e0f0 0x7fc6681533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:08.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.594+0000 7fc66fe9c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66816c0e0 con 0x7fc6680b5450 2024-04-17T10:48:08.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.594+0000 7fc66fe9c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66816bf60 con 0x7fc6680ce400 2024-04-17T10:48:08.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.594+0000 7fc66fe9c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc66816c260 con 0x7fc66815e0f0 2024-04-17T10:48:08.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.594+0000 7fc66dc38640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6680b5450 0x7fc668150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:08.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.594+0000 7fc66dc38640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6680b5450 0x7fc668150900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39662/0 (socket says 172.21.15.83:39662) 2024-04-17T10:48:08.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.594+0000 7fc66dc38640 1 -- 172.21.15.83:0/3921469074 learned_addr learned my addr 172.21.15.83:0/3921469074 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:08.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.594+0000 7fc66d437640 1 --2- 172.21.15.83:0/3921469074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6680ce400 0x7fc66814b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:08.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.594+0000 7fc66dc38640 1 -- 172.21.15.83:0/3921469074 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc66815e0f0 msgr2=0x7fc6681533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:08.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.594+0000 7fc66dc38640 1 --2- 172.21.15.83:0/3921469074 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc66815e0f0 0x7fc6681533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:08.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.594+0000 7fc66dc38640 1 -- 172.21.15.83:0/3921469074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6680ce400 msgr2=0x7fc66814b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:08.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.594+0000 7fc66dc38640 1 --2- 172.21.15.83:0/3921469074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6680ce400 0x7fc66814b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:08.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.594+0000 7fc66dc38640 1 -- 172.21.15.83:0/3921469074 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6680cb450 con 0x7fc6680b5450 2024-04-17T10:48:08.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.594+0000 7fc66d437640 1 --2- 172.21.15.83:0/3921469074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6680ce400 0x7fc66814b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:08.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.594+0000 7fc66dc38640 1 --2- 172.21.15.83:0/3921469074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6680b5450 0x7fc668150900 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fc664018fe0 tx=0x7fc66401ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:08.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.594+0000 7fc65e7fc640 1 -- 172.21.15.83:0/3921469074 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6640b0050 con 0x7fc6680b5450 2024-04-17T10:48:08.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.594+0000 7fc65e7fc640 1 -- 172.21.15.83:0/3921469074 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6640bc070 con 0x7fc6680b5450 2024-04-17T10:48:08.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.594+0000 7fc66fe9c640 1 -- 172.21.15.83:0/3921469074 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6680cc400 con 0x7fc6680b5450 2024-04-17T10:48:08.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.594+0000 7fc65e7fc640 1 -- 172.21.15.83:0/3921469074 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6640b5a70 con 0x7fc6680b5450 2024-04-17T10:48:08.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.594+0000 7fc66fe9c640 1 -- 172.21.15.83:0/3921469074 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6680cb700 con 0x7fc6680b5450 2024-04-17T10:48:08.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.594+0000 7fc66fe9c640 1 -- 172.21.15.83:0/3921469074 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6680681a0 con 0x7fc6680b5450 2024-04-17T10:48:08.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.598+0000 7fc65e7fc640 1 -- 172.21.15.83:0/3921469074 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc6640ab4c0 con 0x7fc6680b5450 2024-04-17T10:48:08.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.598+0000 7fc65e7fc640 1 --2- 172.21.15.83:0/3921469074 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc64c066f60 0x7fc64c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:08.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.598+0000 7fc65e7fc640 1 -- 172.21.15.83:0/3921469074 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc66411f970 con 0x7fc6680b5450 2024-04-17T10:48:08.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.598+0000 7fc66d437640 1 --2- 172.21.15.83:0/3921469074 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc64c066f60 0x7fc64c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:08.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.598+0000 7fc66d437640 1 --2- 172.21.15.83:0/3921469074 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc64c066f60 0x7fc64c069420 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc66814c0d0 tx=0x7fc658009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:08.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.602+0000 7fc65e7fc640 1 -- 172.21.15.83:0/3921469074 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc664016050 con 0x7fc6680b5450 2024-04-17T10:48:08.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.758+0000 7fc66fe9c640 1 -- 172.21.15.83:0/3921469074 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc6680681a0 con 0x7fc6680b5450 2024-04-17T10:48:08.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.758+0000 7fc65e7fc640 1 -- 172.21.15.83:0/3921469074 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc6640f0980 con 0x7fc6680b5450 2024-04-17T10:48:08.756 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:48:08.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.762+0000 7fc66fe9c640 1 -- 172.21.15.83:0/3921469074 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc64c066f60 msgr2=0x7fc64c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:08.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.762+0000 7fc66fe9c640 1 --2- 172.21.15.83:0/3921469074 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc64c066f60 0x7fc64c069420 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc66814c0d0 tx=0x7fc658009040 comp rx=0 tx=0).stop 2024-04-17T10:48:08.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.762+0000 7fc66fe9c640 1 -- 172.21.15.83:0/3921469074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6680b5450 msgr2=0x7fc668150900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:08.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.762+0000 7fc66fe9c640 1 --2- 172.21.15.83:0/3921469074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6680b5450 0x7fc668150900 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fc664018fe0 tx=0x7fc66401ac10 comp rx=0 tx=0).stop 2024-04-17T10:48:08.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.762+0000 7fc66fe9c640 1 -- 172.21.15.83:0/3921469074 shutdown_connections 2024-04-17T10:48:08.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.762+0000 7fc66fe9c640 1 --2- 172.21.15.83:0/3921469074 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc64c066f60 0x7fc64c069420 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:08.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.762+0000 7fc66fe9c640 1 --2- 172.21.15.83:0/3921469074 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc66815e0f0 0x7fc6681533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:08.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.762+0000 7fc66fe9c640 1 --2- 172.21.15.83:0/3921469074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6680ce400 0x7fc66814b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:08.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.762+0000 7fc66fe9c640 1 --2- 172.21.15.83:0/3921469074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6680b5450 0x7fc668150900 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:08.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.762+0000 7fc66fe9c640 1 -- 172.21.15.83:0/3921469074 >> 172.21.15.83:0/3921469074 conn(0x7fc6680d0450 msgr2=0x7fc66815e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:08.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.762+0000 7fc66fe9c640 1 -- 172.21.15.83:0/3921469074 shutdown_connections 2024-04-17T10:48:08.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.762+0000 7fc66fe9c640 1 -- 172.21.15.83:0/3921469074 wait complete. 2024-04-17T10:48:08.774 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:48:08.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.898+0000 7f9c8d6d6640 1 Processor -- start 2024-04-17T10:48:08.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.898+0000 7f9c8d6d6640 1 -- start start 2024-04-17T10:48:08.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.898+0000 7f9c8d6d6640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9c8815b6e0 0x7f9c88071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:08.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.898+0000 7f9c8d6d6640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9c8815bac0 0x7f9c880cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:08.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.898+0000 7f9c8d6d6640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c8806c1f0 con 0x7f9c88070470 2024-04-17T10:48:08.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.898+0000 7f9c8d6d6640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c880cdd30 con 0x7f9c8815b6e0 2024-04-17T10:48:08.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.898+0000 7f9c8d6d6640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c88070a10 con 0x7f9c8815bac0 2024-04-17T10:48:08.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.898+0000 7f9c86ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9c8815b6e0 0x7f9c88071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:08.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.898+0000 7f9c86ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9c8815b6e0 0x7f9c88071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39678/0 (socket says 172.21.15.83:39678) 2024-04-17T10:48:08.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.898+0000 7f9c86ffd640 1 -- 172.21.15.83:0/4072070204 learned_addr learned my addr 172.21.15.83:0/4072070204 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:08.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.898+0000 7f9c867fc640 1 --2- 172.21.15.83:0/4072070204 >> v2:172.21.15.167:3300/0 conn(0x7f9c8815bac0 0x7f9c880cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:08.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.898+0000 7f9c86ffd640 1 -- 172.21.15.83:0/4072070204 >> v2:172.21.15.167:3300/0 conn(0x7f9c8815bac0 msgr2=0x7f9c880cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:08.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.898+0000 7f9c86ffd640 1 --2- 172.21.15.83:0/4072070204 >> v2:172.21.15.167:3300/0 conn(0x7f9c8815bac0 0x7f9c880cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:08.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.898+0000 7f9c86ffd640 1 -- 172.21.15.83:0/4072070204 >> v1:172.21.15.83:6789/0 conn(0x7f9c88070470 legacy=0x7f9c8806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:08.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.898+0000 7f9c86ffd640 1 -- 172.21.15.83:0/4072070204 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c88070cf0 con 0x7f9c8815b6e0 2024-04-17T10:48:08.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.898+0000 7f9c85ffb640 1 -- 172.21.15.83:0/4072070204 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3453887981 0 0) 0x7f9c8806c1f0 con 0x7f9c88070470 2024-04-17T10:48:08.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.902+0000 7f9c86ffd640 1 --2- 172.21.15.83:0/4072070204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c8815b6e0 0x7f9c88071cc0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f9c88071c50 tx=0x7f9c7801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63aa61aeff7410a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:08.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.902+0000 7f9c85ffb640 1 -- 172.21.15.83:0/4072070204 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c780b0070 con 0x7f9c8815b6e0 2024-04-17T10:48:08.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.902+0000 7f9c85ffb640 1 -- 172.21.15.83:0/4072070204 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c780ab600 con 0x7f9c8815b6e0 2024-04-17T10:48:08.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.902+0000 7f9c85ffb640 1 -- 172.21.15.83:0/4072070204 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c780ab9d0 con 0x7f9c8815b6e0 2024-04-17T10:48:08.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.902+0000 7f9c8d6d6640 1 -- 172.21.15.83:0/4072070204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c8815b6e0 msgr2=0x7f9c88071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:08.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.902+0000 7f9c8d6d6640 1 --2- 172.21.15.83:0/4072070204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c8815b6e0 0x7f9c88071cc0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f9c88071c50 tx=0x7f9c7801c520 comp rx=0 tx=0).stop 2024-04-17T10:48:08.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.902+0000 7f9c8d6d6640 1 -- 172.21.15.83:0/4072070204 shutdown_connections 2024-04-17T10:48:08.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.902+0000 7f9c8d6d6640 1 --2- 172.21.15.83:0/4072070204 >> v2:172.21.15.167:3300/0 conn(0x7f9c8815bac0 0x7f9c880cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:08.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.902+0000 7f9c8d6d6640 1 --2- 172.21.15.83:0/4072070204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c8815b6e0 0x7f9c88071cc0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:08.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.902+0000 7f9c8d6d6640 1 -- 172.21.15.83:0/4072070204 >> 172.21.15.83:0/4072070204 conn(0x7f9c880b15b0 msgr2=0x7f9c880b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:08.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.902+0000 7f9c8d6d6640 1 -- 172.21.15.83:0/4072070204 shutdown_connections 2024-04-17T10:48:08.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.902+0000 7f9c8d6d6640 1 -- 172.21.15.83:0/4072070204 wait complete. 2024-04-17T10:48:08.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.902+0000 7f9c8d6d6640 1 Processor -- start 2024-04-17T10:48:08.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.902+0000 7f9c8d6d6640 1 -- start start 2024-04-17T10:48:08.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.902+0000 7f9c8d6d6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c88070470 0x7f9c88146320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:08.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.902+0000 7f9c8d6d6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c8815b6e0 0x7f9c880b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:08.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.902+0000 7f9c8d6d6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c8815bac0 0x7f9c88149fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:08.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.902+0000 7f9c8d6d6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8806cee0 con 0x7f9c8815b6e0 2024-04-17T10:48:08.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.902+0000 7f9c8d6d6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8806b730 con 0x7f9c8815bac0 2024-04-17T10:48:08.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.902+0000 7f9c8d6d6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8806d060 con 0x7f9c88070470 2024-04-17T10:48:08.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.902+0000 7f9c867fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c8815b6e0 0x7f9c880b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:08.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.902+0000 7f9c867fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c8815b6e0 0x7f9c880b3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39692/0 (socket says 172.21.15.83:39692) 2024-04-17T10:48:08.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.902+0000 7f9c867fc640 1 -- 172.21.15.83:0/1149304564 learned_addr learned my addr 172.21.15.83:0/1149304564 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:08.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.902+0000 7f9c86ffd640 1 --2- 172.21.15.83:0/1149304564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c88070470 0x7f9c88146320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:08.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.902+0000 7f9c877fe640 1 --2- 172.21.15.83:0/1149304564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c8815bac0 0x7f9c88149fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:08.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.902+0000 7f9c867fc640 1 -- 172.21.15.83:0/1149304564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c88070470 msgr2=0x7f9c88146320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:08.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.902+0000 7f9c867fc640 1 --2- 172.21.15.83:0/1149304564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c88070470 0x7f9c88146320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:08.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.902+0000 7f9c867fc640 1 -- 172.21.15.83:0/1149304564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c8815bac0 msgr2=0x7f9c88149fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:08.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.902+0000 7f9c867fc640 1 --2- 172.21.15.83:0/1149304564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c8815bac0 0x7f9c88149fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:08.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.902+0000 7f9c867fc640 1 -- 172.21.15.83:0/1149304564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c8814ae00 con 0x7f9c8815b6e0 2024-04-17T10:48:08.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.902+0000 7f9c867fc640 1 --2- 172.21.15.83:0/1149304564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c8815b6e0 0x7f9c880b3a20 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00d620 tx=0x7f9c7c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:08.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.902+0000 7f9c6f7fe640 1 -- 172.21.15.83:0/1149304564 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c00b620 con 0x7f9c8815b6e0 2024-04-17T10:48:08.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.906+0000 7f9c6f7fe640 1 -- 172.21.15.83:0/1149304564 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c7c019070 con 0x7f9c8815b6e0 2024-04-17T10:48:08.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.906+0000 7f9c8d6d6640 1 -- 172.21.15.83:0/1149304564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c880b2980 con 0x7f9c8815b6e0 2024-04-17T10:48:08.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.906+0000 7f9c6f7fe640 1 -- 172.21.15.83:0/1149304564 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c00b9e0 con 0x7f9c8815b6e0 2024-04-17T10:48:08.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.906+0000 7f9c8d6d6640 1 -- 172.21.15.83:0/1149304564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c880b2f40 con 0x7f9c8815b6e0 2024-04-17T10:48:08.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.906+0000 7f9c8d6d6640 1 -- 172.21.15.83:0/1149304564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c88074f90 con 0x7f9c8815b6e0 2024-04-17T10:48:08.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.906+0000 7f9c6f7fe640 1 -- 172.21.15.83:0/1149304564 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9c7c00bb80 con 0x7f9c8815b6e0 2024-04-17T10:48:08.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.906+0000 7f9c6f7fe640 1 --2- 172.21.15.83:0/1149304564 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9c74066f90 0x7f9c74069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:08.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.906+0000 7f9c6f7fe640 1 -- 172.21.15.83:0/1149304564 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c7c0853f0 con 0x7f9c8815b6e0 2024-04-17T10:48:08.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.906+0000 7f9c86ffd640 1 --2- 172.21.15.83:0/1149304564 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9c74066f90 0x7f9c74069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:08.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.906+0000 7f9c86ffd640 1 --2- 172.21.15.83:0/1149304564 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9c74066f90 0x7f9c74069450 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9c780ad000 tx=0x7f9c780a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:08.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:08.910+0000 7f9c6f7fe640 1 -- 172.21.15.83:0/1149304564 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9c8814ae00 con 0x7f9c8815b6e0 2024-04-17T10:48:09.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.062+0000 7f9c8d6d6640 1 -- 172.21.15.83:0/1149304564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9c88074480 con 0x7f9c8815b6e0 2024-04-17T10:48:09.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.062+0000 7f9c6f7fe640 1 -- 172.21.15.83:0/1149304564 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9c7c056400 con 0x7f9c8815b6e0 2024-04-17T10:48:09.058 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:48:09.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.062+0000 7f9c8d6d6640 1 -- 172.21.15.83:0/1149304564 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9c74066f90 msgr2=0x7f9c74069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:09.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.062+0000 7f9c8d6d6640 1 --2- 172.21.15.83:0/1149304564 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9c74066f90 0x7f9c74069450 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9c780ad000 tx=0x7f9c780a6000 comp rx=0 tx=0).stop 2024-04-17T10:48:09.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.062+0000 7f9c8d6d6640 1 -- 172.21.15.83:0/1149304564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c8815b6e0 msgr2=0x7f9c880b3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:09.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.062+0000 7f9c8d6d6640 1 --2- 172.21.15.83:0/1149304564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c8815b6e0 0x7f9c880b3a20 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00d620 tx=0x7f9c7c00dae0 comp rx=0 tx=0).stop 2024-04-17T10:48:09.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.062+0000 7f9c8d6d6640 1 -- 172.21.15.83:0/1149304564 shutdown_connections 2024-04-17T10:48:09.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.062+0000 7f9c8d6d6640 1 --2- 172.21.15.83:0/1149304564 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9c74066f90 0x7f9c74069450 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:09.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.062+0000 7f9c8d6d6640 1 --2- 172.21.15.83:0/1149304564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c8815bac0 0x7f9c88149fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:09.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.062+0000 7f9c8d6d6640 1 --2- 172.21.15.83:0/1149304564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c8815b6e0 0x7f9c880b3a20 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:09.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.062+0000 7f9c8d6d6640 1 --2- 172.21.15.83:0/1149304564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c88070470 0x7f9c88146320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:09.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.062+0000 7f9c8d6d6640 1 -- 172.21.15.83:0/1149304564 >> 172.21.15.83:0/1149304564 conn(0x7f9c880b15b0 msgr2=0x7f9c8806ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:09.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.062+0000 7f9c8d6d6640 1 -- 172.21.15.83:0/1149304564 shutdown_connections 2024-04-17T10:48:09.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.062+0000 7f9c8d6d6640 1 -- 172.21.15.83:0/1149304564 wait complete. 2024-04-17T10:48:09.076 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:48:09.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24e26ab640 1 Processor -- start 2024-04-17T10:48:09.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24e26ab640 1 -- start start 2024-04-17T10:48:09.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24e26ab640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24dc15b6e0 0x7f24dc071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:09.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24e26ab640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24dc06c1f0 con 0x7f24dc070470 2024-04-17T10:48:09.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24e26ab640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24dc070a10 con 0x7f24dc15bac0 2024-04-17T10:48:09.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24e26ab640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24dc070cf0 con 0x7f24dc15b6e0 2024-04-17T10:48:09.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24e0c48640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f24dc070470 0x7f24dc06ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38190/0 (socket says 172.21.15.83:38190) 2024-04-17T10:48:09.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24e0c48640 1 -- 172.21.15.83:0/1385636097 learned_addr learned my addr 172.21.15.83:0/1385636097 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:09.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24dbfff640 1 --2- 172.21.15.83:0/1385636097 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24dc15b6e0 0x7f24dc071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:09.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24daffd640 1 -- 172.21.15.83:0/1385636097 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2839434400 0 0) 0x7f24dc06c1f0 con 0x7f24dc070470 2024-04-17T10:48:09.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24daffd640 1 -- 172.21.15.83:0/1385636097 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24c8003530 con 0x7f24dc070470 2024-04-17T10:48:09.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24daffd640 1 -- 172.21.15.83:0/1385636097 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3992843606 0 0) 0x7f24c8003530 con 0x7f24dc070470 2024-04-17T10:48:09.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24daffd640 1 -- 172.21.15.83:0/1385636097 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24dc15b6e0 msgr2=0x7f24dc071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:09.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24daffd640 1 --2- 172.21.15.83:0/1385636097 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24dc15b6e0 0x7f24dc071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:09.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24daffd640 1 -- 172.21.15.83:0/1385636097 >> v1:172.21.15.167:6789/0 conn(0x7f24dc15bac0 legacy=0x7f24dc0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:09.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24daffd640 1 -- 172.21.15.83:0/1385636097 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24dc070fd0 con 0x7f24dc070470 2024-04-17T10:48:09.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24daffd640 1 -- 172.21.15.83:0/1385636097 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f24d4001680 con 0x7f24dc070470 2024-04-17T10:48:09.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24daffd640 1 -- 172.21.15.83:0/1385636097 >> v1:172.21.15.83:6789/0 conn(0x7f24dc070470 legacy=0x7f24dc06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:09.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24daffd640 1 --2- 172.21.15.83:0/1385636097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24c8003a60 0x7f24c8003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:09.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24daffd640 1 --2- 172.21.15.83:0/1385636097 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24dc15b6e0 0x7f24c8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:09.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24daffd640 1 --2- 172.21.15.83:0/1385636097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24c8004d50 0x7f24c80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:09.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24daffd640 1 -- 172.21.15.83:0/1385636097 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24dc070cf0 con 0x7f24c8004d50 2024-04-17T10:48:09.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24daffd640 1 -- 172.21.15.83:0/1385636097 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c800bae0 con 0x7f24c8003a60 2024-04-17T10:48:09.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24daffd640 1 -- 172.21.15.83:0/1385636097 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c800bce0 con 0x7f24dc15b6e0 2024-04-17T10:48:09.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24e0c48640 1 --2- 172.21.15.83:0/1385636097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24c8004d50 0x7f24c80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:09.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24dbfff640 1 --2- 172.21.15.83:0/1385636097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24c8003a60 0x7f24c8003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:09.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24db7fe640 1 --2- 172.21.15.83:0/1385636097 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24dc15b6e0 0x7f24c8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:09.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24e0c48640 1 -- 172.21.15.83:0/1385636097 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24dc15b6e0 msgr2=0x7f24c8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:09.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24e0c48640 1 --2- 172.21.15.83:0/1385636097 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24dc15b6e0 0x7f24c8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:09.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24e0c48640 1 -- 172.21.15.83:0/1385636097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24c8003a60 msgr2=0x7f24c8003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:09.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24e0c48640 1 --2- 172.21.15.83:0/1385636097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24c8003a60 0x7f24c8003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:09.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24e0c48640 1 -- 172.21.15.83:0/1385636097 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c8003530 con 0x7f24c8004d50 2024-04-17T10:48:09.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.202+0000 7f24db7fe640 1 --2- 172.21.15.83:0/1385636097 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24dc15b6e0 0x7f24c8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:09.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.206+0000 7f24e0c48640 1 --2- 172.21.15.83:0/1385636097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24c8004d50 0x7f24c80051d0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f24d4001060 tx=0x7f24d401cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d3725770a4e86d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:09.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.206+0000 7f24daffd640 1 -- 172.21.15.83:0/1385636097 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d4007d20 con 0x7f24c8004d50 2024-04-17T10:48:09.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.206+0000 7f24daffd640 1 -- 172.21.15.83:0/1385636097 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d40a9020 con 0x7f24c8004d50 2024-04-17T10:48:09.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.206+0000 7f24daffd640 1 -- 172.21.15.83:0/1385636097 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d40a5b40 con 0x7f24c8004d50 2024-04-17T10:48:09.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.206+0000 7f24e26ab640 1 -- 172.21.15.83:0/1385636097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24c8004d50 msgr2=0x7f24c80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:09.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.206+0000 7f24e26ab640 1 --2- 172.21.15.83:0/1385636097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24c8004d50 0x7f24c80051d0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f24d4001060 tx=0x7f24d401cc20 comp rx=0 tx=0).stop 2024-04-17T10:48:09.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.206+0000 7f24e0c48640 1 -- 172.21.15.83:0/1385636097 reap_dead start 2024-04-17T10:48:09.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.206+0000 7f24e26ab640 1 -- 172.21.15.83:0/1385636097 shutdown_connections 2024-04-17T10:48:09.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.206+0000 7f24e26ab640 1 -- 172.21.15.83:0/1385636097 >> 172.21.15.83:0/1385636097 conn(0x7f24dc0b15b0 msgr2=0x7f24dc0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:09.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.206+0000 7f24e26ab640 1 -- 172.21.15.83:0/1385636097 shutdown_connections 2024-04-17T10:48:09.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.206+0000 7f24e26ab640 1 -- 172.21.15.83:0/1385636097 wait complete. 2024-04-17T10:48:09.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.206+0000 7f24e26ab640 1 Processor -- start 2024-04-17T10:48:09.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.206+0000 7f24e26ab640 1 -- start start 2024-04-17T10:48:09.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.210+0000 7f24e26ab640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24c8004d50 0x7f24dc1f2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:09.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.210+0000 7f24e26ab640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24dc15e8b0 0x7f24dc0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:09.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.210+0000 7f24e26ab640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24dc15ed30 0x7f24dc0add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:09.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.210+0000 7f24e26ab640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24dc06b6a0 con 0x7f24c8004d50 2024-04-17T10:48:09.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.210+0000 7f24e26ab640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24dc06b520 con 0x7f24dc15ed30 2024-04-17T10:48:09.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.210+0000 7f24e26ab640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24dc071cc0 con 0x7f24dc15e8b0 2024-04-17T10:48:09.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.210+0000 7f24dbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24c8004d50 0x7f24dc1f2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:09.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.210+0000 7f24dbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24c8004d50 0x7f24dc1f2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39698/0 (socket says 172.21.15.83:39698) 2024-04-17T10:48:09.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.210+0000 7f24dbfff640 1 -- 172.21.15.83:0/278309946 learned_addr learned my addr 172.21.15.83:0/278309946 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:09.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.210+0000 7f24e0c48640 1 --2- 172.21.15.83:0/278309946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24dc15ed30 0x7f24dc0add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:09.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.210+0000 7f24db7fe640 1 --2- 172.21.15.83:0/278309946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24dc15e8b0 0x7f24dc0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:09.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.210+0000 7f24dbfff640 1 -- 172.21.15.83:0/278309946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24dc15e8b0 msgr2=0x7f24dc0ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:09.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.210+0000 7f24dbfff640 1 --2- 172.21.15.83:0/278309946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24dc15e8b0 0x7f24dc0ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:09.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.210+0000 7f24dbfff640 1 -- 172.21.15.83:0/278309946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24dc15ed30 msgr2=0x7f24dc0add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:09.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.210+0000 7f24dbfff640 1 --2- 172.21.15.83:0/278309946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24dc15ed30 0x7f24dc0add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:09.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.210+0000 7f24dbfff640 1 -- 172.21.15.83:0/278309946 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24dc143bc0 con 0x7f24c8004d50 2024-04-17T10:48:09.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.210+0000 7f24db7fe640 1 --2- 172.21.15.83:0/278309946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24dc15e8b0 0x7f24dc0ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:09.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.210+0000 7f24dbfff640 1 --2- 172.21.15.83:0/278309946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24c8004d50 0x7f24dc1f2cf0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f24cc007f30 tx=0x7f24cc011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:09.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.210+0000 7f24d8ff9640 1 -- 172.21.15.83:0/278309946 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc01a020 con 0x7f24c8004d50 2024-04-17T10:48:09.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.210+0000 7f24d8ff9640 1 -- 172.21.15.83:0/278309946 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24cc016470 con 0x7f24c8004d50 2024-04-17T10:48:09.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.210+0000 7f24e26ab640 1 -- 172.21.15.83:0/278309946 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24dc143040 con 0x7f24c8004d50 2024-04-17T10:48:09.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.210+0000 7f24d8ff9640 1 -- 172.21.15.83:0/278309946 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc016840 con 0x7f24c8004d50 2024-04-17T10:48:09.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.210+0000 7f24e26ab640 1 -- 172.21.15.83:0/278309946 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24dc143600 con 0x7f24c8004d50 2024-04-17T10:48:09.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.210+0000 7f24e26ab640 1 -- 172.21.15.83:0/278309946 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24dc074f90 con 0x7f24c8004d50 2024-04-17T10:48:09.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.210+0000 7f24d8ff9640 1 -- 172.21.15.83:0/278309946 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f24cc01d020 con 0x7f24c8004d50 2024-04-17T10:48:09.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.210+0000 7f24d8ff9640 1 --2- 172.21.15.83:0/278309946 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f24bc066f60 0x7f24bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:09.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.210+0000 7f24d8ff9640 1 -- 172.21.15.83:0/278309946 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24cc085ec0 con 0x7f24c8004d50 2024-04-17T10:48:09.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.210+0000 7f24db7fe640 1 --2- 172.21.15.83:0/278309946 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f24bc066f60 0x7f24bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:09.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.210+0000 7f24db7fe640 1 --2- 172.21.15.83:0/278309946 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f24bc066f60 0x7f24bc069420 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f24c4007fd0 tx=0x7f24c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:09.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.214+0000 7f24d8ff9640 1 -- 172.21.15.83:0/278309946 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f24cc024030 con 0x7f24c8004d50 2024-04-17T10:48:09.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.374+0000 7f24e26ab640 1 -- 172.21.15.83:0/278309946 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f24dc06ae80 con 0x7f24c8004d50 2024-04-17T10:48:09.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.374+0000 7f24d8ff9640 1 -- 172.21.15.83:0/278309946 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v103) v1 ==== 80+0+0 (secure 0 0 0) 0x7f24cc056eb0 con 0x7f24c8004d50 2024-04-17T10:48:09.370 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-17T10:48:09.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.374+0000 7f24e26ab640 1 -- 172.21.15.83:0/278309946 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f24bc066f60 msgr2=0x7f24bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:09.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.374+0000 7f24e26ab640 1 --2- 172.21.15.83:0/278309946 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f24bc066f60 0x7f24bc069420 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f24c4007fd0 tx=0x7f24c4009040 comp rx=0 tx=0).stop 2024-04-17T10:48:09.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.374+0000 7f24e26ab640 1 -- 172.21.15.83:0/278309946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24c8004d50 msgr2=0x7f24dc1f2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:09.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.374+0000 7f24e26ab640 1 --2- 172.21.15.83:0/278309946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24c8004d50 0x7f24dc1f2cf0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f24cc007f30 tx=0x7f24cc011660 comp rx=0 tx=0).stop 2024-04-17T10:48:09.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.374+0000 7f24e26ab640 1 -- 172.21.15.83:0/278309946 shutdown_connections 2024-04-17T10:48:09.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.374+0000 7f24e26ab640 1 --2- 172.21.15.83:0/278309946 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f24bc066f60 0x7f24bc069420 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:09.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.374+0000 7f24e26ab640 1 --2- 172.21.15.83:0/278309946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24dc15ed30 0x7f24dc0add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:09.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.374+0000 7f24e26ab640 1 --2- 172.21.15.83:0/278309946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24dc15e8b0 0x7f24dc0ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:09.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.374+0000 7f24e26ab640 1 --2- 172.21.15.83:0/278309946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24c8004d50 0x7f24dc1f2cf0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:09.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.374+0000 7f24e26ab640 1 -- 172.21.15.83:0/278309946 >> 172.21.15.83:0/278309946 conn(0x7f24dc0b15b0 msgr2=0x7f24dc0cdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:09.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.374+0000 7f24e26ab640 1 -- 172.21.15.83:0/278309946 shutdown_connections 2024-04-17T10:48:09.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.374+0000 7f24e26ab640 1 -- 172.21.15.83:0/278309946 wait complete. 2024-04-17T10:48:09.387 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2024-04-17T10:48:09.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:09.388 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-17T10:48:09.398 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[73527]: fuse finished with error 0 and tester_r 0 2024-04-17T10:48:09.476 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-17T10:48:09.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:09.476 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T10:48:09.638 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:48:09.638 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:48:09.645 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:48:09.645 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-04-17T10:48:09.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:09.645 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-17T10:48:09.694 INFO:teuthology.orchestra.run.smithi167.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-17T10:48:09.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:48:09.695 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:48:09.695 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:48:09.701 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16063 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:48:09.701 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-17T10:48:09.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:09.701 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-17T10:48:09.753 INFO:teuthology.orchestra.run.smithi083.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-17T10:48:09.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:48:09.754 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:48:09.755 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:48:09.762 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:48:09.762 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-04-17T10:48:09.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:09.762 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-17T10:48:09.810 INFO:teuthology.orchestra.run.smithi167.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-17T10:48:09.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:48:09.811 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:48:09.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.942+0000 7f44f9fa5640 1 Processor -- start 2024-04-17T10:48:09.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.942+0000 7f44f9fa5640 1 -- start start 2024-04-17T10:48:09.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.942+0000 7f44f9fa5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f44f4072330 0x7f44f40cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:09.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.942+0000 7f44f9fa5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f44f40cfa10 0x7f44f406c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:09.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.942+0000 7f44f9fa5640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44f40cfea0 con 0x7f44f415ba30 2024-04-17T10:48:09.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.942+0000 7f44f9fa5640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44f4071ea0 con 0x7f44f40cfa10 2024-04-17T10:48:09.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.942+0000 7f44f9fa5640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44f406e320 con 0x7f44f4072330 2024-04-17T10:48:09.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.942+0000 7f44f37fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f44f415ba30 0x7f44f415c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38206/0 (socket says 172.21.15.83:38206) 2024-04-17T10:48:09.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.942+0000 7f44f37fe640 1 -- 172.21.15.83:0/845216244 learned_addr learned my addr 172.21.15.83:0/845216244 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:09.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.942+0000 7f44f3fff640 1 --2- 172.21.15.83:0/845216244 >> v2:172.21.15.83:3300/0 conn(0x7f44f40cfa10 0x7f44f406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:09.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.942+0000 7f44f27fc640 1 -- 172.21.15.83:0/845216244 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1815359683 0 0) 0x7f44f40cfea0 con 0x7f44f415ba30 2024-04-17T10:48:09.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.942+0000 7f44f27fc640 1 -- 172.21.15.83:0/845216244 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44e0003530 con 0x7f44f415ba30 2024-04-17T10:48:09.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.942+0000 7f44f2ffd640 1 --2- 172.21.15.83:0/845216244 >> v2:172.21.15.167:3300/0 conn(0x7f44f4072330 0x7f44f40cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:09.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.942+0000 7f44f3fff640 1 -- 172.21.15.83:0/845216244 >> v2:172.21.15.167:3300/0 conn(0x7f44f4072330 msgr2=0x7f44f40cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:09.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.942+0000 7f44f3fff640 1 --2- 172.21.15.83:0/845216244 >> v2:172.21.15.167:3300/0 conn(0x7f44f4072330 0x7f44f40cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:09.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.942+0000 7f44f3fff640 1 -- 172.21.15.83:0/845216244 >> v1:172.21.15.83:6789/0 conn(0x7f44f415ba30 legacy=0x7f44f415c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:09.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.942+0000 7f44f3fff640 1 -- 172.21.15.83:0/845216244 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f406e040 con 0x7f44f40cfa10 2024-04-17T10:48:09.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.946+0000 7f44f3fff640 1 --2- 172.21.15.83:0/845216244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44f40cfa10 0x7f44f406c490 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f44f40af2b0 tx=0x7f44ec01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ecf9306003de92c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:09.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.946+0000 7f44f27fc640 1 -- 172.21.15.83:0/845216244 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec0b2070 con 0x7f44f40cfa10 2024-04-17T10:48:09.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.946+0000 7f44f27fc640 1 -- 172.21.15.83:0/845216244 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44ec0a5870 con 0x7f44f40cfa10 2024-04-17T10:48:09.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.946+0000 7f44f27fc640 1 -- 172.21.15.83:0/845216244 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec0a5c40 con 0x7f44f40cfa10 2024-04-17T10:48:09.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.946+0000 7f44f9fa5640 1 -- 172.21.15.83:0/845216244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44f40cfa10 msgr2=0x7f44f406c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:09.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.946+0000 7f44f9fa5640 1 --2- 172.21.15.83:0/845216244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44f40cfa10 0x7f44f406c490 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f44f40af2b0 tx=0x7f44ec01c8a0 comp rx=0 tx=0).stop 2024-04-17T10:48:09.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.946+0000 7f44f9fa5640 1 -- 172.21.15.83:0/845216244 shutdown_connections 2024-04-17T10:48:09.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.946+0000 7f44f9fa5640 1 --2- 172.21.15.83:0/845216244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44f40cfa10 0x7f44f406c490 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:09.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.946+0000 7f44f9fa5640 1 --2- 172.21.15.83:0/845216244 >> v2:172.21.15.167:3300/0 conn(0x7f44f4072330 0x7f44f40cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:09.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.946+0000 7f44f9fa5640 1 -- 172.21.15.83:0/845216244 >> 172.21.15.83:0/845216244 conn(0x7f44f406abe0 msgr2=0x7f44f406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:09.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.946+0000 7f44f9fa5640 1 -- 172.21.15.83:0/845216244 shutdown_connections 2024-04-17T10:48:09.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.946+0000 7f44f9fa5640 1 -- 172.21.15.83:0/845216244 wait complete. 2024-04-17T10:48:09.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.946+0000 7f44f9fa5640 1 Processor -- start 2024-04-17T10:48:09.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.946+0000 7f44f9fa5640 1 -- start start 2024-04-17T10:48:09.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.950+0000 7f44f9fa5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44f4072330 0x7f44f414ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:09.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.950+0000 7f44f9fa5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44f40cfa10 0x7f44f415a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:09.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.950+0000 7f44f9fa5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44f415ba30 0x7f44f414d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:09.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.950+0000 7f44f9fa5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f406de20 con 0x7f44f415ba30 2024-04-17T10:48:09.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.950+0000 7f44f9fa5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f406dca0 con 0x7f44f4072330 2024-04-17T10:48:09.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.950+0000 7f44f9fa5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f4168520 con 0x7f44f40cfa10 2024-04-17T10:48:09.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.950+0000 7f44f3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44f415ba30 0x7f44f414d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:09.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.950+0000 7f44f3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44f415ba30 0x7f44f414d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39706/0 (socket says 172.21.15.83:39706) 2024-04-17T10:48:09.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.950+0000 7f44f3fff640 1 -- 172.21.15.83:0/2277271719 learned_addr learned my addr 172.21.15.83:0/2277271719 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:09.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.950+0000 7f44f2ffd640 1 --2- 172.21.15.83:0/2277271719 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44f40cfa10 0x7f44f415a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:09.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.950+0000 7f44f37fe640 1 --2- 172.21.15.83:0/2277271719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44f4072330 0x7f44f414ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:09.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.950+0000 7f44f3fff640 1 -- 172.21.15.83:0/2277271719 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44f40cfa10 msgr2=0x7f44f415a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:09.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.950+0000 7f44f3fff640 1 --2- 172.21.15.83:0/2277271719 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44f40cfa10 0x7f44f415a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:09.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.950+0000 7f44f3fff640 1 -- 172.21.15.83:0/2277271719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44f4072330 msgr2=0x7f44f414ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:09.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.950+0000 7f44f3fff640 1 --2- 172.21.15.83:0/2277271719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44f4072330 0x7f44f414ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:09.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.950+0000 7f44f3fff640 1 -- 172.21.15.83:0/2277271719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f40adee0 con 0x7f44f415ba30 2024-04-17T10:48:09.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.950+0000 7f44f37fe640 1 --2- 172.21.15.83:0/2277271719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44f4072330 0x7f44f414ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:09.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.950+0000 7f44f3fff640 1 --2- 172.21.15.83:0/2277271719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44f415ba30 0x7f44f414d1b0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f44ec01a4d0 tx=0x7f44ec019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:09.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.950+0000 7f44ebfff640 1 -- 172.21.15.83:0/2277271719 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec0bd020 con 0x7f44f415ba30 2024-04-17T10:48:09.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.950+0000 7f44ebfff640 1 -- 172.21.15.83:0/2277271719 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44ec0b8470 con 0x7f44f415ba30 2024-04-17T10:48:09.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.950+0000 7f44f9fa5640 1 -- 172.21.15.83:0/2277271719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f414e4c0 con 0x7f44f415ba30 2024-04-17T10:48:09.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.950+0000 7f44ebfff640 1 -- 172.21.15.83:0/2277271719 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec0b2040 con 0x7f44f415ba30 2024-04-17T10:48:09.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.950+0000 7f44f9fa5640 1 -- 172.21.15.83:0/2277271719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f40ae1c0 con 0x7f44f415ba30 2024-04-17T10:48:09.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.950+0000 7f44f9fa5640 1 -- 172.21.15.83:0/2277271719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44f4074f90 con 0x7f44f415ba30 2024-04-17T10:48:09.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.950+0000 7f44ebfff640 1 -- 172.21.15.83:0/2277271719 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f44ec0c0020 con 0x7f44f415ba30 2024-04-17T10:48:09.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.950+0000 7f44ebfff640 1 --2- 172.21.15.83:0/2277271719 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f44d4066f90 0x7f44d4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:09.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.950+0000 7f44f37fe640 1 --2- 172.21.15.83:0/2277271719 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f44d4066f90 0x7f44d4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:09.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.950+0000 7f44ebfff640 1 -- 172.21.15.83:0/2277271719 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f44ec11fd70 con 0x7f44f415ba30 2024-04-17T10:48:09.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.950+0000 7f44f37fe640 1 --2- 172.21.15.83:0/2277271719 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f44d4066f90 0x7f44d4069450 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f44e4005890 tx=0x7f44e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:09.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:09.954+0000 7f44ebfff640 1 -- 172.21.15.83:0/2277271719 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f44ec0c7030 con 0x7f44f415ba30 2024-04-17T10:48:10.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.142+0000 7f44f9fa5640 1 -- 172.21.15.83:0/2277271719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44f40cf250 con 0x7f44f415ba30 2024-04-17T10:48:10.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.142+0000 7f44ebfff640 1 -- 172.21.15.83:0/2277271719 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f44ec016520 con 0x7f44f415ba30 2024-04-17T10:48:10.139 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:10.139 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/437268957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":437268957},{"type":"v1","addr":"172.21.15.167:6833","nonce":437268957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":6253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3214640674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3214640674},{"type":"v1","addr":"172.21.15.167:6835","nonce":3214640674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":8046,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/715498323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":715498323},{"type":"v1","addr":"172.21.15.83:6835","nonce":715498323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:47:29.385103+0000","modified":"2024-04-17T10:47:31.389088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8049":{"gid":8049,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2353406669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":2353406669},{"type":"v1","addr":"172.21.15.83:6837","nonce":2353406669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-17T10:48:10.139 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 62 2024-04-17T10:48:10.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.146+0000 7f44f9fa5640 1 -- 172.21.15.83:0/2277271719 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f44d4066f90 msgr2=0x7f44d4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:10.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.146+0000 7f44f9fa5640 1 --2- 172.21.15.83:0/2277271719 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f44d4066f90 0x7f44d4069450 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f44e4005890 tx=0x7f44e4009040 comp rx=0 tx=0).stop 2024-04-17T10:48:10.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.146+0000 7f44f9fa5640 1 -- 172.21.15.83:0/2277271719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44f415ba30 msgr2=0x7f44f414d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:10.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.146+0000 7f44f9fa5640 1 --2- 172.21.15.83:0/2277271719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44f415ba30 0x7f44f414d1b0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f44ec01a4d0 tx=0x7f44ec019640 comp rx=0 tx=0).stop 2024-04-17T10:48:10.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.146+0000 7f44f9fa5640 1 -- 172.21.15.83:0/2277271719 shutdown_connections 2024-04-17T10:48:10.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.146+0000 7f44f9fa5640 1 --2- 172.21.15.83:0/2277271719 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f44d4066f90 0x7f44d4069450 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:10.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.146+0000 7f44f9fa5640 1 --2- 172.21.15.83:0/2277271719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44f415ba30 0x7f44f414d1b0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:10.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.146+0000 7f44f9fa5640 1 --2- 172.21.15.83:0/2277271719 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44f40cfa10 0x7f44f415a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:10.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.146+0000 7f44f9fa5640 1 --2- 172.21.15.83:0/2277271719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44f4072330 0x7f44f414ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:10.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.146+0000 7f44f9fa5640 1 -- 172.21.15.83:0/2277271719 >> 172.21.15.83:0/2277271719 conn(0x7f44f406abe0 msgr2=0x7f44f415acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:10.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.146+0000 7f44f9fa5640 1 -- 172.21.15.83:0/2277271719 shutdown_connections 2024-04-17T10:48:10.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.146+0000 7f44f9fa5640 1 -- 172.21.15.83:0/2277271719 wait complete. 2024-04-17T10:48:10.157 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:48:10.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.266+0000 7f635c76b640 1 Processor -- start 2024-04-17T10:48:10.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.266+0000 7f635c76b640 1 -- start start 2024-04-17T10:48:10.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.270+0000 7f635c76b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f63540b3990 0x7f6354070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:10.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.270+0000 7f635c76b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6354070dc0 con 0x7f63540709b0 2024-04-17T10:48:10.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.270+0000 7f635c76b640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63540b2bd0 con 0x7f63540cf060 2024-04-17T10:48:10.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.270+0000 7f635c76b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63540b2980 con 0x7f63540b3990 2024-04-17T10:48:10.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.270+0000 7f635ad08640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f63540709b0 0x7f635406c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38218/0 (socket says 172.21.15.83:38218) 2024-04-17T10:48:10.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.270+0000 7f6359d06640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f63540b3990 0x7f6354070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:10.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.270+0000 7f635ad08640 1 -- 172.21.15.83:0/1066658190 learned_addr learned my addr 172.21.15.83:0/1066658190 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:10.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.270+0000 7f6359d06640 1 -- 172.21.15.83:0/1066658190 >> v1:172.21.15.167:6789/0 conn(0x7f63540cf060 legacy=0x7f63540b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:10.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.270+0000 7f6359d06640 1 -- 172.21.15.83:0/1066658190 >> v1:172.21.15.83:6789/0 conn(0x7f63540709b0 legacy=0x7f635406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:10.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.270+0000 7f6359d06640 1 -- 172.21.15.83:0/1066658190 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63540b2eb0 con 0x7f63540b3990 2024-04-17T10:48:10.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.270+0000 7f6359505640 1 -- 172.21.15.83:0/1066658190 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1936952287 0 0) 0x7f634c0014a0 con 0x7f63540709b0 2024-04-17T10:48:10.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.270+0000 7f6359d06640 1 --2- 172.21.15.83:0/1066658190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63540b3990 0x7f6354070470 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f634400a960 tx=0x7f634401d840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438da86c842380c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:10.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.270+0000 7f6359505640 1 -- 172.21.15.83:0/1066658190 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63440b3070 con 0x7f63540b3990 2024-04-17T10:48:10.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.270+0000 7f6359505640 1 -- 172.21.15.83:0/1066658190 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63440a68f0 con 0x7f63540b3990 2024-04-17T10:48:10.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.270+0000 7f6359505640 1 -- 172.21.15.83:0/1066658190 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634400b040 con 0x7f63540b3990 2024-04-17T10:48:10.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.270+0000 7f635c76b640 1 -- 172.21.15.83:0/1066658190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63540b3990 msgr2=0x7f6354070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:10.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.270+0000 7f635c76b640 1 --2- 172.21.15.83:0/1066658190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63540b3990 0x7f6354070470 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f634400a960 tx=0x7f634401d840 comp rx=0 tx=0).stop 2024-04-17T10:48:10.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.270+0000 7f635c76b640 1 -- 172.21.15.83:0/1066658190 shutdown_connections 2024-04-17T10:48:10.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.270+0000 7f635c76b640 1 --2- 172.21.15.83:0/1066658190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63540b3990 0x7f6354070470 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:10.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.270+0000 7f635c76b640 1 -- 172.21.15.83:0/1066658190 >> 172.21.15.83:0/1066658190 conn(0x7f635406abe0 msgr2=0x7f635406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:10.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.270+0000 7f635c76b640 1 -- 172.21.15.83:0/1066658190 shutdown_connections 2024-04-17T10:48:10.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.270+0000 7f635c76b640 1 -- 172.21.15.83:0/1066658190 wait complete. 2024-04-17T10:48:10.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.274+0000 7f635c76b640 1 Processor -- start 2024-04-17T10:48:10.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.274+0000 7f635c76b640 1 -- start start 2024-04-17T10:48:10.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.274+0000 7f635c76b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63540709b0 0x7f635415c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:10.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.274+0000 7f635c76b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63540b3990 0x7f6354145c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:10.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.274+0000 7f635c76b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63540cf060 0x7f63541568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:10.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.274+0000 7f635c76b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63540cf440 con 0x7f63540b3990 2024-04-17T10:48:10.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.274+0000 7f635c76b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63540b4770 con 0x7f63540709b0 2024-04-17T10:48:10.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.274+0000 7f635c76b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63540cf5c0 con 0x7f63540cf060 2024-04-17T10:48:10.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.274+0000 7f6359d06640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63540b3990 0x7f6354145c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:10.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.274+0000 7f6359d06640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63540b3990 0x7f6354145c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39736/0 (socket says 172.21.15.83:39736) 2024-04-17T10:48:10.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.274+0000 7f6359d06640 1 -- 172.21.15.83:0/1703575234 learned_addr learned my addr 172.21.15.83:0/1703575234 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:10.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.274+0000 7f6359d06640 1 -- 172.21.15.83:0/1703575234 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63540cf060 msgr2=0x7f63541568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:10.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.274+0000 7f6359d06640 1 --2- 172.21.15.83:0/1703575234 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63540cf060 0x7f63541568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:10.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.274+0000 7f6359d06640 1 -- 172.21.15.83:0/1703575234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63540709b0 msgr2=0x7f635415c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:10.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.274+0000 7f6359d06640 1 --2- 172.21.15.83:0/1703575234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63540709b0 0x7f635415c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:10.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.274+0000 7f6359d06640 1 -- 172.21.15.83:0/1703575234 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6354146150 con 0x7f63540b3990 2024-04-17T10:48:10.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.274+0000 7f6359d06640 1 --2- 172.21.15.83:0/1703575234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63540b3990 0x7f6354145c10 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f63440b1040 tx=0x7f634401c080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:10.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.274+0000 7f634affd640 1 -- 172.21.15.83:0/1703575234 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63440b3040 con 0x7f63540b3990 2024-04-17T10:48:10.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.274+0000 7f634affd640 1 -- 172.21.15.83:0/1703575234 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6344004220 con 0x7f63540b3990 2024-04-17T10:48:10.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.274+0000 7f635c76b640 1 -- 172.21.15.83:0/1703575234 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63541f85c0 con 0x7f63540b3990 2024-04-17T10:48:10.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.274+0000 7f634affd640 1 -- 172.21.15.83:0/1703575234 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6344003620 con 0x7f63540b3990 2024-04-17T10:48:10.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.274+0000 7f635c76b640 1 -- 172.21.15.83:0/1703575234 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f635414d7d0 con 0x7f63540b3990 2024-04-17T10:48:10.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.274+0000 7f635c76b640 1 -- 172.21.15.83:0/1703575234 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6354074f90 con 0x7f63540b3990 2024-04-17T10:48:10.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.274+0000 7f634affd640 1 -- 172.21.15.83:0/1703575234 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f63440037c0 con 0x7f63540b3990 2024-04-17T10:48:10.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.278+0000 7f634affd640 1 --2- 172.21.15.83:0/1703575234 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6338066f60 0x7f6338069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:10.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.278+0000 7f634affd640 1 -- 172.21.15.83:0/1703575234 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f634411f050 con 0x7f63540b3990 2024-04-17T10:48:10.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.278+0000 7f635a507640 1 --2- 172.21.15.83:0/1703575234 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6338066f60 0x7f6338069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:10.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.278+0000 7f635a507640 1 --2- 172.21.15.83:0/1703575234 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6338066f60 0x7f6338069420 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6350000f70 tx=0x7f63500079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:10.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.282+0000 7f634affd640 1 -- 172.21.15.83:0/1703575234 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6354146150 con 0x7f63540b3990 2024-04-17T10:48:10.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.438+0000 7f635c76b640 1 -- 172.21.15.83:0/1703575234 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6354074480 con 0x7f63540b3990 2024-04-17T10:48:10.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.442+0000 7f634affd640 1 -- 172.21.15.83:0/1703575234 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16547 (secure 0 0 0) 0x7f6344017460 con 0x7f63540b3990 2024-04-17T10:48:10.439 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:10.439 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":103,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:47:31.018926+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:47:27.619432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-17T10:47:28.379472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:48:10.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.446+0000 7f635c76b640 1 -- 172.21.15.83:0/1703575234 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6338066f60 msgr2=0x7f6338069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:10.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.446+0000 7f635c76b640 1 --2- 172.21.15.83:0/1703575234 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6338066f60 0x7f6338069420 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6350000f70 tx=0x7f63500079c0 comp rx=0 tx=0).stop 2024-04-17T10:48:10.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.446+0000 7f635c76b640 1 -- 172.21.15.83:0/1703575234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63540b3990 msgr2=0x7f6354145c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:10.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.446+0000 7f635c76b640 1 --2- 172.21.15.83:0/1703575234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63540b3990 0x7f6354145c10 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f63440b1040 tx=0x7f634401c080 comp rx=0 tx=0).stop 2024-04-17T10:48:10.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.446+0000 7f635c76b640 1 -- 172.21.15.83:0/1703575234 shutdown_connections 2024-04-17T10:48:10.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.446+0000 7f635c76b640 1 --2- 172.21.15.83:0/1703575234 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6338066f60 0x7f6338069420 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:10.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.446+0000 7f635c76b640 1 --2- 172.21.15.83:0/1703575234 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63540cf060 0x7f63541568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:10.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.446+0000 7f635c76b640 1 --2- 172.21.15.83:0/1703575234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63540b3990 0x7f6354145c10 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:10.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.446+0000 7f635c76b640 1 --2- 172.21.15.83:0/1703575234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63540709b0 0x7f635415c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:10.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.446+0000 7f635c76b640 1 -- 172.21.15.83:0/1703575234 >> 172.21.15.83:0/1703575234 conn(0x7f635406abe0 msgr2=0x7f63540aedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:10.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.446+0000 7f635c76b640 1 -- 172.21.15.83:0/1703575234 shutdown_connections 2024-04-17T10:48:10.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.446+0000 7f635c76b640 1 -- 172.21.15.83:0/1703575234 wait complete. 2024-04-17T10:48:10.458 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:47:27.619432+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:47:28.379472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:48:10.459 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:48:10.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.574+0000 7f7a00913640 1 Processor -- start 2024-04-17T10:48:10.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.574+0000 7f7a00913640 1 -- start start 2024-04-17T10:48:10.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.574+0000 7f7a00913640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f79fc06ae10 0x7f79fc06b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:10.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.574+0000 7f7a00913640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f79fc070aa0 0x7f79fc070f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:10.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.574+0000 7f7a00913640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79fc1625d0 con 0x7f79fc15e330 2024-04-17T10:48:10.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.574+0000 7f7a00913640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79fc15eeb0 con 0x7f79fc06ae10 2024-04-17T10:48:10.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.574+0000 7f7a00913640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79fc06b730 con 0x7f79fc070aa0 2024-04-17T10:48:10.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.574+0000 7f79fa59c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f79fc06ae10 0x7f79fc06b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:10.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.574+0000 7f79fa59c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f79fc06ae10 0x7f79fc06b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39748/0 (socket says 172.21.15.83:39748) 2024-04-17T10:48:10.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.574+0000 7f79fa59c640 1 -- 172.21.15.83:0/1925929000 learned_addr learned my addr 172.21.15.83:0/1925929000 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:10.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.574+0000 7f79fad9d640 1 --2- 172.21.15.83:0/1925929000 >> v2:172.21.15.167:3300/0 conn(0x7f79fc070aa0 0x7f79fc070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:10.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.574+0000 7f79fa59c640 1 -- 172.21.15.83:0/1925929000 >> v2:172.21.15.167:3300/0 conn(0x7f79fc070aa0 msgr2=0x7f79fc070f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:10.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.574+0000 7f79fa59c640 1 --2- 172.21.15.83:0/1925929000 >> v2:172.21.15.167:3300/0 conn(0x7f79fc070aa0 0x7f79fc070f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:10.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.574+0000 7f79fa59c640 1 -- 172.21.15.83:0/1925929000 >> v1:172.21.15.167:6789/0 conn(0x7f79fc15e330 legacy=0x7f79fc15e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:10.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.574+0000 7f79fa59c640 1 -- 172.21.15.83:0/1925929000 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79fc161810 con 0x7f79fc06ae10 2024-04-17T10:48:10.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.578+0000 7f79fa59c640 1 --2- 172.21.15.83:0/1925929000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79fc06ae10 0x7f79fc06b1f0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f79fc15bc70 tx=0x7f79ec01c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3486444be3c35b25 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:10.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.578+0000 7f79f959a640 1 -- 172.21.15.83:0/1925929000 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0a5610 con 0x7f79fc06ae10 2024-04-17T10:48:10.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.578+0000 7f79f959a640 1 -- 172.21.15.83:0/1925929000 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ec0a57b0 con 0x7f79fc06ae10 2024-04-17T10:48:10.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.578+0000 7f79f959a640 1 -- 172.21.15.83:0/1925929000 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0a5b80 con 0x7f79fc06ae10 2024-04-17T10:48:10.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.578+0000 7f7a00913640 1 -- 172.21.15.83:0/1925929000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79fc06ae10 msgr2=0x7f79fc06b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:10.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.578+0000 7f7a00913640 1 --2- 172.21.15.83:0/1925929000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79fc06ae10 0x7f79fc06b1f0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f79fc15bc70 tx=0x7f79ec01c860 comp rx=0 tx=0).stop 2024-04-17T10:48:10.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.578+0000 7f7a00913640 1 -- 172.21.15.83:0/1925929000 shutdown_connections 2024-04-17T10:48:10.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.578+0000 7f7a00913640 1 --2- 172.21.15.83:0/1925929000 >> v2:172.21.15.167:3300/0 conn(0x7f79fc070aa0 0x7f79fc070f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:10.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.578+0000 7f7a00913640 1 --2- 172.21.15.83:0/1925929000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79fc06ae10 0x7f79fc06b1f0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:10.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.578+0000 7f7a00913640 1 -- 172.21.15.83:0/1925929000 >> 172.21.15.83:0/1925929000 conn(0x7f79fc0b15b0 msgr2=0x7f79fc0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:10.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.578+0000 7f7a00913640 1 -- 172.21.15.83:0/1925929000 shutdown_connections 2024-04-17T10:48:10.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.578+0000 7f7a00913640 1 -- 172.21.15.83:0/1925929000 wait complete. 2024-04-17T10:48:10.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.578+0000 7f7a00913640 1 Processor -- start 2024-04-17T10:48:10.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.578+0000 7f7a00913640 1 -- start start 2024-04-17T10:48:10.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.578+0000 7f7a00913640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79fc06ae10 0x7f79fc157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:10.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.578+0000 7f7a00913640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79fc070aa0 0x7f79fc158740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:10.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.582+0000 7f7a00913640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79fc15e330 0x7f79fc15c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:10.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.582+0000 7f7a00913640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc0cf470 con 0x7f79fc06ae10 2024-04-17T10:48:10.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.582+0000 7f7a00913640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc0cf2f0 con 0x7f79fc070aa0 2024-04-17T10:48:10.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.582+0000 7f7a00913640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc16aa10 con 0x7f79fc15e330 2024-04-17T10:48:10.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.582+0000 7f79fa59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79fc06ae10 0x7f79fc157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:10.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.582+0000 7f79fa59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79fc06ae10 0x7f79fc157ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39764/0 (socket says 172.21.15.83:39764) 2024-04-17T10:48:10.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.582+0000 7f79fa59c640 1 -- 172.21.15.83:0/125729812 learned_addr learned my addr 172.21.15.83:0/125729812 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:10.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.582+0000 7f79fa59c640 1 -- 172.21.15.83:0/125729812 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79fc15e330 msgr2=0x7f79fc15c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:10.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.582+0000 7f79fa59c640 1 --2- 172.21.15.83:0/125729812 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79fc15e330 0x7f79fc15c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:10.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.582+0000 7f79fa59c640 1 -- 172.21.15.83:0/125729812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79fc070aa0 msgr2=0x7f79fc158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:10.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.582+0000 7f79fa59c640 1 --2- 172.21.15.83:0/125729812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79fc070aa0 0x7f79fc158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:10.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.582+0000 7f79fa59c640 1 -- 172.21.15.83:0/125729812 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79fc142d70 con 0x7f79fc06ae10 2024-04-17T10:48:10.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.582+0000 7f79fa59c640 1 --2- 172.21.15.83:0/125729812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79fc06ae10 0x7f79fc157ee0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0af040 tx=0x7f79ec019600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:10.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.582+0000 7f79eaffd640 1 -- 172.21.15.83:0/125729812 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0bc020 con 0x7f79fc06ae10 2024-04-17T10:48:10.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.582+0000 7f79eaffd640 1 -- 172.21.15.83:0/125729812 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ec0b8690 con 0x7f79fc06ae10 2024-04-17T10:48:10.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.582+0000 7f79eaffd640 1 -- 172.21.15.83:0/125729812 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec009510 con 0x7f79fc06ae10 2024-04-17T10:48:10.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.582+0000 7f7a00913640 1 -- 172.21.15.83:0/125729812 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79fc144ef0 con 0x7f79fc06ae10 2024-04-17T10:48:10.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.582+0000 7f7a00913640 1 -- 172.21.15.83:0/125729812 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79fc1454b0 con 0x7f79fc06ae10 2024-04-17T10:48:10.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.582+0000 7f7a00913640 1 -- 172.21.15.83:0/125729812 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79fc074f90 con 0x7f79fc06ae10 2024-04-17T10:48:10.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.582+0000 7f79eaffd640 1 -- 172.21.15.83:0/125729812 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f79ec0bf020 con 0x7f79fc06ae10 2024-04-17T10:48:10.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.582+0000 7f79eaffd640 1 --2- 172.21.15.83:0/125729812 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f79dc066f60 0x7f79dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:10.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.582+0000 7f79eaffd640 1 -- 172.21.15.83:0/125729812 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79ec11ffb0 con 0x7f79fc06ae10 2024-04-17T10:48:10.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.582+0000 7f79f9d9b640 1 --2- 172.21.15.83:0/125729812 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f79dc066f60 0x7f79dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:10.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.582+0000 7f79f9d9b640 1 --2- 172.21.15.83:0/125729812 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f79dc066f60 0x7f79dc069420 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f79fc1fa420 tx=0x7f79f00079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:10.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.586+0000 7f79eaffd640 1 -- 172.21.15.83:0/125729812 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f79ec0c6030 con 0x7f79fc06ae10 2024-04-17T10:48:10.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.738+0000 7f7a00913640 1 -- 172.21.15.83:0/125729812 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f79fc072740 con 0x7f79fc06ae10 2024-04-17T10:48:10.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.738+0000 7f79eaffd640 1 -- 172.21.15.83:0/125729812 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) v1 ==== 93+0+10 (secure 0 0 0) 0x7f79ec0165a0 con 0x7f79fc06ae10 2024-04-17T10:48:10.735 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:48:10.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.742+0000 7f7a00913640 1 -- 172.21.15.83:0/125729812 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f79dc066f60 msgr2=0x7f79dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:10.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.742+0000 7f7a00913640 1 --2- 172.21.15.83:0/125729812 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f79dc066f60 0x7f79dc069420 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f79fc1fa420 tx=0x7f79f00079c0 comp rx=0 tx=0).stop 2024-04-17T10:48:10.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.742+0000 7f7a00913640 1 -- 172.21.15.83:0/125729812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79fc06ae10 msgr2=0x7f79fc157ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:10.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.742+0000 7f7a00913640 1 --2- 172.21.15.83:0/125729812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79fc06ae10 0x7f79fc157ee0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0af040 tx=0x7f79ec019600 comp rx=0 tx=0).stop 2024-04-17T10:48:10.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.742+0000 7f7a00913640 1 -- 172.21.15.83:0/125729812 shutdown_connections 2024-04-17T10:48:10.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.742+0000 7f7a00913640 1 --2- 172.21.15.83:0/125729812 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f79dc066f60 0x7f79dc069420 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:10.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.742+0000 7f7a00913640 1 --2- 172.21.15.83:0/125729812 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79fc15e330 0x7f79fc15c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:10.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.742+0000 7f7a00913640 1 --2- 172.21.15.83:0/125729812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79fc070aa0 0x7f79fc158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:10.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.742+0000 7f7a00913640 1 --2- 172.21.15.83:0/125729812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79fc06ae10 0x7f79fc157ee0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:10.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.742+0000 7f7a00913640 1 -- 172.21.15.83:0/125729812 >> 172.21.15.83:0/125729812 conn(0x7f79fc0b15b0 msgr2=0x7f79fc06d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:10.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.742+0000 7f7a00913640 1 -- 172.21.15.83:0/125729812 shutdown_connections 2024-04-17T10:48:10.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.742+0000 7f7a00913640 1 -- 172.21.15.83:0/125729812 wait complete. 2024-04-17T10:48:10.752 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:48:10.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.874+0000 7fea725a5640 1 Processor -- start 2024-04-17T10:48:10.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.874+0000 7fea725a5640 1 -- start start 2024-04-17T10:48:10.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.878+0000 7fea725a5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea6c0afde0 0x7fea6c15ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:10.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.878+0000 7fea725a5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fea6c0b4910 0x7fea6c15b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:10.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.878+0000 7fea725a5640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea6c15e590 con 0x7fea6c15ae20 2024-04-17T10:48:10.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.878+0000 7fea725a5640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea6c0b01c0 con 0x7fea6c0b4910 2024-04-17T10:48:10.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.878+0000 7fea725a5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c15b8e0 con 0x7fea6c0afde0 2024-04-17T10:48:10.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.878+0000 7fea6bfff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea6c0afde0 0x7fea6c15ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:10.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.878+0000 7fea70b42640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fea6c0b4910 0x7fea6c15b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:10.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.878+0000 7fea6bfff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea6c0afde0 0x7fea6c15ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:44270/0 (socket says 172.21.15.83:44270) 2024-04-17T10:48:10.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.878+0000 7fea6bfff640 1 -- 172.21.15.83:0/3413053278 learned_addr learned my addr 172.21.15.83:0/3413053278 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:10.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.878+0000 7fea6bfff640 1 -- 172.21.15.83:0/3413053278 >> v2:172.21.15.167:3300/0 conn(0x7fea6c0b4910 msgr2=0x7fea6c15b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:10.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.878+0000 7fea6bfff640 1 --2- 172.21.15.83:0/3413053278 >> v2:172.21.15.167:3300/0 conn(0x7fea6c0b4910 0x7fea6c15b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:10.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.878+0000 7fea6bfff640 1 -- 172.21.15.83:0/3413053278 >> v1:172.21.15.167:6789/0 conn(0x7fea6c15ae20 legacy=0x7fea6c0b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:10.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.878+0000 7fea6bfff640 1 -- 172.21.15.83:0/3413053278 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea6c15bbc0 con 0x7fea6c0afde0 2024-04-17T10:48:10.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.878+0000 7fea70b42640 1 --2- 172.21.15.83:0/3413053278 >> v2:172.21.15.167:3300/0 conn(0x7fea6c0b4910 0x7fea6c15b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:48:10.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.882+0000 7fea6bfff640 1 --2- 172.21.15.83:0/3413053278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea6c0afde0 0x7fea6c15ed70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009870 tx=0x7fea5c01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78fec798d9cac195 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:10.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.882+0000 7fea6b7fe640 1 -- 172.21.15.83:0/3413053278 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5c0a9640 con 0x7fea6c0afde0 2024-04-17T10:48:10.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.882+0000 7fea6b7fe640 1 -- 172.21.15.83:0/3413053278 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea5c0a97e0 con 0x7fea6c0afde0 2024-04-17T10:48:10.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.882+0000 7fea6b7fe640 1 -- 172.21.15.83:0/3413053278 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5c0a9bb0 con 0x7fea6c0afde0 2024-04-17T10:48:10.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.882+0000 7fea725a5640 1 -- 172.21.15.83:0/3413053278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea6c0afde0 msgr2=0x7fea6c15ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:10.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.882+0000 7fea725a5640 1 --2- 172.21.15.83:0/3413053278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea6c0afde0 0x7fea6c15ed70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009870 tx=0x7fea5c01c560 comp rx=0 tx=0).stop 2024-04-17T10:48:10.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.882+0000 7fea725a5640 1 -- 172.21.15.83:0/3413053278 shutdown_connections 2024-04-17T10:48:10.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.882+0000 7fea725a5640 1 --2- 172.21.15.83:0/3413053278 >> v2:172.21.15.167:3300/0 conn(0x7fea6c0b4910 0x7fea6c15b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:10.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.882+0000 7fea725a5640 1 --2- 172.21.15.83:0/3413053278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea6c0afde0 0x7fea6c15ed70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:10.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.882+0000 7fea725a5640 1 -- 172.21.15.83:0/3413053278 >> 172.21.15.83:0/3413053278 conn(0x7fea6c06bcf0 msgr2=0x7fea6c06c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:10.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.882+0000 7fea725a5640 1 -- 172.21.15.83:0/3413053278 shutdown_connections 2024-04-17T10:48:10.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.882+0000 7fea725a5640 1 -- 172.21.15.83:0/3413053278 wait complete. 2024-04-17T10:48:10.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.882+0000 7fea725a5640 1 Processor -- start 2024-04-17T10:48:10.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.882+0000 7fea725a5640 1 -- start start 2024-04-17T10:48:10.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.882+0000 7fea725a5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea6c0afde0 0x7fea6c149e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:10.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.882+0000 7fea725a5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea6c0b4910 0x7fea6c06ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:10.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.882+0000 7fea725a5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea6c15ae20 0x7fea6c159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:10.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.882+0000 7fea725a5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c15c970 con 0x7fea6c0b4910 2024-04-17T10:48:10.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.882+0000 7fea725a5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c15c7f0 con 0x7fea6c15ae20 2024-04-17T10:48:10.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.882+0000 7fea725a5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c15caf0 con 0x7fea6c0afde0 2024-04-17T10:48:10.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.886+0000 7fea63fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea6c0b4910 0x7fea6c06ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:10.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.886+0000 7fea63fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea6c0b4910 0x7fea6c06ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39774/0 (socket says 172.21.15.83:39774) 2024-04-17T10:48:10.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.886+0000 7fea63fff640 1 -- 172.21.15.83:0/4143654417 learned_addr learned my addr 172.21.15.83:0/4143654417 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:10.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.886+0000 7fea6bfff640 1 --2- 172.21.15.83:0/4143654417 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea6c0afde0 0x7fea6c149e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:10.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.886+0000 7fea70b42640 1 --2- 172.21.15.83:0/4143654417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea6c15ae20 0x7fea6c159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:10.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.886+0000 7fea63fff640 1 -- 172.21.15.83:0/4143654417 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea6c0afde0 msgr2=0x7fea6c149e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:10.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.886+0000 7fea63fff640 1 --2- 172.21.15.83:0/4143654417 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea6c0afde0 0x7fea6c149e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:10.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.886+0000 7fea63fff640 1 -- 172.21.15.83:0/4143654417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea6c15ae20 msgr2=0x7fea6c159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:10.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.886+0000 7fea63fff640 1 --2- 172.21.15.83:0/4143654417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea6c15ae20 0x7fea6c159a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:10.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.886+0000 7fea63fff640 1 -- 172.21.15.83:0/4143654417 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea6c06b9c0 con 0x7fea6c0b4910 2024-04-17T10:48:10.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.886+0000 7fea63fff640 1 --2- 172.21.15.83:0/4143654417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea6c0b4910 0x7fea6c06ad70 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fea5400c420 tx=0x7fea5400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:10.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.886+0000 7fea697fa640 1 -- 172.21.15.83:0/4143654417 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea540052d0 con 0x7fea6c0b4910 2024-04-17T10:48:10.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.886+0000 7fea697fa640 1 -- 172.21.15.83:0/4143654417 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea54005470 con 0x7fea6c0b4910 2024-04-17T10:48:10.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.886+0000 7fea697fa640 1 -- 172.21.15.83:0/4143654417 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea54007260 con 0x7fea6c0b4910 2024-04-17T10:48:10.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.886+0000 7fea725a5640 1 -- 172.21.15.83:0/4143654417 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea6c1592a0 con 0x7fea6c0b4910 2024-04-17T10:48:10.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.886+0000 7fea725a5640 1 -- 172.21.15.83:0/4143654417 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea6c06b400 con 0x7fea6c0b4910 2024-04-17T10:48:10.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.886+0000 7fea725a5640 1 -- 172.21.15.83:0/4143654417 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea6c074f90 con 0x7fea6c0b4910 2024-04-17T10:48:10.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.886+0000 7fea697fa640 1 -- 172.21.15.83:0/4143654417 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fea54007400 con 0x7fea6c0b4910 2024-04-17T10:48:10.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.886+0000 7fea697fa640 1 --2- 172.21.15.83:0/4143654417 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fea4c066f60 0x7fea4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:10.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.886+0000 7fea697fa640 1 -- 172.21.15.83:0/4143654417 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea54084420 con 0x7fea6c0b4910 2024-04-17T10:48:10.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.886+0000 7fea6bfff640 1 --2- 172.21.15.83:0/4143654417 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fea4c066f60 0x7fea4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:10.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.886+0000 7fea6bfff640 1 --2- 172.21.15.83:0/4143654417 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fea4c066f60 0x7fea4c069420 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0ae000 tx=0x7fea5c0b3040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:10.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:10.890+0000 7fea697fa640 1 -- 172.21.15.83:0/4143654417 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fea54054e70 con 0x7fea6c0b4910 2024-04-17T10:48:11.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.038+0000 7fea725a5640 1 -- 172.21.15.83:0/4143654417 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fea6c158db0 con 0x7fea6c0b4910 2024-04-17T10:48:11.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.038+0000 7fea697fa640 1 -- 172.21.15.83:0/4143654417 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) v1 ==== 104+0+11 (secure 0 0 0) 0x7fea54058f10 con 0x7fea6c0b4910 2024-04-17T10:48:11.034 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:48:11.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.038+0000 7fea725a5640 1 -- 172.21.15.83:0/4143654417 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fea4c066f60 msgr2=0x7fea4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:11.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.038+0000 7fea725a5640 1 --2- 172.21.15.83:0/4143654417 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fea4c066f60 0x7fea4c069420 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0ae000 tx=0x7fea5c0b3040 comp rx=0 tx=0).stop 2024-04-17T10:48:11.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.038+0000 7fea725a5640 1 -- 172.21.15.83:0/4143654417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea6c0b4910 msgr2=0x7fea6c06ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:11.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.038+0000 7fea725a5640 1 --2- 172.21.15.83:0/4143654417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea6c0b4910 0x7fea6c06ad70 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fea5400c420 tx=0x7fea5400c8e0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.038+0000 7fea725a5640 1 -- 172.21.15.83:0/4143654417 shutdown_connections 2024-04-17T10:48:11.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.038+0000 7fea725a5640 1 --2- 172.21.15.83:0/4143654417 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fea4c066f60 0x7fea4c069420 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.038+0000 7fea725a5640 1 --2- 172.21.15.83:0/4143654417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea6c15ae20 0x7fea6c159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.038+0000 7fea725a5640 1 --2- 172.21.15.83:0/4143654417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea6c0b4910 0x7fea6c06ad70 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.038+0000 7fea725a5640 1 --2- 172.21.15.83:0/4143654417 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea6c0afde0 0x7fea6c149e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.038+0000 7fea725a5640 1 -- 172.21.15.83:0/4143654417 >> 172.21.15.83:0/4143654417 conn(0x7fea6c06bcf0 msgr2=0x7fea6c14acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:11.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.038+0000 7fea725a5640 1 -- 172.21.15.83:0/4143654417 shutdown_connections 2024-04-17T10:48:11.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.038+0000 7fea725a5640 1 -- 172.21.15.83:0/4143654417 wait complete. 2024-04-17T10:48:11.052 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:48:11.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.178+0000 7fc5af63c640 1 Processor -- start 2024-04-17T10:48:11.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.178+0000 7fc5af63c640 1 -- start start 2024-04-17T10:48:11.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.178+0000 7fc5af63c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a815b6e0 0x7fc5a8071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:11.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.178+0000 7fc5af63c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc5a815bac0 0x7fc5a80cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:11.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.178+0000 7fc5af63c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc5a8070470 0x7fc5a806ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:11.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.178+0000 7fc5af63c640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a806c1f0 con 0x7fc5a8070470 2024-04-17T10:48:11.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.178+0000 7fc5af63c640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a80cdd30 con 0x7fc5a815bac0 2024-04-17T10:48:11.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.178+0000 7fc5af63c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a8070a10 con 0x7fc5a815b6e0 2024-04-17T10:48:11.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.178+0000 7fc5adbd9640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc5a8070470 0x7fc5a806ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:11.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.178+0000 7fc5adbd9640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc5a8070470 0x7fc5a806ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39788/0 (socket says 172.21.15.83:39788) 2024-04-17T10:48:11.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.178+0000 7fc5adbd9640 1 -- 172.21.15.83:0/4210257405 learned_addr learned my addr 172.21.15.83:0/4210257405 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:11.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.178+0000 7fc5acbd7640 1 --2- 172.21.15.83:0/4210257405 >> v2:172.21.15.167:3300/0 conn(0x7fc5a815bac0 0x7fc5a80cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:11.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.178+0000 7fc5ad3d8640 1 --2- 172.21.15.83:0/4210257405 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a815b6e0 0x7fc5a8071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:11.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.178+0000 7fc5adbd9640 1 -- 172.21.15.83:0/4210257405 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a815b6e0 msgr2=0x7fc5a8071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:11.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.178+0000 7fc5adbd9640 1 --2- 172.21.15.83:0/4210257405 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a815b6e0 0x7fc5a8071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.178+0000 7fc5adbd9640 1 -- 172.21.15.83:0/4210257405 >> v2:172.21.15.167:3300/0 conn(0x7fc5a815bac0 msgr2=0x7fc5a80cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:11.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.178+0000 7fc5adbd9640 1 --2- 172.21.15.83:0/4210257405 >> v2:172.21.15.167:3300/0 conn(0x7fc5a815bac0 0x7fc5a80cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.178+0000 7fc5adbd9640 1 -- 172.21.15.83:0/4210257405 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a8070cf0 con 0x7fc5a8070470 2024-04-17T10:48:11.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.182+0000 7fc5adbd9640 1 --2- 172.21.15.83:0/4210257405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a8070470 0x7fc5a806ae10 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fc5a80b49f0 tx=0x7fc5a001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2f28a5737035ea server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:11.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.182+0000 7fc59ffff640 1 -- 172.21.15.83:0/4210257405 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a00a5670 con 0x7fc5a8070470 2024-04-17T10:48:11.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.182+0000 7fc59ffff640 1 -- 172.21.15.83:0/4210257405 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a00a5810 con 0x7fc5a8070470 2024-04-17T10:48:11.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.182+0000 7fc59ffff640 1 -- 172.21.15.83:0/4210257405 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a00a5be0 con 0x7fc5a8070470 2024-04-17T10:48:11.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.182+0000 7fc5af63c640 1 -- 172.21.15.83:0/4210257405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a8070470 msgr2=0x7fc5a806ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:11.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.182+0000 7fc5af63c640 1 --2- 172.21.15.83:0/4210257405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a8070470 0x7fc5a806ae10 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fc5a80b49f0 tx=0x7fc5a001c9f0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.182+0000 7fc5af63c640 1 -- 172.21.15.83:0/4210257405 shutdown_connections 2024-04-17T10:48:11.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.182+0000 7fc5af63c640 1 --2- 172.21.15.83:0/4210257405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a8070470 0x7fc5a806ae10 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.182+0000 7fc5af63c640 1 --2- 172.21.15.83:0/4210257405 >> v2:172.21.15.167:3300/0 conn(0x7fc5a815bac0 0x7fc5a80cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.182+0000 7fc5af63c640 1 --2- 172.21.15.83:0/4210257405 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a815b6e0 0x7fc5a8071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.182+0000 7fc5af63c640 1 -- 172.21.15.83:0/4210257405 >> 172.21.15.83:0/4210257405 conn(0x7fc5a80b15b0 msgr2=0x7fc5a80b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:11.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.182+0000 7fc5af63c640 1 -- 172.21.15.83:0/4210257405 shutdown_connections 2024-04-17T10:48:11.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.182+0000 7fc5af63c640 1 -- 172.21.15.83:0/4210257405 wait complete. 2024-04-17T10:48:11.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.182+0000 7fc5af63c640 1 Processor -- start 2024-04-17T10:48:11.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.182+0000 7fc5af63c640 1 -- start start 2024-04-17T10:48:11.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.182+0000 7fc5af63c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a8070470 0x7fc5a8146f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:11.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.186+0000 7fc5af63c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a815b6e0 0x7fc5a80aa490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:11.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.186+0000 7fc5af63c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5a815bac0 0x7fc5a815e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:11.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.186+0000 7fc5af63c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a806d270 con 0x7fc5a815b6e0 2024-04-17T10:48:11.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.186+0000 7fc5af63c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a806d0f0 con 0x7fc5a815bac0 2024-04-17T10:48:11.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.186+0000 7fc5af63c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a806b760 con 0x7fc5a8070470 2024-04-17T10:48:11.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.186+0000 7fc5acbd7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a815b6e0 0x7fc5a80aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:11.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.186+0000 7fc5acbd7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a815b6e0 0x7fc5a80aa490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39794/0 (socket says 172.21.15.83:39794) 2024-04-17T10:48:11.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.186+0000 7fc5acbd7640 1 -- 172.21.15.83:0/1492435995 learned_addr learned my addr 172.21.15.83:0/1492435995 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:11.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.186+0000 7fc5adbd9640 1 --2- 172.21.15.83:0/1492435995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5a815bac0 0x7fc5a815e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:11.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.186+0000 7fc5ad3d8640 1 --2- 172.21.15.83:0/1492435995 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a8070470 0x7fc5a8146f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:11.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.186+0000 7fc5acbd7640 1 -- 172.21.15.83:0/1492435995 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a8070470 msgr2=0x7fc5a8146f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:11.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.186+0000 7fc5acbd7640 1 --2- 172.21.15.83:0/1492435995 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a8070470 0x7fc5a8146f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.186+0000 7fc5acbd7640 1 -- 172.21.15.83:0/1492435995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5a815bac0 msgr2=0x7fc5a815e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:11.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.186+0000 7fc5acbd7640 1 --2- 172.21.15.83:0/1492435995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5a815bac0 0x7fc5a815e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.186+0000 7fc5acbd7640 1 -- 172.21.15.83:0/1492435995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a8069060 con 0x7fc5a815b6e0 2024-04-17T10:48:11.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.186+0000 7fc5adbd9640 1 --2- 172.21.15.83:0/1492435995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5a815bac0 0x7fc5a815e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:11.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.186+0000 7fc5acbd7640 1 --2- 172.21.15.83:0/1492435995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a815b6e0 0x7fc5a80aa490 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fc59800d800 tx=0x7fc59800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:11.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.186+0000 7fc59dffb640 1 -- 172.21.15.83:0/1492435995 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598004020 con 0x7fc5a815b6e0 2024-04-17T10:48:11.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.186+0000 7fc59dffb640 1 -- 172.21.15.83:0/1492435995 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5980041c0 con 0x7fc5a815b6e0 2024-04-17T10:48:11.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.186+0000 7fc59dffb640 1 -- 172.21.15.83:0/1492435995 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5980043b0 con 0x7fc5a815b6e0 2024-04-17T10:48:11.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.186+0000 7fc5af63c640 1 -- 172.21.15.83:0/1492435995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a80a9430 con 0x7fc5a815b6e0 2024-04-17T10:48:11.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.186+0000 7fc5af63c640 1 -- 172.21.15.83:0/1492435995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a80a9c90 con 0x7fc5a815b6e0 2024-04-17T10:48:11.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.186+0000 7fc5af63c640 1 -- 172.21.15.83:0/1492435995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a8074f90 con 0x7fc5a815b6e0 2024-04-17T10:48:11.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.186+0000 7fc59dffb640 1 -- 172.21.15.83:0/1492435995 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc59801b020 con 0x7fc5a815b6e0 2024-04-17T10:48:11.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.186+0000 7fc59dffb640 1 --2- 172.21.15.83:0/1492435995 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc58c066f60 0x7fc58c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:11.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.186+0000 7fc5ad3d8640 1 --2- 172.21.15.83:0/1492435995 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc58c066f60 0x7fc58c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:11.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.186+0000 7fc59dffb640 1 -- 172.21.15.83:0/1492435995 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc598084e50 con 0x7fc5a815b6e0 2024-04-17T10:48:11.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.186+0000 7fc5ad3d8640 1 --2- 172.21.15.83:0/1492435995 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc58c066f60 0x7fc58c069420 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009790 tx=0x7fc5a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:11.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.190+0000 7fc59dffb640 1 -- 172.21.15.83:0/1492435995 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc598055dd0 con 0x7fc5a815b6e0 2024-04-17T10:48:11.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.334+0000 7fc5af63c640 1 -- 172.21.15.83:0/1492435995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc5a8074480 con 0x7fc5a815b6e0 2024-04-17T10:48:11.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.338+0000 7fc59dffb640 1 -- 172.21.15.83:0/1492435995 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc5980556d0 con 0x7fc5a815b6e0 2024-04-17T10:48:11.333 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:48:11.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.338+0000 7fc5af63c640 1 -- 172.21.15.83:0/1492435995 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc58c066f60 msgr2=0x7fc58c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:11.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.338+0000 7fc5af63c640 1 --2- 172.21.15.83:0/1492435995 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc58c066f60 0x7fc58c069420 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009790 tx=0x7fc5a4008040 comp rx=0 tx=0).stop 2024-04-17T10:48:11.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.338+0000 7fc5af63c640 1 -- 172.21.15.83:0/1492435995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a815b6e0 msgr2=0x7fc5a80aa490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:11.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.338+0000 7fc5af63c640 1 --2- 172.21.15.83:0/1492435995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a815b6e0 0x7fc5a80aa490 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fc59800d800 tx=0x7fc59800dcc0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.338+0000 7fc5af63c640 1 -- 172.21.15.83:0/1492435995 shutdown_connections 2024-04-17T10:48:11.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.338+0000 7fc5af63c640 1 --2- 172.21.15.83:0/1492435995 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc58c066f60 0x7fc58c069420 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.338+0000 7fc5af63c640 1 --2- 172.21.15.83:0/1492435995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5a815bac0 0x7fc5a815e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.338+0000 7fc5af63c640 1 --2- 172.21.15.83:0/1492435995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a815b6e0 0x7fc5a80aa490 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.338+0000 7fc5af63c640 1 --2- 172.21.15.83:0/1492435995 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a8070470 0x7fc5a8146f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.338+0000 7fc5af63c640 1 -- 172.21.15.83:0/1492435995 >> 172.21.15.83:0/1492435995 conn(0x7fc5a80b15b0 msgr2=0x7fc5a80b00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:11.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.338+0000 7fc5af63c640 1 -- 172.21.15.83:0/1492435995 shutdown_connections 2024-04-17T10:48:11.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.338+0000 7fc5af63c640 1 -- 172.21.15.83:0/1492435995 wait complete. 2024-04-17T10:48:11.351 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:48:11.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.474+0000 7ff78f6c5640 1 Processor -- start 2024-04-17T10:48:11.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.474+0000 7ff78f6c5640 1 -- start start 2024-04-17T10:48:11.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.478+0000 7ff78f6c5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff78815ba30 0x7ff788072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:11.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.478+0000 7ff78f6c5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff7880cfbe0 0x7ff78806c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:11.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.478+0000 7ff78f6c5640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff78806e320 con 0x7ff78815c960 2024-04-17T10:48:11.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.478+0000 7ff78f6c5640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff788071ea0 con 0x7ff78815ba30 2024-04-17T10:48:11.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.478+0000 7ff78f6c5640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff78806e040 con 0x7ff7880cfbe0 2024-04-17T10:48:11.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.478+0000 7ff78d461640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff78815ba30 0x7ff788072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:11.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.478+0000 7ff78d461640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff78815ba30 0x7ff788072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39802/0 (socket says 172.21.15.83:39802) 2024-04-17T10:48:11.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.478+0000 7ff78d461640 1 -- 172.21.15.83:0/1436725578 learned_addr learned my addr 172.21.15.83:0/1436725578 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:11.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.478+0000 7ff78dc62640 1 --2- 172.21.15.83:0/1436725578 >> v2:172.21.15.167:3300/0 conn(0x7ff7880cfbe0 0x7ff78806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:11.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.478+0000 7ff78d461640 1 -- 172.21.15.83:0/1436725578 >> v2:172.21.15.167:3300/0 conn(0x7ff7880cfbe0 msgr2=0x7ff78806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:11.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.478+0000 7ff78d461640 1 --2- 172.21.15.83:0/1436725578 >> v2:172.21.15.167:3300/0 conn(0x7ff7880cfbe0 0x7ff78806c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.478+0000 7ff78d461640 1 -- 172.21.15.83:0/1436725578 >> v1:172.21.15.167:6789/0 conn(0x7ff78815c960 legacy=0x7ff7880cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:11.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.478+0000 7ff78d461640 1 -- 172.21.15.83:0/1436725578 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff788071850 con 0x7ff78815ba30 2024-04-17T10:48:11.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.478+0000 7ff78dc62640 1 --2- 172.21.15.83:0/1436725578 >> v2:172.21.15.167:3300/0 conn(0x7ff7880cfbe0 0x7ff78806c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:11.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.482+0000 7ff78d461640 1 --2- 172.21.15.83:0/1436725578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff78815ba30 0x7ff788072330 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7ff7880b2410 tx=0x7ff78001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2f2d50e9d43d09 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:11.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.482+0000 7ff787fff640 1 -- 172.21.15.83:0/1436725578 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7800b0070 con 0x7ff78815ba30 2024-04-17T10:48:11.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.482+0000 7ff787fff640 1 -- 172.21.15.83:0/1436725578 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7800ab600 con 0x7ff78815ba30 2024-04-17T10:48:11.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.482+0000 7ff787fff640 1 -- 172.21.15.83:0/1436725578 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7800ab9d0 con 0x7ff78815ba30 2024-04-17T10:48:11.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.482+0000 7ff78f6c5640 1 -- 172.21.15.83:0/1436725578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff78815ba30 msgr2=0x7ff788072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:11.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.482+0000 7ff78f6c5640 1 --2- 172.21.15.83:0/1436725578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff78815ba30 0x7ff788072330 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7ff7880b2410 tx=0x7ff78001c430 comp rx=0 tx=0).stop 2024-04-17T10:48:11.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.482+0000 7ff78f6c5640 1 -- 172.21.15.83:0/1436725578 shutdown_connections 2024-04-17T10:48:11.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.482+0000 7ff78f6c5640 1 --2- 172.21.15.83:0/1436725578 >> v2:172.21.15.167:3300/0 conn(0x7ff7880cfbe0 0x7ff78806c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.482+0000 7ff78f6c5640 1 --2- 172.21.15.83:0/1436725578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff78815ba30 0x7ff788072330 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.482+0000 7ff78f6c5640 1 -- 172.21.15.83:0/1436725578 >> 172.21.15.83:0/1436725578 conn(0x7ff78806abe0 msgr2=0x7ff78806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:11.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.482+0000 7ff78f6c5640 1 -- 172.21.15.83:0/1436725578 shutdown_connections 2024-04-17T10:48:11.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.482+0000 7ff78f6c5640 1 -- 172.21.15.83:0/1436725578 wait complete. 2024-04-17T10:48:11.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.482+0000 7ff78f6c5640 1 Processor -- start 2024-04-17T10:48:11.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.482+0000 7ff78f6c5640 1 -- start start 2024-04-17T10:48:11.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.482+0000 7ff78f6c5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7880cfbe0 0x7ff788149150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:11.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.482+0000 7ff78f6c5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff78815ba30 0x7ff788147f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:11.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.482+0000 7ff78f6c5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff78815c960 0x7ff788149ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:11.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.482+0000 7ff78f6c5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78806de20 con 0x7ff78815ba30 2024-04-17T10:48:11.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.482+0000 7ff78f6c5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78806dca0 con 0x7ff7880cfbe0 2024-04-17T10:48:11.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.482+0000 7ff78f6c5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff788168520 con 0x7ff78815c960 2024-04-17T10:48:11.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.482+0000 7ff78cc60640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff78815ba30 0x7ff788147f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:11.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.482+0000 7ff78cc60640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff78815ba30 0x7ff788147f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39812/0 (socket says 172.21.15.83:39812) 2024-04-17T10:48:11.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.482+0000 7ff78cc60640 1 -- 172.21.15.83:0/96481233 learned_addr learned my addr 172.21.15.83:0/96481233 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:11.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.482+0000 7ff78d461640 1 --2- 172.21.15.83:0/96481233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7880cfbe0 0x7ff788149150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:11.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.482+0000 7ff78dc62640 1 --2- 172.21.15.83:0/96481233 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff78815c960 0x7ff788149ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:11.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.486+0000 7ff78cc60640 1 -- 172.21.15.83:0/96481233 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff78815c960 msgr2=0x7ff788149ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:11.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.486+0000 7ff78cc60640 1 --2- 172.21.15.83:0/96481233 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff78815c960 0x7ff788149ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.486+0000 7ff78cc60640 1 -- 172.21.15.83:0/96481233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7880cfbe0 msgr2=0x7ff788149150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:11.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.486+0000 7ff78cc60640 1 --2- 172.21.15.83:0/96481233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7880cfbe0 0x7ff788149150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.486+0000 7ff78cc60640 1 -- 172.21.15.83:0/96481233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78814ba70 con 0x7ff78815ba30 2024-04-17T10:48:11.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.486+0000 7ff78cc60640 1 --2- 172.21.15.83:0/96481233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff78815ba30 0x7ff788147f10 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00c450 tx=0x7ff77c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:11.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.486+0000 7ff785ffb640 1 -- 172.21.15.83:0/96481233 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c012830 con 0x7ff78815ba30 2024-04-17T10:48:11.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.486+0000 7ff785ffb640 1 -- 172.21.15.83:0/96481233 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77c017070 con 0x7ff78815ba30 2024-04-17T10:48:11.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.486+0000 7ff78f6c5640 1 -- 172.21.15.83:0/96481233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff78814bd10 con 0x7ff78815ba30 2024-04-17T10:48:11.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.486+0000 7ff785ffb640 1 -- 172.21.15.83:0/96481233 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c012c30 con 0x7ff78815ba30 2024-04-17T10:48:11.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.486+0000 7ff78f6c5640 1 -- 172.21.15.83:0/96481233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff78814c2d0 con 0x7ff78815ba30 2024-04-17T10:48:11.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.486+0000 7ff78f6c5640 1 -- 172.21.15.83:0/96481233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff788074f90 con 0x7ff78815ba30 2024-04-17T10:48:11.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.486+0000 7ff785ffb640 1 -- 172.21.15.83:0/96481233 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff77c01c460 con 0x7ff78815ba30 2024-04-17T10:48:11.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.486+0000 7ff785ffb640 1 --2- 172.21.15.83:0/96481233 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff774066f60 0x7ff774069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:11.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.486+0000 7ff78d461640 1 --2- 172.21.15.83:0/96481233 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff774066f60 0x7ff774069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:11.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.486+0000 7ff785ffb640 1 -- 172.21.15.83:0/96481233 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff77c084d90 con 0x7ff78815ba30 2024-04-17T10:48:11.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.486+0000 7ff78d461640 1 --2- 172.21.15.83:0/96481233 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff774066f60 0x7ff774069420 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff7800ad000 tx=0x7ff7800a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:11.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.490+0000 7ff785ffb640 1 -- 172.21.15.83:0/96481233 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff77c055e30 con 0x7ff78815ba30 2024-04-17T10:48:11.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.670+0000 7ff78f6c5640 1 -- 172.21.15.83:0/96481233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff788152e30 con 0x7ff78815ba30 2024-04-17T10:48:11.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.670+0000 7ff785ffb640 1 -- 172.21.15.83:0/96481233 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4940 (secure 0 0 0) 0x7ff77c055730 con 0x7ff78815ba30 2024-04-17T10:48:11.668 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:11.668 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/437268957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":437268957},{"type":"v1","addr":"172.21.15.167:6833","nonce":437268957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":6253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3214640674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3214640674},{"type":"v1","addr":"172.21.15.167:6835","nonce":3214640674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":8046,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/715498323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":715498323},{"type":"v1","addr":"172.21.15.83:6835","nonce":715498323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:47:29.385103+0000","modified":"2024-04-17T10:47:31.389088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8049":{"gid":8049,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2353406669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":2353406669},{"type":"v1","addr":"172.21.15.83:6837","nonce":2353406669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-17T10:48:11.668 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 62 2024-04-17T10:48:11.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.674+0000 7ff78f6c5640 1 -- 172.21.15.83:0/96481233 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff774066f60 msgr2=0x7ff774069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:11.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.674+0000 7ff78f6c5640 1 --2- 172.21.15.83:0/96481233 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff774066f60 0x7ff774069420 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff7800ad000 tx=0x7ff7800a6000 comp rx=0 tx=0).stop 2024-04-17T10:48:11.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.674+0000 7ff78f6c5640 1 -- 172.21.15.83:0/96481233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff78815ba30 msgr2=0x7ff788147f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:11.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.674+0000 7ff78f6c5640 1 --2- 172.21.15.83:0/96481233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff78815ba30 0x7ff788147f10 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00c450 tx=0x7ff77c00c910 comp rx=0 tx=0).stop 2024-04-17T10:48:11.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.674+0000 7ff78f6c5640 1 -- 172.21.15.83:0/96481233 shutdown_connections 2024-04-17T10:48:11.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.674+0000 7ff78f6c5640 1 --2- 172.21.15.83:0/96481233 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff774066f60 0x7ff774069420 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.674+0000 7ff78f6c5640 1 --2- 172.21.15.83:0/96481233 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff78815c960 0x7ff788149ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.674+0000 7ff78f6c5640 1 --2- 172.21.15.83:0/96481233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff78815ba30 0x7ff788147f10 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.674+0000 7ff78f6c5640 1 --2- 172.21.15.83:0/96481233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7880cfbe0 0x7ff788149150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.674+0000 7ff78f6c5640 1 -- 172.21.15.83:0/96481233 >> 172.21.15.83:0/96481233 conn(0x7ff78806abe0 msgr2=0x7ff78815aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:11.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.674+0000 7ff78f6c5640 1 -- 172.21.15.83:0/96481233 shutdown_connections 2024-04-17T10:48:11.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.674+0000 7ff78f6c5640 1 -- 172.21.15.83:0/96481233 wait complete. 2024-04-17T10:48:11.688 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:48:11.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.806+0000 7fb223c9a640 1 Processor -- start 2024-04-17T10:48:11.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.806+0000 7fb223c9a640 1 -- start start 2024-04-17T10:48:11.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.810+0000 7fb223c9a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb21c15eeb0 0x7fb21c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:11.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.810+0000 7fb223c9a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb21c0cf780 0x7fb21c06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:11.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.810+0000 7fb223c9a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb21c06e980 con 0x7fb21c15f290 2024-04-17T10:48:11.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.810+0000 7fb223c9a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb21c0cf030 con 0x7fb21c15eeb0 2024-04-17T10:48:11.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.810+0000 7fb223c9a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb21c0cdb90 con 0x7fb21c0cf780 2024-04-17T10:48:11.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.810+0000 7fb221a36640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb21c15eeb0 0x7fb21c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:11.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.810+0000 7fb221a36640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb21c15eeb0 0x7fb21c06d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39822/0 (socket says 172.21.15.83:39822) 2024-04-17T10:48:11.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.810+0000 7fb222237640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb21c15f290 0x7fb21c15f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38222/0 (socket says 172.21.15.83:38222) 2024-04-17T10:48:11.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.810+0000 7fb221a36640 1 -- 172.21.15.83:0/634829089 learned_addr learned my addr 172.21.15.83:0/634829089 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:11.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.810+0000 7fb221a36640 1 -- 172.21.15.83:0/634829089 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb21c0cf780 msgr2=0x7fb21c06bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:11.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.810+0000 7fb221235640 1 --2- 172.21.15.83:0/634829089 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb21c0cf780 0x7fb21c06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:11.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.810+0000 7fb221a36640 1 --2- 172.21.15.83:0/634829089 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb21c0cf780 0x7fb21c06bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.810+0000 7fb220a34640 1 -- 172.21.15.83:0/634829089 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 656746920 0 0) 0x7fb21c06e980 con 0x7fb21c15f290 2024-04-17T10:48:11.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.810+0000 7fb221a36640 1 -- 172.21.15.83:0/634829089 >> v1:172.21.15.83:6789/0 conn(0x7fb21c15f290 legacy=0x7fb21c15f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:11.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.810+0000 7fb221a36640 1 -- 172.21.15.83:0/634829089 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb21c06e0e0 con 0x7fb21c15eeb0 2024-04-17T10:48:11.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.810+0000 7fb221a36640 1 --2- 172.21.15.83:0/634829089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb21c15eeb0 0x7fb21c06d5d0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fb218009870 tx=0x7fb21801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f8e816e4b9a172 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:11.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.810+0000 7fb220a34640 1 -- 172.21.15.83:0/634829089 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2180a9610 con 0x7fb21c15eeb0 2024-04-17T10:48:11.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.810+0000 7fb220a34640 1 -- 172.21.15.83:0/634829089 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2180a97b0 con 0x7fb21c15eeb0 2024-04-17T10:48:11.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.810+0000 7fb220a34640 1 -- 172.21.15.83:0/634829089 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2180a9b80 con 0x7fb21c15eeb0 2024-04-17T10:48:11.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb223c9a640 1 -- 172.21.15.83:0/634829089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb21c15eeb0 msgr2=0x7fb21c06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:11.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb223c9a640 1 --2- 172.21.15.83:0/634829089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb21c15eeb0 0x7fb21c06d5d0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fb218009870 tx=0x7fb21801c530 comp rx=0 tx=0).stop 2024-04-17T10:48:11.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb223c9a640 1 -- 172.21.15.83:0/634829089 shutdown_connections 2024-04-17T10:48:11.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb223c9a640 1 --2- 172.21.15.83:0/634829089 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb21c0cf780 0x7fb21c06bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb223c9a640 1 --2- 172.21.15.83:0/634829089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb21c15eeb0 0x7fb21c06d5d0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb223c9a640 1 -- 172.21.15.83:0/634829089 >> 172.21.15.83:0/634829089 conn(0x7fb21c0cc910 msgr2=0x7fb21c0ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:11.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb223c9a640 1 -- 172.21.15.83:0/634829089 shutdown_connections 2024-04-17T10:48:11.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb223c9a640 1 -- 172.21.15.83:0/634829089 wait complete. 2024-04-17T10:48:11.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb223c9a640 1 Processor -- start 2024-04-17T10:48:11.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb223c9a640 1 -- start start 2024-04-17T10:48:11.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb223c9a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb21c0cf780 0x7fb21c15e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:11.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb223c9a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb21c15eeb0 0x7fb21c15c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:11.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb223c9a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb21c15f290 0x7fb21c09be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:11.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb223c9a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21c168a80 con 0x7fb21c15eeb0 2024-04-17T10:48:11.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb223c9a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21c06c540 con 0x7fb21c0cf780 2024-04-17T10:48:11.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb223c9a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb21c168c00 con 0x7fb21c15f290 2024-04-17T10:48:11.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb221235640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb21c15eeb0 0x7fb21c15c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:11.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb221235640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb21c15eeb0 0x7fb21c15c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39830/0 (socket says 172.21.15.83:39830) 2024-04-17T10:48:11.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb221235640 1 -- 172.21.15.83:0/3242174683 learned_addr learned my addr 172.21.15.83:0/3242174683 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:11.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb222237640 1 --2- 172.21.15.83:0/3242174683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb21c15f290 0x7fb21c09be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:11.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb221a36640 1 --2- 172.21.15.83:0/3242174683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb21c0cf780 0x7fb21c15e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:11.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb221235640 1 -- 172.21.15.83:0/3242174683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb21c15f290 msgr2=0x7fb21c09be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:11.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb221235640 1 --2- 172.21.15.83:0/3242174683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb21c15f290 0x7fb21c09be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb221235640 1 -- 172.21.15.83:0/3242174683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb21c0cf780 msgr2=0x7fb21c15e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:11.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb221235640 1 --2- 172.21.15.83:0/3242174683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb21c0cf780 0x7fb21c15e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb221235640 1 -- 172.21.15.83:0/3242174683 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb21c1f8d80 con 0x7fb21c15eeb0 2024-04-17T10:48:11.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb221235640 1 --2- 172.21.15.83:0/3242174683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb21c15eeb0 0x7fb21c15c100 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00c850 tx=0x7fb20c00cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:11.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb2127fc640 1 -- 172.21.15.83:0/3242174683 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c015070 con 0x7fb21c15eeb0 2024-04-17T10:48:11.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb2127fc640 1 -- 172.21.15.83:0/3242174683 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20c004020 con 0x7fb21c15eeb0 2024-04-17T10:48:11.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb2127fc640 1 -- 172.21.15.83:0/3242174683 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c0043f0 con 0x7fb21c15eeb0 2024-04-17T10:48:11.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb223c9a640 1 -- 172.21.15.83:0/3242174683 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb21c09b630 con 0x7fb21c15eeb0 2024-04-17T10:48:11.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb223c9a640 1 -- 172.21.15.83:0/3242174683 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb21c15c640 con 0x7fb21c15eeb0 2024-04-17T10:48:11.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.814+0000 7fb223c9a640 1 -- 172.21.15.83:0/3242174683 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb21c074f90 con 0x7fb21c15eeb0 2024-04-17T10:48:11.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.818+0000 7fb2127fc640 1 -- 172.21.15.83:0/3242174683 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb20c0077d0 con 0x7fb21c15eeb0 2024-04-17T10:48:11.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.818+0000 7fb2127fc640 1 --2- 172.21.15.83:0/3242174683 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb200066f60 0x7fb200069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:11.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.818+0000 7fb221a36640 1 --2- 172.21.15.83:0/3242174683 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb200066f60 0x7fb200069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:11.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.818+0000 7fb2127fc640 1 -- 172.21.15.83:0/3242174683 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb20c084e00 con 0x7fb21c15eeb0 2024-04-17T10:48:11.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.818+0000 7fb221a36640 1 --2- 172.21.15.83:0/3242174683 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb200066f60 0x7fb200069420 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb2180ae000 tx=0x7fb2180b3040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:11.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.822+0000 7fb2127fc640 1 -- 172.21.15.83:0/3242174683 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb20c055ea0 con 0x7fb21c15eeb0 2024-04-17T10:48:11.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.970+0000 7fb223c9a640 1 -- 172.21.15.83:0/3242174683 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb21c068150 con 0x7fb21c15eeb0 2024-04-17T10:48:11.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.970+0000 7fb2127fc640 1 -- 172.21.15.83:0/3242174683 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16547 (secure 0 0 0) 0x7fb20c0557a0 con 0x7fb21c15eeb0 2024-04-17T10:48:11.968 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:11.968 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":103,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:47:31.018926+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:47:27.619432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-17T10:47:28.379472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:48:11.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.974+0000 7fb223c9a640 1 -- 172.21.15.83:0/3242174683 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb200066f60 msgr2=0x7fb200069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:11.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.974+0000 7fb223c9a640 1 --2- 172.21.15.83:0/3242174683 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb200066f60 0x7fb200069420 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb2180ae000 tx=0x7fb2180b3040 comp rx=0 tx=0).stop 2024-04-17T10:48:11.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.974+0000 7fb223c9a640 1 -- 172.21.15.83:0/3242174683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb21c15eeb0 msgr2=0x7fb21c15c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:11.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.974+0000 7fb223c9a640 1 --2- 172.21.15.83:0/3242174683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb21c15eeb0 0x7fb21c15c100 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00c850 tx=0x7fb20c00cd10 comp rx=0 tx=0).stop 2024-04-17T10:48:11.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.974+0000 7fb223c9a640 1 -- 172.21.15.83:0/3242174683 shutdown_connections 2024-04-17T10:48:11.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.974+0000 7fb223c9a640 1 --2- 172.21.15.83:0/3242174683 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb200066f60 0x7fb200069420 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.974+0000 7fb223c9a640 1 --2- 172.21.15.83:0/3242174683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb21c15f290 0x7fb21c09be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.974+0000 7fb223c9a640 1 --2- 172.21.15.83:0/3242174683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb21c15eeb0 0x7fb21c15c100 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.974+0000 7fb223c9a640 1 --2- 172.21.15.83:0/3242174683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb21c0cf780 0x7fb21c15e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:11.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.974+0000 7fb223c9a640 1 -- 172.21.15.83:0/3242174683 >> 172.21.15.83:0/3242174683 conn(0x7fb21c0cc910 msgr2=0x7fb21c0b06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:11.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.974+0000 7fb223c9a640 1 -- 172.21.15.83:0/3242174683 shutdown_connections 2024-04-17T10:48:11.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:11.974+0000 7fb223c9a640 1 -- 172.21.15.83:0/3242174683 wait complete. 2024-04-17T10:48:11.986 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-17T10:48:11.986 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:48:12.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.110+0000 7fa86851e640 1 Processor -- start 2024-04-17T10:48:12.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.110+0000 7fa86851e640 1 -- start start 2024-04-17T10:48:12.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.110+0000 7fa86851e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa860146e70 0x7fa86006d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:12.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.110+0000 7fa86851e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa860147570 con 0x7fa860146a00 2024-04-17T10:48:12.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.110+0000 7fa86851e640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa86006cbb0 con 0x7fa8600b2d80 2024-04-17T10:48:12.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.110+0000 7fa86851e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa86006c960 con 0x7fa860146e70 2024-04-17T10:48:12.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.110+0000 7fa8662ba640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa860146a00 0x7fa8600b2670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38232/0 (socket says 172.21.15.83:38232) 2024-04-17T10:48:12.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.110+0000 7fa8662ba640 1 -- 172.21.15.83:0/3870205607 learned_addr learned my addr 172.21.15.83:0/3870205607 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:12.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.110+0000 7fa866abb640 1 --2- 172.21.15.83:0/3870205607 >> v2:172.21.15.83:3300/0 conn(0x7fa860146e70 0x7fa86006d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:12.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.110+0000 7fa8652b8640 1 -- 172.21.15.83:0/3870205607 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3229141302 0 0) 0x7fa860147570 con 0x7fa860146a00 2024-04-17T10:48:12.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.110+0000 7fa8652b8640 1 -- 172.21.15.83:0/3870205607 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa84c003530 con 0x7fa860146a00 2024-04-17T10:48:12.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.110+0000 7fa866abb640 1 -- 172.21.15.83:0/3870205607 >> v1:172.21.15.167:6789/0 conn(0x7fa8600b2d80 legacy=0x7fa860070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:12.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.110+0000 7fa866abb640 1 -- 172.21.15.83:0/3870205607 >> v1:172.21.15.83:6789/0 conn(0x7fa860146a00 legacy=0x7fa8600b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:12.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.110+0000 7fa866abb640 1 -- 172.21.15.83:0/3870205607 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86006ce90 con 0x7fa860146e70 2024-04-17T10:48:12.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa866abb640 1 --2- 172.21.15.83:0/3870205607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa860146e70 0x7fa86006d690 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fa858009d90 tx=0x7fa85801c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7719d0942333c931 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:12.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa8652b8640 1 -- 172.21.15.83:0/3870205607 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8580ad070 con 0x7fa860146e70 2024-04-17T10:48:12.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa8652b8640 1 -- 172.21.15.83:0/3870205607 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8580a9680 con 0x7fa860146e70 2024-04-17T10:48:12.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa8652b8640 1 -- 172.21.15.83:0/3870205607 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8580a9a50 con 0x7fa860146e70 2024-04-17T10:48:12.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa86851e640 1 -- 172.21.15.83:0/3870205607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa860146e70 msgr2=0x7fa86006d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:12.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa86851e640 1 --2- 172.21.15.83:0/3870205607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa860146e70 0x7fa86006d690 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fa858009d90 tx=0x7fa85801c6b0 comp rx=0 tx=0).stop 2024-04-17T10:48:12.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa86851e640 1 -- 172.21.15.83:0/3870205607 shutdown_connections 2024-04-17T10:48:12.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa86851e640 1 --2- 172.21.15.83:0/3870205607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa860146e70 0x7fa86006d690 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:12.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa86851e640 1 -- 172.21.15.83:0/3870205607 >> 172.21.15.83:0/3870205607 conn(0x7fa86006abe0 msgr2=0x7fa86006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:12.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa86851e640 1 -- 172.21.15.83:0/3870205607 shutdown_connections 2024-04-17T10:48:12.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa86851e640 1 -- 172.21.15.83:0/3870205607 wait complete. 2024-04-17T10:48:12.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa86851e640 1 Processor -- start 2024-04-17T10:48:12.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa86851e640 1 -- start start 2024-04-17T10:48:12.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa86851e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8600b2d80 0x7fa86015bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:12.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa86851e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa860146a00 0x7fa860160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:12.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa86851e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa860146e70 0x7fa860136f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:12.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa86851e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86016c420 con 0x7fa860146e70 2024-04-17T10:48:12.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa86851e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa860147320 con 0x7fa860146a00 2024-04-17T10:48:12.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa86851e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86016c5a0 con 0x7fa8600b2d80 2024-04-17T10:48:12.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa866abb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa860146e70 0x7fa860136f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:12.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa866abb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa860146e70 0x7fa860136f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39838/0 (socket says 172.21.15.83:39838) 2024-04-17T10:48:12.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa866abb640 1 -- 172.21.15.83:0/3373686678 learned_addr learned my addr 172.21.15.83:0/3373686678 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:12.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa865ab9640 1 --2- 172.21.15.83:0/3373686678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa860146a00 0x7fa860160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:12.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa8662ba640 1 --2- 172.21.15.83:0/3373686678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8600b2d80 0x7fa86015bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:12.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa866abb640 1 -- 172.21.15.83:0/3373686678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8600b2d80 msgr2=0x7fa86015bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:12.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa866abb640 1 --2- 172.21.15.83:0/3373686678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8600b2d80 0x7fa86015bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:12.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa866abb640 1 -- 172.21.15.83:0/3373686678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa860146a00 msgr2=0x7fa860160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:12.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa866abb640 1 --2- 172.21.15.83:0/3373686678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa860146a00 0x7fa860160ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:12.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa866abb640 1 -- 172.21.15.83:0/3373686678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa860156860 con 0x7fa860146e70 2024-04-17T10:48:12.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa865ab9640 1 --2- 172.21.15.83:0/3373686678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa860146a00 0x7fa860160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:12.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa866abb640 1 --2- 172.21.15.83:0/3373686678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa860146e70 0x7fa860136f00 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fa858009d90 tx=0x7fa858019450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:12.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa856ffd640 1 -- 172.21.15.83:0/3373686678 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8580c2070 con 0x7fa860146e70 2024-04-17T10:48:12.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa856ffd640 1 -- 172.21.15.83:0/3373686678 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8580b3810 con 0x7fa860146e70 2024-04-17T10:48:12.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa86851e640 1 -- 172.21.15.83:0/3373686678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa860156580 con 0x7fa860146e70 2024-04-17T10:48:12.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa856ffd640 1 -- 172.21.15.83:0/3373686678 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8580ad040 con 0x7fa860146e70 2024-04-17T10:48:12.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.114+0000 7fa86851e640 1 -- 172.21.15.83:0/3373686678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8601593e0 con 0x7fa860146e70 2024-04-17T10:48:12.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.118+0000 7fa86851e640 1 -- 172.21.15.83:0/3373686678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa860158ef0 con 0x7fa860146e70 2024-04-17T10:48:12.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.118+0000 7fa856ffd640 1 -- 172.21.15.83:0/3373686678 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa8580c5070 con 0x7fa860146e70 2024-04-17T10:48:12.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.118+0000 7fa856ffd640 1 --2- 172.21.15.83:0/3373686678 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa844066f60 0x7fa844069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:12.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.118+0000 7fa856ffd640 1 -- 172.21.15.83:0/3373686678 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa85811ff00 con 0x7fa860146e70 2024-04-17T10:48:12.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.118+0000 7fa8662ba640 1 --2- 172.21.15.83:0/3373686678 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa844066f60 0x7fa844069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:12.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.118+0000 7fa8662ba640 1 --2- 172.21.15.83:0/3373686678 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa844066f60 0x7fa844069420 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0011f0 tx=0x7fa85c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:12.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.122+0000 7fa856ffd640 1 -- 172.21.15.83:0/3373686678 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa8580b8070 con 0x7fa860146e70 2024-04-17T10:48:12.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.314+0000 7fa86851e640 1 -- 172.21.15.83:0/3373686678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa860074f90 con 0x7fa860146e70 2024-04-17T10:48:12.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.318+0000 7fa856ffd640 1 -- 172.21.15.83:0/3373686678 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fa858016560 con 0x7fa860146e70 2024-04-17T10:48:12.313 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:12.313 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/437268957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":437268957},{"type":"v1","addr":"172.21.15.167:6833","nonce":437268957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":6253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3214640674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3214640674},{"type":"v1","addr":"172.21.15.167:6835","nonce":3214640674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":8046,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/715498323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":715498323},{"type":"v1","addr":"172.21.15.83:6835","nonce":715498323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:47:29.385103+0000","modified":"2024-04-17T10:47:31.389088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8049":{"gid":8049,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2353406669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":2353406669},{"type":"v1","addr":"172.21.15.83:6837","nonce":2353406669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-17T10:48:12.314 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 62 2024-04-17T10:48:12.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.318+0000 7fa86851e640 1 -- 172.21.15.83:0/3373686678 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa844066f60 msgr2=0x7fa844069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:12.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.318+0000 7fa86851e640 1 --2- 172.21.15.83:0/3373686678 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa844066f60 0x7fa844069420 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0011f0 tx=0x7fa85c009040 comp rx=0 tx=0).stop 2024-04-17T10:48:12.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.318+0000 7fa86851e640 1 -- 172.21.15.83:0/3373686678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa860146e70 msgr2=0x7fa860136f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:12.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.318+0000 7fa86851e640 1 --2- 172.21.15.83:0/3373686678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa860146e70 0x7fa860136f00 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fa858009d90 tx=0x7fa858019450 comp rx=0 tx=0).stop 2024-04-17T10:48:12.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.318+0000 7fa86851e640 1 -- 172.21.15.83:0/3373686678 shutdown_connections 2024-04-17T10:48:12.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.318+0000 7fa86851e640 1 --2- 172.21.15.83:0/3373686678 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa844066f60 0x7fa844069420 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:12.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.318+0000 7fa86851e640 1 --2- 172.21.15.83:0/3373686678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa860146e70 0x7fa860136f00 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:12.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.318+0000 7fa86851e640 1 --2- 172.21.15.83:0/3373686678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa860146a00 0x7fa860160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:12.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.318+0000 7fa86851e640 1 --2- 172.21.15.83:0/3373686678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8600b2d80 0x7fa86015bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:12.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.318+0000 7fa86851e640 1 -- 172.21.15.83:0/3373686678 >> 172.21.15.83:0/3373686678 conn(0x7fa86006abe0 msgr2=0x7fa86006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:12.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.318+0000 7fa86851e640 1 -- 172.21.15.83:0/3373686678 shutdown_connections 2024-04-17T10:48:12.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.318+0000 7fa86851e640 1 -- 172.21.15.83:0/3373686678 wait complete. 2024-04-17T10:48:12.333 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:48:12.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.454+0000 7fef776c6640 1 Processor -- start 2024-04-17T10:48:12.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.454+0000 7fef776c6640 1 -- start start 2024-04-17T10:48:12.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.458+0000 7fef776c6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef7015ba30 0x7fef70072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:12.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.458+0000 7fef776c6640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fef7015c960 0x7fef700cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:12.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.458+0000 7fef776c6640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef700cfea0 con 0x7fef700cfa10 2024-04-17T10:48:12.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.458+0000 7fef776c6640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef70071ea0 con 0x7fef7015c960 2024-04-17T10:48:12.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.458+0000 7fef776c6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7006e320 con 0x7fef7015ba30 2024-04-17T10:48:12.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.458+0000 7fef74c61640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fef7015c960 0x7fef700cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:12.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.458+0000 7fef74c61640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fef7015c960 0x7fef700cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39842/0 (socket says 172.21.15.83:39842) 2024-04-17T10:48:12.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.458+0000 7fef74c61640 1 -- 172.21.15.83:0/2486110665 learned_addr learned my addr 172.21.15.83:0/2486110665 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:12.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.458+0000 7fef74c61640 1 -- 172.21.15.83:0/2486110665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef7015ba30 msgr2=0x7fef70072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:12.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.458+0000 7fef74c61640 1 --2- 172.21.15.83:0/2486110665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef7015ba30 0x7fef70072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:12.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.458+0000 7fef74c61640 1 -- 172.21.15.83:0/2486110665 >> v1:172.21.15.167:6789/0 conn(0x7fef700cfa10 legacy=0x7fef7006c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:12.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.458+0000 7fef74c61640 1 -- 172.21.15.83:0/2486110665 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7006e040 con 0x7fef7015c960 2024-04-17T10:48:12.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.458+0000 7fef74c61640 1 --2- 172.21.15.83:0/2486110665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef7015c960 0x7fef700cf4d0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fef60009870 tx=0x7fef6001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8869420ccb3bd494 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:12.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.458+0000 7fef67fff640 1 -- 172.21.15.83:0/2486110665 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef600b0070 con 0x7fef7015c960 2024-04-17T10:48:12.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.458+0000 7fef67fff640 1 -- 172.21.15.83:0/2486110665 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef600ab600 con 0x7fef7015c960 2024-04-17T10:48:12.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.458+0000 7fef67fff640 1 -- 172.21.15.83:0/2486110665 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef600ab9d0 con 0x7fef7015c960 2024-04-17T10:48:12.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.458+0000 7fef776c6640 1 -- 172.21.15.83:0/2486110665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef7015c960 msgr2=0x7fef700cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:12.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.458+0000 7fef776c6640 1 --2- 172.21.15.83:0/2486110665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef7015c960 0x7fef700cf4d0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fef60009870 tx=0x7fef6001c490 comp rx=0 tx=0).stop 2024-04-17T10:48:12.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.458+0000 7fef776c6640 1 -- 172.21.15.83:0/2486110665 shutdown_connections 2024-04-17T10:48:12.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.458+0000 7fef776c6640 1 --2- 172.21.15.83:0/2486110665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef7015c960 0x7fef700cf4d0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:12.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.458+0000 7fef776c6640 1 --2- 172.21.15.83:0/2486110665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef7015ba30 0x7fef70072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:12.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.458+0000 7fef776c6640 1 -- 172.21.15.83:0/2486110665 >> 172.21.15.83:0/2486110665 conn(0x7fef7006abe0 msgr2=0x7fef7006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:12.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.458+0000 7fef776c6640 1 -- 172.21.15.83:0/2486110665 shutdown_connections 2024-04-17T10:48:12.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.458+0000 7fef776c6640 1 -- 172.21.15.83:0/2486110665 wait complete. 2024-04-17T10:48:12.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.462+0000 7fef776c6640 1 Processor -- start 2024-04-17T10:48:12.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.462+0000 7fef776c6640 1 -- start start 2024-04-17T10:48:12.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.462+0000 7fef776c6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef700cfa10 0x7fef701515d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:12.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.462+0000 7fef776c6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef7015ba30 0x7fef70154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:12.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.462+0000 7fef776c6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef7015c960 0x7fef70154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:12.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.462+0000 7fef776c6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7006dca0 con 0x7fef700cfa10 2024-04-17T10:48:12.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.462+0000 7fef776c6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef700aeae0 con 0x7fef7015ba30 2024-04-17T10:48:12.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.462+0000 7fef776c6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7006de20 con 0x7fef7015c960 2024-04-17T10:48:12.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.462+0000 7fef75462640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef700cfa10 0x7fef701515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:12.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.462+0000 7fef75462640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef700cfa10 0x7fef701515d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39856/0 (socket says 172.21.15.83:39856) 2024-04-17T10:48:12.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.462+0000 7fef75462640 1 -- 172.21.15.83:0/3663077719 learned_addr learned my addr 172.21.15.83:0/3663077719 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:12.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.462+0000 7fef74c61640 1 --2- 172.21.15.83:0/3663077719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef7015ba30 0x7fef70154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:12.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.462+0000 7fef75c63640 1 --2- 172.21.15.83:0/3663077719 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef7015c960 0x7fef70154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:12.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.462+0000 7fef75462640 1 -- 172.21.15.83:0/3663077719 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef7015c960 msgr2=0x7fef70154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:12.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.462+0000 7fef75462640 1 --2- 172.21.15.83:0/3663077719 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef7015c960 0x7fef70154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:12.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.462+0000 7fef75462640 1 -- 172.21.15.83:0/3663077719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef7015ba30 msgr2=0x7fef70154410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:12.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.462+0000 7fef75462640 1 --2- 172.21.15.83:0/3663077719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef7015ba30 0x7fef70154410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:12.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.462+0000 7fef75462640 1 -- 172.21.15.83:0/3663077719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef70155180 con 0x7fef700cfa10 2024-04-17T10:48:12.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.462+0000 7fef75c63640 1 --2- 172.21.15.83:0/3663077719 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef7015c960 0x7fef70154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:12.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.462+0000 7fef74c61640 1 --2- 172.21.15.83:0/3663077719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef7015ba30 0x7fef70154410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:48:12.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.462+0000 7fef75462640 1 --2- 172.21.15.83:0/3663077719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef700cfa10 0x7fef701515d0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00b550 tx=0x7fef6c00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:12.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.462+0000 7fef65ffb640 1 -- 172.21.15.83:0/3663077719 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c014070 con 0x7fef700cfa10 2024-04-17T10:48:12.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.462+0000 7fef65ffb640 1 -- 172.21.15.83:0/3663077719 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6c010890 con 0x7fef700cfa10 2024-04-17T10:48:12.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.462+0000 7fef776c6640 1 -- 172.21.15.83:0/3663077719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef700b28a0 con 0x7fef700cfa10 2024-04-17T10:48:12.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.462+0000 7fef65ffb640 1 -- 172.21.15.83:0/3663077719 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c010c80 con 0x7fef700cfa10 2024-04-17T10:48:12.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.462+0000 7fef776c6640 1 -- 172.21.15.83:0/3663077719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef700b3100 con 0x7fef700cfa10 2024-04-17T10:48:12.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.462+0000 7fef776c6640 1 -- 172.21.15.83:0/3663077719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef70074f90 con 0x7fef700cfa10 2024-04-17T10:48:12.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.466+0000 7fef65ffb640 1 -- 172.21.15.83:0/3663077719 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fef6c017070 con 0x7fef700cfa10 2024-04-17T10:48:12.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.466+0000 7fef65ffb640 1 --2- 172.21.15.83:0/3663077719 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fef54066f60 0x7fef54069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:12.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.466+0000 7fef65ffb640 1 -- 172.21.15.83:0/3663077719 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef6c084d50 con 0x7fef700cfa10 2024-04-17T10:48:12.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.466+0000 7fef74c61640 1 --2- 172.21.15.83:0/3663077719 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fef54066f60 0x7fef54069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:12.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.466+0000 7fef74c61640 1 --2- 172.21.15.83:0/3663077719 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fef54066f60 0x7fef54069420 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fef7006d810 tx=0x7fef60009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:12.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.470+0000 7fef65ffb640 1 -- 172.21.15.83:0/3663077719 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fef6c088030 con 0x7fef700cfa10 2024-04-17T10:48:12.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.650+0000 7fef776c6640 1 -- 172.21.15.83:0/3663077719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef700b2a70 con 0x7fef700cfa10 2024-04-17T10:48:12.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.650+0000 7fef65ffb640 1 -- 172.21.15.83:0/3663077719 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fef700b2a70 con 0x7fef700cfa10 2024-04-17T10:48:12.648 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:12.648 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/437268957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":437268957},{"type":"v1","addr":"172.21.15.167:6833","nonce":437268957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":6253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3214640674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3214640674},{"type":"v1","addr":"172.21.15.167:6835","nonce":3214640674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":8046,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/715498323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":715498323},{"type":"v1","addr":"172.21.15.83:6835","nonce":715498323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:47:29.385103+0000","modified":"2024-04-17T10:47:31.389088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8049":{"gid":8049,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2353406669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":2353406669},{"type":"v1","addr":"172.21.15.83:6837","nonce":2353406669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-17T10:48:12.648 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 62 2024-04-17T10:48:12.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.654+0000 7fef776c6640 1 -- 172.21.15.83:0/3663077719 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fef54066f60 msgr2=0x7fef54069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:12.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.654+0000 7fef776c6640 1 --2- 172.21.15.83:0/3663077719 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fef54066f60 0x7fef54069420 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fef7006d810 tx=0x7fef60009040 comp rx=0 tx=0).stop 2024-04-17T10:48:12.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.654+0000 7fef776c6640 1 -- 172.21.15.83:0/3663077719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef700cfa10 msgr2=0x7fef701515d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:12.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.654+0000 7fef776c6640 1 --2- 172.21.15.83:0/3663077719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef700cfa10 0x7fef701515d0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00b550 tx=0x7fef6c00ba10 comp rx=0 tx=0).stop 2024-04-17T10:48:12.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.654+0000 7fef776c6640 1 -- 172.21.15.83:0/3663077719 shutdown_connections 2024-04-17T10:48:12.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.654+0000 7fef776c6640 1 --2- 172.21.15.83:0/3663077719 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fef54066f60 0x7fef54069420 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:12.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.654+0000 7fef776c6640 1 --2- 172.21.15.83:0/3663077719 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef7015c960 0x7fef70154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:12.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.654+0000 7fef776c6640 1 --2- 172.21.15.83:0/3663077719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef7015ba30 0x7fef70154410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:12.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.654+0000 7fef776c6640 1 --2- 172.21.15.83:0/3663077719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef700cfa10 0x7fef701515d0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:12.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.654+0000 7fef776c6640 1 -- 172.21.15.83:0/3663077719 >> 172.21.15.83:0/3663077719 conn(0x7fef7006abe0 msgr2=0x7fef7015e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:12.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.654+0000 7fef776c6640 1 -- 172.21.15.83:0/3663077719 shutdown_connections 2024-04-17T10:48:12.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.654+0000 7fef776c6640 1 -- 172.21.15.83:0/3663077719 wait complete. 2024-04-17T10:48:12.666 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:48:12.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.778+0000 7f7702a1d640 1 Processor -- start 2024-04-17T10:48:12.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.778+0000 7f7702a1d640 1 -- start start 2024-04-17T10:48:12.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.782+0000 7f7702a1d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f76fc0afd00 0x7f76fc06e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:12.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.782+0000 7f7702a1d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76fc15f1b0 con 0x7f76fc0b4270 2024-04-17T10:48:12.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.782+0000 7f7702a1d640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76fc06dce0 con 0x7f76fc15eb90 2024-04-17T10:48:12.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.782+0000 7f7702a1d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76fc06dfc0 con 0x7f76fc0afd00 2024-04-17T10:48:12.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.782+0000 7f76fbfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f76fc0afd00 0x7f76fc06e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:12.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.782+0000 7f76fb7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f76fc0b4270 0x7f76fc15e480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38236/0 (socket says 172.21.15.83:38236) 2024-04-17T10:48:12.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.782+0000 7f76fb7fe640 1 -- 172.21.15.83:0/3636386199 learned_addr learned my addr 172.21.15.83:0/3636386199 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:12.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.782+0000 7f76fbfff640 1 -- 172.21.15.83:0/3636386199 >> v1:172.21.15.167:6789/0 conn(0x7f76fc15eb90 legacy=0x7f76fc15bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:12.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.782+0000 7f76fbfff640 1 -- 172.21.15.83:0/3636386199 >> v1:172.21.15.83:6789/0 conn(0x7f76fc0b4270 legacy=0x7f76fc15e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:12.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.782+0000 7f76fbfff640 1 -- 172.21.15.83:0/3636386199 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76fc06e2a0 con 0x7f76fc0afd00 2024-04-17T10:48:12.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.782+0000 7f76fbfff640 1 --2- 172.21.15.83:0/3636386199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76fc0afd00 0x7f76fc06e7c0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f76f4009d90 tx=0x7f76f401eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f3d4a4d04bd9503 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:12.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.782+0000 7f76faffd640 1 -- 172.21.15.83:0/3636386199 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f40b4070 con 0x7f76fc0afd00 2024-04-17T10:48:12.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.782+0000 7f76faffd640 1 -- 172.21.15.83:0/3636386199 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f40a7910 con 0x7f76fc0afd00 2024-04-17T10:48:12.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.782+0000 7f76faffd640 1 -- 172.21.15.83:0/3636386199 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f40a7ce0 con 0x7f76fc0afd00 2024-04-17T10:48:12.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.786+0000 7f7702a1d640 1 -- 172.21.15.83:0/3636386199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76fc0afd00 msgr2=0x7f76fc06e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:12.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.786+0000 7f7702a1d640 1 --2- 172.21.15.83:0/3636386199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76fc0afd00 0x7f76fc06e7c0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f76f4009d90 tx=0x7f76f401eb40 comp rx=0 tx=0).stop 2024-04-17T10:48:12.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.786+0000 7f7702a1d640 1 -- 172.21.15.83:0/3636386199 shutdown_connections 2024-04-17T10:48:12.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.786+0000 7f7702a1d640 1 --2- 172.21.15.83:0/3636386199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76fc0afd00 0x7f76fc06e7c0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:12.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.786+0000 7f7702a1d640 1 -- 172.21.15.83:0/3636386199 >> 172.21.15.83:0/3636386199 conn(0x7f76fc06bcf0 msgr2=0x7f76fc06c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:12.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.786+0000 7f7702a1d640 1 -- 172.21.15.83:0/3636386199 shutdown_connections 2024-04-17T10:48:12.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.786+0000 7f7702a1d640 1 -- 172.21.15.83:0/3636386199 wait complete. 2024-04-17T10:48:12.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.786+0000 7f7702a1d640 1 Processor -- start 2024-04-17T10:48:12.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.786+0000 7f7702a1d640 1 -- start start 2024-04-17T10:48:12.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.786+0000 7f7702a1d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76fc0afd00 0x7f76fc14cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:12.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.786+0000 7f7702a1d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76fc0b4270 0x7f76fc0cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:12.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.786+0000 7f7702a1d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76fc15eb90 0x7f76fc14eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:12.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.786+0000 7f7702a1d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76fc06b3f0 con 0x7f76fc15eb90 2024-04-17T10:48:12.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.786+0000 7f7702a1d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76fc15cb00 con 0x7f76fc0b4270 2024-04-17T10:48:12.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.786+0000 7f7702a1d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76fc06b570 con 0x7f76fc0afd00 2024-04-17T10:48:12.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.786+0000 7f7700fba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76fc15eb90 0x7f76fc14eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:12.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.786+0000 7f7700fba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76fc15eb90 0x7f76fc14eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39868/0 (socket says 172.21.15.83:39868) 2024-04-17T10:48:12.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.786+0000 7f7700fba640 1 -- 172.21.15.83:0/974320533 learned_addr learned my addr 172.21.15.83:0/974320533 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:12.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.786+0000 7f76fb7fe640 1 --2- 172.21.15.83:0/974320533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76fc0b4270 0x7f76fc0cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:12.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.786+0000 7f7700fba640 1 -- 172.21.15.83:0/974320533 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76fc0afd00 msgr2=0x7f76fc14cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:12.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.786+0000 7f7700fba640 1 --2- 172.21.15.83:0/974320533 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76fc0afd00 0x7f76fc14cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:12.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.786+0000 7f7700fba640 1 -- 172.21.15.83:0/974320533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76fc0b4270 msgr2=0x7f76fc0cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:12.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.786+0000 7f7700fba640 1 --2- 172.21.15.83:0/974320533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76fc0b4270 0x7f76fc0cf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:12.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.786+0000 7f7700fba640 1 -- 172.21.15.83:0/974320533 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76fc0cd750 con 0x7f76fc15eb90 2024-04-17T10:48:12.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.786+0000 7f76fb7fe640 1 --2- 172.21.15.83:0/974320533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76fc0b4270 0x7f76fc0cf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:12.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.786+0000 7f7700fba640 1 --2- 172.21.15.83:0/974320533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76fc15eb90 0x7f76fc14eeb0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f76e400b410 tx=0x7f76e400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:12.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.786+0000 7f76f8ff9640 1 -- 172.21.15.83:0/974320533 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e4015020 con 0x7f76fc15eb90 2024-04-17T10:48:12.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.786+0000 7f76f8ff9640 1 -- 172.21.15.83:0/974320533 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e4011870 con 0x7f76fc15eb90 2024-04-17T10:48:12.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.786+0000 7f7702a1d640 1 -- 172.21.15.83:0/974320533 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76fc0cfe90 con 0x7f76fc15eb90 2024-04-17T10:48:12.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.786+0000 7f76f8ff9640 1 -- 172.21.15.83:0/974320533 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e4011c60 con 0x7f76fc15eb90 2024-04-17T10:48:12.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.786+0000 7f7702a1d640 1 -- 172.21.15.83:0/974320533 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76fc0cdfd0 con 0x7f76fc15eb90 2024-04-17T10:48:12.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.786+0000 7f7702a1d640 1 -- 172.21.15.83:0/974320533 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76fc074f90 con 0x7f76fc15eb90 2024-04-17T10:48:12.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.790+0000 7f76f8ff9640 1 -- 172.21.15.83:0/974320533 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f76e4018020 con 0x7f76fc15eb90 2024-04-17T10:48:12.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.790+0000 7f76f8ff9640 1 --2- 172.21.15.83:0/974320533 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f76dc066f60 0x7f76dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:12.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.790+0000 7f76fbfff640 1 --2- 172.21.15.83:0/974320533 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f76dc066f60 0x7f76dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:12.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.790+0000 7f76f8ff9640 1 -- 172.21.15.83:0/974320533 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f76e4084e40 con 0x7f76fc15eb90 2024-04-17T10:48:12.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.790+0000 7f76fbfff640 1 --2- 172.21.15.83:0/974320533 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f76dc066f60 0x7f76dc069420 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f76f40b1000 tx=0x7f76f4004ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:12.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.794+0000 7f76f8ff9640 1 -- 172.21.15.83:0/974320533 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f76e401f020 con 0x7f76fc15eb90 2024-04-17T10:48:12.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.938+0000 7f7702a1d640 1 -- 172.21.15.83:0/974320533 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76fc068150 con 0x7f76fc15eb90 2024-04-17T10:48:12.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.942+0000 7f76f8ff9640 1 -- 172.21.15.83:0/974320533 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16547 (secure 0 0 0) 0x7f76e40172c0 con 0x7f76fc15eb90 2024-04-17T10:48:12.938 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:12.938 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":103,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:47:31.018926+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:47:27.619432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-17T10:47:28.379472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:48:12.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.942+0000 7f7702a1d640 1 -- 172.21.15.83:0/974320533 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f76dc066f60 msgr2=0x7f76dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:12.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.942+0000 7f7702a1d640 1 --2- 172.21.15.83:0/974320533 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f76dc066f60 0x7f76dc069420 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f76f40b1000 tx=0x7f76f4004ac0 comp rx=0 tx=0).stop 2024-04-17T10:48:12.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.942+0000 7f7702a1d640 1 -- 172.21.15.83:0/974320533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76fc15eb90 msgr2=0x7f76fc14eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:12.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.942+0000 7f7702a1d640 1 --2- 172.21.15.83:0/974320533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76fc15eb90 0x7f76fc14eeb0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f76e400b410 tx=0x7f76e400b8d0 comp rx=0 tx=0).stop 2024-04-17T10:48:12.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.942+0000 7f7702a1d640 1 -- 172.21.15.83:0/974320533 shutdown_connections 2024-04-17T10:48:12.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.942+0000 7f7702a1d640 1 --2- 172.21.15.83:0/974320533 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f76dc066f60 0x7f76dc069420 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:12.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.942+0000 7f7702a1d640 1 --2- 172.21.15.83:0/974320533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76fc15eb90 0x7f76fc14eeb0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:12.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.942+0000 7f7702a1d640 1 --2- 172.21.15.83:0/974320533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76fc0b4270 0x7f76fc0cf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:12.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.942+0000 7f7702a1d640 1 --2- 172.21.15.83:0/974320533 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76fc0afd00 0x7f76fc14cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:12.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.942+0000 7f7702a1d640 1 -- 172.21.15.83:0/974320533 >> 172.21.15.83:0/974320533 conn(0x7f76fc06bcf0 msgr2=0x7f76fc15cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:12.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.942+0000 7f7702a1d640 1 -- 172.21.15.83:0/974320533 shutdown_connections 2024-04-17T10:48:12.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:12.942+0000 7f7702a1d640 1 -- 172.21.15.83:0/974320533 wait complete. 2024-04-17T10:48:12.956 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-17T10:48:13.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f4e9b0640 1 Processor -- start 2024-04-17T10:48:13.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f4e9b0640 1 -- start start 2024-04-17T10:48:13.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f4e9b0640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5f4815dbe0 0x7f5f480c5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f4e9b0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f4806f150 0x7f5f480ca520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f4e9b0640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f480c6850 con 0x7f5f480caa60 2024-04-17T10:48:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f4e9b0640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f480caeb0 con 0x7f5f4815dbe0 2024-04-17T10:48:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f4e9b0640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f480c5fb0 con 0x7f5f4806f150 2024-04-17T10:48:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f4cf4d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5f480caa60 0x7f5f48161830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38238/0 (socket says 172.21.15.83:38238) 2024-04-17T10:48:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f4cf4d640 1 -- 172.21.15.83:0/2427683885 learned_addr learned my addr 172.21.15.83:0/2427683885 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f47fff640 1 --2- 172.21.15.83:0/2427683885 >> v2:172.21.15.167:3300/0 conn(0x7f5f4815dbe0 0x7f5f480c5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f3ffff640 1 --2- 172.21.15.83:0/2427683885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f4806f150 0x7f5f480ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f477fe640 1 -- 172.21.15.83:0/2427683885 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3803609269 0 0) 0x7f5f480c6850 con 0x7f5f480caa60 2024-04-17T10:48:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f477fe640 1 -- 172.21.15.83:0/2427683885 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f34003530 con 0x7f5f480caa60 2024-04-17T10:48:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f477fe640 1 -- 172.21.15.83:0/2427683885 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4260477262 0 0) 0x7f5f34003530 con 0x7f5f480caa60 2024-04-17T10:48:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f477fe640 1 -- 172.21.15.83:0/2427683885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f4806f150 msgr2=0x7f5f480ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f477fe640 1 --2- 172.21.15.83:0/2427683885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f4806f150 0x7f5f480ca520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f477fe640 1 -- 172.21.15.83:0/2427683885 >> v2:172.21.15.167:3300/0 conn(0x7f5f4815dbe0 msgr2=0x7f5f480c5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:13.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f477fe640 1 --2- 172.21.15.83:0/2427683885 >> v2:172.21.15.167:3300/0 conn(0x7f5f4815dbe0 0x7f5f480c5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:13.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f477fe640 1 -- 172.21.15.83:0/2427683885 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f480c6290 con 0x7f5f480caa60 2024-04-17T10:48:13.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f477fe640 1 -- 172.21.15.83:0/2427683885 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f5f40001a00 con 0x7f5f480caa60 2024-04-17T10:48:13.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f477fe640 1 -- 172.21.15.83:0/2427683885 >> v1:172.21.15.83:6789/0 conn(0x7f5f480caa60 legacy=0x7f5f48161830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:13.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f477fe640 1 --2- 172.21.15.83:0/2427683885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f34003a30 0x7f5f34003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:13.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f477fe640 1 --2- 172.21.15.83:0/2427683885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f4806f150 0x7f5f34004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:13.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f477fe640 1 --2- 172.21.15.83:0/2427683885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f34004d50 0x7f5f340051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:13.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f477fe640 1 -- 172.21.15.83:0/2427683885 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f480caeb0 con 0x7f5f34003a30 2024-04-17T10:48:13.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f477fe640 1 -- 172.21.15.83:0/2427683885 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f480c5fb0 con 0x7f5f34004d50 2024-04-17T10:48:13.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f477fe640 1 -- 172.21.15.83:0/2427683885 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3400b9d0 con 0x7f5f4806f150 2024-04-17T10:48:13.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f47fff640 1 --2- 172.21.15.83:0/2427683885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f34003a30 0x7f5f34003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:13.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f3ffff640 1 --2- 172.21.15.83:0/2427683885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f4806f150 0x7f5f34004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:13.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f4cf4d640 1 --2- 172.21.15.83:0/2427683885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f34004d50 0x7f5f340051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:13.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f47fff640 1 -- 172.21.15.83:0/2427683885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f4806f150 msgr2=0x7f5f34004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:13.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f47fff640 1 --2- 172.21.15.83:0/2427683885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f4806f150 0x7f5f34004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:13.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f47fff640 1 -- 172.21.15.83:0/2427683885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f34004d50 msgr2=0x7f5f340051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:13.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f47fff640 1 --2- 172.21.15.83:0/2427683885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f34004d50 0x7f5f340051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:13.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f47fff640 1 -- 172.21.15.83:0/2427683885 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f34003530 con 0x7f5f34003a30 2024-04-17T10:48:13.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.082+0000 7f5f4cf4d640 1 --2- 172.21.15.83:0/2427683885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f34004d50 0x7f5f340051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:13.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.086+0000 7f5f47fff640 1 --2- 172.21.15.83:0/2427683885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f34003a30 0x7f5f34003e50 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009ae0 tx=0x7f5f3801ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c4bca9c3698b03 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:13.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.086+0000 7f5f477fe640 1 -- 172.21.15.83:0/2427683885 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f380b2070 con 0x7f5f34003a30 2024-04-17T10:48:13.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.086+0000 7f5f477fe640 1 -- 172.21.15.83:0/2427683885 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f380a5670 con 0x7f5f34003a30 2024-04-17T10:48:13.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.086+0000 7f5f477fe640 1 -- 172.21.15.83:0/2427683885 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f380096b0 con 0x7f5f34003a30 2024-04-17T10:48:13.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.086+0000 7f5f4e9b0640 1 -- 172.21.15.83:0/2427683885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f34003a30 msgr2=0x7f5f34003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.086+0000 7f5f4e9b0640 1 --2- 172.21.15.83:0/2427683885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f34003a30 0x7f5f34003e50 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009ae0 tx=0x7f5f3801ca80 comp rx=0 tx=0).stop 2024-04-17T10:48:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.086+0000 7f5f4cf4d640 1 -- 172.21.15.83:0/2427683885 reap_dead start 2024-04-17T10:48:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.086+0000 7f5f4e9b0640 1 -- 172.21.15.83:0/2427683885 shutdown_connections 2024-04-17T10:48:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.086+0000 7f5f4e9b0640 1 -- 172.21.15.83:0/2427683885 >> 172.21.15.83:0/2427683885 conn(0x7f5f4806abe0 msgr2=0x7f5f4806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.086+0000 7f5f4e9b0640 1 -- 172.21.15.83:0/2427683885 shutdown_connections 2024-04-17T10:48:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.086+0000 7f5f4e9b0640 1 -- 172.21.15.83:0/2427683885 wait complete. 2024-04-17T10:48:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.086+0000 7f5f4e9b0640 1 Processor -- start 2024-04-17T10:48:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.086+0000 7f5f4e9b0640 1 -- start start 2024-04-17T10:48:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.090+0000 7f5f4e9b0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f34003a30 0x7f5f4815a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.090+0000 7f5f4e9b0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f480b50b0 0x7f5f480b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.090+0000 7f5f4e9b0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f480b0c30 0x7f5f480c7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.090+0000 7f5f4e9b0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4815e140 con 0x7f5f34003a30 2024-04-17T10:48:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.090+0000 7f5f4e9b0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4815dfc0 con 0x7f5f480b0c30 2024-04-17T10:48:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.090+0000 7f5f4e9b0640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f48168b60 con 0x7f5f480b50b0 2024-04-17T10:48:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.090+0000 7f5f47fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f34003a30 0x7f5f4815a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.090+0000 7f5f47fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f34003a30 0x7f5f4815a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39886/0 (socket says 172.21.15.83:39886) 2024-04-17T10:48:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.090+0000 7f5f47fff640 1 -- 172.21.15.83:0/1148131705 learned_addr learned my addr 172.21.15.83:0/1148131705 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.090+0000 7f5f3ffff640 1 --2- 172.21.15.83:0/1148131705 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f480b50b0 0x7f5f480b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.090+0000 7f5f4cf4d640 1 --2- 172.21.15.83:0/1148131705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f480b0c30 0x7f5f480c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.090+0000 7f5f47fff640 1 -- 172.21.15.83:0/1148131705 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f480b50b0 msgr2=0x7f5f480b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.090+0000 7f5f47fff640 1 --2- 172.21.15.83:0/1148131705 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f480b50b0 0x7f5f480b06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.090+0000 7f5f47fff640 1 -- 172.21.15.83:0/1148131705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f480b0c30 msgr2=0x7f5f480c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.090+0000 7f5f47fff640 1 --2- 172.21.15.83:0/1148131705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f480b0c30 0x7f5f480c7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.090+0000 7f5f47fff640 1 -- 172.21.15.83:0/1148131705 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f480af2c0 con 0x7f5f34003a30 2024-04-17T10:48:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.090+0000 7f5f3ffff640 1 --2- 172.21.15.83:0/1148131705 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f480b50b0 0x7f5f480b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.090+0000 7f5f4cf4d640 1 --2- 172.21.15.83:0/1148131705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f480b0c30 0x7f5f480c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.090+0000 7f5f47fff640 1 --2- 172.21.15.83:0/1148131705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f34003a30 0x7f5f4815a570 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f5f3801c8b0 tx=0x7f5f34005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:13.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.090+0000 7f5f457fa640 1 -- 172.21.15.83:0/1148131705 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f380bc020 con 0x7f5f34003a30 2024-04-17T10:48:13.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.090+0000 7f5f457fa640 1 -- 172.21.15.83:0/1148131705 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f38004020 con 0x7f5f34003a30 2024-04-17T10:48:13.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.090+0000 7f5f4e9b0640 1 -- 172.21.15.83:0/1148131705 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f480b4010 con 0x7f5f34003a30 2024-04-17T10:48:13.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.090+0000 7f5f457fa640 1 -- 172.21.15.83:0/1148131705 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f380b2040 con 0x7f5f34003a30 2024-04-17T10:48:13.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.090+0000 7f5f4e9b0640 1 -- 172.21.15.83:0/1148131705 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f480b45d0 con 0x7f5f34003a30 2024-04-17T10:48:13.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.090+0000 7f5f4e9b0640 1 -- 172.21.15.83:0/1148131705 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f48074f90 con 0x7f5f34003a30 2024-04-17T10:48:13.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.090+0000 7f5f457fa640 1 -- 172.21.15.83:0/1148131705 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5f380bf070 con 0x7f5f34003a30 2024-04-17T10:48:13.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.090+0000 7f5f457fa640 1 --2- 172.21.15.83:0/1148131705 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5f28066f60 0x7f5f28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:13.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.090+0000 7f5f457fa640 1 -- 172.21.15.83:0/1148131705 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f3811ea80 con 0x7f5f34003a30 2024-04-17T10:48:13.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.090+0000 7f5f3ffff640 1 --2- 172.21.15.83:0/1148131705 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5f28066f60 0x7f5f28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:13.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.090+0000 7f5f3ffff640 1 --2- 172.21.15.83:0/1148131705 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5f28066f60 0x7f5f28069420 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5f4806ce40 tx=0x7f5f30008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:13.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.094+0000 7f5f457fa640 1 -- 172.21.15.83:0/1148131705 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5f380167c0 con 0x7f5f34003a30 2024-04-17T10:48:13.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.274+0000 7f5f4e9b0640 1 -- 172.21.15.83:0/1148131705 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5f48066bc0 con 0x7f5f34003a30 2024-04-17T10:48:13.271 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-04-17T10:48:13.274+0000 7f35c3b81640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T10:48:13.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.278+0000 7f5f457fa640 1 -- 172.21.15.83:0/1148131705 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v63) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5f380efa90 con 0x7f5f34003a30 2024-04-17T10:48:13.274 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-17T10:48:13.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.278+0000 7f5f4e9b0640 1 -- 172.21.15.83:0/1148131705 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5f28066f60 msgr2=0x7f5f28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:13.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.278+0000 7f5f4e9b0640 1 --2- 172.21.15.83:0/1148131705 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5f28066f60 0x7f5f28069420 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5f4806ce40 tx=0x7f5f30008040 comp rx=0 tx=0).stop 2024-04-17T10:48:13.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.278+0000 7f5f4e9b0640 1 -- 172.21.15.83:0/1148131705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f34003a30 msgr2=0x7f5f4815a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:13.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.278+0000 7f5f4e9b0640 1 --2- 172.21.15.83:0/1148131705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f34003a30 0x7f5f4815a570 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f5f3801c8b0 tx=0x7f5f34005840 comp rx=0 tx=0).stop 2024-04-17T10:48:13.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.282+0000 7f5f4e9b0640 1 -- 172.21.15.83:0/1148131705 shutdown_connections 2024-04-17T10:48:13.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.282+0000 7f5f4e9b0640 1 --2- 172.21.15.83:0/1148131705 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5f28066f60 0x7f5f28069420 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:13.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.282+0000 7f5f4e9b0640 1 --2- 172.21.15.83:0/1148131705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f480b0c30 0x7f5f480c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:13.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.282+0000 7f5f4e9b0640 1 --2- 172.21.15.83:0/1148131705 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f480b50b0 0x7f5f480b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:13.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.282+0000 7f5f4e9b0640 1 --2- 172.21.15.83:0/1148131705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f34003a30 0x7f5f4815a570 secure :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f5f3801c8b0 tx=0x7f5f34005840 comp rx=0 tx=0).stop 2024-04-17T10:48:13.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.282+0000 7f5f4e9b0640 1 -- 172.21.15.83:0/1148131705 >> 172.21.15.83:0/1148131705 conn(0x7f5f4806abe0 msgr2=0x7f5f4806d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:13.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.282+0000 7f5f4e9b0640 1 -- 172.21.15.83:0/1148131705 shutdown_connections 2024-04-17T10:48:13.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.282+0000 7f5f4e9b0640 1 -- 172.21.15.83:0/1148131705 wait complete. 2024-04-17T10:48:13.296 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-17T10:48:13.357 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-04-17T10:48:13.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.430+0000 7fbda63ee640 1 Processor -- start 2024-04-17T10:48:13.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.430+0000 7fbda63ee640 1 -- start start 2024-04-17T10:48:13.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.430+0000 7fbda63ee640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbda015a9c0 0x7fbda00cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:13.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.430+0000 7fbda63ee640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbda006dc00 0x7fbda00ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:13.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.430+0000 7fbda63ee640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbda006e170 con 0x7fbda00d0710 2024-04-17T10:48:13.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.430+0000 7fbda63ee640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda006ed80 con 0x7fbda006dc00 2024-04-17T10:48:13.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.430+0000 7fbda63ee640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda006e450 con 0x7fbda015a9c0 2024-04-17T10:48:13.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.430+0000 7fbd9ffff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbda00d0710 0x7fbda00d0af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38252/0 (socket says 172.21.15.83:38252) 2024-04-17T10:48:13.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.430+0000 7fbd9ffff640 1 -- 172.21.15.83:0/979275607 learned_addr learned my addr 172.21.15.83:0/979275607 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:13.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.430+0000 7fbda498b640 1 --2- 172.21.15.83:0/979275607 >> v2:172.21.15.83:3300/0 conn(0x7fbda006dc00 0x7fbda00ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:13.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.430+0000 7fbd9f7fe640 1 --2- 172.21.15.83:0/979275607 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbda015a9c0 0x7fbda00cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:13.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.430+0000 7fbd9effd640 1 -- 172.21.15.83:0/979275607 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2456580845 0 0) 0x7fbda006e170 con 0x7fbda00d0710 2024-04-17T10:48:13.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.430+0000 7fbd9effd640 1 -- 172.21.15.83:0/979275607 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd8c003530 con 0x7fbda00d0710 2024-04-17T10:48:13.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.430+0000 7fbda498b640 1 -- 172.21.15.83:0/979275607 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbda015a9c0 msgr2=0x7fbda00cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:13.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.430+0000 7fbda498b640 1 --2- 172.21.15.83:0/979275607 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbda015a9c0 0x7fbda00cdb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:13.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.430+0000 7fbda498b640 1 -- 172.21.15.83:0/979275607 >> v1:172.21.15.83:6789/0 conn(0x7fbda00d0710 legacy=0x7fbda00d0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:13.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.430+0000 7fbda498b640 1 -- 172.21.15.83:0/979275607 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda006e730 con 0x7fbda006dc00 2024-04-17T10:48:13.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.434+0000 7fbda498b640 1 --2- 172.21.15.83:0/979275607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda006dc00 0x7fbda00ce0b0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fbd980097c0 tx=0x7fbd9801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=374a35097497ed42 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:13.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.434+0000 7fbd9effd640 1 -- 172.21.15.83:0/979275607 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd980b2070 con 0x7fbda006dc00 2024-04-17T10:48:13.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.434+0000 7fbd9effd640 1 -- 172.21.15.83:0/979275607 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd980ad680 con 0x7fbda006dc00 2024-04-17T10:48:13.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.434+0000 7fbd9effd640 1 -- 172.21.15.83:0/979275607 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd980ada30 con 0x7fbda006dc00 2024-04-17T10:48:13.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.434+0000 7fbda63ee640 1 -- 172.21.15.83:0/979275607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda006dc00 msgr2=0x7fbda00ce0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:13.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.434+0000 7fbda63ee640 1 --2- 172.21.15.83:0/979275607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda006dc00 0x7fbda00ce0b0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fbd980097c0 tx=0x7fbd9801c470 comp rx=0 tx=0).stop 2024-04-17T10:48:13.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.434+0000 7fbda63ee640 1 -- 172.21.15.83:0/979275607 shutdown_connections 2024-04-17T10:48:13.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.434+0000 7fbda63ee640 1 --2- 172.21.15.83:0/979275607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda006dc00 0x7fbda00ce0b0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:13.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.434+0000 7fbda63ee640 1 --2- 172.21.15.83:0/979275607 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbda015a9c0 0x7fbda00cdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:13.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.434+0000 7fbda63ee640 1 -- 172.21.15.83:0/979275607 >> 172.21.15.83:0/979275607 conn(0x7fbda006bcf0 msgr2=0x7fbda006c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:13.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.434+0000 7fbda63ee640 1 -- 172.21.15.83:0/979275607 shutdown_connections 2024-04-17T10:48:13.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.434+0000 7fbda63ee640 1 -- 172.21.15.83:0/979275607 wait complete. 2024-04-17T10:48:13.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.434+0000 7fbda63ee640 1 Processor -- start 2024-04-17T10:48:13.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.434+0000 7fbda63ee640 1 -- start start 2024-04-17T10:48:13.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.438+0000 7fbda63ee640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbda006dc00 0x7fbda013b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:13.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.438+0000 7fbda63ee640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda00d0710 0x7fbda006abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:13.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.438+0000 7fbda63ee640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda015a9c0 0x7fbda006b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:13.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.438+0000 7fbda63ee640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda0167760 con 0x7fbda00d0710 2024-04-17T10:48:13.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.438+0000 7fbda63ee640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda01675e0 con 0x7fbda015a9c0 2024-04-17T10:48:13.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.438+0000 7fbda63ee640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda01678e0 con 0x7fbda006dc00 2024-04-17T10:48:13.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.438+0000 7fbd9f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda00d0710 0x7fbda006abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:13.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.438+0000 7fbd9f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda00d0710 0x7fbda006abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39906/0 (socket says 172.21.15.83:39906) 2024-04-17T10:48:13.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.438+0000 7fbd9f7fe640 1 -- 172.21.15.83:0/4223020644 learned_addr learned my addr 172.21.15.83:0/4223020644 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:13.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.438+0000 7fbd9ffff640 1 --2- 172.21.15.83:0/4223020644 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbda006dc00 0x7fbda013b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:13.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.438+0000 7fbda498b640 1 --2- 172.21.15.83:0/4223020644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda015a9c0 0x7fbda006b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:13.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.438+0000 7fbd9f7fe640 1 -- 172.21.15.83:0/4223020644 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbda006dc00 msgr2=0x7fbda013b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:13.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.438+0000 7fbd9f7fe640 1 --2- 172.21.15.83:0/4223020644 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbda006dc00 0x7fbda013b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:13.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.438+0000 7fbd9f7fe640 1 -- 172.21.15.83:0/4223020644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda015a9c0 msgr2=0x7fbda006b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:13.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.438+0000 7fbd9f7fe640 1 --2- 172.21.15.83:0/4223020644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda015a9c0 0x7fbda006b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:13.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.438+0000 7fbd9f7fe640 1 -- 172.21.15.83:0/4223020644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda0067cc0 con 0x7fbda00d0710 2024-04-17T10:48:13.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.438+0000 7fbd9ffff640 1 --2- 172.21.15.83:0/4223020644 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbda006dc00 0x7fbda013b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:13.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.438+0000 7fbda498b640 1 --2- 172.21.15.83:0/4223020644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda015a9c0 0x7fbda006b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:13.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.438+0000 7fbd9f7fe640 1 --2- 172.21.15.83:0/4223020644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda00d0710 0x7fbda006abe0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000e8a0 tx=0x7fbd9000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:13.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.438+0000 7fbd9cff9640 1 -- 172.21.15.83:0/4223020644 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd90017070 con 0x7fbda00d0710 2024-04-17T10:48:13.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.438+0000 7fbd9cff9640 1 -- 172.21.15.83:0/4223020644 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9000c940 con 0x7fbda00d0710 2024-04-17T10:48:13.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.438+0000 7fbda63ee640 1 -- 172.21.15.83:0/4223020644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda00b2e50 con 0x7fbda00d0710 2024-04-17T10:48:13.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.438+0000 7fbd9cff9640 1 -- 172.21.15.83:0/4223020644 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9000ccf0 con 0x7fbda00d0710 2024-04-17T10:48:13.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.438+0000 7fbda63ee640 1 -- 172.21.15.83:0/4223020644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda01504c0 con 0x7fbda00d0710 2024-04-17T10:48:13.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.438+0000 7fbda63ee640 1 -- 172.21.15.83:0/4223020644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda00b3300 con 0x7fbda00d0710 2024-04-17T10:48:13.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.438+0000 7fbd9cff9640 1 -- 172.21.15.83:0/4223020644 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbd9001a070 con 0x7fbda00d0710 2024-04-17T10:48:13.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.438+0000 7fbd9cff9640 1 --2- 172.21.15.83:0/4223020644 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbd80066f90 0x7fbd80069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:13.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.438+0000 7fbd9cff9640 1 -- 172.21.15.83:0/4223020644 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbd90085690 con 0x7fbda00d0710 2024-04-17T10:48:13.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.438+0000 7fbd9ffff640 1 --2- 172.21.15.83:0/4223020644 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbd80066f90 0x7fbd80069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:13.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.442+0000 7fbd9ffff640 1 --2- 172.21.15.83:0/4223020644 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbd80066f90 0x7fbd80069450 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fbd88005c10 tx=0x7fbd88009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:13.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.442+0000 7fbd9cff9640 1 -- 172.21.15.83:0/4223020644 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbd90056720 con 0x7fbda00d0710 2024-04-17T10:48:13.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:13.638+0000 7fbda63ee640 1 -- 172.21.15.83:0/4223020644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbda0074f90 con 0x7fbda00d0710 2024-04-17T10:48:14.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.282+0000 7fbd9cff9640 1 -- 172.21.15.83:0/4223020644 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v64) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbd90056020 con 0x7fbda00d0710 2024-04-17T10:48:14.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.282+0000 7fbda63ee640 1 -- 172.21.15.83:0/4223020644 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbd80066f90 msgr2=0x7fbd80069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:14.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.282+0000 7fbda63ee640 1 --2- 172.21.15.83:0/4223020644 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbd80066f90 0x7fbd80069450 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fbd88005c10 tx=0x7fbd88009040 comp rx=0 tx=0).stop 2024-04-17T10:48:14.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.282+0000 7fbda63ee640 1 -- 172.21.15.83:0/4223020644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda00d0710 msgr2=0x7fbda006abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:14.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.282+0000 7fbda63ee640 1 --2- 172.21.15.83:0/4223020644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda00d0710 0x7fbda006abe0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000e8a0 tx=0x7fbd9000eca0 comp rx=0 tx=0).stop 2024-04-17T10:48:14.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.286+0000 7fbda63ee640 1 -- 172.21.15.83:0/4223020644 shutdown_connections 2024-04-17T10:48:14.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.286+0000 7fbda63ee640 1 --2- 172.21.15.83:0/4223020644 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbd80066f90 0x7fbd80069450 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:14.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.286+0000 7fbda63ee640 1 --2- 172.21.15.83:0/4223020644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda015a9c0 0x7fbda006b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:14.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.286+0000 7fbda63ee640 1 --2- 172.21.15.83:0/4223020644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda00d0710 0x7fbda006abe0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:14.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.286+0000 7fbda63ee640 1 --2- 172.21.15.83:0/4223020644 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbda006dc00 0x7fbda013b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:14.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.286+0000 7fbda63ee640 1 -- 172.21.15.83:0/4223020644 >> 172.21.15.83:0/4223020644 conn(0x7fbda006bcf0 msgr2=0x7fbda006c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:14.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.286+0000 7fbda63ee640 1 -- 172.21.15.83:0/4223020644 shutdown_connections 2024-04-17T10:48:14.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.286+0000 7fbda63ee640 1 -- 172.21.15.83:0/4223020644 wait complete. 2024-04-17T10:48:14.301 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-17T10:48:14.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.430+0000 7fa3bf77c640 1 Processor -- start 2024-04-17T10:48:14.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.430+0000 7fa3bf77c640 1 -- start start 2024-04-17T10:48:14.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.430+0000 7fa3bf77c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa3b815cb40 0x7fa3b806abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:14.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.430+0000 7fa3bf77c640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3b80b39d0 con 0x7fa3b80aedf0 2024-04-17T10:48:14.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.430+0000 7fa3bf77c640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3b8070c40 con 0x7fa3b806b120 2024-04-17T10:48:14.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.430+0000 7fa3bf77c640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3b8071290 con 0x7fa3b815cb40 2024-04-17T10:48:14.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.430+0000 7fa3bd518640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa3b80aedf0 0x7fa3b80b30f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38254/0 (socket says 172.21.15.83:38254) 2024-04-17T10:48:14.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.430+0000 7fa3bd518640 1 -- 172.21.15.83:0/4113727369 learned_addr learned my addr 172.21.15.83:0/4113727369 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:14.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.430+0000 7fa3affff640 1 -- 172.21.15.83:0/4113727369 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 974203432 0 0) 0x7fa3b80b39d0 con 0x7fa3b80aedf0 2024-04-17T10:48:14.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.430+0000 7fa3affff640 1 -- 172.21.15.83:0/4113727369 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3a4003530 con 0x7fa3b80aedf0 2024-04-17T10:48:14.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.430+0000 7fa3bcd17640 1 --2- 172.21.15.83:0/4113727369 >> v2:172.21.15.167:3300/0 conn(0x7fa3b815cb40 0x7fa3b806abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:14.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.430+0000 7fa3affff640 1 -- 172.21.15.83:0/4113727369 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2738876570 0 0) 0x7fa3a4003530 con 0x7fa3b80aedf0 2024-04-17T10:48:14.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.430+0000 7fa3affff640 1 -- 172.21.15.83:0/4113727369 >> v2:172.21.15.167:3300/0 conn(0x7fa3b815cb40 msgr2=0x7fa3b806abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:14.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.430+0000 7fa3affff640 1 --2- 172.21.15.83:0/4113727369 >> v2:172.21.15.167:3300/0 conn(0x7fa3b815cb40 0x7fa3b806abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:14.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.430+0000 7fa3affff640 1 -- 172.21.15.83:0/4113727369 >> v1:172.21.15.167:6789/0 conn(0x7fa3b806b120 legacy=0x7fa3b8071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:14.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.430+0000 7fa3affff640 1 -- 172.21.15.83:0/4113727369 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b8070f20 con 0x7fa3b80aedf0 2024-04-17T10:48:14.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.430+0000 7fa3affff640 1 -- 172.21.15.83:0/4113727369 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fa3b4001a00 con 0x7fa3b80aedf0 2024-04-17T10:48:14.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.430+0000 7fa3affff640 1 -- 172.21.15.83:0/4113727369 >> v1:172.21.15.83:6789/0 conn(0x7fa3b80aedf0 legacy=0x7fa3b80b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:14.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.430+0000 7fa3affff640 1 --2- 172.21.15.83:0/4113727369 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3a4003a10 0x7fa3a4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:14.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.430+0000 7fa3affff640 1 --2- 172.21.15.83:0/4113727369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3a4004810 0x7fa3a4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:14.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.430+0000 7fa3affff640 1 --2- 172.21.15.83:0/4113727369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3a4005220 0x7fa3a400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:14.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.430+0000 7fa3affff640 1 -- 172.21.15.83:0/4113727369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b8071290 con 0x7fa3a4005220 2024-04-17T10:48:14.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.430+0000 7fa3affff640 1 -- 172.21.15.83:0/4113727369 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a400c170 con 0x7fa3a4004810 2024-04-17T10:48:14.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.430+0000 7fa3affff640 1 -- 172.21.15.83:0/4113727369 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a400c370 con 0x7fa3a4003a10 2024-04-17T10:48:14.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.430+0000 7fa3bdd19640 1 --2- 172.21.15.83:0/4113727369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3a4005220 0x7fa3a400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:14.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.430+0000 7fa3bd518640 1 --2- 172.21.15.83:0/4113727369 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3a4003a10 0x7fa3a4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:14.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.430+0000 7fa3bcd17640 1 --2- 172.21.15.83:0/4113727369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3a4004810 0x7fa3a4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:14.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.430+0000 7fa3bdd19640 1 -- 172.21.15.83:0/4113727369 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3a4003a10 msgr2=0x7fa3a4003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:14.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.430+0000 7fa3bdd19640 1 --2- 172.21.15.83:0/4113727369 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3a4003a10 0x7fa3a4003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:14.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.430+0000 7fa3bdd19640 1 -- 172.21.15.83:0/4113727369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3a4004810 msgr2=0x7fa3a4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:14.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.430+0000 7fa3bdd19640 1 --2- 172.21.15.83:0/4113727369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3a4004810 0x7fa3a4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:14.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.430+0000 7fa3bdd19640 1 -- 172.21.15.83:0/4113727369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a4003530 con 0x7fa3a4005220 2024-04-17T10:48:14.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.434+0000 7fa3bdd19640 1 -- 172.21.15.83:0/4113727369 reap_dead start 2024-04-17T10:48:14.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.434+0000 7fa3bdd19640 1 --2- 172.21.15.83:0/4113727369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3a4005220 0x7fa3a400b9d0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0001780 tx=0x7fa3b001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117dff70082279f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:14.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.434+0000 7fa3affff640 1 -- 172.21.15.83:0/4113727369 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b00013a0 con 0x7fa3a4005220 2024-04-17T10:48:14.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.434+0000 7fa3affff640 1 -- 172.21.15.83:0/4113727369 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b00a9070 con 0x7fa3a4005220 2024-04-17T10:48:14.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.434+0000 7fa3affff640 1 -- 172.21.15.83:0/4113727369 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0003600 con 0x7fa3a4005220 2024-04-17T10:48:14.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.434+0000 7fa3bf77c640 1 -- 172.21.15.83:0/4113727369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3a4005220 msgr2=0x7fa3a400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:14.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.434+0000 7fa3bf77c640 1 --2- 172.21.15.83:0/4113727369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3a4005220 0x7fa3a400b9d0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0001780 tx=0x7fa3b001c420 comp rx=0 tx=0).stop 2024-04-17T10:48:14.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.434+0000 7fa3bf77c640 1 -- 172.21.15.83:0/4113727369 shutdown_connections 2024-04-17T10:48:14.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.434+0000 7fa3bf77c640 1 --2- 172.21.15.83:0/4113727369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3a4005220 0x7fa3a400b9d0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:14.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.434+0000 7fa3bf77c640 1 -- 172.21.15.83:0/4113727369 >> 172.21.15.83:0/4113727369 conn(0x7fa3b80cc050 msgr2=0x7fa3b80cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:14.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.434+0000 7fa3bf77c640 1 -- 172.21.15.83:0/4113727369 shutdown_connections 2024-04-17T10:48:14.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.434+0000 7fa3bf77c640 1 -- 172.21.15.83:0/4113727369 wait complete. 2024-04-17T10:48:14.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.434+0000 7fa3bf77c640 1 Processor -- start 2024-04-17T10:48:14.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.438+0000 7fa3bf77c640 1 -- start start 2024-04-17T10:48:14.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.438+0000 7fa3bf77c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3a4005220 0x7fa3b8151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:14.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.438+0000 7fa3bf77c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3b8156b70 0x7fa3b80b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:14.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.438+0000 7fa3bf77c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3b8156ff0 0x7fa3b80b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:14.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.438+0000 7fa3bf77c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b806c2e0 con 0x7fa3b8156b70 2024-04-17T10:48:14.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.438+0000 7fa3bf77c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b806c160 con 0x7fa3b8156ff0 2024-04-17T10:48:14.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.438+0000 7fa3bf77c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b806abe0 con 0x7fa3a4005220 2024-04-17T10:48:14.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.438+0000 7fa3bcd17640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3b8156b70 0x7fa3b80b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:14.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.438+0000 7fa3bcd17640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3b8156b70 0x7fa3b80b0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39930/0 (socket says 172.21.15.83:39930) 2024-04-17T10:48:14.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.438+0000 7fa3bcd17640 1 -- 172.21.15.83:0/3884702095 learned_addr learned my addr 172.21.15.83:0/3884702095 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:14.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.438+0000 7fa3bdd19640 1 --2- 172.21.15.83:0/3884702095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3b8156ff0 0x7fa3b80b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:14.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.438+0000 7fa3bd518640 1 --2- 172.21.15.83:0/3884702095 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3a4005220 0x7fa3b8151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:14.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.438+0000 7fa3bcd17640 1 -- 172.21.15.83:0/3884702095 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3a4005220 msgr2=0x7fa3b8151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:14.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.438+0000 7fa3bcd17640 1 --2- 172.21.15.83:0/3884702095 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3a4005220 0x7fa3b8151300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:14.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.438+0000 7fa3bcd17640 1 -- 172.21.15.83:0/3884702095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3b8156ff0 msgr2=0x7fa3b80b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:14.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.438+0000 7fa3bcd17640 1 --2- 172.21.15.83:0/3884702095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3b8156ff0 0x7fa3b80b07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:14.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.438+0000 7fa3bcd17640 1 -- 172.21.15.83:0/3884702095 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b81531a0 con 0x7fa3b8156b70 2024-04-17T10:48:14.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.438+0000 7fa3bcd17640 1 --2- 172.21.15.83:0/3884702095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3b8156b70 0x7fa3b80b0280 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fa3a800f5b0 tx=0x7fa3a800fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:14.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.438+0000 7fa3adffb640 1 -- 172.21.15.83:0/3884702095 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a8013810 con 0x7fa3b8156b70 2024-04-17T10:48:14.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.438+0000 7fa3adffb640 1 -- 172.21.15.83:0/3884702095 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a801a070 con 0x7fa3b8156b70 2024-04-17T10:48:14.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.438+0000 7fa3bf77c640 1 -- 172.21.15.83:0/3884702095 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3b8153480 con 0x7fa3b8156b70 2024-04-17T10:48:14.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.438+0000 7fa3adffb640 1 -- 172.21.15.83:0/3884702095 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a8013bf0 con 0x7fa3b8156b70 2024-04-17T10:48:14.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.438+0000 7fa3bf77c640 1 -- 172.21.15.83:0/3884702095 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b8153a40 con 0x7fa3b8156b70 2024-04-17T10:48:14.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.438+0000 7fa3bf77c640 1 -- 172.21.15.83:0/3884702095 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3b8075000 con 0x7fa3b8156b70 2024-04-17T10:48:14.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.438+0000 7fa3adffb640 1 -- 172.21.15.83:0/3884702095 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa3a8012070 con 0x7fa3b8156b70 2024-04-17T10:48:14.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.438+0000 7fa3adffb640 1 --2- 172.21.15.83:0/3884702095 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa39c066f60 0x7fa39c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:14.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.438+0000 7fa3adffb640 1 -- 172.21.15.83:0/3884702095 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa3a8084ef0 con 0x7fa3b8156b70 2024-04-17T10:48:14.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.438+0000 7fa3bd518640 1 --2- 172.21.15.83:0/3884702095 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa39c066f60 0x7fa39c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:14.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.442+0000 7fa3bd518640 1 --2- 172.21.15.83:0/3884702095 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa39c066f60 0x7fa39c069420 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa3b40013e0 tx=0x7fa3b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:14.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.446+0000 7fa3adffb640 1 -- 172.21.15.83:0/3884702095 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa3a8055f30 con 0x7fa3b8156b70 2024-04-17T10:48:14.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:14.606+0000 7fa3bf77c640 1 -- 172.21.15.83:0/3884702095 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa3b806e6d0 con 0x7fa3b8156b70 2024-04-17T10:48:15.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.286+0000 7fa3adffb640 1 -- 172.21.15.83:0/3884702095 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v105) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa3a8055830 con 0x7fa3b8156b70 2024-04-17T10:48:15.285 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2024-04-17T10:48:15.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.294+0000 7fa3bf77c640 1 -- 172.21.15.83:0/3884702095 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa39c066f60 msgr2=0x7fa39c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:15.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.294+0000 7fa3bf77c640 1 --2- 172.21.15.83:0/3884702095 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa39c066f60 0x7fa39c069420 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa3b40013e0 tx=0x7fa3b4009040 comp rx=0 tx=0).stop 2024-04-17T10:48:15.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.294+0000 7fa3bf77c640 1 -- 172.21.15.83:0/3884702095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3b8156b70 msgr2=0x7fa3b80b0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:15.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.294+0000 7fa3bf77c640 1 --2- 172.21.15.83:0/3884702095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3b8156b70 0x7fa3b80b0280 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fa3a800f5b0 tx=0x7fa3a800fa70 comp rx=0 tx=0).stop 2024-04-17T10:48:15.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.294+0000 7fa3bf77c640 1 -- 172.21.15.83:0/3884702095 shutdown_connections 2024-04-17T10:48:15.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.298+0000 7fa3bf77c640 1 --2- 172.21.15.83:0/3884702095 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa39c066f60 0x7fa39c069420 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:15.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.298+0000 7fa3bf77c640 1 --2- 172.21.15.83:0/3884702095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3b8156ff0 0x7fa3b80b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:15.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.298+0000 7fa3bf77c640 1 --2- 172.21.15.83:0/3884702095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3b8156b70 0x7fa3b80b0280 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:15.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.298+0000 7fa3bf77c640 1 --2- 172.21.15.83:0/3884702095 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3a4005220 0x7fa3b8151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:15.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.298+0000 7fa3bf77c640 1 -- 172.21.15.83:0/3884702095 >> 172.21.15.83:0/3884702095 conn(0x7fa3b80cc050 msgr2=0x7fa3b806c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:15.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.298+0000 7fa3bf77c640 1 -- 172.21.15.83:0/3884702095 shutdown_connections 2024-04-17T10:48:15.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.298+0000 7fa3bf77c640 1 -- 172.21.15.83:0/3884702095 wait complete. 2024-04-17T10:48:15.316 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-17T10:48:15.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.446+0000 7ff0be3ef640 1 Processor -- start 2024-04-17T10:48:15.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.446+0000 7ff0be3ef640 1 -- start start 2024-04-17T10:48:15.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.446+0000 7ff0be3ef640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0b80aedf0 0x7ff0b80b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:15.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.446+0000 7ff0be3ef640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0b80b39d0 con 0x7ff0b806b2f0 2024-04-17T10:48:15.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.446+0000 7ff0be3ef640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0b8070c40 con 0x7ff0b815cb40 2024-04-17T10:48:15.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.446+0000 7ff0be3ef640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b8071290 con 0x7ff0b80aedf0 2024-04-17T10:48:15.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.446+0000 7ff0bc98c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff0b806b2f0 0x7ff0b8071720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38258/0 (socket says 172.21.15.83:38258) 2024-04-17T10:48:15.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.446+0000 7ff0bc98c640 1 -- 172.21.15.83:0/880208412 learned_addr learned my addr 172.21.15.83:0/880208412 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:15.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.446+0000 7ff0b7fff640 1 --2- 172.21.15.83:0/880208412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0b80aedf0 0x7ff0b80b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:15.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.446+0000 7ff0b6ffd640 1 -- 172.21.15.83:0/880208412 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1809721929 0 0) 0x7ff0b80b39d0 con 0x7ff0b806b2f0 2024-04-17T10:48:15.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.446+0000 7ff0b6ffd640 1 -- 172.21.15.83:0/880208412 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0a4003530 con 0x7ff0b806b2f0 2024-04-17T10:48:15.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.446+0000 7ff0b6ffd640 1 -- 172.21.15.83:0/880208412 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3565210632 0 0) 0x7ff0a4003530 con 0x7ff0b806b2f0 2024-04-17T10:48:15.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.446+0000 7ff0b6ffd640 1 -- 172.21.15.83:0/880208412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0b80aedf0 msgr2=0x7ff0b80b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:15.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.446+0000 7ff0b6ffd640 1 --2- 172.21.15.83:0/880208412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0b80aedf0 0x7ff0b80b30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:15.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.446+0000 7ff0b6ffd640 1 -- 172.21.15.83:0/880208412 >> v1:172.21.15.167:6789/0 conn(0x7ff0b815cb40 legacy=0x7ff0b806abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:15.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.446+0000 7ff0b6ffd640 1 -- 172.21.15.83:0/880208412 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b8070f20 con 0x7ff0b806b2f0 2024-04-17T10:48:15.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.446+0000 7ff0b6ffd640 1 -- 172.21.15.83:0/880208412 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7ff0a8001680 con 0x7ff0b806b2f0 2024-04-17T10:48:15.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.446+0000 7ff0b6ffd640 1 -- 172.21.15.83:0/880208412 >> v1:172.21.15.83:6789/0 conn(0x7ff0b806b2f0 legacy=0x7ff0b8071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:15.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.446+0000 7ff0b6ffd640 1 --2- 172.21.15.83:0/880208412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0a40039f0 0x7ff0a4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:15.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.446+0000 7ff0b6ffd640 1 --2- 172.21.15.83:0/880208412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0b80aedf0 0x7ff0a4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:15.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.446+0000 7ff0b6ffd640 1 --2- 172.21.15.83:0/880208412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0a4004d50 0x7ff0a40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:15.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.446+0000 7ff0b6ffd640 1 -- 172.21.15.83:0/880208412 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b8071290 con 0x7ff0a4004d50 2024-04-17T10:48:15.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.446+0000 7ff0b6ffd640 1 -- 172.21.15.83:0/880208412 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a400bb00 con 0x7ff0a40039f0 2024-04-17T10:48:15.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.446+0000 7ff0b6ffd640 1 -- 172.21.15.83:0/880208412 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a400bd00 con 0x7ff0b80aedf0 2024-04-17T10:48:15.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.450+0000 7ff0bc98c640 1 --2- 172.21.15.83:0/880208412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0a4004d50 0x7ff0a40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:15.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.450+0000 7ff0b7fff640 1 --2- 172.21.15.83:0/880208412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0a40039f0 0x7ff0a4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:15.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.450+0000 7ff0b77fe640 1 --2- 172.21.15.83:0/880208412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0b80aedf0 0x7ff0a4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:15.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.450+0000 7ff0bc98c640 1 -- 172.21.15.83:0/880208412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0b80aedf0 msgr2=0x7ff0a4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:15.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.450+0000 7ff0bc98c640 1 --2- 172.21.15.83:0/880208412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0b80aedf0 0x7ff0a4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:15.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.450+0000 7ff0bc98c640 1 -- 172.21.15.83:0/880208412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0a40039f0 msgr2=0x7ff0a4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:15.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.450+0000 7ff0bc98c640 1 --2- 172.21.15.83:0/880208412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0a40039f0 0x7ff0a4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:15.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.450+0000 7ff0bc98c640 1 -- 172.21.15.83:0/880208412 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a4003530 con 0x7ff0a4004d50 2024-04-17T10:48:15.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.450+0000 7ff0b77fe640 1 --2- 172.21.15.83:0/880208412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0b80aedf0 0x7ff0a4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:15.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.450+0000 7ff0bc98c640 1 --2- 172.21.15.83:0/880208412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0a4004d50 0x7ff0a40051d0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7ff0a800c240 tx=0x7ff0a801f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc2630d28e19e17 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:15.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.450+0000 7ff0b6ffd640 1 -- 172.21.15.83:0/880208412 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8007d20 con 0x7ff0a4004d50 2024-04-17T10:48:15.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.450+0000 7ff0b6ffd640 1 -- 172.21.15.83:0/880208412 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a8005ae0 con 0x7ff0a4004d50 2024-04-17T10:48:15.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.450+0000 7ff0b6ffd640 1 -- 172.21.15.83:0/880208412 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8007070 con 0x7ff0a4004d50 2024-04-17T10:48:15.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.450+0000 7ff0be3ef640 1 -- 172.21.15.83:0/880208412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0a4004d50 msgr2=0x7ff0a40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:15.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.450+0000 7ff0be3ef640 1 --2- 172.21.15.83:0/880208412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0a4004d50 0x7ff0a40051d0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7ff0a800c240 tx=0x7ff0a801f3f0 comp rx=0 tx=0).stop 2024-04-17T10:48:15.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.450+0000 7ff0bc98c640 1 -- 172.21.15.83:0/880208412 reap_dead start 2024-04-17T10:48:15.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.450+0000 7ff0be3ef640 1 -- 172.21.15.83:0/880208412 shutdown_connections 2024-04-17T10:48:15.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.450+0000 7ff0be3ef640 1 -- 172.21.15.83:0/880208412 >> 172.21.15.83:0/880208412 conn(0x7ff0b80cc050 msgr2=0x7ff0b80cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:15.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.450+0000 7ff0be3ef640 1 -- 172.21.15.83:0/880208412 shutdown_connections 2024-04-17T10:48:15.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.450+0000 7ff0be3ef640 1 -- 172.21.15.83:0/880208412 wait complete. 2024-04-17T10:48:15.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.454+0000 7ff0be3ef640 1 Processor -- start 2024-04-17T10:48:15.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.454+0000 7ff0be3ef640 1 -- start start 2024-04-17T10:48:15.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.454+0000 7ff0be3ef640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0b814dab0 0x7ff0b80b0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:15.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.454+0000 7ff0be3ef640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0b80b0760 0x7ff0b8154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:15.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.454+0000 7ff0be3ef640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0b8155200 0x7ff0b8159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:15.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.454+0000 7ff0be3ef640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b806c2e0 con 0x7ff0b80b0760 2024-04-17T10:48:15.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.454+0000 7ff0be3ef640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b806c160 con 0x7ff0b8155200 2024-04-17T10:48:15.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.454+0000 7ff0be3ef640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b80b30f0 con 0x7ff0b814dab0 2024-04-17T10:48:15.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.454+0000 7ff0b77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0b80b0760 0x7ff0b8154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:15.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.454+0000 7ff0b77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0b80b0760 0x7ff0b8154cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39942/0 (socket says 172.21.15.83:39942) 2024-04-17T10:48:15.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.454+0000 7ff0b77fe640 1 -- 172.21.15.83:0/2834082015 learned_addr learned my addr 172.21.15.83:0/2834082015 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:15.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.454+0000 7ff0b7fff640 1 --2- 172.21.15.83:0/2834082015 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0b814dab0 0x7ff0b80b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:15.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.454+0000 7ff0bc98c640 1 --2- 172.21.15.83:0/2834082015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0b8155200 0x7ff0b8159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:15.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.454+0000 7ff0b77fe640 1 -- 172.21.15.83:0/2834082015 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0b814dab0 msgr2=0x7ff0b80b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:15.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.454+0000 7ff0b77fe640 1 --2- 172.21.15.83:0/2834082015 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0b814dab0 0x7ff0b80b0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:15.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.454+0000 7ff0b77fe640 1 -- 172.21.15.83:0/2834082015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0b8155200 msgr2=0x7ff0b8159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:15.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.454+0000 7ff0b77fe640 1 --2- 172.21.15.83:0/2834082015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0b8155200 0x7ff0b8159520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:15.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.454+0000 7ff0b77fe640 1 -- 172.21.15.83:0/2834082015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b8155680 con 0x7ff0b80b0760 2024-04-17T10:48:15.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.454+0000 7ff0b77fe640 1 --2- 172.21.15.83:0/2834082015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0b80b0760 0x7ff0b8154cc0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000d7a0 tx=0x7ff0a000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:15.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.454+0000 7ff0b4ff9640 1 -- 172.21.15.83:0/2834082015 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0005270 con 0x7ff0b80b0760 2024-04-17T10:48:15.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.454+0000 7ff0b4ff9640 1 -- 172.21.15.83:0/2834082015 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a0017070 con 0x7ff0b80b0760 2024-04-17T10:48:15.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.454+0000 7ff0b4ff9640 1 -- 172.21.15.83:0/2834082015 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a000b6b0 con 0x7ff0b80b0760 2024-04-17T10:48:15.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.454+0000 7ff0be3ef640 1 -- 172.21.15.83:0/2834082015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b814f160 con 0x7ff0b80b0760 2024-04-17T10:48:15.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.454+0000 7ff0be3ef640 1 -- 172.21.15.83:0/2834082015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b814f720 con 0x7ff0b80b0760 2024-04-17T10:48:15.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.454+0000 7ff0be3ef640 1 -- 172.21.15.83:0/2834082015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b8075000 con 0x7ff0b80b0760 2024-04-17T10:48:15.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.454+0000 7ff0b4ff9640 1 -- 172.21.15.83:0/2834082015 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff0a000b850 con 0x7ff0b80b0760 2024-04-17T10:48:15.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.458+0000 7ff0b4ff9640 1 --2- 172.21.15.83:0/2834082015 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff098066f60 0x7ff098069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:15.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.458+0000 7ff0b4ff9640 1 -- 172.21.15.83:0/2834082015 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5357+0+0 (secure 0 0 0) 0x7ff0a0084dd0 con 0x7ff0b80b0760 2024-04-17T10:48:15.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.458+0000 7ff0b7fff640 1 --2- 172.21.15.83:0/2834082015 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff098066f60 0x7ff098069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:15.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.458+0000 7ff0b7fff640 1 --2- 172.21.15.83:0/2834082015 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff098066f60 0x7ff098069420 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0007520 tx=0x7ff0b000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:15.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.462+0000 7ff0b4ff9640 1 -- 172.21.15.83:0/2834082015 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff0a0055f40 con 0x7ff0b80b0760 2024-04-17T10:48:15.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:15.622+0000 7ff0be3ef640 1 -- 172.21.15.83:0/2834082015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff0b80b2cc0 con 0x7ff0b80b0760 2024-04-17T10:48:16.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.290+0000 7ff0b4ff9640 1 -- 172.21.15.83:0/2834082015 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v106) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff0a0086020 con 0x7ff0b80b0760 2024-04-17T10:48:16.288 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2024-04-17T10:48:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.294+0000 7ff0be3ef640 1 -- 172.21.15.83:0/2834082015 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff098066f60 msgr2=0x7ff098069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.294+0000 7ff0be3ef640 1 --2- 172.21.15.83:0/2834082015 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff098066f60 0x7ff098069420 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0007520 tx=0x7ff0b000b040 comp rx=0 tx=0).stop 2024-04-17T10:48:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.294+0000 7ff0be3ef640 1 -- 172.21.15.83:0/2834082015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0b80b0760 msgr2=0x7ff0b8154cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.294+0000 7ff0be3ef640 1 --2- 172.21.15.83:0/2834082015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0b80b0760 0x7ff0b8154cc0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000d7a0 tx=0x7ff0a000dc60 comp rx=0 tx=0).stop 2024-04-17T10:48:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.294+0000 7ff0be3ef640 1 -- 172.21.15.83:0/2834082015 shutdown_connections 2024-04-17T10:48:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.294+0000 7ff0be3ef640 1 --2- 172.21.15.83:0/2834082015 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff098066f60 0x7ff098069420 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.294+0000 7ff0be3ef640 1 --2- 172.21.15.83:0/2834082015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0b8155200 0x7ff0b8159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.294+0000 7ff0be3ef640 1 --2- 172.21.15.83:0/2834082015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0b80b0760 0x7ff0b8154cc0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.294+0000 7ff0be3ef640 1 --2- 172.21.15.83:0/2834082015 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0b814dab0 0x7ff0b80b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.294+0000 7ff0be3ef640 1 -- 172.21.15.83:0/2834082015 >> 172.21.15.83:0/2834082015 conn(0x7ff0b80cc050 msgr2=0x7ff0b8071570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.294+0000 7ff0be3ef640 1 -- 172.21.15.83:0/2834082015 shutdown_connections 2024-04-17T10:48:16.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.294+0000 7ff0be3ef640 1 -- 172.21.15.83:0/2834082015 wait complete. 2024-04-17T10:48:16.322 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-17T10:48:16.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.446+0000 7f70d5657640 1 Processor -- start 2024-04-17T10:48:16.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.446+0000 7f70d5657640 1 -- start start 2024-04-17T10:48:16.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70d5657640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70d015e0f0 0x7f70d00b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:16.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70d5657640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f70d00b5450 0x7f70d00b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:16.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70d5657640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70d015e4d0 con 0x7f70d00ce400 2024-04-17T10:48:16.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70d5657640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d00b5df0 con 0x7f70d00b5450 2024-04-17T10:48:16.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70d5657640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d0162b40 con 0x7f70d015e0f0 2024-04-17T10:48:16.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70c67fc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f70d00ce400 0x7f70d0161c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38270/0 (socket says 172.21.15.83:38270) 2024-04-17T10:48:16.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70c67fc640 1 -- 172.21.15.83:0/4241573061 learned_addr learned my addr 172.21.15.83:0/4241573061 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:16.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70ceffd640 1 --2- 172.21.15.83:0/4241573061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70d015e0f0 0x7f70d00b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:16.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70cf7fe640 1 --2- 172.21.15.83:0/4241573061 >> v2:172.21.15.167:3300/0 conn(0x7f70d00b5450 0x7f70d00b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:16.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70ce7fc640 1 -- 172.21.15.83:0/4241573061 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2231275970 0 0) 0x7f70d015e4d0 con 0x7f70d00ce400 2024-04-17T10:48:16.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70ce7fc640 1 -- 172.21.15.83:0/4241573061 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70bc003530 con 0x7f70d00ce400 2024-04-17T10:48:16.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70ce7fc640 1 -- 172.21.15.83:0/4241573061 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 443316259 0 0) 0x7f70bc003530 con 0x7f70d00ce400 2024-04-17T10:48:16.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70ce7fc640 1 -- 172.21.15.83:0/4241573061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70d015e0f0 msgr2=0x7f70d00b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:16.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70ce7fc640 1 --2- 172.21.15.83:0/4241573061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70d015e0f0 0x7f70d00b6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:16.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70ce7fc640 1 -- 172.21.15.83:0/4241573061 >> v2:172.21.15.167:3300/0 conn(0x7f70d00b5450 msgr2=0x7f70d00b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:16.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70ce7fc640 1 --2- 172.21.15.83:0/4241573061 >> v2:172.21.15.167:3300/0 conn(0x7f70d00b5450 0x7f70d00b58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:16.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70ce7fc640 1 -- 172.21.15.83:0/4241573061 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d0162580 con 0x7f70d00ce400 2024-04-17T10:48:16.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70ce7fc640 1 -- 172.21.15.83:0/4241573061 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f70b8001680 con 0x7f70d00ce400 2024-04-17T10:48:16.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70ce7fc640 1 -- 172.21.15.83:0/4241573061 >> v1:172.21.15.83:6789/0 conn(0x7f70d00ce400 legacy=0x7f70d0161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:16.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70ce7fc640 1 --2- 172.21.15.83:0/4241573061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70bc003a60 0x7f70bc003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:16.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70ce7fc640 1 --2- 172.21.15.83:0/4241573061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70bc004810 0x7f70bc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:16.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70ce7fc640 1 --2- 172.21.15.83:0/4241573061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70d015e0f0 0x7f70bc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:16.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70ce7fc640 1 -- 172.21.15.83:0/4241573061 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d0162b40 con 0x7f70bc003a60 2024-04-17T10:48:16.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70ce7fc640 1 -- 172.21.15.83:0/4241573061 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc00b9d0 con 0x7f70bc004810 2024-04-17T10:48:16.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70ce7fc640 1 -- 172.21.15.83:0/4241573061 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc00bbd0 con 0x7f70d015e0f0 2024-04-17T10:48:16.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70ceffd640 1 --2- 172.21.15.83:0/4241573061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70bc003a60 0x7f70bc003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:16.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70cf7fe640 1 --2- 172.21.15.83:0/4241573061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70d015e0f0 0x7f70bc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:16.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70ceffd640 1 -- 172.21.15.83:0/4241573061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70d015e0f0 msgr2=0x7f70bc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:16.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70ceffd640 1 --2- 172.21.15.83:0/4241573061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70d015e0f0 0x7f70bc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:16.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70ceffd640 1 -- 172.21.15.83:0/4241573061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70bc004810 msgr2=0x7f70bc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:16.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70ceffd640 1 --2- 172.21.15.83:0/4241573061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70bc004810 0x7f70bc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:16.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70ceffd640 1 -- 172.21.15.83:0/4241573061 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70bc003530 con 0x7f70bc003a60 2024-04-17T10:48:16.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70cf7fe640 1 --2- 172.21.15.83:0/4241573061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70d015e0f0 0x7f70bc0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:16.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.450+0000 7f70ceffd640 1 --2- 172.21.15.83:0/4241573061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70bc003a60 0x7f70bc003e80 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f70c0009ae0 tx=0x7f70c001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f23af136311519 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:16.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70ce7fc640 1 -- 172.21.15.83:0/4241573061 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c00b2070 con 0x7f70bc003a60 2024-04-17T10:48:16.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70ce7fc640 1 -- 172.21.15.83:0/4241573061 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c00a5530 con 0x7f70bc003a60 2024-04-17T10:48:16.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70ce7fc640 1 -- 172.21.15.83:0/4241573061 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c00096b0 con 0x7f70bc003a60 2024-04-17T10:48:16.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70d5657640 1 -- 172.21.15.83:0/4241573061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70bc003a60 msgr2=0x7f70bc003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:16.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70d5657640 1 --2- 172.21.15.83:0/4241573061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70bc003a60 0x7f70bc003e80 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f70c0009ae0 tx=0x7f70c001ca70 comp rx=0 tx=0).stop 2024-04-17T10:48:16.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70cf7fe640 1 -- 172.21.15.83:0/4241573061 reap_dead start 2024-04-17T10:48:16.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70d5657640 1 -- 172.21.15.83:0/4241573061 shutdown_connections 2024-04-17T10:48:16.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70d5657640 1 -- 172.21.15.83:0/4241573061 >> 172.21.15.83:0/4241573061 conn(0x7f70d00d0450 msgr2=0x7f70d00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:16.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70d5657640 1 -- 172.21.15.83:0/4241573061 shutdown_connections 2024-04-17T10:48:16.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70d5657640 1 -- 172.21.15.83:0/4241573061 wait complete. 2024-04-17T10:48:16.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70d5657640 1 Processor -- start 2024-04-17T10:48:16.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70d5657640 1 -- start start 2024-04-17T10:48:16.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70d5657640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70d00c9ed0 0x7f70d0155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:16.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70d5657640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70d01572c0 0x7f70d015e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:16.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70d5657640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70d0146b90 0x7f70d015ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:16.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70d5657640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d00d2d50 con 0x7f70d01572c0 2024-04-17T10:48:16.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70d5657640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d00b7110 con 0x7f70d0146b90 2024-04-17T10:48:16.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70d5657640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d00d2ed0 con 0x7f70d00c9ed0 2024-04-17T10:48:16.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70c67fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70d01572c0 0x7f70d015e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:16.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70c67fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70d01572c0 0x7f70d015e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39968/0 (socket says 172.21.15.83:39968) 2024-04-17T10:48:16.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70c67fc640 1 -- 172.21.15.83:0/651577593 learned_addr learned my addr 172.21.15.83:0/651577593 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:16.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70ceffd640 1 --2- 172.21.15.83:0/651577593 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70d00c9ed0 0x7f70d0155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:16.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70c67fc640 1 -- 172.21.15.83:0/651577593 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70d00c9ed0 msgr2=0x7f70d0155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:16.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70c67fc640 1 --2- 172.21.15.83:0/651577593 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70d00c9ed0 0x7f70d0155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:16.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70cf7fe640 1 --2- 172.21.15.83:0/651577593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70d0146b90 0x7f70d015ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:16.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70c67fc640 1 -- 172.21.15.83:0/651577593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70d0146b90 msgr2=0x7f70d015ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:16.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70c67fc640 1 --2- 172.21.15.83:0/651577593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70d0146b90 0x7f70d015ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:16.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70c67fc640 1 -- 172.21.15.83:0/651577593 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d0156220 con 0x7f70d01572c0 2024-04-17T10:48:16.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70ceffd640 1 --2- 172.21.15.83:0/651577593 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70d00c9ed0 0x7f70d0155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:16.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70c67fc640 1 --2- 172.21.15.83:0/651577593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70d01572c0 0x7f70d015e680 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f70b800c6b0 tx=0x7f70b800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:16.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70c7fff640 1 -- 172.21.15.83:0/651577593 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b8018070 con 0x7f70d01572c0 2024-04-17T10:48:16.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70c7fff640 1 -- 172.21.15.83:0/651577593 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b8004ab0 con 0x7f70d01572c0 2024-04-17T10:48:16.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70c7fff640 1 -- 172.21.15.83:0/651577593 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b80070d0 con 0x7f70d01572c0 2024-04-17T10:48:16.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70d5657640 1 -- 172.21.15.83:0/651577593 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d0156500 con 0x7f70d01572c0 2024-04-17T10:48:16.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.454+0000 7f70d5657640 1 -- 172.21.15.83:0/651577593 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d0156ac0 con 0x7f70d01572c0 2024-04-17T10:48:16.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.458+0000 7f70d5657640 1 -- 172.21.15.83:0/651577593 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70d0074f90 con 0x7f70d01572c0 2024-04-17T10:48:16.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.458+0000 7f70c7fff640 1 -- 172.21.15.83:0/651577593 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f70b801b070 con 0x7f70d01572c0 2024-04-17T10:48:16.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.458+0000 7f70c7fff640 1 --2- 172.21.15.83:0/651577593 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f70b0066f90 0x7f70b0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:16.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.458+0000 7f70ceffd640 1 --2- 172.21.15.83:0/651577593 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f70b0066f90 0x7f70b0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:16.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.458+0000 7f70c7fff640 1 -- 172.21.15.83:0/651577593 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f70b80840d0 con 0x7f70d01572c0 2024-04-17T10:48:16.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.458+0000 7f70ceffd640 1 --2- 172.21.15.83:0/651577593 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f70b0066f90 0x7f70b0069450 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f70c00af000 tx=0x7f70c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:16.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.462+0000 7f70c7fff640 1 -- 172.21.15.83:0/651577593 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f70b80553e0 con 0x7f70d01572c0 2024-04-17T10:48:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.618+0000 7f70d5657640 1 -- 172.21.15.83:0/651577593 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f70d00d17c0 con 0x7f70d01572c0 2024-04-17T10:48:16.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.626+0000 7f70c7fff640 1 -- 172.21.15.83:0/651577593 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f70b8054ce0 con 0x7f70d01572c0 2024-04-17T10:48:16.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.626+0000 7f70d5657640 1 -- 172.21.15.83:0/651577593 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f70b0066f90 msgr2=0x7f70b0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:16.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.626+0000 7f70d5657640 1 --2- 172.21.15.83:0/651577593 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f70b0066f90 0x7f70b0069450 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f70c00af000 tx=0x7f70c0008040 comp rx=0 tx=0).stop 2024-04-17T10:48:16.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.626+0000 7f70d5657640 1 -- 172.21.15.83:0/651577593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70d01572c0 msgr2=0x7f70d015e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:16.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.626+0000 7f70d5657640 1 --2- 172.21.15.83:0/651577593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70d01572c0 0x7f70d015e680 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f70b800c6b0 tx=0x7f70b800cb70 comp rx=0 tx=0).stop 2024-04-17T10:48:16.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.626+0000 7f70d5657640 1 -- 172.21.15.83:0/651577593 shutdown_connections 2024-04-17T10:48:16.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.626+0000 7f70d5657640 1 --2- 172.21.15.83:0/651577593 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f70b0066f90 0x7f70b0069450 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:16.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.626+0000 7f70d5657640 1 --2- 172.21.15.83:0/651577593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70d0146b90 0x7f70d015ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:16.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.626+0000 7f70d5657640 1 --2- 172.21.15.83:0/651577593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70d01572c0 0x7f70d015e680 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:16.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.626+0000 7f70d5657640 1 --2- 172.21.15.83:0/651577593 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70d00c9ed0 0x7f70d0155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:16.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.626+0000 7f70d5657640 1 -- 172.21.15.83:0/651577593 >> 172.21.15.83:0/651577593 conn(0x7f70d00d0450 msgr2=0x7f70d015c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:16.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.626+0000 7f70d5657640 1 -- 172.21.15.83:0/651577593 shutdown_connections 2024-04-17T10:48:16.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.626+0000 7f70d5657640 1 -- 172.21.15.83:0/651577593 wait complete. 2024-04-17T10:48:16.641 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume' 2024-04-17T10:48:16.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.754+0000 7fd834386640 1 Processor -- start 2024-04-17T10:48:16.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.754+0000 7fd834386640 1 -- start start 2024-04-17T10:48:16.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.754+0000 7fd834386640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd82c072330 0x7fd82c0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:16.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.754+0000 7fd834386640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd82c0cfa10 0x7fd82c06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:16.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.754+0000 7fd834386640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd82c0cfea0 con 0x7fd82c15ba30 2024-04-17T10:48:16.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.754+0000 7fd834386640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd82c071ea0 con 0x7fd82c0cfa10 2024-04-17T10:48:16.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.754+0000 7fd834386640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd82c06e320 con 0x7fd82c072330 2024-04-17T10:48:16.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.754+0000 7fd832923640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd82c0cfa10 0x7fd82c06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:16.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.754+0000 7fd832923640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd82c0cfa10 0x7fd82c06c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42252/0 (socket says 172.21.15.83:42252) 2024-04-17T10:48:16.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.754+0000 7fd832923640 1 -- 172.21.15.83:0/540237495 learned_addr learned my addr 172.21.15.83:0/540237495 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:16.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.754+0000 7fd831921640 1 --2- 172.21.15.83:0/540237495 >> v2:172.21.15.167:3300/0 conn(0x7fd82c072330 0x7fd82c0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:16.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.754+0000 7fd832923640 1 -- 172.21.15.83:0/540237495 >> v2:172.21.15.167:3300/0 conn(0x7fd82c072330 msgr2=0x7fd82c0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:16.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.754+0000 7fd832923640 1 --2- 172.21.15.83:0/540237495 >> v2:172.21.15.167:3300/0 conn(0x7fd82c072330 0x7fd82c0cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:16.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.754+0000 7fd832923640 1 -- 172.21.15.83:0/540237495 >> v1:172.21.15.167:6789/0 conn(0x7fd82c15ba30 legacy=0x7fd82c15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:16.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.754+0000 7fd832923640 1 -- 172.21.15.83:0/540237495 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd82c06e040 con 0x7fd82c0cfa10 2024-04-17T10:48:16.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.758+0000 7fd832923640 1 --2- 172.21.15.83:0/540237495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd82c0cfa10 0x7fd82c06c490 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0af2b0 tx=0x7fd81c01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d7092ed1de2623 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:16.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.758+0000 7fd831120640 1 -- 172.21.15.83:0/540237495 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81c0a56f0 con 0x7fd82c0cfa10 2024-04-17T10:48:16.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.758+0000 7fd831120640 1 -- 172.21.15.83:0/540237495 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd81c0a5890 con 0x7fd82c0cfa10 2024-04-17T10:48:16.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.758+0000 7fd831120640 1 -- 172.21.15.83:0/540237495 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81c0a5c60 con 0x7fd82c0cfa10 2024-04-17T10:48:16.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.758+0000 7fd834386640 1 -- 172.21.15.83:0/540237495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd82c0cfa10 msgr2=0x7fd82c06c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:16.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.758+0000 7fd834386640 1 --2- 172.21.15.83:0/540237495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd82c0cfa10 0x7fd82c06c490 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0af2b0 tx=0x7fd81c01c920 comp rx=0 tx=0).stop 2024-04-17T10:48:16.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.758+0000 7fd834386640 1 -- 172.21.15.83:0/540237495 shutdown_connections 2024-04-17T10:48:16.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.758+0000 7fd834386640 1 --2- 172.21.15.83:0/540237495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd82c0cfa10 0x7fd82c06c490 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:16.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.758+0000 7fd834386640 1 --2- 172.21.15.83:0/540237495 >> v2:172.21.15.167:3300/0 conn(0x7fd82c072330 0x7fd82c0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:16.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.758+0000 7fd834386640 1 -- 172.21.15.83:0/540237495 >> 172.21.15.83:0/540237495 conn(0x7fd82c06abe0 msgr2=0x7fd82c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:16.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.758+0000 7fd834386640 1 -- 172.21.15.83:0/540237495 shutdown_connections 2024-04-17T10:48:16.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.758+0000 7fd834386640 1 -- 172.21.15.83:0/540237495 wait complete. 2024-04-17T10:48:16.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.758+0000 7fd834386640 1 Processor -- start 2024-04-17T10:48:16.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.758+0000 7fd834386640 1 -- start start 2024-04-17T10:48:16.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.758+0000 7fd834386640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd82c072330 0x7fd82c15d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:16.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.758+0000 7fd834386640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd82c0cfa10 0x7fd82c14f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:16.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.758+0000 7fd834386640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd82c15ba30 0x7fd82c15db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:16.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.758+0000 7fd834386640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82c06de20 con 0x7fd82c072330 2024-04-17T10:48:16.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.758+0000 7fd834386640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82c06dca0 con 0x7fd82c0cfa10 2024-04-17T10:48:16.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.758+0000 7fd834386640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd82c168520 con 0x7fd82c15ba30 2024-04-17T10:48:16.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.758+0000 7fd832122640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd82c072330 0x7fd82c15d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:16.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.758+0000 7fd832122640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd82c072330 0x7fd82c15d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42268/0 (socket says 172.21.15.83:42268) 2024-04-17T10:48:16.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.758+0000 7fd832122640 1 -- 172.21.15.83:0/2061803363 learned_addr learned my addr 172.21.15.83:0/2061803363 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:16.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.762+0000 7fd831921640 1 --2- 172.21.15.83:0/2061803363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd82c0cfa10 0x7fd82c14f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:16.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.762+0000 7fd832923640 1 --2- 172.21.15.83:0/2061803363 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd82c15ba30 0x7fd82c15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:16.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.762+0000 7fd832122640 1 -- 172.21.15.83:0/2061803363 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd82c15ba30 msgr2=0x7fd82c15db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:16.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.762+0000 7fd832122640 1 --2- 172.21.15.83:0/2061803363 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd82c15ba30 0x7fd82c15db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:16.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.762+0000 7fd832122640 1 -- 172.21.15.83:0/2061803363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd82c0cfa10 msgr2=0x7fd82c14f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:16.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.762+0000 7fd832122640 1 --2- 172.21.15.83:0/2061803363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd82c0cfa10 0x7fd82c14f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:16.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.762+0000 7fd832122640 1 -- 172.21.15.83:0/2061803363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd82c15e050 con 0x7fd82c072330 2024-04-17T10:48:16.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.762+0000 7fd831921640 1 --2- 172.21.15.83:0/2061803363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd82c0cfa10 0x7fd82c14f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:16.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.762+0000 7fd832122640 1 --2- 172.21.15.83:0/2061803363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd82c072330 0x7fd82c15d090 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fd82400c450 tx=0x7fd82400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:16.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.762+0000 7fd82affd640 1 -- 172.21.15.83:0/2061803363 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd824012830 con 0x7fd82c072330 2024-04-17T10:48:16.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.762+0000 7fd82affd640 1 -- 172.21.15.83:0/2061803363 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd824017070 con 0x7fd82c072330 2024-04-17T10:48:16.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.762+0000 7fd834386640 1 -- 172.21.15.83:0/2061803363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd82c152540 con 0x7fd82c072330 2024-04-17T10:48:16.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.762+0000 7fd82affd640 1 -- 172.21.15.83:0/2061803363 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd824012c30 con 0x7fd82c072330 2024-04-17T10:48:16.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.762+0000 7fd834386640 1 -- 172.21.15.83:0/2061803363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd82c152b00 con 0x7fd82c072330 2024-04-17T10:48:16.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.762+0000 7fd834386640 1 -- 172.21.15.83:0/2061803363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd82c074f90 con 0x7fd82c072330 2024-04-17T10:48:16.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.762+0000 7fd82affd640 1 -- 172.21.15.83:0/2061803363 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd82401c460 con 0x7fd82c072330 2024-04-17T10:48:16.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.762+0000 7fd82affd640 1 --2- 172.21.15.83:0/2061803363 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd818066f60 0x7fd818069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:16.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.762+0000 7fd82affd640 1 -- 172.21.15.83:0/2061803363 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd8240849f0 con 0x7fd82c072330 2024-04-17T10:48:16.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.762+0000 7fd831921640 1 --2- 172.21.15.83:0/2061803363 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd818066f60 0x7fd818069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:16.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.762+0000 7fd831921640 1 --2- 172.21.15.83:0/2061803363 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd818066f60 0x7fd818069420 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd82c151a70 tx=0x7fd820008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:16.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.766+0000 7fd82affd640 1 -- 172.21.15.83:0/2061803363 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd824055d90 con 0x7fd82c072330 2024-04-17T10:48:16.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:16.970+0000 7fd834386640 1 -- 172.21.15.83:0/2061803363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]} v 0) v1 -- 0x7fd82c0b1080 con 0x7fd82c072330 2024-04-17T10:48:17.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.318+0000 7fd82affd640 1 -- 172.21.15.83:0/2061803363 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]}]=0 v447) v1 ==== 155+0+0 (secure 0 0 0) 0x7fd824055690 con 0x7fd82c072330 2024-04-17T10:48:17.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.322+0000 7fd834386640 1 -- 172.21.15.83:0/2061803363 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd818066f60 msgr2=0x7fd818069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:17.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.322+0000 7fd834386640 1 --2- 172.21.15.83:0/2061803363 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd818066f60 0x7fd818069420 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd82c151a70 tx=0x7fd820008040 comp rx=0 tx=0).stop 2024-04-17T10:48:17.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.322+0000 7fd834386640 1 -- 172.21.15.83:0/2061803363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd82c072330 msgr2=0x7fd82c15d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:17.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.322+0000 7fd834386640 1 --2- 172.21.15.83:0/2061803363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd82c072330 0x7fd82c15d090 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fd82400c450 tx=0x7fd82400c910 comp rx=0 tx=0).stop 2024-04-17T10:48:17.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.322+0000 7fd834386640 1 -- 172.21.15.83:0/2061803363 shutdown_connections 2024-04-17T10:48:17.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.322+0000 7fd834386640 1 --2- 172.21.15.83:0/2061803363 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd818066f60 0x7fd818069420 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:17.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.322+0000 7fd834386640 1 --2- 172.21.15.83:0/2061803363 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd82c15ba30 0x7fd82c15db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:17.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.322+0000 7fd834386640 1 --2- 172.21.15.83:0/2061803363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd82c0cfa10 0x7fd82c14f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:17.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.322+0000 7fd834386640 1 --2- 172.21.15.83:0/2061803363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd82c072330 0x7fd82c15d090 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:17.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.322+0000 7fd834386640 1 -- 172.21.15.83:0/2061803363 >> 172.21.15.83:0/2061803363 conn(0x7fd82c06abe0 msgr2=0x7fd82c155530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:17.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.322+0000 7fd834386640 1 -- 172.21.15.83:0/2061803363 shutdown_connections 2024-04-17T10:48:17.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.322+0000 7fd834386640 1 -- 172.21.15.83:0/2061803363 wait complete. 2024-04-17T10:48:17.337 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-17T10:48:17.337 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume_with_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-17T10:48:17.338 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:48:17.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.462+0000 7fa8739ad640 1 Processor -- start 2024-04-17T10:48:17.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.462+0000 7fa8739ad640 1 -- start start 2024-04-17T10:48:17.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.462+0000 7fa8739ad640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa86c15ddd0 0x7fa86c0d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:17.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.462+0000 7fa8739ad640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa86c0b2f70 0x7fa86c0d2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:17.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.462+0000 7fa8739ad640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa86c0d18c0 con 0x7fa86c06b740 2024-04-17T10:48:17.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.462+0000 7fa8739ad640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa86c0d1e40 con 0x7fa86c0b2f70 2024-04-17T10:48:17.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.462+0000 7fa8739ad640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa86c0d1b60 con 0x7fa86c15ddd0 2024-04-17T10:48:17.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.462+0000 7fa870f48640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa86c0b2f70 0x7fa86c0d2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:17.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.462+0000 7fa870f48640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa86c0b2f70 0x7fa86c0d2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42276/0 (socket says 172.21.15.83:42276) 2024-04-17T10:48:17.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.462+0000 7fa870f48640 1 -- 172.21.15.83:0/2711019821 learned_addr learned my addr 172.21.15.83:0/2711019821 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:17.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.462+0000 7fa870f48640 1 -- 172.21.15.83:0/2711019821 >> v2:172.21.15.167:3300/0 conn(0x7fa86c15ddd0 msgr2=0x7fa86c0d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:17.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.462+0000 7fa870f48640 1 --2- 172.21.15.83:0/2711019821 >> v2:172.21.15.167:3300/0 conn(0x7fa86c15ddd0 0x7fa86c0d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:17.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.462+0000 7fa870f48640 1 -- 172.21.15.83:0/2711019821 >> v1:172.21.15.167:6789/0 conn(0x7fa86c06b740 legacy=0x7fa86c06bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:17.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.462+0000 7fa870f48640 1 -- 172.21.15.83:0/2711019821 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86c06b220 con 0x7fa86c0b2f70 2024-04-17T10:48:17.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.466+0000 7fa870f48640 1 --2- 172.21.15.83:0/2711019821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa86c0b2f70 0x7fa86c0d2560 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fa86c08c0d0 tx=0x7fa85c01c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae01b31f2b2963a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:17.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.466+0000 7fa863fff640 1 -- 172.21.15.83:0/2711019821 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c0b0070 con 0x7fa86c0b2f70 2024-04-17T10:48:17.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.466+0000 7fa863fff640 1 -- 172.21.15.83:0/2711019821 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85c0ab600 con 0x7fa86c0b2f70 2024-04-17T10:48:17.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.466+0000 7fa863fff640 1 -- 172.21.15.83:0/2711019821 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c0ab9d0 con 0x7fa86c0b2f70 2024-04-17T10:48:17.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.466+0000 7fa8739ad640 1 -- 172.21.15.83:0/2711019821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa86c0b2f70 msgr2=0x7fa86c0d2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:17.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.466+0000 7fa8739ad640 1 --2- 172.21.15.83:0/2711019821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa86c0b2f70 0x7fa86c0d2560 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fa86c08c0d0 tx=0x7fa85c01c360 comp rx=0 tx=0).stop 2024-04-17T10:48:17.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.466+0000 7fa8739ad640 1 -- 172.21.15.83:0/2711019821 shutdown_connections 2024-04-17T10:48:17.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.466+0000 7fa8739ad640 1 --2- 172.21.15.83:0/2711019821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa86c0b2f70 0x7fa86c0d2560 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:17.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.466+0000 7fa8739ad640 1 --2- 172.21.15.83:0/2711019821 >> v2:172.21.15.167:3300/0 conn(0x7fa86c15ddd0 0x7fa86c0d2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:17.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.466+0000 7fa8739ad640 1 -- 172.21.15.83:0/2711019821 >> 172.21.15.83:0/2711019821 conn(0x7fa86c0d3a40 msgr2=0x7fa86c0d3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:17.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.466+0000 7fa8739ad640 1 -- 172.21.15.83:0/2711019821 shutdown_connections 2024-04-17T10:48:17.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.466+0000 7fa8739ad640 1 -- 172.21.15.83:0/2711019821 wait complete. 2024-04-17T10:48:17.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.466+0000 7fa8739ad640 1 Processor -- start 2024-04-17T10:48:17.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.466+0000 7fa8739ad640 1 -- start start 2024-04-17T10:48:17.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.466+0000 7fa8739ad640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa86c06b740 0x7fa86c0b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:17.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.466+0000 7fa8739ad640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa86c0b2f70 0x7fa86c0b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:17.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.466+0000 7fa8739ad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa86c15ddd0 0x7fa86c0b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:17.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.466+0000 7fa8739ad640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c16abd0 con 0x7fa86c15ddd0 2024-04-17T10:48:17.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.466+0000 7fa8739ad640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c16aa50 con 0x7fa86c0b2f70 2024-04-17T10:48:17.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.466+0000 7fa8739ad640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c16ad50 con 0x7fa86c06b740 2024-04-17T10:48:17.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.466+0000 7fa871f4a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa86c15ddd0 0x7fa86c0b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:17.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.466+0000 7fa871f4a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa86c15ddd0 0x7fa86c0b48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42288/0 (socket says 172.21.15.83:42288) 2024-04-17T10:48:17.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.466+0000 7fa871f4a640 1 -- 172.21.15.83:0/3844413145 learned_addr learned my addr 172.21.15.83:0/3844413145 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:17.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.466+0000 7fa870f48640 1 --2- 172.21.15.83:0/3844413145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa86c0b2f70 0x7fa86c0b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:17.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.466+0000 7fa871f4a640 1 -- 172.21.15.83:0/3844413145 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa86c06b740 msgr2=0x7fa86c0b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:17.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.466+0000 7fa871749640 1 --2- 172.21.15.83:0/3844413145 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa86c06b740 0x7fa86c0b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:17.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.466+0000 7fa871f4a640 1 --2- 172.21.15.83:0/3844413145 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa86c06b740 0x7fa86c0b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:17.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.466+0000 7fa871f4a640 1 -- 172.21.15.83:0/3844413145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa86c0b2f70 msgr2=0x7fa86c0b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:17.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.466+0000 7fa871f4a640 1 --2- 172.21.15.83:0/3844413145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa86c0b2f70 0x7fa86c0b4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:17.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.466+0000 7fa871f4a640 1 -- 172.21.15.83:0/3844413145 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86c0b5d60 con 0x7fa86c15ddd0 2024-04-17T10:48:17.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.466+0000 7fa870f48640 1 --2- 172.21.15.83:0/3844413145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa86c0b2f70 0x7fa86c0b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:48:17.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.466+0000 7fa871749640 1 --2- 172.21.15.83:0/3844413145 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa86c06b740 0x7fa86c0b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:17.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.470+0000 7fa871f4a640 1 --2- 172.21.15.83:0/3844413145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa86c15ddd0 0x7fa86c0b48b0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fa86400c7b0 tx=0x7fa86400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:17.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.470+0000 7fa861ffb640 1 -- 172.21.15.83:0/3844413145 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa864017070 con 0x7fa86c15ddd0 2024-04-17T10:48:17.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.470+0000 7fa861ffb640 1 -- 172.21.15.83:0/3844413145 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa864003db0 con 0x7fa86c15ddd0 2024-04-17T10:48:17.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.470+0000 7fa8739ad640 1 -- 172.21.15.83:0/3844413145 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa86c0b6040 con 0x7fa86c15ddd0 2024-04-17T10:48:17.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.470+0000 7fa861ffb640 1 -- 172.21.15.83:0/3844413145 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8640052b0 con 0x7fa86c15ddd0 2024-04-17T10:48:17.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.470+0000 7fa8739ad640 1 -- 172.21.15.83:0/3844413145 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa86c0b3630 con 0x7fa86c15ddd0 2024-04-17T10:48:17.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.470+0000 7fa8739ad640 1 -- 172.21.15.83:0/3844413145 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa86c074f90 con 0x7fa86c15ddd0 2024-04-17T10:48:17.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.470+0000 7fa861ffb640 1 -- 172.21.15.83:0/3844413145 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa86401a070 con 0x7fa86c15ddd0 2024-04-17T10:48:17.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.470+0000 7fa861ffb640 1 --2- 172.21.15.83:0/3844413145 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa850066f60 0x7fa850069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:17.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.470+0000 7fa871749640 1 --2- 172.21.15.83:0/3844413145 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa850066f60 0x7fa850069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:17.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.470+0000 7fa861ffb640 1 -- 172.21.15.83:0/3844413145 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa864084480 con 0x7fa86c15ddd0 2024-04-17T10:48:17.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.470+0000 7fa871749640 1 --2- 172.21.15.83:0/3844413145 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa850066f60 0x7fa850069420 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa86800a870 tx=0x7fa868008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:17.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.474+0000 7fa861ffb640 1 -- 172.21.15.83:0/3844413145 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa8640556f0 con 0x7fa86c15ddd0 2024-04-17T10:48:17.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.634+0000 7fa8739ad640 1 -- 172.21.15.83:0/3844413145 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa86c161130 con 0x7fa86c15ddd0 2024-04-17T10:48:17.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.638+0000 7fa861ffb640 1 -- 172.21.15.83:0/3844413145 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+12958 (secure 0 0 0) 0x7fa864054ff0 con 0x7fa86c15ddd0 2024-04-17T10:48:17.634 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:17.634 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":106,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:48:16.293100+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/2353406669":"2024-04-18T10:48:13.280219+0000","172.21.15.83:6836/2353406669":"2024-04-18T10:48:13.280219+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:48:17.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.638+0000 7fa8739ad640 1 -- 172.21.15.83:0/3844413145 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa850066f60 msgr2=0x7fa850069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:17.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.638+0000 7fa8739ad640 1 --2- 172.21.15.83:0/3844413145 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa850066f60 0x7fa850069420 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa86800a870 tx=0x7fa868008040 comp rx=0 tx=0).stop 2024-04-17T10:48:17.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.638+0000 7fa8739ad640 1 -- 172.21.15.83:0/3844413145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa86c15ddd0 msgr2=0x7fa86c0b48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:17.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.638+0000 7fa8739ad640 1 --2- 172.21.15.83:0/3844413145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa86c15ddd0 0x7fa86c0b48b0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fa86400c7b0 tx=0x7fa86400cc70 comp rx=0 tx=0).stop 2024-04-17T10:48:17.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.638+0000 7fa8739ad640 1 -- 172.21.15.83:0/3844413145 shutdown_connections 2024-04-17T10:48:17.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.638+0000 7fa8739ad640 1 --2- 172.21.15.83:0/3844413145 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa850066f60 0x7fa850069420 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:17.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.638+0000 7fa8739ad640 1 --2- 172.21.15.83:0/3844413145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa86c15ddd0 0x7fa86c0b48b0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:17.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.638+0000 7fa8739ad640 1 --2- 172.21.15.83:0/3844413145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa86c0b2f70 0x7fa86c0b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:17.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.638+0000 7fa8739ad640 1 --2- 172.21.15.83:0/3844413145 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa86c06b740 0x7fa86c0b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:17.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.642+0000 7fa8739ad640 1 -- 172.21.15.83:0/3844413145 >> 172.21.15.83:0/3844413145 conn(0x7fa86c0d3a40 msgr2=0x7fa86c0d3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:17.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.642+0000 7fa8739ad640 1 -- 172.21.15.83:0/3844413145 shutdown_connections 2024-04-17T10:48:17.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.642+0000 7fa8739ad640 1 -- 172.21.15.83:0/3844413145 wait complete. 2024-04-17T10:48:17.654 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:48:17.654 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:48:17.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbf4e31640 1 Processor -- start 2024-04-17T10:48:17.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbf4e31640 1 -- start start 2024-04-17T10:48:17.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbf4e31640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbf015a8a0 0x7fcbf006e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:17.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbf4e31640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcbf00b36f0 0x7fcbf006bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:17.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbf4e31640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbf00afbb0 con 0x7fcbf006c230 2024-04-17T10:48:17.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbf4e31640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbf006d760 con 0x7fcbf00b36f0 2024-04-17T10:48:17.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbf4e31640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf006da40 con 0x7fcbf015a8a0 2024-04-17T10:48:17.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbeed9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcbf006c230 0x7fcbf00cd720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51216/0 (socket says 172.21.15.83:51216) 2024-04-17T10:48:17.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbeed9d640 1 -- 172.21.15.83:0/2527371095 learned_addr learned my addr 172.21.15.83:0/2527371095 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:17.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbee59c640 1 --2- 172.21.15.83:0/2527371095 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbf015a8a0 0x7fcbf006e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:17.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbedd9b640 1 --2- 172.21.15.83:0/2527371095 >> v2:172.21.15.167:3300/0 conn(0x7fcbf00b36f0 0x7fcbf006bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:17.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbed59a640 1 -- 172.21.15.83:0/2527371095 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1306062388 0 0) 0x7fcbf00afbb0 con 0x7fcbf006c230 2024-04-17T10:48:17.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbed59a640 1 -- 172.21.15.83:0/2527371095 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbd8003530 con 0x7fcbf006c230 2024-04-17T10:48:17.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbed59a640 1 -- 172.21.15.83:0/2527371095 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3831378163 0 0) 0x7fcbd8003530 con 0x7fcbf006c230 2024-04-17T10:48:17.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbed59a640 1 -- 172.21.15.83:0/2527371095 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbf015a8a0 msgr2=0x7fcbf006e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:17.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbed59a640 1 --2- 172.21.15.83:0/2527371095 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbf015a8a0 0x7fcbf006e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:17.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbed59a640 1 -- 172.21.15.83:0/2527371095 >> v2:172.21.15.167:3300/0 conn(0x7fcbf00b36f0 msgr2=0x7fcbf006bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:17.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbed59a640 1 --2- 172.21.15.83:0/2527371095 >> v2:172.21.15.167:3300/0 conn(0x7fcbf00b36f0 0x7fcbf006bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:17.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbed59a640 1 -- 172.21.15.83:0/2527371095 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf006dd20 con 0x7fcbf006c230 2024-04-17T10:48:17.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbed59a640 1 -- 172.21.15.83:0/2527371095 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fcbe8001a00 con 0x7fcbf006c230 2024-04-17T10:48:17.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbed59a640 1 -- 172.21.15.83:0/2527371095 >> v1:172.21.15.83:6789/0 conn(0x7fcbf006c230 legacy=0x7fcbf00cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:17.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbed59a640 1 --2- 172.21.15.83:0/2527371095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbd8003a30 0x7fcbd8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:17.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbed59a640 1 --2- 172.21.15.83:0/2527371095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbd8004810 0x7fcbd8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:17.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbed59a640 1 --2- 172.21.15.83:0/2527371095 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbf015a8a0 0x7fcbd80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:17.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbed59a640 1 -- 172.21.15.83:0/2527371095 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf006d760 con 0x7fcbd8003a30 2024-04-17T10:48:17.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbed59a640 1 -- 172.21.15.83:0/2527371095 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf006da40 con 0x7fcbd8004810 2024-04-17T10:48:17.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbed59a640 1 -- 172.21.15.83:0/2527371095 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd800b9d0 con 0x7fcbf015a8a0 2024-04-17T10:48:17.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbee59c640 1 --2- 172.21.15.83:0/2527371095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbd8003a30 0x7fcbd8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:17.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbee59c640 1 -- 172.21.15.83:0/2527371095 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbf015a8a0 msgr2=0x7fcbd80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:17.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbeed9d640 1 --2- 172.21.15.83:0/2527371095 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbf015a8a0 0x7fcbd80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:17.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbedd9b640 1 --2- 172.21.15.83:0/2527371095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbd8004810 0x7fcbd8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:17.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbee59c640 1 --2- 172.21.15.83:0/2527371095 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbf015a8a0 0x7fcbd80051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:17.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbee59c640 1 -- 172.21.15.83:0/2527371095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbd8004810 msgr2=0x7fcbd8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:17.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbee59c640 1 --2- 172.21.15.83:0/2527371095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbd8004810 0x7fcbd8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:17.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbee59c640 1 -- 172.21.15.83:0/2527371095 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd8003530 con 0x7fcbd8003a30 2024-04-17T10:48:17.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.786+0000 7fcbedd9b640 1 --2- 172.21.15.83:0/2527371095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbd8004810 0x7fcbd8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:17.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.790+0000 7fcbee59c640 1 --2- 172.21.15.83:0/2527371095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbd8003a30 0x7fcbd8003e50 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0009b10 tx=0x7fcbe001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37a3664eb2d3414 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:17.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.790+0000 7fcbed59a640 1 -- 172.21.15.83:0/2527371095 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe00096b0 con 0x7fcbd8003a30 2024-04-17T10:48:17.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.790+0000 7fcbed59a640 1 -- 172.21.15.83:0/2527371095 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe00a9470 con 0x7fcbd8003a30 2024-04-17T10:48:17.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.790+0000 7fcbed59a640 1 -- 172.21.15.83:0/2527371095 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe00a9860 con 0x7fcbd8003a30 2024-04-17T10:48:17.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.790+0000 7fcbf4e31640 1 -- 172.21.15.83:0/2527371095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbd8003a30 msgr2=0x7fcbd8003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:17.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.790+0000 7fcbf4e31640 1 --2- 172.21.15.83:0/2527371095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbd8003a30 0x7fcbd8003e50 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0009b10 tx=0x7fcbe001ca70 comp rx=0 tx=0).stop 2024-04-17T10:48:17.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.790+0000 7fcbeed9d640 1 -- 172.21.15.83:0/2527371095 reap_dead start 2024-04-17T10:48:17.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.790+0000 7fcbf4e31640 1 -- 172.21.15.83:0/2527371095 shutdown_connections 2024-04-17T10:48:17.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.790+0000 7fcbf4e31640 1 -- 172.21.15.83:0/2527371095 >> 172.21.15.83:0/2527371095 conn(0x7fcbf00b23f0 msgr2=0x7fcbf00b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:17.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.790+0000 7fcbf4e31640 1 -- 172.21.15.83:0/2527371095 shutdown_connections 2024-04-17T10:48:17.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.790+0000 7fcbf4e31640 1 -- 172.21.15.83:0/2527371095 wait complete. 2024-04-17T10:48:17.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.790+0000 7fcbf4e31640 1 Processor -- start 2024-04-17T10:48:17.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.790+0000 7fcbf4e31640 1 -- start start 2024-04-17T10:48:17.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.794+0000 7fcbf4e31640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbd8003a30 0x7fcbf01575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:17.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.794+0000 7fcbf4e31640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbf0155930 0x7fcbf014ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:17.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.794+0000 7fcbf4e31640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbf014d8f0 0x7fcbf00b0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:17.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.794+0000 7fcbf4e31640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf00b3c50 con 0x7fcbf014d8f0 2024-04-17T10:48:17.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.794+0000 7fcbf4e31640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf00b3ad0 con 0x7fcbf0155930 2024-04-17T10:48:17.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.794+0000 7fcbf4e31640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf006e800 con 0x7fcbd8003a30 2024-04-17T10:48:17.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.794+0000 7fcbeed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbf014d8f0 0x7fcbf00b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:17.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.794+0000 7fcbeed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbf014d8f0 0x7fcbf00b0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42306/0 (socket says 172.21.15.83:42306) 2024-04-17T10:48:17.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.794+0000 7fcbeed9d640 1 -- 172.21.15.83:0/498710967 learned_addr learned my addr 172.21.15.83:0/498710967 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:17.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.794+0000 7fcbedd9b640 1 --2- 172.21.15.83:0/498710967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbf0155930 0x7fcbf014ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:17.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.794+0000 7fcbee59c640 1 --2- 172.21.15.83:0/498710967 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbd8003a30 0x7fcbf01575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:17.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.794+0000 7fcbeed9d640 1 -- 172.21.15.83:0/498710967 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbd8003a30 msgr2=0x7fcbf01575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:17.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.794+0000 7fcbeed9d640 1 --2- 172.21.15.83:0/498710967 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbd8003a30 0x7fcbf01575f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:17.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.794+0000 7fcbeed9d640 1 -- 172.21.15.83:0/498710967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbf0155930 msgr2=0x7fcbf014ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:17.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.794+0000 7fcbeed9d640 1 --2- 172.21.15.83:0/498710967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbf0155930 0x7fcbf014ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:17.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.794+0000 7fcbeed9d640 1 -- 172.21.15.83:0/498710967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf0155db0 con 0x7fcbf014d8f0 2024-04-17T10:48:17.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.794+0000 7fcbedd9b640 1 --2- 172.21.15.83:0/498710967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbf0155930 0x7fcbf014ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:48:17.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.794+0000 7fcbeed9d640 1 --2- 172.21.15.83:0/498710967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbf014d8f0 0x7fcbf00b0730 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00b5400 tx=0x7fcbe800b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:17.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.794+0000 7fcbdeffd640 1 -- 172.21.15.83:0/498710967 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe800b8d0 con 0x7fcbf014d8f0 2024-04-17T10:48:17.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.794+0000 7fcbdeffd640 1 -- 172.21.15.83:0/498710967 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe800ba70 con 0x7fcbf014d8f0 2024-04-17T10:48:17.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.794+0000 7fcbf4e31640 1 -- 172.21.15.83:0/498710967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf0154150 con 0x7fcbf014d8f0 2024-04-17T10:48:17.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.794+0000 7fcbdeffd640 1 -- 172.21.15.83:0/498710967 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe800be60 con 0x7fcbf014d8f0 2024-04-17T10:48:17.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.794+0000 7fcbf4e31640 1 -- 172.21.15.83:0/498710967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf01543f0 con 0x7fcbf014d8f0 2024-04-17T10:48:17.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.794+0000 7fcbf4e31640 1 -- 172.21.15.83:0/498710967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbf0074f90 con 0x7fcbf014d8f0 2024-04-17T10:48:17.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.794+0000 7fcbdeffd640 1 -- 172.21.15.83:0/498710967 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcbe801b020 con 0x7fcbf014d8f0 2024-04-17T10:48:17.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.794+0000 7fcbdeffd640 1 --2- 172.21.15.83:0/498710967 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcbd0066f60 0x7fcbd0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:17.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.794+0000 7fcbdeffd640 1 -- 172.21.15.83:0/498710967 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcbe8084e60 con 0x7fcbf014d8f0 2024-04-17T10:48:17.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.794+0000 7fcbee59c640 1 --2- 172.21.15.83:0/498710967 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcbd0066f60 0x7fcbd0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:17.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.794+0000 7fcbee59c640 1 --2- 172.21.15.83:0/498710967 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcbd0066f60 0x7fcbd0069420 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fcbe00ae000 tx=0x7fcbe00b3040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:17.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.798+0000 7fcbdeffd640 1 -- 172.21.15.83:0/498710967 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcbf0155db0 con 0x7fcbf014d8f0 2024-04-17T10:48:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.958+0000 7fcbf4e31640 1 -- 172.21.15.83:0/498710967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcbf006f340 con 0x7fcbf014d8f0 2024-04-17T10:48:17.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.958+0000 7fcbdeffd640 1 -- 172.21.15.83:0/498710967 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcbe8056200 con 0x7fcbf014d8f0 2024-04-17T10:48:17.955 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:48:17.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.958+0000 7fcbf4e31640 1 -- 172.21.15.83:0/498710967 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcbd0066f60 msgr2=0x7fcbd0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:17.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.958+0000 7fcbf4e31640 1 --2- 172.21.15.83:0/498710967 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcbd0066f60 0x7fcbd0069420 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fcbe00ae000 tx=0x7fcbe00b3040 comp rx=0 tx=0).stop 2024-04-17T10:48:17.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.958+0000 7fcbf4e31640 1 -- 172.21.15.83:0/498710967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbf014d8f0 msgr2=0x7fcbf00b0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:17.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.958+0000 7fcbf4e31640 1 --2- 172.21.15.83:0/498710967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbf014d8f0 0x7fcbf00b0730 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00b5400 tx=0x7fcbe800b0d0 comp rx=0 tx=0).stop 2024-04-17T10:48:17.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.958+0000 7fcbf4e31640 1 -- 172.21.15.83:0/498710967 shutdown_connections 2024-04-17T10:48:17.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.958+0000 7fcbf4e31640 1 --2- 172.21.15.83:0/498710967 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcbd0066f60 0x7fcbd0069420 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:17.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.958+0000 7fcbf4e31640 1 --2- 172.21.15.83:0/498710967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbf014d8f0 0x7fcbf00b0730 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:17.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.958+0000 7fcbf4e31640 1 --2- 172.21.15.83:0/498710967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbf0155930 0x7fcbf014ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:17.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.958+0000 7fcbf4e31640 1 --2- 172.21.15.83:0/498710967 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbd8003a30 0x7fcbf01575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:17.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.958+0000 7fcbf4e31640 1 -- 172.21.15.83:0/498710967 >> 172.21.15.83:0/498710967 conn(0x7fcbf00b23f0 msgr2=0x7fcbf00b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:17.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.958+0000 7fcbf4e31640 1 -- 172.21.15.83:0/498710967 shutdown_connections 2024-04-17T10:48:17.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:17.958+0000 7fcbf4e31640 1 -- 172.21.15.83:0/498710967 wait complete. 2024-04-17T10:48:17.975 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort' 2024-04-17T10:48:18.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.098+0000 7fc58486b640 1 Processor -- start 2024-04-17T10:48:18.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.098+0000 7fc58486b640 1 -- start start 2024-04-17T10:48:18.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.098+0000 7fc58486b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5800b0a70 0x7fc5800d2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:18.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.098+0000 7fc58486b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc580162150 0x7fc58006c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:18.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.098+0000 7fc58486b640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc58006e980 con 0x7fc5800ce750 2024-04-17T10:48:18.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.098+0000 7fc58486b640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5801625e0 con 0x7fc580162150 2024-04-17T10:48:18.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.098+0000 7fc58486b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc58006f290 con 0x7fc5800b0a70 2024-04-17T10:48:18.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.098+0000 7fc57dd9b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5800b0a70 0x7fc5800d2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:18.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.098+0000 7fc57e59c640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fc5800ce750 0x7fc580161a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:58512/0 (socket says 172.21.15.83:58512) 2024-04-17T10:48:18.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.098+0000 7fc57e59c640 1 -- 172.21.15.83:0/703881858 learned_addr learned my addr 172.21.15.83:0/703881858 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:18.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.098+0000 7fc57ed9d640 1 --2- 172.21.15.83:0/703881858 >> v2:172.21.15.167:3300/0 conn(0x7fc580162150 0x7fc58006c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:18.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.098+0000 7fc57d59a640 1 -- 172.21.15.83:0/703881858 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3798573530 0 0) 0x7fc58006e980 con 0x7fc5800ce750 2024-04-17T10:48:18.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.098+0000 7fc57d59a640 1 -- 172.21.15.83:0/703881858 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc568003530 con 0x7fc5800ce750 2024-04-17T10:48:18.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.098+0000 7fc57dd9b640 1 -- 172.21.15.83:0/703881858 >> v2:172.21.15.167:3300/0 conn(0x7fc580162150 msgr2=0x7fc58006c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:18.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.098+0000 7fc57dd9b640 1 --2- 172.21.15.83:0/703881858 >> v2:172.21.15.167:3300/0 conn(0x7fc580162150 0x7fc58006c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:18.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.098+0000 7fc57dd9b640 1 -- 172.21.15.83:0/703881858 >> v1:172.21.15.167:6789/0 conn(0x7fc5800ce750 legacy=0x7fc580161a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:18.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.098+0000 7fc57dd9b640 1 -- 172.21.15.83:0/703881858 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc580162810 con 0x7fc5800b0a70 2024-04-17T10:48:18.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.102+0000 7fc57dd9b640 1 --2- 172.21.15.83:0/703881858 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5800b0a70 0x7fc5800d2290 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc58006f410 tx=0x7fc57001cf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c98ef1c269229bb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.102+0000 7fc57d59a640 1 -- 172.21.15.83:0/703881858 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5700b1070 con 0x7fc5800b0a70 2024-04-17T10:48:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.102+0000 7fc57d59a640 1 -- 172.21.15.83:0/703881858 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5700ac410 con 0x7fc5800b0a70 2024-04-17T10:48:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.102+0000 7fc57d59a640 1 -- 172.21.15.83:0/703881858 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5700ac7e0 con 0x7fc5800b0a70 2024-04-17T10:48:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.102+0000 7fc58486b640 1 -- 172.21.15.83:0/703881858 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5800b0a70 msgr2=0x7fc5800d2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.102+0000 7fc58486b640 1 --2- 172.21.15.83:0/703881858 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5800b0a70 0x7fc5800d2290 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc58006f410 tx=0x7fc57001cf00 comp rx=0 tx=0).stop 2024-04-17T10:48:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.102+0000 7fc58486b640 1 -- 172.21.15.83:0/703881858 shutdown_connections 2024-04-17T10:48:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.102+0000 7fc58486b640 1 --2- 172.21.15.83:0/703881858 >> v2:172.21.15.167:3300/0 conn(0x7fc580162150 0x7fc58006c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.102+0000 7fc58486b640 1 --2- 172.21.15.83:0/703881858 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5800b0a70 0x7fc5800d2290 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.102+0000 7fc58486b640 1 -- 172.21.15.83:0/703881858 >> 172.21.15.83:0/703881858 conn(0x7fc5800d1770 msgr2=0x7fc5800d1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.102+0000 7fc58486b640 1 -- 172.21.15.83:0/703881858 shutdown_connections 2024-04-17T10:48:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.102+0000 7fc58486b640 1 -- 172.21.15.83:0/703881858 wait complete. 2024-04-17T10:48:18.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.102+0000 7fc58486b640 1 Processor -- start 2024-04-17T10:48:18.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.102+0000 7fc58486b640 1 -- start start 2024-04-17T10:48:18.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.106+0000 7fc58486b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5800b0a70 0x7fc58014e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:18.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.106+0000 7fc58486b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5800ce750 0x7fc580161070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:18.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.106+0000 7fc58486b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc580162150 0x7fc5800b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:18.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.106+0000 7fc58486b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58006d5d0 con 0x7fc580162150 2024-04-17T10:48:18.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.106+0000 7fc58486b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58006d450 con 0x7fc5800ce750 2024-04-17T10:48:18.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.106+0000 7fc58486b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc58006d750 con 0x7fc5800b0a70 2024-04-17T10:48:18.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.106+0000 7fc57ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc580162150 0x7fc5800b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:18.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.106+0000 7fc57ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc580162150 0x7fc5800b2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42312/0 (socket says 172.21.15.83:42312) 2024-04-17T10:48:18.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.106+0000 7fc57ed9d640 1 -- 172.21.15.83:0/3366603014 learned_addr learned my addr 172.21.15.83:0/3366603014 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:18.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.106+0000 7fc57e59c640 1 --2- 172.21.15.83:0/3366603014 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5800b0a70 0x7fc58014e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:18.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.106+0000 7fc57dd9b640 1 --2- 172.21.15.83:0/3366603014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5800ce750 0x7fc580161070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:18.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.106+0000 7fc57ed9d640 1 -- 172.21.15.83:0/3366603014 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5800b0a70 msgr2=0x7fc58014e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:18.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.106+0000 7fc57ed9d640 1 --2- 172.21.15.83:0/3366603014 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5800b0a70 0x7fc58014e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:18.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.106+0000 7fc57ed9d640 1 -- 172.21.15.83:0/3366603014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5800ce750 msgr2=0x7fc580161070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:18.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.106+0000 7fc57ed9d640 1 --2- 172.21.15.83:0/3366603014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5800ce750 0x7fc580161070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:18.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.106+0000 7fc57ed9d640 1 -- 172.21.15.83:0/3366603014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc58014ec10 con 0x7fc580162150 2024-04-17T10:48:18.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.106+0000 7fc57e59c640 1 --2- 172.21.15.83:0/3366603014 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5800b0a70 0x7fc58014e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:48:18.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.106+0000 7fc57dd9b640 1 --2- 172.21.15.83:0/3366603014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5800ce750 0x7fc580161070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:48:18.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.106+0000 7fc57ed9d640 1 --2- 172.21.15.83:0/3366603014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc580162150 0x7fc5800b2fa0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fc57800efc0 tx=0x7fc57800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:18.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.106+0000 7fc56effd640 1 -- 172.21.15.83:0/3366603014 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc578018020 con 0x7fc580162150 2024-04-17T10:48:18.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.106+0000 7fc56effd640 1 -- 172.21.15.83:0/3366603014 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc578014480 con 0x7fc580162150 2024-04-17T10:48:18.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.106+0000 7fc58486b640 1 -- 172.21.15.83:0/3366603014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc58014eef0 con 0x7fc580162150 2024-04-17T10:48:18.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.106+0000 7fc56effd640 1 -- 172.21.15.83:0/3366603014 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc578014870 con 0x7fc580162150 2024-04-17T10:48:18.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.106+0000 7fc58486b640 1 -- 172.21.15.83:0/3366603014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc58014f4b0 con 0x7fc580162150 2024-04-17T10:48:18.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.106+0000 7fc58486b640 1 -- 172.21.15.83:0/3366603014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc580074f90 con 0x7fc580162150 2024-04-17T10:48:18.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.106+0000 7fc56effd640 1 -- 172.21.15.83:0/3366603014 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc57801b020 con 0x7fc580162150 2024-04-17T10:48:18.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.106+0000 7fc56effd640 1 --2- 172.21.15.83:0/3366603014 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc560066f60 0x7fc560069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:18.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.106+0000 7fc57e59c640 1 --2- 172.21.15.83:0/3366603014 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc560066f60 0x7fc560069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:18.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.106+0000 7fc56effd640 1 -- 172.21.15.83:0/3366603014 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc578085bc0 con 0x7fc580162150 2024-04-17T10:48:18.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.110+0000 7fc57e59c640 1 --2- 172.21.15.83:0/3366603014 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc560066f60 0x7fc560069420 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc574005890 tx=0x7fc574009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:18.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.110+0000 7fc56effd640 1 -- 172.21.15.83:0/3366603014 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc578022030 con 0x7fc580162150 2024-04-17T10:48:18.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:18.330+0000 7fc58486b640 1 -- 172.21.15.83:0/3366603014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]} v 0) v1 -- 0x7fc5800b76f0 con 0x7fc580162150 2024-04-17T10:48:19.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.326+0000 7fc56effd640 1 -- 172.21.15.83:0/3366603014 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]}]=0 v449) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc578010040 con 0x7fc580162150 2024-04-17T10:48:19.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.330+0000 7fc58486b640 1 -- 172.21.15.83:0/3366603014 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc560066f60 msgr2=0x7fc560069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:19.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.330+0000 7fc58486b640 1 --2- 172.21.15.83:0/3366603014 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc560066f60 0x7fc560069420 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc574005890 tx=0x7fc574009040 comp rx=0 tx=0).stop 2024-04-17T10:48:19.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.330+0000 7fc58486b640 1 -- 172.21.15.83:0/3366603014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc580162150 msgr2=0x7fc5800b2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:19.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.330+0000 7fc58486b640 1 --2- 172.21.15.83:0/3366603014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc580162150 0x7fc5800b2fa0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fc57800efc0 tx=0x7fc57800c490 comp rx=0 tx=0).stop 2024-04-17T10:48:19.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.330+0000 7fc58486b640 1 -- 172.21.15.83:0/3366603014 shutdown_connections 2024-04-17T10:48:19.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.330+0000 7fc58486b640 1 --2- 172.21.15.83:0/3366603014 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc560066f60 0x7fc560069420 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:19.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.330+0000 7fc58486b640 1 --2- 172.21.15.83:0/3366603014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc580162150 0x7fc5800b2fa0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:19.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.330+0000 7fc58486b640 1 --2- 172.21.15.83:0/3366603014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5800ce750 0x7fc580161070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:19.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.330+0000 7fc58486b640 1 --2- 172.21.15.83:0/3366603014 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5800b0a70 0x7fc58014e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:19.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.330+0000 7fc58486b640 1 -- 172.21.15.83:0/3366603014 >> 172.21.15.83:0/3366603014 conn(0x7fc5800d1770 msgr2=0x7fc58006bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:19.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.334+0000 7fc58486b640 1 -- 172.21.15.83:0/3366603014 shutdown_connections 2024-04-17T10:48:19.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.334+0000 7fc58486b640 1 -- 172.21.15.83:0/3366603014 wait complete. 2024-04-17T10:48:19.351 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-17T10:48:19.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.482+0000 7fa050f43640 1 Processor -- start 2024-04-17T10:48:19.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.482+0000 7fa050f43640 1 -- start start 2024-04-17T10:48:19.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.486+0000 7fa050f43640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa04c15bb70 0x7fa04c06f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:19.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.486+0000 7fa050f43640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa04c0721b0 0x7fa04c06abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:19.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.486+0000 7fa050f43640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa04c06b120 0x7fa04c06d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:19.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.486+0000 7fa050f43640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa04c06b590 con 0x7fa04c06b120 2024-04-17T10:48:19.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.486+0000 7fa050f43640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa04c0b4930 con 0x7fa04c15bb70 2024-04-17T10:48:19.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.486+0000 7fa050f43640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa04c06fa20 con 0x7fa04c0721b0 2024-04-17T10:48:19.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.486+0000 7fa04ad9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa04c06b120 0x7fa04c06d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:19.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.486+0000 7fa04ad9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa04c06b120 0x7fa04c06d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42314/0 (socket says 172.21.15.83:42314) 2024-04-17T10:48:19.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.486+0000 7fa04ad9d640 1 -- 172.21.15.83:0/422666583 learned_addr learned my addr 172.21.15.83:0/422666583 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:19.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.486+0000 7fa04a59c640 1 --2- 172.21.15.83:0/422666583 >> v2:172.21.15.167:3300/0 conn(0x7fa04c15bb70 0x7fa04c06f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:19.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.486+0000 7fa043fff640 1 --2- 172.21.15.83:0/422666583 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa04c0721b0 0x7fa04c06abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:19.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.486+0000 7fa04ad9d640 1 -- 172.21.15.83:0/422666583 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa04c0721b0 msgr2=0x7fa04c06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:19.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.486+0000 7fa04ad9d640 1 --2- 172.21.15.83:0/422666583 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa04c0721b0 0x7fa04c06abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:19.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.486+0000 7fa04ad9d640 1 -- 172.21.15.83:0/422666583 >> v2:172.21.15.167:3300/0 conn(0x7fa04c15bb70 msgr2=0x7fa04c06f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:19.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.486+0000 7fa04ad9d640 1 --2- 172.21.15.83:0/422666583 >> v2:172.21.15.167:3300/0 conn(0x7fa04c15bb70 0x7fa04c06f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:19.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.486+0000 7fa04ad9d640 1 -- 172.21.15.83:0/422666583 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa04c0b4b60 con 0x7fa04c06b120 2024-04-17T10:48:19.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.486+0000 7fa04ad9d640 1 --2- 172.21.15.83:0/422666583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa04c06b120 0x7fa04c06d550 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fa04c0b34b0 tx=0x7fa04401c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d287210000f77c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:19.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.486+0000 7fa049d9b640 1 -- 172.21.15.83:0/422666583 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0440a5580 con 0x7fa04c06b120 2024-04-17T10:48:19.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.486+0000 7fa049d9b640 1 -- 172.21.15.83:0/422666583 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0440a5720 con 0x7fa04c06b120 2024-04-17T10:48:19.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.486+0000 7fa049d9b640 1 -- 172.21.15.83:0/422666583 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0440a5af0 con 0x7fa04c06b120 2024-04-17T10:48:19.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa050f43640 1 -- 172.21.15.83:0/422666583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa04c06b120 msgr2=0x7fa04c06d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:19.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa050f43640 1 --2- 172.21.15.83:0/422666583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa04c06b120 0x7fa04c06d550 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fa04c0b34b0 tx=0x7fa04401c700 comp rx=0 tx=0).stop 2024-04-17T10:48:19.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa050f43640 1 -- 172.21.15.83:0/422666583 shutdown_connections 2024-04-17T10:48:19.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa050f43640 1 --2- 172.21.15.83:0/422666583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa04c06b120 0x7fa04c06d550 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:19.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa050f43640 1 --2- 172.21.15.83:0/422666583 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa04c0721b0 0x7fa04c06abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:19.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa050f43640 1 --2- 172.21.15.83:0/422666583 >> v2:172.21.15.167:3300/0 conn(0x7fa04c15bb70 0x7fa04c06f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:19.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa050f43640 1 -- 172.21.15.83:0/422666583 >> 172.21.15.83:0/422666583 conn(0x7fa04c0d06c0 msgr2=0x7fa04c0d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:19.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa050f43640 1 -- 172.21.15.83:0/422666583 shutdown_connections 2024-04-17T10:48:19.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa050f43640 1 -- 172.21.15.83:0/422666583 wait complete. 2024-04-17T10:48:19.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa050f43640 1 Processor -- start 2024-04-17T10:48:19.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa050f43640 1 -- start start 2024-04-17T10:48:19.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa050f43640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa04c06b120 0x7fa04c0aed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:19.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa050f43640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa04c0721b0 0x7fa04c0a9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:19.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa050f43640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa04c13cfb0 0x7fa04c158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:19.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa050f43640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04c15c0e0 con 0x7fa04c06b120 2024-04-17T10:48:19.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa050f43640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04c15bf60 con 0x7fa04c0721b0 2024-04-17T10:48:19.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa050f43640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa04c06f5f0 con 0x7fa04c13cfb0 2024-04-17T10:48:19.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa04a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa04c06b120 0x7fa04c0aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:19.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa04a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa04c06b120 0x7fa04c0aed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42330/0 (socket says 172.21.15.83:42330) 2024-04-17T10:48:19.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa04a59c640 1 -- 172.21.15.83:0/108049016 learned_addr learned my addr 172.21.15.83:0/108049016 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:19.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa043fff640 1 --2- 172.21.15.83:0/108049016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa04c0721b0 0x7fa04c0a9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:19.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa04ad9d640 1 --2- 172.21.15.83:0/108049016 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa04c13cfb0 0x7fa04c158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:19.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa04a59c640 1 -- 172.21.15.83:0/108049016 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa04c13cfb0 msgr2=0x7fa04c158bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:19.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa04a59c640 1 --2- 172.21.15.83:0/108049016 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa04c13cfb0 0x7fa04c158bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:19.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa04a59c640 1 -- 172.21.15.83:0/108049016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa04c0721b0 msgr2=0x7fa04c0a9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:19.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa04a59c640 1 --2- 172.21.15.83:0/108049016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa04c0721b0 0x7fa04c0a9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:19.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa04a59c640 1 -- 172.21.15.83:0/108049016 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa04c14e3f0 con 0x7fa04c06b120 2024-04-17T10:48:19.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa04ad9d640 1 --2- 172.21.15.83:0/108049016 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa04c13cfb0 0x7fa04c158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:19.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa04a59c640 1 --2- 172.21.15.83:0/108049016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa04c06b120 0x7fa04c0aed10 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00d5f0 tx=0x7fa03c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:19.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa042ffd640 1 -- 172.21.15.83:0/108049016 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c0125f0 con 0x7fa04c06b120 2024-04-17T10:48:19.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa042ffd640 1 -- 172.21.15.83:0/108049016 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa03c019070 con 0x7fa04c06b120 2024-04-17T10:48:19.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa050f43640 1 -- 172.21.15.83:0/108049016 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa04c14e690 con 0x7fa04c06b120 2024-04-17T10:48:19.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa042ffd640 1 -- 172.21.15.83:0/108049016 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c0129b0 con 0x7fa04c06b120 2024-04-17T10:48:19.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.490+0000 7fa050f43640 1 -- 172.21.15.83:0/108049016 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa04c14ec50 con 0x7fa04c06b120 2024-04-17T10:48:19.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.494+0000 7fa042ffd640 1 -- 172.21.15.83:0/108049016 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa03c012b50 con 0x7fa04c06b120 2024-04-17T10:48:19.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.494+0000 7fa050f43640 1 -- 172.21.15.83:0/108049016 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa04c14f080 con 0x7fa04c06b120 2024-04-17T10:48:19.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.494+0000 7fa042ffd640 1 --2- 172.21.15.83:0/108049016 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa02c066f60 0x7fa02c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:19.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.494+0000 7fa042ffd640 1 -- 172.21.15.83:0/108049016 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa03c085090 con 0x7fa04c06b120 2024-04-17T10:48:19.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.494+0000 7fa043fff640 1 --2- 172.21.15.83:0/108049016 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa02c066f60 0x7fa02c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:19.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.494+0000 7fa043fff640 1 --2- 172.21.15.83:0/108049016 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa02c066f60 0x7fa02c069420 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa034006fd0 tx=0x7fa034008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:19.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.498+0000 7fa042ffd640 1 -- 172.21.15.83:0/108049016 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa03c011340 con 0x7fa04c06b120 2024-04-17T10:48:19.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.654+0000 7fa050f43640 1 -- 172.21.15.83:0/108049016 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa04c074f90 con 0x7fa04c06b120 2024-04-17T10:48:19.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.658+0000 7fa042ffd640 1 -- 172.21.15.83:0/108049016 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa03c056430 con 0x7fa04c06b120 2024-04-17T10:48:19.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.662+0000 7fa050f43640 1 -- 172.21.15.83:0/108049016 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa02c066f60 msgr2=0x7fa02c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:19.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.662+0000 7fa050f43640 1 --2- 172.21.15.83:0/108049016 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa02c066f60 0x7fa02c069420 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa034006fd0 tx=0x7fa034008040 comp rx=0 tx=0).stop 2024-04-17T10:48:19.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.662+0000 7fa050f43640 1 -- 172.21.15.83:0/108049016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa04c06b120 msgr2=0x7fa04c0aed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:19.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.662+0000 7fa050f43640 1 --2- 172.21.15.83:0/108049016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa04c06b120 0x7fa04c0aed10 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00d5f0 tx=0x7fa03c00dab0 comp rx=0 tx=0).stop 2024-04-17T10:48:19.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.662+0000 7fa050f43640 1 -- 172.21.15.83:0/108049016 shutdown_connections 2024-04-17T10:48:19.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.662+0000 7fa050f43640 1 --2- 172.21.15.83:0/108049016 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa02c066f60 0x7fa02c069420 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:19.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.662+0000 7fa050f43640 1 --2- 172.21.15.83:0/108049016 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa04c13cfb0 0x7fa04c158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:19.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.662+0000 7fa050f43640 1 --2- 172.21.15.83:0/108049016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa04c0721b0 0x7fa04c0a9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:19.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.662+0000 7fa050f43640 1 --2- 172.21.15.83:0/108049016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa04c06b120 0x7fa04c0aed10 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:19.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.662+0000 7fa050f43640 1 -- 172.21.15.83:0/108049016 >> 172.21.15.83:0/108049016 conn(0x7fa04c0d06c0 msgr2=0x7fa04c06baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:19.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.662+0000 7fa050f43640 1 -- 172.21.15.83:0/108049016 shutdown_connections 2024-04-17T10:48:19.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:19.662+0000 7fa050f43640 1 -- 172.21.15.83:0/108049016 wait complete. 2024-04-17T10:48:19.676 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T10:48:19.676 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:48:19.730 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:48:19.775 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T10:48:19.775 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:48:19.775 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:48:19.781 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16063 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:48:19.781 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:48:19.782 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:48:19.822 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:48:19.822 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:48:19.822 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:48:19.829 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16063 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:48:19.829 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:48:19.830 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:48:19.869 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:48:19.870 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-17T10:48:19.870 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:48:20.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.002+0000 7ff99c95e640 1 Processor -- start 2024-04-17T10:48:20.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.002+0000 7ff99c95e640 1 -- start start 2024-04-17T10:48:20.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.006+0000 7ff99c95e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff9980ce980 0x7ff9980b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:20.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.006+0000 7ff99c95e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff9980cee00 0x7ff9980b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:20.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.006+0000 7ff99c95e640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff998162a00 con 0x7ff99815dfa0 2024-04-17T10:48:20.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.006+0000 7ff99c95e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff998161e80 con 0x7ff9980cee00 2024-04-17T10:48:20.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.006+0000 7ff99c95e640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff998162160 con 0x7ff9980ce980 2024-04-17T10:48:20.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.006+0000 7ff996d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff9980cee00 0x7ff9980b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:20.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.006+0000 7ff996d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff9980cee00 0x7ff9980b5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42336/0 (socket says 172.21.15.83:42336) 2024-04-17T10:48:20.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.006+0000 7ff996d9d640 1 -- 172.21.15.83:0/37277570 learned_addr learned my addr 172.21.15.83:0/37277570 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:20.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.006+0000 7ff995d9b640 1 --2- 172.21.15.83:0/37277570 >> v2:172.21.15.167:3300/0 conn(0x7ff9980ce980 0x7ff9980b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:20.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.006+0000 7ff996d9d640 1 -- 172.21.15.83:0/37277570 >> v2:172.21.15.167:3300/0 conn(0x7ff9980ce980 msgr2=0x7ff9980b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:20.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.006+0000 7ff996d9d640 1 --2- 172.21.15.83:0/37277570 >> v2:172.21.15.167:3300/0 conn(0x7ff9980ce980 0x7ff9980b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:20.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.006+0000 7ff996d9d640 1 -- 172.21.15.83:0/37277570 >> v1:172.21.15.167:6789/0 conn(0x7ff99815dfa0 legacy=0x7ff9980b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:20.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.006+0000 7ff996d9d640 1 -- 172.21.15.83:0/37277570 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff998162440 con 0x7ff9980cee00 2024-04-17T10:48:20.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.006+0000 7ff996d9d640 1 --2- 172.21.15.83:0/37277570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9980cee00 0x7ff9980b5970 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7ff99808c8b0 tx=0x7ff99001c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5211b0e6c6d9863 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:20.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.006+0000 7ff99559a640 1 -- 172.21.15.83:0/37277570 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9900a5750 con 0x7ff9980cee00 2024-04-17T10:48:20.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.006+0000 7ff99559a640 1 -- 172.21.15.83:0/37277570 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9900a58f0 con 0x7ff9980cee00 2024-04-17T10:48:20.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.006+0000 7ff99559a640 1 -- 172.21.15.83:0/37277570 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9900a5cc0 con 0x7ff9980cee00 2024-04-17T10:48:20.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.010+0000 7ff99c95e640 1 -- 172.21.15.83:0/37277570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9980cee00 msgr2=0x7ff9980b5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:20.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.010+0000 7ff99c95e640 1 --2- 172.21.15.83:0/37277570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9980cee00 0x7ff9980b5970 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7ff99808c8b0 tx=0x7ff99001c8f0 comp rx=0 tx=0).stop 2024-04-17T10:48:20.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.010+0000 7ff99c95e640 1 -- 172.21.15.83:0/37277570 shutdown_connections 2024-04-17T10:48:20.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.010+0000 7ff99c95e640 1 --2- 172.21.15.83:0/37277570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9980cee00 0x7ff9980b5970 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:20.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.010+0000 7ff99c95e640 1 --2- 172.21.15.83:0/37277570 >> v2:172.21.15.167:3300/0 conn(0x7ff9980ce980 0x7ff9980b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:20.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.010+0000 7ff99c95e640 1 -- 172.21.15.83:0/37277570 >> 172.21.15.83:0/37277570 conn(0x7ff9980d0600 msgr2=0x7ff9980d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:20.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.010+0000 7ff99c95e640 1 -- 172.21.15.83:0/37277570 shutdown_connections 2024-04-17T10:48:20.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.010+0000 7ff99c95e640 1 -- 172.21.15.83:0/37277570 wait complete. 2024-04-17T10:48:20.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.010+0000 7ff99c95e640 1 Processor -- start 2024-04-17T10:48:20.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.010+0000 7ff99c95e640 1 -- start start 2024-04-17T10:48:20.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.010+0000 7ff99c95e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9980ce980 0x7ff9980c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:20.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.010+0000 7ff99c95e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9980cee00 0x7ff9980c5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:20.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.010+0000 7ff99c95e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff99815dfa0 0x7ff99814c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:20.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.010+0000 7ff99c95e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9980d26c0 con 0x7ff9980cee00 2024-04-17T10:48:20.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.010+0000 7ff99c95e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9980d2540 con 0x7ff99815dfa0 2024-04-17T10:48:20.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.010+0000 7ff99c95e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9980d2840 con 0x7ff9980ce980 2024-04-17T10:48:20.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.010+0000 7ff995d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9980cee00 0x7ff9980c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:20.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.010+0000 7ff995d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9980cee00 0x7ff9980c5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42344/0 (socket says 172.21.15.83:42344) 2024-04-17T10:48:20.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.010+0000 7ff995d9b640 1 -- 172.21.15.83:0/3151708679 learned_addr learned my addr 172.21.15.83:0/3151708679 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:20.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.010+0000 7ff995d9b640 1 -- 172.21.15.83:0/3151708679 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9980ce980 msgr2=0x7ff9980c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:20.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.010+0000 7ff995d9b640 1 --2- 172.21.15.83:0/3151708679 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9980ce980 0x7ff9980c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:20.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.010+0000 7ff995d9b640 1 -- 172.21.15.83:0/3151708679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff99815dfa0 msgr2=0x7ff99814c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:20.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.010+0000 7ff996d9d640 1 --2- 172.21.15.83:0/3151708679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff99815dfa0 0x7ff99814c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:20.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.010+0000 7ff995d9b640 1 --2- 172.21.15.83:0/3151708679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff99815dfa0 0x7ff99814c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:20.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.010+0000 7ff995d9b640 1 -- 172.21.15.83:0/3151708679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9980c8440 con 0x7ff9980cee00 2024-04-17T10:48:20.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.010+0000 7ff995d9b640 1 --2- 172.21.15.83:0/3151708679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9980cee00 0x7ff9980c5220 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7ff98800d620 tx=0x7ff98800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:20.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.010+0000 7ff986ffd640 1 -- 172.21.15.83:0/3151708679 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98800b620 con 0x7ff9980cee00 2024-04-17T10:48:20.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.010+0000 7ff986ffd640 1 -- 172.21.15.83:0/3151708679 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff988019070 con 0x7ff9980cee00 2024-04-17T10:48:20.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.010+0000 7ff99c95e640 1 -- 172.21.15.83:0/3151708679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9980c8720 con 0x7ff9980cee00 2024-04-17T10:48:20.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.010+0000 7ff986ffd640 1 -- 172.21.15.83:0/3151708679 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98800b9e0 con 0x7ff9980cee00 2024-04-17T10:48:20.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.010+0000 7ff99c95e640 1 -- 172.21.15.83:0/3151708679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9980c6ad0 con 0x7ff9980cee00 2024-04-17T10:48:20.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.010+0000 7ff99c95e640 1 -- 172.21.15.83:0/3151708679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff998074f90 con 0x7ff9980cee00 2024-04-17T10:48:20.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.014+0000 7ff986ffd640 1 -- 172.21.15.83:0/3151708679 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff98800bb80 con 0x7ff9980cee00 2024-04-17T10:48:20.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.014+0000 7ff986ffd640 1 --2- 172.21.15.83:0/3151708679 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff978066fe0 0x7ff9780694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:20.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.014+0000 7ff986ffd640 1 -- 172.21.15.83:0/3151708679 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff988085a50 con 0x7ff9980cee00 2024-04-17T10:48:20.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.014+0000 7ff99659c640 1 --2- 172.21.15.83:0/3151708679 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff978066fe0 0x7ff9780694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:20.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.014+0000 7ff99659c640 1 --2- 172.21.15.83:0/3151708679 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff978066fe0 0x7ff9780694a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ff98c009670 tx=0x7ff98c006bd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:20.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.018+0000 7ff986ffd640 1 -- 172.21.15.83:0/3151708679 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff988056ed0 con 0x7ff9980cee00 2024-04-17T10:48:20.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.206+0000 7ff99c95e640 1 -- 172.21.15.83:0/3151708679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff998159930 con 0x7ff9980cee00 2024-04-17T10:48:20.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.210+0000 7ff986ffd640 1 -- 172.21.15.83:0/3151708679 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+3699 (secure 0 0 0) 0x7ff9880567d0 con 0x7ff9980cee00 2024-04-17T10:48:20.205 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:20.205 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":64,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/437268957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":437268957},{"type":"v1","addr":"172.21.15.167:6833","nonce":437268957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":6253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3214640674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3214640674},{"type":"v1","addr":"172.21.15.167:6835","nonce":3214640674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":8046,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/715498323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":715498323},{"type":"v1","addr":"172.21.15.83:6835","nonce":715498323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":8532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2410559109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":2410559109},{"type":"v1","addr":"172.21.15.83:6837","nonce":2410559109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64}],"filesystems":[]} 2024-04-17T10:48:20.206 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 64 2024-04-17T10:48:20.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.210+0000 7ff99c95e640 1 -- 172.21.15.83:0/3151708679 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff978066fe0 msgr2=0x7ff9780694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:20.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.210+0000 7ff99c95e640 1 --2- 172.21.15.83:0/3151708679 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff978066fe0 0x7ff9780694a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ff98c009670 tx=0x7ff98c006bd0 comp rx=0 tx=0).stop 2024-04-17T10:48:20.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.210+0000 7ff99c95e640 1 -- 172.21.15.83:0/3151708679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9980cee00 msgr2=0x7ff9980c5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:20.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.210+0000 7ff99c95e640 1 --2- 172.21.15.83:0/3151708679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9980cee00 0x7ff9980c5220 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7ff98800d620 tx=0x7ff98800dae0 comp rx=0 tx=0).stop 2024-04-17T10:48:20.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.210+0000 7ff99c95e640 1 -- 172.21.15.83:0/3151708679 shutdown_connections 2024-04-17T10:48:20.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.210+0000 7ff99c95e640 1 --2- 172.21.15.83:0/3151708679 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff978066fe0 0x7ff9780694a0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:20.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.210+0000 7ff99c95e640 1 --2- 172.21.15.83:0/3151708679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff99815dfa0 0x7ff99814c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:20.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.210+0000 7ff99c95e640 1 --2- 172.21.15.83:0/3151708679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9980cee00 0x7ff9980c5220 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:20.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.210+0000 7ff99c95e640 1 --2- 172.21.15.83:0/3151708679 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9980ce980 0x7ff9980c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:20.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.210+0000 7ff99c95e640 1 -- 172.21.15.83:0/3151708679 >> 172.21.15.83:0/3151708679 conn(0x7ff9980d0600 msgr2=0x7ff9981582c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:20.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.210+0000 7ff99c95e640 1 -- 172.21.15.83:0/3151708679 shutdown_connections 2024-04-17T10:48:20.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.210+0000 7ff99c95e640 1 -- 172.21.15.83:0/3151708679 wait complete. 2024-04-17T10:48:20.225 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-17T10:48:20.225 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-17T10:48:20.226 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-17T10:48:20.226 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:48:20.226 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-17T10:48:20.226 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-17T10:48:20.226 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-17T10:48:20.226 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:48:20.227 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-17T10:48:20.227 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-17T10:48:20.227 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-17T10:48:20.227 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:48:20.227 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-17T10:48:20.227 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-17T10:48:20.228 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-17T10:48:20.228 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:48:20.237 INFO:tasks.ceph.mds.c:Stopped 2024-04-17T10:48:20.237 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-17T10:48:20.238 INFO:tasks.ceph.mds.a:Stopped 2024-04-17T10:48:20.238 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-17T10:48:20.240 INFO:tasks.ceph.mds.b:Stopped 2024-04-17T10:48:20.240 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-17T10:48:20.242 INFO:tasks.ceph.mds.d:Stopped 2024-04-17T10:48:20.242 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-17T10:48:20.281 INFO:tasks.ceph.mds.c:Started 2024-04-17T10:48:20.296 DEBUG:teuthology.parallel:result is None 2024-04-17T10:48:20.296 INFO:tasks.ceph.mds.b:Started 2024-04-17T10:48:20.297 DEBUG:teuthology.parallel:result is None 2024-04-17T10:48:20.297 INFO:tasks.ceph.mds.a:Started 2024-04-17T10:48:20.298 DEBUG:teuthology.parallel:result is None 2024-04-17T10:48:20.298 INFO:tasks.ceph.mds.d:Started 2024-04-17T10:48:20.298 DEBUG:teuthology.parallel:result is None 2024-04-17T10:48:20.299 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-17T10:48:20.350 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-04-17T10:48:20.358 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-04-17T10:48:20.359 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2024-04-17T10:48:20.373 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2024-04-17T10:48:20.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.422+0000 7fe20b6e4640 1 Processor -- start 2024-04-17T10:48:20.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.422+0000 7fe20b6e4640 1 -- start start 2024-04-17T10:48:20.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.422+0000 7fe20b6e4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe20415e0f0 0x7fe2040b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:20.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.422+0000 7fe20b6e4640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe2040ce400 0x7fe204161c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:20.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.422+0000 7fe20b6e4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe2040b5450 0x7fe2040b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:20.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.422+0000 7fe20b6e4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2040b5df0 con 0x7fe2040b5450 2024-04-17T10:48:20.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.422+0000 7fe20b6e4640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe204162b40 con 0x7fe2040ce400 2024-04-17T10:48:20.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.422+0000 7fe20b6e4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe20415e4d0 con 0x7fe20415e0f0 2024-04-17T10:48:20.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.422+0000 7fe209c81640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe2040b5450 0x7fe2040b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:20.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.422+0000 7fe209c81640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe2040b5450 0x7fe2040b58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42370/0 (socket says 172.21.15.83:42370) 2024-04-17T10:48:20.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.422+0000 7fe209c81640 1 -- 172.21.15.83:0/2256114130 learned_addr learned my addr 172.21.15.83:0/2256114130 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:20.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.422+0000 7fe208c7f640 1 --2- 172.21.15.83:0/2256114130 >> v2:172.21.15.167:3300/0 conn(0x7fe2040ce400 0x7fe204161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:20.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.422+0000 7fe209480640 1 --2- 172.21.15.83:0/2256114130 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe20415e0f0 0x7fe2040b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:20.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.422+0000 7fe209c81640 1 -- 172.21.15.83:0/2256114130 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe20415e0f0 msgr2=0x7fe2040b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:20.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.422+0000 7fe209c81640 1 --2- 172.21.15.83:0/2256114130 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe20415e0f0 0x7fe2040b6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:20.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.422+0000 7fe209c81640 1 -- 172.21.15.83:0/2256114130 >> v2:172.21.15.167:3300/0 conn(0x7fe2040ce400 msgr2=0x7fe204161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:20.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.422+0000 7fe209c81640 1 --2- 172.21.15.83:0/2256114130 >> v2:172.21.15.167:3300/0 conn(0x7fe2040ce400 0x7fe204161c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:20.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.422+0000 7fe209c81640 1 -- 172.21.15.83:0/2256114130 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe204162580 con 0x7fe2040b5450 2024-04-17T10:48:20.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.422+0000 7fe209480640 1 --2- 172.21.15.83:0/2256114130 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe20415e0f0 0x7fe2040b6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:20.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.422+0000 7fe208c7f640 1 --2- 172.21.15.83:0/2256114130 >> v2:172.21.15.167:3300/0 conn(0x7fe2040ce400 0x7fe204161c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:20.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.426+0000 7fe209c81640 1 --2- 172.21.15.83:0/2256114130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2040b5450 0x7fe2040b58b0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fe2040d4a30 tx=0x7fe1fc01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125dd5e153928c2d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:20.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.426+0000 7fe1fbfff640 1 -- 172.21.15.83:0/2256114130 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc0a56d0 con 0x7fe2040b5450 2024-04-17T10:48:20.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.426+0000 7fe1fbfff640 1 -- 172.21.15.83:0/2256114130 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1fc0a5870 con 0x7fe2040b5450 2024-04-17T10:48:20.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.426+0000 7fe1fbfff640 1 -- 172.21.15.83:0/2256114130 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc0a5c40 con 0x7fe2040b5450 2024-04-17T10:48:20.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.426+0000 7fe20b6e4640 1 -- 172.21.15.83:0/2256114130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2040b5450 msgr2=0x7fe2040b58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:20.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.426+0000 7fe20b6e4640 1 --2- 172.21.15.83:0/2256114130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2040b5450 0x7fe2040b58b0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fe2040d4a30 tx=0x7fe1fc01c960 comp rx=0 tx=0).stop 2024-04-17T10:48:20.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.426+0000 7fe20b6e4640 1 -- 172.21.15.83:0/2256114130 shutdown_connections 2024-04-17T10:48:20.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.426+0000 7fe20b6e4640 1 --2- 172.21.15.83:0/2256114130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2040b5450 0x7fe2040b58b0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:20.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.426+0000 7fe20b6e4640 1 --2- 172.21.15.83:0/2256114130 >> v2:172.21.15.167:3300/0 conn(0x7fe2040ce400 0x7fe204161c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:20.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.426+0000 7fe20b6e4640 1 --2- 172.21.15.83:0/2256114130 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe20415e0f0 0x7fe2040b6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:20.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.426+0000 7fe20b6e4640 1 -- 172.21.15.83:0/2256114130 >> 172.21.15.83:0/2256114130 conn(0x7fe2040d0450 msgr2=0x7fe2040d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:20.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.426+0000 7fe20b6e4640 1 -- 172.21.15.83:0/2256114130 shutdown_connections 2024-04-17T10:48:20.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.426+0000 7fe20b6e4640 1 -- 172.21.15.83:0/2256114130 wait complete. 2024-04-17T10:48:20.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.426+0000 7fe20b6e4640 1 Processor -- start 2024-04-17T10:48:20.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.426+0000 7fe20b6e4640 1 -- start start 2024-04-17T10:48:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.426+0000 7fe20b6e4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2040b5450 0x7fe204153a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.426+0000 7fe20b6e4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2040ce400 0x7fe2040c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.426+0000 7fe20b6e4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe20415e0f0 0x7fe20415ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.426+0000 7fe20b6e4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2041622d0 con 0x7fe2040b5450 2024-04-17T10:48:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.426+0000 7fe20b6e4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe204162150 con 0x7fe20415e0f0 2024-04-17T10:48:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.426+0000 7fe20b6e4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2040d2d50 con 0x7fe2040ce400 2024-04-17T10:48:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.426+0000 7fe209480640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2040b5450 0x7fe204153a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.426+0000 7fe209480640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2040b5450 0x7fe204153a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42386/0 (socket says 172.21.15.83:42386) 2024-04-17T10:48:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.426+0000 7fe209480640 1 -- 172.21.15.83:0/3283379175 learned_addr learned my addr 172.21.15.83:0/3283379175 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.426+0000 7fe209480640 1 -- 172.21.15.83:0/3283379175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2040ce400 msgr2=0x7fe2040c8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.426+0000 7fe209480640 1 --2- 172.21.15.83:0/3283379175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2040ce400 0x7fe2040c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.426+0000 7fe209480640 1 -- 172.21.15.83:0/3283379175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe20415e0f0 msgr2=0x7fe20415ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.426+0000 7fe209c81640 1 --2- 172.21.15.83:0/3283379175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe20415e0f0 0x7fe20415ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.426+0000 7fe209480640 1 --2- 172.21.15.83:0/3283379175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe20415e0f0 0x7fe20415ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.430+0000 7fe209480640 1 -- 172.21.15.83:0/3283379175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2040c8f00 con 0x7fe2040b5450 2024-04-17T10:48:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.430+0000 7fe209480640 1 --2- 172.21.15.83:0/3283379175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2040b5450 0x7fe204153a00 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fe20000c820 tx=0x7fe20000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.430+0000 7fe1f9ffb640 1 -- 172.21.15.83:0/3283379175 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe200015070 con 0x7fe2040b5450 2024-04-17T10:48:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.430+0000 7fe1f9ffb640 1 -- 172.21.15.83:0/3283379175 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe200007a30 con 0x7fe2040b5450 2024-04-17T10:48:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.430+0000 7fe20b6e4640 1 -- 172.21.15.83:0/3283379175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2040c91e0 con 0x7fe2040b5450 2024-04-17T10:48:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.430+0000 7fe1f9ffb640 1 -- 172.21.15.83:0/3283379175 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe200007e40 con 0x7fe2040b5450 2024-04-17T10:48:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.430+0000 7fe20b6e4640 1 -- 172.21.15.83:0/3283379175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2040c97a0 con 0x7fe2040b5450 2024-04-17T10:48:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.430+0000 7fe20b6e4640 1 -- 172.21.15.83:0/3283379175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2040681a0 con 0x7fe2040b5450 2024-04-17T10:48:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.430+0000 7fe1f9ffb640 1 -- 172.21.15.83:0/3283379175 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe200018070 con 0x7fe2040b5450 2024-04-17T10:48:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.430+0000 7fe1f9ffb640 1 --2- 172.21.15.83:0/3283379175 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe1e8066f60 0x7fe1e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.430+0000 7fe1f9ffb640 1 -- 172.21.15.83:0/3283379175 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe2000850b0 con 0x7fe2040b5450 2024-04-17T10:48:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.430+0000 7fe208c7f640 1 --2- 172.21.15.83:0/3283379175 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe1e8066f60 0x7fe1e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.430+0000 7fe208c7f640 1 --2- 172.21.15.83:0/3283379175 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe1e8066f60 0x7fe1e8069420 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe204147980 tx=0x7fe1f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:20.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.434+0000 7fe1f9ffb640 1 -- 172.21.15.83:0/3283379175 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe2000564a0 con 0x7fe2040b5450 2024-04-17T10:48:20.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:20.598+0000 7fe20b6e4640 1 -- 172.21.15.83:0/3283379175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe204074f90 con 0x7fe2040b5450 2024-04-17T10:48:21.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.346+0000 7fe1f9ffb640 1 -- 172.21.15.83:0/3283379175 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v108) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe200055da0 con 0x7fe2040b5450 2024-04-17T10:48:21.343 INFO:teuthology.orchestra.run.smithi083.stderr: removed all blocklist entries 2024-04-17T10:48:21.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.350+0000 7fe20b6e4640 1 -- 172.21.15.83:0/3283379175 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe1e8066f60 msgr2=0x7fe1e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:21.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.350+0000 7fe20b6e4640 1 --2- 172.21.15.83:0/3283379175 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe1e8066f60 0x7fe1e8069420 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe204147980 tx=0x7fe1f4008040 comp rx=0 tx=0).stop 2024-04-17T10:48:21.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.350+0000 7fe20b6e4640 1 -- 172.21.15.83:0/3283379175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2040b5450 msgr2=0x7fe204153a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:21.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.350+0000 7fe20b6e4640 1 --2- 172.21.15.83:0/3283379175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2040b5450 0x7fe204153a00 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fe20000c820 tx=0x7fe20000cce0 comp rx=0 tx=0).stop 2024-04-17T10:48:21.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.350+0000 7fe20b6e4640 1 -- 172.21.15.83:0/3283379175 shutdown_connections 2024-04-17T10:48:21.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.350+0000 7fe20b6e4640 1 --2- 172.21.15.83:0/3283379175 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe1e8066f60 0x7fe1e8069420 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:21.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.350+0000 7fe20b6e4640 1 --2- 172.21.15.83:0/3283379175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe20415e0f0 0x7fe20415ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:21.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.350+0000 7fe20b6e4640 1 --2- 172.21.15.83:0/3283379175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2040ce400 0x7fe2040c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:21.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.350+0000 7fe20b6e4640 1 --2- 172.21.15.83:0/3283379175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2040b5450 0x7fe204153a00 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:21.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.350+0000 7fe20b6e4640 1 -- 172.21.15.83:0/3283379175 >> 172.21.15.83:0/3283379175 conn(0x7fe2040d0450 msgr2=0x7fe2040cf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:21.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.350+0000 7fe20b6e4640 1 -- 172.21.15.83:0/3283379175 shutdown_connections 2024-04-17T10:48:21.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.350+0000 7fe20b6e4640 1 -- 172.21.15.83:0/3283379175 wait complete. 2024-04-17T10:48:21.368 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-17T10:48:21.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.486+0000 7fb8ffcc4640 1 Processor -- start 2024-04-17T10:48:21.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.486+0000 7fb8ffcc4640 1 -- start start 2024-04-17T10:48:21.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.486+0000 7fb8ffcc4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb8f80aedd0 0x7fb8f806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:21.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.486+0000 7fb8ffcc4640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb8f806dae0 0x7fb8f80b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:21.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.486+0000 7fb8ffcc4640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8f815e440 con 0x7fb8f815db50 2024-04-17T10:48:21.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.486+0000 7fb8ffcc4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8f815e720 con 0x7fb8f80aedd0 2024-04-17T10:48:21.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.486+0000 7fb8ffcc4640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8f815ea00 con 0x7fb8f806dae0 2024-04-17T10:48:21.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.486+0000 7fb8fd25f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb8f80aedd0 0x7fb8f806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:21.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.486+0000 7fb8fd25f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb8f80aedd0 0x7fb8f806d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42390/0 (socket says 172.21.15.83:42390) 2024-04-17T10:48:21.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.486+0000 7fb8fd25f640 1 -- 172.21.15.83:0/433595379 learned_addr learned my addr 172.21.15.83:0/433595379 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:21.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.486+0000 7fb8fe261640 1 --2- 172.21.15.83:0/433595379 >> v2:172.21.15.167:3300/0 conn(0x7fb8f806dae0 0x7fb8f80b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:21.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.486+0000 7fb8fd25f640 1 -- 172.21.15.83:0/433595379 >> v2:172.21.15.167:3300/0 conn(0x7fb8f806dae0 msgr2=0x7fb8f80b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:21.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.486+0000 7fb8fd25f640 1 --2- 172.21.15.83:0/433595379 >> v2:172.21.15.167:3300/0 conn(0x7fb8f806dae0 0x7fb8f80b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:21.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.486+0000 7fb8fd25f640 1 -- 172.21.15.83:0/433595379 >> v1:172.21.15.167:6789/0 conn(0x7fb8f815db50 legacy=0x7fb8f8070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:21.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.486+0000 7fb8fd25f640 1 -- 172.21.15.83:0/433595379 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f815ece0 con 0x7fb8f80aedd0 2024-04-17T10:48:21.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.486+0000 7fb8fe261640 1 --2- 172.21.15.83:0/433595379 >> v2:172.21.15.167:3300/0 conn(0x7fb8f806dae0 0x7fb8f80b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:21.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.490+0000 7fb8fd25f640 1 --2- 172.21.15.83:0/433595379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8f80aedd0 0x7fb8f806d5a0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8009870 tx=0x7fb8e801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac3d56792d262c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:21.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.490+0000 7fb8fca5e640 1 -- 172.21.15.83:0/433595379 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e80b0070 con 0x7fb8f80aedd0 2024-04-17T10:48:21.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.490+0000 7fb8fca5e640 1 -- 172.21.15.83:0/433595379 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e80ab600 con 0x7fb8f80aedd0 2024-04-17T10:48:21.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.490+0000 7fb8fca5e640 1 -- 172.21.15.83:0/433595379 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e80ab9d0 con 0x7fb8f80aedd0 2024-04-17T10:48:21.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.490+0000 7fb8ffcc4640 1 -- 172.21.15.83:0/433595379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8f80aedd0 msgr2=0x7fb8f806d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:21.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.490+0000 7fb8ffcc4640 1 --2- 172.21.15.83:0/433595379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8f80aedd0 0x7fb8f806d5a0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8009870 tx=0x7fb8e801c520 comp rx=0 tx=0).stop 2024-04-17T10:48:21.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.490+0000 7fb8ffcc4640 1 -- 172.21.15.83:0/433595379 shutdown_connections 2024-04-17T10:48:21.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.490+0000 7fb8ffcc4640 1 --2- 172.21.15.83:0/433595379 >> v2:172.21.15.167:3300/0 conn(0x7fb8f806dae0 0x7fb8f80b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:21.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.490+0000 7fb8ffcc4640 1 --2- 172.21.15.83:0/433595379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8f80aedd0 0x7fb8f806d5a0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:21.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.490+0000 7fb8ffcc4640 1 -- 172.21.15.83:0/433595379 >> 172.21.15.83:0/433595379 conn(0x7fb8f806abe0 msgr2=0x7fb8f806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:21.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.490+0000 7fb8ffcc4640 1 -- 172.21.15.83:0/433595379 shutdown_connections 2024-04-17T10:48:21.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.490+0000 7fb8ffcc4640 1 -- 172.21.15.83:0/433595379 wait complete. 2024-04-17T10:48:21.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.494+0000 7fb8ffcc4640 1 Processor -- start 2024-04-17T10:48:21.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.494+0000 7fb8ffcc4640 1 -- start start 2024-04-17T10:48:21.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.494+0000 7fb8ffcc4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8f806dae0 0x7fb8f8143ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:21.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.494+0000 7fb8ffcc4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8f80aedd0 0x7fb8f81549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:21.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.494+0000 7fb8ffcc4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8f815db50 0x7fb8f80cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:21.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.494+0000 7fb8ffcc4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f80b3ef0 con 0x7fb8f815db50 2024-04-17T10:48:21.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.494+0000 7fb8ffcc4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f80b3d70 con 0x7fb8f806dae0 2024-04-17T10:48:21.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.494+0000 7fb8ffcc4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f80b4070 con 0x7fb8f80aedd0 2024-04-17T10:48:21.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.494+0000 7fb8fe261640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8f815db50 0x7fb8f80cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:21.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.494+0000 7fb8fe261640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8f815db50 0x7fb8f80cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42402/0 (socket says 172.21.15.83:42402) 2024-04-17T10:48:21.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.494+0000 7fb8fe261640 1 -- 172.21.15.83:0/1160418346 learned_addr learned my addr 172.21.15.83:0/1160418346 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:21.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.494+0000 7fb8fe261640 1 -- 172.21.15.83:0/1160418346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8f80aedd0 msgr2=0x7fb8f81549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:21.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.494+0000 7fb8fd25f640 1 --2- 172.21.15.83:0/1160418346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8f80aedd0 0x7fb8f81549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:21.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.494+0000 7fb8fda60640 1 --2- 172.21.15.83:0/1160418346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8f806dae0 0x7fb8f8143ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:21.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.494+0000 7fb8fe261640 1 --2- 172.21.15.83:0/1160418346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8f80aedd0 0x7fb8f81549f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:21.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.494+0000 7fb8fe261640 1 -- 172.21.15.83:0/1160418346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8f806dae0 msgr2=0x7fb8f8143ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:21.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.494+0000 7fb8fe261640 1 --2- 172.21.15.83:0/1160418346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8f806dae0 0x7fb8f8143ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:21.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.494+0000 7fb8fe261640 1 -- 172.21.15.83:0/1160418346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f8153f00 con 0x7fb8f815db50 2024-04-17T10:48:21.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.494+0000 7fb8fe261640 1 --2- 172.21.15.83:0/1160418346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8f815db50 0x7fb8f80cc7e0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fb8f000cf20 tx=0x7fb8f000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:21.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.494+0000 7fb8ee7fc640 1 -- 172.21.15.83:0/1160418346 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f001a070 con 0x7fb8f815db50 2024-04-17T10:48:21.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.494+0000 7fb8ee7fc640 1 -- 172.21.15.83:0/1160418346 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f000d840 con 0x7fb8f815db50 2024-04-17T10:48:21.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.494+0000 7fb8ffcc4640 1 -- 172.21.15.83:0/1160418346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f81541e0 con 0x7fb8f815db50 2024-04-17T10:48:21.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.494+0000 7fb8ee7fc640 1 -- 172.21.15.83:0/1160418346 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f000dc30 con 0x7fb8f815db50 2024-04-17T10:48:21.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.494+0000 7fb8ffcc4640 1 -- 172.21.15.83:0/1160418346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f8154f30 con 0x7fb8f815db50 2024-04-17T10:48:21.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.494+0000 7fb8ffcc4640 1 -- 172.21.15.83:0/1160418346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f8074f90 con 0x7fb8f815db50 2024-04-17T10:48:21.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.494+0000 7fb8ee7fc640 1 -- 172.21.15.83:0/1160418346 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb8f001d070 con 0x7fb8f815db50 2024-04-17T10:48:21.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.494+0000 7fb8ee7fc640 1 --2- 172.21.15.83:0/1160418346 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb8dc066f90 0x7fb8dc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:21.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.498+0000 7fb8ee7fc640 1 -- 172.21.15.83:0/1160418346 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb8f0085590 con 0x7fb8f815db50 2024-04-17T10:48:21.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.498+0000 7fb8fda60640 1 --2- 172.21.15.83:0/1160418346 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb8dc066f90 0x7fb8dc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:21.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.498+0000 7fb8fda60640 1 --2- 172.21.15.83:0/1160418346 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb8dc066f90 0x7fb8dc069450 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4007fd0 tx=0x7fb8f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:21.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.498+0000 7fb8ee7fc640 1 -- 172.21.15.83:0/1160418346 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb8f00568f0 con 0x7fb8f815db50 2024-04-17T10:48:21.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.698+0000 7fb8ffcc4640 1 -- 172.21.15.83:0/1160418346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb8f80b2af0 con 0x7fb8f815db50 2024-04-17T10:48:21.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.702+0000 7fb8ee7fc640 1 -- 172.21.15.83:0/1160418346 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fb8f00561f0 con 0x7fb8f815db50 2024-04-17T10:48:21.697 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:21.697 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:48:21.697 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_dump": [ 2024-04-17T10:48:21.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:48:21.697 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.a", 2024-04-17T10:48:21.697 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mfpxsNxAANpL7rcWevXwd2KMftSQnEQ==", 2024-04-17T10:48:21.697 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:48:21.698 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:48:21.698 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:48:21.698 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:48:21.698 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:48:21.698 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:48:21.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:48:21.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:48:21.698 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.b", 2024-04-17T10:48:21.698 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mXlI9ChAAYdcrGVmtYdtWWgCLVmpTTg==", 2024-04-17T10:48:21.699 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:48:21.699 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:48:21.699 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:48:21.699 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:48:21.699 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:48:21.699 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:48:21.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:48:21.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:48:21.699 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.c", 2024-04-17T10:48:21.699 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mupo3BBAAyKH801vpFLXYs9mfzSlT0w==", 2024-04-17T10:48:21.699 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:48:21.699 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:48:21.699 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:48:21.700 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:48:21.700 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:48:21.700 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:48:21.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:48:21.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:48:21.700 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.d", 2024-04-17T10:48:21.700 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mbm++EhAAIBMOtQoD+kPeMqR6gK5ctw==", 2024-04-17T10:48:21.700 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:48:21.700 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:48:21.700 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:48:21.700 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:48:21.700 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:48:21.700 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:48:21.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:48:21.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:48:21.701 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.0", 2024-04-17T10:48:21.701 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBXph9m1HWhIRAAOq+9nGq7OGonUCE2aeiAzQ==", 2024-04-17T10:48:21.701 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:48:21.701 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:48:21.701 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:48:21.701 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:48:21.701 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:48:21.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:48:21.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:48:21.701 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.1", 2024-04-17T10:48:21.701 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBaph9mlMmtJRAAXHLMjXfm/+WjIFURLkbAfA==", 2024-04-17T10:48:21.701 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:48:21.701 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:48:21.702 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:48:21.702 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:48:21.702 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:48:21.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:48:21.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:48:21.702 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.2", 2024-04-17T10:48:21.702 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBdph9mBsNPLRAAtR5hwGdDkjLDLS19QUDDYQ==", 2024-04-17T10:48:21.702 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:48:21.702 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:48:21.702 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:48:21.702 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:48:21.702 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:48:21.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:48:21.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:48:21.703 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.3", 2024-04-17T10:48:21.703 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBgph9mZ4EwNRAAIyL7e0EK1523dPXmbOOkzw==", 2024-04-17T10:48:21.703 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:48:21.703 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:48:21.703 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:48:21.703 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:48:21.703 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:48:21.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:48:21.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:48:21.703 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.4", 2024-04-17T10:48:21.703 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBmph9mphAsMhAAtF4uV28Nb66PNjuugB12Hw==", 2024-04-17T10:48:21.703 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:48:21.704 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:48:21.704 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:48:21.704 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:48:21.704 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:48:21.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:48:21.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:48:21.704 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.5", 2024-04-17T10:48:21.704 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBpph9mHJQNNxAA72eOFYFbhakQac1JKRNjzw==", 2024-04-17T10:48:21.704 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:48:21.704 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:48:21.704 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:48:21.704 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:48:21.704 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:48:21.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:48:21.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:48:21.705 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.6", 2024-04-17T10:48:21.705 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBtph9mb6RKAxAAoVX6Cp7l7j3Lm1ES2Sq+fQ==", 2024-04-17T10:48:21.705 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:48:21.705 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:48:21.705 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:48:21.705 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:48:21.705 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:48:21.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:48:21.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:48:21.705 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.7", 2024-04-17T10:48:21.706 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBwph9mdrdrBxAApDwjml2ahsLbRol4uL/BxA==", 2024-04-17T10:48:21.706 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:48:21.706 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:48:21.706 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:48:21.706 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:48:21.706 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:48:21.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:48:21.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:48:21.706 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.0", 2024-04-17T10:48:21.706 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mHGmXGBAAZF+JiAG9YrFtbW+dCy2IEQ==", 2024-04-17T10:48:21.706 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:48:21.707 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:48:21.707 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow r", 2024-04-17T10:48:21.707 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-17T10:48:21.707 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:48:21.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:48:21.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:48:21.707 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.1", 2024-04-17T10:48:21.707 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mB7wgIBAABdavAo/uzVuSjfkkcYtqJg==", 2024-04-17T10:48:21.707 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:48:21.707 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:48:21.707 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-17T10:48:21.707 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-17T10:48:21.707 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-17T10:48:21.708 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:48:21.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:48:21.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:48:21.708 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.2", 2024-04-17T10:48:21.708 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9m+wKgHRAAe8IUFx34ItLpsaDqSYyEgA==", 2024-04-17T10:48:21.708 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:48:21.708 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:48:21.708 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-17T10:48:21.708 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-17T10:48:21.708 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-17T10:48:21.708 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:48:21.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:48:21.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:48:21.708 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.3", 2024-04-17T10:48:21.709 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mfJ0yJRAAHH7rRh/BUIujTT04AF2l0w==", 2024-04-17T10:48:21.709 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:48:21.709 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:48:21.709 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-17T10:48:21.709 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-17T10:48:21.709 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-17T10:48:21.709 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:48:21.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:48:21.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:48:21.709 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.admin", 2024-04-17T10:48:21.709 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mjJ3GIBAA/2qaCbRiLJ5X1cv3ZGdfoA==", 2024-04-17T10:48:21.709 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:48:21.709 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-17T10:48:21.709 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:48:21.710 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:48:21.710 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:48:21.710 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:48:21.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:48:21.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:48:21.710 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.x", 2024-04-17T10:48:21.710 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mMFptNBAA/G0zEfMZTCy9Iox+MwHG3Q==", 2024-04-17T10:48:21.710 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:48:21.710 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-17T10:48:21.710 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-04-17T10:48:21.710 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:48:21.710 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:48:21.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:48:21.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:48:21.711 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.y", 2024-04-17T10:48:21.711 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mjFbRMRAAjY2d8POBzp8hEp9GSO0ylQ==", 2024-04-17T10:48:21.711 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:48:21.711 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-17T10:48:21.711 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-04-17T10:48:21.711 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:48:21.711 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:48:21.711 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:48:21.711 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-17T10:48:21.711 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:48:21.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.702+0000 7fb8ffcc4640 1 -- 172.21.15.83:0/1160418346 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb8dc066f90 msgr2=0x7fb8dc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:21.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.702+0000 7fb8ffcc4640 1 --2- 172.21.15.83:0/1160418346 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb8dc066f90 0x7fb8dc069450 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4007fd0 tx=0x7fb8f4009040 comp rx=0 tx=0).stop 2024-04-17T10:48:21.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.702+0000 7fb8ffcc4640 1 -- 172.21.15.83:0/1160418346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8f815db50 msgr2=0x7fb8f80cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:21.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.702+0000 7fb8ffcc4640 1 --2- 172.21.15.83:0/1160418346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8f815db50 0x7fb8f80cc7e0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fb8f000cf20 tx=0x7fb8f000a7d0 comp rx=0 tx=0).stop 2024-04-17T10:48:21.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.702+0000 7fb8ffcc4640 1 -- 172.21.15.83:0/1160418346 shutdown_connections 2024-04-17T10:48:21.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.702+0000 7fb8ffcc4640 1 --2- 172.21.15.83:0/1160418346 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb8dc066f90 0x7fb8dc069450 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:21.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.702+0000 7fb8ffcc4640 1 --2- 172.21.15.83:0/1160418346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8f815db50 0x7fb8f80cc7e0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:21.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.702+0000 7fb8ffcc4640 1 --2- 172.21.15.83:0/1160418346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8f80aedd0 0x7fb8f81549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:21.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.702+0000 7fb8ffcc4640 1 --2- 172.21.15.83:0/1160418346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8f806dae0 0x7fb8f8143ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:21.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.702+0000 7fb8ffcc4640 1 -- 172.21.15.83:0/1160418346 >> 172.21.15.83:0/1160418346 conn(0x7fb8f806abe0 msgr2=0x7fb8f806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:21.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.702+0000 7fb8ffcc4640 1 -- 172.21.15.83:0/1160418346 shutdown_connections 2024-04-17T10:48:21.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.702+0000 7fb8ffcc4640 1 -- 172.21.15.83:0/1160418346 wait complete. 2024-04-17T10:48:21.716 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:48:21.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.838+0000 7f9bf5add640 1 Processor -- start 2024-04-17T10:48:21.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.838+0000 7f9bf5add640 1 -- start start 2024-04-17T10:48:21.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.838+0000 7f9bf5add640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9bf00b2d80 0x7f9bf0146e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:21.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.838+0000 7f9bf5add640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bf006d690 0x7f9bf01620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:21.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.838+0000 7f9bf5add640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bf0147570 con 0x7f9bf0146a00 2024-04-17T10:48:21.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.838+0000 7f9bf5add640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf0162610 con 0x7f9bf00b2d80 2024-04-17T10:48:21.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.838+0000 7f9bf5add640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf006c960 con 0x7f9bf006d690 2024-04-17T10:48:21.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.838+0000 7f9bef7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9bf0146a00 0x7f9bf00b2670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51232/0 (socket says 172.21.15.83:51232) 2024-04-17T10:48:21.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.838+0000 7f9beeffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9bf00b2d80 0x7f9bf0146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:21.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.838+0000 7f9bef7fe640 1 -- 172.21.15.83:0/1780448649 learned_addr learned my addr 172.21.15.83:0/1780448649 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:21.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.838+0000 7f9beffff640 1 --2- 172.21.15.83:0/1780448649 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bf006d690 0x7f9bf01620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:21.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.838+0000 7f9bee7fc640 1 -- 172.21.15.83:0/1780448649 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2561302817 0 0) 0x7f9bf0147570 con 0x7f9bf0146a00 2024-04-17T10:48:21.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.838+0000 7f9bee7fc640 1 -- 172.21.15.83:0/1780448649 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bd8003530 con 0x7f9bf0146a00 2024-04-17T10:48:21.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.838+0000 7f9beeffd640 1 -- 172.21.15.83:0/1780448649 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bf006d690 msgr2=0x7f9bf01620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:21.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.838+0000 7f9beeffd640 1 --2- 172.21.15.83:0/1780448649 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bf006d690 0x7f9bf01620d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:21.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.838+0000 7f9beeffd640 1 -- 172.21.15.83:0/1780448649 >> v1:172.21.15.83:6789/0 conn(0x7f9bf0146a00 legacy=0x7f9bf00b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:21.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.838+0000 7f9beeffd640 1 -- 172.21.15.83:0/1780448649 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf006cbb0 con 0x7f9bf00b2d80 2024-04-17T10:48:21.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.842+0000 7f9beeffd640 1 --2- 172.21.15.83:0/1780448649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bf00b2d80 0x7f9bf0146e70 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f9be00097c0 tx=0x7f9be001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67a8e67f64f2a6e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:21.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.842+0000 7f9bee7fc640 1 -- 172.21.15.83:0/1780448649 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be00b1070 con 0x7f9bf00b2d80 2024-04-17T10:48:21.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.842+0000 7f9bee7fc640 1 -- 172.21.15.83:0/1780448649 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be00ac410 con 0x7f9bf00b2d80 2024-04-17T10:48:21.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.842+0000 7f9bee7fc640 1 -- 172.21.15.83:0/1780448649 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be00ac7e0 con 0x7f9bf00b2d80 2024-04-17T10:48:21.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.842+0000 7f9bf5add640 1 -- 172.21.15.83:0/1780448649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bf00b2d80 msgr2=0x7f9bf0146e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:21.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.842+0000 7f9bf5add640 1 --2- 172.21.15.83:0/1780448649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bf00b2d80 0x7f9bf0146e70 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f9be00097c0 tx=0x7f9be001c4a0 comp rx=0 tx=0).stop 2024-04-17T10:48:21.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.842+0000 7f9bf5add640 1 -- 172.21.15.83:0/1780448649 shutdown_connections 2024-04-17T10:48:21.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.842+0000 7f9bf5add640 1 --2- 172.21.15.83:0/1780448649 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bf006d690 0x7f9bf01620d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:21.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.842+0000 7f9bf5add640 1 --2- 172.21.15.83:0/1780448649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bf00b2d80 0x7f9bf0146e70 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:21.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.842+0000 7f9bf5add640 1 -- 172.21.15.83:0/1780448649 >> 172.21.15.83:0/1780448649 conn(0x7f9bf006abe0 msgr2=0x7f9bf006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:21.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.842+0000 7f9bf5add640 1 -- 172.21.15.83:0/1780448649 shutdown_connections 2024-04-17T10:48:21.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.842+0000 7f9bf5add640 1 -- 172.21.15.83:0/1780448649 wait complete. 2024-04-17T10:48:21.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.842+0000 7f9bf5add640 1 Processor -- start 2024-04-17T10:48:21.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.842+0000 7f9bf5add640 1 -- start start 2024-04-17T10:48:21.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.842+0000 7f9bf5add640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bf006d690 0x7f9bf0139180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:21.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.842+0000 7f9bf5add640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bf00b2d80 0x7f9bf013c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:21.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.842+0000 7f9bf5add640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bf0146a00 0x7f9bf013c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:21.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.842+0000 7f9bf5add640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf0071010 con 0x7f9bf0146a00 2024-04-17T10:48:21.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.842+0000 7f9bf5add640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf0070e90 con 0x7f9bf006d690 2024-04-17T10:48:21.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.842+0000 7f9bf5add640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf013ace0 con 0x7f9bf00b2d80 2024-04-17T10:48:21.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.842+0000 7f9beffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bf0146a00 0x7f9bf013c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:21.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.842+0000 7f9beffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bf0146a00 0x7f9bf013c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42420/0 (socket says 172.21.15.83:42420) 2024-04-17T10:48:21.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.842+0000 7f9beffff640 1 -- 172.21.15.83:0/3796102734 learned_addr learned my addr 172.21.15.83:0/3796102734 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:21.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.842+0000 7f9beeffd640 1 --2- 172.21.15.83:0/3796102734 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bf00b2d80 0x7f9bf013c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:21.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.842+0000 7f9bef7fe640 1 --2- 172.21.15.83:0/3796102734 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bf006d690 0x7f9bf0139180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:21.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.842+0000 7f9beffff640 1 -- 172.21.15.83:0/3796102734 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bf00b2d80 msgr2=0x7f9bf013c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:21.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.842+0000 7f9beffff640 1 --2- 172.21.15.83:0/3796102734 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bf00b2d80 0x7f9bf013c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:21.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.842+0000 7f9beffff640 1 -- 172.21.15.83:0/3796102734 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bf006d690 msgr2=0x7f9bf0139180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:21.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.842+0000 7f9beffff640 1 --2- 172.21.15.83:0/3796102734 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bf006d690 0x7f9bf0139180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:21.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.842+0000 7f9beffff640 1 -- 172.21.15.83:0/3796102734 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf013a720 con 0x7f9bf0146a00 2024-04-17T10:48:21.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.842+0000 7f9beffff640 1 --2- 172.21.15.83:0/3796102734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bf0146a00 0x7f9bf013c840 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f9be8007e40 tx=0x7f9be800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:21.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.846+0000 7f9bd7fff640 1 -- 172.21.15.83:0/3796102734 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be8014620 con 0x7f9bf0146a00 2024-04-17T10:48:21.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.846+0000 7f9bd7fff640 1 -- 172.21.15.83:0/3796102734 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be8019070 con 0x7f9bf0146a00 2024-04-17T10:48:21.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.846+0000 7f9bf5add640 1 -- 172.21.15.83:0/3796102734 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf013aa00 con 0x7f9bf0146a00 2024-04-17T10:48:21.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.846+0000 7f9bd7fff640 1 -- 172.21.15.83:0/3796102734 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be8014a20 con 0x7f9bf0146a00 2024-04-17T10:48:21.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.846+0000 7f9bf5add640 1 -- 172.21.15.83:0/3796102734 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf01396c0 con 0x7f9bf0146a00 2024-04-17T10:48:21.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.846+0000 7f9bf5add640 1 -- 172.21.15.83:0/3796102734 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf0074f90 con 0x7f9bf0146a00 2024-04-17T10:48:21.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.846+0000 7f9bd7fff640 1 -- 172.21.15.83:0/3796102734 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9be8014bc0 con 0x7f9bf0146a00 2024-04-17T10:48:21.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.846+0000 7f9bd7fff640 1 --2- 172.21.15.83:0/3796102734 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9bdc066f60 0x7f9bdc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:21.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.846+0000 7f9bd7fff640 1 -- 172.21.15.83:0/3796102734 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9be8085530 con 0x7f9bf0146a00 2024-04-17T10:48:21.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.846+0000 7f9bef7fe640 1 --2- 172.21.15.83:0/3796102734 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9bdc066f60 0x7f9bdc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:21.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.846+0000 7f9bef7fe640 1 --2- 172.21.15.83:0/3796102734 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9bdc066f60 0x7f9bdc069420 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9be4007fd0 tx=0x7f9be4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:21.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:21.850+0000 7f9bd7fff640 1 -- 172.21.15.83:0/3796102734 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9be8056920 con 0x7f9bf0146a00 2024-04-17T10:48:22.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.010+0000 7f9bf5add640 1 -- 172.21.15.83:0/3796102734 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9bf006dfe0 con 0x7f9bf0146a00 2024-04-17T10:48:22.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.010+0000 7f9bd7fff640 1 -- 172.21.15.83:0/3796102734 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+12829 (secure 0 0 0) 0x7f9be8056220 con 0x7f9bf0146a00 2024-04-17T10:48:22.006 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:22.007 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":110,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:48:21.363900+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:48:22.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.010+0000 7f9bf5add640 1 -- 172.21.15.83:0/3796102734 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9bdc066f60 msgr2=0x7f9bdc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:22.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.010+0000 7f9bf5add640 1 --2- 172.21.15.83:0/3796102734 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9bdc066f60 0x7f9bdc069420 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9be4007fd0 tx=0x7f9be4009040 comp rx=0 tx=0).stop 2024-04-17T10:48:22.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.010+0000 7f9bf5add640 1 -- 172.21.15.83:0/3796102734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bf0146a00 msgr2=0x7f9bf013c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:22.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.010+0000 7f9bf5add640 1 --2- 172.21.15.83:0/3796102734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bf0146a00 0x7f9bf013c840 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f9be8007e40 tx=0x7f9be800e4b0 comp rx=0 tx=0).stop 2024-04-17T10:48:22.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.010+0000 7f9bf5add640 1 -- 172.21.15.83:0/3796102734 shutdown_connections 2024-04-17T10:48:22.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.010+0000 7f9bf5add640 1 --2- 172.21.15.83:0/3796102734 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9bdc066f60 0x7f9bdc069420 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:22.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.010+0000 7f9bf5add640 1 --2- 172.21.15.83:0/3796102734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bf0146a00 0x7f9bf013c840 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:22.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.010+0000 7f9bf5add640 1 --2- 172.21.15.83:0/3796102734 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bf00b2d80 0x7f9bf013c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:22.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.010+0000 7f9bf5add640 1 --2- 172.21.15.83:0/3796102734 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bf006d690 0x7f9bf0139180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:22.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.014+0000 7f9bf5add640 1 -- 172.21.15.83:0/3796102734 >> 172.21.15.83:0/3796102734 conn(0x7f9bf006abe0 msgr2=0x7f9bf0071e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:22.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.014+0000 7f9bf5add640 1 -- 172.21.15.83:0/3796102734 shutdown_connections 2024-04-17T10:48:22.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.014+0000 7f9bf5add640 1 -- 172.21.15.83:0/3796102734 wait complete. 2024-04-17T10:48:22.026 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:48:22.026 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:48:22.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.154+0000 7ff718dae640 1 Processor -- start 2024-04-17T10:48:22.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.154+0000 7ff718dae640 1 -- start start 2024-04-17T10:48:22.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.154+0000 7ff718dae640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff71406b520 0x7ff7140b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:22.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.154+0000 7ff718dae640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff7140b4040 0x7ff71406cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:22.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.154+0000 7ff718dae640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff71415bd70 con 0x7ff714154de0 2024-04-17T10:48:22.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.154+0000 7ff718dae640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7140b4870 con 0x7ff7140b4040 2024-04-17T10:48:22.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.154+0000 7ff718dae640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff71415bf60 con 0x7ff71406b520 2024-04-17T10:48:22.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.154+0000 7ff712d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff7140b4040 0x7ff71406cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:22.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.154+0000 7ff712d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff7140b4040 0x7ff71406cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42428/0 (socket says 172.21.15.83:42428) 2024-04-17T10:48:22.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.154+0000 7ff712d9d640 1 -- 172.21.15.83:0/2313796352 learned_addr learned my addr 172.21.15.83:0/2313796352 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:22.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.154+0000 7ff711d9b640 1 --2- 172.21.15.83:0/2313796352 >> v2:172.21.15.167:3300/0 conn(0x7ff71406b520 0x7ff7140b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:22.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.154+0000 7ff712d9d640 1 -- 172.21.15.83:0/2313796352 >> v2:172.21.15.167:3300/0 conn(0x7ff71406b520 msgr2=0x7ff7140b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:22.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.154+0000 7ff712d9d640 1 --2- 172.21.15.83:0/2313796352 >> v2:172.21.15.167:3300/0 conn(0x7ff71406b520 0x7ff7140b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:22.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.154+0000 7ff712d9d640 1 -- 172.21.15.83:0/2313796352 >> v1:172.21.15.167:6789/0 conn(0x7ff714154de0 legacy=0x7ff71406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:22.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.154+0000 7ff712d9d640 1 -- 172.21.15.83:0/2313796352 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff71415afb0 con 0x7ff7140b4040 2024-04-17T10:48:22.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.154+0000 7ff712d9d640 1 --2- 172.21.15.83:0/2313796352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7140b4040 0x7ff71406cbe0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7ff7140b2480 tx=0x7ff70c01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ddc0f6cbd8c63 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:22.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff71159a640 1 -- 172.21.15.83:0/2313796352 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70c0a5760 con 0x7ff7140b4040 2024-04-17T10:48:22.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff71159a640 1 -- 172.21.15.83:0/2313796352 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff70c0a5900 con 0x7ff7140b4040 2024-04-17T10:48:22.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff71159a640 1 -- 172.21.15.83:0/2313796352 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70c0a5cd0 con 0x7ff7140b4040 2024-04-17T10:48:22.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff718dae640 1 -- 172.21.15.83:0/2313796352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7140b4040 msgr2=0x7ff71406cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:22.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff718dae640 1 --2- 172.21.15.83:0/2313796352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7140b4040 0x7ff71406cbe0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7ff7140b2480 tx=0x7ff70c01c9b0 comp rx=0 tx=0).stop 2024-04-17T10:48:22.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff718dae640 1 -- 172.21.15.83:0/2313796352 shutdown_connections 2024-04-17T10:48:22.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff718dae640 1 --2- 172.21.15.83:0/2313796352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7140b4040 0x7ff71406cbe0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:22.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff718dae640 1 --2- 172.21.15.83:0/2313796352 >> v2:172.21.15.167:3300/0 conn(0x7ff71406b520 0x7ff7140b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:22.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff718dae640 1 -- 172.21.15.83:0/2313796352 >> 172.21.15.83:0/2313796352 conn(0x7ff7140b15b0 msgr2=0x7ff7140b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:22.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff718dae640 1 -- 172.21.15.83:0/2313796352 shutdown_connections 2024-04-17T10:48:22.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff718dae640 1 -- 172.21.15.83:0/2313796352 wait complete. 2024-04-17T10:48:22.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff718dae640 1 Processor -- start 2024-04-17T10:48:22.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff718dae640 1 -- start start 2024-04-17T10:48:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff718dae640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff71406b520 0x7ff71415a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff718dae640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7140b4040 0x7ff71413b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff718dae640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff714154de0 0x7ff7140ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff718dae640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7140b44a0 con 0x7ff714154de0 2024-04-17T10:48:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff718dae640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7141551c0 con 0x7ff7140b4040 2024-04-17T10:48:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff718dae640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7140b4620 con 0x7ff71406b520 2024-04-17T10:48:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff712d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff714154de0 0x7ff7140ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff712d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff714154de0 0x7ff7140ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42442/0 (socket says 172.21.15.83:42442) 2024-04-17T10:48:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff712d9d640 1 -- 172.21.15.83:0/3677721832 learned_addr learned my addr 172.21.15.83:0/3677721832 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff711d9b640 1 --2- 172.21.15.83:0/3677721832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7140b4040 0x7ff71413b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff71259c640 1 --2- 172.21.15.83:0/3677721832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff71406b520 0x7ff71415a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff712d9d640 1 -- 172.21.15.83:0/3677721832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff71406b520 msgr2=0x7ff71415a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff712d9d640 1 --2- 172.21.15.83:0/3677721832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff71406b520 0x7ff71415a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff712d9d640 1 -- 172.21.15.83:0/3677721832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7140b4040 msgr2=0x7ff71413b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff712d9d640 1 --2- 172.21.15.83:0/3677721832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7140b4040 0x7ff71413b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff712d9d640 1 -- 172.21.15.83:0/3677721832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7140afe40 con 0x7ff714154de0 2024-04-17T10:48:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff71259c640 1 --2- 172.21.15.83:0/3677721832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff71406b520 0x7ff71415a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff712d9d640 1 --2- 172.21.15.83:0/3677721832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff714154de0 0x7ff7140ad840 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7ff70c01a5e0 tx=0x7ff70c019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff702ffd640 1 -- 172.21.15.83:0/3677721832 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70c0bc020 con 0x7ff714154de0 2024-04-17T10:48:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff702ffd640 1 -- 172.21.15.83:0/3677721832 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff70c0b8670 con 0x7ff714154de0 2024-04-17T10:48:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff718dae640 1 -- 172.21.15.83:0/3677721832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7140b0120 con 0x7ff714154de0 2024-04-17T10:48:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.158+0000 7ff702ffd640 1 -- 172.21.15.83:0/3677721832 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70c0b8a60 con 0x7ff714154de0 2024-04-17T10:48:22.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.162+0000 7ff718dae640 1 -- 172.21.15.83:0/3677721832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7140afb10 con 0x7ff714154de0 2024-04-17T10:48:22.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.162+0000 7ff718dae640 1 -- 172.21.15.83:0/3677721832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff714074f90 con 0x7ff714154de0 2024-04-17T10:48:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.162+0000 7ff702ffd640 1 -- 172.21.15.83:0/3677721832 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff70c0bf020 con 0x7ff714154de0 2024-04-17T10:48:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.162+0000 7ff702ffd640 1 --2- 172.21.15.83:0/3677721832 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff6f4066f60 0x7ff6f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.162+0000 7ff702ffd640 1 -- 172.21.15.83:0/3677721832 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff70c11fef0 con 0x7ff714154de0 2024-04-17T10:48:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.162+0000 7ff71259c640 1 --2- 172.21.15.83:0/3677721832 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff6f4066f60 0x7ff6f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.162+0000 7ff71259c640 1 --2- 172.21.15.83:0/3677721832 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff6f4066f60 0x7ff6f4069420 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ff70400a850 tx=0x7ff704008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:22.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.166+0000 7ff702ffd640 1 -- 172.21.15.83:0/3677721832 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff70c0c6030 con 0x7ff714154de0 2024-04-17T10:48:22.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.314+0000 7ff718dae640 1 -- 172.21.15.83:0/3677721832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff7140b02f0 con 0x7ff714154de0 2024-04-17T10:48:22.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.314+0000 7ff702ffd640 1 -- 172.21.15.83:0/3677721832 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff70c0165a0 con 0x7ff714154de0 2024-04-17T10:48:22.311 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:48:22.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.314+0000 7ff718dae640 1 -- 172.21.15.83:0/3677721832 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff6f4066f60 msgr2=0x7ff6f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:22.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.318+0000 7ff718dae640 1 --2- 172.21.15.83:0/3677721832 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff6f4066f60 0x7ff6f4069420 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ff70400a850 tx=0x7ff704008040 comp rx=0 tx=0).stop 2024-04-17T10:48:22.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.318+0000 7ff718dae640 1 -- 172.21.15.83:0/3677721832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff714154de0 msgr2=0x7ff7140ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:22.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.318+0000 7ff718dae640 1 --2- 172.21.15.83:0/3677721832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff714154de0 0x7ff7140ad840 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7ff70c01a5e0 tx=0x7ff70c019750 comp rx=0 tx=0).stop 2024-04-17T10:48:22.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.318+0000 7ff718dae640 1 -- 172.21.15.83:0/3677721832 shutdown_connections 2024-04-17T10:48:22.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.318+0000 7ff718dae640 1 --2- 172.21.15.83:0/3677721832 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff6f4066f60 0x7ff6f4069420 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:22.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.318+0000 7ff718dae640 1 --2- 172.21.15.83:0/3677721832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff714154de0 0x7ff7140ad840 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:22.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.318+0000 7ff718dae640 1 --2- 172.21.15.83:0/3677721832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7140b4040 0x7ff71413b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:22.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.318+0000 7ff718dae640 1 --2- 172.21.15.83:0/3677721832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff71406b520 0x7ff71415a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:22.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.318+0000 7ff718dae640 1 -- 172.21.15.83:0/3677721832 >> 172.21.15.83:0/3677721832 conn(0x7ff7140b15b0 msgr2=0x7ff71406ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:22.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.318+0000 7ff718dae640 1 -- 172.21.15.83:0/3677721832 shutdown_connections 2024-04-17T10:48:22.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.318+0000 7ff718dae640 1 -- 172.21.15.83:0/3677721832 wait complete. 2024-04-17T10:48:22.329 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-17T10:48:22.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.438+0000 7feb14465640 1 Processor -- start 2024-04-17T10:48:22.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.438+0000 7feb14465640 1 -- start start 2024-04-17T10:48:22.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.442+0000 7feb14465640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7feb0c15db50 0x7feb0c070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:22.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.442+0000 7feb14465640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb0c06dcb0 0x7feb0c0b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:22.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.442+0000 7feb14465640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb0c0709b0 con 0x7feb0c0aedd0 2024-04-17T10:48:22.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.442+0000 7feb14465640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb0c15e440 con 0x7feb0c15db50 2024-04-17T10:48:22.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.442+0000 7feb14465640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c15e720 con 0x7feb0c06dcb0 2024-04-17T10:48:22.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.442+0000 7feb12a02640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb0c06dcb0 0x7feb0c0b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:22.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.442+0000 7feb11a00640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7feb0c0aedd0 0x7feb0c06d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:58564/0 (socket says 172.21.15.83:58564) 2024-04-17T10:48:22.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.442+0000 7feb11a00640 1 -- 172.21.15.83:0/4061929273 learned_addr learned my addr 172.21.15.83:0/4061929273 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:22.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.442+0000 7feb12201640 1 --2- 172.21.15.83:0/4061929273 >> v2:172.21.15.167:3300/0 conn(0x7feb0c15db50 0x7feb0c070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:22.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.442+0000 7feb12201640 1 -- 172.21.15.83:0/4061929273 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb0c06dcb0 msgr2=0x7feb0c0b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:22.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.442+0000 7feb12201640 1 --2- 172.21.15.83:0/4061929273 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb0c06dcb0 0x7feb0c0b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:22.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.442+0000 7feb12201640 1 -- 172.21.15.83:0/4061929273 >> v1:172.21.15.167:6789/0 conn(0x7feb0c0aedd0 legacy=0x7feb0c06d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:22.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.442+0000 7feb12201640 1 -- 172.21.15.83:0/4061929273 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0c15ea00 con 0x7feb0c15db50 2024-04-17T10:48:22.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.442+0000 7feb12a02640 1 --2- 172.21.15.83:0/4061929273 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb0c06dcb0 0x7feb0c0b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:48:22.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.442+0000 7feb12201640 1 --2- 172.21.15.83:0/4061929273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb0c15db50 0x7feb0c070470 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7feb080097c0 tx=0x7feb0801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=141176de3ce08e0e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:22.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb111ff640 1 -- 172.21.15.83:0/4061929273 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb080a7070 con 0x7feb0c15db50 2024-04-17T10:48:22.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb111ff640 1 -- 172.21.15.83:0/4061929273 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb080b0070 con 0x7feb0c15db50 2024-04-17T10:48:22.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb111ff640 1 -- 172.21.15.83:0/4061929273 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb080ab890 con 0x7feb0c15db50 2024-04-17T10:48:22.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb14465640 1 -- 172.21.15.83:0/4061929273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb0c15db50 msgr2=0x7feb0c070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:22.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb14465640 1 --2- 172.21.15.83:0/4061929273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb0c15db50 0x7feb0c070470 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7feb080097c0 tx=0x7feb0801c470 comp rx=0 tx=0).stop 2024-04-17T10:48:22.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb14465640 1 -- 172.21.15.83:0/4061929273 shutdown_connections 2024-04-17T10:48:22.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb14465640 1 --2- 172.21.15.83:0/4061929273 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb0c06dcb0 0x7feb0c0b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:22.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb14465640 1 --2- 172.21.15.83:0/4061929273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb0c15db50 0x7feb0c070470 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:22.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb14465640 1 -- 172.21.15.83:0/4061929273 >> 172.21.15.83:0/4061929273 conn(0x7feb0c06abe0 msgr2=0x7feb0c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:22.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb14465640 1 -- 172.21.15.83:0/4061929273 shutdown_connections 2024-04-17T10:48:22.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb14465640 1 -- 172.21.15.83:0/4061929273 wait complete. 2024-04-17T10:48:22.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb14465640 1 Processor -- start 2024-04-17T10:48:22.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb14465640 1 -- start start 2024-04-17T10:48:22.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb14465640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb0c06dcb0 0x7feb0c149190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:22.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb14465640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb0c0aedd0 0x7feb0c15bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:22.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb14465640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb0c15db50 0x7feb0c0cd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:22.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb14465640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c0b3ce0 con 0x7feb0c06dcb0 2024-04-17T10:48:22.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb14465640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c0b3b60 con 0x7feb0c15db50 2024-04-17T10:48:22.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb14465640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c0b3e60 con 0x7feb0c0aedd0 2024-04-17T10:48:22.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb12201640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb0c06dcb0 0x7feb0c149190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:22.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb12201640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb0c06dcb0 0x7feb0c149190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42446/0 (socket says 172.21.15.83:42446) 2024-04-17T10:48:22.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb12201640 1 -- 172.21.15.83:0/2080733263 learned_addr learned my addr 172.21.15.83:0/2080733263 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:22.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb12201640 1 -- 172.21.15.83:0/2080733263 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb0c0aedd0 msgr2=0x7feb0c15bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:22.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb12201640 1 --2- 172.21.15.83:0/2080733263 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb0c0aedd0 0x7feb0c15bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:22.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb12201640 1 -- 172.21.15.83:0/2080733263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb0c15db50 msgr2=0x7feb0c0cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:22.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb12201640 1 --2- 172.21.15.83:0/2080733263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb0c15db50 0x7feb0c0cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:22.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb12201640 1 -- 172.21.15.83:0/2080733263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0c15c7a0 con 0x7feb0c06dcb0 2024-04-17T10:48:22.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb12201640 1 --2- 172.21.15.83:0/2080733263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb0c06dcb0 0x7feb0c149190 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7feb080ad040 tx=0x7feb0801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:22.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb02ffd640 1 -- 172.21.15.83:0/2080733263 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb080bf610 con 0x7feb0c06dcb0 2024-04-17T10:48:22.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb02ffd640 1 -- 172.21.15.83:0/2080733263 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb080bf7d0 con 0x7feb0c06dcb0 2024-04-17T10:48:22.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb14465640 1 -- 172.21.15.83:0/2080733263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb0c15ca80 con 0x7feb0c06dcb0 2024-04-17T10:48:22.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb02ffd640 1 -- 172.21.15.83:0/2080733263 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb080bfbc0 con 0x7feb0c06dcb0 2024-04-17T10:48:22.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.446+0000 7feb14465640 1 -- 172.21.15.83:0/2080733263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb0c15d040 con 0x7feb0c06dcb0 2024-04-17T10:48:22.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.450+0000 7feb14465640 1 -- 172.21.15.83:0/2080733263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0c074f90 con 0x7feb0c06dcb0 2024-04-17T10:48:22.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.450+0000 7feb02ffd640 1 -- 172.21.15.83:0/2080733263 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feb080bd020 con 0x7feb0c06dcb0 2024-04-17T10:48:22.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.450+0000 7feb02ffd640 1 --2- 172.21.15.83:0/2080733263 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feaf0066f60 0x7feaf0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:22.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.450+0000 7feb11a00640 1 --2- 172.21.15.83:0/2080733263 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feaf0066f60 0x7feaf0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:22.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.450+0000 7feb02ffd640 1 -- 172.21.15.83:0/2080733263 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 4857+0+0 (secure 0 0 0) 0x7feb0811f2d0 con 0x7feb0c06dcb0 2024-04-17T10:48:22.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.450+0000 7feb11a00640 1 --2- 172.21.15.83:0/2080733263 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feaf0066f60 0x7feaf0069420 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7feb0c1f9b60 tx=0x7feafc007a50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:22.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.454+0000 7feb02ffd640 1 -- 172.21.15.83:0/2080733263 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feb08016090 con 0x7feb0c06dcb0 2024-04-17T10:48:22.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.614+0000 7feb14465640 1 -- 172.21.15.83:0/2080733263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7feb0c0b2af0 con 0x7feb0c06dcb0 2024-04-17T10:48:22.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.618+0000 7feb02ffd640 1 -- 172.21.15.83:0/2080733263 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v110) v1 ==== 84+0+65 (secure 0 0 0) 0x7feb080f0630 con 0x7feb0c06dcb0 2024-04-17T10:48:22.613 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:22.613 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-04-17T10:48:22.613 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:48:22.613 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2024-04-17T10:48:22.613 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2024-04-17T10:48:22.613 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:48:22.614 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-04-17T10:48:22.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.618+0000 7feb14465640 1 -- 172.21.15.83:0/2080733263 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feaf0066f60 msgr2=0x7feaf0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:22.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.618+0000 7feb14465640 1 --2- 172.21.15.83:0/2080733263 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feaf0066f60 0x7feaf0069420 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7feb0c1f9b60 tx=0x7feafc007a50 comp rx=0 tx=0).stop 2024-04-17T10:48:22.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.618+0000 7feb14465640 1 -- 172.21.15.83:0/2080733263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb0c06dcb0 msgr2=0x7feb0c149190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:22.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.618+0000 7feb14465640 1 --2- 172.21.15.83:0/2080733263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb0c06dcb0 0x7feb0c149190 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7feb080ad040 tx=0x7feb0801acb0 comp rx=0 tx=0).stop 2024-04-17T10:48:22.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.618+0000 7feb14465640 1 -- 172.21.15.83:0/2080733263 shutdown_connections 2024-04-17T10:48:22.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.618+0000 7feb14465640 1 --2- 172.21.15.83:0/2080733263 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feaf0066f60 0x7feaf0069420 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:22.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.618+0000 7feb14465640 1 --2- 172.21.15.83:0/2080733263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb0c15db50 0x7feb0c0cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:22.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.618+0000 7feb14465640 1 --2- 172.21.15.83:0/2080733263 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb0c0aedd0 0x7feb0c15bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:22.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.618+0000 7feb14465640 1 --2- 172.21.15.83:0/2080733263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb0c06dcb0 0x7feb0c149190 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:22.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.618+0000 7feb14465640 1 -- 172.21.15.83:0/2080733263 >> 172.21.15.83:0/2080733263 conn(0x7feb0c06abe0 msgr2=0x7feb0c0b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:22.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.618+0000 7feb14465640 1 -- 172.21.15.83:0/2080733263 shutdown_connections 2024-04-17T10:48:22.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.618+0000 7feb14465640 1 -- 172.21.15.83:0/2080733263 wait complete. 2024-04-17T10:48:22.632 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-17T10:48:22.632 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-17T10:48:22.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.762+0000 7f279ca19640 1 Processor -- start 2024-04-17T10:48:22.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.762+0000 7f279ca19640 1 -- start start 2024-04-17T10:48:22.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.762+0000 7f279ca19640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f27980aedf0 0x7f27980b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:22.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.762+0000 7f279ca19640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f279815cb40 0x7f279806abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:22.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.762+0000 7f279ca19640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27980b39d0 con 0x7f279806b120 2024-04-17T10:48:22.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.762+0000 7f279ca19640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2798072730 con 0x7f27980aedf0 2024-04-17T10:48:22.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.762+0000 7f279ca19640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2798071290 con 0x7f279815cb40 2024-04-17T10:48:22.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.762+0000 7f2796d9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f279806b120 0x7f279815f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51238/0 (socket says 172.21.15.83:51238) 2024-04-17T10:48:22.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.762+0000 7f279659c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f27980aedf0 0x7f27980b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:22.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.762+0000 7f2796d9d640 1 -- 172.21.15.83:0/774457678 learned_addr learned my addr 172.21.15.83:0/774457678 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:22.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.766+0000 7f2795d9b640 1 --2- 172.21.15.83:0/774457678 >> v2:172.21.15.167:3300/0 conn(0x7f279815cb40 0x7f279806abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:22.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.766+0000 7f279659c640 1 -- 172.21.15.83:0/774457678 >> v2:172.21.15.167:3300/0 conn(0x7f279815cb40 msgr2=0x7f279806abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:22.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.766+0000 7f279659c640 1 --2- 172.21.15.83:0/774457678 >> v2:172.21.15.167:3300/0 conn(0x7f279815cb40 0x7f279806abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:22.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.766+0000 7f279659c640 1 -- 172.21.15.83:0/774457678 >> v1:172.21.15.83:6789/0 conn(0x7f279806b120 legacy=0x7f279815f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:22.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.766+0000 7f279659c640 1 -- 172.21.15.83:0/774457678 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2798070c40 con 0x7f27980aedf0 2024-04-17T10:48:22.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.766+0000 7f279559a640 1 -- 172.21.15.83:0/774457678 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 581044507 0 0) 0x7f27980b39d0 con 0x7f279806b120 2024-04-17T10:48:22.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.766+0000 7f279659c640 1 --2- 172.21.15.83:0/774457678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27980aedf0 0x7f27980b30f0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f27980b2920 tx=0x7f278801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866455ad30daa067 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:22.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.766+0000 7f279559a640 1 -- 172.21.15.83:0/774457678 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27880b0070 con 0x7f27980aedf0 2024-04-17T10:48:22.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.766+0000 7f279559a640 1 -- 172.21.15.83:0/774457678 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27880ab600 con 0x7f27980aedf0 2024-04-17T10:48:22.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.766+0000 7f279559a640 1 -- 172.21.15.83:0/774457678 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27880ab9d0 con 0x7f27980aedf0 2024-04-17T10:48:22.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.766+0000 7f279ca19640 1 -- 172.21.15.83:0/774457678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27980aedf0 msgr2=0x7f27980b30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:22.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.766+0000 7f279ca19640 1 --2- 172.21.15.83:0/774457678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27980aedf0 0x7f27980b30f0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f27980b2920 tx=0x7f278801c480 comp rx=0 tx=0).stop 2024-04-17T10:48:22.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.766+0000 7f279ca19640 1 -- 172.21.15.83:0/774457678 shutdown_connections 2024-04-17T10:48:22.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.766+0000 7f279ca19640 1 --2- 172.21.15.83:0/774457678 >> v2:172.21.15.167:3300/0 conn(0x7f279815cb40 0x7f279806abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:22.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.766+0000 7f279ca19640 1 --2- 172.21.15.83:0/774457678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27980aedf0 0x7f27980b30f0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:22.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.766+0000 7f279ca19640 1 -- 172.21.15.83:0/774457678 >> 172.21.15.83:0/774457678 conn(0x7f27980cc050 msgr2=0x7f27980cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:22.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.766+0000 7f279ca19640 1 -- 172.21.15.83:0/774457678 shutdown_connections 2024-04-17T10:48:22.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.766+0000 7f279ca19640 1 -- 172.21.15.83:0/774457678 wait complete. 2024-04-17T10:48:22.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.770+0000 7f279ca19640 1 Processor -- start 2024-04-17T10:48:22.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.770+0000 7f279ca19640 1 -- start start 2024-04-17T10:48:22.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.770+0000 7f279ca19640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f279806b120 0x7f2798158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:22.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.770+0000 7f279ca19640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27980aedf0 0x7f2798148c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:22.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.770+0000 7f279ca19640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f279815cb40 0x7f27980ce340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:22.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.770+0000 7f279ca19640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279806c2e0 con 0x7f27980aedf0 2024-04-17T10:48:22.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.770+0000 7f279ca19640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279806c160 con 0x7f279815cb40 2024-04-17T10:48:22.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.770+0000 7f279ca19640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2798168890 con 0x7f279806b120 2024-04-17T10:48:22.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.770+0000 7f2795d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27980aedf0 0x7f2798148c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:22.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.770+0000 7f2795d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27980aedf0 0x7f2798148c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42470/0 (socket says 172.21.15.83:42470) 2024-04-17T10:48:22.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.770+0000 7f2795d9b640 1 -- 172.21.15.83:0/1670689377 learned_addr learned my addr 172.21.15.83:0/1670689377 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:22.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.770+0000 7f2795d9b640 1 -- 172.21.15.83:0/1670689377 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f279806b120 msgr2=0x7f2798158560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:22.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.770+0000 7f2795d9b640 1 --2- 172.21.15.83:0/1670689377 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f279806b120 0x7f2798158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:22.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.770+0000 7f2795d9b640 1 -- 172.21.15.83:0/1670689377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f279815cb40 msgr2=0x7f27980ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:22.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.770+0000 7f2796d9d640 1 --2- 172.21.15.83:0/1670689377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f279815cb40 0x7f27980ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:22.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.770+0000 7f2795d9b640 1 --2- 172.21.15.83:0/1670689377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f279815cb40 0x7f27980ce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:22.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.770+0000 7f2795d9b640 1 -- 172.21.15.83:0/1670689377 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2798158c40 con 0x7f27980aedf0 2024-04-17T10:48:22.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.770+0000 7f2795d9b640 1 --2- 172.21.15.83:0/1670689377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27980aedf0 0x7f2798148c10 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f278c00d800 tx=0x7f278c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:22.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.770+0000 7f2786ffd640 1 -- 172.21.15.83:0/1670689377 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278c00b6c0 con 0x7f27980aedf0 2024-04-17T10:48:22.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.770+0000 7f2786ffd640 1 -- 172.21.15.83:0/1670689377 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f278c019070 con 0x7f27980aedf0 2024-04-17T10:48:22.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.770+0000 7f279ca19640 1 -- 172.21.15.83:0/1670689377 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2798158f20 con 0x7f27980aedf0 2024-04-17T10:48:22.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.770+0000 7f2786ffd640 1 -- 172.21.15.83:0/1670689377 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278c00ba60 con 0x7f27980aedf0 2024-04-17T10:48:22.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.770+0000 7f279ca19640 1 -- 172.21.15.83:0/1670689377 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27981594e0 con 0x7f27980aedf0 2024-04-17T10:48:22.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.770+0000 7f279ca19640 1 -- 172.21.15.83:0/1670689377 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2798075000 con 0x7f27980aedf0 2024-04-17T10:48:22.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.770+0000 7f2786ffd640 1 -- 172.21.15.83:0/1670689377 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f278c00bc00 con 0x7f27980aedf0 2024-04-17T10:48:22.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.770+0000 7f2786ffd640 1 --2- 172.21.15.83:0/1670689377 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2778066f90 0x7f2778069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:22.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.774+0000 7f2786ffd640 1 -- 172.21.15.83:0/1670689377 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f278c085140 con 0x7f27980aedf0 2024-04-17T10:48:22.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.774+0000 7f279659c640 1 --2- 172.21.15.83:0/1670689377 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2778066f90 0x7f2778069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:22.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.774+0000 7f279659c640 1 --2- 172.21.15.83:0/1670689377 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2778066f90 0x7f2778069450 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f27880ad000 tx=0x7f2788008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:22.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.778+0000 7f2786ffd640 1 -- 172.21.15.83:0/1670689377 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f278c0564a0 con 0x7f27980aedf0 2024-04-17T10:48:22.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:22.938+0000 7f279ca19640 1 -- 172.21.15.83:0/1670689377 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f27980b2cc0 con 0x7f27980aedf0 2024-04-17T10:48:23.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.354+0000 7f2786ffd640 1 -- 172.21.15.83:0/1670689377 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v111) v1 ==== 138+0+0 (secure 0 0 0) 0x7f278c055da0 con 0x7f27980aedf0 2024-04-17T10:48:23.349 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2024-04-17T10:48:23.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.354+0000 7f279ca19640 1 -- 172.21.15.83:0/1670689377 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2778066f90 msgr2=0x7f2778069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:23.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.354+0000 7f279ca19640 1 --2- 172.21.15.83:0/1670689377 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2778066f90 0x7f2778069450 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f27880ad000 tx=0x7f2788008040 comp rx=0 tx=0).stop 2024-04-17T10:48:23.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.354+0000 7f279ca19640 1 -- 172.21.15.83:0/1670689377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27980aedf0 msgr2=0x7f2798148c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:23.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.354+0000 7f279ca19640 1 --2- 172.21.15.83:0/1670689377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27980aedf0 0x7f2798148c10 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f278c00d800 tx=0x7f278c00dcc0 comp rx=0 tx=0).stop 2024-04-17T10:48:23.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.358+0000 7f279ca19640 1 -- 172.21.15.83:0/1670689377 shutdown_connections 2024-04-17T10:48:23.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.358+0000 7f279ca19640 1 --2- 172.21.15.83:0/1670689377 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2778066f90 0x7f2778069450 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:23.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.358+0000 7f279ca19640 1 --2- 172.21.15.83:0/1670689377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f279815cb40 0x7f27980ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:23.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.358+0000 7f279ca19640 1 --2- 172.21.15.83:0/1670689377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27980aedf0 0x7f2798148c10 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:23.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.358+0000 7f279ca19640 1 --2- 172.21.15.83:0/1670689377 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f279806b120 0x7f2798158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:23.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.358+0000 7f279ca19640 1 -- 172.21.15.83:0/1670689377 >> 172.21.15.83:0/1670689377 conn(0x7f27980cc050 msgr2=0x7f279815d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:23.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.358+0000 7f279ca19640 1 -- 172.21.15.83:0/1670689377 shutdown_connections 2024-04-17T10:48:23.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.358+0000 7f279ca19640 1 -- 172.21.15.83:0/1670689377 wait complete. 2024-04-17T10:48:23.380 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-17T10:48:23.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.502+0000 7f4885d3b640 1 Processor -- start 2024-04-17T10:48:23.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.502+0000 7f4885d3b640 1 -- start start 2024-04-17T10:48:23.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.502+0000 7f4885d3b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f488015bc10 0x7f48800d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:23.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.502+0000 7f4885d3b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f48800d0490 0x7f48800cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:23.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.502+0000 7f4885d3b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48800cd3d0 0x7f488006e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:23.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.502+0000 7f4885d3b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48800ced00 con 0x7f48800d0490 2024-04-17T10:48:23.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.502+0000 7f4885d3b640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48800cefe0 con 0x7f488015bc10 2024-04-17T10:48:23.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.502+0000 7f4885d3b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48800cc690 con 0x7f48800cd3d0 2024-04-17T10:48:23.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.502+0000 7f4877fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f48800d0490 0x7f48800cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:23.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.506+0000 7f4877fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f48800d0490 0x7f48800cce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42476/0 (socket says 172.21.15.83:42476) 2024-04-17T10:48:23.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.506+0000 7f4877fff640 1 -- 172.21.15.83:0/1389487916 learned_addr learned my addr 172.21.15.83:0/1389487916 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:23.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.506+0000 7f487f7fe640 1 --2- 172.21.15.83:0/1389487916 >> v2:172.21.15.167:3300/0 conn(0x7f488015bc10 0x7f48800d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:23.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.506+0000 7f487ffff640 1 --2- 172.21.15.83:0/1389487916 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48800cd3d0 0x7f488006e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:23.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.506+0000 7f4877fff640 1 -- 172.21.15.83:0/1389487916 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48800cd3d0 msgr2=0x7f488006e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:23.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.506+0000 7f4877fff640 1 --2- 172.21.15.83:0/1389487916 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48800cd3d0 0x7f488006e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:23.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.506+0000 7f4877fff640 1 -- 172.21.15.83:0/1389487916 >> v2:172.21.15.167:3300/0 conn(0x7f488015bc10 msgr2=0x7f48800d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:23.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.506+0000 7f4877fff640 1 --2- 172.21.15.83:0/1389487916 >> v2:172.21.15.167:3300/0 conn(0x7f488015bc10 0x7f48800d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:23.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.506+0000 7f4877fff640 1 -- 172.21.15.83:0/1389487916 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48800cc970 con 0x7f48800d0490 2024-04-17T10:48:23.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.506+0000 7f4877fff640 1 --2- 172.21.15.83:0/1389487916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48800d0490 0x7f48800cce90 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f48800678a0 tx=0x7f486801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90f3e70cb5eaaae server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:23.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.506+0000 7f487effd640 1 -- 172.21.15.83:0/1389487916 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48680b0070 con 0x7f48800d0490 2024-04-17T10:48:23.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.506+0000 7f487effd640 1 -- 172.21.15.83:0/1389487916 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48680ab600 con 0x7f48800d0490 2024-04-17T10:48:23.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.506+0000 7f487effd640 1 -- 172.21.15.83:0/1389487916 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48680ab9d0 con 0x7f48800d0490 2024-04-17T10:48:23.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.506+0000 7f4885d3b640 1 -- 172.21.15.83:0/1389487916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48800d0490 msgr2=0x7f48800cce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:23.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.506+0000 7f4885d3b640 1 --2- 172.21.15.83:0/1389487916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48800d0490 0x7f48800cce90 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f48800678a0 tx=0x7f486801c490 comp rx=0 tx=0).stop 2024-04-17T10:48:23.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.506+0000 7f4885d3b640 1 -- 172.21.15.83:0/1389487916 shutdown_connections 2024-04-17T10:48:23.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.506+0000 7f4885d3b640 1 --2- 172.21.15.83:0/1389487916 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48800cd3d0 0x7f488006e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:23.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.506+0000 7f4885d3b640 1 --2- 172.21.15.83:0/1389487916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48800d0490 0x7f48800cce90 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:23.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.506+0000 7f4885d3b640 1 --2- 172.21.15.83:0/1389487916 >> v2:172.21.15.167:3300/0 conn(0x7f488015bc10 0x7f48800d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:23.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.506+0000 7f4885d3b640 1 -- 172.21.15.83:0/1389487916 >> 172.21.15.83:0/1389487916 conn(0x7f48800d17e0 msgr2=0x7f48800d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:23.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.506+0000 7f4885d3b640 1 -- 172.21.15.83:0/1389487916 shutdown_connections 2024-04-17T10:48:23.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.506+0000 7f4885d3b640 1 -- 172.21.15.83:0/1389487916 wait complete. 2024-04-17T10:48:23.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.510+0000 7f4885d3b640 1 Processor -- start 2024-04-17T10:48:23.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.510+0000 7f4885d3b640 1 -- start start 2024-04-17T10:48:23.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.510+0000 7f4885d3b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48800cd3d0 0x7f48800b4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:23.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.510+0000 7f4885d3b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48800d0490 0x7f48800c44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:23.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.510+0000 7f4885d3b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f488015bc10 0x7f4880157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:23.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.510+0000 7f4885d3b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488015f9f0 con 0x7f48800cd3d0 2024-04-17T10:48:23.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.510+0000 7f4885d3b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488015f870 con 0x7f488015bc10 2024-04-17T10:48:23.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.510+0000 7f4885d3b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488015fb70 con 0x7f48800d0490 2024-04-17T10:48:23.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.510+0000 7f487f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48800cd3d0 0x7f48800b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:23.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.510+0000 7f487f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48800cd3d0 0x7f48800b4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42486/0 (socket says 172.21.15.83:42486) 2024-04-17T10:48:23.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.510+0000 7f487f7fe640 1 -- 172.21.15.83:0/3439736848 learned_addr learned my addr 172.21.15.83:0/3439736848 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:23.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.510+0000 7f4877fff640 1 --2- 172.21.15.83:0/3439736848 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48800d0490 0x7f48800c44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:23.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.510+0000 7f487ffff640 1 --2- 172.21.15.83:0/3439736848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f488015bc10 0x7f4880157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:23.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.510+0000 7f487f7fe640 1 -- 172.21.15.83:0/3439736848 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48800d0490 msgr2=0x7f48800c44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:23.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.510+0000 7f487f7fe640 1 --2- 172.21.15.83:0/3439736848 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48800d0490 0x7f48800c44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:23.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.510+0000 7f487f7fe640 1 -- 172.21.15.83:0/3439736848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f488015bc10 msgr2=0x7f4880157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:23.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.510+0000 7f487f7fe640 1 --2- 172.21.15.83:0/3439736848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f488015bc10 0x7f4880157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:23.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.510+0000 7f487f7fe640 1 -- 172.21.15.83:0/3439736848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f488015f450 con 0x7f48800cd3d0 2024-04-17T10:48:23.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.510+0000 7f487ffff640 1 --2- 172.21.15.83:0/3439736848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f488015bc10 0x7f4880157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:23.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.510+0000 7f487f7fe640 1 --2- 172.21.15.83:0/3439736848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48800cd3d0 0x7f48800b4400 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f487000d5f0 tx=0x7f487000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:23.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.510+0000 7f487cff9640 1 -- 172.21.15.83:0/3439736848 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48700125f0 con 0x7f48800cd3d0 2024-04-17T10:48:23.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.510+0000 7f487cff9640 1 -- 172.21.15.83:0/3439736848 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4870019070 con 0x7f48800cd3d0 2024-04-17T10:48:23.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.510+0000 7f4885d3b640 1 -- 172.21.15.83:0/3439736848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48800b4940 con 0x7f48800cd3d0 2024-04-17T10:48:23.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.510+0000 7f487cff9640 1 -- 172.21.15.83:0/3439736848 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48700129b0 con 0x7f48800cd3d0 2024-04-17T10:48:23.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.510+0000 7f4885d3b640 1 -- 172.21.15.83:0/3439736848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48801fb1e0 con 0x7f48800cd3d0 2024-04-17T10:48:23.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.510+0000 7f4885d3b640 1 -- 172.21.15.83:0/3439736848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4880074f90 con 0x7f48800cd3d0 2024-04-17T10:48:23.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.510+0000 7f487cff9640 1 -- 172.21.15.83:0/3439736848 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4870012b50 con 0x7f48800cd3d0 2024-04-17T10:48:23.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.514+0000 7f487cff9640 1 --2- 172.21.15.83:0/3439736848 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4860066f60 0x7f4860069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:23.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.514+0000 7f487cff9640 1 -- 172.21.15.83:0/3439736848 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f48700851d0 con 0x7f48800cd3d0 2024-04-17T10:48:23.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.514+0000 7f4877fff640 1 --2- 172.21.15.83:0/3439736848 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4860066f60 0x7f4860069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:23.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.514+0000 7f4877fff640 1 --2- 172.21.15.83:0/3439736848 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4860066f60 0x7f4860069420 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f48680ad000 tx=0x7f4868008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:23.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.518+0000 7f487cff9640 1 -- 172.21.15.83:0/3439736848 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4870056450 con 0x7f48800cd3d0 2024-04-17T10:48:23.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:23.662+0000 7f4885d3b640 1 -- 172.21.15.83:0/3439736848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f488006d6a0 con 0x7f48800cd3d0 2024-04-17T10:48:24.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.358+0000 7f487cff9640 1 -- 172.21.15.83:0/3439736848 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v112) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4870055d50 con 0x7f48800cd3d0 2024-04-17T10:48:24.354 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2024-04-17T10:48:24.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.362+0000 7f4885d3b640 1 -- 172.21.15.83:0/3439736848 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4860066f60 msgr2=0x7f4860069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:24.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.362+0000 7f4885d3b640 1 --2- 172.21.15.83:0/3439736848 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4860066f60 0x7f4860069420 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f48680ad000 tx=0x7f4868008040 comp rx=0 tx=0).stop 2024-04-17T10:48:24.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.362+0000 7f4885d3b640 1 -- 172.21.15.83:0/3439736848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48800cd3d0 msgr2=0x7f48800b4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:24.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.362+0000 7f4885d3b640 1 --2- 172.21.15.83:0/3439736848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48800cd3d0 0x7f48800b4400 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f487000d5f0 tx=0x7f487000dab0 comp rx=0 tx=0).stop 2024-04-17T10:48:24.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.362+0000 7f4885d3b640 1 -- 172.21.15.83:0/3439736848 shutdown_connections 2024-04-17T10:48:24.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.362+0000 7f4885d3b640 1 --2- 172.21.15.83:0/3439736848 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4860066f60 0x7f4860069420 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:24.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.362+0000 7f4885d3b640 1 --2- 172.21.15.83:0/3439736848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f488015bc10 0x7f4880157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:24.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.362+0000 7f4885d3b640 1 --2- 172.21.15.83:0/3439736848 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48800d0490 0x7f48800c44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:24.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.362+0000 7f4885d3b640 1 --2- 172.21.15.83:0/3439736848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48800cd3d0 0x7f48800b4400 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:24.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.362+0000 7f4885d3b640 1 -- 172.21.15.83:0/3439736848 >> 172.21.15.83:0/3439736848 conn(0x7f48800d17e0 msgr2=0x7f4880158d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:24.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.362+0000 7f4885d3b640 1 -- 172.21.15.83:0/3439736848 shutdown_connections 2024-04-17T10:48:24.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.362+0000 7f4885d3b640 1 -- 172.21.15.83:0/3439736848 wait complete. 2024-04-17T10:48:24.389 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-17T10:48:24.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.518+0000 7f94232eb640 1 Processor -- start 2024-04-17T10:48:24.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.518+0000 7f94232eb640 1 -- start start 2024-04-17T10:48:24.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.518+0000 7f94232eb640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f941c0ce400 0x7f941c0b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:24.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.518+0000 7f94232eb640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f941c0b5450 0x7f941c0b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:24.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.518+0000 7f94232eb640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f941c15e4d0 con 0x7f941c15e0f0 2024-04-17T10:48:24.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.518+0000 7f94232eb640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f941c162b40 con 0x7f941c0ce400 2024-04-17T10:48:24.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.518+0000 7f94232eb640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f941c162320 con 0x7f941c0b5450 2024-04-17T10:48:24.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.518+0000 7f9420886640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f941c0ce400 0x7f941c0b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:24.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.518+0000 7f9421087640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f941c15e0f0 0x7f941c161c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51246/0 (socket says 172.21.15.83:51246) 2024-04-17T10:48:24.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.518+0000 7f9421087640 1 -- 172.21.15.83:0/643159380 learned_addr learned my addr 172.21.15.83:0/643159380 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:24.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.522+0000 7f9420886640 1 -- 172.21.15.83:0/643159380 >> v2:172.21.15.167:3300/0 conn(0x7f941c0b5450 msgr2=0x7f941c0b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:24.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.522+0000 7f9420886640 1 --2- 172.21.15.83:0/643159380 >> v2:172.21.15.167:3300/0 conn(0x7f941c0b5450 0x7f941c0b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:24.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.522+0000 7f9420886640 1 -- 172.21.15.83:0/643159380 >> v1:172.21.15.83:6789/0 conn(0x7f941c15e0f0 legacy=0x7f941c161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:24.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.522+0000 7f9420886640 1 -- 172.21.15.83:0/643159380 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941c162580 con 0x7f941c0ce400 2024-04-17T10:48:24.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.522+0000 7f9413fff640 1 -- 172.21.15.83:0/643159380 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1703665082 0 0) 0x7f941c15e4d0 con 0x7f941c15e0f0 2024-04-17T10:48:24.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.522+0000 7f9420886640 1 --2- 172.21.15.83:0/643159380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f941c0ce400 0x7f941c0b6d00 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f941c0d11a0 tx=0x7f940c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9655e626e99ce206 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:24.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.522+0000 7f9413fff640 1 -- 172.21.15.83:0/643159380 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c0b0070 con 0x7f941c0ce400 2024-04-17T10:48:24.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.522+0000 7f9413fff640 1 -- 172.21.15.83:0/643159380 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940c0ab600 con 0x7f941c0ce400 2024-04-17T10:48:24.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.522+0000 7f9413fff640 1 -- 172.21.15.83:0/643159380 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c0a4e60 con 0x7f941c0ce400 2024-04-17T10:48:24.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.522+0000 7f94232eb640 1 -- 172.21.15.83:0/643159380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f941c0ce400 msgr2=0x7f941c0b6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:24.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.522+0000 7f94232eb640 1 --2- 172.21.15.83:0/643159380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f941c0ce400 0x7f941c0b6d00 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f941c0d11a0 tx=0x7f940c01c3d0 comp rx=0 tx=0).stop 2024-04-17T10:48:24.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.522+0000 7f94232eb640 1 -- 172.21.15.83:0/643159380 shutdown_connections 2024-04-17T10:48:24.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.522+0000 7f94232eb640 1 --2- 172.21.15.83:0/643159380 >> v2:172.21.15.167:3300/0 conn(0x7f941c0b5450 0x7f941c0b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:24.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.522+0000 7f94232eb640 1 --2- 172.21.15.83:0/643159380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f941c0ce400 0x7f941c0b6d00 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:24.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.522+0000 7f94232eb640 1 -- 172.21.15.83:0/643159380 >> 172.21.15.83:0/643159380 conn(0x7f941c0d0450 msgr2=0x7f941c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:24.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.522+0000 7f94232eb640 1 -- 172.21.15.83:0/643159380 shutdown_connections 2024-04-17T10:48:24.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.522+0000 7f94232eb640 1 -- 172.21.15.83:0/643159380 wait complete. 2024-04-17T10:48:24.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.526+0000 7f94232eb640 1 Processor -- start 2024-04-17T10:48:24.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.526+0000 7f94232eb640 1 -- start start 2024-04-17T10:48:24.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.526+0000 7f94232eb640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f941c0b5450 0x7f941c153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:24.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.526+0000 7f94232eb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f941c0ce400 0x7f941c160fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:24.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.526+0000 7f94232eb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f941c15e0f0 0x7f941c15f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:24.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.526+0000 7f94232eb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941c0d2ed0 con 0x7f941c0ce400 2024-04-17T10:48:24.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.526+0000 7f94232eb640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941c0d2d50 con 0x7f941c15e0f0 2024-04-17T10:48:24.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.526+0000 7f94232eb640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f941c0d3050 con 0x7f941c0b5450 2024-04-17T10:48:24.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.526+0000 7f9420886640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f941c0ce400 0x7f941c160fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:24.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.526+0000 7f9420886640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f941c0ce400 0x7f941c160fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42504/0 (socket says 172.21.15.83:42504) 2024-04-17T10:48:24.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.526+0000 7f9420886640 1 -- 172.21.15.83:0/603103266 learned_addr learned my addr 172.21.15.83:0/603103266 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:24.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.526+0000 7f9420886640 1 -- 172.21.15.83:0/603103266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f941c0b5450 msgr2=0x7f941c153ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:24.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.526+0000 7f9421087640 1 --2- 172.21.15.83:0/603103266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f941c0b5450 0x7f941c153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:24.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.526+0000 7f9421888640 1 --2- 172.21.15.83:0/603103266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f941c15e0f0 0x7f941c15f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:24.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.526+0000 7f9420886640 1 --2- 172.21.15.83:0/603103266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f941c0b5450 0x7f941c153ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:24.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.526+0000 7f9420886640 1 -- 172.21.15.83:0/603103266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f941c15e0f0 msgr2=0x7f941c15f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:24.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.526+0000 7f9420886640 1 --2- 172.21.15.83:0/603103266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f941c15e0f0 0x7f941c15f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:24.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.526+0000 7f9420886640 1 -- 172.21.15.83:0/603103266 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941c1614e0 con 0x7f941c0ce400 2024-04-17T10:48:24.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.526+0000 7f9421888640 1 --2- 172.21.15.83:0/603103266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f941c15e0f0 0x7f941c15f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:24.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.526+0000 7f9420886640 1 --2- 172.21.15.83:0/603103266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f941c0ce400 0x7f941c160fa0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f940c0ae040 tx=0x7f940c01ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:24.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.526+0000 7f9411ffb640 1 -- 172.21.15.83:0/603103266 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c0b0050 con 0x7f941c0ce400 2024-04-17T10:48:24.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.526+0000 7f9411ffb640 1 -- 172.21.15.83:0/603103266 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940c0bc070 con 0x7f941c0ce400 2024-04-17T10:48:24.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.526+0000 7f94232eb640 1 -- 172.21.15.83:0/603103266 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f941c15e8a0 con 0x7f941c0ce400 2024-04-17T10:48:24.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.526+0000 7f9411ffb640 1 -- 172.21.15.83:0/603103266 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c0b5850 con 0x7f941c0ce400 2024-04-17T10:48:24.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.526+0000 7f94232eb640 1 -- 172.21.15.83:0/603103266 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f941c1536c0 con 0x7f941c0ce400 2024-04-17T10:48:24.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.526+0000 7f94232eb640 1 -- 172.21.15.83:0/603103266 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f941c0681a0 con 0x7f941c0ce400 2024-04-17T10:48:24.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.526+0000 7f9411ffb640 1 -- 172.21.15.83:0/603103266 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f940c0b59f0 con 0x7f941c0ce400 2024-04-17T10:48:24.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.530+0000 7f9411ffb640 1 --2- 172.21.15.83:0/603103266 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9400066f60 0x7f9400069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:24.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.530+0000 7f9411ffb640 1 -- 172.21.15.83:0/603103266 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f940c11f710 con 0x7f941c0ce400 2024-04-17T10:48:24.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.530+0000 7f9421087640 1 --2- 172.21.15.83:0/603103266 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9400066f60 0x7f9400069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:24.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.530+0000 7f9421087640 1 --2- 172.21.15.83:0/603103266 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9400066f60 0x7f9400069420 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9418000f70 tx=0x7f9418009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:24.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.534+0000 7f9411ffb640 1 -- 172.21.15.83:0/603103266 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f940c016050 con 0x7f941c0ce400 2024-04-17T10:48:24.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:24.730+0000 7f94232eb640 1 -- 172.21.15.83:0/603103266 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f941c074f90 con 0x7f941c0ce400 2024-04-17T10:48:25.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.402+0000 7f9411ffb640 1 -- 172.21.15.83:0/603103266 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v67) v1 ==== 174+0+0 (secure 0 0 0) 0x7f940c0f0810 con 0x7f941c0ce400 2024-04-17T10:48:25.403 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 16 and data pool 17 2024-04-17T10:48:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.410+0000 7f94232eb640 1 -- 172.21.15.83:0/603103266 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9400066f60 msgr2=0x7f9400069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.410+0000 7f94232eb640 1 --2- 172.21.15.83:0/603103266 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9400066f60 0x7f9400069420 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9418000f70 tx=0x7f9418009040 comp rx=0 tx=0).stop 2024-04-17T10:48:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.410+0000 7f94232eb640 1 -- 172.21.15.83:0/603103266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f941c0ce400 msgr2=0x7f941c160fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.410+0000 7f94232eb640 1 --2- 172.21.15.83:0/603103266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f941c0ce400 0x7f941c160fa0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f940c0ae040 tx=0x7f940c01ac10 comp rx=0 tx=0).stop 2024-04-17T10:48:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.410+0000 7f94232eb640 1 -- 172.21.15.83:0/603103266 shutdown_connections 2024-04-17T10:48:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.410+0000 7f94232eb640 1 --2- 172.21.15.83:0/603103266 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9400066f60 0x7f9400069420 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.410+0000 7f94232eb640 1 --2- 172.21.15.83:0/603103266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f941c15e0f0 0x7f941c15f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.410+0000 7f94232eb640 1 --2- 172.21.15.83:0/603103266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f941c0ce400 0x7f941c160fa0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.410+0000 7f94232eb640 1 --2- 172.21.15.83:0/603103266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f941c0b5450 0x7f941c153ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.410+0000 7f94232eb640 1 -- 172.21.15.83:0/603103266 >> 172.21.15.83:0/603103266 conn(0x7f941c0d0450 msgr2=0x7f941c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:25.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.410+0000 7f94232eb640 1 -- 172.21.15.83:0/603103266 shutdown_connections 2024-04-17T10:48:25.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.410+0000 7f94232eb640 1 -- 172.21.15.83:0/603103266 wait complete. 2024-04-17T10:48:25.438 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:48:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.558+0000 7f04a9adc640 1 Processor -- start 2024-04-17T10:48:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.558+0000 7f04a9adc640 1 -- start start 2024-04-17T10:48:25.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.562+0000 7f04a9adc640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f04a415db50 0x7f04a4070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:25.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.562+0000 7f04a9adc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04a40aedd0 0x7f04a406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:25.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.562+0000 7f04a9adc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f04a406dae0 0x7f04a40cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:25.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.562+0000 7f04a9adc640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04a40709b0 con 0x7f04a406dae0 2024-04-17T10:48:25.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.562+0000 7f04a9adc640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04a415e440 con 0x7f04a415db50 2024-04-17T10:48:25.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.562+0000 7f04a9adc640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04a415e720 con 0x7f04a40aedd0 2024-04-17T10:48:25.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.562+0000 7f04a3fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f04a406dae0 0x7f04a40cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:25.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.562+0000 7f04a3fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f04a406dae0 0x7f04a40cf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42514/0 (socket says 172.21.15.83:42514) 2024-04-17T10:48:25.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.562+0000 7f04a3fff640 1 -- 172.21.15.83:0/2713209893 learned_addr learned my addr 172.21.15.83:0/2713209893 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:25.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.562+0000 7f04a3fff640 1 -- 172.21.15.83:0/2713209893 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04a40aedd0 msgr2=0x7f04a406d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:25.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.562+0000 7f04a2ffd640 1 --2- 172.21.15.83:0/2713209893 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04a40aedd0 0x7f04a406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:25.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.562+0000 7f04a37fe640 1 --2- 172.21.15.83:0/2713209893 >> v2:172.21.15.167:3300/0 conn(0x7f04a415db50 0x7f04a4070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:25.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.562+0000 7f04a3fff640 1 --2- 172.21.15.83:0/2713209893 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04a40aedd0 0x7f04a406d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:25.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.562+0000 7f04a3fff640 1 -- 172.21.15.83:0/2713209893 >> v2:172.21.15.167:3300/0 conn(0x7f04a415db50 msgr2=0x7f04a4070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:25.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.562+0000 7f04a3fff640 1 --2- 172.21.15.83:0/2713209893 >> v2:172.21.15.167:3300/0 conn(0x7f04a415db50 0x7f04a4070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:25.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.562+0000 7f04a3fff640 1 -- 172.21.15.83:0/2713209893 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a415ea00 con 0x7f04a406dae0 2024-04-17T10:48:25.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.562+0000 7f04a3fff640 1 --2- 172.21.15.83:0/2713209893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04a406dae0 0x7f04a40cf550 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f04a406e960 tx=0x7f049c01c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e9a123f8ff4db1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:25.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.562+0000 7f04a27fc640 1 -- 172.21.15.83:0/2713209893 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c0a64b0 con 0x7f04a406dae0 2024-04-17T10:48:25.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.562+0000 7f04a27fc640 1 -- 172.21.15.83:0/2713209893 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f049c0a6650 con 0x7f04a406dae0 2024-04-17T10:48:25.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.562+0000 7f04a27fc640 1 -- 172.21.15.83:0/2713209893 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c0a6a20 con 0x7f04a406dae0 2024-04-17T10:48:25.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.562+0000 7f04a9adc640 1 -- 172.21.15.83:0/2713209893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04a406dae0 msgr2=0x7f04a40cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:25.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.562+0000 7f04a9adc640 1 --2- 172.21.15.83:0/2713209893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04a406dae0 0x7f04a40cf550 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f04a406e960 tx=0x7f049c01c730 comp rx=0 tx=0).stop 2024-04-17T10:48:25.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.562+0000 7f04a9adc640 1 -- 172.21.15.83:0/2713209893 shutdown_connections 2024-04-17T10:48:25.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.562+0000 7f04a9adc640 1 --2- 172.21.15.83:0/2713209893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04a406dae0 0x7f04a40cf550 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:25.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.562+0000 7f04a9adc640 1 --2- 172.21.15.83:0/2713209893 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04a40aedd0 0x7f04a406d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:25.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.562+0000 7f04a9adc640 1 --2- 172.21.15.83:0/2713209893 >> v2:172.21.15.167:3300/0 conn(0x7f04a415db50 0x7f04a4070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:25.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.562+0000 7f04a9adc640 1 -- 172.21.15.83:0/2713209893 >> 172.21.15.83:0/2713209893 conn(0x7f04a406abe0 msgr2=0x7f04a406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:25.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.562+0000 7f04a9adc640 1 -- 172.21.15.83:0/2713209893 shutdown_connections 2024-04-17T10:48:25.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.562+0000 7f04a9adc640 1 -- 172.21.15.83:0/2713209893 wait complete. 2024-04-17T10:48:25.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.566+0000 7f04a9adc640 1 Processor -- start 2024-04-17T10:48:25.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.566+0000 7f04a9adc640 1 -- start start 2024-04-17T10:48:25.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.566+0000 7f04a9adc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04a406dae0 0x7f04a4157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:25.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.566+0000 7f04a9adc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04a40aedd0 0x7f04a4159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:25.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.566+0000 7f04a9adc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04a415db50 0x7f04a415a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:25.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.566+0000 7f04a9adc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a406c610 con 0x7f04a406dae0 2024-04-17T10:48:25.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.566+0000 7f04a9adc640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a406c490 con 0x7f04a415db50 2024-04-17T10:48:25.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.566+0000 7f04a9adc640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04a406c790 con 0x7f04a40aedd0 2024-04-17T10:48:25.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.566+0000 7f04a37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04a406dae0 0x7f04a4157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:25.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.566+0000 7f04a37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04a406dae0 0x7f04a4157e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42524/0 (socket says 172.21.15.83:42524) 2024-04-17T10:48:25.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.566+0000 7f04a37fe640 1 -- 172.21.15.83:0/2323380381 learned_addr learned my addr 172.21.15.83:0/2323380381 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:25.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.566+0000 7f04a2ffd640 1 --2- 172.21.15.83:0/2323380381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04a40aedd0 0x7f04a4159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:25.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.566+0000 7f04a3fff640 1 --2- 172.21.15.83:0/2323380381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04a415db50 0x7f04a415a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:25.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.566+0000 7f04a37fe640 1 -- 172.21.15.83:0/2323380381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04a40aedd0 msgr2=0x7f04a4159d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:25.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.566+0000 7f04a37fe640 1 --2- 172.21.15.83:0/2323380381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04a40aedd0 0x7f04a4159d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:25.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.566+0000 7f04a37fe640 1 -- 172.21.15.83:0/2323380381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04a415db50 msgr2=0x7f04a415a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:25.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.566+0000 7f04a37fe640 1 --2- 172.21.15.83:0/2323380381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04a415db50 0x7f04a415a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:25.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.566+0000 7f04a37fe640 1 -- 172.21.15.83:0/2323380381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a415a7b0 con 0x7f04a406dae0 2024-04-17T10:48:25.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.566+0000 7f04a37fe640 1 --2- 172.21.15.83:0/2323380381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04a406dae0 0x7f04a4157e30 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f049400c7f0 tx=0x7f049400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:25.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.566+0000 7f048bfff640 1 -- 172.21.15.83:0/2323380381 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0494015070 con 0x7f04a406dae0 2024-04-17T10:48:25.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.566+0000 7f048bfff640 1 -- 172.21.15.83:0/2323380381 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0494007820 con 0x7f04a406dae0 2024-04-17T10:48:25.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.566+0000 7f04a9adc640 1 -- 172.21.15.83:0/2323380381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04a413fb60 con 0x7f04a406dae0 2024-04-17T10:48:25.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.566+0000 7f048bfff640 1 -- 172.21.15.83:0/2323380381 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0494007c10 con 0x7f04a406dae0 2024-04-17T10:48:25.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.566+0000 7f04a9adc640 1 -- 172.21.15.83:0/2323380381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a4167ae0 con 0x7f04a406dae0 2024-04-17T10:48:25.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.566+0000 7f04a9adc640 1 -- 172.21.15.83:0/2323380381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04a4074f90 con 0x7f04a406dae0 2024-04-17T10:48:25.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.566+0000 7f048bfff640 1 -- 172.21.15.83:0/2323380381 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0494018070 con 0x7f04a406dae0 2024-04-17T10:48:25.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.570+0000 7f048bfff640 1 --2- 172.21.15.83:0/2323380381 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0490066f90 0x7f0490069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:25.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.570+0000 7f048bfff640 1 -- 172.21.15.83:0/2323380381 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0494084ea0 con 0x7f04a406dae0 2024-04-17T10:48:25.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.570+0000 7f04a2ffd640 1 --2- 172.21.15.83:0/2323380381 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0490066f90 0x7f0490069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:25.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.570+0000 7f04a2ffd640 1 --2- 172.21.15.83:0/2323380381 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0490066f90 0x7f0490069450 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f04a4154200 tx=0x7f0498008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:25.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.574+0000 7f048bfff640 1 -- 172.21.15.83:0/2323380381 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0494055e20 con 0x7f04a406dae0 2024-04-17T10:48:25.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.730+0000 7f04a9adc640 1 -- 172.21.15.83:0/2323380381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04a4071320 con 0x7f04a406dae0 2024-04-17T10:48:25.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.734+0000 7f048bfff640 1 -- 172.21.15.83:0/2323380381 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+16568 (secure 0 0 0) 0x7f0494055720 con 0x7f04a406dae0 2024-04-17T10:48:25.729 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:25.729 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":113,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:48:24.738025+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:48:22.981915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-17T10:48:23.700138+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:48:25.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.734+0000 7f04a9adc640 1 -- 172.21.15.83:0/2323380381 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0490066f90 msgr2=0x7f0490069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:25.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.734+0000 7f04a9adc640 1 --2- 172.21.15.83:0/2323380381 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0490066f90 0x7f0490069450 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f04a4154200 tx=0x7f0498008040 comp rx=0 tx=0).stop 2024-04-17T10:48:25.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.734+0000 7f04a9adc640 1 -- 172.21.15.83:0/2323380381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04a406dae0 msgr2=0x7f04a4157e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:25.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.734+0000 7f04a9adc640 1 --2- 172.21.15.83:0/2323380381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04a406dae0 0x7f04a4157e30 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f049400c7f0 tx=0x7f049400ccb0 comp rx=0 tx=0).stop 2024-04-17T10:48:25.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.734+0000 7f04a9adc640 1 -- 172.21.15.83:0/2323380381 shutdown_connections 2024-04-17T10:48:25.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.734+0000 7f04a9adc640 1 --2- 172.21.15.83:0/2323380381 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0490066f90 0x7f0490069450 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:25.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.734+0000 7f04a9adc640 1 --2- 172.21.15.83:0/2323380381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04a415db50 0x7f04a415a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:25.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.734+0000 7f04a9adc640 1 --2- 172.21.15.83:0/2323380381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04a40aedd0 0x7f04a4159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:25.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.734+0000 7f04a9adc640 1 --2- 172.21.15.83:0/2323380381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04a406dae0 0x7f04a4157e30 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:25.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.734+0000 7f04a9adc640 1 -- 172.21.15.83:0/2323380381 >> 172.21.15.83:0/2323380381 conn(0x7f04a406abe0 msgr2=0x7f04a4070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:25.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.734+0000 7f04a9adc640 1 -- 172.21.15.83:0/2323380381 shutdown_connections 2024-04-17T10:48:25.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.734+0000 7f04a9adc640 1 -- 172.21.15.83:0/2323380381 wait complete. 2024-04-17T10:48:25.749 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:48:25.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.874+0000 7f41ff4db640 1 Processor -- start 2024-04-17T10:48:25.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.874+0000 7f41ff4db640 1 -- start start 2024-04-17T10:48:25.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.878+0000 7f41ff4db640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f41f8072330 0x7f41f80cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:25.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.878+0000 7f41ff4db640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f41f80cfa10 0x7f41f806c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:25.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.878+0000 7f41ff4db640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41f80cfea0 con 0x7f41f815ba30 2024-04-17T10:48:25.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.878+0000 7f41ff4db640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41f8071ea0 con 0x7f41f8072330 2024-04-17T10:48:25.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.878+0000 7f41ff4db640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41f806e320 con 0x7f41f80cfa10 2024-04-17T10:48:25.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.878+0000 7f41fca76640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f41f8072330 0x7f41f80cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:25.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.878+0000 7f41fca76640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f41f8072330 0x7f41f80cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42528/0 (socket says 172.21.15.83:42528) 2024-04-17T10:48:25.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.878+0000 7f41fca76640 1 -- 172.21.15.83:0/3639106837 learned_addr learned my addr 172.21.15.83:0/3639106837 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:25.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.878+0000 7f41fda78640 1 --2- 172.21.15.83:0/3639106837 >> v2:172.21.15.167:3300/0 conn(0x7f41f80cfa10 0x7f41f806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:25.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.878+0000 7f41fca76640 1 -- 172.21.15.83:0/3639106837 >> v2:172.21.15.167:3300/0 conn(0x7f41f80cfa10 msgr2=0x7f41f806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:25.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.878+0000 7f41fca76640 1 --2- 172.21.15.83:0/3639106837 >> v2:172.21.15.167:3300/0 conn(0x7f41f80cfa10 0x7f41f806c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:25.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.878+0000 7f41fca76640 1 -- 172.21.15.83:0/3639106837 >> v1:172.21.15.167:6789/0 conn(0x7f41f815ba30 legacy=0x7f41f815c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:25.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.878+0000 7f41fca76640 1 -- 172.21.15.83:0/3639106837 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f806e040 con 0x7f41f8072330 2024-04-17T10:48:25.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.878+0000 7f41fda78640 1 --2- 172.21.15.83:0/3639106837 >> v2:172.21.15.167:3300/0 conn(0x7f41f80cfa10 0x7f41f806c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:25.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.878+0000 7f41fca76640 1 --2- 172.21.15.83:0/3639106837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41f8072330 0x7f41f80cf4d0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f41f80676a0 tx=0x7f41e801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7204b8cc5fefcdd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:25.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.878+0000 7f41effff640 1 -- 172.21.15.83:0/3639106837 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e80b2070 con 0x7f41f8072330 2024-04-17T10:48:25.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.878+0000 7f41effff640 1 -- 172.21.15.83:0/3639106837 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e80a9070 con 0x7f41f8072330 2024-04-17T10:48:25.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.878+0000 7f41effff640 1 -- 172.21.15.83:0/3639106837 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e80ad880 con 0x7f41f8072330 2024-04-17T10:48:25.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.882+0000 7f41ff4db640 1 -- 172.21.15.83:0/3639106837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41f8072330 msgr2=0x7f41f80cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:25.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.882+0000 7f41ff4db640 1 --2- 172.21.15.83:0/3639106837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41f8072330 0x7f41f80cf4d0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f41f80676a0 tx=0x7f41e801c450 comp rx=0 tx=0).stop 2024-04-17T10:48:25.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.882+0000 7f41ff4db640 1 -- 172.21.15.83:0/3639106837 shutdown_connections 2024-04-17T10:48:25.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.882+0000 7f41ff4db640 1 --2- 172.21.15.83:0/3639106837 >> v2:172.21.15.167:3300/0 conn(0x7f41f80cfa10 0x7f41f806c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:25.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.882+0000 7f41ff4db640 1 --2- 172.21.15.83:0/3639106837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41f8072330 0x7f41f80cf4d0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:25.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.882+0000 7f41ff4db640 1 -- 172.21.15.83:0/3639106837 >> 172.21.15.83:0/3639106837 conn(0x7f41f806abe0 msgr2=0x7f41f806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:25.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.882+0000 7f41ff4db640 1 -- 172.21.15.83:0/3639106837 shutdown_connections 2024-04-17T10:48:25.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.882+0000 7f41ff4db640 1 -- 172.21.15.83:0/3639106837 wait complete. 2024-04-17T10:48:25.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.882+0000 7f41ff4db640 1 Processor -- start 2024-04-17T10:48:25.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.882+0000 7f41ff4db640 1 -- start start 2024-04-17T10:48:25.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.882+0000 7f41ff4db640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41f8072330 0x7f41f8156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:25.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.882+0000 7f41ff4db640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41f80cfa10 0x7f41f80b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:25.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.882+0000 7f41ff4db640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41f815ba30 0x7f41f80b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:25.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.882+0000 7f41ff4db640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f806de20 con 0x7f41f8072330 2024-04-17T10:48:25.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.882+0000 7f41ff4db640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f806dca0 con 0x7f41f815ba30 2024-04-17T10:48:25.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.882+0000 7f41ff4db640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f8168520 con 0x7f41f80cfa10 2024-04-17T10:48:25.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.882+0000 7f41fd277640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41f8072330 0x7f41f8156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:25.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.882+0000 7f41fd277640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41f8072330 0x7f41f8156930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42532/0 (socket says 172.21.15.83:42532) 2024-04-17T10:48:25.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.882+0000 7f41fd277640 1 -- 172.21.15.83:0/4153425143 learned_addr learned my addr 172.21.15.83:0/4153425143 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:25.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.882+0000 7f41fda78640 1 --2- 172.21.15.83:0/4153425143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41f815ba30 0x7f41f80b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:25.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.882+0000 7f41fca76640 1 --2- 172.21.15.83:0/4153425143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41f80cfa10 0x7f41f80b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:25.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.882+0000 7f41fd277640 1 -- 172.21.15.83:0/4153425143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41f80cfa10 msgr2=0x7f41f80b34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:25.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.882+0000 7f41fd277640 1 --2- 172.21.15.83:0/4153425143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41f80cfa10 0x7f41f80b34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:25.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.882+0000 7f41fd277640 1 -- 172.21.15.83:0/4153425143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41f815ba30 msgr2=0x7f41f80b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:25.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.882+0000 7f41fd277640 1 --2- 172.21.15.83:0/4153425143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41f815ba30 0x7f41f80b39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:25.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.882+0000 7f41fd277640 1 -- 172.21.15.83:0/4153425143 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f80cf000 con 0x7f41f8072330 2024-04-17T10:48:25.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.882+0000 7f41fd277640 1 --2- 172.21.15.83:0/4153425143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41f8072330 0x7f41f8156930 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f41f400c450 tx=0x7f41f400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:25.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.882+0000 7f41edffb640 1 -- 172.21.15.83:0/4153425143 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f4011790 con 0x7f41f8072330 2024-04-17T10:48:25.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.882+0000 7f41edffb640 1 -- 172.21.15.83:0/4153425143 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f4017070 con 0x7f41f8072330 2024-04-17T10:48:25.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.882+0000 7f41ff4db640 1 -- 172.21.15.83:0/4153425143 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f80b29c0 con 0x7f41f8072330 2024-04-17T10:48:25.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.882+0000 7f41edffb640 1 -- 172.21.15.83:0/4153425143 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f4011b90 con 0x7f41f8072330 2024-04-17T10:48:25.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.882+0000 7f41ff4db640 1 -- 172.21.15.83:0/4153425143 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f80b2f80 con 0x7f41f8072330 2024-04-17T10:48:25.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.886+0000 7f41ff4db640 1 -- 172.21.15.83:0/4153425143 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41f8074f90 con 0x7f41f8072330 2024-04-17T10:48:25.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.886+0000 7f41edffb640 1 -- 172.21.15.83:0/4153425143 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f41f4011d30 con 0x7f41f8072330 2024-04-17T10:48:25.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.886+0000 7f41edffb640 1 --2- 172.21.15.83:0/4153425143 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f41dc066f60 0x7f41dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:25.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.886+0000 7f41edffb640 1 -- 172.21.15.83:0/4153425143 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f41f4085980 con 0x7f41f8072330 2024-04-17T10:48:25.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.886+0000 7f41fca76640 1 --2- 172.21.15.83:0/4153425143 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f41dc066f60 0x7f41dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:25.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.886+0000 7f41fca76640 1 --2- 172.21.15.83:0/4153425143 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f41dc066f60 0x7f41dc069420 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f41e80af000 tx=0x7f41e80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:25.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:25.890+0000 7f41edffb640 1 -- 172.21.15.83:0/4153425143 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f41f4056a00 con 0x7f41f8072330 2024-04-17T10:48:26.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.046+0000 7f41ff4db640 1 -- 172.21.15.83:0/4153425143 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41f80b2770 con 0x7f41f8072330 2024-04-17T10:48:26.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.050+0000 7f41edffb640 1 -- 172.21.15.83:0/4153425143 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+16568 (secure 0 0 0) 0x7f41f4056300 con 0x7f41f8072330 2024-04-17T10:48:26.046 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:26.046 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":113,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:48:24.738025+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:48:22.981915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-17T10:48:23.700138+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:48:26.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.054+0000 7f41ff4db640 1 -- 172.21.15.83:0/4153425143 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f41dc066f60 msgr2=0x7f41dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:26.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.054+0000 7f41ff4db640 1 --2- 172.21.15.83:0/4153425143 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f41dc066f60 0x7f41dc069420 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f41e80af000 tx=0x7f41e80a7000 comp rx=0 tx=0).stop 2024-04-17T10:48:26.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.054+0000 7f41ff4db640 1 -- 172.21.15.83:0/4153425143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41f8072330 msgr2=0x7f41f8156930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:26.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.054+0000 7f41ff4db640 1 --2- 172.21.15.83:0/4153425143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41f8072330 0x7f41f8156930 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f41f400c450 tx=0x7f41f400c910 comp rx=0 tx=0).stop 2024-04-17T10:48:26.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.054+0000 7f41ff4db640 1 -- 172.21.15.83:0/4153425143 shutdown_connections 2024-04-17T10:48:26.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.054+0000 7f41ff4db640 1 --2- 172.21.15.83:0/4153425143 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f41dc066f60 0x7f41dc069420 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:26.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.054+0000 7f41ff4db640 1 --2- 172.21.15.83:0/4153425143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41f815ba30 0x7f41f80b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:26.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.054+0000 7f41ff4db640 1 --2- 172.21.15.83:0/4153425143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41f80cfa10 0x7f41f80b34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:26.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.054+0000 7f41ff4db640 1 --2- 172.21.15.83:0/4153425143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41f8072330 0x7f41f8156930 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:26.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.054+0000 7f41ff4db640 1 -- 172.21.15.83:0/4153425143 >> 172.21.15.83:0/4153425143 conn(0x7f41f806abe0 msgr2=0x7f41f80b04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:26.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.054+0000 7f41ff4db640 1 -- 172.21.15.83:0/4153425143 shutdown_connections 2024-04-17T10:48:26.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.054+0000 7f41ff4db640 1 -- 172.21.15.83:0/4153425143 wait complete. 2024-04-17T10:48:26.067 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-17T10:48:26.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.190+0000 7fe98194e640 1 Processor -- start 2024-04-17T10:48:26.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.190+0000 7fe98194e640 1 -- start start 2024-04-17T10:48:26.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.190+0000 7fe98194e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe97c15b6e0 0x7fe97c071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:26.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.190+0000 7fe98194e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe97c070470 0x7fe97c06ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:26.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.190+0000 7fe98194e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe97c06c1f0 con 0x7fe97c15bac0 2024-04-17T10:48:26.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.190+0000 7fe98194e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe97c070a10 con 0x7fe97c15b6e0 2024-04-17T10:48:26.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.190+0000 7fe98194e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c070cf0 con 0x7fe97c070470 2024-04-17T10:48:26.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.190+0000 7fe97affd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe97c15b6e0 0x7fe97c071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:26.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.190+0000 7fe97affd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe97c15b6e0 0x7fe97c071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42540/0 (socket says 172.21.15.83:42540) 2024-04-17T10:48:26.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.190+0000 7fe97affd640 1 -- 172.21.15.83:0/2377708852 learned_addr learned my addr 172.21.15.83:0/2377708852 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:26.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.190+0000 7fe97b7fe640 1 --2- 172.21.15.83:0/2377708852 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe97c070470 0x7fe97c06ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:26.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.190+0000 7fe97affd640 1 -- 172.21.15.83:0/2377708852 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe97c070470 msgr2=0x7fe97c06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:26.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.190+0000 7fe97affd640 1 --2- 172.21.15.83:0/2377708852 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe97c070470 0x7fe97c06ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:26.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.190+0000 7fe97affd640 1 -- 172.21.15.83:0/2377708852 >> v1:172.21.15.83:6789/0 conn(0x7fe97c15bac0 legacy=0x7fe97c0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:26.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.190+0000 7fe97affd640 1 -- 172.21.15.83:0/2377708852 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe97c070fd0 con 0x7fe97c15b6e0 2024-04-17T10:48:26.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.190+0000 7fe979ffb640 1 -- 172.21.15.83:0/2377708852 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4263254172 0 0) 0x7fe97c06c1f0 con 0x7fe97c15bac0 2024-04-17T10:48:26.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.190+0000 7fe97b7fe640 1 --2- 172.21.15.83:0/2377708852 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe97c070470 0x7fe97c06ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:26.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.194+0000 7fe97affd640 1 --2- 172.21.15.83:0/2377708852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe97c15b6e0 0x7fe97c071cc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fe974009870 tx=0x7fe97401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af10ecf444285cf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:26.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.194+0000 7fe979ffb640 1 -- 172.21.15.83:0/2377708852 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9740b0070 con 0x7fe97c15b6e0 2024-04-17T10:48:26.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.194+0000 7fe979ffb640 1 -- 172.21.15.83:0/2377708852 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9740ab600 con 0x7fe97c15b6e0 2024-04-17T10:48:26.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.194+0000 7fe979ffb640 1 -- 172.21.15.83:0/2377708852 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9740ab9d0 con 0x7fe97c15b6e0 2024-04-17T10:48:26.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.194+0000 7fe98194e640 1 -- 172.21.15.83:0/2377708852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe97c15b6e0 msgr2=0x7fe97c071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:26.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.194+0000 7fe98194e640 1 --2- 172.21.15.83:0/2377708852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe97c15b6e0 0x7fe97c071cc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fe974009870 tx=0x7fe97401c4d0 comp rx=0 tx=0).stop 2024-04-17T10:48:26.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.194+0000 7fe98194e640 1 -- 172.21.15.83:0/2377708852 shutdown_connections 2024-04-17T10:48:26.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.194+0000 7fe98194e640 1 --2- 172.21.15.83:0/2377708852 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe97c070470 0x7fe97c06ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:26.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.194+0000 7fe98194e640 1 --2- 172.21.15.83:0/2377708852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe97c15b6e0 0x7fe97c071cc0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:26.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.194+0000 7fe98194e640 1 -- 172.21.15.83:0/2377708852 >> 172.21.15.83:0/2377708852 conn(0x7fe97c0b15b0 msgr2=0x7fe97c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:26.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.194+0000 7fe98194e640 1 -- 172.21.15.83:0/2377708852 shutdown_connections 2024-04-17T10:48:26.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.194+0000 7fe98194e640 1 -- 172.21.15.83:0/2377708852 wait complete. 2024-04-17T10:48:26.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.194+0000 7fe98194e640 1 Processor -- start 2024-04-17T10:48:26.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.194+0000 7fe98194e640 1 -- start start 2024-04-17T10:48:26.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.194+0000 7fe98194e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe97c070470 0x7fe97c148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:26.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.194+0000 7fe98194e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe97c15b6e0 0x7fe97c1445e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:26.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.194+0000 7fe98194e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe97c15bac0 0x7fe97c1439b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:26.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.194+0000 7fe98194e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c06d2b0 con 0x7fe97c15bac0 2024-04-17T10:48:26.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.194+0000 7fe98194e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c06d130 con 0x7fe97c070470 2024-04-17T10:48:26.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.194+0000 7fe98194e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c06b760 con 0x7fe97c15b6e0 2024-04-17T10:48:26.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.194+0000 7fe97b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe97c15bac0 0x7fe97c1439b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:26.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.194+0000 7fe97b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe97c15bac0 0x7fe97c1439b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42550/0 (socket says 172.21.15.83:42550) 2024-04-17T10:48:26.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.194+0000 7fe97b7fe640 1 -- 172.21.15.83:0/3533446730 learned_addr learned my addr 172.21.15.83:0/3533446730 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:26.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.194+0000 7fe97a7fc640 1 --2- 172.21.15.83:0/3533446730 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe97c15b6e0 0x7fe97c1445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:26.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.194+0000 7fe97b7fe640 1 -- 172.21.15.83:0/3533446730 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe97c15b6e0 msgr2=0x7fe97c1445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:26.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.194+0000 7fe97affd640 1 --2- 172.21.15.83:0/3533446730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe97c070470 0x7fe97c148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:26.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.194+0000 7fe97b7fe640 1 --2- 172.21.15.83:0/3533446730 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe97c15b6e0 0x7fe97c1445e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:26.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.194+0000 7fe97b7fe640 1 -- 172.21.15.83:0/3533446730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe97c070470 msgr2=0x7fe97c148710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:26.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.194+0000 7fe97b7fe640 1 --2- 172.21.15.83:0/3533446730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe97c070470 0x7fe97c148710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:26.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.198+0000 7fe97b7fe640 1 -- 172.21.15.83:0/3533446730 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe97c14a160 con 0x7fe97c15bac0 2024-04-17T10:48:26.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.198+0000 7fe97b7fe640 1 --2- 172.21.15.83:0/3533446730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe97c15bac0 0x7fe97c1439b0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00cd50 tx=0x7fe96c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:26.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.198+0000 7fe973fff640 1 -- 172.21.15.83:0/3533446730 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c00f680 con 0x7fe97c15bac0 2024-04-17T10:48:26.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.198+0000 7fe973fff640 1 -- 172.21.15.83:0/3533446730 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe96c00f840 con 0x7fe97c15bac0 2024-04-17T10:48:26.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.198+0000 7fe98194e640 1 -- 172.21.15.83:0/3533446730 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe97c14a440 con 0x7fe97c15bac0 2024-04-17T10:48:26.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.198+0000 7fe973fff640 1 -- 172.21.15.83:0/3533446730 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c00fc30 con 0x7fe97c15bac0 2024-04-17T10:48:26.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.198+0000 7fe98194e640 1 -- 172.21.15.83:0/3533446730 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe97c15dba0 con 0x7fe97c15bac0 2024-04-17T10:48:26.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.198+0000 7fe98194e640 1 -- 172.21.15.83:0/3533446730 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe97c15dda0 con 0x7fe97c15bac0 2024-04-17T10:48:26.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.198+0000 7fe973fff640 1 -- 172.21.15.83:0/3533446730 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe96c0213f0 con 0x7fe97c15bac0 2024-04-17T10:48:26.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.198+0000 7fe973fff640 1 --2- 172.21.15.83:0/3533446730 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe95c066f60 0x7fe95c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:26.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.198+0000 7fe973fff640 1 -- 172.21.15.83:0/3533446730 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe96c085de0 con 0x7fe97c15bac0 2024-04-17T10:48:26.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.198+0000 7fe97affd640 1 --2- 172.21.15.83:0/3533446730 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe95c066f60 0x7fe95c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:26.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.198+0000 7fe97affd640 1 --2- 172.21.15.83:0/3533446730 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe95c066f60 0x7fe95c069420 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe9740ad000 tx=0x7fe974008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:26.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.202+0000 7fe973fff640 1 -- 172.21.15.83:0/3533446730 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe97c15dda0 con 0x7fe97c15bac0 2024-04-17T10:48:26.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.382+0000 7fe98194e640 1 -- 172.21.15.83:0/3533446730 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe97c074f90 con 0x7fe97c15bac0 2024-04-17T10:48:26.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.426+0000 7fe973fff640 1 -- 172.21.15.83:0/3533446730 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v68) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe96c014030 con 0x7fe97c15bac0 2024-04-17T10:48:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.430+0000 7fe98194e640 1 -- 172.21.15.83:0/3533446730 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe95c066f60 msgr2=0x7fe95c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.430+0000 7fe98194e640 1 --2- 172.21.15.83:0/3533446730 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe95c066f60 0x7fe95c069420 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe9740ad000 tx=0x7fe974008040 comp rx=0 tx=0).stop 2024-04-17T10:48:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.430+0000 7fe98194e640 1 -- 172.21.15.83:0/3533446730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe97c15bac0 msgr2=0x7fe97c1439b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:26.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.430+0000 7fe98194e640 1 --2- 172.21.15.83:0/3533446730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe97c15bac0 0x7fe97c1439b0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00cd50 tx=0x7fe96c00a650 comp rx=0 tx=0).stop 2024-04-17T10:48:26.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.430+0000 7fe98194e640 1 -- 172.21.15.83:0/3533446730 shutdown_connections 2024-04-17T10:48:26.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.430+0000 7fe98194e640 1 --2- 172.21.15.83:0/3533446730 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe95c066f60 0x7fe95c069420 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:26.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.430+0000 7fe98194e640 1 --2- 172.21.15.83:0/3533446730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe97c15bac0 0x7fe97c1439b0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:26.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.430+0000 7fe98194e640 1 --2- 172.21.15.83:0/3533446730 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe97c15b6e0 0x7fe97c1445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:26.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.430+0000 7fe98194e640 1 --2- 172.21.15.83:0/3533446730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe97c070470 0x7fe97c148710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:26.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.430+0000 7fe98194e640 1 -- 172.21.15.83:0/3533446730 >> 172.21.15.83:0/3533446730 conn(0x7fe97c0b15b0 msgr2=0x7fe97c0ce7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:26.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.430+0000 7fe98194e640 1 -- 172.21.15.83:0/3533446730 shutdown_connections 2024-04-17T10:48:26.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.430+0000 7fe98194e640 1 -- 172.21.15.83:0/3533446730 wait complete. 2024-04-17T10:48:26.442 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-17T10:48:26.442 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-17T10:48:26.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.562+0000 7f996c13a640 1 Processor -- start 2024-04-17T10:48:26.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.562+0000 7f996c13a640 1 -- start start 2024-04-17T10:48:26.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.566+0000 7f996c13a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f996415e330 0x7f996415e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:26.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.566+0000 7f996c13a640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9964070aa0 0x7f9964070f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:26.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.566+0000 7f996c13a640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99641625d0 con 0x7f996406ae10 2024-04-17T10:48:26.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.566+0000 7f996c13a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9964161810 con 0x7f996415e330 2024-04-17T10:48:26.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.566+0000 7f996c13a640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9964161af0 con 0x7f9964070aa0 2024-04-17T10:48:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.566+0000 7f99696d5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f996415e330 0x7f996415e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.566+0000 7f99696d5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f996415e330 0x7f996415e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42562/0 (socket says 172.21.15.83:42562) 2024-04-17T10:48:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.566+0000 7f99696d5640 1 -- 172.21.15.83:0/2085327736 learned_addr learned my addr 172.21.15.83:0/2085327736 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.566+0000 7f99696d5640 1 -- 172.21.15.83:0/2085327736 >> v2:172.21.15.167:3300/0 conn(0x7f9964070aa0 msgr2=0x7f9964070f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.566+0000 7f99696d5640 1 --2- 172.21.15.83:0/2085327736 >> v2:172.21.15.167:3300/0 conn(0x7f9964070aa0 0x7f9964070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.566+0000 7f99696d5640 1 -- 172.21.15.83:0/2085327736 >> v1:172.21.15.167:6789/0 conn(0x7f996406ae10 legacy=0x7f996406b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.566+0000 7f99696d5640 1 -- 172.21.15.83:0/2085327736 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9964161dd0 con 0x7f996415e330 2024-04-17T10:48:26.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.566+0000 7f99696d5640 1 --2- 172.21.15.83:0/2085327736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f996415e330 0x7f996415e7a0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f996415bc70 tx=0x7f995401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc16e14493e741aa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:26.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.566+0000 7f9968ed4640 1 -- 172.21.15.83:0/2085327736 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99540b0070 con 0x7f996415e330 2024-04-17T10:48:26.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.566+0000 7f9968ed4640 1 -- 172.21.15.83:0/2085327736 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99540ab600 con 0x7f996415e330 2024-04-17T10:48:26.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.566+0000 7f9968ed4640 1 -- 172.21.15.83:0/2085327736 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99540ab9d0 con 0x7f996415e330 2024-04-17T10:48:26.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.566+0000 7f996c13a640 1 -- 172.21.15.83:0/2085327736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f996415e330 msgr2=0x7f996415e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:26.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.566+0000 7f996c13a640 1 --2- 172.21.15.83:0/2085327736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f996415e330 0x7f996415e7a0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f996415bc70 tx=0x7f995401c390 comp rx=0 tx=0).stop 2024-04-17T10:48:26.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.570+0000 7f996c13a640 1 -- 172.21.15.83:0/2085327736 shutdown_connections 2024-04-17T10:48:26.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.570+0000 7f996c13a640 1 --2- 172.21.15.83:0/2085327736 >> v2:172.21.15.167:3300/0 conn(0x7f9964070aa0 0x7f9964070f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:26.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.570+0000 7f996c13a640 1 --2- 172.21.15.83:0/2085327736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f996415e330 0x7f996415e7a0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:26.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.570+0000 7f996c13a640 1 -- 172.21.15.83:0/2085327736 >> 172.21.15.83:0/2085327736 conn(0x7f99640b15b0 msgr2=0x7f99640b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:26.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.570+0000 7f996c13a640 1 -- 172.21.15.83:0/2085327736 shutdown_connections 2024-04-17T10:48:26.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.570+0000 7f996c13a640 1 -- 172.21.15.83:0/2085327736 wait complete. 2024-04-17T10:48:26.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.570+0000 7f996c13a640 1 Processor -- start 2024-04-17T10:48:26.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.570+0000 7f996c13a640 1 -- start start 2024-04-17T10:48:26.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.570+0000 7f996c13a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f996406ae10 0x7f9964144000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:26.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.570+0000 7f996c13a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9964070aa0 0x7f996415ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:26.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.570+0000 7f996c13a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f996415e330 0x7f996415c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:26.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.570+0000 7f996c13a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996416ab90 con 0x7f9964070aa0 2024-04-17T10:48:26.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.570+0000 7f996c13a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996416aa10 con 0x7f996406ae10 2024-04-17T10:48:26.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.570+0000 7f996c13a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f996416ad10 con 0x7f996415e330 2024-04-17T10:48:26.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.570+0000 7f99696d5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9964070aa0 0x7f996415ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:26.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.570+0000 7f99696d5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9964070aa0 0x7f996415ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42566/0 (socket says 172.21.15.83:42566) 2024-04-17T10:48:26.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.570+0000 7f99696d5640 1 -- 172.21.15.83:0/1188163913 learned_addr learned my addr 172.21.15.83:0/1188163913 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:26.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.570+0000 7f996a6d7640 1 --2- 172.21.15.83:0/1188163913 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f996415e330 0x7f996415c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:26.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.570+0000 7f9969ed6640 1 --2- 172.21.15.83:0/1188163913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f996406ae10 0x7f9964144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:26.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.570+0000 7f99696d5640 1 -- 172.21.15.83:0/1188163913 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f996415e330 msgr2=0x7f996415c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:26.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.570+0000 7f99696d5640 1 --2- 172.21.15.83:0/1188163913 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f996415e330 0x7f996415c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:26.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.570+0000 7f99696d5640 1 -- 172.21.15.83:0/1188163913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f996406ae10 msgr2=0x7f9964144000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:26.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.570+0000 7f99696d5640 1 --2- 172.21.15.83:0/1188163913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f996406ae10 0x7f9964144000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:26.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.570+0000 7f99696d5640 1 -- 172.21.15.83:0/1188163913 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99641595e0 con 0x7f9964070aa0 2024-04-17T10:48:26.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.570+0000 7f99696d5640 1 --2- 172.21.15.83:0/1188163913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9964070aa0 0x7f996415ac40 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f99540ae040 tx=0x7f9954019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:26.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.570+0000 7f995a7fc640 1 -- 172.21.15.83:0/1188163913 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99540b0050 con 0x7f9964070aa0 2024-04-17T10:48:26.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.570+0000 7f995a7fc640 1 -- 172.21.15.83:0/1188163913 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99540bc070 con 0x7f9964070aa0 2024-04-17T10:48:26.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.570+0000 7f996c13a640 1 -- 172.21.15.83:0/1188163913 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99641598c0 con 0x7f9964070aa0 2024-04-17T10:48:26.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.570+0000 7f995a7fc640 1 -- 172.21.15.83:0/1188163913 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99540b5a50 con 0x7f9964070aa0 2024-04-17T10:48:26.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.570+0000 7f996c13a640 1 -- 172.21.15.83:0/1188163913 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9964159e80 con 0x7f9964070aa0 2024-04-17T10:48:26.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.570+0000 7f996c13a640 1 -- 172.21.15.83:0/1188163913 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9964074f90 con 0x7f9964070aa0 2024-04-17T10:48:26.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.574+0000 7f995a7fc640 1 -- 172.21.15.83:0/1188163913 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f99540b5bf0 con 0x7f9964070aa0 2024-04-17T10:48:26.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.574+0000 7f995a7fc640 1 --2- 172.21.15.83:0/1188163913 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9948066f90 0x7f9948069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:26.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.574+0000 7f995a7fc640 1 -- 172.21.15.83:0/1188163913 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f995411f8c0 con 0x7f9964070aa0 2024-04-17T10:48:26.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.574+0000 7f9969ed6640 1 --2- 172.21.15.83:0/1188163913 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9948066f90 0x7f9948069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:26.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.574+0000 7f9969ed6640 1 --2- 172.21.15.83:0/1188163913 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9948066f90 0x7f9948069450 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f996000b810 tx=0x7f9960008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:26.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.578+0000 7f995a7fc640 1 -- 172.21.15.83:0/1188163913 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f99540160d0 con 0x7f9964070aa0 2024-04-17T10:48:26.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:26.770+0000 7f996c13a640 1 -- 172.21.15.83:0/1188163913 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9964071d90 con 0x7f9964070aa0 2024-04-17T10:48:27.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.430+0000 7f995a7fc640 1 -- 172.21.15.83:0/1188163913 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v69) v1 ==== 122+0+0 (secure 0 0 0) 0x7f99540f0960 con 0x7f9964070aa0 2024-04-17T10:48:27.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.434+0000 7f996c13a640 1 -- 172.21.15.83:0/1188163913 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9948066f90 msgr2=0x7f9948069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:27.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.434+0000 7f996c13a640 1 --2- 172.21.15.83:0/1188163913 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9948066f90 0x7f9948069450 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f996000b810 tx=0x7f9960008040 comp rx=0 tx=0).stop 2024-04-17T10:48:27.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.434+0000 7f996c13a640 1 -- 172.21.15.83:0/1188163913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9964070aa0 msgr2=0x7f996415ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:27.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.434+0000 7f996c13a640 1 --2- 172.21.15.83:0/1188163913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9964070aa0 0x7f996415ac40 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f99540ae040 tx=0x7f9954019130 comp rx=0 tx=0).stop 2024-04-17T10:48:27.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.434+0000 7f996c13a640 1 -- 172.21.15.83:0/1188163913 shutdown_connections 2024-04-17T10:48:27.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.434+0000 7f996c13a640 1 --2- 172.21.15.83:0/1188163913 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9948066f90 0x7f9948069450 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:27.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.434+0000 7f996c13a640 1 --2- 172.21.15.83:0/1188163913 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f996415e330 0x7f996415c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:27.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.434+0000 7f996c13a640 1 --2- 172.21.15.83:0/1188163913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9964070aa0 0x7f996415ac40 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:27.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.434+0000 7f996c13a640 1 --2- 172.21.15.83:0/1188163913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f996406ae10 0x7f9964144000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:27.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.434+0000 7f996c13a640 1 -- 172.21.15.83:0/1188163913 >> 172.21.15.83:0/1188163913 conn(0x7f99640b15b0 msgr2=0x7f996406b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:27.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.434+0000 7f996c13a640 1 -- 172.21.15.83:0/1188163913 shutdown_connections 2024-04-17T10:48:27.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.434+0000 7f996c13a640 1 -- 172.21.15.83:0/1188163913 wait complete. 2024-04-17T10:48:27.451 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-17T10:48:27.451 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:48:27.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.578+0000 7f7e1997c640 1 Processor -- start 2024-04-17T10:48:27.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.578+0000 7f7e1997c640 1 -- start start 2024-04-17T10:48:27.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e1997c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7e1406dcb0 0x7f7e14165650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:27.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e1997c640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e1415e440 con 0x7f7e140aedd0 2024-04-17T10:48:27.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e1997c640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e1415e720 con 0x7f7e1415db50 2024-04-17T10:48:27.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e1997c640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e1415ea00 con 0x7f7e1406dcb0 2024-04-17T10:48:27.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e0bfff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7e140aedd0 0x7f7e1406d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40928/0 (socket says 172.21.15.83:40928) 2024-04-17T10:48:27.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e0bfff640 1 -- 172.21.15.83:0/2199026687 learned_addr learned my addr 172.21.15.83:0/2199026687 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:27.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e127fc640 1 -- 172.21.15.83:0/2199026687 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1580165106 0 0) 0x7f7e1415e440 con 0x7f7e140aedd0 2024-04-17T10:48:27.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e127fc640 1 -- 172.21.15.83:0/2199026687 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e00003530 con 0x7f7e140aedd0 2024-04-17T10:48:27.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e137fe640 1 --2- 172.21.15.83:0/2199026687 >> v2:172.21.15.167:3300/0 conn(0x7f7e1406dcb0 0x7f7e14165650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:27.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e127fc640 1 -- 172.21.15.83:0/2199026687 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 847879253 0 0) 0x7f7e00003530 con 0x7f7e140aedd0 2024-04-17T10:48:27.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e127fc640 1 -- 172.21.15.83:0/2199026687 >> v2:172.21.15.167:3300/0 conn(0x7f7e1406dcb0 msgr2=0x7f7e14165650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:27.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e127fc640 1 --2- 172.21.15.83:0/2199026687 >> v2:172.21.15.167:3300/0 conn(0x7f7e1406dcb0 0x7f7e14165650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:27.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e127fc640 1 -- 172.21.15.83:0/2199026687 >> v1:172.21.15.167:6789/0 conn(0x7f7e1415db50 legacy=0x7f7e14070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:27.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e127fc640 1 -- 172.21.15.83:0/2199026687 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e1415ece0 con 0x7f7e140aedd0 2024-04-17T10:48:27.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e127fc640 1 -- 172.21.15.83:0/2199026687 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f7dfc001680 con 0x7f7e140aedd0 2024-04-17T10:48:27.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e137fe640 1 --2- 172.21.15.83:0/2199026687 >> v2:172.21.15.167:3300/0 conn(0x7f7e1406dcb0 0x7f7e14165650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:48:27.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e127fc640 1 -- 172.21.15.83:0/2199026687 >> v1:172.21.15.83:6789/0 conn(0x7f7e140aedd0 legacy=0x7f7e1406d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:27.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e127fc640 1 --2- 172.21.15.83:0/2199026687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e000039f0 0x7f7e00003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:27.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e127fc640 1 --2- 172.21.15.83:0/2199026687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e00004810 0x7f7e00004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:27.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e127fc640 1 --2- 172.21.15.83:0/2199026687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e00005220 0x7f7e0000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:27.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e127fc640 1 -- 172.21.15.83:0/2199026687 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1415ea00 con 0x7f7e000039f0 2024-04-17T10:48:27.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e127fc640 1 -- 172.21.15.83:0/2199026687 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0000c150 con 0x7f7e00004810 2024-04-17T10:48:27.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e127fc640 1 -- 172.21.15.83:0/2199026687 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0000c350 con 0x7f7e00005220 2024-04-17T10:48:27.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e12ffd640 1 --2- 172.21.15.83:0/2199026687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e000039f0 0x7f7e00003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:27.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e137fe640 1 --2- 172.21.15.83:0/2199026687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e00005220 0x7f7e0000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:27.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e0bfff640 1 --2- 172.21.15.83:0/2199026687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e00004810 0x7f7e00004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:27.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e12ffd640 1 -- 172.21.15.83:0/2199026687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e00005220 msgr2=0x7f7e0000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:27.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e12ffd640 1 --2- 172.21.15.83:0/2199026687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e00005220 0x7f7e0000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:27.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e12ffd640 1 -- 172.21.15.83:0/2199026687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e00004810 msgr2=0x7f7e00004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:27.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e12ffd640 1 --2- 172.21.15.83:0/2199026687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e00004810 0x7f7e00004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:27.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e12ffd640 1 -- 172.21.15.83:0/2199026687 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e00003530 con 0x7f7e000039f0 2024-04-17T10:48:27.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e137fe640 1 --2- 172.21.15.83:0/2199026687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e00005220 0x7f7e0000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:27.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e0bfff640 1 --2- 172.21.15.83:0/2199026687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e00004810 0x7f7e00004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:27.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.582+0000 7f7e137fe640 1 -- 172.21.15.83:0/2199026687 reap_dead start 2024-04-17T10:48:27.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.586+0000 7f7e12ffd640 1 --2- 172.21.15.83:0/2199026687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e000039f0 0x7f7e00003e10 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f7e04001750 tx=0x7f7e0401c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d19761010c808c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:27.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.586+0000 7f7e127fc640 1 -- 172.21.15.83:0/2199026687 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e040013a0 con 0x7f7e000039f0 2024-04-17T10:48:27.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.586+0000 7f7e127fc640 1 -- 172.21.15.83:0/2199026687 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e040050d0 con 0x7f7e000039f0 2024-04-17T10:48:27.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.586+0000 7f7e127fc640 1 -- 172.21.15.83:0/2199026687 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e04003600 con 0x7f7e000039f0 2024-04-17T10:48:27.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.586+0000 7f7e1997c640 1 -- 172.21.15.83:0/2199026687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e000039f0 msgr2=0x7f7e00003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:27.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.586+0000 7f7e1997c640 1 --2- 172.21.15.83:0/2199026687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e000039f0 0x7f7e00003e10 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f7e04001750 tx=0x7f7e0401c370 comp rx=0 tx=0).stop 2024-04-17T10:48:27.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.586+0000 7f7e1997c640 1 -- 172.21.15.83:0/2199026687 shutdown_connections 2024-04-17T10:48:27.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.586+0000 7f7e1997c640 1 --2- 172.21.15.83:0/2199026687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e000039f0 0x7f7e00003e10 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:27.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.586+0000 7f7e1997c640 1 -- 172.21.15.83:0/2199026687 >> 172.21.15.83:0/2199026687 conn(0x7f7e1406abe0 msgr2=0x7f7e1406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:27.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.586+0000 7f7e1997c640 1 -- 172.21.15.83:0/2199026687 shutdown_connections 2024-04-17T10:48:27.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.586+0000 7f7e1997c640 1 -- 172.21.15.83:0/2199026687 wait complete. 2024-04-17T10:48:27.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.586+0000 7f7e1997c640 1 Processor -- start 2024-04-17T10:48:27.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.586+0000 7f7e1997c640 1 -- start start 2024-04-17T10:48:27.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.586+0000 7f7e1997c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e000039f0 0x7f7e141543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:27.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.586+0000 7f7e1997c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e141401c0 0x7f7e140b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:27.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.586+0000 7f7e1997c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e14140640 0x7f7e140b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:27.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.586+0000 7f7e1997c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1415e0b0 con 0x7f7e000039f0 2024-04-17T10:48:27.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.586+0000 7f7e1997c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1415df30 con 0x7f7e141401c0 2024-04-17T10:48:27.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.586+0000 7f7e1997c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e140cfd60 con 0x7f7e14140640 2024-04-17T10:48:27.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.586+0000 7f7e12ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e000039f0 0x7f7e141543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:27.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.586+0000 7f7e12ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e000039f0 0x7f7e141543c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44618/0 (socket says 172.21.15.83:44618) 2024-04-17T10:48:27.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.586+0000 7f7e12ffd640 1 -- 172.21.15.83:0/73987129 learned_addr learned my addr 172.21.15.83:0/73987129 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:27.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.586+0000 7f7e0bfff640 1 --2- 172.21.15.83:0/73987129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e141401c0 0x7f7e140b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:27.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.586+0000 7f7e137fe640 1 --2- 172.21.15.83:0/73987129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e14140640 0x7f7e140b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:27.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.586+0000 7f7e12ffd640 1 -- 172.21.15.83:0/73987129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e14140640 msgr2=0x7f7e140b0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:27.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.586+0000 7f7e12ffd640 1 --2- 172.21.15.83:0/73987129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e14140640 0x7f7e140b0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:27.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.586+0000 7f7e12ffd640 1 -- 172.21.15.83:0/73987129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e141401c0 msgr2=0x7f7e140b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:27.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.586+0000 7f7e12ffd640 1 --2- 172.21.15.83:0/73987129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e141401c0 0x7f7e140b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:27.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.586+0000 7f7e12ffd640 1 -- 172.21.15.83:0/73987129 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e14140f50 con 0x7f7e000039f0 2024-04-17T10:48:27.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.586+0000 7f7e0bfff640 1 --2- 172.21.15.83:0/73987129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e141401c0 0x7f7e140b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:27.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.586+0000 7f7e12ffd640 1 --2- 172.21.15.83:0/73987129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e000039f0 0x7f7e141543c0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f7e04001750 tx=0x7f7e00005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:27.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.590+0000 7f7e10ff9640 1 -- 172.21.15.83:0/73987129 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e04007040 con 0x7f7e000039f0 2024-04-17T10:48:27.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.590+0000 7f7e10ff9640 1 -- 172.21.15.83:0/73987129 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e040bb070 con 0x7f7e000039f0 2024-04-17T10:48:27.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.590+0000 7f7e1997c640 1 -- 172.21.15.83:0/73987129 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e14141230 con 0x7f7e000039f0 2024-04-17T10:48:27.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.590+0000 7f7e10ff9640 1 -- 172.21.15.83:0/73987129 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e04007400 con 0x7f7e000039f0 2024-04-17T10:48:27.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.590+0000 7f7e1997c640 1 -- 172.21.15.83:0/73987129 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e141417f0 con 0x7f7e000039f0 2024-04-17T10:48:27.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.590+0000 7f7e1997c640 1 -- 172.21.15.83:0/73987129 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e14074f90 con 0x7f7e000039f0 2024-04-17T10:48:27.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.590+0000 7f7e10ff9640 1 -- 172.21.15.83:0/73987129 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7e040b4040 con 0x7f7e000039f0 2024-04-17T10:48:27.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.590+0000 7f7e10ff9640 1 --2- 172.21.15.83:0/73987129 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7df4066f60 0x7f7df4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:27.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.590+0000 7f7e10ff9640 1 -- 172.21.15.83:0/73987129 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e0411ed80 con 0x7f7e000039f0 2024-04-17T10:48:27.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.590+0000 7f7e0bfff640 1 --2- 172.21.15.83:0/73987129 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7df4066f60 0x7f7df4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:27.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.590+0000 7f7e0bfff640 1 --2- 172.21.15.83:0/73987129 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7df4066f60 0x7f7df4069420 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc001210 tx=0x7f7dfc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:27.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.594+0000 7f7e10ff9640 1 -- 172.21.15.83:0/73987129 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7e040160b0 con 0x7f7e000039f0 2024-04-17T10:48:27.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.774+0000 7f7e1997c640 1 -- 172.21.15.83:0/73987129 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e140cbdb0 con 0x7f7e000039f0 2024-04-17T10:48:27.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.774+0000 7f7e10ff9640 1 -- 172.21.15.83:0/73987129 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f7e040efe20 con 0x7f7e000039f0 2024-04-17T10:48:27.771 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:27.771 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5786,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1553533838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1553533838},{"type":"v1","addr":"172.21.15.167:6833","nonce":1553533838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/674536913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":674536913},{"type":"v1","addr":"172.21.15.167:6835","nonce":674536913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8598,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4183890457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4183890457},{"type":"v1","addr":"172.21.15.83:6835","nonce":4183890457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:48:24.738443+0000","modified":"2024-04-17T10:48:26.776273+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8604":{"gid":8604,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1479417998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1479417998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1479417998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-17T10:48:27.771 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 69 2024-04-17T10:48:27.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.778+0000 7f7e1997c640 1 -- 172.21.15.83:0/73987129 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7df4066f60 msgr2=0x7f7df4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:27.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.778+0000 7f7e1997c640 1 --2- 172.21.15.83:0/73987129 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7df4066f60 0x7f7df4069420 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc001210 tx=0x7f7dfc009040 comp rx=0 tx=0).stop 2024-04-17T10:48:27.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.778+0000 7f7e1997c640 1 -- 172.21.15.83:0/73987129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e000039f0 msgr2=0x7f7e141543c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:27.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.778+0000 7f7e1997c640 1 --2- 172.21.15.83:0/73987129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e000039f0 0x7f7e141543c0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f7e04001750 tx=0x7f7e00005840 comp rx=0 tx=0).stop 2024-04-17T10:48:27.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.778+0000 7f7e1997c640 1 -- 172.21.15.83:0/73987129 shutdown_connections 2024-04-17T10:48:27.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.778+0000 7f7e1997c640 1 --2- 172.21.15.83:0/73987129 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7df4066f60 0x7f7df4069420 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:27.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.778+0000 7f7e1997c640 1 --2- 172.21.15.83:0/73987129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e14140640 0x7f7e140b0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:27.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.778+0000 7f7e1997c640 1 --2- 172.21.15.83:0/73987129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e141401c0 0x7f7e140b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:27.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.778+0000 7f7e1997c640 1 --2- 172.21.15.83:0/73987129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e000039f0 0x7f7e141543c0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:27.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.778+0000 7f7e1997c640 1 -- 172.21.15.83:0/73987129 >> 172.21.15.83:0/73987129 conn(0x7f7e1406abe0 msgr2=0x7f7e140b3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:27.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.778+0000 7f7e1997c640 1 -- 172.21.15.83:0/73987129 shutdown_connections 2024-04-17T10:48:27.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.778+0000 7f7e1997c640 1 -- 172.21.15.83:0/73987129 wait complete. 2024-04-17T10:48:27.790 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:48:27.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac601cf640 1 Processor -- start 2024-04-17T10:48:27.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac601cf640 1 -- start start 2024-04-17T10:48:27.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac601cf640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fac580ce980 0x7fac580b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:27.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac601cf640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac58162a00 con 0x7fac580cee00 2024-04-17T10:48:27.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac601cf640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac58161e80 con 0x7fac5815dfa0 2024-04-17T10:48:27.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac601cf640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac58162160 con 0x7fac580ce980 2024-04-17T10:48:27.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac5e76c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fac580cee00 0x7fac580d1d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40944/0 (socket says 172.21.15.83:40944) 2024-04-17T10:48:27.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac5e76c640 1 -- 172.21.15.83:0/613603702 learned_addr learned my addr 172.21.15.83:0/613603702 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:27.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac5d76a640 1 --2- 172.21.15.83:0/613603702 >> v2:172.21.15.167:3300/0 conn(0x7fac580ce980 0x7fac580b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:27.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac5cf69640 1 -- 172.21.15.83:0/613603702 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1240828768 0 0) 0x7fac58162a00 con 0x7fac580cee00 2024-04-17T10:48:27.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac5cf69640 1 -- 172.21.15.83:0/613603702 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac44003530 con 0x7fac580cee00 2024-04-17T10:48:27.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac5cf69640 1 -- 172.21.15.83:0/613603702 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3230375580 0 0) 0x7fac44003530 con 0x7fac580cee00 2024-04-17T10:48:27.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac5cf69640 1 -- 172.21.15.83:0/613603702 >> v2:172.21.15.167:3300/0 conn(0x7fac580ce980 msgr2=0x7fac580b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:27.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac5cf69640 1 --2- 172.21.15.83:0/613603702 >> v2:172.21.15.167:3300/0 conn(0x7fac580ce980 0x7fac580b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:27.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac5cf69640 1 -- 172.21.15.83:0/613603702 >> v1:172.21.15.167:6789/0 conn(0x7fac5815dfa0 legacy=0x7fac580b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:27.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac5cf69640 1 -- 172.21.15.83:0/613603702 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac58162440 con 0x7fac580cee00 2024-04-17T10:48:27.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac5cf69640 1 -- 172.21.15.83:0/613603702 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fac50001a00 con 0x7fac580cee00 2024-04-17T10:48:27.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac5cf69640 1 -- 172.21.15.83:0/613603702 >> v1:172.21.15.83:6789/0 conn(0x7fac580cee00 legacy=0x7fac580d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:27.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac5cf69640 1 --2- 172.21.15.83:0/613603702 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac440039c0 0x7fac44003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:27.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac5cf69640 1 --2- 172.21.15.83:0/613603702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac44004810 0x7fac44004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:27.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac5cf69640 1 --2- 172.21.15.83:0/613603702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac440051d0 0x7fac4400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:27.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac5cf69640 1 -- 172.21.15.83:0/613603702 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac58162160 con 0x7fac44004810 2024-04-17T10:48:27.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac5cf69640 1 -- 172.21.15.83:0/613603702 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4400c170 con 0x7fac440051d0 2024-04-17T10:48:27.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac5cf69640 1 -- 172.21.15.83:0/613603702 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4400c370 con 0x7fac440039c0 2024-04-17T10:48:27.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac5d76a640 1 --2- 172.21.15.83:0/613603702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac44004810 0x7fac44004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:27.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac5e76c640 1 --2- 172.21.15.83:0/613603702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac440051d0 0x7fac4400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:27.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac5df6b640 1 --2- 172.21.15.83:0/613603702 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac440039c0 0x7fac44003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:27.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac5d76a640 1 -- 172.21.15.83:0/613603702 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac440039c0 msgr2=0x7fac44003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:27.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac5d76a640 1 --2- 172.21.15.83:0/613603702 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac440039c0 0x7fac44003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:27.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac5d76a640 1 -- 172.21.15.83:0/613603702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac440051d0 msgr2=0x7fac4400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:27.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac5d76a640 1 --2- 172.21.15.83:0/613603702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac440051d0 0x7fac4400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:27.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac5e76c640 1 -- 172.21.15.83:0/613603702 reap_dead start 2024-04-17T10:48:27.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.906+0000 7fac5d76a640 1 -- 172.21.15.83:0/613603702 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac44003530 con 0x7fac44004810 2024-04-17T10:48:27.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.910+0000 7fac5d76a640 1 --2- 172.21.15.83:0/613603702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac44004810 0x7fac44004c90 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fac54009ac0 tx=0x7fac5401ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5454acfeb501dbd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:27.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.910+0000 7fac5cf69640 1 -- 172.21.15.83:0/613603702 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac540b2070 con 0x7fac44004810 2024-04-17T10:48:27.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.910+0000 7fac5cf69640 1 -- 172.21.15.83:0/613603702 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac540a5590 con 0x7fac44004810 2024-04-17T10:48:27.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.910+0000 7fac5cf69640 1 -- 172.21.15.83:0/613603702 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac54009690 con 0x7fac44004810 2024-04-17T10:48:27.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.910+0000 7fac601cf640 1 -- 172.21.15.83:0/613603702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac44004810 msgr2=0x7fac44004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:27.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.910+0000 7fac601cf640 1 --2- 172.21.15.83:0/613603702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac44004810 0x7fac44004c90 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fac54009ac0 tx=0x7fac5401ca20 comp rx=0 tx=0).stop 2024-04-17T10:48:27.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.910+0000 7fac601cf640 1 -- 172.21.15.83:0/613603702 shutdown_connections 2024-04-17T10:48:27.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.910+0000 7fac601cf640 1 --2- 172.21.15.83:0/613603702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac44004810 0x7fac44004c90 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:27.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.910+0000 7fac601cf640 1 -- 172.21.15.83:0/613603702 >> 172.21.15.83:0/613603702 conn(0x7fac580d0600 msgr2=0x7fac580d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:27.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.910+0000 7fac601cf640 1 -- 172.21.15.83:0/613603702 shutdown_connections 2024-04-17T10:48:27.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.910+0000 7fac601cf640 1 -- 172.21.15.83:0/613603702 wait complete. 2024-04-17T10:48:27.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.914+0000 7fac601cf640 1 Processor -- start 2024-04-17T10:48:27.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.914+0000 7fac601cf640 1 -- start start 2024-04-17T10:48:27.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.914+0000 7fac601cf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac44004810 0x7fac5815e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:27.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.914+0000 7fac601cf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac58160d40 0x7fac580c6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:27.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.914+0000 7fac601cf640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac581611c0 0x7fac580c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:27.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.914+0000 7fac601cf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac580b71b0 con 0x7fac44004810 2024-04-17T10:48:27.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.914+0000 7fac601cf640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac580b7030 con 0x7fac58160d40 2024-04-17T10:48:27.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.914+0000 7fac601cf640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac580d1d30 con 0x7fac581611c0 2024-04-17T10:48:27.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.914+0000 7fac5df6b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac44004810 0x7fac5815e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:27.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.914+0000 7fac5df6b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac44004810 0x7fac5815e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44642/0 (socket says 172.21.15.83:44642) 2024-04-17T10:48:27.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.914+0000 7fac5df6b640 1 -- 172.21.15.83:0/3469679295 learned_addr learned my addr 172.21.15.83:0/3469679295 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:27.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.914+0000 7fac5d76a640 1 --2- 172.21.15.83:0/3469679295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac58160d40 0x7fac580c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:27.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.914+0000 7fac5df6b640 1 -- 172.21.15.83:0/3469679295 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac581611c0 msgr2=0x7fac580c6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:27.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.914+0000 7fac5e76c640 1 --2- 172.21.15.83:0/3469679295 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac581611c0 0x7fac580c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:27.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.914+0000 7fac5df6b640 1 --2- 172.21.15.83:0/3469679295 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac581611c0 0x7fac580c6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:27.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.914+0000 7fac5df6b640 1 -- 172.21.15.83:0/3469679295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac58160d40 msgr2=0x7fac580c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:27.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.914+0000 7fac5df6b640 1 --2- 172.21.15.83:0/3469679295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac58160d40 0x7fac580c6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:27.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.914+0000 7fac5df6b640 1 -- 172.21.15.83:0/3469679295 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac5815ed80 con 0x7fac44004810 2024-04-17T10:48:27.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.914+0000 7fac5d76a640 1 --2- 172.21.15.83:0/3469679295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac58160d40 0x7fac580c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:27.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.914+0000 7fac5df6b640 1 --2- 172.21.15.83:0/3469679295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac44004810 0x7fac5815e610 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fac4800e4c0 tx=0x7fac4800e980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:27.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.914+0000 7fac4e7fc640 1 -- 172.21.15.83:0/3469679295 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac480052e0 con 0x7fac44004810 2024-04-17T10:48:27.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.914+0000 7fac4e7fc640 1 -- 172.21.15.83:0/3469679295 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac48018070 con 0x7fac44004810 2024-04-17T10:48:27.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.914+0000 7fac601cf640 1 -- 172.21.15.83:0/3469679295 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac5815f060 con 0x7fac44004810 2024-04-17T10:48:27.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.914+0000 7fac4e7fc640 1 -- 172.21.15.83:0/3469679295 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac48003840 con 0x7fac44004810 2024-04-17T10:48:27.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.914+0000 7fac601cf640 1 -- 172.21.15.83:0/3469679295 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac5815f620 con 0x7fac44004810 2024-04-17T10:48:27.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.914+0000 7fac601cf640 1 -- 172.21.15.83:0/3469679295 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac580681a0 con 0x7fac44004810 2024-04-17T10:48:27.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.914+0000 7fac4e7fc640 1 -- 172.21.15.83:0/3469679295 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fac480039e0 con 0x7fac44004810 2024-04-17T10:48:27.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.918+0000 7fac4e7fc640 1 --2- 172.21.15.83:0/3469679295 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fac3c066f60 0x7fac3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:27.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.918+0000 7fac4e7fc640 1 -- 172.21.15.83:0/3469679295 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac48084c40 con 0x7fac44004810 2024-04-17T10:48:27.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.918+0000 7fac5d76a640 1 --2- 172.21.15.83:0/3469679295 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fac3c066f60 0x7fac3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:27.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.918+0000 7fac5d76a640 1 --2- 172.21.15.83:0/3469679295 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fac3c066f60 0x7fac3c069420 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fac540af000 tx=0x7fac54008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:27.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:27.922+0000 7fac4e7fc640 1 -- 172.21.15.83:0/3469679295 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fac48055d10 con 0x7fac44004810 2024-04-17T10:48:28.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.078+0000 7fac601cf640 1 -- 172.21.15.83:0/3469679295 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac58074f90 con 0x7fac44004810 2024-04-17T10:48:28.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.078+0000 7fac4e7fc640 1 -- 172.21.15.83:0/3469679295 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+16555 (secure 0 0 0) 0x7fac48005480 con 0x7fac44004810 2024-04-17T10:48:28.076 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:28.076 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":114,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:48:26.365134+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:48:22.981915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-17T10:48:23.700138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:48:28.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.082+0000 7fac601cf640 1 -- 172.21.15.83:0/3469679295 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fac3c066f60 msgr2=0x7fac3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:28.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.082+0000 7fac601cf640 1 --2- 172.21.15.83:0/3469679295 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fac3c066f60 0x7fac3c069420 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fac540af000 tx=0x7fac54008040 comp rx=0 tx=0).stop 2024-04-17T10:48:28.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.082+0000 7fac601cf640 1 -- 172.21.15.83:0/3469679295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac44004810 msgr2=0x7fac5815e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:28.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.082+0000 7fac601cf640 1 --2- 172.21.15.83:0/3469679295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac44004810 0x7fac5815e610 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fac4800e4c0 tx=0x7fac4800e980 comp rx=0 tx=0).stop 2024-04-17T10:48:28.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.082+0000 7fac601cf640 1 -- 172.21.15.83:0/3469679295 shutdown_connections 2024-04-17T10:48:28.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.082+0000 7fac601cf640 1 --2- 172.21.15.83:0/3469679295 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fac3c066f60 0x7fac3c069420 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:28.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.082+0000 7fac601cf640 1 --2- 172.21.15.83:0/3469679295 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac581611c0 0x7fac580c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:28.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.082+0000 7fac601cf640 1 --2- 172.21.15.83:0/3469679295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac58160d40 0x7fac580c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:28.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.082+0000 7fac601cf640 1 --2- 172.21.15.83:0/3469679295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac44004810 0x7fac5815e610 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:28.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.082+0000 7fac601cf640 1 -- 172.21.15.83:0/3469679295 >> 172.21.15.83:0/3469679295 conn(0x7fac580d0600 msgr2=0x7fac580caa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:28.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.082+0000 7fac601cf640 1 -- 172.21.15.83:0/3469679295 shutdown_connections 2024-04-17T10:48:28.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.082+0000 7fac601cf640 1 -- 172.21.15.83:0/3469679295 wait complete. 2024-04-17T10:48:28.095 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-17T10:48:28.095 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:48:28.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.210+0000 7f9ee5ff6640 1 Processor -- start 2024-04-17T10:48:28.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.210+0000 7f9ee5ff6640 1 -- start start 2024-04-17T10:48:28.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.210+0000 7f9ee5ff6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ee015dfa0 0x7f9ee00b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:28.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.210+0000 7f9ee5ff6640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9ee00b5b40 0x7f9ee00d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:28.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.210+0000 7f9ee5ff6640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ee0162a00 con 0x7f9ee00ce980 2024-04-17T10:48:28.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.210+0000 7f9ee5ff6640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ee0161e80 con 0x7f9ee00b5b40 2024-04-17T10:48:28.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.210+0000 7f9ee5ff6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee0162160 con 0x7f9ee015dfa0 2024-04-17T10:48:28.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.210+0000 7f9edffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9ee00b5b40 0x7f9ee00d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:28.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.210+0000 7f9edffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9ee00b5b40 0x7f9ee00d1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44656/0 (socket says 172.21.15.83:44656) 2024-04-17T10:48:28.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.210+0000 7f9edffff640 1 -- 172.21.15.83:0/2157476530 learned_addr learned my addr 172.21.15.83:0/2157476530 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:28.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.210+0000 7f9edf7fe640 1 --2- 172.21.15.83:0/2157476530 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ee015dfa0 0x7f9ee00b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:28.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.210+0000 7f9edffff640 1 -- 172.21.15.83:0/2157476530 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ee015dfa0 msgr2=0x7f9ee00b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:28.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.210+0000 7f9edffff640 1 --2- 172.21.15.83:0/2157476530 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ee015dfa0 0x7f9ee00b6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:28.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.210+0000 7f9edffff640 1 -- 172.21.15.83:0/2157476530 >> v1:172.21.15.167:6789/0 conn(0x7f9ee00ce980 legacy=0x7f9ee00b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:28.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.210+0000 7f9edffff640 1 -- 172.21.15.83:0/2157476530 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee0162440 con 0x7f9ee00b5b40 2024-04-17T10:48:28.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.210+0000 7f9edf7fe640 1 --2- 172.21.15.83:0/2157476530 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ee015dfa0 0x7f9ee00b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:28.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.214+0000 7f9edffff640 1 --2- 172.21.15.83:0/2157476530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ee00b5b40 0x7f9ee00d1d30 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009e50 tx=0x7f9ed801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb550c424bdeb222 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:28.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.214+0000 7f9ede7fc640 1 -- 172.21.15.83:0/2157476530 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed80a56c0 con 0x7f9ee00b5b40 2024-04-17T10:48:28.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.214+0000 7f9ede7fc640 1 -- 172.21.15.83:0/2157476530 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed80a5860 con 0x7f9ee00b5b40 2024-04-17T10:48:28.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.214+0000 7f9ede7fc640 1 -- 172.21.15.83:0/2157476530 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed80a5c30 con 0x7f9ee00b5b40 2024-04-17T10:48:28.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.214+0000 7f9ee5ff6640 1 -- 172.21.15.83:0/2157476530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ee00b5b40 msgr2=0x7f9ee00d1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:28.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.214+0000 7f9ee5ff6640 1 --2- 172.21.15.83:0/2157476530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ee00b5b40 0x7f9ee00d1d30 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009e50 tx=0x7f9ed801c930 comp rx=0 tx=0).stop 2024-04-17T10:48:28.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.214+0000 7f9ee5ff6640 1 -- 172.21.15.83:0/2157476530 shutdown_connections 2024-04-17T10:48:28.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.214+0000 7f9ee5ff6640 1 --2- 172.21.15.83:0/2157476530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ee00b5b40 0x7f9ee00d1d30 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:28.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.214+0000 7f9ee5ff6640 1 --2- 172.21.15.83:0/2157476530 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ee015dfa0 0x7f9ee00b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:28.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.214+0000 7f9ee5ff6640 1 -- 172.21.15.83:0/2157476530 >> 172.21.15.83:0/2157476530 conn(0x7f9ee00d0600 msgr2=0x7f9ee00d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:28.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.214+0000 7f9ee5ff6640 1 -- 172.21.15.83:0/2157476530 shutdown_connections 2024-04-17T10:48:28.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.214+0000 7f9ee5ff6640 1 -- 172.21.15.83:0/2157476530 wait complete. 2024-04-17T10:48:28.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.214+0000 7f9ee5ff6640 1 Processor -- start 2024-04-17T10:48:28.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.214+0000 7f9ee5ff6640 1 -- start start 2024-04-17T10:48:28.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.214+0000 7f9ee5ff6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ee00b5b40 0x7f9ee014fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:28.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.214+0000 7f9ee5ff6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ee00ce980 0x7f9ee0154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:28.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.214+0000 7f9ee5ff6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ee015dfa0 0x7f9ee0151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:28.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.214+0000 7f9ee5ff6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee016ae50 con 0x7f9ee00b5b40 2024-04-17T10:48:28.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.214+0000 7f9ee5ff6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee0161b10 con 0x7f9ee015dfa0 2024-04-17T10:48:28.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.214+0000 7f9ee5ff6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee016afd0 con 0x7f9ee00ce980 2024-04-17T10:48:28.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.214+0000 7f9edf7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ee00b5b40 0x7f9ee014fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:28.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.214+0000 7f9edf7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ee00b5b40 0x7f9ee014fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44664/0 (socket says 172.21.15.83:44664) 2024-04-17T10:48:28.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.214+0000 7f9edf7fe640 1 -- 172.21.15.83:0/4166810570 learned_addr learned my addr 172.21.15.83:0/4166810570 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:28.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.214+0000 7f9edffff640 1 --2- 172.21.15.83:0/4166810570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ee015dfa0 0x7f9ee0151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:28.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.214+0000 7f9edf7fe640 1 -- 172.21.15.83:0/4166810570 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ee00ce980 msgr2=0x7f9ee0154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:28.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.214+0000 7f9edeffd640 1 --2- 172.21.15.83:0/4166810570 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ee00ce980 0x7f9ee0154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:28.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.214+0000 7f9edf7fe640 1 --2- 172.21.15.83:0/4166810570 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ee00ce980 0x7f9ee0154470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:28.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.214+0000 7f9edf7fe640 1 -- 172.21.15.83:0/4166810570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ee015dfa0 msgr2=0x7f9ee0151b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:28.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.214+0000 7f9edf7fe640 1 --2- 172.21.15.83:0/4166810570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ee015dfa0 0x7f9ee0151b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:28.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.214+0000 7f9edf7fe640 1 -- 172.21.15.83:0/4166810570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee014c120 con 0x7f9ee00b5b40 2024-04-17T10:48:28.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.214+0000 7f9edffff640 1 --2- 172.21.15.83:0/4166810570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ee015dfa0 0x7f9ee0151b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:28.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.214+0000 7f9edeffd640 1 --2- 172.21.15.83:0/4166810570 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ee00ce980 0x7f9ee0154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:28.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.218+0000 7f9edf7fe640 1 --2- 172.21.15.83:0/4166810570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ee00b5b40 0x7f9ee014fea0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400c7f0 tx=0x7f9ed400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:28.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.218+0000 7f9ec7fff640 1 -- 172.21.15.83:0/4166810570 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4007a30 con 0x7f9ee00b5b40 2024-04-17T10:48:28.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.218+0000 7f9ec7fff640 1 -- 172.21.15.83:0/4166810570 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed4007bd0 con 0x7f9ee00b5b40 2024-04-17T10:48:28.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.218+0000 7f9ee5ff6640 1 -- 172.21.15.83:0/4166810570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee014bb60 con 0x7f9ee00b5b40 2024-04-17T10:48:28.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.218+0000 7f9ec7fff640 1 -- 172.21.15.83:0/4166810570 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed40113f0 con 0x7f9ee00b5b40 2024-04-17T10:48:28.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.218+0000 7f9ee5ff6640 1 -- 172.21.15.83:0/4166810570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee014ca50 con 0x7f9ee00b5b40 2024-04-17T10:48:28.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.218+0000 7f9ee5ff6640 1 -- 172.21.15.83:0/4166810570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee0074f90 con 0x7f9ee00b5b40 2024-04-17T10:48:28.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.218+0000 7f9ec7fff640 1 -- 172.21.15.83:0/4166810570 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9ed4011590 con 0x7f9ee00b5b40 2024-04-17T10:48:28.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.218+0000 7f9ec7fff640 1 --2- 172.21.15.83:0/4166810570 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9ecc066f60 0x7f9ecc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:28.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.218+0000 7f9ec7fff640 1 -- 172.21.15.83:0/4166810570 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ed40851b0 con 0x7f9ee00b5b40 2024-04-17T10:48:28.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.218+0000 7f9edeffd640 1 --2- 172.21.15.83:0/4166810570 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9ecc066f60 0x7f9ecc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:28.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.218+0000 7f9edeffd640 1 --2- 172.21.15.83:0/4166810570 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9ecc066f60 0x7f9ecc069420 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9ee014fad0 tx=0x7f9ed0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:28.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.222+0000 7f9ec7fff640 1 -- 172.21.15.83:0/4166810570 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9ed4015020 con 0x7f9ee00b5b40 2024-04-17T10:48:28.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.366+0000 7f9ee5ff6640 1 -- 172.21.15.83:0/4166810570 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9ee00d0cc0 con 0x7f9ecc066f60 2024-04-17T10:48:28.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.374+0000 7f9ec7fff640 1 -- 172.21.15.83:0/4166810570 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247089 (secure 0 0 0) 0x7f9ed0054550 con 0x7f9ecc066f60 2024-04-17T10:48:28.375 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:28.377 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:48:28.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.382+0000 7f9ee5ff6640 1 -- 172.21.15.83:0/4166810570 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9ecc066f60 msgr2=0x7f9ecc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:28.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.382+0000 7f9ee5ff6640 1 --2- 172.21.15.83:0/4166810570 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9ecc066f60 0x7f9ecc069420 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9ee014fad0 tx=0x7f9ed0009040 comp rx=0 tx=0).stop 2024-04-17T10:48:28.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.382+0000 7f9ee5ff6640 1 -- 172.21.15.83:0/4166810570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ee00b5b40 msgr2=0x7f9ee014fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:28.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.382+0000 7f9ee5ff6640 1 --2- 172.21.15.83:0/4166810570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ee00b5b40 0x7f9ee014fea0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400c7f0 tx=0x7f9ed400ccb0 comp rx=0 tx=0).stop 2024-04-17T10:48:28.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.382+0000 7f9ee5ff6640 1 -- 172.21.15.83:0/4166810570 shutdown_connections 2024-04-17T10:48:28.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.382+0000 7f9ee5ff6640 1 --2- 172.21.15.83:0/4166810570 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9ecc066f60 0x7f9ecc069420 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:28.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.382+0000 7f9ee5ff6640 1 --2- 172.21.15.83:0/4166810570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ee015dfa0 0x7f9ee0151b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:28.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.382+0000 7f9ee5ff6640 1 --2- 172.21.15.83:0/4166810570 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ee00ce980 0x7f9ee0154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:28.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.382+0000 7f9ee5ff6640 1 --2- 172.21.15.83:0/4166810570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ee00b5b40 0x7f9ee014fea0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:28.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.382+0000 7f9ee5ff6640 1 -- 172.21.15.83:0/4166810570 >> 172.21.15.83:0/4166810570 conn(0x7f9ee00d0600 msgr2=0x7f9ee015cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:28.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.382+0000 7f9ee5ff6640 1 -- 172.21.15.83:0/4166810570 shutdown_connections 2024-04-17T10:48:28.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.382+0000 7f9ee5ff6640 1 -- 172.21.15.83:0/4166810570 wait complete. 2024-04-17T10:48:28.392 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":422,"stamp":"2024-04-17T10:48:28.363691+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":537644,"kb_used_data":11944,"kb_used_omap":12,"kb_used_meta":525555,"kb_avail":754437076,"statfs":{"total":773094113280,"available":772543565824,"internally_reserved":0,"allocated":12230656,"data_stored":5958136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":538168880},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.003886"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.387123+0000","last_change":"2024-04-17T10:48:24.387123+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.387123+0000","last_undegraded":"2024-04-17T10:48:24.387123+0000","last_fullsized":"2024-04-17T10:48:24.387123+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.370522+0000","last_change":"2024-04-17T10:48:24.370522+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.370522+0000","last_undegraded":"2024-04-17T10:48:24.370522+0000","last_fullsized":"2024-04-17T10:48:24.370522+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.379509+0000","last_change":"2024-04-17T10:48:24.379509+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.379509+0000","last_undegraded":"2024-04-17T10:48:24.379509+0000","last_fullsized":"2024-04-17T10:48:24.379509+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.387122+0000","last_change":"2024-04-17T10:48:24.387122+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.387122+0000","last_undegraded":"2024-04-17T10:48:24.387122+0000","last_fullsized":"2024-04-17T10:48:24.387122+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.380195+0000","last_change":"2024-04-17T10:48:24.380195+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.380195+0000","last_undegraded":"2024-04-17T10:48:24.380195+0000","last_fullsized":"2024-04-17T10:48:24.380195+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.378608+0000","last_change":"2024-04-17T10:48:24.378608+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.378608+0000","last_undegraded":"2024-04-17T10:48:24.378608+0000","last_fullsized":"2024-04-17T10:48:24.378608+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.384126+0000","last_change":"2024-04-17T10:48:24.384126+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.384126+0000","last_undegraded":"2024-04-17T10:48:24.384126+0000","last_fullsized":"2024-04-17T10:48:24.384126+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.380148+0000","last_change":"2024-04-17T10:48:24.380148+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.380148+0000","last_undegraded":"2024-04-17T10:48:24.380148+0000","last_fullsized":"2024-04-17T10:48:24.380148+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.386611+0000","last_change":"2024-04-17T10:48:24.386611+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.386611+0000","last_undegraded":"2024-04-17T10:48:24.386611+0000","last_fullsized":"2024-04-17T10:48:24.386611+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.389198+0000","last_change":"2024-04-17T10:48:24.389198+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.389198+0000","last_undegraded":"2024-04-17T10:48:24.389198+0000","last_fullsized":"2024-04-17T10:48:24.389198+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.370124+0000","last_change":"2024-04-17T10:48:24.370124+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.370124+0000","last_undegraded":"2024-04-17T10:48:24.370124+0000","last_fullsized":"2024-04-17T10:48:24.370124+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.386065+0000","last_change":"2024-04-17T10:48:24.386065+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.386065+0000","last_undegraded":"2024-04-17T10:48:24.386065+0000","last_fullsized":"2024-04-17T10:48:24.386065+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.374379+0000","last_change":"2024-04-17T10:48:24.374379+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.374379+0000","last_undegraded":"2024-04-17T10:48:24.374379+0000","last_fullsized":"2024-04-17T10:48:24.374379+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.376316+0000","last_change":"2024-04-17T10:48:24.376316+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.376316+0000","last_undegraded":"2024-04-17T10:48:24.376316+0000","last_fullsized":"2024-04-17T10:48:24.376316+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.375310+0000","last_change":"2024-04-17T10:48:24.375310+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.375310+0000","last_undegraded":"2024-04-17T10:48:24.375310+0000","last_fullsized":"2024-04-17T10:48:24.375310+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.385989+0000","last_change":"2024-04-17T10:48:24.385989+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.385989+0000","last_undegraded":"2024-04-17T10:48:24.385989+0000","last_fullsized":"2024-04-17T10:48:24.385989+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.373575+0000","last_change":"2024-04-17T10:48:24.373575+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.373575+0000","last_undegraded":"2024-04-17T10:48:24.373575+0000","last_fullsized":"2024-04-17T10:48:24.373575+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.385756+0000","last_change":"2024-04-17T10:48:24.385756+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.385756+0000","last_undegraded":"2024-04-17T10:48:24.385756+0000","last_fullsized":"2024-04-17T10:48:24.385756+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.378920+0000","last_change":"2024-04-17T10:48:24.378920+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.378920+0000","last_undegraded":"2024-04-17T10:48:24.378920+0000","last_fullsized":"2024-04-17T10:48:24.378920+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.371256+0000","last_change":"2024-04-17T10:48:24.371256+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.371256+0000","last_undegraded":"2024-04-17T10:48:24.371256+0000","last_fullsized":"2024-04-17T10:48:24.371256+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.370141+0000","last_change":"2024-04-17T10:48:24.370141+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.370141+0000","last_undegraded":"2024-04-17T10:48:24.370141+0000","last_fullsized":"2024-04-17T10:48:24.370141+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.393879+0000","last_change":"2024-04-17T10:48:24.393879+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.393879+0000","last_undegraded":"2024-04-17T10:48:24.393879+0000","last_fullsized":"2024-04-17T10:48:24.393879+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.373549+0000","last_change":"2024-04-17T10:48:24.373549+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.373549+0000","last_undegraded":"2024-04-17T10:48:24.373549+0000","last_fullsized":"2024-04-17T10:48:24.373549+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.393172+0000","last_change":"2024-04-17T10:48:24.393172+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.393172+0000","last_undegraded":"2024-04-17T10:48:24.393172+0000","last_fullsized":"2024-04-17T10:48:24.393172+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.371705+0000","last_change":"2024-04-17T10:48:24.371705+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.371705+0000","last_undegraded":"2024-04-17T10:48:24.371705+0000","last_fullsized":"2024-04-17T10:48:24.371705+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.386061+0000","last_change":"2024-04-17T10:48:24.386061+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.386061+0000","last_undegraded":"2024-04-17T10:48:24.386061+0000","last_fullsized":"2024-04-17T10:48:24.386061+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.374580+0000","last_change":"2024-04-17T10:48:24.374580+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.374580+0000","last_undegraded":"2024-04-17T10:48:24.374580+0000","last_fullsized":"2024-04-17T10:48:24.374580+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.393499+0000","last_change":"2024-04-17T10:48:24.393499+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.393499+0000","last_undegraded":"2024-04-17T10:48:24.393499+0000","last_fullsized":"2024-04-17T10:48:24.393499+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.372097+0000","last_change":"2024-04-17T10:48:24.372097+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.372097+0000","last_undegraded":"2024-04-17T10:48:24.372097+0000","last_fullsized":"2024-04-17T10:48:24.372097+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.389706+0000","last_change":"2024-04-17T10:48:24.389706+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.389706+0000","last_undegraded":"2024-04-17T10:48:24.389706+0000","last_fullsized":"2024-04-17T10:48:24.389706+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.385558+0000","last_change":"2024-04-17T10:48:24.385558+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.385558+0000","last_undegraded":"2024-04-17T10:48:24.385558+0000","last_fullsized":"2024-04-17T10:48:24.385558+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.376758+0000","last_change":"2024-04-17T10:48:24.376758+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.376758+0000","last_undegraded":"2024-04-17T10:48:24.376758+0000","last_fullsized":"2024-04-17T10:48:24.376758+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.375305+0000","last_change":"2024-04-17T10:48:24.375305+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.375305+0000","last_undegraded":"2024-04-17T10:48:24.375305+0000","last_fullsized":"2024-04-17T10:48:24.375305+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401517+0000","last_change":"2024-04-17T10:48:24.396879+0000","last_active":"2024-04-17T10:48:24.401517+0000","last_peered":"2024-04-17T10:48:24.401517+0000","last_clean":"2024-04-17T10:48:24.401517+0000","last_became_active":"2024-04-17T10:48:24.396126+0000","last_became_peered":"2024-04-17T10:48:24.396126+0000","last_unstale":"2024-04-17T10:48:24.401517+0000","last_undegraded":"2024-04-17T10:48:24.401517+0000","last_fullsized":"2024-04-17T10:48:24.401517+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:35:03.367753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.376492+0000","last_change":"2024-04-17T10:48:24.376492+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.376492+0000","last_undegraded":"2024-04-17T10:48:24.376492+0000","last_fullsized":"2024-04-17T10:48:24.376492+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.399197+0000","last_change":"2024-04-17T10:48:24.398459+0000","last_active":"2024-04-17T10:48:24.399197+0000","last_peered":"2024-04-17T10:48:24.399197+0000","last_clean":"2024-04-17T10:48:24.399197+0000","last_became_active":"2024-04-17T10:48:24.398098+0000","last_became_peered":"2024-04-17T10:48:24.398098+0000","last_unstale":"2024-04-17T10:48:24.399197+0000","last_undegraded":"2024-04-17T10:48:24.399197+0000","last_fullsized":"2024-04-17T10:48:24.399197+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:23:07.107332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.379331+0000","last_change":"2024-04-17T10:48:24.379331+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.379331+0000","last_undegraded":"2024-04-17T10:48:24.379331+0000","last_fullsized":"2024-04-17T10:48:24.379331+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.402080+0000","last_change":"2024-04-17T10:48:24.400890+0000","last_active":"2024-04-17T10:48:24.402080+0000","last_peered":"2024-04-17T10:48:24.402080+0000","last_clean":"2024-04-17T10:48:24.402080+0000","last_became_active":"2024-04-17T10:48:24.400472+0000","last_became_peered":"2024-04-17T10:48:24.400472+0000","last_unstale":"2024-04-17T10:48:24.402080+0000","last_undegraded":"2024-04-17T10:48:24.402080+0000","last_fullsized":"2024-04-17T10:48:24.402080+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:20:02.731934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.376494+0000","last_change":"2024-04-17T10:48:24.376494+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.376494+0000","last_undegraded":"2024-04-17T10:48:24.376494+0000","last_fullsized":"2024-04-17T10:48:24.376494+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.402280+0000","last_change":"2024-04-17T10:48:24.401643+0000","last_active":"2024-04-17T10:48:24.402280+0000","last_peered":"2024-04-17T10:48:24.402280+0000","last_clean":"2024-04-17T10:48:24.402280+0000","last_became_active":"2024-04-17T10:48:24.400656+0000","last_became_peered":"2024-04-17T10:48:24.400656+0000","last_unstale":"2024-04-17T10:48:24.402280+0000","last_undegraded":"2024-04-17T10:48:24.402280+0000","last_fullsized":"2024-04-17T10:48:24.402280+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:11:46.753399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.385378+0000","last_change":"2024-04-17T10:48:24.385378+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.385378+0000","last_undegraded":"2024-04-17T10:48:24.385378+0000","last_fullsized":"2024-04-17T10:48:24.385378+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.404934+0000","last_change":"2024-04-17T10:48:24.403866+0000","last_active":"2024-04-17T10:48:24.404934+0000","last_peered":"2024-04-17T10:48:24.404934+0000","last_clean":"2024-04-17T10:48:24.404934+0000","last_became_active":"2024-04-17T10:48:24.403129+0000","last_became_peered":"2024-04-17T10:48:24.403129+0000","last_unstale":"2024-04-17T10:48:24.404934+0000","last_undegraded":"2024-04-17T10:48:24.404934+0000","last_fullsized":"2024-04-17T10:48:24.404934+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:34:15.196696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.371493+0000","last_change":"2024-04-17T10:48:24.371493+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.371493+0000","last_undegraded":"2024-04-17T10:48:24.371493+0000","last_fullsized":"2024-04-17T10:48:24.371493+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.402426+0000","last_change":"2024-04-17T10:48:24.399589+0000","last_active":"2024-04-17T10:48:24.402426+0000","last_peered":"2024-04-17T10:48:24.402426+0000","last_clean":"2024-04-17T10:48:24.402426+0000","last_became_active":"2024-04-17T10:48:24.399097+0000","last_became_peered":"2024-04-17T10:48:24.399097+0000","last_unstale":"2024-04-17T10:48:24.402426+0000","last_undegraded":"2024-04-17T10:48:24.402426+0000","last_fullsized":"2024-04-17T10:48:24.402426+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:01:53.668168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.385357+0000","last_change":"2024-04-17T10:48:24.385357+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.385357+0000","last_undegraded":"2024-04-17T10:48:24.385357+0000","last_fullsized":"2024-04-17T10:48:24.385357+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.379620+0000","last_change":"2024-04-17T10:48:24.376241+0000","last_active":"2024-04-17T10:48:24.379620+0000","last_peered":"2024-04-17T10:48:24.379620+0000","last_clean":"2024-04-17T10:48:24.379620+0000","last_became_active":"2024-04-17T10:48:24.375430+0000","last_became_peered":"2024-04-17T10:48:24.375430+0000","last_unstale":"2024-04-17T10:48:24.379620+0000","last_undegraded":"2024-04-17T10:48:24.379620+0000","last_fullsized":"2024-04-17T10:48:24.379620+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:46:07.697646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.384767+0000","last_change":"2024-04-17T10:48:24.384767+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.384767+0000","last_undegraded":"2024-04-17T10:48:24.384767+0000","last_fullsized":"2024-04-17T10:48:24.384767+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.385628+0000","last_change":"2024-04-17T10:48:24.382363+0000","last_active":"2024-04-17T10:48:24.385628+0000","last_peered":"2024-04-17T10:48:24.385628+0000","last_clean":"2024-04-17T10:48:24.385628+0000","last_became_active":"2024-04-17T10:48:24.381493+0000","last_became_peered":"2024-04-17T10:48:24.381493+0000","last_unstale":"2024-04-17T10:48:24.385628+0000","last_undegraded":"2024-04-17T10:48:24.385628+0000","last_fullsized":"2024-04-17T10:48:24.385628+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:06:35.936896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.389344+0000","last_change":"2024-04-17T10:48:24.389344+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.389344+0000","last_undegraded":"2024-04-17T10:48:24.389344+0000","last_fullsized":"2024-04-17T10:48:24.389344+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401814+0000","last_change":"2024-04-17T10:48:24.400839+0000","last_active":"2024-04-17T10:48:24.401814+0000","last_peered":"2024-04-17T10:48:24.401814+0000","last_clean":"2024-04-17T10:48:24.401814+0000","last_became_active":"2024-04-17T10:48:24.400489+0000","last_became_peered":"2024-04-17T10:48:24.400489+0000","last_unstale":"2024-04-17T10:48:24.401814+0000","last_undegraded":"2024-04-17T10:48:24.401814+0000","last_fullsized":"2024-04-17T10:48:24.401814+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:42:13.356381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.375839+0000","last_change":"2024-04-17T10:48:24.375839+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.375839+0000","last_undegraded":"2024-04-17T10:48:24.375839+0000","last_fullsized":"2024-04-17T10:48:24.375839+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.402374+0000","last_change":"2024-04-17T10:48:24.399209+0000","last_active":"2024-04-17T10:48:24.402374+0000","last_peered":"2024-04-17T10:48:24.402374+0000","last_clean":"2024-04-17T10:48:24.402374+0000","last_became_active":"2024-04-17T10:48:24.398789+0000","last_became_peered":"2024-04-17T10:48:24.398789+0000","last_unstale":"2024-04-17T10:48:24.402374+0000","last_undegraded":"2024-04-17T10:48:24.402374+0000","last_fullsized":"2024-04-17T10:48:24.402374+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:33:31.610895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.391756+0000","last_change":"2024-04-17T10:48:24.391756+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.391756+0000","last_undegraded":"2024-04-17T10:48:24.391756+0000","last_fullsized":"2024-04-17T10:48:24.391756+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.402960+0000","last_change":"2024-04-17T10:48:24.402346+0000","last_active":"2024-04-17T10:48:24.402960+0000","last_peered":"2024-04-17T10:48:24.402960+0000","last_clean":"2024-04-17T10:48:24.402960+0000","last_became_active":"2024-04-17T10:48:24.401900+0000","last_became_peered":"2024-04-17T10:48:24.401900+0000","last_unstale":"2024-04-17T10:48:24.402960+0000","last_undegraded":"2024-04-17T10:48:24.402960+0000","last_fullsized":"2024-04-17T10:48:24.402960+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:23:07.107332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.390503+0000","last_change":"2024-04-17T10:48:24.390503+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.390503+0000","last_undegraded":"2024-04-17T10:48:24.390503+0000","last_fullsized":"2024-04-17T10:48:24.390503+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.382233+0000","last_change":"2024-04-17T10:48:24.379794+0000","last_active":"2024-04-17T10:48:24.382233+0000","last_peered":"2024-04-17T10:48:24.382233+0000","last_clean":"2024-04-17T10:48:24.382233+0000","last_became_active":"2024-04-17T10:48:24.379413+0000","last_became_peered":"2024-04-17T10:48:24.379413+0000","last_unstale":"2024-04-17T10:48:24.382233+0000","last_undegraded":"2024-04-17T10:48:24.382233+0000","last_fullsized":"2024-04-17T10:48:24.382233+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:42:10.087214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.391471+0000","last_change":"2024-04-17T10:48:24.391471+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.391471+0000","last_undegraded":"2024-04-17T10:48:24.391471+0000","last_fullsized":"2024-04-17T10:48:24.391471+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401761+0000","last_change":"2024-04-17T10:48:24.400108+0000","last_active":"2024-04-17T10:48:24.401761+0000","last_peered":"2024-04-17T10:48:24.401761+0000","last_clean":"2024-04-17T10:48:24.401761+0000","last_became_active":"2024-04-17T10:48:24.397908+0000","last_became_peered":"2024-04-17T10:48:24.397908+0000","last_unstale":"2024-04-17T10:48:24.401761+0000","last_undegraded":"2024-04-17T10:48:24.401761+0000","last_fullsized":"2024-04-17T10:48:24.401761+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:11:46.753399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.393082+0000","last_change":"2024-04-17T10:48:24.393082+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.393082+0000","last_undegraded":"2024-04-17T10:48:24.393082+0000","last_fullsized":"2024-04-17T10:48:24.393082+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.392836+0000","last_change":"2024-04-17T10:48:24.391544+0000","last_active":"2024-04-17T10:48:24.392836+0000","last_peered":"2024-04-17T10:48:24.392836+0000","last_clean":"2024-04-17T10:48:24.392836+0000","last_became_active":"2024-04-17T10:48:24.388072+0000","last_became_peered":"2024-04-17T10:48:24.388072+0000","last_unstale":"2024-04-17T10:48:24.392836+0000","last_undegraded":"2024-04-17T10:48:24.392836+0000","last_fullsized":"2024-04-17T10:48:24.392836+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:03:44.300221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.392104+0000","last_change":"2024-04-17T10:48:24.392104+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.392104+0000","last_undegraded":"2024-04-17T10:48:24.392104+0000","last_fullsized":"2024-04-17T10:48:24.392104+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.389451+0000","last_change":"2024-04-17T10:48:24.383457+0000","last_active":"2024-04-17T10:48:24.389451+0000","last_peered":"2024-04-17T10:48:24.389451+0000","last_clean":"2024-04-17T10:48:24.389451+0000","last_became_active":"2024-04-17T10:48:24.382932+0000","last_became_peered":"2024-04-17T10:48:24.382932+0000","last_unstale":"2024-04-17T10:48:24.389451+0000","last_undegraded":"2024-04-17T10:48:24.389451+0000","last_fullsized":"2024-04-17T10:48:24.389451+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:53:06.465373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.376491+0000","last_change":"2024-04-17T10:48:24.376491+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.376491+0000","last_undegraded":"2024-04-17T10:48:24.376491+0000","last_fullsized":"2024-04-17T10:48:24.376491+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.399747+0000","last_change":"2024-04-17T10:48:24.399014+0000","last_active":"2024-04-17T10:48:24.399747+0000","last_peered":"2024-04-17T10:48:24.399747+0000","last_clean":"2024-04-17T10:48:24.399747+0000","last_became_active":"2024-04-17T10:48:24.398705+0000","last_became_peered":"2024-04-17T10:48:24.398705+0000","last_unstale":"2024-04-17T10:48:24.399747+0000","last_undegraded":"2024-04-17T10:48:24.399747+0000","last_fullsized":"2024-04-17T10:48:24.399747+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:01:53.668168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.374704+0000","last_change":"2024-04-17T10:48:24.374704+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.374704+0000","last_undegraded":"2024-04-17T10:48:24.374704+0000","last_fullsized":"2024-04-17T10:48:24.374704+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.390926+0000","last_change":"2024-04-17T10:48:24.387073+0000","last_active":"2024-04-17T10:48:24.390926+0000","last_peered":"2024-04-17T10:48:24.390926+0000","last_clean":"2024-04-17T10:48:24.390926+0000","last_became_active":"2024-04-17T10:48:24.386344+0000","last_became_peered":"2024-04-17T10:48:24.386344+0000","last_unstale":"2024-04-17T10:48:24.390926+0000","last_undegraded":"2024-04-17T10:48:24.390926+0000","last_fullsized":"2024-04-17T10:48:24.390926+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:09:38.552029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.373427+0000","last_change":"2024-04-17T10:48:24.373427+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.373427+0000","last_undegraded":"2024-04-17T10:48:24.373427+0000","last_fullsized":"2024-04-17T10:48:24.373427+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.385549+0000","last_change":"2024-04-17T10:48:24.382682+0000","last_active":"2024-04-17T10:48:24.385549+0000","last_peered":"2024-04-17T10:48:24.385549+0000","last_clean":"2024-04-17T10:48:24.385549+0000","last_became_active":"2024-04-17T10:48:24.381331+0000","last_became_peered":"2024-04-17T10:48:24.381331+0000","last_unstale":"2024-04-17T10:48:24.385549+0000","last_undegraded":"2024-04-17T10:48:24.385549+0000","last_fullsized":"2024-04-17T10:48:24.385549+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:15:43.459938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.373479+0000","last_change":"2024-04-17T10:48:24.373479+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.373479+0000","last_undegraded":"2024-04-17T10:48:24.373479+0000","last_fullsized":"2024-04-17T10:48:24.373479+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401395+0000","last_change":"2024-04-17T10:48:24.396200+0000","last_active":"2024-04-17T10:48:24.401395+0000","last_peered":"2024-04-17T10:48:24.401395+0000","last_clean":"2024-04-17T10:48:24.401395+0000","last_became_active":"2024-04-17T10:48:24.395802+0000","last_became_peered":"2024-04-17T10:48:24.395802+0000","last_unstale":"2024-04-17T10:48:24.401395+0000","last_undegraded":"2024-04-17T10:48:24.401395+0000","last_fullsized":"2024-04-17T10:48:24.401395+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:13:48.163535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.374303+0000","last_change":"2024-04-17T10:48:24.374303+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.374303+0000","last_undegraded":"2024-04-17T10:48:24.374303+0000","last_fullsized":"2024-04-17T10:48:24.374303+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401739+0000","last_change":"2024-04-17T10:48:24.397774+0000","last_active":"2024-04-17T10:48:24.401739+0000","last_peered":"2024-04-17T10:48:24.401739+0000","last_clean":"2024-04-17T10:48:24.401739+0000","last_became_active":"2024-04-17T10:48:24.397453+0000","last_became_peered":"2024-04-17T10:48:24.397453+0000","last_unstale":"2024-04-17T10:48:24.401739+0000","last_undegraded":"2024-04-17T10:48:24.401739+0000","last_fullsized":"2024-04-17T10:48:24.401739+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:20:23.871607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.384650+0000","last_change":"2024-04-17T10:48:24.384650+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.384650+0000","last_undegraded":"2024-04-17T10:48:24.384650+0000","last_fullsized":"2024-04-17T10:48:24.384650+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401589+0000","last_change":"2024-04-17T10:48:24.400725+0000","last_active":"2024-04-17T10:48:24.401589+0000","last_peered":"2024-04-17T10:48:24.401589+0000","last_clean":"2024-04-17T10:48:24.401589+0000","last_became_active":"2024-04-17T10:48:24.400444+0000","last_became_peered":"2024-04-17T10:48:24.400444+0000","last_unstale":"2024-04-17T10:48:24.401589+0000","last_undegraded":"2024-04-17T10:48:24.401589+0000","last_fullsized":"2024-04-17T10:48:24.401589+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:05:55.306345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.379208+0000","last_change":"2024-04-17T10:48:24.379208+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.379208+0000","last_undegraded":"2024-04-17T10:48:24.379208+0000","last_fullsized":"2024-04-17T10:48:24.379208+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401440+0000","last_change":"2024-04-17T10:48:24.397713+0000","last_active":"2024-04-17T10:48:24.401440+0000","last_peered":"2024-04-17T10:48:24.401440+0000","last_clean":"2024-04-17T10:48:24.401440+0000","last_became_active":"2024-04-17T10:48:24.397264+0000","last_became_peered":"2024-04-17T10:48:24.397264+0000","last_unstale":"2024-04-17T10:48:24.401440+0000","last_undegraded":"2024-04-17T10:48:24.401440+0000","last_fullsized":"2024-04-17T10:48:24.401440+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:20:02.731934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.390331+0000","last_change":"2024-04-17T10:48:24.390331+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.390331+0000","last_undegraded":"2024-04-17T10:48:24.390331+0000","last_fullsized":"2024-04-17T10:48:24.390331+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.395732+0000","last_change":"2024-04-17T10:48:24.393648+0000","last_active":"2024-04-17T10:48:24.395732+0000","last_peered":"2024-04-17T10:48:24.395732+0000","last_clean":"2024-04-17T10:48:24.395732+0000","last_became_active":"2024-04-17T10:48:24.392862+0000","last_became_peered":"2024-04-17T10:48:24.392862+0000","last_unstale":"2024-04-17T10:48:24.395732+0000","last_undegraded":"2024-04-17T10:48:24.395732+0000","last_fullsized":"2024-04-17T10:48:24.395732+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:33:31.610895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.384853+0000","last_change":"2024-04-17T10:48:24.384853+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.384853+0000","last_undegraded":"2024-04-17T10:48:24.384853+0000","last_fullsized":"2024-04-17T10:48:24.384853+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.394096+0000","last_change":"2024-04-17T10:48:24.389637+0000","last_active":"2024-04-17T10:48:24.394096+0000","last_peered":"2024-04-17T10:48:24.394096+0000","last_clean":"2024-04-17T10:48:24.394096+0000","last_became_active":"2024-04-17T10:48:24.389212+0000","last_became_peered":"2024-04-17T10:48:24.389212+0000","last_unstale":"2024-04-17T10:48:24.394096+0000","last_undegraded":"2024-04-17T10:48:24.394096+0000","last_fullsized":"2024-04-17T10:48:24.394096+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:36:28.852444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.384672+0000","last_change":"2024-04-17T10:48:24.384672+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.384672+0000","last_undegraded":"2024-04-17T10:48:24.384672+0000","last_fullsized":"2024-04-17T10:48:24.384672+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.400185+0000","last_change":"2024-04-17T10:48:24.398460+0000","last_active":"2024-04-17T10:48:24.400185+0000","last_peered":"2024-04-17T10:48:24.400185+0000","last_clean":"2024-04-17T10:48:24.400185+0000","last_became_active":"2024-04-17T10:48:24.397860+0000","last_became_peered":"2024-04-17T10:48:24.397860+0000","last_unstale":"2024-04-17T10:48:24.400185+0000","last_undegraded":"2024-04-17T10:48:24.400185+0000","last_fullsized":"2024-04-17T10:48:24.400185+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:20:18.572385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.384643+0000","last_change":"2024-04-17T10:48:24.384643+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.384643+0000","last_undegraded":"2024-04-17T10:48:24.384643+0000","last_fullsized":"2024-04-17T10:48:24.384643+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.395517+0000","last_change":"2024-04-17T10:48:24.391043+0000","last_active":"2024-04-17T10:48:24.395517+0000","last_peered":"2024-04-17T10:48:24.395517+0000","last_clean":"2024-04-17T10:48:24.395517+0000","last_became_active":"2024-04-17T10:48:24.390673+0000","last_became_peered":"2024-04-17T10:48:24.390673+0000","last_unstale":"2024-04-17T10:48:24.395517+0000","last_undegraded":"2024-04-17T10:48:24.395517+0000","last_fullsized":"2024-04-17T10:48:24.395517+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:36:28.852444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.375853+0000","last_change":"2024-04-17T10:48:24.375853+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.375853+0000","last_undegraded":"2024-04-17T10:48:24.375853+0000","last_fullsized":"2024-04-17T10:48:24.375853+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.400107+0000","last_change":"2024-04-17T10:48:24.398215+0000","last_active":"2024-04-17T10:48:24.400107+0000","last_peered":"2024-04-17T10:48:24.400107+0000","last_clean":"2024-04-17T10:48:24.400107+0000","last_became_active":"2024-04-17T10:48:24.394779+0000","last_became_peered":"2024-04-17T10:48:24.394779+0000","last_unstale":"2024-04-17T10:48:24.400107+0000","last_undegraded":"2024-04-17T10:48:24.400107+0000","last_fullsized":"2024-04-17T10:48:24.400107+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:54:47.216960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.370200+0000","last_change":"2024-04-17T10:48:24.370200+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.370200+0000","last_undegraded":"2024-04-17T10:48:24.370200+0000","last_fullsized":"2024-04-17T10:48:24.370200+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401418+0000","last_change":"2024-04-17T10:48:24.396294+0000","last_active":"2024-04-17T10:48:24.401418+0000","last_peered":"2024-04-17T10:48:24.401418+0000","last_clean":"2024-04-17T10:48:24.401418+0000","last_became_active":"2024-04-17T10:48:24.395673+0000","last_became_peered":"2024-04-17T10:48:24.395673+0000","last_unstale":"2024-04-17T10:48:24.401418+0000","last_undegraded":"2024-04-17T10:48:24.401418+0000","last_fullsized":"2024-04-17T10:48:24.401418+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:16:43.238162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.370856+0000","last_change":"2024-04-17T10:48:24.370856+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.370856+0000","last_undegraded":"2024-04-17T10:48:24.370856+0000","last_fullsized":"2024-04-17T10:48:24.370856+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.387789+0000","last_change":"2024-04-17T10:48:24.386727+0000","last_active":"2024-04-17T10:48:24.387789+0000","last_peered":"2024-04-17T10:48:24.387789+0000","last_clean":"2024-04-17T10:48:24.387789+0000","last_became_active":"2024-04-17T10:48:24.386334+0000","last_became_peered":"2024-04-17T10:48:24.386334+0000","last_unstale":"2024-04-17T10:48:24.387789+0000","last_undegraded":"2024-04-17T10:48:24.387789+0000","last_fullsized":"2024-04-17T10:48:24.387789+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:09.646139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.378187+0000","last_change":"2024-04-17T10:48:24.378187+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.378187+0000","last_undegraded":"2024-04-17T10:48:24.378187+0000","last_fullsized":"2024-04-17T10:48:24.378187+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.402062+0000","last_change":"2024-04-17T10:48:24.401190+0000","last_active":"2024-04-17T10:48:24.402062+0000","last_peered":"2024-04-17T10:48:24.402062+0000","last_clean":"2024-04-17T10:48:24.402062+0000","last_became_active":"2024-04-17T10:48:24.400909+0000","last_became_peered":"2024-04-17T10:48:24.400909+0000","last_unstale":"2024-04-17T10:48:24.402062+0000","last_undegraded":"2024-04-17T10:48:24.402062+0000","last_fullsized":"2024-04-17T10:48:24.402062+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:20:23.871607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.383491+0000","last_change":"2024-04-17T10:48:24.383491+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.383491+0000","last_undegraded":"2024-04-17T10:48:24.383491+0000","last_fullsized":"2024-04-17T10:48:24.383491+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.395697+0000","last_change":"2024-04-17T10:48:24.393593+0000","last_active":"2024-04-17T10:48:24.395697+0000","last_peered":"2024-04-17T10:48:24.395697+0000","last_clean":"2024-04-17T10:48:24.395697+0000","last_became_active":"2024-04-17T10:48:24.392876+0000","last_became_peered":"2024-04-17T10:48:24.392876+0000","last_unstale":"2024-04-17T10:48:24.395697+0000","last_undegraded":"2024-04-17T10:48:24.395697+0000","last_fullsized":"2024-04-17T10:48:24.395697+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:54:47.216960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.390419+0000","last_change":"2024-04-17T10:48:24.390419+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.390419+0000","last_undegraded":"2024-04-17T10:48:24.390419+0000","last_fullsized":"2024-04-17T10:48:24.390419+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401545+0000","last_change":"2024-04-17T10:48:24.396850+0000","last_active":"2024-04-17T10:48:24.401545+0000","last_peered":"2024-04-17T10:48:24.401545+0000","last_clean":"2024-04-17T10:48:24.401545+0000","last_became_active":"2024-04-17T10:48:24.396194+0000","last_became_peered":"2024-04-17T10:48:24.396194+0000","last_unstale":"2024-04-17T10:48:24.401545+0000","last_undegraded":"2024-04-17T10:48:24.401545+0000","last_fullsized":"2024-04-17T10:48:24.401545+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:20:02.139331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":359,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.387430+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:48:24.387430+0000","last_peered":"2024-04-17T10:48:24.387430+0000","last_clean":"2024-04-17T10:48:24.387430+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:48:24.387430+0000","last_undegraded":"2024-04-17T10:48:24.387430+0000","last_fullsized":"2024-04-17T10:48:24.387430+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574977,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59476,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312364,"statfs":{"total":96636764160,"available":96575860736,"internally_reserved":0,"allocated":1974272,"data_stored":1187543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574976,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68624,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303216,"statfs":{"total":96636764160,"available":96566493184,"internally_reserved":0,"allocated":1380352,"data_stored":597175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574977,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54736,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317104,"statfs":{"total":96636764160,"available":96580714496,"internally_reserved":0,"allocated":1380352,"data_stored":597175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68636,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303204,"statfs":{"total":96636764160,"available":96566480896,"internally_reserved":0,"allocated":1380352,"data_stored":597175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574978,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68624,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303216,"statfs":{"total":96636764160,"available":96566493184,"internally_reserved":0,"allocated":1380352,"data_stored":597175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58908,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312932,"statfs":{"total":96636764160,"available":96576442368,"internally_reserved":0,"allocated":1380352,"data_stored":597175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94996,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94276844,"statfs":{"total":96636764160,"available":96539488256,"internally_reserved":0,"allocated":1380352,"data_stored":597175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574976,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63644,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94308196,"statfs":{"total":96636764160,"available":96571592704,"internally_reserved":0,"allocated":1974272,"data_stored":1187543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:48:28.397 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:48:28.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.558+0000 7fe3d5460640 1 Processor -- start 2024-04-17T10:48:28.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.558+0000 7fe3d5460640 1 -- start start 2024-04-17T10:48:28.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.558+0000 7fe3d5460640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe3d015db50 0x7fe3d0070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:28.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.558+0000 7fe3d5460640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3d00709b0 con 0x7fe3d00aedd0 2024-04-17T10:48:28.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.558+0000 7fe3d5460640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3d015e440 con 0x7fe3d006dcb0 2024-04-17T10:48:28.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.558+0000 7fe3d5460640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3d015e720 con 0x7fe3d015db50 2024-04-17T10:48:28.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.558+0000 7fe3ce7fc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe3d00aedd0 0x7fe3d006d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40948/0 (socket says 172.21.15.83:40948) 2024-04-17T10:48:28.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.558+0000 7fe3ce7fc640 1 -- 172.21.15.83:0/3353276780 learned_addr learned my addr 172.21.15.83:0/3353276780 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:28.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.558+0000 7fe3cdffb640 1 -- 172.21.15.83:0/3353276780 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1330002106 0 0) 0x7fe3c4001120 con 0x7fe3d00aedd0 2024-04-17T10:48:28.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.558+0000 7fe3cdffb640 1 -- 172.21.15.83:0/3353276780 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3b8003530 con 0x7fe3d00aedd0 2024-04-17T10:48:28.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.558+0000 7fe3ceffd640 1 --2- 172.21.15.83:0/3353276780 >> v2:172.21.15.167:3300/0 conn(0x7fe3d015db50 0x7fe3d0070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:28.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.558+0000 7fe3cdffb640 1 -- 172.21.15.83:0/3353276780 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2263678934 0 0) 0x7fe3b8003530 con 0x7fe3d00aedd0 2024-04-17T10:48:28.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.558+0000 7fe3cdffb640 1 -- 172.21.15.83:0/3353276780 >> v2:172.21.15.167:3300/0 conn(0x7fe3d015db50 msgr2=0x7fe3d0070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:28.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.558+0000 7fe3cdffb640 1 --2- 172.21.15.83:0/3353276780 >> v2:172.21.15.167:3300/0 conn(0x7fe3d015db50 0x7fe3d0070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:28.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.558+0000 7fe3cdffb640 1 -- 172.21.15.83:0/3353276780 >> v1:172.21.15.167:6789/0 conn(0x7fe3d006dcb0 legacy=0x7fe3d00b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:28.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.558+0000 7fe3cdffb640 1 -- 172.21.15.83:0/3353276780 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d015ea00 con 0x7fe3d00aedd0 2024-04-17T10:48:28.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.558+0000 7fe3cdffb640 1 -- 172.21.15.83:0/3353276780 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fe3c4001850 con 0x7fe3d00aedd0 2024-04-17T10:48:28.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.558+0000 7fe3cdffb640 1 -- 172.21.15.83:0/3353276780 >> v1:172.21.15.83:6789/0 conn(0x7fe3d00aedd0 legacy=0x7fe3d006d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:28.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.558+0000 7fe3cdffb640 1 --2- 172.21.15.83:0/3353276780 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3b80039c0 0x7fe3b8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:28.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.558+0000 7fe3cdffb640 1 --2- 172.21.15.83:0/3353276780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3b8004810 0x7fe3b8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:28.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.558+0000 7fe3cdffb640 1 --2- 172.21.15.83:0/3353276780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3b80051d0 0x7fe3b800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:28.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.558+0000 7fe3cdffb640 1 -- 172.21.15.83:0/3353276780 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d015e720 con 0x7fe3b80051d0 2024-04-17T10:48:28.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.558+0000 7fe3cdffb640 1 -- 172.21.15.83:0/3353276780 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b800e330 con 0x7fe3b8004810 2024-04-17T10:48:28.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.558+0000 7fe3cdffb640 1 -- 172.21.15.83:0/3353276780 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b800e530 con 0x7fe3b80039c0 2024-04-17T10:48:28.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.558+0000 7fe3cf7fe640 1 --2- 172.21.15.83:0/3353276780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3b80051d0 0x7fe3b800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:28.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.562+0000 7fe3ceffd640 1 --2- 172.21.15.83:0/3353276780 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3b80039c0 0x7fe3b8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:28.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.562+0000 7fe3ce7fc640 1 --2- 172.21.15.83:0/3353276780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3b8004810 0x7fe3b8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:28.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.562+0000 7fe3cf7fe640 1 -- 172.21.15.83:0/3353276780 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3b80039c0 msgr2=0x7fe3b8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:28.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.562+0000 7fe3cf7fe640 1 --2- 172.21.15.83:0/3353276780 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3b80039c0 0x7fe3b8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:28.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.562+0000 7fe3cf7fe640 1 -- 172.21.15.83:0/3353276780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3b8004810 msgr2=0x7fe3b8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:28.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.562+0000 7fe3cf7fe640 1 --2- 172.21.15.83:0/3353276780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3b8004810 0x7fe3b8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:28.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.562+0000 7fe3cf7fe640 1 -- 172.21.15.83:0/3353276780 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b8003530 con 0x7fe3b80051d0 2024-04-17T10:48:28.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.562+0000 7fe3cf7fe640 1 -- 172.21.15.83:0/3353276780 reap_dead start 2024-04-17T10:48:28.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.562+0000 7fe3cf7fe640 1 --2- 172.21.15.83:0/3353276780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3b80051d0 0x7fe3b800b9d0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8001e40 tx=0x7fe3c801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c56adca8c00d1e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:28.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.562+0000 7fe3cdffb640 1 -- 172.21.15.83:0/3353276780 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8001a90 con 0x7fe3b80051d0 2024-04-17T10:48:28.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.562+0000 7fe3cdffb640 1 -- 172.21.15.83:0/3353276780 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c8003b80 con 0x7fe3b80051d0 2024-04-17T10:48:28.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.562+0000 7fe3cdffb640 1 -- 172.21.15.83:0/3353276780 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8005020 con 0x7fe3b80051d0 2024-04-17T10:48:28.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.562+0000 7fe3d5460640 1 -- 172.21.15.83:0/3353276780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3b80051d0 msgr2=0x7fe3b800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:28.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.562+0000 7fe3d5460640 1 --2- 172.21.15.83:0/3353276780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3b80051d0 0x7fe3b800b9d0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8001e40 tx=0x7fe3c801c760 comp rx=0 tx=0).stop 2024-04-17T10:48:28.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.562+0000 7fe3d5460640 1 -- 172.21.15.83:0/3353276780 shutdown_connections 2024-04-17T10:48:28.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.562+0000 7fe3d5460640 1 --2- 172.21.15.83:0/3353276780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3b80051d0 0x7fe3b800b9d0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:28.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.562+0000 7fe3d5460640 1 -- 172.21.15.83:0/3353276780 >> 172.21.15.83:0/3353276780 conn(0x7fe3d006abe0 msgr2=0x7fe3d006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:28.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.562+0000 7fe3d5460640 1 -- 172.21.15.83:0/3353276780 shutdown_connections 2024-04-17T10:48:28.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.562+0000 7fe3d5460640 1 -- 172.21.15.83:0/3353276780 wait complete. 2024-04-17T10:48:28.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.566+0000 7fe3d5460640 1 Processor -- start 2024-04-17T10:48:28.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.566+0000 7fe3d5460640 1 -- start start 2024-04-17T10:48:28.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.566+0000 7fe3d5460640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3b80051d0 0x7fe3d015c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:28.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.566+0000 7fe3d5460640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3d0156870 0x7fe3d00a9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:28.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.566+0000 7fe3d5460640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3d00b19a0 0x7fe3d00b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:28.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.566+0000 7fe3d5460640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d015efc0 con 0x7fe3d00b19a0 2024-04-17T10:48:28.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.566+0000 7fe3d5460640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d015e2c0 con 0x7fe3d0156870 2024-04-17T10:48:28.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.566+0000 7fe3d5460640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d015df30 con 0x7fe3b80051d0 2024-04-17T10:48:28.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.566+0000 7fe3cf7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3d00b19a0 0x7fe3d00b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:28.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.566+0000 7fe3cf7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3d00b19a0 0x7fe3d00b1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44682/0 (socket says 172.21.15.83:44682) 2024-04-17T10:48:28.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.566+0000 7fe3cf7fe640 1 -- 172.21.15.83:0/1489837594 learned_addr learned my addr 172.21.15.83:0/1489837594 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:28.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.566+0000 7fe3ce7fc640 1 --2- 172.21.15.83:0/1489837594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3d0156870 0x7fe3d00a9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:28.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.566+0000 7fe3cf7fe640 1 -- 172.21.15.83:0/1489837594 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3b80051d0 msgr2=0x7fe3d015c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:28.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.566+0000 7fe3cf7fe640 1 --2- 172.21.15.83:0/1489837594 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3b80051d0 0x7fe3d015c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:28.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.566+0000 7fe3cf7fe640 1 -- 172.21.15.83:0/1489837594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3d0156870 msgr2=0x7fe3d00a9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:28.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.566+0000 7fe3cf7fe640 1 --2- 172.21.15.83:0/1489837594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3d0156870 0x7fe3d00a9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:28.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.566+0000 7fe3cf7fe640 1 -- 172.21.15.83:0/1489837594 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d00ac380 con 0x7fe3d00b19a0 2024-04-17T10:48:28.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.566+0000 7fe3cf7fe640 1 --2- 172.21.15.83:0/1489837594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3d00b19a0 0x7fe3d00b1e50 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fe3c801c590 tx=0x7fe3b8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:28.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.566+0000 7fe3b77fe640 1 -- 172.21.15.83:0/1489837594 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c80b6070 con 0x7fe3d00b19a0 2024-04-17T10:48:28.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.566+0000 7fe3b77fe640 1 -- 172.21.15.83:0/1489837594 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c80ad070 con 0x7fe3d00b19a0 2024-04-17T10:48:28.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.566+0000 7fe3d5460640 1 -- 172.21.15.83:0/1489837594 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3d00aa690 con 0x7fe3d00b19a0 2024-04-17T10:48:28.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.566+0000 7fe3b77fe640 1 -- 172.21.15.83:0/1489837594 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8007dc0 con 0x7fe3d00b19a0 2024-04-17T10:48:28.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.566+0000 7fe3d5460640 1 -- 172.21.15.83:0/1489837594 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d00aac50 con 0x7fe3d00b19a0 2024-04-17T10:48:28.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.566+0000 7fe3d5460640 1 -- 172.21.15.83:0/1489837594 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3d0074f90 con 0x7fe3d00b19a0 2024-04-17T10:48:28.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.566+0000 7fe3b77fe640 1 -- 172.21.15.83:0/1489837594 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe3c80b9070 con 0x7fe3d00b19a0 2024-04-17T10:48:28.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.566+0000 7fe3b77fe640 1 --2- 172.21.15.83:0/1489837594 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe3bc066f60 0x7fe3bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:28.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.566+0000 7fe3ceffd640 1 --2- 172.21.15.83:0/1489837594 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe3bc066f60 0x7fe3bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:28.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.566+0000 7fe3b77fe640 1 -- 172.21.15.83:0/1489837594 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3c811e240 con 0x7fe3d00b19a0 2024-04-17T10:48:28.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.566+0000 7fe3ceffd640 1 --2- 172.21.15.83:0/1489837594 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe3bc066f60 0x7fe3bc069420 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000d880 tx=0x7fe3c000d260 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:28.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.570+0000 7fe3b77fe640 1 -- 172.21.15.83:0/1489837594 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe3d00ac380 con 0x7fe3d00b19a0 2024-04-17T10:48:28.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.714+0000 7fe3d5460640 1 -- 172.21.15.83:0/1489837594 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe3d00b2af0 con 0x7fe3bc066f60 2024-04-17T10:48:28.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.722+0000 7fe3b77fe640 1 -- 172.21.15.83:0/1489837594 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247089 (secure 0 0 0) 0x7fe3d00b2af0 con 0x7fe3bc066f60 2024-04-17T10:48:28.722 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:28.724 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:48:28.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.726+0000 7fe3d5460640 1 -- 172.21.15.83:0/1489837594 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe3bc066f60 msgr2=0x7fe3bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:28.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.726+0000 7fe3d5460640 1 --2- 172.21.15.83:0/1489837594 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe3bc066f60 0x7fe3bc069420 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000d880 tx=0x7fe3c000d260 comp rx=0 tx=0).stop 2024-04-17T10:48:28.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.726+0000 7fe3d5460640 1 -- 172.21.15.83:0/1489837594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3d00b19a0 msgr2=0x7fe3d00b1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:28.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.726+0000 7fe3d5460640 1 --2- 172.21.15.83:0/1489837594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3d00b19a0 0x7fe3d00b1e50 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fe3c801c590 tx=0x7fe3b8005840 comp rx=0 tx=0).stop 2024-04-17T10:48:28.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.726+0000 7fe3d5460640 1 -- 172.21.15.83:0/1489837594 shutdown_connections 2024-04-17T10:48:28.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.726+0000 7fe3d5460640 1 --2- 172.21.15.83:0/1489837594 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe3bc066f60 0x7fe3bc069420 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:28.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.726+0000 7fe3d5460640 1 --2- 172.21.15.83:0/1489837594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3d00b19a0 0x7fe3d00b1e50 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:28.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.726+0000 7fe3d5460640 1 --2- 172.21.15.83:0/1489837594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3d0156870 0x7fe3d00a9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:28.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.726+0000 7fe3d5460640 1 --2- 172.21.15.83:0/1489837594 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3b80051d0 0x7fe3d015c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:28.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.726+0000 7fe3d5460640 1 -- 172.21.15.83:0/1489837594 >> 172.21.15.83:0/1489837594 conn(0x7fe3d006abe0 msgr2=0x7fe3d00b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:28.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.726+0000 7fe3d5460640 1 -- 172.21.15.83:0/1489837594 shutdown_connections 2024-04-17T10:48:28.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.726+0000 7fe3d5460640 1 -- 172.21.15.83:0/1489837594 wait complete. 2024-04-17T10:48:28.739 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":422,"stamp":"2024-04-17T10:48:28.363691+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":537644,"kb_used_data":11944,"kb_used_omap":12,"kb_used_meta":525555,"kb_avail":754437076,"statfs":{"total":773094113280,"available":772543565824,"internally_reserved":0,"allocated":12230656,"data_stored":5958136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":538168880},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.003886"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.387123+0000","last_change":"2024-04-17T10:48:24.387123+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.387123+0000","last_undegraded":"2024-04-17T10:48:24.387123+0000","last_fullsized":"2024-04-17T10:48:24.387123+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.370522+0000","last_change":"2024-04-17T10:48:24.370522+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.370522+0000","last_undegraded":"2024-04-17T10:48:24.370522+0000","last_fullsized":"2024-04-17T10:48:24.370522+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.379509+0000","last_change":"2024-04-17T10:48:24.379509+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.379509+0000","last_undegraded":"2024-04-17T10:48:24.379509+0000","last_fullsized":"2024-04-17T10:48:24.379509+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.387122+0000","last_change":"2024-04-17T10:48:24.387122+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.387122+0000","last_undegraded":"2024-04-17T10:48:24.387122+0000","last_fullsized":"2024-04-17T10:48:24.387122+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.380195+0000","last_change":"2024-04-17T10:48:24.380195+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.380195+0000","last_undegraded":"2024-04-17T10:48:24.380195+0000","last_fullsized":"2024-04-17T10:48:24.380195+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.378608+0000","last_change":"2024-04-17T10:48:24.378608+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.378608+0000","last_undegraded":"2024-04-17T10:48:24.378608+0000","last_fullsized":"2024-04-17T10:48:24.378608+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.384126+0000","last_change":"2024-04-17T10:48:24.384126+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.384126+0000","last_undegraded":"2024-04-17T10:48:24.384126+0000","last_fullsized":"2024-04-17T10:48:24.384126+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.380148+0000","last_change":"2024-04-17T10:48:24.380148+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.380148+0000","last_undegraded":"2024-04-17T10:48:24.380148+0000","last_fullsized":"2024-04-17T10:48:24.380148+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.386611+0000","last_change":"2024-04-17T10:48:24.386611+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.386611+0000","last_undegraded":"2024-04-17T10:48:24.386611+0000","last_fullsized":"2024-04-17T10:48:24.386611+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.389198+0000","last_change":"2024-04-17T10:48:24.389198+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.389198+0000","last_undegraded":"2024-04-17T10:48:24.389198+0000","last_fullsized":"2024-04-17T10:48:24.389198+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.370124+0000","last_change":"2024-04-17T10:48:24.370124+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.370124+0000","last_undegraded":"2024-04-17T10:48:24.370124+0000","last_fullsized":"2024-04-17T10:48:24.370124+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.386065+0000","last_change":"2024-04-17T10:48:24.386065+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.386065+0000","last_undegraded":"2024-04-17T10:48:24.386065+0000","last_fullsized":"2024-04-17T10:48:24.386065+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.374379+0000","last_change":"2024-04-17T10:48:24.374379+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.374379+0000","last_undegraded":"2024-04-17T10:48:24.374379+0000","last_fullsized":"2024-04-17T10:48:24.374379+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.376316+0000","last_change":"2024-04-17T10:48:24.376316+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.376316+0000","last_undegraded":"2024-04-17T10:48:24.376316+0000","last_fullsized":"2024-04-17T10:48:24.376316+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.375310+0000","last_change":"2024-04-17T10:48:24.375310+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.375310+0000","last_undegraded":"2024-04-17T10:48:24.375310+0000","last_fullsized":"2024-04-17T10:48:24.375310+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.385989+0000","last_change":"2024-04-17T10:48:24.385989+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.385989+0000","last_undegraded":"2024-04-17T10:48:24.385989+0000","last_fullsized":"2024-04-17T10:48:24.385989+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.373575+0000","last_change":"2024-04-17T10:48:24.373575+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.373575+0000","last_undegraded":"2024-04-17T10:48:24.373575+0000","last_fullsized":"2024-04-17T10:48:24.373575+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.385756+0000","last_change":"2024-04-17T10:48:24.385756+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.385756+0000","last_undegraded":"2024-04-17T10:48:24.385756+0000","last_fullsized":"2024-04-17T10:48:24.385756+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.378920+0000","last_change":"2024-04-17T10:48:24.378920+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.378920+0000","last_undegraded":"2024-04-17T10:48:24.378920+0000","last_fullsized":"2024-04-17T10:48:24.378920+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.371256+0000","last_change":"2024-04-17T10:48:24.371256+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.371256+0000","last_undegraded":"2024-04-17T10:48:24.371256+0000","last_fullsized":"2024-04-17T10:48:24.371256+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.370141+0000","last_change":"2024-04-17T10:48:24.370141+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.370141+0000","last_undegraded":"2024-04-17T10:48:24.370141+0000","last_fullsized":"2024-04-17T10:48:24.370141+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.393879+0000","last_change":"2024-04-17T10:48:24.393879+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.393879+0000","last_undegraded":"2024-04-17T10:48:24.393879+0000","last_fullsized":"2024-04-17T10:48:24.393879+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.373549+0000","last_change":"2024-04-17T10:48:24.373549+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.373549+0000","last_undegraded":"2024-04-17T10:48:24.373549+0000","last_fullsized":"2024-04-17T10:48:24.373549+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.393172+0000","last_change":"2024-04-17T10:48:24.393172+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.393172+0000","last_undegraded":"2024-04-17T10:48:24.393172+0000","last_fullsized":"2024-04-17T10:48:24.393172+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.371705+0000","last_change":"2024-04-17T10:48:24.371705+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.371705+0000","last_undegraded":"2024-04-17T10:48:24.371705+0000","last_fullsized":"2024-04-17T10:48:24.371705+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.386061+0000","last_change":"2024-04-17T10:48:24.386061+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.386061+0000","last_undegraded":"2024-04-17T10:48:24.386061+0000","last_fullsized":"2024-04-17T10:48:24.386061+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.374580+0000","last_change":"2024-04-17T10:48:24.374580+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.374580+0000","last_undegraded":"2024-04-17T10:48:24.374580+0000","last_fullsized":"2024-04-17T10:48:24.374580+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.393499+0000","last_change":"2024-04-17T10:48:24.393499+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.393499+0000","last_undegraded":"2024-04-17T10:48:24.393499+0000","last_fullsized":"2024-04-17T10:48:24.393499+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.372097+0000","last_change":"2024-04-17T10:48:24.372097+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.372097+0000","last_undegraded":"2024-04-17T10:48:24.372097+0000","last_fullsized":"2024-04-17T10:48:24.372097+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.389706+0000","last_change":"2024-04-17T10:48:24.389706+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.389706+0000","last_undegraded":"2024-04-17T10:48:24.389706+0000","last_fullsized":"2024-04-17T10:48:24.389706+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.385558+0000","last_change":"2024-04-17T10:48:24.385558+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.385558+0000","last_undegraded":"2024-04-17T10:48:24.385558+0000","last_fullsized":"2024-04-17T10:48:24.385558+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.376758+0000","last_change":"2024-04-17T10:48:24.376758+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.376758+0000","last_undegraded":"2024-04-17T10:48:24.376758+0000","last_fullsized":"2024-04-17T10:48:24.376758+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.375305+0000","last_change":"2024-04-17T10:48:24.375305+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.375305+0000","last_undegraded":"2024-04-17T10:48:24.375305+0000","last_fullsized":"2024-04-17T10:48:24.375305+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401517+0000","last_change":"2024-04-17T10:48:24.396879+0000","last_active":"2024-04-17T10:48:24.401517+0000","last_peered":"2024-04-17T10:48:24.401517+0000","last_clean":"2024-04-17T10:48:24.401517+0000","last_became_active":"2024-04-17T10:48:24.396126+0000","last_became_peered":"2024-04-17T10:48:24.396126+0000","last_unstale":"2024-04-17T10:48:24.401517+0000","last_undegraded":"2024-04-17T10:48:24.401517+0000","last_fullsized":"2024-04-17T10:48:24.401517+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:35:03.367753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.376492+0000","last_change":"2024-04-17T10:48:24.376492+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.376492+0000","last_undegraded":"2024-04-17T10:48:24.376492+0000","last_fullsized":"2024-04-17T10:48:24.376492+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.399197+0000","last_change":"2024-04-17T10:48:24.398459+0000","last_active":"2024-04-17T10:48:24.399197+0000","last_peered":"2024-04-17T10:48:24.399197+0000","last_clean":"2024-04-17T10:48:24.399197+0000","last_became_active":"2024-04-17T10:48:24.398098+0000","last_became_peered":"2024-04-17T10:48:24.398098+0000","last_unstale":"2024-04-17T10:48:24.399197+0000","last_undegraded":"2024-04-17T10:48:24.399197+0000","last_fullsized":"2024-04-17T10:48:24.399197+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:23:07.107332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.379331+0000","last_change":"2024-04-17T10:48:24.379331+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.379331+0000","last_undegraded":"2024-04-17T10:48:24.379331+0000","last_fullsized":"2024-04-17T10:48:24.379331+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.402080+0000","last_change":"2024-04-17T10:48:24.400890+0000","last_active":"2024-04-17T10:48:24.402080+0000","last_peered":"2024-04-17T10:48:24.402080+0000","last_clean":"2024-04-17T10:48:24.402080+0000","last_became_active":"2024-04-17T10:48:24.400472+0000","last_became_peered":"2024-04-17T10:48:24.400472+0000","last_unstale":"2024-04-17T10:48:24.402080+0000","last_undegraded":"2024-04-17T10:48:24.402080+0000","last_fullsized":"2024-04-17T10:48:24.402080+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:20:02.731934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.376494+0000","last_change":"2024-04-17T10:48:24.376494+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.376494+0000","last_undegraded":"2024-04-17T10:48:24.376494+0000","last_fullsized":"2024-04-17T10:48:24.376494+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.402280+0000","last_change":"2024-04-17T10:48:24.401643+0000","last_active":"2024-04-17T10:48:24.402280+0000","last_peered":"2024-04-17T10:48:24.402280+0000","last_clean":"2024-04-17T10:48:24.402280+0000","last_became_active":"2024-04-17T10:48:24.400656+0000","last_became_peered":"2024-04-17T10:48:24.400656+0000","last_unstale":"2024-04-17T10:48:24.402280+0000","last_undegraded":"2024-04-17T10:48:24.402280+0000","last_fullsized":"2024-04-17T10:48:24.402280+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:11:46.753399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.385378+0000","last_change":"2024-04-17T10:48:24.385378+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.385378+0000","last_undegraded":"2024-04-17T10:48:24.385378+0000","last_fullsized":"2024-04-17T10:48:24.385378+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.404934+0000","last_change":"2024-04-17T10:48:24.403866+0000","last_active":"2024-04-17T10:48:24.404934+0000","last_peered":"2024-04-17T10:48:24.404934+0000","last_clean":"2024-04-17T10:48:24.404934+0000","last_became_active":"2024-04-17T10:48:24.403129+0000","last_became_peered":"2024-04-17T10:48:24.403129+0000","last_unstale":"2024-04-17T10:48:24.404934+0000","last_undegraded":"2024-04-17T10:48:24.404934+0000","last_fullsized":"2024-04-17T10:48:24.404934+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:34:15.196696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.371493+0000","last_change":"2024-04-17T10:48:24.371493+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.371493+0000","last_undegraded":"2024-04-17T10:48:24.371493+0000","last_fullsized":"2024-04-17T10:48:24.371493+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.402426+0000","last_change":"2024-04-17T10:48:24.399589+0000","last_active":"2024-04-17T10:48:24.402426+0000","last_peered":"2024-04-17T10:48:24.402426+0000","last_clean":"2024-04-17T10:48:24.402426+0000","last_became_active":"2024-04-17T10:48:24.399097+0000","last_became_peered":"2024-04-17T10:48:24.399097+0000","last_unstale":"2024-04-17T10:48:24.402426+0000","last_undegraded":"2024-04-17T10:48:24.402426+0000","last_fullsized":"2024-04-17T10:48:24.402426+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:01:53.668168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.385357+0000","last_change":"2024-04-17T10:48:24.385357+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.385357+0000","last_undegraded":"2024-04-17T10:48:24.385357+0000","last_fullsized":"2024-04-17T10:48:24.385357+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.379620+0000","last_change":"2024-04-17T10:48:24.376241+0000","last_active":"2024-04-17T10:48:24.379620+0000","last_peered":"2024-04-17T10:48:24.379620+0000","last_clean":"2024-04-17T10:48:24.379620+0000","last_became_active":"2024-04-17T10:48:24.375430+0000","last_became_peered":"2024-04-17T10:48:24.375430+0000","last_unstale":"2024-04-17T10:48:24.379620+0000","last_undegraded":"2024-04-17T10:48:24.379620+0000","last_fullsized":"2024-04-17T10:48:24.379620+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:46:07.697646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.384767+0000","last_change":"2024-04-17T10:48:24.384767+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.384767+0000","last_undegraded":"2024-04-17T10:48:24.384767+0000","last_fullsized":"2024-04-17T10:48:24.384767+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.385628+0000","last_change":"2024-04-17T10:48:24.382363+0000","last_active":"2024-04-17T10:48:24.385628+0000","last_peered":"2024-04-17T10:48:24.385628+0000","last_clean":"2024-04-17T10:48:24.385628+0000","last_became_active":"2024-04-17T10:48:24.381493+0000","last_became_peered":"2024-04-17T10:48:24.381493+0000","last_unstale":"2024-04-17T10:48:24.385628+0000","last_undegraded":"2024-04-17T10:48:24.385628+0000","last_fullsized":"2024-04-17T10:48:24.385628+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:06:35.936896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.389344+0000","last_change":"2024-04-17T10:48:24.389344+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.389344+0000","last_undegraded":"2024-04-17T10:48:24.389344+0000","last_fullsized":"2024-04-17T10:48:24.389344+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401814+0000","last_change":"2024-04-17T10:48:24.400839+0000","last_active":"2024-04-17T10:48:24.401814+0000","last_peered":"2024-04-17T10:48:24.401814+0000","last_clean":"2024-04-17T10:48:24.401814+0000","last_became_active":"2024-04-17T10:48:24.400489+0000","last_became_peered":"2024-04-17T10:48:24.400489+0000","last_unstale":"2024-04-17T10:48:24.401814+0000","last_undegraded":"2024-04-17T10:48:24.401814+0000","last_fullsized":"2024-04-17T10:48:24.401814+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:42:13.356381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.375839+0000","last_change":"2024-04-17T10:48:24.375839+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.375839+0000","last_undegraded":"2024-04-17T10:48:24.375839+0000","last_fullsized":"2024-04-17T10:48:24.375839+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.402374+0000","last_change":"2024-04-17T10:48:24.399209+0000","last_active":"2024-04-17T10:48:24.402374+0000","last_peered":"2024-04-17T10:48:24.402374+0000","last_clean":"2024-04-17T10:48:24.402374+0000","last_became_active":"2024-04-17T10:48:24.398789+0000","last_became_peered":"2024-04-17T10:48:24.398789+0000","last_unstale":"2024-04-17T10:48:24.402374+0000","last_undegraded":"2024-04-17T10:48:24.402374+0000","last_fullsized":"2024-04-17T10:48:24.402374+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:33:31.610895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.391756+0000","last_change":"2024-04-17T10:48:24.391756+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.391756+0000","last_undegraded":"2024-04-17T10:48:24.391756+0000","last_fullsized":"2024-04-17T10:48:24.391756+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.402960+0000","last_change":"2024-04-17T10:48:24.402346+0000","last_active":"2024-04-17T10:48:24.402960+0000","last_peered":"2024-04-17T10:48:24.402960+0000","last_clean":"2024-04-17T10:48:24.402960+0000","last_became_active":"2024-04-17T10:48:24.401900+0000","last_became_peered":"2024-04-17T10:48:24.401900+0000","last_unstale":"2024-04-17T10:48:24.402960+0000","last_undegraded":"2024-04-17T10:48:24.402960+0000","last_fullsized":"2024-04-17T10:48:24.402960+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:23:07.107332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.390503+0000","last_change":"2024-04-17T10:48:24.390503+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.390503+0000","last_undegraded":"2024-04-17T10:48:24.390503+0000","last_fullsized":"2024-04-17T10:48:24.390503+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.382233+0000","last_change":"2024-04-17T10:48:24.379794+0000","last_active":"2024-04-17T10:48:24.382233+0000","last_peered":"2024-04-17T10:48:24.382233+0000","last_clean":"2024-04-17T10:48:24.382233+0000","last_became_active":"2024-04-17T10:48:24.379413+0000","last_became_peered":"2024-04-17T10:48:24.379413+0000","last_unstale":"2024-04-17T10:48:24.382233+0000","last_undegraded":"2024-04-17T10:48:24.382233+0000","last_fullsized":"2024-04-17T10:48:24.382233+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:42:10.087214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.391471+0000","last_change":"2024-04-17T10:48:24.391471+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.391471+0000","last_undegraded":"2024-04-17T10:48:24.391471+0000","last_fullsized":"2024-04-17T10:48:24.391471+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401761+0000","last_change":"2024-04-17T10:48:24.400108+0000","last_active":"2024-04-17T10:48:24.401761+0000","last_peered":"2024-04-17T10:48:24.401761+0000","last_clean":"2024-04-17T10:48:24.401761+0000","last_became_active":"2024-04-17T10:48:24.397908+0000","last_became_peered":"2024-04-17T10:48:24.397908+0000","last_unstale":"2024-04-17T10:48:24.401761+0000","last_undegraded":"2024-04-17T10:48:24.401761+0000","last_fullsized":"2024-04-17T10:48:24.401761+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:11:46.753399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.393082+0000","last_change":"2024-04-17T10:48:24.393082+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.393082+0000","last_undegraded":"2024-04-17T10:48:24.393082+0000","last_fullsized":"2024-04-17T10:48:24.393082+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.392836+0000","last_change":"2024-04-17T10:48:24.391544+0000","last_active":"2024-04-17T10:48:24.392836+0000","last_peered":"2024-04-17T10:48:24.392836+0000","last_clean":"2024-04-17T10:48:24.392836+0000","last_became_active":"2024-04-17T10:48:24.388072+0000","last_became_peered":"2024-04-17T10:48:24.388072+0000","last_unstale":"2024-04-17T10:48:24.392836+0000","last_undegraded":"2024-04-17T10:48:24.392836+0000","last_fullsized":"2024-04-17T10:48:24.392836+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:03:44.300221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.392104+0000","last_change":"2024-04-17T10:48:24.392104+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.392104+0000","last_undegraded":"2024-04-17T10:48:24.392104+0000","last_fullsized":"2024-04-17T10:48:24.392104+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.389451+0000","last_change":"2024-04-17T10:48:24.383457+0000","last_active":"2024-04-17T10:48:24.389451+0000","last_peered":"2024-04-17T10:48:24.389451+0000","last_clean":"2024-04-17T10:48:24.389451+0000","last_became_active":"2024-04-17T10:48:24.382932+0000","last_became_peered":"2024-04-17T10:48:24.382932+0000","last_unstale":"2024-04-17T10:48:24.389451+0000","last_undegraded":"2024-04-17T10:48:24.389451+0000","last_fullsized":"2024-04-17T10:48:24.389451+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:53:06.465373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.376491+0000","last_change":"2024-04-17T10:48:24.376491+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.376491+0000","last_undegraded":"2024-04-17T10:48:24.376491+0000","last_fullsized":"2024-04-17T10:48:24.376491+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.399747+0000","last_change":"2024-04-17T10:48:24.399014+0000","last_active":"2024-04-17T10:48:24.399747+0000","last_peered":"2024-04-17T10:48:24.399747+0000","last_clean":"2024-04-17T10:48:24.399747+0000","last_became_active":"2024-04-17T10:48:24.398705+0000","last_became_peered":"2024-04-17T10:48:24.398705+0000","last_unstale":"2024-04-17T10:48:24.399747+0000","last_undegraded":"2024-04-17T10:48:24.399747+0000","last_fullsized":"2024-04-17T10:48:24.399747+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:01:53.668168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.374704+0000","last_change":"2024-04-17T10:48:24.374704+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.374704+0000","last_undegraded":"2024-04-17T10:48:24.374704+0000","last_fullsized":"2024-04-17T10:48:24.374704+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.390926+0000","last_change":"2024-04-17T10:48:24.387073+0000","last_active":"2024-04-17T10:48:24.390926+0000","last_peered":"2024-04-17T10:48:24.390926+0000","last_clean":"2024-04-17T10:48:24.390926+0000","last_became_active":"2024-04-17T10:48:24.386344+0000","last_became_peered":"2024-04-17T10:48:24.386344+0000","last_unstale":"2024-04-17T10:48:24.390926+0000","last_undegraded":"2024-04-17T10:48:24.390926+0000","last_fullsized":"2024-04-17T10:48:24.390926+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:09:38.552029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.373427+0000","last_change":"2024-04-17T10:48:24.373427+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.373427+0000","last_undegraded":"2024-04-17T10:48:24.373427+0000","last_fullsized":"2024-04-17T10:48:24.373427+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.385549+0000","last_change":"2024-04-17T10:48:24.382682+0000","last_active":"2024-04-17T10:48:24.385549+0000","last_peered":"2024-04-17T10:48:24.385549+0000","last_clean":"2024-04-17T10:48:24.385549+0000","last_became_active":"2024-04-17T10:48:24.381331+0000","last_became_peered":"2024-04-17T10:48:24.381331+0000","last_unstale":"2024-04-17T10:48:24.385549+0000","last_undegraded":"2024-04-17T10:48:24.385549+0000","last_fullsized":"2024-04-17T10:48:24.385549+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:15:43.459938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.373479+0000","last_change":"2024-04-17T10:48:24.373479+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.373479+0000","last_undegraded":"2024-04-17T10:48:24.373479+0000","last_fullsized":"2024-04-17T10:48:24.373479+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401395+0000","last_change":"2024-04-17T10:48:24.396200+0000","last_active":"2024-04-17T10:48:24.401395+0000","last_peered":"2024-04-17T10:48:24.401395+0000","last_clean":"2024-04-17T10:48:24.401395+0000","last_became_active":"2024-04-17T10:48:24.395802+0000","last_became_peered":"2024-04-17T10:48:24.395802+0000","last_unstale":"2024-04-17T10:48:24.401395+0000","last_undegraded":"2024-04-17T10:48:24.401395+0000","last_fullsized":"2024-04-17T10:48:24.401395+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:13:48.163535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.374303+0000","last_change":"2024-04-17T10:48:24.374303+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.374303+0000","last_undegraded":"2024-04-17T10:48:24.374303+0000","last_fullsized":"2024-04-17T10:48:24.374303+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401739+0000","last_change":"2024-04-17T10:48:24.397774+0000","last_active":"2024-04-17T10:48:24.401739+0000","last_peered":"2024-04-17T10:48:24.401739+0000","last_clean":"2024-04-17T10:48:24.401739+0000","last_became_active":"2024-04-17T10:48:24.397453+0000","last_became_peered":"2024-04-17T10:48:24.397453+0000","last_unstale":"2024-04-17T10:48:24.401739+0000","last_undegraded":"2024-04-17T10:48:24.401739+0000","last_fullsized":"2024-04-17T10:48:24.401739+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:20:23.871607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.384650+0000","last_change":"2024-04-17T10:48:24.384650+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.384650+0000","last_undegraded":"2024-04-17T10:48:24.384650+0000","last_fullsized":"2024-04-17T10:48:24.384650+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401589+0000","last_change":"2024-04-17T10:48:24.400725+0000","last_active":"2024-04-17T10:48:24.401589+0000","last_peered":"2024-04-17T10:48:24.401589+0000","last_clean":"2024-04-17T10:48:24.401589+0000","last_became_active":"2024-04-17T10:48:24.400444+0000","last_became_peered":"2024-04-17T10:48:24.400444+0000","last_unstale":"2024-04-17T10:48:24.401589+0000","last_undegraded":"2024-04-17T10:48:24.401589+0000","last_fullsized":"2024-04-17T10:48:24.401589+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:05:55.306345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.379208+0000","last_change":"2024-04-17T10:48:24.379208+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.379208+0000","last_undegraded":"2024-04-17T10:48:24.379208+0000","last_fullsized":"2024-04-17T10:48:24.379208+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401440+0000","last_change":"2024-04-17T10:48:24.397713+0000","last_active":"2024-04-17T10:48:24.401440+0000","last_peered":"2024-04-17T10:48:24.401440+0000","last_clean":"2024-04-17T10:48:24.401440+0000","last_became_active":"2024-04-17T10:48:24.397264+0000","last_became_peered":"2024-04-17T10:48:24.397264+0000","last_unstale":"2024-04-17T10:48:24.401440+0000","last_undegraded":"2024-04-17T10:48:24.401440+0000","last_fullsized":"2024-04-17T10:48:24.401440+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:20:02.731934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.390331+0000","last_change":"2024-04-17T10:48:24.390331+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.390331+0000","last_undegraded":"2024-04-17T10:48:24.390331+0000","last_fullsized":"2024-04-17T10:48:24.390331+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.395732+0000","last_change":"2024-04-17T10:48:24.393648+0000","last_active":"2024-04-17T10:48:24.395732+0000","last_peered":"2024-04-17T10:48:24.395732+0000","last_clean":"2024-04-17T10:48:24.395732+0000","last_became_active":"2024-04-17T10:48:24.392862+0000","last_became_peered":"2024-04-17T10:48:24.392862+0000","last_unstale":"2024-04-17T10:48:24.395732+0000","last_undegraded":"2024-04-17T10:48:24.395732+0000","last_fullsized":"2024-04-17T10:48:24.395732+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:33:31.610895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.384853+0000","last_change":"2024-04-17T10:48:24.384853+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.384853+0000","last_undegraded":"2024-04-17T10:48:24.384853+0000","last_fullsized":"2024-04-17T10:48:24.384853+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.394096+0000","last_change":"2024-04-17T10:48:24.389637+0000","last_active":"2024-04-17T10:48:24.394096+0000","last_peered":"2024-04-17T10:48:24.394096+0000","last_clean":"2024-04-17T10:48:24.394096+0000","last_became_active":"2024-04-17T10:48:24.389212+0000","last_became_peered":"2024-04-17T10:48:24.389212+0000","last_unstale":"2024-04-17T10:48:24.394096+0000","last_undegraded":"2024-04-17T10:48:24.394096+0000","last_fullsized":"2024-04-17T10:48:24.394096+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:36:28.852444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.384672+0000","last_change":"2024-04-17T10:48:24.384672+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.384672+0000","last_undegraded":"2024-04-17T10:48:24.384672+0000","last_fullsized":"2024-04-17T10:48:24.384672+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.400185+0000","last_change":"2024-04-17T10:48:24.398460+0000","last_active":"2024-04-17T10:48:24.400185+0000","last_peered":"2024-04-17T10:48:24.400185+0000","last_clean":"2024-04-17T10:48:24.400185+0000","last_became_active":"2024-04-17T10:48:24.397860+0000","last_became_peered":"2024-04-17T10:48:24.397860+0000","last_unstale":"2024-04-17T10:48:24.400185+0000","last_undegraded":"2024-04-17T10:48:24.400185+0000","last_fullsized":"2024-04-17T10:48:24.400185+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:20:18.572385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.384643+0000","last_change":"2024-04-17T10:48:24.384643+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.384643+0000","last_undegraded":"2024-04-17T10:48:24.384643+0000","last_fullsized":"2024-04-17T10:48:24.384643+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.395517+0000","last_change":"2024-04-17T10:48:24.391043+0000","last_active":"2024-04-17T10:48:24.395517+0000","last_peered":"2024-04-17T10:48:24.395517+0000","last_clean":"2024-04-17T10:48:24.395517+0000","last_became_active":"2024-04-17T10:48:24.390673+0000","last_became_peered":"2024-04-17T10:48:24.390673+0000","last_unstale":"2024-04-17T10:48:24.395517+0000","last_undegraded":"2024-04-17T10:48:24.395517+0000","last_fullsized":"2024-04-17T10:48:24.395517+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:36:28.852444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.375853+0000","last_change":"2024-04-17T10:48:24.375853+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.375853+0000","last_undegraded":"2024-04-17T10:48:24.375853+0000","last_fullsized":"2024-04-17T10:48:24.375853+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.400107+0000","last_change":"2024-04-17T10:48:24.398215+0000","last_active":"2024-04-17T10:48:24.400107+0000","last_peered":"2024-04-17T10:48:24.400107+0000","last_clean":"2024-04-17T10:48:24.400107+0000","last_became_active":"2024-04-17T10:48:24.394779+0000","last_became_peered":"2024-04-17T10:48:24.394779+0000","last_unstale":"2024-04-17T10:48:24.400107+0000","last_undegraded":"2024-04-17T10:48:24.400107+0000","last_fullsized":"2024-04-17T10:48:24.400107+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:54:47.216960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.370200+0000","last_change":"2024-04-17T10:48:24.370200+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.370200+0000","last_undegraded":"2024-04-17T10:48:24.370200+0000","last_fullsized":"2024-04-17T10:48:24.370200+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401418+0000","last_change":"2024-04-17T10:48:24.396294+0000","last_active":"2024-04-17T10:48:24.401418+0000","last_peered":"2024-04-17T10:48:24.401418+0000","last_clean":"2024-04-17T10:48:24.401418+0000","last_became_active":"2024-04-17T10:48:24.395673+0000","last_became_peered":"2024-04-17T10:48:24.395673+0000","last_unstale":"2024-04-17T10:48:24.401418+0000","last_undegraded":"2024-04-17T10:48:24.401418+0000","last_fullsized":"2024-04-17T10:48:24.401418+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:16:43.238162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.370856+0000","last_change":"2024-04-17T10:48:24.370856+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.370856+0000","last_undegraded":"2024-04-17T10:48:24.370856+0000","last_fullsized":"2024-04-17T10:48:24.370856+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.387789+0000","last_change":"2024-04-17T10:48:24.386727+0000","last_active":"2024-04-17T10:48:24.387789+0000","last_peered":"2024-04-17T10:48:24.387789+0000","last_clean":"2024-04-17T10:48:24.387789+0000","last_became_active":"2024-04-17T10:48:24.386334+0000","last_became_peered":"2024-04-17T10:48:24.386334+0000","last_unstale":"2024-04-17T10:48:24.387789+0000","last_undegraded":"2024-04-17T10:48:24.387789+0000","last_fullsized":"2024-04-17T10:48:24.387789+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:09.646139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.378187+0000","last_change":"2024-04-17T10:48:24.378187+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.378187+0000","last_undegraded":"2024-04-17T10:48:24.378187+0000","last_fullsized":"2024-04-17T10:48:24.378187+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.402062+0000","last_change":"2024-04-17T10:48:24.401190+0000","last_active":"2024-04-17T10:48:24.402062+0000","last_peered":"2024-04-17T10:48:24.402062+0000","last_clean":"2024-04-17T10:48:24.402062+0000","last_became_active":"2024-04-17T10:48:24.400909+0000","last_became_peered":"2024-04-17T10:48:24.400909+0000","last_unstale":"2024-04-17T10:48:24.402062+0000","last_undegraded":"2024-04-17T10:48:24.402062+0000","last_fullsized":"2024-04-17T10:48:24.402062+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:20:23.871607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.383491+0000","last_change":"2024-04-17T10:48:24.383491+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.383491+0000","last_undegraded":"2024-04-17T10:48:24.383491+0000","last_fullsized":"2024-04-17T10:48:24.383491+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.395697+0000","last_change":"2024-04-17T10:48:24.393593+0000","last_active":"2024-04-17T10:48:24.395697+0000","last_peered":"2024-04-17T10:48:24.395697+0000","last_clean":"2024-04-17T10:48:24.395697+0000","last_became_active":"2024-04-17T10:48:24.392876+0000","last_became_peered":"2024-04-17T10:48:24.392876+0000","last_unstale":"2024-04-17T10:48:24.395697+0000","last_undegraded":"2024-04-17T10:48:24.395697+0000","last_fullsized":"2024-04-17T10:48:24.395697+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:54:47.216960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.390419+0000","last_change":"2024-04-17T10:48:24.390419+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.390419+0000","last_undegraded":"2024-04-17T10:48:24.390419+0000","last_fullsized":"2024-04-17T10:48:24.390419+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401545+0000","last_change":"2024-04-17T10:48:24.396850+0000","last_active":"2024-04-17T10:48:24.401545+0000","last_peered":"2024-04-17T10:48:24.401545+0000","last_clean":"2024-04-17T10:48:24.401545+0000","last_became_active":"2024-04-17T10:48:24.396194+0000","last_became_peered":"2024-04-17T10:48:24.396194+0000","last_unstale":"2024-04-17T10:48:24.401545+0000","last_undegraded":"2024-04-17T10:48:24.401545+0000","last_fullsized":"2024-04-17T10:48:24.401545+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:20:02.139331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":359,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.387430+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:48:24.387430+0000","last_peered":"2024-04-17T10:48:24.387430+0000","last_clean":"2024-04-17T10:48:24.387430+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:48:24.387430+0000","last_undegraded":"2024-04-17T10:48:24.387430+0000","last_fullsized":"2024-04-17T10:48:24.387430+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574977,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59476,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312364,"statfs":{"total":96636764160,"available":96575860736,"internally_reserved":0,"allocated":1974272,"data_stored":1187543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574976,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68624,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303216,"statfs":{"total":96636764160,"available":96566493184,"internally_reserved":0,"allocated":1380352,"data_stored":597175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574977,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54736,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317104,"statfs":{"total":96636764160,"available":96580714496,"internally_reserved":0,"allocated":1380352,"data_stored":597175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68636,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303204,"statfs":{"total":96636764160,"available":96566480896,"internally_reserved":0,"allocated":1380352,"data_stored":597175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574978,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68624,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303216,"statfs":{"total":96636764160,"available":96566493184,"internally_reserved":0,"allocated":1380352,"data_stored":597175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58908,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312932,"statfs":{"total":96636764160,"available":96576442368,"internally_reserved":0,"allocated":1380352,"data_stored":597175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94996,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94276844,"statfs":{"total":96636764160,"available":96539488256,"internally_reserved":0,"allocated":1380352,"data_stored":597175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574976,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63644,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94308196,"statfs":{"total":96636764160,"available":96571592704,"internally_reserved":0,"allocated":1974272,"data_stored":1187543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:48:28.744 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:48:28.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.902+0000 7fe6823cd640 1 Processor -- start 2024-04-17T10:48:28.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.902+0000 7fe6823cd640 1 -- start start 2024-04-17T10:48:28.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.906+0000 7fe6823cd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe67c0cfbe0 0x7fe67c06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:28.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.906+0000 7fe6823cd640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe67c06e320 con 0x7fe67c15ba30 2024-04-17T10:48:28.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.906+0000 7fe6823cd640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe67c06e040 con 0x7fe67c072330 2024-04-17T10:48:28.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.906+0000 7fe6823cd640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe67c071ea0 con 0x7fe67c0cfbe0 2024-04-17T10:48:28.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.906+0000 7fe67bfff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe67c15ba30 0x7fe67c15c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40958/0 (socket says 172.21.15.83:40958) 2024-04-17T10:48:28.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.906+0000 7fe67bfff640 1 -- 172.21.15.83:0/204602646 learned_addr learned my addr 172.21.15.83:0/204602646 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:28.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.906+0000 7fe67affd640 1 -- 172.21.15.83:0/204602646 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1535295502 0 0) 0x7fe67c06e320 con 0x7fe67c15ba30 2024-04-17T10:48:28.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.906+0000 7fe67affd640 1 -- 172.21.15.83:0/204602646 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe664003530 con 0x7fe67c15ba30 2024-04-17T10:48:28.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.906+0000 7fe68096a640 1 --2- 172.21.15.83:0/204602646 >> v2:172.21.15.167:3300/0 conn(0x7fe67c0cfbe0 0x7fe67c06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:28.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.906+0000 7fe67affd640 1 -- 172.21.15.83:0/204602646 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1964704348 0 0) 0x7fe664003530 con 0x7fe67c15ba30 2024-04-17T10:48:28.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.906+0000 7fe67affd640 1 -- 172.21.15.83:0/204602646 >> v2:172.21.15.167:3300/0 conn(0x7fe67c0cfbe0 msgr2=0x7fe67c06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:28.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.906+0000 7fe67affd640 1 --2- 172.21.15.83:0/204602646 >> v2:172.21.15.167:3300/0 conn(0x7fe67c0cfbe0 0x7fe67c06c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:28.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.906+0000 7fe67affd640 1 -- 172.21.15.83:0/204602646 >> v1:172.21.15.167:6789/0 conn(0x7fe67c072330 legacy=0x7fe67c0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:28.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.906+0000 7fe67affd640 1 -- 172.21.15.83:0/204602646 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe67c071850 con 0x7fe67c15ba30 2024-04-17T10:48:28.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.906+0000 7fe67affd640 1 -- 172.21.15.83:0/204602646 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fe66c0016a0 con 0x7fe67c15ba30 2024-04-17T10:48:28.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.906+0000 7fe67affd640 1 -- 172.21.15.83:0/204602646 >> v1:172.21.15.83:6789/0 conn(0x7fe67c15ba30 legacy=0x7fe67c15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:28.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.906+0000 7fe67affd640 1 --2- 172.21.15.83:0/204602646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6640039c0 0x7fe664003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:28.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.906+0000 7fe67affd640 1 --2- 172.21.15.83:0/204602646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe664004810 0x7fe664004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:28.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.906+0000 7fe67affd640 1 --2- 172.21.15.83:0/204602646 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6640051d0 0x7fe66400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:28.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.906+0000 7fe67affd640 1 -- 172.21.15.83:0/204602646 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c071ea0 con 0x7fe6640039c0 2024-04-17T10:48:28.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.906+0000 7fe67affd640 1 -- 172.21.15.83:0/204602646 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66400c170 con 0x7fe664004810 2024-04-17T10:48:28.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.906+0000 7fe67affd640 1 -- 172.21.15.83:0/204602646 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe66400c370 con 0x7fe6640051d0 2024-04-17T10:48:28.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.906+0000 7fe67bfff640 1 --2- 172.21.15.83:0/204602646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6640039c0 0x7fe664003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:28.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.906+0000 7fe67bfff640 1 -- 172.21.15.83:0/204602646 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6640051d0 msgr2=0x7fe66400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:28.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.906+0000 7fe67bfff640 1 --2- 172.21.15.83:0/204602646 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6640051d0 0x7fe66400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:28.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.906+0000 7fe67bfff640 1 -- 172.21.15.83:0/204602646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe664004810 msgr2=0x7fe664004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:28.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.906+0000 7fe67bfff640 1 --2- 172.21.15.83:0/204602646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe664004810 0x7fe664004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:28.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.906+0000 7fe68096a640 1 -- 172.21.15.83:0/204602646 reap_dead start 2024-04-17T10:48:28.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.906+0000 7fe67bfff640 1 -- 172.21.15.83:0/204602646 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe664003530 con 0x7fe6640039c0 2024-04-17T10:48:28.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe67bfff640 1 --2- 172.21.15.83:0/204602646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6640039c0 0x7fe664003de0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fe66c00c210 tx=0x7fe66c01cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e98615631db7994a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:28.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe67affd640 1 -- 172.21.15.83:0/204602646 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c007d20 con 0x7fe6640039c0 2024-04-17T10:48:28.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe67affd640 1 -- 172.21.15.83:0/204602646 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66c0a9020 con 0x7fe6640039c0 2024-04-17T10:48:28.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe67affd640 1 -- 172.21.15.83:0/204602646 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c005b20 con 0x7fe6640039c0 2024-04-17T10:48:28.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe6823cd640 1 -- 172.21.15.83:0/204602646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6640039c0 msgr2=0x7fe664003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:28.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe6823cd640 1 --2- 172.21.15.83:0/204602646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6640039c0 0x7fe664003de0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fe66c00c210 tx=0x7fe66c01cc00 comp rx=0 tx=0).stop 2024-04-17T10:48:28.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe6823cd640 1 -- 172.21.15.83:0/204602646 shutdown_connections 2024-04-17T10:48:28.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe6823cd640 1 --2- 172.21.15.83:0/204602646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6640039c0 0x7fe664003de0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:28.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe6823cd640 1 -- 172.21.15.83:0/204602646 >> 172.21.15.83:0/204602646 conn(0x7fe67c06abe0 msgr2=0x7fe67c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:28.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe6823cd640 1 -- 172.21.15.83:0/204602646 shutdown_connections 2024-04-17T10:48:28.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe6823cd640 1 -- 172.21.15.83:0/204602646 wait complete. 2024-04-17T10:48:28.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe6823cd640 1 Processor -- start 2024-04-17T10:48:28.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe6823cd640 1 -- start start 2024-04-17T10:48:28.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe6823cd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6640039c0 0x7fe67c15a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:28.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe6823cd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe67c151ac0 0x7fe67c151f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:28.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe6823cd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe67c1f4f60 0x7fe67c1f53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:28.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe6823cd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c0cfa60 con 0x7fe67c151ac0 2024-04-17T10:48:28.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe6823cd640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c0cf8e0 con 0x7fe67c1f4f60 2024-04-17T10:48:28.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe6823cd640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c15cd70 con 0x7fe6640039c0 2024-04-17T10:48:28.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe67b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe67c151ac0 0x7fe67c151f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:28.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe67b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe67c151ac0 0x7fe67c151f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44694/0 (socket says 172.21.15.83:44694) 2024-04-17T10:48:28.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe67b7fe640 1 -- 172.21.15.83:0/2185920809 learned_addr learned my addr 172.21.15.83:0/2185920809 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:28.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe68096a640 1 --2- 172.21.15.83:0/2185920809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe67c1f4f60 0x7fe67c1f53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:28.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe67b7fe640 1 -- 172.21.15.83:0/2185920809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6640039c0 msgr2=0x7fe67c15a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:28.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe67b7fe640 1 --2- 172.21.15.83:0/2185920809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6640039c0 0x7fe67c15a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:28.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe67b7fe640 1 -- 172.21.15.83:0/2185920809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe67c1f4f60 msgr2=0x7fe67c1f53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:28.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe67b7fe640 1 --2- 172.21.15.83:0/2185920809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe67c1f4f60 0x7fe67c1f53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:28.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe67b7fe640 1 -- 172.21.15.83:0/2185920809 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe67c1f84f0 con 0x7fe67c151ac0 2024-04-17T10:48:28.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe67b7fe640 1 --2- 172.21.15.83:0/2185920809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe67c151ac0 0x7fe67c151f40 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fe67000c460 tx=0x7fe67000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:28.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe678ff9640 1 -- 172.21.15.83:0/2185920809 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe670017070 con 0x7fe67c151ac0 2024-04-17T10:48:28.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe678ff9640 1 -- 172.21.15.83:0/2185920809 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6700052f0 con 0x7fe67c151ac0 2024-04-17T10:48:28.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe6823cd640 1 -- 172.21.15.83:0/2185920809 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe67c1f87d0 con 0x7fe67c151ac0 2024-04-17T10:48:28.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe678ff9640 1 -- 172.21.15.83:0/2185920809 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe670003840 con 0x7fe67c151ac0 2024-04-17T10:48:28.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.910+0000 7fe6823cd640 1 -- 172.21.15.83:0/2185920809 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe67c1f8d90 con 0x7fe67c151ac0 2024-04-17T10:48:28.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.914+0000 7fe6823cd640 1 -- 172.21.15.83:0/2185920809 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe67c074f90 con 0x7fe67c151ac0 2024-04-17T10:48:28.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.914+0000 7fe678ff9640 1 -- 172.21.15.83:0/2185920809 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe67001a070 con 0x7fe67c151ac0 2024-04-17T10:48:28.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.914+0000 7fe678ff9640 1 --2- 172.21.15.83:0/2185920809 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe668066f60 0x7fe668069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:28.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.914+0000 7fe678ff9640 1 -- 172.21.15.83:0/2185920809 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe670084380 con 0x7fe67c151ac0 2024-04-17T10:48:28.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.914+0000 7fe67bfff640 1 --2- 172.21.15.83:0/2185920809 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe668066f60 0x7fe668069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:28.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.914+0000 7fe67bfff640 1 --2- 172.21.15.83:0/2185920809 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe668066f60 0x7fe668069420 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fe66c0aa000 tx=0x7fe67c06e210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:28.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:28.918+0000 7fe678ff9640 1 -- 172.21.15.83:0/2185920809 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe670055390 con 0x7fe67c151ac0 2024-04-17T10:48:29.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.070+0000 7fe6823cd640 1 -- 172.21.15.83:0/2185920809 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe67c06d5a0 con 0x7fe668066f60 2024-04-17T10:48:29.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.074+0000 7fe678ff9640 1 -- 172.21.15.83:0/2185920809 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247089 (secure 0 0 0) 0x7fe67c06d5a0 con 0x7fe668066f60 2024-04-17T10:48:29.073 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:29.074 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:48:29.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.078+0000 7fe6823cd640 1 -- 172.21.15.83:0/2185920809 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe668066f60 msgr2=0x7fe668069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:29.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.078+0000 7fe6823cd640 1 --2- 172.21.15.83:0/2185920809 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe668066f60 0x7fe668069420 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fe66c0aa000 tx=0x7fe67c06e210 comp rx=0 tx=0).stop 2024-04-17T10:48:29.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.078+0000 7fe6823cd640 1 -- 172.21.15.83:0/2185920809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe67c151ac0 msgr2=0x7fe67c151f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:29.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.078+0000 7fe6823cd640 1 --2- 172.21.15.83:0/2185920809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe67c151ac0 0x7fe67c151f40 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fe67000c460 tx=0x7fe67000c920 comp rx=0 tx=0).stop 2024-04-17T10:48:29.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.078+0000 7fe6823cd640 1 -- 172.21.15.83:0/2185920809 shutdown_connections 2024-04-17T10:48:29.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.078+0000 7fe6823cd640 1 --2- 172.21.15.83:0/2185920809 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe668066f60 0x7fe668069420 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:29.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.078+0000 7fe6823cd640 1 --2- 172.21.15.83:0/2185920809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe67c1f4f60 0x7fe67c1f53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:29.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.078+0000 7fe6823cd640 1 --2- 172.21.15.83:0/2185920809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe67c151ac0 0x7fe67c151f40 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:29.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.078+0000 7fe6823cd640 1 --2- 172.21.15.83:0/2185920809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6640039c0 0x7fe67c15a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:29.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.078+0000 7fe6823cd640 1 -- 172.21.15.83:0/2185920809 >> 172.21.15.83:0/2185920809 conn(0x7fe67c06abe0 msgr2=0x7fe67c15e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:29.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.078+0000 7fe6823cd640 1 -- 172.21.15.83:0/2185920809 shutdown_connections 2024-04-17T10:48:29.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.078+0000 7fe6823cd640 1 -- 172.21.15.83:0/2185920809 wait complete. 2024-04-17T10:48:29.089 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":422,"stamp":"2024-04-17T10:48:28.363691+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":537644,"kb_used_data":11944,"kb_used_omap":12,"kb_used_meta":525555,"kb_avail":754437076,"statfs":{"total":773094113280,"available":772543565824,"internally_reserved":0,"allocated":12230656,"data_stored":5958136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":538168880},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.003886"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.387123+0000","last_change":"2024-04-17T10:48:24.387123+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.387123+0000","last_undegraded":"2024-04-17T10:48:24.387123+0000","last_fullsized":"2024-04-17T10:48:24.387123+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.370522+0000","last_change":"2024-04-17T10:48:24.370522+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.370522+0000","last_undegraded":"2024-04-17T10:48:24.370522+0000","last_fullsized":"2024-04-17T10:48:24.370522+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.379509+0000","last_change":"2024-04-17T10:48:24.379509+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.379509+0000","last_undegraded":"2024-04-17T10:48:24.379509+0000","last_fullsized":"2024-04-17T10:48:24.379509+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.387122+0000","last_change":"2024-04-17T10:48:24.387122+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.387122+0000","last_undegraded":"2024-04-17T10:48:24.387122+0000","last_fullsized":"2024-04-17T10:48:24.387122+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.380195+0000","last_change":"2024-04-17T10:48:24.380195+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.380195+0000","last_undegraded":"2024-04-17T10:48:24.380195+0000","last_fullsized":"2024-04-17T10:48:24.380195+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.378608+0000","last_change":"2024-04-17T10:48:24.378608+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.378608+0000","last_undegraded":"2024-04-17T10:48:24.378608+0000","last_fullsized":"2024-04-17T10:48:24.378608+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.384126+0000","last_change":"2024-04-17T10:48:24.384126+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.384126+0000","last_undegraded":"2024-04-17T10:48:24.384126+0000","last_fullsized":"2024-04-17T10:48:24.384126+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.380148+0000","last_change":"2024-04-17T10:48:24.380148+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.380148+0000","last_undegraded":"2024-04-17T10:48:24.380148+0000","last_fullsized":"2024-04-17T10:48:24.380148+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.386611+0000","last_change":"2024-04-17T10:48:24.386611+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.386611+0000","last_undegraded":"2024-04-17T10:48:24.386611+0000","last_fullsized":"2024-04-17T10:48:24.386611+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.389198+0000","last_change":"2024-04-17T10:48:24.389198+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.389198+0000","last_undegraded":"2024-04-17T10:48:24.389198+0000","last_fullsized":"2024-04-17T10:48:24.389198+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.370124+0000","last_change":"2024-04-17T10:48:24.370124+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.370124+0000","last_undegraded":"2024-04-17T10:48:24.370124+0000","last_fullsized":"2024-04-17T10:48:24.370124+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.386065+0000","last_change":"2024-04-17T10:48:24.386065+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.386065+0000","last_undegraded":"2024-04-17T10:48:24.386065+0000","last_fullsized":"2024-04-17T10:48:24.386065+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.374379+0000","last_change":"2024-04-17T10:48:24.374379+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.374379+0000","last_undegraded":"2024-04-17T10:48:24.374379+0000","last_fullsized":"2024-04-17T10:48:24.374379+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.376316+0000","last_change":"2024-04-17T10:48:24.376316+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.376316+0000","last_undegraded":"2024-04-17T10:48:24.376316+0000","last_fullsized":"2024-04-17T10:48:24.376316+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.375310+0000","last_change":"2024-04-17T10:48:24.375310+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.375310+0000","last_undegraded":"2024-04-17T10:48:24.375310+0000","last_fullsized":"2024-04-17T10:48:24.375310+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.385989+0000","last_change":"2024-04-17T10:48:24.385989+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.385989+0000","last_undegraded":"2024-04-17T10:48:24.385989+0000","last_fullsized":"2024-04-17T10:48:24.385989+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.373575+0000","last_change":"2024-04-17T10:48:24.373575+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.373575+0000","last_undegraded":"2024-04-17T10:48:24.373575+0000","last_fullsized":"2024-04-17T10:48:24.373575+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.385756+0000","last_change":"2024-04-17T10:48:24.385756+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.385756+0000","last_undegraded":"2024-04-17T10:48:24.385756+0000","last_fullsized":"2024-04-17T10:48:24.385756+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.378920+0000","last_change":"2024-04-17T10:48:24.378920+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.378920+0000","last_undegraded":"2024-04-17T10:48:24.378920+0000","last_fullsized":"2024-04-17T10:48:24.378920+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.371256+0000","last_change":"2024-04-17T10:48:24.371256+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.371256+0000","last_undegraded":"2024-04-17T10:48:24.371256+0000","last_fullsized":"2024-04-17T10:48:24.371256+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.370141+0000","last_change":"2024-04-17T10:48:24.370141+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.370141+0000","last_undegraded":"2024-04-17T10:48:24.370141+0000","last_fullsized":"2024-04-17T10:48:24.370141+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.393879+0000","last_change":"2024-04-17T10:48:24.393879+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.393879+0000","last_undegraded":"2024-04-17T10:48:24.393879+0000","last_fullsized":"2024-04-17T10:48:24.393879+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.373549+0000","last_change":"2024-04-17T10:48:24.373549+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.373549+0000","last_undegraded":"2024-04-17T10:48:24.373549+0000","last_fullsized":"2024-04-17T10:48:24.373549+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.393172+0000","last_change":"2024-04-17T10:48:24.393172+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.393172+0000","last_undegraded":"2024-04-17T10:48:24.393172+0000","last_fullsized":"2024-04-17T10:48:24.393172+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.371705+0000","last_change":"2024-04-17T10:48:24.371705+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.371705+0000","last_undegraded":"2024-04-17T10:48:24.371705+0000","last_fullsized":"2024-04-17T10:48:24.371705+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.386061+0000","last_change":"2024-04-17T10:48:24.386061+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.386061+0000","last_undegraded":"2024-04-17T10:48:24.386061+0000","last_fullsized":"2024-04-17T10:48:24.386061+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.374580+0000","last_change":"2024-04-17T10:48:24.374580+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.374580+0000","last_undegraded":"2024-04-17T10:48:24.374580+0000","last_fullsized":"2024-04-17T10:48:24.374580+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.393499+0000","last_change":"2024-04-17T10:48:24.393499+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.393499+0000","last_undegraded":"2024-04-17T10:48:24.393499+0000","last_fullsized":"2024-04-17T10:48:24.393499+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.372097+0000","last_change":"2024-04-17T10:48:24.372097+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.372097+0000","last_undegraded":"2024-04-17T10:48:24.372097+0000","last_fullsized":"2024-04-17T10:48:24.372097+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.389706+0000","last_change":"2024-04-17T10:48:24.389706+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.389706+0000","last_undegraded":"2024-04-17T10:48:24.389706+0000","last_fullsized":"2024-04-17T10:48:24.389706+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.385558+0000","last_change":"2024-04-17T10:48:24.385558+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.385558+0000","last_undegraded":"2024-04-17T10:48:24.385558+0000","last_fullsized":"2024-04-17T10:48:24.385558+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.376758+0000","last_change":"2024-04-17T10:48:24.376758+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.376758+0000","last_undegraded":"2024-04-17T10:48:24.376758+0000","last_fullsized":"2024-04-17T10:48:24.376758+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.375305+0000","last_change":"2024-04-17T10:48:24.375305+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.375305+0000","last_undegraded":"2024-04-17T10:48:24.375305+0000","last_fullsized":"2024-04-17T10:48:24.375305+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401517+0000","last_change":"2024-04-17T10:48:24.396879+0000","last_active":"2024-04-17T10:48:24.401517+0000","last_peered":"2024-04-17T10:48:24.401517+0000","last_clean":"2024-04-17T10:48:24.401517+0000","last_became_active":"2024-04-17T10:48:24.396126+0000","last_became_peered":"2024-04-17T10:48:24.396126+0000","last_unstale":"2024-04-17T10:48:24.401517+0000","last_undegraded":"2024-04-17T10:48:24.401517+0000","last_fullsized":"2024-04-17T10:48:24.401517+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:35:03.367753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.376492+0000","last_change":"2024-04-17T10:48:24.376492+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.376492+0000","last_undegraded":"2024-04-17T10:48:24.376492+0000","last_fullsized":"2024-04-17T10:48:24.376492+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.399197+0000","last_change":"2024-04-17T10:48:24.398459+0000","last_active":"2024-04-17T10:48:24.399197+0000","last_peered":"2024-04-17T10:48:24.399197+0000","last_clean":"2024-04-17T10:48:24.399197+0000","last_became_active":"2024-04-17T10:48:24.398098+0000","last_became_peered":"2024-04-17T10:48:24.398098+0000","last_unstale":"2024-04-17T10:48:24.399197+0000","last_undegraded":"2024-04-17T10:48:24.399197+0000","last_fullsized":"2024-04-17T10:48:24.399197+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:23:07.107332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.379331+0000","last_change":"2024-04-17T10:48:24.379331+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.379331+0000","last_undegraded":"2024-04-17T10:48:24.379331+0000","last_fullsized":"2024-04-17T10:48:24.379331+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.402080+0000","last_change":"2024-04-17T10:48:24.400890+0000","last_active":"2024-04-17T10:48:24.402080+0000","last_peered":"2024-04-17T10:48:24.402080+0000","last_clean":"2024-04-17T10:48:24.402080+0000","last_became_active":"2024-04-17T10:48:24.400472+0000","last_became_peered":"2024-04-17T10:48:24.400472+0000","last_unstale":"2024-04-17T10:48:24.402080+0000","last_undegraded":"2024-04-17T10:48:24.402080+0000","last_fullsized":"2024-04-17T10:48:24.402080+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:20:02.731934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.376494+0000","last_change":"2024-04-17T10:48:24.376494+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.376494+0000","last_undegraded":"2024-04-17T10:48:24.376494+0000","last_fullsized":"2024-04-17T10:48:24.376494+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.402280+0000","last_change":"2024-04-17T10:48:24.401643+0000","last_active":"2024-04-17T10:48:24.402280+0000","last_peered":"2024-04-17T10:48:24.402280+0000","last_clean":"2024-04-17T10:48:24.402280+0000","last_became_active":"2024-04-17T10:48:24.400656+0000","last_became_peered":"2024-04-17T10:48:24.400656+0000","last_unstale":"2024-04-17T10:48:24.402280+0000","last_undegraded":"2024-04-17T10:48:24.402280+0000","last_fullsized":"2024-04-17T10:48:24.402280+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:11:46.753399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.385378+0000","last_change":"2024-04-17T10:48:24.385378+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.385378+0000","last_undegraded":"2024-04-17T10:48:24.385378+0000","last_fullsized":"2024-04-17T10:48:24.385378+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.404934+0000","last_change":"2024-04-17T10:48:24.403866+0000","last_active":"2024-04-17T10:48:24.404934+0000","last_peered":"2024-04-17T10:48:24.404934+0000","last_clean":"2024-04-17T10:48:24.404934+0000","last_became_active":"2024-04-17T10:48:24.403129+0000","last_became_peered":"2024-04-17T10:48:24.403129+0000","last_unstale":"2024-04-17T10:48:24.404934+0000","last_undegraded":"2024-04-17T10:48:24.404934+0000","last_fullsized":"2024-04-17T10:48:24.404934+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:34:15.196696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.371493+0000","last_change":"2024-04-17T10:48:24.371493+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.371493+0000","last_undegraded":"2024-04-17T10:48:24.371493+0000","last_fullsized":"2024-04-17T10:48:24.371493+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.402426+0000","last_change":"2024-04-17T10:48:24.399589+0000","last_active":"2024-04-17T10:48:24.402426+0000","last_peered":"2024-04-17T10:48:24.402426+0000","last_clean":"2024-04-17T10:48:24.402426+0000","last_became_active":"2024-04-17T10:48:24.399097+0000","last_became_peered":"2024-04-17T10:48:24.399097+0000","last_unstale":"2024-04-17T10:48:24.402426+0000","last_undegraded":"2024-04-17T10:48:24.402426+0000","last_fullsized":"2024-04-17T10:48:24.402426+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:01:53.668168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.385357+0000","last_change":"2024-04-17T10:48:24.385357+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.385357+0000","last_undegraded":"2024-04-17T10:48:24.385357+0000","last_fullsized":"2024-04-17T10:48:24.385357+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.379620+0000","last_change":"2024-04-17T10:48:24.376241+0000","last_active":"2024-04-17T10:48:24.379620+0000","last_peered":"2024-04-17T10:48:24.379620+0000","last_clean":"2024-04-17T10:48:24.379620+0000","last_became_active":"2024-04-17T10:48:24.375430+0000","last_became_peered":"2024-04-17T10:48:24.375430+0000","last_unstale":"2024-04-17T10:48:24.379620+0000","last_undegraded":"2024-04-17T10:48:24.379620+0000","last_fullsized":"2024-04-17T10:48:24.379620+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:46:07.697646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.384767+0000","last_change":"2024-04-17T10:48:24.384767+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.384767+0000","last_undegraded":"2024-04-17T10:48:24.384767+0000","last_fullsized":"2024-04-17T10:48:24.384767+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.385628+0000","last_change":"2024-04-17T10:48:24.382363+0000","last_active":"2024-04-17T10:48:24.385628+0000","last_peered":"2024-04-17T10:48:24.385628+0000","last_clean":"2024-04-17T10:48:24.385628+0000","last_became_active":"2024-04-17T10:48:24.381493+0000","last_became_peered":"2024-04-17T10:48:24.381493+0000","last_unstale":"2024-04-17T10:48:24.385628+0000","last_undegraded":"2024-04-17T10:48:24.385628+0000","last_fullsized":"2024-04-17T10:48:24.385628+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:06:35.936896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.389344+0000","last_change":"2024-04-17T10:48:24.389344+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.389344+0000","last_undegraded":"2024-04-17T10:48:24.389344+0000","last_fullsized":"2024-04-17T10:48:24.389344+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401814+0000","last_change":"2024-04-17T10:48:24.400839+0000","last_active":"2024-04-17T10:48:24.401814+0000","last_peered":"2024-04-17T10:48:24.401814+0000","last_clean":"2024-04-17T10:48:24.401814+0000","last_became_active":"2024-04-17T10:48:24.400489+0000","last_became_peered":"2024-04-17T10:48:24.400489+0000","last_unstale":"2024-04-17T10:48:24.401814+0000","last_undegraded":"2024-04-17T10:48:24.401814+0000","last_fullsized":"2024-04-17T10:48:24.401814+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:42:13.356381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.375839+0000","last_change":"2024-04-17T10:48:24.375839+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.375839+0000","last_undegraded":"2024-04-17T10:48:24.375839+0000","last_fullsized":"2024-04-17T10:48:24.375839+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.402374+0000","last_change":"2024-04-17T10:48:24.399209+0000","last_active":"2024-04-17T10:48:24.402374+0000","last_peered":"2024-04-17T10:48:24.402374+0000","last_clean":"2024-04-17T10:48:24.402374+0000","last_became_active":"2024-04-17T10:48:24.398789+0000","last_became_peered":"2024-04-17T10:48:24.398789+0000","last_unstale":"2024-04-17T10:48:24.402374+0000","last_undegraded":"2024-04-17T10:48:24.402374+0000","last_fullsized":"2024-04-17T10:48:24.402374+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:33:31.610895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.391756+0000","last_change":"2024-04-17T10:48:24.391756+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.391756+0000","last_undegraded":"2024-04-17T10:48:24.391756+0000","last_fullsized":"2024-04-17T10:48:24.391756+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.402960+0000","last_change":"2024-04-17T10:48:24.402346+0000","last_active":"2024-04-17T10:48:24.402960+0000","last_peered":"2024-04-17T10:48:24.402960+0000","last_clean":"2024-04-17T10:48:24.402960+0000","last_became_active":"2024-04-17T10:48:24.401900+0000","last_became_peered":"2024-04-17T10:48:24.401900+0000","last_unstale":"2024-04-17T10:48:24.402960+0000","last_undegraded":"2024-04-17T10:48:24.402960+0000","last_fullsized":"2024-04-17T10:48:24.402960+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:23:07.107332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.390503+0000","last_change":"2024-04-17T10:48:24.390503+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.390503+0000","last_undegraded":"2024-04-17T10:48:24.390503+0000","last_fullsized":"2024-04-17T10:48:24.390503+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.382233+0000","last_change":"2024-04-17T10:48:24.379794+0000","last_active":"2024-04-17T10:48:24.382233+0000","last_peered":"2024-04-17T10:48:24.382233+0000","last_clean":"2024-04-17T10:48:24.382233+0000","last_became_active":"2024-04-17T10:48:24.379413+0000","last_became_peered":"2024-04-17T10:48:24.379413+0000","last_unstale":"2024-04-17T10:48:24.382233+0000","last_undegraded":"2024-04-17T10:48:24.382233+0000","last_fullsized":"2024-04-17T10:48:24.382233+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:42:10.087214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.391471+0000","last_change":"2024-04-17T10:48:24.391471+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.391471+0000","last_undegraded":"2024-04-17T10:48:24.391471+0000","last_fullsized":"2024-04-17T10:48:24.391471+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401761+0000","last_change":"2024-04-17T10:48:24.400108+0000","last_active":"2024-04-17T10:48:24.401761+0000","last_peered":"2024-04-17T10:48:24.401761+0000","last_clean":"2024-04-17T10:48:24.401761+0000","last_became_active":"2024-04-17T10:48:24.397908+0000","last_became_peered":"2024-04-17T10:48:24.397908+0000","last_unstale":"2024-04-17T10:48:24.401761+0000","last_undegraded":"2024-04-17T10:48:24.401761+0000","last_fullsized":"2024-04-17T10:48:24.401761+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:11:46.753399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.393082+0000","last_change":"2024-04-17T10:48:24.393082+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.393082+0000","last_undegraded":"2024-04-17T10:48:24.393082+0000","last_fullsized":"2024-04-17T10:48:24.393082+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.392836+0000","last_change":"2024-04-17T10:48:24.391544+0000","last_active":"2024-04-17T10:48:24.392836+0000","last_peered":"2024-04-17T10:48:24.392836+0000","last_clean":"2024-04-17T10:48:24.392836+0000","last_became_active":"2024-04-17T10:48:24.388072+0000","last_became_peered":"2024-04-17T10:48:24.388072+0000","last_unstale":"2024-04-17T10:48:24.392836+0000","last_undegraded":"2024-04-17T10:48:24.392836+0000","last_fullsized":"2024-04-17T10:48:24.392836+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:03:44.300221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.392104+0000","last_change":"2024-04-17T10:48:24.392104+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.392104+0000","last_undegraded":"2024-04-17T10:48:24.392104+0000","last_fullsized":"2024-04-17T10:48:24.392104+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.389451+0000","last_change":"2024-04-17T10:48:24.383457+0000","last_active":"2024-04-17T10:48:24.389451+0000","last_peered":"2024-04-17T10:48:24.389451+0000","last_clean":"2024-04-17T10:48:24.389451+0000","last_became_active":"2024-04-17T10:48:24.382932+0000","last_became_peered":"2024-04-17T10:48:24.382932+0000","last_unstale":"2024-04-17T10:48:24.389451+0000","last_undegraded":"2024-04-17T10:48:24.389451+0000","last_fullsized":"2024-04-17T10:48:24.389451+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:53:06.465373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.376491+0000","last_change":"2024-04-17T10:48:24.376491+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.376491+0000","last_undegraded":"2024-04-17T10:48:24.376491+0000","last_fullsized":"2024-04-17T10:48:24.376491+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.399747+0000","last_change":"2024-04-17T10:48:24.399014+0000","last_active":"2024-04-17T10:48:24.399747+0000","last_peered":"2024-04-17T10:48:24.399747+0000","last_clean":"2024-04-17T10:48:24.399747+0000","last_became_active":"2024-04-17T10:48:24.398705+0000","last_became_peered":"2024-04-17T10:48:24.398705+0000","last_unstale":"2024-04-17T10:48:24.399747+0000","last_undegraded":"2024-04-17T10:48:24.399747+0000","last_fullsized":"2024-04-17T10:48:24.399747+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:01:53.668168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.374704+0000","last_change":"2024-04-17T10:48:24.374704+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.374704+0000","last_undegraded":"2024-04-17T10:48:24.374704+0000","last_fullsized":"2024-04-17T10:48:24.374704+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.390926+0000","last_change":"2024-04-17T10:48:24.387073+0000","last_active":"2024-04-17T10:48:24.390926+0000","last_peered":"2024-04-17T10:48:24.390926+0000","last_clean":"2024-04-17T10:48:24.390926+0000","last_became_active":"2024-04-17T10:48:24.386344+0000","last_became_peered":"2024-04-17T10:48:24.386344+0000","last_unstale":"2024-04-17T10:48:24.390926+0000","last_undegraded":"2024-04-17T10:48:24.390926+0000","last_fullsized":"2024-04-17T10:48:24.390926+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:09:38.552029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.373427+0000","last_change":"2024-04-17T10:48:24.373427+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.373427+0000","last_undegraded":"2024-04-17T10:48:24.373427+0000","last_fullsized":"2024-04-17T10:48:24.373427+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.385549+0000","last_change":"2024-04-17T10:48:24.382682+0000","last_active":"2024-04-17T10:48:24.385549+0000","last_peered":"2024-04-17T10:48:24.385549+0000","last_clean":"2024-04-17T10:48:24.385549+0000","last_became_active":"2024-04-17T10:48:24.381331+0000","last_became_peered":"2024-04-17T10:48:24.381331+0000","last_unstale":"2024-04-17T10:48:24.385549+0000","last_undegraded":"2024-04-17T10:48:24.385549+0000","last_fullsized":"2024-04-17T10:48:24.385549+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:15:43.459938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.373479+0000","last_change":"2024-04-17T10:48:24.373479+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.373479+0000","last_undegraded":"2024-04-17T10:48:24.373479+0000","last_fullsized":"2024-04-17T10:48:24.373479+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401395+0000","last_change":"2024-04-17T10:48:24.396200+0000","last_active":"2024-04-17T10:48:24.401395+0000","last_peered":"2024-04-17T10:48:24.401395+0000","last_clean":"2024-04-17T10:48:24.401395+0000","last_became_active":"2024-04-17T10:48:24.395802+0000","last_became_peered":"2024-04-17T10:48:24.395802+0000","last_unstale":"2024-04-17T10:48:24.401395+0000","last_undegraded":"2024-04-17T10:48:24.401395+0000","last_fullsized":"2024-04-17T10:48:24.401395+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:13:48.163535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.374303+0000","last_change":"2024-04-17T10:48:24.374303+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.374303+0000","last_undegraded":"2024-04-17T10:48:24.374303+0000","last_fullsized":"2024-04-17T10:48:24.374303+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401739+0000","last_change":"2024-04-17T10:48:24.397774+0000","last_active":"2024-04-17T10:48:24.401739+0000","last_peered":"2024-04-17T10:48:24.401739+0000","last_clean":"2024-04-17T10:48:24.401739+0000","last_became_active":"2024-04-17T10:48:24.397453+0000","last_became_peered":"2024-04-17T10:48:24.397453+0000","last_unstale":"2024-04-17T10:48:24.401739+0000","last_undegraded":"2024-04-17T10:48:24.401739+0000","last_fullsized":"2024-04-17T10:48:24.401739+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:20:23.871607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.384650+0000","last_change":"2024-04-17T10:48:24.384650+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.384650+0000","last_undegraded":"2024-04-17T10:48:24.384650+0000","last_fullsized":"2024-04-17T10:48:24.384650+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401589+0000","last_change":"2024-04-17T10:48:24.400725+0000","last_active":"2024-04-17T10:48:24.401589+0000","last_peered":"2024-04-17T10:48:24.401589+0000","last_clean":"2024-04-17T10:48:24.401589+0000","last_became_active":"2024-04-17T10:48:24.400444+0000","last_became_peered":"2024-04-17T10:48:24.400444+0000","last_unstale":"2024-04-17T10:48:24.401589+0000","last_undegraded":"2024-04-17T10:48:24.401589+0000","last_fullsized":"2024-04-17T10:48:24.401589+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:05:55.306345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.379208+0000","last_change":"2024-04-17T10:48:24.379208+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.379208+0000","last_undegraded":"2024-04-17T10:48:24.379208+0000","last_fullsized":"2024-04-17T10:48:24.379208+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401440+0000","last_change":"2024-04-17T10:48:24.397713+0000","last_active":"2024-04-17T10:48:24.401440+0000","last_peered":"2024-04-17T10:48:24.401440+0000","last_clean":"2024-04-17T10:48:24.401440+0000","last_became_active":"2024-04-17T10:48:24.397264+0000","last_became_peered":"2024-04-17T10:48:24.397264+0000","last_unstale":"2024-04-17T10:48:24.401440+0000","last_undegraded":"2024-04-17T10:48:24.401440+0000","last_fullsized":"2024-04-17T10:48:24.401440+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:20:02.731934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.390331+0000","last_change":"2024-04-17T10:48:24.390331+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.390331+0000","last_undegraded":"2024-04-17T10:48:24.390331+0000","last_fullsized":"2024-04-17T10:48:24.390331+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.395732+0000","last_change":"2024-04-17T10:48:24.393648+0000","last_active":"2024-04-17T10:48:24.395732+0000","last_peered":"2024-04-17T10:48:24.395732+0000","last_clean":"2024-04-17T10:48:24.395732+0000","last_became_active":"2024-04-17T10:48:24.392862+0000","last_became_peered":"2024-04-17T10:48:24.392862+0000","last_unstale":"2024-04-17T10:48:24.395732+0000","last_undegraded":"2024-04-17T10:48:24.395732+0000","last_fullsized":"2024-04-17T10:48:24.395732+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:33:31.610895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.384853+0000","last_change":"2024-04-17T10:48:24.384853+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.384853+0000","last_undegraded":"2024-04-17T10:48:24.384853+0000","last_fullsized":"2024-04-17T10:48:24.384853+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.394096+0000","last_change":"2024-04-17T10:48:24.389637+0000","last_active":"2024-04-17T10:48:24.394096+0000","last_peered":"2024-04-17T10:48:24.394096+0000","last_clean":"2024-04-17T10:48:24.394096+0000","last_became_active":"2024-04-17T10:48:24.389212+0000","last_became_peered":"2024-04-17T10:48:24.389212+0000","last_unstale":"2024-04-17T10:48:24.394096+0000","last_undegraded":"2024-04-17T10:48:24.394096+0000","last_fullsized":"2024-04-17T10:48:24.394096+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:36:28.852444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.384672+0000","last_change":"2024-04-17T10:48:24.384672+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.384672+0000","last_undegraded":"2024-04-17T10:48:24.384672+0000","last_fullsized":"2024-04-17T10:48:24.384672+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.400185+0000","last_change":"2024-04-17T10:48:24.398460+0000","last_active":"2024-04-17T10:48:24.400185+0000","last_peered":"2024-04-17T10:48:24.400185+0000","last_clean":"2024-04-17T10:48:24.400185+0000","last_became_active":"2024-04-17T10:48:24.397860+0000","last_became_peered":"2024-04-17T10:48:24.397860+0000","last_unstale":"2024-04-17T10:48:24.400185+0000","last_undegraded":"2024-04-17T10:48:24.400185+0000","last_fullsized":"2024-04-17T10:48:24.400185+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:20:18.572385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.384643+0000","last_change":"2024-04-17T10:48:24.384643+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.384643+0000","last_undegraded":"2024-04-17T10:48:24.384643+0000","last_fullsized":"2024-04-17T10:48:24.384643+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.395517+0000","last_change":"2024-04-17T10:48:24.391043+0000","last_active":"2024-04-17T10:48:24.395517+0000","last_peered":"2024-04-17T10:48:24.395517+0000","last_clean":"2024-04-17T10:48:24.395517+0000","last_became_active":"2024-04-17T10:48:24.390673+0000","last_became_peered":"2024-04-17T10:48:24.390673+0000","last_unstale":"2024-04-17T10:48:24.395517+0000","last_undegraded":"2024-04-17T10:48:24.395517+0000","last_fullsized":"2024-04-17T10:48:24.395517+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:36:28.852444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.375853+0000","last_change":"2024-04-17T10:48:24.375853+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.375853+0000","last_undegraded":"2024-04-17T10:48:24.375853+0000","last_fullsized":"2024-04-17T10:48:24.375853+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.400107+0000","last_change":"2024-04-17T10:48:24.398215+0000","last_active":"2024-04-17T10:48:24.400107+0000","last_peered":"2024-04-17T10:48:24.400107+0000","last_clean":"2024-04-17T10:48:24.400107+0000","last_became_active":"2024-04-17T10:48:24.394779+0000","last_became_peered":"2024-04-17T10:48:24.394779+0000","last_unstale":"2024-04-17T10:48:24.400107+0000","last_undegraded":"2024-04-17T10:48:24.400107+0000","last_fullsized":"2024-04-17T10:48:24.400107+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:54:47.216960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.370200+0000","last_change":"2024-04-17T10:48:24.370200+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.370200+0000","last_undegraded":"2024-04-17T10:48:24.370200+0000","last_fullsized":"2024-04-17T10:48:24.370200+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401418+0000","last_change":"2024-04-17T10:48:24.396294+0000","last_active":"2024-04-17T10:48:24.401418+0000","last_peered":"2024-04-17T10:48:24.401418+0000","last_clean":"2024-04-17T10:48:24.401418+0000","last_became_active":"2024-04-17T10:48:24.395673+0000","last_became_peered":"2024-04-17T10:48:24.395673+0000","last_unstale":"2024-04-17T10:48:24.401418+0000","last_undegraded":"2024-04-17T10:48:24.401418+0000","last_fullsized":"2024-04-17T10:48:24.401418+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:16:43.238162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.370856+0000","last_change":"2024-04-17T10:48:24.370856+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.370856+0000","last_undegraded":"2024-04-17T10:48:24.370856+0000","last_fullsized":"2024-04-17T10:48:24.370856+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.387789+0000","last_change":"2024-04-17T10:48:24.386727+0000","last_active":"2024-04-17T10:48:24.387789+0000","last_peered":"2024-04-17T10:48:24.387789+0000","last_clean":"2024-04-17T10:48:24.387789+0000","last_became_active":"2024-04-17T10:48:24.386334+0000","last_became_peered":"2024-04-17T10:48:24.386334+0000","last_unstale":"2024-04-17T10:48:24.387789+0000","last_undegraded":"2024-04-17T10:48:24.387789+0000","last_fullsized":"2024-04-17T10:48:24.387789+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:09.646139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.378187+0000","last_change":"2024-04-17T10:48:24.378187+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.378187+0000","last_undegraded":"2024-04-17T10:48:24.378187+0000","last_fullsized":"2024-04-17T10:48:24.378187+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.402062+0000","last_change":"2024-04-17T10:48:24.401190+0000","last_active":"2024-04-17T10:48:24.402062+0000","last_peered":"2024-04-17T10:48:24.402062+0000","last_clean":"2024-04-17T10:48:24.402062+0000","last_became_active":"2024-04-17T10:48:24.400909+0000","last_became_peered":"2024-04-17T10:48:24.400909+0000","last_unstale":"2024-04-17T10:48:24.402062+0000","last_undegraded":"2024-04-17T10:48:24.402062+0000","last_fullsized":"2024-04-17T10:48:24.402062+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:20:23.871607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.383491+0000","last_change":"2024-04-17T10:48:24.383491+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.383491+0000","last_undegraded":"2024-04-17T10:48:24.383491+0000","last_fullsized":"2024-04-17T10:48:24.383491+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.395697+0000","last_change":"2024-04-17T10:48:24.393593+0000","last_active":"2024-04-17T10:48:24.395697+0000","last_peered":"2024-04-17T10:48:24.395697+0000","last_clean":"2024-04-17T10:48:24.395697+0000","last_became_active":"2024-04-17T10:48:24.392876+0000","last_became_peered":"2024-04-17T10:48:24.392876+0000","last_unstale":"2024-04-17T10:48:24.395697+0000","last_undegraded":"2024-04-17T10:48:24.395697+0000","last_fullsized":"2024-04-17T10:48:24.395697+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:54:47.216960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.390419+0000","last_change":"2024-04-17T10:48:24.390419+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.390419+0000","last_undegraded":"2024-04-17T10:48:24.390419+0000","last_fullsized":"2024-04-17T10:48:24.390419+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401545+0000","last_change":"2024-04-17T10:48:24.396850+0000","last_active":"2024-04-17T10:48:24.401545+0000","last_peered":"2024-04-17T10:48:24.401545+0000","last_clean":"2024-04-17T10:48:24.401545+0000","last_became_active":"2024-04-17T10:48:24.396194+0000","last_became_peered":"2024-04-17T10:48:24.396194+0000","last_unstale":"2024-04-17T10:48:24.401545+0000","last_undegraded":"2024-04-17T10:48:24.401545+0000","last_fullsized":"2024-04-17T10:48:24.401545+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:20:02.139331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":359,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.387430+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:48:24.387430+0000","last_peered":"2024-04-17T10:48:24.387430+0000","last_clean":"2024-04-17T10:48:24.387430+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:48:24.387430+0000","last_undegraded":"2024-04-17T10:48:24.387430+0000","last_fullsized":"2024-04-17T10:48:24.387430+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574977,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59476,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312364,"statfs":{"total":96636764160,"available":96575860736,"internally_reserved":0,"allocated":1974272,"data_stored":1187543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574976,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68624,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303216,"statfs":{"total":96636764160,"available":96566493184,"internally_reserved":0,"allocated":1380352,"data_stored":597175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574977,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54736,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317104,"statfs":{"total":96636764160,"available":96580714496,"internally_reserved":0,"allocated":1380352,"data_stored":597175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68636,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303204,"statfs":{"total":96636764160,"available":96566480896,"internally_reserved":0,"allocated":1380352,"data_stored":597175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574978,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68624,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303216,"statfs":{"total":96636764160,"available":96566493184,"internally_reserved":0,"allocated":1380352,"data_stored":597175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58908,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312932,"statfs":{"total":96636764160,"available":96576442368,"internally_reserved":0,"allocated":1380352,"data_stored":597175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94996,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94276844,"statfs":{"total":96636764160,"available":96539488256,"internally_reserved":0,"allocated":1380352,"data_stored":597175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574976,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63644,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94308196,"statfs":{"total":96636764160,"available":96571592704,"internally_reserved":0,"allocated":1974272,"data_stored":1187543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:48:29.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3f is not active+clean 2024-04-17T10:48:29.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.387123+0000', 'last_change': '2024-04-17T10:48:24.387123+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.387123+0000', 'last_undegraded': '2024-04-17T10:48:24.387123+0000', 'last_fullsized': '2024-04-17T10:48:24.387123+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:48:29.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3e is not active+clean 2024-04-17T10:48:29.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.370522+0000', 'last_change': '2024-04-17T10:48:24.370522+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.370522+0000', 'last_undegraded': '2024-04-17T10:48:24.370522+0000', 'last_fullsized': '2024-04-17T10:48:24.370522+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:48:29.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3d is not active+clean 2024-04-17T10:48:29.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.379509+0000', 'last_change': '2024-04-17T10:48:24.379509+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.379509+0000', 'last_undegraded': '2024-04-17T10:48:24.379509+0000', 'last_fullsized': '2024-04-17T10:48:24.379509+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:48:29.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3c is not active+clean 2024-04-17T10:48:29.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.387122+0000', 'last_change': '2024-04-17T10:48:24.387122+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.387122+0000', 'last_undegraded': '2024-04-17T10:48:24.387122+0000', 'last_fullsized': '2024-04-17T10:48:24.387122+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:48:29.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3b is not active+clean 2024-04-17T10:48:29.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.380195+0000', 'last_change': '2024-04-17T10:48:24.380195+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.380195+0000', 'last_undegraded': '2024-04-17T10:48:24.380195+0000', 'last_fullsized': '2024-04-17T10:48:24.380195+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:48:29.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3a is not active+clean 2024-04-17T10:48:29.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.378608+0000', 'last_change': '2024-04-17T10:48:24.378608+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.378608+0000', 'last_undegraded': '2024-04-17T10:48:24.378608+0000', 'last_fullsized': '2024-04-17T10:48:24.378608+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:48:29.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.39 is not active+clean 2024-04-17T10:48:29.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.384126+0000', 'last_change': '2024-04-17T10:48:24.384126+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.384126+0000', 'last_undegraded': '2024-04-17T10:48:24.384126+0000', 'last_fullsized': '2024-04-17T10:48:24.384126+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:48:29.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.38 is not active+clean 2024-04-17T10:48:29.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.380148+0000', 'last_change': '2024-04-17T10:48:24.380148+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.380148+0000', 'last_undegraded': '2024-04-17T10:48:24.380148+0000', 'last_fullsized': '2024-04-17T10:48:24.380148+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:48:29.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.37 is not active+clean 2024-04-17T10:48:29.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.386611+0000', 'last_change': '2024-04-17T10:48:24.386611+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.386611+0000', 'last_undegraded': '2024-04-17T10:48:24.386611+0000', 'last_fullsized': '2024-04-17T10:48:24.386611+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:48:29.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.36 is not active+clean 2024-04-17T10:48:29.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.389198+0000', 'last_change': '2024-04-17T10:48:24.389198+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.389198+0000', 'last_undegraded': '2024-04-17T10:48:24.389198+0000', 'last_fullsized': '2024-04-17T10:48:24.389198+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:48:29.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.35 is not active+clean 2024-04-17T10:48:29.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.370124+0000', 'last_change': '2024-04-17T10:48:24.370124+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.370124+0000', 'last_undegraded': '2024-04-17T10:48:24.370124+0000', 'last_fullsized': '2024-04-17T10:48:24.370124+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:48:29.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.34 is not active+clean 2024-04-17T10:48:29.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.386065+0000', 'last_change': '2024-04-17T10:48:24.386065+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.386065+0000', 'last_undegraded': '2024-04-17T10:48:24.386065+0000', 'last_fullsized': '2024-04-17T10:48:24.386065+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:48:29.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.33 is not active+clean 2024-04-17T10:48:29.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.374379+0000', 'last_change': '2024-04-17T10:48:24.374379+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.374379+0000', 'last_undegraded': '2024-04-17T10:48:24.374379+0000', 'last_fullsized': '2024-04-17T10:48:24.374379+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:48:29.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.32 is not active+clean 2024-04-17T10:48:29.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.376316+0000', 'last_change': '2024-04-17T10:48:24.376316+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.376316+0000', 'last_undegraded': '2024-04-17T10:48:24.376316+0000', 'last_fullsized': '2024-04-17T10:48:24.376316+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:48:29.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.31 is not active+clean 2024-04-17T10:48:29.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.375310+0000', 'last_change': '2024-04-17T10:48:24.375310+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.375310+0000', 'last_undegraded': '2024-04-17T10:48:24.375310+0000', 'last_fullsized': '2024-04-17T10:48:24.375310+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:48:29.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.30 is not active+clean 2024-04-17T10:48:29.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.385989+0000', 'last_change': '2024-04-17T10:48:24.385989+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.385989+0000', 'last_undegraded': '2024-04-17T10:48:24.385989+0000', 'last_fullsized': '2024-04-17T10:48:24.385989+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:48:29.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2f is not active+clean 2024-04-17T10:48:29.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.373575+0000', 'last_change': '2024-04-17T10:48:24.373575+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.373575+0000', 'last_undegraded': '2024-04-17T10:48:24.373575+0000', 'last_fullsized': '2024-04-17T10:48:24.373575+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:48:29.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2e is not active+clean 2024-04-17T10:48:29.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.385756+0000', 'last_change': '2024-04-17T10:48:24.385756+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.385756+0000', 'last_undegraded': '2024-04-17T10:48:24.385756+0000', 'last_fullsized': '2024-04-17T10:48:24.385756+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:48:29.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2d is not active+clean 2024-04-17T10:48:29.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.378920+0000', 'last_change': '2024-04-17T10:48:24.378920+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.378920+0000', 'last_undegraded': '2024-04-17T10:48:24.378920+0000', 'last_fullsized': '2024-04-17T10:48:24.378920+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:48:29.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2c is not active+clean 2024-04-17T10:48:29.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.371256+0000', 'last_change': '2024-04-17T10:48:24.371256+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.371256+0000', 'last_undegraded': '2024-04-17T10:48:24.371256+0000', 'last_fullsized': '2024-04-17T10:48:24.371256+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:48:29.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2b is not active+clean 2024-04-17T10:48:29.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.370141+0000', 'last_change': '2024-04-17T10:48:24.370141+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.370141+0000', 'last_undegraded': '2024-04-17T10:48:24.370141+0000', 'last_fullsized': '2024-04-17T10:48:24.370141+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:48:29.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2a is not active+clean 2024-04-17T10:48:29.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.393879+0000', 'last_change': '2024-04-17T10:48:24.393879+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.393879+0000', 'last_undegraded': '2024-04-17T10:48:24.393879+0000', 'last_fullsized': '2024-04-17T10:48:24.393879+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:48:29.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.29 is not active+clean 2024-04-17T10:48:29.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.373549+0000', 'last_change': '2024-04-17T10:48:24.373549+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.373549+0000', 'last_undegraded': '2024-04-17T10:48:24.373549+0000', 'last_fullsized': '2024-04-17T10:48:24.373549+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:48:29.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.28 is not active+clean 2024-04-17T10:48:29.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.393172+0000', 'last_change': '2024-04-17T10:48:24.393172+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.393172+0000', 'last_undegraded': '2024-04-17T10:48:24.393172+0000', 'last_fullsized': '2024-04-17T10:48:24.393172+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:48:29.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.27 is not active+clean 2024-04-17T10:48:29.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.371705+0000', 'last_change': '2024-04-17T10:48:24.371705+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.371705+0000', 'last_undegraded': '2024-04-17T10:48:24.371705+0000', 'last_fullsized': '2024-04-17T10:48:24.371705+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:48:29.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.26 is not active+clean 2024-04-17T10:48:29.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.386061+0000', 'last_change': '2024-04-17T10:48:24.386061+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.386061+0000', 'last_undegraded': '2024-04-17T10:48:24.386061+0000', 'last_fullsized': '2024-04-17T10:48:24.386061+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:48:29.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.25 is not active+clean 2024-04-17T10:48:29.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.374580+0000', 'last_change': '2024-04-17T10:48:24.374580+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.374580+0000', 'last_undegraded': '2024-04-17T10:48:24.374580+0000', 'last_fullsized': '2024-04-17T10:48:24.374580+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:48:29.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.24 is not active+clean 2024-04-17T10:48:29.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.393499+0000', 'last_change': '2024-04-17T10:48:24.393499+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.393499+0000', 'last_undegraded': '2024-04-17T10:48:24.393499+0000', 'last_fullsized': '2024-04-17T10:48:24.393499+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:48:29.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.23 is not active+clean 2024-04-17T10:48:29.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.372097+0000', 'last_change': '2024-04-17T10:48:24.372097+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.372097+0000', 'last_undegraded': '2024-04-17T10:48:24.372097+0000', 'last_fullsized': '2024-04-17T10:48:24.372097+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:48:29.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.22 is not active+clean 2024-04-17T10:48:29.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.389706+0000', 'last_change': '2024-04-17T10:48:24.389706+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.389706+0000', 'last_undegraded': '2024-04-17T10:48:24.389706+0000', 'last_fullsized': '2024-04-17T10:48:24.389706+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:48:29.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.21 is not active+clean 2024-04-17T10:48:29.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.385558+0000', 'last_change': '2024-04-17T10:48:24.385558+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.385558+0000', 'last_undegraded': '2024-04-17T10:48:24.385558+0000', 'last_fullsized': '2024-04-17T10:48:24.385558+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:48:29.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.20 is not active+clean 2024-04-17T10:48:29.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.376758+0000', 'last_change': '2024-04-17T10:48:24.376758+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.376758+0000', 'last_undegraded': '2024-04-17T10:48:24.376758+0000', 'last_fullsized': '2024-04-17T10:48:24.376758+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:48:29.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1e is not active+clean 2024-04-17T10:48:29.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.375305+0000', 'last_change': '2024-04-17T10:48:24.375305+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.375305+0000', 'last_undegraded': '2024-04-17T10:48:24.375305+0000', 'last_fullsized': '2024-04-17T10:48:24.375305+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:48:29.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1f is not active+clean 2024-04-17T10:48:29.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.376492+0000', 'last_change': '2024-04-17T10:48:24.376492+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.376492+0000', 'last_undegraded': '2024-04-17T10:48:24.376492+0000', 'last_fullsized': '2024-04-17T10:48:24.376492+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:48:29.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1c is not active+clean 2024-04-17T10:48:29.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.379331+0000', 'last_change': '2024-04-17T10:48:24.379331+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.379331+0000', 'last_undegraded': '2024-04-17T10:48:24.379331+0000', 'last_fullsized': '2024-04-17T10:48:24.379331+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:48:29.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1d is not active+clean 2024-04-17T10:48:29.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.376494+0000', 'last_change': '2024-04-17T10:48:24.376494+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.376494+0000', 'last_undegraded': '2024-04-17T10:48:24.376494+0000', 'last_fullsized': '2024-04-17T10:48:24.376494+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:48:29.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1a is not active+clean 2024-04-17T10:48:29.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.385378+0000', 'last_change': '2024-04-17T10:48:24.385378+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.385378+0000', 'last_undegraded': '2024-04-17T10:48:24.385378+0000', 'last_fullsized': '2024-04-17T10:48:24.385378+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:48:29.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1b is not active+clean 2024-04-17T10:48:29.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.371493+0000', 'last_change': '2024-04-17T10:48:24.371493+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.371493+0000', 'last_undegraded': '2024-04-17T10:48:24.371493+0000', 'last_fullsized': '2024-04-17T10:48:24.371493+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:48:29.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.18 is not active+clean 2024-04-17T10:48:29.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.385357+0000', 'last_change': '2024-04-17T10:48:24.385357+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.385357+0000', 'last_undegraded': '2024-04-17T10:48:24.385357+0000', 'last_fullsized': '2024-04-17T10:48:24.385357+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:48:29.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.19 is not active+clean 2024-04-17T10:48:29.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.384767+0000', 'last_change': '2024-04-17T10:48:24.384767+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.384767+0000', 'last_undegraded': '2024-04-17T10:48:24.384767+0000', 'last_fullsized': '2024-04-17T10:48:24.384767+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:48:29.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.16 is not active+clean 2024-04-17T10:48:29.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.389344+0000', 'last_change': '2024-04-17T10:48:24.389344+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.389344+0000', 'last_undegraded': '2024-04-17T10:48:24.389344+0000', 'last_fullsized': '2024-04-17T10:48:24.389344+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:48:29.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.17 is not active+clean 2024-04-17T10:48:29.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.375839+0000', 'last_change': '2024-04-17T10:48:24.375839+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.375839+0000', 'last_undegraded': '2024-04-17T10:48:24.375839+0000', 'last_fullsized': '2024-04-17T10:48:24.375839+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:48:29.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.14 is not active+clean 2024-04-17T10:48:29.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.391756+0000', 'last_change': '2024-04-17T10:48:24.391756+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.391756+0000', 'last_undegraded': '2024-04-17T10:48:24.391756+0000', 'last_fullsized': '2024-04-17T10:48:24.391756+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:48:29.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.15 is not active+clean 2024-04-17T10:48:29.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.390503+0000', 'last_change': '2024-04-17T10:48:24.390503+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.390503+0000', 'last_undegraded': '2024-04-17T10:48:24.390503+0000', 'last_fullsized': '2024-04-17T10:48:24.390503+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:48:29.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.12 is not active+clean 2024-04-17T10:48:29.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.391471+0000', 'last_change': '2024-04-17T10:48:24.391471+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.391471+0000', 'last_undegraded': '2024-04-17T10:48:24.391471+0000', 'last_fullsized': '2024-04-17T10:48:24.391471+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:48:29.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.13 is not active+clean 2024-04-17T10:48:29.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.393082+0000', 'last_change': '2024-04-17T10:48:24.393082+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.393082+0000', 'last_undegraded': '2024-04-17T10:48:24.393082+0000', 'last_fullsized': '2024-04-17T10:48:24.393082+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:48:29.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.11 is not active+clean 2024-04-17T10:48:29.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.392104+0000', 'last_change': '2024-04-17T10:48:24.392104+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.392104+0000', 'last_undegraded': '2024-04-17T10:48:24.392104+0000', 'last_fullsized': '2024-04-17T10:48:24.392104+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:48:29.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.e is not active+clean 2024-04-17T10:48:29.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.376491+0000', 'last_change': '2024-04-17T10:48:24.376491+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.376491+0000', 'last_undegraded': '2024-04-17T10:48:24.376491+0000', 'last_fullsized': '2024-04-17T10:48:24.376491+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:48:29.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.f is not active+clean 2024-04-17T10:48:29.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.374704+0000', 'last_change': '2024-04-17T10:48:24.374704+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.374704+0000', 'last_undegraded': '2024-04-17T10:48:24.374704+0000', 'last_fullsized': '2024-04-17T10:48:24.374704+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:48:29.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.c is not active+clean 2024-04-17T10:48:29.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.373427+0000', 'last_change': '2024-04-17T10:48:24.373427+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.373427+0000', 'last_undegraded': '2024-04-17T10:48:24.373427+0000', 'last_fullsized': '2024-04-17T10:48:24.373427+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:48:29.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.d is not active+clean 2024-04-17T10:48:29.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.373479+0000', 'last_change': '2024-04-17T10:48:24.373479+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.373479+0000', 'last_undegraded': '2024-04-17T10:48:24.373479+0000', 'last_fullsized': '2024-04-17T10:48:24.373479+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:48:29.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.a is not active+clean 2024-04-17T10:48:29.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.374303+0000', 'last_change': '2024-04-17T10:48:24.374303+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.374303+0000', 'last_undegraded': '2024-04-17T10:48:24.374303+0000', 'last_fullsized': '2024-04-17T10:48:24.374303+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:48:29.104 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.b is not active+clean 2024-04-17T10:48:29.104 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.384650+0000', 'last_change': '2024-04-17T10:48:24.384650+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.384650+0000', 'last_undegraded': '2024-04-17T10:48:24.384650+0000', 'last_fullsized': '2024-04-17T10:48:24.384650+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:48:29.104 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.8 is not active+clean 2024-04-17T10:48:29.104 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.379208+0000', 'last_change': '2024-04-17T10:48:24.379208+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.379208+0000', 'last_undegraded': '2024-04-17T10:48:24.379208+0000', 'last_fullsized': '2024-04-17T10:48:24.379208+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:48:29.104 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.9 is not active+clean 2024-04-17T10:48:29.104 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.390331+0000', 'last_change': '2024-04-17T10:48:24.390331+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.390331+0000', 'last_undegraded': '2024-04-17T10:48:24.390331+0000', 'last_fullsized': '2024-04-17T10:48:24.390331+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:48:29.104 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.6 is not active+clean 2024-04-17T10:48:29.104 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.384853+0000', 'last_change': '2024-04-17T10:48:24.384853+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.384853+0000', 'last_undegraded': '2024-04-17T10:48:24.384853+0000', 'last_fullsized': '2024-04-17T10:48:24.384853+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:48:29.104 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.7 is not active+clean 2024-04-17T10:48:29.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.384672+0000', 'last_change': '2024-04-17T10:48:24.384672+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.384672+0000', 'last_undegraded': '2024-04-17T10:48:24.384672+0000', 'last_fullsized': '2024-04-17T10:48:24.384672+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:48:29.105 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.4 is not active+clean 2024-04-17T10:48:29.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.384643+0000', 'last_change': '2024-04-17T10:48:24.384643+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.384643+0000', 'last_undegraded': '2024-04-17T10:48:24.384643+0000', 'last_fullsized': '2024-04-17T10:48:24.384643+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:48:29.105 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.5 is not active+clean 2024-04-17T10:48:29.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.375853+0000', 'last_change': '2024-04-17T10:48:24.375853+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.375853+0000', 'last_undegraded': '2024-04-17T10:48:24.375853+0000', 'last_fullsized': '2024-04-17T10:48:24.375853+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:48:29.105 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2 is not active+clean 2024-04-17T10:48:29.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.370200+0000', 'last_change': '2024-04-17T10:48:24.370200+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.370200+0000', 'last_undegraded': '2024-04-17T10:48:24.370200+0000', 'last_fullsized': '2024-04-17T10:48:24.370200+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:48:29.105 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3 is not active+clean 2024-04-17T10:48:29.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.370856+0000', 'last_change': '2024-04-17T10:48:24.370856+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.370856+0000', 'last_undegraded': '2024-04-17T10:48:24.370856+0000', 'last_fullsized': '2024-04-17T10:48:24.370856+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:48:29.105 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.0 is not active+clean 2024-04-17T10:48:29.106 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.378187+0000', 'last_change': '2024-04-17T10:48:24.378187+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.378187+0000', 'last_undegraded': '2024-04-17T10:48:24.378187+0000', 'last_fullsized': '2024-04-17T10:48:24.378187+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:48:29.106 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1 is not active+clean 2024-04-17T10:48:29.106 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.383491+0000', 'last_change': '2024-04-17T10:48:24.383491+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.383491+0000', 'last_undegraded': '2024-04-17T10:48:24.383491+0000', 'last_fullsized': '2024-04-17T10:48:24.383491+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:48:29.106 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.10 is not active+clean 2024-04-17T10:48:29.106 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:48:24.390419+0000', 'last_change': '2024-04-17T10:48:24.390419+0000', 'last_active': '2024-04-17T10:48:24.357930+0000', 'last_peered': '2024-04-17T10:48:24.357930+0000', 'last_clean': '2024-04-17T10:48:24.357930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:48:24.390419+0000', 'last_undegraded': '2024-04-17T10:48:24.390419+0000', 'last_fullsized': '2024-04-17T10:48:24.390419+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'last_clean_scrub_stamp': '2024-04-17T10:48:24.357930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:48:29.107 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-17T10:48:29.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.250+0000 7faa284fe640 1 Processor -- start 2024-04-17T10:48:29.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.250+0000 7faa284fe640 1 -- start start 2024-04-17T10:48:29.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.250+0000 7faa284fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa2015db50 0x7faa20070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:29.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.250+0000 7faa284fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faa2006dcb0 0x7faa200b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:29.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.250+0000 7faa284fe640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa200709b0 con 0x7faa200aedd0 2024-04-17T10:48:29.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.250+0000 7faa284fe640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa2015e440 con 0x7faa2006dcb0 2024-04-17T10:48:29.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.250+0000 7faa284fe640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa2015e720 con 0x7faa2015db50 2024-04-17T10:48:29.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.250+0000 7faa26a9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faa2006dcb0 0x7faa200b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:29.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.250+0000 7faa25a99640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7faa200aedd0 0x7faa2006d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40972/0 (socket says 172.21.15.83:40972) 2024-04-17T10:48:29.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.250+0000 7faa25a99640 1 -- 172.21.15.83:0/2330197351 learned_addr learned my addr 172.21.15.83:0/2330197351 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:29.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.250+0000 7faa26a9b640 1 -- 172.21.15.83:0/2330197351 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa2015db50 msgr2=0x7faa20070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:29.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.250+0000 7faa2629a640 1 --2- 172.21.15.83:0/2330197351 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa2015db50 0x7faa20070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:29.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.250+0000 7faa25298640 1 -- 172.21.15.83:0/2330197351 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 540246319 0 0) 0x7faa14001120 con 0x7faa200aedd0 2024-04-17T10:48:29.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.250+0000 7faa26a9b640 1 --2- 172.21.15.83:0/2330197351 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa2015db50 0x7faa20070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:29.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.250+0000 7faa26a9b640 1 -- 172.21.15.83:0/2330197351 >> v1:172.21.15.83:6789/0 conn(0x7faa200aedd0 legacy=0x7faa2006d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:29.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.250+0000 7faa26a9b640 1 -- 172.21.15.83:0/2330197351 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2015ea00 con 0x7faa2006dcb0 2024-04-17T10:48:29.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.250+0000 7faa2629a640 1 --2- 172.21.15.83:0/2330197351 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa2015db50 0x7faa20070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:29.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.254+0000 7faa26a9b640 1 --2- 172.21.15.83:0/2330197351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa2006dcb0 0x7faa200b2e10 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7faa180097c0 tx=0x7faa1801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6be3e082cbf1099 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:29.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.254+0000 7faa25298640 1 -- 172.21.15.83:0/2330197351 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa180a7070 con 0x7faa2006dcb0 2024-04-17T10:48:29.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.254+0000 7faa25298640 1 -- 172.21.15.83:0/2330197351 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa180b0070 con 0x7faa2006dcb0 2024-04-17T10:48:29.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.254+0000 7faa25298640 1 -- 172.21.15.83:0/2330197351 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa180ab890 con 0x7faa2006dcb0 2024-04-17T10:48:29.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.254+0000 7faa284fe640 1 -- 172.21.15.83:0/2330197351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa2006dcb0 msgr2=0x7faa200b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:29.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.254+0000 7faa284fe640 1 --2- 172.21.15.83:0/2330197351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa2006dcb0 0x7faa200b2e10 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7faa180097c0 tx=0x7faa1801c470 comp rx=0 tx=0).stop 2024-04-17T10:48:29.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.254+0000 7faa284fe640 1 -- 172.21.15.83:0/2330197351 shutdown_connections 2024-04-17T10:48:29.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.254+0000 7faa284fe640 1 --2- 172.21.15.83:0/2330197351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa2006dcb0 0x7faa200b2e10 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:29.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.254+0000 7faa284fe640 1 --2- 172.21.15.83:0/2330197351 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa2015db50 0x7faa20070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:29.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.254+0000 7faa284fe640 1 -- 172.21.15.83:0/2330197351 >> 172.21.15.83:0/2330197351 conn(0x7faa2006abe0 msgr2=0x7faa2006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:29.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.254+0000 7faa284fe640 1 -- 172.21.15.83:0/2330197351 shutdown_connections 2024-04-17T10:48:29.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.254+0000 7faa284fe640 1 -- 172.21.15.83:0/2330197351 wait complete. 2024-04-17T10:48:29.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.254+0000 7faa284fe640 1 Processor -- start 2024-04-17T10:48:29.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.254+0000 7faa284fe640 1 -- start start 2024-04-17T10:48:29.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.258+0000 7faa284fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa2006dcb0 0x7faa201512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:29.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.258+0000 7faa284fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa200aedd0 0x7faa200adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:29.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.258+0000 7faa284fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa2015db50 0x7faa2015b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:29.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.258+0000 7faa284fe640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa200b3ce0 con 0x7faa200aedd0 2024-04-17T10:48:29.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.258+0000 7faa284fe640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa200b3b60 con 0x7faa2006dcb0 2024-04-17T10:48:29.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.258+0000 7faa284fe640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa200b3e60 con 0x7faa2015db50 2024-04-17T10:48:29.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.258+0000 7faa25a99640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa200aedd0 0x7faa200adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:29.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.258+0000 7faa25a99640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa200aedd0 0x7faa200adaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44722/0 (socket says 172.21.15.83:44722) 2024-04-17T10:48:29.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.258+0000 7faa25a99640 1 -- 172.21.15.83:0/4272960832 learned_addr learned my addr 172.21.15.83:0/4272960832 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:29.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.258+0000 7faa26a9b640 1 --2- 172.21.15.83:0/4272960832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa2015db50 0x7faa2015b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:29.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.258+0000 7faa2629a640 1 --2- 172.21.15.83:0/4272960832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa2006dcb0 0x7faa201512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:29.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.258+0000 7faa25a99640 1 -- 172.21.15.83:0/4272960832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa2015db50 msgr2=0x7faa2015b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:29.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.258+0000 7faa25a99640 1 --2- 172.21.15.83:0/4272960832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa2015db50 0x7faa2015b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:29.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.258+0000 7faa25a99640 1 -- 172.21.15.83:0/4272960832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa2006dcb0 msgr2=0x7faa201512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:29.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.258+0000 7faa25a99640 1 --2- 172.21.15.83:0/4272960832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa2006dcb0 0x7faa201512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:29.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.258+0000 7faa25a99640 1 -- 172.21.15.83:0/4272960832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa20154290 con 0x7faa200aedd0 2024-04-17T10:48:29.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.258+0000 7faa2629a640 1 --2- 172.21.15.83:0/4272960832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa2006dcb0 0x7faa201512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:29.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.258+0000 7faa25a99640 1 --2- 172.21.15.83:0/4272960832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa200aedd0 0x7faa200adaa0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7faa1400c780 tx=0x7faa1400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:29.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.258+0000 7faa1effd640 1 -- 172.21.15.83:0/4272960832 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa14018070 con 0x7faa200aedd0 2024-04-17T10:48:29.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.258+0000 7faa1effd640 1 -- 172.21.15.83:0/4272960832 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa140072f0 con 0x7faa200aedd0 2024-04-17T10:48:29.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.258+0000 7faa284fe640 1 -- 172.21.15.83:0/4272960832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa201531f0 con 0x7faa200aedd0 2024-04-17T10:48:29.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.258+0000 7faa1effd640 1 -- 172.21.15.83:0/4272960832 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa140047f0 con 0x7faa200aedd0 2024-04-17T10:48:29.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.258+0000 7faa284fe640 1 -- 172.21.15.83:0/4272960832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa20153a90 con 0x7faa200aedd0 2024-04-17T10:48:29.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.258+0000 7faa284fe640 1 -- 172.21.15.83:0/4272960832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa20074f90 con 0x7faa200aedd0 2024-04-17T10:48:29.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.258+0000 7faa1effd640 1 -- 172.21.15.83:0/4272960832 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faa1401b070 con 0x7faa200aedd0 2024-04-17T10:48:29.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.258+0000 7faa1effd640 1 --2- 172.21.15.83:0/4272960832 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faa0c066f60 0x7faa0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:29.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.258+0000 7faa1effd640 1 -- 172.21.15.83:0/4272960832 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa140843d0 con 0x7faa200aedd0 2024-04-17T10:48:29.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.258+0000 7faa2629a640 1 --2- 172.21.15.83:0/4272960832 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faa0c066f60 0x7faa0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:29.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.258+0000 7faa2629a640 1 --2- 172.21.15.83:0/4272960832 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faa0c066f60 0x7faa0c069420 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7faa10007f40 tx=0x7faa10007490 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:29.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.262+0000 7faa1effd640 1 -- 172.21.15.83:0/4272960832 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faa200709b0 con 0x7faa200aedd0 2024-04-17T10:48:29.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.466+0000 7faa284fe640 1 -- 172.21.15.83:0/4272960832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7faa20155dc0 con 0x7faa200aedd0 2024-04-17T10:48:29.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.466+0000 7faa1effd640 1 -- 172.21.15.83:0/4272960832 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1584 (secure 0 0 0) 0x7faa14055470 con 0x7faa200aedd0 2024-04-17T10:48:29.462 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:29.462 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":642,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":114,"num_osds":8,"num_up_osds":8,"osd_up_since":1713350279,"num_in_osds":8,"osd_in_since":1713350270,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":550547456,"bytes_avail":772543565824,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":69,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8604}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2024-04-17T10:48:06.358835+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-17T10:48:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.466+0000 7faa284fe640 1 -- 172.21.15.83:0/4272960832 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faa0c066f60 msgr2=0x7faa0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.466+0000 7faa284fe640 1 --2- 172.21.15.83:0/4272960832 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faa0c066f60 0x7faa0c069420 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7faa10007f40 tx=0x7faa10007490 comp rx=0 tx=0).stop 2024-04-17T10:48:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.466+0000 7faa284fe640 1 -- 172.21.15.83:0/4272960832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa200aedd0 msgr2=0x7faa200adaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.466+0000 7faa284fe640 1 --2- 172.21.15.83:0/4272960832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa200aedd0 0x7faa200adaa0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7faa1400c780 tx=0x7faa1400cc40 comp rx=0 tx=0).stop 2024-04-17T10:48:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.466+0000 7faa284fe640 1 -- 172.21.15.83:0/4272960832 shutdown_connections 2024-04-17T10:48:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.466+0000 7faa284fe640 1 --2- 172.21.15.83:0/4272960832 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faa0c066f60 0x7faa0c069420 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.466+0000 7faa284fe640 1 --2- 172.21.15.83:0/4272960832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa2015db50 0x7faa2015b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.466+0000 7faa284fe640 1 --2- 172.21.15.83:0/4272960832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa200aedd0 0x7faa200adaa0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.466+0000 7faa284fe640 1 --2- 172.21.15.83:0/4272960832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa2006dcb0 0x7faa201512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:29.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.466+0000 7faa284fe640 1 -- 172.21.15.83:0/4272960832 >> 172.21.15.83:0/4272960832 conn(0x7faa2006abe0 msgr2=0x7faa200ae580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:29.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.466+0000 7faa284fe640 1 -- 172.21.15.83:0/4272960832 shutdown_connections 2024-04-17T10:48:29.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.466+0000 7faa284fe640 1 -- 172.21.15.83:0/4272960832 wait complete. 2024-04-17T10:48:29.479 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-17T10:48:29.480 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:48:29.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.602+0000 7f5de068d640 1 Processor -- start 2024-04-17T10:48:29.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.602+0000 7f5de068d640 1 -- start start 2024-04-17T10:48:29.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.606+0000 7f5de068d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5dd815ba30 0x7f5dd8072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:29.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.606+0000 7f5de068d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5dd80cfbe0 0x7f5dd806c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:29.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.606+0000 7f5de068d640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dd806e320 con 0x7f5dd815c960 2024-04-17T10:48:29.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.606+0000 7f5de068d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd8071ea0 con 0x7f5dd815ba30 2024-04-17T10:48:29.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.606+0000 7f5de068d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd806e040 con 0x7f5dd80cfbe0 2024-04-17T10:48:29.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.606+0000 7f5dde429640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5dd815ba30 0x7f5dd8072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:29.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.606+0000 7f5dde429640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5dd815ba30 0x7f5dd8072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44724/0 (socket says 172.21.15.83:44724) 2024-04-17T10:48:29.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.606+0000 7f5dde429640 1 -- 172.21.15.83:0/1152353220 learned_addr learned my addr 172.21.15.83:0/1152353220 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:29.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.606+0000 7f5ddec2a640 1 --2- 172.21.15.83:0/1152353220 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5dd80cfbe0 0x7f5dd806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:29.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.606+0000 7f5dde429640 1 -- 172.21.15.83:0/1152353220 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5dd80cfbe0 msgr2=0x7f5dd806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:29.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.606+0000 7f5dde429640 1 --2- 172.21.15.83:0/1152353220 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5dd80cfbe0 0x7f5dd806c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:29.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.606+0000 7f5dde429640 1 -- 172.21.15.83:0/1152353220 >> v1:172.21.15.167:6789/0 conn(0x7f5dd815c960 legacy=0x7f5dd80cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:29.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.606+0000 7f5dde429640 1 -- 172.21.15.83:0/1152353220 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd8071850 con 0x7f5dd815ba30 2024-04-17T10:48:29.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.606+0000 7f5ddec2a640 1 --2- 172.21.15.83:0/1152353220 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5dd80cfbe0 0x7f5dd806c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:29.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.606+0000 7f5dde429640 1 --2- 172.21.15.83:0/1152353220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd815ba30 0x7f5dd8072330 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0009870 tx=0x7f5dd001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03dbf322ab33022 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:29.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.606+0000 7f5ddd427640 1 -- 172.21.15.83:0/1152353220 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd00b0070 con 0x7f5dd815ba30 2024-04-17T10:48:29.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.606+0000 7f5ddd427640 1 -- 172.21.15.83:0/1152353220 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd00ab600 con 0x7f5dd815ba30 2024-04-17T10:48:29.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.606+0000 7f5ddd427640 1 -- 172.21.15.83:0/1152353220 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd00ab9d0 con 0x7f5dd815ba30 2024-04-17T10:48:29.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.610+0000 7f5de068d640 1 -- 172.21.15.83:0/1152353220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd815ba30 msgr2=0x7f5dd8072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:29.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.610+0000 7f5de068d640 1 --2- 172.21.15.83:0/1152353220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd815ba30 0x7f5dd8072330 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0009870 tx=0x7f5dd001c490 comp rx=0 tx=0).stop 2024-04-17T10:48:29.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.610+0000 7f5de068d640 1 -- 172.21.15.83:0/1152353220 shutdown_connections 2024-04-17T10:48:29.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.610+0000 7f5de068d640 1 --2- 172.21.15.83:0/1152353220 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5dd80cfbe0 0x7f5dd806c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:29.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.610+0000 7f5de068d640 1 --2- 172.21.15.83:0/1152353220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd815ba30 0x7f5dd8072330 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:29.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.610+0000 7f5de068d640 1 -- 172.21.15.83:0/1152353220 >> 172.21.15.83:0/1152353220 conn(0x7f5dd806abe0 msgr2=0x7f5dd806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:29.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.610+0000 7f5de068d640 1 -- 172.21.15.83:0/1152353220 shutdown_connections 2024-04-17T10:48:29.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.610+0000 7f5de068d640 1 -- 172.21.15.83:0/1152353220 wait complete. 2024-04-17T10:48:29.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.610+0000 7f5de068d640 1 Processor -- start 2024-04-17T10:48:29.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.610+0000 7f5de068d640 1 -- start start 2024-04-17T10:48:29.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.610+0000 7f5de068d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd80cfbe0 0x7f5dd8143130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:29.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.610+0000 7f5de068d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dd815ba30 0x7f5dd8158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:29.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.610+0000 7f5de068d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5dd815c960 0x7f5dd81528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:29.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.610+0000 7f5de068d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd806de20 con 0x7f5dd80cfbe0 2024-04-17T10:48:29.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.610+0000 7f5de068d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd806dca0 con 0x7f5dd815ba30 2024-04-17T10:48:29.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.610+0000 7f5de068d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd8168520 con 0x7f5dd815c960 2024-04-17T10:48:29.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.610+0000 7f5dde429640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd80cfbe0 0x7f5dd8143130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:29.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.610+0000 7f5dde429640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd80cfbe0 0x7f5dd8143130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44732/0 (socket says 172.21.15.83:44732) 2024-04-17T10:48:29.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.610+0000 7f5dde429640 1 -- 172.21.15.83:0/3471286163 learned_addr learned my addr 172.21.15.83:0/3471286163 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:29.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.610+0000 7f5ddec2a640 1 --2- 172.21.15.83:0/3471286163 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5dd815c960 0x7f5dd81528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:29.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.610+0000 7f5dddc28640 1 --2- 172.21.15.83:0/3471286163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dd815ba30 0x7f5dd8158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:29.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.610+0000 7f5dde429640 1 -- 172.21.15.83:0/3471286163 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5dd815c960 msgr2=0x7f5dd81528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:29.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.610+0000 7f5dde429640 1 --2- 172.21.15.83:0/3471286163 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5dd815c960 0x7f5dd81528f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:29.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.610+0000 7f5dde429640 1 -- 172.21.15.83:0/3471286163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dd815ba30 msgr2=0x7f5dd8158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:29.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.610+0000 7f5dde429640 1 --2- 172.21.15.83:0/3471286163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dd815ba30 0x7f5dd8158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:29.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.610+0000 7f5dde429640 1 -- 172.21.15.83:0/3471286163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd815a810 con 0x7f5dd80cfbe0 2024-04-17T10:48:29.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.610+0000 7f5dde429640 1 --2- 172.21.15.83:0/3471286163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd80cfbe0 0x7f5dd8143130 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00ae040 tx=0x7f5dd001acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:29.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.610+0000 7f5dceffd640 1 -- 172.21.15.83:0/3471286163 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd00ba020 con 0x7f5dd80cfbe0 2024-04-17T10:48:29.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.610+0000 7f5dceffd640 1 -- 172.21.15.83:0/3471286163 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd00b6640 con 0x7f5dd80cfbe0 2024-04-17T10:48:29.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.610+0000 7f5de068d640 1 -- 172.21.15.83:0/3471286163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd8159770 con 0x7f5dd80cfbe0 2024-04-17T10:48:29.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.610+0000 7f5dceffd640 1 -- 172.21.15.83:0/3471286163 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd00b0040 con 0x7f5dd80cfbe0 2024-04-17T10:48:29.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.610+0000 7f5de068d640 1 -- 172.21.15.83:0/3471286163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd8159d30 con 0x7f5dd80cfbe0 2024-04-17T10:48:29.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.614+0000 7f5de068d640 1 -- 172.21.15.83:0/3471286163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd8074f90 con 0x7f5dd80cfbe0 2024-04-17T10:48:29.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.614+0000 7f5dceffd640 1 -- 172.21.15.83:0/3471286163 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5dd00bd020 con 0x7f5dd80cfbe0 2024-04-17T10:48:29.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.614+0000 7f5dceffd640 1 --2- 172.21.15.83:0/3471286163 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5dbc066f60 0x7f5dbc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:29.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.614+0000 7f5dceffd640 1 -- 172.21.15.83:0/3471286163 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5dd011f7f0 con 0x7f5dd80cfbe0 2024-04-17T10:48:29.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.614+0000 7f5dddc28640 1 --2- 172.21.15.83:0/3471286163 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5dbc066f60 0x7f5dbc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:29.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.614+0000 7f5dddc28640 1 --2- 172.21.15.83:0/3471286163 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5dbc066f60 0x7f5dbc069420 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8152300 tx=0x7f5dc8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:29.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.618+0000 7f5dceffd640 1 -- 172.21.15.83:0/3471286163 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5dd00c4030 con 0x7f5dd80cfbe0 2024-04-17T10:48:29.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.762+0000 7f5de068d640 1 -- 172.21.15.83:0/3471286163 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5dd80aeea0 con 0x7f5dbc066f60 2024-04-17T10:48:29.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.778+0000 7f5dceffd640 1 -- 172.21.15.83:0/3471286163 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247089 (secure 0 0 0) 0x7f5dd80aeea0 con 0x7f5dbc066f60 2024-04-17T10:48:29.777 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:48:29.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.782+0000 7f5de068d640 1 -- 172.21.15.83:0/3471286163 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5dbc066f60 msgr2=0x7f5dbc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:29.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.782+0000 7f5de068d640 1 --2- 172.21.15.83:0/3471286163 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5dbc066f60 0x7f5dbc069420 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8152300 tx=0x7f5dc8009040 comp rx=0 tx=0).stop 2024-04-17T10:48:29.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.782+0000 7f5de068d640 1 -- 172.21.15.83:0/3471286163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd80cfbe0 msgr2=0x7f5dd8143130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:29.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.782+0000 7f5de068d640 1 --2- 172.21.15.83:0/3471286163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd80cfbe0 0x7f5dd8143130 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00ae040 tx=0x7f5dd001acd0 comp rx=0 tx=0).stop 2024-04-17T10:48:29.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.782+0000 7f5de068d640 1 -- 172.21.15.83:0/3471286163 shutdown_connections 2024-04-17T10:48:29.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.782+0000 7f5de068d640 1 --2- 172.21.15.83:0/3471286163 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5dbc066f60 0x7f5dbc069420 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:29.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.782+0000 7f5de068d640 1 --2- 172.21.15.83:0/3471286163 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5dd815c960 0x7f5dd81528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:29.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.782+0000 7f5de068d640 1 --2- 172.21.15.83:0/3471286163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dd815ba30 0x7f5dd8158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:29.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.782+0000 7f5de068d640 1 --2- 172.21.15.83:0/3471286163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd80cfbe0 0x7f5dd8143130 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:29.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.782+0000 7f5de068d640 1 -- 172.21.15.83:0/3471286163 >> 172.21.15.83:0/3471286163 conn(0x7f5dd806abe0 msgr2=0x7f5dd815d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:29.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.782+0000 7f5de068d640 1 -- 172.21.15.83:0/3471286163 shutdown_connections 2024-04-17T10:48:29.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:29.782+0000 7f5de068d640 1 -- 172.21.15.83:0/3471286163 wait complete. 2024-04-17T10:48:29.795 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":422,"stamp":"2024-04-17T10:48:28.363691+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":537644,"kb_used_data":11944,"kb_used_omap":12,"kb_used_meta":525555,"kb_avail":754437076,"statfs":{"total":773094113280,"available":772543565824,"internally_reserved":0,"allocated":12230656,"data_stored":5958136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":538168880},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.003886"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.387123+0000","last_change":"2024-04-17T10:48:24.387123+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.387123+0000","last_undegraded":"2024-04-17T10:48:24.387123+0000","last_fullsized":"2024-04-17T10:48:24.387123+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.370522+0000","last_change":"2024-04-17T10:48:24.370522+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.370522+0000","last_undegraded":"2024-04-17T10:48:24.370522+0000","last_fullsized":"2024-04-17T10:48:24.370522+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.379509+0000","last_change":"2024-04-17T10:48:24.379509+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.379509+0000","last_undegraded":"2024-04-17T10:48:24.379509+0000","last_fullsized":"2024-04-17T10:48:24.379509+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.387122+0000","last_change":"2024-04-17T10:48:24.387122+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.387122+0000","last_undegraded":"2024-04-17T10:48:24.387122+0000","last_fullsized":"2024-04-17T10:48:24.387122+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.380195+0000","last_change":"2024-04-17T10:48:24.380195+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.380195+0000","last_undegraded":"2024-04-17T10:48:24.380195+0000","last_fullsized":"2024-04-17T10:48:24.380195+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.378608+0000","last_change":"2024-04-17T10:48:24.378608+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.378608+0000","last_undegraded":"2024-04-17T10:48:24.378608+0000","last_fullsized":"2024-04-17T10:48:24.378608+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.384126+0000","last_change":"2024-04-17T10:48:24.384126+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.384126+0000","last_undegraded":"2024-04-17T10:48:24.384126+0000","last_fullsized":"2024-04-17T10:48:24.384126+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.380148+0000","last_change":"2024-04-17T10:48:24.380148+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.380148+0000","last_undegraded":"2024-04-17T10:48:24.380148+0000","last_fullsized":"2024-04-17T10:48:24.380148+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.386611+0000","last_change":"2024-04-17T10:48:24.386611+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.386611+0000","last_undegraded":"2024-04-17T10:48:24.386611+0000","last_fullsized":"2024-04-17T10:48:24.386611+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.389198+0000","last_change":"2024-04-17T10:48:24.389198+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.389198+0000","last_undegraded":"2024-04-17T10:48:24.389198+0000","last_fullsized":"2024-04-17T10:48:24.389198+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.370124+0000","last_change":"2024-04-17T10:48:24.370124+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.370124+0000","last_undegraded":"2024-04-17T10:48:24.370124+0000","last_fullsized":"2024-04-17T10:48:24.370124+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.386065+0000","last_change":"2024-04-17T10:48:24.386065+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.386065+0000","last_undegraded":"2024-04-17T10:48:24.386065+0000","last_fullsized":"2024-04-17T10:48:24.386065+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.374379+0000","last_change":"2024-04-17T10:48:24.374379+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.374379+0000","last_undegraded":"2024-04-17T10:48:24.374379+0000","last_fullsized":"2024-04-17T10:48:24.374379+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.376316+0000","last_change":"2024-04-17T10:48:24.376316+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.376316+0000","last_undegraded":"2024-04-17T10:48:24.376316+0000","last_fullsized":"2024-04-17T10:48:24.376316+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.375310+0000","last_change":"2024-04-17T10:48:24.375310+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.375310+0000","last_undegraded":"2024-04-17T10:48:24.375310+0000","last_fullsized":"2024-04-17T10:48:24.375310+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.385989+0000","last_change":"2024-04-17T10:48:24.385989+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.385989+0000","last_undegraded":"2024-04-17T10:48:24.385989+0000","last_fullsized":"2024-04-17T10:48:24.385989+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.373575+0000","last_change":"2024-04-17T10:48:24.373575+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.373575+0000","last_undegraded":"2024-04-17T10:48:24.373575+0000","last_fullsized":"2024-04-17T10:48:24.373575+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.385756+0000","last_change":"2024-04-17T10:48:24.385756+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.385756+0000","last_undegraded":"2024-04-17T10:48:24.385756+0000","last_fullsized":"2024-04-17T10:48:24.385756+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.378920+0000","last_change":"2024-04-17T10:48:24.378920+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.378920+0000","last_undegraded":"2024-04-17T10:48:24.378920+0000","last_fullsized":"2024-04-17T10:48:24.378920+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.371256+0000","last_change":"2024-04-17T10:48:24.371256+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.371256+0000","last_undegraded":"2024-04-17T10:48:24.371256+0000","last_fullsized":"2024-04-17T10:48:24.371256+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.370141+0000","last_change":"2024-04-17T10:48:24.370141+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.370141+0000","last_undegraded":"2024-04-17T10:48:24.370141+0000","last_fullsized":"2024-04-17T10:48:24.370141+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.393879+0000","last_change":"2024-04-17T10:48:24.393879+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.393879+0000","last_undegraded":"2024-04-17T10:48:24.393879+0000","last_fullsized":"2024-04-17T10:48:24.393879+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.373549+0000","last_change":"2024-04-17T10:48:24.373549+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.373549+0000","last_undegraded":"2024-04-17T10:48:24.373549+0000","last_fullsized":"2024-04-17T10:48:24.373549+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.393172+0000","last_change":"2024-04-17T10:48:24.393172+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.393172+0000","last_undegraded":"2024-04-17T10:48:24.393172+0000","last_fullsized":"2024-04-17T10:48:24.393172+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.371705+0000","last_change":"2024-04-17T10:48:24.371705+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.371705+0000","last_undegraded":"2024-04-17T10:48:24.371705+0000","last_fullsized":"2024-04-17T10:48:24.371705+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.386061+0000","last_change":"2024-04-17T10:48:24.386061+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.386061+0000","last_undegraded":"2024-04-17T10:48:24.386061+0000","last_fullsized":"2024-04-17T10:48:24.386061+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.374580+0000","last_change":"2024-04-17T10:48:24.374580+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.374580+0000","last_undegraded":"2024-04-17T10:48:24.374580+0000","last_fullsized":"2024-04-17T10:48:24.374580+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.393499+0000","last_change":"2024-04-17T10:48:24.393499+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.393499+0000","last_undegraded":"2024-04-17T10:48:24.393499+0000","last_fullsized":"2024-04-17T10:48:24.393499+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.372097+0000","last_change":"2024-04-17T10:48:24.372097+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.372097+0000","last_undegraded":"2024-04-17T10:48:24.372097+0000","last_fullsized":"2024-04-17T10:48:24.372097+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.389706+0000","last_change":"2024-04-17T10:48:24.389706+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.389706+0000","last_undegraded":"2024-04-17T10:48:24.389706+0000","last_fullsized":"2024-04-17T10:48:24.389706+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.385558+0000","last_change":"2024-04-17T10:48:24.385558+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.385558+0000","last_undegraded":"2024-04-17T10:48:24.385558+0000","last_fullsized":"2024-04-17T10:48:24.385558+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.376758+0000","last_change":"2024-04-17T10:48:24.376758+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.376758+0000","last_undegraded":"2024-04-17T10:48:24.376758+0000","last_fullsized":"2024-04-17T10:48:24.376758+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.375305+0000","last_change":"2024-04-17T10:48:24.375305+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.375305+0000","last_undegraded":"2024-04-17T10:48:24.375305+0000","last_fullsized":"2024-04-17T10:48:24.375305+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401517+0000","last_change":"2024-04-17T10:48:24.396879+0000","last_active":"2024-04-17T10:48:24.401517+0000","last_peered":"2024-04-17T10:48:24.401517+0000","last_clean":"2024-04-17T10:48:24.401517+0000","last_became_active":"2024-04-17T10:48:24.396126+0000","last_became_peered":"2024-04-17T10:48:24.396126+0000","last_unstale":"2024-04-17T10:48:24.401517+0000","last_undegraded":"2024-04-17T10:48:24.401517+0000","last_fullsized":"2024-04-17T10:48:24.401517+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:35:03.367753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.376492+0000","last_change":"2024-04-17T10:48:24.376492+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.376492+0000","last_undegraded":"2024-04-17T10:48:24.376492+0000","last_fullsized":"2024-04-17T10:48:24.376492+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.399197+0000","last_change":"2024-04-17T10:48:24.398459+0000","last_active":"2024-04-17T10:48:24.399197+0000","last_peered":"2024-04-17T10:48:24.399197+0000","last_clean":"2024-04-17T10:48:24.399197+0000","last_became_active":"2024-04-17T10:48:24.398098+0000","last_became_peered":"2024-04-17T10:48:24.398098+0000","last_unstale":"2024-04-17T10:48:24.399197+0000","last_undegraded":"2024-04-17T10:48:24.399197+0000","last_fullsized":"2024-04-17T10:48:24.399197+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:23:07.107332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.379331+0000","last_change":"2024-04-17T10:48:24.379331+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.379331+0000","last_undegraded":"2024-04-17T10:48:24.379331+0000","last_fullsized":"2024-04-17T10:48:24.379331+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.402080+0000","last_change":"2024-04-17T10:48:24.400890+0000","last_active":"2024-04-17T10:48:24.402080+0000","last_peered":"2024-04-17T10:48:24.402080+0000","last_clean":"2024-04-17T10:48:24.402080+0000","last_became_active":"2024-04-17T10:48:24.400472+0000","last_became_peered":"2024-04-17T10:48:24.400472+0000","last_unstale":"2024-04-17T10:48:24.402080+0000","last_undegraded":"2024-04-17T10:48:24.402080+0000","last_fullsized":"2024-04-17T10:48:24.402080+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:20:02.731934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.376494+0000","last_change":"2024-04-17T10:48:24.376494+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.376494+0000","last_undegraded":"2024-04-17T10:48:24.376494+0000","last_fullsized":"2024-04-17T10:48:24.376494+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.402280+0000","last_change":"2024-04-17T10:48:24.401643+0000","last_active":"2024-04-17T10:48:24.402280+0000","last_peered":"2024-04-17T10:48:24.402280+0000","last_clean":"2024-04-17T10:48:24.402280+0000","last_became_active":"2024-04-17T10:48:24.400656+0000","last_became_peered":"2024-04-17T10:48:24.400656+0000","last_unstale":"2024-04-17T10:48:24.402280+0000","last_undegraded":"2024-04-17T10:48:24.402280+0000","last_fullsized":"2024-04-17T10:48:24.402280+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:11:46.753399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.385378+0000","last_change":"2024-04-17T10:48:24.385378+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.385378+0000","last_undegraded":"2024-04-17T10:48:24.385378+0000","last_fullsized":"2024-04-17T10:48:24.385378+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.404934+0000","last_change":"2024-04-17T10:48:24.403866+0000","last_active":"2024-04-17T10:48:24.404934+0000","last_peered":"2024-04-17T10:48:24.404934+0000","last_clean":"2024-04-17T10:48:24.404934+0000","last_became_active":"2024-04-17T10:48:24.403129+0000","last_became_peered":"2024-04-17T10:48:24.403129+0000","last_unstale":"2024-04-17T10:48:24.404934+0000","last_undegraded":"2024-04-17T10:48:24.404934+0000","last_fullsized":"2024-04-17T10:48:24.404934+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:34:15.196696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.371493+0000","last_change":"2024-04-17T10:48:24.371493+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.371493+0000","last_undegraded":"2024-04-17T10:48:24.371493+0000","last_fullsized":"2024-04-17T10:48:24.371493+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.402426+0000","last_change":"2024-04-17T10:48:24.399589+0000","last_active":"2024-04-17T10:48:24.402426+0000","last_peered":"2024-04-17T10:48:24.402426+0000","last_clean":"2024-04-17T10:48:24.402426+0000","last_became_active":"2024-04-17T10:48:24.399097+0000","last_became_peered":"2024-04-17T10:48:24.399097+0000","last_unstale":"2024-04-17T10:48:24.402426+0000","last_undegraded":"2024-04-17T10:48:24.402426+0000","last_fullsized":"2024-04-17T10:48:24.402426+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:01:53.668168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.385357+0000","last_change":"2024-04-17T10:48:24.385357+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.385357+0000","last_undegraded":"2024-04-17T10:48:24.385357+0000","last_fullsized":"2024-04-17T10:48:24.385357+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.379620+0000","last_change":"2024-04-17T10:48:24.376241+0000","last_active":"2024-04-17T10:48:24.379620+0000","last_peered":"2024-04-17T10:48:24.379620+0000","last_clean":"2024-04-17T10:48:24.379620+0000","last_became_active":"2024-04-17T10:48:24.375430+0000","last_became_peered":"2024-04-17T10:48:24.375430+0000","last_unstale":"2024-04-17T10:48:24.379620+0000","last_undegraded":"2024-04-17T10:48:24.379620+0000","last_fullsized":"2024-04-17T10:48:24.379620+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:46:07.697646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.384767+0000","last_change":"2024-04-17T10:48:24.384767+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.384767+0000","last_undegraded":"2024-04-17T10:48:24.384767+0000","last_fullsized":"2024-04-17T10:48:24.384767+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.385628+0000","last_change":"2024-04-17T10:48:24.382363+0000","last_active":"2024-04-17T10:48:24.385628+0000","last_peered":"2024-04-17T10:48:24.385628+0000","last_clean":"2024-04-17T10:48:24.385628+0000","last_became_active":"2024-04-17T10:48:24.381493+0000","last_became_peered":"2024-04-17T10:48:24.381493+0000","last_unstale":"2024-04-17T10:48:24.385628+0000","last_undegraded":"2024-04-17T10:48:24.385628+0000","last_fullsized":"2024-04-17T10:48:24.385628+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:06:35.936896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.389344+0000","last_change":"2024-04-17T10:48:24.389344+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.389344+0000","last_undegraded":"2024-04-17T10:48:24.389344+0000","last_fullsized":"2024-04-17T10:48:24.389344+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401814+0000","last_change":"2024-04-17T10:48:24.400839+0000","last_active":"2024-04-17T10:48:24.401814+0000","last_peered":"2024-04-17T10:48:24.401814+0000","last_clean":"2024-04-17T10:48:24.401814+0000","last_became_active":"2024-04-17T10:48:24.400489+0000","last_became_peered":"2024-04-17T10:48:24.400489+0000","last_unstale":"2024-04-17T10:48:24.401814+0000","last_undegraded":"2024-04-17T10:48:24.401814+0000","last_fullsized":"2024-04-17T10:48:24.401814+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:42:13.356381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.375839+0000","last_change":"2024-04-17T10:48:24.375839+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.375839+0000","last_undegraded":"2024-04-17T10:48:24.375839+0000","last_fullsized":"2024-04-17T10:48:24.375839+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.402374+0000","last_change":"2024-04-17T10:48:24.399209+0000","last_active":"2024-04-17T10:48:24.402374+0000","last_peered":"2024-04-17T10:48:24.402374+0000","last_clean":"2024-04-17T10:48:24.402374+0000","last_became_active":"2024-04-17T10:48:24.398789+0000","last_became_peered":"2024-04-17T10:48:24.398789+0000","last_unstale":"2024-04-17T10:48:24.402374+0000","last_undegraded":"2024-04-17T10:48:24.402374+0000","last_fullsized":"2024-04-17T10:48:24.402374+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:33:31.610895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.391756+0000","last_change":"2024-04-17T10:48:24.391756+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.391756+0000","last_undegraded":"2024-04-17T10:48:24.391756+0000","last_fullsized":"2024-04-17T10:48:24.391756+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.402960+0000","last_change":"2024-04-17T10:48:24.402346+0000","last_active":"2024-04-17T10:48:24.402960+0000","last_peered":"2024-04-17T10:48:24.402960+0000","last_clean":"2024-04-17T10:48:24.402960+0000","last_became_active":"2024-04-17T10:48:24.401900+0000","last_became_peered":"2024-04-17T10:48:24.401900+0000","last_unstale":"2024-04-17T10:48:24.402960+0000","last_undegraded":"2024-04-17T10:48:24.402960+0000","last_fullsized":"2024-04-17T10:48:24.402960+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:23:07.107332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.390503+0000","last_change":"2024-04-17T10:48:24.390503+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.390503+0000","last_undegraded":"2024-04-17T10:48:24.390503+0000","last_fullsized":"2024-04-17T10:48:24.390503+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.382233+0000","last_change":"2024-04-17T10:48:24.379794+0000","last_active":"2024-04-17T10:48:24.382233+0000","last_peered":"2024-04-17T10:48:24.382233+0000","last_clean":"2024-04-17T10:48:24.382233+0000","last_became_active":"2024-04-17T10:48:24.379413+0000","last_became_peered":"2024-04-17T10:48:24.379413+0000","last_unstale":"2024-04-17T10:48:24.382233+0000","last_undegraded":"2024-04-17T10:48:24.382233+0000","last_fullsized":"2024-04-17T10:48:24.382233+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:42:10.087214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.391471+0000","last_change":"2024-04-17T10:48:24.391471+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.391471+0000","last_undegraded":"2024-04-17T10:48:24.391471+0000","last_fullsized":"2024-04-17T10:48:24.391471+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401761+0000","last_change":"2024-04-17T10:48:24.400108+0000","last_active":"2024-04-17T10:48:24.401761+0000","last_peered":"2024-04-17T10:48:24.401761+0000","last_clean":"2024-04-17T10:48:24.401761+0000","last_became_active":"2024-04-17T10:48:24.397908+0000","last_became_peered":"2024-04-17T10:48:24.397908+0000","last_unstale":"2024-04-17T10:48:24.401761+0000","last_undegraded":"2024-04-17T10:48:24.401761+0000","last_fullsized":"2024-04-17T10:48:24.401761+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:11:46.753399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.393082+0000","last_change":"2024-04-17T10:48:24.393082+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.393082+0000","last_undegraded":"2024-04-17T10:48:24.393082+0000","last_fullsized":"2024-04-17T10:48:24.393082+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.392836+0000","last_change":"2024-04-17T10:48:24.391544+0000","last_active":"2024-04-17T10:48:24.392836+0000","last_peered":"2024-04-17T10:48:24.392836+0000","last_clean":"2024-04-17T10:48:24.392836+0000","last_became_active":"2024-04-17T10:48:24.388072+0000","last_became_peered":"2024-04-17T10:48:24.388072+0000","last_unstale":"2024-04-17T10:48:24.392836+0000","last_undegraded":"2024-04-17T10:48:24.392836+0000","last_fullsized":"2024-04-17T10:48:24.392836+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:03:44.300221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.392104+0000","last_change":"2024-04-17T10:48:24.392104+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.392104+0000","last_undegraded":"2024-04-17T10:48:24.392104+0000","last_fullsized":"2024-04-17T10:48:24.392104+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.389451+0000","last_change":"2024-04-17T10:48:24.383457+0000","last_active":"2024-04-17T10:48:24.389451+0000","last_peered":"2024-04-17T10:48:24.389451+0000","last_clean":"2024-04-17T10:48:24.389451+0000","last_became_active":"2024-04-17T10:48:24.382932+0000","last_became_peered":"2024-04-17T10:48:24.382932+0000","last_unstale":"2024-04-17T10:48:24.389451+0000","last_undegraded":"2024-04-17T10:48:24.389451+0000","last_fullsized":"2024-04-17T10:48:24.389451+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:53:06.465373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.376491+0000","last_change":"2024-04-17T10:48:24.376491+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.376491+0000","last_undegraded":"2024-04-17T10:48:24.376491+0000","last_fullsized":"2024-04-17T10:48:24.376491+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.399747+0000","last_change":"2024-04-17T10:48:24.399014+0000","last_active":"2024-04-17T10:48:24.399747+0000","last_peered":"2024-04-17T10:48:24.399747+0000","last_clean":"2024-04-17T10:48:24.399747+0000","last_became_active":"2024-04-17T10:48:24.398705+0000","last_became_peered":"2024-04-17T10:48:24.398705+0000","last_unstale":"2024-04-17T10:48:24.399747+0000","last_undegraded":"2024-04-17T10:48:24.399747+0000","last_fullsized":"2024-04-17T10:48:24.399747+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:01:53.668168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.374704+0000","last_change":"2024-04-17T10:48:24.374704+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.374704+0000","last_undegraded":"2024-04-17T10:48:24.374704+0000","last_fullsized":"2024-04-17T10:48:24.374704+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.390926+0000","last_change":"2024-04-17T10:48:24.387073+0000","last_active":"2024-04-17T10:48:24.390926+0000","last_peered":"2024-04-17T10:48:24.390926+0000","last_clean":"2024-04-17T10:48:24.390926+0000","last_became_active":"2024-04-17T10:48:24.386344+0000","last_became_peered":"2024-04-17T10:48:24.386344+0000","last_unstale":"2024-04-17T10:48:24.390926+0000","last_undegraded":"2024-04-17T10:48:24.390926+0000","last_fullsized":"2024-04-17T10:48:24.390926+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:09:38.552029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.373427+0000","last_change":"2024-04-17T10:48:24.373427+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.373427+0000","last_undegraded":"2024-04-17T10:48:24.373427+0000","last_fullsized":"2024-04-17T10:48:24.373427+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.385549+0000","last_change":"2024-04-17T10:48:24.382682+0000","last_active":"2024-04-17T10:48:24.385549+0000","last_peered":"2024-04-17T10:48:24.385549+0000","last_clean":"2024-04-17T10:48:24.385549+0000","last_became_active":"2024-04-17T10:48:24.381331+0000","last_became_peered":"2024-04-17T10:48:24.381331+0000","last_unstale":"2024-04-17T10:48:24.385549+0000","last_undegraded":"2024-04-17T10:48:24.385549+0000","last_fullsized":"2024-04-17T10:48:24.385549+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:15:43.459938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.373479+0000","last_change":"2024-04-17T10:48:24.373479+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.373479+0000","last_undegraded":"2024-04-17T10:48:24.373479+0000","last_fullsized":"2024-04-17T10:48:24.373479+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401395+0000","last_change":"2024-04-17T10:48:24.396200+0000","last_active":"2024-04-17T10:48:24.401395+0000","last_peered":"2024-04-17T10:48:24.401395+0000","last_clean":"2024-04-17T10:48:24.401395+0000","last_became_active":"2024-04-17T10:48:24.395802+0000","last_became_peered":"2024-04-17T10:48:24.395802+0000","last_unstale":"2024-04-17T10:48:24.401395+0000","last_undegraded":"2024-04-17T10:48:24.401395+0000","last_fullsized":"2024-04-17T10:48:24.401395+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:13:48.163535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.374303+0000","last_change":"2024-04-17T10:48:24.374303+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.374303+0000","last_undegraded":"2024-04-17T10:48:24.374303+0000","last_fullsized":"2024-04-17T10:48:24.374303+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401739+0000","last_change":"2024-04-17T10:48:24.397774+0000","last_active":"2024-04-17T10:48:24.401739+0000","last_peered":"2024-04-17T10:48:24.401739+0000","last_clean":"2024-04-17T10:48:24.401739+0000","last_became_active":"2024-04-17T10:48:24.397453+0000","last_became_peered":"2024-04-17T10:48:24.397453+0000","last_unstale":"2024-04-17T10:48:24.401739+0000","last_undegraded":"2024-04-17T10:48:24.401739+0000","last_fullsized":"2024-04-17T10:48:24.401739+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:20:23.871607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.384650+0000","last_change":"2024-04-17T10:48:24.384650+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.384650+0000","last_undegraded":"2024-04-17T10:48:24.384650+0000","last_fullsized":"2024-04-17T10:48:24.384650+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401589+0000","last_change":"2024-04-17T10:48:24.400725+0000","last_active":"2024-04-17T10:48:24.401589+0000","last_peered":"2024-04-17T10:48:24.401589+0000","last_clean":"2024-04-17T10:48:24.401589+0000","last_became_active":"2024-04-17T10:48:24.400444+0000","last_became_peered":"2024-04-17T10:48:24.400444+0000","last_unstale":"2024-04-17T10:48:24.401589+0000","last_undegraded":"2024-04-17T10:48:24.401589+0000","last_fullsized":"2024-04-17T10:48:24.401589+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:05:55.306345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.379208+0000","last_change":"2024-04-17T10:48:24.379208+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.379208+0000","last_undegraded":"2024-04-17T10:48:24.379208+0000","last_fullsized":"2024-04-17T10:48:24.379208+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401440+0000","last_change":"2024-04-17T10:48:24.397713+0000","last_active":"2024-04-17T10:48:24.401440+0000","last_peered":"2024-04-17T10:48:24.401440+0000","last_clean":"2024-04-17T10:48:24.401440+0000","last_became_active":"2024-04-17T10:48:24.397264+0000","last_became_peered":"2024-04-17T10:48:24.397264+0000","last_unstale":"2024-04-17T10:48:24.401440+0000","last_undegraded":"2024-04-17T10:48:24.401440+0000","last_fullsized":"2024-04-17T10:48:24.401440+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:20:02.731934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.390331+0000","last_change":"2024-04-17T10:48:24.390331+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.390331+0000","last_undegraded":"2024-04-17T10:48:24.390331+0000","last_fullsized":"2024-04-17T10:48:24.390331+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.395732+0000","last_change":"2024-04-17T10:48:24.393648+0000","last_active":"2024-04-17T10:48:24.395732+0000","last_peered":"2024-04-17T10:48:24.395732+0000","last_clean":"2024-04-17T10:48:24.395732+0000","last_became_active":"2024-04-17T10:48:24.392862+0000","last_became_peered":"2024-04-17T10:48:24.392862+0000","last_unstale":"2024-04-17T10:48:24.395732+0000","last_undegraded":"2024-04-17T10:48:24.395732+0000","last_fullsized":"2024-04-17T10:48:24.395732+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:33:31.610895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.384853+0000","last_change":"2024-04-17T10:48:24.384853+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.384853+0000","last_undegraded":"2024-04-17T10:48:24.384853+0000","last_fullsized":"2024-04-17T10:48:24.384853+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.394096+0000","last_change":"2024-04-17T10:48:24.389637+0000","last_active":"2024-04-17T10:48:24.394096+0000","last_peered":"2024-04-17T10:48:24.394096+0000","last_clean":"2024-04-17T10:48:24.394096+0000","last_became_active":"2024-04-17T10:48:24.389212+0000","last_became_peered":"2024-04-17T10:48:24.389212+0000","last_unstale":"2024-04-17T10:48:24.394096+0000","last_undegraded":"2024-04-17T10:48:24.394096+0000","last_fullsized":"2024-04-17T10:48:24.394096+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:36:28.852444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.384672+0000","last_change":"2024-04-17T10:48:24.384672+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.384672+0000","last_undegraded":"2024-04-17T10:48:24.384672+0000","last_fullsized":"2024-04-17T10:48:24.384672+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.400185+0000","last_change":"2024-04-17T10:48:24.398460+0000","last_active":"2024-04-17T10:48:24.400185+0000","last_peered":"2024-04-17T10:48:24.400185+0000","last_clean":"2024-04-17T10:48:24.400185+0000","last_became_active":"2024-04-17T10:48:24.397860+0000","last_became_peered":"2024-04-17T10:48:24.397860+0000","last_unstale":"2024-04-17T10:48:24.400185+0000","last_undegraded":"2024-04-17T10:48:24.400185+0000","last_fullsized":"2024-04-17T10:48:24.400185+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:20:18.572385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.384643+0000","last_change":"2024-04-17T10:48:24.384643+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.384643+0000","last_undegraded":"2024-04-17T10:48:24.384643+0000","last_fullsized":"2024-04-17T10:48:24.384643+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.395517+0000","last_change":"2024-04-17T10:48:24.391043+0000","last_active":"2024-04-17T10:48:24.395517+0000","last_peered":"2024-04-17T10:48:24.395517+0000","last_clean":"2024-04-17T10:48:24.395517+0000","last_became_active":"2024-04-17T10:48:24.390673+0000","last_became_peered":"2024-04-17T10:48:24.390673+0000","last_unstale":"2024-04-17T10:48:24.395517+0000","last_undegraded":"2024-04-17T10:48:24.395517+0000","last_fullsized":"2024-04-17T10:48:24.395517+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:36:28.852444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.375853+0000","last_change":"2024-04-17T10:48:24.375853+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.375853+0000","last_undegraded":"2024-04-17T10:48:24.375853+0000","last_fullsized":"2024-04-17T10:48:24.375853+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.400107+0000","last_change":"2024-04-17T10:48:24.398215+0000","last_active":"2024-04-17T10:48:24.400107+0000","last_peered":"2024-04-17T10:48:24.400107+0000","last_clean":"2024-04-17T10:48:24.400107+0000","last_became_active":"2024-04-17T10:48:24.394779+0000","last_became_peered":"2024-04-17T10:48:24.394779+0000","last_unstale":"2024-04-17T10:48:24.400107+0000","last_undegraded":"2024-04-17T10:48:24.400107+0000","last_fullsized":"2024-04-17T10:48:24.400107+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:54:47.216960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.370200+0000","last_change":"2024-04-17T10:48:24.370200+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.370200+0000","last_undegraded":"2024-04-17T10:48:24.370200+0000","last_fullsized":"2024-04-17T10:48:24.370200+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401418+0000","last_change":"2024-04-17T10:48:24.396294+0000","last_active":"2024-04-17T10:48:24.401418+0000","last_peered":"2024-04-17T10:48:24.401418+0000","last_clean":"2024-04-17T10:48:24.401418+0000","last_became_active":"2024-04-17T10:48:24.395673+0000","last_became_peered":"2024-04-17T10:48:24.395673+0000","last_unstale":"2024-04-17T10:48:24.401418+0000","last_undegraded":"2024-04-17T10:48:24.401418+0000","last_fullsized":"2024-04-17T10:48:24.401418+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:16:43.238162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.370856+0000","last_change":"2024-04-17T10:48:24.370856+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.370856+0000","last_undegraded":"2024-04-17T10:48:24.370856+0000","last_fullsized":"2024-04-17T10:48:24.370856+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.387789+0000","last_change":"2024-04-17T10:48:24.386727+0000","last_active":"2024-04-17T10:48:24.387789+0000","last_peered":"2024-04-17T10:48:24.387789+0000","last_clean":"2024-04-17T10:48:24.387789+0000","last_became_active":"2024-04-17T10:48:24.386334+0000","last_became_peered":"2024-04-17T10:48:24.386334+0000","last_unstale":"2024-04-17T10:48:24.387789+0000","last_undegraded":"2024-04-17T10:48:24.387789+0000","last_fullsized":"2024-04-17T10:48:24.387789+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:09.646139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.378187+0000","last_change":"2024-04-17T10:48:24.378187+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.378187+0000","last_undegraded":"2024-04-17T10:48:24.378187+0000","last_fullsized":"2024-04-17T10:48:24.378187+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.402062+0000","last_change":"2024-04-17T10:48:24.401190+0000","last_active":"2024-04-17T10:48:24.402062+0000","last_peered":"2024-04-17T10:48:24.402062+0000","last_clean":"2024-04-17T10:48:24.402062+0000","last_became_active":"2024-04-17T10:48:24.400909+0000","last_became_peered":"2024-04-17T10:48:24.400909+0000","last_unstale":"2024-04-17T10:48:24.402062+0000","last_undegraded":"2024-04-17T10:48:24.402062+0000","last_fullsized":"2024-04-17T10:48:24.402062+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:20:23.871607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.383491+0000","last_change":"2024-04-17T10:48:24.383491+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.383491+0000","last_undegraded":"2024-04-17T10:48:24.383491+0000","last_fullsized":"2024-04-17T10:48:24.383491+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.395697+0000","last_change":"2024-04-17T10:48:24.393593+0000","last_active":"2024-04-17T10:48:24.395697+0000","last_peered":"2024-04-17T10:48:24.395697+0000","last_clean":"2024-04-17T10:48:24.395697+0000","last_became_active":"2024-04-17T10:48:24.392876+0000","last_became_peered":"2024-04-17T10:48:24.392876+0000","last_unstale":"2024-04-17T10:48:24.395697+0000","last_undegraded":"2024-04-17T10:48:24.395697+0000","last_fullsized":"2024-04-17T10:48:24.395697+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:54:47.216960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-04-17T10:48:24.390419+0000","last_change":"2024-04-17T10:48:24.390419+0000","last_active":"2024-04-17T10:48:24.357930+0000","last_peered":"2024-04-17T10:48:24.357930+0000","last_clean":"2024-04-17T10:48:24.357930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:48:24.390419+0000","last_undegraded":"2024-04-17T10:48:24.390419+0000","last_fullsized":"2024-04-17T10:48:24.390419+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.401545+0000","last_change":"2024-04-17T10:48:24.396850+0000","last_active":"2024-04-17T10:48:24.401545+0000","last_peered":"2024-04-17T10:48:24.401545+0000","last_clean":"2024-04-17T10:48:24.401545+0000","last_became_active":"2024-04-17T10:48:24.396194+0000","last_became_peered":"2024-04-17T10:48:24.396194+0000","last_unstale":"2024-04-17T10:48:24.401545+0000","last_undegraded":"2024-04-17T10:48:24.401545+0000","last_fullsized":"2024-04-17T10:48:24.401545+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:20:02.139331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":359,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-17T10:48:24.387430+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:48:24.387430+0000","last_peered":"2024-04-17T10:48:24.387430+0000","last_clean":"2024-04-17T10:48:24.387430+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:48:24.387430+0000","last_undegraded":"2024-04-17T10:48:24.387430+0000","last_fullsized":"2024-04-17T10:48:24.387430+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574977,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59476,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312364,"statfs":{"total":96636764160,"available":96575860736,"internally_reserved":0,"allocated":1974272,"data_stored":1187543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574976,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68624,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303216,"statfs":{"total":96636764160,"available":96566493184,"internally_reserved":0,"allocated":1380352,"data_stored":597175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574977,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54736,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317104,"statfs":{"total":96636764160,"available":96580714496,"internally_reserved":0,"allocated":1380352,"data_stored":597175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68636,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303204,"statfs":{"total":96636764160,"available":96566480896,"internally_reserved":0,"allocated":1380352,"data_stored":597175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574978,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68624,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303216,"statfs":{"total":96636764160,"available":96566493184,"internally_reserved":0,"allocated":1380352,"data_stored":597175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58908,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312932,"statfs":{"total":96636764160,"available":96576442368,"internally_reserved":0,"allocated":1380352,"data_stored":597175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94996,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94276844,"statfs":{"total":96636764160,"available":96539488256,"internally_reserved":0,"allocated":1380352,"data_stored":597175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574976,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63644,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94308196,"statfs":{"total":96636764160,"available":96571592704,"internally_reserved":0,"allocated":1974272,"data_stored":1187543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:48:31.348 INFO:tasks.ceph.mds.a.smithi083.stderr:2024-04-17T10:48:31.350+0000 7f8bae298640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T10:48:32.800 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:48:32.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.934+0000 7fa494099640 1 Processor -- start 2024-04-17T10:48:32.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.934+0000 7fa494099640 1 -- start start 2024-04-17T10:48:32.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.934+0000 7fa494099640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa48c15de30 0x7fa48c0ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:32.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.934+0000 7fa494099640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa48c0b6cd0 0x7fa48c0d2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:32.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.934+0000 7fa494099640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa48c0ced00 con 0x7fa48c0ce860 2024-04-17T10:48:32.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.934+0000 7fa494099640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa48c15e210 con 0x7fa48c0b6cd0 2024-04-17T10:48:32.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.934+0000 7fa494099640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa48c0cefe0 con 0x7fa48c15de30 2024-04-17T10:48:32.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.934+0000 7fa492636640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa48c0b6cd0 0x7fa48c0d2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:32.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.934+0000 7fa491634640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa48c0ce860 0x7fa48c0d02d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40980/0 (socket says 172.21.15.83:40980) 2024-04-17T10:48:32.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.934+0000 7fa491634640 1 -- 172.21.15.83:0/1055915951 learned_addr learned my addr 172.21.15.83:0/1055915951 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:32.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.934+0000 7fa491e35640 1 --2- 172.21.15.83:0/1055915951 >> v2:172.21.15.167:3300/0 conn(0x7fa48c15de30 0x7fa48c0ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:32.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.934+0000 7fa492636640 1 -- 172.21.15.83:0/1055915951 >> v2:172.21.15.167:3300/0 conn(0x7fa48c15de30 msgr2=0x7fa48c0ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:32.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.934+0000 7fa492636640 1 --2- 172.21.15.83:0/1055915951 >> v2:172.21.15.167:3300/0 conn(0x7fa48c15de30 0x7fa48c0ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:32.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.934+0000 7fa492636640 1 -- 172.21.15.83:0/1055915951 >> v1:172.21.15.83:6789/0 conn(0x7fa48c0ce860 legacy=0x7fa48c0d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:32.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.934+0000 7fa492636640 1 -- 172.21.15.83:0/1055915951 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa48c0d2040 con 0x7fa48c0b6cd0 2024-04-17T10:48:32.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.938+0000 7fa492636640 1 --2- 172.21.15.83:0/1055915951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa48c0b6cd0 0x7fa48c0d2840 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fa48c08c650 tx=0x7fa48401c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7cfbdfdf049b5c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:32.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.938+0000 7fa490e33640 1 -- 172.21.15.83:0/1055915951 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4840b2070 con 0x7fa48c0b6cd0 2024-04-17T10:48:32.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.938+0000 7fa490e33640 1 -- 172.21.15.83:0/1055915951 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4840a5720 con 0x7fa48c0b6cd0 2024-04-17T10:48:32.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.938+0000 7fa490e33640 1 -- 172.21.15.83:0/1055915951 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4840a5af0 con 0x7fa48c0b6cd0 2024-04-17T10:48:32.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.938+0000 7fa494099640 1 -- 172.21.15.83:0/1055915951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa48c0b6cd0 msgr2=0x7fa48c0d2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:32.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.938+0000 7fa494099640 1 --2- 172.21.15.83:0/1055915951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa48c0b6cd0 0x7fa48c0d2840 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fa48c08c650 tx=0x7fa48401c700 comp rx=0 tx=0).stop 2024-04-17T10:48:32.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.938+0000 7fa494099640 1 -- 172.21.15.83:0/1055915951 shutdown_connections 2024-04-17T10:48:32.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.938+0000 7fa494099640 1 --2- 172.21.15.83:0/1055915951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa48c0b6cd0 0x7fa48c0d2840 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:32.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.938+0000 7fa494099640 1 --2- 172.21.15.83:0/1055915951 >> v2:172.21.15.167:3300/0 conn(0x7fa48c15de30 0x7fa48c0ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:32.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.938+0000 7fa494099640 1 -- 172.21.15.83:0/1055915951 >> 172.21.15.83:0/1055915951 conn(0x7fa48c06b0b0 msgr2=0x7fa48c06b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:32.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.938+0000 7fa494099640 1 -- 172.21.15.83:0/1055915951 shutdown_connections 2024-04-17T10:48:32.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.938+0000 7fa494099640 1 -- 172.21.15.83:0/1055915951 wait complete. 2024-04-17T10:48:32.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.938+0000 7fa494099640 1 Processor -- start 2024-04-17T10:48:32.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.938+0000 7fa494099640 1 -- start start 2024-04-17T10:48:32.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.938+0000 7fa494099640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa48c0b6cd0 0x7fa48c154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:32.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.938+0000 7fa494099640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa48c0ce860 0x7fa48c156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:32.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.938+0000 7fa494099640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa48c15de30 0x7fa48c0b7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:32.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.938+0000 7fa494099640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c0d3710 con 0x7fa48c0b6cd0 2024-04-17T10:48:32.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.938+0000 7fa494099640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c0d3590 con 0x7fa48c0ce860 2024-04-17T10:48:32.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.938+0000 7fa494099640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c16abe0 con 0x7fa48c15de30 2024-04-17T10:48:32.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.938+0000 7fa491e35640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa48c0b6cd0 0x7fa48c154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:32.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.938+0000 7fa491e35640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa48c0b6cd0 0x7fa48c154a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44754/0 (socket says 172.21.15.83:44754) 2024-04-17T10:48:32.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.938+0000 7fa491e35640 1 -- 172.21.15.83:0/1739841220 learned_addr learned my addr 172.21.15.83:0/1739841220 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:32.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.938+0000 7fa491634640 1 --2- 172.21.15.83:0/1739841220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa48c0ce860 0x7fa48c156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:32.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.938+0000 7fa492636640 1 --2- 172.21.15.83:0/1739841220 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa48c15de30 0x7fa48c0b7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:32.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.938+0000 7fa491e35640 1 -- 172.21.15.83:0/1739841220 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa48c15de30 msgr2=0x7fa48c0b7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:32.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.938+0000 7fa491e35640 1 --2- 172.21.15.83:0/1739841220 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa48c15de30 0x7fa48c0b7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:32.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.938+0000 7fa491e35640 1 -- 172.21.15.83:0/1739841220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa48c0ce860 msgr2=0x7fa48c156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:32.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.938+0000 7fa491e35640 1 --2- 172.21.15.83:0/1739841220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa48c0ce860 0x7fa48c156ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:32.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.938+0000 7fa491e35640 1 -- 172.21.15.83:0/1739841220 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa48c0c7900 con 0x7fa48c0b6cd0 2024-04-17T10:48:32.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.938+0000 7fa491634640 1 --2- 172.21.15.83:0/1739841220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa48c0ce860 0x7fa48c156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:32.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.938+0000 7fa491e35640 1 --2- 172.21.15.83:0/1739841220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa48c0b6cd0 0x7fa48c154a60 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fa48800d800 tx=0x7fa48800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:32.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.942+0000 7fa4827fc640 1 -- 172.21.15.83:0/1739841220 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48800b6c0 con 0x7fa48c0b6cd0 2024-04-17T10:48:32.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.942+0000 7fa4827fc640 1 -- 172.21.15.83:0/1739841220 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa488019070 con 0x7fa48c0b6cd0 2024-04-17T10:48:32.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.942+0000 7fa4827fc640 1 -- 172.21.15.83:0/1739841220 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48800ba60 con 0x7fa48c0b6cd0 2024-04-17T10:48:32.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.942+0000 7fa494099640 1 -- 172.21.15.83:0/1739841220 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa48c0c7be0 con 0x7fa48c0b6cd0 2024-04-17T10:48:32.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.942+0000 7fa494099640 1 -- 172.21.15.83:0/1739841220 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa48c0c81a0 con 0x7fa48c0b6cd0 2024-04-17T10:48:32.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.942+0000 7fa494099640 1 -- 172.21.15.83:0/1739841220 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa48c074f90 con 0x7fa48c0b6cd0 2024-04-17T10:48:32.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.942+0000 7fa4827fc640 1 -- 172.21.15.83:0/1739841220 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa48800bc00 con 0x7fa48c0b6cd0 2024-04-17T10:48:32.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.942+0000 7fa4827fc640 1 --2- 172.21.15.83:0/1739841220 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa470066f60 0x7fa470069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:32.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.942+0000 7fa491634640 1 --2- 172.21.15.83:0/1739841220 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa470066f60 0x7fa470069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:32.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.942+0000 7fa4827fc640 1 -- 172.21.15.83:0/1739841220 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa488085570 con 0x7fa48c0b6cd0 2024-04-17T10:48:32.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.942+0000 7fa491634640 1 --2- 172.21.15.83:0/1739841220 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa470066f60 0x7fa470069420 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0cf8a0 tx=0x7fa47c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:32.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:32.946+0000 7fa4827fc640 1 -- 172.21.15.83:0/1739841220 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa4880564e0 con 0x7fa48c0b6cd0 2024-04-17T10:48:33.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.102+0000 7fa494099640 1 -- 172.21.15.83:0/1739841220 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa48c0d17e0 con 0x7fa470066f60 2024-04-17T10:48:33.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.110+0000 7fa4827fc640 1 -- 172.21.15.83:0/1739841220 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252483 (secure 0 0 0) 0x7fa48c0d17e0 con 0x7fa470066f60 2024-04-17T10:48:33.108 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:33.110 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:48:33.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.114+0000 7fa494099640 1 -- 172.21.15.83:0/1739841220 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa470066f60 msgr2=0x7fa470069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:33.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.114+0000 7fa494099640 1 --2- 172.21.15.83:0/1739841220 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa470066f60 0x7fa470069420 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0cf8a0 tx=0x7fa47c009040 comp rx=0 tx=0).stop 2024-04-17T10:48:33.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.114+0000 7fa494099640 1 -- 172.21.15.83:0/1739841220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa48c0b6cd0 msgr2=0x7fa48c154a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:33.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.114+0000 7fa494099640 1 --2- 172.21.15.83:0/1739841220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa48c0b6cd0 0x7fa48c154a60 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fa48800d800 tx=0x7fa48800dcc0 comp rx=0 tx=0).stop 2024-04-17T10:48:33.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.114+0000 7fa494099640 1 -- 172.21.15.83:0/1739841220 shutdown_connections 2024-04-17T10:48:33.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.114+0000 7fa494099640 1 --2- 172.21.15.83:0/1739841220 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa470066f60 0x7fa470069420 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:33.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.114+0000 7fa494099640 1 --2- 172.21.15.83:0/1739841220 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa48c15de30 0x7fa48c0b7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:33.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.114+0000 7fa494099640 1 --2- 172.21.15.83:0/1739841220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa48c0ce860 0x7fa48c156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:33.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.114+0000 7fa494099640 1 --2- 172.21.15.83:0/1739841220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa48c0b6cd0 0x7fa48c154a60 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:33.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.114+0000 7fa494099640 1 -- 172.21.15.83:0/1739841220 >> 172.21.15.83:0/1739841220 conn(0x7fa48c06b0b0 msgr2=0x7fa48c06b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:33.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.114+0000 7fa494099640 1 -- 172.21.15.83:0/1739841220 shutdown_connections 2024-04-17T10:48:33.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.114+0000 7fa494099640 1 -- 172.21.15.83:0/1739841220 wait complete. 2024-04-17T10:48:33.126 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":424,"stamp":"2024-04-17T10:48:32.365216+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":180,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":537976,"kb_used_data":12296,"kb_used_omap":12,"kb_used_meta":525555,"kb_avail":754436744,"statfs":{"total":773094113280,"available":772543225856,"internally_reserved":0,"allocated":12591104,"data_stored":6117756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":538168880},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.998823"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.769910+0000","last_change":"2024-04-17T10:48:25.407898+0000","last_active":"2024-04-17T10:48:26.769910+0000","last_peered":"2024-04-17T10:48:26.769910+0000","last_clean":"2024-04-17T10:48:26.769910+0000","last_became_active":"2024-04-17T10:48:25.407565+0000","last_became_peered":"2024-04-17T10:48:25.407565+0000","last_unstale":"2024-04-17T10:48:26.769910+0000","last_undegraded":"2024-04-17T10:48:26.769910+0000","last_fullsized":"2024-04-17T10:48:26.769910+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:12:33.377897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.374694+0000","last_change":"2024-04-17T10:48:25.410384+0000","last_active":"2024-04-17T10:48:26.374694+0000","last_peered":"2024-04-17T10:48:26.374694+0000","last_clean":"2024-04-17T10:48:26.374694+0000","last_became_active":"2024-04-17T10:48:25.409599+0000","last_became_peered":"2024-04-17T10:48:25.409599+0000","last_unstale":"2024-04-17T10:48:26.374694+0000","last_undegraded":"2024-04-17T10:48:26.374694+0000","last_fullsized":"2024-04-17T10:48:26.374694+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:08:59.966356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.373044+0000","last_change":"2024-04-17T10:48:25.396326+0000","last_active":"2024-04-17T10:48:26.373044+0000","last_peered":"2024-04-17T10:48:26.373044+0000","last_clean":"2024-04-17T10:48:26.373044+0000","last_became_active":"2024-04-17T10:48:25.395546+0000","last_became_peered":"2024-04-17T10:48:25.395546+0000","last_unstale":"2024-04-17T10:48:26.373044+0000","last_undegraded":"2024-04-17T10:48:26.373044+0000","last_fullsized":"2024-04-17T10:48:26.373044+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:39:54.936657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.768301+0000","last_change":"2024-04-17T10:48:25.412277+0000","last_active":"2024-04-17T10:48:26.768301+0000","last_peered":"2024-04-17T10:48:26.768301+0000","last_clean":"2024-04-17T10:48:26.768301+0000","last_became_active":"2024-04-17T10:48:25.411971+0000","last_became_peered":"2024-04-17T10:48:25.411971+0000","last_unstale":"2024-04-17T10:48:26.768301+0000","last_undegraded":"2024-04-17T10:48:26.768301+0000","last_fullsized":"2024-04-17T10:48:26.768301+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:18:53.100724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.371449+0000","last_change":"2024-04-17T10:48:25.397178+0000","last_active":"2024-04-17T10:48:26.371449+0000","last_peered":"2024-04-17T10:48:26.371449+0000","last_clean":"2024-04-17T10:48:26.371449+0000","last_became_active":"2024-04-17T10:48:25.396460+0000","last_became_peered":"2024-04-17T10:48:25.396460+0000","last_unstale":"2024-04-17T10:48:26.371449+0000","last_undegraded":"2024-04-17T10:48:26.371449+0000","last_fullsized":"2024-04-17T10:48:26.371449+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:27:05.556140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.766226+0000","last_change":"2024-04-17T10:48:25.416373+0000","last_active":"2024-04-17T10:48:26.766226+0000","last_peered":"2024-04-17T10:48:26.766226+0000","last_clean":"2024-04-17T10:48:26.766226+0000","last_became_active":"2024-04-17T10:48:25.415518+0000","last_became_peered":"2024-04-17T10:48:25.415518+0000","last_unstale":"2024-04-17T10:48:26.766226+0000","last_undegraded":"2024-04-17T10:48:26.766226+0000","last_fullsized":"2024-04-17T10:48:26.766226+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:29:30.392165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.765977+0000","last_change":"2024-04-17T10:48:25.400297+0000","last_active":"2024-04-17T10:48:26.765977+0000","last_peered":"2024-04-17T10:48:26.765977+0000","last_clean":"2024-04-17T10:48:26.765977+0000","last_became_active":"2024-04-17T10:48:25.399408+0000","last_became_peered":"2024-04-17T10:48:25.399408+0000","last_unstale":"2024-04-17T10:48:26.765977+0000","last_undegraded":"2024-04-17T10:48:26.765977+0000","last_fullsized":"2024-04-17T10:48:26.765977+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:20:19.582524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.372451+0000","last_change":"2024-04-17T10:48:25.410349+0000","last_active":"2024-04-17T10:48:26.372451+0000","last_peered":"2024-04-17T10:48:26.372451+0000","last_clean":"2024-04-17T10:48:26.372451+0000","last_became_active":"2024-04-17T10:48:25.409364+0000","last_became_peered":"2024-04-17T10:48:25.409364+0000","last_unstale":"2024-04-17T10:48:26.372451+0000","last_undegraded":"2024-04-17T10:48:26.372451+0000","last_fullsized":"2024-04-17T10:48:26.372451+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:12:10.319753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.770038+0000","last_change":"2024-04-17T10:48:25.412684+0000","last_active":"2024-04-17T10:48:26.770038+0000","last_peered":"2024-04-17T10:48:26.770038+0000","last_clean":"2024-04-17T10:48:26.770038+0000","last_became_active":"2024-04-17T10:48:25.411988+0000","last_became_peered":"2024-04-17T10:48:25.411988+0000","last_unstale":"2024-04-17T10:48:26.770038+0000","last_undegraded":"2024-04-17T10:48:26.770038+0000","last_fullsized":"2024-04-17T10:48:26.770038+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:17:56.010145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.769319+0000","last_change":"2024-04-17T10:48:25.392775+0000","last_active":"2024-04-17T10:48:26.769319+0000","last_peered":"2024-04-17T10:48:26.769319+0000","last_clean":"2024-04-17T10:48:26.769319+0000","last_became_active":"2024-04-17T10:48:25.391662+0000","last_became_peered":"2024-04-17T10:48:25.391662+0000","last_unstale":"2024-04-17T10:48:26.769319+0000","last_undegraded":"2024-04-17T10:48:26.769319+0000","last_fullsized":"2024-04-17T10:48:26.769319+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:35:40.155349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.373941+0000","last_change":"2024-04-17T10:48:25.388692+0000","last_active":"2024-04-17T10:48:26.373941+0000","last_peered":"2024-04-17T10:48:26.373941+0000","last_clean":"2024-04-17T10:48:26.373941+0000","last_became_active":"2024-04-17T10:48:25.387978+0000","last_became_peered":"2024-04-17T10:48:25.387978+0000","last_unstale":"2024-04-17T10:48:26.373941+0000","last_undegraded":"2024-04-17T10:48:26.373941+0000","last_fullsized":"2024-04-17T10:48:26.373941+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:05:11.412575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.768363+0000","last_change":"2024-04-17T10:48:25.405691+0000","last_active":"2024-04-17T10:48:26.768363+0000","last_peered":"2024-04-17T10:48:26.768363+0000","last_clean":"2024-04-17T10:48:26.768363+0000","last_became_active":"2024-04-17T10:48:25.400439+0000","last_became_peered":"2024-04-17T10:48:25.400439+0000","last_unstale":"2024-04-17T10:48:26.768363+0000","last_undegraded":"2024-04-17T10:48:26.768363+0000","last_fullsized":"2024-04-17T10:48:26.768363+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:34:56.501236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.373878+0000","last_change":"2024-04-17T10:48:25.388415+0000","last_active":"2024-04-17T10:48:26.373878+0000","last_peered":"2024-04-17T10:48:26.373878+0000","last_clean":"2024-04-17T10:48:26.373878+0000","last_became_active":"2024-04-17T10:48:25.385696+0000","last_became_peered":"2024-04-17T10:48:25.385696+0000","last_unstale":"2024-04-17T10:48:26.373878+0000","last_undegraded":"2024-04-17T10:48:26.373878+0000","last_fullsized":"2024-04-17T10:48:26.373878+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:33:20.676205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.372393+0000","last_change":"2024-04-17T10:48:25.395611+0000","last_active":"2024-04-17T10:48:26.372393+0000","last_peered":"2024-04-17T10:48:26.372393+0000","last_clean":"2024-04-17T10:48:26.372393+0000","last_became_active":"2024-04-17T10:48:25.394991+0000","last_became_peered":"2024-04-17T10:48:25.394991+0000","last_unstale":"2024-04-17T10:48:26.372393+0000","last_undegraded":"2024-04-17T10:48:26.372393+0000","last_fullsized":"2024-04-17T10:48:26.372393+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:11:44.983590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.767282+0000","last_change":"2024-04-17T10:48:25.407868+0000","last_active":"2024-04-17T10:48:26.767282+0000","last_peered":"2024-04-17T10:48:26.767282+0000","last_clean":"2024-04-17T10:48:26.767282+0000","last_became_active":"2024-04-17T10:48:25.407489+0000","last_became_peered":"2024-04-17T10:48:25.407489+0000","last_unstale":"2024-04-17T10:48:26.767282+0000","last_undegraded":"2024-04-17T10:48:26.767282+0000","last_fullsized":"2024-04-17T10:48:26.767282+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:16:26.938565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.765637+0000","last_change":"2024-04-17T10:48:25.404544+0000","last_active":"2024-04-17T10:48:26.765637+0000","last_peered":"2024-04-17T10:48:26.765637+0000","last_clean":"2024-04-17T10:48:26.765637+0000","last_became_active":"2024-04-17T10:48:25.401709+0000","last_became_peered":"2024-04-17T10:48:25.401709+0000","last_unstale":"2024-04-17T10:48:26.765637+0000","last_undegraded":"2024-04-17T10:48:26.765637+0000","last_fullsized":"2024-04-17T10:48:26.765637+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:15:32.248494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.374808+0000","last_change":"2024-04-17T10:48:25.396532+0000","last_active":"2024-04-17T10:48:26.374808+0000","last_peered":"2024-04-17T10:48:26.374808+0000","last_clean":"2024-04-17T10:48:26.374808+0000","last_became_active":"2024-04-17T10:48:25.395815+0000","last_became_peered":"2024-04-17T10:48:25.395815+0000","last_unstale":"2024-04-17T10:48:26.374808+0000","last_undegraded":"2024-04-17T10:48:26.374808+0000","last_fullsized":"2024-04-17T10:48:26.374808+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:30:40.212726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.766110+0000","last_change":"2024-04-17T10:48:25.406085+0000","last_active":"2024-04-17T10:48:26.766110+0000","last_peered":"2024-04-17T10:48:26.766110+0000","last_clean":"2024-04-17T10:48:26.766110+0000","last_became_active":"2024-04-17T10:48:25.405759+0000","last_became_peered":"2024-04-17T10:48:25.405759+0000","last_unstale":"2024-04-17T10:48:26.766110+0000","last_undegraded":"2024-04-17T10:48:26.766110+0000","last_fullsized":"2024-04-17T10:48:26.766110+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:36:29.862583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.372966+0000","last_change":"2024-04-17T10:48:25.391324+0000","last_active":"2024-04-17T10:48:26.372966+0000","last_peered":"2024-04-17T10:48:26.372966+0000","last_clean":"2024-04-17T10:48:26.372966+0000","last_became_active":"2024-04-17T10:48:25.389979+0000","last_became_peered":"2024-04-17T10:48:25.389979+0000","last_unstale":"2024-04-17T10:48:26.372966+0000","last_undegraded":"2024-04-17T10:48:26.372966+0000","last_fullsized":"2024-04-17T10:48:26.372966+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:51:19.242590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.374292+0000","last_change":"2024-04-17T10:48:25.379428+0000","last_active":"2024-04-17T10:48:26.374292+0000","last_peered":"2024-04-17T10:48:26.374292+0000","last_clean":"2024-04-17T10:48:26.374292+0000","last_became_active":"2024-04-17T10:48:25.378889+0000","last_became_peered":"2024-04-17T10:48:25.378889+0000","last_unstale":"2024-04-17T10:48:26.374292+0000","last_undegraded":"2024-04-17T10:48:26.374292+0000","last_fullsized":"2024-04-17T10:48:26.374292+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:48:34.698270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.373354+0000","last_change":"2024-04-17T10:48:25.398870+0000","last_active":"2024-04-17T10:48:26.373354+0000","last_peered":"2024-04-17T10:48:26.373354+0000","last_clean":"2024-04-17T10:48:26.373354+0000","last_became_active":"2024-04-17T10:48:25.396704+0000","last_became_peered":"2024-04-17T10:48:25.396704+0000","last_unstale":"2024-04-17T10:48:26.373354+0000","last_undegraded":"2024-04-17T10:48:26.373354+0000","last_fullsized":"2024-04-17T10:48:26.373354+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:23:08.117471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.770118+0000","last_change":"2024-04-17T10:48:25.392701+0000","last_active":"2024-04-17T10:48:26.770118+0000","last_peered":"2024-04-17T10:48:26.770118+0000","last_clean":"2024-04-17T10:48:26.770118+0000","last_became_active":"2024-04-17T10:48:25.391909+0000","last_became_peered":"2024-04-17T10:48:25.391909+0000","last_unstale":"2024-04-17T10:48:26.770118+0000","last_undegraded":"2024-04-17T10:48:26.770118+0000","last_fullsized":"2024-04-17T10:48:26.770118+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:54:52.255925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.371678+0000","last_change":"2024-04-17T10:48:25.379971+0000","last_active":"2024-04-17T10:48:26.371678+0000","last_peered":"2024-04-17T10:48:26.371678+0000","last_clean":"2024-04-17T10:48:26.371678+0000","last_became_active":"2024-04-17T10:48:25.379247+0000","last_became_peered":"2024-04-17T10:48:25.379247+0000","last_unstale":"2024-04-17T10:48:26.371678+0000","last_undegraded":"2024-04-17T10:48:26.371678+0000","last_fullsized":"2024-04-17T10:48:26.371678+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:06:36.947035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.769039+0000","last_change":"2024-04-17T10:48:25.405401+0000","last_active":"2024-04-17T10:48:26.769039+0000","last_peered":"2024-04-17T10:48:26.769039+0000","last_clean":"2024-04-17T10:48:26.769039+0000","last_became_active":"2024-04-17T10:48:25.404401+0000","last_became_peered":"2024-04-17T10:48:25.404401+0000","last_unstale":"2024-04-17T10:48:26.769039+0000","last_undegraded":"2024-04-17T10:48:26.769039+0000","last_fullsized":"2024-04-17T10:48:26.769039+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:39:31.303515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.374824+0000","last_change":"2024-04-17T10:48:25.397267+0000","last_active":"2024-04-17T10:48:26.374824+0000","last_peered":"2024-04-17T10:48:26.374824+0000","last_clean":"2024-04-17T10:48:26.374824+0000","last_became_active":"2024-04-17T10:48:25.395674+0000","last_became_peered":"2024-04-17T10:48:25.395674+0000","last_unstale":"2024-04-17T10:48:26.374824+0000","last_undegraded":"2024-04-17T10:48:26.374824+0000","last_fullsized":"2024-04-17T10:48:26.374824+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:06:54.102396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.769470+0000","last_change":"2024-04-17T10:48:25.411334+0000","last_active":"2024-04-17T10:48:26.769470+0000","last_peered":"2024-04-17T10:48:26.769470+0000","last_clean":"2024-04-17T10:48:26.769470+0000","last_became_active":"2024-04-17T10:48:25.410982+0000","last_became_peered":"2024-04-17T10:48:25.410982+0000","last_unstale":"2024-04-17T10:48:26.769470+0000","last_undegraded":"2024-04-17T10:48:26.769470+0000","last_fullsized":"2024-04-17T10:48:26.769470+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:35:06.841596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.766862+0000","last_change":"2024-04-17T10:48:25.416423+0000","last_active":"2024-04-17T10:48:26.766862+0000","last_peered":"2024-04-17T10:48:26.766862+0000","last_clean":"2024-04-17T10:48:26.766862+0000","last_became_active":"2024-04-17T10:48:25.415437+0000","last_became_peered":"2024-04-17T10:48:25.415437+0000","last_unstale":"2024-04-17T10:48:26.766862+0000","last_undegraded":"2024-04-17T10:48:26.766862+0000","last_fullsized":"2024-04-17T10:48:26.766862+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:17:41.530053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.771029+0000","last_change":"2024-04-17T10:48:25.412327+0000","last_active":"2024-04-17T10:48:26.771029+0000","last_peered":"2024-04-17T10:48:26.771029+0000","last_clean":"2024-04-17T10:48:26.771029+0000","last_became_active":"2024-04-17T10:48:25.412023+0000","last_became_peered":"2024-04-17T10:48:25.412023+0000","last_unstale":"2024-04-17T10:48:26.771029+0000","last_undegraded":"2024-04-17T10:48:26.771029+0000","last_fullsized":"2024-04-17T10:48:26.771029+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:39:54.936657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.371859+0000","last_change":"2024-04-17T10:48:25.398115+0000","last_active":"2024-04-17T10:48:26.371859+0000","last_peered":"2024-04-17T10:48:26.371859+0000","last_clean":"2024-04-17T10:48:26.371859+0000","last_became_active":"2024-04-17T10:48:25.395510+0000","last_became_peered":"2024-04-17T10:48:25.395510+0000","last_unstale":"2024-04-17T10:48:26.371859+0000","last_undegraded":"2024-04-17T10:48:26.371859+0000","last_fullsized":"2024-04-17T10:48:26.371859+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:01:54.678307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.766121+0000","last_change":"2024-04-17T10:48:25.407648+0000","last_active":"2024-04-17T10:48:26.766121+0000","last_peered":"2024-04-17T10:48:26.766121+0000","last_clean":"2024-04-17T10:48:26.766121+0000","last_became_active":"2024-04-17T10:48:25.407312+0000","last_became_peered":"2024-04-17T10:48:25.407312+0000","last_unstale":"2024-04-17T10:48:26.766121+0000","last_undegraded":"2024-04-17T10:48:26.766121+0000","last_fullsized":"2024-04-17T10:48:26.766121+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:25:04.560732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.766791+0000","last_change":"2024-04-17T10:48:25.404873+0000","last_active":"2024-04-17T10:48:26.766791+0000","last_peered":"2024-04-17T10:48:26.766791+0000","last_clean":"2024-04-17T10:48:26.766791+0000","last_became_active":"2024-04-17T10:48:25.401939+0000","last_became_peered":"2024-04-17T10:48:25.401939+0000","last_unstale":"2024-04-17T10:48:26.766791+0000","last_undegraded":"2024-04-17T10:48:26.766791+0000","last_fullsized":"2024-04-17T10:48:26.766791+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:10:50.140781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.766533+0000","last_change":"2024-04-17T10:48:25.416396+0000","last_active":"2024-04-17T10:48:26.766533+0000","last_peered":"2024-04-17T10:48:26.766533+0000","last_clean":"2024-04-17T10:48:26.766533+0000","last_became_active":"2024-04-17T10:48:25.410188+0000","last_became_peered":"2024-04-17T10:48:25.410188+0000","last_unstale":"2024-04-17T10:48:26.766533+0000","last_undegraded":"2024-04-17T10:48:26.766533+0000","last_fullsized":"2024-04-17T10:48:26.766533+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:45:45.553626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.767012+0000","last_change":"2024-04-17T10:48:25.409763+0000","last_active":"2024-04-17T10:48:26.767012+0000","last_peered":"2024-04-17T10:48:26.767012+0000","last_clean":"2024-04-17T10:48:26.767012+0000","last_became_active":"2024-04-17T10:48:25.409417+0000","last_became_peered":"2024-04-17T10:48:25.409417+0000","last_unstale":"2024-04-17T10:48:26.767012+0000","last_undegraded":"2024-04-17T10:48:26.767012+0000","last_fullsized":"2024-04-17T10:48:26.767012+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:27:46.941314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"113'4","reported_seq":18,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.766816+0000","last_change":"2024-04-17T10:48:24.396879+0000","last_active":"2024-04-17T10:48:26.766816+0000","last_peered":"2024-04-17T10:48:26.766816+0000","last_clean":"2024-04-17T10:48:26.766816+0000","last_became_active":"2024-04-17T10:48:24.396126+0000","last_became_peered":"2024-04-17T10:48:24.396126+0000","last_unstale":"2024-04-17T10:48:26.766816+0000","last_undegraded":"2024-04-17T10:48:26.766816+0000","last_fullsized":"2024-04-17T10:48:26.766816+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:27:13.904253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.767542+0000","last_change":"2024-04-17T10:48:25.411819+0000","last_active":"2024-04-17T10:48:26.767542+0000","last_peered":"2024-04-17T10:48:26.767542+0000","last_clean":"2024-04-17T10:48:26.767542+0000","last_became_active":"2024-04-17T10:48:25.405741+0000","last_became_peered":"2024-04-17T10:48:25.405741+0000","last_unstale":"2024-04-17T10:48:26.767542+0000","last_undegraded":"2024-04-17T10:48:26.767542+0000","last_fullsized":"2024-04-17T10:48:26.767542+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:01:54.678307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.372656+0000","last_change":"2024-04-17T10:48:24.398459+0000","last_active":"2024-04-17T10:48:26.372656+0000","last_peered":"2024-04-17T10:48:26.372656+0000","last_clean":"2024-04-17T10:48:26.372656+0000","last_became_active":"2024-04-17T10:48:24.398098+0000","last_became_peered":"2024-04-17T10:48:24.398098+0000","last_unstale":"2024-04-17T10:48:26.372656+0000","last_undegraded":"2024-04-17T10:48:26.372656+0000","last_fullsized":"2024-04-17T10:48:26.372656+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:05:10.402436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.766527+0000","last_change":"2024-04-17T10:48:25.415998+0000","last_active":"2024-04-17T10:48:26.766527+0000","last_peered":"2024-04-17T10:48:26.766527+0000","last_clean":"2024-04-17T10:48:26.766527+0000","last_became_active":"2024-04-17T10:48:25.414966+0000","last_became_peered":"2024-04-17T10:48:25.414966+0000","last_unstale":"2024-04-17T10:48:26.766527+0000","last_undegraded":"2024-04-17T10:48:26.766527+0000","last_fullsized":"2024-04-17T10:48:26.766527+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:12:43.861809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.371023+0000","last_change":"2024-04-17T10:48:24.400890+0000","last_active":"2024-04-17T10:48:26.371023+0000","last_peered":"2024-04-17T10:48:26.371023+0000","last_clean":"2024-04-17T10:48:26.371023+0000","last_became_active":"2024-04-17T10:48:24.400472+0000","last_became_peered":"2024-04-17T10:48:24.400472+0000","last_unstale":"2024-04-17T10:48:26.371023+0000","last_undegraded":"2024-04-17T10:48:26.371023+0000","last_fullsized":"2024-04-17T10:48:26.371023+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:10:35.468320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.373814+0000","last_change":"2024-04-17T10:48:25.393014+0000","last_active":"2024-04-17T10:48:26.373814+0000","last_peered":"2024-04-17T10:48:26.373814+0000","last_clean":"2024-04-17T10:48:26.373814+0000","last_became_active":"2024-04-17T10:48:25.392190+0000","last_became_peered":"2024-04-17T10:48:25.392190+0000","last_unstale":"2024-04-17T10:48:26.373814+0000","last_undegraded":"2024-04-17T10:48:26.373814+0000","last_fullsized":"2024-04-17T10:48:26.373814+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:15:32.248494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.371249+0000","last_change":"2024-04-17T10:48:24.401643+0000","last_active":"2024-04-17T10:48:26.371249+0000","last_peered":"2024-04-17T10:48:26.371249+0000","last_clean":"2024-04-17T10:48:26.371249+0000","last_became_active":"2024-04-17T10:48:24.400656+0000","last_became_peered":"2024-04-17T10:48:24.400656+0000","last_unstale":"2024-04-17T10:48:26.371249+0000","last_undegraded":"2024-04-17T10:48:26.371249+0000","last_fullsized":"2024-04-17T10:48:26.371249+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:57:15.491907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.767198+0000","last_change":"2024-04-17T10:48:25.393375+0000","last_active":"2024-04-17T10:48:26.767198+0000","last_peered":"2024-04-17T10:48:26.767198+0000","last_clean":"2024-04-17T10:48:26.767198+0000","last_became_active":"2024-04-17T10:48:25.392451+0000","last_became_peered":"2024-04-17T10:48:25.392451+0000","last_unstale":"2024-04-17T10:48:26.767198+0000","last_undegraded":"2024-04-17T10:48:26.767198+0000","last_fullsized":"2024-04-17T10:48:26.767198+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:21:26.180858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.765928+0000","last_change":"2024-04-17T10:48:24.403866+0000","last_active":"2024-04-17T10:48:26.765928+0000","last_peered":"2024-04-17T10:48:26.765928+0000","last_clean":"2024-04-17T10:48:26.765928+0000","last_became_active":"2024-04-17T10:48:24.403129+0000","last_became_peered":"2024-04-17T10:48:24.403129+0000","last_unstale":"2024-04-17T10:48:26.765928+0000","last_undegraded":"2024-04-17T10:48:26.765928+0000","last_fullsized":"2024-04-17T10:48:26.765928+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:46:16.221800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.372228+0000","last_change":"2024-04-17T10:48:25.409337+0000","last_active":"2024-04-17T10:48:26.372228+0000","last_peered":"2024-04-17T10:48:26.372228+0000","last_clean":"2024-04-17T10:48:26.372228+0000","last_became_active":"2024-04-17T10:48:25.407690+0000","last_became_peered":"2024-04-17T10:48:25.407690+0000","last_unstale":"2024-04-17T10:48:26.372228+0000","last_undegraded":"2024-04-17T10:48:26.372228+0000","last_fullsized":"2024-04-17T10:48:26.372228+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:54:48.227100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.770353+0000","last_change":"2024-04-17T10:48:24.399589+0000","last_active":"2024-04-17T10:48:26.770353+0000","last_peered":"2024-04-17T10:48:26.770353+0000","last_clean":"2024-04-17T10:48:26.770353+0000","last_became_active":"2024-04-17T10:48:24.399097+0000","last_became_peered":"2024-04-17T10:48:24.399097+0000","last_unstale":"2024-04-17T10:48:26.770353+0000","last_undegraded":"2024-04-17T10:48:26.770353+0000","last_fullsized":"2024-04-17T10:48:26.770353+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:08:58.956217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.765688+0000","last_change":"2024-04-17T10:48:25.413668+0000","last_active":"2024-04-17T10:48:26.765688+0000","last_peered":"2024-04-17T10:48:26.765688+0000","last_clean":"2024-04-17T10:48:26.765688+0000","last_became_active":"2024-04-17T10:48:25.413239+0000","last_became_peered":"2024-04-17T10:48:25.413239+0000","last_unstale":"2024-04-17T10:48:26.765688+0000","last_undegraded":"2024-04-17T10:48:26.765688+0000","last_fullsized":"2024-04-17T10:48:26.765688+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:33:20.676205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.371654+0000","last_change":"2024-04-17T10:48:24.376241+0000","last_active":"2024-04-17T10:48:26.371654+0000","last_peered":"2024-04-17T10:48:26.371654+0000","last_clean":"2024-04-17T10:48:26.371654+0000","last_became_active":"2024-04-17T10:48:24.375430+0000","last_became_peered":"2024-04-17T10:48:24.375430+0000","last_unstale":"2024-04-17T10:48:26.371654+0000","last_undegraded":"2024-04-17T10:48:26.371654+0000","last_fullsized":"2024-04-17T10:48:26.371654+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:10:37.882986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.765556+0000","last_change":"2024-04-17T10:48:25.400558+0000","last_active":"2024-04-17T10:48:26.765556+0000","last_peered":"2024-04-17T10:48:26.765556+0000","last_clean":"2024-04-17T10:48:26.765556+0000","last_became_active":"2024-04-17T10:48:25.399941+0000","last_became_peered":"2024-04-17T10:48:25.399941+0000","last_unstale":"2024-04-17T10:48:26.765556+0000","last_undegraded":"2024-04-17T10:48:26.765556+0000","last_fullsized":"2024-04-17T10:48:26.765556+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:03:45.310360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.766069+0000","last_change":"2024-04-17T10:48:24.382363+0000","last_active":"2024-04-17T10:48:26.766069+0000","last_peered":"2024-04-17T10:48:26.766069+0000","last_clean":"2024-04-17T10:48:26.766069+0000","last_became_active":"2024-04-17T10:48:24.381493+0000","last_became_peered":"2024-04-17T10:48:24.381493+0000","last_unstale":"2024-04-17T10:48:26.766069+0000","last_undegraded":"2024-04-17T10:48:26.766069+0000","last_fullsized":"2024-04-17T10:48:26.766069+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:48:33.688131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.766924+0000","last_change":"2024-04-17T10:48:25.388751+0000","last_active":"2024-04-17T10:48:26.766924+0000","last_peered":"2024-04-17T10:48:26.766924+0000","last_clean":"2024-04-17T10:48:26.766924+0000","last_became_active":"2024-04-17T10:48:25.387565+0000","last_became_peered":"2024-04-17T10:48:25.387565+0000","last_unstale":"2024-04-17T10:48:26.766924+0000","last_undegraded":"2024-04-17T10:48:26.766924+0000","last_fullsized":"2024-04-17T10:48:26.766924+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:44:51.344574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.371156+0000","last_change":"2024-04-17T10:48:24.400839+0000","last_active":"2024-04-17T10:48:26.371156+0000","last_peered":"2024-04-17T10:48:26.371156+0000","last_clean":"2024-04-17T10:48:26.371156+0000","last_became_active":"2024-04-17T10:48:24.400489+0000","last_became_peered":"2024-04-17T10:48:24.400489+0000","last_unstale":"2024-04-17T10:48:26.371156+0000","last_undegraded":"2024-04-17T10:48:26.371156+0000","last_fullsized":"2024-04-17T10:48:26.371156+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:12:53.533150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.767614+0000","last_change":"2024-04-17T10:48:25.414228+0000","last_active":"2024-04-17T10:48:26.767614+0000","last_peered":"2024-04-17T10:48:26.767614+0000","last_clean":"2024-04-17T10:48:26.767614+0000","last_became_active":"2024-04-17T10:48:25.413814+0000","last_became_peered":"2024-04-17T10:48:25.413814+0000","last_unstale":"2024-04-17T10:48:26.767614+0000","last_undegraded":"2024-04-17T10:48:26.767614+0000","last_fullsized":"2024-04-17T10:48:26.767614+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:23:08.117471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.770404+0000","last_change":"2024-04-17T10:48:24.399209+0000","last_active":"2024-04-17T10:48:26.770404+0000","last_peered":"2024-04-17T10:48:26.770404+0000","last_clean":"2024-04-17T10:48:26.770404+0000","last_became_active":"2024-04-17T10:48:24.398789+0000","last_became_peered":"2024-04-17T10:48:24.398789+0000","last_unstale":"2024-04-17T10:48:26.770404+0000","last_undegraded":"2024-04-17T10:48:26.770404+0000","last_fullsized":"2024-04-17T10:48:26.770404+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:23:07.107332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.771219+0000","last_change":"2024-04-17T10:48:25.397844+0000","last_active":"2024-04-17T10:48:26.771219+0000","last_peered":"2024-04-17T10:48:26.771219+0000","last_clean":"2024-04-17T10:48:26.771219+0000","last_became_active":"2024-04-17T10:48:25.396170+0000","last_became_peered":"2024-04-17T10:48:25.396170+0000","last_unstale":"2024-04-17T10:48:26.771219+0000","last_undegraded":"2024-04-17T10:48:26.771219+0000","last_fullsized":"2024-04-17T10:48:26.771219+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:22:51.921074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"113'3","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.769154+0000","last_change":"2024-04-17T10:48:24.402346+0000","last_active":"2024-04-17T10:48:26.769154+0000","last_peered":"2024-04-17T10:48:26.769154+0000","last_clean":"2024-04-17T10:48:26.769154+0000","last_became_active":"2024-04-17T10:48:24.401900+0000","last_became_peered":"2024-04-17T10:48:24.401900+0000","last_unstale":"2024-04-17T10:48:26.769154+0000","last_undegraded":"2024-04-17T10:48:26.769154+0000","last_fullsized":"2024-04-17T10:48:26.769154+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:06:53.092256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.767136+0000","last_change":"2024-04-17T10:48:25.398079+0000","last_active":"2024-04-17T10:48:26.767136+0000","last_peered":"2024-04-17T10:48:26.767136+0000","last_clean":"2024-04-17T10:48:26.767136+0000","last_became_active":"2024-04-17T10:48:25.397112+0000","last_became_peered":"2024-04-17T10:48:25.397112+0000","last_unstale":"2024-04-17T10:48:26.767136+0000","last_undegraded":"2024-04-17T10:48:26.767136+0000","last_fullsized":"2024-04-17T10:48:26.767136+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:44:39.086403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"113'6","reported_seq":20,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.768610+0000","last_change":"2024-04-17T10:48:24.379794+0000","last_active":"2024-04-17T10:48:26.768610+0000","last_peered":"2024-04-17T10:48:26.768610+0000","last_clean":"2024-04-17T10:48:26.768610+0000","last_became_active":"2024-04-17T10:48:24.379413+0000","last_became_peered":"2024-04-17T10:48:24.379413+0000","last_unstale":"2024-04-17T10:48:26.768610+0000","last_undegraded":"2024-04-17T10:48:26.768610+0000","last_fullsized":"2024-04-17T10:48:26.768610+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:26:51.328958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.770263+0000","last_change":"2024-04-17T10:48:25.399222+0000","last_active":"2024-04-17T10:48:26.770263+0000","last_peered":"2024-04-17T10:48:26.770263+0000","last_clean":"2024-04-17T10:48:26.770263+0000","last_became_active":"2024-04-17T10:48:25.398549+0000","last_became_peered":"2024-04-17T10:48:25.398549+0000","last_unstale":"2024-04-17T10:48:26.770263+0000","last_undegraded":"2024-04-17T10:48:26.770263+0000","last_fullsized":"2024-04-17T10:48:26.770263+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:43:40.208184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.767364+0000","last_change":"2024-04-17T10:48:24.400108+0000","last_active":"2024-04-17T10:48:26.767364+0000","last_peered":"2024-04-17T10:48:26.767364+0000","last_clean":"2024-04-17T10:48:26.767364+0000","last_became_active":"2024-04-17T10:48:24.397908+0000","last_became_peered":"2024-04-17T10:48:24.397908+0000","last_unstale":"2024-04-17T10:48:26.767364+0000","last_undegraded":"2024-04-17T10:48:26.767364+0000","last_fullsized":"2024-04-17T10:48:26.767364+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:08:54.500148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.770720+0000","last_change":"2024-04-17T10:48:25.405089+0000","last_active":"2024-04-17T10:48:26.770720+0000","last_peered":"2024-04-17T10:48:26.770720+0000","last_clean":"2024-04-17T10:48:26.770720+0000","last_became_active":"2024-04-17T10:48:25.394073+0000","last_became_peered":"2024-04-17T10:48:25.394073+0000","last_unstale":"2024-04-17T10:48:26.770720+0000","last_undegraded":"2024-04-17T10:48:26.770720+0000","last_fullsized":"2024-04-17T10:48:26.770720+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:51:19.242590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.766018+0000","last_change":"2024-04-17T10:48:24.391544+0000","last_active":"2024-04-17T10:48:26.766018+0000","last_peered":"2024-04-17T10:48:26.766018+0000","last_clean":"2024-04-17T10:48:26.766018+0000","last_became_active":"2024-04-17T10:48:24.388072+0000","last_became_peered":"2024-04-17T10:48:24.388072+0000","last_unstale":"2024-04-17T10:48:26.766018+0000","last_undegraded":"2024-04-17T10:48:26.766018+0000","last_fullsized":"2024-04-17T10:48:26.766018+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:44:50.334435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.769958+0000","last_change":"2024-04-17T10:48:25.396154+0000","last_active":"2024-04-17T10:48:26.769958+0000","last_peered":"2024-04-17T10:48:26.769958+0000","last_clean":"2024-04-17T10:48:26.769958+0000","last_became_active":"2024-04-17T10:48:25.395480+0000","last_became_peered":"2024-04-17T10:48:25.395480+0000","last_unstale":"2024-04-17T10:48:26.769958+0000","last_undegraded":"2024-04-17T10:48:26.769958+0000","last_fullsized":"2024-04-17T10:48:26.769958+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:12:10.319753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"113'8","reported_seq":22,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.370735+0000","last_change":"2024-04-17T10:48:24.383457+0000","last_active":"2024-04-17T10:48:26.370735+0000","last_peered":"2024-04-17T10:48:26.370735+0000","last_clean":"2024-04-17T10:48:26.370735+0000","last_became_active":"2024-04-17T10:48:24.382932+0000","last_became_peered":"2024-04-17T10:48:24.382932+0000","last_unstale":"2024-04-17T10:48:26.370735+0000","last_undegraded":"2024-04-17T10:48:26.370735+0000","last_fullsized":"2024-04-17T10:48:26.370735+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:08:54.500148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.371568+0000","last_change":"2024-04-17T10:48:25.390823+0000","last_active":"2024-04-17T10:48:26.371568+0000","last_peered":"2024-04-17T10:48:26.371568+0000","last_clean":"2024-04-17T10:48:26.371568+0000","last_became_active":"2024-04-17T10:48:25.390046+0000","last_became_peered":"2024-04-17T10:48:25.390046+0000","last_unstale":"2024-04-17T10:48:26.371568+0000","last_undegraded":"2024-04-17T10:48:26.371568+0000","last_fullsized":"2024-04-17T10:48:26.371568+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:57:44.448586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.770479+0000","last_change":"2024-04-17T10:48:24.399014+0000","last_active":"2024-04-17T10:48:26.770479+0000","last_peered":"2024-04-17T10:48:26.770479+0000","last_clean":"2024-04-17T10:48:26.770479+0000","last_became_active":"2024-04-17T10:48:24.398705+0000","last_became_peered":"2024-04-17T10:48:24.398705+0000","last_unstale":"2024-04-17T10:48:26.770479+0000","last_undegraded":"2024-04-17T10:48:26.770479+0000","last_fullsized":"2024-04-17T10:48:26.770479+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:17:15.758181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.767297+0000","last_change":"2024-04-17T10:48:25.417619+0000","last_active":"2024-04-17T10:48:26.767297+0000","last_peered":"2024-04-17T10:48:26.767297+0000","last_clean":"2024-04-17T10:48:26.767297+0000","last_became_active":"2024-04-17T10:48:25.415994+0000","last_became_peered":"2024-04-17T10:48:25.415994+0000","last_unstale":"2024-04-17T10:48:26.767297+0000","last_undegraded":"2024-04-17T10:48:26.767297+0000","last_fullsized":"2024-04-17T10:48:26.767297+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:24:51.714582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.371665+0000","last_change":"2024-04-17T10:48:24.387073+0000","last_active":"2024-04-17T10:48:26.371665+0000","last_peered":"2024-04-17T10:48:26.371665+0000","last_clean":"2024-04-17T10:48:26.371665+0000","last_became_active":"2024-04-17T10:48:24.386344+0000","last_became_peered":"2024-04-17T10:48:24.386344+0000","last_unstale":"2024-04-17T10:48:26.371665+0000","last_undegraded":"2024-04-17T10:48:26.371665+0000","last_fullsized":"2024-04-17T10:48:26.371665+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:08:51.075827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.372744+0000","last_change":"2024-04-17T10:48:25.407161+0000","last_active":"2024-04-17T10:48:26.372744+0000","last_peered":"2024-04-17T10:48:26.372744+0000","last_clean":"2024-04-17T10:48:26.372744+0000","last_became_active":"2024-04-17T10:48:25.406425+0000","last_became_peered":"2024-04-17T10:48:25.406425+0000","last_unstale":"2024-04-17T10:48:26.372744+0000","last_undegraded":"2024-04-17T10:48:26.372744+0000","last_fullsized":"2024-04-17T10:48:26.372744+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:48:50.522575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.766428+0000","last_change":"2024-04-17T10:48:24.382682+0000","last_active":"2024-04-17T10:48:26.766428+0000","last_peered":"2024-04-17T10:48:26.766428+0000","last_clean":"2024-04-17T10:48:26.766428+0000","last_became_active":"2024-04-17T10:48:24.381331+0000","last_became_peered":"2024-04-17T10:48:24.381331+0000","last_unstale":"2024-04-17T10:48:26.766428+0000","last_undegraded":"2024-04-17T10:48:26.766428+0000","last_fullsized":"2024-04-17T10:48:26.766428+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:44:38.076264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.373431+0000","last_change":"2024-04-17T10:48:25.390693+0000","last_active":"2024-04-17T10:48:26.373431+0000","last_peered":"2024-04-17T10:48:26.373431+0000","last_clean":"2024-04-17T10:48:26.373431+0000","last_became_active":"2024-04-17T10:48:25.386621+0000","last_became_peered":"2024-04-17T10:48:25.386621+0000","last_unstale":"2024-04-17T10:48:26.373431+0000","last_undegraded":"2024-04-17T10:48:26.373431+0000","last_fullsized":"2024-04-17T10:48:26.373431+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:17:03.401358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.765596+0000","last_change":"2024-04-17T10:48:24.396200+0000","last_active":"2024-04-17T10:48:26.765596+0000","last_peered":"2024-04-17T10:48:26.765596+0000","last_clean":"2024-04-17T10:48:26.765596+0000","last_became_active":"2024-04-17T10:48:24.395802+0000","last_became_peered":"2024-04-17T10:48:24.395802+0000","last_unstale":"2024-04-17T10:48:26.765596+0000","last_undegraded":"2024-04-17T10:48:26.765596+0000","last_fullsized":"2024-04-17T10:48:26.765596+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:06:15.720512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.372391+0000","last_change":"2024-04-17T10:48:25.396222+0000","last_active":"2024-04-17T10:48:26.372391+0000","last_peered":"2024-04-17T10:48:26.372391+0000","last_clean":"2024-04-17T10:48:26.372391+0000","last_became_active":"2024-04-17T10:48:25.395841+0000","last_became_peered":"2024-04-17T10:48:25.395841+0000","last_unstale":"2024-04-17T10:48:26.372391+0000","last_undegraded":"2024-04-17T10:48:26.372391+0000","last_fullsized":"2024-04-17T10:48:26.372391+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:43:40.208184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.767631+0000","last_change":"2024-04-17T10:48:24.397774+0000","last_active":"2024-04-17T10:48:26.767631+0000","last_peered":"2024-04-17T10:48:26.767631+0000","last_clean":"2024-04-17T10:48:26.767631+0000","last_became_active":"2024-04-17T10:48:24.397453+0000","last_became_peered":"2024-04-17T10:48:24.397453+0000","last_unstale":"2024-04-17T10:48:26.767631+0000","last_undegraded":"2024-04-17T10:48:26.767631+0000","last_fullsized":"2024-04-17T10:48:26.767631+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:28:23.849059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.767472+0000","last_change":"2024-04-17T10:48:25.405798+0000","last_active":"2024-04-17T10:48:26.767472+0000","last_peered":"2024-04-17T10:48:26.767472+0000","last_clean":"2024-04-17T10:48:26.767472+0000","last_became_active":"2024-04-17T10:48:25.402057+0000","last_became_peered":"2024-04-17T10:48:25.402057+0000","last_unstale":"2024-04-17T10:48:26.767472+0000","last_undegraded":"2024-04-17T10:48:26.767472+0000","last_fullsized":"2024-04-17T10:48:26.767472+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:17:14.009930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.370820+0000","last_change":"2024-04-17T10:48:24.400725+0000","last_active":"2024-04-17T10:48:26.370820+0000","last_peered":"2024-04-17T10:48:26.370820+0000","last_clean":"2024-04-17T10:48:26.370820+0000","last_became_active":"2024-04-17T10:48:24.400444+0000","last_became_peered":"2024-04-17T10:48:24.400444+0000","last_unstale":"2024-04-17T10:48:26.370820+0000","last_undegraded":"2024-04-17T10:48:26.370820+0000","last_fullsized":"2024-04-17T10:48:26.370820+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:28:23.849059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.372654+0000","last_change":"2024-04-17T10:48:25.391263+0000","last_active":"2024-04-17T10:48:26.372654+0000","last_peered":"2024-04-17T10:48:26.372654+0000","last_clean":"2024-04-17T10:48:26.372654+0000","last_became_active":"2024-04-17T10:48:25.386320+0000","last_became_peered":"2024-04-17T10:48:25.386320+0000","last_unstale":"2024-04-17T10:48:26.372654+0000","last_undegraded":"2024-04-17T10:48:26.372654+0000","last_fullsized":"2024-04-17T10:48:26.372654+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:54:52.255925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.767513+0000","last_change":"2024-04-17T10:48:24.397713+0000","last_active":"2024-04-17T10:48:26.767513+0000","last_peered":"2024-04-17T10:48:26.767513+0000","last_clean":"2024-04-17T10:48:26.767513+0000","last_became_active":"2024-04-17T10:48:24.397264+0000","last_became_peered":"2024-04-17T10:48:24.397264+0000","last_unstale":"2024-04-17T10:48:26.767513+0000","last_undegraded":"2024-04-17T10:48:26.767513+0000","last_fullsized":"2024-04-17T10:48:26.767513+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:57:15.491907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.770360+0000","last_change":"2024-04-17T10:48:25.393538+0000","last_active":"2024-04-17T10:48:26.770360+0000","last_peered":"2024-04-17T10:48:26.770360+0000","last_clean":"2024-04-17T10:48:26.770360+0000","last_became_active":"2024-04-17T10:48:25.390793+0000","last_became_peered":"2024-04-17T10:48:25.390793+0000","last_unstale":"2024-04-17T10:48:26.770360+0000","last_undegraded":"2024-04-17T10:48:26.770360+0000","last_fullsized":"2024-04-17T10:48:26.770360+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:48:50.522575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.766301+0000","last_change":"2024-04-17T10:48:24.393648+0000","last_active":"2024-04-17T10:48:26.766301+0000","last_peered":"2024-04-17T10:48:26.766301+0000","last_clean":"2024-04-17T10:48:26.766301+0000","last_became_active":"2024-04-17T10:48:24.392862+0000","last_became_peered":"2024-04-17T10:48:24.392862+0000","last_unstale":"2024-04-17T10:48:26.766301+0000","last_undegraded":"2024-04-17T10:48:26.766301+0000","last_fullsized":"2024-04-17T10:48:26.766301+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:08:58.956217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.766536+0000","last_change":"2024-04-17T10:48:25.412654+0000","last_active":"2024-04-17T10:48:26.766536+0000","last_peered":"2024-04-17T10:48:26.766536+0000","last_clean":"2024-04-17T10:48:26.766536+0000","last_became_active":"2024-04-17T10:48:25.411953+0000","last_became_peered":"2024-04-17T10:48:25.411953+0000","last_unstale":"2024-04-17T10:48:26.766536+0000","last_undegraded":"2024-04-17T10:48:26.766536+0000","last_fullsized":"2024-04-17T10:48:26.766536+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:33:32.621034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.771756+0000","last_change":"2024-04-17T10:48:24.389637+0000","last_active":"2024-04-17T10:48:26.771756+0000","last_peered":"2024-04-17T10:48:26.771756+0000","last_clean":"2024-04-17T10:48:26.771756+0000","last_became_active":"2024-04-17T10:48:24.389212+0000","last_became_peered":"2024-04-17T10:48:24.389212+0000","last_unstale":"2024-04-17T10:48:26.771756+0000","last_undegraded":"2024-04-17T10:48:26.771756+0000","last_fullsized":"2024-04-17T10:48:26.771756+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:05:10.402436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.770311+0000","last_change":"2024-04-17T10:48:25.413910+0000","last_active":"2024-04-17T10:48:26.770311+0000","last_peered":"2024-04-17T10:48:26.770311+0000","last_clean":"2024-04-17T10:48:26.770311+0000","last_became_active":"2024-04-17T10:48:25.413211+0000","last_became_peered":"2024-04-17T10:48:25.413211+0000","last_unstale":"2024-04-17T10:48:26.770311+0000","last_undegraded":"2024-04-17T10:48:26.770311+0000","last_fullsized":"2024-04-17T10:48:26.770311+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:53:36.586041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"114'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.767347+0000","last_change":"2024-04-17T10:48:24.398460+0000","last_active":"2024-04-17T10:48:26.767347+0000","last_peered":"2024-04-17T10:48:26.767347+0000","last_clean":"2024-04-17T10:48:26.767347+0000","last_became_active":"2024-04-17T10:48:24.397860+0000","last_became_peered":"2024-04-17T10:48:24.397860+0000","last_unstale":"2024-04-17T10:48:26.767347+0000","last_undegraded":"2024-04-17T10:48:26.767347+0000","last_fullsized":"2024-04-17T10:48:26.767347+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:22:24.746310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.768675+0000","last_change":"2024-04-17T10:48:25.412967+0000","last_active":"2024-04-17T10:48:26.768675+0000","last_peered":"2024-04-17T10:48:26.768675+0000","last_clean":"2024-04-17T10:48:26.768675+0000","last_became_active":"2024-04-17T10:48:25.412620+0000","last_became_peered":"2024-04-17T10:48:25.412620+0000","last_unstale":"2024-04-17T10:48:26.768675+0000","last_undegraded":"2024-04-17T10:48:26.768675+0000","last_fullsized":"2024-04-17T10:48:26.768675+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:51:57.769471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.769441+0000","last_change":"2024-04-17T10:48:24.391043+0000","last_active":"2024-04-17T10:48:26.769441+0000","last_peered":"2024-04-17T10:48:26.769441+0000","last_clean":"2024-04-17T10:48:26.769441+0000","last_became_active":"2024-04-17T10:48:24.390673+0000","last_became_peered":"2024-04-17T10:48:24.390673+0000","last_unstale":"2024-04-17T10:48:26.769441+0000","last_undegraded":"2024-04-17T10:48:26.769441+0000","last_fullsized":"2024-04-17T10:48:26.769441+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:19:31.408326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.766186+0000","last_change":"2024-04-17T10:48:25.405277+0000","last_active":"2024-04-17T10:48:26.766186+0000","last_peered":"2024-04-17T10:48:26.766186+0000","last_clean":"2024-04-17T10:48:26.766186+0000","last_became_active":"2024-04-17T10:48:25.404734+0000","last_became_peered":"2024-04-17T10:48:25.404734+0000","last_unstale":"2024-04-17T10:48:26.766186+0000","last_undegraded":"2024-04-17T10:48:26.766186+0000","last_fullsized":"2024-04-17T10:48:26.766186+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:54:48.227100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.771428+0000","last_change":"2024-04-17T10:48:24.398215+0000","last_active":"2024-04-17T10:48:26.771428+0000","last_peered":"2024-04-17T10:48:26.771428+0000","last_clean":"2024-04-17T10:48:26.771428+0000","last_became_active":"2024-04-17T10:48:24.394779+0000","last_became_peered":"2024-04-17T10:48:24.394779+0000","last_unstale":"2024-04-17T10:48:26.771428+0000","last_undegraded":"2024-04-17T10:48:26.771428+0000","last_fullsized":"2024-04-17T10:48:26.771428+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:48:33.688131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.371860+0000","last_change":"2024-04-17T10:48:25.389974+0000","last_active":"2024-04-17T10:48:26.371860+0000","last_peered":"2024-04-17T10:48:26.371860+0000","last_clean":"2024-04-17T10:48:26.371860+0000","last_became_active":"2024-04-17T10:48:25.389407+0000","last_became_peered":"2024-04-17T10:48:25.389407+0000","last_unstale":"2024-04-17T10:48:26.371860+0000","last_undegraded":"2024-04-17T10:48:26.371860+0000","last_fullsized":"2024-04-17T10:48:26.371860+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:36:29.862583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.766618+0000","last_change":"2024-04-17T10:48:24.396294+0000","last_active":"2024-04-17T10:48:26.766618+0000","last_peered":"2024-04-17T10:48:26.766618+0000","last_clean":"2024-04-17T10:48:26.766618+0000","last_became_active":"2024-04-17T10:48:24.395673+0000","last_became_peered":"2024-04-17T10:48:24.395673+0000","last_unstale":"2024-04-17T10:48:26.766618+0000","last_undegraded":"2024-04-17T10:48:26.766618+0000","last_fullsized":"2024-04-17T10:48:26.766618+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:46:07.697646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.372627+0000","last_change":"2024-04-17T10:48:25.398260+0000","last_active":"2024-04-17T10:48:26.372627+0000","last_peered":"2024-04-17T10:48:26.372627+0000","last_clean":"2024-04-17T10:48:26.372627+0000","last_became_active":"2024-04-17T10:48:25.394791+0000","last_became_peered":"2024-04-17T10:48:25.394791+0000","last_unstale":"2024-04-17T10:48:26.372627+0000","last_undegraded":"2024-04-17T10:48:26.372627+0000","last_fullsized":"2024-04-17T10:48:26.372627+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:33:32.621034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.371870+0000","last_change":"2024-04-17T10:48:24.386727+0000","last_active":"2024-04-17T10:48:26.371870+0000","last_peered":"2024-04-17T10:48:26.371870+0000","last_clean":"2024-04-17T10:48:26.371870+0000","last_became_active":"2024-04-17T10:48:24.386334+0000","last_became_peered":"2024-04-17T10:48:24.386334+0000","last_unstale":"2024-04-17T10:48:26.371870+0000","last_undegraded":"2024-04-17T10:48:26.371870+0000","last_fullsized":"2024-04-17T10:48:26.371870+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:41:37.374175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.372778+0000","last_change":"2024-04-17T10:48:25.400147+0000","last_active":"2024-04-17T10:48:26.372778+0000","last_peered":"2024-04-17T10:48:26.372778+0000","last_clean":"2024-04-17T10:48:26.372778+0000","last_became_active":"2024-04-17T10:48:25.397205+0000","last_became_peered":"2024-04-17T10:48:25.397205+0000","last_unstale":"2024-04-17T10:48:26.372778+0000","last_undegraded":"2024-04-17T10:48:26.372778+0000","last_fullsized":"2024-04-17T10:48:26.372778+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:22:51.921074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.370814+0000","last_change":"2024-04-17T10:48:24.401190+0000","last_active":"2024-04-17T10:48:26.370814+0000","last_peered":"2024-04-17T10:48:26.370814+0000","last_clean":"2024-04-17T10:48:26.370814+0000","last_became_active":"2024-04-17T10:48:24.400909+0000","last_became_peered":"2024-04-17T10:48:24.400909+0000","last_unstale":"2024-04-17T10:48:26.370814+0000","last_undegraded":"2024-04-17T10:48:26.370814+0000","last_fullsized":"2024-04-17T10:48:26.370814+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:42:19.089676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.766988+0000","last_change":"2024-04-17T10:48:25.412830+0000","last_active":"2024-04-17T10:48:26.766988+0000","last_peered":"2024-04-17T10:48:26.766988+0000","last_clean":"2024-04-17T10:48:26.766988+0000","last_became_active":"2024-04-17T10:48:25.412377+0000","last_became_peered":"2024-04-17T10:48:25.412377+0000","last_unstale":"2024-04-17T10:48:26.766988+0000","last_undegraded":"2024-04-17T10:48:26.766988+0000","last_fullsized":"2024-04-17T10:48:26.766988+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:33:44.837620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.766533+0000","last_change":"2024-04-17T10:48:24.393593+0000","last_active":"2024-04-17T10:48:26.766533+0000","last_peered":"2024-04-17T10:48:26.766533+0000","last_clean":"2024-04-17T10:48:26.766533+0000","last_became_active":"2024-04-17T10:48:24.392876+0000","last_became_peered":"2024-04-17T10:48:24.392876+0000","last_unstale":"2024-04-17T10:48:26.766533+0000","last_undegraded":"2024-04-17T10:48:26.766533+0000","last_fullsized":"2024-04-17T10:48:26.766533+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:30:39.202587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.769338+0000","last_change":"2024-04-17T10:48:25.400002+0000","last_active":"2024-04-17T10:48:26.769338+0000","last_peered":"2024-04-17T10:48:26.769338+0000","last_clean":"2024-04-17T10:48:26.769338+0000","last_became_active":"2024-04-17T10:48:25.389947+0000","last_became_peered":"2024-04-17T10:48:25.389947+0000","last_unstale":"2024-04-17T10:48:26.769338+0000","last_undegraded":"2024-04-17T10:48:26.769338+0000","last_fullsized":"2024-04-17T10:48:26.769338+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:24.357930+0000","last_clean_scrub_stamp":"2024-04-17T10:48:24.357930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:11:44.983590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.766225+0000","last_change":"2024-04-17T10:48:24.396850+0000","last_active":"2024-04-17T10:48:26.766225+0000","last_peered":"2024-04-17T10:48:26.766225+0000","last_clean":"2024-04-17T10:48:26.766225+0000","last_became_active":"2024-04-17T10:48:24.396194+0000","last_became_peered":"2024-04-17T10:48:24.396194+0000","last_unstale":"2024-04-17T10:48:26.766225+0000","last_undegraded":"2024-04-17T10:48:26.766225+0000","last_fullsized":"2024-04-17T10:48:26.766225+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:48:23.347791+0000","last_clean_scrub_stamp":"2024-04-17T10:48:23.347791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:37:15.553530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":363,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-17T10:48:26.373397+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:48:26.373397+0000","last_peered":"2024-04-17T10:48:26.373397+0000","last_clean":"2024-04-17T10:48:26.373397+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:48:26.373397+0000","last_undegraded":"2024-04-17T10:48:26.373397+0000","last_fullsized":"2024-04-17T10:48:26.373397+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574978,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59520,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312320,"statfs":{"total":96636764160,"available":96575815680,"internally_reserved":0,"allocated":2019328,"data_stored":1207509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574977,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68664,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303176,"statfs":{"total":96636764160,"available":96566452224,"internally_reserved":0,"allocated":1421312,"data_stored":616595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574978,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54780,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317060,"statfs":{"total":96636764160,"available":96580669440,"internally_reserved":0,"allocated":1425408,"data_stored":617129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574977,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68668,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303172,"statfs":{"total":96636764160,"available":96566448128,"internally_reserved":0,"allocated":1417216,"data_stored":616525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574979,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68676,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303164,"statfs":{"total":96636764160,"available":96566439936,"internally_reserved":0,"allocated":1433600,"data_stored":618207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574977,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58940,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312900,"statfs":{"total":96636764160,"available":96576409600,"internally_reserved":0,"allocated":1417216,"data_stored":616525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574977,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95036,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94276804,"statfs":{"total":96636764160,"available":96539447296,"internally_reserved":0,"allocated":1425408,"data_stored":616649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574977,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63692,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94308148,"statfs":{"total":96636764160,"available":96571543552,"internally_reserved":0,"allocated":2031616,"data_stored":1208617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:48:33.130 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-17T10:48:33.130 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-17T10:48:33.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f102b22e640 1 Processor -- start 2024-04-17T10:48:33.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f102b22e640 1 -- start start 2024-04-17T10:48:33.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f102b22e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f10240b4090 0x7f102406c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:33.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f102b22e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10240af160 con 0x7f1024154670 2024-04-17T10:48:33.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f102b22e640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10240b2ff0 con 0x7f10240b4520 2024-04-17T10:48:33.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f102b22e640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1024154a50 con 0x7f10240b4090 2024-04-17T10:48:33.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f1028fca640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1024154670 0x7f10240aea50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40990/0 (socket says 172.21.15.83:40990) 2024-04-17T10:48:33.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f1028fca640 1 -- 172.21.15.83:0/1628182239 learned_addr learned my addr 172.21.15.83:0/1628182239 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:33.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f1023fff640 1 --2- 172.21.15.83:0/1628182239 >> v2:172.21.15.167:3300/0 conn(0x7f10240b4090 0x7f102406c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:33.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f10237fe640 1 -- 172.21.15.83:0/1628182239 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1971574684 0 0) 0x7f10240af160 con 0x7f1024154670 2024-04-17T10:48:33.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f10237fe640 1 -- 172.21.15.83:0/1628182239 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1014003530 con 0x7f1024154670 2024-04-17T10:48:33.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f10237fe640 1 -- 172.21.15.83:0/1628182239 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3473484656 0 0) 0x7f1014003530 con 0x7f1024154670 2024-04-17T10:48:33.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f10237fe640 1 -- 172.21.15.83:0/1628182239 >> v2:172.21.15.167:3300/0 conn(0x7f10240b4090 msgr2=0x7f102406c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:33.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f10237fe640 1 --2- 172.21.15.83:0/1628182239 >> v2:172.21.15.167:3300/0 conn(0x7f10240b4090 0x7f102406c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:33.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f10237fe640 1 -- 172.21.15.83:0/1628182239 >> v1:172.21.15.167:6789/0 conn(0x7f10240b4520 legacy=0x7f102406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:33.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f10237fe640 1 -- 172.21.15.83:0/1628182239 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10240b32d0 con 0x7f1024154670 2024-04-17T10:48:33.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f10237fe640 1 -- 172.21.15.83:0/1628182239 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f1018001680 con 0x7f1024154670 2024-04-17T10:48:33.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f10237fe640 1 -- 172.21.15.83:0/1628182239 >> v1:172.21.15.83:6789/0 conn(0x7f1024154670 legacy=0x7f10240aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:33.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f10237fe640 1 --2- 172.21.15.83:0/1628182239 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1014003a30 0x7f1014003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:33.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f10237fe640 1 --2- 172.21.15.83:0/1628182239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1014004810 0x7f1014004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:33.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f10237fe640 1 --2- 172.21.15.83:0/1628182239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10140051d0 0x7f101400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:33.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f10237fe640 1 -- 172.21.15.83:0/1628182239 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1024154a50 con 0x7f1014004810 2024-04-17T10:48:33.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f10237fe640 1 -- 172.21.15.83:0/1628182239 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101400c100 con 0x7f10140051d0 2024-04-17T10:48:33.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f10237fe640 1 -- 172.21.15.83:0/1628182239 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f101400c300 con 0x7f1014003a30 2024-04-17T10:48:33.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f1023fff640 1 --2- 172.21.15.83:0/1628182239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1014004810 0x7f1014004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:33.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f10297cb640 1 --2- 172.21.15.83:0/1628182239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10140051d0 0x7f101400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:33.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f1028fca640 1 --2- 172.21.15.83:0/1628182239 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1014003a30 0x7f1014003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:33.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f1023fff640 1 -- 172.21.15.83:0/1628182239 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1014003a30 msgr2=0x7f1014003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:33.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f1023fff640 1 --2- 172.21.15.83:0/1628182239 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1014003a30 0x7f1014003e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:33.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f1023fff640 1 -- 172.21.15.83:0/1628182239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10140051d0 msgr2=0x7f101400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:33.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f1023fff640 1 --2- 172.21.15.83:0/1628182239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10140051d0 0x7f101400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:33.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f1023fff640 1 -- 172.21.15.83:0/1628182239 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1014003530 con 0x7f1014004810 2024-04-17T10:48:33.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f10297cb640 1 --2- 172.21.15.83:0/1628182239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10140051d0 0x7f101400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:33.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.294+0000 7f10297cb640 1 -- 172.21.15.83:0/1628182239 reap_dead start 2024-04-17T10:48:33.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.298+0000 7f1023fff640 1 --2- 172.21.15.83:0/1628182239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1014004810 0x7f1014004c90 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f1010009ae0 tx=0x7f101001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978f4d642409338f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:33.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.298+0000 7f10237fe640 1 -- 172.21.15.83:0/1628182239 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10100b2070 con 0x7f1014004810 2024-04-17T10:48:33.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.298+0000 7f10237fe640 1 -- 172.21.15.83:0/1628182239 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10100a5660 con 0x7f1014004810 2024-04-17T10:48:33.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.298+0000 7f10237fe640 1 -- 172.21.15.83:0/1628182239 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10100096b0 con 0x7f1014004810 2024-04-17T10:48:33.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.298+0000 7f102b22e640 1 -- 172.21.15.83:0/1628182239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1014004810 msgr2=0x7f1014004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:33.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.298+0000 7f102b22e640 1 --2- 172.21.15.83:0/1628182239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1014004810 0x7f1014004c90 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f1010009ae0 tx=0x7f101001ca70 comp rx=0 tx=0).stop 2024-04-17T10:48:33.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.298+0000 7f102b22e640 1 -- 172.21.15.83:0/1628182239 shutdown_connections 2024-04-17T10:48:33.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.298+0000 7f102b22e640 1 --2- 172.21.15.83:0/1628182239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1014004810 0x7f1014004c90 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:33.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.298+0000 7f102b22e640 1 -- 172.21.15.83:0/1628182239 >> 172.21.15.83:0/1628182239 conn(0x7f10240b15b0 msgr2=0x7f10240b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:33.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.298+0000 7f102b22e640 1 -- 172.21.15.83:0/1628182239 shutdown_connections 2024-04-17T10:48:33.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.298+0000 7f102b22e640 1 -- 172.21.15.83:0/1628182239 wait complete. 2024-04-17T10:48:33.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.298+0000 7f102b22e640 1 Processor -- start 2024-04-17T10:48:33.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.298+0000 7f102b22e640 1 -- start start 2024-04-17T10:48:33.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.302+0000 7f102b22e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1014004810 0x7f10240adf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:33.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.302+0000 7f102b22e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1024152df0 0x7f1024153270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:33.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.302+0000 7f102b22e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f102413abb0 0x7f102413b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:33.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.302+0000 7f102b22e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10240b3890 con 0x7f1014004810 2024-04-17T10:48:33.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.302+0000 7f102b22e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102406b520 con 0x7f1024152df0 2024-04-17T10:48:33.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.302+0000 7f102b22e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10240b3b70 con 0x7f102413abb0 2024-04-17T10:48:33.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.302+0000 7f1028fca640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1014004810 0x7f10240adf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:33.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.302+0000 7f1028fca640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1014004810 0x7f10240adf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44780/0 (socket says 172.21.15.83:44780) 2024-04-17T10:48:33.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.302+0000 7f1028fca640 1 -- 172.21.15.83:0/3823841490 learned_addr learned my addr 172.21.15.83:0/3823841490 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:33.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.302+0000 7f1023fff640 1 --2- 172.21.15.83:0/3823841490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1024152df0 0x7f1024153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:33.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.302+0000 7f10297cb640 1 --2- 172.21.15.83:0/3823841490 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f102413abb0 0x7f102413b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:33.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.302+0000 7f1028fca640 1 -- 172.21.15.83:0/3823841490 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f102413abb0 msgr2=0x7f102413b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:33.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.302+0000 7f1028fca640 1 --2- 172.21.15.83:0/3823841490 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f102413abb0 0x7f102413b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:33.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.302+0000 7f1028fca640 1 -- 172.21.15.83:0/3823841490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1024152df0 msgr2=0x7f1024153270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:33.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.302+0000 7f1028fca640 1 --2- 172.21.15.83:0/3823841490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1024152df0 0x7f1024153270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:33.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.302+0000 7f1028fca640 1 -- 172.21.15.83:0/3823841490 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f102413b8d0 con 0x7f1014004810 2024-04-17T10:48:33.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.302+0000 7f1023fff640 1 --2- 172.21.15.83:0/3823841490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1024152df0 0x7f1024153270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:33.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.302+0000 7f1028fca640 1 --2- 172.21.15.83:0/3823841490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1014004810 0x7f10240adf40 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f1018012de0 tx=0x7f10180106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:33.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.302+0000 7f10217fa640 1 -- 172.21.15.83:0/3823841490 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101801d630 con 0x7f1014004810 2024-04-17T10:48:33.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.302+0000 7f10217fa640 1 -- 172.21.15.83:0/3823841490 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1018017030 con 0x7f1014004810 2024-04-17T10:48:33.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.302+0000 7f102b22e640 1 -- 172.21.15.83:0/3823841490 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f102414d240 con 0x7f1014004810 2024-04-17T10:48:33.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.302+0000 7f10217fa640 1 -- 172.21.15.83:0/3823841490 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101801da10 con 0x7f1014004810 2024-04-17T10:48:33.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.302+0000 7f102b22e640 1 -- 172.21.15.83:0/3823841490 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f102413bec0 con 0x7f1014004810 2024-04-17T10:48:33.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.302+0000 7f102b22e640 1 -- 172.21.15.83:0/3823841490 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1024074f90 con 0x7f1014004810 2024-04-17T10:48:33.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.302+0000 7f10217fa640 1 -- 172.21.15.83:0/3823841490 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1018013400 con 0x7f1014004810 2024-04-17T10:48:33.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.302+0000 7f10217fa640 1 --2- 172.21.15.83:0/3823841490 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1008066f90 0x7f1008069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:33.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.302+0000 7f1023fff640 1 --2- 172.21.15.83:0/3823841490 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1008066f90 0x7f1008069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:33.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.302+0000 7f10217fa640 1 -- 172.21.15.83:0/3823841490 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f101808a950 con 0x7f1014004810 2024-04-17T10:48:33.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.302+0000 7f1023fff640 1 --2- 172.21.15.83:0/3823841490 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1008066f90 0x7f1008069450 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f10100af000 tx=0x7f1010008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:33.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.306+0000 7f10217fa640 1 -- 172.21.15.83:0/3823841490 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f101805b960 con 0x7f1014004810 2024-04-17T10:48:33.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.514+0000 7f102b22e640 1 -- 172.21.15.83:0/3823841490 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f10240b2a00 con 0x7f1014004810 2024-04-17T10:48:33.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.518+0000 7f10217fa640 1 -- 172.21.15.83:0/3823841490 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7f101805b260 con 0x7f1014004810 2024-04-17T10:48:33.514 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-04-17T10:48:33.514 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBUph9mHGmXGBAAZF+JiAG9YrFtbW+dCy2IEQ== 2024-04-17T10:48:33.514 INFO:teuthology.orchestra.run.smithi083.stdout: caps mds = "allow" 2024-04-17T10:48:33.514 INFO:teuthology.orchestra.run.smithi083.stdout: caps mon = "allow r" 2024-04-17T10:48:33.515 INFO:teuthology.orchestra.run.smithi083.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-17T10:48:33.515 INFO:teuthology.orchestra.run.smithi083.stderr:updated caps for client.0 2024-04-17T10:48:33.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.522+0000 7f102b22e640 1 -- 172.21.15.83:0/3823841490 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1008066f90 msgr2=0x7f1008069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:33.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.522+0000 7f102b22e640 1 --2- 172.21.15.83:0/3823841490 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1008066f90 0x7f1008069450 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f10100af000 tx=0x7f1010008040 comp rx=0 tx=0).stop 2024-04-17T10:48:33.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.522+0000 7f102b22e640 1 -- 172.21.15.83:0/3823841490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1014004810 msgr2=0x7f10240adf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:33.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.522+0000 7f102b22e640 1 --2- 172.21.15.83:0/3823841490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1014004810 0x7f10240adf40 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f1018012de0 tx=0x7f10180106e0 comp rx=0 tx=0).stop 2024-04-17T10:48:33.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.522+0000 7f102b22e640 1 -- 172.21.15.83:0/3823841490 shutdown_connections 2024-04-17T10:48:33.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.522+0000 7f102b22e640 1 --2- 172.21.15.83:0/3823841490 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1008066f90 0x7f1008069450 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:33.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.522+0000 7f102b22e640 1 --2- 172.21.15.83:0/3823841490 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f102413abb0 0x7f102413b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:33.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.522+0000 7f102b22e640 1 --2- 172.21.15.83:0/3823841490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1024152df0 0x7f1024153270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:33.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.522+0000 7f102b22e640 1 --2- 172.21.15.83:0/3823841490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1014004810 0x7f10240adf40 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:33.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.522+0000 7f102b22e640 1 -- 172.21.15.83:0/3823841490 >> 172.21.15.83:0/3823841490 conn(0x7f10240b15b0 msgr2=0x7f10240af8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:33.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.522+0000 7f102b22e640 1 -- 172.21.15.83:0/3823841490 shutdown_connections 2024-04-17T10:48:33.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.522+0000 7f102b22e640 1 -- 172.21.15.83:0/3823841490 wait complete. 2024-04-17T10:48:33.532 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:48:33.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.654+0000 7f2c29880640 1 Processor -- start 2024-04-17T10:48:33.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.654+0000 7f2c29880640 1 -- start start 2024-04-17T10:48:33.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.654+0000 7f2c29880640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2c24146b40 0x7f2c2406d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:33.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.654+0000 7f2c29880640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c24146f20 0x7f2c240720d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:33.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.654+0000 7f2c29880640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c24161ae0 con 0x7f2c2406abe0 2024-04-17T10:48:33.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.654+0000 7f2c29880640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c24161dc0 con 0x7f2c24146b40 2024-04-17T10:48:33.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.654+0000 7f2c29880640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c241620a0 con 0x7f2c24146f20 2024-04-17T10:48:33.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.654+0000 7f2c22ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2c24146b40 0x7f2c2406d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:33.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.654+0000 7f2c22ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2c24146b40 0x7f2c2406d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44790/0 (socket says 172.21.15.83:44790) 2024-04-17T10:48:33.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.654+0000 7f2c22ffd640 1 -- 172.21.15.83:0/1819254687 learned_addr learned my addr 172.21.15.83:0/1819254687 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:33.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.654+0000 7f2c22ffd640 1 -- 172.21.15.83:0/1819254687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c24146f20 msgr2=0x7f2c240720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:33.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.654+0000 7f2c22ffd640 1 --2- 172.21.15.83:0/1819254687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c24146f20 0x7f2c240720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:33.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.654+0000 7f2c22ffd640 1 -- 172.21.15.83:0/1819254687 >> v1:172.21.15.83:6789/0 conn(0x7f2c2406abe0 legacy=0x7f2c240d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:33.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.654+0000 7f2c22ffd640 1 -- 172.21.15.83:0/1819254687 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c24162380 con 0x7f2c24146b40 2024-04-17T10:48:33.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.654+0000 7f2c22ffd640 1 --2- 172.21.15.83:0/1819254687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c24146b40 0x7f2c2406d340 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f2c14009870 tx=0x7f2c1401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a2b577e5e7a11d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:33.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c227fc640 1 -- 172.21.15.83:0/1819254687 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c140b0070 con 0x7f2c24146b40 2024-04-17T10:48:33.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c227fc640 1 -- 172.21.15.83:0/1819254687 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c140ab600 con 0x7f2c24146b40 2024-04-17T10:48:33.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c227fc640 1 -- 172.21.15.83:0/1819254687 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c140ab9d0 con 0x7f2c24146b40 2024-04-17T10:48:33.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c29880640 1 -- 172.21.15.83:0/1819254687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c24146b40 msgr2=0x7f2c2406d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:33.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c29880640 1 --2- 172.21.15.83:0/1819254687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c24146b40 0x7f2c2406d340 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f2c14009870 tx=0x7f2c1401c560 comp rx=0 tx=0).stop 2024-04-17T10:48:33.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c29880640 1 -- 172.21.15.83:0/1819254687 shutdown_connections 2024-04-17T10:48:33.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c29880640 1 --2- 172.21.15.83:0/1819254687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c24146f20 0x7f2c240720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:33.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c29880640 1 --2- 172.21.15.83:0/1819254687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c24146b40 0x7f2c2406d340 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:33.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c29880640 1 -- 172.21.15.83:0/1819254687 >> 172.21.15.83:0/1819254687 conn(0x7f2c2406bcf0 msgr2=0x7f2c2406c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:33.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c29880640 1 -- 172.21.15.83:0/1819254687 shutdown_connections 2024-04-17T10:48:33.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c29880640 1 -- 172.21.15.83:0/1819254687 wait complete. 2024-04-17T10:48:33.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c29880640 1 Processor -- start 2024-04-17T10:48:33.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c29880640 1 -- start start 2024-04-17T10:48:33.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c29880640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c2406abe0 0x7f2c2415d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:33.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c29880640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c24146b40 0x7f2c241450c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:33.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c29880640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c24146f20 0x7f2c24145600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:33.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c29880640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2416ac20 con 0x7f2c2406abe0 2024-04-17T10:48:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c29880640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2416aaa0 con 0x7f2c24146b40 2024-04-17T10:48:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c29880640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2416ada0 con 0x7f2c24146f20 2024-04-17T10:48:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c22ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c2406abe0 0x7f2c2415d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c22ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c2406abe0 0x7f2c2415d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44796/0 (socket says 172.21.15.83:44796) 2024-04-17T10:48:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c22ffd640 1 -- 172.21.15.83:0/793293583 learned_addr learned my addr 172.21.15.83:0/793293583 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c22ffd640 1 -- 172.21.15.83:0/793293583 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c24146f20 msgr2=0x7f2c24145600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c22ffd640 1 --2- 172.21.15.83:0/793293583 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c24146f20 0x7f2c24145600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c22ffd640 1 -- 172.21.15.83:0/793293583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c24146b40 msgr2=0x7f2c241450c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c22ffd640 1 --2- 172.21.15.83:0/793293583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c24146b40 0x7f2c241450c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c22ffd640 1 -- 172.21.15.83:0/793293583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c241380d0 con 0x7f2c2406abe0 2024-04-17T10:48:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c22ffd640 1 --2- 172.21.15.83:0/793293583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c2406abe0 0x7f2c2415d450 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f2c1401a190 tx=0x7f2c14019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c20ff9640 1 -- 172.21.15.83:0/793293583 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c140b0040 con 0x7f2c2406abe0 2024-04-17T10:48:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c20ff9640 1 -- 172.21.15.83:0/793293583 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c140b9070 con 0x7f2c2406abe0 2024-04-17T10:48:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c29880640 1 -- 172.21.15.83:0/793293583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c241383b0 con 0x7f2c2406abe0 2024-04-17T10:48:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c20ff9640 1 -- 172.21.15.83:0/793293583 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c140b4880 con 0x7f2c2406abe0 2024-04-17T10:48:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.658+0000 7f2c29880640 1 -- 172.21.15.83:0/793293583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c24138970 con 0x7f2c2406abe0 2024-04-17T10:48:33.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.662+0000 7f2c29880640 1 -- 172.21.15.83:0/793293583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c24074f90 con 0x7f2c2406abe0 2024-04-17T10:48:33.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.662+0000 7f2c20ff9640 1 -- 172.21.15.83:0/793293583 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2c140ab4c0 con 0x7f2c2406abe0 2024-04-17T10:48:33.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.662+0000 7f2c20ff9640 1 --2- 172.21.15.83:0/793293583 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2c04066f60 0x7f2c04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:33.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.662+0000 7f2c20ff9640 1 -- 172.21.15.83:0/793293583 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c1411ed50 con 0x7f2c2406abe0 2024-04-17T10:48:33.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.662+0000 7f2c1a7fc640 1 --2- 172.21.15.83:0/793293583 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2c04066f60 0x7f2c04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:33.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.662+0000 7f2c1a7fc640 1 --2- 172.21.15.83:0/793293583 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2c04066f60 0x7f2c04069420 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2c240d0e60 tx=0x7f2c0c006ca0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:33.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.666+0000 7f2c20ff9640 1 -- 172.21.15.83:0/793293583 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2c14122030 con 0x7f2c2406abe0 2024-04-17T10:48:33.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.862+0000 7f2c29880640 1 -- 172.21.15.83:0/793293583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c24071bf0 con 0x7f2c2406abe0 2024-04-17T10:48:33.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.862+0000 7f2c20ff9640 1 -- 172.21.15.83:0/793293583 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f2c14016150 con 0x7f2c2406abe0 2024-04-17T10:48:33.860 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:33.860 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5786,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1553533838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1553533838},{"type":"v1","addr":"172.21.15.167:6833","nonce":1553533838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/674536913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":674536913},{"type":"v1","addr":"172.21.15.167:6835","nonce":674536913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8598,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4183890457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4183890457},{"type":"v1","addr":"172.21.15.83:6835","nonce":4183890457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:48:24.738443+0000","modified":"2024-04-17T10:48:26.776273+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8604":{"gid":8604,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1479417998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1479417998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1479417998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-17T10:48:33.861 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 69 2024-04-17T10:48:33.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.866+0000 7f2c29880640 1 -- 172.21.15.83:0/793293583 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2c04066f60 msgr2=0x7f2c04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:33.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.866+0000 7f2c29880640 1 --2- 172.21.15.83:0/793293583 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2c04066f60 0x7f2c04069420 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2c240d0e60 tx=0x7f2c0c006ca0 comp rx=0 tx=0).stop 2024-04-17T10:48:33.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.866+0000 7f2c29880640 1 -- 172.21.15.83:0/793293583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c2406abe0 msgr2=0x7f2c2415d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:33.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.866+0000 7f2c29880640 1 --2- 172.21.15.83:0/793293583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c2406abe0 0x7f2c2415d450 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f2c1401a190 tx=0x7f2c14019300 comp rx=0 tx=0).stop 2024-04-17T10:48:33.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.866+0000 7f2c29880640 1 -- 172.21.15.83:0/793293583 shutdown_connections 2024-04-17T10:48:33.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.866+0000 7f2c29880640 1 --2- 172.21.15.83:0/793293583 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2c04066f60 0x7f2c04069420 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:33.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.866+0000 7f2c29880640 1 --2- 172.21.15.83:0/793293583 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c24146f20 0x7f2c24145600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:33.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.866+0000 7f2c29880640 1 --2- 172.21.15.83:0/793293583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c24146b40 0x7f2c241450c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:33.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.866+0000 7f2c29880640 1 --2- 172.21.15.83:0/793293583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c2406abe0 0x7f2c2415d450 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:33.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.866+0000 7f2c29880640 1 -- 172.21.15.83:0/793293583 >> 172.21.15.83:0/793293583 conn(0x7f2c2406bcf0 msgr2=0x7f2c240b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:33.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.866+0000 7f2c29880640 1 -- 172.21.15.83:0/793293583 shutdown_connections 2024-04-17T10:48:33.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:33.866+0000 7f2c29880640 1 -- 172.21.15.83:0/793293583 wait complete. 2024-04-17T10:48:34.283 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 69, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T10:48:24.738443+0000', 'modified': '2024-04-17T10:48:26.776273+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8604}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8604': {'gid': 8604, 'name': 'a', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6837/1479417998', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6836', 'nonce': 1479417998}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 1479417998}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:48:34.283 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T10:48:34.283 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-17T10:48:34.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.422+0000 7fe986acd640 1 Processor -- start 2024-04-17T10:48:34.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.422+0000 7fe986acd640 1 -- start start 2024-04-17T10:48:34.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.422+0000 7fe986acd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe98015bd50 0x7fe9800cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:34.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.422+0000 7fe986acd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe9800727c0 0x7fe9800ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:34.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.422+0000 7fe986acd640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe98009e100 con 0x7fe9800669a0 2024-04-17T10:48:34.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.422+0000 7fe986acd640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9800cd130 con 0x7fe9800727c0 2024-04-17T10:48:34.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.422+0000 7fe986acd640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9800cc520 con 0x7fe98015bd50 2024-04-17T10:48:34.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.422+0000 7fe98506a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe9800727c0 0x7fe9800ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:34.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.422+0000 7fe98506a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe9800727c0 0x7fe9800ce050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44804/0 (socket says 172.21.15.83:44804) 2024-04-17T10:48:34.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.422+0000 7fe98506a640 1 -- 172.21.15.83:0/3307837044 learned_addr learned my addr 172.21.15.83:0/3307837044 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:34.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.422+0000 7fe984869640 1 --2- 172.21.15.83:0/3307837044 >> v2:172.21.15.167:3300/0 conn(0x7fe98015bd50 0x7fe9800cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:34.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.422+0000 7fe98506a640 1 -- 172.21.15.83:0/3307837044 >> v2:172.21.15.167:3300/0 conn(0x7fe98015bd50 msgr2=0x7fe9800cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:34.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.422+0000 7fe98506a640 1 --2- 172.21.15.83:0/3307837044 >> v2:172.21.15.167:3300/0 conn(0x7fe98015bd50 0x7fe9800cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.422+0000 7fe98506a640 1 -- 172.21.15.83:0/3307837044 >> v1:172.21.15.167:6789/0 conn(0x7fe9800669a0 legacy=0x7fe9800720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:34.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.422+0000 7fe98506a640 1 -- 172.21.15.83:0/3307837044 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9800cc800 con 0x7fe9800727c0 2024-04-17T10:48:34.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.426+0000 7fe98506a640 1 --2- 172.21.15.83:0/3307837044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9800727c0 0x7fe9800ce050 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fe980070050 tx=0x7fe97801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0ae62504ee3255 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:34.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.426+0000 7fe97f7fe640 1 -- 172.21.15.83:0/3307837044 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9780a5710 con 0x7fe9800727c0 2024-04-17T10:48:34.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.426+0000 7fe97f7fe640 1 -- 172.21.15.83:0/3307837044 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9780a58b0 con 0x7fe9800727c0 2024-04-17T10:48:34.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.426+0000 7fe97f7fe640 1 -- 172.21.15.83:0/3307837044 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9780a5c80 con 0x7fe9800727c0 2024-04-17T10:48:34.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.426+0000 7fe986acd640 1 -- 172.21.15.83:0/3307837044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9800727c0 msgr2=0x7fe9800ce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:34.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.426+0000 7fe986acd640 1 --2- 172.21.15.83:0/3307837044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9800727c0 0x7fe9800ce050 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fe980070050 tx=0x7fe97801c960 comp rx=0 tx=0).stop 2024-04-17T10:48:34.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.426+0000 7fe986acd640 1 -- 172.21.15.83:0/3307837044 shutdown_connections 2024-04-17T10:48:34.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.426+0000 7fe986acd640 1 --2- 172.21.15.83:0/3307837044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9800727c0 0x7fe9800ce050 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.426+0000 7fe986acd640 1 --2- 172.21.15.83:0/3307837044 >> v2:172.21.15.167:3300/0 conn(0x7fe98015bd50 0x7fe9800cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.426+0000 7fe986acd640 1 -- 172.21.15.83:0/3307837044 >> 172.21.15.83:0/3307837044 conn(0x7fe9800b3b40 msgr2=0x7fe9800b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:34.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.426+0000 7fe986acd640 1 -- 172.21.15.83:0/3307837044 shutdown_connections 2024-04-17T10:48:34.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.426+0000 7fe986acd640 1 -- 172.21.15.83:0/3307837044 wait complete. 2024-04-17T10:48:34.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.426+0000 7fe986acd640 1 Processor -- start 2024-04-17T10:48:34.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.426+0000 7fe986acd640 1 -- start start 2024-04-17T10:48:34.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.430+0000 7fe986acd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9800669a0 0x7fe98013fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:34.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.430+0000 7fe986acd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9800727c0 0x7fe980158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:34.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.430+0000 7fe986acd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe98015bd50 0x7fe98009bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:34.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.430+0000 7fe986acd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9800cef20 con 0x7fe9800727c0 2024-04-17T10:48:34.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.430+0000 7fe986acd640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9800ceda0 con 0x7fe9800669a0 2024-04-17T10:48:34.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.430+0000 7fe986acd640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe980066db0 con 0x7fe98015bd50 2024-04-17T10:48:34.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.430+0000 7fe97ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9800727c0 0x7fe980158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:34.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.430+0000 7fe97ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9800727c0 0x7fe980158ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44820/0 (socket says 172.21.15.83:44820) 2024-04-17T10:48:34.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.430+0000 7fe97ffff640 1 -- 172.21.15.83:0/747988315 learned_addr learned my addr 172.21.15.83:0/747988315 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:34.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.430+0000 7fe984869640 1 --2- 172.21.15.83:0/747988315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9800669a0 0x7fe98013fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:34.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.430+0000 7fe98506a640 1 --2- 172.21.15.83:0/747988315 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe98015bd50 0x7fe98009bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:34.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.430+0000 7fe97ffff640 1 -- 172.21.15.83:0/747988315 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe98015bd50 msgr2=0x7fe98009bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:34.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.430+0000 7fe97ffff640 1 --2- 172.21.15.83:0/747988315 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe98015bd50 0x7fe98009bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.430+0000 7fe97ffff640 1 -- 172.21.15.83:0/747988315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9800669a0 msgr2=0x7fe98013fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:34.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.430+0000 7fe97ffff640 1 --2- 172.21.15.83:0/747988315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9800669a0 0x7fe98013fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.430+0000 7fe97ffff640 1 -- 172.21.15.83:0/747988315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe980159420 con 0x7fe9800727c0 2024-04-17T10:48:34.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.430+0000 7fe97ffff640 1 --2- 172.21.15.83:0/747988315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9800727c0 0x7fe980158ee0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fe96800c420 tx=0x7fe96800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:34.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.430+0000 7fe97d7fa640 1 -- 172.21.15.83:0/747988315 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9680052d0 con 0x7fe9800727c0 2024-04-17T10:48:34.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.430+0000 7fe97d7fa640 1 -- 172.21.15.83:0/747988315 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe968005470 con 0x7fe9800727c0 2024-04-17T10:48:34.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.430+0000 7fe97d7fa640 1 -- 172.21.15.83:0/747988315 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe968007260 con 0x7fe9800727c0 2024-04-17T10:48:34.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.430+0000 7fe986acd640 1 -- 172.21.15.83:0/747988315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe98009c4c0 con 0x7fe9800727c0 2024-04-17T10:48:34.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.430+0000 7fe986acd640 1 -- 172.21.15.83:0/747988315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe98006ce90 con 0x7fe9800727c0 2024-04-17T10:48:34.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.430+0000 7fe97d7fa640 1 -- 172.21.15.83:0/747988315 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe968007400 con 0x7fe9800727c0 2024-04-17T10:48:34.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.430+0000 7fe97d7fa640 1 --2- 172.21.15.83:0/747988315 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe960066f60 0x7fe960069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:34.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.430+0000 7fe984869640 1 --2- 172.21.15.83:0/747988315 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe960066f60 0x7fe960069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:34.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.430+0000 7fe97d7fa640 1 -- 172.21.15.83:0/747988315 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe968084540 con 0x7fe9800727c0 2024-04-17T10:48:34.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.430+0000 7fe984869640 1 --2- 172.21.15.83:0/747988315 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe960066f60 0x7fe960069420 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe9801fa530 tx=0x7fe970008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:34.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe989c3a000 1 Processor -- start 2024-04-17T10:48:34.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe989c3a000 1 -- start start 2024-04-17T10:48:34.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe989c3a000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561717228920 0x561717228d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:34.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe989c3a000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561717229240 0x5617171de9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:34.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe989c3a000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617171def30 0x5617171df310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:34.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe989c3a000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617170c65f0 con 0x561717229240 2024-04-17T10:48:34.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe989c3a000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56171702dda0 con 0x5617171def30 2024-04-17T10:48:34.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe989c3a000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x561716fb6150 con 0x561717228920 2024-04-17T10:48:34.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe97ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561717229240 0x5617171de9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:34.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe97ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561717229240 0x5617171de9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44830/0 (socket says 172.21.15.83:44830) 2024-04-17T10:48:34.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe97ffff640 1 -- 172.21.15.83:0/2081452540 learned_addr learned my addr 172.21.15.83:0/2081452540 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:34.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe98506a640 1 --2- 172.21.15.83:0/2081452540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617171def30 0x5617171df310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:34.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe97ffff640 1 -- 172.21.15.83:0/2081452540 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561717228920 msgr2=0x561717228d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:34.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe984869640 1 --2- 172.21.15.83:0/2081452540 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561717228920 0x561717228d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:34.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe97ffff640 1 --2- 172.21.15.83:0/2081452540 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561717228920 0x561717228d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe97ffff640 1 -- 172.21.15.83:0/2081452540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617171def30 msgr2=0x5617171df310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:34.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe97ffff640 1 --2- 172.21.15.83:0/2081452540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617171def30 0x5617171df310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe97ffff640 1 -- 172.21.15.83:0/2081452540 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561716fcfca0 con 0x561717229240 2024-04-17T10:48:34.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe98506a640 1 --2- 172.21.15.83:0/2081452540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617171def30 0x5617171df310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:34.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe97ffff640 1 --2- 172.21.15.83:0/2081452540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561717229240 0x5617171de9f0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fe96805d020 tx=0x7fe96800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8004a4a6e19b1a7c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:34.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe9767fc640 1 -- 172.21.15.83:0/2081452540 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe968057070 con 0x561717229240 2024-04-17T10:48:34.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe9767fc640 1 -- 172.21.15.83:0/2081452540 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe968052ee0 con 0x561717229240 2024-04-17T10:48:34.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe9767fc640 1 -- 172.21.15.83:0/2081452540 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9680514b0 con 0x561717229240 2024-04-17T10:48:34.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe989c3a000 1 -- 172.21.15.83:0/2081452540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561717229240 msgr2=0x5617171de9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:34.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe989c3a000 1 --2- 172.21.15.83:0/2081452540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561717229240 0x5617171de9f0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fe96805d020 tx=0x7fe96800cf90 comp rx=0 tx=0).stop 2024-04-17T10:48:34.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe989c3a000 1 -- 172.21.15.83:0/2081452540 shutdown_connections 2024-04-17T10:48:34.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe989c3a000 1 --2- 172.21.15.83:0/2081452540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617171def30 0x5617171df310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe989c3a000 1 --2- 172.21.15.83:0/2081452540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561717229240 0x5617171de9f0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe989c3a000 1 --2- 172.21.15.83:0/2081452540 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561717228920 0x561717228d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe989c3a000 1 -- 172.21.15.83:0/2081452540 >> 172.21.15.83:0/2081452540 conn(0x5617171e52a0 msgr2=0x5617171dd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:34.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe989c3a000 1 -- 172.21.15.83:0/2081452540 shutdown_connections 2024-04-17T10:48:34.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe989c3a000 1 -- 172.21.15.83:0/2081452540 wait complete. 2024-04-17T10:48:34.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe989c3a000 1 -- start start 2024-04-17T10:48:34.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe989c3a000 1 Processor -- start 2024-04-17T10:48:34.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe989c3a000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5617171def30 0x56171722fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:34.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe989c3a000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561717228920 0x561717230230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:34.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe989c3a000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561717229240 0x5617171e2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:34.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe989c3a000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x561716fe8990 con 0x561717229240 2024-04-17T10:48:34.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe989c3a000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561717029e40 con 0x561717228920 2024-04-17T10:48:34.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe989c3a000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x561716fb9900 con 0x5617171def30 2024-04-17T10:48:34.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe989c3a000 10 client.0 _ll_register_callbacks cb 0x5617170e11c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:48:34.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.434+0000 7fe98506a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561717229240 0x5617171e2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:34.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe98506a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561717229240 0x5617171e2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44836/0 (socket says 172.21.15.83:44836) 2024-04-17T10:48:34.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe98506a640 1 -- 172.21.15.83:0/3836029541 learned_addr learned my addr 172.21.15.83:0/3836029541 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:34.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe984869640 1 --2- 172.21.15.83:0/3836029541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5617171def30 0x56171722fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:34.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe97ffff640 1 --2- 172.21.15.83:0/3836029541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561717228920 0x561717230230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:34.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe98506a640 1 -- 172.21.15.83:0/3836029541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5617171def30 msgr2=0x56171722fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:34.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe98506a640 1 --2- 172.21.15.83:0/3836029541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5617171def30 0x56171722fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe98506a640 1 -- 172.21.15.83:0/3836029541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561717228920 msgr2=0x561717230230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:34.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe98506a640 1 --2- 172.21.15.83:0/3836029541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561717228920 0x561717230230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe98506a640 1 -- 172.21.15.83:0/3836029541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561716fe6630 con 0x561717229240 2024-04-17T10:48:34.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe98506a640 1 --2- 172.21.15.83:0/3836029541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561717229240 0x5617171e2aa0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fe97801a590 tx=0x7fe97801b1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeec453677baac77 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:34.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe95f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:48:34.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe95f7fe640 1 -- 172.21.15.83:0/3836029541 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9780bd070 con 0x561717229240 2024-04-17T10:48:34.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe95f7fe640 1 -- 172.21.15.83:0/3836029541 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9780a5a50 con 0x561717229240 2024-04-17T10:48:34.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe989c3a000 1 -- 172.21.15.83:0/3836029541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe968052ee0 con 0x561717229240 2024-04-17T10:48:34.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe95f7fe640 1 -- 172.21.15.83:0/3836029541 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9780c8050 con 0x561717229240 2024-04-17T10:48:34.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe989c3a000 10 client.8772 fetch_fsmap learned FSMap version 69 2024-04-17T10:48:34.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe989c3a000 1 -- 172.21.15.83:0/3836029541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x561716fb63a0 con 0x561717229240 2024-04-17T10:48:34.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe95f7fe640 1 -- 172.21.15.83:0/3836029541 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe968052ee0 con 0x561717229240 2024-04-17T10:48:34.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe95f7fe640 1 -- 172.21.15.83:0/3836029541 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe9780cb020 con 0x561717229240 2024-04-17T10:48:34.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe989c3a000 10 client.8772 fetch_fsmap finished waiting for FSMap version 69 2024-04-17T10:48:34.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe989c3a000 10 client.8772 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T10:48:34.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe989c3a000 20 client.8772 populate_metadata read hostname 'smithi083' 2024-04-17T10:48:34.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe989c3a000 1 --2- 172.21.15.83:0/3836029541 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x5617171e4160 0x561717239d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe989c3a000 4 client.8772 mds_command: new command op to 8604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:48:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe989c3a000 1 -- 172.21.15.83:0/3836029541 --> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561716fc9a50 con 0x5617171e4160 2024-04-17T10:48:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe984869640 1 --2- 172.21.15.83:0/3836029541 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x5617171e4160 0x561717239d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe95f7fe640 10 client.8772 ms_handle_connect on v2:172.21.15.83:6836/1479417998 2024-04-17T10:48:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe984869640 1 --2- 172.21.15.83:0/3836029541 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x5617171e4160 0x561717239d50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c7218e93bbb1f3d server_cookie=4745591b96a62427 in_seq=0 out_seq=0 2024-04-17T10:48:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe95f7fe640 1 -- 172.21.15.83:0/3836029541 <== mds.0 v2:172.21.15.83:6836/1479417998 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561716fc9a50 con 0x5617171e4160 2024-04-17T10:48:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe95f7fe640 10 client.8772 handle_command_reply: tid=0 2024-04-17T10:48:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe989c3a000 1 client.8772 shutdown 2024-04-17T10:48:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe989c3a000 1 -- 172.21.15.83:0/3836029541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561717229240 msgr2=0x5617171e2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe989c3a000 1 --2- 172.21.15.83:0/3836029541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561717229240 0x5617171e2aa0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fe97801a590 tx=0x7fe97801b1a0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe989c3a000 1 -- 172.21.15.83:0/3836029541 shutdown_connections 2024-04-17T10:48:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe989c3a000 1 --2- 172.21.15.83:0/3836029541 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x5617171e4160 0x561717239d50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe989c3a000 1 --2- 172.21.15.83:0/3836029541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561717229240 0x5617171e2aa0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe989c3a000 1 --2- 172.21.15.83:0/3836029541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561717228920 0x561717230230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe95f7fe640 0 client.8772 ms_handle_reset on v2:172.21.15.83:6836/1479417998 2024-04-17T10:48:34.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe989c3a000 1 --2- 172.21.15.83:0/3836029541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5617171def30 0x56171722fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe989c3a000 1 -- 172.21.15.83:0/3836029541 >> 172.21.15.83:0/3836029541 conn(0x5617171e52a0 msgr2=0x5617171dd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:34.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe989c3a000 1 -- 172.21.15.83:0/3836029541 shutdown_connections 2024-04-17T10:48:34.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe989c3a000 1 -- 172.21.15.83:0/3836029541 wait complete. 2024-04-17T10:48:34.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.438+0000 7fe989c3a000 20 client.8772 trim_cache size 0 max 16384 2024-04-17T10:48:34.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.454+0000 7fe989c3a000 1 Processor -- start 2024-04-17T10:48:34.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.454+0000 7fe989c3a000 1 -- start start 2024-04-17T10:48:34.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.454+0000 7fe989c3a000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617171def30 0x561717283a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:34.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.454+0000 7fe989c3a000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5617171e4160 0x561717283f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:34.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.454+0000 7fe989c3a000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561717228920 0x5617172844b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:34.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.454+0000 7fe989c3a000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9800b1980 con 0x561717228920 2024-04-17T10:48:34.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.454+0000 7fe989c3a000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98015dd30 con 0x5617171def30 2024-04-17T10:48:34.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.454+0000 7fe989c3a000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9800b13b0 con 0x5617171e4160 2024-04-17T10:48:34.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.454+0000 7fe98506a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561717228920 0x5617172844b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:34.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.454+0000 7fe98506a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561717228920 0x5617172844b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44838/0 (socket says 172.21.15.83:44838) 2024-04-17T10:48:34.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.454+0000 7fe98506a640 1 -- 172.21.15.83:0/2694123756 learned_addr learned my addr 172.21.15.83:0/2694123756 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:34.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.454+0000 7fe97ffff640 1 --2- 172.21.15.83:0/2694123756 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5617171e4160 0x561717283f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:34.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.454+0000 7fe984869640 1 --2- 172.21.15.83:0/2694123756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617171def30 0x561717283a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:34.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.454+0000 7fe98506a640 1 -- 172.21.15.83:0/2694123756 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5617171e4160 msgr2=0x561717283f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:34.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.454+0000 7fe98506a640 1 --2- 172.21.15.83:0/2694123756 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5617171e4160 0x561717283f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.454+0000 7fe98506a640 1 -- 172.21.15.83:0/2694123756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617171def30 msgr2=0x561717283a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:34.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.454+0000 7fe98506a640 1 --2- 172.21.15.83:0/2694123756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617171def30 0x561717283a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.454+0000 7fe98506a640 1 -- 172.21.15.83:0/2694123756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561717229070 con 0x561717228920 2024-04-17T10:48:34.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.454+0000 7fe97ffff640 1 --2- 172.21.15.83:0/2694123756 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5617171e4160 0x561717283f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:34.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.454+0000 7fe984869640 1 --2- 172.21.15.83:0/2694123756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617171def30 0x561717283a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:34.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.454+0000 7fe98506a640 1 --2- 172.21.15.83:0/2694123756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561717228920 0x5617172844b0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fe97800c090 tx=0x7fe9780b7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d6c618cf77d86e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:34.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.454+0000 7fe975ffb640 1 -- 172.21.15.83:0/2694123756 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9780bd070 con 0x561717228920 2024-04-17T10:48:34.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.454+0000 7fe975ffb640 1 -- 172.21.15.83:0/2694123756 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9780a5710 con 0x561717228920 2024-04-17T10:48:34.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.454+0000 7fe975ffb640 1 -- 172.21.15.83:0/2694123756 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978009510 con 0x561717228920 2024-04-17T10:48:34.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 1 -- 172.21.15.83:0/2694123756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561717228920 msgr2=0x5617172844b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:34.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 1 --2- 172.21.15.83:0/2694123756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561717228920 0x5617172844b0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fe97800c090 tx=0x7fe9780b7000 comp rx=0 tx=0).stop 2024-04-17T10:48:34.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 1 -- 172.21.15.83:0/2694123756 shutdown_connections 2024-04-17T10:48:34.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 1 --2- 172.21.15.83:0/2694123756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561717228920 0x5617172844b0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 1 --2- 172.21.15.83:0/2694123756 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5617171e4160 0x561717283f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 1 --2- 172.21.15.83:0/2694123756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617171def30 0x561717283a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 1 -- 172.21.15.83:0/2694123756 >> 172.21.15.83:0/2694123756 conn(0x5617171e52a0 msgr2=0x5617171e26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:34.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 1 -- 172.21.15.83:0/2694123756 shutdown_connections 2024-04-17T10:48:34.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 1 -- 172.21.15.83:0/2694123756 wait complete. 2024-04-17T10:48:34.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 1 -- start start 2024-04-17T10:48:34.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 1 Processor -- start 2024-04-17T10:48:34.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617171def30 0x56171726d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:34.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5617171e4160 0x56171726d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:34.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561717228920 0x56171726dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:34.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617170323d0 con 0x561717228920 2024-04-17T10:48:34.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617170e9d50 con 0x5617171def30 2024-04-17T10:48:34.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x561716fc8980 con 0x5617171e4160 2024-04-17T10:48:34.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 10 client.0 _ll_register_callbacks cb 0x7fe968051160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:48:34.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe98506a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561717228920 0x56171726dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:34.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe98506a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561717228920 0x56171726dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44854/0 (socket says 172.21.15.83:44854) 2024-04-17T10:48:34.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe98506a640 1 -- 172.21.15.83:0/326505883 learned_addr learned my addr 172.21.15.83:0/326505883 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:34.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe98506a640 1 -- 172.21.15.83:0/326505883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5617171e4160 msgr2=0x56171726d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:34.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe97ffff640 1 --2- 172.21.15.83:0/326505883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5617171e4160 0x56171726d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:34.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe984869640 1 --2- 172.21.15.83:0/326505883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617171def30 0x56171726d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:34.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe98506a640 1 --2- 172.21.15.83:0/326505883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5617171e4160 0x56171726d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe98506a640 1 -- 172.21.15.83:0/326505883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617171def30 msgr2=0x56171726d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:34.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe98506a640 1 --2- 172.21.15.83:0/326505883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617171def30 0x56171726d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe98506a640 1 -- 172.21.15.83:0/326505883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5617170af600 con 0x561717228920 2024-04-17T10:48:34.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe984869640 1 --2- 172.21.15.83:0/326505883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617171def30 0x56171726d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:34.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe98506a640 1 --2- 172.21.15.83:0/326505883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561717228920 0x56171726dd20 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fe978002240 tx=0x7fe978009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c852efd658cb347 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:34.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe95f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:48:34.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe95f7fe640 1 -- 172.21.15.83:0/326505883 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978003bb0 con 0x561717228920 2024-04-17T10:48:34.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe95f7fe640 1 -- 172.21.15.83:0/326505883 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe978003d50 con 0x561717228920 2024-04-17T10:48:34.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 1 -- 172.21.15.83:0/326505883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9780a5710 con 0x561717228920 2024-04-17T10:48:34.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe95f7fe640 1 -- 172.21.15.83:0/326505883 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9780b8850 con 0x561717228920 2024-04-17T10:48:34.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 10 client.8778 fetch_fsmap learned FSMap version 69 2024-04-17T10:48:34.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 1 -- 172.21.15.83:0/326505883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x561716fc29b0 con 0x561717228920 2024-04-17T10:48:34.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe95f7fe640 1 -- 172.21.15.83:0/326505883 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9780a5710 con 0x561717228920 2024-04-17T10:48:34.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe95f7fe640 1 -- 172.21.15.83:0/326505883 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe9780cb040 con 0x561717228920 2024-04-17T10:48:34.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 10 client.8778 fetch_fsmap finished waiting for FSMap version 69 2024-04-17T10:48:34.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 10 client.8778 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T10:48:34.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 20 client.8778 populate_metadata read hostname 'smithi083' 2024-04-17T10:48:34.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 1 --2- 172.21.15.83:0/326505883 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x561717229240 0x561717235240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:34.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 4 client.8778 mds_command: new command op to 8604 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T10:48:34.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 1 -- 172.21.15.83:0/326505883 --> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56171703b2e0 con 0x561717229240 2024-04-17T10:48:34.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe984869640 1 --2- 172.21.15.83:0/326505883 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x561717229240 0x561717235240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:34.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe95f7fe640 10 client.8778 ms_handle_connect on v2:172.21.15.83:6836/1479417998 2024-04-17T10:48:34.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe984869640 1 --2- 172.21.15.83:0/326505883 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x561717229240 0x561717235240 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c64fb0279a307dd3 server_cookie=fb99243397cee73f in_seq=0 out_seq=0 2024-04-17T10:48:34.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe95f7fe640 1 -- 172.21.15.83:0/326505883 <== mds.0 v2:172.21.15.83:6836/1479417998 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x56171703b2e0 con 0x561717229240 2024-04-17T10:48:34.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe95f7fe640 10 client.8778 handle_command_reply: tid=0 2024-04-17T10:48:34.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 1 client.8778 shutdown 2024-04-17T10:48:34.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 1 -- 172.21.15.83:0/326505883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561717228920 msgr2=0x56171726dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:34.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 1 --2- 172.21.15.83:0/326505883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561717228920 0x56171726dd20 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fe978002240 tx=0x7fe978009810 comp rx=0 tx=0).stop 2024-04-17T10:48:34.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 1 -- 172.21.15.83:0/326505883 shutdown_connections 2024-04-17T10:48:34.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 1 --2- 172.21.15.83:0/326505883 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x561717229240 0x561717235240 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 1 --2- 172.21.15.83:0/326505883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561717228920 0x56171726dd20 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 1 --2- 172.21.15.83:0/326505883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5617171e4160 0x56171726d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 1 --2- 172.21.15.83:0/326505883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617171def30 0x56171726d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe95f7fe640 0 client.8778 ms_handle_reset on v2:172.21.15.83:6836/1479417998 2024-04-17T10:48:34.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 1 -- 172.21.15.83:0/326505883 >> 172.21.15.83:0/326505883 conn(0x5617171e52a0 msgr2=0x5617172307a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:34.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 1 -- 172.21.15.83:0/326505883 shutdown_connections 2024-04-17T10:48:34.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 1 -- 172.21.15.83:0/326505883 wait complete. 2024-04-17T10:48:34.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.458+0000 7fe989c3a000 20 client.8778 trim_cache size 0 max 16384 2024-04-17T10:48:34.464 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:48:34.464 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "3de9b7be-af7f-488a-969e-2b6b8db63bcd", 2024-04-17T10:48:34.464 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-04-17T10:48:34.464 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 8604, 2024-04-17T10:48:34.464 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-17T10:48:34.465 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-17T10:48:34.465 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-17T10:48:34.465 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 9.0570793129999991, 2024-04-17T10:48:34.465 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 69, 2024-04-17T10:48:34.465 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 114, 2024-04-17T10:48:34.465 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 114, 2024-04-17T10:48:34.465 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 14.096881971 2024-04-17T10:48:34.465 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:48:34.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.462+0000 7fe986acd640 1 -- 172.21.15.83:0/747988315 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe960066f60 msgr2=0x7fe960069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:34.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.462+0000 7fe986acd640 1 --2- 172.21.15.83:0/747988315 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe960066f60 0x7fe960069420 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe9801fa530 tx=0x7fe970008040 comp rx=0 tx=0).stop 2024-04-17T10:48:34.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.462+0000 7fe986acd640 1 -- 172.21.15.83:0/747988315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9800727c0 msgr2=0x7fe980158ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:34.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.462+0000 7fe986acd640 1 --2- 172.21.15.83:0/747988315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9800727c0 0x7fe980158ee0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fe96800c420 tx=0x7fe96800c8e0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.462+0000 7fe986acd640 1 -- 172.21.15.83:0/747988315 shutdown_connections 2024-04-17T10:48:34.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.462+0000 7fe986acd640 1 --2- 172.21.15.83:0/747988315 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe960066f60 0x7fe960069420 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.462+0000 7fe986acd640 1 --2- 172.21.15.83:0/747988315 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe98015bd50 0x7fe98009bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.462+0000 7fe986acd640 1 --2- 172.21.15.83:0/747988315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9800727c0 0x7fe980158ee0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.462+0000 7fe986acd640 1 --2- 172.21.15.83:0/747988315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9800669a0 0x7fe98013fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.462+0000 7fe986acd640 1 -- 172.21.15.83:0/747988315 >> 172.21.15.83:0/747988315 conn(0x7fe9800b3b40 msgr2=0x7fe98015e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:34.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.462+0000 7fe986acd640 1 -- 172.21.15.83:0/747988315 shutdown_connections 2024-04-17T10:48:34.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.462+0000 7fe986acd640 1 -- 172.21.15.83:0/747988315 wait complete. 2024-04-17T10:48:34.474 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T10:48:34.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.586+0000 7f6200ead640 1 Processor -- start 2024-04-17T10:48:34.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.586+0000 7f6200ead640 1 -- start start 2024-04-17T10:48:34.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.586+0000 7f6200ead640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f61fc15dfa0 0x7f61fc0b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:34.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.586+0000 7f6200ead640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f61fc0b5b40 0x7f61fc0d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:34.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.586+0000 7f6200ead640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61fc162a00 con 0x7f61fc0ce980 2024-04-17T10:48:34.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.586+0000 7f6200ead640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61fc161e80 con 0x7f61fc15dfa0 2024-04-17T10:48:34.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.586+0000 7f6200ead640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61fc162160 con 0x7f61fc0b5b40 2024-04-17T10:48:34.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.586+0000 7f61fa59c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f61fc15dfa0 0x7f61fc0b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:34.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.586+0000 7f61fa59c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f61fc15dfa0 0x7f61fc0b6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44870/0 (socket says 172.21.15.83:44870) 2024-04-17T10:48:34.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.586+0000 7f61fa59c640 1 -- 172.21.15.83:0/227168953 learned_addr learned my addr 172.21.15.83:0/227168953 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:34.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.586+0000 7f61fa59c640 1 -- 172.21.15.83:0/227168953 >> v2:172.21.15.167:3300/0 conn(0x7f61fc0b5b40 msgr2=0x7f61fc0d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:34.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.586+0000 7f61fa59c640 1 --2- 172.21.15.83:0/227168953 >> v2:172.21.15.167:3300/0 conn(0x7f61fc0b5b40 0x7f61fc0d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.586+0000 7f61fa59c640 1 -- 172.21.15.83:0/227168953 >> v1:172.21.15.167:6789/0 conn(0x7f61fc0ce980 legacy=0x7f61fc0b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:34.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.586+0000 7f61fa59c640 1 -- 172.21.15.83:0/227168953 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61fc162440 con 0x7f61fc15dfa0 2024-04-17T10:48:34.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.590+0000 7f61fa59c640 1 --2- 172.21.15.83:0/227168953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61fc15dfa0 0x7f61fc0b6c20 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f61fc08c3f0 tx=0x7f61ec01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a02e7ae696562ba0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:34.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.590+0000 7f61f9d9b640 1 -- 172.21.15.83:0/227168953 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec0b0070 con 0x7f61fc15dfa0 2024-04-17T10:48:34.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.590+0000 7f61f9d9b640 1 -- 172.21.15.83:0/227168953 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ec004230 con 0x7f61fc15dfa0 2024-04-17T10:48:34.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.590+0000 7f61f9d9b640 1 -- 172.21.15.83:0/227168953 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec003620 con 0x7f61fc15dfa0 2024-04-17T10:48:34.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.590+0000 7f6200ead640 1 -- 172.21.15.83:0/227168953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61fc15dfa0 msgr2=0x7f61fc0b6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:34.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.590+0000 7f6200ead640 1 --2- 172.21.15.83:0/227168953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61fc15dfa0 0x7f61fc0b6c20 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f61fc08c3f0 tx=0x7f61ec01c440 comp rx=0 tx=0).stop 2024-04-17T10:48:34.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.590+0000 7f6200ead640 1 -- 172.21.15.83:0/227168953 shutdown_connections 2024-04-17T10:48:34.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.590+0000 7f6200ead640 1 --2- 172.21.15.83:0/227168953 >> v2:172.21.15.167:3300/0 conn(0x7f61fc0b5b40 0x7f61fc0d1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.590+0000 7f6200ead640 1 --2- 172.21.15.83:0/227168953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61fc15dfa0 0x7f61fc0b6c20 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.590+0000 7f6200ead640 1 -- 172.21.15.83:0/227168953 >> 172.21.15.83:0/227168953 conn(0x7f61fc0d0600 msgr2=0x7f61fc0d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:34.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.590+0000 7f6200ead640 1 -- 172.21.15.83:0/227168953 shutdown_connections 2024-04-17T10:48:34.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.590+0000 7f6200ead640 1 -- 172.21.15.83:0/227168953 wait complete. 2024-04-17T10:48:34.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.590+0000 7f6200ead640 1 Processor -- start 2024-04-17T10:48:34.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.590+0000 7f6200ead640 1 -- start start 2024-04-17T10:48:34.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.590+0000 7f6200ead640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61fc0b5b40 0x7f61fc0c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:34.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.590+0000 7f6200ead640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61fc0ce980 0x7f61fc0d39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:34.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.590+0000 7f6200ead640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61fc15dfa0 0x7f61fc0d3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:34.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.590+0000 7f6200ead640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc16ae50 con 0x7f61fc15dfa0 2024-04-17T10:48:34.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.590+0000 7f6200ead640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc161b10 con 0x7f61fc0ce980 2024-04-17T10:48:34.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.590+0000 7f6200ead640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc16afd0 con 0x7f61fc0b5b40 2024-04-17T10:48:34.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.590+0000 7f61fad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61fc15dfa0 0x7f61fc0d3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:34.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.594+0000 7f61fad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61fc15dfa0 0x7f61fc0d3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44872/0 (socket says 172.21.15.83:44872) 2024-04-17T10:48:34.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.594+0000 7f61fad9d640 1 -- 172.21.15.83:0/2986379738 learned_addr learned my addr 172.21.15.83:0/2986379738 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:34.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.594+0000 7f61fa59c640 1 --2- 172.21.15.83:0/2986379738 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61fc0b5b40 0x7f61fc0c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:34.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.594+0000 7f61f3fff640 1 --2- 172.21.15.83:0/2986379738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61fc0ce980 0x7f61fc0d39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:34.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.594+0000 7f61fad9d640 1 -- 172.21.15.83:0/2986379738 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61fc0b5b40 msgr2=0x7f61fc0c5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:34.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.594+0000 7f61fad9d640 1 --2- 172.21.15.83:0/2986379738 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61fc0b5b40 0x7f61fc0c5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.594+0000 7f61fad9d640 1 -- 172.21.15.83:0/2986379738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61fc0ce980 msgr2=0x7f61fc0d39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:34.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.594+0000 7f61fad9d640 1 --2- 172.21.15.83:0/2986379738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61fc0ce980 0x7f61fc0d39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.594+0000 7f61fad9d640 1 -- 172.21.15.83:0/2986379738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61fc152810 con 0x7f61fc15dfa0 2024-04-17T10:48:34.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.594+0000 7f61f3fff640 1 --2- 172.21.15.83:0/2986379738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61fc0ce980 0x7f61fc0d39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:34.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.594+0000 7f61fad9d640 1 --2- 172.21.15.83:0/2986379738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61fc15dfa0 0x7f61fc0d3ef0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f61f400b9e0 tx=0x7f61f400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:34.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.594+0000 7f61f2ffd640 1 -- 172.21.15.83:0/2986379738 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f4019070 con 0x7f61fc15dfa0 2024-04-17T10:48:34.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.594+0000 7f61f2ffd640 1 -- 172.21.15.83:0/2986379738 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f400cb40 con 0x7f61fc15dfa0 2024-04-17T10:48:34.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.594+0000 7f61f2ffd640 1 -- 172.21.15.83:0/2986379738 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f4014440 con 0x7f61fc15dfa0 2024-04-17T10:48:34.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.594+0000 7f6200ead640 1 -- 172.21.15.83:0/2986379738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61fc152af0 con 0x7f61fc15dfa0 2024-04-17T10:48:34.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.594+0000 7f6200ead640 1 -- 172.21.15.83:0/2986379738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61fc1530b0 con 0x7f61fc15dfa0 2024-04-17T10:48:34.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.594+0000 7f6200ead640 1 -- 172.21.15.83:0/2986379738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61fc0681a0 con 0x7f61fc15dfa0 2024-04-17T10:48:34.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.594+0000 7f61f2ffd640 1 -- 172.21.15.83:0/2986379738 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f61f401c070 con 0x7f61fc15dfa0 2024-04-17T10:48:34.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.594+0000 7f61f2ffd640 1 --2- 172.21.15.83:0/2986379738 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f61dc066f60 0x7f61dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:34.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.594+0000 7f61f2ffd640 1 -- 172.21.15.83:0/2986379738 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61f4084cc0 con 0x7f61fc15dfa0 2024-04-17T10:48:34.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.594+0000 7f61fa59c640 1 --2- 172.21.15.83:0/2986379738 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f61dc066f60 0x7f61dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:34.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.594+0000 7f61fa59c640 1 --2- 172.21.15.83:0/2986379738 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f61dc066f60 0x7f61dc069420 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f61ec0ad000 tx=0x7f61ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:34.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.598+0000 7f61f2ffd640 1 -- 172.21.15.83:0/2986379738 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f61f4055cd0 con 0x7f61fc15dfa0 2024-04-17T10:48:34.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.778+0000 7f6200ead640 1 -- 172.21.15.83:0/2986379738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f61fc0b6be0 con 0x7f61fc15dfa0 2024-04-17T10:48:34.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.778+0000 7f61f2ffd640 1 -- 172.21.15.83:0/2986379738 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v69) v1 ==== 53+0+73 (secure 0 0 0) 0x7f61f40555d0 con 0x7f61fc15dfa0 2024-04-17T10:48:34.775 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T10:48:34.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.782+0000 7f6200ead640 1 -- 172.21.15.83:0/2986379738 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f61dc066f60 msgr2=0x7f61dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:34.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.782+0000 7f6200ead640 1 --2- 172.21.15.83:0/2986379738 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f61dc066f60 0x7f61dc069420 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f61ec0ad000 tx=0x7f61ec008040 comp rx=0 tx=0).stop 2024-04-17T10:48:34.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.782+0000 7f6200ead640 1 -- 172.21.15.83:0/2986379738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61fc15dfa0 msgr2=0x7f61fc0d3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:34.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.782+0000 7f6200ead640 1 --2- 172.21.15.83:0/2986379738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61fc15dfa0 0x7f61fc0d3ef0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f61f400b9e0 tx=0x7f61f400bea0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.782+0000 7f6200ead640 1 -- 172.21.15.83:0/2986379738 shutdown_connections 2024-04-17T10:48:34.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.782+0000 7f6200ead640 1 --2- 172.21.15.83:0/2986379738 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f61dc066f60 0x7f61dc069420 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.782+0000 7f6200ead640 1 --2- 172.21.15.83:0/2986379738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61fc15dfa0 0x7f61fc0d3ef0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.782+0000 7f6200ead640 1 --2- 172.21.15.83:0/2986379738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61fc0ce980 0x7f61fc0d39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.782+0000 7f6200ead640 1 --2- 172.21.15.83:0/2986379738 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61fc0b5b40 0x7f61fc0c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:34.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.782+0000 7f6200ead640 1 -- 172.21.15.83:0/2986379738 >> 172.21.15.83:0/2986379738 conn(0x7f61fc0d0600 msgr2=0x7f61fc0cd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:34.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.782+0000 7f6200ead640 1 -- 172.21.15.83:0/2986379738 shutdown_connections 2024-04-17T10:48:34.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:34.782+0000 7f6200ead640 1 -- 172.21.15.83:0/2986379738 wait complete. 2024-04-17T10:48:34.793 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T10:48:34.793 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T10:48:34.793 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-04-17T10:48:34.793 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T10:48:34.793 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T10:48:34.793 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T10:48:34.793 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T10:48:34.793 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T10:48:34.794 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:48:34.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:34.794 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-04-17T10:48:34.801 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T10:48:34.801 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T10:48:34.801 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T10:48:34.801 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:48:34.801 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-17T10:48:34.801 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:48:34.801 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:48:34.801 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-04-17T10:48:34.801 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-04-17T10:48:34.801 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:48:34.802 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-04-17T10:48:34.802 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-04-17T10:48:34.802 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:48:34.802 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:48:34.802 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T10:48:34.802 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:48:34.802 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-17T10:48:34.802 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 41869sec preferred_lft 41869sec 2024-04-17T10:48:34.802 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-04-17T10:48:34.802 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:48:34.802 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T10:48:34.802 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:37:0e:23:b2 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:48:34.802 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T10:48:34.803 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:48:34.803 INFO:teuthology.orchestra.run.smithi083.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T10:48:34.803 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-17T10:48:34.803 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T10:48:34.803 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:48:34.803 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-17T10:48:34.803 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:48:34.803 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:48:34.803 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:48:34.803 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-17T10:48:34.803 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:48:34.803 INFO:teuthology.orchestra.run.smithi083.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:48:34.803 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:48:34.803 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-17T10:48:34.804 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:48:34.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:34.804 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T10:48:34.854 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-04-17T10:48:34.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:34.856 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-04-17T10:48:34.909 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-04-17T10:48:34.909 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T10:48:34.909 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-17T10:48:34.909 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-17T10:48:34.909 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-17T10:48:34.909 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T10:48:34.910 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-17T10:48:34.910 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:48:34.910 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:48:34.910 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:48:34.910 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T10:48:34.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:34.910 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2024-04-17T10:48:34.910 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:34.910 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-17T10:48:34.910 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-17T10:48:34.911 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-17T10:48:34.911 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:34.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:34.983 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-17T10:48:35.029 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-17T10:48:35.029 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T10:48:35.030 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:48:35.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:35.030 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-04-17T10:48:35.086 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-17T10:48:35.086 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:48:35.086 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T10:48:35.133 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:48:35.133 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:48:35.134 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:48:35.185 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:48:35.246 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:48:35.296 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:48:35.296 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:48:35.296 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:48:35.296 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:48:35.296 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:48:35.296 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:48:35.296 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:48:35.296 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:48:35.297 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:48:35.297 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:48:35.297 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:48:35.297 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:48:35.297 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:48:35.297 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:48:35.297 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:48:35.297 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:48:35.297 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:48:35.297 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:48:35.297 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:48:35.297 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:48:35.297 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:48:35.297 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:48:35.298 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:48:35.298 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:48:35.298 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:48:35.298 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:48:35.298 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:48:35.298 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:48:35.298 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:48:35.298 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:48:35.298 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:48:35.298 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:48:35.298 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:48:35.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:35.299 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:48:35.345 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T10:48:35.345 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-17T10:48:35.390 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:48:35.403 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:48:35.452 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:48:35.452 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:48:35.452 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:48:35.452 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:48:35.452 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:48:35.452 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:48:35.453 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:48:35.453 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:48:35.453 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:48:35.453 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:48:35.453 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:48:35.453 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:48:35.453 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:48:35.453 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:48:35.453 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:48:35.453 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:48:35.453 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:48:35.453 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:48:35.453 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:48:35.453 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:48:35.454 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:48:35.454 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:48:35.454 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:48:35.454 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:48:35.454 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:48:35.454 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:48:35.454 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:48:35.454 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:48:35.454 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:48:35.454 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:48:35.454 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:48:35.454 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:48:35.454 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:48:35.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:35.455 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:48:35.470 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:2024-04-17T10:48:35.474+0000 7f569202c580 -1 init, newargv = 0x55cea4bb9050 newargc=15 2024-04-17T10:48:35.470 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[76725]: starting ceph client 2024-04-17T10:48:35.474 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[76725]: starting fuse 2024-04-17T10:48:36.576 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:48:36.593 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:48:36.644 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:48:36.644 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:48:36.644 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:48:36.644 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:48:36.644 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:48:36.644 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:48:36.644 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:48:36.644 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:48:36.644 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:48:36.644 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:48:36.644 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:48:36.644 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:48:36.645 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:48:36.645 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:48:36.645 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:48:36.645 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:48:36.645 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:48:36.645 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:48:36.645 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:48:36.645 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:48:36.645 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:48:36.645 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:48:36.645 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:48:36.645 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:48:36.645 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:48:36.646 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:48:36.646 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:48:36.646 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:48:36.646 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:48:36.646 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:48:36.646 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:48:36.646 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:48:36.646 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:48:36.646 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-17T10:48:36.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:36.647 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:48:36.693 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-04-17T10:48:36.693 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-17T10:48:36.694 DEBUG:teuthology.orchestra.run.smithi083:> sudo stdin-killer -- python3 -c ' 2024-04-17T10:48:36.694 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2024-04-17T10:48:36.694 DEBUG:teuthology.orchestra.run.smithi083:> import re 2024-04-17T10:48:36.694 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-04-17T10:48:36.694 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2024-04-17T10:48:36.694 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:48:36.694 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2024-04-17T10:48:36.694 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-17T10:48:36.694 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2024-04-17T10:48:36.694 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-17T10:48:36.694 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:48:36.694 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2024-04-17T10:48:36.695 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2024-04-17T10:48:36.695 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2024-04-17T10:48:36.695 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2024-04-17T10:48:36.695 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:48:36.695 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2024-04-17T10:48:36.695 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T10:48:36.695 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T10:48:36.695 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T10:48:36.695 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2024-04-17T10:48:36.695 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2024-04-17T10:48:36.695 DEBUG:teuthology.orchestra.run.smithi083:> return f 2024-04-17T10:48:36.695 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T10:48:36.695 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:48:36.695 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.0")) 2024-04-17T10:48:36.695 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-04-17T10:48:36.818 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.0.76725.asok 2024-04-17T10:48:36.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:48:36.830 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.76725.asok 2024-04-17T10:48:36.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:36.830 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.76725.asok status 2024-04-17T10:48:36.984 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:48:36.984 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2024-04-17T10:48:36.984 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-17T10:48:36.984 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-17T10:48:36.984 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "0", 2024-04-17T10:48:36.984 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-04-17T10:48:36.984 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-17T10:48:36.985 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "76725", 2024-04-17T10:48:36.985 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-04-17T10:48:36.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:48:36.985 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2024-04-17T10:48:36.985 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2024-04-17T10:48:36.985 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 8790, 2024-04-17T10:48:36.985 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2024-04-17T10:48:36.985 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-04-17T10:48:36.985 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-04-17T10:48:36.985 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 8790 2024-04-17T10:48:36.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:48:36.985 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-04-17T10:48:36.985 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-17T10:48:36.986 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:48:36.986 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4112045818 2024-04-17T10:48:36.986 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:48:36.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:48:36.986 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-04-17T10:48:36.986 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-17T10:48:36.986 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:48:36.986 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4112045818 2024-04-17T10:48:36.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:48:36.986 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.8790 192.168.0.1:0/4112045818", 2024-04-17T10:48:36.986 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.1:0/4112045818", 2024-04-17T10:48:36.986 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 1, 2024-04-17T10:48:36.987 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 69, 2024-04-17T10:48:36.987 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 114, 2024-04-17T10:48:36.987 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2024-04-17T10:48:36.987 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2024-04-17T10:48:36.987 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2024-04-17T10:48:36.987 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:48:36.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:36.998 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2024-04-17T10:48:36.998 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-17T10:48:37.005 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2024-04-17T10:48:37.005 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-17T10:48:37.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:37.005 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:48:37.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:37.062 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:48:37.062 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:37.062 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.0 2024-04-17T10:48:37.062 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:48:37.062 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:48:37.063 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:48:37.063 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:37.144 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.144 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.144 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162922 s, 64.4 MB/s 2024-04-17T10:48:37.161 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.161 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.161 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132552 s, 79.1 MB/s 2024-04-17T10:48:37.181 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.182 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.182 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018027 s, 58.2 MB/s 2024-04-17T10:48:37.196 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.196 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.196 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112477 s, 93.2 MB/s 2024-04-17T10:48:37.213 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.213 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.213 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141385 s, 74.2 MB/s 2024-04-17T10:48:37.231 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.231 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.231 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141406 s, 74.2 MB/s 2024-04-17T10:48:37.249 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.249 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.249 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157092 s, 66.7 MB/s 2024-04-17T10:48:37.263 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.263 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.264 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113473 s, 92.4 MB/s 2024-04-17T10:48:37.278 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.278 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.278 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117604 s, 89.2 MB/s 2024-04-17T10:48:37.294 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.294 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.294 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129821 s, 80.8 MB/s 2024-04-17T10:48:37.310 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.310 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.310 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135195 s, 77.6 MB/s 2024-04-17T10:48:37.326 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.326 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.326 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126855 s, 82.7 MB/s 2024-04-17T10:48:37.341 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.341 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.341 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012366 s, 84.8 MB/s 2024-04-17T10:48:37.359 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.359 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.359 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155016 s, 67.6 MB/s 2024-04-17T10:48:37.379 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.379 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.379 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170391 s, 61.5 MB/s 2024-04-17T10:48:37.397 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.397 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.397 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144592 s, 72.5 MB/s 2024-04-17T10:48:37.415 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.415 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.415 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147439 s, 71.1 MB/s 2024-04-17T10:48:37.433 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.433 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.434 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150445 s, 69.7 MB/s 2024-04-17T10:48:37.452 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.452 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.452 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150703 s, 69.6 MB/s 2024-04-17T10:48:37.470 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.470 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.470 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150606 s, 69.6 MB/s 2024-04-17T10:48:37.488 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.488 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.488 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142464 s, 73.6 MB/s 2024-04-17T10:48:37.506 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.506 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.506 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140846 s, 74.4 MB/s 2024-04-17T10:48:37.521 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.521 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.521 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115923 s, 90.5 MB/s 2024-04-17T10:48:37.537 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.538 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.538 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134834 s, 77.8 MB/s 2024-04-17T10:48:37.555 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.556 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.556 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150363 s, 69.7 MB/s 2024-04-17T10:48:37.571 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.571 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.572 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120572 s, 87.0 MB/s 2024-04-17T10:48:37.589 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.589 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.589 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142466 s, 73.6 MB/s 2024-04-17T10:48:37.604 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.604 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.604 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125469 s, 83.6 MB/s 2024-04-17T10:48:37.620 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.620 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.620 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127136 s, 82.5 MB/s 2024-04-17T10:48:37.636 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.636 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.636 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133811 s, 78.4 MB/s 2024-04-17T10:48:37.653 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.653 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.654 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142656 s, 73.5 MB/s 2024-04-17T10:48:37.670 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.670 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.670 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138467 s, 75.7 MB/s 2024-04-17T10:48:37.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:37.671 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:48:37.672 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:37.672 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.1 2024-04-17T10:48:37.672 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:48:37.672 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:48:37.672 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:48:37.672 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:37.736 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.737 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.737 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011984 s, 87.5 MB/s 2024-04-17T10:48:37.753 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.753 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.754 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139784 s, 75.0 MB/s 2024-04-17T10:48:37.772 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.772 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.772 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157242 s, 66.7 MB/s 2024-04-17T10:48:37.789 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.789 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.789 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013614 s, 77.0 MB/s 2024-04-17T10:48:37.805 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.806 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.806 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139288 s, 75.3 MB/s 2024-04-17T10:48:37.824 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.824 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.824 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153809 s, 68.2 MB/s 2024-04-17T10:48:37.841 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.841 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.842 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148317 s, 70.7 MB/s 2024-04-17T10:48:37.859 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.859 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.859 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145252 s, 72.2 MB/s 2024-04-17T10:48:37.876 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.877 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.877 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142643 s, 73.5 MB/s 2024-04-17T10:48:37.892 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.893 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.893 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128163 s, 81.8 MB/s 2024-04-17T10:48:37.909 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.909 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.909 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135379 s, 77.5 MB/s 2024-04-17T10:48:37.928 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.928 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.929 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159386 s, 65.8 MB/s 2024-04-17T10:48:37.946 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.946 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.946 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143886 s, 72.9 MB/s 2024-04-17T10:48:37.961 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.961 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.961 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119583 s, 87.7 MB/s 2024-04-17T10:48:37.978 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.978 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.978 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133618 s, 78.5 MB/s 2024-04-17T10:48:37.993 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:37.993 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:37.994 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122829 s, 85.4 MB/s 2024-04-17T10:48:38.010 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.010 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.010 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134112 s, 78.2 MB/s 2024-04-17T10:48:38.027 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.027 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.027 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140454 s, 74.7 MB/s 2024-04-17T10:48:38.045 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.045 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.045 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150738 s, 69.6 MB/s 2024-04-17T10:48:38.062 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.062 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.062 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140741 s, 74.5 MB/s 2024-04-17T10:48:38.077 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.077 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.077 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012565 s, 83.5 MB/s 2024-04-17T10:48:38.092 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.092 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.092 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118954 s, 88.1 MB/s 2024-04-17T10:48:38.110 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.110 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.110 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144799 s, 72.4 MB/s 2024-04-17T10:48:38.126 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.126 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.126 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110166 s, 95.2 MB/s 2024-04-17T10:48:38.138 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.138 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.138 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116166 s, 90.3 MB/s 2024-04-17T10:48:38.152 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.152 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.152 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111577 s, 94.0 MB/s 2024-04-17T10:48:38.166 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.166 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.166 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112871 s, 92.9 MB/s 2024-04-17T10:48:38.181 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.182 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.182 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012584 s, 83.3 MB/s 2024-04-17T10:48:38.197 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.197 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.197 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126544 s, 82.9 MB/s 2024-04-17T10:48:38.210 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.210 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.210 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103887 s, 101 MB/s 2024-04-17T10:48:38.226 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.226 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.227 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013439 s, 78.0 MB/s 2024-04-17T10:48:38.241 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.241 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.241 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123442 s, 84.9 MB/s 2024-04-17T10:48:38.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:38.243 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:48:38.243 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:38.243 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.2 2024-04-17T10:48:38.243 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:48:38.243 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:48:38.243 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:48:38.243 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:38.313 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.313 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.313 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146157 s, 71.7 MB/s 2024-04-17T10:48:38.329 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.329 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.329 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134193 s, 78.1 MB/s 2024-04-17T10:48:38.338 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:48:38.340 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:48:38.344 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.345 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.345 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126325 s, 83.0 MB/s 2024-04-17T10:48:38.361 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.362 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.362 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133273 s, 78.7 MB/s 2024-04-17T10:48:38.376 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.376 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.376 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119461 s, 87.8 MB/s 2024-04-17T10:48:38.391 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.391 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.391 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122261 s, 85.8 MB/s 2024-04-17T10:48:38.404 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.404 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.405 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110369 s, 95.0 MB/s 2024-04-17T10:48:38.422 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.422 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.422 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146471 s, 71.6 MB/s 2024-04-17T10:48:38.436 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.436 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.436 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109917 s, 95.4 MB/s 2024-04-17T10:48:38.450 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.450 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.450 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120191 s, 87.2 MB/s 2024-04-17T10:48:38.466 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.467 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.467 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137519 s, 76.2 MB/s 2024-04-17T10:48:38.479 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.479 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.480 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104036 s, 101 MB/s 2024-04-17T10:48:38.493 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.494 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.494 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113773 s, 92.2 MB/s 2024-04-17T10:48:38.508 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.508 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.508 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121493 s, 86.3 MB/s 2024-04-17T10:48:38.523 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.523 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.523 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118816 s, 88.3 MB/s 2024-04-17T10:48:38.537 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.538 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.538 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011858 s, 88.4 MB/s 2024-04-17T10:48:38.555 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.555 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.555 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137986 s, 76.0 MB/s 2024-04-17T10:48:38.568 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.568 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.568 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00979723 s, 107 MB/s 2024-04-17T10:48:38.584 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.584 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.584 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135519 s, 77.4 MB/s 2024-04-17T10:48:38.596 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.596 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.596 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00939382 s, 112 MB/s 2024-04-17T10:48:38.611 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.611 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.611 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122977 s, 85.3 MB/s 2024-04-17T10:48:38.623 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.623 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.623 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00948467 s, 111 MB/s 2024-04-17T10:48:38.635 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.635 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.635 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00942247 s, 111 MB/s 2024-04-17T10:48:38.651 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.651 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.651 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132722 s, 79.0 MB/s 2024-04-17T10:48:38.666 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.666 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.667 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122987 s, 85.3 MB/s 2024-04-17T10:48:38.680 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.680 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.680 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111942 s, 93.7 MB/s 2024-04-17T10:48:38.697 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.697 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.697 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134914 s, 77.7 MB/s 2024-04-17T10:48:38.713 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.713 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.713 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014018 s, 74.8 MB/s 2024-04-17T10:48:38.729 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.729 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.729 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128107 s, 81.9 MB/s 2024-04-17T10:48:38.742 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.742 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.743 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105914 s, 99.0 MB/s 2024-04-17T10:48:38.759 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.759 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.759 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134401 s, 78.0 MB/s 2024-04-17T10:48:38.774 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.774 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.774 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123131 s, 85.2 MB/s 2024-04-17T10:48:38.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:38.775 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:48:38.776 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:38.776 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.3 2024-04-17T10:48:38.776 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:48:38.776 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:48:38.776 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:48:38.776 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:38.843 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.844 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.844 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126885 s, 82.6 MB/s 2024-04-17T10:48:38.860 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.860 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.860 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133627 s, 78.5 MB/s 2024-04-17T10:48:38.877 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.877 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.877 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013988 s, 75.0 MB/s 2024-04-17T10:48:38.894 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.894 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.894 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139289 s, 75.3 MB/s 2024-04-17T10:48:38.908 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.908 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.908 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114546 s, 91.5 MB/s 2024-04-17T10:48:38.924 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.924 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.924 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133284 s, 78.7 MB/s 2024-04-17T10:48:38.943 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.943 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.943 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014082 s, 74.5 MB/s 2024-04-17T10:48:38.956 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.956 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.956 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114952 s, 91.2 MB/s 2024-04-17T10:48:38.970 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.970 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.970 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115763 s, 90.6 MB/s 2024-04-17T10:48:38.984 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:38.984 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:38.984 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114461 s, 91.6 MB/s 2024-04-17T10:48:39.000 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.000 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.000 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127821 s, 82.0 MB/s 2024-04-17T10:48:39.016 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.016 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.016 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134446 s, 78.0 MB/s 2024-04-17T10:48:39.033 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.033 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.033 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143808 s, 72.9 MB/s 2024-04-17T10:48:39.048 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.048 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.048 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115161 s, 91.1 MB/s 2024-04-17T10:48:39.064 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.064 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.065 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138465 s, 75.7 MB/s 2024-04-17T10:48:39.078 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.079 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.079 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113118 s, 92.7 MB/s 2024-04-17T10:48:39.094 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.094 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.094 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124072 s, 84.5 MB/s 2024-04-17T10:48:39.110 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.110 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.110 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130619 s, 80.3 MB/s 2024-04-17T10:48:39.127 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.127 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.127 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136875 s, 76.6 MB/s 2024-04-17T10:48:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135958 s, 77.1 MB/s 2024-04-17T10:48:39.160 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.160 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.160 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136971 s, 76.6 MB/s 2024-04-17T10:48:39.177 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.177 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.177 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137019 s, 76.5 MB/s 2024-04-17T10:48:39.193 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.193 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.193 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129911 s, 80.7 MB/s 2024-04-17T10:48:39.207 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.207 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.207 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110658 s, 94.8 MB/s 2024-04-17T10:48:39.223 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.223 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.223 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133122 s, 78.8 MB/s 2024-04-17T10:48:39.237 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.237 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.237 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107406 s, 97.6 MB/s 2024-04-17T10:48:39.250 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.250 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.250 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105821 s, 99.1 MB/s 2024-04-17T10:48:39.265 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.265 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.265 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118409 s, 88.6 MB/s 2024-04-17T10:48:39.282 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.282 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.282 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143523 s, 73.1 MB/s 2024-04-17T10:48:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145042 s, 72.3 MB/s 2024-04-17T10:48:39.317 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.317 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.318 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015206 s, 69.0 MB/s 2024-04-17T10:48:39.335 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.336 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.336 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146615 s, 71.5 MB/s 2024-04-17T10:48:39.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:39.337 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:48:39.337 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:39.337 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.4 2024-04-17T10:48:39.337 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:48:39.337 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:48:39.337 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:48:39.337 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:39.406 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.406 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.406 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125516 s, 83.5 MB/s 2024-04-17T10:48:39.419 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.419 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.419 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103214 s, 102 MB/s 2024-04-17T10:48:39.433 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.433 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.434 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120746 s, 86.8 MB/s 2024-04-17T10:48:39.448 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.448 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.448 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115644 s, 90.7 MB/s 2024-04-17T10:48:39.464 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.464 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.464 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134024 s, 78.2 MB/s 2024-04-17T10:48:39.477 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.477 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.477 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00989282 s, 106 MB/s 2024-04-17T10:48:39.490 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.491 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.491 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108361 s, 96.8 MB/s 2024-04-17T10:48:39.507 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.507 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.507 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132515 s, 79.1 MB/s 2024-04-17T10:48:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.521 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01121 s, 93.5 MB/s 2024-04-17T10:48:39.533 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.533 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.533 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100888 s, 104 MB/s 2024-04-17T10:48:39.548 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.549 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.549 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124632 s, 84.1 MB/s 2024-04-17T10:48:39.566 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.566 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.566 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148239 s, 70.7 MB/s 2024-04-17T10:48:39.583 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.583 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.583 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136914 s, 76.6 MB/s 2024-04-17T10:48:39.598 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.598 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.598 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115677 s, 90.6 MB/s 2024-04-17T10:48:39.613 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.614 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.614 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012344 s, 84.9 MB/s 2024-04-17T10:48:39.629 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.629 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.629 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124151 s, 84.5 MB/s 2024-04-17T10:48:39.645 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.645 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.645 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130168 s, 80.6 MB/s 2024-04-17T10:48:39.662 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.662 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.662 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141617 s, 74.0 MB/s 2024-04-17T10:48:39.680 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.681 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.681 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141224 s, 74.2 MB/s 2024-04-17T10:48:39.697 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.697 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.697 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01396 s, 75.1 MB/s 2024-04-17T10:48:39.714 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.714 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.714 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135347 s, 77.5 MB/s 2024-04-17T10:48:39.732 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.732 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.732 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149295 s, 70.2 MB/s 2024-04-17T10:48:39.748 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.748 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.748 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132285 s, 79.3 MB/s 2024-04-17T10:48:39.765 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.765 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.765 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127987 s, 81.9 MB/s 2024-04-17T10:48:39.783 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.783 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.783 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145443 s, 72.1 MB/s 2024-04-17T10:48:39.801 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.801 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.801 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014581 s, 71.9 MB/s 2024-04-17T10:48:39.818 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.819 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.819 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141653 s, 74.0 MB/s 2024-04-17T10:48:39.834 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.834 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.834 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127637 s, 82.2 MB/s 2024-04-17T10:48:39.851 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.851 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.851 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012836 s, 81.7 MB/s 2024-04-17T10:48:39.867 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.868 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.868 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135628 s, 77.3 MB/s 2024-04-17T10:48:39.886 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.886 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.886 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151979 s, 69.0 MB/s 2024-04-17T10:48:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144355 s, 72.6 MB/s 2024-04-17T10:48:39.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:39.906 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:48:39.906 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:39.906 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.5 2024-04-17T10:48:39.906 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:48:39.906 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:48:39.906 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:48:39.906 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:39.974 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.974 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.974 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123567 s, 84.9 MB/s 2024-04-17T10:48:39.990 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:39.990 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:39.990 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128982 s, 81.3 MB/s 2024-04-17T10:48:40.006 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.007 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.007 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141741 s, 74.0 MB/s 2024-04-17T10:48:40.023 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.024 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.024 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137569 s, 76.2 MB/s 2024-04-17T10:48:40.041 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.041 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.041 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144824 s, 72.4 MB/s 2024-04-17T10:48:40.056 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.057 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.057 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012118 s, 86.5 MB/s 2024-04-17T10:48:40.074 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.074 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.075 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151642 s, 69.1 MB/s 2024-04-17T10:48:40.087 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.087 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.087 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0096659 s, 108 MB/s 2024-04-17T10:48:40.102 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.102 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.102 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122367 s, 85.7 MB/s 2024-04-17T10:48:40.117 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.117 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.117 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123436 s, 84.9 MB/s 2024-04-17T10:48:40.134 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.134 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.134 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144022 s, 72.8 MB/s 2024-04-17T10:48:40.147 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.147 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.147 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101098 s, 104 MB/s 2024-04-17T10:48:40.162 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.162 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.162 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123977 s, 84.6 MB/s 2024-04-17T10:48:40.180 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.180 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.180 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137893 s, 76.0 MB/s 2024-04-17T10:48:40.192 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.192 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.192 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00998244 s, 105 MB/s 2024-04-17T10:48:40.207 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.207 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.207 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126856 s, 82.7 MB/s 2024-04-17T10:48:40.224 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.225 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.225 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141727 s, 74.0 MB/s 2024-04-17T10:48:40.240 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.240 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.240 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126706 s, 82.8 MB/s 2024-04-17T10:48:40.253 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.253 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.253 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010614 s, 98.8 MB/s 2024-04-17T10:48:40.269 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.270 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.270 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134077 s, 78.2 MB/s 2024-04-17T10:48:40.287 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.287 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.287 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142989 s, 73.3 MB/s 2024-04-17T10:48:40.298 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.299 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.299 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00904109 s, 116 MB/s 2024-04-17T10:48:40.314 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.314 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.314 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131506 s, 79.7 MB/s 2024-04-17T10:48:40.329 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.329 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.329 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123051 s, 85.2 MB/s 2024-04-17T10:48:40.345 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.345 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.345 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125791 s, 83.4 MB/s 2024-04-17T10:48:40.359 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.359 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.360 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118249 s, 88.7 MB/s 2024-04-17T10:48:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108943 s, 96.2 MB/s 2024-04-17T10:48:40.390 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.390 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.390 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143774 s, 72.9 MB/s 2024-04-17T10:48:40.404 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.404 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.404 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116291 s, 90.2 MB/s 2024-04-17T10:48:40.419 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.419 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.419 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122062 s, 85.9 MB/s 2024-04-17T10:48:40.433 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.433 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.433 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116493 s, 90.0 MB/s 2024-04-17T10:48:40.444 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.444 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.444 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848823 s, 124 MB/s 2024-04-17T10:48:40.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:40.446 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:48:40.446 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:40.446 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.6 2024-04-17T10:48:40.446 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:48:40.446 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:48:40.446 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:48:40.446 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:40.512 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.512 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.513 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126678 s, 82.8 MB/s 2024-04-17T10:48:40.525 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.525 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.525 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103239 s, 102 MB/s 2024-04-17T10:48:40.536 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.536 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.536 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00900723 s, 116 MB/s 2024-04-17T10:48:40.551 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.551 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.551 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118659 s, 88.4 MB/s 2024-04-17T10:48:40.566 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.566 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.566 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127297 s, 82.4 MB/s 2024-04-17T10:48:40.583 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.583 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.583 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140402 s, 74.7 MB/s 2024-04-17T10:48:40.597 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.597 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.597 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116926 s, 89.7 MB/s 2024-04-17T10:48:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121144 s, 86.6 MB/s 2024-04-17T10:48:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011828 s, 88.7 MB/s 2024-04-17T10:48:40.642 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.642 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.642 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128581 s, 81.5 MB/s 2024-04-17T10:48:40.653 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.653 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.653 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848585 s, 124 MB/s 2024-04-17T10:48:40.669 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.669 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.669 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131706 s, 79.6 MB/s 2024-04-17T10:48:40.684 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.684 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.684 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01302 s, 80.5 MB/s 2024-04-17T10:48:40.696 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00981063 s, 107 MB/s 2024-04-17T10:48:40.708 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.708 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.708 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00898827 s, 117 MB/s 2024-04-17T10:48:40.723 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.723 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.723 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120669 s, 86.9 MB/s 2024-04-17T10:48:40.737 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.737 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.737 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118714 s, 88.3 MB/s 2024-04-17T10:48:40.753 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.753 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.753 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133954 s, 78.3 MB/s 2024-04-17T10:48:40.769 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.769 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.769 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136414 s, 76.9 MB/s 2024-04-17T10:48:40.782 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.785 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.786 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104654 s, 100 MB/s 2024-04-17T10:48:40.795 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.795 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.795 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00986865 s, 106 MB/s 2024-04-17T10:48:40.809 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.810 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.810 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012006 s, 87.3 MB/s 2024-04-17T10:48:40.824 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.824 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.824 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118168 s, 88.7 MB/s 2024-04-17T10:48:40.838 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.838 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.838 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106943 s, 98.0 MB/s 2024-04-17T10:48:40.850 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.850 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.850 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00883765 s, 119 MB/s 2024-04-17T10:48:40.864 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.864 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.864 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105294 s, 99.6 MB/s 2024-04-17T10:48:40.880 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.880 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.880 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130065 s, 80.6 MB/s 2024-04-17T10:48:40.895 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.895 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.895 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119319 s, 87.9 MB/s 2024-04-17T10:48:40.910 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.910 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.910 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012262 s, 85.5 MB/s 2024-04-17T10:48:40.927 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.927 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.927 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135491 s, 77.4 MB/s 2024-04-17T10:48:40.941 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.941 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.941 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121704 s, 86.2 MB/s 2024-04-17T10:48:40.958 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:40.959 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:40.959 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141965 s, 73.9 MB/s 2024-04-17T10:48:40.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:40.961 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:48:40.961 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:40.961 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.7 2024-04-17T10:48:40.961 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:48:40.961 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:48:40.962 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:48:40.962 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:41.034 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.035 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.035 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117648 s, 89.1 MB/s 2024-04-17T10:48:41.050 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.050 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.050 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126007 s, 83.2 MB/s 2024-04-17T10:48:41.065 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.066 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.066 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123448 s, 84.9 MB/s 2024-04-17T10:48:41.081 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.081 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.081 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128578 s, 81.6 MB/s 2024-04-17T10:48:41.095 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.095 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.095 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116559 s, 90.0 MB/s 2024-04-17T10:48:41.110 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.110 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.110 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115159 s, 91.1 MB/s 2024-04-17T10:48:41.125 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.125 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.125 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128004 s, 81.9 MB/s 2024-04-17T10:48:41.139 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.139 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.139 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111408 s, 94.1 MB/s 2024-04-17T10:48:41.152 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.152 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.152 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101082 s, 104 MB/s 2024-04-17T10:48:41.167 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.167 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.167 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012529 s, 83.7 MB/s 2024-04-17T10:48:41.181 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.181 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.181 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108814 s, 96.4 MB/s 2024-04-17T10:48:41.198 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.198 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.198 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131034 s, 80.0 MB/s 2024-04-17T10:48:41.209 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.210 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.210 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00955961 s, 110 MB/s 2024-04-17T10:48:41.224 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.224 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.224 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01143 s, 91.7 MB/s 2024-04-17T10:48:41.239 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.239 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.240 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124915 s, 83.9 MB/s 2024-04-17T10:48:41.253 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112451 s, 93.2 MB/s 2024-04-17T10:48:41.268 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.269 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.269 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121748 s, 86.1 MB/s 2024-04-17T10:48:41.280 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.281 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.281 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00936395 s, 112 MB/s 2024-04-17T10:48:41.292 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.292 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.292 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00907952 s, 115 MB/s 2024-04-17T10:48:41.305 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.305 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.305 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00962674 s, 109 MB/s 2024-04-17T10:48:41.321 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.321 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.321 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135358 s, 77.5 MB/s 2024-04-17T10:48:41.337 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.337 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.337 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128259 s, 81.8 MB/s 2024-04-17T10:48:41.352 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.352 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.352 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012305 s, 85.2 MB/s 2024-04-17T10:48:41.368 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.368 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.368 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129508 s, 81.0 MB/s 2024-04-17T10:48:41.384 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.384 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.384 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127621 s, 82.2 MB/s 2024-04-17T10:48:41.400 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.400 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.400 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121417 s, 86.4 MB/s 2024-04-17T10:48:41.414 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.415 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.415 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113591 s, 92.3 MB/s 2024-04-17T10:48:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126588 s, 82.8 MB/s 2024-04-17T10:48:41.444 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.444 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.444 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119046 s, 88.1 MB/s 2024-04-17T10:48:41.459 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.459 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.459 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124577 s, 84.2 MB/s 2024-04-17T10:48:41.471 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.471 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.471 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00891928 s, 118 MB/s 2024-04-17T10:48:41.488 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.488 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.488 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143506 s, 73.1 MB/s 2024-04-17T10:48:41.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:41.489 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:48:41.489 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:41.489 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.8 2024-04-17T10:48:41.489 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:48:41.489 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:48:41.489 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:48:41.489 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:41.556 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.557 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.557 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011634 s, 90.1 MB/s 2024-04-17T10:48:41.569 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.569 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.569 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00942829 s, 111 MB/s 2024-04-17T10:48:41.582 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.583 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.583 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114315 s, 91.7 MB/s 2024-04-17T10:48:41.597 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.597 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.597 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121112 s, 86.6 MB/s 2024-04-17T10:48:41.612 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.612 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.613 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124664 s, 84.1 MB/s 2024-04-17T10:48:41.626 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.626 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.626 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111604 s, 94.0 MB/s 2024-04-17T10:48:41.638 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.638 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.638 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00899587 s, 117 MB/s 2024-04-17T10:48:41.653 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.653 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.653 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119696 s, 87.6 MB/s 2024-04-17T10:48:41.666 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.666 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.666 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111464 s, 94.1 MB/s 2024-04-17T10:48:41.681 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.681 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.681 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127528 s, 82.2 MB/s 2024-04-17T10:48:41.694 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.694 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.695 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113783 s, 92.2 MB/s 2024-04-17T10:48:41.708 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.709 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.709 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117082 s, 89.6 MB/s 2024-04-17T10:48:41.719 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.720 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.720 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861935 s, 122 MB/s 2024-04-17T10:48:41.733 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.733 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.733 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109339 s, 95.9 MB/s 2024-04-17T10:48:41.749 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.749 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.749 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134585 s, 77.9 MB/s 2024-04-17T10:48:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851023 s, 123 MB/s 2024-04-17T10:48:41.775 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.775 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.775 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012343 s, 85.0 MB/s 2024-04-17T10:48:41.789 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.789 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.790 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123625 s, 84.8 MB/s 2024-04-17T10:48:41.803 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.803 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.804 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111928 s, 93.7 MB/s 2024-04-17T10:48:41.816 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.816 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.816 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105566 s, 99.3 MB/s 2024-04-17T10:48:41.830 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.830 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.830 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011079 s, 94.6 MB/s 2024-04-17T10:48:41.842 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.843 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.843 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107415 s, 97.6 MB/s 2024-04-17T10:48:41.855 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.855 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.855 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0099836 s, 105 MB/s 2024-04-17T10:48:41.866 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.866 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.866 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00872019 s, 120 MB/s 2024-04-17T10:48:41.877 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.877 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.878 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00922923 s, 114 MB/s 2024-04-17T10:48:41.891 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.891 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.891 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105475 s, 99.4 MB/s 2024-04-17T10:48:41.904 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.904 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.904 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00935951 s, 112 MB/s 2024-04-17T10:48:41.918 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.919 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.919 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120619 s, 86.9 MB/s 2024-04-17T10:48:41.933 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.933 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.933 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125452 s, 83.6 MB/s 2024-04-17T10:48:41.946 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.946 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.946 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00938837 s, 112 MB/s 2024-04-17T10:48:41.960 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.960 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.960 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01202 s, 87.2 MB/s 2024-04-17T10:48:41.976 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:41.976 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:41.976 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013326 s, 78.7 MB/s 2024-04-17T10:48:41.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:41.977 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:48:41.977 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:41.977 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.9 2024-04-17T10:48:41.977 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:48:41.977 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:48:41.977 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:48:41.977 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:42.046 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.046 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.046 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127821 s, 82.0 MB/s 2024-04-17T10:48:42.063 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.063 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.063 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139715 s, 75.1 MB/s 2024-04-17T10:48:42.075 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.075 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.076 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101386 s, 103 MB/s 2024-04-17T10:48:42.088 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.088 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.088 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00961951 s, 109 MB/s 2024-04-17T10:48:42.102 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.102 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.102 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116156 s, 90.3 MB/s 2024-04-17T10:48:42.117 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.117 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.117 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125574 s, 83.5 MB/s 2024-04-17T10:48:42.133 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.133 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.133 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133741 s, 78.4 MB/s 2024-04-17T10:48:42.147 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.148 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.148 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122204 s, 85.8 MB/s 2024-04-17T10:48:42.163 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.163 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.163 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127701 s, 82.1 MB/s 2024-04-17T10:48:42.176 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.176 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.176 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106587 s, 98.4 MB/s 2024-04-17T10:48:42.190 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.190 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.190 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116986 s, 89.6 MB/s 2024-04-17T10:48:42.203 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.203 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.203 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00998322 s, 105 MB/s 2024-04-17T10:48:42.218 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.218 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.218 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125346 s, 83.7 MB/s 2024-04-17T10:48:42.234 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.234 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.234 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135903 s, 77.2 MB/s 2024-04-17T10:48:42.247 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.248 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.248 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107293 s, 97.7 MB/s 2024-04-17T10:48:42.262 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.262 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.262 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116623 s, 89.9 MB/s 2024-04-17T10:48:42.276 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.276 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.276 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120226 s, 87.2 MB/s 2024-04-17T10:48:42.291 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.291 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.291 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126169 s, 83.1 MB/s 2024-04-17T10:48:42.306 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.306 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.306 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012313 s, 85.2 MB/s 2024-04-17T10:48:42.322 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.322 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.322 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012981 s, 80.8 MB/s 2024-04-17T10:48:42.337 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.337 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.337 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127607 s, 82.2 MB/s 2024-04-17T10:48:42.349 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.349 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.349 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00886438 s, 118 MB/s 2024-04-17T10:48:42.363 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.363 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.363 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120952 s, 86.7 MB/s 2024-04-17T10:48:42.375 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.375 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.375 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00924307 s, 113 MB/s 2024-04-17T10:48:42.388 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.389 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.389 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105253 s, 99.6 MB/s 2024-04-17T10:48:42.403 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.403 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.403 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115386 s, 90.9 MB/s 2024-04-17T10:48:42.419 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.419 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.419 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135094 s, 77.6 MB/s 2024-04-17T10:48:42.435 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.435 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.435 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130732 s, 80.2 MB/s 2024-04-17T10:48:42.449 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.449 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.450 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121887 s, 86.0 MB/s 2024-04-17T10:48:42.467 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.467 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.467 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014764 s, 71.0 MB/s 2024-04-17T10:48:42.482 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.482 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.482 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121808 s, 86.1 MB/s 2024-04-17T10:48:42.499 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.499 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.499 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138205 s, 75.9 MB/s 2024-04-17T10:48:42.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:42.500 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:48:42.501 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:42.501 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.10 2024-04-17T10:48:42.501 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:48:42.501 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:48:42.501 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:48:42.501 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:42.570 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.570 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.570 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143676 s, 73.0 MB/s 2024-04-17T10:48:42.583 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.583 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.583 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010236 s, 102 MB/s 2024-04-17T10:48:42.598 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.598 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.598 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123627 s, 84.8 MB/s 2024-04-17T10:48:42.612 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.612 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.612 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110818 s, 94.6 MB/s 2024-04-17T10:48:42.630 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.630 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.630 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127896 s, 82.0 MB/s 2024-04-17T10:48:42.643 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.643 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.643 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010195 s, 103 MB/s 2024-04-17T10:48:42.662 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.662 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.662 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150138 s, 69.8 MB/s 2024-04-17T10:48:42.678 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.678 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.678 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121807 s, 86.1 MB/s 2024-04-17T10:48:42.695 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.695 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.695 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138024 s, 76.0 MB/s 2024-04-17T10:48:42.711 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.711 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.711 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130961 s, 80.1 MB/s 2024-04-17T10:48:42.726 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.726 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.726 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113802 s, 92.1 MB/s 2024-04-17T10:48:42.742 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.742 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.742 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132337 s, 79.2 MB/s 2024-04-17T10:48:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138746 s, 75.6 MB/s 2024-04-17T10:48:42.775 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.775 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.776 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130632 s, 80.3 MB/s 2024-04-17T10:48:42.789 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.789 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.789 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109484 s, 95.8 MB/s 2024-04-17T10:48:42.802 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.803 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.803 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106364 s, 98.6 MB/s 2024-04-17T10:48:42.815 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.815 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.815 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00999539 s, 105 MB/s 2024-04-17T10:48:42.829 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.829 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.829 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105955 s, 99.0 MB/s 2024-04-17T10:48:42.846 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.846 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.846 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138067 s, 75.9 MB/s 2024-04-17T10:48:42.859 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.859 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.859 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103245 s, 102 MB/s 2024-04-17T10:48:42.874 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.874 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.874 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127834 s, 82.0 MB/s 2024-04-17T10:48:42.890 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.891 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.891 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013213 s, 79.4 MB/s 2024-04-17T10:48:42.907 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.907 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.907 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125736 s, 83.4 MB/s 2024-04-17T10:48:42.920 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.920 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.920 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101329 s, 103 MB/s 2024-04-17T10:48:42.935 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.935 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.935 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122209 s, 85.8 MB/s 2024-04-17T10:48:42.951 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.951 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.951 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127544 s, 82.2 MB/s 2024-04-17T10:48:42.964 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.964 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.965 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105355 s, 99.5 MB/s 2024-04-17T10:48:42.977 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.977 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.977 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0099763 s, 105 MB/s 2024-04-17T10:48:42.994 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:42.994 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:42.994 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140639 s, 74.6 MB/s 2024-04-17T10:48:43.007 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.007 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.007 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100003 s, 105 MB/s 2024-04-17T10:48:43.023 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.023 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.023 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132155 s, 79.3 MB/s 2024-04-17T10:48:43.039 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.039 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.039 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126648 s, 82.8 MB/s 2024-04-17T10:48:43.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:43.041 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:48:43.041 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:43.041 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.11 2024-04-17T10:48:43.041 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:48:43.041 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:48:43.042 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:48:43.042 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133751 s, 78.4 MB/s 2024-04-17T10:48:43.129 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.129 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.130 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126547 s, 82.9 MB/s 2024-04-17T10:48:43.144 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.144 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.144 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011963 s, 87.7 MB/s 2024-04-17T10:48:43.160 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.160 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.161 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129832 s, 80.8 MB/s 2024-04-17T10:48:43.176 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.176 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.176 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128193 s, 81.8 MB/s 2024-04-17T10:48:43.193 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.193 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.194 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146121 s, 71.8 MB/s 2024-04-17T10:48:43.206 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.206 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.206 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100127 s, 105 MB/s 2024-04-17T10:48:43.224 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.224 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.224 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144434 s, 72.6 MB/s 2024-04-17T10:48:43.240 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.240 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.240 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133407 s, 78.6 MB/s 2024-04-17T10:48:43.255 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.255 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.256 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132285 s, 79.3 MB/s 2024-04-17T10:48:43.272 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.272 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.272 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133066 s, 78.8 MB/s 2024-04-17T10:48:43.284 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.284 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.284 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0097733 s, 107 MB/s 2024-04-17T10:48:43.298 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.298 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.298 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107296 s, 97.7 MB/s 2024-04-17T10:48:43.311 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.311 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.311 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00986071 s, 106 MB/s 2024-04-17T10:48:43.329 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.329 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.329 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149589 s, 70.1 MB/s 2024-04-17T10:48:43.343 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.344 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.344 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123195 s, 85.1 MB/s 2024-04-17T10:48:43.360 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.360 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.360 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133133 s, 78.8 MB/s 2024-04-17T10:48:43.377 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.377 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.377 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013872 s, 75.6 MB/s 2024-04-17T10:48:43.392 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.392 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.392 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124362 s, 84.3 MB/s 2024-04-17T10:48:43.406 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.406 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.406 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112102 s, 93.5 MB/s 2024-04-17T10:48:43.424 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.424 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.424 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151822 s, 69.1 MB/s 2024-04-17T10:48:43.439 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.439 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.439 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012318 s, 85.1 MB/s 2024-04-17T10:48:43.452 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.452 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.453 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103639 s, 101 MB/s 2024-04-17T10:48:43.469 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.469 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.469 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134156 s, 78.2 MB/s 2024-04-17T10:48:43.485 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.485 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.485 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134504 s, 78.0 MB/s 2024-04-17T10:48:43.510 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.510 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.510 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834809 s, 126 MB/s 2024-04-17T10:48:43.524 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.524 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.524 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121915 s, 86.0 MB/s 2024-04-17T10:48:43.540 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.540 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.540 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127541 s, 82.2 MB/s 2024-04-17T10:48:43.555 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.556 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.556 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129343 s, 81.1 MB/s 2024-04-17T10:48:43.569 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.570 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.570 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113869 s, 92.1 MB/s 2024-04-17T10:48:43.584 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.584 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.584 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122884 s, 85.3 MB/s 2024-04-17T10:48:43.599 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.599 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.599 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118878 s, 88.2 MB/s 2024-04-17T10:48:43.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:43.600 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:48:43.600 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:43.600 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.12 2024-04-17T10:48:43.600 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:48:43.601 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:48:43.601 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:48:43.601 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:43.669 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.669 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.670 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112595 s, 93.1 MB/s 2024-04-17T10:48:43.683 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.684 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.684 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114709 s, 91.4 MB/s 2024-04-17T10:48:43.698 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.698 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.698 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114553 s, 91.5 MB/s 2024-04-17T10:48:43.712 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.712 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.712 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111651 s, 93.9 MB/s 2024-04-17T10:48:43.728 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.729 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.729 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136085 s, 77.1 MB/s 2024-04-17T10:48:43.743 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.743 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.744 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121707 s, 86.2 MB/s 2024-04-17T10:48:43.756 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.756 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.756 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101341 s, 103 MB/s 2024-04-17T10:48:43.770 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.770 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.770 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110229 s, 95.1 MB/s 2024-04-17T10:48:43.786 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.786 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.786 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132876 s, 78.9 MB/s 2024-04-17T10:48:43.798 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.798 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.799 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00926216 s, 113 MB/s 2024-04-17T10:48:43.815 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.815 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.815 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136633 s, 76.7 MB/s 2024-04-17T10:48:43.830 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.830 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.830 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123673 s, 84.8 MB/s 2024-04-17T10:48:43.845 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.845 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.845 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120955 s, 86.7 MB/s 2024-04-17T10:48:43.859 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.859 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.860 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01068 s, 98.2 MB/s 2024-04-17T10:48:43.874 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.874 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.874 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115892 s, 90.5 MB/s 2024-04-17T10:48:43.889 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.889 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.890 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128924 s, 81.3 MB/s 2024-04-17T10:48:43.902 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.902 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.902 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00984566 s, 107 MB/s 2024-04-17T10:48:43.918 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.919 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.919 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136637 s, 76.7 MB/s 2024-04-17T10:48:43.933 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.934 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.934 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123291 s, 85.0 MB/s 2024-04-17T10:48:43.948 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.948 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.948 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118957 s, 88.1 MB/s 2024-04-17T10:48:43.964 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.964 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.964 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130843 s, 80.1 MB/s 2024-04-17T10:48:43.977 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.977 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.977 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100365 s, 104 MB/s 2024-04-17T10:48:43.995 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:43.995 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:43.995 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014616 s, 71.7 MB/s 2024-04-17T10:48:44.011 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.011 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.011 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130055 s, 80.6 MB/s 2024-04-17T10:48:44.027 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.027 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.027 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132537 s, 79.1 MB/s 2024-04-17T10:48:44.043 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.043 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.043 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125258 s, 83.7 MB/s 2024-04-17T10:48:44.058 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.059 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.059 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128081 s, 81.9 MB/s 2024-04-17T10:48:44.074 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.074 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.074 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119792 s, 87.5 MB/s 2024-04-17T10:48:44.088 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.088 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.088 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119889 s, 87.5 MB/s 2024-04-17T10:48:44.103 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.104 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.104 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122251 s, 85.8 MB/s 2024-04-17T10:48:44.119 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.119 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.119 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126638 s, 82.8 MB/s 2024-04-17T10:48:44.134 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.134 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.135 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126871 s, 82.6 MB/s 2024-04-17T10:48:44.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:44.136 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:48:44.136 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:44.136 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.13 2024-04-17T10:48:44.136 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:48:44.136 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:48:44.136 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:48:44.136 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:44.198 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.198 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.198 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00948547 s, 111 MB/s 2024-04-17T10:48:44.213 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.213 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.214 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129534 s, 81.0 MB/s 2024-04-17T10:48:44.226 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.226 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.226 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00968165 s, 108 MB/s 2024-04-17T10:48:44.240 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.240 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.240 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111909 s, 93.7 MB/s 2024-04-17T10:48:44.255 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.255 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.255 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124558 s, 84.2 MB/s 2024-04-17T10:48:44.269 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.269 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.269 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108208 s, 96.9 MB/s 2024-04-17T10:48:44.280 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.281 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.281 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00941829 s, 111 MB/s 2024-04-17T10:48:44.292 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.293 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.293 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00924084 s, 113 MB/s 2024-04-17T10:48:44.308 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.308 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.308 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127846 s, 82.0 MB/s 2024-04-17T10:48:44.320 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.320 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.320 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00919261 s, 114 MB/s 2024-04-17T10:48:44.332 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.332 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.332 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0090071 s, 116 MB/s 2024-04-17T10:48:44.346 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.346 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.346 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107482 s, 97.6 MB/s 2024-04-17T10:48:44.357 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.357 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.357 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00873743 s, 120 MB/s 2024-04-17T10:48:44.371 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.371 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.371 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115278 s, 91.0 MB/s 2024-04-17T10:48:44.386 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.386 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.386 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123442 s, 84.9 MB/s 2024-04-17T10:48:44.400 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.400 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.400 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011936 s, 87.8 MB/s 2024-04-17T10:48:44.415 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.415 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.415 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120345 s, 87.1 MB/s 2024-04-17T10:48:44.430 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.430 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.430 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125473 s, 83.6 MB/s 2024-04-17T10:48:44.444 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.444 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.444 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110001 s, 95.3 MB/s 2024-04-17T10:48:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124479 s, 84.2 MB/s 2024-04-17T10:48:44.475 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.475 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.476 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139802 s, 75.0 MB/s 2024-04-17T10:48:44.490 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.490 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.490 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112221 s, 93.4 MB/s 2024-04-17T10:48:44.543 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.544 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.544 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195548 s, 53.6 MB/s 2024-04-17T10:48:44.558 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.559 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.559 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121398 s, 86.4 MB/s 2024-04-17T10:48:44.574 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.574 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.575 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132139 s, 79.4 MB/s 2024-04-17T10:48:44.590 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.590 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.590 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012635 s, 83.0 MB/s 2024-04-17T10:48:44.603 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.603 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.603 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00967079 s, 108 MB/s 2024-04-17T10:48:44.616 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.616 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.616 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107041 s, 98.0 MB/s 2024-04-17T10:48:44.631 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.632 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.632 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119936 s, 87.4 MB/s 2024-04-17T10:48:44.646 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.647 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.647 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119563 s, 87.7 MB/s 2024-04-17T10:48:44.659 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.660 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.660 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00946552 s, 111 MB/s 2024-04-17T10:48:44.676 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.676 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.676 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119753 s, 87.6 MB/s 2024-04-17T10:48:44.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:44.677 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:48:44.677 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:44.677 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.14 2024-04-17T10:48:44.678 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:48:44.678 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:48:44.678 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:48:44.678 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:44.742 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00983702 s, 107 MB/s 2024-04-17T10:48:44.759 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.759 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.759 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131344 s, 79.8 MB/s 2024-04-17T10:48:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113598 s, 92.3 MB/s 2024-04-17T10:48:44.787 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.787 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.787 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113311 s, 92.5 MB/s 2024-04-17T10:48:44.801 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.802 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.802 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115199 s, 91.0 MB/s 2024-04-17T10:48:44.816 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.816 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.816 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119655 s, 87.6 MB/s 2024-04-17T10:48:44.828 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.828 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.828 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0089322 s, 117 MB/s 2024-04-17T10:48:44.839 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.840 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.840 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00858661 s, 122 MB/s 2024-04-17T10:48:44.855 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.855 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.855 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123782 s, 84.7 MB/s 2024-04-17T10:48:44.871 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.871 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.871 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130875 s, 80.1 MB/s 2024-04-17T10:48:44.885 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.885 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.885 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110371 s, 95.0 MB/s 2024-04-17T10:48:44.897 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.897 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.897 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830487 s, 126 MB/s 2024-04-17T10:48:44.911 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.911 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.911 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115215 s, 91.0 MB/s 2024-04-17T10:48:44.926 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.926 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.926 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122183 s, 85.8 MB/s 2024-04-17T10:48:44.939 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.940 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.940 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107478 s, 97.6 MB/s 2024-04-17T10:48:44.951 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.951 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.952 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0089459 s, 117 MB/s 2024-04-17T10:48:44.967 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.967 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.967 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122902 s, 85.3 MB/s 2024-04-17T10:48:44.980 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.981 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.981 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110092 s, 95.2 MB/s 2024-04-17T10:48:44.995 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:44.995 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:44.995 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117749 s, 89.1 MB/s 2024-04-17T10:48:45.010 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.010 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.010 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115098 s, 91.1 MB/s 2024-04-17T10:48:45.025 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.025 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.025 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122744 s, 85.4 MB/s 2024-04-17T10:48:45.044 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.044 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.044 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149537 s, 70.1 MB/s 2024-04-17T10:48:45.059 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.059 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.059 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011707 s, 89.6 MB/s 2024-04-17T10:48:45.076 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.076 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.076 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127942 s, 82.0 MB/s 2024-04-17T10:48:45.094 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.094 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.094 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130057 s, 80.6 MB/s 2024-04-17T10:48:45.111 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.112 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.112 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134432 s, 78.0 MB/s 2024-04-17T10:48:45.128 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.129 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.129 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126359 s, 83.0 MB/s 2024-04-17T10:48:45.146 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.146 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.146 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123884 s, 84.6 MB/s 2024-04-17T10:48:45.163 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.163 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.163 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125673 s, 83.4 MB/s 2024-04-17T10:48:45.181 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.181 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.181 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135357 s, 77.5 MB/s 2024-04-17T10:48:45.197 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.197 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.197 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127002 s, 82.6 MB/s 2024-04-17T10:48:45.213 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.213 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.213 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135519 s, 77.4 MB/s 2024-04-17T10:48:45.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:45.214 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:48:45.215 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:45.215 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.15 2024-04-17T10:48:45.215 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:48:45.215 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:48:45.215 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:48:45.215 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:45.283 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.283 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.284 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128654 s, 81.5 MB/s 2024-04-17T10:48:45.299 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.299 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.299 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136115 s, 77.0 MB/s 2024-04-17T10:48:45.315 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.316 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.316 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013019 s, 80.5 MB/s 2024-04-17T10:48:45.331 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.331 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.332 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124755 s, 84.1 MB/s 2024-04-17T10:48:45.345 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.346 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.346 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112549 s, 93.2 MB/s 2024-04-17T10:48:45.358 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.359 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.359 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0099615 s, 105 MB/s 2024-04-17T10:48:45.373 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.373 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.374 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118513 s, 88.5 MB/s 2024-04-17T10:48:45.388 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.388 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.388 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112631 s, 93.1 MB/s 2024-04-17T10:48:45.404 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.404 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.404 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134423 s, 78.0 MB/s 2024-04-17T10:48:45.420 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.420 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.420 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013122 s, 79.9 MB/s 2024-04-17T10:48:45.433 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.434 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.434 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105969 s, 99.0 MB/s 2024-04-17T10:48:45.447 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.447 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.447 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109981 s, 95.3 MB/s 2024-04-17T10:48:45.462 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.462 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.462 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128753 s, 81.4 MB/s 2024-04-17T10:48:45.477 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.477 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.477 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120567 s, 87.0 MB/s 2024-04-17T10:48:45.493 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.493 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.493 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124794 s, 84.0 MB/s 2024-04-17T10:48:45.519 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.519 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.519 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118243 s, 88.7 MB/s 2024-04-17T10:48:45.532 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.532 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.532 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011457 s, 91.5 MB/s 2024-04-17T10:48:45.545 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.545 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.545 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105668 s, 99.2 MB/s 2024-04-17T10:48:45.557 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.557 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.557 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01001 s, 105 MB/s 2024-04-17T10:48:45.567 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.567 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.567 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770115 s, 136 MB/s 2024-04-17T10:48:45.577 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.577 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.577 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076139 s, 138 MB/s 2024-04-17T10:48:45.587 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.587 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.587 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776517 s, 135 MB/s 2024-04-17T10:48:45.600 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.600 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.601 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111896 s, 93.7 MB/s 2024-04-17T10:48:45.611 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.611 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.611 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00863825 s, 121 MB/s 2024-04-17T10:48:45.622 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.622 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.622 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854096 s, 123 MB/s 2024-04-17T10:48:45.636 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.636 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.636 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112475 s, 93.2 MB/s 2024-04-17T10:48:45.649 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.649 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.649 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103349 s, 101 MB/s 2024-04-17T10:48:45.663 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.663 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.663 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012243 s, 85.6 MB/s 2024-04-17T10:48:45.677 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.677 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.678 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113459 s, 92.4 MB/s 2024-04-17T10:48:45.688 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.688 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.688 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840659 s, 125 MB/s 2024-04-17T10:48:45.699 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.699 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.699 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815662 s, 129 MB/s 2024-04-17T10:48:45.713 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.714 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.714 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123099 s, 85.2 MB/s 2024-04-17T10:48:45.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:45.715 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:48:45.715 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:45.715 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.16 2024-04-17T10:48:45.715 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:48:45.715 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:48:45.715 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:48:45.715 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:45.783 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.784 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.784 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114164 s, 91.8 MB/s 2024-04-17T10:48:45.797 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.798 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.798 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011348 s, 92.4 MB/s 2024-04-17T10:48:45.810 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.810 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.810 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00965293 s, 109 MB/s 2024-04-17T10:48:45.826 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.826 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.826 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125555 s, 83.5 MB/s 2024-04-17T10:48:45.840 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.840 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.840 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119792 s, 87.5 MB/s 2024-04-17T10:48:45.855 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.855 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.855 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127644 s, 82.1 MB/s 2024-04-17T10:48:45.869 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.869 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.869 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109035 s, 96.2 MB/s 2024-04-17T10:48:45.883 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.883 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.884 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012148 s, 86.3 MB/s 2024-04-17T10:48:45.897 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.897 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.898 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108447 s, 96.7 MB/s 2024-04-17T10:48:45.909 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.909 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.909 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00906508 s, 116 MB/s 2024-04-17T10:48:45.923 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.923 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.923 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112377 s, 93.3 MB/s 2024-04-17T10:48:45.938 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.938 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.938 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121937 s, 86.0 MB/s 2024-04-17T10:48:45.949 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.949 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.949 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00913594 s, 115 MB/s 2024-04-17T10:48:45.962 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.962 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.962 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00992836 s, 106 MB/s 2024-04-17T10:48:45.977 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.977 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.978 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126278 s, 83.0 MB/s 2024-04-17T10:48:45.992 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:45.992 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:45.992 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124215 s, 84.4 MB/s 2024-04-17T10:48:46.005 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.005 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.005 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00973098 s, 108 MB/s 2024-04-17T10:48:46.020 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.020 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.020 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124696 s, 84.1 MB/s 2024-04-17T10:48:46.034 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.034 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.034 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116289 s, 90.2 MB/s 2024-04-17T10:48:46.049 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.049 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.049 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117283 s, 89.4 MB/s 2024-04-17T10:48:46.062 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.062 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.063 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010872 s, 96.4 MB/s 2024-04-17T10:48:46.077 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.077 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.077 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117552 s, 89.2 MB/s 2024-04-17T10:48:46.089 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.089 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.089 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00915552 s, 115 MB/s 2024-04-17T10:48:46.103 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.103 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.103 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120101 s, 87.3 MB/s 2024-04-17T10:48:46.117 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.117 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.118 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111358 s, 94.2 MB/s 2024-04-17T10:48:46.129 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.129 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.129 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00892304 s, 118 MB/s 2024-04-17T10:48:46.146 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.146 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.146 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128203 s, 81.8 MB/s 2024-04-17T10:48:46.163 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.163 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.164 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134623 s, 77.9 MB/s 2024-04-17T10:48:46.181 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.182 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.182 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144799 s, 72.4 MB/s 2024-04-17T10:48:46.197 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.197 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.197 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111212 s, 94.3 MB/s 2024-04-17T10:48:46.211 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.212 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.212 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105241 s, 99.6 MB/s 2024-04-17T10:48:46.227 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.227 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.228 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129989 s, 80.7 MB/s 2024-04-17T10:48:46.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:46.229 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:48:46.229 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:46.229 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.17 2024-04-17T10:48:46.229 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:48:46.229 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:48:46.229 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:48:46.229 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:46.294 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.294 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.295 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00930166 s, 113 MB/s 2024-04-17T10:48:46.308 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.309 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.309 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103662 s, 101 MB/s 2024-04-17T10:48:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124676 s, 84.1 MB/s 2024-04-17T10:48:46.339 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.339 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.339 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128148 s, 81.8 MB/s 2024-04-17T10:48:46.356 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.356 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.356 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013212 s, 79.4 MB/s 2024-04-17T10:48:46.372 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.372 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.372 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121701 s, 86.2 MB/s 2024-04-17T10:48:46.386 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.387 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.387 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107415 s, 97.6 MB/s 2024-04-17T10:48:46.401 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.402 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.402 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00984737 s, 106 MB/s 2024-04-17T10:48:46.415 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.415 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.415 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120712 s, 86.9 MB/s 2024-04-17T10:48:46.429 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.429 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.429 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112647 s, 93.1 MB/s 2024-04-17T10:48:46.444 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.445 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.445 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131067 s, 80.0 MB/s 2024-04-17T10:48:46.456 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.456 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.456 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086714 s, 121 MB/s 2024-04-17T10:48:46.472 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.472 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.472 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132203 s, 79.3 MB/s 2024-04-17T10:48:46.487 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.488 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.488 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128627 s, 81.5 MB/s 2024-04-17T10:48:46.508 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.508 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.508 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177594 s, 59.0 MB/s 2024-04-17T10:48:46.538 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.538 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.538 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122134 s, 85.9 MB/s 2024-04-17T10:48:46.549 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.549 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.549 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00856638 s, 122 MB/s 2024-04-17T10:48:46.563 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.563 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.563 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114238 s, 91.8 MB/s 2024-04-17T10:48:46.574 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.574 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.574 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834523 s, 126 MB/s 2024-04-17T10:48:46.584 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.585 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.585 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00857014 s, 122 MB/s 2024-04-17T10:48:46.600 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.600 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.600 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117215 s, 89.5 MB/s 2024-04-17T10:48:46.614 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.615 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.615 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125201 s, 83.8 MB/s 2024-04-17T10:48:46.628 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.628 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.628 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114829 s, 91.3 MB/s 2024-04-17T10:48:46.642 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.643 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.643 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121225 s, 86.5 MB/s 2024-04-17T10:48:46.656 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.656 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.656 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011077 s, 94.7 MB/s 2024-04-17T10:48:46.668 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.668 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.668 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00853971 s, 123 MB/s 2024-04-17T10:48:46.680 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.680 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.680 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0098063 s, 107 MB/s 2024-04-17T10:48:46.695 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.695 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.695 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122457 s, 85.6 MB/s 2024-04-17T10:48:46.710 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.710 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.711 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118363 s, 88.6 MB/s 2024-04-17T10:48:46.720 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.721 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.721 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829803 s, 126 MB/s 2024-04-17T10:48:46.733 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.733 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.733 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00899126 s, 117 MB/s 2024-04-17T10:48:46.746 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.746 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.746 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109702 s, 95.6 MB/s 2024-04-17T10:48:46.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:46.747 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:48:46.747 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:46.748 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.18 2024-04-17T10:48:46.748 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:48:46.748 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:48:46.748 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:48:46.748 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:46.820 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.820 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.820 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133599 s, 78.5 MB/s 2024-04-17T10:48:46.837 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.837 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.837 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127941 s, 82.0 MB/s 2024-04-17T10:48:46.853 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.854 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.854 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124858 s, 84.0 MB/s 2024-04-17T10:48:46.870 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.870 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.870 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129655 s, 80.9 MB/s 2024-04-17T10:48:46.885 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.885 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.885 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117879 s, 89.0 MB/s 2024-04-17T10:48:46.899 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.900 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.900 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118498 s, 88.5 MB/s 2024-04-17T10:48:46.915 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.915 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.915 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114105 s, 91.9 MB/s 2024-04-17T10:48:46.928 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.928 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.928 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114303 s, 91.7 MB/s 2024-04-17T10:48:46.941 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.941 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.941 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104222 s, 101 MB/s 2024-04-17T10:48:46.956 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.956 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.957 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123169 s, 85.1 MB/s 2024-04-17T10:48:46.973 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.973 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.973 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136301 s, 76.9 MB/s 2024-04-17T10:48:46.986 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:46.986 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:46.986 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102967 s, 102 MB/s 2024-04-17T10:48:47.003 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.003 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.003 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143473 s, 73.1 MB/s 2024-04-17T10:48:47.020 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.020 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.020 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014074 s, 74.5 MB/s 2024-04-17T10:48:47.033 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.033 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.033 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102623 s, 102 MB/s 2024-04-17T10:48:47.048 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.048 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.048 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122023 s, 85.9 MB/s 2024-04-17T10:48:47.064 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.064 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.064 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126283 s, 83.0 MB/s 2024-04-17T10:48:47.079 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.080 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.080 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130899 s, 80.1 MB/s 2024-04-17T10:48:47.093 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.094 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.094 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104868 s, 100 MB/s 2024-04-17T10:48:47.104 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.104 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.105 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00895066 s, 117 MB/s 2024-04-17T10:48:47.119 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.119 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.119 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122648 s, 85.5 MB/s 2024-04-17T10:48:47.134 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.134 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.134 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124067 s, 84.5 MB/s 2024-04-17T10:48:47.147 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.147 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.147 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100286 s, 105 MB/s 2024-04-17T10:48:47.164 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.164 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.164 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141864 s, 73.9 MB/s 2024-04-17T10:48:47.176 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.176 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.176 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00921607 s, 114 MB/s 2024-04-17T10:48:47.193 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.193 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.193 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138109 s, 75.9 MB/s 2024-04-17T10:48:47.204 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.204 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.204 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00868602 s, 121 MB/s 2024-04-17T10:48:47.216 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.216 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.216 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00939061 s, 112 MB/s 2024-04-17T10:48:47.230 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.231 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.231 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120548 s, 87.0 MB/s 2024-04-17T10:48:47.242 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.243 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.243 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00931976 s, 113 MB/s 2024-04-17T10:48:47.258 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.258 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.258 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125177 s, 83.8 MB/s 2024-04-17T10:48:47.272 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.272 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.272 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117599 s, 89.2 MB/s 2024-04-17T10:48:47.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:47.274 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:48:47.274 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:47.274 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.19 2024-04-17T10:48:47.274 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:48:47.274 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:48:47.274 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:48:47.274 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:47.338 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.338 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.338 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104504 s, 100 MB/s 2024-04-17T10:48:47.347 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.347 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.348 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787979 s, 133 MB/s 2024-04-17T10:48:47.361 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.361 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.361 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116358 s, 90.1 MB/s 2024-04-17T10:48:47.371 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.371 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.371 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709751 s, 148 MB/s 2024-04-17T10:48:47.384 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.384 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.384 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115099 s, 91.1 MB/s 2024-04-17T10:48:47.398 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.398 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.398 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113253 s, 92.6 MB/s 2024-04-17T10:48:47.409 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.410 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.410 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00965366 s, 109 MB/s 2024-04-17T10:48:47.421 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.421 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.421 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00944457 s, 111 MB/s 2024-04-17T10:48:47.437 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.437 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.437 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133115 s, 78.8 MB/s 2024-04-17T10:48:47.452 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.452 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.452 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123324 s, 85.0 MB/s 2024-04-17T10:48:47.462 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.462 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.462 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737888 s, 142 MB/s 2024-04-17T10:48:47.474 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.474 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.474 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100937 s, 104 MB/s 2024-04-17T10:48:47.484 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.484 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.484 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752527 s, 139 MB/s 2024-04-17T10:48:47.493 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.493 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.493 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725493 s, 145 MB/s 2024-04-17T10:48:47.512 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.512 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.512 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161725 s, 64.8 MB/s 2024-04-17T10:48:47.535 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.535 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.536 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713174 s, 147 MB/s 2024-04-17T10:48:47.547 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.547 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.548 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00965225 s, 109 MB/s 2024-04-17T10:48:47.559 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.559 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.559 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00970363 s, 108 MB/s 2024-04-17T10:48:47.572 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.572 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.572 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108284 s, 96.8 MB/s 2024-04-17T10:48:47.586 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.586 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.586 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115158 s, 91.1 MB/s 2024-04-17T10:48:47.595 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.595 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.595 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751376 s, 140 MB/s 2024-04-17T10:48:47.607 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.607 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.607 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00954593 s, 110 MB/s 2024-04-17T10:48:47.618 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.619 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.619 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0098056 s, 107 MB/s 2024-04-17T10:48:47.633 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.633 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.633 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119272 s, 87.9 MB/s 2024-04-17T10:48:47.646 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.646 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.646 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114048 s, 91.9 MB/s 2024-04-17T10:48:47.657 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.657 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.657 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801722 s, 131 MB/s 2024-04-17T10:48:47.670 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.670 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.670 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118907 s, 88.2 MB/s 2024-04-17T10:48:47.680 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.680 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.680 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797352 s, 132 MB/s 2024-04-17T10:48:47.697 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.697 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.698 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116941 s, 89.7 MB/s 2024-04-17T10:48:47.708 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.708 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.708 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114351 s, 91.7 MB/s 2024-04-17T10:48:47.718 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.718 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.718 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754339 s, 139 MB/s 2024-04-17T10:48:47.734 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.734 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.735 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134005 s, 78.2 MB/s 2024-04-17T10:48:47.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:47.736 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:48:47.736 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:47.736 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.20 2024-04-17T10:48:47.736 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:48:47.736 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:48:47.736 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:48:47.736 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:47.802 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.802 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.802 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108951 s, 96.2 MB/s 2024-04-17T10:48:47.816 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.816 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.817 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117711 s, 89.1 MB/s 2024-04-17T10:48:47.830 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.830 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.830 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116659 s, 89.9 MB/s 2024-04-17T10:48:47.840 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.840 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.840 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820372 s, 128 MB/s 2024-04-17T10:48:47.852 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.853 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.853 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102302 s, 102 MB/s 2024-04-17T10:48:47.866 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.866 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.866 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01122 s, 93.5 MB/s 2024-04-17T10:48:47.875 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.875 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.875 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728301 s, 144 MB/s 2024-04-17T10:48:47.889 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.889 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.889 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108616 s, 96.5 MB/s 2024-04-17T10:48:47.900 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.901 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.901 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010415 s, 101 MB/s 2024-04-17T10:48:47.914 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.914 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.914 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111296 s, 94.2 MB/s 2024-04-17T10:48:47.924 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.924 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.924 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771671 s, 136 MB/s 2024-04-17T10:48:47.942 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.942 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.942 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127085 s, 82.5 MB/s 2024-04-17T10:48:47.950 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.950 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.951 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102264 s, 103 MB/s 2024-04-17T10:48:47.963 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.963 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.963 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100645 s, 104 MB/s 2024-04-17T10:48:47.976 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.976 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.976 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109942 s, 95.4 MB/s 2024-04-17T10:48:47.989 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:47.989 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:47.989 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105038 s, 99.8 MB/s 2024-04-17T10:48:48.002 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.003 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.003 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119995 s, 87.4 MB/s 2024-04-17T10:48:48.014 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.014 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.014 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880718 s, 119 MB/s 2024-04-17T10:48:48.023 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.023 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.023 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747951 s, 140 MB/s 2024-04-17T10:48:48.033 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.033 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.033 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746144 s, 141 MB/s 2024-04-17T10:48:48.042 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.043 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.043 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778071 s, 135 MB/s 2024-04-17T10:48:48.052 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.052 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.052 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074068 s, 142 MB/s 2024-04-17T10:48:48.061 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.062 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.062 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758951 s, 138 MB/s 2024-04-17T10:48:48.071 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.071 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.071 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742297 s, 141 MB/s 2024-04-17T10:48:48.084 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.084 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.084 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105647 s, 99.3 MB/s 2024-04-17T10:48:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131009 s, 80.0 MB/s 2024-04-17T10:48:48.108 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.108 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.108 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729168 s, 144 MB/s 2024-04-17T10:48:48.122 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.122 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.123 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118791 s, 88.3 MB/s 2024-04-17T10:48:48.136 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.137 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.137 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118949 s, 88.2 MB/s 2024-04-17T10:48:48.149 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.149 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.149 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101542 s, 103 MB/s 2024-04-17T10:48:48.164 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.164 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.164 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127117 s, 82.5 MB/s 2024-04-17T10:48:48.176 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.176 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.176 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010471 s, 100 MB/s 2024-04-17T10:48:48.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:48.177 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:48:48.178 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:48.178 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.21 2024-04-17T10:48:48.178 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:48:48.178 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:48:48.178 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:48:48.178 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:48.238 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.238 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.238 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819335 s, 128 MB/s 2024-04-17T10:48:48.254 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.254 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.254 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131786 s, 79.6 MB/s 2024-04-17T10:48:48.267 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.267 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.267 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108259 s, 96.9 MB/s 2024-04-17T10:48:48.281 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.282 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.282 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123154 s, 85.1 MB/s 2024-04-17T10:48:48.296 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.296 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.296 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120136 s, 87.3 MB/s 2024-04-17T10:48:48.309 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.310 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.310 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114227 s, 91.8 MB/s 2024-04-17T10:48:48.326 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.326 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.326 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137638 s, 76.2 MB/s 2024-04-17T10:48:48.341 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.341 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.341 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123825 s, 84.7 MB/s 2024-04-17T10:48:48.356 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.356 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.357 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01296 s, 80.9 MB/s 2024-04-17T10:48:48.371 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.371 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.371 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128785 s, 81.4 MB/s 2024-04-17T10:48:48.387 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.388 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.388 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013594 s, 77.1 MB/s 2024-04-17T10:48:48.402 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.402 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.402 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121249 s, 86.5 MB/s 2024-04-17T10:48:48.418 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.418 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.418 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130627 s, 80.3 MB/s 2024-04-17T10:48:48.436 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.436 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.436 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149302 s, 70.2 MB/s 2024-04-17T10:48:48.452 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.452 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.452 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133901 s, 78.3 MB/s 2024-04-17T10:48:48.468 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.468 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.468 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133454 s, 78.6 MB/s 2024-04-17T10:48:48.483 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.483 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.483 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127793 s, 82.1 MB/s 2024-04-17T10:48:48.499 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.499 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.499 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127235 s, 82.4 MB/s 2024-04-17T10:48:48.517 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.517 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.517 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154835 s, 67.7 MB/s 2024-04-17T10:48:48.546 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.549 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.549 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129962 s, 80.7 MB/s 2024-04-17T10:48:48.561 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.561 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.561 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117771 s, 89.0 MB/s 2024-04-17T10:48:48.578 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.578 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.578 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013708 s, 76.5 MB/s 2024-04-17T10:48:48.595 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.595 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.595 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143826 s, 72.9 MB/s 2024-04-17T10:48:48.611 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.611 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.611 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135029 s, 77.7 MB/s 2024-04-17T10:48:48.626 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.627 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.627 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130993 s, 80.0 MB/s 2024-04-17T10:48:48.644 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.644 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.644 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138474 s, 75.7 MB/s 2024-04-17T10:48:48.661 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.661 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.661 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013768 s, 76.2 MB/s 2024-04-17T10:48:48.678 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.678 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.678 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142001 s, 73.8 MB/s 2024-04-17T10:48:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108198 s, 96.9 MB/s 2024-04-17T10:48:48.707 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.707 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.707 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123583 s, 84.8 MB/s 2024-04-17T10:48:48.720 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.720 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.720 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101317 s, 103 MB/s 2024-04-17T10:48:48.735 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.735 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.736 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130106 s, 80.6 MB/s 2024-04-17T10:48:48.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:48.739 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:48:48.739 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:48.740 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.22 2024-04-17T10:48:48.740 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:48:48.740 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:48:48.740 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:48:48.740 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:48.807 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.807 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.807 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134375 s, 78.0 MB/s 2024-04-17T10:48:48.823 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.823 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.823 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133526 s, 78.5 MB/s 2024-04-17T10:48:48.839 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.839 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.839 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135204 s, 77.6 MB/s 2024-04-17T10:48:48.864 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.864 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.864 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149045 s, 70.4 MB/s 2024-04-17T10:48:48.872 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.873 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.873 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125415 s, 83.6 MB/s 2024-04-17T10:48:48.888 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.888 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.888 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120672 s, 86.9 MB/s 2024-04-17T10:48:48.903 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.903 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.903 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125451 s, 83.6 MB/s 2024-04-17T10:48:48.918 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.918 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.918 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120857 s, 86.8 MB/s 2024-04-17T10:48:48.934 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.935 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.935 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135162 s, 77.6 MB/s 2024-04-17T10:48:48.949 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.949 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.949 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120365 s, 87.1 MB/s 2024-04-17T10:48:48.965 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.965 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.965 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132495 s, 79.1 MB/s 2024-04-17T10:48:48.981 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.981 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.981 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012819 s, 81.8 MB/s 2024-04-17T10:48:48.997 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:48.998 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:48.998 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134676 s, 77.9 MB/s 2024-04-17T10:48:49.014 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.014 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.014 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131825 s, 79.5 MB/s 2024-04-17T10:48:49.030 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.030 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.031 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013213 s, 79.4 MB/s 2024-04-17T10:48:49.045 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.045 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.045 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113138 s, 92.7 MB/s 2024-04-17T10:48:49.062 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.062 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.062 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134901 s, 77.7 MB/s 2024-04-17T10:48:49.078 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.078 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.078 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126273 s, 83.0 MB/s 2024-04-17T10:48:49.097 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.097 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.097 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145348 s, 72.1 MB/s 2024-04-17T10:48:49.113 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.113 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.113 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131168 s, 79.9 MB/s 2024-04-17T10:48:49.127 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.127 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.127 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106843 s, 98.1 MB/s 2024-04-17T10:48:49.143 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.143 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.143 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125243 s, 83.7 MB/s 2024-04-17T10:48:49.162 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.162 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.162 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130951 s, 80.1 MB/s 2024-04-17T10:48:49.174 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.174 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.174 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124488 s, 84.2 MB/s 2024-04-17T10:48:49.190 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.190 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.190 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120256 s, 87.2 MB/s 2024-04-17T10:48:49.207 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.207 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.207 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134276 s, 78.1 MB/s 2024-04-17T10:48:49.222 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.222 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.222 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124244 s, 84.4 MB/s 2024-04-17T10:48:49.237 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.237 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.237 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118704 s, 88.3 MB/s 2024-04-17T10:48:49.251 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.251 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.251 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112263 s, 93.4 MB/s 2024-04-17T10:48:49.266 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.266 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.266 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122967 s, 85.3 MB/s 2024-04-17T10:48:49.283 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.283 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.283 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137015 s, 76.5 MB/s 2024-04-17T10:48:49.298 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.298 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.299 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124854 s, 84.0 MB/s 2024-04-17T10:48:49.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:49.300 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:48:49.300 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:49.300 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.23 2024-04-17T10:48:49.300 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:48:49.300 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:48:49.300 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:48:49.300 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:49.373 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.373 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.373 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139915 s, 74.9 MB/s 2024-04-17T10:48:49.390 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.390 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.390 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136512 s, 76.8 MB/s 2024-04-17T10:48:49.406 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.407 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.407 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130724 s, 80.2 MB/s 2024-04-17T10:48:49.420 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.420 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.420 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010533 s, 99.6 MB/s 2024-04-17T10:48:49.436 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.436 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.437 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137728 s, 76.1 MB/s 2024-04-17T10:48:49.452 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.453 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.453 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131781 s, 79.6 MB/s 2024-04-17T10:48:49.468 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.468 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.468 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126371 s, 83.0 MB/s 2024-04-17T10:48:49.482 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.482 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.482 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111265 s, 94.2 MB/s 2024-04-17T10:48:49.497 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.497 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.497 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118992 s, 88.1 MB/s 2024-04-17T10:48:49.513 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.514 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.514 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141113 s, 74.3 MB/s 2024-04-17T10:48:49.552 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.552 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.553 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138791 s, 75.6 MB/s 2024-04-17T10:48:49.567 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.567 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.567 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011447 s, 91.6 MB/s 2024-04-17T10:48:49.582 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.582 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.583 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125802 s, 83.4 MB/s 2024-04-17T10:48:49.598 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132557 s, 79.1 MB/s 2024-04-17T10:48:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013973 s, 75.0 MB/s 2024-04-17T10:48:49.634 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.634 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.634 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141689 s, 74.0 MB/s 2024-04-17T10:48:49.653 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.653 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.654 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148858 s, 70.4 MB/s 2024-04-17T10:48:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142058 s, 73.8 MB/s 2024-04-17T10:48:49.689 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.689 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.689 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135181 s, 77.6 MB/s 2024-04-17T10:48:49.709 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.709 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.709 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155997 s, 67.2 MB/s 2024-04-17T10:48:49.726 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.727 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.727 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133797 s, 78.4 MB/s 2024-04-17T10:48:49.744 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.744 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.745 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132859 s, 78.9 MB/s 2024-04-17T10:48:49.761 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.761 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129994 s, 80.7 MB/s 2024-04-17T10:48:49.779 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.780 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.780 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139634 s, 75.1 MB/s 2024-04-17T10:48:49.796 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.796 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.797 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128544 s, 81.6 MB/s 2024-04-17T10:48:49.813 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.813 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.813 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129911 s, 80.7 MB/s 2024-04-17T10:48:49.831 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.831 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.831 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148305 s, 70.7 MB/s 2024-04-17T10:48:49.848 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.848 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.848 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128283 s, 81.7 MB/s 2024-04-17T10:48:49.862 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.862 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.862 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110181 s, 95.2 MB/s 2024-04-17T10:48:49.877 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.877 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.877 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115815 s, 90.5 MB/s 2024-04-17T10:48:49.893 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.893 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.893 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126484 s, 82.9 MB/s 2024-04-17T10:48:49.910 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.910 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.910 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114149 s, 91.9 MB/s 2024-04-17T10:48:49.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:49.911 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:48:49.911 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:49.911 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.24 2024-04-17T10:48:49.911 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:48:49.911 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:48:49.911 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:48:49.912 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:49.983 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.983 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.983 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121571 s, 86.3 MB/s 2024-04-17T10:48:49.998 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:49.998 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:49.998 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107386 s, 97.6 MB/s 2024-04-17T10:48:50.012 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.012 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.012 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00981238 s, 107 MB/s 2024-04-17T10:48:50.025 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.025 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.025 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00975798 s, 107 MB/s 2024-04-17T10:48:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.041 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.041 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124457 s, 84.3 MB/s 2024-04-17T10:48:50.055 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.055 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.055 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116919 s, 89.7 MB/s 2024-04-17T10:48:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137622 s, 76.2 MB/s 2024-04-17T10:48:50.084 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.085 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.085 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0097013 s, 108 MB/s 2024-04-17T10:48:50.099 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.099 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.099 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112612 s, 93.1 MB/s 2024-04-17T10:48:50.113 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.113 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.113 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111738 s, 93.8 MB/s 2024-04-17T10:48:50.127 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.127 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.127 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108991 s, 96.2 MB/s 2024-04-17T10:48:50.140 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.140 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.141 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106213 s, 98.7 MB/s 2024-04-17T10:48:50.153 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.154 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.154 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00997172 s, 105 MB/s 2024-04-17T10:48:50.165 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.166 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.166 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00912254 s, 115 MB/s 2024-04-17T10:48:50.177 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.177 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.177 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835158 s, 126 MB/s 2024-04-17T10:48:50.190 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.190 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.190 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00964108 s, 109 MB/s 2024-04-17T10:48:50.203 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.203 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.203 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108832 s, 96.3 MB/s 2024-04-17T10:48:50.218 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.218 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.218 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119219 s, 88.0 MB/s 2024-04-17T10:48:50.230 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.230 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.230 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00893485 s, 117 MB/s 2024-04-17T10:48:50.246 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.246 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.246 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126223 s, 83.1 MB/s 2024-04-17T10:48:50.261 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.261 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.261 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120463 s, 87.0 MB/s 2024-04-17T10:48:50.273 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.273 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.273 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00893116 s, 117 MB/s 2024-04-17T10:48:50.286 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.286 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.286 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105051 s, 99.8 MB/s 2024-04-17T10:48:50.298 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.298 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.298 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850768 s, 123 MB/s 2024-04-17T10:48:50.312 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.312 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.313 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115369 s, 90.9 MB/s 2024-04-17T10:48:50.329 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.329 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.329 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134868 s, 77.7 MB/s 2024-04-17T10:48:50.341 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.341 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.341 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00990702 s, 106 MB/s 2024-04-17T10:48:50.355 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.355 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.355 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116706 s, 89.8 MB/s 2024-04-17T10:48:50.371 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.371 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.371 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125523 s, 83.5 MB/s 2024-04-17T10:48:50.382 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.382 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.382 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00875459 s, 120 MB/s 2024-04-17T10:48:50.396 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.396 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.396 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114241 s, 91.8 MB/s 2024-04-17T10:48:50.408 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.408 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.408 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00941178 s, 111 MB/s 2024-04-17T10:48:50.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:50.409 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:48:50.409 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:50.409 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.25 2024-04-17T10:48:50.410 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:48:50.410 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:48:50.410 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:48:50.410 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:50.472 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.472 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.472 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0098686 s, 106 MB/s 2024-04-17T10:48:50.486 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.486 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.487 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121149 s, 86.6 MB/s 2024-04-17T10:48:50.501 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.501 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.501 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114157 s, 91.9 MB/s 2024-04-17T10:48:50.520 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.521 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.521 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173169 s, 60.6 MB/s 2024-04-17T10:48:50.544 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.545 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.545 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750533 s, 140 MB/s 2024-04-17T10:48:50.559 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.559 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.559 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126375 s, 83.0 MB/s 2024-04-17T10:48:50.572 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.572 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.572 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109699 s, 95.6 MB/s 2024-04-17T10:48:50.585 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.585 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.585 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104542 s, 100 MB/s 2024-04-17T10:48:50.595 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.595 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.595 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766148 s, 137 MB/s 2024-04-17T10:48:50.610 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.610 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.610 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128204 s, 81.8 MB/s 2024-04-17T10:48:50.622 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.622 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.623 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104366 s, 100 MB/s 2024-04-17T10:48:50.636 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.636 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.636 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112995 s, 92.8 MB/s 2024-04-17T10:48:50.649 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.649 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.649 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106477 s, 98.5 MB/s 2024-04-17T10:48:50.663 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.664 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.664 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124633 s, 84.1 MB/s 2024-04-17T10:48:50.674 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.674 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.674 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816435 s, 128 MB/s 2024-04-17T10:48:50.688 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.688 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.688 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119062 s, 88.1 MB/s 2024-04-17T10:48:50.701 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.701 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.701 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103863 s, 101 MB/s 2024-04-17T10:48:50.715 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.716 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.716 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126427 s, 82.9 MB/s 2024-04-17T10:48:50.729 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.729 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.729 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106387 s, 98.6 MB/s 2024-04-17T10:48:50.741 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.741 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.741 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104125 s, 101 MB/s 2024-04-17T10:48:50.755 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.755 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.755 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108312 s, 96.8 MB/s 2024-04-17T10:48:50.766 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.766 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.766 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845352 s, 124 MB/s 2024-04-17T10:48:50.776 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.777 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.777 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845512 s, 124 MB/s 2024-04-17T10:48:50.791 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.791 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.792 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121419 s, 86.4 MB/s 2024-04-17T10:48:50.802 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.802 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.802 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855892 s, 123 MB/s 2024-04-17T10:48:50.815 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.815 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.815 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106804 s, 98.2 MB/s 2024-04-17T10:48:50.830 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.830 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.831 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126833 s, 82.7 MB/s 2024-04-17T10:48:50.845 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.845 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.845 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124298 s, 84.4 MB/s 2024-04-17T10:48:50.860 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.860 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.860 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122355 s, 85.7 MB/s 2024-04-17T10:48:50.870 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.871 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.871 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837532 s, 125 MB/s 2024-04-17T10:48:50.881 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.881 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.882 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847987 s, 124 MB/s 2024-04-17T10:48:50.896 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.896 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.896 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121696 s, 86.2 MB/s 2024-04-17T10:48:50.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:50.897 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:48:50.897 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:50.897 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.26 2024-04-17T10:48:50.898 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:48:50.898 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:48:50.898 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:48:50.898 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:50.968 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:50.968 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:50.968 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142083 s, 73.8 MB/s 2024-04-17T10:48:50.984 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.018 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.018 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130884 s, 80.1 MB/s 2024-04-17T10:48:51.019 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.019 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.019 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112515 s, 93.2 MB/s 2024-04-17T10:48:51.019 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.019 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.019 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00976005 s, 107 MB/s 2024-04-17T10:48:51.028 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.028 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.028 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140011 s, 74.9 MB/s 2024-04-17T10:48:51.044 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.044 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.045 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132472 s, 79.2 MB/s 2024-04-17T10:48:51.062 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.062 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.062 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143889 s, 72.9 MB/s 2024-04-17T10:48:51.075 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.075 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.075 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108553 s, 96.6 MB/s 2024-04-17T10:48:51.091 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.091 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.091 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126928 s, 82.6 MB/s 2024-04-17T10:48:51.107 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.108 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.108 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137024 s, 76.5 MB/s 2024-04-17T10:48:51.122 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.122 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.122 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118485 s, 88.5 MB/s 2024-04-17T10:48:51.138 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.138 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.138 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124792 s, 84.0 MB/s 2024-04-17T10:48:51.154 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.154 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.154 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133038 s, 78.8 MB/s 2024-04-17T10:48:51.169 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.169 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.169 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122224 s, 85.8 MB/s 2024-04-17T10:48:51.184 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.184 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.184 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117705 s, 89.1 MB/s 2024-04-17T10:48:51.200 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.200 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.200 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131948 s, 79.5 MB/s 2024-04-17T10:48:51.214 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.214 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.214 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111686 s, 93.9 MB/s 2024-04-17T10:48:51.229 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.229 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.229 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012374 s, 84.7 MB/s 2024-04-17T10:48:51.246 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.246 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.246 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135406 s, 77.4 MB/s 2024-04-17T10:48:51.259 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.259 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.259 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00986115 s, 106 MB/s 2024-04-17T10:48:51.271 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.271 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.271 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101625 s, 103 MB/s 2024-04-17T10:48:51.286 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.286 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.286 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118738 s, 88.3 MB/s 2024-04-17T10:48:51.299 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.299 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.299 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101817 s, 103 MB/s 2024-04-17T10:48:51.315 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.315 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.315 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135191 s, 77.6 MB/s 2024-04-17T10:48:51.330 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.331 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.331 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125395 s, 83.6 MB/s 2024-04-17T10:48:51.343 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.343 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.344 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100312 s, 105 MB/s 2024-04-17T10:48:51.359 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.359 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.359 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123464 s, 84.9 MB/s 2024-04-17T10:48:51.376 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.376 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.377 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138365 s, 75.8 MB/s 2024-04-17T10:48:51.393 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.393 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.394 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126096 s, 83.2 MB/s 2024-04-17T10:48:51.405 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.405 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.405 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102521 s, 102 MB/s 2024-04-17T10:48:51.418 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.418 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.418 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104279 s, 101 MB/s 2024-04-17T10:48:51.431 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.432 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.432 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010473 s, 100 MB/s 2024-04-17T10:48:51.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:51.433 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:48:51.433 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:51.433 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.27 2024-04-17T10:48:51.433 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:48:51.433 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:48:51.433 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:48:51.433 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:51.507 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.507 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.507 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149029 s, 70.4 MB/s 2024-04-17T10:48:51.519 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.519 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.519 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00907775 s, 116 MB/s 2024-04-17T10:48:51.557 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.557 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.557 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011091 s, 94.5 MB/s 2024-04-17T10:48:51.572 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.572 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.572 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126509 s, 82.9 MB/s 2024-04-17T10:48:51.589 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.589 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.589 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138318 s, 75.8 MB/s 2024-04-17T10:48:51.602 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.602 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.603 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113932 s, 92.0 MB/s 2024-04-17T10:48:51.617 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.618 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.618 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012359 s, 84.8 MB/s 2024-04-17T10:48:51.630 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.630 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.630 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00912864 s, 115 MB/s 2024-04-17T10:48:51.641 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.641 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.641 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854341 s, 123 MB/s 2024-04-17T10:48:51.654 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.654 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.654 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100988 s, 104 MB/s 2024-04-17T10:48:51.667 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.667 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.667 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107567 s, 97.5 MB/s 2024-04-17T10:48:51.681 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.681 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.681 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109655 s, 95.6 MB/s 2024-04-17T10:48:51.693 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.693 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.693 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100426 s, 104 MB/s 2024-04-17T10:48:51.707 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.707 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.707 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011292 s, 92.9 MB/s 2024-04-17T10:48:51.722 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.722 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.722 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127868 s, 82.0 MB/s 2024-04-17T10:48:51.735 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.736 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.736 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108741 s, 96.4 MB/s 2024-04-17T10:48:51.750 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.750 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.750 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121046 s, 86.6 MB/s 2024-04-17T10:48:51.764 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.765 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.765 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112137 s, 93.5 MB/s 2024-04-17T10:48:51.778 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.778 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.778 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108892 s, 96.3 MB/s 2024-04-17T10:48:51.794 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.794 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.794 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012853 s, 81.6 MB/s 2024-04-17T10:48:51.806 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.806 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.806 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00942028 s, 111 MB/s 2024-04-17T10:48:51.819 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.820 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.820 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111387 s, 94.1 MB/s 2024-04-17T10:48:51.835 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.836 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.836 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131699 s, 79.6 MB/s 2024-04-17T10:48:51.847 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.847 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.847 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865819 s, 121 MB/s 2024-04-17T10:48:51.862 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.862 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.862 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124172 s, 84.4 MB/s 2024-04-17T10:48:51.876 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.876 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.876 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111133 s, 94.4 MB/s 2024-04-17T10:48:51.891 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.891 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.891 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124171 s, 84.4 MB/s 2024-04-17T10:48:51.905 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.905 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.905 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114541 s, 91.5 MB/s 2024-04-17T10:48:51.920 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.920 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.920 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125628 s, 83.5 MB/s 2024-04-17T10:48:51.934 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.934 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.934 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111254 s, 94.3 MB/s 2024-04-17T10:48:51.950 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.950 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.950 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136062 s, 77.1 MB/s 2024-04-17T10:48:51.966 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:51.966 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:51.966 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132833 s, 78.9 MB/s 2024-04-17T10:48:51.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:51.968 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:48:51.968 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:51.968 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.28 2024-04-17T10:48:51.968 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:48:51.968 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:48:51.968 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:48:51.968 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:52.033 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.033 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.033 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142633 s, 73.5 MB/s 2024-04-17T10:48:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124181 s, 84.4 MB/s 2024-04-17T10:48:52.063 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.063 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.063 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012611 s, 83.1 MB/s 2024-04-17T10:48:52.078 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.078 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.079 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01255 s, 83.6 MB/s 2024-04-17T10:48:52.095 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.095 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.095 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133126 s, 78.8 MB/s 2024-04-17T10:48:52.111 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.111 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.111 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137668 s, 76.2 MB/s 2024-04-17T10:48:52.126 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.126 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.126 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121102 s, 86.6 MB/s 2024-04-17T10:48:52.140 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.140 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.140 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113504 s, 92.4 MB/s 2024-04-17T10:48:52.154 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.154 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.154 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113119 s, 92.7 MB/s 2024-04-17T10:48:52.169 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.169 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.169 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128866 s, 81.4 MB/s 2024-04-17T10:48:52.178 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.178 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.178 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715546 s, 147 MB/s 2024-04-17T10:48:52.190 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.190 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.191 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100386 s, 104 MB/s 2024-04-17T10:48:52.202 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.202 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.202 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00963492 s, 109 MB/s 2024-04-17T10:48:52.216 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.216 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.216 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118256 s, 88.7 MB/s 2024-04-17T10:48:52.229 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.229 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.229 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110635 s, 94.8 MB/s 2024-04-17T10:48:52.242 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.243 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.243 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750753 s, 140 MB/s 2024-04-17T10:48:52.253 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.253 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.253 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115193 s, 91.0 MB/s 2024-04-17T10:48:52.262 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.262 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.262 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713886 s, 147 MB/s 2024-04-17T10:48:52.276 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.276 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.276 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116907 s, 89.7 MB/s 2024-04-17T10:48:52.288 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.288 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.288 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102807 s, 102 MB/s 2024-04-17T10:48:52.301 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.301 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.301 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111453 s, 94.1 MB/s 2024-04-17T10:48:52.314 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.314 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.314 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103275 s, 102 MB/s 2024-04-17T10:48:52.327 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.327 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.327 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115034 s, 91.2 MB/s 2024-04-17T10:48:52.339 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.339 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.339 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00989959 s, 106 MB/s 2024-04-17T10:48:52.353 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.353 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.353 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118028 s, 88.8 MB/s 2024-04-17T10:48:52.367 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.367 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.367 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120921 s, 86.7 MB/s 2024-04-17T10:48:52.380 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.380 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.380 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110338 s, 95.0 MB/s 2024-04-17T10:48:52.390 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.390 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.390 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761755 s, 138 MB/s 2024-04-17T10:48:52.402 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.402 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.402 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101011 s, 104 MB/s 2024-04-17T10:48:52.417 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.417 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.417 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012925 s, 81.1 MB/s 2024-04-17T10:48:52.426 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.426 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.427 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720323 s, 146 MB/s 2024-04-17T10:48:52.441 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.441 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.441 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122477 s, 85.6 MB/s 2024-04-17T10:48:52.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:52.442 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:48:52.442 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:52.442 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.29 2024-04-17T10:48:52.442 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:48:52.446 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:48:52.446 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:48:52.446 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:52.513 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.513 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.513 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108312 s, 96.8 MB/s 2024-04-17T10:48:52.529 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.529 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.529 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128535 s, 81.6 MB/s 2024-04-17T10:48:52.568 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.568 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.568 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123295 s, 85.0 MB/s 2024-04-17T10:48:52.581 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.581 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.582 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106974 s, 98.0 MB/s 2024-04-17T10:48:52.592 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.592 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.592 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819571 s, 128 MB/s 2024-04-17T10:48:52.606 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.606 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.606 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011174 s, 93.8 MB/s 2024-04-17T10:48:52.621 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.621 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.621 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120201 s, 87.2 MB/s 2024-04-17T10:48:52.634 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.635 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.635 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108069 s, 97.0 MB/s 2024-04-17T10:48:52.648 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.648 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.648 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110918 s, 94.5 MB/s 2024-04-17T10:48:52.663 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.663 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.663 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125018 s, 83.9 MB/s 2024-04-17T10:48:52.675 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.675 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.676 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00960911 s, 109 MB/s 2024-04-17T10:48:52.690 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.690 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.690 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121858 s, 86.0 MB/s 2024-04-17T10:48:52.701 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.701 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.702 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861715 s, 122 MB/s 2024-04-17T10:48:52.713 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.713 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.713 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00906688 s, 116 MB/s 2024-04-17T10:48:52.727 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.728 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.728 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121984 s, 86.0 MB/s 2024-04-17T10:48:52.739 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.739 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.739 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00881471 s, 119 MB/s 2024-04-17T10:48:52.753 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.753 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.753 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113425 s, 92.4 MB/s 2024-04-17T10:48:52.765 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.765 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.765 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00903109 s, 116 MB/s 2024-04-17T10:48:52.780 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.780 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.780 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127581 s, 82.2 MB/s 2024-04-17T10:48:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122142 s, 85.8 MB/s 2024-04-17T10:48:52.807 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.807 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.807 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00958348 s, 109 MB/s 2024-04-17T10:48:52.823 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.823 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.823 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134832 s, 77.8 MB/s 2024-04-17T10:48:52.835 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.836 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.836 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00981168 s, 107 MB/s 2024-04-17T10:48:52.848 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.848 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.848 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0094501 s, 111 MB/s 2024-04-17T10:48:52.862 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.862 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.862 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113483 s, 92.4 MB/s 2024-04-17T10:48:52.876 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.876 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.876 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118404 s, 88.6 MB/s 2024-04-17T10:48:52.888 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.888 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.888 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00930807 s, 113 MB/s 2024-04-17T10:48:52.900 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.900 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.900 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00949154 s, 110 MB/s 2024-04-17T10:48:52.912 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.912 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.912 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00907783 s, 116 MB/s 2024-04-17T10:48:52.926 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.926 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.926 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112375 s, 93.3 MB/s 2024-04-17T10:48:52.941 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.941 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.941 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012465 s, 84.1 MB/s 2024-04-17T10:48:52.953 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:52.953 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:52.953 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00928145 s, 113 MB/s 2024-04-17T10:48:52.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:52.954 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:48:52.954 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:52.954 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.30 2024-04-17T10:48:52.954 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:48:52.954 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:48:52.954 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:48:52.954 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:53.021 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.021 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.022 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130721 s, 80.2 MB/s 2024-04-17T10:48:53.037 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.037 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.037 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129824 s, 80.8 MB/s 2024-04-17T10:48:53.053 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.053 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.053 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127378 s, 82.3 MB/s 2024-04-17T10:48:53.069 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.069 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.069 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125933 s, 83.3 MB/s 2024-04-17T10:48:53.085 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.085 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.085 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127136 s, 82.5 MB/s 2024-04-17T10:48:53.103 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.103 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.103 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143948 s, 72.8 MB/s 2024-04-17T10:48:53.121 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.121 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.121 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143812 s, 72.9 MB/s 2024-04-17T10:48:53.137 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.137 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.137 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129315 s, 81.1 MB/s 2024-04-17T10:48:53.158 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.158 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.158 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171571 s, 61.1 MB/s 2024-04-17T10:48:53.174 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.174 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.174 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128533 s, 81.6 MB/s 2024-04-17T10:48:53.188 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.188 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.188 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010657 s, 98.4 MB/s 2024-04-17T10:48:53.203 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.203 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.203 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119552 s, 87.7 MB/s 2024-04-17T10:48:53.219 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.219 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.220 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135231 s, 77.5 MB/s 2024-04-17T10:48:53.234 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.234 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.234 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118253 s, 88.7 MB/s 2024-04-17T10:48:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134998 s, 77.7 MB/s 2024-04-17T10:48:53.269 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.269 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.269 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150551 s, 69.6 MB/s 2024-04-17T10:48:53.285 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.286 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.286 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131598 s, 79.7 MB/s 2024-04-17T10:48:53.303 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.304 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.304 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142301 s, 73.7 MB/s 2024-04-17T10:48:53.321 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.322 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.322 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147306 s, 71.2 MB/s 2024-04-17T10:48:53.337 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.338 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.338 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127594 s, 82.2 MB/s 2024-04-17T10:48:53.355 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.355 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.355 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135235 s, 77.5 MB/s 2024-04-17T10:48:53.372 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.372 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.372 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136934 s, 76.6 MB/s 2024-04-17T10:48:53.388 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.388 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.388 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130292 s, 80.5 MB/s 2024-04-17T10:48:53.401 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.401 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.402 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104415 s, 100 MB/s 2024-04-17T10:48:53.414 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.414 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.415 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00992322 s, 106 MB/s 2024-04-17T10:48:53.427 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.427 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.427 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00951138 s, 110 MB/s 2024-04-17T10:48:53.443 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.443 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.444 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139395 s, 75.2 MB/s 2024-04-17T10:48:53.459 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.459 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.459 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126766 s, 82.7 MB/s 2024-04-17T10:48:53.471 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.471 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.472 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00978001 s, 107 MB/s 2024-04-17T10:48:53.485 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.485 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.485 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010415 s, 101 MB/s 2024-04-17T10:48:53.499 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.500 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.500 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121871 s, 86.0 MB/s 2024-04-17T10:48:53.515 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.515 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.515 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126685 s, 82.8 MB/s 2024-04-17T10:48:53.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:53.517 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:48:53.517 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:48:53.517 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.31 2024-04-17T10:48:53.517 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:48:53.517 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:48:53.517 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:48:53.517 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:48:53.583 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.583 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.583 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011868 s, 88.4 MB/s 2024-04-17T10:48:53.599 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.599 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.599 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129185 s, 81.2 MB/s 2024-04-17T10:48:53.614 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.614 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.614 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123907 s, 84.6 MB/s 2024-04-17T10:48:53.629 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.630 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.630 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129367 s, 81.1 MB/s 2024-04-17T10:48:53.644 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.644 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.644 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116715 s, 89.8 MB/s 2024-04-17T10:48:53.656 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.656 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.656 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00961034 s, 109 MB/s 2024-04-17T10:48:53.671 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.672 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.672 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128307 s, 81.7 MB/s 2024-04-17T10:48:53.687 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.687 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.687 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126708 s, 82.8 MB/s 2024-04-17T10:48:53.700 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.700 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.700 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108745 s, 96.4 MB/s 2024-04-17T10:48:53.715 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.715 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.715 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123092 s, 85.2 MB/s 2024-04-17T10:48:53.729 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.729 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.729 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110308 s, 95.1 MB/s 2024-04-17T10:48:53.741 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.741 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.741 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00885255 s, 118 MB/s 2024-04-17T10:48:53.754 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.754 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.755 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011175 s, 93.8 MB/s 2024-04-17T10:48:53.770 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.770 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.770 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129193 s, 81.2 MB/s 2024-04-17T10:48:53.783 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.783 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.783 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00980599 s, 107 MB/s 2024-04-17T10:48:53.799 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.799 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.799 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128744 s, 81.4 MB/s 2024-04-17T10:48:53.816 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.816 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.816 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131011 s, 80.0 MB/s 2024-04-17T10:48:53.828 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.828 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.829 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00992264 s, 106 MB/s 2024-04-17T10:48:53.845 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.845 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.845 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135835 s, 77.2 MB/s 2024-04-17T10:48:53.863 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.863 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.863 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146716 s, 71.5 MB/s 2024-04-17T10:48:53.876 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.876 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.876 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107609 s, 97.4 MB/s 2024-04-17T10:48:53.892 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.892 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.892 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123573 s, 84.9 MB/s 2024-04-17T10:48:53.908 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.908 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.908 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135932 s, 77.1 MB/s 2024-04-17T10:48:53.922 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.923 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.923 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104963 s, 99.9 MB/s 2024-04-17T10:48:53.939 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.940 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.940 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013992 s, 74.9 MB/s 2024-04-17T10:48:53.953 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.953 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.953 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106661 s, 98.3 MB/s 2024-04-17T10:48:53.967 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.967 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.967 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109258 s, 96.0 MB/s 2024-04-17T10:48:53.981 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.981 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.981 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112184 s, 93.5 MB/s 2024-04-17T10:48:53.997 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:53.997 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:53.997 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126353 s, 83.0 MB/s 2024-04-17T10:48:54.012 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:54.012 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:54.012 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011731 s, 89.4 MB/s 2024-04-17T10:48:54.028 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:54.028 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:54.028 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131538 s, 79.7 MB/s 2024-04-17T10:48:54.046 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:48:54.046 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:48:54.047 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150479 s, 69.7 MB/s 2024-04-17T10:48:54.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:54.048 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub start /scrub_control_test_path recursive 2024-04-17T10:48:54.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.229+0000 7f91f1a26640 1 Processor -- start 2024-04-17T10:48:54.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.229+0000 7f91f1a26640 1 -- start start 2024-04-17T10:48:54.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.229+0000 7f91f1a26640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91ec0aebd0 0x7f91ec0ced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.229+0000 7f91f1a26640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f91ec15b990 0x7f91ec0cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.229+0000 7f91f1a26640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91ec0cf3e0 con 0x7f91ec0cd6c0 2024-04-17T10:48:54.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.229+0000 7f91f1a26640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91ec15e5c0 con 0x7f91ec15b990 2024-04-17T10:48:54.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.229+0000 7f91f1a26640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec0b4230 con 0x7f91ec0aebd0 2024-04-17T10:48:54.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.229+0000 7f91eb7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f91ec0cd6c0 0x7f91ec06abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55896/0 (socket says 172.21.15.83:55896) 2024-04-17T10:48:54.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.229+0000 7f91eb7fe640 1 -- 172.21.15.83:0/1390070124 learned_addr learned my addr 172.21.15.83:0/1390070124 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:54.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.229+0000 7f91eaffd640 1 --2- 172.21.15.83:0/1390070124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91ec0aebd0 0x7f91ec0ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.229+0000 7f91ea7fc640 1 --2- 172.21.15.83:0/1390070124 >> v2:172.21.15.167:3300/0 conn(0x7f91ec15b990 0x7f91ec0cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.229+0000 7f91e9ffb640 1 -- 172.21.15.83:0/1390070124 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1280051984 0 0) 0x7f91ec0cf3e0 con 0x7f91ec0cd6c0 2024-04-17T10:48:54.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.229+0000 7f91e9ffb640 1 -- 172.21.15.83:0/1390070124 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91d4003530 con 0x7f91ec0cd6c0 2024-04-17T10:48:54.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.229+0000 7f91e9ffb640 1 -- 172.21.15.83:0/1390070124 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2332168435 0 0) 0x7f91d4003530 con 0x7f91ec0cd6c0 2024-04-17T10:48:54.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.229+0000 7f91e9ffb640 1 -- 172.21.15.83:0/1390070124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91ec0aebd0 msgr2=0x7f91ec0ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:54.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.229+0000 7f91e9ffb640 1 --2- 172.21.15.83:0/1390070124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91ec0aebd0 0x7f91ec0ced50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.229+0000 7f91e9ffb640 1 -- 172.21.15.83:0/1390070124 >> v2:172.21.15.167:3300/0 conn(0x7f91ec15b990 msgr2=0x7f91ec0cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:54.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.229+0000 7f91e9ffb640 1 --2- 172.21.15.83:0/1390070124 >> v2:172.21.15.167:3300/0 conn(0x7f91ec15b990 0x7f91ec0cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.229+0000 7f91e9ffb640 1 -- 172.21.15.83:0/1390070124 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ec0b4510 con 0x7f91ec0cd6c0 2024-04-17T10:48:54.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.229+0000 7f91e9ffb640 1 -- 172.21.15.83:0/1390070124 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f91e4001a00 con 0x7f91ec0cd6c0 2024-04-17T10:48:54.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.229+0000 7f91e9ffb640 1 -- 172.21.15.83:0/1390070124 >> v1:172.21.15.83:6789/0 conn(0x7f91ec0cd6c0 legacy=0x7f91ec06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.229+0000 7f91e9ffb640 1 --2- 172.21.15.83:0/1390070124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91ec0aebd0 0x7f91d40039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.229+0000 7f91e9ffb640 1 --2- 172.21.15.83:0/1390070124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91d4003f40 0x7f91d4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.229+0000 7f91e9ffb640 1 --2- 172.21.15.83:0/1390070124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91d4004d50 0x7f91d40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.229+0000 7f91e9ffb640 1 -- 172.21.15.83:0/1390070124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec15e5c0 con 0x7f91d4003f40 2024-04-17T10:48:54.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.229+0000 7f91e9ffb640 1 -- 172.21.15.83:0/1390070124 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec0b4230 con 0x7f91d4004d50 2024-04-17T10:48:54.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.229+0000 7f91e9ffb640 1 -- 172.21.15.83:0/1390070124 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d400b9d0 con 0x7f91ec0aebd0 2024-04-17T10:48:54.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.233+0000 7f91ea7fc640 1 --2- 172.21.15.83:0/1390070124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91d4003f40 0x7f91d4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.233+0000 7f91eaffd640 1 --2- 172.21.15.83:0/1390070124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91ec0aebd0 0x7f91d40039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.233+0000 7f91eb7fe640 1 --2- 172.21.15.83:0/1390070124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91d4004d50 0x7f91d40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.233+0000 7f91ea7fc640 1 -- 172.21.15.83:0/1390070124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91ec0aebd0 msgr2=0x7f91d40039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:54.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.233+0000 7f91ea7fc640 1 --2- 172.21.15.83:0/1390070124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91ec0aebd0 0x7f91d40039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.233+0000 7f91ea7fc640 1 -- 172.21.15.83:0/1390070124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91d4004d50 msgr2=0x7f91d40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:54.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.233+0000 7f91ea7fc640 1 --2- 172.21.15.83:0/1390070124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91d4004d50 0x7f91d40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.233+0000 7f91ea7fc640 1 -- 172.21.15.83:0/1390070124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d4003530 con 0x7f91d4003f40 2024-04-17T10:48:54.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.237+0000 7f91ea7fc640 1 --2- 172.21.15.83:0/1390070124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91d4003f40 0x7f91d4004810 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f91e0009ae0 tx=0x7f91e001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132bcaee738109c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:54.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.237+0000 7f91e9ffb640 1 -- 172.21.15.83:0/1390070124 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e00b2070 con 0x7f91d4003f40 2024-04-17T10:48:54.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.237+0000 7f91e9ffb640 1 -- 172.21.15.83:0/1390070124 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e00a56a0 con 0x7f91d4003f40 2024-04-17T10:48:54.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.237+0000 7f91e9ffb640 1 -- 172.21.15.83:0/1390070124 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e00096b0 con 0x7f91d4003f40 2024-04-17T10:48:54.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.237+0000 7f91f1a26640 1 -- 172.21.15.83:0/1390070124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91d4003f40 msgr2=0x7f91d4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.237+0000 7f91f1a26640 1 --2- 172.21.15.83:0/1390070124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91d4003f40 0x7f91d4004810 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f91e0009ae0 tx=0x7f91e001ca50 comp rx=0 tx=0).stop 2024-04-17T10:48:54.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.237+0000 7f91eb7fe640 1 -- 172.21.15.83:0/1390070124 reap_dead start 2024-04-17T10:48:54.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.237+0000 7f91f1a26640 1 -- 172.21.15.83:0/1390070124 shutdown_connections 2024-04-17T10:48:54.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.237+0000 7f91f1a26640 1 -- 172.21.15.83:0/1390070124 >> 172.21.15.83:0/1390070124 conn(0x7f91ec0cc050 msgr2=0x7f91ec0cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:54.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.237+0000 7f91f1a26640 1 -- 172.21.15.83:0/1390070124 shutdown_connections 2024-04-17T10:48:54.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.237+0000 7f91f1a26640 1 -- 172.21.15.83:0/1390070124 wait complete. 2024-04-17T10:48:54.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.237+0000 7f91f1a26640 1 Processor -- start 2024-04-17T10:48:54.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.237+0000 7f91f1a26640 1 -- start start 2024-04-17T10:48:54.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.237+0000 7f91f1a26640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91ec1f3620 0x7f91ec070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.237+0000 7f91f1a26640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91ec071230 0x7f91ec154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.237+0000 7f91f1a26640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91ec155260 0x7f91ec147990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.237+0000 7f91f1a26640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec0aefb0 con 0x7f91ec071230 2024-04-17T10:48:54.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.237+0000 7f91f1a26640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec15bdf0 con 0x7f91ec1f3620 2024-04-17T10:48:54.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.237+0000 7f91f1a26640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec06b2f0 con 0x7f91ec155260 2024-04-17T10:48:54.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.237+0000 7f91ea7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91ec071230 0x7f91ec154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.237+0000 7f91ea7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91ec071230 0x7f91ec154d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44610/0 (socket says 172.21.15.83:44610) 2024-04-17T10:48:54.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.237+0000 7f91ea7fc640 1 -- 172.21.15.83:0/875097762 learned_addr learned my addr 172.21.15.83:0/875097762 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:54.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.237+0000 7f91eaffd640 1 --2- 172.21.15.83:0/875097762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91ec1f3620 0x7f91ec070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.237+0000 7f91eb7fe640 1 --2- 172.21.15.83:0/875097762 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91ec155260 0x7f91ec147990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.237+0000 7f91ea7fc640 1 -- 172.21.15.83:0/875097762 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91ec155260 msgr2=0x7f91ec147990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.237+0000 7f91ea7fc640 1 --2- 172.21.15.83:0/875097762 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91ec155260 0x7f91ec147990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.237+0000 7f91ea7fc640 1 -- 172.21.15.83:0/875097762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91ec1f3620 msgr2=0x7f91ec070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.237+0000 7f91ea7fc640 1 --2- 172.21.15.83:0/875097762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91ec1f3620 0x7f91ec070cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.237+0000 7f91ea7fc640 1 -- 172.21.15.83:0/875097762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ec071790 con 0x7f91ec071230 2024-04-17T10:48:54.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.237+0000 7f91eb7fe640 1 --2- 172.21.15.83:0/875097762 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91ec155260 0x7f91ec147990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:54.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.241+0000 7f91ea7fc640 1 --2- 172.21.15.83:0/875097762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91ec071230 0x7f91ec154d20 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f91e001b160 tx=0x7f91e0019850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:54.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.241+0000 7f91d3fff640 1 -- 172.21.15.83:0/875097762 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e00bd070 con 0x7f91ec071230 2024-04-17T10:48:54.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.241+0000 7f91d3fff640 1 -- 172.21.15.83:0/875097762 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e00ad440 con 0x7f91ec071230 2024-04-17T10:48:54.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.241+0000 7f91f1a26640 1 -- 172.21.15.83:0/875097762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91ec071a70 con 0x7f91ec071230 2024-04-17T10:48:54.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.241+0000 7f91d3fff640 1 -- 172.21.15.83:0/875097762 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e00b2040 con 0x7f91ec071230 2024-04-17T10:48:54.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.241+0000 7f91f1a26640 1 -- 172.21.15.83:0/875097762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91ec0afaa0 con 0x7f91ec071230 2024-04-17T10:48:54.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.241+0000 7f91d3fff640 1 -- 172.21.15.83:0/875097762 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f91e00c0070 con 0x7f91ec071230 2024-04-17T10:48:54.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.241+0000 7f91d3fff640 1 --2- 172.21.15.83:0/875097762 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f91d8066f30 0x7f91d80693f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.241+0000 7f91d3fff640 1 -- 172.21.15.83:0/875097762 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f91e011ea90 con 0x7f91ec071230 2024-04-17T10:48:54.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.241+0000 7f91eaffd640 1 --2- 172.21.15.83:0/875097762 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f91d8066f30 0x7f91d80693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.241+0000 7f91eaffd640 1 --2- 172.21.15.83:0/875097762 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f91d8066f30 0x7f91d80693f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f91dc0099e0 tx=0x7f91dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:54.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.241+0000 7f91f4b93000 1 Processor -- start 2024-04-17T10:48:54.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.241+0000 7f91f4b93000 1 -- start start 2024-04-17T10:48:54.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91f4b93000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a202e73f0 0x560a202e77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91f4b93000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a202e7d10 0x560a2029dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91f4b93000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560a2029e520 0x560a2029e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91f4b93000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a200ebd50 con 0x560a202e73f0 2024-04-17T10:48:54.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91f4b93000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a20073e40 con 0x560a202e7d10 2024-04-17T10:48:54.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91f4b93000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a200e7df0 con 0x560a2029e520 2024-04-17T10:48:54.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91eaffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a202e73f0 0x560a202e77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91eaffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a202e73f0 0x560a202e77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44614/0 (socket says 172.21.15.83:44614) 2024-04-17T10:48:54.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91eaffd640 1 -- 172.21.15.83:0/2222333131 learned_addr learned my addr 172.21.15.83:0/2222333131 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:54.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91eb7fe640 1 --2- 172.21.15.83:0/2222333131 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560a2029e520 0x560a2029e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91ea7fc640 1 --2- 172.21.15.83:0/2222333131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a202e7d10 0x560a2029dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91eaffd640 1 -- 172.21.15.83:0/2222333131 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560a2029e520 msgr2=0x560a2029e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:54.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91eaffd640 1 --2- 172.21.15.83:0/2222333131 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560a2029e520 0x560a2029e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91eaffd640 1 -- 172.21.15.83:0/2222333131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a202e7d10 msgr2=0x560a2029dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:54.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91eaffd640 1 --2- 172.21.15.83:0/2222333131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a202e7d10 0x560a2029dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91eaffd640 1 -- 172.21.15.83:0/2222333131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a20088d90 con 0x560a202e73f0 2024-04-17T10:48:54.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91eaffd640 1 --2- 172.21.15.83:0/2222333131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a202e73f0 0x560a202e77d0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f91dc014d20 tx=0x7f91dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c05e0de5bc59caf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:54.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91d17fa640 1 -- 172.21.15.83:0/2222333131 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc027070 con 0x560a202e73f0 2024-04-17T10:48:54.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91d17fa640 1 -- 172.21.15.83:0/2222333131 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91dc023450 con 0x560a202e73f0 2024-04-17T10:48:54.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91d17fa640 1 -- 172.21.15.83:0/2222333131 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc023800 con 0x560a202e73f0 2024-04-17T10:48:54.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91f4b93000 1 -- 172.21.15.83:0/2222333131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a202e73f0 msgr2=0x560a202e77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91f4b93000 1 --2- 172.21.15.83:0/2222333131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a202e73f0 0x560a202e77d0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f91dc014d20 tx=0x7f91dc017000 comp rx=0 tx=0).stop 2024-04-17T10:48:54.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91f4b93000 1 -- 172.21.15.83:0/2222333131 shutdown_connections 2024-04-17T10:48:54.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91f4b93000 1 --2- 172.21.15.83:0/2222333131 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560a2029e520 0x560a2029e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91f4b93000 1 --2- 172.21.15.83:0/2222333131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a202e7d10 0x560a2029dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91f4b93000 1 --2- 172.21.15.83:0/2222333131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a202e73f0 0x560a202e77d0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91f4b93000 1 -- 172.21.15.83:0/2222333131 >> 172.21.15.83:0/2222333131 conn(0x560a2029a4d0 msgr2=0x560a2029c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:54.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91f4b93000 1 -- 172.21.15.83:0/2222333131 shutdown_connections 2024-04-17T10:48:54.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91f4b93000 1 -- 172.21.15.83:0/2222333131 wait complete. 2024-04-17T10:48:54.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91f4b93000 1 -- start start 2024-04-17T10:48:54.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91f4b93000 1 Processor -- start 2024-04-17T10:48:54.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91f4b93000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560a2029e520 0x560a202a4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91f4b93000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a202e73f0 0x560a202d50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91f4b93000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a202e7d10 0x560a202d5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91f4b93000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a20091610 con 0x560a202e7d10 2024-04-17T10:48:54.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91f4b93000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a200a69a0 con 0x560a202e73f0 2024-04-17T10:48:54.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91f4b93000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a200f0380 con 0x560a2029e520 2024-04-17T10:48:54.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91f4b93000 10 client.0 _ll_register_callbacks cb 0x560a201a00d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:48:54.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91eb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a202e7d10 0x560a202d5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91eb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a202e7d10 0x560a202d5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44626/0 (socket says 172.21.15.83:44626) 2024-04-17T10:48:54.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91eb7fe640 1 -- 172.21.15.83:0/2496006170 learned_addr learned my addr 172.21.15.83:0/2496006170 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:54.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91eaffd640 1 --2- 172.21.15.83:0/2496006170 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560a2029e520 0x560a202a4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91ea7fc640 1 --2- 172.21.15.83:0/2496006170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a202e73f0 0x560a202d50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91eb7fe640 1 -- 172.21.15.83:0/2496006170 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560a2029e520 msgr2=0x560a202a4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:54.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91eb7fe640 1 --2- 172.21.15.83:0/2496006170 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560a2029e520 0x560a202a4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91eb7fe640 1 -- 172.21.15.83:0/2496006170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a202e73f0 msgr2=0x560a202d50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:54.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91eb7fe640 1 --2- 172.21.15.83:0/2496006170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a202e73f0 0x560a202d50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91eb7fe640 1 -- 172.21.15.83:0/2496006170 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a200e6d40 con 0x560a202e7d10 2024-04-17T10:48:54.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91eb7fe640 1 --2- 172.21.15.83:0/2496006170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a202e7d10 0x560a202d5600 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f91e4000f90 tx=0x7f91e4013dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a053ea9b4951d373 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:54.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.245+0000 7f91ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:48:54.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.249+0000 7f91ca7fc640 1 -- 172.21.15.83:0/2496006170 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e4019070 con 0x560a202e7d10 2024-04-17T10:48:54.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.249+0000 7f91ca7fc640 1 -- 172.21.15.83:0/2496006170 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f91e4011cb0 con 0x560a202e7d10 2024-04-17T10:48:54.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.249+0000 7f91f4b93000 1 -- 172.21.15.83:0/2496006170 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91dc023450 con 0x560a202e7d10 2024-04-17T10:48:54.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.249+0000 7f91ca7fc640 1 -- 172.21.15.83:0/2496006170 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91e4024050 con 0x560a202e7d10 2024-04-17T10:48:54.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.249+0000 7f91f4b93000 10 client.8802 fetch_fsmap learned FSMap version 69 2024-04-17T10:48:54.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.249+0000 7f91f4b93000 1 -- 172.21.15.83:0/2496006170 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x560a2016ef00 con 0x560a202e7d10 2024-04-17T10:48:54.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.249+0000 7f91ca7fc640 1 -- 172.21.15.83:0/2496006170 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91dc023450 con 0x560a202e7d10 2024-04-17T10:48:54.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.249+0000 7f91ca7fc640 1 -- 172.21.15.83:0/2496006170 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f91e4027020 con 0x560a202e7d10 2024-04-17T10:48:54.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.249+0000 7f91f4b93000 10 client.8802 fetch_fsmap finished waiting for FSMap version 69 2024-04-17T10:48:54.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.249+0000 7f91f4b93000 10 client.8802 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-17T10:48:54.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.249+0000 7f91f4b93000 20 client.8802 populate_metadata read hostname 'smithi083' 2024-04-17T10:48:54.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.249+0000 7f91f4b93000 1 --2- 172.21.15.83:0/2496006170 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x560a202ee360 0x560a202ee770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.249+0000 7f91f4b93000 4 client.8802 mds_command: new command op to 8604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:48:54.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.249+0000 7f91f4b93000 1 -- 172.21.15.83:0/2496006170 --> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a20087a20 con 0x560a202ee360 2024-04-17T10:48:54.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.249+0000 7f91eaffd640 1 --2- 172.21.15.83:0/2496006170 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x560a202ee360 0x560a202ee770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.249+0000 7f91eaffd640 1 --2- 172.21.15.83:0/2496006170 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x560a202ee360 0x560a202ee770 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5130d18df35996ed server_cookie=d849d8a1cf36f32d in_seq=0 out_seq=0 2024-04-17T10:48:54.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.249+0000 7f91ca7fc640 10 client.8802 ms_handle_connect on v2:172.21.15.83:6836/1479417998 2024-04-17T10:48:54.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.249+0000 7f91ca7fc640 1 -- 172.21.15.83:0/2496006170 <== mds.0 v2:172.21.15.83:6836/1479417998 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560a20087a20 con 0x560a202ee360 2024-04-17T10:48:54.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.249+0000 7f91ca7fc640 10 client.8802 handle_command_reply: tid=0 2024-04-17T10:48:54.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.249+0000 7f91f4b93000 1 client.8802 shutdown 2024-04-17T10:48:54.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.249+0000 7f91f4b93000 1 -- 172.21.15.83:0/2496006170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a202e7d10 msgr2=0x560a202d5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.249+0000 7f91f4b93000 1 --2- 172.21.15.83:0/2496006170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a202e7d10 0x560a202d5600 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f91e4000f90 tx=0x7f91e4013dd0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.249+0000 7f91f4b93000 1 -- 172.21.15.83:0/2496006170 shutdown_connections 2024-04-17T10:48:54.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.249+0000 7f91f4b93000 1 --2- 172.21.15.83:0/2496006170 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x560a202ee360 0x560a202ee770 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.249+0000 7f91f4b93000 1 --2- 172.21.15.83:0/2496006170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a202e7d10 0x560a202d5600 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.249+0000 7f91ca7fc640 0 client.8802 ms_handle_reset on v2:172.21.15.83:6836/1479417998 2024-04-17T10:48:54.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.249+0000 7f91f4b93000 1 --2- 172.21.15.83:0/2496006170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a202e73f0 0x560a202d50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.249+0000 7f91f4b93000 1 --2- 172.21.15.83:0/2496006170 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560a2029e520 0x560a202a4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.249+0000 7f91f4b93000 1 -- 172.21.15.83:0/2496006170 >> 172.21.15.83:0/2496006170 conn(0x560a2029a4d0 msgr2=0x560a2029d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:54.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.249+0000 7f91f4b93000 1 -- 172.21.15.83:0/2496006170 shutdown_connections 2024-04-17T10:48:54.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.249+0000 7f91f4b93000 1 -- 172.21.15.83:0/2496006170 wait complete. 2024-04-17T10:48:54.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.249+0000 7f91f4b93000 20 client.8802 trim_cache size 0 max 16384 2024-04-17T10:48:54.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.265+0000 7f91f4b93000 1 Processor -- start 2024-04-17T10:48:54.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.265+0000 7f91f4b93000 1 -- start start 2024-04-17T10:48:54.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.265+0000 7f91f4b93000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a2029e520 0x560a20343e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.265+0000 7f91f4b93000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560a202e73f0 0x560a20344390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.265+0000 7f91f4b93000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a202e7d10 0x560a203448d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.265+0000 7f91f4b93000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a202a43f0 con 0x560a2029e520 2024-04-17T10:48:54.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.265+0000 7f91f4b93000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec200a60 con 0x560a202e7d10 2024-04-17T10:48:54.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.265+0000 7f91f4b93000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec1ffc50 con 0x560a202e73f0 2024-04-17T10:48:54.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.265+0000 7f91eaffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a2029e520 0x560a20343e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.265+0000 7f91eaffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a2029e520 0x560a20343e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44640/0 (socket says 172.21.15.83:44640) 2024-04-17T10:48:54.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.265+0000 7f91eaffd640 1 -- 172.21.15.83:0/3661816897 learned_addr learned my addr 172.21.15.83:0/3661816897 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:54.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.265+0000 7f91eaffd640 1 -- 172.21.15.83:0/3661816897 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560a202e73f0 msgr2=0x560a20344390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:54.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.265+0000 7f91eaffd640 1 --2- 172.21.15.83:0/3661816897 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560a202e73f0 0x560a20344390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.265+0000 7f91eaffd640 1 -- 172.21.15.83:0/3661816897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a202e7d10 msgr2=0x560a203448d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:54.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.265+0000 7f91eaffd640 1 --2- 172.21.15.83:0/3661816897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a202e7d10 0x560a203448d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.265+0000 7f91eaffd640 1 -- 172.21.15.83:0/3661816897 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a202e7b40 con 0x560a2029e520 2024-04-17T10:48:54.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.265+0000 7f91eaffd640 1 --2- 172.21.15.83:0/3661816897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a2029e520 0x560a20343e50 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f91dc005d00 tx=0x7f91dc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52bc1158273e66e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:54.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.265+0000 7f91d0ff9640 1 -- 172.21.15.83:0/3661816897 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc027040 con 0x560a2029e520 2024-04-17T10:48:54.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.265+0000 7f91d0ff9640 1 -- 172.21.15.83:0/3661816897 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91dc034b00 con 0x560a2029e520 2024-04-17T10:48:54.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.265+0000 7f91d0ff9640 1 -- 172.21.15.83:0/3661816897 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc0342d0 con 0x560a2029e520 2024-04-17T10:48:54.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.265+0000 7f91f4b93000 1 -- 172.21.15.83:0/3661816897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a2029e520 msgr2=0x560a20343e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.265+0000 7f91f4b93000 1 --2- 172.21.15.83:0/3661816897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a2029e520 0x560a20343e50 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f91dc005d00 tx=0x7f91dc01b000 comp rx=0 tx=0).stop 2024-04-17T10:48:54.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.265+0000 7f91f4b93000 1 -- 172.21.15.83:0/3661816897 shutdown_connections 2024-04-17T10:48:54.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.265+0000 7f91f4b93000 1 --2- 172.21.15.83:0/3661816897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a202e7d10 0x560a203448d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.265+0000 7f91f4b93000 1 --2- 172.21.15.83:0/3661816897 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560a202e73f0 0x560a20344390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.265+0000 7f91f4b93000 1 --2- 172.21.15.83:0/3661816897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a2029e520 0x560a20343e50 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.265+0000 7f91f4b93000 1 -- 172.21.15.83:0/3661816897 >> 172.21.15.83:0/3661816897 conn(0x560a2029a4d0 msgr2=0x560a202a3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:54.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.265+0000 7f91f4b93000 1 -- 172.21.15.83:0/3661816897 shutdown_connections 2024-04-17T10:48:54.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.265+0000 7f91f4b93000 1 -- 172.21.15.83:0/3661816897 wait complete. 2024-04-17T10:48:54.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.265+0000 7f91f4b93000 1 -- start start 2024-04-17T10:48:54.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.265+0000 7f91f4b93000 1 Processor -- start 2024-04-17T10:48:54.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91f4b93000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a2029e520 0x560a20344200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91f4b93000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560a202e73f0 0x560a202ec420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91f4b93000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a202e7d10 0x560a202ec960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91f4b93000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec1ff6b0 con 0x560a202e7d10 2024-04-17T10:48:54.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91f4b93000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec0b0fb0 con 0x560a2029e520 2024-04-17T10:48:54.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91f4b93000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a200a2dc0 con 0x560a202e73f0 2024-04-17T10:48:54.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91f4b93000 10 client.0 _ll_register_callbacks cb 0x7f91dc01a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:48:54.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91eb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a202e7d10 0x560a202ec960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91eb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a202e7d10 0x560a202ec960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44650/0 (socket says 172.21.15.83:44650) 2024-04-17T10:48:54.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91eb7fe640 1 -- 172.21.15.83:0/4268477115 learned_addr learned my addr 172.21.15.83:0/4268477115 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:54.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91ea7fc640 1 --2- 172.21.15.83:0/4268477115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560a202e73f0 0x560a202ec420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91eaffd640 1 --2- 172.21.15.83:0/4268477115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a2029e520 0x560a20344200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91eb7fe640 1 -- 172.21.15.83:0/4268477115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560a202e73f0 msgr2=0x560a202ec420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:54.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91eb7fe640 1 --2- 172.21.15.83:0/4268477115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560a202e73f0 0x560a202ec420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91eb7fe640 1 -- 172.21.15.83:0/4268477115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a2029e520 msgr2=0x560a20344200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:54.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91eb7fe640 1 --2- 172.21.15.83:0/4268477115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a2029e520 0x560a20344200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91eb7fe640 1 -- 172.21.15.83:0/4268477115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a20145330 con 0x560a202e7d10 2024-04-17T10:48:54.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91eaffd640 1 --2- 172.21.15.83:0/4268477115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a2029e520 0x560a20344200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:54.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91eb7fe640 1 --2- 172.21.15.83:0/4268477115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a202e7d10 0x560a202ec960 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f91e4002240 tx=0x7f91ec0b46e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2fa37dda718cb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:54.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:48:54.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91ca7fc640 1 -- 172.21.15.83:0/4268477115 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e4019040 con 0x560a202e7d10 2024-04-17T10:48:54.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91ca7fc640 1 -- 172.21.15.83:0/4268477115 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f91e401e440 con 0x560a202e7d10 2024-04-17T10:48:54.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91f4b93000 1 -- 172.21.15.83:0/4268477115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91dc034b00 con 0x560a202e7d10 2024-04-17T10:48:54.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91ca7fc640 1 -- 172.21.15.83:0/4268477115 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91e40071c0 con 0x560a202e7d10 2024-04-17T10:48:54.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91f4b93000 10 client.8808 fetch_fsmap learned FSMap version 69 2024-04-17T10:48:54.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91f4b93000 1 -- 172.21.15.83:0/4268477115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x560a200b85f0 con 0x560a202e7d10 2024-04-17T10:48:54.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91ca7fc640 1 -- 172.21.15.83:0/4268477115 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91dc034b00 con 0x560a202e7d10 2024-04-17T10:48:54.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91ca7fc640 1 -- 172.21.15.83:0/4268477115 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f91e4027070 con 0x560a202e7d10 2024-04-17T10:48:54.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91f4b93000 10 client.8808 fetch_fsmap finished waiting for FSMap version 69 2024-04-17T10:48:54.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91f4b93000 10 client.8808 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-17T10:48:54.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91f4b93000 20 client.8808 populate_metadata read hostname 'smithi083' 2024-04-17T10:48:54.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91f4b93000 1 --2- 172.21.15.83:0/4268477115 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x560a202ecea0 0x560a202ed2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91f4b93000 4 client.8808 mds_command: new command op to 8604 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-17T10:48:54.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91f4b93000 1 -- 172.21.15.83:0/4268477115 --> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x560a200f9290 con 0x560a202ecea0 2024-04-17T10:48:54.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91eaffd640 1 --2- 172.21.15.83:0/4268477115 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x560a202ecea0 0x560a202ed2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91eaffd640 1 --2- 172.21.15.83:0/4268477115 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x560a202ecea0 0x560a202ed2b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2ecf2e0d8cad14c server_cookie=16f799b257391e4f in_seq=0 out_seq=0 2024-04-17T10:48:54.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.269+0000 7f91ca7fc640 10 client.8808 ms_handle_connect on v2:172.21.15.83:6836/1479417998 2024-04-17T10:48:54.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.273+0000 7f91ca7fc640 1 -- 172.21.15.83:0/4268477115 <== mds.0 v2:172.21.15.83:6836/1479417998 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560a200f9290 con 0x560a202ecea0 2024-04-17T10:48:54.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.273+0000 7f91ca7fc640 10 client.8808 handle_command_reply: tid=0 2024-04-17T10:48:54.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.273+0000 7f91f4b93000 1 client.8808 shutdown 2024-04-17T10:48:54.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.273+0000 7f91f4b93000 1 -- 172.21.15.83:0/4268477115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a202e7d10 msgr2=0x560a202ec960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.273+0000 7f91f4b93000 1 --2- 172.21.15.83:0/4268477115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a202e7d10 0x560a202ec960 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f91e4002240 tx=0x7f91ec0b46e0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.273+0000 7f91f4b93000 1 -- 172.21.15.83:0/4268477115 shutdown_connections 2024-04-17T10:48:54.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.273+0000 7f91f4b93000 1 --2- 172.21.15.83:0/4268477115 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x560a202ecea0 0x560a202ed2b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.273+0000 7f91f4b93000 1 --2- 172.21.15.83:0/4268477115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a202e7d10 0x560a202ec960 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.273+0000 7f91f4b93000 1 --2- 172.21.15.83:0/4268477115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560a202e73f0 0x560a202ec420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.273+0000 7f91ca7fc640 0 client.8808 ms_handle_reset on v2:172.21.15.83:6836/1479417998 2024-04-17T10:48:54.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.273+0000 7f91f4b93000 1 --2- 172.21.15.83:0/4268477115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a2029e520 0x560a20344200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.273+0000 7f91f4b93000 1 -- 172.21.15.83:0/4268477115 >> 172.21.15.83:0/4268477115 conn(0x560a2029a4d0 msgr2=0x560a202e6090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:54.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.273+0000 7f91f4b93000 1 -- 172.21.15.83:0/4268477115 shutdown_connections 2024-04-17T10:48:54.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.273+0000 7f91f4b93000 1 -- 172.21.15.83:0/4268477115 wait complete. 2024-04-17T10:48:54.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.273+0000 7f91f4b93000 20 client.8808 trim_cache size 0 max 16384 2024-04-17T10:48:54.272 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:48:54.273 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-17T10:48:54.273 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "9d8443c1-bb99-4ad5-8eef-ff47c6bdcf48", 2024-04-17T10:48:54.273 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-17T10:48:54.273 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:48:54.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.273+0000 7f91f1a26640 1 -- 172.21.15.83:0/875097762 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f91d8066f30 msgr2=0x7f91d80693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.273+0000 7f91f1a26640 1 --2- 172.21.15.83:0/875097762 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f91d8066f30 0x7f91d80693f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f91dc0099e0 tx=0x7f91dc008040 comp rx=0 tx=0).stop 2024-04-17T10:48:54.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.273+0000 7f91f1a26640 1 -- 172.21.15.83:0/875097762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91ec071230 msgr2=0x7f91ec154d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.273+0000 7f91f1a26640 1 --2- 172.21.15.83:0/875097762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91ec071230 0x7f91ec154d20 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f91e001b160 tx=0x7f91e0019850 comp rx=0 tx=0).stop 2024-04-17T10:48:54.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.273+0000 7f91f1a26640 1 -- 172.21.15.83:0/875097762 shutdown_connections 2024-04-17T10:48:54.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.273+0000 7f91f1a26640 1 --2- 172.21.15.83:0/875097762 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f91d8066f30 0x7f91d80693f0 secure :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f91dc0099e0 tx=0x7f91dc008040 comp rx=0 tx=0).stop 2024-04-17T10:48:54.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.273+0000 7f91f1a26640 1 --2- 172.21.15.83:0/875097762 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91ec155260 0x7f91ec147990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.273+0000 7f91f1a26640 1 --2- 172.21.15.83:0/875097762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91ec071230 0x7f91ec154d20 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.273+0000 7f91f1a26640 1 --2- 172.21.15.83:0/875097762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91ec1f3620 0x7f91ec070cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.273+0000 7f91f1a26640 1 -- 172.21.15.83:0/875097762 >> 172.21.15.83:0/875097762 conn(0x7f91ec0cc050 msgr2=0x7f91ec15a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:54.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.273+0000 7f91f1a26640 1 -- 172.21.15.83:0/875097762 shutdown_connections 2024-04-17T10:48:54.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.273+0000 7f91f1a26640 1 -- 172.21.15.83:0/875097762 wait complete. 2024-04-17T10:48:54.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:54.291 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub pause 2024-04-17T10:48:54.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.429+0000 7f71aa69b640 1 Processor -- start 2024-04-17T10:48:54.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.429+0000 7f71aa69b640 1 -- start start 2024-04-17T10:48:54.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.429+0000 7f71aa69b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f71a4154de0 0x7f71a406ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.429+0000 7f71aa69b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71a406b350 0x7f71a40b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.429+0000 7f71aa69b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f71a40b4040 0x7f71a4071580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.429+0000 7f71aa69b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71a40b4870 con 0x7f71a4154de0 2024-04-17T10:48:54.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.429+0000 7f71aa69b640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71a415bd70 con 0x7f71a40b4040 2024-04-17T10:48:54.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.429+0000 7f71aa69b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a415bf60 con 0x7f71a406b350 2024-04-17T10:48:54.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.429+0000 7f71a3fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f71a4154de0 0x7f71a406ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.429+0000 7f71a3fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f71a4154de0 0x7f71a406ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44666/0 (socket says 172.21.15.83:44666) 2024-04-17T10:48:54.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.429+0000 7f71a3fff640 1 -- 172.21.15.83:0/1012228381 learned_addr learned my addr 172.21.15.83:0/1012228381 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:54.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.429+0000 7f71a8c38640 1 --2- 172.21.15.83:0/1012228381 >> v2:172.21.15.167:3300/0 conn(0x7f71a40b4040 0x7f71a4071580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.429+0000 7f71a3fff640 1 -- 172.21.15.83:0/1012228381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71a406b350 msgr2=0x7f71a40b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:54.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.429+0000 7f71a37fe640 1 --2- 172.21.15.83:0/1012228381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71a406b350 0x7f71a40b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.429+0000 7f71a3fff640 1 --2- 172.21.15.83:0/1012228381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71a406b350 0x7f71a40b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.429+0000 7f71a3fff640 1 -- 172.21.15.83:0/1012228381 >> v2:172.21.15.167:3300/0 conn(0x7f71a40b4040 msgr2=0x7f71a4071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:54.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.429+0000 7f71a3fff640 1 --2- 172.21.15.83:0/1012228381 >> v2:172.21.15.167:3300/0 conn(0x7f71a40b4040 0x7f71a4071580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.429+0000 7f71a3fff640 1 -- 172.21.15.83:0/1012228381 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a415afb0 con 0x7f71a4154de0 2024-04-17T10:48:54.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.429+0000 7f71a8c38640 1 --2- 172.21.15.83:0/1012228381 >> v2:172.21.15.167:3300/0 conn(0x7f71a40b4040 0x7f71a4071580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:54.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.433+0000 7f71a3fff640 1 --2- 172.21.15.83:0/1012228381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71a4154de0 0x7f71a406ae10 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f71a406e900 tx=0x7f719401c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e8eb3147b0c375 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:54.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.433+0000 7f71a2ffd640 1 -- 172.21.15.83:0/1012228381 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71940a64d0 con 0x7f71a4154de0 2024-04-17T10:48:54.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.433+0000 7f71a2ffd640 1 -- 172.21.15.83:0/1012228381 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71940a6670 con 0x7f71a4154de0 2024-04-17T10:48:54.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.433+0000 7f71a2ffd640 1 -- 172.21.15.83:0/1012228381 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71940a6a40 con 0x7f71a4154de0 2024-04-17T10:48:54.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.433+0000 7f71aa69b640 1 -- 172.21.15.83:0/1012228381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71a4154de0 msgr2=0x7f71a406ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.433+0000 7f71aa69b640 1 --2- 172.21.15.83:0/1012228381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71a4154de0 0x7f71a406ae10 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f71a406e900 tx=0x7f719401c730 comp rx=0 tx=0).stop 2024-04-17T10:48:54.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.433+0000 7f71aa69b640 1 -- 172.21.15.83:0/1012228381 shutdown_connections 2024-04-17T10:48:54.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.433+0000 7f71aa69b640 1 --2- 172.21.15.83:0/1012228381 >> v2:172.21.15.167:3300/0 conn(0x7f71a40b4040 0x7f71a4071580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.433+0000 7f71aa69b640 1 --2- 172.21.15.83:0/1012228381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71a406b350 0x7f71a40b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.433+0000 7f71aa69b640 1 --2- 172.21.15.83:0/1012228381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71a4154de0 0x7f71a406ae10 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.433+0000 7f71aa69b640 1 -- 172.21.15.83:0/1012228381 >> 172.21.15.83:0/1012228381 conn(0x7f71a40b15b0 msgr2=0x7f71a40b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:54.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.433+0000 7f71aa69b640 1 -- 172.21.15.83:0/1012228381 shutdown_connections 2024-04-17T10:48:54.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.433+0000 7f71aa69b640 1 -- 172.21.15.83:0/1012228381 wait complete. 2024-04-17T10:48:54.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.437+0000 7f71aa69b640 1 Processor -- start 2024-04-17T10:48:54.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.437+0000 7f71aa69b640 1 -- start start 2024-04-17T10:48:54.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.437+0000 7f71aa69b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71a406b350 0x7f71a413fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.437+0000 7f71aa69b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71a40b4040 0x7f71a4143120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.437+0000 7f71aa69b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71a4154de0 0x7f71a4153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.437+0000 7f71aa69b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a41551c0 con 0x7f71a4154de0 2024-04-17T10:48:54.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.437+0000 7f71aa69b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a415b850 con 0x7f71a40b4040 2024-04-17T10:48:54.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.437+0000 7f71aa69b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a40b4450 con 0x7f71a406b350 2024-04-17T10:48:54.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.437+0000 7f71a8c38640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71a4154de0 0x7f71a4153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.437+0000 7f71a8c38640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71a4154de0 0x7f71a4153bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44678/0 (socket says 172.21.15.83:44678) 2024-04-17T10:48:54.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.437+0000 7f71a8c38640 1 -- 172.21.15.83:0/3968314228 learned_addr learned my addr 172.21.15.83:0/3968314228 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:54.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.437+0000 7f71a37fe640 1 --2- 172.21.15.83:0/3968314228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71a40b4040 0x7f71a4143120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.437+0000 7f71a3fff640 1 --2- 172.21.15.83:0/3968314228 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71a406b350 0x7f71a413fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.437+0000 7f71a8c38640 1 -- 172.21.15.83:0/3968314228 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71a406b350 msgr2=0x7f71a413fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.437+0000 7f71a8c38640 1 --2- 172.21.15.83:0/3968314228 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71a406b350 0x7f71a413fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.437+0000 7f71a8c38640 1 -- 172.21.15.83:0/3968314228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71a40b4040 msgr2=0x7f71a4143120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.437+0000 7f71a8c38640 1 --2- 172.21.15.83:0/3968314228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71a40b4040 0x7f71a4143120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.437+0000 7f71a8c38640 1 -- 172.21.15.83:0/3968314228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a415d890 con 0x7f71a4154de0 2024-04-17T10:48:54.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.437+0000 7f71a3fff640 1 --2- 172.21.15.83:0/3968314228 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71a406b350 0x7f71a413fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:54.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.437+0000 7f71a37fe640 1 --2- 172.21.15.83:0/3968314228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71a40b4040 0x7f71a4143120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:48:54.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.437+0000 7f71a8c38640 1 --2- 172.21.15.83:0/3968314228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71a4154de0 0x7f71a4153bd0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f719c00c820 tx=0x7f719c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:54.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.437+0000 7f71a0ff9640 1 -- 172.21.15.83:0/3968314228 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c015020 con 0x7f71a4154de0 2024-04-17T10:48:54.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.437+0000 7f71a0ff9640 1 -- 172.21.15.83:0/3968314228 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719c004240 con 0x7f71a4154de0 2024-04-17T10:48:54.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.437+0000 7f71aa69b640 1 -- 172.21.15.83:0/3968314228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a4143660 con 0x7f71a4154de0 2024-04-17T10:48:54.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.437+0000 7f71a0ff9640 1 -- 172.21.15.83:0/3968314228 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c003620 con 0x7f71a4154de0 2024-04-17T10:48:54.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.437+0000 7f71aa69b640 1 -- 172.21.15.83:0/3968314228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a4159d40 con 0x7f71a4154de0 2024-04-17T10:48:54.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.437+0000 7f71a0ff9640 1 -- 172.21.15.83:0/3968314228 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f719c018020 con 0x7f71a4154de0 2024-04-17T10:48:54.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.441+0000 7f71a0ff9640 1 --2- 172.21.15.83:0/3968314228 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7190066f60 0x7f7190069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.441+0000 7f71a0ff9640 1 -- 172.21.15.83:0/3968314228 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f719c084f90 con 0x7f71a4154de0 2024-04-17T10:48:54.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.441+0000 7f71a3fff640 1 --2- 172.21.15.83:0/3968314228 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7190066f60 0x7f7190069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.441+0000 7f71a3fff640 1 --2- 172.21.15.83:0/3968314228 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7190066f60 0x7f7190069420 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f71940ae000 tx=0x7f71940a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:54.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.441+0000 7f71ad808000 1 Processor -- start 2024-04-17T10:48:54.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.441+0000 7f71ad808000 1 -- start start 2024-04-17T10:48:54.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.441+0000 7f71ad808000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5594bc174a40 0x5594bc174e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.441+0000 7f71ad808000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bc175360 0x5594bc12f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.441+0000 7f71ad808000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594bc12fd90 0x5594bc130170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.441+0000 7f71ad808000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594bc0121f0 con 0x5594bc175360 2024-04-17T10:48:54.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.441+0000 7f71ad808000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594bbf78e90 con 0x5594bc12fd90 2024-04-17T10:48:54.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.441+0000 7f71ad808000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594bbf00f50 con 0x5594bc174a40 2024-04-17T10:48:54.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.441+0000 7f71a37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bc175360 0x5594bc12f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.441+0000 7f71a37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bc175360 0x5594bc12f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44696/0 (socket says 172.21.15.83:44696) 2024-04-17T10:48:54.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.441+0000 7f71a37fe640 1 -- 172.21.15.83:0/3527244971 learned_addr learned my addr 172.21.15.83:0/3527244971 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:54.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.441+0000 7f71a8c38640 1 --2- 172.21.15.83:0/3527244971 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594bc12fd90 0x5594bc130170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.441+0000 7f71a3fff640 1 --2- 172.21.15.83:0/3527244971 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5594bc174a40 0x5594bc174e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.441+0000 7f71a37fe640 1 -- 172.21.15.83:0/3527244971 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5594bc174a40 msgr2=0x5594bc174e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:54.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.441+0000 7f71a37fe640 1 --2- 172.21.15.83:0/3527244971 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5594bc174a40 0x5594bc174e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.441+0000 7f71a37fe640 1 -- 172.21.15.83:0/3527244971 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594bc12fd90 msgr2=0x5594bc130170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:54.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.441+0000 7f71a37fe640 1 --2- 172.21.15.83:0/3527244971 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594bc12fd90 0x5594bc130170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.441+0000 7f71a37fe640 1 -- 172.21.15.83:0/3527244971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594bbf508b0 con 0x5594bc175360 2024-04-17T10:48:54.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.441+0000 7f71a3fff640 1 --2- 172.21.15.83:0/3527244971 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5594bc174a40 0x5594bc174e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:54.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.441+0000 7f71a37fe640 1 --2- 172.21.15.83:0/3527244971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bc175360 0x5594bc12f850 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f719800ca30 tx=0x7f719800cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eefc4bc02fc69e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:54.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.445+0000 7f7189ffb640 1 -- 172.21.15.83:0/3527244971 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719800eb40 con 0x5594bc175360 2024-04-17T10:48:54.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.445+0000 7f7189ffb640 1 -- 172.21.15.83:0/3527244971 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7198019070 con 0x5594bc175360 2024-04-17T10:48:54.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.445+0000 7f7189ffb640 1 -- 172.21.15.83:0/3527244971 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7198014400 con 0x5594bc175360 2024-04-17T10:48:54.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.445+0000 7f71ad808000 1 -- 172.21.15.83:0/3527244971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bc175360 msgr2=0x5594bc12f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.445+0000 7f71ad808000 1 --2- 172.21.15.83:0/3527244971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bc175360 0x5594bc12f850 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f719800ca30 tx=0x7f719800cef0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.445+0000 7f71ad808000 1 -- 172.21.15.83:0/3527244971 shutdown_connections 2024-04-17T10:48:54.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.445+0000 7f71ad808000 1 --2- 172.21.15.83:0/3527244971 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594bc12fd90 0x5594bc130170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.445+0000 7f71ad808000 1 --2- 172.21.15.83:0/3527244971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bc175360 0x5594bc12f850 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.445+0000 7f71ad808000 1 --2- 172.21.15.83:0/3527244971 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5594bc174a40 0x5594bc174e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.445+0000 7f71ad808000 1 -- 172.21.15.83:0/3527244971 >> 172.21.15.83:0/3527244971 conn(0x5594bc12bd40 msgr2=0x5594bc12e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:54.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.445+0000 7f71ad808000 1 -- 172.21.15.83:0/3527244971 shutdown_connections 2024-04-17T10:48:54.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.445+0000 7f71ad808000 1 -- 172.21.15.83:0/3527244971 wait complete. 2024-04-17T10:48:54.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.445+0000 7f71ad808000 1 -- start start 2024-04-17T10:48:54.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.445+0000 7f71ad808000 1 Processor -- start 2024-04-17T10:48:54.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.445+0000 7f71ad808000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5594bc12fd90 0x5594bc17c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.445+0000 7f71ad808000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bc174a40 0x5594bc17c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.445+0000 7f71ad808000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594bc175360 0x5594bc135e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.445+0000 7f71ad808000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594bbf33540 con 0x5594bc174a40 2024-04-17T10:48:54.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.445+0000 7f71ad808000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594bbf74f30 con 0x5594bc175360 2024-04-17T10:48:54.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.445+0000 7f71ad808000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594bbf3ae50 con 0x5594bc12fd90 2024-04-17T10:48:54.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.445+0000 7f71ad808000 10 client.0 _ll_register_callbacks cb 0x5594bc02d180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:48:54.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.445+0000 7f71a37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bc174a40 0x5594bc17c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.445+0000 7f71a37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bc174a40 0x5594bc17c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44712/0 (socket says 172.21.15.83:44712) 2024-04-17T10:48:54.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.445+0000 7f71a37fe640 1 -- 172.21.15.83:0/1516127260 learned_addr learned my addr 172.21.15.83:0/1516127260 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:54.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.449+0000 7f71a8c38640 1 --2- 172.21.15.83:0/1516127260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594bc175360 0x5594bc135e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.449+0000 7f71a3fff640 1 --2- 172.21.15.83:0/1516127260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5594bc12fd90 0x5594bc17c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.449+0000 7f71a37fe640 1 -- 172.21.15.83:0/1516127260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5594bc12fd90 msgr2=0x5594bc17c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:54.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.449+0000 7f71a37fe640 1 --2- 172.21.15.83:0/1516127260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5594bc12fd90 0x5594bc17c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.449+0000 7f71a37fe640 1 -- 172.21.15.83:0/1516127260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594bc175360 msgr2=0x5594bc135e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:54.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.449+0000 7f71a37fe640 1 --2- 172.21.15.83:0/1516127260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594bc175360 0x5594bc135e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.449+0000 7f71a37fe640 1 -- 172.21.15.83:0/1516127260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594bbf011a0 con 0x5594bc174a40 2024-04-17T10:48:54.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.449+0000 7f71a8c38640 1 --2- 172.21.15.83:0/1516127260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594bc175360 0x5594bc135e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:54.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.449+0000 7f71a37fe640 1 --2- 172.21.15.83:0/1516127260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bc174a40 0x5594bc17c700 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f7198016040 tx=0x7f7198004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e982da217576773 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:54.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.449+0000 7f7182ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:48:54.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.449+0000 7f7182ffd640 1 -- 172.21.15.83:0/1516127260 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7198014b40 con 0x5594bc174a40 2024-04-17T10:48:54.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.449+0000 7f7182ffd640 1 -- 172.21.15.83:0/1516127260 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7198014ce0 con 0x5594bc174a40 2024-04-17T10:48:54.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.449+0000 7f71ad808000 1 -- 172.21.15.83:0/1516127260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7198019070 con 0x5594bc174a40 2024-04-17T10:48:54.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.449+0000 7f7182ffd640 1 -- 172.21.15.83:0/1516127260 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7198030050 con 0x5594bc174a40 2024-04-17T10:48:54.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.449+0000 7f71ad808000 10 client.8820 fetch_fsmap learned FSMap version 69 2024-04-17T10:48:54.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.449+0000 7f71ad808000 1 -- 172.21.15.83:0/1516127260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x5594bbffb840 con 0x5594bc174a40 2024-04-17T10:48:54.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.449+0000 7f7182ffd640 1 -- 172.21.15.83:0/1516127260 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7198019070 con 0x5594bc174a40 2024-04-17T10:48:54.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.453+0000 7f7182ffd640 1 -- 172.21.15.83:0/1516127260 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7198033020 con 0x5594bc174a40 2024-04-17T10:48:54.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.453+0000 7f71ad808000 10 client.8820 fetch_fsmap finished waiting for FSMap version 69 2024-04-17T10:48:54.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.453+0000 7f71ad808000 10 client.8820 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-17T10:48:54.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.453+0000 7f71ad808000 20 client.8820 populate_metadata read hostname 'smithi083' 2024-04-17T10:48:54.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.453+0000 7f71ad808000 1 --2- 172.21.15.83:0/1516127260 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x5594bc136ae0 0x5594bc136ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.453+0000 7f71ad808000 4 client.8820 mds_command: new command op to 8604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:48:54.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.453+0000 7f71ad808000 1 -- 172.21.15.83:0/1516127260 --> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5594bbf14be0 con 0x5594bc136ae0 2024-04-17T10:48:54.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.453+0000 7f71a3fff640 1 --2- 172.21.15.83:0/1516127260 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x5594bc136ae0 0x5594bc136ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.453+0000 7f71a3fff640 1 --2- 172.21.15.83:0/1516127260 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x5594bc136ae0 0x5594bc136ec0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9372714eaf124efa server_cookie=e371060414c38497 in_seq=0 out_seq=0 2024-04-17T10:48:54.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.453+0000 7f7182ffd640 10 client.8820 ms_handle_connect on v2:172.21.15.83:6836/1479417998 2024-04-17T10:48:54.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.453+0000 7f7182ffd640 1 -- 172.21.15.83:0/1516127260 <== mds.0 v2:172.21.15.83:6836/1479417998 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5594bbf14be0 con 0x5594bc136ae0 2024-04-17T10:48:54.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.453+0000 7f7182ffd640 10 client.8820 handle_command_reply: tid=0 2024-04-17T10:48:54.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.453+0000 7f71ad808000 1 client.8820 shutdown 2024-04-17T10:48:54.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.453+0000 7f71ad808000 1 -- 172.21.15.83:0/1516127260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bc174a40 msgr2=0x5594bc17c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.453+0000 7f71ad808000 1 --2- 172.21.15.83:0/1516127260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bc174a40 0x5594bc17c700 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f7198016040 tx=0x7f7198004740 comp rx=0 tx=0).stop 2024-04-17T10:48:54.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.453+0000 7f71ad808000 1 -- 172.21.15.83:0/1516127260 shutdown_connections 2024-04-17T10:48:54.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.453+0000 7f71ad808000 1 --2- 172.21.15.83:0/1516127260 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x5594bc136ae0 0x5594bc136ec0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.453+0000 7f71ad808000 1 --2- 172.21.15.83:0/1516127260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594bc175360 0x5594bc135e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.453+0000 7f7182ffd640 0 client.8820 ms_handle_reset on v2:172.21.15.83:6836/1479417998 2024-04-17T10:48:54.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.453+0000 7f71ad808000 1 --2- 172.21.15.83:0/1516127260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bc174a40 0x5594bc17c700 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.453+0000 7f71ad808000 1 --2- 172.21.15.83:0/1516127260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5594bc12fd90 0x5594bc17c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.453+0000 7f71ad808000 1 -- 172.21.15.83:0/1516127260 >> 172.21.15.83:0/1516127260 conn(0x5594bc12bd40 msgr2=0x5594bc12e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:54.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.453+0000 7f71ad808000 1 -- 172.21.15.83:0/1516127260 shutdown_connections 2024-04-17T10:48:54.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.453+0000 7f71ad808000 1 -- 172.21.15.83:0/1516127260 wait complete. 2024-04-17T10:48:54.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.453+0000 7f71ad808000 20 client.8820 trim_cache size 0 max 16384 2024-04-17T10:48:54.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.469+0000 7f71ad808000 1 Processor -- start 2024-04-17T10:48:54.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.469+0000 7f71ad808000 1 -- start start 2024-04-17T10:48:54.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.469+0000 7f71ad808000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bc12fd90 0x5594bc1d1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.469+0000 7f71ad808000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5594bc174a40 0x5594bc1d17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.469+0000 7f71ad808000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594bc175360 0x5594bc1d1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.469+0000 7f71ad808000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a4202ec0 con 0x5594bc12fd90 2024-04-17T10:48:54.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.469+0000 7f71ad808000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a4202080 con 0x5594bc175360 2024-04-17T10:48:54.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.469+0000 7f71ad808000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a4141e00 con 0x5594bc174a40 2024-04-17T10:48:54.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71a3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bc12fd90 0x5594bc1d1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71a3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bc12fd90 0x5594bc1d1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44718/0 (socket says 172.21.15.83:44718) 2024-04-17T10:48:54.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71a3fff640 1 -- 172.21.15.83:0/2582345668 learned_addr learned my addr 172.21.15.83:0/2582345668 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:54.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71a8c38640 1 --2- 172.21.15.83:0/2582345668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594bc175360 0x5594bc1d1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71a3fff640 1 -- 172.21.15.83:0/2582345668 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5594bc174a40 msgr2=0x5594bc1d17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:54.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71a3fff640 1 --2- 172.21.15.83:0/2582345668 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5594bc174a40 0x5594bc1d17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71a3fff640 1 -- 172.21.15.83:0/2582345668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594bc175360 msgr2=0x5594bc1d1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:54.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71a3fff640 1 --2- 172.21.15.83:0/2582345668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594bc175360 0x5594bc1d1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71a3fff640 1 -- 172.21.15.83:0/2582345668 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594bc175190 con 0x5594bc12fd90 2024-04-17T10:48:54.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71a3fff640 1 --2- 172.21.15.83:0/2582345668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bc12fd90 0x5594bc1d1290 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f71940981e0 tx=0x7f7194004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffb0bbe3595499e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:54.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71897fa640 1 -- 172.21.15.83:0/2582345668 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71940ce030 con 0x5594bc12fd90 2024-04-17T10:48:54.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71897fa640 1 -- 172.21.15.83:0/2582345668 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71940a6890 con 0x5594bc12fd90 2024-04-17T10:48:54.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71897fa640 1 -- 172.21.15.83:0/2582345668 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71940a6a40 con 0x5594bc12fd90 2024-04-17T10:48:54.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71ad808000 1 -- 172.21.15.83:0/2582345668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bc12fd90 msgr2=0x5594bc1d1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71ad808000 1 --2- 172.21.15.83:0/2582345668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bc12fd90 0x5594bc1d1290 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f71940981e0 tx=0x7f7194004ac0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71ad808000 1 -- 172.21.15.83:0/2582345668 shutdown_connections 2024-04-17T10:48:54.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71ad808000 1 --2- 172.21.15.83:0/2582345668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594bc175360 0x5594bc1d1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71ad808000 1 --2- 172.21.15.83:0/2582345668 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5594bc174a40 0x5594bc1d17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71ad808000 1 --2- 172.21.15.83:0/2582345668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bc12fd90 0x5594bc1d1290 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71ad808000 1 -- 172.21.15.83:0/2582345668 >> 172.21.15.83:0/2582345668 conn(0x5594bc12bd40 msgr2=0x5594bc1730a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:54.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71ad808000 1 -- 172.21.15.83:0/2582345668 shutdown_connections 2024-04-17T10:48:54.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71ad808000 1 -- 172.21.15.83:0/2582345668 wait complete. 2024-04-17T10:48:54.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71ad808000 1 -- start start 2024-04-17T10:48:54.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71ad808000 1 Processor -- start 2024-04-17T10:48:54.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71ad808000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5594bc12fd90 0x5594bc1d56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71ad808000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bc174a40 0x5594bc1d5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71ad808000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594bc175360 0x5594bc179ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71ad808000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594bc035d10 con 0x5594bc174a40 2024-04-17T10:48:54.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71ad808000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a4201a80 con 0x5594bc175360 2024-04-17T10:48:54.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71ad808000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594bbf74f30 con 0x5594bc12fd90 2024-04-17T10:48:54.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71ad808000 10 client.0 _ll_register_callbacks cb 0x5594bbf3ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:48:54.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71a37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bc174a40 0x5594bc1d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71a37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bc174a40 0x5594bc1d5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44730/0 (socket says 172.21.15.83:44730) 2024-04-17T10:48:54.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71a37fe640 1 -- 172.21.15.83:0/330730678 learned_addr learned my addr 172.21.15.83:0/330730678 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:54.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71a3fff640 1 --2- 172.21.15.83:0/330730678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5594bc12fd90 0x5594bc1d56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71a8c38640 1 --2- 172.21.15.83:0/330730678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594bc175360 0x5594bc179ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71a37fe640 1 -- 172.21.15.83:0/330730678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5594bc12fd90 msgr2=0x5594bc1d56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:54.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71a37fe640 1 --2- 172.21.15.83:0/330730678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5594bc12fd90 0x5594bc1d56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71a37fe640 1 -- 172.21.15.83:0/330730678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594bc175360 msgr2=0x5594bc179ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:54.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71a37fe640 1 --2- 172.21.15.83:0/330730678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594bc175360 0x5594bc179ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71a37fe640 1 -- 172.21.15.83:0/330730678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594bbfd2a10 con 0x5594bc174a40 2024-04-17T10:48:54.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f71a37fe640 1 --2- 172.21.15.83:0/330730678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bc174a40 0x5594bc1d5c00 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f719800cf30 tx=0x7f71980046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c119fef82168f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:54.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f7182ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:48:54.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.473+0000 7f7182ffd640 1 -- 172.21.15.83:0/330730678 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7198004240 con 0x5594bc174a40 2024-04-17T10:48:54.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f7182ffd640 1 -- 172.21.15.83:0/330730678 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f71980043e0 con 0x5594bc174a40 2024-04-17T10:48:54.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71ad808000 1 -- 172.21.15.83:0/330730678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71940a6890 con 0x5594bc174a40 2024-04-17T10:48:54.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f7182ffd640 1 -- 172.21.15.83:0/330730678 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7198041050 con 0x5594bc174a40 2024-04-17T10:48:54.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71ad808000 10 client.8826 fetch_fsmap learned FSMap version 69 2024-04-17T10:48:54.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71ad808000 1 -- 172.21.15.83:0/330730678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x5594bbf0db00 con 0x5594bc174a40 2024-04-17T10:48:54.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f7182ffd640 1 -- 172.21.15.83:0/330730678 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71940a6890 con 0x5594bc174a40 2024-04-17T10:48:54.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f7182ffd640 1 -- 172.21.15.83:0/330730678 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7198044020 con 0x5594bc174a40 2024-04-17T10:48:54.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71ad808000 10 client.8826 fetch_fsmap finished waiting for FSMap version 69 2024-04-17T10:48:54.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71ad808000 10 client.8826 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-17T10:48:54.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71ad808000 20 client.8826 populate_metadata read hostname 'smithi083' 2024-04-17T10:48:54.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71ad808000 1 --2- 172.21.15.83:0/330730678 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x5594bc17a800 0x5594bc17ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71ad808000 4 client.8826 mds_command: new command op to 8604 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-17T10:48:54.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71ad808000 1 -- 172.21.15.83:0/330730678 --> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x5594bbf863d0 con 0x5594bc17a800 2024-04-17T10:48:54.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71a3fff640 1 --2- 172.21.15.83:0/330730678 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x5594bc17a800 0x5594bc17ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f7182ffd640 10 client.8826 ms_handle_connect on v2:172.21.15.83:6836/1479417998 2024-04-17T10:48:54.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71a3fff640 1 --2- 172.21.15.83:0/330730678 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x5594bc17a800 0x5594bc17ac10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6d3cf7f1c6ab006 server_cookie=f1b3bdd8581e7425 in_seq=0 out_seq=0 2024-04-17T10:48:54.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f7182ffd640 1 -- 172.21.15.83:0/330730678 <== mds.0 v2:172.21.15.83:6836/1479417998 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x5594bbf863d0 con 0x5594bc17a800 2024-04-17T10:48:54.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f7182ffd640 10 client.8826 handle_command_reply: tid=0 2024-04-17T10:48:54.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71ad808000 1 client.8826 shutdown 2024-04-17T10:48:54.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71ad808000 1 -- 172.21.15.83:0/330730678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bc174a40 msgr2=0x5594bc1d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71ad808000 1 --2- 172.21.15.83:0/330730678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bc174a40 0x5594bc1d5c00 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f719800cf30 tx=0x7f71980046d0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71ad808000 1 -- 172.21.15.83:0/330730678 shutdown_connections 2024-04-17T10:48:54.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71ad808000 1 --2- 172.21.15.83:0/330730678 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x5594bc17a800 0x5594bc17ac10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71ad808000 1 --2- 172.21.15.83:0/330730678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594bc175360 0x5594bc179ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71ad808000 1 --2- 172.21.15.83:0/330730678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594bc174a40 0x5594bc1d5c00 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71ad808000 1 --2- 172.21.15.83:0/330730678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5594bc12fd90 0x5594bc1d56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71ad808000 1 -- 172.21.15.83:0/330730678 >> 172.21.15.83:0/330730678 conn(0x5594bc12bd40 msgr2=0x5594bc173930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:54.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f7182ffd640 0 client.8826 ms_handle_reset on v2:172.21.15.83:6836/1479417998 2024-04-17T10:48:54.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71ad808000 1 -- 172.21.15.83:0/330730678 shutdown_connections 2024-04-17T10:48:54.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71ad808000 1 -- 172.21.15.83:0/330730678 wait complete. 2024-04-17T10:48:54.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71ad808000 20 client.8826 trim_cache size 0 max 16384 2024-04-17T10:48:54.479 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:48:54.479 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0 2024-04-17T10:48:54.479 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:48:54.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71aa69b640 1 -- 172.21.15.83:0/3968314228 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7190066f60 msgr2=0x7f7190069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71aa69b640 1 --2- 172.21.15.83:0/3968314228 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7190066f60 0x7f7190069420 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f71940ae000 tx=0x7f71940a7000 comp rx=0 tx=0).stop 2024-04-17T10:48:54.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71aa69b640 1 -- 172.21.15.83:0/3968314228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71a4154de0 msgr2=0x7f71a4153bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71aa69b640 1 --2- 172.21.15.83:0/3968314228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71a4154de0 0x7f71a4153bd0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f719c00c820 tx=0x7f719c00cce0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71aa69b640 1 -- 172.21.15.83:0/3968314228 shutdown_connections 2024-04-17T10:48:54.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71aa69b640 1 --2- 172.21.15.83:0/3968314228 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7190066f60 0x7f7190069420 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71aa69b640 1 --2- 172.21.15.83:0/3968314228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71a4154de0 0x7f71a4153bd0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71aa69b640 1 --2- 172.21.15.83:0/3968314228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71a40b4040 0x7f71a4143120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71aa69b640 1 --2- 172.21.15.83:0/3968314228 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71a406b350 0x7f71a413fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71aa69b640 1 -- 172.21.15.83:0/3968314228 >> 172.21.15.83:0/3968314228 conn(0x7f71a40b15b0 msgr2=0x7f71a40b0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:54.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71aa69b640 1 -- 172.21.15.83:0/3968314228 shutdown_connections 2024-04-17T10:48:54.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.477+0000 7f71aa69b640 1 -- 172.21.15.83:0/3968314228 wait complete. 2024-04-17T10:48:54.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:54.490 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-17T10:48:54.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.609+0000 7fd943970640 1 Processor -- start 2024-04-17T10:48:54.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.609+0000 7fd943970640 1 -- start start 2024-04-17T10:48:54.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.609+0000 7fd943970640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd93c0cd5c0 0x7fd93c0720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.609+0000 7fd943970640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd93c09e100 con 0x7fd93c15bd50 2024-04-17T10:48:54.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.609+0000 7fd943970640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd93c0cc520 con 0x7fd93c0725f0 2024-04-17T10:48:54.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.609+0000 7fd943970640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd93c072a60 con 0x7fd93c0cd5c0 2024-04-17T10:48:54.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.609+0000 7fd940f0b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd93c0cd5c0 0x7fd93c0720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.609+0000 7fd940f0b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd93c0cd5c0 0x7fd93c0720b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44744/0 (socket says 172.21.15.83:44744) 2024-04-17T10:48:54.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.609+0000 7fd940f0b640 1 -- 172.21.15.83:0/2948919487 learned_addr learned my addr 172.21.15.83:0/2948919487 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:54.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.609+0000 7fd94170c640 1 --1- 172.21.15.83:0/2948919487 >> v1:172.21.15.83:6789/0 conn(0x7fd93c15bd50 0x7fd93c0669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55906/0 (socket says 172.21.15.83:55906) 2024-04-17T10:48:54.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.609+0000 7fd940f0b640 1 -- 172.21.15.83:0/2948919487 >> v1:172.21.15.167:6789/0 conn(0x7fd93c0725f0 legacy=0x7fd93c0ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:54.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.609+0000 7fd940f0b640 1 -- 172.21.15.83:0/2948919487 >> v1:172.21.15.83:6789/0 conn(0x7fd93c15bd50 legacy=0x7fd93c0669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.609+0000 7fd940f0b640 1 -- 172.21.15.83:0/2948919487 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd93c0cc800 con 0x7fd93c0cd5c0 2024-04-17T10:48:54.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.609+0000 7fd933fff640 1 -- 172.21.15.83:0/2948919487 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3574481428 0 0) 0x7fd93c09e100 con 0x7fd93c15bd50 2024-04-17T10:48:54.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd940f0b640 1 --2- 172.21.15.83:0/2948919487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd93c0cd5c0 0x7fd93c0720b0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fd92c009870 tx=0x7fd92c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9005100438dc8aa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:54.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd933fff640 1 -- 172.21.15.83:0/2948919487 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92c0b1070 con 0x7fd93c0cd5c0 2024-04-17T10:48:54.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd933fff640 1 -- 172.21.15.83:0/2948919487 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92c0ac600 con 0x7fd93c0cd5c0 2024-04-17T10:48:54.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd933fff640 1 -- 172.21.15.83:0/2948919487 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92c0ac9d0 con 0x7fd93c0cd5c0 2024-04-17T10:48:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd943970640 1 -- 172.21.15.83:0/2948919487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd93c0cd5c0 msgr2=0x7fd93c0720b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd943970640 1 --2- 172.21.15.83:0/2948919487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd93c0cd5c0 0x7fd93c0720b0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fd92c009870 tx=0x7fd92c01c490 comp rx=0 tx=0).stop 2024-04-17T10:48:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd943970640 1 -- 172.21.15.83:0/2948919487 shutdown_connections 2024-04-17T10:48:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd943970640 1 --2- 172.21.15.83:0/2948919487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd93c0cd5c0 0x7fd93c0720b0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd943970640 1 -- 172.21.15.83:0/2948919487 >> 172.21.15.83:0/2948919487 conn(0x7fd93c0b3b40 msgr2=0x7fd93c0b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:54.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd943970640 1 -- 172.21.15.83:0/2948919487 shutdown_connections 2024-04-17T10:48:54.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd943970640 1 -- 172.21.15.83:0/2948919487 wait complete. 2024-04-17T10:48:54.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd943970640 1 Processor -- start 2024-04-17T10:48:54.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd943970640 1 -- start start 2024-04-17T10:48:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd943970640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd93c0725f0 0x7fd93c13e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd943970640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd93c0cd5c0 0x7fd93c157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd943970640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd93c15bd50 0x7fd93c09c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd943970640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93c169700 con 0x7fd93c15bd50 2024-04-17T10:48:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd943970640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93c0cd130 con 0x7fd93c0725f0 2024-04-17T10:48:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd943970640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd93c169880 con 0x7fd93c0cd5c0 2024-04-17T10:48:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd941f0d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd93c15bd50 0x7fd93c09c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd941f0d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd93c15bd50 0x7fd93c09c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44752/0 (socket says 172.21.15.83:44752) 2024-04-17T10:48:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd941f0d640 1 -- 172.21.15.83:0/522478146 learned_addr learned my addr 172.21.15.83:0/522478146 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd94170c640 1 --2- 172.21.15.83:0/522478146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd93c0725f0 0x7fd93c13e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd940f0b640 1 --2- 172.21.15.83:0/522478146 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd93c0cd5c0 0x7fd93c157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd941f0d640 1 -- 172.21.15.83:0/522478146 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd93c0cd5c0 msgr2=0x7fd93c157dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd941f0d640 1 --2- 172.21.15.83:0/522478146 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd93c0cd5c0 0x7fd93c157dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd941f0d640 1 -- 172.21.15.83:0/522478146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd93c0725f0 msgr2=0x7fd93c13e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd941f0d640 1 --2- 172.21.15.83:0/522478146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd93c0725f0 0x7fd93c13e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd941f0d640 1 -- 172.21.15.83:0/522478146 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd93c1599d0 con 0x7fd93c15bd50 2024-04-17T10:48:54.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd94170c640 1 --2- 172.21.15.83:0/522478146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd93c0725f0 0x7fd93c13e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:54.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd941f0d640 1 --2- 172.21.15.83:0/522478146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd93c15bd50 0x7fd93c09c4b0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fd93400c7b0 tx=0x7fd93400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:54.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd931ffb640 1 -- 172.21.15.83:0/522478146 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd934004b70 con 0x7fd93c15bd50 2024-04-17T10:48:54.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd931ffb640 1 -- 172.21.15.83:0/522478146 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd934004d10 con 0x7fd93c15bd50 2024-04-17T10:48:54.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.613+0000 7fd931ffb640 1 -- 172.21.15.83:0/522478146 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93400aa90 con 0x7fd93c15bd50 2024-04-17T10:48:54.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.617+0000 7fd943970640 1 -- 172.21.15.83:0/522478146 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd93c156490 con 0x7fd93c15bd50 2024-04-17T10:48:54.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.617+0000 7fd943970640 1 -- 172.21.15.83:0/522478146 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd93c156a50 con 0x7fd93c15bd50 2024-04-17T10:48:54.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.617+0000 7fd931ffb640 1 -- 172.21.15.83:0/522478146 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd93400ac30 con 0x7fd93c15bd50 2024-04-17T10:48:54.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.617+0000 7fd931ffb640 1 --2- 172.21.15.83:0/522478146 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd920066f60 0x7fd920069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.617+0000 7fd931ffb640 1 -- 172.21.15.83:0/522478146 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd934085320 con 0x7fd93c15bd50 2024-04-17T10:48:54.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.617+0000 7fd94170c640 1 --2- 172.21.15.83:0/522478146 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd920066f60 0x7fd920069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.617+0000 7fd94170c640 1 --2- 172.21.15.83:0/522478146 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd920066f60 0x7fd920069420 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd938007fd0 tx=0x7fd938009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:54.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.617+0000 7fd946add000 1 Processor -- start 2024-04-17T10:48:54.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.617+0000 7fd946add000 1 -- start start 2024-04-17T10:48:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.617+0000 7fd946add000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e449bee30 0x564e449bf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.617+0000 7fd946add000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564e449bf750 0x564e44974f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.617+0000 7fd946add000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564e449754b0 0x564e44975890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.617+0000 7fd946add000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e447bdff0 con 0x564e449bee30 2024-04-17T10:48:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.617+0000 7fd946add000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e44746150 con 0x564e449bf750 2024-04-17T10:48:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.617+0000 7fd946add000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e447ba090 con 0x564e449754b0 2024-04-17T10:48:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.617+0000 7fd94170c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e449bee30 0x564e449bf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.617+0000 7fd94170c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e449bee30 0x564e449bf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44766/0 (socket says 172.21.15.83:44766) 2024-04-17T10:48:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd94170c640 1 -- 172.21.15.83:0/3988796985 learned_addr learned my addr 172.21.15.83:0/3988796985 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd94170c640 1 -- 172.21.15.83:0/3988796985 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564e449754b0 msgr2=0x564e44975890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd941f0d640 1 --2- 172.21.15.83:0/3988796985 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564e449754b0 0x564e44975890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd94170c640 1 --2- 172.21.15.83:0/3988796985 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564e449754b0 0x564e44975890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd94170c640 1 -- 172.21.15.83:0/3988796985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564e449bf750 msgr2=0x564e44974f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd94170c640 1 --2- 172.21.15.83:0/3988796985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564e449bf750 0x564e44974f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd94170c640 1 -- 172.21.15.83:0/3988796985 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e449540d0 con 0x564e449bee30 2024-04-17T10:48:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd94170c640 1 --2- 172.21.15.83:0/3988796985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e449bee30 0x564e449bf210 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fd938007360 tx=0x7fd938010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d35b2f53c1971392 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd91effd640 1 -- 172.21.15.83:0/3988796985 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93800d9a0 con 0x564e449bee30 2024-04-17T10:48:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd91effd640 1 -- 172.21.15.83:0/3988796985 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93800db40 con 0x564e449bee30 2024-04-17T10:48:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd91effd640 1 -- 172.21.15.83:0/3988796985 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd938020450 con 0x564e449bee30 2024-04-17T10:48:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd946add000 1 -- 172.21.15.83:0/3988796985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e449bee30 msgr2=0x564e449bf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd946add000 1 --2- 172.21.15.83:0/3988796985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e449bee30 0x564e449bf210 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fd938007360 tx=0x7fd938010000 comp rx=0 tx=0).stop 2024-04-17T10:48:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd946add000 1 -- 172.21.15.83:0/3988796985 shutdown_connections 2024-04-17T10:48:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd946add000 1 --2- 172.21.15.83:0/3988796985 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564e449754b0 0x564e44975890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd946add000 1 --2- 172.21.15.83:0/3988796985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564e449bf750 0x564e44974f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd946add000 1 --2- 172.21.15.83:0/3988796985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e449bee30 0x564e449bf210 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd946add000 1 -- 172.21.15.83:0/3988796985 >> 172.21.15.83:0/3988796985 conn(0x564e4496d020 msgr2=0x564e4496f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd946add000 1 -- 172.21.15.83:0/3988796985 shutdown_connections 2024-04-17T10:48:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd946add000 1 -- 172.21.15.83:0/3988796985 wait complete. 2024-04-17T10:48:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd946add000 1 -- start start 2024-04-17T10:48:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd946add000 1 Processor -- start 2024-04-17T10:48:54.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd946add000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e449754b0 0x564e449be810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd946add000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564e449bee30 0x564e4497b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd946add000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564e449bf750 0x564e4497b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd946add000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e447c2620 con 0x564e449754b0 2024-04-17T10:48:54.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd946add000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e44778690 con 0x564e449bee30 2024-04-17T10:48:54.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd946add000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e44746a40 con 0x564e449bf750 2024-04-17T10:48:54.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd946add000 10 client.0 _ll_register_callbacks cb 0x564e4495be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:48:54.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd94170c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e449754b0 0x564e449be810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd94170c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e449754b0 0x564e449be810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44782/0 (socket says 172.21.15.83:44782) 2024-04-17T10:48:54.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd94170c640 1 -- 172.21.15.83:0/1095842610 learned_addr learned my addr 172.21.15.83:0/1095842610 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:54.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd941f0d640 1 --2- 172.21.15.83:0/1095842610 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564e449bf750 0x564e4497b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd940f0b640 1 --2- 172.21.15.83:0/1095842610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564e449bee30 0x564e4497b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd94170c640 1 -- 172.21.15.83:0/1095842610 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564e449bf750 msgr2=0x564e4497b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:54.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd94170c640 1 --2- 172.21.15.83:0/1095842610 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564e449bf750 0x564e4497b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd94170c640 1 -- 172.21.15.83:0/1095842610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564e449bee30 msgr2=0x564e4497b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:54.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd94170c640 1 --2- 172.21.15.83:0/1095842610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564e449bee30 0x564e4497b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd94170c640 1 -- 172.21.15.83:0/1095842610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e4486b1c0 con 0x564e449754b0 2024-04-17T10:48:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd940f0b640 1 --2- 172.21.15.83:0/1095842610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564e449bee30 0x564e4497b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd94170c640 1 --2- 172.21.15.83:0/1095842610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e449754b0 0x564e449be810 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fd938007cc0 tx=0x7fd938012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbec0b653c135bdb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd917fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:48:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd917fff640 1 -- 172.21.15.83:0/1095842610 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9380224f0 con 0x564e449754b0 2024-04-17T10:48:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd917fff640 1 -- 172.21.15.83:0/1095842610 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd938022690 con 0x564e449754b0 2024-04-17T10:48:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd946add000 1 -- 172.21.15.83:0/1095842610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd93800db40 con 0x564e449754b0 2024-04-17T10:48:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd917fff640 1 -- 172.21.15.83:0/1095842610 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd938022890 con 0x564e449754b0 2024-04-17T10:48:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd946add000 10 client.8841 fetch_fsmap learned FSMap version 69 2024-04-17T10:48:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd946add000 1 -- 172.21.15.83:0/1095842610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x564e44867170 con 0x564e449754b0 2024-04-17T10:48:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.621+0000 7fd917fff640 1 -- 172.21.15.83:0/1095842610 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93800db40 con 0x564e449754b0 2024-04-17T10:48:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.625+0000 7fd917fff640 1 -- 172.21.15.83:0/1095842610 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd938022a30 con 0x564e449754b0 2024-04-17T10:48:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.625+0000 7fd946add000 10 client.8841 fetch_fsmap finished waiting for FSMap version 69 2024-04-17T10:48:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.625+0000 7fd946add000 10 client.8841 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-17T10:48:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.625+0000 7fd946add000 20 client.8841 populate_metadata read hostname 'smithi083' 2024-04-17T10:48:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.625+0000 7fd946add000 1 --2- 172.21.15.83:0/1095842610 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x564e4497c640 0x564e449d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.625+0000 7fd946add000 4 client.8841 mds_command: new command op to 8604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:48:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.625+0000 7fd946add000 1 -- 172.21.15.83:0/1095842610 --> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e44759c60 con 0x564e4497c640 2024-04-17T10:48:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.625+0000 7fd940f0b640 1 --2- 172.21.15.83:0/1095842610 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x564e4497c640 0x564e449d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.625+0000 7fd940f0b640 1 --2- 172.21.15.83:0/1095842610 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x564e4497c640 0x564e449d10f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa27b3db208722af server_cookie=4a437aeb7f695d59 in_seq=0 out_seq=0 2024-04-17T10:48:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.625+0000 7fd917fff640 10 client.8841 ms_handle_connect on v2:172.21.15.83:6836/1479417998 2024-04-17T10:48:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.625+0000 7fd917fff640 1 -- 172.21.15.83:0/1095842610 <== mds.0 v2:172.21.15.83:6836/1479417998 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564e44759c60 con 0x564e4497c640 2024-04-17T10:48:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.625+0000 7fd917fff640 10 client.8841 handle_command_reply: tid=0 2024-04-17T10:48:54.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.625+0000 7fd946add000 1 client.8841 shutdown 2024-04-17T10:48:54.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.625+0000 7fd946add000 1 -- 172.21.15.83:0/1095842610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e449754b0 msgr2=0x564e449be810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.625+0000 7fd946add000 1 --2- 172.21.15.83:0/1095842610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e449754b0 0x564e449be810 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fd938007cc0 tx=0x7fd938012000 comp rx=0 tx=0).stop 2024-04-17T10:48:54.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.625+0000 7fd946add000 1 -- 172.21.15.83:0/1095842610 shutdown_connections 2024-04-17T10:48:54.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.625+0000 7fd946add000 1 --2- 172.21.15.83:0/1095842610 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x564e4497c640 0x564e449d10f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.625+0000 7fd946add000 1 --2- 172.21.15.83:0/1095842610 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564e449bf750 0x564e4497b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.625+0000 7fd946add000 1 --2- 172.21.15.83:0/1095842610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564e449bee30 0x564e4497b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.625+0000 7fd946add000 1 --2- 172.21.15.83:0/1095842610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e449754b0 0x564e449be810 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.625+0000 7fd917fff640 0 client.8841 ms_handle_reset on v2:172.21.15.83:6836/1479417998 2024-04-17T10:48:54.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.625+0000 7fd946add000 1 -- 172.21.15.83:0/1095842610 >> 172.21.15.83:0/1095842610 conn(0x564e4496d020 msgr2=0x564e44976120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:54.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.625+0000 7fd946add000 1 -- 172.21.15.83:0/1095842610 shutdown_connections 2024-04-17T10:48:54.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.625+0000 7fd946add000 1 -- 172.21.15.83:0/1095842610 wait complete. 2024-04-17T10:48:54.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.625+0000 7fd946add000 20 client.8841 trim_cache size 0 max 16384 2024-04-17T10:48:54.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.641+0000 7fd946add000 1 Processor -- start 2024-04-17T10:48:54.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.641+0000 7fd946add000 1 -- start start 2024-04-17T10:48:54.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.641+0000 7fd946add000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e449754b0 0x564e449d3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.641+0000 7fd946add000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564e449bee30 0x564e449d3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.641+0000 7fd946add000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564e449bf750 0x564e4499cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.641+0000 7fd946add000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93c06e9e0 con 0x564e449754b0 2024-04-17T10:48:54.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.641+0000 7fd946add000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93c15b310 con 0x564e449bf750 2024-04-17T10:48:54.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.641+0000 7fd946add000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd93c15e250 con 0x564e449bee30 2024-04-17T10:48:54.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.641+0000 7fd94170c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e449754b0 0x564e449d3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.641+0000 7fd94170c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e449754b0 0x564e449d3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44786/0 (socket says 172.21.15.83:44786) 2024-04-17T10:48:54.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.641+0000 7fd94170c640 1 -- 172.21.15.83:0/1773008956 learned_addr learned my addr 172.21.15.83:0/1773008956 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:54.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.641+0000 7fd94170c640 1 -- 172.21.15.83:0/1773008956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564e449bee30 msgr2=0x564e449d3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:54.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.641+0000 7fd94170c640 1 --2- 172.21.15.83:0/1773008956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564e449bee30 0x564e449d3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.641+0000 7fd94170c640 1 -- 172.21.15.83:0/1773008956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564e449bf750 msgr2=0x564e4499cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:54.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.641+0000 7fd94170c640 1 --2- 172.21.15.83:0/1773008956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564e449bf750 0x564e4499cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.641+0000 7fd94170c640 1 -- 172.21.15.83:0/1773008956 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e44804de0 con 0x564e449754b0 2024-04-17T10:48:54.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.641+0000 7fd94170c640 1 --2- 172.21.15.83:0/1773008956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e449754b0 0x564e449d3110 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fd938005fd0 tx=0x7fd93801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daafd6c1b1bccae3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:54.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.641+0000 7fd91e7fc640 1 -- 172.21.15.83:0/1773008956 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93803d070 con 0x564e449754b0 2024-04-17T10:48:54.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.641+0000 7fd91e7fc640 1 -- 172.21.15.83:0/1773008956 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93800da60 con 0x564e449754b0 2024-04-17T10:48:54.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.641+0000 7fd91e7fc640 1 -- 172.21.15.83:0/1773008956 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9380223a0 con 0x564e449754b0 2024-04-17T10:48:54.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.641+0000 7fd946add000 1 -- 172.21.15.83:0/1773008956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e449754b0 msgr2=0x564e449d3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.641+0000 7fd946add000 1 --2- 172.21.15.83:0/1773008956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e449754b0 0x564e449d3110 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fd938005fd0 tx=0x7fd93801c000 comp rx=0 tx=0).stop 2024-04-17T10:48:54.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.641+0000 7fd946add000 1 -- 172.21.15.83:0/1773008956 shutdown_connections 2024-04-17T10:48:54.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.641+0000 7fd946add000 1 --2- 172.21.15.83:0/1773008956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564e449bf750 0x564e4499cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.641+0000 7fd946add000 1 --2- 172.21.15.83:0/1773008956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564e449bee30 0x564e449d3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.641+0000 7fd946add000 1 --2- 172.21.15.83:0/1773008956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e449754b0 0x564e449d3110 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.641+0000 7fd946add000 1 -- 172.21.15.83:0/1773008956 >> 172.21.15.83:0/1773008956 conn(0x564e4496d020 msgr2=0x564e449cc420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:54.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.641+0000 7fd946add000 1 -- 172.21.15.83:0/1773008956 shutdown_connections 2024-04-17T10:48:54.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.641+0000 7fd946add000 1 -- 172.21.15.83:0/1773008956 wait complete. 2024-04-17T10:48:54.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.641+0000 7fd946add000 1 -- start start 2024-04-17T10:48:54.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.641+0000 7fd946add000 1 Processor -- start 2024-04-17T10:48:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd946add000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e449754b0 0x564e449d2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd946add000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564e449bee30 0x564e449d2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd946add000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564e449bf750 0x564e449d33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd946add000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e44758e00 con 0x564e449754b0 2024-04-17T10:48:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd946add000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93c15ad10 con 0x564e449bee30 2024-04-17T10:48:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd946add000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e447bd300 con 0x564e449bf750 2024-04-17T10:48:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd946add000 10 client.0 _ll_register_callbacks cb 0x564e4493c040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:48:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd94170c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e449754b0 0x564e449d2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd94170c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e449754b0 0x564e449d2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44788/0 (socket says 172.21.15.83:44788) 2024-04-17T10:48:54.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd94170c640 1 -- 172.21.15.83:0/2499289233 learned_addr learned my addr 172.21.15.83:0/2499289233 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:54.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd940f0b640 1 --2- 172.21.15.83:0/2499289233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564e449bee30 0x564e449d2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd94170c640 1 -- 172.21.15.83:0/2499289233 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564e449bf750 msgr2=0x564e449d33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:54.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd94170c640 1 --2- 172.21.15.83:0/2499289233 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564e449bf750 0x564e449d33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd94170c640 1 -- 172.21.15.83:0/2499289233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564e449bee30 msgr2=0x564e449d2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:54.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd94170c640 1 --2- 172.21.15.83:0/2499289233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564e449bee30 0x564e449d2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd94170c640 1 -- 172.21.15.83:0/2499289233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e4475f8e0 con 0x564e449754b0 2024-04-17T10:48:54.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd940f0b640 1 --2- 172.21.15.83:0/2499289233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564e449bee30 0x564e449d2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:54.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd94170c640 1 --2- 172.21.15.83:0/2499289233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e449754b0 0x564e449d2530 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fd93800d6e0 tx=0x7fd938000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d16e749c12f3e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:54.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd917fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:48:54.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd917fff640 1 -- 172.21.15.83:0/2499289233 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93803d070 con 0x564e449754b0 2024-04-17T10:48:54.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd917fff640 1 -- 172.21.15.83:0/2499289233 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd938042af0 con 0x564e449754b0 2024-04-17T10:48:54.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd946add000 1 -- 172.21.15.83:0/2499289233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd93800da60 con 0x564e449754b0 2024-04-17T10:48:54.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd917fff640 1 -- 172.21.15.83:0/2499289233 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd938001300 con 0x564e449754b0 2024-04-17T10:48:54.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd946add000 10 client.8847 fetch_fsmap learned FSMap version 69 2024-04-17T10:48:54.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd946add000 1 -- 172.21.15.83:0/2499289233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x564e448178f0 con 0x564e449754b0 2024-04-17T10:48:54.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd917fff640 1 -- 172.21.15.83:0/2499289233 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93800da60 con 0x564e449754b0 2024-04-17T10:48:54.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd917fff640 1 -- 172.21.15.83:0/2499289233 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd938038690 con 0x564e449754b0 2024-04-17T10:48:54.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd946add000 10 client.8847 fetch_fsmap finished waiting for FSMap version 69 2024-04-17T10:48:54.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd946add000 10 client.8847 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-17T10:48:54.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd946add000 20 client.8847 populate_metadata read hostname 'smithi083' 2024-04-17T10:48:54.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd946add000 1 --2- 172.21.15.83:0/2499289233 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x564e449dcb80 0x564e449dcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd946add000 4 client.8847 mds_command: new command op to 8604 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T10:48:54.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd946add000 1 -- 172.21.15.83:0/2499289233 --> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564e447cb530 con 0x564e449dcb80 2024-04-17T10:48:54.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd940f0b640 1 --2- 172.21.15.83:0/2499289233 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x564e449dcb80 0x564e449dcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd917fff640 10 client.8847 ms_handle_connect on v2:172.21.15.83:6836/1479417998 2024-04-17T10:48:54.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd940f0b640 1 --2- 172.21.15.83:0/2499289233 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x564e449dcb80 0x564e449dcf90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b5e6330560c33e6 server_cookie=71ae724320e70d38 in_seq=0 out_seq=0 2024-04-17T10:48:54.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd917fff640 1 -- 172.21.15.83:0/2499289233 <== mds.0 v2:172.21.15.83:6836/1479417998 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x564e447cb530 con 0x564e449dcb80 2024-04-17T10:48:54.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd917fff640 10 client.8847 handle_command_reply: tid=0 2024-04-17T10:48:54.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd946add000 1 client.8847 shutdown 2024-04-17T10:48:54.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd946add000 1 -- 172.21.15.83:0/2499289233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e449754b0 msgr2=0x564e449d2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd946add000 1 --2- 172.21.15.83:0/2499289233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e449754b0 0x564e449d2530 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fd93800d6e0 tx=0x7fd938000ee0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd946add000 1 -- 172.21.15.83:0/2499289233 shutdown_connections 2024-04-17T10:48:54.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd946add000 1 --2- 172.21.15.83:0/2499289233 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x564e449dcb80 0x564e449dcf90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd946add000 1 --2- 172.21.15.83:0/2499289233 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x564e449bf750 0x564e449d33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd946add000 1 --2- 172.21.15.83:0/2499289233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564e449bee30 0x564e449d2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd946add000 1 --2- 172.21.15.83:0/2499289233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e449754b0 0x564e449d2530 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd917fff640 0 client.8847 ms_handle_reset on v2:172.21.15.83:6836/1479417998 2024-04-17T10:48:54.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd946add000 1 -- 172.21.15.83:0/2499289233 >> 172.21.15.83:0/2499289233 conn(0x564e4496d020 msgr2=0x564e4496f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:54.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd946add000 1 -- 172.21.15.83:0/2499289233 shutdown_connections 2024-04-17T10:48:54.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd946add000 1 -- 172.21.15.83:0/2499289233 wait complete. 2024-04-17T10:48:54.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd946add000 20 client.8847 trim_cache size 0 max 16384 2024-04-17T10:48:54.649 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:48:54.649 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-17T10:48:54.649 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-17T10:48:54.650 INFO:teuthology.orchestra.run.smithi083.stdout: "9d8443c1-bb99-4ad5-8eef-ff47c6bdcf48": { 2024-04-17T10:48:54.650 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/scrub_control_test_path", 2024-04-17T10:48:54.650 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "9d8443c1-bb99-4ad5-8eef-ff47c6bdcf48", 2024-04-17T10:48:54.650 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive" 2024-04-17T10:48:54.650 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:48:54.650 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:48:54.650 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:48:54.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd943970640 1 -- 172.21.15.83:0/522478146 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd920066f60 msgr2=0x7fd920069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd943970640 1 --2- 172.21.15.83:0/522478146 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd920066f60 0x7fd920069420 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd938007fd0 tx=0x7fd938009040 comp rx=0 tx=0).stop 2024-04-17T10:48:54.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd943970640 1 -- 172.21.15.83:0/522478146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd93c15bd50 msgr2=0x7fd93c09c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd943970640 1 --2- 172.21.15.83:0/522478146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd93c15bd50 0x7fd93c09c4b0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fd93400c7b0 tx=0x7fd93400cc70 comp rx=0 tx=0).stop 2024-04-17T10:48:54.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd943970640 1 -- 172.21.15.83:0/522478146 shutdown_connections 2024-04-17T10:48:54.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd943970640 1 --2- 172.21.15.83:0/522478146 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd920066f60 0x7fd920069420 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd943970640 1 --2- 172.21.15.83:0/522478146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd93c15bd50 0x7fd93c09c4b0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd943970640 1 --2- 172.21.15.83:0/522478146 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd93c0cd5c0 0x7fd93c157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd943970640 1 --2- 172.21.15.83:0/522478146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd93c0725f0 0x7fd93c13e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd943970640 1 -- 172.21.15.83:0/522478146 >> 172.21.15.83:0/522478146 conn(0x7fd93c0b3b40 msgr2=0x7fd93c0cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:54.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd943970640 1 -- 172.21.15.83:0/522478146 shutdown_connections 2024-04-17T10:48:54.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.645+0000 7fd943970640 1 -- 172.21.15.83:0/522478146 wait complete. 2024-04-17T10:48:54.659 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:48:54.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.773+0000 7fcc1f9ab640 1 Processor -- start 2024-04-17T10:48:54.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.773+0000 7fcc1f9ab640 1 -- start start 2024-04-17T10:48:54.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.777+0000 7fcc1f9ab640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcc1815db30 0x7fcc180d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.777+0000 7fcc1f9ab640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc180d1ad0 0x7fcc1806b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.777+0000 7fcc1f9ab640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc180cf8f0 con 0x7fcc1815df10 2024-04-17T10:48:54.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.777+0000 7fcc1f9ab640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc180ce4d0 con 0x7fcc1815db30 2024-04-17T10:48:54.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.777+0000 7fcc1f9ab640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc180d0ec0 con 0x7fcc180d1ad0 2024-04-17T10:48:54.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.777+0000 7fcc1d747640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcc1815db30 0x7fcc180d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.777+0000 7fcc1df48640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc180d1ad0 0x7fcc1806b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.777+0000 7fcc1cf46640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fcc1815df10 0x7fcc180cf1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:48432/0 (socket says 172.21.15.83:48432) 2024-04-17T10:48:54.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.777+0000 7fcc1cf46640 1 -- 172.21.15.83:0/2300139350 learned_addr learned my addr 172.21.15.83:0/2300139350 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:54.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.777+0000 7fcc1df48640 1 -- 172.21.15.83:0/2300139350 >> v2:172.21.15.167:3300/0 conn(0x7fcc1815db30 msgr2=0x7fcc180d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:54.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.777+0000 7fcc1df48640 1 --2- 172.21.15.83:0/2300139350 >> v2:172.21.15.167:3300/0 conn(0x7fcc1815db30 0x7fcc180d0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.777+0000 7fcc1df48640 1 -- 172.21.15.83:0/2300139350 >> v1:172.21.15.167:6789/0 conn(0x7fcc1815df10 legacy=0x7fcc180cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.777+0000 7fcc1df48640 1 -- 172.21.15.83:0/2300139350 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc180d0be0 con 0x7fcc180d1ad0 2024-04-17T10:48:54.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.777+0000 7fcc1d747640 1 --2- 172.21.15.83:0/2300139350 >> v2:172.21.15.167:3300/0 conn(0x7fcc1815db30 0x7fcc180d0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:48:54.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.777+0000 7fcc1df48640 1 --2- 172.21.15.83:0/2300139350 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc180d1ad0 0x7fcc1806b0f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009d90 tx=0x7fcc1001c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8e9a23a618f8edd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:54.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.781+0000 7fcc0ffff640 1 -- 172.21.15.83:0/2300139350 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc100b2070 con 0x7fcc180d1ad0 2024-04-17T10:48:54.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.781+0000 7fcc0ffff640 1 -- 172.21.15.83:0/2300139350 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc100a5910 con 0x7fcc180d1ad0 2024-04-17T10:48:54.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.781+0000 7fcc0ffff640 1 -- 172.21.15.83:0/2300139350 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc100a5ce0 con 0x7fcc180d1ad0 2024-04-17T10:48:54.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.781+0000 7fcc1f9ab640 1 -- 172.21.15.83:0/2300139350 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc180d1ad0 msgr2=0x7fcc1806b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.781+0000 7fcc1f9ab640 1 --2- 172.21.15.83:0/2300139350 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc180d1ad0 0x7fcc1806b0f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009d90 tx=0x7fcc1001c940 comp rx=0 tx=0).stop 2024-04-17T10:48:54.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.781+0000 7fcc1f9ab640 1 -- 172.21.15.83:0/2300139350 shutdown_connections 2024-04-17T10:48:54.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.781+0000 7fcc1f9ab640 1 --2- 172.21.15.83:0/2300139350 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc180d1ad0 0x7fcc1806b0f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.781+0000 7fcc1f9ab640 1 --2- 172.21.15.83:0/2300139350 >> v2:172.21.15.167:3300/0 conn(0x7fcc1815db30 0x7fcc180d0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.781+0000 7fcc1f9ab640 1 -- 172.21.15.83:0/2300139350 >> 172.21.15.83:0/2300139350 conn(0x7fcc1809cd50 msgr2=0x7fcc180d39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:54.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.781+0000 7fcc1f9ab640 1 -- 172.21.15.83:0/2300139350 shutdown_connections 2024-04-17T10:48:54.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.781+0000 7fcc1f9ab640 1 -- 172.21.15.83:0/2300139350 wait complete. 2024-04-17T10:48:54.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.781+0000 7fcc1f9ab640 1 Processor -- start 2024-04-17T10:48:54.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.781+0000 7fcc1f9ab640 1 -- start start 2024-04-17T10:48:54.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.781+0000 7fcc1f9ab640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc180d1ad0 0x7fcc18152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.781+0000 7fcc1f9ab640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc1815db30 0x7fcc18152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.781+0000 7fcc1f9ab640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc1815df10 0x7fcc180b3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.781+0000 7fcc1f9ab640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1816ab70 con 0x7fcc1815db30 2024-04-17T10:48:54.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.781+0000 7fcc1f9ab640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc180d0860 con 0x7fcc180d1ad0 2024-04-17T10:48:54.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.781+0000 7fcc1f9ab640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1816acf0 con 0x7fcc1815df10 2024-04-17T10:48:54.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.781+0000 7fcc1cf46640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc1815db30 0x7fcc18152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.781+0000 7fcc1cf46640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc1815db30 0x7fcc18152cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44792/0 (socket says 172.21.15.83:44792) 2024-04-17T10:48:54.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.781+0000 7fcc1cf46640 1 -- 172.21.15.83:0/2970846784 learned_addr learned my addr 172.21.15.83:0/2970846784 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:54.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.781+0000 7fcc1d747640 1 --2- 172.21.15.83:0/2970846784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc180d1ad0 0x7fcc18152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.781+0000 7fcc1df48640 1 --2- 172.21.15.83:0/2970846784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc1815df10 0x7fcc180b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.781+0000 7fcc1cf46640 1 -- 172.21.15.83:0/2970846784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc1815df10 msgr2=0x7fcc180b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.781+0000 7fcc1cf46640 1 --2- 172.21.15.83:0/2970846784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc1815df10 0x7fcc180b3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.781+0000 7fcc1cf46640 1 -- 172.21.15.83:0/2970846784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc180d1ad0 msgr2=0x7fcc18152780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.781+0000 7fcc1cf46640 1 --2- 172.21.15.83:0/2970846784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc180d1ad0 0x7fcc18152780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.781+0000 7fcc1cf46640 1 -- 172.21.15.83:0/2970846784 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1815b7f0 con 0x7fcc1815db30 2024-04-17T10:48:54.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.781+0000 7fcc1d747640 1 --2- 172.21.15.83:0/2970846784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc180d1ad0 0x7fcc18152780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:48:54.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.781+0000 7fcc1df48640 1 --2- 172.21.15.83:0/2970846784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc1815df10 0x7fcc180b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:48:54.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.781+0000 7fcc1cf46640 1 --2- 172.21.15.83:0/2970846784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc1815db30 0x7fcc18152cc0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800c450 tx=0x7fcc0800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:54.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.785+0000 7fcc0dffb640 1 -- 172.21.15.83:0/2970846784 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc08017070 con 0x7fcc1815db30 2024-04-17T10:48:54.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.785+0000 7fcc0dffb640 1 -- 172.21.15.83:0/2970846784 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc08005cf0 con 0x7fcc1815db30 2024-04-17T10:48:54.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.785+0000 7fcc1f9ab640 1 -- 172.21.15.83:0/2970846784 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc1815bad0 con 0x7fcc1815db30 2024-04-17T10:48:54.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.785+0000 7fcc0dffb640 1 -- 172.21.15.83:0/2970846784 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc08007290 con 0x7fcc1815db30 2024-04-17T10:48:54.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.785+0000 7fcc1f9ab640 1 -- 172.21.15.83:0/2970846784 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc1815c090 con 0x7fcc1815db30 2024-04-17T10:48:54.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.785+0000 7fcc1f9ab640 1 -- 172.21.15.83:0/2970846784 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc180681a0 con 0x7fcc1815db30 2024-04-17T10:48:54.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.785+0000 7fcc0dffb640 1 -- 172.21.15.83:0/2970846784 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcc0801a070 con 0x7fcc1815db30 2024-04-17T10:48:54.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.785+0000 7fcc0dffb640 1 --2- 172.21.15.83:0/2970846784 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcbfc066f90 0x7fcbfc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:54.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.785+0000 7fcc1d747640 1 --2- 172.21.15.83:0/2970846784 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcbfc066f90 0x7fcbfc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:54.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.785+0000 7fcc0dffb640 1 -- 172.21.15.83:0/2970846784 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fcc080842b0 con 0x7fcc1815db30 2024-04-17T10:48:54.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.785+0000 7fcc1d747640 1 --2- 172.21.15.83:0/2970846784 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcbfc066f90 0x7fcbfc069450 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fcc14009b10 tx=0x7fcc14008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:54.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.789+0000 7fcc0dffb640 1 -- 172.21.15.83:0/2970846784 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcc08055310 con 0x7fcc1815db30 2024-04-17T10:48:54.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.981+0000 7fcc1f9ab640 1 -- 172.21.15.83:0/2970846784 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc180d1210 con 0x7fcc1815db30 2024-04-17T10:48:54.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.981+0000 7fcc0dffb640 1 -- 172.21.15.83:0/2970846784 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fcc08054c10 con 0x7fcc1815db30 2024-04-17T10:48:54.980 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:54.980 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5786,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1553533838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1553533838},{"type":"v1","addr":"172.21.15.167:6833","nonce":1553533838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/674536913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":674536913},{"type":"v1","addr":"172.21.15.167:6835","nonce":674536913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8598,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4183890457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4183890457},{"type":"v1","addr":"172.21.15.83:6835","nonce":4183890457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:48:24.738443+0000","modified":"2024-04-17T10:48:26.776273+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8604":{"gid":8604,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1479417998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1479417998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1479417998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-17T10:48:54.981 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 69 2024-04-17T10:48:54.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.985+0000 7fcc1f9ab640 1 -- 172.21.15.83:0/2970846784 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcbfc066f90 msgr2=0x7fcbfc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.985+0000 7fcc1f9ab640 1 --2- 172.21.15.83:0/2970846784 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcbfc066f90 0x7fcbfc069450 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fcc14009b10 tx=0x7fcc14008040 comp rx=0 tx=0).stop 2024-04-17T10:48:54.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.985+0000 7fcc1f9ab640 1 -- 172.21.15.83:0/2970846784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc1815db30 msgr2=0x7fcc18152cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:54.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.985+0000 7fcc1f9ab640 1 --2- 172.21.15.83:0/2970846784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc1815db30 0x7fcc18152cc0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800c450 tx=0x7fcc0800c910 comp rx=0 tx=0).stop 2024-04-17T10:48:54.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.985+0000 7fcc1f9ab640 1 -- 172.21.15.83:0/2970846784 shutdown_connections 2024-04-17T10:48:54.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.985+0000 7fcc1f9ab640 1 --2- 172.21.15.83:0/2970846784 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcbfc066f90 0x7fcbfc069450 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.985+0000 7fcc1f9ab640 1 --2- 172.21.15.83:0/2970846784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc1815df10 0x7fcc180b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.985+0000 7fcc1f9ab640 1 --2- 172.21.15.83:0/2970846784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc1815db30 0x7fcc18152cc0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.985+0000 7fcc1f9ab640 1 --2- 172.21.15.83:0/2970846784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc180d1ad0 0x7fcc18152780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:54.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.985+0000 7fcc1f9ab640 1 -- 172.21.15.83:0/2970846784 >> 172.21.15.83:0/2970846784 conn(0x7fcc1809cd50 msgr2=0x7fcc1815e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:54.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.985+0000 7fcc1f9ab640 1 -- 172.21.15.83:0/2970846784 shutdown_connections 2024-04-17T10:48:54.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:54.985+0000 7fcc1f9ab640 1 -- 172.21.15.83:0/2970846784 wait complete. 2024-04-17T10:48:54.999 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-17T10:48:55.000 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-17T10:48:55.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.117+0000 7fde6fa0a640 1 Processor -- start 2024-04-17T10:48:55.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.117+0000 7fde6fa0a640 1 -- start start 2024-04-17T10:48:55.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.117+0000 7fde6fa0a640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fde6815dfa0 0x7fde680b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:55.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.117+0000 7fde6fa0a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde68162a00 con 0x7fde680b5b40 2024-04-17T10:48:55.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.117+0000 7fde6fa0a640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde68161e80 con 0x7fde680ce980 2024-04-17T10:48:55.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.117+0000 7fde6fa0a640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde68162160 con 0x7fde6815dfa0 2024-04-17T10:48:55.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.117+0000 7fde6dfa7640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fde680b5b40 0x7fde680d1d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55918/0 (socket says 172.21.15.83:55918) 2024-04-17T10:48:55.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.117+0000 7fde6dfa7640 1 -- 172.21.15.83:0/3618641150 learned_addr learned my addr 172.21.15.83:0/3618641150 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:55.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde6d7a6640 1 --2- 172.21.15.83:0/3618641150 >> v2:172.21.15.167:3300/0 conn(0x7fde6815dfa0 0x7fde680b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:55.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde5ffff640 1 -- 172.21.15.83:0/3618641150 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 668999391 0 0) 0x7fde68162a00 con 0x7fde680b5b40 2024-04-17T10:48:55.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde5ffff640 1 -- 172.21.15.83:0/3618641150 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde54003530 con 0x7fde680b5b40 2024-04-17T10:48:55.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde5ffff640 1 -- 172.21.15.83:0/3618641150 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3905516129 0 0) 0x7fde54003530 con 0x7fde680b5b40 2024-04-17T10:48:55.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde5ffff640 1 -- 172.21.15.83:0/3618641150 >> v2:172.21.15.167:3300/0 conn(0x7fde6815dfa0 msgr2=0x7fde680b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:55.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde5ffff640 1 --2- 172.21.15.83:0/3618641150 >> v2:172.21.15.167:3300/0 conn(0x7fde6815dfa0 0x7fde680b6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:55.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde5ffff640 1 -- 172.21.15.83:0/3618641150 >> v1:172.21.15.167:6789/0 conn(0x7fde680ce980 legacy=0x7fde680b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:55.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde5ffff640 1 -- 172.21.15.83:0/3618641150 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde68162440 con 0x7fde680b5b40 2024-04-17T10:48:55.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde5ffff640 1 -- 172.21.15.83:0/3618641150 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fde60001a20 con 0x7fde680b5b40 2024-04-17T10:48:55.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde5ffff640 1 -- 172.21.15.83:0/3618641150 >> v1:172.21.15.83:6789/0 conn(0x7fde680b5b40 legacy=0x7fde680d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:55.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde5ffff640 1 --2- 172.21.15.83:0/3618641150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde540039c0 0x7fde54003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:55.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde5ffff640 1 --2- 172.21.15.83:0/3618641150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde54004810 0x7fde54004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:55.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde5ffff640 1 --2- 172.21.15.83:0/3618641150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde540051d0 0x7fde5400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:55.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde5ffff640 1 -- 172.21.15.83:0/3618641150 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde68162160 con 0x7fde540051d0 2024-04-17T10:48:55.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde5ffff640 1 -- 172.21.15.83:0/3618641150 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5400c170 con 0x7fde540039c0 2024-04-17T10:48:55.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde5ffff640 1 -- 172.21.15.83:0/3618641150 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde5400c370 con 0x7fde54004810 2024-04-17T10:48:55.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde6dfa7640 1 --2- 172.21.15.83:0/3618641150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde540051d0 0x7fde5400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:55.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde6d7a6640 1 --2- 172.21.15.83:0/3618641150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde540039c0 0x7fde54003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:55.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde6dfa7640 1 -- 172.21.15.83:0/3618641150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde54004810 msgr2=0x7fde54004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:55.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde6cfa5640 1 --2- 172.21.15.83:0/3618641150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde54004810 0x7fde54004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:55.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde6dfa7640 1 --2- 172.21.15.83:0/3618641150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde54004810 0x7fde54004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:55.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde6dfa7640 1 -- 172.21.15.83:0/3618641150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde540039c0 msgr2=0x7fde54003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:55.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde6dfa7640 1 --2- 172.21.15.83:0/3618641150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde540039c0 0x7fde54003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:55.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde6dfa7640 1 -- 172.21.15.83:0/3618641150 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde54003530 con 0x7fde540051d0 2024-04-17T10:48:55.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde6dfa7640 1 -- 172.21.15.83:0/3618641150 reap_dead start 2024-04-17T10:48:55.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde6dfa7640 1 --2- 172.21.15.83:0/3618641150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde540051d0 0x7fde5400b9d0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fde68067840 tx=0x7fde6001fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f379e1e5bffaec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:55.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde5ffff640 1 -- 172.21.15.83:0/3618641150 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6000a4a0 con 0x7fde540051d0 2024-04-17T10:48:55.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde5ffff640 1 -- 172.21.15.83:0/3618641150 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde600aa490 con 0x7fde540051d0 2024-04-17T10:48:55.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde5ffff640 1 -- 172.21.15.83:0/3618641150 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde600aa860 con 0x7fde540051d0 2024-04-17T10:48:55.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde6fa0a640 1 -- 172.21.15.83:0/3618641150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde540051d0 msgr2=0x7fde5400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:55.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde6fa0a640 1 --2- 172.21.15.83:0/3618641150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde540051d0 0x7fde5400b9d0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fde68067840 tx=0x7fde6001fa60 comp rx=0 tx=0).stop 2024-04-17T10:48:55.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde6fa0a640 1 -- 172.21.15.83:0/3618641150 shutdown_connections 2024-04-17T10:48:55.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde6fa0a640 1 --2- 172.21.15.83:0/3618641150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde540051d0 0x7fde5400b9d0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:55.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde6fa0a640 1 -- 172.21.15.83:0/3618641150 >> 172.21.15.83:0/3618641150 conn(0x7fde680d0600 msgr2=0x7fde680d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:55.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde6fa0a640 1 -- 172.21.15.83:0/3618641150 shutdown_connections 2024-04-17T10:48:55.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.121+0000 7fde6fa0a640 1 -- 172.21.15.83:0/3618641150 wait complete. 2024-04-17T10:48:55.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.125+0000 7fde6fa0a640 1 Processor -- start 2024-04-17T10:48:55.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.125+0000 7fde6fa0a640 1 -- start start 2024-04-17T10:48:55.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.125+0000 7fde6fa0a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde540051d0 0x7fde6815cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:55.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.125+0000 7fde6fa0a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde68159fb0 0x7fde6815bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:55.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.125+0000 7fde6fa0a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde6815c010 0x7fde6815c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:55.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.125+0000 7fde6fa0a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde68161b10 con 0x7fde6815c010 2024-04-17T10:48:55.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.125+0000 7fde6fa0a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6815e380 con 0x7fde68159fb0 2024-04-17T10:48:55.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.125+0000 7fde6fa0a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde68161c90 con 0x7fde540051d0 2024-04-17T10:48:55.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.125+0000 7fde6dfa7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde6815c010 0x7fde6815c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:55.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.125+0000 7fde6dfa7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde6815c010 0x7fde6815c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44822/0 (socket says 172.21.15.83:44822) 2024-04-17T10:48:55.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.125+0000 7fde6dfa7640 1 -- 172.21.15.83:0/1394254972 learned_addr learned my addr 172.21.15.83:0/1394254972 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:55.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.125+0000 7fde6d7a6640 1 --2- 172.21.15.83:0/1394254972 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde540051d0 0x7fde6815cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:55.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.125+0000 7fde6dfa7640 1 -- 172.21.15.83:0/1394254972 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde540051d0 msgr2=0x7fde6815cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:55.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.125+0000 7fde6dfa7640 1 --2- 172.21.15.83:0/1394254972 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde540051d0 0x7fde6815cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:55.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.125+0000 7fde6cfa5640 1 --2- 172.21.15.83:0/1394254972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde68159fb0 0x7fde6815bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:55.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.125+0000 7fde6dfa7640 1 -- 172.21.15.83:0/1394254972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde68159fb0 msgr2=0x7fde6815bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:55.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.125+0000 7fde6dfa7640 1 --2- 172.21.15.83:0/1394254972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde68159fb0 0x7fde6815bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:55.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.125+0000 7fde6dfa7640 1 -- 172.21.15.83:0/1394254972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde6815c9e0 con 0x7fde6815c010 2024-04-17T10:48:55.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.125+0000 7fde6d7a6640 1 --2- 172.21.15.83:0/1394254972 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde540051d0 0x7fde6815cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:55.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.125+0000 7fde6dfa7640 1 --2- 172.21.15.83:0/1394254972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde6815c010 0x7fde6815c4a0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fde60020300 tx=0x7fde54005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:55.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.125+0000 7fde5dffb640 1 -- 172.21.15.83:0/1394254972 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde600bc070 con 0x7fde6815c010 2024-04-17T10:48:55.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.125+0000 7fde5dffb640 1 -- 172.21.15.83:0/1394254972 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde60001260 con 0x7fde6815c010 2024-04-17T10:48:55.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.125+0000 7fde6fa0a640 1 -- 172.21.15.83:0/1394254972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde6815d190 con 0x7fde6815c010 2024-04-17T10:48:55.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.125+0000 7fde5dffb640 1 -- 172.21.15.83:0/1394254972 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde60004d70 con 0x7fde6815c010 2024-04-17T10:48:55.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.125+0000 7fde6fa0a640 1 -- 172.21.15.83:0/1394254972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde680c92a0 con 0x7fde6815c010 2024-04-17T10:48:55.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.125+0000 7fde6fa0a640 1 -- 172.21.15.83:0/1394254972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde680681a0 con 0x7fde6815c010 2024-04-17T10:48:55.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.129+0000 7fde5dffb640 1 -- 172.21.15.83:0/1394254972 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fde600b3070 con 0x7fde6815c010 2024-04-17T10:48:55.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.129+0000 7fde5dffb640 1 --2- 172.21.15.83:0/1394254972 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fde4c066f60 0x7fde4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:55.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.129+0000 7fde5dffb640 1 -- 172.21.15.83:0/1394254972 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fde60121f90 con 0x7fde6815c010 2024-04-17T10:48:55.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.129+0000 7fde6d7a6640 1 --2- 172.21.15.83:0/1394254972 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fde4c066f60 0x7fde4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:55.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.129+0000 7fde6d7a6640 1 --2- 172.21.15.83:0/1394254972 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fde4c066f60 0x7fde4c069420 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fde640099a0 tx=0x7fde64008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:55.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.133+0000 7fde5dffb640 1 -- 172.21.15.83:0/1394254972 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fde60019770 con 0x7fde6815c010 2024-04-17T10:48:55.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.341+0000 7fde6fa0a640 1 -- 172.21.15.83:0/1394254972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fde680d1c30 con 0x7fde6815c010 2024-04-17T10:48:55.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.341+0000 7fde5dffb640 1 -- 172.21.15.83:0/1394254972 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1573 (secure 0 0 0) 0x7fde600f2ed0 con 0x7fde6815c010 2024-04-17T10:48:55.339 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:55.339 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":668,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":115,"num_osds":8,"num_up_osds":8,"osd_up_since":1713350279,"num_in_osds":8,"osd_in_since":1713350270,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":766,"data_bytes":779917098,"bytes_used":2035884032,"bytes_avail":771058229248,"bytes_total":773094113280,"write_bytes_sec":53488829,"read_op_per_sec":0,"write_op_per_sec":53},"fsmap":{"epoch":69,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8604}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2024-04-17T10:48:06.358835+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-17T10:48:55.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.345+0000 7fde6fa0a640 1 -- 172.21.15.83:0/1394254972 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fde4c066f60 msgr2=0x7fde4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:55.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.345+0000 7fde6fa0a640 1 --2- 172.21.15.83:0/1394254972 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fde4c066f60 0x7fde4c069420 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fde640099a0 tx=0x7fde64008040 comp rx=0 tx=0).stop 2024-04-17T10:48:55.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.345+0000 7fde6fa0a640 1 -- 172.21.15.83:0/1394254972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde6815c010 msgr2=0x7fde6815c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:55.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.345+0000 7fde6fa0a640 1 --2- 172.21.15.83:0/1394254972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde6815c010 0x7fde6815c4a0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fde60020300 tx=0x7fde54005840 comp rx=0 tx=0).stop 2024-04-17T10:48:55.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.345+0000 7fde6fa0a640 1 -- 172.21.15.83:0/1394254972 shutdown_connections 2024-04-17T10:48:55.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.345+0000 7fde6fa0a640 1 --2- 172.21.15.83:0/1394254972 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fde4c066f60 0x7fde4c069420 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:55.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.345+0000 7fde6fa0a640 1 --2- 172.21.15.83:0/1394254972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde6815c010 0x7fde6815c4a0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:55.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.345+0000 7fde6fa0a640 1 --2- 172.21.15.83:0/1394254972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde68159fb0 0x7fde6815bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:55.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.345+0000 7fde6fa0a640 1 --2- 172.21.15.83:0/1394254972 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde540051d0 0x7fde6815cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:55.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.345+0000 7fde6fa0a640 1 -- 172.21.15.83:0/1394254972 >> 172.21.15.83:0/1394254972 conn(0x7fde680d0600 msgr2=0x7fde680b7f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:55.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.345+0000 7fde6fa0a640 1 -- 172.21.15.83:0/1394254972 shutdown_connections 2024-04-17T10:48:55.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:55.345+0000 7fde6fa0a640 1 -- 172.21.15.83:0/1394254972 wait complete. 2024-04-17T10:48:55.356 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-17T10:48:56.358 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:48:56.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.493+0000 7f2d6c260640 1 Processor -- start 2024-04-17T10:48:56.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.493+0000 7f2d6c260640 1 -- start start 2024-04-17T10:48:56.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.493+0000 7f2d6c260640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d6406f020 0x7f2d640b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:56.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.493+0000 7f2d6c260640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2d640b5410 0x7f2d640ad840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:56.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.493+0000 7f2d6c260640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6406dcb0 con 0x7f2d64147530 2024-04-17T10:48:56.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.493+0000 7f2d6c260640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d6406eb90 con 0x7f2d640b5410 2024-04-17T10:48:56.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.493+0000 7f2d6c260640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d64147910 con 0x7f2d6406f020 2024-04-17T10:48:56.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.493+0000 7f2d69ffc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2d64147530 0x7f2d6406d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55934/0 (socket says 172.21.15.83:55934) 2024-04-17T10:48:56.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.493+0000 7f2d69ffc640 1 -- 172.21.15.83:0/479276864 learned_addr learned my addr 172.21.15.83:0/479276864 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:56.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.493+0000 7f2d6a7fd640 1 --2- 172.21.15.83:0/479276864 >> v2:172.21.15.83:3300/0 conn(0x7f2d640b5410 0x7f2d640ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:56.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.493+0000 7f2d6a7fd640 1 -- 172.21.15.83:0/479276864 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d6406f020 msgr2=0x7f2d640b4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:56.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.493+0000 7f2d6a7fd640 1 --2- 172.21.15.83:0/479276864 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d6406f020 0x7f2d640b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:56.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.493+0000 7f2d6a7fd640 1 -- 172.21.15.83:0/479276864 >> v1:172.21.15.83:6789/0 conn(0x7f2d64147530 legacy=0x7f2d6406d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:56.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.493+0000 7f2d68ffa640 1 -- 172.21.15.83:0/479276864 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 828560995 0 0) 0x7f2d6406dcb0 con 0x7f2d64147530 2024-04-17T10:48:56.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.493+0000 7f2d6a7fd640 1 -- 172.21.15.83:0/479276864 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6406df80 con 0x7f2d640b5410 2024-04-17T10:48:56.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.497+0000 7f2d6a7fd640 1 --2- 172.21.15.83:0/479276864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d640b5410 0x7f2d640ad840 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c009d90 tx=0x7f2d5c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e98895fd663fd576 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:56.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.497+0000 7f2d68ffa640 1 -- 172.21.15.83:0/479276864 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c0b2070 con 0x7f2d640b5410 2024-04-17T10:48:56.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.497+0000 7f2d68ffa640 1 -- 172.21.15.83:0/479276864 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d5c0a5920 con 0x7f2d640b5410 2024-04-17T10:48:56.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.497+0000 7f2d68ffa640 1 -- 172.21.15.83:0/479276864 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c0a5cf0 con 0x7f2d640b5410 2024-04-17T10:48:56.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.497+0000 7f2d6c260640 1 -- 172.21.15.83:0/479276864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d640b5410 msgr2=0x7f2d640ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:56.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.497+0000 7f2d6c260640 1 --2- 172.21.15.83:0/479276864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d640b5410 0x7f2d640ad840 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c009d90 tx=0x7f2d5c01c970 comp rx=0 tx=0).stop 2024-04-17T10:48:56.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.497+0000 7f2d6c260640 1 -- 172.21.15.83:0/479276864 shutdown_connections 2024-04-17T10:48:56.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.497+0000 7f2d6c260640 1 --2- 172.21.15.83:0/479276864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d640b5410 0x7f2d640ad840 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:56.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.497+0000 7f2d6c260640 1 --2- 172.21.15.83:0/479276864 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d6406f020 0x7f2d640b4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:56.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.497+0000 7f2d6c260640 1 -- 172.21.15.83:0/479276864 >> 172.21.15.83:0/479276864 conn(0x7f2d6406bcf0 msgr2=0x7f2d6406c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:56.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.497+0000 7f2d6c260640 1 -- 172.21.15.83:0/479276864 shutdown_connections 2024-04-17T10:48:56.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.497+0000 7f2d6c260640 1 -- 172.21.15.83:0/479276864 wait complete. 2024-04-17T10:48:56.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.497+0000 7f2d6c260640 1 Processor -- start 2024-04-17T10:48:56.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.497+0000 7f2d6c260640 1 -- start start 2024-04-17T10:48:56.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.501+0000 7f2d6c260640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d6406f020 0x7f2d640b1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:56.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.501+0000 7f2d6c260640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d640b5410 0x7f2d6413ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:56.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.501+0000 7f2d6c260640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d64147530 0x7f2d6415c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:56.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.501+0000 7f2d6c260640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d64071e80 con 0x7f2d6406f020 2024-04-17T10:48:56.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.501+0000 7f2d6c260640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d640b58a0 con 0x7f2d64147530 2024-04-17T10:48:56.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.501+0000 7f2d6c260640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d64072000 con 0x7f2d640b5410 2024-04-17T10:48:56.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.501+0000 7f2d69ffc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d6406f020 0x7f2d640b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:56.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.501+0000 7f2d69ffc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d6406f020 0x7f2d640b1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44844/0 (socket says 172.21.15.83:44844) 2024-04-17T10:48:56.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.501+0000 7f2d69ffc640 1 -- 172.21.15.83:0/2000781698 learned_addr learned my addr 172.21.15.83:0/2000781698 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:56.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.501+0000 7f2d69ffc640 1 -- 172.21.15.83:0/2000781698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d640b5410 msgr2=0x7f2d6413ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:56.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.501+0000 7f2d6a7fd640 1 --2- 172.21.15.83:0/2000781698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d64147530 0x7f2d6415c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:56.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.501+0000 7f2d697fb640 1 --2- 172.21.15.83:0/2000781698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d640b5410 0x7f2d6413ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:56.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.501+0000 7f2d69ffc640 1 --2- 172.21.15.83:0/2000781698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d640b5410 0x7f2d6413ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:56.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.501+0000 7f2d69ffc640 1 -- 172.21.15.83:0/2000781698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d64147530 msgr2=0x7f2d6415c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:56.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.501+0000 7f2d69ffc640 1 --2- 172.21.15.83:0/2000781698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d64147530 0x7f2d6415c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:56.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.501+0000 7f2d69ffc640 1 -- 172.21.15.83:0/2000781698 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d640b1570 con 0x7f2d6406f020 2024-04-17T10:48:56.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.501+0000 7f2d69ffc640 1 --2- 172.21.15.83:0/2000781698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d6406f020 0x7f2d640b1030 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000c480 tx=0x7f2d6000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:56.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.501+0000 7f2d5a7fc640 1 -- 172.21.15.83:0/2000781698 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d60018070 con 0x7f2d6406f020 2024-04-17T10:48:56.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.501+0000 7f2d5a7fc640 1 -- 172.21.15.83:0/2000781698 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d60007500 con 0x7f2d6406f020 2024-04-17T10:48:56.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.501+0000 7f2d6c260640 1 -- 172.21.15.83:0/2000781698 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d640b0590 con 0x7f2d6406f020 2024-04-17T10:48:56.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.501+0000 7f2d5a7fc640 1 -- 172.21.15.83:0/2000781698 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d60004a30 con 0x7f2d6406f020 2024-04-17T10:48:56.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.501+0000 7f2d6c260640 1 -- 172.21.15.83:0/2000781698 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d640b0830 con 0x7f2d6406f020 2024-04-17T10:48:56.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.501+0000 7f2d6c260640 1 -- 172.21.15.83:0/2000781698 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d64074f90 con 0x7f2d6406f020 2024-04-17T10:48:56.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.501+0000 7f2d5a7fc640 1 -- 172.21.15.83:0/2000781698 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2d6001b070 con 0x7f2d6406f020 2024-04-17T10:48:56.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.501+0000 7f2d5a7fc640 1 --2- 172.21.15.83:0/2000781698 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2d48066f60 0x7f2d48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:56.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.501+0000 7f2d5a7fc640 1 -- 172.21.15.83:0/2000781698 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f2d60084340 con 0x7f2d6406f020 2024-04-17T10:48:56.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.501+0000 7f2d697fb640 1 --2- 172.21.15.83:0/2000781698 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2d48066f60 0x7f2d48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:56.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.501+0000 7f2d697fb640 1 --2- 172.21.15.83:0/2000781698 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2d48066f60 0x7f2d48069420 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2d640cfa00 tx=0x7f2d54008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:56.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.505+0000 7f2d5a7fc640 1 -- 172.21.15.83:0/2000781698 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2d640b1570 con 0x7f2d6406f020 2024-04-17T10:48:56.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.701+0000 7f2d6c260640 1 -- 172.21.15.83:0/2000781698 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d64074480 con 0x7f2d6406f020 2024-04-17T10:48:56.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.701+0000 7f2d5a7fc640 1 -- 172.21.15.83:0/2000781698 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f2d600553a0 con 0x7f2d6406f020 2024-04-17T10:48:56.700 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:56.700 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5786,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1553533838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1553533838},{"type":"v1","addr":"172.21.15.167:6833","nonce":1553533838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/674536913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":674536913},{"type":"v1","addr":"172.21.15.167:6835","nonce":674536913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8598,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4183890457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4183890457},{"type":"v1","addr":"172.21.15.83:6835","nonce":4183890457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:48:24.738443+0000","modified":"2024-04-17T10:48:26.776273+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8604":{"gid":8604,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1479417998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1479417998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1479417998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-17T10:48:56.700 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 69 2024-04-17T10:48:56.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.705+0000 7f2d6c260640 1 -- 172.21.15.83:0/2000781698 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2d48066f60 msgr2=0x7f2d48069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:56.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.705+0000 7f2d6c260640 1 --2- 172.21.15.83:0/2000781698 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2d48066f60 0x7f2d48069420 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2d640cfa00 tx=0x7f2d54008040 comp rx=0 tx=0).stop 2024-04-17T10:48:56.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.705+0000 7f2d6c260640 1 -- 172.21.15.83:0/2000781698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d6406f020 msgr2=0x7f2d640b1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:56.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.705+0000 7f2d6c260640 1 --2- 172.21.15.83:0/2000781698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d6406f020 0x7f2d640b1030 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000c480 tx=0x7f2d6000c940 comp rx=0 tx=0).stop 2024-04-17T10:48:56.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.705+0000 7f2d6c260640 1 -- 172.21.15.83:0/2000781698 shutdown_connections 2024-04-17T10:48:56.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.705+0000 7f2d6c260640 1 --2- 172.21.15.83:0/2000781698 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2d48066f60 0x7f2d48069420 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:56.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.705+0000 7f2d6c260640 1 --2- 172.21.15.83:0/2000781698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d64147530 0x7f2d6415c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:56.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.705+0000 7f2d6c260640 1 --2- 172.21.15.83:0/2000781698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d640b5410 0x7f2d6413ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:56.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.705+0000 7f2d6c260640 1 --2- 172.21.15.83:0/2000781698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d6406f020 0x7f2d640b1030 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:56.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.705+0000 7f2d6c260640 1 -- 172.21.15.83:0/2000781698 >> 172.21.15.83:0/2000781698 conn(0x7f2d6406bcf0 msgr2=0x7f2d6415e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:56.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.705+0000 7f2d6c260640 1 -- 172.21.15.83:0/2000781698 shutdown_connections 2024-04-17T10:48:56.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.705+0000 7f2d6c260640 1 -- 172.21.15.83:0/2000781698 wait complete. 2024-04-17T10:48:56.718 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-17T10:48:56.719 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-17T10:48:56.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.837+0000 7ff5627de640 1 Processor -- start 2024-04-17T10:48:56.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.837+0000 7ff5627de640 1 -- start start 2024-04-17T10:48:56.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.837+0000 7ff5627de640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff55c072330 0x7ff55c0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:56.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.837+0000 7ff5627de640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff55c0cfa10 0x7ff55c06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:56.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.837+0000 7ff5627de640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff55c0cfea0 con 0x7ff55c15ba30 2024-04-17T10:48:56.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.837+0000 7ff5627de640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff55c071ea0 con 0x7ff55c072330 2024-04-17T10:48:56.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.837+0000 7ff5627de640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff55c06e320 con 0x7ff55c0cfa10 2024-04-17T10:48:56.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.837+0000 7ff55b7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff55c072330 0x7ff55c0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:56.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.837+0000 7ff55b7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff55c072330 0x7ff55c0cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40346/0 (socket says 172.21.15.83:40346) 2024-04-17T10:48:56.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.837+0000 7ff55b7fe640 1 -- 172.21.15.83:0/620856926 learned_addr learned my addr 172.21.15.83:0/620856926 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:56.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.837+0000 7ff560d7b640 1 --2- 172.21.15.83:0/620856926 >> v2:172.21.15.167:3300/0 conn(0x7ff55c0cfa10 0x7ff55c06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:56.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.837+0000 7ff55b7fe640 1 -- 172.21.15.83:0/620856926 >> v2:172.21.15.167:3300/0 conn(0x7ff55c0cfa10 msgr2=0x7ff55c06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:56.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.837+0000 7ff55b7fe640 1 --2- 172.21.15.83:0/620856926 >> v2:172.21.15.167:3300/0 conn(0x7ff55c0cfa10 0x7ff55c06c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:56.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.837+0000 7ff55b7fe640 1 -- 172.21.15.83:0/620856926 >> v1:172.21.15.167:6789/0 conn(0x7ff55c15ba30 legacy=0x7ff55c15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:56.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.837+0000 7ff55b7fe640 1 -- 172.21.15.83:0/620856926 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff55c06e040 con 0x7ff55c072330 2024-04-17T10:48:56.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.841+0000 7ff55b7fe640 1 --2- 172.21.15.83:0/620856926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff55c072330 0x7ff55c0cf4d0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0676a0 tx=0x7ff54c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3ceb742c191b2e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:56.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.841+0000 7ff55affd640 1 -- 172.21.15.83:0/620856926 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c0b2070 con 0x7ff55c072330 2024-04-17T10:48:56.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.841+0000 7ff55affd640 1 -- 172.21.15.83:0/620856926 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54c0a9070 con 0x7ff55c072330 2024-04-17T10:48:56.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.841+0000 7ff55affd640 1 -- 172.21.15.83:0/620856926 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c0ad880 con 0x7ff55c072330 2024-04-17T10:48:56.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.841+0000 7ff5627de640 1 -- 172.21.15.83:0/620856926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff55c072330 msgr2=0x7ff55c0cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:56.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.841+0000 7ff5627de640 1 --2- 172.21.15.83:0/620856926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff55c072330 0x7ff55c0cf4d0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0676a0 tx=0x7ff54c01c450 comp rx=0 tx=0).stop 2024-04-17T10:48:56.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.841+0000 7ff5627de640 1 -- 172.21.15.83:0/620856926 shutdown_connections 2024-04-17T10:48:56.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.841+0000 7ff5627de640 1 --2- 172.21.15.83:0/620856926 >> v2:172.21.15.167:3300/0 conn(0x7ff55c0cfa10 0x7ff55c06c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:56.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.841+0000 7ff5627de640 1 --2- 172.21.15.83:0/620856926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff55c072330 0x7ff55c0cf4d0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:56.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.841+0000 7ff5627de640 1 -- 172.21.15.83:0/620856926 >> 172.21.15.83:0/620856926 conn(0x7ff55c06abe0 msgr2=0x7ff55c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:56.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.841+0000 7ff5627de640 1 -- 172.21.15.83:0/620856926 shutdown_connections 2024-04-17T10:48:56.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.841+0000 7ff5627de640 1 -- 172.21.15.83:0/620856926 wait complete. 2024-04-17T10:48:56.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.845+0000 7ff5627de640 1 Processor -- start 2024-04-17T10:48:56.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.845+0000 7ff5627de640 1 -- start start 2024-04-17T10:48:56.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.845+0000 7ff5627de640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff55c072330 0x7ff55c14d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:56.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.845+0000 7ff5627de640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff55c0cfa10 0x7ff55c14e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:56.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.845+0000 7ff5627de640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff55c15ba30 0x7ff55c155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:56.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.845+0000 7ff5627de640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55c06de20 con 0x7ff55c15ba30 2024-04-17T10:48:56.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.845+0000 7ff5627de640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55c06dca0 con 0x7ff55c072330 2024-04-17T10:48:56.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.845+0000 7ff5627de640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff55c168520 con 0x7ff55c0cfa10 2024-04-17T10:48:56.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.845+0000 7ff560d7b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff55c15ba30 0x7ff55c155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:56.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.845+0000 7ff560d7b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff55c15ba30 0x7ff55c155730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40354/0 (socket says 172.21.15.83:40354) 2024-04-17T10:48:56.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.845+0000 7ff560d7b640 1 -- 172.21.15.83:0/3780568385 learned_addr learned my addr 172.21.15.83:0/3780568385 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:56.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.845+0000 7ff55b7fe640 1 --2- 172.21.15.83:0/3780568385 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff55c0cfa10 0x7ff55c14e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:56.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.845+0000 7ff55bfff640 1 --2- 172.21.15.83:0/3780568385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff55c072330 0x7ff55c14d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:56.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.845+0000 7ff560d7b640 1 -- 172.21.15.83:0/3780568385 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff55c0cfa10 msgr2=0x7ff55c14e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:56.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.845+0000 7ff560d7b640 1 --2- 172.21.15.83:0/3780568385 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff55c0cfa10 0x7ff55c14e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:56.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.845+0000 7ff560d7b640 1 -- 172.21.15.83:0/3780568385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff55c072330 msgr2=0x7ff55c14d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:56.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.845+0000 7ff560d7b640 1 --2- 172.21.15.83:0/3780568385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff55c072330 0x7ff55c14d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:56.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.845+0000 7ff560d7b640 1 -- 172.21.15.83:0/3780568385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff55c156df0 con 0x7ff55c15ba30 2024-04-17T10:48:56.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.845+0000 7ff560d7b640 1 --2- 172.21.15.83:0/3780568385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff55c15ba30 0x7ff55c155730 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7ff55400cd70 tx=0x7ff55400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:56.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.845+0000 7ff558ff9640 1 -- 172.21.15.83:0/3780568385 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff554003ba0 con 0x7ff55c15ba30 2024-04-17T10:48:56.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.845+0000 7ff558ff9640 1 -- 172.21.15.83:0/3780568385 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff554003d40 con 0x7ff55c15ba30 2024-04-17T10:48:56.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.845+0000 7ff5627de640 1 -- 172.21.15.83:0/3780568385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff55c14d9b0 con 0x7ff55c15ba30 2024-04-17T10:48:56.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.845+0000 7ff558ff9640 1 -- 172.21.15.83:0/3780568385 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff554004240 con 0x7ff55c15ba30 2024-04-17T10:48:56.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.845+0000 7ff5627de640 1 -- 172.21.15.83:0/3780568385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff55c14f450 con 0x7ff55c15ba30 2024-04-17T10:48:56.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.845+0000 7ff5627de640 1 -- 172.21.15.83:0/3780568385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff55c156fc0 con 0x7ff55c15ba30 2024-04-17T10:48:56.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.845+0000 7ff558ff9640 1 -- 172.21.15.83:0/3780568385 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff5540043e0 con 0x7ff55c15ba30 2024-04-17T10:48:56.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.845+0000 7ff558ff9640 1 --2- 172.21.15.83:0/3780568385 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff53c066f60 0x7ff53c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:56.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.849+0000 7ff55bfff640 1 --2- 172.21.15.83:0/3780568385 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff53c066f60 0x7ff53c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:56.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.849+0000 7ff558ff9640 1 -- 172.21.15.83:0/3780568385 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff554085b30 con 0x7ff55c15ba30 2024-04-17T10:48:56.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.849+0000 7ff55bfff640 1 --2- 172.21.15.83:0/3780568385 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff53c066f60 0x7ff53c069420 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff55c1f7b00 tx=0x7ff544009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:56.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:56.853+0000 7ff558ff9640 1 -- 172.21.15.83:0/3780568385 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff55401d340 con 0x7ff55c15ba30 2024-04-17T10:48:57.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.065+0000 7ff5627de640 1 -- 172.21.15.83:0/3780568385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7ff55c06b160 con 0x7ff55c15ba30 2024-04-17T10:48:57.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.069+0000 7ff558ff9640 1 -- 172.21.15.83:0/3780568385 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1675 (secure 0 0 0) 0x7ff554056b00 con 0x7ff55c15ba30 2024-04-17T10:48:57.065 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:57.065 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":669,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":116,"num_osds":8,"num_up_osds":8,"osd_up_since":1713350279,"num_in_osds":8,"osd_in_since":1713350270,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":95},{"state_name":"peering","count":2}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076818817,"bytes_used":2655641600,"bytes_avail":770438471680,"bytes_total":773094113280,"inactive_pgs_ratio":0.020618556067347527,"write_bytes_sec":77589788,"read_op_per_sec":0,"write_op_per_sec":78},"fsmap":{"epoch":69,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8604}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-04-17T10:48:56.371485+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-17T10:48:57.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.069+0000 7ff5627de640 1 -- 172.21.15.83:0/3780568385 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff53c066f60 msgr2=0x7ff53c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.069+0000 7ff5627de640 1 --2- 172.21.15.83:0/3780568385 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff53c066f60 0x7ff53c069420 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff55c1f7b00 tx=0x7ff544009040 comp rx=0 tx=0).stop 2024-04-17T10:48:57.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.069+0000 7ff5627de640 1 -- 172.21.15.83:0/3780568385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff55c15ba30 msgr2=0x7ff55c155730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.069+0000 7ff5627de640 1 --2- 172.21.15.83:0/3780568385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff55c15ba30 0x7ff55c155730 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7ff55400cd70 tx=0x7ff55400a660 comp rx=0 tx=0).stop 2024-04-17T10:48:57.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.069+0000 7ff5627de640 1 -- 172.21.15.83:0/3780568385 shutdown_connections 2024-04-17T10:48:57.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.069+0000 7ff5627de640 1 --2- 172.21.15.83:0/3780568385 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff53c066f60 0x7ff53c069420 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.069+0000 7ff5627de640 1 --2- 172.21.15.83:0/3780568385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff55c15ba30 0x7ff55c155730 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.069+0000 7ff5627de640 1 --2- 172.21.15.83:0/3780568385 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff55c0cfa10 0x7ff55c14e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.069+0000 7ff5627de640 1 --2- 172.21.15.83:0/3780568385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff55c072330 0x7ff55c14d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.069+0000 7ff5627de640 1 -- 172.21.15.83:0/3780568385 >> 172.21.15.83:0/3780568385 conn(0x7ff55c06abe0 msgr2=0x7ff55c159b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:57.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.069+0000 7ff5627de640 1 -- 172.21.15.83:0/3780568385 shutdown_connections 2024-04-17T10:48:57.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.069+0000 7ff5627de640 1 -- 172.21.15.83:0/3780568385 wait complete. 2024-04-17T10:48:57.083 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused'} 2024-04-17T10:48:57.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:57.083 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub abort 2024-04-17T10:48:57.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.205+0000 7f68fc5dc640 1 Processor -- start 2024-04-17T10:48:57.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.205+0000 7f68fc5dc640 1 -- start start 2024-04-17T10:48:57.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.205+0000 7f68fc5dc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f68f415b6e0 0x7f68f4071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.205+0000 7f68fc5dc640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f68f415bac0 0x7f68f40cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.205+0000 7f68fc5dc640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68f406c1f0 con 0x7f68f4070470 2024-04-17T10:48:57.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.205+0000 7f68fc5dc640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68f40cdd30 con 0x7f68f415b6e0 2024-04-17T10:48:57.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.205+0000 7f68fc5dc640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68f4070a10 con 0x7f68f415bac0 2024-04-17T10:48:57.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.205+0000 7f68fa378640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f68f415b6e0 0x7f68f4071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.205+0000 7f68fab79640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f68f4070470 0x7f68f406ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49326/0 (socket says 172.21.15.83:49326) 2024-04-17T10:48:57.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.205+0000 7f68fab79640 1 -- 172.21.15.83:0/123484592 learned_addr learned my addr 172.21.15.83:0/123484592 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:57.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.205+0000 7f68f9b77640 1 --2- 172.21.15.83:0/123484592 >> v2:172.21.15.167:3300/0 conn(0x7f68f415bac0 0x7f68f40cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.205+0000 7f68fa378640 1 -- 172.21.15.83:0/123484592 >> v2:172.21.15.167:3300/0 conn(0x7f68f415bac0 msgr2=0x7f68f40cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:57.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.205+0000 7f68fa378640 1 --2- 172.21.15.83:0/123484592 >> v2:172.21.15.167:3300/0 conn(0x7f68f415bac0 0x7f68f40cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.205+0000 7f68fa378640 1 -- 172.21.15.83:0/123484592 >> v1:172.21.15.83:6789/0 conn(0x7f68f4070470 legacy=0x7f68f406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.205+0000 7f68f9376640 1 -- 172.21.15.83:0/123484592 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2763507186 0 0) 0x7f68f406c1f0 con 0x7f68f4070470 2024-04-17T10:48:57.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.205+0000 7f68fa378640 1 -- 172.21.15.83:0/123484592 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f4070cf0 con 0x7f68f415b6e0 2024-04-17T10:48:57.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.205+0000 7f68f9b77640 1 --2- 172.21.15.83:0/123484592 >> v2:172.21.15.167:3300/0 conn(0x7f68f415bac0 0x7f68f40cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:57.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.209+0000 7f68fa378640 1 --2- 172.21.15.83:0/123484592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68f415b6e0 0x7f68f4071cc0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f68f4071c50 tx=0x7f68f001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8f3407dd06ec60 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:57.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.209+0000 7f68f9376640 1 -- 172.21.15.83:0/123484592 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f00b0070 con 0x7f68f415b6e0 2024-04-17T10:48:57.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.209+0000 7f68f9376640 1 -- 172.21.15.83:0/123484592 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f00ab600 con 0x7f68f415b6e0 2024-04-17T10:48:57.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.209+0000 7f68f9376640 1 -- 172.21.15.83:0/123484592 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f00ab9d0 con 0x7f68f415b6e0 2024-04-17T10:48:57.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.209+0000 7f68fc5dc640 1 -- 172.21.15.83:0/123484592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68f415b6e0 msgr2=0x7f68f4071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.209+0000 7f68fc5dc640 1 --2- 172.21.15.83:0/123484592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68f415b6e0 0x7f68f4071cc0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f68f4071c50 tx=0x7f68f001c480 comp rx=0 tx=0).stop 2024-04-17T10:48:57.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.209+0000 7f68fc5dc640 1 -- 172.21.15.83:0/123484592 shutdown_connections 2024-04-17T10:48:57.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.209+0000 7f68fc5dc640 1 --2- 172.21.15.83:0/123484592 >> v2:172.21.15.167:3300/0 conn(0x7f68f415bac0 0x7f68f40cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.209+0000 7f68fc5dc640 1 --2- 172.21.15.83:0/123484592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68f415b6e0 0x7f68f4071cc0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.209+0000 7f68fc5dc640 1 -- 172.21.15.83:0/123484592 >> 172.21.15.83:0/123484592 conn(0x7f68f40b15b0 msgr2=0x7f68f40b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:57.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.209+0000 7f68fc5dc640 1 -- 172.21.15.83:0/123484592 shutdown_connections 2024-04-17T10:48:57.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.209+0000 7f68fc5dc640 1 -- 172.21.15.83:0/123484592 wait complete. 2024-04-17T10:48:57.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.213+0000 7f68fc5dc640 1 Processor -- start 2024-04-17T10:48:57.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.213+0000 7f68fc5dc640 1 -- start start 2024-04-17T10:48:57.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.213+0000 7f68fc5dc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68f4070470 0x7f68f4155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.213+0000 7f68fc5dc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68f415b6e0 0x7f68f41f6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.213+0000 7f68fc5dc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68f415bac0 0x7f68f414a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.213+0000 7f68fc5dc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f406cee0 con 0x7f68f415b6e0 2024-04-17T10:48:57.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.213+0000 7f68fc5dc640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f406b730 con 0x7f68f415bac0 2024-04-17T10:48:57.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.213+0000 7f68fc5dc640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f406d060 con 0x7f68f4070470 2024-04-17T10:48:57.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.213+0000 7f68f9b77640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68f415b6e0 0x7f68f41f6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.213+0000 7f68f9b77640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68f415b6e0 0x7f68f41f6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40380/0 (socket says 172.21.15.83:40380) 2024-04-17T10:48:57.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.213+0000 7f68f9b77640 1 -- 172.21.15.83:0/742616412 learned_addr learned my addr 172.21.15.83:0/742616412 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:57.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.213+0000 7f68fab79640 1 --2- 172.21.15.83:0/742616412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68f415bac0 0x7f68f414a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.213+0000 7f68fa378640 1 --2- 172.21.15.83:0/742616412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68f4070470 0x7f68f4155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.213+0000 7f68f9b77640 1 -- 172.21.15.83:0/742616412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68f4070470 msgr2=0x7f68f4155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.213+0000 7f68f9b77640 1 --2- 172.21.15.83:0/742616412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68f4070470 0x7f68f4155240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.213+0000 7f68f9b77640 1 -- 172.21.15.83:0/742616412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68f415bac0 msgr2=0x7f68f414a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.213+0000 7f68f9b77640 1 --2- 172.21.15.83:0/742616412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68f415bac0 0x7f68f414a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.213+0000 7f68f9b77640 1 -- 172.21.15.83:0/742616412 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f41f6c90 con 0x7f68f415b6e0 2024-04-17T10:48:57.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.213+0000 7f68f9b77640 1 --2- 172.21.15.83:0/742616412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68f415b6e0 0x7f68f41f6750 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f68e400c820 tx=0x7f68e400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:57.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.213+0000 7f68eaffd640 1 -- 172.21.15.83:0/742616412 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e4015070 con 0x7f68f415b6e0 2024-04-17T10:48:57.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.213+0000 7f68eaffd640 1 -- 172.21.15.83:0/742616412 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e4007a30 con 0x7f68f415b6e0 2024-04-17T10:48:57.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.213+0000 7f68fc5dc640 1 -- 172.21.15.83:0/742616412 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f4156df0 con 0x7f68f415b6e0 2024-04-17T10:48:57.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.213+0000 7f68eaffd640 1 -- 172.21.15.83:0/742616412 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e4007e40 con 0x7f68f415b6e0 2024-04-17T10:48:57.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.213+0000 7f68fc5dc640 1 -- 172.21.15.83:0/742616412 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f414a150 con 0x7f68f415b6e0 2024-04-17T10:48:57.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.213+0000 7f68eaffd640 1 -- 172.21.15.83:0/742616412 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f68e4018070 con 0x7f68f415b6e0 2024-04-17T10:48:57.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.213+0000 7f68eaffd640 1 --2- 172.21.15.83:0/742616412 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f68d8066f90 0x7f68d8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68eaffd640 1 -- 172.21.15.83:0/742616412 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f68e4084e90 con 0x7f68f415b6e0 2024-04-17T10:48:57.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68fa378640 1 --2- 172.21.15.83:0/742616412 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f68d8066f90 0x7f68d8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68fa378640 1 --2- 172.21.15.83:0/742616412 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f68d8066f90 0x7f68d8069450 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f68f00ad000 tx=0x7f68f00a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:57.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68ff749000 1 Processor -- start 2024-04-17T10:48:57.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68ff749000 1 -- start start 2024-04-17T10:48:57.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68ff749000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55970dd07a40 0x55970dd07e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68ff749000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55970dd08360 0x55970dcc2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68ff749000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55970dcc2d90 0x55970dcc3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68ff749000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55970dba51f0 con 0x55970dcc2d90 2024-04-17T10:48:57.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68ff749000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55970db0be90 con 0x55970dd07a40 2024-04-17T10:48:57.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68ff749000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55970da93f50 con 0x55970dd08360 2024-04-17T10:48:57.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68fab79640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55970dcc2d90 0x55970dcc3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68fab79640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55970dcc2d90 0x55970dcc3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40388/0 (socket says 172.21.15.83:40388) 2024-04-17T10:48:57.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68fab79640 1 -- 172.21.15.83:0/1876974035 learned_addr learned my addr 172.21.15.83:0/1876974035 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:57.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68f9b77640 1 --2- 172.21.15.83:0/1876974035 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55970dd08360 0x55970dcc2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68fa378640 1 --2- 172.21.15.83:0/1876974035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55970dd07a40 0x55970dd07e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68fab79640 1 -- 172.21.15.83:0/1876974035 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55970dd08360 msgr2=0x55970dcc2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:57.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68fab79640 1 --2- 172.21.15.83:0/1876974035 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55970dd08360 0x55970dcc2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68fab79640 1 -- 172.21.15.83:0/1876974035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55970dd07a40 msgr2=0x55970dd07e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:57.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68fab79640 1 --2- 172.21.15.83:0/1876974035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55970dd07a40 0x55970dd07e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68fab79640 1 -- 172.21.15.83:0/1876974035 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55970dae38b0 con 0x55970dcc2d90 2024-04-17T10:48:57.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68f9b77640 1 --2- 172.21.15.83:0/1876974035 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55970dd08360 0x55970dcc2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:57.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68fab79640 1 --2- 172.21.15.83:0/1876974035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55970dcc2d90 0x55970dcc3170 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f68ec001e30 tx=0x7f68ec00d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca39e258139465e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:57.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68d7fff640 1 -- 172.21.15.83:0/1876974035 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec00de30 con 0x55970dcc2d90 2024-04-17T10:48:57.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68d7fff640 1 -- 172.21.15.83:0/1876974035 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ec004b70 con 0x55970dcc2d90 2024-04-17T10:48:57.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68d7fff640 1 -- 172.21.15.83:0/1876974035 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec0070d0 con 0x55970dcc2d90 2024-04-17T10:48:57.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68ff749000 1 -- 172.21.15.83:0/1876974035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55970dcc2d90 msgr2=0x55970dcc3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68ff749000 1 --2- 172.21.15.83:0/1876974035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55970dcc2d90 0x55970dcc3170 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f68ec001e30 tx=0x7f68ec00d520 comp rx=0 tx=0).stop 2024-04-17T10:48:57.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68ff749000 1 -- 172.21.15.83:0/1876974035 shutdown_connections 2024-04-17T10:48:57.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68ff749000 1 --2- 172.21.15.83:0/1876974035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55970dcc2d90 0x55970dcc3170 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68ff749000 1 --2- 172.21.15.83:0/1876974035 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55970dd08360 0x55970dcc2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68ff749000 1 --2- 172.21.15.83:0/1876974035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55970dd07a40 0x55970dd07e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68ff749000 1 -- 172.21.15.83:0/1876974035 >> 172.21.15.83:0/1876974035 conn(0x55970dcbed40 msgr2=0x55970dcc1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:57.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68ff749000 1 -- 172.21.15.83:0/1876974035 shutdown_connections 2024-04-17T10:48:57.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68ff749000 1 -- 172.21.15.83:0/1876974035 wait complete. 2024-04-17T10:48:57.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68ff749000 1 -- start start 2024-04-17T10:48:57.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.217+0000 7f68ff749000 1 Processor -- start 2024-04-17T10:48:57.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68ff749000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55970dcc2d90 0x55970dd0f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68ff749000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55970dd07a40 0x55970dd0f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68ff749000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55970dd08360 0x55970dcc8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68ff749000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55970dac6540 con 0x55970dd08360 2024-04-17T10:48:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68ff749000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55970db07f30 con 0x55970dd07a40 2024-04-17T10:48:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68ff749000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55970dac6d70 con 0x55970dcc2d90 2024-04-17T10:48:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68ff749000 10 client.0 _ll_register_callbacks cb 0x55970dbc0180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:48:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68fab79640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55970dd08360 0x55970dcc8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68fab79640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55970dd08360 0x55970dcc8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40394/0 (socket says 172.21.15.83:40394) 2024-04-17T10:48:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68fab79640 1 -- 172.21.15.83:0/2434831441 learned_addr learned my addr 172.21.15.83:0/2434831441 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68fa378640 1 --2- 172.21.15.83:0/2434831441 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55970dcc2d90 0x55970dd0f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68f9b77640 1 --2- 172.21.15.83:0/2434831441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55970dd07a40 0x55970dd0f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68fab79640 1 -- 172.21.15.83:0/2434831441 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55970dcc2d90 msgr2=0x55970dd0f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68fab79640 1 --2- 172.21.15.83:0/2434831441 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55970dcc2d90 0x55970dd0f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68fab79640 1 -- 172.21.15.83:0/2434831441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55970dd07a40 msgr2=0x55970dd0f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68fab79640 1 --2- 172.21.15.83:0/2434831441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55970dd07a40 0x55970dd0f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68fab79640 1 -- 172.21.15.83:0/2434831441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55970da941a0 con 0x55970dd08360 2024-04-17T10:48:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68fab79640 1 --2- 172.21.15.83:0/2434831441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55970dd08360 0x55970dcc8e50 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f68ec0012f0 tx=0x7f68ec001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a93ce07115e819 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:48:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68d57fa640 1 -- 172.21.15.83:0/2434831441 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec023020 con 0x55970dd08360 2024-04-17T10:48:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68ff749000 1 -- 172.21.15.83:0/2434831441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f68ec004b70 con 0x55970dd08360 2024-04-17T10:48:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68d57fa640 1 -- 172.21.15.83:0/2434831441 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f68ec01f740 con 0x55970dd08360 2024-04-17T10:48:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68d57fa640 1 -- 172.21.15.83:0/2434831441 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68ec02d050 con 0x55970dd08360 2024-04-17T10:48:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68ff749000 10 client.8886 fetch_fsmap learned FSMap version 69 2024-04-17T10:48:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68ff749000 1 -- 172.21.15.83:0/2434831441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x55970db8e840 con 0x55970dd08360 2024-04-17T10:48:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68d57fa640 1 -- 172.21.15.83:0/2434831441 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ec004b70 con 0x55970dd08360 2024-04-17T10:48:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68d57fa640 1 -- 172.21.15.83:0/2434831441 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f68ec030020 con 0x55970dd08360 2024-04-17T10:48:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68ff749000 10 client.8886 fetch_fsmap finished waiting for FSMap version 69 2024-04-17T10:48:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68ff749000 10 client.8886 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-17T10:48:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68ff749000 20 client.8886 populate_metadata read hostname 'smithi083' 2024-04-17T10:48:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68ff749000 1 --2- 172.21.15.83:0/2434831441 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x55970dcc9bc0 0x55970dcc9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68ff749000 4 client.8886 mds_command: new command op to 8604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:48:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68ff749000 1 -- 172.21.15.83:0/2434831441 --> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55970daa7be0 con 0x55970dcc9bc0 2024-04-17T10:48:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68fa378640 1 --2- 172.21.15.83:0/2434831441 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x55970dcc9bc0 0x55970dcc9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68fa378640 1 --2- 172.21.15.83:0/2434831441 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x55970dcc9bc0 0x55970dcc9fa0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=417cad903adc2565 server_cookie=40240a40fbab4088 in_seq=0 out_seq=0 2024-04-17T10:48:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68d57fa640 10 client.8886 ms_handle_connect on v2:172.21.15.83:6836/1479417998 2024-04-17T10:48:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68d57fa640 1 -- 172.21.15.83:0/2434831441 <== mds.0 v2:172.21.15.83:6836/1479417998 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55970daa7be0 con 0x55970dcc9bc0 2024-04-17T10:48:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68d57fa640 10 client.8886 handle_command_reply: tid=0 2024-04-17T10:48:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68ff749000 1 client.8886 shutdown 2024-04-17T10:48:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68ff749000 1 -- 172.21.15.83:0/2434831441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55970dd08360 msgr2=0x55970dcc8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68ff749000 1 --2- 172.21.15.83:0/2434831441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55970dd08360 0x55970dcc8e50 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f68ec0012f0 tx=0x7f68ec001260 comp rx=0 tx=0).stop 2024-04-17T10:48:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68ff749000 1 -- 172.21.15.83:0/2434831441 shutdown_connections 2024-04-17T10:48:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68ff749000 1 --2- 172.21.15.83:0/2434831441 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x55970dcc9bc0 0x55970dcc9fa0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68ff749000 1 --2- 172.21.15.83:0/2434831441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55970dd08360 0x55970dcc8e50 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68ff749000 1 --2- 172.21.15.83:0/2434831441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55970dd07a40 0x55970dd0f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68d57fa640 0 client.8886 ms_handle_reset on v2:172.21.15.83:6836/1479417998 2024-04-17T10:48:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68ff749000 1 --2- 172.21.15.83:0/2434831441 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55970dcc2d90 0x55970dd0f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68ff749000 1 -- 172.21.15.83:0/2434831441 >> 172.21.15.83:0/2434831441 conn(0x55970dcbed40 msgr2=0x55970dcc19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68ff749000 1 -- 172.21.15.83:0/2434831441 shutdown_connections 2024-04-17T10:48:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68ff749000 1 -- 172.21.15.83:0/2434831441 wait complete. 2024-04-17T10:48:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.221+0000 7f68ff749000 20 client.8886 trim_cache size 0 max 16384 2024-04-17T10:48:57.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.237+0000 7f68ff749000 1 Processor -- start 2024-04-17T10:48:57.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.237+0000 7f68ff749000 1 -- start start 2024-04-17T10:48:57.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.237+0000 7f68ff749000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55970dcc2d90 0x55970dd64080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.237+0000 7f68ff749000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55970dd07a40 0x55970dd645c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.237+0000 7f68ff749000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55970dd08360 0x55970dd64b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.237+0000 7f68ff749000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f4202ac0 con 0x55970dd08360 2024-04-17T10:48:57.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.237+0000 7f68ff749000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f4201ca0 con 0x55970dcc2d90 2024-04-17T10:48:57.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.237+0000 7f68ff749000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f41f9d20 con 0x55970dd07a40 2024-04-17T10:48:57.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.237+0000 7f68fab79640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55970dd08360 0x55970dd64b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.237+0000 7f68fab79640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55970dd08360 0x55970dd64b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40410/0 (socket says 172.21.15.83:40410) 2024-04-17T10:48:57.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.237+0000 7f68fab79640 1 -- 172.21.15.83:0/382605966 learned_addr learned my addr 172.21.15.83:0/382605966 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:57.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.237+0000 7f68fab79640 1 -- 172.21.15.83:0/382605966 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55970dd07a40 msgr2=0x55970dd645c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:57.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.237+0000 7f68fab79640 1 --2- 172.21.15.83:0/382605966 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55970dd07a40 0x55970dd645c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.237+0000 7f68fab79640 1 -- 172.21.15.83:0/382605966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55970dcc2d90 msgr2=0x55970dd64080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:57.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.237+0000 7f68fab79640 1 --2- 172.21.15.83:0/382605966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55970dcc2d90 0x55970dd64080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.237+0000 7f68fab79640 1 -- 172.21.15.83:0/382605966 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55970db65a10 con 0x55970dd08360 2024-04-17T10:48:57.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.237+0000 7f68fab79640 1 --2- 172.21.15.83:0/382605966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55970dd08360 0x55970dd64b00 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f68ec001d30 tx=0x7f68ec01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c197b6b29ba5e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:57.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.237+0000 7f68d77fe640 1 -- 172.21.15.83:0/382605966 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec036070 con 0x55970dd08360 2024-04-17T10:48:57.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.237+0000 7f68d77fe640 1 -- 172.21.15.83:0/382605966 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ec004b70 con 0x55970dd08360 2024-04-17T10:48:57.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.237+0000 7f68d77fe640 1 -- 172.21.15.83:0/382605966 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec023030 con 0x55970dd08360 2024-04-17T10:48:57.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 1 -- 172.21.15.83:0/382605966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55970dd08360 msgr2=0x55970dd64b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 1 --2- 172.21.15.83:0/382605966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55970dd08360 0x55970dd64b00 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f68ec001d30 tx=0x7f68ec01e000 comp rx=0 tx=0).stop 2024-04-17T10:48:57.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 1 -- 172.21.15.83:0/382605966 shutdown_connections 2024-04-17T10:48:57.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 1 --2- 172.21.15.83:0/382605966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55970dd08360 0x55970dd64b00 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 1 --2- 172.21.15.83:0/382605966 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55970dd07a40 0x55970dd645c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 1 --2- 172.21.15.83:0/382605966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55970dcc2d90 0x55970dd64080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 1 -- 172.21.15.83:0/382605966 >> 172.21.15.83:0/382605966 conn(0x55970dcbed40 msgr2=0x55970dd06180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:57.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 1 -- 172.21.15.83:0/382605966 shutdown_connections 2024-04-17T10:48:57.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 1 -- 172.21.15.83:0/382605966 wait complete. 2024-04-17T10:48:57.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 1 -- start start 2024-04-17T10:48:57.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 1 Processor -- start 2024-04-17T10:48:57.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55970dcc2d90 0x55970dd682a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55970dd07a40 0x55970dd687e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55970dd08360 0x55970dd0ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55970db07f30 con 0x55970dd08360 2024-04-17T10:48:57.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f42016d0 con 0x55970dcc2d90 2024-04-17T10:48:57.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55970dac4df0 con 0x55970dd07a40 2024-04-17T10:48:57.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 10 client.0 _ll_register_callbacks cb 0x55970dacea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:48:57.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68fab79640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55970dd08360 0x55970dd0ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68fab79640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55970dd08360 0x55970dd0ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40412/0 (socket says 172.21.15.83:40412) 2024-04-17T10:48:57.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68fab79640 1 -- 172.21.15.83:0/371983044 learned_addr learned my addr 172.21.15.83:0/371983044 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:57.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68f9b77640 1 --2- 172.21.15.83:0/371983044 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55970dd07a40 0x55970dd687e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68fab79640 1 -- 172.21.15.83:0/371983044 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55970dd07a40 msgr2=0x55970dd687e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:57.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68fab79640 1 --2- 172.21.15.83:0/371983044 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55970dd07a40 0x55970dd687e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68fab79640 1 -- 172.21.15.83:0/371983044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55970dcc2d90 msgr2=0x55970dd682a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:57.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68fa378640 1 --2- 172.21.15.83:0/371983044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55970dcc2d90 0x55970dd682a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68fab79640 1 --2- 172.21.15.83:0/371983044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55970dcc2d90 0x55970dd682a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68fab79640 1 -- 172.21.15.83:0/371983044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55970daa0b00 con 0x55970dd08360 2024-04-17T10:48:57.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68fab79640 1 --2- 172.21.15.83:0/371983044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55970dd08360 0x55970dd0ced0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f68ec001a30 tx=0x7f68ec009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32890a33bbd6462b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:57.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:48:57.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68d57fa640 1 -- 172.21.15.83:0/371983044 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec03f060 con 0x55970dd08360 2024-04-17T10:48:57.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68d57fa640 1 -- 172.21.15.83:0/371983044 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f68ec014430 con 0x55970dd08360 2024-04-17T10:48:57.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 1 -- 172.21.15.83:0/371983044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f68ec004b70 con 0x55970dd08360 2024-04-17T10:48:57.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68d57fa640 1 -- 172.21.15.83:0/371983044 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68ec045050 con 0x55970dd08360 2024-04-17T10:48:57.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 10 client.8892 fetch_fsmap learned FSMap version 69 2024-04-17T10:48:57.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 1 -- 172.21.15.83:0/371983044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x55970dab2f80 con 0x55970dd08360 2024-04-17T10:48:57.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68d57fa640 1 -- 172.21.15.83:0/371983044 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ec004b70 con 0x55970dd08360 2024-04-17T10:48:57.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68d57fa640 1 -- 172.21.15.83:0/371983044 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f68ec048020 con 0x55970dd08360 2024-04-17T10:48:57.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 10 client.8892 fetch_fsmap finished waiting for FSMap version 69 2024-04-17T10:48:57.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 10 client.8892 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-17T10:48:57.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 20 client.8892 populate_metadata read hostname 'smithi083' 2024-04-17T10:48:57.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 1 --2- 172.21.15.83:0/371983044 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x55970dd0d8c0 0x55970dd0dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 4 client.8892 mds_command: new command op to 8604 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-17T10:48:57.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 1 -- 172.21.15.83:0/371983044 --> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x55970db193d0 con 0x55970dd0d8c0 2024-04-17T10:48:57.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68fa378640 1 --2- 172.21.15.83:0/371983044 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x55970dd0d8c0 0x55970dd0dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68fa378640 1 --2- 172.21.15.83:0/371983044 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x55970dd0d8c0 0x55970dd0dcd0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c26b399d11a13d2 server_cookie=4b71f2321de0ec70 in_seq=0 out_seq=0 2024-04-17T10:48:57.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68d57fa640 10 client.8892 ms_handle_connect on v2:172.21.15.83:6836/1479417998 2024-04-17T10:48:57.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68d57fa640 1 -- 172.21.15.83:0/371983044 <== mds.0 v2:172.21.15.83:6836/1479417998 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x7f68f00c7e50 con 0x55970dd0d8c0 2024-04-17T10:48:57.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68d57fa640 10 client.8892 handle_command_reply: tid=0 2024-04-17T10:48:57.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 1 client.8892 shutdown 2024-04-17T10:48:57.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 1 -- 172.21.15.83:0/371983044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55970dd08360 msgr2=0x55970dd0ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 1 --2- 172.21.15.83:0/371983044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55970dd08360 0x55970dd0ced0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f68ec001a30 tx=0x7f68ec009040 comp rx=0 tx=0).stop 2024-04-17T10:48:57.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 1 -- 172.21.15.83:0/371983044 shutdown_connections 2024-04-17T10:48:57.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 1 --2- 172.21.15.83:0/371983044 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x55970dd0d8c0 0x55970dd0dcd0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 1 --2- 172.21.15.83:0/371983044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55970dd08360 0x55970dd0ced0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 1 --2- 172.21.15.83:0/371983044 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55970dd07a40 0x55970dd687e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68d57fa640 0 client.8892 ms_handle_reset on v2:172.21.15.83:6836/1479417998 2024-04-17T10:48:57.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 1 --2- 172.21.15.83:0/371983044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55970dcc2d90 0x55970dd682a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 1 -- 172.21.15.83:0/371983044 >> 172.21.15.83:0/371983044 conn(0x55970dcbed40 msgr2=0x55970dd06a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:57.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 1 -- 172.21.15.83:0/371983044 shutdown_connections 2024-04-17T10:48:57.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.241+0000 7f68ff749000 1 -- 172.21.15.83:0/371983044 wait complete. 2024-04-17T10:48:57.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.245+0000 7f68ff749000 20 client.8892 trim_cache size 0 max 16384 2024-04-17T10:48:57.245 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:48:57.245 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0 2024-04-17T10:48:57.245 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:48:57.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.245+0000 7f68fc5dc640 1 -- 172.21.15.83:0/742616412 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f68d8066f90 msgr2=0x7f68d8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.245+0000 7f68fc5dc640 1 --2- 172.21.15.83:0/742616412 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f68d8066f90 0x7f68d8069450 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f68f00ad000 tx=0x7f68f00a6000 comp rx=0 tx=0).stop 2024-04-17T10:48:57.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.245+0000 7f68fc5dc640 1 -- 172.21.15.83:0/742616412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68f415b6e0 msgr2=0x7f68f41f6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.245+0000 7f68fc5dc640 1 --2- 172.21.15.83:0/742616412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68f415b6e0 0x7f68f41f6750 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f68e400c820 tx=0x7f68e400cce0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.245+0000 7f68fc5dc640 1 -- 172.21.15.83:0/742616412 shutdown_connections 2024-04-17T10:48:57.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.245+0000 7f68fc5dc640 1 --2- 172.21.15.83:0/742616412 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f68d8066f90 0x7f68d8069450 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.245+0000 7f68fc5dc640 1 --2- 172.21.15.83:0/742616412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68f415bac0 0x7f68f414a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.245+0000 7f68fc5dc640 1 --2- 172.21.15.83:0/742616412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68f415b6e0 0x7f68f41f6750 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.245+0000 7f68fc5dc640 1 --2- 172.21.15.83:0/742616412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68f4070470 0x7f68f4155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.245+0000 7f68fc5dc640 1 -- 172.21.15.83:0/742616412 >> 172.21.15.83:0/742616412 conn(0x7f68f40b15b0 msgr2=0x7f68f4152c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:57.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.245+0000 7f68fc5dc640 1 -- 172.21.15.83:0/742616412 shutdown_connections 2024-04-17T10:48:57.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.245+0000 7f68fc5dc640 1 -- 172.21.15.83:0/742616412 wait complete. 2024-04-17T10:48:57.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:57.257 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-17T10:48:57.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.373+0000 7f744f7fd640 1 Processor -- start 2024-04-17T10:48:57.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.373+0000 7f744f7fd640 1 -- start start 2024-04-17T10:48:57.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.373+0000 7f744f7fd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f744815e0f0 0x7f74480b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.373+0000 7f744f7fd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f74480b5450 0x7f74480b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.373+0000 7f744f7fd640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f744815e4d0 con 0x7f74480ce400 2024-04-17T10:48:57.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.373+0000 7f744f7fd640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74480b5df0 con 0x7f74480b5450 2024-04-17T10:48:57.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.373+0000 7f744f7fd640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7448162b40 con 0x7f744815e0f0 2024-04-17T10:48:57.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.373+0000 7f744dd9a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f74480b5450 0x7f74480b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.373+0000 7f744dd9a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f74480b5450 0x7f74480b58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40428/0 (socket says 172.21.15.83:40428) 2024-04-17T10:48:57.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.373+0000 7f744dd9a640 1 -- 172.21.15.83:0/2616976327 learned_addr learned my addr 172.21.15.83:0/2616976327 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:57.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.373+0000 7f744d599640 1 --2- 172.21.15.83:0/2616976327 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f744815e0f0 0x7f74480b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.373+0000 7f744dd9a640 1 -- 172.21.15.83:0/2616976327 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f744815e0f0 msgr2=0x7f74480b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:57.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.373+0000 7f744dd9a640 1 --2- 172.21.15.83:0/2616976327 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f744815e0f0 0x7f74480b6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.373+0000 7f744dd9a640 1 -- 172.21.15.83:0/2616976327 >> v1:172.21.15.167:6789/0 conn(0x7f74480ce400 legacy=0x7f7448161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.373+0000 7f744dd9a640 1 -- 172.21.15.83:0/2616976327 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7448162580 con 0x7f74480b5450 2024-04-17T10:48:57.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.377+0000 7f744dd9a640 1 --2- 172.21.15.83:0/2616976327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74480b5450 0x7f74480b58b0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f7440009e80 tx=0x7f744001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733eca8971cca6b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:57.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.377+0000 7f743ffff640 1 -- 172.21.15.83:0/2616976327 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74400a5640 con 0x7f74480b5450 2024-04-17T10:48:57.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.377+0000 7f743ffff640 1 -- 172.21.15.83:0/2616976327 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74400a57e0 con 0x7f74480b5450 2024-04-17T10:48:57.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.377+0000 7f743ffff640 1 -- 172.21.15.83:0/2616976327 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74400a5bb0 con 0x7f74480b5450 2024-04-17T10:48:57.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.377+0000 7f744f7fd640 1 -- 172.21.15.83:0/2616976327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74480b5450 msgr2=0x7f74480b58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.377+0000 7f744f7fd640 1 --2- 172.21.15.83:0/2616976327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74480b5450 0x7f74480b58b0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f7440009e80 tx=0x7f744001c980 comp rx=0 tx=0).stop 2024-04-17T10:48:57.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.377+0000 7f744f7fd640 1 -- 172.21.15.83:0/2616976327 shutdown_connections 2024-04-17T10:48:57.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.377+0000 7f744f7fd640 1 --2- 172.21.15.83:0/2616976327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74480b5450 0x7f74480b58b0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.377+0000 7f744f7fd640 1 --2- 172.21.15.83:0/2616976327 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f744815e0f0 0x7f74480b6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.377+0000 7f744f7fd640 1 -- 172.21.15.83:0/2616976327 >> 172.21.15.83:0/2616976327 conn(0x7f74480d0450 msgr2=0x7f74480d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:57.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.377+0000 7f744f7fd640 1 -- 172.21.15.83:0/2616976327 shutdown_connections 2024-04-17T10:48:57.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.377+0000 7f744f7fd640 1 -- 172.21.15.83:0/2616976327 wait complete. 2024-04-17T10:48:57.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.381+0000 7f744f7fd640 1 Processor -- start 2024-04-17T10:48:57.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.381+0000 7f744f7fd640 1 -- start start 2024-04-17T10:48:57.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.381+0000 7f744f7fd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74480b5450 0x7f744806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.381+0000 7f744f7fd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74480ce400 0x7f744806d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.381+0000 7f744f7fd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f744815e0f0 0x7f74480c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.381+0000 7f744f7fd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74480d2ed0 con 0x7f74480ce400 2024-04-17T10:48:57.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.381+0000 7f744f7fd640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74480d2d50 con 0x7f74480b5450 2024-04-17T10:48:57.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.381+0000 7f744f7fd640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74480d3050 con 0x7f744815e0f0 2024-04-17T10:48:57.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.381+0000 7f744cd98640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74480ce400 0x7f744806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.381+0000 7f744cd98640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74480ce400 0x7f744806d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40432/0 (socket says 172.21.15.83:40432) 2024-04-17T10:48:57.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.381+0000 7f744cd98640 1 -- 172.21.15.83:0/490483094 learned_addr learned my addr 172.21.15.83:0/490483094 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:57.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.381+0000 7f744d599640 1 --2- 172.21.15.83:0/490483094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74480b5450 0x7f744806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.381+0000 7f744dd9a640 1 --2- 172.21.15.83:0/490483094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f744815e0f0 0x7f74480c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.381+0000 7f744cd98640 1 -- 172.21.15.83:0/490483094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f744815e0f0 msgr2=0x7f74480c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.381+0000 7f744cd98640 1 --2- 172.21.15.83:0/490483094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f744815e0f0 0x7f74480c8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.381+0000 7f744cd98640 1 -- 172.21.15.83:0/490483094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74480b5450 msgr2=0x7f744806ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.381+0000 7f744cd98640 1 --2- 172.21.15.83:0/490483094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74480b5450 0x7f744806ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.381+0000 7f744cd98640 1 -- 172.21.15.83:0/490483094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7448160dd0 con 0x7f74480ce400 2024-04-17T10:48:57.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.381+0000 7f744dd9a640 1 --2- 172.21.15.83:0/490483094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f744815e0f0 0x7f74480c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:57.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.381+0000 7f744cd98640 1 --2- 172.21.15.83:0/490483094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74480ce400 0x7f744806d210 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f743800c430 tx=0x7f743800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:57.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.381+0000 7f743dffb640 1 -- 172.21.15.83:0/490483094 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74380052d0 con 0x7f74480ce400 2024-04-17T10:48:57.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.381+0000 7f743dffb640 1 -- 172.21.15.83:0/490483094 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7438005470 con 0x7f74480ce400 2024-04-17T10:48:57.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.381+0000 7f744f7fd640 1 -- 172.21.15.83:0/490483094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f744815ec60 con 0x7f74480ce400 2024-04-17T10:48:57.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.381+0000 7f743dffb640 1 -- 172.21.15.83:0/490483094 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7438007260 con 0x7f74480ce400 2024-04-17T10:48:57.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.381+0000 7f744f7fd640 1 -- 172.21.15.83:0/490483094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7448161330 con 0x7f74480ce400 2024-04-17T10:48:57.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.381+0000 7f743dffb640 1 -- 172.21.15.83:0/490483094 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7438007400 con 0x7f74480ce400 2024-04-17T10:48:57.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.381+0000 7f743dffb640 1 --2- 172.21.15.83:0/490483094 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f742c066f60 0x7f742c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.381+0000 7f743dffb640 1 -- 172.21.15.83:0/490483094 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7438084580 con 0x7f74480ce400 2024-04-17T10:48:57.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.381+0000 7f744d599640 1 --2- 172.21.15.83:0/490483094 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f742c066f60 0x7f742c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.385+0000 7f744d599640 1 --2- 172.21.15.83:0/490483094 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f742c066f60 0x7f742c069420 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f74480cf2d0 tx=0x7f7444008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:57.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.385+0000 7f745296a000 1 Processor -- start 2024-04-17T10:48:57.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.385+0000 7f745296a000 1 -- start start 2024-04-17T10:48:57.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.385+0000 7f745296a000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589af6bc9d0 0x5589af6bcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.385+0000 7f745296a000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5589af6bd2f0 0x5589af672b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.385+0000 7f745296a000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589af6730a0 0x5589af673480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.385+0000 7f745296a000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589af4bfff0 con 0x5589af6730a0 2024-04-17T10:48:57.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.385+0000 7f745296a000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589af448150 con 0x5589af6bc9d0 2024-04-17T10:48:57.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.385+0000 7f745296a000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5589af4bc090 con 0x5589af6bd2f0 2024-04-17T10:48:57.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.385+0000 7f744dd9a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589af6730a0 0x5589af673480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.385+0000 7f744dd9a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589af6730a0 0x5589af673480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40434/0 (socket says 172.21.15.83:40434) 2024-04-17T10:48:57.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.385+0000 7f744dd9a640 1 -- 172.21.15.83:0/624766564 learned_addr learned my addr 172.21.15.83:0/624766564 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:57.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.385+0000 7f744cd98640 1 --2- 172.21.15.83:0/624766564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5589af6bd2f0 0x5589af672b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.385+0000 7f744d599640 1 --2- 172.21.15.83:0/624766564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589af6bc9d0 0x5589af6bcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.385+0000 7f744dd9a640 1 -- 172.21.15.83:0/624766564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5589af6bd2f0 msgr2=0x5589af672b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:57.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.385+0000 7f744dd9a640 1 --2- 172.21.15.83:0/624766564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5589af6bd2f0 0x5589af672b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.385+0000 7f744dd9a640 1 -- 172.21.15.83:0/624766564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589af6bc9d0 msgr2=0x5589af6bcdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:57.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.385+0000 7f744dd9a640 1 --2- 172.21.15.83:0/624766564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589af6bc9d0 0x5589af6bcdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.385+0000 7f744dd9a640 1 -- 172.21.15.83:0/624766564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589af6560d0 con 0x5589af6730a0 2024-04-17T10:48:57.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.385+0000 7f744dd9a640 1 --2- 172.21.15.83:0/624766564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589af6730a0 0x5589af673480 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f74400af040 tx=0x7f7440004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0d5d95fec38afb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:57.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.385+0000 7f7432ffd640 1 -- 172.21.15.83:0/624766564 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74400be070 con 0x5589af6730a0 2024-04-17T10:48:57.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.385+0000 7f7432ffd640 1 -- 172.21.15.83:0/624766564 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74400a5640 con 0x5589af6730a0 2024-04-17T10:48:57.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.385+0000 7f7432ffd640 1 -- 172.21.15.83:0/624766564 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74400a59f0 con 0x5589af6730a0 2024-04-17T10:48:57.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.385+0000 7f745296a000 1 -- 172.21.15.83:0/624766564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589af6730a0 msgr2=0x5589af673480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.385+0000 7f745296a000 1 --2- 172.21.15.83:0/624766564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589af6730a0 0x5589af673480 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f74400af040 tx=0x7f7440004ac0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.385+0000 7f745296a000 1 -- 172.21.15.83:0/624766564 shutdown_connections 2024-04-17T10:48:57.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.385+0000 7f745296a000 1 --2- 172.21.15.83:0/624766564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589af6730a0 0x5589af673480 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.385+0000 7f745296a000 1 --2- 172.21.15.83:0/624766564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5589af6bd2f0 0x5589af672b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.385+0000 7f745296a000 1 --2- 172.21.15.83:0/624766564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589af6bc9d0 0x5589af6bcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.385+0000 7f745296a000 1 -- 172.21.15.83:0/624766564 >> 172.21.15.83:0/624766564 conn(0x5589af66f020 msgr2=0x5589af671440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:57.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.385+0000 7f745296a000 1 -- 172.21.15.83:0/624766564 shutdown_connections 2024-04-17T10:48:57.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.385+0000 7f745296a000 1 -- 172.21.15.83:0/624766564 wait complete. 2024-04-17T10:48:57.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f745296a000 1 -- start start 2024-04-17T10:48:57.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f745296a000 1 Processor -- start 2024-04-17T10:48:57.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f745296a000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5589af6730a0 0x5589af6a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f745296a000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589af6bc9d0 0x5589af6a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f745296a000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589af6bd2f0 0x5589af6a2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f745296a000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589af484250 con 0x5589af6bc9d0 2024-04-17T10:48:57.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f745296a000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589af47a690 con 0x5589af6bd2f0 2024-04-17T10:48:57.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f745296a000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5589af448a40 con 0x5589af6730a0 2024-04-17T10:48:57.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f745296a000 10 client.0 _ll_register_callbacks cb 0x5589af65de90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:48:57.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f744cd98640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589af6bc9d0 0x5589af6a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f744cd98640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589af6bc9d0 0x5589af6a1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40436/0 (socket says 172.21.15.83:40436) 2024-04-17T10:48:57.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f744cd98640 1 -- 172.21.15.83:0/3155369902 learned_addr learned my addr 172.21.15.83:0/3155369902 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:57.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f744dd9a640 1 --2- 172.21.15.83:0/3155369902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589af6bd2f0 0x5589af6a2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f744d599640 1 --2- 172.21.15.83:0/3155369902 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5589af6730a0 0x5589af6a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f744cd98640 1 -- 172.21.15.83:0/3155369902 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5589af6730a0 msgr2=0x5589af6a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:57.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f744cd98640 1 --2- 172.21.15.83:0/3155369902 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5589af6730a0 0x5589af6a17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f744cd98640 1 -- 172.21.15.83:0/3155369902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589af6bd2f0 msgr2=0x5589af6a2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:57.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f744cd98640 1 --2- 172.21.15.83:0/3155369902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589af6bd2f0 0x5589af6a2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f744cd98640 1 -- 172.21.15.83:0/3155369902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5589af56d1c0 con 0x5589af6bc9d0 2024-04-17T10:48:57.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f744cd98640 1 --2- 172.21.15.83:0/3155369902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589af6bc9d0 0x5589af6a1d30 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f743805d030 tx=0x7f7438054f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9bc2186d58f538 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:57.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f7427fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:48:57.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f7427fff640 1 -- 172.21.15.83:0/3155369902 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743805d0d0 con 0x5589af6bc9d0 2024-04-17T10:48:57.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f7427fff640 1 -- 172.21.15.83:0/3155369902 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f743805d270 con 0x5589af6bc9d0 2024-04-17T10:48:57.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f745296a000 1 -- 172.21.15.83:0/3155369902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74400a5640 con 0x5589af6bc9d0 2024-04-17T10:48:57.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f7427fff640 1 -- 172.21.15.83:0/3155369902 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f743806cbe0 con 0x5589af6bc9d0 2024-04-17T10:48:57.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f745296a000 10 client.8904 fetch_fsmap learned FSMap version 69 2024-04-17T10:48:57.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f745296a000 1 -- 172.21.15.83:0/3155369902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x5589af569170 con 0x5589af6bc9d0 2024-04-17T10:48:57.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f7427fff640 1 -- 172.21.15.83:0/3155369902 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74400a5640 con 0x5589af6bc9d0 2024-04-17T10:48:57.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f7427fff640 1 -- 172.21.15.83:0/3155369902 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f74380542a0 con 0x5589af6bc9d0 2024-04-17T10:48:57.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f745296a000 10 client.8904 fetch_fsmap finished waiting for FSMap version 69 2024-04-17T10:48:57.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f745296a000 10 client.8904 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-17T10:48:57.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f745296a000 20 client.8904 populate_metadata read hostname 'smithi083' 2024-04-17T10:48:57.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f745296a000 1 --2- 172.21.15.83:0/3155369902 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x5589af6ccc80 0x5589af6cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f745296a000 4 client.8904 mds_command: new command op to 8604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:48:57.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f745296a000 1 -- 172.21.15.83:0/3155369902 --> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5589af45bc60 con 0x5589af6ccc80 2024-04-17T10:48:57.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f744d599640 1 --2- 172.21.15.83:0/3155369902 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x5589af6ccc80 0x5589af6cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f7427fff640 10 client.8904 ms_handle_connect on v2:172.21.15.83:6836/1479417998 2024-04-17T10:48:57.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f744d599640 1 --2- 172.21.15.83:0/3155369902 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x5589af6ccc80 0x5589af6cf070 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=512e178a0d61d30 server_cookie=5e7ef681c8d92f1b in_seq=0 out_seq=0 2024-04-17T10:48:57.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f7427fff640 1 -- 172.21.15.83:0/3155369902 <== mds.0 v2:172.21.15.83:6836/1479417998 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5589af45bc60 con 0x5589af6ccc80 2024-04-17T10:48:57.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f7427fff640 10 client.8904 handle_command_reply: tid=0 2024-04-17T10:48:57.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f745296a000 1 client.8904 shutdown 2024-04-17T10:48:57.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f745296a000 1 -- 172.21.15.83:0/3155369902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589af6bc9d0 msgr2=0x5589af6a1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f745296a000 1 --2- 172.21.15.83:0/3155369902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589af6bc9d0 0x5589af6a1d30 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f743805d030 tx=0x7f7438054f10 comp rx=0 tx=0).stop 2024-04-17T10:48:57.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f745296a000 1 -- 172.21.15.83:0/3155369902 shutdown_connections 2024-04-17T10:48:57.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f745296a000 1 --2- 172.21.15.83:0/3155369902 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x5589af6ccc80 0x5589af6cf070 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f745296a000 1 --2- 172.21.15.83:0/3155369902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589af6bd2f0 0x5589af6a2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f745296a000 1 --2- 172.21.15.83:0/3155369902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589af6bc9d0 0x5589af6a1d30 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f7427fff640 0 client.8904 ms_handle_reset on v2:172.21.15.83:6836/1479417998 2024-04-17T10:48:57.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f745296a000 1 --2- 172.21.15.83:0/3155369902 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5589af6730a0 0x5589af6a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f745296a000 1 -- 172.21.15.83:0/3155369902 >> 172.21.15.83:0/3155369902 conn(0x5589af66f020 msgr2=0x5589af671ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:57.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f745296a000 1 -- 172.21.15.83:0/3155369902 shutdown_connections 2024-04-17T10:48:57.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.389+0000 7f745296a000 1 -- 172.21.15.83:0/3155369902 wait complete. 2024-04-17T10:48:57.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.393+0000 7f745296a000 20 client.8904 trim_cache size 0 max 16384 2024-04-17T10:48:57.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f745296a000 1 Processor -- start 2024-04-17T10:48:57.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f745296a000 1 -- start start 2024-04-17T10:48:57.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f745296a000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589af6730a0 0x5589af7190f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f745296a000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5589af6bc9d0 0x5589af719630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f745296a000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589af6bd2f0 0x5589af719b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f745296a000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7448202c80 con 0x5589af6730a0 2024-04-17T10:48:57.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f745296a000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7448201e20 con 0x5589af6bd2f0 2024-04-17T10:48:57.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f745296a000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74482012f0 con 0x5589af6bc9d0 2024-04-17T10:48:57.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f744d599640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589af6730a0 0x5589af7190f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f744d599640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589af6730a0 0x5589af7190f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40442/0 (socket says 172.21.15.83:40442) 2024-04-17T10:48:57.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f744d599640 1 -- 172.21.15.83:0/238672592 learned_addr learned my addr 172.21.15.83:0/238672592 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:57.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f744dd9a640 1 --2- 172.21.15.83:0/238672592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589af6bd2f0 0x5589af719b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f744cd98640 1 --2- 172.21.15.83:0/238672592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5589af6bc9d0 0x5589af719630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f744d599640 1 -- 172.21.15.83:0/238672592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5589af6bc9d0 msgr2=0x5589af719630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:57.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f744d599640 1 --2- 172.21.15.83:0/238672592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5589af6bc9d0 0x5589af719630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f744d599640 1 -- 172.21.15.83:0/238672592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589af6bd2f0 msgr2=0x5589af719b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:57.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f744d599640 1 --2- 172.21.15.83:0/238672592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589af6bd2f0 0x5589af719b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f744d599640 1 -- 172.21.15.83:0/238672592 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589af506de0 con 0x5589af6730a0 2024-04-17T10:48:57.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f744cd98640 1 --2- 172.21.15.83:0/238672592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5589af6bc9d0 0x5589af719630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:57.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f744d599640 1 --2- 172.21.15.83:0/238672592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589af6730a0 0x5589af7190f0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f744400f970 tx=0x7f7444015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b39764ba649085 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:57.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f74327fc640 1 -- 172.21.15.83:0/238672592 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744401b070 con 0x5589af6730a0 2024-04-17T10:48:57.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f74327fc640 1 -- 172.21.15.83:0/238672592 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f744402d070 con 0x5589af6730a0 2024-04-17T10:48:57.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f74327fc640 1 -- 172.21.15.83:0/238672592 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7444014d60 con 0x5589af6730a0 2024-04-17T10:48:57.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f745296a000 1 -- 172.21.15.83:0/238672592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589af6730a0 msgr2=0x5589af7190f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f745296a000 1 --2- 172.21.15.83:0/238672592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589af6730a0 0x5589af7190f0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f744400f970 tx=0x7f7444015000 comp rx=0 tx=0).stop 2024-04-17T10:48:57.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f745296a000 1 -- 172.21.15.83:0/238672592 shutdown_connections 2024-04-17T10:48:57.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f745296a000 1 --2- 172.21.15.83:0/238672592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589af6bd2f0 0x5589af719b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f745296a000 1 --2- 172.21.15.83:0/238672592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5589af6bc9d0 0x5589af719630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f745296a000 1 --2- 172.21.15.83:0/238672592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589af6730a0 0x5589af7190f0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f745296a000 1 -- 172.21.15.83:0/238672592 >> 172.21.15.83:0/238672592 conn(0x5589af66f020 msgr2=0x5589af6c9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:57.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f745296a000 1 -- 172.21.15.83:0/238672592 shutdown_connections 2024-04-17T10:48:57.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f745296a000 1 -- 172.21.15.83:0/238672592 wait complete. 2024-04-17T10:48:57.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f745296a000 1 -- start start 2024-04-17T10:48:57.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.409+0000 7f745296a000 1 Processor -- start 2024-04-17T10:48:57.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f745296a000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589af6730a0 0x5589af6cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f745296a000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589af6bc9d0 0x5589af6cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f745296a000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5589af6bd2f0 0x5589af6cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f745296a000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589af45ae00 con 0x5589af6bc9d0 2024-04-17T10:48:57.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f745296a000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7448201820 con 0x5589af6730a0 2024-04-17T10:48:57.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f745296a000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5589af4bf300 con 0x5589af6bd2f0 2024-04-17T10:48:57.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f745296a000 10 client.0 _ll_register_callbacks cb 0x5589af63e040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:48:57.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f744cd98640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589af6bc9d0 0x5589af6cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f744cd98640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589af6bc9d0 0x5589af6cf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40450/0 (socket says 172.21.15.83:40450) 2024-04-17T10:48:57.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f744cd98640 1 -- 172.21.15.83:0/2194037636 learned_addr learned my addr 172.21.15.83:0/2194037636 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:57.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f744d599640 1 --2- 172.21.15.83:0/2194037636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589af6730a0 0x5589af6cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f744dd9a640 1 --2- 172.21.15.83:0/2194037636 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5589af6bd2f0 0x5589af6cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f744cd98640 1 -- 172.21.15.83:0/2194037636 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5589af6bd2f0 msgr2=0x5589af6cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:57.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f744cd98640 1 --2- 172.21.15.83:0/2194037636 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5589af6bd2f0 0x5589af6cfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f744cd98640 1 -- 172.21.15.83:0/2194037636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589af6730a0 msgr2=0x5589af6cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:57.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f744cd98640 1 --2- 172.21.15.83:0/2194037636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589af6730a0 0x5589af6cf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f744cd98640 1 -- 172.21.15.83:0/2194037636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5589af4618e0 con 0x5589af6bc9d0 2024-04-17T10:48:57.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f744cd98640 1 --2- 172.21.15.83:0/2194037636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589af6bc9d0 0x5589af6cf8c0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f7438000fc0 tx=0x7f743801d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c1c83fd17e0424 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:57.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f7427fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:48:57.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f7427fff640 1 -- 172.21.15.83:0/2194037636 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74380510f0 con 0x5589af6bc9d0 2024-04-17T10:48:57.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f7427fff640 1 -- 172.21.15.83:0/2194037636 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f743807eb50 con 0x5589af6bc9d0 2024-04-17T10:48:57.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f745296a000 1 -- 172.21.15.83:0/2194037636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f744402d070 con 0x5589af6bc9d0 2024-04-17T10:48:57.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f7427fff640 1 -- 172.21.15.83:0/2194037636 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7438055550 con 0x5589af6bc9d0 2024-04-17T10:48:57.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f745296a000 10 client.8910 fetch_fsmap learned FSMap version 69 2024-04-17T10:48:57.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f745296a000 1 -- 172.21.15.83:0/2194037636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x5589af5198f0 con 0x5589af6bc9d0 2024-04-17T10:48:57.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f7427fff640 1 -- 172.21.15.83:0/2194037636 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f744402d070 con 0x5589af6bc9d0 2024-04-17T10:48:57.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f7427fff640 1 -- 172.21.15.83:0/2194037636 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7438050ba0 con 0x5589af6bc9d0 2024-04-17T10:48:57.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f745296a000 10 client.8910 fetch_fsmap finished waiting for FSMap version 69 2024-04-17T10:48:57.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f745296a000 10 client.8910 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-17T10:48:57.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f745296a000 20 client.8910 populate_metadata read hostname 'smithi083' 2024-04-17T10:48:57.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f745296a000 1 --2- 172.21.15.83:0/2194037636 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x5589af6d0a30 0x5589af724f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f745296a000 4 client.8910 mds_command: new command op to 8604 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T10:48:57.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f745296a000 1 -- 172.21.15.83:0/2194037636 --> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5589af4cd530 con 0x5589af6d0a30 2024-04-17T10:48:57.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f744d599640 1 --2- 172.21.15.83:0/2194037636 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x5589af6d0a30 0x5589af724f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f7427fff640 10 client.8910 ms_handle_connect on v2:172.21.15.83:6836/1479417998 2024-04-17T10:48:57.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f744d599640 1 --2- 172.21.15.83:0/2194037636 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x5589af6d0a30 0x5589af724f60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7aa38cdc245d5348 server_cookie=670277e38a11c42b in_seq=0 out_seq=0 2024-04-17T10:48:57.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f7427fff640 1 -- 172.21.15.83:0/2194037636 <== mds.0 v2:172.21.15.83:6836/1479417998 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+69 (crc 0 0 0) 0x5589af4cd530 con 0x5589af6d0a30 2024-04-17T10:48:57.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f7427fff640 10 client.8910 handle_command_reply: tid=0 2024-04-17T10:48:57.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f745296a000 1 client.8910 shutdown 2024-04-17T10:48:57.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f745296a000 1 -- 172.21.15.83:0/2194037636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589af6bc9d0 msgr2=0x5589af6cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f745296a000 1 --2- 172.21.15.83:0/2194037636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589af6bc9d0 0x5589af6cf8c0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f7438000fc0 tx=0x7f743801d040 comp rx=0 tx=0).stop 2024-04-17T10:48:57.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f745296a000 1 -- 172.21.15.83:0/2194037636 shutdown_connections 2024-04-17T10:48:57.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f745296a000 1 --2- 172.21.15.83:0/2194037636 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x5589af6d0a30 0x5589af724f60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f745296a000 1 --2- 172.21.15.83:0/2194037636 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5589af6bd2f0 0x5589af6cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f7427fff640 0 client.8910 ms_handle_reset on v2:172.21.15.83:6836/1479417998 2024-04-17T10:48:57.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f745296a000 1 --2- 172.21.15.83:0/2194037636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589af6bc9d0 0x5589af6cf8c0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f745296a000 1 --2- 172.21.15.83:0/2194037636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589af6730a0 0x5589af6cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f745296a000 1 -- 172.21.15.83:0/2194037636 >> 172.21.15.83:0/2194037636 conn(0x5589af66f020 msgr2=0x5589af6bb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:57.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f745296a000 1 -- 172.21.15.83:0/2194037636 shutdown_connections 2024-04-17T10:48:57.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f745296a000 1 -- 172.21.15.83:0/2194037636 wait complete. 2024-04-17T10:48:57.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f745296a000 20 client.8910 trim_cache size 0 max 16384 2024-04-17T10:48:57.418 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:48:57.418 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-17T10:48:57.419 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-17T10:48:57.419 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:48:57.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f744f7fd640 1 -- 172.21.15.83:0/490483094 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f742c066f60 msgr2=0x7f742c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f744f7fd640 1 --2- 172.21.15.83:0/490483094 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f742c066f60 0x7f742c069420 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f74480cf2d0 tx=0x7f7444008040 comp rx=0 tx=0).stop 2024-04-17T10:48:57.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f744f7fd640 1 -- 172.21.15.83:0/490483094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74480ce400 msgr2=0x7f744806d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f744f7fd640 1 --2- 172.21.15.83:0/490483094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74480ce400 0x7f744806d210 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f743800c430 tx=0x7f743800c8f0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f744f7fd640 1 -- 172.21.15.83:0/490483094 shutdown_connections 2024-04-17T10:48:57.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f744f7fd640 1 --2- 172.21.15.83:0/490483094 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f742c066f60 0x7f742c069420 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f744f7fd640 1 --2- 172.21.15.83:0/490483094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f744815e0f0 0x7f74480c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f744f7fd640 1 --2- 172.21.15.83:0/490483094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74480ce400 0x7f744806d210 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f744f7fd640 1 --2- 172.21.15.83:0/490483094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74480b5450 0x7f744806ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f744f7fd640 1 -- 172.21.15.83:0/490483094 >> 172.21.15.83:0/490483094 conn(0x7f74480d0450 msgr2=0x7f74480c9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:57.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f744f7fd640 1 -- 172.21.15.83:0/490483094 shutdown_connections 2024-04-17T10:48:57.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.413+0000 7f744f7fd640 1 -- 172.21.15.83:0/490483094 wait complete. 2024-04-17T10:48:57.427 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:48:57.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.545+0000 7f3866201640 1 Processor -- start 2024-04-17T10:48:57.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.545+0000 7f3866201640 1 -- start start 2024-04-17T10:48:57.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.545+0000 7f3866201640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f386015ba30 0x7f3860072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.545+0000 7f3866201640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f386015c960 0x7f38600cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.545+0000 7f3866201640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38600cfea0 con 0x7f38600cfa10 2024-04-17T10:48:57.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.545+0000 7f3866201640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3860071ea0 con 0x7f386015ba30 2024-04-17T10:48:57.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.545+0000 7f3866201640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386006e320 con 0x7f386015c960 2024-04-17T10:48:57.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.545+0000 7f385ffff640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f38600cfa10 0x7f386006c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:39434/0 (socket says 172.21.15.83:39434) 2024-04-17T10:48:57.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.545+0000 7f385ffff640 1 -- 172.21.15.83:0/1242405733 learned_addr learned my addr 172.21.15.83:0/1242405733 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:57.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.545+0000 7f385f7fe640 1 --2- 172.21.15.83:0/1242405733 >> v2:172.21.15.167:3300/0 conn(0x7f386015ba30 0x7f3860072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.545+0000 7f385effd640 1 --2- 172.21.15.83:0/1242405733 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f386015c960 0x7f38600cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.545+0000 7f385f7fe640 1 -- 172.21.15.83:0/1242405733 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f386015c960 msgr2=0x7f38600cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:57.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.545+0000 7f385f7fe640 1 --2- 172.21.15.83:0/1242405733 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f386015c960 0x7f38600cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.545+0000 7f385f7fe640 1 -- 172.21.15.83:0/1242405733 >> v1:172.21.15.167:6789/0 conn(0x7f38600cfa10 legacy=0x7f386006c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.545+0000 7f385f7fe640 1 -- 172.21.15.83:0/1242405733 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f386006e040 con 0x7f386015ba30 2024-04-17T10:48:57.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.549+0000 7f385f7fe640 1 --2- 172.21.15.83:0/1242405733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f386015ba30 0x7f3860072330 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f38500097c0 tx=0x7f385001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ca12e57215045f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:57.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.549+0000 7f385e7fc640 1 -- 172.21.15.83:0/1242405733 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38500b0070 con 0x7f386015ba30 2024-04-17T10:48:57.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.549+0000 7f385e7fc640 1 -- 172.21.15.83:0/1242405733 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38500ab600 con 0x7f386015ba30 2024-04-17T10:48:57.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.549+0000 7f385e7fc640 1 -- 172.21.15.83:0/1242405733 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38500ab9d0 con 0x7f386015ba30 2024-04-17T10:48:57.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.549+0000 7f3866201640 1 -- 172.21.15.83:0/1242405733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f386015ba30 msgr2=0x7f3860072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.549+0000 7f3866201640 1 --2- 172.21.15.83:0/1242405733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f386015ba30 0x7f3860072330 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f38500097c0 tx=0x7f385001c410 comp rx=0 tx=0).stop 2024-04-17T10:48:57.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.549+0000 7f3866201640 1 -- 172.21.15.83:0/1242405733 shutdown_connections 2024-04-17T10:48:57.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.549+0000 7f3866201640 1 --2- 172.21.15.83:0/1242405733 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f386015c960 0x7f38600cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.549+0000 7f3866201640 1 --2- 172.21.15.83:0/1242405733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f386015ba30 0x7f3860072330 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.549+0000 7f3866201640 1 -- 172.21.15.83:0/1242405733 >> 172.21.15.83:0/1242405733 conn(0x7f386006abe0 msgr2=0x7f386006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:57.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.549+0000 7f3866201640 1 -- 172.21.15.83:0/1242405733 shutdown_connections 2024-04-17T10:48:57.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.549+0000 7f3866201640 1 -- 172.21.15.83:0/1242405733 wait complete. 2024-04-17T10:48:57.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.549+0000 7f3866201640 1 Processor -- start 2024-04-17T10:48:57.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.549+0000 7f3866201640 1 -- start start 2024-04-17T10:48:57.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.553+0000 7f3866201640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38600cfa10 0x7f3860159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.553+0000 7f3866201640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f386015ba30 0x7f3860153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.553+0000 7f3866201640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f386015c960 0x7f38601535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.553+0000 7f3866201640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386006de20 con 0x7f38600cfa10 2024-04-17T10:48:57.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.553+0000 7f3866201640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386006dca0 con 0x7f386015ba30 2024-04-17T10:48:57.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.553+0000 7f3866201640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386015cd40 con 0x7f386015c960 2024-04-17T10:48:57.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.553+0000 7f385f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38600cfa10 0x7f3860159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.553+0000 7f385f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38600cfa10 0x7f3860159ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40466/0 (socket says 172.21.15.83:40466) 2024-04-17T10:48:57.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.553+0000 7f385f7fe640 1 -- 172.21.15.83:0/1865654092 learned_addr learned my addr 172.21.15.83:0/1865654092 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:57.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.553+0000 7f385ffff640 1 --2- 172.21.15.83:0/1865654092 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f386015c960 0x7f38601535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.553+0000 7f385effd640 1 --2- 172.21.15.83:0/1865654092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f386015ba30 0x7f3860153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.553+0000 7f385f7fe640 1 -- 172.21.15.83:0/1865654092 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f386015c960 msgr2=0x7f38601535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.553+0000 7f385f7fe640 1 --2- 172.21.15.83:0/1865654092 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f386015c960 0x7f38601535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.553+0000 7f385f7fe640 1 -- 172.21.15.83:0/1865654092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f386015ba30 msgr2=0x7f3860153060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.553+0000 7f385f7fe640 1 --2- 172.21.15.83:0/1865654092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f386015ba30 0x7f3860153060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.553+0000 7f385f7fe640 1 -- 172.21.15.83:0/1865654092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f386014f210 con 0x7f38600cfa10 2024-04-17T10:48:57.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.553+0000 7f385f7fe640 1 --2- 172.21.15.83:0/1865654092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38600cfa10 0x7f3860159ec0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f385001a040 tx=0x7f38500191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:57.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.553+0000 7f3847fff640 1 -- 172.21.15.83:0/1865654092 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38500b0050 con 0x7f38600cfa10 2024-04-17T10:48:57.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.553+0000 7f3847fff640 1 -- 172.21.15.83:0/1865654092 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38500bc070 con 0x7f38600cfa10 2024-04-17T10:48:57.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.553+0000 7f3866201640 1 -- 172.21.15.83:0/1865654092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3860154ce0 con 0x7f38600cfa10 2024-04-17T10:48:57.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.553+0000 7f3847fff640 1 -- 172.21.15.83:0/1865654092 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3850006e20 con 0x7f38600cfa10 2024-04-17T10:48:57.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.553+0000 7f3866201640 1 -- 172.21.15.83:0/1865654092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3860153f20 con 0x7f38600cfa10 2024-04-17T10:48:57.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.553+0000 7f3866201640 1 -- 172.21.15.83:0/1865654092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3860074f90 con 0x7f38600cfa10 2024-04-17T10:48:57.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.553+0000 7f3847fff640 1 -- 172.21.15.83:0/1865654092 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f38500b5830 con 0x7f38600cfa10 2024-04-17T10:48:57.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.553+0000 7f3847fff640 1 --2- 172.21.15.83:0/1865654092 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f384c066f60 0x7f384c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.553+0000 7f385effd640 1 --2- 172.21.15.83:0/1865654092 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f384c066f60 0x7f384c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.553+0000 7f3847fff640 1 -- 172.21.15.83:0/1865654092 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f385011f6c0 con 0x7f38600cfa10 2024-04-17T10:48:57.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.557+0000 7f385effd640 1 --2- 172.21.15.83:0/1865654092 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f384c066f60 0x7f384c069420 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3860071430 tx=0x7f3854008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:57.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.561+0000 7f3847fff640 1 -- 172.21.15.83:0/1865654092 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f386014f210 con 0x7f38600cfa10 2024-04-17T10:48:57.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.757+0000 7f3866201640 1 -- 172.21.15.83:0/1865654092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3860152ae0 con 0x7f38600cfa10 2024-04-17T10:48:57.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.757+0000 7f3847fff640 1 -- 172.21.15.83:0/1865654092 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f3850016090 con 0x7f38600cfa10 2024-04-17T10:48:57.755 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:57.755 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5786,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1553533838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1553533838},{"type":"v1","addr":"172.21.15.167:6833","nonce":1553533838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/674536913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":674536913},{"type":"v1","addr":"172.21.15.167:6835","nonce":674536913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8598,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4183890457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4183890457},{"type":"v1","addr":"172.21.15.83:6835","nonce":4183890457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:48:24.738443+0000","modified":"2024-04-17T10:48:26.776273+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8604":{"gid":8604,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1479417998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1479417998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1479417998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-17T10:48:57.755 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 69 2024-04-17T10:48:57.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.761+0000 7f3866201640 1 -- 172.21.15.83:0/1865654092 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f384c066f60 msgr2=0x7f384c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.761+0000 7f3866201640 1 --2- 172.21.15.83:0/1865654092 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f384c066f60 0x7f384c069420 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3860071430 tx=0x7f3854008040 comp rx=0 tx=0).stop 2024-04-17T10:48:57.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.761+0000 7f3866201640 1 -- 172.21.15.83:0/1865654092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38600cfa10 msgr2=0x7f3860159ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.761+0000 7f3866201640 1 --2- 172.21.15.83:0/1865654092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38600cfa10 0x7f3860159ec0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f385001a040 tx=0x7f38500191b0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.761+0000 7f3866201640 1 -- 172.21.15.83:0/1865654092 shutdown_connections 2024-04-17T10:48:57.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.761+0000 7f3866201640 1 --2- 172.21.15.83:0/1865654092 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f384c066f60 0x7f384c069420 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.761+0000 7f3866201640 1 --2- 172.21.15.83:0/1865654092 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f386015c960 0x7f38601535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.761+0000 7f3866201640 1 --2- 172.21.15.83:0/1865654092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f386015ba30 0x7f3860153060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.761+0000 7f3866201640 1 --2- 172.21.15.83:0/1865654092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38600cfa10 0x7f3860159ec0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.761+0000 7f3866201640 1 -- 172.21.15.83:0/1865654092 >> 172.21.15.83:0/1865654092 conn(0x7f386006abe0 msgr2=0x7f386015dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:57.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.761+0000 7f3866201640 1 -- 172.21.15.83:0/1865654092 shutdown_connections 2024-04-17T10:48:57.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.761+0000 7f3866201640 1 -- 172.21.15.83:0/1865654092 wait complete. 2024-04-17T10:48:57.772 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-17T10:48:57.772 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-17T10:48:57.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.881+0000 7f8af7627640 1 Processor -- start 2024-04-17T10:48:57.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.881+0000 7f8af7627640 1 -- start start 2024-04-17T10:48:57.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.881+0000 7f8af7627640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8af0161670 0x7f8af00d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.881+0000 7f8af7627640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8af00cee30 con 0x7f8af00b9820 2024-04-17T10:48:57.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.881+0000 7f8af7627640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8af0161b30 con 0x7f8af00d0df0 2024-04-17T10:48:57.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.881+0000 7f8af7627640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8af00d1220 con 0x7f8af0161670 2024-04-17T10:48:57.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.881+0000 7f8af53c3640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8af00b9820 0x7f8af015eb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49332/0 (socket says 172.21.15.83:49332) 2024-04-17T10:48:57.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.881+0000 7f8af53c3640 1 -- 172.21.15.83:0/3000336776 learned_addr learned my addr 172.21.15.83:0/3000336776 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:57.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.881+0000 7f8af4bc2640 1 --2- 172.21.15.83:0/3000336776 >> v2:172.21.15.167:3300/0 conn(0x7f8af0161670 0x7f8af00d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.881+0000 7f8ae7fff640 1 -- 172.21.15.83:0/3000336776 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1637292869 0 0) 0x7f8af00cee30 con 0x7f8af00b9820 2024-04-17T10:48:57.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.881+0000 7f8ae7fff640 1 -- 172.21.15.83:0/3000336776 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8adc003530 con 0x7f8af00b9820 2024-04-17T10:48:57.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.881+0000 7f8ae7fff640 1 -- 172.21.15.83:0/3000336776 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 735670926 0 0) 0x7f8adc003530 con 0x7f8af00b9820 2024-04-17T10:48:57.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.881+0000 7f8ae7fff640 1 -- 172.21.15.83:0/3000336776 >> v2:172.21.15.167:3300/0 conn(0x7f8af0161670 msgr2=0x7f8af00d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:57.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.881+0000 7f8ae7fff640 1 --2- 172.21.15.83:0/3000336776 >> v2:172.21.15.167:3300/0 conn(0x7f8af0161670 0x7f8af00d08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.881+0000 7f8ae7fff640 1 -- 172.21.15.83:0/3000336776 >> v1:172.21.15.167:6789/0 conn(0x7f8af00d0df0 legacy=0x7f8af00d1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.885+0000 7f8ae7fff640 1 -- 172.21.15.83:0/3000336776 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af0161e10 con 0x7f8af00b9820 2024-04-17T10:48:57.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.885+0000 7f8ae7fff640 1 -- 172.21.15.83:0/3000336776 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f8aec001680 con 0x7f8af00b9820 2024-04-17T10:48:57.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.885+0000 7f8ae7fff640 1 -- 172.21.15.83:0/3000336776 >> v1:172.21.15.83:6789/0 conn(0x7f8af00b9820 legacy=0x7f8af015eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.885+0000 7f8ae7fff640 1 --2- 172.21.15.83:0/3000336776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8adc0039c0 0x7f8adc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.885+0000 7f8ae7fff640 1 --2- 172.21.15.83:0/3000336776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8adc004810 0x7f8adc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.885+0000 7f8ae7fff640 1 --2- 172.21.15.83:0/3000336776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8adc0051d0 0x7f8adc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.885+0000 7f8ae7fff640 1 -- 172.21.15.83:0/3000336776 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af00d1220 con 0x7f8adc0039c0 2024-04-17T10:48:57.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.885+0000 7f8ae7fff640 1 -- 172.21.15.83:0/3000336776 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc00c170 con 0x7f8adc0051d0 2024-04-17T10:48:57.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.885+0000 7f8ae7fff640 1 -- 172.21.15.83:0/3000336776 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc00c370 con 0x7f8adc004810 2024-04-17T10:48:57.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.885+0000 7f8af53c3640 1 --2- 172.21.15.83:0/3000336776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8adc0039c0 0x7f8adc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.885+0000 7f8af53c3640 1 -- 172.21.15.83:0/3000336776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8adc004810 msgr2=0x7f8adc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:57.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.885+0000 7f8af53c3640 1 --2- 172.21.15.83:0/3000336776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8adc004810 0x7f8adc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.885+0000 7f8af53c3640 1 -- 172.21.15.83:0/3000336776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8adc0051d0 msgr2=0x7f8adc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:57.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.885+0000 7f8af5bc4640 1 --2- 172.21.15.83:0/3000336776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8adc0051d0 0x7f8adc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.885+0000 7f8af53c3640 1 --2- 172.21.15.83:0/3000336776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8adc0051d0 0x7f8adc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.885+0000 7f8af53c3640 1 -- 172.21.15.83:0/3000336776 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8adc003530 con 0x7f8adc0039c0 2024-04-17T10:48:57.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.885+0000 7f8af5bc4640 1 --2- 172.21.15.83:0/3000336776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8adc0051d0 0x7f8adc00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:57.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.885+0000 7f8af5bc4640 1 -- 172.21.15.83:0/3000336776 reap_dead start 2024-04-17T10:48:57.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.885+0000 7f8af53c3640 1 --2- 172.21.15.83:0/3000336776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8adc0039c0 0x7f8adc003de0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f8aec00c210 tx=0x7f8aec01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe38121fad7f7427 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:57.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.885+0000 7f8ae7fff640 1 -- 172.21.15.83:0/3000336776 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec007d40 con 0x7f8adc0039c0 2024-04-17T10:48:57.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.885+0000 7f8ae7fff640 1 -- 172.21.15.83:0/3000336776 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aec0a9070 con 0x7f8adc0039c0 2024-04-17T10:48:57.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.885+0000 7f8ae7fff640 1 -- 172.21.15.83:0/3000336776 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec005d80 con 0x7f8adc0039c0 2024-04-17T10:48:57.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.885+0000 7f8af7627640 1 -- 172.21.15.83:0/3000336776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8adc0039c0 msgr2=0x7f8adc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.885+0000 7f8af7627640 1 --2- 172.21.15.83:0/3000336776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8adc0039c0 0x7f8adc003de0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f8aec00c210 tx=0x7f8aec01cca0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.885+0000 7f8af7627640 1 -- 172.21.15.83:0/3000336776 shutdown_connections 2024-04-17T10:48:57.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.885+0000 7f8af7627640 1 --2- 172.21.15.83:0/3000336776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8adc0039c0 0x7f8adc003de0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.885+0000 7f8af7627640 1 -- 172.21.15.83:0/3000336776 >> 172.21.15.83:0/3000336776 conn(0x7f8af006b280 msgr2=0x7f8af006b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:57.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.885+0000 7f8af7627640 1 -- 172.21.15.83:0/3000336776 shutdown_connections 2024-04-17T10:48:57.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.885+0000 7f8af7627640 1 -- 172.21.15.83:0/3000336776 wait complete. 2024-04-17T10:48:57.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.889+0000 7f8af7627640 1 Processor -- start 2024-04-17T10:48:57.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.889+0000 7f8af7627640 1 -- start start 2024-04-17T10:48:57.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.889+0000 7f8af7627640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8adc0039c0 0x7f8af00cd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.889+0000 7f8af7627640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8af0153b80 0x7f8af0145a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.889+0000 7f8af7627640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8af00c65a0 0x7f8af00b81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.889+0000 7f8af7627640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af00b5f00 con 0x7f8af00c65a0 2024-04-17T10:48:57.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.889+0000 7f8af7627640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af00b5d80 con 0x7f8adc0039c0 2024-04-17T10:48:57.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.889+0000 7f8af7627640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af00b6080 con 0x7f8af0153b80 2024-04-17T10:48:57.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.889+0000 7f8af5bc4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8af00c65a0 0x7f8af00b81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.889+0000 7f8af5bc4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8af00c65a0 0x7f8af00b81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40498/0 (socket says 172.21.15.83:40498) 2024-04-17T10:48:57.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.889+0000 7f8af5bc4640 1 -- 172.21.15.83:0/1695342368 learned_addr learned my addr 172.21.15.83:0/1695342368 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:57.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.889+0000 7f8af53c3640 1 --2- 172.21.15.83:0/1695342368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8adc0039c0 0x7f8af00cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.889+0000 7f8af4bc2640 1 --2- 172.21.15.83:0/1695342368 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8af0153b80 0x7f8af0145a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.889+0000 7f8af5bc4640 1 -- 172.21.15.83:0/1695342368 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8af0153b80 msgr2=0x7f8af0145a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.889+0000 7f8af5bc4640 1 --2- 172.21.15.83:0/1695342368 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8af0153b80 0x7f8af0145a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.889+0000 7f8af5bc4640 1 -- 172.21.15.83:0/1695342368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8adc0039c0 msgr2=0x7f8af00cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:57.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.889+0000 7f8af5bc4640 1 --2- 172.21.15.83:0/1695342368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8adc0039c0 0x7f8af00cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:57.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.889+0000 7f8af5bc4640 1 -- 172.21.15.83:0/1695342368 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af01540b0 con 0x7f8af00c65a0 2024-04-17T10:48:57.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.889+0000 7f8af5bc4640 1 --2- 172.21.15.83:0/1695342368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8af00c65a0 0x7f8af00b81d0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f8af008ce50 tx=0x7f8ae800d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:57.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.889+0000 7f8ae5ffb640 1 -- 172.21.15.83:0/1695342368 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8018070 con 0x7f8af00c65a0 2024-04-17T10:48:57.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.889+0000 7f8ae5ffb640 1 -- 172.21.15.83:0/1695342368 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae8005070 con 0x7f8af00c65a0 2024-04-17T10:48:57.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.889+0000 7f8af7627640 1 -- 172.21.15.83:0/1695342368 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af0154390 con 0x7f8af00c65a0 2024-04-17T10:48:57.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.889+0000 7f8ae5ffb640 1 -- 172.21.15.83:0/1695342368 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8005460 con 0x7f8af00c65a0 2024-04-17T10:48:57.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.889+0000 7f8af7627640 1 -- 172.21.15.83:0/1695342368 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af0154950 con 0x7f8af00c65a0 2024-04-17T10:48:57.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.889+0000 7f8af7627640 1 -- 172.21.15.83:0/1695342368 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af0074f90 con 0x7f8af00c65a0 2024-04-17T10:48:57.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.893+0000 7f8ae5ffb640 1 -- 172.21.15.83:0/1695342368 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8ae8005600 con 0x7f8af00c65a0 2024-04-17T10:48:57.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.893+0000 7f8ae5ffb640 1 --2- 172.21.15.83:0/1695342368 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8ad4066f90 0x7f8ad4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:57.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.893+0000 7f8ae5ffb640 1 -- 172.21.15.83:0/1695342368 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8ae8084380 con 0x7f8af00c65a0 2024-04-17T10:48:57.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.893+0000 7f8af53c3640 1 --2- 172.21.15.83:0/1695342368 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8ad4066f90 0x7f8ad4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:57.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.893+0000 7f8af53c3640 1 --2- 172.21.15.83:0/1695342368 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8ad4066f90 0x7f8ad4069450 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0ab000 tx=0x7f8aec0b0040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:57.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:57.897+0000 7f8ae5ffb640 1 -- 172.21.15.83:0/1695342368 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8ae80553e0 con 0x7f8af00c65a0 2024-04-17T10:48:58.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.105+0000 7f8af7627640 1 -- 172.21.15.83:0/1695342368 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f8af00c70a0 con 0x7f8af00c65a0 2024-04-17T10:48:58.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.105+0000 7f8ae5ffb640 1 -- 172.21.15.83:0/1695342368 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1675 (secure 0 0 0) 0x7f8ae8054ce0 con 0x7f8af00c65a0 2024-04-17T10:48:58.102 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:58.102 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":670,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":116,"num_osds":8,"num_up_osds":8,"osd_up_since":1713350279,"num_in_osds":8,"osd_in_since":1713350270,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":95},{"state_name":"peering","count":2}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076818817,"bytes_used":2655641600,"bytes_avail":770438471680,"bytes_total":773094113280,"inactive_pgs_ratio":0.020618556067347527,"write_bytes_sec":77589788,"read_op_per_sec":0,"write_op_per_sec":78},"fsmap":{"epoch":69,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8604}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-04-17T10:48:56.371485+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-17T10:48:58.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.109+0000 7f8af7627640 1 -- 172.21.15.83:0/1695342368 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8ad4066f90 msgr2=0x7f8ad4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:58.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.109+0000 7f8af7627640 1 --2- 172.21.15.83:0/1695342368 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8ad4066f90 0x7f8ad4069450 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0ab000 tx=0x7f8aec0b0040 comp rx=0 tx=0).stop 2024-04-17T10:48:58.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.109+0000 7f8af7627640 1 -- 172.21.15.83:0/1695342368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8af00c65a0 msgr2=0x7f8af00b81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:58.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.109+0000 7f8af7627640 1 --2- 172.21.15.83:0/1695342368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8af00c65a0 0x7f8af00b81d0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f8af008ce50 tx=0x7f8ae800d9d0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.109+0000 7f8af7627640 1 -- 172.21.15.83:0/1695342368 shutdown_connections 2024-04-17T10:48:58.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.109+0000 7f8af7627640 1 --2- 172.21.15.83:0/1695342368 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8ad4066f90 0x7f8ad4069450 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.109+0000 7f8af7627640 1 --2- 172.21.15.83:0/1695342368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8af00c65a0 0x7f8af00b81d0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.109+0000 7f8af7627640 1 --2- 172.21.15.83:0/1695342368 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8af0153b80 0x7f8af0145a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.109+0000 7f8af7627640 1 --2- 172.21.15.83:0/1695342368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8adc0039c0 0x7f8af00cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.109+0000 7f8af7627640 1 -- 172.21.15.83:0/1695342368 >> 172.21.15.83:0/1695342368 conn(0x7f8af006b280 msgr2=0x7f8af00ca180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:58.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.109+0000 7f8af7627640 1 -- 172.21.15.83:0/1695342368 shutdown_connections 2024-04-17T10:48:58.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.109+0000 7f8af7627640 1 -- 172.21.15.83:0/1695342368 wait complete. 2024-04-17T10:48:58.119 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused'} 2024-04-17T10:48:58.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:48:58.120 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub resume 2024-04-17T10:48:58.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.241+0000 7f65a7c36640 1 Processor -- start 2024-04-17T10:48:58.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.241+0000 7f65a7c36640 1 -- start start 2024-04-17T10:48:58.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.241+0000 7f65a7c36640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f65a015b6e0 0x7f65a0071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.241+0000 7f65a7c36640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f65a015bac0 0x7f65a00cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.241+0000 7f65a7c36640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65a006c1f0 con 0x7f65a0070470 2024-04-17T10:48:58.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.241+0000 7f65a7c36640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65a00cdd30 con 0x7f65a015bac0 2024-04-17T10:48:58.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.241+0000 7f65a7c36640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65a0070a10 con 0x7f65a015b6e0 2024-04-17T10:48:58.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.241+0000 7f65a51d1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f65a015bac0 0x7f65a00cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.241+0000 7f65a51d1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f65a015bac0 0x7f65a00cd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40508/0 (socket says 172.21.15.83:40508) 2024-04-17T10:48:58.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.241+0000 7f65a51d1640 1 -- 172.21.15.83:0/2434740174 learned_addr learned my addr 172.21.15.83:0/2434740174 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:58.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.241+0000 7f65a59d2640 1 --2- 172.21.15.83:0/2434740174 >> v2:172.21.15.167:3300/0 conn(0x7f65a015b6e0 0x7f65a0071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.241+0000 7f65a51d1640 1 -- 172.21.15.83:0/2434740174 >> v2:172.21.15.167:3300/0 conn(0x7f65a015b6e0 msgr2=0x7f65a0071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:58.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.241+0000 7f65a51d1640 1 --2- 172.21.15.83:0/2434740174 >> v2:172.21.15.167:3300/0 conn(0x7f65a015b6e0 0x7f65a0071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.241+0000 7f65a51d1640 1 -- 172.21.15.83:0/2434740174 >> v1:172.21.15.167:6789/0 conn(0x7f65a0070470 legacy=0x7f65a006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:58.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.241+0000 7f65a51d1640 1 -- 172.21.15.83:0/2434740174 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a0070cf0 con 0x7f65a015bac0 2024-04-17T10:48:58.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.245+0000 7f65a51d1640 1 --2- 172.21.15.83:0/2434740174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65a015bac0 0x7f65a00cd7f0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f65a00728a0 tx=0x7f659001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a711c8f964fe7d2f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:58.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.245+0000 7f65a49d0640 1 -- 172.21.15.83:0/2434740174 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65900b0070 con 0x7f65a015bac0 2024-04-17T10:48:58.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.245+0000 7f65a49d0640 1 -- 172.21.15.83:0/2434740174 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65900ab600 con 0x7f65a015bac0 2024-04-17T10:48:58.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.245+0000 7f65a49d0640 1 -- 172.21.15.83:0/2434740174 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65900ab9d0 con 0x7f65a015bac0 2024-04-17T10:48:58.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.245+0000 7f65a7c36640 1 -- 172.21.15.83:0/2434740174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65a015bac0 msgr2=0x7f65a00cd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:58.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.245+0000 7f65a7c36640 1 --2- 172.21.15.83:0/2434740174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65a015bac0 0x7f65a00cd7f0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f65a00728a0 tx=0x7f659001c520 comp rx=0 tx=0).stop 2024-04-17T10:48:58.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.245+0000 7f65a7c36640 1 -- 172.21.15.83:0/2434740174 shutdown_connections 2024-04-17T10:48:58.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.245+0000 7f65a7c36640 1 --2- 172.21.15.83:0/2434740174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65a015bac0 0x7f65a00cd7f0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.245+0000 7f65a7c36640 1 --2- 172.21.15.83:0/2434740174 >> v2:172.21.15.167:3300/0 conn(0x7f65a015b6e0 0x7f65a0071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.245+0000 7f65a7c36640 1 -- 172.21.15.83:0/2434740174 >> 172.21.15.83:0/2434740174 conn(0x7f65a00b15b0 msgr2=0x7f65a00b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:58.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.245+0000 7f65a7c36640 1 -- 172.21.15.83:0/2434740174 shutdown_connections 2024-04-17T10:48:58.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.245+0000 7f65a7c36640 1 -- 172.21.15.83:0/2434740174 wait complete. 2024-04-17T10:48:58.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.245+0000 7f65a7c36640 1 Processor -- start 2024-04-17T10:48:58.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.245+0000 7f65a7c36640 1 -- start start 2024-04-17T10:48:58.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.245+0000 7f65a7c36640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65a0070470 0x7f65a0154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.245+0000 7f65a7c36640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65a015b6e0 0x7f65a0143010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.245+0000 7f65a7c36640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65a015bac0 0x7f65a0155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.245+0000 7f65a7c36640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a006cee0 con 0x7f65a0070470 2024-04-17T10:48:58.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.245+0000 7f65a7c36640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a006d060 con 0x7f65a015b6e0 2024-04-17T10:48:58.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.245+0000 7f65a7c36640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a006d1e0 con 0x7f65a015bac0 2024-04-17T10:48:58.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.245+0000 7f65a59d2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65a0070470 0x7f65a0154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.249+0000 7f65a59d2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65a0070470 0x7f65a0154e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40524/0 (socket says 172.21.15.83:40524) 2024-04-17T10:48:58.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.249+0000 7f65a59d2640 1 -- 172.21.15.83:0/2272009253 learned_addr learned my addr 172.21.15.83:0/2272009253 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:58.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.249+0000 7f65a61d3640 1 --2- 172.21.15.83:0/2272009253 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65a015bac0 0x7f65a0155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.249+0000 7f65a51d1640 1 --2- 172.21.15.83:0/2272009253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65a015b6e0 0x7f65a0143010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.249+0000 7f65a59d2640 1 -- 172.21.15.83:0/2272009253 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65a015bac0 msgr2=0x7f65a0155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:58.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.249+0000 7f65a59d2640 1 --2- 172.21.15.83:0/2272009253 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65a015bac0 0x7f65a0155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.249+0000 7f65a59d2640 1 -- 172.21.15.83:0/2272009253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65a015b6e0 msgr2=0x7f65a0143010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:58.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.249+0000 7f65a59d2640 1 --2- 172.21.15.83:0/2272009253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65a015b6e0 0x7f65a0143010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.249+0000 7f65a59d2640 1 -- 172.21.15.83:0/2272009253 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a0150740 con 0x7f65a0070470 2024-04-17T10:48:58.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.249+0000 7f65a59d2640 1 --2- 172.21.15.83:0/2272009253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65a0070470 0x7f65a0154e80 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f659c00d800 tx=0x7f659c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:58.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.249+0000 7f65967fc640 1 -- 172.21.15.83:0/2272009253 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c00b6c0 con 0x7f65a0070470 2024-04-17T10:48:58.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.249+0000 7f65967fc640 1 -- 172.21.15.83:0/2272009253 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c019070 con 0x7f65a0070470 2024-04-17T10:48:58.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.249+0000 7f65a7c36640 1 -- 172.21.15.83:0/2272009253 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a0143550 con 0x7f65a0070470 2024-04-17T10:48:58.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.249+0000 7f65967fc640 1 -- 172.21.15.83:0/2272009253 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c00ba60 con 0x7f65a0070470 2024-04-17T10:48:58.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.249+0000 7f65a7c36640 1 -- 172.21.15.83:0/2272009253 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a01543a0 con 0x7f65a0070470 2024-04-17T10:48:58.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.249+0000 7f65967fc640 1 -- 172.21.15.83:0/2272009253 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f659c00bc00 con 0x7f65a0070470 2024-04-17T10:48:58.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.249+0000 7f65967fc640 1 --2- 172.21.15.83:0/2272009253 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6584066f90 0x7f6584069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.249+0000 7f65967fc640 1 -- 172.21.15.83:0/2272009253 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f659c0854d0 con 0x7f65a0070470 2024-04-17T10:48:58.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.249+0000 7f65a51d1640 1 --2- 172.21.15.83:0/2272009253 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6584066f90 0x7f6584069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.249+0000 7f65a51d1640 1 --2- 172.21.15.83:0/2272009253 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6584066f90 0x7f6584069450 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f65900ad000 tx=0x7f65900a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:58.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.249+0000 7f65aada3000 1 Processor -- start 2024-04-17T10:48:58.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.249+0000 7f65aada3000 1 -- start start 2024-04-17T10:48:58.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65aada3000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2adbc99d0 0x55a2adbc9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65aada3000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a2adbca2f0 0x55a2adb7fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65aada3000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2adb800a0 0x55a2adb80480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65aada3000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2ad9ccff0 con 0x55a2adbc99d0 2024-04-17T10:48:58.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65aada3000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2ad955150 con 0x55a2adb800a0 2024-04-17T10:48:58.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65aada3000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a2ad9c9090 con 0x55a2adbca2f0 2024-04-17T10:48:58.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65a59d2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2adbc99d0 0x55a2adbc9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65a59d2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2adbc99d0 0x55a2adbc9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40532/0 (socket says 172.21.15.83:40532) 2024-04-17T10:48:58.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65a59d2640 1 -- 172.21.15.83:0/3434937314 learned_addr learned my addr 172.21.15.83:0/3434937314 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:58.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65a51d1640 1 --2- 172.21.15.83:0/3434937314 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a2adbca2f0 0x55a2adb7fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65a59d2640 1 -- 172.21.15.83:0/3434937314 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a2adbca2f0 msgr2=0x55a2adb7fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:58.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65a61d3640 1 --2- 172.21.15.83:0/3434937314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2adb800a0 0x55a2adb80480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65a59d2640 1 --2- 172.21.15.83:0/3434937314 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a2adbca2f0 0x55a2adb7fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65a59d2640 1 -- 172.21.15.83:0/3434937314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2adb800a0 msgr2=0x55a2adb80480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:58.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65a59d2640 1 --2- 172.21.15.83:0/3434937314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2adb800a0 0x55a2adb80480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65a59d2640 1 -- 172.21.15.83:0/3434937314 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2adb630d0 con 0x55a2adbc99d0 2024-04-17T10:48:58.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65a59d2640 1 --2- 172.21.15.83:0/3434937314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2adbc99d0 0x55a2adbc9db0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f659c052200 tx=0x7f659c055830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c76991d8ab43a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:58.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65837fe640 1 -- 172.21.15.83:0/3434937314 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c059030 con 0x55a2adbc99d0 2024-04-17T10:48:58.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65837fe640 1 -- 172.21.15.83:0/3434937314 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c05e4b0 con 0x55a2adbc99d0 2024-04-17T10:48:58.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65837fe640 1 -- 172.21.15.83:0/3434937314 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c05c0c0 con 0x55a2adbc99d0 2024-04-17T10:48:58.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65aada3000 1 -- 172.21.15.83:0/3434937314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2adbc99d0 msgr2=0x55a2adbc9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:58.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65aada3000 1 --2- 172.21.15.83:0/3434937314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2adbc99d0 0x55a2adbc9db0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f659c052200 tx=0x7f659c055830 comp rx=0 tx=0).stop 2024-04-17T10:48:58.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65aada3000 1 -- 172.21.15.83:0/3434937314 shutdown_connections 2024-04-17T10:48:58.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65aada3000 1 --2- 172.21.15.83:0/3434937314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2adb800a0 0x55a2adb80480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65aada3000 1 --2- 172.21.15.83:0/3434937314 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a2adbca2f0 0x55a2adb7fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65aada3000 1 --2- 172.21.15.83:0/3434937314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2adbc99d0 0x55a2adbc9db0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65aada3000 1 -- 172.21.15.83:0/3434937314 >> 172.21.15.83:0/3434937314 conn(0x55a2adb7c020 msgr2=0x55a2adb7e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:58.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65aada3000 1 -- 172.21.15.83:0/3434937314 shutdown_connections 2024-04-17T10:48:58.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65aada3000 1 -- 172.21.15.83:0/3434937314 wait complete. 2024-04-17T10:48:58.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65aada3000 1 -- start start 2024-04-17T10:48:58.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65aada3000 1 Processor -- start 2024-04-17T10:48:58.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65aada3000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a2adb800a0 0x55a2adbae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65aada3000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2adbc99d0 0x55a2adbaed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65aada3000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2adbca2f0 0x55a2adbaf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65aada3000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2ad991250 con 0x55a2adbc99d0 2024-04-17T10:48:58.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65aada3000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2ad987690 con 0x55a2adbca2f0 2024-04-17T10:48:58.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65aada3000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a2ad955a40 con 0x55a2adb800a0 2024-04-17T10:48:58.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65aada3000 10 client.0 _ll_register_callbacks cb 0x55a2adb6ae90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:48:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65a51d1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2adbc99d0 0x55a2adbaed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65a51d1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2adbc99d0 0x55a2adbaed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40542/0 (socket says 172.21.15.83:40542) 2024-04-17T10:48:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65a51d1640 1 -- 172.21.15.83:0/3908139878 learned_addr learned my addr 172.21.15.83:0/3908139878 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65a61d3640 1 --2- 172.21.15.83:0/3908139878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2adbca2f0 0x55a2adbaf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65a59d2640 1 --2- 172.21.15.83:0/3908139878 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a2adb800a0 0x55a2adbae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65a51d1640 1 -- 172.21.15.83:0/3908139878 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a2adb800a0 msgr2=0x55a2adbae7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65a51d1640 1 --2- 172.21.15.83:0/3908139878 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a2adb800a0 0x55a2adbae7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65a51d1640 1 -- 172.21.15.83:0/3908139878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2adbca2f0 msgr2=0x55a2adbaf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65a51d1640 1 --2- 172.21.15.83:0/3908139878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2adbca2f0 0x55a2adbaf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65a51d1640 1 -- 172.21.15.83:0/3908139878 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2ada7a1c0 con 0x55a2adbc99d0 2024-04-17T10:48:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65a61d3640 1 --2- 172.21.15.83:0/3908139878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2adbca2f0 0x55a2adbaf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65a59d2640 1 --2- 172.21.15.83:0/3908139878 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a2adb800a0 0x55a2adbae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:48:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65a51d1640 1 --2- 172.21.15.83:0/3908139878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2adbc99d0 0x55a2adbaed30 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f65900ae040 tx=0x7f659001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc8c077d2852315f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f6580ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:48:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f6580ff9640 1 -- 172.21.15.83:0/3908139878 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65900b0040 con 0x55a2adbc99d0 2024-04-17T10:48:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f6580ff9640 1 -- 172.21.15.83:0/3908139878 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f65900b6670 con 0x55a2adbc99d0 2024-04-17T10:48:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.253+0000 7f65aada3000 1 -- 172.21.15.83:0/3908139878 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f659c05e4b0 con 0x55a2adbc99d0 2024-04-17T10:48:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.257+0000 7f6580ff9640 1 -- 172.21.15.83:0/3908139878 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65900b6870 con 0x55a2adbc99d0 2024-04-17T10:48:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.257+0000 7f65aada3000 10 client.8931 fetch_fsmap learned FSMap version 69 2024-04-17T10:48:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.257+0000 7f65aada3000 1 -- 172.21.15.83:0/3908139878 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x55a2ada76170 con 0x55a2adbc99d0 2024-04-17T10:48:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.257+0000 7f6580ff9640 1 -- 172.21.15.83:0/3908139878 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c05e4b0 con 0x55a2adbc99d0 2024-04-17T10:48:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.257+0000 7f6580ff9640 1 -- 172.21.15.83:0/3908139878 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f65900b6a10 con 0x55a2adbc99d0 2024-04-17T10:48:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.257+0000 7f65aada3000 10 client.8931 fetch_fsmap finished waiting for FSMap version 69 2024-04-17T10:48:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.257+0000 7f65aada3000 10 client.8931 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-17T10:48:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.257+0000 7f65aada3000 20 client.8931 populate_metadata read hostname 'smithi083' 2024-04-17T10:48:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.257+0000 7f65aada3000 1 --2- 172.21.15.83:0/3908139878 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x55a2adbd9c80 0x55a2adbdc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.257+0000 7f65aada3000 4 client.8931 mds_command: new command op to 8604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:48:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.257+0000 7f65aada3000 1 -- 172.21.15.83:0/3908139878 --> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a2ad968c60 con 0x55a2adbd9c80 2024-04-17T10:48:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.257+0000 7f65a59d2640 1 --2- 172.21.15.83:0/3908139878 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x55a2adbd9c80 0x55a2adbdc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.257+0000 7f6580ff9640 10 client.8931 ms_handle_connect on v2:172.21.15.83:6836/1479417998 2024-04-17T10:48:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.257+0000 7f65a59d2640 1 --2- 172.21.15.83:0/3908139878 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x55a2adbd9c80 0x55a2adbdc070 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50dd7a7a35b32ecf server_cookie=fce5d47cb62f38ae in_seq=0 out_seq=0 2024-04-17T10:48:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.257+0000 7f6580ff9640 1 -- 172.21.15.83:0/3908139878 <== mds.0 v2:172.21.15.83:6836/1479417998 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a2ad968c60 con 0x55a2adbd9c80 2024-04-17T10:48:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.257+0000 7f6580ff9640 10 client.8931 handle_command_reply: tid=0 2024-04-17T10:48:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.257+0000 7f65aada3000 1 client.8931 shutdown 2024-04-17T10:48:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.257+0000 7f65aada3000 1 -- 172.21.15.83:0/3908139878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2adbc99d0 msgr2=0x55a2adbaed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.257+0000 7f65aada3000 1 --2- 172.21.15.83:0/3908139878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2adbc99d0 0x55a2adbaed30 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f65900ae040 tx=0x7f659001ad60 comp rx=0 tx=0).stop 2024-04-17T10:48:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.257+0000 7f65aada3000 1 -- 172.21.15.83:0/3908139878 shutdown_connections 2024-04-17T10:48:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.257+0000 7f65aada3000 1 --2- 172.21.15.83:0/3908139878 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x55a2adbd9c80 0x55a2adbdc070 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.257+0000 7f65aada3000 1 --2- 172.21.15.83:0/3908139878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2adbca2f0 0x55a2adbaf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.257+0000 7f65aada3000 1 --2- 172.21.15.83:0/3908139878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2adbc99d0 0x55a2adbaed30 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.257+0000 7f65aada3000 1 --2- 172.21.15.83:0/3908139878 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a2adb800a0 0x55a2adbae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.257+0000 7f65aada3000 1 -- 172.21.15.83:0/3908139878 >> 172.21.15.83:0/3908139878 conn(0x55a2adb7c020 msgr2=0x55a2adb7eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:58.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.257+0000 7f6580ff9640 0 client.8931 ms_handle_reset on v2:172.21.15.83:6836/1479417998 2024-04-17T10:48:58.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.257+0000 7f65aada3000 1 -- 172.21.15.83:0/3908139878 shutdown_connections 2024-04-17T10:48:58.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.257+0000 7f65aada3000 1 -- 172.21.15.83:0/3908139878 wait complete. 2024-04-17T10:48:58.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.257+0000 7f65aada3000 20 client.8931 trim_cache size 0 max 16384 2024-04-17T10:48:58.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.269+0000 7f65aada3000 1 Processor -- start 2024-04-17T10:48:58.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.269+0000 7f65aada3000 1 -- start start 2024-04-17T10:48:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65aada3000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2adb800a0 0x55a2adbddec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65aada3000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2adbc99d0 0x55a2adbc8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65aada3000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a2adbca2f0 0x55a2adbc85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65aada3000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a0202cc0 con 0x55a2adb800a0 2024-04-17T10:48:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65aada3000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a0201ec0 con 0x55a2adbc99d0 2024-04-17T10:48:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65aada3000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a0159a00 con 0x55a2adbca2f0 2024-04-17T10:48:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65a59d2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2adb800a0 0x55a2adbddec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65a59d2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2adb800a0 0x55a2adbddec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40548/0 (socket says 172.21.15.83:40548) 2024-04-17T10:48:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65a59d2640 1 -- 172.21.15.83:0/3456939076 learned_addr learned my addr 172.21.15.83:0/3456939076 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65a59d2640 1 -- 172.21.15.83:0/3456939076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a2adbca2f0 msgr2=0x55a2adbc85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65a59d2640 1 --2- 172.21.15.83:0/3456939076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a2adbca2f0 0x55a2adbc85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65a59d2640 1 -- 172.21.15.83:0/3456939076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2adbc99d0 msgr2=0x55a2adbc8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65a59d2640 1 --2- 172.21.15.83:0/3456939076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2adbc99d0 0x55a2adbc8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65a59d2640 1 -- 172.21.15.83:0/3456939076 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2ada13de0 con 0x55a2adb800a0 2024-04-17T10:48:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65a59d2640 1 --2- 172.21.15.83:0/3456939076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2adb800a0 0x55a2adbddec0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f659c052200 tx=0x7f659c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84f6afaeab627cd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f6582ffd640 1 -- 172.21.15.83:0/3456939076 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c059040 con 0x55a2adb800a0 2024-04-17T10:48:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f6582ffd640 1 -- 172.21.15.83:0/3456939076 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c053ec0 con 0x55a2adb800a0 2024-04-17T10:48:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f6582ffd640 1 -- 172.21.15.83:0/3456939076 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c05e4b0 con 0x55a2adb800a0 2024-04-17T10:48:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65aada3000 1 -- 172.21.15.83:0/3456939076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2adb800a0 msgr2=0x55a2adbddec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65aada3000 1 --2- 172.21.15.83:0/3456939076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2adb800a0 0x55a2adbddec0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f659c052200 tx=0x7f659c015000 comp rx=0 tx=0).stop 2024-04-17T10:48:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65aada3000 1 -- 172.21.15.83:0/3456939076 shutdown_connections 2024-04-17T10:48:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65aada3000 1 --2- 172.21.15.83:0/3456939076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a2adbca2f0 0x55a2adbc85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65aada3000 1 --2- 172.21.15.83:0/3456939076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2adbc99d0 0x55a2adbc8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65aada3000 1 --2- 172.21.15.83:0/3456939076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2adb800a0 0x55a2adbddec0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65aada3000 1 -- 172.21.15.83:0/3456939076 >> 172.21.15.83:0/3456939076 conn(0x55a2adb7c020 msgr2=0x55a2adbac470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65aada3000 1 -- 172.21.15.83:0/3456939076 shutdown_connections 2024-04-17T10:48:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65aada3000 1 -- 172.21.15.83:0/3456939076 wait complete. 2024-04-17T10:48:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65aada3000 1 -- start start 2024-04-17T10:48:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65aada3000 1 Processor -- start 2024-04-17T10:48:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65aada3000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2adb800a0 0x55a2adb7f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65aada3000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2adbc99d0 0x55a2adbdbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65aada3000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a2adbca2f0 0x55a2adbdc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65aada3000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2ad967e00 con 0x55a2adbc99d0 2024-04-17T10:48:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65aada3000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a015d490 con 0x55a2adb800a0 2024-04-17T10:48:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65aada3000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a2ad9cc300 con 0x55a2adbca2f0 2024-04-17T10:48:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65aada3000 10 client.0 _ll_register_callbacks cb 0x55a2adb4b040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:48:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65a51d1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2adbc99d0 0x55a2adbdbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65a51d1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2adbc99d0 0x55a2adbdbc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40564/0 (socket says 172.21.15.83:40564) 2024-04-17T10:48:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65a51d1640 1 -- 172.21.15.83:0/515171246 learned_addr learned my addr 172.21.15.83:0/515171246 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65a51d1640 1 -- 172.21.15.83:0/515171246 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a2adbca2f0 msgr2=0x55a2adbdc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65a51d1640 1 --2- 172.21.15.83:0/515171246 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a2adbca2f0 0x55a2adbdc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65a51d1640 1 -- 172.21.15.83:0/515171246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2adb800a0 msgr2=0x55a2adb7f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65a51d1640 1 --2- 172.21.15.83:0/515171246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2adb800a0 0x55a2adb7f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65a51d1640 1 -- 172.21.15.83:0/515171246 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2ad96e8e0 con 0x55a2adbc99d0 2024-04-17T10:48:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65a51d1640 1 --2- 172.21.15.83:0/515171246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2adbc99d0 0x55a2adbdbc90 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f6590000c10 tx=0x7f65900192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1bdea3c2fd5a6d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f6580ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:48:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f6580ff9640 1 -- 172.21.15.83:0/515171246 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65900b0040 con 0x55a2adbc99d0 2024-04-17T10:48:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f6580ff9640 1 -- 172.21.15.83:0/515171246 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f65900d2450 con 0x55a2adbc99d0 2024-04-17T10:48:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.273+0000 7f65aada3000 1 -- 172.21.15.83:0/515171246 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a2ad9da530 con 0x55a2adbc99d0 2024-04-17T10:48:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f6580ff9640 1 -- 172.21.15.83:0/515171246 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65900d2650 con 0x55a2adbc99d0 2024-04-17T10:48:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65aada3000 10 client.8937 fetch_fsmap learned FSMap version 69 2024-04-17T10:48:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65aada3000 1 -- 172.21.15.83:0/515171246 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x55a2ada268f0 con 0x55a2adbc99d0 2024-04-17T10:48:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f6580ff9640 1 -- 172.21.15.83:0/515171246 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a2ad9da530 con 0x55a2adbc99d0 2024-04-17T10:48:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f6580ff9640 1 -- 172.21.15.83:0/515171246 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f65900b7900 con 0x55a2adbc99d0 2024-04-17T10:48:58.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65aada3000 10 client.8937 fetch_fsmap finished waiting for FSMap version 69 2024-04-17T10:48:58.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65aada3000 10 client.8937 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-17T10:48:58.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65aada3000 20 client.8937 populate_metadata read hostname 'smithi083' 2024-04-17T10:48:58.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65aada3000 1 --2- 172.21.15.83:0/515171246 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x55a2adc29330 0x55a2adc29740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65aada3000 4 client.8937 mds_command: new command op to 8604 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-17T10:48:58.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65aada3000 1 -- 172.21.15.83:0/515171246 --> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x55a2ad9d4880 con 0x55a2adc29330 2024-04-17T10:48:58.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65a59d2640 1 --2- 172.21.15.83:0/515171246 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x55a2adc29330 0x55a2adc29740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f6580ff9640 10 client.8937 ms_handle_connect on v2:172.21.15.83:6836/1479417998 2024-04-17T10:48:58.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65a59d2640 1 --2- 172.21.15.83:0/515171246 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x55a2adc29330 0x55a2adc29740 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea0272ede588c4f8 server_cookie=61f9961609f4511a in_seq=0 out_seq=0 2024-04-17T10:48:58.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f6580ff9640 1 -- 172.21.15.83:0/515171246 <== mds.0 v2:172.21.15.83:6836/1479417998 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55a2ad9d4880 con 0x55a2adc29330 2024-04-17T10:48:58.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f6580ff9640 10 client.8937 handle_command_reply: tid=0 2024-04-17T10:48:58.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65aada3000 1 client.8937 shutdown 2024-04-17T10:48:58.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65aada3000 1 -- 172.21.15.83:0/515171246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2adbc99d0 msgr2=0x55a2adbdbc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:58.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65aada3000 1 --2- 172.21.15.83:0/515171246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2adbc99d0 0x55a2adbdbc90 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f6590000c10 tx=0x7f65900192c0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65aada3000 1 -- 172.21.15.83:0/515171246 shutdown_connections 2024-04-17T10:48:58.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65aada3000 1 --2- 172.21.15.83:0/515171246 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x55a2adc29330 0x55a2adc29740 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65aada3000 1 --2- 172.21.15.83:0/515171246 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a2adbca2f0 0x55a2adbdc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65aada3000 1 --2- 172.21.15.83:0/515171246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2adbc99d0 0x55a2adbdbc90 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f6580ff9640 0 client.8937 ms_handle_reset on v2:172.21.15.83:6836/1479417998 2024-04-17T10:48:58.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65aada3000 1 --2- 172.21.15.83:0/515171246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2adb800a0 0x55a2adb7f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65aada3000 1 -- 172.21.15.83:0/515171246 >> 172.21.15.83:0/515171246 conn(0x55a2adb7c020 msgr2=0x55a2adbd6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:58.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65aada3000 1 -- 172.21.15.83:0/515171246 shutdown_connections 2024-04-17T10:48:58.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65aada3000 1 -- 172.21.15.83:0/515171246 wait complete. 2024-04-17T10:48:58.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65aada3000 20 client.8937 trim_cache size 0 max 16384 2024-04-17T10:48:58.280 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:48:58.280 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0 2024-04-17T10:48:58.280 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:48:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65a7c36640 1 -- 172.21.15.83:0/2272009253 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6584066f90 msgr2=0x7f6584069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65a7c36640 1 --2- 172.21.15.83:0/2272009253 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6584066f90 0x7f6584069450 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f65900ad000 tx=0x7f65900a6000 comp rx=0 tx=0).stop 2024-04-17T10:48:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65a7c36640 1 -- 172.21.15.83:0/2272009253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65a0070470 msgr2=0x7f65a0154e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65a7c36640 1 --2- 172.21.15.83:0/2272009253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65a0070470 0x7f65a0154e80 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f659c00d800 tx=0x7f659c00dcc0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65a7c36640 1 -- 172.21.15.83:0/2272009253 shutdown_connections 2024-04-17T10:48:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65a7c36640 1 --2- 172.21.15.83:0/2272009253 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6584066f90 0x7f6584069450 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65a7c36640 1 --2- 172.21.15.83:0/2272009253 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65a015bac0 0x7f65a0155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65a7c36640 1 --2- 172.21.15.83:0/2272009253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65a015b6e0 0x7f65a0143010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65a7c36640 1 --2- 172.21.15.83:0/2272009253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65a0070470 0x7f65a0154e80 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65a7c36640 1 -- 172.21.15.83:0/2272009253 >> 172.21.15.83:0/2272009253 conn(0x7f65a00b15b0 msgr2=0x7f65a00ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65a7c36640 1 -- 172.21.15.83:0/2272009253 shutdown_connections 2024-04-17T10:48:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.277+0000 7f65a7c36640 1 -- 172.21.15.83:0/2272009253 wait complete. 2024-04-17T10:48:58.289 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-17T10:48:58.289 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-17T10:48:58.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.405+0000 7f1a2c148640 1 Processor -- start 2024-04-17T10:48:58.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.405+0000 7f1a2c148640 1 -- start start 2024-04-17T10:48:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.409+0000 7f1a2c148640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a24154de0 0x7f1a2406ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.409+0000 7f1a2c148640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1a2406b350 0x7f1a240b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.409+0000 7f1a2c148640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a2415bd70 con 0x7f1a240b4040 2024-04-17T10:48:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.409+0000 7f1a2c148640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a240b4870 con 0x7f1a2406b350 2024-04-17T10:48:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.409+0000 7f1a2c148640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2415bf60 con 0x7f1a24154de0 2024-04-17T10:48:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.409+0000 7f1a296e3640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1a2406b350 0x7f1a240b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.409+0000 7f1a2a6e5640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1a240b4040 0x7f1a24071580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49334/0 (socket says 172.21.15.83:49334) 2024-04-17T10:48:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.409+0000 7f1a296e3640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1a2406b350 0x7f1a240b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40570/0 (socket says 172.21.15.83:40570) 2024-04-17T10:48:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.409+0000 7f1a2a6e5640 1 -- 172.21.15.83:0/4111294471 learned_addr learned my addr 172.21.15.83:0/4111294471 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.409+0000 7f1a29ee4640 1 --2- 172.21.15.83:0/4111294471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a24154de0 0x7f1a2406ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.409+0000 7f1a296e3640 1 -- 172.21.15.83:0/4111294471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a24154de0 msgr2=0x7f1a2406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.409+0000 7f1a296e3640 1 --2- 172.21.15.83:0/4111294471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a24154de0 0x7f1a2406ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.409+0000 7f1a296e3640 1 -- 172.21.15.83:0/4111294471 >> v1:172.21.15.83:6789/0 conn(0x7f1a240b4040 legacy=0x7f1a24071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.409+0000 7f1a296e3640 1 -- 172.21.15.83:0/4111294471 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a2415afb0 con 0x7f1a2406b350 2024-04-17T10:48:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.409+0000 7f1a296e3640 1 --2- 172.21.15.83:0/4111294471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a2406b350 0x7f1a240b3b00 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f1a140097c0 tx=0x7f1a1401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5935147472d0e947 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.409+0000 7f1a28ee2640 1 -- 172.21.15.83:0/4111294471 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a140b0070 con 0x7f1a2406b350 2024-04-17T10:48:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.409+0000 7f1a28ee2640 1 -- 172.21.15.83:0/4111294471 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a140a4e50 con 0x7f1a2406b350 2024-04-17T10:48:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.409+0000 7f1a28ee2640 1 -- 172.21.15.83:0/4111294471 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a140ab820 con 0x7f1a2406b350 2024-04-17T10:48:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.409+0000 7f1a2c148640 1 -- 172.21.15.83:0/4111294471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a2406b350 msgr2=0x7f1a240b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.409+0000 7f1a2c148640 1 --2- 172.21.15.83:0/4111294471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a2406b350 0x7f1a240b3b00 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f1a140097c0 tx=0x7f1a1401c4a0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.409+0000 7f1a2c148640 1 -- 172.21.15.83:0/4111294471 shutdown_connections 2024-04-17T10:48:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.409+0000 7f1a2c148640 1 --2- 172.21.15.83:0/4111294471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a2406b350 0x7f1a240b3b00 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.409+0000 7f1a2c148640 1 --2- 172.21.15.83:0/4111294471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a24154de0 0x7f1a2406ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.409+0000 7f1a2c148640 1 -- 172.21.15.83:0/4111294471 >> 172.21.15.83:0/4111294471 conn(0x7f1a240b15b0 msgr2=0x7f1a240b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.409+0000 7f1a2c148640 1 -- 172.21.15.83:0/4111294471 shutdown_connections 2024-04-17T10:48:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.409+0000 7f1a2c148640 1 -- 172.21.15.83:0/4111294471 wait complete. 2024-04-17T10:48:58.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.413+0000 7f1a2c148640 1 Processor -- start 2024-04-17T10:48:58.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.413+0000 7f1a2c148640 1 -- start start 2024-04-17T10:48:58.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.413+0000 7f1a2c148640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a2406b350 0x7f1a2414c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.413+0000 7f1a2c148640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a240b4040 0x7f1a24153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.413+0000 7f1a2c148640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a24154de0 0x7f1a2415cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.413+0000 7f1a2c148640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a240b4450 con 0x7f1a2406b350 2024-04-17T10:48:58.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.413+0000 7f1a2c148640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a241551c0 con 0x7f1a24154de0 2024-04-17T10:48:58.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.413+0000 7f1a2c148640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a240b45d0 con 0x7f1a240b4040 2024-04-17T10:48:58.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.413+0000 7f1a29ee4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a2406b350 0x7f1a2414c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.413+0000 7f1a29ee4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a2406b350 0x7f1a2414c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40572/0 (socket says 172.21.15.83:40572) 2024-04-17T10:48:58.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.413+0000 7f1a29ee4640 1 -- 172.21.15.83:0/2430517115 learned_addr learned my addr 172.21.15.83:0/2430517115 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:58.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.413+0000 7f1a29ee4640 1 -- 172.21.15.83:0/2430517115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a240b4040 msgr2=0x7f1a24153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:58.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.413+0000 7f1a296e3640 1 --2- 172.21.15.83:0/2430517115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a240b4040 0x7f1a24153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.413+0000 7f1a29ee4640 1 --2- 172.21.15.83:0/2430517115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a240b4040 0x7f1a24153f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.413+0000 7f1a29ee4640 1 -- 172.21.15.83:0/2430517115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a24154de0 msgr2=0x7f1a2415cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:58.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.413+0000 7f1a2a6e5640 1 --2- 172.21.15.83:0/2430517115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a24154de0 0x7f1a2415cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.413+0000 7f1a29ee4640 1 --2- 172.21.15.83:0/2430517115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a24154de0 0x7f1a2415cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.413+0000 7f1a29ee4640 1 -- 172.21.15.83:0/2430517115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a24166730 con 0x7f1a2406b350 2024-04-17T10:48:58.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.413+0000 7f1a29ee4640 1 --2- 172.21.15.83:0/2430517115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a2406b350 0x7f1a2414c430 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000d830 tx=0x7f1a2000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:58.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.413+0000 7f1a1a7fc640 1 -- 172.21.15.83:0/2430517115 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2000b6c0 con 0x7f1a2406b350 2024-04-17T10:48:58.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.413+0000 7f1a1a7fc640 1 -- 172.21.15.83:0/2430517115 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a20019070 con 0x7f1a2406b350 2024-04-17T10:48:58.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.413+0000 7f1a2c148640 1 -- 172.21.15.83:0/2430517115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a24166a10 con 0x7f1a2406b350 2024-04-17T10:48:58.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.413+0000 7f1a1a7fc640 1 -- 172.21.15.83:0/2430517115 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2000ba60 con 0x7f1a2406b350 2024-04-17T10:48:58.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.413+0000 7f1a2c148640 1 -- 172.21.15.83:0/2430517115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a24166fd0 con 0x7f1a2406b350 2024-04-17T10:48:58.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.413+0000 7f1a1a7fc640 1 -- 172.21.15.83:0/2430517115 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1a2001e460 con 0x7f1a2406b350 2024-04-17T10:48:58.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a1a7fc640 1 --2- 172.21.15.83:0/2430517115 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1a08066f90 0x7f1a08069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a1a7fc640 1 -- 172.21.15.83:0/2430517115 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1a20085730 con 0x7f1a2406b350 2024-04-17T10:48:58.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a296e3640 1 --2- 172.21.15.83:0/2430517115 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1a08066f90 0x7f1a08069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a296e3640 1 --2- 172.21.15.83:0/2430517115 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1a08066f90 0x7f1a08069450 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1a140ad000 tx=0x7f1a14008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:58.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a2f2b5000 1 Processor -- start 2024-04-17T10:48:58.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a2f2b5000 1 -- start start 2024-04-17T10:48:58.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a2f2b5000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b332d79d0 0x557b332d7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a2f2b5000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557b332d82f0 0x557b3328db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a2f2b5000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b3328e0a0 0x557b3328e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a2f2b5000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b330daff0 con 0x557b332d79d0 2024-04-17T10:48:58.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a2f2b5000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b33063150 con 0x557b3328e0a0 2024-04-17T10:48:58.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a2f2b5000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b330d7090 con 0x557b332d82f0 2024-04-17T10:48:58.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a29ee4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b332d79d0 0x557b332d7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a29ee4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b332d79d0 0x557b332d7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40586/0 (socket says 172.21.15.83:40586) 2024-04-17T10:48:58.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a29ee4640 1 -- 172.21.15.83:0/1136073996 learned_addr learned my addr 172.21.15.83:0/1136073996 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:58.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a29ee4640 1 -- 172.21.15.83:0/1136073996 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557b332d82f0 msgr2=0x557b3328db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:58.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a29ee4640 1 --2- 172.21.15.83:0/1136073996 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557b332d82f0 0x557b3328db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a29ee4640 1 -- 172.21.15.83:0/1136073996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b3328e0a0 msgr2=0x557b3328e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a29ee4640 1 --2- 172.21.15.83:0/1136073996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b3328e0a0 0x557b3328e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a29ee4640 1 -- 172.21.15.83:0/1136073996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b332710d0 con 0x557b332d79d0 2024-04-17T10:48:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a29ee4640 1 --2- 172.21.15.83:0/1136073996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b332d79d0 0x557b332d7db0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f1a2005e1e0 tx=0x7f1a20056840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be6f14f9e3ec3ef server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a077fe640 1 -- 172.21.15.83:0/1136073996 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a20082070 con 0x557b332d79d0 2024-04-17T10:48:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a077fe640 1 -- 172.21.15.83:0/1136073996 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2005e710 con 0x557b332d79d0 2024-04-17T10:48:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a077fe640 1 -- 172.21.15.83:0/1136073996 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2006d070 con 0x557b332d79d0 2024-04-17T10:48:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a2f2b5000 1 -- 172.21.15.83:0/1136073996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b332d79d0 msgr2=0x557b332d7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a2f2b5000 1 --2- 172.21.15.83:0/1136073996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b332d79d0 0x557b332d7db0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f1a2005e1e0 tx=0x7f1a20056840 comp rx=0 tx=0).stop 2024-04-17T10:48:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a2f2b5000 1 -- 172.21.15.83:0/1136073996 shutdown_connections 2024-04-17T10:48:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a2f2b5000 1 --2- 172.21.15.83:0/1136073996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b3328e0a0 0x557b3328e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a2f2b5000 1 --2- 172.21.15.83:0/1136073996 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557b332d82f0 0x557b3328db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a2f2b5000 1 --2- 172.21.15.83:0/1136073996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b332d79d0 0x557b332d7db0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a2f2b5000 1 -- 172.21.15.83:0/1136073996 >> 172.21.15.83:0/1136073996 conn(0x557b3328a020 msgr2=0x557b3328c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a2f2b5000 1 -- 172.21.15.83:0/1136073996 shutdown_connections 2024-04-17T10:48:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a2f2b5000 1 -- 172.21.15.83:0/1136073996 wait complete. 2024-04-17T10:48:58.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a2f2b5000 1 -- start start 2024-04-17T10:48:58.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.417+0000 7f1a2f2b5000 1 Processor -- start 2024-04-17T10:48:58.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2f2b5000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b3328e0a0 0x557b332bc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2f2b5000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557b332d79d0 0x557b332bcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2f2b5000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b332d82f0 0x557b332bd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2f2b5000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b3309f250 con 0x557b332d82f0 2024-04-17T10:48:58.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2f2b5000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b33095690 con 0x557b3328e0a0 2024-04-17T10:48:58.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2f2b5000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b33063a40 con 0x557b332d79d0 2024-04-17T10:48:58.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2f2b5000 10 client.0 _ll_register_callbacks cb 0x557b33278e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:48:58.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2a6e5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b332d82f0 0x557b332bd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2a6e5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b332d82f0 0x557b332bd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40596/0 (socket says 172.21.15.83:40596) 2024-04-17T10:48:58.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2a6e5640 1 -- 172.21.15.83:0/2210562906 learned_addr learned my addr 172.21.15.83:0/2210562906 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:58.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a29ee4640 1 --2- 172.21.15.83:0/2210562906 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b3328e0a0 0x557b332bc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a296e3640 1 --2- 172.21.15.83:0/2210562906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557b332d79d0 0x557b332bcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2a6e5640 1 -- 172.21.15.83:0/2210562906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557b332d79d0 msgr2=0x557b332bcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:58.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2a6e5640 1 --2- 172.21.15.83:0/2210562906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557b332d79d0 0x557b332bcd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2a6e5640 1 -- 172.21.15.83:0/2210562906 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b3328e0a0 msgr2=0x557b332bc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:58.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2a6e5640 1 --2- 172.21.15.83:0/2210562906 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b3328e0a0 0x557b332bc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2a6e5640 1 -- 172.21.15.83:0/2210562906 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557b331881c0 con 0x557b332d82f0 2024-04-17T10:48:58.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2a6e5640 1 --2- 172.21.15.83:0/2210562906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b332d82f0 0x557b332bd270 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00d7a0 tx=0x7f1a1c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f03b6eb9dadaf852 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:58.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:48:58.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a04ff9640 1 -- 172.21.15.83:0/2210562906 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c00b8b0 con 0x557b332d82f0 2024-04-17T10:48:58.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a04ff9640 1 -- 172.21.15.83:0/2210562906 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1a1c00ba50 con 0x557b332d82f0 2024-04-17T10:48:58.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2f2b5000 1 -- 172.21.15.83:0/2210562906 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a2005e710 con 0x557b332d82f0 2024-04-17T10:48:58.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a04ff9640 1 -- 172.21.15.83:0/2210562906 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a1c01f050 con 0x557b332d82f0 2024-04-17T10:48:58.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2f2b5000 10 client.8952 fetch_fsmap learned FSMap version 69 2024-04-17T10:48:58.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2f2b5000 1 -- 172.21.15.83:0/2210562906 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x557b33184170 con 0x557b332d82f0 2024-04-17T10:48:58.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a04ff9640 1 -- 172.21.15.83:0/2210562906 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2005e710 con 0x557b332d82f0 2024-04-17T10:48:58.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a04ff9640 1 -- 172.21.15.83:0/2210562906 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1a1c022020 con 0x557b332d82f0 2024-04-17T10:48:58.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2f2b5000 10 client.8952 fetch_fsmap finished waiting for FSMap version 69 2024-04-17T10:48:58.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2f2b5000 10 client.8952 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-17T10:48:58.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2f2b5000 20 client.8952 populate_metadata read hostname 'smithi083' 2024-04-17T10:48:58.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2f2b5000 1 --2- 172.21.15.83:0/2210562906 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x557b332e7c80 0x557b332ea070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2f2b5000 4 client.8952 mds_command: new command op to 8604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:48:58.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2f2b5000 1 -- 172.21.15.83:0/2210562906 --> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b33076c60 con 0x557b332e7c80 2024-04-17T10:48:58.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a29ee4640 1 --2- 172.21.15.83:0/2210562906 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x557b332e7c80 0x557b332ea070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a04ff9640 10 client.8952 ms_handle_connect on v2:172.21.15.83:6836/1479417998 2024-04-17T10:48:58.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a29ee4640 1 --2- 172.21.15.83:0/2210562906 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x557b332e7c80 0x557b332ea070 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b81efbad13e160d server_cookie=42f3f87732ac02ac in_seq=0 out_seq=0 2024-04-17T10:48:58.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a04ff9640 1 -- 172.21.15.83:0/2210562906 <== mds.0 v2:172.21.15.83:6836/1479417998 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557b33076c60 con 0x557b332e7c80 2024-04-17T10:48:58.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a04ff9640 10 client.8952 handle_command_reply: tid=0 2024-04-17T10:48:58.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2f2b5000 1 client.8952 shutdown 2024-04-17T10:48:58.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2f2b5000 1 -- 172.21.15.83:0/2210562906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b332d82f0 msgr2=0x557b332bd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:58.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2f2b5000 1 --2- 172.21.15.83:0/2210562906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b332d82f0 0x557b332bd270 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00d7a0 tx=0x7f1a1c00dc60 comp rx=0 tx=0).stop 2024-04-17T10:48:58.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2f2b5000 1 -- 172.21.15.83:0/2210562906 shutdown_connections 2024-04-17T10:48:58.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2f2b5000 1 --2- 172.21.15.83:0/2210562906 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x557b332e7c80 0x557b332ea070 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2f2b5000 1 --2- 172.21.15.83:0/2210562906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b332d82f0 0x557b332bd270 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2f2b5000 1 --2- 172.21.15.83:0/2210562906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557b332d79d0 0x557b332bcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a04ff9640 0 client.8952 ms_handle_reset on v2:172.21.15.83:6836/1479417998 2024-04-17T10:48:58.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2f2b5000 1 --2- 172.21.15.83:0/2210562906 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b3328e0a0 0x557b332bc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2f2b5000 1 -- 172.21.15.83:0/2210562906 >> 172.21.15.83:0/2210562906 conn(0x557b3328a020 msgr2=0x557b3328cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:58.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2f2b5000 1 -- 172.21.15.83:0/2210562906 shutdown_connections 2024-04-17T10:48:58.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2f2b5000 1 -- 172.21.15.83:0/2210562906 wait complete. 2024-04-17T10:48:58.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.421+0000 7f1a2f2b5000 20 client.8952 trim_cache size 0 max 16384 2024-04-17T10:48:58.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.437+0000 7f1a2f2b5000 1 Processor -- start 2024-04-17T10:48:58.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.437+0000 7f1a2f2b5000 1 -- start start 2024-04-17T10:48:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.437+0000 7f1a2f2b5000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b3328e0a0 0x557b3328da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.437+0000 7f1a2f2b5000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557b332d79d0 0x557b332e4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.437+0000 7f1a2f2b5000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b332d82f0 0x557b332e5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.437+0000 7f1a2f2b5000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a242030b0 con 0x557b332d82f0 2024-04-17T10:48:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.437+0000 7f1a2f2b5000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a242022b0 con 0x557b3328e0a0 2024-04-17T10:48:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.437+0000 7f1a2f2b5000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a241680e0 con 0x557b332d79d0 2024-04-17T10:48:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.437+0000 7f1a2a6e5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b332d82f0 0x557b332e5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.437+0000 7f1a2a6e5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b332d82f0 0x557b332e5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40600/0 (socket says 172.21.15.83:40600) 2024-04-17T10:48:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.437+0000 7f1a2a6e5640 1 -- 172.21.15.83:0/3336551760 learned_addr learned my addr 172.21.15.83:0/3336551760 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.437+0000 7f1a2a6e5640 1 -- 172.21.15.83:0/3336551760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557b332d79d0 msgr2=0x557b332e4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.437+0000 7f1a2a6e5640 1 --2- 172.21.15.83:0/3336551760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557b332d79d0 0x557b332e4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.437+0000 7f1a2a6e5640 1 -- 172.21.15.83:0/3336551760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b3328e0a0 msgr2=0x557b3328da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.437+0000 7f1a2a6e5640 1 --2- 172.21.15.83:0/3336551760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b3328e0a0 0x557b3328da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.437+0000 7f1a2a6e5640 1 -- 172.21.15.83:0/3336551760 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b33121de0 con 0x557b332d82f0 2024-04-17T10:48:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.437+0000 7f1a2a6e5640 1 --2- 172.21.15.83:0/3336551760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b332d82f0 0x557b332e5380 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c001230 tx=0x7f1a1c007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d4763b8feaeb6f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.437+0000 7f1a06ffd640 1 -- 172.21.15.83:0/3336551760 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c00b8b0 con 0x557b332d82f0 2024-04-17T10:48:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.437+0000 7f1a06ffd640 1 -- 172.21.15.83:0/3336551760 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1c017070 con 0x557b332d82f0 2024-04-17T10:48:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.437+0000 7f1a06ffd640 1 -- 172.21.15.83:0/3336551760 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c012630 con 0x557b332d82f0 2024-04-17T10:48:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.437+0000 7f1a2f2b5000 1 -- 172.21.15.83:0/3336551760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b332d82f0 msgr2=0x557b332e5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.437+0000 7f1a2f2b5000 1 --2- 172.21.15.83:0/3336551760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b332d82f0 0x557b332e5380 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c001230 tx=0x7f1a1c007000 comp rx=0 tx=0).stop 2024-04-17T10:48:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.437+0000 7f1a2f2b5000 1 -- 172.21.15.83:0/3336551760 shutdown_connections 2024-04-17T10:48:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.437+0000 7f1a2f2b5000 1 --2- 172.21.15.83:0/3336551760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b332d82f0 0x557b332e5380 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.437+0000 7f1a2f2b5000 1 --2- 172.21.15.83:0/3336551760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557b332d79d0 0x557b332e4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.437+0000 7f1a2f2b5000 1 --2- 172.21.15.83:0/3336551760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b3328e0a0 0x557b3328da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.437+0000 7f1a2f2b5000 1 -- 172.21.15.83:0/3336551760 >> 172.21.15.83:0/3336551760 conn(0x557b3328a020 msgr2=0x557b332e4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:58.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.437+0000 7f1a2f2b5000 1 -- 172.21.15.83:0/3336551760 shutdown_connections 2024-04-17T10:48:58.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.437+0000 7f1a2f2b5000 1 -- 172.21.15.83:0/3336551760 wait complete. 2024-04-17T10:48:58.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.437+0000 7f1a2f2b5000 1 -- start start 2024-04-17T10:48:58.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2f2b5000 1 Processor -- start 2024-04-17T10:48:58.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2f2b5000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b3328e0a0 0x557b33337720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2f2b5000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b332d79d0 0x557b33337c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2f2b5000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557b332d82f0 0x557b332e9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2f2b5000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b33075e00 con 0x557b332d79d0 2024-04-17T10:48:58.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2f2b5000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a241686e0 con 0x557b3328e0a0 2024-04-17T10:48:58.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2f2b5000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b330da300 con 0x557b332d82f0 2024-04-17T10:48:58.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2f2b5000 10 client.0 _ll_register_callbacks cb 0x557b33259040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:48:58.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a296e3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b332d79d0 0x557b33337c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a296e3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b332d79d0 0x557b33337c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40614/0 (socket says 172.21.15.83:40614) 2024-04-17T10:48:58.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a296e3640 1 -- 172.21.15.83:0/3367128332 learned_addr learned my addr 172.21.15.83:0/3367128332 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2a6e5640 1 --2- 172.21.15.83:0/3367128332 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557b332d82f0 0x557b332e9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a29ee4640 1 --2- 172.21.15.83:0/3367128332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b3328e0a0 0x557b33337720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a296e3640 1 -- 172.21.15.83:0/3367128332 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557b332d82f0 msgr2=0x557b332e9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a296e3640 1 --2- 172.21.15.83:0/3367128332 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557b332d82f0 0x557b332e9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a296e3640 1 -- 172.21.15.83:0/3367128332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b3328e0a0 msgr2=0x557b33337720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a296e3640 1 --2- 172.21.15.83:0/3367128332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b3328e0a0 0x557b33337720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a296e3640 1 -- 172.21.15.83:0/3367128332 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557b3307c8e0 con 0x557b332d79d0 2024-04-17T10:48:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a296e3640 1 --2- 172.21.15.83:0/3367128332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b332d79d0 0x557b33337c60 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f1a140ae040 tx=0x7f1a14019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69344901aa7a9317 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:48:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a04ff9640 1 -- 172.21.15.83:0/3367128332 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a140ab600 con 0x557b332d79d0 2024-04-17T10:48:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a04ff9640 1 -- 172.21.15.83:0/3367128332 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1a140ab7a0 con 0x557b332d79d0 2024-04-17T10:48:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2f2b5000 1 -- 172.21.15.83:0/3367128332 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a1c017070 con 0x557b332d79d0 2024-04-17T10:48:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a04ff9640 1 -- 172.21.15.83:0/3367128332 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a140ce050 con 0x557b332d79d0 2024-04-17T10:48:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2f2b5000 10 client.8958 fetch_fsmap learned FSMap version 69 2024-04-17T10:48:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2f2b5000 1 -- 172.21.15.83:0/3367128332 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x557b331348f0 con 0x557b332d79d0 2024-04-17T10:48:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a04ff9640 1 -- 172.21.15.83:0/3367128332 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1c017070 con 0x557b332d79d0 2024-04-17T10:48:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a04ff9640 1 -- 172.21.15.83:0/3367128332 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1a140d1020 con 0x557b332d79d0 2024-04-17T10:48:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2f2b5000 10 client.8958 fetch_fsmap finished waiting for FSMap version 69 2024-04-17T10:48:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2f2b5000 10 client.8958 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-17T10:48:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2f2b5000 20 client.8958 populate_metadata read hostname 'smithi083' 2024-04-17T10:48:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2f2b5000 1 --2- 172.21.15.83:0/3367128332 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x557b332ea7f0 0x557b332eac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2f2b5000 4 client.8958 mds_command: new command op to 8604 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T10:48:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2f2b5000 1 -- 172.21.15.83:0/3367128332 --> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557b330e8530 con 0x557b332ea7f0 2024-04-17T10:48:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a29ee4640 1 --2- 172.21.15.83:0/3367128332 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x557b332ea7f0 0x557b332eac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a04ff9640 10 client.8958 ms_handle_connect on v2:172.21.15.83:6836/1479417998 2024-04-17T10:48:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a29ee4640 1 --2- 172.21.15.83:0/3367128332 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x557b332ea7f0 0x557b332eac00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=987384002df2d191 server_cookie=9cd4f81dd0897f16 in_seq=0 out_seq=0 2024-04-17T10:48:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a04ff9640 1 -- 172.21.15.83:0/3367128332 <== mds.0 v2:172.21.15.83:6836/1479417998 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557b330e8530 con 0x557b332ea7f0 2024-04-17T10:48:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a04ff9640 10 client.8958 handle_command_reply: tid=0 2024-04-17T10:48:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2f2b5000 1 client.8958 shutdown 2024-04-17T10:48:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2f2b5000 1 -- 172.21.15.83:0/3367128332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b332d79d0 msgr2=0x557b33337c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2f2b5000 1 --2- 172.21.15.83:0/3367128332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b332d79d0 0x557b33337c60 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f1a140ae040 tx=0x7f1a14019240 comp rx=0 tx=0).stop 2024-04-17T10:48:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2f2b5000 1 -- 172.21.15.83:0/3367128332 shutdown_connections 2024-04-17T10:48:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2f2b5000 1 --2- 172.21.15.83:0/3367128332 >> [v2:172.21.15.83:6836/1479417998,v1:172.21.15.83:6837/1479417998] conn(0x557b332ea7f0 0x557b332eac00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2f2b5000 1 --2- 172.21.15.83:0/3367128332 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557b332d82f0 0x557b332e9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2f2b5000 1 --2- 172.21.15.83:0/3367128332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b332d79d0 0x557b33337c60 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2f2b5000 1 --2- 172.21.15.83:0/3367128332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b3328e0a0 0x557b33337720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2f2b5000 1 -- 172.21.15.83:0/3367128332 >> 172.21.15.83:0/3367128332 conn(0x557b3328a020 msgr2=0x557b332e4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a04ff9640 0 client.8958 ms_handle_reset on v2:172.21.15.83:6836/1479417998 2024-04-17T10:48:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2f2b5000 1 -- 172.21.15.83:0/3367128332 shutdown_connections 2024-04-17T10:48:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2f2b5000 1 -- 172.21.15.83:0/3367128332 wait complete. 2024-04-17T10:48:58.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2f2b5000 20 client.8958 trim_cache size 0 max 16384 2024-04-17T10:48:58.446 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:48:58.446 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-17T10:48:58.446 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-17T10:48:58.446 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:48:58.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2c148640 1 -- 172.21.15.83:0/2430517115 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1a08066f90 msgr2=0x7f1a08069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:58.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2c148640 1 --2- 172.21.15.83:0/2430517115 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1a08066f90 0x7f1a08069450 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1a140ad000 tx=0x7f1a14008040 comp rx=0 tx=0).stop 2024-04-17T10:48:58.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2c148640 1 -- 172.21.15.83:0/2430517115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a2406b350 msgr2=0x7f1a2414c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:58.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2c148640 1 --2- 172.21.15.83:0/2430517115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a2406b350 0x7f1a2414c430 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000d830 tx=0x7f1a2000dcf0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2c148640 1 -- 172.21.15.83:0/2430517115 shutdown_connections 2024-04-17T10:48:58.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2c148640 1 --2- 172.21.15.83:0/2430517115 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1a08066f90 0x7f1a08069450 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2c148640 1 --2- 172.21.15.83:0/2430517115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a24154de0 0x7f1a2415cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2c148640 1 --2- 172.21.15.83:0/2430517115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a240b4040 0x7f1a24153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2c148640 1 --2- 172.21.15.83:0/2430517115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a2406b350 0x7f1a2414c430 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2c148640 1 -- 172.21.15.83:0/2430517115 >> 172.21.15.83:0/2430517115 conn(0x7f1a240b15b0 msgr2=0x7f1a2406d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:58.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2c148640 1 -- 172.21.15.83:0/2430517115 shutdown_connections 2024-04-17T10:48:58.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.441+0000 7f1a2c148640 1 -- 172.21.15.83:0/2430517115 wait complete. 2024-04-17T10:48:58.454 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T10:48:58.455 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:48:58.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.565+0000 7fd6ef1cb640 1 Processor -- start 2024-04-17T10:48:58.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.565+0000 7fd6ef1cb640 1 -- start start 2024-04-17T10:48:58.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6ef1cb640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd6e8146b40 0x7fd6e806d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6ef1cb640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6e8146f20 0x7fd6e80720d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6ef1cb640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e8161ae0 con 0x7fd6e806abe0 2024-04-17T10:48:58.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6ef1cb640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6e8161dc0 con 0x7fd6e8146b40 2024-04-17T10:48:58.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6ef1cb640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e81620a0 con 0x7fd6e8146f20 2024-04-17T10:48:58.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6ed768640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd6e806abe0 0x7fd6e80d07d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49348/0 (socket says 172.21.15.83:49348) 2024-04-17T10:48:58.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6ed768640 1 -- 172.21.15.83:0/1064392110 learned_addr learned my addr 172.21.15.83:0/1064392110 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:58.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6ecf67640 1 --2- 172.21.15.83:0/1064392110 >> v2:172.21.15.167:3300/0 conn(0x7fd6e8146b40 0x7fd6e806d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6e77fe640 1 -- 172.21.15.83:0/1064392110 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1798043447 0 0) 0x7fd6e8161ae0 con 0x7fd6e806abe0 2024-04-17T10:48:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6e77fe640 1 -- 172.21.15.83:0/1064392110 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6d8003530 con 0x7fd6e806abe0 2024-04-17T10:48:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6e7fff640 1 --2- 172.21.15.83:0/1064392110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6e8146f20 0x7fd6e80720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6e77fe640 1 -- 172.21.15.83:0/1064392110 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2071392510 0 0) 0x7fd6d8003530 con 0x7fd6e806abe0 2024-04-17T10:48:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6e77fe640 1 -- 172.21.15.83:0/1064392110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6e8146f20 msgr2=0x7fd6e80720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6e77fe640 1 --2- 172.21.15.83:0/1064392110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6e8146f20 0x7fd6e80720d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6e77fe640 1 -- 172.21.15.83:0/1064392110 >> v2:172.21.15.167:3300/0 conn(0x7fd6e8146b40 msgr2=0x7fd6e806d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6e77fe640 1 --2- 172.21.15.83:0/1064392110 >> v2:172.21.15.167:3300/0 conn(0x7fd6e8146b40 0x7fd6e806d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6e77fe640 1 -- 172.21.15.83:0/1064392110 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e8162380 con 0x7fd6e806abe0 2024-04-17T10:48:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6e77fe640 1 -- 172.21.15.83:0/1064392110 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fd6e0001a00 con 0x7fd6e806abe0 2024-04-17T10:48:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6e77fe640 1 -- 172.21.15.83:0/1064392110 >> v1:172.21.15.83:6789/0 conn(0x7fd6e806abe0 legacy=0x7fd6e80d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6e77fe640 1 --2- 172.21.15.83:0/1064392110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6d80039c0 0x7fd6d8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6e77fe640 1 --2- 172.21.15.83:0/1064392110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6e8146f20 0x7fd6d8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6e77fe640 1 --2- 172.21.15.83:0/1064392110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6d8004d50 0x7fd6d80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6e77fe640 1 -- 172.21.15.83:0/1064392110 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e8161dc0 con 0x7fd6d8004d50 2024-04-17T10:48:58.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6e77fe640 1 -- 172.21.15.83:0/1064392110 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e81620a0 con 0x7fd6d80039c0 2024-04-17T10:48:58.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6e77fe640 1 -- 172.21.15.83:0/1064392110 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d800ba00 con 0x7fd6e8146f20 2024-04-17T10:48:58.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6ed768640 1 --2- 172.21.15.83:0/1064392110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6d8004d50 0x7fd6d80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6ecf67640 1 --2- 172.21.15.83:0/1064392110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6d80039c0 0x7fd6d8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6e7fff640 1 --2- 172.21.15.83:0/1064392110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6e8146f20 0x7fd6d8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6ed768640 1 -- 172.21.15.83:0/1064392110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6e8146f20 msgr2=0x7fd6d8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:58.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6ed768640 1 --2- 172.21.15.83:0/1064392110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6e8146f20 0x7fd6d8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6ed768640 1 -- 172.21.15.83:0/1064392110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6d80039c0 msgr2=0x7fd6d8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:58.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6ed768640 1 --2- 172.21.15.83:0/1064392110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6d80039c0 0x7fd6d8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.569+0000 7fd6ed768640 1 -- 172.21.15.83:0/1064392110 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d8003530 con 0x7fd6d8004d50 2024-04-17T10:48:58.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.573+0000 7fd6ed768640 1 --2- 172.21.15.83:0/1064392110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6d8004d50 0x7fd6d80051d0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fd6e000a8a0 tx=0x7fd6e001d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8b1fcc9b8039f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:58.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.573+0000 7fd6e77fe640 1 -- 172.21.15.83:0/1064392110 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e000a4c0 con 0x7fd6d8004d50 2024-04-17T10:48:58.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.573+0000 7fd6e77fe640 1 -- 172.21.15.83:0/1064392110 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e00a9020 con 0x7fd6d8004d50 2024-04-17T10:48:58.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.573+0000 7fd6e77fe640 1 -- 172.21.15.83:0/1064392110 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e00072f0 con 0x7fd6d8004d50 2024-04-17T10:48:58.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.573+0000 7fd6ef1cb640 1 -- 172.21.15.83:0/1064392110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6d8004d50 msgr2=0x7fd6d80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:58.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.573+0000 7fd6ef1cb640 1 --2- 172.21.15.83:0/1064392110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6d8004d50 0x7fd6d80051d0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fd6e000a8a0 tx=0x7fd6e001d3c0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.573+0000 7fd6ed768640 1 -- 172.21.15.83:0/1064392110 reap_dead start 2024-04-17T10:48:58.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.573+0000 7fd6ef1cb640 1 -- 172.21.15.83:0/1064392110 shutdown_connections 2024-04-17T10:48:58.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.573+0000 7fd6ef1cb640 1 -- 172.21.15.83:0/1064392110 >> 172.21.15.83:0/1064392110 conn(0x7fd6e806bcf0 msgr2=0x7fd6e806c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:58.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.573+0000 7fd6ef1cb640 1 -- 172.21.15.83:0/1064392110 shutdown_connections 2024-04-17T10:48:58.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.573+0000 7fd6ef1cb640 1 -- 172.21.15.83:0/1064392110 wait complete. 2024-04-17T10:48:58.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.577+0000 7fd6ef1cb640 1 Processor -- start 2024-04-17T10:48:58.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.577+0000 7fd6ef1cb640 1 -- start start 2024-04-17T10:48:58.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.577+0000 7fd6ef1cb640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6e813f340 0x7fd6e80b1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.577+0000 7fd6ef1cb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6e80b19d0 0x7fd6e814f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.577+0000 7fd6ef1cb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6e814f860 0x7fd6e80ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.577+0000 7fd6ef1cb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e806b1b0 con 0x7fd6e814f860 2024-04-17T10:48:58.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.577+0000 7fd6ef1cb640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e806b030 con 0x7fd6e80b19d0 2024-04-17T10:48:58.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.577+0000 7fd6ef1cb640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e806b330 con 0x7fd6e813f340 2024-04-17T10:48:58.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.577+0000 7fd6ed768640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6e814f860 0x7fd6e80ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.577+0000 7fd6ed768640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6e814f860 0x7fd6e80ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40644/0 (socket says 172.21.15.83:40644) 2024-04-17T10:48:58.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.577+0000 7fd6ed768640 1 -- 172.21.15.83:0/1374232210 learned_addr learned my addr 172.21.15.83:0/1374232210 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:58.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.577+0000 7fd6e7fff640 1 --2- 172.21.15.83:0/1374232210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6e80b19d0 0x7fd6e814f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.577+0000 7fd6ecf67640 1 --2- 172.21.15.83:0/1374232210 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6e813f340 0x7fd6e80b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.577+0000 7fd6ed768640 1 -- 172.21.15.83:0/1374232210 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6e813f340 msgr2=0x7fd6e80b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:58.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.577+0000 7fd6ed768640 1 --2- 172.21.15.83:0/1374232210 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6e813f340 0x7fd6e80b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.577+0000 7fd6ed768640 1 -- 172.21.15.83:0/1374232210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6e80b19d0 msgr2=0x7fd6e814f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:58.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.577+0000 7fd6ed768640 1 --2- 172.21.15.83:0/1374232210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6e80b19d0 0x7fd6e814f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.577+0000 7fd6ed768640 1 -- 172.21.15.83:0/1374232210 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e815fff0 con 0x7fd6e814f860 2024-04-17T10:48:58.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.577+0000 7fd6ed768640 1 --2- 172.21.15.83:0/1374232210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6e814f860 0x7fd6e80ad840 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0098de0 tx=0x7fd6d8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:58.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.577+0000 7fd6e57fa640 1 -- 172.21.15.83:0/1374232210 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0002020 con 0x7fd6e814f860 2024-04-17T10:48:58.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.577+0000 7fd6e57fa640 1 -- 172.21.15.83:0/1374232210 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e0001210 con 0x7fd6e814f860 2024-04-17T10:48:58.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.577+0000 7fd6ef1cb640 1 -- 172.21.15.83:0/1374232210 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e80b1e30 con 0x7fd6e814f860 2024-04-17T10:48:58.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.577+0000 7fd6e57fa640 1 -- 172.21.15.83:0/1374232210 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e00077b0 con 0x7fd6e814f860 2024-04-17T10:48:58.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.577+0000 7fd6ef1cb640 1 -- 172.21.15.83:0/1374232210 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e814d470 con 0x7fd6e814f860 2024-04-17T10:48:58.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.577+0000 7fd6ef1cb640 1 -- 172.21.15.83:0/1374232210 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e8074f90 con 0x7fd6e814f860 2024-04-17T10:48:58.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.577+0000 7fd6e57fa640 1 -- 172.21.15.83:0/1374232210 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd6e00be060 con 0x7fd6e814f860 2024-04-17T10:48:58.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.581+0000 7fd6e57fa640 1 --2- 172.21.15.83:0/1374232210 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd6cc066f60 0x7fd6cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.581+0000 7fd6e57fa640 1 -- 172.21.15.83:0/1374232210 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd6e011ecc0 con 0x7fd6e814f860 2024-04-17T10:48:58.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.581+0000 7fd6ecf67640 1 --2- 172.21.15.83:0/1374232210 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd6cc066f60 0x7fd6cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.581+0000 7fd6ecf67640 1 --2- 172.21.15.83:0/1374232210 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd6cc066f60 0x7fd6cc069420 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009ae0 tx=0x7fd6d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:58.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.585+0000 7fd6e57fa640 1 -- 172.21.15.83:0/1374232210 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd6e00170d0 con 0x7fd6e814f860 2024-04-17T10:48:58.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.761+0000 7fd6ef1cb640 1 -- 172.21.15.83:0/1374232210 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6e8071bf0 con 0x7fd6e814f860 2024-04-17T10:48:58.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.761+0000 7fd6e57fa640 1 -- 172.21.15.83:0/1374232210 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fd6e00ef530 con 0x7fd6e814f860 2024-04-17T10:48:58.757 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:58.758 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5786,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1553533838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1553533838},{"type":"v1","addr":"172.21.15.167:6833","nonce":1553533838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/674536913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":674536913},{"type":"v1","addr":"172.21.15.167:6835","nonce":674536913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8598,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4183890457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4183890457},{"type":"v1","addr":"172.21.15.83:6835","nonce":4183890457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:48:24.738443+0000","modified":"2024-04-17T10:48:26.776273+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8604":{"gid":8604,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1479417998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1479417998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1479417998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-17T10:48:58.758 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 69 2024-04-17T10:48:58.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.761+0000 7fd6ef1cb640 1 -- 172.21.15.83:0/1374232210 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd6cc066f60 msgr2=0x7fd6cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:58.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.761+0000 7fd6ef1cb640 1 --2- 172.21.15.83:0/1374232210 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd6cc066f60 0x7fd6cc069420 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009ae0 tx=0x7fd6d4008040 comp rx=0 tx=0).stop 2024-04-17T10:48:58.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.761+0000 7fd6ef1cb640 1 -- 172.21.15.83:0/1374232210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6e814f860 msgr2=0x7fd6e80ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:58.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.761+0000 7fd6ef1cb640 1 --2- 172.21.15.83:0/1374232210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6e814f860 0x7fd6e80ad840 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0098de0 tx=0x7fd6d8005840 comp rx=0 tx=0).stop 2024-04-17T10:48:58.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.761+0000 7fd6ef1cb640 1 -- 172.21.15.83:0/1374232210 shutdown_connections 2024-04-17T10:48:58.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.761+0000 7fd6ef1cb640 1 --2- 172.21.15.83:0/1374232210 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd6cc066f60 0x7fd6cc069420 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.761+0000 7fd6ef1cb640 1 --2- 172.21.15.83:0/1374232210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6e814f860 0x7fd6e80ad840 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.761+0000 7fd6ef1cb640 1 --2- 172.21.15.83:0/1374232210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6e80b19d0 0x7fd6e814f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.761+0000 7fd6ef1cb640 1 --2- 172.21.15.83:0/1374232210 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6e813f340 0x7fd6e80b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.761+0000 7fd6ef1cb640 1 -- 172.21.15.83:0/1374232210 >> 172.21.15.83:0/1374232210 conn(0x7fd6e806bcf0 msgr2=0x7fd6e80b51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:58.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.761+0000 7fd6ef1cb640 1 -- 172.21.15.83:0/1374232210 shutdown_connections 2024-04-17T10:48:58.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.761+0000 7fd6ef1cb640 1 -- 172.21.15.83:0/1374232210 wait complete. 2024-04-17T10:48:58.774 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-17T10:48:58.775 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-17T10:48:58.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.893+0000 7fcb30ba9640 1 Processor -- start 2024-04-17T10:48:58.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.893+0000 7fcb30ba9640 1 -- start start 2024-04-17T10:48:58.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.893+0000 7fcb30ba9640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcb2c146a00 0x7fcb2c146e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.893+0000 7fcb30ba9640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb2c06d690 0x7fcb2c1620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.893+0000 7fcb30ba9640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb2c06daf0 con 0x7fcb2c0b2670 2024-04-17T10:48:58.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.893+0000 7fcb30ba9640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb2c162610 con 0x7fcb2c146a00 2024-04-17T10:48:58.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.893+0000 7fcb30ba9640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c06c960 con 0x7fcb2c06d690 2024-04-17T10:48:58.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.893+0000 7fcb2a59c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcb2c146a00 0x7fcb2c146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.893+0000 7fcb29d9b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb2c06d690 0x7fcb2c1620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.893+0000 7fcb2ad9d640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fcb2c0b2670 0x7fcb2c070470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:39458/0 (socket says 172.21.15.83:39458) 2024-04-17T10:48:58.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.893+0000 7fcb2ad9d640 1 -- 172.21.15.83:0/362669217 learned_addr learned my addr 172.21.15.83:0/362669217 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:48:58.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.897+0000 7fcb2a59c640 1 -- 172.21.15.83:0/362669217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb2c06d690 msgr2=0x7fcb2c1620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:48:58.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.897+0000 7fcb2a59c640 1 --2- 172.21.15.83:0/362669217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb2c06d690 0x7fcb2c1620d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.897+0000 7fcb2a59c640 1 -- 172.21.15.83:0/362669217 >> v1:172.21.15.167:6789/0 conn(0x7fcb2c0b2670 legacy=0x7fcb2c070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:58.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.897+0000 7fcb2a59c640 1 -- 172.21.15.83:0/362669217 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb2c147570 con 0x7fcb2c146a00 2024-04-17T10:48:58.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.897+0000 7fcb29d9b640 1 --2- 172.21.15.83:0/362669217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb2c06d690 0x7fcb2c1620d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:48:58.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.897+0000 7fcb2a59c640 1 --2- 172.21.15.83:0/362669217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb2c146a00 0x7fcb2c146e70 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fcb240097c0 tx=0x7fcb2401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac878a9bbb1e619c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:58.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.897+0000 7fcb2959a640 1 -- 172.21.15.83:0/362669217 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb240b1070 con 0x7fcb2c146a00 2024-04-17T10:48:58.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.897+0000 7fcb2959a640 1 -- 172.21.15.83:0/362669217 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb240ac410 con 0x7fcb2c146a00 2024-04-17T10:48:58.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.897+0000 7fcb2959a640 1 -- 172.21.15.83:0/362669217 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb240ac7e0 con 0x7fcb2c146a00 2024-04-17T10:48:58.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.897+0000 7fcb30ba9640 1 -- 172.21.15.83:0/362669217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb2c146a00 msgr2=0x7fcb2c146e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:58.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.897+0000 7fcb30ba9640 1 --2- 172.21.15.83:0/362669217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb2c146a00 0x7fcb2c146e70 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fcb240097c0 tx=0x7fcb2401c4a0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.897+0000 7fcb30ba9640 1 -- 172.21.15.83:0/362669217 shutdown_connections 2024-04-17T10:48:58.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.897+0000 7fcb30ba9640 1 --2- 172.21.15.83:0/362669217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb2c06d690 0x7fcb2c1620d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.897+0000 7fcb30ba9640 1 --2- 172.21.15.83:0/362669217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb2c146a00 0x7fcb2c146e70 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.897+0000 7fcb30ba9640 1 -- 172.21.15.83:0/362669217 >> 172.21.15.83:0/362669217 conn(0x7fcb2c06abe0 msgr2=0x7fcb2c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:58.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.897+0000 7fcb30ba9640 1 -- 172.21.15.83:0/362669217 shutdown_connections 2024-04-17T10:48:58.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.897+0000 7fcb30ba9640 1 -- 172.21.15.83:0/362669217 wait complete. 2024-04-17T10:48:58.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.901+0000 7fcb30ba9640 1 Processor -- start 2024-04-17T10:48:58.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.901+0000 7fcb30ba9640 1 -- start start 2024-04-17T10:48:58.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.901+0000 7fcb30ba9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb2c06d690 0x7fcb2c15fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.901+0000 7fcb30ba9640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb2c0b2670 0x7fcb2c15eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.901+0000 7fcb30ba9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb2c146a00 0x7fcb2c15d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.901+0000 7fcb30ba9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c071110 con 0x7fcb2c146a00 2024-04-17T10:48:58.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.901+0000 7fcb30ba9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c070f90 con 0x7fcb2c06d690 2024-04-17T10:48:58.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.901+0000 7fcb30ba9640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c16b200 con 0x7fcb2c0b2670 2024-04-17T10:48:58.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.901+0000 7fcb2ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb2c146a00 0x7fcb2c15d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.901+0000 7fcb2ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb2c146a00 0x7fcb2c15d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40646/0 (socket says 172.21.15.83:40646) 2024-04-17T10:48:58.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.901+0000 7fcb2ad9d640 1 -- 172.21.15.83:0/3829991589 learned_addr learned my addr 172.21.15.83:0/3829991589 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:48:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.901+0000 7fcb2a59c640 1 --2- 172.21.15.83:0/3829991589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb2c06d690 0x7fcb2c15fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.901+0000 7fcb29d9b640 1 --2- 172.21.15.83:0/3829991589 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb2c0b2670 0x7fcb2c15eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.901+0000 7fcb2ad9d640 1 -- 172.21.15.83:0/3829991589 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb2c0b2670 msgr2=0x7fcb2c15eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.901+0000 7fcb2ad9d640 1 --2- 172.21.15.83:0/3829991589 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb2c0b2670 0x7fcb2c15eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.901+0000 7fcb2ad9d640 1 -- 172.21.15.83:0/3829991589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb2c06d690 msgr2=0x7fcb2c15fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.901+0000 7fcb2ad9d640 1 --2- 172.21.15.83:0/3829991589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb2c06d690 0x7fcb2c15fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.901+0000 7fcb2ad9d640 1 -- 172.21.15.83:0/3829991589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb2c140040 con 0x7fcb2c146a00 2024-04-17T10:48:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.901+0000 7fcb2ad9d640 1 --2- 172.21.15.83:0/3829991589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb2c146a00 0x7fcb2c15d2b0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00c430 tx=0x7fcb1c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.901+0000 7fcb22ffd640 1 -- 172.21.15.83:0/3829991589 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c017070 con 0x7fcb2c146a00 2024-04-17T10:48:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.901+0000 7fcb22ffd640 1 -- 172.21.15.83:0/3829991589 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb1c005cf0 con 0x7fcb2c146a00 2024-04-17T10:48:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.901+0000 7fcb30ba9640 1 -- 172.21.15.83:0/3829991589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb2c1402e0 con 0x7fcb2c146a00 2024-04-17T10:48:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.901+0000 7fcb22ffd640 1 -- 172.21.15.83:0/3829991589 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c0072f0 con 0x7fcb2c146a00 2024-04-17T10:48:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.901+0000 7fcb30ba9640 1 -- 172.21.15.83:0/3829991589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb2c1602a0 con 0x7fcb2c146a00 2024-04-17T10:48:58.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.901+0000 7fcb30ba9640 1 -- 172.21.15.83:0/3829991589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb2c074f90 con 0x7fcb2c146a00 2024-04-17T10:48:58.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.905+0000 7fcb22ffd640 1 -- 172.21.15.83:0/3829991589 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcb1c01a070 con 0x7fcb2c146a00 2024-04-17T10:48:58.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.905+0000 7fcb22ffd640 1 --2- 172.21.15.83:0/3829991589 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcb0c066f60 0x7fcb0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:48:58.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.905+0000 7fcb2a59c640 1 --2- 172.21.15.83:0/3829991589 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcb0c066f60 0x7fcb0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:48:58.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.905+0000 7fcb22ffd640 1 -- 172.21.15.83:0/3829991589 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fcb1c084400 con 0x7fcb2c146a00 2024-04-17T10:48:58.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.905+0000 7fcb2a59c640 1 --2- 172.21.15.83:0/3829991589 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcb0c066f60 0x7fcb0c069420 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fcb240ae000 tx=0x7fcb24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:48:58.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:58.909+0000 7fcb22ffd640 1 -- 172.21.15.83:0/3829991589 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcb1c0553d0 con 0x7fcb2c146a00 2024-04-17T10:48:59.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:59.109+0000 7fcb30ba9640 1 -- 172.21.15.83:0/3829991589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fcb2c0b0690 con 0x7fcb2c146a00 2024-04-17T10:48:59.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:59.109+0000 7fcb22ffd640 1 -- 172.21.15.83:0/3829991589 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1675 (secure 0 0 0) 0x7fcb1c054cd0 con 0x7fcb2c146a00 2024-04-17T10:48:59.106 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:48:59.107 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":671,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":116,"num_osds":8,"num_up_osds":8,"osd_up_since":1713350279,"num_in_osds":8,"osd_in_since":1713350270,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":95},{"state_name":"peering","count":2}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076818817,"bytes_used":2655641600,"bytes_avail":770438471680,"bytes_total":773094113280,"inactive_pgs_ratio":0.020618556067347527,"write_bytes_sec":37109459,"read_op_per_sec":0,"write_op_per_sec":37},"fsmap":{"epoch":69,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8604}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-04-17T10:48:56.371485+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-17T10:48:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:59.113+0000 7fcb30ba9640 1 -- 172.21.15.83:0/3829991589 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcb0c066f60 msgr2=0x7fcb0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:59.113+0000 7fcb30ba9640 1 --2- 172.21.15.83:0/3829991589 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcb0c066f60 0x7fcb0c069420 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fcb240ae000 tx=0x7fcb24008040 comp rx=0 tx=0).stop 2024-04-17T10:48:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:59.113+0000 7fcb30ba9640 1 -- 172.21.15.83:0/3829991589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb2c146a00 msgr2=0x7fcb2c15d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:48:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:59.113+0000 7fcb30ba9640 1 --2- 172.21.15.83:0/3829991589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb2c146a00 0x7fcb2c15d2b0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00c430 tx=0x7fcb1c00c8f0 comp rx=0 tx=0).stop 2024-04-17T10:48:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:59.113+0000 7fcb30ba9640 1 -- 172.21.15.83:0/3829991589 shutdown_connections 2024-04-17T10:48:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:59.113+0000 7fcb30ba9640 1 --2- 172.21.15.83:0/3829991589 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcb0c066f60 0x7fcb0c069420 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:59.113+0000 7fcb30ba9640 1 --2- 172.21.15.83:0/3829991589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb2c146a00 0x7fcb2c15d2b0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:59.113+0000 7fcb30ba9640 1 --2- 172.21.15.83:0/3829991589 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb2c0b2670 0x7fcb2c15eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:59.113+0000 7fcb30ba9640 1 --2- 172.21.15.83:0/3829991589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb2c06d690 0x7fcb2c15fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:48:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:59.113+0000 7fcb30ba9640 1 -- 172.21.15.83:0/3829991589 >> 172.21.15.83:0/3829991589 conn(0x7fcb2c06abe0 msgr2=0x7fcb2c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:48:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:59.113+0000 7fcb30ba9640 1 -- 172.21.15.83:0/3829991589 shutdown_connections 2024-04-17T10:48:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:48:59.113+0000 7fcb30ba9640 1 -- 172.21.15.83:0/3829991589 wait complete. 2024-04-17T10:48:59.124 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused'} 2024-04-17T10:49:00.125 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:49:00.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.245+0000 7fe64cf4b640 1 Processor -- start 2024-04-17T10:49:00.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.249+0000 7fe64cf4b640 1 -- start start 2024-04-17T10:49:00.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.249+0000 7fe64cf4b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe64815db50 0x7fe648070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:00.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.249+0000 7fe64cf4b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe64806dcb0 0x7fe6480b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:00.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.249+0000 7fe64cf4b640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6480709b0 con 0x7fe6480aedd0 2024-04-17T10:49:00.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.249+0000 7fe64cf4b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe64815e440 con 0x7fe64815db50 2024-04-17T10:49:00.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.249+0000 7fe64cf4b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe64815e720 con 0x7fe64806dcb0 2024-04-17T10:49:00.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.249+0000 7fe64659c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe64815db50 0x7fe648070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:00.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.249+0000 7fe64659c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe64815db50 0x7fe648070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40662/0 (socket says 172.21.15.83:40662) 2024-04-17T10:49:00.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.249+0000 7fe64659c640 1 -- 172.21.15.83:0/3155694856 learned_addr learned my addr 172.21.15.83:0/3155694856 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:00.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.249+0000 7fe646d9d640 1 --2- 172.21.15.83:0/3155694856 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe64806dcb0 0x7fe6480b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:00.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.249+0000 7fe64659c640 1 -- 172.21.15.83:0/3155694856 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe64806dcb0 msgr2=0x7fe6480b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:00.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.249+0000 7fe64659c640 1 --2- 172.21.15.83:0/3155694856 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe64806dcb0 0x7fe6480b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:00.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.249+0000 7fe64659c640 1 -- 172.21.15.83:0/3155694856 >> v1:172.21.15.167:6789/0 conn(0x7fe6480aedd0 legacy=0x7fe64806d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:00.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.249+0000 7fe64659c640 1 -- 172.21.15.83:0/3155694856 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe64815ea00 con 0x7fe64815db50 2024-04-17T10:49:00.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.249+0000 7fe64659c640 1 --2- 172.21.15.83:0/3155694856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe64815db50 0x7fe648070470 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fe640009870 tx=0x7fe64001c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e0345eb79c4ae server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:00.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.249+0000 7fe64559a640 1 -- 172.21.15.83:0/3155694856 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6400b0070 con 0x7fe64815db50 2024-04-17T10:49:00.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe64559a640 1 -- 172.21.15.83:0/3155694856 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6400ab600 con 0x7fe64815db50 2024-04-17T10:49:00.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe64559a640 1 -- 172.21.15.83:0/3155694856 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6400ab9d0 con 0x7fe64815db50 2024-04-17T10:49:00.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe64cf4b640 1 -- 172.21.15.83:0/3155694856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe64815db50 msgr2=0x7fe648070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:00.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe64cf4b640 1 --2- 172.21.15.83:0/3155694856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe64815db50 0x7fe648070470 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fe640009870 tx=0x7fe64001c310 comp rx=0 tx=0).stop 2024-04-17T10:49:00.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe64cf4b640 1 -- 172.21.15.83:0/3155694856 shutdown_connections 2024-04-17T10:49:00.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe64cf4b640 1 --2- 172.21.15.83:0/3155694856 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe64806dcb0 0x7fe6480b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:00.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe64cf4b640 1 --2- 172.21.15.83:0/3155694856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe64815db50 0x7fe648070470 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:00.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe64cf4b640 1 -- 172.21.15.83:0/3155694856 >> 172.21.15.83:0/3155694856 conn(0x7fe64806abe0 msgr2=0x7fe64806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:00.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe64cf4b640 1 -- 172.21.15.83:0/3155694856 shutdown_connections 2024-04-17T10:49:00.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe64cf4b640 1 -- 172.21.15.83:0/3155694856 wait complete. 2024-04-17T10:49:00.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe64cf4b640 1 Processor -- start 2024-04-17T10:49:00.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe64cf4b640 1 -- start start 2024-04-17T10:49:00.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe64cf4b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe64806dcb0 0x7fe64815c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:00.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe64cf4b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6480aedd0 0x7fe6481fa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:00.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe64cf4b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe64815db50 0x7fe648155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:00.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe64cf4b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6480b3ce0 con 0x7fe64806dcb0 2024-04-17T10:49:00.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe64cf4b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6480b3b60 con 0x7fe64815db50 2024-04-17T10:49:00.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe64cf4b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6480b3e60 con 0x7fe6480aedd0 2024-04-17T10:49:00.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe64659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe64806dcb0 0x7fe64815c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:00.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe64659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe64806dcb0 0x7fe64815c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40672/0 (socket says 172.21.15.83:40672) 2024-04-17T10:49:00.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe64659c640 1 -- 172.21.15.83:0/2373893806 learned_addr learned my addr 172.21.15.83:0/2373893806 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:00.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe646d9d640 1 --2- 172.21.15.83:0/2373893806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe64815db50 0x7fe648155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:00.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe645d9b640 1 --2- 172.21.15.83:0/2373893806 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6480aedd0 0x7fe6481fa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:00.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe64659c640 1 -- 172.21.15.83:0/2373893806 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6480aedd0 msgr2=0x7fe6481fa0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:00.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe64659c640 1 --2- 172.21.15.83:0/2373893806 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6480aedd0 0x7fe6481fa0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:00.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe64659c640 1 -- 172.21.15.83:0/2373893806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe64815db50 msgr2=0x7fe648155520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:00.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe64659c640 1 --2- 172.21.15.83:0/2373893806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe64815db50 0x7fe648155520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:00.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe64659c640 1 -- 172.21.15.83:0/2373893806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6481fab90 con 0x7fe64806dcb0 2024-04-17T10:49:00.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe646d9d640 1 --2- 172.21.15.83:0/2373893806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe64815db50 0x7fe648155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:00.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe64659c640 1 --2- 172.21.15.83:0/2373893806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe64806dcb0 0x7fe64815c4e0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fe6400ae040 tx=0x7fe64001ab50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:00.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe63effd640 1 -- 172.21.15.83:0/2373893806 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6400b0040 con 0x7fe64806dcb0 2024-04-17T10:49:00.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe63effd640 1 -- 172.21.15.83:0/2373893806 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6400bc070 con 0x7fe64806dcb0 2024-04-17T10:49:00.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe64cf4b640 1 -- 172.21.15.83:0/2373893806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6481fae70 con 0x7fe64806dcb0 2024-04-17T10:49:00.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe63effd640 1 -- 172.21.15.83:0/2373893806 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6400b6880 con 0x7fe64806dcb0 2024-04-17T10:49:00.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.253+0000 7fe64cf4b640 1 -- 172.21.15.83:0/2373893806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe648159f90 con 0x7fe64806dcb0 2024-04-17T10:49:00.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.257+0000 7fe64cf4b640 1 -- 172.21.15.83:0/2373893806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe648074f90 con 0x7fe64806dcb0 2024-04-17T10:49:00.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.257+0000 7fe63effd640 1 -- 172.21.15.83:0/2373893806 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe6400b6a20 con 0x7fe64806dcb0 2024-04-17T10:49:00.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.257+0000 7fe63effd640 1 --2- 172.21.15.83:0/2373893806 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe628066f60 0x7fe628069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:00.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.257+0000 7fe63effd640 1 -- 172.21.15.83:0/2373893806 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe64011f570 con 0x7fe64806dcb0 2024-04-17T10:49:00.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.257+0000 7fe645d9b640 1 --2- 172.21.15.83:0/2373893806 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe628066f60 0x7fe628069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:00.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.257+0000 7fe645d9b640 1 --2- 172.21.15.83:0/2373893806 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe628066f60 0x7fe628069420 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe64806d050 tx=0x7fe630009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:00.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.261+0000 7fe63effd640 1 -- 172.21.15.83:0/2373893806 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe6480709b0 con 0x7fe64806dcb0 2024-04-17T10:49:00.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.481+0000 7fe64cf4b640 1 -- 172.21.15.83:0/2373893806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6480b2af0 con 0x7fe64806dcb0 2024-04-17T10:49:00.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.481+0000 7fe63effd640 1 -- 172.21.15.83:0/2373893806 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fe6400f05d0 con 0x7fe64806dcb0 2024-04-17T10:49:00.479 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:00.480 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5786,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1553533838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1553533838},{"type":"v1","addr":"172.21.15.167:6833","nonce":1553533838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/674536913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":674536913},{"type":"v1","addr":"172.21.15.167:6835","nonce":674536913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8598,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4183890457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4183890457},{"type":"v1","addr":"172.21.15.83:6835","nonce":4183890457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:48:24.738443+0000","modified":"2024-04-17T10:48:26.776273+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8604":{"gid":8604,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1479417998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1479417998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1479417998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-17T10:49:00.480 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 69 2024-04-17T10:49:00.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.485+0000 7fe64cf4b640 1 -- 172.21.15.83:0/2373893806 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe628066f60 msgr2=0x7fe628069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:00.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.485+0000 7fe64cf4b640 1 --2- 172.21.15.83:0/2373893806 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe628066f60 0x7fe628069420 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe64806d050 tx=0x7fe630009040 comp rx=0 tx=0).stop 2024-04-17T10:49:00.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.485+0000 7fe64cf4b640 1 -- 172.21.15.83:0/2373893806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe64806dcb0 msgr2=0x7fe64815c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:00.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.485+0000 7fe64cf4b640 1 --2- 172.21.15.83:0/2373893806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe64806dcb0 0x7fe64815c4e0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fe6400ae040 tx=0x7fe64001ab50 comp rx=0 tx=0).stop 2024-04-17T10:49:00.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.485+0000 7fe64cf4b640 1 -- 172.21.15.83:0/2373893806 shutdown_connections 2024-04-17T10:49:00.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.485+0000 7fe64cf4b640 1 --2- 172.21.15.83:0/2373893806 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe628066f60 0x7fe628069420 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:00.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.485+0000 7fe64cf4b640 1 --2- 172.21.15.83:0/2373893806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe64815db50 0x7fe648155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:00.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.485+0000 7fe64cf4b640 1 --2- 172.21.15.83:0/2373893806 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6480aedd0 0x7fe6481fa0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:00.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.485+0000 7fe64cf4b640 1 --2- 172.21.15.83:0/2373893806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe64806dcb0 0x7fe64815c4e0 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:00.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.485+0000 7fe64cf4b640 1 -- 172.21.15.83:0/2373893806 >> 172.21.15.83:0/2373893806 conn(0x7fe64806abe0 msgr2=0x7fe6480af240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:00.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.485+0000 7fe64cf4b640 1 -- 172.21.15.83:0/2373893806 shutdown_connections 2024-04-17T10:49:00.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.485+0000 7fe64cf4b640 1 -- 172.21.15.83:0/2373893806 wait complete. 2024-04-17T10:49:00.498 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-17T10:49:00.498 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-17T10:49:00.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.613+0000 7fbf2e51c640 1 Processor -- start 2024-04-17T10:49:00.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.613+0000 7fbf2e51c640 1 -- start start 2024-04-17T10:49:00.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.617+0000 7fbf2e51c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbf2815dfa0 0x7fbf280b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:00.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.617+0000 7fbf2e51c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbf280ce980 0x7fbf280b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:00.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.617+0000 7fbf2e51c640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf28162a00 con 0x7fbf280cee30 2024-04-17T10:49:00.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.617+0000 7fbf2e51c640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf28161e80 con 0x7fbf2815dfa0 2024-04-17T10:49:00.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.617+0000 7fbf2e51c640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf28162160 con 0x7fbf280ce980 2024-04-17T10:49:00.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.617+0000 7fbf27fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbf2815dfa0 0x7fbf280b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:00.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.617+0000 7fbf27fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbf2815dfa0 0x7fbf280b6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40682/0 (socket says 172.21.15.83:40682) 2024-04-17T10:49:00.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.617+0000 7fbf27fff640 1 -- 172.21.15.83:0/311534680 learned_addr learned my addr 172.21.15.83:0/311534680 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:00.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.617+0000 7fbf1f7fe640 1 --2- 172.21.15.83:0/311534680 >> v2:172.21.15.167:3300/0 conn(0x7fbf280ce980 0x7fbf280b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:00.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.617+0000 7fbf27fff640 1 -- 172.21.15.83:0/311534680 >> v2:172.21.15.167:3300/0 conn(0x7fbf280ce980 msgr2=0x7fbf280b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:00.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.617+0000 7fbf27fff640 1 --2- 172.21.15.83:0/311534680 >> v2:172.21.15.167:3300/0 conn(0x7fbf280ce980 0x7fbf280b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:00.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.617+0000 7fbf27fff640 1 -- 172.21.15.83:0/311534680 >> v1:172.21.15.167:6789/0 conn(0x7fbf280cee30 legacy=0x7fbf280d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:00.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.617+0000 7fbf27fff640 1 -- 172.21.15.83:0/311534680 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf28162440 con 0x7fbf2815dfa0 2024-04-17T10:49:00.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.617+0000 7fbf27fff640 1 --2- 172.21.15.83:0/311534680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf2815dfa0 0x7fbf280b6c20 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fbf2808c8b0 tx=0x7fbf1801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f031f4c8feda05 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:00.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.617+0000 7fbf277fe640 1 -- 172.21.15.83:0/311534680 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf180b0070 con 0x7fbf2815dfa0 2024-04-17T10:49:00.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf277fe640 1 -- 172.21.15.83:0/311534680 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf180ab600 con 0x7fbf2815dfa0 2024-04-17T10:49:00.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf277fe640 1 -- 172.21.15.83:0/311534680 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf180ab9d0 con 0x7fbf2815dfa0 2024-04-17T10:49:00.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf2e51c640 1 -- 172.21.15.83:0/311534680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf2815dfa0 msgr2=0x7fbf280b6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:00.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf2e51c640 1 --2- 172.21.15.83:0/311534680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf2815dfa0 0x7fbf280b6c20 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fbf2808c8b0 tx=0x7fbf1801c420 comp rx=0 tx=0).stop 2024-04-17T10:49:00.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf2e51c640 1 -- 172.21.15.83:0/311534680 shutdown_connections 2024-04-17T10:49:00.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf2e51c640 1 --2- 172.21.15.83:0/311534680 >> v2:172.21.15.167:3300/0 conn(0x7fbf280ce980 0x7fbf280b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:00.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf2e51c640 1 --2- 172.21.15.83:0/311534680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf2815dfa0 0x7fbf280b6c20 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:00.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf2e51c640 1 -- 172.21.15.83:0/311534680 >> 172.21.15.83:0/311534680 conn(0x7fbf280d0600 msgr2=0x7fbf280d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:00.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf2e51c640 1 -- 172.21.15.83:0/311534680 shutdown_connections 2024-04-17T10:49:00.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf2e51c640 1 -- 172.21.15.83:0/311534680 wait complete. 2024-04-17T10:49:00.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf2e51c640 1 Processor -- start 2024-04-17T10:49:00.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf2e51c640 1 -- start start 2024-04-17T10:49:00.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf2e51c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf280ce980 0x7fbf280c8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:00.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf2e51c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf280cee30 0x7fbf281535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:00.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf2e51c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf2815dfa0 0x7fbf281514e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:00.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf2e51c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf28161b10 con 0x7fbf2815dfa0 2024-04-17T10:49:00.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf2e51c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf280d2850 con 0x7fbf280ce980 2024-04-17T10:49:00.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf2e51c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2816b660 con 0x7fbf280cee30 2024-04-17T10:49:00.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf2cab9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf2815dfa0 0x7fbf281514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:00.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf2cab9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf2815dfa0 0x7fbf281514e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40688/0 (socket says 172.21.15.83:40688) 2024-04-17T10:49:00.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf2cab9640 1 -- 172.21.15.83:0/4224292151 learned_addr learned my addr 172.21.15.83:0/4224292151 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:00.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf27fff640 1 --2- 172.21.15.83:0/4224292151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf280ce980 0x7fbf280c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:00.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf1f7fe640 1 --2- 172.21.15.83:0/4224292151 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf280cee30 0x7fbf281535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:00.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf2cab9640 1 -- 172.21.15.83:0/4224292151 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf280cee30 msgr2=0x7fbf281535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:00.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf2cab9640 1 --2- 172.21.15.83:0/4224292151 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf280cee30 0x7fbf281535d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:00.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf2cab9640 1 -- 172.21.15.83:0/4224292151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf280ce980 msgr2=0x7fbf280c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:00.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf2cab9640 1 --2- 172.21.15.83:0/4224292151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf280ce980 0x7fbf280c8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:00.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf2cab9640 1 -- 172.21.15.83:0/4224292151 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf280c76d0 con 0x7fbf2815dfa0 2024-04-17T10:49:00.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf27fff640 1 --2- 172.21.15.83:0/4224292151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf280ce980 0x7fbf280c8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:49:00.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf2cab9640 1 --2- 172.21.15.83:0/4224292151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf2815dfa0 0x7fbf281514e0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000b970 tx=0x7fbf2000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:00.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf257fa640 1 -- 172.21.15.83:0/4224292151 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf20019070 con 0x7fbf2815dfa0 2024-04-17T10:49:00.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf257fa640 1 -- 172.21.15.83:0/4224292151 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2000cae0 con 0x7fbf2815dfa0 2024-04-17T10:49:00.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf257fa640 1 -- 172.21.15.83:0/4224292151 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf20014440 con 0x7fbf2815dfa0 2024-04-17T10:49:00.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf2e51c640 1 -- 172.21.15.83:0/4224292151 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf280c79b0 con 0x7fbf2815dfa0 2024-04-17T10:49:00.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.621+0000 7fbf2e51c640 1 -- 172.21.15.83:0/4224292151 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf28153b10 con 0x7fbf2815dfa0 2024-04-17T10:49:00.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.625+0000 7fbf2e51c640 1 -- 172.21.15.83:0/4224292151 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf28074f90 con 0x7fbf2815dfa0 2024-04-17T10:49:00.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.625+0000 7fbf257fa640 1 -- 172.21.15.83:0/4224292151 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbf2001c070 con 0x7fbf2815dfa0 2024-04-17T10:49:00.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.625+0000 7fbf257fa640 1 --2- 172.21.15.83:0/4224292151 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbf08066f90 0x7fbf08069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:00.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.625+0000 7fbf27fff640 1 --2- 172.21.15.83:0/4224292151 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbf08066f90 0x7fbf08069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:00.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.625+0000 7fbf257fa640 1 -- 172.21.15.83:0/4224292151 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fbf20084c50 con 0x7fbf2815dfa0 2024-04-17T10:49:00.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.625+0000 7fbf27fff640 1 --2- 172.21.15.83:0/4224292151 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbf08066f90 0x7fbf08069450 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fbf180ad000 tx=0x7fbf180a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:00.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.629+0000 7fbf257fa640 1 -- 172.21.15.83:0/4224292151 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbf20055c10 con 0x7fbf2815dfa0 2024-04-17T10:49:00.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.829+0000 7fbf2e51c640 1 -- 172.21.15.83:0/4224292151 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fbf28154860 con 0x7fbf2815dfa0 2024-04-17T10:49:00.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.829+0000 7fbf257fa640 1 -- 172.21.15.83:0/4224292151 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1773 (secure 0 0 0) 0x7fbf20055510 con 0x7fbf2815dfa0 2024-04-17T10:49:00.826 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:00.826 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":673,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":116,"num_osds":8,"num_up_osds":8,"osd_up_since":1713350279,"num_in_osds":8,"osd_in_since":1713350270,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077883743,"bytes_used":2675843072,"bytes_avail":770418270208,"bytes_total":773094113280,"recovering_objects_per_sec":4,"recovering_bytes_per_sec":4324520,"recovering_keys_per_sec":0,"num_objects_recovered":33,"num_bytes_recovered":34603008,"num_keys_recovered":0,"write_bytes_sec":37245754,"read_op_per_sec":0,"write_op_per_sec":39},"fsmap":{"epoch":69,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8604}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-04-17T10:48:56.371485+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-17T10:49:00.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.829+0000 7fbf2e51c640 1 -- 172.21.15.83:0/4224292151 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbf08066f90 msgr2=0x7fbf08069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:00.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.829+0000 7fbf2e51c640 1 --2- 172.21.15.83:0/4224292151 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbf08066f90 0x7fbf08069450 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fbf180ad000 tx=0x7fbf180a6000 comp rx=0 tx=0).stop 2024-04-17T10:49:00.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.829+0000 7fbf2e51c640 1 -- 172.21.15.83:0/4224292151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf2815dfa0 msgr2=0x7fbf281514e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:00.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.829+0000 7fbf2e51c640 1 --2- 172.21.15.83:0/4224292151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf2815dfa0 0x7fbf281514e0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000b970 tx=0x7fbf2000be30 comp rx=0 tx=0).stop 2024-04-17T10:49:00.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.829+0000 7fbf2e51c640 1 -- 172.21.15.83:0/4224292151 shutdown_connections 2024-04-17T10:49:00.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.829+0000 7fbf2e51c640 1 --2- 172.21.15.83:0/4224292151 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbf08066f90 0x7fbf08069450 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:00.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.829+0000 7fbf2e51c640 1 --2- 172.21.15.83:0/4224292151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf2815dfa0 0x7fbf281514e0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:00.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.829+0000 7fbf2e51c640 1 --2- 172.21.15.83:0/4224292151 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf280cee30 0x7fbf281535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:00.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.829+0000 7fbf2e51c640 1 --2- 172.21.15.83:0/4224292151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf280ce980 0x7fbf280c8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:00.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.829+0000 7fbf2e51c640 1 -- 172.21.15.83:0/4224292151 >> 172.21.15.83:0/4224292151 conn(0x7fbf280d0600 msgr2=0x7fbf2814dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:00.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.829+0000 7fbf2e51c640 1 -- 172.21.15.83:0/4224292151 shutdown_connections 2024-04-17T10:49:00.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:00.829+0000 7fbf2e51c640 1 -- 172.21.15.83:0/4224292151 wait complete. 2024-04-17T10:49:00.843 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused'} 2024-04-17T10:49:01.844 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:49:01.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.981+0000 7f7d5d061640 1 Processor -- start 2024-04-17T10:49:01.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.981+0000 7f7d5d061640 1 -- start start 2024-04-17T10:49:01.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.981+0000 7f7d5d061640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7d5815ba30 0x7f7d58072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:01.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.981+0000 7f7d5d061640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d580cfbe0 0x7f7d5806c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:01.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.981+0000 7f7d5d061640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d5806e320 con 0x7f7d5815c960 2024-04-17T10:49:01.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.981+0000 7f7d5d061640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d58071ea0 con 0x7f7d5815ba30 2024-04-17T10:49:01.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.981+0000 7f7d5d061640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5806e040 con 0x7f7d580cfbe0 2024-04-17T10:49:01.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.981+0000 7f7d56d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7d5815ba30 0x7f7d58072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:01.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.981+0000 7f7d56d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7d5815ba30 0x7f7d58072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40700/0 (socket says 172.21.15.83:40700) 2024-04-17T10:49:01.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.981+0000 7f7d56d9d640 1 -- 172.21.15.83:0/1314937385 learned_addr learned my addr 172.21.15.83:0/1314937385 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:01.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.981+0000 7f7d5759e640 1 --2- 172.21.15.83:0/1314937385 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d580cfbe0 0x7f7d5806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:01.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.981+0000 7f7d56d9d640 1 -- 172.21.15.83:0/1314937385 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d580cfbe0 msgr2=0x7f7d5806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:01.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.981+0000 7f7d56d9d640 1 --2- 172.21.15.83:0/1314937385 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d580cfbe0 0x7f7d5806c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:01.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.981+0000 7f7d56d9d640 1 -- 172.21.15.83:0/1314937385 >> v1:172.21.15.167:6789/0 conn(0x7f7d5815c960 legacy=0x7f7d580cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:01.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.981+0000 7f7d56d9d640 1 -- 172.21.15.83:0/1314937385 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d58071850 con 0x7f7d5815ba30 2024-04-17T10:49:01.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.981+0000 7f7d5759e640 1 --2- 172.21.15.83:0/1314937385 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d580cfbe0 0x7f7d5806c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:01.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.985+0000 7f7d56d9d640 1 --2- 172.21.15.83:0/1314937385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d5815ba30 0x7f7d58072330 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f7d48009870 tx=0x7f7d4801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a46ce24bbf48c22c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:01.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.985+0000 7f7d55d9b640 1 -- 172.21.15.83:0/1314937385 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d480b0070 con 0x7f7d5815ba30 2024-04-17T10:49:01.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.985+0000 7f7d55d9b640 1 -- 172.21.15.83:0/1314937385 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d480ab600 con 0x7f7d5815ba30 2024-04-17T10:49:01.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.985+0000 7f7d55d9b640 1 -- 172.21.15.83:0/1314937385 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d480ab9d0 con 0x7f7d5815ba30 2024-04-17T10:49:01.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.985+0000 7f7d5d061640 1 -- 172.21.15.83:0/1314937385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d5815ba30 msgr2=0x7f7d58072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:01.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.985+0000 7f7d5d061640 1 --2- 172.21.15.83:0/1314937385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d5815ba30 0x7f7d58072330 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f7d48009870 tx=0x7f7d4801c490 comp rx=0 tx=0).stop 2024-04-17T10:49:01.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.985+0000 7f7d5d061640 1 -- 172.21.15.83:0/1314937385 shutdown_connections 2024-04-17T10:49:01.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.985+0000 7f7d5d061640 1 --2- 172.21.15.83:0/1314937385 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d580cfbe0 0x7f7d5806c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:01.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.985+0000 7f7d5d061640 1 --2- 172.21.15.83:0/1314937385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d5815ba30 0x7f7d58072330 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:01.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.985+0000 7f7d5d061640 1 -- 172.21.15.83:0/1314937385 >> 172.21.15.83:0/1314937385 conn(0x7f7d5806abe0 msgr2=0x7f7d5806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:01.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.985+0000 7f7d5d061640 1 -- 172.21.15.83:0/1314937385 shutdown_connections 2024-04-17T10:49:01.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.985+0000 7f7d5d061640 1 -- 172.21.15.83:0/1314937385 wait complete. 2024-04-17T10:49:01.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.985+0000 7f7d5d061640 1 Processor -- start 2024-04-17T10:49:01.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.985+0000 7f7d5d061640 1 -- start start 2024-04-17T10:49:01.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.985+0000 7f7d5d061640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d580cfbe0 0x7f7d58146b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:01.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.985+0000 7f7d5d061640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d5815ba30 0x7f7d58147fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:01.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.985+0000 7f7d5d061640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d5815c960 0x7f7d580af9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:01.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.985+0000 7f7d5d061640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5806de20 con 0x7f7d580cfbe0 2024-04-17T10:49:01.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.985+0000 7f7d5d061640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5806dca0 con 0x7f7d5815ba30 2024-04-17T10:49:01.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.985+0000 7f7d5d061640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d58168520 con 0x7f7d5815c960 2024-04-17T10:49:01.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.985+0000 7f7d56d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d580cfbe0 0x7f7d58146b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:01.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.985+0000 7f7d56d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d580cfbe0 0x7f7d58146b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40710/0 (socket says 172.21.15.83:40710) 2024-04-17T10:49:01.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.985+0000 7f7d56d9d640 1 -- 172.21.15.83:0/4135167723 learned_addr learned my addr 172.21.15.83:0/4135167723 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:01.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.985+0000 7f7d5759e640 1 --2- 172.21.15.83:0/4135167723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d5815c960 0x7f7d580af9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:01.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.985+0000 7f7d56d9d640 1 -- 172.21.15.83:0/4135167723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d5815c960 msgr2=0x7f7d580af9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:01.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.985+0000 7f7d56d9d640 1 --2- 172.21.15.83:0/4135167723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d5815c960 0x7f7d580af9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:01.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.985+0000 7f7d56d9d640 1 -- 172.21.15.83:0/4135167723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d5815ba30 msgr2=0x7f7d58147fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:01.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.985+0000 7f7d5659c640 1 --2- 172.21.15.83:0/4135167723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d5815ba30 0x7f7d58147fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:01.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.985+0000 7f7d56d9d640 1 --2- 172.21.15.83:0/4135167723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d5815ba30 0x7f7d58147fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:01.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.985+0000 7f7d56d9d640 1 -- 172.21.15.83:0/4135167723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d58155840 con 0x7f7d580cfbe0 2024-04-17T10:49:01.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.985+0000 7f7d5659c640 1 --2- 172.21.15.83:0/4135167723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d5815ba30 0x7f7d58147fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:01.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.985+0000 7f7d56d9d640 1 --2- 172.21.15.83:0/4135167723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d580cfbe0 0x7f7d58146b90 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f7d4801ab40 tx=0x7f7d4801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:01.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.989+0000 7f7d3f7fe640 1 -- 172.21.15.83:0/4135167723 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d480b0040 con 0x7f7d580cfbe0 2024-04-17T10:49:01.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.989+0000 7f7d3f7fe640 1 -- 172.21.15.83:0/4135167723 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d480bc070 con 0x7f7d580cfbe0 2024-04-17T10:49:01.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.989+0000 7f7d5d061640 1 -- 172.21.15.83:0/4135167723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d58155b20 con 0x7f7d580cfbe0 2024-04-17T10:49:01.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.989+0000 7f7d3f7fe640 1 -- 172.21.15.83:0/4135167723 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d480b6880 con 0x7f7d580cfbe0 2024-04-17T10:49:01.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.989+0000 7f7d5d061640 1 -- 172.21.15.83:0/4135167723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d581493a0 con 0x7f7d580cfbe0 2024-04-17T10:49:01.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.989+0000 7f7d5d061640 1 -- 172.21.15.83:0/4135167723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d58074f90 con 0x7f7d580cfbe0 2024-04-17T10:49:01.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.989+0000 7f7d3f7fe640 1 -- 172.21.15.83:0/4135167723 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7d480b6a20 con 0x7f7d580cfbe0 2024-04-17T10:49:01.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.989+0000 7f7d3f7fe640 1 --2- 172.21.15.83:0/4135167723 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7d44066f60 0x7f7d44069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:01.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.989+0000 7f7d5659c640 1 --2- 172.21.15.83:0/4135167723 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7d44066f60 0x7f7d44069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:01.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.989+0000 7f7d3f7fe640 1 -- 172.21.15.83:0/4135167723 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7d4811f5a0 con 0x7f7d580cfbe0 2024-04-17T10:49:01.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.989+0000 7f7d5659c640 1 --2- 172.21.15.83:0/4135167723 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7d44066f60 0x7f7d44069420 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7d581564a0 tx=0x7f7d4c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:01.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:01.993+0000 7f7d3f7fe640 1 -- 172.21.15.83:0/4135167723 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7d48016110 con 0x7f7d580cfbe0 2024-04-17T10:49:02.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.189+0000 7f7d5d061640 1 -- 172.21.15.83:0/4135167723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d5815e6e0 con 0x7f7d580cfbe0 2024-04-17T10:49:02.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.189+0000 7f7d3f7fe640 1 -- 172.21.15.83:0/4135167723 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f7d480f05a0 con 0x7f7d580cfbe0 2024-04-17T10:49:02.186 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:02.186 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5786,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1553533838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1553533838},{"type":"v1","addr":"172.21.15.167:6833","nonce":1553533838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/674536913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":674536913},{"type":"v1","addr":"172.21.15.167:6835","nonce":674536913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8598,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4183890457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4183890457},{"type":"v1","addr":"172.21.15.83:6835","nonce":4183890457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:48:24.738443+0000","modified":"2024-04-17T10:48:26.776273+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8604":{"gid":8604,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1479417998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1479417998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1479417998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-17T10:49:02.186 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 69 2024-04-17T10:49:02.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.193+0000 7f7d5d061640 1 -- 172.21.15.83:0/4135167723 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7d44066f60 msgr2=0x7f7d44069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:02.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.193+0000 7f7d5d061640 1 --2- 172.21.15.83:0/4135167723 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7d44066f60 0x7f7d44069420 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7d581564a0 tx=0x7f7d4c009040 comp rx=0 tx=0).stop 2024-04-17T10:49:02.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.193+0000 7f7d5d061640 1 -- 172.21.15.83:0/4135167723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d580cfbe0 msgr2=0x7f7d58146b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:02.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.193+0000 7f7d5d061640 1 --2- 172.21.15.83:0/4135167723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d580cfbe0 0x7f7d58146b90 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f7d4801ab40 tx=0x7f7d4801acd0 comp rx=0 tx=0).stop 2024-04-17T10:49:02.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.193+0000 7f7d5d061640 1 -- 172.21.15.83:0/4135167723 shutdown_connections 2024-04-17T10:49:02.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.193+0000 7f7d5d061640 1 --2- 172.21.15.83:0/4135167723 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7d44066f60 0x7f7d44069420 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:02.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.193+0000 7f7d5d061640 1 --2- 172.21.15.83:0/4135167723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d5815c960 0x7f7d580af9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:02.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.193+0000 7f7d5d061640 1 --2- 172.21.15.83:0/4135167723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d5815ba30 0x7f7d58147fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:02.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.193+0000 7f7d5d061640 1 --2- 172.21.15.83:0/4135167723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d580cfbe0 0x7f7d58146b90 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:02.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.193+0000 7f7d5d061640 1 -- 172.21.15.83:0/4135167723 >> 172.21.15.83:0/4135167723 conn(0x7f7d5806abe0 msgr2=0x7f7d5815d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:02.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.193+0000 7f7d5d061640 1 -- 172.21.15.83:0/4135167723 shutdown_connections 2024-04-17T10:49:02.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.193+0000 7f7d5d061640 1 -- 172.21.15.83:0/4135167723 wait complete. 2024-04-17T10:49:02.206 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-17T10:49:02.206 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-17T10:49:02.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.325+0000 7f94578d5640 1 Processor -- start 2024-04-17T10:49:02.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.325+0000 7f94578d5640 1 -- start start 2024-04-17T10:49:02.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.329+0000 7f94578d5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f945015ba30 0x7f9450072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:02.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.329+0000 7f94578d5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f945015c960 0x7f94500cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:02.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.329+0000 7f94578d5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94500cfa10 0x7f945006c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:02.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.329+0000 7f94578d5640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9450071ea0 con 0x7f945015c960 2024-04-17T10:49:02.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.329+0000 7f94578d5640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94500cfea0 con 0x7f945015ba30 2024-04-17T10:49:02.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.329+0000 7f94578d5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f945006e320 con 0x7f94500cfa10 2024-04-17T10:49:02.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.329+0000 7f9454e70640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f945015c960 0x7f94500cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:02.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.329+0000 7f9454e70640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f945015c960 0x7f94500cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40718/0 (socket says 172.21.15.83:40718) 2024-04-17T10:49:02.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.329+0000 7f9454e70640 1 -- 172.21.15.83:0/563004063 learned_addr learned my addr 172.21.15.83:0/563004063 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:02.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.329+0000 7f9455e72640 1 --2- 172.21.15.83:0/563004063 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94500cfa10 0x7f945006c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:02.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.329+0000 7f9455671640 1 --2- 172.21.15.83:0/563004063 >> v2:172.21.15.167:3300/0 conn(0x7f945015ba30 0x7f9450072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:02.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.329+0000 7f9454e70640 1 -- 172.21.15.83:0/563004063 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94500cfa10 msgr2=0x7f945006c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:02.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.329+0000 7f9454e70640 1 --2- 172.21.15.83:0/563004063 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94500cfa10 0x7f945006c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:02.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.329+0000 7f9454e70640 1 -- 172.21.15.83:0/563004063 >> v2:172.21.15.167:3300/0 conn(0x7f945015ba30 msgr2=0x7f9450072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:02.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.329+0000 7f9454e70640 1 --2- 172.21.15.83:0/563004063 >> v2:172.21.15.167:3300/0 conn(0x7f945015ba30 0x7f9450072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:02.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.329+0000 7f9454e70640 1 -- 172.21.15.83:0/563004063 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945006e040 con 0x7f945015c960 2024-04-17T10:49:02.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.329+0000 7f9454e70640 1 --2- 172.21.15.83:0/563004063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f945015c960 0x7f94500cf4d0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f94500b2410 tx=0x7f944001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4203ae81775dbe99 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:02.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.329+0000 7f9447fff640 1 -- 172.21.15.83:0/563004063 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94400b0070 con 0x7f945015c960 2024-04-17T10:49:02.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.329+0000 7f9447fff640 1 -- 172.21.15.83:0/563004063 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94400ab600 con 0x7f945015c960 2024-04-17T10:49:02.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.329+0000 7f9447fff640 1 -- 172.21.15.83:0/563004063 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94400ab9d0 con 0x7f945015c960 2024-04-17T10:49:02.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.329+0000 7f94578d5640 1 -- 172.21.15.83:0/563004063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f945015c960 msgr2=0x7f94500cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:02.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.329+0000 7f94578d5640 1 --2- 172.21.15.83:0/563004063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f945015c960 0x7f94500cf4d0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f94500b2410 tx=0x7f944001c490 comp rx=0 tx=0).stop 2024-04-17T10:49:02.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.329+0000 7f94578d5640 1 -- 172.21.15.83:0/563004063 shutdown_connections 2024-04-17T10:49:02.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.329+0000 7f94578d5640 1 --2- 172.21.15.83:0/563004063 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94500cfa10 0x7f945006c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:02.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.329+0000 7f94578d5640 1 --2- 172.21.15.83:0/563004063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f945015c960 0x7f94500cf4d0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:02.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.329+0000 7f94578d5640 1 --2- 172.21.15.83:0/563004063 >> v2:172.21.15.167:3300/0 conn(0x7f945015ba30 0x7f9450072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:02.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.329+0000 7f94578d5640 1 -- 172.21.15.83:0/563004063 >> 172.21.15.83:0/563004063 conn(0x7f945006abe0 msgr2=0x7f945006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:02.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.333+0000 7f94578d5640 1 -- 172.21.15.83:0/563004063 shutdown_connections 2024-04-17T10:49:02.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.333+0000 7f94578d5640 1 -- 172.21.15.83:0/563004063 wait complete. 2024-04-17T10:49:02.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.333+0000 7f94578d5640 1 Processor -- start 2024-04-17T10:49:02.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.333+0000 7f94578d5640 1 -- start start 2024-04-17T10:49:02.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.333+0000 7f94578d5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94500cfa10 0x7f9450137b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:02.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.333+0000 7f94578d5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f945015ba30 0x7f9450155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:02.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.333+0000 7f94578d5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f945015c960 0x7f945014cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:02.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.333+0000 7f94578d5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945006dca0 con 0x7f945015ba30 2024-04-17T10:49:02.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.333+0000 7f94578d5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94500aeae0 con 0x7f94500cfa10 2024-04-17T10:49:02.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.333+0000 7f94578d5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f945006de20 con 0x7f945015c960 2024-04-17T10:49:02.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.333+0000 7f9454e70640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f945015ba30 0x7f9450155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:02.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.333+0000 7f9454e70640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f945015ba30 0x7f9450155080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40726/0 (socket says 172.21.15.83:40726) 2024-04-17T10:49:02.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.333+0000 7f9454e70640 1 -- 172.21.15.83:0/3430301481 learned_addr learned my addr 172.21.15.83:0/3430301481 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:02.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.333+0000 7f9455e72640 1 --2- 172.21.15.83:0/3430301481 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f945015c960 0x7f945014cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:02.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.333+0000 7f9455671640 1 --2- 172.21.15.83:0/3430301481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94500cfa10 0x7f9450137b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:02.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.333+0000 7f9454e70640 1 -- 172.21.15.83:0/3430301481 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f945015c960 msgr2=0x7f945014cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:02.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.333+0000 7f9454e70640 1 --2- 172.21.15.83:0/3430301481 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f945015c960 0x7f945014cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:02.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.333+0000 7f9454e70640 1 -- 172.21.15.83:0/3430301481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94500cfa10 msgr2=0x7f9450137b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:02.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.333+0000 7f9454e70640 1 --2- 172.21.15.83:0/3430301481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94500cfa10 0x7f9450137b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:02.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.333+0000 7f9454e70640 1 -- 172.21.15.83:0/3430301481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9450156f10 con 0x7f945015ba30 2024-04-17T10:49:02.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.333+0000 7f9454e70640 1 --2- 172.21.15.83:0/3430301481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f945015ba30 0x7f9450155080 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f944001cd30 tx=0x7f9440019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:02.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.333+0000 7f9445ffb640 1 -- 172.21.15.83:0/3430301481 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94400b8020 con 0x7f945015ba30 2024-04-17T10:49:02.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.333+0000 7f9445ffb640 1 -- 172.21.15.83:0/3430301481 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94400b4410 con 0x7f945015ba30 2024-04-17T10:49:02.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.333+0000 7f9445ffb640 1 -- 172.21.15.83:0/3430301481 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94400b0040 con 0x7f945015ba30 2024-04-17T10:49:02.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.333+0000 7f94578d5640 1 -- 172.21.15.83:0/3430301481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f945014c400 con 0x7f945015ba30 2024-04-17T10:49:02.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.333+0000 7f94578d5640 1 -- 172.21.15.83:0/3430301481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94501555c0 con 0x7f945015ba30 2024-04-17T10:49:02.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.337+0000 7f94578d5640 1 -- 172.21.15.83:0/3430301481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9450074f90 con 0x7f945015ba30 2024-04-17T10:49:02.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.337+0000 7f9445ffb640 1 -- 172.21.15.83:0/3430301481 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f94400bb020 con 0x7f945015ba30 2024-04-17T10:49:02.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.337+0000 7f9445ffb640 1 --2- 172.21.15.83:0/3430301481 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9434066f90 0x7f9434069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:02.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.337+0000 7f9445ffb640 1 -- 172.21.15.83:0/3430301481 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f944011e950 con 0x7f945015ba30 2024-04-17T10:49:02.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.337+0000 7f9455671640 1 --2- 172.21.15.83:0/3430301481 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9434066f90 0x7f9434069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:02.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.337+0000 7f9455671640 1 --2- 172.21.15.83:0/3430301481 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9434066f90 0x7f9434069450 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f945015de00 tx=0x7f944c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:02.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.341+0000 7f9445ffb640 1 -- 172.21.15.83:0/3430301481 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f94400c2070 con 0x7f945015ba30 2024-04-17T10:49:02.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.557+0000 7f94578d5640 1 -- 172.21.15.83:0/3430301481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f94500723c0 con 0x7f945015ba30 2024-04-17T10:49:02.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.557+0000 7f9445ffb640 1 -- 172.21.15.83:0/3430301481 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1750 (secure 0 0 0) 0x7f9440016110 con 0x7f945015ba30 2024-04-17T10:49:02.555 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:02.555 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":675,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":116,"num_osds":8,"num_up_osds":8,"osd_up_since":1713350279,"num_in_osds":8,"osd_in_since":1713350270,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077883743,"bytes_used":2675843072,"bytes_avail":770418270208,"bytes_total":773094113280,"recovering_objects_per_sec":4,"recovering_bytes_per_sec":4324084,"recovering_keys_per_sec":0,"num_objects_recovered":33,"num_bytes_recovered":34603008,"num_keys_recovered":0,"write_bytes_sec":37241992,"read_op_per_sec":0,"write_op_per_sec":39},"fsmap":{"epoch":69,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8604}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2024-04-17T10:49:02.373478+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-17T10:49:02.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.561+0000 7f94578d5640 1 -- 172.21.15.83:0/3430301481 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9434066f90 msgr2=0x7f9434069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:02.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.561+0000 7f94578d5640 1 --2- 172.21.15.83:0/3430301481 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9434066f90 0x7f9434069450 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f945015de00 tx=0x7f944c008040 comp rx=0 tx=0).stop 2024-04-17T10:49:02.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.561+0000 7f94578d5640 1 -- 172.21.15.83:0/3430301481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f945015ba30 msgr2=0x7f9450155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:02.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.561+0000 7f94578d5640 1 --2- 172.21.15.83:0/3430301481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f945015ba30 0x7f9450155080 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f944001cd30 tx=0x7f9440019230 comp rx=0 tx=0).stop 2024-04-17T10:49:02.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.561+0000 7f94578d5640 1 -- 172.21.15.83:0/3430301481 shutdown_connections 2024-04-17T10:49:02.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.561+0000 7f94578d5640 1 --2- 172.21.15.83:0/3430301481 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9434066f90 0x7f9434069450 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:02.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.561+0000 7f94578d5640 1 --2- 172.21.15.83:0/3430301481 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f945015c960 0x7f945014cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:02.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.561+0000 7f94578d5640 1 --2- 172.21.15.83:0/3430301481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f945015ba30 0x7f9450155080 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:02.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.561+0000 7f94578d5640 1 --2- 172.21.15.83:0/3430301481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94500cfa10 0x7f9450137b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:02.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.561+0000 7f94578d5640 1 -- 172.21.15.83:0/3430301481 >> 172.21.15.83:0/3430301481 conn(0x7f945006abe0 msgr2=0x7f94500af6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:02.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.561+0000 7f94578d5640 1 -- 172.21.15.83:0/3430301481 shutdown_connections 2024-04-17T10:49:02.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.561+0000 7f94578d5640 1 -- 172.21.15.83:0/3430301481 wait complete. 2024-04-17T10:49:02.572 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-17T10:49:02.573 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T10:49:02.573 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:49:02.625 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:49:02.675 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T10:49:02.675 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:49:02.675 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:49:02.681 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:49:02.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.845+0000 7f0a4e6aa640 1 Processor -- start 2024-04-17T10:49:02.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.845+0000 7f0a4e6aa640 1 -- start start 2024-04-17T10:49:02.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.849+0000 7f0a4e6aa640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0a4815db50 0x7f0a48070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:02.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.849+0000 7f0a4e6aa640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a480aedd0 0x7f0a4806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:02.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.849+0000 7f0a4e6aa640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0a4806dae0 0x7f0a480cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:02.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.849+0000 7f0a4e6aa640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a480709b0 con 0x7f0a4815db50 2024-04-17T10:49:02.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.849+0000 7f0a4e6aa640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a4815e440 con 0x7f0a4806dae0 2024-04-17T10:49:02.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.849+0000 7f0a4e6aa640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4815e720 con 0x7f0a480aedd0 2024-04-17T10:49:02.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.849+0000 7f0a47fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0a4815db50 0x7f0a48070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:02.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.849+0000 7f0a47fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0a4815db50 0x7f0a48070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40732/0 (socket says 172.21.15.83:40732) 2024-04-17T10:49:02.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.849+0000 7f0a47fff640 1 -- 172.21.15.83:0/3549791623 learned_addr learned my addr 172.21.15.83:0/3549791623 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:02.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.849+0000 7f0a47fff640 1 -- 172.21.15.83:0/3549791623 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a480aedd0 msgr2=0x7f0a4806d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:02.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.849+0000 7f0a47fff640 1 --2- 172.21.15.83:0/3549791623 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a480aedd0 0x7f0a4806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:02.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.849+0000 7f0a47fff640 1 -- 172.21.15.83:0/3549791623 >> v2:172.21.15.167:3300/0 conn(0x7f0a4806dae0 msgr2=0x7f0a480cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:02.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.849+0000 7f0a47fff640 1 --2- 172.21.15.83:0/3549791623 >> v2:172.21.15.167:3300/0 conn(0x7f0a4806dae0 0x7f0a480cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:02.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.849+0000 7f0a47fff640 1 -- 172.21.15.83:0/3549791623 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a4815ea00 con 0x7f0a4815db50 2024-04-17T10:49:02.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.849+0000 7f0a47fff640 1 --2- 172.21.15.83:0/3549791623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4815db50 0x7f0a48070470 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f0a480b2ac0 tx=0x7f0a3801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f9bf34db0ce2c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:02.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.849+0000 7f0a46ffd640 1 -- 172.21.15.83:0/3549791623 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a380b0070 con 0x7f0a4815db50 2024-04-17T10:49:02.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.849+0000 7f0a46ffd640 1 -- 172.21.15.83:0/3549791623 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a380ab600 con 0x7f0a4815db50 2024-04-17T10:49:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.849+0000 7f0a46ffd640 1 -- 172.21.15.83:0/3549791623 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a380ab9d0 con 0x7f0a4815db50 2024-04-17T10:49:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.849+0000 7f0a4e6aa640 1 -- 172.21.15.83:0/3549791623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4815db50 msgr2=0x7f0a48070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.849+0000 7f0a4e6aa640 1 --2- 172.21.15.83:0/3549791623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4815db50 0x7f0a48070470 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f0a480b2ac0 tx=0x7f0a3801c260 comp rx=0 tx=0).stop 2024-04-17T10:49:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a4e6aa640 1 -- 172.21.15.83:0/3549791623 shutdown_connections 2024-04-17T10:49:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a4e6aa640 1 --2- 172.21.15.83:0/3549791623 >> v2:172.21.15.167:3300/0 conn(0x7f0a4806dae0 0x7f0a480cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a4e6aa640 1 --2- 172.21.15.83:0/3549791623 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a480aedd0 0x7f0a4806d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a4e6aa640 1 --2- 172.21.15.83:0/3549791623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4815db50 0x7f0a48070470 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a4e6aa640 1 -- 172.21.15.83:0/3549791623 >> 172.21.15.83:0/3549791623 conn(0x7f0a4806abe0 msgr2=0x7f0a4806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a4e6aa640 1 -- 172.21.15.83:0/3549791623 shutdown_connections 2024-04-17T10:49:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a4e6aa640 1 -- 172.21.15.83:0/3549791623 wait complete. 2024-04-17T10:49:02.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a4e6aa640 1 Processor -- start 2024-04-17T10:49:02.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a4e6aa640 1 -- start start 2024-04-17T10:49:02.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a4e6aa640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a4806dae0 0x7f0a48164090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:02.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a4e6aa640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a480aedd0 0x7f0a481f6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:02.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a4e6aa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4815db50 0x7f0a481f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:02.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a4e6aa640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a480b2f90 con 0x7f0a4815db50 2024-04-17T10:49:02.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a4e6aa640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a480b2e10 con 0x7f0a4806dae0 2024-04-17T10:49:02.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a4e6aa640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a480b3110 con 0x7f0a480aedd0 2024-04-17T10:49:02.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a4cc47640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4815db50 0x7f0a481f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:02.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a4cc47640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4815db50 0x7f0a481f6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40742/0 (socket says 172.21.15.83:40742) 2024-04-17T10:49:02.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a4cc47640 1 -- 172.21.15.83:0/100712967 learned_addr learned my addr 172.21.15.83:0/100712967 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:02.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a47fff640 1 --2- 172.21.15.83:0/100712967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a4806dae0 0x7f0a48164090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:02.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a477fe640 1 --2- 172.21.15.83:0/100712967 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a480aedd0 0x7f0a481f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:02.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a4cc47640 1 -- 172.21.15.83:0/100712967 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a480aedd0 msgr2=0x7f0a481f6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:02.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a4cc47640 1 --2- 172.21.15.83:0/100712967 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a480aedd0 0x7f0a481f6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:02.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a4cc47640 1 -- 172.21.15.83:0/100712967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a4806dae0 msgr2=0x7f0a48164090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:02.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a4cc47640 1 --2- 172.21.15.83:0/100712967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a4806dae0 0x7f0a48164090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:02.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a4cc47640 1 -- 172.21.15.83:0/100712967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a481f70b0 con 0x7f0a4815db50 2024-04-17T10:49:02.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a47fff640 1 --2- 172.21.15.83:0/100712967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a4806dae0 0x7f0a48164090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:02.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a477fe640 1 --2- 172.21.15.83:0/100712967 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a480aedd0 0x7f0a481f6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:02.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a4cc47640 1 --2- 172.21.15.83:0/100712967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4815db50 0x7f0a481f6b70 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000b9a0 tx=0x7f0a4000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:02.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a44ff9640 1 -- 172.21.15.83:0/100712967 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a40019070 con 0x7f0a4815db50 2024-04-17T10:49:02.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a44ff9640 1 -- 172.21.15.83:0/100712967 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4000cbc0 con 0x7f0a4815db50 2024-04-17T10:49:02.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a4e6aa640 1 -- 172.21.15.83:0/100712967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a48160de0 con 0x7f0a4815db50 2024-04-17T10:49:02.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a44ff9640 1 -- 172.21.15.83:0/100712967 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a40014440 con 0x7f0a4815db50 2024-04-17T10:49:02.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a4e6aa640 1 -- 172.21.15.83:0/100712967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a48160b00 con 0x7f0a4815db50 2024-04-17T10:49:02.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.853+0000 7f0a4e6aa640 1 -- 172.21.15.83:0/100712967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a48074f90 con 0x7f0a4815db50 2024-04-17T10:49:02.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.857+0000 7f0a44ff9640 1 -- 172.21.15.83:0/100712967 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0a4001c070 con 0x7f0a4815db50 2024-04-17T10:49:02.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.857+0000 7f0a44ff9640 1 --2- 172.21.15.83:0/100712967 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0a34066f60 0x7f0a34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:02.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.857+0000 7f0a47fff640 1 --2- 172.21.15.83:0/100712967 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0a34066f60 0x7f0a34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:02.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.857+0000 7f0a44ff9640 1 -- 172.21.15.83:0/100712967 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0a40084d20 con 0x7f0a4815db50 2024-04-17T10:49:02.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.857+0000 7f0a47fff640 1 --2- 172.21.15.83:0/100712967 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0a34066f60 0x7f0a34069420 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0a380ad000 tx=0x7f0a38008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:02.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:02.861+0000 7f0a44ff9640 1 -- 172.21.15.83:0/100712967 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0a40055d80 con 0x7f0a4815db50 2024-04-17T10:49:03.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.021+0000 7f0a4e6aa640 1 -- 172.21.15.83:0/100712967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a48071f60 con 0x7f0a4815db50 2024-04-17T10:49:03.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.021+0000 7f0a44ff9640 1 -- 172.21.15.83:0/100712967 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+16649 (secure 0 0 0) 0x7f0a40055680 con 0x7f0a4815db50 2024-04-17T10:49:03.020 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:03.020 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":116,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:48:55.454382+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:48:22.981915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-17T10:48:23.700138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.18","mappings":[{"from":0,"to":5}]},{"pgid":"17.34","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:49:03.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.025+0000 7f0a4e6aa640 1 -- 172.21.15.83:0/100712967 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0a34066f60 msgr2=0x7f0a34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:03.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.025+0000 7f0a4e6aa640 1 --2- 172.21.15.83:0/100712967 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0a34066f60 0x7f0a34069420 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0a380ad000 tx=0x7f0a38008040 comp rx=0 tx=0).stop 2024-04-17T10:49:03.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.025+0000 7f0a4e6aa640 1 -- 172.21.15.83:0/100712967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4815db50 msgr2=0x7f0a481f6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:03.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.025+0000 7f0a4e6aa640 1 --2- 172.21.15.83:0/100712967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4815db50 0x7f0a481f6b70 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000b9a0 tx=0x7f0a4000be60 comp rx=0 tx=0).stop 2024-04-17T10:49:03.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.025+0000 7f0a4e6aa640 1 -- 172.21.15.83:0/100712967 shutdown_connections 2024-04-17T10:49:03.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.025+0000 7f0a4e6aa640 1 --2- 172.21.15.83:0/100712967 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0a34066f60 0x7f0a34069420 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.025+0000 7f0a4e6aa640 1 --2- 172.21.15.83:0/100712967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4815db50 0x7f0a481f6b70 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.025+0000 7f0a4e6aa640 1 --2- 172.21.15.83:0/100712967 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a480aedd0 0x7f0a481f6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.025+0000 7f0a4e6aa640 1 --2- 172.21.15.83:0/100712967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a4806dae0 0x7f0a48164090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.025+0000 7f0a4e6aa640 1 -- 172.21.15.83:0/100712967 >> 172.21.15.83:0/100712967 conn(0x7f0a4806abe0 msgr2=0x7f0a480af2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:03.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.025+0000 7f0a4e6aa640 1 -- 172.21.15.83:0/100712967 shutdown_connections 2024-04-17T10:49:03.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.025+0000 7f0a4e6aa640 1 -- 172.21.15.83:0/100712967 wait complete. 2024-04-17T10:49:03.039 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:48:22.981915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:48:23.700138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:49:03.039 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:49:03.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.169+0000 7fc987421640 1 Processor -- start 2024-04-17T10:49:03.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.169+0000 7fc987421640 1 -- start start 2024-04-17T10:49:03.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.169+0000 7fc987421640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc98015e0f0 0x7fc9800b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:03.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.169+0000 7fc987421640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc9800ce400 0x7fc980161c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:03.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.169+0000 7fc987421640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc98015e4d0 con 0x7fc9800b5450 2024-04-17T10:49:03.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.169+0000 7fc987421640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc980162b40 con 0x7fc98015e0f0 2024-04-17T10:49:03.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.169+0000 7fc987421640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc980162580 con 0x7fc9800ce400 2024-04-17T10:49:03.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.169+0000 7fc9849bc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc9800ce400 0x7fc980161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:03.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.169+0000 7fc9851bd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc98015e0f0 0x7fc9800b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:03.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.169+0000 7fc9851bd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc98015e0f0 0x7fc9800b6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:58780/0 (socket says 172.21.15.83:58780) 2024-04-17T10:49:03.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.169+0000 7fc9849bc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc9800ce400 0x7fc980161c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:37764/0 (socket says 172.21.15.83:37764) 2024-04-17T10:49:03.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.169+0000 7fc9851bd640 1 -- 172.21.15.83:0/4115704185 learned_addr learned my addr 172.21.15.83:0/4115704185 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:03.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.169+0000 7fc9851bd640 1 -- 172.21.15.83:0/4115704185 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc9800ce400 msgr2=0x7fc980161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:03.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.169+0000 7fc9851bd640 1 --2- 172.21.15.83:0/4115704185 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc9800ce400 0x7fc980161c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.169+0000 7fc9851bd640 1 -- 172.21.15.83:0/4115704185 >> v1:172.21.15.167:6789/0 conn(0x7fc9800b5450 legacy=0x7fc9800d2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:03.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.169+0000 7fc9851bd640 1 -- 172.21.15.83:0/4115704185 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc980162860 con 0x7fc98015e0f0 2024-04-17T10:49:03.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.169+0000 7fc9849bc640 1 --2- 172.21.15.83:0/4115704185 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc9800ce400 0x7fc980161c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:49:03.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.173+0000 7fc9851bd640 1 --2- 172.21.15.83:0/4115704185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc98015e0f0 0x7fc9800b6d00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc978009870 tx=0x7fc97801c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea1f0dc9110b9963 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:03.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.173+0000 7fc97ffff640 1 -- 172.21.15.83:0/4115704185 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9780b2070 con 0x7fc98015e0f0 2024-04-17T10:49:03.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.173+0000 7fc97ffff640 1 -- 172.21.15.83:0/4115704185 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9780ad640 con 0x7fc98015e0f0 2024-04-17T10:49:03.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.173+0000 7fc97ffff640 1 -- 172.21.15.83:0/4115704185 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9780ada10 con 0x7fc98015e0f0 2024-04-17T10:49:03.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.173+0000 7fc987421640 1 -- 172.21.15.83:0/4115704185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc98015e0f0 msgr2=0x7fc9800b6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:03.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.173+0000 7fc987421640 1 --2- 172.21.15.83:0/4115704185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc98015e0f0 0x7fc9800b6d00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc978009870 tx=0x7fc97801c4d0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.173+0000 7fc987421640 1 -- 172.21.15.83:0/4115704185 shutdown_connections 2024-04-17T10:49:03.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.173+0000 7fc987421640 1 --2- 172.21.15.83:0/4115704185 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc9800ce400 0x7fc980161c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.173+0000 7fc987421640 1 --2- 172.21.15.83:0/4115704185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc98015e0f0 0x7fc9800b6d00 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.173+0000 7fc987421640 1 -- 172.21.15.83:0/4115704185 >> 172.21.15.83:0/4115704185 conn(0x7fc9800d0450 msgr2=0x7fc9800d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:03.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.173+0000 7fc987421640 1 -- 172.21.15.83:0/4115704185 shutdown_connections 2024-04-17T10:49:03.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.173+0000 7fc987421640 1 -- 172.21.15.83:0/4115704185 wait complete. 2024-04-17T10:49:03.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.173+0000 7fc987421640 1 Processor -- start 2024-04-17T10:49:03.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.173+0000 7fc987421640 1 -- start start 2024-04-17T10:49:03.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.177+0000 7fc987421640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9800b5450 0x7fc9801f6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:03.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.177+0000 7fc987421640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9800ce400 0x7fc98014bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:03.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.177+0000 7fc987421640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc98015e0f0 0x7fc98014d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:03.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.177+0000 7fc987421640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9801622d0 con 0x7fc9800ce400 2024-04-17T10:49:03.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.177+0000 7fc987421640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc980162150 con 0x7fc9800b5450 2024-04-17T10:49:03.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.177+0000 7fc987421640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc98016af50 con 0x7fc98015e0f0 2024-04-17T10:49:03.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.177+0000 7fc9849bc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9800ce400 0x7fc98014bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:03.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.177+0000 7fc9849bc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9800ce400 0x7fc98014bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40758/0 (socket says 172.21.15.83:40758) 2024-04-17T10:49:03.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.177+0000 7fc9849bc640 1 -- 172.21.15.83:0/928339592 learned_addr learned my addr 172.21.15.83:0/928339592 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:03.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.177+0000 7fc9859be640 1 --2- 172.21.15.83:0/928339592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc98015e0f0 0x7fc98014d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:03.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.177+0000 7fc9851bd640 1 --2- 172.21.15.83:0/928339592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9800b5450 0x7fc9801f6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:03.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.177+0000 7fc9849bc640 1 -- 172.21.15.83:0/928339592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc98015e0f0 msgr2=0x7fc98014d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:03.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.177+0000 7fc9849bc640 1 --2- 172.21.15.83:0/928339592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc98015e0f0 0x7fc98014d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.177+0000 7fc9849bc640 1 -- 172.21.15.83:0/928339592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9800b5450 msgr2=0x7fc9801f6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:03.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.177+0000 7fc9849bc640 1 --2- 172.21.15.83:0/928339592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9800b5450 0x7fc9801f6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.177+0000 7fc9849bc640 1 -- 172.21.15.83:0/928339592 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9800caaa0 con 0x7fc9800ce400 2024-04-17T10:49:03.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.177+0000 7fc9849bc640 1 --2- 172.21.15.83:0/928339592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9800ce400 0x7fc98014bba0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fc97400e9d0 tx=0x7fc97400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:03.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.177+0000 7fc97dffb640 1 -- 172.21.15.83:0/928339592 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc974017070 con 0x7fc9800ce400 2024-04-17T10:49:03.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.177+0000 7fc97dffb640 1 -- 172.21.15.83:0/928339592 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97400ca60 con 0x7fc9800ce400 2024-04-17T10:49:03.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.177+0000 7fc987421640 1 -- 172.21.15.83:0/928339592 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9800cad80 con 0x7fc9800ce400 2024-04-17T10:49:03.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.177+0000 7fc97dffb640 1 -- 172.21.15.83:0/928339592 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97400ce50 con 0x7fc9800ce400 2024-04-17T10:49:03.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.177+0000 7fc987421640 1 -- 172.21.15.83:0/928339592 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9800cb340 con 0x7fc9800ce400 2024-04-17T10:49:03.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.177+0000 7fc987421640 1 -- 172.21.15.83:0/928339592 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9800681a0 con 0x7fc9800ce400 2024-04-17T10:49:03.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.177+0000 7fc97dffb640 1 -- 172.21.15.83:0/928339592 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc97401a070 con 0x7fc9800ce400 2024-04-17T10:49:03.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.177+0000 7fc97dffb640 1 --2- 172.21.15.83:0/928339592 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc96c067100 0x7fc96c0695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:03.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.177+0000 7fc9851bd640 1 --2- 172.21.15.83:0/928339592 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc96c067100 0x7fc96c0695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:03.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.177+0000 7fc97dffb640 1 -- 172.21.15.83:0/928339592 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc974085000 con 0x7fc9800ce400 2024-04-17T10:49:03.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.181+0000 7fc9851bd640 1 --2- 172.21.15.83:0/928339592 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc96c067100 0x7fc96c0695c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc9780af000 tx=0x7fc978008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:03.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.181+0000 7fc97dffb640 1 -- 172.21.15.83:0/928339592 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc974055980 con 0x7fc9800ce400 2024-04-17T10:49:03.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.341+0000 7fc987421640 1 -- 172.21.15.83:0/928339592 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc980074f90 con 0x7fc9800ce400 2024-04-17T10:49:03.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.341+0000 7fc97dffb640 1 -- 172.21.15.83:0/928339592 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc974059a20 con 0x7fc9800ce400 2024-04-17T10:49:03.338 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:49:03.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.345+0000 7fc987421640 1 -- 172.21.15.83:0/928339592 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc96c067100 msgr2=0x7fc96c0695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:03.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.345+0000 7fc987421640 1 --2- 172.21.15.83:0/928339592 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc96c067100 0x7fc96c0695c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc9780af000 tx=0x7fc978008040 comp rx=0 tx=0).stop 2024-04-17T10:49:03.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.345+0000 7fc987421640 1 -- 172.21.15.83:0/928339592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9800ce400 msgr2=0x7fc98014bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:03.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.345+0000 7fc987421640 1 --2- 172.21.15.83:0/928339592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9800ce400 0x7fc98014bba0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fc97400e9d0 tx=0x7fc97400ee90 comp rx=0 tx=0).stop 2024-04-17T10:49:03.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.345+0000 7fc987421640 1 -- 172.21.15.83:0/928339592 shutdown_connections 2024-04-17T10:49:03.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.345+0000 7fc987421640 1 --2- 172.21.15.83:0/928339592 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc96c067100 0x7fc96c0695c0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.345+0000 7fc987421640 1 --2- 172.21.15.83:0/928339592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc98015e0f0 0x7fc98014d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.345+0000 7fc987421640 1 --2- 172.21.15.83:0/928339592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9800ce400 0x7fc98014bba0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.345+0000 7fc987421640 1 --2- 172.21.15.83:0/928339592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9800b5450 0x7fc9801f6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.345+0000 7fc987421640 1 -- 172.21.15.83:0/928339592 >> 172.21.15.83:0/928339592 conn(0x7fc9800d0450 msgr2=0x7fc98015e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:03.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.345+0000 7fc987421640 1 -- 172.21.15.83:0/928339592 shutdown_connections 2024-04-17T10:49:03.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.345+0000 7fc987421640 1 -- 172.21.15.83:0/928339592 wait complete. 2024-04-17T10:49:03.357 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:49:03.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.477+0000 7ff996fcf640 1 Processor -- start 2024-04-17T10:49:03.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.477+0000 7ff996fcf640 1 -- start start 2024-04-17T10:49:03.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.481+0000 7ff996fcf640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff990154670 0x7ff9900aea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:03.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.481+0000 7ff996fcf640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff9900b4090 0x7ff99006c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:03.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.481+0000 7ff996fcf640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9900b4520 0x7ff99006cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:03.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.481+0000 7ff996fcf640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff990154a50 con 0x7ff990154670 2024-04-17T10:49:03.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.481+0000 7ff996fcf640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9900b2ff0 con 0x7ff9900b4090 2024-04-17T10:49:03.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.481+0000 7ff996fcf640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9900b32d0 con 0x7ff9900b4520 2024-04-17T10:49:03.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.481+0000 7ff994d6b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff990154670 0x7ff9900aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:03.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.481+0000 7ff994d6b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff990154670 0x7ff9900aea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40766/0 (socket says 172.21.15.83:40766) 2024-04-17T10:49:03.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.481+0000 7ff994d6b640 1 -- 172.21.15.83:0/3858736197 learned_addr learned my addr 172.21.15.83:0/3858736197 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:03.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.481+0000 7ff994d6b640 1 -- 172.21.15.83:0/3858736197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9900b4520 msgr2=0x7ff99006cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:03.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.481+0000 7ff994d6b640 1 --2- 172.21.15.83:0/3858736197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9900b4520 0x7ff99006cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.481+0000 7ff994d6b640 1 -- 172.21.15.83:0/3858736197 >> v2:172.21.15.167:3300/0 conn(0x7ff9900b4090 msgr2=0x7ff99006c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:03.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.481+0000 7ff98ffff640 1 --2- 172.21.15.83:0/3858736197 >> v2:172.21.15.167:3300/0 conn(0x7ff9900b4090 0x7ff99006c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:03.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.481+0000 7ff994d6b640 1 --2- 172.21.15.83:0/3858736197 >> v2:172.21.15.167:3300/0 conn(0x7ff9900b4090 0x7ff99006c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.481+0000 7ff994d6b640 1 -- 172.21.15.83:0/3858736197 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9900b35b0 con 0x7ff990154670 2024-04-17T10:49:03.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.481+0000 7ff994d6b640 1 --2- 172.21.15.83:0/3858736197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff990154670 0x7ff9900aea50 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7ff9900728a0 tx=0x7ff98001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=847c0810c3fd19b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:03.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.485+0000 7ff98f7fe640 1 -- 172.21.15.83:0/3858736197 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9800a5840 con 0x7ff990154670 2024-04-17T10:49:03.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.485+0000 7ff98f7fe640 1 -- 172.21.15.83:0/3858736197 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9800a59e0 con 0x7ff990154670 2024-04-17T10:49:03.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.485+0000 7ff98f7fe640 1 -- 172.21.15.83:0/3858736197 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9800a5db0 con 0x7ff990154670 2024-04-17T10:49:03.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.485+0000 7ff996fcf640 1 -- 172.21.15.83:0/3858736197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff990154670 msgr2=0x7ff9900aea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:03.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.485+0000 7ff996fcf640 1 --2- 172.21.15.83:0/3858736197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff990154670 0x7ff9900aea50 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7ff9900728a0 tx=0x7ff98001c9c0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.485+0000 7ff996fcf640 1 -- 172.21.15.83:0/3858736197 shutdown_connections 2024-04-17T10:49:03.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.485+0000 7ff996fcf640 1 --2- 172.21.15.83:0/3858736197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9900b4520 0x7ff99006cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.485+0000 7ff996fcf640 1 --2- 172.21.15.83:0/3858736197 >> v2:172.21.15.167:3300/0 conn(0x7ff9900b4090 0x7ff99006c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.485+0000 7ff996fcf640 1 --2- 172.21.15.83:0/3858736197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff990154670 0x7ff9900aea50 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.485+0000 7ff996fcf640 1 -- 172.21.15.83:0/3858736197 >> 172.21.15.83:0/3858736197 conn(0x7ff9900b15b0 msgr2=0x7ff9900b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:03.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.485+0000 7ff996fcf640 1 -- 172.21.15.83:0/3858736197 shutdown_connections 2024-04-17T10:49:03.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.485+0000 7ff996fcf640 1 -- 172.21.15.83:0/3858736197 wait complete. 2024-04-17T10:49:03.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.485+0000 7ff996fcf640 1 Processor -- start 2024-04-17T10:49:03.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.485+0000 7ff996fcf640 1 -- start start 2024-04-17T10:49:03.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.485+0000 7ff996fcf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9900b4090 0x7ff990149550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:03.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.485+0000 7ff996fcf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9900b4520 0x7ff990149a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:03.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.485+0000 7ff996fcf640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff990154670 0x7ff990144a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:03.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.485+0000 7ff996fcf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9900af110 con 0x7ff9900b4520 2024-04-17T10:49:03.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.485+0000 7ff996fcf640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9900aef90 con 0x7ff9900b4090 2024-04-17T10:49:03.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.485+0000 7ff996fcf640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff99006ae10 con 0x7ff990154670 2024-04-17T10:49:03.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.485+0000 7ff98ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9900b4520 0x7ff990149a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:03.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.485+0000 7ff98ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9900b4520 0x7ff990149a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40770/0 (socket says 172.21.15.83:40770) 2024-04-17T10:49:03.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.485+0000 7ff98ffff640 1 -- 172.21.15.83:0/4007613110 learned_addr learned my addr 172.21.15.83:0/4007613110 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:03.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.485+0000 7ff994d6b640 1 --2- 172.21.15.83:0/4007613110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9900b4090 0x7ff990149550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:03.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.485+0000 7ff99556c640 1 --2- 172.21.15.83:0/4007613110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff990154670 0x7ff990144a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:03.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.489+0000 7ff98ffff640 1 -- 172.21.15.83:0/4007613110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff990154670 msgr2=0x7ff990144a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:03.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.489+0000 7ff98ffff640 1 --2- 172.21.15.83:0/4007613110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff990154670 0x7ff990144a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.489+0000 7ff98ffff640 1 -- 172.21.15.83:0/4007613110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9900b4090 msgr2=0x7ff990149550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:03.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.489+0000 7ff98ffff640 1 --2- 172.21.15.83:0/4007613110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9900b4090 0x7ff990149550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.489+0000 7ff98ffff640 1 -- 172.21.15.83:0/4007613110 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9901428b0 con 0x7ff9900b4520 2024-04-17T10:49:03.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.489+0000 7ff98ffff640 1 --2- 172.21.15.83:0/4007613110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9900b4520 0x7ff990149a90 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7ff98400eac0 tx=0x7ff98400ef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:03.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.489+0000 7ff98d7fa640 1 -- 172.21.15.83:0/4007613110 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff984007c10 con 0x7ff9900b4520 2024-04-17T10:49:03.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.489+0000 7ff98d7fa640 1 -- 172.21.15.83:0/4007613110 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff984007db0 con 0x7ff9900b4520 2024-04-17T10:49:03.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.489+0000 7ff98d7fa640 1 -- 172.21.15.83:0/4007613110 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff984004240 con 0x7ff9900b4520 2024-04-17T10:49:03.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.489+0000 7ff996fcf640 1 -- 172.21.15.83:0/4007613110 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff990143020 con 0x7ff9900b4520 2024-04-17T10:49:03.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.489+0000 7ff996fcf640 1 -- 172.21.15.83:0/4007613110 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff990148040 con 0x7ff9900b4520 2024-04-17T10:49:03.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.489+0000 7ff996fcf640 1 -- 172.21.15.83:0/4007613110 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff990074f90 con 0x7ff9900b4520 2024-04-17T10:49:03.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.489+0000 7ff98d7fa640 1 -- 172.21.15.83:0/4007613110 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff98400b030 con 0x7ff9900b4520 2024-04-17T10:49:03.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.489+0000 7ff98d7fa640 1 --2- 172.21.15.83:0/4007613110 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff97c066f60 0x7ff97c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:03.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.489+0000 7ff994d6b640 1 --2- 172.21.15.83:0/4007613110 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff97c066f60 0x7ff97c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:03.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.489+0000 7ff98d7fa640 1 -- 172.21.15.83:0/4007613110 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff9840873d0 con 0x7ff9900b4520 2024-04-17T10:49:03.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.489+0000 7ff994d6b640 1 --2- 172.21.15.83:0/4007613110 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff97c066f60 0x7ff97c069420 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff9800ae000 tx=0x7ff9800a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:03.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.493+0000 7ff98d7fa640 1 -- 172.21.15.83:0/4007613110 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff9840583a0 con 0x7ff9900b4520 2024-04-17T10:49:03.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.653+0000 7ff996fcf640 1 -- 172.21.15.83:0/4007613110 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff9900b2a00 con 0x7ff9900b4520 2024-04-17T10:49:03.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.653+0000 7ff98d7fa640 1 -- 172.21.15.83:0/4007613110 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v116) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff984057ca0 con 0x7ff9900b4520 2024-04-17T10:49:03.651 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:49:03.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.657+0000 7ff996fcf640 1 -- 172.21.15.83:0/4007613110 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff97c066f60 msgr2=0x7ff97c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:03.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.657+0000 7ff996fcf640 1 --2- 172.21.15.83:0/4007613110 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff97c066f60 0x7ff97c069420 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff9800ae000 tx=0x7ff9800a7070 comp rx=0 tx=0).stop 2024-04-17T10:49:03.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.657+0000 7ff996fcf640 1 -- 172.21.15.83:0/4007613110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9900b4520 msgr2=0x7ff990149a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:03.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.657+0000 7ff996fcf640 1 --2- 172.21.15.83:0/4007613110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9900b4520 0x7ff990149a90 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7ff98400eac0 tx=0x7ff98400ef80 comp rx=0 tx=0).stop 2024-04-17T10:49:03.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.657+0000 7ff996fcf640 1 -- 172.21.15.83:0/4007613110 shutdown_connections 2024-04-17T10:49:03.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.657+0000 7ff996fcf640 1 --2- 172.21.15.83:0/4007613110 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff97c066f60 0x7ff97c069420 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.657+0000 7ff996fcf640 1 --2- 172.21.15.83:0/4007613110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff990154670 0x7ff990144a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.657+0000 7ff996fcf640 1 --2- 172.21.15.83:0/4007613110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9900b4520 0x7ff990149a90 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.657+0000 7ff996fcf640 1 --2- 172.21.15.83:0/4007613110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9900b4090 0x7ff990149550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.657+0000 7ff996fcf640 1 -- 172.21.15.83:0/4007613110 >> 172.21.15.83:0/4007613110 conn(0x7ff9900b15b0 msgr2=0x7ff99015b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:03.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.657+0000 7ff996fcf640 1 -- 172.21.15.83:0/4007613110 shutdown_connections 2024-04-17T10:49:03.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.657+0000 7ff996fcf640 1 -- 172.21.15.83:0/4007613110 wait complete. 2024-04-17T10:49:03.670 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:49:03.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f8c22a640 1 Processor -- start 2024-04-17T10:49:03.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f8c22a640 1 -- start start 2024-04-17T10:49:03.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f8c22a640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6f841669f0 0x7f6f84168de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:03.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f8c22a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f841625d0 con 0x7f6f8406ae10 2024-04-17T10:49:03.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f8c22a640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f84161810 con 0x7f6f8415e330 2024-04-17T10:49:03.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f8c22a640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f8415eeb0 con 0x7f6f841669f0 2024-04-17T10:49:03.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f89fc6640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6f8406ae10 0x7f6f8406b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49358/0 (socket says 172.21.15.83:49358) 2024-04-17T10:49:03.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f89fc6640 1 -- 172.21.15.83:0/3392235073 learned_addr learned my addr 172.21.15.83:0/3392235073 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:03.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f88fc4640 1 -- 172.21.15.83:0/3392235073 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2574361676 0 0) 0x7f6f841625d0 con 0x7f6f8406ae10 2024-04-17T10:49:03.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f88fc4640 1 -- 172.21.15.83:0/3392235073 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f70003530 con 0x7f6f8406ae10 2024-04-17T10:49:03.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f8a7c7640 1 --2- 172.21.15.83:0/3392235073 >> v2:172.21.15.167:3300/0 conn(0x7f6f841669f0 0x7f6f84168de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:03.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f88fc4640 1 -- 172.21.15.83:0/3392235073 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2527150907 0 0) 0x7f6f70003530 con 0x7f6f8406ae10 2024-04-17T10:49:03.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f88fc4640 1 -- 172.21.15.83:0/3392235073 >> v2:172.21.15.167:3300/0 conn(0x7f6f841669f0 msgr2=0x7f6f84168de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:03.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f88fc4640 1 --2- 172.21.15.83:0/3392235073 >> v2:172.21.15.167:3300/0 conn(0x7f6f841669f0 0x7f6f84168de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f88fc4640 1 -- 172.21.15.83:0/3392235073 >> v1:172.21.15.167:6789/0 conn(0x7f6f8415e330 legacy=0x7f6f8415e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:03.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f88fc4640 1 -- 172.21.15.83:0/3392235073 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f84161af0 con 0x7f6f8406ae10 2024-04-17T10:49:03.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f88fc4640 1 -- 172.21.15.83:0/3392235073 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f6f74001680 con 0x7f6f8406ae10 2024-04-17T10:49:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f88fc4640 1 -- 172.21.15.83:0/3392235073 >> v1:172.21.15.83:6789/0 conn(0x7f6f8406ae10 legacy=0x7f6f8406b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f88fc4640 1 --2- 172.21.15.83:0/3392235073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f70003a30 0x7f6f70003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f88fc4640 1 --2- 172.21.15.83:0/3392235073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f70004810 0x7f6f70004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f88fc4640 1 --2- 172.21.15.83:0/3392235073 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f700052f0 0x7f6f7000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f88fc4640 1 -- 172.21.15.83:0/3392235073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7000c210 con 0x7f6f70004810 2024-04-17T10:49:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f88fc4640 1 -- 172.21.15.83:0/3392235073 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7000c3c0 con 0x7f6f70003a30 2024-04-17T10:49:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f88fc4640 1 -- 172.21.15.83:0/3392235073 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7000c5c0 con 0x7f6f700052f0 2024-04-17T10:49:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f897c5640 1 --2- 172.21.15.83:0/3392235073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f70004810 0x7f6f70004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f89fc6640 1 --2- 172.21.15.83:0/3392235073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f70003a30 0x7f6f70003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f897c5640 1 -- 172.21.15.83:0/3392235073 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f700052f0 msgr2=0x7f6f7000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f897c5640 1 --2- 172.21.15.83:0/3392235073 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f700052f0 0x7f6f7000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f897c5640 1 -- 172.21.15.83:0/3392235073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f70003a30 msgr2=0x7f6f70003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f897c5640 1 --2- 172.21.15.83:0/3392235073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f70003a30 0x7f6f70003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f897c5640 1 -- 172.21.15.83:0/3392235073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f70003530 con 0x7f6f70004810 2024-04-17T10:49:03.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f8a7c7640 1 -- 172.21.15.83:0/3392235073 reap_dead start 2024-04-17T10:49:03.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.797+0000 7f6f89fc6640 1 --2- 172.21.15.83:0/3392235073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f70003a30 0x7f6f70003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:03.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.801+0000 7f6f897c5640 1 --2- 172.21.15.83:0/3392235073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f70004810 0x7f6f70004c90 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f6f80001750 tx=0x7f6f8001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=543e2f40ac290727 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:03.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.801+0000 7f6f88fc4640 1 -- 172.21.15.83:0/3392235073 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f800013a0 con 0x7f6f70004810 2024-04-17T10:49:03.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.801+0000 7f6f88fc4640 1 -- 172.21.15.83:0/3392235073 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f800052f0 con 0x7f6f70004810 2024-04-17T10:49:03.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.801+0000 7f6f88fc4640 1 -- 172.21.15.83:0/3392235073 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f80003840 con 0x7f6f70004810 2024-04-17T10:49:03.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.801+0000 7f6f8c22a640 1 -- 172.21.15.83:0/3392235073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f70004810 msgr2=0x7f6f70004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.801+0000 7f6f8c22a640 1 --2- 172.21.15.83:0/3392235073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f70004810 0x7f6f70004c90 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f6f80001750 tx=0x7f6f8001c3a0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.801+0000 7f6f8c22a640 1 -- 172.21.15.83:0/3392235073 shutdown_connections 2024-04-17T10:49:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.801+0000 7f6f8c22a640 1 --2- 172.21.15.83:0/3392235073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f70004810 0x7f6f70004c90 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.801+0000 7f6f8c22a640 1 -- 172.21.15.83:0/3392235073 >> 172.21.15.83:0/3392235073 conn(0x7f6f840b15b0 msgr2=0x7f6f840b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.801+0000 7f6f8c22a640 1 -- 172.21.15.83:0/3392235073 shutdown_connections 2024-04-17T10:49:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.801+0000 7f6f8c22a640 1 -- 172.21.15.83:0/3392235073 wait complete. 2024-04-17T10:49:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.801+0000 7f6f8c22a640 1 Processor -- start 2024-04-17T10:49:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.801+0000 7f6f8c22a640 1 -- start start 2024-04-17T10:49:03.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.801+0000 7f6f8c22a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f70004810 0x7f6f841f7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:03.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.801+0000 7f6f8c22a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f84165c90 0x7f6f84158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:03.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.801+0000 7f6f8c22a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f84166110 0x7f6f841f9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:03.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.801+0000 7f6f8c22a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8415ecd0 con 0x7f6f84166110 2024-04-17T10:49:03.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.801+0000 7f6f8c22a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8415eb50 con 0x7f6f70004810 2024-04-17T10:49:03.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.801+0000 7f6f8c22a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8406b600 con 0x7f6f84165c90 2024-04-17T10:49:03.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.805+0000 7f6f8a7c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f84166110 0x7f6f841f9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:03.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.805+0000 7f6f8a7c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f84166110 0x7f6f841f9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40792/0 (socket says 172.21.15.83:40792) 2024-04-17T10:49:03.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.805+0000 7f6f8a7c7640 1 -- 172.21.15.83:0/4260174874 learned_addr learned my addr 172.21.15.83:0/4260174874 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:03.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.805+0000 7f6f89fc6640 1 --2- 172.21.15.83:0/4260174874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f70004810 0x7f6f841f7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:03.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.805+0000 7f6f897c5640 1 --2- 172.21.15.83:0/4260174874 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f84165c90 0x7f6f84158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:03.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.805+0000 7f6f8a7c7640 1 -- 172.21.15.83:0/4260174874 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f84165c90 msgr2=0x7f6f84158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:03.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.805+0000 7f6f8a7c7640 1 --2- 172.21.15.83:0/4260174874 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f84165c90 0x7f6f84158e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.805+0000 7f6f8a7c7640 1 -- 172.21.15.83:0/4260174874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f70004810 msgr2=0x7f6f841f7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:03.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.805+0000 7f6f8a7c7640 1 --2- 172.21.15.83:0/4260174874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f70004810 0x7f6f841f7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.805+0000 7f6f8a7c7640 1 -- 172.21.15.83:0/4260174874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f841f7940 con 0x7f6f84166110 2024-04-17T10:49:03.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.805+0000 7f6f897c5640 1 --2- 172.21.15.83:0/4260174874 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f84165c90 0x7f6f84158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:03.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.805+0000 7f6f8a7c7640 1 --2- 172.21.15.83:0/4260174874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f84166110 0x7f6f841f9e00 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f6f840d06a0 tx=0x7f6f7c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:03.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.805+0000 7f6f7a7fc640 1 -- 172.21.15.83:0/4260174874 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c00ed90 con 0x7f6f84166110 2024-04-17T10:49:03.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.805+0000 7f6f7a7fc640 1 -- 172.21.15.83:0/4260174874 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f7c019070 con 0x7f6f84166110 2024-04-17T10:49:03.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.805+0000 7f6f7a7fc640 1 -- 172.21.15.83:0/4260174874 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c014660 con 0x7f6f84166110 2024-04-17T10:49:03.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.805+0000 7f6f8c22a640 1 -- 172.21.15.83:0/4260174874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f8415be40 con 0x7f6f84166110 2024-04-17T10:49:03.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.805+0000 7f6f8c22a640 1 -- 172.21.15.83:0/4260174874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f8415c400 con 0x7f6f84166110 2024-04-17T10:49:03.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.805+0000 7f6f8c22a640 1 -- 172.21.15.83:0/4260174874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f84074f90 con 0x7f6f84166110 2024-04-17T10:49:03.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.805+0000 7f6f7a7fc640 1 -- 172.21.15.83:0/4260174874 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6f7c014800 con 0x7f6f84166110 2024-04-17T10:49:03.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.805+0000 7f6f7a7fc640 1 --2- 172.21.15.83:0/4260174874 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6f68066f60 0x7f6f68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:03.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.805+0000 7f6f7a7fc640 1 -- 172.21.15.83:0/4260174874 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6f7c085620 con 0x7f6f84166110 2024-04-17T10:49:03.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.805+0000 7f6f89fc6640 1 --2- 172.21.15.83:0/4260174874 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6f68066f60 0x7f6f68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:03.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.805+0000 7f6f89fc6640 1 --2- 172.21.15.83:0/4260174874 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6f68066f60 0x7f6f68069420 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f6f8406a390 tx=0x7f6f74009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:03.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.809+0000 7f6f7a7fc640 1 -- 172.21.15.83:0/4260174874 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6f841f7940 con 0x7f6f84166110 2024-04-17T10:49:03.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.969+0000 7f6f8c22a640 1 -- 172.21.15.83:0/4260174874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6f84071a00 con 0x7f6f84166110 2024-04-17T10:49:03.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.969+0000 7f6f7a7fc640 1 -- 172.21.15.83:0/4260174874 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v116) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6f7c0565f0 con 0x7f6f84166110 2024-04-17T10:49:03.966 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:49:03.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.969+0000 7f6f8c22a640 1 -- 172.21.15.83:0/4260174874 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6f68066f60 msgr2=0x7f6f68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:03.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.969+0000 7f6f8c22a640 1 --2- 172.21.15.83:0/4260174874 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6f68066f60 0x7f6f68069420 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f6f8406a390 tx=0x7f6f74009040 comp rx=0 tx=0).stop 2024-04-17T10:49:03.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.969+0000 7f6f8c22a640 1 -- 172.21.15.83:0/4260174874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f84166110 msgr2=0x7f6f841f9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:03.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.969+0000 7f6f8c22a640 1 --2- 172.21.15.83:0/4260174874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f84166110 0x7f6f841f9e00 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f6f840d06a0 tx=0x7f6f7c00e4b0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.969+0000 7f6f8c22a640 1 -- 172.21.15.83:0/4260174874 shutdown_connections 2024-04-17T10:49:03.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.969+0000 7f6f8c22a640 1 --2- 172.21.15.83:0/4260174874 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6f68066f60 0x7f6f68069420 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.969+0000 7f6f8c22a640 1 --2- 172.21.15.83:0/4260174874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f84166110 0x7f6f841f9e00 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.969+0000 7f6f8c22a640 1 --2- 172.21.15.83:0/4260174874 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f84165c90 0x7f6f84158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.969+0000 7f6f8c22a640 1 --2- 172.21.15.83:0/4260174874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f70004810 0x7f6f841f7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:03.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.969+0000 7f6f8c22a640 1 -- 172.21.15.83:0/4260174874 >> 172.21.15.83:0/4260174874 conn(0x7f6f840b15b0 msgr2=0x7f6f8406d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:03.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.973+0000 7f6f8c22a640 1 -- 172.21.15.83:0/4260174874 shutdown_connections 2024-04-17T10:49:03.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:03.973+0000 7f6f8c22a640 1 -- 172.21.15.83:0/4260174874 wait complete. 2024-04-17T10:49:03.985 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:49:04.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.109+0000 7fc1105e9640 1 Processor -- start 2024-04-17T10:49:04.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.109+0000 7fc1105e9640 1 -- start start 2024-04-17T10:49:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.109+0000 7fc1105e9640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc1080ca770 0x7fc1080c6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.109+0000 7fc1105e9640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc10815df90 con 0x7fc1080cd5b0 2024-04-17T10:49:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.109+0000 7fc1105e9640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1080c52d0 con 0x7fc1080cb5b0 2024-04-17T10:49:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.109+0000 7fc1105e9640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1080c5080 con 0x7fc1080ca770 2024-04-17T10:49:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.109+0000 7fc10e385640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc1080ca770 0x7fc1080c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.109+0000 7fc10e385640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc1080ca770 0x7fc1080c6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40802/0 (socket says 172.21.15.83:40802) 2024-04-17T10:49:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.109+0000 7fc10e385640 1 -- 172.21.15.83:0/3549280963 learned_addr learned my addr 172.21.15.83:0/3549280963 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.109+0000 7fc10eb86640 1 --1- 172.21.15.83:0/3549280963 >> v1:172.21.15.83:6789/0 conn(0x7fc1080cd5b0 0x7fc1080ad840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49364/0 (socket says 172.21.15.83:49364) 2024-04-17T10:49:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.109+0000 7fc10e385640 1 -- 172.21.15.83:0/3549280963 >> v1:172.21.15.167:6789/0 conn(0x7fc1080cb5b0 legacy=0x7fc1080cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.109+0000 7fc10e385640 1 -- 172.21.15.83:0/3549280963 >> v1:172.21.15.83:6789/0 conn(0x7fc1080cd5b0 legacy=0x7fc1080ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:04.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.109+0000 7fc10d383640 1 -- 172.21.15.83:0/3549280963 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3106083802 0 0) 0x7fc10815df90 con 0x7fc1080cd5b0 2024-04-17T10:49:04.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.109+0000 7fc10e385640 1 -- 172.21.15.83:0/3549280963 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1080c55b0 con 0x7fc1080ca770 2024-04-17T10:49:04.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.113+0000 7fc10e385640 1 --2- 172.21.15.83:0/3549280963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1080ca770 0x7fc1080c6090 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fc104009870 tx=0x7fc10401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a1e4101681d032 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:04.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.113+0000 7fc10d383640 1 -- 172.21.15.83:0/3549280963 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1040ab430 con 0x7fc1080ca770 2024-04-17T10:49:04.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.113+0000 7fc10d383640 1 -- 172.21.15.83:0/3549280963 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1040b0070 con 0x7fc1080ca770 2024-04-17T10:49:04.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.113+0000 7fc10d383640 1 -- 172.21.15.83:0/3549280963 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1040ab7f0 con 0x7fc1080ca770 2024-04-17T10:49:04.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.113+0000 7fc1105e9640 1 -- 172.21.15.83:0/3549280963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1080ca770 msgr2=0x7fc1080c6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:04.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.113+0000 7fc1105e9640 1 --2- 172.21.15.83:0/3549280963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1080ca770 0x7fc1080c6090 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fc104009870 tx=0x7fc10401c520 comp rx=0 tx=0).stop 2024-04-17T10:49:04.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.113+0000 7fc1105e9640 1 -- 172.21.15.83:0/3549280963 shutdown_connections 2024-04-17T10:49:04.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.113+0000 7fc1105e9640 1 --2- 172.21.15.83:0/3549280963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1080ca770 0x7fc1080c6090 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:04.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.113+0000 7fc1105e9640 1 -- 172.21.15.83:0/3549280963 >> 172.21.15.83:0/3549280963 conn(0x7fc1080cf9e0 msgr2=0x7fc1080cfdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:04.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.113+0000 7fc1105e9640 1 -- 172.21.15.83:0/3549280963 shutdown_connections 2024-04-17T10:49:04.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.113+0000 7fc1105e9640 1 -- 172.21.15.83:0/3549280963 wait complete. 2024-04-17T10:49:04.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.113+0000 7fc1105e9640 1 Processor -- start 2024-04-17T10:49:04.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.113+0000 7fc1105e9640 1 -- start start 2024-04-17T10:49:04.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.113+0000 7fc1105e9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1080ca770 0x7fc1080c9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:04.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.113+0000 7fc1105e9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1080cb5b0 0x7fc1081546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:04.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.113+0000 7fc1105e9640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1080cd5b0 0x7fc10814eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:04.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.113+0000 7fc1105e9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1080ae0d0 con 0x7fc1080ca770 2024-04-17T10:49:04.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.113+0000 7fc1105e9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1080adf50 con 0x7fc1080cb5b0 2024-04-17T10:49:04.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.113+0000 7fc1105e9640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc10816c440 con 0x7fc1080cd5b0 2024-04-17T10:49:04.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.113+0000 7fc10e385640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1080ca770 0x7fc1080c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:04.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.113+0000 7fc10e385640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1080ca770 0x7fc1080c9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40806/0 (socket says 172.21.15.83:40806) 2024-04-17T10:49:04.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.113+0000 7fc10e385640 1 -- 172.21.15.83:0/588910833 learned_addr learned my addr 172.21.15.83:0/588910833 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:04.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.113+0000 7fc10e385640 1 -- 172.21.15.83:0/588910833 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1080cd5b0 msgr2=0x7fc10814eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:04.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.113+0000 7fc10e385640 1 --2- 172.21.15.83:0/588910833 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1080cd5b0 0x7fc10814eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:04.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.113+0000 7fc10e385640 1 -- 172.21.15.83:0/588910833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1080cb5b0 msgr2=0x7fc1081546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:04.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.113+0000 7fc10e385640 1 --2- 172.21.15.83:0/588910833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1080cb5b0 0x7fc1081546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:04.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.113+0000 7fc10e385640 1 -- 172.21.15.83:0/588910833 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc108072740 con 0x7fc1080ca770 2024-04-17T10:49:04.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.117+0000 7fc10e385640 1 --2- 172.21.15.83:0/588910833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1080ca770 0x7fc1080c9470 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fc1040ad040 tx=0x7fc10401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:04.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.117+0000 7fc0feffd640 1 -- 172.21.15.83:0/588910833 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1040be5f0 con 0x7fc1080ca770 2024-04-17T10:49:04.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.117+0000 7fc0feffd640 1 -- 172.21.15.83:0/588910833 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1040be7b0 con 0x7fc1080ca770 2024-04-17T10:49:04.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.117+0000 7fc1105e9640 1 -- 172.21.15.83:0/588910833 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1080c7fb0 con 0x7fc1080ca770 2024-04-17T10:49:04.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.117+0000 7fc0feffd640 1 -- 172.21.15.83:0/588910833 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1040beba0 con 0x7fc1080ca770 2024-04-17T10:49:04.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.117+0000 7fc1105e9640 1 -- 172.21.15.83:0/588910833 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1080c74d0 con 0x7fc1080ca770 2024-04-17T10:49:04.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.117+0000 7fc1105e9640 1 -- 172.21.15.83:0/588910833 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc108074f90 con 0x7fc1080ca770 2024-04-17T10:49:04.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.117+0000 7fc0feffd640 1 -- 172.21.15.83:0/588910833 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc1040b43f0 con 0x7fc1080ca770 2024-04-17T10:49:04.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.117+0000 7fc0feffd640 1 --2- 172.21.15.83:0/588910833 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc0ec066f60 0x7fc0ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:04.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.117+0000 7fc0feffd640 1 -- 172.21.15.83:0/588910833 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc1040a8070 con 0x7fc1080ca770 2024-04-17T10:49:04.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.117+0000 7fc10db84640 1 --2- 172.21.15.83:0/588910833 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc0ec066f60 0x7fc0ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:04.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.117+0000 7fc10db84640 1 --2- 172.21.15.83:0/588910833 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc0ec066f60 0x7fc0ec069420 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fc1081fadd0 tx=0x7fc0f80079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:04.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.121+0000 7fc0feffd640 1 -- 172.21.15.83:0/588910833 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc108072740 con 0x7fc1080ca770 2024-04-17T10:49:04.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.293+0000 7fc1105e9640 1 -- 172.21.15.83:0/588910833 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc10815dbe0 con 0x7fc1080ca770 2024-04-17T10:49:04.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.293+0000 7fc0feffd640 1 -- 172.21.15.83:0/588910833 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v116) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc104016110 con 0x7fc1080ca770 2024-04-17T10:49:04.291 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-04-17T10:49:04.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.293+0000 7fc1105e9640 1 -- 172.21.15.83:0/588910833 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc0ec066f60 msgr2=0x7fc0ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:04.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.293+0000 7fc1105e9640 1 --2- 172.21.15.83:0/588910833 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc0ec066f60 0x7fc0ec069420 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fc1081fadd0 tx=0x7fc0f80079c0 comp rx=0 tx=0).stop 2024-04-17T10:49:04.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.293+0000 7fc1105e9640 1 -- 172.21.15.83:0/588910833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1080ca770 msgr2=0x7fc1080c9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:04.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.293+0000 7fc1105e9640 1 --2- 172.21.15.83:0/588910833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1080ca770 0x7fc1080c9470 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fc1040ad040 tx=0x7fc10401ad60 comp rx=0 tx=0).stop 2024-04-17T10:49:04.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.293+0000 7fc1105e9640 1 -- 172.21.15.83:0/588910833 shutdown_connections 2024-04-17T10:49:04.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.293+0000 7fc1105e9640 1 --2- 172.21.15.83:0/588910833 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc0ec066f60 0x7fc0ec069420 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:04.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.293+0000 7fc1105e9640 1 --2- 172.21.15.83:0/588910833 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1080cd5b0 0x7fc10814eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:04.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.293+0000 7fc1105e9640 1 --2- 172.21.15.83:0/588910833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1080cb5b0 0x7fc1081546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:04.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.293+0000 7fc1105e9640 1 --2- 172.21.15.83:0/588910833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1080ca770 0x7fc1080c9470 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:04.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.293+0000 7fc1105e9640 1 -- 172.21.15.83:0/588910833 >> 172.21.15.83:0/588910833 conn(0x7fc1080cf9e0 msgr2=0x7fc1080cccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:04.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.297+0000 7fc1105e9640 1 -- 172.21.15.83:0/588910833 shutdown_connections 2024-04-17T10:49:04.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.297+0000 7fc1105e9640 1 -- 172.21.15.83:0/588910833 wait complete. 2024-04-17T10:49:04.309 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2024-04-17T10:49:04.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:04.309 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-17T10:49:04.320 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[76725]: fuse finished with error 0 and tester_r 0 2024-04-17T10:49:04.429 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-17T10:49:04.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:04.430 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T10:49:04.487 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:49:04.487 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:49:04.493 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:49:04.493 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-04-17T10:49:04.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:04.494 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-17T10:49:04.542 INFO:teuthology.orchestra.run.smithi167.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-17T10:49:04.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:49:04.543 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:49:04.543 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:49:04.548 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16063 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:49:04.549 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-17T10:49:04.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:04.549 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-17T10:49:04.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:49:04.596 INFO:teuthology.orchestra.run.smithi083.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-17T10:49:04.596 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:49:04.596 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:49:04.603 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:49:04.603 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-04-17T10:49:04.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:04.603 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-17T10:49:04.650 INFO:teuthology.orchestra.run.smithi167.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-17T10:49:04.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:49:04.651 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:49:04.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.781+0000 7f85fbf94640 1 Processor -- start 2024-04-17T10:49:04.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.781+0000 7f85fbf94640 1 -- start start 2024-04-17T10:49:04.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.785+0000 7f85fbf94640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f85f40b9820 0x7f85f415eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:04.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.785+0000 7f85fbf94640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f85f40d0fc0 0x7f85f40d1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:04.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.785+0000 7f85fbf94640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85f415f040 con 0x7f85f4161670 2024-04-17T10:49:04.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.785+0000 7f85fbf94640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85f40cee30 con 0x7f85f40d0fc0 2024-04-17T10:49:04.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.785+0000 7f85fbf94640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85f4161b30 con 0x7f85f40b9820 2024-04-17T10:49:04.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.785+0000 7f85fa531640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f85f40d0fc0 0x7f85f40d1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:04.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.785+0000 7f85f952f640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f85f4161670 0x7f85f40d08b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49368/0 (socket says 172.21.15.83:49368) 2024-04-17T10:49:04.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.785+0000 7f85f952f640 1 -- 172.21.15.83:0/2609866983 learned_addr learned my addr 172.21.15.83:0/2609866983 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:04.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.785+0000 7f85fa531640 1 -- 172.21.15.83:0/2609866983 >> v2:172.21.15.167:3300/0 conn(0x7f85f40b9820 msgr2=0x7f85f415eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:04.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.785+0000 7f85fa531640 1 --2- 172.21.15.83:0/2609866983 >> v2:172.21.15.167:3300/0 conn(0x7f85f40b9820 0x7f85f415eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:04.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.785+0000 7f85fa531640 1 -- 172.21.15.83:0/2609866983 >> v1:172.21.15.83:6789/0 conn(0x7f85f4161670 legacy=0x7f85f40d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:04.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.785+0000 7f85f8d2e640 1 -- 172.21.15.83:0/2609866983 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2754074380 0 0) 0x7f85e4001120 con 0x7f85f4161670 2024-04-17T10:49:04.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.785+0000 7f85fa531640 1 -- 172.21.15.83:0/2609866983 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f4161e10 con 0x7f85f40d0fc0 2024-04-17T10:49:04.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.785+0000 7f85fa531640 1 --2- 172.21.15.83:0/2609866983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85f40d0fc0 0x7f85f40d1e70 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f85f408c5e0 tx=0x7f85ec01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c7318541eebee8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:04.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.785+0000 7f85f8d2e640 1 -- 172.21.15.83:0/2609866983 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec0b2070 con 0x7f85f40d0fc0 2024-04-17T10:49:04.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.785+0000 7f85f8d2e640 1 -- 172.21.15.83:0/2609866983 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ec0a5850 con 0x7f85f40d0fc0 2024-04-17T10:49:04.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.785+0000 7f85f8d2e640 1 -- 172.21.15.83:0/2609866983 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec0a5c20 con 0x7f85f40d0fc0 2024-04-17T10:49:04.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.789+0000 7f85fbf94640 1 -- 172.21.15.83:0/2609866983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85f40d0fc0 msgr2=0x7f85f40d1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:04.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.789+0000 7f85fbf94640 1 --2- 172.21.15.83:0/2609866983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85f40d0fc0 0x7f85f40d1e70 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f85f408c5e0 tx=0x7f85ec01c8a0 comp rx=0 tx=0).stop 2024-04-17T10:49:04.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.789+0000 7f85fbf94640 1 -- 172.21.15.83:0/2609866983 shutdown_connections 2024-04-17T10:49:04.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.789+0000 7f85fbf94640 1 --2- 172.21.15.83:0/2609866983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85f40d0fc0 0x7f85f40d1e70 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:04.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.789+0000 7f85fbf94640 1 --2- 172.21.15.83:0/2609866983 >> v2:172.21.15.167:3300/0 conn(0x7f85f40b9820 0x7f85f415eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:04.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.789+0000 7f85fbf94640 1 -- 172.21.15.83:0/2609866983 >> 172.21.15.83:0/2609866983 conn(0x7f85f406b280 msgr2=0x7f85f406b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:04.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.789+0000 7f85fbf94640 1 -- 172.21.15.83:0/2609866983 shutdown_connections 2024-04-17T10:49:04.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.789+0000 7f85fbf94640 1 -- 172.21.15.83:0/2609866983 wait complete. 2024-04-17T10:49:04.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.789+0000 7f85fbf94640 1 Processor -- start 2024-04-17T10:49:04.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.789+0000 7f85fbf94640 1 -- start start 2024-04-17T10:49:04.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.789+0000 7f85fbf94640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85f40b9820 0x7f85f40b7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:04.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.789+0000 7f85fbf94640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85f40d0fc0 0x7f85f4152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:04.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.789+0000 7f85fbf94640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85f4161670 0x7f85f4152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:04.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.789+0000 7f85fbf94640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f40d2fd0 con 0x7f85f4161670 2024-04-17T10:49:04.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.789+0000 7f85fbf94640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f40d2e50 con 0x7f85f40d0fc0 2024-04-17T10:49:04.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.789+0000 7f85fbf94640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f40d3150 con 0x7f85f40b9820 2024-04-17T10:49:04.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.789+0000 7f85fa531640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85f4161670 0x7f85f4152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:04.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.789+0000 7f85fa531640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85f4161670 0x7f85f4152f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40834/0 (socket says 172.21.15.83:40834) 2024-04-17T10:49:04.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.789+0000 7f85fa531640 1 -- 172.21.15.83:0/2286390760 learned_addr learned my addr 172.21.15.83:0/2286390760 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:04.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.789+0000 7f85fa531640 1 -- 172.21.15.83:0/2286390760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85f40b9820 msgr2=0x7f85f40b7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:04.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.789+0000 7f85fa531640 1 --2- 172.21.15.83:0/2286390760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85f40b9820 0x7f85f40b7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:04.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.789+0000 7f85fa531640 1 -- 172.21.15.83:0/2286390760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85f40d0fc0 msgr2=0x7f85f4152a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:04.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.789+0000 7f85fa531640 1 --2- 172.21.15.83:0/2286390760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85f40d0fc0 0x7f85f4152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:04.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.789+0000 7f85fa531640 1 -- 172.21.15.83:0/2286390760 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f4153480 con 0x7f85f4161670 2024-04-17T10:49:04.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.789+0000 7f85fa531640 1 --2- 172.21.15.83:0/2286390760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85f4161670 0x7f85f4152f40 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f85ec0b0040 tx=0x7f85ec019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:04.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.789+0000 7f85ea7fc640 1 -- 172.21.15.83:0/2286390760 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec0bd020 con 0x7f85f4161670 2024-04-17T10:49:04.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.789+0000 7f85ea7fc640 1 -- 172.21.15.83:0/2286390760 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ec0b8670 con 0x7f85f4161670 2024-04-17T10:49:04.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.789+0000 7f85fbf94640 1 -- 172.21.15.83:0/2286390760 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f4153730 con 0x7f85f4161670 2024-04-17T10:49:04.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.789+0000 7f85ea7fc640 1 -- 172.21.15.83:0/2286390760 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec0b2040 con 0x7f85f4161670 2024-04-17T10:49:04.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.789+0000 7f85fbf94640 1 -- 172.21.15.83:0/2286390760 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f4153cf0 con 0x7f85f4161670 2024-04-17T10:49:04.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.793+0000 7f85fbf94640 1 -- 172.21.15.83:0/2286390760 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85f4074f90 con 0x7f85f4161670 2024-04-17T10:49:04.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.793+0000 7f85ea7fc640 1 -- 172.21.15.83:0/2286390760 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f85ec0c0020 con 0x7f85f4161670 2024-04-17T10:49:04.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.793+0000 7f85ea7fc640 1 --2- 172.21.15.83:0/2286390760 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f85d8066f90 0x7f85d8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:04.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.793+0000 7f85ea7fc640 1 -- 172.21.15.83:0/2286390760 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f85ec11fb20 con 0x7f85f4161670 2024-04-17T10:49:04.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.793+0000 7f85f9d30640 1 --2- 172.21.15.83:0/2286390760 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f85d8066f90 0x7f85d8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:04.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.793+0000 7f85f9d30640 1 --2- 172.21.15.83:0/2286390760 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f85d8066f90 0x7f85d8069450 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f85f0009690 tx=0x7f85f0006c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:04.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.797+0000 7f85ea7fc640 1 -- 172.21.15.83:0/2286390760 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f85ec0c8020 con 0x7f85f4161670 2024-04-17T10:49:04.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.993+0000 7f85fbf94640 1 -- 172.21.15.83:0/2286390760 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85f4160070 con 0x7f85f4161670 2024-04-17T10:49:04.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.997+0000 7f85ea7fc640 1 -- 172.21.15.83:0/2286390760 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f85ec016520 con 0x7f85f4161670 2024-04-17T10:49:04.993 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:04.993 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5786,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1553533838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1553533838},{"type":"v1","addr":"172.21.15.167:6833","nonce":1553533838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/674536913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":674536913},{"type":"v1","addr":"172.21.15.167:6835","nonce":674536913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8598,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4183890457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4183890457},{"type":"v1","addr":"172.21.15.83:6835","nonce":4183890457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:48:24.738443+0000","modified":"2024-04-17T10:48:26.776273+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8604":{"gid":8604,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1479417998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1479417998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1479417998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-17T10:49:04.993 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 69 2024-04-17T10:49:04.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.997+0000 7f85fbf94640 1 -- 172.21.15.83:0/2286390760 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f85d8066f90 msgr2=0x7f85d8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:04.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.997+0000 7f85fbf94640 1 --2- 172.21.15.83:0/2286390760 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f85d8066f90 0x7f85d8069450 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f85f0009690 tx=0x7f85f0006c00 comp rx=0 tx=0).stop 2024-04-17T10:49:04.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.997+0000 7f85fbf94640 1 -- 172.21.15.83:0/2286390760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85f4161670 msgr2=0x7f85f4152f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:04.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.997+0000 7f85fbf94640 1 --2- 172.21.15.83:0/2286390760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85f4161670 0x7f85f4152f40 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f85ec0b0040 tx=0x7f85ec019640 comp rx=0 tx=0).stop 2024-04-17T10:49:04.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.997+0000 7f85fbf94640 1 -- 172.21.15.83:0/2286390760 shutdown_connections 2024-04-17T10:49:04.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.997+0000 7f85fbf94640 1 --2- 172.21.15.83:0/2286390760 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f85d8066f90 0x7f85d8069450 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:04.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.997+0000 7f85fbf94640 1 --2- 172.21.15.83:0/2286390760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85f4161670 0x7f85f4152f40 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:04.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.997+0000 7f85fbf94640 1 --2- 172.21.15.83:0/2286390760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85f40d0fc0 0x7f85f4152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:04.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.997+0000 7f85fbf94640 1 --2- 172.21.15.83:0/2286390760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85f40b9820 0x7f85f40b7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:04.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.997+0000 7f85fbf94640 1 -- 172.21.15.83:0/2286390760 >> 172.21.15.83:0/2286390760 conn(0x7f85f406b280 msgr2=0x7f85f415f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:04.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.997+0000 7f85fbf94640 1 -- 172.21.15.83:0/2286390760 shutdown_connections 2024-04-17T10:49:04.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:04.997+0000 7f85fbf94640 1 -- 172.21.15.83:0/2286390760 wait complete. 2024-04-17T10:49:05.012 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:49:05.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.129+0000 7ff4236a5640 1 Processor -- start 2024-04-17T10:49:05.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.129+0000 7ff4236a5640 1 -- start start 2024-04-17T10:49:05.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff4236a5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff41c15ba30 0x7ff41c072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:05.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff4236a5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff41c0cfbe0 0x7ff41c06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:05.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff4236a5640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff41c06e320 con 0x7ff41c15c960 2024-04-17T10:49:05.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff4236a5640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff41c071ea0 con 0x7ff41c0cfbe0 2024-04-17T10:49:05.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff4236a5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c06e040 con 0x7ff41c15ba30 2024-04-17T10:49:05.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff420c40640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff41c15c960 0x7ff41c0cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49380/0 (socket says 172.21.15.83:49380) 2024-04-17T10:49:05.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff420c40640 1 -- 172.21.15.83:0/2122107741 learned_addr learned my addr 172.21.15.83:0/2122107741 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:05.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff421c42640 1 --2- 172.21.15.83:0/2122107741 >> v2:172.21.15.167:3300/0 conn(0x7ff41c0cfbe0 0x7ff41c06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:05.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff421441640 1 --2- 172.21.15.83:0/2122107741 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff41c15ba30 0x7ff41c072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:05.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff413fff640 1 -- 172.21.15.83:0/2122107741 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1497096335 0 0) 0x7ff41c06e320 con 0x7ff41c15c960 2024-04-17T10:49:05.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff413fff640 1 -- 172.21.15.83:0/2122107741 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff408003530 con 0x7ff41c15c960 2024-04-17T10:49:05.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff413fff640 1 -- 172.21.15.83:0/2122107741 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2760870118 0 0) 0x7ff408003530 con 0x7ff41c15c960 2024-04-17T10:49:05.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff413fff640 1 -- 172.21.15.83:0/2122107741 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff41c15ba30 msgr2=0x7ff41c072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:05.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff413fff640 1 --2- 172.21.15.83:0/2122107741 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff41c15ba30 0x7ff41c072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:05.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff413fff640 1 -- 172.21.15.83:0/2122107741 >> v2:172.21.15.167:3300/0 conn(0x7ff41c0cfbe0 msgr2=0x7ff41c06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff413fff640 1 --2- 172.21.15.83:0/2122107741 >> v2:172.21.15.167:3300/0 conn(0x7ff41c0cfbe0 0x7ff41c06c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff413fff640 1 -- 172.21.15.83:0/2122107741 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff41c071850 con 0x7ff41c15c960 2024-04-17T10:49:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff413fff640 1 -- 172.21.15.83:0/2122107741 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7ff40c0016a0 con 0x7ff41c15c960 2024-04-17T10:49:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff413fff640 1 -- 172.21.15.83:0/2122107741 >> v1:172.21.15.83:6789/0 conn(0x7ff41c15c960 legacy=0x7ff41c0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff413fff640 1 --2- 172.21.15.83:0/2122107741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4080039c0 0x7ff408003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff413fff640 1 --2- 172.21.15.83:0/2122107741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff408004810 0x7ff408004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff413fff640 1 --2- 172.21.15.83:0/2122107741 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff41c15ba30 0x7ff4080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff413fff640 1 -- 172.21.15.83:0/2122107741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c071ea0 con 0x7ff4080039c0 2024-04-17T10:49:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff413fff640 1 -- 172.21.15.83:0/2122107741 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c06e040 con 0x7ff408004810 2024-04-17T10:49:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff413fff640 1 -- 172.21.15.83:0/2122107741 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff40800ba20 con 0x7ff41c15ba30 2024-04-17T10:49:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff421441640 1 --2- 172.21.15.83:0/2122107741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4080039c0 0x7ff408003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff420c40640 1 --2- 172.21.15.83:0/2122107741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff408004810 0x7ff408004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff421c42640 1 --2- 172.21.15.83:0/2122107741 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff41c15ba30 0x7ff4080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff421441640 1 -- 172.21.15.83:0/2122107741 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff41c15ba30 msgr2=0x7ff4080051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:05.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff421441640 1 --2- 172.21.15.83:0/2122107741 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff41c15ba30 0x7ff4080051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:05.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff421441640 1 -- 172.21.15.83:0/2122107741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff408004810 msgr2=0x7ff408004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:05.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff421441640 1 --2- 172.21.15.83:0/2122107741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff408004810 0x7ff408004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:05.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.133+0000 7ff421441640 1 -- 172.21.15.83:0/2122107741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff408003530 con 0x7ff4080039c0 2024-04-17T10:49:05.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff421441640 1 --2- 172.21.15.83:0/2122107741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4080039c0 0x7ff408003de0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7ff418009b10 tx=0x7ff41801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d092e44b7562563 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:05.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff413fff640 1 -- 172.21.15.83:0/2122107741 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4180096b0 con 0x7ff4080039c0 2024-04-17T10:49:05.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff413fff640 1 -- 172.21.15.83:0/2122107741 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4180a9470 con 0x7ff4080039c0 2024-04-17T10:49:05.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff413fff640 1 -- 172.21.15.83:0/2122107741 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4180a9860 con 0x7ff4080039c0 2024-04-17T10:49:05.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff4236a5640 1 -- 172.21.15.83:0/2122107741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4080039c0 msgr2=0x7ff408003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:05.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff4236a5640 1 --2- 172.21.15.83:0/2122107741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4080039c0 0x7ff408003de0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7ff418009b10 tx=0x7ff41801ca70 comp rx=0 tx=0).stop 2024-04-17T10:49:05.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff421c42640 1 -- 172.21.15.83:0/2122107741 reap_dead start 2024-04-17T10:49:05.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff4236a5640 1 -- 172.21.15.83:0/2122107741 shutdown_connections 2024-04-17T10:49:05.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff4236a5640 1 -- 172.21.15.83:0/2122107741 >> 172.21.15.83:0/2122107741 conn(0x7ff41c06abe0 msgr2=0x7ff41c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:05.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff4236a5640 1 -- 172.21.15.83:0/2122107741 shutdown_connections 2024-04-17T10:49:05.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff4236a5640 1 -- 172.21.15.83:0/2122107741 wait complete. 2024-04-17T10:49:05.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff4236a5640 1 Processor -- start 2024-04-17T10:49:05.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff4236a5640 1 -- start start 2024-04-17T10:49:05.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff4236a5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4080039c0 0x7ff41c1f8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:05.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff4236a5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff41c0ce4f0 0x7ff41c159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:05.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff4236a5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff41c14bc70 0x7ff41c15a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:05.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff4236a5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c15cee0 con 0x7ff4080039c0 2024-04-17T10:49:05.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff4236a5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c15cd60 con 0x7ff41c14bc70 2024-04-17T10:49:05.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff4236a5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c166510 con 0x7ff41c0ce4f0 2024-04-17T10:49:05.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff421441640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4080039c0 0x7ff41c1f8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:05.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff421441640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4080039c0 0x7ff41c1f8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40860/0 (socket says 172.21.15.83:40860) 2024-04-17T10:49:05.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff421441640 1 -- 172.21.15.83:0/3324326027 learned_addr learned my addr 172.21.15.83:0/3324326027 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:05.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff421c42640 1 --2- 172.21.15.83:0/3324326027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff41c14bc70 0x7ff41c15a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:05.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff421441640 1 -- 172.21.15.83:0/3324326027 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff41c0ce4f0 msgr2=0x7ff41c159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:05.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff420c40640 1 --2- 172.21.15.83:0/3324326027 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff41c0ce4f0 0x7ff41c159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:05.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff421441640 1 --2- 172.21.15.83:0/3324326027 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff41c0ce4f0 0x7ff41c159f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:05.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff421441640 1 -- 172.21.15.83:0/3324326027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff41c14bc70 msgr2=0x7ff41c15a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:05.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff421441640 1 --2- 172.21.15.83:0/3324326027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff41c14bc70 0x7ff41c15a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:05.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff421441640 1 -- 172.21.15.83:0/3324326027 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff41c1f8f90 con 0x7ff4080039c0 2024-04-17T10:49:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff421c42640 1 --2- 172.21.15.83:0/3324326027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff41c14bc70 0x7ff41c15a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff420c40640 1 --2- 172.21.15.83:0/3324326027 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff41c0ce4f0 0x7ff41c159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff421441640 1 --2- 172.21.15.83:0/3324326027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4080039c0 0x7ff41c1f8a50 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7ff4180ab040 tx=0x7ff418019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff411ffb640 1 -- 172.21.15.83:0/3324326027 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4180bf020 con 0x7ff4080039c0 2024-04-17T10:49:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff411ffb640 1 -- 172.21.15.83:0/3324326027 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4180a9c40 con 0x7ff4080039c0 2024-04-17T10:49:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff4236a5640 1 -- 172.21.15.83:0/3324326027 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff41c0ce9b0 con 0x7ff4080039c0 2024-04-17T10:49:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.137+0000 7ff411ffb640 1 -- 172.21.15.83:0/3324326027 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff418004020 con 0x7ff4080039c0 2024-04-17T10:49:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.141+0000 7ff4236a5640 1 -- 172.21.15.83:0/3324326027 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff41c155480 con 0x7ff4080039c0 2024-04-17T10:49:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.141+0000 7ff4236a5640 1 -- 172.21.15.83:0/3324326027 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff41c0cec90 con 0x7ff4080039c0 2024-04-17T10:49:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.141+0000 7ff411ffb640 1 -- 172.21.15.83:0/3324326027 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff4180c2070 con 0x7ff4080039c0 2024-04-17T10:49:05.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.141+0000 7ff411ffb640 1 --2- 172.21.15.83:0/3324326027 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff400066f60 0x7ff400069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:05.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.141+0000 7ff411ffb640 1 -- 172.21.15.83:0/3324326027 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff41811f480 con 0x7ff4080039c0 2024-04-17T10:49:05.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.141+0000 7ff420c40640 1 --2- 172.21.15.83:0/3324326027 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff400066f60 0x7ff400069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:05.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.141+0000 7ff420c40640 1 --2- 172.21.15.83:0/3324326027 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff400066f60 0x7ff400069420 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff40c001ec0 tx=0x7ff40c000fa0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:05.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.145+0000 7ff411ffb640 1 -- 172.21.15.83:0/3324326027 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff4180c8020 con 0x7ff4080039c0 2024-04-17T10:49:05.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.305+0000 7ff4236a5640 1 -- 172.21.15.83:0/3324326027 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff41c074f90 con 0x7ff4080039c0 2024-04-17T10:49:05.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.309+0000 7ff411ffb640 1 -- 172.21.15.83:0/3324326027 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+16649 (secure 0 0 0) 0x7ff41c1f8f90 con 0x7ff4080039c0 2024-04-17T10:49:05.305 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:05.305 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":116,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:48:55.454382+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:48:22.981915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-17T10:48:23.700138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.18","mappings":[{"from":0,"to":5}]},{"pgid":"17.34","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:49:05.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.309+0000 7ff4236a5640 1 -- 172.21.15.83:0/3324326027 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff400066f60 msgr2=0x7ff400069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:05.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.309+0000 7ff4236a5640 1 --2- 172.21.15.83:0/3324326027 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff400066f60 0x7ff400069420 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff40c001ec0 tx=0x7ff40c000fa0 comp rx=0 tx=0).stop 2024-04-17T10:49:05.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.309+0000 7ff4236a5640 1 -- 172.21.15.83:0/3324326027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4080039c0 msgr2=0x7ff41c1f8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:05.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.309+0000 7ff4236a5640 1 --2- 172.21.15.83:0/3324326027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4080039c0 0x7ff41c1f8a50 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7ff4180ab040 tx=0x7ff418019810 comp rx=0 tx=0).stop 2024-04-17T10:49:05.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.309+0000 7ff4236a5640 1 -- 172.21.15.83:0/3324326027 shutdown_connections 2024-04-17T10:49:05.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.309+0000 7ff4236a5640 1 --2- 172.21.15.83:0/3324326027 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff400066f60 0x7ff400069420 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:05.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.309+0000 7ff4236a5640 1 --2- 172.21.15.83:0/3324326027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff41c14bc70 0x7ff41c15a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:05.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.309+0000 7ff4236a5640 1 --2- 172.21.15.83:0/3324326027 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff41c0ce4f0 0x7ff41c159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:05.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.309+0000 7ff4236a5640 1 --2- 172.21.15.83:0/3324326027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4080039c0 0x7ff41c1f8a50 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:05.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.309+0000 7ff4236a5640 1 -- 172.21.15.83:0/3324326027 >> 172.21.15.83:0/3324326027 conn(0x7ff41c06abe0 msgr2=0x7ff41c070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:05.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.309+0000 7ff4236a5640 1 -- 172.21.15.83:0/3324326027 shutdown_connections 2024-04-17T10:49:05.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.309+0000 7ff4236a5640 1 -- 172.21.15.83:0/3324326027 wait complete. 2024-04-17T10:49:05.326 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:48:22.981915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:48:23.700138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:49:05.326 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:49:05.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.453+0000 7fbc95d68640 1 Processor -- start 2024-04-17T10:49:05.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.453+0000 7fbc95d68640 1 -- start start 2024-04-17T10:49:05.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.453+0000 7fbc95d68640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbc9015aa10 0x7fbc900729d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:05.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.453+0000 7fbc95d68640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbc900cbca0 0x7fbc9006bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:05.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.453+0000 7fbc95d68640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc900cc510 con 0x7fbc9006c130 2024-04-17T10:49:05.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.453+0000 7fbc95d68640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc90072f10 con 0x7fbc900cbca0 2024-04-17T10:49:05.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.453+0000 7fbc95d68640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc9006c560 con 0x7fbc9015aa10 2024-04-17T10:49:05.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.453+0000 7fbc87fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbc900cbca0 0x7fbc9006bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:05.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.453+0000 7fbc8ffff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbc9006c130 0x7fbc9006db70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49386/0 (socket says 172.21.15.83:49386) 2024-04-17T10:49:05.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.453+0000 7fbc8ffff640 1 -- 172.21.15.83:0/1349177511 learned_addr learned my addr 172.21.15.83:0/1349177511 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:05.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.453+0000 7fbc8f7fe640 1 --2- 172.21.15.83:0/1349177511 >> v2:172.21.15.167:3300/0 conn(0x7fbc9015aa10 0x7fbc900729d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:05.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.453+0000 7fbc87fff640 1 -- 172.21.15.83:0/1349177511 >> v2:172.21.15.167:3300/0 conn(0x7fbc9015aa10 msgr2=0x7fbc900729d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:05.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.453+0000 7fbc87fff640 1 --2- 172.21.15.83:0/1349177511 >> v2:172.21.15.167:3300/0 conn(0x7fbc9015aa10 0x7fbc900729d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:05.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.453+0000 7fbc87fff640 1 -- 172.21.15.83:0/1349177511 >> v1:172.21.15.83:6789/0 conn(0x7fbc9006c130 legacy=0x7fbc9006db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:05.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.453+0000 7fbc8effd640 1 -- 172.21.15.83:0/1349177511 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 790176083 0 0) 0x7fbc900cc510 con 0x7fbc9006c130 2024-04-17T10:49:05.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.453+0000 7fbc87fff640 1 -- 172.21.15.83:0/1349177511 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc9015e740 con 0x7fbc900cbca0 2024-04-17T10:49:05.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.453+0000 7fbc8f7fe640 1 --2- 172.21.15.83:0/1349177511 >> v2:172.21.15.167:3300/0 conn(0x7fbc9015aa10 0x7fbc900729d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:05.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc87fff640 1 --2- 172.21.15.83:0/1349177511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc900cbca0 0x7fbc9006bbf0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fbc900cfe70 tx=0x7fbc7801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2be700efe67cc0e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:05.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc8effd640 1 -- 172.21.15.83:0/1349177511 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc780b1070 con 0x7fbc900cbca0 2024-04-17T10:49:05.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc8effd640 1 -- 172.21.15.83:0/1349177511 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc780ac600 con 0x7fbc900cbca0 2024-04-17T10:49:05.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc8effd640 1 -- 172.21.15.83:0/1349177511 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc780ac9d0 con 0x7fbc900cbca0 2024-04-17T10:49:05.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc95d68640 1 -- 172.21.15.83:0/1349177511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc900cbca0 msgr2=0x7fbc9006bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:05.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc95d68640 1 --2- 172.21.15.83:0/1349177511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc900cbca0 0x7fbc9006bbf0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fbc900cfe70 tx=0x7fbc7801c3a0 comp rx=0 tx=0).stop 2024-04-17T10:49:05.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc95d68640 1 -- 172.21.15.83:0/1349177511 shutdown_connections 2024-04-17T10:49:05.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc95d68640 1 --2- 172.21.15.83:0/1349177511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc900cbca0 0x7fbc9006bbf0 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:05.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc95d68640 1 --2- 172.21.15.83:0/1349177511 >> v2:172.21.15.167:3300/0 conn(0x7fbc9015aa10 0x7fbc900729d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:05.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc95d68640 1 -- 172.21.15.83:0/1349177511 >> 172.21.15.83:0/1349177511 conn(0x7fbc900cc990 msgr2=0x7fbc900ccda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:05.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc95d68640 1 -- 172.21.15.83:0/1349177511 shutdown_connections 2024-04-17T10:49:05.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc95d68640 1 -- 172.21.15.83:0/1349177511 wait complete. 2024-04-17T10:49:05.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc95d68640 1 Processor -- start 2024-04-17T10:49:05.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc95d68640 1 -- start start 2024-04-17T10:49:05.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc95d68640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc9006c130 0x7fbc90153df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:05.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc95d68640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc900cbca0 0x7fbc9015b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:05.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc95d68640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc9015aa10 0x7fbc9015b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:05.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc95d68640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9015adf0 con 0x7fbc9006c130 2024-04-17T10:49:05.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc95d68640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9015f2c0 con 0x7fbc9015aa10 2024-04-17T10:49:05.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc95d68640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9015af70 con 0x7fbc900cbca0 2024-04-17T10:49:05.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc8f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc9006c130 0x7fbc90153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:05.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc8f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc9006c130 0x7fbc90153df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40882/0 (socket says 172.21.15.83:40882) 2024-04-17T10:49:05.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc8f7fe640 1 -- 172.21.15.83:0/1677891932 learned_addr learned my addr 172.21.15.83:0/1677891932 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:05.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc8f7fe640 1 -- 172.21.15.83:0/1677891932 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc900cbca0 msgr2=0x7fbc9015b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:05.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc8f7fe640 1 --2- 172.21.15.83:0/1677891932 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc900cbca0 0x7fbc9015b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:05.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc8f7fe640 1 -- 172.21.15.83:0/1677891932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc9015aa10 msgr2=0x7fbc9015b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:05.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc8f7fe640 1 --2- 172.21.15.83:0/1677891932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc9015aa10 0x7fbc9015b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:05.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc8f7fe640 1 -- 172.21.15.83:0/1677891932 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc90159680 con 0x7fbc9006c130 2024-04-17T10:49:05.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc8f7fe640 1 --2- 172.21.15.83:0/1677891932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc9006c130 0x7fbc90153df0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fbc8000c820 tx=0x7fbc8000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:05.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc8cff9640 1 -- 172.21.15.83:0/1677891932 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc80015070 con 0x7fbc9006c130 2024-04-17T10:49:05.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc8cff9640 1 -- 172.21.15.83:0/1677891932 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc80007a30 con 0x7fbc9006c130 2024-04-17T10:49:05.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc95d68640 1 -- 172.21.15.83:0/1677891932 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc90159960 con 0x7fbc9006c130 2024-04-17T10:49:05.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc8cff9640 1 -- 172.21.15.83:0/1677891932 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc80007e00 con 0x7fbc9006c130 2024-04-17T10:49:05.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.457+0000 7fbc95d68640 1 -- 172.21.15.83:0/1677891932 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc90159f20 con 0x7fbc9006c130 2024-04-17T10:49:05.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.461+0000 7fbc95d68640 1 -- 172.21.15.83:0/1677891932 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc90074f90 con 0x7fbc9006c130 2024-04-17T10:49:05.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.461+0000 7fbc8cff9640 1 -- 172.21.15.83:0/1677891932 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbc80018070 con 0x7fbc9006c130 2024-04-17T10:49:05.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.461+0000 7fbc8cff9640 1 --2- 172.21.15.83:0/1677891932 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbc70066f90 0x7fbc70069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:05.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.461+0000 7fbc8cff9640 1 -- 172.21.15.83:0/1677891932 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fbc80084f60 con 0x7fbc9006c130 2024-04-17T10:49:05.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.461+0000 7fbc87fff640 1 --2- 172.21.15.83:0/1677891932 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbc70066f90 0x7fbc70069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:05.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.461+0000 7fbc87fff640 1 --2- 172.21.15.83:0/1677891932 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbc70066f90 0x7fbc70069450 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fbc780ae000 tx=0x7fbc78008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:05.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.465+0000 7fbc8cff9640 1 -- 172.21.15.83:0/1677891932 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbc80055f30 con 0x7fbc9006c130 2024-04-17T10:49:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.621+0000 7fbc95d68640 1 -- 172.21.15.83:0/1677891932 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbc9009b820 con 0x7fbc9006c130 2024-04-17T10:49:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.625+0000 7fbc8cff9640 1 -- 172.21.15.83:0/1677891932 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbc80055830 con 0x7fbc9006c130 2024-04-17T10:49:05.621 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:49:05.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.625+0000 7fbc95d68640 1 -- 172.21.15.83:0/1677891932 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbc70066f90 msgr2=0x7fbc70069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:05.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.625+0000 7fbc95d68640 1 --2- 172.21.15.83:0/1677891932 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbc70066f90 0x7fbc70069450 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fbc780ae000 tx=0x7fbc78008040 comp rx=0 tx=0).stop 2024-04-17T10:49:05.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.625+0000 7fbc95d68640 1 -- 172.21.15.83:0/1677891932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc9006c130 msgr2=0x7fbc90153df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:05.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.625+0000 7fbc95d68640 1 --2- 172.21.15.83:0/1677891932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc9006c130 0x7fbc90153df0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fbc8000c820 tx=0x7fbc8000cce0 comp rx=0 tx=0).stop 2024-04-17T10:49:05.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.625+0000 7fbc95d68640 1 -- 172.21.15.83:0/1677891932 shutdown_connections 2024-04-17T10:49:05.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.625+0000 7fbc95d68640 1 --2- 172.21.15.83:0/1677891932 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbc70066f90 0x7fbc70069450 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:05.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.625+0000 7fbc95d68640 1 --2- 172.21.15.83:0/1677891932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc9015aa10 0x7fbc9015b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:05.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.625+0000 7fbc95d68640 1 --2- 172.21.15.83:0/1677891932 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc900cbca0 0x7fbc9015b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:05.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.625+0000 7fbc95d68640 1 --2- 172.21.15.83:0/1677891932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc9006c130 0x7fbc90153df0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:05.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.625+0000 7fbc95d68640 1 -- 172.21.15.83:0/1677891932 >> 172.21.15.83:0/1677891932 conn(0x7fbc900cc990 msgr2=0x7fbc900cab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:05.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.625+0000 7fbc95d68640 1 -- 172.21.15.83:0/1677891932 shutdown_connections 2024-04-17T10:49:05.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.625+0000 7fbc95d68640 1 -- 172.21.15.83:0/1677891932 wait complete. 2024-04-17T10:49:05.640 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:49:05.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.765+0000 7f01baad6640 1 Processor -- start 2024-04-17T10:49:05.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.765+0000 7f01baad6640 1 -- start start 2024-04-17T10:49:05.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.769+0000 7f01baad6640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f01b415bac0 0x7f01b4071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:05.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.769+0000 7f01baad6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01b4070470 0x7f01b406ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:05.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.769+0000 7f01baad6640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01b406c1f0 con 0x7f01b415b6e0 2024-04-17T10:49:05.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.769+0000 7f01baad6640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01b4070a10 con 0x7f01b415bac0 2024-04-17T10:49:05.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.769+0000 7f01baad6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b4070cf0 con 0x7f01b4070470 2024-04-17T10:49:05.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.769+0000 7f01b3fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f01b415bac0 0x7f01b4071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:05.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.769+0000 7f01b3fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f01b415bac0 0x7f01b4071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40898/0 (socket says 172.21.15.83:40898) 2024-04-17T10:49:05.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.769+0000 7f01b3fff640 1 -- 172.21.15.83:0/85689888 learned_addr learned my addr 172.21.15.83:0/85689888 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:05.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.769+0000 7f01b9073640 1 --2- 172.21.15.83:0/85689888 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01b4070470 0x7f01b406ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:05.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.769+0000 7f01b3fff640 1 -- 172.21.15.83:0/85689888 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01b4070470 msgr2=0x7f01b406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:05.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.769+0000 7f01b3fff640 1 --2- 172.21.15.83:0/85689888 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01b4070470 0x7f01b406ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:05.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.769+0000 7f01b3fff640 1 -- 172.21.15.83:0/85689888 >> v1:172.21.15.167:6789/0 conn(0x7f01b415b6e0 legacy=0x7f01b40cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:05.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.769+0000 7f01b3fff640 1 -- 172.21.15.83:0/85689888 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b4070fd0 con 0x7f01b415bac0 2024-04-17T10:49:05.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.769+0000 7f01b9073640 1 --2- 172.21.15.83:0/85689888 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01b4070470 0x7f01b406ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:05.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.769+0000 7f01b3fff640 1 --2- 172.21.15.83:0/85689888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01b415bac0 0x7f01b4071cc0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f01b4071c50 tx=0x7f01a801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619a199211a2c690 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:05.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.769+0000 7f01b37fe640 1 -- 172.21.15.83:0/85689888 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a80b0070 con 0x7f01b415bac0 2024-04-17T10:49:05.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.769+0000 7f01b37fe640 1 -- 172.21.15.83:0/85689888 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a80ab600 con 0x7f01b415bac0 2024-04-17T10:49:05.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.769+0000 7f01b37fe640 1 -- 172.21.15.83:0/85689888 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a80ab9d0 con 0x7f01b415bac0 2024-04-17T10:49:05.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.773+0000 7f01baad6640 1 -- 172.21.15.83:0/85689888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01b415bac0 msgr2=0x7f01b4071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:05.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.773+0000 7f01baad6640 1 --2- 172.21.15.83:0/85689888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01b415bac0 0x7f01b4071cc0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f01b4071c50 tx=0x7f01a801c450 comp rx=0 tx=0).stop 2024-04-17T10:49:05.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.773+0000 7f01baad6640 1 -- 172.21.15.83:0/85689888 shutdown_connections 2024-04-17T10:49:05.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.773+0000 7f01baad6640 1 --2- 172.21.15.83:0/85689888 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01b4070470 0x7f01b406ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:05.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.773+0000 7f01baad6640 1 --2- 172.21.15.83:0/85689888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01b415bac0 0x7f01b4071cc0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:05.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.773+0000 7f01baad6640 1 -- 172.21.15.83:0/85689888 >> 172.21.15.83:0/85689888 conn(0x7f01b40b15b0 msgr2=0x7f01b40b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:05.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.773+0000 7f01baad6640 1 -- 172.21.15.83:0/85689888 shutdown_connections 2024-04-17T10:49:05.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.773+0000 7f01baad6640 1 -- 172.21.15.83:0/85689888 wait complete. 2024-04-17T10:49:05.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.773+0000 7f01baad6640 1 Processor -- start 2024-04-17T10:49:05.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.773+0000 7f01baad6640 1 -- start start 2024-04-17T10:49:05.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.773+0000 7f01baad6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01b4070470 0x7f01b4152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:05.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.773+0000 7f01baad6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01b415b6e0 0x7f01b40b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:05.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.773+0000 7f01baad6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01b415bac0 0x7f01b4151220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:05.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.773+0000 7f01baad6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b406d2f0 con 0x7f01b4070470 2024-04-17T10:49:05.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.773+0000 7f01baad6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b406d170 con 0x7f01b415bac0 2024-04-17T10:49:05.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.773+0000 7f01baad6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b406b760 con 0x7f01b415b6e0 2024-04-17T10:49:05.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.773+0000 7f01b8872640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01b4070470 0x7f01b4152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:05.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.773+0000 7f01b8872640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01b4070470 0x7f01b4152230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40912/0 (socket says 172.21.15.83:40912) 2024-04-17T10:49:05.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.773+0000 7f01b8872640 1 -- 172.21.15.83:0/3549940160 learned_addr learned my addr 172.21.15.83:0/3549940160 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:05.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.773+0000 7f01b3fff640 1 --2- 172.21.15.83:0/3549940160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01b415b6e0 0x7f01b40b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:05.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.773+0000 7f01b9073640 1 --2- 172.21.15.83:0/3549940160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01b415bac0 0x7f01b4151220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:05.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.773+0000 7f01b8872640 1 -- 172.21.15.83:0/3549940160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01b415b6e0 msgr2=0x7f01b40b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:05.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.773+0000 7f01b8872640 1 --2- 172.21.15.83:0/3549940160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01b415b6e0 0x7f01b40b27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:05.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.773+0000 7f01b8872640 1 -- 172.21.15.83:0/3549940160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01b415bac0 msgr2=0x7f01b4151220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:05.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.773+0000 7f01b8872640 1 --2- 172.21.15.83:0/3549940160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01b415bac0 0x7f01b4151220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:05.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.773+0000 7f01b8872640 1 -- 172.21.15.83:0/3549940160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b414e550 con 0x7f01b4070470 2024-04-17T10:49:05.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.773+0000 7f01b3fff640 1 --2- 172.21.15.83:0/3549940160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01b415b6e0 0x7f01b40b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:05.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.773+0000 7f01b9073640 1 --2- 172.21.15.83:0/3549940160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01b415bac0 0x7f01b4151220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:05.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.773+0000 7f01b8872640 1 --2- 172.21.15.83:0/3549940160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01b4070470 0x7f01b4152230 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f01a400c450 tx=0x7f01a400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:05.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.773+0000 7f01b17fa640 1 -- 172.21.15.83:0/3549940160 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a4012830 con 0x7f01b4070470 2024-04-17T10:49:05.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.773+0000 7f01b17fa640 1 -- 172.21.15.83:0/3549940160 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a4017070 con 0x7f01b4070470 2024-04-17T10:49:05.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.777+0000 7f01baad6640 1 -- 172.21.15.83:0/3549940160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b414ede0 con 0x7f01b4070470 2024-04-17T10:49:05.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.777+0000 7f01b17fa640 1 -- 172.21.15.83:0/3549940160 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a4012c30 con 0x7f01b4070470 2024-04-17T10:49:05.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.777+0000 7f01baad6640 1 -- 172.21.15.83:0/3549940160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b414fa30 con 0x7f01b4070470 2024-04-17T10:49:05.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.777+0000 7f01baad6640 1 -- 172.21.15.83:0/3549940160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01b4074f90 con 0x7f01b4070470 2024-04-17T10:49:05.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.777+0000 7f01b17fa640 1 -- 172.21.15.83:0/3549940160 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f01a401c460 con 0x7f01b4070470 2024-04-17T10:49:05.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.777+0000 7f01b17fa640 1 --2- 172.21.15.83:0/3549940160 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f01a0066f60 0x7f01a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:05.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.777+0000 7f01b17fa640 1 -- 172.21.15.83:0/3549940160 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f01a4084d50 con 0x7f01b4070470 2024-04-17T10:49:05.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.777+0000 7f01b3fff640 1 --2- 172.21.15.83:0/3549940160 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f01a0066f60 0x7f01a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:05.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.777+0000 7f01b3fff640 1 --2- 172.21.15.83:0/3549940160 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f01a0066f60 0x7f01a0069420 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f01a80ad000 tx=0x7f01a80a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:05.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.781+0000 7f01b17fa640 1 -- 172.21.15.83:0/3549940160 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f01a4088030 con 0x7f01b4070470 2024-04-17T10:49:05.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.937+0000 7f01baad6640 1 -- 172.21.15.83:0/3549940160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f01b40b2d20 con 0x7f01b4070470 2024-04-17T10:49:05.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.937+0000 7f01b17fa640 1 -- 172.21.15.83:0/3549940160 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v116) v1 ==== 104+0+11 (secure 0 0 0) 0x7f01a4055db0 con 0x7f01b4070470 2024-04-17T10:49:05.936 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:49:05.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.941+0000 7f01baad6640 1 -- 172.21.15.83:0/3549940160 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f01a0066f60 msgr2=0x7f01a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:05.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.941+0000 7f01baad6640 1 --2- 172.21.15.83:0/3549940160 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f01a0066f60 0x7f01a0069420 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f01a80ad000 tx=0x7f01a80a6000 comp rx=0 tx=0).stop 2024-04-17T10:49:05.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.941+0000 7f01baad6640 1 -- 172.21.15.83:0/3549940160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01b4070470 msgr2=0x7f01b4152230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:05.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.941+0000 7f01baad6640 1 --2- 172.21.15.83:0/3549940160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01b4070470 0x7f01b4152230 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f01a400c450 tx=0x7f01a400c910 comp rx=0 tx=0).stop 2024-04-17T10:49:05.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.941+0000 7f01baad6640 1 -- 172.21.15.83:0/3549940160 shutdown_connections 2024-04-17T10:49:05.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.941+0000 7f01baad6640 1 --2- 172.21.15.83:0/3549940160 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f01a0066f60 0x7f01a0069420 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:05.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.941+0000 7f01baad6640 1 --2- 172.21.15.83:0/3549940160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01b415bac0 0x7f01b4151220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:05.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.941+0000 7f01baad6640 1 --2- 172.21.15.83:0/3549940160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01b415b6e0 0x7f01b40b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:05.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.941+0000 7f01baad6640 1 --2- 172.21.15.83:0/3549940160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01b4070470 0x7f01b4152230 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:05.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.941+0000 7f01baad6640 1 -- 172.21.15.83:0/3549940160 >> 172.21.15.83:0/3549940160 conn(0x7f01b40b15b0 msgr2=0x7f01b40cd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:05.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.941+0000 7f01baad6640 1 -- 172.21.15.83:0/3549940160 shutdown_connections 2024-04-17T10:49:05.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:05.941+0000 7f01baad6640 1 -- 172.21.15.83:0/3549940160 wait complete. 2024-04-17T10:49:05.954 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:49:06.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.073+0000 7fad3359d640 1 Processor -- start 2024-04-17T10:49:06.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.073+0000 7fad3359d640 1 -- start start 2024-04-17T10:49:06.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.077+0000 7fad3359d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad2c154de0 0x7fad2c06ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:06.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.077+0000 7fad3359d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fad2c06b350 0x7fad2c0b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:06.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.077+0000 7fad3359d640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad2c15bd70 con 0x7fad2c0b4040 2024-04-17T10:49:06.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.077+0000 7fad3359d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad2c0b4870 con 0x7fad2c06b350 2024-04-17T10:49:06.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.077+0000 7fad3359d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c15bf60 con 0x7fad2c154de0 2024-04-17T10:49:06.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.077+0000 7fad30b38640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fad2c06b350 0x7fad2c0b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:06.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.077+0000 7fad30b38640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fad2c06b350 0x7fad2c0b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40924/0 (socket says 172.21.15.83:40924) 2024-04-17T10:49:06.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.077+0000 7fad30b38640 1 -- 172.21.15.83:0/1300852375 learned_addr learned my addr 172.21.15.83:0/1300852375 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:06.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.077+0000 7fad31339640 1 --2- 172.21.15.83:0/1300852375 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad2c154de0 0x7fad2c06ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:06.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.077+0000 7fad30b38640 1 -- 172.21.15.83:0/1300852375 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad2c154de0 msgr2=0x7fad2c06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:06.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.077+0000 7fad30b38640 1 --2- 172.21.15.83:0/1300852375 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad2c154de0 0x7fad2c06ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.077+0000 7fad30b38640 1 -- 172.21.15.83:0/1300852375 >> v1:172.21.15.167:6789/0 conn(0x7fad2c0b4040 legacy=0x7fad2c071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:06.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.077+0000 7fad30b38640 1 -- 172.21.15.83:0/1300852375 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2c15afb0 con 0x7fad2c06b350 2024-04-17T10:49:06.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.077+0000 7fad30b38640 1 --2- 172.21.15.83:0/1300852375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad2c06b350 0x7fad2c0b3b00 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fad1c00bb10 tx=0x7fad1c01ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dea444906a97870 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:06.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.077+0000 7fad23fff640 1 -- 172.21.15.83:0/1300852375 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1c0b4070 con 0x7fad2c06b350 2024-04-17T10:49:06.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.077+0000 7fad23fff640 1 -- 172.21.15.83:0/1300852375 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad1c0a7a20 con 0x7fad2c06b350 2024-04-17T10:49:06.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.077+0000 7fad23fff640 1 -- 172.21.15.83:0/1300852375 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1c0a7df0 con 0x7fad2c06b350 2024-04-17T10:49:06.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad3359d640 1 -- 172.21.15.83:0/1300852375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad2c06b350 msgr2=0x7fad2c0b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:06.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad3359d640 1 --2- 172.21.15.83:0/1300852375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad2c06b350 0x7fad2c0b3b00 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fad1c00bb10 tx=0x7fad1c01ec30 comp rx=0 tx=0).stop 2024-04-17T10:49:06.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad3359d640 1 -- 172.21.15.83:0/1300852375 shutdown_connections 2024-04-17T10:49:06.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad3359d640 1 --2- 172.21.15.83:0/1300852375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad2c06b350 0x7fad2c0b3b00 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad3359d640 1 --2- 172.21.15.83:0/1300852375 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad2c154de0 0x7fad2c06ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad3359d640 1 -- 172.21.15.83:0/1300852375 >> 172.21.15.83:0/1300852375 conn(0x7fad2c0b15b0 msgr2=0x7fad2c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:06.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad3359d640 1 -- 172.21.15.83:0/1300852375 shutdown_connections 2024-04-17T10:49:06.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad3359d640 1 -- 172.21.15.83:0/1300852375 wait complete. 2024-04-17T10:49:06.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad3359d640 1 Processor -- start 2024-04-17T10:49:06.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad3359d640 1 -- start start 2024-04-17T10:49:06.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad3359d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad2c06b350 0x7fad2c14c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:06.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad3359d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad2c0b4040 0x7fad2c149930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:06.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad3359d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad2c154de0 0x7fad2c15d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:06.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad3359d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c0b4450 con 0x7fad2c154de0 2024-04-17T10:49:06.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad3359d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c1551c0 con 0x7fad2c06b350 2024-04-17T10:49:06.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad3359d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c0b45d0 con 0x7fad2c0b4040 2024-04-17T10:49:06.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad31b3a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad2c154de0 0x7fad2c15d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:06.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad31b3a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad2c154de0 0x7fad2c15d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40928/0 (socket says 172.21.15.83:40928) 2024-04-17T10:49:06.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad31b3a640 1 -- 172.21.15.83:0/776475711 learned_addr learned my addr 172.21.15.83:0/776475711 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:06.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad31339640 1 --2- 172.21.15.83:0/776475711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad2c06b350 0x7fad2c14c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:06.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad30b38640 1 --2- 172.21.15.83:0/776475711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad2c0b4040 0x7fad2c149930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:06.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad31b3a640 1 -- 172.21.15.83:0/776475711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad2c0b4040 msgr2=0x7fad2c149930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:06.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad31b3a640 1 --2- 172.21.15.83:0/776475711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad2c0b4040 0x7fad2c149930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad31b3a640 1 -- 172.21.15.83:0/776475711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad2c06b350 msgr2=0x7fad2c14c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:06.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad31b3a640 1 --2- 172.21.15.83:0/776475711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad2c06b350 0x7fad2c14c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad31b3a640 1 -- 172.21.15.83:0/776475711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2c14a440 con 0x7fad2c154de0 2024-04-17T10:49:06.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad31339640 1 --2- 172.21.15.83:0/776475711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad2c06b350 0x7fad2c14c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:06.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad30b38640 1 --2- 172.21.15.83:0/776475711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad2c0b4040 0x7fad2c149930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:06.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad31b3a640 1 --2- 172.21.15.83:0/776475711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad2c154de0 0x7fad2c15d060 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fad2400c730 tx=0x7fad2400cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:06.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad21ffb640 1 -- 172.21.15.83:0/776475711 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad24016020 con 0x7fad2c154de0 2024-04-17T10:49:06.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad21ffb640 1 -- 172.21.15.83:0/776475711 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad24006d60 con 0x7fad2c154de0 2024-04-17T10:49:06.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad3359d640 1 -- 172.21.15.83:0/776475711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad2c15a620 con 0x7fad2c154de0 2024-04-17T10:49:06.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad21ffb640 1 -- 172.21.15.83:0/776475711 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad24003bc0 con 0x7fad2c154de0 2024-04-17T10:49:06.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.081+0000 7fad3359d640 1 -- 172.21.15.83:0/776475711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad2c158a30 con 0x7fad2c154de0 2024-04-17T10:49:06.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.085+0000 7fad3359d640 1 -- 172.21.15.83:0/776475711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad2c074f90 con 0x7fad2c154de0 2024-04-17T10:49:06.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.085+0000 7fad21ffb640 1 -- 172.21.15.83:0/776475711 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fad240050d0 con 0x7fad2c154de0 2024-04-17T10:49:06.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.085+0000 7fad21ffb640 1 --2- 172.21.15.83:0/776475711 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fad10066f60 0x7fad10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:06.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.085+0000 7fad21ffb640 1 -- 172.21.15.83:0/776475711 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fad24084c10 con 0x7fad2c154de0 2024-04-17T10:49:06.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.085+0000 7fad31339640 1 --2- 172.21.15.83:0/776475711 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fad10066f60 0x7fad10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:06.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.085+0000 7fad31339640 1 --2- 172.21.15.83:0/776475711 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fad10066f60 0x7fad10069420 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fad2c1505a0 tx=0x7fad28008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:06.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.089+0000 7fad21ffb640 1 -- 172.21.15.83:0/776475711 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fad24020020 con 0x7fad2c154de0 2024-04-17T10:49:06.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.237+0000 7fad3359d640 1 -- 172.21.15.83:0/776475711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fad2c0b1cd0 con 0x7fad2c154de0 2024-04-17T10:49:06.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.241+0000 7fad21ffb640 1 -- 172.21.15.83:0/776475711 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v116) v1 ==== 100+0+11 (secure 0 0 0) 0x7fad24055c70 con 0x7fad2c154de0 2024-04-17T10:49:06.237 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:49:06.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.241+0000 7fad3359d640 1 -- 172.21.15.83:0/776475711 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fad10066f60 msgr2=0x7fad10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:06.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.241+0000 7fad3359d640 1 --2- 172.21.15.83:0/776475711 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fad10066f60 0x7fad10069420 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fad2c1505a0 tx=0x7fad28008040 comp rx=0 tx=0).stop 2024-04-17T10:49:06.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.241+0000 7fad3359d640 1 -- 172.21.15.83:0/776475711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad2c154de0 msgr2=0x7fad2c15d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:06.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.241+0000 7fad3359d640 1 --2- 172.21.15.83:0/776475711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad2c154de0 0x7fad2c15d060 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fad2400c730 tx=0x7fad2400cb30 comp rx=0 tx=0).stop 2024-04-17T10:49:06.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.241+0000 7fad3359d640 1 -- 172.21.15.83:0/776475711 shutdown_connections 2024-04-17T10:49:06.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.241+0000 7fad3359d640 1 --2- 172.21.15.83:0/776475711 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fad10066f60 0x7fad10069420 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.241+0000 7fad3359d640 1 --2- 172.21.15.83:0/776475711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad2c154de0 0x7fad2c15d060 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.241+0000 7fad3359d640 1 --2- 172.21.15.83:0/776475711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad2c0b4040 0x7fad2c149930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.241+0000 7fad3359d640 1 --2- 172.21.15.83:0/776475711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad2c06b350 0x7fad2c14c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.241+0000 7fad3359d640 1 -- 172.21.15.83:0/776475711 >> 172.21.15.83:0/776475711 conn(0x7fad2c0b15b0 msgr2=0x7fad2c0ceda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:06.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.241+0000 7fad3359d640 1 -- 172.21.15.83:0/776475711 shutdown_connections 2024-04-17T10:49:06.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.241+0000 7fad3359d640 1 -- 172.21.15.83:0/776475711 wait complete. 2024-04-17T10:49:06.255 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:49:06.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.373+0000 7fca10dbf640 1 Processor -- start 2024-04-17T10:49:06.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.373+0000 7fca10dbf640 1 -- start start 2024-04-17T10:49:06.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.373+0000 7fca10dbf640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fca0c072330 0x7fca0c0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:06.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.373+0000 7fca10dbf640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca0c0cfa10 0x7fca0c06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:06.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.373+0000 7fca10dbf640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca0c0cfea0 con 0x7fca0c15ba30 2024-04-17T10:49:06.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.373+0000 7fca10dbf640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca0c071ea0 con 0x7fca0c072330 2024-04-17T10:49:06.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.373+0000 7fca10dbf640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c06e320 con 0x7fca0c0cfa10 2024-04-17T10:49:06.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.373+0000 7fca03fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fca0c072330 0x7fca0c0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:06.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.373+0000 7fca03fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fca0c072330 0x7fca0c0cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40936/0 (socket says 172.21.15.83:40936) 2024-04-17T10:49:06.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.373+0000 7fca03fff640 1 -- 172.21.15.83:0/821717448 learned_addr learned my addr 172.21.15.83:0/821717448 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:06.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.373+0000 7fca0ad9d640 1 --2- 172.21.15.83:0/821717448 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca0c0cfa10 0x7fca0c06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:06.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.373+0000 7fca03fff640 1 -- 172.21.15.83:0/821717448 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca0c0cfa10 msgr2=0x7fca0c06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:06.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.373+0000 7fca03fff640 1 --2- 172.21.15.83:0/821717448 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca0c0cfa10 0x7fca0c06c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.373+0000 7fca03fff640 1 -- 172.21.15.83:0/821717448 >> v1:172.21.15.167:6789/0 conn(0x7fca0c15ba30 legacy=0x7fca0c15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:06.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.373+0000 7fca03fff640 1 -- 172.21.15.83:0/821717448 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca0c06e040 con 0x7fca0c072330 2024-04-17T10:49:06.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.373+0000 7fca03fff640 1 --2- 172.21.15.83:0/821717448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca0c072330 0x7fca0c0cf4d0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fca0c072020 tx=0x7fc9f401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1301e852786fb47f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:06.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca09d9b640 1 -- 172.21.15.83:0/821717448 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f40b0070 con 0x7fca0c072330 2024-04-17T10:49:06.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca09d9b640 1 -- 172.21.15.83:0/821717448 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f40ab600 con 0x7fca0c072330 2024-04-17T10:49:06.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca09d9b640 1 -- 172.21.15.83:0/821717448 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f40ab9d0 con 0x7fca0c072330 2024-04-17T10:49:06.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca10dbf640 1 -- 172.21.15.83:0/821717448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca0c072330 msgr2=0x7fca0c0cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:06.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca10dbf640 1 --2- 172.21.15.83:0/821717448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca0c072330 0x7fca0c0cf4d0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fca0c072020 tx=0x7fc9f401c470 comp rx=0 tx=0).stop 2024-04-17T10:49:06.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca10dbf640 1 -- 172.21.15.83:0/821717448 shutdown_connections 2024-04-17T10:49:06.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca10dbf640 1 --2- 172.21.15.83:0/821717448 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca0c0cfa10 0x7fca0c06c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca10dbf640 1 --2- 172.21.15.83:0/821717448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca0c072330 0x7fca0c0cf4d0 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca10dbf640 1 -- 172.21.15.83:0/821717448 >> 172.21.15.83:0/821717448 conn(0x7fca0c06abe0 msgr2=0x7fca0c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:06.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca10dbf640 1 -- 172.21.15.83:0/821717448 shutdown_connections 2024-04-17T10:49:06.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca10dbf640 1 -- 172.21.15.83:0/821717448 wait complete. 2024-04-17T10:49:06.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca10dbf640 1 Processor -- start 2024-04-17T10:49:06.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca10dbf640 1 -- start start 2024-04-17T10:49:06.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca10dbf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca0c072330 0x7fca0c147f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:06.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca10dbf640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca0c0cfa10 0x7fca0c148790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:06.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca10dbf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca0c15ba30 0x7fca0c148cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:06.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca10dbf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c06de20 con 0x7fca0c072330 2024-04-17T10:49:06.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca10dbf640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c06dca0 con 0x7fca0c15ba30 2024-04-17T10:49:06.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca10dbf640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c168520 con 0x7fca0c0cfa10 2024-04-17T10:49:06.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca0a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca0c072330 0x7fca0c147f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:06.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca0a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca0c072330 0x7fca0c147f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40938/0 (socket says 172.21.15.83:40938) 2024-04-17T10:49:06.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca0a59c640 1 -- 172.21.15.83:0/3303065690 learned_addr learned my addr 172.21.15.83:0/3303065690 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:06.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca0a59c640 1 -- 172.21.15.83:0/3303065690 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca0c0cfa10 msgr2=0x7fca0c148790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:06.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca0a59c640 1 --2- 172.21.15.83:0/3303065690 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca0c0cfa10 0x7fca0c148790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca0a59c640 1 -- 172.21.15.83:0/3303065690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca0c15ba30 msgr2=0x7fca0c148cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:06.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca0a59c640 1 --2- 172.21.15.83:0/3303065690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca0c15ba30 0x7fca0c148cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca0a59c640 1 -- 172.21.15.83:0/3303065690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca0c15daa0 con 0x7fca0c072330 2024-04-17T10:49:06.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca0a59c640 1 --2- 172.21.15.83:0/3303065690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca0c072330 0x7fca0c147f60 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc00c450 tx=0x7fc9fc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:06.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca02ffd640 1 -- 172.21.15.83:0/3303065690 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc012830 con 0x7fca0c072330 2024-04-17T10:49:06.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca02ffd640 1 -- 172.21.15.83:0/3303065690 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9fc017070 con 0x7fca0c072330 2024-04-17T10:49:06.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca02ffd640 1 -- 172.21.15.83:0/3303065690 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc012c10 con 0x7fca0c072330 2024-04-17T10:49:06.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca10dbf640 1 -- 172.21.15.83:0/3303065690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca0c15dd80 con 0x7fca0c072330 2024-04-17T10:49:06.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.377+0000 7fca10dbf640 1 -- 172.21.15.83:0/3303065690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca0c15e340 con 0x7fca0c072330 2024-04-17T10:49:06.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.381+0000 7fca10dbf640 1 -- 172.21.15.83:0/3303065690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca0c074f90 con 0x7fca0c072330 2024-04-17T10:49:06.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.381+0000 7fca02ffd640 1 -- 172.21.15.83:0/3303065690 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc9fc01c460 con 0x7fca0c072330 2024-04-17T10:49:06.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.381+0000 7fca02ffd640 1 --2- 172.21.15.83:0/3303065690 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc9f0066f60 0x7fc9f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:06.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.381+0000 7fca03fff640 1 --2- 172.21.15.83:0/3303065690 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc9f0066f60 0x7fc9f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:06.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.381+0000 7fca02ffd640 1 -- 172.21.15.83:0/3303065690 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc9fc084e50 con 0x7fca0c072330 2024-04-17T10:49:06.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.381+0000 7fca03fff640 1 --2- 172.21.15.83:0/3303065690 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc9f0066f60 0x7fc9f0069420 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fca0c0b2410 tx=0x7fc9f4004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:06.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.385+0000 7fca02ffd640 1 -- 172.21.15.83:0/3303065690 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc9fc055e20 con 0x7fca0c072330 2024-04-17T10:49:06.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.565+0000 7fca10dbf640 1 -- 172.21.15.83:0/3303065690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca0c14fd70 con 0x7fca0c072330 2024-04-17T10:49:06.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.565+0000 7fca02ffd640 1 -- 172.21.15.83:0/3303065690 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fc9fc055720 con 0x7fca0c072330 2024-04-17T10:49:06.563 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:06.563 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5786,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1553533838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1553533838},{"type":"v1","addr":"172.21.15.167:6833","nonce":1553533838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/674536913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":674536913},{"type":"v1","addr":"172.21.15.167:6835","nonce":674536913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8598,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4183890457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4183890457},{"type":"v1","addr":"172.21.15.83:6835","nonce":4183890457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:48:24.738443+0000","modified":"2024-04-17T10:48:26.776273+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8604":{"gid":8604,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1479417998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1479417998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1479417998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-17T10:49:06.563 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 69 2024-04-17T10:49:06.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.569+0000 7fca10dbf640 1 -- 172.21.15.83:0/3303065690 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc9f0066f60 msgr2=0x7fc9f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:06.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.569+0000 7fca10dbf640 1 --2- 172.21.15.83:0/3303065690 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc9f0066f60 0x7fc9f0069420 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fca0c0b2410 tx=0x7fc9f4004740 comp rx=0 tx=0).stop 2024-04-17T10:49:06.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.569+0000 7fca10dbf640 1 -- 172.21.15.83:0/3303065690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca0c072330 msgr2=0x7fca0c147f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:06.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.569+0000 7fca10dbf640 1 --2- 172.21.15.83:0/3303065690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca0c072330 0x7fca0c147f60 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc00c450 tx=0x7fc9fc00c910 comp rx=0 tx=0).stop 2024-04-17T10:49:06.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.569+0000 7fca10dbf640 1 -- 172.21.15.83:0/3303065690 shutdown_connections 2024-04-17T10:49:06.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.569+0000 7fca10dbf640 1 --2- 172.21.15.83:0/3303065690 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc9f0066f60 0x7fc9f0069420 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.569+0000 7fca10dbf640 1 --2- 172.21.15.83:0/3303065690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca0c15ba30 0x7fca0c148cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.569+0000 7fca10dbf640 1 --2- 172.21.15.83:0/3303065690 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca0c0cfa10 0x7fca0c148790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.569+0000 7fca10dbf640 1 --2- 172.21.15.83:0/3303065690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca0c072330 0x7fca0c147f60 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.569+0000 7fca10dbf640 1 -- 172.21.15.83:0/3303065690 >> 172.21.15.83:0/3303065690 conn(0x7fca0c06abe0 msgr2=0x7fca0c1521c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:06.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.569+0000 7fca10dbf640 1 -- 172.21.15.83:0/3303065690 shutdown_connections 2024-04-17T10:49:06.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.569+0000 7fca10dbf640 1 -- 172.21.15.83:0/3303065690 wait complete. 2024-04-17T10:49:06.580 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:49:06.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.693+0000 7fa4825c8640 1 Processor -- start 2024-04-17T10:49:06.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.693+0000 7fa4825c8640 1 -- start start 2024-04-17T10:49:06.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.693+0000 7fa4825c8640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa47c0ce980 0x7fa47c0b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:06.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.693+0000 7fa4825c8640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa47c162a00 con 0x7fa47c0cee00 2024-04-17T10:49:06.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.693+0000 7fa4825c8640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa47c161e80 con 0x7fa47c15dfa0 2024-04-17T10:49:06.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.693+0000 7fa4825c8640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa47c162160 con 0x7fa47c0ce980 2024-04-17T10:49:06.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.693+0000 7fa47b7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa47c0ce980 0x7fa47c0b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:06.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.693+0000 7fa480b65640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa47c0cee00 0x7fa47c0d1d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49390/0 (socket says 172.21.15.83:49390) 2024-04-17T10:49:06.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.693+0000 7fa480b65640 1 -- 172.21.15.83:0/3175512877 learned_addr learned my addr 172.21.15.83:0/3175512877 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:06.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.693+0000 7fa47b7fe640 1 -- 172.21.15.83:0/3175512877 >> v1:172.21.15.167:6789/0 conn(0x7fa47c15dfa0 legacy=0x7fa47c0b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:06.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.693+0000 7fa47b7fe640 1 -- 172.21.15.83:0/3175512877 >> v1:172.21.15.83:6789/0 conn(0x7fa47c0cee00 legacy=0x7fa47c0d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:06.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.693+0000 7fa47b7fe640 1 -- 172.21.15.83:0/3175512877 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa47c162440 con 0x7fa47c0ce980 2024-04-17T10:49:06.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.693+0000 7fa47affd640 1 -- 172.21.15.83:0/3175512877 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2282177195 0 0) 0x7fa47c162a00 con 0x7fa47c0cee00 2024-04-17T10:49:06.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa47b7fe640 1 --2- 172.21.15.83:0/3175512877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa47c0ce980 0x7fa47c0b5430 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fa4700097c0 tx=0x7fa47001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ee37ebfd7dbc9e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:06.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa47affd640 1 -- 172.21.15.83:0/3175512877 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4700b0070 con 0x7fa47c0ce980 2024-04-17T10:49:06.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa47affd640 1 -- 172.21.15.83:0/3175512877 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4700ab600 con 0x7fa47c0ce980 2024-04-17T10:49:06.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa47affd640 1 -- 172.21.15.83:0/3175512877 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4700ab9d0 con 0x7fa47c0ce980 2024-04-17T10:49:06.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa4825c8640 1 -- 172.21.15.83:0/3175512877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa47c0ce980 msgr2=0x7fa47c0b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:06.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa4825c8640 1 --2- 172.21.15.83:0/3175512877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa47c0ce980 0x7fa47c0b5430 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fa4700097c0 tx=0x7fa47001c3a0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa4825c8640 1 -- 172.21.15.83:0/3175512877 shutdown_connections 2024-04-17T10:49:06.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa4825c8640 1 --2- 172.21.15.83:0/3175512877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa47c0ce980 0x7fa47c0b5430 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa4825c8640 1 -- 172.21.15.83:0/3175512877 >> 172.21.15.83:0/3175512877 conn(0x7fa47c0d0600 msgr2=0x7fa47c0d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:06.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa4825c8640 1 -- 172.21.15.83:0/3175512877 shutdown_connections 2024-04-17T10:49:06.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa4825c8640 1 -- 172.21.15.83:0/3175512877 wait complete. 2024-04-17T10:49:06.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa4825c8640 1 Processor -- start 2024-04-17T10:49:06.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa4825c8640 1 -- start start 2024-04-17T10:49:06.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa4825c8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa47c0ce980 0x7fa47c1534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:06.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa4825c8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa47c0cee00 0x7fa47c1fa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:06.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa4825c8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa47c15dfa0 0x7fa47c1fa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:06.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa4825c8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c16c670 con 0x7fa47c15dfa0 2024-04-17T10:49:06.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa4825c8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c161b10 con 0x7fa47c0ce980 2024-04-17T10:49:06.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa4825c8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c16c7f0 con 0x7fa47c0cee00 2024-04-17T10:49:06.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa480b65640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa47c15dfa0 0x7fa47c1fa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:06.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa480b65640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa47c15dfa0 0x7fa47c1fa9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40952/0 (socket says 172.21.15.83:40952) 2024-04-17T10:49:06.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa480b65640 1 -- 172.21.15.83:0/691025287 learned_addr learned my addr 172.21.15.83:0/691025287 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:06.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa47bfff640 1 --2- 172.21.15.83:0/691025287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa47c0ce980 0x7fa47c1534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:06.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa47b7fe640 1 --2- 172.21.15.83:0/691025287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa47c0cee00 0x7fa47c1fa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:06.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa480b65640 1 -- 172.21.15.83:0/691025287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa47c0cee00 msgr2=0x7fa47c1fa4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:06.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa480b65640 1 --2- 172.21.15.83:0/691025287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa47c0cee00 0x7fa47c1fa4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa480b65640 1 -- 172.21.15.83:0/691025287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa47c0ce980 msgr2=0x7fa47c1534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:06.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa480b65640 1 --2- 172.21.15.83:0/691025287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa47c0ce980 0x7fa47c1534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa480b65640 1 -- 172.21.15.83:0/691025287 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa47c150e30 con 0x7fa47c15dfa0 2024-04-17T10:49:06.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa480b65640 1 --2- 172.21.15.83:0/691025287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa47c15dfa0 0x7fa47c1fa9f0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fa474001570 tx=0x7fa47400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:06.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa478ff9640 1 -- 172.21.15.83:0/691025287 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa474018070 con 0x7fa47c15dfa0 2024-04-17T10:49:06.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa478ff9640 1 -- 172.21.15.83:0/691025287 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa47400a640 con 0x7fa47c15dfa0 2024-04-17T10:49:06.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa4825c8640 1 -- 172.21.15.83:0/691025287 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa47c151110 con 0x7fa47c15dfa0 2024-04-17T10:49:06.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa478ff9640 1 -- 172.21.15.83:0/691025287 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47400aa30 con 0x7fa47c15dfa0 2024-04-17T10:49:06.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.697+0000 7fa4825c8640 1 -- 172.21.15.83:0/691025287 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa47c1516d0 con 0x7fa47c15dfa0 2024-04-17T10:49:06.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.701+0000 7fa4825c8640 1 -- 172.21.15.83:0/691025287 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa47c0681a0 con 0x7fa47c15dfa0 2024-04-17T10:49:06.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.701+0000 7fa478ff9640 1 -- 172.21.15.83:0/691025287 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa47401b070 con 0x7fa47c15dfa0 2024-04-17T10:49:06.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.701+0000 7fa478ff9640 1 --2- 172.21.15.83:0/691025287 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa468066f60 0x7fa468069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:06.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.701+0000 7fa478ff9640 1 -- 172.21.15.83:0/691025287 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa4740846d0 con 0x7fa47c15dfa0 2024-04-17T10:49:06.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.701+0000 7fa47bfff640 1 --2- 172.21.15.83:0/691025287 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa468066f60 0x7fa468069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:06.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.701+0000 7fa47bfff640 1 --2- 172.21.15.83:0/691025287 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa468066f60 0x7fa468069420 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00b810 tx=0x7fa46c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:06.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.705+0000 7fa478ff9640 1 -- 172.21.15.83:0/691025287 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa4740556a0 con 0x7fa47c15dfa0 2024-04-17T10:49:06.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.853+0000 7fa4825c8640 1 -- 172.21.15.83:0/691025287 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa47c0d17a0 con 0x7fa47c15dfa0 2024-04-17T10:49:06.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.853+0000 7fa478ff9640 1 -- 172.21.15.83:0/691025287 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+16649 (secure 0 0 0) 0x7fa474054fa0 con 0x7fa47c15dfa0 2024-04-17T10:49:06.851 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:06.852 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":116,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:48:55.454382+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:48:22.981915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-17T10:48:23.700138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.18","mappings":[{"from":0,"to":5}]},{"pgid":"17.34","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:49:06.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.857+0000 7fa4825c8640 1 -- 172.21.15.83:0/691025287 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa468066f60 msgr2=0x7fa468069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:06.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.857+0000 7fa4825c8640 1 --2- 172.21.15.83:0/691025287 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa468066f60 0x7fa468069420 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00b810 tx=0x7fa46c008040 comp rx=0 tx=0).stop 2024-04-17T10:49:06.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.857+0000 7fa4825c8640 1 -- 172.21.15.83:0/691025287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa47c15dfa0 msgr2=0x7fa47c1fa9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:06.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.857+0000 7fa4825c8640 1 --2- 172.21.15.83:0/691025287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa47c15dfa0 0x7fa47c1fa9f0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fa474001570 tx=0x7fa47400cca0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.857+0000 7fa4825c8640 1 -- 172.21.15.83:0/691025287 shutdown_connections 2024-04-17T10:49:06.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.857+0000 7fa4825c8640 1 --2- 172.21.15.83:0/691025287 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa468066f60 0x7fa468069420 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.857+0000 7fa4825c8640 1 --2- 172.21.15.83:0/691025287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa47c15dfa0 0x7fa47c1fa9f0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.857+0000 7fa4825c8640 1 --2- 172.21.15.83:0/691025287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa47c0cee00 0x7fa47c1fa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.857+0000 7fa4825c8640 1 --2- 172.21.15.83:0/691025287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa47c0ce980 0x7fa47c1534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.857+0000 7fa4825c8640 1 -- 172.21.15.83:0/691025287 >> 172.21.15.83:0/691025287 conn(0x7fa47c0d0600 msgr2=0x7fa47c0caa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:06.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.857+0000 7fa4825c8640 1 -- 172.21.15.83:0/691025287 shutdown_connections 2024-04-17T10:49:06.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.857+0000 7fa4825c8640 1 -- 172.21.15.83:0/691025287 wait complete. 2024-04-17T10:49:06.870 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-17T10:49:06.870 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:49:06.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.989+0000 7f52bd964640 1 Processor -- start 2024-04-17T10:49:06.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.989+0000 7f52bd964640 1 -- start start 2024-04-17T10:49:06.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.993+0000 7f52bd964640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f52b815db50 0x7f52b8070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:06.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.993+0000 7f52bd964640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52b806dcb0 0x7f52b80b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:06.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.993+0000 7f52bd964640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52b80709b0 con 0x7f52b80aedd0 2024-04-17T10:49:06.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.993+0000 7f52bd964640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52b815e440 con 0x7f52b815db50 2024-04-17T10:49:06.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.993+0000 7f52bd964640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b815e720 con 0x7f52b806dcb0 2024-04-17T10:49:06.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.993+0000 7f52b6ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f52b815db50 0x7f52b8070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:06.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.993+0000 7f52b6ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f52b815db50 0x7f52b8070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45072/0 (socket says 172.21.15.83:45072) 2024-04-17T10:49:06.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.993+0000 7f52b6ffd640 1 -- 172.21.15.83:0/19518769 learned_addr learned my addr 172.21.15.83:0/19518769 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:06.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.993+0000 7f52b77fe640 1 --2- 172.21.15.83:0/19518769 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52b806dcb0 0x7f52b80b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:06.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.993+0000 7f52b6ffd640 1 -- 172.21.15.83:0/19518769 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52b806dcb0 msgr2=0x7f52b80b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:06.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.993+0000 7f52b6ffd640 1 --2- 172.21.15.83:0/19518769 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52b806dcb0 0x7f52b80b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.993+0000 7f52b6ffd640 1 -- 172.21.15.83:0/19518769 >> v1:172.21.15.167:6789/0 conn(0x7f52b80aedd0 legacy=0x7f52b806d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:06.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.993+0000 7f52b6ffd640 1 -- 172.21.15.83:0/19518769 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b815ea00 con 0x7f52b815db50 2024-04-17T10:49:06.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.993+0000 7f52b6ffd640 1 --2- 172.21.15.83:0/19518769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52b815db50 0x7f52b8070470 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f52a800bb10 tx=0x7f52a801e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6d78183c1bac95 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:06.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.993+0000 7f52b5ffb640 1 -- 172.21.15.83:0/19518769 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a80b4070 con 0x7f52b815db50 2024-04-17T10:49:06.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.993+0000 7f52b5ffb640 1 -- 172.21.15.83:0/19518769 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a80a7a70 con 0x7f52b815db50 2024-04-17T10:49:06.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.993+0000 7f52b5ffb640 1 -- 172.21.15.83:0/19518769 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a80a7e40 con 0x7f52b815db50 2024-04-17T10:49:06.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f52bd964640 1 -- 172.21.15.83:0/19518769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52b815db50 msgr2=0x7f52b8070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:06.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f52bd964640 1 --2- 172.21.15.83:0/19518769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52b815db50 0x7f52b8070470 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f52a800bb10 tx=0x7f52a801e9e0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f52bd964640 1 -- 172.21.15.83:0/19518769 shutdown_connections 2024-04-17T10:49:06.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f52bd964640 1 --2- 172.21.15.83:0/19518769 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52b806dcb0 0x7f52b80b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f52bd964640 1 --2- 172.21.15.83:0/19518769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52b815db50 0x7f52b8070470 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f52bd964640 1 -- 172.21.15.83:0/19518769 >> 172.21.15.83:0/19518769 conn(0x7f52b806abe0 msgr2=0x7f52b806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:06.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f52bd964640 1 -- 172.21.15.83:0/19518769 shutdown_connections 2024-04-17T10:49:06.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f52bd964640 1 -- 172.21.15.83:0/19518769 wait complete. 2024-04-17T10:49:06.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f52bd964640 1 Processor -- start 2024-04-17T10:49:06.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f52bd964640 1 -- start start 2024-04-17T10:49:06.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f52bd964640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52b806dcb0 0x7f52b81599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:06.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f52bd964640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52b80aedd0 0x7f52b81572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:06.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f52bd964640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52b815db50 0x7f52b80abc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:06.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f52bd964640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b80b3ce0 con 0x7f52b815db50 2024-04-17T10:49:06.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f52bd964640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b80b3b60 con 0x7f52b80aedd0 2024-04-17T10:49:06.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f52bd964640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b80b3e60 con 0x7f52b806dcb0 2024-04-17T10:49:06.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f52b77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52b815db50 0x7f52b80abc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:06.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f52b77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52b815db50 0x7f52b80abc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45082/0 (socket says 172.21.15.83:45082) 2024-04-17T10:49:06.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f52b77fe640 1 -- 172.21.15.83:0/4115064522 learned_addr learned my addr 172.21.15.83:0/4115064522 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:06.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f52b6ffd640 1 --2- 172.21.15.83:0/4115064522 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52b806dcb0 0x7f52b81599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:06.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f52b77fe640 1 -- 172.21.15.83:0/4115064522 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52b806dcb0 msgr2=0x7f52b81599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:06.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f52b67fc640 1 --2- 172.21.15.83:0/4115064522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52b80aedd0 0x7f52b81572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:06.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f52b77fe640 1 --2- 172.21.15.83:0/4115064522 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52b806dcb0 0x7f52b81599c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f52b77fe640 1 -- 172.21.15.83:0/4115064522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52b80aedd0 msgr2=0x7f52b81572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:06.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f52b77fe640 1 --2- 172.21.15.83:0/4115064522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52b80aedd0 0x7f52b81572a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:06.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f52b77fe640 1 -- 172.21.15.83:0/4115064522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b8157ad0 con 0x7f52b815db50 2024-04-17T10:49:06.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f52b6ffd640 1 --2- 172.21.15.83:0/4115064522 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52b806dcb0 0x7f52b81599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:06.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f52b67fc640 1 --2- 172.21.15.83:0/4115064522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52b80aedd0 0x7f52b81572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:06.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f52b77fe640 1 --2- 172.21.15.83:0/4115064522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52b815db50 0x7f52b80abc10 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f52b0007d40 tx=0x7f52b000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:06.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f529ffff640 1 -- 172.21.15.83:0/4115064522 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b0014420 con 0x7f52b815db50 2024-04-17T10:49:06.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f529ffff640 1 -- 172.21.15.83:0/4115064522 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b0019070 con 0x7f52b815db50 2024-04-17T10:49:06.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f529ffff640 1 -- 172.21.15.83:0/4115064522 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b0014820 con 0x7f52b815db50 2024-04-17T10:49:06.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f52bd964640 1 -- 172.21.15.83:0/4115064522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b8157db0 con 0x7f52b815db50 2024-04-17T10:49:06.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:06.997+0000 7f52bd964640 1 -- 172.21.15.83:0/4115064522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b81577e0 con 0x7f52b815db50 2024-04-17T10:49:06.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.001+0000 7f52bd964640 1 -- 172.21.15.83:0/4115064522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52b8074f90 con 0x7f52b815db50 2024-04-17T10:49:06.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.001+0000 7f529ffff640 1 -- 172.21.15.83:0/4115064522 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f52b00149c0 con 0x7f52b815db50 2024-04-17T10:49:06.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.001+0000 7f529ffff640 1 --2- 172.21.15.83:0/4115064522 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f52a4066f90 0x7f52a4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:06.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.001+0000 7f529ffff640 1 -- 172.21.15.83:0/4115064522 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f52b00857f0 con 0x7f52b815db50 2024-04-17T10:49:06.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.001+0000 7f52b6ffd640 1 --2- 172.21.15.83:0/4115064522 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f52a4066f90 0x7f52a4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:06.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.001+0000 7f52b6ffd640 1 --2- 172.21.15.83:0/4115064522 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f52a4066f90 0x7f52a4069450 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f52a80b1000 tx=0x7f52a80a9070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:07.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.005+0000 7f529ffff640 1 -- 172.21.15.83:0/4115064522 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f52b0056850 con 0x7f52b815db50 2024-04-17T10:49:07.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.189+0000 7f52bd964640 1 -- 172.21.15.83:0/4115064522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52b80b2af0 con 0x7f52b815db50 2024-04-17T10:49:07.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.189+0000 7f529ffff640 1 -- 172.21.15.83:0/4115064522 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f52b0056150 con 0x7f52b815db50 2024-04-17T10:49:07.187 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:07.188 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5786,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1553533838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1553533838},{"type":"v1","addr":"172.21.15.167:6833","nonce":1553533838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/674536913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":674536913},{"type":"v1","addr":"172.21.15.167:6835","nonce":674536913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8598,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4183890457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4183890457},{"type":"v1","addr":"172.21.15.83:6835","nonce":4183890457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:48:24.738443+0000","modified":"2024-04-17T10:48:26.776273+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8604":{"gid":8604,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1479417998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1479417998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1479417998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-17T10:49:07.188 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 69 2024-04-17T10:49:07.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.193+0000 7f52bd964640 1 -- 172.21.15.83:0/4115064522 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f52a4066f90 msgr2=0x7f52a4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:07.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.193+0000 7f52bd964640 1 --2- 172.21.15.83:0/4115064522 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f52a4066f90 0x7f52a4069450 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f52a80b1000 tx=0x7f52a80a9070 comp rx=0 tx=0).stop 2024-04-17T10:49:07.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.193+0000 7f52bd964640 1 -- 172.21.15.83:0/4115064522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52b815db50 msgr2=0x7f52b80abc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:07.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.193+0000 7f52bd964640 1 --2- 172.21.15.83:0/4115064522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52b815db50 0x7f52b80abc10 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f52b0007d40 tx=0x7f52b000e4b0 comp rx=0 tx=0).stop 2024-04-17T10:49:07.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.193+0000 7f52bd964640 1 -- 172.21.15.83:0/4115064522 shutdown_connections 2024-04-17T10:49:07.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.193+0000 7f52bd964640 1 --2- 172.21.15.83:0/4115064522 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f52a4066f90 0x7f52a4069450 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:07.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.193+0000 7f52bd964640 1 --2- 172.21.15.83:0/4115064522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52b815db50 0x7f52b80abc10 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:07.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.193+0000 7f52bd964640 1 --2- 172.21.15.83:0/4115064522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52b80aedd0 0x7f52b81572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:07.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.193+0000 7f52bd964640 1 --2- 172.21.15.83:0/4115064522 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52b806dcb0 0x7f52b81599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:07.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.193+0000 7f52bd964640 1 -- 172.21.15.83:0/4115064522 >> 172.21.15.83:0/4115064522 conn(0x7f52b806abe0 msgr2=0x7f52b806c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:07.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.193+0000 7f52bd964640 1 -- 172.21.15.83:0/4115064522 shutdown_connections 2024-04-17T10:49:07.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.193+0000 7f52bd964640 1 -- 172.21.15.83:0/4115064522 wait complete. 2024-04-17T10:49:07.205 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:49:07.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.325+0000 7f135d9aa640 1 Processor -- start 2024-04-17T10:49:07.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.325+0000 7f135d9aa640 1 -- start start 2024-04-17T10:49:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.329+0000 7f135d9aa640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f135815ba30 0x7f1358072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.329+0000 7f135d9aa640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f13580cfbe0 0x7f135806c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.329+0000 7f135d9aa640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f135806e320 con 0x7f135815c960 2024-04-17T10:49:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.329+0000 7f135d9aa640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1358071ea0 con 0x7f135815ba30 2024-04-17T10:49:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.329+0000 7f135d9aa640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f135806e040 con 0x7f13580cfbe0 2024-04-17T10:49:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.329+0000 7f1356ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f135815ba30 0x7f1358072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.329+0000 7f13567fc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f135815c960 0x7f13580cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:36572/0 (socket says 172.21.15.83:36572) 2024-04-17T10:49:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.329+0000 7f13567fc640 1 -- 172.21.15.83:0/4004502214 learned_addr learned my addr 172.21.15.83:0/4004502214 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.329+0000 7f13577fe640 1 --2- 172.21.15.83:0/4004502214 >> v2:172.21.15.167:3300/0 conn(0x7f13580cfbe0 0x7f135806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.329+0000 7f1356ffd640 1 -- 172.21.15.83:0/4004502214 >> v2:172.21.15.167:3300/0 conn(0x7f13580cfbe0 msgr2=0x7f135806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.329+0000 7f1356ffd640 1 --2- 172.21.15.83:0/4004502214 >> v2:172.21.15.167:3300/0 conn(0x7f13580cfbe0 0x7f135806c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.329+0000 7f1355ffb640 1 -- 172.21.15.83:0/4004502214 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1651318314 0 0) 0x7f135806e320 con 0x7f135815c960 2024-04-17T10:49:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.329+0000 7f1356ffd640 1 -- 172.21.15.83:0/4004502214 >> v1:172.21.15.83:6789/0 conn(0x7f135815c960 legacy=0x7f13580cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.329+0000 7f1356ffd640 1 -- 172.21.15.83:0/4004502214 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1358071850 con 0x7f135815ba30 2024-04-17T10:49:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.329+0000 7f13577fe640 1 --2- 172.21.15.83:0/4004502214 >> v2:172.21.15.167:3300/0 conn(0x7f13580cfbe0 0x7f135806c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.329+0000 7f1356ffd640 1 --2- 172.21.15.83:0/4004502214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f135815ba30 0x7f1358072330 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f13580676a0 tx=0x7f134c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b270610c96283756 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:07.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.329+0000 7f1355ffb640 1 -- 172.21.15.83:0/4004502214 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134c0b0070 con 0x7f135815ba30 2024-04-17T10:49:07.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.329+0000 7f1355ffb640 1 -- 172.21.15.83:0/4004502214 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f134c0ab600 con 0x7f135815ba30 2024-04-17T10:49:07.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.329+0000 7f1355ffb640 1 -- 172.21.15.83:0/4004502214 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134c0ab9d0 con 0x7f135815ba30 2024-04-17T10:49:07.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.333+0000 7f135d9aa640 1 -- 172.21.15.83:0/4004502214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f135815ba30 msgr2=0x7f1358072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:07.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.333+0000 7f135d9aa640 1 --2- 172.21.15.83:0/4004502214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f135815ba30 0x7f1358072330 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f13580676a0 tx=0x7f134c01c3f0 comp rx=0 tx=0).stop 2024-04-17T10:49:07.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.333+0000 7f135d9aa640 1 -- 172.21.15.83:0/4004502214 shutdown_connections 2024-04-17T10:49:07.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.333+0000 7f135d9aa640 1 --2- 172.21.15.83:0/4004502214 >> v2:172.21.15.167:3300/0 conn(0x7f13580cfbe0 0x7f135806c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:07.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.333+0000 7f135d9aa640 1 --2- 172.21.15.83:0/4004502214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f135815ba30 0x7f1358072330 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:07.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.333+0000 7f135d9aa640 1 -- 172.21.15.83:0/4004502214 >> 172.21.15.83:0/4004502214 conn(0x7f135806abe0 msgr2=0x7f135806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:07.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.333+0000 7f135d9aa640 1 -- 172.21.15.83:0/4004502214 shutdown_connections 2024-04-17T10:49:07.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.333+0000 7f135d9aa640 1 -- 172.21.15.83:0/4004502214 wait complete. 2024-04-17T10:49:07.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.333+0000 7f135d9aa640 1 Processor -- start 2024-04-17T10:49:07.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.333+0000 7f135d9aa640 1 -- start start 2024-04-17T10:49:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.333+0000 7f135d9aa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13580cfbe0 0x7f135814fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.333+0000 7f135d9aa640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f135815ba30 0x7f1358150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.333+0000 7f135d9aa640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f135815c960 0x7f135814d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.333+0000 7f135d9aa640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135806de20 con 0x7f13580cfbe0 2024-04-17T10:49:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.333+0000 7f135d9aa640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135806dca0 con 0x7f135815ba30 2024-04-17T10:49:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.333+0000 7f135d9aa640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1358168520 con 0x7f135815c960 2024-04-17T10:49:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.333+0000 7f1356ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13580cfbe0 0x7f135814fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.333+0000 7f1356ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13580cfbe0 0x7f135814fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45100/0 (socket says 172.21.15.83:45100) 2024-04-17T10:49:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.333+0000 7f1356ffd640 1 -- 172.21.15.83:0/2949306660 learned_addr learned my addr 172.21.15.83:0/2949306660 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.333+0000 7f1356ffd640 1 -- 172.21.15.83:0/2949306660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f135815c960 msgr2=0x7f135814d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.333+0000 7f1356ffd640 1 --2- 172.21.15.83:0/2949306660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f135815c960 0x7f135814d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.333+0000 7f1356ffd640 1 -- 172.21.15.83:0/2949306660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f135815ba30 msgr2=0x7f1358150220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.333+0000 7f1356ffd640 1 --2- 172.21.15.83:0/2949306660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f135815ba30 0x7f1358150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:07.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.333+0000 7f1356ffd640 1 -- 172.21.15.83:0/2949306660 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f135815d3e0 con 0x7f13580cfbe0 2024-04-17T10:49:07.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.333+0000 7f1356ffd640 1 --2- 172.21.15.83:0/2949306660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13580cfbe0 0x7f135814fce0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f134c0ae040 tx=0x7f134c01ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:07.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.333+0000 7f133ffff640 1 -- 172.21.15.83:0/2949306660 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134c0b0050 con 0x7f13580cfbe0 2024-04-17T10:49:07.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.333+0000 7f133ffff640 1 -- 172.21.15.83:0/2949306660 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f134c0bc070 con 0x7f13580cfbe0 2024-04-17T10:49:07.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.333+0000 7f135d9aa640 1 -- 172.21.15.83:0/2949306660 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f135814af40 con 0x7f13580cfbe0 2024-04-17T10:49:07.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.333+0000 7f133ffff640 1 -- 172.21.15.83:0/2949306660 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134c0b5a90 con 0x7f13580cfbe0 2024-04-17T10:49:07.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.333+0000 7f135d9aa640 1 -- 172.21.15.83:0/2949306660 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f135814b1e0 con 0x7f13580cfbe0 2024-04-17T10:49:07.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.337+0000 7f135d9aa640 1 -- 172.21.15.83:0/2949306660 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1358074f90 con 0x7f13580cfbe0 2024-04-17T10:49:07.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.337+0000 7f133ffff640 1 -- 172.21.15.83:0/2949306660 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f134c0b5c30 con 0x7f13580cfbe0 2024-04-17T10:49:07.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.337+0000 7f133ffff640 1 --2- 172.21.15.83:0/2949306660 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1344066f90 0x7f1344069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:07.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.337+0000 7f133ffff640 1 -- 172.21.15.83:0/2949306660 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f134c11fa60 con 0x7f13580cfbe0 2024-04-17T10:49:07.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.337+0000 7f13567fc640 1 --2- 172.21.15.83:0/2949306660 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1344066f90 0x7f1344069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:07.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.337+0000 7f13567fc640 1 --2- 172.21.15.83:0/2949306660 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1344066f90 0x7f1344069450 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f13480011f0 tx=0x7f1348007960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:07.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.341+0000 7f133ffff640 1 -- 172.21.15.83:0/2949306660 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f134c016070 con 0x7f13580cfbe0 2024-04-17T10:49:07.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.541+0000 7f135d9aa640 1 -- 172.21.15.83:0/2949306660 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f135814e710 con 0x7f13580cfbe0 2024-04-17T10:49:07.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.541+0000 7f133ffff640 1 -- 172.21.15.83:0/2949306660 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f134c0f0be0 con 0x7f13580cfbe0 2024-04-17T10:49:07.537 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:07.538 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5786,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1553533838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1553533838},{"type":"v1","addr":"172.21.15.167:6833","nonce":1553533838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/674536913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":674536913},{"type":"v1","addr":"172.21.15.167:6835","nonce":674536913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8598,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4183890457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4183890457},{"type":"v1","addr":"172.21.15.83:6835","nonce":4183890457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:48:24.738443+0000","modified":"2024-04-17T10:48:26.776273+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8604":{"gid":8604,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1479417998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1479417998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1479417998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-17T10:49:07.538 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 69 2024-04-17T10:49:07.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.541+0000 7f135d9aa640 1 -- 172.21.15.83:0/2949306660 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1344066f90 msgr2=0x7f1344069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:07.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.541+0000 7f135d9aa640 1 --2- 172.21.15.83:0/2949306660 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1344066f90 0x7f1344069450 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f13480011f0 tx=0x7f1348007960 comp rx=0 tx=0).stop 2024-04-17T10:49:07.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.541+0000 7f135d9aa640 1 -- 172.21.15.83:0/2949306660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13580cfbe0 msgr2=0x7f135814fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:07.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.541+0000 7f135d9aa640 1 --2- 172.21.15.83:0/2949306660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13580cfbe0 0x7f135814fce0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f134c0ae040 tx=0x7f134c01ac30 comp rx=0 tx=0).stop 2024-04-17T10:49:07.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.541+0000 7f135d9aa640 1 -- 172.21.15.83:0/2949306660 shutdown_connections 2024-04-17T10:49:07.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.541+0000 7f135d9aa640 1 --2- 172.21.15.83:0/2949306660 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1344066f90 0x7f1344069450 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:07.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.541+0000 7f135d9aa640 1 --2- 172.21.15.83:0/2949306660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f135815c960 0x7f135814d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:07.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.541+0000 7f135d9aa640 1 --2- 172.21.15.83:0/2949306660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f135815ba30 0x7f1358150220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:07.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.541+0000 7f135d9aa640 1 --2- 172.21.15.83:0/2949306660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13580cfbe0 0x7f135814fce0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:07.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.541+0000 7f135d9aa640 1 -- 172.21.15.83:0/2949306660 >> 172.21.15.83:0/2949306660 conn(0x7f135806abe0 msgr2=0x7f135815ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:07.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.541+0000 7f135d9aa640 1 -- 172.21.15.83:0/2949306660 shutdown_connections 2024-04-17T10:49:07.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.541+0000 7f135d9aa640 1 -- 172.21.15.83:0/2949306660 wait complete. 2024-04-17T10:49:07.556 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:49:07.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.681+0000 7fb2fec99640 1 Processor -- start 2024-04-17T10:49:07.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.681+0000 7fb2fec99640 1 -- start start 2024-04-17T10:49:07.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.681+0000 7fb2fec99640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb2f80ce7f0 0x7fb2f8168dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:07.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.681+0000 7fb2fec99640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2f815f6c0 con 0x7fb2f815de90 2024-04-17T10:49:07.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.681+0000 7fb2fec99640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2f806bd90 con 0x7fb2f80ceca0 2024-04-17T10:49:07.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.681+0000 7fb2fec99640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2f806c030 con 0x7fb2f80ce7f0 2024-04-17T10:49:07.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.681+0000 7fb2fca35640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb2f815de90 0x7fb2f815efb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:36582/0 (socket says 172.21.15.83:36582) 2024-04-17T10:49:07.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.681+0000 7fb2fca35640 1 -- 172.21.15.83:0/3334501104 learned_addr learned my addr 172.21.15.83:0/3334501104 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:07.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.681+0000 7fb2fd236640 1 --2- 172.21.15.83:0/3334501104 >> v2:172.21.15.83:3300/0 conn(0x7fb2f80ce7f0 0x7fb2f8168dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:07.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.681+0000 7fb2f77fe640 1 -- 172.21.15.83:0/3334501104 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1437742721 0 0) 0x7fb2ec001120 con 0x7fb2f815de90 2024-04-17T10:49:07.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.681+0000 7fb2f77fe640 1 -- 172.21.15.83:0/3334501104 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2e8003530 con 0x7fb2f815de90 2024-04-17T10:49:07.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.681+0000 7fb2fd236640 1 -- 172.21.15.83:0/3334501104 >> v1:172.21.15.167:6789/0 conn(0x7fb2f80ceca0 legacy=0x7fb2f80ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:07.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.681+0000 7fb2fd236640 1 -- 172.21.15.83:0/3334501104 >> v1:172.21.15.83:6789/0 conn(0x7fb2f815de90 legacy=0x7fb2f815efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:07.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.681+0000 7fb2fd236640 1 -- 172.21.15.83:0/3334501104 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f806c310 con 0x7fb2f80ce7f0 2024-04-17T10:49:07.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2fd236640 1 --2- 172.21.15.83:0/3334501104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2f80ce7f0 0x7fb2f8168dd0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0009d90 tx=0x7fb2f001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdcbcb4345ef6df3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:07.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2f77fe640 1 -- 172.21.15.83:0/3334501104 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f00b2070 con 0x7fb2f80ce7f0 2024-04-17T10:49:07.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2f77fe640 1 -- 172.21.15.83:0/3334501104 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f00a59c0 con 0x7fb2f80ce7f0 2024-04-17T10:49:07.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2f77fe640 1 -- 172.21.15.83:0/3334501104 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f00a5d90 con 0x7fb2f80ce7f0 2024-04-17T10:49:07.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2fec99640 1 -- 172.21.15.83:0/3334501104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2f80ce7f0 msgr2=0x7fb2f8168dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:07.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2fec99640 1 --2- 172.21.15.83:0/3334501104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2f80ce7f0 0x7fb2f8168dd0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0009d90 tx=0x7fb2f001c900 comp rx=0 tx=0).stop 2024-04-17T10:49:07.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2fec99640 1 -- 172.21.15.83:0/3334501104 shutdown_connections 2024-04-17T10:49:07.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2fec99640 1 --2- 172.21.15.83:0/3334501104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2f80ce7f0 0x7fb2f8168dd0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:07.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2fec99640 1 -- 172.21.15.83:0/3334501104 >> 172.21.15.83:0/3334501104 conn(0x7fb2f80d17b0 msgr2=0x7fb2f80d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:07.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2fec99640 1 -- 172.21.15.83:0/3334501104 shutdown_connections 2024-04-17T10:49:07.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2fec99640 1 -- 172.21.15.83:0/3334501104 wait complete. 2024-04-17T10:49:07.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2fec99640 1 Processor -- start 2024-04-17T10:49:07.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2fec99640 1 -- start start 2024-04-17T10:49:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2fec99640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2f80ce7f0 0x7fb2f8152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2fec99640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2f80ceca0 0x7fb2f80b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2fec99640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2f815de90 0x7fb2f815f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2fec99640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f806cdf0 con 0x7fb2f80ce7f0 2024-04-17T10:49:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2fec99640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f815e270 con 0x7fb2f815de90 2024-04-17T10:49:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2fec99640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f80c80c0 con 0x7fb2f80ceca0 2024-04-17T10:49:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2fca35640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2f80ce7f0 0x7fb2f8152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2fca35640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2f80ce7f0 0x7fb2f8152b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45122/0 (socket says 172.21.15.83:45122) 2024-04-17T10:49:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2fca35640 1 -- 172.21.15.83:0/2706632338 learned_addr learned my addr 172.21.15.83:0/2706632338 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2fca35640 1 -- 172.21.15.83:0/2706632338 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2f80ceca0 msgr2=0x7fb2f80b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2fca35640 1 --2- 172.21.15.83:0/2706632338 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2f80ceca0 0x7fb2f80b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2fca35640 1 -- 172.21.15.83:0/2706632338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2f815de90 msgr2=0x7fb2f815f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2fca35640 1 --2- 172.21.15.83:0/2706632338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2f815de90 0x7fb2f815f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2fca35640 1 -- 172.21.15.83:0/2706632338 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f80c6480 con 0x7fb2f80ce7f0 2024-04-17T10:49:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2fca35640 1 --2- 172.21.15.83:0/2706632338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2f80ce7f0 0x7fb2f8152b20 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec00c760 tx=0x7fb2ec00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2f57fa640 1 -- 172.21.15.83:0/2706632338 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec017070 con 0x7fb2f80ce7f0 2024-04-17T10:49:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2f57fa640 1 -- 172.21.15.83:0/2706632338 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ec005ae0 con 0x7fb2f80ce7f0 2024-04-17T10:49:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2fec99640 1 -- 172.21.15.83:0/2706632338 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f80b5030 con 0x7fb2f80ce7f0 2024-04-17T10:49:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2f57fa640 1 -- 172.21.15.83:0/2706632338 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec007e30 con 0x7fb2f80ce7f0 2024-04-17T10:49:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.685+0000 7fb2fec99640 1 -- 172.21.15.83:0/2706632338 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f815fee0 con 0x7fb2f80ce7f0 2024-04-17T10:49:07.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.689+0000 7fb2fec99640 1 -- 172.21.15.83:0/2706632338 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2f8074f90 con 0x7fb2f80ce7f0 2024-04-17T10:49:07.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.689+0000 7fb2f57fa640 1 -- 172.21.15.83:0/2706632338 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb2ec01a070 con 0x7fb2f80ce7f0 2024-04-17T10:49:07.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.689+0000 7fb2f57fa640 1 --2- 172.21.15.83:0/2706632338 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb2dc066f60 0x7fb2dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:07.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.689+0000 7fb2f57fa640 1 -- 172.21.15.83:0/2706632338 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb2ec0843a0 con 0x7fb2f80ce7f0 2024-04-17T10:49:07.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.689+0000 7fb2f7fff640 1 --2- 172.21.15.83:0/2706632338 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb2dc066f60 0x7fb2dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:07.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.689+0000 7fb2f7fff640 1 --2- 172.21.15.83:0/2706632338 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb2dc066f60 0x7fb2dc069420 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb2f80c63b0 tx=0x7fb2e4006bd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:07.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.693+0000 7fb2f57fa640 1 -- 172.21.15.83:0/2706632338 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb2f815f6c0 con 0x7fb2f80ce7f0 2024-04-17T10:49:07.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.853+0000 7fb2fec99640 1 -- 172.21.15.83:0/2706632338 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2f8072690 con 0x7fb2f80ce7f0 2024-04-17T10:49:07.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.853+0000 7fb2f57fa640 1 -- 172.21.15.83:0/2706632338 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+16649 (secure 0 0 0) 0x7fb2ec055400 con 0x7fb2f80ce7f0 2024-04-17T10:49:07.850 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:07.851 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":116,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:48:55.454382+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:48:22.981915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-17T10:48:23.700138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.18","mappings":[{"from":0,"to":5}]},{"pgid":"17.34","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:49:07.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.857+0000 7fb2fec99640 1 -- 172.21.15.83:0/2706632338 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb2dc066f60 msgr2=0x7fb2dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:07.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.857+0000 7fb2fec99640 1 --2- 172.21.15.83:0/2706632338 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb2dc066f60 0x7fb2dc069420 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb2f80c63b0 tx=0x7fb2e4006bd0 comp rx=0 tx=0).stop 2024-04-17T10:49:07.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.857+0000 7fb2fec99640 1 -- 172.21.15.83:0/2706632338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2f80ce7f0 msgr2=0x7fb2f8152b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:07.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.857+0000 7fb2fec99640 1 --2- 172.21.15.83:0/2706632338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2f80ce7f0 0x7fb2f8152b20 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec00c760 tx=0x7fb2ec00cc20 comp rx=0 tx=0).stop 2024-04-17T10:49:07.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.857+0000 7fb2fec99640 1 -- 172.21.15.83:0/2706632338 shutdown_connections 2024-04-17T10:49:07.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.857+0000 7fb2fec99640 1 --2- 172.21.15.83:0/2706632338 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb2dc066f60 0x7fb2dc069420 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:07.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.857+0000 7fb2fec99640 1 --2- 172.21.15.83:0/2706632338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2f815de90 0x7fb2f815f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:07.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.857+0000 7fb2fec99640 1 --2- 172.21.15.83:0/2706632338 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2f80ceca0 0x7fb2f80b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:07.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.857+0000 7fb2fec99640 1 --2- 172.21.15.83:0/2706632338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2f80ce7f0 0x7fb2f8152b20 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:07.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.857+0000 7fb2fec99640 1 -- 172.21.15.83:0/2706632338 >> 172.21.15.83:0/2706632338 conn(0x7fb2f80d17b0 msgr2=0x7fb2f8153690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:07.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.857+0000 7fb2fec99640 1 -- 172.21.15.83:0/2706632338 shutdown_connections 2024-04-17T10:49:07.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.857+0000 7fb2fec99640 1 -- 172.21.15.83:0/2706632338 wait complete. 2024-04-17T10:49:07.870 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-17T10:49:07.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.993+0000 7f8833932640 1 Processor -- start 2024-04-17T10:49:07.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.993+0000 7f8833932640 1 -- start start 2024-04-17T10:49:07.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.997+0000 7f8833932640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f882c0b5b60 0x7f882c0b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:07.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.997+0000 7f8833932640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f882c15e4d0 con 0x7f882c15e0f0 2024-04-17T10:49:07.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.997+0000 7f8833932640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f882c162580 con 0x7f882c0ce400 2024-04-17T10:49:07.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.997+0000 7f8833932640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f882c162b40 con 0x7f882c0b5b60 2024-04-17T10:49:07.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.997+0000 7f88316ce640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f882c15e0f0 0x7f882c161c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:36590/0 (socket says 172.21.15.83:36590) 2024-04-17T10:49:07.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.997+0000 7f88316ce640 1 -- 172.21.15.83:0/2126814631 learned_addr learned my addr 172.21.15.83:0/2126814631 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:07.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.997+0000 7f8831ecf640 1 --2- 172.21.15.83:0/2126814631 >> v2:172.21.15.83:3300/0 conn(0x7f882c0b5b60 0x7f882c0b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:07.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.997+0000 7f8831ecf640 1 -- 172.21.15.83:0/2126814631 >> v1:172.21.15.167:6789/0 conn(0x7f882c0ce400 legacy=0x7f882c0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:07.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.997+0000 7f8831ecf640 1 -- 172.21.15.83:0/2126814631 >> v1:172.21.15.83:6789/0 conn(0x7f882c15e0f0 legacy=0x7f882c161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:07.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.997+0000 7f8831ecf640 1 -- 172.21.15.83:0/2126814631 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f882c162860 con 0x7f882c0b5b60 2024-04-17T10:49:07.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.997+0000 7f8831ecf640 1 --2- 172.21.15.83:0/2126814631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f882c0b5b60 0x7f882c0b6d00 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f8824009d90 tx=0x7f882401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=101d0ef7070f0a94 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:07.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.997+0000 7f8823fff640 1 -- 172.21.15.83:0/2126814631 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88240b2070 con 0x7f882c0b5b60 2024-04-17T10:49:07.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.997+0000 7f8823fff640 1 -- 172.21.15.83:0/2126814631 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88240a5760 con 0x7f882c0b5b60 2024-04-17T10:49:07.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:07.997+0000 7f8823fff640 1 -- 172.21.15.83:0/2126814631 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88240a5b30 con 0x7f882c0b5b60 2024-04-17T10:49:07.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.001+0000 7f8833932640 1 -- 172.21.15.83:0/2126814631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f882c0b5b60 msgr2=0x7f882c0b6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:07.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.001+0000 7f8833932640 1 --2- 172.21.15.83:0/2126814631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f882c0b5b60 0x7f882c0b6d00 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f8824009d90 tx=0x7f882401c8a0 comp rx=0 tx=0).stop 2024-04-17T10:49:07.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.001+0000 7f8833932640 1 -- 172.21.15.83:0/2126814631 shutdown_connections 2024-04-17T10:49:07.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.001+0000 7f8833932640 1 --2- 172.21.15.83:0/2126814631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f882c0b5b60 0x7f882c0b6d00 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:07.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.001+0000 7f8833932640 1 -- 172.21.15.83:0/2126814631 >> 172.21.15.83:0/2126814631 conn(0x7f882c0d0450 msgr2=0x7f882c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:07.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.001+0000 7f8833932640 1 -- 172.21.15.83:0/2126814631 shutdown_connections 2024-04-17T10:49:07.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.001+0000 7f8833932640 1 -- 172.21.15.83:0/2126814631 wait complete. 2024-04-17T10:49:07.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.001+0000 7f8833932640 1 Processor -- start 2024-04-17T10:49:07.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.001+0000 7f8833932640 1 -- start start 2024-04-17T10:49:08.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.001+0000 7f8833932640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f882c0b5b60 0x7f882c157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:08.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.001+0000 7f8833932640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f882c0ce400 0x7f882c1fa1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:08.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.001+0000 7f8833932640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f882c15e0f0 0x7f882c1fa710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:08.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.001+0000 7f8833932640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882c0d3120 con 0x7f882c15e0f0 2024-04-17T10:49:08.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.001+0000 7f8833932640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882c0d2fa0 con 0x7f882c0b5b60 2024-04-17T10:49:08.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.001+0000 7f8833932640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f882c0d32a0 con 0x7f882c0ce400 2024-04-17T10:49:08.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.001+0000 7f8831ecf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f882c15e0f0 0x7f882c1fa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:08.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.001+0000 7f8831ecf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f882c15e0f0 0x7f882c1fa710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45140/0 (socket says 172.21.15.83:45140) 2024-04-17T10:49:08.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.001+0000 7f8831ecf640 1 -- 172.21.15.83:0/1991557426 learned_addr learned my addr 172.21.15.83:0/1991557426 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:08.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.001+0000 7f88316ce640 1 --2- 172.21.15.83:0/1991557426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f882c0b5b60 0x7f882c157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:08.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.001+0000 7f8830ecd640 1 --2- 172.21.15.83:0/1991557426 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f882c0ce400 0x7f882c1fa1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:08.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.001+0000 7f8831ecf640 1 -- 172.21.15.83:0/1991557426 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f882c0ce400 msgr2=0x7f882c1fa1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:08.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.001+0000 7f8831ecf640 1 --2- 172.21.15.83:0/1991557426 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f882c0ce400 0x7f882c1fa1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:08.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.001+0000 7f8831ecf640 1 -- 172.21.15.83:0/1991557426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f882c0b5b60 msgr2=0x7f882c157170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:08.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.001+0000 7f8831ecf640 1 --2- 172.21.15.83:0/1991557426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f882c0b5b60 0x7f882c157170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:08.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.001+0000 7f8831ecf640 1 -- 172.21.15.83:0/1991557426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f882c154d70 con 0x7f882c15e0f0 2024-04-17T10:49:08.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.001+0000 7f8830ecd640 1 --2- 172.21.15.83:0/1991557426 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f882c0ce400 0x7f882c1fa1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:08.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.001+0000 7f8831ecf640 1 --2- 172.21.15.83:0/1991557426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f882c15e0f0 0x7f882c1fa710 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f882401a4d0 tx=0x7f8824019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:08.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.001+0000 7f8821ffb640 1 -- 172.21.15.83:0/1991557426 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88240b2050 con 0x7f882c15e0f0 2024-04-17T10:49:08.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.001+0000 7f8821ffb640 1 -- 172.21.15.83:0/1991557426 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88240be070 con 0x7f882c15e0f0 2024-04-17T10:49:08.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.001+0000 7f8833932640 1 -- 172.21.15.83:0/1991557426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f882c155050 con 0x7f882c15e0f0 2024-04-17T10:49:08.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.001+0000 7f8821ffb640 1 -- 172.21.15.83:0/1991557426 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88240ada50 con 0x7f882c15e0f0 2024-04-17T10:49:08.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.001+0000 7f8833932640 1 -- 172.21.15.83:0/1991557426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f882c155610 con 0x7f882c15e0f0 2024-04-17T10:49:08.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.005+0000 7f8833932640 1 -- 172.21.15.83:0/1991557426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f882c0681a0 con 0x7f882c15e0f0 2024-04-17T10:49:08.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.005+0000 7f8821ffb640 1 -- 172.21.15.83:0/1991557426 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f88240adbf0 con 0x7f882c15e0f0 2024-04-17T10:49:08.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.005+0000 7f8821ffb640 1 --2- 172.21.15.83:0/1991557426 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8810066f60 0x7f8810069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:08.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.005+0000 7f8821ffb640 1 -- 172.21.15.83:0/1991557426 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f88241201f0 con 0x7f882c15e0f0 2024-04-17T10:49:08.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.005+0000 7f88316ce640 1 --2- 172.21.15.83:0/1991557426 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8810066f60 0x7f8810069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:08.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.005+0000 7f88316ce640 1 --2- 172.21.15.83:0/1991557426 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8810066f60 0x7f8810069420 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f881c000f70 tx=0x7f881c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:08.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.009+0000 7f8821ffb640 1 -- 172.21.15.83:0/1991557426 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8824016520 con 0x7f882c15e0f0 2024-04-17T10:49:08.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.201+0000 7f8833932640 1 -- 172.21.15.83:0/1991557426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f882c074f90 con 0x7f882c15e0f0 2024-04-17T10:49:08.198 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-04-17T10:49:08.201+0000 7f35c3b81640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T10:49:08.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.205+0000 7f8821ffb640 1 -- 172.21.15.83:0/1991557426 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v70) v1 ==== 161+0+0 (secure 0 0 0) 0x7f88240f1250 con 0x7f882c15e0f0 2024-04-17T10:49:08.201 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-17T10:49:08.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.205+0000 7f8833932640 1 -- 172.21.15.83:0/1991557426 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8810066f60 msgr2=0x7f8810069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:08.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.205+0000 7f8833932640 1 --2- 172.21.15.83:0/1991557426 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8810066f60 0x7f8810069420 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f881c000f70 tx=0x7f881c009040 comp rx=0 tx=0).stop 2024-04-17T10:49:08.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.205+0000 7f8833932640 1 -- 172.21.15.83:0/1991557426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f882c15e0f0 msgr2=0x7f882c1fa710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:08.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.205+0000 7f8833932640 1 --2- 172.21.15.83:0/1991557426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f882c15e0f0 0x7f882c1fa710 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f882401a4d0 tx=0x7f8824019640 comp rx=0 tx=0).stop 2024-04-17T10:49:08.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.205+0000 7f8833932640 1 -- 172.21.15.83:0/1991557426 shutdown_connections 2024-04-17T10:49:08.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.205+0000 7f8833932640 1 --2- 172.21.15.83:0/1991557426 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8810066f60 0x7f8810069420 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:08.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.205+0000 7f8833932640 1 --2- 172.21.15.83:0/1991557426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f882c15e0f0 0x7f882c1fa710 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:08.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.205+0000 7f8833932640 1 --2- 172.21.15.83:0/1991557426 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f882c0ce400 0x7f882c1fa1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:08.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.205+0000 7f8833932640 1 --2- 172.21.15.83:0/1991557426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f882c0b5b60 0x7f882c157170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:08.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.205+0000 7f8833932640 1 -- 172.21.15.83:0/1991557426 >> 172.21.15.83:0/1991557426 conn(0x7f882c0d0450 msgr2=0x7f882c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:08.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.205+0000 7f8833932640 1 -- 172.21.15.83:0/1991557426 shutdown_connections 2024-04-17T10:49:08.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.205+0000 7f8833932640 1 -- 172.21.15.83:0/1991557426 wait complete. 2024-04-17T10:49:08.221 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-17T10:49:08.248 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-04-17T10:49:08.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.345+0000 7f97d428e640 1 Processor -- start 2024-04-17T10:49:08.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.345+0000 7f97d428e640 1 -- start start 2024-04-17T10:49:08.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.345+0000 7f97d428e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f97cc15b6e0 0x7f97cc071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:08.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.345+0000 7f97d428e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97cc15bac0 0x7f97cc0cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:08.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.345+0000 7f97d428e640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97cc06c1f0 con 0x7f97cc070470 2024-04-17T10:49:08.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.345+0000 7f97d428e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97cc0cdd30 con 0x7f97cc15b6e0 2024-04-17T10:49:08.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.345+0000 7f97d428e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc070a10 con 0x7f97cc15bac0 2024-04-17T10:49:08.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.345+0000 7f97d202a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f97cc15b6e0 0x7f97cc071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:08.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.345+0000 7f97d202a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f97cc15b6e0 0x7f97cc071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45150/0 (socket says 172.21.15.83:45150) 2024-04-17T10:49:08.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.345+0000 7f97d202a640 1 -- 172.21.15.83:0/2377927753 learned_addr learned my addr 172.21.15.83:0/2377927753 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:08.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.345+0000 7f97d1829640 1 --2- 172.21.15.83:0/2377927753 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97cc15bac0 0x7f97cc0cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:08.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.345+0000 7f97d202a640 1 -- 172.21.15.83:0/2377927753 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97cc15bac0 msgr2=0x7f97cc0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:08.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.345+0000 7f97d202a640 1 --2- 172.21.15.83:0/2377927753 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97cc15bac0 0x7f97cc0cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:08.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.345+0000 7f97d202a640 1 -- 172.21.15.83:0/2377927753 >> v1:172.21.15.167:6789/0 conn(0x7f97cc070470 legacy=0x7f97cc06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:08.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.345+0000 7f97d202a640 1 -- 172.21.15.83:0/2377927753 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97cc070cf0 con 0x7f97cc15b6e0 2024-04-17T10:49:08.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.345+0000 7f97d1829640 1 --2- 172.21.15.83:0/2377927753 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97cc15bac0 0x7f97cc0cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:08.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.349+0000 7f97d202a640 1 --2- 172.21.15.83:0/2377927753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97cc15b6e0 0x7f97cc071cc0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f97cc0728a0 tx=0x7f97bc01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6f417a1aabb251 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:08.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.349+0000 7f97d1028640 1 -- 172.21.15.83:0/2377927753 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc0b0070 con 0x7f97cc15b6e0 2024-04-17T10:49:08.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.349+0000 7f97d1028640 1 -- 172.21.15.83:0/2377927753 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97bc0ab600 con 0x7f97cc15b6e0 2024-04-17T10:49:08.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.349+0000 7f97d1028640 1 -- 172.21.15.83:0/2377927753 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc0ab9d0 con 0x7f97cc15b6e0 2024-04-17T10:49:08.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.349+0000 7f97d428e640 1 -- 172.21.15.83:0/2377927753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97cc15b6e0 msgr2=0x7f97cc071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:08.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.349+0000 7f97d428e640 1 --2- 172.21.15.83:0/2377927753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97cc15b6e0 0x7f97cc071cc0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f97cc0728a0 tx=0x7f97bc01c4e0 comp rx=0 tx=0).stop 2024-04-17T10:49:08.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.349+0000 7f97d428e640 1 -- 172.21.15.83:0/2377927753 shutdown_connections 2024-04-17T10:49:08.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.349+0000 7f97d428e640 1 --2- 172.21.15.83:0/2377927753 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97cc15bac0 0x7f97cc0cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:08.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.349+0000 7f97d428e640 1 --2- 172.21.15.83:0/2377927753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97cc15b6e0 0x7f97cc071cc0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:08.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.349+0000 7f97d428e640 1 -- 172.21.15.83:0/2377927753 >> 172.21.15.83:0/2377927753 conn(0x7f97cc0b15b0 msgr2=0x7f97cc0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:08.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.349+0000 7f97d428e640 1 -- 172.21.15.83:0/2377927753 shutdown_connections 2024-04-17T10:49:08.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.349+0000 7f97d428e640 1 -- 172.21.15.83:0/2377927753 wait complete. 2024-04-17T10:49:08.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.349+0000 7f97d428e640 1 Processor -- start 2024-04-17T10:49:08.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.349+0000 7f97d428e640 1 -- start start 2024-04-17T10:49:08.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.353+0000 7f97d428e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97cc070470 0x7f97cc13a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:08.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.353+0000 7f97d428e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97cc15b6e0 0x7f97cc158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:08.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.353+0000 7f97d428e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97cc15bac0 0x7f97cc159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:08.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.353+0000 7f97d428e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc06cee0 con 0x7f97cc15bac0 2024-04-17T10:49:08.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.353+0000 7f97d428e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc06b730 con 0x7f97cc15b6e0 2024-04-17T10:49:08.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.353+0000 7f97d428e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc06d060 con 0x7f97cc070470 2024-04-17T10:49:08.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.353+0000 7f97d282b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97cc15bac0 0x7f97cc159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:08.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.353+0000 7f97d282b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97cc15bac0 0x7f97cc159110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45166/0 (socket says 172.21.15.83:45166) 2024-04-17T10:49:08.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.353+0000 7f97d282b640 1 -- 172.21.15.83:0/286120800 learned_addr learned my addr 172.21.15.83:0/286120800 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:08.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.353+0000 7f97d1829640 1 --2- 172.21.15.83:0/286120800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97cc15b6e0 0x7f97cc158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:08.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.353+0000 7f97d202a640 1 --2- 172.21.15.83:0/286120800 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97cc070470 0x7f97cc13a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:08.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.353+0000 7f97d282b640 1 -- 172.21.15.83:0/286120800 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97cc070470 msgr2=0x7f97cc13a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:08.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.353+0000 7f97d282b640 1 --2- 172.21.15.83:0/286120800 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97cc070470 0x7f97cc13a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:08.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.353+0000 7f97d282b640 1 -- 172.21.15.83:0/286120800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97cc15b6e0 msgr2=0x7f97cc158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:08.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.353+0000 7f97d282b640 1 --2- 172.21.15.83:0/286120800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97cc15b6e0 0x7f97cc158bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:08.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.353+0000 7f97d282b640 1 -- 172.21.15.83:0/286120800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97cc15e890 con 0x7f97cc15bac0 2024-04-17T10:49:08.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.353+0000 7f97d1829640 1 --2- 172.21.15.83:0/286120800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97cc15b6e0 0x7f97cc158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:08.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.353+0000 7f97d282b640 1 --2- 172.21.15.83:0/286120800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97cc15bac0 0x7f97cc159110 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f97c400c780 tx=0x7f97c400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:08.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.353+0000 7f97c2ffd640 1 -- 172.21.15.83:0/286120800 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c4004b70 con 0x7f97cc15bac0 2024-04-17T10:49:08.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.353+0000 7f97c2ffd640 1 -- 172.21.15.83:0/286120800 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c4004d10 con 0x7f97cc15bac0 2024-04-17T10:49:08.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.353+0000 7f97d428e640 1 -- 172.21.15.83:0/286120800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97cc154120 con 0x7f97cc15bac0 2024-04-17T10:49:08.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.353+0000 7f97c2ffd640 1 -- 172.21.15.83:0/286120800 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c400aa90 con 0x7f97cc15bac0 2024-04-17T10:49:08.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.353+0000 7f97d428e640 1 -- 172.21.15.83:0/286120800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97cc1546e0 con 0x7f97cc15bac0 2024-04-17T10:49:08.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.353+0000 7f97d428e640 1 -- 172.21.15.83:0/286120800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97cc074f90 con 0x7f97cc15bac0 2024-04-17T10:49:08.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.353+0000 7f97c2ffd640 1 -- 172.21.15.83:0/286120800 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f97c400ac30 con 0x7f97cc15bac0 2024-04-17T10:49:08.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.353+0000 7f97c2ffd640 1 --2- 172.21.15.83:0/286120800 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f97b0066f90 0x7f97b0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:08.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.353+0000 7f97c2ffd640 1 -- 172.21.15.83:0/286120800 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f97c4085310 con 0x7f97cc15bac0 2024-04-17T10:49:08.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.353+0000 7f97d202a640 1 --2- 172.21.15.83:0/286120800 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f97b0066f90 0x7f97b0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:08.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.353+0000 7f97d202a640 1 --2- 172.21.15.83:0/286120800 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f97b0066f90 0x7f97b0069450 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0ad000 tx=0x7f97bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:08.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.357+0000 7f97c2ffd640 1 -- 172.21.15.83:0/286120800 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f97c4020070 con 0x7f97cc15bac0 2024-04-17T10:49:08.474 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:49:08.476 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:49:08.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:08.549+0000 7f97d428e640 1 -- 172.21.15.83:0/286120800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f97cc0b1990 con 0x7f97cc15bac0 2024-04-17T10:49:09.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.213+0000 7f97c2ffd640 1 -- 172.21.15.83:0/286120800 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v71) v1 ==== 104+0+0 (secure 0 0 0) 0x7f97c4056290 con 0x7f97cc15bac0 2024-04-17T10:49:09.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.217+0000 7f97d428e640 1 -- 172.21.15.83:0/286120800 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f97b0066f90 msgr2=0x7f97b0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:09.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.217+0000 7f97d428e640 1 --2- 172.21.15.83:0/286120800 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f97b0066f90 0x7f97b0069450 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0ad000 tx=0x7f97bc008040 comp rx=0 tx=0).stop 2024-04-17T10:49:09.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.217+0000 7f97d428e640 1 -- 172.21.15.83:0/286120800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97cc15bac0 msgr2=0x7f97cc159110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:09.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.217+0000 7f97d428e640 1 --2- 172.21.15.83:0/286120800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97cc15bac0 0x7f97cc159110 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f97c400c780 tx=0x7f97c400cc40 comp rx=0 tx=0).stop 2024-04-17T10:49:09.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.217+0000 7f97d428e640 1 -- 172.21.15.83:0/286120800 shutdown_connections 2024-04-17T10:49:09.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.217+0000 7f97d428e640 1 --2- 172.21.15.83:0/286120800 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f97b0066f90 0x7f97b0069450 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:09.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.217+0000 7f97d428e640 1 --2- 172.21.15.83:0/286120800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97cc15bac0 0x7f97cc159110 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:09.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.217+0000 7f97d428e640 1 --2- 172.21.15.83:0/286120800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97cc15b6e0 0x7f97cc158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:09.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.217+0000 7f97d428e640 1 --2- 172.21.15.83:0/286120800 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97cc070470 0x7f97cc13a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:09.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.217+0000 7f97d428e640 1 -- 172.21.15.83:0/286120800 >> 172.21.15.83:0/286120800 conn(0x7f97cc0b15b0 msgr2=0x7f97cc06ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:09.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.217+0000 7f97d428e640 1 -- 172.21.15.83:0/286120800 shutdown_connections 2024-04-17T10:49:09.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.217+0000 7f97d428e640 1 -- 172.21.15.83:0/286120800 wait complete. 2024-04-17T10:49:09.237 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-17T10:49:09.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.365+0000 7f09d7b44640 1 Processor -- start 2024-04-17T10:49:09.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.365+0000 7f09d7b44640 1 -- start start 2024-04-17T10:49:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.365+0000 7f09d7b44640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f09d006ee50 0x7f09d00cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.365+0000 7f09d7b44640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09d00d04f0 con 0x7f09d015f120 2024-04-17T10:49:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.365+0000 7f09d7b44640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09d00707f0 con 0x7f09d00cef20 2024-04-17T10:49:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.365+0000 7f09d7b44640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d00cfd80 con 0x7f09d006ee50 2024-04-17T10:49:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.365+0000 7f09d58e0640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f09d015f120 0x7f09d006e740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:36608/0 (socket says 172.21.15.83:36608) 2024-04-17T10:49:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.365+0000 7f09d58e0640 1 -- 172.21.15.83:0/3144045963 learned_addr learned my addr 172.21.15.83:0/3144045963 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:09.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.365+0000 7f09d48de640 1 -- 172.21.15.83:0/3144045963 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 152934217 0 0) 0x7f09d00d04f0 con 0x7f09d015f120 2024-04-17T10:49:09.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.365+0000 7f09d60e1640 1 --2- 172.21.15.83:0/3144045963 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f09d006ee50 0x7f09d00cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:09.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d48de640 1 -- 172.21.15.83:0/3144045963 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09bc003530 con 0x7f09d015f120 2024-04-17T10:49:09.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d48de640 1 -- 172.21.15.83:0/3144045963 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2483532020 0 0) 0x7f09bc003530 con 0x7f09d015f120 2024-04-17T10:49:09.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d48de640 1 -- 172.21.15.83:0/3144045963 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f09d006ee50 msgr2=0x7f09d00cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:09.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d48de640 1 --2- 172.21.15.83:0/3144045963 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f09d006ee50 0x7f09d00cd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:09.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d48de640 1 -- 172.21.15.83:0/3144045963 >> v1:172.21.15.167:6789/0 conn(0x7f09d00cef20 legacy=0x7f09d00ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:09.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d48de640 1 -- 172.21.15.83:0/3144045963 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d006f380 con 0x7f09d015f120 2024-04-17T10:49:09.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d48de640 1 -- 172.21.15.83:0/3144045963 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f09cc001680 con 0x7f09d015f120 2024-04-17T10:49:09.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d48de640 1 -- 172.21.15.83:0/3144045963 >> v1:172.21.15.83:6789/0 conn(0x7f09d015f120 legacy=0x7f09d006e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:09.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d48de640 1 --2- 172.21.15.83:0/3144045963 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f09d006ee50 0x7f09bc0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:09.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d48de640 1 --2- 172.21.15.83:0/3144045963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09bc003f70 0x7f09bc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:09.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d48de640 1 --2- 172.21.15.83:0/3144045963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09bc004d50 0x7f09bc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:09.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d48de640 1 -- 172.21.15.83:0/3144045963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d00cfd80 con 0x7f09bc003f70 2024-04-17T10:49:09.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d48de640 1 -- 172.21.15.83:0/3144045963 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc00bab0 con 0x7f09bc004d50 2024-04-17T10:49:09.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d48de640 1 -- 172.21.15.83:0/3144045963 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc00bcb0 con 0x7f09d006ee50 2024-04-17T10:49:09.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d50df640 1 --2- 172.21.15.83:0/3144045963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09bc003f70 0x7f09bc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:09.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d58e0640 1 --2- 172.21.15.83:0/3144045963 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f09d006ee50 0x7f09bc0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:09.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d60e1640 1 --2- 172.21.15.83:0/3144045963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09bc004d50 0x7f09bc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:09.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d50df640 1 -- 172.21.15.83:0/3144045963 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f09d006ee50 msgr2=0x7f09bc0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:09.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d50df640 1 --2- 172.21.15.83:0/3144045963 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f09d006ee50 0x7f09bc0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:09.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d50df640 1 -- 172.21.15.83:0/3144045963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09bc004d50 msgr2=0x7f09bc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:09.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d50df640 1 --2- 172.21.15.83:0/3144045963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09bc004d50 0x7f09bc0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:09.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d50df640 1 -- 172.21.15.83:0/3144045963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09bc003530 con 0x7f09bc003f70 2024-04-17T10:49:09.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d58e0640 1 --2- 172.21.15.83:0/3144045963 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f09d006ee50 0x7f09bc0039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:09.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d50df640 1 --2- 172.21.15.83:0/3144045963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09bc003f70 0x7f09bc004810 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f09c0001750 tx=0x7f09c001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63733dd037f2fc61 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:09.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d48de640 1 -- 172.21.15.83:0/3144045963 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c00013a0 con 0x7f09bc003f70 2024-04-17T10:49:09.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d48de640 1 -- 172.21.15.83:0/3144045963 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c00052f0 con 0x7f09bc003f70 2024-04-17T10:49:09.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d48de640 1 -- 172.21.15.83:0/3144045963 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c0003840 con 0x7f09bc003f70 2024-04-17T10:49:09.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d7b44640 1 -- 172.21.15.83:0/3144045963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09bc003f70 msgr2=0x7f09bc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:09.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d7b44640 1 --2- 172.21.15.83:0/3144045963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09bc003f70 0x7f09bc004810 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f09c0001750 tx=0x7f09c001c3e0 comp rx=0 tx=0).stop 2024-04-17T10:49:09.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d60e1640 1 -- 172.21.15.83:0/3144045963 reap_dead start 2024-04-17T10:49:09.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d7b44640 1 -- 172.21.15.83:0/3144045963 shutdown_connections 2024-04-17T10:49:09.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d7b44640 1 -- 172.21.15.83:0/3144045963 >> 172.21.15.83:0/3144045963 conn(0x7f09d00b3330 msgr2=0x7f09d00b3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:09.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d7b44640 1 -- 172.21.15.83:0/3144045963 shutdown_connections 2024-04-17T10:49:09.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.369+0000 7f09d7b44640 1 -- 172.21.15.83:0/3144045963 wait complete. 2024-04-17T10:49:09.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.373+0000 7f09d7b44640 1 Processor -- start 2024-04-17T10:49:09.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.373+0000 7f09d7b44640 1 -- start start 2024-04-17T10:49:09.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.373+0000 7f09d7b44640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09bc003f70 0x7f09d0158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:09.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.373+0000 7f09d7b44640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09d00c5130 0x7f09d00c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:09.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.373+0000 7f09d7b44640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f09d00c5570 0x7f09d00c64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:09.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.373+0000 7f09d7b44640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d015f500 con 0x7f09d00c5130 2024-04-17T10:49:09.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.373+0000 7f09d7b44640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d00d0060 con 0x7f09bc003f70 2024-04-17T10:49:09.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.373+0000 7f09d7b44640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d015f680 con 0x7f09d00c5570 2024-04-17T10:49:09.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.373+0000 7f09d50df640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09d00c5130 0x7f09d00c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:09.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.373+0000 7f09d50df640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09d00c5130 0x7f09d00c5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45184/0 (socket says 172.21.15.83:45184) 2024-04-17T10:49:09.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.373+0000 7f09d50df640 1 -- 172.21.15.83:0/3295375256 learned_addr learned my addr 172.21.15.83:0/3295375256 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:09.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.373+0000 7f09d58e0640 1 --2- 172.21.15.83:0/3295375256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09bc003f70 0x7f09d0158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:09.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.373+0000 7f09d50df640 1 -- 172.21.15.83:0/3295375256 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f09d00c5570 msgr2=0x7f09d00c64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:09.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.373+0000 7f09d50df640 1 --2- 172.21.15.83:0/3295375256 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f09d00c5570 0x7f09d00c64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:09.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.373+0000 7f09d50df640 1 -- 172.21.15.83:0/3295375256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09bc003f70 msgr2=0x7f09d0158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:09.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.373+0000 7f09d50df640 1 --2- 172.21.15.83:0/3295375256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09bc003f70 0x7f09d0158e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:09.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.373+0000 7f09d50df640 1 -- 172.21.15.83:0/3295375256 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d00b4420 con 0x7f09d00c5130 2024-04-17T10:49:09.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.373+0000 7f09d58e0640 1 --2- 172.21.15.83:0/3295375256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09bc003f70 0x7f09d0158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:09.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.373+0000 7f09d50df640 1 --2- 172.21.15.83:0/3295375256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09d00c5130 0x7f09d00c5fb0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f09c0005040 tx=0x7f09bc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:09.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.373+0000 7f09c67fc640 1 -- 172.21.15.83:0/3295375256 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c00013a0 con 0x7f09d00c5130 2024-04-17T10:49:09.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.373+0000 7f09c67fc640 1 -- 172.21.15.83:0/3295375256 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c00bb070 con 0x7f09d00c5130 2024-04-17T10:49:09.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.373+0000 7f09d7b44640 1 -- 172.21.15.83:0/3295375256 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d006c8a0 con 0x7f09d00c5130 2024-04-17T10:49:09.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.373+0000 7f09c67fc640 1 -- 172.21.15.83:0/3295375256 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c0007a40 con 0x7f09d00c5130 2024-04-17T10:49:09.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.373+0000 7f09d7b44640 1 -- 172.21.15.83:0/3295375256 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d006ce60 con 0x7f09d00c5130 2024-04-17T10:49:09.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.373+0000 7f09d7b44640 1 -- 172.21.15.83:0/3295375256 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09d0075000 con 0x7f09d00c5130 2024-04-17T10:49:09.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.377+0000 7f09c67fc640 1 -- 172.21.15.83:0/3295375256 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f09c00b4040 con 0x7f09d00c5130 2024-04-17T10:49:09.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.377+0000 7f09c67fc640 1 --2- 172.21.15.83:0/3295375256 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f09b4066f60 0x7f09b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:09.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.377+0000 7f09c67fc640 1 -- 172.21.15.83:0/3295375256 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f09c011f140 con 0x7f09d00c5130 2024-04-17T10:49:09.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.377+0000 7f09d58e0640 1 --2- 172.21.15.83:0/3295375256 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f09b4066f60 0x7f09b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:09.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.377+0000 7f09d58e0640 1 --2- 172.21.15.83:0/3295375256 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f09b4066f60 0x7f09b4069420 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f09cc000c10 tx=0x7f09cc0011b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:09.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.381+0000 7f09c67fc640 1 -- 172.21.15.83:0/3295375256 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f09c00160f0 con 0x7f09d00c5130 2024-04-17T10:49:09.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:09.529+0000 7f09d7b44640 1 -- 172.21.15.83:0/3295375256 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f09d00b0490 con 0x7f09d00c5130 2024-04-17T10:49:10.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.533+0000 7f09c67fc640 1 -- 172.21.15.83:0/3295375256 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v118) v1 ==== 181+0+0 (secure 0 0 0) 0x7f09c00f0150 con 0x7f09d00c5130 2024-04-17T10:49:10.530 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2024-04-17T10:49:10.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.537+0000 7f09d7b44640 1 -- 172.21.15.83:0/3295375256 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f09b4066f60 msgr2=0x7f09b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:10.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.537+0000 7f09d7b44640 1 --2- 172.21.15.83:0/3295375256 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f09b4066f60 0x7f09b4069420 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f09cc000c10 tx=0x7f09cc0011b0 comp rx=0 tx=0).stop 2024-04-17T10:49:10.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.537+0000 7f09d7b44640 1 -- 172.21.15.83:0/3295375256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09d00c5130 msgr2=0x7f09d00c5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:10.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.537+0000 7f09d7b44640 1 --2- 172.21.15.83:0/3295375256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09d00c5130 0x7f09d00c5fb0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f09c0005040 tx=0x7f09bc005840 comp rx=0 tx=0).stop 2024-04-17T10:49:10.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.537+0000 7f09d7b44640 1 -- 172.21.15.83:0/3295375256 shutdown_connections 2024-04-17T10:49:10.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.537+0000 7f09d7b44640 1 --2- 172.21.15.83:0/3295375256 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f09b4066f60 0x7f09b4069420 secure :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f09cc000c10 tx=0x7f09cc0011b0 comp rx=0 tx=0).stop 2024-04-17T10:49:10.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.537+0000 7f09d7b44640 1 --2- 172.21.15.83:0/3295375256 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f09d00c5570 0x7f09d00c64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:10.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.537+0000 7f09d7b44640 1 --2- 172.21.15.83:0/3295375256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09d00c5130 0x7f09d00c5fb0 secure :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f09c0005040 tx=0x7f09bc005840 comp rx=0 tx=0).stop 2024-04-17T10:49:10.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.537+0000 7f09d7b44640 1 --2- 172.21.15.83:0/3295375256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09bc003f70 0x7f09d0158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:10.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.537+0000 7f09d7b44640 1 -- 172.21.15.83:0/3295375256 >> 172.21.15.83:0/3295375256 conn(0x7f09d00b3330 msgr2=0x7f09d00b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:10.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.537+0000 7f09d7b44640 1 -- 172.21.15.83:0/3295375256 shutdown_connections 2024-04-17T10:49:10.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.537+0000 7f09d7b44640 1 -- 172.21.15.83:0/3295375256 wait complete. 2024-04-17T10:49:10.556 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-17T10:49:10.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.685+0000 7fbf37032640 1 Processor -- start 2024-04-17T10:49:10.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.689+0000 7fbf37032640 1 -- start start 2024-04-17T10:49:10.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.689+0000 7fbf37032640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf300aedf0 0x7fbf300b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:10.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.689+0000 7fbf37032640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbf3015cb40 0x7fbf3006abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:10.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.689+0000 7fbf37032640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf300b39d0 con 0x7fbf3006b120 2024-04-17T10:49:10.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.689+0000 7fbf37032640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf30072730 con 0x7fbf3015cb40 2024-04-17T10:49:10.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.689+0000 7fbf37032640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf30071290 con 0x7fbf300aedf0 2024-04-17T10:49:10.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.689+0000 7fbf2ffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbf3015cb40 0x7fbf3006abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:10.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.689+0000 7fbf355cf640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbf3006b120 0x7fbf3015f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:36614/0 (socket says 172.21.15.83:36614) 2024-04-17T10:49:10.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.689+0000 7fbf355cf640 1 -- 172.21.15.83:0/3240757563 learned_addr learned my addr 172.21.15.83:0/3240757563 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:10.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.689+0000 7fbf34dce640 1 --2- 172.21.15.83:0/3240757563 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf300aedf0 0x7fbf300b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:10.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.689+0000 7fbf2ffff640 1 -- 172.21.15.83:0/3240757563 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf300aedf0 msgr2=0x7fbf300b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:10.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.689+0000 7fbf2ffff640 1 --2- 172.21.15.83:0/3240757563 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf300aedf0 0x7fbf300b30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:10.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.689+0000 7fbf2ffff640 1 -- 172.21.15.83:0/3240757563 >> v1:172.21.15.83:6789/0 conn(0x7fbf3006b120 legacy=0x7fbf3015f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:10.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.689+0000 7fbf2ffff640 1 -- 172.21.15.83:0/3240757563 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf30070c40 con 0x7fbf3015cb40 2024-04-17T10:49:10.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.689+0000 7fbf2f7fe640 1 -- 172.21.15.83:0/3240757563 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 81011643 0 0) 0x7fbf300b39d0 con 0x7fbf3006b120 2024-04-17T10:49:10.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf2ffff640 1 --2- 172.21.15.83:0/3240757563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf3015cb40 0x7fbf3006abe0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c0097c0 tx=0x7fbf1c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b5e98d7d18f142a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:10.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf2f7fe640 1 -- 172.21.15.83:0/3240757563 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c0b0070 con 0x7fbf3015cb40 2024-04-17T10:49:10.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf2f7fe640 1 -- 172.21.15.83:0/3240757563 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1c0ab600 con 0x7fbf3015cb40 2024-04-17T10:49:10.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf2f7fe640 1 -- 172.21.15.83:0/3240757563 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c0ab9d0 con 0x7fbf3015cb40 2024-04-17T10:49:10.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf37032640 1 -- 172.21.15.83:0/3240757563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf3015cb40 msgr2=0x7fbf3006abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:10.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf37032640 1 --2- 172.21.15.83:0/3240757563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf3015cb40 0x7fbf3006abe0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c0097c0 tx=0x7fbf1c01c410 comp rx=0 tx=0).stop 2024-04-17T10:49:10.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf37032640 1 -- 172.21.15.83:0/3240757563 shutdown_connections 2024-04-17T10:49:10.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf37032640 1 --2- 172.21.15.83:0/3240757563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf3015cb40 0x7fbf3006abe0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:10.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf37032640 1 --2- 172.21.15.83:0/3240757563 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf300aedf0 0x7fbf300b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:10.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf37032640 1 -- 172.21.15.83:0/3240757563 >> 172.21.15.83:0/3240757563 conn(0x7fbf300cc050 msgr2=0x7fbf300cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:10.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf37032640 1 -- 172.21.15.83:0/3240757563 shutdown_connections 2024-04-17T10:49:10.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf37032640 1 -- 172.21.15.83:0/3240757563 wait complete. 2024-04-17T10:49:10.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf37032640 1 Processor -- start 2024-04-17T10:49:10.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf37032640 1 -- start start 2024-04-17T10:49:10.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf37032640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf3006b120 0x7fbf30152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:10.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf37032640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf300aedf0 0x7fbf300cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:10.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf37032640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf3015cb40 0x7fbf300ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:10.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf37032640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3006c2e0 con 0x7fbf300aedf0 2024-04-17T10:49:10.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf37032640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3006c160 con 0x7fbf3006b120 2024-04-17T10:49:10.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf37032640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf30168890 con 0x7fbf3015cb40 2024-04-17T10:49:10.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf2ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf300aedf0 0x7fbf300cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:10.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf2ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf300aedf0 0x7fbf300cdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45194/0 (socket says 172.21.15.83:45194) 2024-04-17T10:49:10.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf2ffff640 1 -- 172.21.15.83:0/1029378739 learned_addr learned my addr 172.21.15.83:0/1029378739 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:10.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf34dce640 1 --2- 172.21.15.83:0/1029378739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf3006b120 0x7fbf30152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:10.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf355cf640 1 --2- 172.21.15.83:0/1029378739 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf3015cb40 0x7fbf300ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:10.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf2ffff640 1 -- 172.21.15.83:0/1029378739 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf3015cb40 msgr2=0x7fbf300ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:10.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf2ffff640 1 --2- 172.21.15.83:0/1029378739 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf3015cb40 0x7fbf300ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:10.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf2ffff640 1 -- 172.21.15.83:0/1029378739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf3006b120 msgr2=0x7fbf30152100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:10.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf2ffff640 1 --2- 172.21.15.83:0/1029378739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf3006b120 0x7fbf30152100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:10.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf2ffff640 1 -- 172.21.15.83:0/1029378739 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf300ce3f0 con 0x7fbf300aedf0 2024-04-17T10:49:10.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf34dce640 1 --2- 172.21.15.83:0/1029378739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf3006b120 0x7fbf30152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:10.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf2ffff640 1 --2- 172.21.15.83:0/1029378739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf300aedf0 0x7fbf300cdaa0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c0ae040 tx=0x7fbf1c01ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:10.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf2d7fa640 1 -- 172.21.15.83:0/1029378739 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c0b0050 con 0x7fbf300aedf0 2024-04-17T10:49:10.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf2d7fa640 1 -- 172.21.15.83:0/1029378739 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1c0b1040 con 0x7fbf300aedf0 2024-04-17T10:49:10.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf37032640 1 -- 172.21.15.83:0/1029378739 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf300ce6d0 con 0x7fbf300aedf0 2024-04-17T10:49:10.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf2d7fa640 1 -- 172.21.15.83:0/1029378739 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c004240 con 0x7fbf300aedf0 2024-04-17T10:49:10.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.693+0000 7fbf37032640 1 -- 172.21.15.83:0/1029378739 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf300cec90 con 0x7fbf300aedf0 2024-04-17T10:49:10.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.697+0000 7fbf37032640 1 -- 172.21.15.83:0/1029378739 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf30075000 con 0x7fbf300aedf0 2024-04-17T10:49:10.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.697+0000 7fbf2d7fa640 1 -- 172.21.15.83:0/1029378739 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbf1c0be060 con 0x7fbf300aedf0 2024-04-17T10:49:10.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.697+0000 7fbf2d7fa640 1 --2- 172.21.15.83:0/1029378739 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbf14066f90 0x7fbf14069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:10.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.697+0000 7fbf2d7fa640 1 -- 172.21.15.83:0/1029378739 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5415+0+0 (secure 0 0 0) 0x7fbf1c11e9d0 con 0x7fbf300aedf0 2024-04-17T10:49:10.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.697+0000 7fbf34dce640 1 --2- 172.21.15.83:0/1029378739 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbf14066f90 0x7fbf14069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:10.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.697+0000 7fbf34dce640 1 --2- 172.21.15.83:0/1029378739 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbf14066f90 0x7fbf14069450 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fbf240097a0 tx=0x7fbf24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:10.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.701+0000 7fbf2d7fa640 1 -- 172.21.15.83:0/1029378739 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbf1c016090 con 0x7fbf300aedf0 2024-04-17T10:49:10.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:10.861+0000 7fbf37032640 1 -- 172.21.15.83:0/1029378739 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbf300b2cc0 con 0x7fbf300aedf0 2024-04-17T10:49:11.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.533+0000 7fbf2d7fa640 1 -- 172.21.15.83:0/1029378739 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v119) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbf1c0efa60 con 0x7fbf300aedf0 2024-04-17T10:49:11.532 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2024-04-17T10:49:11.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.537+0000 7fbf37032640 1 -- 172.21.15.83:0/1029378739 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbf14066f90 msgr2=0x7fbf14069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:11.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.537+0000 7fbf37032640 1 --2- 172.21.15.83:0/1029378739 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbf14066f90 0x7fbf14069450 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fbf240097a0 tx=0x7fbf24008040 comp rx=0 tx=0).stop 2024-04-17T10:49:11.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.537+0000 7fbf37032640 1 -- 172.21.15.83:0/1029378739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf300aedf0 msgr2=0x7fbf300cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:11.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.537+0000 7fbf37032640 1 --2- 172.21.15.83:0/1029378739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf300aedf0 0x7fbf300cdaa0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c0ae040 tx=0x7fbf1c01ac50 comp rx=0 tx=0).stop 2024-04-17T10:49:11.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.537+0000 7fbf37032640 1 -- 172.21.15.83:0/1029378739 shutdown_connections 2024-04-17T10:49:11.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.537+0000 7fbf37032640 1 --2- 172.21.15.83:0/1029378739 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbf14066f90 0x7fbf14069450 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:11.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.537+0000 7fbf37032640 1 --2- 172.21.15.83:0/1029378739 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf3015cb40 0x7fbf300ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:11.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.537+0000 7fbf37032640 1 --2- 172.21.15.83:0/1029378739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf300aedf0 0x7fbf300cdaa0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:11.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.537+0000 7fbf37032640 1 --2- 172.21.15.83:0/1029378739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf3006b120 0x7fbf30152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:11.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.537+0000 7fbf37032640 1 -- 172.21.15.83:0/1029378739 >> 172.21.15.83:0/1029378739 conn(0x7fbf300cc050 msgr2=0x7fbf30152f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:11.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.537+0000 7fbf37032640 1 -- 172.21.15.83:0/1029378739 shutdown_connections 2024-04-17T10:49:11.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.537+0000 7fbf37032640 1 -- 172.21.15.83:0/1029378739 wait complete. 2024-04-17T10:49:11.564 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-17T10:49:11.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.701+0000 7f3e55c88640 1 Processor -- start 2024-04-17T10:49:11.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.701+0000 7f3e55c88640 1 -- start start 2024-04-17T10:49:11.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.701+0000 7f3e55c88640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3e5015b6e0 0x7f3e50071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:11.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.701+0000 7f3e55c88640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e50070470 0x7f3e5006ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:11.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.701+0000 7f3e55c88640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e5006c1f0 con 0x7f3e5015bac0 2024-04-17T10:49:11.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.701+0000 7f3e55c88640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e50070a10 con 0x7f3e5015b6e0 2024-04-17T10:49:11.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.701+0000 7f3e55c88640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e50070cf0 con 0x7f3e50070470 2024-04-17T10:49:11.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.701+0000 7f3e4effd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3e5015bac0 0x7f3e500cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:36630/0 (socket says 172.21.15.83:36630) 2024-04-17T10:49:11.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.701+0000 7f3e4effd640 1 -- 172.21.15.83:0/1478702701 learned_addr learned my addr 172.21.15.83:0/1478702701 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:11.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.701+0000 7f3e4f7fe640 1 --2- 172.21.15.83:0/1478702701 >> v2:172.21.15.167:3300/0 conn(0x7f3e5015b6e0 0x7f3e50071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:11.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.701+0000 7f3e4ffff640 1 --2- 172.21.15.83:0/1478702701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e50070470 0x7f3e5006ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:11.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.701+0000 7f3e4e7fc640 1 -- 172.21.15.83:0/1478702701 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 550002998 0 0) 0x7f3e5006c1f0 con 0x7f3e5015bac0 2024-04-17T10:49:11.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.701+0000 7f3e4e7fc640 1 -- 172.21.15.83:0/1478702701 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e38003530 con 0x7f3e5015bac0 2024-04-17T10:49:11.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.701+0000 7f3e4e7fc640 1 -- 172.21.15.83:0/1478702701 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1078199946 0 0) 0x7f3e38003530 con 0x7f3e5015bac0 2024-04-17T10:49:11.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.701+0000 7f3e4f7fe640 1 -- 172.21.15.83:0/1478702701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e50070470 msgr2=0x7f3e5006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:11.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.701+0000 7f3e4f7fe640 1 --2- 172.21.15.83:0/1478702701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e50070470 0x7f3e5006ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:11.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.701+0000 7f3e4f7fe640 1 -- 172.21.15.83:0/1478702701 >> v1:172.21.15.83:6789/0 conn(0x7f3e5015bac0 legacy=0x7f3e500cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:11.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.701+0000 7f3e4f7fe640 1 -- 172.21.15.83:0/1478702701 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e50070fd0 con 0x7f3e5015b6e0 2024-04-17T10:49:11.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.705+0000 7f3e4f7fe640 1 --2- 172.21.15.83:0/1478702701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e5015b6e0 0x7f3e50071cc0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3e400097c0 tx=0x7f3e4001c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ddd3544d436226b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:11.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.705+0000 7f3e4e7fc640 1 -- 172.21.15.83:0/1478702701 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e400b0070 con 0x7f3e5015b6e0 2024-04-17T10:49:11.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.705+0000 7f3e4e7fc640 1 -- 172.21.15.83:0/1478702701 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e400ab400 con 0x7f3e5015b6e0 2024-04-17T10:49:11.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.705+0000 7f3e4e7fc640 1 -- 172.21.15.83:0/1478702701 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e440016a0 con 0x7f3e5015b6e0 2024-04-17T10:49:11.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.705+0000 7f3e55c88640 1 -- 172.21.15.83:0/1478702701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e5015b6e0 msgr2=0x7f3e50071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:11.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.705+0000 7f3e55c88640 1 --2- 172.21.15.83:0/1478702701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e5015b6e0 0x7f3e50071cc0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3e400097c0 tx=0x7f3e4001c330 comp rx=0 tx=0).stop 2024-04-17T10:49:11.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.705+0000 7f3e55c88640 1 -- 172.21.15.83:0/1478702701 shutdown_connections 2024-04-17T10:49:11.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.705+0000 7f3e55c88640 1 --2- 172.21.15.83:0/1478702701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e50070470 0x7f3e5006ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:11.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.705+0000 7f3e55c88640 1 --2- 172.21.15.83:0/1478702701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e5015b6e0 0x7f3e50071cc0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:11.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.705+0000 7f3e55c88640 1 -- 172.21.15.83:0/1478702701 >> 172.21.15.83:0/1478702701 conn(0x7f3e500b15b0 msgr2=0x7f3e500b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:11.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.705+0000 7f3e55c88640 1 -- 172.21.15.83:0/1478702701 shutdown_connections 2024-04-17T10:49:11.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.705+0000 7f3e55c88640 1 -- 172.21.15.83:0/1478702701 wait complete. 2024-04-17T10:49:11.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.709+0000 7f3e55c88640 1 Processor -- start 2024-04-17T10:49:11.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.709+0000 7f3e55c88640 1 -- start start 2024-04-17T10:49:11.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.709+0000 7f3e55c88640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e50070470 0x7f3e5014e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:11.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.709+0000 7f3e55c88640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e5015b6e0 0x7f3e5014c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:11.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.709+0000 7f3e55c88640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e5015bac0 0x7f3e500af350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:11.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.709+0000 7f3e55c88640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5006d2f0 con 0x7f3e5015bac0 2024-04-17T10:49:11.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.709+0000 7f3e55c88640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5006d170 con 0x7f3e5015b6e0 2024-04-17T10:49:11.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.709+0000 7f3e55c88640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5006b760 con 0x7f3e50070470 2024-04-17T10:49:11.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.709+0000 7f3e4ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e5015bac0 0x7f3e500af350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:11.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.709+0000 7f3e4ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e5015bac0 0x7f3e500af350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45198/0 (socket says 172.21.15.83:45198) 2024-04-17T10:49:11.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.709+0000 7f3e4ffff640 1 -- 172.21.15.83:0/2713813024 learned_addr learned my addr 172.21.15.83:0/2713813024 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:11.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.709+0000 7f3e4f7fe640 1 --2- 172.21.15.83:0/2713813024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e50070470 0x7f3e5014e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:11.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.709+0000 7f3e4effd640 1 --2- 172.21.15.83:0/2713813024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e5015b6e0 0x7f3e5014c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:11.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.709+0000 7f3e4ffff640 1 -- 172.21.15.83:0/2713813024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e50070470 msgr2=0x7f3e5014e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:11.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.709+0000 7f3e4ffff640 1 --2- 172.21.15.83:0/2713813024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e50070470 0x7f3e5014e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:11.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.709+0000 7f3e4ffff640 1 -- 172.21.15.83:0/2713813024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e5015b6e0 msgr2=0x7f3e5014c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:11.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.709+0000 7f3e4ffff640 1 --2- 172.21.15.83:0/2713813024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e5015b6e0 0x7f3e5014c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:11.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.709+0000 7f3e4ffff640 1 -- 172.21.15.83:0/2713813024 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e500aef80 con 0x7f3e5015bac0 2024-04-17T10:49:11.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.709+0000 7f3e4effd640 1 --2- 172.21.15.83:0/2713813024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e5015b6e0 0x7f3e5014c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:49:11.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.709+0000 7f3e4ffff640 1 --2- 172.21.15.83:0/2713813024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e5015bac0 0x7f3e500af350 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f3e48009d30 tx=0x7f3e4800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:11.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.709+0000 7f3e37fff640 1 -- 172.21.15.83:0/2713813024 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e48018020 con 0x7f3e5015bac0 2024-04-17T10:49:11.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.709+0000 7f3e37fff640 1 -- 172.21.15.83:0/2713813024 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e48014480 con 0x7f3e5015bac0 2024-04-17T10:49:11.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.709+0000 7f3e55c88640 1 -- 172.21.15.83:0/2713813024 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e5015d320 con 0x7f3e5015bac0 2024-04-17T10:49:11.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.709+0000 7f3e37fff640 1 -- 172.21.15.83:0/2713813024 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e48014870 con 0x7f3e5015bac0 2024-04-17T10:49:11.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.709+0000 7f3e55c88640 1 -- 172.21.15.83:0/2713813024 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e5015d8e0 con 0x7f3e5015bac0 2024-04-17T10:49:11.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.709+0000 7f3e55c88640 1 -- 172.21.15.83:0/2713813024 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e50074f90 con 0x7f3e5015bac0 2024-04-17T10:49:11.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.709+0000 7f3e37fff640 1 -- 172.21.15.83:0/2713813024 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3e4801b020 con 0x7f3e5015bac0 2024-04-17T10:49:11.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.709+0000 7f3e37fff640 1 --2- 172.21.15.83:0/2713813024 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3e3c066f60 0x7f3e3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:11.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.709+0000 7f3e37fff640 1 -- 172.21.15.83:0/2713813024 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3e48085750 con 0x7f3e5015bac0 2024-04-17T10:49:11.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.713+0000 7f3e4f7fe640 1 --2- 172.21.15.83:0/2713813024 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3e3c066f60 0x7f3e3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:11.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.713+0000 7f3e4f7fe640 1 --2- 172.21.15.83:0/2713813024 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3e3c066f60 0x7f3e3c069420 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3e400ad000 tx=0x7f3e400a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:11.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.717+0000 7f3e37fff640 1 -- 172.21.15.83:0/2713813024 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3e48022070 con 0x7f3e5015bac0 2024-04-17T10:49:11.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.901+0000 7f3e55c88640 1 -- 172.21.15.83:0/2713813024 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f3e500cbc00 con 0x7f3e5015bac0 2024-04-17T10:49:11.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.905+0000 7f3e37fff640 1 -- 172.21.15.83:0/2713813024 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3e48010040 con 0x7f3e5015bac0 2024-04-17T10:49:11.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.909+0000 7f3e55c88640 1 -- 172.21.15.83:0/2713813024 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3e3c066f60 msgr2=0x7f3e3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:11.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.909+0000 7f3e55c88640 1 --2- 172.21.15.83:0/2713813024 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3e3c066f60 0x7f3e3c069420 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3e400ad000 tx=0x7f3e400a6000 comp rx=0 tx=0).stop 2024-04-17T10:49:11.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.909+0000 7f3e55c88640 1 -- 172.21.15.83:0/2713813024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e5015bac0 msgr2=0x7f3e500af350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:11.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.909+0000 7f3e55c88640 1 --2- 172.21.15.83:0/2713813024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e5015bac0 0x7f3e500af350 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f3e48009d30 tx=0x7f3e4800c490 comp rx=0 tx=0).stop 2024-04-17T10:49:11.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.909+0000 7f3e55c88640 1 -- 172.21.15.83:0/2713813024 shutdown_connections 2024-04-17T10:49:11.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.909+0000 7f3e55c88640 1 --2- 172.21.15.83:0/2713813024 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3e3c066f60 0x7f3e3c069420 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:11.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.909+0000 7f3e55c88640 1 --2- 172.21.15.83:0/2713813024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e5015bac0 0x7f3e500af350 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:11.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.909+0000 7f3e55c88640 1 --2- 172.21.15.83:0/2713813024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e5015b6e0 0x7f3e5014c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:11.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.909+0000 7f3e55c88640 1 --2- 172.21.15.83:0/2713813024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e50070470 0x7f3e5014e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:11.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.909+0000 7f3e55c88640 1 -- 172.21.15.83:0/2713813024 >> 172.21.15.83:0/2713813024 conn(0x7f3e500b15b0 msgr2=0x7f3e500cfd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:11.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.909+0000 7f3e55c88640 1 -- 172.21.15.83:0/2713813024 shutdown_connections 2024-04-17T10:49:11.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:11.909+0000 7f3e55c88640 1 -- 172.21.15.83:0/2713813024 wait complete. 2024-04-17T10:49:11.922 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort' 2024-04-17T10:49:12.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.045+0000 7f274a0ad640 1 Processor -- start 2024-04-17T10:49:12.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.045+0000 7f274a0ad640 1 -- start start 2024-04-17T10:49:12.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.045+0000 7f274a0ad640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f274406b2f0 0x7f27440cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:12.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.045+0000 7f274a0ad640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2744066a10 0x7f2744072ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:12.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.045+0000 7f274a0ad640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27440cc400 con 0x7f274415bba0 2024-04-17T10:49:12.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.045+0000 7f274a0ad640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2744073000 con 0x7f2744066a10 2024-04-17T10:49:12.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.045+0000 7f274a0ad640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27440cc1b0 con 0x7f274406b2f0 2024-04-17T10:49:12.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.045+0000 7f27437fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f274415bba0 0x7f274406abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:36632/0 (socket says 172.21.15.83:36632) 2024-04-17T10:49:12.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.045+0000 7f27437fe640 1 -- 172.21.15.83:0/1308866621 learned_addr learned my addr 172.21.15.83:0/1308866621 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:12.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.045+0000 7f2743fff640 1 --2- 172.21.15.83:0/1308866621 >> v2:172.21.15.83:3300/0 conn(0x7f2744066a10 0x7f2744072ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:12.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.045+0000 7f27427fc640 1 -- 172.21.15.83:0/1308866621 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2775070602 0 0) 0x7f27440cc400 con 0x7f274415bba0 2024-04-17T10:49:12.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.045+0000 7f27427fc640 1 -- 172.21.15.83:0/1308866621 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f272c003530 con 0x7f274415bba0 2024-04-17T10:49:12.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.045+0000 7f2742ffd640 1 --2- 172.21.15.83:0/1308866621 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f274406b2f0 0x7f27440cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:12.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.045+0000 7f2743fff640 1 -- 172.21.15.83:0/1308866621 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f274406b2f0 msgr2=0x7f27440cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:12.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.045+0000 7f2743fff640 1 --2- 172.21.15.83:0/1308866621 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f274406b2f0 0x7f27440cd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:12.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.045+0000 7f2743fff640 1 -- 172.21.15.83:0/1308866621 >> v1:172.21.15.83:6789/0 conn(0x7f274415bba0 legacy=0x7f274406abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:12.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.045+0000 7f2743fff640 1 -- 172.21.15.83:0/1308866621 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27440cc6e0 con 0x7f2744066a10 2024-04-17T10:49:12.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.049+0000 7f2743fff640 1 --2- 172.21.15.83:0/1308866621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2744066a10 0x7f2744072ac0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f273c009d90 tx=0x7f273c01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6706ade732ed2bb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:12.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.049+0000 7f27427fc640 1 -- 172.21.15.83:0/1308866621 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273c0b2070 con 0x7f2744066a10 2024-04-17T10:49:12.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.049+0000 7f27427fc640 1 -- 172.21.15.83:0/1308866621 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273c0a5a00 con 0x7f2744066a10 2024-04-17T10:49:12.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.049+0000 7f27427fc640 1 -- 172.21.15.83:0/1308866621 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273c0a5dd0 con 0x7f2744066a10 2024-04-17T10:49:12.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.049+0000 7f274a0ad640 1 -- 172.21.15.83:0/1308866621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2744066a10 msgr2=0x7f2744072ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:12.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.049+0000 7f274a0ad640 1 --2- 172.21.15.83:0/1308866621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2744066a10 0x7f2744072ac0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f273c009d90 tx=0x7f273c01c900 comp rx=0 tx=0).stop 2024-04-17T10:49:12.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.049+0000 7f274a0ad640 1 -- 172.21.15.83:0/1308866621 shutdown_connections 2024-04-17T10:49:12.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.049+0000 7f274a0ad640 1 --2- 172.21.15.83:0/1308866621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2744066a10 0x7f2744072ac0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:12.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.049+0000 7f274a0ad640 1 --2- 172.21.15.83:0/1308866621 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f274406b2f0 0x7f27440cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:12.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.049+0000 7f274a0ad640 1 -- 172.21.15.83:0/1308866621 >> 172.21.15.83:0/1308866621 conn(0x7f2744071200 msgr2=0x7f27440b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:12.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.049+0000 7f274a0ad640 1 -- 172.21.15.83:0/1308866621 shutdown_connections 2024-04-17T10:49:12.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.049+0000 7f274a0ad640 1 -- 172.21.15.83:0/1308866621 wait complete. 2024-04-17T10:49:12.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.049+0000 7f274a0ad640 1 Processor -- start 2024-04-17T10:49:12.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.049+0000 7f274a0ad640 1 -- start start 2024-04-17T10:49:12.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.049+0000 7f274a0ad640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2744066a10 0x7f2744158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:12.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.049+0000 7f274a0ad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f274406b2f0 0x7f2744159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:12.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.049+0000 7f274a0ad640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f274415bba0 0x7f2744148fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:12.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.049+0000 7f274a0ad640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27440ce940 con 0x7f274406b2f0 2024-04-17T10:49:12.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.049+0000 7f274a0ad640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274415bf80 con 0x7f2744066a10 2024-04-17T10:49:12.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.049+0000 7f274a0ad640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27440ceac0 con 0x7f274415bba0 2024-04-17T10:49:12.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.053+0000 7f2742ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f274406b2f0 0x7f2744159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:12.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.053+0000 7f2742ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f274406b2f0 0x7f2744159090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45224/0 (socket says 172.21.15.83:45224) 2024-04-17T10:49:12.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.053+0000 7f2742ffd640 1 -- 172.21.15.83:0/3396872872 learned_addr learned my addr 172.21.15.83:0/3396872872 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:12.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.053+0000 7f27437fe640 1 --2- 172.21.15.83:0/3396872872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2744066a10 0x7f2744158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:12.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.053+0000 7f2743fff640 1 --2- 172.21.15.83:0/3396872872 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f274415bba0 0x7f2744148fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:12.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.053+0000 7f2742ffd640 1 -- 172.21.15.83:0/3396872872 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f274415bba0 msgr2=0x7f2744148fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:12.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.053+0000 7f2742ffd640 1 --2- 172.21.15.83:0/3396872872 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f274415bba0 0x7f2744148fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:12.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.053+0000 7f2742ffd640 1 -- 172.21.15.83:0/3396872872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2744066a10 msgr2=0x7f2744158b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:12.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.053+0000 7f2742ffd640 1 --2- 172.21.15.83:0/3396872872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2744066a10 0x7f2744158b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:12.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.053+0000 7f2742ffd640 1 -- 172.21.15.83:0/3396872872 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2744152560 con 0x7f274406b2f0 2024-04-17T10:49:12.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.053+0000 7f27437fe640 1 --2- 172.21.15.83:0/3396872872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2744066a10 0x7f2744158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:12.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.053+0000 7f2742ffd640 1 --2- 172.21.15.83:0/3396872872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f274406b2f0 0x7f2744159090 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f273800c7f0 tx=0x7f273800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:12.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.053+0000 7f272bfff640 1 -- 172.21.15.83:0/3396872872 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2738015070 con 0x7f274406b2f0 2024-04-17T10:49:12.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.053+0000 7f272bfff640 1 -- 172.21.15.83:0/3396872872 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2738004240 con 0x7f274406b2f0 2024-04-17T10:49:12.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.053+0000 7f274a0ad640 1 -- 172.21.15.83:0/3396872872 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2744152840 con 0x7f274406b2f0 2024-04-17T10:49:12.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.053+0000 7f272bfff640 1 -- 172.21.15.83:0/3396872872 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2738003620 con 0x7f274406b2f0 2024-04-17T10:49:12.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.053+0000 7f274a0ad640 1 -- 172.21.15.83:0/3396872872 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2744158300 con 0x7f274406b2f0 2024-04-17T10:49:12.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.053+0000 7f274a0ad640 1 -- 172.21.15.83:0/3396872872 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2744074f90 con 0x7f274406b2f0 2024-04-17T10:49:12.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.053+0000 7f272bfff640 1 -- 172.21.15.83:0/3396872872 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f27380039d0 con 0x7f274406b2f0 2024-04-17T10:49:12.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.053+0000 7f272bfff640 1 --2- 172.21.15.83:0/3396872872 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2730066f60 0x7f2730069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:12.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.053+0000 7f272bfff640 1 -- 172.21.15.83:0/3396872872 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2738084db0 con 0x7f274406b2f0 2024-04-17T10:49:12.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.053+0000 7f27437fe640 1 --2- 172.21.15.83:0/3396872872 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2730066f60 0x7f2730069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:12.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.053+0000 7f27437fe640 1 --2- 172.21.15.83:0/3396872872 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2730066f60 0x7f2730069420 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2734005890 tx=0x7f2734009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:12.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.057+0000 7f272bfff640 1 -- 172.21.15.83:0/3396872872 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f27380560c0 con 0x7f274406b2f0 2024-04-17T10:49:12.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.277+0000 7f274a0ad640 1 -- 172.21.15.83:0/3396872872 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]} v 0) v1 -- 0x7f27440671b0 con 0x7f274406b2f0 2024-04-17T10:49:12.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.553+0000 7f272bfff640 1 -- 172.21.15.83:0/3396872872 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]}]=0 v493) v1 ==== 166+0+0 (secure 0 0 0) 0x7f27380559c0 con 0x7f274406b2f0 2024-04-17T10:49:12.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.557+0000 7f274a0ad640 1 -- 172.21.15.83:0/3396872872 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2730066f60 msgr2=0x7f2730069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:12.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.557+0000 7f274a0ad640 1 --2- 172.21.15.83:0/3396872872 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2730066f60 0x7f2730069420 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2734005890 tx=0x7f2734009040 comp rx=0 tx=0).stop 2024-04-17T10:49:12.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.557+0000 7f274a0ad640 1 -- 172.21.15.83:0/3396872872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f274406b2f0 msgr2=0x7f2744159090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:12.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.557+0000 7f274a0ad640 1 --2- 172.21.15.83:0/3396872872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f274406b2f0 0x7f2744159090 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f273800c7f0 tx=0x7f273800ccb0 comp rx=0 tx=0).stop 2024-04-17T10:49:12.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.557+0000 7f274a0ad640 1 -- 172.21.15.83:0/3396872872 shutdown_connections 2024-04-17T10:49:12.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.557+0000 7f274a0ad640 1 --2- 172.21.15.83:0/3396872872 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2730066f60 0x7f2730069420 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:12.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.557+0000 7f274a0ad640 1 --2- 172.21.15.83:0/3396872872 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f274415bba0 0x7f2744148fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:12.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.557+0000 7f274a0ad640 1 --2- 172.21.15.83:0/3396872872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f274406b2f0 0x7f2744159090 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:12.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.557+0000 7f274a0ad640 1 --2- 172.21.15.83:0/3396872872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2744066a10 0x7f2744158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:12.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.557+0000 7f274a0ad640 1 -- 172.21.15.83:0/3396872872 >> 172.21.15.83:0/3396872872 conn(0x7f2744071200 msgr2=0x7f27440ae050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:12.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.557+0000 7f274a0ad640 1 -- 172.21.15.83:0/3396872872 shutdown_connections 2024-04-17T10:49:12.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.557+0000 7f274a0ad640 1 -- 172.21.15.83:0/3396872872 wait complete. 2024-04-17T10:49:12.576 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume_with_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-17T10:49:12.577 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_task_status_on_mds_failover (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-17T10:49:12.577 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:49:12.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.705+0000 7fa30cf85640 1 Processor -- start 2024-04-17T10:49:12.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.705+0000 7fa30cf85640 1 -- start start 2024-04-17T10:49:12.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.705+0000 7fa30cf85640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa30815a900 0x7fa308072ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:12.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.705+0000 7fa30cf85640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa30806c300 0x7fa3080cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:12.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.705+0000 7fa30cf85640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa30806d050 con 0x7fa30806cbf0 2024-04-17T10:49:12.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.705+0000 7fa30cf85640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa308071000 con 0x7fa30815a900 2024-04-17T10:49:12.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.705+0000 7fa30cf85640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa30806ef90 con 0x7fa30806c300 2024-04-17T10:49:12.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.705+0000 7fa30659c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa30815a900 0x7fa308072ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:12.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.705+0000 7fa30659c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa30815a900 0x7fa308072ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45226/0 (socket says 172.21.15.83:45226) 2024-04-17T10:49:12.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.705+0000 7fa30659c640 1 -- 172.21.15.83:0/630408374 learned_addr learned my addr 172.21.15.83:0/630408374 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:12.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.705+0000 7fa30659c640 1 -- 172.21.15.83:0/630408374 >> v2:172.21.15.167:3300/0 conn(0x7fa30806c300 msgr2=0x7fa3080cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:12.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.705+0000 7fa30659c640 1 --2- 172.21.15.83:0/630408374 >> v2:172.21.15.167:3300/0 conn(0x7fa30806c300 0x7fa3080cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:12.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.705+0000 7fa30659c640 1 -- 172.21.15.83:0/630408374 >> v1:172.21.15.167:6789/0 conn(0x7fa30806cbf0 legacy=0x7fa30806bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:12.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.705+0000 7fa30659c640 1 -- 172.21.15.83:0/630408374 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa30806f270 con 0x7fa30815a900 2024-04-17T10:49:12.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.705+0000 7fa30659c640 1 --2- 172.21.15.83:0/630408374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa30815a900 0x7fa308072ab0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fa3080b2ac0 tx=0x7fa2f801c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde1015a26bed5af server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:12.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.705+0000 7fa30559a640 1 -- 172.21.15.83:0/630408374 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f80a5780 con 0x7fa30815a900 2024-04-17T10:49:12.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.709+0000 7fa30559a640 1 -- 172.21.15.83:0/630408374 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f80a5920 con 0x7fa30815a900 2024-04-17T10:49:12.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.709+0000 7fa30559a640 1 -- 172.21.15.83:0/630408374 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f80a5cf0 con 0x7fa30815a900 2024-04-17T10:49:12.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.709+0000 7fa30cf85640 1 -- 172.21.15.83:0/630408374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa30815a900 msgr2=0x7fa308072ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:12.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.709+0000 7fa30cf85640 1 --2- 172.21.15.83:0/630408374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa30815a900 0x7fa308072ab0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fa3080b2ac0 tx=0x7fa2f801c9d0 comp rx=0 tx=0).stop 2024-04-17T10:49:12.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.709+0000 7fa30cf85640 1 -- 172.21.15.83:0/630408374 shutdown_connections 2024-04-17T10:49:12.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.709+0000 7fa30cf85640 1 --2- 172.21.15.83:0/630408374 >> v2:172.21.15.167:3300/0 conn(0x7fa30806c300 0x7fa3080cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:12.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.709+0000 7fa30cf85640 1 --2- 172.21.15.83:0/630408374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa30815a900 0x7fa308072ab0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:12.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.709+0000 7fa30cf85640 1 -- 172.21.15.83:0/630408374 >> 172.21.15.83:0/630408374 conn(0x7fa3080cc910 msgr2=0x7fa3080ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:12.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.709+0000 7fa30cf85640 1 -- 172.21.15.83:0/630408374 shutdown_connections 2024-04-17T10:49:12.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.709+0000 7fa30cf85640 1 -- 172.21.15.83:0/630408374 wait complete. 2024-04-17T10:49:12.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.709+0000 7fa30cf85640 1 Processor -- start 2024-04-17T10:49:12.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.709+0000 7fa30cf85640 1 -- start start 2024-04-17T10:49:12.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.709+0000 7fa30cf85640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa30806c300 0x7fa3080b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:12.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.709+0000 7fa30cf85640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa30806cbf0 0x7fa308157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:12.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.709+0000 7fa30cf85640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa30815a900 0x7fa3080b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:12.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.709+0000 7fa30cf85640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3080cb880 con 0x7fa30806cbf0 2024-04-17T10:49:12.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.709+0000 7fa30cf85640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3080cb700 con 0x7fa30815a900 2024-04-17T10:49:12.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.709+0000 7fa30cf85640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3080cba00 con 0x7fa30806c300 2024-04-17T10:49:12.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.713+0000 7fa305d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa30806cbf0 0x7fa308157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:12.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.713+0000 7fa305d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa30806cbf0 0x7fa308157ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45232/0 (socket says 172.21.15.83:45232) 2024-04-17T10:49:12.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.713+0000 7fa305d9b640 1 -- 172.21.15.83:0/465224792 learned_addr learned my addr 172.21.15.83:0/465224792 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:12.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.713+0000 7fa30659c640 1 --2- 172.21.15.83:0/465224792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa30806c300 0x7fa3080b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:12.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.713+0000 7fa306d9d640 1 --2- 172.21.15.83:0/465224792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa30815a900 0x7fa3080b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:12.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.713+0000 7fa305d9b640 1 -- 172.21.15.83:0/465224792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa30806c300 msgr2=0x7fa3080b0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:12.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.713+0000 7fa305d9b640 1 --2- 172.21.15.83:0/465224792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa30806c300 0x7fa3080b0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:12.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.713+0000 7fa305d9b640 1 -- 172.21.15.83:0/465224792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa30815a900 msgr2=0x7fa3080b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:12.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.713+0000 7fa305d9b640 1 --2- 172.21.15.83:0/465224792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa30815a900 0x7fa3080b5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:12.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.713+0000 7fa305d9b640 1 -- 172.21.15.83:0/465224792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3080b1320 con 0x7fa30806cbf0 2024-04-17T10:49:12.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.713+0000 7fa306d9d640 1 --2- 172.21.15.83:0/465224792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa30815a900 0x7fa3080b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:12.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.713+0000 7fa305d9b640 1 --2- 172.21.15.83:0/465224792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa30806cbf0 0x7fa308157ef0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000c450 tx=0x7fa2f000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:12.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.713+0000 7fa2feffd640 1 -- 172.21.15.83:0/465224792 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0012830 con 0x7fa30806cbf0 2024-04-17T10:49:12.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.713+0000 7fa2feffd640 1 -- 172.21.15.83:0/465224792 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f0017070 con 0x7fa30806cbf0 2024-04-17T10:49:12.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.713+0000 7fa2feffd640 1 -- 172.21.15.83:0/465224792 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0012c30 con 0x7fa30806cbf0 2024-04-17T10:49:12.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.713+0000 7fa30cf85640 1 -- 172.21.15.83:0/465224792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa308156e50 con 0x7fa30806cbf0 2024-04-17T10:49:12.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.713+0000 7fa30cf85640 1 -- 172.21.15.83:0/465224792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa308157410 con 0x7fa30806cbf0 2024-04-17T10:49:12.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.713+0000 7fa30cf85640 1 -- 172.21.15.83:0/465224792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3080c9a90 con 0x7fa30806cbf0 2024-04-17T10:49:12.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.713+0000 7fa2feffd640 1 -- 172.21.15.83:0/465224792 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa2f001c460 con 0x7fa30806cbf0 2024-04-17T10:49:12.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.713+0000 7fa2feffd640 1 --2- 172.21.15.83:0/465224792 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa2e8066f60 0x7fa2e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:12.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.713+0000 7fa2feffd640 1 -- 172.21.15.83:0/465224792 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa2f0084b70 con 0x7fa30806cbf0 2024-04-17T10:49:12.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.713+0000 7fa30659c640 1 --2- 172.21.15.83:0/465224792 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa2e8066f60 0x7fa2e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:12.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.713+0000 7fa30659c640 1 --2- 172.21.15.83:0/465224792 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa2e8066f60 0x7fa2e8069420 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fa2f80ae000 tx=0x7fa2f80a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:12.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.717+0000 7fa2feffd640 1 -- 172.21.15.83:0/465224792 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa3080c9a90 con 0x7fa30806cbf0 2024-04-17T10:49:12.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.881+0000 7fa30cf85640 1 -- 172.21.15.83:0/465224792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa308068150 con 0x7fa30806cbf0 2024-04-17T10:49:12.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.885+0000 7fa2feffd640 1 -- 172.21.15.83:0/465224792 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+12958 (secure 0 0 0) 0x7fa2f0055e80 con 0x7fa30806cbf0 2024-04-17T10:49:12.881 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:12.881 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":119,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:49:11.538556+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/1479417998":"2024-04-18T10:49:08.207502+0000","172.21.15.83:6836/1479417998":"2024-04-18T10:49:08.207502+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:49:12.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.885+0000 7fa30cf85640 1 -- 172.21.15.83:0/465224792 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa2e8066f60 msgr2=0x7fa2e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:12.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.885+0000 7fa30cf85640 1 --2- 172.21.15.83:0/465224792 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa2e8066f60 0x7fa2e8069420 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fa2f80ae000 tx=0x7fa2f80a7070 comp rx=0 tx=0).stop 2024-04-17T10:49:12.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.885+0000 7fa30cf85640 1 -- 172.21.15.83:0/465224792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa30806cbf0 msgr2=0x7fa308157ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:12.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.885+0000 7fa30cf85640 1 --2- 172.21.15.83:0/465224792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa30806cbf0 0x7fa308157ef0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000c450 tx=0x7fa2f000c910 comp rx=0 tx=0).stop 2024-04-17T10:49:12.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.885+0000 7fa30cf85640 1 -- 172.21.15.83:0/465224792 shutdown_connections 2024-04-17T10:49:12.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.885+0000 7fa30cf85640 1 --2- 172.21.15.83:0/465224792 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa2e8066f60 0x7fa2e8069420 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:12.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.885+0000 7fa30cf85640 1 --2- 172.21.15.83:0/465224792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa30815a900 0x7fa3080b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:12.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.885+0000 7fa30cf85640 1 --2- 172.21.15.83:0/465224792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa30806cbf0 0x7fa308157ef0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:12.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.885+0000 7fa30cf85640 1 --2- 172.21.15.83:0/465224792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa30806c300 0x7fa3080b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:12.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.885+0000 7fa30cf85640 1 -- 172.21.15.83:0/465224792 >> 172.21.15.83:0/465224792 conn(0x7fa3080cc910 msgr2=0x7fa30806e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:12.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.885+0000 7fa30cf85640 1 -- 172.21.15.83:0/465224792 shutdown_connections 2024-04-17T10:49:12.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:12.885+0000 7fa30cf85640 1 -- 172.21.15.83:0/465224792 wait complete. 2024-04-17T10:49:12.900 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:49:12.901 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:49:13.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.025+0000 7f37b78c8640 1 Processor -- start 2024-04-17T10:49:13.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.025+0000 7f37b78c8640 1 -- start start 2024-04-17T10:49:13.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.025+0000 7f37b78c8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37b00ce400 0x7f37b00b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:13.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.025+0000 7f37b78c8640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f37b00b5450 0x7f37b00b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:13.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.025+0000 7f37b78c8640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37b015e4d0 con 0x7f37b015e0f0 2024-04-17T10:49:13.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.025+0000 7f37b78c8640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37b00b5df0 con 0x7f37b00b5450 2024-04-17T10:49:13.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.025+0000 7f37b78c8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b0162b40 con 0x7f37b00ce400 2024-04-17T10:49:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.025+0000 7f37b5e65640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f37b00b5450 0x7f37b00b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.025+0000 7f37b4e63640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37b00ce400 0x7f37b00b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.025+0000 7f37b5664640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f37b015e0f0 0x7f37b0161c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:53870/0 (socket says 172.21.15.83:53870) 2024-04-17T10:49:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.025+0000 7f37b5664640 1 -- 172.21.15.83:0/1068744705 learned_addr learned my addr 172.21.15.83:0/1068744705 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.025+0000 7f37b5e65640 1 -- 172.21.15.83:0/1068744705 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37b00ce400 msgr2=0x7f37b00b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.025+0000 7f37b5e65640 1 --2- 172.21.15.83:0/1068744705 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37b00ce400 0x7f37b00b6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.025+0000 7f37b5e65640 1 -- 172.21.15.83:0/1068744705 >> v1:172.21.15.167:6789/0 conn(0x7f37b015e0f0 legacy=0x7f37b0161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.025+0000 7f37b5e65640 1 -- 172.21.15.83:0/1068744705 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b0162580 con 0x7f37b00b5450 2024-04-17T10:49:13.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.029+0000 7f37b5e65640 1 --2- 172.21.15.83:0/1068744705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37b00b5450 0x7f37b00b58b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009dc0 tx=0x7f37a801c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=307bd3de6136ced9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:13.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.029+0000 7f37a7fff640 1 -- 172.21.15.83:0/1068744705 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a80b2070 con 0x7f37b00b5450 2024-04-17T10:49:13.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.029+0000 7f37a7fff640 1 -- 172.21.15.83:0/1068744705 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a80a5780 con 0x7f37b00b5450 2024-04-17T10:49:13.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.029+0000 7f37a7fff640 1 -- 172.21.15.83:0/1068744705 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a80a5b50 con 0x7f37b00b5450 2024-04-17T10:49:13.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.029+0000 7f37b78c8640 1 -- 172.21.15.83:0/1068744705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37b00b5450 msgr2=0x7f37b00b58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:13.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.029+0000 7f37b78c8640 1 --2- 172.21.15.83:0/1068744705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37b00b5450 0x7f37b00b58b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009dc0 tx=0x7f37a801c8c0 comp rx=0 tx=0).stop 2024-04-17T10:49:13.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.029+0000 7f37b78c8640 1 -- 172.21.15.83:0/1068744705 shutdown_connections 2024-04-17T10:49:13.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.029+0000 7f37b78c8640 1 --2- 172.21.15.83:0/1068744705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37b00b5450 0x7f37b00b58b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:13.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.029+0000 7f37b78c8640 1 --2- 172.21.15.83:0/1068744705 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37b00ce400 0x7f37b00b6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:13.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.029+0000 7f37b78c8640 1 -- 172.21.15.83:0/1068744705 >> 172.21.15.83:0/1068744705 conn(0x7f37b00d0450 msgr2=0x7f37b00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:13.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.029+0000 7f37b78c8640 1 -- 172.21.15.83:0/1068744705 shutdown_connections 2024-04-17T10:49:13.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.029+0000 7f37b78c8640 1 -- 172.21.15.83:0/1068744705 wait complete. 2024-04-17T10:49:13.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.029+0000 7f37b78c8640 1 Processor -- start 2024-04-17T10:49:13.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.029+0000 7f37b78c8640 1 -- start start 2024-04-17T10:49:13.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.033+0000 7f37b78c8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37b00b5450 0x7f37b0149050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:13.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.033+0000 7f37b78c8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37b00ce400 0x7f37b015a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:13.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.033+0000 7f37b78c8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37b015e0f0 0x7f37b015a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:13.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.033+0000 7f37b78c8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b00d2ed0 con 0x7f37b00ce400 2024-04-17T10:49:13.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.033+0000 7f37b78c8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b00d2d50 con 0x7f37b015e0f0 2024-04-17T10:49:13.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.033+0000 7f37b78c8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b00d3050 con 0x7f37b00b5450 2024-04-17T10:49:13.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.033+0000 7f37b4e63640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37b00ce400 0x7f37b015a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:13.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.033+0000 7f37b4e63640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37b00ce400 0x7f37b015a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45242/0 (socket says 172.21.15.83:45242) 2024-04-17T10:49:13.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.033+0000 7f37b4e63640 1 -- 172.21.15.83:0/1405635634 learned_addr learned my addr 172.21.15.83:0/1405635634 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:13.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.033+0000 7f37b5664640 1 --2- 172.21.15.83:0/1405635634 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37b00b5450 0x7f37b0149050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:13.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.033+0000 7f37b5e65640 1 --2- 172.21.15.83:0/1405635634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37b015e0f0 0x7f37b015a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:13.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.033+0000 7f37b4e63640 1 -- 172.21.15.83:0/1405635634 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37b00b5450 msgr2=0x7f37b0149050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:13.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.033+0000 7f37b4e63640 1 --2- 172.21.15.83:0/1405635634 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37b00b5450 0x7f37b0149050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:13.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.033+0000 7f37b4e63640 1 -- 172.21.15.83:0/1405635634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37b015e0f0 msgr2=0x7f37b015a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:13.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.033+0000 7f37b4e63640 1 --2- 172.21.15.83:0/1405635634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37b015e0f0 0x7f37b015a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:13.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.033+0000 7f37b4e63640 1 -- 172.21.15.83:0/1405635634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b00c2170 con 0x7f37b00ce400 2024-04-17T10:49:13.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.033+0000 7f37b5e65640 1 --2- 172.21.15.83:0/1405635634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37b015e0f0 0x7f37b015a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:13.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.033+0000 7f37b4e63640 1 --2- 172.21.15.83:0/1405635634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37b00ce400 0x7f37b015a300 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f37ac00e900 tx=0x7f37ac00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:13.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.033+0000 7f37a5ffb640 1 -- 172.21.15.83:0/1405635634 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac017070 con 0x7f37b00ce400 2024-04-17T10:49:13.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.033+0000 7f37a5ffb640 1 -- 172.21.15.83:0/1405635634 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ac00ca00 con 0x7f37b00ce400 2024-04-17T10:49:13.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.033+0000 7f37b78c8640 1 -- 172.21.15.83:0/1405635634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37b00c2450 con 0x7f37b00ce400 2024-04-17T10:49:13.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.033+0000 7f37a5ffb640 1 -- 172.21.15.83:0/1405635634 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac00cdf0 con 0x7f37b00ce400 2024-04-17T10:49:13.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.033+0000 7f37b78c8640 1 -- 172.21.15.83:0/1405635634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b00c2a10 con 0x7f37b00ce400 2024-04-17T10:49:13.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.033+0000 7f37b78c8640 1 -- 172.21.15.83:0/1405635634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37b00681a0 con 0x7f37b00ce400 2024-04-17T10:49:13.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.033+0000 7f37a5ffb640 1 -- 172.21.15.83:0/1405635634 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f37ac01a070 con 0x7f37b00ce400 2024-04-17T10:49:13.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.033+0000 7f37a5ffb640 1 --2- 172.21.15.83:0/1405635634 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3794066f60 0x7f3794069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:13.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.033+0000 7f37a5ffb640 1 -- 172.21.15.83:0/1405635634 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f37ac085090 con 0x7f37b00ce400 2024-04-17T10:49:13.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.033+0000 7f37b5664640 1 --2- 172.21.15.83:0/1405635634 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3794066f60 0x7f3794069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:13.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.033+0000 7f37b5664640 1 --2- 172.21.15.83:0/1405635634 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3794066f60 0x7f3794069420 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f37a0000f70 tx=0x7f37a0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:13.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.037+0000 7f37a5ffb640 1 -- 172.21.15.83:0/1405635634 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f37ac056430 con 0x7f37b00ce400 2024-04-17T10:49:13.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.197+0000 7f37b78c8640 1 -- 172.21.15.83:0/1405635634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f37b0074f90 con 0x7f37b00ce400 2024-04-17T10:49:13.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.197+0000 7f37a5ffb640 1 -- 172.21.15.83:0/1405635634 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) v1 ==== 93+0+10 (secure 0 0 0) 0x7f37ac055d30 con 0x7f37b00ce400 2024-04-17T10:49:13.195 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:49:13.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.201+0000 7f37b78c8640 1 -- 172.21.15.83:0/1405635634 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3794066f60 msgr2=0x7f3794069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:13.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.201+0000 7f37b78c8640 1 --2- 172.21.15.83:0/1405635634 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3794066f60 0x7f3794069420 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f37a0000f70 tx=0x7f37a0009040 comp rx=0 tx=0).stop 2024-04-17T10:49:13.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.201+0000 7f37b78c8640 1 -- 172.21.15.83:0/1405635634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37b00ce400 msgr2=0x7f37b015a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:13.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.201+0000 7f37b78c8640 1 --2- 172.21.15.83:0/1405635634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37b00ce400 0x7f37b015a300 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f37ac00e900 tx=0x7f37ac00edc0 comp rx=0 tx=0).stop 2024-04-17T10:49:13.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.201+0000 7f37b78c8640 1 -- 172.21.15.83:0/1405635634 shutdown_connections 2024-04-17T10:49:13.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.201+0000 7f37b78c8640 1 --2- 172.21.15.83:0/1405635634 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3794066f60 0x7f3794069420 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:13.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.201+0000 7f37b78c8640 1 --2- 172.21.15.83:0/1405635634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37b015e0f0 0x7f37b015a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:13.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.201+0000 7f37b78c8640 1 --2- 172.21.15.83:0/1405635634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37b00ce400 0x7f37b015a300 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:13.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.201+0000 7f37b78c8640 1 --2- 172.21.15.83:0/1405635634 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37b00b5450 0x7f37b0149050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:13.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.201+0000 7f37b78c8640 1 -- 172.21.15.83:0/1405635634 >> 172.21.15.83:0/1405635634 conn(0x7f37b00d0450 msgr2=0x7f37b00b8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:13.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.201+0000 7f37b78c8640 1 -- 172.21.15.83:0/1405635634 shutdown_connections 2024-04-17T10:49:13.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.201+0000 7f37b78c8640 1 -- 172.21.15.83:0/1405635634 wait complete. 2024-04-17T10:49:13.213 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover' 2024-04-17T10:49:13.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.329+0000 7f19fd6ee640 1 Processor -- start 2024-04-17T10:49:13.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.329+0000 7f19fd6ee640 1 -- start start 2024-04-17T10:49:13.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19fd6ee640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19f806abe0 0x7f19f80afe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:13.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19fd6ee640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f19f815bac0 0x7f19f806afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:13.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19fd6ee640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19f806b500 con 0x7f19f80d0610 2024-04-17T10:49:13.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19fd6ee640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19f80d0b40 con 0x7f19f815bac0 2024-04-17T10:49:13.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19fd6ee640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f80d0e20 con 0x7f19f806abe0 2024-04-17T10:49:13.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19f77fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f19f80d0610 0x7f19f806d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:36640/0 (socket says 172.21.15.83:36640) 2024-04-17T10:49:13.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19f77fe640 1 -- 172.21.15.83:0/2569675347 learned_addr learned my addr 172.21.15.83:0/2569675347 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:13.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19f5ffb640 1 -- 172.21.15.83:0/2569675347 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1958087008 0 0) 0x7f19f806b500 con 0x7f19f80d0610 2024-04-17T10:49:13.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19f5ffb640 1 -- 172.21.15.83:0/2569675347 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19e0003530 con 0x7f19f80d0610 2024-04-17T10:49:13.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19f6ffd640 1 --2- 172.21.15.83:0/2569675347 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19f806abe0 0x7f19f80afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:13.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19f67fc640 1 --2- 172.21.15.83:0/2569675347 >> v2:172.21.15.167:3300/0 conn(0x7f19f815bac0 0x7f19f806afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:13.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19f5ffb640 1 -- 172.21.15.83:0/2569675347 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4124491374 0 0) 0x7f19e0003530 con 0x7f19f80d0610 2024-04-17T10:49:13.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19f5ffb640 1 -- 172.21.15.83:0/2569675347 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19f806abe0 msgr2=0x7f19f80afe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:13.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19f5ffb640 1 --2- 172.21.15.83:0/2569675347 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19f806abe0 0x7f19f80afe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:13.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19f5ffb640 1 -- 172.21.15.83:0/2569675347 >> v2:172.21.15.167:3300/0 conn(0x7f19f815bac0 msgr2=0x7f19f806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:13.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19f5ffb640 1 --2- 172.21.15.83:0/2569675347 >> v2:172.21.15.167:3300/0 conn(0x7f19f815bac0 0x7f19f806afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:13.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19f5ffb640 1 -- 172.21.15.83:0/2569675347 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f80d1100 con 0x7f19f80d0610 2024-04-17T10:49:13.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19f5ffb640 1 -- 172.21.15.83:0/2569675347 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f19e8001680 con 0x7f19f80d0610 2024-04-17T10:49:13.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19f5ffb640 1 -- 172.21.15.83:0/2569675347 >> v1:172.21.15.83:6789/0 conn(0x7f19f80d0610 legacy=0x7f19f806d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:13.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19f5ffb640 1 --2- 172.21.15.83:0/2569675347 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19f806abe0 0x7f19e00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:13.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19f5ffb640 1 --2- 172.21.15.83:0/2569675347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19e0003ff0 0x7f19e0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:13.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19f5ffb640 1 --2- 172.21.15.83:0/2569675347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19e0004d50 0x7f19e00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:13.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19f5ffb640 1 -- 172.21.15.83:0/2569675347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f80d0b40 con 0x7f19e0004d50 2024-04-17T10:49:13.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19f5ffb640 1 -- 172.21.15.83:0/2569675347 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f80d0e20 con 0x7f19e0003ff0 2024-04-17T10:49:13.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19f5ffb640 1 -- 172.21.15.83:0/2569675347 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e000bad0 con 0x7f19f806abe0 2024-04-17T10:49:13.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19f77fe640 1 --2- 172.21.15.83:0/2569675347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19e0004d50 0x7f19e00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:13.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19f6ffd640 1 --2- 172.21.15.83:0/2569675347 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19f806abe0 0x7f19e00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:13.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19f67fc640 1 --2- 172.21.15.83:0/2569675347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19e0003ff0 0x7f19e0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:13.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19f77fe640 1 -- 172.21.15.83:0/2569675347 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19f806abe0 msgr2=0x7f19e00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:13.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19f77fe640 1 --2- 172.21.15.83:0/2569675347 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19f806abe0 0x7f19e00039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:13.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19f77fe640 1 -- 172.21.15.83:0/2569675347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19e0003ff0 msgr2=0x7f19e0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:13.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19f77fe640 1 --2- 172.21.15.83:0/2569675347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19e0003ff0 0x7f19e0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:13.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19f77fe640 1 -- 172.21.15.83:0/2569675347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e0003530 con 0x7f19e0004d50 2024-04-17T10:49:13.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19f67fc640 1 --2- 172.21.15.83:0/2569675347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19e0003ff0 0x7f19e0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:13.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19f77fe640 1 --2- 172.21.15.83:0/2569675347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19e0004d50 0x7f19e00051d0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f19e800c210 tx=0x7f19e801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7fb77e1bd0ccbc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:13.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.333+0000 7f19f5ffb640 1 -- 172.21.15.83:0/2569675347 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e8007d60 con 0x7f19e0004d50 2024-04-17T10:49:13.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19f5ffb640 1 -- 172.21.15.83:0/2569675347 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e80a9020 con 0x7f19e0004d50 2024-04-17T10:49:13.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19f5ffb640 1 -- 172.21.15.83:0/2569675347 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e80a5be0 con 0x7f19e0004d50 2024-04-17T10:49:13.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19fd6ee640 1 -- 172.21.15.83:0/2569675347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19e0004d50 msgr2=0x7f19e00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:13.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19fd6ee640 1 --2- 172.21.15.83:0/2569675347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19e0004d50 0x7f19e00051d0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f19e800c210 tx=0x7f19e801cca0 comp rx=0 tx=0).stop 2024-04-17T10:49:13.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19f77fe640 1 -- 172.21.15.83:0/2569675347 reap_dead start 2024-04-17T10:49:13.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19fd6ee640 1 -- 172.21.15.83:0/2569675347 shutdown_connections 2024-04-17T10:49:13.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19fd6ee640 1 -- 172.21.15.83:0/2569675347 >> 172.21.15.83:0/2569675347 conn(0x7f19f806bcf0 msgr2=0x7f19f806c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:13.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19fd6ee640 1 -- 172.21.15.83:0/2569675347 shutdown_connections 2024-04-17T10:49:13.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19fd6ee640 1 -- 172.21.15.83:0/2569675347 wait complete. 2024-04-17T10:49:13.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19fd6ee640 1 Processor -- start 2024-04-17T10:49:13.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19fd6ee640 1 -- start start 2024-04-17T10:49:13.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19fd6ee640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19e0004d50 0x7f19f815da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:13.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19fd6ee640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19f815df50 0x7f19f81f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:13.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19fd6ee640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19f81f41c0 0x7f19f81f6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:13.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19fd6ee640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f80afb00 con 0x7f19f815df50 2024-04-17T10:49:13.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19fd6ee640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f80ce6d0 con 0x7f19f81f41c0 2024-04-17T10:49:13.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19fd6ee640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f80cfc70 con 0x7f19e0004d50 2024-04-17T10:49:13.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19f67fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19f815df50 0x7f19f81f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:13.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19f67fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19f815df50 0x7f19f81f3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45270/0 (socket says 172.21.15.83:45270) 2024-04-17T10:49:13.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19f67fc640 1 -- 172.21.15.83:0/2517900907 learned_addr learned my addr 172.21.15.83:0/2517900907 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:13.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19f6ffd640 1 --2- 172.21.15.83:0/2517900907 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19e0004d50 0x7f19f815da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:13.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19f77fe640 1 --2- 172.21.15.83:0/2517900907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19f81f41c0 0x7f19f81f6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:13.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19f67fc640 1 -- 172.21.15.83:0/2517900907 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19e0004d50 msgr2=0x7f19f815da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:13.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19f67fc640 1 --2- 172.21.15.83:0/2517900907 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19e0004d50 0x7f19f815da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:13.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19f67fc640 1 -- 172.21.15.83:0/2517900907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19f81f41c0 msgr2=0x7f19f81f6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:13.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19f67fc640 1 --2- 172.21.15.83:0/2517900907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19f81f41c0 0x7f19f81f6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:13.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19f67fc640 1 -- 172.21.15.83:0/2517900907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f8150a00 con 0x7f19f815df50 2024-04-17T10:49:13.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19f77fe640 1 --2- 172.21.15.83:0/2517900907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19f81f41c0 0x7f19f81f6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:13.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19f67fc640 1 --2- 172.21.15.83:0/2517900907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19f815df50 0x7f19f81f3c80 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00e840 tx=0x7f19ec00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:13.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19df7fe640 1 -- 172.21.15.83:0/2517900907 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec017070 con 0x7f19f815df50 2024-04-17T10:49:13.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19df7fe640 1 -- 172.21.15.83:0/2517900907 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19ec00c870 con 0x7f19f815df50 2024-04-17T10:49:13.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19fd6ee640 1 -- 172.21.15.83:0/2517900907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f8150ce0 con 0x7f19f815df50 2024-04-17T10:49:13.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19df7fe640 1 -- 172.21.15.83:0/2517900907 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec00f040 con 0x7f19f815df50 2024-04-17T10:49:13.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.337+0000 7f19fd6ee640 1 -- 172.21.15.83:0/2517900907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f81512a0 con 0x7f19f815df50 2024-04-17T10:49:13.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.341+0000 7f19fd6ee640 1 -- 172.21.15.83:0/2517900907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f8074f90 con 0x7f19f815df50 2024-04-17T10:49:13.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.341+0000 7f19df7fe640 1 -- 172.21.15.83:0/2517900907 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f19ec01a070 con 0x7f19f815df50 2024-04-17T10:49:13.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.341+0000 7f19df7fe640 1 --2- 172.21.15.83:0/2517900907 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f19e4066f60 0x7f19e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:13.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.341+0000 7f19df7fe640 1 -- 172.21.15.83:0/2517900907 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f19ec0848e0 con 0x7f19f815df50 2024-04-17T10:49:13.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.341+0000 7f19f6ffd640 1 --2- 172.21.15.83:0/2517900907 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f19e4066f60 0x7f19e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:13.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.341+0000 7f19f6ffd640 1 --2- 172.21.15.83:0/2517900907 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f19e4066f60 0x7f19e4069420 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f19f0004330 tx=0x7f19f000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:13.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.345+0000 7f19df7fe640 1 -- 172.21.15.83:0/2517900907 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f19ec055bf0 con 0x7f19f815df50 2024-04-17T10:49:13.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:13.561+0000 7f19fd6ee640 1 -- 172.21.15.83:0/2517900907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]} v 0) v1 -- 0x7f19f8158bf0 con 0x7f19f815df50 2024-04-17T10:49:14.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.565+0000 7f19df7fe640 1 -- 172.21.15.83:0/2517900907 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]}]=0 v495) v1 ==== 169+0+0 (secure 0 0 0) 0x7f19ec0554f0 con 0x7f19f815df50 2024-04-17T10:49:14.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.565+0000 7f19fd6ee640 1 -- 172.21.15.83:0/2517900907 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f19e4066f60 msgr2=0x7f19e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:14.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.565+0000 7f19fd6ee640 1 --2- 172.21.15.83:0/2517900907 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f19e4066f60 0x7f19e4069420 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f19f0004330 tx=0x7f19f000b040 comp rx=0 tx=0).stop 2024-04-17T10:49:14.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.565+0000 7f19fd6ee640 1 -- 172.21.15.83:0/2517900907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19f815df50 msgr2=0x7f19f81f3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:14.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.565+0000 7f19fd6ee640 1 --2- 172.21.15.83:0/2517900907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19f815df50 0x7f19f81f3c80 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00e840 tx=0x7f19ec00ed00 comp rx=0 tx=0).stop 2024-04-17T10:49:14.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.565+0000 7f19fd6ee640 1 -- 172.21.15.83:0/2517900907 shutdown_connections 2024-04-17T10:49:14.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.565+0000 7f19fd6ee640 1 --2- 172.21.15.83:0/2517900907 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f19e4066f60 0x7f19e4069420 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:14.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.565+0000 7f19fd6ee640 1 --2- 172.21.15.83:0/2517900907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19f81f41c0 0x7f19f81f6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:14.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.565+0000 7f19fd6ee640 1 --2- 172.21.15.83:0/2517900907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19f815df50 0x7f19f81f3c80 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:14.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.565+0000 7f19fd6ee640 1 --2- 172.21.15.83:0/2517900907 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19e0004d50 0x7f19f815da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:14.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.569+0000 7f19fd6ee640 1 -- 172.21.15.83:0/2517900907 >> 172.21.15.83:0/2517900907 conn(0x7f19f806bcf0 msgr2=0x7f19f8159640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:14.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.569+0000 7f19fd6ee640 1 -- 172.21.15.83:0/2517900907 shutdown_connections 2024-04-17T10:49:14.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.569+0000 7f19fd6ee640 1 -- 172.21.15.83:0/2517900907 wait complete. 2024-04-17T10:49:14.584 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-17T10:49:14.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.713+0000 7fe0e8eb5640 1 Processor -- start 2024-04-17T10:49:14.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.713+0000 7fe0e8eb5640 1 -- start start 2024-04-17T10:49:14.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.713+0000 7fe0e8eb5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0e4154de0 0x7fe0e406ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:14.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.713+0000 7fe0e8eb5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe0e406b350 0x7fe0e40b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:14.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.713+0000 7fe0e8eb5640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0e415bd70 con 0x7fe0e40b4040 2024-04-17T10:49:14.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.713+0000 7fe0e8eb5640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0e40b4870 con 0x7fe0e406b350 2024-04-17T10:49:14.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.713+0000 7fe0e8eb5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e415bf60 con 0x7fe0e4154de0 2024-04-17T10:49:14.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.713+0000 7fe0e1d9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe0e406b350 0x7fe0e40b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:14.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.713+0000 7fe0e1d9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe0e406b350 0x7fe0e40b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45286/0 (socket says 172.21.15.83:45286) 2024-04-17T10:49:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.713+0000 7fe0e1d9b640 1 -- 172.21.15.83:0/430003505 learned_addr learned my addr 172.21.15.83:0/430003505 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.713+0000 7fe0e2d9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe0e40b4040 0x7fe0e4071580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:36652/0 (socket says 172.21.15.83:36652) 2024-04-17T10:49:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.713+0000 7fe0e259c640 1 --2- 172.21.15.83:0/430003505 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0e4154de0 0x7fe0e406ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.713+0000 7fe0e1d9b640 1 -- 172.21.15.83:0/430003505 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0e4154de0 msgr2=0x7fe0e406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.713+0000 7fe0e1d9b640 1 --2- 172.21.15.83:0/430003505 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0e4154de0 0x7fe0e406ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.713+0000 7fe0e1d9b640 1 -- 172.21.15.83:0/430003505 >> v1:172.21.15.83:6789/0 conn(0x7fe0e40b4040 legacy=0x7fe0e4071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.713+0000 7fe0e1d9b640 1 -- 172.21.15.83:0/430003505 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e415afb0 con 0x7fe0e406b350 2024-04-17T10:49:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.713+0000 7fe0e159a640 1 -- 172.21.15.83:0/430003505 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3736410080 0 0) 0x7fe0e415bd70 con 0x7fe0e40b4040 2024-04-17T10:49:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.713+0000 7fe0e259c640 1 --2- 172.21.15.83:0/430003505 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0e4154de0 0x7fe0e406ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:14.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0e1d9b640 1 --2- 172.21.15.83:0/430003505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e406b350 0x7fe0e40b3b00 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4009870 tx=0x7fe0d401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd2b65f523ef9b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:14.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0e159a640 1 -- 172.21.15.83:0/430003505 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d40b0070 con 0x7fe0e406b350 2024-04-17T10:49:14.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0e159a640 1 -- 172.21.15.83:0/430003505 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d40ab600 con 0x7fe0e406b350 2024-04-17T10:49:14.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0e159a640 1 -- 172.21.15.83:0/430003505 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d40ab9d0 con 0x7fe0e406b350 2024-04-17T10:49:14.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0e8eb5640 1 -- 172.21.15.83:0/430003505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e406b350 msgr2=0x7fe0e40b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:14.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0e8eb5640 1 --2- 172.21.15.83:0/430003505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e406b350 0x7fe0e40b3b00 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4009870 tx=0x7fe0d401c560 comp rx=0 tx=0).stop 2024-04-17T10:49:14.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0e8eb5640 1 -- 172.21.15.83:0/430003505 shutdown_connections 2024-04-17T10:49:14.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0e8eb5640 1 --2- 172.21.15.83:0/430003505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e406b350 0x7fe0e40b3b00 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:14.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0e8eb5640 1 --2- 172.21.15.83:0/430003505 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0e4154de0 0x7fe0e406ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:14.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0e8eb5640 1 -- 172.21.15.83:0/430003505 >> 172.21.15.83:0/430003505 conn(0x7fe0e40b15b0 msgr2=0x7fe0e40b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:14.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0e8eb5640 1 -- 172.21.15.83:0/430003505 shutdown_connections 2024-04-17T10:49:14.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0e8eb5640 1 -- 172.21.15.83:0/430003505 wait complete. 2024-04-17T10:49:14.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0e8eb5640 1 Processor -- start 2024-04-17T10:49:14.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0e8eb5640 1 -- start start 2024-04-17T10:49:14.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0e8eb5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0e406b350 0x7fe0e414bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:14.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0e8eb5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e40b4040 0x7fe0e40af250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:14.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0e8eb5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0e4154de0 0x7fe0e40af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:14.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0e8eb5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e40b4490 con 0x7fe0e40b4040 2024-04-17T10:49:14.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0e8eb5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e41551c0 con 0x7fe0e406b350 2024-04-17T10:49:14.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0e8eb5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e40b4610 con 0x7fe0e4154de0 2024-04-17T10:49:14.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0e1d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e40b4040 0x7fe0e40af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:14.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0e1d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e40b4040 0x7fe0e40af250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45298/0 (socket says 172.21.15.83:45298) 2024-04-17T10:49:14.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0e1d9b640 1 -- 172.21.15.83:0/3594249011 learned_addr learned my addr 172.21.15.83:0/3594249011 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:14.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0e1d9b640 1 -- 172.21.15.83:0/3594249011 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0e4154de0 msgr2=0x7fe0e40af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:14.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0e1d9b640 1 --2- 172.21.15.83:0/3594249011 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0e4154de0 0x7fe0e40af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:14.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0e1d9b640 1 -- 172.21.15.83:0/3594249011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0e406b350 msgr2=0x7fe0e414bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:14.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0e1d9b640 1 --2- 172.21.15.83:0/3594249011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0e406b350 0x7fe0e414bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:14.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0e1d9b640 1 -- 172.21.15.83:0/3594249011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e4150fb0 con 0x7fe0e40b4040 2024-04-17T10:49:14.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0e1d9b640 1 --2- 172.21.15.83:0/3594249011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e40b4040 0x7fe0e40af250 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fe0d40ae040 tx=0x7fe0d4019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:14.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0d2ffd640 1 -- 172.21.15.83:0/3594249011 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d40b0040 con 0x7fe0e40b4040 2024-04-17T10:49:14.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0d2ffd640 1 -- 172.21.15.83:0/3594249011 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d40b9070 con 0x7fe0e40b4040 2024-04-17T10:49:14.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0e8eb5640 1 -- 172.21.15.83:0/3594249011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0e4151290 con 0x7fe0e40b4040 2024-04-17T10:49:14.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0d2ffd640 1 -- 172.21.15.83:0/3594249011 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d40b4880 con 0x7fe0e40b4040 2024-04-17T10:49:14.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.717+0000 7fe0e8eb5640 1 -- 172.21.15.83:0/3594249011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0e4151850 con 0x7fe0e40b4040 2024-04-17T10:49:14.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.721+0000 7fe0e8eb5640 1 -- 172.21.15.83:0/3594249011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0e4074f90 con 0x7fe0e40b4040 2024-04-17T10:49:14.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.721+0000 7fe0d2ffd640 1 -- 172.21.15.83:0/3594249011 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe0d40ab4c0 con 0x7fe0e40b4040 2024-04-17T10:49:14.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.721+0000 7fe0d2ffd640 1 --2- 172.21.15.83:0/3594249011 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe0c4066f60 0x7fe0c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:14.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.721+0000 7fe0d2ffd640 1 -- 172.21.15.83:0/3594249011 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe0d411eae0 con 0x7fe0e40b4040 2024-04-17T10:49:14.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.721+0000 7fe0e259c640 1 --2- 172.21.15.83:0/3594249011 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe0c4066f60 0x7fe0c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:14.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.721+0000 7fe0e259c640 1 --2- 172.21.15.83:0/3594249011 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe0c4066f60 0x7fe0c4069420 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe0e40b0210 tx=0x7fe0d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:14.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.725+0000 7fe0d2ffd640 1 -- 172.21.15.83:0/3594249011 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe0d4016150 con 0x7fe0e40b4040 2024-04-17T10:49:14.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.873+0000 7fe0e8eb5640 1 -- 172.21.15.83:0/3594249011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe0e4071700 con 0x7fe0e40b4040 2024-04-17T10:49:14.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.877+0000 7fe0d2ffd640 1 -- 172.21.15.83:0/3594249011 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe0d40efdf0 con 0x7fe0e40b4040 2024-04-17T10:49:14.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.881+0000 7fe0e8eb5640 1 -- 172.21.15.83:0/3594249011 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe0c4066f60 msgr2=0x7fe0c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:14.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.881+0000 7fe0e8eb5640 1 --2- 172.21.15.83:0/3594249011 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe0c4066f60 0x7fe0c4069420 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe0e40b0210 tx=0x7fe0d8008040 comp rx=0 tx=0).stop 2024-04-17T10:49:14.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.881+0000 7fe0e8eb5640 1 -- 172.21.15.83:0/3594249011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e40b4040 msgr2=0x7fe0e40af250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:14.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.881+0000 7fe0e8eb5640 1 --2- 172.21.15.83:0/3594249011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e40b4040 0x7fe0e40af250 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fe0d40ae040 tx=0x7fe0d4019300 comp rx=0 tx=0).stop 2024-04-17T10:49:14.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.881+0000 7fe0e8eb5640 1 -- 172.21.15.83:0/3594249011 shutdown_connections 2024-04-17T10:49:14.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.881+0000 7fe0e8eb5640 1 --2- 172.21.15.83:0/3594249011 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe0c4066f60 0x7fe0c4069420 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:14.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.881+0000 7fe0e8eb5640 1 --2- 172.21.15.83:0/3594249011 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0e4154de0 0x7fe0e40af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:14.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.881+0000 7fe0e8eb5640 1 --2- 172.21.15.83:0/3594249011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e40b4040 0x7fe0e40af250 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:14.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.881+0000 7fe0e8eb5640 1 --2- 172.21.15.83:0/3594249011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0e406b350 0x7fe0e414bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:14.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.881+0000 7fe0e8eb5640 1 -- 172.21.15.83:0/3594249011 >> 172.21.15.83:0/3594249011 conn(0x7fe0e40b15b0 msgr2=0x7fe0e406d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:14.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.881+0000 7fe0e8eb5640 1 -- 172.21.15.83:0/3594249011 shutdown_connections 2024-04-17T10:49:14.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:14.881+0000 7fe0e8eb5640 1 -- 172.21.15.83:0/3594249011 wait complete. 2024-04-17T10:49:14.893 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T10:49:14.893 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:49:14.942 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:49:14.991 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T10:49:14.991 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:49:14.991 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:49:14.997 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16063 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:49:14.997 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:49:14.997 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:49:15.003 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:49:15.003 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:49:15.003 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:49:15.045 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16063 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:49:15.045 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:49:15.045 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:49:15.050 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:49:15.050 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-17T10:49:15.050 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:49:15.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.213+0000 7f2c6b3fb640 1 Processor -- start 2024-04-17T10:49:15.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.213+0000 7f2c6b3fb640 1 -- start start 2024-04-17T10:49:15.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.213+0000 7f2c6b3fb640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2c6415ba30 0x7f2c64072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:15.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.213+0000 7f2c6b3fb640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c6406e320 con 0x7f2c6415c960 2024-04-17T10:49:15.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.213+0000 7f2c6b3fb640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c6406e040 con 0x7f2c640cfbe0 2024-04-17T10:49:15.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.213+0000 7f2c6b3fb640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c64071ea0 con 0x7f2c6415ba30 2024-04-17T10:49:15.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.213+0000 7f2c68996640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2c6415c960 0x7f2c640cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:36668/0 (socket says 172.21.15.83:36668) 2024-04-17T10:49:15.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.213+0000 7f2c69197640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2c6415ba30 0x7f2c64072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:15.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.213+0000 7f2c68996640 1 -- 172.21.15.83:0/1704951440 learned_addr learned my addr 172.21.15.83:0/1704951440 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:15.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.213+0000 7f2c69197640 1 -- 172.21.15.83:0/1704951440 >> v1:172.21.15.167:6789/0 conn(0x7f2c640cfbe0 legacy=0x7f2c6406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:15.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.213+0000 7f2c69197640 1 -- 172.21.15.83:0/1704951440 >> v1:172.21.15.83:6789/0 conn(0x7f2c6415c960 legacy=0x7f2c640cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:15.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.213+0000 7f2c69197640 1 -- 172.21.15.83:0/1704951440 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c64071850 con 0x7f2c6415ba30 2024-04-17T10:49:15.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.213+0000 7f2c5bfff640 1 -- 172.21.15.83:0/1704951440 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4224378370 0 0) 0x7f2c6406e320 con 0x7f2c6415c960 2024-04-17T10:49:15.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.217+0000 7f2c69197640 1 --2- 172.21.15.83:0/1704951440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c6415ba30 0x7f2c64072330 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f2c600097c0 tx=0x7f2c6001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58084f82b31865c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:15.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.217+0000 7f2c5bfff640 1 -- 172.21.15.83:0/1704951440 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c600b0070 con 0x7f2c6415ba30 2024-04-17T10:49:15.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.217+0000 7f2c5bfff640 1 -- 172.21.15.83:0/1704951440 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c600ab600 con 0x7f2c6415ba30 2024-04-17T10:49:15.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.217+0000 7f2c5bfff640 1 -- 172.21.15.83:0/1704951440 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c600ab9d0 con 0x7f2c6415ba30 2024-04-17T10:49:15.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.217+0000 7f2c6b3fb640 1 -- 172.21.15.83:0/1704951440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c6415ba30 msgr2=0x7f2c64072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:15.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.217+0000 7f2c6b3fb640 1 --2- 172.21.15.83:0/1704951440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c6415ba30 0x7f2c64072330 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f2c600097c0 tx=0x7f2c6001c3d0 comp rx=0 tx=0).stop 2024-04-17T10:49:15.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.217+0000 7f2c6b3fb640 1 -- 172.21.15.83:0/1704951440 shutdown_connections 2024-04-17T10:49:15.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.217+0000 7f2c6b3fb640 1 --2- 172.21.15.83:0/1704951440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c6415ba30 0x7f2c64072330 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:15.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.217+0000 7f2c6b3fb640 1 -- 172.21.15.83:0/1704951440 >> 172.21.15.83:0/1704951440 conn(0x7f2c6406abe0 msgr2=0x7f2c6406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:15.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.217+0000 7f2c6b3fb640 1 -- 172.21.15.83:0/1704951440 shutdown_connections 2024-04-17T10:49:15.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.217+0000 7f2c6b3fb640 1 -- 172.21.15.83:0/1704951440 wait complete. 2024-04-17T10:49:15.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.221+0000 7f2c6b3fb640 1 Processor -- start 2024-04-17T10:49:15.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.221+0000 7f2c6b3fb640 1 -- start start 2024-04-17T10:49:15.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.221+0000 7f2c6b3fb640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c640cfbe0 0x7f2c640adc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:15.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.221+0000 7f2c6b3fb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c6415ba30 0x7f2c6414cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:15.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.221+0000 7f2c6b3fb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c6415c960 0x7f2c64146bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:15.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.221+0000 7f2c6b3fb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6406de20 con 0x7f2c6415ba30 2024-04-17T10:49:15.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.221+0000 7f2c6b3fb640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6406dca0 con 0x7f2c6415c960 2024-04-17T10:49:15.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.221+0000 7f2c6b3fb640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c64169530 con 0x7f2c640cfbe0 2024-04-17T10:49:15.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.221+0000 7f2c68996640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c6415ba30 0x7f2c6414cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:15.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.221+0000 7f2c68996640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c6415ba30 0x7f2c6414cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45312/0 (socket says 172.21.15.83:45312) 2024-04-17T10:49:15.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.221+0000 7f2c68996640 1 -- 172.21.15.83:0/3991847000 learned_addr learned my addr 172.21.15.83:0/3991847000 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:15.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.221+0000 7f2c68996640 1 -- 172.21.15.83:0/3991847000 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c640cfbe0 msgr2=0x7f2c640adc70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-17T10:49:15.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.221+0000 7f2c68996640 1 --2- 172.21.15.83:0/3991847000 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c640cfbe0 0x7f2c640adc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:15.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.221+0000 7f2c68996640 1 -- 172.21.15.83:0/3991847000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c6415c960 msgr2=0x7f2c64146bf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-17T10:49:15.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.221+0000 7f2c68996640 1 --2- 172.21.15.83:0/3991847000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c6415c960 0x7f2c64146bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:15.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.221+0000 7f2c68996640 1 -- 172.21.15.83:0/3991847000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c64155370 con 0x7f2c6415ba30 2024-04-17T10:49:15.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.221+0000 7f2c68996640 1 --2- 172.21.15.83:0/3991847000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c6415ba30 0x7f2c6414cd80 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f2c5400c410 tx=0x7f2c5400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:15.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.221+0000 7f2c59ffb640 1 -- 172.21.15.83:0/3991847000 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c54017070 con 0x7f2c6415ba30 2024-04-17T10:49:15.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.221+0000 7f2c59ffb640 1 -- 172.21.15.83:0/3991847000 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c54005cf0 con 0x7f2c6415ba30 2024-04-17T10:49:15.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.221+0000 7f2c6b3fb640 1 -- 172.21.15.83:0/3991847000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c640ced60 con 0x7f2c6415ba30 2024-04-17T10:49:15.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.221+0000 7f2c59ffb640 1 -- 172.21.15.83:0/3991847000 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c54007310 con 0x7f2c6415ba30 2024-04-17T10:49:15.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.221+0000 7f2c6b3fb640 1 -- 172.21.15.83:0/3991847000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c640cf320 con 0x7f2c6415ba30 2024-04-17T10:49:15.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.221+0000 7f2c6b3fb640 1 -- 172.21.15.83:0/3991847000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c64074f90 con 0x7f2c6415ba30 2024-04-17T10:49:15.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.225+0000 7f2c59ffb640 1 -- 172.21.15.83:0/3991847000 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2c5401a070 con 0x7f2c6415ba30 2024-04-17T10:49:15.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.225+0000 7f2c59ffb640 1 --2- 172.21.15.83:0/3991847000 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2c48066f60 0x7f2c48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:15.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.225+0000 7f2c59ffb640 1 -- 172.21.15.83:0/3991847000 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2c54084180 con 0x7f2c6415ba30 2024-04-17T10:49:15.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.225+0000 7f2c69197640 1 --2- 172.21.15.83:0/3991847000 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2c48066f60 0x7f2c48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:15.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.225+0000 7f2c69197640 1 --2- 172.21.15.83:0/3991847000 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2c48066f60 0x7f2c48069420 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2c600ad000 tx=0x7f2c60008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:15.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.229+0000 7f2c59ffb640 1 -- 172.21.15.83:0/3991847000 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2c540553f0 con 0x7f2c6415ba30 2024-04-17T10:49:15.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.409+0000 7f2c6b3fb640 1 -- 172.21.15.83:0/3991847000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c6415eb00 con 0x7f2c6415ba30 2024-04-17T10:49:15.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.409+0000 7f2c59ffb640 1 -- 172.21.15.83:0/3991847000 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f2c54054cf0 con 0x7f2c6415ba30 2024-04-17T10:49:15.407 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:15.407 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":71,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5786,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1553533838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1553533838},{"type":"v1","addr":"172.21.15.167:6833","nonce":1553533838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/674536913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":674536913},{"type":"v1","addr":"172.21.15.167:6835","nonce":674536913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8598,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4183890457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4183890457},{"type":"v1","addr":"172.21.15.83:6835","nonce":4183890457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":9114,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1684302968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1684302968},{"type":"v1","addr":"172.21.15.83:6837","nonce":1684302968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[]} 2024-04-17T10:49:15.408 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 71 2024-04-17T10:49:15.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.413+0000 7f2c6b3fb640 1 -- 172.21.15.83:0/3991847000 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2c48066f60 msgr2=0x7f2c48069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:15.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.413+0000 7f2c6b3fb640 1 --2- 172.21.15.83:0/3991847000 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2c48066f60 0x7f2c48069420 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2c600ad000 tx=0x7f2c60008040 comp rx=0 tx=0).stop 2024-04-17T10:49:15.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.413+0000 7f2c6b3fb640 1 -- 172.21.15.83:0/3991847000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c6415ba30 msgr2=0x7f2c6414cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:15.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.413+0000 7f2c6b3fb640 1 --2- 172.21.15.83:0/3991847000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c6415ba30 0x7f2c6414cd80 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f2c5400c410 tx=0x7f2c5400c8d0 comp rx=0 tx=0).stop 2024-04-17T10:49:15.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.413+0000 7f2c6b3fb640 1 -- 172.21.15.83:0/3991847000 shutdown_connections 2024-04-17T10:49:15.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.413+0000 7f2c6b3fb640 1 --2- 172.21.15.83:0/3991847000 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2c48066f60 0x7f2c48069420 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:15.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.413+0000 7f2c6b3fb640 1 --2- 172.21.15.83:0/3991847000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c6415c960 0x7f2c64146bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:15.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.413+0000 7f2c6b3fb640 1 --2- 172.21.15.83:0/3991847000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c6415ba30 0x7f2c6414cd80 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:15.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.413+0000 7f2c6b3fb640 1 --2- 172.21.15.83:0/3991847000 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c640cfbe0 0x7f2c640adc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:15.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.413+0000 7f2c6b3fb640 1 -- 172.21.15.83:0/3991847000 >> 172.21.15.83:0/3991847000 conn(0x7f2c6406abe0 msgr2=0x7f2c640af2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:15.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.413+0000 7f2c6b3fb640 1 -- 172.21.15.83:0/3991847000 shutdown_connections 2024-04-17T10:49:15.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.413+0000 7f2c6b3fb640 1 -- 172.21.15.83:0/3991847000 wait complete. 2024-04-17T10:49:15.426 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-17T10:49:15.426 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-17T10:49:15.427 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-17T10:49:15.427 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:49:15.427 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-17T10:49:15.427 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-17T10:49:15.427 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-17T10:49:15.427 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:49:15.428 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-17T10:49:15.428 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-17T10:49:15.428 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-17T10:49:15.428 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:49:15.428 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-17T10:49:15.428 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-17T10:49:15.428 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-17T10:49:15.429 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:49:15.437 INFO:tasks.ceph.mds.a:Stopped 2024-04-17T10:49:15.437 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-17T10:49:15.440 INFO:tasks.ceph.mds.c:Stopped 2024-04-17T10:49:15.440 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-17T10:49:15.442 INFO:tasks.ceph.mds.d:Stopped 2024-04-17T10:49:15.442 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-17T10:49:15.442 INFO:tasks.ceph.mds.b:Stopped 2024-04-17T10:49:15.443 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-17T10:49:15.484 INFO:tasks.ceph.mds.a:Started 2024-04-17T10:49:15.485 DEBUG:teuthology.parallel:result is None 2024-04-17T10:49:15.486 INFO:tasks.ceph.mds.c:Started 2024-04-17T10:49:15.486 DEBUG:teuthology.parallel:result is None 2024-04-17T10:49:15.487 INFO:tasks.ceph.mds.d:Started 2024-04-17T10:49:15.487 DEBUG:teuthology.parallel:result is None 2024-04-17T10:49:15.487 INFO:tasks.ceph.mds.b:Started 2024-04-17T10:49:15.488 DEBUG:teuthology.parallel:result is None 2024-04-17T10:49:15.488 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-17T10:49:15.543 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-04-17T10:49:15.549 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-04-17T10:49:15.551 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2024-04-17T10:49:15.559 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2024-04-17T10:49:15.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.605+0000 7fafe0af5640 1 Processor -- start 2024-04-17T10:49:15.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.605+0000 7fafe0af5640 1 -- start start 2024-04-17T10:49:15.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.605+0000 7fafe0af5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafdc15f600 0x7fafdc0af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:15.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.605+0000 7fafe0af5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fafdc15f9e0 0x7fafdc0add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:15.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.605+0000 7fafe0af5640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafdc0cc720 con 0x7fafdc0ca520 2024-04-17T10:49:15.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.605+0000 7fafe0af5640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafdc0713d0 con 0x7fafdc15f9e0 2024-04-17T10:49:15.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.605+0000 7fafe0af5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc0acf80 con 0x7fafdc15f600 2024-04-17T10:49:15.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.605+0000 7fafd9d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fafdc0ca520 0x7fafdc070bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:36692/0 (socket says 172.21.15.83:36692) 2024-04-17T10:49:15.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.605+0000 7fafd9d9b640 1 -- 172.21.15.83:0/1199220676 learned_addr learned my addr 172.21.15.83:0/1199220676 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:15.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.605+0000 7fafdad9d640 1 --2- 172.21.15.83:0/1199220676 >> v2:172.21.15.167:3300/0 conn(0x7fafdc15f9e0 0x7fafdc0add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:15.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.605+0000 7fafda59c640 1 --2- 172.21.15.83:0/1199220676 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafdc15f600 0x7fafdc0af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:15.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.605+0000 7fafd959a640 1 -- 172.21.15.83:0/1199220676 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 782305677 0 0) 0x7fafdc0cc720 con 0x7fafdc0ca520 2024-04-17T10:49:15.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.605+0000 7fafd959a640 1 -- 172.21.15.83:0/1199220676 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafc4003530 con 0x7fafdc0ca520 2024-04-17T10:49:15.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.605+0000 7fafda59c640 1 -- 172.21.15.83:0/1199220676 >> v2:172.21.15.167:3300/0 conn(0x7fafdc15f9e0 msgr2=0x7fafdc0add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:15.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.605+0000 7fafda59c640 1 --2- 172.21.15.83:0/1199220676 >> v2:172.21.15.167:3300/0 conn(0x7fafdc15f9e0 0x7fafdc0add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:15.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.605+0000 7fafda59c640 1 -- 172.21.15.83:0/1199220676 >> v1:172.21.15.83:6789/0 conn(0x7fafdc0ca520 legacy=0x7fafdc070bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:15.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.605+0000 7fafda59c640 1 -- 172.21.15.83:0/1199220676 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafdc0ad260 con 0x7fafdc15f600 2024-04-17T10:49:15.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.609+0000 7fafda59c640 1 --2- 172.21.15.83:0/1199220676 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafdc15f600 0x7fafdc0af4d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009d90 tx=0x7fafcc01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c409994c943bcf30 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:15.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.609+0000 7fafd959a640 1 -- 172.21.15.83:0/1199220676 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc0b2070 con 0x7fafdc15f600 2024-04-17T10:49:15.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.609+0000 7fafd959a640 1 -- 172.21.15.83:0/1199220676 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafcc0a5830 con 0x7fafdc15f600 2024-04-17T10:49:15.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.609+0000 7fafd959a640 1 -- 172.21.15.83:0/1199220676 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc0a5be0 con 0x7fafdc15f600 2024-04-17T10:49:15.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.609+0000 7fafe0af5640 1 -- 172.21.15.83:0/1199220676 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafdc15f600 msgr2=0x7fafdc0af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:15.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.609+0000 7fafe0af5640 1 --2- 172.21.15.83:0/1199220676 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafdc15f600 0x7fafdc0af4d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009d90 tx=0x7fafcc01c970 comp rx=0 tx=0).stop 2024-04-17T10:49:15.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.609+0000 7fafe0af5640 1 -- 172.21.15.83:0/1199220676 shutdown_connections 2024-04-17T10:49:15.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.609+0000 7fafe0af5640 1 --2- 172.21.15.83:0/1199220676 >> v2:172.21.15.167:3300/0 conn(0x7fafdc15f9e0 0x7fafdc0add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:15.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.609+0000 7fafe0af5640 1 --2- 172.21.15.83:0/1199220676 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafdc15f600 0x7fafdc0af4d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:15.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.609+0000 7fafe0af5640 1 -- 172.21.15.83:0/1199220676 >> 172.21.15.83:0/1199220676 conn(0x7fafdc06abe0 msgr2=0x7fafdc06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:15.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.609+0000 7fafe0af5640 1 -- 172.21.15.83:0/1199220676 shutdown_connections 2024-04-17T10:49:15.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.609+0000 7fafe0af5640 1 -- 172.21.15.83:0/1199220676 wait complete. 2024-04-17T10:49:15.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.609+0000 7fafe0af5640 1 Processor -- start 2024-04-17T10:49:15.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.609+0000 7fafe0af5640 1 -- start start 2024-04-17T10:49:15.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.609+0000 7fafe0af5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafdc0ca520 0x7fafdc0b4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:15.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.609+0000 7fafe0af5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafdc15f600 0x7fafdc06b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:15.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.609+0000 7fafe0af5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafdc15f9e0 0x7fafdc06bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:15.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.609+0000 7fafe0af5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc15b540 con 0x7fafdc15f9e0 2024-04-17T10:49:15.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.609+0000 7fafe0af5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc15b3c0 con 0x7fafdc0ca520 2024-04-17T10:49:15.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.609+0000 7fafe0af5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc15b6c0 con 0x7fafdc15f600 2024-04-17T10:49:15.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.609+0000 7fafdad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafdc15f9e0 0x7fafdc06bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:15.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.609+0000 7fafdad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafdc15f9e0 0x7fafdc06bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45350/0 (socket says 172.21.15.83:45350) 2024-04-17T10:49:15.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.609+0000 7fafdad9d640 1 -- 172.21.15.83:0/3684807781 learned_addr learned my addr 172.21.15.83:0/3684807781 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:15.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.609+0000 7fafda59c640 1 --2- 172.21.15.83:0/3684807781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafdc0ca520 0x7fafdc0b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:15.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.609+0000 7fafd9d9b640 1 --2- 172.21.15.83:0/3684807781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafdc15f600 0x7fafdc06b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.609+0000 7fafdad9d640 1 -- 172.21.15.83:0/3684807781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafdc15f600 msgr2=0x7fafdc06b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.609+0000 7fafdad9d640 1 --2- 172.21.15.83:0/3684807781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafdc15f600 0x7fafdc06b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.609+0000 7fafdad9d640 1 -- 172.21.15.83:0/3684807781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafdc0ca520 msgr2=0x7fafdc0b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.609+0000 7fafdad9d640 1 --2- 172.21.15.83:0/3684807781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafdc0ca520 0x7fafdc0b4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.609+0000 7fafdad9d640 1 -- 172.21.15.83:0/3684807781 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafdc0b4560 con 0x7fafdc15f9e0 2024-04-17T10:49:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.609+0000 7fafdad9d640 1 --2- 172.21.15.83:0/3684807781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafdc15f9e0 0x7fafdc06bcc0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fafd400e7f0 tx=0x7fafd400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.613+0000 7fafcaffd640 1 -- 172.21.15.83:0/3684807781 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd4017070 con 0x7fafdc15f9e0 2024-04-17T10:49:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.613+0000 7fafcaffd640 1 -- 172.21.15.83:0/3684807781 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd400c940 con 0x7fafdc15f9e0 2024-04-17T10:49:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.613+0000 7fafe0af5640 1 -- 172.21.15.83:0/3684807781 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafdc0b4800 con 0x7fafdc15f9e0 2024-04-17T10:49:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.613+0000 7fafcaffd640 1 -- 172.21.15.83:0/3684807781 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd400eea0 con 0x7fafdc15f9e0 2024-04-17T10:49:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.613+0000 7fafe0af5640 1 -- 172.21.15.83:0/3684807781 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafdc0b4dc0 con 0x7fafdc15f9e0 2024-04-17T10:49:15.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.613+0000 7fafe0af5640 1 -- 172.21.15.83:0/3684807781 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafdc074f90 con 0x7fafdc15f9e0 2024-04-17T10:49:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.613+0000 7fafcaffd640 1 -- 172.21.15.83:0/3684807781 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fafd401a070 con 0x7fafdc15f9e0 2024-04-17T10:49:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.613+0000 7fafcaffd640 1 --2- 172.21.15.83:0/3684807781 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fafbc066f60 0x7fafbc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.613+0000 7fafcaffd640 1 -- 172.21.15.83:0/3684807781 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fafd4084b80 con 0x7fafdc15f9e0 2024-04-17T10:49:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.613+0000 7fafda59c640 1 --2- 172.21.15.83:0/3684807781 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fafbc066f60 0x7fafbc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:15.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.613+0000 7fafda59c640 1 --2- 172.21.15.83:0/3684807781 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fafbc066f60 0x7fafbc069420 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fafcc0af000 tx=0x7fafcc0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:15.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.617+0000 7fafcaffd640 1 -- 172.21.15.83:0/3684807781 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fafd4055820 con 0x7fafdc15f9e0 2024-04-17T10:49:15.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:15.777+0000 7fafe0af5640 1 -- 172.21.15.83:0/3684807781 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fafdc0c6aa0 con 0x7fafdc15f9e0 2024-04-17T10:49:16.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.585+0000 7fafcaffd640 1 -- 172.21.15.83:0/3684807781 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v121) v1 ==== 97+0+0 (secure 0 0 0) 0x7fafd40598c0 con 0x7fafdc15f9e0 2024-04-17T10:49:16.636 INFO:teuthology.orchestra.run.smithi083.stderr: removed all blocklist entries 2024-04-17T10:49:16.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.589+0000 7fafe0af5640 1 -- 172.21.15.83:0/3684807781 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fafbc066f60 msgr2=0x7fafbc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:16.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.589+0000 7fafe0af5640 1 --2- 172.21.15.83:0/3684807781 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fafbc066f60 0x7fafbc069420 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fafcc0af000 tx=0x7fafcc0a7070 comp rx=0 tx=0).stop 2024-04-17T10:49:16.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.589+0000 7fafe0af5640 1 -- 172.21.15.83:0/3684807781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafdc15f9e0 msgr2=0x7fafdc06bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:16.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.589+0000 7fafe0af5640 1 --2- 172.21.15.83:0/3684807781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafdc15f9e0 0x7fafdc06bcc0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fafd400e7f0 tx=0x7fafd400ecb0 comp rx=0 tx=0).stop 2024-04-17T10:49:16.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.589+0000 7fafe0af5640 1 -- 172.21.15.83:0/3684807781 shutdown_connections 2024-04-17T10:49:16.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.589+0000 7fafe0af5640 1 --2- 172.21.15.83:0/3684807781 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fafbc066f60 0x7fafbc069420 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:16.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.589+0000 7fafe0af5640 1 --2- 172.21.15.83:0/3684807781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafdc15f9e0 0x7fafdc06bcc0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:16.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.589+0000 7fafe0af5640 1 --2- 172.21.15.83:0/3684807781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafdc15f600 0x7fafdc06b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:16.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.589+0000 7fafe0af5640 1 --2- 172.21.15.83:0/3684807781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafdc0ca520 0x7fafdc0b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:16.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.589+0000 7fafe0af5640 1 -- 172.21.15.83:0/3684807781 >> 172.21.15.83:0/3684807781 conn(0x7fafdc06abe0 msgr2=0x7fafdc0afae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:16.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.589+0000 7fafe0af5640 1 -- 172.21.15.83:0/3684807781 shutdown_connections 2024-04-17T10:49:16.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.589+0000 7fafe0af5640 1 -- 172.21.15.83:0/3684807781 wait complete. 2024-04-17T10:49:16.638 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-17T10:49:16.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.761+0000 7fd968cac640 1 Processor -- start 2024-04-17T10:49:16.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.761+0000 7fd968cac640 1 -- start start 2024-04-17T10:49:16.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.761+0000 7fd968cac640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd96415db50 0x7fd964070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:16.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.761+0000 7fd968cac640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd96406dcb0 0x7fd9640b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:16.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.761+0000 7fd968cac640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9640709b0 con 0x7fd9640aedd0 2024-04-17T10:49:16.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.761+0000 7fd968cac640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd96415e440 con 0x7fd96415db50 2024-04-17T10:49:16.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.761+0000 7fd968cac640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd96415e720 con 0x7fd96406dcb0 2024-04-17T10:49:16.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.761+0000 7fd96259c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd96415db50 0x7fd964070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:16.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.761+0000 7fd96259c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd96415db50 0x7fd964070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35404/0 (socket says 172.21.15.83:35404) 2024-04-17T10:49:16.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.761+0000 7fd96259c640 1 -- 172.21.15.83:0/112259066 learned_addr learned my addr 172.21.15.83:0/112259066 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:16.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.761+0000 7fd962d9d640 1 --2- 172.21.15.83:0/112259066 >> v2:172.21.15.167:3300/0 conn(0x7fd96406dcb0 0x7fd9640b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:16.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.761+0000 7fd96259c640 1 -- 172.21.15.83:0/112259066 >> v2:172.21.15.167:3300/0 conn(0x7fd96406dcb0 msgr2=0x7fd9640b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:16.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.761+0000 7fd96259c640 1 --2- 172.21.15.83:0/112259066 >> v2:172.21.15.167:3300/0 conn(0x7fd96406dcb0 0x7fd9640b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:16.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.761+0000 7fd96259c640 1 -- 172.21.15.83:0/112259066 >> v1:172.21.15.167:6789/0 conn(0x7fd9640aedd0 legacy=0x7fd96406d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:16.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.761+0000 7fd96259c640 1 -- 172.21.15.83:0/112259066 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd96415ea00 con 0x7fd96415db50 2024-04-17T10:49:16.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd96259c640 1 --2- 172.21.15.83:0/112259066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96415db50 0x7fd964070470 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fd954009870 tx=0x7fd95401c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8209e66527c49920 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:16.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd96159a640 1 -- 172.21.15.83:0/112259066 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9540b0070 con 0x7fd96415db50 2024-04-17T10:49:16.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd96159a640 1 -- 172.21.15.83:0/112259066 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9540ab600 con 0x7fd96415db50 2024-04-17T10:49:16.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd96159a640 1 -- 172.21.15.83:0/112259066 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9540ab9d0 con 0x7fd96415db50 2024-04-17T10:49:16.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd968cac640 1 -- 172.21.15.83:0/112259066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96415db50 msgr2=0x7fd964070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:16.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd968cac640 1 --2- 172.21.15.83:0/112259066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96415db50 0x7fd964070470 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fd954009870 tx=0x7fd95401c2d0 comp rx=0 tx=0).stop 2024-04-17T10:49:16.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd968cac640 1 -- 172.21.15.83:0/112259066 shutdown_connections 2024-04-17T10:49:16.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd968cac640 1 --2- 172.21.15.83:0/112259066 >> v2:172.21.15.167:3300/0 conn(0x7fd96406dcb0 0x7fd9640b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:16.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd968cac640 1 --2- 172.21.15.83:0/112259066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96415db50 0x7fd964070470 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:16.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd968cac640 1 -- 172.21.15.83:0/112259066 >> 172.21.15.83:0/112259066 conn(0x7fd96406abe0 msgr2=0x7fd96406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:16.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd968cac640 1 -- 172.21.15.83:0/112259066 shutdown_connections 2024-04-17T10:49:16.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd968cac640 1 -- 172.21.15.83:0/112259066 wait complete. 2024-04-17T10:49:16.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd968cac640 1 Processor -- start 2024-04-17T10:49:16.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd968cac640 1 -- start start 2024-04-17T10:49:16.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd968cac640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd96406dcb0 0x7fd96413c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:16.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd968cac640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9640aedd0 0x7fd9640a9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:16.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd968cac640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96415db50 0x7fd964154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:16.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd968cac640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9640b3ce0 con 0x7fd96415db50 2024-04-17T10:49:16.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd968cac640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9640b3b60 con 0x7fd9640aedd0 2024-04-17T10:49:16.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd968cac640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9640b3e60 con 0x7fd96406dcb0 2024-04-17T10:49:16.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd962d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96415db50 0x7fd964154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:16.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd962d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96415db50 0x7fd964154780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35408/0 (socket says 172.21.15.83:35408) 2024-04-17T10:49:16.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd962d9d640 1 -- 172.21.15.83:0/2530144367 learned_addr learned my addr 172.21.15.83:0/2530144367 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:16.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd962d9d640 1 -- 172.21.15.83:0/2530144367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd96406dcb0 msgr2=0x7fd96413c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:16.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd96259c640 1 --2- 172.21.15.83:0/2530144367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd96406dcb0 0x7fd96413c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:16.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd962d9d640 1 --2- 172.21.15.83:0/2530144367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd96406dcb0 0x7fd96413c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:16.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd961d9b640 1 --2- 172.21.15.83:0/2530144367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9640aedd0 0x7fd9640a9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:16.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd962d9d640 1 -- 172.21.15.83:0/2530144367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9640aedd0 msgr2=0x7fd9640a9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:16.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd962d9d640 1 --2- 172.21.15.83:0/2530144367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9640aedd0 0x7fd9640a9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:16.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd962d9d640 1 -- 172.21.15.83:0/2530144367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd96413d440 con 0x7fd96415db50 2024-04-17T10:49:16.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd962d9d640 1 --2- 172.21.15.83:0/2530144367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96415db50 0x7fd964154780 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fd95c007e30 tx=0x7fd95c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:16.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd95affd640 1 -- 172.21.15.83:0/2530144367 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c014620 con 0x7fd96415db50 2024-04-17T10:49:16.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd95affd640 1 -- 172.21.15.83:0/2530144367 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd95c019070 con 0x7fd96415db50 2024-04-17T10:49:16.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.765+0000 7fd95affd640 1 -- 172.21.15.83:0/2530144367 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c014a00 con 0x7fd96415db50 2024-04-17T10:49:16.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.769+0000 7fd968cac640 1 -- 172.21.15.83:0/2530144367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd96413d720 con 0x7fd96415db50 2024-04-17T10:49:16.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.769+0000 7fd968cac640 1 -- 172.21.15.83:0/2530144367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd96413dce0 con 0x7fd96415db50 2024-04-17T10:49:16.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.769+0000 7fd968cac640 1 -- 172.21.15.83:0/2530144367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd964074f90 con 0x7fd96415db50 2024-04-17T10:49:16.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.769+0000 7fd95affd640 1 -- 172.21.15.83:0/2530144367 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd95c014ba0 con 0x7fd96415db50 2024-04-17T10:49:16.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.769+0000 7fd95affd640 1 --2- 172.21.15.83:0/2530144367 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd944066f60 0x7fd944069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:16.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.769+0000 7fd95affd640 1 -- 172.21.15.83:0/2530144367 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd95c085570 con 0x7fd96415db50 2024-04-17T10:49:16.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.769+0000 7fd96259c640 1 --2- 172.21.15.83:0/2530144367 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd944066f60 0x7fd944069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:16.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.769+0000 7fd96259c640 1 --2- 172.21.15.83:0/2530144367 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd944066f60 0x7fd944069420 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd9540ad000 tx=0x7fd954008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:16.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.773+0000 7fd95affd640 1 -- 172.21.15.83:0/2530144367 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd95c0568d0 con 0x7fd96415db50 2024-04-17T10:49:16.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.993+0000 7fd968cac640 1 -- 172.21.15.83:0/2530144367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd964071780 con 0x7fd96415db50 2024-04-17T10:49:16.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.993+0000 7fd95affd640 1 -- 172.21.15.83:0/2530144367 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fd95c0561d0 con 0x7fd96415db50 2024-04-17T10:49:16.991 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:16.991 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:49:16.991 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_dump": [ 2024-04-17T10:49:16.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:49:16.991 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.a", 2024-04-17T10:49:16.991 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mfpxsNxAANpL7rcWevXwd2KMftSQnEQ==", 2024-04-17T10:49:16.991 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:49:16.991 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:49:16.991 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:49:16.991 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:49:16.992 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:49:16.992 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:49:16.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:49:16.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:49:16.992 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.b", 2024-04-17T10:49:16.992 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mXlI9ChAAYdcrGVmtYdtWWgCLVmpTTg==", 2024-04-17T10:49:16.992 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:49:16.992 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:49:16.992 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:49:16.992 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:49:16.992 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:49:16.992 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:49:16.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:49:16.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:49:16.993 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.c", 2024-04-17T10:49:16.993 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mupo3BBAAyKH801vpFLXYs9mfzSlT0w==", 2024-04-17T10:49:16.993 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:49:16.993 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:49:16.993 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:49:16.993 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:49:16.993 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:49:16.993 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:49:16.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:49:16.993 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:49:16.993 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.d", 2024-04-17T10:49:16.993 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mbm++EhAAIBMOtQoD+kPeMqR6gK5ctw==", 2024-04-17T10:49:16.993 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:49:16.994 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:49:16.994 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:49:16.994 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:49:16.994 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:49:16.994 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:49:16.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:49:16.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:49:16.994 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.0", 2024-04-17T10:49:16.994 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBXph9m1HWhIRAAOq+9nGq7OGonUCE2aeiAzQ==", 2024-04-17T10:49:16.994 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:49:16.994 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:49:16.994 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:49:16.994 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:49:16.994 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:49:16.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:49:16.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:49:16.995 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.1", 2024-04-17T10:49:16.995 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBaph9mlMmtJRAAXHLMjXfm/+WjIFURLkbAfA==", 2024-04-17T10:49:16.995 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:49:16.995 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:49:16.995 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:49:16.995 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:49:16.995 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:49:16.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:49:16.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:49:16.995 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.2", 2024-04-17T10:49:16.995 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBdph9mBsNPLRAAtR5hwGdDkjLDLS19QUDDYQ==", 2024-04-17T10:49:16.995 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:49:16.995 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:49:16.996 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:49:16.996 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:49:16.996 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:49:16.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:49:16.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:49:16.996 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.3", 2024-04-17T10:49:16.996 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBgph9mZ4EwNRAAIyL7e0EK1523dPXmbOOkzw==", 2024-04-17T10:49:16.996 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:49:16.996 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:49:16.996 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:49:16.996 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:49:16.996 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:49:16.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:49:16.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:49:16.997 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.4", 2024-04-17T10:49:16.997 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBmph9mphAsMhAAtF4uV28Nb66PNjuugB12Hw==", 2024-04-17T10:49:16.997 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:49:16.997 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:49:16.997 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:49:16.997 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:49:16.997 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:49:16.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:49:16.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:49:16.997 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.5", 2024-04-17T10:49:16.997 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBpph9mHJQNNxAA72eOFYFbhakQac1JKRNjzw==", 2024-04-17T10:49:16.997 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:49:16.997 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:49:16.998 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:49:16.998 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:49:16.998 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:49:16.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:49:16.998 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:49:16.998 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.6", 2024-04-17T10:49:16.998 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBtph9mb6RKAxAAoVX6Cp7l7j3Lm1ES2Sq+fQ==", 2024-04-17T10:49:16.998 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:49:16.998 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:49:16.998 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:49:16.998 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:49:16.998 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:49:16.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:49:16.998 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:49:16.999 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.7", 2024-04-17T10:49:16.999 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBwph9mdrdrBxAApDwjml2ahsLbRol4uL/BxA==", 2024-04-17T10:49:16.999 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:49:16.999 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:49:16.999 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:49:16.999 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:49:16.999 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:49:16.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:49:16.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:49:16.999 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.0", 2024-04-17T10:49:16.999 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mHGmXGBAAZF+JiAG9YrFtbW+dCy2IEQ==", 2024-04-17T10:49:16.999 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:49:16.999 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:49:17.000 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow r", 2024-04-17T10:49:17.000 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-17T10:49:17.000 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:49:17.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:49:17.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:49:17.000 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.1", 2024-04-17T10:49:17.000 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mB7wgIBAABdavAo/uzVuSjfkkcYtqJg==", 2024-04-17T10:49:17.000 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:49:17.000 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:49:17.000 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-17T10:49:17.000 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-17T10:49:17.000 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-17T10:49:17.000 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:49:17.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:49:17.001 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:49:17.001 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.2", 2024-04-17T10:49:17.001 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9m+wKgHRAAe8IUFx34ItLpsaDqSYyEgA==", 2024-04-17T10:49:17.001 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:49:17.001 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:49:17.001 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-17T10:49:17.001 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-17T10:49:17.001 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-17T10:49:17.001 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:49:17.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:49:17.001 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:49:17.001 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.3", 2024-04-17T10:49:17.001 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mfJ0yJRAAHH7rRh/BUIujTT04AF2l0w==", 2024-04-17T10:49:17.002 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:49:17.002 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:49:17.002 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-17T10:49:17.002 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-17T10:49:17.002 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-17T10:49:17.002 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:49:17.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:49:17.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:49:17.002 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.admin", 2024-04-17T10:49:17.002 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mjJ3GIBAA/2qaCbRiLJ5X1cv3ZGdfoA==", 2024-04-17T10:49:17.002 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:49:17.002 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-17T10:49:17.002 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:49:17.003 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:49:17.003 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:49:17.003 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:49:17.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:49:17.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:49:17.003 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.x", 2024-04-17T10:49:17.003 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mMFptNBAA/G0zEfMZTCy9Iox+MwHG3Q==", 2024-04-17T10:49:17.003 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:49:17.003 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-17T10:49:17.003 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-04-17T10:49:17.003 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:49:17.003 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:49:17.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:49:17.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:49:17.003 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.y", 2024-04-17T10:49:17.004 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mjFbRMRAAjY2d8POBzp8hEp9GSO0ylQ==", 2024-04-17T10:49:17.004 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:49:17.004 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-17T10:49:17.004 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-04-17T10:49:17.004 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:49:17.004 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:49:17.004 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:49:17.004 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-17T10:49:17.004 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:49:17.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.997+0000 7fd968cac640 1 -- 172.21.15.83:0/2530144367 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd944066f60 msgr2=0x7fd944069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:17.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.997+0000 7fd968cac640 1 --2- 172.21.15.83:0/2530144367 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd944066f60 0x7fd944069420 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd9540ad000 tx=0x7fd954008040 comp rx=0 tx=0).stop 2024-04-17T10:49:17.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.997+0000 7fd968cac640 1 -- 172.21.15.83:0/2530144367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96415db50 msgr2=0x7fd964154780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:17.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.997+0000 7fd968cac640 1 --2- 172.21.15.83:0/2530144367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96415db50 0x7fd964154780 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fd95c007e30 tx=0x7fd95c00e4b0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.997+0000 7fd968cac640 1 -- 172.21.15.83:0/2530144367 shutdown_connections 2024-04-17T10:49:17.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.997+0000 7fd968cac640 1 --2- 172.21.15.83:0/2530144367 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd944066f60 0x7fd944069420 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.997+0000 7fd968cac640 1 --2- 172.21.15.83:0/2530144367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96415db50 0x7fd964154780 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.997+0000 7fd968cac640 1 --2- 172.21.15.83:0/2530144367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9640aedd0 0x7fd9640a9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.997+0000 7fd968cac640 1 --2- 172.21.15.83:0/2530144367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd96406dcb0 0x7fd96413c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.997+0000 7fd968cac640 1 -- 172.21.15.83:0/2530144367 >> 172.21.15.83:0/2530144367 conn(0x7fd96406abe0 msgr2=0x7fd96406d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:17.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.997+0000 7fd968cac640 1 -- 172.21.15.83:0/2530144367 shutdown_connections 2024-04-17T10:49:17.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:16.997+0000 7fd968cac640 1 -- 172.21.15.83:0/2530144367 wait complete. 2024-04-17T10:49:17.010 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:49:17.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.129+0000 7feb26516640 1 Processor -- start 2024-04-17T10:49:17.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.129+0000 7feb26516640 1 -- start start 2024-04-17T10:49:17.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.129+0000 7feb26516640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feb2015ba30 0x7feb20072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:17.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.129+0000 7feb26516640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb2006e320 con 0x7feb2015c960 2024-04-17T10:49:17.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.129+0000 7feb26516640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb2006e040 con 0x7feb200cfbe0 2024-04-17T10:49:17.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.129+0000 7feb26516640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb20071ea0 con 0x7feb2015ba30 2024-04-17T10:49:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.129+0000 7feb1ffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feb2015ba30 0x7feb20072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.129+0000 7feb1f7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7feb2015c960 0x7feb200cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41380/0 (socket says 172.21.15.83:41380) 2024-04-17T10:49:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.129+0000 7feb1f7fe640 1 -- 172.21.15.83:0/1202673432 learned_addr learned my addr 172.21.15.83:0/1202673432 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.129+0000 7feb1ffff640 1 -- 172.21.15.83:0/1202673432 >> v1:172.21.15.167:6789/0 conn(0x7feb200cfbe0 legacy=0x7feb2006c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.129+0000 7feb1ffff640 1 -- 172.21.15.83:0/1202673432 >> v1:172.21.15.83:6789/0 conn(0x7feb2015c960 legacy=0x7feb200cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.129+0000 7feb1effd640 1 -- 172.21.15.83:0/1202673432 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 584272505 0 0) 0x7feb2006e320 con 0x7feb2015c960 2024-04-17T10:49:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.129+0000 7feb1ffff640 1 -- 172.21.15.83:0/1202673432 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb20071850 con 0x7feb2015ba30 2024-04-17T10:49:17.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb1ffff640 1 --2- 172.21.15.83:0/1202673432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb2015ba30 0x7feb20072330 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7feb18009d90 tx=0x7feb1801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb96619a0d6e0a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:17.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb1effd640 1 -- 172.21.15.83:0/1202673432 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb180b2070 con 0x7feb2015ba30 2024-04-17T10:49:17.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb1effd640 1 -- 172.21.15.83:0/1202673432 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb180a57b0 con 0x7feb2015ba30 2024-04-17T10:49:17.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb1effd640 1 -- 172.21.15.83:0/1202673432 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb180a5b80 con 0x7feb2015ba30 2024-04-17T10:49:17.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb26516640 1 -- 172.21.15.83:0/1202673432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb2015ba30 msgr2=0x7feb20072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:17.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb26516640 1 --2- 172.21.15.83:0/1202673432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb2015ba30 0x7feb20072330 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7feb18009d90 tx=0x7feb1801c8a0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb26516640 1 -- 172.21.15.83:0/1202673432 shutdown_connections 2024-04-17T10:49:17.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb26516640 1 --2- 172.21.15.83:0/1202673432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb2015ba30 0x7feb20072330 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb26516640 1 -- 172.21.15.83:0/1202673432 >> 172.21.15.83:0/1202673432 conn(0x7feb2006abe0 msgr2=0x7feb2006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:17.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb26516640 1 -- 172.21.15.83:0/1202673432 shutdown_connections 2024-04-17T10:49:17.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb26516640 1 -- 172.21.15.83:0/1202673432 wait complete. 2024-04-17T10:49:17.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb26516640 1 Processor -- start 2024-04-17T10:49:17.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb26516640 1 -- start start 2024-04-17T10:49:17.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb26516640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb200cfbe0 0x7feb2014b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:17.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb26516640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb2015ba30 0x7feb20143bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:17.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb26516640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb2015c960 0x7feb20147940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:17.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb26516640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2006de20 con 0x7feb200cfbe0 2024-04-17T10:49:17.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb26516640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2006dca0 con 0x7feb2015c960 2024-04-17T10:49:17.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb26516640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb20169530 con 0x7feb2015ba30 2024-04-17T10:49:17.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb1ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb200cfbe0 0x7feb2014b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:17.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb1ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb200cfbe0 0x7feb2014b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35436/0 (socket says 172.21.15.83:35436) 2024-04-17T10:49:17.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb1ffff640 1 -- 172.21.15.83:0/3774269475 learned_addr learned my addr 172.21.15.83:0/3774269475 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:17.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb1ffff640 1 -- 172.21.15.83:0/3774269475 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb2015ba30 msgr2=0x7feb20143bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:17.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb1ffff640 1 --2- 172.21.15.83:0/3774269475 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb2015ba30 0x7feb20143bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb1ffff640 1 -- 172.21.15.83:0/3774269475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb2015c960 msgr2=0x7feb20147940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:17.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb1ffff640 1 --2- 172.21.15.83:0/3774269475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb2015c960 0x7feb20147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb1ffff640 1 -- 172.21.15.83:0/3774269475 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb20152500 con 0x7feb200cfbe0 2024-04-17T10:49:17.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb1ffff640 1 --2- 172.21.15.83:0/3774269475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb200cfbe0 0x7feb2014b3d0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7feb18009d90 tx=0x7feb2006d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:17.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb1cff9640 1 -- 172.21.15.83:0/3774269475 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb180bc070 con 0x7feb200cfbe0 2024-04-17T10:49:17.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb1cff9640 1 -- 172.21.15.83:0/3774269475 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb18003ba0 con 0x7feb200cfbe0 2024-04-17T10:49:17.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb26516640 1 -- 172.21.15.83:0/3774269475 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb201527a0 con 0x7feb200cfbe0 2024-04-17T10:49:17.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb1cff9640 1 -- 172.21.15.83:0/3774269475 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb180b2040 con 0x7feb200cfbe0 2024-04-17T10:49:17.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.133+0000 7feb26516640 1 -- 172.21.15.83:0/3774269475 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb20152d60 con 0x7feb200cfbe0 2024-04-17T10:49:17.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.137+0000 7feb26516640 1 -- 172.21.15.83:0/3774269475 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb20074f90 con 0x7feb200cfbe0 2024-04-17T10:49:17.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.137+0000 7feb1cff9640 1 -- 172.21.15.83:0/3774269475 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feb180b5070 con 0x7feb200cfbe0 2024-04-17T10:49:17.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.137+0000 7feb1cff9640 1 --2- 172.21.15.83:0/3774269475 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feb0c066f60 0x7feb0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:17.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.137+0000 7feb1cff9640 1 -- 172.21.15.83:0/3774269475 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 4857+0+0 (secure 0 0 0) 0x7feb1811f5b0 con 0x7feb200cfbe0 2024-04-17T10:49:17.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.137+0000 7feb1f7fe640 1 --2- 172.21.15.83:0/3774269475 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feb0c066f60 0x7feb0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:17.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.137+0000 7feb1f7fe640 1 --2- 172.21.15.83:0/3774269475 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feb0c066f60 0x7feb0c069420 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7feb14005890 tx=0x7feb14007a10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:17.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.141+0000 7feb1cff9640 1 -- 172.21.15.83:0/3774269475 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feb180c5070 con 0x7feb200cfbe0 2024-04-17T10:49:17.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.301+0000 7feb26516640 1 -- 172.21.15.83:0/3774269475 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb20152f30 con 0x7feb200cfbe0 2024-04-17T10:49:17.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.301+0000 7feb1cff9640 1 -- 172.21.15.83:0/3774269475 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+12829 (secure 0 0 0) 0x7feb18016520 con 0x7feb200cfbe0 2024-04-17T10:49:17.299 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:17.299 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":123,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:49:16.604179+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:49:17.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.305+0000 7feb26516640 1 -- 172.21.15.83:0/3774269475 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feb0c066f60 msgr2=0x7feb0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:17.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.305+0000 7feb26516640 1 --2- 172.21.15.83:0/3774269475 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feb0c066f60 0x7feb0c069420 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7feb14005890 tx=0x7feb14007a10 comp rx=0 tx=0).stop 2024-04-17T10:49:17.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.305+0000 7feb26516640 1 -- 172.21.15.83:0/3774269475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb200cfbe0 msgr2=0x7feb2014b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:17.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.305+0000 7feb26516640 1 --2- 172.21.15.83:0/3774269475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb200cfbe0 0x7feb2014b3d0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7feb18009d90 tx=0x7feb2006d3b0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.305+0000 7feb26516640 1 -- 172.21.15.83:0/3774269475 shutdown_connections 2024-04-17T10:49:17.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.305+0000 7feb26516640 1 --2- 172.21.15.83:0/3774269475 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feb0c066f60 0x7feb0c069420 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.305+0000 7feb26516640 1 --2- 172.21.15.83:0/3774269475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb2015c960 0x7feb20147940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.305+0000 7feb26516640 1 --2- 172.21.15.83:0/3774269475 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb2015ba30 0x7feb20143bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.305+0000 7feb26516640 1 --2- 172.21.15.83:0/3774269475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb200cfbe0 0x7feb2014b3d0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.305+0000 7feb26516640 1 -- 172.21.15.83:0/3774269475 >> 172.21.15.83:0/3774269475 conn(0x7feb2006abe0 msgr2=0x7feb2015bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:17.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.305+0000 7feb26516640 1 -- 172.21.15.83:0/3774269475 shutdown_connections 2024-04-17T10:49:17.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.305+0000 7feb26516640 1 -- 172.21.15.83:0/3774269475 wait complete. 2024-04-17T10:49:17.319 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:49:17.319 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:49:17.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.445+0000 7f78ac295640 1 Processor -- start 2024-04-17T10:49:17.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.445+0000 7f78ac295640 1 -- start start 2024-04-17T10:49:17.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.445+0000 7f78ac295640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f78a406ae10 0x7f78a406b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:17.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.445+0000 7f78ac295640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f78a415e330 0x7f78a415e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:17.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.445+0000 7f78ac295640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78a40ad840 0x7f78a40adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:17.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.445+0000 7f78ac295640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a41625d0 con 0x7f78a415e330 2024-04-17T10:49:17.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.445+0000 7f78ac295640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a406b730 con 0x7f78a406ae10 2024-04-17T10:49:17.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.445+0000 7f78ac295640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a4161810 con 0x7f78a40ad840 2024-04-17T10:49:17.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.445+0000 7f78a9830640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f78a415e330 0x7f78a415e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:17.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.445+0000 7f78a9830640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f78a415e330 0x7f78a415e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35444/0 (socket says 172.21.15.83:35444) 2024-04-17T10:49:17.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.445+0000 7f78a9830640 1 -- 172.21.15.83:0/1512661009 learned_addr learned my addr 172.21.15.83:0/1512661009 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:17.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.445+0000 7f78aa832640 1 --2- 172.21.15.83:0/1512661009 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78a40ad840 0x7f78a40adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:17.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.445+0000 7f78aa031640 1 --2- 172.21.15.83:0/1512661009 >> v2:172.21.15.167:3300/0 conn(0x7f78a406ae10 0x7f78a406b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:17.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.445+0000 7f78a9830640 1 -- 172.21.15.83:0/1512661009 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78a40ad840 msgr2=0x7f78a40adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:17.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.445+0000 7f78a9830640 1 --2- 172.21.15.83:0/1512661009 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78a40ad840 0x7f78a40adcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.445+0000 7f78a9830640 1 -- 172.21.15.83:0/1512661009 >> v2:172.21.15.167:3300/0 conn(0x7f78a406ae10 msgr2=0x7f78a406b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:17.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.445+0000 7f78a9830640 1 --2- 172.21.15.83:0/1512661009 >> v2:172.21.15.167:3300/0 conn(0x7f78a406ae10 0x7f78a406b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.445+0000 7f78a9830640 1 -- 172.21.15.83:0/1512661009 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a4161af0 con 0x7f78a415e330 2024-04-17T10:49:17.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.445+0000 7f78aa031640 1 --2- 172.21.15.83:0/1512661009 >> v2:172.21.15.167:3300/0 conn(0x7f78a406ae10 0x7f78a406b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:17.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78a9830640 1 --2- 172.21.15.83:0/1512661009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a415e330 0x7f78a415e7a0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f78a40aece0 tx=0x7f789801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85cfec0c141e5a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:17.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78a902f640 1 -- 172.21.15.83:0/1512661009 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78980b0070 con 0x7f78a415e330 2024-04-17T10:49:17.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78a902f640 1 -- 172.21.15.83:0/1512661009 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78980ab600 con 0x7f78a415e330 2024-04-17T10:49:17.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78a902f640 1 -- 172.21.15.83:0/1512661009 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78980ab9d0 con 0x7f78a415e330 2024-04-17T10:49:17.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78ac295640 1 -- 172.21.15.83:0/1512661009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a415e330 msgr2=0x7f78a415e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:17.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78ac295640 1 --2- 172.21.15.83:0/1512661009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a415e330 0x7f78a415e7a0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f78a40aece0 tx=0x7f789801c490 comp rx=0 tx=0).stop 2024-04-17T10:49:17.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78ac295640 1 -- 172.21.15.83:0/1512661009 shutdown_connections 2024-04-17T10:49:17.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78ac295640 1 --2- 172.21.15.83:0/1512661009 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78a40ad840 0x7f78a40adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78ac295640 1 --2- 172.21.15.83:0/1512661009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a415e330 0x7f78a415e7a0 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78ac295640 1 --2- 172.21.15.83:0/1512661009 >> v2:172.21.15.167:3300/0 conn(0x7f78a406ae10 0x7f78a406b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78ac295640 1 -- 172.21.15.83:0/1512661009 >> 172.21.15.83:0/1512661009 conn(0x7f78a40b15b0 msgr2=0x7f78a40b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:17.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78ac295640 1 -- 172.21.15.83:0/1512661009 shutdown_connections 2024-04-17T10:49:17.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78ac295640 1 -- 172.21.15.83:0/1512661009 wait complete. 2024-04-17T10:49:17.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78ac295640 1 Processor -- start 2024-04-17T10:49:17.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78ac295640 1 -- start start 2024-04-17T10:49:17.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78ac295640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78a406ae10 0x7f78a4157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:17.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78ac295640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a40ad840 0x7f78a406c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:17.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78ac295640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78a415e330 0x7f78a406c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:17.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78ac295640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a40cf470 con 0x7f78a40ad840 2024-04-17T10:49:17.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78ac295640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a40cf2f0 con 0x7f78a415e330 2024-04-17T10:49:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78ac295640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a415ece0 con 0x7f78a406ae10 2024-04-17T10:49:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78a9830640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a40ad840 0x7f78a406c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78a9830640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a40ad840 0x7f78a406c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35450/0 (socket says 172.21.15.83:35450) 2024-04-17T10:49:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78a9830640 1 -- 172.21.15.83:0/3218722687 learned_addr learned my addr 172.21.15.83:0/3218722687 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78aa832640 1 --2- 172.21.15.83:0/3218722687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78a415e330 0x7f78a406c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78aa031640 1 --2- 172.21.15.83:0/3218722687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78a406ae10 0x7f78a4157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78a9830640 1 -- 172.21.15.83:0/3218722687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78a406ae10 msgr2=0x7f78a4157300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78a9830640 1 --2- 172.21.15.83:0/3218722687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78a406ae10 0x7f78a4157300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78a9830640 1 -- 172.21.15.83:0/3218722687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78a415e330 msgr2=0x7f78a406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78a9830640 1 --2- 172.21.15.83:0/3218722687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78a415e330 0x7f78a406c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78a9830640 1 -- 172.21.15.83:0/3218722687 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a4158ce0 con 0x7f78a40ad840 2024-04-17T10:49:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78aa031640 1 --2- 172.21.15.83:0/3218722687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78a406ae10 0x7f78a4157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.449+0000 7f78a9830640 1 --2- 172.21.15.83:0/3218722687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a40ad840 0x7f78a406c390 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f789801ab40 tx=0x7f7898019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.453+0000 7f7896ffd640 1 -- 172.21.15.83:0/3218722687 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78980b0040 con 0x7f78a40ad840 2024-04-17T10:49:17.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.453+0000 7f7896ffd640 1 -- 172.21.15.83:0/3218722687 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78980b9070 con 0x7f78a40ad840 2024-04-17T10:49:17.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.453+0000 7f78ac295640 1 -- 172.21.15.83:0/3218722687 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a4140170 con 0x7f78a40ad840 2024-04-17T10:49:17.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.453+0000 7f7896ffd640 1 -- 172.21.15.83:0/3218722687 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78980b4890 con 0x7f78a40ad840 2024-04-17T10:49:17.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.453+0000 7f78ac295640 1 -- 172.21.15.83:0/3218722687 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a4145c40 con 0x7f78a40ad840 2024-04-17T10:49:17.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.453+0000 7f78ac295640 1 -- 172.21.15.83:0/3218722687 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a4074f90 con 0x7f78a40ad840 2024-04-17T10:49:17.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.453+0000 7f7896ffd640 1 -- 172.21.15.83:0/3218722687 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f78980ab4c0 con 0x7f78a40ad840 2024-04-17T10:49:17.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.453+0000 7f7896ffd640 1 --2- 172.21.15.83:0/3218722687 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7888066f60 0x7f7888069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:17.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.453+0000 7f7896ffd640 1 -- 172.21.15.83:0/3218722687 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f789811e530 con 0x7f78a40ad840 2024-04-17T10:49:17.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.453+0000 7f78aa031640 1 --2- 172.21.15.83:0/3218722687 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7888066f60 0x7f7888069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:17.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.453+0000 7f78aa031640 1 --2- 172.21.15.83:0/3218722687 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7888066f60 0x7f7888069420 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f78a0006fd0 tx=0x7f78a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:17.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.457+0000 7f7896ffd640 1 -- 172.21.15.83:0/3218722687 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f78a4158ce0 con 0x7f78a40ad840 2024-04-17T10:49:17.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.609+0000 7f78ac295640 1 -- 172.21.15.83:0/3218722687 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f78a4072740 con 0x7f78a40ad840 2024-04-17T10:49:17.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.613+0000 7f7896ffd640 1 -- 172.21.15.83:0/3218722687 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7898016110 con 0x7f78a40ad840 2024-04-17T10:49:17.609 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:49:17.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.613+0000 7f78ac295640 1 -- 172.21.15.83:0/3218722687 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7888066f60 msgr2=0x7f7888069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:17.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.613+0000 7f78ac295640 1 --2- 172.21.15.83:0/3218722687 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7888066f60 0x7f7888069420 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f78a0006fd0 tx=0x7f78a0008040 comp rx=0 tx=0).stop 2024-04-17T10:49:17.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.613+0000 7f78ac295640 1 -- 172.21.15.83:0/3218722687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a40ad840 msgr2=0x7f78a406c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:17.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.613+0000 7f78ac295640 1 --2- 172.21.15.83:0/3218722687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a40ad840 0x7f78a406c390 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f789801ab40 tx=0x7f7898019230 comp rx=0 tx=0).stop 2024-04-17T10:49:17.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.613+0000 7f78ac295640 1 -- 172.21.15.83:0/3218722687 shutdown_connections 2024-04-17T10:49:17.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.613+0000 7f78ac295640 1 --2- 172.21.15.83:0/3218722687 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7888066f60 0x7f7888069420 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.613+0000 7f78ac295640 1 --2- 172.21.15.83:0/3218722687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78a415e330 0x7f78a406c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.613+0000 7f78ac295640 1 --2- 172.21.15.83:0/3218722687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a40ad840 0x7f78a406c390 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.613+0000 7f78ac295640 1 --2- 172.21.15.83:0/3218722687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78a406ae10 0x7f78a4157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.613+0000 7f78ac295640 1 -- 172.21.15.83:0/3218722687 >> 172.21.15.83:0/3218722687 conn(0x7f78a40b15b0 msgr2=0x7f78a406cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:17.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.613+0000 7f78ac295640 1 -- 172.21.15.83:0/3218722687 shutdown_connections 2024-04-17T10:49:17.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.613+0000 7f78ac295640 1 -- 172.21.15.83:0/3218722687 wait complete. 2024-04-17T10:49:17.627 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-17T10:49:17.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.741+0000 7f86ef74b640 1 Processor -- start 2024-04-17T10:49:17.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.741+0000 7f86ef74b640 1 -- start start 2024-04-17T10:49:17.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.741+0000 7f86ef74b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f86e815eeb0 0x7f86e806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:17.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.741+0000 7f86ef74b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86e806e980 con 0x7f86e80cf780 2024-04-17T10:49:17.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.741+0000 7f86ef74b640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86e806e0e0 con 0x7f86e806c300 2024-04-17T10:49:17.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.741+0000 7f86ef74b640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86e80cdb90 con 0x7f86e815eeb0 2024-04-17T10:49:17.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.741+0000 7f86ecce6640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f86e80cf780 0x7f86e806bbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41388/0 (socket says 172.21.15.83:41388) 2024-04-17T10:49:17.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.741+0000 7f86ecce6640 1 -- 172.21.15.83:0/4107745243 learned_addr learned my addr 172.21.15.83:0/4107745243 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:17.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.741+0000 7f86ed4e7640 1 --2- 172.21.15.83:0/4107745243 >> v2:172.21.15.167:3300/0 conn(0x7f86e815eeb0 0x7f86e806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:17.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.741+0000 7f86dbfff640 1 -- 172.21.15.83:0/4107745243 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3909543118 0 0) 0x7f86e806e980 con 0x7f86e80cf780 2024-04-17T10:49:17.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.741+0000 7f86dbfff640 1 -- 172.21.15.83:0/4107745243 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86d4003530 con 0x7f86e80cf780 2024-04-17T10:49:17.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.741+0000 7f86dbfff640 1 -- 172.21.15.83:0/4107745243 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1090429565 0 0) 0x7f86d4003530 con 0x7f86e80cf780 2024-04-17T10:49:17.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.741+0000 7f86dbfff640 1 -- 172.21.15.83:0/4107745243 >> v2:172.21.15.167:3300/0 conn(0x7f86e815eeb0 msgr2=0x7f86e806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:17.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.741+0000 7f86dbfff640 1 --2- 172.21.15.83:0/4107745243 >> v2:172.21.15.167:3300/0 conn(0x7f86e815eeb0 0x7f86e806d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.741+0000 7f86dbfff640 1 -- 172.21.15.83:0/4107745243 >> v1:172.21.15.167:6789/0 conn(0x7f86e806c300 legacy=0x7f86e80ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:17.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.741+0000 7f86dbfff640 1 -- 172.21.15.83:0/4107745243 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e806e3c0 con 0x7f86e80cf780 2024-04-17T10:49:17.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.741+0000 7f86dbfff640 1 -- 172.21.15.83:0/4107745243 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f86dc001a00 con 0x7f86e80cf780 2024-04-17T10:49:17.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.741+0000 7f86dbfff640 1 -- 172.21.15.83:0/4107745243 >> v1:172.21.15.83:6789/0 conn(0x7f86e80cf780 legacy=0x7f86e806bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:17.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.741+0000 7f86dbfff640 1 --2- 172.21.15.83:0/4107745243 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86d40039c0 0x7f86d4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:17.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.741+0000 7f86dbfff640 1 --2- 172.21.15.83:0/4107745243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86d4004810 0x7f86d4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:17.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.741+0000 7f86dbfff640 1 --2- 172.21.15.83:0/4107745243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86d40051d0 0x7f86d400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:17.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.741+0000 7f86dbfff640 1 -- 172.21.15.83:0/4107745243 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e80cdb90 con 0x7f86d4004810 2024-04-17T10:49:17.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.741+0000 7f86dbfff640 1 -- 172.21.15.83:0/4107745243 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d400c070 con 0x7f86d40051d0 2024-04-17T10:49:17.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.741+0000 7f86dbfff640 1 -- 172.21.15.83:0/4107745243 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d400c270 con 0x7f86d40039c0 2024-04-17T10:49:17.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.741+0000 7f86ecce6640 1 --2- 172.21.15.83:0/4107745243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86d4004810 0x7f86d4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:17.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.741+0000 7f86edce8640 1 --2- 172.21.15.83:0/4107745243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86d40051d0 0x7f86d400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:17.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.741+0000 7f86ecce6640 1 -- 172.21.15.83:0/4107745243 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86d40039c0 msgr2=0x7f86d4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:17.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.741+0000 7f86ecce6640 1 --2- 172.21.15.83:0/4107745243 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86d40039c0 0x7f86d4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.741+0000 7f86ecce6640 1 -- 172.21.15.83:0/4107745243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86d40051d0 msgr2=0x7f86d400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:17.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.741+0000 7f86ecce6640 1 --2- 172.21.15.83:0/4107745243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86d40051d0 0x7f86d400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.741+0000 7f86edce8640 1 -- 172.21.15.83:0/4107745243 reap_dead start 2024-04-17T10:49:17.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.741+0000 7f86ecce6640 1 -- 172.21.15.83:0/4107745243 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d4003530 con 0x7f86d4004810 2024-04-17T10:49:17.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.745+0000 7f86ecce6640 1 --2- 172.21.15.83:0/4107745243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86d4004810 0x7f86d4004c90 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f86dc00a810 tx=0x7f86dc01faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272f967e2dc4a5ea server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:17.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.745+0000 7f86dbfff640 1 -- 172.21.15.83:0/4107745243 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86dc00a460 con 0x7f86d4004810 2024-04-17T10:49:17.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.745+0000 7f86dbfff640 1 -- 172.21.15.83:0/4107745243 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86dc0aa6d0 con 0x7f86d4004810 2024-04-17T10:49:17.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.745+0000 7f86dbfff640 1 -- 172.21.15.83:0/4107745243 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86dc0aab00 con 0x7f86d4004810 2024-04-17T10:49:17.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.745+0000 7f86ef74b640 1 -- 172.21.15.83:0/4107745243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86d4004810 msgr2=0x7f86d4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:17.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.745+0000 7f86ef74b640 1 --2- 172.21.15.83:0/4107745243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86d4004810 0x7f86d4004c90 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f86dc00a810 tx=0x7f86dc01faa0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.745+0000 7f86ef74b640 1 -- 172.21.15.83:0/4107745243 shutdown_connections 2024-04-17T10:49:17.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.745+0000 7f86ef74b640 1 --2- 172.21.15.83:0/4107745243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86d4004810 0x7f86d4004c90 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.745+0000 7f86ef74b640 1 -- 172.21.15.83:0/4107745243 >> 172.21.15.83:0/4107745243 conn(0x7f86e80cc910 msgr2=0x7f86e80ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:17.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.745+0000 7f86ef74b640 1 -- 172.21.15.83:0/4107745243 shutdown_connections 2024-04-17T10:49:17.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.745+0000 7f86ef74b640 1 -- 172.21.15.83:0/4107745243 wait complete. 2024-04-17T10:49:17.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.745+0000 7f86ef74b640 1 Processor -- start 2024-04-17T10:49:17.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.745+0000 7f86ef74b640 1 -- start start 2024-04-17T10:49:17.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.749+0000 7f86ef74b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86d4004810 0x7f86e814ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:17.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.749+0000 7f86ef74b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86e80c4d90 0x7f86e809ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:17.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.749+0000 7f86ef74b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86e8157d60 0x7f86e809bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:17.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.749+0000 7f86ef74b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e8070d70 con 0x7f86e8157d60 2024-04-17T10:49:17.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.749+0000 7f86ef74b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e8070bf0 con 0x7f86d4004810 2024-04-17T10:49:17.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.749+0000 7f86ef74b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e815f290 con 0x7f86e80c4d90 2024-04-17T10:49:17.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.749+0000 7f86edce8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86e8157d60 0x7f86e809bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:17.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.749+0000 7f86edce8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86e8157d60 0x7f86e809bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35464/0 (socket says 172.21.15.83:35464) 2024-04-17T10:49:17.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.749+0000 7f86edce8640 1 -- 172.21.15.83:0/1117704069 learned_addr learned my addr 172.21.15.83:0/1117704069 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:17.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.749+0000 7f86ed4e7640 1 --2- 172.21.15.83:0/1117704069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86d4004810 0x7f86e814ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:17.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.749+0000 7f86ecce6640 1 --2- 172.21.15.83:0/1117704069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86e80c4d90 0x7f86e809ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:17.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.749+0000 7f86edce8640 1 -- 172.21.15.83:0/1117704069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86e80c4d90 msgr2=0x7f86e809ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:17.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.749+0000 7f86edce8640 1 --2- 172.21.15.83:0/1117704069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86e80c4d90 0x7f86e809ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.749+0000 7f86edce8640 1 -- 172.21.15.83:0/1117704069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86d4004810 msgr2=0x7f86e814ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:17.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.749+0000 7f86edce8640 1 --2- 172.21.15.83:0/1117704069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86d4004810 0x7f86e814ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.749+0000 7f86edce8640 1 -- 172.21.15.83:0/1117704069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e81f53a0 con 0x7f86e8157d60 2024-04-17T10:49:17.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.749+0000 7f86ed4e7640 1 --2- 172.21.15.83:0/1117704069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86d4004810 0x7f86e814ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:49:17.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.749+0000 7f86edce8640 1 --2- 172.21.15.83:0/1117704069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86e8157d60 0x7f86e809bf60 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f86e80709d0 tx=0x7f86e0007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:17.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.749+0000 7f86d9ffb640 1 -- 172.21.15.83:0/1117704069 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e0019070 con 0x7f86e8157d60 2024-04-17T10:49:17.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.749+0000 7f86d9ffb640 1 -- 172.21.15.83:0/1117704069 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e0005110 con 0x7f86e8157d60 2024-04-17T10:49:17.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.749+0000 7f86ef74b640 1 -- 172.21.15.83:0/1117704069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86e81f5680 con 0x7f86e8157d60 2024-04-17T10:49:17.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.749+0000 7f86d9ffb640 1 -- 172.21.15.83:0/1117704069 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e0007030 con 0x7f86e8157d60 2024-04-17T10:49:17.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.749+0000 7f86ef74b640 1 -- 172.21.15.83:0/1117704069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e81f5c40 con 0x7f86e8157d60 2024-04-17T10:49:17.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.749+0000 7f86ef74b640 1 -- 172.21.15.83:0/1117704069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86e8074f90 con 0x7f86e8157d60 2024-04-17T10:49:17.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.749+0000 7f86d9ffb640 1 -- 172.21.15.83:0/1117704069 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f86e00071d0 con 0x7f86e8157d60 2024-04-17T10:49:17.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.749+0000 7f86d9ffb640 1 --2- 172.21.15.83:0/1117704069 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f86cc066f90 0x7f86cc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:17.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.749+0000 7f86d9ffb640 1 -- 172.21.15.83:0/1117704069 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f86e0083ee0 con 0x7f86e8157d60 2024-04-17T10:49:17.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.749+0000 7f86ed4e7640 1 --2- 172.21.15.83:0/1117704069 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f86cc066f90 0x7f86cc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:17.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.749+0000 7f86ed4e7640 1 --2- 172.21.15.83:0/1117704069 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f86cc066f90 0x7f86cc069450 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f86e806a390 tx=0x7f86e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:17.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.753+0000 7f86d9ffb640 1 -- 172.21.15.83:0/1117704069 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f86e00552d0 con 0x7f86e8157d60 2024-04-17T10:49:17.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.913+0000 7f86ef74b640 1 -- 172.21.15.83:0/1117704069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f86e8068150 con 0x7f86e8157d60 2024-04-17T10:49:17.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.913+0000 7f86d9ffb640 1 -- 172.21.15.83:0/1117704069 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v123) v1 ==== 84+0+65 (secure 0 0 0) 0x7f86e0054bd0 con 0x7f86e8157d60 2024-04-17T10:49:17.910 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:17.910 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-04-17T10:49:17.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:49:17.910 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2024-04-17T10:49:17.911 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2024-04-17T10:49:17.911 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:49:17.911 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-04-17T10:49:17.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.917+0000 7f86ef74b640 1 -- 172.21.15.83:0/1117704069 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f86cc066f90 msgr2=0x7f86cc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:17.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.917+0000 7f86ef74b640 1 --2- 172.21.15.83:0/1117704069 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f86cc066f90 0x7f86cc069450 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f86e806a390 tx=0x7f86e4008040 comp rx=0 tx=0).stop 2024-04-17T10:49:17.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.917+0000 7f86ef74b640 1 -- 172.21.15.83:0/1117704069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86e8157d60 msgr2=0x7f86e809bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:17.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.917+0000 7f86ef74b640 1 --2- 172.21.15.83:0/1117704069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86e8157d60 0x7f86e809bf60 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f86e80709d0 tx=0x7f86e0007f90 comp rx=0 tx=0).stop 2024-04-17T10:49:17.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.917+0000 7f86ef74b640 1 -- 172.21.15.83:0/1117704069 shutdown_connections 2024-04-17T10:49:17.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.917+0000 7f86ef74b640 1 --2- 172.21.15.83:0/1117704069 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f86cc066f90 0x7f86cc069450 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.917+0000 7f86ef74b640 1 --2- 172.21.15.83:0/1117704069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86e8157d60 0x7f86e809bf60 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.917+0000 7f86ef74b640 1 --2- 172.21.15.83:0/1117704069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86e80c4d90 0x7f86e809ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.917+0000 7f86ef74b640 1 --2- 172.21.15.83:0/1117704069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86d4004810 0x7f86e814ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:17.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.917+0000 7f86ef74b640 1 -- 172.21.15.83:0/1117704069 >> 172.21.15.83:0/1117704069 conn(0x7f86e80cc910 msgr2=0x7f86e80719c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:17.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.917+0000 7f86ef74b640 1 -- 172.21.15.83:0/1117704069 shutdown_connections 2024-04-17T10:49:17.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:17.917+0000 7f86ef74b640 1 -- 172.21.15.83:0/1117704069 wait complete. 2024-04-17T10:49:17.928 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-17T10:49:17.929 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-17T10:49:18.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.045+0000 7f8d4bc93640 1 Processor -- start 2024-04-17T10:49:18.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.045+0000 7f8d4bc93640 1 -- start start 2024-04-17T10:49:18.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.045+0000 7f8d4bc93640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8d440aebd0 0x7f8d440ced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:18.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.045+0000 7f8d4bc93640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d4415b990 0x7f8d440cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:18.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.045+0000 7f8d4bc93640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8d440cd6c0 0x7f8d4406abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:18.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.045+0000 7f8d4bc93640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d4415e5c0 con 0x7f8d440cd6c0 2024-04-17T10:49:18.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.045+0000 7f8d4bc93640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d440cf3e0 con 0x7f8d440aebd0 2024-04-17T10:49:18.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.045+0000 7f8d4bc93640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d440b4230 con 0x7f8d4415b990 2024-04-17T10:49:18.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.045+0000 7f8d4a230640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8d440cd6c0 0x7f8d4406abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:18.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.045+0000 7f8d4a230640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8d440cd6c0 0x7f8d4406abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35470/0 (socket says 172.21.15.83:35470) 2024-04-17T10:49:18.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.045+0000 7f8d4a230640 1 -- 172.21.15.83:0/3098954007 learned_addr learned my addr 172.21.15.83:0/3098954007 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:18.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.045+0000 7f8d4922e640 1 --2- 172.21.15.83:0/3098954007 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d4415b990 0x7f8d440cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:18.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.045+0000 7f8d49a2f640 1 --2- 172.21.15.83:0/3098954007 >> v2:172.21.15.167:3300/0 conn(0x7f8d440aebd0 0x7f8d440ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:18.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.045+0000 7f8d4a230640 1 -- 172.21.15.83:0/3098954007 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d4415b990 msgr2=0x7f8d440cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:18.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.045+0000 7f8d4a230640 1 --2- 172.21.15.83:0/3098954007 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d4415b990 0x7f8d440cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:18.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.045+0000 7f8d4a230640 1 -- 172.21.15.83:0/3098954007 >> v2:172.21.15.167:3300/0 conn(0x7f8d440aebd0 msgr2=0x7f8d440ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:18.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.045+0000 7f8d4a230640 1 --2- 172.21.15.83:0/3098954007 >> v2:172.21.15.167:3300/0 conn(0x7f8d440aebd0 0x7f8d440ced50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:18.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.045+0000 7f8d4a230640 1 -- 172.21.15.83:0/3098954007 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d440b4510 con 0x7f8d440cd6c0 2024-04-17T10:49:18.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.049+0000 7f8d4a230640 1 --2- 172.21.15.83:0/3098954007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d440cd6c0 0x7f8d4406abe0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c009e50 tx=0x7f8d3c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce38eadabcb5c1e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:18.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.049+0000 7f8d48a2d640 1 -- 172.21.15.83:0/3098954007 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c0a5560 con 0x7f8d440cd6c0 2024-04-17T10:49:18.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.049+0000 7f8d48a2d640 1 -- 172.21.15.83:0/3098954007 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3c0a5700 con 0x7f8d440cd6c0 2024-04-17T10:49:18.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.049+0000 7f8d48a2d640 1 -- 172.21.15.83:0/3098954007 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c0a5ad0 con 0x7f8d440cd6c0 2024-04-17T10:49:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.049+0000 7f8d4bc93640 1 -- 172.21.15.83:0/3098954007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d440cd6c0 msgr2=0x7f8d4406abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.049+0000 7f8d4bc93640 1 --2- 172.21.15.83:0/3098954007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d440cd6c0 0x7f8d4406abe0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c009e50 tx=0x7f8d3c01c9f0 comp rx=0 tx=0).stop 2024-04-17T10:49:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.049+0000 7f8d4bc93640 1 -- 172.21.15.83:0/3098954007 shutdown_connections 2024-04-17T10:49:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.049+0000 7f8d4bc93640 1 --2- 172.21.15.83:0/3098954007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d440cd6c0 0x7f8d4406abe0 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.049+0000 7f8d4bc93640 1 --2- 172.21.15.83:0/3098954007 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d4415b990 0x7f8d440cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.049+0000 7f8d4bc93640 1 --2- 172.21.15.83:0/3098954007 >> v2:172.21.15.167:3300/0 conn(0x7f8d440aebd0 0x7f8d440ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.049+0000 7f8d4bc93640 1 -- 172.21.15.83:0/3098954007 >> 172.21.15.83:0/3098954007 conn(0x7f8d440cc050 msgr2=0x7f8d440cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.049+0000 7f8d4bc93640 1 -- 172.21.15.83:0/3098954007 shutdown_connections 2024-04-17T10:49:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.049+0000 7f8d4bc93640 1 -- 172.21.15.83:0/3098954007 wait complete. 2024-04-17T10:49:18.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.049+0000 7f8d4bc93640 1 Processor -- start 2024-04-17T10:49:18.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.053+0000 7f8d4bc93640 1 -- start start 2024-04-17T10:49:18.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.053+0000 7f8d4bc93640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d440aebd0 0x7f8d44148c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:18.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.053+0000 7f8d4bc93640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d4415b990 0x7f8d440adca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:18.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.053+0000 7f8d4bc93640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d44150540 0x7f8d441670d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:18.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.053+0000 7f8d4bc93640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d44167c50 con 0x7f8d4415b990 2024-04-17T10:49:18.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.053+0000 7f8d4bc93640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d44167ad0 con 0x7f8d440aebd0 2024-04-17T10:49:18.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.053+0000 7f8d4bc93640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d44167dd0 con 0x7f8d44150540 2024-04-17T10:49:18.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.053+0000 7f8d4922e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d4415b990 0x7f8d440adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:18.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.053+0000 7f8d4922e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d4415b990 0x7f8d440adca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35484/0 (socket says 172.21.15.83:35484) 2024-04-17T10:49:18.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.053+0000 7f8d4922e640 1 -- 172.21.15.83:0/2895559040 learned_addr learned my addr 172.21.15.83:0/2895559040 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:18.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.053+0000 7f8d49a2f640 1 --2- 172.21.15.83:0/2895559040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d440aebd0 0x7f8d44148c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:18.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.053+0000 7f8d4a230640 1 --2- 172.21.15.83:0/2895559040 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d44150540 0x7f8d441670d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:18.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.053+0000 7f8d4922e640 1 -- 172.21.15.83:0/2895559040 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d44150540 msgr2=0x7f8d441670d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:18.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.053+0000 7f8d4922e640 1 --2- 172.21.15.83:0/2895559040 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d44150540 0x7f8d441670d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:18.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.053+0000 7f8d4922e640 1 -- 172.21.15.83:0/2895559040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d440aebd0 msgr2=0x7f8d44148c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:18.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.053+0000 7f8d4922e640 1 --2- 172.21.15.83:0/2895559040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d440aebd0 0x7f8d44148c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:18.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.053+0000 7f8d4922e640 1 -- 172.21.15.83:0/2895559040 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d44150a40 con 0x7f8d4415b990 2024-04-17T10:49:18.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.053+0000 7f8d4922e640 1 --2- 172.21.15.83:0/2895559040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d4415b990 0x7f8d440adca0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400c820 tx=0x7f8d3400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:18.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.053+0000 7f8d3a7fc640 1 -- 172.21.15.83:0/2895559040 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d34015070 con 0x7f8d4415b990 2024-04-17T10:49:18.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.053+0000 7f8d3a7fc640 1 -- 172.21.15.83:0/2895559040 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d34007a30 con 0x7f8d4415b990 2024-04-17T10:49:18.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.053+0000 7f8d4bc93640 1 -- 172.21.15.83:0/2895559040 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d44150d20 con 0x7f8d4415b990 2024-04-17T10:49:18.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.053+0000 7f8d3a7fc640 1 -- 172.21.15.83:0/2895559040 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d34007e20 con 0x7f8d4415b990 2024-04-17T10:49:18.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.053+0000 7f8d4bc93640 1 -- 172.21.15.83:0/2895559040 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d440ae1e0 con 0x7f8d4415b990 2024-04-17T10:49:18.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.053+0000 7f8d4bc93640 1 -- 172.21.15.83:0/2895559040 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d44075000 con 0x7f8d4415b990 2024-04-17T10:49:18.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.053+0000 7f8d3a7fc640 1 -- 172.21.15.83:0/2895559040 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8d34018070 con 0x7f8d4415b990 2024-04-17T10:49:18.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.053+0000 7f8d3a7fc640 1 --2- 172.21.15.83:0/2895559040 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8d28066f60 0x7f8d28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:18.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.053+0000 7f8d3a7fc640 1 -- 172.21.15.83:0/2895559040 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8d34084b20 con 0x7f8d4415b990 2024-04-17T10:49:18.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.053+0000 7f8d49a2f640 1 --2- 172.21.15.83:0/2895559040 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8d28066f60 0x7f8d28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:18.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.053+0000 7f8d49a2f640 1 --2- 172.21.15.83:0/2895559040 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8d28066f60 0x7f8d28069420 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f8d40006fd0 tx=0x7f8d40008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:18.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.057+0000 7f8d3a7fc640 1 -- 172.21.15.83:0/2895559040 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8d34055ef0 con 0x7f8d4415b990 2024-04-17T10:49:18.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.205+0000 7f8d4bc93640 1 -- 172.21.15.83:0/2895559040 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8d440cd430 con 0x7f8d4415b990 2024-04-17T10:49:18.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.589+0000 7f8d3a7fc640 1 -- 172.21.15.83:0/2895559040 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v124) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8d340557f0 con 0x7f8d4415b990 2024-04-17T10:49:18.587 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2024-04-17T10:49:18.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.593+0000 7f8d4bc93640 1 -- 172.21.15.83:0/2895559040 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8d28066f60 msgr2=0x7f8d28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:18.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.593+0000 7f8d4bc93640 1 --2- 172.21.15.83:0/2895559040 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8d28066f60 0x7f8d28069420 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f8d40006fd0 tx=0x7f8d40008040 comp rx=0 tx=0).stop 2024-04-17T10:49:18.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.593+0000 7f8d4bc93640 1 -- 172.21.15.83:0/2895559040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d4415b990 msgr2=0x7f8d440adca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:18.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.593+0000 7f8d4bc93640 1 --2- 172.21.15.83:0/2895559040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d4415b990 0x7f8d440adca0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400c820 tx=0x7f8d3400cce0 comp rx=0 tx=0).stop 2024-04-17T10:49:18.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.593+0000 7f8d4bc93640 1 -- 172.21.15.83:0/2895559040 shutdown_connections 2024-04-17T10:49:18.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.593+0000 7f8d4bc93640 1 --2- 172.21.15.83:0/2895559040 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8d28066f60 0x7f8d28069420 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:18.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.593+0000 7f8d4bc93640 1 --2- 172.21.15.83:0/2895559040 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d44150540 0x7f8d441670d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:18.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.593+0000 7f8d4bc93640 1 --2- 172.21.15.83:0/2895559040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d4415b990 0x7f8d440adca0 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:18.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.593+0000 7f8d4bc93640 1 --2- 172.21.15.83:0/2895559040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d440aebd0 0x7f8d44148c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:18.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.593+0000 7f8d4bc93640 1 -- 172.21.15.83:0/2895559040 >> 172.21.15.83:0/2895559040 conn(0x7f8d440cc050 msgr2=0x7f8d440b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:18.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.593+0000 7f8d4bc93640 1 -- 172.21.15.83:0/2895559040 shutdown_connections 2024-04-17T10:49:18.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.593+0000 7f8d4bc93640 1 -- 172.21.15.83:0/2895559040 wait complete. 2024-04-17T10:49:18.615 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-17T10:49:18.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.745+0000 7f42aa923640 1 Processor -- start 2024-04-17T10:49:18.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.745+0000 7f42aa923640 1 -- start start 2024-04-17T10:49:18.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.749+0000 7f42aa923640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42a40d26c0 0x7f42a40d17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:18.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.749+0000 7f42aa923640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a40ce930 con 0x7f42a40d0970 2024-04-17T10:49:18.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.749+0000 7f42aa923640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a40d4890 con 0x7f42a40ce4e0 2024-04-17T10:49:18.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.749+0000 7f42aa923640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a40cf7d0 con 0x7f42a40d26c0 2024-04-17T10:49:18.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.749+0000 7f42a3fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f42a40d0970 0x7f42a40d1fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41404/0 (socket says 172.21.15.83:41404) 2024-04-17T10:49:18.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.749+0000 7f42a3fff640 1 -- 172.21.15.83:0/3511295352 learned_addr learned my addr 172.21.15.83:0/3511295352 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:18.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.749+0000 7f42a2ffd640 1 -- 172.21.15.83:0/3511295352 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 75020722 0 0) 0x7f42a40ce930 con 0x7f42a40d0970 2024-04-17T10:49:18.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.749+0000 7f42a37fe640 1 --2- 172.21.15.83:0/3511295352 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42a40d26c0 0x7f42a40d17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:18.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.749+0000 7f42a2ffd640 1 -- 172.21.15.83:0/3511295352 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f428c003530 con 0x7f42a40d0970 2024-04-17T10:49:18.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.749+0000 7f42a2ffd640 1 -- 172.21.15.83:0/3511295352 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2023428198 0 0) 0x7f428c003530 con 0x7f42a40d0970 2024-04-17T10:49:18.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.749+0000 7f42a2ffd640 1 -- 172.21.15.83:0/3511295352 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42a40d26c0 msgr2=0x7f42a40d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:18.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.749+0000 7f42a2ffd640 1 --2- 172.21.15.83:0/3511295352 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42a40d26c0 0x7f42a40d17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:18.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.749+0000 7f42a2ffd640 1 -- 172.21.15.83:0/3511295352 >> v1:172.21.15.167:6789/0 conn(0x7f42a40ce4e0 legacy=0x7f42a415d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:18.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.749+0000 7f42a2ffd640 1 -- 172.21.15.83:0/3511295352 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a40cebc0 con 0x7f42a40d0970 2024-04-17T10:49:18.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.749+0000 7f42a2ffd640 1 -- 172.21.15.83:0/3511295352 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f4294001a00 con 0x7f42a40d0970 2024-04-17T10:49:18.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.749+0000 7f42a2ffd640 1 -- 172.21.15.83:0/3511295352 >> v1:172.21.15.83:6789/0 conn(0x7f42a40d0970 legacy=0x7f42a40d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:18.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.749+0000 7f42a2ffd640 1 --2- 172.21.15.83:0/3511295352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f428c003a60 0x7f428c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:18.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.749+0000 7f42a2ffd640 1 --2- 172.21.15.83:0/3511295352 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42a40d26c0 0x7f428c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:18.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.749+0000 7f42a2ffd640 1 --2- 172.21.15.83:0/3511295352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f428c004d50 0x7f428c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:18.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.749+0000 7f42a2ffd640 1 -- 172.21.15.83:0/3511295352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a40cf7d0 con 0x7f428c004d50 2024-04-17T10:49:18.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.749+0000 7f42a2ffd640 1 -- 172.21.15.83:0/3511295352 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428c00bae0 con 0x7f428c003a60 2024-04-17T10:49:18.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.749+0000 7f42a2ffd640 1 -- 172.21.15.83:0/3511295352 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f428c00bce0 con 0x7f42a40d26c0 2024-04-17T10:49:18.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.749+0000 7f42a8ec0640 1 --2- 172.21.15.83:0/3511295352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f428c004d50 0x7f428c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:18.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.749+0000 7f42a3fff640 1 --2- 172.21.15.83:0/3511295352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f428c003a60 0x7f428c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:18.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.749+0000 7f42a37fe640 1 --2- 172.21.15.83:0/3511295352 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42a40d26c0 0x7f428c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:18.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.749+0000 7f42a8ec0640 1 -- 172.21.15.83:0/3511295352 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42a40d26c0 msgr2=0x7f428c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:18.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.749+0000 7f42a8ec0640 1 --2- 172.21.15.83:0/3511295352 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42a40d26c0 0x7f428c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:18.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.749+0000 7f42a8ec0640 1 -- 172.21.15.83:0/3511295352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f428c003a60 msgr2=0x7f428c003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:18.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.749+0000 7f42a8ec0640 1 --2- 172.21.15.83:0/3511295352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f428c003a60 0x7f428c003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:18.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.749+0000 7f42a8ec0640 1 -- 172.21.15.83:0/3511295352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f428c003530 con 0x7f428c004d50 2024-04-17T10:49:18.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.749+0000 7f42a3fff640 1 --2- 172.21.15.83:0/3511295352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f428c003a60 0x7f428c003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:49:18.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.753+0000 7f42a8ec0640 1 --2- 172.21.15.83:0/3511295352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f428c004d50 0x7f428c0051d0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f429c001750 tx=0x7f429c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfbbbddd1e705e74 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:18.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.753+0000 7f42a2ffd640 1 -- 172.21.15.83:0/3511295352 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c0013a0 con 0x7f428c004d50 2024-04-17T10:49:18.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.753+0000 7f42a2ffd640 1 -- 172.21.15.83:0/3511295352 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429c0052f0 con 0x7f428c004d50 2024-04-17T10:49:18.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.753+0000 7f42a2ffd640 1 -- 172.21.15.83:0/3511295352 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c003840 con 0x7f428c004d50 2024-04-17T10:49:18.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.753+0000 7f42aa923640 1 -- 172.21.15.83:0/3511295352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f428c004d50 msgr2=0x7f428c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:18.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.753+0000 7f42aa923640 1 --2- 172.21.15.83:0/3511295352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f428c004d50 0x7f428c0051d0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f429c001750 tx=0x7f429c01c3e0 comp rx=0 tx=0).stop 2024-04-17T10:49:18.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.753+0000 7f42a8ec0640 1 -- 172.21.15.83:0/3511295352 reap_dead start 2024-04-17T10:49:18.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.753+0000 7f42aa923640 1 -- 172.21.15.83:0/3511295352 shutdown_connections 2024-04-17T10:49:18.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.753+0000 7f42aa923640 1 -- 172.21.15.83:0/3511295352 >> 172.21.15.83:0/3511295352 conn(0x7f42a409cd50 msgr2=0x7f42a40d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:18.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.753+0000 7f42aa923640 1 -- 172.21.15.83:0/3511295352 shutdown_connections 2024-04-17T10:49:18.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.753+0000 7f42aa923640 1 -- 172.21.15.83:0/3511295352 wait complete. 2024-04-17T10:49:18.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.753+0000 7f42aa923640 1 Processor -- start 2024-04-17T10:49:18.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.753+0000 7f42aa923640 1 -- start start 2024-04-17T10:49:18.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.753+0000 7f42aa923640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42a4151d10 0x7f42a40b4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:18.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.753+0000 7f42aa923640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42a41520f0 0x7f42a40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:18.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.753+0000 7f42aa923640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42a40cc410 0x7f42a40cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:18.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.753+0000 7f42aa923640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a40cf460 con 0x7f42a4151d10 2024-04-17T10:49:18.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.757+0000 7f42aa923640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a40cf180 con 0x7f42a41520f0 2024-04-17T10:49:18.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.757+0000 7f42aa923640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a40d0d50 con 0x7f42a40cc410 2024-04-17T10:49:18.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.757+0000 7f42a3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42a4151d10 0x7f42a40b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:18.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.757+0000 7f42a3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42a4151d10 0x7f42a40b4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35506/0 (socket says 172.21.15.83:35506) 2024-04-17T10:49:18.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.757+0000 7f42a3fff640 1 -- 172.21.15.83:0/1508862273 learned_addr learned my addr 172.21.15.83:0/1508862273 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:18.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.757+0000 7f42a37fe640 1 --2- 172.21.15.83:0/1508862273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42a41520f0 0x7f42a40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:18.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.757+0000 7f42a8ec0640 1 --2- 172.21.15.83:0/1508862273 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42a40cc410 0x7f42a40cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:18.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.757+0000 7f42a3fff640 1 -- 172.21.15.83:0/1508862273 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42a40cc410 msgr2=0x7f42a40cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:18.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.757+0000 7f42a3fff640 1 --2- 172.21.15.83:0/1508862273 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42a40cc410 0x7f42a40cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:18.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.757+0000 7f42a3fff640 1 -- 172.21.15.83:0/1508862273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42a41520f0 msgr2=0x7f42a40b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:18.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.757+0000 7f42a3fff640 1 --2- 172.21.15.83:0/1508862273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42a41520f0 0x7f42a40b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:18.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.757+0000 7f42a3fff640 1 -- 172.21.15.83:0/1508862273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a40cd190 con 0x7f42a4151d10 2024-04-17T10:49:18.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.757+0000 7f42a37fe640 1 --2- 172.21.15.83:0/1508862273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42a41520f0 0x7f42a40b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:18.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.757+0000 7f42a8ec0640 1 --2- 172.21.15.83:0/1508862273 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42a40cc410 0x7f42a40cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:18.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.757+0000 7f42a3fff640 1 --2- 172.21.15.83:0/1508862273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42a4151d10 0x7f42a40b4920 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f429400e410 tx=0x7f429400e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:18.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.757+0000 7f42a0ff9640 1 -- 172.21.15.83:0/1508862273 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4294012590 con 0x7f42a4151d10 2024-04-17T10:49:18.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.757+0000 7f42a0ff9640 1 -- 172.21.15.83:0/1508862273 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4294018070 con 0x7f42a4151d10 2024-04-17T10:49:18.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.757+0000 7f42aa923640 1 -- 172.21.15.83:0/1508862273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a40cab40 con 0x7f42a4151d10 2024-04-17T10:49:18.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.757+0000 7f42a0ff9640 1 -- 172.21.15.83:0/1508862273 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4294012970 con 0x7f42a4151d10 2024-04-17T10:49:18.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.757+0000 7f42aa923640 1 -- 172.21.15.83:0/1508862273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a40cb100 con 0x7f42a4151d10 2024-04-17T10:49:18.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.757+0000 7f42aa923640 1 -- 172.21.15.83:0/1508862273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a40681a0 con 0x7f42a4151d10 2024-04-17T10:49:18.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.757+0000 7f42a0ff9640 1 -- 172.21.15.83:0/1508862273 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4294012b10 con 0x7f42a4151d10 2024-04-17T10:49:18.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.757+0000 7f42a0ff9640 1 --2- 172.21.15.83:0/1508862273 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4290066f90 0x7f4290069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:18.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.757+0000 7f42a37fe640 1 --2- 172.21.15.83:0/1508862273 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4290066f90 0x7f4290069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:18.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.757+0000 7f42a0ff9640 1 -- 172.21.15.83:0/1508862273 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f4294085a20 con 0x7f42a4151d10 2024-04-17T10:49:18.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.757+0000 7f42a37fe640 1 --2- 172.21.15.83:0/1508862273 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4290066f90 0x7f4290069450 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f42a40c7450 tx=0x7f4298008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:18.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.761+0000 7f42a0ff9640 1 -- 172.21.15.83:0/1508862273 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f42940112e0 con 0x7f42a4151d10 2024-04-17T10:49:18.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:18.909+0000 7f42aa923640 1 -- 172.21.15.83:0/1508862273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f42a4074f90 con 0x7f42a4151d10 2024-04-17T10:49:19.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.593+0000 7f42a0ff9640 1 -- 172.21.15.83:0/1508862273 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v125) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4294056b80 con 0x7f42a4151d10 2024-04-17T10:49:19.591 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2024-04-17T10:49:19.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.597+0000 7f42aa923640 1 -- 172.21.15.83:0/1508862273 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4290066f90 msgr2=0x7f4290069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:19.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.597+0000 7f42aa923640 1 --2- 172.21.15.83:0/1508862273 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4290066f90 0x7f4290069450 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f42a40c7450 tx=0x7f4298008040 comp rx=0 tx=0).stop 2024-04-17T10:49:19.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.597+0000 7f42aa923640 1 -- 172.21.15.83:0/1508862273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42a4151d10 msgr2=0x7f42a40b4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:19.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.597+0000 7f42aa923640 1 --2- 172.21.15.83:0/1508862273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42a4151d10 0x7f42a40b4920 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f429400e410 tx=0x7f429400e8d0 comp rx=0 tx=0).stop 2024-04-17T10:49:19.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.597+0000 7f42aa923640 1 -- 172.21.15.83:0/1508862273 shutdown_connections 2024-04-17T10:49:19.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.597+0000 7f42aa923640 1 --2- 172.21.15.83:0/1508862273 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4290066f90 0x7f4290069450 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:19.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.597+0000 7f42aa923640 1 --2- 172.21.15.83:0/1508862273 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42a40cc410 0x7f42a40cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:19.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.597+0000 7f42aa923640 1 --2- 172.21.15.83:0/1508862273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42a41520f0 0x7f42a40b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:19.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.597+0000 7f42aa923640 1 --2- 172.21.15.83:0/1508862273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42a4151d10 0x7f42a40b4920 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:19.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.597+0000 7f42aa923640 1 -- 172.21.15.83:0/1508862273 >> 172.21.15.83:0/1508862273 conn(0x7f42a409cd50 msgr2=0x7f42a40b40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:19.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.597+0000 7f42aa923640 1 -- 172.21.15.83:0/1508862273 shutdown_connections 2024-04-17T10:49:19.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.597+0000 7f42aa923640 1 -- 172.21.15.83:0/1508862273 wait complete. 2024-04-17T10:49:19.626 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-17T10:49:19.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.749+0000 7fc4fcc3d640 1 Processor -- start 2024-04-17T10:49:19.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.749+0000 7fc4fcc3d640 1 -- start start 2024-04-17T10:49:19.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.753+0000 7fc4fcc3d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc4f806b520 0x7fc4f80b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:19.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.753+0000 7fc4fcc3d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc4f80b4040 0x7fc4f806cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:19.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.753+0000 7fc4fcc3d640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4f815bd70 con 0x7fc4f8154de0 2024-04-17T10:49:19.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.753+0000 7fc4fcc3d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4f80b4870 con 0x7fc4f806b520 2024-04-17T10:49:19.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.753+0000 7fc4fcc3d640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4f815bf60 con 0x7fc4f80b4040 2024-04-17T10:49:19.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.753+0000 7fc4f5d9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc4f806b520 0x7fc4f80b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:19.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.753+0000 7fc4f5d9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc4f806b520 0x7fc4f80b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35518/0 (socket says 172.21.15.83:35518) 2024-04-17T10:49:19.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.753+0000 7fc4f5d9b640 1 -- 172.21.15.83:0/2200427558 learned_addr learned my addr 172.21.15.83:0/2200427558 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:19.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.753+0000 7fc4f5d9b640 1 -- 172.21.15.83:0/2200427558 >> v2:172.21.15.167:3300/0 conn(0x7fc4f80b4040 msgr2=0x7fc4f806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:19.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.753+0000 7fc4f5d9b640 1 --2- 172.21.15.83:0/2200427558 >> v2:172.21.15.167:3300/0 conn(0x7fc4f80b4040 0x7fc4f806cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:19.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.753+0000 7fc4f5d9b640 1 -- 172.21.15.83:0/2200427558 >> v1:172.21.15.167:6789/0 conn(0x7fc4f8154de0 legacy=0x7fc4f806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:19.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.753+0000 7fc4f5d9b640 1 -- 172.21.15.83:0/2200427558 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f815afb0 con 0x7fc4f806b520 2024-04-17T10:49:19.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.753+0000 7fc4f5d9b640 1 --2- 172.21.15.83:0/2200427558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4f806b520 0x7fc4f80b3b00 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fc4f806e8d0 tx=0x7fc4ec01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7397682be4145f88 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:19.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.753+0000 7fc4f559a640 1 -- 172.21.15.83:0/2200427558 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec0b1070 con 0x7fc4f806b520 2024-04-17T10:49:19.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.753+0000 7fc4f559a640 1 -- 172.21.15.83:0/2200427558 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4ec0ac630 con 0x7fc4f806b520 2024-04-17T10:49:19.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.753+0000 7fc4f559a640 1 -- 172.21.15.83:0/2200427558 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec0a8050 con 0x7fc4f806b520 2024-04-17T10:49:19.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4fcc3d640 1 -- 172.21.15.83:0/2200427558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4f806b520 msgr2=0x7fc4f80b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:19.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4fcc3d640 1 --2- 172.21.15.83:0/2200427558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4f806b520 0x7fc4f80b3b00 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fc4f806e8d0 tx=0x7fc4ec01c4e0 comp rx=0 tx=0).stop 2024-04-17T10:49:19.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4fcc3d640 1 -- 172.21.15.83:0/2200427558 shutdown_connections 2024-04-17T10:49:19.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4fcc3d640 1 --2- 172.21.15.83:0/2200427558 >> v2:172.21.15.167:3300/0 conn(0x7fc4f80b4040 0x7fc4f806cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:19.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4fcc3d640 1 --2- 172.21.15.83:0/2200427558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4f806b520 0x7fc4f80b3b00 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:19.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4fcc3d640 1 -- 172.21.15.83:0/2200427558 >> 172.21.15.83:0/2200427558 conn(0x7fc4f80b15b0 msgr2=0x7fc4f80b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:19.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4fcc3d640 1 -- 172.21.15.83:0/2200427558 shutdown_connections 2024-04-17T10:49:19.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4fcc3d640 1 -- 172.21.15.83:0/2200427558 wait complete. 2024-04-17T10:49:19.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4fcc3d640 1 Processor -- start 2024-04-17T10:49:19.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4fcc3d640 1 -- start start 2024-04-17T10:49:19.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4fcc3d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4f806b520 0x7fc4f814e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:19.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4fcc3d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc4f80b4040 0x7fc4f8148eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:19.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4fcc3d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4f8154de0 0x7fc4f8156130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:19.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4fcc3d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f80b4480 con 0x7fc4f8154de0 2024-04-17T10:49:19.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4fcc3d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f81551c0 con 0x7fc4f806b520 2024-04-17T10:49:19.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4fcc3d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f80b4600 con 0x7fc4f80b4040 2024-04-17T10:49:19.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4f6d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4f8154de0 0x7fc4f8156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:19.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4f6d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4f8154de0 0x7fc4f8156130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35526/0 (socket says 172.21.15.83:35526) 2024-04-17T10:49:19.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4f6d9d640 1 -- 172.21.15.83:0/2713962763 learned_addr learned my addr 172.21.15.83:0/2713962763 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:19.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4f659c640 1 --2- 172.21.15.83:0/2713962763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4f806b520 0x7fc4f814e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:19.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4f5d9b640 1 --2- 172.21.15.83:0/2713962763 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc4f80b4040 0x7fc4f8148eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:19.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4f6d9d640 1 -- 172.21.15.83:0/2713962763 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc4f80b4040 msgr2=0x7fc4f8148eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:19.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4f6d9d640 1 --2- 172.21.15.83:0/2713962763 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc4f80b4040 0x7fc4f8148eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:19.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4f6d9d640 1 -- 172.21.15.83:0/2713962763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4f806b520 msgr2=0x7fc4f814e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:19.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4f6d9d640 1 --2- 172.21.15.83:0/2713962763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4f806b520 0x7fc4f814e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:19.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4f6d9d640 1 -- 172.21.15.83:0/2713962763 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f8156a10 con 0x7fc4f8154de0 2024-04-17T10:49:19.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4f659c640 1 --2- 172.21.15.83:0/2713962763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4f806b520 0x7fc4f814e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:19.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4f6d9d640 1 --2- 172.21.15.83:0/2713962763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4f8154de0 0x7fc4f8156130 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fc4f000b9a0 tx=0x7fc4f000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:19.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4e6ffd640 1 -- 172.21.15.83:0/2713962763 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0019070 con 0x7fc4f8154de0 2024-04-17T10:49:19.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4e6ffd640 1 -- 172.21.15.83:0/2713962763 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f000cbc0 con 0x7fc4f8154de0 2024-04-17T10:49:19.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4fcc3d640 1 -- 172.21.15.83:0/2713962763 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4f8156cf0 con 0x7fc4f8154de0 2024-04-17T10:49:19.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4e6ffd640 1 -- 172.21.15.83:0/2713962763 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0014440 con 0x7fc4f8154de0 2024-04-17T10:49:19.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.757+0000 7fc4fcc3d640 1 -- 172.21.15.83:0/2713962763 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4f81572b0 con 0x7fc4f8154de0 2024-04-17T10:49:19.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.761+0000 7fc4fcc3d640 1 -- 172.21.15.83:0/2713962763 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4f8074f90 con 0x7fc4f8154de0 2024-04-17T10:49:19.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.761+0000 7fc4e6ffd640 1 -- 172.21.15.83:0/2713962763 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc4f001c070 con 0x7fc4f8154de0 2024-04-17T10:49:19.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.761+0000 7fc4e6ffd640 1 --2- 172.21.15.83:0/2713962763 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc4d8066f60 0x7fc4d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:19.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.761+0000 7fc4e6ffd640 1 -- 172.21.15.83:0/2713962763 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fc4f00850e0 con 0x7fc4f8154de0 2024-04-17T10:49:19.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.761+0000 7fc4f659c640 1 --2- 172.21.15.83:0/2713962763 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc4d8066f60 0x7fc4d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:19.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.761+0000 7fc4f659c640 1 --2- 172.21.15.83:0/2713962763 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc4d8066f60 0x7fc4d8069420 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc4f81548e0 tx=0x7fc4e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:19.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.765+0000 7fc4e6ffd640 1 -- 172.21.15.83:0/2713962763 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc4f0023020 con 0x7fc4f8154de0 2024-04-17T10:49:19.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:19.953+0000 7fc4fcc3d640 1 -- 172.21.15.83:0/2713962763 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc4f80b3280 con 0x7fc4f8154de0 2024-04-17T10:49:20.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.637+0000 7fc4e6ffd640 1 -- 172.21.15.83:0/2713962763 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v74) v1 ==== 174+0+0 (secure 0 0 0) 0x7fc4f0056230 con 0x7fc4f8154de0 2024-04-17T10:49:20.638 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 18 and data pool 19 2024-04-17T10:49:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.649+0000 7fc4fcc3d640 1 -- 172.21.15.83:0/2713962763 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc4d8066f60 msgr2=0x7fc4d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:20.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.649+0000 7fc4fcc3d640 1 --2- 172.21.15.83:0/2713962763 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc4d8066f60 0x7fc4d8069420 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc4f81548e0 tx=0x7fc4e8008040 comp rx=0 tx=0).stop 2024-04-17T10:49:20.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.649+0000 7fc4fcc3d640 1 -- 172.21.15.83:0/2713962763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4f8154de0 msgr2=0x7fc4f8156130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:20.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.649+0000 7fc4fcc3d640 1 --2- 172.21.15.83:0/2713962763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4f8154de0 0x7fc4f8156130 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fc4f000b9a0 tx=0x7fc4f000be60 comp rx=0 tx=0).stop 2024-04-17T10:49:20.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.649+0000 7fc4fcc3d640 1 -- 172.21.15.83:0/2713962763 shutdown_connections 2024-04-17T10:49:20.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.649+0000 7fc4fcc3d640 1 --2- 172.21.15.83:0/2713962763 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc4d8066f60 0x7fc4d8069420 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:20.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.649+0000 7fc4fcc3d640 1 --2- 172.21.15.83:0/2713962763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4f8154de0 0x7fc4f8156130 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:20.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.649+0000 7fc4fcc3d640 1 --2- 172.21.15.83:0/2713962763 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc4f80b4040 0x7fc4f8148eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:20.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.649+0000 7fc4fcc3d640 1 --2- 172.21.15.83:0/2713962763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4f806b520 0x7fc4f814e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:20.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.649+0000 7fc4fcc3d640 1 -- 172.21.15.83:0/2713962763 >> 172.21.15.83:0/2713962763 conn(0x7fc4f80b15b0 msgr2=0x7fc4f80b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:20.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.649+0000 7fc4fcc3d640 1 -- 172.21.15.83:0/2713962763 shutdown_connections 2024-04-17T10:49:20.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.653+0000 7fc4fcc3d640 1 -- 172.21.15.83:0/2713962763 wait complete. 2024-04-17T10:49:20.673 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:49:20.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.789+0000 7fdfe244f640 1 Processor -- start 2024-04-17T10:49:20.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.789+0000 7fdfe244f640 1 -- start start 2024-04-17T10:49:20.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.789+0000 7fdfe244f640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdfdc15df40 0x7fdfdc06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:20.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.789+0000 7fdfe244f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdfdc06afa0 0x7fdfdc06e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:20.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.789+0000 7fdfe244f640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfdc0ad1a0 con 0x7fdfdc0b5c80 2024-04-17T10:49:20.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.789+0000 7fdfe244f640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfdc06ee40 con 0x7fdfdc06afa0 2024-04-17T10:49:20.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.789+0000 7fdfe244f640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfdc0ad480 con 0x7fdfdc15df40 2024-04-17T10:49:20.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.789+0000 7fdfdb7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdfdc06afa0 0x7fdfdc06e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:20.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.789+0000 7fdfdb7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdfdc06afa0 0x7fdfdc06e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35530/0 (socket says 172.21.15.83:35530) 2024-04-17T10:49:20.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.789+0000 7fdfdb7fe640 1 -- 172.21.15.83:0/743279581 learned_addr learned my addr 172.21.15.83:0/743279581 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:20.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.789+0000 7fdfdbfff640 1 --2- 172.21.15.83:0/743279581 >> v2:172.21.15.167:3300/0 conn(0x7fdfdc15df40 0x7fdfdc06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:20.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.789+0000 7fdfdb7fe640 1 -- 172.21.15.83:0/743279581 >> v2:172.21.15.167:3300/0 conn(0x7fdfdc15df40 msgr2=0x7fdfdc06e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:20.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.789+0000 7fdfdb7fe640 1 --2- 172.21.15.83:0/743279581 >> v2:172.21.15.167:3300/0 conn(0x7fdfdc15df40 0x7fdfdc06e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:20.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.789+0000 7fdfdb7fe640 1 -- 172.21.15.83:0/743279581 >> v1:172.21.15.167:6789/0 conn(0x7fdfdc0b5c80 legacy=0x7fdfdc06b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:20.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.789+0000 7fdfdb7fe640 1 -- 172.21.15.83:0/743279581 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfdc0ad760 con 0x7fdfdc06afa0 2024-04-17T10:49:20.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.793+0000 7fdfdb7fe640 1 --2- 172.21.15.83:0/743279581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfdc06afa0 0x7fdfdc06e900 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc0746f0 tx=0x7fdfd001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e2a0074d6828d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:20.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.793+0000 7fdfdaffd640 1 -- 172.21.15.83:0/743279581 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd00b0070 con 0x7fdfdc06afa0 2024-04-17T10:49:20.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.793+0000 7fdfdaffd640 1 -- 172.21.15.83:0/743279581 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfd00a4e50 con 0x7fdfdc06afa0 2024-04-17T10:49:20.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.793+0000 7fdfdaffd640 1 -- 172.21.15.83:0/743279581 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd00ab820 con 0x7fdfdc06afa0 2024-04-17T10:49:20.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.793+0000 7fdfe244f640 1 -- 172.21.15.83:0/743279581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfdc06afa0 msgr2=0x7fdfdc06e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:20.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.793+0000 7fdfe244f640 1 --2- 172.21.15.83:0/743279581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfdc06afa0 0x7fdfdc06e900 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc0746f0 tx=0x7fdfd001c3c0 comp rx=0 tx=0).stop 2024-04-17T10:49:20.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.793+0000 7fdfe244f640 1 -- 172.21.15.83:0/743279581 shutdown_connections 2024-04-17T10:49:20.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.793+0000 7fdfe244f640 1 --2- 172.21.15.83:0/743279581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfdc06afa0 0x7fdfdc06e900 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:20.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.793+0000 7fdfe244f640 1 --2- 172.21.15.83:0/743279581 >> v2:172.21.15.167:3300/0 conn(0x7fdfdc15df40 0x7fdfdc06e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:20.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.793+0000 7fdfe244f640 1 -- 172.21.15.83:0/743279581 >> 172.21.15.83:0/743279581 conn(0x7fdfdc0b4eb0 msgr2=0x7fdfdc0b5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:20.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.793+0000 7fdfe244f640 1 -- 172.21.15.83:0/743279581 shutdown_connections 2024-04-17T10:49:20.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.793+0000 7fdfe244f640 1 -- 172.21.15.83:0/743279581 wait complete. 2024-04-17T10:49:20.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.793+0000 7fdfe244f640 1 Processor -- start 2024-04-17T10:49:20.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.793+0000 7fdfe244f640 1 -- start start 2024-04-17T10:49:20.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.797+0000 7fdfe244f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfdc06afa0 0x7fdfdc15b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:20.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.797+0000 7fdfe244f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfdc0b5c80 0x7fdfdc14f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:20.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.797+0000 7fdfe244f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfdc15df40 0x7fdfdc14fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:20.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.797+0000 7fdfe244f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfdc16adf0 con 0x7fdfdc15df40 2024-04-17T10:49:20.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.797+0000 7fdfe244f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfdc16ac70 con 0x7fdfdc0b5c80 2024-04-17T10:49:20.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.797+0000 7fdfe244f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfdc16af70 con 0x7fdfdc06afa0 2024-04-17T10:49:20.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.797+0000 7fdfe09ec640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfdc15df40 0x7fdfdc14fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:20.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.797+0000 7fdfe09ec640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfdc15df40 0x7fdfdc14fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35538/0 (socket says 172.21.15.83:35538) 2024-04-17T10:49:20.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.797+0000 7fdfe09ec640 1 -- 172.21.15.83:0/1560020340 learned_addr learned my addr 172.21.15.83:0/1560020340 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:20.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.797+0000 7fdfe09ec640 1 -- 172.21.15.83:0/1560020340 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfdc06afa0 msgr2=0x7fdfdc15b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:20.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.797+0000 7fdfdbfff640 1 --2- 172.21.15.83:0/1560020340 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfdc06afa0 0x7fdfdc15b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:20.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.797+0000 7fdfdb7fe640 1 --2- 172.21.15.83:0/1560020340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfdc0b5c80 0x7fdfdc14f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:20.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.797+0000 7fdfe09ec640 1 --2- 172.21.15.83:0/1560020340 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfdc06afa0 0x7fdfdc15b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:20.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.797+0000 7fdfe09ec640 1 -- 172.21.15.83:0/1560020340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfdc0b5c80 msgr2=0x7fdfdc14f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:20.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.797+0000 7fdfe09ec640 1 --2- 172.21.15.83:0/1560020340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfdc0b5c80 0x7fdfdc14f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:20.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.797+0000 7fdfe09ec640 1 -- 172.21.15.83:0/1560020340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfdc13cc70 con 0x7fdfdc15df40 2024-04-17T10:49:20.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.797+0000 7fdfdb7fe640 1 --2- 172.21.15.83:0/1560020340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfdc0b5c80 0x7fdfdc14f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:20.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.797+0000 7fdfdbfff640 1 --2- 172.21.15.83:0/1560020340 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfdc06afa0 0x7fdfdc15b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:20.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.797+0000 7fdfe09ec640 1 --2- 172.21.15.83:0/1560020340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfdc15df40 0x7fdfdc14fb50 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00c3c0 tx=0x7fdfcc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:20.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.797+0000 7fdfd8ff9640 1 -- 172.21.15.83:0/1560020340 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc011790 con 0x7fdfdc15df40 2024-04-17T10:49:20.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.797+0000 7fdfd8ff9640 1 -- 172.21.15.83:0/1560020340 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfcc017070 con 0x7fdfdc15df40 2024-04-17T10:49:20.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.797+0000 7fdfe244f640 1 -- 172.21.15.83:0/1560020340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfdc13cf10 con 0x7fdfdc15df40 2024-04-17T10:49:20.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.797+0000 7fdfd8ff9640 1 -- 172.21.15.83:0/1560020340 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc011b70 con 0x7fdfdc15df40 2024-04-17T10:49:20.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.797+0000 7fdfe244f640 1 -- 172.21.15.83:0/1560020340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfdc13d4d0 con 0x7fdfdc15df40 2024-04-17T10:49:20.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.797+0000 7fdfe244f640 1 -- 172.21.15.83:0/1560020340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfdc074f90 con 0x7fdfdc15df40 2024-04-17T10:49:20.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.797+0000 7fdfd8ff9640 1 -- 172.21.15.83:0/1560020340 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdfcc011d10 con 0x7fdfdc15df40 2024-04-17T10:49:20.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.797+0000 7fdfd8ff9640 1 --2- 172.21.15.83:0/1560020340 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdfc8066f60 0x7fdfc8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:20.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.797+0000 7fdfd8ff9640 1 -- 172.21.15.83:0/1560020340 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdfcc084e50 con 0x7fdfdc15df40 2024-04-17T10:49:20.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.797+0000 7fdfdbfff640 1 --2- 172.21.15.83:0/1560020340 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdfc8066f60 0x7fdfc8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:20.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.797+0000 7fdfdbfff640 1 --2- 172.21.15.83:0/1560020340 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdfc8066f60 0x7fdfc8069420 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009f50 tx=0x7fdfd400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:20.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.801+0000 7fdfd8ff9640 1 -- 172.21.15.83:0/1560020340 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdfcc055e60 con 0x7fdfdc15df40 2024-04-17T10:49:20.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.961+0000 7fdfe244f640 1 -- 172.21.15.83:0/1560020340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdfdc068140 con 0x7fdfdc15df40 2024-04-17T10:49:20.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.965+0000 7fdfd8ff9640 1 -- 172.21.15.83:0/1560020340 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+16620 (secure 0 0 0) 0x7fdfcc055760 con 0x7fdfdc15df40 2024-04-17T10:49:20.962 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:20.962 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":126,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:49:19.961945+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:49:18.247824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-17T10:49:18.951409+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:49:20.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.965+0000 7fdfe244f640 1 -- 172.21.15.83:0/1560020340 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdfc8066f60 msgr2=0x7fdfc8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:20.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.965+0000 7fdfe244f640 1 --2- 172.21.15.83:0/1560020340 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdfc8066f60 0x7fdfc8069420 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009f50 tx=0x7fdfd400b040 comp rx=0 tx=0).stop 2024-04-17T10:49:20.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.965+0000 7fdfe244f640 1 -- 172.21.15.83:0/1560020340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfdc15df40 msgr2=0x7fdfdc14fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:20.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.965+0000 7fdfe244f640 1 --2- 172.21.15.83:0/1560020340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfdc15df40 0x7fdfdc14fb50 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00c3c0 tx=0x7fdfcc00c880 comp rx=0 tx=0).stop 2024-04-17T10:49:20.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.969+0000 7fdfe244f640 1 -- 172.21.15.83:0/1560020340 shutdown_connections 2024-04-17T10:49:20.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.969+0000 7fdfe244f640 1 --2- 172.21.15.83:0/1560020340 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdfc8066f60 0x7fdfc8069420 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:20.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.969+0000 7fdfe244f640 1 --2- 172.21.15.83:0/1560020340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfdc15df40 0x7fdfdc14fb50 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:20.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.969+0000 7fdfe244f640 1 --2- 172.21.15.83:0/1560020340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfdc0b5c80 0x7fdfdc14f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:20.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.969+0000 7fdfe244f640 1 --2- 172.21.15.83:0/1560020340 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfdc06afa0 0x7fdfdc15b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:20.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.969+0000 7fdfe244f640 1 -- 172.21.15.83:0/1560020340 >> 172.21.15.83:0/1560020340 conn(0x7fdfdc0b4eb0 msgr2=0x7fdfdc155e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:20.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.969+0000 7fdfe244f640 1 -- 172.21.15.83:0/1560020340 shutdown_connections 2024-04-17T10:49:20.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:20.969+0000 7fdfe244f640 1 -- 172.21.15.83:0/1560020340 wait complete. 2024-04-17T10:49:20.983 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:49:21.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.105+0000 7f50dc01c640 1 Processor -- start 2024-04-17T10:49:21.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.105+0000 7f50dc01c640 1 -- start start 2024-04-17T10:49:21.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.105+0000 7f50dc01c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50d40aecd0 0x7f50d40cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:21.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.105+0000 7f50dc01c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f50d40b3f80 0x7f50d40b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:21.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.105+0000 7f50dc01c640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d4072760 con 0x7f50d4166550 2024-04-17T10:49:21.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.105+0000 7f50dc01c640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50d40cf810 con 0x7f50d40b3f80 2024-04-17T10:49:21.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.105+0000 7f50dc01c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d40b4950 con 0x7f50d40aecd0 2024-04-17T10:49:21.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.105+0000 7f50d9db8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50d40aecd0 0x7f50d40cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:21.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.105+0000 7f50d95b7640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f50d4166550 0x7f50d4071be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:39134/0 (socket says 172.21.15.83:39134) 2024-04-17T10:49:21.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.105+0000 7f50d95b7640 1 -- 172.21.15.83:0/602094514 learned_addr learned my addr 172.21.15.83:0/602094514 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:21.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.105+0000 7f50da5b9640 1 --2- 172.21.15.83:0/602094514 >> v2:172.21.15.167:3300/0 conn(0x7f50d40b3f80 0x7f50d40b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:21.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.105+0000 7f50d9db8640 1 -- 172.21.15.83:0/602094514 >> v2:172.21.15.167:3300/0 conn(0x7f50d40b3f80 msgr2=0x7f50d40b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:21.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.105+0000 7f50d9db8640 1 --2- 172.21.15.83:0/602094514 >> v2:172.21.15.167:3300/0 conn(0x7f50d40b3f80 0x7f50d40b4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:21.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.105+0000 7f50d9db8640 1 -- 172.21.15.83:0/602094514 >> v1:172.21.15.167:6789/0 conn(0x7f50d4166550 legacy=0x7f50d4071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:21.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.105+0000 7f50d9db8640 1 -- 172.21.15.83:0/602094514 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d40cf200 con 0x7f50d40aecd0 2024-04-17T10:49:21.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.109+0000 7f50d9db8640 1 --2- 172.21.15.83:0/602094514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50d40aecd0 0x7f50d40cfca0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f50c40097c0 tx=0x7f50c401c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c7dc82859c9568e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:21.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.109+0000 7f50d8db6640 1 -- 172.21.15.83:0/602094514 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c40b0070 con 0x7f50d40aecd0 2024-04-17T10:49:21.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.109+0000 7f50d8db6640 1 -- 172.21.15.83:0/602094514 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c40ab600 con 0x7f50d40aecd0 2024-04-17T10:49:21.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.109+0000 7f50d8db6640 1 -- 172.21.15.83:0/602094514 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c40ab9d0 con 0x7f50d40aecd0 2024-04-17T10:49:21.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.109+0000 7f50dc01c640 1 -- 172.21.15.83:0/602094514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50d40aecd0 msgr2=0x7f50d40cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:21.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.109+0000 7f50dc01c640 1 --2- 172.21.15.83:0/602094514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50d40aecd0 0x7f50d40cfca0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f50c40097c0 tx=0x7f50c401c410 comp rx=0 tx=0).stop 2024-04-17T10:49:21.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.109+0000 7f50dc01c640 1 -- 172.21.15.83:0/602094514 shutdown_connections 2024-04-17T10:49:21.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.109+0000 7f50dc01c640 1 --2- 172.21.15.83:0/602094514 >> v2:172.21.15.167:3300/0 conn(0x7f50d40b3f80 0x7f50d40b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:21.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.109+0000 7f50dc01c640 1 --2- 172.21.15.83:0/602094514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50d40aecd0 0x7f50d40cfca0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:21.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.109+0000 7f50dc01c640 1 -- 172.21.15.83:0/602094514 >> 172.21.15.83:0/602094514 conn(0x7f50d406abe0 msgr2=0x7f50d406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:21.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.109+0000 7f50dc01c640 1 -- 172.21.15.83:0/602094514 shutdown_connections 2024-04-17T10:49:21.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.109+0000 7f50dc01c640 1 -- 172.21.15.83:0/602094514 wait complete. 2024-04-17T10:49:21.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.109+0000 7f50dc01c640 1 Processor -- start 2024-04-17T10:49:21.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.109+0000 7f50dc01c640 1 -- start start 2024-04-17T10:49:21.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.113+0000 7f50dc01c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50d40aecd0 0x7f50d4152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:21.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.113+0000 7f50dc01c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50d40b3f80 0x7f50d41fadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:21.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.113+0000 7f50dc01c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50d4166550 0x7f50d41532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:21.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.113+0000 7f50dc01c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d40b3000 con 0x7f50d4166550 2024-04-17T10:49:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.113+0000 7f50dc01c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d40b2e80 con 0x7f50d40aecd0 2024-04-17T10:49:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.113+0000 7f50dc01c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d40b3180 con 0x7f50d40b3f80 2024-04-17T10:49:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.113+0000 7f50da5b9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50d4166550 0x7f50d41532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.113+0000 7f50da5b9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50d4166550 0x7f50d41532a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35542/0 (socket says 172.21.15.83:35542) 2024-04-17T10:49:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.113+0000 7f50da5b9640 1 -- 172.21.15.83:0/3097496159 learned_addr learned my addr 172.21.15.83:0/3097496159 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.113+0000 7f50d9db8640 1 --2- 172.21.15.83:0/3097496159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50d40aecd0 0x7f50d4152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.113+0000 7f50d95b7640 1 --2- 172.21.15.83:0/3097496159 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50d40b3f80 0x7f50d41fadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.113+0000 7f50da5b9640 1 -- 172.21.15.83:0/3097496159 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50d40b3f80 msgr2=0x7f50d41fadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.113+0000 7f50da5b9640 1 --2- 172.21.15.83:0/3097496159 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50d40b3f80 0x7f50d41fadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.113+0000 7f50da5b9640 1 -- 172.21.15.83:0/3097496159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50d40aecd0 msgr2=0x7f50d4152ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.113+0000 7f50da5b9640 1 --2- 172.21.15.83:0/3097496159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50d40aecd0 0x7f50d4152ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.113+0000 7f50da5b9640 1 -- 172.21.15.83:0/3097496159 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d415e020 con 0x7f50d4166550 2024-04-17T10:49:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.113+0000 7f50da5b9640 1 --2- 172.21.15.83:0/3097496159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50d4166550 0x7f50d41532a0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00e960 tx=0x7f50cc00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.113+0000 7f50ca7fc640 1 -- 172.21.15.83:0/3097496159 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc017070 con 0x7f50d4166550 2024-04-17T10:49:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.113+0000 7f50ca7fc640 1 -- 172.21.15.83:0/3097496159 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50cc00c9a0 con 0x7f50d4166550 2024-04-17T10:49:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.113+0000 7f50ca7fc640 1 -- 172.21.15.83:0/3097496159 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc00cd70 con 0x7f50d4166550 2024-04-17T10:49:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.113+0000 7f50dc01c640 1 -- 172.21.15.83:0/3097496159 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d415e300 con 0x7f50d4166550 2024-04-17T10:49:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.113+0000 7f50dc01c640 1 -- 172.21.15.83:0/3097496159 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d4155500 con 0x7f50d4166550 2024-04-17T10:49:21.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.113+0000 7f50dc01c640 1 -- 172.21.15.83:0/3097496159 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50d4074f90 con 0x7f50d4166550 2024-04-17T10:49:21.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.113+0000 7f50ca7fc640 1 -- 172.21.15.83:0/3097496159 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f50cc01a070 con 0x7f50d4166550 2024-04-17T10:49:21.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.113+0000 7f50ca7fc640 1 --2- 172.21.15.83:0/3097496159 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f50b8066f60 0x7f50b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:21.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.113+0000 7f50ca7fc640 1 -- 172.21.15.83:0/3097496159 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50cc085330 con 0x7f50d4166550 2024-04-17T10:49:21.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.113+0000 7f50d9db8640 1 --2- 172.21.15.83:0/3097496159 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f50b8066f60 0x7f50b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:21.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.113+0000 7f50d9db8640 1 --2- 172.21.15.83:0/3097496159 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f50b8066f60 0x7f50b8069420 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f50c40ad000 tx=0x7f50c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:21.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.117+0000 7f50ca7fc640 1 -- 172.21.15.83:0/3097496159 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f50cc055cf0 con 0x7f50d4166550 2024-04-17T10:49:21.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.277+0000 7f50dc01c640 1 -- 172.21.15.83:0/3097496159 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50d41556d0 con 0x7f50d4166550 2024-04-17T10:49:21.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.281+0000 7f50ca7fc640 1 -- 172.21.15.83:0/3097496159 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+16620 (secure 0 0 0) 0x7f50cc059d90 con 0x7f50d4166550 2024-04-17T10:49:21.277 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:21.278 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":126,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:49:19.961945+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:49:18.247824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-17T10:49:18.951409+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:49:21.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.281+0000 7f50dc01c640 1 -- 172.21.15.83:0/3097496159 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f50b8066f60 msgr2=0x7f50b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:21.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.281+0000 7f50dc01c640 1 --2- 172.21.15.83:0/3097496159 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f50b8066f60 0x7f50b8069420 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f50c40ad000 tx=0x7f50c4008040 comp rx=0 tx=0).stop 2024-04-17T10:49:21.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.281+0000 7f50dc01c640 1 -- 172.21.15.83:0/3097496159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50d4166550 msgr2=0x7f50d41532a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:21.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.281+0000 7f50dc01c640 1 --2- 172.21.15.83:0/3097496159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50d4166550 0x7f50d41532a0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00e960 tx=0x7f50cc00ee20 comp rx=0 tx=0).stop 2024-04-17T10:49:21.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.281+0000 7f50dc01c640 1 -- 172.21.15.83:0/3097496159 shutdown_connections 2024-04-17T10:49:21.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.281+0000 7f50dc01c640 1 --2- 172.21.15.83:0/3097496159 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f50b8066f60 0x7f50b8069420 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:21.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.281+0000 7f50dc01c640 1 --2- 172.21.15.83:0/3097496159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50d4166550 0x7f50d41532a0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:21.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.281+0000 7f50dc01c640 1 --2- 172.21.15.83:0/3097496159 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50d40b3f80 0x7f50d41fadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:21.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.281+0000 7f50dc01c640 1 --2- 172.21.15.83:0/3097496159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50d40aecd0 0x7f50d4152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:21.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.281+0000 7f50dc01c640 1 -- 172.21.15.83:0/3097496159 >> 172.21.15.83:0/3097496159 conn(0x7f50d406abe0 msgr2=0x7f50d406ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:21.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.281+0000 7f50dc01c640 1 -- 172.21.15.83:0/3097496159 shutdown_connections 2024-04-17T10:49:21.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.281+0000 7f50dc01c640 1 -- 172.21.15.83:0/3097496159 wait complete. 2024-04-17T10:49:21.297 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-17T10:49:21.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.425+0000 7f49018df640 1 Processor -- start 2024-04-17T10:49:21.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.425+0000 7f49018df640 1 -- start start 2024-04-17T10:49:21.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.425+0000 7f49018df640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f48fc15b6e0 0x7f48fc071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:21.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.425+0000 7f49018df640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f48fc070470 0x7f48fc06ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:21.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.425+0000 7f49018df640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48fc06c1f0 con 0x7f48fc15bac0 2024-04-17T10:49:21.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.425+0000 7f49018df640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48fc070a10 con 0x7f48fc070470 2024-04-17T10:49:21.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.425+0000 7f49018df640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48fc070cf0 con 0x7f48fc15b6e0 2024-04-17T10:49:21.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.425+0000 7f48fb7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f48fc070470 0x7f48fc06ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:21.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.425+0000 7f48fb7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f48fc070470 0x7f48fc06ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35550/0 (socket says 172.21.15.83:35550) 2024-04-17T10:49:21.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.425+0000 7f48fb7fe640 1 -- 172.21.15.83:0/80044906 learned_addr learned my addr 172.21.15.83:0/80044906 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:21.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.425+0000 7f48faffd640 1 --2- 172.21.15.83:0/80044906 >> v2:172.21.15.167:3300/0 conn(0x7f48fc15b6e0 0x7f48fc071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:21.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.425+0000 7f48fb7fe640 1 -- 172.21.15.83:0/80044906 >> v2:172.21.15.167:3300/0 conn(0x7f48fc15b6e0 msgr2=0x7f48fc071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:21.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.425+0000 7f48fb7fe640 1 --2- 172.21.15.83:0/80044906 >> v2:172.21.15.167:3300/0 conn(0x7f48fc15b6e0 0x7f48fc071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:21.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.425+0000 7f48fb7fe640 1 -- 172.21.15.83:0/80044906 >> v1:172.21.15.83:6789/0 conn(0x7f48fc15bac0 legacy=0x7f48fc0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:21.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.425+0000 7f48fb7fe640 1 -- 172.21.15.83:0/80044906 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48fc070fd0 con 0x7f48fc070470 2024-04-17T10:49:21.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.429+0000 7f48fb7fe640 1 --2- 172.21.15.83:0/80044906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48fc070470 0x7f48fc06ae10 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0728a0 tx=0x7f48f401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9efb5a2e13d04b90 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:21.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.429+0000 7f48fa7fc640 1 -- 172.21.15.83:0/80044906 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f40a56c0 con 0x7f48fc070470 2024-04-17T10:49:21.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.429+0000 7f48fa7fc640 1 -- 172.21.15.83:0/80044906 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f40a5860 con 0x7f48fc070470 2024-04-17T10:49:21.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.429+0000 7f48fa7fc640 1 -- 172.21.15.83:0/80044906 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f40a5c30 con 0x7f48fc070470 2024-04-17T10:49:21.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.429+0000 7f49018df640 1 -- 172.21.15.83:0/80044906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48fc070470 msgr2=0x7f48fc06ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:21.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.429+0000 7f49018df640 1 --2- 172.21.15.83:0/80044906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48fc070470 0x7f48fc06ae10 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0728a0 tx=0x7f48f401c9f0 comp rx=0 tx=0).stop 2024-04-17T10:49:21.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.429+0000 7f49018df640 1 -- 172.21.15.83:0/80044906 shutdown_connections 2024-04-17T10:49:21.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.429+0000 7f49018df640 1 --2- 172.21.15.83:0/80044906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48fc070470 0x7f48fc06ae10 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:21.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.429+0000 7f49018df640 1 --2- 172.21.15.83:0/80044906 >> v2:172.21.15.167:3300/0 conn(0x7f48fc15b6e0 0x7f48fc071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:21.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.429+0000 7f49018df640 1 -- 172.21.15.83:0/80044906 >> 172.21.15.83:0/80044906 conn(0x7f48fc0b15b0 msgr2=0x7f48fc0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:21.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.429+0000 7f49018df640 1 -- 172.21.15.83:0/80044906 shutdown_connections 2024-04-17T10:49:21.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.429+0000 7f49018df640 1 -- 172.21.15.83:0/80044906 wait complete. 2024-04-17T10:49:21.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.429+0000 7f49018df640 1 Processor -- start 2024-04-17T10:49:21.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.429+0000 7f49018df640 1 -- start start 2024-04-17T10:49:21.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.429+0000 7f49018df640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48fc070470 0x7f48fc141310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:21.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.429+0000 7f49018df640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48fc15b6e0 0x7f48fc149160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:21.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.429+0000 7f49018df640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48fc15bac0 0x7f48fc145500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:21.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.429+0000 7f49018df640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc06d2b0 con 0x7f48fc15b6e0 2024-04-17T10:49:21.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.429+0000 7f49018df640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc06d130 con 0x7f48fc15bac0 2024-04-17T10:49:21.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.429+0000 7f49018df640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc06b760 con 0x7f48fc070470 2024-04-17T10:49:21.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.429+0000 7f48f27fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48fc15b6e0 0x7f48fc149160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:21.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.429+0000 7f48f27fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48fc15b6e0 0x7f48fc149160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35562/0 (socket says 172.21.15.83:35562) 2024-04-17T10:49:21.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.429+0000 7f48f27fc640 1 -- 172.21.15.83:0/879881260 learned_addr learned my addr 172.21.15.83:0/879881260 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:21.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.429+0000 7f48fb7fe640 1 --2- 172.21.15.83:0/879881260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48fc15bac0 0x7f48fc145500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:21.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.429+0000 7f48faffd640 1 --2- 172.21.15.83:0/879881260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48fc070470 0x7f48fc141310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:21.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.429+0000 7f48f27fc640 1 -- 172.21.15.83:0/879881260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48fc070470 msgr2=0x7f48fc141310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:21.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.429+0000 7f48f27fc640 1 --2- 172.21.15.83:0/879881260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48fc070470 0x7f48fc141310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:21.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.429+0000 7f48f27fc640 1 -- 172.21.15.83:0/879881260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48fc15bac0 msgr2=0x7f48fc145500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:21.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.429+0000 7f48f27fc640 1 --2- 172.21.15.83:0/879881260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48fc15bac0 0x7f48fc145500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:21.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.429+0000 7f48f27fc640 1 -- 172.21.15.83:0/879881260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48fc149be0 con 0x7f48fc15b6e0 2024-04-17T10:49:21.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.429+0000 7f48fb7fe640 1 --2- 172.21.15.83:0/879881260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48fc15bac0 0x7f48fc145500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:21.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.429+0000 7f48faffd640 1 --2- 172.21.15.83:0/879881260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48fc070470 0x7f48fc141310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:21.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.433+0000 7f48f27fc640 1 --2- 172.21.15.83:0/879881260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48fc15b6e0 0x7f48fc149160 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f48e400c3c0 tx=0x7f48e400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:21.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.433+0000 7f48f8ff9640 1 -- 172.21.15.83:0/879881260 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e4017070 con 0x7f48fc15b6e0 2024-04-17T10:49:21.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.433+0000 7f48f8ff9640 1 -- 172.21.15.83:0/879881260 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e40052f0 con 0x7f48fc15b6e0 2024-04-17T10:49:21.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.433+0000 7f49018df640 1 -- 172.21.15.83:0/879881260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48fc149ec0 con 0x7f48fc15b6e0 2024-04-17T10:49:21.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.433+0000 7f48f8ff9640 1 -- 172.21.15.83:0/879881260 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e4003840 con 0x7f48fc15b6e0 2024-04-17T10:49:21.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.433+0000 7f49018df640 1 -- 172.21.15.83:0/879881260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48fc146250 con 0x7f48fc15b6e0 2024-04-17T10:49:21.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.433+0000 7f49018df640 1 -- 172.21.15.83:0/879881260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48fc074f90 con 0x7f48fc15b6e0 2024-04-17T10:49:21.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.433+0000 7f48f8ff9640 1 -- 172.21.15.83:0/879881260 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f48e401a070 con 0x7f48fc15b6e0 2024-04-17T10:49:21.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.433+0000 7f48f8ff9640 1 --2- 172.21.15.83:0/879881260 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f48dc066f90 0x7f48dc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:21.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.433+0000 7f48f8ff9640 1 -- 172.21.15.83:0/879881260 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f48e4084210 con 0x7f48fc15b6e0 2024-04-17T10:49:21.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.433+0000 7f48faffd640 1 --2- 172.21.15.83:0/879881260 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f48dc066f90 0x7f48dc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:21.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.433+0000 7f48faffd640 1 --2- 172.21.15.83:0/879881260 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f48dc066f90 0x7f48dc069450 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f48ec009840 tx=0x7f48ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:21.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.437+0000 7f48f8ff9640 1 -- 172.21.15.83:0/879881260 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f48e40552b0 con 0x7f48fc15b6e0 2024-04-17T10:49:21.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.625+0000 7f49018df640 1 -- 172.21.15.83:0/879881260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f48fc152c00 con 0x7f48fc15b6e0 2024-04-17T10:49:21.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.645+0000 7f48f8ff9640 1 -- 172.21.15.83:0/879881260 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v75) v1 ==== 118+0+0 (secure 0 0 0) 0x7f48e4054bb0 con 0x7f48fc15b6e0 2024-04-17T10:49:21.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.649+0000 7f49018df640 1 -- 172.21.15.83:0/879881260 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f48dc066f90 msgr2=0x7f48dc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:21.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.649+0000 7f49018df640 1 --2- 172.21.15.83:0/879881260 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f48dc066f90 0x7f48dc069450 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f48ec009840 tx=0x7f48ec008040 comp rx=0 tx=0).stop 2024-04-17T10:49:21.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.649+0000 7f49018df640 1 -- 172.21.15.83:0/879881260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48fc15b6e0 msgr2=0x7f48fc149160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:21.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.649+0000 7f49018df640 1 --2- 172.21.15.83:0/879881260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48fc15b6e0 0x7f48fc149160 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f48e400c3c0 tx=0x7f48e400c880 comp rx=0 tx=0).stop 2024-04-17T10:49:21.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.649+0000 7f49018df640 1 -- 172.21.15.83:0/879881260 shutdown_connections 2024-04-17T10:49:21.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.649+0000 7f49018df640 1 --2- 172.21.15.83:0/879881260 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f48dc066f90 0x7f48dc069450 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.649+0000 7f49018df640 1 --2- 172.21.15.83:0/879881260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48fc15bac0 0x7f48fc145500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.649+0000 7f49018df640 1 --2- 172.21.15.83:0/879881260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48fc15b6e0 0x7f48fc149160 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.649+0000 7f49018df640 1 --2- 172.21.15.83:0/879881260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48fc070470 0x7f48fc141310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.649+0000 7f49018df640 1 -- 172.21.15.83:0/879881260 >> 172.21.15.83:0/879881260 conn(0x7f48fc0b15b0 msgr2=0x7f48fc0cd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.649+0000 7f49018df640 1 -- 172.21.15.83:0/879881260 shutdown_connections 2024-04-17T10:49:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.649+0000 7f49018df640 1 -- 172.21.15.83:0/879881260 wait complete. 2024-04-17T10:49:21.661 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-17T10:49:21.661 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-17T10:49:21.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.781+0000 7f12bc0d8640 1 Processor -- start 2024-04-17T10:49:21.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.781+0000 7f12bc0d8640 1 -- start start 2024-04-17T10:49:21.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.785+0000 7f12bc0d8640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f12b415b6e0 0x7f12b4071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:21.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.785+0000 7f12bc0d8640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12b406c1f0 con 0x7f12b4070470 2024-04-17T10:49:21.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.785+0000 7f12bc0d8640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12b4070a10 con 0x7f12b415bac0 2024-04-17T10:49:21.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.785+0000 7f12bc0d8640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12b4070cf0 con 0x7f12b415b6e0 2024-04-17T10:49:21.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.785+0000 7f12b9e74640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f12b415b6e0 0x7f12b4071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:21.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.785+0000 7f12ba675640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f12b4070470 0x7f12b406ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41416/0 (socket says 172.21.15.83:41416) 2024-04-17T10:49:21.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.785+0000 7f12ba675640 1 -- 172.21.15.83:0/196845339 learned_addr learned my addr 172.21.15.83:0/196845339 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:21.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.785+0000 7f12b9e74640 1 -- 172.21.15.83:0/196845339 >> v1:172.21.15.167:6789/0 conn(0x7f12b415bac0 legacy=0x7f12b40cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:21.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.785+0000 7f12b9e74640 1 -- 172.21.15.83:0/196845339 >> v1:172.21.15.83:6789/0 conn(0x7f12b4070470 legacy=0x7f12b406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:21.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.785+0000 7f12b9e74640 1 -- 172.21.15.83:0/196845339 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b4070fd0 con 0x7f12b415b6e0 2024-04-17T10:49:21.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.785+0000 7f12b8e72640 1 -- 172.21.15.83:0/196845339 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1668296962 0 0) 0x7f12b406c1f0 con 0x7f12b4070470 2024-04-17T10:49:21.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.785+0000 7f12b9e74640 1 --2- 172.21.15.83:0/196845339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12b415b6e0 0x7f12b4071cc0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f12b00097c0 tx=0x7f12b001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22bed380316449f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:21.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.785+0000 7f12b8e72640 1 -- 172.21.15.83:0/196845339 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b00b0070 con 0x7f12b415b6e0 2024-04-17T10:49:21.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.785+0000 7f12b8e72640 1 -- 172.21.15.83:0/196845339 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b00ab600 con 0x7f12b415b6e0 2024-04-17T10:49:21.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.785+0000 7f12b8e72640 1 -- 172.21.15.83:0/196845339 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b00ab9d0 con 0x7f12b415b6e0 2024-04-17T10:49:21.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.789+0000 7f12bc0d8640 1 -- 172.21.15.83:0/196845339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12b415b6e0 msgr2=0x7f12b4071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:21.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.789+0000 7f12bc0d8640 1 --2- 172.21.15.83:0/196845339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12b415b6e0 0x7f12b4071cc0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f12b00097c0 tx=0x7f12b001c460 comp rx=0 tx=0).stop 2024-04-17T10:49:21.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.789+0000 7f12bc0d8640 1 -- 172.21.15.83:0/196845339 shutdown_connections 2024-04-17T10:49:21.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.789+0000 7f12bc0d8640 1 --2- 172.21.15.83:0/196845339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12b415b6e0 0x7f12b4071cc0 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:21.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.789+0000 7f12bc0d8640 1 -- 172.21.15.83:0/196845339 >> 172.21.15.83:0/196845339 conn(0x7f12b40b15b0 msgr2=0x7f12b40b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:21.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.789+0000 7f12bc0d8640 1 -- 172.21.15.83:0/196845339 shutdown_connections 2024-04-17T10:49:21.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.789+0000 7f12bc0d8640 1 -- 172.21.15.83:0/196845339 wait complete. 2024-04-17T10:49:21.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.789+0000 7f12bc0d8640 1 Processor -- start 2024-04-17T10:49:21.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.789+0000 7f12bc0d8640 1 -- start start 2024-04-17T10:49:21.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.789+0000 7f12bc0d8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12b4070470 0x7f12b4148900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:21.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.789+0000 7f12bc0d8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12b415b6e0 0x7f12b415a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:21.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.789+0000 7f12bc0d8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12b415bac0 0x7f12b41f63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:21.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.789+0000 7f12bc0d8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b406cee0 con 0x7f12b415bac0 2024-04-17T10:49:21.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.789+0000 7f12bc0d8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b406d060 con 0x7f12b4070470 2024-04-17T10:49:21.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.789+0000 7f12bc0d8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b406d1e0 con 0x7f12b415b6e0 2024-04-17T10:49:21.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.789+0000 7f12ba675640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12b415bac0 0x7f12b41f63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:21.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.789+0000 7f12ba675640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12b415bac0 0x7f12b41f63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35578/0 (socket says 172.21.15.83:35578) 2024-04-17T10:49:21.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.789+0000 7f12ba675640 1 -- 172.21.15.83:0/2093402187 learned_addr learned my addr 172.21.15.83:0/2093402187 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:21.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.789+0000 7f12b9673640 1 --2- 172.21.15.83:0/2093402187 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12b415b6e0 0x7f12b415a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:21.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.789+0000 7f12b9e74640 1 --2- 172.21.15.83:0/2093402187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12b4070470 0x7f12b4148900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.789+0000 7f12ba675640 1 -- 172.21.15.83:0/2093402187 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12b415b6e0 msgr2=0x7f12b415a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.789+0000 7f12ba675640 1 --2- 172.21.15.83:0/2093402187 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12b415b6e0 0x7f12b415a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.789+0000 7f12ba675640 1 -- 172.21.15.83:0/2093402187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12b4070470 msgr2=0x7f12b4148900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.789+0000 7f12ba675640 1 --2- 172.21.15.83:0/2093402187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12b4070470 0x7f12b4148900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.789+0000 7f12ba675640 1 -- 172.21.15.83:0/2093402187 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b4149330 con 0x7f12b415bac0 2024-04-17T10:49:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.789+0000 7f12ba675640 1 --2- 172.21.15.83:0/2093402187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12b415bac0 0x7f12b41f63a0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f12ac00cb20 tx=0x7f12ac00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.789+0000 7f12aa7fc640 1 -- 172.21.15.83:0/2093402187 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ac00aca0 con 0x7f12b415bac0 2024-04-17T10:49:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.789+0000 7f12aa7fc640 1 -- 172.21.15.83:0/2093402187 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12ac012400 con 0x7f12b415bac0 2024-04-17T10:49:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.789+0000 7f12aa7fc640 1 -- 172.21.15.83:0/2093402187 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ac00ae60 con 0x7f12b415bac0 2024-04-17T10:49:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.789+0000 7f12bc0d8640 1 -- 172.21.15.83:0/2093402187 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b4149670 con 0x7f12b415bac0 2024-04-17T10:49:21.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.789+0000 7f12bc0d8640 1 -- 172.21.15.83:0/2093402187 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b4149eb0 con 0x7f12b415bac0 2024-04-17T10:49:21.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.793+0000 7f12bc0d8640 1 -- 172.21.15.83:0/2093402187 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12b414a0b0 con 0x7f12b415bac0 2024-04-17T10:49:21.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.793+0000 7f12aa7fc640 1 -- 172.21.15.83:0/2093402187 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f12ac0127f0 con 0x7f12b415bac0 2024-04-17T10:49:21.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.793+0000 7f12aa7fc640 1 --2- 172.21.15.83:0/2093402187 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1298066f60 0x7f1298069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:21.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.793+0000 7f12b9e74640 1 --2- 172.21.15.83:0/2093402187 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1298066f60 0x7f1298069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:21.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.793+0000 7f12aa7fc640 1 -- 172.21.15.83:0/2093402187 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f12ac0856f0 con 0x7f12b415bac0 2024-04-17T10:49:21.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.793+0000 7f12b9e74640 1 --2- 172.21.15.83:0/2093402187 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1298066f60 0x7f1298069420 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f12b00ad000 tx=0x7f12b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:21.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.797+0000 7f12aa7fc640 1 -- 172.21.15.83:0/2093402187 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f12b414a0b0 con 0x7f12b415bac0 2024-04-17T10:49:21.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:21.989+0000 7f12bc0d8640 1 -- 172.21.15.83:0/2093402187 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f12b4074f90 con 0x7f12b415bac0 2024-04-17T10:49:22.585 INFO:tasks.ceph.mds.c.smithi083.stderr:2024-04-17T10:49:22.589+0000 7fd2473e7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T10:49:22.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.657+0000 7f12aa7fc640 1 -- 172.21.15.83:0/2093402187 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v76) v1 ==== 122+0+0 (secure 0 0 0) 0x7f12b4149670 con 0x7f12b415bac0 2024-04-17T10:49:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.661+0000 7f12bc0d8640 1 -- 172.21.15.83:0/2093402187 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1298066f60 msgr2=0x7f1298069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.661+0000 7f12bc0d8640 1 --2- 172.21.15.83:0/2093402187 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1298066f60 0x7f1298069420 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f12b00ad000 tx=0x7f12b0008040 comp rx=0 tx=0).stop 2024-04-17T10:49:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.661+0000 7f12bc0d8640 1 -- 172.21.15.83:0/2093402187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12b415bac0 msgr2=0x7f12b41f63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.661+0000 7f12bc0d8640 1 --2- 172.21.15.83:0/2093402187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12b415bac0 0x7f12b41f63a0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f12ac00cb20 tx=0x7f12ac00a060 comp rx=0 tx=0).stop 2024-04-17T10:49:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.661+0000 7f12bc0d8640 1 -- 172.21.15.83:0/2093402187 shutdown_connections 2024-04-17T10:49:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.661+0000 7f12bc0d8640 1 --2- 172.21.15.83:0/2093402187 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1298066f60 0x7f1298069420 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.661+0000 7f12bc0d8640 1 --2- 172.21.15.83:0/2093402187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12b415bac0 0x7f12b41f63a0 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.661+0000 7f12bc0d8640 1 --2- 172.21.15.83:0/2093402187 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12b415b6e0 0x7f12b415a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.661+0000 7f12bc0d8640 1 --2- 172.21.15.83:0/2093402187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12b4070470 0x7f12b4148900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.661+0000 7f12bc0d8640 1 -- 172.21.15.83:0/2093402187 >> 172.21.15.83:0/2093402187 conn(0x7f12b40b15b0 msgr2=0x7f12b40cd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.661+0000 7f12bc0d8640 1 -- 172.21.15.83:0/2093402187 shutdown_connections 2024-04-17T10:49:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.661+0000 7f12bc0d8640 1 -- 172.21.15.83:0/2093402187 wait complete. 2024-04-17T10:49:22.676 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-17T10:49:22.676 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:49:22.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.805+0000 7f4a88d62640 1 Processor -- start 2024-04-17T10:49:22.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.805+0000 7f4a88d62640 1 -- start start 2024-04-17T10:49:22.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.805+0000 7f4a88d62640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4a8415ba30 0x7f4a84072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:22.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.805+0000 7f4a88d62640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a840cfbe0 0x7f4a8406c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:22.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.805+0000 7f4a88d62640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a8406e320 con 0x7f4a8415c960 2024-04-17T10:49:22.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.805+0000 7f4a88d62640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a84071ea0 con 0x7f4a8415ba30 2024-04-17T10:49:22.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.805+0000 7f4a88d62640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8406e040 con 0x7f4a840cfbe0 2024-04-17T10:49:22.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.805+0000 7f4a8259c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4a8415ba30 0x7f4a84072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:22.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.805+0000 7f4a81d9b640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f4a8415c960 0x7f4a840cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:39146/0 (socket says 172.21.15.83:39146) 2024-04-17T10:49:22.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.805+0000 7f4a81d9b640 1 -- 172.21.15.83:0/3395607289 learned_addr learned my addr 172.21.15.83:0/3395607289 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:22.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.805+0000 7f4a82d9d640 1 --2- 172.21.15.83:0/3395607289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a840cfbe0 0x7f4a8406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:22.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.805+0000 7f4a8259c640 1 -- 172.21.15.83:0/3395607289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a840cfbe0 msgr2=0x7f4a8406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:22.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.805+0000 7f4a8259c640 1 --2- 172.21.15.83:0/3395607289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a840cfbe0 0x7f4a8406c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:22.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.805+0000 7f4a8259c640 1 -- 172.21.15.83:0/3395607289 >> v1:172.21.15.167:6789/0 conn(0x7f4a8415c960 legacy=0x7f4a840cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:22.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.805+0000 7f4a8259c640 1 -- 172.21.15.83:0/3395607289 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a84071850 con 0x7f4a8415ba30 2024-04-17T10:49:22.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.809+0000 7f4a8259c640 1 --2- 172.21.15.83:0/3395607289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a8415ba30 0x7f4a84072330 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4a780097c0 tx=0x7f4a7801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edfbb00f15dc5795 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:22.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.809+0000 7f4a8159a640 1 -- 172.21.15.83:0/3395607289 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a780b0070 con 0x7f4a8415ba30 2024-04-17T10:49:22.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.809+0000 7f4a8159a640 1 -- 172.21.15.83:0/3395607289 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a780ab600 con 0x7f4a8415ba30 2024-04-17T10:49:22.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.809+0000 7f4a8159a640 1 -- 172.21.15.83:0/3395607289 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a780ab9d0 con 0x7f4a8415ba30 2024-04-17T10:49:22.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.809+0000 7f4a88d62640 1 -- 172.21.15.83:0/3395607289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a8415ba30 msgr2=0x7f4a84072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:22.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.809+0000 7f4a88d62640 1 --2- 172.21.15.83:0/3395607289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a8415ba30 0x7f4a84072330 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4a780097c0 tx=0x7f4a7801c410 comp rx=0 tx=0).stop 2024-04-17T10:49:22.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.809+0000 7f4a88d62640 1 -- 172.21.15.83:0/3395607289 shutdown_connections 2024-04-17T10:49:22.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.809+0000 7f4a88d62640 1 --2- 172.21.15.83:0/3395607289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a840cfbe0 0x7f4a8406c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:22.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.809+0000 7f4a88d62640 1 --2- 172.21.15.83:0/3395607289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a8415ba30 0x7f4a84072330 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:22.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.809+0000 7f4a88d62640 1 -- 172.21.15.83:0/3395607289 >> 172.21.15.83:0/3395607289 conn(0x7f4a8406abe0 msgr2=0x7f4a8406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:22.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.809+0000 7f4a88d62640 1 -- 172.21.15.83:0/3395607289 shutdown_connections 2024-04-17T10:49:22.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.809+0000 7f4a88d62640 1 -- 172.21.15.83:0/3395607289 wait complete. 2024-04-17T10:49:22.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.809+0000 7f4a88d62640 1 Processor -- start 2024-04-17T10:49:22.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.809+0000 7f4a88d62640 1 -- start start 2024-04-17T10:49:22.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.809+0000 7f4a88d62640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a840cfbe0 0x7f4a841544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:22.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.809+0000 7f4a88d62640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a8415ba30 0x7f4a841f9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:22.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.809+0000 7f4a88d62640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a8415c960 0x7f4a841fa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:22.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.809+0000 7f4a88d62640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8415cee0 con 0x7f4a8415c960 2024-04-17T10:49:22.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.809+0000 7f4a88d62640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8415cd60 con 0x7f4a8415ba30 2024-04-17T10:49:22.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.809+0000 7f4a88d62640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a84168520 con 0x7f4a840cfbe0 2024-04-17T10:49:22.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.809+0000 7f4a82d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a8415c960 0x7f4a841fa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:22.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.809+0000 7f4a82d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a8415c960 0x7f4a841fa3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35594/0 (socket says 172.21.15.83:35594) 2024-04-17T10:49:22.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.809+0000 7f4a82d9d640 1 -- 172.21.15.83:0/3452736517 learned_addr learned my addr 172.21.15.83:0/3452736517 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:22.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.809+0000 7f4a81d9b640 1 --2- 172.21.15.83:0/3452736517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a8415ba30 0x7f4a841f9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:22.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.809+0000 7f4a8259c640 1 --2- 172.21.15.83:0/3452736517 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a840cfbe0 0x7f4a841544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:22.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.809+0000 7f4a82d9d640 1 -- 172.21.15.83:0/3452736517 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a840cfbe0 msgr2=0x7f4a841544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:22.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.809+0000 7f4a82d9d640 1 --2- 172.21.15.83:0/3452736517 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a840cfbe0 0x7f4a841544d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:22.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.809+0000 7f4a82d9d640 1 -- 172.21.15.83:0/3452736517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a8415ba30 msgr2=0x7f4a841f9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:22.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.809+0000 7f4a82d9d640 1 --2- 172.21.15.83:0/3452736517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a8415ba30 0x7f4a841f9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:22.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.809+0000 7f4a82d9d640 1 -- 172.21.15.83:0/3452736517 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8414b170 con 0x7f4a8415c960 2024-04-17T10:49:22.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.809+0000 7f4a82d9d640 1 --2- 172.21.15.83:0/3452736517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a8415c960 0x7f4a841fa3c0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00efc0 tx=0x7f4a7c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:22.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.809+0000 7f4a72ffd640 1 -- 172.21.15.83:0/3452736517 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c018020 con 0x7f4a8415c960 2024-04-17T10:49:22.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.813+0000 7f4a72ffd640 1 -- 172.21.15.83:0/3452736517 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a7c014480 con 0x7f4a8415c960 2024-04-17T10:49:22.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.813+0000 7f4a72ffd640 1 -- 172.21.15.83:0/3452736517 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c014870 con 0x7f4a8415c960 2024-04-17T10:49:22.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.813+0000 7f4a88d62640 1 -- 172.21.15.83:0/3452736517 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a8414b450 con 0x7f4a8415c960 2024-04-17T10:49:22.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.813+0000 7f4a88d62640 1 -- 172.21.15.83:0/3452736517 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a8414ba10 con 0x7f4a8415c960 2024-04-17T10:49:22.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.813+0000 7f4a88d62640 1 -- 172.21.15.83:0/3452736517 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a84074f90 con 0x7f4a8415c960 2024-04-17T10:49:22.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.813+0000 7f4a72ffd640 1 -- 172.21.15.83:0/3452736517 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4a7c01b020 con 0x7f4a8415c960 2024-04-17T10:49:22.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.813+0000 7f4a72ffd640 1 --2- 172.21.15.83:0/3452736517 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4a64066f60 0x7f4a64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:22.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.813+0000 7f4a72ffd640 1 -- 172.21.15.83:0/3452736517 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a7c085a50 con 0x7f4a8415c960 2024-04-17T10:49:22.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.813+0000 7f4a8259c640 1 --2- 172.21.15.83:0/3452736517 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4a64066f60 0x7f4a64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:22.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.813+0000 7f4a8259c640 1 --2- 172.21.15.83:0/3452736517 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4a64066f60 0x7f4a64069420 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4a780ad000 tx=0x7f4a780a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:22.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:22.817+0000 7f4a72ffd640 1 -- 172.21.15.83:0/3452736517 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4a7c022070 con 0x7f4a8415c960 2024-04-17T10:49:23.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.005+0000 7f4a88d62640 1 -- 172.21.15.83:0/3452736517 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a841562d0 con 0x7f4a8415c960 2024-04-17T10:49:23.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.005+0000 7f4a72ffd640 1 -- 172.21.15.83:0/3452736517 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f4a7c010040 con 0x7f4a8415c960 2024-04-17T10:49:23.002 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:23.002 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5942,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3888166846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3888166846},{"type":"v1","addr":"172.21.15.167:6833","nonce":3888166846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":6772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1167913722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1167913722},{"type":"v1","addr":"172.21.15.167:6835","nonce":1167913722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":9192,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3552333026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3552333026},{"type":"v1","addr":"172.21.15.83:6835","nonce":3552333026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:49:19.962680+0000","modified":"2024-04-17T10:49:21.996802+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9198":{"gid":9198,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1559222491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1559222491},{"type":"v1","addr":"172.21.15.83:6837","nonce":1559222491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-17T10:49:23.002 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 76 2024-04-17T10:49:23.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.005+0000 7f4a88d62640 1 -- 172.21.15.83:0/3452736517 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4a64066f60 msgr2=0x7f4a64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:23.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.005+0000 7f4a88d62640 1 --2- 172.21.15.83:0/3452736517 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4a64066f60 0x7f4a64069420 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4a780ad000 tx=0x7f4a780a6000 comp rx=0 tx=0).stop 2024-04-17T10:49:23.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.005+0000 7f4a88d62640 1 -- 172.21.15.83:0/3452736517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a8415c960 msgr2=0x7f4a841fa3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:23.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.005+0000 7f4a88d62640 1 --2- 172.21.15.83:0/3452736517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a8415c960 0x7f4a841fa3c0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00efc0 tx=0x7f4a7c00c490 comp rx=0 tx=0).stop 2024-04-17T10:49:23.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.005+0000 7f4a88d62640 1 -- 172.21.15.83:0/3452736517 shutdown_connections 2024-04-17T10:49:23.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.005+0000 7f4a88d62640 1 --2- 172.21.15.83:0/3452736517 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4a64066f60 0x7f4a64069420 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:23.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.005+0000 7f4a88d62640 1 --2- 172.21.15.83:0/3452736517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a8415c960 0x7f4a841fa3c0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:23.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.005+0000 7f4a88d62640 1 --2- 172.21.15.83:0/3452736517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a8415ba30 0x7f4a841f9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:23.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.005+0000 7f4a88d62640 1 --2- 172.21.15.83:0/3452736517 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a840cfbe0 0x7f4a841544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:23.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.005+0000 7f4a88d62640 1 -- 172.21.15.83:0/3452736517 >> 172.21.15.83:0/3452736517 conn(0x7f4a8406abe0 msgr2=0x7f4a84148a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:23.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.005+0000 7f4a88d62640 1 -- 172.21.15.83:0/3452736517 shutdown_connections 2024-04-17T10:49:23.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.009+0000 7f4a88d62640 1 -- 172.21.15.83:0/3452736517 wait complete. 2024-04-17T10:49:23.020 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:49:23.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.133+0000 7f2a44892640 1 Processor -- start 2024-04-17T10:49:23.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.133+0000 7f2a44892640 1 -- start start 2024-04-17T10:49:23.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.133+0000 7f2a44892640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a4015ba30 0x7f2a40072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:23.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.133+0000 7f2a44892640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2a4015c960 0x7f2a400cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:23.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.133+0000 7f2a44892640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2a400cfa10 0x7f2a4006c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:23.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.133+0000 7f2a44892640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a40071ea0 con 0x7f2a4015c960 2024-04-17T10:49:23.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.133+0000 7f2a44892640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a400cfea0 con 0x7f2a400cfa10 2024-04-17T10:49:23.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.133+0000 7f2a44892640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4006e320 con 0x7f2a4015ba30 2024-04-17T10:49:23.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.133+0000 7f2a37fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2a4015c960 0x7f2a400cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:23.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.133+0000 7f2a37fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2a4015c960 0x7f2a400cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35608/0 (socket says 172.21.15.83:35608) 2024-04-17T10:49:23.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.133+0000 7f2a37fff640 1 -- 172.21.15.83:0/3216289576 learned_addr learned my addr 172.21.15.83:0/3216289576 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:23.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.133+0000 7f2a3ed9d640 1 --2- 172.21.15.83:0/3216289576 >> v2:172.21.15.167:3300/0 conn(0x7f2a400cfa10 0x7f2a4006c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:23.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.133+0000 7f2a3e59c640 1 --2- 172.21.15.83:0/3216289576 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a4015ba30 0x7f2a40072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:23.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.133+0000 7f2a37fff640 1 -- 172.21.15.83:0/3216289576 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a4015ba30 msgr2=0x7f2a40072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:23.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.133+0000 7f2a37fff640 1 --2- 172.21.15.83:0/3216289576 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a4015ba30 0x7f2a40072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:23.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.133+0000 7f2a37fff640 1 -- 172.21.15.83:0/3216289576 >> v2:172.21.15.167:3300/0 conn(0x7f2a400cfa10 msgr2=0x7f2a4006c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:23.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.133+0000 7f2a37fff640 1 --2- 172.21.15.83:0/3216289576 >> v2:172.21.15.167:3300/0 conn(0x7f2a400cfa10 0x7f2a4006c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:23.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.133+0000 7f2a37fff640 1 -- 172.21.15.83:0/3216289576 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a4006e040 con 0x7f2a4015c960 2024-04-17T10:49:23.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.133+0000 7f2a3ed9d640 1 --2- 172.21.15.83:0/3216289576 >> v2:172.21.15.167:3300/0 conn(0x7f2a400cfa10 0x7f2a4006c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:23.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a37fff640 1 --2- 172.21.15.83:0/3216289576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a4015c960 0x7f2a400cf4d0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f2a4006d810 tx=0x7f2a2801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cedf4cac90cbd40d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:23.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a3dd9b640 1 -- 172.21.15.83:0/3216289576 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a280b0070 con 0x7f2a4015c960 2024-04-17T10:49:23.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a3dd9b640 1 -- 172.21.15.83:0/3216289576 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a280ab600 con 0x7f2a4015c960 2024-04-17T10:49:23.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a3dd9b640 1 -- 172.21.15.83:0/3216289576 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a280ab9d0 con 0x7f2a4015c960 2024-04-17T10:49:23.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a44892640 1 -- 172.21.15.83:0/3216289576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a4015c960 msgr2=0x7f2a400cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:23.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a44892640 1 --2- 172.21.15.83:0/3216289576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a4015c960 0x7f2a400cf4d0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f2a4006d810 tx=0x7f2a2801c490 comp rx=0 tx=0).stop 2024-04-17T10:49:23.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a44892640 1 -- 172.21.15.83:0/3216289576 shutdown_connections 2024-04-17T10:49:23.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a44892640 1 --2- 172.21.15.83:0/3216289576 >> v2:172.21.15.167:3300/0 conn(0x7f2a400cfa10 0x7f2a4006c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:23.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a44892640 1 --2- 172.21.15.83:0/3216289576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a4015c960 0x7f2a400cf4d0 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:23.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a44892640 1 --2- 172.21.15.83:0/3216289576 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a4015ba30 0x7f2a40072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:23.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a44892640 1 -- 172.21.15.83:0/3216289576 >> 172.21.15.83:0/3216289576 conn(0x7f2a4006abe0 msgr2=0x7f2a4006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:23.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a44892640 1 -- 172.21.15.83:0/3216289576 shutdown_connections 2024-04-17T10:49:23.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a44892640 1 -- 172.21.15.83:0/3216289576 wait complete. 2024-04-17T10:49:23.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a44892640 1 Processor -- start 2024-04-17T10:49:23.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a44892640 1 -- start start 2024-04-17T10:49:23.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a44892640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a400cfa10 0x7f2a401378d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:23.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a44892640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a4015ba30 0x7f2a400af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:23.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a44892640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a4015c960 0x7f2a400afc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:23.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a44892640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4006dca0 con 0x7f2a400cfa10 2024-04-17T10:49:23.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a44892640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a400aeae0 con 0x7f2a4015ba30 2024-04-17T10:49:23.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a44892640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4006de20 con 0x7f2a4015c960 2024-04-17T10:49:23.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a3e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a400cfa10 0x7f2a401378d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:23.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a3e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a400cfa10 0x7f2a401378d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35624/0 (socket says 172.21.15.83:35624) 2024-04-17T10:49:23.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a3e59c640 1 -- 172.21.15.83:0/3574133834 learned_addr learned my addr 172.21.15.83:0/3574133834 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:23.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a37fff640 1 --2- 172.21.15.83:0/3574133834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a4015ba30 0x7f2a400af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:23.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a3ed9d640 1 --2- 172.21.15.83:0/3574133834 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a4015c960 0x7f2a400afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:23.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a3e59c640 1 -- 172.21.15.83:0/3574133834 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a4015c960 msgr2=0x7f2a400afc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:23.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a3e59c640 1 --2- 172.21.15.83:0/3574133834 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a4015c960 0x7f2a400afc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:23.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a3e59c640 1 -- 172.21.15.83:0/3574133834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a4015ba30 msgr2=0x7f2a400af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:23.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a3e59c640 1 --2- 172.21.15.83:0/3574133834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a4015ba30 0x7f2a400af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:23.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a3e59c640 1 -- 172.21.15.83:0/3574133834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a40139e40 con 0x7f2a400cfa10 2024-04-17T10:49:23.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a37fff640 1 --2- 172.21.15.83:0/3574133834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a4015ba30 0x7f2a400af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:23.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.137+0000 7f2a3e59c640 1 --2- 172.21.15.83:0/3574133834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a400cfa10 0x7f2a401378d0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000c7f0 tx=0x7f2a3000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:23.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.141+0000 7f2a36ffd640 1 -- 172.21.15.83:0/3574133834 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a30015070 con 0x7f2a400cfa10 2024-04-17T10:49:23.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.141+0000 7f2a36ffd640 1 -- 172.21.15.83:0/3574133834 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a30007820 con 0x7f2a400cfa10 2024-04-17T10:49:23.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.141+0000 7f2a44892640 1 -- 172.21.15.83:0/3574133834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a40138da0 con 0x7f2a400cfa10 2024-04-17T10:49:23.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.141+0000 7f2a36ffd640 1 -- 172.21.15.83:0/3574133834 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a30007c10 con 0x7f2a400cfa10 2024-04-17T10:49:23.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.141+0000 7f2a44892640 1 -- 172.21.15.83:0/3574133834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a40139360 con 0x7f2a400cfa10 2024-04-17T10:49:23.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.141+0000 7f2a44892640 1 -- 172.21.15.83:0/3574133834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a40074f90 con 0x7f2a400cfa10 2024-04-17T10:49:23.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.141+0000 7f2a36ffd640 1 -- 172.21.15.83:0/3574133834 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2a30018070 con 0x7f2a400cfa10 2024-04-17T10:49:23.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.141+0000 7f2a36ffd640 1 --2- 172.21.15.83:0/3574133834 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2a20066f60 0x7f2a20069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:23.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.141+0000 7f2a37fff640 1 --2- 172.21.15.83:0/3574133834 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2a20066f60 0x7f2a20069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:23.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.141+0000 7f2a36ffd640 1 -- 172.21.15.83:0/3574133834 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a300850c0 con 0x7f2a400cfa10 2024-04-17T10:49:23.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.141+0000 7f2a37fff640 1 --2- 172.21.15.83:0/3574133834 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2a20066f60 0x7f2a20069420 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2a40070060 tx=0x7f2a28008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:23.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.145+0000 7f2a36ffd640 1 -- 172.21.15.83:0/3574133834 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2a300560d0 con 0x7f2a400cfa10 2024-04-17T10:49:23.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.305+0000 7f2a44892640 1 -- 172.21.15.83:0/3574133834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a400aee90 con 0x7f2a400cfa10 2024-04-17T10:49:23.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.309+0000 7f2a36ffd640 1 -- 172.21.15.83:0/3574133834 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16607 (secure 0 0 0) 0x7f2a300559d0 con 0x7f2a400cfa10 2024-04-17T10:49:23.306 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:23.306 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":127,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:49:21.641129+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:49:18.247824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-17T10:49:18.951409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:49:23.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.309+0000 7f2a44892640 1 -- 172.21.15.83:0/3574133834 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2a20066f60 msgr2=0x7f2a20069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:23.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.309+0000 7f2a44892640 1 --2- 172.21.15.83:0/3574133834 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2a20066f60 0x7f2a20069420 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2a40070060 tx=0x7f2a28008040 comp rx=0 tx=0).stop 2024-04-17T10:49:23.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.309+0000 7f2a44892640 1 -- 172.21.15.83:0/3574133834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a400cfa10 msgr2=0x7f2a401378d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:23.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.309+0000 7f2a44892640 1 --2- 172.21.15.83:0/3574133834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a400cfa10 0x7f2a401378d0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000c7f0 tx=0x7f2a3000ccb0 comp rx=0 tx=0).stop 2024-04-17T10:49:23.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.309+0000 7f2a44892640 1 -- 172.21.15.83:0/3574133834 shutdown_connections 2024-04-17T10:49:23.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.309+0000 7f2a44892640 1 --2- 172.21.15.83:0/3574133834 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2a20066f60 0x7f2a20069420 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:23.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.309+0000 7f2a44892640 1 --2- 172.21.15.83:0/3574133834 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a4015c960 0x7f2a400afc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:23.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.309+0000 7f2a44892640 1 --2- 172.21.15.83:0/3574133834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a4015ba30 0x7f2a400af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:23.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.309+0000 7f2a44892640 1 --2- 172.21.15.83:0/3574133834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a400cfa10 0x7f2a401378d0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:23.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.309+0000 7f2a44892640 1 -- 172.21.15.83:0/3574133834 >> 172.21.15.83:0/3574133834 conn(0x7f2a4006abe0 msgr2=0x7f2a4014ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:23.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.309+0000 7f2a44892640 1 -- 172.21.15.83:0/3574133834 shutdown_connections 2024-04-17T10:49:23.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.309+0000 7f2a44892640 1 -- 172.21.15.83:0/3574133834 wait complete. 2024-04-17T10:49:23.324 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-17T10:49:23.325 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:49:23.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad89369640 1 Processor -- start 2024-04-17T10:49:23.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad89369640 1 -- start start 2024-04-17T10:49:23.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad89369640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad840b55e0 0x7fad840ad840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:23.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad89369640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad8406dcb0 con 0x7fad84147530 2024-04-17T10:49:23.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad89369640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad8406df80 con 0x7fad8406f020 2024-04-17T10:49:23.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad89369640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad8406eb90 con 0x7fad840b55e0 2024-04-17T10:49:23.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad82ffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fad84147530 0x7fad8406d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41422/0 (socket says 172.21.15.83:41422) 2024-04-17T10:49:23.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad82ffd640 1 -- 172.21.15.83:0/4119689453 learned_addr learned my addr 172.21.15.83:0/4119689453 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:23.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad837fe640 1 --2- 172.21.15.83:0/4119689453 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad840b55e0 0x7fad840ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:23.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad81ffb640 1 -- 172.21.15.83:0/4119689453 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3624880237 0 0) 0x7fad8406dcb0 con 0x7fad84147530 2024-04-17T10:49:23.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad81ffb640 1 -- 172.21.15.83:0/4119689453 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad6c003530 con 0x7fad84147530 2024-04-17T10:49:23.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad81ffb640 1 -- 172.21.15.83:0/4119689453 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2798267767 0 0) 0x7fad6c003530 con 0x7fad84147530 2024-04-17T10:49:23.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad81ffb640 1 -- 172.21.15.83:0/4119689453 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad840b55e0 msgr2=0x7fad840ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:23.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad81ffb640 1 --2- 172.21.15.83:0/4119689453 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad840b55e0 0x7fad840ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:23.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad81ffb640 1 -- 172.21.15.83:0/4119689453 >> v1:172.21.15.167:6789/0 conn(0x7fad8406f020 legacy=0x7fad840b4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:23.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad81ffb640 1 -- 172.21.15.83:0/4119689453 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8406e260 con 0x7fad84147530 2024-04-17T10:49:23.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad81ffb640 1 -- 172.21.15.83:0/4119689453 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fad74001680 con 0x7fad84147530 2024-04-17T10:49:23.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad81ffb640 1 -- 172.21.15.83:0/4119689453 >> v1:172.21.15.83:6789/0 conn(0x7fad84147530 legacy=0x7fad8406d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:23.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad81ffb640 1 --2- 172.21.15.83:0/4119689453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad6c0039f0 0x7fad6c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:23.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad81ffb640 1 --2- 172.21.15.83:0/4119689453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad6c0043a0 0x7fad6c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:23.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad81ffb640 1 --2- 172.21.15.83:0/4119689453 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad840b55e0 0x7fad6c005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:23.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad81ffb640 1 -- 172.21.15.83:0/4119689453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8406eb90 con 0x7fad6c0039f0 2024-04-17T10:49:23.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad81ffb640 1 -- 172.21.15.83:0/4119689453 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c00bab0 con 0x7fad6c0043a0 2024-04-17T10:49:23.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad81ffb640 1 -- 172.21.15.83:0/4119689453 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c00bcb0 con 0x7fad840b55e0 2024-04-17T10:49:23.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad82ffd640 1 --2- 172.21.15.83:0/4119689453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad6c0039f0 0x7fad6c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:23.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad827fc640 1 --2- 172.21.15.83:0/4119689453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad6c0043a0 0x7fad6c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:23.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad837fe640 1 --2- 172.21.15.83:0/4119689453 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad840b55e0 0x7fad6c005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:23.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad82ffd640 1 -- 172.21.15.83:0/4119689453 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad840b55e0 msgr2=0x7fad6c005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:23.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad82ffd640 1 --2- 172.21.15.83:0/4119689453 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad840b55e0 0x7fad6c005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:23.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad82ffd640 1 -- 172.21.15.83:0/4119689453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad6c0043a0 msgr2=0x7fad6c004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:23.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad82ffd640 1 --2- 172.21.15.83:0/4119689453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad6c0043a0 0x7fad6c004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:23.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad82ffd640 1 -- 172.21.15.83:0/4119689453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6c003530 con 0x7fad6c0039f0 2024-04-17T10:49:23.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.453+0000 7fad82ffd640 1 --2- 172.21.15.83:0/4119689453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad6c0039f0 0x7fad6c003dd0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fad7400c190 tx=0x7fad7401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e6e6867cb7dbaf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:23.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad81ffb640 1 -- 172.21.15.83:0/4119689453 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad74007cd0 con 0x7fad6c0039f0 2024-04-17T10:49:23.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad81ffb640 1 -- 172.21.15.83:0/4119689453 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad740a9070 con 0x7fad6c0039f0 2024-04-17T10:49:23.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad81ffb640 1 -- 172.21.15.83:0/4119689453 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad74005d20 con 0x7fad6c0039f0 2024-04-17T10:49:23.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad89369640 1 -- 172.21.15.83:0/4119689453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad6c0039f0 msgr2=0x7fad6c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:23.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad89369640 1 --2- 172.21.15.83:0/4119689453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad6c0039f0 0x7fad6c003dd0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fad7400c190 tx=0x7fad7401cc40 comp rx=0 tx=0).stop 2024-04-17T10:49:23.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad837fe640 1 -- 172.21.15.83:0/4119689453 reap_dead start 2024-04-17T10:49:23.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad89369640 1 -- 172.21.15.83:0/4119689453 shutdown_connections 2024-04-17T10:49:23.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad89369640 1 -- 172.21.15.83:0/4119689453 >> 172.21.15.83:0/4119689453 conn(0x7fad8406bcf0 msgr2=0x7fad8406c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:23.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad89369640 1 -- 172.21.15.83:0/4119689453 shutdown_connections 2024-04-17T10:49:23.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad89369640 1 -- 172.21.15.83:0/4119689453 wait complete. 2024-04-17T10:49:23.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad89369640 1 Processor -- start 2024-04-17T10:49:23.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad89369640 1 -- start start 2024-04-17T10:49:23.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad89369640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad84132150 0x7fad840b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:23.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad89369640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad84132530 0x7fad840b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:23.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad89369640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad840b1000 0x7fad840b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:23.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad89369640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad84072000 con 0x7fad840b1000 2024-04-17T10:49:23.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad89369640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad84071e80 con 0x7fad84132530 2024-04-17T10:49:23.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad89369640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad840ae050 con 0x7fad84132150 2024-04-17T10:49:23.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad837fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad840b1000 0x7fad840b14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:23.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad837fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad840b1000 0x7fad840b14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35640/0 (socket says 172.21.15.83:35640) 2024-04-17T10:49:23.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad837fe640 1 -- 172.21.15.83:0/356238871 learned_addr learned my addr 172.21.15.83:0/356238871 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:23.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad82ffd640 1 --2- 172.21.15.83:0/356238871 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad84132150 0x7fad840b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:23.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad827fc640 1 --2- 172.21.15.83:0/356238871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad84132530 0x7fad840b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:23.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad837fe640 1 -- 172.21.15.83:0/356238871 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad84132150 msgr2=0x7fad840b25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:23.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad837fe640 1 --2- 172.21.15.83:0/356238871 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad84132150 0x7fad840b25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:23.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad837fe640 1 -- 172.21.15.83:0/356238871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad84132530 msgr2=0x7fad840b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:23.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad837fe640 1 --2- 172.21.15.83:0/356238871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad84132530 0x7fad840b2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:23.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad837fe640 1 -- 172.21.15.83:0/356238871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad840b1da0 con 0x7fad840b1000 2024-04-17T10:49:23.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad837fe640 1 --2- 172.21.15.83:0/356238871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad840b1000 0x7fad840b14b0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fad84072ea0 tx=0x7fad7c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:23.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad6b7fe640 1 -- 172.21.15.83:0/356238871 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c011e10 con 0x7fad840b1000 2024-04-17T10:49:23.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad6b7fe640 1 -- 172.21.15.83:0/356238871 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7c01b070 con 0x7fad840b1000 2024-04-17T10:49:23.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad89369640 1 -- 172.21.15.83:0/356238871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad84161650 con 0x7fad840b1000 2024-04-17T10:49:23.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad6b7fe640 1 -- 172.21.15.83:0/356238871 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c0039a0 con 0x7fad840b1000 2024-04-17T10:49:23.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.457+0000 7fad89369640 1 -- 172.21.15.83:0/356238871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad84161c10 con 0x7fad840b1000 2024-04-17T10:49:23.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.461+0000 7fad89369640 1 -- 172.21.15.83:0/356238871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad84161e10 con 0x7fad840b1000 2024-04-17T10:49:23.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.461+0000 7fad6b7fe640 1 -- 172.21.15.83:0/356238871 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fad7c003b40 con 0x7fad840b1000 2024-04-17T10:49:23.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.461+0000 7fad6b7fe640 1 --2- 172.21.15.83:0/356238871 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fad70066f90 0x7fad70069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:23.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.461+0000 7fad82ffd640 1 --2- 172.21.15.83:0/356238871 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fad70066f90 0x7fad70069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:23.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.461+0000 7fad6b7fe640 1 -- 172.21.15.83:0/356238871 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad7c085740 con 0x7fad840b1000 2024-04-17T10:49:23.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.461+0000 7fad82ffd640 1 --2- 172.21.15.83:0/356238871 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fad70066f90 0x7fad70069450 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fad740ab000 tx=0x7fad740a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:23.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.465+0000 7fad6b7fe640 1 -- 172.21.15.83:0/356238871 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fad84161e10 con 0x7fad840b1000 2024-04-17T10:49:23.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.625+0000 7fad89369640 1 -- 172.21.15.83:0/356238871 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fad84160530 con 0x7fad70066f90 2024-04-17T10:49:23.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.637+0000 7fad6b7fe640 1 -- 172.21.15.83:0/356238871 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240257 (secure 0 0 0) 0x7fad84160530 con 0x7fad70066f90 2024-04-17T10:49:23.636 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:23.638 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:49:23.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.641+0000 7fad89369640 1 -- 172.21.15.83:0/356238871 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fad70066f90 msgr2=0x7fad70069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:23.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.641+0000 7fad89369640 1 --2- 172.21.15.83:0/356238871 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fad70066f90 0x7fad70069450 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fad740ab000 tx=0x7fad740a7070 comp rx=0 tx=0).stop 2024-04-17T10:49:23.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.641+0000 7fad89369640 1 -- 172.21.15.83:0/356238871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad840b1000 msgr2=0x7fad840b14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:23.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.641+0000 7fad89369640 1 --2- 172.21.15.83:0/356238871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad840b1000 0x7fad840b14b0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fad84072ea0 tx=0x7fad7c0115a0 comp rx=0 tx=0).stop 2024-04-17T10:49:23.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.641+0000 7fad89369640 1 -- 172.21.15.83:0/356238871 shutdown_connections 2024-04-17T10:49:23.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.641+0000 7fad89369640 1 --2- 172.21.15.83:0/356238871 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fad70066f90 0x7fad70069450 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:23.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.641+0000 7fad89369640 1 --2- 172.21.15.83:0/356238871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad840b1000 0x7fad840b14b0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:23.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.641+0000 7fad89369640 1 --2- 172.21.15.83:0/356238871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad84132530 0x7fad840b2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:23.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.641+0000 7fad89369640 1 --2- 172.21.15.83:0/356238871 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad84132150 0x7fad840b25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:23.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.641+0000 7fad89369640 1 -- 172.21.15.83:0/356238871 >> 172.21.15.83:0/356238871 conn(0x7fad8406bcf0 msgr2=0x7fad8415fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:23.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.641+0000 7fad89369640 1 -- 172.21.15.83:0/356238871 shutdown_connections 2024-04-17T10:49:23.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.641+0000 7fad89369640 1 -- 172.21.15.83:0/356238871 wait complete. 2024-04-17T10:49:23.654 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":462,"stamp":"2024-04-17T10:49:22.378709+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":518140,"kb_used_data":13256,"kb_used_omap":12,"kb_used_meta":504755,"kb_avail":754456580,"statfs":{"total":773094113280,"available":772563537920,"internally_reserved":0,"allocated":13574144,"data_stored":6560936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":516869680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001074"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.601284+0000","last_change":"2024-04-17T10:49:18.601284+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.601284+0000","last_undegraded":"2024-04-17T10:49:18.601284+0000","last_fullsized":"2024-04-17T10:49:18.601284+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.606434+0000","last_change":"2024-04-17T10:49:18.606434+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.606434+0000","last_undegraded":"2024-04-17T10:49:18.606434+0000","last_fullsized":"2024-04-17T10:49:18.606434+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.607163+0000","last_change":"2024-04-17T10:49:18.607163+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.607163+0000","last_undegraded":"2024-04-17T10:49:18.607163+0000","last_fullsized":"2024-04-17T10:49:18.607163+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.604468+0000","last_change":"2024-04-17T10:49:18.604468+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.604468+0000","last_undegraded":"2024-04-17T10:49:18.604468+0000","last_fullsized":"2024-04-17T10:49:18.604468+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.605190+0000","last_change":"2024-04-17T10:49:18.605190+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.605190+0000","last_undegraded":"2024-04-17T10:49:18.605190+0000","last_fullsized":"2024-04-17T10:49:18.605190+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.607670+0000","last_change":"2024-04-17T10:49:18.607670+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.607670+0000","last_undegraded":"2024-04-17T10:49:18.607670+0000","last_fullsized":"2024-04-17T10:49:18.607670+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.599977+0000","last_change":"2024-04-17T10:49:18.599977+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.599977+0000","last_undegraded":"2024-04-17T10:49:18.599977+0000","last_fullsized":"2024-04-17T10:49:18.599977+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.602747+0000","last_change":"2024-04-17T10:49:18.602747+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.602747+0000","last_undegraded":"2024-04-17T10:49:18.602747+0000","last_fullsized":"2024-04-17T10:49:18.602747+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.607858+0000","last_change":"2024-04-17T10:49:18.607858+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.607858+0000","last_undegraded":"2024-04-17T10:49:18.607858+0000","last_fullsized":"2024-04-17T10:49:18.607858+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.606892+0000","last_change":"2024-04-17T10:49:18.606892+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.606892+0000","last_undegraded":"2024-04-17T10:49:18.606892+0000","last_fullsized":"2024-04-17T10:49:18.606892+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.606429+0000","last_change":"2024-04-17T10:49:18.606429+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.606429+0000","last_undegraded":"2024-04-17T10:49:18.606429+0000","last_fullsized":"2024-04-17T10:49:18.606429+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.610020+0000","last_change":"2024-04-17T10:49:18.610020+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.610020+0000","last_undegraded":"2024-04-17T10:49:18.610020+0000","last_fullsized":"2024-04-17T10:49:18.610020+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.603716+0000","last_change":"2024-04-17T10:49:18.603716+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.603716+0000","last_undegraded":"2024-04-17T10:49:18.603716+0000","last_fullsized":"2024-04-17T10:49:18.603716+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.603472+0000","last_change":"2024-04-17T10:49:18.603472+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.603472+0000","last_undegraded":"2024-04-17T10:49:18.603472+0000","last_fullsized":"2024-04-17T10:49:18.603472+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.607126+0000","last_change":"2024-04-17T10:49:18.607126+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.607126+0000","last_undegraded":"2024-04-17T10:49:18.607126+0000","last_fullsized":"2024-04-17T10:49:18.607126+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.606500+0000","last_change":"2024-04-17T10:49:18.606500+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.606500+0000","last_undegraded":"2024-04-17T10:49:18.606500+0000","last_fullsized":"2024-04-17T10:49:18.606500+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.608023+0000","last_change":"2024-04-17T10:49:18.608023+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.608023+0000","last_undegraded":"2024-04-17T10:49:18.608023+0000","last_fullsized":"2024-04-17T10:49:18.608023+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.601414+0000","last_change":"2024-04-17T10:49:18.601414+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.601414+0000","last_undegraded":"2024-04-17T10:49:18.601414+0000","last_fullsized":"2024-04-17T10:49:18.601414+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.606274+0000","last_change":"2024-04-17T10:49:18.606274+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.606274+0000","last_undegraded":"2024-04-17T10:49:18.606274+0000","last_fullsized":"2024-04-17T10:49:18.606274+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.606819+0000","last_change":"2024-04-17T10:49:18.606819+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.606819+0000","last_undegraded":"2024-04-17T10:49:18.606819+0000","last_fullsized":"2024-04-17T10:49:18.606819+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.606846+0000","last_change":"2024-04-17T10:49:18.606846+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.606846+0000","last_undegraded":"2024-04-17T10:49:18.606846+0000","last_fullsized":"2024-04-17T10:49:18.606846+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.607464+0000","last_change":"2024-04-17T10:49:18.607464+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.607464+0000","last_undegraded":"2024-04-17T10:49:18.607464+0000","last_fullsized":"2024-04-17T10:49:18.607464+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.605424+0000","last_change":"2024-04-17T10:49:18.605424+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.605424+0000","last_undegraded":"2024-04-17T10:49:18.605424+0000","last_fullsized":"2024-04-17T10:49:18.605424+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.603806+0000","last_change":"2024-04-17T10:49:18.603806+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.603806+0000","last_undegraded":"2024-04-17T10:49:18.603806+0000","last_fullsized":"2024-04-17T10:49:18.603806+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.599394+0000","last_change":"2024-04-17T10:49:18.599394+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.599394+0000","last_undegraded":"2024-04-17T10:49:18.599394+0000","last_fullsized":"2024-04-17T10:49:18.599394+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.602627+0000","last_change":"2024-04-17T10:49:18.602627+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.602627+0000","last_undegraded":"2024-04-17T10:49:18.602627+0000","last_fullsized":"2024-04-17T10:49:18.602627+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.609076+0000","last_change":"2024-04-17T10:49:18.609076+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.609076+0000","last_undegraded":"2024-04-17T10:49:18.609076+0000","last_fullsized":"2024-04-17T10:49:18.609076+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.602589+0000","last_change":"2024-04-17T10:49:18.602589+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.602589+0000","last_undegraded":"2024-04-17T10:49:18.602589+0000","last_fullsized":"2024-04-17T10:49:18.602589+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.599419+0000","last_change":"2024-04-17T10:49:18.599419+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.599419+0000","last_undegraded":"2024-04-17T10:49:18.599419+0000","last_fullsized":"2024-04-17T10:49:18.599419+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.606829+0000","last_change":"2024-04-17T10:49:18.606829+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.606829+0000","last_undegraded":"2024-04-17T10:49:18.606829+0000","last_fullsized":"2024-04-17T10:49:18.606829+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.604444+0000","last_change":"2024-04-17T10:49:18.604444+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.604444+0000","last_undegraded":"2024-04-17T10:49:18.604444+0000","last_fullsized":"2024-04-17T10:49:18.604444+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.599931+0000","last_change":"2024-04-17T10:49:18.599931+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.599931+0000","last_undegraded":"2024-04-17T10:49:18.599931+0000","last_fullsized":"2024-04-17T10:49:18.599931+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":383,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-17T10:49:18.601893+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:49:18.601893+0000","last_peered":"2024-04-17T10:49:18.601893+0000","last_clean":"2024-04-17T10:49:18.601893+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:49:18.601893+0000","last_undegraded":"2024-04-17T10:49:18.601893+0000","last_fullsized":"2024-04-17T10:49:18.601893+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574988,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47160,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324680,"statfs":{"total":96636764160,"available":96588472320,"internally_reserved":0,"allocated":2142208,"data_stored":1262893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68788,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303052,"statfs":{"total":96636764160,"available":96566325248,"internally_reserved":0,"allocated":1548288,"data_stored":672525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54908,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316932,"statfs":{"total":96636764160,"available":96580538368,"internally_reserved":0,"allocated":1548288,"data_stored":672525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574987,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72948,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298892,"statfs":{"total":96636764160,"available":96562065408,"internally_reserved":0,"allocated":1548288,"data_stored":672525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72956,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298884,"statfs":{"total":96636764160,"available":96562057216,"internally_reserved":0,"allocated":1548288,"data_stored":672525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59060,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312780,"statfs":{"total":96636764160,"available":96576286720,"internally_reserved":0,"allocated":1548288,"data_stored":672525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95160,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94276680,"statfs":{"total":96636764160,"available":96539320320,"internally_reserved":0,"allocated":1548288,"data_stored":672525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574987,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47160,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324680,"statfs":{"total":96636764160,"available":96588472320,"internally_reserved":0,"allocated":2142208,"data_stored":1262893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:49:23.659 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:49:23.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.821+0000 7f0ca7355640 1 Processor -- start 2024-04-17T10:49:23.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.821+0000 7f0ca7355640 1 -- start start 2024-04-17T10:49:23.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.821+0000 7f0ca7355640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0ca0072330 0x7f0ca00cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:23.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.821+0000 7f0ca7355640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ca00cfa10 0x7f0ca006c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:23.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.821+0000 7f0ca7355640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ca00cfea0 con 0x7f0ca015ba30 2024-04-17T10:49:23.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.821+0000 7f0ca7355640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ca0071ea0 con 0x7f0ca0072330 2024-04-17T10:49:23.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.821+0000 7f0ca7355640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca006e320 con 0x7f0ca00cfa10 2024-04-17T10:49:23.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.821+0000 7f0ca58f2640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ca00cfa10 0x7f0ca006c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:23.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.821+0000 7f0ca48f0640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0ca0072330 0x7f0ca00cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:23.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.821+0000 7f0ca50f1640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f0ca015ba30 0x7f0ca015c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:39172/0 (socket says 172.21.15.83:39172) 2024-04-17T10:49:23.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.821+0000 7f0ca50f1640 1 -- 172.21.15.83:0/3436898456 learned_addr learned my addr 172.21.15.83:0/3436898456 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:23.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.821+0000 7f0ca48f0640 1 -- 172.21.15.83:0/3436898456 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ca00cfa10 msgr2=0x7f0ca006c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:23.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.821+0000 7f0ca48f0640 1 --2- 172.21.15.83:0/3436898456 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ca00cfa10 0x7f0ca006c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:23.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.821+0000 7f0ca48f0640 1 -- 172.21.15.83:0/3436898456 >> v1:172.21.15.167:6789/0 conn(0x7f0ca015ba30 legacy=0x7f0ca015c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:23.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.821+0000 7f0ca48f0640 1 -- 172.21.15.83:0/3436898456 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca006e040 con 0x7f0ca0072330 2024-04-17T10:49:23.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.821+0000 7f0ca58f2640 1 --2- 172.21.15.83:0/3436898456 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ca00cfa10 0x7f0ca006c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:49:23.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.825+0000 7f0ca48f0640 1 --2- 172.21.15.83:0/3436898456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ca0072330 0x7f0ca00cf4d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0072020 tx=0x7f0c9401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25c082a16c7f1135 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:23.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.825+0000 7f0c9ffff640 1 -- 172.21.15.83:0/3436898456 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c940b0070 con 0x7f0ca0072330 2024-04-17T10:49:23.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.825+0000 7f0c9ffff640 1 -- 172.21.15.83:0/3436898456 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c940ab600 con 0x7f0ca0072330 2024-04-17T10:49:23.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.825+0000 7f0c9ffff640 1 -- 172.21.15.83:0/3436898456 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c940ab9d0 con 0x7f0ca0072330 2024-04-17T10:49:23.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.825+0000 7f0ca7355640 1 -- 172.21.15.83:0/3436898456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ca0072330 msgr2=0x7f0ca00cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:23.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.825+0000 7f0ca7355640 1 --2- 172.21.15.83:0/3436898456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ca0072330 0x7f0ca00cf4d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0072020 tx=0x7f0c9401c410 comp rx=0 tx=0).stop 2024-04-17T10:49:23.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.825+0000 7f0ca7355640 1 -- 172.21.15.83:0/3436898456 shutdown_connections 2024-04-17T10:49:23.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.825+0000 7f0ca7355640 1 --2- 172.21.15.83:0/3436898456 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ca00cfa10 0x7f0ca006c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:23.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.825+0000 7f0ca7355640 1 --2- 172.21.15.83:0/3436898456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ca0072330 0x7f0ca00cf4d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:23.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.825+0000 7f0ca7355640 1 -- 172.21.15.83:0/3436898456 >> 172.21.15.83:0/3436898456 conn(0x7f0ca006abe0 msgr2=0x7f0ca006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:23.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.825+0000 7f0ca7355640 1 -- 172.21.15.83:0/3436898456 shutdown_connections 2024-04-17T10:49:23.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.825+0000 7f0ca7355640 1 -- 172.21.15.83:0/3436898456 wait complete. 2024-04-17T10:49:23.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.825+0000 7f0ca7355640 1 Processor -- start 2024-04-17T10:49:23.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.825+0000 7f0ca7355640 1 -- start start 2024-04-17T10:49:23.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.825+0000 7f0ca7355640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca0072330 0x7f0ca015eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:23.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.825+0000 7f0ca7355640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ca00cfa10 0x7f0ca01597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:23.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.825+0000 7f0ca7355640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ca015ba30 0x7f0ca0148180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:23.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.825+0000 7f0ca7355640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca006de20 con 0x7f0ca0072330 2024-04-17T10:49:23.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.825+0000 7f0ca7355640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca006dca0 con 0x7f0ca00cfa10 2024-04-17T10:49:23.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.825+0000 7f0ca7355640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca0168520 con 0x7f0ca015ba30 2024-04-17T10:49:23.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.825+0000 7f0ca50f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca0072330 0x7f0ca015eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:23.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.825+0000 7f0ca50f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca0072330 0x7f0ca015eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35644/0 (socket says 172.21.15.83:35644) 2024-04-17T10:49:23.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.825+0000 7f0ca50f1640 1 -- 172.21.15.83:0/4049797011 learned_addr learned my addr 172.21.15.83:0/4049797011 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.829+0000 7f0ca48f0640 1 --2- 172.21.15.83:0/4049797011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ca00cfa10 0x7f0ca01597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.829+0000 7f0ca58f2640 1 --2- 172.21.15.83:0/4049797011 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ca015ba30 0x7f0ca0148180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.829+0000 7f0ca50f1640 1 -- 172.21.15.83:0/4049797011 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ca015ba30 msgr2=0x7f0ca0148180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.829+0000 7f0ca50f1640 1 --2- 172.21.15.83:0/4049797011 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ca015ba30 0x7f0ca0148180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.829+0000 7f0ca50f1640 1 -- 172.21.15.83:0/4049797011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ca00cfa10 msgr2=0x7f0ca01597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.829+0000 7f0ca50f1640 1 --2- 172.21.15.83:0/4049797011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ca00cfa10 0x7f0ca01597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.829+0000 7f0ca50f1640 1 -- 172.21.15.83:0/4049797011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca015de20 con 0x7f0ca0072330 2024-04-17T10:49:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.829+0000 7f0ca50f1640 1 --2- 172.21.15.83:0/4049797011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca0072330 0x7f0ca015eb80 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f0c9800c450 tx=0x7f0c9800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.829+0000 7f0c9dffb640 1 -- 172.21.15.83:0/4049797011 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c98017070 con 0x7f0ca0072330 2024-04-17T10:49:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.829+0000 7f0c9dffb640 1 -- 172.21.15.83:0/4049797011 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c98005cf0 con 0x7f0ca0072330 2024-04-17T10:49:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.829+0000 7f0c9dffb640 1 -- 172.21.15.83:0/4049797011 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c98007290 con 0x7f0ca0072330 2024-04-17T10:49:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.829+0000 7f0ca7355640 1 -- 172.21.15.83:0/4049797011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca0160170 con 0x7f0ca0072330 2024-04-17T10:49:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.829+0000 7f0ca7355640 1 -- 172.21.15.83:0/4049797011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca015e380 con 0x7f0ca0072330 2024-04-17T10:49:23.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.829+0000 7f0ca7355640 1 -- 172.21.15.83:0/4049797011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ca0074f90 con 0x7f0ca0072330 2024-04-17T10:49:23.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.829+0000 7f0c9dffb640 1 -- 172.21.15.83:0/4049797011 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0c9801a070 con 0x7f0ca0072330 2024-04-17T10:49:23.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.829+0000 7f0c9dffb640 1 --2- 172.21.15.83:0/4049797011 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0c8c066f60 0x7f0c8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:23.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.829+0000 7f0c9dffb640 1 -- 172.21.15.83:0/4049797011 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c98084760 con 0x7f0ca0072330 2024-04-17T10:49:23.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.829+0000 7f0ca48f0640 1 --2- 172.21.15.83:0/4049797011 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0c8c066f60 0x7f0c8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:23.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.829+0000 7f0ca48f0640 1 --2- 172.21.15.83:0/4049797011 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0c8c066f60 0x7f0c8c069420 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f0c940ad000 tx=0x7f0c94008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:23.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.833+0000 7f0c9dffb640 1 -- 172.21.15.83:0/4049797011 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0c98021020 con 0x7f0ca0072330 2024-04-17T10:49:23.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:23.989+0000 7f0ca7355640 1 -- 172.21.15.83:0/4049797011 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0ca015b130 con 0x7f0c8c066f60 2024-04-17T10:49:23.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.001+0000 7f0c9dffb640 1 -- 172.21.15.83:0/4049797011 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240257 (secure 0 0 0) 0x7f0ca015b130 con 0x7f0c8c066f60 2024-04-17T10:49:23.998 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:24.000 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:49:24.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.005+0000 7f0ca7355640 1 -- 172.21.15.83:0/4049797011 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0c8c066f60 msgr2=0x7f0c8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:24.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.005+0000 7f0ca7355640 1 --2- 172.21.15.83:0/4049797011 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0c8c066f60 0x7f0c8c069420 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f0c940ad000 tx=0x7f0c94008040 comp rx=0 tx=0).stop 2024-04-17T10:49:24.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.005+0000 7f0ca7355640 1 -- 172.21.15.83:0/4049797011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca0072330 msgr2=0x7f0ca015eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:24.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.005+0000 7f0ca7355640 1 --2- 172.21.15.83:0/4049797011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca0072330 0x7f0ca015eb80 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f0c9800c450 tx=0x7f0c9800c910 comp rx=0 tx=0).stop 2024-04-17T10:49:24.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.005+0000 7f0ca7355640 1 -- 172.21.15.83:0/4049797011 shutdown_connections 2024-04-17T10:49:24.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.005+0000 7f0ca7355640 1 --2- 172.21.15.83:0/4049797011 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0c8c066f60 0x7f0c8c069420 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:24.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.005+0000 7f0ca7355640 1 --2- 172.21.15.83:0/4049797011 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ca015ba30 0x7f0ca0148180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:24.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.005+0000 7f0ca7355640 1 --2- 172.21.15.83:0/4049797011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ca00cfa10 0x7f0ca01597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:24.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.005+0000 7f0ca7355640 1 --2- 172.21.15.83:0/4049797011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca0072330 0x7f0ca015eb80 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:24.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.005+0000 7f0ca7355640 1 -- 172.21.15.83:0/4049797011 >> 172.21.15.83:0/4049797011 conn(0x7f0ca006abe0 msgr2=0x7f0ca015a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:24.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.005+0000 7f0ca7355640 1 -- 172.21.15.83:0/4049797011 shutdown_connections 2024-04-17T10:49:24.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.005+0000 7f0ca7355640 1 -- 172.21.15.83:0/4049797011 wait complete. 2024-04-17T10:49:24.015 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":462,"stamp":"2024-04-17T10:49:22.378709+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":518140,"kb_used_data":13256,"kb_used_omap":12,"kb_used_meta":504755,"kb_avail":754456580,"statfs":{"total":773094113280,"available":772563537920,"internally_reserved":0,"allocated":13574144,"data_stored":6560936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":516869680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001074"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.601284+0000","last_change":"2024-04-17T10:49:18.601284+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.601284+0000","last_undegraded":"2024-04-17T10:49:18.601284+0000","last_fullsized":"2024-04-17T10:49:18.601284+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.606434+0000","last_change":"2024-04-17T10:49:18.606434+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.606434+0000","last_undegraded":"2024-04-17T10:49:18.606434+0000","last_fullsized":"2024-04-17T10:49:18.606434+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.607163+0000","last_change":"2024-04-17T10:49:18.607163+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.607163+0000","last_undegraded":"2024-04-17T10:49:18.607163+0000","last_fullsized":"2024-04-17T10:49:18.607163+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.604468+0000","last_change":"2024-04-17T10:49:18.604468+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.604468+0000","last_undegraded":"2024-04-17T10:49:18.604468+0000","last_fullsized":"2024-04-17T10:49:18.604468+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.605190+0000","last_change":"2024-04-17T10:49:18.605190+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.605190+0000","last_undegraded":"2024-04-17T10:49:18.605190+0000","last_fullsized":"2024-04-17T10:49:18.605190+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.607670+0000","last_change":"2024-04-17T10:49:18.607670+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.607670+0000","last_undegraded":"2024-04-17T10:49:18.607670+0000","last_fullsized":"2024-04-17T10:49:18.607670+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.599977+0000","last_change":"2024-04-17T10:49:18.599977+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.599977+0000","last_undegraded":"2024-04-17T10:49:18.599977+0000","last_fullsized":"2024-04-17T10:49:18.599977+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.602747+0000","last_change":"2024-04-17T10:49:18.602747+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.602747+0000","last_undegraded":"2024-04-17T10:49:18.602747+0000","last_fullsized":"2024-04-17T10:49:18.602747+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.607858+0000","last_change":"2024-04-17T10:49:18.607858+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.607858+0000","last_undegraded":"2024-04-17T10:49:18.607858+0000","last_fullsized":"2024-04-17T10:49:18.607858+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.606892+0000","last_change":"2024-04-17T10:49:18.606892+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.606892+0000","last_undegraded":"2024-04-17T10:49:18.606892+0000","last_fullsized":"2024-04-17T10:49:18.606892+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.606429+0000","last_change":"2024-04-17T10:49:18.606429+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.606429+0000","last_undegraded":"2024-04-17T10:49:18.606429+0000","last_fullsized":"2024-04-17T10:49:18.606429+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.610020+0000","last_change":"2024-04-17T10:49:18.610020+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.610020+0000","last_undegraded":"2024-04-17T10:49:18.610020+0000","last_fullsized":"2024-04-17T10:49:18.610020+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.603716+0000","last_change":"2024-04-17T10:49:18.603716+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.603716+0000","last_undegraded":"2024-04-17T10:49:18.603716+0000","last_fullsized":"2024-04-17T10:49:18.603716+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.603472+0000","last_change":"2024-04-17T10:49:18.603472+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.603472+0000","last_undegraded":"2024-04-17T10:49:18.603472+0000","last_fullsized":"2024-04-17T10:49:18.603472+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.607126+0000","last_change":"2024-04-17T10:49:18.607126+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.607126+0000","last_undegraded":"2024-04-17T10:49:18.607126+0000","last_fullsized":"2024-04-17T10:49:18.607126+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.606500+0000","last_change":"2024-04-17T10:49:18.606500+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.606500+0000","last_undegraded":"2024-04-17T10:49:18.606500+0000","last_fullsized":"2024-04-17T10:49:18.606500+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.608023+0000","last_change":"2024-04-17T10:49:18.608023+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.608023+0000","last_undegraded":"2024-04-17T10:49:18.608023+0000","last_fullsized":"2024-04-17T10:49:18.608023+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.601414+0000","last_change":"2024-04-17T10:49:18.601414+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.601414+0000","last_undegraded":"2024-04-17T10:49:18.601414+0000","last_fullsized":"2024-04-17T10:49:18.601414+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.606274+0000","last_change":"2024-04-17T10:49:18.606274+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.606274+0000","last_undegraded":"2024-04-17T10:49:18.606274+0000","last_fullsized":"2024-04-17T10:49:18.606274+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.606819+0000","last_change":"2024-04-17T10:49:18.606819+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.606819+0000","last_undegraded":"2024-04-17T10:49:18.606819+0000","last_fullsized":"2024-04-17T10:49:18.606819+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.606846+0000","last_change":"2024-04-17T10:49:18.606846+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.606846+0000","last_undegraded":"2024-04-17T10:49:18.606846+0000","last_fullsized":"2024-04-17T10:49:18.606846+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.607464+0000","last_change":"2024-04-17T10:49:18.607464+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.607464+0000","last_undegraded":"2024-04-17T10:49:18.607464+0000","last_fullsized":"2024-04-17T10:49:18.607464+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.605424+0000","last_change":"2024-04-17T10:49:18.605424+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.605424+0000","last_undegraded":"2024-04-17T10:49:18.605424+0000","last_fullsized":"2024-04-17T10:49:18.605424+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.603806+0000","last_change":"2024-04-17T10:49:18.603806+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.603806+0000","last_undegraded":"2024-04-17T10:49:18.603806+0000","last_fullsized":"2024-04-17T10:49:18.603806+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.599394+0000","last_change":"2024-04-17T10:49:18.599394+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.599394+0000","last_undegraded":"2024-04-17T10:49:18.599394+0000","last_fullsized":"2024-04-17T10:49:18.599394+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.602627+0000","last_change":"2024-04-17T10:49:18.602627+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.602627+0000","last_undegraded":"2024-04-17T10:49:18.602627+0000","last_fullsized":"2024-04-17T10:49:18.602627+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.609076+0000","last_change":"2024-04-17T10:49:18.609076+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.609076+0000","last_undegraded":"2024-04-17T10:49:18.609076+0000","last_fullsized":"2024-04-17T10:49:18.609076+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.602589+0000","last_change":"2024-04-17T10:49:18.602589+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.602589+0000","last_undegraded":"2024-04-17T10:49:18.602589+0000","last_fullsized":"2024-04-17T10:49:18.602589+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.599419+0000","last_change":"2024-04-17T10:49:18.599419+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.599419+0000","last_undegraded":"2024-04-17T10:49:18.599419+0000","last_fullsized":"2024-04-17T10:49:18.599419+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.606829+0000","last_change":"2024-04-17T10:49:18.606829+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.606829+0000","last_undegraded":"2024-04-17T10:49:18.606829+0000","last_fullsized":"2024-04-17T10:49:18.606829+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.604444+0000","last_change":"2024-04-17T10:49:18.604444+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.604444+0000","last_undegraded":"2024-04-17T10:49:18.604444+0000","last_fullsized":"2024-04-17T10:49:18.604444+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.599931+0000","last_change":"2024-04-17T10:49:18.599931+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.599931+0000","last_undegraded":"2024-04-17T10:49:18.599931+0000","last_fullsized":"2024-04-17T10:49:18.599931+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":383,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-17T10:49:18.601893+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:49:18.601893+0000","last_peered":"2024-04-17T10:49:18.601893+0000","last_clean":"2024-04-17T10:49:18.601893+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:49:18.601893+0000","last_undegraded":"2024-04-17T10:49:18.601893+0000","last_fullsized":"2024-04-17T10:49:18.601893+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574988,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47160,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324680,"statfs":{"total":96636764160,"available":96588472320,"internally_reserved":0,"allocated":2142208,"data_stored":1262893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68788,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303052,"statfs":{"total":96636764160,"available":96566325248,"internally_reserved":0,"allocated":1548288,"data_stored":672525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54908,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316932,"statfs":{"total":96636764160,"available":96580538368,"internally_reserved":0,"allocated":1548288,"data_stored":672525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574987,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72948,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298892,"statfs":{"total":96636764160,"available":96562065408,"internally_reserved":0,"allocated":1548288,"data_stored":672525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72956,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298884,"statfs":{"total":96636764160,"available":96562057216,"internally_reserved":0,"allocated":1548288,"data_stored":672525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59060,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312780,"statfs":{"total":96636764160,"available":96576286720,"internally_reserved":0,"allocated":1548288,"data_stored":672525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95160,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94276680,"statfs":{"total":96636764160,"available":96539320320,"internally_reserved":0,"allocated":1548288,"data_stored":672525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574987,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47160,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324680,"statfs":{"total":96636764160,"available":96588472320,"internally_reserved":0,"allocated":2142208,"data_stored":1262893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:49:24.018 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:49:24.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.185+0000 7f0c9ea71640 1 Processor -- start 2024-04-17T10:49:24.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.185+0000 7f0c9ea71640 1 -- start start 2024-04-17T10:49:24.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.185+0000 7f0c9ea71640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0c9815db50 0x7f0c98070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:24.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.185+0000 7f0c9ea71640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0c980aedd0 0x7f0c9806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:24.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.185+0000 7f0c9ea71640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c980709b0 con 0x7f0c9806dae0 2024-04-17T10:49:24.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.185+0000 7f0c9ea71640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c9815e440 con 0x7f0c980aedd0 2024-04-17T10:49:24.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.185+0000 7f0c9ea71640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c9815e720 con 0x7f0c9815db50 2024-04-17T10:49:24.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.185+0000 7f0c97fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0c980aedd0 0x7f0c9806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:24.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.185+0000 7f0c97fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0c980aedd0 0x7f0c9806d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35648/0 (socket says 172.21.15.83:35648) 2024-04-17T10:49:24.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.185+0000 7f0c97fff640 1 -- 172.21.15.83:0/287862747 learned_addr learned my addr 172.21.15.83:0/287862747 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:24.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.185+0000 7f0c9c80d640 1 --2- 172.21.15.83:0/287862747 >> v2:172.21.15.167:3300/0 conn(0x7f0c9815db50 0x7f0c98070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:24.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.189+0000 7f0c97fff640 1 -- 172.21.15.83:0/287862747 >> v2:172.21.15.167:3300/0 conn(0x7f0c9815db50 msgr2=0x7f0c98070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:24.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.189+0000 7f0c97fff640 1 --2- 172.21.15.83:0/287862747 >> v2:172.21.15.167:3300/0 conn(0x7f0c9815db50 0x7f0c98070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:24.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.189+0000 7f0c97fff640 1 -- 172.21.15.83:0/287862747 >> v1:172.21.15.167:6789/0 conn(0x7f0c9806dae0 legacy=0x7f0c980cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:24.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.189+0000 7f0c97fff640 1 -- 172.21.15.83:0/287862747 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c9815ea00 con 0x7f0c980aedd0 2024-04-17T10:49:24.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.189+0000 7f0c9c80d640 1 --2- 172.21.15.83:0/287862747 >> v2:172.21.15.167:3300/0 conn(0x7f0c9815db50 0x7f0c98070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:24.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.189+0000 7f0c97fff640 1 --2- 172.21.15.83:0/287862747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c980aedd0 0x7f0c9806d5a0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c009870 tx=0x7f0c8c01c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd692bbc31d0dfd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:24.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.189+0000 7f0c977fe640 1 -- 172.21.15.83:0/287862747 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c0b0070 con 0x7f0c980aedd0 2024-04-17T10:49:24.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.189+0000 7f0c977fe640 1 -- 172.21.15.83:0/287862747 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c8c0ab600 con 0x7f0c980aedd0 2024-04-17T10:49:24.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.189+0000 7f0c977fe640 1 -- 172.21.15.83:0/287862747 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c0ab9d0 con 0x7f0c980aedd0 2024-04-17T10:49:24.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.189+0000 7f0c9ea71640 1 -- 172.21.15.83:0/287862747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c980aedd0 msgr2=0x7f0c9806d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:24.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.189+0000 7f0c9ea71640 1 --2- 172.21.15.83:0/287862747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c980aedd0 0x7f0c9806d5a0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c009870 tx=0x7f0c8c01c300 comp rx=0 tx=0).stop 2024-04-17T10:49:24.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.189+0000 7f0c9ea71640 1 -- 172.21.15.83:0/287862747 shutdown_connections 2024-04-17T10:49:24.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.189+0000 7f0c9ea71640 1 --2- 172.21.15.83:0/287862747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c980aedd0 0x7f0c9806d5a0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:24.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.189+0000 7f0c9ea71640 1 --2- 172.21.15.83:0/287862747 >> v2:172.21.15.167:3300/0 conn(0x7f0c9815db50 0x7f0c98070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:24.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.189+0000 7f0c9ea71640 1 -- 172.21.15.83:0/287862747 >> 172.21.15.83:0/287862747 conn(0x7f0c9806abe0 msgr2=0x7f0c9806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:24.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.189+0000 7f0c9ea71640 1 -- 172.21.15.83:0/287862747 shutdown_connections 2024-04-17T10:49:24.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.189+0000 7f0c9ea71640 1 -- 172.21.15.83:0/287862747 wait complete. 2024-04-17T10:49:24.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.193+0000 7f0c9ea71640 1 Processor -- start 2024-04-17T10:49:24.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.193+0000 7f0c9ea71640 1 -- start start 2024-04-17T10:49:24.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.193+0000 7f0c9ea71640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c9806dae0 0x7f0c98155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:24.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.193+0000 7f0c9ea71640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c980aedd0 0x7f0c981421c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:24.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.193+0000 7f0c9ea71640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c9815db50 0x7f0c980cd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:24.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.193+0000 7f0c9ea71640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c981677e0 con 0x7f0c9806dae0 2024-04-17T10:49:24.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.193+0000 7f0c9ea71640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c98167660 con 0x7f0c980aedd0 2024-04-17T10:49:24.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.193+0000 7f0c9ea71640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c98167960 con 0x7f0c9815db50 2024-04-17T10:49:24.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.193+0000 7f0c9c80d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c9806dae0 0x7f0c98155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:24.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.193+0000 7f0c9c80d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c9806dae0 0x7f0c98155740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35664/0 (socket says 172.21.15.83:35664) 2024-04-17T10:49:24.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.193+0000 7f0c9c80d640 1 -- 172.21.15.83:0/2017082038 learned_addr learned my addr 172.21.15.83:0/2017082038 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:24.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.193+0000 7f0c9d00e640 1 --2- 172.21.15.83:0/2017082038 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c9815db50 0x7f0c980cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:24.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.193+0000 7f0c97fff640 1 --2- 172.21.15.83:0/2017082038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c980aedd0 0x7f0c981421c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:24.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.193+0000 7f0c9c80d640 1 -- 172.21.15.83:0/2017082038 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c9815db50 msgr2=0x7f0c980cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:24.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.193+0000 7f0c9c80d640 1 --2- 172.21.15.83:0/2017082038 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c9815db50 0x7f0c980cd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:24.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.193+0000 7f0c9c80d640 1 -- 172.21.15.83:0/2017082038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c980aedd0 msgr2=0x7f0c981421c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:24.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.193+0000 7f0c9c80d640 1 --2- 172.21.15.83:0/2017082038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c980aedd0 0x7f0c981421c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:24.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.193+0000 7f0c9c80d640 1 -- 172.21.15.83:0/2017082038 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c981545a0 con 0x7f0c9806dae0 2024-04-17T10:49:24.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.193+0000 7f0c9d00e640 1 --2- 172.21.15.83:0/2017082038 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c9815db50 0x7f0c980cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:24.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.193+0000 7f0c97fff640 1 --2- 172.21.15.83:0/2017082038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c980aedd0 0x7f0c981421c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:24.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.193+0000 7f0c9c80d640 1 --2- 172.21.15.83:0/2017082038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c9806dae0 0x7f0c98155740 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800c7f0 tx=0x7f0c8800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:24.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.193+0000 7f0c957fa640 1 -- 172.21.15.83:0/2017082038 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c88015070 con 0x7f0c9806dae0 2024-04-17T10:49:24.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.193+0000 7f0c957fa640 1 -- 172.21.15.83:0/2017082038 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c88007820 con 0x7f0c9806dae0 2024-04-17T10:49:24.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.193+0000 7f0c9ea71640 1 -- 172.21.15.83:0/2017082038 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c98153500 con 0x7f0c9806dae0 2024-04-17T10:49:24.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.193+0000 7f0c957fa640 1 -- 172.21.15.83:0/2017082038 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c88007c10 con 0x7f0c9806dae0 2024-04-17T10:49:24.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.193+0000 7f0c9ea71640 1 -- 172.21.15.83:0/2017082038 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c98153ac0 con 0x7f0c9806dae0 2024-04-17T10:49:24.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.193+0000 7f0c9ea71640 1 -- 172.21.15.83:0/2017082038 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c98074f90 con 0x7f0c9806dae0 2024-04-17T10:49:24.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.193+0000 7f0c957fa640 1 -- 172.21.15.83:0/2017082038 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0c88018070 con 0x7f0c9806dae0 2024-04-17T10:49:24.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.197+0000 7f0c957fa640 1 --2- 172.21.15.83:0/2017082038 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0c84066f60 0x7f0c84069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:24.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.197+0000 7f0c97fff640 1 --2- 172.21.15.83:0/2017082038 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0c84066f60 0x7f0c84069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:24.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.197+0000 7f0c957fa640 1 -- 172.21.15.83:0/2017082038 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c88085140 con 0x7f0c9806dae0 2024-04-17T10:49:24.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.197+0000 7f0c97fff640 1 --2- 172.21.15.83:0/2017082038 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0c84066f60 0x7f0c84069420 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c0ad000 tx=0x7f0c8c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:24.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.201+0000 7f0c957fa640 1 -- 172.21.15.83:0/2017082038 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0c88056150 con 0x7f0c9806dae0 2024-04-17T10:49:24.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.353+0000 7f0c9ea71640 1 -- 172.21.15.83:0/2017082038 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0c980b2af0 con 0x7f0c84066f60 2024-04-17T10:49:24.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.369+0000 7f0c957fa640 1 -- 172.21.15.83:0/2017082038 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240257 (secure 0 0 0) 0x7f0c980b2af0 con 0x7f0c84066f60 2024-04-17T10:49:24.365 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:24.367 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:49:24.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.373+0000 7f0c9ea71640 1 -- 172.21.15.83:0/2017082038 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0c84066f60 msgr2=0x7f0c84069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:24.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.373+0000 7f0c9ea71640 1 --2- 172.21.15.83:0/2017082038 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0c84066f60 0x7f0c84069420 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c0ad000 tx=0x7f0c8c008040 comp rx=0 tx=0).stop 2024-04-17T10:49:24.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.373+0000 7f0c9ea71640 1 -- 172.21.15.83:0/2017082038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c9806dae0 msgr2=0x7f0c98155740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:24.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.373+0000 7f0c9ea71640 1 --2- 172.21.15.83:0/2017082038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c9806dae0 0x7f0c98155740 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800c7f0 tx=0x7f0c8800ccb0 comp rx=0 tx=0).stop 2024-04-17T10:49:24.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.373+0000 7f0c9ea71640 1 -- 172.21.15.83:0/2017082038 shutdown_connections 2024-04-17T10:49:24.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.373+0000 7f0c9ea71640 1 --2- 172.21.15.83:0/2017082038 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0c84066f60 0x7f0c84069420 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:24.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.373+0000 7f0c9ea71640 1 --2- 172.21.15.83:0/2017082038 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c9815db50 0x7f0c980cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:24.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.373+0000 7f0c9ea71640 1 --2- 172.21.15.83:0/2017082038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c980aedd0 0x7f0c981421c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:24.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.373+0000 7f0c9ea71640 1 --2- 172.21.15.83:0/2017082038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c9806dae0 0x7f0c98155740 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:24.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.373+0000 7f0c9ea71640 1 -- 172.21.15.83:0/2017082038 >> 172.21.15.83:0/2017082038 conn(0x7f0c9806abe0 msgr2=0x7f0c980af1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:24.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.373+0000 7f0c9ea71640 1 -- 172.21.15.83:0/2017082038 shutdown_connections 2024-04-17T10:49:24.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.373+0000 7f0c9ea71640 1 -- 172.21.15.83:0/2017082038 wait complete. 2024-04-17T10:49:24.384 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":462,"stamp":"2024-04-17T10:49:22.378709+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":518140,"kb_used_data":13256,"kb_used_omap":12,"kb_used_meta":504755,"kb_avail":754456580,"statfs":{"total":773094113280,"available":772563537920,"internally_reserved":0,"allocated":13574144,"data_stored":6560936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":516869680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001074"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.601284+0000","last_change":"2024-04-17T10:49:18.601284+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.601284+0000","last_undegraded":"2024-04-17T10:49:18.601284+0000","last_fullsized":"2024-04-17T10:49:18.601284+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.606434+0000","last_change":"2024-04-17T10:49:18.606434+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.606434+0000","last_undegraded":"2024-04-17T10:49:18.606434+0000","last_fullsized":"2024-04-17T10:49:18.606434+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.607163+0000","last_change":"2024-04-17T10:49:18.607163+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.607163+0000","last_undegraded":"2024-04-17T10:49:18.607163+0000","last_fullsized":"2024-04-17T10:49:18.607163+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.604468+0000","last_change":"2024-04-17T10:49:18.604468+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.604468+0000","last_undegraded":"2024-04-17T10:49:18.604468+0000","last_fullsized":"2024-04-17T10:49:18.604468+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.605190+0000","last_change":"2024-04-17T10:49:18.605190+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.605190+0000","last_undegraded":"2024-04-17T10:49:18.605190+0000","last_fullsized":"2024-04-17T10:49:18.605190+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.607670+0000","last_change":"2024-04-17T10:49:18.607670+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.607670+0000","last_undegraded":"2024-04-17T10:49:18.607670+0000","last_fullsized":"2024-04-17T10:49:18.607670+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.599977+0000","last_change":"2024-04-17T10:49:18.599977+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.599977+0000","last_undegraded":"2024-04-17T10:49:18.599977+0000","last_fullsized":"2024-04-17T10:49:18.599977+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.602747+0000","last_change":"2024-04-17T10:49:18.602747+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.602747+0000","last_undegraded":"2024-04-17T10:49:18.602747+0000","last_fullsized":"2024-04-17T10:49:18.602747+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.607858+0000","last_change":"2024-04-17T10:49:18.607858+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.607858+0000","last_undegraded":"2024-04-17T10:49:18.607858+0000","last_fullsized":"2024-04-17T10:49:18.607858+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.606892+0000","last_change":"2024-04-17T10:49:18.606892+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.606892+0000","last_undegraded":"2024-04-17T10:49:18.606892+0000","last_fullsized":"2024-04-17T10:49:18.606892+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.606429+0000","last_change":"2024-04-17T10:49:18.606429+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.606429+0000","last_undegraded":"2024-04-17T10:49:18.606429+0000","last_fullsized":"2024-04-17T10:49:18.606429+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.610020+0000","last_change":"2024-04-17T10:49:18.610020+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.610020+0000","last_undegraded":"2024-04-17T10:49:18.610020+0000","last_fullsized":"2024-04-17T10:49:18.610020+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.603716+0000","last_change":"2024-04-17T10:49:18.603716+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.603716+0000","last_undegraded":"2024-04-17T10:49:18.603716+0000","last_fullsized":"2024-04-17T10:49:18.603716+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.603472+0000","last_change":"2024-04-17T10:49:18.603472+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.603472+0000","last_undegraded":"2024-04-17T10:49:18.603472+0000","last_fullsized":"2024-04-17T10:49:18.603472+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.607126+0000","last_change":"2024-04-17T10:49:18.607126+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.607126+0000","last_undegraded":"2024-04-17T10:49:18.607126+0000","last_fullsized":"2024-04-17T10:49:18.607126+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.606500+0000","last_change":"2024-04-17T10:49:18.606500+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.606500+0000","last_undegraded":"2024-04-17T10:49:18.606500+0000","last_fullsized":"2024-04-17T10:49:18.606500+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.608023+0000","last_change":"2024-04-17T10:49:18.608023+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.608023+0000","last_undegraded":"2024-04-17T10:49:18.608023+0000","last_fullsized":"2024-04-17T10:49:18.608023+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.601414+0000","last_change":"2024-04-17T10:49:18.601414+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.601414+0000","last_undegraded":"2024-04-17T10:49:18.601414+0000","last_fullsized":"2024-04-17T10:49:18.601414+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.606274+0000","last_change":"2024-04-17T10:49:18.606274+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.606274+0000","last_undegraded":"2024-04-17T10:49:18.606274+0000","last_fullsized":"2024-04-17T10:49:18.606274+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.606819+0000","last_change":"2024-04-17T10:49:18.606819+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.606819+0000","last_undegraded":"2024-04-17T10:49:18.606819+0000","last_fullsized":"2024-04-17T10:49:18.606819+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.606846+0000","last_change":"2024-04-17T10:49:18.606846+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.606846+0000","last_undegraded":"2024-04-17T10:49:18.606846+0000","last_fullsized":"2024-04-17T10:49:18.606846+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.607464+0000","last_change":"2024-04-17T10:49:18.607464+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.607464+0000","last_undegraded":"2024-04-17T10:49:18.607464+0000","last_fullsized":"2024-04-17T10:49:18.607464+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.605424+0000","last_change":"2024-04-17T10:49:18.605424+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.605424+0000","last_undegraded":"2024-04-17T10:49:18.605424+0000","last_fullsized":"2024-04-17T10:49:18.605424+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.603806+0000","last_change":"2024-04-17T10:49:18.603806+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.603806+0000","last_undegraded":"2024-04-17T10:49:18.603806+0000","last_fullsized":"2024-04-17T10:49:18.603806+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.599394+0000","last_change":"2024-04-17T10:49:18.599394+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.599394+0000","last_undegraded":"2024-04-17T10:49:18.599394+0000","last_fullsized":"2024-04-17T10:49:18.599394+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.602627+0000","last_change":"2024-04-17T10:49:18.602627+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.602627+0000","last_undegraded":"2024-04-17T10:49:18.602627+0000","last_fullsized":"2024-04-17T10:49:18.602627+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.609076+0000","last_change":"2024-04-17T10:49:18.609076+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.609076+0000","last_undegraded":"2024-04-17T10:49:18.609076+0000","last_fullsized":"2024-04-17T10:49:18.609076+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.602589+0000","last_change":"2024-04-17T10:49:18.602589+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.602589+0000","last_undegraded":"2024-04-17T10:49:18.602589+0000","last_fullsized":"2024-04-17T10:49:18.602589+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.599419+0000","last_change":"2024-04-17T10:49:18.599419+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.599419+0000","last_undegraded":"2024-04-17T10:49:18.599419+0000","last_fullsized":"2024-04-17T10:49:18.599419+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.606829+0000","last_change":"2024-04-17T10:49:18.606829+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.606829+0000","last_undegraded":"2024-04-17T10:49:18.606829+0000","last_fullsized":"2024-04-17T10:49:18.606829+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.604444+0000","last_change":"2024-04-17T10:49:18.604444+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.604444+0000","last_undegraded":"2024-04-17T10:49:18.604444+0000","last_fullsized":"2024-04-17T10:49:18.604444+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.599931+0000","last_change":"2024-04-17T10:49:18.599931+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.599931+0000","last_undegraded":"2024-04-17T10:49:18.599931+0000","last_fullsized":"2024-04-17T10:49:18.599931+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":383,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-17T10:49:18.601893+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:49:18.601893+0000","last_peered":"2024-04-17T10:49:18.601893+0000","last_clean":"2024-04-17T10:49:18.601893+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:49:18.601893+0000","last_undegraded":"2024-04-17T10:49:18.601893+0000","last_fullsized":"2024-04-17T10:49:18.601893+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574988,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47160,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324680,"statfs":{"total":96636764160,"available":96588472320,"internally_reserved":0,"allocated":2142208,"data_stored":1262893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68788,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303052,"statfs":{"total":96636764160,"available":96566325248,"internally_reserved":0,"allocated":1548288,"data_stored":672525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54908,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316932,"statfs":{"total":96636764160,"available":96580538368,"internally_reserved":0,"allocated":1548288,"data_stored":672525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574987,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72948,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298892,"statfs":{"total":96636764160,"available":96562065408,"internally_reserved":0,"allocated":1548288,"data_stored":672525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72956,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298884,"statfs":{"total":96636764160,"available":96562057216,"internally_reserved":0,"allocated":1548288,"data_stored":672525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59060,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312780,"statfs":{"total":96636764160,"available":96576286720,"internally_reserved":0,"allocated":1548288,"data_stored":672525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95160,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94276680,"statfs":{"total":96636764160,"available":96539320320,"internally_reserved":0,"allocated":1548288,"data_stored":672525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574987,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47160,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324680,"statfs":{"total":96636764160,"available":96588472320,"internally_reserved":0,"allocated":2142208,"data_stored":1262893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:49:24.387 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3f is not active+clean 2024-04-17T10:49:24.387 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.388 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3e is not active+clean 2024-04-17T10:49:24.388 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.388 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3d is not active+clean 2024-04-17T10:49:24.388 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.388 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3c is not active+clean 2024-04-17T10:49:24.388 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.388 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3b is not active+clean 2024-04-17T10:49:24.388 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.388 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3a is not active+clean 2024-04-17T10:49:24.388 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.389 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.39 is not active+clean 2024-04-17T10:49:24.389 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.389 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.38 is not active+clean 2024-04-17T10:49:24.389 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.389 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.37 is not active+clean 2024-04-17T10:49:24.389 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.389 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.36 is not active+clean 2024-04-17T10:49:24.389 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.389 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.35 is not active+clean 2024-04-17T10:49:24.389 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.390 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.34 is not active+clean 2024-04-17T10:49:24.390 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.390 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.33 is not active+clean 2024-04-17T10:49:24.390 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.390 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.32 is not active+clean 2024-04-17T10:49:24.390 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.390 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.31 is not active+clean 2024-04-17T10:49:24.390 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.390 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.30 is not active+clean 2024-04-17T10:49:24.390 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.391 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2f is not active+clean 2024-04-17T10:49:24.391 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.391 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2e is not active+clean 2024-04-17T10:49:24.391 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.391 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2d is not active+clean 2024-04-17T10:49:24.391 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.391 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2c is not active+clean 2024-04-17T10:49:24.391 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.391 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2b is not active+clean 2024-04-17T10:49:24.391 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.392 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2a is not active+clean 2024-04-17T10:49:24.392 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.392 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.29 is not active+clean 2024-04-17T10:49:24.392 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.392 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.28 is not active+clean 2024-04-17T10:49:24.392 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.392 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.27 is not active+clean 2024-04-17T10:49:24.392 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.392 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.26 is not active+clean 2024-04-17T10:49:24.392 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.25 is not active+clean 2024-04-17T10:49:24.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.24 is not active+clean 2024-04-17T10:49:24.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.23 is not active+clean 2024-04-17T10:49:24.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.22 is not active+clean 2024-04-17T10:49:24.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.21 is not active+clean 2024-04-17T10:49:24.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.20 is not active+clean 2024-04-17T10:49:24.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1e is not active+clean 2024-04-17T10:49:24.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1f is not active+clean 2024-04-17T10:49:24.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:49:18.601284+0000', 'last_change': '2024-04-17T10:49:18.601284+0000', 'last_active': '2024-04-17T10:49:18.586291+0000', 'last_peered': '2024-04-17T10:49:18.586291+0000', 'last_clean': '2024-04-17T10:49:18.586291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:18.601284+0000', 'last_undegraded': '2024-04-17T10:49:18.601284+0000', 'last_fullsized': '2024-04-17T10:49:18.601284+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:49:24.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1f is not active+clean 2024-04-17T10:49:24.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1e is not active+clean 2024-04-17T10:49:24.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:49:18.606434+0000', 'last_change': '2024-04-17T10:49:18.606434+0000', 'last_active': '2024-04-17T10:49:18.586291+0000', 'last_peered': '2024-04-17T10:49:18.586291+0000', 'last_clean': '2024-04-17T10:49:18.586291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:18.606434+0000', 'last_undegraded': '2024-04-17T10:49:18.606434+0000', 'last_fullsized': '2024-04-17T10:49:18.606434+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:49:24.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1c is not active+clean 2024-04-17T10:49:24.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1d is not active+clean 2024-04-17T10:49:24.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:49:18.607163+0000', 'last_change': '2024-04-17T10:49:18.607163+0000', 'last_active': '2024-04-17T10:49:18.586291+0000', 'last_peered': '2024-04-17T10:49:18.586291+0000', 'last_clean': '2024-04-17T10:49:18.586291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:18.607163+0000', 'last_undegraded': '2024-04-17T10:49:18.607163+0000', 'last_fullsized': '2024-04-17T10:49:18.607163+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:49:24.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1d is not active+clean 2024-04-17T10:49:24.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1c is not active+clean 2024-04-17T10:49:24.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:49:18.604468+0000', 'last_change': '2024-04-17T10:49:18.604468+0000', 'last_active': '2024-04-17T10:49:18.586291+0000', 'last_peered': '2024-04-17T10:49:18.586291+0000', 'last_clean': '2024-04-17T10:49:18.586291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:18.604468+0000', 'last_undegraded': '2024-04-17T10:49:18.604468+0000', 'last_fullsized': '2024-04-17T10:49:18.604468+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:49:24.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1a is not active+clean 2024-04-17T10:49:24.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1b is not active+clean 2024-04-17T10:49:24.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:49:18.605190+0000', 'last_change': '2024-04-17T10:49:18.605190+0000', 'last_active': '2024-04-17T10:49:18.586291+0000', 'last_peered': '2024-04-17T10:49:18.586291+0000', 'last_clean': '2024-04-17T10:49:18.586291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:18.605190+0000', 'last_undegraded': '2024-04-17T10:49:18.605190+0000', 'last_fullsized': '2024-04-17T10:49:18.605190+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:49:24.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1b is not active+clean 2024-04-17T10:49:24.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1a is not active+clean 2024-04-17T10:49:24.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:49:18.607670+0000', 'last_change': '2024-04-17T10:49:18.607670+0000', 'last_active': '2024-04-17T10:49:18.586291+0000', 'last_peered': '2024-04-17T10:49:18.586291+0000', 'last_clean': '2024-04-17T10:49:18.586291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:18.607670+0000', 'last_undegraded': '2024-04-17T10:49:18.607670+0000', 'last_fullsized': '2024-04-17T10:49:18.607670+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:49:24.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.18 is not active+clean 2024-04-17T10:49:24.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.19 is not active+clean 2024-04-17T10:49:24.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:49:18.599977+0000', 'last_change': '2024-04-17T10:49:18.599977+0000', 'last_active': '2024-04-17T10:49:18.586291+0000', 'last_peered': '2024-04-17T10:49:18.586291+0000', 'last_clean': '2024-04-17T10:49:18.586291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:18.599977+0000', 'last_undegraded': '2024-04-17T10:49:18.599977+0000', 'last_fullsized': '2024-04-17T10:49:18.599977+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:49:24.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.19 is not active+clean 2024-04-17T10:49:24.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.18 is not active+clean 2024-04-17T10:49:24.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:49:18.602747+0000', 'last_change': '2024-04-17T10:49:18.602747+0000', 'last_active': '2024-04-17T10:49:18.586291+0000', 'last_peered': '2024-04-17T10:49:18.586291+0000', 'last_clean': '2024-04-17T10:49:18.586291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:18.602747+0000', 'last_undegraded': '2024-04-17T10:49:18.602747+0000', 'last_fullsized': '2024-04-17T10:49:18.602747+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:49:24.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.16 is not active+clean 2024-04-17T10:49:24.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.17 is not active+clean 2024-04-17T10:49:24.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:49:18.607858+0000', 'last_change': '2024-04-17T10:49:18.607858+0000', 'last_active': '2024-04-17T10:49:18.586291+0000', 'last_peered': '2024-04-17T10:49:18.586291+0000', 'last_clean': '2024-04-17T10:49:18.586291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:18.607858+0000', 'last_undegraded': '2024-04-17T10:49:18.607858+0000', 'last_fullsized': '2024-04-17T10:49:18.607858+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:49:24.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.17 is not active+clean 2024-04-17T10:49:24.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.16 is not active+clean 2024-04-17T10:49:24.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:49:18.606892+0000', 'last_change': '2024-04-17T10:49:18.606892+0000', 'last_active': '2024-04-17T10:49:18.586291+0000', 'last_peered': '2024-04-17T10:49:18.586291+0000', 'last_clean': '2024-04-17T10:49:18.586291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:18.606892+0000', 'last_undegraded': '2024-04-17T10:49:18.606892+0000', 'last_fullsized': '2024-04-17T10:49:18.606892+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:49:24.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.14 is not active+clean 2024-04-17T10:49:24.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.15 is not active+clean 2024-04-17T10:49:24.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:49:18.606429+0000', 'last_change': '2024-04-17T10:49:18.606429+0000', 'last_active': '2024-04-17T10:49:18.586291+0000', 'last_peered': '2024-04-17T10:49:18.586291+0000', 'last_clean': '2024-04-17T10:49:18.586291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:18.606429+0000', 'last_undegraded': '2024-04-17T10:49:18.606429+0000', 'last_fullsized': '2024-04-17T10:49:18.606429+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:49:24.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.15 is not active+clean 2024-04-17T10:49:24.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.14 is not active+clean 2024-04-17T10:49:24.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:49:18.610020+0000', 'last_change': '2024-04-17T10:49:18.610020+0000', 'last_active': '2024-04-17T10:49:18.586291+0000', 'last_peered': '2024-04-17T10:49:18.586291+0000', 'last_clean': '2024-04-17T10:49:18.586291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:18.610020+0000', 'last_undegraded': '2024-04-17T10:49:18.610020+0000', 'last_fullsized': '2024-04-17T10:49:18.610020+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:49:24.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.13 is not active+clean 2024-04-17T10:49:24.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.12 is not active+clean 2024-04-17T10:49:24.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:49:18.603716+0000', 'last_change': '2024-04-17T10:49:18.603716+0000', 'last_active': '2024-04-17T10:49:18.586291+0000', 'last_peered': '2024-04-17T10:49:18.586291+0000', 'last_clean': '2024-04-17T10:49:18.586291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:18.603716+0000', 'last_undegraded': '2024-04-17T10:49:18.603716+0000', 'last_fullsized': '2024-04-17T10:49:18.603716+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:49:24.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.10 is not active+clean 2024-04-17T10:49:24.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.11 is not active+clean 2024-04-17T10:49:24.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:49:18.603472+0000', 'last_change': '2024-04-17T10:49:18.603472+0000', 'last_active': '2024-04-17T10:49:18.586291+0000', 'last_peered': '2024-04-17T10:49:18.586291+0000', 'last_clean': '2024-04-17T10:49:18.586291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:18.603472+0000', 'last_undegraded': '2024-04-17T10:49:18.603472+0000', 'last_fullsized': '2024-04-17T10:49:18.603472+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:49:24.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.11 is not active+clean 2024-04-17T10:49:24.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.10 is not active+clean 2024-04-17T10:49:24.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:49:18.607126+0000', 'last_change': '2024-04-17T10:49:18.607126+0000', 'last_active': '2024-04-17T10:49:18.586291+0000', 'last_peered': '2024-04-17T10:49:18.586291+0000', 'last_clean': '2024-04-17T10:49:18.586291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:18.607126+0000', 'last_undegraded': '2024-04-17T10:49:18.607126+0000', 'last_fullsized': '2024-04-17T10:49:18.607126+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:49:24.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.e is not active+clean 2024-04-17T10:49:24.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.f is not active+clean 2024-04-17T10:49:24.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:49:18.606500+0000', 'last_change': '2024-04-17T10:49:18.606500+0000', 'last_active': '2024-04-17T10:49:18.586291+0000', 'last_peered': '2024-04-17T10:49:18.586291+0000', 'last_clean': '2024-04-17T10:49:18.586291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:18.606500+0000', 'last_undegraded': '2024-04-17T10:49:18.606500+0000', 'last_fullsized': '2024-04-17T10:49:18.606500+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:49:24.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.f is not active+clean 2024-04-17T10:49:24.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.e is not active+clean 2024-04-17T10:49:24.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:49:18.608023+0000', 'last_change': '2024-04-17T10:49:18.608023+0000', 'last_active': '2024-04-17T10:49:18.586291+0000', 'last_peered': '2024-04-17T10:49:18.586291+0000', 'last_clean': '2024-04-17T10:49:18.586291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:18.608023+0000', 'last_undegraded': '2024-04-17T10:49:18.608023+0000', 'last_fullsized': '2024-04-17T10:49:18.608023+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:49:24.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.c is not active+clean 2024-04-17T10:49:24.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.d is not active+clean 2024-04-17T10:49:24.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:49:18.601414+0000', 'last_change': '2024-04-17T10:49:18.601414+0000', 'last_active': '2024-04-17T10:49:18.586291+0000', 'last_peered': '2024-04-17T10:49:18.586291+0000', 'last_clean': '2024-04-17T10:49:18.586291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:18.601414+0000', 'last_undegraded': '2024-04-17T10:49:18.601414+0000', 'last_fullsized': '2024-04-17T10:49:18.601414+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:49:24.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.d is not active+clean 2024-04-17T10:49:24.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.c is not active+clean 2024-04-17T10:49:24.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:49:18.606274+0000', 'last_change': '2024-04-17T10:49:18.606274+0000', 'last_active': '2024-04-17T10:49:18.586291+0000', 'last_peered': '2024-04-17T10:49:18.586291+0000', 'last_clean': '2024-04-17T10:49:18.586291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:18.606274+0000', 'last_undegraded': '2024-04-17T10:49:18.606274+0000', 'last_fullsized': '2024-04-17T10:49:18.606274+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:49:24.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.a is not active+clean 2024-04-17T10:49:24.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.b is not active+clean 2024-04-17T10:49:24.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:49:18.606819+0000', 'last_change': '2024-04-17T10:49:18.606819+0000', 'last_active': '2024-04-17T10:49:18.586291+0000', 'last_peered': '2024-04-17T10:49:18.586291+0000', 'last_clean': '2024-04-17T10:49:18.586291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:18.606819+0000', 'last_undegraded': '2024-04-17T10:49:18.606819+0000', 'last_fullsized': '2024-04-17T10:49:18.606819+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:49:24.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.b is not active+clean 2024-04-17T10:49:24.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.a is not active+clean 2024-04-17T10:49:24.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:49:18.606846+0000', 'last_change': '2024-04-17T10:49:18.606846+0000', 'last_active': '2024-04-17T10:49:18.586291+0000', 'last_peered': '2024-04-17T10:49:18.586291+0000', 'last_clean': '2024-04-17T10:49:18.586291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:18.606846+0000', 'last_undegraded': '2024-04-17T10:49:18.606846+0000', 'last_fullsized': '2024-04-17T10:49:18.606846+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:49:24.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.8 is not active+clean 2024-04-17T10:49:24.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.9 is not active+clean 2024-04-17T10:49:24.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:49:18.607464+0000', 'last_change': '2024-04-17T10:49:18.607464+0000', 'last_active': '2024-04-17T10:49:18.586291+0000', 'last_peered': '2024-04-17T10:49:18.586291+0000', 'last_clean': '2024-04-17T10:49:18.586291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:18.607464+0000', 'last_undegraded': '2024-04-17T10:49:18.607464+0000', 'last_fullsized': '2024-04-17T10:49:18.607464+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:49:24.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.9 is not active+clean 2024-04-17T10:49:24.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.8 is not active+clean 2024-04-17T10:49:24.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:49:18.605424+0000', 'last_change': '2024-04-17T10:49:18.605424+0000', 'last_active': '2024-04-17T10:49:18.586291+0000', 'last_peered': '2024-04-17T10:49:18.586291+0000', 'last_clean': '2024-04-17T10:49:18.586291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:18.605424+0000', 'last_undegraded': '2024-04-17T10:49:18.605424+0000', 'last_fullsized': '2024-04-17T10:49:18.605424+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:49:24.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.6 is not active+clean 2024-04-17T10:49:24.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.7 is not active+clean 2024-04-17T10:49:24.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:49:18.603806+0000', 'last_change': '2024-04-17T10:49:18.603806+0000', 'last_active': '2024-04-17T10:49:18.586291+0000', 'last_peered': '2024-04-17T10:49:18.586291+0000', 'last_clean': '2024-04-17T10:49:18.586291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:18.603806+0000', 'last_undegraded': '2024-04-17T10:49:18.603806+0000', 'last_fullsized': '2024-04-17T10:49:18.603806+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:49:24.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.7 is not active+clean 2024-04-17T10:49:24.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.6 is not active+clean 2024-04-17T10:49:24.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:49:18.599394+0000', 'last_change': '2024-04-17T10:49:18.599394+0000', 'last_active': '2024-04-17T10:49:18.586291+0000', 'last_peered': '2024-04-17T10:49:18.586291+0000', 'last_clean': '2024-04-17T10:49:18.586291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:18.599394+0000', 'last_undegraded': '2024-04-17T10:49:18.599394+0000', 'last_fullsized': '2024-04-17T10:49:18.599394+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:49:24.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.4 is not active+clean 2024-04-17T10:49:24.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.5 is not active+clean 2024-04-17T10:49:24.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:49:18.602627+0000', 'last_change': '2024-04-17T10:49:18.602627+0000', 'last_active': '2024-04-17T10:49:18.586291+0000', 'last_peered': '2024-04-17T10:49:18.586291+0000', 'last_clean': '2024-04-17T10:49:18.586291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:18.602627+0000', 'last_undegraded': '2024-04-17T10:49:18.602627+0000', 'last_fullsized': '2024-04-17T10:49:18.602627+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:49:24.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.5 is not active+clean 2024-04-17T10:49:24.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.4 is not active+clean 2024-04-17T10:49:24.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:49:18.609076+0000', 'last_change': '2024-04-17T10:49:18.609076+0000', 'last_active': '2024-04-17T10:49:18.586291+0000', 'last_peered': '2024-04-17T10:49:18.586291+0000', 'last_clean': '2024-04-17T10:49:18.586291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:18.609076+0000', 'last_undegraded': '2024-04-17T10:49:18.609076+0000', 'last_fullsized': '2024-04-17T10:49:18.609076+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:49:24.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2 is not active+clean 2024-04-17T10:49:24.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3 is not active+clean 2024-04-17T10:49:24.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:49:18.602589+0000', 'last_change': '2024-04-17T10:49:18.602589+0000', 'last_active': '2024-04-17T10:49:18.586291+0000', 'last_peered': '2024-04-17T10:49:18.586291+0000', 'last_clean': '2024-04-17T10:49:18.586291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:18.602589+0000', 'last_undegraded': '2024-04-17T10:49:18.602589+0000', 'last_fullsized': '2024-04-17T10:49:18.602589+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:49:24.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3 is not active+clean 2024-04-17T10:49:24.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2 is not active+clean 2024-04-17T10:49:24.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:49:18.599419+0000', 'last_change': '2024-04-17T10:49:18.599419+0000', 'last_active': '2024-04-17T10:49:18.586291+0000', 'last_peered': '2024-04-17T10:49:18.586291+0000', 'last_clean': '2024-04-17T10:49:18.586291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:18.599419+0000', 'last_undegraded': '2024-04-17T10:49:18.599419+0000', 'last_fullsized': '2024-04-17T10:49:18.599419+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:49:24.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.0 is not active+clean 2024-04-17T10:49:24.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1 is not active+clean 2024-04-17T10:49:24.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:49:18.606829+0000', 'last_change': '2024-04-17T10:49:18.606829+0000', 'last_active': '2024-04-17T10:49:18.586291+0000', 'last_peered': '2024-04-17T10:49:18.586291+0000', 'last_clean': '2024-04-17T10:49:18.586291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:18.606829+0000', 'last_undegraded': '2024-04-17T10:49:18.606829+0000', 'last_fullsized': '2024-04-17T10:49:18.606829+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:49:24.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1 is not active+clean 2024-04-17T10:49:24.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.0 is not active+clean 2024-04-17T10:49:24.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:49:18.604444+0000', 'last_change': '2024-04-17T10:49:18.604444+0000', 'last_active': '2024-04-17T10:49:18.586291+0000', 'last_peered': '2024-04-17T10:49:18.586291+0000', 'last_clean': '2024-04-17T10:49:18.586291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:18.604444+0000', 'last_undegraded': '2024-04-17T10:49:18.604444+0000', 'last_fullsized': '2024-04-17T10:49:18.604444+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:49:24.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.12 is not active+clean 2024-04-17T10:49:24.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T10:49:19.595335+0000', 'last_change': '2024-04-17T10:49:19.595335+0000', 'last_active': '2024-04-17T10:49:19.595335+0000', 'last_peered': '2024-04-17T10:49:19.595335+0000', 'last_clean': '2024-04-17T10:49:19.595335+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:19.595335+0000', 'last_undegraded': '2024-04-17T10:49:19.595335+0000', 'last_fullsized': '2024-04-17T10:49:19.595335+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:19.595335+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T10:49:24.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.13 is not active+clean 2024-04-17T10:49:24.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:49:18.599931+0000', 'last_change': '2024-04-17T10:49:18.599931+0000', 'last_active': '2024-04-17T10:49:18.586291+0000', 'last_peered': '2024-04-17T10:49:18.586291+0000', 'last_clean': '2024-04-17T10:49:18.586291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:49:18.599931+0000', 'last_undegraded': '2024-04-17T10:49:18.599931+0000', 'last_fullsized': '2024-04-17T10:49:18.599931+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'last_clean_scrub_stamp': '2024-04-17T10:49:18.586291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:49:24.407 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-17T10:49:24.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.557+0000 7f9f8adba640 1 Processor -- start 2024-04-17T10:49:24.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.557+0000 7f9f8adba640 1 -- start start 2024-04-17T10:49:24.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.561+0000 7f9f8adba640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9f840b3990 0x7f9f84070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:24.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.561+0000 7f9f8adba640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9f840709b0 0x7f9f8406c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:24.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.561+0000 7f9f8adba640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f84070dc0 con 0x7f9f840cf060 2024-04-17T10:49:24.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.561+0000 7f9f8adba640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f840b2980 con 0x7f9f840709b0 2024-04-17T10:49:24.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.561+0000 7f9f8adba640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f840cf440 con 0x7f9f840b3990 2024-04-17T10:49:24.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.561+0000 7f9f89357640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9f840709b0 0x7f9f8406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:24.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.561+0000 7f9f89357640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9f840709b0 0x7f9f8406c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35670/0 (socket says 172.21.15.83:35670) 2024-04-17T10:49:24.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.561+0000 7f9f89357640 1 -- 172.21.15.83:0/2695702382 learned_addr learned my addr 172.21.15.83:0/2695702382 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:24.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.561+0000 7f9f7bfff640 1 --2- 172.21.15.83:0/2695702382 >> v2:172.21.15.167:3300/0 conn(0x7f9f840b3990 0x7f9f84070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:24.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.561+0000 7f9f89357640 1 -- 172.21.15.83:0/2695702382 >> v2:172.21.15.167:3300/0 conn(0x7f9f840b3990 msgr2=0x7f9f84070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:24.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.561+0000 7f9f89357640 1 --2- 172.21.15.83:0/2695702382 >> v2:172.21.15.167:3300/0 conn(0x7f9f840b3990 0x7f9f84070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:24.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.561+0000 7f9f89357640 1 -- 172.21.15.83:0/2695702382 >> v1:172.21.15.167:6789/0 conn(0x7f9f840cf060 legacy=0x7f9f840b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:24.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.561+0000 7f9f89357640 1 -- 172.21.15.83:0/2695702382 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f840b2bd0 con 0x7f9f840709b0 2024-04-17T10:49:24.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.561+0000 7f9f89357640 1 --2- 172.21.15.83:0/2695702382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f840709b0 0x7f9f8406c490 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f9f840b2380 tx=0x7f9f7c01c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da5d9384018ca0a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:24.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.561+0000 7f9f7b7fe640 1 -- 172.21.15.83:0/2695702382 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c0a5660 con 0x7f9f840709b0 2024-04-17T10:49:24.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.561+0000 7f9f7b7fe640 1 -- 172.21.15.83:0/2695702382 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f7c0a5800 con 0x7f9f840709b0 2024-04-17T10:49:24.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.561+0000 7f9f7b7fe640 1 -- 172.21.15.83:0/2695702382 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c0a5bd0 con 0x7f9f840709b0 2024-04-17T10:49:24.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f8adba640 1 -- 172.21.15.83:0/2695702382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f840709b0 msgr2=0x7f9f8406c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:24.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f8adba640 1 --2- 172.21.15.83:0/2695702382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f840709b0 0x7f9f8406c490 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f9f840b2380 tx=0x7f9f7c01c7a0 comp rx=0 tx=0).stop 2024-04-17T10:49:24.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f8adba640 1 -- 172.21.15.83:0/2695702382 shutdown_connections 2024-04-17T10:49:24.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f8adba640 1 --2- 172.21.15.83:0/2695702382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f840709b0 0x7f9f8406c490 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:24.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f8adba640 1 --2- 172.21.15.83:0/2695702382 >> v2:172.21.15.167:3300/0 conn(0x7f9f840b3990 0x7f9f84070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:24.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f8adba640 1 -- 172.21.15.83:0/2695702382 >> 172.21.15.83:0/2695702382 conn(0x7f9f8406abe0 msgr2=0x7f9f8406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:24.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f8adba640 1 -- 172.21.15.83:0/2695702382 shutdown_connections 2024-04-17T10:49:24.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f8adba640 1 -- 172.21.15.83:0/2695702382 wait complete. 2024-04-17T10:49:24.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f8adba640 1 Processor -- start 2024-04-17T10:49:24.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f8adba640 1 -- start start 2024-04-17T10:49:24.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f8adba640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f840709b0 0x7f9f84163910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:24.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f8adba640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f840b3990 0x7f9f84151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:24.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f8adba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f840cf060 0x7f9f841590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:24.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f8adba640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f840b4770 con 0x7f9f840cf060 2024-04-17T10:49:24.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f8adba640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8415b310 con 0x7f9f840709b0 2024-04-17T10:49:24.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f8adba640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f840cf5c0 con 0x7f9f840b3990 2024-04-17T10:49:24.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f89357640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f840cf060 0x7f9f841590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:24.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f89357640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f840cf060 0x7f9f841590c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35686/0 (socket says 172.21.15.83:35686) 2024-04-17T10:49:24.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f89357640 1 -- 172.21.15.83:0/3395966219 learned_addr learned my addr 172.21.15.83:0/3395966219 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:24.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f7bfff640 1 --2- 172.21.15.83:0/3395966219 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f840b3990 0x7f9f84151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:24.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f88b56640 1 --2- 172.21.15.83:0/3395966219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f840709b0 0x7f9f84163910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:24.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f89357640 1 -- 172.21.15.83:0/3395966219 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f840b3990 msgr2=0x7f9f84151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:24.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f89357640 1 --2- 172.21.15.83:0/3395966219 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f840b3990 0x7f9f84151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:24.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f89357640 1 -- 172.21.15.83:0/3395966219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f840709b0 msgr2=0x7f9f84163910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:24.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f89357640 1 --2- 172.21.15.83:0/3395966219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f840709b0 0x7f9f84163910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:24.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f89357640 1 -- 172.21.15.83:0/3395966219 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f841628b0 con 0x7f9f840cf060 2024-04-17T10:49:24.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f88b56640 1 --2- 172.21.15.83:0/3395966219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f840709b0 0x7f9f84163910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:24.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f89357640 1 --2- 172.21.15.83:0/3395966219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f840cf060 0x7f9f841590c0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c01a3d0 tx=0x7f9f7c019540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:24.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f797fa640 1 -- 172.21.15.83:0/3395966219 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c009510 con 0x7f9f840cf060 2024-04-17T10:49:24.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f797fa640 1 -- 172.21.15.83:0/3395966219 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f7c0be070 con 0x7f9f840cf060 2024-04-17T10:49:24.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f8adba640 1 -- 172.21.15.83:0/3395966219 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f84162b50 con 0x7f9f840cf060 2024-04-17T10:49:24.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f797fa640 1 -- 172.21.15.83:0/3395966219 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c0b88d0 con 0x7f9f840cf060 2024-04-17T10:49:24.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.565+0000 7f9f8adba640 1 -- 172.21.15.83:0/3395966219 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f84163110 con 0x7f9f840cf060 2024-04-17T10:49:24.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.569+0000 7f9f8adba640 1 -- 172.21.15.83:0/3395966219 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f84074f90 con 0x7f9f840cf060 2024-04-17T10:49:24.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.569+0000 7f9f797fa640 1 -- 172.21.15.83:0/3395966219 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9f8406d1e0 con 0x7f9f840cf060 2024-04-17T10:49:24.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.569+0000 7f9f797fa640 1 --2- 172.21.15.83:0/3395966219 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9f70066f60 0x7f9f70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:24.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.569+0000 7f9f797fa640 1 -- 172.21.15.83:0/3395966219 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f7c11ffa0 con 0x7f9f840cf060 2024-04-17T10:49:24.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.569+0000 7f9f88b56640 1 --2- 172.21.15.83:0/3395966219 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9f70066f60 0x7f9f70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:24.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.569+0000 7f9f88b56640 1 --2- 172.21.15.83:0/3395966219 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9f70066f60 0x7f9f70069420 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000a850 tx=0x7f9f80008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:24.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.573+0000 7f9f797fa640 1 -- 172.21.15.83:0/3395966219 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9f84070dc0 con 0x7f9f840cf060 2024-04-17T10:49:24.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.789+0000 7f9f8adba640 1 -- 172.21.15.83:0/3395966219 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f9f8406c760 con 0x7f9f840cf060 2024-04-17T10:49:24.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.789+0000 7f9f797fa640 1 -- 172.21.15.83:0/3395966219 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1530 (secure 0 0 0) 0x7f9f7c0f10d0 con 0x7f9f840cf060 2024-04-17T10:49:24.784 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:24.784 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":697,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":127,"num_osds":8,"num_up_osds":8,"osd_up_since":1713350279,"num_in_osds":8,"osd_in_since":1713350270,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":530575360,"bytes_avail":772563537920,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":76,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":9198}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-04-17T10:49:08.374763+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-17T10:49:24.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.793+0000 7f9f8adba640 1 -- 172.21.15.83:0/3395966219 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9f70066f60 msgr2=0x7f9f70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:24.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.793+0000 7f9f8adba640 1 --2- 172.21.15.83:0/3395966219 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9f70066f60 0x7f9f70069420 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000a850 tx=0x7f9f80008040 comp rx=0 tx=0).stop 2024-04-17T10:49:24.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.793+0000 7f9f8adba640 1 -- 172.21.15.83:0/3395966219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f840cf060 msgr2=0x7f9f841590c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:24.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.793+0000 7f9f8adba640 1 --2- 172.21.15.83:0/3395966219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f840cf060 0x7f9f841590c0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c01a3d0 tx=0x7f9f7c019540 comp rx=0 tx=0).stop 2024-04-17T10:49:24.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.793+0000 7f9f8adba640 1 -- 172.21.15.83:0/3395966219 shutdown_connections 2024-04-17T10:49:24.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.793+0000 7f9f8adba640 1 --2- 172.21.15.83:0/3395966219 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9f70066f60 0x7f9f70069420 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:24.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.793+0000 7f9f8adba640 1 --2- 172.21.15.83:0/3395966219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f840cf060 0x7f9f841590c0 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:24.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.793+0000 7f9f8adba640 1 --2- 172.21.15.83:0/3395966219 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f840b3990 0x7f9f84151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:24.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.793+0000 7f9f8adba640 1 --2- 172.21.15.83:0/3395966219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f840709b0 0x7f9f84163910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:24.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.793+0000 7f9f8adba640 1 -- 172.21.15.83:0/3395966219 >> 172.21.15.83:0/3395966219 conn(0x7f9f8406abe0 msgr2=0x7f9f840b3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:24.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.793+0000 7f9f8adba640 1 -- 172.21.15.83:0/3395966219 shutdown_connections 2024-04-17T10:49:24.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.793+0000 7f9f8adba640 1 -- 172.21.15.83:0/3395966219 wait complete. 2024-04-17T10:49:24.802 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-17T10:49:24.802 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:49:24.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.929+0000 7f4643973640 1 Processor -- start 2024-04-17T10:49:24.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.929+0000 7f4643973640 1 -- start start 2024-04-17T10:49:24.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.933+0000 7f4643973640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f463c0b0ce0 0x7f463c0cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:24.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.933+0000 7f4643973640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f463c0cf0a0 con 0x7f463c0c5140 2024-04-17T10:49:24.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.933+0000 7f4643973640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f463c0cf380 con 0x7f463c0b1e50 2024-04-17T10:49:24.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.933+0000 7f4643973640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f463c0b2c40 con 0x7f463c0b0ce0 2024-04-17T10:49:24.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.933+0000 7f4641f10640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f463c0c5140 0x7f463c165d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41430/0 (socket says 172.21.15.83:41430) 2024-04-17T10:49:24.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.933+0000 7f4641f10640 1 -- 172.21.15.83:0/2758461529 learned_addr learned my addr 172.21.15.83:0/2758461529 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:24.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.933+0000 7f464170f640 1 --2- 172.21.15.83:0/2758461529 >> v2:172.21.15.167:3300/0 conn(0x7f463c0b0ce0 0x7f463c0cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:24.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.933+0000 7f462ffff640 1 -- 172.21.15.83:0/2758461529 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2188064079 0 0) 0x7f463c0cf0a0 con 0x7f463c0c5140 2024-04-17T10:49:24.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.933+0000 7f462ffff640 1 -- 172.21.15.83:0/2758461529 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4628003530 con 0x7f463c0c5140 2024-04-17T10:49:24.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.933+0000 7f462ffff640 1 -- 172.21.15.83:0/2758461529 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1815894556 0 0) 0x7f4628003530 con 0x7f463c0c5140 2024-04-17T10:49:24.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.933+0000 7f462ffff640 1 -- 172.21.15.83:0/2758461529 >> v2:172.21.15.167:3300/0 conn(0x7f463c0b0ce0 msgr2=0x7f463c0cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:24.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.933+0000 7f462ffff640 1 --2- 172.21.15.83:0/2758461529 >> v2:172.21.15.167:3300/0 conn(0x7f463c0b0ce0 0x7f463c0cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:24.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.933+0000 7f462ffff640 1 -- 172.21.15.83:0/2758461529 >> v1:172.21.15.167:6789/0 conn(0x7f463c0b1e50 legacy=0x7f463c0c4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:24.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.933+0000 7f462ffff640 1 -- 172.21.15.83:0/2758461529 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f463c0b2f20 con 0x7f463c0c5140 2024-04-17T10:49:24.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.933+0000 7f462ffff640 1 -- 172.21.15.83:0/2758461529 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f4638001680 con 0x7f463c0c5140 2024-04-17T10:49:24.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.933+0000 7f462ffff640 1 -- 172.21.15.83:0/2758461529 >> v1:172.21.15.83:6789/0 conn(0x7f463c0c5140 legacy=0x7f463c165d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:24.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.933+0000 7f462ffff640 1 --2- 172.21.15.83:0/2758461529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46280039c0 0x7f4628003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:24.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.933+0000 7f462ffff640 1 --2- 172.21.15.83:0/2758461529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4628004810 0x7f4628004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:24.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.933+0000 7f462ffff640 1 --2- 172.21.15.83:0/2758461529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46280051d0 0x7f462800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:24.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.933+0000 7f462ffff640 1 -- 172.21.15.83:0/2758461529 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463c0b2c40 con 0x7f46280039c0 2024-04-17T10:49:24.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.933+0000 7f462ffff640 1 -- 172.21.15.83:0/2758461529 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462800c170 con 0x7f46280051d0 2024-04-17T10:49:24.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.933+0000 7f462ffff640 1 -- 172.21.15.83:0/2758461529 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462800c370 con 0x7f4628004810 2024-04-17T10:49:24.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.933+0000 7f464170f640 1 --2- 172.21.15.83:0/2758461529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46280039c0 0x7f4628003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:24.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.933+0000 7f4641f10640 1 --2- 172.21.15.83:0/2758461529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46280051d0 0x7f462800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:24.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.933+0000 7f4640f0e640 1 --2- 172.21.15.83:0/2758461529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4628004810 0x7f4628004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:24.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.933+0000 7f464170f640 1 -- 172.21.15.83:0/2758461529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4628004810 msgr2=0x7f4628004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:24.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.933+0000 7f464170f640 1 --2- 172.21.15.83:0/2758461529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4628004810 0x7f4628004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:24.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.933+0000 7f464170f640 1 -- 172.21.15.83:0/2758461529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46280051d0 msgr2=0x7f462800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:24.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.933+0000 7f464170f640 1 --2- 172.21.15.83:0/2758461529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46280051d0 0x7f462800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:24.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.933+0000 7f464170f640 1 -- 172.21.15.83:0/2758461529 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4628003530 con 0x7f46280039c0 2024-04-17T10:49:24.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.933+0000 7f4641f10640 1 -- 172.21.15.83:0/2758461529 reap_dead start 2024-04-17T10:49:24.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.937+0000 7f464170f640 1 --2- 172.21.15.83:0/2758461529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46280039c0 0x7f4628003de0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f4630009ae0 tx=0x7f463001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd41fa46183b8de server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:24.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.937+0000 7f462ffff640 1 -- 172.21.15.83:0/2758461529 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46300096b0 con 0x7f46280039c0 2024-04-17T10:49:24.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.937+0000 7f462ffff640 1 -- 172.21.15.83:0/2758461529 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46300a6450 con 0x7f46280039c0 2024-04-17T10:49:24.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.937+0000 7f462ffff640 1 -- 172.21.15.83:0/2758461529 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46300a6840 con 0x7f46280039c0 2024-04-17T10:49:24.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.937+0000 7f4643973640 1 -- 172.21.15.83:0/2758461529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46280039c0 msgr2=0x7f4628003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:24.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.937+0000 7f4643973640 1 --2- 172.21.15.83:0/2758461529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46280039c0 0x7f4628003de0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f4630009ae0 tx=0x7f463001ca70 comp rx=0 tx=0).stop 2024-04-17T10:49:24.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.937+0000 7f4643973640 1 -- 172.21.15.83:0/2758461529 shutdown_connections 2024-04-17T10:49:24.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.937+0000 7f4643973640 1 --2- 172.21.15.83:0/2758461529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46280039c0 0x7f4628003de0 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:24.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.937+0000 7f4643973640 1 -- 172.21.15.83:0/2758461529 >> 172.21.15.83:0/2758461529 conn(0x7f463c09a9d0 msgr2=0x7f463c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:24.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.937+0000 7f4643973640 1 -- 172.21.15.83:0/2758461529 shutdown_connections 2024-04-17T10:49:24.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.937+0000 7f4643973640 1 -- 172.21.15.83:0/2758461529 wait complete. 2024-04-17T10:49:24.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.937+0000 7f4643973640 1 Processor -- start 2024-04-17T10:49:24.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.937+0000 7f4643973640 1 -- start start 2024-04-17T10:49:24.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.937+0000 7f4643973640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46280039c0 0x7f463c161e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:24.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.937+0000 7f4643973640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f463c09f710 0x7f463c0c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:24.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.937+0000 7f4643973640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f463c09c6a0 0x7f463c0cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:24.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.937+0000 7f4643973640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463c0c4fc0 con 0x7f463c09c6a0 2024-04-17T10:49:24.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.937+0000 7f4643973640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463c0c4e40 con 0x7f463c09f710 2024-04-17T10:49:24.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.937+0000 7f4643973640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463c0b10c0 con 0x7f46280039c0 2024-04-17T10:49:24.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.937+0000 7f4641f10640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f463c09c6a0 0x7f463c0cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:24.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.937+0000 7f4641f10640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f463c09c6a0 0x7f463c0cac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35706/0 (socket says 172.21.15.83:35706) 2024-04-17T10:49:24.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.937+0000 7f4641f10640 1 -- 172.21.15.83:0/104301477 learned_addr learned my addr 172.21.15.83:0/104301477 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:24.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.937+0000 7f464170f640 1 --2- 172.21.15.83:0/104301477 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46280039c0 0x7f463c161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:24.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.937+0000 7f4640f0e640 1 --2- 172.21.15.83:0/104301477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f463c09f710 0x7f463c0c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:24.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.937+0000 7f4641f10640 1 -- 172.21.15.83:0/104301477 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46280039c0 msgr2=0x7f463c161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:24.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.937+0000 7f4641f10640 1 --2- 172.21.15.83:0/104301477 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46280039c0 0x7f463c161e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:24.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.937+0000 7f4641f10640 1 -- 172.21.15.83:0/104301477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f463c09f710 msgr2=0x7f463c0c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:24.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.937+0000 7f4641f10640 1 --2- 172.21.15.83:0/104301477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f463c09f710 0x7f463c0c7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:24.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.937+0000 7f4641f10640 1 -- 172.21.15.83:0/104301477 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f463c162390 con 0x7f463c09c6a0 2024-04-17T10:49:24.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.937+0000 7f4640f0e640 1 --2- 172.21.15.83:0/104301477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f463c09f710 0x7f463c0c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:49:24.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.941+0000 7f4641f10640 1 --2- 172.21.15.83:0/104301477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f463c09c6a0 0x7f463c0cac30 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f463c0ce130 tx=0x7f463800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:24.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.941+0000 7f462dffb640 1 -- 172.21.15.83:0/104301477 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4638019070 con 0x7f463c09c6a0 2024-04-17T10:49:24.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.941+0000 7f462dffb640 1 -- 172.21.15.83:0/104301477 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463800c690 con 0x7f463c09c6a0 2024-04-17T10:49:24.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.941+0000 7f4643973640 1 -- 172.21.15.83:0/104301477 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f463c0c8bd0 con 0x7f463c09c6a0 2024-04-17T10:49:24.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.941+0000 7f462dffb640 1 -- 172.21.15.83:0/104301477 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463800ca80 con 0x7f463c09c6a0 2024-04-17T10:49:24.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.941+0000 7f4643973640 1 -- 172.21.15.83:0/104301477 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f463c0c9190 con 0x7f463c09c6a0 2024-04-17T10:49:24.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.941+0000 7f4643973640 1 -- 172.21.15.83:0/104301477 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f463c074f90 con 0x7f463c09c6a0 2024-04-17T10:49:24.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.941+0000 7f462dffb640 1 -- 172.21.15.83:0/104301477 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f463801c070 con 0x7f463c09c6a0 2024-04-17T10:49:24.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.941+0000 7f462dffb640 1 --2- 172.21.15.83:0/104301477 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4620066f60 0x7f4620069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:24.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.941+0000 7f464170f640 1 --2- 172.21.15.83:0/104301477 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4620066f60 0x7f4620069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:24.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.941+0000 7f462dffb640 1 -- 172.21.15.83:0/104301477 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f46380844c0 con 0x7f463c09c6a0 2024-04-17T10:49:24.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.941+0000 7f464170f640 1 --2- 172.21.15.83:0/104301477 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4620066f60 0x7f4620069420 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f46300ad000 tx=0x7f46300b2040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:24.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:24.945+0000 7f462dffb640 1 -- 172.21.15.83:0/104301477 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f463c162390 con 0x7f463c09c6a0 2024-04-17T10:49:25.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:25.097+0000 7f4643973640 1 -- 172.21.15.83:0/104301477 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f463c09b280 con 0x7f4620066f60 2024-04-17T10:49:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:25.105+0000 7f462dffb640 1 -- 172.21.15.83:0/104301477 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240257 (secure 0 0 0) 0x7f46300f9aa0 con 0x7f4620066f60 2024-04-17T10:49:25.099 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:25.100 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:49:25.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:25.105+0000 7f4643973640 1 -- 172.21.15.83:0/104301477 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4620066f60 msgr2=0x7f4620069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:25.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:25.105+0000 7f4643973640 1 --2- 172.21.15.83:0/104301477 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4620066f60 0x7f4620069420 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f46300ad000 tx=0x7f46300b2040 comp rx=0 tx=0).stop 2024-04-17T10:49:25.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:25.105+0000 7f4643973640 1 -- 172.21.15.83:0/104301477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f463c09c6a0 msgr2=0x7f463c0cac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:25.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:25.105+0000 7f4643973640 1 --2- 172.21.15.83:0/104301477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f463c09c6a0 0x7f463c0cac30 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f463c0ce130 tx=0x7f463800eb90 comp rx=0 tx=0).stop 2024-04-17T10:49:25.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:25.105+0000 7f4643973640 1 -- 172.21.15.83:0/104301477 shutdown_connections 2024-04-17T10:49:25.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:25.105+0000 7f4643973640 1 --2- 172.21.15.83:0/104301477 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4620066f60 0x7f4620069420 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:25.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:25.105+0000 7f4643973640 1 --2- 172.21.15.83:0/104301477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f463c09c6a0 0x7f463c0cac30 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:25.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:25.105+0000 7f4643973640 1 --2- 172.21.15.83:0/104301477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f463c09f710 0x7f463c0c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:25.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:25.105+0000 7f4643973640 1 --2- 172.21.15.83:0/104301477 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46280039c0 0x7f463c161e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:25.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:25.105+0000 7f4643973640 1 -- 172.21.15.83:0/104301477 >> 172.21.15.83:0/104301477 conn(0x7f463c09a9d0 msgr2=0x7f463c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:25.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:25.109+0000 7f4643973640 1 -- 172.21.15.83:0/104301477 shutdown_connections 2024-04-17T10:49:25.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:25.109+0000 7f4643973640 1 -- 172.21.15.83:0/104301477 wait complete. 2024-04-17T10:49:25.115 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":463,"stamp":"2024-04-17T10:49:24.379214+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":518140,"kb_used_data":13256,"kb_used_omap":12,"kb_used_meta":504755,"kb_avail":754456580,"statfs":{"total":773094113280,"available":772563537920,"internally_reserved":0,"allocated":13574144,"data_stored":6560936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":516869680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.781117"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.601284+0000","last_change":"2024-04-17T10:49:18.601284+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.601284+0000","last_undegraded":"2024-04-17T10:49:18.601284+0000","last_fullsized":"2024-04-17T10:49:18.601284+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.606434+0000","last_change":"2024-04-17T10:49:18.606434+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.606434+0000","last_undegraded":"2024-04-17T10:49:18.606434+0000","last_fullsized":"2024-04-17T10:49:18.606434+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.607163+0000","last_change":"2024-04-17T10:49:18.607163+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.607163+0000","last_undegraded":"2024-04-17T10:49:18.607163+0000","last_fullsized":"2024-04-17T10:49:18.607163+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.604468+0000","last_change":"2024-04-17T10:49:18.604468+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.604468+0000","last_undegraded":"2024-04-17T10:49:18.604468+0000","last_fullsized":"2024-04-17T10:49:18.604468+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.605190+0000","last_change":"2024-04-17T10:49:18.605190+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.605190+0000","last_undegraded":"2024-04-17T10:49:18.605190+0000","last_fullsized":"2024-04-17T10:49:18.605190+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.607670+0000","last_change":"2024-04-17T10:49:18.607670+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.607670+0000","last_undegraded":"2024-04-17T10:49:18.607670+0000","last_fullsized":"2024-04-17T10:49:18.607670+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.599977+0000","last_change":"2024-04-17T10:49:18.599977+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.599977+0000","last_undegraded":"2024-04-17T10:49:18.599977+0000","last_fullsized":"2024-04-17T10:49:18.599977+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.602747+0000","last_change":"2024-04-17T10:49:18.602747+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.602747+0000","last_undegraded":"2024-04-17T10:49:18.602747+0000","last_fullsized":"2024-04-17T10:49:18.602747+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.607858+0000","last_change":"2024-04-17T10:49:18.607858+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.607858+0000","last_undegraded":"2024-04-17T10:49:18.607858+0000","last_fullsized":"2024-04-17T10:49:18.607858+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.606892+0000","last_change":"2024-04-17T10:49:18.606892+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.606892+0000","last_undegraded":"2024-04-17T10:49:18.606892+0000","last_fullsized":"2024-04-17T10:49:18.606892+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.606429+0000","last_change":"2024-04-17T10:49:18.606429+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.606429+0000","last_undegraded":"2024-04-17T10:49:18.606429+0000","last_fullsized":"2024-04-17T10:49:18.606429+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.610020+0000","last_change":"2024-04-17T10:49:18.610020+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.610020+0000","last_undegraded":"2024-04-17T10:49:18.610020+0000","last_fullsized":"2024-04-17T10:49:18.610020+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.603716+0000","last_change":"2024-04-17T10:49:18.603716+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.603716+0000","last_undegraded":"2024-04-17T10:49:18.603716+0000","last_fullsized":"2024-04-17T10:49:18.603716+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.603472+0000","last_change":"2024-04-17T10:49:18.603472+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.603472+0000","last_undegraded":"2024-04-17T10:49:18.603472+0000","last_fullsized":"2024-04-17T10:49:18.603472+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.607126+0000","last_change":"2024-04-17T10:49:18.607126+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.607126+0000","last_undegraded":"2024-04-17T10:49:18.607126+0000","last_fullsized":"2024-04-17T10:49:18.607126+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.606500+0000","last_change":"2024-04-17T10:49:18.606500+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.606500+0000","last_undegraded":"2024-04-17T10:49:18.606500+0000","last_fullsized":"2024-04-17T10:49:18.606500+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.608023+0000","last_change":"2024-04-17T10:49:18.608023+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.608023+0000","last_undegraded":"2024-04-17T10:49:18.608023+0000","last_fullsized":"2024-04-17T10:49:18.608023+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.601414+0000","last_change":"2024-04-17T10:49:18.601414+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.601414+0000","last_undegraded":"2024-04-17T10:49:18.601414+0000","last_fullsized":"2024-04-17T10:49:18.601414+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.606274+0000","last_change":"2024-04-17T10:49:18.606274+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.606274+0000","last_undegraded":"2024-04-17T10:49:18.606274+0000","last_fullsized":"2024-04-17T10:49:18.606274+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.606819+0000","last_change":"2024-04-17T10:49:18.606819+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.606819+0000","last_undegraded":"2024-04-17T10:49:18.606819+0000","last_fullsized":"2024-04-17T10:49:18.606819+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.606846+0000","last_change":"2024-04-17T10:49:18.606846+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.606846+0000","last_undegraded":"2024-04-17T10:49:18.606846+0000","last_fullsized":"2024-04-17T10:49:18.606846+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.607464+0000","last_change":"2024-04-17T10:49:18.607464+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.607464+0000","last_undegraded":"2024-04-17T10:49:18.607464+0000","last_fullsized":"2024-04-17T10:49:18.607464+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.605424+0000","last_change":"2024-04-17T10:49:18.605424+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.605424+0000","last_undegraded":"2024-04-17T10:49:18.605424+0000","last_fullsized":"2024-04-17T10:49:18.605424+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.603806+0000","last_change":"2024-04-17T10:49:18.603806+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.603806+0000","last_undegraded":"2024-04-17T10:49:18.603806+0000","last_fullsized":"2024-04-17T10:49:18.603806+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.599394+0000","last_change":"2024-04-17T10:49:18.599394+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.599394+0000","last_undegraded":"2024-04-17T10:49:18.599394+0000","last_fullsized":"2024-04-17T10:49:18.599394+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.602627+0000","last_change":"2024-04-17T10:49:18.602627+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.602627+0000","last_undegraded":"2024-04-17T10:49:18.602627+0000","last_fullsized":"2024-04-17T10:49:18.602627+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.609076+0000","last_change":"2024-04-17T10:49:18.609076+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.609076+0000","last_undegraded":"2024-04-17T10:49:18.609076+0000","last_fullsized":"2024-04-17T10:49:18.609076+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.602589+0000","last_change":"2024-04-17T10:49:18.602589+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.602589+0000","last_undegraded":"2024-04-17T10:49:18.602589+0000","last_fullsized":"2024-04-17T10:49:18.602589+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.599419+0000","last_change":"2024-04-17T10:49:18.599419+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.599419+0000","last_undegraded":"2024-04-17T10:49:18.599419+0000","last_fullsized":"2024-04-17T10:49:18.599419+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.606829+0000","last_change":"2024-04-17T10:49:18.606829+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.606829+0000","last_undegraded":"2024-04-17T10:49:18.606829+0000","last_fullsized":"2024-04-17T10:49:18.606829+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.604444+0000","last_change":"2024-04-17T10:49:18.604444+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.604444+0000","last_undegraded":"2024-04-17T10:49:18.604444+0000","last_fullsized":"2024-04-17T10:49:18.604444+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T10:49:19.595335+0000","last_change":"2024-04-17T10:49:19.595335+0000","last_active":"2024-04-17T10:49:19.595335+0000","last_peered":"2024-04-17T10:49:19.595335+0000","last_clean":"2024-04-17T10:49:19.595335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:19.595335+0000","last_undegraded":"2024-04-17T10:49:19.595335+0000","last_fullsized":"2024-04-17T10:49:19.595335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-17T10:49:18.599931+0000","last_change":"2024-04-17T10:49:18.599931+0000","last_active":"2024-04-17T10:49:18.586291+0000","last_peered":"2024-04-17T10:49:18.586291+0000","last_clean":"2024-04-17T10:49:18.586291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:49:18.599931+0000","last_undegraded":"2024-04-17T10:49:18.599931+0000","last_fullsized":"2024-04-17T10:49:18.599931+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":383,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-17T10:49:18.601893+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:49:18.601893+0000","last_peered":"2024-04-17T10:49:18.601893+0000","last_clean":"2024-04-17T10:49:18.601893+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:49:18.601893+0000","last_undegraded":"2024-04-17T10:49:18.601893+0000","last_fullsized":"2024-04-17T10:49:18.601893+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574988,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47160,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324680,"statfs":{"total":96636764160,"available":96588472320,"internally_reserved":0,"allocated":2142208,"data_stored":1262893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68788,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303052,"statfs":{"total":96636764160,"available":96566325248,"internally_reserved":0,"allocated":1548288,"data_stored":672525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54908,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316932,"statfs":{"total":96636764160,"available":96580538368,"internally_reserved":0,"allocated":1548288,"data_stored":672525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574987,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72948,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298892,"statfs":{"total":96636764160,"available":96562065408,"internally_reserved":0,"allocated":1548288,"data_stored":672525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72956,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298884,"statfs":{"total":96636764160,"available":96562057216,"internally_reserved":0,"allocated":1548288,"data_stored":672525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59060,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312780,"statfs":{"total":96636764160,"available":96576286720,"internally_reserved":0,"allocated":1548288,"data_stored":672525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95160,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94276680,"statfs":{"total":96636764160,"available":96539320320,"internally_reserved":0,"allocated":1548288,"data_stored":672525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574987,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47160,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324680,"statfs":{"total":96636764160,"available":96588472320,"internally_reserved":0,"allocated":2142208,"data_stored":1262893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:49:28.120 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:49:28.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.261+0000 7f9193b27640 1 Processor -- start 2024-04-17T10:49:28.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.261+0000 7f9193b27640 1 -- start start 2024-04-17T10:49:28.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.261+0000 7f9193b27640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f918c15dac0 0x7f918c06e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:28.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.261+0000 7f9193b27640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f918c06eb20 0x7f918c161710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:28.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.261+0000 7f9193b27640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f918c06bd90 con 0x7f918c161c50 2024-04-17T10:49:28.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.261+0000 7f9193b27640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f918c06c9a0 con 0x7f918c06eb20 2024-04-17T10:49:28.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.261+0000 7f9193b27640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f918c06c070 con 0x7f918c15dac0 2024-04-17T10:49:28.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.261+0000 7f91910c2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f918c06eb20 0x7f918c161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:28.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.261+0000 7f91910c2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f918c06eb20 0x7f918c161710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57876/0 (socket says 172.21.15.83:57876) 2024-04-17T10:49:28.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.261+0000 7f91910c2640 1 -- 172.21.15.83:0/2781942639 learned_addr learned my addr 172.21.15.83:0/2781942639 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:28.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.261+0000 7f91918c3640 1 --2- 172.21.15.83:0/2781942639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f918c15dac0 0x7f918c06e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:28.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.261+0000 7f91910c2640 1 -- 172.21.15.83:0/2781942639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f918c15dac0 msgr2=0x7f918c06e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:28.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.261+0000 7f91910c2640 1 --2- 172.21.15.83:0/2781942639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f918c15dac0 0x7f918c06e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:28.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.261+0000 7f91910c2640 1 -- 172.21.15.83:0/2781942639 >> v1:172.21.15.167:6789/0 conn(0x7f918c161c50 legacy=0x7f918c06ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:28.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.261+0000 7f91910c2640 1 -- 172.21.15.83:0/2781942639 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f918c06c350 con 0x7f918c06eb20 2024-04-17T10:49:28.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.265+0000 7f91910c2640 1 --2- 172.21.15.83:0/2781942639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f918c06eb20 0x7f918c161710 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f917c009870 tx=0x7f917c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f97023327cff6b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:28.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.265+0000 7f91908c1640 1 -- 172.21.15.83:0/2781942639 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c0a7070 con 0x7f918c06eb20 2024-04-17T10:49:28.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.265+0000 7f91908c1640 1 -- 172.21.15.83:0/2781942639 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917c0b0070 con 0x7f918c06eb20 2024-04-17T10:49:28.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.265+0000 7f91908c1640 1 -- 172.21.15.83:0/2781942639 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c0ab890 con 0x7f918c06eb20 2024-04-17T10:49:28.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.265+0000 7f9193b27640 1 -- 172.21.15.83:0/2781942639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f918c06eb20 msgr2=0x7f918c161710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:28.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.265+0000 7f9193b27640 1 --2- 172.21.15.83:0/2781942639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f918c06eb20 0x7f918c161710 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f917c009870 tx=0x7f917c01c560 comp rx=0 tx=0).stop 2024-04-17T10:49:28.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.265+0000 7f9193b27640 1 -- 172.21.15.83:0/2781942639 shutdown_connections 2024-04-17T10:49:28.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.265+0000 7f9193b27640 1 --2- 172.21.15.83:0/2781942639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f918c06eb20 0x7f918c161710 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:28.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.265+0000 7f9193b27640 1 --2- 172.21.15.83:0/2781942639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f918c15dac0 0x7f918c06e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:28.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.265+0000 7f9193b27640 1 -- 172.21.15.83:0/2781942639 >> 172.21.15.83:0/2781942639 conn(0x7f918c0b3220 msgr2=0x7f918c0b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:28.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.265+0000 7f9193b27640 1 -- 172.21.15.83:0/2781942639 shutdown_connections 2024-04-17T10:49:28.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.265+0000 7f9193b27640 1 -- 172.21.15.83:0/2781942639 wait complete. 2024-04-17T10:49:28.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.265+0000 7f9193b27640 1 Processor -- start 2024-04-17T10:49:28.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.265+0000 7f9193b27640 1 -- start start 2024-04-17T10:49:28.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.265+0000 7f9193b27640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f918c06eb20 0x7f918c148ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:28.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.265+0000 7f9193b27640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f918c15dac0 0x7f918c1f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:28.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.265+0000 7f9193b27640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f918c161c50 0x7f918c0b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:28.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.265+0000 7f9193b27640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918c162510 con 0x7f918c06eb20 2024-04-17T10:49:28.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.265+0000 7f9193b27640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918c162390 con 0x7f918c15dac0 2024-04-17T10:49:28.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.265+0000 7f9193b27640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f918c16aab0 con 0x7f918c161c50 2024-04-17T10:49:28.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.265+0000 7f91918c3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f918c06eb20 0x7f918c148ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:28.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.265+0000 7f91918c3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f918c06eb20 0x7f918c148ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57886/0 (socket says 172.21.15.83:57886) 2024-04-17T10:49:28.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.265+0000 7f91918c3640 1 -- 172.21.15.83:0/1724171768 learned_addr learned my addr 172.21.15.83:0/1724171768 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:28.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.265+0000 7f91910c2640 1 --2- 172.21.15.83:0/1724171768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f918c15dac0 0x7f918c1f6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:28.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.265+0000 7f91920c4640 1 --2- 172.21.15.83:0/1724171768 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f918c161c50 0x7f918c0b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:28.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.265+0000 7f91918c3640 1 -- 172.21.15.83:0/1724171768 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f918c161c50 msgr2=0x7f918c0b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:28.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.265+0000 7f91918c3640 1 --2- 172.21.15.83:0/1724171768 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f918c161c50 0x7f918c0b22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:28.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.265+0000 7f91918c3640 1 -- 172.21.15.83:0/1724171768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f918c15dac0 msgr2=0x7f918c1f6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:28.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.265+0000 7f91918c3640 1 --2- 172.21.15.83:0/1724171768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f918c15dac0 0x7f918c1f6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:28.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.265+0000 7f91918c3640 1 -- 172.21.15.83:0/1724171768 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f918c1f7790 con 0x7f918c06eb20 2024-04-17T10:49:28.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.269+0000 7f91918c3640 1 --2- 172.21.15.83:0/1724171768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f918c06eb20 0x7f918c148ca0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f918800d800 tx=0x7f918800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:28.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.269+0000 7f91827fc640 1 -- 172.21.15.83:0/1724171768 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918800b6c0 con 0x7f918c06eb20 2024-04-17T10:49:28.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.269+0000 7f91827fc640 1 -- 172.21.15.83:0/1724171768 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9188019070 con 0x7f918c06eb20 2024-04-17T10:49:28.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.269+0000 7f9193b27640 1 -- 172.21.15.83:0/1724171768 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f918c1f7a70 con 0x7f918c06eb20 2024-04-17T10:49:28.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.269+0000 7f91827fc640 1 -- 172.21.15.83:0/1724171768 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918800ba60 con 0x7f918c06eb20 2024-04-17T10:49:28.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.269+0000 7f9193b27640 1 -- 172.21.15.83:0/1724171768 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f918c1f8030 con 0x7f918c06eb20 2024-04-17T10:49:28.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.269+0000 7f9193b27640 1 -- 172.21.15.83:0/1724171768 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f918c074f90 con 0x7f918c06eb20 2024-04-17T10:49:28.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.269+0000 7f91827fc640 1 -- 172.21.15.83:0/1724171768 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f918800bc00 con 0x7f918c06eb20 2024-04-17T10:49:28.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.269+0000 7f91827fc640 1 --2- 172.21.15.83:0/1724171768 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9170066f90 0x7f9170069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:28.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.269+0000 7f91827fc640 1 -- 172.21.15.83:0/1724171768 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9188085490 con 0x7f918c06eb20 2024-04-17T10:49:28.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.269+0000 7f91910c2640 1 --2- 172.21.15.83:0/1724171768 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9170066f90 0x7f9170069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:28.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.269+0000 7f91910c2640 1 --2- 172.21.15.83:0/1724171768 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9170066f90 0x7f9170069450 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f917c0af000 tx=0x7f917c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:28.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.273+0000 7f91827fc640 1 -- 172.21.15.83:0/1724171768 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f91880564a0 con 0x7f918c06eb20 2024-04-17T10:49:28.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.425+0000 7f9193b27640 1 -- 172.21.15.83:0/1724171768 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f918c1f7530 con 0x7f9170066f90 2024-04-17T10:49:28.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.433+0000 7f91827fc640 1 -- 172.21.15.83:0/1724171768 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252486 (secure 0 0 0) 0x7f917c0b0040 con 0x7f9170066f90 2024-04-17T10:49:28.428 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:28.430 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:49:28.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.437+0000 7f9193b27640 1 -- 172.21.15.83:0/1724171768 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9170066f90 msgr2=0x7f9170069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:28.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.437+0000 7f9193b27640 1 --2- 172.21.15.83:0/1724171768 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9170066f90 0x7f9170069450 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f917c0af000 tx=0x7f917c0a7000 comp rx=0 tx=0).stop 2024-04-17T10:49:28.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.437+0000 7f9193b27640 1 -- 172.21.15.83:0/1724171768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f918c06eb20 msgr2=0x7f918c148ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:28.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.437+0000 7f9193b27640 1 --2- 172.21.15.83:0/1724171768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f918c06eb20 0x7f918c148ca0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f918800d800 tx=0x7f918800dcc0 comp rx=0 tx=0).stop 2024-04-17T10:49:28.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.437+0000 7f9193b27640 1 -- 172.21.15.83:0/1724171768 shutdown_connections 2024-04-17T10:49:28.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.437+0000 7f9193b27640 1 --2- 172.21.15.83:0/1724171768 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9170066f90 0x7f9170069450 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:28.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.437+0000 7f9193b27640 1 --2- 172.21.15.83:0/1724171768 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f918c161c50 0x7f918c0b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:28.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.437+0000 7f9193b27640 1 --2- 172.21.15.83:0/1724171768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f918c15dac0 0x7f918c1f6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:28.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.437+0000 7f9193b27640 1 --2- 172.21.15.83:0/1724171768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f918c06eb20 0x7f918c148ca0 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:28.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.437+0000 7f9193b27640 1 -- 172.21.15.83:0/1724171768 >> 172.21.15.83:0/1724171768 conn(0x7f918c0b3220 msgr2=0x7f918c0af850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:28.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.437+0000 7f9193b27640 1 -- 172.21.15.83:0/1724171768 shutdown_connections 2024-04-17T10:49:28.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.437+0000 7f9193b27640 1 -- 172.21.15.83:0/1724171768 wait complete. 2024-04-17T10:49:28.445 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2024-04-17T10:49:28.380523+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":518488,"kb_used_data":13608,"kb_used_omap":12,"kb_used_meta":504755,"kb_avail":754456232,"statfs":{"total":773094113280,"available":772563181568,"internally_reserved":0,"allocated":13934592,"data_stored":6720556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":516869680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002439"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.839594+0000","last_change":"2024-04-17T10:49:20.673590+0000","last_active":"2024-04-17T10:49:22.839594+0000","last_peered":"2024-04-17T10:49:22.839594+0000","last_clean":"2024-04-17T10:49:22.839594+0000","last_became_active":"2024-04-17T10:49:20.673262+0000","last_became_peered":"2024-04-17T10:49:20.673262+0000","last_unstale":"2024-04-17T10:49:22.839594+0000","last_undegraded":"2024-04-17T10:49:22.839594+0000","last_fullsized":"2024-04-17T10:49:22.839594+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:24:03.354876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.655837+0000","last_change":"2024-04-17T10:49:20.673168+0000","last_active":"2024-04-17T10:49:21.655837+0000","last_peered":"2024-04-17T10:49:21.655837+0000","last_clean":"2024-04-17T10:49:21.655837+0000","last_became_active":"2024-04-17T10:49:20.672858+0000","last_became_peered":"2024-04-17T10:49:20.672858+0000","last_unstale":"2024-04-17T10:49:21.655837+0000","last_undegraded":"2024-04-17T10:49:21.655837+0000","last_fullsized":"2024-04-17T10:49:21.655837+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:45:45.445011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.657631+0000","last_change":"2024-04-17T10:49:20.645864+0000","last_active":"2024-04-17T10:49:21.657631+0000","last_peered":"2024-04-17T10:49:21.657631+0000","last_clean":"2024-04-17T10:49:21.657631+0000","last_became_active":"2024-04-17T10:49:20.644650+0000","last_became_peered":"2024-04-17T10:49:20.644650+0000","last_unstale":"2024-04-17T10:49:21.657631+0000","last_undegraded":"2024-04-17T10:49:21.657631+0000","last_fullsized":"2024-04-17T10:49:21.657631+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:13:28.615303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.839317+0000","last_change":"2024-04-17T10:49:20.645367+0000","last_active":"2024-04-17T10:49:22.839317+0000","last_peered":"2024-04-17T10:49:22.839317+0000","last_clean":"2024-04-17T10:49:22.839317+0000","last_became_active":"2024-04-17T10:49:20.645030+0000","last_became_peered":"2024-04-17T10:49:20.645030+0000","last_unstale":"2024-04-17T10:49:22.839317+0000","last_undegraded":"2024-04-17T10:49:22.839317+0000","last_fullsized":"2024-04-17T10:49:22.839317+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:37:25.099988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.655978+0000","last_change":"2024-04-17T10:49:20.670475+0000","last_active":"2024-04-17T10:49:21.655978+0000","last_peered":"2024-04-17T10:49:21.655978+0000","last_clean":"2024-04-17T10:49:21.655978+0000","last_became_active":"2024-04-17T10:49:20.668963+0000","last_became_peered":"2024-04-17T10:49:20.668963+0000","last_unstale":"2024-04-17T10:49:21.655978+0000","last_undegraded":"2024-04-17T10:49:21.655978+0000","last_fullsized":"2024-04-17T10:49:21.655978+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:47.987142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.593741+0000","last_change":"2024-04-17T10:49:20.671666+0000","last_active":"2024-04-17T10:49:22.593741+0000","last_peered":"2024-04-17T10:49:22.593741+0000","last_clean":"2024-04-17T10:49:22.593741+0000","last_became_active":"2024-04-17T10:49:20.668971+0000","last_became_peered":"2024-04-17T10:49:20.668971+0000","last_unstale":"2024-04-17T10:49:22.593741+0000","last_undegraded":"2024-04-17T10:49:22.593741+0000","last_fullsized":"2024-04-17T10:49:22.593741+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:51:56.596470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.593361+0000","last_change":"2024-04-17T10:49:20.671605+0000","last_active":"2024-04-17T10:49:22.593361+0000","last_peered":"2024-04-17T10:49:22.593361+0000","last_clean":"2024-04-17T10:49:22.593361+0000","last_became_active":"2024-04-17T10:49:20.668896+0000","last_became_peered":"2024-04-17T10:49:20.668896+0000","last_unstale":"2024-04-17T10:49:22.593361+0000","last_undegraded":"2024-04-17T10:49:22.593361+0000","last_fullsized":"2024-04-17T10:49:22.593361+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:18:51.247550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.655373+0000","last_change":"2024-04-17T10:49:20.676420+0000","last_active":"2024-04-17T10:49:21.655373+0000","last_peered":"2024-04-17T10:49:21.655373+0000","last_clean":"2024-04-17T10:49:21.655373+0000","last_became_active":"2024-04-17T10:49:20.664363+0000","last_became_peered":"2024-04-17T10:49:20.664363+0000","last_unstale":"2024-04-17T10:49:21.655373+0000","last_undegraded":"2024-04-17T10:49:21.655373+0000","last_fullsized":"2024-04-17T10:49:21.655373+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:19:48.338129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.657803+0000","last_change":"2024-04-17T10:49:20.677952+0000","last_active":"2024-04-17T10:49:21.657803+0000","last_peered":"2024-04-17T10:49:21.657803+0000","last_clean":"2024-04-17T10:49:21.657803+0000","last_became_active":"2024-04-17T10:49:20.676762+0000","last_became_peered":"2024-04-17T10:49:20.676762+0000","last_unstale":"2024-04-17T10:49:21.657803+0000","last_undegraded":"2024-04-17T10:49:21.657803+0000","last_fullsized":"2024-04-17T10:49:21.657803+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:54:31.823446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.655957+0000","last_change":"2024-04-17T10:49:20.673085+0000","last_active":"2024-04-17T10:49:21.655957+0000","last_peered":"2024-04-17T10:49:21.655957+0000","last_clean":"2024-04-17T10:49:21.655957+0000","last_became_active":"2024-04-17T10:49:20.672790+0000","last_became_peered":"2024-04-17T10:49:20.672790+0000","last_unstale":"2024-04-17T10:49:21.655957+0000","last_undegraded":"2024-04-17T10:49:21.655957+0000","last_fullsized":"2024-04-17T10:49:21.655957+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:40:26.540920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.308125+0000","last_change":"2024-04-17T10:49:20.655457+0000","last_active":"2024-04-17T10:49:22.308125+0000","last_peered":"2024-04-17T10:49:22.308125+0000","last_clean":"2024-04-17T10:49:22.308125+0000","last_became_active":"2024-04-17T10:49:20.655166+0000","last_became_peered":"2024-04-17T10:49:20.655166+0000","last_unstale":"2024-04-17T10:49:22.308125+0000","last_undegraded":"2024-04-17T10:49:22.308125+0000","last_fullsized":"2024-04-17T10:49:22.308125+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:48:33.213849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.657344+0000","last_change":"2024-04-17T10:49:20.677298+0000","last_active":"2024-04-17T10:49:21.657344+0000","last_peered":"2024-04-17T10:49:21.657344+0000","last_clean":"2024-04-17T10:49:21.657344+0000","last_became_active":"2024-04-17T10:49:20.676225+0000","last_became_peered":"2024-04-17T10:49:20.676225+0000","last_unstale":"2024-04-17T10:49:21.657344+0000","last_undegraded":"2024-04-17T10:49:21.657344+0000","last_fullsized":"2024-04-17T10:49:21.657344+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:36:02.079001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.839307+0000","last_change":"2024-04-17T10:49:20.673062+0000","last_active":"2024-04-17T10:49:22.839307+0000","last_peered":"2024-04-17T10:49:22.839307+0000","last_clean":"2024-04-17T10:49:22.839307+0000","last_became_active":"2024-04-17T10:49:20.672741+0000","last_became_peered":"2024-04-17T10:49:20.672741+0000","last_unstale":"2024-04-17T10:49:22.839307+0000","last_undegraded":"2024-04-17T10:49:22.839307+0000","last_fullsized":"2024-04-17T10:49:22.839307+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:07:32.184441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.655669+0000","last_change":"2024-04-17T10:49:20.651363+0000","last_active":"2024-04-17T10:49:21.655669+0000","last_peered":"2024-04-17T10:49:21.655669+0000","last_clean":"2024-04-17T10:49:21.655669+0000","last_became_active":"2024-04-17T10:49:20.650579+0000","last_became_peered":"2024-04-17T10:49:20.650579+0000","last_unstale":"2024-04-17T10:49:21.655669+0000","last_undegraded":"2024-04-17T10:49:21.655669+0000","last_fullsized":"2024-04-17T10:49:21.655669+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:18:12.005726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.839088+0000","last_change":"2024-04-17T10:49:20.645661+0000","last_active":"2024-04-17T10:49:22.839088+0000","last_peered":"2024-04-17T10:49:22.839088+0000","last_clean":"2024-04-17T10:49:22.839088+0000","last_became_active":"2024-04-17T10:49:20.645178+0000","last_became_peered":"2024-04-17T10:49:20.645178+0000","last_unstale":"2024-04-17T10:49:22.839088+0000","last_undegraded":"2024-04-17T10:49:22.839088+0000","last_fullsized":"2024-04-17T10:49:22.839088+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:23:20.993854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.412381+0000","last_change":"2024-04-17T10:49:20.651350+0000","last_active":"2024-04-17T10:49:22.412381+0000","last_peered":"2024-04-17T10:49:22.412381+0000","last_clean":"2024-04-17T10:49:22.412381+0000","last_became_active":"2024-04-17T10:49:20.651032+0000","last_became_peered":"2024-04-17T10:49:20.651032+0000","last_unstale":"2024-04-17T10:49:22.412381+0000","last_undegraded":"2024-04-17T10:49:22.412381+0000","last_fullsized":"2024-04-17T10:49:22.412381+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:09:35.357557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.594325+0000","last_change":"2024-04-17T10:49:20.672168+0000","last_active":"2024-04-17T10:49:22.594325+0000","last_peered":"2024-04-17T10:49:22.594325+0000","last_clean":"2024-04-17T10:49:22.594325+0000","last_became_active":"2024-04-17T10:49:20.669049+0000","last_became_peered":"2024-04-17T10:49:20.669049+0000","last_unstale":"2024-04-17T10:49:22.594325+0000","last_undegraded":"2024-04-17T10:49:22.594325+0000","last_fullsized":"2024-04-17T10:49:22.594325+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:18:51.718455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.412224+0000","last_change":"2024-04-17T10:49:20.673169+0000","last_active":"2024-04-17T10:49:22.412224+0000","last_peered":"2024-04-17T10:49:22.412224+0000","last_clean":"2024-04-17T10:49:22.412224+0000","last_became_active":"2024-04-17T10:49:20.672886+0000","last_became_peered":"2024-04-17T10:49:20.672886+0000","last_unstale":"2024-04-17T10:49:22.412224+0000","last_undegraded":"2024-04-17T10:49:22.412224+0000","last_fullsized":"2024-04-17T10:49:22.412224+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:14:35.759031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.308380+0000","last_change":"2024-04-17T10:49:20.673581+0000","last_active":"2024-04-17T10:49:22.308380+0000","last_peered":"2024-04-17T10:49:22.308380+0000","last_clean":"2024-04-17T10:49:22.308380+0000","last_became_active":"2024-04-17T10:49:20.673288+0000","last_became_peered":"2024-04-17T10:49:20.673288+0000","last_unstale":"2024-04-17T10:49:22.308380+0000","last_undegraded":"2024-04-17T10:49:22.308380+0000","last_fullsized":"2024-04-17T10:49:22.308380+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:02:02.176762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.412231+0000","last_change":"2024-04-17T10:49:20.657349+0000","last_active":"2024-04-17T10:49:22.412231+0000","last_peered":"2024-04-17T10:49:22.412231+0000","last_clean":"2024-04-17T10:49:22.412231+0000","last_became_active":"2024-04-17T10:49:20.657060+0000","last_became_peered":"2024-04-17T10:49:20.657060+0000","last_unstale":"2024-04-17T10:49:22.412231+0000","last_undegraded":"2024-04-17T10:49:22.412231+0000","last_fullsized":"2024-04-17T10:49:22.412231+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:06:58.356442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.655521+0000","last_change":"2024-04-17T10:49:20.671634+0000","last_active":"2024-04-17T10:49:21.655521+0000","last_peered":"2024-04-17T10:49:21.655521+0000","last_clean":"2024-04-17T10:49:21.655521+0000","last_became_active":"2024-04-17T10:49:20.670215+0000","last_became_peered":"2024-04-17T10:49:20.670215+0000","last_unstale":"2024-04-17T10:49:21.655521+0000","last_undegraded":"2024-04-17T10:49:21.655521+0000","last_fullsized":"2024-04-17T10:49:21.655521+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:06:58.356442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.307949+0000","last_change":"2024-04-17T10:49:20.644233+0000","last_active":"2024-04-17T10:49:22.307949+0000","last_peered":"2024-04-17T10:49:22.307949+0000","last_clean":"2024-04-17T10:49:22.307949+0000","last_became_active":"2024-04-17T10:49:20.643654+0000","last_became_peered":"2024-04-17T10:49:20.643654+0000","last_unstale":"2024-04-17T10:49:22.307949+0000","last_undegraded":"2024-04-17T10:49:22.307949+0000","last_fullsized":"2024-04-17T10:49:22.307949+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:22:20.516161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.657149+0000","last_change":"2024-04-17T10:49:20.675819+0000","last_active":"2024-04-17T10:49:21.657149+0000","last_peered":"2024-04-17T10:49:21.657149+0000","last_clean":"2024-04-17T10:49:21.657149+0000","last_became_active":"2024-04-17T10:49:20.665878+0000","last_became_peered":"2024-04-17T10:49:20.665878+0000","last_unstale":"2024-04-17T10:49:21.657149+0000","last_undegraded":"2024-04-17T10:49:21.657149+0000","last_fullsized":"2024-04-17T10:49:21.657149+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:36:35.392754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.307715+0000","last_change":"2024-04-17T10:49:20.655517+0000","last_active":"2024-04-17T10:49:22.307715+0000","last_peered":"2024-04-17T10:49:22.307715+0000","last_clean":"2024-04-17T10:49:22.307715+0000","last_became_active":"2024-04-17T10:49:20.655151+0000","last_became_peered":"2024-04-17T10:49:20.655151+0000","last_unstale":"2024-04-17T10:49:22.307715+0000","last_undegraded":"2024-04-17T10:49:22.307715+0000","last_fullsized":"2024-04-17T10:49:22.307715+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:32:30.053642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.655988+0000","last_change":"2024-04-17T10:49:20.651250+0000","last_active":"2024-04-17T10:49:21.655988+0000","last_peered":"2024-04-17T10:49:21.655988+0000","last_clean":"2024-04-17T10:49:21.655988+0000","last_became_active":"2024-04-17T10:49:20.650514+0000","last_became_peered":"2024-04-17T10:49:20.650514+0000","last_unstale":"2024-04-17T10:49:21.655988+0000","last_undegraded":"2024-04-17T10:49:21.655988+0000","last_fullsized":"2024-04-17T10:49:21.655988+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:12:40.220995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.412304+0000","last_change":"2024-04-17T10:49:20.648696+0000","last_active":"2024-04-17T10:49:22.412304+0000","last_peered":"2024-04-17T10:49:22.412304+0000","last_clean":"2024-04-17T10:49:22.412304+0000","last_became_active":"2024-04-17T10:49:20.648307+0000","last_became_peered":"2024-04-17T10:49:20.648307+0000","last_unstale":"2024-04-17T10:49:22.412304+0000","last_undegraded":"2024-04-17T10:49:22.412304+0000","last_fullsized":"2024-04-17T10:49:22.412304+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:47.987142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.839306+0000","last_change":"2024-04-17T10:49:20.673283+0000","last_active":"2024-04-17T10:49:22.839306+0000","last_peered":"2024-04-17T10:49:22.839306+0000","last_clean":"2024-04-17T10:49:22.839306+0000","last_became_active":"2024-04-17T10:49:20.672964+0000","last_became_peered":"2024-04-17T10:49:20.672964+0000","last_unstale":"2024-04-17T10:49:22.839306+0000","last_undegraded":"2024-04-17T10:49:22.839306+0000","last_fullsized":"2024-04-17T10:49:22.839306+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:43:06.334758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.700059+0000","last_change":"2024-04-17T10:49:20.653417+0000","last_active":"2024-04-17T10:49:21.700059+0000","last_peered":"2024-04-17T10:49:21.700059+0000","last_clean":"2024-04-17T10:49:21.700059+0000","last_became_active":"2024-04-17T10:49:20.652694+0000","last_became_peered":"2024-04-17T10:49:20.652694+0000","last_unstale":"2024-04-17T10:49:21.700059+0000","last_undegraded":"2024-04-17T10:49:21.700059+0000","last_fullsized":"2024-04-17T10:49:21.700059+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:34:40.075025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.839186+0000","last_change":"2024-04-17T10:49:20.642091+0000","last_active":"2024-04-17T10:49:22.839186+0000","last_peered":"2024-04-17T10:49:22.839186+0000","last_clean":"2024-04-17T10:49:22.839186+0000","last_became_active":"2024-04-17T10:49:20.641382+0000","last_became_peered":"2024-04-17T10:49:20.641382+0000","last_unstale":"2024-04-17T10:49:22.839186+0000","last_undegraded":"2024-04-17T10:49:22.839186+0000","last_fullsized":"2024-04-17T10:49:22.839186+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:56.907193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.839118+0000","last_change":"2024-04-17T10:49:20.640113+0000","last_active":"2024-04-17T10:49:22.839118+0000","last_peered":"2024-04-17T10:49:22.839118+0000","last_clean":"2024-04-17T10:49:22.839118+0000","last_became_active":"2024-04-17T10:49:20.638854+0000","last_became_peered":"2024-04-17T10:49:20.638854+0000","last_unstale":"2024-04-17T10:49:22.839118+0000","last_undegraded":"2024-04-17T10:49:22.839118+0000","last_fullsized":"2024-04-17T10:49:22.839118+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:15:03.754123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.656369+0000","last_change":"2024-04-17T10:49:20.649023+0000","last_active":"2024-04-17T10:49:21.656369+0000","last_peered":"2024-04-17T10:49:21.656369+0000","last_clean":"2024-04-17T10:49:21.656369+0000","last_became_active":"2024-04-17T10:49:20.648267+0000","last_became_peered":"2024-04-17T10:49:20.648267+0000","last_unstale":"2024-04-17T10:49:21.656369+0000","last_undegraded":"2024-04-17T10:49:21.656369+0000","last_fullsized":"2024-04-17T10:49:21.656369+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:22:20.516161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.593364+0000","last_change":"2024-04-17T10:49:20.662818+0000","last_active":"2024-04-17T10:49:22.593364+0000","last_peered":"2024-04-17T10:49:22.593364+0000","last_clean":"2024-04-17T10:49:22.593364+0000","last_became_active":"2024-04-17T10:49:20.662088+0000","last_became_peered":"2024-04-17T10:49:20.662088+0000","last_unstale":"2024-04-17T10:49:22.593364+0000","last_undegraded":"2024-04-17T10:49:22.593364+0000","last_fullsized":"2024-04-17T10:49:22.593364+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:13:28.615303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.412631+0000","last_change":"2024-04-17T10:49:20.666759+0000","last_active":"2024-04-17T10:49:22.412631+0000","last_peered":"2024-04-17T10:49:22.412631+0000","last_clean":"2024-04-17T10:49:22.412631+0000","last_became_active":"2024-04-17T10:49:20.666394+0000","last_became_peered":"2024-04-17T10:49:20.666394+0000","last_unstale":"2024-04-17T10:49:22.412631+0000","last_undegraded":"2024-04-17T10:49:22.412631+0000","last_fullsized":"2024-04-17T10:49:22.412631+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:22:20.516161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"126'4","reported_seq":18,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.656539+0000","last_change":"2024-04-17T10:49:19.615386+0000","last_active":"2024-04-17T10:49:21.656539+0000","last_peered":"2024-04-17T10:49:21.656539+0000","last_clean":"2024-04-17T10:49:21.656539+0000","last_became_active":"2024-04-17T10:49:19.614666+0000","last_became_peered":"2024-04-17T10:49:19.614666+0000","last_unstale":"2024-04-17T10:49:21.656539+0000","last_undegraded":"2024-04-17T10:49:21.656539+0000","last_fullsized":"2024-04-17T10:49:21.656539+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:23:21.981832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.658077+0000","last_change":"2024-04-17T10:49:20.650178+0000","last_active":"2024-04-17T10:49:21.658077+0000","last_peered":"2024-04-17T10:49:21.658077+0000","last_clean":"2024-04-17T10:49:21.658077+0000","last_became_active":"2024-04-17T10:49:20.649160+0000","last_became_peered":"2024-04-17T10:49:20.649160+0000","last_unstale":"2024-04-17T10:49:21.658077+0000","last_undegraded":"2024-04-17T10:49:21.658077+0000","last_fullsized":"2024-04-17T10:49:21.658077+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:51:56.596470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"126'1","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.699998+0000","last_change":"2024-04-17T10:49:19.631369+0000","last_active":"2024-04-17T10:49:21.699998+0000","last_peered":"2024-04-17T10:49:21.699998+0000","last_clean":"2024-04-17T10:49:21.699998+0000","last_became_active":"2024-04-17T10:49:19.626237+0000","last_became_peered":"2024-04-17T10:49:19.626237+0000","last_unstale":"2024-04-17T10:49:21.699998+0000","last_undegraded":"2024-04-17T10:49:21.699998+0000","last_fullsized":"2024-04-17T10:49:21.699998+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:13:04.548114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.658043+0000","last_change":"2024-04-17T10:49:20.673608+0000","last_active":"2024-04-17T10:49:21.658043+0000","last_peered":"2024-04-17T10:49:21.658043+0000","last_clean":"2024-04-17T10:49:21.658043+0000","last_became_active":"2024-04-17T10:49:20.672397+0000","last_became_peered":"2024-04-17T10:49:20.672397+0000","last_unstale":"2024-04-17T10:49:21.658043+0000","last_undegraded":"2024-04-17T10:49:21.658043+0000","last_fullsized":"2024-04-17T10:49:21.658043+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:48:33.213849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"126'2","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.308237+0000","last_change":"2024-04-17T10:49:19.621933+0000","last_active":"2024-04-17T10:49:22.308237+0000","last_peered":"2024-04-17T10:49:22.308237+0000","last_clean":"2024-04-17T10:49:22.308237+0000","last_became_active":"2024-04-17T10:49:19.620766+0000","last_became_peered":"2024-04-17T10:49:19.620766+0000","last_unstale":"2024-04-17T10:49:22.308237+0000","last_undegraded":"2024-04-17T10:49:22.308237+0000","last_fullsized":"2024-04-17T10:49:22.308237+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:02:17.679654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.655750+0000","last_change":"2024-04-17T10:49:20.652798+0000","last_active":"2024-04-17T10:49:21.655750+0000","last_peered":"2024-04-17T10:49:21.655750+0000","last_clean":"2024-04-17T10:49:21.655750+0000","last_became_active":"2024-04-17T10:49:20.651893+0000","last_became_peered":"2024-04-17T10:49:20.651893+0000","last_unstale":"2024-04-17T10:49:21.655750+0000","last_undegraded":"2024-04-17T10:49:21.655750+0000","last_fullsized":"2024-04-17T10:49:21.655750+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:31:50.340912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"126'2","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.655977+0000","last_change":"2024-04-17T10:49:19.631659+0000","last_active":"2024-04-17T10:49:21.655977+0000","last_peered":"2024-04-17T10:49:21.655977+0000","last_clean":"2024-04-17T10:49:21.655977+0000","last_became_active":"2024-04-17T10:49:19.631149+0000","last_became_peered":"2024-04-17T10:49:19.631149+0000","last_unstale":"2024-04-17T10:49:21.655977+0000","last_undegraded":"2024-04-17T10:49:21.655977+0000","last_fullsized":"2024-04-17T10:49:21.655977+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:45:33.314765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.411954+0000","last_change":"2024-04-17T10:49:20.679486+0000","last_active":"2024-04-17T10:49:22.411954+0000","last_peered":"2024-04-17T10:49:22.411954+0000","last_clean":"2024-04-17T10:49:22.411954+0000","last_became_active":"2024-04-17T10:49:20.678959+0000","last_became_peered":"2024-04-17T10:49:20.678959+0000","last_unstale":"2024-04-17T10:49:22.411954+0000","last_undegraded":"2024-04-17T10:49:22.411954+0000","last_fullsized":"2024-04-17T10:49:22.411954+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:02:18.688697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"126'1","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.655514+0000","last_change":"2024-04-17T10:49:19.631021+0000","last_active":"2024-04-17T10:49:21.655514+0000","last_peered":"2024-04-17T10:49:21.655514+0000","last_clean":"2024-04-17T10:49:21.655514+0000","last_became_active":"2024-04-17T10:49:19.630663+0000","last_became_peered":"2024-04-17T10:49:19.630663+0000","last_unstale":"2024-04-17T10:49:21.655514+0000","last_undegraded":"2024-04-17T10:49:21.655514+0000","last_fullsized":"2024-04-17T10:49:21.655514+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:15:02.745079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.593760+0000","last_change":"2024-04-17T10:49:20.647030+0000","last_active":"2024-04-17T10:49:22.593760+0000","last_peered":"2024-04-17T10:49:22.593760+0000","last_clean":"2024-04-17T10:49:22.593760+0000","last_became_active":"2024-04-17T10:49:20.646237+0000","last_became_peered":"2024-04-17T10:49:20.646237+0000","last_unstale":"2024-04-17T10:49:22.593760+0000","last_undegraded":"2024-04-17T10:49:22.593760+0000","last_fullsized":"2024-04-17T10:49:22.593760+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:54:31.823446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"126'1","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.411433+0000","last_change":"2024-04-17T10:49:19.632508+0000","last_active":"2024-04-17T10:49:22.411433+0000","last_peered":"2024-04-17T10:49:22.411433+0000","last_clean":"2024-04-17T10:49:22.411433+0000","last_became_active":"2024-04-17T10:49:19.632155+0000","last_became_peered":"2024-04-17T10:49:19.632155+0000","last_unstale":"2024-04-17T10:49:22.411433+0000","last_undegraded":"2024-04-17T10:49:22.411433+0000","last_fullsized":"2024-04-17T10:49:22.411433+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:54.418708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.655747+0000","last_change":"2024-04-17T10:49:20.665989+0000","last_active":"2024-04-17T10:49:21.655747+0000","last_peered":"2024-04-17T10:49:21.655747+0000","last_clean":"2024-04-17T10:49:21.655747+0000","last_became_active":"2024-04-17T10:49:20.663455+0000","last_became_peered":"2024-04-17T10:49:20.663455+0000","last_unstale":"2024-04-17T10:49:21.655747+0000","last_undegraded":"2024-04-17T10:49:21.655747+0000","last_fullsized":"2024-04-17T10:49:21.655747+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:35:51.738641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.593017+0000","last_change":"2024-04-17T10:49:19.623660+0000","last_active":"2024-04-17T10:49:22.593017+0000","last_peered":"2024-04-17T10:49:22.593017+0000","last_clean":"2024-04-17T10:49:22.593017+0000","last_became_active":"2024-04-17T10:49:19.622920+0000","last_became_peered":"2024-04-17T10:49:19.622920+0000","last_unstale":"2024-04-17T10:49:22.593017+0000","last_undegraded":"2024-04-17T10:49:22.593017+0000","last_fullsized":"2024-04-17T10:49:22.593017+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:16:26.476856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.655419+0000","last_change":"2024-04-17T10:49:20.651804+0000","last_active":"2024-04-17T10:49:21.655419+0000","last_peered":"2024-04-17T10:49:21.655419+0000","last_clean":"2024-04-17T10:49:21.655419+0000","last_became_active":"2024-04-17T10:49:20.650953+0000","last_became_peered":"2024-04-17T10:49:20.650953+0000","last_unstale":"2024-04-17T10:49:21.655419+0000","last_undegraded":"2024-04-17T10:49:21.655419+0000","last_fullsized":"2024-04-17T10:49:21.655419+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:27:47.576503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.655501+0000","last_change":"2024-04-17T10:49:19.637500+0000","last_active":"2024-04-17T10:49:21.655501+0000","last_peered":"2024-04-17T10:49:21.655501+0000","last_clean":"2024-04-17T10:49:21.655501+0000","last_became_active":"2024-04-17T10:49:19.636944+0000","last_became_peered":"2024-04-17T10:49:19.636944+0000","last_unstale":"2024-04-17T10:49:21.655501+0000","last_undegraded":"2024-04-17T10:49:21.655501+0000","last_fullsized":"2024-04-17T10:49:21.655501+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:55.898149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.839439+0000","last_change":"2024-04-17T10:49:20.645074+0000","last_active":"2024-04-17T10:49:22.839439+0000","last_peered":"2024-04-17T10:49:22.839439+0000","last_clean":"2024-04-17T10:49:22.839439+0000","last_became_active":"2024-04-17T10:49:20.644452+0000","last_became_peered":"2024-04-17T10:49:20.644452+0000","last_unstale":"2024-04-17T10:49:22.839439+0000","last_undegraded":"2024-04-17T10:49:22.839439+0000","last_fullsized":"2024-04-17T10:49:22.839439+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:34:27.858440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.700478+0000","last_change":"2024-04-17T10:49:19.637215+0000","last_active":"2024-04-17T10:49:21.700478+0000","last_peered":"2024-04-17T10:49:21.700478+0000","last_clean":"2024-04-17T10:49:21.700478+0000","last_became_active":"2024-04-17T10:49:19.635663+0000","last_became_peered":"2024-04-17T10:49:19.635663+0000","last_unstale":"2024-04-17T10:49:21.700478+0000","last_undegraded":"2024-04-17T10:49:21.700478+0000","last_fullsized":"2024-04-17T10:49:21.700478+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:55:46.484287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.700352+0000","last_change":"2024-04-17T10:49:20.669943+0000","last_active":"2024-04-17T10:49:21.700352+0000","last_peered":"2024-04-17T10:49:21.700352+0000","last_clean":"2024-04-17T10:49:21.700352+0000","last_became_active":"2024-04-17T10:49:20.668073+0000","last_became_peered":"2024-04-17T10:49:20.668073+0000","last_unstale":"2024-04-17T10:49:21.700352+0000","last_undegraded":"2024-04-17T10:49:21.700352+0000","last_fullsized":"2024-04-17T10:49:21.700352+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:22:21.418264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.412432+0000","last_change":"2024-04-17T10:49:19.637636+0000","last_active":"2024-04-17T10:49:22.412432+0000","last_peered":"2024-04-17T10:49:22.412432+0000","last_clean":"2024-04-17T10:49:22.412432+0000","last_became_active":"2024-04-17T10:49:19.637270+0000","last_became_peered":"2024-04-17T10:49:19.637270+0000","last_unstale":"2024-04-17T10:49:22.412432+0000","last_undegraded":"2024-04-17T10:49:22.412432+0000","last_fullsized":"2024-04-17T10:49:22.412432+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:32:04.769207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.658078+0000","last_change":"2024-04-17T10:49:20.652785+0000","last_active":"2024-04-17T10:49:21.658078+0000","last_peered":"2024-04-17T10:49:21.658078+0000","last_clean":"2024-04-17T10:49:21.658078+0000","last_became_active":"2024-04-17T10:49:20.651343+0000","last_became_peered":"2024-04-17T10:49:20.651343+0000","last_unstale":"2024-04-17T10:49:21.658078+0000","last_undegraded":"2024-04-17T10:49:21.658078+0000","last_fullsized":"2024-04-17T10:49:21.658078+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:02:02.176762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"126'3","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.699983+0000","last_change":"2024-04-17T10:49:19.635729+0000","last_active":"2024-04-17T10:49:21.699983+0000","last_peered":"2024-04-17T10:49:21.699983+0000","last_clean":"2024-04-17T10:49:21.699983+0000","last_became_active":"2024-04-17T10:49:19.635152+0000","last_became_peered":"2024-04-17T10:49:19.635152+0000","last_unstale":"2024-04-17T10:49:21.699983+0000","last_undegraded":"2024-04-17T10:49:21.699983+0000","last_fullsized":"2024-04-17T10:49:21.699983+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:40:25.531877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.656073+0000","last_change":"2024-04-17T10:49:20.641864+0000","last_active":"2024-04-17T10:49:21.656073+0000","last_peered":"2024-04-17T10:49:21.656073+0000","last_clean":"2024-04-17T10:49:21.656073+0000","last_became_active":"2024-04-17T10:49:20.641143+0000","last_became_peered":"2024-04-17T10:49:20.641143+0000","last_unstale":"2024-04-17T10:49:21.656073+0000","last_undegraded":"2024-04-17T10:49:21.656073+0000","last_fullsized":"2024-04-17T10:49:21.656073+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:32:30.053642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"126'6","reported_seq":20,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.657086+0000","last_change":"2024-04-17T10:49:19.634477+0000","last_active":"2024-04-17T10:49:21.657086+0000","last_peered":"2024-04-17T10:49:21.657086+0000","last_clean":"2024-04-17T10:49:21.657086+0000","last_became_active":"2024-04-17T10:49:19.629537+0000","last_became_peered":"2024-04-17T10:49:19.629537+0000","last_unstale":"2024-04-17T10:49:21.657086+0000","last_undegraded":"2024-04-17T10:49:21.657086+0000","last_fullsized":"2024-04-17T10:49:21.657086+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:49:44.750937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.839371+0000","last_change":"2024-04-17T10:49:20.649492+0000","last_active":"2024-04-17T10:49:22.839371+0000","last_peered":"2024-04-17T10:49:22.839371+0000","last_clean":"2024-04-17T10:49:22.839371+0000","last_became_active":"2024-04-17T10:49:20.649158+0000","last_became_peered":"2024-04-17T10:49:20.649158+0000","last_unstale":"2024-04-17T10:49:22.839371+0000","last_undegraded":"2024-04-17T10:49:22.839371+0000","last_fullsized":"2024-04-17T10:49:22.839371+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:55:43.464505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.839186+0000","last_change":"2024-04-17T10:49:19.618690+0000","last_active":"2024-04-17T10:49:22.839186+0000","last_peered":"2024-04-17T10:49:22.839186+0000","last_clean":"2024-04-17T10:49:22.839186+0000","last_became_active":"2024-04-17T10:49:19.618112+0000","last_became_peered":"2024-04-17T10:49:19.618112+0000","last_unstale":"2024-04-17T10:49:22.839186+0000","last_undegraded":"2024-04-17T10:49:22.839186+0000","last_fullsized":"2024-04-17T10:49:22.839186+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:28:09.142754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.655453+0000","last_change":"2024-04-17T10:49:20.654064+0000","last_active":"2024-04-17T10:49:21.655453+0000","last_peered":"2024-04-17T10:49:21.655453+0000","last_clean":"2024-04-17T10:49:21.655453+0000","last_became_active":"2024-04-17T10:49:20.653818+0000","last_became_peered":"2024-04-17T10:49:20.653818+0000","last_unstale":"2024-04-17T10:49:21.655453+0000","last_undegraded":"2024-04-17T10:49:21.655453+0000","last_fullsized":"2024-04-17T10:49:21.655453+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:46:39.896263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.655478+0000","last_change":"2024-04-17T10:49:19.629396+0000","last_active":"2024-04-17T10:49:21.655478+0000","last_peered":"2024-04-17T10:49:21.655478+0000","last_clean":"2024-04-17T10:49:21.655478+0000","last_became_active":"2024-04-17T10:49:19.627077+0000","last_became_peered":"2024-04-17T10:49:19.627077+0000","last_unstale":"2024-04-17T10:49:21.655478+0000","last_undegraded":"2024-04-17T10:49:21.655478+0000","last_fullsized":"2024-04-17T10:49:21.655478+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:23:19.984810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.656737+0000","last_change":"2024-04-17T10:49:20.677538+0000","last_active":"2024-04-17T10:49:21.656737+0000","last_peered":"2024-04-17T10:49:21.656737+0000","last_clean":"2024-04-17T10:49:21.656737+0000","last_became_active":"2024-04-17T10:49:20.675711+0000","last_became_peered":"2024-04-17T10:49:20.675711+0000","last_unstale":"2024-04-17T10:49:21.656737+0000","last_undegraded":"2024-04-17T10:49:21.656737+0000","last_fullsized":"2024-04-17T10:49:21.656737+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:52:53.006876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"126'8","reported_seq":22,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.656034+0000","last_change":"2024-04-17T10:49:19.639007+0000","last_active":"2024-04-17T10:49:21.656034+0000","last_peered":"2024-04-17T10:49:21.656034+0000","last_clean":"2024-04-17T10:49:21.656034+0000","last_became_active":"2024-04-17T10:49:19.637054+0000","last_became_peered":"2024-04-17T10:49:19.637054+0000","last_unstale":"2024-04-17T10:49:21.656034+0000","last_undegraded":"2024-04-17T10:49:21.656034+0000","last_fullsized":"2024-04-17T10:49:21.656034+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:52:13.470951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.657708+0000","last_change":"2024-04-17T10:49:20.636094+0000","last_active":"2024-04-17T10:49:21.657708+0000","last_peered":"2024-04-17T10:49:21.657708+0000","last_clean":"2024-04-17T10:49:21.657708+0000","last_became_active":"2024-04-17T10:49:20.635302+0000","last_became_peered":"2024-04-17T10:49:20.635302+0000","last_unstale":"2024-04-17T10:49:21.657708+0000","last_undegraded":"2024-04-17T10:49:21.657708+0000","last_fullsized":"2024-04-17T10:49:21.657708+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:18:51.247550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"126'1","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.700531+0000","last_change":"2024-04-17T10:49:19.633718+0000","last_active":"2024-04-17T10:49:21.700531+0000","last_peered":"2024-04-17T10:49:21.700531+0000","last_clean":"2024-04-17T10:49:21.700531+0000","last_became_active":"2024-04-17T10:49:19.631782+0000","last_became_peered":"2024-04-17T10:49:19.631782+0000","last_unstale":"2024-04-17T10:49:21.700531+0000","last_undegraded":"2024-04-17T10:49:21.700531+0000","last_fullsized":"2024-04-17T10:49:21.700531+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:44:34.436545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.700406+0000","last_change":"2024-04-17T10:49:20.673735+0000","last_active":"2024-04-17T10:49:21.700406+0000","last_peered":"2024-04-17T10:49:21.700406+0000","last_clean":"2024-04-17T10:49:21.700406+0000","last_became_active":"2024-04-17T10:49:20.672578+0000","last_became_peered":"2024-04-17T10:49:20.672578+0000","last_unstale":"2024-04-17T10:49:21.700406+0000","last_undegraded":"2024-04-17T10:49:21.700406+0000","last_fullsized":"2024-04-17T10:49:21.700406+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:18:09.247335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"126'2","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.308294+0000","last_change":"2024-04-17T10:49:19.634375+0000","last_active":"2024-04-17T10:49:22.308294+0000","last_peered":"2024-04-17T10:49:22.308294+0000","last_clean":"2024-04-17T10:49:22.308294+0000","last_became_active":"2024-04-17T10:49:19.634064+0000","last_became_peered":"2024-04-17T10:49:19.634064+0000","last_unstale":"2024-04-17T10:49:22.308294+0000","last_undegraded":"2024-04-17T10:49:22.308294+0000","last_fullsized":"2024-04-17T10:49:22.308294+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:42:48.327411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.412274+0000","last_change":"2024-04-17T10:49:20.649199+0000","last_active":"2024-04-17T10:49:22.412274+0000","last_peered":"2024-04-17T10:49:22.412274+0000","last_clean":"2024-04-17T10:49:22.412274+0000","last_became_active":"2024-04-17T10:49:20.647551+0000","last_became_peered":"2024-04-17T10:49:20.647551+0000","last_unstale":"2024-04-17T10:49:22.412274+0000","last_undegraded":"2024-04-17T10:49:22.412274+0000","last_fullsized":"2024-04-17T10:49:22.412274+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:31:50.340912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"126'1","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.593843+0000","last_change":"2024-04-17T10:49:19.626246+0000","last_active":"2024-04-17T10:49:22.593843+0000","last_peered":"2024-04-17T10:49:22.593843+0000","last_clean":"2024-04-17T10:49:22.593843+0000","last_became_active":"2024-04-17T10:49:19.625926+0000","last_became_peered":"2024-04-17T10:49:19.625926+0000","last_unstale":"2024-04-17T10:49:22.593843+0000","last_undegraded":"2024-04-17T10:49:22.593843+0000","last_fullsized":"2024-04-17T10:49:22.593843+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:34:14.904567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.656012+0000","last_change":"2024-04-17T10:49:20.651824+0000","last_active":"2024-04-17T10:49:21.656012+0000","last_peered":"2024-04-17T10:49:21.656012+0000","last_clean":"2024-04-17T10:49:21.656012+0000","last_became_active":"2024-04-17T10:49:20.650616+0000","last_became_peered":"2024-04-17T10:49:20.650616+0000","last_unstale":"2024-04-17T10:49:21.656012+0000","last_undegraded":"2024-04-17T10:49:21.656012+0000","last_fullsized":"2024-04-17T10:49:21.656012+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:13:05.557158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.700055+0000","last_change":"2024-04-17T10:49:19.626019+0000","last_active":"2024-04-17T10:49:21.700055+0000","last_peered":"2024-04-17T10:49:21.700055+0000","last_clean":"2024-04-17T10:49:21.700055+0000","last_became_active":"2024-04-17T10:49:19.625217+0000","last_became_peered":"2024-04-17T10:49:19.625217+0000","last_unstale":"2024-04-17T10:49:21.700055+0000","last_undegraded":"2024-04-17T10:49:21.700055+0000","last_fullsized":"2024-04-17T10:49:21.700055+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:12:39.211951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.839191+0000","last_change":"2024-04-17T10:49:20.679333+0000","last_active":"2024-04-17T10:49:22.839191+0000","last_peered":"2024-04-17T10:49:22.839191+0000","last_clean":"2024-04-17T10:49:22.839191+0000","last_became_active":"2024-04-17T10:49:20.679003+0000","last_became_peered":"2024-04-17T10:49:20.679003+0000","last_unstale":"2024-04-17T10:49:22.839191+0000","last_undegraded":"2024-04-17T10:49:22.839191+0000","last_fullsized":"2024-04-17T10:49:22.839191+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:48.350486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"126'2","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.699789+0000","last_change":"2024-04-17T10:49:19.623322+0000","last_active":"2024-04-17T10:49:21.699789+0000","last_peered":"2024-04-17T10:49:21.699789+0000","last_clean":"2024-04-17T10:49:21.699789+0000","last_became_active":"2024-04-17T10:49:19.622547+0000","last_became_peered":"2024-04-17T10:49:19.622547+0000","last_unstale":"2024-04-17T10:49:21.699789+0000","last_undegraded":"2024-04-17T10:49:21.699789+0000","last_fullsized":"2024-04-17T10:49:21.699789+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:49:44.750937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.593835+0000","last_change":"2024-04-17T10:49:20.658450+0000","last_active":"2024-04-17T10:49:22.593835+0000","last_peered":"2024-04-17T10:49:22.593835+0000","last_clean":"2024-04-17T10:49:22.593835+0000","last_became_active":"2024-04-17T10:49:20.657632+0000","last_became_peered":"2024-04-17T10:49:20.657632+0000","last_unstale":"2024-04-17T10:49:22.593835+0000","last_undegraded":"2024-04-17T10:49:22.593835+0000","last_fullsized":"2024-04-17T10:49:22.593835+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:49:59.308085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.411768+0000","last_change":"2024-04-17T10:49:19.639310+0000","last_active":"2024-04-17T10:49:22.411768+0000","last_peered":"2024-04-17T10:49:22.411768+0000","last_clean":"2024-04-17T10:49:22.411768+0000","last_became_active":"2024-04-17T10:49:19.638990+0000","last_became_peered":"2024-04-17T10:49:19.638990+0000","last_unstale":"2024-04-17T10:49:22.411768+0000","last_undegraded":"2024-04-17T10:49:22.411768+0000","last_fullsized":"2024-04-17T10:49:22.411768+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:19:20.437632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.655459+0000","last_change":"2024-04-17T10:49:20.650146+0000","last_active":"2024-04-17T10:49:21.655459+0000","last_peered":"2024-04-17T10:49:21.655459+0000","last_clean":"2024-04-17T10:49:21.655459+0000","last_became_active":"2024-04-17T10:49:20.649800+0000","last_became_peered":"2024-04-17T10:49:20.649800+0000","last_unstale":"2024-04-17T10:49:21.655459+0000","last_undegraded":"2024-04-17T10:49:21.655459+0000","last_fullsized":"2024-04-17T10:49:21.655459+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:20:27.655870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.411290+0000","last_change":"2024-04-17T10:49:19.637010+0000","last_active":"2024-04-17T10:49:22.411290+0000","last_peered":"2024-04-17T10:49:22.411290+0000","last_clean":"2024-04-17T10:49:22.411290+0000","last_became_active":"2024-04-17T10:49:19.636648+0000","last_became_peered":"2024-04-17T10:49:19.636648+0000","last_unstale":"2024-04-17T10:49:22.411290+0000","last_undegraded":"2024-04-17T10:49:22.411290+0000","last_fullsized":"2024-04-17T10:49:22.411290+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:40:34.190480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.655990+0000","last_change":"2024-04-17T10:49:20.650342+0000","last_active":"2024-04-17T10:49:21.655990+0000","last_peered":"2024-04-17T10:49:21.655990+0000","last_clean":"2024-04-17T10:49:21.655990+0000","last_became_active":"2024-04-17T10:49:20.649605+0000","last_became_peered":"2024-04-17T10:49:20.649605+0000","last_unstale":"2024-04-17T10:49:21.655990+0000","last_undegraded":"2024-04-17T10:49:21.655990+0000","last_fullsized":"2024-04-17T10:49:21.655990+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:09:35.357557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.307564+0000","last_change":"2024-04-17T10:49:19.637022+0000","last_active":"2024-04-17T10:49:22.307564+0000","last_peered":"2024-04-17T10:49:22.307564+0000","last_clean":"2024-04-17T10:49:22.307564+0000","last_became_active":"2024-04-17T10:49:19.636699+0000","last_became_peered":"2024-04-17T10:49:19.636699+0000","last_unstale":"2024-04-17T10:49:22.307564+0000","last_undegraded":"2024-04-17T10:49:22.307564+0000","last_fullsized":"2024-04-17T10:49:22.307564+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:39:14.915890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.594281+0000","last_change":"2024-04-17T10:49:20.659370+0000","last_active":"2024-04-17T10:49:22.594281+0000","last_peered":"2024-04-17T10:49:22.594281+0000","last_clean":"2024-04-17T10:49:22.594281+0000","last_became_active":"2024-04-17T10:49:20.658659+0000","last_became_peered":"2024-04-17T10:49:20.658659+0000","last_unstale":"2024-04-17T10:49:22.594281+0000","last_undegraded":"2024-04-17T10:49:22.594281+0000","last_fullsized":"2024-04-17T10:49:22.594281+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:28:17.451627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"126'2","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.839624+0000","last_change":"2024-04-17T10:49:19.634764+0000","last_active":"2024-04-17T10:49:22.839624+0000","last_peered":"2024-04-17T10:49:22.839624+0000","last_clean":"2024-04-17T10:49:22.839624+0000","last_became_active":"2024-04-17T10:49:19.634325+0000","last_became_peered":"2024-04-17T10:49:19.634325+0000","last_unstale":"2024-04-17T10:49:22.839624+0000","last_undegraded":"2024-04-17T10:49:22.839624+0000","last_fullsized":"2024-04-17T10:49:22.839624+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:47:11.460301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.839562+0000","last_change":"2024-04-17T10:49:20.646495+0000","last_active":"2024-04-17T10:49:22.839562+0000","last_peered":"2024-04-17T10:49:22.839562+0000","last_clean":"2024-04-17T10:49:22.839562+0000","last_became_active":"2024-04-17T10:49:20.645519+0000","last_became_peered":"2024-04-17T10:49:20.645519+0000","last_unstale":"2024-04-17T10:49:22.839562+0000","last_undegraded":"2024-04-17T10:49:22.839562+0000","last_fullsized":"2024-04-17T10:49:22.839562+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:02:49.915712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"126'1","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.656123+0000","last_change":"2024-04-17T10:49:19.617903+0000","last_active":"2024-04-17T10:49:21.656123+0000","last_peered":"2024-04-17T10:49:21.656123+0000","last_clean":"2024-04-17T10:49:21.656123+0000","last_became_active":"2024-04-17T10:49:19.614491+0000","last_became_peered":"2024-04-17T10:49:19.614491+0000","last_unstale":"2024-04-17T10:49:21.656123+0000","last_undegraded":"2024-04-17T10:49:21.656123+0000","last_fullsized":"2024-04-17T10:49:21.656123+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:19:20.437632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.655826+0000","last_change":"2024-04-17T10:49:20.650536+0000","last_active":"2024-04-17T10:49:21.655826+0000","last_peered":"2024-04-17T10:49:21.655826+0000","last_clean":"2024-04-17T10:49:21.655826+0000","last_became_active":"2024-04-17T10:49:20.650202+0000","last_became_peered":"2024-04-17T10:49:20.650202+0000","last_unstale":"2024-04-17T10:49:21.655826+0000","last_undegraded":"2024-04-17T10:49:21.655826+0000","last_fullsized":"2024-04-17T10:49:21.655826+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:19:26.194611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.656062+0000","last_change":"2024-04-17T10:49:19.636371+0000","last_active":"2024-04-17T10:49:21.656062+0000","last_peered":"2024-04-17T10:49:21.656062+0000","last_clean":"2024-04-17T10:49:21.656062+0000","last_became_active":"2024-04-17T10:49:19.635705+0000","last_became_peered":"2024-04-17T10:49:19.635705+0000","last_unstale":"2024-04-17T10:49:21.656062+0000","last_undegraded":"2024-04-17T10:49:21.656062+0000","last_fullsized":"2024-04-17T10:49:21.656062+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:47.341443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.593971+0000","last_change":"2024-04-17T10:49:20.670622+0000","last_active":"2024-04-17T10:49:22.593971+0000","last_peered":"2024-04-17T10:49:22.593971+0000","last_clean":"2024-04-17T10:49:22.593971+0000","last_became_active":"2024-04-17T10:49:20.668895+0000","last_became_peered":"2024-04-17T10:49:20.668895+0000","last_unstale":"2024-04-17T10:49:22.593971+0000","last_undegraded":"2024-04-17T10:49:22.593971+0000","last_fullsized":"2024-04-17T10:49:22.593971+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:51:52.124354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.307750+0000","last_change":"2024-04-17T10:49:19.631724+0000","last_active":"2024-04-17T10:49:22.307750+0000","last_peered":"2024-04-17T10:49:22.307750+0000","last_clean":"2024-04-17T10:49:22.307750+0000","last_became_active":"2024-04-17T10:49:19.631389+0000","last_became_peered":"2024-04-17T10:49:19.631389+0000","last_unstale":"2024-04-17T10:49:22.307750+0000","last_undegraded":"2024-04-17T10:49:22.307750+0000","last_fullsized":"2024-04-17T10:49:22.307750+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:27:52.266342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.655353+0000","last_change":"2024-04-17T10:49:20.679862+0000","last_active":"2024-04-17T10:49:21.655353+0000","last_peered":"2024-04-17T10:49:21.655353+0000","last_clean":"2024-04-17T10:49:21.655353+0000","last_became_active":"2024-04-17T10:49:20.678569+0000","last_became_peered":"2024-04-17T10:49:20.678569+0000","last_unstale":"2024-04-17T10:49:21.655353+0000","last_undegraded":"2024-04-17T10:49:21.655353+0000","last_fullsized":"2024-04-17T10:49:21.655353+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:14:35.759031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.655643+0000","last_change":"2024-04-17T10:49:19.630151+0000","last_active":"2024-04-17T10:49:21.655643+0000","last_peered":"2024-04-17T10:49:21.655643+0000","last_clean":"2024-04-17T10:49:21.655643+0000","last_became_active":"2024-04-17T10:49:19.629807+0000","last_became_peered":"2024-04-17T10:49:19.629807+0000","last_unstale":"2024-04-17T10:49:21.655643+0000","last_undegraded":"2024-04-17T10:49:21.655643+0000","last_fullsized":"2024-04-17T10:49:21.655643+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:43:05.325714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.700049+0000","last_change":"2024-04-17T10:49:20.648839+0000","last_active":"2024-04-17T10:49:21.700049+0000","last_peered":"2024-04-17T10:49:21.700049+0000","last_clean":"2024-04-17T10:49:21.700049+0000","last_became_active":"2024-04-17T10:49:20.648142+0000","last_became_peered":"2024-04-17T10:49:20.648142+0000","last_unstale":"2024-04-17T10:49:21.700049+0000","last_undegraded":"2024-04-17T10:49:21.700049+0000","last_fullsized":"2024-04-17T10:49:21.700049+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:11:45.378186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.655278+0000","last_change":"2024-04-17T10:49:19.628673+0000","last_active":"2024-04-17T10:49:21.655278+0000","last_peered":"2024-04-17T10:49:21.655278+0000","last_clean":"2024-04-17T10:49:21.655278+0000","last_became_active":"2024-04-17T10:49:19.627044+0000","last_became_peered":"2024-04-17T10:49:19.627044+0000","last_unstale":"2024-04-17T10:49:21.655278+0000","last_undegraded":"2024-04-17T10:49:21.655278+0000","last_fullsized":"2024-04-17T10:49:21.655278+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:54.418708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.412614+0000","last_change":"2024-04-17T10:49:20.660068+0000","last_active":"2024-04-17T10:49:22.412614+0000","last_peered":"2024-04-17T10:49:22.412614+0000","last_clean":"2024-04-17T10:49:22.412614+0000","last_became_active":"2024-04-17T10:49:20.658828+0000","last_became_peered":"2024-04-17T10:49:20.658828+0000","last_unstale":"2024-04-17T10:49:22.412614+0000","last_undegraded":"2024-04-17T10:49:22.412614+0000","last_fullsized":"2024-04-17T10:49:22.412614+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:32:30.053642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.411385+0000","last_change":"2024-04-17T10:49:19.638400+0000","last_active":"2024-04-17T10:49:22.411385+0000","last_peered":"2024-04-17T10:49:22.411385+0000","last_clean":"2024-04-17T10:49:22.411385+0000","last_became_active":"2024-04-17T10:49:19.638049+0000","last_became_peered":"2024-04-17T10:49:19.638049+0000","last_unstale":"2024-04-17T10:49:22.411385+0000","last_undegraded":"2024-04-17T10:49:22.411385+0000","last_fullsized":"2024-04-17T10:49:22.411385+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:23:21.981832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.655533+0000","last_change":"2024-04-17T10:49:20.650419+0000","last_active":"2024-04-17T10:49:21.655533+0000","last_peered":"2024-04-17T10:49:21.655533+0000","last_clean":"2024-04-17T10:49:21.655533+0000","last_became_active":"2024-04-17T10:49:20.650102+0000","last_became_peered":"2024-04-17T10:49:20.650102+0000","last_unstale":"2024-04-17T10:49:21.655533+0000","last_undegraded":"2024-04-17T10:49:21.655533+0000","last_fullsized":"2024-04-17T10:49:21.655533+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:25:40.450492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:22.308413+0000","last_change":"2024-04-17T10:49:19.625628+0000","last_active":"2024-04-17T10:49:22.308413+0000","last_peered":"2024-04-17T10:49:22.308413+0000","last_clean":"2024-04-17T10:49:22.308413+0000","last_became_active":"2024-04-17T10:49:19.624710+0000","last_became_peered":"2024-04-17T10:49:19.624710+0000","last_unstale":"2024-04-17T10:49:22.308413+0000","last_undegraded":"2024-04-17T10:49:22.308413+0000","last_fullsized":"2024-04-17T10:49:22.308413+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:15:01.882215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.655129+0000","last_change":"2024-04-17T10:49:20.644683+0000","last_active":"2024-04-17T10:49:21.655129+0000","last_peered":"2024-04-17T10:49:21.655129+0000","last_clean":"2024-04-17T10:49:21.655129+0000","last_became_active":"2024-04-17T10:49:20.643984+0000","last_became_peered":"2024-04-17T10:49:20.643984+0000","last_unstale":"2024-04-17T10:49:21.655129+0000","last_undegraded":"2024-04-17T10:49:21.655129+0000","last_fullsized":"2024-04-17T10:49:21.655129+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:19.595335+0000","last_clean_scrub_stamp":"2024-04-17T10:49:19.595335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:02:18.688697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"126'2","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.656038+0000","last_change":"2024-04-17T10:49:19.623140+0000","last_active":"2024-04-17T10:49:21.656038+0000","last_peered":"2024-04-17T10:49:21.656038+0000","last_clean":"2024-04-17T10:49:21.656038+0000","last_became_active":"2024-04-17T10:49:19.622247+0000","last_became_peered":"2024-04-17T10:49:19.622247+0000","last_unstale":"2024-04-17T10:49:21.656038+0000","last_undegraded":"2024-04-17T10:49:21.656038+0000","last_fullsized":"2024-04-17T10:49:21.656038+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:49:18.586291+0000","last_clean_scrub_stamp":"2024-04-17T10:49:18.586291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:32:04.769207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":389,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-17T10:49:21.654739+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:49:21.654739+0000","last_peered":"2024-04-17T10:49:21.654739+0000","last_clean":"2024-04-17T10:49:21.654739+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:49:21.654739+0000","last_undegraded":"2024-04-17T10:49:21.654739+0000","last_fullsized":"2024-04-17T10:49:21.654739+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574989,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47224,"kb_used_data":2152,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324616,"statfs":{"total":96636764160,"available":96588406784,"internally_reserved":0,"allocated":2203648,"data_stored":1284493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574988,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68840,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303000,"statfs":{"total":96636764160,"available":96566272000,"internally_reserved":0,"allocated":1597440,"data_stored":693487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574989,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54948,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316892,"statfs":{"total":96636764160,"available":96580497408,"internally_reserved":0,"allocated":1593344,"data_stored":691987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574988,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72988,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298852,"statfs":{"total":96636764160,"available":96562024448,"internally_reserved":0,"allocated":1589248,"data_stored":691945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574990,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72996,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298844,"statfs":{"total":96636764160,"available":96562016256,"internally_reserved":0,"allocated":1593344,"data_stored":692035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574988,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59104,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312736,"statfs":{"total":96636764160,"available":96576241664,"internally_reserved":0,"allocated":1593344,"data_stored":692491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574988,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95192,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94276648,"statfs":{"total":96636764160,"available":96539287552,"internally_reserved":0,"allocated":1585152,"data_stored":691875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574988,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47196,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324644,"statfs":{"total":96636764160,"available":96588435456,"internally_reserved":0,"allocated":2179072,"data_stored":1282243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:49:28.448 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-17T10:49:28.449 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-17T10:49:28.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.625+0000 7fc5b35bf640 1 Processor -- start 2024-04-17T10:49:28.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.625+0000 7fc5b35bf640 1 -- start start 2024-04-17T10:49:28.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.625+0000 7fc5b35bf640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc5ac161ee0 0x7fc5ac06cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:28.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.625+0000 7fc5b35bf640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5ac1622c0 0x7fc5ac06ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:28.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.625+0000 7fc5b35bf640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc5ac0ae390 0x7fc5ac0acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:28.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.625+0000 7fc5b35bf640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5ac0cf120 con 0x7fc5ac0ae390 2024-04-17T10:49:28.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.625+0000 7fc5b35bf640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5ac0cf370 con 0x7fc5ac161ee0 2024-04-17T10:49:28.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.625+0000 7fc5b35bf640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac0cf650 con 0x7fc5ac1622c0 2024-04-17T10:49:28.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.625+0000 7fc5b1b5c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc5ac0ae390 0x7fc5ac0acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:28.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.625+0000 7fc5b1b5c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc5ac0ae390 0x7fc5ac0acf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57898/0 (socket says 172.21.15.83:57898) 2024-04-17T10:49:28.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.625+0000 7fc5b1b5c640 1 -- 172.21.15.83:0/3127377025 learned_addr learned my addr 172.21.15.83:0/3127377025 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:28.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.625+0000 7fc5b135b640 1 --2- 172.21.15.83:0/3127377025 >> v2:172.21.15.167:3300/0 conn(0x7fc5ac161ee0 0x7fc5ac06cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:28.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.625+0000 7fc5b0b5a640 1 --2- 172.21.15.83:0/3127377025 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5ac1622c0 0x7fc5ac06ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:28.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.625+0000 7fc5b1b5c640 1 -- 172.21.15.83:0/3127377025 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5ac1622c0 msgr2=0x7fc5ac06ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:28.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.625+0000 7fc5b1b5c640 1 --2- 172.21.15.83:0/3127377025 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5ac1622c0 0x7fc5ac06ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:28.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.625+0000 7fc5b1b5c640 1 -- 172.21.15.83:0/3127377025 >> v2:172.21.15.167:3300/0 conn(0x7fc5ac161ee0 msgr2=0x7fc5ac06cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:28.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.625+0000 7fc5b1b5c640 1 --2- 172.21.15.83:0/3127377025 >> v2:172.21.15.167:3300/0 conn(0x7fc5ac161ee0 0x7fc5ac06cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:28.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.625+0000 7fc5b1b5c640 1 -- 172.21.15.83:0/3127377025 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ac0cf930 con 0x7fc5ac0ae390 2024-04-17T10:49:28.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.629+0000 7fc5b1b5c640 1 --2- 172.21.15.83:0/3127377025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5ac0ae390 0x7fc5ac0acf30 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac066ba0 tx=0x7fc5a801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6199553bad1ee5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:28.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.629+0000 7fc5a3fff640 1 -- 172.21.15.83:0/3127377025 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a80b0070 con 0x7fc5ac0ae390 2024-04-17T10:49:28.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.629+0000 7fc5a3fff640 1 -- 172.21.15.83:0/3127377025 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a80ab600 con 0x7fc5ac0ae390 2024-04-17T10:49:28.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.629+0000 7fc5a3fff640 1 -- 172.21.15.83:0/3127377025 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a80ab9d0 con 0x7fc5ac0ae390 2024-04-17T10:49:28.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.629+0000 7fc5b35bf640 1 -- 172.21.15.83:0/3127377025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5ac0ae390 msgr2=0x7fc5ac0acf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:28.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.629+0000 7fc5b35bf640 1 --2- 172.21.15.83:0/3127377025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5ac0ae390 0x7fc5ac0acf30 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac066ba0 tx=0x7fc5a801c430 comp rx=0 tx=0).stop 2024-04-17T10:49:28.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.629+0000 7fc5b35bf640 1 -- 172.21.15.83:0/3127377025 shutdown_connections 2024-04-17T10:49:28.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.629+0000 7fc5b35bf640 1 --2- 172.21.15.83:0/3127377025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5ac0ae390 0x7fc5ac0acf30 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:28.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.629+0000 7fc5b35bf640 1 --2- 172.21.15.83:0/3127377025 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5ac1622c0 0x7fc5ac06ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:28.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.629+0000 7fc5b35bf640 1 --2- 172.21.15.83:0/3127377025 >> v2:172.21.15.167:3300/0 conn(0x7fc5ac161ee0 0x7fc5ac06cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:28.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.629+0000 7fc5b35bf640 1 -- 172.21.15.83:0/3127377025 >> 172.21.15.83:0/3127377025 conn(0x7fc5ac0b1e20 msgr2=0x7fc5ac0b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:28.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.629+0000 7fc5b35bf640 1 -- 172.21.15.83:0/3127377025 shutdown_connections 2024-04-17T10:49:28.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.629+0000 7fc5b35bf640 1 -- 172.21.15.83:0/3127377025 wait complete. 2024-04-17T10:49:28.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.629+0000 7fc5b35bf640 1 Processor -- start 2024-04-17T10:49:28.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.629+0000 7fc5b35bf640 1 -- start start 2024-04-17T10:49:28.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.629+0000 7fc5b35bf640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5ac0ae390 0x7fc5ac1626a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:28.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.629+0000 7fc5b35bf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5ac161ee0 0x7fc5ac072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:28.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.629+0000 7fc5b35bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5ac1622c0 0x7fc5ac154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:28.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.629+0000 7fc5b35bf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac06ba40 con 0x7fc5ac1622c0 2024-04-17T10:49:28.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.629+0000 7fc5b35bf640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac06b8c0 con 0x7fc5ac161ee0 2024-04-17T10:49:28.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.629+0000 7fc5b35bf640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac06bbc0 con 0x7fc5ac0ae390 2024-04-17T10:49:28.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.629+0000 7fc5b1b5c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5ac1622c0 0x7fc5ac154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:28.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.633+0000 7fc5b1b5c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5ac1622c0 0x7fc5ac154e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57908/0 (socket says 172.21.15.83:57908) 2024-04-17T10:49:28.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.633+0000 7fc5b1b5c640 1 -- 172.21.15.83:0/2767700420 learned_addr learned my addr 172.21.15.83:0/2767700420 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:28.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.633+0000 7fc5b135b640 1 --2- 172.21.15.83:0/2767700420 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5ac0ae390 0x7fc5ac1626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:28.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.633+0000 7fc5b0b5a640 1 --2- 172.21.15.83:0/2767700420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5ac161ee0 0x7fc5ac072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:28.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.633+0000 7fc5b1b5c640 1 -- 172.21.15.83:0/2767700420 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5ac0ae390 msgr2=0x7fc5ac1626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:28.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.633+0000 7fc5b1b5c640 1 --2- 172.21.15.83:0/2767700420 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5ac0ae390 0x7fc5ac1626a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:28.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.633+0000 7fc5b1b5c640 1 -- 172.21.15.83:0/2767700420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5ac161ee0 msgr2=0x7fc5ac072ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:28.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.633+0000 7fc5b1b5c640 1 --2- 172.21.15.83:0/2767700420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5ac161ee0 0x7fc5ac072ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:28.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.633+0000 7fc5b1b5c640 1 -- 172.21.15.83:0/2767700420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ac073020 con 0x7fc5ac1622c0 2024-04-17T10:49:28.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.633+0000 7fc5b0b5a640 1 --2- 172.21.15.83:0/2767700420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5ac161ee0 0x7fc5ac072ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:28.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.633+0000 7fc5b1b5c640 1 --2- 172.21.15.83:0/2767700420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5ac1622c0 0x7fc5ac154e10 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fc5a80ae040 tx=0x7fc5a801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:28.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.633+0000 7fc5a1ffb640 1 -- 172.21.15.83:0/2767700420 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a80ba020 con 0x7fc5ac1622c0 2024-04-17T10:49:28.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.633+0000 7fc5a1ffb640 1 -- 172.21.15.83:0/2767700420 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a80b6420 con 0x7fc5ac1622c0 2024-04-17T10:49:28.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.633+0000 7fc5b35bf640 1 -- 172.21.15.83:0/2767700420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5ac15f670 con 0x7fc5ac1622c0 2024-04-17T10:49:28.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.633+0000 7fc5a1ffb640 1 -- 172.21.15.83:0/2767700420 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a80b0040 con 0x7fc5ac1622c0 2024-04-17T10:49:28.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.633+0000 7fc5b35bf640 1 -- 172.21.15.83:0/2767700420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5ac165130 con 0x7fc5ac1622c0 2024-04-17T10:49:28.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.633+0000 7fc5b35bf640 1 -- 172.21.15.83:0/2767700420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5ac15f840 con 0x7fc5ac1622c0 2024-04-17T10:49:28.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.633+0000 7fc5a1ffb640 1 -- 172.21.15.83:0/2767700420 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc5a80bd020 con 0x7fc5ac1622c0 2024-04-17T10:49:28.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.633+0000 7fc5a1ffb640 1 --2- 172.21.15.83:0/2767700420 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc590066f60 0x7fc590069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:28.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.633+0000 7fc5a1ffb640 1 -- 172.21.15.83:0/2767700420 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5a811f280 con 0x7fc5ac1622c0 2024-04-17T10:49:28.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.633+0000 7fc5b135b640 1 --2- 172.21.15.83:0/2767700420 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc590066f60 0x7fc590069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:28.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.633+0000 7fc5b135b640 1 --2- 172.21.15.83:0/2767700420 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc590066f60 0x7fc590069420 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009eb0 tx=0x7fc5a400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:28.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.637+0000 7fc5a1ffb640 1 -- 172.21.15.83:0/2767700420 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc5a80c4030 con 0x7fc5ac1622c0 2024-04-17T10:49:28.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.857+0000 7fc5b35bf640 1 -- 172.21.15.83:0/2767700420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc5ac074f90 con 0x7fc5ac1622c0 2024-04-17T10:49:28.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.861+0000 7fc5a1ffb640 1 -- 172.21.15.83:0/2767700420 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+147 (secure 0 0 0) 0x7fc5a8016110 con 0x7fc5ac1622c0 2024-04-17T10:49:28.854 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-04-17T10:49:28.854 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBUph9mHGmXGBAAZF+JiAG9YrFtbW+dCy2IEQ== 2024-04-17T10:49:28.854 INFO:teuthology.orchestra.run.smithi083.stdout: caps mds = "allow" 2024-04-17T10:49:28.854 INFO:teuthology.orchestra.run.smithi083.stdout: caps mon = "allow r" 2024-04-17T10:49:28.854 INFO:teuthology.orchestra.run.smithi083.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-17T10:49:28.855 INFO:teuthology.orchestra.run.smithi083.stderr:updated caps for client.0 2024-04-17T10:49:28.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.865+0000 7fc5b35bf640 1 -- 172.21.15.83:0/2767700420 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc590066f60 msgr2=0x7fc590069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:28.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.865+0000 7fc5b35bf640 1 --2- 172.21.15.83:0/2767700420 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc590066f60 0x7fc590069420 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009eb0 tx=0x7fc5a400b040 comp rx=0 tx=0).stop 2024-04-17T10:49:28.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.865+0000 7fc5b35bf640 1 -- 172.21.15.83:0/2767700420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5ac1622c0 msgr2=0x7fc5ac154e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:28.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.865+0000 7fc5b35bf640 1 --2- 172.21.15.83:0/2767700420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5ac1622c0 0x7fc5ac154e10 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fc5a80ae040 tx=0x7fc5a801acd0 comp rx=0 tx=0).stop 2024-04-17T10:49:28.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.865+0000 7fc5b35bf640 1 -- 172.21.15.83:0/2767700420 shutdown_connections 2024-04-17T10:49:28.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.865+0000 7fc5b35bf640 1 --2- 172.21.15.83:0/2767700420 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc590066f60 0x7fc590069420 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:28.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.865+0000 7fc5b35bf640 1 --2- 172.21.15.83:0/2767700420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5ac1622c0 0x7fc5ac154e10 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:28.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.865+0000 7fc5b35bf640 1 --2- 172.21.15.83:0/2767700420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5ac161ee0 0x7fc5ac072ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:28.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.865+0000 7fc5b35bf640 1 --2- 172.21.15.83:0/2767700420 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5ac0ae390 0x7fc5ac1626a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:28.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.865+0000 7fc5b35bf640 1 -- 172.21.15.83:0/2767700420 >> 172.21.15.83:0/2767700420 conn(0x7fc5ac0b1e20 msgr2=0x7fc5ac0b0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:28.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.865+0000 7fc5b35bf640 1 -- 172.21.15.83:0/2767700420 shutdown_connections 2024-04-17T10:49:28.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.865+0000 7fc5b35bf640 1 -- 172.21.15.83:0/2767700420 wait complete. 2024-04-17T10:49:28.875 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:49:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.997+0000 7fe479d21640 1 Processor -- start 2024-04-17T10:49:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:28.997+0000 7fe479d21640 1 -- start start 2024-04-17T10:49:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.001+0000 7fe479d21640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe47415db50 0x7fe474070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.001+0000 7fe479d21640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe47406dcb0 0x7fe4740b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.001+0000 7fe479d21640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4740709b0 con 0x7fe4740aedd0 2024-04-17T10:49:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.001+0000 7fe479d21640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe47415e440 con 0x7fe47406dcb0 2024-04-17T10:49:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.001+0000 7fe479d21640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe47415e720 con 0x7fe47415db50 2024-04-17T10:49:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.001+0000 7fe473fff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe47406dcb0 0x7fe4740b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.001+0000 7fe4737fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe47415db50 0x7fe474070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.001+0000 7fe472ffd640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fe4740aedd0 0x7fe47406d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:39576/0 (socket says 172.21.15.83:39576) 2024-04-17T10:49:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.001+0000 7fe472ffd640 1 -- 172.21.15.83:0/4048980123 learned_addr learned my addr 172.21.15.83:0/4048980123 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:28.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.001+0000 7fe473fff640 1 -- 172.21.15.83:0/4048980123 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe47415db50 msgr2=0x7fe474070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:28.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.001+0000 7fe473fff640 1 --2- 172.21.15.83:0/4048980123 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe47415db50 0x7fe474070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:28.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.001+0000 7fe473fff640 1 -- 172.21.15.83:0/4048980123 >> v1:172.21.15.167:6789/0 conn(0x7fe4740aedd0 legacy=0x7fe47406d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:28.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.001+0000 7fe473fff640 1 -- 172.21.15.83:0/4048980123 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe47415ea00 con 0x7fe47406dcb0 2024-04-17T10:49:28.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.005+0000 7fe473fff640 1 --2- 172.21.15.83:0/4048980123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe47406dcb0 0x7fe4740b2e10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe46c0097c0 tx=0x7fe46c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b032fe4c06e127e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:28.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.005+0000 7fe4727fc640 1 -- 172.21.15.83:0/4048980123 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c0a7070 con 0x7fe47406dcb0 2024-04-17T10:49:28.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.005+0000 7fe4727fc640 1 -- 172.21.15.83:0/4048980123 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe46c0b0070 con 0x7fe47406dcb0 2024-04-17T10:49:28.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.005+0000 7fe4727fc640 1 -- 172.21.15.83:0/4048980123 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c0ab890 con 0x7fe47406dcb0 2024-04-17T10:49:28.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.005+0000 7fe479d21640 1 -- 172.21.15.83:0/4048980123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe47406dcb0 msgr2=0x7fe4740b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:28.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.005+0000 7fe479d21640 1 --2- 172.21.15.83:0/4048980123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe47406dcb0 0x7fe4740b2e10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe46c0097c0 tx=0x7fe46c01c470 comp rx=0 tx=0).stop 2024-04-17T10:49:28.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.005+0000 7fe479d21640 1 -- 172.21.15.83:0/4048980123 shutdown_connections 2024-04-17T10:49:28.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.005+0000 7fe479d21640 1 --2- 172.21.15.83:0/4048980123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe47406dcb0 0x7fe4740b2e10 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:28.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.005+0000 7fe479d21640 1 --2- 172.21.15.83:0/4048980123 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe47415db50 0x7fe474070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:28.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.005+0000 7fe479d21640 1 -- 172.21.15.83:0/4048980123 >> 172.21.15.83:0/4048980123 conn(0x7fe47406abe0 msgr2=0x7fe47406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:28.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.005+0000 7fe479d21640 1 -- 172.21.15.83:0/4048980123 shutdown_connections 2024-04-17T10:49:28.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.005+0000 7fe479d21640 1 -- 172.21.15.83:0/4048980123 wait complete. 2024-04-17T10:49:28.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.005+0000 7fe479d21640 1 Processor -- start 2024-04-17T10:49:28.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.005+0000 7fe479d21640 1 -- start start 2024-04-17T10:49:29.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.005+0000 7fe479d21640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe47406dcb0 0x7fe47415bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.005+0000 7fe479d21640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4740aedd0 0x7fe474156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.005+0000 7fe479d21640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe47415db50 0x7fe474144df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.005+0000 7fe479d21640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4740b3ce0 con 0x7fe47406dcb0 2024-04-17T10:49:29.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.005+0000 7fe479d21640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4740b3b60 con 0x7fe4740aedd0 2024-04-17T10:49:29.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.005+0000 7fe479d21640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4740b3e60 con 0x7fe47415db50 2024-04-17T10:49:29.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.005+0000 7fe4737fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe47406dcb0 0x7fe47415bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:29.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.005+0000 7fe4737fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe47406dcb0 0x7fe47415bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57918/0 (socket says 172.21.15.83:57918) 2024-04-17T10:49:29.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.005+0000 7fe4737fe640 1 -- 172.21.15.83:0/812347924 learned_addr learned my addr 172.21.15.83:0/812347924 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:29.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.005+0000 7fe472ffd640 1 --2- 172.21.15.83:0/812347924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4740aedd0 0x7fe474156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:29.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.005+0000 7fe473fff640 1 --2- 172.21.15.83:0/812347924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe47415db50 0x7fe474144df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:29.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.005+0000 7fe4737fe640 1 -- 172.21.15.83:0/812347924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe47415db50 msgr2=0x7fe474144df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:29.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.005+0000 7fe4737fe640 1 --2- 172.21.15.83:0/812347924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe47415db50 0x7fe474144df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.005+0000 7fe4737fe640 1 -- 172.21.15.83:0/812347924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4740aedd0 msgr2=0x7fe474156410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:29.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.005+0000 7fe4737fe640 1 --2- 172.21.15.83:0/812347924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4740aedd0 0x7fe474156410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.005+0000 7fe4737fe640 1 -- 172.21.15.83:0/812347924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4740a9980 con 0x7fe47406dcb0 2024-04-17T10:49:29.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.009+0000 7fe4737fe640 1 --2- 172.21.15.83:0/812347924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe47406dcb0 0x7fe47415bb10 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fe464009d10 tx=0x7fe46400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:29.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.009+0000 7fe46bfff640 1 -- 172.21.15.83:0/812347924 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe464018020 con 0x7fe47406dcb0 2024-04-17T10:49:29.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.009+0000 7fe46bfff640 1 -- 172.21.15.83:0/812347924 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe464014480 con 0x7fe47406dcb0 2024-04-17T10:49:29.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.009+0000 7fe479d21640 1 -- 172.21.15.83:0/812347924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4740a9c60 con 0x7fe47406dcb0 2024-04-17T10:49:29.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.009+0000 7fe46bfff640 1 -- 172.21.15.83:0/812347924 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe464014830 con 0x7fe47406dcb0 2024-04-17T10:49:29.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.009+0000 7fe479d21640 1 -- 172.21.15.83:0/812347924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4740aa220 con 0x7fe47406dcb0 2024-04-17T10:49:29.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.009+0000 7fe479d21640 1 -- 172.21.15.83:0/812347924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe474074f90 con 0x7fe47406dcb0 2024-04-17T10:49:29.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.009+0000 7fe46bfff640 1 -- 172.21.15.83:0/812347924 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe46401b020 con 0x7fe47406dcb0 2024-04-17T10:49:29.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.009+0000 7fe46bfff640 1 --2- 172.21.15.83:0/812347924 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe454066f60 0x7fe454069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.009+0000 7fe46bfff640 1 -- 172.21.15.83:0/812347924 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe464085af0 con 0x7fe47406dcb0 2024-04-17T10:49:29.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.009+0000 7fe472ffd640 1 --2- 172.21.15.83:0/812347924 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe454066f60 0x7fe454069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:29.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.009+0000 7fe472ffd640 1 --2- 172.21.15.83:0/812347924 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe454066f60 0x7fe454069420 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fe47415d300 tx=0x7fe45c007e90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:29.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.013+0000 7fe46bfff640 1 -- 172.21.15.83:0/812347924 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe464022070 con 0x7fe47406dcb0 2024-04-17T10:49:29.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.205+0000 7fe479d21640 1 -- 172.21.15.83:0/812347924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4740d0300 con 0x7fe47406dcb0 2024-04-17T10:49:29.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.205+0000 7fe46bfff640 1 -- 172.21.15.83:0/812347924 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fe464010040 con 0x7fe47406dcb0 2024-04-17T10:49:29.200 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:29.200 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5942,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3888166846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3888166846},{"type":"v1","addr":"172.21.15.167:6833","nonce":3888166846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":6772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1167913722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1167913722},{"type":"v1","addr":"172.21.15.167:6835","nonce":1167913722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":9192,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3552333026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3552333026},{"type":"v1","addr":"172.21.15.83:6835","nonce":3552333026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:49:19.962680+0000","modified":"2024-04-17T10:49:21.996802+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9198":{"gid":9198,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1559222491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1559222491},{"type":"v1","addr":"172.21.15.83:6837","nonce":1559222491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-17T10:49:29.200 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 76 2024-04-17T10:49:29.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.209+0000 7fe479d21640 1 -- 172.21.15.83:0/812347924 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe454066f60 msgr2=0x7fe454069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:29.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.209+0000 7fe479d21640 1 --2- 172.21.15.83:0/812347924 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe454066f60 0x7fe454069420 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fe47415d300 tx=0x7fe45c007e90 comp rx=0 tx=0).stop 2024-04-17T10:49:29.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.209+0000 7fe479d21640 1 -- 172.21.15.83:0/812347924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe47406dcb0 msgr2=0x7fe47415bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:29.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.209+0000 7fe479d21640 1 --2- 172.21.15.83:0/812347924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe47406dcb0 0x7fe47415bb10 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fe464009d10 tx=0x7fe46400c490 comp rx=0 tx=0).stop 2024-04-17T10:49:29.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.209+0000 7fe479d21640 1 -- 172.21.15.83:0/812347924 shutdown_connections 2024-04-17T10:49:29.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.209+0000 7fe479d21640 1 --2- 172.21.15.83:0/812347924 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe454066f60 0x7fe454069420 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.209+0000 7fe479d21640 1 --2- 172.21.15.83:0/812347924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe47415db50 0x7fe474144df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.209+0000 7fe479d21640 1 --2- 172.21.15.83:0/812347924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4740aedd0 0x7fe474156410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.209+0000 7fe479d21640 1 --2- 172.21.15.83:0/812347924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe47406dcb0 0x7fe47415bb10 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.209+0000 7fe479d21640 1 -- 172.21.15.83:0/812347924 >> 172.21.15.83:0/812347924 conn(0x7fe47406abe0 msgr2=0x7fe4740cf420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:29.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.209+0000 7fe479d21640 1 -- 172.21.15.83:0/812347924 shutdown_connections 2024-04-17T10:49:29.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.209+0000 7fe479d21640 1 -- 172.21.15.83:0/812347924 wait complete. 2024-04-17T10:49:29.622 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 76, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T10:49:19.962680+0000', 'modified': '2024-04-17T10:49:21.996802+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9198}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9198': {'gid': 9198, 'name': 'c', 'rank': 0, 'incarnation': 74, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6837/1559222491', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6836', 'nonce': 1559222491}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 1559222491}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:49:29.622 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T10:49:29.622 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-17T10:49:29.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.761+0000 7f679c335640 1 Processor -- start 2024-04-17T10:49:29.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.761+0000 7f679c335640 1 -- start start 2024-04-17T10:49:29.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.761+0000 7f679c335640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f679415b6e0 0x7f6794071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.761+0000 7f679c335640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6794070470 0x7f679406ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.761+0000 7f679c335640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f679406c1f0 con 0x7f679415bac0 2024-04-17T10:49:29.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.761+0000 7f679c335640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6794070a10 con 0x7f6794070470 2024-04-17T10:49:29.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.761+0000 7f679c335640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6794070cf0 con 0x7f679415b6e0 2024-04-17T10:49:29.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.761+0000 7f67998d0640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f679415bac0 0x7f67940cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59192/0 (socket says 172.21.15.83:59192) 2024-04-17T10:49:29.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.761+0000 7f67998d0640 1 -- 172.21.15.83:0/2480911287 learned_addr learned my addr 172.21.15.83:0/2480911287 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:29.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.761+0000 7f679a0d1640 1 --2- 172.21.15.83:0/2480911287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f679415b6e0 0x7f6794071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:29.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.761+0000 7f679a8d2640 1 --2- 172.21.15.83:0/2480911287 >> v2:172.21.15.167:3300/0 conn(0x7f6794070470 0x7f679406ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:29.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.761+0000 7f67990cf640 1 -- 172.21.15.83:0/2480911287 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 594229133 0 0) 0x7f679406c1f0 con 0x7f679415bac0 2024-04-17T10:49:29.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.761+0000 7f67990cf640 1 -- 172.21.15.83:0/2480911287 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6780003530 con 0x7f679415bac0 2024-04-17T10:49:29.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f67990cf640 1 -- 172.21.15.83:0/2480911287 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2240313626 0 0) 0x7f6780003530 con 0x7f679415bac0 2024-04-17T10:49:29.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f67990cf640 1 -- 172.21.15.83:0/2480911287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f679415b6e0 msgr2=0x7f6794071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:29.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f67990cf640 1 --2- 172.21.15.83:0/2480911287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f679415b6e0 0x7f6794071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f67990cf640 1 -- 172.21.15.83:0/2480911287 >> v2:172.21.15.167:3300/0 conn(0x7f6794070470 msgr2=0x7f679406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:29.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f67990cf640 1 --2- 172.21.15.83:0/2480911287 >> v2:172.21.15.167:3300/0 conn(0x7f6794070470 0x7f679406ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f67990cf640 1 -- 172.21.15.83:0/2480911287 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6794070fd0 con 0x7f679415bac0 2024-04-17T10:49:29.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f67990cf640 1 -- 172.21.15.83:0/2480911287 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f6784001680 con 0x7f679415bac0 2024-04-17T10:49:29.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f67990cf640 1 -- 172.21.15.83:0/2480911287 >> v1:172.21.15.83:6789/0 conn(0x7f679415bac0 legacy=0x7f67940cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:29.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f67990cf640 1 --2- 172.21.15.83:0/2480911287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6780003a30 0x7f6780003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f67990cf640 1 --2- 172.21.15.83:0/2480911287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f679415b6e0 0x7f6780004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f67990cf640 1 --2- 172.21.15.83:0/2480911287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6780004d50 0x7f67800051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f67990cf640 1 -- 172.21.15.83:0/2480911287 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6794070a10 con 0x7f6780004d50 2024-04-17T10:49:29.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f67990cf640 1 -- 172.21.15.83:0/2480911287 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6794070cf0 con 0x7f6780003a30 2024-04-17T10:49:29.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f67990cf640 1 -- 172.21.15.83:0/2480911287 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678000b9d0 con 0x7f679415b6e0 2024-04-17T10:49:29.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f679a8d2640 1 --2- 172.21.15.83:0/2480911287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6780004d50 0x7f67800051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:29.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f679a0d1640 1 --2- 172.21.15.83:0/2480911287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6780003a30 0x7f6780003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:29.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f67998d0640 1 --2- 172.21.15.83:0/2480911287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f679415b6e0 0x7f6780004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:29.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f679a8d2640 1 -- 172.21.15.83:0/2480911287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f679415b6e0 msgr2=0x7f6780004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:29.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f679a8d2640 1 --2- 172.21.15.83:0/2480911287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f679415b6e0 0x7f6780004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f679a8d2640 1 -- 172.21.15.83:0/2480911287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6780003a30 msgr2=0x7f6780003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:29.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f679a8d2640 1 --2- 172.21.15.83:0/2480911287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6780003a30 0x7f6780003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f679a8d2640 1 -- 172.21.15.83:0/2480911287 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6780003530 con 0x7f6780004d50 2024-04-17T10:49:29.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f67998d0640 1 --2- 172.21.15.83:0/2480911287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f679415b6e0 0x7f6780004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:29.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f679a8d2640 1 --2- 172.21.15.83:0/2480911287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6780004d50 0x7f67800051d0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f678c009fc0 tx=0x7f678c01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d563d3dfc742f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:29.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f67990cf640 1 -- 172.21.15.83:0/2480911287 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c0b2070 con 0x7f6780004d50 2024-04-17T10:49:29.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f67990cf640 1 -- 172.21.15.83:0/2480911287 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678c0a5cd0 con 0x7f6780004d50 2024-04-17T10:49:29.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f67990cf640 1 -- 172.21.15.83:0/2480911287 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c009c10 con 0x7f6780004d50 2024-04-17T10:49:29.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f679c335640 1 -- 172.21.15.83:0/2480911287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6780004d50 msgr2=0x7f67800051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:29.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f679c335640 1 --2- 172.21.15.83:0/2480911287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6780004d50 0x7f67800051d0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f678c009fc0 tx=0x7f678c01ccc0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f679a8d2640 1 -- 172.21.15.83:0/2480911287 reap_dead start 2024-04-17T10:49:29.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f679c335640 1 -- 172.21.15.83:0/2480911287 shutdown_connections 2024-04-17T10:49:29.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f679c335640 1 -- 172.21.15.83:0/2480911287 >> 172.21.15.83:0/2480911287 conn(0x7f67940b15b0 msgr2=0x7f67940b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:29.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f679c335640 1 -- 172.21.15.83:0/2480911287 shutdown_connections 2024-04-17T10:49:29.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.765+0000 7f679c335640 1 -- 172.21.15.83:0/2480911287 wait complete. 2024-04-17T10:49:29.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.769+0000 7f679c335640 1 Processor -- start 2024-04-17T10:49:29.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.769+0000 7f679c335640 1 -- start start 2024-04-17T10:49:29.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.769+0000 7f679c335640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f679414ee00 0x7f6794156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.769+0000 7f679c335640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f679414f1e0 0x7f67941f1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.769+0000 7f679c335640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67941599c0 0x7f6794159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.769+0000 7f679c335640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679406b4d0 con 0x7f679414ee00 2024-04-17T10:49:29.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.769+0000 7f679c335640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679406b350 con 0x7f679414f1e0 2024-04-17T10:49:29.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.769+0000 7f679c335640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f679406b650 con 0x7f67941599c0 2024-04-17T10:49:29.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.769+0000 7f679a0d1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f679414ee00 0x7f6794156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:29.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.769+0000 7f679a0d1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f679414ee00 0x7f6794156790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57938/0 (socket says 172.21.15.83:57938) 2024-04-17T10:49:29.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.769+0000 7f679a0d1640 1 -- 172.21.15.83:0/2183668139 learned_addr learned my addr 172.21.15.83:0/2183668139 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:29.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.769+0000 7f679a0d1640 1 -- 172.21.15.83:0/2183668139 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67941599c0 msgr2=0x7f6794159e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:29.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.769+0000 7f679a8d2640 1 --2- 172.21.15.83:0/2183668139 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67941599c0 0x7f6794159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:29.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.769+0000 7f67998d0640 1 --2- 172.21.15.83:0/2183668139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f679414f1e0 0x7f67941f1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:29.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.769+0000 7f679a0d1640 1 --2- 172.21.15.83:0/2183668139 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67941599c0 0x7f6794159e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.769+0000 7f679a0d1640 1 -- 172.21.15.83:0/2183668139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f679414f1e0 msgr2=0x7f67941f1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:29.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.769+0000 7f679a0d1640 1 --2- 172.21.15.83:0/2183668139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f679414f1e0 0x7f67941f1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.769+0000 7f679a0d1640 1 -- 172.21.15.83:0/2183668139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67941f21f0 con 0x7f679414ee00 2024-04-17T10:49:29.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.769+0000 7f679a8d2640 1 --2- 172.21.15.83:0/2183668139 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67941599c0 0x7f6794159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:29.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.769+0000 7f679a0d1640 1 --2- 172.21.15.83:0/2183668139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f679414ee00 0x7f6794156790 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f6790011640 tx=0x7f6790011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:29.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.769+0000 7f678affd640 1 -- 172.21.15.83:0/2183668139 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679000f920 con 0x7f679414ee00 2024-04-17T10:49:29.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.769+0000 7f678affd640 1 -- 172.21.15.83:0/2183668139 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679000fac0 con 0x7f679414ee00 2024-04-17T10:49:29.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.769+0000 7f679c335640 1 -- 172.21.15.83:0/2183668139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f679414cdc0 con 0x7f679414ee00 2024-04-17T10:49:29.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.769+0000 7f678affd640 1 -- 172.21.15.83:0/2183668139 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679000fe70 con 0x7f679414ee00 2024-04-17T10:49:29.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.769+0000 7f679c335640 1 -- 172.21.15.83:0/2183668139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f679414d340 con 0x7f679414ee00 2024-04-17T10:49:29.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f678affd640 1 -- 172.21.15.83:0/2183668139 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6790015460 con 0x7f679414ee00 2024-04-17T10:49:29.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f678affd640 1 --2- 172.21.15.83:0/2183668139 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6778066f90 0x7f6778069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f678affd640 1 -- 172.21.15.83:0/2183668139 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6790085960 con 0x7f679414ee00 2024-04-17T10:49:29.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f67998d0640 1 --2- 172.21.15.83:0/2183668139 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6778066f90 0x7f6778069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:29.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f67998d0640 1 --2- 172.21.15.83:0/2183668139 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6778066f90 0x7f6778069450 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6784000c10 tx=0x7f6784009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:29.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f679f4a2000 1 Processor -- start 2024-04-17T10:49:29.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f679f4a2000 1 -- start start 2024-04-17T10:49:29.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f679f4a2000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2ca01920 0x55ea2ca01d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f679f4a2000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ea2ca02240 0x55ea2c9b79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f679f4a2000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea2c9b7f30 0x55ea2c9b8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f679f4a2000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea2c89f5f0 con 0x55ea2ca01920 2024-04-17T10:49:29.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f679f4a2000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea2c806da0 con 0x55ea2c9b7f30 2024-04-17T10:49:29.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f679f4a2000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ea2c78f150 con 0x55ea2ca02240 2024-04-17T10:49:29.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f679a0d1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2ca01920 0x55ea2ca01d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:29.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f679a0d1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2ca01920 0x55ea2ca01d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57942/0 (socket says 172.21.15.83:57942) 2024-04-17T10:49:29.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f679a0d1640 1 -- 172.21.15.83:0/3048618309 learned_addr learned my addr 172.21.15.83:0/3048618309 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:29.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f67998d0640 1 --2- 172.21.15.83:0/3048618309 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ea2ca02240 0x55ea2c9b79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:29.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f679a0d1640 1 -- 172.21.15.83:0/3048618309 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ea2ca02240 msgr2=0x55ea2c9b79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:29.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f679a0d1640 1 --2- 172.21.15.83:0/3048618309 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ea2ca02240 0x55ea2c9b79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f679a0d1640 1 -- 172.21.15.83:0/3048618309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea2c9b7f30 msgr2=0x55ea2c9b8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:29.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f679a8d2640 1 --2- 172.21.15.83:0/3048618309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea2c9b7f30 0x55ea2c9b8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:29.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f679a0d1640 1 --2- 172.21.15.83:0/3048618309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea2c9b7f30 0x55ea2c9b8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f679a0d1640 1 -- 172.21.15.83:0/3048618309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea2c7a8ca0 con 0x55ea2ca01920 2024-04-17T10:49:29.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f679a0d1640 1 --2- 172.21.15.83:0/3048618309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2ca01920 0x55ea2ca01d00 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f6790083f40 tx=0x7f6790056440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f8e4bcd90137ef server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:29.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f6777fff640 1 -- 172.21.15.83:0/3048618309 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6790082070 con 0x55ea2ca01920 2024-04-17T10:49:29.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f6777fff640 1 -- 172.21.15.83:0/3048618309 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6790054600 con 0x55ea2ca01920 2024-04-17T10:49:29.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f6777fff640 1 -- 172.21.15.83:0/3048618309 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6790052970 con 0x55ea2ca01920 2024-04-17T10:49:29.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f679f4a2000 1 -- 172.21.15.83:0/3048618309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2ca01920 msgr2=0x55ea2ca01d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:29.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f679f4a2000 1 --2- 172.21.15.83:0/3048618309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2ca01920 0x55ea2ca01d00 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f6790083f40 tx=0x7f6790056440 comp rx=0 tx=0).stop 2024-04-17T10:49:29.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f679f4a2000 1 -- 172.21.15.83:0/3048618309 shutdown_connections 2024-04-17T10:49:29.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f679f4a2000 1 --2- 172.21.15.83:0/3048618309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea2c9b7f30 0x55ea2c9b8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f679f4a2000 1 --2- 172.21.15.83:0/3048618309 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ea2ca02240 0x55ea2c9b79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f679f4a2000 1 --2- 172.21.15.83:0/3048618309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2ca01920 0x55ea2ca01d00 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f679f4a2000 1 -- 172.21.15.83:0/3048618309 >> 172.21.15.83:0/3048618309 conn(0x55ea2c9be2a0 msgr2=0x55ea2c9b62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:29.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f679f4a2000 1 -- 172.21.15.83:0/3048618309 shutdown_connections 2024-04-17T10:49:29.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.773+0000 7f679f4a2000 1 -- 172.21.15.83:0/3048618309 wait complete. 2024-04-17T10:49:29.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679f4a2000 1 -- start start 2024-04-17T10:49:29.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679f4a2000 1 Processor -- start 2024-04-17T10:49:29.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679f4a2000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ea2c9b7f30 0x55ea2ca08d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679f4a2000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2ca01920 0x55ea2ca092a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679f4a2000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea2ca02240 0x55ea2c9bbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679f4a2000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea2c7c1990 con 0x55ea2ca01920 2024-04-17T10:49:29.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679f4a2000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea2c802e40 con 0x55ea2ca02240 2024-04-17T10:49:29.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679f4a2000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ea2c792900 con 0x55ea2c9b7f30 2024-04-17T10:49:29.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679f4a2000 10 client.0 _ll_register_callbacks cb 0x55ea2c8ba1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:49:29.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f67998d0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2ca01920 0x55ea2ca092a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:29.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f67998d0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2ca01920 0x55ea2ca092a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57948/0 (socket says 172.21.15.83:57948) 2024-04-17T10:49:29.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f67998d0640 1 -- 172.21.15.83:0/1238625169 learned_addr learned my addr 172.21.15.83:0/1238625169 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:29.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679a8d2640 1 --2- 172.21.15.83:0/1238625169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea2ca02240 0x55ea2c9bbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:29.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679a0d1640 1 --2- 172.21.15.83:0/1238625169 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ea2c9b7f30 0x55ea2ca08d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:29.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f67998d0640 1 -- 172.21.15.83:0/1238625169 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ea2c9b7f30 msgr2=0x55ea2ca08d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:29.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f67998d0640 1 --2- 172.21.15.83:0/1238625169 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ea2c9b7f30 0x55ea2ca08d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f67998d0640 1 -- 172.21.15.83:0/1238625169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea2ca02240 msgr2=0x55ea2c9bbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:29.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f67998d0640 1 --2- 172.21.15.83:0/1238625169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea2ca02240 0x55ea2c9bbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f67998d0640 1 -- 172.21.15.83:0/1238625169 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea2c7bf630 con 0x55ea2ca01920 2024-04-17T10:49:29.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679a8d2640 1 --2- 172.21.15.83:0/1238625169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea2ca02240 0x55ea2c9bbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:29.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679a0d1640 1 --2- 172.21.15.83:0/1238625169 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ea2c9b7f30 0x55ea2ca08d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:29.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f67998d0640 1 --2- 172.21.15.83:0/1238625169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2ca01920 0x55ea2ca092a0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f6784007400 tx=0x7f6784000ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5581f3b0b7456279 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:29.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f67757fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:49:29.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f67757fa640 1 -- 172.21.15.83:0/1238625169 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6784013890 con 0x55ea2ca01920 2024-04-17T10:49:29.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f67757fa640 1 -- 172.21.15.83:0/1238625169 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6784021a80 con 0x55ea2ca01920 2024-04-17T10:49:29.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679f4a2000 1 -- 172.21.15.83:0/1238625169 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6790054600 con 0x55ea2ca01920 2024-04-17T10:49:29.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f67757fa640 1 -- 172.21.15.83:0/1238625169 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f678400ee50 con 0x55ea2ca01920 2024-04-17T10:49:29.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679f4a2000 10 client.9348 fetch_fsmap learned FSMap version 76 2024-04-17T10:49:29.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679f4a2000 1 -- 172.21.15.83:0/1238625169 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x55ea2c78f3a0 con 0x55ea2ca01920 2024-04-17T10:49:29.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f67757fa640 1 -- 172.21.15.83:0/1238625169 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6790054600 con 0x55ea2ca01920 2024-04-17T10:49:29.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f67757fa640 1 -- 172.21.15.83:0/1238625169 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 76) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6784021c80 con 0x55ea2ca01920 2024-04-17T10:49:29.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679f4a2000 10 client.9348 fetch_fsmap finished waiting for FSMap version 76 2024-04-17T10:49:29.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679f4a2000 10 client.9348 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-17T10:49:29.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679f4a2000 20 client.9348 populate_metadata read hostname 'smithi083' 2024-04-17T10:49:29.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679f4a2000 1 --2- 172.21.15.83:0/1238625169 >> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] conn(0x55ea2c9bc4b0 0x55ea2c9bc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679f4a2000 4 client.9348 mds_command: new command op to 9198 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:49:29.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679f4a2000 1 -- 172.21.15.83:0/1238625169 --> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea2c7a2a50 con 0x55ea2c9bc4b0 2024-04-17T10:49:29.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679a0d1640 1 --2- 172.21.15.83:0/1238625169 >> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] conn(0x55ea2c9bc4b0 0x55ea2c9bc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:29.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f67757fa640 10 client.9348 ms_handle_connect on v2:172.21.15.83:6836/1559222491 2024-04-17T10:49:29.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679a0d1640 1 --2- 172.21.15.83:0/1238625169 >> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] conn(0x55ea2c9bc4b0 0x55ea2c9bc890 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73544f08fad23225 server_cookie=8f2468a982175bde in_seq=0 out_seq=0 2024-04-17T10:49:29.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f67757fa640 1 -- 172.21.15.83:0/1238625169 <== mds.0 v2:172.21.15.83:6836/1559222491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ea2c7a2a50 con 0x55ea2c9bc4b0 2024-04-17T10:49:29.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f67757fa640 10 client.9348 handle_command_reply: tid=0 2024-04-17T10:49:29.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679f4a2000 1 client.9348 shutdown 2024-04-17T10:49:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679f4a2000 1 -- 172.21.15.83:0/1238625169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2ca01920 msgr2=0x55ea2ca092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679f4a2000 1 --2- 172.21.15.83:0/1238625169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2ca01920 0x55ea2ca092a0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f6784007400 tx=0x7f6784000ff0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679f4a2000 1 -- 172.21.15.83:0/1238625169 shutdown_connections 2024-04-17T10:49:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679f4a2000 1 --2- 172.21.15.83:0/1238625169 >> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] conn(0x55ea2c9bc4b0 0x55ea2c9bc890 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679f4a2000 1 --2- 172.21.15.83:0/1238625169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea2ca02240 0x55ea2c9bbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679f4a2000 1 --2- 172.21.15.83:0/1238625169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2ca01920 0x55ea2ca092a0 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f67757fa640 0 client.9348 ms_handle_reset on v2:172.21.15.83:6836/1559222491 2024-04-17T10:49:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679f4a2000 1 --2- 172.21.15.83:0/1238625169 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ea2c9b7f30 0x55ea2ca08d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.777+0000 7f679f4a2000 1 -- 172.21.15.83:0/1238625169 >> 172.21.15.83:0/1238625169 conn(0x55ea2c9be2a0 msgr2=0x55ea2c9b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.781+0000 7f679f4a2000 1 -- 172.21.15.83:0/1238625169 shutdown_connections 2024-04-17T10:49:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.781+0000 7f679f4a2000 1 -- 172.21.15.83:0/1238625169 wait complete. 2024-04-17T10:49:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.781+0000 7f679f4a2000 20 client.9348 trim_cache size 0 max 16384 2024-04-17T10:49:29.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.793+0000 7f679f4a2000 1 Processor -- start 2024-04-17T10:49:29.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.793+0000 7f679f4a2000 1 -- start start 2024-04-17T10:49:29.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679f4a2000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2c9b7f30 0x55ea2ca658c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679f4a2000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea2ca01920 0x55ea2ca65e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679f4a2000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ea2ca02240 0x55ea2ca66340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679f4a2000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67941fd910 con 0x55ea2c9b7f30 2024-04-17T10:49:29.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679f4a2000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67941fcd70 con 0x55ea2ca01920 2024-04-17T10:49:29.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679f4a2000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67941fd310 con 0x55ea2ca02240 2024-04-17T10:49:29.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679a0d1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2c9b7f30 0x55ea2ca658c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:29.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679a0d1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2c9b7f30 0x55ea2ca658c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57954/0 (socket says 172.21.15.83:57954) 2024-04-17T10:49:29.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679a0d1640 1 -- 172.21.15.83:0/3461497180 learned_addr learned my addr 172.21.15.83:0/3461497180 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:29.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679a0d1640 1 -- 172.21.15.83:0/3461497180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ea2ca02240 msgr2=0x55ea2ca66340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:29.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679a0d1640 1 --2- 172.21.15.83:0/3461497180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ea2ca02240 0x55ea2ca66340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679a0d1640 1 -- 172.21.15.83:0/3461497180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea2ca01920 msgr2=0x55ea2ca65e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:29.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679a0d1640 1 --2- 172.21.15.83:0/3461497180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea2ca01920 0x55ea2ca65e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679a0d1640 1 -- 172.21.15.83:0/3461497180 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea2ca02070 con 0x55ea2c9b7f30 2024-04-17T10:49:29.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679a0d1640 1 --2- 172.21.15.83:0/3461497180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2c9b7f30 0x55ea2ca658c0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f6790010010 tx=0x7f6790058000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab32a5966766769 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:29.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f67777fe640 1 -- 172.21.15.83:0/3461497180 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6790082070 con 0x55ea2c9b7f30 2024-04-17T10:49:29.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f67777fe640 1 -- 172.21.15.83:0/3461497180 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6790083040 con 0x55ea2c9b7f30 2024-04-17T10:49:29.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f67777fe640 1 -- 172.21.15.83:0/3461497180 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6790071040 con 0x55ea2c9b7f30 2024-04-17T10:49:29.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679f4a2000 1 -- 172.21.15.83:0/3461497180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2c9b7f30 msgr2=0x55ea2ca658c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:29.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679f4a2000 1 --2- 172.21.15.83:0/3461497180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2c9b7f30 0x55ea2ca658c0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f6790010010 tx=0x7f6790058000 comp rx=0 tx=0).stop 2024-04-17T10:49:29.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679f4a2000 1 -- 172.21.15.83:0/3461497180 shutdown_connections 2024-04-17T10:49:29.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679f4a2000 1 --2- 172.21.15.83:0/3461497180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ea2ca02240 0x55ea2ca66340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679f4a2000 1 --2- 172.21.15.83:0/3461497180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea2ca01920 0x55ea2ca65e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679f4a2000 1 --2- 172.21.15.83:0/3461497180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2c9b7f30 0x55ea2ca658c0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679f4a2000 1 -- 172.21.15.83:0/3461497180 >> 172.21.15.83:0/3461497180 conn(0x55ea2c9be2a0 msgr2=0x55ea2c9bb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:29.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679f4a2000 1 -- 172.21.15.83:0/3461497180 shutdown_connections 2024-04-17T10:49:29.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679f4a2000 1 -- 172.21.15.83:0/3461497180 wait complete. 2024-04-17T10:49:29.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679f4a2000 1 -- start start 2024-04-17T10:49:29.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679f4a2000 1 Processor -- start 2024-04-17T10:49:29.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679f4a2000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea2c9b7f30 0x55ea2ca0e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679f4a2000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2ca01920 0x55ea2ca66640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679f4a2000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ea2ca02240 0x55ea2ca66b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679f4a2000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea2c80b3d0 con 0x55ea2ca01920 2024-04-17T10:49:29.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679f4a2000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea2c8c2d50 con 0x55ea2c9b7f30 2024-04-17T10:49:29.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679f4a2000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ea2c7a1980 con 0x55ea2ca02240 2024-04-17T10:49:29.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679f4a2000 10 client.0 _ll_register_callbacks cb 0x7f67900560d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:49:29.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f67998d0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2ca01920 0x55ea2ca66640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:29.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f67998d0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2ca01920 0x55ea2ca66640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57968/0 (socket says 172.21.15.83:57968) 2024-04-17T10:49:29.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f67998d0640 1 -- 172.21.15.83:0/1367988091 learned_addr learned my addr 172.21.15.83:0/1367988091 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:29.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679a0d1640 1 --2- 172.21.15.83:0/1367988091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea2c9b7f30 0x55ea2ca0e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:29.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679a8d2640 1 --2- 172.21.15.83:0/1367988091 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ea2ca02240 0x55ea2ca66b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:29.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f67998d0640 1 -- 172.21.15.83:0/1367988091 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ea2ca02240 msgr2=0x55ea2ca66b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:29.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f67998d0640 1 --2- 172.21.15.83:0/1367988091 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ea2ca02240 0x55ea2ca66b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f67998d0640 1 -- 172.21.15.83:0/1367988091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea2c9b7f30 msgr2=0x55ea2ca0e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:29.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f67998d0640 1 --2- 172.21.15.83:0/1367988091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea2c9b7f30 0x55ea2ca0e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f67998d0640 1 -- 172.21.15.83:0/1367988091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea2c888600 con 0x55ea2ca01920 2024-04-17T10:49:29.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.797+0000 7f679a0d1640 1 --2- 172.21.15.83:0/1367988091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea2c9b7f30 0x55ea2ca0e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:29.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f67998d0640 1 --2- 172.21.15.83:0/1367988091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2ca01920 0x55ea2ca66640 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f6784001c60 tx=0x7f67940ccac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f7d35f16d02060 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:29.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f67757fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:49:29.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f67757fa640 1 -- 172.21.15.83:0/1367988091 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6784030070 con 0x55ea2ca01920 2024-04-17T10:49:29.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f67757fa640 1 -- 172.21.15.83:0/1367988091 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f678402c840 con 0x55ea2ca01920 2024-04-17T10:49:29.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679f4a2000 1 -- 172.21.15.83:0/1367988091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6790083040 con 0x55ea2ca01920 2024-04-17T10:49:29.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f67757fa640 1 -- 172.21.15.83:0/1367988091 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f678403c050 con 0x55ea2ca01920 2024-04-17T10:49:29.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679f4a2000 10 client.9354 fetch_fsmap learned FSMap version 76 2024-04-17T10:49:29.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679f4a2000 1 -- 172.21.15.83:0/1367988091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x55ea2c79b9b0 con 0x55ea2ca01920 2024-04-17T10:49:29.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f67757fa640 1 -- 172.21.15.83:0/1367988091 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6790083040 con 0x55ea2ca01920 2024-04-17T10:49:29.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f67757fa640 1 -- 172.21.15.83:0/1367988091 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 76) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f678403f020 con 0x55ea2ca01920 2024-04-17T10:49:29.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679f4a2000 10 client.9354 fetch_fsmap finished waiting for FSMap version 76 2024-04-17T10:49:29.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679f4a2000 10 client.9354 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-17T10:49:29.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679f4a2000 20 client.9354 populate_metadata read hostname 'smithi083' 2024-04-17T10:49:29.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679f4a2000 1 --2- 172.21.15.83:0/1367988091 >> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] conn(0x55ea2ca670c0 0x55ea2ca674a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679f4a2000 4 client.9354 mds_command: new command op to 9198 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T10:49:29.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679f4a2000 1 -- 172.21.15.83:0/1367988091 --> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ea2c8142e0 con 0x55ea2ca670c0 2024-04-17T10:49:29.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679a0d1640 1 --2- 172.21.15.83:0/1367988091 >> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] conn(0x55ea2ca670c0 0x55ea2ca674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:29.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f67757fa640 10 client.9354 ms_handle_connect on v2:172.21.15.83:6836/1559222491 2024-04-17T10:49:29.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679a0d1640 1 --2- 172.21.15.83:0/1367988091 >> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] conn(0x55ea2ca670c0 0x55ea2ca674a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cd60e415f2c02aa server_cookie=ac04ac7b9d9fcbd6 in_seq=0 out_seq=0 2024-04-17T10:49:29.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f67757fa640 1 -- 172.21.15.83:0/1367988091 <== mds.0 v2:172.21.15.83:6836/1559222491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55ea2c8142e0 con 0x55ea2ca670c0 2024-04-17T10:49:29.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f67757fa640 10 client.9354 handle_command_reply: tid=0 2024-04-17T10:49:29.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679f4a2000 1 client.9354 shutdown 2024-04-17T10:49:29.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679f4a2000 1 -- 172.21.15.83:0/1367988091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2ca01920 msgr2=0x55ea2ca66640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:29.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679f4a2000 1 --2- 172.21.15.83:0/1367988091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2ca01920 0x55ea2ca66640 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f6784001c60 tx=0x7f67940ccac0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679f4a2000 1 -- 172.21.15.83:0/1367988091 shutdown_connections 2024-04-17T10:49:29.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679f4a2000 1 --2- 172.21.15.83:0/1367988091 >> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] conn(0x55ea2ca670c0 0x55ea2ca674a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679f4a2000 1 --2- 172.21.15.83:0/1367988091 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ea2ca02240 0x55ea2ca66b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679f4a2000 1 --2- 172.21.15.83:0/1367988091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2ca01920 0x55ea2ca66640 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f67757fa640 0 client.9354 ms_handle_reset on v2:172.21.15.83:6836/1559222491 2024-04-17T10:49:29.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679f4a2000 1 --2- 172.21.15.83:0/1367988091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea2c9b7f30 0x55ea2ca0e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679f4a2000 1 -- 172.21.15.83:0/1367988091 >> 172.21.15.83:0/1367988091 conn(0x55ea2c9be2a0 msgr2=0x55ea2ca09950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:29.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679f4a2000 1 -- 172.21.15.83:0/1367988091 shutdown_connections 2024-04-17T10:49:29.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679f4a2000 1 -- 172.21.15.83:0/1367988091 wait complete. 2024-04-17T10:49:29.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679f4a2000 20 client.9354 trim_cache size 0 max 16384 2024-04-17T10:49:29.800 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:49:29.800 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "3de9b7be-af7f-488a-969e-2b6b8db63bcd", 2024-04-17T10:49:29.800 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-04-17T10:49:29.800 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 9198, 2024-04-17T10:49:29.800 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-17T10:49:29.800 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-17T10:49:29.800 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-17T10:49:29.800 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 9.1592572879999992, 2024-04-17T10:49:29.800 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 76, 2024-04-17T10:49:29.800 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 127, 2024-04-17T10:49:29.800 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 127, 2024-04-17T10:49:29.800 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 14.242279128 2024-04-17T10:49:29.801 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:49:29.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679c335640 1 -- 172.21.15.83:0/2183668139 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6778066f90 msgr2=0x7f6778069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:29.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679c335640 1 --2- 172.21.15.83:0/2183668139 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6778066f90 0x7f6778069450 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6784000c10 tx=0x7f6784009040 comp rx=0 tx=0).stop 2024-04-17T10:49:29.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679c335640 1 -- 172.21.15.83:0/2183668139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f679414ee00 msgr2=0x7f6794156790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:29.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679c335640 1 --2- 172.21.15.83:0/2183668139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f679414ee00 0x7f6794156790 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f6790011640 tx=0x7f6790011b00 comp rx=0 tx=0).stop 2024-04-17T10:49:29.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679c335640 1 -- 172.21.15.83:0/2183668139 shutdown_connections 2024-04-17T10:49:29.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679c335640 1 --2- 172.21.15.83:0/2183668139 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6778066f90 0x7f6778069450 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679c335640 1 --2- 172.21.15.83:0/2183668139 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67941599c0 0x7f6794159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679c335640 1 --2- 172.21.15.83:0/2183668139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f679414f1e0 0x7f67941f1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679c335640 1 --2- 172.21.15.83:0/2183668139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f679414ee00 0x7f6794156790 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679c335640 1 -- 172.21.15.83:0/2183668139 >> 172.21.15.83:0/2183668139 conn(0x7f67940b15b0 msgr2=0x7f679406d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:29.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679c335640 1 -- 172.21.15.83:0/2183668139 shutdown_connections 2024-04-17T10:49:29.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.801+0000 7f679c335640 1 -- 172.21.15.83:0/2183668139 wait complete. 2024-04-17T10:49:29.811 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T10:49:29.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.929+0000 7f0a16da5640 1 Processor -- start 2024-04-17T10:49:29.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.929+0000 7f0a16da5640 1 -- start start 2024-04-17T10:49:29.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.933+0000 7f0a16da5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0a1015ba30 0x7f0a10072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.933+0000 7f0a16da5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a1015c960 0x7f0a100cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.933+0000 7f0a16da5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0a100cfa10 0x7f0a1006c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.933+0000 7f0a16da5640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a10071ea0 con 0x7f0a100cfa10 2024-04-17T10:49:29.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.933+0000 7f0a16da5640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a100cfea0 con 0x7f0a1015ba30 2024-04-17T10:49:29.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.933+0000 7f0a16da5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1006e320 con 0x7f0a1015c960 2024-04-17T10:49:29.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.933+0000 7f0a15342640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0a100cfa10 0x7f0a1006c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:29.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.933+0000 7f0a15342640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0a100cfa10 0x7f0a1006c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57972/0 (socket says 172.21.15.83:57972) 2024-04-17T10:49:29.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.933+0000 7f0a15342640 1 -- 172.21.15.83:0/3660525017 learned_addr learned my addr 172.21.15.83:0/3660525017 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:29.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.933+0000 7f0a0ffff640 1 --2- 172.21.15.83:0/3660525017 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a1015c960 0x7f0a100cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:29.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.933+0000 7f0a14b41640 1 --2- 172.21.15.83:0/3660525017 >> v2:172.21.15.167:3300/0 conn(0x7f0a1015ba30 0x7f0a10072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:29.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.933+0000 7f0a15342640 1 -- 172.21.15.83:0/3660525017 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a1015c960 msgr2=0x7f0a100cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:29.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.933+0000 7f0a15342640 1 --2- 172.21.15.83:0/3660525017 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a1015c960 0x7f0a100cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.933+0000 7f0a15342640 1 -- 172.21.15.83:0/3660525017 >> v2:172.21.15.167:3300/0 conn(0x7f0a1015ba30 msgr2=0x7f0a10072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:29.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.933+0000 7f0a15342640 1 --2- 172.21.15.83:0/3660525017 >> v2:172.21.15.167:3300/0 conn(0x7f0a1015ba30 0x7f0a10072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.933+0000 7f0a15342640 1 -- 172.21.15.83:0/3660525017 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1006e040 con 0x7f0a100cfa10 2024-04-17T10:49:29.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.933+0000 7f0a15342640 1 --2- 172.21.15.83:0/3660525017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a100cfa10 0x7f0a1006c490 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f0a100b2410 tx=0x7f0a0001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2adf37e09e59713 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:29.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.933+0000 7f0a0f7fe640 1 -- 172.21.15.83:0/3660525017 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a000b0070 con 0x7f0a100cfa10 2024-04-17T10:49:29.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.933+0000 7f0a0f7fe640 1 -- 172.21.15.83:0/3660525017 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a000ab600 con 0x7f0a100cfa10 2024-04-17T10:49:29.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.933+0000 7f0a0f7fe640 1 -- 172.21.15.83:0/3660525017 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a000ab9d0 con 0x7f0a100cfa10 2024-04-17T10:49:29.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.933+0000 7f0a16da5640 1 -- 172.21.15.83:0/3660525017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a100cfa10 msgr2=0x7f0a1006c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:29.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.933+0000 7f0a16da5640 1 --2- 172.21.15.83:0/3660525017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a100cfa10 0x7f0a1006c490 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f0a100b2410 tx=0x7f0a0001c490 comp rx=0 tx=0).stop 2024-04-17T10:49:29.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.933+0000 7f0a16da5640 1 -- 172.21.15.83:0/3660525017 shutdown_connections 2024-04-17T10:49:29.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.933+0000 7f0a16da5640 1 --2- 172.21.15.83:0/3660525017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a100cfa10 0x7f0a1006c490 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.933+0000 7f0a16da5640 1 --2- 172.21.15.83:0/3660525017 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a1015c960 0x7f0a100cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.933+0000 7f0a16da5640 1 --2- 172.21.15.83:0/3660525017 >> v2:172.21.15.167:3300/0 conn(0x7f0a1015ba30 0x7f0a10072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.933+0000 7f0a16da5640 1 -- 172.21.15.83:0/3660525017 >> 172.21.15.83:0/3660525017 conn(0x7f0a1006abe0 msgr2=0x7f0a1006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:29.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.933+0000 7f0a16da5640 1 -- 172.21.15.83:0/3660525017 shutdown_connections 2024-04-17T10:49:29.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.933+0000 7f0a16da5640 1 -- 172.21.15.83:0/3660525017 wait complete. 2024-04-17T10:49:29.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.937+0000 7f0a16da5640 1 Processor -- start 2024-04-17T10:49:29.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.937+0000 7f0a16da5640 1 -- start start 2024-04-17T10:49:29.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.937+0000 7f0a16da5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a100cfa10 0x7f0a10143bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.937+0000 7f0a16da5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a1015ba30 0x7f0a100b4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.937+0000 7f0a16da5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a1015c960 0x7f0a100ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.937+0000 7f0a16da5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1006de20 con 0x7f0a100cfa10 2024-04-17T10:49:29.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.937+0000 7f0a16da5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1006dca0 con 0x7f0a1015c960 2024-04-17T10:49:29.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.937+0000 7f0a16da5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1015cd40 con 0x7f0a1015ba30 2024-04-17T10:49:29.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.937+0000 7f0a14b41640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a100cfa10 0x7f0a10143bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:29.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.937+0000 7f0a14b41640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a100cfa10 0x7f0a10143bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57988/0 (socket says 172.21.15.83:57988) 2024-04-17T10:49:29.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.937+0000 7f0a14b41640 1 -- 172.21.15.83:0/1779022493 learned_addr learned my addr 172.21.15.83:0/1779022493 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:29.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.937+0000 7f0a0ffff640 1 --2- 172.21.15.83:0/1779022493 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a1015ba30 0x7f0a100b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:29.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.937+0000 7f0a15342640 1 --2- 172.21.15.83:0/1779022493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a1015c960 0x7f0a100ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.937+0000 7f0a14b41640 1 -- 172.21.15.83:0/1779022493 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a1015ba30 msgr2=0x7f0a100b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.937+0000 7f0a14b41640 1 --2- 172.21.15.83:0/1779022493 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a1015ba30 0x7f0a100b4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.937+0000 7f0a14b41640 1 -- 172.21.15.83:0/1779022493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a1015c960 msgr2=0x7f0a100ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.937+0000 7f0a14b41640 1 --2- 172.21.15.83:0/1779022493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a1015c960 0x7f0a100ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.937+0000 7f0a14b41640 1 -- 172.21.15.83:0/1779022493 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1014b8f0 con 0x7f0a100cfa10 2024-04-17T10:49:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.937+0000 7f0a14b41640 1 --2- 172.21.15.83:0/1779022493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a100cfa10 0x7f0a10143bf0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f0a0800ce70 tx=0x7f0a0800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.937+0000 7f0a0d7fa640 1 -- 172.21.15.83:0/1779022493 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a08003ba0 con 0x7f0a100cfa10 2024-04-17T10:49:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.937+0000 7f0a0d7fa640 1 -- 172.21.15.83:0/1779022493 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a08003d40 con 0x7f0a100cfa10 2024-04-17T10:49:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.937+0000 7f0a16da5640 1 -- 172.21.15.83:0/1779022493 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a1014d2e0 con 0x7f0a100cfa10 2024-04-17T10:49:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.937+0000 7f0a0d7fa640 1 -- 172.21.15.83:0/1779022493 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a08004260 con 0x7f0a100cfa10 2024-04-17T10:49:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.937+0000 7f0a16da5640 1 -- 172.21.15.83:0/1779022493 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a1014b5c0 con 0x7f0a100cfa10 2024-04-17T10:49:29.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.937+0000 7f0a16da5640 1 -- 172.21.15.83:0/1779022493 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a1014d800 con 0x7f0a100cfa10 2024-04-17T10:49:29.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.941+0000 7f0a0d7fa640 1 -- 172.21.15.83:0/1779022493 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0a08004400 con 0x7f0a100cfa10 2024-04-17T10:49:29.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.941+0000 7f0a0d7fa640 1 --2- 172.21.15.83:0/1779022493 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f09f0066f60 0x7f09f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:29.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.941+0000 7f0a0ffff640 1 --2- 172.21.15.83:0/1779022493 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f09f0066f60 0x7f09f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:29.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.941+0000 7f0a0d7fa640 1 -- 172.21.15.83:0/1779022493 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a08085bc0 con 0x7f0a100cfa10 2024-04-17T10:49:29.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.941+0000 7f0a0ffff640 1 --2- 172.21.15.83:0/1779022493 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f09f0066f60 0x7f09f0069420 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f09f800b800 tx=0x7f09f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:29.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:29.945+0000 7f0a0d7fa640 1 -- 172.21.15.83:0/1779022493 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0a08056b40 con 0x7f0a100cfa10 2024-04-17T10:49:30.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:30.133+0000 7f0a16da5640 1 -- 172.21.15.83:0/1779022493 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0a1006b0f0 con 0x7f0a100cfa10 2024-04-17T10:49:30.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:30.133+0000 7f0a0d7fa640 1 -- 172.21.15.83:0/1779022493 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v76) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0a08056440 con 0x7f0a100cfa10 2024-04-17T10:49:30.126 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T10:49:30.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:30.133+0000 7f0a16da5640 1 -- 172.21.15.83:0/1779022493 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f09f0066f60 msgr2=0x7f09f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:30.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:30.137+0000 7f0a16da5640 1 --2- 172.21.15.83:0/1779022493 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f09f0066f60 0x7f09f0069420 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f09f800b800 tx=0x7f09f8008040 comp rx=0 tx=0).stop 2024-04-17T10:49:30.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:30.137+0000 7f0a16da5640 1 -- 172.21.15.83:0/1779022493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a100cfa10 msgr2=0x7f0a10143bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:30.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:30.137+0000 7f0a16da5640 1 --2- 172.21.15.83:0/1779022493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a100cfa10 0x7f0a10143bf0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f0a0800ce70 tx=0x7f0a0800a660 comp rx=0 tx=0).stop 2024-04-17T10:49:30.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:30.137+0000 7f0a16da5640 1 -- 172.21.15.83:0/1779022493 shutdown_connections 2024-04-17T10:49:30.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:30.137+0000 7f0a16da5640 1 --2- 172.21.15.83:0/1779022493 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f09f0066f60 0x7f09f0069420 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:30.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:30.137+0000 7f0a16da5640 1 --2- 172.21.15.83:0/1779022493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a1015c960 0x7f0a100ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:30.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:30.137+0000 7f0a16da5640 1 --2- 172.21.15.83:0/1779022493 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a1015ba30 0x7f0a100b4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:30.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:30.137+0000 7f0a16da5640 1 --2- 172.21.15.83:0/1779022493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a100cfa10 0x7f0a10143bf0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:30.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:30.137+0000 7f0a16da5640 1 -- 172.21.15.83:0/1779022493 >> 172.21.15.83:0/1779022493 conn(0x7f0a1006abe0 msgr2=0x7f0a100b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:30.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:30.137+0000 7f0a16da5640 1 -- 172.21.15.83:0/1779022493 shutdown_connections 2024-04-17T10:49:30.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:30.137+0000 7f0a16da5640 1 -- 172.21.15.83:0/1779022493 wait complete. 2024-04-17T10:49:30.144 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T10:49:30.144 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T10:49:30.144 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-04-17T10:49:30.144 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T10:49:30.144 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T10:49:30.144 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T10:49:30.144 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T10:49:30.144 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T10:49:30.144 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:49:30.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:30.144 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-04-17T10:49:30.150 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T10:49:30.150 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T10:49:30.150 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T10:49:30.150 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:49:30.151 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-17T10:49:30.151 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:49:30.151 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:49:30.151 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-04-17T10:49:30.151 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-04-17T10:49:30.151 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:49:30.151 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-04-17T10:49:30.151 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-04-17T10:49:30.151 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:49:30.151 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:49:30.151 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T10:49:30.151 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:49:30.151 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-17T10:49:30.152 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 41814sec preferred_lft 41814sec 2024-04-17T10:49:30.152 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-04-17T10:49:30.152 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:49:30.152 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T10:49:30.152 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:37:0e:23:b2 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:49:30.152 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T10:49:30.152 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:49:30.152 INFO:teuthology.orchestra.run.smithi083.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T10:49:30.152 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-17T10:49:30.152 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T10:49:30.152 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:49:30.152 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-17T10:49:30.152 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:49:30.153 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:49:30.153 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:49:30.153 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-17T10:49:30.153 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:49:30.153 INFO:teuthology.orchestra.run.smithi083.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:49:30.153 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:49:30.153 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-17T10:49:30.153 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:49:30.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:30.153 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T10:49:30.200 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-04-17T10:49:30.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:30.202 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-04-17T10:49:30.251 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-04-17T10:49:30.252 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T10:49:30.252 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-17T10:49:30.252 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-17T10:49:30.252 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-17T10:49:30.252 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T10:49:30.252 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-17T10:49:30.252 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:49:30.252 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:49:30.252 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:49:30.252 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T10:49:30.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:30.253 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2024-04-17T10:49:30.253 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:30.253 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-17T10:49:30.253 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-17T10:49:30.253 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-17T10:49:30.253 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:30.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:30.327 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-17T10:49:30.380 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-17T10:49:30.381 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T10:49:30.381 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:49:30.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:30.381 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-04-17T10:49:30.435 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-17T10:49:30.435 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:49:30.436 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T10:49:30.485 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:49:30.486 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:49:30.487 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:49:30.532 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:49:30.591 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:49:30.637 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:49:30.637 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:49:30.638 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:49:30.638 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:49:30.638 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:49:30.638 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:49:30.638 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:49:30.638 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:49:30.638 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:49:30.638 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:49:30.638 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:49:30.639 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:49:30.639 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:49:30.639 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:49:30.639 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:49:30.639 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:49:30.639 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:49:30.639 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:49:30.639 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:49:30.639 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:49:30.639 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:49:30.639 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:49:30.639 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:49:30.639 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:49:30.639 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:49:30.640 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:49:30.640 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:49:30.640 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:49:30.640 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:49:30.640 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:49:30.640 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:49:30.640 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:49:30.640 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:49:30.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:30.641 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:49:30.689 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T10:49:30.689 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-17T10:49:30.734 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:49:30.749 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:49:30.797 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:49:30.797 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:49:30.797 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:49:30.797 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:49:30.797 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:49:30.798 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:49:30.798 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:49:30.798 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:49:30.798 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:49:30.798 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:49:30.798 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:49:30.798 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:49:30.798 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:49:30.798 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:49:30.798 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:49:30.798 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:49:30.798 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:49:30.799 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:49:30.799 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:49:30.799 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:49:30.799 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:49:30.799 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:49:30.799 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:49:30.799 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:49:30.799 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:49:30.799 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:49:30.799 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:49:30.799 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:49:30.799 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:49:30.799 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:49:30.799 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:49:30.800 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:49:30.800 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:49:30.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:30.800 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:49:30.809 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:2024-04-17T10:49:30.817+0000 7ff99fb3f580 -1 init, newargv = 0x55930a2ed290 newargc=15 2024-04-17T10:49:30.809 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[80023]: starting ceph client 2024-04-17T10:49:30.815 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[80023]: starting fuse 2024-04-17T10:49:31.915 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:49:31.931 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:49:31.977 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:49:32.000 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:49:32.000 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:49:32.001 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:49:32.001 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:49:32.001 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:49:32.001 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:49:32.001 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:49:32.001 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:49:32.001 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:49:32.001 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:49:32.001 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:49:32.001 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:49:32.001 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:49:32.001 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:49:32.001 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:49:32.001 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:49:32.002 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:49:32.002 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:49:32.002 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:49:32.002 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:49:32.002 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:49:32.002 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:49:32.002 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:49:32.002 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:49:32.002 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:49:32.002 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:49:32.002 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:49:32.002 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:49:32.003 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:49:32.003 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:49:32.003 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:49:32.003 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:49:32.003 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-17T10:49:32.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:32.004 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:49:32.052 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-04-17T10:49:32.053 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-17T10:49:32.053 DEBUG:teuthology.orchestra.run.smithi083:> sudo stdin-killer -- python3 -c ' 2024-04-17T10:49:32.053 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2024-04-17T10:49:32.053 DEBUG:teuthology.orchestra.run.smithi083:> import re 2024-04-17T10:49:32.053 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-04-17T10:49:32.053 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2024-04-17T10:49:32.053 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:49:32.053 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2024-04-17T10:49:32.053 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-17T10:49:32.054 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2024-04-17T10:49:32.054 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-17T10:49:32.054 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:49:32.054 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2024-04-17T10:49:32.054 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2024-04-17T10:49:32.054 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2024-04-17T10:49:32.054 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2024-04-17T10:49:32.054 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:49:32.054 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2024-04-17T10:49:32.054 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T10:49:32.054 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T10:49:32.054 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T10:49:32.054 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2024-04-17T10:49:32.054 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2024-04-17T10:49:32.055 DEBUG:teuthology.orchestra.run.smithi083:> return f 2024-04-17T10:49:32.055 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T10:49:32.055 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:49:32.055 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.0")) 2024-04-17T10:49:32.055 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-04-17T10:49:32.178 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.0.80023.asok 2024-04-17T10:49:32.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:49:32.190 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.80023.asok 2024-04-17T10:49:32.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:32.190 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.80023.asok status 2024-04-17T10:49:32.342 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:49:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2024-04-17T10:49:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-17T10:49:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-17T10:49:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "0", 2024-04-17T10:49:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-04-17T10:49:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-17T10:49:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "80023", 2024-04-17T10:49:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-04-17T10:49:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:49:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2024-04-17T10:49:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2024-04-17T10:49:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 9366, 2024-04-17T10:49:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2024-04-17T10:49:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-04-17T10:49:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-04-17T10:49:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 9366 2024-04-17T10:49:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:49:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-04-17T10:49:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-17T10:49:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:49:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3159455557 2024-04-17T10:49:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:49:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:49:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-04-17T10:49:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-17T10:49:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:49:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3159455557 2024-04-17T10:49:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:49:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.9366 192.168.0.1:0/3159455557", 2024-04-17T10:49:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.1:0/3159455557", 2024-04-17T10:49:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 1, 2024-04-17T10:49:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 76, 2024-04-17T10:49:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 127, 2024-04-17T10:49:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2024-04-17T10:49:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2024-04-17T10:49:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2024-04-17T10:49:32.345 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:49:32.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:32.357 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2024-04-17T10:49:32.357 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-17T10:49:32.362 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2024-04-17T10:49:32.362 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-17T10:49:32.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:32.362 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:49:32.415 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:49:32.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.585+0000 7f1191652640 1 Processor -- start 2024-04-17T10:49:32.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.589+0000 7f1191652640 1 -- start start 2024-04-17T10:49:32.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.589+0000 7f1191652640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f118c15db50 0x7f118c070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:32.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.589+0000 7f1191652640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f118c0aedd0 0x7f118c06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:32.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.589+0000 7f1191652640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f118c0709b0 con 0x7f118c06dae0 2024-04-17T10:49:32.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.589+0000 7f1191652640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f118c15e440 con 0x7f118c15db50 2024-04-17T10:49:32.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.589+0000 7f1191652640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f118c15e720 con 0x7f118c0aedd0 2024-04-17T10:49:32.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.589+0000 7f118affd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f118c15db50 0x7f118c070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:32.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.589+0000 7f118affd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f118c15db50 0x7f118c070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58000/0 (socket says 172.21.15.83:58000) 2024-04-17T10:49:32.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.589+0000 7f118affd640 1 -- 172.21.15.83:0/402307971 learned_addr learned my addr 172.21.15.83:0/402307971 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:32.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.589+0000 7f118a7fc640 1 --2- 172.21.15.83:0/402307971 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f118c0aedd0 0x7f118c06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:32.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.589+0000 7f118affd640 1 -- 172.21.15.83:0/402307971 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f118c0aedd0 msgr2=0x7f118c06d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:32.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.589+0000 7f118affd640 1 --2- 172.21.15.83:0/402307971 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f118c0aedd0 0x7f118c06d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:32.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.589+0000 7f118affd640 1 -- 172.21.15.83:0/402307971 >> v1:172.21.15.167:6789/0 conn(0x7f118c06dae0 legacy=0x7f118c0cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:32.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.589+0000 7f118affd640 1 -- 172.21.15.83:0/402307971 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f118c15ea00 con 0x7f118c15db50 2024-04-17T10:49:32.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.589+0000 7f118affd640 1 --2- 172.21.15.83:0/402307971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f118c15db50 0x7f118c070470 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f117c009870 tx=0x7f117c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c45d1759e5dee647 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:32.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f1189ffb640 1 -- 172.21.15.83:0/402307971 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c0ab650 con 0x7f118c15db50 2024-04-17T10:49:32.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f1189ffb640 1 -- 172.21.15.83:0/402307971 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117c0b0070 con 0x7f118c15db50 2024-04-17T10:49:32.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f1189ffb640 1 -- 172.21.15.83:0/402307971 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c0aba10 con 0x7f118c15db50 2024-04-17T10:49:32.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f1191652640 1 -- 172.21.15.83:0/402307971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f118c15db50 msgr2=0x7f118c070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:32.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f1191652640 1 --2- 172.21.15.83:0/402307971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f118c15db50 0x7f118c070470 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f117c009870 tx=0x7f117c01c560 comp rx=0 tx=0).stop 2024-04-17T10:49:32.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f1191652640 1 -- 172.21.15.83:0/402307971 shutdown_connections 2024-04-17T10:49:32.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f1191652640 1 --2- 172.21.15.83:0/402307971 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f118c0aedd0 0x7f118c06d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:32.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f1191652640 1 --2- 172.21.15.83:0/402307971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f118c15db50 0x7f118c070470 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:32.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f1191652640 1 -- 172.21.15.83:0/402307971 >> 172.21.15.83:0/402307971 conn(0x7f118c06abe0 msgr2=0x7f118c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:32.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f1191652640 1 -- 172.21.15.83:0/402307971 shutdown_connections 2024-04-17T10:49:32.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f1191652640 1 -- 172.21.15.83:0/402307971 wait complete. 2024-04-17T10:49:32.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f1191652640 1 Processor -- start 2024-04-17T10:49:32.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f1191652640 1 -- start start 2024-04-17T10:49:32.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f1191652640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f118c06dae0 0x7f118c0ae660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:32.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f1191652640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f118c0aedd0 0x7f118c1522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:32.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f1191652640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f118c15db50 0x7f118c14dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:32.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f1191652640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c06c610 con 0x7f118c0aedd0 2024-04-17T10:49:32.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f1191652640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c06c490 con 0x7f118c15db50 2024-04-17T10:49:32.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f1191652640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f118c06c790 con 0x7f118c06dae0 2024-04-17T10:49:32.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f118a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f118c0aedd0 0x7f118c1522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:32.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f118a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f118c0aedd0 0x7f118c1522f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58010/0 (socket says 172.21.15.83:58010) 2024-04-17T10:49:32.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f118a7fc640 1 -- 172.21.15.83:0/3088350951 learned_addr learned my addr 172.21.15.83:0/3088350951 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:32.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f118affd640 1 --2- 172.21.15.83:0/3088350951 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f118c06dae0 0x7f118c0ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:32.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f118b7fe640 1 --2- 172.21.15.83:0/3088350951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f118c15db50 0x7f118c14dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:32.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f118a7fc640 1 -- 172.21.15.83:0/3088350951 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f118c06dae0 msgr2=0x7f118c0ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:32.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f118a7fc640 1 --2- 172.21.15.83:0/3088350951 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f118c06dae0 0x7f118c0ae660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:32.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f118a7fc640 1 -- 172.21.15.83:0/3088350951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f118c15db50 msgr2=0x7f118c14dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:32.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f118a7fc640 1 --2- 172.21.15.83:0/3088350951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f118c15db50 0x7f118c14dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:32.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f118a7fc640 1 -- 172.21.15.83:0/3088350951 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f118c14e380 con 0x7f118c0aedd0 2024-04-17T10:49:32.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f118a7fc640 1 --2- 172.21.15.83:0/3088350951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f118c0aedd0 0x7f118c1522f0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f118000d800 tx=0x7f118000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:32.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f11737fe640 1 -- 172.21.15.83:0/3088350951 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118000b6c0 con 0x7f118c0aedd0 2024-04-17T10:49:32.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f11737fe640 1 -- 172.21.15.83:0/3088350951 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1180019070 con 0x7f118c0aedd0 2024-04-17T10:49:32.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f1191652640 1 -- 172.21.15.83:0/3088350951 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f118c14e660 con 0x7f118c0aedd0 2024-04-17T10:49:32.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f11737fe640 1 -- 172.21.15.83:0/3088350951 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118000ba60 con 0x7f118c0aedd0 2024-04-17T10:49:32.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.593+0000 7f1191652640 1 -- 172.21.15.83:0/3088350951 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f118c14ec20 con 0x7f118c0aedd0 2024-04-17T10:49:32.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.597+0000 7f1191652640 1 -- 172.21.15.83:0/3088350951 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f118c074f90 con 0x7f118c0aedd0 2024-04-17T10:49:32.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.597+0000 7f11737fe640 1 -- 172.21.15.83:0/3088350951 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f118000bc00 con 0x7f118c0aedd0 2024-04-17T10:49:32.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.597+0000 7f11737fe640 1 --2- 172.21.15.83:0/3088350951 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1178066f60 0x7f1178069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:32.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.597+0000 7f11737fe640 1 -- 172.21.15.83:0/3088350951 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1180085490 con 0x7f118c0aedd0 2024-04-17T10:49:32.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.597+0000 7f118affd640 1 --2- 172.21.15.83:0/3088350951 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1178066f60 0x7f1178069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:32.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.597+0000 7f118affd640 1 --2- 172.21.15.83:0/3088350951 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1178066f60 0x7f1178069420 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f117c0af000 tx=0x7f117c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:32.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.601+0000 7f11737fe640 1 -- 172.21.15.83:0/3088350951 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f11800564a0 con 0x7f118c0aedd0 2024-04-17T10:49:32.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.793+0000 7f1191652640 1 -- 172.21.15.83:0/3088350951 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f118c0d0710 con 0x7f118c0aedd0 2024-04-17T10:49:32.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.797+0000 7f11737fe640 1 -- 172.21.15.83:0/3088350951 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f1180055da0 con 0x7f118c0aedd0 2024-04-17T10:49:32.788 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:32.788 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5942,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3888166846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3888166846},{"type":"v1","addr":"172.21.15.167:6833","nonce":3888166846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":6772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1167913722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1167913722},{"type":"v1","addr":"172.21.15.167:6835","nonce":1167913722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":9192,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3552333026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3552333026},{"type":"v1","addr":"172.21.15.83:6835","nonce":3552333026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:49:19.962680+0000","modified":"2024-04-17T10:49:21.996802+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9198":{"gid":9198,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1559222491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1559222491},{"type":"v1","addr":"172.21.15.83:6837","nonce":1559222491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-17T10:49:32.789 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 76 2024-04-17T10:49:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.797+0000 7f1191652640 1 -- 172.21.15.83:0/3088350951 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1178066f60 msgr2=0x7f1178069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.797+0000 7f1191652640 1 --2- 172.21.15.83:0/3088350951 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1178066f60 0x7f1178069420 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f117c0af000 tx=0x7f117c0a7000 comp rx=0 tx=0).stop 2024-04-17T10:49:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.797+0000 7f1191652640 1 -- 172.21.15.83:0/3088350951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f118c0aedd0 msgr2=0x7f118c1522f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.797+0000 7f1191652640 1 --2- 172.21.15.83:0/3088350951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f118c0aedd0 0x7f118c1522f0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f118000d800 tx=0x7f118000dcc0 comp rx=0 tx=0).stop 2024-04-17T10:49:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.797+0000 7f1191652640 1 -- 172.21.15.83:0/3088350951 shutdown_connections 2024-04-17T10:49:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.797+0000 7f1191652640 1 --2- 172.21.15.83:0/3088350951 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1178066f60 0x7f1178069420 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.797+0000 7f1191652640 1 --2- 172.21.15.83:0/3088350951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f118c15db50 0x7f118c14dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.797+0000 7f1191652640 1 --2- 172.21.15.83:0/3088350951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f118c0aedd0 0x7f118c1522f0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.797+0000 7f1191652640 1 --2- 172.21.15.83:0/3088350951 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f118c06dae0 0x7f118c0ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.797+0000 7f1191652640 1 -- 172.21.15.83:0/3088350951 >> 172.21.15.83:0/3088350951 conn(0x7f118c06abe0 msgr2=0x7f118c0b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.797+0000 7f1191652640 1 -- 172.21.15.83:0/3088350951 shutdown_connections 2024-04-17T10:49:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.797+0000 7f1191652640 1 -- 172.21.15.83:0/3088350951 wait complete. 2024-04-17T10:49:32.807 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:49:32.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.925+0000 7f47d7f39640 1 Processor -- start 2024-04-17T10:49:32.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.925+0000 7f47d7f39640 1 -- start start 2024-04-17T10:49:32.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.925+0000 7f47d7f39640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f47d00ce980 0x7f47d00b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:32.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.925+0000 7f47d7f39640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47d0162a00 con 0x7f47d00cee00 2024-04-17T10:49:32.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.925+0000 7f47d7f39640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47d0161e80 con 0x7f47d015dfa0 2024-04-17T10:49:32.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.925+0000 7f47d7f39640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d0162160 con 0x7f47d00ce980 2024-04-17T10:49:32.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.925+0000 7f47d64d6640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f47d00cee00 0x7f47d00d1d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59204/0 (socket says 172.21.15.83:59204) 2024-04-17T10:49:32.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.925+0000 7f47d64d6640 1 -- 172.21.15.83:0/3429034186 learned_addr learned my addr 172.21.15.83:0/3429034186 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:32.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.929+0000 7f47d54d4640 1 --2- 172.21.15.83:0/3429034186 >> v2:172.21.15.167:3300/0 conn(0x7f47d00ce980 0x7f47d00b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:32.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.929+0000 7f47d4cd3640 1 -- 172.21.15.83:0/3429034186 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 638063092 0 0) 0x7f47d0162a00 con 0x7f47d00cee00 2024-04-17T10:49:32.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.929+0000 7f47d4cd3640 1 -- 172.21.15.83:0/3429034186 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47bc003530 con 0x7f47d00cee00 2024-04-17T10:49:32.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.929+0000 7f47d4cd3640 1 -- 172.21.15.83:0/3429034186 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 387372564 0 0) 0x7f47bc003530 con 0x7f47d00cee00 2024-04-17T10:49:32.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.929+0000 7f47d4cd3640 1 -- 172.21.15.83:0/3429034186 >> v2:172.21.15.167:3300/0 conn(0x7f47d00ce980 msgr2=0x7f47d00b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:32.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.929+0000 7f47d4cd3640 1 --2- 172.21.15.83:0/3429034186 >> v2:172.21.15.167:3300/0 conn(0x7f47d00ce980 0x7f47d00b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:32.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.929+0000 7f47d4cd3640 1 -- 172.21.15.83:0/3429034186 >> v1:172.21.15.167:6789/0 conn(0x7f47d015dfa0 legacy=0x7f47d00b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:32.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.929+0000 7f47d4cd3640 1 -- 172.21.15.83:0/3429034186 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d0162440 con 0x7f47d00cee00 2024-04-17T10:49:32.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.929+0000 7f47d4cd3640 1 -- 172.21.15.83:0/3429034186 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f47c8001a00 con 0x7f47d00cee00 2024-04-17T10:49:32.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.929+0000 7f47d54d4640 1 --2- 172.21.15.83:0/3429034186 >> v2:172.21.15.167:3300/0 conn(0x7f47d00ce980 0x7f47d00b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:49:32.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.929+0000 7f47d4cd3640 1 -- 172.21.15.83:0/3429034186 >> v1:172.21.15.83:6789/0 conn(0x7f47d00cee00 legacy=0x7f47d00d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:32.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.929+0000 7f47d4cd3640 1 --2- 172.21.15.83:0/3429034186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47bc0039c0 0x7f47bc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:32.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.929+0000 7f47d4cd3640 1 --2- 172.21.15.83:0/3429034186 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f47bc004810 0x7f47bc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:32.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.929+0000 7f47d4cd3640 1 --2- 172.21.15.83:0/3429034186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47bc0052f0 0x7f47bc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:32.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.929+0000 7f47d4cd3640 1 -- 172.21.15.83:0/3429034186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d0162160 con 0x7f47bc0052f0 2024-04-17T10:49:32.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.929+0000 7f47d4cd3640 1 -- 172.21.15.83:0/3429034186 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc00c1c0 con 0x7f47bc0039c0 2024-04-17T10:49:32.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.929+0000 7f47d4cd3640 1 -- 172.21.15.83:0/3429034186 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc00c3c0 con 0x7f47bc004810 2024-04-17T10:49:32.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.929+0000 7f47d64d6640 1 --2- 172.21.15.83:0/3429034186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47bc0052f0 0x7f47bc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:32.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.929+0000 7f47d5cd5640 1 --2- 172.21.15.83:0/3429034186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47bc0039c0 0x7f47bc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:32.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.929+0000 7f47d54d4640 1 --2- 172.21.15.83:0/3429034186 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f47bc004810 0x7f47bc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:32.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.929+0000 7f47d64d6640 1 -- 172.21.15.83:0/3429034186 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f47bc004810 msgr2=0x7f47bc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:32.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.929+0000 7f47d64d6640 1 --2- 172.21.15.83:0/3429034186 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f47bc004810 0x7f47bc004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:32.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.929+0000 7f47d64d6640 1 -- 172.21.15.83:0/3429034186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47bc0039c0 msgr2=0x7f47bc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:32.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.929+0000 7f47d64d6640 1 --2- 172.21.15.83:0/3429034186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47bc0039c0 0x7f47bc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:32.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.929+0000 7f47d64d6640 1 -- 172.21.15.83:0/3429034186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47bc003530 con 0x7f47bc0052f0 2024-04-17T10:49:32.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.929+0000 7f47d64d6640 1 -- 172.21.15.83:0/3429034186 reap_dead start 2024-04-17T10:49:32.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.929+0000 7f47d64d6640 1 --2- 172.21.15.83:0/3429034186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47bc0052f0 0x7f47bc00b9d0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f47c8002240 tx=0x7f47c801fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682fec06a3a3407e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:32.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.929+0000 7f47d4cd3640 1 -- 172.21.15.83:0/3429034186 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c800a4a0 con 0x7f47bc0052f0 2024-04-17T10:49:32.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.929+0000 7f47d4cd3640 1 -- 172.21.15.83:0/3429034186 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c80aa4f0 con 0x7f47bc0052f0 2024-04-17T10:49:32.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.929+0000 7f47d4cd3640 1 -- 172.21.15.83:0/3429034186 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c80aa8c0 con 0x7f47bc0052f0 2024-04-17T10:49:32.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47d7f39640 1 -- 172.21.15.83:0/3429034186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47bc0052f0 msgr2=0x7f47bc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:32.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47d7f39640 1 --2- 172.21.15.83:0/3429034186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47bc0052f0 0x7f47bc00b9d0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f47c8002240 tx=0x7f47c801fac0 comp rx=0 tx=0).stop 2024-04-17T10:49:32.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47d7f39640 1 -- 172.21.15.83:0/3429034186 shutdown_connections 2024-04-17T10:49:32.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47d7f39640 1 --2- 172.21.15.83:0/3429034186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47bc0052f0 0x7f47bc00b9d0 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:32.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47d7f39640 1 -- 172.21.15.83:0/3429034186 >> 172.21.15.83:0/3429034186 conn(0x7f47d00d0600 msgr2=0x7f47d00d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:32.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47d7f39640 1 -- 172.21.15.83:0/3429034186 shutdown_connections 2024-04-17T10:49:32.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47d7f39640 1 -- 172.21.15.83:0/3429034186 wait complete. 2024-04-17T10:49:32.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47d7f39640 1 Processor -- start 2024-04-17T10:49:32.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47d7f39640 1 -- start start 2024-04-17T10:49:32.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47d7f39640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47bc0052f0 0x7f47d00c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:32.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47d7f39640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47d014d8e0 0x7f47d0160570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:32.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47d7f39640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f47d00c74c0 0x7f47d00c7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:32.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47d7f39640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d0161b10 con 0x7f47d014d8e0 2024-04-17T10:49:32.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47d7f39640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d015e380 con 0x7f47bc0052f0 2024-04-17T10:49:32.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47d7f39640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d0161c90 con 0x7f47d00c74c0 2024-04-17T10:49:32.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47d54d4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47d014d8e0 0x7f47d0160570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:32.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47d54d4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47d014d8e0 0x7f47d0160570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58016/0 (socket says 172.21.15.83:58016) 2024-04-17T10:49:32.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47d54d4640 1 -- 172.21.15.83:0/1424850104 learned_addr learned my addr 172.21.15.83:0/1424850104 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:32.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47d64d6640 1 --2- 172.21.15.83:0/1424850104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f47d00c74c0 0x7f47d00c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:32.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47d5cd5640 1 --2- 172.21.15.83:0/1424850104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47bc0052f0 0x7f47d00c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:32.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47d54d4640 1 -- 172.21.15.83:0/1424850104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f47d00c74c0 msgr2=0x7f47d00c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:32.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47d54d4640 1 --2- 172.21.15.83:0/1424850104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f47d00c74c0 0x7f47d00c7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:32.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47d54d4640 1 -- 172.21.15.83:0/1424850104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47bc0052f0 msgr2=0x7f47d00c9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:32.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47d54d4640 1 --2- 172.21.15.83:0/1424850104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47bc0052f0 0x7f47d00c9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:32.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47d54d4640 1 -- 172.21.15.83:0/1424850104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d00c8b30 con 0x7f47d014d8e0 2024-04-17T10:49:32.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47d64d6640 1 --2- 172.21.15.83:0/1424850104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f47d00c74c0 0x7f47d00c7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:32.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47d54d4640 1 --2- 172.21.15.83:0/1424850104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47d014d8e0 0x7f47d0160570 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f47c000e590 tx=0x7f47c000ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:32.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47c67fc640 1 -- 172.21.15.83:0/1424850104 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c0017070 con 0x7f47d014d8e0 2024-04-17T10:49:32.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47c67fc640 1 -- 172.21.15.83:0/1424850104 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c0013610 con 0x7f47d014d8e0 2024-04-17T10:49:32.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47d7f39640 1 -- 172.21.15.83:0/1424850104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d00c8e10 con 0x7f47d014d8e0 2024-04-17T10:49:32.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47c67fc640 1 -- 172.21.15.83:0/1424850104 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c000f040 con 0x7f47d014d8e0 2024-04-17T10:49:32.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47d7f39640 1 -- 172.21.15.83:0/1424850104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d00c93d0 con 0x7f47d014d8e0 2024-04-17T10:49:32.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.933+0000 7f47d7f39640 1 -- 172.21.15.83:0/1424850104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47d00681a0 con 0x7f47d014d8e0 2024-04-17T10:49:32.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.937+0000 7f47c67fc640 1 -- 172.21.15.83:0/1424850104 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f47c001a070 con 0x7f47d014d8e0 2024-04-17T10:49:32.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.937+0000 7f47c67fc640 1 --2- 172.21.15.83:0/1424850104 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f47b4066f90 0x7f47b4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:32.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.937+0000 7f47c67fc640 1 -- 172.21.15.83:0/1424850104 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47c00849e0 con 0x7f47d014d8e0 2024-04-17T10:49:32.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.937+0000 7f47d5cd5640 1 --2- 172.21.15.83:0/1424850104 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f47b4066f90 0x7f47b4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:32.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.937+0000 7f47d5cd5640 1 --2- 172.21.15.83:0/1424850104 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f47b4066f90 0x7f47b4069450 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f47cc0015a0 tx=0x7f47cc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:32.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:32.941+0000 7f47c67fc640 1 -- 172.21.15.83:0/1424850104 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f47c0055a80 con 0x7f47d014d8e0 2024-04-17T10:49:33.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:33.129+0000 7f47d7f39640 1 -- 172.21.15.83:0/1424850104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47d0074f90 con 0x7f47d014d8e0 2024-04-17T10:49:33.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:33.129+0000 7f47c67fc640 1 -- 172.21.15.83:0/1424850104 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f47c0055380 con 0x7f47d014d8e0 2024-04-17T10:49:33.126 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:33.126 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5942,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3888166846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3888166846},{"type":"v1","addr":"172.21.15.167:6833","nonce":3888166846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":6772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1167913722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1167913722},{"type":"v1","addr":"172.21.15.167:6835","nonce":1167913722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":9192,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3552333026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3552333026},{"type":"v1","addr":"172.21.15.83:6835","nonce":3552333026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:49:19.962680+0000","modified":"2024-04-17T10:49:21.996802+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9198":{"gid":9198,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1559222491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1559222491},{"type":"v1","addr":"172.21.15.83:6837","nonce":1559222491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-17T10:49:33.128 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 76 2024-04-17T10:49:33.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:33.133+0000 7f47d7f39640 1 -- 172.21.15.83:0/1424850104 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f47b4066f90 msgr2=0x7f47b4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:33.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:33.133+0000 7f47d7f39640 1 --2- 172.21.15.83:0/1424850104 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f47b4066f90 0x7f47b4069450 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f47cc0015a0 tx=0x7f47cc009040 comp rx=0 tx=0).stop 2024-04-17T10:49:33.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:33.133+0000 7f47d7f39640 1 -- 172.21.15.83:0/1424850104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47d014d8e0 msgr2=0x7f47d0160570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:33.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:33.133+0000 7f47d7f39640 1 --2- 172.21.15.83:0/1424850104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47d014d8e0 0x7f47d0160570 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f47c000e590 tx=0x7f47c000ea50 comp rx=0 tx=0).stop 2024-04-17T10:49:33.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:33.133+0000 7f47d7f39640 1 -- 172.21.15.83:0/1424850104 shutdown_connections 2024-04-17T10:49:33.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:33.133+0000 7f47d7f39640 1 --2- 172.21.15.83:0/1424850104 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f47b4066f90 0x7f47b4069450 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:33.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:33.133+0000 7f47d7f39640 1 --2- 172.21.15.83:0/1424850104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f47d00c74c0 0x7f47d00c7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:33.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:33.133+0000 7f47d7f39640 1 --2- 172.21.15.83:0/1424850104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47d014d8e0 0x7f47d0160570 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:33.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:33.133+0000 7f47d7f39640 1 --2- 172.21.15.83:0/1424850104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47bc0052f0 0x7f47d00c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:33.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:33.133+0000 7f47d7f39640 1 -- 172.21.15.83:0/1424850104 >> 172.21.15.83:0/1424850104 conn(0x7f47d00d0600 msgr2=0x7f47d00b91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:33.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:33.133+0000 7f47d7f39640 1 -- 172.21.15.83:0/1424850104 shutdown_connections 2024-04-17T10:49:33.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:33.133+0000 7f47d7f39640 1 -- 172.21.15.83:0/1424850104 wait complete. 2024-04-17T10:49:33.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:33.143 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:49:33.143 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:33.143 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.0 2024-04-17T10:49:33.143 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:49:33.143 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:49:33.143 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:49:33.143 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:33.180 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.181 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.181 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160541 s, 65.3 MB/s 2024-04-17T10:49:33.197 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.197 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.197 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140811 s, 74.5 MB/s 2024-04-17T10:49:33.208 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.208 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.208 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00922522 s, 114 MB/s 2024-04-17T10:49:33.221 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.221 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.221 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107791 s, 97.3 MB/s 2024-04-17T10:49:33.236 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.236 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.236 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125455 s, 83.6 MB/s 2024-04-17T10:49:33.246 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.246 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.247 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859776 s, 122 MB/s 2024-04-17T10:49:33.259 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.259 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.259 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105216 s, 99.7 MB/s 2024-04-17T10:49:33.274 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.274 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.274 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123302 s, 85.0 MB/s 2024-04-17T10:49:33.289 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.289 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.289 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131847 s, 79.5 MB/s 2024-04-17T10:49:33.299 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.299 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.299 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804545 s, 130 MB/s 2024-04-17T10:49:33.310 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.310 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.310 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0089717 s, 117 MB/s 2024-04-17T10:49:33.323 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.323 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.323 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011124 s, 94.3 MB/s 2024-04-17T10:49:33.336 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.336 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.336 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100737 s, 104 MB/s 2024-04-17T10:49:33.350 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.350 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.350 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122991 s, 85.3 MB/s 2024-04-17T10:49:33.364 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.364 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.365 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121206 s, 86.5 MB/s 2024-04-17T10:49:33.379 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.379 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.379 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122293 s, 85.7 MB/s 2024-04-17T10:49:33.393 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.393 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.393 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116969 s, 89.6 MB/s 2024-04-17T10:49:33.405 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.405 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.405 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104336 s, 100 MB/s 2024-04-17T10:49:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.418 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.418 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103535 s, 101 MB/s 2024-04-17T10:49:33.427 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.427 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.428 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787274 s, 133 MB/s 2024-04-17T10:49:33.437 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.437 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.438 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800874 s, 131 MB/s 2024-04-17T10:49:33.451 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.451 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.451 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114749 s, 91.4 MB/s 2024-04-17T10:49:33.464 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.464 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.464 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106572 s, 98.4 MB/s 2024-04-17T10:49:33.478 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.478 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.478 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119699 s, 87.6 MB/s 2024-04-17T10:49:33.490 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.490 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.491 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103227 s, 102 MB/s 2024-04-17T10:49:33.504 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.504 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.504 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011468 s, 91.4 MB/s 2024-04-17T10:49:33.517 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.517 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.517 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107793 s, 97.3 MB/s 2024-04-17T10:49:33.531 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.531 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.531 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113652 s, 92.3 MB/s 2024-04-17T10:49:33.542 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.542 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.542 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00868368 s, 121 MB/s 2024-04-17T10:49:33.553 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.553 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.553 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00914215 s, 115 MB/s 2024-04-17T10:49:33.567 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.567 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.567 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011998 s, 87.4 MB/s 2024-04-17T10:49:33.583 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.584 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.584 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138638 s, 75.6 MB/s 2024-04-17T10:49:33.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:33.585 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:49:33.585 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:33.585 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.1 2024-04-17T10:49:33.585 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:49:33.585 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:49:33.585 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:49:33.585 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:33.651 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.651 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.651 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128144 s, 81.8 MB/s 2024-04-17T10:49:33.665 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.665 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.665 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114075 s, 91.9 MB/s 2024-04-17T10:49:33.679 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.679 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.679 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117958 s, 88.9 MB/s 2024-04-17T10:49:33.694 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.694 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.694 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012098 s, 86.7 MB/s 2024-04-17T10:49:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00967046 s, 108 MB/s 2024-04-17T10:49:33.719 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.719 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.719 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104416 s, 100 MB/s 2024-04-17T10:49:33.729 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.729 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.729 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788202 s, 133 MB/s 2024-04-17T10:49:33.741 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.741 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.741 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00998764 s, 105 MB/s 2024-04-17T10:49:33.753 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.754 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.754 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104927 s, 99.9 MB/s 2024-04-17T10:49:33.768 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.768 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.768 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120141 s, 87.3 MB/s 2024-04-17T10:49:33.781 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.781 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.781 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011166 s, 93.9 MB/s 2024-04-17T10:49:33.795 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.795 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.795 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122896 s, 85.3 MB/s 2024-04-17T10:49:33.805 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.806 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.806 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083128 s, 126 MB/s 2024-04-17T10:49:33.819 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.819 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.819 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108948 s, 96.2 MB/s 2024-04-17T10:49:33.834 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.834 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.834 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132911 s, 78.9 MB/s 2024-04-17T10:49:33.846 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.846 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.846 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00998207 s, 105 MB/s 2024-04-17T10:49:33.856 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.856 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.856 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813827 s, 129 MB/s 2024-04-17T10:49:33.871 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.871 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.871 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125453 s, 83.6 MB/s 2024-04-17T10:49:33.883 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.883 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.883 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00994901 s, 105 MB/s 2024-04-17T10:49:33.896 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.896 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.896 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107802 s, 97.3 MB/s 2024-04-17T10:49:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00977672 s, 107 MB/s 2024-04-17T10:49:33.920 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.921 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.921 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104216 s, 101 MB/s 2024-04-17T10:49:33.935 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.935 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.935 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125226 s, 83.7 MB/s 2024-04-17T10:49:33.948 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.948 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.948 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108099 s, 97.0 MB/s 2024-04-17T10:49:33.962 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.962 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.962 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116386 s, 90.1 MB/s 2024-04-17T10:49:33.975 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.975 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.975 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108284 s, 96.8 MB/s 2024-04-17T10:49:33.987 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.987 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.987 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00980722 s, 107 MB/s 2024-04-17T10:49:33.999 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:33.999 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:33.999 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102153 s, 103 MB/s 2024-04-17T10:49:34.012 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.012 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.012 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100949 s, 104 MB/s 2024-04-17T10:49:34.024 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.024 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.024 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100297 s, 105 MB/s 2024-04-17T10:49:34.036 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.036 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.037 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107602 s, 97.4 MB/s 2024-04-17T10:49:34.049 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.049 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.049 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105696 s, 99.2 MB/s 2024-04-17T10:49:34.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:34.051 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:49:34.051 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:34.051 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.2 2024-04-17T10:49:34.051 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:49:34.051 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:49:34.051 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:49:34.051 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:34.112 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.112 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.112 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107206 s, 97.8 MB/s 2024-04-17T10:49:34.122 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.122 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.122 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815432 s, 129 MB/s 2024-04-17T10:49:34.135 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.135 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.135 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104346 s, 100 MB/s 2024-04-17T10:49:34.147 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.147 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.147 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103771 s, 101 MB/s 2024-04-17T10:49:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007432 s, 141 MB/s 2024-04-17T10:49:34.170 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.170 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.170 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110901 s, 94.6 MB/s 2024-04-17T10:49:34.183 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.183 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.183 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010476 s, 100 MB/s 2024-04-17T10:49:34.195 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.195 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.195 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102015 s, 103 MB/s 2024-04-17T10:49:34.206 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.206 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880043 s, 119 MB/s 2024-04-17T10:49:34.217 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.217 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.217 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814113 s, 129 MB/s 2024-04-17T10:49:34.229 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.229 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.230 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102014 s, 103 MB/s 2024-04-17T10:49:34.242 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.242 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.242 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00996012 s, 105 MB/s 2024-04-17T10:49:34.253 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.253 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.253 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00894255 s, 117 MB/s 2024-04-17T10:49:34.264 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.264 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.264 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834592 s, 126 MB/s 2024-04-17T10:49:34.278 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.278 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.278 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118974 s, 88.1 MB/s 2024-04-17T10:49:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121384 s, 86.4 MB/s 2024-04-17T10:49:34.305 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.305 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.305 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110005 s, 95.3 MB/s 2024-04-17T10:49:34.320 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.320 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.320 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125612 s, 83.5 MB/s 2024-04-17T10:49:34.334 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.334 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.334 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117702 s, 89.1 MB/s 2024-04-17T10:49:34.348 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.348 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.348 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111746 s, 93.8 MB/s 2024-04-17T10:49:34.360 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.360 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.360 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103193 s, 102 MB/s 2024-04-17T10:49:34.374 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.374 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.374 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111803 s, 93.8 MB/s 2024-04-17T10:49:34.387 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.387 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.387 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109619 s, 95.7 MB/s 2024-04-17T10:49:34.400 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.400 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.400 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106069 s, 98.9 MB/s 2024-04-17T10:49:34.411 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.411 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.411 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00876504 s, 120 MB/s 2024-04-17T10:49:34.424 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.424 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.424 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111122 s, 94.4 MB/s 2024-04-17T10:49:34.435 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.435 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.435 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805394 s, 130 MB/s 2024-04-17T10:49:34.449 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.449 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.449 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118058 s, 88.8 MB/s 2024-04-17T10:49:34.464 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.464 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.464 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125349 s, 83.7 MB/s 2024-04-17T10:49:34.477 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.477 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.477 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108992 s, 96.2 MB/s 2024-04-17T10:49:34.491 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.491 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.492 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121651 s, 86.2 MB/s 2024-04-17T10:49:34.505 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.505 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.505 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113972 s, 92.0 MB/s 2024-04-17T10:49:34.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:34.506 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:49:34.506 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:34.506 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.3 2024-04-17T10:49:34.507 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:49:34.507 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:49:34.507 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:49:34.507 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:34.570 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.571 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.571 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131868 s, 79.5 MB/s 2024-04-17T10:49:34.580 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.580 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.581 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773183 s, 136 MB/s 2024-04-17T10:49:34.594 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.594 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.594 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113944 s, 92.0 MB/s 2024-04-17T10:49:34.607 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.607 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.607 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110385 s, 95.0 MB/s 2024-04-17T10:49:34.621 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.621 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.621 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118275 s, 88.7 MB/s 2024-04-17T10:49:34.632 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.632 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.632 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840902 s, 125 MB/s 2024-04-17T10:49:34.643 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.644 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.644 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00952279 s, 110 MB/s 2024-04-17T10:49:34.656 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.656 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.656 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102571 s, 102 MB/s 2024-04-17T10:49:34.669 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.669 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.669 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105604 s, 99.3 MB/s 2024-04-17T10:49:34.682 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.682 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.682 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110599 s, 94.8 MB/s 2024-04-17T10:49:34.692 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.692 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.692 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794022 s, 132 MB/s 2024-04-17T10:49:34.706 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.707 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.707 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118337 s, 88.6 MB/s 2024-04-17T10:49:34.716 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.716 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.716 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788221 s, 133 MB/s 2024-04-17T10:49:34.729 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.729 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.729 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106302 s, 98.6 MB/s 2024-04-17T10:49:34.739 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.739 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.739 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749776 s, 140 MB/s 2024-04-17T10:49:34.753 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.753 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.753 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105298 s, 99.6 MB/s 2024-04-17T10:49:34.765 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.765 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.765 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114044 s, 91.9 MB/s 2024-04-17T10:49:34.779 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.779 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.779 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116131 s, 90.3 MB/s 2024-04-17T10:49:34.793 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.793 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.793 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117302 s, 89.4 MB/s 2024-04-17T10:49:34.806 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.806 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.806 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113778 s, 92.2 MB/s 2024-04-17T10:49:34.821 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.821 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.822 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131926 s, 79.5 MB/s 2024-04-17T10:49:34.832 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.832 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.832 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808851 s, 130 MB/s 2024-04-17T10:49:34.845 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.846 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.846 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115296 s, 90.9 MB/s 2024-04-17T10:49:34.859 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.859 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.859 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010941 s, 95.8 MB/s 2024-04-17T10:49:34.871 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.871 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.871 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102545 s, 102 MB/s 2024-04-17T10:49:34.884 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.884 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.884 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102329 s, 102 MB/s 2024-04-17T10:49:34.897 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.897 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.897 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107991 s, 97.1 MB/s 2024-04-17T10:49:34.909 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.909 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.909 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102158 s, 103 MB/s 2024-04-17T10:49:34.921 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.921 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.922 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102236 s, 103 MB/s 2024-04-17T10:49:34.932 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.932 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.932 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824174 s, 127 MB/s 2024-04-17T10:49:34.946 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.946 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.946 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123391 s, 85.0 MB/s 2024-04-17T10:49:34.960 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:34.960 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:34.960 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114011 s, 92.0 MB/s 2024-04-17T10:49:34.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:34.961 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:49:34.961 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:34.961 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.4 2024-04-17T10:49:34.961 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:49:34.962 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:49:34.962 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:49:34.962 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:35.023 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.023 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.024 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108245 s, 96.9 MB/s 2024-04-17T10:49:35.038 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.038 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.038 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127523 s, 82.2 MB/s 2024-04-17T10:49:35.052 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.052 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.052 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113536 s, 92.4 MB/s 2024-04-17T10:49:35.061 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.061 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.061 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743664 s, 141 MB/s 2024-04-17T10:49:35.074 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.074 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.074 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011091 s, 94.5 MB/s 2024-04-17T10:49:35.089 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.089 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.089 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013002 s, 80.6 MB/s 2024-04-17T10:49:35.103 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.103 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.103 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118887 s, 88.2 MB/s 2024-04-17T10:49:35.114 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.114 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.114 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00853215 s, 123 MB/s 2024-04-17T10:49:35.128 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.128 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.128 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117789 s, 89.0 MB/s 2024-04-17T10:49:35.141 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.141 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.141 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111364 s, 94.2 MB/s 2024-04-17T10:49:35.155 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.155 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.155 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122898 s, 85.3 MB/s 2024-04-17T10:49:35.168 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.168 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.168 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112078 s, 93.6 MB/s 2024-04-17T10:49:35.182 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.182 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.182 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011159 s, 94.0 MB/s 2024-04-17T10:49:35.193 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.194 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.194 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00958472 s, 109 MB/s 2024-04-17T10:49:35.207 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.207 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.207 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011864 s, 88.4 MB/s 2024-04-17T10:49:35.221 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.221 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.221 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116617 s, 89.9 MB/s 2024-04-17T10:49:35.231 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.231 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.231 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817867 s, 128 MB/s 2024-04-17T10:49:35.241 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.241 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.241 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756506 s, 139 MB/s 2024-04-17T10:49:35.255 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.255 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.255 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122121 s, 85.9 MB/s 2024-04-17T10:49:35.264 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.265 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.265 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761631 s, 138 MB/s 2024-04-17T10:49:35.277 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.277 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.277 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103251 s, 102 MB/s 2024-04-17T10:49:35.289 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.289 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.289 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00982781 s, 107 MB/s 2024-04-17T10:49:35.301 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.301 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.301 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00973593 s, 108 MB/s 2024-04-17T10:49:35.313 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.313 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.313 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0098078 s, 107 MB/s 2024-04-17T10:49:35.325 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.325 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.325 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100592 s, 104 MB/s 2024-04-17T10:49:35.338 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.338 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.338 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106787 s, 98.2 MB/s 2024-04-17T10:49:35.350 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.350 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.350 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100745 s, 104 MB/s 2024-04-17T10:49:35.363 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.363 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.363 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010533 s, 99.6 MB/s 2024-04-17T10:49:35.375 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.375 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.376 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106149 s, 98.8 MB/s 2024-04-17T10:49:35.385 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.385 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.385 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745371 s, 141 MB/s 2024-04-17T10:49:35.399 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.399 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.399 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123879 s, 84.6 MB/s 2024-04-17T10:49:35.409 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.409 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.409 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767792 s, 137 MB/s 2024-04-17T10:49:35.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:35.410 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:49:35.410 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:35.410 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.5 2024-04-17T10:49:35.410 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:49:35.410 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:49:35.410 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:49:35.411 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:35.472 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.473 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.473 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109089 s, 96.1 MB/s 2024-04-17T10:49:35.486 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.486 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.486 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110658 s, 94.8 MB/s 2024-04-17T10:49:35.500 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.501 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.501 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123563 s, 84.9 MB/s 2024-04-17T10:49:35.515 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.515 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.516 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124729 s, 84.1 MB/s 2024-04-17T10:49:35.527 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.527 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.527 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00938449 s, 112 MB/s 2024-04-17T10:49:35.542 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.542 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.542 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124032 s, 84.5 MB/s 2024-04-17T10:49:35.557 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.558 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.558 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128641 s, 81.5 MB/s 2024-04-17T10:49:35.572 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.573 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.573 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118216 s, 88.7 MB/s 2024-04-17T10:49:35.588 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.588 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.589 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125722 s, 83.4 MB/s 2024-04-17T10:49:35.601 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.601 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.601 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106122 s, 98.8 MB/s 2024-04-17T10:49:35.616 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.616 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.616 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011709 s, 89.6 MB/s 2024-04-17T10:49:35.629 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.629 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.629 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109799 s, 95.5 MB/s 2024-04-17T10:49:35.643 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.643 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.644 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123416 s, 85.0 MB/s 2024-04-17T10:49:35.655 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.655 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.655 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00977472 s, 107 MB/s 2024-04-17T10:49:35.668 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.668 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.668 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010955 s, 95.7 MB/s 2024-04-17T10:49:35.678 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.678 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.678 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736284 s, 142 MB/s 2024-04-17T10:49:35.692 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.693 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.693 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124778 s, 84.0 MB/s 2024-04-17T10:49:35.702 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.702 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.702 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746048 s, 141 MB/s 2024-04-17T10:49:35.716 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.717 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.717 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119282 s, 87.9 MB/s 2024-04-17T10:49:35.726 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.726 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.726 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763072 s, 137 MB/s 2024-04-17T10:49:35.738 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.738 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.738 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101818 s, 103 MB/s 2024-04-17T10:49:35.752 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.752 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.752 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114823 s, 91.3 MB/s 2024-04-17T10:49:35.765 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.765 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.765 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112406 s, 93.3 MB/s 2024-04-17T10:49:35.778 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.778 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.778 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105109 s, 99.8 MB/s 2024-04-17T10:49:35.793 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.793 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.793 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125755 s, 83.4 MB/s 2024-04-17T10:49:35.805 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.805 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.805 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00984842 s, 106 MB/s 2024-04-17T10:49:35.819 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.819 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.820 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121759 s, 86.1 MB/s 2024-04-17T10:49:35.829 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.829 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.829 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789581 s, 133 MB/s 2024-04-17T10:49:35.842 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.842 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.842 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010738 s, 97.7 MB/s 2024-04-17T10:49:35.855 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.855 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.855 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107447 s, 97.6 MB/s 2024-04-17T10:49:35.866 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.866 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.866 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824603 s, 127 MB/s 2024-04-17T10:49:35.881 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.881 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.881 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132696 s, 79.0 MB/s 2024-04-17T10:49:35.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:35.883 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:49:35.883 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:35.883 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.6 2024-04-17T10:49:35.883 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:49:35.883 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:49:35.883 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:49:35.883 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:35.942 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.942 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.942 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793666 s, 132 MB/s 2024-04-17T10:49:35.955 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.955 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.955 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107267 s, 97.8 MB/s 2024-04-17T10:49:35.969 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.969 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.969 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125424 s, 83.6 MB/s 2024-04-17T10:49:35.982 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.982 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.982 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110336 s, 95.0 MB/s 2024-04-17T10:49:35.995 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:35.995 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:35.995 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106627 s, 98.3 MB/s 2024-04-17T10:49:36.009 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.009 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.009 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116882 s, 89.7 MB/s 2024-04-17T10:49:36.021 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.021 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.021 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00979085 s, 107 MB/s 2024-04-17T10:49:36.035 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.035 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.035 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115281 s, 91.0 MB/s 2024-04-17T10:49:36.047 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.047 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.047 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00945256 s, 111 MB/s 2024-04-17T10:49:36.060 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.060 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.060 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105586 s, 99.3 MB/s 2024-04-17T10:49:36.076 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.076 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.076 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134528 s, 77.9 MB/s 2024-04-17T10:49:36.087 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.087 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.087 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00872776 s, 120 MB/s 2024-04-17T10:49:36.101 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.101 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.101 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118077 s, 88.8 MB/s 2024-04-17T10:49:36.114 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.114 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.114 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112845 s, 92.9 MB/s 2024-04-17T10:49:36.128 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.128 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.128 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011249 s, 93.2 MB/s 2024-04-17T10:49:36.141 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.141 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.141 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108785 s, 96.4 MB/s 2024-04-17T10:49:36.152 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.152 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.152 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00908633 s, 115 MB/s 2024-04-17T10:49:36.166 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.166 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.166 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115272 s, 91.0 MB/s 2024-04-17T10:49:36.178 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.178 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.178 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00956328 s, 110 MB/s 2024-04-17T10:49:36.192 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.192 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.193 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125604 s, 83.5 MB/s 2024-04-17T10:49:36.207 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.207 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.207 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127342 s, 82.3 MB/s 2024-04-17T10:49:36.223 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.223 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.223 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134181 s, 78.1 MB/s 2024-04-17T10:49:36.235 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.235 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.236 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00991045 s, 106 MB/s 2024-04-17T10:49:36.251 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.251 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.251 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133403 s, 78.6 MB/s 2024-04-17T10:49:36.266 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.266 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.266 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120807 s, 86.8 MB/s 2024-04-17T10:49:36.280 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.280 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.280 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118143 s, 88.8 MB/s 2024-04-17T10:49:36.297 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.297 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.297 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140812 s, 74.5 MB/s 2024-04-17T10:49:36.311 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.311 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.311 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117062 s, 89.6 MB/s 2024-04-17T10:49:36.327 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.327 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.327 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126799 s, 82.7 MB/s 2024-04-17T10:49:36.340 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.340 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.340 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110025 s, 95.3 MB/s 2024-04-17T10:49:36.351 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.351 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.351 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00894701 s, 117 MB/s 2024-04-17T10:49:36.365 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.365 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.365 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117714 s, 89.1 MB/s 2024-04-17T10:49:36.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:36.366 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:49:36.366 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:36.367 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.7 2024-04-17T10:49:36.367 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:49:36.367 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:49:36.367 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:49:36.367 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:36.432 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.433 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.433 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118993 s, 88.1 MB/s 2024-04-17T10:49:36.444 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.445 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.445 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00937258 s, 112 MB/s 2024-04-17T10:49:36.460 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.460 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.460 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124433 s, 84.3 MB/s 2024-04-17T10:49:36.474 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.474 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.474 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110092 s, 95.2 MB/s 2024-04-17T10:49:36.487 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.487 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.487 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110316 s, 95.1 MB/s 2024-04-17T10:49:36.503 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.503 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.503 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127234 s, 82.4 MB/s 2024-04-17T10:49:36.516 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.516 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.516 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010626 s, 98.7 MB/s 2024-04-17T10:49:36.531 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.531 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.531 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125647 s, 83.5 MB/s 2024-04-17T10:49:36.546 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.546 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.546 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121314 s, 86.4 MB/s 2024-04-17T10:49:36.559 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.559 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.559 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106152 s, 98.8 MB/s 2024-04-17T10:49:36.576 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.576 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.576 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134844 s, 77.8 MB/s 2024-04-17T10:49:36.592 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.592 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.592 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013605 s, 77.1 MB/s 2024-04-17T10:49:36.605 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.606 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.606 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106171 s, 98.8 MB/s 2024-04-17T10:49:36.620 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.620 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.621 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01234 s, 85.0 MB/s 2024-04-17T10:49:36.635 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.635 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.635 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112422 s, 93.3 MB/s 2024-04-17T10:49:36.651 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.651 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.651 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131663 s, 79.6 MB/s 2024-04-17T10:49:36.668 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.668 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.668 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146394 s, 71.6 MB/s 2024-04-17T10:49:36.683 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.683 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.683 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119161 s, 88.0 MB/s 2024-04-17T10:49:36.698 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.699 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.699 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130302 s, 80.5 MB/s 2024-04-17T10:49:36.712 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.712 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.712 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108928 s, 96.3 MB/s 2024-04-17T10:49:36.726 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.726 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.726 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117005 s, 89.6 MB/s 2024-04-17T10:49:36.740 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.740 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.740 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010362 s, 101 MB/s 2024-04-17T10:49:36.756 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.756 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.756 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129609 s, 80.9 MB/s 2024-04-17T10:49:36.770 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.770 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.770 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110458 s, 94.9 MB/s 2024-04-17T10:49:36.785 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.785 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.785 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125411 s, 83.6 MB/s 2024-04-17T10:49:36.800 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.800 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.800 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123054 s, 85.2 MB/s 2024-04-17T10:49:36.818 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.818 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.818 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137535 s, 76.2 MB/s 2024-04-17T10:49:36.832 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.832 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.832 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107954 s, 97.1 MB/s 2024-04-17T10:49:36.846 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.846 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.846 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108371 s, 96.8 MB/s 2024-04-17T10:49:36.861 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.861 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.862 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121695 s, 86.2 MB/s 2024-04-17T10:49:36.876 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.876 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.876 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120207 s, 87.2 MB/s 2024-04-17T10:49:36.891 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.891 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.891 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122508 s, 85.6 MB/s 2024-04-17T10:49:36.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:36.892 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:49:36.892 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:36.892 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.8 2024-04-17T10:49:36.892 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:49:36.892 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:49:36.893 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:49:36.893 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:36.962 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.963 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.963 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126916 s, 82.6 MB/s 2024-04-17T10:49:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126296 s, 83.0 MB/s 2024-04-17T10:49:36.990 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:36.991 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:36.991 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00961671 s, 109 MB/s 2024-04-17T10:49:37.004 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.005 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.005 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113096 s, 92.7 MB/s 2024-04-17T10:49:37.021 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.021 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.021 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133711 s, 78.4 MB/s 2024-04-17T10:49:37.032 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.033 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.033 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00913373 s, 115 MB/s 2024-04-17T10:49:37.048 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.048 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.048 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120703 s, 86.9 MB/s 2024-04-17T10:49:37.061 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.062 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.062 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011416 s, 91.9 MB/s 2024-04-17T10:49:37.077 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.077 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.077 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123964 s, 84.6 MB/s 2024-04-17T10:49:37.090 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.090 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.090 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0094221 s, 111 MB/s 2024-04-17T10:49:37.105 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.105 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.105 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121395 s, 86.4 MB/s 2024-04-17T10:49:37.116 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.116 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.116 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00884863 s, 119 MB/s 2024-04-17T10:49:37.132 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.132 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.132 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130633 s, 80.3 MB/s 2024-04-17T10:49:37.147 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.147 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.147 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121206 s, 86.5 MB/s 2024-04-17T10:49:37.162 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.162 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.162 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124461 s, 84.2 MB/s 2024-04-17T10:49:37.176 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.176 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.176 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116846 s, 89.7 MB/s 2024-04-17T10:49:37.189 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.189 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.189 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105511 s, 99.4 MB/s 2024-04-17T10:49:37.202 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.202 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.202 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00911377 s, 115 MB/s 2024-04-17T10:49:37.217 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.217 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.217 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132386 s, 79.2 MB/s 2024-04-17T10:49:37.232 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.232 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.232 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121415 s, 86.4 MB/s 2024-04-17T10:49:37.247 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.247 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.247 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122459 s, 85.6 MB/s 2024-04-17T10:49:37.261 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.261 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.261 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115909 s, 90.5 MB/s 2024-04-17T10:49:37.278 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.278 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.278 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127147 s, 82.5 MB/s 2024-04-17T10:49:37.293 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.293 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.293 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118694 s, 88.3 MB/s 2024-04-17T10:49:37.309 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.309 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.309 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132821 s, 78.9 MB/s 2024-04-17T10:49:37.324 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.324 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.324 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125652 s, 83.5 MB/s 2024-04-17T10:49:37.339 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.339 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.339 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122917 s, 85.3 MB/s 2024-04-17T10:49:37.355 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.355 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.355 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126614 s, 82.8 MB/s 2024-04-17T10:49:37.370 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.370 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.370 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126647 s, 82.8 MB/s 2024-04-17T10:49:37.385 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.385 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.385 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122121 s, 85.9 MB/s 2024-04-17T10:49:37.398 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.399 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.399 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109988 s, 95.3 MB/s 2024-04-17T10:49:37.415 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.415 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.415 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014172 s, 74.0 MB/s 2024-04-17T10:49:37.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:37.417 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:49:37.417 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:37.417 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.9 2024-04-17T10:49:37.417 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:49:37.417 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:49:37.417 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:49:37.417 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:37.482 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.483 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.483 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00999103 s, 105 MB/s 2024-04-17T10:49:37.499 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.499 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.499 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131141 s, 80.0 MB/s 2024-04-17T10:49:37.515 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.516 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.516 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135762 s, 77.2 MB/s 2024-04-17T10:49:37.532 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.532 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.532 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137311 s, 76.4 MB/s 2024-04-17T10:49:37.548 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.548 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.548 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123226 s, 85.1 MB/s 2024-04-17T10:49:37.564 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.564 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.564 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133183 s, 78.7 MB/s 2024-04-17T10:49:37.579 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.579 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.579 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119187 s, 88.0 MB/s 2024-04-17T10:49:37.594 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.594 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.594 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126217 s, 83.1 MB/s 2024-04-17T10:49:37.609 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.609 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.610 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114889 s, 91.3 MB/s 2024-04-17T10:49:37.624 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.625 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.625 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127301 s, 82.4 MB/s 2024-04-17T10:49:37.638 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.638 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.638 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109454 s, 95.8 MB/s 2024-04-17T10:49:37.653 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.654 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.654 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123643 s, 84.8 MB/s 2024-04-17T10:49:37.670 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.671 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.671 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137497 s, 76.3 MB/s 2024-04-17T10:49:37.684 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.685 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.685 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116038 s, 90.4 MB/s 2024-04-17T10:49:37.700 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.700 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.700 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122455 s, 85.6 MB/s 2024-04-17T10:49:37.712 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.712 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.712 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00914662 s, 115 MB/s 2024-04-17T10:49:37.725 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.725 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.725 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00994178 s, 105 MB/s 2024-04-17T10:49:37.737 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.737 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.737 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.009427 s, 111 MB/s 2024-04-17T10:49:37.749 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.749 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.749 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00912976 s, 115 MB/s 2024-04-17T10:49:37.764 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.764 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.764 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125318 s, 83.7 MB/s 2024-04-17T10:49:37.780 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.781 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.781 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135254 s, 77.5 MB/s 2024-04-17T10:49:37.796 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.796 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.796 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123697 s, 84.8 MB/s 2024-04-17T10:49:37.811 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.811 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.811 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127449 s, 82.3 MB/s 2024-04-17T10:49:37.826 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.827 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.827 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128479 s, 81.6 MB/s 2024-04-17T10:49:37.841 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.841 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.841 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116084 s, 90.3 MB/s 2024-04-17T10:49:37.858 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.859 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.859 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150738 s, 69.6 MB/s 2024-04-17T10:49:37.872 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.873 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.873 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111357 s, 94.2 MB/s 2024-04-17T10:49:37.886 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.886 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.886 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010545 s, 99.4 MB/s 2024-04-17T10:49:37.898 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.898 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.898 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00950691 s, 110 MB/s 2024-04-17T10:49:37.913 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.913 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.913 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119778 s, 87.5 MB/s 2024-04-17T10:49:37.928 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.928 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.928 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012234 s, 85.7 MB/s 2024-04-17T10:49:37.941 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:37.941 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:37.941 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100086 s, 105 MB/s 2024-04-17T10:49:37.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:37.942 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:49:37.942 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:37.943 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.10 2024-04-17T10:49:37.943 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:49:37.943 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:49:37.943 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:49:37.943 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:38.007 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.008 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.008 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125134 s, 83.8 MB/s 2024-04-17T10:49:38.022 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.022 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.022 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115142 s, 91.1 MB/s 2024-04-17T10:49:38.036 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.036 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.036 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012129 s, 86.5 MB/s 2024-04-17T10:49:38.051 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.051 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.051 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011876 s, 88.3 MB/s 2024-04-17T10:49:38.065 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.066 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.066 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119849 s, 87.5 MB/s 2024-04-17T10:49:38.079 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.079 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.079 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112139 s, 93.5 MB/s 2024-04-17T10:49:38.096 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.096 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.096 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140086 s, 74.9 MB/s 2024-04-17T10:49:38.108 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.109 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.109 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00994153 s, 105 MB/s 2024-04-17T10:49:38.124 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.124 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.124 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130782 s, 80.2 MB/s 2024-04-17T10:49:38.135 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.135 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.135 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00881345 s, 119 MB/s 2024-04-17T10:49:38.150 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.150 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.150 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123003 s, 85.2 MB/s 2024-04-17T10:49:38.164 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.164 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.164 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114348 s, 91.7 MB/s 2024-04-17T10:49:38.178 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.178 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.178 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118391 s, 88.6 MB/s 2024-04-17T10:49:38.192 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.192 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.192 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116307 s, 90.2 MB/s 2024-04-17T10:49:38.205 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.205 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.205 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105697 s, 99.2 MB/s 2024-04-17T10:49:38.219 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.220 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.220 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121166 s, 86.5 MB/s 2024-04-17T10:49:38.234 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.234 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.234 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123164 s, 85.1 MB/s 2024-04-17T10:49:38.248 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.248 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.248 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110885 s, 94.6 MB/s 2024-04-17T10:49:38.259 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.259 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.259 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842378 s, 124 MB/s 2024-04-17T10:49:38.274 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.274 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.274 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120213 s, 87.2 MB/s 2024-04-17T10:49:38.287 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.287 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.287 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106757 s, 98.2 MB/s 2024-04-17T10:49:38.302 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.302 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.302 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120815 s, 86.8 MB/s 2024-04-17T10:49:38.315 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.315 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.315 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107821 s, 97.3 MB/s 2024-04-17T10:49:38.329 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.330 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.330 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113014 s, 92.8 MB/s 2024-04-17T10:49:38.341 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.341 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.341 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00893106 s, 117 MB/s 2024-04-17T10:49:38.355 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.355 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.356 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121794 s, 86.1 MB/s 2024-04-17T10:49:38.369 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.369 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.369 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109701 s, 95.6 MB/s 2024-04-17T10:49:38.383 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.383 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.383 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115005 s, 91.2 MB/s 2024-04-17T10:49:38.396 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.396 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.396 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107194 s, 97.8 MB/s 2024-04-17T10:49:38.411 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.411 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.412 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01236 s, 84.8 MB/s 2024-04-17T10:49:38.422 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.422 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.422 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848026 s, 124 MB/s 2024-04-17T10:49:38.433 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.434 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.434 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00863719 s, 121 MB/s 2024-04-17T10:49:38.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:38.435 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:49:38.435 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:38.435 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.11 2024-04-17T10:49:38.435 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:49:38.435 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:49:38.435 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:49:38.435 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:38.495 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.495 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.495 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00928666 s, 113 MB/s 2024-04-17T10:49:38.510 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.510 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.510 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119053 s, 88.1 MB/s 2024-04-17T10:49:38.524 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.524 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.524 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112985 s, 92.8 MB/s 2024-04-17T10:49:38.535 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.535 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.535 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852136 s, 123 MB/s 2024-04-17T10:49:38.546 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.546 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.546 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851099 s, 123 MB/s 2024-04-17T10:49:38.559 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.560 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.560 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108066 s, 97.0 MB/s 2024-04-17T10:49:38.573 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.573 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.573 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01109 s, 94.6 MB/s 2024-04-17T10:49:38.587 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.587 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.587 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115266 s, 91.0 MB/s 2024-04-17T10:49:38.601 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.602 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.602 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118762 s, 88.3 MB/s 2024-04-17T10:49:38.613 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.613 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.613 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00872463 s, 120 MB/s 2024-04-17T10:49:38.629 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.629 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.629 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104649 s, 100 MB/s 2024-04-17T10:49:38.631 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:49:38.633 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:49:38.642 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.642 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.643 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120976 s, 86.7 MB/s 2024-04-17T10:49:38.657 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.657 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.657 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118906 s, 88.2 MB/s 2024-04-17T10:49:38.670 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.670 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.670 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010652 s, 98.4 MB/s 2024-04-17T10:49:38.684 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.684 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.684 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011103 s, 94.4 MB/s 2024-04-17T10:49:38.697 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.698 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.698 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106153 s, 98.8 MB/s 2024-04-17T10:49:38.710 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.710 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.711 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103289 s, 102 MB/s 2024-04-17T10:49:38.724 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.724 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.724 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107669 s, 97.4 MB/s 2024-04-17T10:49:38.737 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.737 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.737 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102184 s, 103 MB/s 2024-04-17T10:49:38.749 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.749 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.749 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0098627 s, 106 MB/s 2024-04-17T10:49:38.761 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.761 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.761 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00986093 s, 106 MB/s 2024-04-17T10:49:38.775 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.775 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.775 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112413 s, 93.3 MB/s 2024-04-17T10:49:38.786 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.786 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.786 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00905614 s, 116 MB/s 2024-04-17T10:49:38.799 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.799 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.800 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106526 s, 98.4 MB/s 2024-04-17T10:49:38.812 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.812 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.812 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101118 s, 104 MB/s 2024-04-17T10:49:38.827 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.827 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.827 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123958 s, 84.6 MB/s 2024-04-17T10:49:38.848 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.848 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.848 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103462 s, 101 MB/s 2024-04-17T10:49:38.863 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.863 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.863 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121114 s, 86.6 MB/s 2024-04-17T10:49:38.873 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.873 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.873 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713619 s, 147 MB/s 2024-04-17T10:49:38.887 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.887 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.887 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011351 s, 92.4 MB/s 2024-04-17T10:49:38.900 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.900 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.900 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100286 s, 105 MB/s 2024-04-17T10:49:38.911 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.912 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.912 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871597 s, 120 MB/s 2024-04-17T10:49:38.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:38.912 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:49:38.912 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:38.912 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.12 2024-04-17T10:49:38.912 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:49:38.912 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:49:38.913 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:49:38.913 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:38.973 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.973 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.973 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811906 s, 129 MB/s 2024-04-17T10:49:38.985 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.985 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.986 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104175 s, 101 MB/s 2024-04-17T10:49:38.997 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:38.997 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:38.997 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00987679 s, 106 MB/s 2024-04-17T10:49:39.007 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.007 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.007 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072632 s, 144 MB/s 2024-04-17T10:49:39.023 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.023 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.023 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136609 s, 76.8 MB/s 2024-04-17T10:49:39.032 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.032 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.032 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720541 s, 146 MB/s 2024-04-17T10:49:39.044 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.044 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.044 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00996249 s, 105 MB/s 2024-04-17T10:49:39.057 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.057 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.058 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010772 s, 97.3 MB/s 2024-04-17T10:49:39.067 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.067 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.067 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073928 s, 142 MB/s 2024-04-17T10:49:39.080 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.080 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.080 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114507 s, 91.6 MB/s 2024-04-17T10:49:39.095 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.096 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.096 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129382 s, 81.0 MB/s 2024-04-17T10:49:39.109 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.109 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.109 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011312 s, 92.7 MB/s 2024-04-17T10:49:39.124 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.124 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.124 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127109 s, 82.5 MB/s 2024-04-17T10:49:39.137 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.137 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.137 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105421 s, 99.5 MB/s 2024-04-17T10:49:39.150 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.150 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.150 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00995519 s, 105 MB/s 2024-04-17T10:49:39.162 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103099 s, 102 MB/s 2024-04-17T10:49:39.177 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.177 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.177 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115381 s, 90.9 MB/s 2024-04-17T10:49:39.189 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.189 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.189 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0098435 s, 107 MB/s 2024-04-17T10:49:39.199 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.199 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.199 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721823 s, 145 MB/s 2024-04-17T10:49:39.209 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.209 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.209 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752332 s, 139 MB/s 2024-04-17T10:49:39.222 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.223 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.223 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105744 s, 99.2 MB/s 2024-04-17T10:49:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102582 s, 102 MB/s 2024-04-17T10:49:39.251 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.251 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.251 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128811 s, 81.4 MB/s 2024-04-17T10:49:39.261 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.261 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.261 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760322 s, 138 MB/s 2024-04-17T10:49:39.277 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.277 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.277 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131201 s, 79.9 MB/s 2024-04-17T10:49:39.290 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.290 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.290 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102641 s, 102 MB/s 2024-04-17T10:49:39.303 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.304 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.304 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111849 s, 93.7 MB/s 2024-04-17T10:49:39.316 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.317 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.317 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105339 s, 99.5 MB/s 2024-04-17T10:49:39.331 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.331 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.331 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114168 s, 91.8 MB/s 2024-04-17T10:49:39.345 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.345 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.345 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112873 s, 92.9 MB/s 2024-04-17T10:49:39.358 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.358 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.358 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109865 s, 95.4 MB/s 2024-04-17T10:49:39.370 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.370 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.370 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00948541 s, 111 MB/s 2024-04-17T10:49:39.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:39.371 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:49:39.371 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:39.372 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.13 2024-04-17T10:49:39.372 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:49:39.372 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:49:39.372 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:49:39.372 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:39.440 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.440 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.440 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129262 s, 81.1 MB/s 2024-04-17T10:49:39.453 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.454 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.454 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111893 s, 93.7 MB/s 2024-04-17T10:49:39.467 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.467 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.467 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106509 s, 98.4 MB/s 2024-04-17T10:49:39.482 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.482 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.482 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126539 s, 82.9 MB/s 2024-04-17T10:49:39.497 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.498 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.498 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126919 s, 82.6 MB/s 2024-04-17T10:49:39.507 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.507 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.507 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730797 s, 143 MB/s 2024-04-17T10:49:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104494 s, 100 MB/s 2024-04-17T10:49:39.533 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.534 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.534 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101582 s, 103 MB/s 2024-04-17T10:49:39.547 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.547 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.547 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103088 s, 102 MB/s 2024-04-17T10:49:39.559 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.559 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.559 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00988605 s, 106 MB/s 2024-04-17T10:49:39.571 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.571 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.571 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00926062 s, 113 MB/s 2024-04-17T10:49:39.585 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.585 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.585 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114406 s, 91.7 MB/s 2024-04-17T10:49:39.597 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.597 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.597 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00965937 s, 109 MB/s 2024-04-17T10:49:39.609 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.609 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.609 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842991 s, 124 MB/s 2024-04-17T10:49:39.621 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.621 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.621 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00984835 s, 106 MB/s 2024-04-17T10:49:39.636 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.636 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.636 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124304 s, 84.4 MB/s 2024-04-17T10:49:39.649 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.649 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.649 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101289 s, 104 MB/s 2024-04-17T10:49:39.659 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.659 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.659 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724694 s, 145 MB/s 2024-04-17T10:49:39.672 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.672 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.672 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105831 s, 99.1 MB/s 2024-04-17T10:49:39.683 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.683 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.683 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850439 s, 123 MB/s 2024-04-17T10:49:39.695 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.695 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.695 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00944835 s, 111 MB/s 2024-04-17T10:49:39.705 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.705 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.705 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772866 s, 136 MB/s 2024-04-17T10:49:39.717 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.717 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.717 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00922681 s, 114 MB/s 2024-04-17T10:49:39.728 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.728 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.728 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844732 s, 124 MB/s 2024-04-17T10:49:39.737 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.737 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.737 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714025 s, 147 MB/s 2024-04-17T10:49:39.749 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.749 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.749 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00981713 s, 107 MB/s 2024-04-17T10:49:39.761 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.761 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.761 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100954 s, 104 MB/s 2024-04-17T10:49:39.775 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.775 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.775 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116311 s, 90.2 MB/s 2024-04-17T10:49:39.792 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.792 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.792 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147815 s, 70.9 MB/s 2024-04-17T10:49:39.804 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.804 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.805 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010007 s, 105 MB/s 2024-04-17T10:49:39.819 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.819 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.819 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125493 s, 83.6 MB/s 2024-04-17T10:49:39.881 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.881 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.881 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119985 s, 87.4 MB/s 2024-04-17T10:49:39.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:39.883 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:49:39.883 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:39.883 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.14 2024-04-17T10:49:39.883 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:49:39.883 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:49:39.883 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:49:39.883 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:39.946 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.946 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.946 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104716 s, 100 MB/s 2024-04-17T10:49:39.959 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.959 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.960 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010961 s, 95.7 MB/s 2024-04-17T10:49:39.974 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.974 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.975 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126559 s, 82.9 MB/s 2024-04-17T10:49:39.987 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:39.987 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:39.987 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00940471 s, 111 MB/s 2024-04-17T10:49:40.002 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.002 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.002 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126355 s, 83.0 MB/s 2024-04-17T10:49:40.013 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.013 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.013 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793832 s, 132 MB/s 2024-04-17T10:49:40.027 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.027 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.027 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110792 s, 94.6 MB/s 2024-04-17T10:49:40.041 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.041 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.041 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109408 s, 95.8 MB/s 2024-04-17T10:49:40.054 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.054 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.054 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010787 s, 97.2 MB/s 2024-04-17T10:49:40.069 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.069 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.069 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012488 s, 84.0 MB/s 2024-04-17T10:49:40.082 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.082 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.082 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106053 s, 98.9 MB/s 2024-04-17T10:49:40.095 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.095 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.095 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102069 s, 103 MB/s 2024-04-17T10:49:40.111 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.111 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.111 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127237 s, 82.4 MB/s 2024-04-17T10:49:40.124 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.124 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.125 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111904 s, 93.7 MB/s 2024-04-17T10:49:40.140 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.141 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.141 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133189 s, 78.7 MB/s 2024-04-17T10:49:40.156 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.156 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.156 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125587 s, 83.5 MB/s 2024-04-17T10:49:40.169 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.169 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.170 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00943917 s, 111 MB/s 2024-04-17T10:49:40.180 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.181 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.181 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818766 s, 128 MB/s 2024-04-17T10:49:40.191 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.191 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.191 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741563 s, 141 MB/s 2024-04-17T10:49:40.207 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.207 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.207 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130808 s, 80.2 MB/s 2024-04-17T10:49:40.222 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.222 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.223 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131237 s, 79.9 MB/s 2024-04-17T10:49:40.238 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.238 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.238 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124447 s, 84.3 MB/s 2024-04-17T10:49:40.252 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.252 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.252 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111431 s, 94.1 MB/s 2024-04-17T10:49:40.268 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.268 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.268 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133051 s, 78.8 MB/s 2024-04-17T10:49:40.279 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.279 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.280 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00890814 s, 118 MB/s 2024-04-17T10:49:40.292 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.292 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.293 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102737 s, 102 MB/s 2024-04-17T10:49:40.306 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.306 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.306 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106873 s, 98.1 MB/s 2024-04-17T10:49:40.320 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.320 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.320 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010883 s, 96.3 MB/s 2024-04-17T10:49:40.334 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.334 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.335 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116553 s, 90.0 MB/s 2024-04-17T10:49:40.347 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.347 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.347 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00973253 s, 108 MB/s 2024-04-17T10:49:40.359 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.359 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.359 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00895459 s, 117 MB/s 2024-04-17T10:49:40.372 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.373 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.373 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109318 s, 95.9 MB/s 2024-04-17T10:49:40.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:40.374 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:49:40.374 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:40.374 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.15 2024-04-17T10:49:40.374 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:49:40.374 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:49:40.374 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:49:40.374 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:40.443 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.444 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.444 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104359 s, 100 MB/s 2024-04-17T10:49:40.457 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.457 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.457 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105411 s, 99.5 MB/s 2024-04-17T10:49:40.473 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.473 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.473 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124073 s, 84.5 MB/s 2024-04-17T10:49:40.484 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.484 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.484 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00904154 s, 116 MB/s 2024-04-17T10:49:40.498 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.498 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.498 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108802 s, 96.4 MB/s 2024-04-17T10:49:40.512 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.512 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.512 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115787 s, 90.6 MB/s 2024-04-17T10:49:40.526 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.526 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.526 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109334 s, 95.9 MB/s 2024-04-17T10:49:40.541 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.541 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.541 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121549 s, 86.3 MB/s 2024-04-17T10:49:40.553 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.553 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.553 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00993004 s, 106 MB/s 2024-04-17T10:49:40.570 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.570 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.570 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136731 s, 76.7 MB/s 2024-04-17T10:49:40.585 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.585 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.585 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125521 s, 83.5 MB/s 2024-04-17T10:49:40.602 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.602 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.602 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142097 s, 73.8 MB/s 2024-04-17T10:49:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830058 s, 126 MB/s 2024-04-17T10:49:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109396 s, 95.9 MB/s 2024-04-17T10:49:40.641 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.641 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.641 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130687 s, 80.2 MB/s 2024-04-17T10:49:40.658 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.658 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.658 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014482 s, 72.4 MB/s 2024-04-17T10:49:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101266 s, 104 MB/s 2024-04-17T10:49:40.683 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.683 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.683 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00980331 s, 107 MB/s 2024-04-17T10:49:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117915 s, 88.9 MB/s 2024-04-17T10:49:40.713 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.713 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.713 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130213 s, 80.5 MB/s 2024-04-17T10:49:40.727 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.727 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.727 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118635 s, 88.4 MB/s 2024-04-17T10:49:40.740 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.741 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.741 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114331 s, 91.7 MB/s 2024-04-17T10:49:40.754 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.755 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.755 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109743 s, 95.5 MB/s 2024-04-17T10:49:40.767 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.767 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.768 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109853 s, 95.5 MB/s 2024-04-17T10:49:40.779 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.780 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.780 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100609 s, 104 MB/s 2024-04-17T10:49:40.791 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.791 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.791 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00915957 s, 114 MB/s 2024-04-17T10:49:40.801 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.802 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.802 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830787 s, 126 MB/s 2024-04-17T10:49:40.815 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.815 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.815 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111421 s, 94.1 MB/s 2024-04-17T10:49:40.833 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.834 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.834 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148746 s, 70.5 MB/s 2024-04-17T10:49:40.858 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.858 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.858 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109043 s, 96.2 MB/s 2024-04-17T10:49:40.871 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.871 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.872 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01096 s, 95.7 MB/s 2024-04-17T10:49:40.886 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.886 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.886 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011963 s, 87.7 MB/s 2024-04-17T10:49:40.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:40.887 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:49:40.887 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:40.887 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.16 2024-04-17T10:49:40.887 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:49:40.887 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:49:40.888 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:49:40.888 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:40.955 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.955 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.955 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121747 s, 86.1 MB/s 2024-04-17T10:49:40.969 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.969 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.969 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113121 s, 92.7 MB/s 2024-04-17T10:49:40.981 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.981 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.981 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110333 s, 95.0 MB/s 2024-04-17T10:49:40.997 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:40.997 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:40.998 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013784 s, 76.1 MB/s 2024-04-17T10:49:41.009 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.009 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.009 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00868345 s, 121 MB/s 2024-04-17T10:49:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.024 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120393 s, 87.1 MB/s 2024-04-17T10:49:41.040 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.040 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.040 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143908 s, 72.9 MB/s 2024-04-17T10:49:41.057 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.057 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.057 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150861 s, 69.5 MB/s 2024-04-17T10:49:41.072 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.072 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.072 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118956 s, 88.1 MB/s 2024-04-17T10:49:41.086 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.086 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.086 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122536 s, 85.6 MB/s 2024-04-17T10:49:41.100 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.100 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.100 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117124 s, 89.5 MB/s 2024-04-17T10:49:41.114 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.114 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.114 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119258 s, 87.9 MB/s 2024-04-17T10:49:41.131 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.131 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.131 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134867 s, 77.7 MB/s 2024-04-17T10:49:41.147 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.147 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.148 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133024 s, 78.8 MB/s 2024-04-17T10:49:41.164 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.164 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.164 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139439 s, 75.2 MB/s 2024-04-17T10:49:41.177 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.177 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.177 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010017 s, 105 MB/s 2024-04-17T10:49:41.193 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.194 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.194 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134421 s, 78.0 MB/s 2024-04-17T10:49:41.209 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.209 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.209 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123813 s, 84.7 MB/s 2024-04-17T10:49:41.226 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.226 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.226 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138648 s, 75.6 MB/s 2024-04-17T10:49:41.242 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.242 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.242 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013236 s, 79.2 MB/s 2024-04-17T10:49:41.259 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.259 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.259 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133607 s, 78.5 MB/s 2024-04-17T10:49:41.274 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.274 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.275 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118887 s, 88.2 MB/s 2024-04-17T10:49:41.290 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.290 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.290 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118526 s, 88.5 MB/s 2024-04-17T10:49:41.304 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.304 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.304 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107987 s, 97.1 MB/s 2024-04-17T10:49:41.318 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.318 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.318 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103762 s, 101 MB/s 2024-04-17T10:49:41.333 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.333 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.333 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126652 s, 82.8 MB/s 2024-04-17T10:49:41.350 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.350 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.350 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136454 s, 76.8 MB/s 2024-04-17T10:49:41.363 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.363 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.363 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104219 s, 101 MB/s 2024-04-17T10:49:41.379 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.379 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.379 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129492 s, 81.0 MB/s 2024-04-17T10:49:41.395 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.395 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.395 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121413 s, 86.4 MB/s 2024-04-17T10:49:41.413 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.413 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.413 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137171 s, 76.4 MB/s 2024-04-17T10:49:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013798 s, 76.0 MB/s 2024-04-17T10:49:41.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:41.431 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:49:41.432 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:41.432 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.17 2024-04-17T10:49:41.432 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:49:41.432 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:49:41.432 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:49:41.432 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:41.508 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.509 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.509 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125009 s, 83.9 MB/s 2024-04-17T10:49:41.526 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.526 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.526 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137658 s, 76.2 MB/s 2024-04-17T10:49:41.542 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.543 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.543 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128205 s, 81.8 MB/s 2024-04-17T10:49:41.557 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.557 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.557 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111311 s, 94.2 MB/s 2024-04-17T10:49:41.574 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.574 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.574 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144534 s, 72.5 MB/s 2024-04-17T10:49:41.590 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.590 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.590 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012497 s, 83.9 MB/s 2024-04-17T10:49:41.606 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.606 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.607 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013174 s, 79.6 MB/s 2024-04-17T10:49:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117205 s, 89.5 MB/s 2024-04-17T10:49:41.640 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.640 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.640 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150372 s, 69.7 MB/s 2024-04-17T10:49:41.656 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.656 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.656 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012157 s, 86.3 MB/s 2024-04-17T10:49:41.673 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.673 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.673 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141781 s, 74.0 MB/s 2024-04-17T10:49:41.690 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.690 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.691 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134286 s, 78.1 MB/s 2024-04-17T10:49:41.707 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.708 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.708 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141584 s, 74.1 MB/s 2024-04-17T10:49:41.727 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.727 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.727 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145401 s, 72.1 MB/s 2024-04-17T10:49:41.745 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.745 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.745 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145585 s, 72.0 MB/s 2024-04-17T10:49:41.763 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.763 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.763 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146023 s, 71.8 MB/s 2024-04-17T10:49:41.783 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.783 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.783 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159556 s, 65.7 MB/s 2024-04-17T10:49:41.800 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.800 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.800 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124992 s, 83.9 MB/s 2024-04-17T10:49:41.816 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.817 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.817 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133586 s, 78.5 MB/s 2024-04-17T10:49:41.834 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.834 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.834 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133355 s, 78.6 MB/s 2024-04-17T10:49:41.908 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.908 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.908 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129027 s, 81.3 MB/s 2024-04-17T10:49:41.924 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.924 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.924 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011865 s, 88.4 MB/s 2024-04-17T10:49:41.941 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.941 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.941 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136878 s, 76.6 MB/s 2024-04-17T10:49:41.957 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.957 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.957 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125263 s, 83.7 MB/s 2024-04-17T10:49:41.972 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.973 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.973 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121412 s, 86.4 MB/s 2024-04-17T10:49:41.987 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:41.987 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:41.988 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012191 s, 86.0 MB/s 2024-04-17T10:49:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121963 s, 86.0 MB/s 2024-04-17T10:49:42.019 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.019 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.019 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131469 s, 79.8 MB/s 2024-04-17T10:49:42.037 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.037 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.037 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134345 s, 78.1 MB/s 2024-04-17T10:49:42.055 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.055 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.055 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139123 s, 75.4 MB/s 2024-04-17T10:49:42.071 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.071 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.071 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129904 s, 80.7 MB/s 2024-04-17T10:49:42.088 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.089 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.089 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141767 s, 74.0 MB/s 2024-04-17T10:49:42.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:42.090 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:49:42.090 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:42.090 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.18 2024-04-17T10:49:42.090 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:49:42.090 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:49:42.090 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:49:42.091 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:42.155 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.155 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.155 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00998363 s, 105 MB/s 2024-04-17T10:49:42.169 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.169 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.169 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112758 s, 93.0 MB/s 2024-04-17T10:49:42.184 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.184 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.184 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123964 s, 84.6 MB/s 2024-04-17T10:49:42.195 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.195 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.196 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00911958 s, 115 MB/s 2024-04-17T10:49:42.208 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.208 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.208 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00954714 s, 110 MB/s 2024-04-17T10:49:42.222 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.222 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.223 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114441 s, 91.6 MB/s 2024-04-17T10:49:42.237 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.237 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.238 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012421 s, 84.4 MB/s 2024-04-17T10:49:42.251 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.251 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.251 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109015 s, 96.2 MB/s 2024-04-17T10:49:42.266 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.267 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.267 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129322 s, 81.1 MB/s 2024-04-17T10:49:42.280 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.280 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.280 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111954 s, 93.7 MB/s 2024-04-17T10:49:42.296 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.296 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.296 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134509 s, 78.0 MB/s 2024-04-17T10:49:42.312 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.312 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.312 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120879 s, 86.7 MB/s 2024-04-17T10:49:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.328 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011619 s, 90.2 MB/s 2024-04-17T10:49:42.343 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.343 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.343 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113201 s, 92.6 MB/s 2024-04-17T10:49:42.358 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.358 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.358 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127353 s, 82.3 MB/s 2024-04-17T10:49:42.371 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.371 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.371 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00977497 s, 107 MB/s 2024-04-17T10:49:42.385 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.385 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.385 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116959 s, 89.7 MB/s 2024-04-17T10:49:42.398 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.398 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.398 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100623 s, 104 MB/s 2024-04-17T10:49:42.412 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.412 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.412 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011824 s, 88.7 MB/s 2024-04-17T10:49:42.427 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.427 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.427 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120647 s, 86.9 MB/s 2024-04-17T10:49:42.442 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.442 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.442 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121004 s, 86.7 MB/s 2024-04-17T10:49:42.457 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.457 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.457 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012145 s, 86.3 MB/s 2024-04-17T10:49:42.473 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.473 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.473 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131749 s, 79.6 MB/s 2024-04-17T10:49:42.484 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.485 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.485 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00909674 s, 115 MB/s 2024-04-17T10:49:42.501 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.502 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.502 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139413 s, 75.2 MB/s 2024-04-17T10:49:42.517 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.517 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.517 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118971 s, 88.1 MB/s 2024-04-17T10:49:42.532 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.532 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.532 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121804 s, 86.1 MB/s 2024-04-17T10:49:42.546 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.546 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.546 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116783 s, 89.8 MB/s 2024-04-17T10:49:42.558 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.558 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.558 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00907503 s, 116 MB/s 2024-04-17T10:49:42.573 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.573 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.573 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122249 s, 85.8 MB/s 2024-04-17T10:49:42.587 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.587 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.587 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112324 s, 93.4 MB/s 2024-04-17T10:49:42.600 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.600 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.600 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103125 s, 102 MB/s 2024-04-17T10:49:42.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:42.601 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:49:42.601 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:42.601 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.19 2024-04-17T10:49:42.602 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:49:42.602 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:49:42.602 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:49:42.602 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:42.671 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.671 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.671 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119218 s, 88.0 MB/s 2024-04-17T10:49:42.688 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.688 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.688 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146355 s, 71.6 MB/s 2024-04-17T10:49:42.703 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.703 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.703 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116266 s, 90.2 MB/s 2024-04-17T10:49:42.720 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.720 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.720 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133626 s, 78.5 MB/s 2024-04-17T10:49:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011669 s, 89.9 MB/s 2024-04-17T10:49:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132908 s, 78.9 MB/s 2024-04-17T10:49:42.766 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.766 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.766 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120796 s, 86.8 MB/s 2024-04-17T10:49:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136656 s, 76.7 MB/s 2024-04-17T10:49:42.798 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.798 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.798 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113089 s, 92.7 MB/s 2024-04-17T10:49:42.816 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.816 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.817 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138635 s, 75.6 MB/s 2024-04-17T10:49:42.836 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.837 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.837 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016197 s, 64.7 MB/s 2024-04-17T10:49:42.910 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.910 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.910 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013771 s, 76.1 MB/s 2024-04-17T10:49:42.929 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.929 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.929 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141581 s, 74.1 MB/s 2024-04-17T10:49:42.948 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.948 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.949 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149913 s, 69.9 MB/s 2024-04-17T10:49:42.968 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.968 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.968 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015309 s, 68.5 MB/s 2024-04-17T10:49:42.986 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:42.986 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:42.986 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014866 s, 70.5 MB/s 2024-04-17T10:49:43.002 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.002 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.002 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133776 s, 78.4 MB/s 2024-04-17T10:49:43.018 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.018 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.018 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131438 s, 79.8 MB/s 2024-04-17T10:49:43.034 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.034 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.034 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128443 s, 81.6 MB/s 2024-04-17T10:49:43.048 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.048 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.048 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111497 s, 94.0 MB/s 2024-04-17T10:49:43.064 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.064 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.065 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136289 s, 76.9 MB/s 2024-04-17T10:49:43.080 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.080 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.080 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129542 s, 80.9 MB/s 2024-04-17T10:49:43.097 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.097 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.097 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140288 s, 74.7 MB/s 2024-04-17T10:49:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132659 s, 79.0 MB/s 2024-04-17T10:49:43.130 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.130 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.130 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144543 s, 72.5 MB/s 2024-04-17T10:49:43.147 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.147 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.147 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130238 s, 80.5 MB/s 2024-04-17T10:49:43.163 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.163 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.163 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128363 s, 81.7 MB/s 2024-04-17T10:49:43.179 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.179 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.179 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134221 s, 78.1 MB/s 2024-04-17T10:49:43.193 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.193 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.193 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105784 s, 99.1 MB/s 2024-04-17T10:49:43.210 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.210 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.210 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139723 s, 75.0 MB/s 2024-04-17T10:49:43.226 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.226 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.226 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126776 s, 82.7 MB/s 2024-04-17T10:49:43.239 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.239 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.239 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103394 s, 101 MB/s 2024-04-17T10:49:43.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:43.240 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:49:43.240 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:43.240 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.20 2024-04-17T10:49:43.240 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:49:43.240 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:49:43.241 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:49:43.241 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:43.309 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.309 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.309 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113076 s, 92.7 MB/s 2024-04-17T10:49:43.325 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.325 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.325 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136813 s, 76.6 MB/s 2024-04-17T10:49:43.341 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.342 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.342 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131294 s, 79.9 MB/s 2024-04-17T10:49:43.356 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.356 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.356 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110314 s, 95.1 MB/s 2024-04-17T10:49:43.368 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.368 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.368 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.009311 s, 113 MB/s 2024-04-17T10:49:43.380 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.380 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.380 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0091855 s, 114 MB/s 2024-04-17T10:49:43.394 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.394 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.394 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108812 s, 96.4 MB/s 2024-04-17T10:49:43.405 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.405 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.405 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877902 s, 119 MB/s 2024-04-17T10:49:43.420 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.420 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.420 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115604 s, 90.7 MB/s 2024-04-17T10:49:43.435 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.435 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.435 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120506 s, 87.0 MB/s 2024-04-17T10:49:43.450 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.450 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.450 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118776 s, 88.3 MB/s 2024-04-17T10:49:43.463 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.464 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.464 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109437 s, 95.8 MB/s 2024-04-17T10:49:43.478 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.478 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.478 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117647 s, 89.1 MB/s 2024-04-17T10:49:43.491 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.491 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.492 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011036 s, 95.0 MB/s 2024-04-17T10:49:43.505 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.505 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.506 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011403 s, 92.0 MB/s 2024-04-17T10:49:43.520 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.520 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.520 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118894 s, 88.2 MB/s 2024-04-17T10:49:43.534 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.534 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.534 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010961 s, 95.7 MB/s 2024-04-17T10:49:43.550 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.550 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.550 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127472 s, 82.3 MB/s 2024-04-17T10:49:43.564 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.565 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.565 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122571 s, 85.5 MB/s 2024-04-17T10:49:43.579 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.579 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.579 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116682 s, 89.9 MB/s 2024-04-17T10:49:43.595 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.595 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.595 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132467 s, 79.2 MB/s 2024-04-17T10:49:43.609 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.609 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.609 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110148 s, 95.2 MB/s 2024-04-17T10:49:43.624 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.625 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.625 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125956 s, 83.2 MB/s 2024-04-17T10:49:43.641 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.641 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.641 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132572 s, 79.1 MB/s 2024-04-17T10:49:43.655 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.656 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.656 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120588 s, 87.0 MB/s 2024-04-17T10:49:43.669 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.670 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.670 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116145 s, 90.3 MB/s 2024-04-17T10:49:43.686 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.686 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.686 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124429 s, 84.3 MB/s 2024-04-17T10:49:43.699 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.699 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.700 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106961 s, 98.0 MB/s 2024-04-17T10:49:43.716 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.716 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.716 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131646 s, 79.7 MB/s 2024-04-17T10:49:43.730 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.730 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.730 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112434 s, 93.3 MB/s 2024-04-17T10:49:43.743 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.744 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.744 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110487 s, 94.9 MB/s 2024-04-17T10:49:43.757 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.758 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.758 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113935 s, 92.0 MB/s 2024-04-17T10:49:43.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:43.759 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:49:43.759 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:43.759 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.21 2024-04-17T10:49:43.759 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:49:43.759 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:49:43.759 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:49:43.759 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:43.823 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.823 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.823 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116246 s, 90.2 MB/s 2024-04-17T10:49:43.840 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.840 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.840 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141884 s, 73.9 MB/s 2024-04-17T10:49:43.874 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.874 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.874 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126329 s, 83.0 MB/s 2024-04-17T10:49:43.887 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.887 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.887 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105206 s, 99.7 MB/s 2024-04-17T10:49:43.901 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.901 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.902 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118275 s, 88.7 MB/s 2024-04-17T10:49:43.917 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.917 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.917 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130532 s, 80.3 MB/s 2024-04-17T10:49:43.932 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.932 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.932 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118188 s, 88.7 MB/s 2024-04-17T10:49:43.946 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.946 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.946 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119002 s, 88.1 MB/s 2024-04-17T10:49:43.958 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.958 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.958 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00920517 s, 114 MB/s 2024-04-17T10:49:43.974 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.974 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.975 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133382 s, 78.6 MB/s 2024-04-17T10:49:43.989 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:43.989 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:43.989 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012379 s, 84.7 MB/s 2024-04-17T10:49:44.004 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.004 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.004 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124489 s, 84.2 MB/s 2024-04-17T10:49:44.017 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.017 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.017 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102649 s, 102 MB/s 2024-04-17T10:49:44.032 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.032 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.032 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119439 s, 87.8 MB/s 2024-04-17T10:49:44.046 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.046 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.046 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109698 s, 95.6 MB/s 2024-04-17T10:49:44.062 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.062 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.063 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138505 s, 75.7 MB/s 2024-04-17T10:49:44.077 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.077 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.077 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120296 s, 87.2 MB/s 2024-04-17T10:49:44.091 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.091 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.091 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01022 s, 103 MB/s 2024-04-17T10:49:44.105 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.105 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.105 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115111 s, 91.1 MB/s 2024-04-17T10:49:44.120 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.121 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.121 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126614 s, 82.8 MB/s 2024-04-17T10:49:44.136 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.136 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.136 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123932 s, 84.6 MB/s 2024-04-17T10:49:44.149 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.149 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.149 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102972 s, 102 MB/s 2024-04-17T10:49:44.163 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.163 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.163 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116336 s, 90.1 MB/s 2024-04-17T10:49:44.178 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.178 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.179 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123521 s, 84.9 MB/s 2024-04-17T10:49:44.195 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.195 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.195 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138315 s, 75.8 MB/s 2024-04-17T10:49:44.209 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.210 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.210 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120357 s, 87.1 MB/s 2024-04-17T10:49:44.225 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.225 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.225 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126344 s, 83.0 MB/s 2024-04-17T10:49:44.240 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.240 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.240 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120736 s, 86.8 MB/s 2024-04-17T10:49:44.256 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.256 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.257 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133318 s, 78.7 MB/s 2024-04-17T10:49:44.269 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.269 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.269 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102532 s, 102 MB/s 2024-04-17T10:49:44.284 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.284 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.284 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123964 s, 84.6 MB/s 2024-04-17T10:49:44.300 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.300 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.300 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129917 s, 80.7 MB/s 2024-04-17T10:49:44.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:44.301 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:49:44.301 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:44.301 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.22 2024-04-17T10:49:44.302 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:49:44.302 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:49:44.302 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:49:44.302 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:44.369 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.370 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.370 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00966123 s, 109 MB/s 2024-04-17T10:49:44.382 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.382 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.382 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00932373 s, 112 MB/s 2024-04-17T10:49:44.397 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.397 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.397 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125261 s, 83.7 MB/s 2024-04-17T10:49:44.411 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.411 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.411 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011307 s, 92.7 MB/s 2024-04-17T10:49:44.424 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.424 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.424 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109757 s, 95.5 MB/s 2024-04-17T10:49:44.440 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.440 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.440 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012573 s, 83.4 MB/s 2024-04-17T10:49:44.451 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.451 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.451 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00913524 s, 115 MB/s 2024-04-17T10:49:44.467 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.467 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.467 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130033 s, 80.6 MB/s 2024-04-17T10:49:44.479 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.479 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.479 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00965998 s, 109 MB/s 2024-04-17T10:49:44.493 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.493 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.493 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113203 s, 92.6 MB/s 2024-04-17T10:49:44.505 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.505 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.505 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00892748 s, 117 MB/s 2024-04-17T10:49:44.519 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.519 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.519 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121087 s, 86.6 MB/s 2024-04-17T10:49:44.534 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.534 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.534 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122462 s, 85.6 MB/s 2024-04-17T10:49:44.545 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.545 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.545 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00912471 s, 115 MB/s 2024-04-17T10:49:44.560 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.560 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.560 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119669 s, 87.6 MB/s 2024-04-17T10:49:44.574 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.574 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.575 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122162 s, 85.8 MB/s 2024-04-17T10:49:44.588 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.588 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.588 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011194 s, 93.7 MB/s 2024-04-17T10:49:44.602 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.603 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.603 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119755 s, 87.6 MB/s 2024-04-17T10:49:44.618 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.619 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.619 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134371 s, 78.0 MB/s 2024-04-17T10:49:44.633 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.633 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.633 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121698 s, 86.2 MB/s 2024-04-17T10:49:44.650 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.650 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.650 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142689 s, 73.5 MB/s 2024-04-17T10:49:44.667 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151317 s, 69.3 MB/s 2024-04-17T10:49:44.682 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.682 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.682 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119008 s, 88.1 MB/s 2024-04-17T10:49:44.696 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.696 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.696 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116042 s, 90.4 MB/s 2024-04-17T10:49:44.707 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.707 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.707 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846783 s, 124 MB/s 2024-04-17T10:49:44.719 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.719 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.719 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00929174 s, 113 MB/s 2024-04-17T10:49:44.732 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.732 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.732 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010915 s, 96.1 MB/s 2024-04-17T10:49:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011263 s, 93.1 MB/s 2024-04-17T10:49:44.760 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.760 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.760 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118756 s, 88.3 MB/s 2024-04-17T10:49:44.776 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.776 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.776 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133307 s, 78.7 MB/s 2024-04-17T10:49:44.792 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.792 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.792 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012512 s, 83.8 MB/s 2024-04-17T10:49:44.805 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.805 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.805 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103079 s, 102 MB/s 2024-04-17T10:49:44.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:44.806 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:49:44.807 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:44.807 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.23 2024-04-17T10:49:44.807 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:49:44.807 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:49:44.807 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:49:44.807 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:44.883 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.883 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.883 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136513 s, 76.8 MB/s 2024-04-17T10:49:44.898 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.898 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.898 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124886 s, 84.0 MB/s 2024-04-17T10:49:44.914 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.914 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.914 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130263 s, 80.5 MB/s 2024-04-17T10:49:44.927 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.927 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.927 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109715 s, 95.6 MB/s 2024-04-17T10:49:44.944 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.944 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.944 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126299 s, 83.0 MB/s 2024-04-17T10:49:44.958 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.958 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.958 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102974 s, 102 MB/s 2024-04-17T10:49:44.970 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.971 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.971 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010374 s, 101 MB/s 2024-04-17T10:49:44.986 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.986 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.986 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012639 s, 83.0 MB/s 2024-04-17T10:49:44.998 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:44.998 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:44.998 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00973719 s, 108 MB/s 2024-04-17T10:49:45.013 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.013 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.014 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012745 s, 82.3 MB/s 2024-04-17T10:49:45.027 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.027 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.027 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110229 s, 95.1 MB/s 2024-04-17T10:49:45.042 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.042 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.042 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125706 s, 83.4 MB/s 2024-04-17T10:49:45.056 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.056 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.056 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011887 s, 88.2 MB/s 2024-04-17T10:49:45.072 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.072 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.072 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127898 s, 82.0 MB/s 2024-04-17T10:49:45.086 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.086 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.087 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123531 s, 84.9 MB/s 2024-04-17T10:49:45.098 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.098 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.098 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00907916 s, 115 MB/s 2024-04-17T10:49:45.113 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.113 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.113 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122106 s, 85.9 MB/s 2024-04-17T10:49:45.126 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.126 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.126 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110445 s, 94.9 MB/s 2024-04-17T10:49:45.138 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.138 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.138 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00909733 s, 115 MB/s 2024-04-17T10:49:45.151 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.151 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.151 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110819 s, 94.6 MB/s 2024-04-17T10:49:45.162 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.162 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.162 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842426 s, 124 MB/s 2024-04-17T10:49:45.173 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.173 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.173 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00900992 s, 116 MB/s 2024-04-17T10:49:45.187 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.187 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.187 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118034 s, 88.8 MB/s 2024-04-17T10:49:45.199 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.199 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.199 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00923196 s, 114 MB/s 2024-04-17T10:49:45.213 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.214 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.214 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120113 s, 87.3 MB/s 2024-04-17T10:49:45.229 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.229 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.229 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125019 s, 83.9 MB/s 2024-04-17T10:49:45.244 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.244 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.245 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131945 s, 79.5 MB/s 2024-04-17T10:49:45.259 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.259 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.259 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116873 s, 89.7 MB/s 2024-04-17T10:49:45.275 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.275 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.275 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013918 s, 75.3 MB/s 2024-04-17T10:49:45.289 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.289 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.289 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117723 s, 89.1 MB/s 2024-04-17T10:49:45.302 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.303 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.303 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112294 s, 93.4 MB/s 2024-04-17T10:49:45.317 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.318 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.318 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122051 s, 85.9 MB/s 2024-04-17T10:49:45.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:45.319 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:49:45.319 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:45.320 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.24 2024-04-17T10:49:45.320 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:49:45.320 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:49:45.320 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:49:45.320 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:45.390 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.390 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.390 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011554 s, 90.8 MB/s 2024-04-17T10:49:45.405 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.405 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.405 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012035 s, 87.1 MB/s 2024-04-17T10:49:45.419 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.419 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.419 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012111 s, 86.6 MB/s 2024-04-17T10:49:45.434 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.435 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.435 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117223 s, 89.5 MB/s 2024-04-17T10:49:45.449 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.449 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.449 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118699 s, 88.3 MB/s 2024-04-17T10:49:45.460 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.460 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.460 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819849 s, 128 MB/s 2024-04-17T10:49:45.472 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.472 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.472 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00956387 s, 110 MB/s 2024-04-17T10:49:45.485 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.485 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.485 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107086 s, 97.9 MB/s 2024-04-17T10:49:45.499 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.499 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.499 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115899 s, 90.5 MB/s 2024-04-17T10:49:45.511 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.512 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.512 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100824 s, 104 MB/s 2024-04-17T10:49:45.526 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.526 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.527 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120971 s, 86.7 MB/s 2024-04-17T10:49:45.540 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.540 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.540 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114906 s, 91.3 MB/s 2024-04-17T10:49:45.554 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.555 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.555 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120947 s, 86.7 MB/s 2024-04-17T10:49:45.569 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.569 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.569 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105919 s, 99.0 MB/s 2024-04-17T10:49:45.583 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.583 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.584 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130757 s, 80.2 MB/s 2024-04-17T10:49:45.601 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.602 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.602 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015734 s, 66.6 MB/s 2024-04-17T10:49:45.615 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.615 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.616 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115099 s, 91.1 MB/s 2024-04-17T10:49:45.629 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.630 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.630 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118003 s, 88.9 MB/s 2024-04-17T10:49:45.642 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.643 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.643 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107849 s, 97.2 MB/s 2024-04-17T10:49:45.656 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.656 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.656 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110731 s, 94.7 MB/s 2024-04-17T10:49:45.668 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.668 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.668 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101776 s, 103 MB/s 2024-04-17T10:49:45.683 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.683 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.683 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122427 s, 85.6 MB/s 2024-04-17T10:49:45.697 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.697 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.697 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120626 s, 86.9 MB/s 2024-04-17T10:49:45.711 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.712 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.712 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117609 s, 89.2 MB/s 2024-04-17T10:49:45.728 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.728 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.728 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134933 s, 77.7 MB/s 2024-04-17T10:49:45.738 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.739 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.739 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871501 s, 120 MB/s 2024-04-17T10:49:45.752 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.752 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.752 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114077 s, 91.9 MB/s 2024-04-17T10:49:45.767 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.767 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.767 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012357 s, 84.9 MB/s 2024-04-17T10:49:45.779 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.780 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.780 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109898 s, 95.4 MB/s 2024-04-17T10:49:45.793 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.793 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.793 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011112 s, 94.4 MB/s 2024-04-17T10:49:45.806 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.806 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.807 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114611 s, 91.5 MB/s 2024-04-17T10:49:45.820 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.820 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.820 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117926 s, 88.9 MB/s 2024-04-17T10:49:45.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:45.821 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:49:45.821 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:45.821 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.25 2024-04-17T10:49:45.821 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:49:45.822 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:49:45.822 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:49:45.822 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:45.888 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.888 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.888 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107868 s, 97.2 MB/s 2024-04-17T10:49:45.902 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.902 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.902 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116805 s, 89.8 MB/s 2024-04-17T10:49:45.914 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.915 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.915 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107268 s, 97.8 MB/s 2024-04-17T10:49:45.925 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.925 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.925 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779676 s, 134 MB/s 2024-04-17T10:49:45.937 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.938 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.938 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108706 s, 96.5 MB/s 2024-04-17T10:49:45.947 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.947 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.947 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756103 s, 139 MB/s 2024-04-17T10:49:45.960 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.960 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.960 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111143 s, 94.3 MB/s 2024-04-17T10:49:45.972 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.972 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.972 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101367 s, 103 MB/s 2024-04-17T10:49:45.985 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.986 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.986 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010924 s, 96.0 MB/s 2024-04-17T10:49:45.998 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:45.998 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:45.998 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102172 s, 103 MB/s 2024-04-17T10:49:46.009 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.009 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.009 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00919495 s, 114 MB/s 2024-04-17T10:49:46.022 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.022 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.022 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105578 s, 99.3 MB/s 2024-04-17T10:49:46.035 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.035 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.035 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112902 s, 92.9 MB/s 2024-04-17T10:49:46.047 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.047 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.047 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00976829 s, 107 MB/s 2024-04-17T10:49:46.061 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.061 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.061 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011599 s, 90.4 MB/s 2024-04-17T10:49:46.074 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.074 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.074 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109482 s, 95.8 MB/s 2024-04-17T10:49:46.087 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.087 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.087 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108147 s, 97.0 MB/s 2024-04-17T10:49:46.101 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.101 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.101 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111039 s, 94.4 MB/s 2024-04-17T10:49:46.114 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.114 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.114 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112207 s, 93.5 MB/s 2024-04-17T10:49:46.126 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.126 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.126 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00944473 s, 111 MB/s 2024-04-17T10:49:46.141 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.141 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.141 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122355 s, 85.7 MB/s 2024-04-17T10:49:46.158 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.158 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.158 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131574 s, 79.7 MB/s 2024-04-17T10:49:46.174 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.174 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.174 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131592 s, 79.7 MB/s 2024-04-17T10:49:46.191 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.191 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.191 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130891 s, 80.1 MB/s 2024-04-17T10:49:46.206 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.207 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.207 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012064 s, 86.9 MB/s 2024-04-17T10:49:46.221 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.222 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.222 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117578 s, 89.2 MB/s 2024-04-17T10:49:46.238 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.238 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.238 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136086 s, 77.1 MB/s 2024-04-17T10:49:46.250 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.250 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.250 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00949478 s, 110 MB/s 2024-04-17T10:49:46.266 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.266 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.266 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137949 s, 76.0 MB/s 2024-04-17T10:49:46.280 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.281 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.281 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119255 s, 87.9 MB/s 2024-04-17T10:49:46.297 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.297 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.297 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119556 s, 87.7 MB/s 2024-04-17T10:49:46.314 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.314 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.314 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013791 s, 76.0 MB/s 2024-04-17T10:49:46.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:46.315 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:49:46.315 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:46.316 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.26 2024-04-17T10:49:46.316 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:49:46.316 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:49:46.316 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:49:46.316 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:46.385 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.385 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.386 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101316 s, 103 MB/s 2024-04-17T10:49:46.397 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.397 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.397 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106624 s, 98.3 MB/s 2024-04-17T10:49:46.411 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.411 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.411 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114259 s, 91.8 MB/s 2024-04-17T10:49:46.426 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.426 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.426 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127656 s, 82.1 MB/s 2024-04-17T10:49:46.441 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.441 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.441 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128964 s, 81.3 MB/s 2024-04-17T10:49:46.456 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.456 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.456 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118227 s, 88.7 MB/s 2024-04-17T10:49:46.470 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.470 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.470 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105237 s, 99.6 MB/s 2024-04-17T10:49:46.486 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.487 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.487 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130761 s, 80.2 MB/s 2024-04-17T10:49:46.500 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.500 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.500 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114405 s, 91.7 MB/s 2024-04-17T10:49:46.514 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.514 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.514 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116689 s, 89.9 MB/s 2024-04-17T10:49:46.529 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.529 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.529 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125222 s, 83.7 MB/s 2024-04-17T10:49:46.543 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.543 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.543 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113002 s, 92.8 MB/s 2024-04-17T10:49:46.559 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.559 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.559 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140696 s, 74.5 MB/s 2024-04-17T10:49:46.575 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.575 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.575 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132071 s, 79.4 MB/s 2024-04-17T10:49:46.586 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.586 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.586 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00893974 s, 117 MB/s 2024-04-17T10:49:46.602 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.602 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.602 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132658 s, 79.0 MB/s 2024-04-17T10:49:46.617 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.617 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.617 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127008 s, 82.6 MB/s 2024-04-17T10:49:46.632 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.632 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.632 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124297 s, 84.4 MB/s 2024-04-17T10:49:46.648 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.648 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.648 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133459 s, 78.6 MB/s 2024-04-17T10:49:46.664 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.664 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.665 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132136 s, 79.4 MB/s 2024-04-17T10:49:46.679 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.680 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.680 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123597 s, 84.8 MB/s 2024-04-17T10:49:46.693 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.693 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.693 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110471 s, 94.9 MB/s 2024-04-17T10:49:46.707 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.707 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.707 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111078 s, 94.4 MB/s 2024-04-17T10:49:46.718 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.718 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.719 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00917872 s, 114 MB/s 2024-04-17T10:49:46.733 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.733 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.733 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118331 s, 88.6 MB/s 2024-04-17T10:49:46.746 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.747 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.747 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011271 s, 93.0 MB/s 2024-04-17T10:49:46.760 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.760 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.760 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110377 s, 95.0 MB/s 2024-04-17T10:49:46.774 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.774 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.774 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011978 s, 87.5 MB/s 2024-04-17T10:49:46.789 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.789 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.789 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123799 s, 84.7 MB/s 2024-04-17T10:49:46.805 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.805 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.805 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136298 s, 76.9 MB/s 2024-04-17T10:49:46.819 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.819 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.820 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011933 s, 87.9 MB/s 2024-04-17T10:49:46.834 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.834 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.835 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124561 s, 84.2 MB/s 2024-04-17T10:49:46.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:46.836 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:49:46.836 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:46.836 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.27 2024-04-17T10:49:46.836 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:49:46.836 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:49:46.836 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:49:46.836 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:46.901 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.901 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.901 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106229 s, 98.7 MB/s 2024-04-17T10:49:46.916 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.916 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.916 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117216 s, 89.5 MB/s 2024-04-17T10:49:46.931 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.931 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.931 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118122 s, 88.8 MB/s 2024-04-17T10:49:46.943 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.943 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.943 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010275 s, 102 MB/s 2024-04-17T10:49:46.957 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.958 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.958 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118706 s, 88.3 MB/s 2024-04-17T10:49:46.970 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.970 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.970 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102595 s, 102 MB/s 2024-04-17T10:49:46.982 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.982 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.982 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100703 s, 104 MB/s 2024-04-17T10:49:46.995 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:46.995 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:46.995 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01093 s, 95.9 MB/s 2024-04-17T10:49:47.009 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.010 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.010 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109788 s, 95.5 MB/s 2024-04-17T10:49:47.025 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.025 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.025 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122095 s, 85.9 MB/s 2024-04-17T10:49:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106013 s, 98.9 MB/s 2024-04-17T10:49:47.053 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.053 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.053 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114877 s, 91.3 MB/s 2024-04-17T10:49:47.067 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.067 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.067 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107761 s, 97.3 MB/s 2024-04-17T10:49:47.079 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.079 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.079 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00914141 s, 115 MB/s 2024-04-17T10:49:47.089 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.089 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.089 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779102 s, 135 MB/s 2024-04-17T10:49:47.101 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.101 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.101 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00971214 s, 108 MB/s 2024-04-17T10:49:47.112 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.112 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.112 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00856749 s, 122 MB/s 2024-04-17T10:49:47.126 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.126 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.126 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119401 s, 87.8 MB/s 2024-04-17T10:49:47.140 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.140 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.140 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115555 s, 90.7 MB/s 2024-04-17T10:49:47.154 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.154 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.154 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118519 s, 88.5 MB/s 2024-04-17T10:49:47.166 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.167 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.167 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103702 s, 101 MB/s 2024-04-17T10:49:47.180 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.180 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.180 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108994 s, 96.2 MB/s 2024-04-17T10:49:47.191 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.191 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.191 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00895153 s, 117 MB/s 2024-04-17T10:49:47.206 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.206 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.206 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113566 s, 92.3 MB/s 2024-04-17T10:49:47.216 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.216 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.216 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0088505 s, 118 MB/s 2024-04-17T10:49:47.226 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.226 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.227 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821501 s, 128 MB/s 2024-04-17T10:49:47.242 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.242 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.242 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129408 s, 81.0 MB/s 2024-04-17T10:49:47.254 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.254 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.254 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00997333 s, 105 MB/s 2024-04-17T10:49:47.267 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.267 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.267 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105486 s, 99.4 MB/s 2024-04-17T10:49:47.281 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.281 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.281 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122708 s, 85.5 MB/s 2024-04-17T10:49:47.296 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.423 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.424 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124193 s, 84.4 MB/s 2024-04-17T10:49:47.425 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.425 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.425 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125057 s, 83.8 MB/s 2024-04-17T10:49:47.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:47.425 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:49:47.426 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:47.426 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.28 2024-04-17T10:49:47.426 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:49:47.426 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:49:47.426 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:49:47.426 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:47.502 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.502 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.502 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138094 s, 75.9 MB/s 2024-04-17T10:49:47.516 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.516 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.516 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107952 s, 97.1 MB/s 2024-04-17T10:49:47.533 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.533 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.533 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136291 s, 76.9 MB/s 2024-04-17T10:49:47.549 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.549 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.549 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126574 s, 82.8 MB/s 2024-04-17T10:49:47.565 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.565 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.566 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137038 s, 76.5 MB/s 2024-04-17T10:49:47.582 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.582 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.582 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135109 s, 77.6 MB/s 2024-04-17T10:49:47.598 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.599 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.599 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136681 s, 76.7 MB/s 2024-04-17T10:49:47.615 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.615 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.615 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131476 s, 79.8 MB/s 2024-04-17T10:49:47.631 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.631 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.631 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132204 s, 79.3 MB/s 2024-04-17T10:49:47.648 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.648 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.648 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135284 s, 77.5 MB/s 2024-04-17T10:49:47.665 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.666 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.666 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147135 s, 71.3 MB/s 2024-04-17T10:49:47.680 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.680 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.681 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118426 s, 88.5 MB/s 2024-04-17T10:49:47.694 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.694 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.695 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110734 s, 94.7 MB/s 2024-04-17T10:49:47.711 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.711 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.711 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134545 s, 77.9 MB/s 2024-04-17T10:49:47.725 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.725 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.726 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111662 s, 93.9 MB/s 2024-04-17T10:49:47.742 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.742 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.742 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137101 s, 76.5 MB/s 2024-04-17T10:49:47.757 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.757 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.757 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107799 s, 97.3 MB/s 2024-04-17T10:49:47.774 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.774 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.774 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136855 s, 76.6 MB/s 2024-04-17T10:49:47.788 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.788 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.788 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107435 s, 97.6 MB/s 2024-04-17T10:49:47.802 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.802 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.802 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107934 s, 97.2 MB/s 2024-04-17T10:49:47.821 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.821 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.821 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135345 s, 77.5 MB/s 2024-04-17T10:49:47.835 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.835 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.835 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132293 s, 79.3 MB/s 2024-04-17T10:49:47.852 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.852 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.852 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138852 s, 75.5 MB/s 2024-04-17T10:49:47.894 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.895 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.895 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013832 s, 75.8 MB/s 2024-04-17T10:49:47.910 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.910 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.911 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134114 s, 78.2 MB/s 2024-04-17T10:49:47.927 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.927 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.927 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013423 s, 78.1 MB/s 2024-04-17T10:49:47.943 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.943 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.943 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129314 s, 81.1 MB/s 2024-04-17T10:49:47.960 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.960 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.960 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136824 s, 76.6 MB/s 2024-04-17T10:49:47.976 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.976 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.976 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135076 s, 77.6 MB/s 2024-04-17T10:49:47.993 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:47.993 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:47.993 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142435 s, 73.6 MB/s 2024-04-17T10:49:48.009 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.009 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.009 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128223 s, 81.8 MB/s 2024-04-17T10:49:48.025 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.025 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.025 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128733 s, 81.5 MB/s 2024-04-17T10:49:48.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:48.026 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:49:48.026 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:48.026 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.29 2024-04-17T10:49:48.026 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:49:48.026 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:49:48.026 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:49:48.027 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:48.095 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.096 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.096 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122926 s, 85.3 MB/s 2024-04-17T10:49:48.113 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.113 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.113 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135784 s, 77.2 MB/s 2024-04-17T10:49:48.129 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.129 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.129 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122642 s, 85.5 MB/s 2024-04-17T10:49:48.146 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.147 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.147 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135316 s, 77.5 MB/s 2024-04-17T10:49:48.163 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.163 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.163 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013423 s, 78.1 MB/s 2024-04-17T10:49:48.177 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.177 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.177 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108926 s, 96.3 MB/s 2024-04-17T10:49:48.189 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.189 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.189 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00901601 s, 116 MB/s 2024-04-17T10:49:48.202 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.202 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.202 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101425 s, 103 MB/s 2024-04-17T10:49:48.217 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.217 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.217 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119277 s, 87.9 MB/s 2024-04-17T10:49:48.232 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.232 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.233 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012984 s, 80.8 MB/s 2024-04-17T10:49:48.247 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.247 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.247 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120297 s, 87.2 MB/s 2024-04-17T10:49:48.260 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.261 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.261 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105474 s, 99.4 MB/s 2024-04-17T10:49:48.276 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.276 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.276 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125633 s, 83.5 MB/s 2024-04-17T10:49:48.290 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.290 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.290 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112542 s, 93.2 MB/s 2024-04-17T10:49:48.302 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.302 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.302 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00969547 s, 108 MB/s 2024-04-17T10:49:48.316 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.316 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.316 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111332 s, 94.2 MB/s 2024-04-17T10:49:48.331 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.331 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.331 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124498 s, 84.2 MB/s 2024-04-17T10:49:48.347 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.347 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.347 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123203 s, 85.1 MB/s 2024-04-17T10:49:48.361 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.361 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.361 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114968 s, 91.2 MB/s 2024-04-17T10:49:48.377 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.377 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.378 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137491 s, 76.3 MB/s 2024-04-17T10:49:48.392 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.392 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.392 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118239 s, 88.7 MB/s 2024-04-17T10:49:48.406 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.406 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.406 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108756 s, 96.4 MB/s 2024-04-17T10:49:48.421 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.421 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.421 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128547 s, 81.6 MB/s 2024-04-17T10:49:48.437 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.437 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.437 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127002 s, 82.6 MB/s 2024-04-17T10:49:48.453 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.453 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.453 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131367 s, 79.8 MB/s 2024-04-17T10:49:48.469 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.469 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.469 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123991 s, 84.6 MB/s 2024-04-17T10:49:48.484 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.485 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.485 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123289 s, 85.1 MB/s 2024-04-17T10:49:48.500 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.500 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.500 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130552 s, 80.3 MB/s 2024-04-17T10:49:48.517 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.517 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.517 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142503 s, 73.6 MB/s 2024-04-17T10:49:48.533 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.533 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.533 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129968 s, 80.7 MB/s 2024-04-17T10:49:48.547 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.547 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.547 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102786 s, 102 MB/s 2024-04-17T10:49:48.563 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.564 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.564 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012658 s, 82.8 MB/s 2024-04-17T10:49:48.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:48.565 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:49:48.565 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:48.565 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.30 2024-04-17T10:49:48.565 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:49:48.565 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:49:48.565 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:49:48.565 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:48.630 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.630 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.630 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00964967 s, 109 MB/s 2024-04-17T10:49:48.647 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.647 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.647 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130695 s, 80.2 MB/s 2024-04-17T10:49:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00988535 s, 106 MB/s 2024-04-17T10:49:48.674 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.674 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.674 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115953 s, 90.4 MB/s 2024-04-17T10:49:48.687 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.687 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.687 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101074 s, 104 MB/s 2024-04-17T10:49:48.699 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.700 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.700 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00952959 s, 110 MB/s 2024-04-17T10:49:48.712 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.713 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.713 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106 s, 98.9 MB/s 2024-04-17T10:49:48.724 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.724 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.724 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00883608 s, 119 MB/s 2024-04-17T10:49:48.738 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.738 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.738 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113514 s, 92.4 MB/s 2024-04-17T10:49:48.751 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.751 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.751 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01075 s, 97.5 MB/s 2024-04-17T10:49:48.766 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.766 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.766 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121666 s, 86.2 MB/s 2024-04-17T10:49:48.780 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.781 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.781 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121433 s, 86.4 MB/s 2024-04-17T10:49:48.794 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.794 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.794 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107602 s, 97.4 MB/s 2024-04-17T10:49:48.808 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.808 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.809 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111625 s, 93.9 MB/s 2024-04-17T10:49:48.823 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.823 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.823 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121684 s, 86.2 MB/s 2024-04-17T10:49:48.836 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.836 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.836 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010254 s, 102 MB/s 2024-04-17T10:49:48.849 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.850 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.850 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108312 s, 96.8 MB/s 2024-04-17T10:49:48.861 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.861 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.861 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00916195 s, 114 MB/s 2024-04-17T10:49:48.891 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.891 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.891 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106493 s, 98.5 MB/s 2024-04-17T10:49:48.905 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.905 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.905 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123016 s, 85.2 MB/s 2024-04-17T10:49:48.918 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.918 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.919 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010735 s, 97.7 MB/s 2024-04-17T10:49:48.933 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.933 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.933 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118366 s, 88.6 MB/s 2024-04-17T10:49:48.948 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.948 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.948 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118084 s, 88.8 MB/s 2024-04-17T10:49:48.962 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.962 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.962 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112068 s, 93.6 MB/s 2024-04-17T10:49:48.978 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.978 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.978 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137086 s, 76.5 MB/s 2024-04-17T10:49:48.992 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:48.992 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:48.992 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114155 s, 91.9 MB/s 2024-04-17T10:49:49.007 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.007 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.007 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117865 s, 89.0 MB/s 2024-04-17T10:49:49.018 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.018 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.018 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840858 s, 125 MB/s 2024-04-17T10:49:49.031 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.031 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.031 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103498 s, 101 MB/s 2024-04-17T10:49:49.045 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.045 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.046 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105805 s, 99.1 MB/s 2024-04-17T10:49:49.057 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.057 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.057 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00976385 s, 107 MB/s 2024-04-17T10:49:49.071 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.071 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.071 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113753 s, 92.2 MB/s 2024-04-17T10:49:49.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:49.072 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:49:49.072 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:49:49.072 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.31 2024-04-17T10:49:49.073 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:49:49.073 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:49:49.073 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:49:49.073 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:49:49.138 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.138 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.138 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113892 s, 92.1 MB/s 2024-04-17T10:49:49.149 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.149 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.149 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00857462 s, 122 MB/s 2024-04-17T10:49:49.159 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.159 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.159 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079813 s, 131 MB/s 2024-04-17T10:49:49.170 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.170 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.170 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816247 s, 128 MB/s 2024-04-17T10:49:49.184 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.184 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.184 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115331 s, 90.9 MB/s 2024-04-17T10:49:49.197 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.197 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.197 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109151 s, 96.1 MB/s 2024-04-17T10:49:49.211 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.211 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.211 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117456 s, 89.3 MB/s 2024-04-17T10:49:49.222 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.222 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.222 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076901 s, 136 MB/s 2024-04-17T10:49:49.236 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.236 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.236 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118068 s, 88.8 MB/s 2024-04-17T10:49:49.251 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.251 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.251 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117656 s, 89.1 MB/s 2024-04-17T10:49:49.267 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.267 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.267 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117376 s, 89.3 MB/s 2024-04-17T10:49:49.284 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.284 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.284 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128432 s, 81.6 MB/s 2024-04-17T10:49:49.299 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.299 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.299 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111577 s, 94.0 MB/s 2024-04-17T10:49:49.314 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.314 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.314 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109688 s, 95.6 MB/s 2024-04-17T10:49:49.330 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.330 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.330 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012497 s, 83.9 MB/s 2024-04-17T10:49:49.346 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.347 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.347 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121399 s, 86.4 MB/s 2024-04-17T10:49:49.362 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.362 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.362 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109308 s, 95.9 MB/s 2024-04-17T10:49:49.380 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.380 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.380 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141654 s, 74.0 MB/s 2024-04-17T10:49:49.394 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.395 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.395 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00995489 s, 105 MB/s 2024-04-17T10:49:49.412 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.412 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.412 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130405 s, 80.4 MB/s 2024-04-17T10:49:49.427 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.427 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.427 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110121 s, 95.2 MB/s 2024-04-17T10:49:49.440 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.440 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.440 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00900759 s, 116 MB/s 2024-04-17T10:49:49.451 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.451 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.451 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846646 s, 124 MB/s 2024-04-17T10:49:49.466 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.466 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.466 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123857 s, 84.7 MB/s 2024-04-17T10:49:49.477 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.477 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.477 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838855 s, 125 MB/s 2024-04-17T10:49:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119954 s, 87.4 MB/s 2024-04-17T10:49:49.506 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.506 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.506 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112049 s, 93.6 MB/s 2024-04-17T10:49:49.522 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.522 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.522 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132225 s, 79.3 MB/s 2024-04-17T10:49:49.535 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.536 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.536 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110568 s, 94.8 MB/s 2024-04-17T10:49:49.549 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.550 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.550 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115586 s, 90.7 MB/s 2024-04-17T10:49:49.562 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.563 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.563 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105626 s, 99.3 MB/s 2024-04-17T10:49:49.577 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:49:49.578 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:49:49.578 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012249 s, 85.6 MB/s 2024-04-17T10:49:49.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:49.579 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub start /scrub_control_test_path recursive 2024-04-17T10:49:49.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.753+0000 7fa9d8708640 1 Processor -- start 2024-04-17T10:49:49.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.753+0000 7fa9d8708640 1 -- start start 2024-04-17T10:49:49.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.753+0000 7fa9d8708640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9d015e020 0x7fa9d00cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.753+0000 7fa9d8708640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa9d00d4510 0x7fa9d00d17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.753+0000 7fa9d8708640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d0074800 con 0x7fa9d00d1fd0 2024-04-17T10:49:49.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.753+0000 7fa9d8708640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9d00cf550 con 0x7fa9d00d4510 2024-04-17T10:49:49.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.753+0000 7fa9d8708640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d00cff20 con 0x7fa9d015e020 2024-04-17T10:49:49.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.753+0000 7fa9d5ca3640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa9d00d4510 0x7fa9d00d17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.753+0000 7fa9d5ca3640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa9d00d4510 0x7fa9d00d17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47804/0 (socket says 172.21.15.83:47804) 2024-04-17T10:49:49.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.753+0000 7fa9d5ca3640 1 -- 172.21.15.83:0/4038812723 learned_addr learned my addr 172.21.15.83:0/4038812723 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:49.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.753+0000 7fa9d5ca3640 1 -- 172.21.15.83:0/4038812723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9d015e020 msgr2=0x7fa9d00cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:49.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.753+0000 7fa9d64a4640 1 --2- 172.21.15.83:0/4038812723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9d015e020 0x7fa9d00cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.753+0000 7fa9d5ca3640 1 --2- 172.21.15.83:0/4038812723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9d015e020 0x7fa9d00cf9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.753+0000 7fa9d54a2640 1 -- 172.21.15.83:0/4038812723 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2222217402 0 0) 0x7fa9d0074800 con 0x7fa9d00d1fd0 2024-04-17T10:49:49.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.753+0000 7fa9d5ca3640 1 -- 172.21.15.83:0/4038812723 >> v1:172.21.15.83:6789/0 conn(0x7fa9d00d1fd0 legacy=0x7fa9d006b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:49.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.753+0000 7fa9d5ca3640 1 -- 172.21.15.83:0/4038812723 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d00ce980 con 0x7fa9d00d4510 2024-04-17T10:49:49.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.757+0000 7fa9d5ca3640 1 --2- 172.21.15.83:0/4038812723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9d00d4510 0x7fa9d00d17d0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0009870 tx=0x7fa9c001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1397f07c69fb8f3d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:49.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.757+0000 7fa9d54a2640 1 -- 172.21.15.83:0/4038812723 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c00a95f0 con 0x7fa9d00d4510 2024-04-17T10:49:49.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.757+0000 7fa9d54a2640 1 -- 172.21.15.83:0/4038812723 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c00a9790 con 0x7fa9d00d4510 2024-04-17T10:49:49.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.757+0000 7fa9d54a2640 1 -- 172.21.15.83:0/4038812723 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c00a9b60 con 0x7fa9d00d4510 2024-04-17T10:49:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.757+0000 7fa9d8708640 1 -- 172.21.15.83:0/4038812723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9d00d4510 msgr2=0x7fa9d00d17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.757+0000 7fa9d8708640 1 --2- 172.21.15.83:0/4038812723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9d00d4510 0x7fa9d00d17d0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0009870 tx=0x7fa9c001c530 comp rx=0 tx=0).stop 2024-04-17T10:49:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.757+0000 7fa9d8708640 1 -- 172.21.15.83:0/4038812723 shutdown_connections 2024-04-17T10:49:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.757+0000 7fa9d8708640 1 --2- 172.21.15.83:0/4038812723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9d00d4510 0x7fa9d00d17d0 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.757+0000 7fa9d8708640 1 --2- 172.21.15.83:0/4038812723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9d015e020 0x7fa9d00cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.757+0000 7fa9d8708640 1 -- 172.21.15.83:0/4038812723 >> 172.21.15.83:0/4038812723 conn(0x7fa9d00d39f0 msgr2=0x7fa9d00d3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.757+0000 7fa9d8708640 1 -- 172.21.15.83:0/4038812723 shutdown_connections 2024-04-17T10:49:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.757+0000 7fa9d8708640 1 -- 172.21.15.83:0/4038812723 wait complete. 2024-04-17T10:49:49.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.761+0000 7fa9d8708640 1 Processor -- start 2024-04-17T10:49:49.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.761+0000 7fa9d8708640 1 -- start start 2024-04-17T10:49:49.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.761+0000 7fa9d8708640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9d00d1fd0 0x7fa9d0155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.761+0000 7fa9d8708640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9d00d4510 0x7fa9d00b6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.761+0000 7fa9d8708640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9d015e020 0x7fa9d00c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.761+0000 7fa9d8708640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d006bc10 con 0x7fa9d00d1fd0 2024-04-17T10:49:49.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.761+0000 7fa9d8708640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d00d1d10 con 0x7fa9d015e020 2024-04-17T10:49:49.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.761+0000 7fa9d8708640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d00d27f0 con 0x7fa9d00d4510 2024-04-17T10:49:49.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.761+0000 7fa9d64a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9d00d1fd0 0x7fa9d0155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.761+0000 7fa9d64a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9d00d1fd0 0x7fa9d0155350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47814/0 (socket says 172.21.15.83:47814) 2024-04-17T10:49:49.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.761+0000 7fa9d64a4640 1 -- 172.21.15.83:0/3250720423 learned_addr learned my addr 172.21.15.83:0/3250720423 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:49.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.761+0000 7fa9d5ca3640 1 --2- 172.21.15.83:0/3250720423 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9d00d4510 0x7fa9d00b6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.761+0000 7fa9d6ca5640 1 --2- 172.21.15.83:0/3250720423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9d015e020 0x7fa9d00c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.761+0000 7fa9d64a4640 1 -- 172.21.15.83:0/3250720423 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9d00d4510 msgr2=0x7fa9d00b6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:49.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.761+0000 7fa9d64a4640 1 --2- 172.21.15.83:0/3250720423 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9d00d4510 0x7fa9d00b6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.761+0000 7fa9d64a4640 1 -- 172.21.15.83:0/3250720423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9d015e020 msgr2=0x7fa9d00c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:49.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.761+0000 7fa9d64a4640 1 --2- 172.21.15.83:0/3250720423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9d015e020 0x7fa9d00c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.761+0000 7fa9d64a4640 1 -- 172.21.15.83:0/3250720423 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d00b9140 con 0x7fa9d00d1fd0 2024-04-17T10:49:49.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.761+0000 7fa9d64a4640 1 --2- 172.21.15.83:0/3250720423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9d00d1fd0 0x7fa9d0155350 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00c820 tx=0x7fa9cc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:49.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.761+0000 7fa9c6ffd640 1 -- 172.21.15.83:0/3250720423 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc015070 con 0x7fa9d00d1fd0 2024-04-17T10:49:49.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.761+0000 7fa9c6ffd640 1 -- 172.21.15.83:0/3250720423 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9cc004240 con 0x7fa9d00d1fd0 2024-04-17T10:49:49.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.761+0000 7fa9c6ffd640 1 -- 172.21.15.83:0/3250720423 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc00ce10 con 0x7fa9d00d1fd0 2024-04-17T10:49:49.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.761+0000 7fa9d8708640 1 -- 172.21.15.83:0/3250720423 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d00b9420 con 0x7fa9d00d1fd0 2024-04-17T10:49:49.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.761+0000 7fa9d8708640 1 -- 172.21.15.83:0/3250720423 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d00c7c20 con 0x7fa9d00d1fd0 2024-04-17T10:49:49.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9c6ffd640 1 -- 172.21.15.83:0/3250720423 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa9cc007830 con 0x7fa9d00d1fd0 2024-04-17T10:49:49.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9c6ffd640 1 --2- 172.21.15.83:0/3250720423 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa9b4066f60 0x7fa9b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9c6ffd640 1 -- 172.21.15.83:0/3250720423 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa9cc084e90 con 0x7fa9d00d1fd0 2024-04-17T10:49:49.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9d5ca3640 1 --2- 172.21.15.83:0/3250720423 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa9b4066f60 0x7fa9b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9d5ca3640 1 --2- 172.21.15.83:0/3250720423 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa9b4066f60 0x7fa9b4069420 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fa9c00ae000 tx=0x7fa9c00a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:49.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9db875000 1 Processor -- start 2024-04-17T10:49:49.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9db875000 1 -- start start 2024-04-17T10:49:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9db875000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559c520233f0 0x559c520237d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9db875000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c52023d10 0x559c51fd9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9db875000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c51fda520 0x559c51fda900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9db875000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c51e27d50 con 0x559c52023d10 2024-04-17T10:49:49.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9db875000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c51dafe40 con 0x559c51fda520 2024-04-17T10:49:49.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9db875000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c51e23df0 con 0x559c520233f0 2024-04-17T10:49:49.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9d5ca3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c52023d10 0x559c51fd9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9d5ca3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c52023d10 0x559c51fd9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47826/0 (socket says 172.21.15.83:47826) 2024-04-17T10:49:49.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9d5ca3640 1 -- 172.21.15.83:0/1251024100 learned_addr learned my addr 172.21.15.83:0/1251024100 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:49.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9d6ca5640 1 --2- 172.21.15.83:0/1251024100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c51fda520 0x559c51fda900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9d64a4640 1 --2- 172.21.15.83:0/1251024100 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559c520233f0 0x559c520237d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9d5ca3640 1 -- 172.21.15.83:0/1251024100 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559c520233f0 msgr2=0x559c520237d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:49.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9d5ca3640 1 --2- 172.21.15.83:0/1251024100 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559c520233f0 0x559c520237d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9d5ca3640 1 -- 172.21.15.83:0/1251024100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c51fda520 msgr2=0x559c51fda900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:49.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9d5ca3640 1 --2- 172.21.15.83:0/1251024100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c51fda520 0x559c51fda900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9d5ca3640 1 -- 172.21.15.83:0/1251024100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c51dc4d90 con 0x559c52023d10 2024-04-17T10:49:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9d5ca3640 1 --2- 172.21.15.83:0/1251024100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c52023d10 0x559c51fd9fe0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0000c10 tx=0x7fa9c0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d9a67fe91ea97f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9b3fff640 1 -- 172.21.15.83:0/1251024100 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c00c8480 con 0x559c52023d10 2024-04-17T10:49:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9b3fff640 1 -- 172.21.15.83:0/1251024100 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c00c8620 con 0x559c52023d10 2024-04-17T10:49:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9b3fff640 1 -- 172.21.15.83:0/1251024100 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c00c89d0 con 0x559c52023d10 2024-04-17T10:49:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9db875000 1 -- 172.21.15.83:0/1251024100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c52023d10 msgr2=0x559c51fd9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9db875000 1 --2- 172.21.15.83:0/1251024100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c52023d10 0x559c51fd9fe0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0000c10 tx=0x7fa9c0004740 comp rx=0 tx=0).stop 2024-04-17T10:49:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9db875000 1 -- 172.21.15.83:0/1251024100 shutdown_connections 2024-04-17T10:49:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9db875000 1 --2- 172.21.15.83:0/1251024100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c51fda520 0x559c51fda900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9db875000 1 --2- 172.21.15.83:0/1251024100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c52023d10 0x559c51fd9fe0 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9db875000 1 --2- 172.21.15.83:0/1251024100 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559c520233f0 0x559c520237d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9db875000 1 -- 172.21.15.83:0/1251024100 >> 172.21.15.83:0/1251024100 conn(0x559c51fd64d0 msgr2=0x559c51fd88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9db875000 1 -- 172.21.15.83:0/1251024100 shutdown_connections 2024-04-17T10:49:49.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.765+0000 7fa9db875000 1 -- 172.21.15.83:0/1251024100 wait complete. 2024-04-17T10:49:49.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9db875000 1 -- start start 2024-04-17T10:49:49.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9db875000 1 Processor -- start 2024-04-17T10:49:49.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9db875000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c51fda520 0x559c51fe0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9db875000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c520233f0 0x559c520110c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9db875000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559c52023d10 0x559c52011600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9db875000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c51dcd610 con 0x559c51fda520 2024-04-17T10:49:49.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9db875000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c51de29a0 con 0x559c520233f0 2024-04-17T10:49:49.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9db875000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c51e2c380 con 0x559c52023d10 2024-04-17T10:49:49.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9db875000 10 client.0 _ll_register_callbacks cb 0x559c51edc0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:49:49.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9d64a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c51fda520 0x559c51fe0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9d64a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c51fda520 0x559c51fe0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47838/0 (socket says 172.21.15.83:47838) 2024-04-17T10:49:49.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9d64a4640 1 -- 172.21.15.83:0/344194592 learned_addr learned my addr 172.21.15.83:0/344194592 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:49.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9d5ca3640 1 --2- 172.21.15.83:0/344194592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c520233f0 0x559c520110c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9d6ca5640 1 --2- 172.21.15.83:0/344194592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559c52023d10 0x559c52011600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9d64a4640 1 -- 172.21.15.83:0/344194592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559c52023d10 msgr2=0x559c52011600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9d64a4640 1 --2- 172.21.15.83:0/344194592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559c52023d10 0x559c52011600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9d64a4640 1 -- 172.21.15.83:0/344194592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c520233f0 msgr2=0x559c520110c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9d64a4640 1 --2- 172.21.15.83:0/344194592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c520233f0 0x559c520110c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9d64a4640 1 -- 172.21.15.83:0/344194592 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559c51e22d40 con 0x559c51fda520 2024-04-17T10:49:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9d5ca3640 1 --2- 172.21.15.83:0/344194592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c520233f0 0x559c520110c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9d64a4640 1 --2- 172.21.15.83:0/344194592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c51fda520 0x559c51fe0010 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc007410 tx=0x7fa9cc056070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229dc4473d0bb854 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:49:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9b17fa640 1 -- 172.21.15.83:0/344194592 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc01e040 con 0x559c51fda520 2024-04-17T10:49:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9b17fa640 1 -- 172.21.15.83:0/344194592 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa9cc056b60 con 0x559c51fda520 2024-04-17T10:49:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9db875000 1 -- 172.21.15.83:0/344194592 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9c00c8620 con 0x559c51fda520 2024-04-17T10:49:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9b17fa640 1 -- 172.21.15.83:0/344194592 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9cc083170 con 0x559c51fda520 2024-04-17T10:49:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9db875000 10 client.9393 fetch_fsmap learned FSMap version 76 2024-04-17T10:49:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9db875000 1 -- 172.21.15.83:0/344194592 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x559c51eaaf00 con 0x559c51fda520 2024-04-17T10:49:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9b17fa640 1 -- 172.21.15.83:0/344194592 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c00c8620 con 0x559c51fda520 2024-04-17T10:49:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9b17fa640 1 -- 172.21.15.83:0/344194592 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 76) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa9cc054c80 con 0x559c51fda520 2024-04-17T10:49:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9db875000 10 client.9393 fetch_fsmap finished waiting for FSMap version 76 2024-04-17T10:49:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9db875000 10 client.9393 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-17T10:49:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9db875000 20 client.9393 populate_metadata read hostname 'smithi083' 2024-04-17T10:49:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9db875000 1 --2- 172.21.15.83:0/344194592 >> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] conn(0x559c5202aa40 0x559c5202ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9db875000 4 client.9393 mds_command: new command op to 9198 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:49:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9db875000 1 -- 172.21.15.83:0/344194592 --> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c51dc3a20 con 0x559c5202aa40 2024-04-17T10:49:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9d5ca3640 1 --2- 172.21.15.83:0/344194592 >> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] conn(0x559c5202aa40 0x559c5202ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9d5ca3640 1 --2- 172.21.15.83:0/344194592 >> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] conn(0x559c5202aa40 0x559c5202ae20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95b160ba89c7ca80 server_cookie=be05a7ccd283385d in_seq=0 out_seq=0 2024-04-17T10:49:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9b17fa640 10 client.9393 ms_handle_connect on v2:172.21.15.83:6836/1559222491 2024-04-17T10:49:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9b17fa640 1 -- 172.21.15.83:0/344194592 <== mds.0 v2:172.21.15.83:6836/1559222491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559c51dc3a20 con 0x559c5202aa40 2024-04-17T10:49:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9b17fa640 10 client.9393 handle_command_reply: tid=0 2024-04-17T10:49:49.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9db875000 1 client.9393 shutdown 2024-04-17T10:49:49.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9db875000 1 -- 172.21.15.83:0/344194592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c51fda520 msgr2=0x559c51fe0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:49.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9db875000 1 --2- 172.21.15.83:0/344194592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c51fda520 0x559c51fe0010 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc007410 tx=0x7fa9cc056070 comp rx=0 tx=0).stop 2024-04-17T10:49:49.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9db875000 1 -- 172.21.15.83:0/344194592 shutdown_connections 2024-04-17T10:49:49.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9db875000 1 --2- 172.21.15.83:0/344194592 >> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] conn(0x559c5202aa40 0x559c5202ae20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9db875000 1 --2- 172.21.15.83:0/344194592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559c52023d10 0x559c52011600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9b17fa640 0 client.9393 ms_handle_reset on v2:172.21.15.83:6836/1559222491 2024-04-17T10:49:49.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9db875000 1 --2- 172.21.15.83:0/344194592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c520233f0 0x559c520110c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9db875000 1 --2- 172.21.15.83:0/344194592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c51fda520 0x559c51fe0010 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9db875000 1 -- 172.21.15.83:0/344194592 >> 172.21.15.83:0/344194592 conn(0x559c51fd64d0 msgr2=0x559c51fd9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:49.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9db875000 1 -- 172.21.15.83:0/344194592 shutdown_connections 2024-04-17T10:49:49.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.769+0000 7fa9db875000 1 -- 172.21.15.83:0/344194592 wait complete. 2024-04-17T10:49:49.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.773+0000 7fa9db875000 20 client.9393 trim_cache size 0 max 16384 2024-04-17T10:49:49.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.785+0000 7fa9db875000 1 Processor -- start 2024-04-17T10:49:49.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.785+0000 7fa9db875000 1 -- start start 2024-04-17T10:49:49.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.785+0000 7fa9db875000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c51fda520 0x559c5207fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.785+0000 7fa9db875000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c520233f0 0x559c52080390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.785+0000 7fa9db875000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559c52023d10 0x559c520808d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.785+0000 7fa9db875000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d0203100 con 0x559c51fda520 2024-04-17T10:49:49.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.785+0000 7fa9db875000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d02022c0 con 0x559c520233f0 2024-04-17T10:49:49.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.785+0000 7fa9db875000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d00c9040 con 0x559c52023d10 2024-04-17T10:49:49.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.785+0000 7fa9d64a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c51fda520 0x559c5207fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.785+0000 7fa9d64a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c51fda520 0x559c5207fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47850/0 (socket says 172.21.15.83:47850) 2024-04-17T10:49:49.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.785+0000 7fa9d64a4640 1 -- 172.21.15.83:0/3786182943 learned_addr learned my addr 172.21.15.83:0/3786182943 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:49.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9d64a4640 1 -- 172.21.15.83:0/3786182943 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559c52023d10 msgr2=0x559c520808d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:49.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9d64a4640 1 --2- 172.21.15.83:0/3786182943 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559c52023d10 0x559c520808d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9d64a4640 1 -- 172.21.15.83:0/3786182943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c520233f0 msgr2=0x559c52080390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:49.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9d64a4640 1 --2- 172.21.15.83:0/3786182943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c520233f0 0x559c52080390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9d64a4640 1 -- 172.21.15.83:0/3786182943 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c52023b40 con 0x559c51fda520 2024-04-17T10:49:49.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9d64a4640 1 --2- 172.21.15.83:0/3786182943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c51fda520 0x559c5207fe50 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc05da10 tx=0x7fa9cc058000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ab05bae760ce88 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:49.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9b37fe640 1 -- 172.21.15.83:0/3786182943 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc083170 con 0x559c51fda520 2024-04-17T10:49:49.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9b37fe640 1 -- 172.21.15.83:0/3786182943 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9cc06d200 con 0x559c51fda520 2024-04-17T10:49:49.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9b37fe640 1 -- 172.21.15.83:0/3786182943 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc056d60 con 0x559c51fda520 2024-04-17T10:49:49.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9db875000 1 -- 172.21.15.83:0/3786182943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c51fda520 msgr2=0x559c5207fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:49.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9db875000 1 --2- 172.21.15.83:0/3786182943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c51fda520 0x559c5207fe50 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc05da10 tx=0x7fa9cc058000 comp rx=0 tx=0).stop 2024-04-17T10:49:49.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9db875000 1 -- 172.21.15.83:0/3786182943 shutdown_connections 2024-04-17T10:49:49.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9db875000 1 --2- 172.21.15.83:0/3786182943 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559c52023d10 0x559c520808d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9db875000 1 --2- 172.21.15.83:0/3786182943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c520233f0 0x559c52080390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9db875000 1 --2- 172.21.15.83:0/3786182943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c51fda520 0x559c5207fe50 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9db875000 1 -- 172.21.15.83:0/3786182943 >> 172.21.15.83:0/3786182943 conn(0x559c51fd64d0 msgr2=0x559c51fdf460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:49.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9db875000 1 -- 172.21.15.83:0/3786182943 shutdown_connections 2024-04-17T10:49:49.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9db875000 1 -- 172.21.15.83:0/3786182943 wait complete. 2024-04-17T10:49:49.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9db875000 1 -- start start 2024-04-17T10:49:49.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9db875000 1 Processor -- start 2024-04-17T10:49:49.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9db875000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c51fda520 0x559c52084630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9db875000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559c520233f0 0x559c52084b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9db875000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c52023d10 0x559c52028420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9db875000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c51ddedc0 con 0x559c52023d10 2024-04-17T10:49:49.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9db875000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d0201cf0 con 0x559c51fda520 2024-04-17T10:49:49.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9db875000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c51e27060 con 0x559c520233f0 2024-04-17T10:49:49.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9db875000 10 client.0 _ll_register_callbacks cb 0x7fa9c00096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:49:49.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9d6ca5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c52023d10 0x559c52028420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9d6ca5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c52023d10 0x559c52028420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47862/0 (socket says 172.21.15.83:47862) 2024-04-17T10:49:49.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9d6ca5640 1 -- 172.21.15.83:0/3050382936 learned_addr learned my addr 172.21.15.83:0/3050382936 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:49.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9d5ca3640 1 --2- 172.21.15.83:0/3050382936 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559c520233f0 0x559c52084b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9d6ca5640 1 -- 172.21.15.83:0/3050382936 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559c520233f0 msgr2=0x559c52084b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:49.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9d6ca5640 1 --2- 172.21.15.83:0/3050382936 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559c520233f0 0x559c52084b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9d6ca5640 1 -- 172.21.15.83:0/3050382936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c51fda520 msgr2=0x559c52084630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:49.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9d6ca5640 1 --2- 172.21.15.83:0/3050382936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c51fda520 0x559c52084630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9d6ca5640 1 -- 172.21.15.83:0/3050382936 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559c51e81330 con 0x559c52023d10 2024-04-17T10:49:49.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9d6ca5640 1 --2- 172.21.15.83:0/3050382936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c52023d10 0x559c52028420 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fa9c80104f0 tx=0x7fa9c80108f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=522cb65d223fa8c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:49.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:49:49.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9b17fa640 1 -- 172.21.15.83:0/3050382936 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8003d90 con 0x559c52023d10 2024-04-17T10:49:49.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9b17fa640 1 -- 172.21.15.83:0/3050382936 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa9c80050d0 con 0x559c52023d10 2024-04-17T10:49:49.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9db875000 1 -- 172.21.15.83:0/3050382936 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9cc06d200 con 0x559c52023d10 2024-04-17T10:49:49.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9b17fa640 1 -- 172.21.15.83:0/3050382936 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9c8021050 con 0x559c52023d10 2024-04-17T10:49:49.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9db875000 10 client.9399 fetch_fsmap learned FSMap version 76 2024-04-17T10:49:49.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9db875000 1 -- 172.21.15.83:0/3050382936 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x559c51df45f0 con 0x559c52023d10 2024-04-17T10:49:49.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9b17fa640 1 -- 172.21.15.83:0/3050382936 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9cc06d200 con 0x559c52023d10 2024-04-17T10:49:49.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9b17fa640 1 -- 172.21.15.83:0/3050382936 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 76) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa9c8024020 con 0x559c52023d10 2024-04-17T10:49:49.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9db875000 10 client.9399 fetch_fsmap finished waiting for FSMap version 76 2024-04-17T10:49:49.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9db875000 10 client.9399 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-17T10:49:49.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9db875000 20 client.9399 populate_metadata read hostname 'smithi083' 2024-04-17T10:49:49.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9db875000 1 --2- 172.21.15.83:0/3050382936 >> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] conn(0x559c520291c0 0x559c520295d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9db875000 4 client.9399 mds_command: new command op to 9198 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-17T10:49:49.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9db875000 1 -- 172.21.15.83:0/3050382936 --> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x559c51e35290 con 0x559c520291c0 2024-04-17T10:49:49.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.789+0000 7fa9d64a4640 1 --2- 172.21.15.83:0/3050382936 >> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] conn(0x559c520291c0 0x559c520295d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.793+0000 7fa9b17fa640 10 client.9399 ms_handle_connect on v2:172.21.15.83:6836/1559222491 2024-04-17T10:49:49.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.793+0000 7fa9d64a4640 1 --2- 172.21.15.83:0/3050382936 >> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] conn(0x559c520291c0 0x559c520295d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e2f5b027542f6d6 server_cookie=858796b1fb50f9f7 in_seq=0 out_seq=0 2024-04-17T10:49:49.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.793+0000 7fa9b17fa640 1 -- 172.21.15.83:0/3050382936 <== mds.0 v2:172.21.15.83:6836/1559222491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559c51e35290 con 0x559c520291c0 2024-04-17T10:49:49.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.793+0000 7fa9b17fa640 10 client.9399 handle_command_reply: tid=0 2024-04-17T10:49:49.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.793+0000 7fa9db875000 1 client.9399 shutdown 2024-04-17T10:49:49.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.793+0000 7fa9db875000 1 -- 172.21.15.83:0/3050382936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c52023d10 msgr2=0x559c52028420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:49.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.793+0000 7fa9db875000 1 --2- 172.21.15.83:0/3050382936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c52023d10 0x559c52028420 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fa9c80104f0 tx=0x7fa9c80108f0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.793+0000 7fa9db875000 1 -- 172.21.15.83:0/3050382936 shutdown_connections 2024-04-17T10:49:49.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.793+0000 7fa9db875000 1 --2- 172.21.15.83:0/3050382936 >> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] conn(0x559c520291c0 0x559c520295d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.793+0000 7fa9db875000 1 --2- 172.21.15.83:0/3050382936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c52023d10 0x559c52028420 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.793+0000 7fa9db875000 1 --2- 172.21.15.83:0/3050382936 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559c520233f0 0x559c52084b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.793+0000 7fa9db875000 1 --2- 172.21.15.83:0/3050382936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c51fda520 0x559c52084630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.793+0000 7fa9b17fa640 0 client.9399 ms_handle_reset on v2:172.21.15.83:6836/1559222491 2024-04-17T10:49:49.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.793+0000 7fa9db875000 1 -- 172.21.15.83:0/3050382936 >> 172.21.15.83:0/3050382936 conn(0x559c51fd64d0 msgr2=0x559c52021e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:49.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.793+0000 7fa9db875000 1 -- 172.21.15.83:0/3050382936 shutdown_connections 2024-04-17T10:49:49.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.793+0000 7fa9db875000 1 -- 172.21.15.83:0/3050382936 wait complete. 2024-04-17T10:49:49.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.793+0000 7fa9db875000 20 client.9399 trim_cache size 0 max 16384 2024-04-17T10:49:49.791 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:49:49.791 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-17T10:49:49.791 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "678c1d4a-835d-4af3-b1c0-8ce40445977f", 2024-04-17T10:49:49.791 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-17T10:49:49.791 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:49:49.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.793+0000 7fa9d8708640 1 -- 172.21.15.83:0/3250720423 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa9b4066f60 msgr2=0x7fa9b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:49.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.793+0000 7fa9d8708640 1 --2- 172.21.15.83:0/3250720423 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa9b4066f60 0x7fa9b4069420 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fa9c00ae000 tx=0x7fa9c00a7000 comp rx=0 tx=0).stop 2024-04-17T10:49:49.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.793+0000 7fa9d8708640 1 -- 172.21.15.83:0/3250720423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9d00d1fd0 msgr2=0x7fa9d0155350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:49.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.793+0000 7fa9d8708640 1 --2- 172.21.15.83:0/3250720423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9d00d1fd0 0x7fa9d0155350 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00c820 tx=0x7fa9cc00cce0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.797+0000 7fa9d8708640 1 -- 172.21.15.83:0/3250720423 shutdown_connections 2024-04-17T10:49:49.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.797+0000 7fa9d8708640 1 --2- 172.21.15.83:0/3250720423 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa9b4066f60 0x7fa9b4069420 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.797+0000 7fa9d8708640 1 --2- 172.21.15.83:0/3250720423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9d015e020 0x7fa9d00c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.797+0000 7fa9d8708640 1 --2- 172.21.15.83:0/3250720423 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9d00d4510 0x7fa9d00b6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.797+0000 7fa9d8708640 1 --2- 172.21.15.83:0/3250720423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9d00d1fd0 0x7fa9d0155350 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.797+0000 7fa9d8708640 1 -- 172.21.15.83:0/3250720423 >> 172.21.15.83:0/3250720423 conn(0x7fa9d00d39f0 msgr2=0x7fa9d015cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:49.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.797+0000 7fa9d8708640 1 -- 172.21.15.83:0/3250720423 shutdown_connections 2024-04-17T10:49:49.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.797+0000 7fa9d8708640 1 -- 172.21.15.83:0/3250720423 wait complete. 2024-04-17T10:49:49.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:49.809 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub pause 2024-04-17T10:49:49.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.949+0000 7fcd9246f640 1 Processor -- start 2024-04-17T10:49:49.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.949+0000 7fcd9246f640 1 -- start start 2024-04-17T10:49:49.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.949+0000 7fcd9246f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd8c0ce400 0x7fcd8c0b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.949+0000 7fcd9246f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd8c15e4d0 con 0x7fcd8c15e0f0 2024-04-17T10:49:49.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.949+0000 7fcd9246f640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd8c162580 con 0x7fcd8c0b5450 2024-04-17T10:49:49.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.949+0000 7fcd9246f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c162b40 con 0x7fcd8c0ce400 2024-04-17T10:49:49.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.949+0000 7fcd8bfff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcd8c15e0f0 0x7fcd8c161c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54900/0 (socket says 172.21.15.83:54900) 2024-04-17T10:49:49.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.949+0000 7fcd8bfff640 1 -- 172.21.15.83:0/3491271104 learned_addr learned my addr 172.21.15.83:0/3491271104 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:49.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.949+0000 7fcd8b7fe640 1 --2- 172.21.15.83:0/3491271104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd8c0ce400 0x7fcd8c0b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.949+0000 7fcd8affd640 1 -- 172.21.15.83:0/3491271104 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2522912958 0 0) 0x7fcd8c15e4d0 con 0x7fcd8c15e0f0 2024-04-17T10:49:49.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.949+0000 7fcd8affd640 1 -- 172.21.15.83:0/3491271104 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd74003530 con 0x7fcd8c15e0f0 2024-04-17T10:49:49.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.949+0000 7fcd8affd640 1 -- 172.21.15.83:0/3491271104 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4034803756 0 0) 0x7fcd74003530 con 0x7fcd8c15e0f0 2024-04-17T10:49:49.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.949+0000 7fcd8affd640 1 -- 172.21.15.83:0/3491271104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd8c0ce400 msgr2=0x7fcd8c0b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:49.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.949+0000 7fcd8affd640 1 --2- 172.21.15.83:0/3491271104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd8c0ce400 0x7fcd8c0b6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.949+0000 7fcd8affd640 1 -- 172.21.15.83:0/3491271104 >> v1:172.21.15.167:6789/0 conn(0x7fcd8c0b5450 legacy=0x7fcd8c0d2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:49.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.949+0000 7fcd8affd640 1 -- 172.21.15.83:0/3491271104 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8c162860 con 0x7fcd8c15e0f0 2024-04-17T10:49:49.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.949+0000 7fcd8affd640 1 -- 172.21.15.83:0/3491271104 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fcd7c001680 con 0x7fcd8c15e0f0 2024-04-17T10:49:49.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.949+0000 7fcd8affd640 1 -- 172.21.15.83:0/3491271104 >> v1:172.21.15.83:6789/0 conn(0x7fcd8c15e0f0 legacy=0x7fcd8c161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:49.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.949+0000 7fcd8affd640 1 --2- 172.21.15.83:0/3491271104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd74003a60 0x7fcd74003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.949+0000 7fcd8affd640 1 --2- 172.21.15.83:0/3491271104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd8c0ce400 0x7fcd74004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.949+0000 7fcd8affd640 1 --2- 172.21.15.83:0/3491271104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd74004da0 0x7fcd74005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.949+0000 7fcd8affd640 1 -- 172.21.15.83:0/3491271104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c162b40 con 0x7fcd74004da0 2024-04-17T10:49:49.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.949+0000 7fcd8affd640 1 -- 172.21.15.83:0/3491271104 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7400bc30 con 0x7fcd74003a60 2024-04-17T10:49:49.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.949+0000 7fcd8affd640 1 -- 172.21.15.83:0/3491271104 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7400be30 con 0x7fcd8c0ce400 2024-04-17T10:49:49.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.949+0000 7fcd90a0c640 1 --2- 172.21.15.83:0/3491271104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd74004da0 0x7fcd74005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.949+0000 7fcd8b7fe640 1 --2- 172.21.15.83:0/3491271104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd8c0ce400 0x7fcd74004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.949+0000 7fcd8bfff640 1 --2- 172.21.15.83:0/3491271104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd74003a60 0x7fcd74003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.949+0000 7fcd90a0c640 1 -- 172.21.15.83:0/3491271104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd8c0ce400 msgr2=0x7fcd74004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:49.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.949+0000 7fcd90a0c640 1 --2- 172.21.15.83:0/3491271104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd8c0ce400 0x7fcd74004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.949+0000 7fcd90a0c640 1 -- 172.21.15.83:0/3491271104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd74003a60 msgr2=0x7fcd74003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:49.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.949+0000 7fcd90a0c640 1 --2- 172.21.15.83:0/3491271104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd74003a60 0x7fcd74003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.949+0000 7fcd90a0c640 1 -- 172.21.15.83:0/3491271104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd74003530 con 0x7fcd74004da0 2024-04-17T10:49:49.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.953+0000 7fcd90a0c640 1 --2- 172.21.15.83:0/3491271104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd74004da0 0x7fcd74005220 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fcd84001e70 tx=0x7fcd8401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b2422547603d97a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:49.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.953+0000 7fcd8affd640 1 -- 172.21.15.83:0/3491271104 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd84001a90 con 0x7fcd74004da0 2024-04-17T10:49:49.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.953+0000 7fcd8affd640 1 -- 172.21.15.83:0/3491271104 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd84003b80 con 0x7fcd74004da0 2024-04-17T10:49:49.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.953+0000 7fcd8affd640 1 -- 172.21.15.83:0/3491271104 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd84005080 con 0x7fcd74004da0 2024-04-17T10:49:49.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.953+0000 7fcd9246f640 1 -- 172.21.15.83:0/3491271104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd74004da0 msgr2=0x7fcd74005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:49.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.953+0000 7fcd9246f640 1 --2- 172.21.15.83:0/3491271104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd74004da0 0x7fcd74005220 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fcd84001e70 tx=0x7fcd8401c780 comp rx=0 tx=0).stop 2024-04-17T10:49:49.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.953+0000 7fcd90a0c640 1 -- 172.21.15.83:0/3491271104 reap_dead start 2024-04-17T10:49:49.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.953+0000 7fcd9246f640 1 -- 172.21.15.83:0/3491271104 shutdown_connections 2024-04-17T10:49:49.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.953+0000 7fcd9246f640 1 -- 172.21.15.83:0/3491271104 >> 172.21.15.83:0/3491271104 conn(0x7fcd8c0d0450 msgr2=0x7fcd8c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:49.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.953+0000 7fcd9246f640 1 -- 172.21.15.83:0/3491271104 shutdown_connections 2024-04-17T10:49:49.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.953+0000 7fcd9246f640 1 -- 172.21.15.83:0/3491271104 wait complete. 2024-04-17T10:49:49.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.953+0000 7fcd9246f640 1 Processor -- start 2024-04-17T10:49:49.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.953+0000 7fcd9246f640 1 -- start start 2024-04-17T10:49:49.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.953+0000 7fcd9246f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd74004da0 0x7fcd8c0c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.953+0000 7fcd9246f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd8c1446f0 0x7fcd8c15e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.953+0000 7fcd9246f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd8c15b6f0 0x7fcd8c15bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.953+0000 7fcd9246f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c0d2020 con 0x7fcd8c1446f0 2024-04-17T10:49:49.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.953+0000 7fcd9246f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c162320 con 0x7fcd8c15b6f0 2024-04-17T10:49:49.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.953+0000 7fcd9246f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c0b6d00 con 0x7fcd74004da0 2024-04-17T10:49:49.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.953+0000 7fcd8b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd8c1446f0 0x7fcd8c15e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.953+0000 7fcd8b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd8c1446f0 0x7fcd8c15e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47878/0 (socket says 172.21.15.83:47878) 2024-04-17T10:49:49.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.953+0000 7fcd8b7fe640 1 -- 172.21.15.83:0/1877968929 learned_addr learned my addr 172.21.15.83:0/1877968929 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:49.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.953+0000 7fcd90a0c640 1 --2- 172.21.15.83:0/1877968929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd8c15b6f0 0x7fcd8c15bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.957+0000 7fcd8bfff640 1 --2- 172.21.15.83:0/1877968929 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd74004da0 0x7fcd8c0c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.957+0000 7fcd8b7fe640 1 -- 172.21.15.83:0/1877968929 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd74004da0 msgr2=0x7fcd8c0c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:49.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.957+0000 7fcd8b7fe640 1 --2- 172.21.15.83:0/1877968929 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd74004da0 0x7fcd8c0c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.957+0000 7fcd8b7fe640 1 -- 172.21.15.83:0/1877968929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd8c15b6f0 msgr2=0x7fcd8c15bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:49.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.957+0000 7fcd8b7fe640 1 --2- 172.21.15.83:0/1877968929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd8c15b6f0 0x7fcd8c15bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.957+0000 7fcd8b7fe640 1 -- 172.21.15.83:0/1877968929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8c141c50 con 0x7fcd8c1446f0 2024-04-17T10:49:49.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.957+0000 7fcd8b7fe640 1 --2- 172.21.15.83:0/1877968929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd8c1446f0 0x7fcd8c15e650 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fcd8000f550 tx=0x7fcd8000fa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:49.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.957+0000 7fcd88ff9640 1 -- 172.21.15.83:0/1877968929 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd80019020 con 0x7fcd8c1446f0 2024-04-17T10:49:49.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.957+0000 7fcd88ff9640 1 -- 172.21.15.83:0/1877968929 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd80014870 con 0x7fcd8c1446f0 2024-04-17T10:49:49.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.957+0000 7fcd9246f640 1 -- 172.21.15.83:0/1877968929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd8c141ef0 con 0x7fcd8c1446f0 2024-04-17T10:49:49.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.957+0000 7fcd88ff9640 1 -- 172.21.15.83:0/1877968929 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd80014c40 con 0x7fcd8c1446f0 2024-04-17T10:49:49.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.957+0000 7fcd9246f640 1 -- 172.21.15.83:0/1877968929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8c1424b0 con 0x7fcd8c1446f0 2024-04-17T10:49:49.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.957+0000 7fcd88ff9640 1 -- 172.21.15.83:0/1877968929 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcd8001c020 con 0x7fcd8c1446f0 2024-04-17T10:49:49.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.957+0000 7fcd88ff9640 1 --2- 172.21.15.83:0/1877968929 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd78066f90 0x7fcd78069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.957+0000 7fcd8bfff640 1 --2- 172.21.15.83:0/1877968929 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd78066f90 0x7fcd78069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.957+0000 7fcd88ff9640 1 -- 172.21.15.83:0/1877968929 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd800857f0 con 0x7fcd8c1446f0 2024-04-17T10:49:49.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.957+0000 7fcd8bfff640 1 --2- 172.21.15.83:0/1877968929 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd78066f90 0x7fcd78069450 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c001ec0 tx=0x7fcd7c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:49.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.957+0000 7fcd955dc000 1 Processor -- start 2024-04-17T10:49:49.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.957+0000 7fcd955dc000 1 -- start start 2024-04-17T10:49:49.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd955dc000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c499eeaa40 0x55c499eeae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd955dc000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c499eeb360 0x55c499ea5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd955dc000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c499ea5d90 0x55c499ea6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd955dc000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c499d881f0 con 0x55c499ea5d90 2024-04-17T10:49:49.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd955dc000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c499ceee90 con 0x55c499eeb360 2024-04-17T10:49:49.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd955dc000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c499c76f50 con 0x55c499eeaa40 2024-04-17T10:49:49.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd90a0c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c499ea5d90 0x55c499ea6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd90a0c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c499ea5d90 0x55c499ea6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47888/0 (socket says 172.21.15.83:47888) 2024-04-17T10:49:49.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd90a0c640 1 -- 172.21.15.83:0/2176084520 learned_addr learned my addr 172.21.15.83:0/2176084520 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:49.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd8bfff640 1 --2- 172.21.15.83:0/2176084520 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c499eeaa40 0x55c499eeae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd8b7fe640 1 --2- 172.21.15.83:0/2176084520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c499eeb360 0x55c499ea5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd90a0c640 1 -- 172.21.15.83:0/2176084520 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c499eeaa40 msgr2=0x55c499eeae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:49.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd90a0c640 1 --2- 172.21.15.83:0/2176084520 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c499eeaa40 0x55c499eeae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd90a0c640 1 -- 172.21.15.83:0/2176084520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c499eeb360 msgr2=0x55c499ea5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd90a0c640 1 --2- 172.21.15.83:0/2176084520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c499eeb360 0x55c499ea5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd90a0c640 1 -- 172.21.15.83:0/2176084520 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c499cc68b0 con 0x55c499ea5d90 2024-04-17T10:49:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd8b7fe640 1 --2- 172.21.15.83:0/2176084520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c499eeb360 0x55c499ea5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd90a0c640 1 --2- 172.21.15.83:0/2176084520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c499ea5d90 0x55c499ea6170 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fcd8401d020 tx=0x7fcd840014b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaead8b814cb89b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd71ffb640 1 -- 172.21.15.83:0/2176084520 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd840ba070 con 0x55c499ea5d90 2024-04-17T10:49:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd71ffb640 1 -- 172.21.15.83:0/2176084520 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd84006d60 con 0x55c499ea5d90 2024-04-17T10:49:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd71ffb640 1 -- 172.21.15.83:0/2176084520 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd84003ba0 con 0x55c499ea5d90 2024-04-17T10:49:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd955dc000 1 -- 172.21.15.83:0/2176084520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c499ea5d90 msgr2=0x55c499ea6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd955dc000 1 --2- 172.21.15.83:0/2176084520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c499ea5d90 0x55c499ea6170 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fcd8401d020 tx=0x7fcd840014b0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd955dc000 1 -- 172.21.15.83:0/2176084520 shutdown_connections 2024-04-17T10:49:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd955dc000 1 --2- 172.21.15.83:0/2176084520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c499ea5d90 0x55c499ea6170 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd955dc000 1 --2- 172.21.15.83:0/2176084520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c499eeb360 0x55c499ea5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd955dc000 1 --2- 172.21.15.83:0/2176084520 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c499eeaa40 0x55c499eeae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd955dc000 1 -- 172.21.15.83:0/2176084520 >> 172.21.15.83:0/2176084520 conn(0x55c499ea1d40 msgr2=0x55c499ea4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:49.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd955dc000 1 -- 172.21.15.83:0/2176084520 shutdown_connections 2024-04-17T10:49:49.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd955dc000 1 -- 172.21.15.83:0/2176084520 wait complete. 2024-04-17T10:49:49.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd955dc000 1 -- start start 2024-04-17T10:49:49.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd955dc000 1 Processor -- start 2024-04-17T10:49:49.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd955dc000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c499ea5d90 0x55c499ef21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd955dc000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c499eeaa40 0x55c499ef2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd955dc000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c499eeb360 0x55c499eabea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd955dc000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c499ca9540 con 0x55c499ea5d90 2024-04-17T10:49:49.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd955dc000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c499ceaf30 con 0x55c499eeaa40 2024-04-17T10:49:49.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd955dc000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c499cb9420 con 0x55c499eeb360 2024-04-17T10:49:49.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd955dc000 10 client.0 _ll_register_callbacks cb 0x55c499da3180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:49:49.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd8bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c499ea5d90 0x55c499ef21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd8bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c499ea5d90 0x55c499ef21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47896/0 (socket says 172.21.15.83:47896) 2024-04-17T10:49:49.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd8bfff640 1 -- 172.21.15.83:0/1813538238 learned_addr learned my addr 172.21.15.83:0/1813538238 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:49.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd90a0c640 1 --2- 172.21.15.83:0/1813538238 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c499eeb360 0x55c499eabea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd8bfff640 1 -- 172.21.15.83:0/1813538238 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c499eeb360 msgr2=0x55c499eabea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:49.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd8b7fe640 1 --2- 172.21.15.83:0/1813538238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c499eeaa40 0x55c499ef2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd8bfff640 1 --2- 172.21.15.83:0/1813538238 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c499eeb360 0x55c499eabea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd8bfff640 1 -- 172.21.15.83:0/1813538238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c499eeaa40 msgr2=0x55c499ef2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:49.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd8bfff640 1 --2- 172.21.15.83:0/1813538238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c499eeaa40 0x55c499ef2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd8bfff640 1 -- 172.21.15.83:0/1813538238 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c499c771a0 con 0x55c499ea5d90 2024-04-17T10:49:49.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd8bfff640 1 --2- 172.21.15.83:0/1813538238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c499ea5d90 0x55c499ef21c0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c013e10 tx=0x7fcd7c000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b42c131ca2a3522 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:49.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd6affd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:49:49.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd6affd640 1 -- 172.21.15.83:0/1813538238 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c010040 con 0x55c499ea5d90 2024-04-17T10:49:49.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd6affd640 1 -- 172.21.15.83:0/1813538238 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd7c00e800 con 0x55c499ea5d90 2024-04-17T10:49:49.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd955dc000 1 -- 172.21.15.83:0/1813538238 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd84006d60 con 0x55c499ea5d90 2024-04-17T10:49:49.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd6affd640 1 -- 172.21.15.83:0/1813538238 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd7c00ea00 con 0x55c499ea5d90 2024-04-17T10:49:49.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd955dc000 10 client.9411 fetch_fsmap learned FSMap version 76 2024-04-17T10:49:49.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.961+0000 7fcd955dc000 1 -- 172.21.15.83:0/1813538238 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x55c499d71840 con 0x55c499ea5d90 2024-04-17T10:49:49.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.965+0000 7fcd6affd640 1 -- 172.21.15.83:0/1813538238 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd84006d60 con 0x55c499ea5d90 2024-04-17T10:49:49.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.965+0000 7fcd6affd640 1 -- 172.21.15.83:0/1813538238 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 76) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcd7c00eba0 con 0x55c499ea5d90 2024-04-17T10:49:49.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.965+0000 7fcd955dc000 10 client.9411 fetch_fsmap finished waiting for FSMap version 76 2024-04-17T10:49:49.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.965+0000 7fcd955dc000 10 client.9411 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-17T10:49:49.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.965+0000 7fcd955dc000 20 client.9411 populate_metadata read hostname 'smithi083' 2024-04-17T10:49:49.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.965+0000 7fcd955dc000 1 --2- 172.21.15.83:0/1813538238 >> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] conn(0x55c499eacb80 0x55c499eacf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.965+0000 7fcd955dc000 4 client.9411 mds_command: new command op to 9198 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:49:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.965+0000 7fcd955dc000 1 -- 172.21.15.83:0/1813538238 --> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c499c8abe0 con 0x55c499eacb80 2024-04-17T10:49:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.965+0000 7fcd8b7fe640 1 --2- 172.21.15.83:0/1813538238 >> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] conn(0x55c499eacb80 0x55c499eacf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.965+0000 7fcd6affd640 10 client.9411 ms_handle_connect on v2:172.21.15.83:6836/1559222491 2024-04-17T10:49:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.965+0000 7fcd8b7fe640 1 --2- 172.21.15.83:0/1813538238 >> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] conn(0x55c499eacb80 0x55c499eacf60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88778e82207e0f1a server_cookie=79d47f08f47018d0 in_seq=0 out_seq=0 2024-04-17T10:49:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.965+0000 7fcd6affd640 1 -- 172.21.15.83:0/1813538238 <== mds.0 v2:172.21.15.83:6836/1559222491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c499c8abe0 con 0x55c499eacb80 2024-04-17T10:49:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.965+0000 7fcd6affd640 10 client.9411 handle_command_reply: tid=0 2024-04-17T10:49:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.965+0000 7fcd955dc000 1 client.9411 shutdown 2024-04-17T10:49:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.965+0000 7fcd955dc000 1 -- 172.21.15.83:0/1813538238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c499ea5d90 msgr2=0x55c499ef21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.965+0000 7fcd955dc000 1 --2- 172.21.15.83:0/1813538238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c499ea5d90 0x55c499ef21c0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c013e10 tx=0x7fcd7c000bd0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.965+0000 7fcd955dc000 1 -- 172.21.15.83:0/1813538238 shutdown_connections 2024-04-17T10:49:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.965+0000 7fcd955dc000 1 --2- 172.21.15.83:0/1813538238 >> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] conn(0x55c499eacb80 0x55c499eacf60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.965+0000 7fcd955dc000 1 --2- 172.21.15.83:0/1813538238 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c499eeb360 0x55c499eabea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.965+0000 7fcd955dc000 1 --2- 172.21.15.83:0/1813538238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c499eeaa40 0x55c499ef2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.965+0000 7fcd955dc000 1 --2- 172.21.15.83:0/1813538238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c499ea5d90 0x55c499ef21c0 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.965+0000 7fcd955dc000 1 -- 172.21.15.83:0/1813538238 >> 172.21.15.83:0/1813538238 conn(0x55c499ea1d40 msgr2=0x55c499ea49c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:49.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.965+0000 7fcd6affd640 0 client.9411 ms_handle_reset on v2:172.21.15.83:6836/1559222491 2024-04-17T10:49:49.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.965+0000 7fcd955dc000 1 -- 172.21.15.83:0/1813538238 shutdown_connections 2024-04-17T10:49:49.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.965+0000 7fcd955dc000 1 -- 172.21.15.83:0/1813538238 wait complete. 2024-04-17T10:49:49.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.965+0000 7fcd955dc000 20 client.9411 trim_cache size 0 max 16384 2024-04-17T10:49:49.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.977+0000 7fcd955dc000 1 Processor -- start 2024-04-17T10:49:49.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.977+0000 7fcd955dc000 1 -- start start 2024-04-17T10:49:49.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd955dc000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c499ea5d90 0x55c499f46e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd955dc000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c499eeaa40 0x55c499f473b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd955dc000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c499eeb360 0x55c499f478f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd955dc000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c2004d0 con 0x55c499eeaa40 2024-04-17T10:49:49.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd955dc000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c1ff6d0 con 0x55c499ea5d90 2024-04-17T10:49:49.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd955dc000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c0b7df0 con 0x55c499eeb360 2024-04-17T10:49:49.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd8b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c499eeaa40 0x55c499f473b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd8b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c499eeaa40 0x55c499f473b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47904/0 (socket says 172.21.15.83:47904) 2024-04-17T10:49:49.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd8b7fe640 1 -- 172.21.15.83:0/2273172316 learned_addr learned my addr 172.21.15.83:0/2273172316 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:49.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd8b7fe640 1 -- 172.21.15.83:0/2273172316 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c499eeb360 msgr2=0x55c499f478f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:49.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd8b7fe640 1 --2- 172.21.15.83:0/2273172316 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c499eeb360 0x55c499f478f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd8b7fe640 1 -- 172.21.15.83:0/2273172316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c499ea5d90 msgr2=0x55c499f46e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:49.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd8b7fe640 1 --2- 172.21.15.83:0/2273172316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c499ea5d90 0x55c499f46e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd8b7fe640 1 -- 172.21.15.83:0/2273172316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c499d48a10 con 0x55c499eeaa40 2024-04-17T10:49:49.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd8b7fe640 1 --2- 172.21.15.83:0/2273172316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c499eeaa40 0x55c499f473b0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fcd80006ed0 tx=0x7fcd80020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc77d535cccb541 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:49.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd717fa640 1 -- 172.21.15.83:0/2273172316 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd80081070 con 0x55c499eeaa40 2024-04-17T10:49:49.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd717fa640 1 -- 172.21.15.83:0/2273172316 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8005e800 con 0x55c499eeaa40 2024-04-17T10:49:49.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd717fa640 1 -- 172.21.15.83:0/2273172316 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd800543f0 con 0x55c499eeaa40 2024-04-17T10:49:49.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd955dc000 1 -- 172.21.15.83:0/2273172316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c499eeaa40 msgr2=0x55c499f473b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:49.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd955dc000 1 --2- 172.21.15.83:0/2273172316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c499eeaa40 0x55c499f473b0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fcd80006ed0 tx=0x7fcd80020040 comp rx=0 tx=0).stop 2024-04-17T10:49:49.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd955dc000 1 -- 172.21.15.83:0/2273172316 shutdown_connections 2024-04-17T10:49:49.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd955dc000 1 --2- 172.21.15.83:0/2273172316 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c499eeb360 0x55c499f478f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd955dc000 1 --2- 172.21.15.83:0/2273172316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c499eeaa40 0x55c499f473b0 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd955dc000 1 --2- 172.21.15.83:0/2273172316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c499ea5d90 0x55c499f46e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd955dc000 1 -- 172.21.15.83:0/2273172316 >> 172.21.15.83:0/2273172316 conn(0x55c499ea1d40 msgr2=0x55c499ea4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:49.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd955dc000 1 -- 172.21.15.83:0/2273172316 shutdown_connections 2024-04-17T10:49:49.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd955dc000 1 -- 172.21.15.83:0/2273172316 wait complete. 2024-04-17T10:49:49.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd955dc000 1 -- start start 2024-04-17T10:49:49.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd955dc000 1 Processor -- start 2024-04-17T10:49:49.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd955dc000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c499ea5d90 0x55c499f4b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd955dc000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c499eeaa40 0x55c499f4b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd955dc000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c499eeb360 0x55c499eefa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd955dc000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c1436d0 con 0x55c499ea5d90 2024-04-17T10:49:49.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd955dc000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c499ceaf30 con 0x55c499eeb360 2024-04-17T10:49:49.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd955dc000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c499c922d0 con 0x55c499eeaa40 2024-04-17T10:49:49.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd955dc000 10 client.0 _ll_register_callbacks cb 0x55c499cb1a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:49:49.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd8bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c499ea5d90 0x55c499f4b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd8bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c499ea5d90 0x55c499f4b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47916/0 (socket says 172.21.15.83:47916) 2024-04-17T10:49:49.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd8bfff640 1 -- 172.21.15.83:0/4084363700 learned_addr learned my addr 172.21.15.83:0/4084363700 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:49.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd8bfff640 1 -- 172.21.15.83:0/4084363700 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c499eeaa40 msgr2=0x55c499f4b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:49.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd8bfff640 1 --2- 172.21.15.83:0/4084363700 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c499eeaa40 0x55c499f4b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd8bfff640 1 -- 172.21.15.83:0/4084363700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c499eeb360 msgr2=0x55c499eefa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:49.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd8bfff640 1 --2- 172.21.15.83:0/4084363700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c499eeb360 0x55c499eefa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd8bfff640 1 -- 172.21.15.83:0/4084363700 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c499c83b00 con 0x55c499ea5d90 2024-04-17T10:49:49.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd8bfff640 1 --2- 172.21.15.83:0/4084363700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c499ea5d90 0x55c499f4b2a0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c001490 tx=0x7fcd7c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6faa73708059aace server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:49.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd6affd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:49:49.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd6affd640 1 -- 172.21.15.83:0/4084363700 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c03f060 con 0x55c499ea5d90 2024-04-17T10:49:49.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd6affd640 1 -- 172.21.15.83:0/4084363700 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd7c03ab10 con 0x55c499ea5d90 2024-04-17T10:49:49.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd955dc000 1 -- 172.21.15.83:0/4084363700 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd8005e800 con 0x55c499ea5d90 2024-04-17T10:49:49.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd6affd640 1 -- 172.21.15.83:0/4084363700 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd7c013bd0 con 0x55c499ea5d90 2024-04-17T10:49:49.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd955dc000 10 client.9417 fetch_fsmap learned FSMap version 76 2024-04-17T10:49:49.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd955dc000 1 -- 172.21.15.83:0/4084363700 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x55c499c95f80 con 0x55c499ea5d90 2024-04-17T10:49:49.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.981+0000 7fcd6affd640 1 -- 172.21.15.83:0/4084363700 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8005e800 con 0x55c499ea5d90 2024-04-17T10:49:49.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd6affd640 1 -- 172.21.15.83:0/4084363700 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 76) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcd7c00e3d0 con 0x55c499ea5d90 2024-04-17T10:49:49.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd955dc000 10 client.9417 fetch_fsmap finished waiting for FSMap version 76 2024-04-17T10:49:49.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd955dc000 10 client.9417 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-17T10:49:49.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd955dc000 20 client.9417 populate_metadata read hostname 'smithi083' 2024-04-17T10:49:49.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd955dc000 1 --2- 172.21.15.83:0/4084363700 >> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] conn(0x55c499ef0810 0x55c499ef0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:49.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd955dc000 4 client.9417 mds_command: new command op to 9198 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-17T10:49:49.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd955dc000 1 -- 172.21.15.83:0/4084363700 --> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x55c499cfc3d0 con 0x55c499ef0810 2024-04-17T10:49:49.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd8b7fe640 1 --2- 172.21.15.83:0/4084363700 >> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] conn(0x55c499ef0810 0x55c499ef0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:49.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd6affd640 10 client.9417 ms_handle_connect on v2:172.21.15.83:6836/1559222491 2024-04-17T10:49:49.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd8b7fe640 1 --2- 172.21.15.83:0/4084363700 >> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] conn(0x55c499ef0810 0x55c499ef0c20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68e0e88b68044ea6 server_cookie=47c21c58129e89a4 in_seq=0 out_seq=0 2024-04-17T10:49:49.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd6affd640 1 -- 172.21.15.83:0/4084363700 <== mds.0 v2:172.21.15.83:6836/1559222491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55c499cfc3d0 con 0x55c499ef0810 2024-04-17T10:49:49.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd6affd640 10 client.9417 handle_command_reply: tid=0 2024-04-17T10:49:49.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd955dc000 1 client.9417 shutdown 2024-04-17T10:49:49.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd955dc000 1 -- 172.21.15.83:0/4084363700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c499ea5d90 msgr2=0x55c499f4b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:49.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd955dc000 1 --2- 172.21.15.83:0/4084363700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c499ea5d90 0x55c499f4b2a0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c001490 tx=0x7fcd7c017000 comp rx=0 tx=0).stop 2024-04-17T10:49:49.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd955dc000 1 -- 172.21.15.83:0/4084363700 shutdown_connections 2024-04-17T10:49:49.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd955dc000 1 --2- 172.21.15.83:0/4084363700 >> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] conn(0x55c499ef0810 0x55c499ef0c20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd955dc000 1 --2- 172.21.15.83:0/4084363700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c499eeb360 0x55c499eefa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd6affd640 0 client.9417 ms_handle_reset on v2:172.21.15.83:6836/1559222491 2024-04-17T10:49:49.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd955dc000 1 --2- 172.21.15.83:0/4084363700 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c499eeaa40 0x55c499f4b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd955dc000 1 --2- 172.21.15.83:0/4084363700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c499ea5d90 0x55c499f4b2a0 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd955dc000 1 -- 172.21.15.83:0/4084363700 >> 172.21.15.83:0/4084363700 conn(0x55c499ea1d40 msgr2=0x55c499ea49c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:49.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd955dc000 1 -- 172.21.15.83:0/4084363700 shutdown_connections 2024-04-17T10:49:49.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd955dc000 1 -- 172.21.15.83:0/4084363700 wait complete. 2024-04-17T10:49:49.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd955dc000 20 client.9417 trim_cache size 0 max 16384 2024-04-17T10:49:49.983 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:49:49.983 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0 2024-04-17T10:49:49.983 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:49:49.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd9246f640 1 -- 172.21.15.83:0/1877968929 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd78066f90 msgr2=0x7fcd78069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:49.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd9246f640 1 --2- 172.21.15.83:0/1877968929 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd78066f90 0x7fcd78069450 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c001ec0 tx=0x7fcd7c009040 comp rx=0 tx=0).stop 2024-04-17T10:49:49.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd9246f640 1 -- 172.21.15.83:0/1877968929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd8c1446f0 msgr2=0x7fcd8c15e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:49.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd9246f640 1 --2- 172.21.15.83:0/1877968929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd8c1446f0 0x7fcd8c15e650 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fcd8000f550 tx=0x7fcd8000fa10 comp rx=0 tx=0).stop 2024-04-17T10:49:49.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd9246f640 1 -- 172.21.15.83:0/1877968929 shutdown_connections 2024-04-17T10:49:49.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd9246f640 1 --2- 172.21.15.83:0/1877968929 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd78066f90 0x7fcd78069450 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd9246f640 1 --2- 172.21.15.83:0/1877968929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd8c15b6f0 0x7fcd8c15bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd9246f640 1 --2- 172.21.15.83:0/1877968929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd8c1446f0 0x7fcd8c15e650 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd9246f640 1 --2- 172.21.15.83:0/1877968929 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd74004da0 0x7fcd8c0c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:49.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd9246f640 1 -- 172.21.15.83:0/1877968929 >> 172.21.15.83:0/1877968929 conn(0x7fcd8c0d0450 msgr2=0x7fcd8c1541c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:49.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd9246f640 1 -- 172.21.15.83:0/1877968929 shutdown_connections 2024-04-17T10:49:49.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:49.985+0000 7fcd9246f640 1 -- 172.21.15.83:0/1877968929 wait complete. 2024-04-17T10:49:49.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:49:49.991 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub status 2024-04-17T10:49:50.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.113+0000 7f51bf7c5640 1 Processor -- start 2024-04-17T10:49:50.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.113+0000 7f51bf7c5640 1 -- start start 2024-04-17T10:49:50.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.113+0000 7f51bf7c5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b8154de0 0x7f51b806ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.113+0000 7f51bf7c5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f51b806b350 0x7f51b80b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.113+0000 7f51bf7c5640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51b815bd70 con 0x7f51b80b4040 2024-04-17T10:49:50.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.113+0000 7f51bf7c5640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51b80b4870 con 0x7f51b806b350 2024-04-17T10:49:50.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.113+0000 7f51bf7c5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b815bf60 con 0x7f51b8154de0 2024-04-17T10:49:50.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.113+0000 7f51bdd62640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f51b80b4040 0x7f51b8071580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54914/0 (socket says 172.21.15.83:54914) 2024-04-17T10:49:50.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.113+0000 7f51bdd62640 1 -- 172.21.15.83:0/771432589 learned_addr learned my addr 172.21.15.83:0/771432589 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:50.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.113+0000 7f51bcd60640 1 --2- 172.21.15.83:0/771432589 >> v2:172.21.15.167:3300/0 conn(0x7f51b806b350 0x7f51b80b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.113+0000 7f51bd561640 1 --2- 172.21.15.83:0/771432589 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b8154de0 0x7f51b806ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.113+0000 7f51affff640 1 -- 172.21.15.83:0/771432589 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060139184 0 0) 0x7f51b815bd70 con 0x7f51b80b4040 2024-04-17T10:49:50.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.113+0000 7f51affff640 1 -- 172.21.15.83:0/771432589 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51a4005740 con 0x7f51b80b4040 2024-04-17T10:49:50.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.113+0000 7f51affff640 1 -- 172.21.15.83:0/771432589 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 588559782 0 0) 0x7f51a4005740 con 0x7f51b80b4040 2024-04-17T10:49:50.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.113+0000 7f51affff640 1 -- 172.21.15.83:0/771432589 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b8154de0 msgr2=0x7f51b806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:50.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.113+0000 7f51affff640 1 --2- 172.21.15.83:0/771432589 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b8154de0 0x7f51b806ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.113+0000 7f51affff640 1 -- 172.21.15.83:0/771432589 >> v2:172.21.15.167:3300/0 conn(0x7f51b806b350 msgr2=0x7f51b80b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:50.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.113+0000 7f51affff640 1 --2- 172.21.15.83:0/771432589 >> v2:172.21.15.167:3300/0 conn(0x7f51b806b350 0x7f51b80b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.113+0000 7f51affff640 1 -- 172.21.15.83:0/771432589 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b815afb0 con 0x7f51b80b4040 2024-04-17T10:49:50.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.113+0000 7f51affff640 1 -- 172.21.15.83:0/771432589 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f51b0001a00 con 0x7f51b80b4040 2024-04-17T10:49:50.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.113+0000 7f51affff640 1 -- 172.21.15.83:0/771432589 >> v1:172.21.15.83:6789/0 conn(0x7f51b80b4040 legacy=0x7f51b8071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:50.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.113+0000 7f51affff640 1 --2- 172.21.15.83:0/771432589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51a4005c40 0x7f51a4006060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.113+0000 7f51affff640 1 --2- 172.21.15.83:0/771432589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51a4006a20 0x7f51a4006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.113+0000 7f51affff640 1 --2- 172.21.15.83:0/771432589 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b8154de0 0x7f51a40073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.113+0000 7f51affff640 1 -- 172.21.15.83:0/771432589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b815bf60 con 0x7f51a4005c40 2024-04-17T10:49:50.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.113+0000 7f51affff640 1 -- 172.21.15.83:0/771432589 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a400dbe0 con 0x7f51a4006a20 2024-04-17T10:49:50.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.113+0000 7f51affff640 1 -- 172.21.15.83:0/771432589 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a400dd60 con 0x7f51b8154de0 2024-04-17T10:49:50.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.117+0000 7f51bd561640 1 --2- 172.21.15.83:0/771432589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51a4005c40 0x7f51a4006060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.117+0000 7f51bdd62640 1 --2- 172.21.15.83:0/771432589 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b8154de0 0x7f51a40073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.117+0000 7f51bd561640 1 -- 172.21.15.83:0/771432589 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b8154de0 msgr2=0x7f51a40073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:50.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.117+0000 7f51bd561640 1 --2- 172.21.15.83:0/771432589 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b8154de0 0x7f51a40073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.117+0000 7f51bd561640 1 -- 172.21.15.83:0/771432589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51a4006a20 msgr2=0x7f51a4006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:50.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.117+0000 7f51bcd60640 1 --2- 172.21.15.83:0/771432589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51a4006a20 0x7f51a4006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.117+0000 7f51bd561640 1 --2- 172.21.15.83:0/771432589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51a4006a20 0x7f51a4006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.117+0000 7f51bd561640 1 -- 172.21.15.83:0/771432589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a4005740 con 0x7f51a4005c40 2024-04-17T10:49:50.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.117+0000 7f51bdd62640 1 --2- 172.21.15.83:0/771432589 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b8154de0 0x7f51a40073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:50.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.117+0000 7f51bcd60640 1 --2- 172.21.15.83:0/771432589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51a4006a20 0x7f51a4006ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:50.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.117+0000 7f51bd561640 1 --2- 172.21.15.83:0/771432589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51a4005c40 0x7f51a4006060 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f51b4009b10 tx=0x7f51b401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2dde11fbe5937ea server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:50.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.117+0000 7f51affff640 1 -- 172.21.15.83:0/771432589 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b40096b0 con 0x7f51a4005c40 2024-04-17T10:49:50.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.117+0000 7f51affff640 1 -- 172.21.15.83:0/771432589 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b40a9470 con 0x7f51a4005c40 2024-04-17T10:49:50.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.117+0000 7f51affff640 1 -- 172.21.15.83:0/771432589 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b40a9860 con 0x7f51a4005c40 2024-04-17T10:49:50.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.117+0000 7f51bf7c5640 1 -- 172.21.15.83:0/771432589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51a4005c40 msgr2=0x7f51a4006060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:50.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.117+0000 7f51bf7c5640 1 --2- 172.21.15.83:0/771432589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51a4005c40 0x7f51a4006060 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f51b4009b10 tx=0x7f51b401ca70 comp rx=0 tx=0).stop 2024-04-17T10:49:50.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.117+0000 7f51bdd62640 1 -- 172.21.15.83:0/771432589 reap_dead start 2024-04-17T10:49:50.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.117+0000 7f51bf7c5640 1 -- 172.21.15.83:0/771432589 shutdown_connections 2024-04-17T10:49:50.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.117+0000 7f51bf7c5640 1 -- 172.21.15.83:0/771432589 >> 172.21.15.83:0/771432589 conn(0x7f51b80b15b0 msgr2=0x7f51b80b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:50.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.117+0000 7f51bf7c5640 1 -- 172.21.15.83:0/771432589 shutdown_connections 2024-04-17T10:49:50.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.117+0000 7f51bf7c5640 1 -- 172.21.15.83:0/771432589 wait complete. 2024-04-17T10:49:50.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.121+0000 7f51bf7c5640 1 Processor -- start 2024-04-17T10:49:50.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.121+0000 7f51bf7c5640 1 -- start start 2024-04-17T10:49:50.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.121+0000 7f51bf7c5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51a4005c40 0x7f51b814d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.121+0000 7f51bf7c5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51b8155be0 0x7f51b815f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.121+0000 7f51bf7c5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b8156060 0x7f51b80cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.121+0000 7f51bf7c5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b81551c0 con 0x7f51a4005c40 2024-04-17T10:49:50.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.121+0000 7f51bf7c5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b815b850 con 0x7f51b8155be0 2024-04-17T10:49:50.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.121+0000 7f51bf7c5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b80b4470 con 0x7f51b8156060 2024-04-17T10:49:50.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.121+0000 7f51bd561640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51a4005c40 0x7f51b814d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.121+0000 7f51bd561640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51a4005c40 0x7f51b814d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47932/0 (socket says 172.21.15.83:47932) 2024-04-17T10:49:50.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.121+0000 7f51bd561640 1 -- 172.21.15.83:0/3117895012 learned_addr learned my addr 172.21.15.83:0/3117895012 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:50.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.121+0000 7f51bdd62640 1 --2- 172.21.15.83:0/3117895012 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b8156060 0x7f51b80cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.121+0000 7f51bcd60640 1 --2- 172.21.15.83:0/3117895012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51b8155be0 0x7f51b815f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.121+0000 7f51bd561640 1 -- 172.21.15.83:0/3117895012 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b8156060 msgr2=0x7f51b80cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:50.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.121+0000 7f51bd561640 1 --2- 172.21.15.83:0/3117895012 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b8156060 0x7f51b80cd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.121+0000 7f51bd561640 1 -- 172.21.15.83:0/3117895012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51b8155be0 msgr2=0x7f51b815f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:50.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.121+0000 7f51bd561640 1 --2- 172.21.15.83:0/3117895012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51b8155be0 0x7f51b815f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.121+0000 7f51bd561640 1 -- 172.21.15.83:0/3117895012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b815fbf0 con 0x7f51a4005c40 2024-04-17T10:49:50.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.121+0000 7f51bcd60640 1 --2- 172.21.15.83:0/3117895012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51b8155be0 0x7f51b815f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:50.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.121+0000 7f51bdd62640 1 --2- 172.21.15.83:0/3117895012 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b8156060 0x7f51b80cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:50.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.121+0000 7f51bd561640 1 --2- 172.21.15.83:0/3117895012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51a4005c40 0x7f51b814d090 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f51b401c8a0 tx=0x7f51b4019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:50.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.121+0000 7f51adffb640 1 -- 172.21.15.83:0/3117895012 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b400f000 con 0x7f51a4005c40 2024-04-17T10:49:50.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.121+0000 7f51adffb640 1 -- 172.21.15.83:0/3117895012 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b40a9cc0 con 0x7f51a4005c40 2024-04-17T10:49:50.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.121+0000 7f51bf7c5640 1 -- 172.21.15.83:0/3117895012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b8158ff0 con 0x7f51a4005c40 2024-04-17T10:49:50.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.121+0000 7f51adffb640 1 -- 172.21.15.83:0/3117895012 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b4004260 con 0x7f51a4005c40 2024-04-17T10:49:50.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.121+0000 7f51bf7c5640 1 -- 172.21.15.83:0/3117895012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b81595b0 con 0x7f51a4005c40 2024-04-17T10:49:50.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.121+0000 7f51adffb640 1 -- 172.21.15.83:0/3117895012 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f51b40c2030 con 0x7f51a4005c40 2024-04-17T10:49:50.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51adffb640 1 --2- 172.21.15.83:0/3117895012 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f519c066f60 0x7f519c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51adffb640 1 -- 172.21.15.83:0/3117895012 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51b411f5c0 con 0x7f51a4005c40 2024-04-17T10:49:50.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51bcd60640 1 --2- 172.21.15.83:0/3117895012 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f519c066f60 0x7f519c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51bcd60640 1 --2- 172.21.15.83:0/3117895012 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f519c066f60 0x7f519c069420 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f51a8006e90 tx=0x7f51a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:50.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51c2932000 1 Processor -- start 2024-04-17T10:49:50.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51c2932000 1 -- start start 2024-04-17T10:49:50.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51c2932000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eef6ae99d0 0x55eef6ae9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51c2932000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eef6aea2f0 0x55eef6a9fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51c2932000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eef6aa00a0 0x55eef6aa0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51c2932000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eef68ecff0 con 0x55eef6aa00a0 2024-04-17T10:49:50.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51c2932000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eef6875150 con 0x55eef6ae99d0 2024-04-17T10:49:50.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51c2932000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eef68e9090 con 0x55eef6aea2f0 2024-04-17T10:49:50.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51bdd62640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eef6aa00a0 0x55eef6aa0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51bdd62640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eef6aa00a0 0x55eef6aa0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47938/0 (socket says 172.21.15.83:47938) 2024-04-17T10:49:50.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51bdd62640 1 -- 172.21.15.83:0/4291274949 learned_addr learned my addr 172.21.15.83:0/4291274949 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:50.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51bcd60640 1 --2- 172.21.15.83:0/4291274949 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eef6aea2f0 0x55eef6a9fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51bd561640 1 --2- 172.21.15.83:0/4291274949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eef6ae99d0 0x55eef6ae9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51bdd62640 1 -- 172.21.15.83:0/4291274949 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eef6aea2f0 msgr2=0x55eef6a9fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:50.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51bdd62640 1 --2- 172.21.15.83:0/4291274949 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eef6aea2f0 0x55eef6a9fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51bdd62640 1 -- 172.21.15.83:0/4291274949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eef6ae99d0 msgr2=0x55eef6ae9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:50.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51bdd62640 1 --2- 172.21.15.83:0/4291274949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eef6ae99d0 0x55eef6ae9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51bdd62640 1 -- 172.21.15.83:0/4291274949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eef6a830d0 con 0x55eef6aa00a0 2024-04-17T10:49:50.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51bd561640 1 --2- 172.21.15.83:0/4291274949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eef6ae99d0 0x55eef6ae9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:50.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51bdd62640 1 --2- 172.21.15.83:0/4291274949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eef6aa00a0 0x55eef6aa0480 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f51b0000f90 tx=0x7f51b0002200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6efd5a51ba758a26 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:50.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51a2ffd640 1 -- 172.21.15.83:0/4291274949 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b0018070 con 0x55eef6aa00a0 2024-04-17T10:49:50.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51a2ffd640 1 -- 172.21.15.83:0/4291274949 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b0004d70 con 0x55eef6aa00a0 2024-04-17T10:49:50.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51a2ffd640 1 -- 172.21.15.83:0/4291274949 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b0007330 con 0x55eef6aa00a0 2024-04-17T10:49:50.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51c2932000 1 -- 172.21.15.83:0/4291274949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eef6aa00a0 msgr2=0x55eef6aa0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:50.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51c2932000 1 --2- 172.21.15.83:0/4291274949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eef6aa00a0 0x55eef6aa0480 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f51b0000f90 tx=0x7f51b0002200 comp rx=0 tx=0).stop 2024-04-17T10:49:50.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51c2932000 1 -- 172.21.15.83:0/4291274949 shutdown_connections 2024-04-17T10:49:50.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51c2932000 1 --2- 172.21.15.83:0/4291274949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eef6aa00a0 0x55eef6aa0480 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51c2932000 1 --2- 172.21.15.83:0/4291274949 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eef6aea2f0 0x55eef6a9fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51c2932000 1 --2- 172.21.15.83:0/4291274949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eef6ae99d0 0x55eef6ae9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51c2932000 1 -- 172.21.15.83:0/4291274949 >> 172.21.15.83:0/4291274949 conn(0x55eef6a9c020 msgr2=0x55eef6a9e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:50.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51c2932000 1 -- 172.21.15.83:0/4291274949 shutdown_connections 2024-04-17T10:49:50.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.125+0000 7f51c2932000 1 -- 172.21.15.83:0/4291274949 wait complete. 2024-04-17T10:49:50.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51c2932000 1 -- start start 2024-04-17T10:49:50.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51c2932000 1 Processor -- start 2024-04-17T10:49:50.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51c2932000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eef6aa00a0 0x55eef6ace7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51c2932000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eef6ae99d0 0x55eef6aced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51c2932000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eef6aea2f0 0x55eef6acf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51c2932000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eef68b1250 con 0x55eef6aea2f0 2024-04-17T10:49:50.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51c2932000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eef68a7690 con 0x55eef6aa00a0 2024-04-17T10:49:50.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51c2932000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eef6875a40 con 0x55eef6ae99d0 2024-04-17T10:49:50.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51c2932000 10 client.0 _ll_register_callbacks cb 0x55eef6a8ae90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:49:50.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51bdd62640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eef6aea2f0 0x55eef6acf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51bdd62640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eef6aea2f0 0x55eef6acf270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47942/0 (socket says 172.21.15.83:47942) 2024-04-17T10:49:50.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51bdd62640 1 -- 172.21.15.83:0/857610309 learned_addr learned my addr 172.21.15.83:0/857610309 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:50.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51bcd60640 1 --2- 172.21.15.83:0/857610309 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eef6ae99d0 0x55eef6aced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51bdd62640 1 -- 172.21.15.83:0/857610309 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eef6ae99d0 msgr2=0x55eef6aced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:50.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51bd561640 1 --2- 172.21.15.83:0/857610309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eef6aa00a0 0x55eef6ace7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51bdd62640 1 --2- 172.21.15.83:0/857610309 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eef6ae99d0 0x55eef6aced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51bdd62640 1 -- 172.21.15.83:0/857610309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eef6aa00a0 msgr2=0x55eef6ace7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:50.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51bdd62640 1 --2- 172.21.15.83:0/857610309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eef6aa00a0 0x55eef6ace7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51bdd62640 1 -- 172.21.15.83:0/857610309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eef699a1c0 con 0x55eef6aea2f0 2024-04-17T10:49:50.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51bdd62640 1 --2- 172.21.15.83:0/857610309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eef6aea2f0 0x55eef6acf270 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f51b0002240 tx=0x7f51b0001dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80965efc1e4b577 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:50.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f5197fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:49:50.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f5197fff640 1 -- 172.21.15.83:0/857610309 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b0023020 con 0x55eef6aea2f0 2024-04-17T10:49:50.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f5197fff640 1 -- 172.21.15.83:0/857610309 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51b0013dd0 con 0x55eef6aea2f0 2024-04-17T10:49:50.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51c2932000 1 -- 172.21.15.83:0/857610309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51b0004d70 con 0x55eef6aea2f0 2024-04-17T10:49:50.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f5197fff640 1 -- 172.21.15.83:0/857610309 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51b002d050 con 0x55eef6aea2f0 2024-04-17T10:49:50.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51c2932000 10 client.9429 fetch_fsmap learned FSMap version 76 2024-04-17T10:49:50.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51c2932000 1 -- 172.21.15.83:0/857610309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x55eef6996170 con 0x55eef6aea2f0 2024-04-17T10:49:50.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f5197fff640 1 -- 172.21.15.83:0/857610309 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b0004d70 con 0x55eef6aea2f0 2024-04-17T10:49:50.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f5197fff640 1 -- 172.21.15.83:0/857610309 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 76) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f51b0030020 con 0x55eef6aea2f0 2024-04-17T10:49:50.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51c2932000 10 client.9429 fetch_fsmap finished waiting for FSMap version 76 2024-04-17T10:49:50.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51c2932000 10 client.9429 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-17T10:49:50.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51c2932000 20 client.9429 populate_metadata read hostname 'smithi083' 2024-04-17T10:49:50.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51c2932000 1 --2- 172.21.15.83:0/857610309 >> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] conn(0x55eef6af9c80 0x55eef6afc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51c2932000 4 client.9429 mds_command: new command op to 9198 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:49:50.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51c2932000 1 -- 172.21.15.83:0/857610309 --> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eef6888c60 con 0x55eef6af9c80 2024-04-17T10:49:50.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51bd561640 1 --2- 172.21.15.83:0/857610309 >> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] conn(0x55eef6af9c80 0x55eef6afc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51bd561640 1 --2- 172.21.15.83:0/857610309 >> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] conn(0x55eef6af9c80 0x55eef6afc070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bb0c7e6b765be4d server_cookie=b9250df37adc6dfd in_seq=0 out_seq=0 2024-04-17T10:49:50.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f5197fff640 10 client.9429 ms_handle_connect on v2:172.21.15.83:6836/1559222491 2024-04-17T10:49:50.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f5197fff640 1 -- 172.21.15.83:0/857610309 <== mds.0 v2:172.21.15.83:6836/1559222491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eef6888c60 con 0x55eef6af9c80 2024-04-17T10:49:50.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f5197fff640 10 client.9429 handle_command_reply: tid=0 2024-04-17T10:49:50.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51c2932000 1 client.9429 shutdown 2024-04-17T10:49:50.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51c2932000 1 -- 172.21.15.83:0/857610309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eef6aea2f0 msgr2=0x55eef6acf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:50.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51c2932000 1 --2- 172.21.15.83:0/857610309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eef6aea2f0 0x55eef6acf270 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f51b0002240 tx=0x7f51b0001dd0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51c2932000 1 -- 172.21.15.83:0/857610309 shutdown_connections 2024-04-17T10:49:50.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51c2932000 1 --2- 172.21.15.83:0/857610309 >> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] conn(0x55eef6af9c80 0x55eef6afc070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51c2932000 1 --2- 172.21.15.83:0/857610309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eef6aea2f0 0x55eef6acf270 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51c2932000 1 --2- 172.21.15.83:0/857610309 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eef6ae99d0 0x55eef6aced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f5197fff640 0 client.9429 ms_handle_reset on v2:172.21.15.83:6836/1559222491 2024-04-17T10:49:50.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51c2932000 1 --2- 172.21.15.83:0/857610309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eef6aa00a0 0x55eef6ace7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51c2932000 1 -- 172.21.15.83:0/857610309 >> 172.21.15.83:0/857610309 conn(0x55eef6a9c020 msgr2=0x55eef6a9eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:50.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51c2932000 1 -- 172.21.15.83:0/857610309 shutdown_connections 2024-04-17T10:49:50.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51c2932000 1 -- 172.21.15.83:0/857610309 wait complete. 2024-04-17T10:49:50.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.129+0000 7f51c2932000 20 client.9429 trim_cache size 0 max 16384 2024-04-17T10:49:50.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.145+0000 7f51c2932000 1 Processor -- start 2024-04-17T10:49:50.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.145+0000 7f51c2932000 1 -- start start 2024-04-17T10:49:50.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.145+0000 7f51c2932000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eef6aa00a0 0x55eef6ae8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.145+0000 7f51c2932000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eef6ae99d0 0x55eef6a9f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.145+0000 7f51c2932000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eef6aea2f0 0x55eef6a9f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.145+0000 7f51c2932000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b81ff210 con 0x55eef6aea2f0 2024-04-17T10:49:50.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.145+0000 7f51c2932000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b81fe410 con 0x55eef6ae99d0 2024-04-17T10:49:50.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.145+0000 7f51c2932000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b815a960 con 0x55eef6aa00a0 2024-04-17T10:49:50.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.145+0000 7f51bdd62640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eef6aea2f0 0x55eef6a9f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.145+0000 7f51bdd62640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eef6aea2f0 0x55eef6a9f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47958/0 (socket says 172.21.15.83:47958) 2024-04-17T10:49:50.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.145+0000 7f51bdd62640 1 -- 172.21.15.83:0/1107343878 learned_addr learned my addr 172.21.15.83:0/1107343878 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:50.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.145+0000 7f51bcd60640 1 --2- 172.21.15.83:0/1107343878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eef6ae99d0 0x55eef6a9f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.145+0000 7f51bdd62640 1 -- 172.21.15.83:0/1107343878 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eef6aa00a0 msgr2=0x55eef6ae8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:50.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.145+0000 7f51bdd62640 1 --2- 172.21.15.83:0/1107343878 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eef6aa00a0 0x55eef6ae8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.145+0000 7f51bdd62640 1 -- 172.21.15.83:0/1107343878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eef6ae99d0 msgr2=0x55eef6a9f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:50.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.145+0000 7f51bdd62640 1 --2- 172.21.15.83:0/1107343878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eef6ae99d0 0x55eef6a9f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.145+0000 7f51bdd62640 1 -- 172.21.15.83:0/1107343878 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eef6933de0 con 0x55eef6aea2f0 2024-04-17T10:49:50.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.145+0000 7f51bcd60640 1 --2- 172.21.15.83:0/1107343878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eef6ae99d0 0x55eef6a9f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:50.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51bdd62640 1 --2- 172.21.15.83:0/1107343878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eef6aea2f0 0x55eef6a9f960 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f51b0001a60 tx=0x7f51b001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2baa8454e103765 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:50.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51a27fc640 1 -- 172.21.15.83:0/1107343878 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b0023070 con 0x55eef6aea2f0 2024-04-17T10:49:50.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51a27fc640 1 -- 172.21.15.83:0/1107343878 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b0019040 con 0x55eef6aea2f0 2024-04-17T10:49:50.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51a27fc640 1 -- 172.21.15.83:0/1107343878 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b0018070 con 0x55eef6aea2f0 2024-04-17T10:49:50.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51c2932000 1 -- 172.21.15.83:0/1107343878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eef6aea2f0 msgr2=0x55eef6a9f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:50.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51c2932000 1 --2- 172.21.15.83:0/1107343878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eef6aea2f0 0x55eef6a9f960 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f51b0001a60 tx=0x7f51b001d000 comp rx=0 tx=0).stop 2024-04-17T10:49:50.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51c2932000 1 -- 172.21.15.83:0/1107343878 shutdown_connections 2024-04-17T10:49:50.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51c2932000 1 --2- 172.21.15.83:0/1107343878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eef6aea2f0 0x55eef6a9f960 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51c2932000 1 --2- 172.21.15.83:0/1107343878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eef6ae99d0 0x55eef6a9f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51c2932000 1 --2- 172.21.15.83:0/1107343878 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eef6aa00a0 0x55eef6ae8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51c2932000 1 -- 172.21.15.83:0/1107343878 >> 172.21.15.83:0/1107343878 conn(0x55eef6a9c020 msgr2=0x55eef6ae80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:50.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51c2932000 1 -- 172.21.15.83:0/1107343878 shutdown_connections 2024-04-17T10:49:50.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51c2932000 1 -- 172.21.15.83:0/1107343878 wait complete. 2024-04-17T10:49:50.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51c2932000 1 -- start start 2024-04-17T10:49:50.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51c2932000 1 Processor -- start 2024-04-17T10:49:50.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51c2932000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eef6aa00a0 0x55eef6b4a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51c2932000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eef6ae99d0 0x55eef6b4aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51c2932000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eef6aea2f0 0x55eef6afc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51c2932000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eef6887e00 con 0x55eef6ae99d0 2024-04-17T10:49:50.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51c2932000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b81f6ae0 con 0x55eef6aea2f0 2024-04-17T10:49:50.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51c2932000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eef68ec300 con 0x55eef6aa00a0 2024-04-17T10:49:50.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51c2932000 10 client.0 _ll_register_callbacks cb 0x55eef6a6b040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:49:50.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51bcd60640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eef6ae99d0 0x55eef6b4aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51bcd60640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eef6ae99d0 0x55eef6b4aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47970/0 (socket says 172.21.15.83:47970) 2024-04-17T10:49:50.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51bcd60640 1 -- 172.21.15.83:0/2492872781 learned_addr learned my addr 172.21.15.83:0/2492872781 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:50.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51bdd62640 1 --2- 172.21.15.83:0/2492872781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eef6aea2f0 0x55eef6afc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51bd561640 1 --2- 172.21.15.83:0/2492872781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eef6aa00a0 0x55eef6b4a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51bcd60640 1 -- 172.21.15.83:0/2492872781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eef6aa00a0 msgr2=0x55eef6b4a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:50.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51bcd60640 1 --2- 172.21.15.83:0/2492872781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eef6aa00a0 0x55eef6b4a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51bcd60640 1 -- 172.21.15.83:0/2492872781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eef6aea2f0 msgr2=0x55eef6afc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:50.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51bcd60640 1 --2- 172.21.15.83:0/2492872781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eef6aea2f0 0x55eef6afc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51bcd60640 1 -- 172.21.15.83:0/2492872781 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eef688e8e0 con 0x55eef6ae99d0 2024-04-17T10:49:50.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51bdd62640 1 --2- 172.21.15.83:0/2492872781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eef6aea2f0 0x55eef6afc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:49:50.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51bcd60640 1 --2- 172.21.15.83:0/2492872781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eef6ae99d0 0x55eef6b4aa60 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f51a80156c0 tx=0x7f51a8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f41079d4fb1f030e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:50.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f5197fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:49:50.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f5197fff640 1 -- 172.21.15.83:0/2492872781 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a8023790 con 0x55eef6ae99d0 2024-04-17T10:49:50.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f5197fff640 1 -- 172.21.15.83:0/2492872781 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51a8023930 con 0x55eef6ae99d0 2024-04-17T10:49:50.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51c2932000 1 -- 172.21.15.83:0/2492872781 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51b0019040 con 0x55eef6ae99d0 2024-04-17T10:49:50.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f5197fff640 1 -- 172.21.15.83:0/2492872781 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51a8033050 con 0x55eef6ae99d0 2024-04-17T10:49:50.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51c2932000 10 client.9435 fetch_fsmap learned FSMap version 76 2024-04-17T10:49:50.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51c2932000 1 -- 172.21.15.83:0/2492872781 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x55eef69468f0 con 0x55eef6ae99d0 2024-04-17T10:49:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f5197fff640 1 -- 172.21.15.83:0/2492872781 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b0019040 con 0x55eef6ae99d0 2024-04-17T10:49:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f5197fff640 1 -- 172.21.15.83:0/2492872781 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 76) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f51a8036020 con 0x55eef6ae99d0 2024-04-17T10:49:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51c2932000 10 client.9435 fetch_fsmap finished waiting for FSMap version 76 2024-04-17T10:49:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51c2932000 10 client.9435 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-17T10:49:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51c2932000 20 client.9435 populate_metadata read hostname 'smithi083' 2024-04-17T10:49:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51c2932000 1 --2- 172.21.15.83:0/2492872781 >> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] conn(0x55eef6afcfc0 0x55eef6afd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51c2932000 4 client.9435 mds_command: new command op to 9198 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T10:49:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51c2932000 1 -- 172.21.15.83:0/2492872781 --> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55eef68fa530 con 0x55eef6afcfc0 2024-04-17T10:49:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51bd561640 1 --2- 172.21.15.83:0/2492872781 >> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] conn(0x55eef6afcfc0 0x55eef6afd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f5197fff640 10 client.9435 ms_handle_connect on v2:172.21.15.83:6836/1559222491 2024-04-17T10:49:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.149+0000 7f51bd561640 1 --2- 172.21.15.83:0/2492872781 >> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] conn(0x55eef6afcfc0 0x55eef6afd3d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b31b90e9612224e8 server_cookie=ed9d183c92d3bcdb in_seq=0 out_seq=0 2024-04-17T10:49:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.153+0000 7f5197fff640 1 -- 172.21.15.83:0/2492872781 <== mds.0 v2:172.21.15.83:6836/1559222491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x55eef68fa530 con 0x55eef6afcfc0 2024-04-17T10:49:50.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.153+0000 7f5197fff640 10 client.9435 handle_command_reply: tid=0 2024-04-17T10:49:50.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.153+0000 7f51c2932000 1 client.9435 shutdown 2024-04-17T10:49:50.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.153+0000 7f51c2932000 1 -- 172.21.15.83:0/2492872781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eef6ae99d0 msgr2=0x55eef6b4aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:50.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.153+0000 7f51c2932000 1 --2- 172.21.15.83:0/2492872781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eef6ae99d0 0x55eef6b4aa60 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f51a80156c0 tx=0x7f51a8004740 comp rx=0 tx=0).stop 2024-04-17T10:49:50.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.153+0000 7f51c2932000 1 -- 172.21.15.83:0/2492872781 shutdown_connections 2024-04-17T10:49:50.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.153+0000 7f51c2932000 1 --2- 172.21.15.83:0/2492872781 >> [v2:172.21.15.83:6836/1559222491,v1:172.21.15.83:6837/1559222491] conn(0x55eef6afcfc0 0x55eef6afd3d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.153+0000 7f51c2932000 1 --2- 172.21.15.83:0/2492872781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eef6aea2f0 0x55eef6afc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.153+0000 7f5197fff640 0 client.9435 ms_handle_reset on v2:172.21.15.83:6836/1559222491 2024-04-17T10:49:50.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.153+0000 7f51c2932000 1 --2- 172.21.15.83:0/2492872781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eef6ae99d0 0x55eef6b4aa60 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.153+0000 7f51c2932000 1 --2- 172.21.15.83:0/2492872781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55eef6aa00a0 0x55eef6b4a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.153+0000 7f51c2932000 1 -- 172.21.15.83:0/2492872781 >> 172.21.15.83:0/2492872781 conn(0x55eef6a9c020 msgr2=0x55eef6ae8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:50.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.153+0000 7f51c2932000 1 -- 172.21.15.83:0/2492872781 shutdown_connections 2024-04-17T10:49:50.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.153+0000 7f51c2932000 1 -- 172.21.15.83:0/2492872781 wait complete. 2024-04-17T10:49:50.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.153+0000 7f51c2932000 20 client.9435 trim_cache size 0 max 16384 2024-04-17T10:49:50.152 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:49:50.152 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-17T10:49:50.152 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-17T10:49:50.152 INFO:teuthology.orchestra.run.smithi083.stdout: "678c1d4a-835d-4af3-b1c0-8ce40445977f": { 2024-04-17T10:49:50.152 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/scrub_control_test_path", 2024-04-17T10:49:50.152 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "678c1d4a-835d-4af3-b1c0-8ce40445977f", 2024-04-17T10:49:50.152 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive" 2024-04-17T10:49:50.152 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:49:50.152 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:49:50.152 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:49:50.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.153+0000 7f51bf7c5640 1 -- 172.21.15.83:0/3117895012 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f519c066f60 msgr2=0x7f519c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:50.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.153+0000 7f51bf7c5640 1 --2- 172.21.15.83:0/3117895012 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f519c066f60 0x7f519c069420 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f51a8006e90 tx=0x7f51a8008040 comp rx=0 tx=0).stop 2024-04-17T10:49:50.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.153+0000 7f51bf7c5640 1 -- 172.21.15.83:0/3117895012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51a4005c40 msgr2=0x7f51b814d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:50.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.153+0000 7f51bf7c5640 1 --2- 172.21.15.83:0/3117895012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51a4005c40 0x7f51b814d090 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f51b401c8a0 tx=0x7f51b4019810 comp rx=0 tx=0).stop 2024-04-17T10:49:50.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.153+0000 7f51bf7c5640 1 -- 172.21.15.83:0/3117895012 shutdown_connections 2024-04-17T10:49:50.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.153+0000 7f51bf7c5640 1 --2- 172.21.15.83:0/3117895012 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f519c066f60 0x7f519c069420 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.153+0000 7f51bf7c5640 1 --2- 172.21.15.83:0/3117895012 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b8156060 0x7f51b80cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.153+0000 7f51bf7c5640 1 --2- 172.21.15.83:0/3117895012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51b8155be0 0x7f51b815f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.153+0000 7f51bf7c5640 1 --2- 172.21.15.83:0/3117895012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51a4005c40 0x7f51b814d090 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.153+0000 7f51bf7c5640 1 -- 172.21.15.83:0/3117895012 >> 172.21.15.83:0/3117895012 conn(0x7f51b80b15b0 msgr2=0x7f51b8070470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:50.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.153+0000 7f51bf7c5640 1 -- 172.21.15.83:0/3117895012 shutdown_connections 2024-04-17T10:49:50.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.153+0000 7f51bf7c5640 1 -- 172.21.15.83:0/3117895012 wait complete. 2024-04-17T10:49:50.160 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:49:50.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.277+0000 7f9ddfa21640 1 Processor -- start 2024-04-17T10:49:50.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.277+0000 7f9ddfa21640 1 -- start start 2024-04-17T10:49:50.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.277+0000 7f9ddfa21640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dd80aedd0 0x7f9dd806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.277+0000 7f9ddfa21640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9dd806dae0 0x7f9dd80b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.277+0000 7f9ddfa21640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dd815e440 con 0x7f9dd815db50 2024-04-17T10:49:50.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.277+0000 7f9ddfa21640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dd815e720 con 0x7f9dd806dae0 2024-04-17T10:49:50.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.277+0000 7f9ddfa21640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd815ea00 con 0x7f9dd80aedd0 2024-04-17T10:49:50.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.277+0000 7f9ddd7bd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9dd815db50 0x7f9dd8070470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54924/0 (socket says 172.21.15.83:54924) 2024-04-17T10:49:50.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.277+0000 7f9ddd7bd640 1 -- 172.21.15.83:0/490175824 learned_addr learned my addr 172.21.15.83:0/490175824 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:50.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.277+0000 7f9ddcfbc640 1 --2- 172.21.15.83:0/490175824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dd80aedd0 0x7f9dd806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.277+0000 7f9dddfbe640 1 --2- 172.21.15.83:0/490175824 >> v2:172.21.15.167:3300/0 conn(0x7f9dd806dae0 0x7f9dd80b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.277+0000 7f9dcffff640 1 -- 172.21.15.83:0/490175824 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 488511783 0 0) 0x7f9dd815e440 con 0x7f9dd815db50 2024-04-17T10:49:50.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.277+0000 7f9dcffff640 1 -- 172.21.15.83:0/490175824 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dc4003530 con 0x7f9dd815db50 2024-04-17T10:49:50.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9dcffff640 1 -- 172.21.15.83:0/490175824 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3914331409 0 0) 0x7f9dc4003530 con 0x7f9dd815db50 2024-04-17T10:49:50.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9dcffff640 1 -- 172.21.15.83:0/490175824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dd80aedd0 msgr2=0x7f9dd806d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:50.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9dcffff640 1 --2- 172.21.15.83:0/490175824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dd80aedd0 0x7f9dd806d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9dcffff640 1 -- 172.21.15.83:0/490175824 >> v2:172.21.15.167:3300/0 conn(0x7f9dd806dae0 msgr2=0x7f9dd80b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:50.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9dcffff640 1 --2- 172.21.15.83:0/490175824 >> v2:172.21.15.167:3300/0 conn(0x7f9dd806dae0 0x7f9dd80b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9dcffff640 1 -- 172.21.15.83:0/490175824 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd815ece0 con 0x7f9dd815db50 2024-04-17T10:49:50.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9dcffff640 1 -- 172.21.15.83:0/490175824 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f9dd4001680 con 0x7f9dd815db50 2024-04-17T10:49:50.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9dcffff640 1 -- 172.21.15.83:0/490175824 >> v1:172.21.15.83:6789/0 conn(0x7f9dd815db50 legacy=0x7f9dd8070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:50.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9dcffff640 1 --2- 172.21.15.83:0/490175824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dd80aedd0 0x7f9dc40039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9dcffff640 1 --2- 172.21.15.83:0/490175824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dc4003f40 0x7f9dc4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9dcffff640 1 --2- 172.21.15.83:0/490175824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dc4004d50 0x7f9dc40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9dcffff640 1 -- 172.21.15.83:0/490175824 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd815e720 con 0x7f9dc4003f40 2024-04-17T10:49:50.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9dcffff640 1 -- 172.21.15.83:0/490175824 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd815ea00 con 0x7f9dc4004d50 2024-04-17T10:49:50.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9dcffff640 1 -- 172.21.15.83:0/490175824 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc400dbe0 con 0x7f9dd80aedd0 2024-04-17T10:49:50.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9ddcfbc640 1 --2- 172.21.15.83:0/490175824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dc4003f40 0x7f9dc4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9ddd7bd640 1 --2- 172.21.15.83:0/490175824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dd80aedd0 0x7f9dc40039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9dddfbe640 1 --2- 172.21.15.83:0/490175824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dc4004d50 0x7f9dc40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9ddcfbc640 1 -- 172.21.15.83:0/490175824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dd80aedd0 msgr2=0x7f9dc40039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:50.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9ddcfbc640 1 --2- 172.21.15.83:0/490175824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dd80aedd0 0x7f9dc40039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9ddcfbc640 1 -- 172.21.15.83:0/490175824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dc4004d50 msgr2=0x7f9dc40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:50.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9ddcfbc640 1 --2- 172.21.15.83:0/490175824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dc4004d50 0x7f9dc40051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9ddcfbc640 1 -- 172.21.15.83:0/490175824 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc4003530 con 0x7f9dc4003f40 2024-04-17T10:49:50.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9ddd7bd640 1 --2- 172.21.15.83:0/490175824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dd80aedd0 0x7f9dc40039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:50.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9ddcfbc640 1 --2- 172.21.15.83:0/490175824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dc4003f40 0x7f9dc4004810 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009ae0 tx=0x7f9dc801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dcaf8c319f96a3b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:50.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9dcffff640 1 -- 172.21.15.83:0/490175824 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc80b2070 con 0x7f9dc4003f40 2024-04-17T10:49:50.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9dcffff640 1 -- 172.21.15.83:0/490175824 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc80a5660 con 0x7f9dc4003f40 2024-04-17T10:49:50.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9dcffff640 1 -- 172.21.15.83:0/490175824 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc80096b0 con 0x7f9dc4003f40 2024-04-17T10:49:50.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9ddfa21640 1 -- 172.21.15.83:0/490175824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dc4003f40 msgr2=0x7f9dc4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:50.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9ddfa21640 1 --2- 172.21.15.83:0/490175824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dc4003f40 0x7f9dc4004810 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009ae0 tx=0x7f9dc801ca10 comp rx=0 tx=0).stop 2024-04-17T10:49:50.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9dddfbe640 1 -- 172.21.15.83:0/490175824 reap_dead start 2024-04-17T10:49:50.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9ddfa21640 1 -- 172.21.15.83:0/490175824 shutdown_connections 2024-04-17T10:49:50.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9ddfa21640 1 -- 172.21.15.83:0/490175824 >> 172.21.15.83:0/490175824 conn(0x7f9dd806abe0 msgr2=0x7f9dd806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:50.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9ddfa21640 1 -- 172.21.15.83:0/490175824 shutdown_connections 2024-04-17T10:49:50.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.281+0000 7f9ddfa21640 1 -- 172.21.15.83:0/490175824 wait complete. 2024-04-17T10:49:50.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.285+0000 7f9ddfa21640 1 Processor -- start 2024-04-17T10:49:50.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.285+0000 7f9ddfa21640 1 -- start start 2024-04-17T10:49:50.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.285+0000 7f9ddfa21640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dc4003f40 0x7f9dd814a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.285+0000 7f9ddfa21640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dd80af930 0x7f9dd80afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.285+0000 7f9ddfa21640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dd8157ea0 0x7f9dd8158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.285+0000 7f9ddfa21640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd815e0b0 con 0x7f9dd8157ea0 2024-04-17T10:49:50.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.285+0000 7f9ddfa21640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd815df30 con 0x7f9dd80af930 2024-04-17T10:49:50.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.285+0000 7f9ddfa21640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd80b3350 con 0x7f9dc4003f40 2024-04-17T10:49:50.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.285+0000 7f9dddfbe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dd8157ea0 0x7f9dd8158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.285+0000 7f9dddfbe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dd8157ea0 0x7f9dd8158320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47990/0 (socket says 172.21.15.83:47990) 2024-04-17T10:49:50.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.285+0000 7f9dddfbe640 1 -- 172.21.15.83:0/3592180924 learned_addr learned my addr 172.21.15.83:0/3592180924 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:50.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.285+0000 7f9ddcfbc640 1 --2- 172.21.15.83:0/3592180924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dd80af930 0x7f9dd80afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.285+0000 7f9ddd7bd640 1 --2- 172.21.15.83:0/3592180924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dc4003f40 0x7f9dd814a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.285+0000 7f9dddfbe640 1 -- 172.21.15.83:0/3592180924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dc4003f40 msgr2=0x7f9dd814a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:50.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.285+0000 7f9dddfbe640 1 --2- 172.21.15.83:0/3592180924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dc4003f40 0x7f9dd814a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.285+0000 7f9dddfbe640 1 -- 172.21.15.83:0/3592180924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dd80af930 msgr2=0x7f9dd80afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:50.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.285+0000 7f9dddfbe640 1 --2- 172.21.15.83:0/3592180924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dd80af930 0x7f9dd80afdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.285+0000 7f9dddfbe640 1 -- 172.21.15.83:0/3592180924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd8157780 con 0x7f9dd8157ea0 2024-04-17T10:49:50.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.285+0000 7f9dddfbe640 1 --2- 172.21.15.83:0/3592180924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dd8157ea0 0x7f9dd8158320 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f9dd80d0610 tx=0x7f9dd000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:50.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.285+0000 7f9dcdffb640 1 -- 172.21.15.83:0/3592180924 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd001c070 con 0x7f9dd8157ea0 2024-04-17T10:49:50.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.285+0000 7f9dcdffb640 1 -- 172.21.15.83:0/3592180924 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd0017400 con 0x7f9dd8157ea0 2024-04-17T10:49:50.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.285+0000 7f9ddfa21640 1 -- 172.21.15.83:0/3592180924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dd8158ab0 con 0x7f9dd8157ea0 2024-04-17T10:49:50.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.285+0000 7f9dcdffb640 1 -- 172.21.15.83:0/3592180924 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd00177f0 con 0x7f9dd8157ea0 2024-04-17T10:49:50.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.285+0000 7f9ddfa21640 1 -- 172.21.15.83:0/3592180924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd8159070 con 0x7f9dd8157ea0 2024-04-17T10:49:50.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.285+0000 7f9ddfa21640 1 -- 172.21.15.83:0/3592180924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dd8074f90 con 0x7f9dd8157ea0 2024-04-17T10:49:50.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.285+0000 7f9dcdffb640 1 -- 172.21.15.83:0/3592180924 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9dd001f070 con 0x7f9dd8157ea0 2024-04-17T10:49:50.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.289+0000 7f9dcdffb640 1 --2- 172.21.15.83:0/3592180924 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9dbc066f60 0x7f9dbc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.289+0000 7f9dcdffb640 1 -- 172.21.15.83:0/3592180924 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9dd0085450 con 0x7f9dd8157ea0 2024-04-17T10:49:50.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.289+0000 7f9ddd7bd640 1 --2- 172.21.15.83:0/3592180924 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9dbc066f60 0x7f9dbc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.289+0000 7f9ddd7bd640 1 --2- 172.21.15.83:0/3592180924 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9dbc066f60 0x7f9dbc069420 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4001870 tx=0x7f9dd4001560 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:50.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.293+0000 7f9dcdffb640 1 -- 172.21.15.83:0/3592180924 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9dd8157780 con 0x7f9dd8157ea0 2024-04-17T10:49:50.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.485+0000 7f9ddfa21640 1 -- 172.21.15.83:0/3592180924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9dd806d070 con 0x7f9dd8157ea0 2024-04-17T10:49:50.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.485+0000 7f9dcdffb640 1 -- 172.21.15.83:0/3592180924 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f9dd0012070 con 0x7f9dd8157ea0 2024-04-17T10:49:50.479 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:50.479 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5942,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3888166846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3888166846},{"type":"v1","addr":"172.21.15.167:6833","nonce":3888166846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":6772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1167913722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1167913722},{"type":"v1","addr":"172.21.15.167:6835","nonce":1167913722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":9192,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3552333026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3552333026},{"type":"v1","addr":"172.21.15.83:6835","nonce":3552333026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:49:19.962680+0000","modified":"2024-04-17T10:49:21.996802+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9198":{"gid":9198,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1559222491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1559222491},{"type":"v1","addr":"172.21.15.83:6837","nonce":1559222491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-17T10:49:50.479 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 76 2024-04-17T10:49:50.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.489+0000 7f9ddfa21640 1 -- 172.21.15.83:0/3592180924 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9dbc066f60 msgr2=0x7f9dbc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:50.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.489+0000 7f9ddfa21640 1 --2- 172.21.15.83:0/3592180924 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9dbc066f60 0x7f9dbc069420 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4001870 tx=0x7f9dd4001560 comp rx=0 tx=0).stop 2024-04-17T10:49:50.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.489+0000 7f9ddfa21640 1 -- 172.21.15.83:0/3592180924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dd8157ea0 msgr2=0x7f9dd8158320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:50.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.489+0000 7f9ddfa21640 1 --2- 172.21.15.83:0/3592180924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dd8157ea0 0x7f9dd8158320 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f9dd80d0610 tx=0x7f9dd000f4a0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.489+0000 7f9ddfa21640 1 -- 172.21.15.83:0/3592180924 shutdown_connections 2024-04-17T10:49:50.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.489+0000 7f9ddfa21640 1 --2- 172.21.15.83:0/3592180924 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9dbc066f60 0x7f9dbc069420 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.489+0000 7f9ddfa21640 1 --2- 172.21.15.83:0/3592180924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dd8157ea0 0x7f9dd8158320 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.489+0000 7f9ddfa21640 1 --2- 172.21.15.83:0/3592180924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dd80af930 0x7f9dd80afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.489+0000 7f9ddfa21640 1 --2- 172.21.15.83:0/3592180924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dc4003f40 0x7f9dd814a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.489+0000 7f9ddfa21640 1 -- 172.21.15.83:0/3592180924 >> 172.21.15.83:0/3592180924 conn(0x7f9dd806abe0 msgr2=0x7f9dd80ae690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:50.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.489+0000 7f9ddfa21640 1 -- 172.21.15.83:0/3592180924 shutdown_connections 2024-04-17T10:49:50.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.489+0000 7f9ddfa21640 1 -- 172.21.15.83:0/3592180924 wait complete. 2024-04-17T10:49:50.497 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-17T10:49:50.497 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-17T10:49:50.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.645+0000 7f3f564b7640 1 Processor -- start 2024-04-17T10:49:50.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.645+0000 7f3f564b7640 1 -- start start 2024-04-17T10:49:50.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.645+0000 7f3f564b7640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3f500cf780 0x7f3f5006bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.645+0000 7f3f564b7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f5015f290 0x7f3f5006c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.645+0000 7f3f564b7640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f5006e980 con 0x7f3f5015eeb0 2024-04-17T10:49:50.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.645+0000 7f3f564b7640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f500cf030 con 0x7f3f500cf780 2024-04-17T10:49:50.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.645+0000 7f3f564b7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f500cdb90 con 0x7f3f5015f290 2024-04-17T10:49:50.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.645+0000 7f3f4ffff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3f5015eeb0 0x7f3f5006d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54926/0 (socket says 172.21.15.83:54926) 2024-04-17T10:49:50.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.645+0000 7f3f4ffff640 1 -- 172.21.15.83:0/808923665 learned_addr learned my addr 172.21.15.83:0/808923665 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:50.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.645+0000 7f3f54a54640 1 --2- 172.21.15.83:0/808923665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f5015f290 0x7f3f5006c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.645+0000 7f3f4effd640 1 -- 172.21.15.83:0/808923665 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 798651169 0 0) 0x7f3f5006e980 con 0x7f3f5015eeb0 2024-04-17T10:49:50.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.645+0000 7f3f4effd640 1 -- 172.21.15.83:0/808923665 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f38003530 con 0x7f3f5015eeb0 2024-04-17T10:49:50.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.645+0000 7f3f4f7fe640 1 --2- 172.21.15.83:0/808923665 >> v2:172.21.15.167:3300/0 conn(0x7f3f500cf780 0x7f3f5006bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.645+0000 7f3f4effd640 1 -- 172.21.15.83:0/808923665 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3752425454 0 0) 0x7f3f38003530 con 0x7f3f5015eeb0 2024-04-17T10:49:50.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.645+0000 7f3f4effd640 1 -- 172.21.15.83:0/808923665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f5015f290 msgr2=0x7f3f5006c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:50.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.645+0000 7f3f4effd640 1 --2- 172.21.15.83:0/808923665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f5015f290 0x7f3f5006c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.645+0000 7f3f4effd640 1 -- 172.21.15.83:0/808923665 >> v2:172.21.15.167:3300/0 conn(0x7f3f500cf780 msgr2=0x7f3f5006bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:50.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.645+0000 7f3f4effd640 1 --2- 172.21.15.83:0/808923665 >> v2:172.21.15.167:3300/0 conn(0x7f3f500cf780 0x7f3f5006bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.645+0000 7f3f4effd640 1 -- 172.21.15.83:0/808923665 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5006e0e0 con 0x7f3f5015eeb0 2024-04-17T10:49:50.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.645+0000 7f3f4effd640 1 -- 172.21.15.83:0/808923665 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f3f40001680 con 0x7f3f5015eeb0 2024-04-17T10:49:50.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.645+0000 7f3f4effd640 1 -- 172.21.15.83:0/808923665 >> v1:172.21.15.83:6789/0 conn(0x7f3f5015eeb0 legacy=0x7f3f5006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:50.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.645+0000 7f3f4effd640 1 --2- 172.21.15.83:0/808923665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f5015f290 0x7f3f380039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.645+0000 7f3f4effd640 1 --2- 172.21.15.83:0/808923665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f38003f90 0x7f3f38004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.645+0000 7f3f4effd640 1 --2- 172.21.15.83:0/808923665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f38004d50 0x7f3f380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.645+0000 7f3f4effd640 1 -- 172.21.15.83:0/808923665 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f500cdb90 con 0x7f3f38004d50 2024-04-17T10:49:50.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.645+0000 7f3f4effd640 1 -- 172.21.15.83:0/808923665 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3800b9d0 con 0x7f3f38003f90 2024-04-17T10:49:50.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.645+0000 7f3f4effd640 1 -- 172.21.15.83:0/808923665 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3800bb50 con 0x7f3f5015f290 2024-04-17T10:49:50.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.645+0000 7f3f54a54640 1 --2- 172.21.15.83:0/808923665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f38004d50 0x7f3f380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.645+0000 7f3f4ffff640 1 --2- 172.21.15.83:0/808923665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f5015f290 0x7f3f380039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.649+0000 7f3f54a54640 1 -- 172.21.15.83:0/808923665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f5015f290 msgr2=0x7f3f380039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:50.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.649+0000 7f3f54a54640 1 --2- 172.21.15.83:0/808923665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f5015f290 0x7f3f380039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.649+0000 7f3f4f7fe640 1 --2- 172.21.15.83:0/808923665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f38003f90 0x7f3f38004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.649+0000 7f3f54a54640 1 -- 172.21.15.83:0/808923665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f38003f90 msgr2=0x7f3f38004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:50.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.649+0000 7f3f54a54640 1 --2- 172.21.15.83:0/808923665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f38003f90 0x7f3f38004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.649+0000 7f3f54a54640 1 -- 172.21.15.83:0/808923665 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f38003530 con 0x7f3f38004d50 2024-04-17T10:49:50.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.649+0000 7f3f54a54640 1 --2- 172.21.15.83:0/808923665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f38004d50 0x7f3f380051d0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f3f48009fc0 tx=0x7f3f4801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c66fffd4add212 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:50.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.649+0000 7f3f4effd640 1 -- 172.21.15.83:0/808923665 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f480b2070 con 0x7f3f38004d50 2024-04-17T10:49:50.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.649+0000 7f3f4effd640 1 -- 172.21.15.83:0/808923665 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f480a5c50 con 0x7f3f38004d50 2024-04-17T10:49:50.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.649+0000 7f3f4effd640 1 -- 172.21.15.83:0/808923665 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48009c10 con 0x7f3f38004d50 2024-04-17T10:49:50.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.649+0000 7f3f564b7640 1 -- 172.21.15.83:0/808923665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f38004d50 msgr2=0x7f3f380051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:50.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.649+0000 7f3f564b7640 1 --2- 172.21.15.83:0/808923665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f38004d50 0x7f3f380051d0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f3f48009fc0 tx=0x7f3f4801cc40 comp rx=0 tx=0).stop 2024-04-17T10:49:50.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.649+0000 7f3f54a54640 1 -- 172.21.15.83:0/808923665 reap_dead start 2024-04-17T10:49:50.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.649+0000 7f3f564b7640 1 -- 172.21.15.83:0/808923665 shutdown_connections 2024-04-17T10:49:50.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.649+0000 7f3f564b7640 1 -- 172.21.15.83:0/808923665 >> 172.21.15.83:0/808923665 conn(0x7f3f500cc910 msgr2=0x7f3f500ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:50.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.649+0000 7f3f564b7640 1 -- 172.21.15.83:0/808923665 shutdown_connections 2024-04-17T10:49:50.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.649+0000 7f3f564b7640 1 -- 172.21.15.83:0/808923665 wait complete. 2024-04-17T10:49:50.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.653+0000 7f3f564b7640 1 Processor -- start 2024-04-17T10:49:50.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.653+0000 7f3f564b7640 1 -- start start 2024-04-17T10:49:50.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.653+0000 7f3f564b7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f38004d50 0x7f3f500b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.653+0000 7f3f564b7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f501f15c0 0x7f3f5009e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.653+0000 7f3f564b7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f50070ef0 0x7f3f500713a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.653+0000 7f3f564b7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f50070bf0 con 0x7f3f38004d50 2024-04-17T10:49:50.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.653+0000 7f3f564b7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5006e6a0 con 0x7f3f501f15c0 2024-04-17T10:49:50.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.653+0000 7f3f564b7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f50070d70 con 0x7f3f50070ef0 2024-04-17T10:49:50.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.653+0000 7f3f4ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f38004d50 0x7f3f500b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.653+0000 7f3f4ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f38004d50 0x7f3f500b2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48016/0 (socket says 172.21.15.83:48016) 2024-04-17T10:49:50.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.653+0000 7f3f4ffff640 1 -- 172.21.15.83:0/2650585101 learned_addr learned my addr 172.21.15.83:0/2650585101 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:50.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.653+0000 7f3f54a54640 1 --2- 172.21.15.83:0/2650585101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f50070ef0 0x7f3f500713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.653+0000 7f3f4f7fe640 1 --2- 172.21.15.83:0/2650585101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f501f15c0 0x7f3f5009e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.653+0000 7f3f4ffff640 1 -- 172.21.15.83:0/2650585101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f50070ef0 msgr2=0x7f3f500713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:50.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.653+0000 7f3f4ffff640 1 --2- 172.21.15.83:0/2650585101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f50070ef0 0x7f3f500713a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.653+0000 7f3f4ffff640 1 -- 172.21.15.83:0/2650585101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f501f15c0 msgr2=0x7f3f5009e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:50.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.653+0000 7f3f4ffff640 1 --2- 172.21.15.83:0/2650585101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f501f15c0 0x7f3f5009e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.653+0000 7f3f4ffff640 1 -- 172.21.15.83:0/2650585101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5006d5d0 con 0x7f3f38004d50 2024-04-17T10:49:50.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.653+0000 7f3f4f7fe640 1 --2- 172.21.15.83:0/2650585101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f501f15c0 0x7f3f5009e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:49:50.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.653+0000 7f3f4ffff640 1 --2- 172.21.15.83:0/2650585101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f38004d50 0x7f3f500b2d40 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000e7f0 tx=0x7f3f4000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:50.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.653+0000 7f3f4cff9640 1 -- 172.21.15.83:0/2650585101 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4000c8d0 con 0x7f3f38004d50 2024-04-17T10:49:50.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.653+0000 7f3f4cff9640 1 -- 172.21.15.83:0/2650585101 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f40018070 con 0x7f3f38004d50 2024-04-17T10:49:50.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.653+0000 7f3f564b7640 1 -- 172.21.15.83:0/2650585101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f500a0ad0 con 0x7f3f38004d50 2024-04-17T10:49:50.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.653+0000 7f3f4cff9640 1 -- 172.21.15.83:0/2650585101 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4000ccb0 con 0x7f3f38004d50 2024-04-17T10:49:50.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.653+0000 7f3f564b7640 1 -- 172.21.15.83:0/2650585101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f500b3690 con 0x7f3f38004d50 2024-04-17T10:49:50.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.653+0000 7f3f564b7640 1 -- 172.21.15.83:0/2650585101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f500b3890 con 0x7f3f38004d50 2024-04-17T10:49:50.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.657+0000 7f3f4cff9640 1 -- 172.21.15.83:0/2650585101 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3f4001d460 con 0x7f3f38004d50 2024-04-17T10:49:50.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.657+0000 7f3f4cff9640 1 --2- 172.21.15.83:0/2650585101 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3f3c066f90 0x7f3f3c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:50.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.657+0000 7f3f4cff9640 1 -- 172.21.15.83:0/2650585101 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f40084da0 con 0x7f3f38004d50 2024-04-17T10:49:50.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.657+0000 7f3f4f7fe640 1 --2- 172.21.15.83:0/2650585101 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3f3c066f90 0x7f3f3c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:50.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.657+0000 7f3f4f7fe640 1 --2- 172.21.15.83:0/2650585101 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3f3c066f90 0x7f3f3c069450 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3f50071bb0 tx=0x7f3f44008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:50.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.661+0000 7f3f4cff9640 1 -- 172.21.15.83:0/2650585101 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3f500b3890 con 0x7f3f38004d50 2024-04-17T10:49:50.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.881+0000 7f3f564b7640 1 -- 172.21.15.83:0/2650585101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f3f50074f90 con 0x7f3f38004d50 2024-04-17T10:49:50.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.881+0000 7f3f4cff9640 1 -- 172.21.15.83:0/2650585101 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1446 (secure 0 0 0) 0x7f3f40055e40 con 0x7f3f38004d50 2024-04-17T10:49:50.873 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:50.873 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":723,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":127,"num_osds":8,"num_up_osds":8,"osd_up_since":1713350279,"num_in_osds":8,"osd_in_since":1713350270,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1039,"data_bytes":1067367509,"bytes_used":2594045952,"bytes_avail":770500067328,"bytes_total":773094113280,"write_bytes_sec":80762677,"read_op_per_sec":0,"write_op_per_sec":80},"fsmap":{"epoch":76,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":9198}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-04-17T10:49:08.374763+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-17T10:49:50.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.881+0000 7f3f564b7640 1 -- 172.21.15.83:0/2650585101 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3f3c066f90 msgr2=0x7f3f3c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:50.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.881+0000 7f3f564b7640 1 --2- 172.21.15.83:0/2650585101 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3f3c066f90 0x7f3f3c069450 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3f50071bb0 tx=0x7f3f44008040 comp rx=0 tx=0).stop 2024-04-17T10:49:50.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.881+0000 7f3f564b7640 1 -- 172.21.15.83:0/2650585101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f38004d50 msgr2=0x7f3f500b2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:50.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.881+0000 7f3f564b7640 1 --2- 172.21.15.83:0/2650585101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f38004d50 0x7f3f500b2d40 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000e7f0 tx=0x7f3f4000ecb0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.881+0000 7f3f564b7640 1 -- 172.21.15.83:0/2650585101 shutdown_connections 2024-04-17T10:49:50.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.881+0000 7f3f564b7640 1 --2- 172.21.15.83:0/2650585101 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3f3c066f90 0x7f3f3c069450 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.881+0000 7f3f564b7640 1 --2- 172.21.15.83:0/2650585101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f50070ef0 0x7f3f500713a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.881+0000 7f3f564b7640 1 --2- 172.21.15.83:0/2650585101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f501f15c0 0x7f3f5009e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.881+0000 7f3f564b7640 1 --2- 172.21.15.83:0/2650585101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f38004d50 0x7f3f500b2d40 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:50.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.881+0000 7f3f564b7640 1 -- 172.21.15.83:0/2650585101 >> 172.21.15.83:0/2650585101 conn(0x7f3f500cc910 msgr2=0x7f3f500b0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:50.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.881+0000 7f3f564b7640 1 -- 172.21.15.83:0/2650585101 shutdown_connections 2024-04-17T10:49:50.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:50.881+0000 7f3f564b7640 1 -- 172.21.15.83:0/2650585101 wait complete. 2024-04-17T10:49:50.892 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-17T10:49:51.893 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:49:52.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.021+0000 7f988c5ea640 1 Processor -- start 2024-04-17T10:49:52.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.021+0000 7f988c5ea640 1 -- start start 2024-04-17T10:49:52.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.025+0000 7f988c5ea640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f98840cfbe0 0x7f988406c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:52.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.025+0000 7f988c5ea640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f988406e320 con 0x7f988415ba30 2024-04-17T10:49:52.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.025+0000 7f988c5ea640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f988406e040 con 0x7f9884072330 2024-04-17T10:49:52.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.025+0000 7f988c5ea640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9884071ea0 con 0x7f98840cfbe0 2024-04-17T10:49:52.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.025+0000 7f988ab87640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f98840cfbe0 0x7f988406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:52.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.025+0000 7f988ab87640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f98840cfbe0 0x7f988406c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48028/0 (socket says 172.21.15.83:48028) 2024-04-17T10:49:52.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.025+0000 7f988a386640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f988415ba30 0x7f988415c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54932/0 (socket says 172.21.15.83:54932) 2024-04-17T10:49:52.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.025+0000 7f988ab87640 1 -- 172.21.15.83:0/174097387 learned_addr learned my addr 172.21.15.83:0/174097387 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:52.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.025+0000 7f988ab87640 1 -- 172.21.15.83:0/174097387 >> v1:172.21.15.167:6789/0 conn(0x7f9884072330 legacy=0x7f98840cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:52.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.025+0000 7f988ab87640 1 -- 172.21.15.83:0/174097387 >> v1:172.21.15.83:6789/0 conn(0x7f988415ba30 legacy=0x7f988415c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:52.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.025+0000 7f9889384640 1 -- 172.21.15.83:0/174097387 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 227867189 0 0) 0x7f988406e320 con 0x7f988415ba30 2024-04-17T10:49:52.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.025+0000 7f988ab87640 1 -- 172.21.15.83:0/174097387 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9884071850 con 0x7f98840cfbe0 2024-04-17T10:49:52.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.025+0000 7f988ab87640 1 --2- 172.21.15.83:0/174097387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98840cfbe0 0x7f988406c490 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f987c009e50 tx=0x7f987c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=389ab3c586b6c4f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:52.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.025+0000 7f9889384640 1 -- 172.21.15.83:0/174097387 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c0a5730 con 0x7f98840cfbe0 2024-04-17T10:49:52.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.029+0000 7f9889384640 1 -- 172.21.15.83:0/174097387 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987c0a58d0 con 0x7f98840cfbe0 2024-04-17T10:49:52.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.029+0000 7f9889384640 1 -- 172.21.15.83:0/174097387 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c0a5ca0 con 0x7f98840cfbe0 2024-04-17T10:49:52.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.029+0000 7f988c5ea640 1 -- 172.21.15.83:0/174097387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98840cfbe0 msgr2=0x7f988406c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:52.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.029+0000 7f988c5ea640 1 --2- 172.21.15.83:0/174097387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98840cfbe0 0x7f988406c490 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f987c009e50 tx=0x7f987c01c960 comp rx=0 tx=0).stop 2024-04-17T10:49:52.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.029+0000 7f988c5ea640 1 -- 172.21.15.83:0/174097387 shutdown_connections 2024-04-17T10:49:52.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.029+0000 7f988c5ea640 1 --2- 172.21.15.83:0/174097387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98840cfbe0 0x7f988406c490 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:52.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.029+0000 7f988c5ea640 1 -- 172.21.15.83:0/174097387 >> 172.21.15.83:0/174097387 conn(0x7f988406abe0 msgr2=0x7f988406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:52.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.029+0000 7f988c5ea640 1 -- 172.21.15.83:0/174097387 shutdown_connections 2024-04-17T10:49:52.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.029+0000 7f988c5ea640 1 -- 172.21.15.83:0/174097387 wait complete. 2024-04-17T10:49:52.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.029+0000 7f988c5ea640 1 Processor -- start 2024-04-17T10:49:52.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.029+0000 7f988c5ea640 1 -- start start 2024-04-17T10:49:52.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.029+0000 7f988c5ea640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9884072330 0x7f988414e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:52.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.029+0000 7f988c5ea640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98840cfbe0 0x7f9884144400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:52.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.029+0000 7f988c5ea640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f988415ba30 0x7f988415c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:52.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.029+0000 7f988c5ea640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98841696b0 con 0x7f9884072330 2024-04-17T10:49:52.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.029+0000 7f988c5ea640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9884169530 con 0x7f98840cfbe0 2024-04-17T10:49:52.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.029+0000 7f988c5ea640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9884169830 con 0x7f988415ba30 2024-04-17T10:49:52.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.029+0000 7f988a386640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9884072330 0x7f988414e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:52.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.029+0000 7f988a386640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9884072330 0x7f988414e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48034/0 (socket says 172.21.15.83:48034) 2024-04-17T10:49:52.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.029+0000 7f988a386640 1 -- 172.21.15.83:0/207737345 learned_addr learned my addr 172.21.15.83:0/207737345 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:52.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.029+0000 7f988a386640 1 -- 172.21.15.83:0/207737345 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f988415ba30 msgr2=0x7f988415c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:52.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.029+0000 7f988a386640 1 --2- 172.21.15.83:0/207737345 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f988415ba30 0x7f988415c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:52.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.029+0000 7f988a386640 1 -- 172.21.15.83:0/207737345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98840cfbe0 msgr2=0x7f9884144400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:52.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.029+0000 7f988a386640 1 --2- 172.21.15.83:0/207737345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98840cfbe0 0x7f9884144400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:52.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.029+0000 7f988a386640 1 -- 172.21.15.83:0/207737345 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98841521e0 con 0x7f9884072330 2024-04-17T10:49:52.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.029+0000 7f988a386640 1 --2- 172.21.15.83:0/207737345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9884072330 0x7f988414e140 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f988000c450 tx=0x7f988000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:52.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.029+0000 7f987affd640 1 -- 172.21.15.83:0/207737345 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880018070 con 0x7f9884072330 2024-04-17T10:49:52.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.029+0000 7f987affd640 1 -- 172.21.15.83:0/207737345 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98800074a0 con 0x7f9884072330 2024-04-17T10:49:52.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.029+0000 7f988c5ea640 1 -- 172.21.15.83:0/207737345 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98841524c0 con 0x7f9884072330 2024-04-17T10:49:52.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.029+0000 7f987affd640 1 -- 172.21.15.83:0/207737345 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880004a30 con 0x7f9884072330 2024-04-17T10:49:52.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.029+0000 7f988c5ea640 1 -- 172.21.15.83:0/207737345 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9884152a80 con 0x7f9884072330 2024-04-17T10:49:52.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.033+0000 7f988c5ea640 1 -- 172.21.15.83:0/207737345 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9884074f90 con 0x7f9884072330 2024-04-17T10:49:52.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.033+0000 7f987affd640 1 -- 172.21.15.83:0/207737345 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f988001b070 con 0x7f9884072330 2024-04-17T10:49:52.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.033+0000 7f987affd640 1 --2- 172.21.15.83:0/207737345 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9868066f60 0x7f9868069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:52.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.033+0000 7f987affd640 1 -- 172.21.15.83:0/207737345 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9880084250 con 0x7f9884072330 2024-04-17T10:49:52.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.033+0000 7f9889b85640 1 --2- 172.21.15.83:0/207737345 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9868066f60 0x7f9868069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:52.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.033+0000 7f9889b85640 1 --2- 172.21.15.83:0/207737345 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9868066f60 0x7f9868069420 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f9874007fd0 tx=0x7f98740079a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:52.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.037+0000 7f987affd640 1 -- 172.21.15.83:0/207737345 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9880055260 con 0x7f9884072330 2024-04-17T10:49:52.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.233+0000 7f988c5ea640 1 -- 172.21.15.83:0/207737345 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9884152c80 con 0x7f9884072330 2024-04-17T10:49:52.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.233+0000 7f987affd640 1 -- 172.21.15.83:0/207737345 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f9880054b60 con 0x7f9884072330 2024-04-17T10:49:52.225 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:52.226 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5942,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3888166846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3888166846},{"type":"v1","addr":"172.21.15.167:6833","nonce":3888166846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":6772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1167913722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1167913722},{"type":"v1","addr":"172.21.15.167:6835","nonce":1167913722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":9192,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3552333026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3552333026},{"type":"v1","addr":"172.21.15.83:6835","nonce":3552333026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:49:19.962680+0000","modified":"2024-04-17T10:49:21.996802+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9198":{"gid":9198,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1559222491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1559222491},{"type":"v1","addr":"172.21.15.83:6837","nonce":1559222491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-17T10:49:52.226 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 76 2024-04-17T10:49:52.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.233+0000 7f988c5ea640 1 -- 172.21.15.83:0/207737345 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9868066f60 msgr2=0x7f9868069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:52.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.233+0000 7f988c5ea640 1 --2- 172.21.15.83:0/207737345 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9868066f60 0x7f9868069420 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f9874007fd0 tx=0x7f98740079a0 comp rx=0 tx=0).stop 2024-04-17T10:49:52.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.233+0000 7f988c5ea640 1 -- 172.21.15.83:0/207737345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9884072330 msgr2=0x7f988414e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:52.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.233+0000 7f988c5ea640 1 --2- 172.21.15.83:0/207737345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9884072330 0x7f988414e140 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f988000c450 tx=0x7f988000c910 comp rx=0 tx=0).stop 2024-04-17T10:49:52.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.237+0000 7f988c5ea640 1 -- 172.21.15.83:0/207737345 shutdown_connections 2024-04-17T10:49:52.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.237+0000 7f988c5ea640 1 --2- 172.21.15.83:0/207737345 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9868066f60 0x7f9868069420 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:52.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.237+0000 7f988c5ea640 1 --2- 172.21.15.83:0/207737345 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f988415ba30 0x7f988415c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:52.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.237+0000 7f988c5ea640 1 --2- 172.21.15.83:0/207737345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98840cfbe0 0x7f9884144400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:52.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.237+0000 7f988c5ea640 1 --2- 172.21.15.83:0/207737345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9884072330 0x7f988414e140 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:52.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.237+0000 7f988c5ea640 1 -- 172.21.15.83:0/207737345 >> 172.21.15.83:0/207737345 conn(0x7f988406abe0 msgr2=0x7f9884154050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:52.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.237+0000 7f988c5ea640 1 -- 172.21.15.83:0/207737345 shutdown_connections 2024-04-17T10:49:52.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.237+0000 7f988c5ea640 1 -- 172.21.15.83:0/207737345 wait complete. 2024-04-17T10:49:52.244 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-17T10:49:52.245 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-17T10:49:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.365+0000 7fb7f4182640 1 Processor -- start 2024-04-17T10:49:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.365+0000 7fb7f4182640 1 -- start start 2024-04-17T10:49:52.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.365+0000 7fb7f4182640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb7ec0cfbe0 0x7fb7ec06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:52.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.365+0000 7fb7f4182640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7ec06e320 con 0x7fb7ec072330 2024-04-17T10:49:52.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.365+0000 7fb7f4182640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7ec06e040 con 0x7fb7ec15ba30 2024-04-17T10:49:52.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.365+0000 7fb7f4182640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7ec071ea0 con 0x7fb7ec0cfbe0 2024-04-17T10:49:52.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.365+0000 7fb7f271f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb7ec0cfbe0 0x7fb7ec06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:52.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.365+0000 7fb7f171d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb7ec072330 0x7fb7ec0cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54940/0 (socket says 172.21.15.83:54940) 2024-04-17T10:49:52.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.365+0000 7fb7f171d640 1 -- 172.21.15.83:0/3340540646 learned_addr learned my addr 172.21.15.83:0/3340540646 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:52.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.369+0000 7fb7f271f640 1 -- 172.21.15.83:0/3340540646 >> v1:172.21.15.167:6789/0 conn(0x7fb7ec15ba30 legacy=0x7fb7ec15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:52.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.369+0000 7fb7f271f640 1 -- 172.21.15.83:0/3340540646 >> v1:172.21.15.83:6789/0 conn(0x7fb7ec072330 legacy=0x7fb7ec0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:52.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.369+0000 7fb7f271f640 1 -- 172.21.15.83:0/3340540646 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7ec071850 con 0x7fb7ec0cfbe0 2024-04-17T10:49:52.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.369+0000 7fb7f271f640 1 --2- 172.21.15.83:0/3340540646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7ec0cfbe0 0x7fb7ec06c490 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009d90 tx=0x7fb7e401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad94370d05269fc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:52.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.369+0000 7fb7f0f1c640 1 -- 172.21.15.83:0/3340540646 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e40b2070 con 0x7fb7ec0cfbe0 2024-04-17T10:49:52.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.369+0000 7fb7f0f1c640 1 -- 172.21.15.83:0/3340540646 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e40a58b0 con 0x7fb7ec0cfbe0 2024-04-17T10:49:52.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.369+0000 7fb7f0f1c640 1 -- 172.21.15.83:0/3340540646 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e40a5c80 con 0x7fb7ec0cfbe0 2024-04-17T10:49:52.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.369+0000 7fb7f4182640 1 -- 172.21.15.83:0/3340540646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7ec0cfbe0 msgr2=0x7fb7ec06c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:52.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.369+0000 7fb7f4182640 1 --2- 172.21.15.83:0/3340540646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7ec0cfbe0 0x7fb7ec06c490 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009d90 tx=0x7fb7e401c8e0 comp rx=0 tx=0).stop 2024-04-17T10:49:52.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.369+0000 7fb7f4182640 1 -- 172.21.15.83:0/3340540646 shutdown_connections 2024-04-17T10:49:52.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.369+0000 7fb7f4182640 1 --2- 172.21.15.83:0/3340540646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7ec0cfbe0 0x7fb7ec06c490 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:52.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.369+0000 7fb7f4182640 1 -- 172.21.15.83:0/3340540646 >> 172.21.15.83:0/3340540646 conn(0x7fb7ec06abe0 msgr2=0x7fb7ec06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:52.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.369+0000 7fb7f4182640 1 -- 172.21.15.83:0/3340540646 shutdown_connections 2024-04-17T10:49:52.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.369+0000 7fb7f4182640 1 -- 172.21.15.83:0/3340540646 wait complete. 2024-04-17T10:49:52.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.373+0000 7fb7f4182640 1 Processor -- start 2024-04-17T10:49:52.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.373+0000 7fb7f4182640 1 -- start start 2024-04-17T10:49:52.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.373+0000 7fb7f4182640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb7ec072330 0x7fb7ec14a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:52.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.373+0000 7fb7f4182640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7ec0cfbe0 0x7fb7ec1359e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:52.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.373+0000 7fb7f4182640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7ec15ba30 0x7fb7ec146e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:52.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.373+0000 7fb7f4182640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ec1696b0 con 0x7fb7ec15ba30 2024-04-17T10:49:52.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.373+0000 7fb7f4182640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ec169530 con 0x7fb7ec0cfbe0 2024-04-17T10:49:52.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.373+0000 7fb7f4182640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ec169830 con 0x7fb7ec072330 2024-04-17T10:49:52.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.373+0000 7fb7f271f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7ec15ba30 0x7fb7ec146e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:52.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.373+0000 7fb7f271f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7ec15ba30 0x7fb7ec146e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48056/0 (socket says 172.21.15.83:48056) 2024-04-17T10:49:52.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.373+0000 7fb7f271f640 1 -- 172.21.15.83:0/2370850184 learned_addr learned my addr 172.21.15.83:0/2370850184 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:52.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.373+0000 7fb7f271f640 1 -- 172.21.15.83:0/2370850184 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb7ec072330 msgr2=0x7fb7ec14a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:52.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.373+0000 7fb7f271f640 1 --2- 172.21.15.83:0/2370850184 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb7ec072330 0x7fb7ec14a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:52.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.373+0000 7fb7f271f640 1 -- 172.21.15.83:0/2370850184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7ec0cfbe0 msgr2=0x7fb7ec1359e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:52.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.373+0000 7fb7f171d640 1 --2- 172.21.15.83:0/2370850184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7ec0cfbe0 0x7fb7ec1359e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:52.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.373+0000 7fb7f271f640 1 --2- 172.21.15.83:0/2370850184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7ec0cfbe0 0x7fb7ec1359e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:52.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.373+0000 7fb7f271f640 1 -- 172.21.15.83:0/2370850184 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7ec14a8b0 con 0x7fb7ec15ba30 2024-04-17T10:49:52.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.373+0000 7fb7f271f640 1 --2- 172.21.15.83:0/2370850184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7ec15ba30 0x7fb7ec146e60 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fb7e401a510 tx=0x7fb7e4019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:52.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.373+0000 7fb7e27fc640 1 -- 172.21.15.83:0/2370850184 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e40b2050 con 0x7fb7ec15ba30 2024-04-17T10:49:52.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.373+0000 7fb7e27fc640 1 -- 172.21.15.83:0/2370850184 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e40be070 con 0x7fb7ec15ba30 2024-04-17T10:49:52.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.373+0000 7fb7f4182640 1 -- 172.21.15.83:0/2370850184 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7ec14ab90 con 0x7fb7ec15ba30 2024-04-17T10:49:52.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.373+0000 7fb7e27fc640 1 -- 172.21.15.83:0/2370850184 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e40ada50 con 0x7fb7ec15ba30 2024-04-17T10:49:52.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.373+0000 7fb7f4182640 1 -- 172.21.15.83:0/2370850184 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7ec14b150 con 0x7fb7ec15ba30 2024-04-17T10:49:52.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.373+0000 7fb7f4182640 1 -- 172.21.15.83:0/2370850184 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7ec074f90 con 0x7fb7ec15ba30 2024-04-17T10:49:52.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.373+0000 7fb7e27fc640 1 -- 172.21.15.83:0/2370850184 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb7e40adbf0 con 0x7fb7ec15ba30 2024-04-17T10:49:52.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.373+0000 7fb7e27fc640 1 --2- 172.21.15.83:0/2370850184 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb7d0066f60 0x7fb7d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:52.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.377+0000 7fb7e27fc640 1 -- 172.21.15.83:0/2370850184 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb7e4120270 con 0x7fb7ec15ba30 2024-04-17T10:49:52.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.377+0000 7fb7f1f1e640 1 --2- 172.21.15.83:0/2370850184 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb7d0066f60 0x7fb7d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:52.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.377+0000 7fb7f1f1e640 1 --2- 172.21.15.83:0/2370850184 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb7d0066f60 0x7fb7d0069420 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8000f70 tx=0x7fb7e80079e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:52.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.381+0000 7fb7e27fc640 1 -- 172.21.15.83:0/2370850184 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb7e4016560 con 0x7fb7ec15ba30 2024-04-17T10:49:52.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.597+0000 7fb7f4182640 1 -- 172.21.15.83:0/2370850184 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb7ec15aee0 con 0x7fb7ec15ba30 2024-04-17T10:49:52.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.597+0000 7fb7e27fc640 1 -- 172.21.15.83:0/2370850184 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1446 (secure 0 0 0) 0x7fb7e40f1290 con 0x7fb7ec15ba30 2024-04-17T10:49:52.590 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:52.590 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":725,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":127,"num_osds":8,"num_up_osds":8,"osd_up_since":1713350279,"num_in_osds":8,"osd_in_since":1713350270,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1039,"data_bytes":1067367509,"bytes_used":2594045952,"bytes_avail":770500067328,"bytes_total":773094113280,"write_bytes_sec":80762677,"read_op_per_sec":0,"write_op_per_sec":80},"fsmap":{"epoch":76,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":9198}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-04-17T10:49:08.374763+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-17T10:49:52.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.601+0000 7fb7f4182640 1 -- 172.21.15.83:0/2370850184 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb7d0066f60 msgr2=0x7fb7d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:52.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.601+0000 7fb7f4182640 1 --2- 172.21.15.83:0/2370850184 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb7d0066f60 0x7fb7d0069420 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8000f70 tx=0x7fb7e80079e0 comp rx=0 tx=0).stop 2024-04-17T10:49:52.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.601+0000 7fb7f4182640 1 -- 172.21.15.83:0/2370850184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7ec15ba30 msgr2=0x7fb7ec146e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:52.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.601+0000 7fb7f4182640 1 --2- 172.21.15.83:0/2370850184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7ec15ba30 0x7fb7ec146e60 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fb7e401a510 tx=0x7fb7e4019680 comp rx=0 tx=0).stop 2024-04-17T10:49:52.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.601+0000 7fb7f4182640 1 -- 172.21.15.83:0/2370850184 shutdown_connections 2024-04-17T10:49:52.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.601+0000 7fb7f4182640 1 --2- 172.21.15.83:0/2370850184 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb7d0066f60 0x7fb7d0069420 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:52.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.601+0000 7fb7f4182640 1 --2- 172.21.15.83:0/2370850184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7ec15ba30 0x7fb7ec146e60 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:52.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.601+0000 7fb7f4182640 1 --2- 172.21.15.83:0/2370850184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7ec0cfbe0 0x7fb7ec1359e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:52.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.601+0000 7fb7f4182640 1 --2- 172.21.15.83:0/2370850184 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb7ec072330 0x7fb7ec14a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:52.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.601+0000 7fb7f4182640 1 -- 172.21.15.83:0/2370850184 >> 172.21.15.83:0/2370850184 conn(0x7fb7ec06abe0 msgr2=0x7fb7ec0ce990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:52.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.601+0000 7fb7f4182640 1 -- 172.21.15.83:0/2370850184 shutdown_connections 2024-04-17T10:49:52.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:52.601+0000 7fb7f4182640 1 -- 172.21.15.83:0/2370850184 wait complete. 2024-04-17T10:49:52.608 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-17T10:49:53.610 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:49:53.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.749+0000 7f0eea0fb640 1 Processor -- start 2024-04-17T10:49:53.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.749+0000 7f0eea0fb640 1 -- start start 2024-04-17T10:49:53.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.749+0000 7f0eea0fb640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0ee40b2d80 0x7f0ee4146e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:53.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.749+0000 7f0eea0fb640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0ee406d690 0x7f0ee41620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:53.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.749+0000 7f0eea0fb640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ee4147570 con 0x7f0ee4146a00 2024-04-17T10:49:53.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.749+0000 7f0eea0fb640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ee4162610 con 0x7f0ee406d690 2024-04-17T10:49:53.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.749+0000 7f0eea0fb640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ee406c960 con 0x7f0ee40b2d80 2024-04-17T10:49:53.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.749+0000 7f0ee3fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0ee406d690 0x7f0ee41620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:53.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.749+0000 7f0ee3fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0ee406d690 0x7f0ee41620d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48066/0 (socket says 172.21.15.83:48066) 2024-04-17T10:49:53.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.749+0000 7f0ee3fff640 1 -- 172.21.15.83:0/530826881 learned_addr learned my addr 172.21.15.83:0/530826881 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:53.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.749+0000 7f0ee3fff640 1 -- 172.21.15.83:0/530826881 >> v2:172.21.15.167:3300/0 conn(0x7f0ee40b2d80 msgr2=0x7f0ee4146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:53.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.749+0000 7f0ee3fff640 1 --2- 172.21.15.83:0/530826881 >> v2:172.21.15.167:3300/0 conn(0x7f0ee40b2d80 0x7f0ee4146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:53.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.749+0000 7f0ee3fff640 1 -- 172.21.15.83:0/530826881 >> v1:172.21.15.83:6789/0 conn(0x7f0ee4146a00 legacy=0x7f0ee40b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:53.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.749+0000 7f0ee3fff640 1 -- 172.21.15.83:0/530826881 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee406cbb0 con 0x7f0ee406d690 2024-04-17T10:49:53.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0ee3fff640 1 --2- 172.21.15.83:0/530826881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ee406d690 0x7f0ee41620d0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4067670 tx=0x7f0edc01ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faacf9a0f9c15ece server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:53.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0ee27fc640 1 -- 172.21.15.83:0/530826881 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc0a57f0 con 0x7f0ee406d690 2024-04-17T10:49:53.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0ee27fc640 1 -- 172.21.15.83:0/530826881 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0edc0a5990 con 0x7f0ee406d690 2024-04-17T10:49:53.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0ee27fc640 1 -- 172.21.15.83:0/530826881 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc0a5d60 con 0x7f0ee406d690 2024-04-17T10:49:53.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0eea0fb640 1 -- 172.21.15.83:0/530826881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ee406d690 msgr2=0x7f0ee41620d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:53.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0eea0fb640 1 --2- 172.21.15.83:0/530826881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ee406d690 0x7f0ee41620d0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4067670 tx=0x7f0edc01ca10 comp rx=0 tx=0).stop 2024-04-17T10:49:53.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0eea0fb640 1 -- 172.21.15.83:0/530826881 shutdown_connections 2024-04-17T10:49:53.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0eea0fb640 1 --2- 172.21.15.83:0/530826881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ee406d690 0x7f0ee41620d0 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:53.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0eea0fb640 1 --2- 172.21.15.83:0/530826881 >> v2:172.21.15.167:3300/0 conn(0x7f0ee40b2d80 0x7f0ee4146e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:53.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0eea0fb640 1 -- 172.21.15.83:0/530826881 >> 172.21.15.83:0/530826881 conn(0x7f0ee406abe0 msgr2=0x7f0ee406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:53.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0eea0fb640 1 -- 172.21.15.83:0/530826881 shutdown_connections 2024-04-17T10:49:53.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0eea0fb640 1 -- 172.21.15.83:0/530826881 wait complete. 2024-04-17T10:49:53.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0eea0fb640 1 Processor -- start 2024-04-17T10:49:53.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0eea0fb640 1 -- start start 2024-04-17T10:49:53.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0eea0fb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ee406d690 0x7f0ee413b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:53.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0eea0fb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ee40b2d80 0x7f0ee415faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0eea0fb640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ee4146a00 0x7f0ee415b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0eea0fb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee4071010 con 0x7f0ee406d690 2024-04-17T10:49:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0eea0fb640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee4070e90 con 0x7f0ee40b2d80 2024-04-17T10:49:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0eea0fb640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee413ee50 con 0x7f0ee4146a00 2024-04-17T10:49:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0ee37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ee406d690 0x7f0ee413b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0ee37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ee406d690 0x7f0ee413b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48074/0 (socket says 172.21.15.83:48074) 2024-04-17T10:49:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0ee37fe640 1 -- 172.21.15.83:0/1486454338 learned_addr learned my addr 172.21.15.83:0/1486454338 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0ee2ffd640 1 --2- 172.21.15.83:0/1486454338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ee40b2d80 0x7f0ee415faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0ee37fe640 1 -- 172.21.15.83:0/1486454338 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ee4146a00 msgr2=0x7f0ee415b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0ee37fe640 1 --2- 172.21.15.83:0/1486454338 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ee4146a00 0x7f0ee415b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0ee37fe640 1 -- 172.21.15.83:0/1486454338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ee40b2d80 msgr2=0x7f0ee415faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0ee37fe640 1 --2- 172.21.15.83:0/1486454338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ee40b2d80 0x7f0ee415faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0ee37fe640 1 -- 172.21.15.83:0/1486454338 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee4160340 con 0x7f0ee406d690 2024-04-17T10:49:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0ee37fe640 1 --2- 172.21.15.83:0/1486454338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ee406d690 0x7f0ee413b150 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f0ed400c370 tx=0x7f0ed400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:53.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0ecbfff640 1 -- 172.21.15.83:0/1486454338 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed40052d0 con 0x7f0ee406d690 2024-04-17T10:49:53.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0ecbfff640 1 -- 172.21.15.83:0/1486454338 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed4005470 con 0x7f0ee406d690 2024-04-17T10:49:53.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0eea0fb640 1 -- 172.21.15.83:0/1486454338 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee413e800 con 0x7f0ee406d690 2024-04-17T10:49:53.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.753+0000 7f0ecbfff640 1 -- 172.21.15.83:0/1486454338 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4007260 con 0x7f0ee406d690 2024-04-17T10:49:53.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.757+0000 7f0eea0fb640 1 -- 172.21.15.83:0/1486454338 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee4160030 con 0x7f0ee406d690 2024-04-17T10:49:53.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.757+0000 7f0eea0fb640 1 -- 172.21.15.83:0/1486454338 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ee4074f90 con 0x7f0ee406d690 2024-04-17T10:49:53.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.757+0000 7f0ecbfff640 1 -- 172.21.15.83:0/1486454338 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0ed4007400 con 0x7f0ee406d690 2024-04-17T10:49:53.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.757+0000 7f0ecbfff640 1 --2- 172.21.15.83:0/1486454338 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0ed0066f90 0x7f0ed0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:53.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.757+0000 7f0ecbfff640 1 -- 172.21.15.83:0/1486454338 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ed4084470 con 0x7f0ee406d690 2024-04-17T10:49:53.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.757+0000 7f0ee2ffd640 1 --2- 172.21.15.83:0/1486454338 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0ed0066f90 0x7f0ed0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:53.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.757+0000 7f0ee2ffd640 1 --2- 172.21.15.83:0/1486454338 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0ed0066f90 0x7f0ed0069450 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f0ee41f9bd0 tx=0x7f0ed8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:53.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.761+0000 7f0ecbfff640 1 -- 172.21.15.83:0/1486454338 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0ed4055480 con 0x7f0ee406d690 2024-04-17T10:49:53.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.957+0000 7f0eea0fb640 1 -- 172.21.15.83:0/1486454338 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ee40ae7c0 con 0x7f0ee406d690 2024-04-17T10:49:53.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.957+0000 7f0ecbfff640 1 -- 172.21.15.83:0/1486454338 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f0ed4054d80 con 0x7f0ee406d690 2024-04-17T10:49:53.950 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:53.950 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5942,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3888166846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3888166846},{"type":"v1","addr":"172.21.15.167:6833","nonce":3888166846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":6772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1167913722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1167913722},{"type":"v1","addr":"172.21.15.167:6835","nonce":1167913722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":9192,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3552333026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3552333026},{"type":"v1","addr":"172.21.15.83:6835","nonce":3552333026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:49:19.962680+0000","modified":"2024-04-17T10:49:21.996802+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9198":{"gid":9198,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1559222491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1559222491},{"type":"v1","addr":"172.21.15.83:6837","nonce":1559222491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-17T10:49:53.950 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 76 2024-04-17T10:49:53.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.961+0000 7f0eea0fb640 1 -- 172.21.15.83:0/1486454338 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0ed0066f90 msgr2=0x7f0ed0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:53.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.961+0000 7f0eea0fb640 1 --2- 172.21.15.83:0/1486454338 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0ed0066f90 0x7f0ed0069450 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f0ee41f9bd0 tx=0x7f0ed8008040 comp rx=0 tx=0).stop 2024-04-17T10:49:53.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.961+0000 7f0eea0fb640 1 -- 172.21.15.83:0/1486454338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ee406d690 msgr2=0x7f0ee413b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:53.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.961+0000 7f0eea0fb640 1 --2- 172.21.15.83:0/1486454338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ee406d690 0x7f0ee413b150 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f0ed400c370 tx=0x7f0ed400c830 comp rx=0 tx=0).stop 2024-04-17T10:49:53.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.961+0000 7f0eea0fb640 1 -- 172.21.15.83:0/1486454338 shutdown_connections 2024-04-17T10:49:53.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.961+0000 7f0eea0fb640 1 --2- 172.21.15.83:0/1486454338 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0ed0066f90 0x7f0ed0069450 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:53.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.961+0000 7f0eea0fb640 1 --2- 172.21.15.83:0/1486454338 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ee4146a00 0x7f0ee415b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:53.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.961+0000 7f0eea0fb640 1 --2- 172.21.15.83:0/1486454338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ee40b2d80 0x7f0ee415faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:53.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.961+0000 7f0eea0fb640 1 --2- 172.21.15.83:0/1486454338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ee406d690 0x7f0ee413b150 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:53.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.961+0000 7f0eea0fb640 1 -- 172.21.15.83:0/1486454338 >> 172.21.15.83:0/1486454338 conn(0x7f0ee406abe0 msgr2=0x7f0ee41617f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:53.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.961+0000 7f0eea0fb640 1 -- 172.21.15.83:0/1486454338 shutdown_connections 2024-04-17T10:49:53.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:53.961+0000 7f0eea0fb640 1 -- 172.21.15.83:0/1486454338 wait complete. 2024-04-17T10:49:53.968 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-17T10:49:53.968 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-17T10:49:54.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.089+0000 7f6776782640 1 Processor -- start 2024-04-17T10:49:54.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.089+0000 7f6776782640 1 -- start start 2024-04-17T10:49:54.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.093+0000 7f6776782640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f677006c300 0x7f6770072ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:54.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.093+0000 7f6776782640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f677006cbf0 0x7f67700cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:54.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.093+0000 7f6776782640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f677006d080 con 0x7f677015a900 2024-04-17T10:49:54.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.093+0000 7f6776782640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6770071000 con 0x7f677006c300 2024-04-17T10:49:54.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.093+0000 7f6776782640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f677006ef90 con 0x7f677006cbf0 2024-04-17T10:49:54.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.093+0000 7f67677fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f677006c300 0x7f6770072ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:54.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.093+0000 7f676ffff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f677015a900 0x7f677006bbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54956/0 (socket says 172.21.15.83:54956) 2024-04-17T10:49:54.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.093+0000 7f67677fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f677006c300 0x7f6770072ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48084/0 (socket says 172.21.15.83:48084) 2024-04-17T10:49:54.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.093+0000 7f676ffff640 1 -- 172.21.15.83:0/910318525 learned_addr learned my addr 172.21.15.83:0/910318525 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:49:54.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.093+0000 7f6774d1f640 1 --2- 172.21.15.83:0/910318525 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f677006cbf0 0x7f67700cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:54.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.093+0000 7f67677fe640 1 -- 172.21.15.83:0/910318525 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f677006cbf0 msgr2=0x7f67700cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:54.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.093+0000 7f67677fe640 1 --2- 172.21.15.83:0/910318525 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f677006cbf0 0x7f67700cc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:54.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.093+0000 7f67677fe640 1 -- 172.21.15.83:0/910318525 >> v1:172.21.15.83:6789/0 conn(0x7f677015a900 legacy=0x7f677006bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:54.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.093+0000 7f67677fe640 1 -- 172.21.15.83:0/910318525 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f677006f270 con 0x7f677006c300 2024-04-17T10:49:54.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.093+0000 7f676f7fe640 1 -- 172.21.15.83:0/910318525 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2975172184 0 0) 0x7f677006d080 con 0x7f677015a900 2024-04-17T10:49:54.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.093+0000 7f6774d1f640 1 --2- 172.21.15.83:0/910318525 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f677006cbf0 0x7f67700cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:54.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.093+0000 7f67677fe640 1 --2- 172.21.15.83:0/910318525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f677006c300 0x7f6770072ab0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f67580097c0 tx=0x7f675801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f63b210f715ee1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:54.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.093+0000 7f676f7fe640 1 -- 172.21.15.83:0/910318525 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67580b2070 con 0x7f677006c300 2024-04-17T10:49:54.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.093+0000 7f676f7fe640 1 -- 172.21.15.83:0/910318525 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67580ad640 con 0x7f677006c300 2024-04-17T10:49:54.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.093+0000 7f676f7fe640 1 -- 172.21.15.83:0/910318525 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67580ada10 con 0x7f677006c300 2024-04-17T10:49:54.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f6776782640 1 -- 172.21.15.83:0/910318525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f677006c300 msgr2=0x7f6770072ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:54.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f6776782640 1 --2- 172.21.15.83:0/910318525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f677006c300 0x7f6770072ab0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f67580097c0 tx=0x7f675801c4a0 comp rx=0 tx=0).stop 2024-04-17T10:49:54.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f6776782640 1 -- 172.21.15.83:0/910318525 shutdown_connections 2024-04-17T10:49:54.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f6776782640 1 --2- 172.21.15.83:0/910318525 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f677006cbf0 0x7f67700cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:54.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f6776782640 1 --2- 172.21.15.83:0/910318525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f677006c300 0x7f6770072ab0 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:54.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f6776782640 1 -- 172.21.15.83:0/910318525 >> 172.21.15.83:0/910318525 conn(0x7f67700cc910 msgr2=0x7f67700ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:54.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f6776782640 1 -- 172.21.15.83:0/910318525 shutdown_connections 2024-04-17T10:49:54.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f6776782640 1 -- 172.21.15.83:0/910318525 wait complete. 2024-04-17T10:49:54.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f6776782640 1 Processor -- start 2024-04-17T10:49:54.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f6776782640 1 -- start start 2024-04-17T10:49:54.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f6776782640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f677006c300 0x7f67700b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:54.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f6776782640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f677006cbf0 0x7f67700c3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:54.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f6776782640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f677015a900 0x7f6770156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:54.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f6776782640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67700cb880 con 0x7f677006cbf0 2024-04-17T10:49:54.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f6776782640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67700cb700 con 0x7f677006c300 2024-04-17T10:49:54.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f6776782640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67700cba00 con 0x7f677015a900 2024-04-17T10:49:54.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f67677fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f677006cbf0 0x7f67700c3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:54.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f67677fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f677006cbf0 0x7f67700c3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48098/0 (socket says 172.21.15.83:48098) 2024-04-17T10:49:54.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f67677fe640 1 -- 172.21.15.83:0/1823429401 learned_addr learned my addr 172.21.15.83:0/1823429401 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:54.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f676ffff640 1 --2- 172.21.15.83:0/1823429401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f677006c300 0x7f67700b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:54.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f6774d1f640 1 --2- 172.21.15.83:0/1823429401 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f677015a900 0x7f6770156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:54.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f67677fe640 1 -- 172.21.15.83:0/1823429401 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f677015a900 msgr2=0x7f6770156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:54.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f67677fe640 1 --2- 172.21.15.83:0/1823429401 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f677015a900 0x7f6770156340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:54.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f67677fe640 1 -- 172.21.15.83:0/1823429401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f677006c300 msgr2=0x7f67700b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:54.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f67677fe640 1 --2- 172.21.15.83:0/1823429401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f677006c300 0x7f67700b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:54.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f67677fe640 1 -- 172.21.15.83:0/1823429401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f677015dd70 con 0x7f677006cbf0 2024-04-17T10:49:54.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f6774d1f640 1 --2- 172.21.15.83:0/1823429401 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f677015a900 0x7f6770156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:54.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f67677fe640 1 --2- 172.21.15.83:0/1823429401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f677006cbf0 0x7f67700c3de0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f675801a0d0 tx=0x7f6758019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:54.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f676d7fa640 1 -- 172.21.15.83:0/1823429401 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67580ba070 con 0x7f677006cbf0 2024-04-17T10:49:54.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f676d7fa640 1 -- 172.21.15.83:0/1823429401 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67580b6610 con 0x7f677006cbf0 2024-04-17T10:49:54.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f6776782640 1 -- 172.21.15.83:0/1823429401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f677009b380 con 0x7f677006cbf0 2024-04-17T10:49:54.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f676d7fa640 1 -- 172.21.15.83:0/1823429401 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67580b2040 con 0x7f677006cbf0 2024-04-17T10:49:54.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.097+0000 7f6776782640 1 -- 172.21.15.83:0/1823429401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f677009b5f0 con 0x7f677006cbf0 2024-04-17T10:49:54.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.101+0000 7f6776782640 1 -- 172.21.15.83:0/1823429401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6770074f90 con 0x7f677006cbf0 2024-04-17T10:49:54.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.101+0000 7f676d7fa640 1 -- 172.21.15.83:0/1823429401 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f67580bd070 con 0x7f677006cbf0 2024-04-17T10:49:54.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.101+0000 7f676d7fa640 1 --2- 172.21.15.83:0/1823429401 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6750066f60 0x7f6750069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:54.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.101+0000 7f676d7fa640 1 -- 172.21.15.83:0/1823429401 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f675811f1a0 con 0x7f677006cbf0 2024-04-17T10:49:54.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.101+0000 7f676ffff640 1 --2- 172.21.15.83:0/1823429401 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6750066f60 0x7f6750069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:54.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.101+0000 7f676ffff640 1 --2- 172.21.15.83:0/1823429401 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6750066f60 0x7f6750069420 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6760000f70 tx=0x7f6760007b80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:54.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.105+0000 7f676d7fa640 1 -- 172.21.15.83:0/1823429401 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6758016090 con 0x7f677006cbf0 2024-04-17T10:49:54.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.325+0000 7f6776782640 1 -- 172.21.15.83:0/1823429401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f677006b000 con 0x7f677006cbf0 2024-04-17T10:49:54.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.325+0000 7f676d7fa640 1 -- 172.21.15.83:0/1823429401 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1446 (secure 0 0 0) 0x7f67580f0120 con 0x7f677006cbf0 2024-04-17T10:49:54.319 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:54.319 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":727,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":127,"num_osds":8,"num_up_osds":8,"osd_up_since":1713350279,"num_in_osds":8,"osd_in_since":1713350270,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1039,"data_bytes":1067367509,"bytes_used":2594045952,"bytes_avail":770500067328,"bytes_total":773094113280,"write_bytes_sec":52249507,"read_op_per_sec":0,"write_op_per_sec":52},"fsmap":{"epoch":76,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":9198}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-04-17T10:49:08.374763+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-17T10:49:54.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.329+0000 7f6776782640 1 -- 172.21.15.83:0/1823429401 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6750066f60 msgr2=0x7f6750069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:54.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.329+0000 7f6776782640 1 --2- 172.21.15.83:0/1823429401 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6750066f60 0x7f6750069420 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6760000f70 tx=0x7f6760007b80 comp rx=0 tx=0).stop 2024-04-17T10:49:54.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.329+0000 7f6776782640 1 -- 172.21.15.83:0/1823429401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f677006cbf0 msgr2=0x7f67700c3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:54.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.329+0000 7f6776782640 1 --2- 172.21.15.83:0/1823429401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f677006cbf0 0x7f67700c3de0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f675801a0d0 tx=0x7f6758019240 comp rx=0 tx=0).stop 2024-04-17T10:49:54.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.329+0000 7f6776782640 1 -- 172.21.15.83:0/1823429401 shutdown_connections 2024-04-17T10:49:54.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.329+0000 7f6776782640 1 --2- 172.21.15.83:0/1823429401 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6750066f60 0x7f6750069420 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:54.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.329+0000 7f6776782640 1 --2- 172.21.15.83:0/1823429401 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f677015a900 0x7f6770156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:54.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.329+0000 7f6776782640 1 --2- 172.21.15.83:0/1823429401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f677006cbf0 0x7f67700c3de0 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:54.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.329+0000 7f6776782640 1 --2- 172.21.15.83:0/1823429401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f677006c300 0x7f67700b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:54.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.329+0000 7f6776782640 1 -- 172.21.15.83:0/1823429401 >> 172.21.15.83:0/1823429401 conn(0x7f67700cc910 msgr2=0x7f677015d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:54.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.329+0000 7f6776782640 1 -- 172.21.15.83:0/1823429401 shutdown_connections 2024-04-17T10:49:54.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:54.329+0000 7f6776782640 1 -- 172.21.15.83:0/1823429401 wait complete. 2024-04-17T10:49:54.339 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-17T10:49:55.339 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:49:55.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.477+0000 7f1a13ac2640 1 Processor -- start 2024-04-17T10:49:55.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.477+0000 7f1a13ac2640 1 -- start start 2024-04-17T10:49:55.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.477+0000 7f1a13ac2640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a0c15ba30 0x7f1a0c072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:55.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.477+0000 7f1a13ac2640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1a0c15c960 0x7f1a0c0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:55.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.477+0000 7f1a13ac2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1a0c0cfa10 0x7f1a0c06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:55.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.477+0000 7f1a13ac2640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a0c071ea0 con 0x7f1a0c0cfa10 2024-04-17T10:49:55.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.477+0000 7f1a13ac2640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a0c0cfea0 con 0x7f1a0c15c960 2024-04-17T10:49:55.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.477+0000 7f1a13ac2640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c06e320 con 0x7f1a0c15ba30 2024-04-17T10:49:55.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.477+0000 7f1a1205f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1a0c0cfa10 0x7f1a0c06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:55.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.477+0000 7f1a1205f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1a0c0cfa10 0x7f1a0c06c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48102/0 (socket says 172.21.15.83:48102) 2024-04-17T10:49:55.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.477+0000 7f1a1205f640 1 -- 172.21.15.83:0/2623833416 learned_addr learned my addr 172.21.15.83:0/2623833416 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:55.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.477+0000 7f1a1105d640 1 --2- 172.21.15.83:0/2623833416 >> v2:172.21.15.167:3300/0 conn(0x7f1a0c15c960 0x7f1a0c0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:55.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.477+0000 7f1a1185e640 1 --2- 172.21.15.83:0/2623833416 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a0c15ba30 0x7f1a0c072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:55.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.477+0000 7f1a1205f640 1 -- 172.21.15.83:0/2623833416 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a0c15ba30 msgr2=0x7f1a0c072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:55.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.477+0000 7f1a1205f640 1 --2- 172.21.15.83:0/2623833416 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a0c15ba30 0x7f1a0c072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:55.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.477+0000 7f1a1205f640 1 -- 172.21.15.83:0/2623833416 >> v2:172.21.15.167:3300/0 conn(0x7f1a0c15c960 msgr2=0x7f1a0c0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:55.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.477+0000 7f1a1205f640 1 --2- 172.21.15.83:0/2623833416 >> v2:172.21.15.167:3300/0 conn(0x7f1a0c15c960 0x7f1a0c0cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:55.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.477+0000 7f1a1205f640 1 -- 172.21.15.83:0/2623833416 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0c06e040 con 0x7f1a0c0cfa10 2024-04-17T10:49:55.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.477+0000 7f1a1105d640 1 --2- 172.21.15.83:0/2623833416 >> v2:172.21.15.167:3300/0 conn(0x7f1a0c15c960 0x7f1a0c0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:49:55.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.481+0000 7f1a1205f640 1 --2- 172.21.15.83:0/2623833416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a0c0cfa10 0x7f1a0c06c490 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c070060 tx=0x7f1a0401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b7a74349add1429 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:55.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.481+0000 7f1a1085c640 1 -- 172.21.15.83:0/2623833416 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a040a56d0 con 0x7f1a0c0cfa10 2024-04-17T10:49:55.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.481+0000 7f1a1085c640 1 -- 172.21.15.83:0/2623833416 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a040a5870 con 0x7f1a0c0cfa10 2024-04-17T10:49:55.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.481+0000 7f1a1085c640 1 -- 172.21.15.83:0/2623833416 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a040a5c40 con 0x7f1a0c0cfa10 2024-04-17T10:49:55.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.481+0000 7f1a13ac2640 1 -- 172.21.15.83:0/2623833416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a0c0cfa10 msgr2=0x7f1a0c06c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:55.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.481+0000 7f1a13ac2640 1 --2- 172.21.15.83:0/2623833416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a0c0cfa10 0x7f1a0c06c490 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c070060 tx=0x7f1a0401c960 comp rx=0 tx=0).stop 2024-04-17T10:49:55.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.481+0000 7f1a13ac2640 1 -- 172.21.15.83:0/2623833416 shutdown_connections 2024-04-17T10:49:55.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.481+0000 7f1a13ac2640 1 --2- 172.21.15.83:0/2623833416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a0c0cfa10 0x7f1a0c06c490 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:55.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.481+0000 7f1a13ac2640 1 --2- 172.21.15.83:0/2623833416 >> v2:172.21.15.167:3300/0 conn(0x7f1a0c15c960 0x7f1a0c0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:55.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.481+0000 7f1a13ac2640 1 --2- 172.21.15.83:0/2623833416 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a0c15ba30 0x7f1a0c072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:55.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.481+0000 7f1a13ac2640 1 -- 172.21.15.83:0/2623833416 >> 172.21.15.83:0/2623833416 conn(0x7f1a0c06abe0 msgr2=0x7f1a0c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:55.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.481+0000 7f1a13ac2640 1 -- 172.21.15.83:0/2623833416 shutdown_connections 2024-04-17T10:49:55.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.481+0000 7f1a13ac2640 1 -- 172.21.15.83:0/2623833416 wait complete. 2024-04-17T10:49:55.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.485+0000 7f1a13ac2640 1 Processor -- start 2024-04-17T10:49:55.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.485+0000 7f1a13ac2640 1 -- start start 2024-04-17T10:49:55.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.485+0000 7f1a13ac2640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a0c0cfa10 0x7f1a0c153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:55.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.485+0000 7f1a13ac2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a0c15ba30 0x7f1a0c1485d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:55.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.485+0000 7f1a13ac2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a0c15c960 0x7f1a0c155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:55.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.485+0000 7f1a13ac2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c06de20 con 0x7f1a0c15ba30 2024-04-17T10:49:55.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.485+0000 7f1a13ac2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c06dca0 con 0x7f1a0c15c960 2024-04-17T10:49:55.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.485+0000 7f1a13ac2640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c15cd40 con 0x7f1a0c0cfa10 2024-04-17T10:49:55.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.485+0000 7f1a1105d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a0c15ba30 0x7f1a0c1485d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:55.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.485+0000 7f1a1105d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a0c15ba30 0x7f1a0c1485d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48106/0 (socket says 172.21.15.83:48106) 2024-04-17T10:49:55.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.485+0000 7f1a1105d640 1 -- 172.21.15.83:0/1987131634 learned_addr learned my addr 172.21.15.83:0/1987131634 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:55.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.485+0000 7f1a1205f640 1 --2- 172.21.15.83:0/1987131634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a0c15c960 0x7f1a0c155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:55.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.485+0000 7f1a1185e640 1 --2- 172.21.15.83:0/1987131634 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a0c0cfa10 0x7f1a0c153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:55.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.485+0000 7f1a1105d640 1 -- 172.21.15.83:0/1987131634 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a0c0cfa10 msgr2=0x7f1a0c153d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:55.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.485+0000 7f1a1105d640 1 --2- 172.21.15.83:0/1987131634 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a0c0cfa10 0x7f1a0c153d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:55.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.485+0000 7f1a1105d640 1 -- 172.21.15.83:0/1987131634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a0c15c960 msgr2=0x7f1a0c155550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:55.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.485+0000 7f1a1105d640 1 --2- 172.21.15.83:0/1987131634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a0c15c960 0x7f1a0c155550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:55.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.485+0000 7f1a1105d640 1 -- 172.21.15.83:0/1987131634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0c154290 con 0x7f1a0c15ba30 2024-04-17T10:49:55.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.485+0000 7f1a1205f640 1 --2- 172.21.15.83:0/1987131634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a0c15c960 0x7f1a0c155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:55.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.485+0000 7f1a1105d640 1 --2- 172.21.15.83:0/1987131634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a0c15ba30 0x7f1a0c1485d0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f19fc00d800 tx=0x7f19fc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:55.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.485+0000 7f1a027fc640 1 -- 172.21.15.83:0/1987131634 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc018020 con 0x7f1a0c15ba30 2024-04-17T10:49:55.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.485+0000 7f1a027fc640 1 -- 172.21.15.83:0/1987131634 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19fc0096c0 con 0x7f1a0c15ba30 2024-04-17T10:49:55.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.485+0000 7f1a13ac2640 1 -- 172.21.15.83:0/1987131634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a0c154530 con 0x7f1a0c15ba30 2024-04-17T10:49:55.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.485+0000 7f1a027fc640 1 -- 172.21.15.83:0/1987131634 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc009ab0 con 0x7f1a0c15ba30 2024-04-17T10:49:55.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.485+0000 7f1a13ac2640 1 -- 172.21.15.83:0/1987131634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a0c14b930 con 0x7f1a0c15ba30 2024-04-17T10:49:55.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.485+0000 7f1a13ac2640 1 -- 172.21.15.83:0/1987131634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a0c074f90 con 0x7f1a0c15ba30 2024-04-17T10:49:55.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.485+0000 7f1a027fc640 1 -- 172.21.15.83:0/1987131634 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f19fc01b020 con 0x7f1a0c15ba30 2024-04-17T10:49:55.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.489+0000 7f1a027fc640 1 --2- 172.21.15.83:0/1987131634 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f19f0066f60 0x7f19f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:55.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.489+0000 7f1a1185e640 1 --2- 172.21.15.83:0/1987131634 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f19f0066f60 0x7f19f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:55.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.489+0000 7f1a027fc640 1 -- 172.21.15.83:0/1987131634 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f19fc085690 con 0x7f1a0c15ba30 2024-04-17T10:49:55.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.489+0000 7f1a1185e640 1 --2- 172.21.15.83:0/1987131634 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f19f0066f60 0x7f19f0069420 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1a08009670 tx=0x7f1a08008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:55.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.493+0000 7f1a027fc640 1 -- 172.21.15.83:0/1987131634 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f19fc022020 con 0x7f1a0c15ba30 2024-04-17T10:49:55.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.693+0000 7f1a13ac2640 1 -- 172.21.15.83:0/1987131634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a0c0b24f0 con 0x7f1a0c15ba30 2024-04-17T10:49:55.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.697+0000 7f1a027fc640 1 -- 172.21.15.83:0/1987131634 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f19fc056530 con 0x7f1a0c15ba30 2024-04-17T10:49:55.688 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:55.688 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5942,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3888166846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3888166846},{"type":"v1","addr":"172.21.15.167:6833","nonce":3888166846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":6772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1167913722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1167913722},{"type":"v1","addr":"172.21.15.167:6835","nonce":1167913722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":9192,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3552333026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3552333026},{"type":"v1","addr":"172.21.15.83:6835","nonce":3552333026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:49:19.962680+0000","modified":"2024-04-17T10:49:21.996802+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9198":{"gid":9198,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1559222491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1559222491},{"type":"v1","addr":"172.21.15.83:6837","nonce":1559222491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-17T10:49:55.689 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 76 2024-04-17T10:49:55.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.697+0000 7f1a13ac2640 1 -- 172.21.15.83:0/1987131634 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f19f0066f60 msgr2=0x7f19f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:55.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.697+0000 7f1a13ac2640 1 --2- 172.21.15.83:0/1987131634 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f19f0066f60 0x7f19f0069420 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1a08009670 tx=0x7f1a08008040 comp rx=0 tx=0).stop 2024-04-17T10:49:55.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.697+0000 7f1a13ac2640 1 -- 172.21.15.83:0/1987131634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a0c15ba30 msgr2=0x7f1a0c1485d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:55.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.697+0000 7f1a13ac2640 1 --2- 172.21.15.83:0/1987131634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a0c15ba30 0x7f1a0c1485d0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f19fc00d800 tx=0x7f19fc00dcc0 comp rx=0 tx=0).stop 2024-04-17T10:49:55.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.697+0000 7f1a13ac2640 1 -- 172.21.15.83:0/1987131634 shutdown_connections 2024-04-17T10:49:55.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.697+0000 7f1a13ac2640 1 --2- 172.21.15.83:0/1987131634 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f19f0066f60 0x7f19f0069420 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:55.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.697+0000 7f1a13ac2640 1 --2- 172.21.15.83:0/1987131634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a0c15c960 0x7f1a0c155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:55.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.697+0000 7f1a13ac2640 1 --2- 172.21.15.83:0/1987131634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a0c15ba30 0x7f1a0c1485d0 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:55.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.697+0000 7f1a13ac2640 1 --2- 172.21.15.83:0/1987131634 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a0c0cfa10 0x7f1a0c153d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:55.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.697+0000 7f1a13ac2640 1 -- 172.21.15.83:0/1987131634 >> 172.21.15.83:0/1987131634 conn(0x7f1a0c06abe0 msgr2=0x7f1a0c073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:55.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.697+0000 7f1a13ac2640 1 -- 172.21.15.83:0/1987131634 shutdown_connections 2024-04-17T10:49:55.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.697+0000 7f1a13ac2640 1 -- 172.21.15.83:0/1987131634 wait complete. 2024-04-17T10:49:55.707 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-17T10:49:55.707 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-17T10:49:55.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.829+0000 7fe876894640 1 Processor -- start 2024-04-17T10:49:55.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.829+0000 7fe876894640 1 -- start start 2024-04-17T10:49:55.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.829+0000 7fe876894640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe87015d940 0x7fe8700b5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:55.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.829+0000 7fe876894640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8700b27f0 0x7fe87006cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:55.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.829+0000 7fe876894640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe87006d2d0 0x7fe8700d0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:55.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.829+0000 7fe876894640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe87006e150 con 0x7fe87006d2d0 2024-04-17T10:49:55.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.829+0000 7fe876894640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe87006d740 con 0x7fe87015d940 2024-04-17T10:49:55.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.829+0000 7fe876894640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe87006abe0 con 0x7fe8700b27f0 2024-04-17T10:49:55.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.829+0000 7fe874e31640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe87006d2d0 0x7fe8700d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:55.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.829+0000 7fe874e31640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe87006d2d0 0x7fe8700d0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48122/0 (socket says 172.21.15.83:48122) 2024-04-17T10:49:55.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.829+0000 7fe874e31640 1 -- 172.21.15.83:0/1040198283 learned_addr learned my addr 172.21.15.83:0/1040198283 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:55.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.829+0000 7fe86f7fe640 1 --2- 172.21.15.83:0/1040198283 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8700b27f0 0x7fe87006cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:55.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.829+0000 7fe86ffff640 1 --2- 172.21.15.83:0/1040198283 >> v2:172.21.15.167:3300/0 conn(0x7fe87015d940 0x7fe8700b5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:55.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.829+0000 7fe874e31640 1 -- 172.21.15.83:0/1040198283 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8700b27f0 msgr2=0x7fe87006cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:55.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.829+0000 7fe874e31640 1 --2- 172.21.15.83:0/1040198283 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8700b27f0 0x7fe87006cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:55.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.829+0000 7fe874e31640 1 -- 172.21.15.83:0/1040198283 >> v2:172.21.15.167:3300/0 conn(0x7fe87015d940 msgr2=0x7fe8700b5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:55.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.829+0000 7fe874e31640 1 --2- 172.21.15.83:0/1040198283 >> v2:172.21.15.167:3300/0 conn(0x7fe87015d940 0x7fe8700b5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:55.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.829+0000 7fe874e31640 1 -- 172.21.15.83:0/1040198283 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87006ae30 con 0x7fe87006d2d0 2024-04-17T10:49:55.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.833+0000 7fe874e31640 1 --2- 172.21.15.83:0/1040198283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe87006d2d0 0x7fe8700d0070 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fe870071af0 tx=0x7fe86801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea8968829c36bfef server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:55.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.833+0000 7fe86effd640 1 -- 172.21.15.83:0/1040198283 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8680a55f0 con 0x7fe87006d2d0 2024-04-17T10:49:55.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.833+0000 7fe86effd640 1 -- 172.21.15.83:0/1040198283 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8680a5790 con 0x7fe87006d2d0 2024-04-17T10:49:55.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.833+0000 7fe86effd640 1 -- 172.21.15.83:0/1040198283 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8680a5b60 con 0x7fe87006d2d0 2024-04-17T10:49:55.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.833+0000 7fe876894640 1 -- 172.21.15.83:0/1040198283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe87006d2d0 msgr2=0x7fe8700d0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:55.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.833+0000 7fe876894640 1 --2- 172.21.15.83:0/1040198283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe87006d2d0 0x7fe8700d0070 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fe870071af0 tx=0x7fe86801c9f0 comp rx=0 tx=0).stop 2024-04-17T10:49:55.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.833+0000 7fe876894640 1 -- 172.21.15.83:0/1040198283 shutdown_connections 2024-04-17T10:49:55.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.833+0000 7fe876894640 1 --2- 172.21.15.83:0/1040198283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe87006d2d0 0x7fe8700d0070 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:55.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.833+0000 7fe876894640 1 --2- 172.21.15.83:0/1040198283 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8700b27f0 0x7fe87006cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:55.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.833+0000 7fe876894640 1 --2- 172.21.15.83:0/1040198283 >> v2:172.21.15.167:3300/0 conn(0x7fe87015d940 0x7fe8700b5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:55.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.833+0000 7fe876894640 1 -- 172.21.15.83:0/1040198283 >> 172.21.15.83:0/1040198283 conn(0x7fe8700b4eb0 msgr2=0x7fe8700b5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:55.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.833+0000 7fe876894640 1 -- 172.21.15.83:0/1040198283 shutdown_connections 2024-04-17T10:49:55.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.833+0000 7fe876894640 1 -- 172.21.15.83:0/1040198283 wait complete. 2024-04-17T10:49:55.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.833+0000 7fe876894640 1 Processor -- start 2024-04-17T10:49:55.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.833+0000 7fe876894640 1 -- start start 2024-04-17T10:49:55.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.837+0000 7fe876894640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe87006d2d0 0x7fe87009ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:55.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.837+0000 7fe876894640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8700b27f0 0x7fe87009a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:55.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.837+0000 7fe876894640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe87015d940 0x7fe8700997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:55.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.837+0000 7fe876894640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8700ae400 con 0x7fe87006d2d0 2024-04-17T10:49:55.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.837+0000 7fe876894640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8700ae280 con 0x7fe87015d940 2024-04-17T10:49:55.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.837+0000 7fe876894640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8700d05b0 con 0x7fe8700b27f0 2024-04-17T10:49:55.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.837+0000 7fe86ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe87006d2d0 0x7fe87009ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:55.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.837+0000 7fe86ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe87006d2d0 0x7fe87009ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48136/0 (socket says 172.21.15.83:48136) 2024-04-17T10:49:55.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.837+0000 7fe86ffff640 1 -- 172.21.15.83:0/3443024518 learned_addr learned my addr 172.21.15.83:0/3443024518 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:55.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.837+0000 7fe86ffff640 1 -- 172.21.15.83:0/3443024518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8700b27f0 msgr2=0x7fe87009a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:55.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.837+0000 7fe86ffff640 1 --2- 172.21.15.83:0/3443024518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8700b27f0 0x7fe87009a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:55.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.837+0000 7fe86ffff640 1 -- 172.21.15.83:0/3443024518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe87015d940 msgr2=0x7fe8700997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:55.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.837+0000 7fe874e31640 1 --2- 172.21.15.83:0/3443024518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe87015d940 0x7fe8700997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:55.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.837+0000 7fe86ffff640 1 --2- 172.21.15.83:0/3443024518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe87015d940 0x7fe8700997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:55.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.837+0000 7fe86ffff640 1 -- 172.21.15.83:0/3443024518 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87009a680 con 0x7fe87006d2d0 2024-04-17T10:49:55.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.837+0000 7fe86ffff640 1 --2- 172.21.15.83:0/3443024518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe87006d2d0 0x7fe87009ca20 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fe86000c820 tx=0x7fe86000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:55.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.837+0000 7fe86cff9640 1 -- 172.21.15.83:0/3443024518 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe860015070 con 0x7fe87006d2d0 2024-04-17T10:49:55.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.837+0000 7fe86cff9640 1 -- 172.21.15.83:0/3443024518 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe860007a30 con 0x7fe87006d2d0 2024-04-17T10:49:55.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.837+0000 7fe876894640 1 -- 172.21.15.83:0/3443024518 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe87009d0a0 con 0x7fe87006d2d0 2024-04-17T10:49:55.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.837+0000 7fe86cff9640 1 -- 172.21.15.83:0/3443024518 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe860007e20 con 0x7fe87006d2d0 2024-04-17T10:49:55.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.837+0000 7fe876894640 1 -- 172.21.15.83:0/3443024518 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe87009d660 con 0x7fe87006d2d0 2024-04-17T10:49:55.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.837+0000 7fe876894640 1 -- 172.21.15.83:0/3443024518 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe870074f90 con 0x7fe87006d2d0 2024-04-17T10:49:55.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.837+0000 7fe86cff9640 1 -- 172.21.15.83:0/3443024518 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe860018070 con 0x7fe87006d2d0 2024-04-17T10:49:55.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.837+0000 7fe86cff9640 1 --2- 172.21.15.83:0/3443024518 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe85c066f60 0x7fe85c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:55.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.837+0000 7fe86f7fe640 1 --2- 172.21.15.83:0/3443024518 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe85c066f60 0x7fe85c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:55.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.837+0000 7fe86cff9640 1 -- 172.21.15.83:0/3443024518 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe860084ea0 con 0x7fe87006d2d0 2024-04-17T10:49:55.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.837+0000 7fe86f7fe640 1 --2- 172.21.15.83:0/3443024518 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe85c066f60 0x7fe85c069420 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe864006b20 tx=0x7fe864008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:55.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:55.841+0000 7fe86cff9640 1 -- 172.21.15.83:0/3443024518 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe860055f00 con 0x7fe87006d2d0 2024-04-17T10:49:56.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.065+0000 7fe876894640 1 -- 172.21.15.83:0/3443024518 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe870068140 con 0x7fe87006d2d0 2024-04-17T10:49:56.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.065+0000 7fe86cff9640 1 -- 172.21.15.83:0/3443024518 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7fe860055800 con 0x7fe87006d2d0 2024-04-17T10:49:56.059 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:56.059 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":728,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":128,"num_osds":8,"num_up_osds":8,"osd_up_since":1713350279,"num_in_osds":8,"osd_in_since":1713350270,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1039,"data_bytes":1067367509,"bytes_used":2594045952,"bytes_avail":770500067328,"bytes_total":773094113280,"write_bytes_sec":52250190,"read_op_per_sec":0,"write_op_per_sec":52},"fsmap":{"epoch":76,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":9198}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-04-17T10:49:54.387518+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-17T10:49:56.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.069+0000 7fe876894640 1 -- 172.21.15.83:0/3443024518 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe85c066f60 msgr2=0x7fe85c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:56.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.069+0000 7fe876894640 1 --2- 172.21.15.83:0/3443024518 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe85c066f60 0x7fe85c069420 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe864006b20 tx=0x7fe864008040 comp rx=0 tx=0).stop 2024-04-17T10:49:56.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.069+0000 7fe876894640 1 -- 172.21.15.83:0/3443024518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe87006d2d0 msgr2=0x7fe87009ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:56.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.069+0000 7fe876894640 1 --2- 172.21.15.83:0/3443024518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe87006d2d0 0x7fe87009ca20 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fe86000c820 tx=0x7fe86000cce0 comp rx=0 tx=0).stop 2024-04-17T10:49:56.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.069+0000 7fe876894640 1 -- 172.21.15.83:0/3443024518 shutdown_connections 2024-04-17T10:49:56.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.069+0000 7fe876894640 1 --2- 172.21.15.83:0/3443024518 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe85c066f60 0x7fe85c069420 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:56.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.069+0000 7fe876894640 1 --2- 172.21.15.83:0/3443024518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe87015d940 0x7fe8700997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:56.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.069+0000 7fe876894640 1 --2- 172.21.15.83:0/3443024518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8700b27f0 0x7fe87009a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:56.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.069+0000 7fe876894640 1 --2- 172.21.15.83:0/3443024518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe87006d2d0 0x7fe87009ca20 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:56.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.069+0000 7fe876894640 1 -- 172.21.15.83:0/3443024518 >> 172.21.15.83:0/3443024518 conn(0x7fe8700b4eb0 msgr2=0x7fe87013de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:56.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.069+0000 7fe876894640 1 -- 172.21.15.83:0/3443024518 shutdown_connections 2024-04-17T10:49:56.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.069+0000 7fe876894640 1 -- 172.21.15.83:0/3443024518 wait complete. 2024-04-17T10:49:56.077 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-17T10:49:56.078 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-17T10:49:56.078 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:49:56.089 INFO:tasks.ceph.mds.c:Stopped 2024-04-17T10:49:56.089 INFO:tasks.cephfs.test_scrub_checks:Waiting for promotion of one of the original standbys {'d', 'b', 'a'} 2024-04-17T10:49:56.089 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-04-17T10:49:56.281 INFO:teuthology.orchestra.run.smithi083.stdout:{"mds_beacon_grace":"15.000000"} 2024-04-17T10:49:56.281 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-04-17T10:49:56.282 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:49:56.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.413+0000 7ff670e94640 1 Processor -- start 2024-04-17T10:49:56.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.413+0000 7ff670e94640 1 -- start start 2024-04-17T10:49:56.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.417+0000 7ff670e94640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff66c072330 0x7ff66c0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:56.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.417+0000 7ff670e94640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff66c0cfa10 0x7ff66c06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:56.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.417+0000 7ff670e94640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff66c0cfea0 con 0x7ff66c15ba30 2024-04-17T10:49:56.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.417+0000 7ff670e94640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff66c071ea0 con 0x7ff66c0cfa10 2024-04-17T10:49:56.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.417+0000 7ff670e94640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff66c06e320 con 0x7ff66c072330 2024-04-17T10:49:56.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.417+0000 7ff66ad9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff66c0cfa10 0x7ff66c06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:56.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.417+0000 7ff66ad9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff66c0cfa10 0x7ff66c06c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48138/0 (socket says 172.21.15.83:48138) 2024-04-17T10:49:56.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.417+0000 7ff66ad9d640 1 -- 172.21.15.83:0/3837827382 learned_addr learned my addr 172.21.15.83:0/3837827382 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:56.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.417+0000 7ff669d9b640 1 --2- 172.21.15.83:0/3837827382 >> v2:172.21.15.167:3300/0 conn(0x7ff66c072330 0x7ff66c0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:56.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.417+0000 7ff66ad9d640 1 -- 172.21.15.83:0/3837827382 >> v2:172.21.15.167:3300/0 conn(0x7ff66c072330 msgr2=0x7ff66c0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:56.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.417+0000 7ff66ad9d640 1 --2- 172.21.15.83:0/3837827382 >> v2:172.21.15.167:3300/0 conn(0x7ff66c072330 0x7ff66c0cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:56.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.417+0000 7ff66ad9d640 1 -- 172.21.15.83:0/3837827382 >> v1:172.21.15.167:6789/0 conn(0x7ff66c15ba30 legacy=0x7ff66c15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:49:56.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.417+0000 7ff66ad9d640 1 -- 172.21.15.83:0/3837827382 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff66c06e040 con 0x7ff66c0cfa10 2024-04-17T10:49:56.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.417+0000 7ff669d9b640 1 --2- 172.21.15.83:0/3837827382 >> v2:172.21.15.167:3300/0 conn(0x7ff66c072330 0x7ff66c0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:56.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.417+0000 7ff66ad9d640 1 --2- 172.21.15.83:0/3837827382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff66c0cfa10 0x7ff66c06c490 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7ff66c0af2b0 tx=0x7ff66401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6916e4ad17d5d34 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:56.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.417+0000 7ff66959a640 1 -- 172.21.15.83:0/3837827382 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6640a56f0 con 0x7ff66c0cfa10 2024-04-17T10:49:56.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.417+0000 7ff66959a640 1 -- 172.21.15.83:0/3837827382 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6640a5890 con 0x7ff66c0cfa10 2024-04-17T10:49:56.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.417+0000 7ff66959a640 1 -- 172.21.15.83:0/3837827382 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6640a5c60 con 0x7ff66c0cfa10 2024-04-17T10:49:56.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.421+0000 7ff670e94640 1 -- 172.21.15.83:0/3837827382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff66c0cfa10 msgr2=0x7ff66c06c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:56.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.421+0000 7ff670e94640 1 --2- 172.21.15.83:0/3837827382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff66c0cfa10 0x7ff66c06c490 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7ff66c0af2b0 tx=0x7ff66401c920 comp rx=0 tx=0).stop 2024-04-17T10:49:56.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.421+0000 7ff670e94640 1 -- 172.21.15.83:0/3837827382 shutdown_connections 2024-04-17T10:49:56.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.421+0000 7ff670e94640 1 --2- 172.21.15.83:0/3837827382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff66c0cfa10 0x7ff66c06c490 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:56.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.421+0000 7ff670e94640 1 --2- 172.21.15.83:0/3837827382 >> v2:172.21.15.167:3300/0 conn(0x7ff66c072330 0x7ff66c0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:56.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.421+0000 7ff670e94640 1 -- 172.21.15.83:0/3837827382 >> 172.21.15.83:0/3837827382 conn(0x7ff66c06abe0 msgr2=0x7ff66c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:56.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.421+0000 7ff670e94640 1 -- 172.21.15.83:0/3837827382 shutdown_connections 2024-04-17T10:49:56.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.421+0000 7ff670e94640 1 -- 172.21.15.83:0/3837827382 wait complete. 2024-04-17T10:49:56.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.421+0000 7ff670e94640 1 Processor -- start 2024-04-17T10:49:56.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.421+0000 7ff670e94640 1 -- start start 2024-04-17T10:49:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.421+0000 7ff670e94640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff66c072330 0x7ff66c1547e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.421+0000 7ff670e94640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff66c0cfa10 0x7ff66c1508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.421+0000 7ff670e94640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff66c15ba30 0x7ff66c15e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.421+0000 7ff670e94640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66c06de20 con 0x7ff66c15ba30 2024-04-17T10:49:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.421+0000 7ff670e94640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66c06dca0 con 0x7ff66c072330 2024-04-17T10:49:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.421+0000 7ff670e94640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff66c168520 con 0x7ff66c0cfa10 2024-04-17T10:49:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.421+0000 7ff66ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff66c15ba30 0x7ff66c15e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.421+0000 7ff66ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff66c15ba30 0x7ff66c15e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48154/0 (socket says 172.21.15.83:48154) 2024-04-17T10:49:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.421+0000 7ff66ad9d640 1 -- 172.21.15.83:0/2951965693 learned_addr learned my addr 172.21.15.83:0/2951965693 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:49:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.421+0000 7ff66a59c640 1 --2- 172.21.15.83:0/2951965693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff66c072330 0x7ff66c1547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:56.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.421+0000 7ff66ad9d640 1 -- 172.21.15.83:0/2951965693 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff66c0cfa10 msgr2=0x7ff66c1508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:56.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.421+0000 7ff66ad9d640 1 --2- 172.21.15.83:0/2951965693 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff66c0cfa10 0x7ff66c1508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:56.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.421+0000 7ff66ad9d640 1 -- 172.21.15.83:0/2951965693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff66c072330 msgr2=0x7ff66c1547e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:56.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.421+0000 7ff66ad9d640 1 --2- 172.21.15.83:0/2951965693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff66c072330 0x7ff66c1547e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:56.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.421+0000 7ff66ad9d640 1 -- 172.21.15.83:0/2951965693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff66c152fe0 con 0x7ff66c15ba30 2024-04-17T10:49:56.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.421+0000 7ff66a59c640 1 --2- 172.21.15.83:0/2951965693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff66c072330 0x7ff66c1547e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:49:56.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.421+0000 7ff66ad9d640 1 --2- 172.21.15.83:0/2951965693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff66c15ba30 0x7ff66c15e560 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7ff66401a550 tx=0x7ff6640196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:56.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.421+0000 7ff65affd640 1 -- 172.21.15.83:0/2951965693 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6640bc020 con 0x7ff66c15ba30 2024-04-17T10:49:56.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.421+0000 7ff65affd640 1 -- 172.21.15.83:0/2951965693 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6640b8670 con 0x7ff66c15ba30 2024-04-17T10:49:56.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.421+0000 7ff670e94640 1 -- 172.21.15.83:0/2951965693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff66c153d00 con 0x7ff66c15ba30 2024-04-17T10:49:56.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.421+0000 7ff65affd640 1 -- 172.21.15.83:0/2951965693 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6640b8a60 con 0x7ff66c15ba30 2024-04-17T10:49:56.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.421+0000 7ff670e94640 1 -- 172.21.15.83:0/2951965693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff66c1542c0 con 0x7ff66c15ba30 2024-04-17T10:49:56.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.425+0000 7ff670e94640 1 -- 172.21.15.83:0/2951965693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff66c153af0 con 0x7ff66c15ba30 2024-04-17T10:49:56.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.425+0000 7ff65affd640 1 -- 172.21.15.83:0/2951965693 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff6640bf020 con 0x7ff66c15ba30 2024-04-17T10:49:56.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.425+0000 7ff65affd640 1 --2- 172.21.15.83:0/2951965693 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff64c066f60 0x7ff64c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:49:56.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.425+0000 7ff66a59c640 1 --2- 172.21.15.83:0/2951965693 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff64c066f60 0x7ff64c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:49:56.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.425+0000 7ff65affd640 1 -- 172.21.15.83:0/2951965693 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff664120050 con 0x7ff66c15ba30 2024-04-17T10:49:56.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.425+0000 7ff66a59c640 1 --2- 172.21.15.83:0/2951965693 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff64c066f60 0x7ff64c069420 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7ff66c1f7810 tx=0x7ff65c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:49:56.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.429+0000 7ff65affd640 1 -- 172.21.15.83:0/2951965693 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff6640c6030 con 0x7ff66c15ba30 2024-04-17T10:49:56.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.625+0000 7ff670e94640 1 -- 172.21.15.83:0/2951965693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff66c074f90 con 0x7ff66c15ba30 2024-04-17T10:49:56.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.625+0000 7ff65affd640 1 -- 172.21.15.83:0/2951965693 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4946 (secure 0 0 0) 0x7ff6640165a0 con 0x7ff66c15ba30 2024-04-17T10:49:56.619 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:49:56.619 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5942,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3888166846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3888166846},{"type":"v1","addr":"172.21.15.167:6833","nonce":3888166846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":6772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1167913722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1167913722},{"type":"v1","addr":"172.21.15.167:6835","nonce":1167913722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":9192,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3552333026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3552333026},{"type":"v1","addr":"172.21.15.83:6835","nonce":3552333026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:49:19.962680+0000","modified":"2024-04-17T10:49:21.996802+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9198":{"gid":9198,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1559222491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1559222491},{"type":"v1","addr":"172.21.15.83:6837","nonce":1559222491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-17T10:49:56.619 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 76 2024-04-17T10:49:56.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.629+0000 7ff670e94640 1 -- 172.21.15.83:0/2951965693 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff64c066f60 msgr2=0x7ff64c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:56.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.629+0000 7ff670e94640 1 --2- 172.21.15.83:0/2951965693 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff64c066f60 0x7ff64c069420 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7ff66c1f7810 tx=0x7ff65c009040 comp rx=0 tx=0).stop 2024-04-17T10:49:56.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.629+0000 7ff670e94640 1 -- 172.21.15.83:0/2951965693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff66c15ba30 msgr2=0x7ff66c15e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:49:56.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.629+0000 7ff670e94640 1 --2- 172.21.15.83:0/2951965693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff66c15ba30 0x7ff66c15e560 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7ff66401a550 tx=0x7ff6640196c0 comp rx=0 tx=0).stop 2024-04-17T10:49:56.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.629+0000 7ff670e94640 1 -- 172.21.15.83:0/2951965693 shutdown_connections 2024-04-17T10:49:56.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.629+0000 7ff670e94640 1 --2- 172.21.15.83:0/2951965693 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff64c066f60 0x7ff64c069420 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:56.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.629+0000 7ff670e94640 1 --2- 172.21.15.83:0/2951965693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff66c15ba30 0x7ff66c15e560 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:56.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.629+0000 7ff670e94640 1 --2- 172.21.15.83:0/2951965693 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff66c0cfa10 0x7ff66c1508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:56.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.629+0000 7ff670e94640 1 --2- 172.21.15.83:0/2951965693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff66c072330 0x7ff66c1547e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:49:56.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.629+0000 7ff670e94640 1 -- 172.21.15.83:0/2951965693 >> 172.21.15.83:0/2951965693 conn(0x7ff66c06abe0 msgr2=0x7ff66c151b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:49:56.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.629+0000 7ff670e94640 1 -- 172.21.15.83:0/2951965693 shutdown_connections 2024-04-17T10:49:56.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:49:56.629+0000 7ff670e94640 1 -- 172.21.15.83:0/2951965693 wait complete. 2024-04-17T10:49:56.639 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-17T10:50:01.639 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:50:01.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.777+0000 7f5cc1d0b640 1 Processor -- start 2024-04-17T10:50:01.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.777+0000 7f5cc1d0b640 1 -- start start 2024-04-17T10:50:01.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.777+0000 7f5cc1d0b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5cbc15ba30 0x7f5cbc072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:01.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.777+0000 7f5cc1d0b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5cbc0cfbe0 0x7f5cbc06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:01.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.777+0000 7f5cc1d0b640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cbc06e320 con 0x7f5cbc15c960 2024-04-17T10:50:01.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.777+0000 7f5cc1d0b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cbc071ea0 con 0x7f5cbc0cfbe0 2024-04-17T10:50:01.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.777+0000 7f5cc1d0b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc06e040 con 0x7f5cbc15ba30 2024-04-17T10:50:01.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.777+0000 7f5cbbfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5cbc0cfbe0 0x7f5cbc06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:01.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.777+0000 7f5cbbfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5cbc0cfbe0 0x7f5cbc06c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42672/0 (socket says 172.21.15.83:42672) 2024-04-17T10:50:01.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.777+0000 7f5cbbfff640 1 -- 172.21.15.83:0/211226742 learned_addr learned my addr 172.21.15.83:0/211226742 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:01.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.777+0000 7f5cbb7fe640 1 --2- 172.21.15.83:0/211226742 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5cbc15ba30 0x7f5cbc072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:01.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.777+0000 7f5cbbfff640 1 -- 172.21.15.83:0/211226742 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5cbc15ba30 msgr2=0x7f5cbc072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:01.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.777+0000 7f5cbbfff640 1 --2- 172.21.15.83:0/211226742 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5cbc15ba30 0x7f5cbc072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:01.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.777+0000 7f5cbbfff640 1 -- 172.21.15.83:0/211226742 >> v1:172.21.15.167:6789/0 conn(0x7f5cbc15c960 legacy=0x7f5cbc0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:01.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.777+0000 7f5cbbfff640 1 -- 172.21.15.83:0/211226742 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cbc071850 con 0x7f5cbc0cfbe0 2024-04-17T10:50:01.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.781+0000 7f5cbbfff640 1 --2- 172.21.15.83:0/211226742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cbc0cfbe0 0x7f5cbc06c490 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4009e50 tx=0x7f5cb401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb6d9465c0da9e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:01.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.781+0000 7f5cba7fc640 1 -- 172.21.15.83:0/211226742 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb40a56b0 con 0x7f5cbc0cfbe0 2024-04-17T10:50:01.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.781+0000 7f5cba7fc640 1 -- 172.21.15.83:0/211226742 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb40a5850 con 0x7f5cbc0cfbe0 2024-04-17T10:50:01.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.781+0000 7f5cba7fc640 1 -- 172.21.15.83:0/211226742 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb40a5c20 con 0x7f5cbc0cfbe0 2024-04-17T10:50:01.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.781+0000 7f5cc1d0b640 1 -- 172.21.15.83:0/211226742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cbc0cfbe0 msgr2=0x7f5cbc06c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:01.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.781+0000 7f5cc1d0b640 1 --2- 172.21.15.83:0/211226742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cbc0cfbe0 0x7f5cbc06c490 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4009e50 tx=0x7f5cb401c9a0 comp rx=0 tx=0).stop 2024-04-17T10:50:01.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.781+0000 7f5cc1d0b640 1 -- 172.21.15.83:0/211226742 shutdown_connections 2024-04-17T10:50:01.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.781+0000 7f5cc1d0b640 1 --2- 172.21.15.83:0/211226742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cbc0cfbe0 0x7f5cbc06c490 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:01.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.781+0000 7f5cc1d0b640 1 --2- 172.21.15.83:0/211226742 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5cbc15ba30 0x7f5cbc072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:01.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.781+0000 7f5cc1d0b640 1 -- 172.21.15.83:0/211226742 >> 172.21.15.83:0/211226742 conn(0x7f5cbc06abe0 msgr2=0x7f5cbc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:01.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.781+0000 7f5cc1d0b640 1 -- 172.21.15.83:0/211226742 shutdown_connections 2024-04-17T10:50:01.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.781+0000 7f5cc1d0b640 1 -- 172.21.15.83:0/211226742 wait complete. 2024-04-17T10:50:01.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.781+0000 7f5cc1d0b640 1 Processor -- start 2024-04-17T10:50:01.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.781+0000 7f5cc1d0b640 1 -- start start 2024-04-17T10:50:01.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.781+0000 7f5cc1d0b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cbc0cfbe0 0x7f5cbc139260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:01.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.781+0000 7f5cc1d0b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5cbc15ba30 0x7f5cbc14eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:01.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.781+0000 7f5cc1d0b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cbc15c960 0x7f5cbc135dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:01.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.781+0000 7f5cc1d0b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc15cee0 con 0x7f5cbc15c960 2024-04-17T10:50:01.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.781+0000 7f5cc1d0b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc15cd60 con 0x7f5cbc0cfbe0 2024-04-17T10:50:01.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.781+0000 7f5cc1d0b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc168520 con 0x7f5cbc15ba30 2024-04-17T10:50:01.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.781+0000 7f5cbbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cbc15c960 0x7f5cbc135dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:01.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.781+0000 7f5cbbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cbc15c960 0x7f5cbc135dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42678/0 (socket says 172.21.15.83:42678) 2024-04-17T10:50:01.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.781+0000 7f5cbbfff640 1 -- 172.21.15.83:0/4153032619 learned_addr learned my addr 172.21.15.83:0/4153032619 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:01.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.781+0000 7f5cbb7fe640 1 --2- 172.21.15.83:0/4153032619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cbc0cfbe0 0x7f5cbc139260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:01.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.785+0000 7f5cbaffd640 1 --2- 172.21.15.83:0/4153032619 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5cbc15ba30 0x7f5cbc14eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:01.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.785+0000 7f5cbbfff640 1 -- 172.21.15.83:0/4153032619 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5cbc15ba30 msgr2=0x7f5cbc14eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:01.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.785+0000 7f5cbbfff640 1 --2- 172.21.15.83:0/4153032619 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5cbc15ba30 0x7f5cbc14eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:01.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.785+0000 7f5cbbfff640 1 -- 172.21.15.83:0/4153032619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cbc0cfbe0 msgr2=0x7f5cbc139260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:01.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.785+0000 7f5cbbfff640 1 --2- 172.21.15.83:0/4153032619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cbc0cfbe0 0x7f5cbc139260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:01.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.785+0000 7f5cbbfff640 1 -- 172.21.15.83:0/4153032619 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cbc14f960 con 0x7f5cbc15c960 2024-04-17T10:50:01.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.785+0000 7f5cbb7fe640 1 --2- 172.21.15.83:0/4153032619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cbc0cfbe0 0x7f5cbc139260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:50:01.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.785+0000 7f5cbbfff640 1 --2- 172.21.15.83:0/4153032619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cbc15c960 0x7f5cbc135dd0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4009e50 tx=0x7f5cb4019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:01.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.785+0000 7f5ca3fff640 1 -- 172.21.15.83:0/4153032619 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb40bc020 con 0x7f5cbc15c960 2024-04-17T10:50:01.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.785+0000 7f5ca3fff640 1 -- 172.21.15.83:0/4153032619 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb40b8670 con 0x7f5cbc15c960 2024-04-17T10:50:01.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.785+0000 7f5cc1d0b640 1 -- 172.21.15.83:0/4153032619 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cbc14f3e0 con 0x7f5cbc15c960 2024-04-17T10:50:01.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.785+0000 7f5ca3fff640 1 -- 172.21.15.83:0/4153032619 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4009510 con 0x7f5cbc15c960 2024-04-17T10:50:01.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.785+0000 7f5cc1d0b640 1 -- 172.21.15.83:0/4153032619 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cbc136310 con 0x7f5cbc15c960 2024-04-17T10:50:01.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.785+0000 7f5cc1d0b640 1 -- 172.21.15.83:0/4153032619 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cbc074f90 con 0x7f5cbc15c960 2024-04-17T10:50:01.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.785+0000 7f5ca3fff640 1 -- 172.21.15.83:0/4153032619 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5cb40bf020 con 0x7f5cbc15c960 2024-04-17T10:50:01.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.785+0000 7f5ca3fff640 1 --2- 172.21.15.83:0/4153032619 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5ca8066f60 0x7f5ca8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:01.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.785+0000 7f5ca3fff640 1 -- 172.21.15.83:0/4153032619 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5cb41200d0 con 0x7f5cbc15c960 2024-04-17T10:50:01.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.785+0000 7f5cbb7fe640 1 --2- 172.21.15.83:0/4153032619 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5ca8066f60 0x7f5ca8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:01.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.785+0000 7f5cbb7fe640 1 --2- 172.21.15.83:0/4153032619 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5ca8066f60 0x7f5ca8069420 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5cac007a70 tx=0x7f5cac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:01.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.789+0000 7f5ca3fff640 1 -- 172.21.15.83:0/4153032619 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5cb40c6030 con 0x7f5cbc15c960 2024-04-17T10:50:01.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.985+0000 7f5cc1d0b640 1 -- 172.21.15.83:0/4153032619 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5cbc06b0f0 con 0x7f5cbc15c960 2024-04-17T10:50:01.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.985+0000 7f5ca3fff640 1 -- 172.21.15.83:0/4153032619 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f5cb4016620 con 0x7f5cbc15c960 2024-04-17T10:50:01.977 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:50:01.977 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5942,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3888166846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3888166846},{"type":"v1","addr":"172.21.15.167:6833","nonce":3888166846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":6772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1167913722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1167913722},{"type":"v1","addr":"172.21.15.167:6835","nonce":1167913722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":9192,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3552333026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3552333026},{"type":"v1","addr":"172.21.15.83:6835","nonce":3552333026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:49:19.962680+0000","modified":"2024-04-17T10:49:21.996802+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9198":{"gid":9198,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1559222491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1559222491},{"type":"v1","addr":"172.21.15.83:6837","nonce":1559222491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-17T10:50:01.977 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 76 2024-04-17T10:50:01.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.985+0000 7f5cc1d0b640 1 -- 172.21.15.83:0/4153032619 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5ca8066f60 msgr2=0x7f5ca8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:01.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.985+0000 7f5cc1d0b640 1 --2- 172.21.15.83:0/4153032619 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5ca8066f60 0x7f5ca8069420 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5cac007a70 tx=0x7f5cac008040 comp rx=0 tx=0).stop 2024-04-17T10:50:01.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.985+0000 7f5cc1d0b640 1 -- 172.21.15.83:0/4153032619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cbc15c960 msgr2=0x7f5cbc135dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:01.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.985+0000 7f5cc1d0b640 1 --2- 172.21.15.83:0/4153032619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cbc15c960 0x7f5cbc135dd0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4009e50 tx=0x7f5cb4019740 comp rx=0 tx=0).stop 2024-04-17T10:50:01.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.985+0000 7f5cc1d0b640 1 -- 172.21.15.83:0/4153032619 shutdown_connections 2024-04-17T10:50:01.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.985+0000 7f5cc1d0b640 1 --2- 172.21.15.83:0/4153032619 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5ca8066f60 0x7f5ca8069420 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:01.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.985+0000 7f5cc1d0b640 1 --2- 172.21.15.83:0/4153032619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cbc15c960 0x7f5cbc135dd0 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:01.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.985+0000 7f5cc1d0b640 1 --2- 172.21.15.83:0/4153032619 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5cbc15ba30 0x7f5cbc14eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:01.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.985+0000 7f5cc1d0b640 1 --2- 172.21.15.83:0/4153032619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cbc0cfbe0 0x7f5cbc139260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:01.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.985+0000 7f5cc1d0b640 1 -- 172.21.15.83:0/4153032619 >> 172.21.15.83:0/4153032619 conn(0x7f5cbc06abe0 msgr2=0x7f5cbc1524d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:01.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.985+0000 7f5cc1d0b640 1 -- 172.21.15.83:0/4153032619 shutdown_connections 2024-04-17T10:50:01.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:01.985+0000 7f5cc1d0b640 1 -- 172.21.15.83:0/4153032619 wait complete. 2024-04-17T10:50:01.996 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-17T10:50:06.997 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:50:07.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ec39b2640 1 Processor -- start 2024-04-17T10:50:07.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ec39b2640 1 -- start start 2024-04-17T10:50:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ec39b2640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6ebc0b3990 0x7f6ebc070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ec39b2640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ebc070dc0 con 0x7f6ebc0709b0 2024-04-17T10:50:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ec39b2640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ebc0b2bd0 con 0x7f6ebc0cf060 2024-04-17T10:50:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ec39b2640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ebc0b2980 con 0x7f6ebc0b3990 2024-04-17T10:50:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ec1f4f640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6ebc0709b0 0x7f6ebc06c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39808/0 (socket says 172.21.15.83:39808) 2024-04-17T10:50:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ec1f4f640 1 -- 172.21.15.83:0/3139048454 learned_addr learned my addr 172.21.15.83:0/3139048454 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ec0f4d640 1 --2- 172.21.15.83:0/3139048454 >> v2:172.21.15.167:3300/0 conn(0x7f6ebc0b3990 0x7f6ebc070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ebbfff640 1 -- 172.21.15.83:0/3139048454 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 361194968 0 0) 0x7f6eac001120 con 0x7f6ebc0709b0 2024-04-17T10:50:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ebbfff640 1 -- 172.21.15.83:0/3139048454 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ea4003450 con 0x7f6ebc0709b0 2024-04-17T10:50:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ebbfff640 1 -- 172.21.15.83:0/3139048454 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2931169314 0 0) 0x7f6ea4003450 con 0x7f6ebc0709b0 2024-04-17T10:50:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ebbfff640 1 -- 172.21.15.83:0/3139048454 >> v2:172.21.15.167:3300/0 conn(0x7f6ebc0b3990 msgr2=0x7f6ebc070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ebbfff640 1 --2- 172.21.15.83:0/3139048454 >> v2:172.21.15.167:3300/0 conn(0x7f6ebc0b3990 0x7f6ebc070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ebbfff640 1 -- 172.21.15.83:0/3139048454 >> v1:172.21.15.167:6789/0 conn(0x7f6ebc0cf060 legacy=0x7f6ebc0b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ebbfff640 1 -- 172.21.15.83:0/3139048454 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ebc0b2eb0 con 0x7f6ebc0709b0 2024-04-17T10:50:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ebbfff640 1 -- 172.21.15.83:0/3139048454 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f6eac001850 con 0x7f6ebc0709b0 2024-04-17T10:50:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ebbfff640 1 -- 172.21.15.83:0/3139048454 >> v1:172.21.15.83:6789/0 conn(0x7f6ebc0709b0 legacy=0x7f6ebc06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ebbfff640 1 --2- 172.21.15.83:0/3139048454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ea4003ab0 0x7f6ea4003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ebbfff640 1 --2- 172.21.15.83:0/3139048454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ea4008860 0x7f6ea400acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ebbfff640 1 --2- 172.21.15.83:0/3139048454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ea400b4d0 0x7f6ea400d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ebbfff640 1 -- 172.21.15.83:0/3139048454 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc0b2980 con 0x7f6ea4008860 2024-04-17T10:50:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ebbfff640 1 -- 172.21.15.83:0/3139048454 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea400e270 con 0x7f6ea4003ab0 2024-04-17T10:50:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ebbfff640 1 -- 172.21.15.83:0/3139048454 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea400e470 con 0x7f6ea400b4d0 2024-04-17T10:50:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ec0f4d640 1 --2- 172.21.15.83:0/3139048454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ea4008860 0x7f6ea400acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ec174e640 1 --2- 172.21.15.83:0/3139048454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ea4003ab0 0x7f6ea4003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ec1f4f640 1 --2- 172.21.15.83:0/3139048454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ea400b4d0 0x7f6ea400d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:07.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ec0f4d640 1 -- 172.21.15.83:0/3139048454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ea400b4d0 msgr2=0x7f6ea400d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:07.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ec0f4d640 1 --2- 172.21.15.83:0/3139048454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ea400b4d0 0x7f6ea400d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:07.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ec0f4d640 1 -- 172.21.15.83:0/3139048454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ea4003ab0 msgr2=0x7f6ea4003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:07.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ec0f4d640 1 --2- 172.21.15.83:0/3139048454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ea4003ab0 0x7f6ea4003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:07.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ec0f4d640 1 -- 172.21.15.83:0/3139048454 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea4003450 con 0x7f6ea4008860 2024-04-17T10:50:07.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ec1f4f640 1 --2- 172.21.15.83:0/3139048454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ea400b4d0 0x7f6ea400d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:07.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.141+0000 7f6ec1f4f640 1 -- 172.21.15.83:0/3139048454 reap_dead start 2024-04-17T10:50:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.145+0000 7f6ec0f4d640 1 --2- 172.21.15.83:0/3139048454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ea4008860 0x7f6ea400acd0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0009ae0 tx=0x7f6eb001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2d93a82dbe52ed server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.145+0000 7f6ebbfff640 1 -- 172.21.15.83:0/3139048454 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb00b2070 con 0x7f6ea4008860 2024-04-17T10:50:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.145+0000 7f6ebbfff640 1 -- 172.21.15.83:0/3139048454 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb00a5550 con 0x7f6ea4008860 2024-04-17T10:50:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.145+0000 7f6ebbfff640 1 -- 172.21.15.83:0/3139048454 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb00096b0 con 0x7f6ea4008860 2024-04-17T10:50:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.145+0000 7f6ec39b2640 1 -- 172.21.15.83:0/3139048454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ea4008860 msgr2=0x7f6ea400acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.145+0000 7f6ec39b2640 1 --2- 172.21.15.83:0/3139048454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ea4008860 0x7f6ea400acd0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0009ae0 tx=0x7f6eb001ca70 comp rx=0 tx=0).stop 2024-04-17T10:50:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.145+0000 7f6ec39b2640 1 -- 172.21.15.83:0/3139048454 shutdown_connections 2024-04-17T10:50:07.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.145+0000 7f6ec39b2640 1 --2- 172.21.15.83:0/3139048454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ea4008860 0x7f6ea400acd0 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:07.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.145+0000 7f6ec39b2640 1 -- 172.21.15.83:0/3139048454 >> 172.21.15.83:0/3139048454 conn(0x7f6ebc06abe0 msgr2=0x7f6ebc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:07.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.145+0000 7f6ec39b2640 1 -- 172.21.15.83:0/3139048454 shutdown_connections 2024-04-17T10:50:07.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.145+0000 7f6ec39b2640 1 -- 172.21.15.83:0/3139048454 wait complete. 2024-04-17T10:50:07.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.145+0000 7f6ec39b2640 1 Processor -- start 2024-04-17T10:50:07.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.145+0000 7f6ec39b2640 1 -- start start 2024-04-17T10:50:07.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.149+0000 7f6ec39b2640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ea4008860 0x7f6ebc15e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:07.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.149+0000 7f6ec39b2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ebc15cec0 0x7f6ebc1431d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:07.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.149+0000 7f6ec39b2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ebc1505c0 0x7f6ebc0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:07.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.149+0000 7f6ec39b2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc15b310 con 0x7f6ebc15cec0 2024-04-17T10:50:07.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.149+0000 7f6ec39b2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc0b3470 con 0x7f6ebc1505c0 2024-04-17T10:50:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.149+0000 7f6ec39b2640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc070470 con 0x7f6ea4008860 2024-04-17T10:50:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.149+0000 7f6ec0f4d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ebc15cec0 0x7f6ebc1431d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.149+0000 7f6ec0f4d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ebc15cec0 0x7f6ebc1431d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58660/0 (socket says 172.21.15.83:58660) 2024-04-17T10:50:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.149+0000 7f6ec0f4d640 1 -- 172.21.15.83:0/2536722978 learned_addr learned my addr 172.21.15.83:0/2536722978 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.149+0000 7f6ec174e640 1 --2- 172.21.15.83:0/2536722978 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ea4008860 0x7f6ebc15e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.149+0000 7f6ec1f4f640 1 --2- 172.21.15.83:0/2536722978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ebc1505c0 0x7f6ebc0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.149+0000 7f6ec0f4d640 1 -- 172.21.15.83:0/2536722978 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ea4008860 msgr2=0x7f6ebc15e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.149+0000 7f6ec0f4d640 1 --2- 172.21.15.83:0/2536722978 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ea4008860 0x7f6ebc15e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.149+0000 7f6ec0f4d640 1 -- 172.21.15.83:0/2536722978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ebc1505c0 msgr2=0x7f6ebc0ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.149+0000 7f6ec0f4d640 1 --2- 172.21.15.83:0/2536722978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ebc1505c0 0x7f6ebc0ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.149+0000 7f6ec0f4d640 1 -- 172.21.15.83:0/2536722978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ebc15d350 con 0x7f6ebc15cec0 2024-04-17T10:50:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.149+0000 7f6ec0f4d640 1 --2- 172.21.15.83:0/2536722978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ebc15cec0 0x7f6ebc1431d0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f6eb001c8a0 tx=0x7f6eb0019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.149+0000 7f6eb9ffb640 1 -- 172.21.15.83:0/2536722978 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb00b2040 con 0x7f6ebc15cec0 2024-04-17T10:50:07.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.149+0000 7f6eb9ffb640 1 -- 172.21.15.83:0/2536722978 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb0004020 con 0x7f6ebc15cec0 2024-04-17T10:50:07.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.149+0000 7f6ec39b2640 1 -- 172.21.15.83:0/2536722978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ebc15d630 con 0x7f6ebc15cec0 2024-04-17T10:50:07.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.149+0000 7f6eb9ffb640 1 -- 172.21.15.83:0/2536722978 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0004410 con 0x7f6ebc15cec0 2024-04-17T10:50:07.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.149+0000 7f6ec39b2640 1 -- 172.21.15.83:0/2536722978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ebc15dbf0 con 0x7f6ebc15cec0 2024-04-17T10:50:07.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.149+0000 7f6ec39b2640 1 -- 172.21.15.83:0/2536722978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ebc074f90 con 0x7f6ebc15cec0 2024-04-17T10:50:07.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.149+0000 7f6eb9ffb640 1 -- 172.21.15.83:0/2536722978 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6eb00bf060 con 0x7f6ebc15cec0 2024-04-17T10:50:07.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.149+0000 7f6eb9ffb640 1 --2- 172.21.15.83:0/2536722978 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6ea8066f60 0x7f6ea8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:07.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.149+0000 7f6ec174e640 1 --2- 172.21.15.83:0/2536722978 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6ea8066f60 0x7f6ea8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:07.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.149+0000 7f6eb9ffb640 1 -- 172.21.15.83:0/2536722978 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6eb011ebe0 con 0x7f6ebc15cec0 2024-04-17T10:50:07.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.149+0000 7f6ec174e640 1 --2- 172.21.15.83:0/2536722978 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6ea8066f60 0x7f6ea8069420 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4005210 tx=0x7f6eb4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:07.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.153+0000 7f6eb9ffb640 1 -- 172.21.15.83:0/2536722978 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6eb0016780 con 0x7f6ebc15cec0 2024-04-17T10:50:07.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.345+0000 7f6ec39b2640 1 -- 172.21.15.83:0/2536722978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ebc0aec20 con 0x7f6ebc15cec0 2024-04-17T10:50:07.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.349+0000 7f6eb9ffb640 1 -- 172.21.15.83:0/2536722978 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f6eb00efbb0 con 0x7f6ebc15cec0 2024-04-17T10:50:07.340 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:50:07.340 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5942,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3888166846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3888166846},{"type":"v1","addr":"172.21.15.167:6833","nonce":3888166846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":6772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1167913722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1167913722},{"type":"v1","addr":"172.21.15.167:6835","nonce":1167913722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":9192,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3552333026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3552333026},{"type":"v1","addr":"172.21.15.83:6835","nonce":3552333026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:49:19.962680+0000","modified":"2024-04-17T10:49:21.996802+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9198":{"gid":9198,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1559222491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1559222491},{"type":"v1","addr":"172.21.15.83:6837","nonce":1559222491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-17T10:50:07.340 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 76 2024-04-17T10:50:07.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.349+0000 7f6ec39b2640 1 -- 172.21.15.83:0/2536722978 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6ea8066f60 msgr2=0x7f6ea8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:07.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.349+0000 7f6ec39b2640 1 --2- 172.21.15.83:0/2536722978 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6ea8066f60 0x7f6ea8069420 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4005210 tx=0x7f6eb4009040 comp rx=0 tx=0).stop 2024-04-17T10:50:07.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.349+0000 7f6ec39b2640 1 -- 172.21.15.83:0/2536722978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ebc15cec0 msgr2=0x7f6ebc1431d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:07.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.349+0000 7f6ec39b2640 1 --2- 172.21.15.83:0/2536722978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ebc15cec0 0x7f6ebc1431d0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f6eb001c8a0 tx=0x7f6eb0019810 comp rx=0 tx=0).stop 2024-04-17T10:50:07.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.349+0000 7f6ec39b2640 1 -- 172.21.15.83:0/2536722978 shutdown_connections 2024-04-17T10:50:07.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.349+0000 7f6ec39b2640 1 --2- 172.21.15.83:0/2536722978 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6ea8066f60 0x7f6ea8069420 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:07.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.349+0000 7f6ec39b2640 1 --2- 172.21.15.83:0/2536722978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ebc1505c0 0x7f6ebc0ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:07.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.349+0000 7f6ec39b2640 1 --2- 172.21.15.83:0/2536722978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ebc15cec0 0x7f6ebc1431d0 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:07.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.349+0000 7f6ec39b2640 1 --2- 172.21.15.83:0/2536722978 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ea4008860 0x7f6ebc15e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:07.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.349+0000 7f6ec39b2640 1 -- 172.21.15.83:0/2536722978 >> 172.21.15.83:0/2536722978 conn(0x7f6ebc06abe0 msgr2=0x7f6ebc06d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:07.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.349+0000 7f6ec39b2640 1 -- 172.21.15.83:0/2536722978 shutdown_connections 2024-04-17T10:50:07.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:07.349+0000 7f6ec39b2640 1 -- 172.21.15.83:0/2536722978 wait complete. 2024-04-17T10:50:07.359 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-17T10:50:08.735 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:50:08.738 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:50:12.361 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:50:12.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4c4d22640 1 Processor -- start 2024-04-17T10:50:12.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4c4d22640 1 -- start start 2024-04-17T10:50:12.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4c4d22640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4c00ae5a0 0x7fe4c006adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:12.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4c4d22640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe4c00aea00 0x7fe4c006b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:12.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4c4d22640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4c00b5fb0 con 0x7fe4c015bbe0 2024-04-17T10:50:12.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4c4d22640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4c00670b0 con 0x7fe4c00aea00 2024-04-17T10:50:12.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4c4d22640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c00cff60 con 0x7fe4c00ae5a0 2024-04-17T10:50:12.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4be59c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe4c015bbe0 0x7fe4c00669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39810/0 (socket says 172.21.15.83:39810) 2024-04-17T10:50:12.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4be59c640 1 -- 172.21.15.83:0/2336309283 learned_addr learned my addr 172.21.15.83:0/2336309283 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:12.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4bd59a640 1 -- 172.21.15.83:0/2336309283 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2664938043 0 0) 0x7fe4b8001120 con 0x7fe4c015bbe0 2024-04-17T10:50:12.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4bd59a640 1 -- 172.21.15.83:0/2336309283 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4a8003530 con 0x7fe4c015bbe0 2024-04-17T10:50:12.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4bed9d640 1 --2- 172.21.15.83:0/2336309283 >> v2:172.21.15.167:3300/0 conn(0x7fe4c00aea00 0x7fe4c006b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:12.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4bdd9b640 1 --2- 172.21.15.83:0/2336309283 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4c00ae5a0 0x7fe4c006adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:12.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4bd59a640 1 -- 172.21.15.83:0/2336309283 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 669249902 0 0) 0x7fe4a8003530 con 0x7fe4c015bbe0 2024-04-17T10:50:12.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4bd59a640 1 -- 172.21.15.83:0/2336309283 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4c00ae5a0 msgr2=0x7fe4c006adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:12.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4bd59a640 1 --2- 172.21.15.83:0/2336309283 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4c00ae5a0 0x7fe4c006adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:12.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4bd59a640 1 -- 172.21.15.83:0/2336309283 >> v2:172.21.15.167:3300/0 conn(0x7fe4c00aea00 msgr2=0x7fe4c006b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:12.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4bd59a640 1 --2- 172.21.15.83:0/2336309283 >> v2:172.21.15.167:3300/0 conn(0x7fe4c00aea00 0x7fe4c006b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:12.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4bd59a640 1 -- 172.21.15.83:0/2336309283 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c006bcc0 con 0x7fe4c015bbe0 2024-04-17T10:50:12.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4bd59a640 1 -- 172.21.15.83:0/2336309283 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fe4b8001850 con 0x7fe4c015bbe0 2024-04-17T10:50:12.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4bd59a640 1 -- 172.21.15.83:0/2336309283 >> v1:172.21.15.83:6789/0 conn(0x7fe4c015bbe0 legacy=0x7fe4c00669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:12.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4bd59a640 1 --2- 172.21.15.83:0/2336309283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4a80039c0 0x7fe4a8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:12.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4bd59a640 1 --2- 172.21.15.83:0/2336309283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4a8004810 0x7fe4a8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:12.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4bd59a640 1 --2- 172.21.15.83:0/2336309283 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4c00ae5a0 0x7fe4a80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:12.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4bd59a640 1 -- 172.21.15.83:0/2336309283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c00670b0 con 0x7fe4a8004810 2024-04-17T10:50:12.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4bd59a640 1 -- 172.21.15.83:0/2336309283 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c00cff60 con 0x7fe4a80039c0 2024-04-17T10:50:12.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4bd59a640 1 -- 172.21.15.83:0/2336309283 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a800b9d0 con 0x7fe4c00ae5a0 2024-04-17T10:50:12.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4bdd9b640 1 --2- 172.21.15.83:0/2336309283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4a8004810 0x7fe4a8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:12.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4be59c640 1 --2- 172.21.15.83:0/2336309283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4a80039c0 0x7fe4a8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:12.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4bed9d640 1 --2- 172.21.15.83:0/2336309283 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4c00ae5a0 0x7fe4a80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:12.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4bdd9b640 1 -- 172.21.15.83:0/2336309283 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4c00ae5a0 msgr2=0x7fe4a80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:12.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4bdd9b640 1 --2- 172.21.15.83:0/2336309283 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4c00ae5a0 0x7fe4a80051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:12.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4bdd9b640 1 -- 172.21.15.83:0/2336309283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4a80039c0 msgr2=0x7fe4a8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:12.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4bdd9b640 1 --2- 172.21.15.83:0/2336309283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4a80039c0 0x7fe4a8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:12.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.493+0000 7fe4bdd9b640 1 -- 172.21.15.83:0/2336309283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a8003530 con 0x7fe4a8004810 2024-04-17T10:50:12.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.497+0000 7fe4bdd9b640 1 --2- 172.21.15.83:0/2336309283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4a8004810 0x7fe4a8004c90 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4009b10 tx=0x7fe4b401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fab1d92179aa11d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:12.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.497+0000 7fe4bd59a640 1 -- 172.21.15.83:0/2336309283 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b40096b0 con 0x7fe4a8004810 2024-04-17T10:50:12.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.497+0000 7fe4bd59a640 1 -- 172.21.15.83:0/2336309283 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b40a9470 con 0x7fe4a8004810 2024-04-17T10:50:12.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.497+0000 7fe4bd59a640 1 -- 172.21.15.83:0/2336309283 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b40a9860 con 0x7fe4a8004810 2024-04-17T10:50:12.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.497+0000 7fe4c4d22640 1 -- 172.21.15.83:0/2336309283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4a8004810 msgr2=0x7fe4a8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:12.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.497+0000 7fe4c4d22640 1 --2- 172.21.15.83:0/2336309283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4a8004810 0x7fe4a8004c90 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4009b10 tx=0x7fe4b401ca70 comp rx=0 tx=0).stop 2024-04-17T10:50:12.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.497+0000 7fe4bed9d640 1 -- 172.21.15.83:0/2336309283 reap_dead start 2024-04-17T10:50:12.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.497+0000 7fe4c4d22640 1 -- 172.21.15.83:0/2336309283 shutdown_connections 2024-04-17T10:50:12.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.497+0000 7fe4c4d22640 1 -- 172.21.15.83:0/2336309283 >> 172.21.15.83:0/2336309283 conn(0x7fe4c00b4eb0 msgr2=0x7fe4c00b5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:12.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.497+0000 7fe4c4d22640 1 -- 172.21.15.83:0/2336309283 shutdown_connections 2024-04-17T10:50:12.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.497+0000 7fe4c4d22640 1 -- 172.21.15.83:0/2336309283 wait complete. 2024-04-17T10:50:12.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.497+0000 7fe4c4d22640 1 Processor -- start 2024-04-17T10:50:12.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.497+0000 7fe4c4d22640 1 -- start start 2024-04-17T10:50:12.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.501+0000 7fe4c4d22640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4a8004810 0x7fe4c014a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:12.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.501+0000 7fe4c4d22640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4c00cb6c0 0x7fe4c00cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:12.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.501+0000 7fe4c4d22640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4c00ce270 0x7fe4c00ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:12.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.501+0000 7fe4c4d22640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c015c140 con 0x7fe4c00ce270 2024-04-17T10:50:12.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.501+0000 7fe4c4d22640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c015bfc0 con 0x7fe4a8004810 2024-04-17T10:50:12.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.501+0000 7fe4c4d22640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c00d0be0 con 0x7fe4c00cb6c0 2024-04-17T10:50:12.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.501+0000 7fe4bed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4c00ce270 0x7fe4c00ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:12.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.501+0000 7fe4bed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4c00ce270 0x7fe4c00ce6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58684/0 (socket says 172.21.15.83:58684) 2024-04-17T10:50:12.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.501+0000 7fe4bed9d640 1 -- 172.21.15.83:0/250489953 learned_addr learned my addr 172.21.15.83:0/250489953 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:12.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.501+0000 7fe4be59c640 1 --2- 172.21.15.83:0/250489953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4a8004810 0x7fe4c014a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:12.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.501+0000 7fe4bdd9b640 1 --2- 172.21.15.83:0/250489953 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4c00cb6c0 0x7fe4c00cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:12.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.501+0000 7fe4bed9d640 1 -- 172.21.15.83:0/250489953 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4c00cb6c0 msgr2=0x7fe4c00cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:12.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.501+0000 7fe4bed9d640 1 --2- 172.21.15.83:0/250489953 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4c00cb6c0 0x7fe4c00cbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:12.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.501+0000 7fe4bed9d640 1 -- 172.21.15.83:0/250489953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4a8004810 msgr2=0x7fe4c014a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:12.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.501+0000 7fe4bed9d640 1 --2- 172.21.15.83:0/250489953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4a8004810 0x7fe4c014a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:12.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.501+0000 7fe4bed9d640 1 -- 172.21.15.83:0/250489953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c00ca750 con 0x7fe4c00ce270 2024-04-17T10:50:12.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.501+0000 7fe4bdd9b640 1 --2- 172.21.15.83:0/250489953 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4c00cb6c0 0x7fe4c00cbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:12.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.501+0000 7fe4bed9d640 1 --2- 172.21.15.83:0/250489953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4c00ce270 0x7fe4c00ce6f0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0074620 tx=0x7fe4b000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:12.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.501+0000 7fe4aeffd640 1 -- 172.21.15.83:0/250489953 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b0016620 con 0x7fe4c00ce270 2024-04-17T10:50:12.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.501+0000 7fe4aeffd640 1 -- 172.21.15.83:0/250489953 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b001b070 con 0x7fe4c00ce270 2024-04-17T10:50:12.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.501+0000 7fe4c4d22640 1 -- 172.21.15.83:0/250489953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c00c93d0 con 0x7fe4c00ce270 2024-04-17T10:50:12.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.501+0000 7fe4aeffd640 1 -- 172.21.15.83:0/250489953 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b0016a20 con 0x7fe4c00ce270 2024-04-17T10:50:12.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.501+0000 7fe4c4d22640 1 -- 172.21.15.83:0/250489953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c00c9990 con 0x7fe4c00ce270 2024-04-17T10:50:12.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.501+0000 7fe4c4d22640 1 -- 172.21.15.83:0/250489953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4c0074f90 con 0x7fe4c00ce270 2024-04-17T10:50:12.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.501+0000 7fe4aeffd640 1 -- 172.21.15.83:0/250489953 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe4b0016bc0 con 0x7fe4c00ce270 2024-04-17T10:50:12.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.501+0000 7fe4aeffd640 1 --2- 172.21.15.83:0/250489953 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe4a0066f60 0x7fe4a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:12.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.501+0000 7fe4be59c640 1 --2- 172.21.15.83:0/250489953 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe4a0066f60 0x7fe4a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:12.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.501+0000 7fe4aeffd640 1 -- 172.21.15.83:0/250489953 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fe4b0085b40 con 0x7fe4c00ce270 2024-04-17T10:50:12.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.501+0000 7fe4be59c640 1 --2- 172.21.15.83:0/250489953 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe4a0066f60 0x7fe4a0069420 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8001060 tx=0x7fe4b8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:12.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.505+0000 7fe4aeffd640 1 -- 172.21.15.83:0/250489953 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe4b0024020 con 0x7fe4c00ce270 2024-04-17T10:50:12.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.697+0000 7fe4c4d22640 1 -- 172.21.15.83:0/250489953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4c0068140 con 0x7fe4c00ce270 2024-04-17T10:50:12.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.697+0000 7fe4aeffd640 1 -- 172.21.15.83:0/250489953 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4157 (secure 0 0 0) 0x7fe4b0056b50 con 0x7fe4c00ce270 2024-04-17T10:50:12.691 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:50:12.691 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5942,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3888166846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3888166846},{"type":"v1","addr":"172.21.15.167:6833","nonce":3888166846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":6772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1167913722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1167913722},{"type":"v1","addr":"172.21.15.167:6835","nonce":1167913722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:49:19.962680+0000","modified":"2024-04-17T10:50:12.167656+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9192},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9192":{"gid":9192,"name":"a","rank":0,"incarnation":77,"state":"up:replay","state_seq":1,"addr":"172.21.15.83:6835/3552333026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3552333026},{"type":"v1","addr":"172.21.15.83:6835","nonce":3552333026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-17T10:50:12.691 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 77 2024-04-17T10:50:12.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.701+0000 7fe4c4d22640 1 -- 172.21.15.83:0/250489953 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe4a0066f60 msgr2=0x7fe4a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:12.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.701+0000 7fe4c4d22640 1 --2- 172.21.15.83:0/250489953 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe4a0066f60 0x7fe4a0069420 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8001060 tx=0x7fe4b8009040 comp rx=0 tx=0).stop 2024-04-17T10:50:12.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.701+0000 7fe4c4d22640 1 -- 172.21.15.83:0/250489953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4c00ce270 msgr2=0x7fe4c00ce6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:12.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.701+0000 7fe4c4d22640 1 --2- 172.21.15.83:0/250489953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4c00ce270 0x7fe4c00ce6f0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0074620 tx=0x7fe4b000f520 comp rx=0 tx=0).stop 2024-04-17T10:50:12.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.701+0000 7fe4c4d22640 1 -- 172.21.15.83:0/250489953 shutdown_connections 2024-04-17T10:50:12.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.701+0000 7fe4c4d22640 1 --2- 172.21.15.83:0/250489953 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe4a0066f60 0x7fe4a0069420 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:12.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.701+0000 7fe4c4d22640 1 --2- 172.21.15.83:0/250489953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4c00ce270 0x7fe4c00ce6f0 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:12.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.701+0000 7fe4c4d22640 1 --2- 172.21.15.83:0/250489953 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4c00cb6c0 0x7fe4c00cbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:12.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.701+0000 7fe4c4d22640 1 --2- 172.21.15.83:0/250489953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4a8004810 0x7fe4c014a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:12.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.701+0000 7fe4c4d22640 1 -- 172.21.15.83:0/250489953 >> 172.21.15.83:0/250489953 conn(0x7fe4c00b4eb0 msgr2=0x7fe4c00669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:12.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.701+0000 7fe4c4d22640 1 -- 172.21.15.83:0/250489953 shutdown_connections 2024-04-17T10:50:12.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:12.701+0000 7fe4c4d22640 1 -- 172.21.15.83:0/250489953 wait complete. 2024-04-17T10:50:12.710 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-17T10:50:16.586 INFO:tasks.ceph.mds.a.smithi083.stderr:2024-04-17T10:50:16.592+0000 7fb86a2b6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T10:50:17.710 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:50:17.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.848+0000 7f60eda43640 1 Processor -- start 2024-04-17T10:50:17.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.848+0000 7f60eda43640 1 -- start start 2024-04-17T10:50:17.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.848+0000 7f60eda43640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f60e8070b80 0x7f60e8164d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:17.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.848+0000 7f60eda43640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60e80b2bd0 con 0x7f60e80cf060 2024-04-17T10:50:17.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.848+0000 7f60eda43640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60e80b2eb0 con 0x7f60e80b3990 2024-04-17T10:50:17.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.848+0000 7f60eda43640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60e80b2980 con 0x7f60e8070b80 2024-04-17T10:50:17.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.848+0000 7f60e77fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f60e8070b80 0x7f60e8164d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:17.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.848+0000 7f60e77fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f60e8070b80 0x7f60e8164d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40474/0 (socket says 172.21.15.83:40474) 2024-04-17T10:50:17.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.848+0000 7f60e77fe640 1 -- 172.21.15.83:0/3089376497 learned_addr learned my addr 172.21.15.83:0/3089376497 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:17.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.852+0000 7f60e77fe640 1 -- 172.21.15.83:0/3089376497 >> v1:172.21.15.167:6789/0 conn(0x7f60e80b3990 legacy=0x7f60e8070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:17.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.852+0000 7f60e77fe640 1 -- 172.21.15.83:0/3089376497 >> v1:172.21.15.83:6789/0 conn(0x7f60e80cf060 legacy=0x7f60e80b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:17.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.852+0000 7f60e77fe640 1 -- 172.21.15.83:0/3089376497 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e80b3190 con 0x7f60e8070b80 2024-04-17T10:50:17.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.852+0000 7f60e77fe640 1 --2- 172.21.15.83:0/3089376497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60e8070b80 0x7f60e8164d80 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f60e80b2b00 tx=0x7f60e001ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3606e2932419809 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:17.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.852+0000 7f60e5ffb640 1 -- 172.21.15.83:0/3089376497 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e00ad070 con 0x7f60e8070b80 2024-04-17T10:50:17.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.852+0000 7f60e5ffb640 1 -- 172.21.15.83:0/3089376497 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60e00a7c50 con 0x7f60e8070b80 2024-04-17T10:50:17.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.852+0000 7f60e5ffb640 1 -- 172.21.15.83:0/3089376497 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e00b1400 con 0x7f60e8070b80 2024-04-17T10:50:17.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.852+0000 7f60eda43640 1 -- 172.21.15.83:0/3089376497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60e8070b80 msgr2=0x7f60e8164d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:17.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.852+0000 7f60eda43640 1 --2- 172.21.15.83:0/3089376497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60e8070b80 0x7f60e8164d80 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f60e80b2b00 tx=0x7f60e001ebe0 comp rx=0 tx=0).stop 2024-04-17T10:50:17.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.852+0000 7f60eda43640 1 -- 172.21.15.83:0/3089376497 shutdown_connections 2024-04-17T10:50:17.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.852+0000 7f60eda43640 1 --2- 172.21.15.83:0/3089376497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60e8070b80 0x7f60e8164d80 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:17.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.852+0000 7f60eda43640 1 -- 172.21.15.83:0/3089376497 >> 172.21.15.83:0/3089376497 conn(0x7f60e806abe0 msgr2=0x7f60e806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:17.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.852+0000 7f60eda43640 1 -- 172.21.15.83:0/3089376497 shutdown_connections 2024-04-17T10:50:17.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.852+0000 7f60eda43640 1 -- 172.21.15.83:0/3089376497 wait complete. 2024-04-17T10:50:17.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.856+0000 7f60eda43640 1 Processor -- start 2024-04-17T10:50:17.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.856+0000 7f60eda43640 1 -- start start 2024-04-17T10:50:17.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.856+0000 7f60eda43640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60e8070b80 0x7f60e8150250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:17.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.856+0000 7f60eda43640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60e80b3990 0x7f60e81530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:17.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.856+0000 7f60eda43640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60e80cf060 0x7f60e80abfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:17.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.856+0000 7f60eda43640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e8167450 con 0x7f60e8070b80 2024-04-17T10:50:17.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.856+0000 7f60eda43640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e81672d0 con 0x7f60e80b3990 2024-04-17T10:50:17.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.856+0000 7f60eda43640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60e81675d0 con 0x7f60e80cf060 2024-04-17T10:50:17.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.856+0000 7f60e6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60e8070b80 0x7f60e8150250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:17.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.856+0000 7f60e6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60e8070b80 0x7f60e8150250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40480/0 (socket says 172.21.15.83:40480) 2024-04-17T10:50:17.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.856+0000 7f60e6ffd640 1 -- 172.21.15.83:0/184231676 learned_addr learned my addr 172.21.15.83:0/184231676 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:17.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.856+0000 7f60e77fe640 1 --2- 172.21.15.83:0/184231676 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60e80cf060 0x7f60e80abfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:17.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.856+0000 7f60e6ffd640 1 -- 172.21.15.83:0/184231676 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60e80cf060 msgr2=0x7f60e80abfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:17.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.856+0000 7f60e6ffd640 1 --2- 172.21.15.83:0/184231676 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60e80cf060 0x7f60e80abfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:17.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.856+0000 7f60e6ffd640 1 -- 172.21.15.83:0/184231676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60e80b3990 msgr2=0x7f60e81530f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:17.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.856+0000 7f60e6ffd640 1 --2- 172.21.15.83:0/184231676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60e80b3990 0x7f60e81530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:17.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.856+0000 7f60e6ffd640 1 -- 172.21.15.83:0/184231676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e8153630 con 0x7f60e8070b80 2024-04-17T10:50:17.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.856+0000 7f60e6ffd640 1 --2- 172.21.15.83:0/184231676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60e8070b80 0x7f60e8150250 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f60d800c3f0 tx=0x7f60d800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:17.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.856+0000 7f60cffff640 1 -- 172.21.15.83:0/184231676 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d8017070 con 0x7f60e8070b80 2024-04-17T10:50:17.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.856+0000 7f60cffff640 1 -- 172.21.15.83:0/184231676 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60d80052d0 con 0x7f60e8070b80 2024-04-17T10:50:17.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.856+0000 7f60eda43640 1 -- 172.21.15.83:0/184231676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60e81538b0 con 0x7f60e8070b80 2024-04-17T10:50:17.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.856+0000 7f60cffff640 1 -- 172.21.15.83:0/184231676 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d8003840 con 0x7f60e8070b80 2024-04-17T10:50:17.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.856+0000 7f60eda43640 1 -- 172.21.15.83:0/184231676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60e8153e70 con 0x7f60e8070b80 2024-04-17T10:50:17.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.856+0000 7f60eda43640 1 -- 172.21.15.83:0/184231676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60e8074f90 con 0x7f60e8070b80 2024-04-17T10:50:17.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.860+0000 7f60cffff640 1 -- 172.21.15.83:0/184231676 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f60d801a070 con 0x7f60e8070b80 2024-04-17T10:50:17.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.860+0000 7f60cffff640 1 --2- 172.21.15.83:0/184231676 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f60d4066f60 0x7f60d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:17.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.860+0000 7f60cffff640 1 -- 172.21.15.83:0/184231676 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f60d8084440 con 0x7f60e8070b80 2024-04-17T10:50:17.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.860+0000 7f60e67fc640 1 --2- 172.21.15.83:0/184231676 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f60d4066f60 0x7f60d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:17.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.860+0000 7f60e67fc640 1 --2- 172.21.15.83:0/184231676 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f60d4066f60 0x7f60d4069420 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f60dc000f70 tx=0x7f60dc0079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:17.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:17.864+0000 7f60cffff640 1 -- 172.21.15.83:0/184231676 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f60e8153630 con 0x7f60e8070b80 2024-04-17T10:50:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.056+0000 7f60eda43640 1 -- 172.21.15.83:0/184231676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60e8154070 con 0x7f60e8070b80 2024-04-17T10:50:18.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.056+0000 7f60cffff640 1 -- 172.21.15.83:0/184231676 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4158 (secure 0 0 0) 0x7f60d8005480 con 0x7f60e8070b80 2024-04-17T10:50:18.097 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:50:18.097 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":80,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5942,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3888166846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3888166846},{"type":"v1","addr":"172.21.15.167:6833","nonce":3888166846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":6772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1167913722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1167913722},{"type":"v1","addr":"172.21.15.167:6835","nonce":1167913722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:49:19.962680+0000","modified":"2024-04-17T10:50:14.204149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9192},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9192":{"gid":9192,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":18,"addr":"172.21.15.83:6835/3552333026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3552333026},{"type":"v1","addr":"172.21.15.83:6835","nonce":3552333026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-17T10:50:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 80 2024-04-17T10:50:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.060+0000 7f60eda43640 1 -- 172.21.15.83:0/184231676 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f60d4066f60 msgr2=0x7f60d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.060+0000 7f60eda43640 1 --2- 172.21.15.83:0/184231676 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f60d4066f60 0x7f60d4069420 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f60dc000f70 tx=0x7f60dc0079c0 comp rx=0 tx=0).stop 2024-04-17T10:50:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.060+0000 7f60eda43640 1 -- 172.21.15.83:0/184231676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60e8070b80 msgr2=0x7f60e8150250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.060+0000 7f60eda43640 1 --2- 172.21.15.83:0/184231676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60e8070b80 0x7f60e8150250 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f60d800c3f0 tx=0x7f60d800c8b0 comp rx=0 tx=0).stop 2024-04-17T10:50:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.060+0000 7f60eda43640 1 -- 172.21.15.83:0/184231676 shutdown_connections 2024-04-17T10:50:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.060+0000 7f60eda43640 1 --2- 172.21.15.83:0/184231676 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f60d4066f60 0x7f60d4069420 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.060+0000 7f60eda43640 1 --2- 172.21.15.83:0/184231676 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60e80cf060 0x7f60e80abfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.060+0000 7f60eda43640 1 --2- 172.21.15.83:0/184231676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60e80b3990 0x7f60e81530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.060+0000 7f60eda43640 1 --2- 172.21.15.83:0/184231676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60e8070b80 0x7f60e8150250 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.060+0000 7f60eda43640 1 -- 172.21.15.83:0/184231676 >> 172.21.15.83:0/184231676 conn(0x7f60e806abe0 msgr2=0x7f60e806cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.060+0000 7f60eda43640 1 -- 172.21.15.83:0/184231676 shutdown_connections 2024-04-17T10:50:18.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.060+0000 7f60eda43640 1 -- 172.21.15.83:0/184231676 wait complete. 2024-04-17T10:50:18.100 DEBUG:tasks.ceph_test_case:wait_until_true: success in 20s and 0 retries 2024-04-17T10:50:18.100 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:50:18.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f54317b3640 1 Processor -- start 2024-04-17T10:50:18.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f54317b3640 1 -- start start 2024-04-17T10:50:18.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f54317b3640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f542c0ce8c0 0x7f542c0b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:18.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f54317b3640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f542c0b2c40 0x7f542c0b1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:18.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f54317b3640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f542c06c910 con 0x7f542c15dcf0 2024-04-17T10:50:18.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f54317b3640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f542c0b2a40 con 0x7f542c0b2c40 2024-04-17T10:50:18.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f54317b3640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542c06c6d0 con 0x7f542c0ce8c0 2024-04-17T10:50:18.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f542affd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f542c15dcf0 0x7f542c0ce1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38972/0 (socket says 172.21.15.83:38972) 2024-04-17T10:50:18.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f542affd640 1 -- 172.21.15.83:0/460183468 learned_addr learned my addr 172.21.15.83:0/460183468 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:18.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f542a7fc640 1 --2- 172.21.15.83:0/460183468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f542c0ce8c0 0x7f542c0b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:18.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f542b7fe640 1 --2- 172.21.15.83:0/460183468 >> v2:172.21.15.167:3300/0 conn(0x7f542c0b2c40 0x7f542c0b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:18.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f5429ffb640 1 -- 172.21.15.83:0/460183468 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3045032339 0 0) 0x7f542c06c910 con 0x7f542c15dcf0 2024-04-17T10:50:18.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f5429ffb640 1 -- 172.21.15.83:0/460183468 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5414003530 con 0x7f542c15dcf0 2024-04-17T10:50:18.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f5429ffb640 1 -- 172.21.15.83:0/460183468 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1797227367 0 0) 0x7f5414003530 con 0x7f542c15dcf0 2024-04-17T10:50:18.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f5429ffb640 1 -- 172.21.15.83:0/460183468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f542c0ce8c0 msgr2=0x7f542c0b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:18.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f5429ffb640 1 --2- 172.21.15.83:0/460183468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f542c0ce8c0 0x7f542c0b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:18.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f5429ffb640 1 -- 172.21.15.83:0/460183468 >> v2:172.21.15.167:3300/0 conn(0x7f542c0b2c40 msgr2=0x7f542c0b1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:18.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f5429ffb640 1 --2- 172.21.15.83:0/460183468 >> v2:172.21.15.167:3300/0 conn(0x7f542c0b2c40 0x7f542c0b1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:18.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f5429ffb640 1 -- 172.21.15.83:0/460183468 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542c06cbf0 con 0x7f542c15dcf0 2024-04-17T10:50:18.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f5429ffb640 1 -- 172.21.15.83:0/460183468 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f5420001680 con 0x7f542c15dcf0 2024-04-17T10:50:18.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f5429ffb640 1 -- 172.21.15.83:0/460183468 >> v1:172.21.15.83:6789/0 conn(0x7f542c15dcf0 legacy=0x7f542c0ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:18.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f5429ffb640 1 --2- 172.21.15.83:0/460183468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54140039c0 0x7f5414003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:18.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f5429ffb640 1 --2- 172.21.15.83:0/460183468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5414004810 0x7f5414004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:18.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f5429ffb640 1 --2- 172.21.15.83:0/460183468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f542c0ce8c0 0x7f54140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:18.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f5429ffb640 1 -- 172.21.15.83:0/460183468 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c06c6d0 con 0x7f54140039c0 2024-04-17T10:50:18.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f5429ffb640 1 -- 172.21.15.83:0/460183468 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541400b9d0 con 0x7f5414004810 2024-04-17T10:50:18.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f5429ffb640 1 -- 172.21.15.83:0/460183468 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f541400bb50 con 0x7f542c0ce8c0 2024-04-17T10:50:18.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f542affd640 1 --2- 172.21.15.83:0/460183468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54140039c0 0x7f5414003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:18.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f542a7fc640 1 --2- 172.21.15.83:0/460183468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5414004810 0x7f5414004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:18.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f542affd640 1 -- 172.21.15.83:0/460183468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f542c0ce8c0 msgr2=0x7f54140051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:18.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f542affd640 1 --2- 172.21.15.83:0/460183468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f542c0ce8c0 0x7f54140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:18.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f542affd640 1 -- 172.21.15.83:0/460183468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5414004810 msgr2=0x7f5414004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:18.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f542affd640 1 --2- 172.21.15.83:0/460183468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5414004810 0x7f5414004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:18.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.232+0000 7f542affd640 1 -- 172.21.15.83:0/460183468 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5414003530 con 0x7f54140039c0 2024-04-17T10:50:18.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.236+0000 7f542affd640 1 --2- 172.21.15.83:0/460183468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54140039c0 0x7f5414003de0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f542000c270 tx=0x7f542001cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db9a434b6b20d78 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:18.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.236+0000 7f5429ffb640 1 -- 172.21.15.83:0/460183468 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5420007d40 con 0x7f54140039c0 2024-04-17T10:50:18.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.236+0000 7f5429ffb640 1 -- 172.21.15.83:0/460183468 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54200a9020 con 0x7f54140039c0 2024-04-17T10:50:18.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.236+0000 7f5429ffb640 1 -- 172.21.15.83:0/460183468 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5420005b00 con 0x7f54140039c0 2024-04-17T10:50:18.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.236+0000 7f54317b3640 1 -- 172.21.15.83:0/460183468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54140039c0 msgr2=0x7f5414003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:18.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.236+0000 7f54317b3640 1 --2- 172.21.15.83:0/460183468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54140039c0 0x7f5414003de0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f542000c270 tx=0x7f542001cca0 comp rx=0 tx=0).stop 2024-04-17T10:50:18.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.236+0000 7f542b7fe640 1 -- 172.21.15.83:0/460183468 reap_dead start 2024-04-17T10:50:18.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.236+0000 7f54317b3640 1 -- 172.21.15.83:0/460183468 shutdown_connections 2024-04-17T10:50:18.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.236+0000 7f54317b3640 1 -- 172.21.15.83:0/460183468 >> 172.21.15.83:0/460183468 conn(0x7f542c0b32b0 msgr2=0x7f542c0b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:18.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.236+0000 7f54317b3640 1 -- 172.21.15.83:0/460183468 shutdown_connections 2024-04-17T10:50:18.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.236+0000 7f54317b3640 1 -- 172.21.15.83:0/460183468 wait complete. 2024-04-17T10:50:18.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.236+0000 7f54317b3640 1 Processor -- start 2024-04-17T10:50:18.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.236+0000 7f54317b3640 1 -- start start 2024-04-17T10:50:18.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.236+0000 7f54317b3640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f542c157a60 0x7f542c0b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:18.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.236+0000 7f54317b3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f542c0b4390 0x7f542c0b5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:18.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.236+0000 7f54317b3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f542c158070 0x7f542c0af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:18.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.236+0000 7f54317b3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c0d0bb0 con 0x7f542c158070 2024-04-17T10:50:18.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.236+0000 7f54317b3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c0d0a30 con 0x7f542c0b4390 2024-04-17T10:50:18.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.236+0000 7f54317b3640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542c0d0d30 con 0x7f542c157a60 2024-04-17T10:50:18.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.236+0000 7f542b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f542c158070 0x7f542c0af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:18.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.236+0000 7f542b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f542c158070 0x7f542c0af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40486/0 (socket says 172.21.15.83:40486) 2024-04-17T10:50:18.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.236+0000 7f542b7fe640 1 -- 172.21.15.83:0/109338146 learned_addr learned my addr 172.21.15.83:0/109338146 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:18.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.236+0000 7f542a7fc640 1 --2- 172.21.15.83:0/109338146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f542c0b4390 0x7f542c0b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:18.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.236+0000 7f542affd640 1 --2- 172.21.15.83:0/109338146 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f542c157a60 0x7f542c0b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:18.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.236+0000 7f542b7fe640 1 -- 172.21.15.83:0/109338146 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f542c157a60 msgr2=0x7f542c0b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:18.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.236+0000 7f542b7fe640 1 --2- 172.21.15.83:0/109338146 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f542c157a60 0x7f542c0b4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:18.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.236+0000 7f542b7fe640 1 -- 172.21.15.83:0/109338146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f542c0b4390 msgr2=0x7f542c0b5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:18.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.236+0000 7f542b7fe640 1 --2- 172.21.15.83:0/109338146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f542c0b4390 0x7f542c0b5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:18.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.236+0000 7f542b7fe640 1 -- 172.21.15.83:0/109338146 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542c158800 con 0x7f542c158070 2024-04-17T10:50:18.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.236+0000 7f542affd640 1 --2- 172.21.15.83:0/109338146 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f542c157a60 0x7f542c0b4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.236+0000 7f542a7fc640 1 --2- 172.21.15.83:0/109338146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f542c0b4390 0x7f542c0b5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.240+0000 7f542b7fe640 1 --2- 172.21.15.83:0/109338146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f542c158070 0x7f542c0af8e0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f5424000f90 tx=0x7f542400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.240+0000 7f54137fe640 1 -- 172.21.15.83:0/109338146 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542400ba80 con 0x7f542c158070 2024-04-17T10:50:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.240+0000 7f54137fe640 1 -- 172.21.15.83:0/109338146 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5424019070 con 0x7f542c158070 2024-04-17T10:50:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.240+0000 7f54317b3640 1 -- 172.21.15.83:0/109338146 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542c157e40 con 0x7f542c158070 2024-04-17T10:50:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.240+0000 7f54137fe640 1 -- 172.21.15.83:0/109338146 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542400be60 con 0x7f542c158070 2024-04-17T10:50:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.240+0000 7f54317b3640 1 -- 172.21.15.83:0/109338146 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f542c0af600 con 0x7f542c158070 2024-04-17T10:50:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.240+0000 7f54317b3640 1 -- 172.21.15.83:0/109338146 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f542c074f90 con 0x7f542c158070 2024-04-17T10:50:18.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.240+0000 7f54137fe640 1 -- 172.21.15.83:0/109338146 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5424013070 con 0x7f542c158070 2024-04-17T10:50:18.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.240+0000 7f54137fe640 1 --2- 172.21.15.83:0/109338146 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5418066f60 0x7f5418069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:18.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.240+0000 7f54137fe640 1 -- 172.21.15.83:0/109338146 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f5424085120 con 0x7f542c158070 2024-04-17T10:50:18.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.240+0000 7f542affd640 1 --2- 172.21.15.83:0/109338146 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5418066f60 0x7f5418069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:18.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.240+0000 7f542affd640 1 --2- 172.21.15.83:0/109338146 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5418066f60 0x7f5418069420 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f54200aa000 tx=0x7f542c067830 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:18.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.244+0000 7f54137fe640 1 -- 172.21.15.83:0/109338146 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f54240122c0 con 0x7f542c158070 2024-04-17T10:50:18.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.436+0000 7f54317b3640 1 -- 172.21.15.83:0/109338146 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f542c06c220 con 0x7f542c158070 2024-04-17T10:50:18.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.436+0000 7f54137fe640 1 -- 172.21.15.83:0/109338146 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4158 (secure 0 0 0) 0x7f5424056020 con 0x7f542c158070 2024-04-17T10:50:18.431 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:50:18.431 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":80,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5942,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3888166846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3888166846},{"type":"v1","addr":"172.21.15.167:6833","nonce":3888166846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":6772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1167913722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1167913722},{"type":"v1","addr":"172.21.15.167:6835","nonce":1167913722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:49:19.962680+0000","modified":"2024-04-17T10:50:14.204149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9192},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9192":{"gid":9192,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":18,"addr":"172.21.15.83:6835/3552333026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3552333026},{"type":"v1","addr":"172.21.15.83:6835","nonce":3552333026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-17T10:50:18.432 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 80 2024-04-17T10:50:18.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.440+0000 7f54317b3640 1 -- 172.21.15.83:0/109338146 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5418066f60 msgr2=0x7f5418069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:18.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.440+0000 7f54317b3640 1 --2- 172.21.15.83:0/109338146 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5418066f60 0x7f5418069420 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f54200aa000 tx=0x7f542c067830 comp rx=0 tx=0).stop 2024-04-17T10:50:18.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.440+0000 7f54317b3640 1 -- 172.21.15.83:0/109338146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f542c158070 msgr2=0x7f542c0af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:18.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.440+0000 7f54317b3640 1 --2- 172.21.15.83:0/109338146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f542c158070 0x7f542c0af8e0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f5424000f90 tx=0x7f542400b480 comp rx=0 tx=0).stop 2024-04-17T10:50:18.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.440+0000 7f54317b3640 1 -- 172.21.15.83:0/109338146 shutdown_connections 2024-04-17T10:50:18.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.440+0000 7f54317b3640 1 --2- 172.21.15.83:0/109338146 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5418066f60 0x7f5418069420 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:18.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.440+0000 7f54317b3640 1 --2- 172.21.15.83:0/109338146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f542c158070 0x7f542c0af8e0 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:18.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.440+0000 7f54317b3640 1 --2- 172.21.15.83:0/109338146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f542c0b4390 0x7f542c0b5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:18.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.440+0000 7f54317b3640 1 --2- 172.21.15.83:0/109338146 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f542c157a60 0x7f542c0b4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:18.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.440+0000 7f54317b3640 1 -- 172.21.15.83:0/109338146 >> 172.21.15.83:0/109338146 conn(0x7f542c0b32b0 msgr2=0x7f542c0b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:18.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.440+0000 7f54317b3640 1 -- 172.21.15.83:0/109338146 shutdown_connections 2024-04-17T10:50:18.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.440+0000 7f54317b3640 1 -- 172.21.15.83:0/109338146 wait complete. 2024-04-17T10:50:18.449 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-17T10:50:18.450 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-17T10:50:18.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.568+0000 7f1c529fc640 1 Processor -- start 2024-04-17T10:50:18.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.568+0000 7f1c529fc640 1 -- start start 2024-04-17T10:50:18.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.568+0000 7f1c529fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1c4c072330 0x7f1c4c0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:18.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.568+0000 7f1c529fc640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c4c0cfea0 con 0x7f1c4c15ba30 2024-04-17T10:50:18.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.568+0000 7f1c529fc640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c4c06e320 con 0x7f1c4c0cfa10 2024-04-17T10:50:18.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.568+0000 7f1c529fc640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c4c071ea0 con 0x7f1c4c072330 2024-04-17T10:50:18.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.568+0000 7f1c4b7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1c4c072330 0x7f1c4c0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:18.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.568+0000 7f1c4b7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1c4c072330 0x7f1c4c0cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40492/0 (socket says 172.21.15.83:40492) 2024-04-17T10:50:18.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.568+0000 7f1c4b7fe640 1 -- 172.21.15.83:0/1665577266 learned_addr learned my addr 172.21.15.83:0/1665577266 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:18.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.572+0000 7f1c4b7fe640 1 -- 172.21.15.83:0/1665577266 >> v1:172.21.15.167:6789/0 conn(0x7f1c4c0cfa10 legacy=0x7f1c4c06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:18.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.572+0000 7f1c4b7fe640 1 -- 172.21.15.83:0/1665577266 >> v1:172.21.15.83:6789/0 conn(0x7f1c4c15ba30 legacy=0x7f1c4c15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:18.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.572+0000 7f1c4b7fe640 1 -- 172.21.15.83:0/1665577266 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4c06e040 con 0x7f1c4c072330 2024-04-17T10:50:18.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.572+0000 7f1c4b7fe640 1 --2- 172.21.15.83:0/1665577266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c4c072330 0x7f1c4c0cf4d0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f1c40009870 tx=0x7f1c4001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efec18b2aae64355 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:18.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.572+0000 7f1c4affd640 1 -- 172.21.15.83:0/1665577266 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c400b1070 con 0x7f1c4c072330 2024-04-17T10:50:18.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.572+0000 7f1c4affd640 1 -- 172.21.15.83:0/1665577266 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c400ac600 con 0x7f1c4c072330 2024-04-17T10:50:18.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.572+0000 7f1c4affd640 1 -- 172.21.15.83:0/1665577266 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c400ac9d0 con 0x7f1c4c072330 2024-04-17T10:50:18.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.572+0000 7f1c529fc640 1 -- 172.21.15.83:0/1665577266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c4c072330 msgr2=0x7f1c4c0cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:18.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.572+0000 7f1c529fc640 1 --2- 172.21.15.83:0/1665577266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c4c072330 0x7f1c4c0cf4d0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f1c40009870 tx=0x7f1c4001c490 comp rx=0 tx=0).stop 2024-04-17T10:50:18.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.572+0000 7f1c529fc640 1 -- 172.21.15.83:0/1665577266 shutdown_connections 2024-04-17T10:50:18.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.572+0000 7f1c529fc640 1 --2- 172.21.15.83:0/1665577266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c4c072330 0x7f1c4c0cf4d0 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:18.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.572+0000 7f1c529fc640 1 -- 172.21.15.83:0/1665577266 >> 172.21.15.83:0/1665577266 conn(0x7f1c4c06abe0 msgr2=0x7f1c4c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:18.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.572+0000 7f1c529fc640 1 -- 172.21.15.83:0/1665577266 shutdown_connections 2024-04-17T10:50:18.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.572+0000 7f1c529fc640 1 -- 172.21.15.83:0/1665577266 wait complete. 2024-04-17T10:50:18.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.576+0000 7f1c529fc640 1 Processor -- start 2024-04-17T10:50:18.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.576+0000 7f1c529fc640 1 -- start start 2024-04-17T10:50:18.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.576+0000 7f1c529fc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c4c072330 0x7f1c4c15c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:18.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.576+0000 7f1c529fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c4c0cfa10 0x7f1c4c1479d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:18.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.576+0000 7f1c529fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c4c15ba30 0x7f1c4c0b2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:18.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.576+0000 7f1c529fc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c06de20 con 0x7f1c4c0cfa10 2024-04-17T10:50:18.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.576+0000 7f1c529fc640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c06dca0 con 0x7f1c4c15ba30 2024-04-17T10:50:18.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.576+0000 7f1c529fc640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c169530 con 0x7f1c4c072330 2024-04-17T10:50:18.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.576+0000 7f1c4b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c4c0cfa10 0x7f1c4c1479d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:18.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.576+0000 7f1c4b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c4c0cfa10 0x7f1c4c1479d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40508/0 (socket says 172.21.15.83:40508) 2024-04-17T10:50:18.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.576+0000 7f1c4b7fe640 1 -- 172.21.15.83:0/441183164 learned_addr learned my addr 172.21.15.83:0/441183164 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:18.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.576+0000 7f1c4b7fe640 1 -- 172.21.15.83:0/441183164 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c4c072330 msgr2=0x7f1c4c15c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:18.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.576+0000 7f1c4b7fe640 1 --2- 172.21.15.83:0/441183164 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c4c072330 0x7f1c4c15c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:18.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.576+0000 7f1c4b7fe640 1 -- 172.21.15.83:0/441183164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c4c15ba30 msgr2=0x7f1c4c0b2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:18.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.576+0000 7f1c50f99640 1 --2- 172.21.15.83:0/441183164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c4c15ba30 0x7f1c4c0b2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:18.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.576+0000 7f1c4b7fe640 1 --2- 172.21.15.83:0/441183164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c4c15ba30 0x7f1c4c0b2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:18.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.576+0000 7f1c4b7fe640 1 -- 172.21.15.83:0/441183164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4c0b34a0 con 0x7f1c4c0cfa10 2024-04-17T10:50:18.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.576+0000 7f1c4b7fe640 1 --2- 172.21.15.83:0/441183164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c4c0cfa10 0x7f1c4c1479d0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f1c4001a0c0 tx=0x7f1c40019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:18.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.576+0000 7f1c48ff9640 1 -- 172.21.15.83:0/441183164 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c400b1040 con 0x7f1c4c0cfa10 2024-04-17T10:50:18.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.576+0000 7f1c48ff9640 1 -- 172.21.15.83:0/441183164 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c400bc070 con 0x7f1c4c0cfa10 2024-04-17T10:50:18.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.576+0000 7f1c529fc640 1 -- 172.21.15.83:0/441183164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c4c0b3780 con 0x7f1c4c0cfa10 2024-04-17T10:50:18.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.576+0000 7f1c48ff9640 1 -- 172.21.15.83:0/441183164 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c400b7890 con 0x7f1c4c0cfa10 2024-04-17T10:50:18.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.576+0000 7f1c529fc640 1 -- 172.21.15.83:0/441183164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c4c14ef40 con 0x7f1c4c0cfa10 2024-04-17T10:50:18.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.576+0000 7f1c529fc640 1 -- 172.21.15.83:0/441183164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c4c0b3950 con 0x7f1c4c0cfa10 2024-04-17T10:50:18.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.576+0000 7f1c48ff9640 1 -- 172.21.15.83:0/441183164 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1c400b7a30 con 0x7f1c4c0cfa10 2024-04-17T10:50:18.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.576+0000 7f1c48ff9640 1 --2- 172.21.15.83:0/441183164 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1c38066f60 0x7f1c38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:18.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.576+0000 7f1c48ff9640 1 -- 172.21.15.83:0/441183164 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f1c4011efd0 con 0x7f1c4c0cfa10 2024-04-17T10:50:18.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.576+0000 7f1c4bfff640 1 --2- 172.21.15.83:0/441183164 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1c38066f60 0x7f1c38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:18.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.580+0000 7f1c4bfff640 1 --2- 172.21.15.83:0/441183164 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1c38066f60 0x7f1c38069420 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c000f70 tx=0x7f1c3c0079e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:18.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.584+0000 7f1c48ff9640 1 -- 172.21.15.83:0/441183164 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1c40016110 con 0x7f1c4c0cfa10 2024-04-17T10:50:18.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.792+0000 7f1c529fc640 1 -- 172.21.15.83:0/441183164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f1c4c074f90 con 0x7f1c4c0cfa10 2024-04-17T10:50:18.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.792+0000 7f1c48ff9640 1 -- 172.21.15.83:0/441183164 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1206 (secure 0 0 0) 0x7f1c400f0070 con 0x7f1c4c0cfa10 2024-04-17T10:50:18.786 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:50:18.786 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":751,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":130,"num_osds":8,"num_up_osds":8,"osd_up_since":1713350279,"num_in_osds":8,"osd_in_since":1713350270,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1078426655,"bytes_used":2678358016,"bytes_avail":770415755264,"bytes_total":773094113280,"read_bytes_sec":410483,"write_bytes_sec":204,"read_op_per_sec":1,"write_op_per_sec":0},"fsmap":{"epoch":80,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":9192}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":27,"modified":"2024-04-17T10:50:18.395476+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-17T10:50:18.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.796+0000 7f1c529fc640 1 -- 172.21.15.83:0/441183164 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1c38066f60 msgr2=0x7f1c38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:18.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.796+0000 7f1c529fc640 1 --2- 172.21.15.83:0/441183164 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1c38066f60 0x7f1c38069420 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c000f70 tx=0x7f1c3c0079e0 comp rx=0 tx=0).stop 2024-04-17T10:50:18.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.796+0000 7f1c529fc640 1 -- 172.21.15.83:0/441183164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c4c0cfa10 msgr2=0x7f1c4c1479d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:18.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.796+0000 7f1c529fc640 1 --2- 172.21.15.83:0/441183164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c4c0cfa10 0x7f1c4c1479d0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f1c4001a0c0 tx=0x7f1c40019230 comp rx=0 tx=0).stop 2024-04-17T10:50:18.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.796+0000 7f1c529fc640 1 -- 172.21.15.83:0/441183164 shutdown_connections 2024-04-17T10:50:18.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.796+0000 7f1c529fc640 1 --2- 172.21.15.83:0/441183164 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1c38066f60 0x7f1c38069420 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:18.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.796+0000 7f1c529fc640 1 --2- 172.21.15.83:0/441183164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c4c15ba30 0x7f1c4c0b2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:18.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.796+0000 7f1c529fc640 1 --2- 172.21.15.83:0/441183164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c4c0cfa10 0x7f1c4c1479d0 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:18.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.796+0000 7f1c529fc640 1 --2- 172.21.15.83:0/441183164 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c4c072330 0x7f1c4c15c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:18.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.796+0000 7f1c529fc640 1 -- 172.21.15.83:0/441183164 >> 172.21.15.83:0/441183164 conn(0x7f1c4c06abe0 msgr2=0x7f1c4c15a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:18.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.796+0000 7f1c529fc640 1 -- 172.21.15.83:0/441183164 shutdown_connections 2024-04-17T10:50:18.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:18.796+0000 7f1c529fc640 1 -- 172.21.15.83:0/441183164 wait complete. 2024-04-17T10:50:18.803 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-17T10:50:18.803 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T10:50:18.803 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:50:18.861 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:50:18.916 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T10:50:18.916 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:50:18.916 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:50:18.922 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:50:19.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.084+0000 7fbe8f2c7640 1 Processor -- start 2024-04-17T10:50:19.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.084+0000 7fbe8f2c7640 1 -- start start 2024-04-17T10:50:19.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.088+0000 7fbe8f2c7640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbe8815db50 0x7fbe88070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:19.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.088+0000 7fbe8f2c7640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe880709b0 con 0x7fbe8806dcb0 2024-04-17T10:50:19.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.088+0000 7fbe8f2c7640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe8815e440 con 0x7fbe880aedd0 2024-04-17T10:50:19.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.088+0000 7fbe8f2c7640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe8815e720 con 0x7fbe8815db50 2024-04-17T10:50:19.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.088+0000 7fbe8d063640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbe8815db50 0x7fbe88070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:19.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.088+0000 7fbe8d063640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbe8815db50 0x7fbe88070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40514/0 (socket says 172.21.15.83:40514) 2024-04-17T10:50:19.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.088+0000 7fbe8d063640 1 -- 172.21.15.83:0/3888377771 learned_addr learned my addr 172.21.15.83:0/3888377771 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:19.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.088+0000 7fbe8d063640 1 -- 172.21.15.83:0/3888377771 >> v1:172.21.15.167:6789/0 conn(0x7fbe880aedd0 legacy=0x7fbe8806d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:19.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.088+0000 7fbe8d063640 1 -- 172.21.15.83:0/3888377771 >> v1:172.21.15.83:6789/0 conn(0x7fbe8806dcb0 legacy=0x7fbe880b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:19.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.088+0000 7fbe8d063640 1 -- 172.21.15.83:0/3888377771 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8815ea00 con 0x7fbe8815db50 2024-04-17T10:50:19.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.088+0000 7fbe8d063640 1 --2- 172.21.15.83:0/3888377771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe8815db50 0x7fbe88070470 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fbe840098a0 tx=0x7fbe8401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b419434cfce0f0f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:19.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.088+0000 7fbe7ffff640 1 -- 172.21.15.83:0/3888377771 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe840aa430 con 0x7fbe8815db50 2024-04-17T10:50:19.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.088+0000 7fbe7ffff640 1 -- 172.21.15.83:0/3888377771 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe840af070 con 0x7fbe8815db50 2024-04-17T10:50:19.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.088+0000 7fbe7ffff640 1 -- 172.21.15.83:0/3888377771 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe840aa7f0 con 0x7fbe8815db50 2024-04-17T10:50:19.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.092+0000 7fbe8f2c7640 1 -- 172.21.15.83:0/3888377771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe8815db50 msgr2=0x7fbe88070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:19.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.092+0000 7fbe8f2c7640 1 --2- 172.21.15.83:0/3888377771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe8815db50 0x7fbe88070470 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fbe840098a0 tx=0x7fbe8401c4d0 comp rx=0 tx=0).stop 2024-04-17T10:50:19.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.092+0000 7fbe8f2c7640 1 -- 172.21.15.83:0/3888377771 shutdown_connections 2024-04-17T10:50:19.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.092+0000 7fbe8f2c7640 1 --2- 172.21.15.83:0/3888377771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe8815db50 0x7fbe88070470 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:19.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.092+0000 7fbe8f2c7640 1 -- 172.21.15.83:0/3888377771 >> 172.21.15.83:0/3888377771 conn(0x7fbe8806abe0 msgr2=0x7fbe8806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:19.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.092+0000 7fbe8f2c7640 1 -- 172.21.15.83:0/3888377771 shutdown_connections 2024-04-17T10:50:19.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.092+0000 7fbe8f2c7640 1 -- 172.21.15.83:0/3888377771 wait complete. 2024-04-17T10:50:19.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.092+0000 7fbe8f2c7640 1 Processor -- start 2024-04-17T10:50:19.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.092+0000 7fbe8f2c7640 1 -- start start 2024-04-17T10:50:19.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.092+0000 7fbe8f2c7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe8806dcb0 0x7fbe881fa770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:19.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.092+0000 7fbe8f2c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe880aedd0 0x7fbe88142350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:19.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.092+0000 7fbe8f2c7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe8815db50 0x7fbe880a9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:19.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.092+0000 7fbe8f2c7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe880b3eb0 con 0x7fbe880aedd0 2024-04-17T10:50:19.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.092+0000 7fbe8f2c7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe880b3d30 con 0x7fbe8806dcb0 2024-04-17T10:50:19.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.092+0000 7fbe8f2c7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe880b4030 con 0x7fbe8815db50 2024-04-17T10:50:19.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.092+0000 7fbe8c862640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe880aedd0 0x7fbe88142350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:19.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.092+0000 7fbe8c862640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe880aedd0 0x7fbe88142350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40526/0 (socket says 172.21.15.83:40526) 2024-04-17T10:50:19.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.092+0000 7fbe8c862640 1 -- 172.21.15.83:0/393468123 learned_addr learned my addr 172.21.15.83:0/393468123 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:19.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.092+0000 7fbe8c862640 1 -- 172.21.15.83:0/393468123 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe8815db50 msgr2=0x7fbe880a9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:19.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.092+0000 7fbe8c862640 1 --2- 172.21.15.83:0/393468123 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe8815db50 0x7fbe880a9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:19.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.092+0000 7fbe8c862640 1 -- 172.21.15.83:0/393468123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe8806dcb0 msgr2=0x7fbe881fa770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:19.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.092+0000 7fbe8c862640 1 --2- 172.21.15.83:0/393468123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe8806dcb0 0x7fbe881fa770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:19.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.092+0000 7fbe8c862640 1 -- 172.21.15.83:0/393468123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8815c7d0 con 0x7fbe880aedd0 2024-04-17T10:50:19.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.092+0000 7fbe8c862640 1 --2- 172.21.15.83:0/393468123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe880aedd0 0x7fbe88142350 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800c4c0 tx=0x7fbe78001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:19.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.092+0000 7fbe7dffb640 1 -- 172.21.15.83:0/393468123 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe78017070 con 0x7fbe880aedd0 2024-04-17T10:50:19.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.092+0000 7fbe7dffb640 1 -- 172.21.15.83:0/393468123 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe78003600 con 0x7fbe880aedd0 2024-04-17T10:50:19.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.092+0000 7fbe8f2c7640 1 -- 172.21.15.83:0/393468123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe8815cab0 con 0x7fbe880aedd0 2024-04-17T10:50:19.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.092+0000 7fbe7dffb640 1 -- 172.21.15.83:0/393468123 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe780039f0 con 0x7fbe880aedd0 2024-04-17T10:50:19.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.092+0000 7fbe8f2c7640 1 -- 172.21.15.83:0/393468123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe881f9f70 con 0x7fbe880aedd0 2024-04-17T10:50:19.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.092+0000 7fbe8f2c7640 1 -- 172.21.15.83:0/393468123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe88074f90 con 0x7fbe880aedd0 2024-04-17T10:50:19.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.096+0000 7fbe7dffb640 1 -- 172.21.15.83:0/393468123 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbe78003b90 con 0x7fbe880aedd0 2024-04-17T10:50:19.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.096+0000 7fbe7dffb640 1 --2- 172.21.15.83:0/393468123 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbe6c066f90 0x7fbe6c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:19.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.096+0000 7fbe8d063640 1 --2- 172.21.15.83:0/393468123 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbe6c066f90 0x7fbe6c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:19.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.096+0000 7fbe7dffb640 1 -- 172.21.15.83:0/393468123 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fbe780843f0 con 0x7fbe880aedd0 2024-04-17T10:50:19.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.096+0000 7fbe8d063640 1 --2- 172.21.15.83:0/393468123 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbe6c066f90 0x7fbe6c069450 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fbe840ae000 tx=0x7fbe84004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:19.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.100+0000 7fbe7dffb640 1 -- 172.21.15.83:0/393468123 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbe78055250 con 0x7fbe880aedd0 2024-04-17T10:50:19.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.244+0000 7fbe8f2c7640 1 -- 172.21.15.83:0/393468123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe880b2af0 con 0x7fbe880aedd0 2024-04-17T10:50:19.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.248+0000 7fbe7dffb640 1 -- 172.21.15.83:0/393468123 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+16777 (secure 0 0 0) 0x7fbe78054b50 con 0x7fbe880aedd0 2024-04-17T10:50:19.240 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:50:19.241 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":130,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:50:12.167718+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:49:18.247824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-17T10:49:18.951409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.5","mappings":[{"from":2,"to":0}]},{"pgid":"19.25","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/1559222491":"2024-04-18T10:50:12.167626+0000","172.21.15.83:6836/1559222491":"2024-04-18T10:50:12.167626+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:50:19.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.248+0000 7fbe8f2c7640 1 -- 172.21.15.83:0/393468123 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbe6c066f90 msgr2=0x7fbe6c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:19.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.248+0000 7fbe8f2c7640 1 --2- 172.21.15.83:0/393468123 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbe6c066f90 0x7fbe6c069450 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fbe840ae000 tx=0x7fbe84004740 comp rx=0 tx=0).stop 2024-04-17T10:50:19.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.248+0000 7fbe8f2c7640 1 -- 172.21.15.83:0/393468123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe880aedd0 msgr2=0x7fbe88142350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:19.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.248+0000 7fbe8f2c7640 1 --2- 172.21.15.83:0/393468123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe880aedd0 0x7fbe88142350 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800c4c0 tx=0x7fbe78001e80 comp rx=0 tx=0).stop 2024-04-17T10:50:19.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.248+0000 7fbe8f2c7640 1 -- 172.21.15.83:0/393468123 shutdown_connections 2024-04-17T10:50:19.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.248+0000 7fbe8f2c7640 1 --2- 172.21.15.83:0/393468123 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbe6c066f90 0x7fbe6c069450 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:19.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.248+0000 7fbe8f2c7640 1 --2- 172.21.15.83:0/393468123 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe8815db50 0x7fbe880a9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:19.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.248+0000 7fbe8f2c7640 1 --2- 172.21.15.83:0/393468123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe880aedd0 0x7fbe88142350 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:19.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.248+0000 7fbe8f2c7640 1 --2- 172.21.15.83:0/393468123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe8806dcb0 0x7fbe881fa770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:19.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.248+0000 7fbe8f2c7640 1 -- 172.21.15.83:0/393468123 >> 172.21.15.83:0/393468123 conn(0x7fbe8806abe0 msgr2=0x7fbe880b0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:19.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.248+0000 7fbe8f2c7640 1 -- 172.21.15.83:0/393468123 shutdown_connections 2024-04-17T10:50:19.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.248+0000 7fbe8f2c7640 1 -- 172.21.15.83:0/393468123 wait complete. 2024-04-17T10:50:19.258 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:49:18.247824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:49:18.951409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:50:19.258 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:50:19.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.380+0000 7f133e8f2640 1 Processor -- start 2024-04-17T10:50:19.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.380+0000 7f133e8f2640 1 -- start start 2024-04-17T10:50:19.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f133e8f2640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f133806b520 0x7f13380b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:19.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f133e8f2640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f133815bd70 con 0x7f1338154de0 2024-04-17T10:50:19.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f133e8f2640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f133815bf60 con 0x7f13380b4040 2024-04-17T10:50:19.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f133e8f2640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13380b4870 con 0x7f133806b520 2024-04-17T10:50:19.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f1337fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1338154de0 0x7f133806ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38994/0 (socket says 172.21.15.83:38994) 2024-04-17T10:50:19.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f1337fff640 1 -- 172.21.15.83:0/2030788715 learned_addr learned my addr 172.21.15.83:0/2030788715 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:19.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f13377fe640 1 --2- 172.21.15.83:0/2030788715 >> v2:172.21.15.167:3300/0 conn(0x7f133806b520 0x7f13380b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:19.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f1336ffd640 1 -- 172.21.15.83:0/2030788715 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2826385790 0 0) 0x7f133815bd70 con 0x7f1338154de0 2024-04-17T10:50:19.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f1336ffd640 1 -- 172.21.15.83:0/2030788715 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1320003530 con 0x7f1338154de0 2024-04-17T10:50:19.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f1336ffd640 1 -- 172.21.15.83:0/2030788715 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2222798071 0 0) 0x7f1320003530 con 0x7f1338154de0 2024-04-17T10:50:19.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f1336ffd640 1 -- 172.21.15.83:0/2030788715 >> v2:172.21.15.167:3300/0 conn(0x7f133806b520 msgr2=0x7f13380b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:19.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f1336ffd640 1 --2- 172.21.15.83:0/2030788715 >> v2:172.21.15.167:3300/0 conn(0x7f133806b520 0x7f13380b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:19.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f1336ffd640 1 -- 172.21.15.83:0/2030788715 >> v1:172.21.15.167:6789/0 conn(0x7f13380b4040 legacy=0x7f133806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:19.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f1336ffd640 1 -- 172.21.15.83:0/2030788715 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133815afb0 con 0x7f1338154de0 2024-04-17T10:50:19.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f1336ffd640 1 -- 172.21.15.83:0/2030788715 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f1328001680 con 0x7f1338154de0 2024-04-17T10:50:19.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f13377fe640 1 --2- 172.21.15.83:0/2030788715 >> v2:172.21.15.167:3300/0 conn(0x7f133806b520 0x7f13380b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:50:19.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f1336ffd640 1 -- 172.21.15.83:0/2030788715 >> v1:172.21.15.83:6789/0 conn(0x7f1338154de0 legacy=0x7f133806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:19.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f1336ffd640 1 --2- 172.21.15.83:0/2030788715 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1320003a30 0x7f1320003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:19.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f1336ffd640 1 --2- 172.21.15.83:0/2030788715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1320004810 0x7f1320004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:19.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f1336ffd640 1 --2- 172.21.15.83:0/2030788715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1320005240 0x7f132000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:19.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f1336ffd640 1 -- 172.21.15.83:0/2030788715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132000c170 con 0x7f1320004810 2024-04-17T10:50:19.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f1336ffd640 1 -- 172.21.15.83:0/2030788715 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132000c320 con 0x7f1320005240 2024-04-17T10:50:19.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f1336ffd640 1 -- 172.21.15.83:0/2030788715 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f132000c520 con 0x7f1320003a30 2024-04-17T10:50:19.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f13377fe640 1 --2- 172.21.15.83:0/2030788715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1320004810 0x7f1320004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:19.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f1337fff640 1 --2- 172.21.15.83:0/2030788715 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1320003a30 0x7f1320003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:19.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f133ce8f640 1 --2- 172.21.15.83:0/2030788715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1320005240 0x7f132000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:19.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f13377fe640 1 -- 172.21.15.83:0/2030788715 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1320003a30 msgr2=0x7f1320003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:19.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f13377fe640 1 --2- 172.21.15.83:0/2030788715 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1320003a30 0x7f1320003e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:19.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f13377fe640 1 -- 172.21.15.83:0/2030788715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1320005240 msgr2=0x7f132000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:19.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f13377fe640 1 --2- 172.21.15.83:0/2030788715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1320005240 0x7f132000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:19.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f13377fe640 1 -- 172.21.15.83:0/2030788715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1320003530 con 0x7f1320004810 2024-04-17T10:50:19.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f133ce8f640 1 --2- 172.21.15.83:0/2030788715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1320005240 0x7f132000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:19.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.384+0000 7f133ce8f640 1 -- 172.21.15.83:0/2030788715 reap_dead start 2024-04-17T10:50:19.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f13377fe640 1 --2- 172.21.15.83:0/2030788715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1320004810 0x7f1320004c90 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f132c0098d0 tx=0x7f132c01c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ebd1a35a729e64 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:19.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f1336ffd640 1 -- 172.21.15.83:0/2030788715 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c00f040 con 0x7f1320004810 2024-04-17T10:50:19.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f1336ffd640 1 -- 172.21.15.83:0/2030788715 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132c0a8640 con 0x7f1320004810 2024-04-17T10:50:19.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f1336ffd640 1 -- 172.21.15.83:0/2030788715 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c0a89f0 con 0x7f1320004810 2024-04-17T10:50:19.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f133e8f2640 1 -- 172.21.15.83:0/2030788715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1320004810 msgr2=0x7f1320004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:19.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f133e8f2640 1 --2- 172.21.15.83:0/2030788715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1320004810 0x7f1320004c90 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f132c0098d0 tx=0x7f132c01c5d0 comp rx=0 tx=0).stop 2024-04-17T10:50:19.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f133e8f2640 1 -- 172.21.15.83:0/2030788715 shutdown_connections 2024-04-17T10:50:19.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f133e8f2640 1 --2- 172.21.15.83:0/2030788715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1320004810 0x7f1320004c90 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:19.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f133e8f2640 1 -- 172.21.15.83:0/2030788715 >> 172.21.15.83:0/2030788715 conn(0x7f13380b15b0 msgr2=0x7f13380b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:19.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f133e8f2640 1 -- 172.21.15.83:0/2030788715 shutdown_connections 2024-04-17T10:50:19.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f133e8f2640 1 -- 172.21.15.83:0/2030788715 wait complete. 2024-04-17T10:50:19.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f133e8f2640 1 Processor -- start 2024-04-17T10:50:19.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f133e8f2640 1 -- start start 2024-04-17T10:50:19.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f133e8f2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1320004810 0x7f1338157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:19.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f133e8f2640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1338155ec0 0x7f1338158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:19.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f133e8f2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1338148b50 0x7f1338148fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:19.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f133e8f2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133815b850 con 0x7f1320004810 2024-04-17T10:50:19.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f133e8f2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133815b570 con 0x7f1338148b50 2024-04-17T10:50:19.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f133e8f2640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13381551c0 con 0x7f1338155ec0 2024-04-17T10:50:19.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f1337fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1320004810 0x7f1338157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:19.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f1337fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1320004810 0x7f1338157d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40542/0 (socket says 172.21.15.83:40542) 2024-04-17T10:50:19.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f1337fff640 1 -- 172.21.15.83:0/2090645499 learned_addr learned my addr 172.21.15.83:0/2090645499 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:19.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f1337fff640 1 -- 172.21.15.83:0/2090645499 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1338155ec0 msgr2=0x7f1338158ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:19.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f1337fff640 1 --2- 172.21.15.83:0/2090645499 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1338155ec0 0x7f1338158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:19.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f1337fff640 1 -- 172.21.15.83:0/2090645499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1338148b50 msgr2=0x7f1338148fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:19.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f1337fff640 1 --2- 172.21.15.83:0/2090645499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1338148b50 0x7f1338148fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:19.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f1337fff640 1 -- 172.21.15.83:0/2090645499 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13381563d0 con 0x7f1320004810 2024-04-17T10:50:19.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f1337fff640 1 --2- 172.21.15.83:0/2090645499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1320004810 0x7f1338157d60 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f1328010bb0 tx=0x7f132800e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:19.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f1334ff9640 1 -- 172.21.15.83:0/2090645499 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132800ebe0 con 0x7f1320004810 2024-04-17T10:50:19.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f1334ff9640 1 -- 172.21.15.83:0/2090645499 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1328011040 con 0x7f1320004810 2024-04-17T10:50:19.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f133e8f2640 1 -- 172.21.15.83:0/2090645499 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13381566b0 con 0x7f1320004810 2024-04-17T10:50:19.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f1334ff9640 1 -- 172.21.15.83:0/2090645499 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1328015400 con 0x7f1320004810 2024-04-17T10:50:19.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.388+0000 7f133e8f2640 1 -- 172.21.15.83:0/2090645499 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1338156c70 con 0x7f1320004810 2024-04-17T10:50:19.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.392+0000 7f133e8f2640 1 -- 172.21.15.83:0/2090645499 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1338074f90 con 0x7f1320004810 2024-04-17T10:50:19.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.392+0000 7f1334ff9640 1 -- 172.21.15.83:0/2090645499 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f132801d020 con 0x7f1320004810 2024-04-17T10:50:19.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.392+0000 7f1334ff9640 1 --2- 172.21.15.83:0/2090645499 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1324066f60 0x7f1324069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:19.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.392+0000 7f1334ff9640 1 -- 172.21.15.83:0/2090645499 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f1328088110 con 0x7f1320004810 2024-04-17T10:50:19.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.392+0000 7f13377fe640 1 --2- 172.21.15.83:0/2090645499 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1324066f60 0x7f1324069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:19.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.392+0000 7f13377fe640 1 --2- 172.21.15.83:0/2090645499 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1324066f60 0x7f1324069420 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f132c0af000 tx=0x7f132c004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:19.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.396+0000 7f1334ff9640 1 -- 172.21.15.83:0/2090645499 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1328059120 con 0x7f1320004810 2024-04-17T10:50:19.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.556+0000 7f133e8f2640 1 -- 172.21.15.83:0/2090645499 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1338157100 con 0x7f1320004810 2024-04-17T10:50:19.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.556+0000 7f1334ff9640 1 -- 172.21.15.83:0/2090645499 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v130) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1328058a20 con 0x7f1320004810 2024-04-17T10:50:19.550 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:50:19.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.560+0000 7f133e8f2640 1 -- 172.21.15.83:0/2090645499 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1324066f60 msgr2=0x7f1324069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:19.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.560+0000 7f133e8f2640 1 --2- 172.21.15.83:0/2090645499 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1324066f60 0x7f1324069420 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f132c0af000 tx=0x7f132c004740 comp rx=0 tx=0).stop 2024-04-17T10:50:19.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.560+0000 7f133e8f2640 1 -- 172.21.15.83:0/2090645499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1320004810 msgr2=0x7f1338157d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:19.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.560+0000 7f133e8f2640 1 --2- 172.21.15.83:0/2090645499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1320004810 0x7f1338157d60 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f1328010bb0 tx=0x7f132800e4d0 comp rx=0 tx=0).stop 2024-04-17T10:50:19.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.560+0000 7f133e8f2640 1 -- 172.21.15.83:0/2090645499 shutdown_connections 2024-04-17T10:50:19.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.560+0000 7f133e8f2640 1 --2- 172.21.15.83:0/2090645499 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1324066f60 0x7f1324069420 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:19.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.560+0000 7f133e8f2640 1 --2- 172.21.15.83:0/2090645499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1338148b50 0x7f1338148fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:19.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.560+0000 7f133e8f2640 1 --2- 172.21.15.83:0/2090645499 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1338155ec0 0x7f1338158ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:19.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.560+0000 7f133e8f2640 1 --2- 172.21.15.83:0/2090645499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1320004810 0x7f1338157d60 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:19.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.560+0000 7f133e8f2640 1 -- 172.21.15.83:0/2090645499 >> 172.21.15.83:0/2090645499 conn(0x7f13380b15b0 msgr2=0x7f13380b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:19.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.560+0000 7f133e8f2640 1 -- 172.21.15.83:0/2090645499 shutdown_connections 2024-04-17T10:50:19.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.560+0000 7f133e8f2640 1 -- 172.21.15.83:0/2090645499 wait complete. 2024-04-17T10:50:19.570 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:50:19.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.700+0000 7fec45679640 1 Processor -- start 2024-04-17T10:50:19.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.700+0000 7fec45679640 1 -- start start 2024-04-17T10:50:19.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.700+0000 7fec45679640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fec4015b6e0 0x7fec40071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:19.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.700+0000 7fec45679640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fec4015bac0 0x7fec400cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:19.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.700+0000 7fec45679640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec40070470 0x7fec4006ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:19.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.700+0000 7fec45679640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec4006c1f0 con 0x7fec4015b6e0 2024-04-17T10:50:19.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.700+0000 7fec45679640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec400cdd30 con 0x7fec4015bac0 2024-04-17T10:50:19.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.700+0000 7fec45679640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec40070a10 con 0x7fec40070470 2024-04-17T10:50:19.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.700+0000 7fec3effd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fec4015b6e0 0x7fec40071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:19.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.700+0000 7fec3effd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fec4015b6e0 0x7fec40071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40548/0 (socket says 172.21.15.83:40548) 2024-04-17T10:50:19.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.700+0000 7fec3effd640 1 -- 172.21.15.83:0/521568380 learned_addr learned my addr 172.21.15.83:0/521568380 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:19.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.700+0000 7fec3f7fe640 1 --2- 172.21.15.83:0/521568380 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec40070470 0x7fec4006ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:19.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.700+0000 7fec3e7fc640 1 --2- 172.21.15.83:0/521568380 >> v2:172.21.15.167:3300/0 conn(0x7fec4015bac0 0x7fec400cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:19.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.700+0000 7fec3effd640 1 -- 172.21.15.83:0/521568380 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec40070470 msgr2=0x7fec4006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:19.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.700+0000 7fec3effd640 1 --2- 172.21.15.83:0/521568380 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec40070470 0x7fec4006ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:19.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.700+0000 7fec3effd640 1 -- 172.21.15.83:0/521568380 >> v2:172.21.15.167:3300/0 conn(0x7fec4015bac0 msgr2=0x7fec400cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:19.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.700+0000 7fec3effd640 1 --2- 172.21.15.83:0/521568380 >> v2:172.21.15.167:3300/0 conn(0x7fec4015bac0 0x7fec400cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:19.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.700+0000 7fec3effd640 1 -- 172.21.15.83:0/521568380 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec40070cf0 con 0x7fec4015b6e0 2024-04-17T10:50:19.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.700+0000 7fec3e7fc640 1 --2- 172.21.15.83:0/521568380 >> v2:172.21.15.167:3300/0 conn(0x7fec4015bac0 0x7fec400cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:19.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.704+0000 7fec3effd640 1 --2- 172.21.15.83:0/521568380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec4015b6e0 0x7fec40071cc0 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7fec40071c50 tx=0x7fec3001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d141ae4d99317209 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:19.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.704+0000 7fec3dffb640 1 -- 172.21.15.83:0/521568380 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec300ab5f0 con 0x7fec4015b6e0 2024-04-17T10:50:19.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.704+0000 7fec3dffb640 1 -- 172.21.15.83:0/521568380 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec300b0070 con 0x7fec4015b6e0 2024-04-17T10:50:19.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.704+0000 7fec3dffb640 1 -- 172.21.15.83:0/521568380 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec300ab9b0 con 0x7fec4015b6e0 2024-04-17T10:50:19.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.704+0000 7fec45679640 1 -- 172.21.15.83:0/521568380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec4015b6e0 msgr2=0x7fec40071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:19.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.704+0000 7fec45679640 1 --2- 172.21.15.83:0/521568380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec4015b6e0 0x7fec40071cc0 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7fec40071c50 tx=0x7fec3001c520 comp rx=0 tx=0).stop 2024-04-17T10:50:19.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.704+0000 7fec45679640 1 -- 172.21.15.83:0/521568380 shutdown_connections 2024-04-17T10:50:19.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.704+0000 7fec45679640 1 --2- 172.21.15.83:0/521568380 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec40070470 0x7fec4006ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:19.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.704+0000 7fec45679640 1 --2- 172.21.15.83:0/521568380 >> v2:172.21.15.167:3300/0 conn(0x7fec4015bac0 0x7fec400cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:19.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.704+0000 7fec45679640 1 --2- 172.21.15.83:0/521568380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec4015b6e0 0x7fec40071cc0 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:19.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.704+0000 7fec45679640 1 -- 172.21.15.83:0/521568380 >> 172.21.15.83:0/521568380 conn(0x7fec400b15b0 msgr2=0x7fec400b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:19.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.704+0000 7fec45679640 1 -- 172.21.15.83:0/521568380 shutdown_connections 2024-04-17T10:50:19.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.704+0000 7fec45679640 1 -- 172.21.15.83:0/521568380 wait complete. 2024-04-17T10:50:19.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.704+0000 7fec45679640 1 Processor -- start 2024-04-17T10:50:19.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.704+0000 7fec45679640 1 -- start start 2024-04-17T10:50:19.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.708+0000 7fec45679640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec40070470 0x7fec4015d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:19.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.708+0000 7fec45679640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec4015b6e0 0x7fec40157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:19.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.708+0000 7fec45679640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec4015bac0 0x7fec400cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:19.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.708+0000 7fec45679640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4006d270 con 0x7fec40070470 2024-04-17T10:50:19.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.708+0000 7fec45679640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4006d0f0 con 0x7fec4015bac0 2024-04-17T10:50:19.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.708+0000 7fec45679640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec4006b760 con 0x7fec4015b6e0 2024-04-17T10:50:19.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.708+0000 7fec3effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec40070470 0x7fec4015d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:19.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.708+0000 7fec3effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec40070470 0x7fec4015d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40552/0 (socket says 172.21.15.83:40552) 2024-04-17T10:50:19.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.708+0000 7fec3effd640 1 -- 172.21.15.83:0/559609128 learned_addr learned my addr 172.21.15.83:0/559609128 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:19.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.708+0000 7fec3e7fc640 1 --2- 172.21.15.83:0/559609128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec4015b6e0 0x7fec40157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:19.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.708+0000 7fec3effd640 1 -- 172.21.15.83:0/559609128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec4015b6e0 msgr2=0x7fec40157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:19.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.708+0000 7fec3f7fe640 1 --2- 172.21.15.83:0/559609128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec4015bac0 0x7fec400cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:19.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.708+0000 7fec3effd640 1 --2- 172.21.15.83:0/559609128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec4015b6e0 0x7fec40157680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:19.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.708+0000 7fec3effd640 1 -- 172.21.15.83:0/559609128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec4015bac0 msgr2=0x7fec400cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:19.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.708+0000 7fec3effd640 1 --2- 172.21.15.83:0/559609128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec4015bac0 0x7fec400cdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:19.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.708+0000 7fec3effd640 1 -- 172.21.15.83:0/559609128 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec40164610 con 0x7fec40070470 2024-04-17T10:50:19.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.708+0000 7fec3effd640 1 --2- 172.21.15.83:0/559609128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec40070470 0x7fec4015d380 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7fec300ad040 tx=0x7fec3001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:19.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.708+0000 7fec277fe640 1 -- 172.21.15.83:0/559609128 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec300be5f0 con 0x7fec40070470 2024-04-17T10:50:19.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.708+0000 7fec277fe640 1 -- 172.21.15.83:0/559609128 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec300be7b0 con 0x7fec40070470 2024-04-17T10:50:19.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.708+0000 7fec45679640 1 -- 172.21.15.83:0/559609128 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec401648b0 con 0x7fec40070470 2024-04-17T10:50:19.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.708+0000 7fec277fe640 1 -- 172.21.15.83:0/559609128 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec300beba0 con 0x7fec40070470 2024-04-17T10:50:19.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.708+0000 7fec45679640 1 -- 172.21.15.83:0/559609128 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec40164e70 con 0x7fec40070470 2024-04-17T10:50:19.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.708+0000 7fec45679640 1 -- 172.21.15.83:0/559609128 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec40074f90 con 0x7fec40070470 2024-04-17T10:50:19.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.708+0000 7fec277fe640 1 -- 172.21.15.83:0/559609128 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fec300b43f0 con 0x7fec40070470 2024-04-17T10:50:19.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.708+0000 7fec277fe640 1 --2- 172.21.15.83:0/559609128 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fec2c066f90 0x7fec2c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:19.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.708+0000 7fec277fe640 1 -- 172.21.15.83:0/559609128 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fec3011fa40 con 0x7fec40070470 2024-04-17T10:50:19.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.708+0000 7fec3e7fc640 1 --2- 172.21.15.83:0/559609128 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fec2c066f90 0x7fec2c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:19.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.712+0000 7fec3e7fc640 1 --2- 172.21.15.83:0/559609128 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fec2c066f90 0x7fec2c069450 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fec401f99e0 tx=0x7fec34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:19.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.712+0000 7fec277fe640 1 -- 172.21.15.83:0/559609128 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fec300c4060 con 0x7fec40070470 2024-04-17T10:50:19.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.872+0000 7fec45679640 1 -- 172.21.15.83:0/559609128 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fec40074480 con 0x7fec40070470 2024-04-17T10:50:19.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.872+0000 7fec277fe640 1 -- 172.21.15.83:0/559609128 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v130) v1 ==== 104+0+11 (secure 0 0 0) 0x7fec30016110 con 0x7fec40070470 2024-04-17T10:50:19.864 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:50:19.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.872+0000 7fec45679640 1 -- 172.21.15.83:0/559609128 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fec2c066f90 msgr2=0x7fec2c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:19.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.872+0000 7fec45679640 1 --2- 172.21.15.83:0/559609128 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fec2c066f90 0x7fec2c069450 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fec401f99e0 tx=0x7fec34008040 comp rx=0 tx=0).stop 2024-04-17T10:50:19.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.872+0000 7fec45679640 1 -- 172.21.15.83:0/559609128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec40070470 msgr2=0x7fec4015d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:19.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.872+0000 7fec45679640 1 --2- 172.21.15.83:0/559609128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec40070470 0x7fec4015d380 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7fec300ad040 tx=0x7fec3001ad60 comp rx=0 tx=0).stop 2024-04-17T10:50:19.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.872+0000 7fec45679640 1 -- 172.21.15.83:0/559609128 shutdown_connections 2024-04-17T10:50:19.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.872+0000 7fec45679640 1 --2- 172.21.15.83:0/559609128 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fec2c066f90 0x7fec2c069450 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:19.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.872+0000 7fec45679640 1 --2- 172.21.15.83:0/559609128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec4015bac0 0x7fec400cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:19.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.872+0000 7fec45679640 1 --2- 172.21.15.83:0/559609128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec4015b6e0 0x7fec40157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:19.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.872+0000 7fec45679640 1 --2- 172.21.15.83:0/559609128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec40070470 0x7fec4015d380 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:19.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.872+0000 7fec45679640 1 -- 172.21.15.83:0/559609128 >> 172.21.15.83:0/559609128 conn(0x7fec400b15b0 msgr2=0x7fec400b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:19.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.872+0000 7fec45679640 1 -- 172.21.15.83:0/559609128 shutdown_connections 2024-04-17T10:50:19.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:19.872+0000 7fec45679640 1 -- 172.21.15.83:0/559609128 wait complete. 2024-04-17T10:50:19.883 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:50:20.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.016+0000 7fdc232a6640 1 Processor -- start 2024-04-17T10:50:20.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.016+0000 7fdc232a6640 1 -- start start 2024-04-17T10:50:20.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.016+0000 7fdc232a6640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdc1c06b520 0x7fdc1c0b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:20.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.016+0000 7fdc232a6640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdc1c0b4040 0x7fdc1c06cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:20.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.016+0000 7fdc232a6640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc1c15bd70 con 0x7fdc1c154de0 2024-04-17T10:50:20.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.016+0000 7fdc232a6640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1c0b4870 con 0x7fdc1c0b4040 2024-04-17T10:50:20.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.016+0000 7fdc232a6640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1c15bf60 con 0x7fdc1c06b520 2024-04-17T10:50:20.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.016+0000 7fdc21843640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdc1c0b4040 0x7fdc1c06cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:20.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.016+0000 7fdc21843640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdc1c0b4040 0x7fdc1c06cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40554/0 (socket says 172.21.15.83:40554) 2024-04-17T10:50:20.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.016+0000 7fdc21843640 1 -- 172.21.15.83:0/3365791348 learned_addr learned my addr 172.21.15.83:0/3365791348 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:20.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.016+0000 7fdc20841640 1 --2- 172.21.15.83:0/3365791348 >> v2:172.21.15.167:3300/0 conn(0x7fdc1c06b520 0x7fdc1c0b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:20.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.016+0000 7fdc21843640 1 -- 172.21.15.83:0/3365791348 >> v2:172.21.15.167:3300/0 conn(0x7fdc1c06b520 msgr2=0x7fdc1c0b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:20.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.016+0000 7fdc21843640 1 --2- 172.21.15.83:0/3365791348 >> v2:172.21.15.167:3300/0 conn(0x7fdc1c06b520 0x7fdc1c0b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:20.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.016+0000 7fdc21843640 1 -- 172.21.15.83:0/3365791348 >> v1:172.21.15.167:6789/0 conn(0x7fdc1c154de0 legacy=0x7fdc1c06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:20.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.016+0000 7fdc21843640 1 -- 172.21.15.83:0/3365791348 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1c15afb0 con 0x7fdc1c0b4040 2024-04-17T10:50:20.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.020+0000 7fdc21843640 1 --2- 172.21.15.83:0/3365791348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc1c0b4040 0x7fdc1c06cbe0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c0b2480 tx=0x7fdc1401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76b4cf74d695d51 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:20.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.020+0000 7fdc13fff640 1 -- 172.21.15.83:0/3365791348 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc140b1070 con 0x7fdc1c0b4040 2024-04-17T10:50:20.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.020+0000 7fdc13fff640 1 -- 172.21.15.83:0/3365791348 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc140ac630 con 0x7fdc1c0b4040 2024-04-17T10:50:20.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.020+0000 7fdc13fff640 1 -- 172.21.15.83:0/3365791348 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc140a8050 con 0x7fdc1c0b4040 2024-04-17T10:50:20.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.020+0000 7fdc232a6640 1 -- 172.21.15.83:0/3365791348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc1c0b4040 msgr2=0x7fdc1c06cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:20.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.020+0000 7fdc232a6640 1 --2- 172.21.15.83:0/3365791348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc1c0b4040 0x7fdc1c06cbe0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c0b2480 tx=0x7fdc1401c4e0 comp rx=0 tx=0).stop 2024-04-17T10:50:20.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.020+0000 7fdc232a6640 1 -- 172.21.15.83:0/3365791348 shutdown_connections 2024-04-17T10:50:20.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.020+0000 7fdc232a6640 1 --2- 172.21.15.83:0/3365791348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc1c0b4040 0x7fdc1c06cbe0 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:20.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.020+0000 7fdc232a6640 1 --2- 172.21.15.83:0/3365791348 >> v2:172.21.15.167:3300/0 conn(0x7fdc1c06b520 0x7fdc1c0b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:20.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.020+0000 7fdc232a6640 1 -- 172.21.15.83:0/3365791348 >> 172.21.15.83:0/3365791348 conn(0x7fdc1c0b15b0 msgr2=0x7fdc1c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:20.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.020+0000 7fdc232a6640 1 -- 172.21.15.83:0/3365791348 shutdown_connections 2024-04-17T10:50:20.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.020+0000 7fdc232a6640 1 -- 172.21.15.83:0/3365791348 wait complete. 2024-04-17T10:50:20.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.020+0000 7fdc232a6640 1 Processor -- start 2024-04-17T10:50:20.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.020+0000 7fdc232a6640 1 -- start start 2024-04-17T10:50:20.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.024+0000 7fdc232a6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc1c06b520 0x7fdc1c15c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:20.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.024+0000 7fdc232a6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc1c0b4040 0x7fdc1c150d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:20.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.024+0000 7fdc232a6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc1c154de0 0x7fdc1c151260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:20.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.024+0000 7fdc232a6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c0b44a0 con 0x7fdc1c06b520 2024-04-17T10:50:20.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.024+0000 7fdc232a6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c1551c0 con 0x7fdc1c0b4040 2024-04-17T10:50:20.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.024+0000 7fdc232a6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c0b4620 con 0x7fdc1c154de0 2024-04-17T10:50:20.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.024+0000 7fdc21042640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc1c06b520 0x7fdc1c15c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:20.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.024+0000 7fdc21042640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc1c06b520 0x7fdc1c15c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40566/0 (socket says 172.21.15.83:40566) 2024-04-17T10:50:20.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.024+0000 7fdc21042640 1 -- 172.21.15.83:0/839112027 learned_addr learned my addr 172.21.15.83:0/839112027 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:20.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.024+0000 7fdc20841640 1 --2- 172.21.15.83:0/839112027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc1c0b4040 0x7fdc1c150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:20.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.024+0000 7fdc21843640 1 --2- 172.21.15.83:0/839112027 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc1c154de0 0x7fdc1c151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:20.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.024+0000 7fdc21042640 1 -- 172.21.15.83:0/839112027 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc1c154de0 msgr2=0x7fdc1c151260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:20.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.024+0000 7fdc21042640 1 --2- 172.21.15.83:0/839112027 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc1c154de0 0x7fdc1c151260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:20.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.024+0000 7fdc21042640 1 -- 172.21.15.83:0/839112027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc1c0b4040 msgr2=0x7fdc1c150d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:20.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.024+0000 7fdc21042640 1 --2- 172.21.15.83:0/839112027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc1c0b4040 0x7fdc1c150d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:20.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.024+0000 7fdc21042640 1 -- 172.21.15.83:0/839112027 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1c157740 con 0x7fdc1c06b520 2024-04-17T10:50:20.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.024+0000 7fdc21042640 1 --2- 172.21.15.83:0/839112027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc1c06b520 0x7fdc1c15c590 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800ba80 tx=0x7fdc1800bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:20.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.024+0000 7fdc11ffb640 1 -- 172.21.15.83:0/839112027 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1800cc00 con 0x7fdc1c06b520 2024-04-17T10:50:20.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.024+0000 7fdc11ffb640 1 -- 172.21.15.83:0/839112027 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1800cda0 con 0x7fdc1c06b520 2024-04-17T10:50:20.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.024+0000 7fdc232a6640 1 -- 172.21.15.83:0/839112027 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc1c157a20 con 0x7fdc1c06b520 2024-04-17T10:50:20.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.024+0000 7fdc11ffb640 1 -- 172.21.15.83:0/839112027 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc18003bc0 con 0x7fdc1c06b520 2024-04-17T10:50:20.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.024+0000 7fdc232a6640 1 -- 172.21.15.83:0/839112027 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1c154030 con 0x7fdc1c06b520 2024-04-17T10:50:20.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.024+0000 7fdc232a6640 1 -- 172.21.15.83:0/839112027 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc1c074f90 con 0x7fdc1c06b520 2024-04-17T10:50:20.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.024+0000 7fdc11ffb640 1 -- 172.21.15.83:0/839112027 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdc18006d10 con 0x7fdc1c06b520 2024-04-17T10:50:20.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.024+0000 7fdc11ffb640 1 --2- 172.21.15.83:0/839112027 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdc00066f60 0x7fdc00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:20.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.024+0000 7fdc11ffb640 1 -- 172.21.15.83:0/839112027 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fdc18084d00 con 0x7fdc1c06b520 2024-04-17T10:50:20.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.024+0000 7fdc20841640 1 --2- 172.21.15.83:0/839112027 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdc00066f60 0x7fdc00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:20.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.024+0000 7fdc20841640 1 --2- 172.21.15.83:0/839112027 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdc00066f60 0x7fdc00069420 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c06b1d0 tx=0x7fdc0c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:20.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.028+0000 7fdc11ffb640 1 -- 172.21.15.83:0/839112027 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdc18055c80 con 0x7fdc1c06b520 2024-04-17T10:50:20.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.188+0000 7fdc232a6640 1 -- 172.21.15.83:0/839112027 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdc1c0b3280 con 0x7fdc1c06b520 2024-04-17T10:50:20.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.188+0000 7fdc11ffb640 1 -- 172.21.15.83:0/839112027 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v130) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdc18055580 con 0x7fdc1c06b520 2024-04-17T10:50:20.182 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:50:20.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.192+0000 7fdc232a6640 1 -- 172.21.15.83:0/839112027 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdc00066f60 msgr2=0x7fdc00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:20.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.192+0000 7fdc232a6640 1 --2- 172.21.15.83:0/839112027 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdc00066f60 0x7fdc00069420 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c06b1d0 tx=0x7fdc0c008040 comp rx=0 tx=0).stop 2024-04-17T10:50:20.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.192+0000 7fdc232a6640 1 -- 172.21.15.83:0/839112027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc1c06b520 msgr2=0x7fdc1c15c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:20.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.192+0000 7fdc232a6640 1 --2- 172.21.15.83:0/839112027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc1c06b520 0x7fdc1c15c590 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800ba80 tx=0x7fdc1800bf40 comp rx=0 tx=0).stop 2024-04-17T10:50:20.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.192+0000 7fdc232a6640 1 -- 172.21.15.83:0/839112027 shutdown_connections 2024-04-17T10:50:20.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.192+0000 7fdc232a6640 1 --2- 172.21.15.83:0/839112027 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdc00066f60 0x7fdc00069420 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:20.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.192+0000 7fdc232a6640 1 --2- 172.21.15.83:0/839112027 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc1c154de0 0x7fdc1c151260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:20.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.192+0000 7fdc232a6640 1 --2- 172.21.15.83:0/839112027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc1c0b4040 0x7fdc1c150d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:20.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.192+0000 7fdc232a6640 1 --2- 172.21.15.83:0/839112027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc1c06b520 0x7fdc1c15c590 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:20.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.192+0000 7fdc232a6640 1 -- 172.21.15.83:0/839112027 >> 172.21.15.83:0/839112027 conn(0x7fdc1c0b15b0 msgr2=0x7fdc1c0affa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:20.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.192+0000 7fdc232a6640 1 -- 172.21.15.83:0/839112027 shutdown_connections 2024-04-17T10:50:20.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.192+0000 7fdc232a6640 1 -- 172.21.15.83:0/839112027 wait complete. 2024-04-17T10:50:20.201 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:50:20.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.328+0000 7facb45d5640 1 Processor -- start 2024-04-17T10:50:20.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.328+0000 7facb45d5640 1 -- start start 2024-04-17T10:50:20.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.328+0000 7facb45d5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7facac15b6e0 0x7facac071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:20.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.328+0000 7facb45d5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7facac15bac0 0x7facac0cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:20.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.328+0000 7facb45d5640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facac06c1f0 con 0x7facac070470 2024-04-17T10:50:20.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.328+0000 7facb45d5640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facac0cdd30 con 0x7facac15b6e0 2024-04-17T10:50:20.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.328+0000 7facb45d5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facac070a10 con 0x7facac15bac0 2024-04-17T10:50:20.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.328+0000 7facb2b72640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7facac070470 0x7facac06ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39006/0 (socket says 172.21.15.83:39006) 2024-04-17T10:50:20.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.328+0000 7facb2b72640 1 -- 172.21.15.83:0/1861445862 learned_addr learned my addr 172.21.15.83:0/1861445862 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:20.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb1b70640 1 --2- 172.21.15.83:0/1861445862 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7facac15bac0 0x7facac0cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:20.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb2371640 1 --2- 172.21.15.83:0/1861445862 >> v2:172.21.15.167:3300/0 conn(0x7facac15b6e0 0x7facac071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:20.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb136f640 1 -- 172.21.15.83:0/1861445862 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4250166292 0 0) 0x7facac06c1f0 con 0x7facac070470 2024-04-17T10:50:20.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb136f640 1 -- 172.21.15.83:0/1861445862 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac98003530 con 0x7facac070470 2024-04-17T10:50:20.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb136f640 1 -- 172.21.15.83:0/1861445862 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3478871073 0 0) 0x7fac98003530 con 0x7facac070470 2024-04-17T10:50:20.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb136f640 1 -- 172.21.15.83:0/1861445862 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7facac15bac0 msgr2=0x7facac0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:20.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb136f640 1 --2- 172.21.15.83:0/1861445862 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7facac15bac0 0x7facac0cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:20.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb136f640 1 -- 172.21.15.83:0/1861445862 >> v2:172.21.15.167:3300/0 conn(0x7facac15b6e0 msgr2=0x7facac071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:20.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb136f640 1 --2- 172.21.15.83:0/1861445862 >> v2:172.21.15.167:3300/0 conn(0x7facac15b6e0 0x7facac071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:20.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb136f640 1 -- 172.21.15.83:0/1861445862 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facac070cf0 con 0x7facac070470 2024-04-17T10:50:20.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb136f640 1 -- 172.21.15.83:0/1861445862 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7faca4001a20 con 0x7facac070470 2024-04-17T10:50:20.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb136f640 1 -- 172.21.15.83:0/1861445862 >> v1:172.21.15.83:6789/0 conn(0x7facac070470 legacy=0x7facac06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:20.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb136f640 1 --2- 172.21.15.83:0/1861445862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac98003a30 0x7fac98003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:20.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb136f640 1 --2- 172.21.15.83:0/1861445862 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7facac15bac0 0x7fac98004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:20.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb136f640 1 --2- 172.21.15.83:0/1861445862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac98004d50 0x7fac980051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:20.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb136f640 1 -- 172.21.15.83:0/1861445862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facac0cdd30 con 0x7fac98004d50 2024-04-17T10:50:20.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb136f640 1 -- 172.21.15.83:0/1861445862 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facac070a10 con 0x7fac98003a30 2024-04-17T10:50:20.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb136f640 1 -- 172.21.15.83:0/1861445862 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac9800b9d0 con 0x7facac15bac0 2024-04-17T10:50:20.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb2b72640 1 --2- 172.21.15.83:0/1861445862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac98004d50 0x7fac980051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:20.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb2371640 1 --2- 172.21.15.83:0/1861445862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac98003a30 0x7fac98003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:20.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb1b70640 1 --2- 172.21.15.83:0/1861445862 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7facac15bac0 0x7fac98004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:20.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb2b72640 1 -- 172.21.15.83:0/1861445862 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7facac15bac0 msgr2=0x7fac98004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:20.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb2b72640 1 --2- 172.21.15.83:0/1861445862 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7facac15bac0 0x7fac98004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:20.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb2b72640 1 -- 172.21.15.83:0/1861445862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac98003a30 msgr2=0x7fac98003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:20.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb2b72640 1 --2- 172.21.15.83:0/1861445862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac98003a30 0x7fac98003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:20.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb2b72640 1 -- 172.21.15.83:0/1861445862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac98003530 con 0x7fac98004d50 2024-04-17T10:50:20.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb2b72640 1 --2- 172.21.15.83:0/1861445862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac98004d50 0x7fac980051d0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7faca400a880 tx=0x7faca401d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3dff7063a2c6a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:20.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb136f640 1 -- 172.21.15.83:0/1861445862 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca400a4d0 con 0x7fac98004d50 2024-04-17T10:50:20.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb136f640 1 -- 172.21.15.83:0/1861445862 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faca40ad650 con 0x7fac98004d50 2024-04-17T10:50:20.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb136f640 1 -- 172.21.15.83:0/1861445862 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca40ada40 con 0x7fac98004d50 2024-04-17T10:50:20.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb45d5640 1 -- 172.21.15.83:0/1861445862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac98004d50 msgr2=0x7fac980051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:20.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb45d5640 1 --2- 172.21.15.83:0/1861445862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac98004d50 0x7fac980051d0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7faca400a880 tx=0x7faca401d380 comp rx=0 tx=0).stop 2024-04-17T10:50:20.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb2b72640 1 -- 172.21.15.83:0/1861445862 reap_dead start 2024-04-17T10:50:20.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb45d5640 1 -- 172.21.15.83:0/1861445862 shutdown_connections 2024-04-17T10:50:20.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.332+0000 7facb45d5640 1 -- 172.21.15.83:0/1861445862 >> 172.21.15.83:0/1861445862 conn(0x7facac0b15b0 msgr2=0x7facac0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:20.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.336+0000 7facb45d5640 1 -- 172.21.15.83:0/1861445862 shutdown_connections 2024-04-17T10:50:20.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.336+0000 7facb45d5640 1 -- 172.21.15.83:0/1861445862 wait complete. 2024-04-17T10:50:20.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.336+0000 7facb45d5640 1 Processor -- start 2024-04-17T10:50:20.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.336+0000 7facb45d5640 1 -- start start 2024-04-17T10:50:20.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.336+0000 7facb45d5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facac147f10 0x7facac15d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:20.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.336+0000 7facb45d5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7facac1482f0 0x7facac15a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:20.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.336+0000 7facb45d5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facac06d7e0 0x7facac06dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:20.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.336+0000 7facb45d5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facac06b520 con 0x7facac06d7e0 2024-04-17T10:50:20.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.336+0000 7facb45d5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facac0712b0 con 0x7facac147f10 2024-04-17T10:50:20.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.336+0000 7facb45d5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facac06b6a0 con 0x7facac1482f0 2024-04-17T10:50:20.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.336+0000 7facb2b72640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facac06d7e0 0x7facac06dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:20.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.336+0000 7facb2b72640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facac06d7e0 0x7facac06dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40582/0 (socket says 172.21.15.83:40582) 2024-04-17T10:50:20.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.336+0000 7facb2b72640 1 -- 172.21.15.83:0/1035628648 learned_addr learned my addr 172.21.15.83:0/1035628648 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:20.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.336+0000 7facb2371640 1 --2- 172.21.15.83:0/1035628648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facac147f10 0x7facac15d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:20.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.336+0000 7facb2b72640 1 -- 172.21.15.83:0/1035628648 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7facac1482f0 msgr2=0x7facac15a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:20.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.336+0000 7facb1b70640 1 --2- 172.21.15.83:0/1035628648 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7facac1482f0 0x7facac15a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:20.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.336+0000 7facb2b72640 1 --2- 172.21.15.83:0/1035628648 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7facac1482f0 0x7facac15a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:20.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.336+0000 7facb2b72640 1 -- 172.21.15.83:0/1035628648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facac147f10 msgr2=0x7facac15d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:20.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.336+0000 7facb2b72640 1 --2- 172.21.15.83:0/1035628648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facac147f10 0x7facac15d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:20.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.336+0000 7facb2b72640 1 -- 172.21.15.83:0/1035628648 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facac15c2a0 con 0x7facac06d7e0 2024-04-17T10:50:20.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.336+0000 7facb2b72640 1 --2- 172.21.15.83:0/1035628648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facac06d7e0 0x7facac06dc90 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7faca4000f90 tx=0x7fac98005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:20.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.336+0000 7fac9effd640 1 -- 172.21.15.83:0/1035628648 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca40b2040 con 0x7facac06d7e0 2024-04-17T10:50:20.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.336+0000 7fac9effd640 1 -- 172.21.15.83:0/1035628648 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faca40011d0 con 0x7facac06d7e0 2024-04-17T10:50:20.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.336+0000 7facb45d5640 1 -- 172.21.15.83:0/1035628648 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facac15c580 con 0x7facac06d7e0 2024-04-17T10:50:20.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.336+0000 7fac9effd640 1 -- 172.21.15.83:0/1035628648 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca4002030 con 0x7facac06d7e0 2024-04-17T10:50:20.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.336+0000 7facb45d5640 1 -- 172.21.15.83:0/1035628648 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facac15cb40 con 0x7facac06d7e0 2024-04-17T10:50:20.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.336+0000 7facb45d5640 1 -- 172.21.15.83:0/1035628648 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facac06e1d0 con 0x7facac06d7e0 2024-04-17T10:50:20.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.340+0000 7fac9effd640 1 -- 172.21.15.83:0/1035628648 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faca40b63f0 con 0x7facac06d7e0 2024-04-17T10:50:20.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.340+0000 7fac9effd640 1 --2- 172.21.15.83:0/1035628648 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fac90066f60 0x7fac90069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:20.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.340+0000 7fac9effd640 1 -- 172.21.15.83:0/1035628648 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5853+0+0 (secure 0 0 0) 0x7faca411ebd0 con 0x7facac06d7e0 2024-04-17T10:50:20.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.340+0000 7facb2371640 1 --2- 172.21.15.83:0/1035628648 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fac90066f60 0x7fac90069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:20.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.340+0000 7facb2371640 1 --2- 172.21.15.83:0/1035628648 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fac90066f60 0x7fac90069420 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7faca00098d0 tx=0x7faca0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:20.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.344+0000 7fac9effd640 1 -- 172.21.15.83:0/1035628648 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7facac06e1d0 con 0x7facac06d7e0 2024-04-17T10:50:20.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.508+0000 7facb45d5640 1 -- 172.21.15.83:0/1035628648 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7facac074f90 con 0x7facac06d7e0 2024-04-17T10:50:20.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.512+0000 7fac9effd640 1 -- 172.21.15.83:0/1035628648 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v130) v1 ==== 80+0+122 (secure 0 0 0) 0x7faca4017090 con 0x7facac06d7e0 2024-04-17T10:50:20.503 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6837/1559222491 2024-04-18T10:50:12.167626+0000 2024-04-17T10:50:20.504 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6836/1559222491 2024-04-18T10:50:12.167626+0000 2024-04-17T10:50:20.504 INFO:teuthology.orchestra.run.smithi083.stderr:listed 2 entries 2024-04-17T10:50:20.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.512+0000 7facb45d5640 1 -- 172.21.15.83:0/1035628648 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fac90066f60 msgr2=0x7fac90069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:20.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.512+0000 7facb45d5640 1 --2- 172.21.15.83:0/1035628648 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fac90066f60 0x7fac90069420 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7faca00098d0 tx=0x7faca0008040 comp rx=0 tx=0).stop 2024-04-17T10:50:20.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.512+0000 7facb45d5640 1 -- 172.21.15.83:0/1035628648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facac06d7e0 msgr2=0x7facac06dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:20.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.512+0000 7facb45d5640 1 --2- 172.21.15.83:0/1035628648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facac06d7e0 0x7facac06dc90 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7faca4000f90 tx=0x7fac98005840 comp rx=0 tx=0).stop 2024-04-17T10:50:20.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.512+0000 7facb45d5640 1 -- 172.21.15.83:0/1035628648 shutdown_connections 2024-04-17T10:50:20.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.512+0000 7facb45d5640 1 --2- 172.21.15.83:0/1035628648 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fac90066f60 0x7fac90069420 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:20.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.512+0000 7facb45d5640 1 --2- 172.21.15.83:0/1035628648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facac06d7e0 0x7facac06dc90 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:20.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.512+0000 7facb45d5640 1 --2- 172.21.15.83:0/1035628648 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7facac1482f0 0x7facac15a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:20.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.512+0000 7facb45d5640 1 --2- 172.21.15.83:0/1035628648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facac147f10 0x7facac15d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:20.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.512+0000 7facb45d5640 1 -- 172.21.15.83:0/1035628648 >> 172.21.15.83:0/1035628648 conn(0x7facac0b15b0 msgr2=0x7facac0b0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:20.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.512+0000 7facb45d5640 1 -- 172.21.15.83:0/1035628648 shutdown_connections 2024-04-17T10:50:20.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.512+0000 7facb45d5640 1 -- 172.21.15.83:0/1035628648 wait complete. 2024-04-17T10:50:20.521 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2024-04-17T10:50:20.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:20.522 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-17T10:50:20.533 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[80023]: fuse finished with error 0 and tester_r 0 2024-04-17T10:50:20.535 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-17T10:50:20.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:20.535 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T10:50:20.669 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:50:20.669 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:50:20.676 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:50:20.676 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-04-17T10:50:20.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:20.676 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-17T10:50:20.722 INFO:teuthology.orchestra.run.smithi167.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-17T10:50:20.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:50:20.723 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:50:20.723 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:50:20.731 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16063 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:50:20.731 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-17T10:50:20.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:20.731 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-17T10:50:20.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:50:20.781 INFO:teuthology.orchestra.run.smithi083.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-17T10:50:20.781 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:50:20.781 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:50:20.787 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:50:20.787 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-04-17T10:50:20.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:20.787 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-17T10:50:20.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:50:20.835 INFO:teuthology.orchestra.run.smithi167.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-17T10:50:20.835 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:50:20.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.968+0000 7f463ba69640 1 Processor -- start 2024-04-17T10:50:20.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.968+0000 7f463ba69640 1 -- start start 2024-04-17T10:50:20.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.972+0000 7f463ba69640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f46340aedd0 0x7f463406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:20.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.972+0000 7f463ba69640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f463415e440 con 0x7f463415db50 2024-04-17T10:50:20.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.972+0000 7f463ba69640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f463415e720 con 0x7f463406dae0 2024-04-17T10:50:20.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.972+0000 7f463ba69640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f463415ea00 con 0x7f46340aedd0 2024-04-17T10:50:20.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.972+0000 7f4639805640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f463415db50 0x7f4634070470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39022/0 (socket says 172.21.15.83:39022) 2024-04-17T10:50:20.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.972+0000 7f4639805640 1 -- 172.21.15.83:0/2321184936 learned_addr learned my addr 172.21.15.83:0/2321184936 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:20.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.972+0000 7f4639004640 1 --2- 172.21.15.83:0/2321184936 >> v2:172.21.15.167:3300/0 conn(0x7f46340aedd0 0x7f463406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:20.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.972+0000 7f4638803640 1 -- 172.21.15.83:0/2321184936 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3809085903 0 0) 0x7f463415e440 con 0x7f463415db50 2024-04-17T10:50:20.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.972+0000 7f4638803640 1 -- 172.21.15.83:0/2321184936 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4620003530 con 0x7f463415db50 2024-04-17T10:50:20.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.972+0000 7f4638803640 1 -- 172.21.15.83:0/2321184936 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1937979770 0 0) 0x7f4620003530 con 0x7f463415db50 2024-04-17T10:50:20.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.972+0000 7f4638803640 1 -- 172.21.15.83:0/2321184936 >> v2:172.21.15.167:3300/0 conn(0x7f46340aedd0 msgr2=0x7f463406d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:20.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.972+0000 7f4638803640 1 --2- 172.21.15.83:0/2321184936 >> v2:172.21.15.167:3300/0 conn(0x7f46340aedd0 0x7f463406d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:20.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.972+0000 7f4638803640 1 -- 172.21.15.83:0/2321184936 >> v1:172.21.15.167:6789/0 conn(0x7f463406dae0 legacy=0x7f46340b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:20.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.972+0000 7f4638803640 1 -- 172.21.15.83:0/2321184936 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f463415ece0 con 0x7f463415db50 2024-04-17T10:50:20.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.972+0000 7f4638803640 1 -- 172.21.15.83:0/2321184936 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f4630001680 con 0x7f463415db50 2024-04-17T10:50:20.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.972+0000 7f4638803640 1 -- 172.21.15.83:0/2321184936 >> v1:172.21.15.83:6789/0 conn(0x7f463415db50 legacy=0x7f4634070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:20.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.972+0000 7f4638803640 1 --2- 172.21.15.83:0/2321184936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46200039c0 0x7f4620003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:20.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.972+0000 7f4638803640 1 --2- 172.21.15.83:0/2321184936 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4620004810 0x7f4620004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:20.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.972+0000 7f4638803640 1 --2- 172.21.15.83:0/2321184936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46200051d0 0x7f462000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:20.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.972+0000 7f4638803640 1 -- 172.21.15.83:0/2321184936 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463415ea00 con 0x7f46200051d0 2024-04-17T10:50:20.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.972+0000 7f4638803640 1 -- 172.21.15.83:0/2321184936 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462000c070 con 0x7f46200039c0 2024-04-17T10:50:20.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.972+0000 7f4638803640 1 -- 172.21.15.83:0/2321184936 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462000c270 con 0x7f4620004810 2024-04-17T10:50:20.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.972+0000 7f463a006640 1 --2- 172.21.15.83:0/2321184936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46200051d0 0x7f462000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:20.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.972+0000 7f4639805640 1 --2- 172.21.15.83:0/2321184936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46200039c0 0x7f4620003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:20.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.972+0000 7f4639004640 1 --2- 172.21.15.83:0/2321184936 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4620004810 0x7f4620004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:20.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.972+0000 7f463a006640 1 -- 172.21.15.83:0/2321184936 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4620004810 msgr2=0x7f4620004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:20.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.972+0000 7f463a006640 1 --2- 172.21.15.83:0/2321184936 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4620004810 0x7f4620004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:20.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.972+0000 7f463a006640 1 -- 172.21.15.83:0/2321184936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46200039c0 msgr2=0x7f4620003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:20.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.972+0000 7f463a006640 1 --2- 172.21.15.83:0/2321184936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46200039c0 0x7f4620003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:20.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.972+0000 7f463a006640 1 -- 172.21.15.83:0/2321184936 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4620003530 con 0x7f46200051d0 2024-04-17T10:50:20.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.972+0000 7f4639805640 1 --2- 172.21.15.83:0/2321184936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46200039c0 0x7f4620003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:50:20.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.976+0000 7f463a006640 1 -- 172.21.15.83:0/2321184936 reap_dead start 2024-04-17T10:50:20.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.976+0000 7f463a006640 1 --2- 172.21.15.83:0/2321184936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46200051d0 0x7f462000b9d0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f462c001e70 tx=0x7f462c01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a1f7cca49f7257d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:20.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.976+0000 7f4638803640 1 -- 172.21.15.83:0/2321184936 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c001a90 con 0x7f46200051d0 2024-04-17T10:50:20.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.976+0000 7f4638803640 1 -- 172.21.15.83:0/2321184936 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462c003bc0 con 0x7f46200051d0 2024-04-17T10:50:20.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.976+0000 7f4638803640 1 -- 172.21.15.83:0/2321184936 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c005020 con 0x7f46200051d0 2024-04-17T10:50:20.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.976+0000 7f463ba69640 1 -- 172.21.15.83:0/2321184936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46200051d0 msgr2=0x7f462000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:20.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.976+0000 7f463ba69640 1 --2- 172.21.15.83:0/2321184936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46200051d0 0x7f462000b9d0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f462c001e70 tx=0x7f462c01c780 comp rx=0 tx=0).stop 2024-04-17T10:50:20.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.976+0000 7f463ba69640 1 -- 172.21.15.83:0/2321184936 shutdown_connections 2024-04-17T10:50:20.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.976+0000 7f463ba69640 1 --2- 172.21.15.83:0/2321184936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46200051d0 0x7f462000b9d0 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:20.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.976+0000 7f463ba69640 1 -- 172.21.15.83:0/2321184936 >> 172.21.15.83:0/2321184936 conn(0x7f463406abe0 msgr2=0x7f463406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:20.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.976+0000 7f463ba69640 1 -- 172.21.15.83:0/2321184936 shutdown_connections 2024-04-17T10:50:20.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.976+0000 7f463ba69640 1 -- 172.21.15.83:0/2321184936 wait complete. 2024-04-17T10:50:20.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.976+0000 7f463ba69640 1 Processor -- start 2024-04-17T10:50:20.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.976+0000 7f463ba69640 1 -- start start 2024-04-17T10:50:20.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.976+0000 7f463ba69640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46200051d0 0x7f4634150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:20.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.976+0000 7f463ba69640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46340ad840 0x7f46340adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:20.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.976+0000 7f463ba69640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4634154c80 0x7f4634155100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:20.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.976+0000 7f463ba69640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463415df30 con 0x7f4634154c80 2024-04-17T10:50:20.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.976+0000 7f463ba69640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463415e2c0 con 0x7f46200051d0 2024-04-17T10:50:20.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.976+0000 7f463ba69640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463415e0b0 con 0x7f46340ad840 2024-04-17T10:50:20.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.976+0000 7f463a006640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4634154c80 0x7f4634155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:20.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.976+0000 7f463a006640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4634154c80 0x7f4634155100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40606/0 (socket says 172.21.15.83:40606) 2024-04-17T10:50:20.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.976+0000 7f463a006640 1 -- 172.21.15.83:0/3323854678 learned_addr learned my addr 172.21.15.83:0/3323854678 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:20.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.976+0000 7f4639004640 1 --2- 172.21.15.83:0/3323854678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46340ad840 0x7f46340adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:20.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.976+0000 7f4639805640 1 --2- 172.21.15.83:0/3323854678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46200051d0 0x7f4634150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:20.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.976+0000 7f463a006640 1 -- 172.21.15.83:0/3323854678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46340ad840 msgr2=0x7f46340adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:20.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.976+0000 7f463a006640 1 --2- 172.21.15.83:0/3323854678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46340ad840 0x7f46340adcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:20.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.976+0000 7f463a006640 1 -- 172.21.15.83:0/3323854678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46200051d0 msgr2=0x7f4634150ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:20.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.976+0000 7f463a006640 1 --2- 172.21.15.83:0/3323854678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46200051d0 0x7f4634150ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:20.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.976+0000 7f463a006640 1 -- 172.21.15.83:0/3323854678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4634151430 con 0x7f4634154c80 2024-04-17T10:50:20.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.976+0000 7f4639805640 1 --2- 172.21.15.83:0/3323854678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46200051d0 0x7f4634150ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:20.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.980+0000 7f463a006640 1 --2- 172.21.15.83:0/3323854678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4634154c80 0x7f4634155100 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f462c01c5b0 tx=0x7f4620005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:20.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.980+0000 7f462a7fc640 1 -- 172.21.15.83:0/3323854678 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c007dc0 con 0x7f4634154c80 2024-04-17T10:50:20.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.980+0000 7f462a7fc640 1 -- 172.21.15.83:0/3323854678 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462c003980 con 0x7f4634154c80 2024-04-17T10:50:20.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.980+0000 7f463ba69640 1 -- 172.21.15.83:0/3323854678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46341525c0 con 0x7f4634154c80 2024-04-17T10:50:20.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.980+0000 7f462a7fc640 1 -- 172.21.15.83:0/3323854678 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c006dc0 con 0x7f4634154c80 2024-04-17T10:50:20.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.980+0000 7f463ba69640 1 -- 172.21.15.83:0/3323854678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4634155c20 con 0x7f4634154c80 2024-04-17T10:50:20.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.980+0000 7f463ba69640 1 -- 172.21.15.83:0/3323854678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4634157ad0 con 0x7f4634154c80 2024-04-17T10:50:20.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.980+0000 7f462a7fc640 1 -- 172.21.15.83:0/3323854678 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f462c0a6480 con 0x7f4634154c80 2024-04-17T10:50:20.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.980+0000 7f462a7fc640 1 --2- 172.21.15.83:0/3323854678 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4618066f60 0x7f4618069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:20.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.980+0000 7f462a7fc640 1 -- 172.21.15.83:0/3323854678 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f462c11e9b0 con 0x7f4634154c80 2024-04-17T10:50:20.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.980+0000 7f4639805640 1 --2- 172.21.15.83:0/3323854678 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4618066f60 0x7f4618069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:20.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.980+0000 7f4639805640 1 --2- 172.21.15.83:0/3323854678 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4618066f60 0x7f4618069420 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f46341581b0 tx=0x7f4630000ee0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:20.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:20.984+0000 7f462a7fc640 1 -- 172.21.15.83:0/3323854678 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f462c0b8060 con 0x7f4634154c80 2024-04-17T10:50:21.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.160+0000 7f463ba69640 1 -- 172.21.15.83:0/3323854678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46340ae200 con 0x7f4634154c80 2024-04-17T10:50:21.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.160+0000 7f462a7fc640 1 -- 172.21.15.83:0/3323854678 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4158 (secure 0 0 0) 0x7f4634151430 con 0x7f4634154c80 2024-04-17T10:50:21.154 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:50:21.155 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":80,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5942,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3888166846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3888166846},{"type":"v1","addr":"172.21.15.167:6833","nonce":3888166846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":6772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1167913722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1167913722},{"type":"v1","addr":"172.21.15.167:6835","nonce":1167913722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:49:19.962680+0000","modified":"2024-04-17T10:50:14.204149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9192},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9192":{"gid":9192,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":18,"addr":"172.21.15.83:6835/3552333026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3552333026},{"type":"v1","addr":"172.21.15.83:6835","nonce":3552333026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-17T10:50:21.155 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 80 2024-04-17T10:50:21.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.164+0000 7f463ba69640 1 -- 172.21.15.83:0/3323854678 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4618066f60 msgr2=0x7f4618069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:21.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.164+0000 7f463ba69640 1 --2- 172.21.15.83:0/3323854678 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4618066f60 0x7f4618069420 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f46341581b0 tx=0x7f4630000ee0 comp rx=0 tx=0).stop 2024-04-17T10:50:21.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.164+0000 7f463ba69640 1 -- 172.21.15.83:0/3323854678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4634154c80 msgr2=0x7f4634155100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:21.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.164+0000 7f463ba69640 1 --2- 172.21.15.83:0/3323854678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4634154c80 0x7f4634155100 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f462c01c5b0 tx=0x7f4620005840 comp rx=0 tx=0).stop 2024-04-17T10:50:21.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.164+0000 7f463ba69640 1 -- 172.21.15.83:0/3323854678 shutdown_connections 2024-04-17T10:50:21.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.164+0000 7f463ba69640 1 --2- 172.21.15.83:0/3323854678 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4618066f60 0x7f4618069420 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:21.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.164+0000 7f463ba69640 1 --2- 172.21.15.83:0/3323854678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4634154c80 0x7f4634155100 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:21.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.164+0000 7f463ba69640 1 --2- 172.21.15.83:0/3323854678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46340ad840 0x7f46340adcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:21.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.164+0000 7f463ba69640 1 --2- 172.21.15.83:0/3323854678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46200051d0 0x7f4634150ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:21.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.164+0000 7f463ba69640 1 -- 172.21.15.83:0/3323854678 >> 172.21.15.83:0/3323854678 conn(0x7f463406abe0 msgr2=0x7f4634070e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:21.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.164+0000 7f463ba69640 1 -- 172.21.15.83:0/3323854678 shutdown_connections 2024-04-17T10:50:21.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.164+0000 7f463ba69640 1 -- 172.21.15.83:0/3323854678 wait complete. 2024-04-17T10:50:21.172 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:50:21.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f42945f1640 1 Processor -- start 2024-04-17T10:50:21.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f42945f1640 1 -- start start 2024-04-17T10:50:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f42945f1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f428c072330 0x7f428c0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f42945f1640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f428c0cfa10 0x7f428c06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f42945f1640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f428c0cfea0 con 0x7f428c15ba30 2024-04-17T10:50:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f42945f1640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f428c071ea0 con 0x7f428c0cfa10 2024-04-17T10:50:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f42945f1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f428c06e320 con 0x7f428c072330 2024-04-17T10:50:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f429238d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f428c15ba30 0x7f428c15c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39024/0 (socket says 172.21.15.83:39024) 2024-04-17T10:50:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f429238d640 1 -- 172.21.15.83:0/3561824390 learned_addr learned my addr 172.21.15.83:0/3561824390 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:21.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f429138b640 1 -- 172.21.15.83:0/3561824390 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 105349428 0 0) 0x7f428c0cfea0 con 0x7f428c15ba30 2024-04-17T10:50:21.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f429138b640 1 -- 172.21.15.83:0/3561824390 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4278003530 con 0x7f428c15ba30 2024-04-17T10:50:21.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f4291b8c640 1 --2- 172.21.15.83:0/3561824390 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f428c072330 0x7f428c0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:21.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f4292b8e640 1 --2- 172.21.15.83:0/3561824390 >> v2:172.21.15.167:3300/0 conn(0x7f428c0cfa10 0x7f428c06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f429138b640 1 -- 172.21.15.83:0/3561824390 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2195436732 0 0) 0x7f4278003530 con 0x7f428c15ba30 2024-04-17T10:50:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f429138b640 1 -- 172.21.15.83:0/3561824390 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f428c072330 msgr2=0x7f428c0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f429138b640 1 --2- 172.21.15.83:0/3561824390 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f428c072330 0x7f428c0cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f429138b640 1 -- 172.21.15.83:0/3561824390 >> v2:172.21.15.167:3300/0 conn(0x7f428c0cfa10 msgr2=0x7f428c06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f429138b640 1 --2- 172.21.15.83:0/3561824390 >> v2:172.21.15.167:3300/0 conn(0x7f428c0cfa10 0x7f428c06c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f429138b640 1 -- 172.21.15.83:0/3561824390 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f428c06e040 con 0x7f428c15ba30 2024-04-17T10:50:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f429138b640 1 -- 172.21.15.83:0/3561824390 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f4288001680 con 0x7f428c15ba30 2024-04-17T10:50:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f429138b640 1 -- 172.21.15.83:0/3561824390 >> v1:172.21.15.83:6789/0 conn(0x7f428c15ba30 legacy=0x7f428c15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f429138b640 1 --2- 172.21.15.83:0/3561824390 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f428c072330 0x7f42780039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f429138b640 1 --2- 172.21.15.83:0/3561824390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4278003ff0 0x7f4278004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f429138b640 1 --2- 172.21.15.83:0/3561824390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4278004d50 0x7f42780051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f429138b640 1 -- 172.21.15.83:0/3561824390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428c071ea0 con 0x7f4278003ff0 2024-04-17T10:50:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f429138b640 1 -- 172.21.15.83:0/3561824390 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428c06e320 con 0x7f4278004d50 2024-04-17T10:50:21.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f429138b640 1 -- 172.21.15.83:0/3561824390 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f427800bb90 con 0x7f428c072330 2024-04-17T10:50:21.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f4291b8c640 1 --2- 172.21.15.83:0/3561824390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4278003ff0 0x7f4278004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:21.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f4292b8e640 1 --2- 172.21.15.83:0/3561824390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4278004d50 0x7f42780051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:21.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f429238d640 1 --2- 172.21.15.83:0/3561824390 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f428c072330 0x7f42780039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:21.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f4291b8c640 1 -- 172.21.15.83:0/3561824390 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f428c072330 msgr2=0x7f42780039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:21.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f4291b8c640 1 --2- 172.21.15.83:0/3561824390 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f428c072330 0x7f42780039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:21.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f4291b8c640 1 -- 172.21.15.83:0/3561824390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4278004d50 msgr2=0x7f42780051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:21.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f4291b8c640 1 --2- 172.21.15.83:0/3561824390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4278004d50 0x7f42780051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:21.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.292+0000 7f4291b8c640 1 -- 172.21.15.83:0/3561824390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4278003530 con 0x7f4278003ff0 2024-04-17T10:50:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.296+0000 7f4291b8c640 1 --2- 172.21.15.83:0/3561824390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4278003ff0 0x7f4278004810 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f427c00ba30 tx=0x7f427c01c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f210d094939f26b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.296+0000 7f429138b640 1 -- 172.21.15.83:0/3561824390 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c00b600 con 0x7f4278003ff0 2024-04-17T10:50:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.296+0000 7f429138b640 1 -- 172.21.15.83:0/3561824390 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f427c0b1070 con 0x7f4278003ff0 2024-04-17T10:50:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.296+0000 7f429138b640 1 -- 172.21.15.83:0/3561824390 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c0ac680 con 0x7f4278003ff0 2024-04-17T10:50:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.296+0000 7f42945f1640 1 -- 172.21.15.83:0/3561824390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4278003ff0 msgr2=0x7f4278004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.296+0000 7f42945f1640 1 --2- 172.21.15.83:0/3561824390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4278003ff0 0x7f4278004810 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f427c00ba30 tx=0x7f427c01c6a0 comp rx=0 tx=0).stop 2024-04-17T10:50:21.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.296+0000 7f4292b8e640 1 -- 172.21.15.83:0/3561824390 reap_dead start 2024-04-17T10:50:21.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.296+0000 7f42945f1640 1 -- 172.21.15.83:0/3561824390 shutdown_connections 2024-04-17T10:50:21.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.296+0000 7f42945f1640 1 -- 172.21.15.83:0/3561824390 >> 172.21.15.83:0/3561824390 conn(0x7f428c06abe0 msgr2=0x7f428c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:21.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.296+0000 7f42945f1640 1 -- 172.21.15.83:0/3561824390 shutdown_connections 2024-04-17T10:50:21.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.296+0000 7f42945f1640 1 -- 172.21.15.83:0/3561824390 wait complete. 2024-04-17T10:50:21.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.296+0000 7f42945f1640 1 Processor -- start 2024-04-17T10:50:21.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.296+0000 7f42945f1640 1 -- start start 2024-04-17T10:50:21.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.296+0000 7f42945f1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4278003ff0 0x7f428c148160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:21.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.296+0000 7f42945f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f428c1566b0 0x7f428c1594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:21.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.296+0000 7f42945f1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f428c159a00 0x7f428c156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:21.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.296+0000 7f42945f1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428c06de20 con 0x7f428c1566b0 2024-04-17T10:50:21.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.296+0000 7f42945f1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428c06dca0 con 0x7f428c159a00 2024-04-17T10:50:21.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.296+0000 7f42945f1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f428c166510 con 0x7f4278003ff0 2024-04-17T10:50:21.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.296+0000 7f4291b8c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f428c1566b0 0x7f428c1594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:21.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.296+0000 7f4291b8c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f428c1566b0 0x7f428c1594c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40632/0 (socket says 172.21.15.83:40632) 2024-04-17T10:50:21.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.296+0000 7f4291b8c640 1 -- 172.21.15.83:0/1162350695 learned_addr learned my addr 172.21.15.83:0/1162350695 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:21.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.296+0000 7f4292b8e640 1 --2- 172.21.15.83:0/1162350695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f428c159a00 0x7f428c156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:21.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.296+0000 7f429238d640 1 --2- 172.21.15.83:0/1162350695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4278003ff0 0x7f428c148160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:21.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.296+0000 7f4291b8c640 1 -- 172.21.15.83:0/1162350695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4278003ff0 msgr2=0x7f428c148160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:21.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.296+0000 7f4291b8c640 1 --2- 172.21.15.83:0/1162350695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4278003ff0 0x7f428c148160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:21.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.296+0000 7f4291b8c640 1 -- 172.21.15.83:0/1162350695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f428c159a00 msgr2=0x7f428c156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:21.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.296+0000 7f4291b8c640 1 --2- 172.21.15.83:0/1162350695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f428c159a00 0x7f428c156f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:21.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.296+0000 7f4291b8c640 1 -- 172.21.15.83:0/1162350695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f428c15eff0 con 0x7f428c1566b0 2024-04-17T10:50:21.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.296+0000 7f4291b8c640 1 --2- 172.21.15.83:0/1162350695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f428c1566b0 0x7f428c1594c0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f427c0ae040 tx=0x7f4278005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:21.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.300+0000 7f4282ffd640 1 -- 172.21.15.83:0/1162350695 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c0bb070 con 0x7f428c1566b0 2024-04-17T10:50:21.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.300+0000 7f4282ffd640 1 -- 172.21.15.83:0/1162350695 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f427c004240 con 0x7f428c1566b0 2024-04-17T10:50:21.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.300+0000 7f42945f1640 1 -- 172.21.15.83:0/1162350695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f428c06d5d0 con 0x7f428c1566b0 2024-04-17T10:50:21.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.300+0000 7f4282ffd640 1 -- 172.21.15.83:0/1162350695 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c003620 con 0x7f428c1566b0 2024-04-17T10:50:21.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.300+0000 7f42945f1640 1 -- 172.21.15.83:0/1162350695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f428c155bd0 con 0x7f428c1566b0 2024-04-17T10:50:21.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.300+0000 7f42945f1640 1 -- 172.21.15.83:0/1162350695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f428c159e10 con 0x7f428c1566b0 2024-04-17T10:50:21.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.300+0000 7f4282ffd640 1 -- 172.21.15.83:0/1162350695 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f427c0037c0 con 0x7f428c1566b0 2024-04-17T10:50:21.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.300+0000 7f4282ffd640 1 --2- 172.21.15.83:0/1162350695 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4270066f60 0x7f4270069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:21.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.300+0000 7f4282ffd640 1 -- 172.21.15.83:0/1162350695 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f427c11f2d0 con 0x7f428c1566b0 2024-04-17T10:50:21.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.300+0000 7f429238d640 1 --2- 172.21.15.83:0/1162350695 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4270066f60 0x7f4270069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:21.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.300+0000 7f429238d640 1 --2- 172.21.15.83:0/1162350695 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4270066f60 0x7f4270069420 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f4288000c10 tx=0x7f4288009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:21.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.304+0000 7f4282ffd640 1 -- 172.21.15.83:0/1162350695 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f428c159e10 con 0x7f428c1566b0 2024-04-17T10:50:21.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.460+0000 7f42945f1640 1 -- 172.21.15.83:0/1162350695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f428c074f90 con 0x7f428c1566b0 2024-04-17T10:50:21.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.460+0000 7f4282ffd640 1 -- 172.21.15.83:0/1162350695 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+16777 (secure 0 0 0) 0x7f427c0163b0 con 0x7f428c1566b0 2024-04-17T10:50:21.455 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:50:21.455 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":130,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:50:12.167718+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:49:18.247824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-17T10:49:18.951409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.5","mappings":[{"from":2,"to":0}]},{"pgid":"19.25","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/1559222491":"2024-04-18T10:50:12.167626+0000","172.21.15.83:6836/1559222491":"2024-04-18T10:50:12.167626+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:50:21.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.464+0000 7f42945f1640 1 -- 172.21.15.83:0/1162350695 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4270066f60 msgr2=0x7f4270069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:21.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.464+0000 7f42945f1640 1 --2- 172.21.15.83:0/1162350695 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4270066f60 0x7f4270069420 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f4288000c10 tx=0x7f4288009040 comp rx=0 tx=0).stop 2024-04-17T10:50:21.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.464+0000 7f42945f1640 1 -- 172.21.15.83:0/1162350695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f428c1566b0 msgr2=0x7f428c1594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:21.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.464+0000 7f42945f1640 1 --2- 172.21.15.83:0/1162350695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f428c1566b0 0x7f428c1594c0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f427c0ae040 tx=0x7f4278005840 comp rx=0 tx=0).stop 2024-04-17T10:50:21.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.464+0000 7f42945f1640 1 -- 172.21.15.83:0/1162350695 shutdown_connections 2024-04-17T10:50:21.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.464+0000 7f42945f1640 1 --2- 172.21.15.83:0/1162350695 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4270066f60 0x7f4270069420 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:21.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.464+0000 7f42945f1640 1 --2- 172.21.15.83:0/1162350695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f428c159a00 0x7f428c156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:21.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.464+0000 7f42945f1640 1 --2- 172.21.15.83:0/1162350695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f428c1566b0 0x7f428c1594c0 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:21.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.464+0000 7f42945f1640 1 --2- 172.21.15.83:0/1162350695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4278003ff0 0x7f428c148160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:21.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.464+0000 7f42945f1640 1 -- 172.21.15.83:0/1162350695 >> 172.21.15.83:0/1162350695 conn(0x7f428c06abe0 msgr2=0x7f428c0af580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:21.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.464+0000 7f42945f1640 1 -- 172.21.15.83:0/1162350695 shutdown_connections 2024-04-17T10:50:21.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.464+0000 7f42945f1640 1 -- 172.21.15.83:0/1162350695 wait complete. 2024-04-17T10:50:21.473 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:49:18.247824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:49:18.951409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:50:21.473 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:50:21.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70b1fde640 1 Processor -- start 2024-04-17T10:50:21.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70b1fde640 1 -- start start 2024-04-17T10:50:21.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70b1fde640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f70ac15b6e0 0x7f70ac071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:21.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70b1fde640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70ac15bac0 0x7f70ac0cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:21.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70b1fde640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70ac06c1f0 con 0x7f70ac070470 2024-04-17T10:50:21.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70b1fde640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70ac0cdd30 con 0x7f70ac15b6e0 2024-04-17T10:50:21.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70b1fde640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac070a10 con 0x7f70ac15bac0 2024-04-17T10:50:21.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70abfff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f70ac070470 0x7f70ac06ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39036/0 (socket says 172.21.15.83:39036) 2024-04-17T10:50:21.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70abfff640 1 -- 172.21.15.83:0/3239642036 learned_addr learned my addr 172.21.15.83:0/3239642036 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:21.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70ab7fe640 1 --2- 172.21.15.83:0/3239642036 >> v2:172.21.15.167:3300/0 conn(0x7f70ac15b6e0 0x7f70ac071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:21.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70a3fff640 1 --2- 172.21.15.83:0/3239642036 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70ac15bac0 0x7f70ac0cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:21.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70aaffd640 1 -- 172.21.15.83:0/3239642036 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 46626532 0 0) 0x7f70ac06c1f0 con 0x7f70ac070470 2024-04-17T10:50:21.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70aaffd640 1 -- 172.21.15.83:0/3239642036 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7098003530 con 0x7f70ac070470 2024-04-17T10:50:21.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70aaffd640 1 -- 172.21.15.83:0/3239642036 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1949308558 0 0) 0x7f7098003530 con 0x7f70ac070470 2024-04-17T10:50:21.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70aaffd640 1 -- 172.21.15.83:0/3239642036 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70ac15bac0 msgr2=0x7f70ac0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:21.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70aaffd640 1 --2- 172.21.15.83:0/3239642036 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70ac15bac0 0x7f70ac0cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:21.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70aaffd640 1 -- 172.21.15.83:0/3239642036 >> v2:172.21.15.167:3300/0 conn(0x7f70ac15b6e0 msgr2=0x7f70ac071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:21.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70aaffd640 1 --2- 172.21.15.83:0/3239642036 >> v2:172.21.15.167:3300/0 conn(0x7f70ac15b6e0 0x7f70ac071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:21.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70aaffd640 1 -- 172.21.15.83:0/3239642036 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ac070cf0 con 0x7f70ac070470 2024-04-17T10:50:21.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70aaffd640 1 -- 172.21.15.83:0/3239642036 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f70a4001a20 con 0x7f70ac070470 2024-04-17T10:50:21.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70ab7fe640 1 --2- 172.21.15.83:0/3239642036 >> v2:172.21.15.167:3300/0 conn(0x7f70ac15b6e0 0x7f70ac071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:50:21.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70aaffd640 1 -- 172.21.15.83:0/3239642036 >> v1:172.21.15.83:6789/0 conn(0x7f70ac070470 legacy=0x7f70ac06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:21.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70aaffd640 1 --2- 172.21.15.83:0/3239642036 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70ac15bac0 0x7f7098003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:21.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70aaffd640 1 --2- 172.21.15.83:0/3239642036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7098003fb0 0x7f7098004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:21.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70aaffd640 1 --2- 172.21.15.83:0/3239642036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7098004d50 0x7f70980051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:21.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70aaffd640 1 -- 172.21.15.83:0/3239642036 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac0cdd30 con 0x7f7098004d50 2024-04-17T10:50:21.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70aaffd640 1 -- 172.21.15.83:0/3239642036 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac070a10 con 0x7f7098003fb0 2024-04-17T10:50:21.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70aaffd640 1 -- 172.21.15.83:0/3239642036 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f709800bab0 con 0x7f70ac15bac0 2024-04-17T10:50:21.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70abfff640 1 --2- 172.21.15.83:0/3239642036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7098004d50 0x7f70980051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:21.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70ab7fe640 1 --2- 172.21.15.83:0/3239642036 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70ac15bac0 0x7f7098003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:21.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70a3fff640 1 --2- 172.21.15.83:0/3239642036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7098003fb0 0x7f7098004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:21.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70abfff640 1 -- 172.21.15.83:0/3239642036 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70ac15bac0 msgr2=0x7f7098003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:21.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70abfff640 1 --2- 172.21.15.83:0/3239642036 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70ac15bac0 0x7f7098003a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:21.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70abfff640 1 -- 172.21.15.83:0/3239642036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7098003fb0 msgr2=0x7f7098004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:21.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70abfff640 1 --2- 172.21.15.83:0/3239642036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7098003fb0 0x7f7098004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:21.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.600+0000 7f70abfff640 1 -- 172.21.15.83:0/3239642036 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7098003530 con 0x7f7098004d50 2024-04-17T10:50:21.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.604+0000 7f70abfff640 1 --2- 172.21.15.83:0/3239642036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7098004d50 0x7f70980051d0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f70a400a880 tx=0x7f70a401d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de56c7412d0f91fd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:21.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.604+0000 7f70aaffd640 1 -- 172.21.15.83:0/3239642036 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a400a4d0 con 0x7f7098004d50 2024-04-17T10:50:21.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.604+0000 7f70aaffd640 1 -- 172.21.15.83:0/3239642036 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a40ad650 con 0x7f7098004d50 2024-04-17T10:50:21.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.604+0000 7f70aaffd640 1 -- 172.21.15.83:0/3239642036 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a40ada80 con 0x7f7098004d50 2024-04-17T10:50:21.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.604+0000 7f70b1fde640 1 -- 172.21.15.83:0/3239642036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7098004d50 msgr2=0x7f70980051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:21.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.604+0000 7f70b1fde640 1 --2- 172.21.15.83:0/3239642036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7098004d50 0x7f70980051d0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f70a400a880 tx=0x7f70a401d380 comp rx=0 tx=0).stop 2024-04-17T10:50:21.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.604+0000 7f70abfff640 1 -- 172.21.15.83:0/3239642036 reap_dead start 2024-04-17T10:50:21.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.604+0000 7f70b1fde640 1 -- 172.21.15.83:0/3239642036 shutdown_connections 2024-04-17T10:50:21.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.604+0000 7f70b1fde640 1 -- 172.21.15.83:0/3239642036 >> 172.21.15.83:0/3239642036 conn(0x7f70ac0b15b0 msgr2=0x7f70ac0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.604+0000 7f70b1fde640 1 -- 172.21.15.83:0/3239642036 shutdown_connections 2024-04-17T10:50:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.604+0000 7f70b1fde640 1 -- 172.21.15.83:0/3239642036 wait complete. 2024-04-17T10:50:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.604+0000 7f70b1fde640 1 Processor -- start 2024-04-17T10:50:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.604+0000 7f70b1fde640 1 -- start start 2024-04-17T10:50:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.604+0000 7f70b1fde640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7098004d50 0x7f70ac154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.604+0000 7f70b1fde640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70ac0abd90 0x7f70ac157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.604+0000 7f70b1fde640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70ac157ec0 0x7f70ac155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.604+0000 7f70b1fde640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac06b520 con 0x7f70ac157ec0 2024-04-17T10:50:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.604+0000 7f70b1fde640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac0712b0 con 0x7f70ac0abd90 2024-04-17T10:50:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.604+0000 7f70b1fde640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac06b6a0 con 0x7f7098004d50 2024-04-17T10:50:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.604+0000 7f70abfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70ac157ec0 0x7f70ac155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.604+0000 7f70abfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70ac157ec0 0x7f70ac155a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40656/0 (socket says 172.21.15.83:40656) 2024-04-17T10:50:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.604+0000 7f70abfff640 1 -- 172.21.15.83:0/2417537351 learned_addr learned my addr 172.21.15.83:0/2417537351 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.608+0000 7f70a3fff640 1 --2- 172.21.15.83:0/2417537351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70ac0abd90 0x7f70ac157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.608+0000 7f70ab7fe640 1 --2- 172.21.15.83:0/2417537351 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7098004d50 0x7f70ac154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.608+0000 7f70abfff640 1 -- 172.21.15.83:0/2417537351 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7098004d50 msgr2=0x7f70ac154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.608+0000 7f70abfff640 1 --2- 172.21.15.83:0/2417537351 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7098004d50 0x7f70ac154270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.608+0000 7f70abfff640 1 -- 172.21.15.83:0/2417537351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70ac0abd90 msgr2=0x7f70ac157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.608+0000 7f70abfff640 1 --2- 172.21.15.83:0/2417537351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70ac0abd90 0x7f70ac157980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.608+0000 7f70abfff640 1 -- 172.21.15.83:0/2417537351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ac1547b0 con 0x7f70ac157ec0 2024-04-17T10:50:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.608+0000 7f70abfff640 1 --2- 172.21.15.83:0/2417537351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70ac157ec0 0x7f70ac155a30 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f70a4098da0 tx=0x7f7098005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.608+0000 7f70a8ff9640 1 -- 172.21.15.83:0/2417537351 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a40ad430 con 0x7f70ac157ec0 2024-04-17T10:50:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.608+0000 7f70a8ff9640 1 -- 172.21.15.83:0/2417537351 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a40011d0 con 0x7f70ac157ec0 2024-04-17T10:50:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.608+0000 7f70b1fde640 1 -- 172.21.15.83:0/2417537351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70ac158420 con 0x7f70ac157ec0 2024-04-17T10:50:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.608+0000 7f70a8ff9640 1 -- 172.21.15.83:0/2417537351 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a4001960 con 0x7f70ac157ec0 2024-04-17T10:50:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.608+0000 7f70b1fde640 1 -- 172.21.15.83:0/2417537351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70ac159840 con 0x7f70ac157ec0 2024-04-17T10:50:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.608+0000 7f70b1fde640 1 -- 172.21.15.83:0/2417537351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70ac074f90 con 0x7f70ac157ec0 2024-04-17T10:50:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.608+0000 7f70a8ff9640 1 -- 172.21.15.83:0/2417537351 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f70a40b5070 con 0x7f70ac157ec0 2024-04-17T10:50:21.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.608+0000 7f70a8ff9640 1 --2- 172.21.15.83:0/2417537351 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f708c066f90 0x7f708c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:21.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.608+0000 7f70ab7fe640 1 --2- 172.21.15.83:0/2417537351 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f708c066f90 0x7f708c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:21.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.608+0000 7f70a8ff9640 1 -- 172.21.15.83:0/2417537351 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f70a411ed00 con 0x7f70ac157ec0 2024-04-17T10:50:21.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.608+0000 7f70ab7fe640 1 --2- 172.21.15.83:0/2417537351 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f708c066f90 0x7f708c069450 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f70ac0cc460 tx=0x7f709c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:21.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.612+0000 7f70a8ff9640 1 -- 172.21.15.83:0/2417537351 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f70a4017090 con 0x7f70ac157ec0 2024-04-17T10:50:21.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.760+0000 7f70b1fde640 1 -- 172.21.15.83:0/2417537351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f70ac071e00 con 0x7f70ac157ec0 2024-04-17T10:50:21.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.760+0000 7f70a8ff9640 1 -- 172.21.15.83:0/2417537351 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v130) v1 ==== 93+0+10 (secure 0 0 0) 0x7f70a40ab380 con 0x7f70ac157ec0 2024-04-17T10:50:21.755 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:50:21.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.764+0000 7f70b1fde640 1 -- 172.21.15.83:0/2417537351 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f708c066f90 msgr2=0x7f708c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:21.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.764+0000 7f70b1fde640 1 --2- 172.21.15.83:0/2417537351 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f708c066f90 0x7f708c069450 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f70ac0cc460 tx=0x7f709c008040 comp rx=0 tx=0).stop 2024-04-17T10:50:21.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.764+0000 7f70b1fde640 1 -- 172.21.15.83:0/2417537351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70ac157ec0 msgr2=0x7f70ac155a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:21.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.764+0000 7f70b1fde640 1 --2- 172.21.15.83:0/2417537351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70ac157ec0 0x7f70ac155a30 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f70a4098da0 tx=0x7f7098005840 comp rx=0 tx=0).stop 2024-04-17T10:50:21.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.764+0000 7f70b1fde640 1 -- 172.21.15.83:0/2417537351 shutdown_connections 2024-04-17T10:50:21.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.764+0000 7f70b1fde640 1 --2- 172.21.15.83:0/2417537351 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f708c066f90 0x7f708c069450 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:21.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.764+0000 7f70b1fde640 1 --2- 172.21.15.83:0/2417537351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70ac157ec0 0x7f70ac155a30 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:21.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.764+0000 7f70b1fde640 1 --2- 172.21.15.83:0/2417537351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70ac0abd90 0x7f70ac157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:21.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.764+0000 7f70b1fde640 1 --2- 172.21.15.83:0/2417537351 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7098004d50 0x7f70ac154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:21.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.764+0000 7f70b1fde640 1 -- 172.21.15.83:0/2417537351 >> 172.21.15.83:0/2417537351 conn(0x7f70ac0b15b0 msgr2=0x7f70ac0ce6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:21.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.764+0000 7f70b1fde640 1 -- 172.21.15.83:0/2417537351 shutdown_connections 2024-04-17T10:50:21.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.764+0000 7f70b1fde640 1 -- 172.21.15.83:0/2417537351 wait complete. 2024-04-17T10:50:21.773 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:50:21.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed6c77a640 1 Processor -- start 2024-04-17T10:50:21.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed6c77a640 1 -- start start 2024-04-17T10:50:21.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed6c77a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed6406b520 0x7fed64071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:21.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed6c77a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed6406c1f0 con 0x7fed6415b6e0 2024-04-17T10:50:21.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed6c77a640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed64070a10 con 0x7fed6415bac0 2024-04-17T10:50:21.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed6c77a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed64070cf0 con 0x7fed6406b520 2024-04-17T10:50:21.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed6a516640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fed6415b6e0 0x7fed640cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39050/0 (socket says 172.21.15.83:39050) 2024-04-17T10:50:21.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed6a516640 1 -- 172.21.15.83:0/2573371965 learned_addr learned my addr 172.21.15.83:0/2573371965 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:21.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed6ad17640 1 --2- 172.21.15.83:0/2573371965 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed6406b520 0x7fed64071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:21.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed69514640 1 -- 172.21.15.83:0/2573371965 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1358287654 0 0) 0x7fed6406c1f0 con 0x7fed6415b6e0 2024-04-17T10:50:21.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed69514640 1 -- 172.21.15.83:0/2573371965 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed50003530 con 0x7fed6415b6e0 2024-04-17T10:50:21.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed69514640 1 -- 172.21.15.83:0/2573371965 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2440188689 0 0) 0x7fed50003530 con 0x7fed6415b6e0 2024-04-17T10:50:21.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed69514640 1 -- 172.21.15.83:0/2573371965 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed6406b520 msgr2=0x7fed64071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:21.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed69514640 1 --2- 172.21.15.83:0/2573371965 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed6406b520 0x7fed64071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:21.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed69514640 1 -- 172.21.15.83:0/2573371965 >> v1:172.21.15.167:6789/0 conn(0x7fed6415bac0 legacy=0x7fed6406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:21.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed69514640 1 -- 172.21.15.83:0/2573371965 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed64070fd0 con 0x7fed6415b6e0 2024-04-17T10:50:21.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed69514640 1 -- 172.21.15.83:0/2573371965 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fed60001680 con 0x7fed6415b6e0 2024-04-17T10:50:21.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed69514640 1 -- 172.21.15.83:0/2573371965 >> v1:172.21.15.83:6789/0 conn(0x7fed6415b6e0 legacy=0x7fed640cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:21.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed69514640 1 --2- 172.21.15.83:0/2573371965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed50003a60 0x7fed50003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:21.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed69514640 1 --2- 172.21.15.83:0/2573371965 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed6406b520 0x7fed50004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:21.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed69514640 1 --2- 172.21.15.83:0/2573371965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed50004d50 0x7fed500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:21.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed69514640 1 -- 172.21.15.83:0/2573371965 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed64070cf0 con 0x7fed50003a60 2024-04-17T10:50:21.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed69514640 1 -- 172.21.15.83:0/2573371965 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5000bae0 con 0x7fed50004d50 2024-04-17T10:50:21.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed69514640 1 -- 172.21.15.83:0/2573371965 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed5000bce0 con 0x7fed6406b520 2024-04-17T10:50:21.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed6a516640 1 --2- 172.21.15.83:0/2573371965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed50003a60 0x7fed50003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:21.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed6ad17640 1 --2- 172.21.15.83:0/2573371965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed50004d50 0x7fed500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:21.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed69d15640 1 --2- 172.21.15.83:0/2573371965 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed6406b520 0x7fed50004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:21.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed6a516640 1 -- 172.21.15.83:0/2573371965 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed6406b520 msgr2=0x7fed50004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:21.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed6a516640 1 --2- 172.21.15.83:0/2573371965 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed6406b520 0x7fed50004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:21.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed6a516640 1 -- 172.21.15.83:0/2573371965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed50004d50 msgr2=0x7fed500051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:21.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed6a516640 1 --2- 172.21.15.83:0/2573371965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed50004d50 0x7fed500051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:21.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed6a516640 1 -- 172.21.15.83:0/2573371965 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed50003530 con 0x7fed50003a60 2024-04-17T10:50:21.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.900+0000 7fed69d15640 1 --2- 172.21.15.83:0/2573371965 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed6406b520 0x7fed50004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:50:21.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.904+0000 7fed6a516640 1 --2- 172.21.15.83:0/2573371965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed50003a60 0x7fed50003e60 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fed6000c210 tx=0x7fed6001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d1c8e8c6771fb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:21.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.904+0000 7fed69514640 1 -- 172.21.15.83:0/2573371965 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed60007d40 con 0x7fed50003a60 2024-04-17T10:50:21.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.904+0000 7fed69514640 1 -- 172.21.15.83:0/2573371965 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed600a9070 con 0x7fed50003a60 2024-04-17T10:50:21.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.904+0000 7fed69514640 1 -- 172.21.15.83:0/2573371965 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed60005d40 con 0x7fed50003a60 2024-04-17T10:50:21.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.904+0000 7fed6c77a640 1 -- 172.21.15.83:0/2573371965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed50003a60 msgr2=0x7fed50003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:21.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.904+0000 7fed6c77a640 1 --2- 172.21.15.83:0/2573371965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed50003a60 0x7fed50003e60 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fed6000c210 tx=0x7fed6001cc80 comp rx=0 tx=0).stop 2024-04-17T10:50:21.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.904+0000 7fed6ad17640 1 -- 172.21.15.83:0/2573371965 reap_dead start 2024-04-17T10:50:21.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.904+0000 7fed6c77a640 1 -- 172.21.15.83:0/2573371965 shutdown_connections 2024-04-17T10:50:21.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.904+0000 7fed6c77a640 1 -- 172.21.15.83:0/2573371965 >> 172.21.15.83:0/2573371965 conn(0x7fed640b15b0 msgr2=0x7fed640b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:21.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.904+0000 7fed6c77a640 1 -- 172.21.15.83:0/2573371965 shutdown_connections 2024-04-17T10:50:21.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.904+0000 7fed6c77a640 1 -- 172.21.15.83:0/2573371965 wait complete. 2024-04-17T10:50:21.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.904+0000 7fed6c77a640 1 Processor -- start 2024-04-17T10:50:21.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.904+0000 7fed6c77a640 1 -- start start 2024-04-17T10:50:21.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.908+0000 7fed6c77a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed50003a60 0x7fed641f4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:21.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.908+0000 7fed6c77a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed640a9430 0x7fed640aaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:21.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.908+0000 7fed6c77a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed640a98b0 0x7fed64157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.908+0000 7fed6c77a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6406d2f0 con 0x7fed50003a60 2024-04-17T10:50:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.908+0000 7fed6c77a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6406d170 con 0x7fed640a98b0 2024-04-17T10:50:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.908+0000 7fed6c77a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed64167600 con 0x7fed640a9430 2024-04-17T10:50:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.908+0000 7fed6a516640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed50003a60 0x7fed641f4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.908+0000 7fed6a516640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed50003a60 0x7fed641f4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40676/0 (socket says 172.21.15.83:40676) 2024-04-17T10:50:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.908+0000 7fed6a516640 1 -- 172.21.15.83:0/1399741904 learned_addr learned my addr 172.21.15.83:0/1399741904 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.908+0000 7fed6ad17640 1 --2- 172.21.15.83:0/1399741904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed640a98b0 0x7fed64157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.908+0000 7fed69d15640 1 --2- 172.21.15.83:0/1399741904 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed640a9430 0x7fed640aaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.908+0000 7fed6a516640 1 -- 172.21.15.83:0/1399741904 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed640a9430 msgr2=0x7fed640aaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.908+0000 7fed6a516640 1 --2- 172.21.15.83:0/1399741904 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed640a9430 0x7fed640aaeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.908+0000 7fed6a516640 1 -- 172.21.15.83:0/1399741904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed640a98b0 msgr2=0x7fed64157e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.908+0000 7fed6a516640 1 --2- 172.21.15.83:0/1399741904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed640a98b0 0x7fed64157e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.908+0000 7fed6a516640 1 -- 172.21.15.83:0/1399741904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed640a9e10 con 0x7fed50003a60 2024-04-17T10:50:21.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.908+0000 7fed6ad17640 1 --2- 172.21.15.83:0/1399741904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed640a98b0 0x7fed64157e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:21.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.908+0000 7fed6a516640 1 --2- 172.21.15.83:0/1399741904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed50003a60 0x7fed641f4970 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fed6000c210 tx=0x7fed50005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:21.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.908+0000 7fed5affd640 1 -- 172.21.15.83:0/1399741904 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed600a5870 con 0x7fed50003a60 2024-04-17T10:50:21.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.908+0000 7fed5affd640 1 -- 172.21.15.83:0/1399741904 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed600a5a10 con 0x7fed50003a60 2024-04-17T10:50:21.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.908+0000 7fed6c77a640 1 -- 172.21.15.83:0/1399741904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed640aa3d0 con 0x7fed50003a60 2024-04-17T10:50:21.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.908+0000 7fed5affd640 1 -- 172.21.15.83:0/1399741904 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed600a5dc0 con 0x7fed50003a60 2024-04-17T10:50:21.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.908+0000 7fed6c77a640 1 -- 172.21.15.83:0/1399741904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed640aa990 con 0x7fed50003a60 2024-04-17T10:50:21.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.908+0000 7fed6c77a640 1 -- 172.21.15.83:0/1399741904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed641f5270 con 0x7fed50003a60 2024-04-17T10:50:21.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.908+0000 7fed5affd640 1 -- 172.21.15.83:0/1399741904 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fed600b7030 con 0x7fed50003a60 2024-04-17T10:50:21.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.908+0000 7fed5affd640 1 --2- 172.21.15.83:0/1399741904 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fed48066f90 0x7fed48069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:21.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.908+0000 7fed5affd640 1 -- 172.21.15.83:0/1399741904 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fed6011e3d0 con 0x7fed50003a60 2024-04-17T10:50:21.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.908+0000 7fed69d15640 1 --2- 172.21.15.83:0/1399741904 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fed48066f90 0x7fed48069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:21.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.908+0000 7fed69d15640 1 --2- 172.21.15.83:0/1399741904 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fed48066f90 0x7fed48069450 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fed64150890 tx=0x7fed54009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:21.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:21.912+0000 7fed5affd640 1 -- 172.21.15.83:0/1399741904 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fed60016990 con 0x7fed50003a60 2024-04-17T10:50:22.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.060+0000 7fed6c77a640 1 -- 172.21.15.83:0/1399741904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fed64074f90 con 0x7fed50003a60 2024-04-17T10:50:22.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.060+0000 7fed5affd640 1 -- 172.21.15.83:0/1399741904 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v130) v1 ==== 104+0+11 (secure 0 0 0) 0x7fed600ef2c0 con 0x7fed50003a60 2024-04-17T10:50:22.053 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:50:22.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.060+0000 7fed6c77a640 1 -- 172.21.15.83:0/1399741904 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fed48066f90 msgr2=0x7fed48069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:22.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.060+0000 7fed6c77a640 1 --2- 172.21.15.83:0/1399741904 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fed48066f90 0x7fed48069450 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fed64150890 tx=0x7fed54009040 comp rx=0 tx=0).stop 2024-04-17T10:50:22.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.060+0000 7fed6c77a640 1 -- 172.21.15.83:0/1399741904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed50003a60 msgr2=0x7fed641f4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:22.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.060+0000 7fed6c77a640 1 --2- 172.21.15.83:0/1399741904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed50003a60 0x7fed641f4970 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fed6000c210 tx=0x7fed50005840 comp rx=0 tx=0).stop 2024-04-17T10:50:22.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.060+0000 7fed6c77a640 1 -- 172.21.15.83:0/1399741904 shutdown_connections 2024-04-17T10:50:22.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.060+0000 7fed6c77a640 1 --2- 172.21.15.83:0/1399741904 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fed48066f90 0x7fed48069450 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:22.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.060+0000 7fed6c77a640 1 --2- 172.21.15.83:0/1399741904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed640a98b0 0x7fed64157e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:22.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.060+0000 7fed6c77a640 1 --2- 172.21.15.83:0/1399741904 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed640a9430 0x7fed640aaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:22.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.060+0000 7fed6c77a640 1 --2- 172.21.15.83:0/1399741904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed50003a60 0x7fed641f4970 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:22.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.060+0000 7fed6c77a640 1 -- 172.21.15.83:0/1399741904 >> 172.21.15.83:0/1399741904 conn(0x7fed640b15b0 msgr2=0x7fed64156600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:22.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.064+0000 7fed6c77a640 1 -- 172.21.15.83:0/1399741904 shutdown_connections 2024-04-17T10:50:22.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.064+0000 7fed6c77a640 1 -- 172.21.15.83:0/1399741904 wait complete. 2024-04-17T10:50:22.071 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:50:22.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.200+0000 7f7323eb9640 1 Processor -- start 2024-04-17T10:50:22.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.200+0000 7f7323eb9640 1 -- start start 2024-04-17T10:50:22.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.200+0000 7f7323eb9640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f731c0d0490 0x7f731c0d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:22.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.200+0000 7f7323eb9640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f731c06e500 0x7f731c06e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:22.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.200+0000 7f7323eb9640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f731c0ced00 con 0x7f731c15bc10 2024-04-17T10:50:22.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.200+0000 7f7323eb9640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f731c0cefe0 con 0x7f731c0d0490 2024-04-17T10:50:22.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.200+0000 7f7323eb9640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f731c0cc690 con 0x7f731c06e500 2024-04-17T10:50:22.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.200+0000 7f7321454640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f731c0d0490 0x7f731c0d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:22.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.200+0000 7f7321c55640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f731c15bc10 0x7f731c0cce90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39060/0 (socket says 172.21.15.83:39060) 2024-04-17T10:50:22.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.200+0000 7f7321c55640 1 -- 172.21.15.83:0/3218408161 learned_addr learned my addr 172.21.15.83:0/3218408161 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:22.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.200+0000 7f7321454640 1 -- 172.21.15.83:0/3218408161 >> v2:172.21.15.167:3300/0 conn(0x7f731c06e500 msgr2=0x7f731c06e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:22.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.200+0000 7f7321454640 1 --2- 172.21.15.83:0/3218408161 >> v2:172.21.15.167:3300/0 conn(0x7f731c06e500 0x7f731c06e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:22.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.200+0000 7f7321454640 1 -- 172.21.15.83:0/3218408161 >> v1:172.21.15.83:6789/0 conn(0x7f731c15bc10 legacy=0x7f731c0cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:22.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.200+0000 7f7321454640 1 -- 172.21.15.83:0/3218408161 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f731c0cc970 con 0x7f731c0d0490 2024-04-17T10:50:22.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.204+0000 7f7321454640 1 --2- 172.21.15.83:0/3218408161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f731c0d0490 0x7f731c0d2300 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f731c0b4330 tx=0x7f730c01c280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd532a36c8b0220 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:22.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.204+0000 7f7320c53640 1 -- 172.21.15.83:0/3218408161 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730c0b0070 con 0x7f731c0d0490 2024-04-17T10:50:22.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.204+0000 7f7320c53640 1 -- 172.21.15.83:0/3218408161 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f730c0ab600 con 0x7f731c0d0490 2024-04-17T10:50:22.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.204+0000 7f7320c53640 1 -- 172.21.15.83:0/3218408161 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730c0ab9d0 con 0x7f731c0d0490 2024-04-17T10:50:22.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.204+0000 7f7323eb9640 1 -- 172.21.15.83:0/3218408161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f731c0d0490 msgr2=0x7f731c0d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:22.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.204+0000 7f7323eb9640 1 --2- 172.21.15.83:0/3218408161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f731c0d0490 0x7f731c0d2300 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f731c0b4330 tx=0x7f730c01c280 comp rx=0 tx=0).stop 2024-04-17T10:50:22.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.204+0000 7f7323eb9640 1 -- 172.21.15.83:0/3218408161 shutdown_connections 2024-04-17T10:50:22.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.204+0000 7f7323eb9640 1 --2- 172.21.15.83:0/3218408161 >> v2:172.21.15.167:3300/0 conn(0x7f731c06e500 0x7f731c06e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:22.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.204+0000 7f7323eb9640 1 --2- 172.21.15.83:0/3218408161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f731c0d0490 0x7f731c0d2300 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:22.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.204+0000 7f7323eb9640 1 -- 172.21.15.83:0/3218408161 >> 172.21.15.83:0/3218408161 conn(0x7f731c0d17e0 msgr2=0x7f731c0d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:22.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.204+0000 7f7323eb9640 1 -- 172.21.15.83:0/3218408161 shutdown_connections 2024-04-17T10:50:22.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.204+0000 7f7323eb9640 1 -- 172.21.15.83:0/3218408161 wait complete. 2024-04-17T10:50:22.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.204+0000 7f7323eb9640 1 Processor -- start 2024-04-17T10:50:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.204+0000 7f7323eb9640 1 -- start start 2024-04-17T10:50:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.204+0000 7f7323eb9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f731c06e500 0x7f731c06d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.204+0000 7f7323eb9640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f731c0d0490 0x7f731c15a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.204+0000 7f7323eb9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f731c15bc10 0x7f731c06bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.204+0000 7f7323eb9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731c168b10 con 0x7f731c06e500 2024-04-17T10:50:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.204+0000 7f7323eb9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731c168990 con 0x7f731c15bc10 2024-04-17T10:50:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.204+0000 7f7323eb9640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f731c168c90 con 0x7f731c0d0490 2024-04-17T10:50:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.204+0000 7f7321c55640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f731c06e500 0x7f731c06d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.204+0000 7f7321c55640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f731c06e500 0x7f731c06d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40702/0 (socket says 172.21.15.83:40702) 2024-04-17T10:50:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.204+0000 7f7321c55640 1 -- 172.21.15.83:0/1737832642 learned_addr learned my addr 172.21.15.83:0/1737832642 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.208+0000 7f7321c55640 1 -- 172.21.15.83:0/1737832642 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f731c0d0490 msgr2=0x7f731c15a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.208+0000 7f7321c55640 1 --2- 172.21.15.83:0/1737832642 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f731c0d0490 0x7f731c15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.208+0000 7f7321c55640 1 -- 172.21.15.83:0/1737832642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f731c15bc10 msgr2=0x7f731c06bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:22.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.208+0000 7f7321c55640 1 --2- 172.21.15.83:0/1737832642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f731c15bc10 0x7f731c06bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:22.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.208+0000 7f7321c55640 1 -- 172.21.15.83:0/1737832642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f731c159470 con 0x7f731c06e500 2024-04-17T10:50:22.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.208+0000 7f7321c55640 1 --2- 172.21.15.83:0/1737832642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f731c06e500 0x7f731c06d560 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f731800cb20 tx=0x7f7318002150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:22.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.208+0000 7f73127fc640 1 -- 172.21.15.83:0/1737832642 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7318018070 con 0x7f731c06e500 2024-04-17T10:50:22.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.208+0000 7f73127fc640 1 -- 172.21.15.83:0/1737832642 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731800a7c0 con 0x7f731c06e500 2024-04-17T10:50:22.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.208+0000 7f73127fc640 1 -- 172.21.15.83:0/1737832642 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731800abb0 con 0x7f731c06e500 2024-04-17T10:50:22.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.208+0000 7f7323eb9640 1 -- 172.21.15.83:0/1737832642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f731c06ca70 con 0x7f731c06e500 2024-04-17T10:50:22.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.208+0000 7f7323eb9640 1 -- 172.21.15.83:0/1737832642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f731c06daa0 con 0x7f731c06e500 2024-04-17T10:50:22.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.208+0000 7f7323eb9640 1 -- 172.21.15.83:0/1737832642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f731c153750 con 0x7f731c06e500 2024-04-17T10:50:22.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.208+0000 7f73127fc640 1 -- 172.21.15.83:0/1737832642 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f73180070d0 con 0x7f731c06e500 2024-04-17T10:50:22.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.208+0000 7f73127fc640 1 --2- 172.21.15.83:0/1737832642 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7300066f60 0x7f7300069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:22.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.208+0000 7f73127fc640 1 -- 172.21.15.83:0/1737832642 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f7318084920 con 0x7f731c06e500 2024-04-17T10:50:22.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.208+0000 7f7321454640 1 --2- 172.21.15.83:0/1737832642 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7300066f60 0x7f7300069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:22.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.208+0000 7f7321454640 1 --2- 172.21.15.83:0/1737832642 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7300066f60 0x7f7300069420 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f730c0ad000 tx=0x7f730c004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:22.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.212+0000 7f73127fc640 1 -- 172.21.15.83:0/1737832642 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f73180558a0 con 0x7f731c06e500 2024-04-17T10:50:22.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.372+0000 7f7323eb9640 1 -- 172.21.15.83:0/1737832642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f731c074f90 con 0x7f731c06e500 2024-04-17T10:50:22.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.372+0000 7f73127fc640 1 -- 172.21.15.83:0/1737832642 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v130) v1 ==== 100+0+11 (secure 0 0 0) 0x7f73180551a0 con 0x7f731c06e500 2024-04-17T10:50:22.365 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:50:22.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.372+0000 7f7323eb9640 1 -- 172.21.15.83:0/1737832642 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7300066f60 msgr2=0x7f7300069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:22.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.372+0000 7f7323eb9640 1 --2- 172.21.15.83:0/1737832642 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7300066f60 0x7f7300069420 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f730c0ad000 tx=0x7f730c004740 comp rx=0 tx=0).stop 2024-04-17T10:50:22.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.372+0000 7f7323eb9640 1 -- 172.21.15.83:0/1737832642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f731c06e500 msgr2=0x7f731c06d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:22.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.372+0000 7f7323eb9640 1 --2- 172.21.15.83:0/1737832642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f731c06e500 0x7f731c06d560 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f731800cb20 tx=0x7f7318002150 comp rx=0 tx=0).stop 2024-04-17T10:50:22.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.372+0000 7f7323eb9640 1 -- 172.21.15.83:0/1737832642 shutdown_connections 2024-04-17T10:50:22.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.372+0000 7f7323eb9640 1 --2- 172.21.15.83:0/1737832642 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7300066f60 0x7f7300069420 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:22.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.372+0000 7f7323eb9640 1 --2- 172.21.15.83:0/1737832642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f731c15bc10 0x7f731c06bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:22.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.372+0000 7f7323eb9640 1 --2- 172.21.15.83:0/1737832642 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f731c0d0490 0x7f731c15a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:22.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.372+0000 7f7323eb9640 1 --2- 172.21.15.83:0/1737832642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f731c06e500 0x7f731c06d560 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:22.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.372+0000 7f7323eb9640 1 -- 172.21.15.83:0/1737832642 >> 172.21.15.83:0/1737832642 conn(0x7f731c0d17e0 msgr2=0x7f731c0d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:22.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.376+0000 7f7323eb9640 1 -- 172.21.15.83:0/1737832642 shutdown_connections 2024-04-17T10:50:22.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.376+0000 7f7323eb9640 1 -- 172.21.15.83:0/1737832642 wait complete. 2024-04-17T10:50:22.385 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:50:22.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.516+0000 7f99385f4640 1 Processor -- start 2024-04-17T10:50:22.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.516+0000 7f99385f4640 1 -- start start 2024-04-17T10:50:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.516+0000 7f99385f4640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9930161300 0x7f993006ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.516+0000 7f99385f4640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9930161900 con 0x7f993006c300 2024-04-17T10:50:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.516+0000 7f99385f4640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9930161ba0 con 0x7f993006f360 2024-04-17T10:50:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.516+0000 7f99385f4640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9930161e80 con 0x7f9930161300 2024-04-17T10:50:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.516+0000 7f9936b91640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f993006c300 0x7f9930071750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39070/0 (socket says 172.21.15.83:39070) 2024-04-17T10:50:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.516+0000 7f9936b91640 1 -- 172.21.15.83:0/552992582 learned_addr learned my addr 172.21.15.83:0/552992582 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.516+0000 7f9936390640 1 --2- 172.21.15.83:0/552992582 >> v2:172.21.15.167:3300/0 conn(0x7f9930161300 0x7f993006ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.516+0000 7f993538e640 1 -- 172.21.15.83:0/552992582 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 418348640 0 0) 0x7f9930161900 con 0x7f993006c300 2024-04-17T10:50:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.516+0000 7f993538e640 1 -- 172.21.15.83:0/552992582 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f991c003530 con 0x7f993006c300 2024-04-17T10:50:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.516+0000 7f993538e640 1 -- 172.21.15.83:0/552992582 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1677893359 0 0) 0x7f991c003530 con 0x7f993006c300 2024-04-17T10:50:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.516+0000 7f993538e640 1 -- 172.21.15.83:0/552992582 >> v2:172.21.15.167:3300/0 conn(0x7f9930161300 msgr2=0x7f993006ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.516+0000 7f993538e640 1 --2- 172.21.15.83:0/552992582 >> v2:172.21.15.167:3300/0 conn(0x7f9930161300 0x7f993006ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.516+0000 7f993538e640 1 -- 172.21.15.83:0/552992582 >> v1:172.21.15.167:6789/0 conn(0x7f993006f360 legacy=0x7f993006bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.516+0000 7f993538e640 1 -- 172.21.15.83:0/552992582 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9930162160 con 0x7f993006c300 2024-04-17T10:50:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.516+0000 7f993538e640 1 -- 172.21.15.83:0/552992582 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f9928001a00 con 0x7f993006c300 2024-04-17T10:50:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.516+0000 7f993538e640 1 -- 172.21.15.83:0/552992582 >> v1:172.21.15.83:6789/0 conn(0x7f993006c300 legacy=0x7f9930071750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.516+0000 7f993538e640 1 --2- 172.21.15.83:0/552992582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f991c0039c0 0x7f991c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.516+0000 7f993538e640 1 --2- 172.21.15.83:0/552992582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f991c004810 0x7f991c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.516+0000 7f993538e640 1 --2- 172.21.15.83:0/552992582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f991c005220 0x7f991c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.516+0000 7f993538e640 1 -- 172.21.15.83:0/552992582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9930161e80 con 0x7f991c005220 2024-04-17T10:50:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.516+0000 7f993538e640 1 -- 172.21.15.83:0/552992582 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c00c150 con 0x7f991c004810 2024-04-17T10:50:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.516+0000 7f993538e640 1 -- 172.21.15.83:0/552992582 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991c00c350 con 0x7f991c0039c0 2024-04-17T10:50:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.516+0000 7f9936b91640 1 --2- 172.21.15.83:0/552992582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f991c005220 0x7f991c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.516+0000 7f9935b8f640 1 --2- 172.21.15.83:0/552992582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f991c004810 0x7f991c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.516+0000 7f9936390640 1 --2- 172.21.15.83:0/552992582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f991c0039c0 0x7f991c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.516+0000 7f9936b91640 1 -- 172.21.15.83:0/552992582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f991c0039c0 msgr2=0x7f991c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.516+0000 7f9936b91640 1 --2- 172.21.15.83:0/552992582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f991c0039c0 0x7f991c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.516+0000 7f9936b91640 1 -- 172.21.15.83:0/552992582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f991c004810 msgr2=0x7f991c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.516+0000 7f9936b91640 1 --2- 172.21.15.83:0/552992582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f991c004810 0x7f991c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.516+0000 7f9936b91640 1 -- 172.21.15.83:0/552992582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991c003530 con 0x7f991c005220 2024-04-17T10:50:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.520+0000 7f9936b91640 1 -- 172.21.15.83:0/552992582 reap_dead start 2024-04-17T10:50:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.520+0000 7f9936b91640 1 --2- 172.21.15.83:0/552992582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f991c005220 0x7f991c00b9d0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f99300cfd70 tx=0x7f992801fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afcaf2ad2adb6e84 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.520+0000 7f993538e640 1 -- 172.21.15.83:0/552992582 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992800a4a0 con 0x7f991c005220 2024-04-17T10:50:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.520+0000 7f993538e640 1 -- 172.21.15.83:0/552992582 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99280aa4b0 con 0x7f991c005220 2024-04-17T10:50:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.520+0000 7f993538e640 1 -- 172.21.15.83:0/552992582 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99280aa8a0 con 0x7f991c005220 2024-04-17T10:50:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.520+0000 7f99385f4640 1 -- 172.21.15.83:0/552992582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f991c005220 msgr2=0x7f991c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.520+0000 7f99385f4640 1 --2- 172.21.15.83:0/552992582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f991c005220 0x7f991c00b9d0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f99300cfd70 tx=0x7f992801fa80 comp rx=0 tx=0).stop 2024-04-17T10:50:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.520+0000 7f99385f4640 1 -- 172.21.15.83:0/552992582 shutdown_connections 2024-04-17T10:50:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.520+0000 7f99385f4640 1 --2- 172.21.15.83:0/552992582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f991c005220 0x7f991c00b9d0 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.520+0000 7f99385f4640 1 -- 172.21.15.83:0/552992582 >> 172.21.15.83:0/552992582 conn(0x7f99300cc910 msgr2=0x7f99300ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:22.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.520+0000 7f99385f4640 1 -- 172.21.15.83:0/552992582 shutdown_connections 2024-04-17T10:50:22.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.520+0000 7f99385f4640 1 -- 172.21.15.83:0/552992582 wait complete. 2024-04-17T10:50:22.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.520+0000 7f99385f4640 1 Processor -- start 2024-04-17T10:50:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.520+0000 7f99385f4640 1 -- start start 2024-04-17T10:50:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.524+0000 7f99385f4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f991c005220 0x7f99300c65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.524+0000 7f99385f4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99300a06d0 0x7f99300b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.524+0000 7f99385f4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99300b2f80 0x7f99300b5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.524+0000 7f99385f4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993006efa0 con 0x7f99300b2f80 2024-04-17T10:50:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.524+0000 7f99385f4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993006ee20 con 0x7f991c005220 2024-04-17T10:50:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.524+0000 7f99385f4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993006f120 con 0x7f99300a06d0 2024-04-17T10:50:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.524+0000 7f9936b91640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99300b2f80 0x7f99300b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.524+0000 7f9936b91640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99300b2f80 0x7f99300b5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40714/0 (socket says 172.21.15.83:40714) 2024-04-17T10:50:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.524+0000 7f9936b91640 1 -- 172.21.15.83:0/1202465006 learned_addr learned my addr 172.21.15.83:0/1202465006 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.524+0000 7f9936390640 1 --2- 172.21.15.83:0/1202465006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f991c005220 0x7f99300c65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.524+0000 7f9935b8f640 1 --2- 172.21.15.83:0/1202465006 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99300a06d0 0x7f99300b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:22.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.524+0000 7f9936b91640 1 -- 172.21.15.83:0/1202465006 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99300a06d0 msgr2=0x7f99300b2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:22.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.524+0000 7f9936b91640 1 --2- 172.21.15.83:0/1202465006 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99300a06d0 0x7f99300b2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:22.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.524+0000 7f9936b91640 1 -- 172.21.15.83:0/1202465006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f991c005220 msgr2=0x7f99300c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:22.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.524+0000 7f9936b91640 1 --2- 172.21.15.83:0/1202465006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f991c005220 0x7f99300c65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:22.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.524+0000 7f9936b91640 1 -- 172.21.15.83:0/1202465006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993009f630 con 0x7f99300b2f80 2024-04-17T10:50:22.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.524+0000 7f9936b91640 1 --2- 172.21.15.83:0/1202465006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99300b2f80 0x7f99300b5320 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f9928020320 tx=0x7f991c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:22.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.524+0000 7f9926ffd640 1 -- 172.21.15.83:0/1202465006 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99280bd070 con 0x7f99300b2f80 2024-04-17T10:50:22.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.524+0000 7f9926ffd640 1 -- 172.21.15.83:0/1202465006 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99280017c0 con 0x7f99300b2f80 2024-04-17T10:50:22.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.524+0000 7f99385f4640 1 -- 172.21.15.83:0/1202465006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f993009f910 con 0x7f99300b2f80 2024-04-17T10:50:22.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.524+0000 7f9926ffd640 1 -- 172.21.15.83:0/1202465006 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99280aa660 con 0x7f99300b2f80 2024-04-17T10:50:22.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.524+0000 7f99385f4640 1 -- 172.21.15.83:0/1202465006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f993009fed0 con 0x7f99300b2f80 2024-04-17T10:50:22.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.524+0000 7f99385f4640 1 -- 172.21.15.83:0/1202465006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9930074f90 con 0x7f99300b2f80 2024-04-17T10:50:22.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.524+0000 7f9926ffd640 1 -- 172.21.15.83:0/1202465006 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f99280c0070 con 0x7f99300b2f80 2024-04-17T10:50:22.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.524+0000 7f9926ffd640 1 --2- 172.21.15.83:0/1202465006 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9914066f60 0x7f9914069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:22.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.524+0000 7f9926ffd640 1 -- 172.21.15.83:0/1202465006 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f9928121bf0 con 0x7f99300b2f80 2024-04-17T10:50:22.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.524+0000 7f9936390640 1 --2- 172.21.15.83:0/1202465006 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9914066f60 0x7f9914069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:22.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.524+0000 7f9936390640 1 --2- 172.21.15.83:0/1202465006 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9914066f60 0x7f9914069420 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f992c0099b0 tx=0x7f992c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:22.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.528+0000 7f9926ffd640 1 -- 172.21.15.83:0/1202465006 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9928019790 con 0x7f99300b2f80 2024-04-17T10:50:22.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.712+0000 7f99385f4640 1 -- 172.21.15.83:0/1202465006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9930068150 con 0x7f99300b2f80 2024-04-17T10:50:22.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.716+0000 7f9926ffd640 1 -- 172.21.15.83:0/1202465006 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4158 (secure 0 0 0) 0x7f99280f2c00 con 0x7f99300b2f80 2024-04-17T10:50:22.708 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:50:22.708 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":80,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5942,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3888166846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3888166846},{"type":"v1","addr":"172.21.15.167:6833","nonce":3888166846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":6772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1167913722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1167913722},{"type":"v1","addr":"172.21.15.167:6835","nonce":1167913722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:49:19.962680+0000","modified":"2024-04-17T10:50:14.204149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9192},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9192":{"gid":9192,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":18,"addr":"172.21.15.83:6835/3552333026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3552333026},{"type":"v1","addr":"172.21.15.83:6835","nonce":3552333026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-17T10:50:22.708 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 80 2024-04-17T10:50:22.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.716+0000 7f99385f4640 1 -- 172.21.15.83:0/1202465006 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9914066f60 msgr2=0x7f9914069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:22.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.716+0000 7f99385f4640 1 --2- 172.21.15.83:0/1202465006 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9914066f60 0x7f9914069420 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f992c0099b0 tx=0x7f992c008040 comp rx=0 tx=0).stop 2024-04-17T10:50:22.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.716+0000 7f99385f4640 1 -- 172.21.15.83:0/1202465006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99300b2f80 msgr2=0x7f99300b5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:22.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.716+0000 7f99385f4640 1 --2- 172.21.15.83:0/1202465006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99300b2f80 0x7f99300b5320 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f9928020320 tx=0x7f991c005840 comp rx=0 tx=0).stop 2024-04-17T10:50:22.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.716+0000 7f99385f4640 1 -- 172.21.15.83:0/1202465006 shutdown_connections 2024-04-17T10:50:22.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.716+0000 7f99385f4640 1 --2- 172.21.15.83:0/1202465006 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9914066f60 0x7f9914069420 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:22.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.716+0000 7f99385f4640 1 --2- 172.21.15.83:0/1202465006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99300b2f80 0x7f99300b5320 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:22.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.716+0000 7f99385f4640 1 --2- 172.21.15.83:0/1202465006 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99300a06d0 0x7f99300b2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:22.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.716+0000 7f99385f4640 1 --2- 172.21.15.83:0/1202465006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f991c005220 0x7f99300c65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:22.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.716+0000 7f99385f4640 1 -- 172.21.15.83:0/1202465006 >> 172.21.15.83:0/1202465006 conn(0x7f99300cc910 msgr2=0x7f993009c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:22.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.716+0000 7f99385f4640 1 -- 172.21.15.83:0/1202465006 shutdown_connections 2024-04-17T10:50:22.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.716+0000 7f99385f4640 1 -- 172.21.15.83:0/1202465006 wait complete. 2024-04-17T10:50:22.726 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:50:22.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.852+0000 7f2f3fad1640 1 Processor -- start 2024-04-17T10:50:22.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.852+0000 7f2f3fad1640 1 -- start start 2024-04-17T10:50:22.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.856+0000 7f2f3fad1640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2f38072330 0x7f2f380cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:22.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.856+0000 7f2f3fad1640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f380cfea0 con 0x7f2f380cfa10 2024-04-17T10:50:22.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.856+0000 7f2f3fad1640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f3806e320 con 0x7f2f3815ba30 2024-04-17T10:50:22.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.856+0000 7f2f3fad1640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f38071ea0 con 0x7f2f38072330 2024-04-17T10:50:22.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.856+0000 7f2f3e06e640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2f380cfa10 0x7f2f3806c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39078/0 (socket says 172.21.15.83:39078) 2024-04-17T10:50:22.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.856+0000 7f2f3e06e640 1 -- 172.21.15.83:0/1063236812 learned_addr learned my addr 172.21.15.83:0/1063236812 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:22.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.856+0000 7f2f3d06c640 1 --2- 172.21.15.83:0/1063236812 >> v2:172.21.15.167:3300/0 conn(0x7f2f38072330 0x7f2f380cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:22.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.856+0000 7f2f3c86b640 1 -- 172.21.15.83:0/1063236812 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4226547455 0 0) 0x7f2f380cfea0 con 0x7f2f380cfa10 2024-04-17T10:50:22.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.856+0000 7f2f3c86b640 1 -- 172.21.15.83:0/1063236812 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f24003530 con 0x7f2f380cfa10 2024-04-17T10:50:22.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.856+0000 7f2f3c86b640 1 -- 172.21.15.83:0/1063236812 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3651438728 0 0) 0x7f2f24003530 con 0x7f2f380cfa10 2024-04-17T10:50:22.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.856+0000 7f2f3c86b640 1 -- 172.21.15.83:0/1063236812 >> v2:172.21.15.167:3300/0 conn(0x7f2f38072330 msgr2=0x7f2f380cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:22.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.856+0000 7f2f3c86b640 1 --2- 172.21.15.83:0/1063236812 >> v2:172.21.15.167:3300/0 conn(0x7f2f38072330 0x7f2f380cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:22.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.856+0000 7f2f3c86b640 1 -- 172.21.15.83:0/1063236812 >> v1:172.21.15.167:6789/0 conn(0x7f2f3815ba30 legacy=0x7f2f3815c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:22.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.856+0000 7f2f3c86b640 1 -- 172.21.15.83:0/1063236812 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3806e040 con 0x7f2f380cfa10 2024-04-17T10:50:22.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.856+0000 7f2f3c86b640 1 -- 172.21.15.83:0/1063236812 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f2f30001a00 con 0x7f2f380cfa10 2024-04-17T10:50:22.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.856+0000 7f2f3c86b640 1 -- 172.21.15.83:0/1063236812 >> v1:172.21.15.83:6789/0 conn(0x7f2f380cfa10 legacy=0x7f2f3806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:22.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.856+0000 7f2f3c86b640 1 --2- 172.21.15.83:0/1063236812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f240039c0 0x7f2f24003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:22.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.856+0000 7f2f3c86b640 1 --2- 172.21.15.83:0/1063236812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f24004810 0x7f2f24004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:22.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.856+0000 7f2f3c86b640 1 --2- 172.21.15.83:0/1063236812 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f240051d0 0x7f2f2400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:22.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.856+0000 7f2f3c86b640 1 -- 172.21.15.83:0/1063236812 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f38071ea0 con 0x7f2f240039c0 2024-04-17T10:50:22.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.856+0000 7f2f3c86b640 1 -- 172.21.15.83:0/1063236812 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2400c170 con 0x7f2f24004810 2024-04-17T10:50:22.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.856+0000 7f2f3c86b640 1 -- 172.21.15.83:0/1063236812 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2400c370 con 0x7f2f240051d0 2024-04-17T10:50:22.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.856+0000 7f2f3d86d640 1 --2- 172.21.15.83:0/1063236812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f240039c0 0x7f2f24003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:22.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.856+0000 7f2f3d06c640 1 --2- 172.21.15.83:0/1063236812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f24004810 0x7f2f24004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:22.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.856+0000 7f2f3d86d640 1 -- 172.21.15.83:0/1063236812 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f240051d0 msgr2=0x7f2f2400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:22.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.856+0000 7f2f3d86d640 1 --2- 172.21.15.83:0/1063236812 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f240051d0 0x7f2f2400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:22.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.856+0000 7f2f3d86d640 1 -- 172.21.15.83:0/1063236812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f24004810 msgr2=0x7f2f24004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:22.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.856+0000 7f2f3d86d640 1 --2- 172.21.15.83:0/1063236812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f24004810 0x7f2f24004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:22.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.856+0000 7f2f3d86d640 1 -- 172.21.15.83:0/1063236812 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f24003530 con 0x7f2f240039c0 2024-04-17T10:50:22.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.856+0000 7f2f3e06e640 1 -- 172.21.15.83:0/1063236812 reap_dead start 2024-04-17T10:50:22.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.856+0000 7f2f3d86d640 1 --2- 172.21.15.83:0/1063236812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f240039c0 0x7f2f24003de0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f2f34001780 tx=0x7f2f3401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd5085058d8052f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:22.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f3c86b640 1 -- 172.21.15.83:0/1063236812 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f340013a0 con 0x7f2f240039c0 2024-04-17T10:50:22.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f3c86b640 1 -- 172.21.15.83:0/1063236812 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f340052f0 con 0x7f2f240039c0 2024-04-17T10:50:22.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f3c86b640 1 -- 172.21.15.83:0/1063236812 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f34003840 con 0x7f2f240039c0 2024-04-17T10:50:22.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f3fad1640 1 -- 172.21.15.83:0/1063236812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f240039c0 msgr2=0x7f2f24003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:22.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f3fad1640 1 --2- 172.21.15.83:0/1063236812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f240039c0 0x7f2f24003de0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f2f34001780 tx=0x7f2f3401c420 comp rx=0 tx=0).stop 2024-04-17T10:50:22.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f3fad1640 1 -- 172.21.15.83:0/1063236812 shutdown_connections 2024-04-17T10:50:22.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f3fad1640 1 --2- 172.21.15.83:0/1063236812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f240039c0 0x7f2f24003de0 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:22.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f3fad1640 1 -- 172.21.15.83:0/1063236812 >> 172.21.15.83:0/1063236812 conn(0x7f2f3806abe0 msgr2=0x7f2f3806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:22.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f3fad1640 1 -- 172.21.15.83:0/1063236812 shutdown_connections 2024-04-17T10:50:22.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f3fad1640 1 -- 172.21.15.83:0/1063236812 wait complete. 2024-04-17T10:50:22.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f3fad1640 1 Processor -- start 2024-04-17T10:50:22.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f3fad1640 1 -- start start 2024-04-17T10:50:22.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f3fad1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f240039c0 0x7f2f3815dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:22.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f3fad1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f38155030 0x7f2f380ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:22.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f3fad1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f380b0120 0x7f2f380b0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:22.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f3fad1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3806de20 con 0x7f2f38155030 2024-04-17T10:50:22.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f3fad1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3806dca0 con 0x7f2f380b0120 2024-04-17T10:50:22.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f3fad1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3815cd70 con 0x7f2f240039c0 2024-04-17T10:50:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f3d06c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f38155030 0x7f2f380ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f3d06c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f38155030 0x7f2f380ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40746/0 (socket says 172.21.15.83:40746) 2024-04-17T10:50:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f3d06c640 1 -- 172.21.15.83:0/3460976387 learned_addr learned my addr 172.21.15.83:0/3460976387 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f3e06e640 1 --2- 172.21.15.83:0/3460976387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f380b0120 0x7f2f380b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f3d86d640 1 --2- 172.21.15.83:0/3460976387 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f240039c0 0x7f2f3815dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f3d06c640 1 -- 172.21.15.83:0/3460976387 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f240039c0 msgr2=0x7f2f3815dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f3d06c640 1 --2- 172.21.15.83:0/3460976387 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f240039c0 0x7f2f3815dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f3d06c640 1 -- 172.21.15.83:0/3460976387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f380b0120 msgr2=0x7f2f380b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f3d06c640 1 --2- 172.21.15.83:0/3460976387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f380b0120 0x7f2f380b0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f3d06c640 1 -- 172.21.15.83:0/3460976387 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f381557c0 con 0x7f2f38155030 2024-04-17T10:50:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f3e06e640 1 --2- 172.21.15.83:0/3460976387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f380b0120 0x7f2f380b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f3d06c640 1 --2- 172.21.15.83:0/3460976387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f38155030 0x7f2f380ce940 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f2f28010900 tx=0x7f2f28010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f2e7fc640 1 -- 172.21.15.83:0/3460976387 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2800f040 con 0x7f2f38155030 2024-04-17T10:50:22.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f2e7fc640 1 -- 172.21.15.83:0/3460976387 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2800ec20 con 0x7f2f38155030 2024-04-17T10:50:22.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f2e7fc640 1 -- 172.21.15.83:0/3460976387 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2801c400 con 0x7f2f38155030 2024-04-17T10:50:22.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f3fad1640 1 -- 172.21.15.83:0/3460976387 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f38155aa0 con 0x7f2f38155030 2024-04-17T10:50:22.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.860+0000 7f2f3fad1640 1 -- 172.21.15.83:0/3460976387 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f38156060 con 0x7f2f38155030 2024-04-17T10:50:22.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.864+0000 7f2f3fad1640 1 -- 172.21.15.83:0/3460976387 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f38074f90 con 0x7f2f38155030 2024-04-17T10:50:22.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.864+0000 7f2f2e7fc640 1 -- 172.21.15.83:0/3460976387 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2f2801c5a0 con 0x7f2f38155030 2024-04-17T10:50:22.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.864+0000 7f2f2e7fc640 1 --2- 172.21.15.83:0/3460976387 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2f1c066f90 0x7f2f1c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:22.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.864+0000 7f2f2e7fc640 1 -- 172.21.15.83:0/3460976387 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f2f280857d0 con 0x7f2f38155030 2024-04-17T10:50:22.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.864+0000 7f2f3d86d640 1 --2- 172.21.15.83:0/3460976387 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2f1c066f90 0x7f2f1c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:22.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.864+0000 7f2f3d86d640 1 --2- 172.21.15.83:0/3460976387 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2f1c066f90 0x7f2f1c069450 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f2f34001750 tx=0x7f2f340011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:22.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:22.868+0000 7f2f2e7fc640 1 -- 172.21.15.83:0/3460976387 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2f28018030 con 0x7f2f38155030 2024-04-17T10:50:23.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.012+0000 7f2f3fad1640 1 -- 172.21.15.83:0/3460976387 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f38158e10 con 0x7f2f38155030 2024-04-17T10:50:23.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.016+0000 7f2f2e7fc640 1 -- 172.21.15.83:0/3460976387 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+16777 (secure 0 0 0) 0x7f2f280566b0 con 0x7f2f38155030 2024-04-17T10:50:23.009 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:50:23.009 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":130,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:50:12.167718+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:49:18.247824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-17T10:49:18.951409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.5","mappings":[{"from":2,"to":0}]},{"pgid":"19.25","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/1559222491":"2024-04-18T10:50:12.167626+0000","172.21.15.83:6836/1559222491":"2024-04-18T10:50:12.167626+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:50:23.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.016+0000 7f2f3fad1640 1 -- 172.21.15.83:0/3460976387 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2f1c066f90 msgr2=0x7f2f1c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:23.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.016+0000 7f2f3fad1640 1 --2- 172.21.15.83:0/3460976387 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2f1c066f90 0x7f2f1c069450 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f2f34001750 tx=0x7f2f340011a0 comp rx=0 tx=0).stop 2024-04-17T10:50:23.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.016+0000 7f2f3fad1640 1 -- 172.21.15.83:0/3460976387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f38155030 msgr2=0x7f2f380ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:23.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.016+0000 7f2f3fad1640 1 --2- 172.21.15.83:0/3460976387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f38155030 0x7f2f380ce940 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f2f28010900 tx=0x7f2f28010dc0 comp rx=0 tx=0).stop 2024-04-17T10:50:23.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.016+0000 7f2f3fad1640 1 -- 172.21.15.83:0/3460976387 shutdown_connections 2024-04-17T10:50:23.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.016+0000 7f2f3fad1640 1 --2- 172.21.15.83:0/3460976387 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2f1c066f90 0x7f2f1c069450 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:23.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.016+0000 7f2f3fad1640 1 --2- 172.21.15.83:0/3460976387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f380b0120 0x7f2f380b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:23.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.016+0000 7f2f3fad1640 1 --2- 172.21.15.83:0/3460976387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f38155030 0x7f2f380ce940 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:23.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.016+0000 7f2f3fad1640 1 --2- 172.21.15.83:0/3460976387 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f240039c0 0x7f2f3815dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:23.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.016+0000 7f2f3fad1640 1 -- 172.21.15.83:0/3460976387 >> 172.21.15.83:0/3460976387 conn(0x7f2f3806abe0 msgr2=0x7f2f38152510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:23.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.016+0000 7f2f3fad1640 1 -- 172.21.15.83:0/3460976387 shutdown_connections 2024-04-17T10:50:23.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.016+0000 7f2f3fad1640 1 -- 172.21.15.83:0/3460976387 wait complete. 2024-04-17T10:50:23.027 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-17T10:50:23.027 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:50:23.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.152+0000 7f1a65971640 1 Processor -- start 2024-04-17T10:50:23.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.152+0000 7f1a65971640 1 -- start start 2024-04-17T10:50:23.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.156+0000 7f1a65971640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1a60072330 0x7f1a600cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:23.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.156+0000 7f1a65971640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1a600cfa10 0x7f1a6006c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:23.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.156+0000 7f1a65971640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a600cfea0 con 0x7f1a6015ba30 2024-04-17T10:50:23.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.156+0000 7f1a65971640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a60071ea0 con 0x7f1a60072330 2024-04-17T10:50:23.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.156+0000 7f1a65971640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a6006e320 con 0x7f1a600cfa10 2024-04-17T10:50:23.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.156+0000 7f1a567fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1a60072330 0x7f1a600cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:23.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.156+0000 7f1a567fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1a60072330 0x7f1a600cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40748/0 (socket says 172.21.15.83:40748) 2024-04-17T10:50:23.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.156+0000 7f1a567fc640 1 -- 172.21.15.83:0/168056154 learned_addr learned my addr 172.21.15.83:0/168056154 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:23.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.156+0000 7f1a5f7fe640 1 --2- 172.21.15.83:0/168056154 >> v2:172.21.15.167:3300/0 conn(0x7f1a600cfa10 0x7f1a6006c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:23.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.156+0000 7f1a567fc640 1 -- 172.21.15.83:0/168056154 >> v2:172.21.15.167:3300/0 conn(0x7f1a600cfa10 msgr2=0x7f1a6006c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:23.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.156+0000 7f1a567fc640 1 --2- 172.21.15.83:0/168056154 >> v2:172.21.15.167:3300/0 conn(0x7f1a600cfa10 0x7f1a6006c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:23.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.156+0000 7f1a567fc640 1 -- 172.21.15.83:0/168056154 >> v1:172.21.15.167:6789/0 conn(0x7f1a6015ba30 legacy=0x7f1a6015c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:23.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.156+0000 7f1a567fc640 1 -- 172.21.15.83:0/168056154 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a6006e040 con 0x7f1a60072330 2024-04-17T10:50:23.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.156+0000 7f1a567fc640 1 --2- 172.21.15.83:0/168056154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a60072330 0x7f1a600cf4d0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f1a600676a0 tx=0x7f1a4801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2832106cbce0f164 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:23.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a5e7fc640 1 -- 172.21.15.83:0/168056154 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a480b1070 con 0x7f1a60072330 2024-04-17T10:50:23.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a5e7fc640 1 -- 172.21.15.83:0/168056154 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a480ac600 con 0x7f1a60072330 2024-04-17T10:50:23.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a5e7fc640 1 -- 172.21.15.83:0/168056154 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a480ac9d0 con 0x7f1a60072330 2024-04-17T10:50:23.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a65971640 1 -- 172.21.15.83:0/168056154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a60072330 msgr2=0x7f1a600cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:23.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a65971640 1 --2- 172.21.15.83:0/168056154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a60072330 0x7f1a600cf4d0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f1a600676a0 tx=0x7f1a4801c490 comp rx=0 tx=0).stop 2024-04-17T10:50:23.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a65971640 1 -- 172.21.15.83:0/168056154 shutdown_connections 2024-04-17T10:50:23.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a65971640 1 --2- 172.21.15.83:0/168056154 >> v2:172.21.15.167:3300/0 conn(0x7f1a600cfa10 0x7f1a6006c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:23.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a65971640 1 --2- 172.21.15.83:0/168056154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a60072330 0x7f1a600cf4d0 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:23.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a65971640 1 -- 172.21.15.83:0/168056154 >> 172.21.15.83:0/168056154 conn(0x7f1a6006abe0 msgr2=0x7f1a6006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:23.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a65971640 1 -- 172.21.15.83:0/168056154 shutdown_connections 2024-04-17T10:50:23.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a65971640 1 -- 172.21.15.83:0/168056154 wait complete. 2024-04-17T10:50:23.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a65971640 1 Processor -- start 2024-04-17T10:50:23.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a65971640 1 -- start start 2024-04-17T10:50:23.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a65971640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a60072330 0x7f1a6015ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:23.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a65971640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a600cfa10 0x7f1a6013dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:23.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a65971640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a6015ba30 0x7f1a6013e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:23.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a65971640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6006de20 con 0x7f1a60072330 2024-04-17T10:50:23.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a65971640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6006dca0 con 0x7f1a6015ba30 2024-04-17T10:50:23.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a65971640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a60168520 con 0x7f1a600cfa10 2024-04-17T10:50:23.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a5effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a60072330 0x7f1a6015ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:23.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a5effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a60072330 0x7f1a6015ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40754/0 (socket says 172.21.15.83:40754) 2024-04-17T10:50:23.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a5effd640 1 -- 172.21.15.83:0/2366073731 learned_addr learned my addr 172.21.15.83:0/2366073731 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:23.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a5f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a6015ba30 0x7f1a6013e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:23.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a567fc640 1 --2- 172.21.15.83:0/2366073731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a600cfa10 0x7f1a6013dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:23.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a5effd640 1 -- 172.21.15.83:0/2366073731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a600cfa10 msgr2=0x7f1a6013dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:23.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a5effd640 1 --2- 172.21.15.83:0/2366073731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a600cfa10 0x7f1a6013dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:23.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a5effd640 1 -- 172.21.15.83:0/2366073731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a6015ba30 msgr2=0x7f1a6013e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:23.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a5effd640 1 --2- 172.21.15.83:0/2366073731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a6015ba30 0x7f1a6013e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:23.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a5effd640 1 -- 172.21.15.83:0/2366073731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a60151d50 con 0x7f1a60072330 2024-04-17T10:50:23.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a5effd640 1 --2- 172.21.15.83:0/2366073731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a60072330 0x7f1a6015ce10 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000c450 tx=0x7f1a5000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:23.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a5cff9640 1 -- 172.21.15.83:0/2366073731 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a50016020 con 0x7f1a60072330 2024-04-17T10:50:23.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a5cff9640 1 -- 172.21.15.83:0/2366073731 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a500052d0 con 0x7f1a60072330 2024-04-17T10:50:23.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a65971640 1 -- 172.21.15.83:0/2366073731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a60156dd0 con 0x7f1a60072330 2024-04-17T10:50:23.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a5cff9640 1 -- 172.21.15.83:0/2366073731 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a50007260 con 0x7f1a60072330 2024-04-17T10:50:23.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.160+0000 7f1a65971640 1 -- 172.21.15.83:0/2366073731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a60157390 con 0x7f1a60072330 2024-04-17T10:50:23.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.164+0000 7f1a65971640 1 -- 172.21.15.83:0/2366073731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a60074f90 con 0x7f1a60072330 2024-04-17T10:50:23.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.164+0000 7f1a5cff9640 1 -- 172.21.15.83:0/2366073731 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1a50007400 con 0x7f1a60072330 2024-04-17T10:50:23.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.164+0000 7f1a5cff9640 1 --2- 172.21.15.83:0/2366073731 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1a40066f60 0x7f1a40069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:23.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.164+0000 7f1a5cff9640 1 -- 172.21.15.83:0/2366073731 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f1a50084870 con 0x7f1a60072330 2024-04-17T10:50:23.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.164+0000 7f1a567fc640 1 --2- 172.21.15.83:0/2366073731 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1a40066f60 0x7f1a40069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:23.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.164+0000 7f1a567fc640 1 --2- 172.21.15.83:0/2366073731 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1a40066f60 0x7f1a40069420 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1a480ae000 tx=0x7f1a480a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:23.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.168+0000 7f1a5cff9640 1 -- 172.21.15.83:0/2366073731 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1a50020020 con 0x7f1a60072330 2024-04-17T10:50:23.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.344+0000 7f1a65971640 1 -- 172.21.15.83:0/2366073731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a600cf480 con 0x7f1a60072330 2024-04-17T10:50:23.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.348+0000 7f1a5cff9640 1 -- 172.21.15.83:0/2366073731 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4158 (secure 0 0 0) 0x7f1a50005480 con 0x7f1a60072330 2024-04-17T10:50:23.339 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:50:23.340 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":80,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5942,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3888166846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3888166846},{"type":"v1","addr":"172.21.15.167:6833","nonce":3888166846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":6772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1167913722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1167913722},{"type":"v1","addr":"172.21.15.167:6835","nonce":1167913722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:49:19.962680+0000","modified":"2024-04-17T10:50:14.204149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9192},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9192":{"gid":9192,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":18,"addr":"172.21.15.83:6835/3552333026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3552333026},{"type":"v1","addr":"172.21.15.83:6835","nonce":3552333026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-17T10:50:23.340 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 80 2024-04-17T10:50:23.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.348+0000 7f1a65971640 1 -- 172.21.15.83:0/2366073731 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1a40066f60 msgr2=0x7f1a40069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:23.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.348+0000 7f1a65971640 1 --2- 172.21.15.83:0/2366073731 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1a40066f60 0x7f1a40069420 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1a480ae000 tx=0x7f1a480a7000 comp rx=0 tx=0).stop 2024-04-17T10:50:23.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.348+0000 7f1a65971640 1 -- 172.21.15.83:0/2366073731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a60072330 msgr2=0x7f1a6015ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:23.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.348+0000 7f1a65971640 1 --2- 172.21.15.83:0/2366073731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a60072330 0x7f1a6015ce10 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000c450 tx=0x7f1a5000c910 comp rx=0 tx=0).stop 2024-04-17T10:50:23.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.348+0000 7f1a65971640 1 -- 172.21.15.83:0/2366073731 shutdown_connections 2024-04-17T10:50:23.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.348+0000 7f1a65971640 1 --2- 172.21.15.83:0/2366073731 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1a40066f60 0x7f1a40069420 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:23.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.348+0000 7f1a65971640 1 --2- 172.21.15.83:0/2366073731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a6015ba30 0x7f1a6013e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:23.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.348+0000 7f1a65971640 1 --2- 172.21.15.83:0/2366073731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a600cfa10 0x7f1a6013dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:23.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.348+0000 7f1a65971640 1 --2- 172.21.15.83:0/2366073731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a60072330 0x7f1a6015ce10 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:23.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.348+0000 7f1a65971640 1 -- 172.21.15.83:0/2366073731 >> 172.21.15.83:0/2366073731 conn(0x7f1a6006abe0 msgr2=0x7f1a6015a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:23.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.348+0000 7f1a65971640 1 -- 172.21.15.83:0/2366073731 shutdown_connections 2024-04-17T10:50:23.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.348+0000 7f1a65971640 1 -- 172.21.15.83:0/2366073731 wait complete. 2024-04-17T10:50:23.357 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:50:23.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.484+0000 7fad7d731640 1 Processor -- start 2024-04-17T10:50:23.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.484+0000 7fad7d731640 1 -- start start 2024-04-17T10:50:23.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.484+0000 7fad7d731640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fad78072330 0x7fad780cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:23.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.484+0000 7fad7d731640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad780cfea0 con 0x7fad7815ba30 2024-04-17T10:50:23.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.484+0000 7fad7d731640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad7806e320 con 0x7fad780cfa10 2024-04-17T10:50:23.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.484+0000 7fad7d731640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad78071ea0 con 0x7fad78072330 2024-04-17T10:50:23.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.484+0000 7fad76ffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fad7815ba30 0x7fad7815c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39086/0 (socket says 172.21.15.83:39086) 2024-04-17T10:50:23.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.484+0000 7fad76ffd640 1 -- 172.21.15.83:0/4087933718 learned_addr learned my addr 172.21.15.83:0/4087933718 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:23.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.484+0000 7fad767fc640 1 --2- 172.21.15.83:0/4087933718 >> v2:172.21.15.83:3300/0 conn(0x7fad78072330 0x7fad780cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:23.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.484+0000 7fad75ffb640 1 -- 172.21.15.83:0/4087933718 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 463969237 0 0) 0x7fad780cfea0 con 0x7fad7815ba30 2024-04-17T10:50:23.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.484+0000 7fad75ffb640 1 -- 172.21.15.83:0/4087933718 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad60003530 con 0x7fad7815ba30 2024-04-17T10:50:23.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.484+0000 7fad767fc640 1 -- 172.21.15.83:0/4087933718 >> v1:172.21.15.167:6789/0 conn(0x7fad780cfa10 legacy=0x7fad7806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:23.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.484+0000 7fad767fc640 1 -- 172.21.15.83:0/4087933718 >> v1:172.21.15.83:6789/0 conn(0x7fad7815ba30 legacy=0x7fad7815c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:23.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.484+0000 7fad767fc640 1 -- 172.21.15.83:0/4087933718 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7806e040 con 0x7fad78072330 2024-04-17T10:50:23.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.484+0000 7fad75ffb640 1 -- 172.21.15.83:0/4087933718 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 688571789 0 0) 0x7fad60003530 con 0x7fad7815ba30 2024-04-17T10:50:23.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.488+0000 7fad767fc640 1 --2- 172.21.15.83:0/4087933718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad78072330 0x7fad780cf4d0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0097c0 tx=0x7fad6c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4ad6a9037d5735 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:23.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.488+0000 7fad75ffb640 1 -- 172.21.15.83:0/4087933718 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c0b0070 con 0x7fad78072330 2024-04-17T10:50:23.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.488+0000 7fad75ffb640 1 -- 172.21.15.83:0/4087933718 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6c0ab600 con 0x7fad78072330 2024-04-17T10:50:23.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.488+0000 7fad75ffb640 1 -- 172.21.15.83:0/4087933718 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c0ab9d0 con 0x7fad78072330 2024-04-17T10:50:23.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.488+0000 7fad7d731640 1 -- 172.21.15.83:0/4087933718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad78072330 msgr2=0x7fad780cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:23.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.488+0000 7fad7d731640 1 --2- 172.21.15.83:0/4087933718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad78072330 0x7fad780cf4d0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0097c0 tx=0x7fad6c01c410 comp rx=0 tx=0).stop 2024-04-17T10:50:23.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.488+0000 7fad7d731640 1 -- 172.21.15.83:0/4087933718 shutdown_connections 2024-04-17T10:50:23.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.488+0000 7fad7d731640 1 --2- 172.21.15.83:0/4087933718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad78072330 0x7fad780cf4d0 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:23.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.488+0000 7fad7d731640 1 -- 172.21.15.83:0/4087933718 >> 172.21.15.83:0/4087933718 conn(0x7fad7806abe0 msgr2=0x7fad7806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:23.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.488+0000 7fad7d731640 1 -- 172.21.15.83:0/4087933718 shutdown_connections 2024-04-17T10:50:23.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.488+0000 7fad7d731640 1 -- 172.21.15.83:0/4087933718 wait complete. 2024-04-17T10:50:23.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.488+0000 7fad7d731640 1 Processor -- start 2024-04-17T10:50:23.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.488+0000 7fad7d731640 1 -- start start 2024-04-17T10:50:23.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.488+0000 7fad7d731640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad78072330 0x7fad78150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:23.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.488+0000 7fad7d731640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad780cfa10 0x7fad780b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:23.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.488+0000 7fad7d731640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad7815ba30 0x7fad780b2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:23.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.488+0000 7fad7d731640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7806de20 con 0x7fad7815ba30 2024-04-17T10:50:23.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.488+0000 7fad7d731640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7806dca0 con 0x7fad780cfa10 2024-04-17T10:50:23.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.488+0000 7fad7d731640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad78169530 con 0x7fad78072330 2024-04-17T10:50:23.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.488+0000 7fad777fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad7815ba30 0x7fad780b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:23.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.488+0000 7fad777fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad7815ba30 0x7fad780b2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40768/0 (socket says 172.21.15.83:40768) 2024-04-17T10:50:23.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.488+0000 7fad777fe640 1 -- 172.21.15.83:0/1683932213 learned_addr learned my addr 172.21.15.83:0/1683932213 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:23.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.488+0000 7fad767fc640 1 --2- 172.21.15.83:0/1683932213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad780cfa10 0x7fad780b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:23.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.488+0000 7fad76ffd640 1 --2- 172.21.15.83:0/1683932213 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad78072330 0x7fad78150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:23.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.488+0000 7fad777fe640 1 -- 172.21.15.83:0/1683932213 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad78072330 msgr2=0x7fad78150350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:23.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.488+0000 7fad777fe640 1 --2- 172.21.15.83:0/1683932213 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad78072330 0x7fad78150350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:23.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.488+0000 7fad777fe640 1 -- 172.21.15.83:0/1683932213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad780cfa10 msgr2=0x7fad780b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:23.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.488+0000 7fad777fe640 1 --2- 172.21.15.83:0/1683932213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad780cfa10 0x7fad780b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:23.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.488+0000 7fad777fe640 1 -- 172.21.15.83:0/1683932213 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad780b2fc0 con 0x7fad7815ba30 2024-04-17T10:50:23.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.492+0000 7fad777fe640 1 --2- 172.21.15.83:0/1683932213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad7815ba30 0x7fad780b2770 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fad7000c7a0 tx=0x7fad7000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:23.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.492+0000 7fad5f7fe640 1 -- 172.21.15.83:0/1683932213 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad70003b80 con 0x7fad7815ba30 2024-04-17T10:50:23.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.492+0000 7fad5f7fe640 1 -- 172.21.15.83:0/1683932213 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad70003d20 con 0x7fad7815ba30 2024-04-17T10:50:23.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.492+0000 7fad7d731640 1 -- 172.21.15.83:0/1683932213 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad780b32a0 con 0x7fad7815ba30 2024-04-17T10:50:23.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.492+0000 7fad5f7fe640 1 -- 172.21.15.83:0/1683932213 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7000aa90 con 0x7fad7815ba30 2024-04-17T10:50:23.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.492+0000 7fad7d731640 1 -- 172.21.15.83:0/1683932213 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad780b3860 con 0x7fad7815ba30 2024-04-17T10:50:23.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.492+0000 7fad7d731640 1 -- 172.21.15.83:0/1683932213 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad78074f90 con 0x7fad7815ba30 2024-04-17T10:50:23.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.492+0000 7fad5f7fe640 1 -- 172.21.15.83:0/1683932213 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fad7000ac30 con 0x7fad7815ba30 2024-04-17T10:50:23.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.492+0000 7fad5f7fe640 1 --2- 172.21.15.83:0/1683932213 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fad64066f60 0x7fad64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:23.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.492+0000 7fad5f7fe640 1 -- 172.21.15.83:0/1683932213 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fad70085010 con 0x7fad7815ba30 2024-04-17T10:50:23.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.492+0000 7fad76ffd640 1 --2- 172.21.15.83:0/1683932213 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fad64066f60 0x7fad64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:23.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.492+0000 7fad76ffd640 1 --2- 172.21.15.83:0/1683932213 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fad64066f60 0x7fad64069420 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fad68007fd0 tx=0x7fad68009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:23.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.496+0000 7fad5f7fe640 1 -- 172.21.15.83:0/1683932213 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fad70055f90 con 0x7fad7815ba30 2024-04-17T10:50:23.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.680+0000 7fad7d731640 1 -- 172.21.15.83:0/1683932213 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad780b45a0 con 0x7fad7815ba30 2024-04-17T10:50:23.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.680+0000 7fad5f7fe640 1 -- 172.21.15.83:0/1683932213 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4158 (secure 0 0 0) 0x7fad70055890 con 0x7fad7815ba30 2024-04-17T10:50:23.672 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:50:23.673 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":80,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5942,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3888166846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3888166846},{"type":"v1","addr":"172.21.15.167:6833","nonce":3888166846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":6772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1167913722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1167913722},{"type":"v1","addr":"172.21.15.167:6835","nonce":1167913722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:49:19.962680+0000","modified":"2024-04-17T10:50:14.204149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9192},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9192":{"gid":9192,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":18,"addr":"172.21.15.83:6835/3552333026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3552333026},{"type":"v1","addr":"172.21.15.83:6835","nonce":3552333026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-17T10:50:23.673 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 80 2024-04-17T10:50:23.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.684+0000 7fad7d731640 1 -- 172.21.15.83:0/1683932213 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fad64066f60 msgr2=0x7fad64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:23.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.684+0000 7fad7d731640 1 --2- 172.21.15.83:0/1683932213 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fad64066f60 0x7fad64069420 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fad68007fd0 tx=0x7fad68009040 comp rx=0 tx=0).stop 2024-04-17T10:50:23.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.684+0000 7fad7d731640 1 -- 172.21.15.83:0/1683932213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad7815ba30 msgr2=0x7fad780b2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:23.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.684+0000 7fad7d731640 1 --2- 172.21.15.83:0/1683932213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad7815ba30 0x7fad780b2770 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fad7000c7a0 tx=0x7fad7000cc60 comp rx=0 tx=0).stop 2024-04-17T10:50:23.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.684+0000 7fad7d731640 1 -- 172.21.15.83:0/1683932213 shutdown_connections 2024-04-17T10:50:23.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.684+0000 7fad7d731640 1 --2- 172.21.15.83:0/1683932213 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fad64066f60 0x7fad64069420 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:23.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.684+0000 7fad7d731640 1 --2- 172.21.15.83:0/1683932213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad7815ba30 0x7fad780b2770 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:23.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.684+0000 7fad7d731640 1 --2- 172.21.15.83:0/1683932213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad780cfa10 0x7fad780b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:23.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.684+0000 7fad7d731640 1 --2- 172.21.15.83:0/1683932213 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad78072330 0x7fad78150350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:23.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.684+0000 7fad7d731640 1 -- 172.21.15.83:0/1683932213 >> 172.21.15.83:0/1683932213 conn(0x7fad7806abe0 msgr2=0x7fad78159680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:23.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.684+0000 7fad7d731640 1 -- 172.21.15.83:0/1683932213 shutdown_connections 2024-04-17T10:50:23.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.684+0000 7fad7d731640 1 -- 172.21.15.83:0/1683932213 wait complete. 2024-04-17T10:50:23.692 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:50:23.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.808+0000 7f7cb2a18640 1 Processor -- start 2024-04-17T10:50:23.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.808+0000 7f7cb2a18640 1 -- start start 2024-04-17T10:50:23.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.808+0000 7f7cb2a18640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7cac072330 0x7f7cac0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.808+0000 7f7cb2a18640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7cac0cfa10 0x7f7cac06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.808+0000 7f7cb2a18640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cac0cfea0 con 0x7f7cac15ba30 2024-04-17T10:50:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.808+0000 7f7cb2a18640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cac071ea0 con 0x7f7cac072330 2024-04-17T10:50:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.808+0000 7f7cb2a18640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac06e320 con 0x7f7cac0cfa10 2024-04-17T10:50:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.808+0000 7f7cab7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7cac072330 0x7f7cac0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.808+0000 7f7cab7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7cac072330 0x7f7cac0cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40776/0 (socket says 172.21.15.83:40776) 2024-04-17T10:50:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.808+0000 7f7cab7fe640 1 -- 172.21.15.83:0/2278943092 learned_addr learned my addr 172.21.15.83:0/2278943092 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.808+0000 7f7cab7fe640 1 -- 172.21.15.83:0/2278943092 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7cac0cfa10 msgr2=0x7f7cac06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.808+0000 7f7cab7fe640 1 --2- 172.21.15.83:0/2278943092 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7cac0cfa10 0x7f7cac06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.808+0000 7f7cab7fe640 1 -- 172.21.15.83:0/2278943092 >> v1:172.21.15.167:6789/0 conn(0x7f7cac15ba30 legacy=0x7f7cac15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.808+0000 7f7cab7fe640 1 -- 172.21.15.83:0/2278943092 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cac06e040 con 0x7f7cac072330 2024-04-17T10:50:23.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7cab7fe640 1 --2- 172.21.15.83:0/2278943092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cac072330 0x7f7cac0cf4d0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f7cac072020 tx=0x7f7c9401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6793991c7832291 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:23.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7caaffd640 1 -- 172.21.15.83:0/2278943092 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c940b0070 con 0x7f7cac072330 2024-04-17T10:50:23.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7caaffd640 1 -- 172.21.15.83:0/2278943092 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c940ab600 con 0x7f7cac072330 2024-04-17T10:50:23.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7caaffd640 1 -- 172.21.15.83:0/2278943092 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c940ab9d0 con 0x7f7cac072330 2024-04-17T10:50:23.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7cb2a18640 1 -- 172.21.15.83:0/2278943092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cac072330 msgr2=0x7f7cac0cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:23.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7cb2a18640 1 --2- 172.21.15.83:0/2278943092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cac072330 0x7f7cac0cf4d0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f7cac072020 tx=0x7f7c9401c470 comp rx=0 tx=0).stop 2024-04-17T10:50:23.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7cb2a18640 1 -- 172.21.15.83:0/2278943092 shutdown_connections 2024-04-17T10:50:23.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7cb2a18640 1 --2- 172.21.15.83:0/2278943092 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7cac0cfa10 0x7f7cac06c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:23.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7cb2a18640 1 --2- 172.21.15.83:0/2278943092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cac072330 0x7f7cac0cf4d0 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:23.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7cb2a18640 1 -- 172.21.15.83:0/2278943092 >> 172.21.15.83:0/2278943092 conn(0x7f7cac06abe0 msgr2=0x7f7cac06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:23.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7cb2a18640 1 -- 172.21.15.83:0/2278943092 shutdown_connections 2024-04-17T10:50:23.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7cb2a18640 1 -- 172.21.15.83:0/2278943092 wait complete. 2024-04-17T10:50:23.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7cb2a18640 1 Processor -- start 2024-04-17T10:50:23.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7cb2a18640 1 -- start start 2024-04-17T10:50:23.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7cb2a18640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7cac072330 0x7f7cac1534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:23.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7cb2a18640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cac0cfa10 0x7f7cac158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:23.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7cb2a18640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cac15ba30 0x7f7cac140280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:23.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7cb2a18640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac06de20 con 0x7f7cac0cfa10 2024-04-17T10:50:23.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7cb2a18640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac06dca0 con 0x7f7cac15ba30 2024-04-17T10:50:23.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7cb2a18640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac168520 con 0x7f7cac072330 2024-04-17T10:50:23.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7cab7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cac0cfa10 0x7f7cac158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:23.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7cab7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cac0cfa10 0x7f7cac158d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40786/0 (socket says 172.21.15.83:40786) 2024-04-17T10:50:23.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7cab7fe640 1 -- 172.21.15.83:0/4210625773 learned_addr learned my addr 172.21.15.83:0/4210625773 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:23.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7cab7fe640 1 -- 172.21.15.83:0/4210625773 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7cac072330 msgr2=0x7f7cac1534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:23.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7cab7fe640 1 --2- 172.21.15.83:0/4210625773 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7cac072330 0x7f7cac1534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:23.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7cab7fe640 1 -- 172.21.15.83:0/4210625773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cac15ba30 msgr2=0x7f7cac140280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:23.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7cab7fe640 1 --2- 172.21.15.83:0/4210625773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cac15ba30 0x7f7cac140280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:23.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7cab7fe640 1 -- 172.21.15.83:0/4210625773 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cac1521c0 con 0x7f7cac0cfa10 2024-04-17T10:50:23.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7cab7fe640 1 --2- 172.21.15.83:0/4210625773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cac0cfa10 0x7f7cac158d10 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f7c9401cd10 tx=0x7f7c94019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:23.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7ca8ff9640 1 -- 172.21.15.83:0/4210625773 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c940bb070 con 0x7f7cac0cfa10 2024-04-17T10:50:23.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7ca8ff9640 1 -- 172.21.15.83:0/4210625773 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c940b6640 con 0x7f7cac0cfa10 2024-04-17T10:50:23.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7cb2a18640 1 -- 172.21.15.83:0/4210625773 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cac153bb0 con 0x7f7cac0cfa10 2024-04-17T10:50:23.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7ca8ff9640 1 -- 172.21.15.83:0/4210625773 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c940b0040 con 0x7f7cac0cfa10 2024-04-17T10:50:23.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.812+0000 7f7cb2a18640 1 -- 172.21.15.83:0/4210625773 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cac152460 con 0x7f7cac0cfa10 2024-04-17T10:50:23.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.816+0000 7f7cb2a18640 1 -- 172.21.15.83:0/4210625773 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cac074f90 con 0x7f7cac0cfa10 2024-04-17T10:50:23.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.816+0000 7f7ca8ff9640 1 -- 172.21.15.83:0/4210625773 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7c940b69f0 con 0x7f7cac0cfa10 2024-04-17T10:50:23.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.816+0000 7f7ca8ff9640 1 --2- 172.21.15.83:0/4210625773 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7c8c066f60 0x7f7c8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:23.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.816+0000 7f7ca8ff9640 1 -- 172.21.15.83:0/4210625773 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f7c9411ea00 con 0x7f7cac0cfa10 2024-04-17T10:50:23.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.816+0000 7f7cabfff640 1 --2- 172.21.15.83:0/4210625773 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7c8c066f60 0x7f7c8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:23.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.816+0000 7f7cabfff640 1 --2- 172.21.15.83:0/4210625773 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7c8c066f60 0x7f7c8c069420 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c007fd0 tx=0x7f7c9c007d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:23.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.820+0000 7f7ca8ff9640 1 -- 172.21.15.83:0/4210625773 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7c940160f0 con 0x7f7cac0cfa10 2024-04-17T10:50:23.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.976+0000 7f7cb2a18640 1 -- 172.21.15.83:0/4210625773 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7cac074480 con 0x7f7cac0cfa10 2024-04-17T10:50:23.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.980+0000 7f7ca8ff9640 1 -- 172.21.15.83:0/4210625773 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+16777 (secure 0 0 0) 0x7f7c940efa10 con 0x7f7cac0cfa10 2024-04-17T10:50:23.971 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:50:23.972 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":130,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:50:12.167718+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:49:18.247824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-17T10:49:18.951409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.5","mappings":[{"from":2,"to":0}]},{"pgid":"19.25","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/1559222491":"2024-04-18T10:50:12.167626+0000","172.21.15.83:6836/1559222491":"2024-04-18T10:50:12.167626+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:50:23.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.980+0000 7f7cb2a18640 1 -- 172.21.15.83:0/4210625773 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7c8c066f60 msgr2=0x7f7c8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:23.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.980+0000 7f7cb2a18640 1 --2- 172.21.15.83:0/4210625773 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7c8c066f60 0x7f7c8c069420 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c007fd0 tx=0x7f7c9c007d40 comp rx=0 tx=0).stop 2024-04-17T10:50:23.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.980+0000 7f7cb2a18640 1 -- 172.21.15.83:0/4210625773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cac0cfa10 msgr2=0x7f7cac158d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:23.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.980+0000 7f7cb2a18640 1 --2- 172.21.15.83:0/4210625773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cac0cfa10 0x7f7cac158d10 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f7c9401cd10 tx=0x7f7c94019210 comp rx=0 tx=0).stop 2024-04-17T10:50:23.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.980+0000 7f7cb2a18640 1 -- 172.21.15.83:0/4210625773 shutdown_connections 2024-04-17T10:50:23.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.980+0000 7f7cb2a18640 1 --2- 172.21.15.83:0/4210625773 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7c8c066f60 0x7f7c8c069420 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:23.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.980+0000 7f7cb2a18640 1 --2- 172.21.15.83:0/4210625773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cac15ba30 0x7f7cac140280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:23.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.980+0000 7f7cb2a18640 1 --2- 172.21.15.83:0/4210625773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cac0cfa10 0x7f7cac158d10 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:23.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.980+0000 7f7cb2a18640 1 --2- 172.21.15.83:0/4210625773 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7cac072330 0x7f7cac1534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:23.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.980+0000 7f7cb2a18640 1 -- 172.21.15.83:0/4210625773 >> 172.21.15.83:0/4210625773 conn(0x7f7cac06abe0 msgr2=0x7f7cac159c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:23.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.980+0000 7f7cb2a18640 1 -- 172.21.15.83:0/4210625773 shutdown_connections 2024-04-17T10:50:23.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:23.980+0000 7f7cb2a18640 1 -- 172.21.15.83:0/4210625773 wait complete. 2024-04-17T10:50:23.989 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-17T10:50:24.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.108+0000 7f01d143f640 1 Processor -- start 2024-04-17T10:50:24.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.108+0000 7f01d143f640 1 -- start start 2024-04-17T10:50:24.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.108+0000 7f01d143f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01cc15bbc0 0x7f01cc0d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:24.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.108+0000 7f01d143f640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f01cc0b07b0 0x7f01cc0cfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:24.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.108+0000 7f01d143f640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01cc0ccdb0 con 0x7f01cc0d0020 2024-04-17T10:50:24.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.108+0000 7f01d143f640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01cc0d0490 con 0x7f01cc0b07b0 2024-04-17T10:50:24.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.108+0000 7f01d143f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc0cf420 con 0x7f01cc15bbc0 2024-04-17T10:50:24.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.108+0000 7f01caffd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01cc15bbc0 0x7f01cc0d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:24.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.108+0000 7f01ca7fc640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f01cc0b07b0 0x7f01cc0cfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:24.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.108+0000 7f01caffd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01cc15bbc0 0x7f01cc0d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:43622/0 (socket says 172.21.15.83:43622) 2024-04-17T10:50:24.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.108+0000 7f01caffd640 1 -- 172.21.15.83:0/1491524029 learned_addr learned my addr 172.21.15.83:0/1491524029 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:24.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.108+0000 7f01ca7fc640 1 -- 172.21.15.83:0/1491524029 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01cc15bbc0 msgr2=0x7f01cc0d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:24.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.108+0000 7f01ca7fc640 1 --2- 172.21.15.83:0/1491524029 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01cc15bbc0 0x7f01cc0d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:24.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.108+0000 7f01ca7fc640 1 -- 172.21.15.83:0/1491524029 >> v1:172.21.15.167:6789/0 conn(0x7f01cc0d0020 legacy=0x7f01cc0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:24.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.108+0000 7f01ca7fc640 1 -- 172.21.15.83:0/1491524029 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01cc0cf700 con 0x7f01cc0b07b0 2024-04-17T10:50:24.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.112+0000 7f01ca7fc640 1 --2- 172.21.15.83:0/1491524029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01cc0b07b0 0x7f01cc0cfae0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f01b4009d90 tx=0x7f01b401c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ded535be042537fb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:24.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.112+0000 7f01c9ffb640 1 -- 172.21.15.83:0/1491524029 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b40b2070 con 0x7f01cc0b07b0 2024-04-17T10:50:24.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.112+0000 7f01c9ffb640 1 -- 172.21.15.83:0/1491524029 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b40a5a00 con 0x7f01cc0b07b0 2024-04-17T10:50:24.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.112+0000 7f01c9ffb640 1 -- 172.21.15.83:0/1491524029 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b40a5dd0 con 0x7f01cc0b07b0 2024-04-17T10:50:24.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.112+0000 7f01d143f640 1 -- 172.21.15.83:0/1491524029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01cc0b07b0 msgr2=0x7f01cc0cfae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:24.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.112+0000 7f01d143f640 1 --2- 172.21.15.83:0/1491524029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01cc0b07b0 0x7f01cc0cfae0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f01b4009d90 tx=0x7f01b401c940 comp rx=0 tx=0).stop 2024-04-17T10:50:24.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.112+0000 7f01d143f640 1 -- 172.21.15.83:0/1491524029 shutdown_connections 2024-04-17T10:50:24.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.112+0000 7f01d143f640 1 --2- 172.21.15.83:0/1491524029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01cc0b07b0 0x7f01cc0cfae0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:24.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.112+0000 7f01d143f640 1 --2- 172.21.15.83:0/1491524029 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01cc15bbc0 0x7f01cc0d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:24.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.112+0000 7f01d143f640 1 -- 172.21.15.83:0/1491524029 >> 172.21.15.83:0/1491524029 conn(0x7f01cc0d1850 msgr2=0x7f01cc0d1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:24.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.112+0000 7f01d143f640 1 -- 172.21.15.83:0/1491524029 shutdown_connections 2024-04-17T10:50:24.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.112+0000 7f01d143f640 1 -- 172.21.15.83:0/1491524029 wait complete. 2024-04-17T10:50:24.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.116+0000 7f01d143f640 1 Processor -- start 2024-04-17T10:50:24.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.116+0000 7f01d143f640 1 -- start start 2024-04-17T10:50:24.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.116+0000 7f01d143f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01cc0b07b0 0x7f01cc151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:24.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.116+0000 7f01d143f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01cc0d0020 0x7f01cc15af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:24.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.116+0000 7f01d143f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01cc15bbc0 0x7f01cc156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:24.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.116+0000 7f01d143f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc15c120 con 0x7f01cc0d0020 2024-04-17T10:50:24.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.116+0000 7f01d143f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc15bfa0 con 0x7f01cc15bbc0 2024-04-17T10:50:24.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.116+0000 7f01d143f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc1687f0 con 0x7f01cc0b07b0 2024-04-17T10:50:24.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.116+0000 7f01ca7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01cc0d0020 0x7f01cc15af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:24.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.116+0000 7f01ca7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01cc0d0020 0x7f01cc15af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40796/0 (socket says 172.21.15.83:40796) 2024-04-17T10:50:24.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.116+0000 7f01ca7fc640 1 -- 172.21.15.83:0/2110908662 learned_addr learned my addr 172.21.15.83:0/2110908662 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:24.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.116+0000 7f01cb7fe640 1 --2- 172.21.15.83:0/2110908662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01cc15bbc0 0x7f01cc156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:24.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.116+0000 7f01ca7fc640 1 -- 172.21.15.83:0/2110908662 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01cc0b07b0 msgr2=0x7f01cc151f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:24.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.116+0000 7f01ca7fc640 1 --2- 172.21.15.83:0/2110908662 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01cc0b07b0 0x7f01cc151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:24.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.116+0000 7f01ca7fc640 1 -- 172.21.15.83:0/2110908662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01cc15bbc0 msgr2=0x7f01cc156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:24.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.116+0000 7f01ca7fc640 1 --2- 172.21.15.83:0/2110908662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01cc15bbc0 0x7f01cc156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:24.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.116+0000 7f01ca7fc640 1 -- 172.21.15.83:0/2110908662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01cc0b5ef0 con 0x7f01cc0d0020 2024-04-17T10:50:24.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.116+0000 7f01ca7fc640 1 --2- 172.21.15.83:0/2110908662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01cc0d0020 0x7f01cc15af90 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f01b40b0040 tx=0x7f01b40196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:24.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.116+0000 7f01c37fe640 1 -- 172.21.15.83:0/2110908662 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b40b2050 con 0x7f01cc0d0020 2024-04-17T10:50:24.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.116+0000 7f01c37fe640 1 -- 172.21.15.83:0/2110908662 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b40be070 con 0x7f01cc0d0020 2024-04-17T10:50:24.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.116+0000 7f01c37fe640 1 -- 172.21.15.83:0/2110908662 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b40adb10 con 0x7f01cc0d0020 2024-04-17T10:50:24.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.116+0000 7f01d143f640 1 -- 172.21.15.83:0/2110908662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01cc0b52e0 con 0x7f01cc0d0020 2024-04-17T10:50:24.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.116+0000 7f01d143f640 1 -- 172.21.15.83:0/2110908662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01cc0b58a0 con 0x7f01cc0d0020 2024-04-17T10:50:24.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.116+0000 7f01d143f640 1 -- 172.21.15.83:0/2110908662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01cc074f90 con 0x7f01cc0d0020 2024-04-17T10:50:24.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.116+0000 7f01c37fe640 1 -- 172.21.15.83:0/2110908662 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f01b40adcb0 con 0x7f01cc0d0020 2024-04-17T10:50:24.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.120+0000 7f01c37fe640 1 --2- 172.21.15.83:0/2110908662 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f01ac066f60 0x7f01ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:24.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.120+0000 7f01c37fe640 1 -- 172.21.15.83:0/2110908662 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f01b41203b0 con 0x7f01cc0d0020 2024-04-17T10:50:24.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.120+0000 7f01caffd640 1 --2- 172.21.15.83:0/2110908662 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f01ac066f60 0x7f01ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:24.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.120+0000 7f01caffd640 1 --2- 172.21.15.83:0/2110908662 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f01ac066f60 0x7f01ac069420 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f01c4006f50 tx=0x7f01c4006ea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:24.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.124+0000 7f01c37fe640 1 -- 172.21.15.83:0/2110908662 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f01cc0b5ef0 con 0x7f01cc0d0020 2024-04-17T10:50:24.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.300+0000 7f01d143f640 1 -- 172.21.15.83:0/2110908662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f01cc072770 con 0x7f01cc0d0020 2024-04-17T10:50:24.295 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-04-17T10:50:24.300+0000 7f35c3b81640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T10:50:24.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.304+0000 7f01c37fe640 1 -- 172.21.15.83:0/2110908662 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v81) v1 ==== 161+0+0 (secure 0 0 0) 0x7f01b4016560 con 0x7f01cc0d0020 2024-04-17T10:50:24.297 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-17T10:50:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.308+0000 7f01d143f640 1 -- 172.21.15.83:0/2110908662 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f01ac066f60 msgr2=0x7f01ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.308+0000 7f01d143f640 1 --2- 172.21.15.83:0/2110908662 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f01ac066f60 0x7f01ac069420 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f01c4006f50 tx=0x7f01c4006ea0 comp rx=0 tx=0).stop 2024-04-17T10:50:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.308+0000 7f01d143f640 1 -- 172.21.15.83:0/2110908662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01cc0d0020 msgr2=0x7f01cc15af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.308+0000 7f01d143f640 1 --2- 172.21.15.83:0/2110908662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01cc0d0020 0x7f01cc15af90 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f01b40b0040 tx=0x7f01b40196e0 comp rx=0 tx=0).stop 2024-04-17T10:50:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.308+0000 7f01d143f640 1 -- 172.21.15.83:0/2110908662 shutdown_connections 2024-04-17T10:50:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.308+0000 7f01d143f640 1 --2- 172.21.15.83:0/2110908662 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f01ac066f60 0x7f01ac069420 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.308+0000 7f01d143f640 1 --2- 172.21.15.83:0/2110908662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01cc15bbc0 0x7f01cc156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.308+0000 7f01d143f640 1 --2- 172.21.15.83:0/2110908662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01cc0d0020 0x7f01cc15af90 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.308+0000 7f01d143f640 1 --2- 172.21.15.83:0/2110908662 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01cc0b07b0 0x7f01cc151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.308+0000 7f01d143f640 1 -- 172.21.15.83:0/2110908662 >> 172.21.15.83:0/2110908662 conn(0x7f01cc0d1850 msgr2=0x7f01cc1588a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.308+0000 7f01d143f640 1 -- 172.21.15.83:0/2110908662 shutdown_connections 2024-04-17T10:50:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.308+0000 7f01d143f640 1 -- 172.21.15.83:0/2110908662 wait complete. 2024-04-17T10:50:24.321 INFO:tasks.ceph.mds.a.smithi083.stderr: -534> 2024-04-17T10:50:16.592+0000 7fb86a2b6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T10:50:24.324 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-17T10:50:24.347 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-04-17T10:50:24.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.448+0000 7f99a290e640 1 Processor -- start 2024-04-17T10:50:24.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.448+0000 7f99a290e640 1 -- start start 2024-04-17T10:50:24.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f99a290e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f999c0b0900 0x7f999c0d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:24.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f99a290e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f999c0b2720 0x7f999c0b2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:24.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f99a290e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f999c06c150 con 0x7f999c0ce910 2024-04-17T10:50:24.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f99a290e640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f999c0d1020 con 0x7f999c0b2720 2024-04-17T10:50:24.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f99a290e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f999c06bf00 con 0x7f999c0b0900 2024-04-17T10:50:24.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f999bfff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f999c0ce910 0x7f999c0ae490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39092/0 (socket says 172.21.15.83:39092) 2024-04-17T10:50:24.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f999bfff640 1 -- 172.21.15.83:0/3491441176 learned_addr learned my addr 172.21.15.83:0/3491441176 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:24.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f999affd640 1 -- 172.21.15.83:0/3491441176 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2094588849 0 0) 0x7f999c06c150 con 0x7f999c0ce910 2024-04-17T10:50:24.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f999affd640 1 -- 172.21.15.83:0/3491441176 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9984003530 con 0x7f999c0ce910 2024-04-17T10:50:24.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f999b7fe640 1 --2- 172.21.15.83:0/3491441176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f999c0b0900 0x7f999c0d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:24.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f99a0eab640 1 --2- 172.21.15.83:0/3491441176 >> v2:172.21.15.167:3300/0 conn(0x7f999c0b2720 0x7f999c0b2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:24.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f999affd640 1 -- 172.21.15.83:0/3491441176 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2711855390 0 0) 0x7f9984003530 con 0x7f999c0ce910 2024-04-17T10:50:24.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f999affd640 1 -- 172.21.15.83:0/3491441176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f999c0b0900 msgr2=0x7f999c0d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:24.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f999affd640 1 --2- 172.21.15.83:0/3491441176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f999c0b0900 0x7f999c0d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:24.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f999affd640 1 -- 172.21.15.83:0/3491441176 >> v2:172.21.15.167:3300/0 conn(0x7f999c0b2720 msgr2=0x7f999c0b2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:24.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f999affd640 1 --2- 172.21.15.83:0/3491441176 >> v2:172.21.15.167:3300/0 conn(0x7f999c0b2720 0x7f999c0b2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:24.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f999affd640 1 -- 172.21.15.83:0/3491441176 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f999c0b1c40 con 0x7f999c0ce910 2024-04-17T10:50:24.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f999affd640 1 -- 172.21.15.83:0/3491441176 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f998c001680 con 0x7f999c0ce910 2024-04-17T10:50:24.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f99a0eab640 1 --2- 172.21.15.83:0/3491441176 >> v2:172.21.15.167:3300/0 conn(0x7f999c0b2720 0x7f999c0b2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:50:24.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f999affd640 1 -- 172.21.15.83:0/3491441176 >> v1:172.21.15.83:6789/0 conn(0x7f999c0ce910 legacy=0x7f999c0ae490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:24.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f999affd640 1 --2- 172.21.15.83:0/3491441176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9984003a30 0x7f9984003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:24.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f999affd640 1 --2- 172.21.15.83:0/3491441176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f999c0b0900 0x7f9984004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:24.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f999affd640 1 --2- 172.21.15.83:0/3491441176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9984004d50 0x7f99840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:24.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f999affd640 1 -- 172.21.15.83:0/3491441176 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999c0d1020 con 0x7f9984003a30 2024-04-17T10:50:24.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f999affd640 1 -- 172.21.15.83:0/3491441176 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999c06bf00 con 0x7f9984004d50 2024-04-17T10:50:24.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f999affd640 1 -- 172.21.15.83:0/3491441176 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f998400bb30 con 0x7f999c0b0900 2024-04-17T10:50:24.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f999bfff640 1 --2- 172.21.15.83:0/3491441176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9984003a30 0x7f9984003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:24.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f999b7fe640 1 --2- 172.21.15.83:0/3491441176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f999c0b0900 0x7f9984004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:24.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f999bfff640 1 -- 172.21.15.83:0/3491441176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f999c0b0900 msgr2=0x7f9984004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:24.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f999bfff640 1 --2- 172.21.15.83:0/3491441176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f999c0b0900 0x7f9984004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:24.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f999bfff640 1 -- 172.21.15.83:0/3491441176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9984004d50 msgr2=0x7f99840051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:24.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f999bfff640 1 --2- 172.21.15.83:0/3491441176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9984004d50 0x7f99840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:24.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f999bfff640 1 -- 172.21.15.83:0/3491441176 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9984003530 con 0x7f9984003a30 2024-04-17T10:50:24.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f999b7fe640 1 --2- 172.21.15.83:0/3491441176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f999c0b0900 0x7f9984004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:24.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.452+0000 7f999bfff640 1 --2- 172.21.15.83:0/3491441176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9984003a30 0x7f9984003e50 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f998c001ec0 tx=0x7f998c01cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f7992c04f3f5c45 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:24.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f999affd640 1 -- 172.21.15.83:0/3491441176 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998c007cb0 con 0x7f9984003a30 2024-04-17T10:50:24.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f999affd640 1 -- 172.21.15.83:0/3491441176 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f998c0a9020 con 0x7f9984003a30 2024-04-17T10:50:24.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f999affd640 1 -- 172.21.15.83:0/3491441176 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998c005b20 con 0x7f9984003a30 2024-04-17T10:50:24.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f99a290e640 1 -- 172.21.15.83:0/3491441176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9984003a30 msgr2=0x7f9984003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:24.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f99a290e640 1 --2- 172.21.15.83:0/3491441176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9984003a30 0x7f9984003e50 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f998c001ec0 tx=0x7f998c01cba0 comp rx=0 tx=0).stop 2024-04-17T10:50:24.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f99a0eab640 1 -- 172.21.15.83:0/3491441176 reap_dead start 2024-04-17T10:50:24.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f99a290e640 1 -- 172.21.15.83:0/3491441176 shutdown_connections 2024-04-17T10:50:24.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f99a290e640 1 -- 172.21.15.83:0/3491441176 >> 172.21.15.83:0/3491441176 conn(0x7f999c0d17a0 msgr2=0x7f999c0d1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:24.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f99a290e640 1 -- 172.21.15.83:0/3491441176 shutdown_connections 2024-04-17T10:50:24.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f99a290e640 1 -- 172.21.15.83:0/3491441176 wait complete. 2024-04-17T10:50:24.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f99a290e640 1 Processor -- start 2024-04-17T10:50:24.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f99a290e640 1 -- start start 2024-04-17T10:50:24.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f99a290e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9984003a30 0x7f999c15aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:24.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f99a290e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f999c1608d0 0x7f999c0b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:24.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f99a290e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f999c0b6910 0x7f999c0c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:24.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f99a290e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999c15d820 con 0x7f9984003a30 2024-04-17T10:50:24.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f99a290e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999c15d6a0 con 0x7f999c0b6910 2024-04-17T10:50:24.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f99a290e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f999c15d9a0 con 0x7f999c1608d0 2024-04-17T10:50:24.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f999bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9984003a30 0x7f999c15aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:24.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f999bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9984003a30 0x7f999c15aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40828/0 (socket says 172.21.15.83:40828) 2024-04-17T10:50:24.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f999bfff640 1 -- 172.21.15.83:0/869359737 learned_addr learned my addr 172.21.15.83:0/869359737 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:24.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f999bfff640 1 -- 172.21.15.83:0/869359737 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f999c1608d0 msgr2=0x7f999c0b63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:24.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f999bfff640 1 --2- 172.21.15.83:0/869359737 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f999c1608d0 0x7f999c0b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:24.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f99a0eab640 1 --2- 172.21.15.83:0/869359737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f999c0b6910 0x7f999c0c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:24.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f999bfff640 1 -- 172.21.15.83:0/869359737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f999c0b6910 msgr2=0x7f999c0c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:24.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f999bfff640 1 --2- 172.21.15.83:0/869359737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f999c0b6910 0x7f999c0c6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:24.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f999bfff640 1 -- 172.21.15.83:0/869359737 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f999c15f830 con 0x7f9984003a30 2024-04-17T10:50:24.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f999bfff640 1 --2- 172.21.15.83:0/869359737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9984003a30 0x7f999c15aa00 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f998c0ab040 tx=0x7f9984005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:24.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f9998ff9640 1 -- 172.21.15.83:0/869359737 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998c0bc070 con 0x7f9984003a30 2024-04-17T10:50:24.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f9998ff9640 1 -- 172.21.15.83:0/869359737 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f998c005d00 con 0x7f9984003a30 2024-04-17T10:50:24.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f99a290e640 1 -- 172.21.15.83:0/869359737 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f999c15fb10 con 0x7f9984003a30 2024-04-17T10:50:24.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f9998ff9640 1 -- 172.21.15.83:0/869359737 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998c004a30 con 0x7f9984003a30 2024-04-17T10:50:24.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.456+0000 7f99a290e640 1 -- 172.21.15.83:0/869359737 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f999c1600d0 con 0x7f9984003a30 2024-04-17T10:50:24.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.460+0000 7f99a290e640 1 -- 172.21.15.83:0/869359737 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f999c074f90 con 0x7f9984003a30 2024-04-17T10:50:24.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.460+0000 7f9998ff9640 1 -- 172.21.15.83:0/869359737 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f998c0bf070 con 0x7f9984003a30 2024-04-17T10:50:24.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.460+0000 7f9998ff9640 1 --2- 172.21.15.83:0/869359737 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9988066f90 0x7f9988069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:24.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.460+0000 7f9998ff9640 1 -- 172.21.15.83:0/869359737 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f998c11e2c0 con 0x7f9984003a30 2024-04-17T10:50:24.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.460+0000 7f999b7fe640 1 --2- 172.21.15.83:0/869359737 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9988066f90 0x7f9988069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:24.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.460+0000 7f999b7fe640 1 --2- 172.21.15.83:0/869359737 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9988066f90 0x7f9988069450 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f999c15bf60 tx=0x7f9990008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:24.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.464+0000 7f9998ff9640 1 -- 172.21.15.83:0/869359737 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f998c0b4070 con 0x7f9984003a30 2024-04-17T10:50:24.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:24.660+0000 7f99a290e640 1 -- 172.21.15.83:0/869359737 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f999c0b6cf0 con 0x7f9984003a30 2024-04-17T10:50:25.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.312+0000 7f9998ff9640 1 -- 172.21.15.83:0/869359737 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v82) v1 ==== 104+0+0 (secure 0 0 0) 0x7f998c0168b0 con 0x7f9984003a30 2024-04-17T10:50:25.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.316+0000 7f99a290e640 1 -- 172.21.15.83:0/869359737 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9988066f90 msgr2=0x7f9988069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:25.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.316+0000 7f99a290e640 1 --2- 172.21.15.83:0/869359737 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9988066f90 0x7f9988069450 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f999c15bf60 tx=0x7f9990008040 comp rx=0 tx=0).stop 2024-04-17T10:50:25.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.316+0000 7f99a290e640 1 -- 172.21.15.83:0/869359737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9984003a30 msgr2=0x7f999c15aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:25.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.316+0000 7f99a290e640 1 --2- 172.21.15.83:0/869359737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9984003a30 0x7f999c15aa00 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f998c0ab040 tx=0x7f9984005840 comp rx=0 tx=0).stop 2024-04-17T10:50:25.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.316+0000 7f99a290e640 1 -- 172.21.15.83:0/869359737 shutdown_connections 2024-04-17T10:50:25.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.316+0000 7f99a290e640 1 --2- 172.21.15.83:0/869359737 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9988066f90 0x7f9988069450 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:25.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.316+0000 7f99a290e640 1 --2- 172.21.15.83:0/869359737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f999c0b6910 0x7f999c0c6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:25.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.316+0000 7f99a290e640 1 --2- 172.21.15.83:0/869359737 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f999c1608d0 0x7f999c0b63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:25.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.316+0000 7f99a290e640 1 --2- 172.21.15.83:0/869359737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9984003a30 0x7f999c15aa00 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:25.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.316+0000 7f99a290e640 1 -- 172.21.15.83:0/869359737 >> 172.21.15.83:0/869359737 conn(0x7f999c0d17a0 msgr2=0x7f999c06eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:25.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.316+0000 7f99a290e640 1 -- 172.21.15.83:0/869359737 shutdown_connections 2024-04-17T10:50:25.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.316+0000 7f99a290e640 1 -- 172.21.15.83:0/869359737 wait complete. 2024-04-17T10:50:25.331 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-17T10:50:25.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.516+0000 7fcd6df46640 1 Processor -- start 2024-04-17T10:50:25.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.516+0000 7fcd6df46640 1 -- start start 2024-04-17T10:50:25.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.516+0000 7fcd6df46640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcd680aebd0 0x7fcd680ced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:25.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.516+0000 7fcd6df46640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd6806abe0 0x7fcd6806b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:25.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.516+0000 7fcd6df46640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd680cf3e0 con 0x7fcd6815b990 2024-04-17T10:50:25.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.516+0000 7fcd6df46640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd6815e5c0 con 0x7fcd680aebd0 2024-04-17T10:50:25.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.516+0000 7fcd6df46640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd680cd890 con 0x7fcd6806abe0 2024-04-17T10:50:25.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.516+0000 7fcd677fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcd680aebd0 0x7fcd680ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:25.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.516+0000 7fcd677fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcd680aebd0 0x7fcd680ced50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40830/0 (socket says 172.21.15.83:40830) 2024-04-17T10:50:25.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.516+0000 7fcd677fe640 1 -- 172.21.15.83:0/126728899 learned_addr learned my addr 172.21.15.83:0/126728899 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:25.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.516+0000 7fcd677fe640 1 -- 172.21.15.83:0/126728899 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd6806abe0 msgr2=0x7fcd6806b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:25.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.516+0000 7fcd677fe640 1 --2- 172.21.15.83:0/126728899 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd6806abe0 0x7fcd6806b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:25.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.516+0000 7fcd677fe640 1 -- 172.21.15.83:0/126728899 >> v1:172.21.15.167:6789/0 conn(0x7fcd6815b990 legacy=0x7fcd680cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:25.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.516+0000 7fcd677fe640 1 -- 172.21.15.83:0/126728899 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd680b4230 con 0x7fcd680aebd0 2024-04-17T10:50:25.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.516+0000 7fcd677fe640 1 --2- 172.21.15.83:0/126728899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd680aebd0 0x7fcd680ced50 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009870 tx=0x7fcd5801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d9c77704063472 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:25.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd66ffd640 1 -- 172.21.15.83:0/126728899 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd580b0070 con 0x7fcd680aebd0 2024-04-17T10:50:25.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd66ffd640 1 -- 172.21.15.83:0/126728899 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd580ab600 con 0x7fcd680aebd0 2024-04-17T10:50:25.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd66ffd640 1 -- 172.21.15.83:0/126728899 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd580ab9d0 con 0x7fcd680aebd0 2024-04-17T10:50:25.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd6df46640 1 -- 172.21.15.83:0/126728899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd680aebd0 msgr2=0x7fcd680ced50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:25.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd6df46640 1 --2- 172.21.15.83:0/126728899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd680aebd0 0x7fcd680ced50 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009870 tx=0x7fcd5801c520 comp rx=0 tx=0).stop 2024-04-17T10:50:25.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd6df46640 1 -- 172.21.15.83:0/126728899 shutdown_connections 2024-04-17T10:50:25.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd6df46640 1 --2- 172.21.15.83:0/126728899 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd6806abe0 0x7fcd6806b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:25.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd6df46640 1 --2- 172.21.15.83:0/126728899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd680aebd0 0x7fcd680ced50 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:25.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd6df46640 1 -- 172.21.15.83:0/126728899 >> 172.21.15.83:0/126728899 conn(0x7fcd680cc050 msgr2=0x7fcd680cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:25.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd6df46640 1 -- 172.21.15.83:0/126728899 shutdown_connections 2024-04-17T10:50:25.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd6df46640 1 -- 172.21.15.83:0/126728899 wait complete. 2024-04-17T10:50:25.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd6df46640 1 Processor -- start 2024-04-17T10:50:25.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd6df46640 1 -- start start 2024-04-17T10:50:25.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd6df46640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd6806abe0 0x7fcd6815d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:25.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd6df46640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd680aebd0 0x7fcd6813dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:25.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd6df46640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd6815b990 0x7fcd680cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:25.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd6df46640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd681678d0 con 0x7fcd6815b990 2024-04-17T10:50:25.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd6df46640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd680aefb0 con 0x7fcd6806abe0 2024-04-17T10:50:25.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd6df46640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd68167a50 con 0x7fcd680aebd0 2024-04-17T10:50:25.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd67fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd6815b990 0x7fcd680cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:25.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd67fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd6815b990 0x7fcd680cf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40840/0 (socket says 172.21.15.83:40840) 2024-04-17T10:50:25.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd67fff640 1 -- 172.21.15.83:0/3827439098 learned_addr learned my addr 172.21.15.83:0/3827439098 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:25.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd5ffff640 1 --2- 172.21.15.83:0/3827439098 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd680aebd0 0x7fcd6813dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:25.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd677fe640 1 --2- 172.21.15.83:0/3827439098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd6806abe0 0x7fcd6815d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:25.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd67fff640 1 -- 172.21.15.83:0/3827439098 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd680aebd0 msgr2=0x7fcd6813dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:25.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd67fff640 1 --2- 172.21.15.83:0/3827439098 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd680aebd0 0x7fcd6813dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:25.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd67fff640 1 -- 172.21.15.83:0/3827439098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd6806abe0 msgr2=0x7fcd6815d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:25.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd67fff640 1 --2- 172.21.15.83:0/3827439098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd6806abe0 0x7fcd6815d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:25.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd67fff640 1 -- 172.21.15.83:0/3827439098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd680702e0 con 0x7fcd6815b990 2024-04-17T10:50:25.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd67fff640 1 --2- 172.21.15.83:0/3827439098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd6815b990 0x7fcd680cf5b0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000b9e0 tx=0x7fcd6000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:25.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd64ff9640 1 -- 172.21.15.83:0/3827439098 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd60019070 con 0x7fcd6815b990 2024-04-17T10:50:25.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd64ff9640 1 -- 172.21.15.83:0/3827439098 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6000cac0 con 0x7fcd6815b990 2024-04-17T10:50:25.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd64ff9640 1 -- 172.21.15.83:0/3827439098 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd60014440 con 0x7fcd6815b990 2024-04-17T10:50:25.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd6df46640 1 -- 172.21.15.83:0/3827439098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd6815df00 con 0x7fcd6815b990 2024-04-17T10:50:25.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.520+0000 7fcd6df46640 1 -- 172.21.15.83:0/3827439098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd6815c910 con 0x7fcd6815b990 2024-04-17T10:50:25.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.524+0000 7fcd6df46640 1 -- 172.21.15.83:0/3827439098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd68075000 con 0x7fcd6815b990 2024-04-17T10:50:25.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.524+0000 7fcd64ff9640 1 -- 172.21.15.83:0/3827439098 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcd6001c070 con 0x7fcd6815b990 2024-04-17T10:50:25.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.524+0000 7fcd64ff9640 1 --2- 172.21.15.83:0/3827439098 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd48066f60 0x7fcd48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:25.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.524+0000 7fcd677fe640 1 --2- 172.21.15.83:0/3827439098 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd48066f60 0x7fcd48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:25.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.524+0000 7fcd64ff9640 1 -- 172.21.15.83:0/3827439098 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fcd60084e90 con 0x7fcd6815b990 2024-04-17T10:50:25.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.524+0000 7fcd677fe640 1 --2- 172.21.15.83:0/3827439098 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd48066f60 0x7fcd48069420 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fcd580ad000 tx=0x7fcd58008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:25.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.528+0000 7fcd64ff9640 1 -- 172.21.15.83:0/3827439098 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcd60055d10 con 0x7fcd6815b990 2024-04-17T10:50:25.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:25.688+0000 7fcd6df46640 1 -- 172.21.15.83:0/3827439098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcd681656a0 con 0x7fcd6815b990 2024-04-17T10:50:26.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.512+0000 7fcd64ff9640 1 -- 172.21.15.83:0/3827439098 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v132) v1 ==== 181+0+0 (secure 0 0 0) 0x7fcd600860d0 con 0x7fcd6815b990 2024-04-17T10:50:26.506 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2024-04-17T10:50:26.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.516+0000 7fcd6df46640 1 -- 172.21.15.83:0/3827439098 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd48066f60 msgr2=0x7fcd48069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:26.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.516+0000 7fcd6df46640 1 --2- 172.21.15.83:0/3827439098 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd48066f60 0x7fcd48069420 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fcd580ad000 tx=0x7fcd58008040 comp rx=0 tx=0).stop 2024-04-17T10:50:26.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.516+0000 7fcd6df46640 1 -- 172.21.15.83:0/3827439098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd6815b990 msgr2=0x7fcd680cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:26.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.516+0000 7fcd6df46640 1 --2- 172.21.15.83:0/3827439098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd6815b990 0x7fcd680cf5b0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000b9e0 tx=0x7fcd6000bea0 comp rx=0 tx=0).stop 2024-04-17T10:50:26.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.516+0000 7fcd6df46640 1 -- 172.21.15.83:0/3827439098 shutdown_connections 2024-04-17T10:50:26.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.516+0000 7fcd6df46640 1 --2- 172.21.15.83:0/3827439098 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fcd48066f60 0x7fcd48069420 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:26.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.516+0000 7fcd6df46640 1 --2- 172.21.15.83:0/3827439098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd6815b990 0x7fcd680cf5b0 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:26.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.516+0000 7fcd6df46640 1 --2- 172.21.15.83:0/3827439098 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd680aebd0 0x7fcd6813dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:26.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.516+0000 7fcd6df46640 1 --2- 172.21.15.83:0/3827439098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd6806abe0 0x7fcd6815d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:26.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.516+0000 7fcd6df46640 1 -- 172.21.15.83:0/3827439098 >> 172.21.15.83:0/3827439098 conn(0x7fcd680cc050 msgr2=0x7fcd6815ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:26.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.516+0000 7fcd6df46640 1 -- 172.21.15.83:0/3827439098 shutdown_connections 2024-04-17T10:50:26.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.516+0000 7fcd6df46640 1 -- 172.21.15.83:0/3827439098 wait complete. 2024-04-17T10:50:26.533 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-17T10:50:26.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.668+0000 7f6ee2a44640 1 Processor -- start 2024-04-17T10:50:26.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.672+0000 7f6ee2a44640 1 -- start start 2024-04-17T10:50:26.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.672+0000 7f6ee2a44640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6edc0aedf0 0x7f6edc0b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:26.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.672+0000 7f6ee2a44640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6edc15cb40 0x7f6edc06abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:26.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.672+0000 7f6ee2a44640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6edc0b39d0 con 0x7f6edc06b120 2024-04-17T10:50:26.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.672+0000 7f6ee2a44640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6edc072730 con 0x7f6edc0aedf0 2024-04-17T10:50:26.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.672+0000 7f6ee2a44640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc071290 con 0x7f6edc15cb40 2024-04-17T10:50:26.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.672+0000 7f6edbfff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6edc0aedf0 0x7f6edc0b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:26.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.672+0000 7f6ee0fe1640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f6edc06b120 0x7f6edc15f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:52512/0 (socket says 172.21.15.83:52512) 2024-04-17T10:50:26.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.672+0000 7f6ee0fe1640 1 -- 172.21.15.83:0/4282307593 learned_addr learned my addr 172.21.15.83:0/4282307593 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:26.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.672+0000 7f6edbfff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6edc0aedf0 0x7f6edc0b30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:53894/0 (socket says 172.21.15.83:53894) 2024-04-17T10:50:26.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.672+0000 7f6edb7fe640 1 --2- 172.21.15.83:0/4282307593 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6edc15cb40 0x7f6edc06abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:26.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.672+0000 7f6edbfff640 1 -- 172.21.15.83:0/4282307593 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6edc15cb40 msgr2=0x7f6edc06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:26.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.672+0000 7f6edbfff640 1 --2- 172.21.15.83:0/4282307593 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6edc15cb40 0x7f6edc06abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:26.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.672+0000 7f6edbfff640 1 -- 172.21.15.83:0/4282307593 >> v1:172.21.15.167:6789/0 conn(0x7f6edc06b120 legacy=0x7f6edc15f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:26.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.672+0000 7f6edbfff640 1 -- 172.21.15.83:0/4282307593 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6edc070c40 con 0x7f6edc0aedf0 2024-04-17T10:50:26.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.676+0000 7f6edbfff640 1 --2- 172.21.15.83:0/4282307593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6edc0aedf0 0x7f6edc0b30f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4009d90 tx=0x7f6ed401c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df7d3d8930b49bfb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:26.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.676+0000 7f6edaffd640 1 -- 172.21.15.83:0/4282307593 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed40a66d0 con 0x7f6edc0aedf0 2024-04-17T10:50:26.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.676+0000 7f6edaffd640 1 -- 172.21.15.83:0/4282307593 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed40a6870 con 0x7f6edc0aedf0 2024-04-17T10:50:26.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.676+0000 7f6edaffd640 1 -- 172.21.15.83:0/4282307593 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed40a6c40 con 0x7f6edc0aedf0 2024-04-17T10:50:26.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.676+0000 7f6ee2a44640 1 -- 172.21.15.83:0/4282307593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6edc0aedf0 msgr2=0x7f6edc0b30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:26.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.676+0000 7f6ee2a44640 1 --2- 172.21.15.83:0/4282307593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6edc0aedf0 0x7f6edc0b30f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4009d90 tx=0x7f6ed401c8e0 comp rx=0 tx=0).stop 2024-04-17T10:50:26.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.676+0000 7f6ee2a44640 1 -- 172.21.15.83:0/4282307593 shutdown_connections 2024-04-17T10:50:26.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.676+0000 7f6ee2a44640 1 --2- 172.21.15.83:0/4282307593 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6edc15cb40 0x7f6edc06abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:26.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.676+0000 7f6ee2a44640 1 --2- 172.21.15.83:0/4282307593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6edc0aedf0 0x7f6edc0b30f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:26.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.676+0000 7f6ee2a44640 1 -- 172.21.15.83:0/4282307593 >> 172.21.15.83:0/4282307593 conn(0x7f6edc0cc050 msgr2=0x7f6edc0cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:26.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.676+0000 7f6ee2a44640 1 -- 172.21.15.83:0/4282307593 shutdown_connections 2024-04-17T10:50:26.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.676+0000 7f6ee2a44640 1 -- 172.21.15.83:0/4282307593 wait complete. 2024-04-17T10:50:26.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.676+0000 7f6ee2a44640 1 Processor -- start 2024-04-17T10:50:26.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.676+0000 7f6ee2a44640 1 -- start start 2024-04-17T10:50:26.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.676+0000 7f6ee2a44640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6edc06b120 0x7f6edc147c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:26.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.676+0000 7f6ee2a44640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6edc0aedf0 0x7f6edc1570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:26.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.676+0000 7f6ee2a44640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6edc15cb40 0x7f6edc150b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:26.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.676+0000 7f6ee2a44640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc06c2e0 con 0x7f6edc0aedf0 2024-04-17T10:50:26.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.676+0000 7f6ee2a44640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc06c160 con 0x7f6edc15cb40 2024-04-17T10:50:26.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.676+0000 7f6ee2a44640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc168890 con 0x7f6edc06b120 2024-04-17T10:50:26.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.676+0000 7f6edb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6edc0aedf0 0x7f6edc1570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:26.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.676+0000 7f6edb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6edc0aedf0 0x7f6edc1570b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40854/0 (socket says 172.21.15.83:40854) 2024-04-17T10:50:26.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.676+0000 7f6edb7fe640 1 -- 172.21.15.83:0/2604730989 learned_addr learned my addr 172.21.15.83:0/2604730989 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:26.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.680+0000 7f6ee0fe1640 1 --2- 172.21.15.83:0/2604730989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6edc15cb40 0x7f6edc150b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.680+0000 7f6edb7fe640 1 -- 172.21.15.83:0/2604730989 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6edc06b120 msgr2=0x7f6edc147c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.680+0000 7f6edb7fe640 1 --2- 172.21.15.83:0/2604730989 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6edc06b120 0x7f6edc147c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.680+0000 7f6edb7fe640 1 -- 172.21.15.83:0/2604730989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6edc15cb40 msgr2=0x7f6edc150b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.680+0000 7f6edb7fe640 1 --2- 172.21.15.83:0/2604730989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6edc15cb40 0x7f6edc150b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.680+0000 7f6edb7fe640 1 -- 172.21.15.83:0/2604730989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6edc15aee0 con 0x7f6edc0aedf0 2024-04-17T10:50:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.680+0000 7f6edb7fe640 1 --2- 172.21.15.83:0/2604730989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6edc0aedf0 0x7f6edc1570b0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f6ed000e930 tx=0x7f6ed000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.680+0000 7f6ed8ff9640 1 -- 172.21.15.83:0/2604730989 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed0017070 con 0x7f6edc0aedf0 2024-04-17T10:50:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.680+0000 7f6ed8ff9640 1 -- 172.21.15.83:0/2604730989 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed000ca00 con 0x7f6edc0aedf0 2024-04-17T10:50:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.680+0000 7f6ee2a44640 1 -- 172.21.15.83:0/2604730989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6edc15b1c0 con 0x7f6edc0aedf0 2024-04-17T10:50:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.680+0000 7f6ed8ff9640 1 -- 172.21.15.83:0/2604730989 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed000cdb0 con 0x7f6edc0aedf0 2024-04-17T10:50:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.680+0000 7f6ee2a44640 1 -- 172.21.15.83:0/2604730989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6edc15b780 con 0x7f6edc0aedf0 2024-04-17T10:50:26.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.680+0000 7f6ee2a44640 1 -- 172.21.15.83:0/2604730989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6edc075000 con 0x7f6edc0aedf0 2024-04-17T10:50:26.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.680+0000 7f6ed8ff9640 1 -- 172.21.15.83:0/2604730989 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6ed001a070 con 0x7f6edc0aedf0 2024-04-17T10:50:26.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.680+0000 7f6ed8ff9640 1 --2- 172.21.15.83:0/2604730989 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6ec8066f60 0x7f6ec8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:26.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.680+0000 7f6ed8ff9640 1 -- 172.21.15.83:0/2604730989 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5501+0+0 (secure 0 0 0) 0x7f6ed00852e0 con 0x7f6edc0aedf0 2024-04-17T10:50:26.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.680+0000 7f6edbfff640 1 --2- 172.21.15.83:0/2604730989 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6ec8066f60 0x7f6ec8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:26.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.680+0000 7f6edbfff640 1 --2- 172.21.15.83:0/2604730989 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6ec8066f60 0x7f6ec8069420 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6ed40af000 tx=0x7f6ed4004ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:26.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.684+0000 7f6ed8ff9640 1 -- 172.21.15.83:0/2604730989 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6ed0056510 con 0x7f6edc0aedf0 2024-04-17T10:50:26.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:26.840+0000 7f6ee2a44640 1 -- 172.21.15.83:0/2604730989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6edc0b2cc0 con 0x7f6edc0aedf0 2024-04-17T10:50:27.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.516+0000 7f6ed8ff9640 1 -- 172.21.15.83:0/2604730989 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v133) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6ed0055e10 con 0x7f6edc0aedf0 2024-04-17T10:50:27.508 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2024-04-17T10:50:27.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.520+0000 7f6ee2a44640 1 -- 172.21.15.83:0/2604730989 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6ec8066f60 msgr2=0x7f6ec8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:27.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.520+0000 7f6ee2a44640 1 --2- 172.21.15.83:0/2604730989 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6ec8066f60 0x7f6ec8069420 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6ed40af000 tx=0x7f6ed4004ac0 comp rx=0 tx=0).stop 2024-04-17T10:50:27.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.520+0000 7f6ee2a44640 1 -- 172.21.15.83:0/2604730989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6edc0aedf0 msgr2=0x7f6edc1570b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:27.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.520+0000 7f6ee2a44640 1 --2- 172.21.15.83:0/2604730989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6edc0aedf0 0x7f6edc1570b0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f6ed000e930 tx=0x7f6ed000edf0 comp rx=0 tx=0).stop 2024-04-17T10:50:27.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.520+0000 7f6ee2a44640 1 -- 172.21.15.83:0/2604730989 shutdown_connections 2024-04-17T10:50:27.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.520+0000 7f6ee2a44640 1 --2- 172.21.15.83:0/2604730989 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6ec8066f60 0x7f6ec8069420 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:27.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.520+0000 7f6ee2a44640 1 --2- 172.21.15.83:0/2604730989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6edc15cb40 0x7f6edc150b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:27.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.520+0000 7f6ee2a44640 1 --2- 172.21.15.83:0/2604730989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6edc0aedf0 0x7f6edc1570b0 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:27.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.520+0000 7f6ee2a44640 1 --2- 172.21.15.83:0/2604730989 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6edc06b120 0x7f6edc147c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:27.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.520+0000 7f6ee2a44640 1 -- 172.21.15.83:0/2604730989 >> 172.21.15.83:0/2604730989 conn(0x7f6edc0cc050 msgr2=0x7f6edc15d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:27.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.520+0000 7f6ee2a44640 1 -- 172.21.15.83:0/2604730989 shutdown_connections 2024-04-17T10:50:27.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.520+0000 7f6ee2a44640 1 -- 172.21.15.83:0/2604730989 wait complete. 2024-04-17T10:50:27.545 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-17T10:50:27.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.676+0000 7f39c3572640 1 Processor -- start 2024-04-17T10:50:27.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.676+0000 7f39c3572640 1 -- start start 2024-04-17T10:50:27.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.676+0000 7f39c3572640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f39bc150760 0x7f39bc0b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:27.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.676+0000 7f39c3572640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f39bc150b40 0x7f39bc15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:27.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.676+0000 7f39c3572640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39bc0b2910 con 0x7f39bc15da00 2024-04-17T10:50:27.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.676+0000 7f39c3572640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39bc0b3520 con 0x7f39bc150760 2024-04-17T10:50:27.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.676+0000 7f39c3572640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39bc15de30 con 0x7f39bc150b40 2024-04-17T10:50:27.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.676+0000 7f39c1d6f640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f39bc150b40 0x7f39bc15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.680+0000 7f39c2570640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f39bc150760 0x7f39bc0b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.680+0000 7f39c2570640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f39bc150760 0x7f39bc0b39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44800/0 (socket says 172.21.15.83:44800) 2024-04-17T10:50:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.680+0000 7f39c2570640 1 -- 172.21.15.83:0/2236076429 learned_addr learned my addr 172.21.15.83:0/2236076429 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.680+0000 7f39c2570640 1 -- 172.21.15.83:0/2236076429 >> v2:172.21.15.167:3300/0 conn(0x7f39bc150b40 msgr2=0x7f39bc15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.684+0000 7f39c2570640 1 --2- 172.21.15.83:0/2236076429 >> v2:172.21.15.167:3300/0 conn(0x7f39bc150b40 0x7f39bc15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.684+0000 7f39c2570640 1 -- 172.21.15.83:0/2236076429 >> v1:172.21.15.83:6789/0 conn(0x7f39bc15da00 legacy=0x7f39bc071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.684+0000 7f39c2570640 1 -- 172.21.15.83:0/2236076429 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39bc0b2bf0 con 0x7f39bc150760 2024-04-17T10:50:27.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.688+0000 7f39c1d6f640 1 --2- 172.21.15.83:0/2236076429 >> v2:172.21.15.167:3300/0 conn(0x7f39bc150b40 0x7f39bc15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:27.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.688+0000 7f39c2570640 1 --2- 172.21.15.83:0/2236076429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39bc150760 0x7f39bc0b39b0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f39bc06dbe0 tx=0x7f39b801cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4186b236de59d38 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:27.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.688+0000 7f39c156e640 1 -- 172.21.15.83:0/2236076429 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b80b2070 con 0x7f39bc150760 2024-04-17T10:50:27.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.688+0000 7f39c156e640 1 -- 172.21.15.83:0/2236076429 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b80a5ce0 con 0x7f39bc150760 2024-04-17T10:50:27.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.688+0000 7f39c156e640 1 -- 172.21.15.83:0/2236076429 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b80ad660 con 0x7f39bc150760 2024-04-17T10:50:27.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.692+0000 7f39c3572640 1 -- 172.21.15.83:0/2236076429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39bc150760 msgr2=0x7f39bc0b39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:27.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.692+0000 7f39c3572640 1 --2- 172.21.15.83:0/2236076429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39bc150760 0x7f39bc0b39b0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f39bc06dbe0 tx=0x7f39b801cd20 comp rx=0 tx=0).stop 2024-04-17T10:50:27.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.704+0000 7f39c3572640 1 -- 172.21.15.83:0/2236076429 shutdown_connections 2024-04-17T10:50:27.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.704+0000 7f39c3572640 1 --2- 172.21.15.83:0/2236076429 >> v2:172.21.15.167:3300/0 conn(0x7f39bc150b40 0x7f39bc15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:27.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.704+0000 7f39c3572640 1 --2- 172.21.15.83:0/2236076429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39bc150760 0x7f39bc0b39b0 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:27.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.704+0000 7f39c3572640 1 -- 172.21.15.83:0/2236076429 >> 172.21.15.83:0/2236076429 conn(0x7f39bc06abe0 msgr2=0x7f39bc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:27.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.708+0000 7f39c3572640 1 -- 172.21.15.83:0/2236076429 shutdown_connections 2024-04-17T10:50:27.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.712+0000 7f39c3572640 1 -- 172.21.15.83:0/2236076429 wait complete. 2024-04-17T10:50:27.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.716+0000 7f39c3572640 1 Processor -- start 2024-04-17T10:50:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.716+0000 7f39c3572640 1 -- start start 2024-04-17T10:50:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.716+0000 7f39c3572640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39bc150760 0x7f39bc14d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.716+0000 7f39c3572640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39bc150b40 0x7f39bc0b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.716+0000 7f39c3572640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39bc15da00 0x7f39bc147210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.716+0000 7f39c3572640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc06c940 con 0x7f39bc150b40 2024-04-17T10:50:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.716+0000 7f39c3572640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc06c7c0 con 0x7f39bc150760 2024-04-17T10:50:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.716+0000 7f39c3572640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc06cac0 con 0x7f39bc15da00 2024-04-17T10:50:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.716+0000 7f39c1d6f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39bc150b40 0x7f39bc0b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.716+0000 7f39c1d6f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39bc150b40 0x7f39bc0b0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44816/0 (socket says 172.21.15.83:44816) 2024-04-17T10:50:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.716+0000 7f39c1d6f640 1 -- 172.21.15.83:0/688157656 learned_addr learned my addr 172.21.15.83:0/688157656 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.716+0000 7f39c2570640 1 --2- 172.21.15.83:0/688157656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39bc150760 0x7f39bc14d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.716+0000 7f39c2d71640 1 --2- 172.21.15.83:0/688157656 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39bc15da00 0x7f39bc147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.716+0000 7f39c2d71640 1 -- 172.21.15.83:0/688157656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39bc150760 msgr2=0x7f39bc14d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.716+0000 7f39c2d71640 1 --2- 172.21.15.83:0/688157656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39bc150760 0x7f39bc14d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.716+0000 7f39c2d71640 1 -- 172.21.15.83:0/688157656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39bc150b40 msgr2=0x7f39bc0b0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.716+0000 7f39c2d71640 1 --2- 172.21.15.83:0/688157656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39bc150b40 0x7f39bc0b0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.716+0000 7f39c2d71640 1 -- 172.21.15.83:0/688157656 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39bc1fe9d0 con 0x7f39bc15da00 2024-04-17T10:50:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.716+0000 7f39c2570640 1 --2- 172.21.15.83:0/688157656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39bc150760 0x7f39bc14d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:50:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.720+0000 7f39c2d71640 1 --2- 172.21.15.83:0/688157656 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39bc15da00 0x7f39bc147210 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f39b400c730 tx=0x7f39b400cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.720+0000 7f39b2ffd640 1 -- 172.21.15.83:0/688157656 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b4016020 con 0x7f39bc15da00 2024-04-17T10:50:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.720+0000 7f39b2ffd640 1 -- 172.21.15.83:0/688157656 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b4006d60 con 0x7f39bc15da00 2024-04-17T10:50:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.720+0000 7f39b2ffd640 1 -- 172.21.15.83:0/688157656 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b4003bc0 con 0x7f39bc15da00 2024-04-17T10:50:27.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.720+0000 7f39c3572640 1 -- 172.21.15.83:0/688157656 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39bc1fecb0 con 0x7f39bc15da00 2024-04-17T10:50:27.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.720+0000 7f39c3572640 1 -- 172.21.15.83:0/688157656 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39bc1ff270 con 0x7f39bc15da00 2024-04-17T10:50:27.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.720+0000 7f39c3572640 1 -- 172.21.15.83:0/688157656 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39bc070cf0 con 0x7f39bc15da00 2024-04-17T10:50:27.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.720+0000 7f39b2ffd640 1 -- 172.21.15.83:0/688157656 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f39b40050d0 con 0x7f39bc15da00 2024-04-17T10:50:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.720+0000 7f39b2ffd640 1 --2- 172.21.15.83:0/688157656 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f39a0066f90 0x7f39a0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.720+0000 7f39c2570640 1 --2- 172.21.15.83:0/688157656 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f39a0066f90 0x7f39a0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.720+0000 7f39b2ffd640 1 -- 172.21.15.83:0/688157656 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f39b40847e0 con 0x7f39bc15da00 2024-04-17T10:50:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.724+0000 7f39c2570640 1 --2- 172.21.15.83:0/688157656 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f39a0066f90 0x7f39a0069450 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f39b80af000 tx=0x7f39b80a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:27.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.728+0000 7f39b2ffd640 1 -- 172.21.15.83:0/688157656 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f39b4020020 con 0x7f39bc15da00 2024-04-17T10:50:27.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.884+0000 7f39c3572640 1 -- 172.21.15.83:0/688157656 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f39bc06d8e0 con 0x7f39bc15da00 2024-04-17T10:50:27.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.896+0000 7f39b2ffd640 1 -- 172.21.15.83:0/688157656 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f39b4055430 con 0x7f39bc15da00 2024-04-17T10:50:27.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.896+0000 7f39c3572640 1 -- 172.21.15.83:0/688157656 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f39a0066f90 msgr2=0x7f39a0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:27.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.896+0000 7f39c3572640 1 --2- 172.21.15.83:0/688157656 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f39a0066f90 0x7f39a0069450 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f39b80af000 tx=0x7f39b80a7070 comp rx=0 tx=0).stop 2024-04-17T10:50:27.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.896+0000 7f39c3572640 1 -- 172.21.15.83:0/688157656 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39bc15da00 msgr2=0x7f39bc147210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:27.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.896+0000 7f39c3572640 1 --2- 172.21.15.83:0/688157656 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39bc15da00 0x7f39bc147210 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f39b400c730 tx=0x7f39b400cbf0 comp rx=0 tx=0).stop 2024-04-17T10:50:27.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.896+0000 7f39c3572640 1 -- 172.21.15.83:0/688157656 shutdown_connections 2024-04-17T10:50:27.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.896+0000 7f39c3572640 1 --2- 172.21.15.83:0/688157656 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f39a0066f90 0x7f39a0069450 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:27.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.896+0000 7f39c3572640 1 --2- 172.21.15.83:0/688157656 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39bc15da00 0x7f39bc147210 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:27.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.896+0000 7f39c3572640 1 --2- 172.21.15.83:0/688157656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39bc150b40 0x7f39bc0b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:27.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.896+0000 7f39c3572640 1 --2- 172.21.15.83:0/688157656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39bc150760 0x7f39bc14d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:27.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.896+0000 7f39c3572640 1 -- 172.21.15.83:0/688157656 >> 172.21.15.83:0/688157656 conn(0x7f39bc06abe0 msgr2=0x7f39bc149540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:27.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.896+0000 7f39c3572640 1 -- 172.21.15.83:0/688157656 shutdown_connections 2024-04-17T10:50:27.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:27.896+0000 7f39c3572640 1 -- 172.21.15.83:0/688157656 wait complete. 2024-04-17T10:50:27.907 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover' 2024-04-17T10:50:28.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82cbb23640 1 Processor -- start 2024-04-17T10:50:28.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82cbb23640 1 -- start start 2024-04-17T10:50:28.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82cbb23640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82c4146e70 0x7f82c406d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:28.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82cbb23640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c4147570 con 0x7f82c4146a00 2024-04-17T10:50:28.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82cbb23640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c406cbb0 con 0x7f82c40b2d80 2024-04-17T10:50:28.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82cbb23640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c406c960 con 0x7f82c4146e70 2024-04-17T10:50:28.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82c98bf640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f82c4146a00 0x7f82c40b2670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56252/0 (socket says 172.21.15.83:56252) 2024-04-17T10:50:28.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82c98bf640 1 -- 172.21.15.83:0/490934264 learned_addr learned my addr 172.21.15.83:0/490934264 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:28.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82ca0c0640 1 --2- 172.21.15.83:0/490934264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82c4146e70 0x7f82c406d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:28.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82c88bd640 1 -- 172.21.15.83:0/490934264 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1149704794 0 0) 0x7f82c4147570 con 0x7f82c4146a00 2024-04-17T10:50:28.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82c88bd640 1 -- 172.21.15.83:0/490934264 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82b0003530 con 0x7f82c4146a00 2024-04-17T10:50:28.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82c88bd640 1 -- 172.21.15.83:0/490934264 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1196881320 0 0) 0x7f82b0003530 con 0x7f82c4146a00 2024-04-17T10:50:28.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82c88bd640 1 -- 172.21.15.83:0/490934264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82c4146e70 msgr2=0x7f82c406d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:28.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82c88bd640 1 --2- 172.21.15.83:0/490934264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82c4146e70 0x7f82c406d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:28.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82c88bd640 1 -- 172.21.15.83:0/490934264 >> v1:172.21.15.167:6789/0 conn(0x7f82c40b2d80 legacy=0x7f82c4070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:28.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82c88bd640 1 -- 172.21.15.83:0/490934264 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c406ce90 con 0x7f82c4146a00 2024-04-17T10:50:28.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82c88bd640 1 -- 172.21.15.83:0/490934264 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f82c0001a00 con 0x7f82c4146a00 2024-04-17T10:50:28.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82c88bd640 1 -- 172.21.15.83:0/490934264 >> v1:172.21.15.83:6789/0 conn(0x7f82c4146a00 legacy=0x7f82c40b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:28.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82c88bd640 1 --2- 172.21.15.83:0/490934264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82b00039f0 0x7f82b0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:28.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82c88bd640 1 --2- 172.21.15.83:0/490934264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82b0004810 0x7f82b0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:28.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82c88bd640 1 --2- 172.21.15.83:0/490934264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82c4146e70 0x7f82b00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:28.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82c88bd640 1 -- 172.21.15.83:0/490934264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c406c960 con 0x7f82b00039f0 2024-04-17T10:50:28.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82c88bd640 1 -- 172.21.15.83:0/490934264 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b000bab0 con 0x7f82b0004810 2024-04-17T10:50:28.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82c88bd640 1 -- 172.21.15.83:0/490934264 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b000bcb0 con 0x7f82c4146e70 2024-04-17T10:50:28.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82c98bf640 1 --2- 172.21.15.83:0/490934264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82b00039f0 0x7f82b0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:28.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82ca0c0640 1 --2- 172.21.15.83:0/490934264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82c4146e70 0x7f82b00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:28.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82c90be640 1 --2- 172.21.15.83:0/490934264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82b0004810 0x7f82b0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:28.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82c98bf640 1 -- 172.21.15.83:0/490934264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82c4146e70 msgr2=0x7f82b00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:28.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82c98bf640 1 --2- 172.21.15.83:0/490934264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82c4146e70 0x7f82b00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:28.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82c98bf640 1 -- 172.21.15.83:0/490934264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82b0004810 msgr2=0x7f82b0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:28.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82c98bf640 1 --2- 172.21.15.83:0/490934264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82b0004810 0x7f82b0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:28.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82c98bf640 1 -- 172.21.15.83:0/490934264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b0003530 con 0x7f82b00039f0 2024-04-17T10:50:28.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82ca0c0640 1 --2- 172.21.15.83:0/490934264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82c4146e70 0x7f82b00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:28.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.032+0000 7f82c90be640 1 --2- 172.21.15.83:0/490934264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82b0004810 0x7f82b0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:28.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.036+0000 7f82c98bf640 1 --2- 172.21.15.83:0/490934264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82b00039f0 0x7f82b0003e10 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f82c000a810 tx=0x7f82c001d320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9759ea20c4744c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:28.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.036+0000 7f82c88bd640 1 -- 172.21.15.83:0/490934264 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c000a460 con 0x7f82b00039f0 2024-04-17T10:50:28.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.036+0000 7f82c88bd640 1 -- 172.21.15.83:0/490934264 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c00ad650 con 0x7f82b00039f0 2024-04-17T10:50:28.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.036+0000 7f82c88bd640 1 -- 172.21.15.83:0/490934264 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c00ada40 con 0x7f82b00039f0 2024-04-17T10:50:28.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.036+0000 7f82cbb23640 1 -- 172.21.15.83:0/490934264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82b00039f0 msgr2=0x7f82b0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:28.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.036+0000 7f82cbb23640 1 --2- 172.21.15.83:0/490934264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82b00039f0 0x7f82b0003e10 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f82c000a810 tx=0x7f82c001d320 comp rx=0 tx=0).stop 2024-04-17T10:50:28.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.036+0000 7f82ca0c0640 1 -- 172.21.15.83:0/490934264 reap_dead start 2024-04-17T10:50:28.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.036+0000 7f82cbb23640 1 -- 172.21.15.83:0/490934264 shutdown_connections 2024-04-17T10:50:28.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.036+0000 7f82cbb23640 1 -- 172.21.15.83:0/490934264 >> 172.21.15.83:0/490934264 conn(0x7f82c406abe0 msgr2=0x7f82c406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:28.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.036+0000 7f82cbb23640 1 -- 172.21.15.83:0/490934264 shutdown_connections 2024-04-17T10:50:28.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.036+0000 7f82cbb23640 1 -- 172.21.15.83:0/490934264 wait complete. 2024-04-17T10:50:28.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.036+0000 7f82cbb23640 1 Processor -- start 2024-04-17T10:50:28.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.036+0000 7f82cbb23640 1 -- start start 2024-04-17T10:50:28.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.040+0000 7f82cbb23640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82b00039f0 0x7f82c413d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:28.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.040+0000 7f82cbb23640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82c415ccb0 0x7f82c415c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:28.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.040+0000 7f82cbb23640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82c415d1e0 0x7f82c41f6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:28.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.040+0000 7f82cbb23640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c4070d00 con 0x7f82b00039f0 2024-04-17T10:50:28.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.040+0000 7f82cbb23640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c4070b80 con 0x7f82c415ccb0 2024-04-17T10:50:28.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.040+0000 7f82cbb23640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c4070e80 con 0x7f82c415d1e0 2024-04-17T10:50:28.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.040+0000 7f82c98bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82b00039f0 0x7f82c413d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:28.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.040+0000 7f82c98bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82b00039f0 0x7f82c413d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44826/0 (socket says 172.21.15.83:44826) 2024-04-17T10:50:28.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.040+0000 7f82c98bf640 1 -- 172.21.15.83:0/106277515 learned_addr learned my addr 172.21.15.83:0/106277515 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:28.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.040+0000 7f82c90be640 1 --2- 172.21.15.83:0/106277515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82c415ccb0 0x7f82c415c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:28.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.040+0000 7f82ca0c0640 1 --2- 172.21.15.83:0/106277515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82c415d1e0 0x7f82c41f6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:28.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.040+0000 7f82c98bf640 1 -- 172.21.15.83:0/106277515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82c415d1e0 msgr2=0x7f82c41f6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:28.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.040+0000 7f82c98bf640 1 --2- 172.21.15.83:0/106277515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82c415d1e0 0x7f82c41f6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:28.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.040+0000 7f82c98bf640 1 -- 172.21.15.83:0/106277515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82c415ccb0 msgr2=0x7f82c415c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:28.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.040+0000 7f82c98bf640 1 --2- 172.21.15.83:0/106277515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82c415ccb0 0x7f82c415c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:28.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.040+0000 7f82c98bf640 1 -- 172.21.15.83:0/106277515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c415caa0 con 0x7f82b00039f0 2024-04-17T10:50:28.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.040+0000 7f82ca0c0640 1 --2- 172.21.15.83:0/106277515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82c415d1e0 0x7f82c41f6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:28.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.040+0000 7f82c90be640 1 --2- 172.21.15.83:0/106277515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82c415ccb0 0x7f82c415c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:28.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.040+0000 7f82c98bf640 1 --2- 172.21.15.83:0/106277515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82b00039f0 0x7f82c413d6e0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f82c001d150 tx=0x7f82b0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:28.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.040+0000 7f82ba7fc640 1 -- 172.21.15.83:0/106277515 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c00ad430 con 0x7f82b00039f0 2024-04-17T10:50:28.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.040+0000 7f82ba7fc640 1 -- 172.21.15.83:0/106277515 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c00017c0 con 0x7f82b00039f0 2024-04-17T10:50:28.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.040+0000 7f82ba7fc640 1 -- 172.21.15.83:0/106277515 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c0004b70 con 0x7f82b00039f0 2024-04-17T10:50:28.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.040+0000 7f82cbb23640 1 -- 172.21.15.83:0/106277515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c40b1dd0 con 0x7f82b00039f0 2024-04-17T10:50:28.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.040+0000 7f82cbb23640 1 -- 172.21.15.83:0/106277515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c41f7fb0 con 0x7f82b00039f0 2024-04-17T10:50:28.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.040+0000 7f82cbb23640 1 -- 172.21.15.83:0/106277515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82c4074f90 con 0x7f82b00039f0 2024-04-17T10:50:28.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.040+0000 7f82ba7fc640 1 -- 172.21.15.83:0/106277515 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f82c00b5070 con 0x7f82b00039f0 2024-04-17T10:50:28.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.040+0000 7f82ba7fc640 1 --2- 172.21.15.83:0/106277515 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f82a8066f90 0x7f82a8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:28.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.040+0000 7f82ba7fc640 1 -- 172.21.15.83:0/106277515 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f82c011eee0 con 0x7f82b00039f0 2024-04-17T10:50:28.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.040+0000 7f82c90be640 1 --2- 172.21.15.83:0/106277515 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f82a8066f90 0x7f82a8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:28.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.040+0000 7f82c90be640 1 --2- 172.21.15.83:0/106277515 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f82a8066f90 0x7f82a8069450 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f82b4000c10 tx=0x7f82b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:28.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.044+0000 7f82ba7fc640 1 -- 172.21.15.83:0/106277515 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f82c0017030 con 0x7f82b00039f0 2024-04-17T10:50:28.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.264+0000 7f82cbb23640 1 -- 172.21.15.83:0/106277515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]} v 0) v1 -- 0x7f82c406b240 con 0x7f82b00039f0 2024-04-17T10:50:28.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.532+0000 7f82ba7fc640 1 -- 172.21.15.83:0/106277515 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]}]=0 v550) v1 ==== 166+0+0 (secure 0 0 0) 0x7f82c00f0190 con 0x7f82b00039f0 2024-04-17T10:50:28.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.536+0000 7f82cbb23640 1 -- 172.21.15.83:0/106277515 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f82a8066f90 msgr2=0x7f82a8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:28.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.536+0000 7f82cbb23640 1 --2- 172.21.15.83:0/106277515 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f82a8066f90 0x7f82a8069450 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f82b4000c10 tx=0x7f82b4009040 comp rx=0 tx=0).stop 2024-04-17T10:50:28.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.536+0000 7f82cbb23640 1 -- 172.21.15.83:0/106277515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82b00039f0 msgr2=0x7f82c413d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:28.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.536+0000 7f82cbb23640 1 --2- 172.21.15.83:0/106277515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82b00039f0 0x7f82c413d6e0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f82c001d150 tx=0x7f82b0005840 comp rx=0 tx=0).stop 2024-04-17T10:50:28.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.536+0000 7f82cbb23640 1 -- 172.21.15.83:0/106277515 shutdown_connections 2024-04-17T10:50:28.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.536+0000 7f82cbb23640 1 --2- 172.21.15.83:0/106277515 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f82a8066f90 0x7f82a8069450 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:28.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.536+0000 7f82cbb23640 1 --2- 172.21.15.83:0/106277515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82c415d1e0 0x7f82c41f6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:28.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.536+0000 7f82cbb23640 1 --2- 172.21.15.83:0/106277515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82c415ccb0 0x7f82c415c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:28.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.536+0000 7f82cbb23640 1 --2- 172.21.15.83:0/106277515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82b00039f0 0x7f82c413d6e0 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:28.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.536+0000 7f82cbb23640 1 -- 172.21.15.83:0/106277515 >> 172.21.15.83:0/106277515 conn(0x7f82c406abe0 msgr2=0x7f82c41454c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:28.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.536+0000 7f82cbb23640 1 -- 172.21.15.83:0/106277515 shutdown_connections 2024-04-17T10:50:28.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.536+0000 7f82cbb23640 1 -- 172.21.15.83:0/106277515 wait complete. 2024-04-17T10:50:28.548 INFO:tasks.cephfs_test_runner:test_scrub_task_status_on_mds_failover (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-17T10:50:28.548 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-17T10:50:28.548 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:50:28.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.676+0000 7f3a31e87640 1 Processor -- start 2024-04-17T10:50:28.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.676+0000 7f3a31e87640 1 -- start start 2024-04-17T10:50:28.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.676+0000 7f3a31e87640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a2c15db50 0x7f3a2c070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:28.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.680+0000 7f3a31e87640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3a2c06dcb0 0x7f3a2c0b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:28.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.680+0000 7f3a31e87640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a2c0709b0 con 0x7f3a2c0aedd0 2024-04-17T10:50:28.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.680+0000 7f3a31e87640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a2c15e440 con 0x7f3a2c06dcb0 2024-04-17T10:50:28.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.680+0000 7f3a31e87640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c15e720 con 0x7f3a2c15db50 2024-04-17T10:50:28.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.680+0000 7f3a2bfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3a2c06dcb0 0x7f3a2c0b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:28.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.680+0000 7f3a2bfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3a2c06dcb0 0x7f3a2c0b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44832/0 (socket says 172.21.15.83:44832) 2024-04-17T10:50:28.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.680+0000 7f3a2bfff640 1 -- 172.21.15.83:0/1692532323 learned_addr learned my addr 172.21.15.83:0/1692532323 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:28.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.680+0000 7f3a2b7fe640 1 --2- 172.21.15.83:0/1692532323 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a2c15db50 0x7f3a2c070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:28.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.680+0000 7f3a2bfff640 1 -- 172.21.15.83:0/1692532323 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a2c15db50 msgr2=0x7f3a2c070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:28.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.680+0000 7f3a2bfff640 1 --2- 172.21.15.83:0/1692532323 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a2c15db50 0x7f3a2c070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:28.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.680+0000 7f3a2bfff640 1 -- 172.21.15.83:0/1692532323 >> v1:172.21.15.167:6789/0 conn(0x7f3a2c0aedd0 legacy=0x7f3a2c06d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:28.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.680+0000 7f3a2bfff640 1 -- 172.21.15.83:0/1692532323 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2c15ea00 con 0x7f3a2c06dcb0 2024-04-17T10:50:28.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.680+0000 7f3a2bfff640 1 --2- 172.21.15.83:0/1692532323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a2c06dcb0 0x7f3a2c0b2e10 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f3a24009e50 tx=0x7f3a2401c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce80674ae5fca0fd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:28.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.680+0000 7f3a2a7fc640 1 -- 172.21.15.83:0/1692532323 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a240a5660 con 0x7f3a2c06dcb0 2024-04-17T10:50:28.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.680+0000 7f3a2a7fc640 1 -- 172.21.15.83:0/1692532323 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a240a5800 con 0x7f3a2c06dcb0 2024-04-17T10:50:28.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.680+0000 7f3a2a7fc640 1 -- 172.21.15.83:0/1692532323 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a240a5bd0 con 0x7f3a2c06dcb0 2024-04-17T10:50:28.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.680+0000 7f3a31e87640 1 -- 172.21.15.83:0/1692532323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a2c06dcb0 msgr2=0x7f3a2c0b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:28.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.680+0000 7f3a31e87640 1 --2- 172.21.15.83:0/1692532323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a2c06dcb0 0x7f3a2c0b2e10 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f3a24009e50 tx=0x7f3a2401c7e0 comp rx=0 tx=0).stop 2024-04-17T10:50:28.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.680+0000 7f3a31e87640 1 -- 172.21.15.83:0/1692532323 shutdown_connections 2024-04-17T10:50:28.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.680+0000 7f3a31e87640 1 --2- 172.21.15.83:0/1692532323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a2c06dcb0 0x7f3a2c0b2e10 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:28.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.680+0000 7f3a31e87640 1 --2- 172.21.15.83:0/1692532323 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a2c15db50 0x7f3a2c070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:28.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.680+0000 7f3a31e87640 1 -- 172.21.15.83:0/1692532323 >> 172.21.15.83:0/1692532323 conn(0x7f3a2c06abe0 msgr2=0x7f3a2c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:28.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.680+0000 7f3a31e87640 1 -- 172.21.15.83:0/1692532323 shutdown_connections 2024-04-17T10:50:28.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.680+0000 7f3a31e87640 1 -- 172.21.15.83:0/1692532323 wait complete. 2024-04-17T10:50:28.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.684+0000 7f3a31e87640 1 Processor -- start 2024-04-17T10:50:28.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.684+0000 7f3a31e87640 1 -- start start 2024-04-17T10:50:28.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.684+0000 7f3a31e87640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a2c06dcb0 0x7f3a2c1585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:28.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.684+0000 7f3a31e87640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a2c0aedd0 0x7f3a2c154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:28.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.684+0000 7f3a31e87640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a2c15db50 0x7f3a2c15c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:28.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.684+0000 7f3a31e87640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c0b3ce0 con 0x7f3a2c0aedd0 2024-04-17T10:50:28.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.684+0000 7f3a31e87640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c0b3b60 con 0x7f3a2c15db50 2024-04-17T10:50:28.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.684+0000 7f3a31e87640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c0b3e60 con 0x7f3a2c06dcb0 2024-04-17T10:50:28.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.684+0000 7f3a2affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a2c0aedd0 0x7f3a2c154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:28.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.684+0000 7f3a2affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a2c0aedd0 0x7f3a2c154090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44836/0 (socket says 172.21.15.83:44836) 2024-04-17T10:50:28.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.684+0000 7f3a2affd640 1 -- 172.21.15.83:0/1481484181 learned_addr learned my addr 172.21.15.83:0/1481484181 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:28.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.684+0000 7f3a2b7fe640 1 --2- 172.21.15.83:0/1481484181 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a2c06dcb0 0x7f3a2c1585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:28.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.684+0000 7f3a2bfff640 1 --2- 172.21.15.83:0/1481484181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a2c15db50 0x7f3a2c15c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:28.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.684+0000 7f3a2affd640 1 -- 172.21.15.83:0/1481484181 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a2c06dcb0 msgr2=0x7f3a2c1585e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:28.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.684+0000 7f3a2affd640 1 --2- 172.21.15.83:0/1481484181 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a2c06dcb0 0x7f3a2c1585e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:28.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.684+0000 7f3a2affd640 1 -- 172.21.15.83:0/1481484181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a2c15db50 msgr2=0x7f3a2c15c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:28.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.684+0000 7f3a2affd640 1 --2- 172.21.15.83:0/1481484181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a2c15db50 0x7f3a2c15c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:28.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.684+0000 7f3a2affd640 1 -- 172.21.15.83:0/1481484181 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2c0cd750 con 0x7f3a2c0aedd0 2024-04-17T10:50:28.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.684+0000 7f3a2b7fe640 1 --2- 172.21.15.83:0/1481484181 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a2c06dcb0 0x7f3a2c1585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:50:28.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.684+0000 7f3a2affd640 1 --2- 172.21.15.83:0/1481484181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a2c0aedd0 0x7f3a2c154090 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f3a2000c450 tx=0x7f3a2000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:28.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.684+0000 7f3a13fff640 1 -- 172.21.15.83:0/1481484181 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a20012830 con 0x7f3a2c0aedd0 2024-04-17T10:50:28.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.684+0000 7f3a13fff640 1 -- 172.21.15.83:0/1481484181 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a20017070 con 0x7f3a2c0aedd0 2024-04-17T10:50:28.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.684+0000 7f3a31e87640 1 -- 172.21.15.83:0/1481484181 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a2c158c50 con 0x7f3a2c0aedd0 2024-04-17T10:50:28.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.684+0000 7f3a13fff640 1 -- 172.21.15.83:0/1481484181 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a20012c30 con 0x7f3a2c0aedd0 2024-04-17T10:50:28.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.684+0000 7f3a31e87640 1 -- 172.21.15.83:0/1481484181 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a2c159210 con 0x7f3a2c0aedd0 2024-04-17T10:50:28.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.684+0000 7f3a31e87640 1 -- 172.21.15.83:0/1481484181 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a2c074f90 con 0x7f3a2c0aedd0 2024-04-17T10:50:28.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.688+0000 7f3a13fff640 1 -- 172.21.15.83:0/1481484181 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3a2001c460 con 0x7f3a2c0aedd0 2024-04-17T10:50:28.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.688+0000 7f3a13fff640 1 --2- 172.21.15.83:0/1481484181 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3a18066f60 0x7f3a18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:28.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.688+0000 7f3a13fff640 1 -- 172.21.15.83:0/1481484181 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f3a20084bd0 con 0x7f3a2c0aedd0 2024-04-17T10:50:28.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.688+0000 7f3a2b7fe640 1 --2- 172.21.15.83:0/1481484181 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3a18066f60 0x7f3a18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:28.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.688+0000 7f3a2b7fe640 1 --2- 172.21.15.83:0/1481484181 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3a18066f60 0x7f3a18069420 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00b9f0 tx=0x7f3a1c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:28.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.692+0000 7f3a13fff640 1 -- 172.21.15.83:0/1481484181 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3a20055e80 con 0x7f3a2c0aedd0 2024-04-17T10:50:28.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.848+0000 7f3a31e87640 1 -- 172.21.15.83:0/1481484181 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a2c0b2af0 con 0x7f3a2c0aedd0 2024-04-17T10:50:28.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.848+0000 7f3a13fff640 1 -- 172.21.15.83:0/1481484181 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+13088 (secure 0 0 0) 0x7f3a20055780 con 0x7f3a2c0aedd0 2024-04-17T10:50:28.841 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:50:28.842 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":133,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:50:27.515550+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6835/3552333026":"2024-04-18T10:50:24.304885+0000","172.21.15.83:6834/3552333026":"2024-04-18T10:50:24.304885+0000","172.21.15.83:6837/1559222491":"2024-04-18T10:50:12.167626+0000","172.21.15.83:6836/1559222491":"2024-04-18T10:50:12.167626+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:50:28.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.852+0000 7f3a31e87640 1 -- 172.21.15.83:0/1481484181 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3a18066f60 msgr2=0x7f3a18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:28.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.852+0000 7f3a31e87640 1 --2- 172.21.15.83:0/1481484181 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3a18066f60 0x7f3a18069420 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00b9f0 tx=0x7f3a1c008040 comp rx=0 tx=0).stop 2024-04-17T10:50:28.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.852+0000 7f3a31e87640 1 -- 172.21.15.83:0/1481484181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a2c0aedd0 msgr2=0x7f3a2c154090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:28.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.852+0000 7f3a31e87640 1 --2- 172.21.15.83:0/1481484181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a2c0aedd0 0x7f3a2c154090 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f3a2000c450 tx=0x7f3a2000c910 comp rx=0 tx=0).stop 2024-04-17T10:50:28.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.852+0000 7f3a31e87640 1 -- 172.21.15.83:0/1481484181 shutdown_connections 2024-04-17T10:50:28.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.852+0000 7f3a31e87640 1 --2- 172.21.15.83:0/1481484181 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3a18066f60 0x7f3a18069420 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:28.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.852+0000 7f3a31e87640 1 --2- 172.21.15.83:0/1481484181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a2c15db50 0x7f3a2c15c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:28.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.852+0000 7f3a31e87640 1 --2- 172.21.15.83:0/1481484181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a2c0aedd0 0x7f3a2c154090 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:28.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.852+0000 7f3a31e87640 1 --2- 172.21.15.83:0/1481484181 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a2c06dcb0 0x7f3a2c1585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:28.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.852+0000 7f3a31e87640 1 -- 172.21.15.83:0/1481484181 >> 172.21.15.83:0/1481484181 conn(0x7f3a2c06abe0 msgr2=0x7f3a2c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:28.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.852+0000 7f3a31e87640 1 -- 172.21.15.83:0/1481484181 shutdown_connections 2024-04-17T10:50:28.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.852+0000 7f3a31e87640 1 -- 172.21.15.83:0/1481484181 wait complete. 2024-04-17T10:50:28.860 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:50:28.860 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:50:28.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.992+0000 7f9a4e43d640 1 Processor -- start 2024-04-17T10:50:28.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.992+0000 7f9a4e43d640 1 -- start start 2024-04-17T10:50:28.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.992+0000 7f9a4e43d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a4815e330 0x7f9a4815e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:28.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.992+0000 7f9a4e43d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9a48070aa0 0x7f9a48070f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:28.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.992+0000 7f9a4e43d640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a481625d0 con 0x7f9a4806ae10 2024-04-17T10:50:28.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.992+0000 7f9a4e43d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a48161810 con 0x7f9a48070aa0 2024-04-17T10:50:28.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.992+0000 7f9a4e43d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a48161af0 con 0x7f9a4815e330 2024-04-17T10:50:28.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.992+0000 7f9a4c9da640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9a48070aa0 0x7f9a48070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:28.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.992+0000 7f9a4c9da640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9a48070aa0 0x7f9a48070f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44852/0 (socket says 172.21.15.83:44852) 2024-04-17T10:50:28.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.992+0000 7f9a4c9da640 1 -- 172.21.15.83:0/1790901121 learned_addr learned my addr 172.21.15.83:0/1790901121 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:28.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.992+0000 7f9a3f7fe640 1 --2- 172.21.15.83:0/1790901121 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a4815e330 0x7f9a4815e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:28.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.992+0000 7f9a4c9da640 1 -- 172.21.15.83:0/1790901121 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a4815e330 msgr2=0x7f9a4815e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:28.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.992+0000 7f9a4c9da640 1 --2- 172.21.15.83:0/1790901121 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a4815e330 0x7f9a4815e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:28.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.992+0000 7f9a4c9da640 1 -- 172.21.15.83:0/1790901121 >> v1:172.21.15.167:6789/0 conn(0x7f9a4806ae10 legacy=0x7f9a4806b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:28.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.992+0000 7f9a4c9da640 1 -- 172.21.15.83:0/1790901121 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a48161dd0 con 0x7f9a48070aa0 2024-04-17T10:50:28.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.992+0000 7f9a4c9da640 1 --2- 172.21.15.83:0/1790901121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a48070aa0 0x7f9a48070f30 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f9a40009e50 tx=0x7f9a4001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c868da3b95fee5c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:28.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a477fe640 1 -- 172.21.15.83:0/1790901121 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a400a56b0 con 0x7f9a48070aa0 2024-04-17T10:50:28.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a477fe640 1 -- 172.21.15.83:0/1790901121 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a400a5850 con 0x7f9a48070aa0 2024-04-17T10:50:28.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a477fe640 1 -- 172.21.15.83:0/1790901121 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a400a5c20 con 0x7f9a48070aa0 2024-04-17T10:50:28.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a4e43d640 1 -- 172.21.15.83:0/1790901121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a48070aa0 msgr2=0x7f9a48070f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:28.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a4e43d640 1 --2- 172.21.15.83:0/1790901121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a48070aa0 0x7f9a48070f30 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f9a40009e50 tx=0x7f9a4001c8e0 comp rx=0 tx=0).stop 2024-04-17T10:50:28.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a4e43d640 1 -- 172.21.15.83:0/1790901121 shutdown_connections 2024-04-17T10:50:28.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a4e43d640 1 --2- 172.21.15.83:0/1790901121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a48070aa0 0x7f9a48070f30 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:28.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a4e43d640 1 --2- 172.21.15.83:0/1790901121 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a4815e330 0x7f9a4815e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:28.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a4e43d640 1 -- 172.21.15.83:0/1790901121 >> 172.21.15.83:0/1790901121 conn(0x7f9a480b15b0 msgr2=0x7f9a480b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:28.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a4e43d640 1 -- 172.21.15.83:0/1790901121 shutdown_connections 2024-04-17T10:50:28.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a4e43d640 1 -- 172.21.15.83:0/1790901121 wait complete. 2024-04-17T10:50:28.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a4e43d640 1 Processor -- start 2024-04-17T10:50:28.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a4e43d640 1 -- start start 2024-04-17T10:50:28.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a4e43d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a4806ae10 0x7f9a4815bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a4e43d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a48070aa0 0x7f9a481409c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a4e43d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a4815e330 0x7f9a48140f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a4e43d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4816aa10 con 0x7f9a4815e330 2024-04-17T10:50:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a4e43d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4815eec0 con 0x7f9a4806ae10 2024-04-17T10:50:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a4e43d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4816ab90 con 0x7f9a48070aa0 2024-04-17T10:50:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a4c9da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a4815e330 0x7f9a48140f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a4c9da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a4815e330 0x7f9a48140f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44866/0 (socket says 172.21.15.83:44866) 2024-04-17T10:50:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a4c9da640 1 -- 172.21.15.83:0/3838740658 learned_addr learned my addr 172.21.15.83:0/3838740658 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a3f7fe640 1 --2- 172.21.15.83:0/3838740658 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a48070aa0 0x7f9a481409c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a47fff640 1 --2- 172.21.15.83:0/3838740658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a4806ae10 0x7f9a4815bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a4c9da640 1 -- 172.21.15.83:0/3838740658 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a48070aa0 msgr2=0x7f9a481409c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a4c9da640 1 --2- 172.21.15.83:0/3838740658 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a48070aa0 0x7f9a481409c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a4c9da640 1 -- 172.21.15.83:0/3838740658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a4806ae10 msgr2=0x7f9a4815bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a4c9da640 1 --2- 172.21.15.83:0/3838740658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a4806ae10 0x7f9a4815bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a4c9da640 1 -- 172.21.15.83:0/3838740658 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a48160ad0 con 0x7f9a4815e330 2024-04-17T10:50:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a47fff640 1 --2- 172.21.15.83:0/3838740658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a4806ae10 0x7f9a4815bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a4c9da640 1 --2- 172.21.15.83:0/3838740658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a4815e330 0x7f9a48140f00 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f9a4001a510 tx=0x7f9a40019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a457fa640 1 -- 172.21.15.83:0/3838740658 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a400bc020 con 0x7f9a4815e330 2024-04-17T10:50:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a457fa640 1 -- 172.21.15.83:0/3838740658 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a400b8670 con 0x7f9a4815e330 2024-04-17T10:50:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a4e43d640 1 -- 172.21.15.83:0/3838740658 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a48160db0 con 0x7f9a4815e330 2024-04-17T10:50:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a457fa640 1 -- 172.21.15.83:0/3838740658 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a400b8a60 con 0x7f9a4815e330 2024-04-17T10:50:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:28.996+0000 7f9a4e43d640 1 -- 172.21.15.83:0/3838740658 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a48159a30 con 0x7f9a4815e330 2024-04-17T10:50:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.000+0000 7f9a4e43d640 1 -- 172.21.15.83:0/3838740658 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a48074f90 con 0x7f9a4815e330 2024-04-17T10:50:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.000+0000 7f9a457fa640 1 -- 172.21.15.83:0/3838740658 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9a400bf020 con 0x7f9a4815e330 2024-04-17T10:50:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.000+0000 7f9a457fa640 1 --2- 172.21.15.83:0/3838740658 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9a28066f60 0x7f9a28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.000+0000 7f9a47fff640 1 --2- 172.21.15.83:0/3838740658 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9a28066f60 0x7f9a28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.000+0000 7f9a457fa640 1 -- 172.21.15.83:0/3838740658 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f9a401207a0 con 0x7f9a4815e330 2024-04-17T10:50:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.000+0000 7f9a47fff640 1 --2- 172.21.15.83:0/3838740658 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9a28066f60 0x7f9a28069420 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f9a380016f0 tx=0x7f9a38009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:28.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.004+0000 7f9a457fa640 1 -- 172.21.15.83:0/3838740658 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9a40016620 con 0x7f9a4815e330 2024-04-17T10:50:29.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.148+0000 7f9a4e43d640 1 -- 172.21.15.83:0/3838740658 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9a48072740 con 0x7f9a4815e330 2024-04-17T10:50:29.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.152+0000 7f9a457fa640 1 -- 172.21.15.83:0/3838740658 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v133) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9a400f1da0 con 0x7f9a4815e330 2024-04-17T10:50:29.144 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:50:29.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.152+0000 7f9a4e43d640 1 -- 172.21.15.83:0/3838740658 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9a28066f60 msgr2=0x7f9a28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:29.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.152+0000 7f9a4e43d640 1 --2- 172.21.15.83:0/3838740658 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9a28066f60 0x7f9a28069420 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f9a380016f0 tx=0x7f9a38009040 comp rx=0 tx=0).stop 2024-04-17T10:50:29.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.152+0000 7f9a4e43d640 1 -- 172.21.15.83:0/3838740658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a4815e330 msgr2=0x7f9a48140f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:29.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.152+0000 7f9a4e43d640 1 --2- 172.21.15.83:0/3838740658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a4815e330 0x7f9a48140f00 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f9a4001a510 tx=0x7f9a40019680 comp rx=0 tx=0).stop 2024-04-17T10:50:29.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.152+0000 7f9a4e43d640 1 -- 172.21.15.83:0/3838740658 shutdown_connections 2024-04-17T10:50:29.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.152+0000 7f9a4e43d640 1 --2- 172.21.15.83:0/3838740658 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9a28066f60 0x7f9a28069420 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:29.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.152+0000 7f9a4e43d640 1 --2- 172.21.15.83:0/3838740658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a4815e330 0x7f9a48140f00 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:29.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.152+0000 7f9a4e43d640 1 --2- 172.21.15.83:0/3838740658 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a48070aa0 0x7f9a481409c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:29.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.152+0000 7f9a4e43d640 1 --2- 172.21.15.83:0/3838740658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a4806ae10 0x7f9a4815bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:29.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.152+0000 7f9a4e43d640 1 -- 172.21.15.83:0/3838740658 >> 172.21.15.83:0/3838740658 conn(0x7f9a480b15b0 msgr2=0x7f9a480af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:29.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.152+0000 7f9a4e43d640 1 -- 172.21.15.83:0/3838740658 shutdown_connections 2024-04-17T10:50:29.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.152+0000 7f9a4e43d640 1 -- 172.21.15.83:0/3838740658 wait complete. 2024-04-17T10:50:29.161 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive' 2024-04-17T10:50:29.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.288+0000 7f1d2c022640 1 Processor -- start 2024-04-17T10:50:29.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.288+0000 7f1d2c022640 1 -- start start 2024-04-17T10:50:29.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.288+0000 7f1d2c022640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1d2415dfa0 0x7f1d240b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:29.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.288+0000 7f1d2c022640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1d240ce980 0x7f1d240b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:29.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.288+0000 7f1d2c022640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d24162a00 con 0x7f1d240cee30 2024-04-17T10:50:29.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.288+0000 7f1d2c022640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d24161e80 con 0x7f1d2415dfa0 2024-04-17T10:50:29.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.288+0000 7f1d2c022640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d24162160 con 0x7f1d240ce980 2024-04-17T10:50:29.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.288+0000 7f1d29dbe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1d2415dfa0 0x7f1d240b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:29.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.288+0000 7f1d29dbe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1d2415dfa0 0x7f1d240b6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44880/0 (socket says 172.21.15.83:44880) 2024-04-17T10:50:29.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.288+0000 7f1d29dbe640 1 -- 172.21.15.83:0/3960205336 learned_addr learned my addr 172.21.15.83:0/3960205336 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:29.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.288+0000 7f1d295bd640 1 --2- 172.21.15.83:0/3960205336 >> v2:172.21.15.167:3300/0 conn(0x7f1d240ce980 0x7f1d240b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:29.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.288+0000 7f1d29dbe640 1 -- 172.21.15.83:0/3960205336 >> v2:172.21.15.167:3300/0 conn(0x7f1d240ce980 msgr2=0x7f1d240b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:29.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.288+0000 7f1d29dbe640 1 --2- 172.21.15.83:0/3960205336 >> v2:172.21.15.167:3300/0 conn(0x7f1d240ce980 0x7f1d240b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:29.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.288+0000 7f1d29dbe640 1 -- 172.21.15.83:0/3960205336 >> v1:172.21.15.167:6789/0 conn(0x7f1d240cee30 legacy=0x7f1d240d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:29.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.288+0000 7f1d29dbe640 1 -- 172.21.15.83:0/3960205336 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d24162440 con 0x7f1d2415dfa0 2024-04-17T10:50:29.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.288+0000 7f1d295bd640 1 --2- 172.21.15.83:0/3960205336 >> v2:172.21.15.167:3300/0 conn(0x7f1d240ce980 0x7f1d240b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:29.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d29dbe640 1 --2- 172.21.15.83:0/3960205336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d2415dfa0 0x7f1d240b6c20 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f1d2408c8b0 tx=0x7f1d2001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959f3daabc98b973 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:29.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d28dbc640 1 -- 172.21.15.83:0/3960205336 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d200b0070 con 0x7f1d2415dfa0 2024-04-17T10:50:29.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d28dbc640 1 -- 172.21.15.83:0/3960205336 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d200ab600 con 0x7f1d2415dfa0 2024-04-17T10:50:29.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d28dbc640 1 -- 172.21.15.83:0/3960205336 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d200ab9d0 con 0x7f1d2415dfa0 2024-04-17T10:50:29.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d2c022640 1 -- 172.21.15.83:0/3960205336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d2415dfa0 msgr2=0x7f1d240b6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:29.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d2c022640 1 --2- 172.21.15.83:0/3960205336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d2415dfa0 0x7f1d240b6c20 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f1d2408c8b0 tx=0x7f1d2001c420 comp rx=0 tx=0).stop 2024-04-17T10:50:29.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d2c022640 1 -- 172.21.15.83:0/3960205336 shutdown_connections 2024-04-17T10:50:29.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d2c022640 1 --2- 172.21.15.83:0/3960205336 >> v2:172.21.15.167:3300/0 conn(0x7f1d240ce980 0x7f1d240b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:29.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d2c022640 1 --2- 172.21.15.83:0/3960205336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d2415dfa0 0x7f1d240b6c20 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:29.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d2c022640 1 -- 172.21.15.83:0/3960205336 >> 172.21.15.83:0/3960205336 conn(0x7f1d240d0600 msgr2=0x7f1d240d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:29.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d2c022640 1 -- 172.21.15.83:0/3960205336 shutdown_connections 2024-04-17T10:50:29.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d2c022640 1 -- 172.21.15.83:0/3960205336 wait complete. 2024-04-17T10:50:29.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d2c022640 1 Processor -- start 2024-04-17T10:50:29.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d2c022640 1 -- start start 2024-04-17T10:50:29.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d2c022640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d240ce980 0x7f1d24168660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:29.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d2c022640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d240cee30 0x7f1d2414c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:29.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d2c022640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d2415dfa0 0x7f1d240caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:29.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d2c022640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d24161b10 con 0x7f1d240ce980 2024-04-17T10:50:29.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d2c022640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d240d2850 con 0x7f1d240cee30 2024-04-17T10:50:29.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d2c022640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2416b660 con 0x7f1d2415dfa0 2024-04-17T10:50:29.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d29dbe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d240ce980 0x7f1d24168660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:29.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d29dbe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d240ce980 0x7f1d24168660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44890/0 (socket says 172.21.15.83:44890) 2024-04-17T10:50:29.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d29dbe640 1 -- 172.21.15.83:0/3339576043 learned_addr learned my addr 172.21.15.83:0/3339576043 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:29.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d29dbe640 1 -- 172.21.15.83:0/3339576043 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d2415dfa0 msgr2=0x7f1d240caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:29.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d29dbe640 1 --2- 172.21.15.83:0/3339576043 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d2415dfa0 0x7f1d240caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:29.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d29dbe640 1 -- 172.21.15.83:0/3339576043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d240cee30 msgr2=0x7f1d2414c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:29.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d29dbe640 1 --2- 172.21.15.83:0/3339576043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d240cee30 0x7f1d2414c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:29.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d29dbe640 1 -- 172.21.15.83:0/3339576043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d2414cee0 con 0x7f1d240ce980 2024-04-17T10:50:29.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d29dbe640 1 --2- 172.21.15.83:0/3339576043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d240ce980 0x7f1d24168660 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f1d2001aad0 tx=0x7f1d2001ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:29.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d1a7fc640 1 -- 172.21.15.83:0/3339576043 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d200ba020 con 0x7f1d240ce980 2024-04-17T10:50:29.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d1a7fc640 1 -- 172.21.15.83:0/3339576043 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d200b6640 con 0x7f1d240ce980 2024-04-17T10:50:29.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d2c022640 1 -- 172.21.15.83:0/3339576043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d240c86e0 con 0x7f1d240ce980 2024-04-17T10:50:29.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d1a7fc640 1 -- 172.21.15.83:0/3339576043 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d200b0040 con 0x7f1d240ce980 2024-04-17T10:50:29.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.292+0000 7f1d2c022640 1 -- 172.21.15.83:0/3339576043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d240c8ca0 con 0x7f1d240ce980 2024-04-17T10:50:29.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.296+0000 7f1d2c022640 1 -- 172.21.15.83:0/3339576043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d24074f90 con 0x7f1d240ce980 2024-04-17T10:50:29.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.296+0000 7f1d1a7fc640 1 -- 172.21.15.83:0/3339576043 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1d200bd020 con 0x7f1d240ce980 2024-04-17T10:50:29.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.296+0000 7f1d1a7fc640 1 --2- 172.21.15.83:0/3339576043 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1d08066f60 0x7f1d08069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:29.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.296+0000 7f1d1a7fc640 1 -- 172.21.15.83:0/3339576043 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f1d2011efd0 con 0x7f1d240ce980 2024-04-17T10:50:29.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.296+0000 7f1d295bd640 1 --2- 172.21.15.83:0/3339576043 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1d08066f60 0x7f1d08069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:29.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.296+0000 7f1d295bd640 1 --2- 172.21.15.83:0/3339576043 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1d08066f60 0x7f1d08069420 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f1d241613b0 tx=0x7f1d14008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:29.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.300+0000 7f1d1a7fc640 1 -- 172.21.15.83:0/3339576043 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1d200c4030 con 0x7f1d240ce980 2024-04-17T10:50:29.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.516+0000 7f1d2c022640 1 -- 172.21.15.83:0/3339576043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]} v 0) v1 -- 0x7f1d24151060 con 0x7f1d240ce980 2024-04-17T10:50:29.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.532+0000 7f1d1a7fc640 1 -- 172.21.15.83:0/3339576043 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]}]=0 v551) v1 ==== 162+0+0 (secure 0 0 0) 0x7f1d200160d0 con 0x7f1d240ce980 2024-04-17T10:50:29.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.536+0000 7f1d2c022640 1 -- 172.21.15.83:0/3339576043 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1d08066f60 msgr2=0x7f1d08069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:29.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.536+0000 7f1d2c022640 1 --2- 172.21.15.83:0/3339576043 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1d08066f60 0x7f1d08069420 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f1d241613b0 tx=0x7f1d14008040 comp rx=0 tx=0).stop 2024-04-17T10:50:29.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.536+0000 7f1d2c022640 1 -- 172.21.15.83:0/3339576043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d240ce980 msgr2=0x7f1d24168660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:29.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.536+0000 7f1d2c022640 1 --2- 172.21.15.83:0/3339576043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d240ce980 0x7f1d24168660 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f1d2001aad0 tx=0x7f1d2001ac60 comp rx=0 tx=0).stop 2024-04-17T10:50:29.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.536+0000 7f1d2c022640 1 -- 172.21.15.83:0/3339576043 shutdown_connections 2024-04-17T10:50:29.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.536+0000 7f1d2c022640 1 --2- 172.21.15.83:0/3339576043 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1d08066f60 0x7f1d08069420 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:29.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.536+0000 7f1d2c022640 1 --2- 172.21.15.83:0/3339576043 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d2415dfa0 0x7f1d240caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:29.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.536+0000 7f1d2c022640 1 --2- 172.21.15.83:0/3339576043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d240cee30 0x7f1d2414c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:29.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.536+0000 7f1d2c022640 1 --2- 172.21.15.83:0/3339576043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d240ce980 0x7f1d24168660 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:29.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.536+0000 7f1d2c022640 1 -- 172.21.15.83:0/3339576043 >> 172.21.15.83:0/3339576043 conn(0x7f1d240d0600 msgr2=0x7f1d24151860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:29.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.536+0000 7f1d2c022640 1 -- 172.21.15.83:0/3339576043 shutdown_connections 2024-04-17T10:50:29.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.536+0000 7f1d2c022640 1 -- 172.21.15.83:0/3339576043 wait complete. 2024-04-17T10:50:29.545 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-17T10:50:29.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13fbe4c640 1 Processor -- start 2024-04-17T10:50:29.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13fbe4c640 1 -- start start 2024-04-17T10:50:29.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13fbe4c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f13f415bac0 0x7f13f4071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:29.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13fbe4c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13f4070470 0x7f13f406ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:29.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13fbe4c640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f406c1f0 con 0x7f13f415b6e0 2024-04-17T10:50:29.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13fbe4c640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13f4070a10 con 0x7f13f415bac0 2024-04-17T10:50:29.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13fbe4c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f4070cf0 con 0x7f13f4070470 2024-04-17T10:50:29.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13f9be8640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f13f415b6e0 0x7f13f40cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56266/0 (socket says 172.21.15.83:56266) 2024-04-17T10:50:29.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13f9be8640 1 -- 172.21.15.83:0/1095180113 learned_addr learned my addr 172.21.15.83:0/1095180113 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:29.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13f8be6640 1 -- 172.21.15.83:0/1095180113 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2771993780 0 0) 0x7f13f406c1f0 con 0x7f13f415b6e0 2024-04-17T10:50:29.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13f93e7640 1 --2- 172.21.15.83:0/1095180113 >> v2:172.21.15.167:3300/0 conn(0x7f13f415bac0 0x7f13f4071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:29.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13f8be6640 1 -- 172.21.15.83:0/1095180113 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13e0003530 con 0x7f13f415b6e0 2024-04-17T10:50:29.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13fa3e9640 1 --2- 172.21.15.83:0/1095180113 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13f4070470 0x7f13f406ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:29.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13f8be6640 1 -- 172.21.15.83:0/1095180113 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 714297446 0 0) 0x7f13e0003530 con 0x7f13f415b6e0 2024-04-17T10:50:29.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13f8be6640 1 -- 172.21.15.83:0/1095180113 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13f4070470 msgr2=0x7f13f406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:29.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13f8be6640 1 --2- 172.21.15.83:0/1095180113 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13f4070470 0x7f13f406ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:29.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13f8be6640 1 -- 172.21.15.83:0/1095180113 >> v2:172.21.15.167:3300/0 conn(0x7f13f415bac0 msgr2=0x7f13f4071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:29.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13f8be6640 1 --2- 172.21.15.83:0/1095180113 >> v2:172.21.15.167:3300/0 conn(0x7f13f415bac0 0x7f13f4071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:29.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13f8be6640 1 -- 172.21.15.83:0/1095180113 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f4070fd0 con 0x7f13f415b6e0 2024-04-17T10:50:29.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13f8be6640 1 -- 172.21.15.83:0/1095180113 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f13e80016a0 con 0x7f13f415b6e0 2024-04-17T10:50:29.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13f8be6640 1 -- 172.21.15.83:0/1095180113 >> v1:172.21.15.83:6789/0 conn(0x7f13f415b6e0 legacy=0x7f13f40cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:29.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13f8be6640 1 --2- 172.21.15.83:0/1095180113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13e0003a30 0x7f13e0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:29.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13f8be6640 1 --2- 172.21.15.83:0/1095180113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13e0004810 0x7f13e0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:29.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13f8be6640 1 --2- 172.21.15.83:0/1095180113 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13e00051d0 0x7f13e000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:29.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13f8be6640 1 -- 172.21.15.83:0/1095180113 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f4070a10 con 0x7f13e0004810 2024-04-17T10:50:29.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13f8be6640 1 -- 172.21.15.83:0/1095180113 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f4070cf0 con 0x7f13e0003a30 2024-04-17T10:50:29.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13f8be6640 1 -- 172.21.15.83:0/1095180113 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e000c0a0 con 0x7f13e00051d0 2024-04-17T10:50:29.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13fa3e9640 1 --2- 172.21.15.83:0/1095180113 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13f4070470 0x7f13f406ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:50:29.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13f93e7640 1 --2- 172.21.15.83:0/1095180113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13e0004810 0x7f13e0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:29.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13f9be8640 1 --2- 172.21.15.83:0/1095180113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13e0003a30 0x7f13e0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:29.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13f93e7640 1 -- 172.21.15.83:0/1095180113 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13e00051d0 msgr2=0x7f13e000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:29.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13fa3e9640 1 --2- 172.21.15.83:0/1095180113 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13e00051d0 0x7f13e000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:29.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13f93e7640 1 --2- 172.21.15.83:0/1095180113 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13e00051d0 0x7f13e000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:29.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13f93e7640 1 -- 172.21.15.83:0/1095180113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13e0003a30 msgr2=0x7f13e0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:29.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13f93e7640 1 --2- 172.21.15.83:0/1095180113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13e0003a30 0x7f13e0003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:29.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.664+0000 7f13f93e7640 1 -- 172.21.15.83:0/1095180113 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e0003530 con 0x7f13e0004810 2024-04-17T10:50:29.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.668+0000 7f13f93e7640 1 --2- 172.21.15.83:0/1095180113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13e0004810 0x7f13e0004c90 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f13f0009b10 tx=0x7f13f001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a8f05e61d68f1b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:29.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.668+0000 7f13f8be6640 1 -- 172.21.15.83:0/1095180113 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f00096b0 con 0x7f13e0004810 2024-04-17T10:50:29.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.668+0000 7f13f8be6640 1 -- 172.21.15.83:0/1095180113 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f00a6490 con 0x7f13e0004810 2024-04-17T10:50:29.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.668+0000 7f13f8be6640 1 -- 172.21.15.83:0/1095180113 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f00a6880 con 0x7f13e0004810 2024-04-17T10:50:29.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.668+0000 7f13fbe4c640 1 -- 172.21.15.83:0/1095180113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13e0004810 msgr2=0x7f13e0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:29.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.668+0000 7f13fbe4c640 1 --2- 172.21.15.83:0/1095180113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13e0004810 0x7f13e0004c90 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f13f0009b10 tx=0x7f13f001cab0 comp rx=0 tx=0).stop 2024-04-17T10:50:29.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.668+0000 7f13fa3e9640 1 -- 172.21.15.83:0/1095180113 reap_dead start 2024-04-17T10:50:29.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.668+0000 7f13fbe4c640 1 -- 172.21.15.83:0/1095180113 shutdown_connections 2024-04-17T10:50:29.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.668+0000 7f13fbe4c640 1 -- 172.21.15.83:0/1095180113 >> 172.21.15.83:0/1095180113 conn(0x7f13f40b15b0 msgr2=0x7f13f40b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:29.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.668+0000 7f13fbe4c640 1 -- 172.21.15.83:0/1095180113 shutdown_connections 2024-04-17T10:50:29.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.668+0000 7f13fbe4c640 1 -- 172.21.15.83:0/1095180113 wait complete. 2024-04-17T10:50:29.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.668+0000 7f13fbe4c640 1 Processor -- start 2024-04-17T10:50:29.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.668+0000 7f13fbe4c640 1 -- start start 2024-04-17T10:50:29.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.668+0000 7f13fbe4c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13e0004810 0x7f13f40ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:29.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.668+0000 7f13fbe4c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13f40add80 0x7f13f41f7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:29.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.668+0000 7f13fbe4c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13f41f7c60 0x7f13f41faab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:29.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.668+0000 7f13fbe4c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f406d270 con 0x7f13f40add80 2024-04-17T10:50:29.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.668+0000 7f13fbe4c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f406d0f0 con 0x7f13f41f7c60 2024-04-17T10:50:29.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.668+0000 7f13fbe4c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f41665f0 con 0x7f13e0004810 2024-04-17T10:50:29.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.668+0000 7f13f93e7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13f40add80 0x7f13f41f7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:29.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.668+0000 7f13f93e7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13f40add80 0x7f13f41f7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44904/0 (socket says 172.21.15.83:44904) 2024-04-17T10:50:29.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.668+0000 7f13f93e7640 1 -- 172.21.15.83:0/836128299 learned_addr learned my addr 172.21.15.83:0/836128299 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:29.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.668+0000 7f13f93e7640 1 -- 172.21.15.83:0/836128299 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13e0004810 msgr2=0x7f13f40ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:29.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.668+0000 7f13f93e7640 1 --2- 172.21.15.83:0/836128299 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13e0004810 0x7f13f40ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:29.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.668+0000 7f13f93e7640 1 -- 172.21.15.83:0/836128299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13f41f7c60 msgr2=0x7f13f41faab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:29.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.668+0000 7f13f93e7640 1 --2- 172.21.15.83:0/836128299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13f41f7c60 0x7f13f41faab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:29.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.668+0000 7f13f93e7640 1 -- 172.21.15.83:0/836128299 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f40cceb0 con 0x7f13f40add80 2024-04-17T10:50:29.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.668+0000 7f13f93e7640 1 --2- 172.21.15.83:0/836128299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13f40add80 0x7f13f41f7720 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f13f001cf90 tx=0x7f13e0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:29.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.672+0000 7f13e67fc640 1 -- 172.21.15.83:0/836128299 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f00c0070 con 0x7f13f40add80 2024-04-17T10:50:29.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.672+0000 7f13e67fc640 1 -- 172.21.15.83:0/836128299 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f000f000 con 0x7f13f40add80 2024-04-17T10:50:29.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.672+0000 7f13fbe4c640 1 -- 172.21.15.83:0/836128299 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f415cd30 con 0x7f13f40add80 2024-04-17T10:50:29.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.672+0000 7f13e67fc640 1 -- 172.21.15.83:0/836128299 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f00a6d80 con 0x7f13f40add80 2024-04-17T10:50:29.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.672+0000 7f13fbe4c640 1 -- 172.21.15.83:0/836128299 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f415d2f0 con 0x7f13f40add80 2024-04-17T10:50:29.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.672+0000 7f13fbe4c640 1 -- 172.21.15.83:0/836128299 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f40ae200 con 0x7f13f40add80 2024-04-17T10:50:29.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.672+0000 7f13e67fc640 1 -- 172.21.15.83:0/836128299 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f13f00c3070 con 0x7f13f40add80 2024-04-17T10:50:29.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.672+0000 7f13e67fc640 1 --2- 172.21.15.83:0/836128299 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f13d8066f90 0x7f13d8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:29.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.672+0000 7f13f9be8640 1 --2- 172.21.15.83:0/836128299 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f13d8066f90 0x7f13d8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:29.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.672+0000 7f13e67fc640 1 -- 172.21.15.83:0/836128299 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f13f011ebb0 con 0x7f13f40add80 2024-04-17T10:50:29.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.672+0000 7f13f9be8640 1 --2- 172.21.15.83:0/836128299 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f13d8066f90 0x7f13d8069450 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f13f40b1240 tx=0x7f13e8000ee0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:29.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.676+0000 7f13e67fc640 1 -- 172.21.15.83:0/836128299 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f13f00ac020 con 0x7f13f40add80 2024-04-17T10:50:29.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.824+0000 7f13fbe4c640 1 -- 172.21.15.83:0/836128299 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f13f4074f90 con 0x7f13f40add80 2024-04-17T10:50:29.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.832+0000 7f13e67fc640 1 -- 172.21.15.83:0/836128299 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f13f0122070 con 0x7f13f40add80 2024-04-17T10:50:29.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.832+0000 7f13fbe4c640 1 -- 172.21.15.83:0/836128299 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f13d8066f90 msgr2=0x7f13d8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:29.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.832+0000 7f13fbe4c640 1 --2- 172.21.15.83:0/836128299 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f13d8066f90 0x7f13d8069450 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f13f40b1240 tx=0x7f13e8000ee0 comp rx=0 tx=0).stop 2024-04-17T10:50:29.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.832+0000 7f13fbe4c640 1 -- 172.21.15.83:0/836128299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13f40add80 msgr2=0x7f13f41f7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:29.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.832+0000 7f13fbe4c640 1 --2- 172.21.15.83:0/836128299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13f40add80 0x7f13f41f7720 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f13f001cf90 tx=0x7f13e0005840 comp rx=0 tx=0).stop 2024-04-17T10:50:29.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.832+0000 7f13fbe4c640 1 -- 172.21.15.83:0/836128299 shutdown_connections 2024-04-17T10:50:29.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.832+0000 7f13fbe4c640 1 --2- 172.21.15.83:0/836128299 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f13d8066f90 0x7f13d8069450 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:29.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.836+0000 7f13fbe4c640 1 --2- 172.21.15.83:0/836128299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13f41f7c60 0x7f13f41faab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:29.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.836+0000 7f13fbe4c640 1 --2- 172.21.15.83:0/836128299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13f40add80 0x7f13f41f7720 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:29.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.836+0000 7f13fbe4c640 1 --2- 172.21.15.83:0/836128299 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13e0004810 0x7f13f40ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:29.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.836+0000 7f13fbe4c640 1 -- 172.21.15.83:0/836128299 >> 172.21.15.83:0/836128299 conn(0x7f13f40b15b0 msgr2=0x7f13f40b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:29.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.836+0000 7f13fbe4c640 1 -- 172.21.15.83:0/836128299 shutdown_connections 2024-04-17T10:50:29.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:29.836+0000 7f13fbe4c640 1 -- 172.21.15.83:0/836128299 wait complete. 2024-04-17T10:50:29.843 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T10:50:29.844 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:50:29.897 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:50:29.940 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T10:50:29.940 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:50:29.940 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:50:29.945 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16063 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:50:29.945 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:50:29.946 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:50:29.987 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:50:29.987 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:50:29.987 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:50:29.993 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16063 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:50:29.993 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:50:29.993 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:50:30.035 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:50:30.035 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-17T10:50:30.035 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:50:30.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.168+0000 7fc567c26640 1 Processor -- start 2024-04-17T10:50:30.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.168+0000 7fc567c26640 1 -- start start 2024-04-17T10:50:30.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.168+0000 7fc567c26640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc5600b1ce0 0x7fc56006dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:30.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.168+0000 7fc567c26640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc56006c1b0 0x7fc560164660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:30.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.168+0000 7fc567c26640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc56015b8c0 con 0x7fc56015ae80 2024-04-17T10:50:30.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.168+0000 7fc567c26640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc56015bba0 con 0x7fc5600b1ce0 2024-04-17T10:50:30.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.168+0000 7fc567c26640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc56015be80 con 0x7fc56006c1b0 2024-04-17T10:50:30.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.168+0000 7fc5661c3640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc56006c1b0 0x7fc560164660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:30.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.168+0000 7fc5659c2640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc5600b1ce0 0x7fc56006dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:30.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.168+0000 7fc5659c2640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc5600b1ce0 0x7fc56006dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:60800/0 (socket says 172.21.15.83:60800) 2024-04-17T10:50:30.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.168+0000 7fc5651c1640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fc56015ae80 0x7fc5600b20c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:58910/0 (socket says 172.21.15.83:58910) 2024-04-17T10:50:30.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.168+0000 7fc5659c2640 1 -- 172.21.15.83:0/2323784604 learned_addr learned my addr 172.21.15.83:0/2323784604 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:30.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.168+0000 7fc5659c2640 1 -- 172.21.15.83:0/2323784604 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc56006c1b0 msgr2=0x7fc560164660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:30.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.168+0000 7fc5659c2640 1 --2- 172.21.15.83:0/2323784604 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc56006c1b0 0x7fc560164660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:30.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.168+0000 7fc5659c2640 1 -- 172.21.15.83:0/2323784604 >> v1:172.21.15.167:6789/0 conn(0x7fc56015ae80 legacy=0x7fc5600b20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:30.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.168+0000 7fc5659c2640 1 -- 172.21.15.83:0/2323784604 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc56015c160 con 0x7fc5600b1ce0 2024-04-17T10:50:30.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.168+0000 7fc5661c3640 1 --2- 172.21.15.83:0/2323784604 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc56006c1b0 0x7fc560164660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:50:30.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.172+0000 7fc5659c2640 1 --2- 172.21.15.83:0/2323784604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5600b1ce0 0x7fc56006dc40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc558009e50 tx=0x7fc55801ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdfeb0123fbc3385 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:30.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.172+0000 7fc5649c0640 1 -- 172.21.15.83:0/2323784604 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5580a57e0 con 0x7fc5600b1ce0 2024-04-17T10:50:30.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.172+0000 7fc5649c0640 1 -- 172.21.15.83:0/2323784604 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5580a5980 con 0x7fc5600b1ce0 2024-04-17T10:50:30.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.172+0000 7fc5649c0640 1 -- 172.21.15.83:0/2323784604 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5580a5d50 con 0x7fc5600b1ce0 2024-04-17T10:50:30.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.172+0000 7fc567c26640 1 -- 172.21.15.83:0/2323784604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5600b1ce0 msgr2=0x7fc56006dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:30.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.172+0000 7fc567c26640 1 --2- 172.21.15.83:0/2323784604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5600b1ce0 0x7fc56006dc40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc558009e50 tx=0x7fc55801ca00 comp rx=0 tx=0).stop 2024-04-17T10:50:30.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.172+0000 7fc567c26640 1 -- 172.21.15.83:0/2323784604 shutdown_connections 2024-04-17T10:50:30.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.172+0000 7fc567c26640 1 --2- 172.21.15.83:0/2323784604 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc56006c1b0 0x7fc560164660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:30.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.172+0000 7fc567c26640 1 --2- 172.21.15.83:0/2323784604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5600b1ce0 0x7fc56006dc40 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:30.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.172+0000 7fc567c26640 1 -- 172.21.15.83:0/2323784604 >> 172.21.15.83:0/2323784604 conn(0x7fc5600b4eb0 msgr2=0x7fc5600b5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:30.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.172+0000 7fc567c26640 1 -- 172.21.15.83:0/2323784604 shutdown_connections 2024-04-17T10:50:30.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.172+0000 7fc567c26640 1 -- 172.21.15.83:0/2323784604 wait complete. 2024-04-17T10:50:30.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.176+0000 7fc567c26640 1 Processor -- start 2024-04-17T10:50:30.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.176+0000 7fc567c26640 1 -- start start 2024-04-17T10:50:30.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.176+0000 7fc567c26640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc56006c1b0 0x7fc5600d0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:30.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.176+0000 7fc567c26640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5600b1ce0 0x7fc560158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:30.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.176+0000 7fc567c26640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc56015ae80 0x7fc56014e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:30.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.176+0000 7fc567c26640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc560167750 con 0x7fc5600b1ce0 2024-04-17T10:50:30.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.176+0000 7fc567c26640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5601675d0 con 0x7fc56006c1b0 2024-04-17T10:50:30.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.176+0000 7fc567c26640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5601678d0 con 0x7fc56015ae80 2024-04-17T10:50:30.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.176+0000 7fc5651c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5600b1ce0 0x7fc560158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:30.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.176+0000 7fc5651c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5600b1ce0 0x7fc560158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44908/0 (socket says 172.21.15.83:44908) 2024-04-17T10:50:30.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.176+0000 7fc5651c1640 1 -- 172.21.15.83:0/2422448415 learned_addr learned my addr 172.21.15.83:0/2422448415 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:30.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.176+0000 7fc5659c2640 1 --2- 172.21.15.83:0/2422448415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc56006c1b0 0x7fc5600d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:30.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.176+0000 7fc5661c3640 1 --2- 172.21.15.83:0/2422448415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc56015ae80 0x7fc56014e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:30.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.176+0000 7fc5651c1640 1 -- 172.21.15.83:0/2422448415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc56015ae80 msgr2=0x7fc56014e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:30.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.176+0000 7fc5651c1640 1 --2- 172.21.15.83:0/2422448415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc56015ae80 0x7fc56014e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:30.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.176+0000 7fc5651c1640 1 -- 172.21.15.83:0/2422448415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc56006c1b0 msgr2=0x7fc5600d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:30.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.176+0000 7fc5651c1640 1 --2- 172.21.15.83:0/2422448415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc56006c1b0 0x7fc5600d0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:30.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.176+0000 7fc5651c1640 1 -- 172.21.15.83:0/2422448415 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc56014d350 con 0x7fc5600b1ce0 2024-04-17T10:50:30.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.176+0000 7fc5659c2640 1 --2- 172.21.15.83:0/2422448415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc56006c1b0 0x7fc5600d0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:30.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.176+0000 7fc5651c1640 1 --2- 172.21.15.83:0/2422448415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5600b1ce0 0x7fc560158870 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fc55000c3a0 tx=0x7fc55000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:30.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.176+0000 7fc5567fc640 1 -- 172.21.15.83:0/2422448415 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5500074e0 con 0x7fc5600b1ce0 2024-04-17T10:50:30.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.176+0000 7fc5567fc640 1 -- 172.21.15.83:0/2422448415 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc550007680 con 0x7fc5600b1ce0 2024-04-17T10:50:30.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.176+0000 7fc567c26640 1 -- 172.21.15.83:0/2422448415 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc56014cab0 con 0x7fc5600b1ce0 2024-04-17T10:50:30.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.176+0000 7fc5567fc640 1 -- 172.21.15.83:0/2422448415 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc550004a30 con 0x7fc5600b1ce0 2024-04-17T10:50:30.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.176+0000 7fc567c26640 1 -- 172.21.15.83:0/2422448415 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc56014d070 con 0x7fc5600b1ce0 2024-04-17T10:50:30.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.176+0000 7fc567c26640 1 -- 172.21.15.83:0/2422448415 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc560074f90 con 0x7fc5600b1ce0 2024-04-17T10:50:30.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.176+0000 7fc5567fc640 1 -- 172.21.15.83:0/2422448415 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc55001a020 con 0x7fc5600b1ce0 2024-04-17T10:50:30.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.176+0000 7fc5567fc640 1 --2- 172.21.15.83:0/2422448415 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc544066f60 0x7fc544069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:30.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.176+0000 7fc5567fc640 1 -- 172.21.15.83:0/2422448415 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fc5500845d0 con 0x7fc5600b1ce0 2024-04-17T10:50:30.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.176+0000 7fc5659c2640 1 --2- 172.21.15.83:0/2422448415 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc544066f60 0x7fc544069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:30.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.180+0000 7fc5659c2640 1 --2- 172.21.15.83:0/2422448415 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc544066f60 0x7fc544069420 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc5580ae000 tx=0x7fc5580a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:30.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.184+0000 7fc5567fc640 1 -- 172.21.15.83:0/2422448415 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc5500557e0 con 0x7fc5600b1ce0 2024-04-17T10:50:30.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.372+0000 7fc567c26640 1 -- 172.21.15.83:0/2422448415 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc560068140 con 0x7fc5600b1ce0 2024-04-17T10:50:30.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.376+0000 7fc5567fc640 1 -- 172.21.15.83:0/2422448415 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+2914 (secure 0 0 0) 0x7fc5500550e0 con 0x7fc5600b1ce0 2024-04-17T10:50:30.367 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:50:30.367 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":82,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5942,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3888166846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3888166846},{"type":"v1","addr":"172.21.15.167:6833","nonce":3888166846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":6772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1167913722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1167913722},{"type":"v1","addr":"172.21.15.167:6835","nonce":1167913722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":9645,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3673373809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3673373809},{"type":"v1","addr":"172.21.15.83:6835","nonce":3673373809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":82}],"filesystems":[]} 2024-04-17T10:50:30.368 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 82 2024-04-17T10:50:30.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.376+0000 7fc567c26640 1 -- 172.21.15.83:0/2422448415 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc544066f60 msgr2=0x7fc544069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:30.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.376+0000 7fc567c26640 1 --2- 172.21.15.83:0/2422448415 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc544066f60 0x7fc544069420 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc5580ae000 tx=0x7fc5580a7070 comp rx=0 tx=0).stop 2024-04-17T10:50:30.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.376+0000 7fc567c26640 1 -- 172.21.15.83:0/2422448415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5600b1ce0 msgr2=0x7fc560158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:30.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.376+0000 7fc567c26640 1 --2- 172.21.15.83:0/2422448415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5600b1ce0 0x7fc560158870 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fc55000c3a0 tx=0x7fc55000c860 comp rx=0 tx=0).stop 2024-04-17T10:50:30.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.376+0000 7fc567c26640 1 -- 172.21.15.83:0/2422448415 shutdown_connections 2024-04-17T10:50:30.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.376+0000 7fc567c26640 1 --2- 172.21.15.83:0/2422448415 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc544066f60 0x7fc544069420 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:30.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.376+0000 7fc567c26640 1 --2- 172.21.15.83:0/2422448415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc56015ae80 0x7fc56014e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:30.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.376+0000 7fc567c26640 1 --2- 172.21.15.83:0/2422448415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5600b1ce0 0x7fc560158870 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:30.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.376+0000 7fc567c26640 1 --2- 172.21.15.83:0/2422448415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc56006c1b0 0x7fc5600d0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:30.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.376+0000 7fc567c26640 1 -- 172.21.15.83:0/2422448415 >> 172.21.15.83:0/2422448415 conn(0x7fc5600b4eb0 msgr2=0x7fc560157380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:30.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.376+0000 7fc567c26640 1 -- 172.21.15.83:0/2422448415 shutdown_connections 2024-04-17T10:50:30.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.376+0000 7fc567c26640 1 -- 172.21.15.83:0/2422448415 wait complete. 2024-04-17T10:50:30.387 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-17T10:50:30.387 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-17T10:50:30.387 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-17T10:50:30.388 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:50:30.388 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-17T10:50:30.388 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-17T10:50:30.388 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-17T10:50:30.388 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-17T10:50:30.389 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-17T10:50:30.389 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:50:30.389 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-17T10:50:30.389 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-17T10:50:30.389 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-17T10:50:30.389 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:50:30.391 INFO:tasks.ceph.mds.c:Started 2024-04-17T10:50:30.391 DEBUG:teuthology.parallel:result is None 2024-04-17T10:50:30.397 INFO:tasks.ceph.mds.a:Stopped 2024-04-17T10:50:30.397 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-17T10:50:30.402 INFO:tasks.ceph.mds.b:Stopped 2024-04-17T10:50:30.402 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-17T10:50:30.407 INFO:tasks.ceph.mds.d:Stopped 2024-04-17T10:50:30.407 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-17T10:50:30.440 INFO:tasks.ceph.mds.a:Started 2024-04-17T10:50:30.440 DEBUG:teuthology.parallel:result is None 2024-04-17T10:50:30.451 INFO:tasks.ceph.mds.b:Started 2024-04-17T10:50:30.451 DEBUG:teuthology.parallel:result is None 2024-04-17T10:50:30.452 INFO:tasks.ceph.mds.d:Started 2024-04-17T10:50:30.453 DEBUG:teuthology.parallel:result is None 2024-04-17T10:50:30.453 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-17T10:50:30.456 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-04-17T10:50:30.502 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-04-17T10:50:30.524 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2024-04-17T10:50:30.526 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2024-04-17T10:50:30.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.572+0000 7f6c7fbe1640 1 Processor -- start 2024-04-17T10:50:30.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.572+0000 7f6c7fbe1640 1 -- start start 2024-04-17T10:50:30.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.576+0000 7f6c7fbe1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c7815bc10 0x7f6c780d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:30.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.576+0000 7f6c7fbe1640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6c780d0490 0x7f6c780cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:30.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.576+0000 7f6c7fbe1640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c780ced00 con 0x7f6c780cd3d0 2024-04-17T10:50:30.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.576+0000 7f6c7fbe1640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c780cefe0 con 0x7f6c780d0490 2024-04-17T10:50:30.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.576+0000 7f6c7fbe1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c780cc690 con 0x7f6c7815bc10 2024-04-17T10:50:30.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.576+0000 7f6c7d97d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c7815bc10 0x7f6c780d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:30.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.576+0000 7f6c7d97d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c7815bc10 0x7f6c780d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:41366/0 (socket says 172.21.15.83:41366) 2024-04-17T10:50:30.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.576+0000 7f6c7d97d640 1 -- 172.21.15.83:0/1120643520 learned_addr learned my addr 172.21.15.83:0/1120643520 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:30.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.576+0000 7f6c7d17c640 1 --2- 172.21.15.83:0/1120643520 >> v2:172.21.15.167:3300/0 conn(0x7f6c780d0490 0x7f6c780cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:30.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.576+0000 7f6c7d97d640 1 -- 172.21.15.83:0/1120643520 >> v2:172.21.15.167:3300/0 conn(0x7f6c780d0490 msgr2=0x7f6c780cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:30.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.576+0000 7f6c7d97d640 1 --2- 172.21.15.83:0/1120643520 >> v2:172.21.15.167:3300/0 conn(0x7f6c780d0490 0x7f6c780cce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:30.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.576+0000 7f6c7d97d640 1 -- 172.21.15.83:0/1120643520 >> v1:172.21.15.167:6789/0 conn(0x7f6c780cd3d0 legacy=0x7f6c7806e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:30.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.576+0000 7f6c7d97d640 1 -- 172.21.15.83:0/1120643520 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c780cc970 con 0x7f6c7815bc10 2024-04-17T10:50:30.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.576+0000 7f6c7d97d640 1 --2- 172.21.15.83:0/1120643520 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c7815bc10 0x7f6c780d2300 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6c68009870 tx=0x7f6c6801c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7036f25e0de9e485 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:30.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.576+0000 7f6c7c97b640 1 -- 172.21.15.83:0/1120643520 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c680b1070 con 0x7f6c7815bc10 2024-04-17T10:50:30.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.576+0000 7f6c7c97b640 1 -- 172.21.15.83:0/1120643520 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c680a8050 con 0x7f6c7815bc10 2024-04-17T10:50:30.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.576+0000 7f6c7c97b640 1 -- 172.21.15.83:0/1120643520 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c680ac860 con 0x7f6c7815bc10 2024-04-17T10:50:30.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.576+0000 7f6c7fbe1640 1 -- 172.21.15.83:0/1120643520 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c7815bc10 msgr2=0x7f6c780d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:30.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.576+0000 7f6c7fbe1640 1 --2- 172.21.15.83:0/1120643520 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c7815bc10 0x7f6c780d2300 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6c68009870 tx=0x7f6c6801c4d0 comp rx=0 tx=0).stop 2024-04-17T10:50:30.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.576+0000 7f6c7fbe1640 1 -- 172.21.15.83:0/1120643520 shutdown_connections 2024-04-17T10:50:30.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.576+0000 7f6c7fbe1640 1 --2- 172.21.15.83:0/1120643520 >> v2:172.21.15.167:3300/0 conn(0x7f6c780d0490 0x7f6c780cce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:30.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.576+0000 7f6c7fbe1640 1 --2- 172.21.15.83:0/1120643520 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c7815bc10 0x7f6c780d2300 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:30.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.576+0000 7f6c7fbe1640 1 -- 172.21.15.83:0/1120643520 >> 172.21.15.83:0/1120643520 conn(0x7f6c780d17e0 msgr2=0x7f6c780d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:30.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.576+0000 7f6c7fbe1640 1 -- 172.21.15.83:0/1120643520 shutdown_connections 2024-04-17T10:50:30.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.576+0000 7f6c7fbe1640 1 -- 172.21.15.83:0/1120643520 wait complete. 2024-04-17T10:50:30.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.580+0000 7f6c7fbe1640 1 Processor -- start 2024-04-17T10:50:30.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.580+0000 7f6c7fbe1640 1 -- start start 2024-04-17T10:50:30.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.580+0000 7f6c7fbe1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c780cd3d0 0x7f6c780ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:30.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.580+0000 7f6c7fbe1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c780d0490 0x7f6c781fae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:30.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.580+0000 7f6c7fbe1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c7815bc10 0x7f6c780b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:30.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.580+0000 7f6c7fbe1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c78168990 con 0x7f6c7815bc10 2024-04-17T10:50:30.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.580+0000 7f6c7fbe1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7806ee20 con 0x7f6c780cd3d0 2024-04-17T10:50:30.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.580+0000 7f6c7fbe1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c78168b10 con 0x7f6c780d0490 2024-04-17T10:50:30.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.580+0000 7f6c7e17e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c7815bc10 0x7f6c780b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:30.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.580+0000 7f6c7d17c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c780d0490 0x7f6c781fae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:30.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.580+0000 7f6c7e17e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c7815bc10 0x7f6c780b2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44944/0 (socket says 172.21.15.83:44944) 2024-04-17T10:50:30.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.580+0000 7f6c7e17e640 1 -- 172.21.15.83:0/2566088909 learned_addr learned my addr 172.21.15.83:0/2566088909 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:30.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.580+0000 7f6c7d97d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c780cd3d0 0x7f6c780ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:30.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.580+0000 7f6c7e17e640 1 -- 172.21.15.83:0/2566088909 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c780d0490 msgr2=0x7f6c781fae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:30.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.580+0000 7f6c7e17e640 1 --2- 172.21.15.83:0/2566088909 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c780d0490 0x7f6c781fae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:30.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.580+0000 7f6c7e17e640 1 -- 172.21.15.83:0/2566088909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c780cd3d0 msgr2=0x7f6c780ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:30.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.580+0000 7f6c7e17e640 1 --2- 172.21.15.83:0/2566088909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c780cd3d0 0x7f6c780ca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:30.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.580+0000 7f6c7e17e640 1 -- 172.21.15.83:0/2566088909 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c781f6f00 con 0x7f6c7815bc10 2024-04-17T10:50:30.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.580+0000 7f6c7e17e640 1 --2- 172.21.15.83:0/2566088909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c7815bc10 0x7f6c780b2ff0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000c730 tx=0x7f6c7000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:30.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.580+0000 7f6c6e7fc640 1 -- 172.21.15.83:0/2566088909 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c70003b80 con 0x7f6c7815bc10 2024-04-17T10:50:30.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.580+0000 7f6c6e7fc640 1 -- 172.21.15.83:0/2566088909 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c70003d20 con 0x7f6c7815bc10 2024-04-17T10:50:30.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.580+0000 7f6c7fbe1640 1 -- 172.21.15.83:0/2566088909 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c781f71c0 con 0x7f6c7815bc10 2024-04-17T10:50:30.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.580+0000 7f6c6e7fc640 1 -- 172.21.15.83:0/2566088909 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7000aa30 con 0x7f6c7815bc10 2024-04-17T10:50:30.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.580+0000 7f6c7fbe1640 1 -- 172.21.15.83:0/2566088909 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c781f7780 con 0x7f6c7815bc10 2024-04-17T10:50:30.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.580+0000 7f6c7fbe1640 1 -- 172.21.15.83:0/2566088909 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c78074f90 con 0x7f6c7815bc10 2024-04-17T10:50:30.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.584+0000 7f6c6e7fc640 1 -- 172.21.15.83:0/2566088909 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6c7000abd0 con 0x7f6c7815bc10 2024-04-17T10:50:30.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.584+0000 7f6c6e7fc640 1 --2- 172.21.15.83:0/2566088909 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6c5c066f60 0x7f6c5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:30.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.584+0000 7f6c6e7fc640 1 -- 172.21.15.83:0/2566088909 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f6c70084c00 con 0x7f6c7815bc10 2024-04-17T10:50:30.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.584+0000 7f6c7d97d640 1 --2- 172.21.15.83:0/2566088909 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6c5c066f60 0x7f6c5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:30.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.584+0000 7f6c7d97d640 1 --2- 172.21.15.83:0/2566088909 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6c5c066f60 0x7f6c5c069420 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f6c680ae000 tx=0x7f6c680a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:30.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.588+0000 7f6c6e7fc640 1 -- 172.21.15.83:0/2566088909 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6c7008b080 con 0x7f6c7815bc10 2024-04-17T10:50:30.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.760+0000 7f6c7fbe1640 1 -- 172.21.15.83:0/2566088909 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6c780b7760 con 0x7f6c7815bc10 2024-04-17T10:50:30.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.848+0000 7f6c6e7fc640 1 -- 172.21.15.83:0/2566088909 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v136) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6c70055750 con 0x7f6c7815bc10 2024-04-17T10:50:30.842 INFO:teuthology.orchestra.run.smithi083.stderr: removed all blocklist entries 2024-04-17T10:50:30.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.852+0000 7f6c7fbe1640 1 -- 172.21.15.83:0/2566088909 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6c5c066f60 msgr2=0x7f6c5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:30.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.852+0000 7f6c7fbe1640 1 --2- 172.21.15.83:0/2566088909 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6c5c066f60 0x7f6c5c069420 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f6c680ae000 tx=0x7f6c680a7000 comp rx=0 tx=0).stop 2024-04-17T10:50:30.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.852+0000 7f6c7fbe1640 1 -- 172.21.15.83:0/2566088909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c7815bc10 msgr2=0x7f6c780b2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:30.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.852+0000 7f6c7fbe1640 1 --2- 172.21.15.83:0/2566088909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c7815bc10 0x7f6c780b2ff0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000c730 tx=0x7f6c7000cbf0 comp rx=0 tx=0).stop 2024-04-17T10:50:30.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.852+0000 7f6c7fbe1640 1 -- 172.21.15.83:0/2566088909 shutdown_connections 2024-04-17T10:50:30.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.852+0000 7f6c7fbe1640 1 --2- 172.21.15.83:0/2566088909 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6c5c066f60 0x7f6c5c069420 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:30.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.852+0000 7f6c7fbe1640 1 --2- 172.21.15.83:0/2566088909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c7815bc10 0x7f6c780b2ff0 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:30.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.852+0000 7f6c7fbe1640 1 --2- 172.21.15.83:0/2566088909 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c780d0490 0x7f6c781fae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:30.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.852+0000 7f6c7fbe1640 1 --2- 172.21.15.83:0/2566088909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c780cd3d0 0x7f6c780ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:30.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.852+0000 7f6c7fbe1640 1 -- 172.21.15.83:0/2566088909 >> 172.21.15.83:0/2566088909 conn(0x7f6c780d17e0 msgr2=0x7f6c780b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:30.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.852+0000 7f6c7fbe1640 1 -- 172.21.15.83:0/2566088909 shutdown_connections 2024-04-17T10:50:30.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:30.852+0000 7f6c7fbe1640 1 -- 172.21.15.83:0/2566088909 wait complete. 2024-04-17T10:50:30.864 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-17T10:50:31.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d55696640 1 Processor -- start 2024-04-17T10:50:31.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d55696640 1 -- start start 2024-04-17T10:50:31.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d55696640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8d500cfa20 0x7f8d500cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:31.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d55696640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d500d2340 0x7f8d500ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:31.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d55696640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d500cc330 con 0x7f8d500cf3f0 2024-04-17T10:50:31.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d55696640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d500b1890 con 0x7f8d500cfa20 2024-04-17T10:50:31.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d55696640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d500cc940 con 0x7f8d500d2340 2024-04-17T10:50:31.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d4e7fc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8d500cf3f0 0x7f8d500b1180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56284/0 (socket says 172.21.15.83:56284) 2024-04-17T10:50:31.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d4e7fc640 1 -- 172.21.15.83:0/1238516766 learned_addr learned my addr 172.21.15.83:0/1238516766 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:31.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d4effd640 1 --2- 172.21.15.83:0/1238516766 >> v2:172.21.15.167:3300/0 conn(0x7f8d500cfa20 0x7f8d500cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:31.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d4dffb640 1 -- 172.21.15.83:0/1238516766 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3493729189 0 0) 0x7f8d500cc330 con 0x7f8d500cf3f0 2024-04-17T10:50:31.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d4dffb640 1 -- 172.21.15.83:0/1238516766 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d38003530 con 0x7f8d500cf3f0 2024-04-17T10:50:31.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d4f7fe640 1 --2- 172.21.15.83:0/1238516766 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d500d2340 0x7f8d500ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:31.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d4dffb640 1 -- 172.21.15.83:0/1238516766 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2571965867 0 0) 0x7f8d38003530 con 0x7f8d500cf3f0 2024-04-17T10:50:31.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d4dffb640 1 -- 172.21.15.83:0/1238516766 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d500d2340 msgr2=0x7f8d500ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:31.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d4dffb640 1 --2- 172.21.15.83:0/1238516766 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d500d2340 0x7f8d500ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d4dffb640 1 -- 172.21.15.83:0/1238516766 >> v2:172.21.15.167:3300/0 conn(0x7f8d500cfa20 msgr2=0x7f8d500cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:31.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d4dffb640 1 --2- 172.21.15.83:0/1238516766 >> v2:172.21.15.167:3300/0 conn(0x7f8d500cfa20 0x7f8d500cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d4dffb640 1 -- 172.21.15.83:0/1238516766 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d500d0340 con 0x7f8d500cf3f0 2024-04-17T10:50:31.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d4dffb640 1 -- 172.21.15.83:0/1238516766 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f8d44001680 con 0x7f8d500cf3f0 2024-04-17T10:50:31.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d4dffb640 1 -- 172.21.15.83:0/1238516766 >> v1:172.21.15.83:6789/0 conn(0x7f8d500cf3f0 legacy=0x7f8d500b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:31.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d4dffb640 1 --2- 172.21.15.83:0/1238516766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d380039c0 0x7f8d38003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:31.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d4dffb640 1 --2- 172.21.15.83:0/1238516766 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d500d2340 0x7f8d38004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:31.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d4dffb640 1 --2- 172.21.15.83:0/1238516766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d38004d50 0x7f8d380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:31.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d4dffb640 1 -- 172.21.15.83:0/1238516766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d500b1890 con 0x7f8d380039c0 2024-04-17T10:50:31.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d4dffb640 1 -- 172.21.15.83:0/1238516766 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d500cc940 con 0x7f8d38004d50 2024-04-17T10:50:31.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d4dffb640 1 -- 172.21.15.83:0/1238516766 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3800bb30 con 0x7f8d500d2340 2024-04-17T10:50:31.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d4effd640 1 --2- 172.21.15.83:0/1238516766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d380039c0 0x7f8d38003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:31.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d4e7fc640 1 --2- 172.21.15.83:0/1238516766 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d500d2340 0x7f8d38004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:31.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d4f7fe640 1 --2- 172.21.15.83:0/1238516766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d38004d50 0x7f8d380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:31.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d4effd640 1 -- 172.21.15.83:0/1238516766 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d500d2340 msgr2=0x7f8d38004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:31.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d4effd640 1 --2- 172.21.15.83:0/1238516766 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d500d2340 0x7f8d38004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d4effd640 1 -- 172.21.15.83:0/1238516766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d38004d50 msgr2=0x7f8d380051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:31.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d4effd640 1 --2- 172.21.15.83:0/1238516766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d38004d50 0x7f8d380051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d4effd640 1 -- 172.21.15.83:0/1238516766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d38003530 con 0x7f8d380039c0 2024-04-17T10:50:31.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d4f7fe640 1 --2- 172.21.15.83:0/1238516766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d38004d50 0x7f8d380051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:31.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.032+0000 7f8d4e7fc640 1 --2- 172.21.15.83:0/1238516766 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d500d2340 0x7f8d38004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:31.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.036+0000 7f8d4effd640 1 --2- 172.21.15.83:0/1238516766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d380039c0 0x7f8d38003e00 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f8d40009ab0 tx=0x7f8d4001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f563f4f995444a77 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:31.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.036+0000 7f8d4dffb640 1 -- 172.21.15.83:0/1238516766 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d400b2070 con 0x7f8d380039c0 2024-04-17T10:50:31.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.036+0000 7f8d4dffb640 1 -- 172.21.15.83:0/1238516766 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d400a5550 con 0x7f8d380039c0 2024-04-17T10:50:31.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.036+0000 7f8d4dffb640 1 -- 172.21.15.83:0/1238516766 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d400096b0 con 0x7f8d380039c0 2024-04-17T10:50:31.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.036+0000 7f8d55696640 1 -- 172.21.15.83:0/1238516766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d380039c0 msgr2=0x7f8d38003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:31.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.036+0000 7f8d55696640 1 --2- 172.21.15.83:0/1238516766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d380039c0 0x7f8d38003e00 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f8d40009ab0 tx=0x7f8d4001ca90 comp rx=0 tx=0).stop 2024-04-17T10:50:31.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.036+0000 7f8d4f7fe640 1 -- 172.21.15.83:0/1238516766 reap_dead start 2024-04-17T10:50:31.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.036+0000 7f8d55696640 1 -- 172.21.15.83:0/1238516766 shutdown_connections 2024-04-17T10:50:31.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.036+0000 7f8d55696640 1 -- 172.21.15.83:0/1238516766 >> 172.21.15.83:0/1238516766 conn(0x7f8d500d1820 msgr2=0x7f8d500d1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:31.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.036+0000 7f8d55696640 1 -- 172.21.15.83:0/1238516766 shutdown_connections 2024-04-17T10:50:31.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.036+0000 7f8d55696640 1 -- 172.21.15.83:0/1238516766 wait complete. 2024-04-17T10:50:31.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.036+0000 7f8d55696640 1 Processor -- start 2024-04-17T10:50:31.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.036+0000 7f8d55696640 1 -- start start 2024-04-17T10:50:31.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.040+0000 7f8d55696640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d380039c0 0x7f8d5015e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:31.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.040+0000 7f8d55696640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d50159360 0x7f8d500af570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:31.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.040+0000 7f8d55696640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d501597e0 0x7f8d500afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:31.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.040+0000 7f8d55696640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d500b0f10 con 0x7f8d501597e0 2024-04-17T10:50:31.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.040+0000 7f8d55696640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d500b0d90 con 0x7f8d50159360 2024-04-17T10:50:31.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.040+0000 7f8d55696640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5015be50 con 0x7f8d380039c0 2024-04-17T10:50:31.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.040+0000 7f8d4f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d501597e0 0x7f8d500afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:31.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.040+0000 7f8d4f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d501597e0 0x7f8d500afab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44972/0 (socket says 172.21.15.83:44972) 2024-04-17T10:50:31.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.040+0000 7f8d4f7fe640 1 -- 172.21.15.83:0/1086991652 learned_addr learned my addr 172.21.15.83:0/1086991652 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:31.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.040+0000 7f8d4e7fc640 1 --2- 172.21.15.83:0/1086991652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d50159360 0x7f8d500af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:31.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.040+0000 7f8d4effd640 1 --2- 172.21.15.83:0/1086991652 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d380039c0 0x7f8d5015e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:31.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.040+0000 7f8d4f7fe640 1 -- 172.21.15.83:0/1086991652 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d380039c0 msgr2=0x7f8d5015e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:31.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.040+0000 7f8d4f7fe640 1 --2- 172.21.15.83:0/1086991652 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d380039c0 0x7f8d5015e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.040+0000 7f8d4f7fe640 1 -- 172.21.15.83:0/1086991652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d50159360 msgr2=0x7f8d500af570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:31.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.040+0000 7f8d4f7fe640 1 --2- 172.21.15.83:0/1086991652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d50159360 0x7f8d500af570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.040+0000 7f8d4f7fe640 1 -- 172.21.15.83:0/1086991652 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d5015c270 con 0x7f8d501597e0 2024-04-17T10:50:31.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.040+0000 7f8d4e7fc640 1 --2- 172.21.15.83:0/1086991652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d50159360 0x7f8d500af570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:31.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.040+0000 7f8d4f7fe640 1 --2- 172.21.15.83:0/1086991652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d501597e0 0x7f8d500afab0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f8d500692e0 tx=0x7f8d48009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:31.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.040+0000 7f8d377fe640 1 -- 172.21.15.83:0/1086991652 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d48019070 con 0x7f8d501597e0 2024-04-17T10:50:31.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.040+0000 7f8d377fe640 1 -- 172.21.15.83:0/1086991652 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d48007ce0 con 0x7f8d501597e0 2024-04-17T10:50:31.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.040+0000 7f8d377fe640 1 -- 172.21.15.83:0/1086991652 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d48014640 con 0x7f8d501597e0 2024-04-17T10:50:31.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.040+0000 7f8d55696640 1 -- 172.21.15.83:0/1086991652 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d500ca660 con 0x7f8d501597e0 2024-04-17T10:50:31.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.040+0000 7f8d55696640 1 -- 172.21.15.83:0/1086991652 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d5015ebd0 con 0x7f8d501597e0 2024-04-17T10:50:31.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.040+0000 7f8d55696640 1 -- 172.21.15.83:0/1086991652 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d5015c530 con 0x7f8d501597e0 2024-04-17T10:50:31.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.040+0000 7f8d377fe640 1 -- 172.21.15.83:0/1086991652 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8d4801c070 con 0x7f8d501597e0 2024-04-17T10:50:31.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.040+0000 7f8d377fe640 1 --2- 172.21.15.83:0/1086991652 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8d3c066f60 0x7f8d3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:31.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.040+0000 7f8d4effd640 1 --2- 172.21.15.83:0/1086991652 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8d3c066f60 0x7f8d3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:31.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.040+0000 7f8d377fe640 1 -- 172.21.15.83:0/1086991652 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8d48084720 con 0x7f8d501597e0 2024-04-17T10:50:31.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.040+0000 7f8d4effd640 1 --2- 172.21.15.83:0/1086991652 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8d3c066f60 0x7f8d3c069420 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f8d400af000 tx=0x7f8d400046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:31.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.044+0000 7f8d377fe640 1 -- 172.21.15.83:0/1086991652 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8d48055a80 con 0x7f8d501597e0 2024-04-17T10:50:31.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.244+0000 7f8d55696640 1 -- 172.21.15.83:0/1086991652 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f8d50074f90 con 0x7f8d501597e0 2024-04-17T10:50:31.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.244+0000 7f8d377fe640 1 -- 172.21.15.83:0/1086991652 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f8d48055380 con 0x7f8d501597e0 2024-04-17T10:50:31.236 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:50:31.236 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:50:31.236 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_dump": [ 2024-04-17T10:50:31.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:50:31.237 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.a", 2024-04-17T10:50:31.237 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mfpxsNxAANpL7rcWevXwd2KMftSQnEQ==", 2024-04-17T10:50:31.237 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:50:31.237 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:50:31.237 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:50:31.237 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:50:31.237 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:50:31.237 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:50:31.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:50:31.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:50:31.237 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.b", 2024-04-17T10:50:31.237 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mXlI9ChAAYdcrGVmtYdtWWgCLVmpTTg==", 2024-04-17T10:50:31.237 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:50:31.237 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:50:31.238 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:50:31.238 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:50:31.238 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:50:31.238 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:50:31.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:50:31.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:50:31.238 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.c", 2024-04-17T10:50:31.238 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mupo3BBAAyKH801vpFLXYs9mfzSlT0w==", 2024-04-17T10:50:31.238 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:50:31.238 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:50:31.238 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:50:31.238 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:50:31.238 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:50:31.238 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:50:31.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:50:31.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:50:31.239 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.d", 2024-04-17T10:50:31.239 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mbm++EhAAIBMOtQoD+kPeMqR6gK5ctw==", 2024-04-17T10:50:31.239 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:50:31.239 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:50:31.239 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:50:31.239 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:50:31.239 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:50:31.239 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:50:31.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:50:31.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:50:31.239 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.0", 2024-04-17T10:50:31.239 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBXph9m1HWhIRAAOq+9nGq7OGonUCE2aeiAzQ==", 2024-04-17T10:50:31.240 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:50:31.240 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:50:31.240 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:50:31.240 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:50:31.240 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:50:31.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:50:31.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:50:31.240 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.1", 2024-04-17T10:50:31.240 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBaph9mlMmtJRAAXHLMjXfm/+WjIFURLkbAfA==", 2024-04-17T10:50:31.240 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:50:31.240 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:50:31.240 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:50:31.240 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:50:31.240 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:50:31.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:50:31.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:50:31.241 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.2", 2024-04-17T10:50:31.241 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBdph9mBsNPLRAAtR5hwGdDkjLDLS19QUDDYQ==", 2024-04-17T10:50:31.241 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:50:31.241 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:50:31.241 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:50:31.241 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:50:31.241 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:50:31.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:50:31.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:50:31.241 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.3", 2024-04-17T10:50:31.241 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBgph9mZ4EwNRAAIyL7e0EK1523dPXmbOOkzw==", 2024-04-17T10:50:31.241 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:50:31.241 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:50:31.241 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:50:31.242 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:50:31.242 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:50:31.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:50:31.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:50:31.242 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.4", 2024-04-17T10:50:31.242 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBmph9mphAsMhAAtF4uV28Nb66PNjuugB12Hw==", 2024-04-17T10:50:31.242 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:50:31.242 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:50:31.242 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:50:31.242 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:50:31.242 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:50:31.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:50:31.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:50:31.243 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.5", 2024-04-17T10:50:31.243 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBpph9mHJQNNxAA72eOFYFbhakQac1JKRNjzw==", 2024-04-17T10:50:31.243 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:50:31.243 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:50:31.243 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:50:31.243 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:50:31.243 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:50:31.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:50:31.243 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:50:31.243 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.6", 2024-04-17T10:50:31.243 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBtph9mb6RKAxAAoVX6Cp7l7j3Lm1ES2Sq+fQ==", 2024-04-17T10:50:31.243 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:50:31.243 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:50:31.243 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:50:31.244 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:50:31.244 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:50:31.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:50:31.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:50:31.244 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.7", 2024-04-17T10:50:31.244 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBwph9mdrdrBxAApDwjml2ahsLbRol4uL/BxA==", 2024-04-17T10:50:31.244 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:50:31.244 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-04-17T10:50:31.244 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-04-17T10:50:31.244 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:50:31.244 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:50:31.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:50:31.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:50:31.244 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.0", 2024-04-17T10:50:31.244 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mHGmXGBAAZF+JiAG9YrFtbW+dCy2IEQ==", 2024-04-17T10:50:31.245 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:50:31.245 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:50:31.245 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow r", 2024-04-17T10:50:31.245 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-17T10:50:31.245 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:50:31.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:50:31.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:50:31.245 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.1", 2024-04-17T10:50:31.245 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mB7wgIBAABdavAo/uzVuSjfkkcYtqJg==", 2024-04-17T10:50:31.245 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:50:31.245 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:50:31.245 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-17T10:50:31.245 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-17T10:50:31.245 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-17T10:50:31.245 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:50:31.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:50:31.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:50:31.246 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.2", 2024-04-17T10:50:31.246 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9m+wKgHRAAe8IUFx34ItLpsaDqSYyEgA==", 2024-04-17T10:50:31.246 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:50:31.246 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:50:31.246 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-17T10:50:31.246 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-17T10:50:31.246 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-17T10:50:31.246 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:50:31.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:50:31.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:50:31.246 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.3", 2024-04-17T10:50:31.246 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBUph9mfJ0yJRAAHH7rRh/BUIujTT04AF2l0w==", 2024-04-17T10:50:31.246 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:50:31.246 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-04-17T10:50:31.247 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-04-17T10:50:31.247 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-04-17T10:50:31.247 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-04-17T10:50:31.247 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:50:31.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:50:31.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:50:31.247 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.admin", 2024-04-17T10:50:31.247 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mjJ3GIBAA/2qaCbRiLJ5X1cv3ZGdfoA==", 2024-04-17T10:50:31.247 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:50:31.247 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-17T10:50:31.247 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-04-17T10:50:31.247 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-04-17T10:50:31.247 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:50:31.247 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:50:31.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:50:31.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:50:31.248 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.x", 2024-04-17T10:50:31.248 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mMFptNBAA/G0zEfMZTCy9Iox+MwHG3Q==", 2024-04-17T10:50:31.248 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:50:31.248 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-17T10:50:31.248 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-04-17T10:50:31.248 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:50:31.248 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:50:31.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:50:31.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:50:31.248 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.y", 2024-04-17T10:50:31.248 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBTph9mjFbRMRAAjY2d8POBzp8hEp9GSO0ylQ==", 2024-04-17T10:50:31.248 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-04-17T10:50:31.248 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-04-17T10:50:31.249 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-04-17T10:50:31.249 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-04-17T10:50:31.249 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:50:31.249 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:50:31.249 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-17T10:50:31.249 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:50:31.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.244+0000 7f8d55696640 1 -- 172.21.15.83:0/1086991652 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8d3c066f60 msgr2=0x7f8d3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:31.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.244+0000 7f8d55696640 1 --2- 172.21.15.83:0/1086991652 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8d3c066f60 0x7f8d3c069420 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f8d400af000 tx=0x7f8d400046d0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.244+0000 7f8d55696640 1 -- 172.21.15.83:0/1086991652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d501597e0 msgr2=0x7f8d500afab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:31.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.244+0000 7f8d55696640 1 --2- 172.21.15.83:0/1086991652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d501597e0 0x7f8d500afab0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f8d500692e0 tx=0x7f8d48009cd0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.244+0000 7f8d55696640 1 -- 172.21.15.83:0/1086991652 shutdown_connections 2024-04-17T10:50:31.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.244+0000 7f8d55696640 1 --2- 172.21.15.83:0/1086991652 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8d3c066f60 0x7f8d3c069420 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.244+0000 7f8d55696640 1 --2- 172.21.15.83:0/1086991652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d501597e0 0x7f8d500afab0 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.244+0000 7f8d55696640 1 --2- 172.21.15.83:0/1086991652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d50159360 0x7f8d500af570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.244+0000 7f8d55696640 1 --2- 172.21.15.83:0/1086991652 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d380039c0 0x7f8d5015e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.244+0000 7f8d55696640 1 -- 172.21.15.83:0/1086991652 >> 172.21.15.83:0/1086991652 conn(0x7f8d500d1820 msgr2=0x7f8d500b1180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:31.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.244+0000 7f8d55696640 1 -- 172.21.15.83:0/1086991652 shutdown_connections 2024-04-17T10:50:31.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.244+0000 7f8d55696640 1 -- 172.21.15.83:0/1086991652 wait complete. 2024-04-17T10:50:31.254 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:50:31.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.368+0000 7faa0cdaa640 1 Processor -- start 2024-04-17T10:50:31.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.368+0000 7faa0cdaa640 1 -- start start 2024-04-17T10:50:31.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa0cdaa640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faa0815ba30 0x7faa08072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:31.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa0cdaa640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa0806e320 con 0x7faa080cfbe0 2024-04-17T10:50:31.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa0cdaa640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa0806e040 con 0x7faa0815c960 2024-04-17T10:50:31.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa0cdaa640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa08071ea0 con 0x7faa0815ba30 2024-04-17T10:50:31.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa06d9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7faa080cfbe0 0x7faa0806c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56296/0 (socket says 172.21.15.83:56296) 2024-04-17T10:50:31.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa06d9d640 1 -- 172.21.15.83:0/2129264773 learned_addr learned my addr 172.21.15.83:0/2129264773 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:31.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa0659c640 1 --2- 172.21.15.83:0/2129264773 >> v2:172.21.15.167:3300/0 conn(0x7faa0815ba30 0x7faa08072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:31.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa0559a640 1 -- 172.21.15.83:0/2129264773 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2234237962 0 0) 0x7faa0806e320 con 0x7faa080cfbe0 2024-04-17T10:50:31.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa0559a640 1 -- 172.21.15.83:0/2129264773 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9f0003530 con 0x7faa080cfbe0 2024-04-17T10:50:31.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa0559a640 1 -- 172.21.15.83:0/2129264773 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1836438695 0 0) 0x7fa9f0003530 con 0x7faa080cfbe0 2024-04-17T10:50:31.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa0559a640 1 -- 172.21.15.83:0/2129264773 >> v2:172.21.15.167:3300/0 conn(0x7faa0815ba30 msgr2=0x7faa08072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:31.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa0559a640 1 --2- 172.21.15.83:0/2129264773 >> v2:172.21.15.167:3300/0 conn(0x7faa0815ba30 0x7faa08072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa0559a640 1 -- 172.21.15.83:0/2129264773 >> v1:172.21.15.167:6789/0 conn(0x7faa0815c960 legacy=0x7faa080cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:31.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa0559a640 1 -- 172.21.15.83:0/2129264773 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa08071850 con 0x7faa080cfbe0 2024-04-17T10:50:31.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa0559a640 1 -- 172.21.15.83:0/2129264773 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fa9f80016a0 con 0x7faa080cfbe0 2024-04-17T10:50:31.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa0559a640 1 -- 172.21.15.83:0/2129264773 >> v1:172.21.15.83:6789/0 conn(0x7faa080cfbe0 legacy=0x7faa0806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:31.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa0559a640 1 --2- 172.21.15.83:0/2129264773 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9f00039c0 0x7fa9f0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:31.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa0559a640 1 --2- 172.21.15.83:0/2129264773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9f0004810 0x7fa9f0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:31.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa0559a640 1 --2- 172.21.15.83:0/2129264773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9f00051d0 0x7fa9f000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:31.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa0559a640 1 -- 172.21.15.83:0/2129264773 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa08071ea0 con 0x7fa9f00051d0 2024-04-17T10:50:31.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa0559a640 1 -- 172.21.15.83:0/2129264773 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f000c170 con 0x7fa9f0004810 2024-04-17T10:50:31.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa0559a640 1 -- 172.21.15.83:0/2129264773 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f000c370 con 0x7fa9f00039c0 2024-04-17T10:50:31.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa06d9d640 1 --2- 172.21.15.83:0/2129264773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9f00051d0 0x7fa9f000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:31.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa06d9d640 1 -- 172.21.15.83:0/2129264773 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9f00039c0 msgr2=0x7fa9f0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:31.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa06d9d640 1 --2- 172.21.15.83:0/2129264773 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9f00039c0 0x7fa9f0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa06d9d640 1 -- 172.21.15.83:0/2129264773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9f0004810 msgr2=0x7fa9f0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:31.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa06d9d640 1 --2- 172.21.15.83:0/2129264773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9f0004810 0x7fa9f0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa06d9d640 1 -- 172.21.15.83:0/2129264773 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f0003530 con 0x7fa9f00051d0 2024-04-17T10:50:31.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa06d9d640 1 -- 172.21.15.83:0/2129264773 reap_dead start 2024-04-17T10:50:31.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa06d9d640 1 --2- 172.21.15.83:0/2129264773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9f00051d0 0x7fa9f000b9d0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800c210 tx=0x7fa9f801f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd6de2e1fcf389 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:31.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa0559a640 1 -- 172.21.15.83:0/2129264773 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8007d40 con 0x7fa9f00051d0 2024-04-17T10:50:31.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa0559a640 1 -- 172.21.15.83:0/2129264773 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f8005b00 con 0x7fa9f00051d0 2024-04-17T10:50:31.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.372+0000 7faa0559a640 1 -- 172.21.15.83:0/2129264773 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8007070 con 0x7fa9f00051d0 2024-04-17T10:50:31.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.376+0000 7faa0cdaa640 1 -- 172.21.15.83:0/2129264773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9f00051d0 msgr2=0x7fa9f000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:31.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.376+0000 7faa0cdaa640 1 --2- 172.21.15.83:0/2129264773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9f00051d0 0x7fa9f000b9d0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800c210 tx=0x7fa9f801f3b0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.376+0000 7faa0cdaa640 1 -- 172.21.15.83:0/2129264773 shutdown_connections 2024-04-17T10:50:31.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.376+0000 7faa0cdaa640 1 --2- 172.21.15.83:0/2129264773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9f00051d0 0x7fa9f000b9d0 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.376+0000 7faa0cdaa640 1 -- 172.21.15.83:0/2129264773 >> 172.21.15.83:0/2129264773 conn(0x7faa0806abe0 msgr2=0x7faa0806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:31.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.376+0000 7faa0cdaa640 1 -- 172.21.15.83:0/2129264773 shutdown_connections 2024-04-17T10:50:31.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.376+0000 7faa0cdaa640 1 -- 172.21.15.83:0/2129264773 wait complete. 2024-04-17T10:50:31.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.376+0000 7faa0cdaa640 1 Processor -- start 2024-04-17T10:50:31.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.376+0000 7faa0cdaa640 1 -- start start 2024-04-17T10:50:31.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.376+0000 7faa0cdaa640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9f00051d0 0x7faa080b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:31.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.376+0000 7faa0cdaa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa08156c00 0x7faa08157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:31.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.376+0000 7faa0cdaa640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa08150f40 0x7faa081513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:31.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.376+0000 7faa0cdaa640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa080aeae0 con 0x7faa08156c00 2024-04-17T10:50:31.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.376+0000 7faa0cdaa640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0815cde0 con 0x7faa08150f40 2024-04-17T10:50:31.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.376+0000 7faa0cdaa640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa0806dca0 con 0x7fa9f00051d0 2024-04-17T10:50:31.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.376+0000 7faa05d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa08156c00 0x7faa08157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:31.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.376+0000 7faa05d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa08156c00 0x7faa08157080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44990/0 (socket says 172.21.15.83:44990) 2024-04-17T10:50:31.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.376+0000 7faa05d9b640 1 -- 172.21.15.83:0/708645219 learned_addr learned my addr 172.21.15.83:0/708645219 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:31.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.376+0000 7faa0659c640 1 --2- 172.21.15.83:0/708645219 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9f00051d0 0x7faa080b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:31.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.376+0000 7faa06d9d640 1 --2- 172.21.15.83:0/708645219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa08150f40 0x7faa081513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:31.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.376+0000 7faa05d9b640 1 -- 172.21.15.83:0/708645219 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9f00051d0 msgr2=0x7faa080b0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:31.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.376+0000 7faa05d9b640 1 --2- 172.21.15.83:0/708645219 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9f00051d0 0x7faa080b0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.376+0000 7faa05d9b640 1 -- 172.21.15.83:0/708645219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa08150f40 msgr2=0x7faa081513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:31.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.376+0000 7faa05d9b640 1 --2- 172.21.15.83:0/708645219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa08150f40 0x7faa081513c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.376+0000 7faa05d9b640 1 -- 172.21.15.83:0/708645219 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa080aed00 con 0x7faa08156c00 2024-04-17T10:50:31.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.376+0000 7faa0659c640 1 --2- 172.21.15.83:0/708645219 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9f00051d0 0x7faa080b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:31.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.376+0000 7faa05d9b640 1 --2- 172.21.15.83:0/708645219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa08156c00 0x7faa08157080 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc00c560 tx=0x7fa9fc00ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:31.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.376+0000 7fa9f6ffd640 1 -- 172.21.15.83:0/708645219 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc0052d0 con 0x7faa08156c00 2024-04-17T10:50:31.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.376+0000 7fa9f6ffd640 1 -- 172.21.15.83:0/708645219 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9fc0054b0 con 0x7faa08156c00 2024-04-17T10:50:31.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.376+0000 7faa0cdaa640 1 -- 172.21.15.83:0/708645219 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa080aefe0 con 0x7faa08156c00 2024-04-17T10:50:31.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.376+0000 7fa9f6ffd640 1 -- 172.21.15.83:0/708645219 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc012870 con 0x7faa08156c00 2024-04-17T10:50:31.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.376+0000 7faa0cdaa640 1 -- 172.21.15.83:0/708645219 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa080af5a0 con 0x7faa08156c00 2024-04-17T10:50:31.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.380+0000 7faa0cdaa640 1 -- 172.21.15.83:0/708645219 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa08074f90 con 0x7faa08156c00 2024-04-17T10:50:31.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.380+0000 7fa9f6ffd640 1 -- 172.21.15.83:0/708645219 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa9fc012a10 con 0x7faa08156c00 2024-04-17T10:50:31.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.380+0000 7fa9f6ffd640 1 --2- 172.21.15.83:0/708645219 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa9e8066f60 0x7fa9e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:31.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.380+0000 7fa9f6ffd640 1 -- 172.21.15.83:0/708645219 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa9fc0848b0 con 0x7faa08156c00 2024-04-17T10:50:31.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.380+0000 7faa0659c640 1 --2- 172.21.15.83:0/708645219 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa9e8066f60 0x7fa9e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:31.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.380+0000 7faa0659c640 1 --2- 172.21.15.83:0/708645219 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa9e8066f60 0x7fa9e8069420 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7faa00004b40 tx=0x7faa0000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:31.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.384+0000 7fa9f6ffd640 1 -- 172.21.15.83:0/708645219 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa9fc055c10 con 0x7faa08156c00 2024-04-17T10:50:31.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.536+0000 7faa0cdaa640 1 -- 172.21.15.83:0/708645219 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa080cf7f0 con 0x7faa08156c00 2024-04-17T10:50:31.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.540+0000 7fa9f6ffd640 1 -- 172.21.15.83:0/708645219 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+12829 (secure 0 0 0) 0x7fa9fc055510 con 0x7faa08156c00 2024-04-17T10:50:31.535 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:50:31.535 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":136,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:50:30.848073+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:50:31.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.544+0000 7faa0cdaa640 1 -- 172.21.15.83:0/708645219 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa9e8066f60 msgr2=0x7fa9e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:31.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.544+0000 7faa0cdaa640 1 --2- 172.21.15.83:0/708645219 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa9e8066f60 0x7fa9e8069420 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7faa00004b40 tx=0x7faa0000b040 comp rx=0 tx=0).stop 2024-04-17T10:50:31.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.544+0000 7faa0cdaa640 1 -- 172.21.15.83:0/708645219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa08156c00 msgr2=0x7faa08157080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:31.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.544+0000 7faa0cdaa640 1 --2- 172.21.15.83:0/708645219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa08156c00 0x7faa08157080 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc00c560 tx=0x7fa9fc00ca20 comp rx=0 tx=0).stop 2024-04-17T10:50:31.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.544+0000 7faa0cdaa640 1 -- 172.21.15.83:0/708645219 shutdown_connections 2024-04-17T10:50:31.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.544+0000 7faa0cdaa640 1 --2- 172.21.15.83:0/708645219 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa9e8066f60 0x7fa9e8069420 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.544+0000 7faa0cdaa640 1 --2- 172.21.15.83:0/708645219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa08150f40 0x7faa081513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.544+0000 7faa0cdaa640 1 --2- 172.21.15.83:0/708645219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa08156c00 0x7faa08157080 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.544+0000 7faa0cdaa640 1 --2- 172.21.15.83:0/708645219 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9f00051d0 0x7faa080b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.544+0000 7faa0cdaa640 1 -- 172.21.15.83:0/708645219 >> 172.21.15.83:0/708645219 conn(0x7faa0806abe0 msgr2=0x7faa08158f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:31.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.544+0000 7faa0cdaa640 1 -- 172.21.15.83:0/708645219 shutdown_connections 2024-04-17T10:50:31.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.544+0000 7faa0cdaa640 1 -- 172.21.15.83:0/708645219 wait complete. 2024-04-17T10:50:31.552 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:50:31.553 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:50:31.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.680+0000 7f1327adf640 1 Processor -- start 2024-04-17T10:50:31.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.680+0000 7f1327adf640 1 -- start start 2024-04-17T10:50:31.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.680+0000 7f1327adf640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f132006ae10 0x7f132006b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:31.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.680+0000 7f1327adf640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f132015e330 0x7f132015e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:31.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.680+0000 7f1327adf640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13201625d0 con 0x7f13200ad840 2024-04-17T10:50:31.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.680+0000 7f1327adf640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f132006b730 con 0x7f132015e330 2024-04-17T10:50:31.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.680+0000 7f1327adf640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1320161810 con 0x7f132006ae10 2024-04-17T10:50:31.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.680+0000 7f132507a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f132015e330 0x7f132015e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:31.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.680+0000 7f132607c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f13200ad840 0x7f13200adcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56300/0 (socket says 172.21.15.83:56300) 2024-04-17T10:50:31.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.680+0000 7f132607c640 1 -- 172.21.15.83:0/718298421 learned_addr learned my addr 172.21.15.83:0/718298421 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:31.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.680+0000 7f132587b640 1 --2- 172.21.15.83:0/718298421 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f132006ae10 0x7f132006b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:31.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.680+0000 7f132507a640 1 -- 172.21.15.83:0/718298421 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f132006ae10 msgr2=0x7f132006b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:31.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.680+0000 7f132507a640 1 --2- 172.21.15.83:0/718298421 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f132006ae10 0x7f132006b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.680+0000 7f132507a640 1 -- 172.21.15.83:0/718298421 >> v1:172.21.15.83:6789/0 conn(0x7f13200ad840 legacy=0x7f13200adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:31.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.680+0000 7f1324879640 1 -- 172.21.15.83:0/718298421 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3228500563 0 0) 0x7f13201625d0 con 0x7f13200ad840 2024-04-17T10:50:31.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.680+0000 7f132507a640 1 -- 172.21.15.83:0/718298421 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1320161af0 con 0x7f132015e330 2024-04-17T10:50:31.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.684+0000 7f132507a640 1 --2- 172.21.15.83:0/718298421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f132015e330 0x7f132015e770 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f13140097c0 tx=0x7f131401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c4c8a7bb910828 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:31.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.684+0000 7f1324879640 1 -- 172.21.15.83:0/718298421 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13140b0070 con 0x7f132015e330 2024-04-17T10:50:31.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.684+0000 7f1324879640 1 -- 172.21.15.83:0/718298421 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13140ab600 con 0x7f132015e330 2024-04-17T10:50:31.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.684+0000 7f1324879640 1 -- 172.21.15.83:0/718298421 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13140ab9d0 con 0x7f132015e330 2024-04-17T10:50:31.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.684+0000 7f1327adf640 1 -- 172.21.15.83:0/718298421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f132015e330 msgr2=0x7f132015e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:31.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.684+0000 7f1327adf640 1 --2- 172.21.15.83:0/718298421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f132015e330 0x7f132015e770 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f13140097c0 tx=0x7f131401c410 comp rx=0 tx=0).stop 2024-04-17T10:50:31.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.684+0000 7f1327adf640 1 -- 172.21.15.83:0/718298421 shutdown_connections 2024-04-17T10:50:31.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.684+0000 7f1327adf640 1 --2- 172.21.15.83:0/718298421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f132015e330 0x7f132015e770 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.684+0000 7f1327adf640 1 --2- 172.21.15.83:0/718298421 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f132006ae10 0x7f132006b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.684+0000 7f1327adf640 1 -- 172.21.15.83:0/718298421 >> 172.21.15.83:0/718298421 conn(0x7f13200b15b0 msgr2=0x7f13200b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:31.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.684+0000 7f1327adf640 1 -- 172.21.15.83:0/718298421 shutdown_connections 2024-04-17T10:50:31.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.684+0000 7f1327adf640 1 -- 172.21.15.83:0/718298421 wait complete. 2024-04-17T10:50:31.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.688+0000 7f1327adf640 1 Processor -- start 2024-04-17T10:50:31.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.688+0000 7f1327adf640 1 -- start start 2024-04-17T10:50:31.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.688+0000 7f1327adf640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f132006ae10 0x7f13201439d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:31.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.688+0000 7f1327adf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13200ad840 0x7f132015d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:31.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.688+0000 7f1327adf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f132015e330 0x7f132015a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:31.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.688+0000 7f1327adf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13200cf470 con 0x7f132015e330 2024-04-17T10:50:31.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.688+0000 7f1327adf640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13200cf2f0 con 0x7f13200ad840 2024-04-17T10:50:31.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.688+0000 7f1327adf640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f132015ecb0 con 0x7f132006ae10 2024-04-17T10:50:31.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.688+0000 7f132607c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f132015e330 0x7f132015a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:31.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.688+0000 7f132607c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f132015e330 0x7f132015a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45008/0 (socket says 172.21.15.83:45008) 2024-04-17T10:50:31.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.688+0000 7f132607c640 1 -- 172.21.15.83:0/19369094 learned_addr learned my addr 172.21.15.83:0/19369094 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:31.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.688+0000 7f132607c640 1 -- 172.21.15.83:0/19369094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f132006ae10 msgr2=0x7f13201439d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:31.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.688+0000 7f132587b640 1 --2- 172.21.15.83:0/19369094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f132006ae10 0x7f13201439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:31.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.688+0000 7f132607c640 1 --2- 172.21.15.83:0/19369094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f132006ae10 0x7f13201439d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.688+0000 7f132507a640 1 --2- 172.21.15.83:0/19369094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13200ad840 0x7f132015d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:31.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.688+0000 7f132607c640 1 -- 172.21.15.83:0/19369094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13200ad840 msgr2=0x7f132015d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:31.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.688+0000 7f132607c640 1 --2- 172.21.15.83:0/19369094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13200ad840 0x7f132015d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.688+0000 7f132607c640 1 -- 172.21.15.83:0/19369094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f132015c8d0 con 0x7f132015e330 2024-04-17T10:50:31.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.688+0000 7f132607c640 1 --2- 172.21.15.83:0/19369094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f132015e330 0x7f132015a610 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f131000c7b0 tx=0x7f131000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:31.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.688+0000 7f131e7fc640 1 -- 172.21.15.83:0/19369094 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1310018070 con 0x7f132015e330 2024-04-17T10:50:31.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.688+0000 7f131e7fc640 1 -- 172.21.15.83:0/19369094 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f131000a640 con 0x7f132015e330 2024-04-17T10:50:31.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.688+0000 7f1327adf640 1 -- 172.21.15.83:0/19369094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f132015cbb0 con 0x7f132015e330 2024-04-17T10:50:31.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.688+0000 7f131e7fc640 1 -- 172.21.15.83:0/19369094 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131000aa30 con 0x7f132015e330 2024-04-17T10:50:31.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.688+0000 7f1327adf640 1 -- 172.21.15.83:0/19369094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f132015d170 con 0x7f132015e330 2024-04-17T10:50:31.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.688+0000 7f1327adf640 1 -- 172.21.15.83:0/19369094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1320074f90 con 0x7f132015e330 2024-04-17T10:50:31.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.688+0000 7f131e7fc640 1 -- 172.21.15.83:0/19369094 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f131001b070 con 0x7f132015e330 2024-04-17T10:50:31.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.688+0000 7f131e7fc640 1 --2- 172.21.15.83:0/19369094 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f130c066f60 0x7f130c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:31.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.688+0000 7f132587b640 1 --2- 172.21.15.83:0/19369094 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f130c066f60 0x7f130c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:31.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.688+0000 7f131e7fc640 1 -- 172.21.15.83:0/19369094 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1310084220 con 0x7f132015e330 2024-04-17T10:50:31.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.692+0000 7f132587b640 1 --2- 172.21.15.83:0/19369094 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f130c066f60 0x7f130c069420 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f132015b7d0 tx=0x7f1318008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:31.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.696+0000 7f131e7fc640 1 -- 172.21.15.83:0/19369094 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1310055610 con 0x7f132015e330 2024-04-17T10:50:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.840+0000 7f1327adf640 1 -- 172.21.15.83:0/19369094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1320072740 con 0x7f132015e330 2024-04-17T10:50:31.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.840+0000 7f131e7fc640 1 -- 172.21.15.83:0/19369094 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v136) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1310054f10 con 0x7f132015e330 2024-04-17T10:50:31.834 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:50:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.844+0000 7f1327adf640 1 -- 172.21.15.83:0/19369094 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f130c066f60 msgr2=0x7f130c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.844+0000 7f1327adf640 1 --2- 172.21.15.83:0/19369094 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f130c066f60 0x7f130c069420 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f132015b7d0 tx=0x7f1318008040 comp rx=0 tx=0).stop 2024-04-17T10:50:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.844+0000 7f1327adf640 1 -- 172.21.15.83:0/19369094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f132015e330 msgr2=0x7f132015a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.844+0000 7f1327adf640 1 --2- 172.21.15.83:0/19369094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f132015e330 0x7f132015a610 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f131000c7b0 tx=0x7f131000cc70 comp rx=0 tx=0).stop 2024-04-17T10:50:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.844+0000 7f1327adf640 1 -- 172.21.15.83:0/19369094 shutdown_connections 2024-04-17T10:50:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.844+0000 7f1327adf640 1 --2- 172.21.15.83:0/19369094 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f130c066f60 0x7f130c069420 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.844+0000 7f1327adf640 1 --2- 172.21.15.83:0/19369094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f132015e330 0x7f132015a610 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.844+0000 7f1327adf640 1 --2- 172.21.15.83:0/19369094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13200ad840 0x7f132015d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.844+0000 7f1327adf640 1 --2- 172.21.15.83:0/19369094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f132006ae10 0x7f13201439d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.844+0000 7f1327adf640 1 -- 172.21.15.83:0/19369094 >> 172.21.15.83:0/19369094 conn(0x7f13200b15b0 msgr2=0x7f132006dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.844+0000 7f1327adf640 1 -- 172.21.15.83:0/19369094 shutdown_connections 2024-04-17T10:50:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.844+0000 7f1327adf640 1 -- 172.21.15.83:0/19369094 wait complete. 2024-04-17T10:50:31.853 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-17T10:50:31.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.980+0000 7f53d8e5b640 1 Processor -- start 2024-04-17T10:50:31.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.980+0000 7f53d8e5b640 1 -- start start 2024-04-17T10:50:31.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.980+0000 7f53d8e5b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f53d4072330 0x7f53d40cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:31.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.980+0000 7f53d8e5b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f53d40cfa10 0x7f53d406c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:31.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.980+0000 7f53d8e5b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d40cfea0 con 0x7f53d415ba30 2024-04-17T10:50:31.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.980+0000 7f53d8e5b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53d4071ea0 con 0x7f53d40cfa10 2024-04-17T10:50:31.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.980+0000 7f53d8e5b640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53d406e320 con 0x7f53d4072330 2024-04-17T10:50:31.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.980+0000 7f53d259c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f53d415ba30 0x7f53d415c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56308/0 (socket says 172.21.15.83:56308) 2024-04-17T10:50:31.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.980+0000 7f53d259c640 1 -- 172.21.15.83:0/1520744389 learned_addr learned my addr 172.21.15.83:0/1520744389 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:31.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.980+0000 7f53d2d9d640 1 --2- 172.21.15.83:0/1520744389 >> v2:172.21.15.83:3300/0 conn(0x7f53d40cfa10 0x7f53d406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:31.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.980+0000 7f53c9d9b640 1 --2- 172.21.15.83:0/1520744389 >> v2:172.21.15.167:3300/0 conn(0x7f53d4072330 0x7f53d40cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:31.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.980+0000 7f53d1d9b640 1 -- 172.21.15.83:0/1520744389 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3362912385 0 0) 0x7f53d40cfea0 con 0x7f53d415ba30 2024-04-17T10:50:31.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.980+0000 7f53d1d9b640 1 -- 172.21.15.83:0/1520744389 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53c0003530 con 0x7f53d415ba30 2024-04-17T10:50:31.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.980+0000 7f53d2d9d640 1 -- 172.21.15.83:0/1520744389 >> v2:172.21.15.167:3300/0 conn(0x7f53d4072330 msgr2=0x7f53d40cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:31.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.980+0000 7f53d2d9d640 1 --2- 172.21.15.83:0/1520744389 >> v2:172.21.15.167:3300/0 conn(0x7f53d4072330 0x7f53d40cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.980+0000 7f53d2d9d640 1 -- 172.21.15.83:0/1520744389 >> v1:172.21.15.83:6789/0 conn(0x7f53d415ba30 legacy=0x7f53d415c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:31.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.980+0000 7f53d2d9d640 1 -- 172.21.15.83:0/1520744389 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d406e040 con 0x7f53d40cfa10 2024-04-17T10:50:31.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.984+0000 7f53d2d9d640 1 --2- 172.21.15.83:0/1520744389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53d40cfa10 0x7f53d406c490 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f53d40af2b0 tx=0x7f53cc01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b085b24f725a91 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:31.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.984+0000 7f53d1d9b640 1 -- 172.21.15.83:0/1520744389 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc0b2070 con 0x7f53d40cfa10 2024-04-17T10:50:31.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.984+0000 7f53d1d9b640 1 -- 172.21.15.83:0/1520744389 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53cc0a5870 con 0x7f53d40cfa10 2024-04-17T10:50:31.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.984+0000 7f53d1d9b640 1 -- 172.21.15.83:0/1520744389 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc0a5c40 con 0x7f53d40cfa10 2024-04-17T10:50:31.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.984+0000 7f53d8e5b640 1 -- 172.21.15.83:0/1520744389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53d40cfa10 msgr2=0x7f53d406c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:31.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.984+0000 7f53d8e5b640 1 --2- 172.21.15.83:0/1520744389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53d40cfa10 0x7f53d406c490 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f53d40af2b0 tx=0x7f53cc01c8a0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.984+0000 7f53d8e5b640 1 -- 172.21.15.83:0/1520744389 shutdown_connections 2024-04-17T10:50:31.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.984+0000 7f53d8e5b640 1 --2- 172.21.15.83:0/1520744389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53d40cfa10 0x7f53d406c490 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.984+0000 7f53d8e5b640 1 --2- 172.21.15.83:0/1520744389 >> v2:172.21.15.167:3300/0 conn(0x7f53d4072330 0x7f53d40cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.984+0000 7f53d8e5b640 1 -- 172.21.15.83:0/1520744389 >> 172.21.15.83:0/1520744389 conn(0x7f53d406abe0 msgr2=0x7f53d406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:31.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.984+0000 7f53d8e5b640 1 -- 172.21.15.83:0/1520744389 shutdown_connections 2024-04-17T10:50:31.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.984+0000 7f53d8e5b640 1 -- 172.21.15.83:0/1520744389 wait complete. 2024-04-17T10:50:31.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.984+0000 7f53d8e5b640 1 Processor -- start 2024-04-17T10:50:31.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.984+0000 7f53d8e5b640 1 -- start start 2024-04-17T10:50:31.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.984+0000 7f53d8e5b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53d4072330 0x7f53d40b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:31.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.984+0000 7f53d8e5b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53d40cfa10 0x7f53d4158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:31.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.984+0000 7f53d8e5b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53d415ba30 0x7f53d41393e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:31.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.984+0000 7f53d8e5b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d406de20 con 0x7f53d40cfa10 2024-04-17T10:50:31.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.984+0000 7f53d8e5b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d406dca0 con 0x7f53d4072330 2024-04-17T10:50:31.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.984+0000 7f53d8e5b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d4168520 con 0x7f53d415ba30 2024-04-17T10:50:31.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.984+0000 7f53c9d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53d40cfa10 0x7f53d4158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:31.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.984+0000 7f53c9d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53d40cfa10 0x7f53d4158620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45030/0 (socket says 172.21.15.83:45030) 2024-04-17T10:50:31.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.984+0000 7f53c9d9b640 1 -- 172.21.15.83:0/3206197409 learned_addr learned my addr 172.21.15.83:0/3206197409 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:31.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.984+0000 7f53c9d9b640 1 -- 172.21.15.83:0/3206197409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53d415ba30 msgr2=0x7f53d41393e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:31.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.984+0000 7f53c9d9b640 1 --2- 172.21.15.83:0/3206197409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53d415ba30 0x7f53d41393e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.984+0000 7f53c9d9b640 1 -- 172.21.15.83:0/3206197409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53d4072330 msgr2=0x7f53d40b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:31.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.984+0000 7f53d259c640 1 --2- 172.21.15.83:0/3206197409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53d4072330 0x7f53d40b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:31.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.984+0000 7f53c9d9b640 1 --2- 172.21.15.83:0/3206197409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53d4072330 0x7f53d40b03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:31.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.984+0000 7f53c9d9b640 1 -- 172.21.15.83:0/3206197409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d4139920 con 0x7f53d40cfa10 2024-04-17T10:50:31.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.988+0000 7f53c9d9b640 1 --2- 172.21.15.83:0/3206197409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53d40cfa10 0x7f53d4158620 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f53bc00d620 tx=0x7f53bc00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:31.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.988+0000 7f53cb7fe640 1 -- 172.21.15.83:0/3206197409 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc00b620 con 0x7f53d40cfa10 2024-04-17T10:50:31.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.988+0000 7f53cb7fe640 1 -- 172.21.15.83:0/3206197409 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53bc019070 con 0x7f53d40cfa10 2024-04-17T10:50:31.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.988+0000 7f53d8e5b640 1 -- 172.21.15.83:0/3206197409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d4158db0 con 0x7f53d40cfa10 2024-04-17T10:50:31.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.988+0000 7f53cb7fe640 1 -- 172.21.15.83:0/3206197409 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc00ba20 con 0x7f53d40cfa10 2024-04-17T10:50:31.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.988+0000 7f53d8e5b640 1 -- 172.21.15.83:0/3206197409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d4159370 con 0x7f53d40cfa10 2024-04-17T10:50:31.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.988+0000 7f53d8e5b640 1 -- 172.21.15.83:0/3206197409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53d4074f90 con 0x7f53d40cfa10 2024-04-17T10:50:31.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.988+0000 7f53cb7fe640 1 -- 172.21.15.83:0/3206197409 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f53bc00bbc0 con 0x7f53d40cfa10 2024-04-17T10:50:31.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.988+0000 7f53cb7fe640 1 --2- 172.21.15.83:0/3206197409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f53b4066f90 0x7f53b4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:31.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.988+0000 7f53cb7fe640 1 -- 172.21.15.83:0/3206197409 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f53bc085090 con 0x7f53d40cfa10 2024-04-17T10:50:31.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.988+0000 7f53d259c640 1 --2- 172.21.15.83:0/3206197409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f53b4066f90 0x7f53b4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:31.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.988+0000 7f53d259c640 1 --2- 172.21.15.83:0/3206197409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f53b4066f90 0x7f53b4069450 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f53c4007fd0 tx=0x7f53c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:31.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:31.992+0000 7f53cb7fe640 1 -- 172.21.15.83:0/3206197409 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f53bc056480 con 0x7f53d40cfa10 2024-04-17T10:50:32.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.140+0000 7f53d8e5b640 1 -- 172.21.15.83:0/3206197409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f53d41563f0 con 0x7f53d40cfa10 2024-04-17T10:50:32.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.140+0000 7f53cb7fe640 1 -- 172.21.15.83:0/3206197409 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v136) v1 ==== 84+0+65 (secure 0 0 0) 0x7f53bc055d80 con 0x7f53d40cfa10 2024-04-17T10:50:32.133 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:50:32.133 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-04-17T10:50:32.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-17T10:50:32.134 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2024-04-17T10:50:32.134 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2024-04-17T10:50:32.134 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:50:32.134 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-04-17T10:50:32.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.144+0000 7f53d8e5b640 1 -- 172.21.15.83:0/3206197409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f53b4066f90 msgr2=0x7f53b4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:32.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.144+0000 7f53d8e5b640 1 --2- 172.21.15.83:0/3206197409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f53b4066f90 0x7f53b4069450 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f53c4007fd0 tx=0x7f53c4009040 comp rx=0 tx=0).stop 2024-04-17T10:50:32.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.144+0000 7f53d8e5b640 1 -- 172.21.15.83:0/3206197409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53d40cfa10 msgr2=0x7f53d4158620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:32.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.144+0000 7f53d8e5b640 1 --2- 172.21.15.83:0/3206197409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53d40cfa10 0x7f53d4158620 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f53bc00d620 tx=0x7f53bc00dae0 comp rx=0 tx=0).stop 2024-04-17T10:50:32.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.144+0000 7f53d8e5b640 1 -- 172.21.15.83:0/3206197409 shutdown_connections 2024-04-17T10:50:32.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.144+0000 7f53d8e5b640 1 --2- 172.21.15.83:0/3206197409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f53b4066f90 0x7f53b4069450 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:32.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.144+0000 7f53d8e5b640 1 --2- 172.21.15.83:0/3206197409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53d415ba30 0x7f53d41393e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:32.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.144+0000 7f53d8e5b640 1 --2- 172.21.15.83:0/3206197409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53d40cfa10 0x7f53d4158620 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:32.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.144+0000 7f53d8e5b640 1 --2- 172.21.15.83:0/3206197409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53d4072330 0x7f53d40b03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:32.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.144+0000 7f53d8e5b640 1 -- 172.21.15.83:0/3206197409 >> 172.21.15.83:0/3206197409 conn(0x7f53d406abe0 msgr2=0x7f53d415e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:32.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.144+0000 7f53d8e5b640 1 -- 172.21.15.83:0/3206197409 shutdown_connections 2024-04-17T10:50:32.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.144+0000 7f53d8e5b640 1 -- 172.21.15.83:0/3206197409 wait complete. 2024-04-17T10:50:32.152 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-17T10:50:32.152 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-17T10:50:32.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.276+0000 7fbdc3837640 1 Processor -- start 2024-04-17T10:50:32.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.276+0000 7fbdc3837640 1 -- start start 2024-04-17T10:50:32.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.276+0000 7fbdc3837640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbdbc0afef0 0x7fbdbc15e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:32.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.276+0000 7fbdc3837640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdbc0b4980 0x7fbdbc15b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:32.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.276+0000 7fbdc3837640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdbc0b4fd0 con 0x7fbdbc06dfc0 2024-04-17T10:50:32.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.276+0000 7fbdc3837640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdbc15e8f0 con 0x7fbdbc0afef0 2024-04-17T10:50:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.276+0000 7fbdc3837640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc15b060 con 0x7fbdbc0b4980 2024-04-17T10:50:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.276+0000 7fbdc15d3640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbdbc0afef0 0x7fbdbc15e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.276+0000 7fbdc1dd4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdbc0b4980 0x7fbdbc15b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.276+0000 7fbdc0dd2640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fbdbc06dfc0 0x7fbdbc0b4270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:58950/0 (socket says 172.21.15.83:58950) 2024-04-17T10:50:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.276+0000 7fbdc0dd2640 1 -- 172.21.15.83:0/2853784951 learned_addr learned my addr 172.21.15.83:0/2853784951 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:32.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.276+0000 7fbdc15d3640 1 -- 172.21.15.83:0/2853784951 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdbc0b4980 msgr2=0x7fbdbc15b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:32.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.276+0000 7fbdc15d3640 1 --2- 172.21.15.83:0/2853784951 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdbc0b4980 0x7fbdbc15b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:32.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.276+0000 7fbdc15d3640 1 -- 172.21.15.83:0/2853784951 >> v1:172.21.15.167:6789/0 conn(0x7fbdbc06dfc0 legacy=0x7fbdbc0b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:32.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.276+0000 7fbdc15d3640 1 -- 172.21.15.83:0/2853784951 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdbc15eb20 con 0x7fbdbc0afef0 2024-04-17T10:50:32.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.280+0000 7fbdc15d3640 1 --2- 172.21.15.83:0/2853784951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdbc0afef0 0x7fbdbc15e3b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4009d90 tx=0x7fbdb401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbc802c7213fb1fa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.280+0000 7fbdb3fff640 1 -- 172.21.15.83:0/2853784951 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb40b2070 con 0x7fbdbc0afef0 2024-04-17T10:50:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.280+0000 7fbdb3fff640 1 -- 172.21.15.83:0/2853784951 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb40a5830 con 0x7fbdbc0afef0 2024-04-17T10:50:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.280+0000 7fbdb3fff640 1 -- 172.21.15.83:0/2853784951 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb40a5c00 con 0x7fbdbc0afef0 2024-04-17T10:50:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.280+0000 7fbdc3837640 1 -- 172.21.15.83:0/2853784951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdbc0afef0 msgr2=0x7fbdbc15e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.280+0000 7fbdc3837640 1 --2- 172.21.15.83:0/2853784951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdbc0afef0 0x7fbdbc15e3b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4009d90 tx=0x7fbdb401c970 comp rx=0 tx=0).stop 2024-04-17T10:50:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.280+0000 7fbdc3837640 1 -- 172.21.15.83:0/2853784951 shutdown_connections 2024-04-17T10:50:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.280+0000 7fbdc3837640 1 --2- 172.21.15.83:0/2853784951 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdbc0b4980 0x7fbdbc15b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.280+0000 7fbdc3837640 1 --2- 172.21.15.83:0/2853784951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdbc0afef0 0x7fbdbc15e3b0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.280+0000 7fbdc3837640 1 -- 172.21.15.83:0/2853784951 >> 172.21.15.83:0/2853784951 conn(0x7fbdbc06bcf0 msgr2=0x7fbdbc06c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.280+0000 7fbdc3837640 1 -- 172.21.15.83:0/2853784951 shutdown_connections 2024-04-17T10:50:32.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.280+0000 7fbdc3837640 1 -- 172.21.15.83:0/2853784951 wait complete. 2024-04-17T10:50:32.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.280+0000 7fbdc3837640 1 Processor -- start 2024-04-17T10:50:32.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.284+0000 7fbdc3837640 1 -- start start 2024-04-17T10:50:32.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.284+0000 7fbdc3837640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdbc06dfc0 0x7fbdbc1596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:32.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.284+0000 7fbdc3837640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdbc0afef0 0x7fbdbc14e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:32.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.284+0000 7fbdc3837640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdbc0b4980 0x7fbdbc13b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.284+0000 7fbdc3837640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc15c5d0 con 0x7fbdbc0afef0 2024-04-17T10:50:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.284+0000 7fbdc3837640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc15c450 con 0x7fbdbc06dfc0 2024-04-17T10:50:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.284+0000 7fbdc3837640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc15c750 con 0x7fbdbc0b4980 2024-04-17T10:50:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.284+0000 7fbdc0dd2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdbc0afef0 0x7fbdbc14e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.284+0000 7fbdc0dd2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdbc0afef0 0x7fbdbc14e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45046/0 (socket says 172.21.15.83:45046) 2024-04-17T10:50:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.284+0000 7fbdc0dd2640 1 -- 172.21.15.83:0/89361692 learned_addr learned my addr 172.21.15.83:0/89361692 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.284+0000 7fbdc15d3640 1 --2- 172.21.15.83:0/89361692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdbc06dfc0 0x7fbdbc1596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.284+0000 7fbdc1dd4640 1 --2- 172.21.15.83:0/89361692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdbc0b4980 0x7fbdbc13b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.284+0000 7fbdc0dd2640 1 -- 172.21.15.83:0/89361692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdbc0b4980 msgr2=0x7fbdbc13b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.284+0000 7fbdc0dd2640 1 --2- 172.21.15.83:0/89361692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdbc0b4980 0x7fbdbc13b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.284+0000 7fbdc0dd2640 1 -- 172.21.15.83:0/89361692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdbc06dfc0 msgr2=0x7fbdbc1596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.284+0000 7fbdc0dd2640 1 --2- 172.21.15.83:0/89361692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdbc06dfc0 0x7fbdbc1596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.284+0000 7fbdc0dd2640 1 -- 172.21.15.83:0/89361692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdbc158640 con 0x7fbdbc0afef0 2024-04-17T10:50:32.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.284+0000 7fbdc0dd2640 1 --2- 172.21.15.83:0/89361692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdbc0afef0 0x7fbdbc14e390 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fbdac00c420 tx=0x7fbdac00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:32.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.284+0000 7fbdb1ffb640 1 -- 172.21.15.83:0/89361692 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdac017070 con 0x7fbdbc0afef0 2024-04-17T10:50:32.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.284+0000 7fbdb1ffb640 1 -- 172.21.15.83:0/89361692 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdac005cf0 con 0x7fbdbc0afef0 2024-04-17T10:50:32.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.284+0000 7fbdb1ffb640 1 -- 172.21.15.83:0/89361692 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdac0072f0 con 0x7fbdbc0afef0 2024-04-17T10:50:32.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.284+0000 7fbdc3837640 1 -- 172.21.15.83:0/89361692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdbc158920 con 0x7fbdbc0afef0 2024-04-17T10:50:32.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.284+0000 7fbdc3837640 1 -- 172.21.15.83:0/89361692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdbc158ee0 con 0x7fbdbc0afef0 2024-04-17T10:50:32.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.284+0000 7fbdc3837640 1 -- 172.21.15.83:0/89361692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdbc075000 con 0x7fbdbc0afef0 2024-04-17T10:50:32.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.284+0000 7fbdb1ffb640 1 -- 172.21.15.83:0/89361692 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbdac01a070 con 0x7fbdbc0afef0 2024-04-17T10:50:32.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.284+0000 7fbdb1ffb640 1 --2- 172.21.15.83:0/89361692 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbda0066f60 0x7fbda0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:32.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.284+0000 7fbdb1ffb640 1 -- 172.21.15.83:0/89361692 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbdac083fd0 con 0x7fbdbc0afef0 2024-04-17T10:50:32.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.284+0000 7fbdc15d3640 1 --2- 172.21.15.83:0/89361692 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbda0066f60 0x7fbda0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:32.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.288+0000 7fbdc15d3640 1 --2- 172.21.15.83:0/89361692 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbda0066f60 0x7fbda0069420 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fbdb40af000 tx=0x7fbdb40a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:32.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.292+0000 7fbdb1ffb640 1 -- 172.21.15.83:0/89361692 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbdac054ce0 con 0x7fbdbc0afef0 2024-04-17T10:50:32.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.448+0000 7fbdc3837640 1 -- 172.21.15.83:0/89361692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fbdbc06ed10 con 0x7fbdbc0afef0 2024-04-17T10:50:32.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.852+0000 7fbdb1ffb640 1 -- 172.21.15.83:0/89361692 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v137) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbdac058d80 con 0x7fbdbc0afef0 2024-04-17T10:50:32.844 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2024-04-17T10:50:32.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.852+0000 7fbdc3837640 1 -- 172.21.15.83:0/89361692 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbda0066f60 msgr2=0x7fbda0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:32.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.852+0000 7fbdc3837640 1 --2- 172.21.15.83:0/89361692 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbda0066f60 0x7fbda0069420 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fbdb40af000 tx=0x7fbdb40a7070 comp rx=0 tx=0).stop 2024-04-17T10:50:32.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.852+0000 7fbdc3837640 1 -- 172.21.15.83:0/89361692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdbc0afef0 msgr2=0x7fbdbc14e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:32.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.852+0000 7fbdc3837640 1 --2- 172.21.15.83:0/89361692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdbc0afef0 0x7fbdbc14e390 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fbdac00c420 tx=0x7fbdac00c8e0 comp rx=0 tx=0).stop 2024-04-17T10:50:32.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.852+0000 7fbdc3837640 1 -- 172.21.15.83:0/89361692 shutdown_connections 2024-04-17T10:50:32.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.852+0000 7fbdc3837640 1 --2- 172.21.15.83:0/89361692 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbda0066f60 0x7fbda0069420 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:32.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.852+0000 7fbdc3837640 1 --2- 172.21.15.83:0/89361692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdbc0b4980 0x7fbdbc13b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:32.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.852+0000 7fbdc3837640 1 --2- 172.21.15.83:0/89361692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdbc0afef0 0x7fbdbc14e390 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:32.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.852+0000 7fbdc3837640 1 --2- 172.21.15.83:0/89361692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdbc06dfc0 0x7fbdbc1596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:32.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.852+0000 7fbdc3837640 1 -- 172.21.15.83:0/89361692 >> 172.21.15.83:0/89361692 conn(0x7fbdbc06bcf0 msgr2=0x7fbdbc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:32.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.856+0000 7fbdc3837640 1 -- 172.21.15.83:0/89361692 shutdown_connections 2024-04-17T10:50:32.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:32.856+0000 7fbdc3837640 1 -- 172.21.15.83:0/89361692 wait complete. 2024-04-17T10:50:32.872 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-17T10:50:32.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.004+0000 7f1c864fd640 1 Processor -- start 2024-04-17T10:50:32.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.004+0000 7f1c864fd640 1 -- start start 2024-04-17T10:50:33.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.008+0000 7f1c864fd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1c80161940 0x7f1c8006bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:33.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.008+0000 7f1c864fd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1c8015ed70 0x7f1c8006c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:33.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.008+0000 7f1c864fd640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c800d2430 con 0x7f1c800cfb80 2024-04-17T10:50:33.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.008+0000 7f1c864fd640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c800d2710 con 0x7f1c80161940 2024-04-17T10:50:33.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.008+0000 7f1c864fd640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c800ced00 con 0x7f1c8015ed70 2024-04-17T10:50:33.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.008+0000 7f1c7ffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1c80161940 0x7f1c8006bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:33.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.008+0000 7f1c7ffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1c80161940 0x7f1c8006bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45056/0 (socket says 172.21.15.83:45056) 2024-04-17T10:50:33.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.008+0000 7f1c7ffff640 1 -- 172.21.15.83:0/1562517423 learned_addr learned my addr 172.21.15.83:0/1562517423 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:33.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.008+0000 7f1c7ffff640 1 -- 172.21.15.83:0/1562517423 >> v2:172.21.15.167:3300/0 conn(0x7f1c8015ed70 msgr2=0x7f1c8006c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:33.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.008+0000 7f1c7ffff640 1 --2- 172.21.15.83:0/1562517423 >> v2:172.21.15.167:3300/0 conn(0x7f1c8015ed70 0x7f1c8006c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:33.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.008+0000 7f1c7ffff640 1 -- 172.21.15.83:0/1562517423 >> v1:172.21.15.167:6789/0 conn(0x7f1c800cfb80 legacy=0x7f1c80161d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:33.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.008+0000 7f1c7ffff640 1 -- 172.21.15.83:0/1562517423 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c800cefe0 con 0x7f1c80161940 2024-04-17T10:50:33.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.008+0000 7f1c7ffff640 1 --2- 172.21.15.83:0/1562517423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c80161940 0x7f1c8006bb80 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f1c800723f0 tx=0x7f1c7001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13497e6d2ef0a7f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:33.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.008+0000 7f1c7effd640 1 -- 172.21.15.83:0/1562517423 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c700a64f0 con 0x7f1c80161940 2024-04-17T10:50:33.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.008+0000 7f1c7effd640 1 -- 172.21.15.83:0/1562517423 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c700a6690 con 0x7f1c80161940 2024-04-17T10:50:33.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.008+0000 7f1c7effd640 1 -- 172.21.15.83:0/1562517423 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c700a6a60 con 0x7f1c80161940 2024-04-17T10:50:33.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.008+0000 7f1c864fd640 1 -- 172.21.15.83:0/1562517423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c80161940 msgr2=0x7f1c8006bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:33.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.008+0000 7f1c864fd640 1 --2- 172.21.15.83:0/1562517423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c80161940 0x7f1c8006bb80 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f1c800723f0 tx=0x7f1c7001c920 comp rx=0 tx=0).stop 2024-04-17T10:50:33.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.008+0000 7f1c864fd640 1 -- 172.21.15.83:0/1562517423 shutdown_connections 2024-04-17T10:50:33.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.008+0000 7f1c864fd640 1 --2- 172.21.15.83:0/1562517423 >> v2:172.21.15.167:3300/0 conn(0x7f1c8015ed70 0x7f1c8006c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:33.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.008+0000 7f1c864fd640 1 --2- 172.21.15.83:0/1562517423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c80161940 0x7f1c8006bb80 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:33.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.008+0000 7f1c864fd640 1 -- 172.21.15.83:0/1562517423 >> 172.21.15.83:0/1562517423 conn(0x7f1c800d17e0 msgr2=0x7f1c800d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:33.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.008+0000 7f1c864fd640 1 -- 172.21.15.83:0/1562517423 shutdown_connections 2024-04-17T10:50:33.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.008+0000 7f1c864fd640 1 -- 172.21.15.83:0/1562517423 wait complete. 2024-04-17T10:50:33.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.012+0000 7f1c864fd640 1 Processor -- start 2024-04-17T10:50:33.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.012+0000 7f1c864fd640 1 -- start start 2024-04-17T10:50:33.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.012+0000 7f1c864fd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c800cfb80 0x7f1c801556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:33.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.012+0000 7f1c864fd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c8015ed70 0x7f1c80152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:33.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.012+0000 7f1c864fd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c80161940 0x7f1c80153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:33.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.012+0000 7f1c864fd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8016ac30 con 0x7f1c800cfb80 2024-04-17T10:50:33.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.012+0000 7f1c864fd640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c800d01e0 con 0x7f1c80161940 2024-04-17T10:50:33.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.012+0000 7f1c864fd640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8016adb0 con 0x7f1c8015ed70 2024-04-17T10:50:33.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.012+0000 7f1c7ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c800cfb80 0x7f1c801556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:33.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.012+0000 7f1c7ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c800cfb80 0x7f1c801556c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45058/0 (socket says 172.21.15.83:45058) 2024-04-17T10:50:33.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.012+0000 7f1c7ffff640 1 -- 172.21.15.83:0/2759977768 learned_addr learned my addr 172.21.15.83:0/2759977768 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:33.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.012+0000 7f1c7ffff640 1 -- 172.21.15.83:0/2759977768 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c8015ed70 msgr2=0x7f1c80152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:33.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.012+0000 7f1c7ffff640 1 --2- 172.21.15.83:0/2759977768 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c8015ed70 0x7f1c80152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:33.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.012+0000 7f1c7ffff640 1 -- 172.21.15.83:0/2759977768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c80161940 msgr2=0x7f1c80153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:33.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.012+0000 7f1c7ffff640 1 --2- 172.21.15.83:0/2759977768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c80161940 0x7f1c80153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:33.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.012+0000 7f1c7ffff640 1 -- 172.21.15.83:0/2759977768 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c80160990 con 0x7f1c800cfb80 2024-04-17T10:50:33.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.012+0000 7f1c7ffff640 1 --2- 172.21.15.83:0/2759977768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c800cfb80 0x7f1c801556c0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f1c7001afd0 tx=0x7f1c700196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:33.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.012+0000 7f1c7cff9640 1 -- 172.21.15.83:0/2759977768 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c700a6c50 con 0x7f1c800cfb80 2024-04-17T10:50:33.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.012+0000 7f1c7cff9640 1 -- 172.21.15.83:0/2759977768 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c700a6e10 con 0x7f1c800cfb80 2024-04-17T10:50:33.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.012+0000 7f1c864fd640 1 -- 172.21.15.83:0/2759977768 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c80160c70 con 0x7f1c800cfb80 2024-04-17T10:50:33.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.012+0000 7f1c7cff9640 1 -- 172.21.15.83:0/2759977768 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c700af6c0 con 0x7f1c800cfb80 2024-04-17T10:50:33.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.012+0000 7f1c864fd640 1 -- 172.21.15.83:0/2759977768 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c80152db0 con 0x7f1c800cfb80 2024-04-17T10:50:33.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.012+0000 7f1c864fd640 1 -- 172.21.15.83:0/2759977768 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c80074f90 con 0x7f1c800cfb80 2024-04-17T10:50:33.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.016+0000 7f1c7cff9640 1 -- 172.21.15.83:0/2759977768 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1c700af860 con 0x7f1c800cfb80 2024-04-17T10:50:33.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.016+0000 7f1c7cff9640 1 --2- 172.21.15.83:0/2759977768 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1c6c066f60 0x7f1c6c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:33.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.016+0000 7f1c7f7fe640 1 --2- 172.21.15.83:0/2759977768 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1c6c066f60 0x7f1c6c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:33.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.016+0000 7f1c7cff9640 1 -- 172.21.15.83:0/2759977768 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f1c7011f4a0 con 0x7f1c800cfb80 2024-04-17T10:50:33.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.016+0000 7f1c7f7fe640 1 --2- 172.21.15.83:0/2759977768 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1c6c066f60 0x7f1c6c069420 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f1c74007fd0 tx=0x7f1c740079a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:33.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.020+0000 7f1c7cff9640 1 -- 172.21.15.83:0/2759977768 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1c700165a0 con 0x7f1c800cfb80 2024-04-17T10:50:33.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.164+0000 7f1c864fd640 1 -- 172.21.15.83:0/2759977768 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1c80160e40 con 0x7f1c800cfb80 2024-04-17T10:50:33.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.864+0000 7f1c7cff9640 1 -- 172.21.15.83:0/2759977768 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v138) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1c700f0720 con 0x7f1c800cfb80 2024-04-17T10:50:33.857 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2024-04-17T10:50:33.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.868+0000 7f1c864fd640 1 -- 172.21.15.83:0/2759977768 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1c6c066f60 msgr2=0x7f1c6c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:33.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.868+0000 7f1c864fd640 1 --2- 172.21.15.83:0/2759977768 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1c6c066f60 0x7f1c6c069420 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f1c74007fd0 tx=0x7f1c740079a0 comp rx=0 tx=0).stop 2024-04-17T10:50:33.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.868+0000 7f1c864fd640 1 -- 172.21.15.83:0/2759977768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c800cfb80 msgr2=0x7f1c801556c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:33.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.868+0000 7f1c864fd640 1 --2- 172.21.15.83:0/2759977768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c800cfb80 0x7f1c801556c0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f1c7001afd0 tx=0x7f1c700196c0 comp rx=0 tx=0).stop 2024-04-17T10:50:33.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.868+0000 7f1c864fd640 1 -- 172.21.15.83:0/2759977768 shutdown_connections 2024-04-17T10:50:33.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.868+0000 7f1c864fd640 1 --2- 172.21.15.83:0/2759977768 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1c6c066f60 0x7f1c6c069420 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:33.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.868+0000 7f1c864fd640 1 --2- 172.21.15.83:0/2759977768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c80161940 0x7f1c80153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:33.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.868+0000 7f1c864fd640 1 --2- 172.21.15.83:0/2759977768 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c8015ed70 0x7f1c80152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:33.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.868+0000 7f1c864fd640 1 --2- 172.21.15.83:0/2759977768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c800cfb80 0x7f1c801556c0 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:33.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.868+0000 7f1c864fd640 1 -- 172.21.15.83:0/2759977768 >> 172.21.15.83:0/2759977768 conn(0x7f1c800d17e0 msgr2=0x7f1c80161d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:33.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.868+0000 7f1c864fd640 1 -- 172.21.15.83:0/2759977768 shutdown_connections 2024-04-17T10:50:33.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:33.868+0000 7f1c864fd640 1 -- 172.21.15.83:0/2759977768 wait complete. 2024-04-17T10:50:33.894 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-17T10:50:34.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.020+0000 7f58fd072640 1 Processor -- start 2024-04-17T10:50:34.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.020+0000 7f58fd072640 1 -- start start 2024-04-17T10:50:34.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.020+0000 7f58fd072640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f58f8154de0 0x7f58f806ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:34.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.020+0000 7f58fd072640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f815bd70 con 0x7f58f806b350 2024-04-17T10:50:34.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.020+0000 7f58fd072640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f815bf60 con 0x7f58f80b4210 2024-04-17T10:50:34.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.020+0000 7f58fd072640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f80b4870 con 0x7f58f8154de0 2024-04-17T10:50:34.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.024+0000 7f58f659c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f58f806b350 0x7f58f80b3b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56316/0 (socket says 172.21.15.83:56316) 2024-04-17T10:50:34.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.024+0000 7f58f659c640 1 -- 172.21.15.83:0/45060813 learned_addr learned my addr 172.21.15.83:0/45060813 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:34.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.024+0000 7f58f6d9d640 1 --2- 172.21.15.83:0/45060813 >> v2:172.21.15.167:3300/0 conn(0x7f58f8154de0 0x7f58f806ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:34.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.024+0000 7f58f5d9b640 1 -- 172.21.15.83:0/45060813 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1907980031 0 0) 0x7f58f815bd70 con 0x7f58f806b350 2024-04-17T10:50:34.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.024+0000 7f58f5d9b640 1 -- 172.21.15.83:0/45060813 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58e0003530 con 0x7f58f806b350 2024-04-17T10:50:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.024+0000 7f58f5d9b640 1 -- 172.21.15.83:0/45060813 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3592195997 0 0) 0x7f58e0003530 con 0x7f58f806b350 2024-04-17T10:50:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.024+0000 7f58f5d9b640 1 -- 172.21.15.83:0/45060813 >> v2:172.21.15.167:3300/0 conn(0x7f58f8154de0 msgr2=0x7f58f806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.024+0000 7f58f5d9b640 1 --2- 172.21.15.83:0/45060813 >> v2:172.21.15.167:3300/0 conn(0x7f58f8154de0 0x7f58f806ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.024+0000 7f58f5d9b640 1 -- 172.21.15.83:0/45060813 >> v1:172.21.15.167:6789/0 conn(0x7f58f80b4210 legacy=0x7f58f806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.024+0000 7f58f5d9b640 1 -- 172.21.15.83:0/45060813 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f815afb0 con 0x7f58f806b350 2024-04-17T10:50:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.024+0000 7f58f5d9b640 1 -- 172.21.15.83:0/45060813 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f58ec001680 con 0x7f58f806b350 2024-04-17T10:50:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.024+0000 7f58f5d9b640 1 -- 172.21.15.83:0/45060813 >> v1:172.21.15.83:6789/0 conn(0x7f58f806b350 legacy=0x7f58f80b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.024+0000 7f58f5d9b640 1 --2- 172.21.15.83:0/45060813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58e0003a30 0x7f58e0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.024+0000 7f58f5d9b640 1 --2- 172.21.15.83:0/45060813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e0004810 0x7f58e0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.024+0000 7f58f5d9b640 1 --2- 172.21.15.83:0/45060813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58e00051d0 0x7f58e000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.024+0000 7f58f5d9b640 1 -- 172.21.15.83:0/45060813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e000c0d0 con 0x7f58e0004810 2024-04-17T10:50:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.024+0000 7f58f5d9b640 1 -- 172.21.15.83:0/45060813 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e000c280 con 0x7f58e00051d0 2024-04-17T10:50:34.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.024+0000 7f58f5d9b640 1 -- 172.21.15.83:0/45060813 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e000c480 con 0x7f58e0003a30 2024-04-17T10:50:34.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.024+0000 7f58f659c640 1 --2- 172.21.15.83:0/45060813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e0004810 0x7f58e0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:34.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.024+0000 7f58f759e640 1 --2- 172.21.15.83:0/45060813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58e00051d0 0x7f58e000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:34.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.024+0000 7f58f6d9d640 1 --2- 172.21.15.83:0/45060813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58e0003a30 0x7f58e0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:34.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.024+0000 7f58f659c640 1 -- 172.21.15.83:0/45060813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58e0003a30 msgr2=0x7f58e0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:34.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.024+0000 7f58f659c640 1 --2- 172.21.15.83:0/45060813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58e0003a30 0x7f58e0003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:34.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.024+0000 7f58f659c640 1 -- 172.21.15.83:0/45060813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58e00051d0 msgr2=0x7f58e000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:34.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.024+0000 7f58f659c640 1 --2- 172.21.15.83:0/45060813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58e00051d0 0x7f58e000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:34.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.024+0000 7f58f759e640 1 -- 172.21.15.83:0/45060813 reap_dead start 2024-04-17T10:50:34.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.024+0000 7f58f659c640 1 -- 172.21.15.83:0/45060813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e0003530 con 0x7f58e0004810 2024-04-17T10:50:34.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.024+0000 7f58f659c640 1 --2- 172.21.15.83:0/45060813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e0004810 0x7f58e0004c90 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f58ec00c210 tx=0x7f58ec01f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4f7ae270ddc76a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:34.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.024+0000 7f58f5d9b640 1 -- 172.21.15.83:0/45060813 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec007d40 con 0x7f58e0004810 2024-04-17T10:50:34.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.024+0000 7f58f5d9b640 1 -- 172.21.15.83:0/45060813 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ec005b00 con 0x7f58e0004810 2024-04-17T10:50:34.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.024+0000 7f58f5d9b640 1 -- 172.21.15.83:0/45060813 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec007070 con 0x7f58e0004810 2024-04-17T10:50:34.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.028+0000 7f58fd072640 1 -- 172.21.15.83:0/45060813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e0004810 msgr2=0x7f58e0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:34.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.028+0000 7f58fd072640 1 --2- 172.21.15.83:0/45060813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e0004810 0x7f58e0004c90 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f58ec00c210 tx=0x7f58ec01f3f0 comp rx=0 tx=0).stop 2024-04-17T10:50:34.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.028+0000 7f58fd072640 1 -- 172.21.15.83:0/45060813 shutdown_connections 2024-04-17T10:50:34.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.028+0000 7f58fd072640 1 --2- 172.21.15.83:0/45060813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e0004810 0x7f58e0004c90 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:34.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.028+0000 7f58fd072640 1 -- 172.21.15.83:0/45060813 >> 172.21.15.83:0/45060813 conn(0x7f58f80b15b0 msgr2=0x7f58f80b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:34.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.028+0000 7f58fd072640 1 -- 172.21.15.83:0/45060813 shutdown_connections 2024-04-17T10:50:34.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.028+0000 7f58fd072640 1 -- 172.21.15.83:0/45060813 wait complete. 2024-04-17T10:50:34.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.028+0000 7f58fd072640 1 Processor -- start 2024-04-17T10:50:34.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.028+0000 7f58fd072640 1 -- start start 2024-04-17T10:50:34.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.028+0000 7f58fd072640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58e0004810 0x7f58f81f5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:34.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.028+0000 7f58fd072640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58f8141e70 0x7f58f81ee720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:34.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.028+0000 7f58fd072640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58f8150370 0x7f58f81507f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:34.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.028+0000 7f58fd072640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f815b850 con 0x7f58f8141e70 2024-04-17T10:50:34.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.028+0000 7f58fd072640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f815b570 con 0x7f58e0004810 2024-04-17T10:50:34.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.028+0000 7f58fd072640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f81551c0 con 0x7f58f8150370 2024-04-17T10:50:34.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.028+0000 7f58f659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58f8141e70 0x7f58f81ee720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:34.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.028+0000 7f58f659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58f8141e70 0x7f58f81ee720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45064/0 (socket says 172.21.15.83:45064) 2024-04-17T10:50:34.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.028+0000 7f58f659c640 1 -- 172.21.15.83:0/3450690793 learned_addr learned my addr 172.21.15.83:0/3450690793 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:34.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.028+0000 7f58f6d9d640 1 --2- 172.21.15.83:0/3450690793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58e0004810 0x7f58f81f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:34.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.028+0000 7f58f759e640 1 --2- 172.21.15.83:0/3450690793 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58f8150370 0x7f58f81507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:34.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.028+0000 7f58f659c640 1 -- 172.21.15.83:0/3450690793 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58f8150370 msgr2=0x7f58f81507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:34.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.028+0000 7f58f659c640 1 --2- 172.21.15.83:0/3450690793 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58f8150370 0x7f58f81507f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:34.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.028+0000 7f58f659c640 1 -- 172.21.15.83:0/3450690793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58e0004810 msgr2=0x7f58f81f5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:34.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.028+0000 7f58f659c640 1 --2- 172.21.15.83:0/3450690793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58e0004810 0x7f58f81f5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:34.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.028+0000 7f58f659c640 1 -- 172.21.15.83:0/3450690793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f81f5770 con 0x7f58f8141e70 2024-04-17T10:50:34.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.028+0000 7f58f659c640 1 --2- 172.21.15.83:0/3450690793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58f8141e70 0x7f58f81ee720 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f58ec001980 tx=0x7f58e0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:34.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.028+0000 7f58df7fe640 1 -- 172.21.15.83:0/3450690793 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec001680 con 0x7f58f8141e70 2024-04-17T10:50:34.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.028+0000 7f58df7fe640 1 -- 172.21.15.83:0/3450690793 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ec0b6070 con 0x7f58f8141e70 2024-04-17T10:50:34.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.028+0000 7f58fd072640 1 -- 172.21.15.83:0/3450690793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f81eec60 con 0x7f58f8141e70 2024-04-17T10:50:34.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.028+0000 7f58df7fe640 1 -- 172.21.15.83:0/3450690793 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec0049f0 con 0x7f58f8141e70 2024-04-17T10:50:34.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.028+0000 7f58fd072640 1 -- 172.21.15.83:0/3450690793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f81426d0 con 0x7f58f8141e70 2024-04-17T10:50:34.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.032+0000 7f58fd072640 1 -- 172.21.15.83:0/3450690793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f8074f90 con 0x7f58f8141e70 2024-04-17T10:50:34.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.032+0000 7f58df7fe640 1 -- 172.21.15.83:0/3450690793 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f58ec0b8020 con 0x7f58f8141e70 2024-04-17T10:50:34.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.032+0000 7f58df7fe640 1 --2- 172.21.15.83:0/3450690793 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f58e4066f60 0x7f58e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:34.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.032+0000 7f58f6d9d640 1 --2- 172.21.15.83:0/3450690793 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f58e4066f60 0x7f58e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:34.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.032+0000 7f58df7fe640 1 -- 172.21.15.83:0/3450690793 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f58ec122390 con 0x7f58f8141e70 2024-04-17T10:50:34.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.032+0000 7f58f6d9d640 1 --2- 172.21.15.83:0/3450690793 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f58e4066f60 0x7f58e4069420 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009ab0 tx=0x7f58e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:34.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.036+0000 7f58df7fe640 1 -- 172.21.15.83:0/3450690793 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f58f81f5770 con 0x7f58f8141e70 2024-04-17T10:50:34.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.228+0000 7f58fd072640 1 -- 172.21.15.83:0/3450690793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f58f80b3280 con 0x7f58f8141e70 2024-04-17T10:50:34.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.904+0000 7f58df7fe640 1 -- 172.21.15.83:0/3450690793 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v85) v1 ==== 174+0+0 (secure 0 0 0) 0x7f58ec019100 con 0x7f58f8141e70 2024-04-17T10:50:34.897 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 20 and data pool 21 2024-04-17T10:50:34.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.908+0000 7f58fd072640 1 -- 172.21.15.83:0/3450690793 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f58e4066f60 msgr2=0x7f58e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:34.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.908+0000 7f58fd072640 1 --2- 172.21.15.83:0/3450690793 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f58e4066f60 0x7f58e4069420 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009ab0 tx=0x7f58e8008040 comp rx=0 tx=0).stop 2024-04-17T10:50:34.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.908+0000 7f58fd072640 1 -- 172.21.15.83:0/3450690793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58f8141e70 msgr2=0x7f58f81ee720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:34.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.908+0000 7f58fd072640 1 --2- 172.21.15.83:0/3450690793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58f8141e70 0x7f58f81ee720 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f58ec001980 tx=0x7f58e0005840 comp rx=0 tx=0).stop 2024-04-17T10:50:34.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.908+0000 7f58fd072640 1 -- 172.21.15.83:0/3450690793 shutdown_connections 2024-04-17T10:50:34.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.908+0000 7f58fd072640 1 --2- 172.21.15.83:0/3450690793 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f58e4066f60 0x7f58e4069420 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:34.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.908+0000 7f58fd072640 1 --2- 172.21.15.83:0/3450690793 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58f8150370 0x7f58f81507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:34.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.908+0000 7f58fd072640 1 --2- 172.21.15.83:0/3450690793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58f8141e70 0x7f58f81ee720 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:34.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.908+0000 7f58fd072640 1 --2- 172.21.15.83:0/3450690793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58e0004810 0x7f58f81f5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:34.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.908+0000 7f58fd072640 1 -- 172.21.15.83:0/3450690793 >> 172.21.15.83:0/3450690793 conn(0x7f58f80b15b0 msgr2=0x7f58f80b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:34.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.908+0000 7f58fd072640 1 -- 172.21.15.83:0/3450690793 shutdown_connections 2024-04-17T10:50:34.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:34.908+0000 7f58fd072640 1 -- 172.21.15.83:0/3450690793 wait complete. 2024-04-17T10:50:34.929 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:50:35.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.052+0000 7f085dd3b640 1 Processor -- start 2024-04-17T10:50:35.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.052+0000 7f085dd3b640 1 -- start start 2024-04-17T10:50:35.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.052+0000 7f085dd3b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f085815ba30 0x7f0858072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:35.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.052+0000 7f085dd3b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f085806e320 con 0x7f085815c960 2024-04-17T10:50:35.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.052+0000 7f085dd3b640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f085806e040 con 0x7f08580cfbe0 2024-04-17T10:50:35.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.052+0000 7f085dd3b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0858071ea0 con 0x7f085815ba30 2024-04-17T10:50:35.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.052+0000 7f0856ffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f085815c960 0x7f08580cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56322/0 (socket says 172.21.15.83:56322) 2024-04-17T10:50:35.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.052+0000 7f0856ffd640 1 -- 172.21.15.83:0/3840398916 learned_addr learned my addr 172.21.15.83:0/3840398916 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:35.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.052+0000 7f08577fe640 1 --2- 172.21.15.83:0/3840398916 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f085815ba30 0x7f0858072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:35.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.052+0000 7f08567fc640 1 -- 172.21.15.83:0/3840398916 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2046100774 0 0) 0x7f085806e320 con 0x7f085815c960 2024-04-17T10:50:35.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.052+0000 7f08567fc640 1 -- 172.21.15.83:0/3840398916 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0844003530 con 0x7f085815c960 2024-04-17T10:50:35.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.052+0000 7f08567fc640 1 -- 172.21.15.83:0/3840398916 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1032741064 0 0) 0x7f0844003530 con 0x7f085815c960 2024-04-17T10:50:35.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.052+0000 7f08567fc640 1 -- 172.21.15.83:0/3840398916 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f085815ba30 msgr2=0x7f0858072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:35.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.052+0000 7f08567fc640 1 --2- 172.21.15.83:0/3840398916 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f085815ba30 0x7f0858072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:35.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.052+0000 7f08567fc640 1 -- 172.21.15.83:0/3840398916 >> v1:172.21.15.167:6789/0 conn(0x7f08580cfbe0 legacy=0x7f085806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:35.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.052+0000 7f08567fc640 1 -- 172.21.15.83:0/3840398916 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0858071850 con 0x7f085815c960 2024-04-17T10:50:35.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.052+0000 7f08567fc640 1 -- 172.21.15.83:0/3840398916 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f08400016a0 con 0x7f085815c960 2024-04-17T10:50:35.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.052+0000 7f08567fc640 1 -- 172.21.15.83:0/3840398916 >> v1:172.21.15.83:6789/0 conn(0x7f085815c960 legacy=0x7f08580cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:35.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.052+0000 7f08567fc640 1 --2- 172.21.15.83:0/3840398916 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f085815ba30 0x7f08440039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:35.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.052+0000 7f08567fc640 1 --2- 172.21.15.83:0/3840398916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0844003f70 0x7f0844004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:35.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.052+0000 7f08567fc640 1 --2- 172.21.15.83:0/3840398916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0844004d50 0x7f08440051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:35.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.052+0000 7f08567fc640 1 -- 172.21.15.83:0/3840398916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0858071ea0 con 0x7f0844003f70 2024-04-17T10:50:35.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.052+0000 7f08567fc640 1 -- 172.21.15.83:0/3840398916 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084400bb60 con 0x7f0844004d50 2024-04-17T10:50:35.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.052+0000 7f08567fc640 1 -- 172.21.15.83:0/3840398916 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f084400bd60 con 0x7f085815ba30 2024-04-17T10:50:35.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.052+0000 7f0856ffd640 1 --2- 172.21.15.83:0/3840398916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0844003f70 0x7f0844004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:35.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.056+0000 7f08577fe640 1 --2- 172.21.15.83:0/3840398916 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f085815ba30 0x7f08440039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:35.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.056+0000 7f0857fff640 1 --2- 172.21.15.83:0/3840398916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0844004d50 0x7f08440051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:35.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.056+0000 7f0856ffd640 1 -- 172.21.15.83:0/3840398916 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f085815ba30 msgr2=0x7f08440039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:35.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.056+0000 7f0856ffd640 1 --2- 172.21.15.83:0/3840398916 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f085815ba30 0x7f08440039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:35.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.056+0000 7f0856ffd640 1 -- 172.21.15.83:0/3840398916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0844004d50 msgr2=0x7f08440051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:35.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.056+0000 7f0856ffd640 1 --2- 172.21.15.83:0/3840398916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0844004d50 0x7f08440051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:35.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.056+0000 7f0856ffd640 1 -- 172.21.15.83:0/3840398916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0844003530 con 0x7f0844003f70 2024-04-17T10:50:35.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.056+0000 7f0856ffd640 1 --2- 172.21.15.83:0/3840398916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0844003f70 0x7f0844004810 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f084000c290 tx=0x7f085806d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d4d7dd6dfe2fb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:35.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.056+0000 7f08567fc640 1 -- 172.21.15.83:0/3840398916 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0840007d60 con 0x7f0844003f70 2024-04-17T10:50:35.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.056+0000 7f08567fc640 1 -- 172.21.15.83:0/3840398916 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08400a9020 con 0x7f0844003f70 2024-04-17T10:50:35.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.056+0000 7f08567fc640 1 -- 172.21.15.83:0/3840398916 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0840005b00 con 0x7f0844003f70 2024-04-17T10:50:35.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.056+0000 7f085dd3b640 1 -- 172.21.15.83:0/3840398916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0844003f70 msgr2=0x7f0844004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:35.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.056+0000 7f085dd3b640 1 --2- 172.21.15.83:0/3840398916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0844003f70 0x7f0844004810 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f084000c290 tx=0x7f085806d3b0 comp rx=0 tx=0).stop 2024-04-17T10:50:35.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.056+0000 7f0857fff640 1 -- 172.21.15.83:0/3840398916 reap_dead start 2024-04-17T10:50:35.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.056+0000 7f085dd3b640 1 -- 172.21.15.83:0/3840398916 shutdown_connections 2024-04-17T10:50:35.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.056+0000 7f085dd3b640 1 -- 172.21.15.83:0/3840398916 >> 172.21.15.83:0/3840398916 conn(0x7f085806abe0 msgr2=0x7f085806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:35.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.056+0000 7f085dd3b640 1 -- 172.21.15.83:0/3840398916 shutdown_connections 2024-04-17T10:50:35.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.056+0000 7f085dd3b640 1 -- 172.21.15.83:0/3840398916 wait complete. 2024-04-17T10:50:35.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.060+0000 7f085dd3b640 1 Processor -- start 2024-04-17T10:50:35.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.060+0000 7f085dd3b640 1 -- start start 2024-04-17T10:50:35.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.060+0000 7f085dd3b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0844003f70 0x7f0858165c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:35.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.060+0000 7f085dd3b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0858150e10 0x7f08581f86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:35.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.060+0000 7f085dd3b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08581595c0 0x7f08581512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:35.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.060+0000 7f085dd3b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085806de20 con 0x7f0858150e10 2024-04-17T10:50:35.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.060+0000 7f085dd3b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085806dca0 con 0x7f08581595c0 2024-04-17T10:50:35.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.060+0000 7f085dd3b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0858167520 con 0x7f0844003f70 2024-04-17T10:50:35.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.060+0000 7f0856ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0858150e10 0x7f08581f86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:35.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.060+0000 7f0856ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0858150e10 0x7f08581f86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45090/0 (socket says 172.21.15.83:45090) 2024-04-17T10:50:35.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.060+0000 7f0856ffd640 1 -- 172.21.15.83:0/556660112 learned_addr learned my addr 172.21.15.83:0/556660112 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:35.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.060+0000 7f08577fe640 1 --2- 172.21.15.83:0/556660112 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0844003f70 0x7f0858165c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:35.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.060+0000 7f0857fff640 1 --2- 172.21.15.83:0/556660112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08581595c0 0x7f08581512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:35.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.060+0000 7f0856ffd640 1 -- 172.21.15.83:0/556660112 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0844003f70 msgr2=0x7f0858165c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:35.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.060+0000 7f0856ffd640 1 --2- 172.21.15.83:0/556660112 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0844003f70 0x7f0858165c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:35.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.060+0000 7f0856ffd640 1 -- 172.21.15.83:0/556660112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08581595c0 msgr2=0x7f08581512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:35.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.060+0000 7f0856ffd640 1 --2- 172.21.15.83:0/556660112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08581595c0 0x7f08581512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:35.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.060+0000 7f0856ffd640 1 -- 172.21.15.83:0/556660112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0858158240 con 0x7f0858150e10 2024-04-17T10:50:35.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.060+0000 7f0857fff640 1 --2- 172.21.15.83:0/556660112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08581595c0 0x7f08581512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:35.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.060+0000 7f0856ffd640 1 --2- 172.21.15.83:0/556660112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0858150e10 0x7f08581f86e0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f084001d1e0 tx=0x7f0844005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:35.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.060+0000 7f084ffff640 1 -- 172.21.15.83:0/556660112 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0840007d60 con 0x7f0858150e10 2024-04-17T10:50:35.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.060+0000 7f084ffff640 1 -- 172.21.15.83:0/556660112 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08400070d0 con 0x7f0858150e10 2024-04-17T10:50:35.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.060+0000 7f084ffff640 1 -- 172.21.15.83:0/556660112 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08400016a0 con 0x7f0858150e10 2024-04-17T10:50:35.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.060+0000 7f085dd3b640 1 -- 172.21.15.83:0/556660112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0858158520 con 0x7f0858150e10 2024-04-17T10:50:35.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.060+0000 7f085dd3b640 1 -- 172.21.15.83:0/556660112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0858158ae0 con 0x7f0858150e10 2024-04-17T10:50:35.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.060+0000 7f085dd3b640 1 -- 172.21.15.83:0/556660112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0858074f90 con 0x7f0858150e10 2024-04-17T10:50:35.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.064+0000 7f084ffff640 1 -- 172.21.15.83:0/556660112 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f08400074a0 con 0x7f0858150e10 2024-04-17T10:50:35.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.064+0000 7f084ffff640 1 --2- 172.21.15.83:0/556660112 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0838066f90 0x7f0838069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:35.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.064+0000 7f084ffff640 1 -- 172.21.15.83:0/556660112 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f084011e7b0 con 0x7f0858150e10 2024-04-17T10:50:35.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.064+0000 7f08577fe640 1 --2- 172.21.15.83:0/556660112 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0838066f90 0x7f0838069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:35.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.064+0000 7f08577fe640 1 --2- 172.21.15.83:0/556660112 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0838066f90 0x7f0838069450 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f0848007520 tx=0x7f084800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:35.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.068+0000 7f084ffff640 1 -- 172.21.15.83:0/556660112 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0840016a10 con 0x7f0858150e10 2024-04-17T10:50:35.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.228+0000 7f085dd3b640 1 -- 172.21.15.83:0/556660112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f085806c930 con 0x7f0858150e10 2024-04-17T10:50:35.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.232+0000 7f084ffff640 1 -- 172.21.15.83:0/556660112 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+16620 (secure 0 0 0) 0x7f08400ef7c0 con 0x7f0858150e10 2024-04-17T10:50:35.224 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:50:35.224 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":139,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:50:34.232997+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:50:32.488186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-17T10:50:33.207211+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:50:35.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.232+0000 7f085dd3b640 1 -- 172.21.15.83:0/556660112 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0838066f90 msgr2=0x7f0838069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:35.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.232+0000 7f085dd3b640 1 --2- 172.21.15.83:0/556660112 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0838066f90 0x7f0838069450 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f0848007520 tx=0x7f084800b040 comp rx=0 tx=0).stop 2024-04-17T10:50:35.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.232+0000 7f085dd3b640 1 -- 172.21.15.83:0/556660112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0858150e10 msgr2=0x7f08581f86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:35.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.232+0000 7f085dd3b640 1 --2- 172.21.15.83:0/556660112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0858150e10 0x7f08581f86e0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f084001d1e0 tx=0x7f0844005840 comp rx=0 tx=0).stop 2024-04-17T10:50:35.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.232+0000 7f085dd3b640 1 -- 172.21.15.83:0/556660112 shutdown_connections 2024-04-17T10:50:35.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.232+0000 7f085dd3b640 1 --2- 172.21.15.83:0/556660112 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0838066f90 0x7f0838069450 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:35.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.232+0000 7f085dd3b640 1 --2- 172.21.15.83:0/556660112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08581595c0 0x7f08581512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:35.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.232+0000 7f085dd3b640 1 --2- 172.21.15.83:0/556660112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0858150e10 0x7f08581f86e0 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:35.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.232+0000 7f085dd3b640 1 --2- 172.21.15.83:0/556660112 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0844003f70 0x7f0858165c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:35.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.232+0000 7f085dd3b640 1 -- 172.21.15.83:0/556660112 >> 172.21.15.83:0/556660112 conn(0x7f085806abe0 msgr2=0x7f085806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:35.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.232+0000 7f085dd3b640 1 -- 172.21.15.83:0/556660112 shutdown_connections 2024-04-17T10:50:35.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.232+0000 7f085dd3b640 1 -- 172.21.15.83:0/556660112 wait complete. 2024-04-17T10:50:35.244 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:50:35.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.372+0000 7fea14def640 1 Processor -- start 2024-04-17T10:50:35.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.372+0000 7fea14def640 1 -- start start 2024-04-17T10:50:35.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.372+0000 7fea14def640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fea100cfbe0 0x7fea1006c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:35.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.372+0000 7fea14def640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea1006e320 con 0x7fea1015ba30 2024-04-17T10:50:35.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.372+0000 7fea14def640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea1006e040 con 0x7fea10072330 2024-04-17T10:50:35.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.372+0000 7fea14def640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea10071ea0 con 0x7fea100cfbe0 2024-04-17T10:50:35.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.372+0000 7fea0e59c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fea1015ba30 0x7fea1015c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56328/0 (socket says 172.21.15.83:56328) 2024-04-17T10:50:35.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.372+0000 7fea0e59c640 1 -- 172.21.15.83:0/1822782841 learned_addr learned my addr 172.21.15.83:0/1822782841 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:35.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.372+0000 7fea0ed9d640 1 --2- 172.21.15.83:0/1822782841 >> v2:172.21.15.83:3300/0 conn(0x7fea100cfbe0 0x7fea1006c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:35.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.372+0000 7fea0d59a640 1 -- 172.21.15.83:0/1822782841 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3313077017 0 0) 0x7fea1006e320 con 0x7fea1015ba30 2024-04-17T10:50:35.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.372+0000 7fea0d59a640 1 -- 172.21.15.83:0/1822782841 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9f8003530 con 0x7fea1015ba30 2024-04-17T10:50:35.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.372+0000 7fea0ed9d640 1 -- 172.21.15.83:0/1822782841 >> v1:172.21.15.167:6789/0 conn(0x7fea10072330 legacy=0x7fea100cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:35.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.372+0000 7fea0ed9d640 1 -- 172.21.15.83:0/1822782841 >> v1:172.21.15.83:6789/0 conn(0x7fea1015ba30 legacy=0x7fea1015c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:35.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.372+0000 7fea0ed9d640 1 -- 172.21.15.83:0/1822782841 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea10071850 con 0x7fea100cfbe0 2024-04-17T10:50:35.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.376+0000 7fea0ed9d640 1 --2- 172.21.15.83:0/1822782841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea100cfbe0 0x7fea1006c490 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fea08009d90 tx=0x7fea0801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c5906cfc039cd73 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:35.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.376+0000 7fea0d59a640 1 -- 172.21.15.83:0/1822782841 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea080b2070 con 0x7fea100cfbe0 2024-04-17T10:50:35.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.376+0000 7fea0d59a640 1 -- 172.21.15.83:0/1822782841 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea080a5870 con 0x7fea100cfbe0 2024-04-17T10:50:35.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.376+0000 7fea0d59a640 1 -- 172.21.15.83:0/1822782841 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea080a5c40 con 0x7fea100cfbe0 2024-04-17T10:50:35.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.376+0000 7fea14def640 1 -- 172.21.15.83:0/1822782841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea100cfbe0 msgr2=0x7fea1006c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:35.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.376+0000 7fea14def640 1 --2- 172.21.15.83:0/1822782841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea100cfbe0 0x7fea1006c490 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fea08009d90 tx=0x7fea0801c8a0 comp rx=0 tx=0).stop 2024-04-17T10:50:35.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.376+0000 7fea14def640 1 -- 172.21.15.83:0/1822782841 shutdown_connections 2024-04-17T10:50:35.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.376+0000 7fea14def640 1 --2- 172.21.15.83:0/1822782841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea100cfbe0 0x7fea1006c490 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:35.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.376+0000 7fea14def640 1 -- 172.21.15.83:0/1822782841 >> 172.21.15.83:0/1822782841 conn(0x7fea1006abe0 msgr2=0x7fea1006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:35.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.376+0000 7fea14def640 1 -- 172.21.15.83:0/1822782841 shutdown_connections 2024-04-17T10:50:35.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.376+0000 7fea14def640 1 -- 172.21.15.83:0/1822782841 wait complete. 2024-04-17T10:50:35.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.376+0000 7fea14def640 1 Processor -- start 2024-04-17T10:50:35.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.376+0000 7fea14def640 1 -- start start 2024-04-17T10:50:35.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.380+0000 7fea14def640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea10072330 0x7fea100b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:35.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.380+0000 7fea14def640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea100cfbe0 0x7fea1014a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:35.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.380+0000 7fea14def640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea1015ba30 0x7fea10151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:35.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.380+0000 7fea14def640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea100b28f0 con 0x7fea100cfbe0 2024-04-17T10:50:35.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.380+0000 7fea14def640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea100b2770 con 0x7fea1015ba30 2024-04-17T10:50:35.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.380+0000 7fea14def640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea100b2a70 con 0x7fea10072330 2024-04-17T10:50:35.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.380+0000 7fea0dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea100cfbe0 0x7fea1014a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:35.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.380+0000 7fea0dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea100cfbe0 0x7fea1014a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45110/0 (socket says 172.21.15.83:45110) 2024-04-17T10:50:35.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.380+0000 7fea0dd9b640 1 -- 172.21.15.83:0/870314731 learned_addr learned my addr 172.21.15.83:0/870314731 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:35.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.380+0000 7fea0e59c640 1 --2- 172.21.15.83:0/870314731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea10072330 0x7fea100b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:35.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.380+0000 7fea0ed9d640 1 --2- 172.21.15.83:0/870314731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea1015ba30 0x7fea10151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:35.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.380+0000 7fea0dd9b640 1 -- 172.21.15.83:0/870314731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea10072330 msgr2=0x7fea100b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:35.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.380+0000 7fea0dd9b640 1 --2- 172.21.15.83:0/870314731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea10072330 0x7fea100b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:35.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.380+0000 7fea0dd9b640 1 -- 172.21.15.83:0/870314731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea1015ba30 msgr2=0x7fea10151180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:35.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.380+0000 7fea0dd9b640 1 --2- 172.21.15.83:0/870314731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea1015ba30 0x7fea10151180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:35.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.380+0000 7fea0dd9b640 1 -- 172.21.15.83:0/870314731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1014cc50 con 0x7fea100cfbe0 2024-04-17T10:50:35.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.380+0000 7fea0dd9b640 1 --2- 172.21.15.83:0/870314731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea100cfbe0 0x7fea1014a920 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fea0400c450 tx=0x7fea0400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:35.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.380+0000 7fe9feffd640 1 -- 172.21.15.83:0/870314731 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea04012830 con 0x7fea100cfbe0 2024-04-17T10:50:35.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.380+0000 7fe9feffd640 1 -- 172.21.15.83:0/870314731 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea04017070 con 0x7fea100cfbe0 2024-04-17T10:50:35.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.380+0000 7fea14def640 1 -- 172.21.15.83:0/870314731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea1014cf30 con 0x7fea100cfbe0 2024-04-17T10:50:35.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.380+0000 7fe9feffd640 1 -- 172.21.15.83:0/870314731 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea04012c30 con 0x7fea100cfbe0 2024-04-17T10:50:35.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.380+0000 7fea14def640 1 -- 172.21.15.83:0/870314731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea1014d4f0 con 0x7fea100cfbe0 2024-04-17T10:50:35.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.380+0000 7fea14def640 1 -- 172.21.15.83:0/870314731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea100b1140 con 0x7fea100cfbe0 2024-04-17T10:50:35.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.380+0000 7fe9feffd640 1 -- 172.21.15.83:0/870314731 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fea0401c460 con 0x7fea100cfbe0 2024-04-17T10:50:35.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.380+0000 7fe9feffd640 1 --2- 172.21.15.83:0/870314731 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe9f0066f60 0x7fe9f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:35.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.380+0000 7fe9feffd640 1 -- 172.21.15.83:0/870314731 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea04084f10 con 0x7fea100cfbe0 2024-04-17T10:50:35.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.380+0000 7fea0e59c640 1 --2- 172.21.15.83:0/870314731 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe9f0066f60 0x7fe9f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:35.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.384+0000 7fea0e59c640 1 --2- 172.21.15.83:0/870314731 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe9f0066f60 0x7fe9f0069420 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fea00007fd0 tx=0x7fea00009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:35.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.384+0000 7fe9feffd640 1 -- 172.21.15.83:0/870314731 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fea100b1140 con 0x7fea100cfbe0 2024-04-17T10:50:35.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.544+0000 7fea14def640 1 -- 172.21.15.83:0/870314731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea1014fb40 con 0x7fea100cfbe0 2024-04-17T10:50:35.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.544+0000 7fe9feffd640 1 -- 172.21.15.83:0/870314731 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+16620 (secure 0 0 0) 0x7fea04055e80 con 0x7fea100cfbe0 2024-04-17T10:50:35.539 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:50:35.540 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":139,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:50:34.232997+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:50:32.488186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-17T10:50:33.207211+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:50:35.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.548+0000 7fea14def640 1 -- 172.21.15.83:0/870314731 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe9f0066f60 msgr2=0x7fe9f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:35.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.548+0000 7fea14def640 1 --2- 172.21.15.83:0/870314731 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe9f0066f60 0x7fe9f0069420 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fea00007fd0 tx=0x7fea00009040 comp rx=0 tx=0).stop 2024-04-17T10:50:35.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.548+0000 7fea14def640 1 -- 172.21.15.83:0/870314731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea100cfbe0 msgr2=0x7fea1014a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:35.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.548+0000 7fea14def640 1 --2- 172.21.15.83:0/870314731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea100cfbe0 0x7fea1014a920 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fea0400c450 tx=0x7fea0400c910 comp rx=0 tx=0).stop 2024-04-17T10:50:35.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.548+0000 7fea14def640 1 -- 172.21.15.83:0/870314731 shutdown_connections 2024-04-17T10:50:35.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.548+0000 7fea14def640 1 --2- 172.21.15.83:0/870314731 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe9f0066f60 0x7fe9f0069420 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:35.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.548+0000 7fea14def640 1 --2- 172.21.15.83:0/870314731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea1015ba30 0x7fea10151180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:35.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.548+0000 7fea14def640 1 --2- 172.21.15.83:0/870314731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea100cfbe0 0x7fea1014a920 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:35.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.548+0000 7fea14def640 1 --2- 172.21.15.83:0/870314731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea10072330 0x7fea100b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:35.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.548+0000 7fea14def640 1 -- 172.21.15.83:0/870314731 >> 172.21.15.83:0/870314731 conn(0x7fea1006abe0 msgr2=0x7fea1014f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:35.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.548+0000 7fea14def640 1 -- 172.21.15.83:0/870314731 shutdown_connections 2024-04-17T10:50:35.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.548+0000 7fea14def640 1 -- 172.21.15.83:0/870314731 wait complete. 2024-04-17T10:50:35.559 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-17T10:50:35.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.688+0000 7f790dd13640 1 Processor -- start 2024-04-17T10:50:35.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.688+0000 7f790dd13640 1 -- start start 2024-04-17T10:50:35.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.688+0000 7f790dd13640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f790815dce0 0x7f79080ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:35.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.688+0000 7f790dd13640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f79080cefc0 0x7f79080b6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:35.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.688+0000 7f790dd13640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79080d3070 con 0x7f79080b2f30 2024-04-17T10:50:35.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.688+0000 7f790dd13640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79080d2be0 con 0x7f790815dce0 2024-04-17T10:50:35.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.688+0000 7f790dd13640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79080d1fd0 con 0x7f79080cefc0 2024-04-17T10:50:35.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.688+0000 7f790cd11640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f790815dce0 0x7f79080ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:35.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.688+0000 7f7907fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f79080b2f30 0x7f79080ce8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56344/0 (socket says 172.21.15.83:56344) 2024-04-17T10:50:35.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.688+0000 7f7907fff640 1 -- 172.21.15.83:0/1514948588 learned_addr learned my addr 172.21.15.83:0/1514948588 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:35.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.688+0000 7f790d512640 1 --2- 172.21.15.83:0/1514948588 >> v2:172.21.15.167:3300/0 conn(0x7f79080cefc0 0x7f79080b6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:35.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.688+0000 7f790cd11640 1 -- 172.21.15.83:0/1514948588 >> v2:172.21.15.167:3300/0 conn(0x7f79080cefc0 msgr2=0x7f79080b6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:35.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.688+0000 7f790cd11640 1 --2- 172.21.15.83:0/1514948588 >> v2:172.21.15.167:3300/0 conn(0x7f79080cefc0 0x7f79080b6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:35.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.688+0000 7f790cd11640 1 -- 172.21.15.83:0/1514948588 >> v1:172.21.15.83:6789/0 conn(0x7f79080b2f30 legacy=0x7f79080ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:35.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.688+0000 7f79077fe640 1 -- 172.21.15.83:0/1514948588 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1174557837 0 0) 0x7f79080d3070 con 0x7f79080b2f30 2024-04-17T10:50:35.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.688+0000 7f790cd11640 1 -- 172.21.15.83:0/1514948588 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79080d22b0 con 0x7f790815dce0 2024-04-17T10:50:35.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.688+0000 7f790d512640 1 --2- 172.21.15.83:0/1514948588 >> v2:172.21.15.167:3300/0 conn(0x7f79080cefc0 0x7f79080b6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:35.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.692+0000 7f790cd11640 1 --2- 172.21.15.83:0/1514948588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f790815dce0 0x7f79080ce2b0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f790804ff80 tx=0x7f78f801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6176735849450ba7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:35.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.692+0000 7f79077fe640 1 -- 172.21.15.83:0/1514948588 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f80b5070 con 0x7f790815dce0 2024-04-17T10:50:35.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.692+0000 7f79077fe640 1 -- 172.21.15.83:0/1514948588 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f80a8970 con 0x7f790815dce0 2024-04-17T10:50:35.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.692+0000 7f79077fe640 1 -- 172.21.15.83:0/1514948588 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f80a8d40 con 0x7f790815dce0 2024-04-17T10:50:35.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.692+0000 7f790dd13640 1 -- 172.21.15.83:0/1514948588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f790815dce0 msgr2=0x7f79080ce2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:35.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.692+0000 7f790dd13640 1 --2- 172.21.15.83:0/1514948588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f790815dce0 0x7f79080ce2b0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f790804ff80 tx=0x7f78f801ec70 comp rx=0 tx=0).stop 2024-04-17T10:50:35.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.692+0000 7f790dd13640 1 -- 172.21.15.83:0/1514948588 shutdown_connections 2024-04-17T10:50:35.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.692+0000 7f790dd13640 1 --2- 172.21.15.83:0/1514948588 >> v2:172.21.15.167:3300/0 conn(0x7f79080cefc0 0x7f79080b6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:35.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.692+0000 7f790dd13640 1 --2- 172.21.15.83:0/1514948588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f790815dce0 0x7f79080ce2b0 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:35.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.692+0000 7f790dd13640 1 -- 172.21.15.83:0/1514948588 >> 172.21.15.83:0/1514948588 conn(0x7f79080d3ce0 msgr2=0x7f79080d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:35.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.692+0000 7f790dd13640 1 -- 172.21.15.83:0/1514948588 shutdown_connections 2024-04-17T10:50:35.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.692+0000 7f790dd13640 1 -- 172.21.15.83:0/1514948588 wait complete. 2024-04-17T10:50:35.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.692+0000 7f790dd13640 1 Processor -- start 2024-04-17T10:50:35.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.692+0000 7f790dd13640 1 -- start start 2024-04-17T10:50:35.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.696+0000 7f790dd13640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79080b2f30 0x7f790815f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:35.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.696+0000 7f790dd13640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79080cefc0 0x7f7908160750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:35.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.696+0000 7f790dd13640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f790815dce0 0x7f79081f3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:35.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.696+0000 7f790dd13640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79080d1c10 con 0x7f79080cefc0 2024-04-17T10:50:35.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.696+0000 7f790dd13640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79080d2870 con 0x7f790815dce0 2024-04-17T10:50:35.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.696+0000 7f790dd13640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79080d1d90 con 0x7f79080b2f30 2024-04-17T10:50:35.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.696+0000 7f7907fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79080cefc0 0x7f7908160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:35.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.696+0000 7f7907fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79080cefc0 0x7f7908160750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45136/0 (socket says 172.21.15.83:45136) 2024-04-17T10:50:35.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.696+0000 7f7907fff640 1 -- 172.21.15.83:0/689032745 learned_addr learned my addr 172.21.15.83:0/689032745 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:35.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.696+0000 7f790d512640 1 --2- 172.21.15.83:0/689032745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f790815dce0 0x7f79081f3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:35.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.696+0000 7f790cd11640 1 --2- 172.21.15.83:0/689032745 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79080b2f30 0x7f790815f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:35.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.696+0000 7f7907fff640 1 -- 172.21.15.83:0/689032745 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79080b2f30 msgr2=0x7f790815f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:35.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.696+0000 7f7907fff640 1 --2- 172.21.15.83:0/689032745 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79080b2f30 0x7f790815f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:35.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.696+0000 7f7907fff640 1 -- 172.21.15.83:0/689032745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f790815dce0 msgr2=0x7f79081f3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:35.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.696+0000 7f7907fff640 1 --2- 172.21.15.83:0/689032745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f790815dce0 0x7f79081f3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:35.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.696+0000 7f7907fff640 1 -- 172.21.15.83:0/689032745 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f790815e630 con 0x7f79080cefc0 2024-04-17T10:50:35.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.696+0000 7f7907fff640 1 --2- 172.21.15.83:0/689032745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79080cefc0 0x7f7908160750 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f78fc00c4d0 tx=0x7f78fc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:35.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.696+0000 7f79057fa640 1 -- 172.21.15.83:0/689032745 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc018070 con 0x7f79080cefc0 2024-04-17T10:50:35.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.696+0000 7f79057fa640 1 -- 172.21.15.83:0/689032745 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78fc0074e0 con 0x7f79080cefc0 2024-04-17T10:50:35.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.696+0000 7f79057fa640 1 -- 172.21.15.83:0/689032745 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc004a30 con 0x7f79080cefc0 2024-04-17T10:50:35.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.696+0000 7f790dd13640 1 -- 172.21.15.83:0/689032745 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f790815e910 con 0x7f79080cefc0 2024-04-17T10:50:35.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.696+0000 7f790dd13640 1 -- 172.21.15.83:0/689032745 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f790815eed0 con 0x7f79080cefc0 2024-04-17T10:50:35.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.696+0000 7f790dd13640 1 -- 172.21.15.83:0/689032745 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79080681a0 con 0x7f79080cefc0 2024-04-17T10:50:35.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.696+0000 7f79057fa640 1 -- 172.21.15.83:0/689032745 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f78fc004bd0 con 0x7f79080cefc0 2024-04-17T10:50:35.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.696+0000 7f79057fa640 1 --2- 172.21.15.83:0/689032745 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f78f4066f60 0x7f78f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:35.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.696+0000 7f79057fa640 1 -- 172.21.15.83:0/689032745 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f78fc084370 con 0x7f79080cefc0 2024-04-17T10:50:35.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.696+0000 7f790cd11640 1 --2- 172.21.15.83:0/689032745 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f78f4066f60 0x7f78f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:35.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.700+0000 7f790cd11640 1 --2- 172.21.15.83:0/689032745 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f78f4066f60 0x7f78f4069420 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f78f80b2000 tx=0x7f78f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:35.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.700+0000 7f79057fa640 1 -- 172.21.15.83:0/689032745 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f78fc055380 con 0x7f79080cefc0 2024-04-17T10:50:35.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.896+0000 7f790dd13640 1 -- 172.21.15.83:0/689032745 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f790804efa0 con 0x7f79080cefc0 2024-04-17T10:50:35.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.904+0000 7f79057fa640 1 -- 172.21.15.83:0/689032745 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v86) v1 ==== 118+0+0 (secure 0 0 0) 0x7f78fc05b080 con 0x7f79080cefc0 2024-04-17T10:50:35.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.908+0000 7f790dd13640 1 -- 172.21.15.83:0/689032745 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f78f4066f60 msgr2=0x7f78f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:35.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.908+0000 7f790dd13640 1 --2- 172.21.15.83:0/689032745 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f78f4066f60 0x7f78f4069420 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f78f80b2000 tx=0x7f78f8008040 comp rx=0 tx=0).stop 2024-04-17T10:50:35.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.908+0000 7f790dd13640 1 -- 172.21.15.83:0/689032745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79080cefc0 msgr2=0x7f7908160750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:35.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.908+0000 7f790dd13640 1 --2- 172.21.15.83:0/689032745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79080cefc0 0x7f7908160750 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f78fc00c4d0 tx=0x7f78fc00c990 comp rx=0 tx=0).stop 2024-04-17T10:50:35.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.908+0000 7f790dd13640 1 -- 172.21.15.83:0/689032745 shutdown_connections 2024-04-17T10:50:35.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.908+0000 7f790dd13640 1 --2- 172.21.15.83:0/689032745 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f78f4066f60 0x7f78f4069420 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:35.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.908+0000 7f790dd13640 1 --2- 172.21.15.83:0/689032745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f790815dce0 0x7f79081f3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:35.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.908+0000 7f790dd13640 1 --2- 172.21.15.83:0/689032745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79080cefc0 0x7f7908160750 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:35.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.908+0000 7f790dd13640 1 --2- 172.21.15.83:0/689032745 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79080b2f30 0x7f790815f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:35.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.908+0000 7f790dd13640 1 -- 172.21.15.83:0/689032745 >> 172.21.15.83:0/689032745 conn(0x7f79080d3ce0 msgr2=0x7f79080cc930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:35.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.908+0000 7f790dd13640 1 -- 172.21.15.83:0/689032745 shutdown_connections 2024-04-17T10:50:35.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:35.908+0000 7f790dd13640 1 -- 172.21.15.83:0/689032745 wait complete. 2024-04-17T10:50:35.920 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-17T10:50:35.920 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-17T10:50:36.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.044+0000 7f1ab0274640 1 Processor -- start 2024-04-17T10:50:36.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.044+0000 7f1ab0274640 1 -- start start 2024-04-17T10:50:36.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.044+0000 7f1ab0274640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1aa80af6a0 0x7f1aa80afb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:36.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.044+0000 7f1ab0274640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa806e470 con 0x7f1aa8071880 2024-04-17T10:50:36.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.044+0000 7f1ab0274640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa806e710 con 0x7f1aa815bba0 2024-04-17T10:50:36.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.044+0000 7f1ab0274640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa806e9f0 con 0x7f1aa80af6a0 2024-04-17T10:50:36.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.044+0000 7f1aae811640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1aa8071880 0x7f1aa8071d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56360/0 (socket says 172.21.15.83:56360) 2024-04-17T10:50:36.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.044+0000 7f1aae811640 1 -- 172.21.15.83:0/1669156110 learned_addr learned my addr 172.21.15.83:0/1669156110 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:36.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.044+0000 7f1aad80f640 1 --2- 172.21.15.83:0/1669156110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1aa80af6a0 0x7f1aa80afb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:36.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.044+0000 7f1aad00e640 1 -- 172.21.15.83:0/1669156110 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 958451673 0 0) 0x7f1aa806e470 con 0x7f1aa8071880 2024-04-17T10:50:36.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.044+0000 7f1aad00e640 1 -- 172.21.15.83:0/1669156110 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a94003530 con 0x7f1aa8071880 2024-04-17T10:50:36.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.044+0000 7f1aad00e640 1 -- 172.21.15.83:0/1669156110 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1941312072 0 0) 0x7f1a94003530 con 0x7f1aa8071880 2024-04-17T10:50:36.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.044+0000 7f1aad00e640 1 -- 172.21.15.83:0/1669156110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1aa80af6a0 msgr2=0x7f1aa80afb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:36.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.044+0000 7f1aad00e640 1 --2- 172.21.15.83:0/1669156110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1aa80af6a0 0x7f1aa80afb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:36.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.044+0000 7f1aad00e640 1 -- 172.21.15.83:0/1669156110 >> v1:172.21.15.167:6789/0 conn(0x7f1aa815bba0 legacy=0x7f1aa815bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:36.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.044+0000 7f1aad00e640 1 -- 172.21.15.83:0/1669156110 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa8072590 con 0x7f1aa8071880 2024-04-17T10:50:36.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.044+0000 7f1aad00e640 1 -- 172.21.15.83:0/1669156110 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f1aa4001a00 con 0x7f1aa8071880 2024-04-17T10:50:36.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.044+0000 7f1aad00e640 1 -- 172.21.15.83:0/1669156110 >> v1:172.21.15.83:6789/0 conn(0x7f1aa8071880 legacy=0x7f1aa8071d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:36.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.044+0000 7f1aad00e640 1 --2- 172.21.15.83:0/1669156110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a94003a60 0x7f1a94003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:36.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.044+0000 7f1aad00e640 1 --2- 172.21.15.83:0/1669156110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1aa80af6a0 0x7f1a94004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:36.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.044+0000 7f1aad00e640 1 --2- 172.21.15.83:0/1669156110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a94004da0 0x7f1a94005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:36.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.044+0000 7f1aad00e640 1 -- 172.21.15.83:0/1669156110 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa806e9f0 con 0x7f1a94003a60 2024-04-17T10:50:36.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.044+0000 7f1aad00e640 1 -- 172.21.15.83:0/1669156110 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9400bb90 con 0x7f1a94004da0 2024-04-17T10:50:36.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.044+0000 7f1aad00e640 1 -- 172.21.15.83:0/1669156110 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9400bd90 con 0x7f1aa80af6a0 2024-04-17T10:50:36.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.044+0000 7f1aae010640 1 --2- 172.21.15.83:0/1669156110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a94003a60 0x7f1a94003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:36.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.044+0000 7f1aae811640 1 --2- 172.21.15.83:0/1669156110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a94004da0 0x7f1a94005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:36.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.044+0000 7f1aad80f640 1 --2- 172.21.15.83:0/1669156110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1aa80af6a0 0x7f1a94004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:36.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.044+0000 7f1aae010640 1 -- 172.21.15.83:0/1669156110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1aa80af6a0 msgr2=0x7f1a94004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:36.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.044+0000 7f1aae010640 1 --2- 172.21.15.83:0/1669156110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1aa80af6a0 0x7f1a94004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:36.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.044+0000 7f1aae010640 1 -- 172.21.15.83:0/1669156110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a94004da0 msgr2=0x7f1a94005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:36.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.044+0000 7f1aae010640 1 --2- 172.21.15.83:0/1669156110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a94004da0 0x7f1a94005220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:36.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.044+0000 7f1aae010640 1 -- 172.21.15.83:0/1669156110 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a94003530 con 0x7f1a94003a60 2024-04-17T10:50:36.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.048+0000 7f1aae010640 1 --2- 172.21.15.83:0/1669156110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a94003a60 0x7f1a94003e80 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0001780 tx=0x7f1aa001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7df99b7127182e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:36.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.048+0000 7f1aad00e640 1 -- 172.21.15.83:0/1669156110 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa00013a0 con 0x7f1a94003a60 2024-04-17T10:50:36.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.048+0000 7f1aad00e640 1 -- 172.21.15.83:0/1669156110 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa00052f0 con 0x7f1a94003a60 2024-04-17T10:50:36.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.048+0000 7f1aad00e640 1 -- 172.21.15.83:0/1669156110 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0003840 con 0x7f1a94003a60 2024-04-17T10:50:36.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.048+0000 7f1ab0274640 1 -- 172.21.15.83:0/1669156110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a94003a60 msgr2=0x7f1a94003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:36.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.048+0000 7f1ab0274640 1 --2- 172.21.15.83:0/1669156110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a94003a60 0x7f1a94003e80 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0001780 tx=0x7f1aa001c420 comp rx=0 tx=0).stop 2024-04-17T10:50:36.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.048+0000 7f1aae811640 1 -- 172.21.15.83:0/1669156110 reap_dead start 2024-04-17T10:50:36.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.048+0000 7f1ab0274640 1 -- 172.21.15.83:0/1669156110 shutdown_connections 2024-04-17T10:50:36.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.048+0000 7f1ab0274640 1 -- 172.21.15.83:0/1669156110 >> 172.21.15.83:0/1669156110 conn(0x7f1aa80b2aa0 msgr2=0x7f1aa80d06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:36.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.048+0000 7f1ab0274640 1 -- 172.21.15.83:0/1669156110 shutdown_connections 2024-04-17T10:50:36.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.048+0000 7f1ab0274640 1 -- 172.21.15.83:0/1669156110 wait complete. 2024-04-17T10:50:36.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.048+0000 7f1ab0274640 1 Processor -- start 2024-04-17T10:50:36.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.048+0000 7f1ab0274640 1 -- start start 2024-04-17T10:50:36.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.052+0000 7f1ab0274640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a94003a60 0x7f1aa814a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:36.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.052+0000 7f1ab0274640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1aa80b4f10 0x7f1aa80b3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:36.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.052+0000 7f1ab0274640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1aa80ac9b0 0x7f1aa8160d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:36.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.052+0000 7f1ab0274640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa806fdf0 con 0x7f1aa80b4f10 2024-04-17T10:50:36.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.052+0000 7f1ab0274640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa806fc70 con 0x7f1a94003a60 2024-04-17T10:50:36.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.052+0000 7f1ab0274640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa8167770 con 0x7f1aa80ac9b0 2024-04-17T10:50:36.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.052+0000 7f1aad80f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1aa80b4f10 0x7f1aa80b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:36.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.052+0000 7f1aad80f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1aa80b4f10 0x7f1aa80b3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45156/0 (socket says 172.21.15.83:45156) 2024-04-17T10:50:36.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.052+0000 7f1aad80f640 1 -- 172.21.15.83:0/443821462 learned_addr learned my addr 172.21.15.83:0/443821462 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:36.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.052+0000 7f1aae010640 1 --2- 172.21.15.83:0/443821462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a94003a60 0x7f1aa814a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:36.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.052+0000 7f1aae811640 1 --2- 172.21.15.83:0/443821462 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1aa80ac9b0 0x7f1aa8160d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:36.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.052+0000 7f1aad80f640 1 -- 172.21.15.83:0/443821462 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1aa80ac9b0 msgr2=0x7f1aa8160d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:36.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.052+0000 7f1aad80f640 1 --2- 172.21.15.83:0/443821462 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1aa80ac9b0 0x7f1aa8160d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:36.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.052+0000 7f1aad80f640 1 -- 172.21.15.83:0/443821462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a94003a60 msgr2=0x7f1aa814a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:36.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.052+0000 7f1aad80f640 1 --2- 172.21.15.83:0/443821462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a94003a60 0x7f1aa814a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:36.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.052+0000 7f1aad80f640 1 -- 172.21.15.83:0/443821462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa80b4110 con 0x7f1aa80b4f10 2024-04-17T10:50:36.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.052+0000 7f1aad80f640 1 --2- 172.21.15.83:0/443821462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1aa80b4f10 0x7f1aa80b3810 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f1a9800e810 tx=0x7f1a9800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:36.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.052+0000 7f1a9effd640 1 -- 172.21.15.83:0/443821462 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a98017070 con 0x7f1aa80b4f10 2024-04-17T10:50:36.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.052+0000 7f1a9effd640 1 -- 172.21.15.83:0/443821462 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9800c6d0 con 0x7f1aa80b4f10 2024-04-17T10:50:36.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.052+0000 7f1ab0274640 1 -- 172.21.15.83:0/443821462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa80b43f0 con 0x7f1aa80b4f10 2024-04-17T10:50:36.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.052+0000 7f1a9effd640 1 -- 172.21.15.83:0/443821462 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9800f040 con 0x7f1aa80b4f10 2024-04-17T10:50:36.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.052+0000 7f1ab0274640 1 -- 172.21.15.83:0/443821462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa80b3d50 con 0x7f1aa80b4f10 2024-04-17T10:50:36.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.052+0000 7f1ab0274640 1 -- 172.21.15.83:0/443821462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa80a9430 con 0x7f1aa80b4f10 2024-04-17T10:50:36.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.052+0000 7f1a9effd640 1 -- 172.21.15.83:0/443821462 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1a9801a070 con 0x7f1aa80b4f10 2024-04-17T10:50:36.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.052+0000 7f1a9effd640 1 --2- 172.21.15.83:0/443821462 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1a8c066f90 0x7f1a8c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:36.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.052+0000 7f1aae010640 1 --2- 172.21.15.83:0/443821462 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1a8c066f90 0x7f1a8c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:36.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.052+0000 7f1a9effd640 1 -- 172.21.15.83:0/443821462 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a98084aa0 con 0x7f1aa80b4f10 2024-04-17T10:50:36.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.052+0000 7f1aae010640 1 --2- 172.21.15.83:0/443821462 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1a8c066f90 0x7f1a8c069450 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0001750 tx=0x7f1aa00011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:36.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.056+0000 7f1a9effd640 1 -- 172.21.15.83:0/443821462 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1a98055ab0 con 0x7f1aa80b4f10 2024-04-17T10:50:36.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.248+0000 7f1ab0274640 1 -- 172.21.15.83:0/443821462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1aa8074f90 con 0x7f1aa80b4f10 2024-04-17T10:50:36.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.920+0000 7f1a9effd640 1 -- 172.21.15.83:0/443821462 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v87) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1a980553b0 con 0x7f1aa80b4f10 2024-04-17T10:50:36.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.924+0000 7f1ab0274640 1 -- 172.21.15.83:0/443821462 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1a8c066f90 msgr2=0x7f1a8c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:36.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.924+0000 7f1ab0274640 1 --2- 172.21.15.83:0/443821462 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1a8c066f90 0x7f1a8c069450 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0001750 tx=0x7f1aa00011a0 comp rx=0 tx=0).stop 2024-04-17T10:50:36.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.924+0000 7f1ab0274640 1 -- 172.21.15.83:0/443821462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1aa80b4f10 msgr2=0x7f1aa80b3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:36.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.924+0000 7f1ab0274640 1 --2- 172.21.15.83:0/443821462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1aa80b4f10 0x7f1aa80b3810 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f1a9800e810 tx=0x7f1a9800ecd0 comp rx=0 tx=0).stop 2024-04-17T10:50:36.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.924+0000 7f1ab0274640 1 -- 172.21.15.83:0/443821462 shutdown_connections 2024-04-17T10:50:36.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.924+0000 7f1ab0274640 1 --2- 172.21.15.83:0/443821462 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1a8c066f90 0x7f1a8c069450 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:36.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.924+0000 7f1ab0274640 1 --2- 172.21.15.83:0/443821462 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1aa80ac9b0 0x7f1aa8160d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:36.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.924+0000 7f1ab0274640 1 --2- 172.21.15.83:0/443821462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1aa80b4f10 0x7f1aa80b3810 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:36.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.924+0000 7f1ab0274640 1 --2- 172.21.15.83:0/443821462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a94003a60 0x7f1aa814a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:36.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.924+0000 7f1ab0274640 1 -- 172.21.15.83:0/443821462 >> 172.21.15.83:0/443821462 conn(0x7f1aa80b2aa0 msgr2=0x7f1aa8159f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:36.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.924+0000 7f1ab0274640 1 -- 172.21.15.83:0/443821462 shutdown_connections 2024-04-17T10:50:36.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:36.924+0000 7f1ab0274640 1 -- 172.21.15.83:0/443821462 wait complete. 2024-04-17T10:50:36.935 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-17T10:50:36.935 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:50:37.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.052+0000 7fd4bcbaa640 1 Processor -- start 2024-04-17T10:50:37.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.052+0000 7fd4bcbaa640 1 -- start start 2024-04-17T10:50:37.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.056+0000 7fd4bcbaa640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd4b80b2d80 0x7fd4b8146e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:37.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.056+0000 7fd4bcbaa640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4b806d690 0x7fd4b81620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:37.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.056+0000 7fd4bcbaa640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4b8147570 con 0x7fd4b8146a00 2024-04-17T10:50:37.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.056+0000 7fd4bcbaa640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4b8162610 con 0x7fd4b80b2d80 2024-04-17T10:50:37.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.056+0000 7fd4bcbaa640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b806c960 con 0x7fd4b806d690 2024-04-17T10:50:37.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.056+0000 7fd4b5d9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd4b80b2d80 0x7fd4b8146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:37.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.056+0000 7fd4b659c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd4b8146a00 0x7fd4b80b2670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57146/0 (socket says 172.21.15.83:57146) 2024-04-17T10:50:37.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.056+0000 7fd4b659c640 1 -- 172.21.15.83:0/3311003454 learned_addr learned my addr 172.21.15.83:0/3311003454 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:37.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.056+0000 7fd4b6d9d640 1 --2- 172.21.15.83:0/3311003454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4b806d690 0x7fd4b81620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:37.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.056+0000 7fd4b5d9b640 1 -- 172.21.15.83:0/3311003454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4b806d690 msgr2=0x7fd4b81620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:37.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.056+0000 7fd4b5d9b640 1 --2- 172.21.15.83:0/3311003454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4b806d690 0x7fd4b81620d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.056+0000 7fd4b5d9b640 1 -- 172.21.15.83:0/3311003454 >> v1:172.21.15.83:6789/0 conn(0x7fd4b8146a00 legacy=0x7fd4b80b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:37.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.056+0000 7fd4b5d9b640 1 -- 172.21.15.83:0/3311003454 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b806cbb0 con 0x7fd4b80b2d80 2024-04-17T10:50:37.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.056+0000 7fd4b5d9b640 1 --2- 172.21.15.83:0/3311003454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4b80b2d80 0x7fd4b8146e70 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7fd4a00097c0 tx=0x7fd4a001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc2808441fe6afc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:37.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.056+0000 7fd4b559a640 1 -- 172.21.15.83:0/3311003454 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a00b1070 con 0x7fd4b80b2d80 2024-04-17T10:50:37.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.056+0000 7fd4b559a640 1 -- 172.21.15.83:0/3311003454 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a00ac410 con 0x7fd4b80b2d80 2024-04-17T10:50:37.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.056+0000 7fd4b559a640 1 -- 172.21.15.83:0/3311003454 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a00ac7e0 con 0x7fd4b80b2d80 2024-04-17T10:50:37.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.056+0000 7fd4bcbaa640 1 -- 172.21.15.83:0/3311003454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4b80b2d80 msgr2=0x7fd4b8146e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:37.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.056+0000 7fd4bcbaa640 1 --2- 172.21.15.83:0/3311003454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4b80b2d80 0x7fd4b8146e70 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7fd4a00097c0 tx=0x7fd4a001c4a0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.056+0000 7fd4bcbaa640 1 -- 172.21.15.83:0/3311003454 shutdown_connections 2024-04-17T10:50:37.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.060+0000 7fd4bcbaa640 1 --2- 172.21.15.83:0/3311003454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4b806d690 0x7fd4b81620d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.060+0000 7fd4bcbaa640 1 --2- 172.21.15.83:0/3311003454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4b80b2d80 0x7fd4b8146e70 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.060+0000 7fd4bcbaa640 1 -- 172.21.15.83:0/3311003454 >> 172.21.15.83:0/3311003454 conn(0x7fd4b806abe0 msgr2=0x7fd4b806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:37.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.060+0000 7fd4bcbaa640 1 -- 172.21.15.83:0/3311003454 shutdown_connections 2024-04-17T10:50:37.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.060+0000 7fd4bcbaa640 1 -- 172.21.15.83:0/3311003454 wait complete. 2024-04-17T10:50:37.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.060+0000 7fd4bcbaa640 1 Processor -- start 2024-04-17T10:50:37.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.060+0000 7fd4bcbaa640 1 -- start start 2024-04-17T10:50:37.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.060+0000 7fd4bcbaa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4b806d690 0x7fd4b815e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:37.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.060+0000 7fd4bcbaa640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4b80b2d80 0x7fd4b8139a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:37.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.060+0000 7fd4bcbaa640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4b8146a00 0x7fd4b815ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:37.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.060+0000 7fd4bcbaa640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b8071010 con 0x7fd4b806d690 2024-04-17T10:50:37.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.060+0000 7fd4bcbaa640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b8070e90 con 0x7fd4b80b2d80 2024-04-17T10:50:37.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.060+0000 7fd4bcbaa640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b815a430 con 0x7fd4b8146a00 2024-04-17T10:50:37.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.060+0000 7fd4b659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4b806d690 0x7fd4b815e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:37.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.060+0000 7fd4b659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4b806d690 0x7fd4b815e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35258/0 (socket says 172.21.15.83:35258) 2024-04-17T10:50:37.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.060+0000 7fd4b659c640 1 -- 172.21.15.83:0/1839370138 learned_addr learned my addr 172.21.15.83:0/1839370138 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:37.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.060+0000 7fd4b659c640 1 -- 172.21.15.83:0/1839370138 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4b8146a00 msgr2=0x7fd4b815ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:37.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.060+0000 7fd4b659c640 1 --2- 172.21.15.83:0/1839370138 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4b8146a00 0x7fd4b815ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.060+0000 7fd4b659c640 1 -- 172.21.15.83:0/1839370138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4b80b2d80 msgr2=0x7fd4b8139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:37.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.060+0000 7fd4b659c640 1 --2- 172.21.15.83:0/1839370138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4b80b2d80 0x7fd4b8139a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.060+0000 7fd4b659c640 1 -- 172.21.15.83:0/1839370138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b813a380 con 0x7fd4b806d690 2024-04-17T10:50:37.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.060+0000 7fd4b659c640 1 --2- 172.21.15.83:0/1839370138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4b806d690 0x7fd4b815e0d0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800c370 tx=0x7fd4a800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:37.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.060+0000 7fd4aeffd640 1 -- 172.21.15.83:0/1839370138 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a8017070 con 0x7fd4b806d690 2024-04-17T10:50:37.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.060+0000 7fd4aeffd640 1 -- 172.21.15.83:0/1839370138 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a8005cf0 con 0x7fd4b806d690 2024-04-17T10:50:37.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.060+0000 7fd4bcbaa640 1 -- 172.21.15.83:0/1839370138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4b813a660 con 0x7fd4b806d690 2024-04-17T10:50:37.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.060+0000 7fd4aeffd640 1 -- 172.21.15.83:0/1839370138 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a80072d0 con 0x7fd4b806d690 2024-04-17T10:50:37.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.060+0000 7fd4bcbaa640 1 -- 172.21.15.83:0/1839370138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4b815a0c0 con 0x7fd4b806d690 2024-04-17T10:50:37.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.064+0000 7fd4bcbaa640 1 -- 172.21.15.83:0/1839370138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4b815e6d0 con 0x7fd4b806d690 2024-04-17T10:50:37.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.064+0000 7fd4aeffd640 1 -- 172.21.15.83:0/1839370138 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd4a801a070 con 0x7fd4b806d690 2024-04-17T10:50:37.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.064+0000 7fd4aeffd640 1 --2- 172.21.15.83:0/1839370138 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd49c066f60 0x7fd49c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:37.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.064+0000 7fd4aeffd640 1 -- 172.21.15.83:0/1839370138 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4a8084270 con 0x7fd4b806d690 2024-04-17T10:50:37.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.064+0000 7fd4b5d9b640 1 --2- 172.21.15.83:0/1839370138 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd49c066f60 0x7fd49c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:37.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.064+0000 7fd4b5d9b640 1 --2- 172.21.15.83:0/1839370138 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd49c066f60 0x7fd49c069420 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd4a00ae000 tx=0x7fd4a0004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:37.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.068+0000 7fd4aeffd640 1 -- 172.21.15.83:0/1839370138 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd4a8054c30 con 0x7fd4b806d690 2024-04-17T10:50:37.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.244+0000 7fd4bcbaa640 1 -- 172.21.15.83:0/1839370138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4b806dd00 con 0x7fd4b806d690 2024-04-17T10:50:37.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.248+0000 7fd4aeffd640 1 -- 172.21.15.83:0/1839370138 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4948 (secure 0 0 0) 0x7fd4a8058cd0 con 0x7fd4b806d690 2024-04-17T10:50:37.239 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:50:37.240 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":87,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:50:36.251622+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9711},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9711":{"gid":9711,"name":"a","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/3575114599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3575114599},{"type":"v1","addr":"172.21.15.83:6837","nonce":3575114599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:50:37.240 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 87 2024-04-17T10:50:37.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.248+0000 7fd4bcbaa640 1 -- 172.21.15.83:0/1839370138 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd49c066f60 msgr2=0x7fd49c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:37.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.248+0000 7fd4bcbaa640 1 --2- 172.21.15.83:0/1839370138 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd49c066f60 0x7fd49c069420 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd4a00ae000 tx=0x7fd4a0004740 comp rx=0 tx=0).stop 2024-04-17T10:50:37.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.248+0000 7fd4bcbaa640 1 -- 172.21.15.83:0/1839370138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4b806d690 msgr2=0x7fd4b815e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:37.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.248+0000 7fd4bcbaa640 1 --2- 172.21.15.83:0/1839370138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4b806d690 0x7fd4b815e0d0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800c370 tx=0x7fd4a800c830 comp rx=0 tx=0).stop 2024-04-17T10:50:37.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.248+0000 7fd4bcbaa640 1 -- 172.21.15.83:0/1839370138 shutdown_connections 2024-04-17T10:50:37.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.248+0000 7fd4bcbaa640 1 --2- 172.21.15.83:0/1839370138 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd49c066f60 0x7fd49c069420 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.248+0000 7fd4bcbaa640 1 --2- 172.21.15.83:0/1839370138 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4b8146a00 0x7fd4b815ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.248+0000 7fd4bcbaa640 1 --2- 172.21.15.83:0/1839370138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4b80b2d80 0x7fd4b8139a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.248+0000 7fd4bcbaa640 1 --2- 172.21.15.83:0/1839370138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4b806d690 0x7fd4b815e0d0 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.248+0000 7fd4bcbaa640 1 -- 172.21.15.83:0/1839370138 >> 172.21.15.83:0/1839370138 conn(0x7fd4b806abe0 msgr2=0x7fd4b813c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:37.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.248+0000 7fd4bcbaa640 1 -- 172.21.15.83:0/1839370138 shutdown_connections 2024-04-17T10:50:37.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.248+0000 7fd4bcbaa640 1 -- 172.21.15.83:0/1839370138 wait complete. 2024-04-17T10:50:37.256 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:50:37.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.380+0000 7f52b0c87640 1 Processor -- start 2024-04-17T10:50:37.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.384+0000 7f52b0c87640 1 -- start start 2024-04-17T10:50:37.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.384+0000 7f52b0c87640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52ac0cf060 0x7f52ac0b4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:37.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.384+0000 7f52b0c87640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f52ac0b3990 0x7f52ac070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:37.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.384+0000 7f52b0c87640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f52ac0b45a0 0x7f52ac0709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:37.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.384+0000 7f52b0c87640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52ac0b2980 con 0x7f52ac0b45a0 2024-04-17T10:50:37.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.384+0000 7f52b0c87640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52ac0cf440 con 0x7f52ac0b3990 2024-04-17T10:50:37.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.384+0000 7f52b0c87640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac0b2bd0 con 0x7f52ac0cf060 2024-04-17T10:50:37.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.384+0000 7f52aad9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f52ac0b45a0 0x7f52ac0709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:37.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.384+0000 7f52aad9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f52ac0b45a0 0x7f52ac0709b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35270/0 (socket says 172.21.15.83:35270) 2024-04-17T10:50:37.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.384+0000 7f52aad9d640 1 -- 172.21.15.83:0/1685396745 learned_addr learned my addr 172.21.15.83:0/1685396745 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:37.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.384+0000 7f52aad9d640 1 -- 172.21.15.83:0/1685396745 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52ac0cf060 msgr2=0x7f52ac0b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:37.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.384+0000 7f52aa59c640 1 --2- 172.21.15.83:0/1685396745 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52ac0cf060 0x7f52ac0b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:37.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.384+0000 7f52a9d9b640 1 --2- 172.21.15.83:0/1685396745 >> v2:172.21.15.167:3300/0 conn(0x7f52ac0b3990 0x7f52ac070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:37.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.384+0000 7f52aad9d640 1 --2- 172.21.15.83:0/1685396745 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52ac0cf060 0x7f52ac0b4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.384+0000 7f52aad9d640 1 -- 172.21.15.83:0/1685396745 >> v2:172.21.15.167:3300/0 conn(0x7f52ac0b3990 msgr2=0x7f52ac070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:37.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.384+0000 7f52aad9d640 1 --2- 172.21.15.83:0/1685396745 >> v2:172.21.15.167:3300/0 conn(0x7f52ac0b3990 0x7f52ac070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.384+0000 7f52aad9d640 1 -- 172.21.15.83:0/1685396745 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ac0b2eb0 con 0x7f52ac0b45a0 2024-04-17T10:50:37.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.384+0000 7f52aad9d640 1 --2- 172.21.15.83:0/1685396745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52ac0b45a0 0x7f52ac0709b0 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f52ac070060 tx=0x7f529c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ba7d3575fe6028 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:37.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52a959a640 1 -- 172.21.15.83:0/1685396745 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529c0aa610 con 0x7f52ac0b45a0 2024-04-17T10:50:37.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52a959a640 1 -- 172.21.15.83:0/1685396745 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529c0af070 con 0x7f52ac0b45a0 2024-04-17T10:50:37.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52a959a640 1 -- 172.21.15.83:0/1685396745 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529c0aa9d0 con 0x7f52ac0b45a0 2024-04-17T10:50:37.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52b0c87640 1 -- 172.21.15.83:0/1685396745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52ac0b45a0 msgr2=0x7f52ac0709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:37.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52b0c87640 1 --2- 172.21.15.83:0/1685396745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52ac0b45a0 0x7f52ac0709b0 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f52ac070060 tx=0x7f529c01c4f0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52b0c87640 1 -- 172.21.15.83:0/1685396745 shutdown_connections 2024-04-17T10:50:37.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52b0c87640 1 --2- 172.21.15.83:0/1685396745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52ac0b45a0 0x7f52ac0709b0 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52b0c87640 1 --2- 172.21.15.83:0/1685396745 >> v2:172.21.15.167:3300/0 conn(0x7f52ac0b3990 0x7f52ac070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52b0c87640 1 --2- 172.21.15.83:0/1685396745 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52ac0cf060 0x7f52ac0b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52b0c87640 1 -- 172.21.15.83:0/1685396745 >> 172.21.15.83:0/1685396745 conn(0x7f52ac06abe0 msgr2=0x7f52ac06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:37.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52b0c87640 1 -- 172.21.15.83:0/1685396745 shutdown_connections 2024-04-17T10:50:37.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52b0c87640 1 -- 172.21.15.83:0/1685396745 wait complete. 2024-04-17T10:50:37.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52b0c87640 1 Processor -- start 2024-04-17T10:50:37.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52b0c87640 1 -- start start 2024-04-17T10:50:37.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52b0c87640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52ac0b3990 0x7f52ac153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:37.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52b0c87640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52ac0b45a0 0x7f52ac1566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:37.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52b0c87640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52ac0cf060 0x7f52ac156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:37.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52b0c87640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac15e380 con 0x7f52ac0b3990 2024-04-17T10:50:37.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52b0c87640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac15e200 con 0x7f52ac0b45a0 2024-04-17T10:50:37.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52b0c87640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac15e500 con 0x7f52ac0cf060 2024-04-17T10:50:37.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52aa59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52ac0b3990 0x7f52ac153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:37.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52aa59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52ac0b3990 0x7f52ac153b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35272/0 (socket says 172.21.15.83:35272) 2024-04-17T10:50:37.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52aa59c640 1 -- 172.21.15.83:0/2717359860 learned_addr learned my addr 172.21.15.83:0/2717359860 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:37.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52aad9d640 1 --2- 172.21.15.83:0/2717359860 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52ac0cf060 0x7f52ac156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:37.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52a9d9b640 1 --2- 172.21.15.83:0/2717359860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52ac0b45a0 0x7f52ac1566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:37.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52aa59c640 1 -- 172.21.15.83:0/2717359860 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52ac0cf060 msgr2=0x7f52ac156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:37.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52aa59c640 1 --2- 172.21.15.83:0/2717359860 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52ac0cf060 0x7f52ac156c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52aa59c640 1 -- 172.21.15.83:0/2717359860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52ac0b45a0 msgr2=0x7f52ac1566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:37.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52aa59c640 1 --2- 172.21.15.83:0/2717359860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52ac0b45a0 0x7f52ac1566c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52aa59c640 1 -- 172.21.15.83:0/2717359860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ac14eee0 con 0x7f52ac0b3990 2024-04-17T10:50:37.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52aa59c640 1 --2- 172.21.15.83:0/2717359860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52ac0b3990 0x7f52ac153b20 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f52a400cd00 tx=0x7f52a400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:37.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f529affd640 1 -- 172.21.15.83:0/2717359860 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a40039a0 con 0x7f52ac0b3990 2024-04-17T10:50:37.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f529affd640 1 -- 172.21.15.83:0/2717359860 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a4003b40 con 0x7f52ac0b3990 2024-04-17T10:50:37.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52b0c87640 1 -- 172.21.15.83:0/2717359860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52ac14f180 con 0x7f52ac0b3990 2024-04-17T10:50:37.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f529affd640 1 -- 172.21.15.83:0/2717359860 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a4004020 con 0x7f52ac0b3990 2024-04-17T10:50:37.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.388+0000 7f52b0c87640 1 -- 172.21.15.83:0/2717359860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52ac14f740 con 0x7f52ac0b3990 2024-04-17T10:50:37.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.392+0000 7f52b0c87640 1 -- 172.21.15.83:0/2717359860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52ac074f90 con 0x7f52ac0b3990 2024-04-17T10:50:37.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.392+0000 7f529affd640 1 -- 172.21.15.83:0/2717359860 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f52a40041c0 con 0x7f52ac0b3990 2024-04-17T10:50:37.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.392+0000 7f529affd640 1 --2- 172.21.15.83:0/2717359860 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f528c066f60 0x7f528c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:37.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.392+0000 7f529affd640 1 -- 172.21.15.83:0/2717359860 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f52a4085470 con 0x7f52ac0b3990 2024-04-17T10:50:37.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.392+0000 7f52a9d9b640 1 --2- 172.21.15.83:0/2717359860 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f528c066f60 0x7f528c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:37.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.392+0000 7f52a9d9b640 1 --2- 172.21.15.83:0/2717359860 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f528c066f60 0x7f528c069420 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f52ac06d9b0 tx=0x7f52a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:37.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.396+0000 7f529affd640 1 -- 172.21.15.83:0/2717359860 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f52a4014030 con 0x7f52ac0b3990 2024-04-17T10:50:37.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.544+0000 7f52b0c87640 1 -- 172.21.15.83:0/2717359860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52ac151990 con 0x7f52ac0b3990 2024-04-17T10:50:37.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.548+0000 7f529affd640 1 -- 172.21.15.83:0/2717359860 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+16607 (secure 0 0 0) 0x7f52a4056480 con 0x7f52ac0b3990 2024-04-17T10:50:37.540 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:50:37.540 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":140,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:50:35.906462+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:50:32.488186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-17T10:50:33.207211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:50:37.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.548+0000 7f52b0c87640 1 -- 172.21.15.83:0/2717359860 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f528c066f60 msgr2=0x7f528c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:37.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.548+0000 7f52b0c87640 1 --2- 172.21.15.83:0/2717359860 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f528c066f60 0x7f528c069420 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f52ac06d9b0 tx=0x7f52a0008040 comp rx=0 tx=0).stop 2024-04-17T10:50:37.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.548+0000 7f52b0c87640 1 -- 172.21.15.83:0/2717359860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52ac0b3990 msgr2=0x7f52ac153b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:37.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.548+0000 7f52b0c87640 1 --2- 172.21.15.83:0/2717359860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52ac0b3990 0x7f52ac153b20 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f52a400cd00 tx=0x7f52a400a5c0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.548+0000 7f52b0c87640 1 -- 172.21.15.83:0/2717359860 shutdown_connections 2024-04-17T10:50:37.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.548+0000 7f52b0c87640 1 --2- 172.21.15.83:0/2717359860 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f528c066f60 0x7f528c069420 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.548+0000 7f52b0c87640 1 --2- 172.21.15.83:0/2717359860 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52ac0cf060 0x7f52ac156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.548+0000 7f52b0c87640 1 --2- 172.21.15.83:0/2717359860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52ac0b45a0 0x7f52ac1566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.548+0000 7f52b0c87640 1 --2- 172.21.15.83:0/2717359860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52ac0b3990 0x7f52ac153b20 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.548+0000 7f52b0c87640 1 -- 172.21.15.83:0/2717359860 >> 172.21.15.83:0/2717359860 conn(0x7f52ac06abe0 msgr2=0x7f52ac154e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:37.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.548+0000 7f52b0c87640 1 -- 172.21.15.83:0/2717359860 shutdown_connections 2024-04-17T10:50:37.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.548+0000 7f52b0c87640 1 -- 172.21.15.83:0/2717359860 wait complete. 2024-04-17T10:50:37.557 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-17T10:50:37.557 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:50:37.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.668+0000 7f46d959e640 1 Processor -- start 2024-04-17T10:50:37.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.668+0000 7f46d959e640 1 -- start start 2024-04-17T10:50:37.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.672+0000 7f46d959e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46d415ba30 0x7f46d4072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:37.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.672+0000 7f46d959e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f46d415c960 0x7f46d40cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:37.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.672+0000 7f46d959e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46d40cfea0 con 0x7f46d40cfa10 2024-04-17T10:50:37.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.672+0000 7f46d959e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46d4071ea0 con 0x7f46d415c960 2024-04-17T10:50:37.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.672+0000 7f46d959e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d406e320 con 0x7f46d415ba30 2024-04-17T10:50:37.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.672+0000 7f46d27fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f46d415c960 0x7f46d40cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:37.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.672+0000 7f46d37fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f46d40cfa10 0x7f46d406c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57150/0 (socket says 172.21.15.83:57150) 2024-04-17T10:50:37.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.672+0000 7f46d37fe640 1 -- 172.21.15.83:0/3076673807 learned_addr learned my addr 172.21.15.83:0/3076673807 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:37.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.672+0000 7f46d27fc640 1 -- 172.21.15.83:0/3076673807 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46d415ba30 msgr2=0x7f46d4072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:37.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.672+0000 7f46d27fc640 1 --2- 172.21.15.83:0/3076673807 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46d415ba30 0x7f46d4072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.672+0000 7f46d27fc640 1 -- 172.21.15.83:0/3076673807 >> v1:172.21.15.83:6789/0 conn(0x7f46d40cfa10 legacy=0x7f46d406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:37.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.672+0000 7f46d27fc640 1 -- 172.21.15.83:0/3076673807 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d406e040 con 0x7f46d415c960 2024-04-17T10:50:37.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.672+0000 7f46d1ffb640 1 -- 172.21.15.83:0/3076673807 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3856410865 0 0) 0x7f46d40cfea0 con 0x7f46d40cfa10 2024-04-17T10:50:37.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.672+0000 7f46d27fc640 1 --2- 172.21.15.83:0/3076673807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46d415c960 0x7f46d40cf4d0 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f46c80097c0 tx=0x7f46c801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e18debf4d2ebf28 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:37.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.672+0000 7f46d1ffb640 1 -- 172.21.15.83:0/3076673807 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c80b0070 con 0x7f46d415c960 2024-04-17T10:50:37.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.672+0000 7f46d1ffb640 1 -- 172.21.15.83:0/3076673807 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c80ab600 con 0x7f46d415c960 2024-04-17T10:50:37.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.672+0000 7f46d1ffb640 1 -- 172.21.15.83:0/3076673807 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c80ab9d0 con 0x7f46d415c960 2024-04-17T10:50:37.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.672+0000 7f46d959e640 1 -- 172.21.15.83:0/3076673807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46d415c960 msgr2=0x7f46d40cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:37.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.672+0000 7f46d959e640 1 --2- 172.21.15.83:0/3076673807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46d415c960 0x7f46d40cf4d0 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f46c80097c0 tx=0x7f46c801c410 comp rx=0 tx=0).stop 2024-04-17T10:50:37.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.676+0000 7f46d959e640 1 -- 172.21.15.83:0/3076673807 shutdown_connections 2024-04-17T10:50:37.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.676+0000 7f46d959e640 1 --2- 172.21.15.83:0/3076673807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46d415c960 0x7f46d40cf4d0 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.676+0000 7f46d959e640 1 --2- 172.21.15.83:0/3076673807 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46d415ba30 0x7f46d4072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.676+0000 7f46d959e640 1 -- 172.21.15.83:0/3076673807 >> 172.21.15.83:0/3076673807 conn(0x7f46d406abe0 msgr2=0x7f46d406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:37.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.676+0000 7f46d959e640 1 -- 172.21.15.83:0/3076673807 shutdown_connections 2024-04-17T10:50:37.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.676+0000 7f46d959e640 1 -- 172.21.15.83:0/3076673807 wait complete. 2024-04-17T10:50:37.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.676+0000 7f46d959e640 1 Processor -- start 2024-04-17T10:50:37.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.676+0000 7f46d959e640 1 -- start start 2024-04-17T10:50:37.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.676+0000 7f46d959e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46d40cfa10 0x7f46d413a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:37.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.676+0000 7f46d959e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46d415ba30 0x7f46d40b2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:37.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.676+0000 7f46d959e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46d415c960 0x7f46d414e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:37.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.676+0000 7f46d959e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d406dca0 con 0x7f46d415c960 2024-04-17T10:50:37.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.676+0000 7f46d959e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d40aeae0 con 0x7f46d415ba30 2024-04-17T10:50:37.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.676+0000 7f46d959e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d406de20 con 0x7f46d40cfa10 2024-04-17T10:50:37.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.676+0000 7f46d37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46d415c960 0x7f46d414e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:37.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.676+0000 7f46d37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46d415c960 0x7f46d414e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35290/0 (socket says 172.21.15.83:35290) 2024-04-17T10:50:37.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.676+0000 7f46d37fe640 1 -- 172.21.15.83:0/4065561602 learned_addr learned my addr 172.21.15.83:0/4065561602 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:37.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.676+0000 7f46d2ffd640 1 --2- 172.21.15.83:0/4065561602 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46d40cfa10 0x7f46d413a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:37.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.676+0000 7f46d37fe640 1 -- 172.21.15.83:0/4065561602 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46d40cfa10 msgr2=0x7f46d413a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:37.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.676+0000 7f46d27fc640 1 --2- 172.21.15.83:0/4065561602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46d415ba30 0x7f46d40b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:37.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.676+0000 7f46d37fe640 1 --2- 172.21.15.83:0/4065561602 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46d40cfa10 0x7f46d413a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.676+0000 7f46d37fe640 1 -- 172.21.15.83:0/4065561602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46d415ba30 msgr2=0x7f46d40b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:37.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.676+0000 7f46d37fe640 1 --2- 172.21.15.83:0/4065561602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46d415ba30 0x7f46d40b2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.676+0000 7f46d37fe640 1 -- 172.21.15.83:0/4065561602 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d40b0fe0 con 0x7f46d415c960 2024-04-17T10:50:37.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.676+0000 7f46d2ffd640 1 --2- 172.21.15.83:0/4065561602 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46d40cfa10 0x7f46d413a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:37.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.676+0000 7f46d37fe640 1 --2- 172.21.15.83:0/4065561602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46d415c960 0x7f46d414e9b0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f46cc00c700 tx=0x7f46cc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:37.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.676+0000 7f46bb7fe640 1 -- 172.21.15.83:0/4065561602 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc018070 con 0x7f46d415c960 2024-04-17T10:50:37.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.676+0000 7f46bb7fe640 1 -- 172.21.15.83:0/4065561602 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46cc00ce10 con 0x7f46d415c960 2024-04-17T10:50:37.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.676+0000 7f46d959e640 1 -- 172.21.15.83:0/4065561602 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d414b760 con 0x7f46d415c960 2024-04-17T10:50:37.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.676+0000 7f46bb7fe640 1 -- 172.21.15.83:0/4065561602 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc00a860 con 0x7f46d415c960 2024-04-17T10:50:37.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.676+0000 7f46d959e640 1 -- 172.21.15.83:0/4065561602 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d414bd20 con 0x7f46d415c960 2024-04-17T10:50:37.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.676+0000 7f46d959e640 1 -- 172.21.15.83:0/4065561602 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d4074f90 con 0x7f46d415c960 2024-04-17T10:50:37.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.680+0000 7f46bb7fe640 1 -- 172.21.15.83:0/4065561602 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f46cc01b070 con 0x7f46d415c960 2024-04-17T10:50:37.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.680+0000 7f46bb7fe640 1 --2- 172.21.15.83:0/4065561602 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f46c0066f60 0x7f46c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:37.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.680+0000 7f46d2ffd640 1 --2- 172.21.15.83:0/4065561602 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f46c0066f60 0x7f46c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:37.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.680+0000 7f46bb7fe640 1 -- 172.21.15.83:0/4065561602 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f46cc0845d0 con 0x7f46d415c960 2024-04-17T10:50:37.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.680+0000 7f46d2ffd640 1 --2- 172.21.15.83:0/4065561602 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f46c0066f60 0x7f46c0069420 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f46c400a8b0 tx=0x7f46c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:37.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.684+0000 7f46bb7fe640 1 -- 172.21.15.83:0/4065561602 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f46cc0555e0 con 0x7f46d415c960 2024-04-17T10:50:37.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.824+0000 7f46d959e640 1 -- 172.21.15.83:0/4065561602 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f46d40cf2a0 con 0x7f46c0066f60 2024-04-17T10:50:37.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.840+0000 7f46bb7fe640 1 -- 172.21.15.83:0/4065561602 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247089 (secure 0 0 0) 0x7f46d40cf2a0 con 0x7f46c0066f60 2024-04-17T10:50:37.838 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:50:37.839 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:50:37.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.844+0000 7f46d959e640 1 -- 172.21.15.83:0/4065561602 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f46c0066f60 msgr2=0x7f46c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:37.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.844+0000 7f46d959e640 1 --2- 172.21.15.83:0/4065561602 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f46c0066f60 0x7f46c0069420 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f46c400a8b0 tx=0x7f46c4008040 comp rx=0 tx=0).stop 2024-04-17T10:50:37.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.848+0000 7f46d959e640 1 -- 172.21.15.83:0/4065561602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46d415c960 msgr2=0x7f46d414e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:37.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.848+0000 7f46d959e640 1 --2- 172.21.15.83:0/4065561602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46d415c960 0x7f46d414e9b0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f46cc00c700 tx=0x7f46cc00cbc0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.848+0000 7f46d959e640 1 -- 172.21.15.83:0/4065561602 shutdown_connections 2024-04-17T10:50:37.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.848+0000 7f46d959e640 1 --2- 172.21.15.83:0/4065561602 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f46c0066f60 0x7f46c0069420 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.848+0000 7f46d959e640 1 --2- 172.21.15.83:0/4065561602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46d415c960 0x7f46d414e9b0 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.848+0000 7f46d959e640 1 --2- 172.21.15.83:0/4065561602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46d415ba30 0x7f46d40b2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.848+0000 7f46d959e640 1 --2- 172.21.15.83:0/4065561602 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46d40cfa10 0x7f46d413a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:37.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.848+0000 7f46d959e640 1 -- 172.21.15.83:0/4065561602 >> 172.21.15.83:0/4065561602 conn(0x7f46d406abe0 msgr2=0x7f46d406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:37.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.848+0000 7f46d959e640 1 -- 172.21.15.83:0/4065561602 shutdown_connections 2024-04-17T10:50:37.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:37.848+0000 7f46d959e640 1 -- 172.21.15.83:0/4065561602 wait complete. 2024-04-17T10:50:37.855 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":512,"stamp":"2024-04-17T10:50:36.400745+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":494524,"kb_used_data":14592,"kb_used_omap":12,"kb_used_meta":479795,"kb_avail":754480196,"statfs":{"total":773094113280,"available":772587720704,"internally_reserved":0,"allocated":14942208,"data_stored":7189694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":491310640},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001315"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.876422+0000","last_change":"2024-04-17T10:50:33.876422+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.876422+0000","last_undegraded":"2024-04-17T10:50:33.876422+0000","last_fullsized":"2024-04-17T10:50:33.876422+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.878304+0000","last_change":"2024-04-17T10:50:33.878304+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.878304+0000","last_undegraded":"2024-04-17T10:50:33.878304+0000","last_fullsized":"2024-04-17T10:50:33.878304+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.880142+0000","last_change":"2024-04-17T10:50:33.880142+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.880142+0000","last_undegraded":"2024-04-17T10:50:33.880142+0000","last_fullsized":"2024-04-17T10:50:33.880142+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.871442+0000","last_change":"2024-04-17T10:50:33.871442+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.871442+0000","last_undegraded":"2024-04-17T10:50:33.871442+0000","last_fullsized":"2024-04-17T10:50:33.871442+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.879260+0000","last_change":"2024-04-17T10:50:33.879260+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.879260+0000","last_undegraded":"2024-04-17T10:50:33.879260+0000","last_fullsized":"2024-04-17T10:50:33.879260+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.875526+0000","last_change":"2024-04-17T10:50:33.875526+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.875526+0000","last_undegraded":"2024-04-17T10:50:33.875526+0000","last_fullsized":"2024-04-17T10:50:33.875526+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.884342+0000","last_change":"2024-04-17T10:50:33.884342+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.884342+0000","last_undegraded":"2024-04-17T10:50:33.884342+0000","last_fullsized":"2024-04-17T10:50:33.884342+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.875967+0000","last_change":"2024-04-17T10:50:33.875967+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.875967+0000","last_undegraded":"2024-04-17T10:50:33.875967+0000","last_fullsized":"2024-04-17T10:50:33.875967+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.881528+0000","last_change":"2024-04-17T10:50:33.881528+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.881528+0000","last_undegraded":"2024-04-17T10:50:33.881528+0000","last_fullsized":"2024-04-17T10:50:33.881528+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.874184+0000","last_change":"2024-04-17T10:50:33.874184+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.874184+0000","last_undegraded":"2024-04-17T10:50:33.874184+0000","last_fullsized":"2024-04-17T10:50:33.874184+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.884228+0000","last_change":"2024-04-17T10:50:33.884228+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.884228+0000","last_undegraded":"2024-04-17T10:50:33.884228+0000","last_fullsized":"2024-04-17T10:50:33.884228+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.870313+0000","last_change":"2024-04-17T10:50:33.870313+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.870313+0000","last_undegraded":"2024-04-17T10:50:33.870313+0000","last_fullsized":"2024-04-17T10:50:33.870313+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.878811+0000","last_change":"2024-04-17T10:50:33.878811+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.878811+0000","last_undegraded":"2024-04-17T10:50:33.878811+0000","last_fullsized":"2024-04-17T10:50:33.878811+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.875112+0000","last_change":"2024-04-17T10:50:33.875112+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.875112+0000","last_undegraded":"2024-04-17T10:50:33.875112+0000","last_fullsized":"2024-04-17T10:50:33.875112+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.869759+0000","last_change":"2024-04-17T10:50:33.869759+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.869759+0000","last_undegraded":"2024-04-17T10:50:33.869759+0000","last_fullsized":"2024-04-17T10:50:33.869759+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.883895+0000","last_change":"2024-04-17T10:50:33.883895+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.883895+0000","last_undegraded":"2024-04-17T10:50:33.883895+0000","last_fullsized":"2024-04-17T10:50:33.883895+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.871401+0000","last_change":"2024-04-17T10:50:33.871401+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.871401+0000","last_undegraded":"2024-04-17T10:50:33.871401+0000","last_fullsized":"2024-04-17T10:50:33.871401+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.881276+0000","last_change":"2024-04-17T10:50:33.881276+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.881276+0000","last_undegraded":"2024-04-17T10:50:33.881276+0000","last_fullsized":"2024-04-17T10:50:33.881276+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.883559+0000","last_change":"2024-04-17T10:50:33.883559+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.883559+0000","last_undegraded":"2024-04-17T10:50:33.883559+0000","last_fullsized":"2024-04-17T10:50:33.883559+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.885482+0000","last_change":"2024-04-17T10:50:33.885482+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.885482+0000","last_undegraded":"2024-04-17T10:50:33.885482+0000","last_fullsized":"2024-04-17T10:50:33.885482+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.871386+0000","last_change":"2024-04-17T10:50:33.871386+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.871386+0000","last_undegraded":"2024-04-17T10:50:33.871386+0000","last_fullsized":"2024-04-17T10:50:33.871386+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.874084+0000","last_change":"2024-04-17T10:50:33.874084+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.874084+0000","last_undegraded":"2024-04-17T10:50:33.874084+0000","last_fullsized":"2024-04-17T10:50:33.874084+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.869757+0000","last_change":"2024-04-17T10:50:33.869757+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.869757+0000","last_undegraded":"2024-04-17T10:50:33.869757+0000","last_fullsized":"2024-04-17T10:50:33.869757+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.873283+0000","last_change":"2024-04-17T10:50:33.873283+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.873283+0000","last_undegraded":"2024-04-17T10:50:33.873283+0000","last_fullsized":"2024-04-17T10:50:33.873283+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.883856+0000","last_change":"2024-04-17T10:50:33.883856+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.883856+0000","last_undegraded":"2024-04-17T10:50:33.883856+0000","last_fullsized":"2024-04-17T10:50:33.883856+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.881781+0000","last_change":"2024-04-17T10:50:33.881781+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.881781+0000","last_undegraded":"2024-04-17T10:50:33.881781+0000","last_fullsized":"2024-04-17T10:50:33.881781+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.881470+0000","last_change":"2024-04-17T10:50:33.881470+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.881470+0000","last_undegraded":"2024-04-17T10:50:33.881470+0000","last_fullsized":"2024-04-17T10:50:33.881470+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.885926+0000","last_change":"2024-04-17T10:50:33.885926+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.885926+0000","last_undegraded":"2024-04-17T10:50:33.885926+0000","last_fullsized":"2024-04-17T10:50:33.885926+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.875431+0000","last_change":"2024-04-17T10:50:33.875431+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.875431+0000","last_undegraded":"2024-04-17T10:50:33.875431+0000","last_fullsized":"2024-04-17T10:50:33.875431+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.873345+0000","last_change":"2024-04-17T10:50:33.873345+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.873345+0000","last_undegraded":"2024-04-17T10:50:33.873345+0000","last_fullsized":"2024-04-17T10:50:33.873345+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.878027+0000","last_change":"2024-04-17T10:50:33.878027+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.878027+0000","last_undegraded":"2024-04-17T10:50:33.878027+0000","last_fullsized":"2024-04-17T10:50:33.878027+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.880017+0000","last_change":"2024-04-17T10:50:33.880017+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.880017+0000","last_undegraded":"2024-04-17T10:50:33.880017+0000","last_fullsized":"2024-04-17T10:50:33.880017+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.871860+0000","last_change":"2024-04-17T10:50:33.871860+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.871860+0000","last_undegraded":"2024-04-17T10:50:33.871860+0000","last_fullsized":"2024-04-17T10:50:33.871860+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.896574+0000","last_change":"2024-04-17T10:50:33.891540+0000","last_active":"2024-04-17T10:50:33.896574+0000","last_peered":"2024-04-17T10:50:33.896574+0000","last_clean":"2024-04-17T10:50:33.896574+0000","last_became_active":"2024-04-17T10:50:33.890837+0000","last_became_peered":"2024-04-17T10:50:33.890837+0000","last_unstale":"2024-04-17T10:50:33.896574+0000","last_undegraded":"2024-04-17T10:50:33.896574+0000","last_fullsized":"2024-04-17T10:50:33.896574+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:16:19.474043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.880475+0000","last_change":"2024-04-17T10:50:33.880475+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.880475+0000","last_undegraded":"2024-04-17T10:50:33.880475+0000","last_fullsized":"2024-04-17T10:50:33.880475+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.893863+0000","last_change":"2024-04-17T10:50:33.890842+0000","last_active":"2024-04-17T10:50:33.893863+0000","last_peered":"2024-04-17T10:50:33.893863+0000","last_clean":"2024-04-17T10:50:33.893863+0000","last_became_active":"2024-04-17T10:50:33.890472+0000","last_became_peered":"2024-04-17T10:50:33.890472+0000","last_unstale":"2024-04-17T10:50:33.893863+0000","last_undegraded":"2024-04-17T10:50:33.893863+0000","last_fullsized":"2024-04-17T10:50:33.893863+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:52:36.198523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.876197+0000","last_change":"2024-04-17T10:50:33.876197+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.876197+0000","last_undegraded":"2024-04-17T10:50:33.876197+0000","last_fullsized":"2024-04-17T10:50:33.876197+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.900479+0000","last_change":"2024-04-17T10:50:33.899787+0000","last_active":"2024-04-17T10:50:33.900479+0000","last_peered":"2024-04-17T10:50:33.900479+0000","last_clean":"2024-04-17T10:50:33.900479+0000","last_became_active":"2024-04-17T10:50:33.899273+0000","last_became_peered":"2024-04-17T10:50:33.899273+0000","last_unstale":"2024-04-17T10:50:33.900479+0000","last_undegraded":"2024-04-17T10:50:33.900479+0000","last_fullsized":"2024-04-17T10:50:33.900479+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:08:11.608039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.870653+0000","last_change":"2024-04-17T10:50:33.870653+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.870653+0000","last_undegraded":"2024-04-17T10:50:33.870653+0000","last_fullsized":"2024-04-17T10:50:33.870653+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.887943+0000","last_change":"2024-04-17T10:50:33.884473+0000","last_active":"2024-04-17T10:50:33.887943+0000","last_peered":"2024-04-17T10:50:33.887943+0000","last_clean":"2024-04-17T10:50:33.887943+0000","last_became_active":"2024-04-17T10:50:33.883898+0000","last_became_peered":"2024-04-17T10:50:33.883898+0000","last_unstale":"2024-04-17T10:50:33.887943+0000","last_undegraded":"2024-04-17T10:50:33.887943+0000","last_fullsized":"2024-04-17T10:50:33.887943+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:03:31.940294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.877422+0000","last_change":"2024-04-17T10:50:33.877422+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.877422+0000","last_undegraded":"2024-04-17T10:50:33.877422+0000","last_fullsized":"2024-04-17T10:50:33.877422+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.891469+0000","last_change":"2024-04-17T10:50:33.888560+0000","last_active":"2024-04-17T10:50:33.891469+0000","last_peered":"2024-04-17T10:50:33.891469+0000","last_clean":"2024-04-17T10:50:33.891469+0000","last_became_active":"2024-04-17T10:50:33.888155+0000","last_became_peered":"2024-04-17T10:50:33.888155+0000","last_unstale":"2024-04-17T10:50:33.891469+0000","last_undegraded":"2024-04-17T10:50:33.891469+0000","last_fullsized":"2024-04-17T10:50:33.891469+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:19:11.890360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.872397+0000","last_change":"2024-04-17T10:50:33.872397+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.872397+0000","last_undegraded":"2024-04-17T10:50:33.872397+0000","last_fullsized":"2024-04-17T10:50:33.872397+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.898062+0000","last_change":"2024-04-17T10:50:33.896476+0000","last_active":"2024-04-17T10:50:33.898062+0000","last_peered":"2024-04-17T10:50:33.898062+0000","last_clean":"2024-04-17T10:50:33.898062+0000","last_became_active":"2024-04-17T10:50:33.895863+0000","last_became_peered":"2024-04-17T10:50:33.895863+0000","last_unstale":"2024-04-17T10:50:33.898062+0000","last_undegraded":"2024-04-17T10:50:33.898062+0000","last_fullsized":"2024-04-17T10:50:33.898062+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:49:13.741700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.878606+0000","last_change":"2024-04-17T10:50:33.878606+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.878606+0000","last_undegraded":"2024-04-17T10:50:33.878606+0000","last_fullsized":"2024-04-17T10:50:33.878606+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.898833+0000","last_change":"2024-04-17T10:50:33.895471+0000","last_active":"2024-04-17T10:50:33.898833+0000","last_peered":"2024-04-17T10:50:33.898833+0000","last_clean":"2024-04-17T10:50:33.898833+0000","last_became_active":"2024-04-17T10:50:33.895069+0000","last_became_peered":"2024-04-17T10:50:33.895069+0000","last_unstale":"2024-04-17T10:50:33.898833+0000","last_undegraded":"2024-04-17T10:50:33.898833+0000","last_fullsized":"2024-04-17T10:50:33.898833+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:35:29.165208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.879870+0000","last_change":"2024-04-17T10:50:33.879870+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.879870+0000","last_undegraded":"2024-04-17T10:50:33.879870+0000","last_fullsized":"2024-04-17T10:50:33.879870+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.887894+0000","last_change":"2024-04-17T10:50:33.885177+0000","last_active":"2024-04-17T10:50:33.887894+0000","last_peered":"2024-04-17T10:50:33.887894+0000","last_clean":"2024-04-17T10:50:33.887894+0000","last_became_active":"2024-04-17T10:50:33.884763+0000","last_became_peered":"2024-04-17T10:50:33.884763+0000","last_unstale":"2024-04-17T10:50:33.887894+0000","last_undegraded":"2024-04-17T10:50:33.887894+0000","last_fullsized":"2024-04-17T10:50:33.887894+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:49.010628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.876872+0000","last_change":"2024-04-17T10:50:33.876872+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.876872+0000","last_undegraded":"2024-04-17T10:50:33.876872+0000","last_fullsized":"2024-04-17T10:50:33.876872+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.896605+0000","last_change":"2024-04-17T10:50:33.895658+0000","last_active":"2024-04-17T10:50:33.896605+0000","last_peered":"2024-04-17T10:50:33.896605+0000","last_clean":"2024-04-17T10:50:33.896605+0000","last_became_active":"2024-04-17T10:50:33.894228+0000","last_became_peered":"2024-04-17T10:50:33.894228+0000","last_unstale":"2024-04-17T10:50:33.896605+0000","last_undegraded":"2024-04-17T10:50:33.896605+0000","last_fullsized":"2024-04-17T10:50:33.896605+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:02:11.481439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.872072+0000","last_change":"2024-04-17T10:50:33.872072+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.872072+0000","last_undegraded":"2024-04-17T10:50:33.872072+0000","last_fullsized":"2024-04-17T10:50:33.872072+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.896231+0000","last_change":"2024-04-17T10:50:33.890039+0000","last_active":"2024-04-17T10:50:33.896231+0000","last_peered":"2024-04-17T10:50:33.896231+0000","last_clean":"2024-04-17T10:50:33.896231+0000","last_became_active":"2024-04-17T10:50:33.889635+0000","last_became_peered":"2024-04-17T10:50:33.889635+0000","last_unstale":"2024-04-17T10:50:33.896231+0000","last_undegraded":"2024-04-17T10:50:33.896231+0000","last_fullsized":"2024-04-17T10:50:33.896231+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:02:11.481439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.879906+0000","last_change":"2024-04-17T10:50:33.879906+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.879906+0000","last_undegraded":"2024-04-17T10:50:33.879906+0000","last_fullsized":"2024-04-17T10:50:33.879906+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.904390+0000","last_change":"2024-04-17T10:50:33.898934+0000","last_active":"2024-04-17T10:50:33.904390+0000","last_peered":"2024-04-17T10:50:33.904390+0000","last_clean":"2024-04-17T10:50:33.904390+0000","last_became_active":"2024-04-17T10:50:33.897331+0000","last_became_peered":"2024-04-17T10:50:33.897331+0000","last_unstale":"2024-04-17T10:50:33.904390+0000","last_undegraded":"2024-04-17T10:50:33.904390+0000","last_fullsized":"2024-04-17T10:50:33.904390+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:02:11.481439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.881117+0000","last_change":"2024-04-17T10:50:33.881117+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.881117+0000","last_undegraded":"2024-04-17T10:50:33.881117+0000","last_fullsized":"2024-04-17T10:50:33.881117+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.901174+0000","last_change":"2024-04-17T10:50:33.900576+0000","last_active":"2024-04-17T10:50:33.901174+0000","last_peered":"2024-04-17T10:50:33.901174+0000","last_clean":"2024-04-17T10:50:33.901174+0000","last_became_active":"2024-04-17T10:50:33.899933+0000","last_became_peered":"2024-04-17T10:50:33.899933+0000","last_unstale":"2024-04-17T10:50:33.901174+0000","last_undegraded":"2024-04-17T10:50:33.901174+0000","last_fullsized":"2024-04-17T10:50:33.901174+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:08:11.608039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.877267+0000","last_change":"2024-04-17T10:50:33.877267+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.877267+0000","last_undegraded":"2024-04-17T10:50:33.877267+0000","last_fullsized":"2024-04-17T10:50:33.877267+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.897145+0000","last_change":"2024-04-17T10:50:33.890641+0000","last_active":"2024-04-17T10:50:33.897145+0000","last_peered":"2024-04-17T10:50:33.897145+0000","last_clean":"2024-04-17T10:50:33.897145+0000","last_became_active":"2024-04-17T10:50:33.889972+0000","last_became_peered":"2024-04-17T10:50:33.889972+0000","last_unstale":"2024-04-17T10:50:33.897145+0000","last_undegraded":"2024-04-17T10:50:33.897145+0000","last_fullsized":"2024-04-17T10:50:33.897145+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:29:06.526982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.875628+0000","last_change":"2024-04-17T10:50:33.875628+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.875628+0000","last_undegraded":"2024-04-17T10:50:33.875628+0000","last_fullsized":"2024-04-17T10:50:33.875628+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.899430+0000","last_change":"2024-04-17T10:50:33.897839+0000","last_active":"2024-04-17T10:50:33.899430+0000","last_peered":"2024-04-17T10:50:33.899430+0000","last_clean":"2024-04-17T10:50:33.899430+0000","last_became_active":"2024-04-17T10:50:33.897168+0000","last_became_peered":"2024-04-17T10:50:33.897168+0000","last_unstale":"2024-04-17T10:50:33.899430+0000","last_undegraded":"2024-04-17T10:50:33.899430+0000","last_fullsized":"2024-04-17T10:50:33.899430+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:25:56.552880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.878461+0000","last_change":"2024-04-17T10:50:33.878461+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.878461+0000","last_undegraded":"2024-04-17T10:50:33.878461+0000","last_fullsized":"2024-04-17T10:50:33.878461+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.899185+0000","last_change":"2024-04-17T10:50:33.895104+0000","last_active":"2024-04-17T10:50:33.899185+0000","last_peered":"2024-04-17T10:50:33.899185+0000","last_clean":"2024-04-17T10:50:33.899185+0000","last_became_active":"2024-04-17T10:50:33.894652+0000","last_became_peered":"2024-04-17T10:50:33.894652+0000","last_unstale":"2024-04-17T10:50:33.899185+0000","last_undegraded":"2024-04-17T10:50:33.899185+0000","last_fullsized":"2024-04-17T10:50:33.899185+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:17:40.737496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.871177+0000","last_change":"2024-04-17T10:50:33.871177+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.871177+0000","last_undegraded":"2024-04-17T10:50:33.871177+0000","last_fullsized":"2024-04-17T10:50:33.871177+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.894016+0000","last_change":"2024-04-17T10:50:33.892680+0000","last_active":"2024-04-17T10:50:33.894016+0000","last_peered":"2024-04-17T10:50:33.894016+0000","last_clean":"2024-04-17T10:50:33.894016+0000","last_became_active":"2024-04-17T10:50:33.891996+0000","last_became_peered":"2024-04-17T10:50:33.891996+0000","last_unstale":"2024-04-17T10:50:33.894016+0000","last_undegraded":"2024-04-17T10:50:33.894016+0000","last_fullsized":"2024-04-17T10:50:33.894016+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:36.242473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.880302+0000","last_change":"2024-04-17T10:50:33.880302+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.880302+0000","last_undegraded":"2024-04-17T10:50:33.880302+0000","last_fullsized":"2024-04-17T10:50:33.880302+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.899936+0000","last_change":"2024-04-17T10:50:33.898138+0000","last_active":"2024-04-17T10:50:33.899936+0000","last_peered":"2024-04-17T10:50:33.899936+0000","last_clean":"2024-04-17T10:50:33.899936+0000","last_became_active":"2024-04-17T10:50:33.896506+0000","last_became_peered":"2024-04-17T10:50:33.896506+0000","last_unstale":"2024-04-17T10:50:33.899936+0000","last_undegraded":"2024-04-17T10:50:33.899936+0000","last_fullsized":"2024-04-17T10:50:33.899936+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:35:46.656830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.880316+0000","last_change":"2024-04-17T10:50:33.880316+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.880316+0000","last_undegraded":"2024-04-17T10:50:33.880316+0000","last_fullsized":"2024-04-17T10:50:33.880316+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.890096+0000","last_change":"2024-04-17T10:50:33.887380+0000","last_active":"2024-04-17T10:50:33.890096+0000","last_peered":"2024-04-17T10:50:33.890096+0000","last_clean":"2024-04-17T10:50:33.890096+0000","last_became_active":"2024-04-17T10:50:33.887003+0000","last_became_peered":"2024-04-17T10:50:33.887003+0000","last_unstale":"2024-04-17T10:50:33.890096+0000","last_undegraded":"2024-04-17T10:50:33.890096+0000","last_fullsized":"2024-04-17T10:50:33.890096+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:25:00.410077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.877105+0000","last_change":"2024-04-17T10:50:33.877105+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.877105+0000","last_undegraded":"2024-04-17T10:50:33.877105+0000","last_fullsized":"2024-04-17T10:50:33.877105+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.893790+0000","last_change":"2024-04-17T10:50:33.890628+0000","last_active":"2024-04-17T10:50:33.893790+0000","last_peered":"2024-04-17T10:50:33.893790+0000","last_clean":"2024-04-17T10:50:33.893790+0000","last_became_active":"2024-04-17T10:50:33.890232+0000","last_became_peered":"2024-04-17T10:50:33.890232+0000","last_unstale":"2024-04-17T10:50:33.893790+0000","last_undegraded":"2024-04-17T10:50:33.893790+0000","last_fullsized":"2024-04-17T10:50:33.893790+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:55.719217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.872686+0000","last_change":"2024-04-17T10:50:33.872686+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.872686+0000","last_undegraded":"2024-04-17T10:50:33.872686+0000","last_fullsized":"2024-04-17T10:50:33.872686+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.904423+0000","last_change":"2024-04-17T10:50:33.901531+0000","last_active":"2024-04-17T10:50:33.904423+0000","last_peered":"2024-04-17T10:50:33.904423+0000","last_clean":"2024-04-17T10:50:33.904423+0000","last_became_active":"2024-04-17T10:50:33.901224+0000","last_became_peered":"2024-04-17T10:50:33.901224+0000","last_unstale":"2024-04-17T10:50:33.904423+0000","last_undegraded":"2024-04-17T10:50:33.904423+0000","last_fullsized":"2024-04-17T10:50:33.904423+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:16:19.474043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.891453+0000","last_change":"2024-04-17T10:50:33.891453+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.891453+0000","last_undegraded":"2024-04-17T10:50:33.891453+0000","last_fullsized":"2024-04-17T10:50:33.891453+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.896430+0000","last_change":"2024-04-17T10:50:33.891385+0000","last_active":"2024-04-17T10:50:33.896430+0000","last_peered":"2024-04-17T10:50:33.896430+0000","last_clean":"2024-04-17T10:50:33.896430+0000","last_became_active":"2024-04-17T10:50:33.890896+0000","last_became_peered":"2024-04-17T10:50:33.890896+0000","last_unstale":"2024-04-17T10:50:33.896430+0000","last_undegraded":"2024-04-17T10:50:33.896430+0000","last_fullsized":"2024-04-17T10:50:33.896430+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:35:46.656830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.870668+0000","last_change":"2024-04-17T10:50:33.870668+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.870668+0000","last_undegraded":"2024-04-17T10:50:33.870668+0000","last_fullsized":"2024-04-17T10:50:33.870668+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.896755+0000","last_change":"2024-04-17T10:50:33.893344+0000","last_active":"2024-04-17T10:50:33.896755+0000","last_peered":"2024-04-17T10:50:33.896755+0000","last_clean":"2024-04-17T10:50:33.896755+0000","last_became_active":"2024-04-17T10:50:33.892680+0000","last_became_peered":"2024-04-17T10:50:33.892680+0000","last_unstale":"2024-04-17T10:50:33.896755+0000","last_undegraded":"2024-04-17T10:50:33.896755+0000","last_fullsized":"2024-04-17T10:50:33.896755+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:01:25.099953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.884791+0000","last_change":"2024-04-17T10:50:33.884791+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.884791+0000","last_undegraded":"2024-04-17T10:50:33.884791+0000","last_fullsized":"2024-04-17T10:50:33.884791+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.900021+0000","last_change":"2024-04-17T10:50:33.897362+0000","last_active":"2024-04-17T10:50:33.900021+0000","last_peered":"2024-04-17T10:50:33.900021+0000","last_clean":"2024-04-17T10:50:33.900021+0000","last_became_active":"2024-04-17T10:50:33.896738+0000","last_became_peered":"2024-04-17T10:50:33.896738+0000","last_unstale":"2024-04-17T10:50:33.900021+0000","last_undegraded":"2024-04-17T10:50:33.900021+0000","last_fullsized":"2024-04-17T10:50:33.900021+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:49.010628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.875977+0000","last_change":"2024-04-17T10:50:33.875977+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.875977+0000","last_undegraded":"2024-04-17T10:50:33.875977+0000","last_fullsized":"2024-04-17T10:50:33.875977+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.896611+0000","last_change":"2024-04-17T10:50:33.893658+0000","last_active":"2024-04-17T10:50:33.896611+0000","last_peered":"2024-04-17T10:50:33.896611+0000","last_clean":"2024-04-17T10:50:33.896611+0000","last_became_active":"2024-04-17T10:50:33.893023+0000","last_became_peered":"2024-04-17T10:50:33.893023+0000","last_unstale":"2024-04-17T10:50:33.896611+0000","last_undegraded":"2024-04-17T10:50:33.896611+0000","last_fullsized":"2024-04-17T10:50:33.896611+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:48:29.238257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.880974+0000","last_change":"2024-04-17T10:50:33.880974+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.880974+0000","last_undegraded":"2024-04-17T10:50:33.880974+0000","last_fullsized":"2024-04-17T10:50:33.880974+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.892795+0000","last_change":"2024-04-17T10:50:33.887529+0000","last_active":"2024-04-17T10:50:33.892795+0000","last_peered":"2024-04-17T10:50:33.892795+0000","last_clean":"2024-04-17T10:50:33.892795+0000","last_became_active":"2024-04-17T10:50:33.887152+0000","last_became_peered":"2024-04-17T10:50:33.887152+0000","last_unstale":"2024-04-17T10:50:33.892795+0000","last_undegraded":"2024-04-17T10:50:33.892795+0000","last_fullsized":"2024-04-17T10:50:33.892795+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:04.970052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.878592+0000","last_change":"2024-04-17T10:50:33.878592+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.878592+0000","last_undegraded":"2024-04-17T10:50:33.878592+0000","last_fullsized":"2024-04-17T10:50:33.878592+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.892986+0000","last_change":"2024-04-17T10:50:33.889203+0000","last_active":"2024-04-17T10:50:33.892986+0000","last_peered":"2024-04-17T10:50:33.892986+0000","last_clean":"2024-04-17T10:50:33.892986+0000","last_became_active":"2024-04-17T10:50:33.888312+0000","last_became_peered":"2024-04-17T10:50:33.888312+0000","last_unstale":"2024-04-17T10:50:33.892986+0000","last_undegraded":"2024-04-17T10:50:33.892986+0000","last_fullsized":"2024-04-17T10:50:33.892986+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:23:11.975227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.876607+0000","last_change":"2024-04-17T10:50:33.876607+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.876607+0000","last_undegraded":"2024-04-17T10:50:33.876607+0000","last_fullsized":"2024-04-17T10:50:33.876607+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.892508+0000","last_change":"2024-04-17T10:50:33.882648+0000","last_active":"2024-04-17T10:50:33.892508+0000","last_peered":"2024-04-17T10:50:33.892508+0000","last_clean":"2024-04-17T10:50:33.892508+0000","last_became_active":"2024-04-17T10:50:33.882297+0000","last_became_peered":"2024-04-17T10:50:33.882297+0000","last_unstale":"2024-04-17T10:50:33.892508+0000","last_undegraded":"2024-04-17T10:50:33.892508+0000","last_fullsized":"2024-04-17T10:50:33.892508+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:29:30.703224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.879901+0000","last_change":"2024-04-17T10:50:33.879901+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.879901+0000","last_undegraded":"2024-04-17T10:50:33.879901+0000","last_fullsized":"2024-04-17T10:50:33.879901+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.904388+0000","last_change":"2024-04-17T10:50:33.900315+0000","last_active":"2024-04-17T10:50:33.904388+0000","last_peered":"2024-04-17T10:50:33.904388+0000","last_clean":"2024-04-17T10:50:33.904388+0000","last_became_active":"2024-04-17T10:50:33.899987+0000","last_became_peered":"2024-04-17T10:50:33.899987+0000","last_unstale":"2024-04-17T10:50:33.904388+0000","last_undegraded":"2024-04-17T10:50:33.904388+0000","last_fullsized":"2024-04-17T10:50:33.904388+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:35:46.656830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.878334+0000","last_change":"2024-04-17T10:50:33.878334+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.878334+0000","last_undegraded":"2024-04-17T10:50:33.878334+0000","last_fullsized":"2024-04-17T10:50:33.878334+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.896788+0000","last_change":"2024-04-17T10:50:33.894342+0000","last_active":"2024-04-17T10:50:33.896788+0000","last_peered":"2024-04-17T10:50:33.896788+0000","last_clean":"2024-04-17T10:50:33.896788+0000","last_became_active":"2024-04-17T10:50:33.893720+0000","last_became_peered":"2024-04-17T10:50:33.893720+0000","last_unstale":"2024-04-17T10:50:33.896788+0000","last_undegraded":"2024-04-17T10:50:33.896788+0000","last_fullsized":"2024-04-17T10:50:33.896788+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:21:59.523164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.869757+0000","last_change":"2024-04-17T10:50:33.869757+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.869757+0000","last_undegraded":"2024-04-17T10:50:33.869757+0000","last_fullsized":"2024-04-17T10:50:33.869757+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.898205+0000","last_change":"2024-04-17T10:50:33.895914+0000","last_active":"2024-04-17T10:50:33.898205+0000","last_peered":"2024-04-17T10:50:33.898205+0000","last_clean":"2024-04-17T10:50:33.898205+0000","last_became_active":"2024-04-17T10:50:33.893274+0000","last_became_peered":"2024-04-17T10:50:33.893274+0000","last_unstale":"2024-04-17T10:50:33.898205+0000","last_undegraded":"2024-04-17T10:50:33.898205+0000","last_fullsized":"2024-04-17T10:50:33.898205+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:44:02.588052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.877441+0000","last_change":"2024-04-17T10:50:33.877441+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.877441+0000","last_undegraded":"2024-04-17T10:50:33.877441+0000","last_fullsized":"2024-04-17T10:50:33.877441+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.896924+0000","last_change":"2024-04-17T10:50:33.893329+0000","last_active":"2024-04-17T10:50:33.896924+0000","last_peered":"2024-04-17T10:50:33.896924+0000","last_clean":"2024-04-17T10:50:33.896924+0000","last_became_active":"2024-04-17T10:50:33.892918+0000","last_became_peered":"2024-04-17T10:50:33.892918+0000","last_unstale":"2024-04-17T10:50:33.896924+0000","last_undegraded":"2024-04-17T10:50:33.896924+0000","last_fullsized":"2024-04-17T10:50:33.896924+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:43:18.553237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.885117+0000","last_change":"2024-04-17T10:50:33.885117+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.885117+0000","last_undegraded":"2024-04-17T10:50:33.885117+0000","last_fullsized":"2024-04-17T10:50:33.885117+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.898263+0000","last_change":"2024-04-17T10:50:33.896124+0000","last_active":"2024-04-17T10:50:33.898263+0000","last_peered":"2024-04-17T10:50:33.898263+0000","last_clean":"2024-04-17T10:50:33.898263+0000","last_became_active":"2024-04-17T10:50:33.895638+0000","last_became_peered":"2024-04-17T10:50:33.895638+0000","last_unstale":"2024-04-17T10:50:33.898263+0000","last_undegraded":"2024-04-17T10:50:33.898263+0000","last_fullsized":"2024-04-17T10:50:33.898263+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:03:31.940294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":412,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.876578+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:50:33.876578+0000","last_peered":"2024-04-17T10:50:33.876578+0000","last_clean":"2024-04-17T10:50:33.876578+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:50:33.876578+0000","last_undegraded":"2024-04-17T10:50:33.876578+0000","last_fullsized":"2024-04-17T10:50:33.876578+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575003,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47324,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324516,"statfs":{"total":96636764160,"available":96588304384,"internally_reserved":0,"allocated":2310144,"data_stored":1339919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575002,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68952,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302888,"statfs":{"total":96636764160,"available":96566157312,"internally_reserved":0,"allocated":1716224,"data_stored":749551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575003,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55080,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316760,"statfs":{"total":96636764160,"available":96580362240,"internally_reserved":0,"allocated":1728512,"data_stored":755826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575002,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73124,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298716,"statfs":{"total":96636764160,"available":96561885184,"internally_reserved":0,"allocated":1716224,"data_stored":749551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575004,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73112,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298728,"statfs":{"total":96636764160,"available":96561897472,"internally_reserved":0,"allocated":1716224,"data_stored":749551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575002,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46748,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325092,"statfs":{"total":96636764160,"available":96588894208,"internally_reserved":0,"allocated":1716224,"data_stored":749551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575002,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82848,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288992,"statfs":{"total":96636764160,"available":96551927808,"internally_reserved":0,"allocated":1716224,"data_stored":749551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575002,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47336,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324504,"statfs":{"total":96636764160,"available":96588292096,"internally_reserved":0,"allocated":2322432,"data_stored":1346194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:50:37.859 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:50:38.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.024+0000 7faaf0622640 1 Processor -- start 2024-04-17T10:50:38.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.024+0000 7faaf0622640 1 -- start start 2024-04-17T10:50:38.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.024+0000 7faaf0622640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faae815ba30 0x7faae8072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:38.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.024+0000 7faaf0622640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faae80cfbe0 0x7faae806c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:38.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.024+0000 7faaf0622640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faae806e320 con 0x7faae815c960 2024-04-17T10:50:38.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.024+0000 7faaf0622640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faae8071ea0 con 0x7faae815ba30 2024-04-17T10:50:38.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.024+0000 7faaf0622640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faae806e040 con 0x7faae80cfbe0 2024-04-17T10:50:38.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.024+0000 7faaee3be640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faae815ba30 0x7faae8072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:38.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.024+0000 7faaee3be640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faae815ba30 0x7faae8072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35296/0 (socket says 172.21.15.83:35296) 2024-04-17T10:50:38.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.024+0000 7faaee3be640 1 -- 172.21.15.83:0/2905931541 learned_addr learned my addr 172.21.15.83:0/2905931541 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:38.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.024+0000 7faaeebbf640 1 --2- 172.21.15.83:0/2905931541 >> v2:172.21.15.167:3300/0 conn(0x7faae80cfbe0 0x7faae806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:38.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.024+0000 7faaee3be640 1 -- 172.21.15.83:0/2905931541 >> v2:172.21.15.167:3300/0 conn(0x7faae80cfbe0 msgr2=0x7faae806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:38.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.024+0000 7faaee3be640 1 --2- 172.21.15.83:0/2905931541 >> v2:172.21.15.167:3300/0 conn(0x7faae80cfbe0 0x7faae806c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:38.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.024+0000 7faaee3be640 1 -- 172.21.15.83:0/2905931541 >> v1:172.21.15.167:6789/0 conn(0x7faae815c960 legacy=0x7faae80cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:38.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.024+0000 7faaee3be640 1 -- 172.21.15.83:0/2905931541 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae8071850 con 0x7faae815ba30 2024-04-17T10:50:38.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.028+0000 7faaee3be640 1 --2- 172.21.15.83:0/2905931541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faae815ba30 0x7faae8072330 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7faae80b2410 tx=0x7faae401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9cc1871c5bcadd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:38.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.028+0000 7faaed3bc640 1 -- 172.21.15.83:0/2905931541 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae40b0070 con 0x7faae815ba30 2024-04-17T10:50:38.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.028+0000 7faaed3bc640 1 -- 172.21.15.83:0/2905931541 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faae40ab600 con 0x7faae815ba30 2024-04-17T10:50:38.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.028+0000 7faaed3bc640 1 -- 172.21.15.83:0/2905931541 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae40ab9d0 con 0x7faae815ba30 2024-04-17T10:50:38.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.028+0000 7faaf0622640 1 -- 172.21.15.83:0/2905931541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faae815ba30 msgr2=0x7faae8072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:38.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.028+0000 7faaf0622640 1 --2- 172.21.15.83:0/2905931541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faae815ba30 0x7faae8072330 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7faae80b2410 tx=0x7faae401c430 comp rx=0 tx=0).stop 2024-04-17T10:50:38.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.028+0000 7faaf0622640 1 -- 172.21.15.83:0/2905931541 shutdown_connections 2024-04-17T10:50:38.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.028+0000 7faaf0622640 1 --2- 172.21.15.83:0/2905931541 >> v2:172.21.15.167:3300/0 conn(0x7faae80cfbe0 0x7faae806c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:38.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.028+0000 7faaf0622640 1 --2- 172.21.15.83:0/2905931541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faae815ba30 0x7faae8072330 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:38.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.028+0000 7faaf0622640 1 -- 172.21.15.83:0/2905931541 >> 172.21.15.83:0/2905931541 conn(0x7faae806abe0 msgr2=0x7faae806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:38.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.028+0000 7faaf0622640 1 -- 172.21.15.83:0/2905931541 shutdown_connections 2024-04-17T10:50:38.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.028+0000 7faaf0622640 1 -- 172.21.15.83:0/2905931541 wait complete. 2024-04-17T10:50:38.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.028+0000 7faaf0622640 1 Processor -- start 2024-04-17T10:50:38.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.028+0000 7faaf0622640 1 -- start start 2024-04-17T10:50:38.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.032+0000 7faaf0622640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faae80cfbe0 0x7faae8158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:38.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.032+0000 7faaf0622640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faae815ba30 0x7faae8140330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:38.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.032+0000 7faaf0622640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faae815c960 0x7faae80ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:38.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.032+0000 7faaf0622640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae806de20 con 0x7faae815ba30 2024-04-17T10:50:38.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.032+0000 7faaf0622640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae806dca0 con 0x7faae80cfbe0 2024-04-17T10:50:38.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.032+0000 7faaf0622640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faae8168520 con 0x7faae815c960 2024-04-17T10:50:38.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.032+0000 7faaedbbd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faae815ba30 0x7faae8140330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:38.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.032+0000 7faaedbbd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faae815ba30 0x7faae8140330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35310/0 (socket says 172.21.15.83:35310) 2024-04-17T10:50:38.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.032+0000 7faaedbbd640 1 -- 172.21.15.83:0/3174167878 learned_addr learned my addr 172.21.15.83:0/3174167878 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:38.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.032+0000 7faaedbbd640 1 -- 172.21.15.83:0/3174167878 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faae815c960 msgr2=0x7faae80ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:38.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.032+0000 7faaedbbd640 1 --2- 172.21.15.83:0/3174167878 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faae815c960 0x7faae80ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:38.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.032+0000 7faaedbbd640 1 -- 172.21.15.83:0/3174167878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faae80cfbe0 msgr2=0x7faae8158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:38.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.032+0000 7faaedbbd640 1 --2- 172.21.15.83:0/3174167878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faae80cfbe0 0x7faae8158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:38.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.032+0000 7faaedbbd640 1 -- 172.21.15.83:0/3174167878 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae80b24b0 con 0x7faae815ba30 2024-04-17T10:50:38.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.032+0000 7faaedbbd640 1 --2- 172.21.15.83:0/3174167878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faae815ba30 0x7faae8140330 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7faad800c450 tx=0x7faad800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:38.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.032+0000 7faadeffd640 1 -- 172.21.15.83:0/3174167878 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad8012830 con 0x7faae815ba30 2024-04-17T10:50:38.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.032+0000 7faadeffd640 1 -- 172.21.15.83:0/3174167878 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad8017070 con 0x7faae815ba30 2024-04-17T10:50:38.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.032+0000 7faaf0622640 1 -- 172.21.15.83:0/3174167878 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faae80b2790 con 0x7faae815ba30 2024-04-17T10:50:38.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.032+0000 7faadeffd640 1 -- 172.21.15.83:0/3174167878 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad8012c30 con 0x7faae815ba30 2024-04-17T10:50:38.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.032+0000 7faaf0622640 1 -- 172.21.15.83:0/3174167878 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae80b2d50 con 0x7faae815ba30 2024-04-17T10:50:38.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.032+0000 7faaf0622640 1 -- 172.21.15.83:0/3174167878 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faae8074f90 con 0x7faae815ba30 2024-04-17T10:50:38.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.032+0000 7faadeffd640 1 -- 172.21.15.83:0/3174167878 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faad801c460 con 0x7faae815ba30 2024-04-17T10:50:38.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.032+0000 7faadeffd640 1 --2- 172.21.15.83:0/3174167878 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faacc066f90 0x7faacc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:38.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.032+0000 7faadeffd640 1 -- 172.21.15.83:0/3174167878 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faad8084d70 con 0x7faae815ba30 2024-04-17T10:50:38.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.032+0000 7faaee3be640 1 --2- 172.21.15.83:0/3174167878 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faacc066f90 0x7faacc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:38.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.032+0000 7faaee3be640 1 --2- 172.21.15.83:0/3174167878 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faacc066f90 0x7faacc069450 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7faae40ad000 tx=0x7faae4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:38.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.036+0000 7faadeffd640 1 -- 172.21.15.83:0/3174167878 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faad8055e10 con 0x7faae815ba30 2024-04-17T10:50:38.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.180+0000 7faaf0622640 1 -- 172.21.15.83:0/3174167878 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faae8157ea0 con 0x7faacc066f90 2024-04-17T10:50:38.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.196+0000 7faadeffd640 1 -- 172.21.15.83:0/3174167878 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247089 (secure 0 0 0) 0x7faae8157ea0 con 0x7faacc066f90 2024-04-17T10:50:38.191 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:50:38.193 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:50:38.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.200+0000 7faaf0622640 1 -- 172.21.15.83:0/3174167878 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faacc066f90 msgr2=0x7faacc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:38.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.200+0000 7faaf0622640 1 --2- 172.21.15.83:0/3174167878 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faacc066f90 0x7faacc069450 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7faae40ad000 tx=0x7faae4008040 comp rx=0 tx=0).stop 2024-04-17T10:50:38.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.200+0000 7faaf0622640 1 -- 172.21.15.83:0/3174167878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faae815ba30 msgr2=0x7faae8140330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:38.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.200+0000 7faaf0622640 1 --2- 172.21.15.83:0/3174167878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faae815ba30 0x7faae8140330 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7faad800c450 tx=0x7faad800c910 comp rx=0 tx=0).stop 2024-04-17T10:50:38.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.200+0000 7faaf0622640 1 -- 172.21.15.83:0/3174167878 shutdown_connections 2024-04-17T10:50:38.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.200+0000 7faaf0622640 1 --2- 172.21.15.83:0/3174167878 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faacc066f90 0x7faacc069450 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:38.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.200+0000 7faaf0622640 1 --2- 172.21.15.83:0/3174167878 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faae815c960 0x7faae80ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:38.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.200+0000 7faaf0622640 1 --2- 172.21.15.83:0/3174167878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faae815ba30 0x7faae8140330 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:38.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.200+0000 7faaf0622640 1 --2- 172.21.15.83:0/3174167878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faae80cfbe0 0x7faae8158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:38.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.200+0000 7faaf0622640 1 -- 172.21.15.83:0/3174167878 >> 172.21.15.83:0/3174167878 conn(0x7faae806abe0 msgr2=0x7faae80ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:38.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.200+0000 7faaf0622640 1 -- 172.21.15.83:0/3174167878 shutdown_connections 2024-04-17T10:50:38.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.200+0000 7faaf0622640 1 -- 172.21.15.83:0/3174167878 wait complete. 2024-04-17T10:50:38.208 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":512,"stamp":"2024-04-17T10:50:36.400745+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":494524,"kb_used_data":14592,"kb_used_omap":12,"kb_used_meta":479795,"kb_avail":754480196,"statfs":{"total":773094113280,"available":772587720704,"internally_reserved":0,"allocated":14942208,"data_stored":7189694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":491310640},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001315"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.876422+0000","last_change":"2024-04-17T10:50:33.876422+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.876422+0000","last_undegraded":"2024-04-17T10:50:33.876422+0000","last_fullsized":"2024-04-17T10:50:33.876422+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.878304+0000","last_change":"2024-04-17T10:50:33.878304+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.878304+0000","last_undegraded":"2024-04-17T10:50:33.878304+0000","last_fullsized":"2024-04-17T10:50:33.878304+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.880142+0000","last_change":"2024-04-17T10:50:33.880142+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.880142+0000","last_undegraded":"2024-04-17T10:50:33.880142+0000","last_fullsized":"2024-04-17T10:50:33.880142+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.871442+0000","last_change":"2024-04-17T10:50:33.871442+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.871442+0000","last_undegraded":"2024-04-17T10:50:33.871442+0000","last_fullsized":"2024-04-17T10:50:33.871442+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.879260+0000","last_change":"2024-04-17T10:50:33.879260+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.879260+0000","last_undegraded":"2024-04-17T10:50:33.879260+0000","last_fullsized":"2024-04-17T10:50:33.879260+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.875526+0000","last_change":"2024-04-17T10:50:33.875526+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.875526+0000","last_undegraded":"2024-04-17T10:50:33.875526+0000","last_fullsized":"2024-04-17T10:50:33.875526+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.884342+0000","last_change":"2024-04-17T10:50:33.884342+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.884342+0000","last_undegraded":"2024-04-17T10:50:33.884342+0000","last_fullsized":"2024-04-17T10:50:33.884342+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.875967+0000","last_change":"2024-04-17T10:50:33.875967+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.875967+0000","last_undegraded":"2024-04-17T10:50:33.875967+0000","last_fullsized":"2024-04-17T10:50:33.875967+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.881528+0000","last_change":"2024-04-17T10:50:33.881528+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.881528+0000","last_undegraded":"2024-04-17T10:50:33.881528+0000","last_fullsized":"2024-04-17T10:50:33.881528+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.874184+0000","last_change":"2024-04-17T10:50:33.874184+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.874184+0000","last_undegraded":"2024-04-17T10:50:33.874184+0000","last_fullsized":"2024-04-17T10:50:33.874184+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.884228+0000","last_change":"2024-04-17T10:50:33.884228+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.884228+0000","last_undegraded":"2024-04-17T10:50:33.884228+0000","last_fullsized":"2024-04-17T10:50:33.884228+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.870313+0000","last_change":"2024-04-17T10:50:33.870313+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.870313+0000","last_undegraded":"2024-04-17T10:50:33.870313+0000","last_fullsized":"2024-04-17T10:50:33.870313+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.878811+0000","last_change":"2024-04-17T10:50:33.878811+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.878811+0000","last_undegraded":"2024-04-17T10:50:33.878811+0000","last_fullsized":"2024-04-17T10:50:33.878811+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.875112+0000","last_change":"2024-04-17T10:50:33.875112+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.875112+0000","last_undegraded":"2024-04-17T10:50:33.875112+0000","last_fullsized":"2024-04-17T10:50:33.875112+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.869759+0000","last_change":"2024-04-17T10:50:33.869759+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.869759+0000","last_undegraded":"2024-04-17T10:50:33.869759+0000","last_fullsized":"2024-04-17T10:50:33.869759+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.883895+0000","last_change":"2024-04-17T10:50:33.883895+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.883895+0000","last_undegraded":"2024-04-17T10:50:33.883895+0000","last_fullsized":"2024-04-17T10:50:33.883895+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.871401+0000","last_change":"2024-04-17T10:50:33.871401+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.871401+0000","last_undegraded":"2024-04-17T10:50:33.871401+0000","last_fullsized":"2024-04-17T10:50:33.871401+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.881276+0000","last_change":"2024-04-17T10:50:33.881276+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.881276+0000","last_undegraded":"2024-04-17T10:50:33.881276+0000","last_fullsized":"2024-04-17T10:50:33.881276+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.883559+0000","last_change":"2024-04-17T10:50:33.883559+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.883559+0000","last_undegraded":"2024-04-17T10:50:33.883559+0000","last_fullsized":"2024-04-17T10:50:33.883559+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.885482+0000","last_change":"2024-04-17T10:50:33.885482+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.885482+0000","last_undegraded":"2024-04-17T10:50:33.885482+0000","last_fullsized":"2024-04-17T10:50:33.885482+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.871386+0000","last_change":"2024-04-17T10:50:33.871386+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.871386+0000","last_undegraded":"2024-04-17T10:50:33.871386+0000","last_fullsized":"2024-04-17T10:50:33.871386+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.874084+0000","last_change":"2024-04-17T10:50:33.874084+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.874084+0000","last_undegraded":"2024-04-17T10:50:33.874084+0000","last_fullsized":"2024-04-17T10:50:33.874084+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.869757+0000","last_change":"2024-04-17T10:50:33.869757+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.869757+0000","last_undegraded":"2024-04-17T10:50:33.869757+0000","last_fullsized":"2024-04-17T10:50:33.869757+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.873283+0000","last_change":"2024-04-17T10:50:33.873283+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.873283+0000","last_undegraded":"2024-04-17T10:50:33.873283+0000","last_fullsized":"2024-04-17T10:50:33.873283+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.883856+0000","last_change":"2024-04-17T10:50:33.883856+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.883856+0000","last_undegraded":"2024-04-17T10:50:33.883856+0000","last_fullsized":"2024-04-17T10:50:33.883856+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.881781+0000","last_change":"2024-04-17T10:50:33.881781+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.881781+0000","last_undegraded":"2024-04-17T10:50:33.881781+0000","last_fullsized":"2024-04-17T10:50:33.881781+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.881470+0000","last_change":"2024-04-17T10:50:33.881470+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.881470+0000","last_undegraded":"2024-04-17T10:50:33.881470+0000","last_fullsized":"2024-04-17T10:50:33.881470+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.885926+0000","last_change":"2024-04-17T10:50:33.885926+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.885926+0000","last_undegraded":"2024-04-17T10:50:33.885926+0000","last_fullsized":"2024-04-17T10:50:33.885926+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.875431+0000","last_change":"2024-04-17T10:50:33.875431+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.875431+0000","last_undegraded":"2024-04-17T10:50:33.875431+0000","last_fullsized":"2024-04-17T10:50:33.875431+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.873345+0000","last_change":"2024-04-17T10:50:33.873345+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.873345+0000","last_undegraded":"2024-04-17T10:50:33.873345+0000","last_fullsized":"2024-04-17T10:50:33.873345+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.878027+0000","last_change":"2024-04-17T10:50:33.878027+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.878027+0000","last_undegraded":"2024-04-17T10:50:33.878027+0000","last_fullsized":"2024-04-17T10:50:33.878027+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.880017+0000","last_change":"2024-04-17T10:50:33.880017+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.880017+0000","last_undegraded":"2024-04-17T10:50:33.880017+0000","last_fullsized":"2024-04-17T10:50:33.880017+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.871860+0000","last_change":"2024-04-17T10:50:33.871860+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.871860+0000","last_undegraded":"2024-04-17T10:50:33.871860+0000","last_fullsized":"2024-04-17T10:50:33.871860+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.896574+0000","last_change":"2024-04-17T10:50:33.891540+0000","last_active":"2024-04-17T10:50:33.896574+0000","last_peered":"2024-04-17T10:50:33.896574+0000","last_clean":"2024-04-17T10:50:33.896574+0000","last_became_active":"2024-04-17T10:50:33.890837+0000","last_became_peered":"2024-04-17T10:50:33.890837+0000","last_unstale":"2024-04-17T10:50:33.896574+0000","last_undegraded":"2024-04-17T10:50:33.896574+0000","last_fullsized":"2024-04-17T10:50:33.896574+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:16:19.474043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.880475+0000","last_change":"2024-04-17T10:50:33.880475+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.880475+0000","last_undegraded":"2024-04-17T10:50:33.880475+0000","last_fullsized":"2024-04-17T10:50:33.880475+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.893863+0000","last_change":"2024-04-17T10:50:33.890842+0000","last_active":"2024-04-17T10:50:33.893863+0000","last_peered":"2024-04-17T10:50:33.893863+0000","last_clean":"2024-04-17T10:50:33.893863+0000","last_became_active":"2024-04-17T10:50:33.890472+0000","last_became_peered":"2024-04-17T10:50:33.890472+0000","last_unstale":"2024-04-17T10:50:33.893863+0000","last_undegraded":"2024-04-17T10:50:33.893863+0000","last_fullsized":"2024-04-17T10:50:33.893863+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:52:36.198523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.876197+0000","last_change":"2024-04-17T10:50:33.876197+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.876197+0000","last_undegraded":"2024-04-17T10:50:33.876197+0000","last_fullsized":"2024-04-17T10:50:33.876197+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.900479+0000","last_change":"2024-04-17T10:50:33.899787+0000","last_active":"2024-04-17T10:50:33.900479+0000","last_peered":"2024-04-17T10:50:33.900479+0000","last_clean":"2024-04-17T10:50:33.900479+0000","last_became_active":"2024-04-17T10:50:33.899273+0000","last_became_peered":"2024-04-17T10:50:33.899273+0000","last_unstale":"2024-04-17T10:50:33.900479+0000","last_undegraded":"2024-04-17T10:50:33.900479+0000","last_fullsized":"2024-04-17T10:50:33.900479+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:08:11.608039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.870653+0000","last_change":"2024-04-17T10:50:33.870653+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.870653+0000","last_undegraded":"2024-04-17T10:50:33.870653+0000","last_fullsized":"2024-04-17T10:50:33.870653+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.887943+0000","last_change":"2024-04-17T10:50:33.884473+0000","last_active":"2024-04-17T10:50:33.887943+0000","last_peered":"2024-04-17T10:50:33.887943+0000","last_clean":"2024-04-17T10:50:33.887943+0000","last_became_active":"2024-04-17T10:50:33.883898+0000","last_became_peered":"2024-04-17T10:50:33.883898+0000","last_unstale":"2024-04-17T10:50:33.887943+0000","last_undegraded":"2024-04-17T10:50:33.887943+0000","last_fullsized":"2024-04-17T10:50:33.887943+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:03:31.940294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.877422+0000","last_change":"2024-04-17T10:50:33.877422+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.877422+0000","last_undegraded":"2024-04-17T10:50:33.877422+0000","last_fullsized":"2024-04-17T10:50:33.877422+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.891469+0000","last_change":"2024-04-17T10:50:33.888560+0000","last_active":"2024-04-17T10:50:33.891469+0000","last_peered":"2024-04-17T10:50:33.891469+0000","last_clean":"2024-04-17T10:50:33.891469+0000","last_became_active":"2024-04-17T10:50:33.888155+0000","last_became_peered":"2024-04-17T10:50:33.888155+0000","last_unstale":"2024-04-17T10:50:33.891469+0000","last_undegraded":"2024-04-17T10:50:33.891469+0000","last_fullsized":"2024-04-17T10:50:33.891469+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:19:11.890360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.872397+0000","last_change":"2024-04-17T10:50:33.872397+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.872397+0000","last_undegraded":"2024-04-17T10:50:33.872397+0000","last_fullsized":"2024-04-17T10:50:33.872397+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.898062+0000","last_change":"2024-04-17T10:50:33.896476+0000","last_active":"2024-04-17T10:50:33.898062+0000","last_peered":"2024-04-17T10:50:33.898062+0000","last_clean":"2024-04-17T10:50:33.898062+0000","last_became_active":"2024-04-17T10:50:33.895863+0000","last_became_peered":"2024-04-17T10:50:33.895863+0000","last_unstale":"2024-04-17T10:50:33.898062+0000","last_undegraded":"2024-04-17T10:50:33.898062+0000","last_fullsized":"2024-04-17T10:50:33.898062+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:49:13.741700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.878606+0000","last_change":"2024-04-17T10:50:33.878606+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.878606+0000","last_undegraded":"2024-04-17T10:50:33.878606+0000","last_fullsized":"2024-04-17T10:50:33.878606+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.898833+0000","last_change":"2024-04-17T10:50:33.895471+0000","last_active":"2024-04-17T10:50:33.898833+0000","last_peered":"2024-04-17T10:50:33.898833+0000","last_clean":"2024-04-17T10:50:33.898833+0000","last_became_active":"2024-04-17T10:50:33.895069+0000","last_became_peered":"2024-04-17T10:50:33.895069+0000","last_unstale":"2024-04-17T10:50:33.898833+0000","last_undegraded":"2024-04-17T10:50:33.898833+0000","last_fullsized":"2024-04-17T10:50:33.898833+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:35:29.165208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.879870+0000","last_change":"2024-04-17T10:50:33.879870+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.879870+0000","last_undegraded":"2024-04-17T10:50:33.879870+0000","last_fullsized":"2024-04-17T10:50:33.879870+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.887894+0000","last_change":"2024-04-17T10:50:33.885177+0000","last_active":"2024-04-17T10:50:33.887894+0000","last_peered":"2024-04-17T10:50:33.887894+0000","last_clean":"2024-04-17T10:50:33.887894+0000","last_became_active":"2024-04-17T10:50:33.884763+0000","last_became_peered":"2024-04-17T10:50:33.884763+0000","last_unstale":"2024-04-17T10:50:33.887894+0000","last_undegraded":"2024-04-17T10:50:33.887894+0000","last_fullsized":"2024-04-17T10:50:33.887894+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:49.010628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.876872+0000","last_change":"2024-04-17T10:50:33.876872+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.876872+0000","last_undegraded":"2024-04-17T10:50:33.876872+0000","last_fullsized":"2024-04-17T10:50:33.876872+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.896605+0000","last_change":"2024-04-17T10:50:33.895658+0000","last_active":"2024-04-17T10:50:33.896605+0000","last_peered":"2024-04-17T10:50:33.896605+0000","last_clean":"2024-04-17T10:50:33.896605+0000","last_became_active":"2024-04-17T10:50:33.894228+0000","last_became_peered":"2024-04-17T10:50:33.894228+0000","last_unstale":"2024-04-17T10:50:33.896605+0000","last_undegraded":"2024-04-17T10:50:33.896605+0000","last_fullsized":"2024-04-17T10:50:33.896605+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:02:11.481439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.872072+0000","last_change":"2024-04-17T10:50:33.872072+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.872072+0000","last_undegraded":"2024-04-17T10:50:33.872072+0000","last_fullsized":"2024-04-17T10:50:33.872072+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.896231+0000","last_change":"2024-04-17T10:50:33.890039+0000","last_active":"2024-04-17T10:50:33.896231+0000","last_peered":"2024-04-17T10:50:33.896231+0000","last_clean":"2024-04-17T10:50:33.896231+0000","last_became_active":"2024-04-17T10:50:33.889635+0000","last_became_peered":"2024-04-17T10:50:33.889635+0000","last_unstale":"2024-04-17T10:50:33.896231+0000","last_undegraded":"2024-04-17T10:50:33.896231+0000","last_fullsized":"2024-04-17T10:50:33.896231+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:02:11.481439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.879906+0000","last_change":"2024-04-17T10:50:33.879906+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.879906+0000","last_undegraded":"2024-04-17T10:50:33.879906+0000","last_fullsized":"2024-04-17T10:50:33.879906+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.904390+0000","last_change":"2024-04-17T10:50:33.898934+0000","last_active":"2024-04-17T10:50:33.904390+0000","last_peered":"2024-04-17T10:50:33.904390+0000","last_clean":"2024-04-17T10:50:33.904390+0000","last_became_active":"2024-04-17T10:50:33.897331+0000","last_became_peered":"2024-04-17T10:50:33.897331+0000","last_unstale":"2024-04-17T10:50:33.904390+0000","last_undegraded":"2024-04-17T10:50:33.904390+0000","last_fullsized":"2024-04-17T10:50:33.904390+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:02:11.481439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.881117+0000","last_change":"2024-04-17T10:50:33.881117+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.881117+0000","last_undegraded":"2024-04-17T10:50:33.881117+0000","last_fullsized":"2024-04-17T10:50:33.881117+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.901174+0000","last_change":"2024-04-17T10:50:33.900576+0000","last_active":"2024-04-17T10:50:33.901174+0000","last_peered":"2024-04-17T10:50:33.901174+0000","last_clean":"2024-04-17T10:50:33.901174+0000","last_became_active":"2024-04-17T10:50:33.899933+0000","last_became_peered":"2024-04-17T10:50:33.899933+0000","last_unstale":"2024-04-17T10:50:33.901174+0000","last_undegraded":"2024-04-17T10:50:33.901174+0000","last_fullsized":"2024-04-17T10:50:33.901174+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:08:11.608039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.877267+0000","last_change":"2024-04-17T10:50:33.877267+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.877267+0000","last_undegraded":"2024-04-17T10:50:33.877267+0000","last_fullsized":"2024-04-17T10:50:33.877267+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.897145+0000","last_change":"2024-04-17T10:50:33.890641+0000","last_active":"2024-04-17T10:50:33.897145+0000","last_peered":"2024-04-17T10:50:33.897145+0000","last_clean":"2024-04-17T10:50:33.897145+0000","last_became_active":"2024-04-17T10:50:33.889972+0000","last_became_peered":"2024-04-17T10:50:33.889972+0000","last_unstale":"2024-04-17T10:50:33.897145+0000","last_undegraded":"2024-04-17T10:50:33.897145+0000","last_fullsized":"2024-04-17T10:50:33.897145+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:29:06.526982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.875628+0000","last_change":"2024-04-17T10:50:33.875628+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.875628+0000","last_undegraded":"2024-04-17T10:50:33.875628+0000","last_fullsized":"2024-04-17T10:50:33.875628+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.899430+0000","last_change":"2024-04-17T10:50:33.897839+0000","last_active":"2024-04-17T10:50:33.899430+0000","last_peered":"2024-04-17T10:50:33.899430+0000","last_clean":"2024-04-17T10:50:33.899430+0000","last_became_active":"2024-04-17T10:50:33.897168+0000","last_became_peered":"2024-04-17T10:50:33.897168+0000","last_unstale":"2024-04-17T10:50:33.899430+0000","last_undegraded":"2024-04-17T10:50:33.899430+0000","last_fullsized":"2024-04-17T10:50:33.899430+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:25:56.552880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.878461+0000","last_change":"2024-04-17T10:50:33.878461+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.878461+0000","last_undegraded":"2024-04-17T10:50:33.878461+0000","last_fullsized":"2024-04-17T10:50:33.878461+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.899185+0000","last_change":"2024-04-17T10:50:33.895104+0000","last_active":"2024-04-17T10:50:33.899185+0000","last_peered":"2024-04-17T10:50:33.899185+0000","last_clean":"2024-04-17T10:50:33.899185+0000","last_became_active":"2024-04-17T10:50:33.894652+0000","last_became_peered":"2024-04-17T10:50:33.894652+0000","last_unstale":"2024-04-17T10:50:33.899185+0000","last_undegraded":"2024-04-17T10:50:33.899185+0000","last_fullsized":"2024-04-17T10:50:33.899185+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:17:40.737496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.871177+0000","last_change":"2024-04-17T10:50:33.871177+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.871177+0000","last_undegraded":"2024-04-17T10:50:33.871177+0000","last_fullsized":"2024-04-17T10:50:33.871177+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.894016+0000","last_change":"2024-04-17T10:50:33.892680+0000","last_active":"2024-04-17T10:50:33.894016+0000","last_peered":"2024-04-17T10:50:33.894016+0000","last_clean":"2024-04-17T10:50:33.894016+0000","last_became_active":"2024-04-17T10:50:33.891996+0000","last_became_peered":"2024-04-17T10:50:33.891996+0000","last_unstale":"2024-04-17T10:50:33.894016+0000","last_undegraded":"2024-04-17T10:50:33.894016+0000","last_fullsized":"2024-04-17T10:50:33.894016+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:36.242473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.880302+0000","last_change":"2024-04-17T10:50:33.880302+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.880302+0000","last_undegraded":"2024-04-17T10:50:33.880302+0000","last_fullsized":"2024-04-17T10:50:33.880302+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.899936+0000","last_change":"2024-04-17T10:50:33.898138+0000","last_active":"2024-04-17T10:50:33.899936+0000","last_peered":"2024-04-17T10:50:33.899936+0000","last_clean":"2024-04-17T10:50:33.899936+0000","last_became_active":"2024-04-17T10:50:33.896506+0000","last_became_peered":"2024-04-17T10:50:33.896506+0000","last_unstale":"2024-04-17T10:50:33.899936+0000","last_undegraded":"2024-04-17T10:50:33.899936+0000","last_fullsized":"2024-04-17T10:50:33.899936+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:35:46.656830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.880316+0000","last_change":"2024-04-17T10:50:33.880316+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.880316+0000","last_undegraded":"2024-04-17T10:50:33.880316+0000","last_fullsized":"2024-04-17T10:50:33.880316+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.890096+0000","last_change":"2024-04-17T10:50:33.887380+0000","last_active":"2024-04-17T10:50:33.890096+0000","last_peered":"2024-04-17T10:50:33.890096+0000","last_clean":"2024-04-17T10:50:33.890096+0000","last_became_active":"2024-04-17T10:50:33.887003+0000","last_became_peered":"2024-04-17T10:50:33.887003+0000","last_unstale":"2024-04-17T10:50:33.890096+0000","last_undegraded":"2024-04-17T10:50:33.890096+0000","last_fullsized":"2024-04-17T10:50:33.890096+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:25:00.410077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.877105+0000","last_change":"2024-04-17T10:50:33.877105+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.877105+0000","last_undegraded":"2024-04-17T10:50:33.877105+0000","last_fullsized":"2024-04-17T10:50:33.877105+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.893790+0000","last_change":"2024-04-17T10:50:33.890628+0000","last_active":"2024-04-17T10:50:33.893790+0000","last_peered":"2024-04-17T10:50:33.893790+0000","last_clean":"2024-04-17T10:50:33.893790+0000","last_became_active":"2024-04-17T10:50:33.890232+0000","last_became_peered":"2024-04-17T10:50:33.890232+0000","last_unstale":"2024-04-17T10:50:33.893790+0000","last_undegraded":"2024-04-17T10:50:33.893790+0000","last_fullsized":"2024-04-17T10:50:33.893790+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:55.719217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.872686+0000","last_change":"2024-04-17T10:50:33.872686+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.872686+0000","last_undegraded":"2024-04-17T10:50:33.872686+0000","last_fullsized":"2024-04-17T10:50:33.872686+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.904423+0000","last_change":"2024-04-17T10:50:33.901531+0000","last_active":"2024-04-17T10:50:33.904423+0000","last_peered":"2024-04-17T10:50:33.904423+0000","last_clean":"2024-04-17T10:50:33.904423+0000","last_became_active":"2024-04-17T10:50:33.901224+0000","last_became_peered":"2024-04-17T10:50:33.901224+0000","last_unstale":"2024-04-17T10:50:33.904423+0000","last_undegraded":"2024-04-17T10:50:33.904423+0000","last_fullsized":"2024-04-17T10:50:33.904423+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:16:19.474043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.891453+0000","last_change":"2024-04-17T10:50:33.891453+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.891453+0000","last_undegraded":"2024-04-17T10:50:33.891453+0000","last_fullsized":"2024-04-17T10:50:33.891453+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.896430+0000","last_change":"2024-04-17T10:50:33.891385+0000","last_active":"2024-04-17T10:50:33.896430+0000","last_peered":"2024-04-17T10:50:33.896430+0000","last_clean":"2024-04-17T10:50:33.896430+0000","last_became_active":"2024-04-17T10:50:33.890896+0000","last_became_peered":"2024-04-17T10:50:33.890896+0000","last_unstale":"2024-04-17T10:50:33.896430+0000","last_undegraded":"2024-04-17T10:50:33.896430+0000","last_fullsized":"2024-04-17T10:50:33.896430+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:35:46.656830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.870668+0000","last_change":"2024-04-17T10:50:33.870668+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.870668+0000","last_undegraded":"2024-04-17T10:50:33.870668+0000","last_fullsized":"2024-04-17T10:50:33.870668+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.896755+0000","last_change":"2024-04-17T10:50:33.893344+0000","last_active":"2024-04-17T10:50:33.896755+0000","last_peered":"2024-04-17T10:50:33.896755+0000","last_clean":"2024-04-17T10:50:33.896755+0000","last_became_active":"2024-04-17T10:50:33.892680+0000","last_became_peered":"2024-04-17T10:50:33.892680+0000","last_unstale":"2024-04-17T10:50:33.896755+0000","last_undegraded":"2024-04-17T10:50:33.896755+0000","last_fullsized":"2024-04-17T10:50:33.896755+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:01:25.099953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.884791+0000","last_change":"2024-04-17T10:50:33.884791+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.884791+0000","last_undegraded":"2024-04-17T10:50:33.884791+0000","last_fullsized":"2024-04-17T10:50:33.884791+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.900021+0000","last_change":"2024-04-17T10:50:33.897362+0000","last_active":"2024-04-17T10:50:33.900021+0000","last_peered":"2024-04-17T10:50:33.900021+0000","last_clean":"2024-04-17T10:50:33.900021+0000","last_became_active":"2024-04-17T10:50:33.896738+0000","last_became_peered":"2024-04-17T10:50:33.896738+0000","last_unstale":"2024-04-17T10:50:33.900021+0000","last_undegraded":"2024-04-17T10:50:33.900021+0000","last_fullsized":"2024-04-17T10:50:33.900021+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:49.010628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.875977+0000","last_change":"2024-04-17T10:50:33.875977+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.875977+0000","last_undegraded":"2024-04-17T10:50:33.875977+0000","last_fullsized":"2024-04-17T10:50:33.875977+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.896611+0000","last_change":"2024-04-17T10:50:33.893658+0000","last_active":"2024-04-17T10:50:33.896611+0000","last_peered":"2024-04-17T10:50:33.896611+0000","last_clean":"2024-04-17T10:50:33.896611+0000","last_became_active":"2024-04-17T10:50:33.893023+0000","last_became_peered":"2024-04-17T10:50:33.893023+0000","last_unstale":"2024-04-17T10:50:33.896611+0000","last_undegraded":"2024-04-17T10:50:33.896611+0000","last_fullsized":"2024-04-17T10:50:33.896611+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:48:29.238257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.880974+0000","last_change":"2024-04-17T10:50:33.880974+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.880974+0000","last_undegraded":"2024-04-17T10:50:33.880974+0000","last_fullsized":"2024-04-17T10:50:33.880974+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.892795+0000","last_change":"2024-04-17T10:50:33.887529+0000","last_active":"2024-04-17T10:50:33.892795+0000","last_peered":"2024-04-17T10:50:33.892795+0000","last_clean":"2024-04-17T10:50:33.892795+0000","last_became_active":"2024-04-17T10:50:33.887152+0000","last_became_peered":"2024-04-17T10:50:33.887152+0000","last_unstale":"2024-04-17T10:50:33.892795+0000","last_undegraded":"2024-04-17T10:50:33.892795+0000","last_fullsized":"2024-04-17T10:50:33.892795+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:04.970052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.878592+0000","last_change":"2024-04-17T10:50:33.878592+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.878592+0000","last_undegraded":"2024-04-17T10:50:33.878592+0000","last_fullsized":"2024-04-17T10:50:33.878592+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.892986+0000","last_change":"2024-04-17T10:50:33.889203+0000","last_active":"2024-04-17T10:50:33.892986+0000","last_peered":"2024-04-17T10:50:33.892986+0000","last_clean":"2024-04-17T10:50:33.892986+0000","last_became_active":"2024-04-17T10:50:33.888312+0000","last_became_peered":"2024-04-17T10:50:33.888312+0000","last_unstale":"2024-04-17T10:50:33.892986+0000","last_undegraded":"2024-04-17T10:50:33.892986+0000","last_fullsized":"2024-04-17T10:50:33.892986+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:23:11.975227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.876607+0000","last_change":"2024-04-17T10:50:33.876607+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.876607+0000","last_undegraded":"2024-04-17T10:50:33.876607+0000","last_fullsized":"2024-04-17T10:50:33.876607+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.892508+0000","last_change":"2024-04-17T10:50:33.882648+0000","last_active":"2024-04-17T10:50:33.892508+0000","last_peered":"2024-04-17T10:50:33.892508+0000","last_clean":"2024-04-17T10:50:33.892508+0000","last_became_active":"2024-04-17T10:50:33.882297+0000","last_became_peered":"2024-04-17T10:50:33.882297+0000","last_unstale":"2024-04-17T10:50:33.892508+0000","last_undegraded":"2024-04-17T10:50:33.892508+0000","last_fullsized":"2024-04-17T10:50:33.892508+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:29:30.703224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.879901+0000","last_change":"2024-04-17T10:50:33.879901+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.879901+0000","last_undegraded":"2024-04-17T10:50:33.879901+0000","last_fullsized":"2024-04-17T10:50:33.879901+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.904388+0000","last_change":"2024-04-17T10:50:33.900315+0000","last_active":"2024-04-17T10:50:33.904388+0000","last_peered":"2024-04-17T10:50:33.904388+0000","last_clean":"2024-04-17T10:50:33.904388+0000","last_became_active":"2024-04-17T10:50:33.899987+0000","last_became_peered":"2024-04-17T10:50:33.899987+0000","last_unstale":"2024-04-17T10:50:33.904388+0000","last_undegraded":"2024-04-17T10:50:33.904388+0000","last_fullsized":"2024-04-17T10:50:33.904388+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:35:46.656830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.878334+0000","last_change":"2024-04-17T10:50:33.878334+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.878334+0000","last_undegraded":"2024-04-17T10:50:33.878334+0000","last_fullsized":"2024-04-17T10:50:33.878334+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.896788+0000","last_change":"2024-04-17T10:50:33.894342+0000","last_active":"2024-04-17T10:50:33.896788+0000","last_peered":"2024-04-17T10:50:33.896788+0000","last_clean":"2024-04-17T10:50:33.896788+0000","last_became_active":"2024-04-17T10:50:33.893720+0000","last_became_peered":"2024-04-17T10:50:33.893720+0000","last_unstale":"2024-04-17T10:50:33.896788+0000","last_undegraded":"2024-04-17T10:50:33.896788+0000","last_fullsized":"2024-04-17T10:50:33.896788+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:21:59.523164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.869757+0000","last_change":"2024-04-17T10:50:33.869757+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.869757+0000","last_undegraded":"2024-04-17T10:50:33.869757+0000","last_fullsized":"2024-04-17T10:50:33.869757+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.898205+0000","last_change":"2024-04-17T10:50:33.895914+0000","last_active":"2024-04-17T10:50:33.898205+0000","last_peered":"2024-04-17T10:50:33.898205+0000","last_clean":"2024-04-17T10:50:33.898205+0000","last_became_active":"2024-04-17T10:50:33.893274+0000","last_became_peered":"2024-04-17T10:50:33.893274+0000","last_unstale":"2024-04-17T10:50:33.898205+0000","last_undegraded":"2024-04-17T10:50:33.898205+0000","last_fullsized":"2024-04-17T10:50:33.898205+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:44:02.588052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.877441+0000","last_change":"2024-04-17T10:50:33.877441+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.877441+0000","last_undegraded":"2024-04-17T10:50:33.877441+0000","last_fullsized":"2024-04-17T10:50:33.877441+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.896924+0000","last_change":"2024-04-17T10:50:33.893329+0000","last_active":"2024-04-17T10:50:33.896924+0000","last_peered":"2024-04-17T10:50:33.896924+0000","last_clean":"2024-04-17T10:50:33.896924+0000","last_became_active":"2024-04-17T10:50:33.892918+0000","last_became_peered":"2024-04-17T10:50:33.892918+0000","last_unstale":"2024-04-17T10:50:33.896924+0000","last_undegraded":"2024-04-17T10:50:33.896924+0000","last_fullsized":"2024-04-17T10:50:33.896924+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:43:18.553237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.885117+0000","last_change":"2024-04-17T10:50:33.885117+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.885117+0000","last_undegraded":"2024-04-17T10:50:33.885117+0000","last_fullsized":"2024-04-17T10:50:33.885117+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.898263+0000","last_change":"2024-04-17T10:50:33.896124+0000","last_active":"2024-04-17T10:50:33.898263+0000","last_peered":"2024-04-17T10:50:33.898263+0000","last_clean":"2024-04-17T10:50:33.898263+0000","last_became_active":"2024-04-17T10:50:33.895638+0000","last_became_peered":"2024-04-17T10:50:33.895638+0000","last_unstale":"2024-04-17T10:50:33.898263+0000","last_undegraded":"2024-04-17T10:50:33.898263+0000","last_fullsized":"2024-04-17T10:50:33.898263+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:03:31.940294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":412,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.876578+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:50:33.876578+0000","last_peered":"2024-04-17T10:50:33.876578+0000","last_clean":"2024-04-17T10:50:33.876578+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:50:33.876578+0000","last_undegraded":"2024-04-17T10:50:33.876578+0000","last_fullsized":"2024-04-17T10:50:33.876578+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575003,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47324,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324516,"statfs":{"total":96636764160,"available":96588304384,"internally_reserved":0,"allocated":2310144,"data_stored":1339919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575002,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68952,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302888,"statfs":{"total":96636764160,"available":96566157312,"internally_reserved":0,"allocated":1716224,"data_stored":749551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575003,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55080,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316760,"statfs":{"total":96636764160,"available":96580362240,"internally_reserved":0,"allocated":1728512,"data_stored":755826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575002,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73124,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298716,"statfs":{"total":96636764160,"available":96561885184,"internally_reserved":0,"allocated":1716224,"data_stored":749551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575004,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73112,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298728,"statfs":{"total":96636764160,"available":96561897472,"internally_reserved":0,"allocated":1716224,"data_stored":749551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575002,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46748,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325092,"statfs":{"total":96636764160,"available":96588894208,"internally_reserved":0,"allocated":1716224,"data_stored":749551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575002,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82848,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288992,"statfs":{"total":96636764160,"available":96551927808,"internally_reserved":0,"allocated":1716224,"data_stored":749551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575002,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47336,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324504,"statfs":{"total":96636764160,"available":96588292096,"internally_reserved":0,"allocated":2322432,"data_stored":1346194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:50:38.212 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:50:38.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f3721f0e640 1 Processor -- start 2024-04-17T10:50:38.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f3721f0e640 1 -- start start 2024-04-17T10:50:38.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f3721f0e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f371c0b0a70 0x7f371c0d2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:38.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f3721f0e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f371c06e980 con 0x7f371c0ce750 2024-04-17T10:50:38.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f3721f0e640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f371c162810 con 0x7f371c162150 2024-04-17T10:50:38.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f3721f0e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f371c1625e0 con 0x7f371c0b0a70 2024-04-17T10:50:38.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f371b7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f371c0ce750 0x7f371c161a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57152/0 (socket says 172.21.15.83:57152) 2024-04-17T10:50:38.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f371b7fe640 1 -- 172.21.15.83:0/1204327881 learned_addr learned my addr 172.21.15.83:0/1204327881 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:38.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f371affd640 1 --2- 172.21.15.83:0/1204327881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f371c0b0a70 0x7f371c0d2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:38.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f371a7fc640 1 -- 172.21.15.83:0/1204327881 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2427910280 0 0) 0x7f371c06e980 con 0x7f371c0ce750 2024-04-17T10:50:38.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f371a7fc640 1 -- 172.21.15.83:0/1204327881 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3704003530 con 0x7f371c0ce750 2024-04-17T10:50:38.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f371a7fc640 1 -- 172.21.15.83:0/1204327881 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3567123157 0 0) 0x7f3704003530 con 0x7f371c0ce750 2024-04-17T10:50:38.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f371a7fc640 1 -- 172.21.15.83:0/1204327881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f371c0b0a70 msgr2=0x7f371c0d2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:38.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f371a7fc640 1 --2- 172.21.15.83:0/1204327881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f371c0b0a70 0x7f371c0d2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:38.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f371a7fc640 1 -- 172.21.15.83:0/1204327881 >> v1:172.21.15.167:6789/0 conn(0x7f371c162150 legacy=0x7f371c06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:38.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f371a7fc640 1 -- 172.21.15.83:0/1204327881 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f371c06ec20 con 0x7f371c0ce750 2024-04-17T10:50:38.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f371a7fc640 1 -- 172.21.15.83:0/1204327881 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f370c001680 con 0x7f371c0ce750 2024-04-17T10:50:38.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f371a7fc640 1 -- 172.21.15.83:0/1204327881 >> v1:172.21.15.83:6789/0 conn(0x7f371c0ce750 legacy=0x7f371c161a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:38.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f371a7fc640 1 --2- 172.21.15.83:0/1204327881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37040039f0 0x7f3704003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:38.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f371a7fc640 1 --2- 172.21.15.83:0/1204327881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3704004810 0x7f3704004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:38.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f371a7fc640 1 --2- 172.21.15.83:0/1204327881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f371c0b0a70 0x7f37040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:38.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f371a7fc640 1 -- 172.21.15.83:0/1204327881 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370400ba80 con 0x7f37040039f0 2024-04-17T10:50:38.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f371a7fc640 1 -- 172.21.15.83:0/1204327881 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370400bc00 con 0x7f3704004810 2024-04-17T10:50:38.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f371a7fc640 1 -- 172.21.15.83:0/1204327881 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f370400be00 con 0x7f371c0b0a70 2024-04-17T10:50:38.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f371b7fe640 1 --2- 172.21.15.83:0/1204327881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37040039f0 0x7f3704003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:38.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f371affd640 1 --2- 172.21.15.83:0/1204327881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3704004810 0x7f3704004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:38.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f371bfff640 1 --2- 172.21.15.83:0/1204327881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f371c0b0a70 0x7f37040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:38.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f371b7fe640 1 -- 172.21.15.83:0/1204327881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f371c0b0a70 msgr2=0x7f37040051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:38.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f371b7fe640 1 --2- 172.21.15.83:0/1204327881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f371c0b0a70 0x7f37040051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:38.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f371b7fe640 1 -- 172.21.15.83:0/1204327881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3704004810 msgr2=0x7f3704004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:38.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f371b7fe640 1 --2- 172.21.15.83:0/1204327881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3704004810 0x7f3704004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:38.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f371b7fe640 1 -- 172.21.15.83:0/1204327881 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3704003530 con 0x7f37040039f0 2024-04-17T10:50:38.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f371affd640 1 --2- 172.21.15.83:0/1204327881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3704004810 0x7f3704004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:38.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.380+0000 7f371bfff640 1 --2- 172.21.15.83:0/1204327881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f371c0b0a70 0x7f37040051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:38.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.384+0000 7f371b7fe640 1 --2- 172.21.15.83:0/1204327881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37040039f0 0x7f3704003e10 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f370c00c210 tx=0x7f370c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c194fc43b22acd11 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:38.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.384+0000 7f371a7fc640 1 -- 172.21.15.83:0/1204327881 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370c007d20 con 0x7f37040039f0 2024-04-17T10:50:38.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.384+0000 7f371a7fc640 1 -- 172.21.15.83:0/1204327881 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f370c0a9070 con 0x7f37040039f0 2024-04-17T10:50:38.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.384+0000 7f371a7fc640 1 -- 172.21.15.83:0/1204327881 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370c005d60 con 0x7f37040039f0 2024-04-17T10:50:38.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.384+0000 7f3721f0e640 1 -- 172.21.15.83:0/1204327881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37040039f0 msgr2=0x7f3704003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:38.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.384+0000 7f3721f0e640 1 --2- 172.21.15.83:0/1204327881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37040039f0 0x7f3704003e10 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f370c00c210 tx=0x7f370c01cc40 comp rx=0 tx=0).stop 2024-04-17T10:50:38.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.384+0000 7f371bfff640 1 -- 172.21.15.83:0/1204327881 reap_dead start 2024-04-17T10:50:38.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.384+0000 7f3721f0e640 1 -- 172.21.15.83:0/1204327881 shutdown_connections 2024-04-17T10:50:38.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.384+0000 7f3721f0e640 1 -- 172.21.15.83:0/1204327881 >> 172.21.15.83:0/1204327881 conn(0x7f371c0d1770 msgr2=0x7f371c0d1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:38.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.384+0000 7f3721f0e640 1 -- 172.21.15.83:0/1204327881 shutdown_connections 2024-04-17T10:50:38.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.384+0000 7f3721f0e640 1 -- 172.21.15.83:0/1204327881 wait complete. 2024-04-17T10:50:38.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.384+0000 7f3721f0e640 1 Processor -- start 2024-04-17T10:50:38.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.384+0000 7f3721f0e640 1 -- start start 2024-04-17T10:50:38.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.384+0000 7f3721f0e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37040039f0 0x7f371c0b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:38.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.384+0000 7f3721f0e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f371c0b6a60 0x7f371c0b4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:38.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.384+0000 7f3721f0e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f371c0b6ee0 0x7f371c0b4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:38.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.384+0000 7f3721f0e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371c06f290 con 0x7f37040039f0 2024-04-17T10:50:38.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.384+0000 7f3721f0e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371c169f90 con 0x7f371c0b6ee0 2024-04-17T10:50:38.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.384+0000 7f3721f0e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f371c16a110 con 0x7f371c0b6a60 2024-04-17T10:50:38.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.384+0000 7f371b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37040039f0 0x7f371c0b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:38.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.384+0000 7f371b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37040039f0 0x7f371c0b10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35322/0 (socket says 172.21.15.83:35322) 2024-04-17T10:50:38.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.384+0000 7f371b7fe640 1 -- 172.21.15.83:0/1660949815 learned_addr learned my addr 172.21.15.83:0/1660949815 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:38.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.384+0000 7f371affd640 1 --2- 172.21.15.83:0/1660949815 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f371c0b6a60 0x7f371c0b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:38.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.384+0000 7f371bfff640 1 --2- 172.21.15.83:0/1660949815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f371c0b6ee0 0x7f371c0b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:38.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.384+0000 7f371b7fe640 1 -- 172.21.15.83:0/1660949815 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f371c0b6a60 msgr2=0x7f371c0b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:38.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.384+0000 7f371b7fe640 1 --2- 172.21.15.83:0/1660949815 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f371c0b6a60 0x7f371c0b4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:38.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.384+0000 7f371b7fe640 1 -- 172.21.15.83:0/1660949815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f371c0b6ee0 msgr2=0x7f371c0b4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:38.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.384+0000 7f371b7fe640 1 --2- 172.21.15.83:0/1660949815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f371c0b6ee0 0x7f371c0b4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:38.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.388+0000 7f371b7fe640 1 -- 172.21.15.83:0/1660949815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f371c0b50a0 con 0x7f37040039f0 2024-04-17T10:50:38.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.388+0000 7f371b7fe640 1 --2- 172.21.15.83:0/1660949815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37040039f0 0x7f371c0b10d0 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f370c01ca70 tx=0x7f3704005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:38.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.388+0000 7f3703fff640 1 -- 172.21.15.83:0/1660949815 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370c005d40 con 0x7f37040039f0 2024-04-17T10:50:38.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.388+0000 7f3703fff640 1 -- 172.21.15.83:0/1660949815 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f370c0047f0 con 0x7f37040039f0 2024-04-17T10:50:38.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.388+0000 7f3703fff640 1 -- 172.21.15.83:0/1660949815 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370c004bc0 con 0x7f37040039f0 2024-04-17T10:50:38.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.388+0000 7f3721f0e640 1 -- 172.21.15.83:0/1660949815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f371c166fb0 con 0x7f37040039f0 2024-04-17T10:50:38.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.388+0000 7f3721f0e640 1 -- 172.21.15.83:0/1660949815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f371c167570 con 0x7f37040039f0 2024-04-17T10:50:38.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.388+0000 7f3721f0e640 1 -- 172.21.15.83:0/1660949815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f371c074f90 con 0x7f37040039f0 2024-04-17T10:50:38.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.388+0000 7f3703fff640 1 -- 172.21.15.83:0/1660949815 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f370c007040 con 0x7f37040039f0 2024-04-17T10:50:38.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.388+0000 7f3703fff640 1 --2- 172.21.15.83:0/1660949815 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3708066f90 0x7f3708069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:38.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.388+0000 7f3703fff640 1 -- 172.21.15.83:0/1660949815 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f370c11e500 con 0x7f37040039f0 2024-04-17T10:50:38.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.388+0000 7f371affd640 1 --2- 172.21.15.83:0/1660949815 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3708066f90 0x7f3708069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:38.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.388+0000 7f371affd640 1 --2- 172.21.15.83:0/1660949815 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3708066f90 0x7f3708069450 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f371c0c5c00 tx=0x7f3710008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:38.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.392+0000 7f3703fff640 1 -- 172.21.15.83:0/1660949815 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f370c016950 con 0x7f37040039f0 2024-04-17T10:50:38.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.544+0000 7f3721f0e640 1 -- 172.21.15.83:0/1660949815 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f371c0b76f0 con 0x7f3708066f90 2024-04-17T10:50:38.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.564+0000 7f3703fff640 1 -- 172.21.15.83:0/1660949815 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247089 (secure 0 0 0) 0x7f371c0b76f0 con 0x7f3708066f90 2024-04-17T10:50:38.558 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:50:38.559 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:50:38.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.568+0000 7f3721f0e640 1 -- 172.21.15.83:0/1660949815 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3708066f90 msgr2=0x7f3708069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:38.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.568+0000 7f3721f0e640 1 --2- 172.21.15.83:0/1660949815 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3708066f90 0x7f3708069450 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f371c0c5c00 tx=0x7f3710008040 comp rx=0 tx=0).stop 2024-04-17T10:50:38.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.568+0000 7f3721f0e640 1 -- 172.21.15.83:0/1660949815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37040039f0 msgr2=0x7f371c0b10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:38.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.568+0000 7f3721f0e640 1 --2- 172.21.15.83:0/1660949815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37040039f0 0x7f371c0b10d0 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f370c01ca70 tx=0x7f3704005840 comp rx=0 tx=0).stop 2024-04-17T10:50:38.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.568+0000 7f3721f0e640 1 -- 172.21.15.83:0/1660949815 shutdown_connections 2024-04-17T10:50:38.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.568+0000 7f3721f0e640 1 --2- 172.21.15.83:0/1660949815 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3708066f90 0x7f3708069450 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:38.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.568+0000 7f3721f0e640 1 --2- 172.21.15.83:0/1660949815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f371c0b6ee0 0x7f371c0b4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:38.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.568+0000 7f3721f0e640 1 --2- 172.21.15.83:0/1660949815 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f371c0b6a60 0x7f371c0b4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:38.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.568+0000 7f3721f0e640 1 --2- 172.21.15.83:0/1660949815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37040039f0 0x7f371c0b10d0 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:38.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.568+0000 7f3721f0e640 1 -- 172.21.15.83:0/1660949815 >> 172.21.15.83:0/1660949815 conn(0x7f371c0d1770 msgr2=0x7f371c06beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:38.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.568+0000 7f3721f0e640 1 -- 172.21.15.83:0/1660949815 shutdown_connections 2024-04-17T10:50:38.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.568+0000 7f3721f0e640 1 -- 172.21.15.83:0/1660949815 wait complete. 2024-04-17T10:50:38.575 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":513,"stamp":"2024-04-17T10:50:38.401239+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":494524,"kb_used_data":14592,"kb_used_omap":12,"kb_used_meta":479795,"kb_avail":754480196,"statfs":{"total":773094113280,"available":772587720704,"internally_reserved":0,"allocated":14942208,"data_stored":7189694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":491310640},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.545239"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.876422+0000","last_change":"2024-04-17T10:50:33.876422+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.876422+0000","last_undegraded":"2024-04-17T10:50:33.876422+0000","last_fullsized":"2024-04-17T10:50:33.876422+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.878304+0000","last_change":"2024-04-17T10:50:33.878304+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.878304+0000","last_undegraded":"2024-04-17T10:50:33.878304+0000","last_fullsized":"2024-04-17T10:50:33.878304+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.880142+0000","last_change":"2024-04-17T10:50:33.880142+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.880142+0000","last_undegraded":"2024-04-17T10:50:33.880142+0000","last_fullsized":"2024-04-17T10:50:33.880142+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.871442+0000","last_change":"2024-04-17T10:50:33.871442+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.871442+0000","last_undegraded":"2024-04-17T10:50:33.871442+0000","last_fullsized":"2024-04-17T10:50:33.871442+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.879260+0000","last_change":"2024-04-17T10:50:33.879260+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.879260+0000","last_undegraded":"2024-04-17T10:50:33.879260+0000","last_fullsized":"2024-04-17T10:50:33.879260+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.875526+0000","last_change":"2024-04-17T10:50:33.875526+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.875526+0000","last_undegraded":"2024-04-17T10:50:33.875526+0000","last_fullsized":"2024-04-17T10:50:33.875526+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.884342+0000","last_change":"2024-04-17T10:50:33.884342+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.884342+0000","last_undegraded":"2024-04-17T10:50:33.884342+0000","last_fullsized":"2024-04-17T10:50:33.884342+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.875967+0000","last_change":"2024-04-17T10:50:33.875967+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.875967+0000","last_undegraded":"2024-04-17T10:50:33.875967+0000","last_fullsized":"2024-04-17T10:50:33.875967+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.881528+0000","last_change":"2024-04-17T10:50:33.881528+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.881528+0000","last_undegraded":"2024-04-17T10:50:33.881528+0000","last_fullsized":"2024-04-17T10:50:33.881528+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.874184+0000","last_change":"2024-04-17T10:50:33.874184+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.874184+0000","last_undegraded":"2024-04-17T10:50:33.874184+0000","last_fullsized":"2024-04-17T10:50:33.874184+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.884228+0000","last_change":"2024-04-17T10:50:33.884228+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.884228+0000","last_undegraded":"2024-04-17T10:50:33.884228+0000","last_fullsized":"2024-04-17T10:50:33.884228+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.870313+0000","last_change":"2024-04-17T10:50:33.870313+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.870313+0000","last_undegraded":"2024-04-17T10:50:33.870313+0000","last_fullsized":"2024-04-17T10:50:33.870313+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.878811+0000","last_change":"2024-04-17T10:50:33.878811+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.878811+0000","last_undegraded":"2024-04-17T10:50:33.878811+0000","last_fullsized":"2024-04-17T10:50:33.878811+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.875112+0000","last_change":"2024-04-17T10:50:33.875112+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.875112+0000","last_undegraded":"2024-04-17T10:50:33.875112+0000","last_fullsized":"2024-04-17T10:50:33.875112+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.869759+0000","last_change":"2024-04-17T10:50:33.869759+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.869759+0000","last_undegraded":"2024-04-17T10:50:33.869759+0000","last_fullsized":"2024-04-17T10:50:33.869759+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.883895+0000","last_change":"2024-04-17T10:50:33.883895+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.883895+0000","last_undegraded":"2024-04-17T10:50:33.883895+0000","last_fullsized":"2024-04-17T10:50:33.883895+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.871401+0000","last_change":"2024-04-17T10:50:33.871401+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.871401+0000","last_undegraded":"2024-04-17T10:50:33.871401+0000","last_fullsized":"2024-04-17T10:50:33.871401+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.881276+0000","last_change":"2024-04-17T10:50:33.881276+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.881276+0000","last_undegraded":"2024-04-17T10:50:33.881276+0000","last_fullsized":"2024-04-17T10:50:33.881276+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.883559+0000","last_change":"2024-04-17T10:50:33.883559+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.883559+0000","last_undegraded":"2024-04-17T10:50:33.883559+0000","last_fullsized":"2024-04-17T10:50:33.883559+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.885482+0000","last_change":"2024-04-17T10:50:33.885482+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.885482+0000","last_undegraded":"2024-04-17T10:50:33.885482+0000","last_fullsized":"2024-04-17T10:50:33.885482+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.871386+0000","last_change":"2024-04-17T10:50:33.871386+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.871386+0000","last_undegraded":"2024-04-17T10:50:33.871386+0000","last_fullsized":"2024-04-17T10:50:33.871386+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.874084+0000","last_change":"2024-04-17T10:50:33.874084+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.874084+0000","last_undegraded":"2024-04-17T10:50:33.874084+0000","last_fullsized":"2024-04-17T10:50:33.874084+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.869757+0000","last_change":"2024-04-17T10:50:33.869757+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.869757+0000","last_undegraded":"2024-04-17T10:50:33.869757+0000","last_fullsized":"2024-04-17T10:50:33.869757+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.873283+0000","last_change":"2024-04-17T10:50:33.873283+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.873283+0000","last_undegraded":"2024-04-17T10:50:33.873283+0000","last_fullsized":"2024-04-17T10:50:33.873283+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.883856+0000","last_change":"2024-04-17T10:50:33.883856+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.883856+0000","last_undegraded":"2024-04-17T10:50:33.883856+0000","last_fullsized":"2024-04-17T10:50:33.883856+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.881781+0000","last_change":"2024-04-17T10:50:33.881781+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.881781+0000","last_undegraded":"2024-04-17T10:50:33.881781+0000","last_fullsized":"2024-04-17T10:50:33.881781+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.881470+0000","last_change":"2024-04-17T10:50:33.881470+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.881470+0000","last_undegraded":"2024-04-17T10:50:33.881470+0000","last_fullsized":"2024-04-17T10:50:33.881470+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.885926+0000","last_change":"2024-04-17T10:50:33.885926+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.885926+0000","last_undegraded":"2024-04-17T10:50:33.885926+0000","last_fullsized":"2024-04-17T10:50:33.885926+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.875431+0000","last_change":"2024-04-17T10:50:33.875431+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.875431+0000","last_undegraded":"2024-04-17T10:50:33.875431+0000","last_fullsized":"2024-04-17T10:50:33.875431+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.873345+0000","last_change":"2024-04-17T10:50:33.873345+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.873345+0000","last_undegraded":"2024-04-17T10:50:33.873345+0000","last_fullsized":"2024-04-17T10:50:33.873345+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.878027+0000","last_change":"2024-04-17T10:50:33.878027+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.878027+0000","last_undegraded":"2024-04-17T10:50:33.878027+0000","last_fullsized":"2024-04-17T10:50:33.878027+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.880017+0000","last_change":"2024-04-17T10:50:33.880017+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.880017+0000","last_undegraded":"2024-04-17T10:50:33.880017+0000","last_fullsized":"2024-04-17T10:50:33.880017+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.871860+0000","last_change":"2024-04-17T10:50:33.871860+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.871860+0000","last_undegraded":"2024-04-17T10:50:33.871860+0000","last_fullsized":"2024-04-17T10:50:33.871860+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.896574+0000","last_change":"2024-04-17T10:50:33.891540+0000","last_active":"2024-04-17T10:50:33.896574+0000","last_peered":"2024-04-17T10:50:33.896574+0000","last_clean":"2024-04-17T10:50:33.896574+0000","last_became_active":"2024-04-17T10:50:33.890837+0000","last_became_peered":"2024-04-17T10:50:33.890837+0000","last_unstale":"2024-04-17T10:50:33.896574+0000","last_undegraded":"2024-04-17T10:50:33.896574+0000","last_fullsized":"2024-04-17T10:50:33.896574+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:16:19.474043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.880475+0000","last_change":"2024-04-17T10:50:33.880475+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.880475+0000","last_undegraded":"2024-04-17T10:50:33.880475+0000","last_fullsized":"2024-04-17T10:50:33.880475+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.893863+0000","last_change":"2024-04-17T10:50:33.890842+0000","last_active":"2024-04-17T10:50:33.893863+0000","last_peered":"2024-04-17T10:50:33.893863+0000","last_clean":"2024-04-17T10:50:33.893863+0000","last_became_active":"2024-04-17T10:50:33.890472+0000","last_became_peered":"2024-04-17T10:50:33.890472+0000","last_unstale":"2024-04-17T10:50:33.893863+0000","last_undegraded":"2024-04-17T10:50:33.893863+0000","last_fullsized":"2024-04-17T10:50:33.893863+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:52:36.198523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.876197+0000","last_change":"2024-04-17T10:50:33.876197+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.876197+0000","last_undegraded":"2024-04-17T10:50:33.876197+0000","last_fullsized":"2024-04-17T10:50:33.876197+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.900479+0000","last_change":"2024-04-17T10:50:33.899787+0000","last_active":"2024-04-17T10:50:33.900479+0000","last_peered":"2024-04-17T10:50:33.900479+0000","last_clean":"2024-04-17T10:50:33.900479+0000","last_became_active":"2024-04-17T10:50:33.899273+0000","last_became_peered":"2024-04-17T10:50:33.899273+0000","last_unstale":"2024-04-17T10:50:33.900479+0000","last_undegraded":"2024-04-17T10:50:33.900479+0000","last_fullsized":"2024-04-17T10:50:33.900479+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:08:11.608039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.870653+0000","last_change":"2024-04-17T10:50:33.870653+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.870653+0000","last_undegraded":"2024-04-17T10:50:33.870653+0000","last_fullsized":"2024-04-17T10:50:33.870653+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.887943+0000","last_change":"2024-04-17T10:50:33.884473+0000","last_active":"2024-04-17T10:50:33.887943+0000","last_peered":"2024-04-17T10:50:33.887943+0000","last_clean":"2024-04-17T10:50:33.887943+0000","last_became_active":"2024-04-17T10:50:33.883898+0000","last_became_peered":"2024-04-17T10:50:33.883898+0000","last_unstale":"2024-04-17T10:50:33.887943+0000","last_undegraded":"2024-04-17T10:50:33.887943+0000","last_fullsized":"2024-04-17T10:50:33.887943+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:03:31.940294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.877422+0000","last_change":"2024-04-17T10:50:33.877422+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.877422+0000","last_undegraded":"2024-04-17T10:50:33.877422+0000","last_fullsized":"2024-04-17T10:50:33.877422+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.891469+0000","last_change":"2024-04-17T10:50:33.888560+0000","last_active":"2024-04-17T10:50:33.891469+0000","last_peered":"2024-04-17T10:50:33.891469+0000","last_clean":"2024-04-17T10:50:33.891469+0000","last_became_active":"2024-04-17T10:50:33.888155+0000","last_became_peered":"2024-04-17T10:50:33.888155+0000","last_unstale":"2024-04-17T10:50:33.891469+0000","last_undegraded":"2024-04-17T10:50:33.891469+0000","last_fullsized":"2024-04-17T10:50:33.891469+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:19:11.890360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.872397+0000","last_change":"2024-04-17T10:50:33.872397+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.872397+0000","last_undegraded":"2024-04-17T10:50:33.872397+0000","last_fullsized":"2024-04-17T10:50:33.872397+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.898062+0000","last_change":"2024-04-17T10:50:33.896476+0000","last_active":"2024-04-17T10:50:33.898062+0000","last_peered":"2024-04-17T10:50:33.898062+0000","last_clean":"2024-04-17T10:50:33.898062+0000","last_became_active":"2024-04-17T10:50:33.895863+0000","last_became_peered":"2024-04-17T10:50:33.895863+0000","last_unstale":"2024-04-17T10:50:33.898062+0000","last_undegraded":"2024-04-17T10:50:33.898062+0000","last_fullsized":"2024-04-17T10:50:33.898062+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:49:13.741700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.878606+0000","last_change":"2024-04-17T10:50:33.878606+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.878606+0000","last_undegraded":"2024-04-17T10:50:33.878606+0000","last_fullsized":"2024-04-17T10:50:33.878606+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.898833+0000","last_change":"2024-04-17T10:50:33.895471+0000","last_active":"2024-04-17T10:50:33.898833+0000","last_peered":"2024-04-17T10:50:33.898833+0000","last_clean":"2024-04-17T10:50:33.898833+0000","last_became_active":"2024-04-17T10:50:33.895069+0000","last_became_peered":"2024-04-17T10:50:33.895069+0000","last_unstale":"2024-04-17T10:50:33.898833+0000","last_undegraded":"2024-04-17T10:50:33.898833+0000","last_fullsized":"2024-04-17T10:50:33.898833+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:35:29.165208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.879870+0000","last_change":"2024-04-17T10:50:33.879870+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.879870+0000","last_undegraded":"2024-04-17T10:50:33.879870+0000","last_fullsized":"2024-04-17T10:50:33.879870+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.887894+0000","last_change":"2024-04-17T10:50:33.885177+0000","last_active":"2024-04-17T10:50:33.887894+0000","last_peered":"2024-04-17T10:50:33.887894+0000","last_clean":"2024-04-17T10:50:33.887894+0000","last_became_active":"2024-04-17T10:50:33.884763+0000","last_became_peered":"2024-04-17T10:50:33.884763+0000","last_unstale":"2024-04-17T10:50:33.887894+0000","last_undegraded":"2024-04-17T10:50:33.887894+0000","last_fullsized":"2024-04-17T10:50:33.887894+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:49.010628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.876872+0000","last_change":"2024-04-17T10:50:33.876872+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.876872+0000","last_undegraded":"2024-04-17T10:50:33.876872+0000","last_fullsized":"2024-04-17T10:50:33.876872+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.896605+0000","last_change":"2024-04-17T10:50:33.895658+0000","last_active":"2024-04-17T10:50:33.896605+0000","last_peered":"2024-04-17T10:50:33.896605+0000","last_clean":"2024-04-17T10:50:33.896605+0000","last_became_active":"2024-04-17T10:50:33.894228+0000","last_became_peered":"2024-04-17T10:50:33.894228+0000","last_unstale":"2024-04-17T10:50:33.896605+0000","last_undegraded":"2024-04-17T10:50:33.896605+0000","last_fullsized":"2024-04-17T10:50:33.896605+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:02:11.481439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.872072+0000","last_change":"2024-04-17T10:50:33.872072+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.872072+0000","last_undegraded":"2024-04-17T10:50:33.872072+0000","last_fullsized":"2024-04-17T10:50:33.872072+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.896231+0000","last_change":"2024-04-17T10:50:33.890039+0000","last_active":"2024-04-17T10:50:33.896231+0000","last_peered":"2024-04-17T10:50:33.896231+0000","last_clean":"2024-04-17T10:50:33.896231+0000","last_became_active":"2024-04-17T10:50:33.889635+0000","last_became_peered":"2024-04-17T10:50:33.889635+0000","last_unstale":"2024-04-17T10:50:33.896231+0000","last_undegraded":"2024-04-17T10:50:33.896231+0000","last_fullsized":"2024-04-17T10:50:33.896231+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:02:11.481439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.879906+0000","last_change":"2024-04-17T10:50:33.879906+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.879906+0000","last_undegraded":"2024-04-17T10:50:33.879906+0000","last_fullsized":"2024-04-17T10:50:33.879906+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.904390+0000","last_change":"2024-04-17T10:50:33.898934+0000","last_active":"2024-04-17T10:50:33.904390+0000","last_peered":"2024-04-17T10:50:33.904390+0000","last_clean":"2024-04-17T10:50:33.904390+0000","last_became_active":"2024-04-17T10:50:33.897331+0000","last_became_peered":"2024-04-17T10:50:33.897331+0000","last_unstale":"2024-04-17T10:50:33.904390+0000","last_undegraded":"2024-04-17T10:50:33.904390+0000","last_fullsized":"2024-04-17T10:50:33.904390+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:02:11.481439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.881117+0000","last_change":"2024-04-17T10:50:33.881117+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.881117+0000","last_undegraded":"2024-04-17T10:50:33.881117+0000","last_fullsized":"2024-04-17T10:50:33.881117+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.901174+0000","last_change":"2024-04-17T10:50:33.900576+0000","last_active":"2024-04-17T10:50:33.901174+0000","last_peered":"2024-04-17T10:50:33.901174+0000","last_clean":"2024-04-17T10:50:33.901174+0000","last_became_active":"2024-04-17T10:50:33.899933+0000","last_became_peered":"2024-04-17T10:50:33.899933+0000","last_unstale":"2024-04-17T10:50:33.901174+0000","last_undegraded":"2024-04-17T10:50:33.901174+0000","last_fullsized":"2024-04-17T10:50:33.901174+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:08:11.608039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.877267+0000","last_change":"2024-04-17T10:50:33.877267+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.877267+0000","last_undegraded":"2024-04-17T10:50:33.877267+0000","last_fullsized":"2024-04-17T10:50:33.877267+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.897145+0000","last_change":"2024-04-17T10:50:33.890641+0000","last_active":"2024-04-17T10:50:33.897145+0000","last_peered":"2024-04-17T10:50:33.897145+0000","last_clean":"2024-04-17T10:50:33.897145+0000","last_became_active":"2024-04-17T10:50:33.889972+0000","last_became_peered":"2024-04-17T10:50:33.889972+0000","last_unstale":"2024-04-17T10:50:33.897145+0000","last_undegraded":"2024-04-17T10:50:33.897145+0000","last_fullsized":"2024-04-17T10:50:33.897145+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:29:06.526982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.875628+0000","last_change":"2024-04-17T10:50:33.875628+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.875628+0000","last_undegraded":"2024-04-17T10:50:33.875628+0000","last_fullsized":"2024-04-17T10:50:33.875628+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.899430+0000","last_change":"2024-04-17T10:50:33.897839+0000","last_active":"2024-04-17T10:50:33.899430+0000","last_peered":"2024-04-17T10:50:33.899430+0000","last_clean":"2024-04-17T10:50:33.899430+0000","last_became_active":"2024-04-17T10:50:33.897168+0000","last_became_peered":"2024-04-17T10:50:33.897168+0000","last_unstale":"2024-04-17T10:50:33.899430+0000","last_undegraded":"2024-04-17T10:50:33.899430+0000","last_fullsized":"2024-04-17T10:50:33.899430+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:25:56.552880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.878461+0000","last_change":"2024-04-17T10:50:33.878461+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.878461+0000","last_undegraded":"2024-04-17T10:50:33.878461+0000","last_fullsized":"2024-04-17T10:50:33.878461+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.899185+0000","last_change":"2024-04-17T10:50:33.895104+0000","last_active":"2024-04-17T10:50:33.899185+0000","last_peered":"2024-04-17T10:50:33.899185+0000","last_clean":"2024-04-17T10:50:33.899185+0000","last_became_active":"2024-04-17T10:50:33.894652+0000","last_became_peered":"2024-04-17T10:50:33.894652+0000","last_unstale":"2024-04-17T10:50:33.899185+0000","last_undegraded":"2024-04-17T10:50:33.899185+0000","last_fullsized":"2024-04-17T10:50:33.899185+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:17:40.737496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.871177+0000","last_change":"2024-04-17T10:50:33.871177+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.871177+0000","last_undegraded":"2024-04-17T10:50:33.871177+0000","last_fullsized":"2024-04-17T10:50:33.871177+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.894016+0000","last_change":"2024-04-17T10:50:33.892680+0000","last_active":"2024-04-17T10:50:33.894016+0000","last_peered":"2024-04-17T10:50:33.894016+0000","last_clean":"2024-04-17T10:50:33.894016+0000","last_became_active":"2024-04-17T10:50:33.891996+0000","last_became_peered":"2024-04-17T10:50:33.891996+0000","last_unstale":"2024-04-17T10:50:33.894016+0000","last_undegraded":"2024-04-17T10:50:33.894016+0000","last_fullsized":"2024-04-17T10:50:33.894016+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:36.242473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.880302+0000","last_change":"2024-04-17T10:50:33.880302+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.880302+0000","last_undegraded":"2024-04-17T10:50:33.880302+0000","last_fullsized":"2024-04-17T10:50:33.880302+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.899936+0000","last_change":"2024-04-17T10:50:33.898138+0000","last_active":"2024-04-17T10:50:33.899936+0000","last_peered":"2024-04-17T10:50:33.899936+0000","last_clean":"2024-04-17T10:50:33.899936+0000","last_became_active":"2024-04-17T10:50:33.896506+0000","last_became_peered":"2024-04-17T10:50:33.896506+0000","last_unstale":"2024-04-17T10:50:33.899936+0000","last_undegraded":"2024-04-17T10:50:33.899936+0000","last_fullsized":"2024-04-17T10:50:33.899936+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:35:46.656830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.880316+0000","last_change":"2024-04-17T10:50:33.880316+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.880316+0000","last_undegraded":"2024-04-17T10:50:33.880316+0000","last_fullsized":"2024-04-17T10:50:33.880316+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.890096+0000","last_change":"2024-04-17T10:50:33.887380+0000","last_active":"2024-04-17T10:50:33.890096+0000","last_peered":"2024-04-17T10:50:33.890096+0000","last_clean":"2024-04-17T10:50:33.890096+0000","last_became_active":"2024-04-17T10:50:33.887003+0000","last_became_peered":"2024-04-17T10:50:33.887003+0000","last_unstale":"2024-04-17T10:50:33.890096+0000","last_undegraded":"2024-04-17T10:50:33.890096+0000","last_fullsized":"2024-04-17T10:50:33.890096+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:25:00.410077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.877105+0000","last_change":"2024-04-17T10:50:33.877105+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.877105+0000","last_undegraded":"2024-04-17T10:50:33.877105+0000","last_fullsized":"2024-04-17T10:50:33.877105+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.893790+0000","last_change":"2024-04-17T10:50:33.890628+0000","last_active":"2024-04-17T10:50:33.893790+0000","last_peered":"2024-04-17T10:50:33.893790+0000","last_clean":"2024-04-17T10:50:33.893790+0000","last_became_active":"2024-04-17T10:50:33.890232+0000","last_became_peered":"2024-04-17T10:50:33.890232+0000","last_unstale":"2024-04-17T10:50:33.893790+0000","last_undegraded":"2024-04-17T10:50:33.893790+0000","last_fullsized":"2024-04-17T10:50:33.893790+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:55.719217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.872686+0000","last_change":"2024-04-17T10:50:33.872686+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.872686+0000","last_undegraded":"2024-04-17T10:50:33.872686+0000","last_fullsized":"2024-04-17T10:50:33.872686+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.904423+0000","last_change":"2024-04-17T10:50:33.901531+0000","last_active":"2024-04-17T10:50:33.904423+0000","last_peered":"2024-04-17T10:50:33.904423+0000","last_clean":"2024-04-17T10:50:33.904423+0000","last_became_active":"2024-04-17T10:50:33.901224+0000","last_became_peered":"2024-04-17T10:50:33.901224+0000","last_unstale":"2024-04-17T10:50:33.904423+0000","last_undegraded":"2024-04-17T10:50:33.904423+0000","last_fullsized":"2024-04-17T10:50:33.904423+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:16:19.474043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.891453+0000","last_change":"2024-04-17T10:50:33.891453+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.891453+0000","last_undegraded":"2024-04-17T10:50:33.891453+0000","last_fullsized":"2024-04-17T10:50:33.891453+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.896430+0000","last_change":"2024-04-17T10:50:33.891385+0000","last_active":"2024-04-17T10:50:33.896430+0000","last_peered":"2024-04-17T10:50:33.896430+0000","last_clean":"2024-04-17T10:50:33.896430+0000","last_became_active":"2024-04-17T10:50:33.890896+0000","last_became_peered":"2024-04-17T10:50:33.890896+0000","last_unstale":"2024-04-17T10:50:33.896430+0000","last_undegraded":"2024-04-17T10:50:33.896430+0000","last_fullsized":"2024-04-17T10:50:33.896430+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:35:46.656830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.870668+0000","last_change":"2024-04-17T10:50:33.870668+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.870668+0000","last_undegraded":"2024-04-17T10:50:33.870668+0000","last_fullsized":"2024-04-17T10:50:33.870668+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.896755+0000","last_change":"2024-04-17T10:50:33.893344+0000","last_active":"2024-04-17T10:50:33.896755+0000","last_peered":"2024-04-17T10:50:33.896755+0000","last_clean":"2024-04-17T10:50:33.896755+0000","last_became_active":"2024-04-17T10:50:33.892680+0000","last_became_peered":"2024-04-17T10:50:33.892680+0000","last_unstale":"2024-04-17T10:50:33.896755+0000","last_undegraded":"2024-04-17T10:50:33.896755+0000","last_fullsized":"2024-04-17T10:50:33.896755+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:01:25.099953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.884791+0000","last_change":"2024-04-17T10:50:33.884791+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.884791+0000","last_undegraded":"2024-04-17T10:50:33.884791+0000","last_fullsized":"2024-04-17T10:50:33.884791+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.900021+0000","last_change":"2024-04-17T10:50:33.897362+0000","last_active":"2024-04-17T10:50:33.900021+0000","last_peered":"2024-04-17T10:50:33.900021+0000","last_clean":"2024-04-17T10:50:33.900021+0000","last_became_active":"2024-04-17T10:50:33.896738+0000","last_became_peered":"2024-04-17T10:50:33.896738+0000","last_unstale":"2024-04-17T10:50:33.900021+0000","last_undegraded":"2024-04-17T10:50:33.900021+0000","last_fullsized":"2024-04-17T10:50:33.900021+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:49.010628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.875977+0000","last_change":"2024-04-17T10:50:33.875977+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.875977+0000","last_undegraded":"2024-04-17T10:50:33.875977+0000","last_fullsized":"2024-04-17T10:50:33.875977+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.896611+0000","last_change":"2024-04-17T10:50:33.893658+0000","last_active":"2024-04-17T10:50:33.896611+0000","last_peered":"2024-04-17T10:50:33.896611+0000","last_clean":"2024-04-17T10:50:33.896611+0000","last_became_active":"2024-04-17T10:50:33.893023+0000","last_became_peered":"2024-04-17T10:50:33.893023+0000","last_unstale":"2024-04-17T10:50:33.896611+0000","last_undegraded":"2024-04-17T10:50:33.896611+0000","last_fullsized":"2024-04-17T10:50:33.896611+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:48:29.238257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.880974+0000","last_change":"2024-04-17T10:50:33.880974+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.880974+0000","last_undegraded":"2024-04-17T10:50:33.880974+0000","last_fullsized":"2024-04-17T10:50:33.880974+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.892795+0000","last_change":"2024-04-17T10:50:33.887529+0000","last_active":"2024-04-17T10:50:33.892795+0000","last_peered":"2024-04-17T10:50:33.892795+0000","last_clean":"2024-04-17T10:50:33.892795+0000","last_became_active":"2024-04-17T10:50:33.887152+0000","last_became_peered":"2024-04-17T10:50:33.887152+0000","last_unstale":"2024-04-17T10:50:33.892795+0000","last_undegraded":"2024-04-17T10:50:33.892795+0000","last_fullsized":"2024-04-17T10:50:33.892795+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:04.970052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.878592+0000","last_change":"2024-04-17T10:50:33.878592+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.878592+0000","last_undegraded":"2024-04-17T10:50:33.878592+0000","last_fullsized":"2024-04-17T10:50:33.878592+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.892986+0000","last_change":"2024-04-17T10:50:33.889203+0000","last_active":"2024-04-17T10:50:33.892986+0000","last_peered":"2024-04-17T10:50:33.892986+0000","last_clean":"2024-04-17T10:50:33.892986+0000","last_became_active":"2024-04-17T10:50:33.888312+0000","last_became_peered":"2024-04-17T10:50:33.888312+0000","last_unstale":"2024-04-17T10:50:33.892986+0000","last_undegraded":"2024-04-17T10:50:33.892986+0000","last_fullsized":"2024-04-17T10:50:33.892986+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:23:11.975227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.876607+0000","last_change":"2024-04-17T10:50:33.876607+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.876607+0000","last_undegraded":"2024-04-17T10:50:33.876607+0000","last_fullsized":"2024-04-17T10:50:33.876607+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.892508+0000","last_change":"2024-04-17T10:50:33.882648+0000","last_active":"2024-04-17T10:50:33.892508+0000","last_peered":"2024-04-17T10:50:33.892508+0000","last_clean":"2024-04-17T10:50:33.892508+0000","last_became_active":"2024-04-17T10:50:33.882297+0000","last_became_peered":"2024-04-17T10:50:33.882297+0000","last_unstale":"2024-04-17T10:50:33.892508+0000","last_undegraded":"2024-04-17T10:50:33.892508+0000","last_fullsized":"2024-04-17T10:50:33.892508+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:29:30.703224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.879901+0000","last_change":"2024-04-17T10:50:33.879901+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.879901+0000","last_undegraded":"2024-04-17T10:50:33.879901+0000","last_fullsized":"2024-04-17T10:50:33.879901+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.904388+0000","last_change":"2024-04-17T10:50:33.900315+0000","last_active":"2024-04-17T10:50:33.904388+0000","last_peered":"2024-04-17T10:50:33.904388+0000","last_clean":"2024-04-17T10:50:33.904388+0000","last_became_active":"2024-04-17T10:50:33.899987+0000","last_became_peered":"2024-04-17T10:50:33.899987+0000","last_unstale":"2024-04-17T10:50:33.904388+0000","last_undegraded":"2024-04-17T10:50:33.904388+0000","last_fullsized":"2024-04-17T10:50:33.904388+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:35:46.656830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.878334+0000","last_change":"2024-04-17T10:50:33.878334+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.878334+0000","last_undegraded":"2024-04-17T10:50:33.878334+0000","last_fullsized":"2024-04-17T10:50:33.878334+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.896788+0000","last_change":"2024-04-17T10:50:33.894342+0000","last_active":"2024-04-17T10:50:33.896788+0000","last_peered":"2024-04-17T10:50:33.896788+0000","last_clean":"2024-04-17T10:50:33.896788+0000","last_became_active":"2024-04-17T10:50:33.893720+0000","last_became_peered":"2024-04-17T10:50:33.893720+0000","last_unstale":"2024-04-17T10:50:33.896788+0000","last_undegraded":"2024-04-17T10:50:33.896788+0000","last_fullsized":"2024-04-17T10:50:33.896788+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:21:59.523164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.869757+0000","last_change":"2024-04-17T10:50:33.869757+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.869757+0000","last_undegraded":"2024-04-17T10:50:33.869757+0000","last_fullsized":"2024-04-17T10:50:33.869757+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.898205+0000","last_change":"2024-04-17T10:50:33.895914+0000","last_active":"2024-04-17T10:50:33.898205+0000","last_peered":"2024-04-17T10:50:33.898205+0000","last_clean":"2024-04-17T10:50:33.898205+0000","last_became_active":"2024-04-17T10:50:33.893274+0000","last_became_peered":"2024-04-17T10:50:33.893274+0000","last_unstale":"2024-04-17T10:50:33.898205+0000","last_undegraded":"2024-04-17T10:50:33.898205+0000","last_fullsized":"2024-04-17T10:50:33.898205+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:44:02.588052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.877441+0000","last_change":"2024-04-17T10:50:33.877441+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.877441+0000","last_undegraded":"2024-04-17T10:50:33.877441+0000","last_fullsized":"2024-04-17T10:50:33.877441+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.896924+0000","last_change":"2024-04-17T10:50:33.893329+0000","last_active":"2024-04-17T10:50:33.896924+0000","last_peered":"2024-04-17T10:50:33.896924+0000","last_clean":"2024-04-17T10:50:33.896924+0000","last_became_active":"2024-04-17T10:50:33.892918+0000","last_became_peered":"2024-04-17T10:50:33.892918+0000","last_unstale":"2024-04-17T10:50:33.896924+0000","last_undegraded":"2024-04-17T10:50:33.896924+0000","last_fullsized":"2024-04-17T10:50:33.896924+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:43:18.553237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.885117+0000","last_change":"2024-04-17T10:50:33.885117+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.885117+0000","last_undegraded":"2024-04-17T10:50:33.885117+0000","last_fullsized":"2024-04-17T10:50:33.885117+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.898263+0000","last_change":"2024-04-17T10:50:33.896124+0000","last_active":"2024-04-17T10:50:33.898263+0000","last_peered":"2024-04-17T10:50:33.898263+0000","last_clean":"2024-04-17T10:50:33.898263+0000","last_became_active":"2024-04-17T10:50:33.895638+0000","last_became_peered":"2024-04-17T10:50:33.895638+0000","last_unstale":"2024-04-17T10:50:33.898263+0000","last_undegraded":"2024-04-17T10:50:33.898263+0000","last_fullsized":"2024-04-17T10:50:33.898263+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:03:31.940294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":412,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.876578+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:50:33.876578+0000","last_peered":"2024-04-17T10:50:33.876578+0000","last_clean":"2024-04-17T10:50:33.876578+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:50:33.876578+0000","last_undegraded":"2024-04-17T10:50:33.876578+0000","last_fullsized":"2024-04-17T10:50:33.876578+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575003,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47324,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324516,"statfs":{"total":96636764160,"available":96588304384,"internally_reserved":0,"allocated":2310144,"data_stored":1339919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575002,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68952,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302888,"statfs":{"total":96636764160,"available":96566157312,"internally_reserved":0,"allocated":1716224,"data_stored":749551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575003,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55080,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316760,"statfs":{"total":96636764160,"available":96580362240,"internally_reserved":0,"allocated":1728512,"data_stored":755826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575002,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73124,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298716,"statfs":{"total":96636764160,"available":96561885184,"internally_reserved":0,"allocated":1716224,"data_stored":749551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575004,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73112,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298728,"statfs":{"total":96636764160,"available":96561897472,"internally_reserved":0,"allocated":1716224,"data_stored":749551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575002,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46748,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325092,"statfs":{"total":96636764160,"available":96588894208,"internally_reserved":0,"allocated":1716224,"data_stored":749551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575002,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82848,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288992,"statfs":{"total":96636764160,"available":96551927808,"internally_reserved":0,"allocated":1716224,"data_stored":749551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575002,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47336,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324504,"statfs":{"total":96636764160,"available":96588292096,"internally_reserved":0,"allocated":2322432,"data_stored":1346194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:50:38.579 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3f is not active+clean 2024-04-17T10:50:38.579 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.876422+0000', 'last_change': '2024-04-17T10:50:33.876422+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.876422+0000', 'last_undegraded': '2024-04-17T10:50:33.876422+0000', 'last_fullsized': '2024-04-17T10:50:33.876422+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:50:38.579 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3e is not active+clean 2024-04-17T10:50:38.579 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.878304+0000', 'last_change': '2024-04-17T10:50:33.878304+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.878304+0000', 'last_undegraded': '2024-04-17T10:50:33.878304+0000', 'last_fullsized': '2024-04-17T10:50:33.878304+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:50:38.580 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3d is not active+clean 2024-04-17T10:50:38.580 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.880142+0000', 'last_change': '2024-04-17T10:50:33.880142+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.880142+0000', 'last_undegraded': '2024-04-17T10:50:33.880142+0000', 'last_fullsized': '2024-04-17T10:50:33.880142+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:50:38.580 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3c is not active+clean 2024-04-17T10:50:38.580 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.871442+0000', 'last_change': '2024-04-17T10:50:33.871442+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.871442+0000', 'last_undegraded': '2024-04-17T10:50:33.871442+0000', 'last_fullsized': '2024-04-17T10:50:33.871442+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:50:38.580 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3b is not active+clean 2024-04-17T10:50:38.580 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.879260+0000', 'last_change': '2024-04-17T10:50:33.879260+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.879260+0000', 'last_undegraded': '2024-04-17T10:50:33.879260+0000', 'last_fullsized': '2024-04-17T10:50:33.879260+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:50:38.580 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3a is not active+clean 2024-04-17T10:50:38.580 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.875526+0000', 'last_change': '2024-04-17T10:50:33.875526+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.875526+0000', 'last_undegraded': '2024-04-17T10:50:33.875526+0000', 'last_fullsized': '2024-04-17T10:50:33.875526+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:50:38.580 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.39 is not active+clean 2024-04-17T10:50:38.580 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.884342+0000', 'last_change': '2024-04-17T10:50:33.884342+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.884342+0000', 'last_undegraded': '2024-04-17T10:50:33.884342+0000', 'last_fullsized': '2024-04-17T10:50:33.884342+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:50:38.581 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.38 is not active+clean 2024-04-17T10:50:38.581 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.875967+0000', 'last_change': '2024-04-17T10:50:33.875967+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.875967+0000', 'last_undegraded': '2024-04-17T10:50:33.875967+0000', 'last_fullsized': '2024-04-17T10:50:33.875967+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:50:38.581 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.37 is not active+clean 2024-04-17T10:50:38.581 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.881528+0000', 'last_change': '2024-04-17T10:50:33.881528+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.881528+0000', 'last_undegraded': '2024-04-17T10:50:33.881528+0000', 'last_fullsized': '2024-04-17T10:50:33.881528+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:50:38.581 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.36 is not active+clean 2024-04-17T10:50:38.581 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.874184+0000', 'last_change': '2024-04-17T10:50:33.874184+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.874184+0000', 'last_undegraded': '2024-04-17T10:50:33.874184+0000', 'last_fullsized': '2024-04-17T10:50:33.874184+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:50:38.581 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.35 is not active+clean 2024-04-17T10:50:38.581 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.884228+0000', 'last_change': '2024-04-17T10:50:33.884228+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.884228+0000', 'last_undegraded': '2024-04-17T10:50:33.884228+0000', 'last_fullsized': '2024-04-17T10:50:33.884228+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:50:38.581 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.34 is not active+clean 2024-04-17T10:50:38.581 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.870313+0000', 'last_change': '2024-04-17T10:50:33.870313+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.870313+0000', 'last_undegraded': '2024-04-17T10:50:33.870313+0000', 'last_fullsized': '2024-04-17T10:50:33.870313+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:50:38.582 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.33 is not active+clean 2024-04-17T10:50:38.582 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.878811+0000', 'last_change': '2024-04-17T10:50:33.878811+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.878811+0000', 'last_undegraded': '2024-04-17T10:50:33.878811+0000', 'last_fullsized': '2024-04-17T10:50:33.878811+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:50:38.582 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.32 is not active+clean 2024-04-17T10:50:38.582 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.875112+0000', 'last_change': '2024-04-17T10:50:33.875112+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.875112+0000', 'last_undegraded': '2024-04-17T10:50:33.875112+0000', 'last_fullsized': '2024-04-17T10:50:33.875112+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:50:38.582 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.31 is not active+clean 2024-04-17T10:50:38.582 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.869759+0000', 'last_change': '2024-04-17T10:50:33.869759+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.869759+0000', 'last_undegraded': '2024-04-17T10:50:33.869759+0000', 'last_fullsized': '2024-04-17T10:50:33.869759+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:50:38.582 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.30 is not active+clean 2024-04-17T10:50:38.582 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.883895+0000', 'last_change': '2024-04-17T10:50:33.883895+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.883895+0000', 'last_undegraded': '2024-04-17T10:50:33.883895+0000', 'last_fullsized': '2024-04-17T10:50:33.883895+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:50:38.582 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2f is not active+clean 2024-04-17T10:50:38.582 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.871401+0000', 'last_change': '2024-04-17T10:50:33.871401+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.871401+0000', 'last_undegraded': '2024-04-17T10:50:33.871401+0000', 'last_fullsized': '2024-04-17T10:50:33.871401+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:50:38.583 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2e is not active+clean 2024-04-17T10:50:38.583 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.881276+0000', 'last_change': '2024-04-17T10:50:33.881276+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.881276+0000', 'last_undegraded': '2024-04-17T10:50:33.881276+0000', 'last_fullsized': '2024-04-17T10:50:33.881276+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:50:38.583 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2d is not active+clean 2024-04-17T10:50:38.583 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.883559+0000', 'last_change': '2024-04-17T10:50:33.883559+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.883559+0000', 'last_undegraded': '2024-04-17T10:50:33.883559+0000', 'last_fullsized': '2024-04-17T10:50:33.883559+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:50:38.583 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2c is not active+clean 2024-04-17T10:50:38.583 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.885482+0000', 'last_change': '2024-04-17T10:50:33.885482+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.885482+0000', 'last_undegraded': '2024-04-17T10:50:33.885482+0000', 'last_fullsized': '2024-04-17T10:50:33.885482+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:50:38.583 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2b is not active+clean 2024-04-17T10:50:38.583 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.871386+0000', 'last_change': '2024-04-17T10:50:33.871386+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.871386+0000', 'last_undegraded': '2024-04-17T10:50:33.871386+0000', 'last_fullsized': '2024-04-17T10:50:33.871386+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:50:38.583 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2a is not active+clean 2024-04-17T10:50:38.583 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.874084+0000', 'last_change': '2024-04-17T10:50:33.874084+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.874084+0000', 'last_undegraded': '2024-04-17T10:50:33.874084+0000', 'last_fullsized': '2024-04-17T10:50:33.874084+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:50:38.583 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.29 is not active+clean 2024-04-17T10:50:38.584 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.869757+0000', 'last_change': '2024-04-17T10:50:33.869757+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.869757+0000', 'last_undegraded': '2024-04-17T10:50:33.869757+0000', 'last_fullsized': '2024-04-17T10:50:33.869757+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:50:38.584 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.28 is not active+clean 2024-04-17T10:50:38.584 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.873283+0000', 'last_change': '2024-04-17T10:50:33.873283+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.873283+0000', 'last_undegraded': '2024-04-17T10:50:33.873283+0000', 'last_fullsized': '2024-04-17T10:50:33.873283+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:50:38.584 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.27 is not active+clean 2024-04-17T10:50:38.584 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.883856+0000', 'last_change': '2024-04-17T10:50:33.883856+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.883856+0000', 'last_undegraded': '2024-04-17T10:50:33.883856+0000', 'last_fullsized': '2024-04-17T10:50:33.883856+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:50:38.584 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.26 is not active+clean 2024-04-17T10:50:38.584 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.881781+0000', 'last_change': '2024-04-17T10:50:33.881781+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.881781+0000', 'last_undegraded': '2024-04-17T10:50:33.881781+0000', 'last_fullsized': '2024-04-17T10:50:33.881781+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:50:38.584 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.25 is not active+clean 2024-04-17T10:50:38.584 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.881470+0000', 'last_change': '2024-04-17T10:50:33.881470+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.881470+0000', 'last_undegraded': '2024-04-17T10:50:33.881470+0000', 'last_fullsized': '2024-04-17T10:50:33.881470+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:50:38.584 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.24 is not active+clean 2024-04-17T10:50:38.584 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.885926+0000', 'last_change': '2024-04-17T10:50:33.885926+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.885926+0000', 'last_undegraded': '2024-04-17T10:50:33.885926+0000', 'last_fullsized': '2024-04-17T10:50:33.885926+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:50:38.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.23 is not active+clean 2024-04-17T10:50:38.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.875431+0000', 'last_change': '2024-04-17T10:50:33.875431+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.875431+0000', 'last_undegraded': '2024-04-17T10:50:33.875431+0000', 'last_fullsized': '2024-04-17T10:50:33.875431+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:50:38.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.22 is not active+clean 2024-04-17T10:50:38.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.873345+0000', 'last_change': '2024-04-17T10:50:33.873345+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.873345+0000', 'last_undegraded': '2024-04-17T10:50:33.873345+0000', 'last_fullsized': '2024-04-17T10:50:33.873345+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:50:38.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.21 is not active+clean 2024-04-17T10:50:38.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.878027+0000', 'last_change': '2024-04-17T10:50:33.878027+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.878027+0000', 'last_undegraded': '2024-04-17T10:50:33.878027+0000', 'last_fullsized': '2024-04-17T10:50:33.878027+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:50:38.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.20 is not active+clean 2024-04-17T10:50:38.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.880017+0000', 'last_change': '2024-04-17T10:50:33.880017+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.880017+0000', 'last_undegraded': '2024-04-17T10:50:33.880017+0000', 'last_fullsized': '2024-04-17T10:50:33.880017+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:50:38.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1e is not active+clean 2024-04-17T10:50:38.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.871860+0000', 'last_change': '2024-04-17T10:50:33.871860+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.871860+0000', 'last_undegraded': '2024-04-17T10:50:33.871860+0000', 'last_fullsized': '2024-04-17T10:50:33.871860+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:50:38.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1f is not active+clean 2024-04-17T10:50:38.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.880475+0000', 'last_change': '2024-04-17T10:50:33.880475+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.880475+0000', 'last_undegraded': '2024-04-17T10:50:33.880475+0000', 'last_fullsized': '2024-04-17T10:50:33.880475+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:50:38.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1c is not active+clean 2024-04-17T10:50:38.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.876197+0000', 'last_change': '2024-04-17T10:50:33.876197+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.876197+0000', 'last_undegraded': '2024-04-17T10:50:33.876197+0000', 'last_fullsized': '2024-04-17T10:50:33.876197+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:50:38.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1d is not active+clean 2024-04-17T10:50:38.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.870653+0000', 'last_change': '2024-04-17T10:50:33.870653+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.870653+0000', 'last_undegraded': '2024-04-17T10:50:33.870653+0000', 'last_fullsized': '2024-04-17T10:50:33.870653+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:50:38.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1a is not active+clean 2024-04-17T10:50:38.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.877422+0000', 'last_change': '2024-04-17T10:50:33.877422+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.877422+0000', 'last_undegraded': '2024-04-17T10:50:33.877422+0000', 'last_fullsized': '2024-04-17T10:50:33.877422+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:50:38.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1b is not active+clean 2024-04-17T10:50:38.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.872397+0000', 'last_change': '2024-04-17T10:50:33.872397+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.872397+0000', 'last_undegraded': '2024-04-17T10:50:33.872397+0000', 'last_fullsized': '2024-04-17T10:50:33.872397+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:50:38.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.18 is not active+clean 2024-04-17T10:50:38.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.878606+0000', 'last_change': '2024-04-17T10:50:33.878606+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.878606+0000', 'last_undegraded': '2024-04-17T10:50:33.878606+0000', 'last_fullsized': '2024-04-17T10:50:33.878606+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:50:38.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.19 is not active+clean 2024-04-17T10:50:38.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.879870+0000', 'last_change': '2024-04-17T10:50:33.879870+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.879870+0000', 'last_undegraded': '2024-04-17T10:50:33.879870+0000', 'last_fullsized': '2024-04-17T10:50:33.879870+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:50:38.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.16 is not active+clean 2024-04-17T10:50:38.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.876872+0000', 'last_change': '2024-04-17T10:50:33.876872+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.876872+0000', 'last_undegraded': '2024-04-17T10:50:33.876872+0000', 'last_fullsized': '2024-04-17T10:50:33.876872+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:50:38.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.17 is not active+clean 2024-04-17T10:50:38.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.872072+0000', 'last_change': '2024-04-17T10:50:33.872072+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.872072+0000', 'last_undegraded': '2024-04-17T10:50:33.872072+0000', 'last_fullsized': '2024-04-17T10:50:33.872072+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:50:38.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.15 is not active+clean 2024-04-17T10:50:38.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.879906+0000', 'last_change': '2024-04-17T10:50:33.879906+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.879906+0000', 'last_undegraded': '2024-04-17T10:50:33.879906+0000', 'last_fullsized': '2024-04-17T10:50:33.879906+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:50:38.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.12 is not active+clean 2024-04-17T10:50:38.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.881117+0000', 'last_change': '2024-04-17T10:50:33.881117+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.881117+0000', 'last_undegraded': '2024-04-17T10:50:33.881117+0000', 'last_fullsized': '2024-04-17T10:50:33.881117+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:50:38.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.13 is not active+clean 2024-04-17T10:50:38.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.877267+0000', 'last_change': '2024-04-17T10:50:33.877267+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.877267+0000', 'last_undegraded': '2024-04-17T10:50:33.877267+0000', 'last_fullsized': '2024-04-17T10:50:33.877267+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:50:38.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.10 is not active+clean 2024-04-17T10:50:38.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.875628+0000', 'last_change': '2024-04-17T10:50:33.875628+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.875628+0000', 'last_undegraded': '2024-04-17T10:50:33.875628+0000', 'last_fullsized': '2024-04-17T10:50:33.875628+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:50:38.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.11 is not active+clean 2024-04-17T10:50:38.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.878461+0000', 'last_change': '2024-04-17T10:50:33.878461+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.878461+0000', 'last_undegraded': '2024-04-17T10:50:33.878461+0000', 'last_fullsized': '2024-04-17T10:50:33.878461+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:50:38.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.e is not active+clean 2024-04-17T10:50:38.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.871177+0000', 'last_change': '2024-04-17T10:50:33.871177+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.871177+0000', 'last_undegraded': '2024-04-17T10:50:33.871177+0000', 'last_fullsized': '2024-04-17T10:50:33.871177+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:50:38.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.f is not active+clean 2024-04-17T10:50:38.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.880302+0000', 'last_change': '2024-04-17T10:50:33.880302+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.880302+0000', 'last_undegraded': '2024-04-17T10:50:33.880302+0000', 'last_fullsized': '2024-04-17T10:50:33.880302+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:50:38.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.c is not active+clean 2024-04-17T10:50:38.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.880316+0000', 'last_change': '2024-04-17T10:50:33.880316+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.880316+0000', 'last_undegraded': '2024-04-17T10:50:33.880316+0000', 'last_fullsized': '2024-04-17T10:50:33.880316+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:50:38.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.d is not active+clean 2024-04-17T10:50:38.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.877105+0000', 'last_change': '2024-04-17T10:50:33.877105+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.877105+0000', 'last_undegraded': '2024-04-17T10:50:33.877105+0000', 'last_fullsized': '2024-04-17T10:50:33.877105+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:50:38.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.a is not active+clean 2024-04-17T10:50:38.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.872686+0000', 'last_change': '2024-04-17T10:50:33.872686+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.872686+0000', 'last_undegraded': '2024-04-17T10:50:33.872686+0000', 'last_fullsized': '2024-04-17T10:50:33.872686+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:50:38.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.b is not active+clean 2024-04-17T10:50:38.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.891453+0000', 'last_change': '2024-04-17T10:50:33.891453+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.891453+0000', 'last_undegraded': '2024-04-17T10:50:33.891453+0000', 'last_fullsized': '2024-04-17T10:50:33.891453+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:50:38.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.8 is not active+clean 2024-04-17T10:50:38.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.870668+0000', 'last_change': '2024-04-17T10:50:33.870668+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.870668+0000', 'last_undegraded': '2024-04-17T10:50:33.870668+0000', 'last_fullsized': '2024-04-17T10:50:33.870668+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T10:50:38.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.9 is not active+clean 2024-04-17T10:50:38.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.884791+0000', 'last_change': '2024-04-17T10:50:33.884791+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.884791+0000', 'last_undegraded': '2024-04-17T10:50:33.884791+0000', 'last_fullsized': '2024-04-17T10:50:33.884791+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:50:38.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.6 is not active+clean 2024-04-17T10:50:38.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.875977+0000', 'last_change': '2024-04-17T10:50:33.875977+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.875977+0000', 'last_undegraded': '2024-04-17T10:50:33.875977+0000', 'last_fullsized': '2024-04-17T10:50:33.875977+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:50:38.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.7 is not active+clean 2024-04-17T10:50:38.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.880974+0000', 'last_change': '2024-04-17T10:50:33.880974+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.880974+0000', 'last_undegraded': '2024-04-17T10:50:33.880974+0000', 'last_fullsized': '2024-04-17T10:50:33.880974+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T10:50:38.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.4 is not active+clean 2024-04-17T10:50:38.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.878592+0000', 'last_change': '2024-04-17T10:50:33.878592+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.878592+0000', 'last_undegraded': '2024-04-17T10:50:33.878592+0000', 'last_fullsized': '2024-04-17T10:50:33.878592+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T10:50:38.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.5 is not active+clean 2024-04-17T10:50:38.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.876607+0000', 'last_change': '2024-04-17T10:50:33.876607+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.876607+0000', 'last_undegraded': '2024-04-17T10:50:33.876607+0000', 'last_fullsized': '2024-04-17T10:50:33.876607+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T10:50:38.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2 is not active+clean 2024-04-17T10:50:38.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.879901+0000', 'last_change': '2024-04-17T10:50:33.879901+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.879901+0000', 'last_undegraded': '2024-04-17T10:50:33.879901+0000', 'last_fullsized': '2024-04-17T10:50:33.879901+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T10:50:38.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3 is not active+clean 2024-04-17T10:50:38.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.878334+0000', 'last_change': '2024-04-17T10:50:33.878334+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.878334+0000', 'last_undegraded': '2024-04-17T10:50:33.878334+0000', 'last_fullsized': '2024-04-17T10:50:33.878334+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:50:38.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.0 is not active+clean 2024-04-17T10:50:38.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.869757+0000', 'last_change': '2024-04-17T10:50:33.869757+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.869757+0000', 'last_undegraded': '2024-04-17T10:50:33.869757+0000', 'last_fullsized': '2024-04-17T10:50:33.869757+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T10:50:38.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1 is not active+clean 2024-04-17T10:50:38.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.877441+0000', 'last_change': '2024-04-17T10:50:33.877441+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.877441+0000', 'last_undegraded': '2024-04-17T10:50:33.877441+0000', 'last_fullsized': '2024-04-17T10:50:33.877441+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T10:50:38.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.14 is not active+clean 2024-04-17T10:50:38.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-17T10:50:33.885117+0000', 'last_change': '2024-04-17T10:50:33.885117+0000', 'last_active': '2024-04-17T10:50:33.854740+0000', 'last_peered': '2024-04-17T10:50:33.854740+0000', 'last_clean': '2024-04-17T10:50:33.854740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T10:50:33.885117+0000', 'last_undegraded': '2024-04-17T10:50:33.885117+0000', 'last_fullsized': '2024-04-17T10:50:33.885117+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'last_clean_scrub_stamp': '2024-04-17T10:50:33.854740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-17T10:50:38.592 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-17T10:50:38.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.744+0000 7f4573ecd640 1 Processor -- start 2024-04-17T10:50:38.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.744+0000 7f4573ecd640 1 -- start start 2024-04-17T10:50:38.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.744+0000 7f4573ecd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f456c15ba30 0x7f456c072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:38.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.744+0000 7f4573ecd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f456c15c960 0x7f456c0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:38.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.744+0000 7f4573ecd640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f456c0cfea0 con 0x7f456c0cfa10 2024-04-17T10:50:38.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.744+0000 7f4573ecd640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f456c071ea0 con 0x7f456c15c960 2024-04-17T10:50:38.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.744+0000 7f4573ecd640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f456c06e320 con 0x7f456c15ba30 2024-04-17T10:50:38.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.744+0000 7f457246a640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f456c0cfa10 0x7f456c06c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57166/0 (socket says 172.21.15.83:57166) 2024-04-17T10:50:38.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.744+0000 7f457246a640 1 -- 172.21.15.83:0/1494217585 learned_addr learned my addr 172.21.15.83:0/1494217585 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:38.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.744+0000 7f4571468640 1 --2- 172.21.15.83:0/1494217585 >> v2:172.21.15.83:3300/0 conn(0x7f456c15c960 0x7f456c0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:38.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.744+0000 7f4571c69640 1 --2- 172.21.15.83:0/1494217585 >> v2:172.21.15.167:3300/0 conn(0x7f456c15ba30 0x7f456c072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:38.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.744+0000 7f4570c67640 1 -- 172.21.15.83:0/1494217585 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3706138834 0 0) 0x7f456c0cfea0 con 0x7f456c0cfa10 2024-04-17T10:50:38.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.744+0000 7f4570c67640 1 -- 172.21.15.83:0/1494217585 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4558003530 con 0x7f456c0cfa10 2024-04-17T10:50:38.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.744+0000 7f4571468640 1 -- 172.21.15.83:0/1494217585 >> v2:172.21.15.167:3300/0 conn(0x7f456c15ba30 msgr2=0x7f456c072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:38.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.744+0000 7f4571468640 1 --2- 172.21.15.83:0/1494217585 >> v2:172.21.15.167:3300/0 conn(0x7f456c15ba30 0x7f456c072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:38.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.744+0000 7f4571468640 1 -- 172.21.15.83:0/1494217585 >> v1:172.21.15.83:6789/0 conn(0x7f456c0cfa10 legacy=0x7f456c06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:38.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.744+0000 7f4571468640 1 -- 172.21.15.83:0/1494217585 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f456c06e040 con 0x7f456c15c960 2024-04-17T10:50:38.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.748+0000 7f4571468640 1 --2- 172.21.15.83:0/1494217585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f456c15c960 0x7f456c0cf4d0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f456c0b2410 tx=0x7f455c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f1cf18bce39ba server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:38.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.748+0000 7f4570c67640 1 -- 172.21.15.83:0/1494217585 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455c0b0070 con 0x7f456c15c960 2024-04-17T10:50:38.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.748+0000 7f4570c67640 1 -- 172.21.15.83:0/1494217585 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f455c0ab600 con 0x7f456c15c960 2024-04-17T10:50:38.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.748+0000 7f4570c67640 1 -- 172.21.15.83:0/1494217585 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455c0ab9d0 con 0x7f456c15c960 2024-04-17T10:50:38.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.748+0000 7f4573ecd640 1 -- 172.21.15.83:0/1494217585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f456c15c960 msgr2=0x7f456c0cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:38.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.748+0000 7f4573ecd640 1 --2- 172.21.15.83:0/1494217585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f456c15c960 0x7f456c0cf4d0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f456c0b2410 tx=0x7f455c01c3f0 comp rx=0 tx=0).stop 2024-04-17T10:50:38.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.748+0000 7f4573ecd640 1 -- 172.21.15.83:0/1494217585 shutdown_connections 2024-04-17T10:50:38.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.748+0000 7f4573ecd640 1 --2- 172.21.15.83:0/1494217585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f456c15c960 0x7f456c0cf4d0 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:38.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.748+0000 7f4573ecd640 1 --2- 172.21.15.83:0/1494217585 >> v2:172.21.15.167:3300/0 conn(0x7f456c15ba30 0x7f456c072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:38.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.748+0000 7f4573ecd640 1 -- 172.21.15.83:0/1494217585 >> 172.21.15.83:0/1494217585 conn(0x7f456c06abe0 msgr2=0x7f456c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:38.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.748+0000 7f4573ecd640 1 -- 172.21.15.83:0/1494217585 shutdown_connections 2024-04-17T10:50:38.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.748+0000 7f4573ecd640 1 -- 172.21.15.83:0/1494217585 wait complete. 2024-04-17T10:50:38.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.748+0000 7f4573ecd640 1 Processor -- start 2024-04-17T10:50:38.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.748+0000 7f4573ecd640 1 -- start start 2024-04-17T10:50:38.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.752+0000 7f4573ecd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f456c0cfa10 0x7f456c144510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:38.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.752+0000 7f4573ecd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f456c15ba30 0x7f456c1502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:38.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.752+0000 7f4573ecd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f456c15c960 0x7f456c147ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:38.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.752+0000 7f4573ecd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456c06dca0 con 0x7f456c0cfa10 2024-04-17T10:50:38.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.752+0000 7f4573ecd640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456c0aeae0 con 0x7f456c15ba30 2024-04-17T10:50:38.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.752+0000 7f4573ecd640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f456c06de20 con 0x7f456c15c960 2024-04-17T10:50:38.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.752+0000 7f4571c69640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f456c0cfa10 0x7f456c144510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:38.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.752+0000 7f4571c69640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f456c0cfa10 0x7f456c144510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35338/0 (socket says 172.21.15.83:35338) 2024-04-17T10:50:38.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.752+0000 7f4571c69640 1 -- 172.21.15.83:0/450493791 learned_addr learned my addr 172.21.15.83:0/450493791 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:38.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.752+0000 7f4571c69640 1 -- 172.21.15.83:0/450493791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f456c15c960 msgr2=0x7f456c147ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:38.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.752+0000 7f4571c69640 1 --2- 172.21.15.83:0/450493791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f456c15c960 0x7f456c147ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:38.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.752+0000 7f4571c69640 1 -- 172.21.15.83:0/450493791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f456c15ba30 msgr2=0x7f456c1502d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:38.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.752+0000 7f4571c69640 1 --2- 172.21.15.83:0/450493791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f456c15ba30 0x7f456c1502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:38.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.752+0000 7f4571c69640 1 -- 172.21.15.83:0/450493791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f456c1f7e10 con 0x7f456c0cfa10 2024-04-17T10:50:38.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.752+0000 7f4571c69640 1 --2- 172.21.15.83:0/450493791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f456c0cfa10 0x7f456c144510 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f456800d5f0 tx=0x7f456800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:38.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.752+0000 7f45627fc640 1 -- 172.21.15.83:0/450493791 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45680125f0 con 0x7f456c0cfa10 2024-04-17T10:50:38.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.752+0000 7f45627fc640 1 -- 172.21.15.83:0/450493791 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4568019070 con 0x7f456c0cfa10 2024-04-17T10:50:38.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.752+0000 7f4573ecd640 1 -- 172.21.15.83:0/450493791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f456c1f80f0 con 0x7f456c0cfa10 2024-04-17T10:50:38.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.752+0000 7f45627fc640 1 -- 172.21.15.83:0/450493791 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45680129d0 con 0x7f456c0cfa10 2024-04-17T10:50:38.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.752+0000 7f4573ecd640 1 -- 172.21.15.83:0/450493791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f456c1f86b0 con 0x7f456c0cfa10 2024-04-17T10:50:38.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.752+0000 7f4573ecd640 1 -- 172.21.15.83:0/450493791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f456c074f90 con 0x7f456c0cfa10 2024-04-17T10:50:38.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.752+0000 7f45627fc640 1 -- 172.21.15.83:0/450493791 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4568012b70 con 0x7f456c0cfa10 2024-04-17T10:50:38.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.752+0000 7f45627fc640 1 --2- 172.21.15.83:0/450493791 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4550066f90 0x7f4550069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:38.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.752+0000 7f45627fc640 1 -- 172.21.15.83:0/450493791 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4568085470 con 0x7f456c0cfa10 2024-04-17T10:50:38.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.752+0000 7f4571468640 1 --2- 172.21.15.83:0/450493791 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4550066f90 0x7f4550069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:38.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.752+0000 7f4571468640 1 --2- 172.21.15.83:0/450493791 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4550066f90 0x7f4550069450 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f455c0ad000 tx=0x7f455c0046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:38.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.756+0000 7f45627fc640 1 -- 172.21.15.83:0/450493791 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4568011340 con 0x7f456c0cfa10 2024-04-17T10:50:38.889 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:50:38.891 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:50:38.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.956+0000 7f4573ecd640 1 -- 172.21.15.83:0/450493791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f456c0cf6a0 con 0x7f456c0cfa10 2024-04-17T10:50:38.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.956+0000 7f45627fc640 1 -- 172.21.15.83:0/450493791 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1038 (secure 0 0 0) 0x7f4568056480 con 0x7f456c0cfa10 2024-04-17T10:50:38.951 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:50:38.951 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":771,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":140,"num_osds":8,"num_up_osds":8,"osd_up_since":1713350279,"num_in_osds":8,"osd_in_since":1713350270,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":506392576,"bytes_avail":772587720704,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":87,"id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"a","status":"up:active","gid":9711}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":28,"modified":"2024-04-17T10:50:24.397189+0000","services":{}},"progress_events":{}} 2024-04-17T10:50:38.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.960+0000 7f4573ecd640 1 -- 172.21.15.83:0/450493791 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4550066f90 msgr2=0x7f4550069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:38.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.960+0000 7f4573ecd640 1 --2- 172.21.15.83:0/450493791 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4550066f90 0x7f4550069450 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f455c0ad000 tx=0x7f455c0046d0 comp rx=0 tx=0).stop 2024-04-17T10:50:38.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.960+0000 7f4573ecd640 1 -- 172.21.15.83:0/450493791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f456c0cfa10 msgr2=0x7f456c144510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:38.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.960+0000 7f4573ecd640 1 --2- 172.21.15.83:0/450493791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f456c0cfa10 0x7f456c144510 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f456800d5f0 tx=0x7f456800dab0 comp rx=0 tx=0).stop 2024-04-17T10:50:38.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.960+0000 7f4573ecd640 1 -- 172.21.15.83:0/450493791 shutdown_connections 2024-04-17T10:50:38.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.960+0000 7f4573ecd640 1 --2- 172.21.15.83:0/450493791 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4550066f90 0x7f4550069450 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:38.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.960+0000 7f4573ecd640 1 --2- 172.21.15.83:0/450493791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f456c15c960 0x7f456c147ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:38.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.960+0000 7f4573ecd640 1 --2- 172.21.15.83:0/450493791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f456c15ba30 0x7f456c1502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:38.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.960+0000 7f4573ecd640 1 --2- 172.21.15.83:0/450493791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f456c0cfa10 0x7f456c144510 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:38.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.960+0000 7f4573ecd640 1 -- 172.21.15.83:0/450493791 >> 172.21.15.83:0/450493791 conn(0x7f456c06abe0 msgr2=0x7f456c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:38.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.960+0000 7f4573ecd640 1 -- 172.21.15.83:0/450493791 shutdown_connections 2024-04-17T10:50:38.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:38.960+0000 7f4573ecd640 1 -- 172.21.15.83:0/450493791 wait complete. 2024-04-17T10:50:38.968 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-17T10:50:38.968 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:50:39.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e8697640 1 Processor -- start 2024-04-17T10:50:39.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e8697640 1 -- start start 2024-04-17T10:50:39.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e8697640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66e00ce980 0x7f66e00b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:39.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e8697640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66e0162a00 con 0x7f66e00cee00 2024-04-17T10:50:39.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e8697640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66e0161e80 con 0x7f66e015dfa0 2024-04-17T10:50:39.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e8697640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e0162160 con 0x7f66e00ce980 2024-04-17T10:50:39.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e6c34640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f66e00cee00 0x7f66e00d1d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57182/0 (socket says 172.21.15.83:57182) 2024-04-17T10:50:39.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e6c34640 1 -- 172.21.15.83:0/3055229603 learned_addr learned my addr 172.21.15.83:0/3055229603 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:39.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e5c32640 1 --2- 172.21.15.83:0/3055229603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66e00ce980 0x7f66e00b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:39.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e5431640 1 -- 172.21.15.83:0/3055229603 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 471876782 0 0) 0x7f66e0162a00 con 0x7f66e00cee00 2024-04-17T10:50:39.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e5431640 1 -- 172.21.15.83:0/3055229603 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66cc003530 con 0x7f66e00cee00 2024-04-17T10:50:39.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e5431640 1 -- 172.21.15.83:0/3055229603 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3252418448 0 0) 0x7f66cc003530 con 0x7f66e00cee00 2024-04-17T10:50:39.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e5431640 1 -- 172.21.15.83:0/3055229603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66e00ce980 msgr2=0x7f66e00b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:39.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e5431640 1 --2- 172.21.15.83:0/3055229603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66e00ce980 0x7f66e00b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:39.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e5431640 1 -- 172.21.15.83:0/3055229603 >> v1:172.21.15.167:6789/0 conn(0x7f66e015dfa0 legacy=0x7f66e00b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:39.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e5431640 1 -- 172.21.15.83:0/3055229603 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e0162440 con 0x7f66e00cee00 2024-04-17T10:50:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e5431640 1 -- 172.21.15.83:0/3055229603 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f66d8001a00 con 0x7f66e00cee00 2024-04-17T10:50:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e5431640 1 -- 172.21.15.83:0/3055229603 >> v1:172.21.15.83:6789/0 conn(0x7f66e00cee00 legacy=0x7f66e00d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e5431640 1 --2- 172.21.15.83:0/3055229603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66cc0039f0 0x7f66cc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e5431640 1 --2- 172.21.15.83:0/3055229603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66cc004810 0x7f66cc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e5431640 1 --2- 172.21.15.83:0/3055229603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66e00ce980 0x7f66cc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e5431640 1 -- 172.21.15.83:0/3055229603 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e0162160 con 0x7f66cc004810 2024-04-17T10:50:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e5431640 1 -- 172.21.15.83:0/3055229603 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66cc00bb00 con 0x7f66cc0039f0 2024-04-17T10:50:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e5431640 1 -- 172.21.15.83:0/3055229603 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66cc00bd00 con 0x7f66e00ce980 2024-04-17T10:50:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e5c32640 1 --2- 172.21.15.83:0/3055229603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66cc004810 0x7f66cc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e6c34640 1 --2- 172.21.15.83:0/3055229603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66e00ce980 0x7f66cc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e6433640 1 --2- 172.21.15.83:0/3055229603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66cc0039f0 0x7f66cc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e5c32640 1 -- 172.21.15.83:0/3055229603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66e00ce980 msgr2=0x7f66cc005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e5c32640 1 --2- 172.21.15.83:0/3055229603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66e00ce980 0x7f66cc005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e5c32640 1 -- 172.21.15.83:0/3055229603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66cc0039f0 msgr2=0x7f66cc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e5c32640 1 --2- 172.21.15.83:0/3055229603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66cc0039f0 0x7f66cc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:39.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e5c32640 1 -- 172.21.15.83:0/3055229603 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66cc003530 con 0x7f66cc004810 2024-04-17T10:50:39.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e6c34640 1 --2- 172.21.15.83:0/3055229603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66e00ce980 0x7f66cc005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:39.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.144+0000 7f66e6433640 1 --2- 172.21.15.83:0/3055229603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66cc0039f0 0x7f66cc003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:39.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.148+0000 7f66e5c32640 1 --2- 172.21.15.83:0/3055229603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66cc004810 0x7f66cc004c90 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f66d0009ab0 tx=0x7f66d001ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa18879b387e3f78 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:39.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.148+0000 7f66e5431640 1 -- 172.21.15.83:0/3055229603 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d00b2070 con 0x7f66cc004810 2024-04-17T10:50:39.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.148+0000 7f66e5431640 1 -- 172.21.15.83:0/3055229603 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d00a5590 con 0x7f66cc004810 2024-04-17T10:50:39.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.148+0000 7f66e5431640 1 -- 172.21.15.83:0/3055229603 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d00096b0 con 0x7f66cc004810 2024-04-17T10:50:39.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.148+0000 7f66e8697640 1 -- 172.21.15.83:0/3055229603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66cc004810 msgr2=0x7f66cc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:39.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.148+0000 7f66e8697640 1 --2- 172.21.15.83:0/3055229603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66cc004810 0x7f66cc004c90 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f66d0009ab0 tx=0x7f66d001ca20 comp rx=0 tx=0).stop 2024-04-17T10:50:39.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.148+0000 7f66e6c34640 1 -- 172.21.15.83:0/3055229603 reap_dead start 2024-04-17T10:50:39.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.148+0000 7f66e8697640 1 -- 172.21.15.83:0/3055229603 shutdown_connections 2024-04-17T10:50:39.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.148+0000 7f66e8697640 1 -- 172.21.15.83:0/3055229603 >> 172.21.15.83:0/3055229603 conn(0x7f66e00d0600 msgr2=0x7f66e00d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:39.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.148+0000 7f66e8697640 1 -- 172.21.15.83:0/3055229603 shutdown_connections 2024-04-17T10:50:39.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.148+0000 7f66e8697640 1 -- 172.21.15.83:0/3055229603 wait complete. 2024-04-17T10:50:39.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.148+0000 7f66e8697640 1 Processor -- start 2024-04-17T10:50:39.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.148+0000 7f66e8697640 1 -- start start 2024-04-17T10:50:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.148+0000 7f66e8697640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66cc004810 0x7f66e00c74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.152+0000 7f66e8697640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66e00d3c10 0x7f66e00cd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.152+0000 7f66e8697640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e00d4090 0x7f66e01520f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.152+0000 7f66e8697640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e0169e50 con 0x7f66e00d4090 2024-04-17T10:50:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.152+0000 7f66e8697640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e0161b10 con 0x7f66cc004810 2024-04-17T10:50:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.152+0000 7f66e8697640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e0169fd0 con 0x7f66e00d3c10 2024-04-17T10:50:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.152+0000 7f66e6c34640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e00d4090 0x7f66e01520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.152+0000 7f66e6c34640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e00d4090 0x7f66e01520f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35360/0 (socket says 172.21.15.83:35360) 2024-04-17T10:50:39.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.152+0000 7f66e6c34640 1 -- 172.21.15.83:0/569901788 learned_addr learned my addr 172.21.15.83:0/569901788 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:39.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.152+0000 7f66e6433640 1 --2- 172.21.15.83:0/569901788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66cc004810 0x7f66e00c74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:39.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.152+0000 7f66e5c32640 1 --2- 172.21.15.83:0/569901788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66e00d3c10 0x7f66e00cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:39.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.152+0000 7f66e6c34640 1 -- 172.21.15.83:0/569901788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66e00d3c10 msgr2=0x7f66e00cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:39.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.152+0000 7f66e6c34640 1 --2- 172.21.15.83:0/569901788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66e00d3c10 0x7f66e00cd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:39.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.152+0000 7f66e6c34640 1 -- 172.21.15.83:0/569901788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66cc004810 msgr2=0x7f66e00c74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:39.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.152+0000 7f66e6c34640 1 --2- 172.21.15.83:0/569901788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66cc004810 0x7f66e00c74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:39.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.152+0000 7f66e6c34640 1 -- 172.21.15.83:0/569901788 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e00c7a00 con 0x7f66e00d4090 2024-04-17T10:50:39.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.152+0000 7f66e5c32640 1 --2- 172.21.15.83:0/569901788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66e00d3c10 0x7f66e00cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:39.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.152+0000 7f66e6c34640 1 --2- 172.21.15.83:0/569901788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e00d4090 0x7f66e01520f0 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f66e006dae0 tx=0x7f66d800b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:39.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.152+0000 7f66d6ffd640 1 -- 172.21.15.83:0/569901788 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d800b8d0 con 0x7f66e00d4090 2024-04-17T10:50:39.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.152+0000 7f66d6ffd640 1 -- 172.21.15.83:0/569901788 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d800ba70 con 0x7f66e00d4090 2024-04-17T10:50:39.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.152+0000 7f66e8697640 1 -- 172.21.15.83:0/569901788 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66e01534f0 con 0x7f66e00d4090 2024-04-17T10:50:39.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.152+0000 7f66d6ffd640 1 -- 172.21.15.83:0/569901788 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d800be60 con 0x7f66e00d4090 2024-04-17T10:50:39.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.152+0000 7f66e8697640 1 -- 172.21.15.83:0/569901788 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e01552a0 con 0x7f66e00d4090 2024-04-17T10:50:39.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.152+0000 7f66e8697640 1 -- 172.21.15.83:0/569901788 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66e00681a0 con 0x7f66e00d4090 2024-04-17T10:50:39.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.152+0000 7f66d6ffd640 1 -- 172.21.15.83:0/569901788 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f66d801b060 con 0x7f66e00d4090 2024-04-17T10:50:39.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.152+0000 7f66d6ffd640 1 --2- 172.21.15.83:0/569901788 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f66c4066f90 0x7f66c4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:39.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.152+0000 7f66e6433640 1 --2- 172.21.15.83:0/569901788 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f66c4066f90 0x7f66c4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:39.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.152+0000 7f66d6ffd640 1 -- 172.21.15.83:0/569901788 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f66d8084f20 con 0x7f66e00d4090 2024-04-17T10:50:39.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.152+0000 7f66e6433640 1 --2- 172.21.15.83:0/569901788 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f66c4066f90 0x7f66c4069450 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f66dc001760 tx=0x7f66dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:39.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.156+0000 7f66d6ffd640 1 -- 172.21.15.83:0/569901788 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f66e01534f0 con 0x7f66e00d4090 2024-04-17T10:50:39.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.304+0000 7f66e8697640 1 -- 172.21.15.83:0/569901788 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f66e00d1810 con 0x7f66c4066f90 2024-04-17T10:50:39.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.316+0000 7f66d6ffd640 1 -- 172.21.15.83:0/569901788 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247089 (secure 0 0 0) 0x7f66e00d1810 con 0x7f66c4066f90 2024-04-17T10:50:39.313 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:50:39.315 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:50:39.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.320+0000 7f66e8697640 1 -- 172.21.15.83:0/569901788 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f66c4066f90 msgr2=0x7f66c4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:39.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.320+0000 7f66e8697640 1 --2- 172.21.15.83:0/569901788 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f66c4066f90 0x7f66c4069450 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f66dc001760 tx=0x7f66dc009040 comp rx=0 tx=0).stop 2024-04-17T10:50:39.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.320+0000 7f66e8697640 1 -- 172.21.15.83:0/569901788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e00d4090 msgr2=0x7f66e01520f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:39.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.320+0000 7f66e8697640 1 --2- 172.21.15.83:0/569901788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e00d4090 0x7f66e01520f0 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f66e006dae0 tx=0x7f66d800b0d0 comp rx=0 tx=0).stop 2024-04-17T10:50:39.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.320+0000 7f66e8697640 1 -- 172.21.15.83:0/569901788 shutdown_connections 2024-04-17T10:50:39.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.320+0000 7f66e8697640 1 --2- 172.21.15.83:0/569901788 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f66c4066f90 0x7f66c4069450 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:39.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.320+0000 7f66e8697640 1 --2- 172.21.15.83:0/569901788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e00d4090 0x7f66e01520f0 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:39.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.320+0000 7f66e8697640 1 --2- 172.21.15.83:0/569901788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66e00d3c10 0x7f66e00cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:39.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.320+0000 7f66e8697640 1 --2- 172.21.15.83:0/569901788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66cc004810 0x7f66e00c74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:39.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.320+0000 7f66e8697640 1 -- 172.21.15.83:0/569901788 >> 172.21.15.83:0/569901788 conn(0x7f66e00d0600 msgr2=0x7f66e0164630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:39.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.320+0000 7f66e8697640 1 -- 172.21.15.83:0/569901788 shutdown_connections 2024-04-17T10:50:39.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:39.320+0000 7f66e8697640 1 -- 172.21.15.83:0/569901788 wait complete. 2024-04-17T10:50:39.328 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":513,"stamp":"2024-04-17T10:50:38.401239+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":494524,"kb_used_data":14592,"kb_used_omap":12,"kb_used_meta":479795,"kb_avail":754480196,"statfs":{"total":773094113280,"available":772587720704,"internally_reserved":0,"allocated":14942208,"data_stored":7189694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":491310640},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.545239"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.876422+0000","last_change":"2024-04-17T10:50:33.876422+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.876422+0000","last_undegraded":"2024-04-17T10:50:33.876422+0000","last_fullsized":"2024-04-17T10:50:33.876422+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.878304+0000","last_change":"2024-04-17T10:50:33.878304+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.878304+0000","last_undegraded":"2024-04-17T10:50:33.878304+0000","last_fullsized":"2024-04-17T10:50:33.878304+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.880142+0000","last_change":"2024-04-17T10:50:33.880142+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.880142+0000","last_undegraded":"2024-04-17T10:50:33.880142+0000","last_fullsized":"2024-04-17T10:50:33.880142+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.871442+0000","last_change":"2024-04-17T10:50:33.871442+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.871442+0000","last_undegraded":"2024-04-17T10:50:33.871442+0000","last_fullsized":"2024-04-17T10:50:33.871442+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.879260+0000","last_change":"2024-04-17T10:50:33.879260+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.879260+0000","last_undegraded":"2024-04-17T10:50:33.879260+0000","last_fullsized":"2024-04-17T10:50:33.879260+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.875526+0000","last_change":"2024-04-17T10:50:33.875526+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.875526+0000","last_undegraded":"2024-04-17T10:50:33.875526+0000","last_fullsized":"2024-04-17T10:50:33.875526+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.884342+0000","last_change":"2024-04-17T10:50:33.884342+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.884342+0000","last_undegraded":"2024-04-17T10:50:33.884342+0000","last_fullsized":"2024-04-17T10:50:33.884342+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.875967+0000","last_change":"2024-04-17T10:50:33.875967+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.875967+0000","last_undegraded":"2024-04-17T10:50:33.875967+0000","last_fullsized":"2024-04-17T10:50:33.875967+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.881528+0000","last_change":"2024-04-17T10:50:33.881528+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.881528+0000","last_undegraded":"2024-04-17T10:50:33.881528+0000","last_fullsized":"2024-04-17T10:50:33.881528+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.874184+0000","last_change":"2024-04-17T10:50:33.874184+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.874184+0000","last_undegraded":"2024-04-17T10:50:33.874184+0000","last_fullsized":"2024-04-17T10:50:33.874184+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.884228+0000","last_change":"2024-04-17T10:50:33.884228+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.884228+0000","last_undegraded":"2024-04-17T10:50:33.884228+0000","last_fullsized":"2024-04-17T10:50:33.884228+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.870313+0000","last_change":"2024-04-17T10:50:33.870313+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.870313+0000","last_undegraded":"2024-04-17T10:50:33.870313+0000","last_fullsized":"2024-04-17T10:50:33.870313+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.878811+0000","last_change":"2024-04-17T10:50:33.878811+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.878811+0000","last_undegraded":"2024-04-17T10:50:33.878811+0000","last_fullsized":"2024-04-17T10:50:33.878811+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.875112+0000","last_change":"2024-04-17T10:50:33.875112+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.875112+0000","last_undegraded":"2024-04-17T10:50:33.875112+0000","last_fullsized":"2024-04-17T10:50:33.875112+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.869759+0000","last_change":"2024-04-17T10:50:33.869759+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.869759+0000","last_undegraded":"2024-04-17T10:50:33.869759+0000","last_fullsized":"2024-04-17T10:50:33.869759+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.883895+0000","last_change":"2024-04-17T10:50:33.883895+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.883895+0000","last_undegraded":"2024-04-17T10:50:33.883895+0000","last_fullsized":"2024-04-17T10:50:33.883895+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.871401+0000","last_change":"2024-04-17T10:50:33.871401+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.871401+0000","last_undegraded":"2024-04-17T10:50:33.871401+0000","last_fullsized":"2024-04-17T10:50:33.871401+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.881276+0000","last_change":"2024-04-17T10:50:33.881276+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.881276+0000","last_undegraded":"2024-04-17T10:50:33.881276+0000","last_fullsized":"2024-04-17T10:50:33.881276+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.883559+0000","last_change":"2024-04-17T10:50:33.883559+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.883559+0000","last_undegraded":"2024-04-17T10:50:33.883559+0000","last_fullsized":"2024-04-17T10:50:33.883559+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.885482+0000","last_change":"2024-04-17T10:50:33.885482+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.885482+0000","last_undegraded":"2024-04-17T10:50:33.885482+0000","last_fullsized":"2024-04-17T10:50:33.885482+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.871386+0000","last_change":"2024-04-17T10:50:33.871386+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.871386+0000","last_undegraded":"2024-04-17T10:50:33.871386+0000","last_fullsized":"2024-04-17T10:50:33.871386+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.874084+0000","last_change":"2024-04-17T10:50:33.874084+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.874084+0000","last_undegraded":"2024-04-17T10:50:33.874084+0000","last_fullsized":"2024-04-17T10:50:33.874084+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.869757+0000","last_change":"2024-04-17T10:50:33.869757+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.869757+0000","last_undegraded":"2024-04-17T10:50:33.869757+0000","last_fullsized":"2024-04-17T10:50:33.869757+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.873283+0000","last_change":"2024-04-17T10:50:33.873283+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.873283+0000","last_undegraded":"2024-04-17T10:50:33.873283+0000","last_fullsized":"2024-04-17T10:50:33.873283+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.883856+0000","last_change":"2024-04-17T10:50:33.883856+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.883856+0000","last_undegraded":"2024-04-17T10:50:33.883856+0000","last_fullsized":"2024-04-17T10:50:33.883856+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.881781+0000","last_change":"2024-04-17T10:50:33.881781+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.881781+0000","last_undegraded":"2024-04-17T10:50:33.881781+0000","last_fullsized":"2024-04-17T10:50:33.881781+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.881470+0000","last_change":"2024-04-17T10:50:33.881470+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.881470+0000","last_undegraded":"2024-04-17T10:50:33.881470+0000","last_fullsized":"2024-04-17T10:50:33.881470+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.885926+0000","last_change":"2024-04-17T10:50:33.885926+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.885926+0000","last_undegraded":"2024-04-17T10:50:33.885926+0000","last_fullsized":"2024-04-17T10:50:33.885926+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.875431+0000","last_change":"2024-04-17T10:50:33.875431+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.875431+0000","last_undegraded":"2024-04-17T10:50:33.875431+0000","last_fullsized":"2024-04-17T10:50:33.875431+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.873345+0000","last_change":"2024-04-17T10:50:33.873345+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.873345+0000","last_undegraded":"2024-04-17T10:50:33.873345+0000","last_fullsized":"2024-04-17T10:50:33.873345+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.878027+0000","last_change":"2024-04-17T10:50:33.878027+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.878027+0000","last_undegraded":"2024-04-17T10:50:33.878027+0000","last_fullsized":"2024-04-17T10:50:33.878027+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.880017+0000","last_change":"2024-04-17T10:50:33.880017+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.880017+0000","last_undegraded":"2024-04-17T10:50:33.880017+0000","last_fullsized":"2024-04-17T10:50:33.880017+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.871860+0000","last_change":"2024-04-17T10:50:33.871860+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.871860+0000","last_undegraded":"2024-04-17T10:50:33.871860+0000","last_fullsized":"2024-04-17T10:50:33.871860+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.896574+0000","last_change":"2024-04-17T10:50:33.891540+0000","last_active":"2024-04-17T10:50:33.896574+0000","last_peered":"2024-04-17T10:50:33.896574+0000","last_clean":"2024-04-17T10:50:33.896574+0000","last_became_active":"2024-04-17T10:50:33.890837+0000","last_became_peered":"2024-04-17T10:50:33.890837+0000","last_unstale":"2024-04-17T10:50:33.896574+0000","last_undegraded":"2024-04-17T10:50:33.896574+0000","last_fullsized":"2024-04-17T10:50:33.896574+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:16:19.474043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.880475+0000","last_change":"2024-04-17T10:50:33.880475+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.880475+0000","last_undegraded":"2024-04-17T10:50:33.880475+0000","last_fullsized":"2024-04-17T10:50:33.880475+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.893863+0000","last_change":"2024-04-17T10:50:33.890842+0000","last_active":"2024-04-17T10:50:33.893863+0000","last_peered":"2024-04-17T10:50:33.893863+0000","last_clean":"2024-04-17T10:50:33.893863+0000","last_became_active":"2024-04-17T10:50:33.890472+0000","last_became_peered":"2024-04-17T10:50:33.890472+0000","last_unstale":"2024-04-17T10:50:33.893863+0000","last_undegraded":"2024-04-17T10:50:33.893863+0000","last_fullsized":"2024-04-17T10:50:33.893863+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:52:36.198523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.876197+0000","last_change":"2024-04-17T10:50:33.876197+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.876197+0000","last_undegraded":"2024-04-17T10:50:33.876197+0000","last_fullsized":"2024-04-17T10:50:33.876197+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.900479+0000","last_change":"2024-04-17T10:50:33.899787+0000","last_active":"2024-04-17T10:50:33.900479+0000","last_peered":"2024-04-17T10:50:33.900479+0000","last_clean":"2024-04-17T10:50:33.900479+0000","last_became_active":"2024-04-17T10:50:33.899273+0000","last_became_peered":"2024-04-17T10:50:33.899273+0000","last_unstale":"2024-04-17T10:50:33.900479+0000","last_undegraded":"2024-04-17T10:50:33.900479+0000","last_fullsized":"2024-04-17T10:50:33.900479+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:08:11.608039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.870653+0000","last_change":"2024-04-17T10:50:33.870653+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.870653+0000","last_undegraded":"2024-04-17T10:50:33.870653+0000","last_fullsized":"2024-04-17T10:50:33.870653+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.887943+0000","last_change":"2024-04-17T10:50:33.884473+0000","last_active":"2024-04-17T10:50:33.887943+0000","last_peered":"2024-04-17T10:50:33.887943+0000","last_clean":"2024-04-17T10:50:33.887943+0000","last_became_active":"2024-04-17T10:50:33.883898+0000","last_became_peered":"2024-04-17T10:50:33.883898+0000","last_unstale":"2024-04-17T10:50:33.887943+0000","last_undegraded":"2024-04-17T10:50:33.887943+0000","last_fullsized":"2024-04-17T10:50:33.887943+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:03:31.940294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.877422+0000","last_change":"2024-04-17T10:50:33.877422+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.877422+0000","last_undegraded":"2024-04-17T10:50:33.877422+0000","last_fullsized":"2024-04-17T10:50:33.877422+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.891469+0000","last_change":"2024-04-17T10:50:33.888560+0000","last_active":"2024-04-17T10:50:33.891469+0000","last_peered":"2024-04-17T10:50:33.891469+0000","last_clean":"2024-04-17T10:50:33.891469+0000","last_became_active":"2024-04-17T10:50:33.888155+0000","last_became_peered":"2024-04-17T10:50:33.888155+0000","last_unstale":"2024-04-17T10:50:33.891469+0000","last_undegraded":"2024-04-17T10:50:33.891469+0000","last_fullsized":"2024-04-17T10:50:33.891469+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:19:11.890360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.872397+0000","last_change":"2024-04-17T10:50:33.872397+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.872397+0000","last_undegraded":"2024-04-17T10:50:33.872397+0000","last_fullsized":"2024-04-17T10:50:33.872397+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.898062+0000","last_change":"2024-04-17T10:50:33.896476+0000","last_active":"2024-04-17T10:50:33.898062+0000","last_peered":"2024-04-17T10:50:33.898062+0000","last_clean":"2024-04-17T10:50:33.898062+0000","last_became_active":"2024-04-17T10:50:33.895863+0000","last_became_peered":"2024-04-17T10:50:33.895863+0000","last_unstale":"2024-04-17T10:50:33.898062+0000","last_undegraded":"2024-04-17T10:50:33.898062+0000","last_fullsized":"2024-04-17T10:50:33.898062+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:49:13.741700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.878606+0000","last_change":"2024-04-17T10:50:33.878606+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.878606+0000","last_undegraded":"2024-04-17T10:50:33.878606+0000","last_fullsized":"2024-04-17T10:50:33.878606+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.898833+0000","last_change":"2024-04-17T10:50:33.895471+0000","last_active":"2024-04-17T10:50:33.898833+0000","last_peered":"2024-04-17T10:50:33.898833+0000","last_clean":"2024-04-17T10:50:33.898833+0000","last_became_active":"2024-04-17T10:50:33.895069+0000","last_became_peered":"2024-04-17T10:50:33.895069+0000","last_unstale":"2024-04-17T10:50:33.898833+0000","last_undegraded":"2024-04-17T10:50:33.898833+0000","last_fullsized":"2024-04-17T10:50:33.898833+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:35:29.165208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.879870+0000","last_change":"2024-04-17T10:50:33.879870+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.879870+0000","last_undegraded":"2024-04-17T10:50:33.879870+0000","last_fullsized":"2024-04-17T10:50:33.879870+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.887894+0000","last_change":"2024-04-17T10:50:33.885177+0000","last_active":"2024-04-17T10:50:33.887894+0000","last_peered":"2024-04-17T10:50:33.887894+0000","last_clean":"2024-04-17T10:50:33.887894+0000","last_became_active":"2024-04-17T10:50:33.884763+0000","last_became_peered":"2024-04-17T10:50:33.884763+0000","last_unstale":"2024-04-17T10:50:33.887894+0000","last_undegraded":"2024-04-17T10:50:33.887894+0000","last_fullsized":"2024-04-17T10:50:33.887894+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:49.010628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.876872+0000","last_change":"2024-04-17T10:50:33.876872+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.876872+0000","last_undegraded":"2024-04-17T10:50:33.876872+0000","last_fullsized":"2024-04-17T10:50:33.876872+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.896605+0000","last_change":"2024-04-17T10:50:33.895658+0000","last_active":"2024-04-17T10:50:33.896605+0000","last_peered":"2024-04-17T10:50:33.896605+0000","last_clean":"2024-04-17T10:50:33.896605+0000","last_became_active":"2024-04-17T10:50:33.894228+0000","last_became_peered":"2024-04-17T10:50:33.894228+0000","last_unstale":"2024-04-17T10:50:33.896605+0000","last_undegraded":"2024-04-17T10:50:33.896605+0000","last_fullsized":"2024-04-17T10:50:33.896605+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:02:11.481439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.872072+0000","last_change":"2024-04-17T10:50:33.872072+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.872072+0000","last_undegraded":"2024-04-17T10:50:33.872072+0000","last_fullsized":"2024-04-17T10:50:33.872072+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.896231+0000","last_change":"2024-04-17T10:50:33.890039+0000","last_active":"2024-04-17T10:50:33.896231+0000","last_peered":"2024-04-17T10:50:33.896231+0000","last_clean":"2024-04-17T10:50:33.896231+0000","last_became_active":"2024-04-17T10:50:33.889635+0000","last_became_peered":"2024-04-17T10:50:33.889635+0000","last_unstale":"2024-04-17T10:50:33.896231+0000","last_undegraded":"2024-04-17T10:50:33.896231+0000","last_fullsized":"2024-04-17T10:50:33.896231+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:02:11.481439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.879906+0000","last_change":"2024-04-17T10:50:33.879906+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.879906+0000","last_undegraded":"2024-04-17T10:50:33.879906+0000","last_fullsized":"2024-04-17T10:50:33.879906+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.904390+0000","last_change":"2024-04-17T10:50:33.898934+0000","last_active":"2024-04-17T10:50:33.904390+0000","last_peered":"2024-04-17T10:50:33.904390+0000","last_clean":"2024-04-17T10:50:33.904390+0000","last_became_active":"2024-04-17T10:50:33.897331+0000","last_became_peered":"2024-04-17T10:50:33.897331+0000","last_unstale":"2024-04-17T10:50:33.904390+0000","last_undegraded":"2024-04-17T10:50:33.904390+0000","last_fullsized":"2024-04-17T10:50:33.904390+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:02:11.481439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.881117+0000","last_change":"2024-04-17T10:50:33.881117+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.881117+0000","last_undegraded":"2024-04-17T10:50:33.881117+0000","last_fullsized":"2024-04-17T10:50:33.881117+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.901174+0000","last_change":"2024-04-17T10:50:33.900576+0000","last_active":"2024-04-17T10:50:33.901174+0000","last_peered":"2024-04-17T10:50:33.901174+0000","last_clean":"2024-04-17T10:50:33.901174+0000","last_became_active":"2024-04-17T10:50:33.899933+0000","last_became_peered":"2024-04-17T10:50:33.899933+0000","last_unstale":"2024-04-17T10:50:33.901174+0000","last_undegraded":"2024-04-17T10:50:33.901174+0000","last_fullsized":"2024-04-17T10:50:33.901174+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:08:11.608039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.877267+0000","last_change":"2024-04-17T10:50:33.877267+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.877267+0000","last_undegraded":"2024-04-17T10:50:33.877267+0000","last_fullsized":"2024-04-17T10:50:33.877267+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.897145+0000","last_change":"2024-04-17T10:50:33.890641+0000","last_active":"2024-04-17T10:50:33.897145+0000","last_peered":"2024-04-17T10:50:33.897145+0000","last_clean":"2024-04-17T10:50:33.897145+0000","last_became_active":"2024-04-17T10:50:33.889972+0000","last_became_peered":"2024-04-17T10:50:33.889972+0000","last_unstale":"2024-04-17T10:50:33.897145+0000","last_undegraded":"2024-04-17T10:50:33.897145+0000","last_fullsized":"2024-04-17T10:50:33.897145+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:29:06.526982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.875628+0000","last_change":"2024-04-17T10:50:33.875628+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.875628+0000","last_undegraded":"2024-04-17T10:50:33.875628+0000","last_fullsized":"2024-04-17T10:50:33.875628+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.899430+0000","last_change":"2024-04-17T10:50:33.897839+0000","last_active":"2024-04-17T10:50:33.899430+0000","last_peered":"2024-04-17T10:50:33.899430+0000","last_clean":"2024-04-17T10:50:33.899430+0000","last_became_active":"2024-04-17T10:50:33.897168+0000","last_became_peered":"2024-04-17T10:50:33.897168+0000","last_unstale":"2024-04-17T10:50:33.899430+0000","last_undegraded":"2024-04-17T10:50:33.899430+0000","last_fullsized":"2024-04-17T10:50:33.899430+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:25:56.552880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.878461+0000","last_change":"2024-04-17T10:50:33.878461+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.878461+0000","last_undegraded":"2024-04-17T10:50:33.878461+0000","last_fullsized":"2024-04-17T10:50:33.878461+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.899185+0000","last_change":"2024-04-17T10:50:33.895104+0000","last_active":"2024-04-17T10:50:33.899185+0000","last_peered":"2024-04-17T10:50:33.899185+0000","last_clean":"2024-04-17T10:50:33.899185+0000","last_became_active":"2024-04-17T10:50:33.894652+0000","last_became_peered":"2024-04-17T10:50:33.894652+0000","last_unstale":"2024-04-17T10:50:33.899185+0000","last_undegraded":"2024-04-17T10:50:33.899185+0000","last_fullsized":"2024-04-17T10:50:33.899185+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:17:40.737496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.871177+0000","last_change":"2024-04-17T10:50:33.871177+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.871177+0000","last_undegraded":"2024-04-17T10:50:33.871177+0000","last_fullsized":"2024-04-17T10:50:33.871177+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.894016+0000","last_change":"2024-04-17T10:50:33.892680+0000","last_active":"2024-04-17T10:50:33.894016+0000","last_peered":"2024-04-17T10:50:33.894016+0000","last_clean":"2024-04-17T10:50:33.894016+0000","last_became_active":"2024-04-17T10:50:33.891996+0000","last_became_peered":"2024-04-17T10:50:33.891996+0000","last_unstale":"2024-04-17T10:50:33.894016+0000","last_undegraded":"2024-04-17T10:50:33.894016+0000","last_fullsized":"2024-04-17T10:50:33.894016+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:36.242473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.880302+0000","last_change":"2024-04-17T10:50:33.880302+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.880302+0000","last_undegraded":"2024-04-17T10:50:33.880302+0000","last_fullsized":"2024-04-17T10:50:33.880302+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.899936+0000","last_change":"2024-04-17T10:50:33.898138+0000","last_active":"2024-04-17T10:50:33.899936+0000","last_peered":"2024-04-17T10:50:33.899936+0000","last_clean":"2024-04-17T10:50:33.899936+0000","last_became_active":"2024-04-17T10:50:33.896506+0000","last_became_peered":"2024-04-17T10:50:33.896506+0000","last_unstale":"2024-04-17T10:50:33.899936+0000","last_undegraded":"2024-04-17T10:50:33.899936+0000","last_fullsized":"2024-04-17T10:50:33.899936+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:35:46.656830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.880316+0000","last_change":"2024-04-17T10:50:33.880316+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.880316+0000","last_undegraded":"2024-04-17T10:50:33.880316+0000","last_fullsized":"2024-04-17T10:50:33.880316+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.890096+0000","last_change":"2024-04-17T10:50:33.887380+0000","last_active":"2024-04-17T10:50:33.890096+0000","last_peered":"2024-04-17T10:50:33.890096+0000","last_clean":"2024-04-17T10:50:33.890096+0000","last_became_active":"2024-04-17T10:50:33.887003+0000","last_became_peered":"2024-04-17T10:50:33.887003+0000","last_unstale":"2024-04-17T10:50:33.890096+0000","last_undegraded":"2024-04-17T10:50:33.890096+0000","last_fullsized":"2024-04-17T10:50:33.890096+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:25:00.410077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.877105+0000","last_change":"2024-04-17T10:50:33.877105+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.877105+0000","last_undegraded":"2024-04-17T10:50:33.877105+0000","last_fullsized":"2024-04-17T10:50:33.877105+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.893790+0000","last_change":"2024-04-17T10:50:33.890628+0000","last_active":"2024-04-17T10:50:33.893790+0000","last_peered":"2024-04-17T10:50:33.893790+0000","last_clean":"2024-04-17T10:50:33.893790+0000","last_became_active":"2024-04-17T10:50:33.890232+0000","last_became_peered":"2024-04-17T10:50:33.890232+0000","last_unstale":"2024-04-17T10:50:33.893790+0000","last_undegraded":"2024-04-17T10:50:33.893790+0000","last_fullsized":"2024-04-17T10:50:33.893790+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:55.719217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.872686+0000","last_change":"2024-04-17T10:50:33.872686+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.872686+0000","last_undegraded":"2024-04-17T10:50:33.872686+0000","last_fullsized":"2024-04-17T10:50:33.872686+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.904423+0000","last_change":"2024-04-17T10:50:33.901531+0000","last_active":"2024-04-17T10:50:33.904423+0000","last_peered":"2024-04-17T10:50:33.904423+0000","last_clean":"2024-04-17T10:50:33.904423+0000","last_became_active":"2024-04-17T10:50:33.901224+0000","last_became_peered":"2024-04-17T10:50:33.901224+0000","last_unstale":"2024-04-17T10:50:33.904423+0000","last_undegraded":"2024-04-17T10:50:33.904423+0000","last_fullsized":"2024-04-17T10:50:33.904423+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:16:19.474043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.891453+0000","last_change":"2024-04-17T10:50:33.891453+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.891453+0000","last_undegraded":"2024-04-17T10:50:33.891453+0000","last_fullsized":"2024-04-17T10:50:33.891453+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.896430+0000","last_change":"2024-04-17T10:50:33.891385+0000","last_active":"2024-04-17T10:50:33.896430+0000","last_peered":"2024-04-17T10:50:33.896430+0000","last_clean":"2024-04-17T10:50:33.896430+0000","last_became_active":"2024-04-17T10:50:33.890896+0000","last_became_peered":"2024-04-17T10:50:33.890896+0000","last_unstale":"2024-04-17T10:50:33.896430+0000","last_undegraded":"2024-04-17T10:50:33.896430+0000","last_fullsized":"2024-04-17T10:50:33.896430+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:35:46.656830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.870668+0000","last_change":"2024-04-17T10:50:33.870668+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.870668+0000","last_undegraded":"2024-04-17T10:50:33.870668+0000","last_fullsized":"2024-04-17T10:50:33.870668+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.896755+0000","last_change":"2024-04-17T10:50:33.893344+0000","last_active":"2024-04-17T10:50:33.896755+0000","last_peered":"2024-04-17T10:50:33.896755+0000","last_clean":"2024-04-17T10:50:33.896755+0000","last_became_active":"2024-04-17T10:50:33.892680+0000","last_became_peered":"2024-04-17T10:50:33.892680+0000","last_unstale":"2024-04-17T10:50:33.896755+0000","last_undegraded":"2024-04-17T10:50:33.896755+0000","last_fullsized":"2024-04-17T10:50:33.896755+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:01:25.099953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.884791+0000","last_change":"2024-04-17T10:50:33.884791+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.884791+0000","last_undegraded":"2024-04-17T10:50:33.884791+0000","last_fullsized":"2024-04-17T10:50:33.884791+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.900021+0000","last_change":"2024-04-17T10:50:33.897362+0000","last_active":"2024-04-17T10:50:33.900021+0000","last_peered":"2024-04-17T10:50:33.900021+0000","last_clean":"2024-04-17T10:50:33.900021+0000","last_became_active":"2024-04-17T10:50:33.896738+0000","last_became_peered":"2024-04-17T10:50:33.896738+0000","last_unstale":"2024-04-17T10:50:33.900021+0000","last_undegraded":"2024-04-17T10:50:33.900021+0000","last_fullsized":"2024-04-17T10:50:33.900021+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:49.010628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.875977+0000","last_change":"2024-04-17T10:50:33.875977+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.875977+0000","last_undegraded":"2024-04-17T10:50:33.875977+0000","last_fullsized":"2024-04-17T10:50:33.875977+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.896611+0000","last_change":"2024-04-17T10:50:33.893658+0000","last_active":"2024-04-17T10:50:33.896611+0000","last_peered":"2024-04-17T10:50:33.896611+0000","last_clean":"2024-04-17T10:50:33.896611+0000","last_became_active":"2024-04-17T10:50:33.893023+0000","last_became_peered":"2024-04-17T10:50:33.893023+0000","last_unstale":"2024-04-17T10:50:33.896611+0000","last_undegraded":"2024-04-17T10:50:33.896611+0000","last_fullsized":"2024-04-17T10:50:33.896611+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:48:29.238257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.880974+0000","last_change":"2024-04-17T10:50:33.880974+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.880974+0000","last_undegraded":"2024-04-17T10:50:33.880974+0000","last_fullsized":"2024-04-17T10:50:33.880974+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.892795+0000","last_change":"2024-04-17T10:50:33.887529+0000","last_active":"2024-04-17T10:50:33.892795+0000","last_peered":"2024-04-17T10:50:33.892795+0000","last_clean":"2024-04-17T10:50:33.892795+0000","last_became_active":"2024-04-17T10:50:33.887152+0000","last_became_peered":"2024-04-17T10:50:33.887152+0000","last_unstale":"2024-04-17T10:50:33.892795+0000","last_undegraded":"2024-04-17T10:50:33.892795+0000","last_fullsized":"2024-04-17T10:50:33.892795+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:04.970052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.878592+0000","last_change":"2024-04-17T10:50:33.878592+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.878592+0000","last_undegraded":"2024-04-17T10:50:33.878592+0000","last_fullsized":"2024-04-17T10:50:33.878592+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.892986+0000","last_change":"2024-04-17T10:50:33.889203+0000","last_active":"2024-04-17T10:50:33.892986+0000","last_peered":"2024-04-17T10:50:33.892986+0000","last_clean":"2024-04-17T10:50:33.892986+0000","last_became_active":"2024-04-17T10:50:33.888312+0000","last_became_peered":"2024-04-17T10:50:33.888312+0000","last_unstale":"2024-04-17T10:50:33.892986+0000","last_undegraded":"2024-04-17T10:50:33.892986+0000","last_fullsized":"2024-04-17T10:50:33.892986+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:23:11.975227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.876607+0000","last_change":"2024-04-17T10:50:33.876607+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.876607+0000","last_undegraded":"2024-04-17T10:50:33.876607+0000","last_fullsized":"2024-04-17T10:50:33.876607+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.892508+0000","last_change":"2024-04-17T10:50:33.882648+0000","last_active":"2024-04-17T10:50:33.892508+0000","last_peered":"2024-04-17T10:50:33.892508+0000","last_clean":"2024-04-17T10:50:33.892508+0000","last_became_active":"2024-04-17T10:50:33.882297+0000","last_became_peered":"2024-04-17T10:50:33.882297+0000","last_unstale":"2024-04-17T10:50:33.892508+0000","last_undegraded":"2024-04-17T10:50:33.892508+0000","last_fullsized":"2024-04-17T10:50:33.892508+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:29:30.703224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.879901+0000","last_change":"2024-04-17T10:50:33.879901+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.879901+0000","last_undegraded":"2024-04-17T10:50:33.879901+0000","last_fullsized":"2024-04-17T10:50:33.879901+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.904388+0000","last_change":"2024-04-17T10:50:33.900315+0000","last_active":"2024-04-17T10:50:33.904388+0000","last_peered":"2024-04-17T10:50:33.904388+0000","last_clean":"2024-04-17T10:50:33.904388+0000","last_became_active":"2024-04-17T10:50:33.899987+0000","last_became_peered":"2024-04-17T10:50:33.899987+0000","last_unstale":"2024-04-17T10:50:33.904388+0000","last_undegraded":"2024-04-17T10:50:33.904388+0000","last_fullsized":"2024-04-17T10:50:33.904388+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:35:46.656830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.878334+0000","last_change":"2024-04-17T10:50:33.878334+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.878334+0000","last_undegraded":"2024-04-17T10:50:33.878334+0000","last_fullsized":"2024-04-17T10:50:33.878334+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.896788+0000","last_change":"2024-04-17T10:50:33.894342+0000","last_active":"2024-04-17T10:50:33.896788+0000","last_peered":"2024-04-17T10:50:33.896788+0000","last_clean":"2024-04-17T10:50:33.896788+0000","last_became_active":"2024-04-17T10:50:33.893720+0000","last_became_peered":"2024-04-17T10:50:33.893720+0000","last_unstale":"2024-04-17T10:50:33.896788+0000","last_undegraded":"2024-04-17T10:50:33.896788+0000","last_fullsized":"2024-04-17T10:50:33.896788+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:21:59.523164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.869757+0000","last_change":"2024-04-17T10:50:33.869757+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.869757+0000","last_undegraded":"2024-04-17T10:50:33.869757+0000","last_fullsized":"2024-04-17T10:50:33.869757+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.898205+0000","last_change":"2024-04-17T10:50:33.895914+0000","last_active":"2024-04-17T10:50:33.898205+0000","last_peered":"2024-04-17T10:50:33.898205+0000","last_clean":"2024-04-17T10:50:33.898205+0000","last_became_active":"2024-04-17T10:50:33.893274+0000","last_became_peered":"2024-04-17T10:50:33.893274+0000","last_unstale":"2024-04-17T10:50:33.898205+0000","last_undegraded":"2024-04-17T10:50:33.898205+0000","last_fullsized":"2024-04-17T10:50:33.898205+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:44:02.588052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.877441+0000","last_change":"2024-04-17T10:50:33.877441+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.877441+0000","last_undegraded":"2024-04-17T10:50:33.877441+0000","last_fullsized":"2024-04-17T10:50:33.877441+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.896924+0000","last_change":"2024-04-17T10:50:33.893329+0000","last_active":"2024-04-17T10:50:33.896924+0000","last_peered":"2024-04-17T10:50:33.896924+0000","last_clean":"2024-04-17T10:50:33.896924+0000","last_became_active":"2024-04-17T10:50:33.892918+0000","last_became_peered":"2024-04-17T10:50:33.892918+0000","last_unstale":"2024-04-17T10:50:33.896924+0000","last_undegraded":"2024-04-17T10:50:33.896924+0000","last_fullsized":"2024-04-17T10:50:33.896924+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:43:18.553237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-17T10:50:33.885117+0000","last_change":"2024-04-17T10:50:33.885117+0000","last_active":"2024-04-17T10:50:33.854740+0000","last_peered":"2024-04-17T10:50:33.854740+0000","last_clean":"2024-04-17T10:50:33.854740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T10:50:33.885117+0000","last_undegraded":"2024-04-17T10:50:33.885117+0000","last_fullsized":"2024-04-17T10:50:33.885117+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.898263+0000","last_change":"2024-04-17T10:50:33.896124+0000","last_active":"2024-04-17T10:50:33.898263+0000","last_peered":"2024-04-17T10:50:33.898263+0000","last_clean":"2024-04-17T10:50:33.898263+0000","last_became_active":"2024-04-17T10:50:33.895638+0000","last_became_peered":"2024-04-17T10:50:33.895638+0000","last_unstale":"2024-04-17T10:50:33.898263+0000","last_undegraded":"2024-04-17T10:50:33.898263+0000","last_fullsized":"2024-04-17T10:50:33.898263+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:03:31.940294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":412,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-17T10:50:33.876578+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:50:33.876578+0000","last_peered":"2024-04-17T10:50:33.876578+0000","last_clean":"2024-04-17T10:50:33.876578+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:50:33.876578+0000","last_undegraded":"2024-04-17T10:50:33.876578+0000","last_fullsized":"2024-04-17T10:50:33.876578+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575003,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47324,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324516,"statfs":{"total":96636764160,"available":96588304384,"internally_reserved":0,"allocated":2310144,"data_stored":1339919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575002,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68952,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302888,"statfs":{"total":96636764160,"available":96566157312,"internally_reserved":0,"allocated":1716224,"data_stored":749551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575003,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55080,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316760,"statfs":{"total":96636764160,"available":96580362240,"internally_reserved":0,"allocated":1728512,"data_stored":755826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575002,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73124,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298716,"statfs":{"total":96636764160,"available":96561885184,"internally_reserved":0,"allocated":1716224,"data_stored":749551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575004,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73112,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298728,"statfs":{"total":96636764160,"available":96561897472,"internally_reserved":0,"allocated":1716224,"data_stored":749551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575002,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46748,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325092,"statfs":{"total":96636764160,"available":96588894208,"internally_reserved":0,"allocated":1716224,"data_stored":749551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575002,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82848,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288992,"statfs":{"total":96636764160,"available":96551927808,"internally_reserved":0,"allocated":1716224,"data_stored":749551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575002,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47336,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324504,"statfs":{"total":96636764160,"available":96588292096,"internally_reserved":0,"allocated":2322432,"data_stored":1346194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:50:40.846 INFO:tasks.ceph.mds.a.smithi083.stderr:2024-04-17T10:50:40.852+0000 7fe08f24a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T10:50:42.333 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:50:42.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.472+0000 7f9dabb42640 1 Processor -- start 2024-04-17T10:50:42.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.472+0000 7f9dabb42640 1 -- start start 2024-04-17T10:50:42.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.472+0000 7f9dabb42640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9da415ddd0 0x7f9da40d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:42.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.472+0000 7f9dabb42640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9da415e1b0 0x7f9da40d2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:42.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.472+0000 7f9dabb42640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da40d1940 con 0x7f9da40b2b90 2024-04-17T10:50:42.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.472+0000 7f9dabb42640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9da40cfc70 con 0x7f9da415ddd0 2024-04-17T10:50:42.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.472+0000 7f9dabb42640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9da40d1c20 con 0x7f9da415e1b0 2024-04-17T10:50:42.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.472+0000 7f9da98de640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9da415ddd0 0x7f9da40d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:42.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.472+0000 7f9daa0df640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9da40b2b90 0x7f9da40ce730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57192/0 (socket says 172.21.15.83:57192) 2024-04-17T10:50:42.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.472+0000 7f9daa0df640 1 -- 172.21.15.83:0/2630135899 learned_addr learned my addr 172.21.15.83:0/2630135899 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:42.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.472+0000 7f9da98de640 1 -- 172.21.15.83:0/2630135899 >> v2:172.21.15.167:3300/0 conn(0x7f9da415e1b0 msgr2=0x7f9da40d2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:42.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.472+0000 7f9da98de640 1 --2- 172.21.15.83:0/2630135899 >> v2:172.21.15.167:3300/0 conn(0x7f9da415e1b0 0x7f9da40d2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:42.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.472+0000 7f9da98de640 1 -- 172.21.15.83:0/2630135899 >> v1:172.21.15.83:6789/0 conn(0x7f9da40b2b90 legacy=0x7f9da40ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:42.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.472+0000 7f9da98de640 1 -- 172.21.15.83:0/2630135899 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da40cf900 con 0x7f9da415ddd0 2024-04-17T10:50:42.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9da98de640 1 --2- 172.21.15.83:0/2630135899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9da415ddd0 0x7f9da40d2020 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f9da40732f0 tx=0x7f9da001c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170684227d8032be server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:42.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9da88dc640 1 -- 172.21.15.83:0/2630135899 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da00b1070 con 0x7f9da415ddd0 2024-04-17T10:50:42.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9da88dc640 1 -- 172.21.15.83:0/2630135899 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da00a4880 con 0x7f9da415ddd0 2024-04-17T10:50:42.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9da88dc640 1 -- 172.21.15.83:0/2630135899 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da00a4c50 con 0x7f9da415ddd0 2024-04-17T10:50:42.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9dabb42640 1 -- 172.21.15.83:0/2630135899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9da415ddd0 msgr2=0x7f9da40d2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:42.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9dabb42640 1 --2- 172.21.15.83:0/2630135899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9da415ddd0 0x7f9da40d2020 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f9da40732f0 tx=0x7f9da001c040 comp rx=0 tx=0).stop 2024-04-17T10:50:42.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9dabb42640 1 -- 172.21.15.83:0/2630135899 shutdown_connections 2024-04-17T10:50:42.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9dabb42640 1 --2- 172.21.15.83:0/2630135899 >> v2:172.21.15.167:3300/0 conn(0x7f9da415e1b0 0x7f9da40d2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:42.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9dabb42640 1 --2- 172.21.15.83:0/2630135899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9da415ddd0 0x7f9da40d2020 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:42.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9dabb42640 1 -- 172.21.15.83:0/2630135899 >> 172.21.15.83:0/2630135899 conn(0x7f9da40d3a40 msgr2=0x7f9da40d3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:42.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9dabb42640 1 -- 172.21.15.83:0/2630135899 shutdown_connections 2024-04-17T10:50:42.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9dabb42640 1 -- 172.21.15.83:0/2630135899 wait complete. 2024-04-17T10:50:42.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9dabb42640 1 Processor -- start 2024-04-17T10:50:42.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9dabb42640 1 -- start start 2024-04-17T10:50:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9dabb42640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9da40b2b90 0x7f9da40b85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9dabb42640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9da415ddd0 0x7f9da40b7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9dabb42640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9da415e1b0 0x7f9da40b8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9dabb42640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da416bb70 con 0x7f9da415ddd0 2024-04-17T10:50:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9dabb42640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da416b9f0 con 0x7f9da40b2b90 2024-04-17T10:50:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9dabb42640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9da416bcf0 con 0x7f9da415e1b0 2024-04-17T10:50:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9da90dd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9da415ddd0 0x7f9da40b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9da90dd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9da415ddd0 0x7f9da40b7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35378/0 (socket says 172.21.15.83:35378) 2024-04-17T10:50:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9da90dd640 1 -- 172.21.15.83:0/244388806 learned_addr learned my addr 172.21.15.83:0/244388806 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9da98de640 1 --2- 172.21.15.83:0/244388806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9da40b2b90 0x7f9da40b85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9daa0df640 1 --2- 172.21.15.83:0/244388806 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9da415e1b0 0x7f9da40b8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:42.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9da90dd640 1 -- 172.21.15.83:0/244388806 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9da415e1b0 msgr2=0x7f9da40b8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:42.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9da90dd640 1 --2- 172.21.15.83:0/244388806 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9da415e1b0 0x7f9da40b8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:42.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9da90dd640 1 -- 172.21.15.83:0/244388806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9da40b2b90 msgr2=0x7f9da40b85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:42.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9da90dd640 1 --2- 172.21.15.83:0/244388806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9da40b2b90 0x7f9da40b85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:42.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9da90dd640 1 -- 172.21.15.83:0/244388806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da40b5990 con 0x7f9da415ddd0 2024-04-17T10:50:42.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9da98de640 1 --2- 172.21.15.83:0/244388806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9da40b2b90 0x7f9da40b85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:42.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9daa0df640 1 --2- 172.21.15.83:0/244388806 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9da415e1b0 0x7f9da40b8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:42.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.476+0000 7f9da90dd640 1 --2- 172.21.15.83:0/244388806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9da415ddd0 0x7f9da40b7600 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400b440 tx=0x7f9d9400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:42.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.480+0000 7f9d9a7fc640 1 -- 172.21.15.83:0/244388806 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d94015020 con 0x7f9da415ddd0 2024-04-17T10:50:42.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.480+0000 7f9d9a7fc640 1 -- 172.21.15.83:0/244388806 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d940118b0 con 0x7f9da415ddd0 2024-04-17T10:50:42.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.480+0000 7f9dabb42640 1 -- 172.21.15.83:0/244388806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9da40b5c70 con 0x7f9da415ddd0 2024-04-17T10:50:42.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.480+0000 7f9d9a7fc640 1 -- 172.21.15.83:0/244388806 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d94011ca0 con 0x7f9da415ddd0 2024-04-17T10:50:42.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.480+0000 7f9dabb42640 1 -- 172.21.15.83:0/244388806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da40b6230 con 0x7f9da415ddd0 2024-04-17T10:50:42.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.480+0000 7f9dabb42640 1 -- 172.21.15.83:0/244388806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9da40681a0 con 0x7f9da415ddd0 2024-04-17T10:50:42.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.480+0000 7f9d9a7fc640 1 -- 172.21.15.83:0/244388806 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9d94018020 con 0x7f9da415ddd0 2024-04-17T10:50:42.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.480+0000 7f9d9a7fc640 1 --2- 172.21.15.83:0/244388806 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9d88066f90 0x7f9d88069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:42.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.480+0000 7f9da98de640 1 --2- 172.21.15.83:0/244388806 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9d88066f90 0x7f9d88069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:42.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.480+0000 7f9d9a7fc640 1 -- 172.21.15.83:0/244388806 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9d94085110 con 0x7f9da415ddd0 2024-04-17T10:50:42.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.480+0000 7f9da98de640 1 --2- 172.21.15.83:0/244388806 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9d88066f90 0x7f9d88069450 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f9da00ae000 tx=0x7f9da0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:42.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.484+0000 7f9d9a7fc640 1 -- 172.21.15.83:0/244388806 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9d9401f020 con 0x7f9da415ddd0 2024-04-17T10:50:42.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.640+0000 7f9dabb42640 1 -- 172.21.15.83:0/244388806 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9da40b7b40 con 0x7f9d88066f90 2024-04-17T10:50:42.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.648+0000 7f9d9a7fc640 1 -- 172.21.15.83:0/244388806 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252492 (secure 0 0 0) 0x7f9da40b7b40 con 0x7f9d88066f90 2024-04-17T10:50:42.645 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:50:42.647 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:50:42.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.656+0000 7f9dabb42640 1 -- 172.21.15.83:0/244388806 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9d88066f90 msgr2=0x7f9d88069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:42.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.656+0000 7f9dabb42640 1 --2- 172.21.15.83:0/244388806 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9d88066f90 0x7f9d88069450 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f9da00ae000 tx=0x7f9da0008040 comp rx=0 tx=0).stop 2024-04-17T10:50:42.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.656+0000 7f9dabb42640 1 -- 172.21.15.83:0/244388806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9da415ddd0 msgr2=0x7f9da40b7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:42.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.656+0000 7f9dabb42640 1 --2- 172.21.15.83:0/244388806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9da415ddd0 0x7f9da40b7600 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400b440 tx=0x7f9d9400b900 comp rx=0 tx=0).stop 2024-04-17T10:50:42.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.656+0000 7f9dabb42640 1 -- 172.21.15.83:0/244388806 shutdown_connections 2024-04-17T10:50:42.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.656+0000 7f9dabb42640 1 --2- 172.21.15.83:0/244388806 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9d88066f90 0x7f9d88069450 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:42.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.656+0000 7f9dabb42640 1 --2- 172.21.15.83:0/244388806 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9da415e1b0 0x7f9da40b8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:42.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.656+0000 7f9dabb42640 1 --2- 172.21.15.83:0/244388806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9da415ddd0 0x7f9da40b7600 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:42.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.656+0000 7f9dabb42640 1 --2- 172.21.15.83:0/244388806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9da40b2b90 0x7f9da40b85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:42.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.656+0000 7f9dabb42640 1 -- 172.21.15.83:0/244388806 >> 172.21.15.83:0/244388806 conn(0x7f9da40d3a40 msgr2=0x7f9da41554d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:42.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.656+0000 7f9dabb42640 1 -- 172.21.15.83:0/244388806 shutdown_connections 2024-04-17T10:50:42.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.656+0000 7f9dabb42640 1 -- 172.21.15.83:0/244388806 wait complete. 2024-04-17T10:50:42.665 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":515,"stamp":"2024-04-17T10:50:42.402618+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":494840,"kb_used_data":14920,"kb_used_omap":12,"kb_used_meta":479795,"kb_avail":754479880,"statfs":{"total":773094113280,"available":772587397120,"internally_reserved":0,"allocated":15278080,"data_stored":7336764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":491310640},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002613"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.917451+0000","last_change":"2024-04-17T10:50:34.911673+0000","last_active":"2024-04-17T10:50:35.917451+0000","last_peered":"2024-04-17T10:50:35.917451+0000","last_clean":"2024-04-17T10:50:35.917451+0000","last_became_active":"2024-04-17T10:50:34.911305+0000","last_became_peered":"2024-04-17T10:50:34.911305+0000","last_unstale":"2024-04-17T10:50:35.917451+0000","last_undegraded":"2024-04-17T10:50:35.917451+0000","last_fullsized":"2024-04-17T10:50:35.917451+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:10:49.616962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.024549+0000","last_change":"2024-04-17T10:50:34.895926+0000","last_active":"2024-04-17T10:50:36.024549+0000","last_peered":"2024-04-17T10:50:36.024549+0000","last_clean":"2024-04-17T10:50:36.024549+0000","last_became_active":"2024-04-17T10:50:34.895540+0000","last_became_peered":"2024-04-17T10:50:34.895540+0000","last_unstale":"2024-04-17T10:50:36.024549+0000","last_undegraded":"2024-04-17T10:50:36.024549+0000","last_fullsized":"2024-04-17T10:50:36.024549+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:52:34.754522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.958670+0000","last_change":"2024-04-17T10:50:34.894302+0000","last_active":"2024-04-17T10:50:36.958670+0000","last_peered":"2024-04-17T10:50:36.958670+0000","last_clean":"2024-04-17T10:50:36.958670+0000","last_became_active":"2024-04-17T10:50:34.893514+0000","last_became_peered":"2024-04-17T10:50:34.893514+0000","last_unstale":"2024-04-17T10:50:36.958670+0000","last_undegraded":"2024-04-17T10:50:36.958670+0000","last_fullsized":"2024-04-17T10:50:36.958670+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:26:15.444880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.918935+0000","last_change":"2024-04-17T10:50:34.933670+0000","last_active":"2024-04-17T10:50:35.918935+0000","last_peered":"2024-04-17T10:50:35.918935+0000","last_clean":"2024-04-17T10:50:35.918935+0000","last_became_active":"2024-04-17T10:50:34.930335+0000","last_became_peered":"2024-04-17T10:50:34.930335+0000","last_unstale":"2024-04-17T10:50:35.918935+0000","last_undegraded":"2024-04-17T10:50:35.918935+0000","last_fullsized":"2024-04-17T10:50:35.918935+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:19:45.217622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.957867+0000","last_change":"2024-04-17T10:50:34.910616+0000","last_active":"2024-04-17T10:50:36.957867+0000","last_peered":"2024-04-17T10:50:36.957867+0000","last_clean":"2024-04-17T10:50:36.957867+0000","last_became_active":"2024-04-17T10:50:34.909610+0000","last_became_peered":"2024-04-17T10:50:34.909610+0000","last_unstale":"2024-04-17T10:50:36.957867+0000","last_undegraded":"2024-04-17T10:50:36.957867+0000","last_fullsized":"2024-04-17T10:50:36.957867+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:33:04.600317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.917157+0000","last_change":"2024-04-17T10:50:34.907319+0000","last_active":"2024-04-17T10:50:35.917157+0000","last_peered":"2024-04-17T10:50:35.917157+0000","last_clean":"2024-04-17T10:50:35.917157+0000","last_became_active":"2024-04-17T10:50:34.906608+0000","last_became_peered":"2024-04-17T10:50:34.906608+0000","last_unstale":"2024-04-17T10:50:35.917157+0000","last_undegraded":"2024-04-17T10:50:35.917157+0000","last_fullsized":"2024-04-17T10:50:35.917157+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:21:02.246547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.958819+0000","last_change":"2024-04-17T10:50:34.917487+0000","last_active":"2024-04-17T10:50:36.958819+0000","last_peered":"2024-04-17T10:50:36.958819+0000","last_clean":"2024-04-17T10:50:36.958819+0000","last_became_active":"2024-04-17T10:50:34.917180+0000","last_became_peered":"2024-04-17T10:50:34.917180+0000","last_unstale":"2024-04-17T10:50:36.958819+0000","last_undegraded":"2024-04-17T10:50:36.958819+0000","last_fullsized":"2024-04-17T10:50:36.958819+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:43:13.249127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.916370+0000","last_change":"2024-04-17T10:50:34.898863+0000","last_active":"2024-04-17T10:50:35.916370+0000","last_peered":"2024-04-17T10:50:35.916370+0000","last_clean":"2024-04-17T10:50:35.916370+0000","last_became_active":"2024-04-17T10:50:34.898178+0000","last_became_peered":"2024-04-17T10:50:34.898178+0000","last_unstale":"2024-04-17T10:50:35.916370+0000","last_undegraded":"2024-04-17T10:50:35.916370+0000","last_fullsized":"2024-04-17T10:50:35.916370+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:44:03.595860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.024500+0000","last_change":"2024-04-17T10:50:34.902118+0000","last_active":"2024-04-17T10:50:36.024500+0000","last_peered":"2024-04-17T10:50:36.024500+0000","last_clean":"2024-04-17T10:50:36.024500+0000","last_became_active":"2024-04-17T10:50:34.901259+0000","last_became_peered":"2024-04-17T10:50:34.901259+0000","last_unstale":"2024-04-17T10:50:36.024500+0000","last_undegraded":"2024-04-17T10:50:36.024500+0000","last_fullsized":"2024-04-17T10:50:36.024500+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:35:32.874037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.918217+0000","last_change":"2024-04-17T10:50:34.934837+0000","last_active":"2024-04-17T10:50:35.918217+0000","last_peered":"2024-04-17T10:50:35.918217+0000","last_clean":"2024-04-17T10:50:35.918217+0000","last_became_active":"2024-04-17T10:50:34.934301+0000","last_became_peered":"2024-04-17T10:50:34.934301+0000","last_unstale":"2024-04-17T10:50:35.918217+0000","last_undegraded":"2024-04-17T10:50:35.918217+0000","last_fullsized":"2024-04-17T10:50:35.918217+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:46:52.682058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.916400+0000","last_change":"2024-04-17T10:50:34.906169+0000","last_active":"2024-04-17T10:50:35.916400+0000","last_peered":"2024-04-17T10:50:35.916400+0000","last_clean":"2024-04-17T10:50:35.916400+0000","last_became_active":"2024-04-17T10:50:34.905338+0000","last_became_peered":"2024-04-17T10:50:34.905338+0000","last_unstale":"2024-04-17T10:50:35.916400+0000","last_undegraded":"2024-04-17T10:50:35.916400+0000","last_fullsized":"2024-04-17T10:50:35.916400+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:29:07.534791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.917544+0000","last_change":"2024-04-17T10:50:34.890549+0000","last_active":"2024-04-17T10:50:35.917544+0000","last_peered":"2024-04-17T10:50:35.917544+0000","last_clean":"2024-04-17T10:50:35.917544+0000","last_became_active":"2024-04-17T10:50:34.889776+0000","last_became_peered":"2024-04-17T10:50:34.889776+0000","last_unstale":"2024-04-17T10:50:35.917544+0000","last_undegraded":"2024-04-17T10:50:35.917544+0000","last_fullsized":"2024-04-17T10:50:35.917544+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:44:49.330467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.959306+0000","last_change":"2024-04-17T10:50:34.909461+0000","last_active":"2024-04-17T10:50:36.959306+0000","last_peered":"2024-04-17T10:50:36.959306+0000","last_clean":"2024-04-17T10:50:36.959306+0000","last_became_active":"2024-04-17T10:50:34.908741+0000","last_became_peered":"2024-04-17T10:50:34.908741+0000","last_unstale":"2024-04-17T10:50:36.959306+0000","last_undegraded":"2024-04-17T10:50:36.959306+0000","last_fullsized":"2024-04-17T10:50:36.959306+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:33:44.313047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.919606+0000","last_change":"2024-04-17T10:50:34.922558+0000","last_active":"2024-04-17T10:50:35.919606+0000","last_peered":"2024-04-17T10:50:35.919606+0000","last_clean":"2024-04-17T10:50:35.919606+0000","last_became_active":"2024-04-17T10:50:34.921794+0000","last_became_peered":"2024-04-17T10:50:34.921794+0000","last_unstale":"2024-04-17T10:50:35.919606+0000","last_undegraded":"2024-04-17T10:50:35.919606+0000","last_fullsized":"2024-04-17T10:50:35.919606+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:36:09.518340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.916214+0000","last_change":"2024-04-17T10:50:34.891356+0000","last_active":"2024-04-17T10:50:35.916214+0000","last_peered":"2024-04-17T10:50:35.916214+0000","last_clean":"2024-04-17T10:50:35.916214+0000","last_became_active":"2024-04-17T10:50:34.890600+0000","last_became_peered":"2024-04-17T10:50:34.890600+0000","last_unstale":"2024-04-17T10:50:35.916214+0000","last_undegraded":"2024-04-17T10:50:35.916214+0000","last_fullsized":"2024-04-17T10:50:35.916214+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:25:16.724413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.958021+0000","last_change":"2024-04-17T10:50:34.905014+0000","last_active":"2024-04-17T10:50:36.958021+0000","last_peered":"2024-04-17T10:50:36.958021+0000","last_clean":"2024-04-17T10:50:36.958021+0000","last_became_active":"2024-04-17T10:50:34.904514+0000","last_became_peered":"2024-04-17T10:50:34.904514+0000","last_unstale":"2024-04-17T10:50:36.958021+0000","last_undegraded":"2024-04-17T10:50:36.958021+0000","last_fullsized":"2024-04-17T10:50:36.958021+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:51:13.567490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.918979+0000","last_change":"2024-04-17T10:50:34.902199+0000","last_active":"2024-04-17T10:50:35.918979+0000","last_peered":"2024-04-17T10:50:35.918979+0000","last_clean":"2024-04-17T10:50:35.918979+0000","last_became_active":"2024-04-17T10:50:34.900390+0000","last_became_peered":"2024-04-17T10:50:34.900390+0000","last_unstale":"2024-04-17T10:50:35.918979+0000","last_undegraded":"2024-04-17T10:50:35.918979+0000","last_fullsized":"2024-04-17T10:50:35.918979+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:32:06.491956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.958576+0000","last_change":"2024-04-17T10:50:34.904447+0000","last_active":"2024-04-17T10:50:36.958576+0000","last_peered":"2024-04-17T10:50:36.958576+0000","last_clean":"2024-04-17T10:50:36.958576+0000","last_became_active":"2024-04-17T10:50:34.903666+0000","last_became_peered":"2024-04-17T10:50:34.903666+0000","last_unstale":"2024-04-17T10:50:36.958576+0000","last_undegraded":"2024-04-17T10:50:36.958576+0000","last_fullsized":"2024-04-17T10:50:36.958576+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:21:04.322874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.916750+0000","last_change":"2024-04-17T10:50:34.903106+0000","last_active":"2024-04-17T10:50:35.916750+0000","last_peered":"2024-04-17T10:50:35.916750+0000","last_clean":"2024-04-17T10:50:35.916750+0000","last_became_active":"2024-04-17T10:50:34.902561+0000","last_became_peered":"2024-04-17T10:50:34.902561+0000","last_unstale":"2024-04-17T10:50:35.916750+0000","last_undegraded":"2024-04-17T10:50:35.916750+0000","last_fullsized":"2024-04-17T10:50:35.916750+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:08:12.615848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.959209+0000","last_change":"2024-04-17T10:50:34.892564+0000","last_active":"2024-04-17T10:50:36.959209+0000","last_peered":"2024-04-17T10:50:36.959209+0000","last_clean":"2024-04-17T10:50:36.959209+0000","last_became_active":"2024-04-17T10:50:34.892265+0000","last_became_peered":"2024-04-17T10:50:34.892265+0000","last_unstale":"2024-04-17T10:50:36.959209+0000","last_undegraded":"2024-04-17T10:50:36.959209+0000","last_fullsized":"2024-04-17T10:50:36.959209+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:46:33.608695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.917930+0000","last_change":"2024-04-17T10:50:34.934704+0000","last_active":"2024-04-17T10:50:35.917930+0000","last_peered":"2024-04-17T10:50:35.917930+0000","last_clean":"2024-04-17T10:50:35.917930+0000","last_became_active":"2024-04-17T10:50:34.934223+0000","last_became_peered":"2024-04-17T10:50:34.934223+0000","last_unstale":"2024-04-17T10:50:35.917930+0000","last_undegraded":"2024-04-17T10:50:35.917930+0000","last_fullsized":"2024-04-17T10:50:35.917930+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:17:41.737984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.917571+0000","last_change":"2024-04-17T10:50:34.915422+0000","last_active":"2024-04-17T10:50:35.917571+0000","last_peered":"2024-04-17T10:50:35.917571+0000","last_clean":"2024-04-17T10:50:35.917571+0000","last_became_active":"2024-04-17T10:50:34.913669+0000","last_became_peered":"2024-04-17T10:50:34.913669+0000","last_unstale":"2024-04-17T10:50:35.917571+0000","last_undegraded":"2024-04-17T10:50:35.917571+0000","last_fullsized":"2024-04-17T10:50:35.917571+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:23:56.679391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.916847+0000","last_change":"2024-04-17T10:50:34.915256+0000","last_active":"2024-04-17T10:50:35.916847+0000","last_peered":"2024-04-17T10:50:35.916847+0000","last_clean":"2024-04-17T10:50:35.916847+0000","last_became_active":"2024-04-17T10:50:34.914583+0000","last_became_peered":"2024-04-17T10:50:34.914583+0000","last_unstale":"2024-04-17T10:50:35.916847+0000","last_undegraded":"2024-04-17T10:50:35.916847+0000","last_fullsized":"2024-04-17T10:50:35.916847+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:43:29.261985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.916258+0000","last_change":"2024-04-17T10:50:34.911165+0000","last_active":"2024-04-17T10:50:35.916258+0000","last_peered":"2024-04-17T10:50:35.916258+0000","last_clean":"2024-04-17T10:50:35.916258+0000","last_became_active":"2024-04-17T10:50:34.910530+0000","last_became_peered":"2024-04-17T10:50:34.910530+0000","last_unstale":"2024-04-17T10:50:35.916258+0000","last_undegraded":"2024-04-17T10:50:35.916258+0000","last_fullsized":"2024-04-17T10:50:35.916258+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:32:34.024840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.958344+0000","last_change":"2024-04-17T10:50:34.905973+0000","last_active":"2024-04-17T10:50:36.958344+0000","last_peered":"2024-04-17T10:50:36.958344+0000","last_clean":"2024-04-17T10:50:36.958344+0000","last_became_active":"2024-04-17T10:50:34.904886+0000","last_became_peered":"2024-04-17T10:50:34.904886+0000","last_unstale":"2024-04-17T10:50:36.958344+0000","last_undegraded":"2024-04-17T10:50:36.958344+0000","last_fullsized":"2024-04-17T10:50:36.958344+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:05.977861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.916575+0000","last_change":"2024-04-17T10:50:34.900751+0000","last_active":"2024-04-17T10:50:35.916575+0000","last_peered":"2024-04-17T10:50:35.916575+0000","last_clean":"2024-04-17T10:50:35.916575+0000","last_became_active":"2024-04-17T10:50:34.900345+0000","last_became_peered":"2024-04-17T10:50:34.900345+0000","last_unstale":"2024-04-17T10:50:35.916575+0000","last_undegraded":"2024-04-17T10:50:35.916575+0000","last_fullsized":"2024-04-17T10:50:35.916575+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:03:32.948103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.958657+0000","last_change":"2024-04-17T10:50:34.902783+0000","last_active":"2024-04-17T10:50:36.958657+0000","last_peered":"2024-04-17T10:50:36.958657+0000","last_clean":"2024-04-17T10:50:36.958657+0000","last_became_active":"2024-04-17T10:50:34.900417+0000","last_became_peered":"2024-04-17T10:50:34.900417+0000","last_unstale":"2024-04-17T10:50:36.958657+0000","last_undegraded":"2024-04-17T10:50:36.958657+0000","last_fullsized":"2024-04-17T10:50:36.958657+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:25:57.560689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.959431+0000","last_change":"2024-04-17T10:50:34.902379+0000","last_active":"2024-04-17T10:50:36.959431+0000","last_peered":"2024-04-17T10:50:36.959431+0000","last_clean":"2024-04-17T10:50:36.959431+0000","last_became_active":"2024-04-17T10:50:34.901860+0000","last_became_peered":"2024-04-17T10:50:34.901860+0000","last_unstale":"2024-04-17T10:50:36.959431+0000","last_undegraded":"2024-04-17T10:50:36.959431+0000","last_fullsized":"2024-04-17T10:50:36.959431+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:50:39.469298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.916231+0000","last_change":"2024-04-17T10:50:34.906495+0000","last_active":"2024-04-17T10:50:35.916231+0000","last_peered":"2024-04-17T10:50:35.916231+0000","last_clean":"2024-04-17T10:50:35.916231+0000","last_became_active":"2024-04-17T10:50:34.905424+0000","last_became_peered":"2024-04-17T10:50:34.905424+0000","last_unstale":"2024-04-17T10:50:35.916231+0000","last_undegraded":"2024-04-17T10:50:35.916231+0000","last_fullsized":"2024-04-17T10:50:35.916231+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:16:17.150663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.917953+0000","last_change":"2024-04-17T10:50:34.920776+0000","last_active":"2024-04-17T10:50:35.917953+0000","last_peered":"2024-04-17T10:50:35.917953+0000","last_clean":"2024-04-17T10:50:35.917953+0000","last_became_active":"2024-04-17T10:50:34.915077+0000","last_became_peered":"2024-04-17T10:50:34.915077+0000","last_unstale":"2024-04-17T10:50:35.917953+0000","last_undegraded":"2024-04-17T10:50:35.917953+0000","last_fullsized":"2024-04-17T10:50:35.917953+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:21:04.322874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.024091+0000","last_change":"2024-04-17T10:50:34.894447+0000","last_active":"2024-04-17T10:50:36.024091+0000","last_peered":"2024-04-17T10:50:36.024091+0000","last_clean":"2024-04-17T10:50:36.024091+0000","last_became_active":"2024-04-17T10:50:34.889948+0000","last_became_peered":"2024-04-17T10:50:34.889948+0000","last_unstale":"2024-04-17T10:50:36.024091+0000","last_undegraded":"2024-04-17T10:50:36.024091+0000","last_fullsized":"2024-04-17T10:50:36.024091+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:22:02.344005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.958022+0000","last_change":"2024-04-17T10:50:34.895347+0000","last_active":"2024-04-17T10:50:36.958022+0000","last_peered":"2024-04-17T10:50:36.958022+0000","last_clean":"2024-04-17T10:50:36.958022+0000","last_became_active":"2024-04-17T10:50:34.895045+0000","last_became_peered":"2024-04-17T10:50:34.895045+0000","last_unstale":"2024-04-17T10:50:36.958022+0000","last_undegraded":"2024-04-17T10:50:36.958022+0000","last_fullsized":"2024-04-17T10:50:36.958022+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:53:06.383760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.918663+0000","last_change":"2024-04-17T10:50:34.901601+0000","last_active":"2024-04-17T10:50:35.918663+0000","last_peered":"2024-04-17T10:50:35.918663+0000","last_clean":"2024-04-17T10:50:35.918663+0000","last_became_active":"2024-04-17T10:50:34.899585+0000","last_became_peered":"2024-04-17T10:50:34.899585+0000","last_unstale":"2024-04-17T10:50:35.918663+0000","last_undegraded":"2024-04-17T10:50:35.918663+0000","last_fullsized":"2024-04-17T10:50:35.918663+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:29:20.372510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"139'4","reported_seq":18,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.918561+0000","last_change":"2024-04-17T10:50:33.891540+0000","last_active":"2024-04-17T10:50:35.918561+0000","last_peered":"2024-04-17T10:50:35.918561+0000","last_clean":"2024-04-17T10:50:35.918561+0000","last_became_active":"2024-04-17T10:50:33.890837+0000","last_became_peered":"2024-04-17T10:50:33.890837+0000","last_unstale":"2024-04-17T10:50:35.918561+0000","last_undegraded":"2024-04-17T10:50:35.918561+0000","last_fullsized":"2024-04-17T10:50:35.918561+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:23:39.326201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.024819+0000","last_change":"2024-04-17T10:50:34.897444+0000","last_active":"2024-04-17T10:50:36.024819+0000","last_peered":"2024-04-17T10:50:36.024819+0000","last_clean":"2024-04-17T10:50:36.024819+0000","last_became_active":"2024-04-17T10:50:34.896675+0000","last_became_peered":"2024-04-17T10:50:34.896675+0000","last_unstale":"2024-04-17T10:50:36.024819+0000","last_undegraded":"2024-04-17T10:50:36.024819+0000","last_fullsized":"2024-04-17T10:50:36.024819+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:25:16.724413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.024597+0000","last_change":"2024-04-17T10:50:33.890842+0000","last_active":"2024-04-17T10:50:36.024597+0000","last_peered":"2024-04-17T10:50:36.024597+0000","last_clean":"2024-04-17T10:50:36.024597+0000","last_became_active":"2024-04-17T10:50:33.890472+0000","last_became_peered":"2024-04-17T10:50:33.890472+0000","last_unstale":"2024-04-17T10:50:36.024597+0000","last_undegraded":"2024-04-17T10:50:36.024597+0000","last_fullsized":"2024-04-17T10:50:36.024597+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:48:29.238257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.916267+0000","last_change":"2024-04-17T10:50:34.941374+0000","last_active":"2024-04-17T10:50:35.916267+0000","last_peered":"2024-04-17T10:50:35.916267+0000","last_clean":"2024-04-17T10:50:35.916267+0000","last_became_active":"2024-04-17T10:50:34.941071+0000","last_became_peered":"2024-04-17T10:50:34.941071+0000","last_unstale":"2024-04-17T10:50:35.916267+0000","last_undegraded":"2024-04-17T10:50:35.916267+0000","last_fullsized":"2024-04-17T10:50:35.916267+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:40:30.184339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.919011+0000","last_change":"2024-04-17T10:50:33.899787+0000","last_active":"2024-04-17T10:50:35.919011+0000","last_peered":"2024-04-17T10:50:35.919011+0000","last_clean":"2024-04-17T10:50:35.919011+0000","last_became_active":"2024-04-17T10:50:33.899273+0000","last_became_peered":"2024-04-17T10:50:33.899273+0000","last_unstale":"2024-04-17T10:50:35.919011+0000","last_undegraded":"2024-04-17T10:50:35.919011+0000","last_fullsized":"2024-04-17T10:50:35.919011+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:33:03.592509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.917784+0000","last_change":"2024-04-17T10:50:34.891393+0000","last_active":"2024-04-17T10:50:35.917784+0000","last_peered":"2024-04-17T10:50:35.917784+0000","last_clean":"2024-04-17T10:50:35.917784+0000","last_became_active":"2024-04-17T10:50:34.889921+0000","last_became_peered":"2024-04-17T10:50:34.889921+0000","last_unstale":"2024-04-17T10:50:35.917784+0000","last_undegraded":"2024-04-17T10:50:35.917784+0000","last_fullsized":"2024-04-17T10:50:35.917784+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:54:51.339338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.918937+0000","last_change":"2024-04-17T10:50:33.884473+0000","last_active":"2024-04-17T10:50:35.918937+0000","last_peered":"2024-04-17T10:50:35.918937+0000","last_clean":"2024-04-17T10:50:35.918937+0000","last_became_active":"2024-04-17T10:50:33.883898+0000","last_became_peered":"2024-04-17T10:50:33.883898+0000","last_unstale":"2024-04-17T10:50:35.918937+0000","last_undegraded":"2024-04-17T10:50:35.918937+0000","last_fullsized":"2024-04-17T10:50:35.918937+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:23:33.767758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.024141+0000","last_change":"2024-04-17T10:50:34.889125+0000","last_active":"2024-04-17T10:50:36.024141+0000","last_peered":"2024-04-17T10:50:36.024141+0000","last_clean":"2024-04-17T10:50:36.024141+0000","last_became_active":"2024-04-17T10:50:34.888697+0000","last_became_peered":"2024-04-17T10:50:34.888697+0000","last_unstale":"2024-04-17T10:50:36.024141+0000","last_undegraded":"2024-04-17T10:50:36.024141+0000","last_fullsized":"2024-04-17T10:50:36.024141+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:52.994993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.958207+0000","last_change":"2024-04-17T10:50:33.888560+0000","last_active":"2024-04-17T10:50:36.958207+0000","last_peered":"2024-04-17T10:50:36.958207+0000","last_clean":"2024-04-17T10:50:36.958207+0000","last_became_active":"2024-04-17T10:50:33.888155+0000","last_became_peered":"2024-04-17T10:50:33.888155+0000","last_unstale":"2024-04-17T10:50:36.958207+0000","last_undegraded":"2024-04-17T10:50:36.958207+0000","last_fullsized":"2024-04-17T10:50:36.958207+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:12:58.629783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.918452+0000","last_change":"2024-04-17T10:50:34.923638+0000","last_active":"2024-04-17T10:50:35.918452+0000","last_peered":"2024-04-17T10:50:35.918452+0000","last_clean":"2024-04-17T10:50:35.918452+0000","last_became_active":"2024-04-17T10:50:34.922951+0000","last_became_peered":"2024-04-17T10:50:34.922951+0000","last_unstale":"2024-04-17T10:50:35.918452+0000","last_undegraded":"2024-04-17T10:50:35.918452+0000","last_fullsized":"2024-04-17T10:50:35.918452+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:58:00.074911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.957830+0000","last_change":"2024-04-17T10:50:33.896476+0000","last_active":"2024-04-17T10:50:36.957830+0000","last_peered":"2024-04-17T10:50:36.957830+0000","last_clean":"2024-04-17T10:50:36.957830+0000","last_became_active":"2024-04-17T10:50:33.895863+0000","last_became_peered":"2024-04-17T10:50:33.895863+0000","last_unstale":"2024-04-17T10:50:36.957830+0000","last_undegraded":"2024-04-17T10:50:36.957830+0000","last_fullsized":"2024-04-17T10:50:36.957830+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:26.920392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.024205+0000","last_change":"2024-04-17T10:50:34.896841+0000","last_active":"2024-04-17T10:50:36.024205+0000","last_peered":"2024-04-17T10:50:36.024205+0000","last_clean":"2024-04-17T10:50:36.024205+0000","last_became_active":"2024-04-17T10:50:34.896270+0000","last_became_peered":"2024-04-17T10:50:34.896270+0000","last_unstale":"2024-04-17T10:50:36.024205+0000","last_undegraded":"2024-04-17T10:50:36.024205+0000","last_fullsized":"2024-04-17T10:50:36.024205+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:17:05.403285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.958121+0000","last_change":"2024-04-17T10:50:33.895471+0000","last_active":"2024-04-17T10:50:36.958121+0000","last_peered":"2024-04-17T10:50:36.958121+0000","last_clean":"2024-04-17T10:50:36.958121+0000","last_became_active":"2024-04-17T10:50:33.895069+0000","last_became_peered":"2024-04-17T10:50:33.895069+0000","last_unstale":"2024-04-17T10:50:36.958121+0000","last_undegraded":"2024-04-17T10:50:36.958121+0000","last_fullsized":"2024-04-17T10:50:36.958121+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:34.669861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.958274+0000","last_change":"2024-04-17T10:50:34.902650+0000","last_active":"2024-04-17T10:50:36.958274+0000","last_peered":"2024-04-17T10:50:36.958274+0000","last_clean":"2024-04-17T10:50:36.958274+0000","last_became_active":"2024-04-17T10:50:34.901880+0000","last_became_peered":"2024-04-17T10:50:34.901880+0000","last_unstale":"2024-04-17T10:50:36.958274+0000","last_undegraded":"2024-04-17T10:50:36.958274+0000","last_fullsized":"2024-04-17T10:50:36.958274+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:21:02.246547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.916851+0000","last_change":"2024-04-17T10:50:33.885177+0000","last_active":"2024-04-17T10:50:35.916851+0000","last_peered":"2024-04-17T10:50:35.916851+0000","last_clean":"2024-04-17T10:50:35.916851+0000","last_became_active":"2024-04-17T10:50:33.884763+0000","last_became_peered":"2024-04-17T10:50:33.884763+0000","last_unstale":"2024-04-17T10:50:35.916851+0000","last_undegraded":"2024-04-17T10:50:35.916851+0000","last_fullsized":"2024-04-17T10:50:35.916851+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:21:01.238739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.959767+0000","last_change":"2024-04-17T10:50:34.891541+0000","last_active":"2024-04-17T10:50:36.959767+0000","last_peered":"2024-04-17T10:50:36.959767+0000","last_clean":"2024-04-17T10:50:36.959767+0000","last_became_active":"2024-04-17T10:50:34.890785+0000","last_became_peered":"2024-04-17T10:50:34.890785+0000","last_unstale":"2024-04-17T10:50:36.959767+0000","last_undegraded":"2024-04-17T10:50:36.959767+0000","last_fullsized":"2024-04-17T10:50:36.959767+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:49:47.473255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.024841+0000","last_change":"2024-04-17T10:50:33.895658+0000","last_active":"2024-04-17T10:50:36.024841+0000","last_peered":"2024-04-17T10:50:36.024841+0000","last_clean":"2024-04-17T10:50:36.024841+0000","last_became_active":"2024-04-17T10:50:33.894228+0000","last_became_peered":"2024-04-17T10:50:33.894228+0000","last_unstale":"2024-04-17T10:50:36.024841+0000","last_undegraded":"2024-04-17T10:50:36.024841+0000","last_fullsized":"2024-04-17T10:50:36.024841+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:36:02.424435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.918323+0000","last_change":"2024-04-17T10:50:34.902963+0000","last_active":"2024-04-17T10:50:35.918323+0000","last_peered":"2024-04-17T10:50:35.918323+0000","last_clean":"2024-04-17T10:50:35.918323+0000","last_became_active":"2024-04-17T10:50:34.900442+0000","last_became_peered":"2024-04-17T10:50:34.900442+0000","last_unstale":"2024-04-17T10:50:35.918323+0000","last_undegraded":"2024-04-17T10:50:35.918323+0000","last_fullsized":"2024-04-17T10:50:35.918323+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:37.045623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.919442+0000","last_change":"2024-04-17T10:50:33.890039+0000","last_active":"2024-04-17T10:50:35.919442+0000","last_peered":"2024-04-17T10:50:35.919442+0000","last_clean":"2024-04-17T10:50:35.919442+0000","last_became_active":"2024-04-17T10:50:33.889635+0000","last_became_peered":"2024-04-17T10:50:33.889635+0000","last_unstale":"2024-04-17T10:50:35.919442+0000","last_undegraded":"2024-04-17T10:50:35.919442+0000","last_fullsized":"2024-04-17T10:50:35.919442+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:16:34.733445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.958839+0000","last_change":"2024-04-17T10:50:34.893858+0000","last_active":"2024-04-17T10:50:36.958839+0000","last_peered":"2024-04-17T10:50:36.958839+0000","last_clean":"2024-04-17T10:50:36.958839+0000","last_became_active":"2024-04-17T10:50:34.892211+0000","last_became_peered":"2024-04-17T10:50:34.892211+0000","last_unstale":"2024-04-17T10:50:36.958839+0000","last_undegraded":"2024-04-17T10:50:36.958839+0000","last_fullsized":"2024-04-17T10:50:36.958839+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:11:12.923426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"139'6","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.959562+0000","last_change":"2024-04-17T10:50:33.898934+0000","last_active":"2024-04-17T10:50:36.959562+0000","last_peered":"2024-04-17T10:50:36.959562+0000","last_clean":"2024-04-17T10:50:36.959562+0000","last_became_active":"2024-04-17T10:50:33.897331+0000","last_became_peered":"2024-04-17T10:50:33.897331+0000","last_unstale":"2024-04-17T10:50:36.959562+0000","last_undegraded":"2024-04-17T10:50:36.959562+0000","last_fullsized":"2024-04-17T10:50:36.959562+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:48:29.238257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.957986+0000","last_change":"2024-04-17T10:50:34.930739+0000","last_active":"2024-04-17T10:50:36.957986+0000","last_peered":"2024-04-17T10:50:36.957986+0000","last_clean":"2024-04-17T10:50:36.957986+0000","last_became_active":"2024-04-17T10:50:34.930348+0000","last_became_peered":"2024-04-17T10:50:34.930348+0000","last_unstale":"2024-04-17T10:50:36.957986+0000","last_undegraded":"2024-04-17T10:50:36.957986+0000","last_fullsized":"2024-04-17T10:50:36.957986+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:51:02.471554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.958117+0000","last_change":"2024-04-17T10:50:33.900576+0000","last_active":"2024-04-17T10:50:36.958117+0000","last_peered":"2024-04-17T10:50:36.958117+0000","last_clean":"2024-04-17T10:50:36.958117+0000","last_became_active":"2024-04-17T10:50:33.899933+0000","last_became_peered":"2024-04-17T10:50:33.899933+0000","last_unstale":"2024-04-17T10:50:36.958117+0000","last_undegraded":"2024-04-17T10:50:36.958117+0000","last_fullsized":"2024-04-17T10:50:36.958117+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:21:01.238739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.024374+0000","last_change":"2024-04-17T10:50:34.891215+0000","last_active":"2024-04-17T10:50:36.024374+0000","last_peered":"2024-04-17T10:50:36.024374+0000","last_clean":"2024-04-17T10:50:36.024374+0000","last_became_active":"2024-04-17T10:50:34.890033+0000","last_became_peered":"2024-04-17T10:50:34.890033+0000","last_unstale":"2024-04-17T10:50:36.024374+0000","last_undegraded":"2024-04-17T10:50:36.024374+0000","last_fullsized":"2024-04-17T10:50:36.024374+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:18:43.211742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.958256+0000","last_change":"2024-04-17T10:50:33.890641+0000","last_active":"2024-04-17T10:50:36.958256+0000","last_peered":"2024-04-17T10:50:36.958256+0000","last_clean":"2024-04-17T10:50:36.958256+0000","last_became_active":"2024-04-17T10:50:33.889972+0000","last_became_peered":"2024-04-17T10:50:33.889972+0000","last_unstale":"2024-04-17T10:50:36.958256+0000","last_undegraded":"2024-04-17T10:50:36.958256+0000","last_fullsized":"2024-04-17T10:50:36.958256+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:23:33.767758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.957784+0000","last_change":"2024-04-17T10:50:34.908881+0000","last_active":"2024-04-17T10:50:36.957784+0000","last_peered":"2024-04-17T10:50:36.957784+0000","last_clean":"2024-04-17T10:50:36.957784+0000","last_became_active":"2024-04-17T10:50:34.907247+0000","last_became_peered":"2024-04-17T10:50:34.907247+0000","last_unstale":"2024-04-17T10:50:36.957784+0000","last_undegraded":"2024-04-17T10:50:36.957784+0000","last_fullsized":"2024-04-17T10:50:36.957784+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:08:12.615848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.916224+0000","last_change":"2024-04-17T10:50:33.897839+0000","last_active":"2024-04-17T10:50:35.916224+0000","last_peered":"2024-04-17T10:50:35.916224+0000","last_clean":"2024-04-17T10:50:35.916224+0000","last_became_active":"2024-04-17T10:50:33.897168+0000","last_became_peered":"2024-04-17T10:50:33.897168+0000","last_unstale":"2024-04-17T10:50:35.916224+0000","last_undegraded":"2024-04-17T10:50:35.916224+0000","last_fullsized":"2024-04-17T10:50:35.916224+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:46:32.600887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.957128+0000","last_change":"2024-04-17T10:50:34.897888+0000","last_active":"2024-04-17T10:50:36.957128+0000","last_peered":"2024-04-17T10:50:36.957128+0000","last_clean":"2024-04-17T10:50:36.957128+0000","last_became_active":"2024-04-17T10:50:34.897116+0000","last_became_peered":"2024-04-17T10:50:34.897116+0000","last_unstale":"2024-04-17T10:50:36.957128+0000","last_undegraded":"2024-04-17T10:50:36.957128+0000","last_fullsized":"2024-04-17T10:50:36.957128+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:24:20.846974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"139'8","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.958208+0000","last_change":"2024-04-17T10:50:33.895104+0000","last_active":"2024-04-17T10:50:36.958208+0000","last_peered":"2024-04-17T10:50:36.958208+0000","last_clean":"2024-04-17T10:50:36.958208+0000","last_became_active":"2024-04-17T10:50:33.894652+0000","last_became_peered":"2024-04-17T10:50:33.894652+0000","last_unstale":"2024-04-17T10:50:36.958208+0000","last_undegraded":"2024-04-17T10:50:36.958208+0000","last_fullsized":"2024-04-17T10:50:36.958208+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:35:53.326622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.919049+0000","last_change":"2024-04-17T10:50:34.902964+0000","last_active":"2024-04-17T10:50:35.919049+0000","last_peered":"2024-04-17T10:50:35.919049+0000","last_clean":"2024-04-17T10:50:35.919049+0000","last_became_active":"2024-04-17T10:50:34.900597+0000","last_became_peered":"2024-04-17T10:50:34.900597+0000","last_unstale":"2024-04-17T10:50:35.919049+0000","last_undegraded":"2024-04-17T10:50:35.919049+0000","last_fullsized":"2024-04-17T10:50:35.919049+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:13:23.672734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.959465+0000","last_change":"2024-04-17T10:50:33.892680+0000","last_active":"2024-04-17T10:50:36.959465+0000","last_peered":"2024-04-17T10:50:36.959465+0000","last_clean":"2024-04-17T10:50:36.959465+0000","last_became_active":"2024-04-17T10:50:33.891996+0000","last_became_peered":"2024-04-17T10:50:33.891996+0000","last_unstale":"2024-04-17T10:50:36.959465+0000","last_undegraded":"2024-04-17T10:50:36.959465+0000","last_fullsized":"2024-04-17T10:50:36.959465+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:09:32.560749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.958460+0000","last_change":"2024-04-17T10:50:34.889037+0000","last_active":"2024-04-17T10:50:36.958460+0000","last_peered":"2024-04-17T10:50:36.958460+0000","last_clean":"2024-04-17T10:50:36.958460+0000","last_became_active":"2024-04-17T10:50:34.888099+0000","last_became_peered":"2024-04-17T10:50:34.888099+0000","last_unstale":"2024-04-17T10:50:36.958460+0000","last_undegraded":"2024-04-17T10:50:36.958460+0000","last_fullsized":"2024-04-17T10:50:36.958460+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:23:12.983036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.024651+0000","last_change":"2024-04-17T10:50:33.898138+0000","last_active":"2024-04-17T10:50:36.024651+0000","last_peered":"2024-04-17T10:50:36.024651+0000","last_clean":"2024-04-17T10:50:36.024651+0000","last_became_active":"2024-04-17T10:50:33.896506+0000","last_became_peered":"2024-04-17T10:50:33.896506+0000","last_unstale":"2024-04-17T10:50:36.024651+0000","last_undegraded":"2024-04-17T10:50:36.024651+0000","last_fullsized":"2024-04-17T10:50:36.024651+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:21:59.523164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.958094+0000","last_change":"2024-04-17T10:50:34.932556+0000","last_active":"2024-04-17T10:50:36.958094+0000","last_peered":"2024-04-17T10:50:36.958094+0000","last_clean":"2024-04-17T10:50:36.958094+0000","last_became_active":"2024-04-17T10:50:34.931946+0000","last_became_peered":"2024-04-17T10:50:34.931946+0000","last_unstale":"2024-04-17T10:50:36.958094+0000","last_undegraded":"2024-04-17T10:50:36.958094+0000","last_fullsized":"2024-04-17T10:50:36.958094+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:43:29.261985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.959005+0000","last_change":"2024-04-17T10:50:33.887380+0000","last_active":"2024-04-17T10:50:36.959005+0000","last_peered":"2024-04-17T10:50:36.959005+0000","last_clean":"2024-04-17T10:50:36.959005+0000","last_became_active":"2024-04-17T10:50:33.887003+0000","last_became_peered":"2024-04-17T10:50:33.887003+0000","last_unstale":"2024-04-17T10:50:36.959005+0000","last_undegraded":"2024-04-17T10:50:36.959005+0000","last_fullsized":"2024-04-17T10:50:36.959005+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:19:22.498932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.916627+0000","last_change":"2024-04-17T10:50:34.907357+0000","last_active":"2024-04-17T10:50:35.916627+0000","last_peered":"2024-04-17T10:50:35.916627+0000","last_clean":"2024-04-17T10:50:35.916627+0000","last_became_active":"2024-04-17T10:50:34.906730+0000","last_became_peered":"2024-04-17T10:50:34.906730+0000","last_unstale":"2024-04-17T10:50:35.916627+0000","last_undegraded":"2024-04-17T10:50:35.916627+0000","last_fullsized":"2024-04-17T10:50:35.916627+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:50.018436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.024284+0000","last_change":"2024-04-17T10:50:33.890628+0000","last_active":"2024-04-17T10:50:36.024284+0000","last_peered":"2024-04-17T10:50:36.024284+0000","last_clean":"2024-04-17T10:50:36.024284+0000","last_became_active":"2024-04-17T10:50:33.890232+0000","last_became_peered":"2024-04-17T10:50:33.890232+0000","last_unstale":"2024-04-17T10:50:36.024284+0000","last_undegraded":"2024-04-17T10:50:36.024284+0000","last_fullsized":"2024-04-17T10:50:36.024284+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:01:25.099953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.918331+0000","last_change":"2024-04-17T10:50:34.915983+0000","last_active":"2024-04-17T10:50:35.918331+0000","last_peered":"2024-04-17T10:50:35.918331+0000","last_clean":"2024-04-17T10:50:35.918331+0000","last_became_active":"2024-04-17T10:50:34.914622+0000","last_became_peered":"2024-04-17T10:50:34.914622+0000","last_unstale":"2024-04-17T10:50:35.918331+0000","last_undegraded":"2024-04-17T10:50:35.918331+0000","last_fullsized":"2024-04-17T10:50:35.918331+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:26:15.444880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.959387+0000","last_change":"2024-04-17T10:50:33.901531+0000","last_active":"2024-04-17T10:50:36.959387+0000","last_peered":"2024-04-17T10:50:36.959387+0000","last_clean":"2024-04-17T10:50:36.959387+0000","last_became_active":"2024-04-17T10:50:33.901224+0000","last_became_peered":"2024-04-17T10:50:33.901224+0000","last_unstale":"2024-04-17T10:50:36.959387+0000","last_undegraded":"2024-04-17T10:50:36.959387+0000","last_fullsized":"2024-04-17T10:50:36.959387+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:01:25.099953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.959423+0000","last_change":"2024-04-17T10:50:34.939826+0000","last_active":"2024-04-17T10:50:36.959423+0000","last_peered":"2024-04-17T10:50:36.959423+0000","last_clean":"2024-04-17T10:50:36.959423+0000","last_became_active":"2024-04-17T10:50:34.939223+0000","last_became_peered":"2024-04-17T10:50:34.939223+0000","last_unstale":"2024-04-17T10:50:36.959423+0000","last_undegraded":"2024-04-17T10:50:36.959423+0000","last_fullsized":"2024-04-17T10:50:36.959423+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:52:17.331394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.916688+0000","last_change":"2024-04-17T10:50:33.891385+0000","last_active":"2024-04-17T10:50:35.916688+0000","last_peered":"2024-04-17T10:50:35.916688+0000","last_clean":"2024-04-17T10:50:35.916688+0000","last_became_active":"2024-04-17T10:50:33.890896+0000","last_became_peered":"2024-04-17T10:50:33.890896+0000","last_unstale":"2024-04-17T10:50:35.916688+0000","last_undegraded":"2024-04-17T10:50:35.916688+0000","last_fullsized":"2024-04-17T10:50:35.916688+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:21:50.888389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.916602+0000","last_change":"2024-04-17T10:50:34.894864+0000","last_active":"2024-04-17T10:50:35.916602+0000","last_peered":"2024-04-17T10:50:35.916602+0000","last_clean":"2024-04-17T10:50:35.916602+0000","last_became_active":"2024-04-17T10:50:34.894105+0000","last_became_peered":"2024-04-17T10:50:34.894105+0000","last_unstale":"2024-04-17T10:50:35.916602+0000","last_undegraded":"2024-04-17T10:50:35.916602+0000","last_fullsized":"2024-04-17T10:50:35.916602+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:51:02.471554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.916130+0000","last_change":"2024-04-17T10:50:33.893344+0000","last_active":"2024-04-17T10:50:35.916130+0000","last_peered":"2024-04-17T10:50:35.916130+0000","last_clean":"2024-04-17T10:50:35.916130+0000","last_became_active":"2024-04-17T10:50:33.892680+0000","last_became_peered":"2024-04-17T10:50:33.892680+0000","last_unstale":"2024-04-17T10:50:35.916130+0000","last_undegraded":"2024-04-17T10:50:35.916130+0000","last_fullsized":"2024-04-17T10:50:35.916130+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:09:49.416649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.958886+0000","last_change":"2024-04-17T10:50:34.939599+0000","last_active":"2024-04-17T10:50:36.958886+0000","last_peered":"2024-04-17T10:50:36.958886+0000","last_clean":"2024-04-17T10:50:36.958886+0000","last_became_active":"2024-04-17T10:50:34.939298+0000","last_became_peered":"2024-04-17T10:50:34.939298+0000","last_unstale":"2024-04-17T10:50:36.958886+0000","last_undegraded":"2024-04-17T10:50:36.958886+0000","last_fullsized":"2024-04-17T10:50:36.958886+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:52:50.981124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.956731+0000","last_change":"2024-04-17T10:50:33.897362+0000","last_active":"2024-04-17T10:50:36.956731+0000","last_peered":"2024-04-17T10:50:36.956731+0000","last_clean":"2024-04-17T10:50:36.956731+0000","last_became_active":"2024-04-17T10:50:33.896738+0000","last_became_peered":"2024-04-17T10:50:33.896738+0000","last_unstale":"2024-04-17T10:50:36.956731+0000","last_undegraded":"2024-04-17T10:50:36.956731+0000","last_fullsized":"2024-04-17T10:50:36.956731+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:10:48.609154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.959882+0000","last_change":"2024-04-17T10:50:34.903846+0000","last_active":"2024-04-17T10:50:36.959882+0000","last_peered":"2024-04-17T10:50:36.959882+0000","last_clean":"2024-04-17T10:50:36.959882+0000","last_became_active":"2024-04-17T10:50:34.903080+0000","last_became_peered":"2024-04-17T10:50:34.903080+0000","last_unstale":"2024-04-17T10:50:36.959882+0000","last_undegraded":"2024-04-17T10:50:36.959882+0000","last_fullsized":"2024-04-17T10:50:36.959882+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:03:16.436167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.919236+0000","last_change":"2024-04-17T10:50:33.893658+0000","last_active":"2024-04-17T10:50:35.919236+0000","last_peered":"2024-04-17T10:50:35.919236+0000","last_clean":"2024-04-17T10:50:35.919236+0000","last_became_active":"2024-04-17T10:50:33.893023+0000","last_became_peered":"2024-04-17T10:50:33.893023+0000","last_unstale":"2024-04-17T10:50:35.919236+0000","last_undegraded":"2024-04-17T10:50:35.919236+0000","last_fullsized":"2024-04-17T10:50:35.919236+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:22:19.505182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.958299+0000","last_change":"2024-04-17T10:50:34.931045+0000","last_active":"2024-04-17T10:50:36.958299+0000","last_peered":"2024-04-17T10:50:36.958299+0000","last_clean":"2024-04-17T10:50:36.958299+0000","last_became_active":"2024-04-17T10:50:34.930352+0000","last_became_peered":"2024-04-17T10:50:34.930352+0000","last_unstale":"2024-04-17T10:50:36.958299+0000","last_undegraded":"2024-04-17T10:50:36.958299+0000","last_fullsized":"2024-04-17T10:50:36.958299+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:23:56.679391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.916629+0000","last_change":"2024-04-17T10:50:33.887529+0000","last_active":"2024-04-17T10:50:35.916629+0000","last_peered":"2024-04-17T10:50:35.916629+0000","last_clean":"2024-04-17T10:50:35.916629+0000","last_became_active":"2024-04-17T10:50:33.887152+0000","last_became_peered":"2024-04-17T10:50:33.887152+0000","last_unstale":"2024-04-17T10:50:35.916629+0000","last_undegraded":"2024-04-17T10:50:35.916629+0000","last_fullsized":"2024-04-17T10:50:35.916629+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:49:42.514723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.024326+0000","last_change":"2024-04-17T10:50:34.935646+0000","last_active":"2024-04-17T10:50:36.024326+0000","last_peered":"2024-04-17T10:50:36.024326+0000","last_clean":"2024-04-17T10:50:36.024326+0000","last_became_active":"2024-04-17T10:50:34.935413+0000","last_became_peered":"2024-04-17T10:50:34.935413+0000","last_unstale":"2024-04-17T10:50:36.024326+0000","last_undegraded":"2024-04-17T10:50:36.024326+0000","last_fullsized":"2024-04-17T10:50:36.024326+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:12:20.384464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.916803+0000","last_change":"2024-04-17T10:50:33.889203+0000","last_active":"2024-04-17T10:50:35.916803+0000","last_peered":"2024-04-17T10:50:35.916803+0000","last_clean":"2024-04-17T10:50:35.916803+0000","last_became_active":"2024-04-17T10:50:33.888312+0000","last_became_peered":"2024-04-17T10:50:33.888312+0000","last_unstale":"2024-04-17T10:50:35.916803+0000","last_undegraded":"2024-04-17T10:50:35.916803+0000","last_fullsized":"2024-04-17T10:50:35.916803+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:20:34.698272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.916823+0000","last_change":"2024-04-17T10:50:34.941301+0000","last_active":"2024-04-17T10:50:35.916823+0000","last_peered":"2024-04-17T10:50:35.916823+0000","last_clean":"2024-04-17T10:50:35.916823+0000","last_became_active":"2024-04-17T10:50:34.941029+0000","last_became_peered":"2024-04-17T10:50:34.941029+0000","last_unstale":"2024-04-17T10:50:35.916823+0000","last_undegraded":"2024-04-17T10:50:35.916823+0000","last_fullsized":"2024-04-17T10:50:35.916823+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:33:04.600317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.916366+0000","last_change":"2024-04-17T10:50:33.882648+0000","last_active":"2024-04-17T10:50:35.916366+0000","last_peered":"2024-04-17T10:50:35.916366+0000","last_clean":"2024-04-17T10:50:35.916366+0000","last_became_active":"2024-04-17T10:50:33.882297+0000","last_became_peered":"2024-04-17T10:50:33.882297+0000","last_unstale":"2024-04-17T10:50:35.916366+0000","last_undegraded":"2024-04-17T10:50:35.916366+0000","last_fullsized":"2024-04-17T10:50:35.916366+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:17:39.813633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.958199+0000","last_change":"2024-04-17T10:50:34.925187+0000","last_active":"2024-04-17T10:50:36.958199+0000","last_peered":"2024-04-17T10:50:36.958199+0000","last_clean":"2024-04-17T10:50:36.958199+0000","last_became_active":"2024-04-17T10:50:34.924689+0000","last_became_peered":"2024-04-17T10:50:34.924689+0000","last_unstale":"2024-04-17T10:50:36.958199+0000","last_undegraded":"2024-04-17T10:50:36.958199+0000","last_fullsized":"2024-04-17T10:50:36.958199+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:29:31.711033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.959527+0000","last_change":"2024-04-17T10:50:33.900315+0000","last_active":"2024-04-17T10:50:36.959527+0000","last_peered":"2024-04-17T10:50:36.959527+0000","last_clean":"2024-04-17T10:50:36.959527+0000","last_became_active":"2024-04-17T10:50:33.899987+0000","last_became_peered":"2024-04-17T10:50:33.899987+0000","last_unstale":"2024-04-17T10:50:36.959527+0000","last_undegraded":"2024-04-17T10:50:36.959527+0000","last_fullsized":"2024-04-17T10:50:36.959527+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:21:59.523164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.959740+0000","last_change":"2024-04-17T10:50:34.889756+0000","last_active":"2024-04-17T10:50:36.959740+0000","last_peered":"2024-04-17T10:50:36.959740+0000","last_clean":"2024-04-17T10:50:36.959740+0000","last_became_active":"2024-04-17T10:50:34.889080+0000","last_became_peered":"2024-04-17T10:50:34.889080+0000","last_unstale":"2024-04-17T10:50:36.959740+0000","last_undegraded":"2024-04-17T10:50:36.959740+0000","last_fullsized":"2024-04-17T10:50:36.959740+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:23:34.775566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.916435+0000","last_change":"2024-04-17T10:50:33.894342+0000","last_active":"2024-04-17T10:50:35.916435+0000","last_peered":"2024-04-17T10:50:35.916435+0000","last_clean":"2024-04-17T10:50:35.916435+0000","last_became_active":"2024-04-17T10:50:33.893720+0000","last_became_peered":"2024-04-17T10:50:33.893720+0000","last_unstale":"2024-04-17T10:50:35.916435+0000","last_undegraded":"2024-04-17T10:50:35.916435+0000","last_fullsized":"2024-04-17T10:50:35.916435+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:36:02.424435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.916750+0000","last_change":"2024-04-17T10:50:34.892573+0000","last_active":"2024-04-17T10:50:35.916750+0000","last_peered":"2024-04-17T10:50:35.916750+0000","last_clean":"2024-04-17T10:50:35.916750+0000","last_became_active":"2024-04-17T10:50:34.891398+0000","last_became_peered":"2024-04-17T10:50:34.891398+0000","last_unstale":"2024-04-17T10:50:35.916750+0000","last_undegraded":"2024-04-17T10:50:35.916750+0000","last_fullsized":"2024-04-17T10:50:35.916750+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:26:18.865699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.957169+0000","last_change":"2024-04-17T10:50:33.895914+0000","last_active":"2024-04-17T10:50:36.957169+0000","last_peered":"2024-04-17T10:50:36.957169+0000","last_clean":"2024-04-17T10:50:36.957169+0000","last_became_active":"2024-04-17T10:50:33.893274+0000","last_became_peered":"2024-04-17T10:50:33.893274+0000","last_unstale":"2024-04-17T10:50:36.957169+0000","last_undegraded":"2024-04-17T10:50:36.957169+0000","last_fullsized":"2024-04-17T10:50:36.957169+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:33:03.592509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.958666+0000","last_change":"2024-04-17T10:50:34.897346+0000","last_active":"2024-04-17T10:50:36.958666+0000","last_peered":"2024-04-17T10:50:36.958666+0000","last_clean":"2024-04-17T10:50:36.958666+0000","last_became_active":"2024-04-17T10:50:34.896413+0000","last_became_peered":"2024-04-17T10:50:34.896413+0000","last_unstale":"2024-04-17T10:50:36.958666+0000","last_undegraded":"2024-04-17T10:50:36.958666+0000","last_fullsized":"2024-04-17T10:50:36.958666+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:10:49.616962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.023987+0000","last_change":"2024-04-17T10:50:33.893329+0000","last_active":"2024-04-17T10:50:36.023987+0000","last_peered":"2024-04-17T10:50:36.023987+0000","last_clean":"2024-04-17T10:50:36.023987+0000","last_became_active":"2024-04-17T10:50:33.892918+0000","last_became_peered":"2024-04-17T10:50:33.892918+0000","last_unstale":"2024-04-17T10:50:36.023987+0000","last_undegraded":"2024-04-17T10:50:36.023987+0000","last_fullsized":"2024-04-17T10:50:36.023987+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:16:19.474043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.959473+0000","last_change":"2024-04-17T10:50:34.939627+0000","last_active":"2024-04-17T10:50:36.959473+0000","last_peered":"2024-04-17T10:50:36.959473+0000","last_clean":"2024-04-17T10:50:36.959473+0000","last_became_active":"2024-04-17T10:50:34.939241+0000","last_became_peered":"2024-04-17T10:50:34.939241+0000","last_unstale":"2024-04-17T10:50:36.959473+0000","last_undegraded":"2024-04-17T10:50:36.959473+0000","last_fullsized":"2024-04-17T10:50:36.959473+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:33.854740+0000","last_clean_scrub_stamp":"2024-04-17T10:50:33.854740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:22:02.344005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"139'3","reported_seq":17,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:36.958813+0000","last_change":"2024-04-17T10:50:33.896124+0000","last_active":"2024-04-17T10:50:36.958813+0000","last_peered":"2024-04-17T10:50:36.958813+0000","last_clean":"2024-04-17T10:50:36.958813+0000","last_became_active":"2024-04-17T10:50:33.895638+0000","last_became_peered":"2024-04-17T10:50:33.895638+0000","last_unstale":"2024-04-17T10:50:36.958813+0000","last_undegraded":"2024-04-17T10:50:36.958813+0000","last_fullsized":"2024-04-17T10:50:36.958813+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:50:32.846932+0000","last_clean_scrub_stamp":"2024-04-17T10:50:32.846932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:33:43.305239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":416,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-17T10:50:35.916578+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:50:35.916578+0000","last_peered":"2024-04-17T10:50:35.916578+0000","last_clean":"2024-04-17T10:50:35.916578+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:50:35.916578+0000","last_undegraded":"2024-04-17T10:50:35.916578+0000","last_fullsized":"2024-04-17T10:50:35.916578+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575004,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47368,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324472,"statfs":{"total":96636764160,"available":96588259328,"internally_reserved":0,"allocated":2355200,"data_stored":1359921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575003,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69000,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302840,"statfs":{"total":96636764160,"available":96566108160,"internally_reserved":0,"allocated":1765376,"data_stored":769047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575004,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55104,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316736,"statfs":{"total":96636764160,"available":96580337664,"internally_reserved":0,"allocated":1757184,"data_stored":768923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575003,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73156,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298684,"statfs":{"total":96636764160,"available":96561852416,"internally_reserved":0,"allocated":1753088,"data_stored":768901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575005,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73152,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298688,"statfs":{"total":96636764160,"available":96561856512,"internally_reserved":0,"allocated":1757184,"data_stored":768991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575003,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46796,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325044,"statfs":{"total":96636764160,"available":96588845056,"internally_reserved":0,"allocated":1769472,"data_stored":770583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575003,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82888,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288952,"statfs":{"total":96636764160,"available":96551886848,"internally_reserved":0,"allocated":1757184,"data_stored":769483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575003,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47376,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324464,"statfs":{"total":96636764160,"available":96588251136,"internally_reserved":0,"allocated":2363392,"data_stored":1360915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:50:42.669 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-17T10:50:42.669 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-17T10:50:42.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.848+0000 7f2fda486640 1 Processor -- start 2024-04-17T10:50:42.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.848+0000 7f2fda486640 1 -- start start 2024-04-17T10:50:42.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.848+0000 7f2fda486640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2fd40af150 0x7f2fd40af530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:42.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.848+0000 7f2fda486640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2fd4072210 0x7f2fd415e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:42.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.848+0000 7f2fda486640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fd4072940 0x7f2fd40669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:42.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.848+0000 7f2fda486640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd4071d80 con 0x7f2fd40af150 2024-04-17T10:50:42.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.848+0000 7f2fda486640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd4071170 con 0x7f2fd4072210 2024-04-17T10:50:42.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.848+0000 7f2fda486640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd4071450 con 0x7f2fd4072940 2024-04-17T10:50:42.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.848+0000 7f2fd3fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2fd40af150 0x7f2fd40af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:42.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.848+0000 7f2fd3fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2fd40af150 0x7f2fd40af530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35394/0 (socket says 172.21.15.83:35394) 2024-04-17T10:50:42.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.848+0000 7f2fd3fff640 1 -- 172.21.15.83:0/4053924763 learned_addr learned my addr 172.21.15.83:0/4053924763 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:42.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.848+0000 7f2fd37fe640 1 --2- 172.21.15.83:0/4053924763 >> v2:172.21.15.167:3300/0 conn(0x7f2fd4072210 0x7f2fd415e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:42.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.848+0000 7f2fd8a23640 1 --2- 172.21.15.83:0/4053924763 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fd4072940 0x7f2fd40669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:42.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.848+0000 7f2fd3fff640 1 -- 172.21.15.83:0/4053924763 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fd4072940 msgr2=0x7f2fd40669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:42.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.848+0000 7f2fd3fff640 1 --2- 172.21.15.83:0/4053924763 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fd4072940 0x7f2fd40669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:42.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.848+0000 7f2fd3fff640 1 -- 172.21.15.83:0/4053924763 >> v2:172.21.15.167:3300/0 conn(0x7f2fd4072210 msgr2=0x7f2fd415e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:42.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.848+0000 7f2fd3fff640 1 --2- 172.21.15.83:0/4053924763 >> v2:172.21.15.167:3300/0 conn(0x7f2fd4072210 0x7f2fd415e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:42.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.848+0000 7f2fd3fff640 1 -- 172.21.15.83:0/4053924763 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd4071730 con 0x7f2fd40af150 2024-04-17T10:50:42.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.848+0000 7f2fd8a23640 1 --2- 172.21.15.83:0/4053924763 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fd4072940 0x7f2fd40669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:42.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.848+0000 7f2fd37fe640 1 --2- 172.21.15.83:0/4053924763 >> v2:172.21.15.167:3300/0 conn(0x7f2fd4072210 0x7f2fd415e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:42.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.852+0000 7f2fd3fff640 1 --2- 172.21.15.83:0/4053924763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fd40af150 0x7f2fd40af530 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f2fd40cfde0 tx=0x7f2fcc01ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c26e1e228d4a1ea server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:42.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.852+0000 7f2fd2ffd640 1 -- 172.21.15.83:0/4053924763 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc0b4070 con 0x7f2fd40af150 2024-04-17T10:50:42.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.852+0000 7f2fd2ffd640 1 -- 172.21.15.83:0/4053924763 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fcc0a7b60 con 0x7f2fd40af150 2024-04-17T10:50:42.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.852+0000 7f2fd2ffd640 1 -- 172.21.15.83:0/4053924763 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc0af440 con 0x7f2fd40af150 2024-04-17T10:50:42.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.852+0000 7f2fda486640 1 -- 172.21.15.83:0/4053924763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fd40af150 msgr2=0x7f2fd40af530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:42.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.852+0000 7f2fda486640 1 --2- 172.21.15.83:0/4053924763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fd40af150 0x7f2fd40af530 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f2fd40cfde0 tx=0x7f2fcc01ebc0 comp rx=0 tx=0).stop 2024-04-17T10:50:42.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.852+0000 7f2fda486640 1 -- 172.21.15.83:0/4053924763 shutdown_connections 2024-04-17T10:50:42.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.852+0000 7f2fda486640 1 --2- 172.21.15.83:0/4053924763 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fd4072940 0x7f2fd40669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:42.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.852+0000 7f2fda486640 1 --2- 172.21.15.83:0/4053924763 >> v2:172.21.15.167:3300/0 conn(0x7f2fd4072210 0x7f2fd415e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:42.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.852+0000 7f2fda486640 1 --2- 172.21.15.83:0/4053924763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fd40af150 0x7f2fd40af530 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:42.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.852+0000 7f2fda486640 1 -- 172.21.15.83:0/4053924763 >> 172.21.15.83:0/4053924763 conn(0x7f2fd406abe0 msgr2=0x7f2fd406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:42.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.852+0000 7f2fda486640 1 -- 172.21.15.83:0/4053924763 shutdown_connections 2024-04-17T10:50:42.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.852+0000 7f2fda486640 1 -- 172.21.15.83:0/4053924763 wait complete. 2024-04-17T10:50:42.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.852+0000 7f2fda486640 1 Processor -- start 2024-04-17T10:50:42.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.852+0000 7f2fda486640 1 -- start start 2024-04-17T10:50:42.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.852+0000 7f2fda486640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fd4072210 0x7f2fd414e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:42.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.852+0000 7f2fda486640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fd4072940 0x7f2fd40c9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:42.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.852+0000 7f2fda486640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fd40af150 0x7f2fd40c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:42.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.856+0000 7f2fda486640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd40c5f00 con 0x7f2fd40af150 2024-04-17T10:50:42.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.856+0000 7f2fda486640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd40c5d80 con 0x7f2fd4072210 2024-04-17T10:50:42.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.856+0000 7f2fda486640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd40c6080 con 0x7f2fd4072940 2024-04-17T10:50:42.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.856+0000 7f2fd8a23640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fd40af150 0x7f2fd40c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:42.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.856+0000 7f2fd8a23640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fd40af150 0x7f2fd40c96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35402/0 (socket says 172.21.15.83:35402) 2024-04-17T10:50:42.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.856+0000 7f2fd8a23640 1 -- 172.21.15.83:0/2935521853 learned_addr learned my addr 172.21.15.83:0/2935521853 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:42.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.856+0000 7f2fd37fe640 1 --2- 172.21.15.83:0/2935521853 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fd4072940 0x7f2fd40c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:42.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.856+0000 7f2fd3fff640 1 --2- 172.21.15.83:0/2935521853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fd4072210 0x7f2fd414e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:42.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.856+0000 7f2fd8a23640 1 -- 172.21.15.83:0/2935521853 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fd4072940 msgr2=0x7f2fd40c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:42.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.856+0000 7f2fd8a23640 1 --2- 172.21.15.83:0/2935521853 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fd4072940 0x7f2fd40c9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:42.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.856+0000 7f2fd8a23640 1 -- 172.21.15.83:0/2935521853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fd4072210 msgr2=0x7f2fd414e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:42.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.856+0000 7f2fd8a23640 1 --2- 172.21.15.83:0/2935521853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fd4072210 0x7f2fd414e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:42.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.856+0000 7f2fd8a23640 1 -- 172.21.15.83:0/2935521853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd40c7bb0 con 0x7f2fd40af150 2024-04-17T10:50:42.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.856+0000 7f2fd37fe640 1 --2- 172.21.15.83:0/2935521853 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fd4072940 0x7f2fd40c9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:42.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.856+0000 7f2fd3fff640 1 --2- 172.21.15.83:0/2935521853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fd4072210 0x7f2fd414e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:50:42.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.856+0000 7f2fd8a23640 1 --2- 172.21.15.83:0/2935521853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fd40af150 0x7f2fd40c96d0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400c7f0 tx=0x7f2fc400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:42.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.856+0000 7f2fd0ff9640 1 -- 172.21.15.83:0/2935521853 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4015070 con 0x7f2fd40af150 2024-04-17T10:50:42.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.856+0000 7f2fd0ff9640 1 -- 172.21.15.83:0/2935521853 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc4007860 con 0x7f2fd40af150 2024-04-17T10:50:42.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.856+0000 7f2fda486640 1 -- 172.21.15.83:0/2935521853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd40c7e90 con 0x7f2fd40af150 2024-04-17T10:50:42.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.856+0000 7f2fd0ff9640 1 -- 172.21.15.83:0/2935521853 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4007c50 con 0x7f2fd40af150 2024-04-17T10:50:42.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.856+0000 7f2fda486640 1 -- 172.21.15.83:0/2935521853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd40c8450 con 0x7f2fd40af150 2024-04-17T10:50:42.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.856+0000 7f2fda486640 1 -- 172.21.15.83:0/2935521853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd4074f90 con 0x7f2fd40af150 2024-04-17T10:50:42.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.856+0000 7f2fd0ff9640 1 -- 172.21.15.83:0/2935521853 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2fc4018070 con 0x7f2fd40af150 2024-04-17T10:50:42.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.856+0000 7f2fd0ff9640 1 --2- 172.21.15.83:0/2935521853 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2fc0066f60 0x7f2fc0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:42.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.856+0000 7f2fd0ff9640 1 -- 172.21.15.83:0/2935521853 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2fc4085140 con 0x7f2fd40af150 2024-04-17T10:50:42.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.856+0000 7f2fd3fff640 1 --2- 172.21.15.83:0/2935521853 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2fc0066f60 0x7f2fc0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:42.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.856+0000 7f2fd3fff640 1 --2- 172.21.15.83:0/2935521853 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2fc0066f60 0x7f2fc0069420 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0b1000 tx=0x7f2fcc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:42.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:42.860+0000 7f2fd0ff9640 1 -- 172.21.15.83:0/2935521853 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2fc40561e0 con 0x7f2fd40af150 2024-04-17T10:50:43.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.080+0000 7f2fda486640 1 -- 172.21.15.83:0/2935521853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f2fd40ce590 con 0x7f2fd40af150 2024-04-17T10:50:43.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.084+0000 7f2fd0ff9640 1 -- 172.21.15.83:0/2935521853 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+147 (secure 0 0 0) 0x7f2fc4055ae0 con 0x7f2fd40af150 2024-04-17T10:50:43.077 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-04-17T10:50:43.078 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBUph9mHGmXGBAAZF+JiAG9YrFtbW+dCy2IEQ== 2024-04-17T10:50:43.078 INFO:teuthology.orchestra.run.smithi083.stdout: caps mds = "allow" 2024-04-17T10:50:43.078 INFO:teuthology.orchestra.run.smithi083.stdout: caps mon = "allow r" 2024-04-17T10:50:43.078 INFO:teuthology.orchestra.run.smithi083.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-17T10:50:43.078 INFO:teuthology.orchestra.run.smithi083.stderr:updated caps for client.0 2024-04-17T10:50:43.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.088+0000 7f2fda486640 1 -- 172.21.15.83:0/2935521853 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2fc0066f60 msgr2=0x7f2fc0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:43.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.088+0000 7f2fda486640 1 --2- 172.21.15.83:0/2935521853 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2fc0066f60 0x7f2fc0069420 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0b1000 tx=0x7f2fcc008040 comp rx=0 tx=0).stop 2024-04-17T10:50:43.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.088+0000 7f2fda486640 1 -- 172.21.15.83:0/2935521853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fd40af150 msgr2=0x7f2fd40c96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:43.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.088+0000 7f2fda486640 1 --2- 172.21.15.83:0/2935521853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fd40af150 0x7f2fd40c96d0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400c7f0 tx=0x7f2fc400ccb0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.088+0000 7f2fda486640 1 -- 172.21.15.83:0/2935521853 shutdown_connections 2024-04-17T10:50:43.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.088+0000 7f2fda486640 1 --2- 172.21.15.83:0/2935521853 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2fc0066f60 0x7f2fc0069420 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.088+0000 7f2fda486640 1 --2- 172.21.15.83:0/2935521853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fd40af150 0x7f2fd40c96d0 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.088+0000 7f2fda486640 1 --2- 172.21.15.83:0/2935521853 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fd4072940 0x7f2fd40c9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.088+0000 7f2fda486640 1 --2- 172.21.15.83:0/2935521853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fd4072210 0x7f2fd414e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.088+0000 7f2fda486640 1 -- 172.21.15.83:0/2935521853 >> 172.21.15.83:0/2935521853 conn(0x7f2fd406abe0 msgr2=0x7f2fd406d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:43.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.088+0000 7f2fda486640 1 -- 172.21.15.83:0/2935521853 shutdown_connections 2024-04-17T10:50:43.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.088+0000 7f2fda486640 1 -- 172.21.15.83:0/2935521853 wait complete. 2024-04-17T10:50:43.098 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:50:43.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.220+0000 7fc43e2cc640 1 Processor -- start 2024-04-17T10:50:43.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.220+0000 7fc43e2cc640 1 -- start start 2024-04-17T10:50:43.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.220+0000 7fc43e2cc640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc4380aedd0 0x7fc43806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:43.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.220+0000 7fc43e2cc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc43806dae0 0x7fc4380b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:43.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.220+0000 7fc43e2cc640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc43815e440 con 0x7fc43815db50 2024-04-17T10:50:43.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.220+0000 7fc43e2cc640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc43815e720 con 0x7fc43806dae0 2024-04-17T10:50:43.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.220+0000 7fc43e2cc640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc43815ea00 con 0x7fc4380aedd0 2024-04-17T10:50:43.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.220+0000 7fc43c869640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc43806dae0 0x7fc4380b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:43.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.220+0000 7fc437fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc43815db50 0x7fc438070470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57200/0 (socket says 172.21.15.83:57200) 2024-04-17T10:50:43.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.220+0000 7fc437fff640 1 -- 172.21.15.83:0/4063729679 learned_addr learned my addr 172.21.15.83:0/4063729679 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:43.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.220+0000 7fc43c869640 1 -- 172.21.15.83:0/4063729679 >> v2:172.21.15.167:3300/0 conn(0x7fc4380aedd0 msgr2=0x7fc43806d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:43.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.220+0000 7fc43c869640 1 --2- 172.21.15.83:0/4063729679 >> v2:172.21.15.167:3300/0 conn(0x7fc4380aedd0 0x7fc43806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.220+0000 7fc43c869640 1 -- 172.21.15.83:0/4063729679 >> v1:172.21.15.83:6789/0 conn(0x7fc43815db50 legacy=0x7fc438070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:43.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.220+0000 7fc43c869640 1 -- 172.21.15.83:0/4063729679 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc43815ece0 con 0x7fc43806dae0 2024-04-17T10:50:43.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.220+0000 7fc43c869640 1 --2- 172.21.15.83:0/4063729679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc43806dae0 0x7fc4380b2e10 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fc430009d90 tx=0x7fc43001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f166965c469c95fe server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:43.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.220+0000 7fc436ffd640 1 -- 172.21.15.83:0/4063729679 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4300b2070 con 0x7fc43806dae0 2024-04-17T10:50:43.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.224+0000 7fc436ffd640 1 -- 172.21.15.83:0/4063729679 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4300a57f0 con 0x7fc43806dae0 2024-04-17T10:50:43.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.224+0000 7fc436ffd640 1 -- 172.21.15.83:0/4063729679 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4300a5bc0 con 0x7fc43806dae0 2024-04-17T10:50:43.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.224+0000 7fc43e2cc640 1 -- 172.21.15.83:0/4063729679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc43806dae0 msgr2=0x7fc4380b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:43.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.224+0000 7fc43e2cc640 1 --2- 172.21.15.83:0/4063729679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc43806dae0 0x7fc4380b2e10 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fc430009d90 tx=0x7fc43001c930 comp rx=0 tx=0).stop 2024-04-17T10:50:43.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.224+0000 7fc43e2cc640 1 -- 172.21.15.83:0/4063729679 shutdown_connections 2024-04-17T10:50:43.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.224+0000 7fc43e2cc640 1 --2- 172.21.15.83:0/4063729679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc43806dae0 0x7fc4380b2e10 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.224+0000 7fc43e2cc640 1 --2- 172.21.15.83:0/4063729679 >> v2:172.21.15.167:3300/0 conn(0x7fc4380aedd0 0x7fc43806d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.224+0000 7fc43e2cc640 1 -- 172.21.15.83:0/4063729679 >> 172.21.15.83:0/4063729679 conn(0x7fc43806abe0 msgr2=0x7fc43806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:43.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.224+0000 7fc43e2cc640 1 -- 172.21.15.83:0/4063729679 shutdown_connections 2024-04-17T10:50:43.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.224+0000 7fc43e2cc640 1 -- 172.21.15.83:0/4063729679 wait complete. 2024-04-17T10:50:43.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.224+0000 7fc43e2cc640 1 Processor -- start 2024-04-17T10:50:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.224+0000 7fc43e2cc640 1 -- start start 2024-04-17T10:50:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.224+0000 7fc43e2cc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc43806dae0 0x7fc438144200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.224+0000 7fc43e2cc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4380aedd0 0x7fc43815bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.224+0000 7fc43e2cc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc43815db50 0x7fc43815c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.224+0000 7fc43e2cc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4380b3ef0 con 0x7fc43815db50 2024-04-17T10:50:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.224+0000 7fc43e2cc640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4380b3d70 con 0x7fc4380aedd0 2024-04-17T10:50:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.224+0000 7fc43e2cc640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4380b4070 con 0x7fc43806dae0 2024-04-17T10:50:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.224+0000 7fc43c869640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc43815db50 0x7fc43815c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.224+0000 7fc43c869640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc43815db50 0x7fc43815c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35424/0 (socket says 172.21.15.83:35424) 2024-04-17T10:50:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.224+0000 7fc43c869640 1 -- 172.21.15.83:0/841583091 learned_addr learned my addr 172.21.15.83:0/841583091 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.224+0000 7fc43c869640 1 -- 172.21.15.83:0/841583091 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc43806dae0 msgr2=0x7fc438144200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.224+0000 7fc43c869640 1 --2- 172.21.15.83:0/841583091 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc43806dae0 0x7fc438144200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.224+0000 7fc43c869640 1 -- 172.21.15.83:0/841583091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4380aedd0 msgr2=0x7fc43815bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:43.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.224+0000 7fc43c869640 1 --2- 172.21.15.83:0/841583091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4380aedd0 0x7fc43815bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.224+0000 7fc43c869640 1 -- 172.21.15.83:0/841583091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4380ccb20 con 0x7fc43815db50 2024-04-17T10:50:43.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.224+0000 7fc43c869640 1 --2- 172.21.15.83:0/841583091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc43815db50 0x7fc43815c1c0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fc4300b0040 tx=0x7fc4300196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:43.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.224+0000 7fc434ff9640 1 -- 172.21.15.83:0/841583091 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4300b2050 con 0x7fc43815db50 2024-04-17T10:50:43.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.224+0000 7fc434ff9640 1 -- 172.21.15.83:0/841583091 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4300be070 con 0x7fc43815db50 2024-04-17T10:50:43.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.224+0000 7fc43e2cc640 1 -- 172.21.15.83:0/841583091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc438152890 con 0x7fc43815db50 2024-04-17T10:50:43.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.224+0000 7fc434ff9640 1 -- 172.21.15.83:0/841583091 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4300ada50 con 0x7fc43815db50 2024-04-17T10:50:43.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.224+0000 7fc43e2cc640 1 -- 172.21.15.83:0/841583091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc438144740 con 0x7fc43815db50 2024-04-17T10:50:43.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.228+0000 7fc43e2cc640 1 -- 172.21.15.83:0/841583091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc438074f90 con 0x7fc43815db50 2024-04-17T10:50:43.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.228+0000 7fc434ff9640 1 -- 172.21.15.83:0/841583091 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc4300adbf0 con 0x7fc43815db50 2024-04-17T10:50:43.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.228+0000 7fc434ff9640 1 --2- 172.21.15.83:0/841583091 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc424066f60 0x7fc424069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:43.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.228+0000 7fc437fff640 1 --2- 172.21.15.83:0/841583091 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc424066f60 0x7fc424069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:43.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.228+0000 7fc434ff9640 1 -- 172.21.15.83:0/841583091 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4301203d0 con 0x7fc43815db50 2024-04-17T10:50:43.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.228+0000 7fc437fff640 1 --2- 172.21.15.83:0/841583091 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc424066f60 0x7fc424069420 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fc428000f70 tx=0x7fc428007a90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:43.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.232+0000 7fc434ff9640 1 -- 172.21.15.83:0/841583091 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc430016520 con 0x7fc43815db50 2024-04-17T10:50:43.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.416+0000 7fc43e2cc640 1 -- 172.21.15.83:0/841583091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4380b2af0 con 0x7fc43815db50 2024-04-17T10:50:43.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.420+0000 7fc434ff9640 1 -- 172.21.15.83:0/841583091 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4948 (secure 0 0 0) 0x7fc4300ab340 con 0x7fc43815db50 2024-04-17T10:50:43.412 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:50:43.412 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":87,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:50:36.251622+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9711},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9711":{"gid":9711,"name":"a","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/3575114599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3575114599},{"type":"v1","addr":"172.21.15.83:6837","nonce":3575114599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:50:43.413 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 87 2024-04-17T10:50:43.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.420+0000 7fc43e2cc640 1 -- 172.21.15.83:0/841583091 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc424066f60 msgr2=0x7fc424069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:43.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.420+0000 7fc43e2cc640 1 --2- 172.21.15.83:0/841583091 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc424066f60 0x7fc424069420 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fc428000f70 tx=0x7fc428007a90 comp rx=0 tx=0).stop 2024-04-17T10:50:43.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.420+0000 7fc43e2cc640 1 -- 172.21.15.83:0/841583091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc43815db50 msgr2=0x7fc43815c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:43.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.420+0000 7fc43e2cc640 1 --2- 172.21.15.83:0/841583091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc43815db50 0x7fc43815c1c0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fc4300b0040 tx=0x7fc4300196d0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.420+0000 7fc43e2cc640 1 -- 172.21.15.83:0/841583091 shutdown_connections 2024-04-17T10:50:43.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.420+0000 7fc43e2cc640 1 --2- 172.21.15.83:0/841583091 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc424066f60 0x7fc424069420 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.420+0000 7fc43e2cc640 1 --2- 172.21.15.83:0/841583091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc43815db50 0x7fc43815c1c0 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.420+0000 7fc43e2cc640 1 --2- 172.21.15.83:0/841583091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4380aedd0 0x7fc43815bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.420+0000 7fc43e2cc640 1 --2- 172.21.15.83:0/841583091 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc43806dae0 0x7fc438144200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.420+0000 7fc43e2cc640 1 -- 172.21.15.83:0/841583091 >> 172.21.15.83:0/841583091 conn(0x7fc43806abe0 msgr2=0x7fc43806c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:43.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.420+0000 7fc43e2cc640 1 -- 172.21.15.83:0/841583091 shutdown_connections 2024-04-17T10:50:43.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.420+0000 7fc43e2cc640 1 -- 172.21.15.83:0/841583091 wait complete. 2024-04-17T10:50:43.832 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 87, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:50:36.251622+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9711}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9711': {'gid': 9711, 'name': 'a', 'rank': 0, 'incarnation': 85, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6837/3575114599', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6836', 'nonce': 3575114599}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 3575114599}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:50:43.832 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T10:50:43.833 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-17T10:50:43.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.964+0000 7f5aa232d640 1 Processor -- start 2024-04-17T10:50:43.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.964+0000 7f5aa232d640 1 -- start start 2024-04-17T10:50:43.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.964+0000 7f5aa232d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a9c0b4210 0x7f5a9c15e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:43.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.964+0000 7f5aa232d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a9c15bd70 con 0x7f5a9c06b520 2024-04-17T10:50:43.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.964+0000 7f5aa232d640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a9c15bf60 con 0x7f5a9c154de0 2024-04-17T10:50:43.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.964+0000 7f5aa232d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c0b4870 con 0x7f5a9c0b4210 2024-04-17T10:50:43.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.964+0000 7f5a9b7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5a9c06b520 0x7f5a9c0b3b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57210/0 (socket says 172.21.15.83:57210) 2024-04-17T10:50:43.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.964+0000 7f5a9b7fe640 1 -- 172.21.15.83:0/1948687870 learned_addr learned my addr 172.21.15.83:0/1948687870 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:43.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.964+0000 7f5aa08ca640 1 --2- 172.21.15.83:0/1948687870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a9c0b4210 0x7f5a9c15e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:43.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.964+0000 7f5a9affd640 1 -- 172.21.15.83:0/1948687870 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3710820742 0 0) 0x7f5a9c15bd70 con 0x7f5a9c06b520 2024-04-17T10:50:43.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.964+0000 7f5a9affd640 1 -- 172.21.15.83:0/1948687870 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a84003530 con 0x7f5a9c06b520 2024-04-17T10:50:43.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.964+0000 7f5a9affd640 1 -- 172.21.15.83:0/1948687870 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1457659523 0 0) 0x7f5a84003530 con 0x7f5a9c06b520 2024-04-17T10:50:43.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.964+0000 7f5a9affd640 1 -- 172.21.15.83:0/1948687870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a9c0b4210 msgr2=0x7f5a9c15e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:43.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.964+0000 7f5a9affd640 1 --2- 172.21.15.83:0/1948687870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a9c0b4210 0x7f5a9c15e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.964+0000 7f5a9affd640 1 -- 172.21.15.83:0/1948687870 >> v1:172.21.15.167:6789/0 conn(0x7f5a9c154de0 legacy=0x7f5a9c06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:43.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.964+0000 7f5a9affd640 1 -- 172.21.15.83:0/1948687870 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9c15afb0 con 0x7f5a9c06b520 2024-04-17T10:50:43.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.964+0000 7f5a9affd640 1 -- 172.21.15.83:0/1948687870 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f5a900016a0 con 0x7f5a9c06b520 2024-04-17T10:50:43.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.964+0000 7f5a9affd640 1 -- 172.21.15.83:0/1948687870 >> v1:172.21.15.83:6789/0 conn(0x7f5a9c06b520 legacy=0x7f5a9c0b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:43.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.964+0000 7f5a9affd640 1 --2- 172.21.15.83:0/1948687870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a84003a60 0x7f5a84003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:43.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.964+0000 7f5a9affd640 1 --2- 172.21.15.83:0/1948687870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a84004810 0x7f5a84004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:43.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.964+0000 7f5a9affd640 1 --2- 172.21.15.83:0/1948687870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a9c0b4210 0x7f5a840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.964+0000 7f5a9affd640 1 -- 172.21.15.83:0/1948687870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8400baa0 con 0x7f5a84004810 2024-04-17T10:50:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.964+0000 7f5a9affd640 1 -- 172.21.15.83:0/1948687870 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8400bc50 con 0x7f5a84003a60 2024-04-17T10:50:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.964+0000 7f5a9affd640 1 -- 172.21.15.83:0/1948687870 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8400be50 con 0x7f5a9c0b4210 2024-04-17T10:50:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.964+0000 7f5a9b7fe640 1 --2- 172.21.15.83:0/1948687870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a84004810 0x7f5a84004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.968+0000 7f5a9b7fe640 1 -- 172.21.15.83:0/1948687870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a9c0b4210 msgr2=0x7f5a840051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.968+0000 7f5aa08ca640 1 --2- 172.21.15.83:0/1948687870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a9c0b4210 0x7f5a840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.968+0000 7f5a9b7fe640 1 --2- 172.21.15.83:0/1948687870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a9c0b4210 0x7f5a840051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.968+0000 7f5a9b7fe640 1 -- 172.21.15.83:0/1948687870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a84003a60 msgr2=0x7f5a84003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.968+0000 7f5a9bfff640 1 --2- 172.21.15.83:0/1948687870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a84003a60 0x7f5a84003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.968+0000 7f5a9b7fe640 1 --2- 172.21.15.83:0/1948687870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a84003a60 0x7f5a84003e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.968+0000 7f5a9b7fe640 1 -- 172.21.15.83:0/1948687870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a84003530 con 0x7f5a84004810 2024-04-17T10:50:43.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.968+0000 7f5a9b7fe640 1 --2- 172.21.15.83:0/1948687870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a84004810 0x7f5a84004c90 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000c290 tx=0x7f5a9001f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be47e8374ca056ea server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:43.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.968+0000 7f5a9affd640 1 -- 172.21.15.83:0/1948687870 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a90007d60 con 0x7f5a84004810 2024-04-17T10:50:43.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.968+0000 7f5a9affd640 1 -- 172.21.15.83:0/1948687870 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a900ac070 con 0x7f5a84004810 2024-04-17T10:50:43.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.968+0000 7f5a9affd640 1 -- 172.21.15.83:0/1948687870 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a90005b20 con 0x7f5a84004810 2024-04-17T10:50:43.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.968+0000 7f5aa232d640 1 -- 172.21.15.83:0/1948687870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a84004810 msgr2=0x7f5a84004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:43.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.968+0000 7f5aa232d640 1 --2- 172.21.15.83:0/1948687870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a84004810 0x7f5a84004c90 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000c290 tx=0x7f5a9001f450 comp rx=0 tx=0).stop 2024-04-17T10:50:43.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.968+0000 7f5aa08ca640 1 -- 172.21.15.83:0/1948687870 reap_dead start 2024-04-17T10:50:43.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.968+0000 7f5aa232d640 1 -- 172.21.15.83:0/1948687870 shutdown_connections 2024-04-17T10:50:43.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.968+0000 7f5aa232d640 1 -- 172.21.15.83:0/1948687870 >> 172.21.15.83:0/1948687870 conn(0x7f5a9c0b15b0 msgr2=0x7f5a9c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:43.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.968+0000 7f5aa232d640 1 -- 172.21.15.83:0/1948687870 shutdown_connections 2024-04-17T10:50:43.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.968+0000 7f5aa232d640 1 -- 172.21.15.83:0/1948687870 wait complete. 2024-04-17T10:50:43.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.972+0000 7f5aa232d640 1 Processor -- start 2024-04-17T10:50:43.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.972+0000 7f5aa232d640 1 -- start start 2024-04-17T10:50:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.972+0000 7f5aa232d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a84004810 0x7f5a9c14df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.972+0000 7f5aa232d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a9c156be0 0x7f5a9c157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.972+0000 7f5aa232d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a9c157060 0x7f5a9c157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.972+0000 7f5aa232d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c15b850 con 0x7f5a84004810 2024-04-17T10:50:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.972+0000 7f5aa232d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c15b570 con 0x7f5a9c157060 2024-04-17T10:50:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.972+0000 7f5aa232d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c1551c0 con 0x7f5a9c156be0 2024-04-17T10:50:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.972+0000 7f5a9bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a84004810 0x7f5a9c14df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.972+0000 7f5a9bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a84004810 0x7f5a9c14df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35448/0 (socket says 172.21.15.83:35448) 2024-04-17T10:50:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.972+0000 7f5a9bfff640 1 -- 172.21.15.83:0/3344157061 learned_addr learned my addr 172.21.15.83:0/3344157061 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.972+0000 7f5a9bfff640 1 -- 172.21.15.83:0/3344157061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a9c156be0 msgr2=0x7f5a9c157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.972+0000 7f5a9bfff640 1 --2- 172.21.15.83:0/3344157061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a9c156be0 0x7f5a9c157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.972+0000 7f5a9bfff640 1 -- 172.21.15.83:0/3344157061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a9c157060 msgr2=0x7f5a9c157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.972+0000 7f5a9bfff640 1 --2- 172.21.15.83:0/3344157061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a9c157060 0x7f5a9c157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.972+0000 7f5a9bfff640 1 -- 172.21.15.83:0/3344157061 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9c15a840 con 0x7f5a84004810 2024-04-17T10:50:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.972+0000 7f5a9bfff640 1 --2- 172.21.15.83:0/3344157061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a84004810 0x7f5a9c14df00 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c00d4c0 tx=0x7f5a8c00d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.972+0000 7f5a98ff9640 1 -- 172.21.15.83:0/3344157061 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c005480 con 0x7f5a84004810 2024-04-17T10:50:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.972+0000 7f5a98ff9640 1 -- 172.21.15.83:0/3344157061 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8c017070 con 0x7f5a84004810 2024-04-17T10:50:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.972+0000 7f5aa232d640 1 -- 172.21.15.83:0/3344157061 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a9c155e20 con 0x7f5a84004810 2024-04-17T10:50:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.972+0000 7f5a98ff9640 1 -- 172.21.15.83:0/3344157061 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c003840 con 0x7f5a84004810 2024-04-17T10:50:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.972+0000 7f5aa232d640 1 -- 172.21.15.83:0/3344157061 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a9c1563e0 con 0x7f5a84004810 2024-04-17T10:50:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.976+0000 7f5a98ff9640 1 -- 172.21.15.83:0/3344157061 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5a8c0039e0 con 0x7f5a84004810 2024-04-17T10:50:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.976+0000 7f5a98ff9640 1 --2- 172.21.15.83:0/3344157061 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5a88066f60 0x7f5a88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.976+0000 7f5a9b7fe640 1 --2- 172.21.15.83:0/3344157061 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5a88066f60 0x7f5a88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.976+0000 7f5a98ff9640 1 -- 172.21.15.83:0/3344157061 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a8c084ef0 con 0x7f5a84004810 2024-04-17T10:50:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.976+0000 7f5a9b7fe640 1 --2- 172.21.15.83:0/3344157061 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5a88066f60 0x7f5a88069420 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f5a900ae000 tx=0x7f5a90001a70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.976+0000 7f5aa549a000 1 Processor -- start 2024-04-17T10:50:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.976+0000 7f5aa549a000 1 -- start start 2024-04-17T10:50:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.976+0000 7f5aa549a000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559fb4662920 0x559fb4662d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.976+0000 7f5aa549a000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559fb4663240 0x559fb46189f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.976+0000 7f5aa549a000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fb4618f30 0x559fb4619310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.976+0000 7f5aa549a000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fb45005f0 con 0x559fb4618f30 2024-04-17T10:50:43.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.976+0000 7f5aa549a000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fb4467da0 con 0x559fb4663240 2024-04-17T10:50:43.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.976+0000 7f5aa549a000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x559fb43f0150 con 0x559fb4662920 2024-04-17T10:50:43.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.976+0000 7f5aa08ca640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fb4618f30 0x559fb4619310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:43.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.976+0000 7f5aa08ca640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fb4618f30 0x559fb4619310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35450/0 (socket says 172.21.15.83:35450) 2024-04-17T10:50:43.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.976+0000 7f5aa08ca640 1 -- 172.21.15.83:0/4292152175 learned_addr learned my addr 172.21.15.83:0/4292152175 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:43.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.976+0000 7f5a9bfff640 1 --2- 172.21.15.83:0/4292152175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559fb4662920 0x559fb4662d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:43.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.976+0000 7f5a9b7fe640 1 --2- 172.21.15.83:0/4292152175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559fb4663240 0x559fb46189f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:43.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.976+0000 7f5aa08ca640 1 -- 172.21.15.83:0/4292152175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559fb4662920 msgr2=0x559fb4662d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:43.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.976+0000 7f5aa08ca640 1 --2- 172.21.15.83:0/4292152175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559fb4662920 0x559fb4662d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.976+0000 7f5aa08ca640 1 -- 172.21.15.83:0/4292152175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559fb4663240 msgr2=0x559fb46189f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:43.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.976+0000 7f5aa08ca640 1 --2- 172.21.15.83:0/4292152175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559fb4663240 0x559fb46189f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.976+0000 7f5aa08ca640 1 -- 172.21.15.83:0/4292152175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fb4409ca0 con 0x559fb4618f30 2024-04-17T10:50:43.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.976+0000 7f5aa08ca640 1 --2- 172.21.15.83:0/4292152175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fb4618f30 0x559fb4619310 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c0b2430 tx=0x7f5a94007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bedf67f57ec97e71 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:43.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.976+0000 7f5a81ffb640 1 -- 172.21.15.83:0/4292152175 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a940039a0 con 0x559fb4618f30 2024-04-17T10:50:43.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.976+0000 7f5a81ffb640 1 -- 172.21.15.83:0/4292152175 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a94003b40 con 0x559fb4618f30 2024-04-17T10:50:43.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.976+0000 7f5a81ffb640 1 -- 172.21.15.83:0/4292152175 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a94004020 con 0x559fb4618f30 2024-04-17T10:50:43.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.976+0000 7f5aa549a000 1 -- 172.21.15.83:0/4292152175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fb4618f30 msgr2=0x559fb4619310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:43.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.976+0000 7f5aa549a000 1 --2- 172.21.15.83:0/4292152175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fb4618f30 0x559fb4619310 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c0b2430 tx=0x7f5a94007870 comp rx=0 tx=0).stop 2024-04-17T10:50:43.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5aa549a000 1 -- 172.21.15.83:0/4292152175 shutdown_connections 2024-04-17T10:50:43.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5aa549a000 1 --2- 172.21.15.83:0/4292152175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fb4618f30 0x559fb4619310 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5aa549a000 1 --2- 172.21.15.83:0/4292152175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559fb4663240 0x559fb46189f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5aa549a000 1 --2- 172.21.15.83:0/4292152175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559fb4662920 0x559fb4662d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5aa549a000 1 -- 172.21.15.83:0/4292152175 >> 172.21.15.83:0/4292152175 conn(0x559fb461f2a0 msgr2=0x559fb46172d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:43.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5aa549a000 1 -- 172.21.15.83:0/4292152175 shutdown_connections 2024-04-17T10:50:43.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5aa549a000 1 -- 172.21.15.83:0/4292152175 wait complete. 2024-04-17T10:50:43.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5aa549a000 1 -- start start 2024-04-17T10:50:43.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5aa549a000 1 Processor -- start 2024-04-17T10:50:43.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5aa549a000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559fb4618f30 0x559fb4669c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:43.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5aa549a000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fb4662920 0x559fb466a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:43.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5aa549a000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559fb4663240 0x559fb461caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:43.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5aa549a000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fb4422990 con 0x559fb4662920 2024-04-17T10:50:43.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5aa549a000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fb4463e40 con 0x559fb4618f30 2024-04-17T10:50:43.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5aa549a000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x559fb43f3900 con 0x559fb4663240 2024-04-17T10:50:43.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5aa549a000 10 client.0 _ll_register_callbacks cb 0x559fb451b1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:50:43.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5a9b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fb4662920 0x559fb466a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:43.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5a9b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fb4662920 0x559fb466a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35452/0 (socket says 172.21.15.83:35452) 2024-04-17T10:50:43.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5a9b7fe640 1 -- 172.21.15.83:0/24271911 learned_addr learned my addr 172.21.15.83:0/24271911 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:43.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5a9bfff640 1 --2- 172.21.15.83:0/24271911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559fb4618f30 0x559fb4669c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:43.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5aa08ca640 1 --2- 172.21.15.83:0/24271911 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559fb4663240 0x559fb461caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:43.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5a9b7fe640 1 -- 172.21.15.83:0/24271911 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559fb4663240 msgr2=0x559fb461caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:43.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5a9b7fe640 1 --2- 172.21.15.83:0/24271911 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559fb4663240 0x559fb461caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5a9b7fe640 1 -- 172.21.15.83:0/24271911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559fb4618f30 msgr2=0x559fb4669c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:43.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5a9b7fe640 1 --2- 172.21.15.83:0/24271911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559fb4618f30 0x559fb4669c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5a9b7fe640 1 -- 172.21.15.83:0/24271911 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559fb4420630 con 0x559fb4662920 2024-04-17T10:50:43.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5a9b7fe640 1 --2- 172.21.15.83:0/24271911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fb4662920 0x559fb466a1c0 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f5a90001060 tx=0x7f5a9001c1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ff04ddc93816fb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:43.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5a7affd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:50:43.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5a7affd640 1 -- 172.21.15.83:0/24271911 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a900cb060 con 0x559fb4662920 2024-04-17T10:50:43.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5a7affd640 1 -- 172.21.15.83:0/24271911 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a90004d40 con 0x559fb4662920 2024-04-17T10:50:43.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5aa549a000 1 -- 172.21.15.83:0/24271911 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a94003b40 con 0x559fb4662920 2024-04-17T10:50:43.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5a7affd640 1 -- 172.21.15.83:0/24271911 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a900d1050 con 0x559fb4662920 2024-04-17T10:50:43.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5aa549a000 10 client.9888 fetch_fsmap learned FSMap version 87 2024-04-17T10:50:43.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5aa549a000 1 -- 172.21.15.83:0/24271911 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x559fb43f03a0 con 0x559fb4662920 2024-04-17T10:50:43.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5a7affd640 1 -- 172.21.15.83:0/24271911 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a94003b40 con 0x559fb4662920 2024-04-17T10:50:43.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5a7affd640 1 -- 172.21.15.83:0/24271911 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 87) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5a900d4020 con 0x559fb4662920 2024-04-17T10:50:43.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5aa549a000 10 client.9888 fetch_fsmap finished waiting for FSMap version 87 2024-04-17T10:50:43.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5aa549a000 10 client.9888 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T10:50:43.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5aa549a000 20 client.9888 populate_metadata read hostname 'smithi083' 2024-04-17T10:50:43.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5aa549a000 1 --2- 172.21.15.83:0/24271911 >> [v2:172.21.15.83:6836/3575114599,v1:172.21.15.83:6837/3575114599] conn(0x559fb4623af0 0x559fb461d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:43.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5aa549a000 4 client.9888 mds_command: new command op to 9711 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:50:43.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5aa549a000 1 -- 172.21.15.83:0/24271911 --> [v2:172.21.15.83:6836/3575114599,v1:172.21.15.83:6837/3575114599] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559fb4403a50 con 0x559fb4623af0 2024-04-17T10:50:43.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5a9bfff640 1 --2- 172.21.15.83:0/24271911 >> [v2:172.21.15.83:6836/3575114599,v1:172.21.15.83:6837/3575114599] conn(0x559fb4623af0 0x559fb461d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:43.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5a7affd640 10 client.9888 ms_handle_connect on v2:172.21.15.83:6836/3575114599 2024-04-17T10:50:43.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5a9bfff640 1 --2- 172.21.15.83:0/24271911 >> [v2:172.21.15.83:6836/3575114599,v1:172.21.15.83:6837/3575114599] conn(0x559fb4623af0 0x559fb461d4b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74bdcd41dee7db12 server_cookie=d10175c2a9b1f899 in_seq=0 out_seq=0 2024-04-17T10:50:43.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5a7affd640 1 -- 172.21.15.83:0/24271911 <== mds.0 v2:172.21.15.83:6836/3575114599 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559fb4403a50 con 0x559fb4623af0 2024-04-17T10:50:43.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5a7affd640 10 client.9888 handle_command_reply: tid=0 2024-04-17T10:50:43.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.980+0000 7f5aa549a000 1 client.9888 shutdown 2024-04-17T10:50:43.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.984+0000 7f5aa549a000 1 -- 172.21.15.83:0/24271911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fb4662920 msgr2=0x559fb466a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:43.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.984+0000 7f5aa549a000 1 --2- 172.21.15.83:0/24271911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fb4662920 0x559fb466a1c0 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f5a90001060 tx=0x7f5a9001c1f0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.984+0000 7f5aa549a000 1 -- 172.21.15.83:0/24271911 shutdown_connections 2024-04-17T10:50:43.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.984+0000 7f5aa549a000 1 --2- 172.21.15.83:0/24271911 >> [v2:172.21.15.83:6836/3575114599,v1:172.21.15.83:6837/3575114599] conn(0x559fb4623af0 0x559fb461d4b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.984+0000 7f5aa549a000 1 --2- 172.21.15.83:0/24271911 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559fb4663240 0x559fb461caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.984+0000 7f5aa549a000 1 --2- 172.21.15.83:0/24271911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fb4662920 0x559fb466a1c0 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.984+0000 7f5aa549a000 1 --2- 172.21.15.83:0/24271911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559fb4618f30 0x559fb4669c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.984+0000 7f5a7affd640 0 client.9888 ms_handle_reset on v2:172.21.15.83:6836/3575114599 2024-04-17T10:50:43.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.984+0000 7f5aa549a000 1 -- 172.21.15.83:0/24271911 >> 172.21.15.83:0/24271911 conn(0x559fb461f2a0 msgr2=0x559fb46179e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:43.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.984+0000 7f5aa549a000 1 -- 172.21.15.83:0/24271911 shutdown_connections 2024-04-17T10:50:43.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.984+0000 7f5aa549a000 1 -- 172.21.15.83:0/24271911 wait complete. 2024-04-17T10:50:43.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.984+0000 7f5aa549a000 20 client.9888 trim_cache size 0 max 16384 2024-04-17T10:50:43.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.996+0000 7f5aa549a000 1 Processor -- start 2024-04-17T10:50:43.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:43.996+0000 7f5aa549a000 1 -- start start 2024-04-17T10:50:43.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5aa549a000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fb4618f30 0x559fb46c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:43.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5aa549a000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559fb4662920 0x559fb46c7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:43.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5aa549a000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559fb4663240 0x559fb46c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:43.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5aa549a000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c1ff450 con 0x559fb4618f30 2024-04-17T10:50:43.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5aa549a000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c1594b0 con 0x559fb4663240 2024-04-17T10:50:43.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5aa549a000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c1feea0 con 0x559fb4662920 2024-04-17T10:50:43.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5a9bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fb4618f30 0x559fb46c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:43.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5a9bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fb4618f30 0x559fb46c6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35460/0 (socket says 172.21.15.83:35460) 2024-04-17T10:50:43.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5a9bfff640 1 -- 172.21.15.83:0/376408583 learned_addr learned my addr 172.21.15.83:0/376408583 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:43.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5a9bfff640 1 -- 172.21.15.83:0/376408583 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559fb4662920 msgr2=0x559fb46c7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:43.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5a9bfff640 1 --2- 172.21.15.83:0/376408583 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559fb4662920 0x559fb46c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5a9bfff640 1 -- 172.21.15.83:0/376408583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559fb4663240 msgr2=0x559fb46c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:43.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5a9bfff640 1 --2- 172.21.15.83:0/376408583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559fb4663240 0x559fb46c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5a9bfff640 1 -- 172.21.15.83:0/376408583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fb4663070 con 0x559fb4618f30 2024-04-17T10:50:43.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5a9bfff640 1 --2- 172.21.15.83:0/376408583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fb4618f30 0x559fb46c6f40 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c00de80 tx=0x7f5a8c083250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3f77d2e4ad70df server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:43.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5a817fa640 1 -- 172.21.15.83:0/376408583 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c080070 con 0x559fb4618f30 2024-04-17T10:50:43.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5a817fa640 1 -- 172.21.15.83:0/376408583 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8c053a80 con 0x559fb4618f30 2024-04-17T10:50:43.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5a817fa640 1 -- 172.21.15.83:0/376408583 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c06d190 con 0x559fb4618f30 2024-04-17T10:50:43.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5aa549a000 1 -- 172.21.15.83:0/376408583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fb4618f30 msgr2=0x559fb46c6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:43.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5aa549a000 1 --2- 172.21.15.83:0/376408583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fb4618f30 0x559fb46c6f40 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c00de80 tx=0x7f5a8c083250 comp rx=0 tx=0).stop 2024-04-17T10:50:43.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5aa549a000 1 -- 172.21.15.83:0/376408583 shutdown_connections 2024-04-17T10:50:43.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5aa549a000 1 --2- 172.21.15.83:0/376408583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559fb4663240 0x559fb46c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5aa549a000 1 --2- 172.21.15.83:0/376408583 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559fb4662920 0x559fb46c7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5aa549a000 1 --2- 172.21.15.83:0/376408583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fb4618f30 0x559fb46c6f40 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:43.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5aa549a000 1 -- 172.21.15.83:0/376408583 >> 172.21.15.83:0/376408583 conn(0x559fb461f2a0 msgr2=0x559fb464ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:43.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5aa549a000 1 -- 172.21.15.83:0/376408583 shutdown_connections 2024-04-17T10:50:43.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5aa549a000 1 -- 172.21.15.83:0/376408583 wait complete. 2024-04-17T10:50:43.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5aa549a000 1 -- start start 2024-04-17T10:50:43.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5aa549a000 1 Processor -- start 2024-04-17T10:50:43.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5aa549a000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fb4618f30 0x559fb466ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:43.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5aa549a000 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559fb4662920 0x559fb46affe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:43.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5aa549a000 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559fb4663240 0x559fb46b0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:43.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5aa549a000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fb446c3d0 con 0x559fb4618f30 2024-04-17T10:50:43.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5aa549a000 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fb4523d50 con 0x559fb4663240 2024-04-17T10:50:43.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5aa549a000 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x559fb4402980 con 0x559fb4662920 2024-04-17T10:50:43.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5aa549a000 10 client.0 _ll_register_callbacks cb 0x7f5a94009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:50:43.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5a9bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fb4618f30 0x559fb466ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:43.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5a9bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fb4618f30 0x559fb466ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35468/0 (socket says 172.21.15.83:35468) 2024-04-17T10:50:43.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5a9bfff640 1 -- 172.21.15.83:0/1887821932 learned_addr learned my addr 172.21.15.83:0/1887821932 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:43.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5aa08ca640 1 --2- 172.21.15.83:0/1887821932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559fb4663240 0x559fb46b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:43.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5a9b7fe640 1 --2- 172.21.15.83:0/1887821932 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559fb4662920 0x559fb46affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:44.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5a9bfff640 1 -- 172.21.15.83:0/1887821932 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559fb4662920 msgr2=0x559fb46affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:44.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5a9bfff640 1 --2- 172.21.15.83:0/1887821932 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559fb4662920 0x559fb46affe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:44.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5a9bfff640 1 -- 172.21.15.83:0/1887821932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559fb4663240 msgr2=0x559fb46b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:44.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5a9bfff640 1 --2- 172.21.15.83:0/1887821932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559fb4663240 0x559fb46b0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:44.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5a9bfff640 1 -- 172.21.15.83:0/1887821932 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559fb44e9600 con 0x559fb4618f30 2024-04-17T10:50:44.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5a9bfff640 1 --2- 172.21.15.83:0/1887821932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fb4618f30 0x559fb466ea60 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c055710 tx=0x7f5a900aa000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f26274b875bcaa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:44.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5a7affd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-17T10:50:44.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5a7affd640 1 -- 172.21.15.83:0/1887821932 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c080040 con 0x559fb4618f30 2024-04-17T10:50:44.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5a7affd640 1 -- 172.21.15.83:0/1887821932 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a8c083400 con 0x559fb4618f30 2024-04-17T10:50:44.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5aa549a000 1 -- 172.21.15.83:0/1887821932 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a8c053a80 con 0x559fb4618f30 2024-04-17T10:50:44.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5a7affd640 1 -- 172.21.15.83:0/1887821932 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a8c003cd0 con 0x559fb4618f30 2024-04-17T10:50:44.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5aa549a000 10 client.9894 fetch_fsmap learned FSMap version 87 2024-04-17T10:50:44.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5aa549a000 1 -- 172.21.15.83:0/1887821932 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x559fb43fc9b0 con 0x559fb4618f30 2024-04-17T10:50:44.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.000+0000 7f5a7affd640 1 -- 172.21.15.83:0/1887821932 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8c053a80 con 0x559fb4618f30 2024-04-17T10:50:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5a7affd640 1 -- 172.21.15.83:0/1887821932 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 87) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5a8c07cce0 con 0x559fb4618f30 2024-04-17T10:50:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5aa549a000 10 client.9894 fetch_fsmap finished waiting for FSMap version 87 2024-04-17T10:50:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5aa549a000 10 client.9894 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T10:50:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5aa549a000 20 client.9894 populate_metadata read hostname 'smithi083' 2024-04-17T10:50:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5aa549a000 1 --2- 172.21.15.83:0/1887821932 >> [v2:172.21.15.83:6836/3575114599,v1:172.21.15.83:6837/3575114599] conn(0x559fb46b0a60 0x559fb46cc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5aa549a000 4 client.9894 mds_command: new command op to 9711 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T10:50:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5aa549a000 1 -- 172.21.15.83:0/1887821932 --> [v2:172.21.15.83:6836/3575114599,v1:172.21.15.83:6837/3575114599] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559fb44752e0 con 0x559fb46b0a60 2024-04-17T10:50:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5a9b7fe640 1 --2- 172.21.15.83:0/1887821932 >> [v2:172.21.15.83:6836/3575114599,v1:172.21.15.83:6837/3575114599] conn(0x559fb46b0a60 0x559fb46cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5a9b7fe640 1 --2- 172.21.15.83:0/1887821932 >> [v2:172.21.15.83:6836/3575114599,v1:172.21.15.83:6837/3575114599] conn(0x559fb46b0a60 0x559fb46cc060 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f3106dd3ab0cd2e server_cookie=84e862c0d5b76104 in_seq=0 out_seq=0 2024-04-17T10:50:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5a7affd640 10 client.9894 ms_handle_connect on v2:172.21.15.83:6836/3575114599 2024-04-17T10:50:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5a7affd640 1 -- 172.21.15.83:0/1887821932 <== mds.0 v2:172.21.15.83:6836/3575114599 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x559fb44752e0 con 0x559fb46b0a60 2024-04-17T10:50:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5a7affd640 10 client.9894 handle_command_reply: tid=0 2024-04-17T10:50:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5aa549a000 1 client.9894 shutdown 2024-04-17T10:50:44.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5aa549a000 1 -- 172.21.15.83:0/1887821932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fb4618f30 msgr2=0x559fb466ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:44.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5aa549a000 1 --2- 172.21.15.83:0/1887821932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fb4618f30 0x559fb466ea60 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c055710 tx=0x7f5a900aa000 comp rx=0 tx=0).stop 2024-04-17T10:50:44.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5aa549a000 1 -- 172.21.15.83:0/1887821932 shutdown_connections 2024-04-17T10:50:44.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5aa549a000 1 --2- 172.21.15.83:0/1887821932 >> [v2:172.21.15.83:6836/3575114599,v1:172.21.15.83:6837/3575114599] conn(0x559fb46b0a60 0x559fb46cc060 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:44.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5aa549a000 1 --2- 172.21.15.83:0/1887821932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559fb4663240 0x559fb46b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:44.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5a7affd640 0 client.9894 ms_handle_reset on v2:172.21.15.83:6836/3575114599 2024-04-17T10:50:44.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5aa549a000 1 --2- 172.21.15.83:0/1887821932 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x559fb4662920 0x559fb46affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:44.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5aa549a000 1 --2- 172.21.15.83:0/1887821932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559fb4618f30 0x559fb466ea60 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:44.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5aa549a000 1 -- 172.21.15.83:0/1887821932 >> 172.21.15.83:0/1887821932 conn(0x559fb461f2a0 msgr2=0x559fb461d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:44.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5aa549a000 1 -- 172.21.15.83:0/1887821932 shutdown_connections 2024-04-17T10:50:44.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5aa549a000 1 -- 172.21.15.83:0/1887821932 wait complete. 2024-04-17T10:50:44.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5aa549a000 20 client.9894 trim_cache size 0 max 16384 2024-04-17T10:50:44.003 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:50:44.003 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "3de9b7be-af7f-488a-969e-2b6b8db63bcd", 2024-04-17T10:50:44.003 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-04-17T10:50:44.003 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 9711, 2024-04-17T10:50:44.003 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-17T10:50:44.003 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-17T10:50:44.003 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-17T10:50:44.003 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 9.0982258070000004, 2024-04-17T10:50:44.003 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 87, 2024-04-17T10:50:44.003 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 140, 2024-04-17T10:50:44.003 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 140, 2024-04-17T10:50:44.004 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 13.492673725 2024-04-17T10:50:44.004 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:50:44.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5aa232d640 1 -- 172.21.15.83:0/3344157061 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5a88066f60 msgr2=0x7f5a88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:44.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5aa232d640 1 --2- 172.21.15.83:0/3344157061 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5a88066f60 0x7f5a88069420 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f5a900ae000 tx=0x7f5a90001a70 comp rx=0 tx=0).stop 2024-04-17T10:50:44.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5aa232d640 1 -- 172.21.15.83:0/3344157061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a84004810 msgr2=0x7f5a9c14df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:44.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5aa232d640 1 --2- 172.21.15.83:0/3344157061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a84004810 0x7f5a9c14df00 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c00d4c0 tx=0x7f5a8c00d980 comp rx=0 tx=0).stop 2024-04-17T10:50:44.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5aa232d640 1 -- 172.21.15.83:0/3344157061 shutdown_connections 2024-04-17T10:50:44.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5aa232d640 1 --2- 172.21.15.83:0/3344157061 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5a88066f60 0x7f5a88069420 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:44.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5aa232d640 1 --2- 172.21.15.83:0/3344157061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a9c157060 0x7f5a9c157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:44.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5aa232d640 1 --2- 172.21.15.83:0/3344157061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a9c156be0 0x7f5a9c157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:44.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5aa232d640 1 --2- 172.21.15.83:0/3344157061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a84004810 0x7f5a9c14df00 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:44.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5aa232d640 1 -- 172.21.15.83:0/3344157061 >> 172.21.15.83:0/3344157061 conn(0x7f5a9c0b15b0 msgr2=0x7f5a9c0afc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:44.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5aa232d640 1 -- 172.21.15.83:0/3344157061 shutdown_connections 2024-04-17T10:50:44.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.004+0000 7f5aa232d640 1 -- 172.21.15.83:0/3344157061 wait complete. 2024-04-17T10:50:44.012 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T10:50:44.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.124+0000 7fbaba1e9640 1 Processor -- start 2024-04-17T10:50:44.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.124+0000 7fbaba1e9640 1 -- start start 2024-04-17T10:50:44.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.124+0000 7fbaba1e9640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbab415de90 0x7fbab415efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:44.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.124+0000 7fbaba1e9640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbab40ce7f0 0x7fbab40cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:44.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.124+0000 7fbaba1e9640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab406bd90 con 0x7fbab40ceca0 2024-04-17T10:50:44.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.124+0000 7fbaba1e9640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbab406c030 con 0x7fbab415de90 2024-04-17T10:50:44.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.124+0000 7fbaba1e9640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbab406c310 con 0x7fbab40ce7f0 2024-04-17T10:50:44.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.124+0000 7fbab37fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbab415de90 0x7fbab415efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:44.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.124+0000 7fbab2ffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbab40ceca0 0x7fbab40ce0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57220/0 (socket says 172.21.15.83:57220) 2024-04-17T10:50:44.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.124+0000 7fbab2ffd640 1 -- 172.21.15.83:0/4076239214 learned_addr learned my addr 172.21.15.83:0/4076239214 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:50:44.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.124+0000 7fbab3fff640 1 --2- 172.21.15.83:0/4076239214 >> v2:172.21.15.167:3300/0 conn(0x7fbab40ce7f0 0x7fbab40cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:44.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.124+0000 7fbab37fe640 1 -- 172.21.15.83:0/4076239214 >> v2:172.21.15.167:3300/0 conn(0x7fbab40ce7f0 msgr2=0x7fbab40cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:50:44.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.124+0000 7fbab37fe640 1 --2- 172.21.15.83:0/4076239214 >> v2:172.21.15.167:3300/0 conn(0x7fbab40ce7f0 0x7fbab40cf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:44.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.124+0000 7fbab37fe640 1 -- 172.21.15.83:0/4076239214 >> v1:172.21.15.83:6789/0 conn(0x7fbab40ceca0 legacy=0x7fbab40ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:44.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.124+0000 7fbab27fc640 1 -- 172.21.15.83:0/4076239214 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2602417613 0 0) 0x7fbab406bd90 con 0x7fbab40ceca0 2024-04-17T10:50:44.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.124+0000 7fbab37fe640 1 -- 172.21.15.83:0/4076239214 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab406c5f0 con 0x7fbab415de90 2024-04-17T10:50:44.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fbab37fe640 1 --2- 172.21.15.83:0/4076239214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbab415de90 0x7fbab415efb0 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7fbab4162b40 tx=0x7fbaa801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c7a2da360e5bc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:44.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fbab27fc640 1 -- 172.21.15.83:0/4076239214 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa80b0070 con 0x7fbab415de90 2024-04-17T10:50:44.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fbab27fc640 1 -- 172.21.15.83:0/4076239214 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa80ab600 con 0x7fbab415de90 2024-04-17T10:50:44.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fbab27fc640 1 -- 172.21.15.83:0/4076239214 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa80ab9d0 con 0x7fbab415de90 2024-04-17T10:50:44.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fbaba1e9640 1 -- 172.21.15.83:0/4076239214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbab415de90 msgr2=0x7fbab415efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:44.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fbaba1e9640 1 --2- 172.21.15.83:0/4076239214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbab415de90 0x7fbab415efb0 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7fbab4162b40 tx=0x7fbaa801c450 comp rx=0 tx=0).stop 2024-04-17T10:50:44.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fbaba1e9640 1 -- 172.21.15.83:0/4076239214 shutdown_connections 2024-04-17T10:50:44.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fbaba1e9640 1 --2- 172.21.15.83:0/4076239214 >> v2:172.21.15.167:3300/0 conn(0x7fbab40ce7f0 0x7fbab40cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:44.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fbaba1e9640 1 --2- 172.21.15.83:0/4076239214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbab415de90 0x7fbab415efb0 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:44.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fbaba1e9640 1 -- 172.21.15.83:0/4076239214 >> 172.21.15.83:0/4076239214 conn(0x7fbab40d17b0 msgr2=0x7fbab40d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:44.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fbaba1e9640 1 -- 172.21.15.83:0/4076239214 shutdown_connections 2024-04-17T10:50:44.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fbaba1e9640 1 -- 172.21.15.83:0/4076239214 wait complete. 2024-04-17T10:50:44.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fbaba1e9640 1 Processor -- start 2024-04-17T10:50:44.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fbaba1e9640 1 -- start start 2024-04-17T10:50:44.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fbaba1e9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbab40ce7f0 0x7fbab40cb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:44.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fbaba1e9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbab40ceca0 0x7fbab40ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:44.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fbaba1e9640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbab415de90 0x7fbab4155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:44.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fbaba1e9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab415f670 con 0x7fbab40ceca0 2024-04-17T10:50:44.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fbaba1e9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab415f4f0 con 0x7fbab40ce7f0 2024-04-17T10:50:44.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fbaba1e9640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab40d0620 con 0x7fbab415de90 2024-04-17T10:50:44.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fbab2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbab40ceca0 0x7fbab40ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:44.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fbab2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbab40ceca0 0x7fbab40ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35488/0 (socket says 172.21.15.83:35488) 2024-04-17T10:50:44.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fbab2ffd640 1 -- 172.21.15.83:0/3326052409 learned_addr learned my addr 172.21.15.83:0/3326052409 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:50:44.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fbab2ffd640 1 -- 172.21.15.83:0/3326052409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbab415de90 msgr2=0x7fbab4155270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:44.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fbab2ffd640 1 --2- 172.21.15.83:0/3326052409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbab415de90 0x7fbab4155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:44.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fbab2ffd640 1 -- 172.21.15.83:0/3326052409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbab40ce7f0 msgr2=0x7fbab40cb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:44.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fbab2ffd640 1 --2- 172.21.15.83:0/3326052409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbab40ce7f0 0x7fbab40cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:44.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fbab2ffd640 1 -- 172.21.15.83:0/3326052409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab41549a0 con 0x7fbab40ceca0 2024-04-17T10:50:44.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fbab2ffd640 1 --2- 172.21.15.83:0/3326052409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbab40ceca0 0x7fbab40ccff0 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7fbaa400c480 tx=0x7fbaa400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:44.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fba9bfff640 1 -- 172.21.15.83:0/3326052409 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4018070 con 0x7fbab40ceca0 2024-04-17T10:50:44.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fba9bfff640 1 -- 172.21.15.83:0/3326052409 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa40074e0 con 0x7fbab40ceca0 2024-04-17T10:50:44.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fbaba1e9640 1 -- 172.21.15.83:0/3326052409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab4154c80 con 0x7fbab40ceca0 2024-04-17T10:50:44.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fba9bfff640 1 -- 172.21.15.83:0/3326052409 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4004a30 con 0x7fbab40ceca0 2024-04-17T10:50:44.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.128+0000 7fbaba1e9640 1 -- 172.21.15.83:0/3326052409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab40ccad0 con 0x7fbab40ceca0 2024-04-17T10:50:44.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.132+0000 7fbaba1e9640 1 -- 172.21.15.83:0/3326052409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbab4155070 con 0x7fbab40ceca0 2024-04-17T10:50:44.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.132+0000 7fba9bfff640 1 -- 172.21.15.83:0/3326052409 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbaa401b070 con 0x7fbab40ceca0 2024-04-17T10:50:44.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.132+0000 7fba9bfff640 1 --2- 172.21.15.83:0/3326052409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbaa0066f60 0x7fbaa0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:50:44.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.132+0000 7fba9bfff640 1 -- 172.21.15.83:0/3326052409 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbaa4084350 con 0x7fbab40ceca0 2024-04-17T10:50:44.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.132+0000 7fbab37fe640 1 --2- 172.21.15.83:0/3326052409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbaa0066f60 0x7fbaa0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:50:44.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.132+0000 7fbab37fe640 1 --2- 172.21.15.83:0/3326052409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbaa0066f60 0x7fbaa0069420 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fbaa80ad000 tx=0x7fbaa8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:50:44.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.136+0000 7fba9bfff640 1 -- 172.21.15.83:0/3326052409 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbab4155070 con 0x7fbab40ceca0 2024-04-17T10:50:44.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.324+0000 7fbaba1e9640 1 -- 172.21.15.83:0/3326052409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbab4074f90 con 0x7fbab40ceca0 2024-04-17T10:50:44.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.324+0000 7fba9bfff640 1 -- 172.21.15.83:0/3326052409 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v87) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbaa40553f0 con 0x7fbab40ceca0 2024-04-17T10:50:44.319 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T10:50:44.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.328+0000 7fbaba1e9640 1 -- 172.21.15.83:0/3326052409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbaa0066f60 msgr2=0x7fbaa0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:44.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.328+0000 7fbaba1e9640 1 --2- 172.21.15.83:0/3326052409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbaa0066f60 0x7fbaa0069420 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fbaa80ad000 tx=0x7fbaa8008040 comp rx=0 tx=0).stop 2024-04-17T10:50:44.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.328+0000 7fbaba1e9640 1 -- 172.21.15.83:0/3326052409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbab40ceca0 msgr2=0x7fbab40ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:50:44.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.328+0000 7fbaba1e9640 1 --2- 172.21.15.83:0/3326052409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbab40ceca0 0x7fbab40ccff0 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7fbaa400c480 tx=0x7fbaa400c940 comp rx=0 tx=0).stop 2024-04-17T10:50:44.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.328+0000 7fbaba1e9640 1 -- 172.21.15.83:0/3326052409 shutdown_connections 2024-04-17T10:50:44.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.328+0000 7fbaba1e9640 1 --2- 172.21.15.83:0/3326052409 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbaa0066f60 0x7fbaa0069420 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:44.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.328+0000 7fbaba1e9640 1 --2- 172.21.15.83:0/3326052409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbab415de90 0x7fbab4155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:44.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.328+0000 7fbaba1e9640 1 --2- 172.21.15.83:0/3326052409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbab40ceca0 0x7fbab40ccff0 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:44.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.328+0000 7fbaba1e9640 1 --2- 172.21.15.83:0/3326052409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbab40ce7f0 0x7fbab40cb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:50:44.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.328+0000 7fbaba1e9640 1 -- 172.21.15.83:0/3326052409 >> 172.21.15.83:0/3326052409 conn(0x7fbab40d17b0 msgr2=0x7fbab406ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:50:44.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.328+0000 7fbaba1e9640 1 -- 172.21.15.83:0/3326052409 shutdown_connections 2024-04-17T10:50:44.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:44.328+0000 7fbaba1e9640 1 -- 172.21.15.83:0/3326052409 wait complete. 2024-04-17T10:50:44.336 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T10:50:44.336 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T10:50:44.337 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-04-17T10:50:44.337 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T10:50:44.337 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T10:50:44.337 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T10:50:44.337 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T10:50:44.337 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T10:50:44.337 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:50:44.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:44.337 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-04-17T10:50:44.343 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T10:50:44.343 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T10:50:44.344 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T10:50:44.344 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:50:44.344 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-17T10:50:44.344 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:50:44.344 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:50:44.344 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-04-17T10:50:44.344 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-04-17T10:50:44.344 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:50:44.344 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-04-17T10:50:44.344 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-04-17T10:50:44.344 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:50:44.344 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:50:44.344 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T10:50:44.344 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:50:44.345 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-17T10:50:44.345 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 41740sec preferred_lft 41740sec 2024-04-17T10:50:44.345 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-04-17T10:50:44.345 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:50:44.345 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T10:50:44.345 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:37:0e:23:b2 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:50:44.345 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T10:50:44.345 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:50:44.345 INFO:teuthology.orchestra.run.smithi083.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T10:50:44.345 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-17T10:50:44.345 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T10:50:44.345 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:50:44.345 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-17T10:50:44.345 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:50:44.346 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:50:44.346 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:50:44.346 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-17T10:50:44.346 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:50:44.346 INFO:teuthology.orchestra.run.smithi083.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:50:44.346 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:50:44.346 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-17T10:50:44.346 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:50:44.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:44.346 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T10:50:44.397 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-04-17T10:50:44.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:44.399 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-04-17T10:50:44.452 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-04-17T10:50:44.452 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T10:50:44.452 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-17T10:50:44.452 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-17T10:50:44.452 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-17T10:50:44.452 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T10:50:44.452 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-17T10:50:44.452 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:50:44.453 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:50:44.453 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T10:50:44.453 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T10:50:44.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:44.453 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2024-04-17T10:50:44.453 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:50:44.453 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-17T10:50:44.453 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-17T10:50:44.453 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-17T10:50:44.453 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:50:44.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:44.527 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-17T10:50:44.576 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-17T10:50:44.576 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T10:50:44.576 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:50:44.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:44.576 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-04-17T10:50:44.630 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-17T10:50:44.631 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:50:44.631 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T10:50:44.680 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:50:44.680 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:50:44.681 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:50:44.727 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:50:44.785 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:50:44.833 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:50:44.833 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:50:44.833 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:50:44.833 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:50:44.833 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:50:44.833 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:50:44.833 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:50:44.833 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:50:44.833 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:50:44.833 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:50:44.834 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:50:44.834 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:50:44.834 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:50:44.834 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:50:44.834 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:50:44.834 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:50:44.834 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:50:44.834 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:50:44.834 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:50:44.834 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:50:44.834 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:50:44.834 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:50:44.834 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:50:44.834 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:50:44.835 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:50:44.835 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:50:44.835 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:50:44.835 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:50:44.835 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:50:44.835 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:50:44.835 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:50:44.835 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:50:44.835 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:50:44.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:44.836 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:50:44.884 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T10:50:44.884 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-17T10:50:44.929 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:50:44.942 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:50:44.993 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:50:44.993 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:50:44.993 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:50:44.993 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:50:44.993 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:50:44.993 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:50:44.994 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:50:44.994 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:50:44.994 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:50:44.994 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:50:44.994 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:50:44.994 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:50:44.994 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:50:44.994 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:50:44.994 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:50:44.994 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:50:44.994 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:50:44.994 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:50:44.994 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:50:44.994 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:50:44.995 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:50:44.995 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:50:44.995 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:50:44.995 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:50:44.995 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:50:44.995 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:50:44.995 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:50:44.995 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:50:44.995 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:50:44.995 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:50:44.995 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:50:44.995 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:50:44.995 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:50:44.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:44.996 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:50:45.001 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:2024-04-17T10:50:45.008+0000 7f2137c75580 -1 init, newargv = 0x55d5103a0b90 newargc=15 2024-04-17T10:50:45.001 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[83273]: starting ceph client 2024-04-17T10:50:45.006 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[83273]: starting fuse 2024-04-17T10:50:46.106 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-17T10:50:46.122 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:50:46.169 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-17T10:50:46.169 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-17T10:50:46.169 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-17T10:50:46.169 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-17T10:50:46.169 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-17T10:50:46.169 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-17T10:50:46.169 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-17T10:50:46.169 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-17T10:50:46.169 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-04-17T10:50:46.169 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-17T10:50:46.169 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-17T10:50:46.169 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-17T10:50:46.169 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:50:46.170 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-17T10:50:46.170 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-17T10:50:46.170 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-17T10:50:46.170 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-17T10:50:46.170 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-04-17T10:50:46.170 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-17T10:50:46.170 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-17T10:50:46.170 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:50:46.170 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-04-17T10:50:46.170 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-17T10:50:46.170 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-17T10:50:46.170 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:50:46.170 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:50:46.170 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:50:46.171 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:50:46.171 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-17T10:50:46.171 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:50:46.171 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:50:46.171 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:50:46.171 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:50:46.171 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-17T10:50:46.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:46.172 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-17T10:50:46.220 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-04-17T10:50:46.220 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-17T10:50:46.220 DEBUG:teuthology.orchestra.run.smithi083:> sudo stdin-killer -- python3 -c ' 2024-04-17T10:50:46.220 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2024-04-17T10:50:46.220 DEBUG:teuthology.orchestra.run.smithi083:> import re 2024-04-17T10:50:46.220 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-04-17T10:50:46.220 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2024-04-17T10:50:46.220 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:50:46.221 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2024-04-17T10:50:46.221 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-17T10:50:46.221 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2024-04-17T10:50:46.221 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-17T10:50:46.221 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:50:46.221 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2024-04-17T10:50:46.221 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2024-04-17T10:50:46.221 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2024-04-17T10:50:46.221 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2024-04-17T10:50:46.221 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:50:46.221 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2024-04-17T10:50:46.221 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T10:50:46.221 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T10:50:46.221 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T10:50:46.221 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2024-04-17T10:50:46.222 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2024-04-17T10:50:46.222 DEBUG:teuthology.orchestra.run.smithi083:> return f 2024-04-17T10:50:46.222 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T10:50:46.222 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-17T10:50:46.222 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.0")) 2024-04-17T10:50:46.222 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-04-17T10:50:46.342 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.0.83273.asok 2024-04-17T10:50:46.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:50:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:50:46.354 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.83273.asok 2024-04-17T10:50:46.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:46.354 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.83273.asok status 2024-04-17T10:50:46.497 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:50:46.497 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2024-04-17T10:50:46.497 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-17T10:50:46.497 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-17T10:50:46.497 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "0", 2024-04-17T10:50:46.497 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-04-17T10:50:46.497 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-17T10:50:46.498 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "83273", 2024-04-17T10:50:46.498 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-04-17T10:50:46.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:50:46.498 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2024-04-17T10:50:46.498 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2024-04-17T10:50:46.498 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 9909, 2024-04-17T10:50:46.498 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2024-04-17T10:50:46.498 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-04-17T10:50:46.498 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-04-17T10:50:46.498 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 9909 2024-04-17T10:50:46.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:50:46.498 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-04-17T10:50:46.498 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-17T10:50:46.499 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:50:46.499 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2666966067 2024-04-17T10:50:46.499 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-17T10:50:46.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:50:46.499 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-04-17T10:50:46.499 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-17T10:50:46.499 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:50:46.499 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2666966067 2024-04-17T10:50:46.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-17T10:50:46.499 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.9909 192.168.0.1:0/2666966067", 2024-04-17T10:50:46.499 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.1:0/2666966067", 2024-04-17T10:50:46.499 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 1, 2024-04-17T10:50:46.499 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 87, 2024-04-17T10:50:46.499 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 140, 2024-04-17T10:50:46.500 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2024-04-17T10:50:46.500 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2024-04-17T10:50:46.500 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2024-04-17T10:50:46.500 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:50:46.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:46.511 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2024-04-17T10:50:46.511 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-17T10:50:46.517 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2024-04-17T10:50:46.518 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-17T10:50:46.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:46.518 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:50:46.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:46.571 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:50:46.571 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:50:46.571 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.0 2024-04-17T10:50:46.571 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:50:46.571 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:50:46.571 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:50:46.571 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:50:46.655 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:46.655 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:46.655 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212148 s, 49.4 MB/s 2024-04-17T10:50:46.674 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:46.674 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:46.674 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01667 s, 62.9 MB/s 2024-04-17T10:50:46.695 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:46.695 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:46.695 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173603 s, 60.4 MB/s 2024-04-17T10:50:46.713 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:46.713 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:46.713 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014939 s, 70.2 MB/s 2024-04-17T10:50:46.730 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:46.730 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:46.731 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146816 s, 71.4 MB/s 2024-04-17T10:50:46.746 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:46.746 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:46.746 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124405 s, 84.3 MB/s 2024-04-17T10:50:46.764 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:46.764 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:46.764 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155294 s, 67.5 MB/s 2024-04-17T10:50:46.780 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:46.780 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:46.780 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131032 s, 80.0 MB/s 2024-04-17T10:50:46.798 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:46.798 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:46.798 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152 s, 69.0 MB/s 2024-04-17T10:50:46.813 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:46.813 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:46.813 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116661 s, 89.9 MB/s 2024-04-17T10:50:46.830 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:46.830 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:46.830 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137012 s, 76.5 MB/s 2024-04-17T10:50:46.847 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:46.847 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:46.847 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140268 s, 74.8 MB/s 2024-04-17T10:50:46.863 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:46.863 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:46.863 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132413 s, 79.2 MB/s 2024-04-17T10:50:46.880 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:46.880 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:46.880 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143529 s, 73.1 MB/s 2024-04-17T10:50:46.895 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:46.895 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:46.895 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123981 s, 84.6 MB/s 2024-04-17T10:50:46.913 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:46.913 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:46.914 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152088 s, 68.9 MB/s 2024-04-17T10:50:46.931 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:46.931 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:46.931 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149032 s, 70.4 MB/s 2024-04-17T10:50:46.948 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:46.949 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:46.949 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146095 s, 71.8 MB/s 2024-04-17T10:50:46.964 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:46.964 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:46.964 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127397 s, 82.3 MB/s 2024-04-17T10:50:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109779 s, 95.5 MB/s 2024-04-17T10:50:46.996 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:46.996 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:46.996 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150021 s, 69.9 MB/s 2024-04-17T10:50:47.011 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.012 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.012 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131901 s, 79.5 MB/s 2024-04-17T10:50:47.028 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.028 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.029 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140382 s, 74.7 MB/s 2024-04-17T10:50:47.043 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.043 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.043 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116849 s, 89.7 MB/s 2024-04-17T10:50:47.060 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.060 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.060 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142354 s, 73.7 MB/s 2024-04-17T10:50:47.081 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.082 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.082 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130994 s, 80.0 MB/s 2024-04-17T10:50:47.090 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.090 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.090 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108719 s, 96.4 MB/s 2024-04-17T10:50:47.105 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.105 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.105 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128214 s, 81.8 MB/s 2024-04-17T10:50:47.120 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.120 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.120 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114431 s, 91.6 MB/s 2024-04-17T10:50:47.136 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.136 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.136 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136528 s, 76.8 MB/s 2024-04-17T10:50:47.151 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.151 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.151 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114018 s, 92.0 MB/s 2024-04-17T10:50:47.169 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.169 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.169 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134259 s, 78.1 MB/s 2024-04-17T10:50:47.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:47.169 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:50:47.170 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:50:47.170 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.1 2024-04-17T10:50:47.170 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:50:47.170 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:50:47.170 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:50:47.170 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:50:47.236 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.236 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.236 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117956 s, 88.9 MB/s 2024-04-17T10:50:47.251 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.251 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.251 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114862 s, 91.3 MB/s 2024-04-17T10:50:47.267 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.267 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.267 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134379 s, 78.0 MB/s 2024-04-17T10:50:47.281 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.281 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.281 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112119 s, 93.5 MB/s 2024-04-17T10:50:47.294 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.294 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.294 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101957 s, 103 MB/s 2024-04-17T10:50:47.311 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.311 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.311 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144358 s, 72.6 MB/s 2024-04-17T10:50:47.325 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.325 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.325 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011098 s, 94.5 MB/s 2024-04-17T10:50:47.341 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.341 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.341 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131907 s, 79.5 MB/s 2024-04-17T10:50:47.356 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.356 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.356 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125025 s, 83.9 MB/s 2024-04-17T10:50:47.373 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.373 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.373 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137208 s, 76.4 MB/s 2024-04-17T10:50:47.387 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.387 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.387 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011299 s, 92.8 MB/s 2024-04-17T10:50:47.405 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.405 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.405 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139885 s, 75.0 MB/s 2024-04-17T10:50:47.418 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.418 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.419 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105773 s, 99.1 MB/s 2024-04-17T10:50:47.435 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.435 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.435 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140066 s, 74.9 MB/s 2024-04-17T10:50:47.450 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.450 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.450 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118556 s, 88.4 MB/s 2024-04-17T10:50:47.467 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.467 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.467 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141152 s, 74.3 MB/s 2024-04-17T10:50:47.480 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.480 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.480 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104649 s, 100 MB/s 2024-04-17T10:50:47.497 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.497 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.497 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014429 s, 72.7 MB/s 2024-04-17T10:50:47.510 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.510 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.510 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010386 s, 101 MB/s 2024-04-17T10:50:47.527 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.527 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.527 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135778 s, 77.2 MB/s 2024-04-17T10:50:47.543 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.543 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.543 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139371 s, 75.2 MB/s 2024-04-17T10:50:47.561 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.561 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.561 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148445 s, 70.6 MB/s 2024-04-17T10:50:47.579 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.579 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.579 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147895 s, 70.9 MB/s 2024-04-17T10:50:47.597 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.597 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.597 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146692 s, 71.5 MB/s 2024-04-17T10:50:47.614 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.614 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.614 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137586 s, 76.2 MB/s 2024-04-17T10:50:47.629 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.629 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.629 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120104 s, 87.3 MB/s 2024-04-17T10:50:47.644 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.644 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.644 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120971 s, 86.7 MB/s 2024-04-17T10:50:47.659 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.659 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.659 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126354 s, 83.0 MB/s 2024-04-17T10:50:47.677 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.677 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.677 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151906 s, 69.0 MB/s 2024-04-17T10:50:47.692 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.692 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.692 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117165 s, 89.5 MB/s 2024-04-17T10:50:47.709 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.710 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.710 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148008 s, 70.8 MB/s 2024-04-17T10:50:47.727 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.727 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.728 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014302 s, 73.3 MB/s 2024-04-17T10:50:47.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:47.729 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:50:47.729 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:50:47.729 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.2 2024-04-17T10:50:47.730 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:50:47.730 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:50:47.730 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:50:47.730 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:50:47.796 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.796 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.796 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116325 s, 90.1 MB/s 2024-04-17T10:50:47.810 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.810 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.810 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011185 s, 93.7 MB/s 2024-04-17T10:50:47.826 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.826 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.826 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131737 s, 79.6 MB/s 2024-04-17T10:50:47.843 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.843 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.843 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132924 s, 78.9 MB/s 2024-04-17T10:50:47.857 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.857 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.857 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114606 s, 91.5 MB/s 2024-04-17T10:50:47.872 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.873 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.873 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125779 s, 83.4 MB/s 2024-04-17T10:50:47.889 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.889 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.890 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138379 s, 75.8 MB/s 2024-04-17T10:50:47.906 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.906 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.907 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135953 s, 77.1 MB/s 2024-04-17T10:50:47.923 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.923 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.924 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125019 s, 83.9 MB/s 2024-04-17T10:50:47.938 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.938 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.938 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011747 s, 89.3 MB/s 2024-04-17T10:50:47.955 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.955 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.955 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140813 s, 74.5 MB/s 2024-04-17T10:50:47.971 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.971 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.972 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131209 s, 79.9 MB/s 2024-04-17T10:50:47.985 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:47.985 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:47.985 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103957 s, 101 MB/s 2024-04-17T10:50:48.001 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.001 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.001 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126467 s, 82.9 MB/s 2024-04-17T10:50:48.016 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.016 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.016 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01279 s, 82.0 MB/s 2024-04-17T10:50:48.033 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.033 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.033 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013803 s, 76.0 MB/s 2024-04-17T10:50:48.048 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.048 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.048 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129268 s, 81.1 MB/s 2024-04-17T10:50:48.065 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.065 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.065 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135111 s, 77.6 MB/s 2024-04-17T10:50:48.079 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.079 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.080 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120751 s, 86.8 MB/s 2024-04-17T10:50:48.097 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.097 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.097 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143627 s, 73.0 MB/s 2024-04-17T10:50:48.117 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.117 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.117 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167141 s, 62.7 MB/s 2024-04-17T10:50:48.133 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.133 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.133 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121858 s, 86.0 MB/s 2024-04-17T10:50:48.150 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.150 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.150 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132559 s, 79.1 MB/s 2024-04-17T10:50:48.165 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.166 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.166 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119639 s, 87.6 MB/s 2024-04-17T10:50:48.179 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.179 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.179 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01083 s, 96.8 MB/s 2024-04-17T10:50:48.195 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.195 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.195 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126426 s, 82.9 MB/s 2024-04-17T10:50:48.213 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.213 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.213 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148704 s, 70.5 MB/s 2024-04-17T10:50:48.229 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.229 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.229 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013172 s, 79.6 MB/s 2024-04-17T10:50:48.246 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.246 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.246 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142517 s, 73.6 MB/s 2024-04-17T10:50:48.264 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.264 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.264 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140095 s, 74.8 MB/s 2024-04-17T10:50:48.282 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.282 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.282 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142476 s, 73.6 MB/s 2024-04-17T10:50:48.302 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.302 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.302 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156021 s, 67.2 MB/s 2024-04-17T10:50:48.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:48.303 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:50:48.303 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:50:48.303 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.3 2024-04-17T10:50:48.303 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:50:48.303 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:50:48.303 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:50:48.303 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:50:48.377 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.377 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.377 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142255 s, 73.7 MB/s 2024-04-17T10:50:48.392 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.392 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.393 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119914 s, 87.4 MB/s 2024-04-17T10:50:48.407 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.407 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.407 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116296 s, 90.2 MB/s 2024-04-17T10:50:48.423 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.423 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.423 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132715 s, 79.0 MB/s 2024-04-17T10:50:48.437 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.437 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.437 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111491 s, 94.1 MB/s 2024-04-17T10:50:48.449 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.449 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.449 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00945565 s, 111 MB/s 2024-04-17T10:50:48.465 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.465 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.465 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128553 s, 81.6 MB/s 2024-04-17T10:50:48.481 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.481 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.481 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129713 s, 80.8 MB/s 2024-04-17T10:50:48.496 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.497 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.497 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130585 s, 80.3 MB/s 2024-04-17T10:50:48.513 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.513 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.513 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013581 s, 77.2 MB/s 2024-04-17T10:50:48.528 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.528 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.528 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127483 s, 82.3 MB/s 2024-04-17T10:50:48.545 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.545 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.545 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142515 s, 73.6 MB/s 2024-04-17T10:50:48.558 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.558 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.558 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104383 s, 100 MB/s 2024-04-17T10:50:48.576 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.576 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.576 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147641 s, 71.0 MB/s 2024-04-17T10:50:48.592 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.592 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.592 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131252 s, 79.9 MB/s 2024-04-17T10:50:48.607 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.608 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.608 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131017 s, 80.0 MB/s 2024-04-17T10:50:48.622 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.622 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.622 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118942 s, 88.2 MB/s 2024-04-17T10:50:48.639 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.639 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.639 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138423 s, 75.8 MB/s 2024-04-17T10:50:48.653 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.654 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.654 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011921 s, 88.0 MB/s 2024-04-17T10:50:48.669 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.670 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.670 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131663 s, 79.6 MB/s 2024-04-17T10:50:48.684 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.684 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.684 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120969 s, 86.7 MB/s 2024-04-17T10:50:48.697 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.697 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.697 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00975756 s, 107 MB/s 2024-04-17T10:50:48.712 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.712 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.712 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012722 s, 82.4 MB/s 2024-04-17T10:50:48.729 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.729 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.729 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136016 s, 77.1 MB/s 2024-04-17T10:50:48.741 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.741 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.741 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00994035 s, 105 MB/s 2024-04-17T10:50:48.757 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.757 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.757 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126151 s, 83.1 MB/s 2024-04-17T10:50:48.773 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.773 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.773 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134248 s, 78.1 MB/s 2024-04-17T10:50:48.791 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.791 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.791 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151513 s, 69.2 MB/s 2024-04-17T10:50:48.806 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.806 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.806 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121773 s, 86.1 MB/s 2024-04-17T10:50:48.822 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.822 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.822 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132434 s, 79.2 MB/s 2024-04-17T10:50:48.838 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.838 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.838 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125982 s, 83.2 MB/s 2024-04-17T10:50:48.856 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.857 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.857 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158062 s, 66.3 MB/s 2024-04-17T10:50:48.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:48.858 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:50:48.858 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:50:48.858 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.4 2024-04-17T10:50:48.859 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:50:48.859 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:50:48.859 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:50:48.859 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:50:48.924 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.924 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.924 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133256 s, 78.7 MB/s 2024-04-17T10:50:48.942 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.942 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.942 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146383 s, 71.6 MB/s 2024-04-17T10:50:48.956 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.956 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.956 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110208 s, 95.1 MB/s 2024-04-17T10:50:48.970 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.970 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.970 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010829 s, 96.8 MB/s 2024-04-17T10:50:48.983 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.984 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.984 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112165 s, 93.5 MB/s 2024-04-17T10:50:48.999 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:48.999 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:48.999 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123359 s, 85.0 MB/s 2024-04-17T10:50:49.015 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.015 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.015 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141262 s, 74.2 MB/s 2024-04-17T10:50:49.031 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.031 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.031 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131179 s, 79.9 MB/s 2024-04-17T10:50:49.046 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.046 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.047 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127801 s, 82.0 MB/s 2024-04-17T10:50:49.062 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.062 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.062 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124311 s, 84.4 MB/s 2024-04-17T10:50:49.077 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.077 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.077 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123917 s, 84.6 MB/s 2024-04-17T10:50:49.093 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.093 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.093 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130241 s, 80.5 MB/s 2024-04-17T10:50:49.108 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.108 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.108 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122356 s, 85.7 MB/s 2024-04-17T10:50:49.124 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.124 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.124 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132065 s, 79.4 MB/s 2024-04-17T10:50:49.139 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.139 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.139 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118786 s, 88.3 MB/s 2024-04-17T10:50:49.154 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.154 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.154 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128391 s, 81.7 MB/s 2024-04-17T10:50:49.171 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.171 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.171 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013525 s, 77.5 MB/s 2024-04-17T10:50:49.184 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.185 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.185 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107576 s, 97.5 MB/s 2024-04-17T10:50:49.199 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.200 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.200 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124478 s, 84.2 MB/s 2024-04-17T10:50:49.217 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.217 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.217 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142145 s, 73.8 MB/s 2024-04-17T10:50:49.235 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.235 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.235 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141851 s, 73.9 MB/s 2024-04-17T10:50:49.253 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.253 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.253 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143459 s, 73.1 MB/s 2024-04-17T10:50:49.270 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.270 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.270 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138877 s, 75.5 MB/s 2024-04-17T10:50:49.287 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.287 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.287 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133846 s, 78.3 MB/s 2024-04-17T10:50:49.302 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.302 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.302 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123868 s, 84.7 MB/s 2024-04-17T10:50:49.318 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.318 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.318 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130051 s, 80.6 MB/s 2024-04-17T10:50:49.333 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.333 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.333 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118656 s, 88.4 MB/s 2024-04-17T10:50:49.349 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.349 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.349 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132927 s, 78.9 MB/s 2024-04-17T10:50:49.364 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.364 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.364 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121977 s, 86.0 MB/s 2024-04-17T10:50:49.379 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.379 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.379 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116665 s, 89.9 MB/s 2024-04-17T10:50:49.396 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.396 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.396 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133912 s, 78.3 MB/s 2024-04-17T10:50:49.412 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.412 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.413 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139043 s, 75.4 MB/s 2024-04-17T10:50:49.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:49.414 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:50:49.414 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:50:49.414 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.5 2024-04-17T10:50:49.414 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:50:49.414 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:50:49.414 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:50:49.414 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:50:49.484 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.484 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.484 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160946 s, 65.2 MB/s 2024-04-17T10:50:49.500 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.500 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.501 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133434 s, 78.6 MB/s 2024-04-17T10:50:49.519 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.519 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.519 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155275 s, 67.5 MB/s 2024-04-17T10:50:49.536 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.536 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.536 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138318 s, 75.8 MB/s 2024-04-17T10:50:49.554 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.554 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.554 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153872 s, 68.1 MB/s 2024-04-17T10:50:49.568 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.568 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.568 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113495 s, 92.4 MB/s 2024-04-17T10:50:49.583 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.583 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.583 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123898 s, 84.6 MB/s 2024-04-17T10:50:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125761 s, 83.4 MB/s 2024-04-17T10:50:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143227 s, 73.2 MB/s 2024-04-17T10:50:49.631 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.631 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.631 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124932 s, 83.9 MB/s 2024-04-17T10:50:49.647 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.647 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.647 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128937 s, 81.3 MB/s 2024-04-17T10:50:49.664 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.664 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.665 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142772 s, 73.4 MB/s 2024-04-17T10:50:49.682 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.683 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.683 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146167 s, 71.7 MB/s 2024-04-17T10:50:49.700 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.700 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.700 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146575 s, 71.5 MB/s 2024-04-17T10:50:49.718 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.718 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.718 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140353 s, 74.7 MB/s 2024-04-17T10:50:49.731 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.731 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.731 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104455 s, 100 MB/s 2024-04-17T10:50:49.746 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.746 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.746 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122175 s, 85.8 MB/s 2024-04-17T10:50:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.763 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.763 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013596 s, 77.1 MB/s 2024-04-17T10:50:49.778 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.778 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.778 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126827 s, 82.7 MB/s 2024-04-17T10:50:49.794 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.794 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.795 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137148 s, 76.5 MB/s 2024-04-17T10:50:49.810 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.810 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.810 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133647 s, 78.5 MB/s 2024-04-17T10:50:49.824 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.824 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.825 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115067 s, 91.1 MB/s 2024-04-17T10:50:49.837 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.837 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.837 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00965762 s, 109 MB/s 2024-04-17T10:50:49.851 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.852 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.852 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119732 s, 87.6 MB/s 2024-04-17T10:50:49.866 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.866 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.866 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123016 s, 85.2 MB/s 2024-04-17T10:50:49.881 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.881 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.881 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119551 s, 87.7 MB/s 2024-04-17T10:50:49.898 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.898 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.898 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142161 s, 73.8 MB/s 2024-04-17T10:50:49.913 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.914 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.914 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01291 s, 81.2 MB/s 2024-04-17T10:50:49.928 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.929 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.929 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127108 s, 82.5 MB/s 2024-04-17T10:50:49.945 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.945 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.946 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130647 s, 80.3 MB/s 2024-04-17T10:50:49.961 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.961 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.961 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137782 s, 76.1 MB/s 2024-04-17T10:50:49.977 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:49.977 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:49.977 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139274 s, 75.3 MB/s 2024-04-17T10:50:49.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:49.979 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:50:49.979 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:50:49.979 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.6 2024-04-17T10:50:49.979 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:50:49.979 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:50:49.979 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:50:49.979 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:50:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123368 s, 85.0 MB/s 2024-04-17T10:50:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144534 s, 72.5 MB/s 2024-04-17T10:50:50.084 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.084 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.084 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150981 s, 69.5 MB/s 2024-04-17T10:50:50.100 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.101 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.101 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135215 s, 77.5 MB/s 2024-04-17T10:50:50.118 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.119 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.119 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153999 s, 68.1 MB/s 2024-04-17T10:50:50.137 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.137 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.137 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147802 s, 70.9 MB/s 2024-04-17T10:50:50.151 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.152 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.152 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113806 s, 92.1 MB/s 2024-04-17T10:50:50.164 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.165 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.165 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00995052 s, 105 MB/s 2024-04-17T10:50:50.180 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.180 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.180 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127923 s, 82.0 MB/s 2024-04-17T10:50:50.194 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.194 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.194 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110829 s, 94.6 MB/s 2024-04-17T10:50:50.207 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.207 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.207 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00993358 s, 106 MB/s 2024-04-17T10:50:50.223 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.223 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.223 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135168 s, 77.6 MB/s 2024-04-17T10:50:50.235 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.235 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.235 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00984457 s, 107 MB/s 2024-04-17T10:50:50.250 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.250 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.250 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123616 s, 84.8 MB/s 2024-04-17T10:50:50.264 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.264 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.264 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112402 s, 93.3 MB/s 2024-04-17T10:50:50.281 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.281 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.281 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140751 s, 74.5 MB/s 2024-04-17T10:50:50.298 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.298 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.298 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141459 s, 74.1 MB/s 2024-04-17T10:50:50.316 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.316 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.316 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152217 s, 68.9 MB/s 2024-04-17T10:50:50.332 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.332 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.332 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131398 s, 79.8 MB/s 2024-04-17T10:50:50.347 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.348 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.348 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120933 s, 86.7 MB/s 2024-04-17T10:50:50.365 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.365 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.365 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138877 s, 75.5 MB/s 2024-04-17T10:50:50.381 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.382 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.382 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014031 s, 74.7 MB/s 2024-04-17T10:50:50.398 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.398 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.398 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136236 s, 77.0 MB/s 2024-04-17T10:50:50.414 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.414 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.414 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126293 s, 83.0 MB/s 2024-04-17T10:50:50.429 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.429 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.429 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119107 s, 88.0 MB/s 2024-04-17T10:50:50.444 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128718 s, 81.5 MB/s 2024-04-17T10:50:50.460 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.460 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.460 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127234 s, 82.4 MB/s 2024-04-17T10:50:50.476 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.476 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.476 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130151 s, 80.6 MB/s 2024-04-17T10:50:50.494 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.494 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.495 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153799 s, 68.2 MB/s 2024-04-17T10:50:50.508 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.508 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.508 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110331 s, 95.0 MB/s 2024-04-17T10:50:50.526 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.526 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.526 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145348 s, 72.1 MB/s 2024-04-17T10:50:50.538 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.538 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.538 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00947965 s, 111 MB/s 2024-04-17T10:50:50.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:50.540 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:50:50.540 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:50:50.540 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.7 2024-04-17T10:50:50.540 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:50:50.540 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:50:50.540 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:50:50.540 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:50:50.605 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.605 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.605 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116422 s, 90.1 MB/s 2024-04-17T10:50:50.615 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.615 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.615 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798614 s, 131 MB/s 2024-04-17T10:50:50.630 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.630 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.630 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122305 s, 85.7 MB/s 2024-04-17T10:50:50.643 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.644 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.644 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109585 s, 95.7 MB/s 2024-04-17T10:50:50.656 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.656 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.656 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106314 s, 98.6 MB/s 2024-04-17T10:50:50.670 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.670 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.670 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111874 s, 93.7 MB/s 2024-04-17T10:50:50.683 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.683 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.684 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109931 s, 95.4 MB/s 2024-04-17T10:50:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117695 s, 89.1 MB/s 2024-04-17T10:50:50.710 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.710 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.710 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100654 s, 104 MB/s 2024-04-17T10:50:50.722 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.722 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.722 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845243 s, 124 MB/s 2024-04-17T10:50:50.737 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.737 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.737 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108529 s, 96.6 MB/s 2024-04-17T10:50:50.752 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.752 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.752 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113226 s, 92.6 MB/s 2024-04-17T10:50:50.766 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.766 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.766 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105627 s, 99.3 MB/s 2024-04-17T10:50:50.781 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.781 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.781 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115367 s, 90.9 MB/s 2024-04-17T10:50:50.796 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.796 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.796 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109268 s, 96.0 MB/s 2024-04-17T10:50:50.812 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.812 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.812 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121921 s, 86.0 MB/s 2024-04-17T10:50:50.827 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.827 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.827 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011872 s, 88.3 MB/s 2024-04-17T10:50:50.841 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.841 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.841 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105517 s, 99.4 MB/s 2024-04-17T10:50:50.854 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.854 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.854 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104454 s, 100 MB/s 2024-04-17T10:50:50.868 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.868 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.868 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115949 s, 90.4 MB/s 2024-04-17T10:50:50.883 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.883 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.883 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125092 s, 83.8 MB/s 2024-04-17T10:50:50.898 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.898 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.898 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125167 s, 83.8 MB/s 2024-04-17T10:50:50.908 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.908 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.908 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759965 s, 138 MB/s 2024-04-17T10:50:50.921 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.921 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.921 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105966 s, 99.0 MB/s 2024-04-17T10:50:50.934 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.934 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.934 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109179 s, 96.0 MB/s 2024-04-17T10:50:50.946 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.946 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.947 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103653 s, 101 MB/s 2024-04-17T10:50:50.961 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.961 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.962 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126118 s, 83.1 MB/s 2024-04-17T10:50:50.974 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.974 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.974 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107932 s, 97.2 MB/s 2024-04-17T10:50:50.988 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:50.988 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:50.988 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112056 s, 93.6 MB/s 2024-04-17T10:50:51.001 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.002 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.002 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109744 s, 95.5 MB/s 2024-04-17T10:50:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115939 s, 90.4 MB/s 2024-04-17T10:50:51.025 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.025 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.025 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753638 s, 139 MB/s 2024-04-17T10:50:51.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:51.026 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:50:51.027 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:50:51.027 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.8 2024-04-17T10:50:51.027 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:50:51.027 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:50:51.027 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:50:51.027 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:50:51.091 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.091 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.091 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110773 s, 94.7 MB/s 2024-04-17T10:50:51.105 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.105 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.105 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108099 s, 97.0 MB/s 2024-04-17T10:50:51.121 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.121 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.121 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126537 s, 82.9 MB/s 2024-04-17T10:50:51.135 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.135 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.135 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011259 s, 93.1 MB/s 2024-04-17T10:50:51.152 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.152 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.152 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134283 s, 78.1 MB/s 2024-04-17T10:50:51.162 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.162 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.162 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798261 s, 131 MB/s 2024-04-17T10:50:51.177 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.178 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.178 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127964 s, 81.9 MB/s 2024-04-17T10:50:51.190 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.190 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.190 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107897 s, 97.2 MB/s 2024-04-17T10:50:51.205 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.205 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.205 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012148 s, 86.3 MB/s 2024-04-17T10:50:51.214 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.214 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.214 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740418 s, 142 MB/s 2024-04-17T10:50:51.228 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.228 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.228 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113002 s, 92.8 MB/s 2024-04-17T10:50:51.242 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.242 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.242 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118333 s, 88.6 MB/s 2024-04-17T10:50:51.255 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.255 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.255 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102702 s, 102 MB/s 2024-04-17T10:50:51.269 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.269 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.269 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117337 s, 89.4 MB/s 2024-04-17T10:50:51.280 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.280 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.280 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00876842 s, 120 MB/s 2024-04-17T10:50:51.292 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.292 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.292 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00984811 s, 106 MB/s 2024-04-17T10:50:51.302 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.303 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.303 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846615 s, 124 MB/s 2024-04-17T10:50:51.314 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.314 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.314 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00959523 s, 109 MB/s 2024-04-17T10:50:51.325 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.325 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.325 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809182 s, 130 MB/s 2024-04-17T10:50:51.339 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.339 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.339 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118851 s, 88.2 MB/s 2024-04-17T10:50:51.353 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.353 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.354 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125355 s, 83.6 MB/s 2024-04-17T10:50:51.369 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.369 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.369 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128149 s, 81.8 MB/s 2024-04-17T10:50:51.384 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.385 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.385 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133219 s, 78.7 MB/s 2024-04-17T10:50:51.396 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.397 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.397 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00964455 s, 109 MB/s 2024-04-17T10:50:51.410 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.411 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.411 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120811 s, 86.8 MB/s 2024-04-17T10:50:51.424 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.424 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.424 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115325 s, 90.9 MB/s 2024-04-17T10:50:51.439 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.439 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.439 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120863 s, 86.8 MB/s 2024-04-17T10:50:51.448 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.448 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.448 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757201 s, 138 MB/s 2024-04-17T10:50:51.462 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.463 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.463 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117795 s, 89.0 MB/s 2024-04-17T10:50:51.475 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.475 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.475 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101623 s, 103 MB/s 2024-04-17T10:50:51.489 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.490 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.490 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120313 s, 87.2 MB/s 2024-04-17T10:50:51.504 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.504 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.504 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126247 s, 83.1 MB/s 2024-04-17T10:50:51.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:51.506 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:50:51.506 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:50:51.506 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.9 2024-04-17T10:50:51.506 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:50:51.506 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:50:51.506 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:50:51.506 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:50:51.571 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.571 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.571 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123379 s, 85.0 MB/s 2024-04-17T10:50:51.585 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.585 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.586 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117052 s, 89.6 MB/s 2024-04-17T10:50:51.599 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.599 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.599 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111893 s, 93.7 MB/s 2024-04-17T10:50:51.614 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.614 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.614 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125765 s, 83.4 MB/s 2024-04-17T10:50:51.629 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.629 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.629 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129254 s, 81.1 MB/s 2024-04-17T10:50:51.643 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.643 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.643 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117544 s, 89.2 MB/s 2024-04-17T10:50:51.657 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.657 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.657 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116243 s, 90.2 MB/s 2024-04-17T10:50:51.667 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.667 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.667 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767174 s, 137 MB/s 2024-04-17T10:50:51.682 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.683 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.683 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131751 s, 79.6 MB/s 2024-04-17T10:50:51.694 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.694 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.694 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00883781 s, 119 MB/s 2024-04-17T10:50:51.706 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.706 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.706 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0099772 s, 105 MB/s 2024-04-17T10:50:51.721 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.721 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.721 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121858 s, 86.0 MB/s 2024-04-17T10:50:51.733 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.733 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.733 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00967323 s, 108 MB/s 2024-04-17T10:50:51.746 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.746 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.746 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112846 s, 92.9 MB/s 2024-04-17T10:50:51.758 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.758 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.758 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00939846 s, 112 MB/s 2024-04-17T10:50:51.769 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.769 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.769 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855639 s, 123 MB/s 2024-04-17T10:50:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00980443 s, 107 MB/s 2024-04-17T10:50:51.794 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.794 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.794 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107285 s, 97.7 MB/s 2024-04-17T10:50:51.807 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.807 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.807 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104204 s, 101 MB/s 2024-04-17T10:50:51.818 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.818 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.818 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877753 s, 119 MB/s 2024-04-17T10:50:51.828 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.828 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.828 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783352 s, 134 MB/s 2024-04-17T10:50:51.840 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.841 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.841 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00991169 s, 106 MB/s 2024-04-17T10:50:51.854 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.855 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.855 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114842 s, 91.3 MB/s 2024-04-17T10:50:51.868 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.869 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.869 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113503 s, 92.4 MB/s 2024-04-17T10:50:51.881 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.881 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.881 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100859 s, 104 MB/s 2024-04-17T10:50:51.897 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.897 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.897 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129873 s, 80.7 MB/s 2024-04-17T10:50:51.911 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.911 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.911 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115019 s, 91.2 MB/s 2024-04-17T10:50:51.924 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.924 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.924 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104801 s, 100 MB/s 2024-04-17T10:50:51.936 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.936 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.937 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00980824 s, 107 MB/s 2024-04-17T10:50:51.950 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.950 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.950 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107971 s, 97.1 MB/s 2024-04-17T10:50:51.965 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.965 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.965 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126155 s, 83.1 MB/s 2024-04-17T10:50:51.980 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:51.980 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:51.980 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120512 s, 87.0 MB/s 2024-04-17T10:50:51.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:51.981 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:50:51.981 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:50:51.981 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.10 2024-04-17T10:50:51.981 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:50:51.981 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:50:51.982 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:50:51.982 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:50:52.044 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.045 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.045 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00953729 s, 110 MB/s 2024-04-17T10:50:52.059 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.059 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.059 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113936 s, 92.0 MB/s 2024-04-17T10:50:52.074 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.074 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.074 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124863 s, 84.0 MB/s 2024-04-17T10:50:52.090 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.090 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.090 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133494 s, 78.5 MB/s 2024-04-17T10:50:52.105 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.105 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.105 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121742 s, 86.1 MB/s 2024-04-17T10:50:52.120 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.120 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.121 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122047 s, 85.9 MB/s 2024-04-17T10:50:52.136 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.136 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.136 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119694 s, 87.6 MB/s 2024-04-17T10:50:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.150 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111311 s, 94.2 MB/s 2024-04-17T10:50:52.165 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.165 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.165 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131378 s, 79.8 MB/s 2024-04-17T10:50:52.176 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.176 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.176 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839775 s, 125 MB/s 2024-04-17T10:50:52.190 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.190 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.190 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114124 s, 91.9 MB/s 2024-04-17T10:50:52.204 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.204 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.204 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108841 s, 96.3 MB/s 2024-04-17T10:50:52.220 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.220 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.220 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139522 s, 75.2 MB/s 2024-04-17T10:50:52.235 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.236 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.236 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128581 s, 81.5 MB/s 2024-04-17T10:50:52.249 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.249 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.249 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109822 s, 95.5 MB/s 2024-04-17T10:50:52.265 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.265 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.265 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133783 s, 78.4 MB/s 2024-04-17T10:50:52.280 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.280 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.280 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011837 s, 88.6 MB/s 2024-04-17T10:50:52.295 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.295 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.295 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121331 s, 86.4 MB/s 2024-04-17T10:50:52.310 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.310 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.310 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124204 s, 84.4 MB/s 2024-04-17T10:50:52.324 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.324 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.324 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117968 s, 88.9 MB/s 2024-04-17T10:50:52.341 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.341 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.341 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014223 s, 73.7 MB/s 2024-04-17T10:50:52.357 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.357 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.357 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125348 s, 83.7 MB/s 2024-04-17T10:50:52.372 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.373 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.373 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131507 s, 79.7 MB/s 2024-04-17T10:50:52.387 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.387 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.387 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114917 s, 91.2 MB/s 2024-04-17T10:50:52.402 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.402 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.402 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121292 s, 86.5 MB/s 2024-04-17T10:50:52.416 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.416 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.416 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118514 s, 88.5 MB/s 2024-04-17T10:50:52.432 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.432 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.432 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130718 s, 80.2 MB/s 2024-04-17T10:50:52.448 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.448 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.448 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124405 s, 84.3 MB/s 2024-04-17T10:50:52.460 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.460 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.460 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00902541 s, 116 MB/s 2024-04-17T10:50:52.472 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.473 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.473 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101406 s, 103 MB/s 2024-04-17T10:50:52.488 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.488 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.488 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121835 s, 86.1 MB/s 2024-04-17T10:50:52.505 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.505 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.505 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148833 s, 70.5 MB/s 2024-04-17T10:50:52.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:52.506 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:50:52.506 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:50:52.506 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.11 2024-04-17T10:50:52.507 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:50:52.507 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:50:52.507 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:50:52.507 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:50:52.578 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.578 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.578 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125514 s, 83.5 MB/s 2024-04-17T10:50:52.594 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.594 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.594 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131074 s, 80.0 MB/s 2024-04-17T10:50:52.610 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.610 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.610 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012732 s, 82.4 MB/s 2024-04-17T10:50:52.626 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.626 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.626 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139942 s, 74.9 MB/s 2024-04-17T10:50:52.641 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.641 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.641 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123382 s, 85.0 MB/s 2024-04-17T10:50:52.656 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.656 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.656 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122784 s, 85.4 MB/s 2024-04-17T10:50:52.671 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.671 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.672 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120918 s, 86.7 MB/s 2024-04-17T10:50:52.687 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.687 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.687 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012722 s, 82.4 MB/s 2024-04-17T10:50:52.702 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.702 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.702 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125988 s, 83.2 MB/s 2024-04-17T10:50:52.717 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.717 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.717 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117871 s, 89.0 MB/s 2024-04-17T10:50:52.730 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.730 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.730 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00976853 s, 107 MB/s 2024-04-17T10:50:52.742 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.742 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.742 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00967962 s, 108 MB/s 2024-04-17T10:50:52.756 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.756 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.756 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011283 s, 92.9 MB/s 2024-04-17T10:50:52.772 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.772 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.772 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127957 s, 81.9 MB/s 2024-04-17T10:50:52.786 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.786 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.786 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111671 s, 93.9 MB/s 2024-04-17T10:50:52.803 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.803 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.803 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129234 s, 81.1 MB/s 2024-04-17T10:50:52.815 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.815 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.816 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00987585 s, 106 MB/s 2024-04-17T10:50:52.831 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.832 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.832 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132115 s, 79.4 MB/s 2024-04-17T10:50:52.846 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.846 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.846 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012033 s, 87.1 MB/s 2024-04-17T10:50:52.858 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.858 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.859 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00943194 s, 111 MB/s 2024-04-17T10:50:52.873 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.873 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.873 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113521 s, 92.4 MB/s 2024-04-17T10:50:52.887 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.887 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.887 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118784 s, 88.3 MB/s 2024-04-17T10:50:52.901 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.901 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.901 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110252 s, 95.1 MB/s 2024-04-17T10:50:52.917 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.917 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.917 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126707 s, 82.8 MB/s 2024-04-17T10:50:52.932 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.932 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.932 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127811 s, 82.0 MB/s 2024-04-17T10:50:52.946 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.946 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.947 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011373 s, 92.2 MB/s 2024-04-17T10:50:52.961 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.961 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.961 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117954 s, 88.9 MB/s 2024-04-17T10:50:52.976 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.976 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.976 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126209 s, 83.1 MB/s 2024-04-17T10:50:52.991 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:52.991 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:52.991 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011759 s, 89.2 MB/s 2024-04-17T10:50:53.007 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.007 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.007 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128673 s, 81.5 MB/s 2024-04-17T10:50:53.026 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.026 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.026 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162432 s, 64.6 MB/s 2024-04-17T10:50:53.054 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.054 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.054 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130005 s, 80.7 MB/s 2024-04-17T10:50:53.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:53.055 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:50:53.055 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:50:53.056 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.12 2024-04-17T10:50:53.056 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:50:53.056 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:50:53.056 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:50:53.056 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:50:53.123 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.123 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.123 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113777 s, 92.2 MB/s 2024-04-17T10:50:53.139 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.140 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.140 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140874 s, 74.4 MB/s 2024-04-17T10:50:53.155 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.155 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.155 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125829 s, 83.3 MB/s 2024-04-17T10:50:53.167 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.167 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.167 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00949222 s, 110 MB/s 2024-04-17T10:50:53.183 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.183 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.183 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130864 s, 80.1 MB/s 2024-04-17T10:50:53.199 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.199 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.199 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130547 s, 80.3 MB/s 2024-04-17T10:50:53.213 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.214 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.214 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116398 s, 90.1 MB/s 2024-04-17T10:50:53.231 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.231 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.231 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143595 s, 73.0 MB/s 2024-04-17T10:50:53.243 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.243 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.243 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00929792 s, 113 MB/s 2024-04-17T10:50:53.258 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.258 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.258 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122979 s, 85.3 MB/s 2024-04-17T10:50:53.272 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.272 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.272 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120479 s, 87.0 MB/s 2024-04-17T10:50:53.285 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.286 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.286 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104238 s, 101 MB/s 2024-04-17T10:50:53.300 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.300 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.300 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011809 s, 88.8 MB/s 2024-04-17T10:50:53.314 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.314 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.314 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110064 s, 95.3 MB/s 2024-04-17T10:50:53.329 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.329 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.329 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118624 s, 88.4 MB/s 2024-04-17T10:50:53.343 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.343 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.343 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011442 s, 91.6 MB/s 2024-04-17T10:50:53.356 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.356 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.356 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108771 s, 96.4 MB/s 2024-04-17T10:50:53.368 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.369 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.369 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00952006 s, 110 MB/s 2024-04-17T10:50:53.382 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.383 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.383 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112031 s, 93.6 MB/s 2024-04-17T10:50:53.396 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.396 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.396 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111952 s, 93.7 MB/s 2024-04-17T10:50:53.411 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.411 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.411 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011849 s, 88.5 MB/s 2024-04-17T10:50:53.427 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.427 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.427 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134119 s, 78.2 MB/s 2024-04-17T10:50:53.440 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.440 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.440 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107365 s, 97.7 MB/s 2024-04-17T10:50:53.456 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.456 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.456 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128792 s, 81.4 MB/s 2024-04-17T10:50:53.471 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.471 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.471 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012822 s, 81.8 MB/s 2024-04-17T10:50:53.488 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.488 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.488 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137309 s, 76.4 MB/s 2024-04-17T10:50:53.501 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.501 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.502 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111363 s, 94.2 MB/s 2024-04-17T10:50:53.516 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.517 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.517 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122843 s, 85.4 MB/s 2024-04-17T10:50:53.534 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.534 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.534 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147514 s, 71.1 MB/s 2024-04-17T10:50:53.549 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.549 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.549 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119999 s, 87.4 MB/s 2024-04-17T10:50:53.564 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.564 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.564 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123421 s, 85.0 MB/s 2024-04-17T10:50:53.581 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.581 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.581 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134639 s, 77.9 MB/s 2024-04-17T10:50:53.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:53.582 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:50:53.582 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:50:53.582 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.13 2024-04-17T10:50:53.582 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:50:53.582 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:50:53.583 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:50:53.583 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:50:53.645 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.645 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.646 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107909 s, 97.2 MB/s 2024-04-17T10:50:53.660 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.660 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.660 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121404 s, 86.4 MB/s 2024-04-17T10:50:53.675 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.675 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.675 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127005 s, 82.6 MB/s 2024-04-17T10:50:53.690 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.690 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.690 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119147 s, 88.0 MB/s 2024-04-17T10:50:53.701 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.701 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.702 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00927976 s, 113 MB/s 2024-04-17T10:50:53.712 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.712 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.712 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835661 s, 125 MB/s 2024-04-17T10:50:53.726 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.726 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.726 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111195 s, 94.3 MB/s 2024-04-17T10:50:53.739 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.739 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.739 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106603 s, 98.4 MB/s 2024-04-17T10:50:53.753 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.753 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.753 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114011 s, 92.0 MB/s 2024-04-17T10:50:53.767 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.768 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.768 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119494 s, 87.8 MB/s 2024-04-17T10:50:53.782 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.782 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.782 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119838 s, 87.5 MB/s 2024-04-17T10:50:53.797 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.797 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.798 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132031 s, 79.4 MB/s 2024-04-17T10:50:53.812 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.812 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.813 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125942 s, 83.3 MB/s 2024-04-17T10:50:53.828 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.828 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.828 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127905 s, 82.0 MB/s 2024-04-17T10:50:53.842 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.842 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.842 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121673 s, 86.2 MB/s 2024-04-17T10:50:53.855 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.855 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.855 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108216 s, 96.9 MB/s 2024-04-17T10:50:53.872 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.872 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.872 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139647 s, 75.1 MB/s 2024-04-17T10:50:53.885 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.885 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.885 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107632 s, 97.4 MB/s 2024-04-17T10:50:53.900 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.900 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.900 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125967 s, 83.2 MB/s 2024-04-17T10:50:53.911 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.911 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.911 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085538 s, 123 MB/s 2024-04-17T10:50:53.924 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.925 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.925 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109024 s, 96.2 MB/s 2024-04-17T10:50:53.935 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.935 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.935 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083538 s, 126 MB/s 2024-04-17T10:50:53.949 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.949 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.949 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116108 s, 90.3 MB/s 2024-04-17T10:50:53.963 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.963 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.963 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110906 s, 94.5 MB/s 2024-04-17T10:50:53.977 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.977 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.977 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117124 s, 89.5 MB/s 2024-04-17T10:50:53.991 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:53.991 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:53.991 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116771 s, 89.8 MB/s 2024-04-17T10:50:54.010 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.010 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.011 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168449 s, 62.2 MB/s 2024-04-17T10:50:54.025 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.026 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.026 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122428 s, 85.6 MB/s 2024-04-17T10:50:54.063 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.063 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.063 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123954 s, 84.6 MB/s 2024-04-17T10:50:54.079 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.079 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.079 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121546 s, 86.3 MB/s 2024-04-17T10:50:54.094 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.094 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.094 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121879 s, 86.0 MB/s 2024-04-17T10:50:54.110 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.110 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.110 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122251 s, 85.8 MB/s 2024-04-17T10:50:54.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:54.111 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:50:54.111 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:50:54.111 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.14 2024-04-17T10:50:54.112 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:50:54.112 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:50:54.112 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:50:54.112 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:50:54.181 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.182 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.182 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123414 s, 85.0 MB/s 2024-04-17T10:50:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128614 s, 81.5 MB/s 2024-04-17T10:50:54.215 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.215 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.215 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137493 s, 76.3 MB/s 2024-04-17T10:50:54.231 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.231 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.231 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131163 s, 79.9 MB/s 2024-04-17T10:50:54.245 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.245 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.245 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106808 s, 98.2 MB/s 2024-04-17T10:50:54.260 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.260 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.261 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120165 s, 87.3 MB/s 2024-04-17T10:50:54.276 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.276 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.276 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128454 s, 81.6 MB/s 2024-04-17T10:50:54.291 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.291 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.291 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119773 s, 87.5 MB/s 2024-04-17T10:50:54.306 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.306 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.306 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123288 s, 85.1 MB/s 2024-04-17T10:50:54.320 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.320 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.320 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101202 s, 104 MB/s 2024-04-17T10:50:54.333 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.333 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.333 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106642 s, 98.3 MB/s 2024-04-17T10:50:54.347 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.347 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.347 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107156 s, 97.9 MB/s 2024-04-17T10:50:54.360 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.360 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.360 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00979164 s, 107 MB/s 2024-04-17T10:50:54.374 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.374 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.374 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117617 s, 89.2 MB/s 2024-04-17T10:50:54.389 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.389 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.389 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115556 s, 90.7 MB/s 2024-04-17T10:50:54.404 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.404 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.404 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124238 s, 84.4 MB/s 2024-04-17T10:50:54.419 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.419 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.419 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116494 s, 90.0 MB/s 2024-04-17T10:50:54.431 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.431 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.431 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00912679 s, 115 MB/s 2024-04-17T10:50:54.446 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.446 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.446 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011982 s, 87.5 MB/s 2024-04-17T10:50:54.462 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.462 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.462 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130093 s, 80.6 MB/s 2024-04-17T10:50:54.478 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.478 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.478 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130289 s, 80.5 MB/s 2024-04-17T10:50:54.492 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.492 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.492 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110084 s, 95.3 MB/s 2024-04-17T10:50:54.506 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.506 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.506 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110937 s, 94.5 MB/s 2024-04-17T10:50:54.521 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.521 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.521 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113117 s, 92.7 MB/s 2024-04-17T10:50:54.531 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.531 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.531 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775636 s, 135 MB/s 2024-04-17T10:50:54.546 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.546 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.546 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123367 s, 85.0 MB/s 2024-04-17T10:50:54.562 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.562 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.562 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115634 s, 90.7 MB/s 2024-04-17T10:50:54.575 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.575 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.575 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0088501 s, 118 MB/s 2024-04-17T10:50:54.589 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.589 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.590 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105216 s, 99.7 MB/s 2024-04-17T10:50:54.605 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.605 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.605 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011777 s, 89.0 MB/s 2024-04-17T10:50:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00892086 s, 118 MB/s 2024-04-17T10:50:54.633 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.633 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.633 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119462 s, 87.8 MB/s 2024-04-17T10:50:54.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:54.635 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:50:54.635 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:50:54.635 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.15 2024-04-17T10:50:54.635 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:50:54.635 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:50:54.635 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:50:54.635 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:50:54.705 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.705 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.705 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128951 s, 81.3 MB/s 2024-04-17T10:50:54.721 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.721 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.721 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133091 s, 78.8 MB/s 2024-04-17T10:50:54.736 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.736 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.736 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118037 s, 88.8 MB/s 2024-04-17T10:50:54.746 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.746 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.746 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799575 s, 131 MB/s 2024-04-17T10:50:54.758 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.758 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.758 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00990871 s, 106 MB/s 2024-04-17T10:50:54.771 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.772 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.772 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106398 s, 98.6 MB/s 2024-04-17T10:50:54.786 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.786 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.787 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122043 s, 85.9 MB/s 2024-04-17T10:50:54.800 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.800 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.800 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111955 s, 93.7 MB/s 2024-04-17T10:50:54.814 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.814 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.815 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123084 s, 85.2 MB/s 2024-04-17T10:50:54.830 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.830 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.830 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127462 s, 82.3 MB/s 2024-04-17T10:50:54.844 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.844 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.844 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122038 s, 85.9 MB/s 2024-04-17T10:50:54.858 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.858 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.858 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110947 s, 94.5 MB/s 2024-04-17T10:50:54.869 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.869 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.869 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00858446 s, 122 MB/s 2024-04-17T10:50:54.882 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.882 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.883 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113549 s, 92.3 MB/s 2024-04-17T10:50:54.896 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.896 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.896 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115892 s, 90.5 MB/s 2024-04-17T10:50:54.907 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.907 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.907 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851365 s, 123 MB/s 2024-04-17T10:50:54.919 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.919 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.919 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100982 s, 104 MB/s 2024-04-17T10:50:54.933 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.933 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.933 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114674 s, 91.4 MB/s 2024-04-17T10:50:54.945 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.946 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.946 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00966117 s, 109 MB/s 2024-04-17T10:50:54.956 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.956 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.956 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815496 s, 129 MB/s 2024-04-17T10:50:54.969 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.969 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.969 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111485 s, 94.1 MB/s 2024-04-17T10:50:54.981 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.982 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.982 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103284 s, 102 MB/s 2024-04-17T10:50:54.996 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:54.996 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:54.996 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123837 s, 84.7 MB/s 2024-04-17T10:50:55.008 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.008 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.008 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00989989 s, 106 MB/s 2024-04-17T10:50:55.023 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.023 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.023 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126645 s, 82.8 MB/s 2024-04-17T10:50:55.051 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.052 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.052 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011709 s, 89.6 MB/s 2024-04-17T10:50:55.063 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.063 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.063 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00881019 s, 119 MB/s 2024-04-17T10:50:55.072 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.073 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.073 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774942 s, 135 MB/s 2024-04-17T10:50:55.087 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.087 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.088 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126826 s, 82.7 MB/s 2024-04-17T10:50:55.102 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.102 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.102 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124939 s, 83.9 MB/s 2024-04-17T10:50:55.116 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.116 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.116 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112768 s, 93.0 MB/s 2024-04-17T10:50:55.131 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.131 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.131 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115463 s, 90.8 MB/s 2024-04-17T10:50:55.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:55.131 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:50:55.131 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:50:55.131 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.16 2024-04-17T10:50:55.131 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:50:55.131 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:50:55.132 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:50:55.132 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:50:55.200 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.201 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.201 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126214 s, 83.1 MB/s 2024-04-17T10:50:55.217 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.217 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.217 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137376 s, 76.3 MB/s 2024-04-17T10:50:55.232 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.232 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.232 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130904 s, 80.1 MB/s 2024-04-17T10:50:55.247 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.247 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.247 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126267 s, 83.0 MB/s 2024-04-17T10:50:55.262 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.262 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.262 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123048 s, 85.2 MB/s 2024-04-17T10:50:55.277 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.277 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.277 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124396 s, 84.3 MB/s 2024-04-17T10:50:55.292 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.292 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.292 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012265 s, 85.5 MB/s 2024-04-17T10:50:55.305 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.305 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.305 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106558 s, 98.4 MB/s 2024-04-17T10:50:55.318 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.319 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.319 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103838 s, 101 MB/s 2024-04-17T10:50:55.333 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.333 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.333 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115863 s, 90.5 MB/s 2024-04-17T10:50:55.350 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.350 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.350 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140625 s, 74.6 MB/s 2024-04-17T10:50:55.366 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.366 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.366 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127409 s, 82.3 MB/s 2024-04-17T10:50:55.382 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.382 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.382 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131253 s, 79.9 MB/s 2024-04-17T10:50:55.396 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.396 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.396 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107476 s, 97.6 MB/s 2024-04-17T10:50:55.413 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.413 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.413 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138679 s, 75.6 MB/s 2024-04-17T10:50:55.427 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.427 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.428 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011374 s, 92.2 MB/s 2024-04-17T10:50:55.443 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.443 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.443 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126065 s, 83.2 MB/s 2024-04-17T10:50:55.458 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.458 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.459 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125693 s, 83.4 MB/s 2024-04-17T10:50:55.479 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.479 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.479 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179072 s, 58.6 MB/s 2024-04-17T10:50:55.497 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.497 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.497 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146734 s, 71.5 MB/s 2024-04-17T10:50:55.510 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.511 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.511 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112071 s, 93.6 MB/s 2024-04-17T10:50:55.528 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.528 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.529 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014408 s, 72.8 MB/s 2024-04-17T10:50:55.544 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.544 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.544 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123612 s, 84.8 MB/s 2024-04-17T10:50:55.560 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.560 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.560 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130875 s, 80.1 MB/s 2024-04-17T10:50:55.572 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.572 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.572 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00946798 s, 111 MB/s 2024-04-17T10:50:55.587 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.587 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.587 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125825 s, 83.3 MB/s 2024-04-17T10:50:55.602 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.602 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.602 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119796 s, 87.5 MB/s 2024-04-17T10:50:55.613 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.613 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.613 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086415 s, 121 MB/s 2024-04-17T10:50:55.626 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.626 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.626 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010358 s, 101 MB/s 2024-04-17T10:50:55.643 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.643 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.643 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116577 s, 89.9 MB/s 2024-04-17T10:50:55.656 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.656 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.656 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101283 s, 104 MB/s 2024-04-17T10:50:55.669 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.669 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.669 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115866 s, 90.5 MB/s 2024-04-17T10:50:55.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:55.671 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:50:55.671 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:50:55.671 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.17 2024-04-17T10:50:55.671 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:50:55.671 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:50:55.671 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:50:55.671 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:50:55.735 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.735 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.735 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104802 s, 100 MB/s 2024-04-17T10:50:55.748 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.748 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.748 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102586 s, 102 MB/s 2024-04-17T10:50:55.763 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.763 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.763 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126208 s, 83.1 MB/s 2024-04-17T10:50:55.773 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.773 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.773 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00863414 s, 121 MB/s 2024-04-17T10:50:55.786 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.786 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.786 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010532 s, 99.6 MB/s 2024-04-17T10:50:55.799 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.799 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.799 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105692 s, 99.2 MB/s 2024-04-17T10:50:55.811 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.812 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.812 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101257 s, 104 MB/s 2024-04-17T10:50:55.824 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.824 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.825 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104072 s, 101 MB/s 2024-04-17T10:50:55.837 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.837 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.837 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104387 s, 100 MB/s 2024-04-17T10:50:55.851 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.851 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.851 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118881 s, 88.2 MB/s 2024-04-17T10:50:55.865 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.866 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.866 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118936 s, 88.2 MB/s 2024-04-17T10:50:55.879 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.879 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.879 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111615 s, 93.9 MB/s 2024-04-17T10:50:55.892 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.892 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.893 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110715 s, 94.7 MB/s 2024-04-17T10:50:55.908 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.908 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.908 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130702 s, 80.2 MB/s 2024-04-17T10:50:55.924 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.924 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.924 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134384 s, 78.0 MB/s 2024-04-17T10:50:55.939 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.939 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.939 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126891 s, 82.6 MB/s 2024-04-17T10:50:55.952 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.952 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.952 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109979 s, 95.3 MB/s 2024-04-17T10:50:55.963 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.963 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.963 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847588 s, 124 MB/s 2024-04-17T10:50:55.977 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.977 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.977 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114648 s, 91.5 MB/s 2024-04-17T10:50:55.990 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:55.990 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:55.990 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113203 s, 92.6 MB/s 2024-04-17T10:50:56.003 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.003 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.004 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107427 s, 97.6 MB/s 2024-04-17T10:50:56.015 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.015 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.015 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00949374 s, 110 MB/s 2024-04-17T10:50:56.040 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.040 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.040 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216685 s, 48.4 MB/s 2024-04-17T10:50:56.065 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.065 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.066 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101657 s, 103 MB/s 2024-04-17T10:50:56.081 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.081 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.082 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131243 s, 79.9 MB/s 2024-04-17T10:50:56.096 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.096 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.096 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117264 s, 89.4 MB/s 2024-04-17T10:50:56.110 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.110 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.110 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113753 s, 92.2 MB/s 2024-04-17T10:50:56.122 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.122 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.122 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00925538 s, 113 MB/s 2024-04-17T10:50:56.137 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.138 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.138 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124278 s, 84.4 MB/s 2024-04-17T10:50:56.153 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.153 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.153 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128568 s, 81.6 MB/s 2024-04-17T10:50:56.167 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.167 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.167 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108284 s, 96.8 MB/s 2024-04-17T10:50:56.181 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.182 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.182 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122346 s, 85.7 MB/s 2024-04-17T10:50:56.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:56.183 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:50:56.183 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:50:56.183 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.18 2024-04-17T10:50:56.183 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:50:56.183 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:50:56.183 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:50:56.183 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:50:56.252 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.252 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.252 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116674 s, 89.9 MB/s 2024-04-17T10:50:56.267 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.267 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.267 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012155 s, 86.3 MB/s 2024-04-17T10:50:56.284 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.284 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.284 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126595 s, 82.8 MB/s 2024-04-17T10:50:56.300 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.300 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.300 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113261 s, 92.6 MB/s 2024-04-17T10:50:56.314 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.314 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.314 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103893 s, 101 MB/s 2024-04-17T10:50:56.331 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.331 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.331 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134438 s, 78.0 MB/s 2024-04-17T10:50:56.347 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.347 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.347 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131965 s, 79.5 MB/s 2024-04-17T10:50:56.364 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.364 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.365 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129387 s, 81.0 MB/s 2024-04-17T10:50:56.380 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.381 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.381 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124229 s, 84.4 MB/s 2024-04-17T10:50:56.396 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.396 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.396 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119706 s, 87.6 MB/s 2024-04-17T10:50:56.410 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.410 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.410 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106245 s, 98.7 MB/s 2024-04-17T10:50:56.426 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.426 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.426 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01188 s, 88.3 MB/s 2024-04-17T10:50:56.441 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.441 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.441 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121518 s, 86.3 MB/s 2024-04-17T10:50:56.458 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.458 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.458 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131807 s, 79.6 MB/s 2024-04-17T10:50:56.480 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.480 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.480 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189657 s, 55.3 MB/s 2024-04-17T10:50:56.496 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.496 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.496 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128921 s, 81.3 MB/s 2024-04-17T10:50:56.516 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.516 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.516 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161251 s, 65.0 MB/s 2024-04-17T10:50:56.533 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.533 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.533 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135353 s, 77.5 MB/s 2024-04-17T10:50:56.549 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.550 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.550 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129755 s, 80.8 MB/s 2024-04-17T10:50:56.566 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.566 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.566 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142208 s, 73.7 MB/s 2024-04-17T10:50:56.583 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.583 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.583 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147135 s, 71.3 MB/s 2024-04-17T10:50:56.599 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.599 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.599 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128455 s, 81.6 MB/s 2024-04-17T10:50:56.614 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.614 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.614 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123151 s, 85.1 MB/s 2024-04-17T10:50:56.626 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.627 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.627 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00964633 s, 109 MB/s 2024-04-17T10:50:56.642 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.642 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.642 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123023 s, 85.2 MB/s 2024-04-17T10:50:56.654 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.654 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.654 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00997058 s, 105 MB/s 2024-04-17T10:50:56.668 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.668 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.668 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115153 s, 91.1 MB/s 2024-04-17T10:50:56.683 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.683 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.683 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121342 s, 86.4 MB/s 2024-04-17T10:50:56.699 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.699 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.699 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129405 s, 81.0 MB/s 2024-04-17T10:50:56.715 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.715 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.715 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135204 s, 77.6 MB/s 2024-04-17T10:50:56.732 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.732 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.732 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137626 s, 76.2 MB/s 2024-04-17T10:50:56.744 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.744 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.744 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00988999 s, 106 MB/s 2024-04-17T10:50:56.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:56.745 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:50:56.745 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:50:56.746 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.19 2024-04-17T10:50:56.746 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:50:56.746 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:50:56.746 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:50:56.746 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:50:56.818 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.818 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.818 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146293 s, 71.7 MB/s 2024-04-17T10:50:56.834 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.835 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.835 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132409 s, 79.2 MB/s 2024-04-17T10:50:56.848 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.848 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.848 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010221 s, 103 MB/s 2024-04-17T10:50:56.861 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.861 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.861 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102128 s, 103 MB/s 2024-04-17T10:50:56.876 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.877 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.877 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012272 s, 85.4 MB/s 2024-04-17T10:50:56.891 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.891 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.892 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012162 s, 86.2 MB/s 2024-04-17T10:50:56.907 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.907 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.908 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130622 s, 80.3 MB/s 2024-04-17T10:50:56.922 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.922 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.922 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110096 s, 95.2 MB/s 2024-04-17T10:50:56.935 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.935 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.936 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105444 s, 99.4 MB/s 2024-04-17T10:50:56.952 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.953 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.953 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136623 s, 76.7 MB/s 2024-04-17T10:50:56.969 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.969 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.969 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132777 s, 79.0 MB/s 2024-04-17T10:50:56.983 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.984 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.984 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109461 s, 95.8 MB/s 2024-04-17T10:50:56.997 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:56.997 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:56.998 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107033 s, 98.0 MB/s 2024-04-17T10:50:57.012 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.012 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.012 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115441 s, 90.8 MB/s 2024-04-17T10:50:57.028 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.028 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.028 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130411 s, 80.4 MB/s 2024-04-17T10:50:57.046 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.046 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.046 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148908 s, 70.4 MB/s 2024-04-17T10:50:57.065 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.065 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.065 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00857382 s, 122 MB/s 2024-04-17T10:50:57.079 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.079 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.079 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116857 s, 89.7 MB/s 2024-04-17T10:50:57.093 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.094 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.094 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116879 s, 89.7 MB/s 2024-04-17T10:50:57.109 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.109 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.109 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125153 s, 83.8 MB/s 2024-04-17T10:50:57.123 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.123 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.123 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117994 s, 88.9 MB/s 2024-04-17T10:50:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134246 s, 78.1 MB/s 2024-04-17T10:50:57.154 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.154 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.154 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111191 s, 94.3 MB/s 2024-04-17T10:50:57.170 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.170 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.170 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132145 s, 79.4 MB/s 2024-04-17T10:50:57.184 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.184 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.184 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111255 s, 94.2 MB/s 2024-04-17T10:50:57.199 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.199 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.199 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011855 s, 88.4 MB/s 2024-04-17T10:50:57.213 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.213 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.213 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113613 s, 92.3 MB/s 2024-04-17T10:50:57.228 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.228 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.228 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126087 s, 83.2 MB/s 2024-04-17T10:50:57.244 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.244 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.244 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127332 s, 82.3 MB/s 2024-04-17T10:50:57.260 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.260 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.260 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012896 s, 81.3 MB/s 2024-04-17T10:50:57.274 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.274 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.274 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011717 s, 89.5 MB/s 2024-04-17T10:50:57.289 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.289 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.290 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119236 s, 87.9 MB/s 2024-04-17T10:50:57.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:57.291 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:50:57.291 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:50:57.291 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.20 2024-04-17T10:50:57.291 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:50:57.291 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:50:57.291 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:50:57.291 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:50:57.363 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.363 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.363 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143071 s, 73.3 MB/s 2024-04-17T10:50:57.376 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.376 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.376 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105836 s, 99.1 MB/s 2024-04-17T10:50:57.391 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.391 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.391 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124188 s, 84.4 MB/s 2024-04-17T10:50:57.407 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.407 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.407 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124685 s, 84.1 MB/s 2024-04-17T10:50:57.422 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.422 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.422 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124777 s, 84.0 MB/s 2024-04-17T10:50:57.437 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.437 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.438 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123775 s, 84.7 MB/s 2024-04-17T10:50:57.454 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.454 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.454 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134906 s, 77.7 MB/s 2024-04-17T10:50:57.469 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.469 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.469 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122519 s, 85.6 MB/s 2024-04-17T10:50:57.484 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.484 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.484 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128943 s, 81.3 MB/s 2024-04-17T10:50:57.500 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.500 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.501 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135308 s, 77.5 MB/s 2024-04-17T10:50:57.518 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.518 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.518 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141185 s, 74.3 MB/s 2024-04-17T10:50:57.532 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.532 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.533 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112001 s, 93.6 MB/s 2024-04-17T10:50:57.548 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.548 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.548 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013021 s, 80.5 MB/s 2024-04-17T10:50:57.564 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.564 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.564 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131109 s, 80.0 MB/s 2024-04-17T10:50:57.578 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.578 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.579 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117802 s, 89.0 MB/s 2024-04-17T10:50:57.593 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.593 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.593 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118274 s, 88.7 MB/s 2024-04-17T10:50:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140988 s, 74.4 MB/s 2024-04-17T10:50:57.623 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.623 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.623 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102537 s, 102 MB/s 2024-04-17T10:50:57.639 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.639 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.640 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127316 s, 82.4 MB/s 2024-04-17T10:50:57.654 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.654 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.654 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114086 s, 91.9 MB/s 2024-04-17T10:50:57.667 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.667 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.668 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105331 s, 99.6 MB/s 2024-04-17T10:50:57.683 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.683 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.683 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125298 s, 83.7 MB/s 2024-04-17T10:50:57.695 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.696 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.696 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00969286 s, 108 MB/s 2024-04-17T10:50:57.710 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.711 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.711 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126166 s, 83.1 MB/s 2024-04-17T10:50:57.725 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.725 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.725 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116261 s, 90.2 MB/s 2024-04-17T10:50:57.739 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.739 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.739 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116201 s, 90.2 MB/s 2024-04-17T10:50:57.752 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.752 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.753 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107943 s, 97.1 MB/s 2024-04-17T10:50:57.768 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.768 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.768 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130443 s, 80.4 MB/s 2024-04-17T10:50:57.783 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.783 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.783 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121503 s, 86.3 MB/s 2024-04-17T10:50:57.798 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.798 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.798 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127125 s, 82.5 MB/s 2024-04-17T10:50:57.812 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.812 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.812 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121782 s, 86.1 MB/s 2024-04-17T10:50:57.828 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.828 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.828 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126816 s, 82.7 MB/s 2024-04-17T10:50:57.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:57.829 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:50:57.829 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:50:57.830 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.21 2024-04-17T10:50:57.830 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:50:57.830 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:50:57.830 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:50:57.830 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:50:57.895 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.895 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.895 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128986 s, 81.3 MB/s 2024-04-17T10:50:57.909 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.909 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.909 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115912 s, 90.5 MB/s 2024-04-17T10:50:57.923 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.923 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.923 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110498 s, 94.9 MB/s 2024-04-17T10:50:57.937 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.937 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.937 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011043 s, 95.0 MB/s 2024-04-17T10:50:57.951 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.951 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.951 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119633 s, 87.6 MB/s 2024-04-17T10:50:57.965 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.965 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.965 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113953 s, 92.0 MB/s 2024-04-17T10:50:57.976 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.976 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.976 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842186 s, 125 MB/s 2024-04-17T10:50:57.987 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:57.987 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:57.988 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860458 s, 122 MB/s 2024-04-17T10:50:58.001 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.001 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.002 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115143 s, 91.1 MB/s 2024-04-17T10:50:58.015 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.015 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.016 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113993 s, 92.0 MB/s 2024-04-17T10:50:58.029 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.029 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.029 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010929 s, 95.9 MB/s 2024-04-17T10:50:58.044 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.045 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.045 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131435 s, 79.8 MB/s 2024-04-17T10:50:58.082 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.082 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.082 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107854 s, 97.2 MB/s 2024-04-17T10:50:58.091 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.091 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.091 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073205 s, 143 MB/s 2024-04-17T10:50:58.104 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.104 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.104 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110133 s, 95.2 MB/s 2024-04-17T10:50:58.116 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.116 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.117 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103636 s, 101 MB/s 2024-04-17T10:50:58.129 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.129 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.129 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102727 s, 102 MB/s 2024-04-17T10:50:58.138 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.138 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.138 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739088 s, 142 MB/s 2024-04-17T10:50:58.152 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.152 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.152 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115856 s, 90.5 MB/s 2024-04-17T10:50:58.162 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.163 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.163 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810282 s, 129 MB/s 2024-04-17T10:50:58.177 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.177 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.177 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113729 s, 92.2 MB/s 2024-04-17T10:50:58.189 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.189 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.189 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00945204 s, 111 MB/s 2024-04-17T10:50:58.199 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.199 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.200 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741819 s, 141 MB/s 2024-04-17T10:50:58.213 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.213 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.213 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104736 s, 100 MB/s 2024-04-17T10:50:58.227 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.227 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.227 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115009 s, 91.2 MB/s 2024-04-17T10:50:58.239 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.239 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.240 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101497 s, 103 MB/s 2024-04-17T10:50:58.251 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.251 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.251 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00978236 s, 107 MB/s 2024-04-17T10:50:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102816 s, 102 MB/s 2024-04-17T10:50:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109131 s, 96.1 MB/s 2024-04-17T10:50:58.290 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.290 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.290 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110641 s, 94.8 MB/s 2024-04-17T10:50:58.302 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.303 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.303 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104098 s, 101 MB/s 2024-04-17T10:50:58.314 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.315 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.315 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00964291 s, 109 MB/s 2024-04-17T10:50:58.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:58.316 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:50:58.316 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:50:58.316 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.22 2024-04-17T10:50:58.316 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:50:58.316 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:50:58.316 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:50:58.316 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:50:58.380 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.381 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.381 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00989651 s, 106 MB/s 2024-04-17T10:50:58.394 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.394 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.394 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111642 s, 93.9 MB/s 2024-04-17T10:50:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117265 s, 89.4 MB/s 2024-04-17T10:50:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.418 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.418 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748413 s, 140 MB/s 2024-04-17T10:50:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105207 s, 99.7 MB/s 2024-04-17T10:50:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109743 s, 95.5 MB/s 2024-04-17T10:50:58.456 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.456 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.456 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105755 s, 99.2 MB/s 2024-04-17T10:50:58.469 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.469 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.469 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116781 s, 89.8 MB/s 2024-04-17T10:50:58.482 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.482 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.482 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109457 s, 95.8 MB/s 2024-04-17T10:50:58.495 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.496 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.496 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109647 s, 95.6 MB/s 2024-04-17T10:50:58.508 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.508 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.508 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106183 s, 98.8 MB/s 2024-04-17T10:50:58.521 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.521 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.521 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107051 s, 98.0 MB/s 2024-04-17T10:50:58.535 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.536 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.536 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118935 s, 88.2 MB/s 2024-04-17T10:50:58.545 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.545 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.545 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744971 s, 141 MB/s 2024-04-17T10:50:58.560 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.560 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.560 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131856 s, 79.5 MB/s 2024-04-17T10:50:58.572 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.572 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.572 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00986487 s, 106 MB/s 2024-04-17T10:50:58.586 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.586 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.586 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120324 s, 87.1 MB/s 2024-04-17T10:50:58.599 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.599 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.599 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115636 s, 90.7 MB/s 2024-04-17T10:50:58.610 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.610 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.610 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817175 s, 128 MB/s 2024-04-17T10:50:58.624 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.624 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.624 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123287 s, 85.1 MB/s 2024-04-17T10:50:58.637 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.637 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.637 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106972 s, 98.0 MB/s 2024-04-17T10:50:58.646 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.647 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.647 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736557 s, 142 MB/s 2024-04-17T10:50:58.656 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.656 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.657 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074065 s, 142 MB/s 2024-04-17T10:50:58.670 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.670 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.670 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115592 s, 90.7 MB/s 2024-04-17T10:50:58.679 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.679 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.679 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749266 s, 140 MB/s 2024-04-17T10:50:58.691 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.691 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.691 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100511 s, 104 MB/s 2024-04-17T10:50:58.702 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.702 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.702 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00921467 s, 114 MB/s 2024-04-17T10:50:58.716 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.716 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.716 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118638 s, 88.4 MB/s 2024-04-17T10:50:58.729 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.729 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.729 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108532 s, 96.6 MB/s 2024-04-17T10:50:58.743 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.743 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.743 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115917 s, 90.5 MB/s 2024-04-17T10:50:58.756 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.756 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.756 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104781 s, 100 MB/s 2024-04-17T10:50:58.765 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.766 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.766 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763117 s, 137 MB/s 2024-04-17T10:50:58.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:58.767 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:50:58.767 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:50:58.767 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.23 2024-04-17T10:50:58.767 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:50:58.767 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:50:58.767 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:50:58.767 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:50:58.832 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.833 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.833 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119973 s, 87.4 MB/s 2024-04-17T10:50:58.845 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.845 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.845 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010416 s, 101 MB/s 2024-04-17T10:50:58.855 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.856 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.856 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080964 s, 130 MB/s 2024-04-17T10:50:58.868 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.868 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.868 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100054 s, 105 MB/s 2024-04-17T10:50:58.878 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.878 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.878 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781933 s, 134 MB/s 2024-04-17T10:50:58.890 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.890 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.890 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102112 s, 103 MB/s 2024-04-17T10:50:58.903 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.903 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.903 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106863 s, 98.1 MB/s 2024-04-17T10:50:58.917 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.917 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.918 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124753 s, 84.1 MB/s 2024-04-17T10:50:58.930 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.930 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.930 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100615 s, 104 MB/s 2024-04-17T10:50:58.942 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.942 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.942 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102012 s, 103 MB/s 2024-04-17T10:50:58.955 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.955 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.955 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102114 s, 103 MB/s 2024-04-17T10:50:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115879 s, 90.5 MB/s 2024-04-17T10:50:58.982 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.094 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.094 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109868 s, 95.4 MB/s 2024-04-17T10:50:59.096 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.096 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.096 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760861 s, 138 MB/s 2024-04-17T10:50:59.096 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.096 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.096 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120533 s, 87.0 MB/s 2024-04-17T10:50:59.096 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100241 s, 105 MB/s 2024-04-17T10:50:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119513 s, 87.7 MB/s 2024-04-17T10:50:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124199 s, 84.4 MB/s 2024-04-17T10:50:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113889 s, 92.1 MB/s 2024-04-17T10:50:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103997 s, 101 MB/s 2024-04-17T10:50:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00863617 s, 121 MB/s 2024-04-17T10:50:59.105 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.105 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.105 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707719 s, 148 MB/s 2024-04-17T10:50:59.118 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.119 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.119 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110645 s, 94.8 MB/s 2024-04-17T10:50:59.131 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.131 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.131 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100364 s, 104 MB/s 2024-04-17T10:50:59.146 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.146 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.146 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106268 s, 98.7 MB/s 2024-04-17T10:50:59.159 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.159 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.159 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119098 s, 88.0 MB/s 2024-04-17T10:50:59.173 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.174 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.174 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111174 s, 94.3 MB/s 2024-04-17T10:50:59.189 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.190 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.190 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128815 s, 81.4 MB/s 2024-04-17T10:50:59.205 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.205 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.205 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123006 s, 85.2 MB/s 2024-04-17T10:50:59.219 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.219 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.219 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113235 s, 92.6 MB/s 2024-04-17T10:50:59.232 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.232 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.232 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108685 s, 96.5 MB/s 2024-04-17T10:50:59.246 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.246 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.246 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110481 s, 94.9 MB/s 2024-04-17T10:50:59.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:59.247 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:50:59.247 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:50:59.247 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.24 2024-04-17T10:50:59.247 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:50:59.247 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:50:59.247 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:50:59.247 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:50:59.313 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.313 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.313 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112245 s, 93.4 MB/s 2024-04-17T10:50:59.327 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.327 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.327 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122059 s, 85.9 MB/s 2024-04-17T10:50:59.338 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.338 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.338 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828417 s, 127 MB/s 2024-04-17T10:50:59.352 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.352 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.352 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118598 s, 88.4 MB/s 2024-04-17T10:50:59.367 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.367 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.367 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127061 s, 82.5 MB/s 2024-04-17T10:50:59.379 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.380 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.380 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00965536 s, 109 MB/s 2024-04-17T10:50:59.392 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.392 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.392 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00999114 s, 105 MB/s 2024-04-17T10:50:59.406 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.406 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.406 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120782 s, 86.8 MB/s 2024-04-17T10:50:59.417 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.417 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.417 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877504 s, 119 MB/s 2024-04-17T10:50:59.431 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.431 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.432 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120006 s, 87.4 MB/s 2024-04-17T10:50:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104542 s, 100 MB/s 2024-04-17T10:50:59.459 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.459 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.459 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117584 s, 89.2 MB/s 2024-04-17T10:50:59.472 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.472 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.473 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117941 s, 88.9 MB/s 2024-04-17T10:50:59.485 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.485 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.485 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104537 s, 100 MB/s 2024-04-17T10:50:59.495 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.495 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.495 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756174 s, 139 MB/s 2024-04-17T10:50:59.504 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.504 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.504 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749871 s, 140 MB/s 2024-04-17T10:50:59.519 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.520 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.520 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131651 s, 79.6 MB/s 2024-04-17T10:50:59.533 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.533 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.533 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116759 s, 89.8 MB/s 2024-04-17T10:50:59.547 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.547 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.547 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115444 s, 90.8 MB/s 2024-04-17T10:50:59.561 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.562 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.562 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122335 s, 85.7 MB/s 2024-04-17T10:50:59.575 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.575 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.575 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115594 s, 90.7 MB/s 2024-04-17T10:50:59.586 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.586 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.586 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00892786 s, 117 MB/s 2024-04-17T10:50:59.601 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.601 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.602 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128482 s, 81.6 MB/s 2024-04-17T10:50:59.615 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.616 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.616 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011975 s, 87.6 MB/s 2024-04-17T10:50:59.629 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.629 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.629 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010953 s, 95.7 MB/s 2024-04-17T10:50:59.643 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.643 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.643 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115384 s, 90.9 MB/s 2024-04-17T10:50:59.657 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.657 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.657 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121386 s, 86.4 MB/s 2024-04-17T10:50:59.671 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.671 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.671 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115926 s, 90.5 MB/s 2024-04-17T10:50:59.686 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.686 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.686 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123957 s, 84.6 MB/s 2024-04-17T10:50:59.704 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.704 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.704 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162158 s, 64.7 MB/s 2024-04-17T10:50:59.715 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.715 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.716 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0090417 s, 116 MB/s 2024-04-17T10:50:59.728 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.728 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.729 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108071 s, 97.0 MB/s 2024-04-17T10:50:59.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:50:59.730 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:50:59.730 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:50:59.730 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.25 2024-04-17T10:50:59.730 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:50:59.730 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:50:59.730 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:50:59.730 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:50:59.793 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.793 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.793 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111395 s, 94.1 MB/s 2024-04-17T10:50:59.802 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.802 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.802 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729483 s, 144 MB/s 2024-04-17T10:50:59.817 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.817 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.817 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126468 s, 82.9 MB/s 2024-04-17T10:50:59.831 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.831 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.831 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119499 s, 87.7 MB/s 2024-04-17T10:50:59.844 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.844 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.844 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110123 s, 95.2 MB/s 2024-04-17T10:50:59.857 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.857 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.857 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105057 s, 99.8 MB/s 2024-04-17T10:50:59.873 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.873 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.873 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129201 s, 81.2 MB/s 2024-04-17T10:50:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119878 s, 87.5 MB/s 2024-04-17T10:50:59.904 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.904 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.904 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118823 s, 88.2 MB/s 2024-04-17T10:50:59.921 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.921 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.921 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133017 s, 78.8 MB/s 2024-04-17T10:50:59.936 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.936 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.936 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115164 s, 91.1 MB/s 2024-04-17T10:50:59.953 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.953 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.953 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128728 s, 81.5 MB/s 2024-04-17T10:50:59.967 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.967 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.967 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110308 s, 95.1 MB/s 2024-04-17T10:50:59.979 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.979 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.979 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010088 s, 104 MB/s 2024-04-17T10:50:59.993 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:50:59.993 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:50:59.993 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118398 s, 88.6 MB/s 2024-04-17T10:51:00.006 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.006 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.006 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100462 s, 104 MB/s 2024-04-17T10:51:00.019 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.019 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.019 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107562 s, 97.5 MB/s 2024-04-17T10:51:00.029 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.030 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.030 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793015 s, 132 MB/s 2024-04-17T10:51:00.043 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.043 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.043 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116484 s, 90.0 MB/s 2024-04-17T10:51:00.070 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.070 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.070 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815064 s, 129 MB/s 2024-04-17T10:51:00.084 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.084 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.084 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118767 s, 88.3 MB/s 2024-04-17T10:51:00.097 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.097 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.098 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111127 s, 94.4 MB/s 2024-04-17T10:51:00.110 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.110 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.110 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104886 s, 100 MB/s 2024-04-17T10:51:00.124 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.124 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.124 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113782 s, 92.2 MB/s 2024-04-17T10:51:00.138 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.138 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.138 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109808 s, 95.5 MB/s 2024-04-17T10:51:00.150 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.150 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.150 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00988302 s, 106 MB/s 2024-04-17T10:51:00.167 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.167 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.167 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111434 s, 94.1 MB/s 2024-04-17T10:51:00.177 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.177 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.177 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106725 s, 98.2 MB/s 2024-04-17T10:51:00.191 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.191 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.191 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115944 s, 90.4 MB/s 2024-04-17T10:51:00.204 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.204 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.204 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106782 s, 98.2 MB/s 2024-04-17T10:51:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125575 s, 83.5 MB/s 2024-04-17T10:51:00.234 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.234 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.234 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121272 s, 86.5 MB/s 2024-04-17T10:51:00.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:51:00.235 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:51:00.235 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:51:00.235 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.26 2024-04-17T10:51:00.235 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:51:00.235 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:51:00.235 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:51:00.236 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:51:00.304 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.304 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.304 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119283 s, 87.9 MB/s 2024-04-17T10:51:00.319 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.320 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.320 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130318 s, 80.5 MB/s 2024-04-17T10:51:00.334 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.334 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.334 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113504 s, 92.4 MB/s 2024-04-17T10:51:00.347 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.347 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.347 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105911 s, 99.0 MB/s 2024-04-17T10:51:00.362 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.362 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.362 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122458 s, 85.6 MB/s 2024-04-17T10:51:00.377 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.377 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.377 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122419 s, 85.7 MB/s 2024-04-17T10:51:00.391 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.391 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.391 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117087 s, 89.6 MB/s 2024-04-17T10:51:00.411 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.411 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.411 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154865 s, 67.7 MB/s 2024-04-17T10:51:00.427 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.427 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.427 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122439 s, 85.6 MB/s 2024-04-17T10:51:00.446 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.446 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.446 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153585 s, 68.3 MB/s 2024-04-17T10:51:00.463 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.463 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.463 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122423 s, 85.7 MB/s 2024-04-17T10:51:00.481 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.481 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.481 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141421 s, 74.1 MB/s 2024-04-17T10:51:00.499 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.499 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.499 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141112 s, 74.3 MB/s 2024-04-17T10:51:00.518 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.518 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.518 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149926 s, 69.9 MB/s 2024-04-17T10:51:00.533 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.533 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.533 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114278 s, 91.8 MB/s 2024-04-17T10:51:00.548 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.548 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.548 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012066 s, 86.9 MB/s 2024-04-17T10:51:00.561 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.561 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.561 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107029 s, 98.0 MB/s 2024-04-17T10:51:00.575 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.575 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.575 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114841 s, 91.3 MB/s 2024-04-17T10:51:00.588 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.588 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.589 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112209 s, 93.4 MB/s 2024-04-17T10:51:00.601 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.601 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.601 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101819 s, 103 MB/s 2024-04-17T10:51:00.615 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.615 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.615 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011411 s, 91.9 MB/s 2024-04-17T10:51:00.629 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.629 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.629 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118861 s, 88.2 MB/s 2024-04-17T10:51:00.642 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.642 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.642 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110145 s, 95.2 MB/s 2024-04-17T10:51:00.656 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.656 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.656 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112568 s, 93.2 MB/s 2024-04-17T10:51:00.668 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.669 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.669 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106777 s, 98.2 MB/s 2024-04-17T10:51:00.683 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.683 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.683 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120864 s, 86.8 MB/s 2024-04-17T10:51:00.697 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.697 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.697 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115581 s, 90.7 MB/s 2024-04-17T10:51:00.710 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.711 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.711 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113433 s, 92.4 MB/s 2024-04-17T10:51:00.724 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.724 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.724 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113131 s, 92.7 MB/s 2024-04-17T10:51:00.735 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.735 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.735 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826615 s, 127 MB/s 2024-04-17T10:51:00.748 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.748 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.748 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111399 s, 94.1 MB/s 2024-04-17T10:51:00.763 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.763 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.763 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119225 s, 87.9 MB/s 2024-04-17T10:51:00.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:51:00.764 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:51:00.764 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:51:00.764 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.27 2024-04-17T10:51:00.764 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:51:00.764 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:51:00.764 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:51:00.764 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:51:00.827 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.827 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.827 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00942969 s, 111 MB/s 2024-04-17T10:51:00.841 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.841 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.841 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114218 s, 91.8 MB/s 2024-04-17T10:51:00.855 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.855 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.855 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01181 s, 88.8 MB/s 2024-04-17T10:51:00.868 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.869 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.869 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117479 s, 89.3 MB/s 2024-04-17T10:51:00.882 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.882 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.882 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110737 s, 94.7 MB/s 2024-04-17T10:51:00.895 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.896 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.896 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113729 s, 92.2 MB/s 2024-04-17T10:51:00.906 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.906 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.906 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832295 s, 126 MB/s 2024-04-17T10:51:00.919 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.919 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.919 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107948 s, 97.1 MB/s 2024-04-17T10:51:00.933 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.933 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.933 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119001 s, 88.1 MB/s 2024-04-17T10:51:00.945 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.945 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.945 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00977041 s, 107 MB/s 2024-04-17T10:51:00.960 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.960 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.960 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130022 s, 80.6 MB/s 2024-04-17T10:51:00.974 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.974 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.974 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111762 s, 93.8 MB/s 2024-04-17T10:51:00.988 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.988 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.988 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119035 s, 88.1 MB/s 2024-04-17T10:51:00.999 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:00.999 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:00.999 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830555 s, 126 MB/s 2024-04-17T10:51:01.012 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.013 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.013 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117895 s, 88.9 MB/s 2024-04-17T10:51:01.027 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.027 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.027 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122531 s, 85.6 MB/s 2024-04-17T10:51:01.037 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.038 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.038 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836418 s, 125 MB/s 2024-04-17T10:51:01.056 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.057 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.057 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169161 s, 62.0 MB/s 2024-04-17T10:51:01.075 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.075 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.075 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100912 s, 104 MB/s 2024-04-17T10:51:01.088 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.088 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.088 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112692 s, 93.0 MB/s 2024-04-17T10:51:01.100 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.100 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.100 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0097491 s, 108 MB/s 2024-04-17T10:51:01.114 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.114 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.114 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011875 s, 88.3 MB/s 2024-04-17T10:51:01.123 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.124 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.124 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737713 s, 142 MB/s 2024-04-17T10:51:01.135 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.135 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.135 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00954345 s, 110 MB/s 2024-04-17T10:51:01.147 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.147 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.147 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00950189 s, 110 MB/s 2024-04-17T10:51:01.157 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.157 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.157 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804662 s, 130 MB/s 2024-04-17T10:51:01.170 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.170 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.170 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104975 s, 99.9 MB/s 2024-04-17T10:51:01.181 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.181 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.181 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00962134 s, 109 MB/s 2024-04-17T10:51:01.195 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.195 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.195 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119039 s, 88.1 MB/s 2024-04-17T10:51:01.210 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.210 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.210 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012668 s, 82.8 MB/s 2024-04-17T10:51:01.222 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.222 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.223 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103276 s, 102 MB/s 2024-04-17T10:51:01.235 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.235 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.235 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00957141 s, 110 MB/s 2024-04-17T10:51:01.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:51:01.235 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:51:01.235 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:51:01.236 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.28 2024-04-17T10:51:01.236 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:51:01.236 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:51:01.236 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:51:01.236 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:51:01.302 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.302 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.302 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125115 s, 83.8 MB/s 2024-04-17T10:51:01.317 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.317 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.317 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127431 s, 82.3 MB/s 2024-04-17T10:51:01.328 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.328 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.329 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00972093 s, 108 MB/s 2024-04-17T10:51:01.343 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.343 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.343 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126617 s, 82.8 MB/s 2024-04-17T10:51:01.357 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.357 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.357 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113594 s, 92.3 MB/s 2024-04-17T10:51:01.369 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.369 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.369 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105486 s, 99.4 MB/s 2024-04-17T10:51:01.379 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.379 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.380 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785005 s, 134 MB/s 2024-04-17T10:51:01.392 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.393 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.393 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111502 s, 94.0 MB/s 2024-04-17T10:51:01.402 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.402 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.402 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075409 s, 139 MB/s 2024-04-17T10:51:01.416 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.416 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.416 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120517 s, 87.0 MB/s 2024-04-17T10:51:01.429 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.429 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.429 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103429 s, 101 MB/s 2024-04-17T10:51:01.439 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.440 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.440 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845711 s, 124 MB/s 2024-04-17T10:51:01.451 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.451 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.451 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822197 s, 128 MB/s 2024-04-17T10:51:01.466 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.466 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.466 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012929 s, 81.1 MB/s 2024-04-17T10:51:01.475 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.475 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.475 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709365 s, 148 MB/s 2024-04-17T10:51:01.488 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.488 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.488 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106685 s, 98.3 MB/s 2024-04-17T10:51:01.499 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.499 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.499 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00900951 s, 116 MB/s 2024-04-17T10:51:01.516 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.516 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.516 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147954 s, 70.9 MB/s 2024-04-17T10:51:01.530 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.530 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.530 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115677 s, 90.6 MB/s 2024-04-17T10:51:01.541 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.541 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.541 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867229 s, 121 MB/s 2024-04-17T10:51:01.555 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.555 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.555 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121535 s, 86.3 MB/s 2024-04-17T10:51:01.568 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.568 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.568 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107829 s, 97.2 MB/s 2024-04-17T10:51:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.583 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119766 s, 87.6 MB/s 2024-04-17T10:51:01.597 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.597 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.598 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125993 s, 83.2 MB/s 2024-04-17T10:51:01.610 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.610 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.610 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00954478 s, 110 MB/s 2024-04-17T10:51:01.622 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.623 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.623 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106446 s, 98.5 MB/s 2024-04-17T10:51:01.637 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.637 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.637 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125534 s, 83.5 MB/s 2024-04-17T10:51:01.651 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.651 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.651 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109736 s, 95.6 MB/s 2024-04-17T10:51:01.668 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.668 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.668 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146922 s, 71.4 MB/s 2024-04-17T10:51:01.678 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.679 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.679 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834987 s, 126 MB/s 2024-04-17T10:51:01.693 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.693 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.693 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117096 s, 89.5 MB/s 2024-04-17T10:51:01.706 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.706 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.706 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109775 s, 95.5 MB/s 2024-04-17T10:51:01.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:51:01.707 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:51:01.707 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:51:01.708 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.29 2024-04-17T10:51:01.708 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:51:01.708 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:51:01.708 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:51:01.708 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:51:01.769 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.769 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.769 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773628 s, 136 MB/s 2024-04-17T10:51:01.785 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.786 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.786 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138034 s, 76.0 MB/s 2024-04-17T10:51:01.800 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.800 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.800 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121925 s, 86.0 MB/s 2024-04-17T10:51:01.817 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.817 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.817 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139295 s, 75.3 MB/s 2024-04-17T10:51:01.831 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.831 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.831 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114752 s, 91.4 MB/s 2024-04-17T10:51:01.843 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.843 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.843 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0095958 s, 109 MB/s 2024-04-17T10:51:01.853 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.854 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.854 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781432 s, 134 MB/s 2024-04-17T10:51:01.868 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.868 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.868 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114505 s, 91.6 MB/s 2024-04-17T10:51:01.883 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.883 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.883 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129487 s, 81.0 MB/s 2024-04-17T10:51:01.896 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.897 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.897 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108808 s, 96.4 MB/s 2024-04-17T10:51:01.913 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.913 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.913 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137967 s, 76.0 MB/s 2024-04-17T10:51:01.928 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.928 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.928 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01234 s, 85.0 MB/s 2024-04-17T10:51:01.941 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.941 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.941 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105779 s, 99.1 MB/s 2024-04-17T10:51:01.955 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.955 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.955 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115288 s, 91.0 MB/s 2024-04-17T10:51:01.969 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.969 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.969 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114315 s, 91.7 MB/s 2024-04-17T10:51:01.984 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.984 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.984 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120263 s, 87.2 MB/s 2024-04-17T10:51:01.997 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:01.997 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:01.998 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110319 s, 95.0 MB/s 2024-04-17T10:51:02.013 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.013 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.013 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01257 s, 83.4 MB/s 2024-04-17T10:51:02.029 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.029 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.029 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013004 s, 80.6 MB/s 2024-04-17T10:51:02.044 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.044 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.044 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126609 s, 82.8 MB/s 2024-04-17T10:51:02.060 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.060 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.060 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129531 s, 81.0 MB/s 2024-04-17T10:51:02.108 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.108 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.108 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011635 s, 90.1 MB/s 2024-04-17T10:51:02.124 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.124 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.124 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127341 s, 82.3 MB/s 2024-04-17T10:51:02.138 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.139 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.139 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113887 s, 92.1 MB/s 2024-04-17T10:51:02.155 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.155 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.155 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132145 s, 79.4 MB/s 2024-04-17T10:51:02.172 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.172 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.172 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135645 s, 77.3 MB/s 2024-04-17T10:51:02.189 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.189 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.189 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127149 s, 82.5 MB/s 2024-04-17T10:51:02.205 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.205 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.205 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130698 s, 80.2 MB/s 2024-04-17T10:51:02.218 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.218 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.218 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00896469 s, 117 MB/s 2024-04-17T10:51:02.236 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.236 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.236 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149387 s, 70.2 MB/s 2024-04-17T10:51:02.250 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.250 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.250 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114793 s, 91.3 MB/s 2024-04-17T10:51:02.265 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.265 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.265 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126379 s, 83.0 MB/s 2024-04-17T10:51:02.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:51:02.267 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:51:02.267 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:51:02.267 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.30 2024-04-17T10:51:02.267 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:51:02.267 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:51:02.267 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:51:02.267 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:51:02.337 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.337 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.337 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136727 s, 76.7 MB/s 2024-04-17T10:51:02.354 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.354 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.354 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137032 s, 76.5 MB/s 2024-04-17T10:51:02.368 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.368 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.368 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112797 s, 93.0 MB/s 2024-04-17T10:51:02.383 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.383 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.383 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120584 s, 87.0 MB/s 2024-04-17T10:51:02.399 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.399 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.399 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125348 s, 83.7 MB/s 2024-04-17T10:51:02.415 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.415 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.415 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129752 s, 80.8 MB/s 2024-04-17T10:51:02.431 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.431 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.431 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129214 s, 81.2 MB/s 2024-04-17T10:51:02.448 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.449 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.449 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144411 s, 72.6 MB/s 2024-04-17T10:51:02.466 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.466 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.466 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133991 s, 78.3 MB/s 2024-04-17T10:51:02.483 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.483 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.483 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142144 s, 73.8 MB/s 2024-04-17T10:51:02.498 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.498 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.498 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125264 s, 83.7 MB/s 2024-04-17T10:51:02.515 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.515 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.515 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131272 s, 79.9 MB/s 2024-04-17T10:51:02.528 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.528 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.528 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00989187 s, 106 MB/s 2024-04-17T10:51:02.542 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.542 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.543 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119959 s, 87.4 MB/s 2024-04-17T10:51:02.554 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.555 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.555 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0095631 s, 110 MB/s 2024-04-17T10:51:02.567 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.567 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.567 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0099741 s, 105 MB/s 2024-04-17T10:51:02.582 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.582 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.582 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119883 s, 87.5 MB/s 2024-04-17T10:51:02.594 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.594 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.594 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00942468 s, 111 MB/s 2024-04-17T10:51:02.605 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.605 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.605 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860914 s, 122 MB/s 2024-04-17T10:51:02.619 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.619 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.619 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118067 s, 88.8 MB/s 2024-04-17T10:51:02.630 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.630 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.630 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798428 s, 131 MB/s 2024-04-17T10:51:02.644 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.644 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.644 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117777 s, 89.0 MB/s 2024-04-17T10:51:02.658 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.659 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.659 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116659 s, 89.9 MB/s 2024-04-17T10:51:02.673 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.673 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.673 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126994 s, 82.6 MB/s 2024-04-17T10:51:02.687 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.687 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.687 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111824 s, 93.8 MB/s 2024-04-17T10:51:02.701 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.701 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.701 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120948 s, 86.7 MB/s 2024-04-17T10:51:02.714 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.715 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.715 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108079 s, 97.0 MB/s 2024-04-17T10:51:02.726 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.726 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.726 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00897029 s, 117 MB/s 2024-04-17T10:51:02.739 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.739 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.739 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010657 s, 98.4 MB/s 2024-04-17T10:51:02.751 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.751 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.751 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010014 s, 105 MB/s 2024-04-17T10:51:02.766 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.766 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.766 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119438 s, 87.8 MB/s 2024-04-17T10:51:02.777 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.777 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.778 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00966076 s, 109 MB/s 2024-04-17T10:51:02.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:51:02.779 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-17T10:51:02.779 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-17T10:51:02.779 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p scrub_control_test_path/dir.31 2024-04-17T10:51:02.779 DEBUG:teuthology.orchestra.run.smithi083:> for ((i = 0; i < 32; i++)); do 2024-04-17T10:51:02.779 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-17T10:51:02.779 DEBUG:teuthology.orchestra.run.smithi083:> done 2024-04-17T10:51:02.779 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-17T10:51:02.847 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.847 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.847 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119873 s, 87.5 MB/s 2024-04-17T10:51:02.860 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.860 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.860 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112857 s, 92.9 MB/s 2024-04-17T10:51:02.874 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.874 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.874 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107875 s, 97.2 MB/s 2024-04-17T10:51:02.886 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.886 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.886 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00916866 s, 114 MB/s 2024-04-17T10:51:02.898 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.898 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.898 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0097608 s, 107 MB/s 2024-04-17T10:51:02.912 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.912 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.912 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116192 s, 90.2 MB/s 2024-04-17T10:51:02.922 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.922 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.922 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762776 s, 137 MB/s 2024-04-17T10:51:02.935 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.935 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.935 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010558 s, 99.3 MB/s 2024-04-17T10:51:02.948 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.948 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.948 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112598 s, 93.1 MB/s 2024-04-17T10:51:02.959 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.959 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.959 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780967 s, 134 MB/s 2024-04-17T10:51:02.974 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.975 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.975 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137369 s, 76.3 MB/s 2024-04-17T10:51:02.987 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:02.987 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106179 s, 98.8 MB/s 2024-04-17T10:51:03.003 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:03.003 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:03.004 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129095 s, 81.2 MB/s 2024-04-17T10:51:03.016 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:03.016 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:03.016 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100504 s, 104 MB/s 2024-04-17T10:51:03.030 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:03.030 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:03.030 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010857 s, 96.6 MB/s 2024-04-17T10:51:03.040 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:03.040 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:03.040 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801097 s, 131 MB/s 2024-04-17T10:51:03.054 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:03.054 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:03.054 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116715 s, 89.8 MB/s 2024-04-17T10:51:03.091 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:03.091 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:03.091 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105229 s, 99.6 MB/s 2024-04-17T10:51:03.105 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:03.105 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:03.105 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011474 s, 91.4 MB/s 2024-04-17T10:51:03.115 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:03.115 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:03.115 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758768 s, 138 MB/s 2024-04-17T10:51:03.128 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:03.128 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:03.128 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107476 s, 97.6 MB/s 2024-04-17T10:51:03.140 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:03.140 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:03.140 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00991157 s, 106 MB/s 2024-04-17T10:51:03.153 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:03.153 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:03.153 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108794 s, 96.4 MB/s 2024-04-17T10:51:03.165 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:03.165 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:03.165 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00987738 s, 106 MB/s 2024-04-17T10:51:03.180 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:03.180 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:03.180 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123314 s, 85.0 MB/s 2024-04-17T10:51:03.192 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:03.193 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:03.193 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010459 s, 100 MB/s 2024-04-17T10:51:03.206 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:03.206 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:03.206 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110239 s, 95.1 MB/s 2024-04-17T10:51:03.221 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:03.221 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:03.221 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125203 s, 83.8 MB/s 2024-04-17T10:51:03.236 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:03.237 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:03.237 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125248 s, 83.7 MB/s 2024-04-17T10:51:03.249 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:03.249 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:03.249 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103799 s, 101 MB/s 2024-04-17T10:51:03.263 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:03.263 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:03.264 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116682 s, 89.9 MB/s 2024-04-17T10:51:03.279 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-17T10:51:03.279 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-17T10:51:03.279 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128973 s, 81.3 MB/s 2024-04-17T10:51:03.280 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 1 2024-04-17T10:51:03.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.460+0000 7fc7c7611640 1 Processor -- start 2024-04-17T10:51:03.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.460+0000 7fc7c7611640 1 -- start start 2024-04-17T10:51:03.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.460+0000 7fc7c7611640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc7c0154670 0x7fc7c00aea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:03.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.460+0000 7fc7c7611640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc7c00b4090 0x7fc7c006c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:03.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.460+0000 7fc7c7611640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c00b2ff0 con 0x7fc7c00b4520 2024-04-17T10:51:03.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.460+0000 7fc7c7611640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c0154a50 con 0x7fc7c00b4090 2024-04-17T10:51:03.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.460+0000 7fc7c7611640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c00b32d0 con 0x7fc7c0154670 2024-04-17T10:51:03.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.460+0000 7fc7c4bac640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc7c00b4090 0x7fc7c006c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:03.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.460+0000 7fc7c4bac640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc7c00b4090 0x7fc7c006c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60518/0 (socket says 172.21.15.83:60518) 2024-04-17T10:51:03.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.460+0000 7fc7c4bac640 1 -- 172.21.15.83:0/3620714523 learned_addr learned my addr 172.21.15.83:0/3620714523 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:03.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.460+0000 7fc7c4bac640 1 -- 172.21.15.83:0/3620714523 >> v2:172.21.15.167:3300/0 conn(0x7fc7c0154670 msgr2=0x7fc7c00aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:03.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.460+0000 7fc7c4bac640 1 --2- 172.21.15.83:0/3620714523 >> v2:172.21.15.167:3300/0 conn(0x7fc7c0154670 0x7fc7c00aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:03.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.460+0000 7fc7c4bac640 1 -- 172.21.15.83:0/3620714523 >> v1:172.21.15.167:6789/0 conn(0x7fc7c00b4520 legacy=0x7fc7c0070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:03.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.460+0000 7fc7c4bac640 1 -- 172.21.15.83:0/3620714523 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c00b35b0 con 0x7fc7c00b4090 2024-04-17T10:51:03.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.464+0000 7fc7c4bac640 1 --2- 172.21.15.83:0/3620714523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7c00b4090 0x7fc7c006c570 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7fc7c006bd70 tx=0x7fc7b401d880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ec6198b845d751 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:03.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.464+0000 7fc7bffff640 1 -- 172.21.15.83:0/3620714523 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b40b3070 con 0x7fc7c00b4090 2024-04-17T10:51:03.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.464+0000 7fc7bffff640 1 -- 172.21.15.83:0/3620714523 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b40a6930 con 0x7fc7c00b4090 2024-04-17T10:51:03.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.464+0000 7fc7bffff640 1 -- 172.21.15.83:0/3620714523 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b400b040 con 0x7fc7c00b4090 2024-04-17T10:51:03.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.464+0000 7fc7c7611640 1 -- 172.21.15.83:0/3620714523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7c00b4090 msgr2=0x7fc7c006c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:03.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.464+0000 7fc7c7611640 1 --2- 172.21.15.83:0/3620714523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7c00b4090 0x7fc7c006c570 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7fc7c006bd70 tx=0x7fc7b401d880 comp rx=0 tx=0).stop 2024-04-17T10:51:03.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.464+0000 7fc7c7611640 1 -- 172.21.15.83:0/3620714523 shutdown_connections 2024-04-17T10:51:03.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.464+0000 7fc7c7611640 1 --2- 172.21.15.83:0/3620714523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7c00b4090 0x7fc7c006c570 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:03.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.464+0000 7fc7c7611640 1 --2- 172.21.15.83:0/3620714523 >> v2:172.21.15.167:3300/0 conn(0x7fc7c0154670 0x7fc7c00aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:03.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.464+0000 7fc7c7611640 1 -- 172.21.15.83:0/3620714523 >> 172.21.15.83:0/3620714523 conn(0x7fc7c00b15b0 msgr2=0x7fc7c00b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:03.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.464+0000 7fc7c7611640 1 -- 172.21.15.83:0/3620714523 shutdown_connections 2024-04-17T10:51:03.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.464+0000 7fc7c7611640 1 -- 172.21.15.83:0/3620714523 wait complete. 2024-04-17T10:51:03.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.468+0000 7fc7c7611640 1 Processor -- start 2024-04-17T10:51:03.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.468+0000 7fc7c7611640 1 -- start start 2024-04-17T10:51:03.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.468+0000 7fc7c7611640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7c00b4090 0x7fc7c01571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:03.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.468+0000 7fc7c7611640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7c00b4520 0x7fc7c01f4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:03.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.468+0000 7fc7c7611640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7c0154670 0x7fc7c013fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:03.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.468+0000 7fc7c7611640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c00aef90 con 0x7fc7c0154670 2024-04-17T10:51:03.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.468+0000 7fc7c7611640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c006cec0 con 0x7fc7c00b4520 2024-04-17T10:51:03.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.468+0000 7fc7c7611640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c00af110 con 0x7fc7c00b4090 2024-04-17T10:51:03.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.468+0000 7fc7c5bae640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7c0154670 0x7fc7c013fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:03.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.468+0000 7fc7c5bae640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7c0154670 0x7fc7c013fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60522/0 (socket says 172.21.15.83:60522) 2024-04-17T10:51:03.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.468+0000 7fc7c5bae640 1 -- 172.21.15.83:0/3040820746 learned_addr learned my addr 172.21.15.83:0/3040820746 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:03.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.468+0000 7fc7c4bac640 1 --2- 172.21.15.83:0/3040820746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7c00b4520 0x7fc7c01f4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:03.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.468+0000 7fc7c53ad640 1 --2- 172.21.15.83:0/3040820746 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7c00b4090 0x7fc7c01571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:03.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.468+0000 7fc7c5bae640 1 -- 172.21.15.83:0/3040820746 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7c00b4090 msgr2=0x7fc7c01571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:03.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.468+0000 7fc7c5bae640 1 --2- 172.21.15.83:0/3040820746 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7c00b4090 0x7fc7c01571c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:03.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.468+0000 7fc7c5bae640 1 -- 172.21.15.83:0/3040820746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7c00b4520 msgr2=0x7fc7c01f4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:03.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.468+0000 7fc7c5bae640 1 --2- 172.21.15.83:0/3040820746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7c00b4520 0x7fc7c01f4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:03.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.468+0000 7fc7c5bae640 1 -- 172.21.15.83:0/3040820746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c0156120 con 0x7fc7c0154670 2024-04-17T10:51:03.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.468+0000 7fc7c4bac640 1 --2- 172.21.15.83:0/3040820746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7c00b4520 0x7fc7c01f4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:51:03.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.468+0000 7fc7c5bae640 1 --2- 172.21.15.83:0/3040820746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7c0154670 0x7fc7c013fee0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7fc7b800c780 tx=0x7fc7b800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:03.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.468+0000 7fc7bdffb640 1 -- 172.21.15.83:0/3040820746 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8017070 con 0x7fc7c0154670 2024-04-17T10:51:03.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.468+0000 7fc7bdffb640 1 -- 172.21.15.83:0/3040820746 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b8003db0 con 0x7fc7c0154670 2024-04-17T10:51:03.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.468+0000 7fc7c7611640 1 -- 172.21.15.83:0/3040820746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c0156400 con 0x7fc7c0154670 2024-04-17T10:51:03.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.468+0000 7fc7bdffb640 1 -- 172.21.15.83:0/3040820746 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b80052b0 con 0x7fc7c0154670 2024-04-17T10:51:03.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.468+0000 7fc7c7611640 1 -- 172.21.15.83:0/3040820746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c01569c0 con 0x7fc7c0154670 2024-04-17T10:51:03.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.468+0000 7fc7c7611640 1 -- 172.21.15.83:0/3040820746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c0074f90 con 0x7fc7c0154670 2024-04-17T10:51:03.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.468+0000 7fc7bdffb640 1 -- 172.21.15.83:0/3040820746 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc7b801a070 con 0x7fc7c0154670 2024-04-17T10:51:03.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.472+0000 7fc7bdffb640 1 --2- 172.21.15.83:0/3040820746 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc7ac066f90 0x7fc7ac069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:03.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.472+0000 7fc7bdffb640 1 -- 172.21.15.83:0/3040820746 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fc7b8084660 con 0x7fc7c0154670 2024-04-17T10:51:03.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.472+0000 7fc7c53ad640 1 --2- 172.21.15.83:0/3040820746 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc7ac066f90 0x7fc7ac069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:03.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.472+0000 7fc7c53ad640 1 --2- 172.21.15.83:0/3040820746 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc7ac066f90 0x7fc7ac069450 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0006fd0 tx=0x7fc7b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:03.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.476+0000 7fc7bdffb640 1 -- 172.21.15.83:0/3040820746 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc7b8055650 con 0x7fc7c0154670 2024-04-17T10:51:03.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:03.668+0000 7fc7c7611640 1 -- 172.21.15.83:0/3040820746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7fc7c00b2a00 con 0x7fc7c0154670 2024-04-17T10:51:04.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.472+0000 7fc7bdffb640 1 -- 172.21.15.83:0/3040820746 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v88) v1 ==== 105+0+0 (secure 0 0 0) 0x7fc7b8054f50 con 0x7fc7c0154670 2024-04-17T10:51:04.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.476+0000 7fc7c7611640 1 -- 172.21.15.83:0/3040820746 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc7ac066f90 msgr2=0x7fc7ac069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:04.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.476+0000 7fc7c7611640 1 --2- 172.21.15.83:0/3040820746 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc7ac066f90 0x7fc7ac069450 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0006fd0 tx=0x7fc7b0008040 comp rx=0 tx=0).stop 2024-04-17T10:51:04.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.476+0000 7fc7c7611640 1 -- 172.21.15.83:0/3040820746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7c0154670 msgr2=0x7fc7c013fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:04.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.476+0000 7fc7c7611640 1 --2- 172.21.15.83:0/3040820746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7c0154670 0x7fc7c013fee0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7fc7b800c780 tx=0x7fc7b800cc40 comp rx=0 tx=0).stop 2024-04-17T10:51:04.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.476+0000 7fc7c7611640 1 -- 172.21.15.83:0/3040820746 shutdown_connections 2024-04-17T10:51:04.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.476+0000 7fc7c7611640 1 --2- 172.21.15.83:0/3040820746 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc7ac066f90 0x7fc7ac069450 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:04.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.476+0000 7fc7c7611640 1 --2- 172.21.15.83:0/3040820746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7c0154670 0x7fc7c013fee0 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:04.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.476+0000 7fc7c7611640 1 --2- 172.21.15.83:0/3040820746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7c00b4520 0x7fc7c01f4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:04.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.476+0000 7fc7c7611640 1 --2- 172.21.15.83:0/3040820746 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7c00b4090 0x7fc7c01571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:04.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.476+0000 7fc7c7611640 1 -- 172.21.15.83:0/3040820746 >> 172.21.15.83:0/3040820746 conn(0x7fc7c00b15b0 msgr2=0x7fc7c00b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:04.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.476+0000 7fc7c7611640 1 -- 172.21.15.83:0/3040820746 shutdown_connections 2024-04-17T10:51:04.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.476+0000 7fc7c7611640 1 -- 172.21.15.83:0/3040820746 wait complete. 2024-04-17T10:51:04.489 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-17T10:51:04.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.620+0000 7efdf3536640 1 Processor -- start 2024-04-17T10:51:04.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.620+0000 7efdf3536640 1 -- start start 2024-04-17T10:51:04.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.620+0000 7efdf3536640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7efdec15bab0 0x7efdec15be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:04.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.620+0000 7efdf3536640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efdec06c260 0x7efdec0acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:04.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.620+0000 7efdf3536640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efdec0ad470 0x7efdec072060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:04.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.620+0000 7efdf3536640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdec0ad8a0 con 0x7efdec06c260 2024-04-17T10:51:04.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.620+0000 7efdf3536640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdec06aef0 con 0x7efdec15bab0 2024-04-17T10:51:04.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.620+0000 7efdf3536640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdec06b1d0 con 0x7efdec0ad470 2024-04-17T10:51:04.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.620+0000 7efdf0ad1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efdec06c260 0x7efdec0acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:04.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.620+0000 7efdf0ad1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efdec06c260 0x7efdec0acf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60524/0 (socket says 172.21.15.83:60524) 2024-04-17T10:51:04.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.620+0000 7efdf0ad1640 1 -- 172.21.15.83:0/1755041732 learned_addr learned my addr 172.21.15.83:0/1755041732 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:04.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.620+0000 7efdf12d2640 1 --2- 172.21.15.83:0/1755041732 >> v2:172.21.15.167:3300/0 conn(0x7efdec15bab0 0x7efdec15be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:04.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.620+0000 7efdf1ad3640 1 --2- 172.21.15.83:0/1755041732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efdec0ad470 0x7efdec072060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:04.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.620+0000 7efdf0ad1640 1 -- 172.21.15.83:0/1755041732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efdec0ad470 msgr2=0x7efdec072060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:04.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.620+0000 7efdf0ad1640 1 --2- 172.21.15.83:0/1755041732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efdec0ad470 0x7efdec072060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:04.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.620+0000 7efdf0ad1640 1 -- 172.21.15.83:0/1755041732 >> v2:172.21.15.167:3300/0 conn(0x7efdec15bab0 msgr2=0x7efdec15be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:04.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.620+0000 7efdf0ad1640 1 --2- 172.21.15.83:0/1755041732 >> v2:172.21.15.167:3300/0 conn(0x7efdec15bab0 0x7efdec15be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:04.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.620+0000 7efdf0ad1640 1 -- 172.21.15.83:0/1755041732 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdec06b4b0 con 0x7efdec06c260 2024-04-17T10:51:04.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.624+0000 7efdf0ad1640 1 --2- 172.21.15.83:0/1755041732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdec06c260 0x7efdec0acf30 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7efdec0cfe40 tx=0x7efde001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07a3a5d6f3527ae server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:04.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.624+0000 7efdebfff640 1 -- 172.21.15.83:0/1755041732 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde00aa610 con 0x7efdec06c260 2024-04-17T10:51:04.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.624+0000 7efdebfff640 1 -- 172.21.15.83:0/1755041732 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde00af070 con 0x7efdec06c260 2024-04-17T10:51:04.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.624+0000 7efdebfff640 1 -- 172.21.15.83:0/1755041732 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde00aa9d0 con 0x7efdec06c260 2024-04-17T10:51:04.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.624+0000 7efdf3536640 1 -- 172.21.15.83:0/1755041732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdec06c260 msgr2=0x7efdec0acf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:04.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.624+0000 7efdf3536640 1 --2- 172.21.15.83:0/1755041732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdec06c260 0x7efdec0acf30 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7efdec0cfe40 tx=0x7efde001c4f0 comp rx=0 tx=0).stop 2024-04-17T10:51:04.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.624+0000 7efdf3536640 1 -- 172.21.15.83:0/1755041732 shutdown_connections 2024-04-17T10:51:04.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.624+0000 7efdf3536640 1 --2- 172.21.15.83:0/1755041732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efdec0ad470 0x7efdec072060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:04.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.624+0000 7efdf3536640 1 --2- 172.21.15.83:0/1755041732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdec06c260 0x7efdec0acf30 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:04.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.624+0000 7efdf3536640 1 --2- 172.21.15.83:0/1755041732 >> v2:172.21.15.167:3300/0 conn(0x7efdec15bab0 0x7efdec15be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:04.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.624+0000 7efdf3536640 1 -- 172.21.15.83:0/1755041732 >> 172.21.15.83:0/1755041732 conn(0x7efdec0b1e20 msgr2=0x7efdec0b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:04.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.624+0000 7efdf3536640 1 -- 172.21.15.83:0/1755041732 shutdown_connections 2024-04-17T10:51:04.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.624+0000 7efdf3536640 1 -- 172.21.15.83:0/1755041732 wait complete. 2024-04-17T10:51:04.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.624+0000 7efdf3536640 1 Processor -- start 2024-04-17T10:51:04.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.628+0000 7efdf3536640 1 -- start start 2024-04-17T10:51:04.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.628+0000 7efdf3536640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efdec06c260 0x7efdec0ce290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:04.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.628+0000 7efdf3536640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdec0ad470 0x7efdec1406d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:04.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.628+0000 7efdf3536640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdec15bab0 0x7efdec152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:04.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.628+0000 7efdf3536640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdec0ca8c0 con 0x7efdec0ad470 2024-04-17T10:51:04.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.628+0000 7efdf3536640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdec0ca740 con 0x7efdec15bab0 2024-04-17T10:51:04.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.628+0000 7efdf3536640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdec0caa40 con 0x7efdec06c260 2024-04-17T10:51:04.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.628+0000 7efdf0ad1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdec0ad470 0x7efdec1406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:04.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.628+0000 7efdf0ad1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdec0ad470 0x7efdec1406d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60534/0 (socket says 172.21.15.83:60534) 2024-04-17T10:51:04.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.628+0000 7efdf0ad1640 1 -- 172.21.15.83:0/2663457525 learned_addr learned my addr 172.21.15.83:0/2663457525 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:04.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.628+0000 7efdf12d2640 1 --2- 172.21.15.83:0/2663457525 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efdec06c260 0x7efdec0ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:04.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.628+0000 7efdf1ad3640 1 --2- 172.21.15.83:0/2663457525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdec15bab0 0x7efdec152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:04.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.628+0000 7efdf0ad1640 1 -- 172.21.15.83:0/2663457525 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efdec06c260 msgr2=0x7efdec0ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:04.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.628+0000 7efdf0ad1640 1 --2- 172.21.15.83:0/2663457525 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efdec06c260 0x7efdec0ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:04.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.628+0000 7efdf0ad1640 1 -- 172.21.15.83:0/2663457525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdec15bab0 msgr2=0x7efdec152a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:04.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.628+0000 7efdf0ad1640 1 --2- 172.21.15.83:0/2663457525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdec15bab0 0x7efdec152a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:04.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.628+0000 7efdf0ad1640 1 -- 172.21.15.83:0/2663457525 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdec140f30 con 0x7efdec0ad470 2024-04-17T10:51:04.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.628+0000 7efdf1ad3640 1 --2- 172.21.15.83:0/2663457525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdec15bab0 0x7efdec152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:51:04.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.628+0000 7efdf0ad1640 1 --2- 172.21.15.83:0/2663457525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdec0ad470 0x7efdec1406d0 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7efde001aba0 tx=0x7efde001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:04.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.628+0000 7efde9ffb640 1 -- 172.21.15.83:0/2663457525 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde00bd5f0 con 0x7efdec0ad470 2024-04-17T10:51:04.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.628+0000 7efde9ffb640 1 -- 172.21.15.83:0/2663457525 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde00bd790 con 0x7efdec0ad470 2024-04-17T10:51:04.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.628+0000 7efdf3536640 1 -- 172.21.15.83:0/2663457525 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdec141210 con 0x7efdec0ad470 2024-04-17T10:51:04.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.628+0000 7efde9ffb640 1 -- 172.21.15.83:0/2663457525 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde00bdb80 con 0x7efdec0ad470 2024-04-17T10:51:04.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.628+0000 7efdf3536640 1 -- 172.21.15.83:0/2663457525 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdec152210 con 0x7efdec0ad470 2024-04-17T10:51:04.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.628+0000 7efdf3536640 1 -- 172.21.15.83:0/2663457525 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdec074f90 con 0x7efdec0ad470 2024-04-17T10:51:04.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.628+0000 7efde9ffb640 1 -- 172.21.15.83:0/2663457525 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efde00b33f0 con 0x7efdec0ad470 2024-04-17T10:51:04.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.628+0000 7efde9ffb640 1 --2- 172.21.15.83:0/2663457525 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efdd8066f90 0x7efdd8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:04.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.628+0000 7efde9ffb640 1 -- 172.21.15.83:0/2663457525 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5738+0+0 (secure 0 0 0) 0x7efde011fd10 con 0x7efdec0ad470 2024-04-17T10:51:04.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.628+0000 7efdf12d2640 1 --2- 172.21.15.83:0/2663457525 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efdd8066f90 0x7efdd8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:04.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.632+0000 7efdf12d2640 1 --2- 172.21.15.83:0/2663457525 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efdd8066f90 0x7efdd8069450 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7efde400b930 tx=0x7efde4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:04.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.632+0000 7efde9ffb640 1 -- 172.21.15.83:0/2663457525 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efde00c3060 con 0x7efdec0ad470 2024-04-17T10:51:04.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.824+0000 7efdf3536640 1 -- 172.21.15.83:0/2663457525 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7efdec06edd0 con 0x7efdec0ad470 2024-04-17T10:51:04.818 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-04-17T10:51:04.824+0000 7f35c3b81640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T10:51:04.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.828+0000 7efde9ffb640 1 -- 172.21.15.83:0/2663457525 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v89) v1 ==== 161+0+0 (secure 0 0 0) 0x7efde00a7070 con 0x7efdec0ad470 2024-04-17T10:51:04.823 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-17T10:51:04.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.832+0000 7efdf3536640 1 -- 172.21.15.83:0/2663457525 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efdd8066f90 msgr2=0x7efdd8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:04.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.832+0000 7efdf3536640 1 --2- 172.21.15.83:0/2663457525 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efdd8066f90 0x7efdd8069450 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7efde400b930 tx=0x7efde4008040 comp rx=0 tx=0).stop 2024-04-17T10:51:04.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.832+0000 7efdf3536640 1 -- 172.21.15.83:0/2663457525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdec0ad470 msgr2=0x7efdec1406d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:04.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.832+0000 7efdf3536640 1 --2- 172.21.15.83:0/2663457525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdec0ad470 0x7efdec1406d0 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7efde001aba0 tx=0x7efde001ad30 comp rx=0 tx=0).stop 2024-04-17T10:51:04.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.832+0000 7efdf3536640 1 -- 172.21.15.83:0/2663457525 shutdown_connections 2024-04-17T10:51:04.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.832+0000 7efdf3536640 1 --2- 172.21.15.83:0/2663457525 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efdd8066f90 0x7efdd8069450 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:04.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.832+0000 7efdf3536640 1 --2- 172.21.15.83:0/2663457525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdec15bab0 0x7efdec152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:04.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.832+0000 7efdf3536640 1 --2- 172.21.15.83:0/2663457525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdec0ad470 0x7efdec1406d0 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:04.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.832+0000 7efdf3536640 1 --2- 172.21.15.83:0/2663457525 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efdec06c260 0x7efdec0ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:04.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.832+0000 7efdf3536640 1 -- 172.21.15.83:0/2663457525 >> 172.21.15.83:0/2663457525 conn(0x7efdec0b1e20 msgr2=0x7efdec0ae090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:04.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.832+0000 7efdf3536640 1 -- 172.21.15.83:0/2663457525 shutdown_connections 2024-04-17T10:51:04.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.832+0000 7efdf3536640 1 -- 172.21.15.83:0/2663457525 wait complete. 2024-04-17T10:51:04.844 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-17T10:51:04.913 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-04-17T10:51:04.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.972+0000 7fb4d955e640 1 Processor -- start 2024-04-17T10:51:04.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.972+0000 7fb4d955e640 1 -- start start 2024-04-17T10:51:04.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.972+0000 7fb4d955e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4d40ce400 0x7fb4d40b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:04.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.972+0000 7fb4d955e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb4d40b5450 0x7fb4d40b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:04.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.972+0000 7fb4d955e640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d415e4d0 con 0x7fb4d415e0f0 2024-04-17T10:51:04.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.972+0000 7fb4d955e640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4d40b5df0 con 0x7fb4d40b5450 2024-04-17T10:51:04.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.972+0000 7fb4d955e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d4162b40 con 0x7fb4d40ce400 2024-04-17T10:51:04.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.972+0000 7fb4d37fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb4d40b5450 0x7fb4d40b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:04.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.972+0000 7fb4d2ffd640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fb4d415e0f0 0x7fb4d4161c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:57082/0 (socket says 172.21.15.83:57082) 2024-04-17T10:51:04.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.972+0000 7fb4d27fc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4d40ce400 0x7fb4d40b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:04.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.972+0000 7fb4d2ffd640 1 -- 172.21.15.83:0/2851053838 learned_addr learned my addr 172.21.15.83:0/2851053838 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:51:04.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.972+0000 7fb4d27fc640 1 -- 172.21.15.83:0/2851053838 >> v2:172.21.15.167:3300/0 conn(0x7fb4d40b5450 msgr2=0x7fb4d40b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:04.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.972+0000 7fb4d27fc640 1 --2- 172.21.15.83:0/2851053838 >> v2:172.21.15.167:3300/0 conn(0x7fb4d40b5450 0x7fb4d40b58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:04.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.972+0000 7fb4d27fc640 1 -- 172.21.15.83:0/2851053838 >> v1:172.21.15.167:6789/0 conn(0x7fb4d415e0f0 legacy=0x7fb4d4161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:04.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.972+0000 7fb4d27fc640 1 -- 172.21.15.83:0/2851053838 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d4162580 con 0x7fb4d40ce400 2024-04-17T10:51:04.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.972+0000 7fb4d37fe640 1 --2- 172.21.15.83:0/2851053838 >> v2:172.21.15.167:3300/0 conn(0x7fb4d40b5450 0x7fb4d40b58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:51:04.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.976+0000 7fb4d27fc640 1 --2- 172.21.15.83:0/2851053838 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4d40ce400 0x7fb4d40b6d00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0097c0 tx=0x7fb4cc01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f2f5512fc38149a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:04.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.976+0000 7fb4d1ffb640 1 -- 172.21.15.83:0/2851053838 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc0b0070 con 0x7fb4d40ce400 2024-04-17T10:51:04.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.976+0000 7fb4d1ffb640 1 -- 172.21.15.83:0/2851053838 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4cc0ab600 con 0x7fb4d40ce400 2024-04-17T10:51:04.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.976+0000 7fb4d1ffb640 1 -- 172.21.15.83:0/2851053838 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc0ab9d0 con 0x7fb4d40ce400 2024-04-17T10:51:04.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.976+0000 7fb4d955e640 1 -- 172.21.15.83:0/2851053838 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4d40ce400 msgr2=0x7fb4d40b6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:04.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.976+0000 7fb4d955e640 1 --2- 172.21.15.83:0/2851053838 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4d40ce400 0x7fb4d40b6d00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0097c0 tx=0x7fb4cc01c410 comp rx=0 tx=0).stop 2024-04-17T10:51:04.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.976+0000 7fb4d955e640 1 -- 172.21.15.83:0/2851053838 shutdown_connections 2024-04-17T10:51:04.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.976+0000 7fb4d955e640 1 --2- 172.21.15.83:0/2851053838 >> v2:172.21.15.167:3300/0 conn(0x7fb4d40b5450 0x7fb4d40b58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:04.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.976+0000 7fb4d955e640 1 --2- 172.21.15.83:0/2851053838 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4d40ce400 0x7fb4d40b6d00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:04.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.976+0000 7fb4d955e640 1 -- 172.21.15.83:0/2851053838 >> 172.21.15.83:0/2851053838 conn(0x7fb4d40d0450 msgr2=0x7fb4d40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:04.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.976+0000 7fb4d955e640 1 -- 172.21.15.83:0/2851053838 shutdown_connections 2024-04-17T10:51:04.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.976+0000 7fb4d955e640 1 -- 172.21.15.83:0/2851053838 wait complete. 2024-04-17T10:51:04.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.976+0000 7fb4d955e640 1 Processor -- start 2024-04-17T10:51:04.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.976+0000 7fb4d955e640 1 -- start start 2024-04-17T10:51:04.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.980+0000 7fb4d955e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4d40b5450 0x7fb4d414d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:04.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.980+0000 7fb4d955e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4d40ce400 0x7fb4d4153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:04.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.980+0000 7fb4d955e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4d415e0f0 0x7fb4d414b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:04.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.980+0000 7fb4d955e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d40d2ed0 con 0x7fb4d40ce400 2024-04-17T10:51:04.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.980+0000 7fb4d955e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d40d2d50 con 0x7fb4d415e0f0 2024-04-17T10:51:04.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.980+0000 7fb4d955e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d40d3050 con 0x7fb4d40b5450 2024-04-17T10:51:04.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.980+0000 7fb4d27fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4d40ce400 0x7fb4d4153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:04.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.980+0000 7fb4d27fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4d40ce400 0x7fb4d4153b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60570/0 (socket says 172.21.15.83:60570) 2024-04-17T10:51:04.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.980+0000 7fb4d27fc640 1 -- 172.21.15.83:0/647085897 learned_addr learned my addr 172.21.15.83:0/647085897 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:04.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.980+0000 7fb4d2ffd640 1 --2- 172.21.15.83:0/647085897 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4d40b5450 0x7fb4d414d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:04.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.980+0000 7fb4d37fe640 1 --2- 172.21.15.83:0/647085897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4d415e0f0 0x7fb4d414b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:04.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.980+0000 7fb4d27fc640 1 -- 172.21.15.83:0/647085897 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4d40b5450 msgr2=0x7fb4d414d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:04.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.980+0000 7fb4d27fc640 1 --2- 172.21.15.83:0/647085897 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4d40b5450 0x7fb4d414d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:04.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.980+0000 7fb4d27fc640 1 -- 172.21.15.83:0/647085897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4d415e0f0 msgr2=0x7fb4d414b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:04.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.980+0000 7fb4d27fc640 1 --2- 172.21.15.83:0/647085897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4d415e0f0 0x7fb4d414b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:04.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.980+0000 7fb4d27fc640 1 -- 172.21.15.83:0/647085897 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d41540c0 con 0x7fb4d40ce400 2024-04-17T10:51:04.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.980+0000 7fb4d27fc640 1 --2- 172.21.15.83:0/647085897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4d40ce400 0x7fb4d4153b80 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0ae040 tx=0x7fb4cc0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:04.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.980+0000 7fb4cb7fe640 1 -- 172.21.15.83:0/647085897 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc0b0040 con 0x7fb4d40ce400 2024-04-17T10:51:04.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.980+0000 7fb4cb7fe640 1 -- 172.21.15.83:0/647085897 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4cc0b9070 con 0x7fb4d40ce400 2024-04-17T10:51:04.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.980+0000 7fb4d955e640 1 -- 172.21.15.83:0/647085897 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4d4160fc0 con 0x7fb4d40ce400 2024-04-17T10:51:04.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.980+0000 7fb4cb7fe640 1 -- 172.21.15.83:0/647085897 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc0b4690 con 0x7fb4d40ce400 2024-04-17T10:51:04.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.980+0000 7fb4d955e640 1 -- 172.21.15.83:0/647085897 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d414b940 con 0x7fb4d40ce400 2024-04-17T10:51:04.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.980+0000 7fb4d955e640 1 -- 172.21.15.83:0/647085897 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4d40681a0 con 0x7fb4d40ce400 2024-04-17T10:51:04.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.980+0000 7fb4cb7fe640 1 -- 172.21.15.83:0/647085897 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb4cc0b4830 con 0x7fb4d40ce400 2024-04-17T10:51:04.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.980+0000 7fb4cb7fe640 1 --2- 172.21.15.83:0/647085897 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb4b4066f60 0x7fb4b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:04.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.980+0000 7fb4cb7fe640 1 -- 172.21.15.83:0/647085897 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fb4cc11ee50 con 0x7fb4d40ce400 2024-04-17T10:51:04.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.980+0000 7fb4d2ffd640 1 --2- 172.21.15.83:0/647085897 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb4b4066f60 0x7fb4b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:04.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.980+0000 7fb4d2ffd640 1 --2- 172.21.15.83:0/647085897 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb4b4066f60 0x7fb4b4069420 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc000f70 tx=0x7fb4bc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:04.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:04.984+0000 7fb4cb7fe640 1 -- 172.21.15.83:0/647085897 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb4cc016090 con 0x7fb4d40ce400 2024-04-17T10:51:05.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.168+0000 7fb4d955e640 1 -- 172.21.15.83:0/647085897 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fb4d4074f90 con 0x7fb4d40ce400 2024-04-17T10:51:05.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.832+0000 7fb4cb7fe640 1 -- 172.21.15.83:0/647085897 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v90) v1 ==== 121+0+0 (secure 0 0 0) 0x7fb4cc0efe00 con 0x7fb4d40ce400 2024-04-17T10:51:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.836+0000 7fb4d955e640 1 -- 172.21.15.83:0/647085897 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb4b4066f60 msgr2=0x7fb4b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.836+0000 7fb4d955e640 1 --2- 172.21.15.83:0/647085897 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb4b4066f60 0x7fb4b4069420 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc000f70 tx=0x7fb4bc009040 comp rx=0 tx=0).stop 2024-04-17T10:51:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.836+0000 7fb4d955e640 1 -- 172.21.15.83:0/647085897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4d40ce400 msgr2=0x7fb4d4153b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.836+0000 7fb4d955e640 1 --2- 172.21.15.83:0/647085897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4d40ce400 0x7fb4d4153b80 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0ae040 tx=0x7fb4cc0191b0 comp rx=0 tx=0).stop 2024-04-17T10:51:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.836+0000 7fb4d955e640 1 -- 172.21.15.83:0/647085897 shutdown_connections 2024-04-17T10:51:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.836+0000 7fb4d955e640 1 --2- 172.21.15.83:0/647085897 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb4b4066f60 0x7fb4b4069420 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.836+0000 7fb4d955e640 1 --2- 172.21.15.83:0/647085897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4d415e0f0 0x7fb4d414b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.836+0000 7fb4d955e640 1 --2- 172.21.15.83:0/647085897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4d40ce400 0x7fb4d4153b80 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.836+0000 7fb4d955e640 1 --2- 172.21.15.83:0/647085897 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4d40b5450 0x7fb4d414d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.836+0000 7fb4d955e640 1 -- 172.21.15.83:0/647085897 >> 172.21.15.83:0/647085897 conn(0x7fb4d40d0450 msgr2=0x7fb4d40b7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.836+0000 7fb4d955e640 1 -- 172.21.15.83:0/647085897 shutdown_connections 2024-04-17T10:51:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.836+0000 7fb4d955e640 1 -- 172.21.15.83:0/647085897 wait complete. 2024-04-17T10:51:05.849 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:51:05.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.984+0000 7f538d78b640 1 Processor -- start 2024-04-17T10:51:05.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.984+0000 7f538d78b640 1 -- start start 2024-04-17T10:51:05.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.984+0000 7f538d78b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f538815ba30 0x7f5388072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:05.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.984+0000 7f538d78b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f538815c960 0x7f53880cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:05.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.984+0000 7f538d78b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f53880cfa10 0x7f538806c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:05.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.984+0000 7f538d78b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5388071ea0 con 0x7f538815c960 2024-04-17T10:51:05.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.984+0000 7f538d78b640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53880cfea0 con 0x7f53880cfa10 2024-04-17T10:51:05.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.984+0000 7f538d78b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f538806e320 con 0x7f538815ba30 2024-04-17T10:51:05.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.984+0000 7f53867fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f538815c960 0x7f53880cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:05.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.984+0000 7f53867fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f538815c960 0x7f53880cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60580/0 (socket says 172.21.15.83:60580) 2024-04-17T10:51:05.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.984+0000 7f53867fc640 1 -- 172.21.15.83:0/4269634633 learned_addr learned my addr 172.21.15.83:0/4269634633 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:05.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.984+0000 7f5386ffd640 1 --2- 172.21.15.83:0/4269634633 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f538815ba30 0x7f5388072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:05.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.984+0000 7f53877fe640 1 --2- 172.21.15.83:0/4269634633 >> v2:172.21.15.167:3300/0 conn(0x7f53880cfa10 0x7f538806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:05.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.984+0000 7f53867fc640 1 -- 172.21.15.83:0/4269634633 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f538815ba30 msgr2=0x7f5388072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:05.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.984+0000 7f53867fc640 1 --2- 172.21.15.83:0/4269634633 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f538815ba30 0x7f5388072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:05.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.984+0000 7f53867fc640 1 -- 172.21.15.83:0/4269634633 >> v2:172.21.15.167:3300/0 conn(0x7f53880cfa10 msgr2=0x7f538806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:05.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.984+0000 7f53867fc640 1 --2- 172.21.15.83:0/4269634633 >> v2:172.21.15.167:3300/0 conn(0x7f53880cfa10 0x7f538806c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:05.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.984+0000 7f53867fc640 1 -- 172.21.15.83:0/4269634633 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538806e040 con 0x7f538815c960 2024-04-17T10:51:05.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.984+0000 7f5386ffd640 1 --2- 172.21.15.83:0/4269634633 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f538815ba30 0x7f5388072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:51:05.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.984+0000 7f53877fe640 1 --2- 172.21.15.83:0/4269634633 >> v2:172.21.15.167:3300/0 conn(0x7f53880cfa10 0x7f538806c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:51:05.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.988+0000 7f53867fc640 1 --2- 172.21.15.83:0/4269634633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f538815c960 0x7f53880cf4d0 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f538806d810 tx=0x7f537c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974fbbbc8a2020bd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:05.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.988+0000 7f5385ffb640 1 -- 172.21.15.83:0/4269634633 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c0b0070 con 0x7f538815c960 2024-04-17T10:51:05.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.988+0000 7f5385ffb640 1 -- 172.21.15.83:0/4269634633 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f537c0ab600 con 0x7f538815c960 2024-04-17T10:51:05.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.988+0000 7f5385ffb640 1 -- 172.21.15.83:0/4269634633 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c0ab9d0 con 0x7f538815c960 2024-04-17T10:51:05.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.988+0000 7f538d78b640 1 -- 172.21.15.83:0/4269634633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f538815c960 msgr2=0x7f53880cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:05.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.988+0000 7f538d78b640 1 --2- 172.21.15.83:0/4269634633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f538815c960 0x7f53880cf4d0 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f538806d810 tx=0x7f537c01c490 comp rx=0 tx=0).stop 2024-04-17T10:51:05.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.988+0000 7f538d78b640 1 -- 172.21.15.83:0/4269634633 shutdown_connections 2024-04-17T10:51:05.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.988+0000 7f538d78b640 1 --2- 172.21.15.83:0/4269634633 >> v2:172.21.15.167:3300/0 conn(0x7f53880cfa10 0x7f538806c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:05.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.988+0000 7f538d78b640 1 --2- 172.21.15.83:0/4269634633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f538815c960 0x7f53880cf4d0 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.988+0000 7f538d78b640 1 --2- 172.21.15.83:0/4269634633 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f538815ba30 0x7f5388072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.988+0000 7f538d78b640 1 -- 172.21.15.83:0/4269634633 >> 172.21.15.83:0/4269634633 conn(0x7f538806abe0 msgr2=0x7f538806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.988+0000 7f538d78b640 1 -- 172.21.15.83:0/4269634633 shutdown_connections 2024-04-17T10:51:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.988+0000 7f538d78b640 1 -- 172.21.15.83:0/4269634633 wait complete. 2024-04-17T10:51:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.988+0000 7f538d78b640 1 Processor -- start 2024-04-17T10:51:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.988+0000 7f538d78b640 1 -- start start 2024-04-17T10:51:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.988+0000 7f538d78b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53880cfa10 0x7f5388137b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.988+0000 7f538d78b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f538815ba30 0x7f538814cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.988+0000 7f538d78b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f538815c960 0x7f53880cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.988+0000 7f538d78b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538806dca0 con 0x7f538815ba30 2024-04-17T10:51:05.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.988+0000 7f538d78b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53880aeae0 con 0x7f538815c960 2024-04-17T10:51:05.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.988+0000 7f538d78b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f538806de20 con 0x7f53880cfa10 2024-04-17T10:51:05.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.988+0000 7f53867fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f538815ba30 0x7f538814cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:05.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.988+0000 7f53867fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f538815ba30 0x7f538814cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60586/0 (socket says 172.21.15.83:60586) 2024-04-17T10:51:05.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.988+0000 7f53867fc640 1 -- 172.21.15.83:0/2649565750 learned_addr learned my addr 172.21.15.83:0/2649565750 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:05.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.988+0000 7f5386ffd640 1 --2- 172.21.15.83:0/2649565750 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53880cfa10 0x7f5388137b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:05.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.988+0000 7f53877fe640 1 --2- 172.21.15.83:0/2649565750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f538815c960 0x7f53880cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:05.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.988+0000 7f53867fc640 1 -- 172.21.15.83:0/2649565750 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53880cfa10 msgr2=0x7f5388137b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:05.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.988+0000 7f53867fc640 1 --2- 172.21.15.83:0/2649565750 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53880cfa10 0x7f5388137b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:05.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.988+0000 7f53867fc640 1 -- 172.21.15.83:0/2649565750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f538815c960 msgr2=0x7f53880cdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:05.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.988+0000 7f53867fc640 1 --2- 172.21.15.83:0/2649565750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f538815c960 0x7f53880cdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:05.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.988+0000 7f53867fc640 1 -- 172.21.15.83:0/2649565750 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5388156b10 con 0x7f538815ba30 2024-04-17T10:51:05.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.992+0000 7f53867fc640 1 --2- 172.21.15.83:0/2649565750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f538815ba30 0x7f538814cda0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f537c01cd30 tx=0x7f537c019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:05.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.992+0000 7f536f7fe640 1 -- 172.21.15.83:0/2649565750 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c0b9070 con 0x7f538815ba30 2024-04-17T10:51:05.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.992+0000 7f536f7fe640 1 -- 172.21.15.83:0/2649565750 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f537c0b4640 con 0x7f538815ba30 2024-04-17T10:51:05.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.992+0000 7f538d78b640 1 -- 172.21.15.83:0/2649565750 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5388156df0 con 0x7f538815ba30 2024-04-17T10:51:05.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.992+0000 7f536f7fe640 1 -- 172.21.15.83:0/2649565750 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c0b0040 con 0x7f538815ba30 2024-04-17T10:51:05.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.992+0000 7f538d78b640 1 -- 172.21.15.83:0/2649565750 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5388156750 con 0x7f538815ba30 2024-04-17T10:51:05.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.992+0000 7f538d78b640 1 -- 172.21.15.83:0/2649565750 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5388074f90 con 0x7f538815ba30 2024-04-17T10:51:05.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.992+0000 7f536f7fe640 1 -- 172.21.15.83:0/2649565750 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f537c0bc070 con 0x7f538815ba30 2024-04-17T10:51:05.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.992+0000 7f536f7fe640 1 --2- 172.21.15.83:0/2649565750 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5374066f60 0x7f5374069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:05.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.992+0000 7f536f7fe640 1 -- 172.21.15.83:0/2649565750 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f537c11e1b0 con 0x7f538815ba30 2024-04-17T10:51:05.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.992+0000 7f5386ffd640 1 --2- 172.21.15.83:0/2649565750 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5374066f60 0x7f5374069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:05.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.992+0000 7f5386ffd640 1 --2- 172.21.15.83:0/2649565750 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5374066f60 0x7f5374069420 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f5378006fd0 tx=0x7f5378008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:05.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:05.996+0000 7f536f7fe640 1 -- 172.21.15.83:0/2649565750 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f537c016110 con 0x7f538815ba30 2024-04-17T10:51:06.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:06.192+0000 7f538d78b640 1 -- 172.21.15.83:0/2649565750 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53880b2530 con 0x7f538815ba30 2024-04-17T10:51:06.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:06.192+0000 7f536f7fe640 1 -- 172.21.15.83:0/2649565750 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f537c0ef0c0 con 0x7f538815ba30 2024-04-17T10:51:06.184 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:51:06.185 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:51:06.185 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:51:06.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:06.192+0000 7f538d78b640 1 -- 172.21.15.83:0/2649565750 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5374066f60 msgr2=0x7f5374069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:06.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:06.192+0000 7f538d78b640 1 --2- 172.21.15.83:0/2649565750 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5374066f60 0x7f5374069420 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f5378006fd0 tx=0x7f5378008040 comp rx=0 tx=0).stop 2024-04-17T10:51:06.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:06.192+0000 7f538d78b640 1 -- 172.21.15.83:0/2649565750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f538815ba30 msgr2=0x7f538814cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:06.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:06.192+0000 7f538d78b640 1 --2- 172.21.15.83:0/2649565750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f538815ba30 0x7f538814cda0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f537c01cd30 tx=0x7f537c019230 comp rx=0 tx=0).stop 2024-04-17T10:51:06.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:06.192+0000 7f538d78b640 1 -- 172.21.15.83:0/2649565750 shutdown_connections 2024-04-17T10:51:06.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:06.192+0000 7f538d78b640 1 --2- 172.21.15.83:0/2649565750 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5374066f60 0x7f5374069420 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:06.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:06.192+0000 7f538d78b640 1 --2- 172.21.15.83:0/2649565750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f538815c960 0x7f53880cdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:06.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:06.192+0000 7f538d78b640 1 --2- 172.21.15.83:0/2649565750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f538815ba30 0x7f538814cda0 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:06.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:06.192+0000 7f538d78b640 1 --2- 172.21.15.83:0/2649565750 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53880cfa10 0x7f5388137b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:06.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:06.192+0000 7f538d78b640 1 -- 172.21.15.83:0/2649565750 >> 172.21.15.83:0/2649565750 conn(0x7f538806abe0 msgr2=0x7f538815e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:06.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:06.196+0000 7f538d78b640 1 -- 172.21.15.83:0/2649565750 shutdown_connections 2024-04-17T10:51:06.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:06.196+0000 7f538d78b640 1 -- 172.21.15.83:0/2649565750 wait complete. 2024-04-17T10:51:06.606 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:51:06.607 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:51:07.607 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:51:07.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.744+0000 7fbda9fa5640 1 Processor -- start 2024-04-17T10:51:07.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.744+0000 7fbda9fa5640 1 -- start start 2024-04-17T10:51:07.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.748+0000 7fbda9fa5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbda415f1d0 0x7fbda415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:07.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.748+0000 7fbda9fa5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbda40d22d0 0x7fbda40ae520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:07.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.748+0000 7fbda9fa5640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbda415f5b0 con 0x7fbda40b2de0 2024-04-17T10:51:07.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.748+0000 7fbda9fa5640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda40b2620 con 0x7fbda40d22d0 2024-04-17T10:51:07.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.748+0000 7fbda9fa5640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda40b2340 con 0x7fbda415f1d0 2024-04-17T10:51:07.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.748+0000 7fbda3fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbda40d22d0 0x7fbda40ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:07.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.748+0000 7fbda3fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbda40d22d0 0x7fbda40ae520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35342/0 (socket says 172.21.15.83:35342) 2024-04-17T10:51:07.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.748+0000 7fbda3fff640 1 -- 172.21.15.83:0/110290389 learned_addr learned my addr 172.21.15.83:0/110290389 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:07.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.748+0000 7fbda37fe640 1 --2- 172.21.15.83:0/110290389 >> v2:172.21.15.167:3300/0 conn(0x7fbda415f1d0 0x7fbda415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:07.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.748+0000 7fbda3fff640 1 -- 172.21.15.83:0/110290389 >> v2:172.21.15.167:3300/0 conn(0x7fbda415f1d0 msgr2=0x7fbda415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:07.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.748+0000 7fbda3fff640 1 --2- 172.21.15.83:0/110290389 >> v2:172.21.15.167:3300/0 conn(0x7fbda415f1d0 0x7fbda415f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:07.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.748+0000 7fbda3fff640 1 -- 172.21.15.83:0/110290389 >> v1:172.21.15.167:6789/0 conn(0x7fbda40b2de0 legacy=0x7fbda415fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:07.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.748+0000 7fbda3fff640 1 -- 172.21.15.83:0/110290389 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda40b1760 con 0x7fbda40d22d0 2024-04-17T10:51:07.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.748+0000 7fbda3fff640 1 --2- 172.21.15.83:0/110290389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda40d22d0 0x7fbda40ae520 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fbda4071730 tx=0x7fbd9c01c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf9905dc303e58f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:07.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.748+0000 7fbda2ffd640 1 -- 172.21.15.83:0/110290389 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c0a5840 con 0x7fbda40d22d0 2024-04-17T10:51:07.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.748+0000 7fbda2ffd640 1 -- 172.21.15.83:0/110290389 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9c0a59e0 con 0x7fbda40d22d0 2024-04-17T10:51:07.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.748+0000 7fbda2ffd640 1 -- 172.21.15.83:0/110290389 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c0a5db0 con 0x7fbda40d22d0 2024-04-17T10:51:07.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbda9fa5640 1 -- 172.21.15.83:0/110290389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda40d22d0 msgr2=0x7fbda40ae520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:07.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbda9fa5640 1 --2- 172.21.15.83:0/110290389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda40d22d0 0x7fbda40ae520 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fbda4071730 tx=0x7fbd9c01c980 comp rx=0 tx=0).stop 2024-04-17T10:51:07.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbda9fa5640 1 -- 172.21.15.83:0/110290389 shutdown_connections 2024-04-17T10:51:07.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbda9fa5640 1 --2- 172.21.15.83:0/110290389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda40d22d0 0x7fbda40ae520 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:07.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbda9fa5640 1 --2- 172.21.15.83:0/110290389 >> v2:172.21.15.167:3300/0 conn(0x7fbda415f1d0 0x7fbda415f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:07.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbda9fa5640 1 -- 172.21.15.83:0/110290389 >> 172.21.15.83:0/110290389 conn(0x7fbda40d17b0 msgr2=0x7fbda40d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:07.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbda9fa5640 1 -- 172.21.15.83:0/110290389 shutdown_connections 2024-04-17T10:51:07.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbda9fa5640 1 -- 172.21.15.83:0/110290389 wait complete. 2024-04-17T10:51:07.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbda9fa5640 1 Processor -- start 2024-04-17T10:51:07.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbda9fa5640 1 -- start start 2024-04-17T10:51:07.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbda9fa5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda40d22d0 0x7fbda414ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:07.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbda9fa5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbda415f1d0 0x7fbda415d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:07.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbda9fa5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda4150ae0 0x7fbda40c7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:07.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbda9fa5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda40af270 con 0x7fbda4150ae0 2024-04-17T10:51:07.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbda9fa5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda4160690 con 0x7fbda40d22d0 2024-04-17T10:51:07.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbda9fa5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda40af3f0 con 0x7fbda415f1d0 2024-04-17T10:51:07.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbda3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda4150ae0 0x7fbda40c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:07.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbda3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda4150ae0 0x7fbda40c7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35356/0 (socket says 172.21.15.83:35356) 2024-04-17T10:51:07.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbda3fff640 1 -- 172.21.15.83:0/343071750 learned_addr learned my addr 172.21.15.83:0/343071750 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:07.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbda37fe640 1 --2- 172.21.15.83:0/343071750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda40d22d0 0x7fbda414ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:07.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbd9affd640 1 --2- 172.21.15.83:0/343071750 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbda415f1d0 0x7fbda415d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:07.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbda3fff640 1 -- 172.21.15.83:0/343071750 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbda415f1d0 msgr2=0x7fbda415d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:07.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbda3fff640 1 --2- 172.21.15.83:0/343071750 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbda415f1d0 0x7fbda415d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:07.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbda3fff640 1 -- 172.21.15.83:0/343071750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda40d22d0 msgr2=0x7fbda414ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:07.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbda3fff640 1 --2- 172.21.15.83:0/343071750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda40d22d0 0x7fbda414ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:07.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbda3fff640 1 -- 172.21.15.83:0/343071750 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda415e200 con 0x7fbda4150ae0 2024-04-17T10:51:07.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbda37fe640 1 --2- 172.21.15.83:0/343071750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda40d22d0 0x7fbda414ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:51:07.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbd9affd640 1 --2- 172.21.15.83:0/343071750 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbda415f1d0 0x7fbda415d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:51:07.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbda3fff640 1 --2- 172.21.15.83:0/343071750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda4150ae0 0x7fbda40c7310 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0af040 tx=0x7fbd9c019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:07.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbda0ff9640 1 -- 172.21.15.83:0/343071750 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c0b8880 con 0x7fbda4150ae0 2024-04-17T10:51:07.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbda0ff9640 1 -- 172.21.15.83:0/343071750 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9c0be070 con 0x7fbda4150ae0 2024-04-17T10:51:07.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbda9fa5640 1 -- 172.21.15.83:0/343071750 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda415e4e0 con 0x7fbda4150ae0 2024-04-17T10:51:07.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbda0ff9640 1 -- 172.21.15.83:0/343071750 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c0b8ca0 con 0x7fbda4150ae0 2024-04-17T10:51:07.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.752+0000 7fbda9fa5640 1 -- 172.21.15.83:0/343071750 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda415dd60 con 0x7fbda4150ae0 2024-04-17T10:51:07.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.756+0000 7fbda9fa5640 1 -- 172.21.15.83:0/343071750 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda415e6b0 con 0x7fbda4150ae0 2024-04-17T10:51:07.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.756+0000 7fbda0ff9640 1 -- 172.21.15.83:0/343071750 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbd9c0c3460 con 0x7fbda4150ae0 2024-04-17T10:51:07.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.756+0000 7fbda0ff9640 1 --2- 172.21.15.83:0/343071750 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbd84066f90 0x7fbd84069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:07.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.756+0000 7fbda0ff9640 1 -- 172.21.15.83:0/343071750 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fbd9c120440 con 0x7fbda4150ae0 2024-04-17T10:51:07.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.756+0000 7fbda37fe640 1 --2- 172.21.15.83:0/343071750 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbd84066f90 0x7fbd84069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:07.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.756+0000 7fbda37fe640 1 --2- 172.21.15.83:0/343071750 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbd84066f90 0x7fbd84069450 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fbd94006fd0 tx=0x7fbd94008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:07.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.760+0000 7fbda0ff9640 1 -- 172.21.15.83:0/343071750 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbda415e6b0 con 0x7fbda4150ae0 2024-04-17T10:51:07.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.948+0000 7fbda9fa5640 1 -- 172.21.15.83:0/343071750 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbda4074f90 con 0x7fbda4150ae0 2024-04-17T10:51:07.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.948+0000 7fbda0ff9640 1 -- 172.21.15.83:0/343071750 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fbd9c0165a0 con 0x7fbda4150ae0 2024-04-17T10:51:07.941 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:51:07.941 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:51:07.942 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:51:07.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.952+0000 7fbda9fa5640 1 -- 172.21.15.83:0/343071750 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbd84066f90 msgr2=0x7fbd84069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:07.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.952+0000 7fbda9fa5640 1 --2- 172.21.15.83:0/343071750 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbd84066f90 0x7fbd84069450 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fbd94006fd0 tx=0x7fbd94008040 comp rx=0 tx=0).stop 2024-04-17T10:51:07.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.952+0000 7fbda9fa5640 1 -- 172.21.15.83:0/343071750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda4150ae0 msgr2=0x7fbda40c7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:07.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.952+0000 7fbda9fa5640 1 --2- 172.21.15.83:0/343071750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda4150ae0 0x7fbda40c7310 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0af040 tx=0x7fbd9c019720 comp rx=0 tx=0).stop 2024-04-17T10:51:07.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.952+0000 7fbda9fa5640 1 -- 172.21.15.83:0/343071750 shutdown_connections 2024-04-17T10:51:07.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.952+0000 7fbda9fa5640 1 --2- 172.21.15.83:0/343071750 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbd84066f90 0x7fbd84069450 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:07.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.952+0000 7fbda9fa5640 1 --2- 172.21.15.83:0/343071750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbda4150ae0 0x7fbda40c7310 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:07.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.952+0000 7fbda9fa5640 1 --2- 172.21.15.83:0/343071750 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbda415f1d0 0x7fbda415d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:07.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.952+0000 7fbda9fa5640 1 --2- 172.21.15.83:0/343071750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda40d22d0 0x7fbda414ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:07.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.952+0000 7fbda9fa5640 1 -- 172.21.15.83:0/343071750 >> 172.21.15.83:0/343071750 conn(0x7fbda40d17b0 msgr2=0x7fbda41513f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:07.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.952+0000 7fbda9fa5640 1 -- 172.21.15.83:0/343071750 shutdown_connections 2024-04-17T10:51:07.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:07.952+0000 7fbda9fa5640 1 -- 172.21.15.83:0/343071750 wait complete. 2024-04-17T10:51:08.361 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:51:08.361 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:51:08.947 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:51:08.949 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:51:09.362 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:51:09.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.500+0000 7fe4058e5640 1 Processor -- start 2024-04-17T10:51:09.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.500+0000 7fe4058e5640 1 -- start start 2024-04-17T10:51:09.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.504+0000 7fe4058e5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe40015ba30 0x7fe400072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:09.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.504+0000 7fe4058e5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4000cfbe0 0x7fe40006c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:09.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.504+0000 7fe4058e5640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe40006e320 con 0x7fe40015c960 2024-04-17T10:51:09.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.504+0000 7fe4058e5640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe400071ea0 con 0x7fe40015ba30 2024-04-17T10:51:09.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.504+0000 7fe4058e5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40006e040 con 0x7fe4000cfbe0 2024-04-17T10:51:09.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.504+0000 7fe3feffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe40015ba30 0x7fe400072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:09.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.504+0000 7fe3feffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe40015ba30 0x7fe400072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35362/0 (socket says 172.21.15.83:35362) 2024-04-17T10:51:09.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.504+0000 7fe3feffd640 1 -- 172.21.15.83:0/2197174493 learned_addr learned my addr 172.21.15.83:0/2197174493 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:09.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.504+0000 7fe3ff7fe640 1 --2- 172.21.15.83:0/2197174493 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4000cfbe0 0x7fe40006c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:09.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.504+0000 7fe3feffd640 1 -- 172.21.15.83:0/2197174493 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4000cfbe0 msgr2=0x7fe40006c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:09.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.504+0000 7fe3feffd640 1 --2- 172.21.15.83:0/2197174493 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4000cfbe0 0x7fe40006c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:09.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.504+0000 7fe3feffd640 1 -- 172.21.15.83:0/2197174493 >> v1:172.21.15.167:6789/0 conn(0x7fe40015c960 legacy=0x7fe4000cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:09.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.504+0000 7fe3feffd640 1 -- 172.21.15.83:0/2197174493 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe400071850 con 0x7fe40015ba30 2024-04-17T10:51:09.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.504+0000 7fe3feffd640 1 --2- 172.21.15.83:0/2197174493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe40015ba30 0x7fe400072330 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0009870 tx=0x7fe3f001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=472fa3dff4a24e5e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:09.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.504+0000 7fe3fe7fc640 1 -- 172.21.15.83:0/2197174493 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f00b0070 con 0x7fe40015ba30 2024-04-17T10:51:09.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.504+0000 7fe3fe7fc640 1 -- 172.21.15.83:0/2197174493 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f00ab600 con 0x7fe40015ba30 2024-04-17T10:51:09.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.504+0000 7fe3fe7fc640 1 -- 172.21.15.83:0/2197174493 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f00ab9d0 con 0x7fe40015ba30 2024-04-17T10:51:09.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.508+0000 7fe4058e5640 1 -- 172.21.15.83:0/2197174493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe40015ba30 msgr2=0x7fe400072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:09.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.508+0000 7fe4058e5640 1 --2- 172.21.15.83:0/2197174493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe40015ba30 0x7fe400072330 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0009870 tx=0x7fe3f001c490 comp rx=0 tx=0).stop 2024-04-17T10:51:09.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.508+0000 7fe4058e5640 1 -- 172.21.15.83:0/2197174493 shutdown_connections 2024-04-17T10:51:09.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.508+0000 7fe4058e5640 1 --2- 172.21.15.83:0/2197174493 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4000cfbe0 0x7fe40006c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:09.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.508+0000 7fe4058e5640 1 --2- 172.21.15.83:0/2197174493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe40015ba30 0x7fe400072330 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:09.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.508+0000 7fe4058e5640 1 -- 172.21.15.83:0/2197174493 >> 172.21.15.83:0/2197174493 conn(0x7fe40006abe0 msgr2=0x7fe40006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:09.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.508+0000 7fe4058e5640 1 -- 172.21.15.83:0/2197174493 shutdown_connections 2024-04-17T10:51:09.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.508+0000 7fe4058e5640 1 -- 172.21.15.83:0/2197174493 wait complete. 2024-04-17T10:51:09.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.508+0000 7fe4058e5640 1 Processor -- start 2024-04-17T10:51:09.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.508+0000 7fe4058e5640 1 -- start start 2024-04-17T10:51:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.508+0000 7fe4058e5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4000cfbe0 0x7fe400154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.508+0000 7fe4058e5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe40015ba30 0x7fe40014f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.508+0000 7fe4058e5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe40015c960 0x7fe400147c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.508+0000 7fe4058e5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40006de20 con 0x7fe40015c960 2024-04-17T10:51:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.508+0000 7fe4058e5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40006dca0 con 0x7fe4000cfbe0 2024-04-17T10:51:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.508+0000 7fe4058e5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe400168520 con 0x7fe40015ba30 2024-04-17T10:51:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.508+0000 7fe3ff7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe40015c960 0x7fe400147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.508+0000 7fe3ff7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe40015c960 0x7fe400147c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35376/0 (socket says 172.21.15.83:35376) 2024-04-17T10:51:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.508+0000 7fe3ff7fe640 1 -- 172.21.15.83:0/4053705949 learned_addr learned my addr 172.21.15.83:0/4053705949 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:09.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.508+0000 7fe3feffd640 1 --2- 172.21.15.83:0/4053705949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4000cfbe0 0x7fe400154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:09.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.508+0000 7fe3ff7fe640 1 -- 172.21.15.83:0/4053705949 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe40015ba30 msgr2=0x7fe40014f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:09.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.508+0000 7fe3ff7fe640 1 --2- 172.21.15.83:0/4053705949 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe40015ba30 0x7fe40014f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:09.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.508+0000 7fe3ff7fe640 1 -- 172.21.15.83:0/4053705949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4000cfbe0 msgr2=0x7fe400154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:09.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.508+0000 7fe3ff7fe640 1 --2- 172.21.15.83:0/4053705949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4000cfbe0 0x7fe400154c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:09.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.508+0000 7fe3ff7fe640 1 -- 172.21.15.83:0/4053705949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe400155f40 con 0x7fe40015c960 2024-04-17T10:51:09.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.508+0000 7fe3ff7fe640 1 --2- 172.21.15.83:0/4053705949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe40015c960 0x7fe400147c30 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800cc30 tx=0x7fe3f800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:09.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.508+0000 7fe3fcff9640 1 -- 172.21.15.83:0/4053705949 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f800ce40 con 0x7fe40015c960 2024-04-17T10:51:09.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.508+0000 7fe3fcff9640 1 -- 172.21.15.83:0/4053705949 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f80039a0 con 0x7fe40015c960 2024-04-17T10:51:09.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.508+0000 7fe3fcff9640 1 -- 172.21.15.83:0/4053705949 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8003d50 con 0x7fe40015c960 2024-04-17T10:51:09.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.508+0000 7fe4058e5640 1 -- 172.21.15.83:0/4053705949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe400155330 con 0x7fe40015c960 2024-04-17T10:51:09.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.508+0000 7fe4058e5640 1 -- 172.21.15.83:0/4053705949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4001558f0 con 0x7fe40015c960 2024-04-17T10:51:09.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.508+0000 7fe4058e5640 1 -- 172.21.15.83:0/4053705949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe400074f90 con 0x7fe40015c960 2024-04-17T10:51:09.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.512+0000 7fe3fcff9640 1 -- 172.21.15.83:0/4053705949 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe3f8003f10 con 0x7fe40015c960 2024-04-17T10:51:09.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.512+0000 7fe3fcff9640 1 --2- 172.21.15.83:0/4053705949 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe3e0066f90 0x7fe3e0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:09.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.512+0000 7fe3fcff9640 1 -- 172.21.15.83:0/4053705949 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fe3f80858e0 con 0x7fe40015c960 2024-04-17T10:51:09.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.512+0000 7fe3feffd640 1 --2- 172.21.15.83:0/4053705949 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe3e0066f90 0x7fe3e0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:09.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.512+0000 7fe3feffd640 1 --2- 172.21.15.83:0/4053705949 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe3e0066f90 0x7fe3e0069450 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe3f00ad000 tx=0x7fe3f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:09.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.516+0000 7fe3fcff9640 1 -- 172.21.15.83:0/4053705949 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe3f8014030 con 0x7fe40015c960 2024-04-17T10:51:09.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.692+0000 7fe4058e5640 1 -- 172.21.15.83:0/4053705949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4001561f0 con 0x7fe40015c960 2024-04-17T10:51:09.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.692+0000 7fe3fcff9640 1 -- 172.21.15.83:0/4053705949 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fe3f8056900 con 0x7fe40015c960 2024-04-17T10:51:09.685 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:51:09.685 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:51:09.685 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:51:09.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.692+0000 7fe4058e5640 1 -- 172.21.15.83:0/4053705949 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe3e0066f90 msgr2=0x7fe3e0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:09.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.692+0000 7fe4058e5640 1 --2- 172.21.15.83:0/4053705949 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe3e0066f90 0x7fe3e0069450 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe3f00ad000 tx=0x7fe3f0008040 comp rx=0 tx=0).stop 2024-04-17T10:51:09.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.692+0000 7fe4058e5640 1 -- 172.21.15.83:0/4053705949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe40015c960 msgr2=0x7fe400147c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:09.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.692+0000 7fe4058e5640 1 --2- 172.21.15.83:0/4053705949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe40015c960 0x7fe400147c30 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800cc30 tx=0x7fe3f800a5c0 comp rx=0 tx=0).stop 2024-04-17T10:51:09.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.696+0000 7fe4058e5640 1 -- 172.21.15.83:0/4053705949 shutdown_connections 2024-04-17T10:51:09.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.696+0000 7fe4058e5640 1 --2- 172.21.15.83:0/4053705949 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe3e0066f90 0x7fe3e0069450 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:09.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.696+0000 7fe4058e5640 1 --2- 172.21.15.83:0/4053705949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe40015c960 0x7fe400147c30 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:09.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.696+0000 7fe4058e5640 1 --2- 172.21.15.83:0/4053705949 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe40015ba30 0x7fe40014f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:09.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.696+0000 7fe4058e5640 1 --2- 172.21.15.83:0/4053705949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4000cfbe0 0x7fe400154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:09.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.696+0000 7fe4058e5640 1 -- 172.21.15.83:0/4053705949 >> 172.21.15.83:0/4053705949 conn(0x7fe40006abe0 msgr2=0x7fe40015d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:09.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.696+0000 7fe4058e5640 1 -- 172.21.15.83:0/4053705949 shutdown_connections 2024-04-17T10:51:09.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:09.696+0000 7fe4058e5640 1 -- 172.21.15.83:0/4053705949 wait complete. 2024-04-17T10:51:10.105 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:51:10.105 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:51:11.106 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:51:11.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c46be5640 1 Processor -- start 2024-04-17T10:51:11.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c46be5640 1 -- start start 2024-04-17T10:51:11.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c46be5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c4015db50 0x7f1c40070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:11.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c46be5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1c4006dcb0 0x7f1c400b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:11.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c46be5640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c400709b0 con 0x7f1c400aedd0 2024-04-17T10:51:11.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c46be5640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c4015e440 con 0x7f1c4006dcb0 2024-04-17T10:51:11.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c46be5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4015e720 con 0x7f1c4015db50 2024-04-17T10:51:11.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c3ffff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1c400aedd0 0x7f1c4006d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46104/0 (socket says 172.21.15.83:46104) 2024-04-17T10:51:11.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c3ffff640 1 -- 172.21.15.83:0/3731336514 learned_addr learned my addr 172.21.15.83:0/3731336514 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:51:11.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c45182640 1 --2- 172.21.15.83:0/3731336514 >> v2:172.21.15.167:3300/0 conn(0x7f1c4006dcb0 0x7f1c400b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:11.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c3f7fe640 1 -- 172.21.15.83:0/3731336514 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 492598135 0 0) 0x7f1c2c001120 con 0x7f1c400aedd0 2024-04-17T10:51:11.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c44981640 1 --2- 172.21.15.83:0/3731336514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c4015db50 0x7f1c40070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:11.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c3f7fe640 1 -- 172.21.15.83:0/3731336514 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c30003530 con 0x7f1c400aedd0 2024-04-17T10:51:11.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c3f7fe640 1 -- 172.21.15.83:0/3731336514 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 649423275 0 0) 0x7f1c30003530 con 0x7f1c400aedd0 2024-04-17T10:51:11.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c3f7fe640 1 -- 172.21.15.83:0/3731336514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c4015db50 msgr2=0x7f1c40070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:11.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c3f7fe640 1 --2- 172.21.15.83:0/3731336514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c4015db50 0x7f1c40070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:11.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c3f7fe640 1 -- 172.21.15.83:0/3731336514 >> v2:172.21.15.167:3300/0 conn(0x7f1c4006dcb0 msgr2=0x7f1c400b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:11.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c3f7fe640 1 --2- 172.21.15.83:0/3731336514 >> v2:172.21.15.167:3300/0 conn(0x7f1c4006dcb0 0x7f1c400b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:11.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c3f7fe640 1 -- 172.21.15.83:0/3731336514 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4015ea00 con 0x7f1c400aedd0 2024-04-17T10:51:11.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c3f7fe640 1 -- 172.21.15.83:0/3731336514 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f1c2c001850 con 0x7f1c400aedd0 2024-04-17T10:51:11.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c3f7fe640 1 -- 172.21.15.83:0/3731336514 >> v1:172.21.15.83:6789/0 conn(0x7f1c400aedd0 legacy=0x7f1c4006d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:11.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c3f7fe640 1 --2- 172.21.15.83:0/3731336514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c300039c0 0x7f1c30003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:11.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c3f7fe640 1 --2- 172.21.15.83:0/3731336514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c4015db50 0x7f1c30004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:11.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c3f7fe640 1 --2- 172.21.15.83:0/3731336514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c30004d50 0x7f1c300051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:11.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c3f7fe640 1 -- 172.21.15.83:0/3731336514 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4015e440 con 0x7f1c300039c0 2024-04-17T10:51:11.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c3f7fe640 1 -- 172.21.15.83:0/3731336514 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4015e720 con 0x7f1c30004d50 2024-04-17T10:51:11.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c3f7fe640 1 -- 172.21.15.83:0/3731336514 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3000dbe0 con 0x7f1c4015db50 2024-04-17T10:51:11.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c44981640 1 --2- 172.21.15.83:0/3731336514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c300039c0 0x7f1c30003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:11.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c44981640 1 -- 172.21.15.83:0/3731336514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c4015db50 msgr2=0x7f1c30004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:11.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c44981640 1 --2- 172.21.15.83:0/3731336514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c4015db50 0x7f1c30004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:11.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c44981640 1 -- 172.21.15.83:0/3731336514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c30004d50 msgr2=0x7f1c300051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:11.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c44981640 1 --2- 172.21.15.83:0/3731336514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c30004d50 0x7f1c300051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:11.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.244+0000 7f1c44981640 1 -- 172.21.15.83:0/3731336514 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c30003530 con 0x7f1c300039c0 2024-04-17T10:51:11.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.248+0000 7f1c44981640 1 --2- 172.21.15.83:0/3731336514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c300039c0 0x7f1c30003de0 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f1c34009ae0 tx=0x7f1c3401ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4563cc10207bd15 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:11.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.248+0000 7f1c3f7fe640 1 -- 172.21.15.83:0/3731336514 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c340b2070 con 0x7f1c300039c0 2024-04-17T10:51:11.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.248+0000 7f1c3f7fe640 1 -- 172.21.15.83:0/3731336514 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c340a5560 con 0x7f1c300039c0 2024-04-17T10:51:11.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.248+0000 7f1c3f7fe640 1 -- 172.21.15.83:0/3731336514 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c340096b0 con 0x7f1c300039c0 2024-04-17T10:51:11.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.248+0000 7f1c46be5640 1 -- 172.21.15.83:0/3731336514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c300039c0 msgr2=0x7f1c30003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:11.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.248+0000 7f1c46be5640 1 --2- 172.21.15.83:0/3731336514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c300039c0 0x7f1c30003de0 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f1c34009ae0 tx=0x7f1c3401ca80 comp rx=0 tx=0).stop 2024-04-17T10:51:11.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.248+0000 7f1c45182640 1 -- 172.21.15.83:0/3731336514 reap_dead start 2024-04-17T10:51:11.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.248+0000 7f1c46be5640 1 -- 172.21.15.83:0/3731336514 shutdown_connections 2024-04-17T10:51:11.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.248+0000 7f1c46be5640 1 -- 172.21.15.83:0/3731336514 >> 172.21.15.83:0/3731336514 conn(0x7f1c4006abe0 msgr2=0x7f1c4006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:11.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.248+0000 7f1c46be5640 1 -- 172.21.15.83:0/3731336514 shutdown_connections 2024-04-17T10:51:11.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.248+0000 7f1c46be5640 1 -- 172.21.15.83:0/3731336514 wait complete. 2024-04-17T10:51:11.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.248+0000 7f1c46be5640 1 Processor -- start 2024-04-17T10:51:11.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.248+0000 7f1c46be5640 1 -- start start 2024-04-17T10:51:11.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.248+0000 7f1c46be5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c300039c0 0x7f1c4014f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:11.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.248+0000 7f1c46be5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c400abc40 0x7f1c4014c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:11.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.248+0000 7f1c46be5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c400ca820 0x7f1c4015c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:11.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.248+0000 7f1c46be5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c400b3ce0 con 0x7f1c400ca820 2024-04-17T10:51:11.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.248+0000 7f1c46be5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c400b3b60 con 0x7f1c300039c0 2024-04-17T10:51:11.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.248+0000 7f1c46be5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c400b3e60 con 0x7f1c400abc40 2024-04-17T10:51:11.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.248+0000 7f1c45182640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c400ca820 0x7f1c4015c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:11.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.248+0000 7f1c45182640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c400ca820 0x7f1c4015c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35392/0 (socket says 172.21.15.83:35392) 2024-04-17T10:51:11.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.248+0000 7f1c45182640 1 -- 172.21.15.83:0/3842657761 learned_addr learned my addr 172.21.15.83:0/3842657761 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:11.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.248+0000 7f1c3ffff640 1 --2- 172.21.15.83:0/3842657761 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c400abc40 0x7f1c4014c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:11.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.252+0000 7f1c45182640 1 -- 172.21.15.83:0/3842657761 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c400abc40 msgr2=0x7f1c4014c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:11.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.252+0000 7f1c44981640 1 --2- 172.21.15.83:0/3842657761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c300039c0 0x7f1c4014f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:11.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.252+0000 7f1c45182640 1 --2- 172.21.15.83:0/3842657761 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c400abc40 0x7f1c4014c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:11.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.252+0000 7f1c45182640 1 -- 172.21.15.83:0/3842657761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c300039c0 msgr2=0x7f1c4014f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:11.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.252+0000 7f1c45182640 1 --2- 172.21.15.83:0/3842657761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c300039c0 0x7f1c4014f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:11.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.252+0000 7f1c45182640 1 -- 172.21.15.83:0/3842657761 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c400ab860 con 0x7f1c400ca820 2024-04-17T10:51:11.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.252+0000 7f1c45182640 1 --2- 172.21.15.83:0/3842657761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c400ca820 0x7f1c4015c420 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f1c4006e960 tx=0x7f1c3800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:11.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.252+0000 7f1c3d7fa640 1 -- 172.21.15.83:0/3842657761 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c38018020 con 0x7f1c400ca820 2024-04-17T10:51:11.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.252+0000 7f1c3d7fa640 1 -- 172.21.15.83:0/3842657761 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3800bae0 con 0x7f1c400ca820 2024-04-17T10:51:11.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.252+0000 7f1c46be5640 1 -- 172.21.15.83:0/3842657761 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c400ac0c0 con 0x7f1c400ca820 2024-04-17T10:51:11.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.252+0000 7f1c3d7fa640 1 -- 172.21.15.83:0/3842657761 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c38014400 con 0x7f1c400ca820 2024-04-17T10:51:11.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.252+0000 7f1c46be5640 1 -- 172.21.15.83:0/3842657761 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c4014f7f0 con 0x7f1c400ca820 2024-04-17T10:51:11.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.252+0000 7f1c46be5640 1 -- 172.21.15.83:0/3842657761 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c40074f90 con 0x7f1c400ca820 2024-04-17T10:51:11.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.252+0000 7f1c3d7fa640 1 -- 172.21.15.83:0/3842657761 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1c3801b020 con 0x7f1c400ca820 2024-04-17T10:51:11.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.252+0000 7f1c3d7fa640 1 --2- 172.21.15.83:0/3842657761 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1c24066f90 0x7f1c24069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:11.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.252+0000 7f1c3d7fa640 1 -- 172.21.15.83:0/3842657761 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f1c38085540 con 0x7f1c400ca820 2024-04-17T10:51:11.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.252+0000 7f1c44981640 1 --2- 172.21.15.83:0/3842657761 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1c24066f90 0x7f1c24069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:11.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.252+0000 7f1c44981640 1 --2- 172.21.15.83:0/3842657761 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1c24066f90 0x7f1c24069450 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f1c340af000 tx=0x7f1c340046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:11.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.256+0000 7f1c3d7fa640 1 -- 172.21.15.83:0/3842657761 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1c38022030 con 0x7f1c400ca820 2024-04-17T10:51:11.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.444+0000 7f1c46be5640 1 -- 172.21.15.83:0/3842657761 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c400aea60 con 0x7f1c400ca820 2024-04-17T10:51:11.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.448+0000 7f1c3d7fa640 1 -- 172.21.15.83:0/3842657761 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f1c38083e20 con 0x7f1c400ca820 2024-04-17T10:51:11.439 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:51:11.439 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:51:11.439 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:51:11.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.448+0000 7f1c46be5640 1 -- 172.21.15.83:0/3842657761 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1c24066f90 msgr2=0x7f1c24069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:11.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.448+0000 7f1c46be5640 1 --2- 172.21.15.83:0/3842657761 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1c24066f90 0x7f1c24069450 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f1c340af000 tx=0x7f1c340046d0 comp rx=0 tx=0).stop 2024-04-17T10:51:11.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.448+0000 7f1c46be5640 1 -- 172.21.15.83:0/3842657761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c400ca820 msgr2=0x7f1c4015c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:11.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.448+0000 7f1c46be5640 1 --2- 172.21.15.83:0/3842657761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c400ca820 0x7f1c4015c420 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f1c4006e960 tx=0x7f1c3800b480 comp rx=0 tx=0).stop 2024-04-17T10:51:11.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.448+0000 7f1c46be5640 1 -- 172.21.15.83:0/3842657761 shutdown_connections 2024-04-17T10:51:11.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.448+0000 7f1c46be5640 1 --2- 172.21.15.83:0/3842657761 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1c24066f90 0x7f1c24069450 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:11.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.448+0000 7f1c46be5640 1 --2- 172.21.15.83:0/3842657761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c400ca820 0x7f1c4015c420 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:11.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.448+0000 7f1c46be5640 1 --2- 172.21.15.83:0/3842657761 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c400abc40 0x7f1c4014c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:11.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.448+0000 7f1c46be5640 1 --2- 172.21.15.83:0/3842657761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c300039c0 0x7f1c4014f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:11.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.448+0000 7f1c46be5640 1 -- 172.21.15.83:0/3842657761 >> 172.21.15.83:0/3842657761 conn(0x7f1c4006abe0 msgr2=0x7f1c400af510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:11.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.448+0000 7f1c46be5640 1 -- 172.21.15.83:0/3842657761 shutdown_connections 2024-04-17T10:51:11.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:11.448+0000 7f1c46be5640 1 -- 172.21.15.83:0/3842657761 wait complete. 2024-04-17T10:51:11.859 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:51:11.859 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:51:12.860 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:51:12.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.000+0000 7f5aef582640 1 Processor -- start 2024-04-17T10:51:12.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.000+0000 7f5aef582640 1 -- start start 2024-04-17T10:51:12.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.000+0000 7f5aef582640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5ae8146a00 0x7f5ae8146e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:12.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.000+0000 7f5aef582640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5ae806d690 0x7f5ae81620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:12.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.000+0000 7f5aef582640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ae80b2670 0x7f5ae80b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:12.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.000+0000 7f5aef582640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ae8162610 con 0x7f5ae806d690 2024-04-17T10:51:12.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.000+0000 7f5aef582640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ae806c960 con 0x7f5ae8146a00 2024-04-17T10:51:12.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.000+0000 7f5aef582640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae806d200 con 0x7f5ae80b2670 2024-04-17T10:51:12.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.000+0000 7f5aecb1d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5ae806d690 0x7f5ae81620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:12.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.000+0000 7f5aecb1d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5ae806d690 0x7f5ae81620d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35406/0 (socket says 172.21.15.83:35406) 2024-04-17T10:51:12.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.000+0000 7f5aecb1d640 1 -- 172.21.15.83:0/3035030973 learned_addr learned my addr 172.21.15.83:0/3035030973 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:12.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.000+0000 7f5aed31e640 1 --2- 172.21.15.83:0/3035030973 >> v2:172.21.15.167:3300/0 conn(0x7f5ae8146a00 0x7f5ae8146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:12.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.000+0000 7f5aedb1f640 1 --2- 172.21.15.83:0/3035030973 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ae80b2670 0x7f5ae80b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:12.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.000+0000 7f5aecb1d640 1 -- 172.21.15.83:0/3035030973 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ae80b2670 msgr2=0x7f5ae80b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:12.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.000+0000 7f5aecb1d640 1 --2- 172.21.15.83:0/3035030973 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ae80b2670 0x7f5ae80b2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:12.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.000+0000 7f5aecb1d640 1 -- 172.21.15.83:0/3035030973 >> v2:172.21.15.167:3300/0 conn(0x7f5ae8146a00 msgr2=0x7f5ae8146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:12.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.000+0000 7f5aecb1d640 1 --2- 172.21.15.83:0/3035030973 >> v2:172.21.15.167:3300/0 conn(0x7f5ae8146a00 0x7f5ae8146e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:12.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.000+0000 7f5aecb1d640 1 -- 172.21.15.83:0/3035030973 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae806daf0 con 0x7f5ae806d690 2024-04-17T10:51:12.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.000+0000 7f5aedb1f640 1 --2- 172.21.15.83:0/3035030973 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ae80b2670 0x7f5ae80b2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:51:12.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.004+0000 7f5aecb1d640 1 --2- 172.21.15.83:0/3035030973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ae806d690 0x7f5ae81620d0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f5ae806ffe0 tx=0x7f5ad801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e218296db846bcd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:12.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.004+0000 7f5adffff640 1 -- 172.21.15.83:0/3035030973 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad80ab5f0 con 0x7f5ae806d690 2024-04-17T10:51:12.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.004+0000 7f5adffff640 1 -- 172.21.15.83:0/3035030973 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad80b0070 con 0x7f5ae806d690 2024-04-17T10:51:12.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.004+0000 7f5adffff640 1 -- 172.21.15.83:0/3035030973 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad80ab9b0 con 0x7f5ae806d690 2024-04-17T10:51:12.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.004+0000 7f5aef582640 1 -- 172.21.15.83:0/3035030973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ae806d690 msgr2=0x7f5ae81620d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:12.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.004+0000 7f5aef582640 1 --2- 172.21.15.83:0/3035030973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ae806d690 0x7f5ae81620d0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f5ae806ffe0 tx=0x7f5ad801c520 comp rx=0 tx=0).stop 2024-04-17T10:51:12.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.004+0000 7f5aef582640 1 -- 172.21.15.83:0/3035030973 shutdown_connections 2024-04-17T10:51:12.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.004+0000 7f5aef582640 1 --2- 172.21.15.83:0/3035030973 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ae80b2670 0x7f5ae80b2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:12.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.004+0000 7f5aef582640 1 --2- 172.21.15.83:0/3035030973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ae806d690 0x7f5ae81620d0 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:12.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.004+0000 7f5aef582640 1 --2- 172.21.15.83:0/3035030973 >> v2:172.21.15.167:3300/0 conn(0x7f5ae8146a00 0x7f5ae8146e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:12.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.004+0000 7f5aef582640 1 -- 172.21.15.83:0/3035030973 >> 172.21.15.83:0/3035030973 conn(0x7f5ae806abe0 msgr2=0x7f5ae806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:12.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.004+0000 7f5aef582640 1 -- 172.21.15.83:0/3035030973 shutdown_connections 2024-04-17T10:51:12.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.004+0000 7f5aef582640 1 -- 172.21.15.83:0/3035030973 wait complete. 2024-04-17T10:51:12.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.004+0000 7f5aef582640 1 Processor -- start 2024-04-17T10:51:12.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.004+0000 7f5aef582640 1 -- start start 2024-04-17T10:51:13.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.004+0000 7f5aef582640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ae806d690 0x7f5ae8142750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:13.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.004+0000 7f5aef582640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ae80b2670 0x7f5ae81391c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:13.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.004+0000 7f5aef582640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ae8146a00 0x7f5ae8071280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:13.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.004+0000 7f5aef582640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae8070e00 con 0x7f5ae80b2670 2024-04-17T10:51:13.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.004+0000 7f5aef582640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae8070c80 con 0x7f5ae806d690 2024-04-17T10:51:13.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.004+0000 7f5aef582640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae8070f80 con 0x7f5ae8146a00 2024-04-17T10:51:13.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.004+0000 7f5aecb1d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ae80b2670 0x7f5ae81391c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:13.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.004+0000 7f5aecb1d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ae80b2670 0x7f5ae81391c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35418/0 (socket says 172.21.15.83:35418) 2024-04-17T10:51:13.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.004+0000 7f5aecb1d640 1 -- 172.21.15.83:0/436258960 learned_addr learned my addr 172.21.15.83:0/436258960 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:13.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.008+0000 7f5aed31e640 1 --2- 172.21.15.83:0/436258960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ae806d690 0x7f5ae8142750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:13.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.008+0000 7f5aedb1f640 1 --2- 172.21.15.83:0/436258960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ae8146a00 0x7f5ae8071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:13.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.008+0000 7f5aecb1d640 1 -- 172.21.15.83:0/436258960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ae8146a00 msgr2=0x7f5ae8071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:13.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.008+0000 7f5aecb1d640 1 --2- 172.21.15.83:0/436258960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ae8146a00 0x7f5ae8071280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:13.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.008+0000 7f5aecb1d640 1 -- 172.21.15.83:0/436258960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ae806d690 msgr2=0x7f5ae8142750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:13.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.008+0000 7f5aecb1d640 1 --2- 172.21.15.83:0/436258960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ae806d690 0x7f5ae8142750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:13.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.008+0000 7f5aecb1d640 1 -- 172.21.15.83:0/436258960 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae81f6b10 con 0x7f5ae80b2670 2024-04-17T10:51:13.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.008+0000 7f5aecb1d640 1 --2- 172.21.15.83:0/436258960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ae80b2670 0x7f5ae81391c0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f5ad801cdc0 tx=0x7f5ad801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:13.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.008+0000 7f5addffb640 1 -- 172.21.15.83:0/436258960 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad80be5f0 con 0x7f5ae80b2670 2024-04-17T10:51:13.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.008+0000 7f5addffb640 1 -- 172.21.15.83:0/436258960 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad80be7b0 con 0x7f5ae80b2670 2024-04-17T10:51:13.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.008+0000 7f5aef582640 1 -- 172.21.15.83:0/436258960 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae8140af0 con 0x7f5ae80b2670 2024-04-17T10:51:13.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.008+0000 7f5addffb640 1 -- 172.21.15.83:0/436258960 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad80beba0 con 0x7f5ae80b2670 2024-04-17T10:51:13.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.008+0000 7f5aef582640 1 -- 172.21.15.83:0/436258960 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae81410b0 con 0x7f5ae80b2670 2024-04-17T10:51:13.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.008+0000 7f5aef582640 1 -- 172.21.15.83:0/436258960 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ae8074f90 con 0x7f5ae80b2670 2024-04-17T10:51:13.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.008+0000 7f5addffb640 1 -- 172.21.15.83:0/436258960 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5ad80b43f0 con 0x7f5ae80b2670 2024-04-17T10:51:13.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.008+0000 7f5addffb640 1 --2- 172.21.15.83:0/436258960 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5acc066f60 0x7f5acc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:13.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.008+0000 7f5addffb640 1 -- 172.21.15.83:0/436258960 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f5ad811f1a0 con 0x7f5ae80b2670 2024-04-17T10:51:13.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.008+0000 7f5aed31e640 1 --2- 172.21.15.83:0/436258960 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5acc066f60 0x7f5acc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:13.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.008+0000 7f5aed31e640 1 --2- 172.21.15.83:0/436258960 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5acc066f60 0x7f5acc069420 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f5ae400b9d0 tx=0x7f5ae4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:13.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.012+0000 7f5addffb640 1 -- 172.21.15.83:0/436258960 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5ad8016110 con 0x7f5ae80b2670 2024-04-17T10:51:13.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.208+0000 7f5aef582640 1 -- 172.21.15.83:0/436258960 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ae813c400 con 0x7f5ae80b2670 2024-04-17T10:51:13.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.208+0000 7f5addffb640 1 -- 172.21.15.83:0/436258960 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f5ad80efa80 con 0x7f5ae80b2670 2024-04-17T10:51:13.201 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:51:13.201 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:51:13.202 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:51:13.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.212+0000 7f5aef582640 1 -- 172.21.15.83:0/436258960 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5acc066f60 msgr2=0x7f5acc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:13.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.212+0000 7f5aef582640 1 --2- 172.21.15.83:0/436258960 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5acc066f60 0x7f5acc069420 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f5ae400b9d0 tx=0x7f5ae4008040 comp rx=0 tx=0).stop 2024-04-17T10:51:13.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.212+0000 7f5aef582640 1 -- 172.21.15.83:0/436258960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ae80b2670 msgr2=0x7f5ae81391c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:13.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.212+0000 7f5aef582640 1 --2- 172.21.15.83:0/436258960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ae80b2670 0x7f5ae81391c0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f5ad801cdc0 tx=0x7f5ad801ad60 comp rx=0 tx=0).stop 2024-04-17T10:51:13.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.212+0000 7f5aef582640 1 -- 172.21.15.83:0/436258960 shutdown_connections 2024-04-17T10:51:13.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.212+0000 7f5aef582640 1 --2- 172.21.15.83:0/436258960 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5acc066f60 0x7f5acc069420 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:13.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.212+0000 7f5aef582640 1 --2- 172.21.15.83:0/436258960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ae8146a00 0x7f5ae8071280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:13.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.212+0000 7f5aef582640 1 --2- 172.21.15.83:0/436258960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ae80b2670 0x7f5ae81391c0 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:13.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.212+0000 7f5aef582640 1 --2- 172.21.15.83:0/436258960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ae806d690 0x7f5ae8142750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:13.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.212+0000 7f5aef582640 1 -- 172.21.15.83:0/436258960 >> 172.21.15.83:0/436258960 conn(0x7f5ae806abe0 msgr2=0x7f5ae813f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:13.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.212+0000 7f5aef582640 1 -- 172.21.15.83:0/436258960 shutdown_connections 2024-04-17T10:51:13.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:13.212+0000 7f5aef582640 1 -- 172.21.15.83:0/436258960 wait complete. 2024-04-17T10:51:13.623 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:51:13.623 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:51:14.624 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:51:14.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9eb14e4640 1 Processor -- start 2024-04-17T10:51:14.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9eb14e4640 1 -- start start 2024-04-17T10:51:14.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9eb14e4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9eac06fb70 0x7f9eac0713a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:14.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9eb14e4640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eac06baf0 con 0x7f9eac0ccbf0 2024-04-17T10:51:14.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9eb14e4640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eac071bf0 con 0x7f9eac15f550 2024-04-17T10:51:14.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9eb14e4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac071ed0 con 0x7f9eac06fb70 2024-04-17T10:51:14.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9ea3fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9eac0ccbf0 0x7f9eac06b3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46108/0 (socket says 172.21.15.83:46108) 2024-04-17T10:51:14.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9ea3fff640 1 -- 172.21.15.83:0/2556435153 learned_addr learned my addr 172.21.15.83:0/2556435153 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:51:14.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9eaaffd640 1 --2- 172.21.15.83:0/2556435153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9eac06fb70 0x7f9eac0713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:14.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9eaa7fc640 1 -- 172.21.15.83:0/2556435153 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4223959431 0 0) 0x7f9eac06baf0 con 0x7f9eac0ccbf0 2024-04-17T10:51:14.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9eaa7fc640 1 -- 172.21.15.83:0/2556435153 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e98003530 con 0x7f9eac0ccbf0 2024-04-17T10:51:14.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9eaa7fc640 1 -- 172.21.15.83:0/2556435153 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 286760109 0 0) 0x7f9e98003530 con 0x7f9eac0ccbf0 2024-04-17T10:51:14.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9eaa7fc640 1 -- 172.21.15.83:0/2556435153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9eac06fb70 msgr2=0x7f9eac0713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:14.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9eaa7fc640 1 --2- 172.21.15.83:0/2556435153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9eac06fb70 0x7f9eac0713a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:14.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9eaa7fc640 1 -- 172.21.15.83:0/2556435153 >> v1:172.21.15.167:6789/0 conn(0x7f9eac15f550 legacy=0x7f9eac0c5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:14.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9eaa7fc640 1 -- 172.21.15.83:0/2556435153 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eac0721b0 con 0x7f9eac0ccbf0 2024-04-17T10:51:14.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9eaa7fc640 1 -- 172.21.15.83:0/2556435153 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f9e9c001680 con 0x7f9eac0ccbf0 2024-04-17T10:51:14.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9eaa7fc640 1 -- 172.21.15.83:0/2556435153 >> v1:172.21.15.83:6789/0 conn(0x7f9eac0ccbf0 legacy=0x7f9eac06b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:14.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9eaa7fc640 1 --2- 172.21.15.83:0/2556435153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e980039f0 0x7f9e98003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:14.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9eaa7fc640 1 --2- 172.21.15.83:0/2556435153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9eac06fb70 0x7f9e98004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:14.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9eaa7fc640 1 --2- 172.21.15.83:0/2556435153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e98004d50 0x7f9e980051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:14.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9eaa7fc640 1 -- 172.21.15.83:0/2556435153 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac071ed0 con 0x7f9e98004d50 2024-04-17T10:51:14.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9eaa7fc640 1 -- 172.21.15.83:0/2556435153 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9800bb60 con 0x7f9e980039f0 2024-04-17T10:51:14.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9eaa7fc640 1 -- 172.21.15.83:0/2556435153 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9800bd30 con 0x7f9eac06fb70 2024-04-17T10:51:14.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9eab7fe640 1 --2- 172.21.15.83:0/2556435153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e98004d50 0x7f9e980051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:14.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9eaaffd640 1 --2- 172.21.15.83:0/2556435153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e980039f0 0x7f9e98003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:14.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9ea3fff640 1 --2- 172.21.15.83:0/2556435153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9eac06fb70 0x7f9e98004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:14.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9eab7fe640 1 -- 172.21.15.83:0/2556435153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9eac06fb70 msgr2=0x7f9e98004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:14.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9eab7fe640 1 --2- 172.21.15.83:0/2556435153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9eac06fb70 0x7f9e98004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:14.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9eab7fe640 1 -- 172.21.15.83:0/2556435153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e980039f0 msgr2=0x7f9e98003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:14.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9eab7fe640 1 --2- 172.21.15.83:0/2556435153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e980039f0 0x7f9e98003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:14.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9eab7fe640 1 -- 172.21.15.83:0/2556435153 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e98003530 con 0x7f9e98004d50 2024-04-17T10:51:14.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.768+0000 7f9eaaffd640 1 --2- 172.21.15.83:0/2556435153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e980039f0 0x7f9e98003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:51:14.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.772+0000 7f9eab7fe640 1 --2- 172.21.15.83:0/2556435153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e98004d50 0x7f9e980051d0 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4001ea0 tx=0x7f9ea401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56cf52652f19633 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:14.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.772+0000 7f9eaa7fc640 1 -- 172.21.15.83:0/2556435153 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4001a90 con 0x7f9e98004d50 2024-04-17T10:51:14.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.772+0000 7f9eaa7fc640 1 -- 172.21.15.83:0/2556435153 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea4003b80 con 0x7f9e98004d50 2024-04-17T10:51:14.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.772+0000 7f9eaa7fc640 1 -- 172.21.15.83:0/2556435153 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4005080 con 0x7f9e98004d50 2024-04-17T10:51:14.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.772+0000 7f9eb14e4640 1 -- 172.21.15.83:0/2556435153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e98004d50 msgr2=0x7f9e980051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:14.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.772+0000 7f9eb14e4640 1 --2- 172.21.15.83:0/2556435153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e98004d50 0x7f9e980051d0 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4001ea0 tx=0x7f9ea401c780 comp rx=0 tx=0).stop 2024-04-17T10:51:14.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.772+0000 7f9eab7fe640 1 -- 172.21.15.83:0/2556435153 reap_dead start 2024-04-17T10:51:14.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.772+0000 7f9eb14e4640 1 -- 172.21.15.83:0/2556435153 shutdown_connections 2024-04-17T10:51:14.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.772+0000 7f9eb14e4640 1 -- 172.21.15.83:0/2556435153 >> 172.21.15.83:0/2556435153 conn(0x7f9eac06bcf0 msgr2=0x7f9eac06c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:14.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.772+0000 7f9eb14e4640 1 -- 172.21.15.83:0/2556435153 shutdown_connections 2024-04-17T10:51:14.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.772+0000 7f9eb14e4640 1 -- 172.21.15.83:0/2556435153 wait complete. 2024-04-17T10:51:14.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.772+0000 7f9eb14e4640 1 Processor -- start 2024-04-17T10:51:14.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.772+0000 7f9eb14e4640 1 -- start start 2024-04-17T10:51:14.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.776+0000 7f9eb14e4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e98004d50 0x7f9eac06e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:14.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.776+0000 7f9eb14e4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9eac0ca200 0x7f9eac14de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:14.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.776+0000 7f9eb14e4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9eac14e3d0 0x7f9eac1f49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:14.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.776+0000 7f9eb14e4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac0c5c60 con 0x7f9e98004d50 2024-04-17T10:51:14.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.776+0000 7f9eb14e4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac06aec0 con 0x7f9eac14e3d0 2024-04-17T10:51:14.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.776+0000 7f9eb14e4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac0c5de0 con 0x7f9eac0ca200 2024-04-17T10:51:14.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.776+0000 7f9eaaffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e98004d50 0x7f9eac06e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:14.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.776+0000 7f9eaaffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e98004d50 0x7f9eac06e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35442/0 (socket says 172.21.15.83:35442) 2024-04-17T10:51:14.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.776+0000 7f9eaaffd640 1 -- 172.21.15.83:0/4026101386 learned_addr learned my addr 172.21.15.83:0/4026101386 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:14.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.776+0000 7f9eaaffd640 1 -- 172.21.15.83:0/4026101386 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9eac0ca200 msgr2=0x7f9eac14de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:14.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.776+0000 7f9eaaffd640 1 --2- 172.21.15.83:0/4026101386 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9eac0ca200 0x7f9eac14de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:14.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.776+0000 7f9eaaffd640 1 -- 172.21.15.83:0/4026101386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9eac14e3d0 msgr2=0x7f9eac1f49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:14.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.776+0000 7f9eab7fe640 1 --2- 172.21.15.83:0/4026101386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9eac14e3d0 0x7f9eac1f49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:14.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.776+0000 7f9eaaffd640 1 --2- 172.21.15.83:0/4026101386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9eac14e3d0 0x7f9eac1f49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:14.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.776+0000 7f9eaaffd640 1 -- 172.21.15.83:0/4026101386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eac1f56f0 con 0x7f9e98004d50 2024-04-17T10:51:14.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.776+0000 7f9eaaffd640 1 --2- 172.21.15.83:0/4026101386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e98004d50 0x7f9eac06e970 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f9e9400f510 tx=0x7f9e9400f9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:14.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.776+0000 7f9ea37fe640 1 -- 172.21.15.83:0/4026101386 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e94019020 con 0x7f9e98004d50 2024-04-17T10:51:14.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.776+0000 7f9ea37fe640 1 -- 172.21.15.83:0/4026101386 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e94014870 con 0x7f9e98004d50 2024-04-17T10:51:14.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.776+0000 7f9eb14e4640 1 -- 172.21.15.83:0/4026101386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eac0c9720 con 0x7f9e98004d50 2024-04-17T10:51:14.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.776+0000 7f9ea37fe640 1 -- 172.21.15.83:0/4026101386 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e94014c40 con 0x7f9e98004d50 2024-04-17T10:51:14.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.776+0000 7f9eb14e4640 1 -- 172.21.15.83:0/4026101386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eac0c9ce0 con 0x7f9e98004d50 2024-04-17T10:51:14.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.776+0000 7f9eb14e4640 1 -- 172.21.15.83:0/4026101386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eac074f90 con 0x7f9e98004d50 2024-04-17T10:51:14.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.776+0000 7f9ea37fe640 1 -- 172.21.15.83:0/4026101386 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9e9401c020 con 0x7f9e98004d50 2024-04-17T10:51:14.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.776+0000 7f9ea37fe640 1 --2- 172.21.15.83:0/4026101386 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9e8c066f60 0x7f9e8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:14.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.776+0000 7f9ea3fff640 1 --2- 172.21.15.83:0/4026101386 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9e8c066f60 0x7f9e8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:14.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.776+0000 7f9ea37fe640 1 -- 172.21.15.83:0/4026101386 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f9e94085830 con 0x7f9e98004d50 2024-04-17T10:51:14.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.776+0000 7f9ea3fff640 1 --2- 172.21.15.83:0/4026101386 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9e8c066f60 0x7f9e8c069420 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c001ec0 tx=0x7f9e9c001d90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:14.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.780+0000 7f9ea37fe640 1 -- 172.21.15.83:0/4026101386 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9e94023020 con 0x7f9e98004d50 2024-04-17T10:51:14.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.972+0000 7f9eb14e4640 1 -- 172.21.15.83:0/4026101386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9eac06c9c0 con 0x7f9e98004d50 2024-04-17T10:51:14.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.976+0000 7f9ea37fe640 1 -- 172.21.15.83:0/4026101386 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f9e94026350 con 0x7f9e98004d50 2024-04-17T10:51:14.968 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:51:14.968 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:51:14.968 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:51:14.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.976+0000 7f9eb14e4640 1 -- 172.21.15.83:0/4026101386 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9e8c066f60 msgr2=0x7f9e8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:14.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.976+0000 7f9eb14e4640 1 --2- 172.21.15.83:0/4026101386 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9e8c066f60 0x7f9e8c069420 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c001ec0 tx=0x7f9e9c001d90 comp rx=0 tx=0).stop 2024-04-17T10:51:14.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.976+0000 7f9eb14e4640 1 -- 172.21.15.83:0/4026101386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e98004d50 msgr2=0x7f9eac06e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:14.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.976+0000 7f9eb14e4640 1 --2- 172.21.15.83:0/4026101386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e98004d50 0x7f9eac06e970 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f9e9400f510 tx=0x7f9e9400f9d0 comp rx=0 tx=0).stop 2024-04-17T10:51:14.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.976+0000 7f9eb14e4640 1 -- 172.21.15.83:0/4026101386 shutdown_connections 2024-04-17T10:51:14.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.976+0000 7f9eb14e4640 1 --2- 172.21.15.83:0/4026101386 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9e8c066f60 0x7f9e8c069420 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:14.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.976+0000 7f9eb14e4640 1 --2- 172.21.15.83:0/4026101386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9eac14e3d0 0x7f9eac1f49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:14.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.976+0000 7f9eb14e4640 1 --2- 172.21.15.83:0/4026101386 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9eac0ca200 0x7f9eac14de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:14.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.976+0000 7f9eb14e4640 1 --2- 172.21.15.83:0/4026101386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e98004d50 0x7f9eac06e970 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:14.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.976+0000 7f9eb14e4640 1 -- 172.21.15.83:0/4026101386 >> 172.21.15.83:0/4026101386 conn(0x7f9eac06bcf0 msgr2=0x7f9eac15e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:14.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.976+0000 7f9eb14e4640 1 -- 172.21.15.83:0/4026101386 shutdown_connections 2024-04-17T10:51:14.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:14.976+0000 7f9eb14e4640 1 -- 172.21.15.83:0/4026101386 wait complete. 2024-04-17T10:51:15.388 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:51:15.388 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:51:16.389 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:51:16.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.528+0000 7fc3e64e4640 1 Processor -- start 2024-04-17T10:51:16.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.528+0000 7fc3e64e4640 1 -- start start 2024-04-17T10:51:16.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.528+0000 7fc3e64e4640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc3e0072330 0x7fc3e00cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:16.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.528+0000 7fc3e64e4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc3e00cfa10 0x7fc3e006c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:16.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.528+0000 7fc3e64e4640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3e00cfea0 con 0x7fc3e015ba30 2024-04-17T10:51:16.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.528+0000 7fc3e64e4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3e0071ea0 con 0x7fc3e00cfa10 2024-04-17T10:51:16.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.528+0000 7fc3e64e4640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3e006e320 con 0x7fc3e0072330 2024-04-17T10:51:16.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.528+0000 7fc3e4a81640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc3e00cfa10 0x7fc3e006c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:16.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.528+0000 7fc3e4a81640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc3e00cfa10 0x7fc3e006c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35446/0 (socket says 172.21.15.83:35446) 2024-04-17T10:51:16.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.528+0000 7fc3e4a81640 1 -- 172.21.15.83:0/4260737556 learned_addr learned my addr 172.21.15.83:0/4260737556 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:16.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.528+0000 7fc3df7fe640 1 --2- 172.21.15.83:0/4260737556 >> v2:172.21.15.167:3300/0 conn(0x7fc3e0072330 0x7fc3e00cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:16.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.528+0000 7fc3e4a81640 1 -- 172.21.15.83:0/4260737556 >> v2:172.21.15.167:3300/0 conn(0x7fc3e0072330 msgr2=0x7fc3e00cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:16.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.528+0000 7fc3e4a81640 1 --2- 172.21.15.83:0/4260737556 >> v2:172.21.15.167:3300/0 conn(0x7fc3e0072330 0x7fc3e00cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:16.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.528+0000 7fc3e4a81640 1 -- 172.21.15.83:0/4260737556 >> v1:172.21.15.167:6789/0 conn(0x7fc3e015ba30 legacy=0x7fc3e015c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:16.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.528+0000 7fc3e4a81640 1 -- 172.21.15.83:0/4260737556 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e006e040 con 0x7fc3e00cfa10 2024-04-17T10:51:16.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.528+0000 7fc3df7fe640 1 --2- 172.21.15.83:0/4260737556 >> v2:172.21.15.167:3300/0 conn(0x7fc3e0072330 0x7fc3e00cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:51:16.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.532+0000 7fc3e4a81640 1 --2- 172.21.15.83:0/4260737556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e00cfa10 0x7fc3e006c490 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7fc3e00af2b0 tx=0x7fc3d801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc96c3dca9d18385 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:16.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.532+0000 7fc3deffd640 1 -- 172.21.15.83:0/4260737556 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d80a56f0 con 0x7fc3e00cfa10 2024-04-17T10:51:16.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.532+0000 7fc3deffd640 1 -- 172.21.15.83:0/4260737556 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d80a5890 con 0x7fc3e00cfa10 2024-04-17T10:51:16.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.532+0000 7fc3deffd640 1 -- 172.21.15.83:0/4260737556 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d80a5c60 con 0x7fc3e00cfa10 2024-04-17T10:51:16.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.532+0000 7fc3e64e4640 1 -- 172.21.15.83:0/4260737556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e00cfa10 msgr2=0x7fc3e006c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:16.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.532+0000 7fc3e64e4640 1 --2- 172.21.15.83:0/4260737556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e00cfa10 0x7fc3e006c490 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7fc3e00af2b0 tx=0x7fc3d801c920 comp rx=0 tx=0).stop 2024-04-17T10:51:16.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.532+0000 7fc3e64e4640 1 -- 172.21.15.83:0/4260737556 shutdown_connections 2024-04-17T10:51:16.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.532+0000 7fc3e64e4640 1 --2- 172.21.15.83:0/4260737556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e00cfa10 0x7fc3e006c490 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:16.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.532+0000 7fc3e64e4640 1 --2- 172.21.15.83:0/4260737556 >> v2:172.21.15.167:3300/0 conn(0x7fc3e0072330 0x7fc3e00cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:16.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.532+0000 7fc3e64e4640 1 -- 172.21.15.83:0/4260737556 >> 172.21.15.83:0/4260737556 conn(0x7fc3e006abe0 msgr2=0x7fc3e006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:16.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.532+0000 7fc3e64e4640 1 -- 172.21.15.83:0/4260737556 shutdown_connections 2024-04-17T10:51:16.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.532+0000 7fc3e64e4640 1 -- 172.21.15.83:0/4260737556 wait complete. 2024-04-17T10:51:16.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.532+0000 7fc3e64e4640 1 Processor -- start 2024-04-17T10:51:16.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.532+0000 7fc3e64e4640 1 -- start start 2024-04-17T10:51:16.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.532+0000 7fc3e64e4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3e0072330 0x7fc3e01547e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.532+0000 7fc3e64e4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e00cfa10 0x7fc3e01508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.532+0000 7fc3e64e4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3e015ba30 0x7fc3e015e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.532+0000 7fc3e64e4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e006de20 con 0x7fc3e00cfa10 2024-04-17T10:51:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.532+0000 7fc3e64e4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e006dca0 con 0x7fc3e015ba30 2024-04-17T10:51:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.532+0000 7fc3e64e4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e0168520 con 0x7fc3e0072330 2024-04-17T10:51:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.532+0000 7fc3df7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e00cfa10 0x7fc3e01508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.532+0000 7fc3df7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e00cfa10 0x7fc3e01508d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35458/0 (socket says 172.21.15.83:35458) 2024-04-17T10:51:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.532+0000 7fc3df7fe640 1 -- 172.21.15.83:0/1026926440 learned_addr learned my addr 172.21.15.83:0/1026926440 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.532+0000 7fc3e4a81640 1 --2- 172.21.15.83:0/1026926440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3e015ba30 0x7fc3e015e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.532+0000 7fc3dffff640 1 --2- 172.21.15.83:0/1026926440 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3e0072330 0x7fc3e01547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.532+0000 7fc3df7fe640 1 -- 172.21.15.83:0/1026926440 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3e0072330 msgr2=0x7fc3e01547e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.532+0000 7fc3df7fe640 1 --2- 172.21.15.83:0/1026926440 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3e0072330 0x7fc3e01547e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.532+0000 7fc3df7fe640 1 -- 172.21.15.83:0/1026926440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3e015ba30 msgr2=0x7fc3e015e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.532+0000 7fc3df7fe640 1 --2- 172.21.15.83:0/1026926440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3e015ba30 0x7fc3e015e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.532+0000 7fc3df7fe640 1 -- 172.21.15.83:0/1026926440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e0152fe0 con 0x7fc3e00cfa10 2024-04-17T10:51:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.532+0000 7fc3dffff640 1 --2- 172.21.15.83:0/1026926440 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3e0072330 0x7fc3e01547e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:51:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.532+0000 7fc3df7fe640 1 --2- 172.21.15.83:0/1026926440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e00cfa10 0x7fc3e01508d0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7fc3c800c7f0 tx=0x7fc3c800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.536+0000 7fc3dcff9640 1 -- 172.21.15.83:0/1026926440 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8015070 con 0x7fc3e00cfa10 2024-04-17T10:51:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.536+0000 7fc3dcff9640 1 -- 172.21.15.83:0/1026926440 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c8007820 con 0x7fc3e00cfa10 2024-04-17T10:51:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.536+0000 7fc3e64e4640 1 -- 172.21.15.83:0/1026926440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3e0153d00 con 0x7fc3e00cfa10 2024-04-17T10:51:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.536+0000 7fc3dcff9640 1 -- 172.21.15.83:0/1026926440 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8007c10 con 0x7fc3e00cfa10 2024-04-17T10:51:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.536+0000 7fc3e64e4640 1 -- 172.21.15.83:0/1026926440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e01542c0 con 0x7fc3e00cfa10 2024-04-17T10:51:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.536+0000 7fc3e64e4640 1 -- 172.21.15.83:0/1026926440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3e0153af0 con 0x7fc3e00cfa10 2024-04-17T10:51:16.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.536+0000 7fc3dcff9640 1 -- 172.21.15.83:0/1026926440 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc3c8018070 con 0x7fc3e00cfa10 2024-04-17T10:51:16.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.536+0000 7fc3dcff9640 1 --2- 172.21.15.83:0/1026926440 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc3c0066f60 0x7fc3c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:16.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.536+0000 7fc3dcff9640 1 -- 172.21.15.83:0/1026926440 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fc3c80856b0 con 0x7fc3e00cfa10 2024-04-17T10:51:16.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.536+0000 7fc3dffff640 1 --2- 172.21.15.83:0/1026926440 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc3c0066f60 0x7fc3c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:16.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.536+0000 7fc3dffff640 1 --2- 172.21.15.83:0/1026926440 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc3c0066f60 0x7fc3c0069420 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fc3e01f7810 tx=0x7fc3d0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:16.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.540+0000 7fc3dcff9640 1 -- 172.21.15.83:0/1026926440 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc3c8056690 con 0x7fc3e00cfa10 2024-04-17T10:51:16.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.732+0000 7fc3e64e4640 1 -- 172.21.15.83:0/1026926440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3e0074f90 con 0x7fc3e00cfa10 2024-04-17T10:51:16.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.732+0000 7fc3dcff9640 1 -- 172.21.15.83:0/1026926440 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fc3c8055f90 con 0x7fc3e00cfa10 2024-04-17T10:51:16.725 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:51:16.725 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:51:16.726 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:51:16.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.732+0000 7fc3e64e4640 1 -- 172.21.15.83:0/1026926440 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc3c0066f60 msgr2=0x7fc3c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:16.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.732+0000 7fc3e64e4640 1 --2- 172.21.15.83:0/1026926440 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc3c0066f60 0x7fc3c0069420 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fc3e01f7810 tx=0x7fc3d0009040 comp rx=0 tx=0).stop 2024-04-17T10:51:16.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.732+0000 7fc3e64e4640 1 -- 172.21.15.83:0/1026926440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e00cfa10 msgr2=0x7fc3e01508d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:16.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.732+0000 7fc3e64e4640 1 --2- 172.21.15.83:0/1026926440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e00cfa10 0x7fc3e01508d0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7fc3c800c7f0 tx=0x7fc3c800ccb0 comp rx=0 tx=0).stop 2024-04-17T10:51:16.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.732+0000 7fc3e64e4640 1 -- 172.21.15.83:0/1026926440 shutdown_connections 2024-04-17T10:51:16.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.732+0000 7fc3e64e4640 1 --2- 172.21.15.83:0/1026926440 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc3c0066f60 0x7fc3c0069420 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:16.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.732+0000 7fc3e64e4640 1 --2- 172.21.15.83:0/1026926440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3e015ba30 0x7fc3e015e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:16.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.732+0000 7fc3e64e4640 1 --2- 172.21.15.83:0/1026926440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e00cfa10 0x7fc3e01508d0 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:16.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.732+0000 7fc3e64e4640 1 --2- 172.21.15.83:0/1026926440 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3e0072330 0x7fc3e01547e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:16.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.732+0000 7fc3e64e4640 1 -- 172.21.15.83:0/1026926440 >> 172.21.15.83:0/1026926440 conn(0x7fc3e006abe0 msgr2=0x7fc3e0151b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:16.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.736+0000 7fc3e64e4640 1 -- 172.21.15.83:0/1026926440 shutdown_connections 2024-04-17T10:51:16.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:16.736+0000 7fc3e64e4640 1 -- 172.21.15.83:0/1026926440 wait complete. 2024-04-17T10:51:17.145 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:51:17.145 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:51:18.146 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:51:18.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.284+0000 7ff349681640 1 Processor -- start 2024-04-17T10:51:18.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.284+0000 7ff349681640 1 -- start start 2024-04-17T10:51:18.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.284+0000 7ff349681640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff34415ba30 0x7ff344072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:18.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.284+0000 7ff349681640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff34415c960 0x7ff3440cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:18.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.284+0000 7ff349681640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff3440cfa10 0x7ff34406c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:18.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.284+0000 7ff349681640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff344071ea0 con 0x7ff3440cfa10 2024-04-17T10:51:18.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.284+0000 7ff349681640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3440cfea0 con 0x7ff34415ba30 2024-04-17T10:51:18.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.284+0000 7ff349681640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff34406e320 con 0x7ff34415c960 2024-04-17T10:51:18.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.284+0000 7ff3437fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff3440cfa10 0x7ff34406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:18.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.284+0000 7ff3437fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff3440cfa10 0x7ff34406c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58486/0 (socket says 172.21.15.83:58486) 2024-04-17T10:51:18.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.284+0000 7ff3437fe640 1 -- 172.21.15.83:0/2861785528 learned_addr learned my addr 172.21.15.83:0/2861785528 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:18.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.284+0000 7ff3437fe640 1 -- 172.21.15.83:0/2861785528 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff34415c960 msgr2=0x7ff3440cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:18.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.284+0000 7ff3437fe640 1 --2- 172.21.15.83:0/2861785528 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff34415c960 0x7ff3440cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:18.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.284+0000 7ff3437fe640 1 -- 172.21.15.83:0/2861785528 >> v2:172.21.15.167:3300/0 conn(0x7ff34415ba30 msgr2=0x7ff344072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:18.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.284+0000 7ff3437fe640 1 --2- 172.21.15.83:0/2861785528 >> v2:172.21.15.167:3300/0 conn(0x7ff34415ba30 0x7ff344072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:18.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.284+0000 7ff3437fe640 1 -- 172.21.15.83:0/2861785528 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34406e040 con 0x7ff3440cfa10 2024-04-17T10:51:18.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.288+0000 7ff3437fe640 1 --2- 172.21.15.83:0/2861785528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3440cfa10 0x7ff34406c490 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7ff3440b2410 tx=0x7ff33c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c81865de533169e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:18.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.288+0000 7ff341ffb640 1 -- 172.21.15.83:0/2861785528 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c0a56d0 con 0x7ff3440cfa10 2024-04-17T10:51:18.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.288+0000 7ff341ffb640 1 -- 172.21.15.83:0/2861785528 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff33c0a5870 con 0x7ff3440cfa10 2024-04-17T10:51:18.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.288+0000 7ff341ffb640 1 -- 172.21.15.83:0/2861785528 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c0a5c40 con 0x7ff3440cfa10 2024-04-17T10:51:18.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.288+0000 7ff349681640 1 -- 172.21.15.83:0/2861785528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3440cfa10 msgr2=0x7ff34406c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:18.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.288+0000 7ff349681640 1 --2- 172.21.15.83:0/2861785528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3440cfa10 0x7ff34406c490 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7ff3440b2410 tx=0x7ff33c01c960 comp rx=0 tx=0).stop 2024-04-17T10:51:18.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.288+0000 7ff349681640 1 -- 172.21.15.83:0/2861785528 shutdown_connections 2024-04-17T10:51:18.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.288+0000 7ff349681640 1 --2- 172.21.15.83:0/2861785528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3440cfa10 0x7ff34406c490 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:18.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.288+0000 7ff349681640 1 --2- 172.21.15.83:0/2861785528 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff34415c960 0x7ff3440cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:18.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.288+0000 7ff349681640 1 --2- 172.21.15.83:0/2861785528 >> v2:172.21.15.167:3300/0 conn(0x7ff34415ba30 0x7ff344072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:18.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.288+0000 7ff349681640 1 -- 172.21.15.83:0/2861785528 >> 172.21.15.83:0/2861785528 conn(0x7ff34406abe0 msgr2=0x7ff34406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:18.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.288+0000 7ff349681640 1 -- 172.21.15.83:0/2861785528 shutdown_connections 2024-04-17T10:51:18.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.288+0000 7ff349681640 1 -- 172.21.15.83:0/2861785528 wait complete. 2024-04-17T10:51:18.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.288+0000 7ff349681640 1 Processor -- start 2024-04-17T10:51:18.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.288+0000 7ff349681640 1 -- start start 2024-04-17T10:51:18.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.288+0000 7ff349681640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3440cfa10 0x7ff34413d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:18.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.288+0000 7ff349681640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff34415ba30 0x7ff3440b3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:18.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.288+0000 7ff349681640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff34415c960 0x7ff34414f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:18.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.288+0000 7ff349681640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34406dca0 con 0x7ff3440cfa10 2024-04-17T10:51:18.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.288+0000 7ff349681640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3440aeae0 con 0x7ff34415ba30 2024-04-17T10:51:18.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.288+0000 7ff349681640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff34406de20 con 0x7ff34415c960 2024-04-17T10:51:18.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.288+0000 7ff342ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3440cfa10 0x7ff34413d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:18.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.288+0000 7ff342ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3440cfa10 0x7ff34413d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58502/0 (socket says 172.21.15.83:58502) 2024-04-17T10:51:18.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.288+0000 7ff342ffd640 1 -- 172.21.15.83:0/3539667301 learned_addr learned my addr 172.21.15.83:0/3539667301 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:18.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.288+0000 7ff3427fc640 1 --2- 172.21.15.83:0/3539667301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff34415ba30 0x7ff3440b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:18.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.288+0000 7ff3437fe640 1 --2- 172.21.15.83:0/3539667301 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff34415c960 0x7ff34414f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:18.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.288+0000 7ff342ffd640 1 -- 172.21.15.83:0/3539667301 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff34415c960 msgr2=0x7ff34414f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:18.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.288+0000 7ff342ffd640 1 --2- 172.21.15.83:0/3539667301 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff34415c960 0x7ff34414f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:18.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.288+0000 7ff342ffd640 1 -- 172.21.15.83:0/3539667301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff34415ba30 msgr2=0x7ff3440b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:18.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.288+0000 7ff342ffd640 1 --2- 172.21.15.83:0/3539667301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff34415ba30 0x7ff3440b3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:18.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.288+0000 7ff342ffd640 1 -- 172.21.15.83:0/3539667301 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3440b3b10 con 0x7ff3440cfa10 2024-04-17T10:51:18.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.288+0000 7ff3427fc640 1 --2- 172.21.15.83:0/3539667301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff34415ba30 0x7ff3440b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:51:18.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.292+0000 7ff342ffd640 1 --2- 172.21.15.83:0/3539667301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3440cfa10 0x7ff34413d300 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7ff33400b410 tx=0x7ff33400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:18.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.292+0000 7ff32b7fe640 1 -- 172.21.15.83:0/3539667301 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff334015020 con 0x7ff3440cfa10 2024-04-17T10:51:18.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.292+0000 7ff32b7fe640 1 -- 172.21.15.83:0/3539667301 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff334011830 con 0x7ff3440cfa10 2024-04-17T10:51:18.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.292+0000 7ff32b7fe640 1 -- 172.21.15.83:0/3539667301 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff334011c20 con 0x7ff3440cfa10 2024-04-17T10:51:18.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.292+0000 7ff349681640 1 -- 172.21.15.83:0/3539667301 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3440b3df0 con 0x7ff3440cfa10 2024-04-17T10:51:18.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.292+0000 7ff349681640 1 -- 172.21.15.83:0/3539667301 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3440b43b0 con 0x7ff3440cfa10 2024-04-17T10:51:18.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.292+0000 7ff349681640 1 -- 172.21.15.83:0/3539667301 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff344074f90 con 0x7ff3440cfa10 2024-04-17T10:51:18.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.292+0000 7ff32b7fe640 1 -- 172.21.15.83:0/3539667301 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff334018020 con 0x7ff3440cfa10 2024-04-17T10:51:18.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.292+0000 7ff32b7fe640 1 --2- 172.21.15.83:0/3539667301 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff330066f60 0x7ff330069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:18.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.292+0000 7ff32b7fe640 1 -- 172.21.15.83:0/3539667301 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7ff334084fa0 con 0x7ff3440cfa10 2024-04-17T10:51:18.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.292+0000 7ff3427fc640 1 --2- 172.21.15.83:0/3539667301 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff330066f60 0x7ff330069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:18.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.292+0000 7ff3427fc640 1 --2- 172.21.15.83:0/3539667301 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff330066f60 0x7ff330069420 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ff34415de90 tx=0x7ff338008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:18.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.296+0000 7ff32b7fe640 1 -- 172.21.15.83:0/3539667301 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff33401f020 con 0x7ff3440cfa10 2024-04-17T10:51:18.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.492+0000 7ff349681640 1 -- 172.21.15.83:0/3539667301 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff34406c7b0 con 0x7ff3440cfa10 2024-04-17T10:51:18.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.496+0000 7ff32b7fe640 1 -- 172.21.15.83:0/3539667301 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7ff3340172c0 con 0x7ff3440cfa10 2024-04-17T10:51:18.487 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:51:18.487 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:51:18.488 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:51:18.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.496+0000 7ff349681640 1 -- 172.21.15.83:0/3539667301 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff330066f60 msgr2=0x7ff330069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:18.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.496+0000 7ff349681640 1 --2- 172.21.15.83:0/3539667301 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff330066f60 0x7ff330069420 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ff34415de90 tx=0x7ff338008040 comp rx=0 tx=0).stop 2024-04-17T10:51:18.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.496+0000 7ff349681640 1 -- 172.21.15.83:0/3539667301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3440cfa10 msgr2=0x7ff34413d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:18.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.496+0000 7ff349681640 1 --2- 172.21.15.83:0/3539667301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3440cfa10 0x7ff34413d300 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7ff33400b410 tx=0x7ff33400b8d0 comp rx=0 tx=0).stop 2024-04-17T10:51:18.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.496+0000 7ff349681640 1 -- 172.21.15.83:0/3539667301 shutdown_connections 2024-04-17T10:51:18.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.496+0000 7ff349681640 1 --2- 172.21.15.83:0/3539667301 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff330066f60 0x7ff330069420 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:18.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.496+0000 7ff349681640 1 --2- 172.21.15.83:0/3539667301 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff34415c960 0x7ff34414f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:18.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.496+0000 7ff349681640 1 --2- 172.21.15.83:0/3539667301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff34415ba30 0x7ff3440b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:18.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.496+0000 7ff349681640 1 --2- 172.21.15.83:0/3539667301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3440cfa10 0x7ff34413d300 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:18.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.496+0000 7ff349681640 1 -- 172.21.15.83:0/3539667301 >> 172.21.15.83:0/3539667301 conn(0x7ff34406abe0 msgr2=0x7ff344152c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:18.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.496+0000 7ff349681640 1 -- 172.21.15.83:0/3539667301 shutdown_connections 2024-04-17T10:51:18.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:18.496+0000 7ff349681640 1 -- 172.21.15.83:0/3539667301 wait complete. 2024-04-17T10:51:18.908 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:51:18.908 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:51:19.909 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:51:20.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.048+0000 7feb45cf0640 1 Processor -- start 2024-04-17T10:51:20.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.048+0000 7feb45cf0640 1 -- start start 2024-04-17T10:51:20.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.052+0000 7feb45cf0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb4015b770 0x7feb4006d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:20.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.052+0000 7feb45cf0640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7feb40072220 0x7feb400cd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:20.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.052+0000 7feb45cf0640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb4006e540 con 0x7feb400cbde0 2024-04-17T10:51:20.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.052+0000 7feb45cf0640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb4006d160 con 0x7feb40072220 2024-04-17T10:51:20.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.052+0000 7feb45cf0640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb4006c270 con 0x7feb4015b770 2024-04-17T10:51:20.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.052+0000 7feb454ef640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7feb40072220 0x7feb400cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:20.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.052+0000 7feb44cee640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb4015b770 0x7feb4006d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:20.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.052+0000 7feb454ef640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7feb40072220 0x7feb400cd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:45402/0 (socket says 172.21.15.83:45402) 2024-04-17T10:51:20.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.052+0000 7feb454ef640 1 -- 172.21.15.83:0/2925292427 learned_addr learned my addr 172.21.15.83:0/2925292427 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:20.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.052+0000 7feb44cee640 1 -- 172.21.15.83:0/2925292427 >> v2:172.21.15.167:3300/0 conn(0x7feb40072220 msgr2=0x7feb400cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:20.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.052+0000 7feb44cee640 1 --2- 172.21.15.83:0/2925292427 >> v2:172.21.15.167:3300/0 conn(0x7feb40072220 0x7feb400cd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:20.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.052+0000 7feb44cee640 1 -- 172.21.15.83:0/2925292427 >> v1:172.21.15.167:6789/0 conn(0x7feb400cbde0 legacy=0x7feb40071b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:20.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.052+0000 7feb44cee640 1 -- 172.21.15.83:0/2925292427 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4006c550 con 0x7feb4015b770 2024-04-17T10:51:20.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.052+0000 7feb454ef640 1 --2- 172.21.15.83:0/2925292427 >> v2:172.21.15.167:3300/0 conn(0x7feb40072220 0x7feb400cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:51:20.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.052+0000 7feb44cee640 1 --2- 172.21.15.83:0/2925292427 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb4015b770 0x7feb4006d5f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7feb340097c0 tx=0x7feb3401c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=997efad95914385d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:20.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb3f7fe640 1 -- 172.21.15.83:0/2925292427 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb340b0070 con 0x7feb4015b770 2024-04-17T10:51:20.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb3f7fe640 1 -- 172.21.15.83:0/2925292427 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb340ab600 con 0x7feb4015b770 2024-04-17T10:51:20.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb3f7fe640 1 -- 172.21.15.83:0/2925292427 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb340ab9d0 con 0x7feb4015b770 2024-04-17T10:51:20.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb45cf0640 1 -- 172.21.15.83:0/2925292427 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb4015b770 msgr2=0x7feb4006d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:20.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb45cf0640 1 --2- 172.21.15.83:0/2925292427 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb4015b770 0x7feb4006d5f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7feb340097c0 tx=0x7feb3401c3e0 comp rx=0 tx=0).stop 2024-04-17T10:51:20.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb45cf0640 1 -- 172.21.15.83:0/2925292427 shutdown_connections 2024-04-17T10:51:20.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb45cf0640 1 --2- 172.21.15.83:0/2925292427 >> v2:172.21.15.167:3300/0 conn(0x7feb40072220 0x7feb400cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:20.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb45cf0640 1 --2- 172.21.15.83:0/2925292427 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb4015b770 0x7feb4006d5f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:20.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb45cf0640 1 -- 172.21.15.83:0/2925292427 >> 172.21.15.83:0/2925292427 conn(0x7feb400b1160 msgr2=0x7feb400b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:20.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb45cf0640 1 -- 172.21.15.83:0/2925292427 shutdown_connections 2024-04-17T10:51:20.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb45cf0640 1 -- 172.21.15.83:0/2925292427 wait complete. 2024-04-17T10:51:20.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb45cf0640 1 Processor -- start 2024-04-17T10:51:20.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb45cf0640 1 -- start start 2024-04-17T10:51:20.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb45cf0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb40072220 0x7feb4014fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:20.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb45cf0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb400cbde0 0x7feb4015dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:20.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb45cf0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb4015b770 0x7feb4015f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:20.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb45cf0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4015bb50 con 0x7feb40072220 2024-04-17T10:51:20.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb45cf0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4006db30 con 0x7feb400cbde0 2024-04-17T10:51:20.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb45cf0640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb4015bcd0 con 0x7feb4015b770 2024-04-17T10:51:20.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb44cee640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb40072220 0x7feb4014fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:20.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb44cee640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb40072220 0x7feb4014fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58510/0 (socket says 172.21.15.83:58510) 2024-04-17T10:51:20.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb44cee640 1 -- 172.21.15.83:0/1135564584 learned_addr learned my addr 172.21.15.83:0/1135564584 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:20.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb454ef640 1 --2- 172.21.15.83:0/1135564584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb4015b770 0x7feb4015f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:20.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb3ffff640 1 --2- 172.21.15.83:0/1135564584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb400cbde0 0x7feb4015dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:20.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb44cee640 1 -- 172.21.15.83:0/1135564584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb4015b770 msgr2=0x7feb4015f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:20.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb44cee640 1 --2- 172.21.15.83:0/1135564584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb4015b770 0x7feb4015f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:20.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb44cee640 1 -- 172.21.15.83:0/1135564584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb400cbde0 msgr2=0x7feb4015dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:20.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb44cee640 1 --2- 172.21.15.83:0/1135564584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb400cbde0 0x7feb4015dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:20.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb44cee640 1 -- 172.21.15.83:0/1135564584 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb401f6f90 con 0x7feb40072220 2024-04-17T10:51:20.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb44cee640 1 --2- 172.21.15.83:0/1135564584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb40072220 0x7feb4014fb90 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7feb340097c0 tx=0x7feb34019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:20.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb3d7fa640 1 -- 172.21.15.83:0/1135564584 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb340ba070 con 0x7feb40072220 2024-04-17T10:51:20.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb45cf0640 1 -- 172.21.15.83:0/1135564584 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb4015e8a0 con 0x7feb40072220 2024-04-17T10:51:20.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.056+0000 7feb3d7fa640 1 -- 172.21.15.83:0/1135564584 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb340b5460 con 0x7feb40072220 2024-04-17T10:51:20.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.060+0000 7feb3d7fa640 1 -- 172.21.15.83:0/1135564584 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb340b0040 con 0x7feb40072220 2024-04-17T10:51:20.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.060+0000 7feb45cf0640 1 -- 172.21.15.83:0/1135564584 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb401f7230 con 0x7feb40072220 2024-04-17T10:51:20.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.060+0000 7feb45cf0640 1 -- 172.21.15.83:0/1135564584 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb4004efa0 con 0x7feb40072220 2024-04-17T10:51:20.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.060+0000 7feb3d7fa640 1 -- 172.21.15.83:0/1135564584 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feb340bd070 con 0x7feb40072220 2024-04-17T10:51:20.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.060+0000 7feb3d7fa640 1 --2- 172.21.15.83:0/1135564584 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feb2c066f60 0x7feb2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:20.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.060+0000 7feb3d7fa640 1 -- 172.21.15.83:0/1135564584 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7feb3411f4b0 con 0x7feb40072220 2024-04-17T10:51:20.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.060+0000 7feb3ffff640 1 --2- 172.21.15.83:0/1135564584 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feb2c066f60 0x7feb2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:20.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.060+0000 7feb3ffff640 1 --2- 172.21.15.83:0/1135564584 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feb2c066f60 0x7feb2c069420 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7feb40165330 tx=0x7feb30009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:20.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.064+0000 7feb3d7fa640 1 -- 172.21.15.83:0/1135564584 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feb34016090 con 0x7feb40072220 2024-04-17T10:51:20.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.256+0000 7feb45cf0640 1 -- 172.21.15.83:0/1135564584 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb400b1e80 con 0x7feb40072220 2024-04-17T10:51:20.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.256+0000 7feb3d7fa640 1 -- 172.21.15.83:0/1135564584 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7feb340efea0 con 0x7feb40072220 2024-04-17T10:51:20.251 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:51:20.251 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:51:20.251 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:51:20.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.260+0000 7feb45cf0640 1 -- 172.21.15.83:0/1135564584 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feb2c066f60 msgr2=0x7feb2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:20.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.260+0000 7feb45cf0640 1 --2- 172.21.15.83:0/1135564584 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feb2c066f60 0x7feb2c069420 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7feb40165330 tx=0x7feb30009040 comp rx=0 tx=0).stop 2024-04-17T10:51:20.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.260+0000 7feb45cf0640 1 -- 172.21.15.83:0/1135564584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb40072220 msgr2=0x7feb4014fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:20.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.260+0000 7feb45cf0640 1 --2- 172.21.15.83:0/1135564584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb40072220 0x7feb4014fb90 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7feb340097c0 tx=0x7feb34019180 comp rx=0 tx=0).stop 2024-04-17T10:51:20.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.260+0000 7feb45cf0640 1 -- 172.21.15.83:0/1135564584 shutdown_connections 2024-04-17T10:51:20.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.260+0000 7feb45cf0640 1 --2- 172.21.15.83:0/1135564584 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feb2c066f60 0x7feb2c069420 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:20.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.260+0000 7feb45cf0640 1 --2- 172.21.15.83:0/1135564584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb4015b770 0x7feb4015f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:20.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.260+0000 7feb45cf0640 1 --2- 172.21.15.83:0/1135564584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb400cbde0 0x7feb4015dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:20.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.260+0000 7feb45cf0640 1 --2- 172.21.15.83:0/1135564584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb40072220 0x7feb4014fb90 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:20.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.260+0000 7feb45cf0640 1 -- 172.21.15.83:0/1135564584 >> 172.21.15.83:0/1135564584 conn(0x7feb400b1160 msgr2=0x7feb400b1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:20.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.260+0000 7feb45cf0640 1 -- 172.21.15.83:0/1135564584 shutdown_connections 2024-04-17T10:51:20.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:20.260+0000 7feb45cf0640 1 -- 172.21.15.83:0/1135564584 wait complete. 2024-04-17T10:51:20.673 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:51:20.673 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:51:21.674 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:51:21.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.812+0000 7ff7e9340640 1 Processor -- start 2024-04-17T10:51:21.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.812+0000 7ff7e9340640 1 -- start start 2024-04-17T10:51:21.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.816+0000 7ff7e9340640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7e40b2d80 0x7ff7e4146e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:21.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.816+0000 7ff7e9340640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e406db10 con 0x7ff7e4146a00 2024-04-17T10:51:21.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.816+0000 7ff7e9340640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e4147570 con 0x7ff7e406d690 2024-04-17T10:51:21.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.816+0000 7ff7e9340640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e406c960 con 0x7ff7e40b2d80 2024-04-17T10:51:21.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.816+0000 7ff7e2ffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff7e4146a00 0x7ff7e40b2670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46858/0 (socket says 172.21.15.83:46858) 2024-04-17T10:51:21.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.816+0000 7ff7e2ffd640 1 -- 172.21.15.83:0/956344552 learned_addr learned my addr 172.21.15.83:0/956344552 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:51:21.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.816+0000 7ff7e1ffb640 1 -- 172.21.15.83:0/956344552 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3572543643 0 0) 0x7ff7e406db10 con 0x7ff7e4146a00 2024-04-17T10:51:21.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.816+0000 7ff7e1ffb640 1 -- 172.21.15.83:0/956344552 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7cc003530 con 0x7ff7e4146a00 2024-04-17T10:51:21.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.816+0000 7ff7e27fc640 1 --2- 172.21.15.83:0/956344552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7e40b2d80 0x7ff7e4146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:21.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.816+0000 7ff7e1ffb640 1 -- 172.21.15.83:0/956344552 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2526071764 0 0) 0x7ff7cc003530 con 0x7ff7e4146a00 2024-04-17T10:51:21.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.816+0000 7ff7e1ffb640 1 -- 172.21.15.83:0/956344552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7e40b2d80 msgr2=0x7ff7e4146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:21.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.816+0000 7ff7e1ffb640 1 --2- 172.21.15.83:0/956344552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7e40b2d80 0x7ff7e4146e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:21.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.816+0000 7ff7e1ffb640 1 -- 172.21.15.83:0/956344552 >> v1:172.21.15.167:6789/0 conn(0x7ff7e406d690 legacy=0x7ff7e4070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:21.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.816+0000 7ff7e1ffb640 1 -- 172.21.15.83:0/956344552 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e406cbb0 con 0x7ff7e4146a00 2024-04-17T10:51:21.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.816+0000 7ff7e1ffb640 1 -- 172.21.15.83:0/956344552 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7ff7d4001680 con 0x7ff7e4146a00 2024-04-17T10:51:21.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.816+0000 7ff7e1ffb640 1 -- 172.21.15.83:0/956344552 >> v1:172.21.15.83:6789/0 conn(0x7ff7e4146a00 legacy=0x7ff7e40b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:21.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.816+0000 7ff7e1ffb640 1 --2- 172.21.15.83:0/956344552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7cc0039f0 0x7ff7cc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:21.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.816+0000 7ff7e1ffb640 1 --2- 172.21.15.83:0/956344552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7e40b2d80 0x7ff7cc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:21.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.816+0000 7ff7e1ffb640 1 --2- 172.21.15.83:0/956344552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7cc004d50 0x7ff7cc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:21.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.816+0000 7ff7e1ffb640 1 -- 172.21.15.83:0/956344552 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e406c960 con 0x7ff7cc0039f0 2024-04-17T10:51:21.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.816+0000 7ff7e1ffb640 1 -- 172.21.15.83:0/956344552 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc00ba80 con 0x7ff7cc004d50 2024-04-17T10:51:21.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.816+0000 7ff7e1ffb640 1 -- 172.21.15.83:0/956344552 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc00bc80 con 0x7ff7e40b2d80 2024-04-17T10:51:21.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.816+0000 7ff7e2ffd640 1 --2- 172.21.15.83:0/956344552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7cc0039f0 0x7ff7cc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:21.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.816+0000 7ff7e27fc640 1 --2- 172.21.15.83:0/956344552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7e40b2d80 0x7ff7cc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:21.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.816+0000 7ff7e2ffd640 1 -- 172.21.15.83:0/956344552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7e40b2d80 msgr2=0x7ff7cc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:21.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.816+0000 7ff7e2ffd640 1 --2- 172.21.15.83:0/956344552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7e40b2d80 0x7ff7cc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:21.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.816+0000 7ff7e2ffd640 1 -- 172.21.15.83:0/956344552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7cc004d50 msgr2=0x7ff7cc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:21.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.816+0000 7ff7e37fe640 1 --2- 172.21.15.83:0/956344552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7cc004d50 0x7ff7cc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:21.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.816+0000 7ff7e2ffd640 1 --2- 172.21.15.83:0/956344552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7cc004d50 0x7ff7cc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:21.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.816+0000 7ff7e2ffd640 1 -- 172.21.15.83:0/956344552 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7cc003530 con 0x7ff7cc0039f0 2024-04-17T10:51:21.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.816+0000 7ff7e37fe640 1 --2- 172.21.15.83:0/956344552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7cc004d50 0x7ff7cc0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:51:21.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.820+0000 7ff7e2ffd640 1 --2- 172.21.15.83:0/956344552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7cc0039f0 0x7ff7cc003e10 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7ff7d400c190 tx=0x7ff7d401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e2c0cb9033f1b8c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:21.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.820+0000 7ff7e1ffb640 1 -- 172.21.15.83:0/956344552 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d4007d00 con 0x7ff7cc0039f0 2024-04-17T10:51:21.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.820+0000 7ff7e1ffb640 1 -- 172.21.15.83:0/956344552 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d40a9020 con 0x7ff7cc0039f0 2024-04-17T10:51:21.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.820+0000 7ff7e1ffb640 1 -- 172.21.15.83:0/956344552 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d4005b00 con 0x7ff7cc0039f0 2024-04-17T10:51:21.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.820+0000 7ff7e9340640 1 -- 172.21.15.83:0/956344552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7cc0039f0 msgr2=0x7ff7cc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:21.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.820+0000 7ff7e9340640 1 --2- 172.21.15.83:0/956344552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7cc0039f0 0x7ff7cc003e10 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7ff7d400c190 tx=0x7ff7d401cc40 comp rx=0 tx=0).stop 2024-04-17T10:51:21.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.820+0000 7ff7e37fe640 1 -- 172.21.15.83:0/956344552 reap_dead start 2024-04-17T10:51:21.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.820+0000 7ff7e9340640 1 -- 172.21.15.83:0/956344552 shutdown_connections 2024-04-17T10:51:21.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.820+0000 7ff7e9340640 1 -- 172.21.15.83:0/956344552 >> 172.21.15.83:0/956344552 conn(0x7ff7e406abe0 msgr2=0x7ff7e406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:21.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.820+0000 7ff7e9340640 1 -- 172.21.15.83:0/956344552 shutdown_connections 2024-04-17T10:51:21.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.820+0000 7ff7e9340640 1 -- 172.21.15.83:0/956344552 wait complete. 2024-04-17T10:51:21.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.820+0000 7ff7e9340640 1 Processor -- start 2024-04-17T10:51:21.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.820+0000 7ff7e9340640 1 -- start start 2024-04-17T10:51:21.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.820+0000 7ff7e9340640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7e413be60 0x7ff7e415ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:21.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.820+0000 7ff7e9340640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7e413c240 0x7ff7e415cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:21.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.820+0000 7ff7e9340640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7e40b0e10 0x7ff7e40b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:21.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.820+0000 7ff7e9340640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e4070d00 con 0x7ff7e413c240 2024-04-17T10:51:21.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.820+0000 7ff7e9340640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e4070b80 con 0x7ff7e413be60 2024-04-17T10:51:21.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.820+0000 7ff7e9340640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e4070e80 con 0x7ff7e40b0e10 2024-04-17T10:51:21.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.820+0000 7ff7e27fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7e413c240 0x7ff7e415cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:21.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.820+0000 7ff7e27fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7e413c240 0x7ff7e415cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58522/0 (socket says 172.21.15.83:58522) 2024-04-17T10:51:21.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.820+0000 7ff7e27fc640 1 -- 172.21.15.83:0/3838351053 learned_addr learned my addr 172.21.15.83:0/3838351053 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:21.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.820+0000 7ff7e2ffd640 1 --2- 172.21.15.83:0/3838351053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7e413be60 0x7ff7e415ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:21.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.820+0000 7ff7e37fe640 1 --2- 172.21.15.83:0/3838351053 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7e40b0e10 0x7ff7e40b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:21.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.820+0000 7ff7e27fc640 1 -- 172.21.15.83:0/3838351053 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7e40b0e10 msgr2=0x7ff7e40b12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:21.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.820+0000 7ff7e27fc640 1 --2- 172.21.15.83:0/3838351053 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7e40b0e10 0x7ff7e40b12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:21.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.820+0000 7ff7e27fc640 1 -- 172.21.15.83:0/3838351053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7e413be60 msgr2=0x7ff7e415ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:21.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.820+0000 7ff7e27fc640 1 --2- 172.21.15.83:0/3838351053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7e413be60 0x7ff7e415ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:21.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.820+0000 7ff7e27fc640 1 -- 172.21.15.83:0/3838351053 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e40af280 con 0x7ff7e413c240 2024-04-17T10:51:21.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.820+0000 7ff7e37fe640 1 --2- 172.21.15.83:0/3838351053 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7e40b0e10 0x7ff7e40b12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:51:21.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.820+0000 7ff7e27fc640 1 --2- 172.21.15.83:0/3838351053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7e413c240 0x7ff7e415cf40 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8011590 tx=0x7ff7d8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:21.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.820+0000 7ff7cb7fe640 1 -- 172.21.15.83:0/3838351053 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8019070 con 0x7ff7e413c240 2024-04-17T10:51:21.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.824+0000 7ff7cb7fe640 1 -- 172.21.15.83:0/3838351053 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d8003620 con 0x7ff7e413c240 2024-04-17T10:51:21.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.824+0000 7ff7e9340640 1 -- 172.21.15.83:0/3838351053 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e40af560 con 0x7ff7e413c240 2024-04-17T10:51:21.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.824+0000 7ff7cb7fe640 1 -- 172.21.15.83:0/3838351053 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8003a10 con 0x7ff7e413c240 2024-04-17T10:51:21.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.824+0000 7ff7e9340640 1 -- 172.21.15.83:0/3838351053 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e40afb20 con 0x7ff7e413c240 2024-04-17T10:51:21.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.824+0000 7ff7e9340640 1 -- 172.21.15.83:0/3838351053 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7e4074f90 con 0x7ff7e413c240 2024-04-17T10:51:21.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.824+0000 7ff7cb7fe640 1 -- 172.21.15.83:0/3838351053 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff7d801c070 con 0x7ff7e413c240 2024-04-17T10:51:21.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.824+0000 7ff7cb7fe640 1 --2- 172.21.15.83:0/3838351053 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff7d0066f90 0x7ff7d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:21.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.824+0000 7ff7cb7fe640 1 -- 172.21.15.83:0/3838351053 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7ff7d8085330 con 0x7ff7e413c240 2024-04-17T10:51:21.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.824+0000 7ff7e2ffd640 1 --2- 172.21.15.83:0/3838351053 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff7d0066f90 0x7ff7d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:21.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.824+0000 7ff7e2ffd640 1 --2- 172.21.15.83:0/3838351053 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff7d0066f90 0x7ff7d0069450 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40aa000 tx=0x7ff7e4146de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:21.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:21.828+0000 7ff7cb7fe640 1 -- 172.21.15.83:0/3838351053 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff7d8056360 con 0x7ff7e413c240 2024-04-17T10:51:22.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:22.020+0000 7ff7e9340640 1 -- 172.21.15.83:0/3838351053 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7e413d620 con 0x7ff7e413c240 2024-04-17T10:51:22.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:22.020+0000 7ff7cb7fe640 1 -- 172.21.15.83:0/3838351053 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7ff7d8055c60 con 0x7ff7e413c240 2024-04-17T10:51:22.012 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:51:22.012 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:51:22.013 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:51:22.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:22.020+0000 7ff7e9340640 1 -- 172.21.15.83:0/3838351053 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff7d0066f90 msgr2=0x7ff7d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:22.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:22.020+0000 7ff7e9340640 1 --2- 172.21.15.83:0/3838351053 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff7d0066f90 0x7ff7d0069450 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40aa000 tx=0x7ff7e4146de0 comp rx=0 tx=0).stop 2024-04-17T10:51:22.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:22.020+0000 7ff7e9340640 1 -- 172.21.15.83:0/3838351053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7e413c240 msgr2=0x7ff7e415cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:22.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:22.020+0000 7ff7e9340640 1 --2- 172.21.15.83:0/3838351053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7e413c240 0x7ff7e415cf40 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8011590 tx=0x7ff7d8009f90 comp rx=0 tx=0).stop 2024-04-17T10:51:22.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:22.020+0000 7ff7e9340640 1 -- 172.21.15.83:0/3838351053 shutdown_connections 2024-04-17T10:51:22.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:22.020+0000 7ff7e9340640 1 --2- 172.21.15.83:0/3838351053 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff7d0066f90 0x7ff7d0069450 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:22.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:22.020+0000 7ff7e9340640 1 --2- 172.21.15.83:0/3838351053 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7e40b0e10 0x7ff7e40b12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:22.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:22.020+0000 7ff7e9340640 1 --2- 172.21.15.83:0/3838351053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7e413c240 0x7ff7e415cf40 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:22.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:22.020+0000 7ff7e9340640 1 --2- 172.21.15.83:0/3838351053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7e413be60 0x7ff7e415ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:22.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:22.020+0000 7ff7e9340640 1 -- 172.21.15.83:0/3838351053 >> 172.21.15.83:0/3838351053 conn(0x7ff7e406abe0 msgr2=0x7ff7e4166c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:22.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:22.020+0000 7ff7e9340640 1 -- 172.21.15.83:0/3838351053 shutdown_connections 2024-04-17T10:51:22.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:22.020+0000 7ff7e9340640 1 -- 172.21.15.83:0/3838351053 wait complete. 2024-04-17T10:51:22.431 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:51:22.431 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:51:23.433 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:51:23.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.572+0000 7febd8b14640 1 Processor -- start 2024-04-17T10:51:23.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.572+0000 7febd8b14640 1 -- start start 2024-04-17T10:51:23.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.576+0000 7febd8b14640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febd4072330 0x7febd40cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:23.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.576+0000 7febd8b14640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd40cfea0 con 0x7febd415ba30 2024-04-17T10:51:23.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.576+0000 7febd8b14640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd406e320 con 0x7febd40cfa10 2024-04-17T10:51:23.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.576+0000 7febd8b14640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd4071ea0 con 0x7febd4072330 2024-04-17T10:51:23.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.576+0000 7febd259c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7febd415ba30 0x7febd415c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46872/0 (socket says 172.21.15.83:46872) 2024-04-17T10:51:23.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.576+0000 7febd259c640 1 -- 172.21.15.83:0/3985146406 learned_addr learned my addr 172.21.15.83:0/3985146406 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:51:23.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.576+0000 7febcbfff640 1 --2- 172.21.15.83:0/3985146406 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febd4072330 0x7febd40cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:23.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.576+0000 7febd1d9b640 1 -- 172.21.15.83:0/3985146406 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 520763305 0 0) 0x7febd40cfea0 con 0x7febd415ba30 2024-04-17T10:51:23.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.576+0000 7febd1d9b640 1 -- 172.21.15.83:0/3985146406 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febc0003530 con 0x7febd415ba30 2024-04-17T10:51:23.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.576+0000 7febd1d9b640 1 -- 172.21.15.83:0/3985146406 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2283671520 0 0) 0x7febc0003530 con 0x7febd415ba30 2024-04-17T10:51:23.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.576+0000 7febd1d9b640 1 -- 172.21.15.83:0/3985146406 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febd4072330 msgr2=0x7febd40cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:23.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.576+0000 7febd1d9b640 1 --2- 172.21.15.83:0/3985146406 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febd4072330 0x7febd40cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:23.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.576+0000 7febd1d9b640 1 -- 172.21.15.83:0/3985146406 >> v1:172.21.15.167:6789/0 conn(0x7febd40cfa10 legacy=0x7febd406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:23.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.576+0000 7febd1d9b640 1 -- 172.21.15.83:0/3985146406 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd406e040 con 0x7febd415ba30 2024-04-17T10:51:23.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.576+0000 7febd1d9b640 1 -- 172.21.15.83:0/3985146406 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7febbc001680 con 0x7febd415ba30 2024-04-17T10:51:23.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.576+0000 7febd1d9b640 1 -- 172.21.15.83:0/3985146406 >> v1:172.21.15.83:6789/0 conn(0x7febd415ba30 legacy=0x7febd415c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:23.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.576+0000 7febd1d9b640 1 --2- 172.21.15.83:0/3985146406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febc00039f0 0x7febc0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:23.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.576+0000 7febd1d9b640 1 --2- 172.21.15.83:0/3985146406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febc0004810 0x7febc0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:23.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.576+0000 7febd1d9b640 1 --2- 172.21.15.83:0/3985146406 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febd4072330 0x7febc0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:23.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.576+0000 7febd1d9b640 1 -- 172.21.15.83:0/3985146406 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd4071ea0 con 0x7febc0004810 2024-04-17T10:51:23.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.576+0000 7febd1d9b640 1 -- 172.21.15.83:0/3985146406 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc000bb00 con 0x7febc00039f0 2024-04-17T10:51:23.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.576+0000 7febd1d9b640 1 -- 172.21.15.83:0/3985146406 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febc000bd00 con 0x7febd4072330 2024-04-17T10:51:23.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.576+0000 7febcbfff640 1 --2- 172.21.15.83:0/3985146406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febc0004810 0x7febc0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:23.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.576+0000 7febd259c640 1 --2- 172.21.15.83:0/3985146406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febc00039f0 0x7febc0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:23.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.576+0000 7febd2d9d640 1 --2- 172.21.15.83:0/3985146406 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febd4072330 0x7febc0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:23.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.576+0000 7febcbfff640 1 -- 172.21.15.83:0/3985146406 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febd4072330 msgr2=0x7febc0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:23.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.576+0000 7febcbfff640 1 --2- 172.21.15.83:0/3985146406 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febd4072330 0x7febc0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:23.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.576+0000 7febcbfff640 1 -- 172.21.15.83:0/3985146406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febc00039f0 msgr2=0x7febc0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:23.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.576+0000 7febcbfff640 1 --2- 172.21.15.83:0/3985146406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febc00039f0 0x7febc0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:23.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.576+0000 7febcbfff640 1 -- 172.21.15.83:0/3985146406 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc0003530 con 0x7febc0004810 2024-04-17T10:51:23.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.576+0000 7febd2d9d640 1 --2- 172.21.15.83:0/3985146406 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febd4072330 0x7febc0005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:51:23.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.580+0000 7febcbfff640 1 --2- 172.21.15.83:0/3985146406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febc0004810 0x7febc0004c90 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7febc4009ae0 tx=0x7febd406d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac6f496acae735e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:23.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.580+0000 7febd1d9b640 1 -- 172.21.15.83:0/3985146406 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc40b2070 con 0x7febc0004810 2024-04-17T10:51:23.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.580+0000 7febd1d9b640 1 -- 172.21.15.83:0/3985146406 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc40a5550 con 0x7febc0004810 2024-04-17T10:51:23.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.580+0000 7febd1d9b640 1 -- 172.21.15.83:0/3985146406 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc40096b0 con 0x7febc0004810 2024-04-17T10:51:23.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.580+0000 7febd8b14640 1 -- 172.21.15.83:0/3985146406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febc0004810 msgr2=0x7febc0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:23.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.580+0000 7febd8b14640 1 --2- 172.21.15.83:0/3985146406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febc0004810 0x7febc0004c90 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7febc4009ae0 tx=0x7febd406d3b0 comp rx=0 tx=0).stop 2024-04-17T10:51:23.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.580+0000 7febd2d9d640 1 -- 172.21.15.83:0/3985146406 reap_dead start 2024-04-17T10:51:23.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.580+0000 7febd8b14640 1 -- 172.21.15.83:0/3985146406 shutdown_connections 2024-04-17T10:51:23.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.580+0000 7febd8b14640 1 -- 172.21.15.83:0/3985146406 >> 172.21.15.83:0/3985146406 conn(0x7febd406abe0 msgr2=0x7febd406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:23.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.580+0000 7febd8b14640 1 -- 172.21.15.83:0/3985146406 shutdown_connections 2024-04-17T10:51:23.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.580+0000 7febd8b14640 1 -- 172.21.15.83:0/3985146406 wait complete. 2024-04-17T10:51:23.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.580+0000 7febd8b14640 1 Processor -- start 2024-04-17T10:51:23.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.580+0000 7febd8b14640 1 -- start start 2024-04-17T10:51:23.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.580+0000 7febd8b14640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febd4150050 0x7febd4152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:23.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.580+0000 7febd8b14640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febd4150f20 0x7febd4155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:23.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.580+0000 7febd8b14640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febd40ce5c0 0x7febd40cea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:23.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.580+0000 7febd8b14640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd406de20 con 0x7febd4150050 2024-04-17T10:51:23.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.580+0000 7febd8b14640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd406dca0 con 0x7febd40ce5c0 2024-04-17T10:51:23.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.580+0000 7febd8b14640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd4167520 con 0x7febd4150f20 2024-04-17T10:51:23.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.580+0000 7febd259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febd4150050 0x7febd4152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:23.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.580+0000 7febd259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febd4150050 0x7febd4152b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58540/0 (socket says 172.21.15.83:58540) 2024-04-17T10:51:23.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.580+0000 7febd259c640 1 -- 172.21.15.83:0/3792303363 learned_addr learned my addr 172.21.15.83:0/3792303363 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:23.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.580+0000 7febcbfff640 1 --2- 172.21.15.83:0/3792303363 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febd4150f20 0x7febd4155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:23.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.580+0000 7febd259c640 1 -- 172.21.15.83:0/3792303363 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febd4150f20 msgr2=0x7febd4155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:23.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.580+0000 7febd259c640 1 --2- 172.21.15.83:0/3792303363 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febd4150f20 0x7febd4155190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:23.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.580+0000 7febd259c640 1 -- 172.21.15.83:0/3792303363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febd40ce5c0 msgr2=0x7febd40cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:23.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.580+0000 7febd2d9d640 1 --2- 172.21.15.83:0/3792303363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febd40ce5c0 0x7febd40cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:23.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.580+0000 7febd259c640 1 --2- 172.21.15.83:0/3792303363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febd40ce5c0 0x7febd40cea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:23.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.580+0000 7febd259c640 1 -- 172.21.15.83:0/3792303363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd4150430 con 0x7febd4150050 2024-04-17T10:51:23.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.580+0000 7febd259c640 1 --2- 172.21.15.83:0/3792303363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febd4150050 0x7febd4152b20 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7febbc00e830 tx=0x7febbc00ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:23.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.584+0000 7febcaffd640 1 -- 172.21.15.83:0/3792303363 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc00c8d0 con 0x7febd4150050 2024-04-17T10:51:23.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.584+0000 7febcaffd640 1 -- 172.21.15.83:0/3792303363 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febbc018070 con 0x7febd4150050 2024-04-17T10:51:23.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.584+0000 7febcaffd640 1 -- 172.21.15.83:0/3792303363 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc00ccd0 con 0x7febd4150050 2024-04-17T10:51:23.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.584+0000 7febd8b14640 1 -- 172.21.15.83:0/3792303363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd4151450 con 0x7febd4150050 2024-04-17T10:51:23.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.584+0000 7febd8b14640 1 -- 172.21.15.83:0/3792303363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd4154990 con 0x7febd4150050 2024-04-17T10:51:23.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.584+0000 7febd8b14640 1 -- 172.21.15.83:0/3792303363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febd40cef80 con 0x7febd4150050 2024-04-17T10:51:23.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.584+0000 7febcaffd640 1 -- 172.21.15.83:0/3792303363 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7febbc013420 con 0x7febd4150050 2024-04-17T10:51:23.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.584+0000 7febcaffd640 1 --2- 172.21.15.83:0/3792303363 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7febb4066f90 0x7febb4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:23.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.584+0000 7febcaffd640 1 -- 172.21.15.83:0/3792303363 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7febbc084c80 con 0x7febd4150050 2024-04-17T10:51:23.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.584+0000 7febcbfff640 1 --2- 172.21.15.83:0/3792303363 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7febb4066f90 0x7febb4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:23.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.584+0000 7febcbfff640 1 --2- 172.21.15.83:0/3792303363 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7febb4066f90 0x7febb4069450 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7febc40af000 tx=0x7febc4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:23.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.588+0000 7febcaffd640 1 -- 172.21.15.83:0/3792303363 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7febbc055cf0 con 0x7febd4150050 2024-04-17T10:51:23.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.780+0000 7febd8b14640 1 -- 172.21.15.83:0/3792303363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febd41544a0 con 0x7febd4150050 2024-04-17T10:51:23.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.780+0000 7febcaffd640 1 -- 172.21.15.83:0/3792303363 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7febbc0555f0 con 0x7febd4150050 2024-04-17T10:51:23.774 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:51:23.774 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:51:23.774 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:51:23.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.784+0000 7febd8b14640 1 -- 172.21.15.83:0/3792303363 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7febb4066f90 msgr2=0x7febb4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:23.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.784+0000 7febd8b14640 1 --2- 172.21.15.83:0/3792303363 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7febb4066f90 0x7febb4069450 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7febc40af000 tx=0x7febc4008040 comp rx=0 tx=0).stop 2024-04-17T10:51:23.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.784+0000 7febd8b14640 1 -- 172.21.15.83:0/3792303363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febd4150050 msgr2=0x7febd4152b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:23.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.784+0000 7febd8b14640 1 --2- 172.21.15.83:0/3792303363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febd4150050 0x7febd4152b20 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7febbc00e830 tx=0x7febbc00ecf0 comp rx=0 tx=0).stop 2024-04-17T10:51:23.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.784+0000 7febd8b14640 1 -- 172.21.15.83:0/3792303363 shutdown_connections 2024-04-17T10:51:23.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.784+0000 7febd8b14640 1 --2- 172.21.15.83:0/3792303363 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7febb4066f90 0x7febb4069450 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:23.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.784+0000 7febd8b14640 1 --2- 172.21.15.83:0/3792303363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febd40ce5c0 0x7febd40cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:23.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.784+0000 7febd8b14640 1 --2- 172.21.15.83:0/3792303363 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febd4150f20 0x7febd4155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:23.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.784+0000 7febd8b14640 1 --2- 172.21.15.83:0/3792303363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febd4150050 0x7febd4152b20 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:23.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.784+0000 7febd8b14640 1 -- 172.21.15.83:0/3792303363 >> 172.21.15.83:0/3792303363 conn(0x7febd406abe0 msgr2=0x7febd406c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:23.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.784+0000 7febd8b14640 1 -- 172.21.15.83:0/3792303363 shutdown_connections 2024-04-17T10:51:23.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:23.784+0000 7febd8b14640 1 -- 172.21.15.83:0/3792303363 wait complete. 2024-04-17T10:51:24.195 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:51:24.195 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:51:25.196 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:51:25.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.335+0000 7f9b1dea3640 1 Processor -- start 2024-04-17T10:51:25.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.335+0000 7f9b1dea3640 1 -- start start 2024-04-17T10:51:25.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.335+0000 7f9b1dea3640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9b1815b250 0x7f9b180ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:25.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.335+0000 7f9b1dea3640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9b180713a0 0x7f9b180717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:25.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.335+0000 7f9b1dea3640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b1815ed20 con 0x7f9b1815e480 2024-04-17T10:51:25.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.335+0000 7f9b1dea3640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b18071d30 con 0x7f9b180713a0 2024-04-17T10:51:25.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.335+0000 7f9b1dea3640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b1815f000 con 0x7f9b1815b250 2024-04-17T10:51:25.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.335+0000 7f9b17fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9b180713a0 0x7f9b180717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.335+0000 7f9b16ffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9b1815e480 0x7f9b1815b630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46878/0 (socket says 172.21.15.83:46878) 2024-04-17T10:51:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.335+0000 7f9b16ffd640 1 -- 172.21.15.83:0/1621110762 learned_addr learned my addr 172.21.15.83:0/1621110762 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:51:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.335+0000 7f9b177fe640 1 --2- 172.21.15.83:0/1621110762 >> v2:172.21.15.167:3300/0 conn(0x7f9b1815b250 0x7f9b180ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.335+0000 7f9b17fff640 1 -- 172.21.15.83:0/1621110762 >> v2:172.21.15.167:3300/0 conn(0x7f9b1815b250 msgr2=0x7f9b180ce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.335+0000 7f9b167fc640 1 -- 172.21.15.83:0/1621110762 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2818258463 0 0) 0x7f9b1815ed20 con 0x7f9b1815e480 2024-04-17T10:51:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.335+0000 7f9b17fff640 1 --2- 172.21.15.83:0/1621110762 >> v2:172.21.15.167:3300/0 conn(0x7f9b1815b250 0x7f9b180ce1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.335+0000 7f9b17fff640 1 -- 172.21.15.83:0/1621110762 >> v1:172.21.15.83:6789/0 conn(0x7f9b1815e480 legacy=0x7f9b1815b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.335+0000 7f9b17fff640 1 -- 172.21.15.83:0/1621110762 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1815f2e0 con 0x7f9b180713a0 2024-04-17T10:51:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.335+0000 7f9b177fe640 1 --2- 172.21.15.83:0/1621110762 >> v2:172.21.15.167:3300/0 conn(0x7f9b1815b250 0x7f9b180ce1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:51:25.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.339+0000 7f9b17fff640 1 --2- 172.21.15.83:0/1621110762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b180713a0 0x7f9b180717f0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f9b18070a80 tx=0x7f9b1001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b68bae4844834 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:25.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.339+0000 7f9b167fc640 1 -- 172.21.15.83:0/1621110762 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b100b2070 con 0x7f9b180713a0 2024-04-17T10:51:25.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.339+0000 7f9b167fc640 1 -- 172.21.15.83:0/1621110762 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b100a58e0 con 0x7f9b180713a0 2024-04-17T10:51:25.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.339+0000 7f9b167fc640 1 -- 172.21.15.83:0/1621110762 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b100a5cb0 con 0x7f9b180713a0 2024-04-17T10:51:25.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.339+0000 7f9b1dea3640 1 -- 172.21.15.83:0/1621110762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b180713a0 msgr2=0x7f9b180717f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:25.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.339+0000 7f9b1dea3640 1 --2- 172.21.15.83:0/1621110762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b180713a0 0x7f9b180717f0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f9b18070a80 tx=0x7f9b1001c930 comp rx=0 tx=0).stop 2024-04-17T10:51:25.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.339+0000 7f9b1dea3640 1 -- 172.21.15.83:0/1621110762 shutdown_connections 2024-04-17T10:51:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.339+0000 7f9b1dea3640 1 --2- 172.21.15.83:0/1621110762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b180713a0 0x7f9b180717f0 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.339+0000 7f9b1dea3640 1 --2- 172.21.15.83:0/1621110762 >> v2:172.21.15.167:3300/0 conn(0x7f9b1815b250 0x7f9b180ce1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.339+0000 7f9b1dea3640 1 -- 172.21.15.83:0/1621110762 >> 172.21.15.83:0/1621110762 conn(0x7f9b1806bcf0 msgr2=0x7f9b1806c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.339+0000 7f9b1dea3640 1 -- 172.21.15.83:0/1621110762 shutdown_connections 2024-04-17T10:51:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.339+0000 7f9b1dea3640 1 -- 172.21.15.83:0/1621110762 wait complete. 2024-04-17T10:51:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.339+0000 7f9b1dea3640 1 Processor -- start 2024-04-17T10:51:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.339+0000 7f9b1dea3640 1 -- start start 2024-04-17T10:51:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.339+0000 7f9b1dea3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b180713a0 0x7f9b181585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.339+0000 7f9b1dea3640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b1815b250 0x7f9b18149b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.339+0000 7f9b1dea3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b1815e480 0x7f9b1814a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.339+0000 7f9b1dea3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1815e8d0 con 0x7f9b180713a0 2024-04-17T10:51:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.339+0000 7f9b1dea3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1815bd40 con 0x7f9b1815e480 2024-04-17T10:51:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.339+0000 7f9b1dea3640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1815ea50 con 0x7f9b1815b250 2024-04-17T10:51:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.339+0000 7f9b177fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b180713a0 0x7f9b181585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.339+0000 7f9b177fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b180713a0 0x7f9b181585a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58550/0 (socket says 172.21.15.83:58550) 2024-04-17T10:51:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.339+0000 7f9b177fe640 1 -- 172.21.15.83:0/265504547 learned_addr learned my addr 172.21.15.83:0/265504547 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.339+0000 7f9b17fff640 1 --2- 172.21.15.83:0/265504547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b1815e480 0x7f9b1814a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.339+0000 7f9b16ffd640 1 --2- 172.21.15.83:0/265504547 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b1815b250 0x7f9b18149b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.339+0000 7f9b177fe640 1 -- 172.21.15.83:0/265504547 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b1815b250 msgr2=0x7f9b18149b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.339+0000 7f9b177fe640 1 --2- 172.21.15.83:0/265504547 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b1815b250 0x7f9b18149b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.339+0000 7f9b177fe640 1 -- 172.21.15.83:0/265504547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b1815e480 msgr2=0x7f9b1814a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.339+0000 7f9b177fe640 1 --2- 172.21.15.83:0/265504547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b1815e480 0x7f9b1814a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.339+0000 7f9b177fe640 1 -- 172.21.15.83:0/265504547 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b18159af0 con 0x7f9b180713a0 2024-04-17T10:51:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.339+0000 7f9b16ffd640 1 --2- 172.21.15.83:0/265504547 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b1815b250 0x7f9b18149b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:51:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.339+0000 7f9b177fe640 1 --2- 172.21.15.83:0/265504547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b180713a0 0x7f9b181585a0 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800c7c0 tx=0x7f9b0800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.343+0000 7f9afffff640 1 -- 172.21.15.83:0/265504547 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b08007a30 con 0x7f9b180713a0 2024-04-17T10:51:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.343+0000 7f9afffff640 1 -- 172.21.15.83:0/265504547 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b08007bd0 con 0x7f9b180713a0 2024-04-17T10:51:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.343+0000 7f9b1dea3640 1 -- 172.21.15.83:0/265504547 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b180b18d0 con 0x7f9b180713a0 2024-04-17T10:51:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.343+0000 7f9afffff640 1 -- 172.21.15.83:0/265504547 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b080113f0 con 0x7f9b180713a0 2024-04-17T10:51:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.343+0000 7f9b1dea3640 1 -- 172.21.15.83:0/265504547 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b181592f0 con 0x7f9b180713a0 2024-04-17T10:51:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.343+0000 7f9b1dea3640 1 -- 172.21.15.83:0/265504547 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b18074f90 con 0x7f9b180713a0 2024-04-17T10:51:25.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.343+0000 7f9afffff640 1 -- 172.21.15.83:0/265504547 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9b08011590 con 0x7f9b180713a0 2024-04-17T10:51:25.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.343+0000 7f9afffff640 1 --2- 172.21.15.83:0/265504547 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9b04066f60 0x7f9b04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:25.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.343+0000 7f9b16ffd640 1 --2- 172.21.15.83:0/265504547 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9b04066f60 0x7f9b04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:25.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.343+0000 7f9afffff640 1 -- 172.21.15.83:0/265504547 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f9b08085740 con 0x7f9b180713a0 2024-04-17T10:51:25.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.343+0000 7f9b16ffd640 1 --2- 172.21.15.83:0/265504547 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9b04066f60 0x7f9b04069420 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c007fd0 tx=0x7f9b0c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:25.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.347+0000 7f9afffff640 1 -- 172.21.15.83:0/265504547 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9b08015020 con 0x7f9b180713a0 2024-04-17T10:51:25.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.539+0000 7f9b1dea3640 1 -- 172.21.15.83:0/265504547 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b18159f50 con 0x7f9b180713a0 2024-04-17T10:51:25.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.543+0000 7f9afffff640 1 -- 172.21.15.83:0/265504547 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f9b080566e0 con 0x7f9b180713a0 2024-04-17T10:51:25.535 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:51:25.535 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:51:25.535 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:51:25.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.543+0000 7f9b1dea3640 1 -- 172.21.15.83:0/265504547 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9b04066f60 msgr2=0x7f9b04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:25.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.543+0000 7f9b1dea3640 1 --2- 172.21.15.83:0/265504547 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9b04066f60 0x7f9b04069420 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c007fd0 tx=0x7f9b0c009040 comp rx=0 tx=0).stop 2024-04-17T10:51:25.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.543+0000 7f9b1dea3640 1 -- 172.21.15.83:0/265504547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b180713a0 msgr2=0x7f9b181585a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:25.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.543+0000 7f9b1dea3640 1 --2- 172.21.15.83:0/265504547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b180713a0 0x7f9b181585a0 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800c7c0 tx=0x7f9b0800cc80 comp rx=0 tx=0).stop 2024-04-17T10:51:25.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.543+0000 7f9b1dea3640 1 -- 172.21.15.83:0/265504547 shutdown_connections 2024-04-17T10:51:25.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.543+0000 7f9b1dea3640 1 --2- 172.21.15.83:0/265504547 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9b04066f60 0x7f9b04069420 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:25.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.543+0000 7f9b1dea3640 1 --2- 172.21.15.83:0/265504547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b1815e480 0x7f9b1814a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:25.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.543+0000 7f9b1dea3640 1 --2- 172.21.15.83:0/265504547 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b1815b250 0x7f9b18149b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:25.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.543+0000 7f9b1dea3640 1 --2- 172.21.15.83:0/265504547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b180713a0 0x7f9b181585a0 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:25.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.543+0000 7f9b1dea3640 1 -- 172.21.15.83:0/265504547 >> 172.21.15.83:0/265504547 conn(0x7f9b1806bcf0 msgr2=0x7f9b1815b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:25.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.543+0000 7f9b1dea3640 1 -- 172.21.15.83:0/265504547 shutdown_connections 2024-04-17T10:51:25.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:25.543+0000 7f9b1dea3640 1 -- 172.21.15.83:0/265504547 wait complete. 2024-04-17T10:51:25.957 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:51:25.957 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:51:26.958 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:51:27.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f6558c3f640 1 Processor -- start 2024-04-17T10:51:27.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f6558c3f640 1 -- start start 2024-04-17T10:51:27.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f6558c3f640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f65540b2d80 0x7f6554146e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:27.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f6558c3f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f655406d690 0x7f65541620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:27.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f6558c3f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6554147570 con 0x7f6554146a00 2024-04-17T10:51:27.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f6558c3f640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6554162610 con 0x7f65540b2d80 2024-04-17T10:51:27.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f6558c3f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f655406c960 con 0x7f655406d690 2024-04-17T10:51:27.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f655259c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6554146a00 0x7f65540b2670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47884/0 (socket says 172.21.15.83:47884) 2024-04-17T10:51:27.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f655259c640 1 -- 172.21.15.83:0/2345792609 learned_addr learned my addr 172.21.15.83:0/2345792609 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:51:27.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f6551d9b640 1 -- 172.21.15.83:0/2345792609 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3551455728 0 0) 0x7f6554147570 con 0x7f6554146a00 2024-04-17T10:51:27.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f6551d9b640 1 -- 172.21.15.83:0/2345792609 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6540003530 con 0x7f6554146a00 2024-04-17T10:51:27.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f6549d9b640 1 --2- 172.21.15.83:0/2345792609 >> v2:172.21.15.167:3300/0 conn(0x7f65540b2d80 0x7f6554146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:27.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f6552d9d640 1 --2- 172.21.15.83:0/2345792609 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f655406d690 0x7f65541620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:27.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f6551d9b640 1 -- 172.21.15.83:0/2345792609 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1220608151 0 0) 0x7f6540003530 con 0x7f6554146a00 2024-04-17T10:51:27.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f6551d9b640 1 -- 172.21.15.83:0/2345792609 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f655406d690 msgr2=0x7f65541620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:27.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f6551d9b640 1 --2- 172.21.15.83:0/2345792609 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f655406d690 0x7f65541620d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:27.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f6551d9b640 1 -- 172.21.15.83:0/2345792609 >> v2:172.21.15.167:3300/0 conn(0x7f65540b2d80 msgr2=0x7f6554146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:27.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f6551d9b640 1 --2- 172.21.15.83:0/2345792609 >> v2:172.21.15.167:3300/0 conn(0x7f65540b2d80 0x7f6554146e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:27.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f6551d9b640 1 -- 172.21.15.83:0/2345792609 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655406cbb0 con 0x7f6554146a00 2024-04-17T10:51:27.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f6551d9b640 1 -- 172.21.15.83:0/2345792609 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f6544001680 con 0x7f6554146a00 2024-04-17T10:51:27.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f6551d9b640 1 -- 172.21.15.83:0/2345792609 >> v1:172.21.15.83:6789/0 conn(0x7f6554146a00 legacy=0x7f65540b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:27.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f6551d9b640 1 --2- 172.21.15.83:0/2345792609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65400039c0 0x7f6540003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:27.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f6551d9b640 1 --2- 172.21.15.83:0/2345792609 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f655406d690 0x7f6540004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:27.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f6551d9b640 1 --2- 172.21.15.83:0/2345792609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6540004d50 0x7f65400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:27.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f6551d9b640 1 -- 172.21.15.83:0/2345792609 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655406c960 con 0x7f6540004d50 2024-04-17T10:51:27.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f6551d9b640 1 -- 172.21.15.83:0/2345792609 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654000ba80 con 0x7f65400039c0 2024-04-17T10:51:27.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f6551d9b640 1 -- 172.21.15.83:0/2345792609 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f654000bc30 con 0x7f655406d690 2024-04-17T10:51:27.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f6552d9d640 1 --2- 172.21.15.83:0/2345792609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6540004d50 0x7f65400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:27.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f655259c640 1 --2- 172.21.15.83:0/2345792609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65400039c0 0x7f6540003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:27.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f6549d9b640 1 --2- 172.21.15.83:0/2345792609 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f655406d690 0x7f6540004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:27.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f6552d9d640 1 -- 172.21.15.83:0/2345792609 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f655406d690 msgr2=0x7f6540004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:27.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f6552d9d640 1 --2- 172.21.15.83:0/2345792609 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f655406d690 0x7f6540004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:27.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f6552d9d640 1 -- 172.21.15.83:0/2345792609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65400039c0 msgr2=0x7f6540003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:27.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f6552d9d640 1 --2- 172.21.15.83:0/2345792609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65400039c0 0x7f6540003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:27.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.103+0000 7f6552d9d640 1 -- 172.21.15.83:0/2345792609 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6540003530 con 0x7f6540004d50 2024-04-17T10:51:27.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.107+0000 7f6552d9d640 1 --2- 172.21.15.83:0/2345792609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6540004d50 0x7f65400051d0 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f654c007360 tx=0x7f654c01cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba82bc9f5052f11c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:27.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.107+0000 7f6551d9b640 1 -- 172.21.15.83:0/2345792609 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c0b2070 con 0x7f6540004d50 2024-04-17T10:51:27.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.107+0000 7f6551d9b640 1 -- 172.21.15.83:0/2345792609 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654c0a5d40 con 0x7f6540004d50 2024-04-17T10:51:27.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.107+0000 7f6551d9b640 1 -- 172.21.15.83:0/2345792609 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c009be0 con 0x7f6540004d50 2024-04-17T10:51:27.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.107+0000 7f6558c3f640 1 -- 172.21.15.83:0/2345792609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6540004d50 msgr2=0x7f65400051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:27.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.107+0000 7f6558c3f640 1 --2- 172.21.15.83:0/2345792609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6540004d50 0x7f65400051d0 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f654c007360 tx=0x7f654c01cde0 comp rx=0 tx=0).stop 2024-04-17T10:51:27.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.107+0000 7f6552d9d640 1 -- 172.21.15.83:0/2345792609 reap_dead start 2024-04-17T10:51:27.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.107+0000 7f6558c3f640 1 -- 172.21.15.83:0/2345792609 shutdown_connections 2024-04-17T10:51:27.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.107+0000 7f6558c3f640 1 -- 172.21.15.83:0/2345792609 >> 172.21.15.83:0/2345792609 conn(0x7f655406abe0 msgr2=0x7f655406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:27.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.107+0000 7f6558c3f640 1 -- 172.21.15.83:0/2345792609 shutdown_connections 2024-04-17T10:51:27.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.107+0000 7f6558c3f640 1 -- 172.21.15.83:0/2345792609 wait complete. 2024-04-17T10:51:27.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.107+0000 7f6558c3f640 1 Processor -- start 2024-04-17T10:51:27.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.107+0000 7f6558c3f640 1 -- start start 2024-04-17T10:51:27.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.107+0000 7f6558c3f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6540004d50 0x7f65540c45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:27.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.107+0000 7f6558c3f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65541f7240 0x7f655413f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:27.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.107+0000 7f6558c3f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6554140df0 0x7f6554141270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:27.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.107+0000 7f6558c3f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655406db40 con 0x7f6540004d50 2024-04-17T10:51:27.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.107+0000 7f6558c3f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655406d200 con 0x7f65541f7240 2024-04-17T10:51:27.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.107+0000 7f6558c3f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65541620d0 con 0x7f6554140df0 2024-04-17T10:51:27.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.111+0000 7f655259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6540004d50 0x7f65540c45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:27.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.111+0000 7f655259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6540004d50 0x7f65540c45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37854/0 (socket says 172.21.15.83:37854) 2024-04-17T10:51:27.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.111+0000 7f655259c640 1 -- 172.21.15.83:0/1819576401 learned_addr learned my addr 172.21.15.83:0/1819576401 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:27.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.111+0000 7f6549d9b640 1 --2- 172.21.15.83:0/1819576401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65541f7240 0x7f655413f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:27.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.111+0000 7f6552d9d640 1 --2- 172.21.15.83:0/1819576401 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6554140df0 0x7f6554141270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:27.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.111+0000 7f655259c640 1 -- 172.21.15.83:0/1819576401 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6554140df0 msgr2=0x7f6554141270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:27.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.111+0000 7f655259c640 1 --2- 172.21.15.83:0/1819576401 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6554140df0 0x7f6554141270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:27.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.111+0000 7f655259c640 1 -- 172.21.15.83:0/1819576401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65541f7240 msgr2=0x7f655413f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:27.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.111+0000 7f655259c640 1 --2- 172.21.15.83:0/1819576401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65541f7240 0x7f655413f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:27.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.111+0000 7f655259c640 1 -- 172.21.15.83:0/1819576401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65540c4ca0 con 0x7f6540004d50 2024-04-17T10:51:27.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.111+0000 7f6552d9d640 1 --2- 172.21.15.83:0/1819576401 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6554140df0 0x7f6554141270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:51:27.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.111+0000 7f6549d9b640 1 --2- 172.21.15.83:0/1819576401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65541f7240 0x7f655413f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:51:27.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.111+0000 7f655259c640 1 --2- 172.21.15.83:0/1819576401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6540004d50 0x7f65540c45e0 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f654400e820 tx=0x7f654400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:27.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.111+0000 7f654b7fe640 1 -- 172.21.15.83:0/1819576401 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654400c870 con 0x7f6540004d50 2024-04-17T10:51:27.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.111+0000 7f654b7fe640 1 -- 172.21.15.83:0/1819576401 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6544018070 con 0x7f6540004d50 2024-04-17T10:51:27.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.111+0000 7f6558c3f640 1 -- 172.21.15.83:0/1819576401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65541f44a0 con 0x7f6540004d50 2024-04-17T10:51:27.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.111+0000 7f654b7fe640 1 -- 172.21.15.83:0/1819576401 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654400cc30 con 0x7f6540004d50 2024-04-17T10:51:27.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.111+0000 7f6558c3f640 1 -- 172.21.15.83:0/1819576401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65541f4740 con 0x7f6540004d50 2024-04-17T10:51:27.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.111+0000 7f6558c3f640 1 -- 172.21.15.83:0/1819576401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6554074f90 con 0x7f6540004d50 2024-04-17T10:51:27.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.111+0000 7f654b7fe640 1 -- 172.21.15.83:0/1819576401 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6544013420 con 0x7f6540004d50 2024-04-17T10:51:27.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.111+0000 7f654b7fe640 1 --2- 172.21.15.83:0/1819576401 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6534066f90 0x7f6534069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:27.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.111+0000 7f654b7fe640 1 -- 172.21.15.83:0/1819576401 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f6544084bc0 con 0x7f6540004d50 2024-04-17T10:51:27.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.111+0000 7f6549d9b640 1 --2- 172.21.15.83:0/1819576401 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6534066f90 0x7f6534069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:27.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.111+0000 7f6549d9b640 1 --2- 172.21.15.83:0/1819576401 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6534066f90 0x7f6534069450 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f65540b2310 tx=0x7f653c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:27.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.115+0000 7f654b7fe640 1 -- 172.21.15.83:0/1819576401 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6544055b60 con 0x7f6540004d50 2024-04-17T10:51:27.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.303+0000 7f6558c3f640 1 -- 172.21.15.83:0/1819576401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65540b2470 con 0x7f6540004d50 2024-04-17T10:51:27.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.303+0000 7f654b7fe640 1 -- 172.21.15.83:0/1819576401 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f6544055460 con 0x7f6540004d50 2024-04-17T10:51:27.299 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:51:27.299 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:51:27.299 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:51:27.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.307+0000 7f6558c3f640 1 -- 172.21.15.83:0/1819576401 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6534066f90 msgr2=0x7f6534069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:27.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.307+0000 7f6558c3f640 1 --2- 172.21.15.83:0/1819576401 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6534066f90 0x7f6534069450 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f65540b2310 tx=0x7f653c008040 comp rx=0 tx=0).stop 2024-04-17T10:51:27.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.307+0000 7f6558c3f640 1 -- 172.21.15.83:0/1819576401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6540004d50 msgr2=0x7f65540c45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:27.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.307+0000 7f6558c3f640 1 --2- 172.21.15.83:0/1819576401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6540004d50 0x7f65540c45e0 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f654400e820 tx=0x7f654400ece0 comp rx=0 tx=0).stop 2024-04-17T10:51:27.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.307+0000 7f6558c3f640 1 -- 172.21.15.83:0/1819576401 shutdown_connections 2024-04-17T10:51:27.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.307+0000 7f6558c3f640 1 --2- 172.21.15.83:0/1819576401 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6534066f90 0x7f6534069450 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:27.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.307+0000 7f6558c3f640 1 --2- 172.21.15.83:0/1819576401 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6554140df0 0x7f6554141270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:27.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.307+0000 7f6558c3f640 1 --2- 172.21.15.83:0/1819576401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65541f7240 0x7f655413f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:27.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.307+0000 7f6558c3f640 1 --2- 172.21.15.83:0/1819576401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6540004d50 0x7f65540c45e0 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:27.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.307+0000 7f6558c3f640 1 -- 172.21.15.83:0/1819576401 >> 172.21.15.83:0/1819576401 conn(0x7f655406abe0 msgr2=0x7f6554160900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:27.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.307+0000 7f6558c3f640 1 -- 172.21.15.83:0/1819576401 shutdown_connections 2024-04-17T10:51:27.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:27.307+0000 7f6558c3f640 1 -- 172.21.15.83:0/1819576401 wait complete. 2024-04-17T10:51:27.718 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:51:27.718 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:51:28.720 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:51:28.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.859+0000 7f60af360640 1 Processor -- start 2024-04-17T10:51:28.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.859+0000 7f60af360640 1 -- start start 2024-04-17T10:51:28.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.859+0000 7f60af360640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60a815ba30 0x7f60a8072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:28.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.859+0000 7f60af360640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f60a80cfbe0 0x7f60a806c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:28.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.859+0000 7f60af360640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60a806e320 con 0x7f60a815c960 2024-04-17T10:51:28.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.859+0000 7f60af360640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60a8071ea0 con 0x7f60a80cfbe0 2024-04-17T10:51:28.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.859+0000 7f60af360640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60a806e040 con 0x7f60a815ba30 2024-04-17T10:51:28.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.859+0000 7f60ad8fd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f60a80cfbe0 0x7f60a806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:28.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.859+0000 7f60ac8fb640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f60a815c960 0x7f60a80cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:57330/0 (socket says 172.21.15.83:57330) 2024-04-17T10:51:28.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.859+0000 7f60ad0fc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60a815ba30 0x7f60a8072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:28.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.859+0000 7f60ac8fb640 1 -- 172.21.15.83:0/493466972 learned_addr learned my addr 172.21.15.83:0/493466972 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:51:28.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.859+0000 7f60ad8fd640 1 -- 172.21.15.83:0/493466972 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60a815ba30 msgr2=0x7f60a8072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:28.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.859+0000 7f60ad8fd640 1 --2- 172.21.15.83:0/493466972 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60a815ba30 0x7f60a8072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:28.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.859+0000 7f60ad8fd640 1 -- 172.21.15.83:0/493466972 >> v1:172.21.15.167:6789/0 conn(0x7f60a815c960 legacy=0x7f60a80cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:28.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.859+0000 7f60ad8fd640 1 -- 172.21.15.83:0/493466972 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a8071850 con 0x7f60a80cfbe0 2024-04-17T10:51:28.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.859+0000 7f60ad0fc640 1 --2- 172.21.15.83:0/493466972 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60a815ba30 0x7f60a8072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:51:28.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.863+0000 7f60ad8fd640 1 --2- 172.21.15.83:0/493466972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60a80cfbe0 0x7f60a806c490 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f60a00097c0 tx=0x7f60a001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb0cbda396bc8a77 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:28.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.863+0000 7f60a7fff640 1 -- 172.21.15.83:0/493466972 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a00b0070 con 0x7f60a80cfbe0 2024-04-17T10:51:28.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60a7fff640 1 -- 172.21.15.83:0/493466972 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60a00ab600 con 0x7f60a80cfbe0 2024-04-17T10:51:28.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60a7fff640 1 -- 172.21.15.83:0/493466972 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a00ab9d0 con 0x7f60a80cfbe0 2024-04-17T10:51:28.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60af360640 1 -- 172.21.15.83:0/493466972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60a80cfbe0 msgr2=0x7f60a806c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:28.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60af360640 1 --2- 172.21.15.83:0/493466972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60a80cfbe0 0x7f60a806c490 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f60a00097c0 tx=0x7f60a001c410 comp rx=0 tx=0).stop 2024-04-17T10:51:28.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60af360640 1 -- 172.21.15.83:0/493466972 shutdown_connections 2024-04-17T10:51:28.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60af360640 1 --2- 172.21.15.83:0/493466972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60a80cfbe0 0x7f60a806c490 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:28.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60af360640 1 --2- 172.21.15.83:0/493466972 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60a815ba30 0x7f60a8072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:28.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60af360640 1 -- 172.21.15.83:0/493466972 >> 172.21.15.83:0/493466972 conn(0x7f60a806abe0 msgr2=0x7f60a806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:28.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60af360640 1 -- 172.21.15.83:0/493466972 shutdown_connections 2024-04-17T10:51:28.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60af360640 1 -- 172.21.15.83:0/493466972 wait complete. 2024-04-17T10:51:28.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60af360640 1 Processor -- start 2024-04-17T10:51:28.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60af360640 1 -- start start 2024-04-17T10:51:28.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60af360640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60a80cfbe0 0x7f60a8158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:28.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60af360640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60a815ba30 0x7f60a814a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:28.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60af360640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60a815c960 0x7f60a81568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:28.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60af360640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a815cee0 con 0x7f60a815ba30 2024-04-17T10:51:28.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60af360640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a815cd60 con 0x7f60a815c960 2024-04-17T10:51:28.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60af360640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60a8168520 con 0x7f60a80cfbe0 2024-04-17T10:51:28.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60ac8fb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60a815ba30 0x7f60a814a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:28.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60ac8fb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60a815ba30 0x7f60a814a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37862/0 (socket says 172.21.15.83:37862) 2024-04-17T10:51:28.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60ac8fb640 1 -- 172.21.15.83:0/4066949695 learned_addr learned my addr 172.21.15.83:0/4066949695 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:28.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60ad0fc640 1 --2- 172.21.15.83:0/4066949695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60a80cfbe0 0x7f60a8158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:28.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60ad8fd640 1 --2- 172.21.15.83:0/4066949695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60a815c960 0x7f60a81568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:28.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60ac8fb640 1 -- 172.21.15.83:0/4066949695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60a80cfbe0 msgr2=0x7f60a8158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:28.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60ac8fb640 1 --2- 172.21.15.83:0/4066949695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60a80cfbe0 0x7f60a8158a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:28.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60ac8fb640 1 -- 172.21.15.83:0/4066949695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60a815c960 msgr2=0x7f60a81568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:28.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60ac8fb640 1 --2- 172.21.15.83:0/4066949695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60a815c960 0x7f60a81568d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:28.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60ac8fb640 1 -- 172.21.15.83:0/4066949695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a8158f70 con 0x7f60a815ba30 2024-04-17T10:51:28.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60ac8fb640 1 --2- 172.21.15.83:0/4066949695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60a815ba30 0x7f60a814a260 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f609c007fd0 tx=0x7f609c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:28.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60a5ffb640 1 -- 172.21.15.83:0/4066949695 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c017070 con 0x7f60a815ba30 2024-04-17T10:51:28.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60a5ffb640 1 -- 172.21.15.83:0/4066949695 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f609c005d10 con 0x7f60a815ba30 2024-04-17T10:51:28.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60af360640 1 -- 172.21.15.83:0/4066949695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60a8159240 con 0x7f60a815ba30 2024-04-17T10:51:28.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60a5ffb640 1 -- 172.21.15.83:0/4066949695 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c007310 con 0x7f60a815ba30 2024-04-17T10:51:28.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.867+0000 7f60af360640 1 -- 172.21.15.83:0/4066949695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60a8159800 con 0x7f60a815ba30 2024-04-17T10:51:28.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.871+0000 7f60af360640 1 -- 172.21.15.83:0/4066949695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60a8074f90 con 0x7f60a815ba30 2024-04-17T10:51:28.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.871+0000 7f60a5ffb640 1 -- 172.21.15.83:0/4066949695 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f609c01a070 con 0x7f60a815ba30 2024-04-17T10:51:28.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.871+0000 7f60a5ffb640 1 --2- 172.21.15.83:0/4066949695 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6094066f90 0x7f6094069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:28.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.871+0000 7f60a5ffb640 1 -- 172.21.15.83:0/4066949695 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f609c084530 con 0x7f60a815ba30 2024-04-17T10:51:28.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.871+0000 7f60ad0fc640 1 --2- 172.21.15.83:0/4066949695 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6094066f90 0x7f6094069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:28.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.871+0000 7f60ad0fc640 1 --2- 172.21.15.83:0/4066949695 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6094066f90 0x7f6094069450 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6098009bd0 tx=0x7f609800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:28.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:28.875+0000 7f60a5ffb640 1 -- 172.21.15.83:0/4066949695 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f609c0553c0 con 0x7f60a815ba30 2024-04-17T10:51:29.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:29.051+0000 7f60af360640 1 -- 172.21.15.83:0/4066949695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60a8072330 con 0x7f60a815ba30 2024-04-17T10:51:29.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:29.055+0000 7f60a5ffb640 1 -- 172.21.15.83:0/4066949695 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f609c054cc0 con 0x7f60a815ba30 2024-04-17T10:51:29.048 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:51:29.048 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:51:29.048 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:51:29.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:29.055+0000 7f60af360640 1 -- 172.21.15.83:0/4066949695 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6094066f90 msgr2=0x7f6094069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:29.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:29.055+0000 7f60af360640 1 --2- 172.21.15.83:0/4066949695 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6094066f90 0x7f6094069450 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6098009bd0 tx=0x7f609800b040 comp rx=0 tx=0).stop 2024-04-17T10:51:29.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:29.055+0000 7f60af360640 1 -- 172.21.15.83:0/4066949695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60a815ba30 msgr2=0x7f60a814a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:29.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:29.055+0000 7f60af360640 1 --2- 172.21.15.83:0/4066949695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60a815ba30 0x7f60a814a260 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f609c007fd0 tx=0x7f609c00c890 comp rx=0 tx=0).stop 2024-04-17T10:51:29.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:29.055+0000 7f60af360640 1 -- 172.21.15.83:0/4066949695 shutdown_connections 2024-04-17T10:51:29.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:29.055+0000 7f60af360640 1 --2- 172.21.15.83:0/4066949695 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6094066f90 0x7f6094069450 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:29.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:29.055+0000 7f60af360640 1 --2- 172.21.15.83:0/4066949695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60a815c960 0x7f60a81568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:29.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:29.055+0000 7f60af360640 1 --2- 172.21.15.83:0/4066949695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60a815ba30 0x7f60a814a260 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:29.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:29.055+0000 7f60af360640 1 --2- 172.21.15.83:0/4066949695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60a80cfbe0 0x7f60a8158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:29.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:29.055+0000 7f60af360640 1 -- 172.21.15.83:0/4066949695 >> 172.21.15.83:0/4066949695 conn(0x7f60a806abe0 msgr2=0x7f60a80b1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:29.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:29.055+0000 7f60af360640 1 -- 172.21.15.83:0/4066949695 shutdown_connections 2024-04-17T10:51:29.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:29.055+0000 7f60af360640 1 -- 172.21.15.83:0/4066949695 wait complete. 2024-04-17T10:51:29.468 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:51:29.468 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:51:30.469 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:51:30.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.607+0000 7f5259b71640 1 Processor -- start 2024-04-17T10:51:30.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.607+0000 7f5259b71640 1 -- start start 2024-04-17T10:51:30.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.607+0000 7f5259b71640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f525415ba30 0x7f5254072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:30.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.607+0000 7f5259b71640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f525415c960 0x7f52540cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:30.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.607+0000 7f5259b71640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52540cfea0 con 0x7f52540cfa10 2024-04-17T10:51:30.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.607+0000 7f5259b71640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5254071ea0 con 0x7f525415ba30 2024-04-17T10:51:30.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.607+0000 7f5259b71640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f525406e320 con 0x7f525415c960 2024-04-17T10:51:30.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.607+0000 7f52537fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f525415ba30 0x7f5254072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:30.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.607+0000 7f5253fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f52540cfa10 0x7f525406c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47894/0 (socket says 172.21.15.83:47894) 2024-04-17T10:51:30.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.607+0000 7f5253fff640 1 -- 172.21.15.83:0/1693026038 learned_addr learned my addr 172.21.15.83:0/1693026038 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:51:30.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.607+0000 7f52537fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f525415ba30 0x7f5254072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37868/0 (socket says 172.21.15.83:37868) 2024-04-17T10:51:30.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.607+0000 7f52537fe640 1 -- 172.21.15.83:0/1693026038 >> v2:172.21.15.167:3300/0 conn(0x7f525415c960 msgr2=0x7f52540cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:30.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.607+0000 7f524affd640 1 --2- 172.21.15.83:0/1693026038 >> v2:172.21.15.167:3300/0 conn(0x7f525415c960 0x7f52540cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:30.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.607+0000 7f52537fe640 1 --2- 172.21.15.83:0/1693026038 >> v2:172.21.15.167:3300/0 conn(0x7f525415c960 0x7f52540cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:30.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.607+0000 7f5252ffd640 1 -- 172.21.15.83:0/1693026038 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2362440770 0 0) 0x7f52540cfea0 con 0x7f52540cfa10 2024-04-17T10:51:30.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.607+0000 7f52537fe640 1 -- 172.21.15.83:0/1693026038 >> v1:172.21.15.83:6789/0 conn(0x7f52540cfa10 legacy=0x7f525406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:30.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.607+0000 7f52537fe640 1 -- 172.21.15.83:0/1693026038 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525406e040 con 0x7f525415ba30 2024-04-17T10:51:30.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.607+0000 7f52537fe640 1 --2- 172.21.15.83:0/1693026038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f525415ba30 0x7f5254072330 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f52540af2b0 tx=0x7f524401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85b9f17ace915d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:30.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f5252ffd640 1 -- 172.21.15.83:0/1693026038 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52440b0070 con 0x7f525415ba30 2024-04-17T10:51:30.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f5252ffd640 1 -- 172.21.15.83:0/1693026038 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52440ab600 con 0x7f525415ba30 2024-04-17T10:51:30.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f5252ffd640 1 -- 172.21.15.83:0/1693026038 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52440ab9d0 con 0x7f525415ba30 2024-04-17T10:51:30.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f5259b71640 1 -- 172.21.15.83:0/1693026038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f525415ba30 msgr2=0x7f5254072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:30.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f5259b71640 1 --2- 172.21.15.83:0/1693026038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f525415ba30 0x7f5254072330 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f52540af2b0 tx=0x7f524401c3f0 comp rx=0 tx=0).stop 2024-04-17T10:51:30.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f5259b71640 1 -- 172.21.15.83:0/1693026038 shutdown_connections 2024-04-17T10:51:30.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f5259b71640 1 --2- 172.21.15.83:0/1693026038 >> v2:172.21.15.167:3300/0 conn(0x7f525415c960 0x7f52540cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:30.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f5259b71640 1 --2- 172.21.15.83:0/1693026038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f525415ba30 0x7f5254072330 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:30.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f5259b71640 1 -- 172.21.15.83:0/1693026038 >> 172.21.15.83:0/1693026038 conn(0x7f525406abe0 msgr2=0x7f525406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:30.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f5259b71640 1 -- 172.21.15.83:0/1693026038 shutdown_connections 2024-04-17T10:51:30.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f5259b71640 1 -- 172.21.15.83:0/1693026038 wait complete. 2024-04-17T10:51:30.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f5259b71640 1 Processor -- start 2024-04-17T10:51:30.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f5259b71640 1 -- start start 2024-04-17T10:51:30.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f5259b71640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52540cfa10 0x7f5254151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:30.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f5259b71640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f525415ba30 0x7f52541459f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:30.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f5259b71640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f525415c960 0x7f52540ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:30.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f5259b71640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525406de20 con 0x7f52540cfa10 2024-04-17T10:51:30.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f5259b71640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525406dca0 con 0x7f525415ba30 2024-04-17T10:51:30.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f5259b71640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f525415cd40 con 0x7f525415c960 2024-04-17T10:51:30.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f52537fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52540cfa10 0x7f5254151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:30.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f52537fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52540cfa10 0x7f5254151390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37880/0 (socket says 172.21.15.83:37880) 2024-04-17T10:51:30.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f52537fe640 1 -- 172.21.15.83:0/1274511662 learned_addr learned my addr 172.21.15.83:0/1274511662 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:30.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f52537fe640 1 -- 172.21.15.83:0/1274511662 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f525415c960 msgr2=0x7f52540ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:30.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f52537fe640 1 --2- 172.21.15.83:0/1274511662 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f525415c960 0x7f52540ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:30.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f52537fe640 1 -- 172.21.15.83:0/1274511662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f525415ba30 msgr2=0x7f52541459f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:30.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f52537fe640 1 --2- 172.21.15.83:0/1274511662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f525415ba30 0x7f52541459f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:30.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f52537fe640 1 -- 172.21.15.83:0/1274511662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52540b47b0 con 0x7f52540cfa10 2024-04-17T10:51:30.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f52537fe640 1 --2- 172.21.15.83:0/1274511662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52540cfa10 0x7f5254151390 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f524401a020 tx=0x7f5244019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:30.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f5250ff9640 1 -- 172.21.15.83:0/1274511662 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52440b0050 con 0x7f52540cfa10 2024-04-17T10:51:30.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f5250ff9640 1 -- 172.21.15.83:0/1274511662 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52440bc070 con 0x7f52540cfa10 2024-04-17T10:51:30.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f5259b71640 1 -- 172.21.15.83:0/1274511662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52540b3c30 con 0x7f52540cfa10 2024-04-17T10:51:30.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f5250ff9640 1 -- 172.21.15.83:0/1274511662 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52440b5830 con 0x7f52540cfa10 2024-04-17T10:51:30.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.611+0000 7f5259b71640 1 -- 172.21.15.83:0/1274511662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52540b41f0 con 0x7f52540cfa10 2024-04-17T10:51:30.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.615+0000 7f5259b71640 1 -- 172.21.15.83:0/1274511662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5254074f90 con 0x7f52540cfa10 2024-04-17T10:51:30.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.615+0000 7f5250ff9640 1 -- 172.21.15.83:0/1274511662 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f52440b59d0 con 0x7f52540cfa10 2024-04-17T10:51:30.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.615+0000 7f5250ff9640 1 --2- 172.21.15.83:0/1274511662 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5234066f60 0x7f5234069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:30.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.615+0000 7f5250ff9640 1 -- 172.21.15.83:0/1274511662 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f524411f8d0 con 0x7f52540cfa10 2024-04-17T10:51:30.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.615+0000 7f524affd640 1 --2- 172.21.15.83:0/1274511662 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5234066f60 0x7f5234069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:30.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.615+0000 7f524affd640 1 --2- 172.21.15.83:0/1274511662 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5234066f60 0x7f5234069420 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f5254159760 tx=0x7f523c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:30.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.619+0000 7f5250ff9640 1 -- 172.21.15.83:0/1274511662 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5244016070 con 0x7f52540cfa10 2024-04-17T10:51:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.803+0000 7f5259b71640 1 -- 172.21.15.83:0/1274511662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52540cbba0 con 0x7f52540cfa10 2024-04-17T10:51:30.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.803+0000 7f5250ff9640 1 -- 172.21.15.83:0/1274511662 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f524411f070 con 0x7f52540cfa10 2024-04-17T10:51:30.798 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:51:30.798 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:51:30.798 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:51:30.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.807+0000 7f5259b71640 1 -- 172.21.15.83:0/1274511662 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5234066f60 msgr2=0x7f5234069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:30.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.807+0000 7f5259b71640 1 --2- 172.21.15.83:0/1274511662 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5234066f60 0x7f5234069420 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f5254159760 tx=0x7f523c008040 comp rx=0 tx=0).stop 2024-04-17T10:51:30.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.807+0000 7f5259b71640 1 -- 172.21.15.83:0/1274511662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52540cfa10 msgr2=0x7f5254151390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:30.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.807+0000 7f5259b71640 1 --2- 172.21.15.83:0/1274511662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52540cfa10 0x7f5254151390 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f524401a020 tx=0x7f5244019190 comp rx=0 tx=0).stop 2024-04-17T10:51:30.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.807+0000 7f5259b71640 1 -- 172.21.15.83:0/1274511662 shutdown_connections 2024-04-17T10:51:30.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.807+0000 7f5259b71640 1 --2- 172.21.15.83:0/1274511662 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5234066f60 0x7f5234069420 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:30.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.807+0000 7f5259b71640 1 --2- 172.21.15.83:0/1274511662 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f525415c960 0x7f52540ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:30.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.807+0000 7f5259b71640 1 --2- 172.21.15.83:0/1274511662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f525415ba30 0x7f52541459f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:30.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.807+0000 7f5259b71640 1 --2- 172.21.15.83:0/1274511662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52540cfa10 0x7f5254151390 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:30.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.807+0000 7f5259b71640 1 -- 172.21.15.83:0/1274511662 >> 172.21.15.83:0/1274511662 conn(0x7f525406abe0 msgr2=0x7f5254159fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:30.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.807+0000 7f5259b71640 1 -- 172.21.15.83:0/1274511662 shutdown_connections 2024-04-17T10:51:30.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:30.807+0000 7f5259b71640 1 -- 172.21.15.83:0/1274511662 wait complete. 2024-04-17T10:51:31.218 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:51:31.218 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:51:32.219 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:51:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.359+0000 7fded457d640 1 Processor -- start 2024-04-17T10:51:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.359+0000 7fded457d640 1 -- start start 2024-04-17T10:51:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.359+0000 7fded457d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdecc0aedd0 0x7fdecc06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.359+0000 7fded457d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdecc06dae0 0x7fdecc0b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.359+0000 7fded457d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdecc15e440 con 0x7fdecc15db50 2024-04-17T10:51:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.359+0000 7fded457d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdecc15e720 con 0x7fdecc06dae0 2024-04-17T10:51:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.359+0000 7fded457d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc15ea00 con 0x7fdecc0aedd0 2024-04-17T10:51:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.359+0000 7fded2b1a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdecc06dae0 0x7fdecc0b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.359+0000 7fded2b1a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdecc06dae0 0x7fdecc0b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37892/0 (socket says 172.21.15.83:37892) 2024-04-17T10:51:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.359+0000 7fded2319640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fdecc15db50 0x7fdecc070470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47902/0 (socket says 172.21.15.83:47902) 2024-04-17T10:51:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.359+0000 7fded2b1a640 1 -- 172.21.15.83:0/2444041537 learned_addr learned my addr 172.21.15.83:0/2444041537 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.359+0000 7fded1b18640 1 --2- 172.21.15.83:0/2444041537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdecc0aedd0 0x7fdecc06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.359+0000 7fded2b1a640 1 -- 172.21.15.83:0/2444041537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdecc0aedd0 msgr2=0x7fdecc06d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.359+0000 7fded2b1a640 1 --2- 172.21.15.83:0/2444041537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdecc0aedd0 0x7fdecc06d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.359+0000 7fded2b1a640 1 -- 172.21.15.83:0/2444041537 >> v1:172.21.15.83:6789/0 conn(0x7fdecc15db50 legacy=0x7fdecc070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.359+0000 7fded2b1a640 1 -- 172.21.15.83:0/2444041537 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdecc15ece0 con 0x7fdecc06dae0 2024-04-17T10:51:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.359+0000 7fded1317640 1 -- 172.21.15.83:0/2444041537 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1556434287 0 0) 0x7fdecc15e440 con 0x7fdecc15db50 2024-04-17T10:51:32.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.359+0000 7fded2b1a640 1 --2- 172.21.15.83:0/2444041537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdecc06dae0 0x7fdecc0b2e10 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7fdec4009870 tx=0x7fdec401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=194d126040af0d00 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:32.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.359+0000 7fded1317640 1 -- 172.21.15.83:0/2444041537 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec40b2070 con 0x7fdecc06dae0 2024-04-17T10:51:32.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.359+0000 7fded1317640 1 -- 172.21.15.83:0/2444041537 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec40a9070 con 0x7fdecc06dae0 2024-04-17T10:51:32.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.359+0000 7fded1317640 1 -- 172.21.15.83:0/2444041537 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec40ad880 con 0x7fdecc06dae0 2024-04-17T10:51:32.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.363+0000 7fded457d640 1 -- 172.21.15.83:0/2444041537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdecc06dae0 msgr2=0x7fdecc0b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:32.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.363+0000 7fded457d640 1 --2- 172.21.15.83:0/2444041537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdecc06dae0 0x7fdecc0b2e10 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7fdec4009870 tx=0x7fdec401c560 comp rx=0 tx=0).stop 2024-04-17T10:51:32.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.363+0000 7fded457d640 1 -- 172.21.15.83:0/2444041537 shutdown_connections 2024-04-17T10:51:32.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.363+0000 7fded457d640 1 --2- 172.21.15.83:0/2444041537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdecc06dae0 0x7fdecc0b2e10 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:32.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.363+0000 7fded457d640 1 --2- 172.21.15.83:0/2444041537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdecc0aedd0 0x7fdecc06d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:32.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.363+0000 7fded457d640 1 -- 172.21.15.83:0/2444041537 >> 172.21.15.83:0/2444041537 conn(0x7fdecc06abe0 msgr2=0x7fdecc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:32.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.363+0000 7fded457d640 1 -- 172.21.15.83:0/2444041537 shutdown_connections 2024-04-17T10:51:32.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.363+0000 7fded457d640 1 -- 172.21.15.83:0/2444041537 wait complete. 2024-04-17T10:51:32.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.363+0000 7fded457d640 1 Processor -- start 2024-04-17T10:51:32.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.363+0000 7fded457d640 1 -- start start 2024-04-17T10:51:32.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.363+0000 7fded457d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdecc06dae0 0x7fdecc150990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:32.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.363+0000 7fded457d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdecc0aedd0 0x7fdecc13ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:32.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.363+0000 7fded457d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdecc15db50 0x7fdecc0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:32.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.363+0000 7fded457d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc0b3ef0 con 0x7fdecc06dae0 2024-04-17T10:51:32.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.363+0000 7fded457d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc0b3d70 con 0x7fdecc0aedd0 2024-04-17T10:51:32.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.363+0000 7fded457d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc0b4070 con 0x7fdecc15db50 2024-04-17T10:51:32.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.363+0000 7fded2319640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdecc06dae0 0x7fdecc150990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:32.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.363+0000 7fded2319640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdecc06dae0 0x7fdecc150990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37902/0 (socket says 172.21.15.83:37902) 2024-04-17T10:51:32.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.363+0000 7fded2319640 1 -- 172.21.15.83:0/4139580441 learned_addr learned my addr 172.21.15.83:0/4139580441 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:32.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.363+0000 7fded2b1a640 1 --2- 172.21.15.83:0/4139580441 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdecc15db50 0x7fdecc0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:32.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.363+0000 7fded1b18640 1 --2- 172.21.15.83:0/4139580441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdecc0aedd0 0x7fdecc13ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:32.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.363+0000 7fded2319640 1 -- 172.21.15.83:0/4139580441 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdecc15db50 msgr2=0x7fdecc0b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:32.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.363+0000 7fded2319640 1 --2- 172.21.15.83:0/4139580441 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdecc15db50 0x7fdecc0b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:32.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.363+0000 7fded2319640 1 -- 172.21.15.83:0/4139580441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdecc0aedd0 msgr2=0x7fdecc13ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:32.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.363+0000 7fded2319640 1 --2- 172.21.15.83:0/4139580441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdecc0aedd0 0x7fdecc13ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:32.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.363+0000 7fded2319640 1 -- 172.21.15.83:0/4139580441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdecc158890 con 0x7fdecc06dae0 2024-04-17T10:51:32.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.363+0000 7fded2319640 1 --2- 172.21.15.83:0/4139580441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdecc06dae0 0x7fdecc150990 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7fdec800c7f0 tx=0x7fdec800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:32.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.363+0000 7fdec2ffd640 1 -- 172.21.15.83:0/4139580441 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec8018070 con 0x7fdecc06dae0 2024-04-17T10:51:32.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.363+0000 7fdec2ffd640 1 -- 172.21.15.83:0/4139580441 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec800a5e0 con 0x7fdecc06dae0 2024-04-17T10:51:32.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.363+0000 7fded457d640 1 -- 172.21.15.83:0/4139580441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdecc158b70 con 0x7fdecc06dae0 2024-04-17T10:51:32.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.363+0000 7fdec2ffd640 1 -- 172.21.15.83:0/4139580441 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec800a9d0 con 0x7fdecc06dae0 2024-04-17T10:51:32.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.363+0000 7fded457d640 1 -- 172.21.15.83:0/4139580441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdecc159130 con 0x7fdecc06dae0 2024-04-17T10:51:32.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.367+0000 7fded457d640 1 -- 172.21.15.83:0/4139580441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdecc074f90 con 0x7fdecc06dae0 2024-04-17T10:51:32.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.367+0000 7fdec2ffd640 1 -- 172.21.15.83:0/4139580441 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdec801b070 con 0x7fdecc06dae0 2024-04-17T10:51:32.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.367+0000 7fdec2ffd640 1 --2- 172.21.15.83:0/4139580441 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdeb0066f60 0x7fdeb0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:32.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.367+0000 7fdec2ffd640 1 -- 172.21.15.83:0/4139580441 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fdec8084690 con 0x7fdecc06dae0 2024-04-17T10:51:32.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.367+0000 7fded1b18640 1 --2- 172.21.15.83:0/4139580441 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdeb0066f60 0x7fdeb0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:32.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.367+0000 7fded1b18640 1 --2- 172.21.15.83:0/4139580441 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdeb0066f60 0x7fdeb0069420 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fdecc0cff90 tx=0x7fdebc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:32.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.371+0000 7fdec2ffd640 1 -- 172.21.15.83:0/4139580441 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdec8055630 con 0x7fdecc06dae0 2024-04-17T10:51:32.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.567+0000 7fded457d640 1 -- 172.21.15.83:0/4139580441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdecc0b2af0 con 0x7fdecc06dae0 2024-04-17T10:51:32.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.567+0000 7fdec2ffd640 1 -- 172.21.15.83:0/4139580441 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fdec8054f30 con 0x7fdecc06dae0 2024-04-17T10:51:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:51:32.560 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:51:32.561 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:51:32.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.567+0000 7fded457d640 1 -- 172.21.15.83:0/4139580441 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdeb0066f60 msgr2=0x7fdeb0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:32.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.567+0000 7fded457d640 1 --2- 172.21.15.83:0/4139580441 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdeb0066f60 0x7fdeb0069420 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fdecc0cff90 tx=0x7fdebc008040 comp rx=0 tx=0).stop 2024-04-17T10:51:32.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.567+0000 7fded457d640 1 -- 172.21.15.83:0/4139580441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdecc06dae0 msgr2=0x7fdecc150990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:32.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.567+0000 7fded457d640 1 --2- 172.21.15.83:0/4139580441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdecc06dae0 0x7fdecc150990 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7fdec800c7f0 tx=0x7fdec800ccb0 comp rx=0 tx=0).stop 2024-04-17T10:51:32.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.567+0000 7fded457d640 1 -- 172.21.15.83:0/4139580441 shutdown_connections 2024-04-17T10:51:32.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.567+0000 7fded457d640 1 --2- 172.21.15.83:0/4139580441 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdeb0066f60 0x7fdeb0069420 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:32.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.567+0000 7fded457d640 1 --2- 172.21.15.83:0/4139580441 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdecc15db50 0x7fdecc0b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:32.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.567+0000 7fded457d640 1 --2- 172.21.15.83:0/4139580441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdecc0aedd0 0x7fdecc13ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:32.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.567+0000 7fded457d640 1 --2- 172.21.15.83:0/4139580441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdecc06dae0 0x7fdecc150990 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:32.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.567+0000 7fded457d640 1 -- 172.21.15.83:0/4139580441 >> 172.21.15.83:0/4139580441 conn(0x7fdecc06abe0 msgr2=0x7fdecc06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:32.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.567+0000 7fded457d640 1 -- 172.21.15.83:0/4139580441 shutdown_connections 2024-04-17T10:51:32.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:32.567+0000 7fded457d640 1 -- 172.21.15.83:0/4139580441 wait complete. 2024-04-17T10:51:32.982 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:51:32.982 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:51:33.984 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:51:34.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.119+0000 7ffb9412d640 1 Processor -- start 2024-04-17T10:51:34.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.119+0000 7ffb9412d640 1 -- start start 2024-04-17T10:51:34.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.123+0000 7ffb9412d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb8c15ba30 0x7ffb8c072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:34.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.123+0000 7ffb9412d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffb8c15c960 0x7ffb8c0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:34.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.123+0000 7ffb9412d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb8c0cfea0 con 0x7ffb8c0cfa10 2024-04-17T10:51:34.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.123+0000 7ffb9412d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb8c071ea0 con 0x7ffb8c15c960 2024-04-17T10:51:34.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.123+0000 7ffb9412d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c06e320 con 0x7ffb8c15ba30 2024-04-17T10:51:34.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.123+0000 7ffb916c8640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffb8c15c960 0x7ffb8c0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:34.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.123+0000 7ffb926ca640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ffb8c0cfa10 0x7ffb8c06c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47904/0 (socket says 172.21.15.83:47904) 2024-04-17T10:51:34.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.123+0000 7ffb926ca640 1 -- 172.21.15.83:0/181560564 learned_addr learned my addr 172.21.15.83:0/181560564 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:51:34.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.123+0000 7ffb91ec9640 1 --2- 172.21.15.83:0/181560564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb8c15ba30 0x7ffb8c072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:34.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.123+0000 7ffb916c8640 1 -- 172.21.15.83:0/181560564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb8c15ba30 msgr2=0x7ffb8c072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:34.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.123+0000 7ffb916c8640 1 --2- 172.21.15.83:0/181560564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb8c15ba30 0x7ffb8c072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:34.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.123+0000 7ffb916c8640 1 -- 172.21.15.83:0/181560564 >> v1:172.21.15.83:6789/0 conn(0x7ffb8c0cfa10 legacy=0x7ffb8c06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:34.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.123+0000 7ffb916c8640 1 -- 172.21.15.83:0/181560564 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c06e040 con 0x7ffb8c15c960 2024-04-17T10:51:34.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.123+0000 7ffb90ec7640 1 -- 172.21.15.83:0/181560564 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4234130072 0 0) 0x7ffb8c0cfea0 con 0x7ffb8c0cfa10 2024-04-17T10:51:34.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.123+0000 7ffb916c8640 1 --2- 172.21.15.83:0/181560564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb8c15c960 0x7ffb8c0cf4d0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7ffb800097c0 tx=0x7ffb8001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=250feb915c25f43a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:34.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.123+0000 7ffb90ec7640 1 -- 172.21.15.83:0/181560564 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb800b0070 con 0x7ffb8c15c960 2024-04-17T10:51:34.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.123+0000 7ffb90ec7640 1 -- 172.21.15.83:0/181560564 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb800ab600 con 0x7ffb8c15c960 2024-04-17T10:51:34.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.123+0000 7ffb90ec7640 1 -- 172.21.15.83:0/181560564 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb800ab9d0 con 0x7ffb8c15c960 2024-04-17T10:51:34.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb9412d640 1 -- 172.21.15.83:0/181560564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb8c15c960 msgr2=0x7ffb8c0cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:34.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb9412d640 1 --2- 172.21.15.83:0/181560564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb8c15c960 0x7ffb8c0cf4d0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7ffb800097c0 tx=0x7ffb8001c410 comp rx=0 tx=0).stop 2024-04-17T10:51:34.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb9412d640 1 -- 172.21.15.83:0/181560564 shutdown_connections 2024-04-17T10:51:34.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb9412d640 1 --2- 172.21.15.83:0/181560564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb8c15c960 0x7ffb8c0cf4d0 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:34.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb9412d640 1 --2- 172.21.15.83:0/181560564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb8c15ba30 0x7ffb8c072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:34.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb9412d640 1 -- 172.21.15.83:0/181560564 >> 172.21.15.83:0/181560564 conn(0x7ffb8c06abe0 msgr2=0x7ffb8c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:34.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb9412d640 1 -- 172.21.15.83:0/181560564 shutdown_connections 2024-04-17T10:51:34.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb9412d640 1 -- 172.21.15.83:0/181560564 wait complete. 2024-04-17T10:51:34.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb9412d640 1 Processor -- start 2024-04-17T10:51:34.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb9412d640 1 -- start start 2024-04-17T10:51:34.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb9412d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb8c0cfa10 0x7ffb8c14a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:34.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb9412d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb8c15ba30 0x7ffb8c143820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:34.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb9412d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb8c15c960 0x7ffb8c14fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:34.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb9412d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c06dca0 con 0x7ffb8c0cfa10 2024-04-17T10:51:34.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb9412d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c0aeae0 con 0x7ffb8c15c960 2024-04-17T10:51:34.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb9412d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c06de20 con 0x7ffb8c15ba30 2024-04-17T10:51:34.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb91ec9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb8c0cfa10 0x7ffb8c14a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:34.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb91ec9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb8c0cfa10 0x7ffb8c14a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37922/0 (socket says 172.21.15.83:37922) 2024-04-17T10:51:34.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb91ec9640 1 -- 172.21.15.83:0/1987496168 learned_addr learned my addr 172.21.15.83:0/1987496168 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:34.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb916c8640 1 --2- 172.21.15.83:0/1987496168 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb8c15ba30 0x7ffb8c143820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:34.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb926ca640 1 --2- 172.21.15.83:0/1987496168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb8c15c960 0x7ffb8c14fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:34.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb91ec9640 1 -- 172.21.15.83:0/1987496168 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb8c15ba30 msgr2=0x7ffb8c143820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:34.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb91ec9640 1 --2- 172.21.15.83:0/1987496168 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb8c15ba30 0x7ffb8c143820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:34.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb91ec9640 1 -- 172.21.15.83:0/1987496168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb8c15c960 msgr2=0x7ffb8c14fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:34.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb91ec9640 1 --2- 172.21.15.83:0/1987496168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb8c15c960 0x7ffb8c14fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:34.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb91ec9640 1 -- 172.21.15.83:0/1987496168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c14adc0 con 0x7ffb8c0cfa10 2024-04-17T10:51:34.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb926ca640 1 --2- 172.21.15.83:0/1987496168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb8c15c960 0x7ffb8c14fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:51:34.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb91ec9640 1 --2- 172.21.15.83:0/1987496168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb8c0cfa10 0x7ffb8c14a880 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7ffb84007f20 tx=0x7ffb8400e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:34.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb8a7fc640 1 -- 172.21.15.83:0/1987496168 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb84014620 con 0x7ffb8c0cfa10 2024-04-17T10:51:34.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb8a7fc640 1 -- 172.21.15.83:0/1987496168 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb84019070 con 0x7ffb8c0cfa10 2024-04-17T10:51:34.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb9412d640 1 -- 172.21.15.83:0/1987496168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb8c1440c0 con 0x7ffb8c0cfa10 2024-04-17T10:51:34.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb8a7fc640 1 -- 172.21.15.83:0/1987496168 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb84014a00 con 0x7ffb8c0cfa10 2024-04-17T10:51:34.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb9412d640 1 -- 172.21.15.83:0/1987496168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb8c0af5c0 con 0x7ffb8c0cfa10 2024-04-17T10:51:34.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.127+0000 7ffb9412d640 1 -- 172.21.15.83:0/1987496168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb8c074f90 con 0x7ffb8c0cfa10 2024-04-17T10:51:34.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.131+0000 7ffb8a7fc640 1 -- 172.21.15.83:0/1987496168 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffb84014ba0 con 0x7ffb8c0cfa10 2024-04-17T10:51:34.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.131+0000 7ffb8a7fc640 1 --2- 172.21.15.83:0/1987496168 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffb78066f60 0x7ffb78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:34.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.131+0000 7ffb8a7fc640 1 -- 172.21.15.83:0/1987496168 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7ffb840859a0 con 0x7ffb8c0cfa10 2024-04-17T10:51:34.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.131+0000 7ffb916c8640 1 --2- 172.21.15.83:0/1987496168 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffb78066f60 0x7ffb78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:34.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.131+0000 7ffb916c8640 1 --2- 172.21.15.83:0/1987496168 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffb78066f60 0x7ffb78069420 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ffb800ad000 tx=0x7ffb80008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:34.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.135+0000 7ffb8a7fc640 1 -- 172.21.15.83:0/1987496168 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffb84056940 con 0x7ffb8c0cfa10 2024-04-17T10:51:34.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.327+0000 7ffb9412d640 1 -- 172.21.15.83:0/1987496168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb8c0af090 con 0x7ffb8c0cfa10 2024-04-17T10:51:34.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.327+0000 7ffb8a7fc640 1 -- 172.21.15.83:0/1987496168 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7ffb84056240 con 0x7ffb8c0cfa10 2024-04-17T10:51:34.322 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:51:34.322 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:51:34.322 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:51:34.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.331+0000 7ffb9412d640 1 -- 172.21.15.83:0/1987496168 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffb78066f60 msgr2=0x7ffb78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:34.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.331+0000 7ffb9412d640 1 --2- 172.21.15.83:0/1987496168 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffb78066f60 0x7ffb78069420 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ffb800ad000 tx=0x7ffb80008040 comp rx=0 tx=0).stop 2024-04-17T10:51:34.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.331+0000 7ffb9412d640 1 -- 172.21.15.83:0/1987496168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb8c0cfa10 msgr2=0x7ffb8c14a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:34.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.331+0000 7ffb9412d640 1 --2- 172.21.15.83:0/1987496168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb8c0cfa10 0x7ffb8c14a880 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7ffb84007f20 tx=0x7ffb8400e570 comp rx=0 tx=0).stop 2024-04-17T10:51:34.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.331+0000 7ffb9412d640 1 -- 172.21.15.83:0/1987496168 shutdown_connections 2024-04-17T10:51:34.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.331+0000 7ffb9412d640 1 --2- 172.21.15.83:0/1987496168 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ffb78066f60 0x7ffb78069420 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:34.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.331+0000 7ffb9412d640 1 --2- 172.21.15.83:0/1987496168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb8c15c960 0x7ffb8c14fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:34.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.331+0000 7ffb9412d640 1 --2- 172.21.15.83:0/1987496168 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb8c15ba30 0x7ffb8c143820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:34.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.331+0000 7ffb9412d640 1 --2- 172.21.15.83:0/1987496168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb8c0cfa10 0x7ffb8c14a880 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:34.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.331+0000 7ffb9412d640 1 -- 172.21.15.83:0/1987496168 >> 172.21.15.83:0/1987496168 conn(0x7ffb8c06abe0 msgr2=0x7ffb8c06c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:34.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.331+0000 7ffb9412d640 1 -- 172.21.15.83:0/1987496168 shutdown_connections 2024-04-17T10:51:34.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:34.331+0000 7ffb9412d640 1 -- 172.21.15.83:0/1987496168 wait complete. 2024-04-17T10:51:34.743 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:51:34.743 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:51:35.745 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:51:35.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.883+0000 7fd7b5935640 1 Processor -- start 2024-04-17T10:51:35.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.883+0000 7fd7b5935640 1 -- start start 2024-04-17T10:51:35.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.883+0000 7fd7b5935640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd7b00b3990 0x7fd7b0070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:35.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.887+0000 7fd7b5935640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b00709b0 0x7fd7b006c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:35.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.887+0000 7fd7b5935640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7b00b2bd0 con 0x7fd7b00cf060 2024-04-17T10:51:35.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.887+0000 7fd7b5935640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7b00b2980 con 0x7fd7b00b3990 2024-04-17T10:51:35.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.887+0000 7fd7b5935640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b00cf440 con 0x7fd7b00709b0 2024-04-17T10:51:35.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.887+0000 7fd7ae7fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd7b00b3990 0x7fd7b0070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:35.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.887+0000 7fd7ae7fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd7b00b3990 0x7fd7b0070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37936/0 (socket says 172.21.15.83:37936) 2024-04-17T10:51:35.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.887+0000 7fd7ae7fc640 1 -- 172.21.15.83:0/757368525 learned_addr learned my addr 172.21.15.83:0/757368525 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:35.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.887+0000 7fd7af7fe640 1 --2- 172.21.15.83:0/757368525 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b00709b0 0x7fd7b006c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:35.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.887+0000 7fd7ae7fc640 1 -- 172.21.15.83:0/757368525 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b00709b0 msgr2=0x7fd7b006c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:35.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.887+0000 7fd7ae7fc640 1 --2- 172.21.15.83:0/757368525 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b00709b0 0x7fd7b006c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:35.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.887+0000 7fd7ae7fc640 1 -- 172.21.15.83:0/757368525 >> v1:172.21.15.167:6789/0 conn(0x7fd7b00cf060 legacy=0x7fd7b00b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:35.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.887+0000 7fd7ae7fc640 1 -- 172.21.15.83:0/757368525 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b00b2eb0 con 0x7fd7b00b3990 2024-04-17T10:51:35.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.887+0000 7fd7ae7fc640 1 --2- 172.21.15.83:0/757368525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7b00b3990 0x7fd7b0070470 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4006fd0 tx=0x7fd7a401e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba1c897e66c812e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:35.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.887+0000 7fd7adffb640 1 -- 172.21.15.83:0/757368525 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a40ad070 con 0x7fd7b00b3990 2024-04-17T10:51:35.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.887+0000 7fd7adffb640 1 -- 172.21.15.83:0/757368525 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a4004230 con 0x7fd7b00b3990 2024-04-17T10:51:35.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.887+0000 7fd7adffb640 1 -- 172.21.15.83:0/757368525 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a40a76f0 con 0x7fd7b00b3990 2024-04-17T10:51:35.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.887+0000 7fd7b5935640 1 -- 172.21.15.83:0/757368525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7b00b3990 msgr2=0x7fd7b0070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:35.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.887+0000 7fd7b5935640 1 --2- 172.21.15.83:0/757368525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7b00b3990 0x7fd7b0070470 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4006fd0 tx=0x7fd7a401e930 comp rx=0 tx=0).stop 2024-04-17T10:51:35.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.887+0000 7fd7b5935640 1 -- 172.21.15.83:0/757368525 shutdown_connections 2024-04-17T10:51:35.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.887+0000 7fd7b5935640 1 --2- 172.21.15.83:0/757368525 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b00709b0 0x7fd7b006c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:35.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.887+0000 7fd7b5935640 1 --2- 172.21.15.83:0/757368525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7b00b3990 0x7fd7b0070470 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:35.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.887+0000 7fd7b5935640 1 -- 172.21.15.83:0/757368525 >> 172.21.15.83:0/757368525 conn(0x7fd7b006abe0 msgr2=0x7fd7b006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:35.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.887+0000 7fd7b5935640 1 -- 172.21.15.83:0/757368525 shutdown_connections 2024-04-17T10:51:35.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.887+0000 7fd7b5935640 1 -- 172.21.15.83:0/757368525 wait complete. 2024-04-17T10:51:35.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.891+0000 7fd7b5935640 1 Processor -- start 2024-04-17T10:51:35.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.891+0000 7fd7b5935640 1 -- start start 2024-04-17T10:51:35.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.891+0000 7fd7b5935640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b00709b0 0x7fd7b0144270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:35.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.891+0000 7fd7b5935640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b00b3990 0x7fd7b00aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:35.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.891+0000 7fd7b5935640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7b00cf060 0x7fd7b00b0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:35.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.891+0000 7fd7b5935640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b006c9d0 con 0x7fd7b00cf060 2024-04-17T10:51:35.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.891+0000 7fd7b5935640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b00b4770 con 0x7fd7b00b3990 2024-04-17T10:51:35.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.891+0000 7fd7b5935640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b006cb50 con 0x7fd7b00709b0 2024-04-17T10:51:35.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.891+0000 7fd7af7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7b00cf060 0x7fd7b00b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:35.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.891+0000 7fd7af7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7b00cf060 0x7fd7b00b0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37938/0 (socket says 172.21.15.83:37938) 2024-04-17T10:51:35.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.891+0000 7fd7af7fe640 1 -- 172.21.15.83:0/1711434637 learned_addr learned my addr 172.21.15.83:0/1711434637 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:35.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.891+0000 7fd7ae7fc640 1 --2- 172.21.15.83:0/1711434637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b00b3990 0x7fd7b00aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:35.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.891+0000 7fd7aeffd640 1 --2- 172.21.15.83:0/1711434637 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b00709b0 0x7fd7b0144270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:35.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.891+0000 7fd7af7fe640 1 -- 172.21.15.83:0/1711434637 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b00709b0 msgr2=0x7fd7b0144270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:35.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.891+0000 7fd7af7fe640 1 --2- 172.21.15.83:0/1711434637 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b00709b0 0x7fd7b0144270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:35.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.891+0000 7fd7af7fe640 1 -- 172.21.15.83:0/1711434637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b00b3990 msgr2=0x7fd7b00aff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:35.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.891+0000 7fd7af7fe640 1 --2- 172.21.15.83:0/1711434637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b00b3990 0x7fd7b00aff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:35.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.891+0000 7fd7af7fe640 1 -- 172.21.15.83:0/1711434637 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b01578f0 con 0x7fd7b00cf060 2024-04-17T10:51:35.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.891+0000 7fd7af7fe640 1 --2- 172.21.15.83:0/1711434637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7b00cf060 0x7fd7b00b0460 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8007e40 tx=0x7fd7a800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:35.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.891+0000 7fd797fff640 1 -- 172.21.15.83:0/1711434637 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8014620 con 0x7fd7b00cf060 2024-04-17T10:51:35.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.891+0000 7fd797fff640 1 -- 172.21.15.83:0/1711434637 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a8019070 con 0x7fd7b00cf060 2024-04-17T10:51:35.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.891+0000 7fd7b5935640 1 -- 172.21.15.83:0/1711434637 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b0157610 con 0x7fd7b00cf060 2024-04-17T10:51:35.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.891+0000 7fd797fff640 1 -- 172.21.15.83:0/1711434637 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8014a20 con 0x7fd7b00cf060 2024-04-17T10:51:35.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.891+0000 7fd7b5935640 1 -- 172.21.15.83:0/1711434637 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b0151dd0 con 0x7fd7b00cf060 2024-04-17T10:51:35.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.891+0000 7fd7b5935640 1 -- 172.21.15.83:0/1711434637 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7b0074f90 con 0x7fd7b00cf060 2024-04-17T10:51:35.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.891+0000 7fd797fff640 1 -- 172.21.15.83:0/1711434637 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd7a8014bc0 con 0x7fd7b00cf060 2024-04-17T10:51:35.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.895+0000 7fd797fff640 1 --2- 172.21.15.83:0/1711434637 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd79c066f60 0x7fd79c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:35.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.895+0000 7fd797fff640 1 -- 172.21.15.83:0/1711434637 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fd7a80858b0 con 0x7fd7b00cf060 2024-04-17T10:51:35.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.895+0000 7fd7aeffd640 1 --2- 172.21.15.83:0/1711434637 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd79c066f60 0x7fd79c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:35.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.895+0000 7fd7aeffd640 1 --2- 172.21.15.83:0/1711434637 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd79c066f60 0x7fd79c069420 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0000f70 tx=0x7fd7a0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:35.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:35.899+0000 7fd797fff640 1 -- 172.21.15.83:0/1711434637 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd7a8011260 con 0x7fd7b00cf060 2024-04-17T10:51:36.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:36.079+0000 7fd7b5935640 1 -- 172.21.15.83:0/1711434637 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7b0074480 con 0x7fd7b00cf060 2024-04-17T10:51:36.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:36.083+0000 7fd797fff640 1 -- 172.21.15.83:0/1711434637 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fd7a80568e0 con 0x7fd7b00cf060 2024-04-17T10:51:36.076 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:51:36.076 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:51:36.076 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:51:36.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:36.083+0000 7fd7b5935640 1 -- 172.21.15.83:0/1711434637 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd79c066f60 msgr2=0x7fd79c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:36.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:36.083+0000 7fd7b5935640 1 --2- 172.21.15.83:0/1711434637 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd79c066f60 0x7fd79c069420 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0000f70 tx=0x7fd7a0009040 comp rx=0 tx=0).stop 2024-04-17T10:51:36.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:36.083+0000 7fd7b5935640 1 -- 172.21.15.83:0/1711434637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7b00cf060 msgr2=0x7fd7b00b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:36.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:36.083+0000 7fd7b5935640 1 --2- 172.21.15.83:0/1711434637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7b00cf060 0x7fd7b00b0460 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8007e40 tx=0x7fd7a800e4b0 comp rx=0 tx=0).stop 2024-04-17T10:51:36.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:36.083+0000 7fd7b5935640 1 -- 172.21.15.83:0/1711434637 shutdown_connections 2024-04-17T10:51:36.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:36.083+0000 7fd7b5935640 1 --2- 172.21.15.83:0/1711434637 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd79c066f60 0x7fd79c069420 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:36.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:36.083+0000 7fd7b5935640 1 --2- 172.21.15.83:0/1711434637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7b00cf060 0x7fd7b00b0460 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:36.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:36.083+0000 7fd7b5935640 1 --2- 172.21.15.83:0/1711434637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b00b3990 0x7fd7b00aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:36.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:36.083+0000 7fd7b5935640 1 --2- 172.21.15.83:0/1711434637 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b00709b0 0x7fd7b0144270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:36.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:36.083+0000 7fd7b5935640 1 -- 172.21.15.83:0/1711434637 >> 172.21.15.83:0/1711434637 conn(0x7fd7b006abe0 msgr2=0x7fd7b00aec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:36.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:36.083+0000 7fd7b5935640 1 -- 172.21.15.83:0/1711434637 shutdown_connections 2024-04-17T10:51:36.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:36.083+0000 7fd7b5935640 1 -- 172.21.15.83:0/1711434637 wait complete. 2024-04-17T10:51:36.495 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:51:36.495 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:51:37.497 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:51:37.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.631+0000 7fa609fac640 1 Processor -- start 2024-04-17T10:51:37.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.631+0000 7fa609fac640 1 -- start start 2024-04-17T10:51:37.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.635+0000 7fa609fac640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa60415ba30 0x7fa604072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:37.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.635+0000 7fa609fac640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa60415c960 0x7fa6040cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:37.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.635+0000 7fa609fac640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6040cfea0 con 0x7fa6040cfa10 2024-04-17T10:51:37.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.635+0000 7fa609fac640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa604071ea0 con 0x7fa60415ba30 2024-04-17T10:51:37.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.635+0000 7fa609fac640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa60406e320 con 0x7fa60415c960 2024-04-17T10:51:37.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.635+0000 7fa6037fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa60415ba30 0x7fa604072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:37.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.635+0000 7fa6037fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa60415ba30 0x7fa604072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46456/0 (socket says 172.21.15.83:46456) 2024-04-17T10:51:37.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.635+0000 7fa6037fe640 1 -- 172.21.15.83:0/938269877 learned_addr learned my addr 172.21.15.83:0/938269877 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:37.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.635+0000 7fa603fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa6040cfa10 0x7fa60406c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35124/0 (socket says 172.21.15.83:35124) 2024-04-17T10:51:37.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.635+0000 7fa602ffd640 1 --2- 172.21.15.83:0/938269877 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa60415c960 0x7fa6040cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:37.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.635+0000 7fa6037fe640 1 -- 172.21.15.83:0/938269877 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa60415c960 msgr2=0x7fa6040cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:37.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.635+0000 7fa6037fe640 1 --2- 172.21.15.83:0/938269877 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa60415c960 0x7fa6040cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:37.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.635+0000 7fa6037fe640 1 -- 172.21.15.83:0/938269877 >> v1:172.21.15.83:6789/0 conn(0x7fa6040cfa10 legacy=0x7fa60406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:37.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.635+0000 7fa6027fc640 1 -- 172.21.15.83:0/938269877 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2187946247 0 0) 0x7fa6040cfea0 con 0x7fa6040cfa10 2024-04-17T10:51:37.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.635+0000 7fa6037fe640 1 -- 172.21.15.83:0/938269877 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa60406e040 con 0x7fa60415ba30 2024-04-17T10:51:37.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.635+0000 7fa6037fe640 1 --2- 172.21.15.83:0/938269877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa60415ba30 0x7fa604072330 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4009870 tx=0x7fa5f401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd9f24ad3ff388f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:37.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa6027fc640 1 -- 172.21.15.83:0/938269877 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f40b1070 con 0x7fa60415ba30 2024-04-17T10:51:37.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa6027fc640 1 -- 172.21.15.83:0/938269877 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f40ac620 con 0x7fa60415ba30 2024-04-17T10:51:37.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa6027fc640 1 -- 172.21.15.83:0/938269877 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f40ac9f0 con 0x7fa60415ba30 2024-04-17T10:51:37.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa609fac640 1 -- 172.21.15.83:0/938269877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa60415ba30 msgr2=0x7fa604072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:37.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa609fac640 1 --2- 172.21.15.83:0/938269877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa60415ba30 0x7fa604072330 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4009870 tx=0x7fa5f401c4d0 comp rx=0 tx=0).stop 2024-04-17T10:51:37.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa609fac640 1 -- 172.21.15.83:0/938269877 shutdown_connections 2024-04-17T10:51:37.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa609fac640 1 --2- 172.21.15.83:0/938269877 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa60415c960 0x7fa6040cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:37.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa609fac640 1 --2- 172.21.15.83:0/938269877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa60415ba30 0x7fa604072330 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:37.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa609fac640 1 -- 172.21.15.83:0/938269877 >> 172.21.15.83:0/938269877 conn(0x7fa60406abe0 msgr2=0x7fa60406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:37.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa609fac640 1 -- 172.21.15.83:0/938269877 shutdown_connections 2024-04-17T10:51:37.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa609fac640 1 -- 172.21.15.83:0/938269877 wait complete. 2024-04-17T10:51:37.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa609fac640 1 Processor -- start 2024-04-17T10:51:37.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa609fac640 1 -- start start 2024-04-17T10:51:37.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa609fac640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6040cfa10 0x7fa6040cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:37.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa609fac640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa60415ba30 0x7fa6040b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:37.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa609fac640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa60415c960 0x7fa60414fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:37.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa609fac640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60406de20 con 0x7fa60415ba30 2024-04-17T10:51:37.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa609fac640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60406dca0 con 0x7fa60415c960 2024-04-17T10:51:37.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa609fac640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa60415cd40 con 0x7fa6040cfa10 2024-04-17T10:51:37.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa602ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa60415ba30 0x7fa6040b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:37.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa602ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa60415ba30 0x7fa6040b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46468/0 (socket says 172.21.15.83:46468) 2024-04-17T10:51:37.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa602ffd640 1 -- 172.21.15.83:0/869345731 learned_addr learned my addr 172.21.15.83:0/869345731 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:37.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa6037fe640 1 --2- 172.21.15.83:0/869345731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6040cfa10 0x7fa6040cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:37.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa603fff640 1 --2- 172.21.15.83:0/869345731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa60415c960 0x7fa60414fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:37.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa602ffd640 1 -- 172.21.15.83:0/869345731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6040cfa10 msgr2=0x7fa6040cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:37.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa602ffd640 1 --2- 172.21.15.83:0/869345731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6040cfa10 0x7fa6040cd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:37.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa602ffd640 1 -- 172.21.15.83:0/869345731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa60415c960 msgr2=0x7fa60414fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:37.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa602ffd640 1 --2- 172.21.15.83:0/869345731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa60415c960 0x7fa60414fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:37.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa602ffd640 1 -- 172.21.15.83:0/869345731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6040cdb30 con 0x7fa60415ba30 2024-04-17T10:51:37.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa603fff640 1 --2- 172.21.15.83:0/869345731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa60415c960 0x7fa60414fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:51:37.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa602ffd640 1 --2- 172.21.15.83:0/869345731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa60415ba30 0x7fa6040b3fe0 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800d5f0 tx=0x7fa5f800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:37.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa5ebfff640 1 -- 172.21.15.83:0/869345731 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f80125f0 con 0x7fa60415ba30 2024-04-17T10:51:37.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa5ebfff640 1 -- 172.21.15.83:0/869345731 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f8019070 con 0x7fa60415ba30 2024-04-17T10:51:37.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa609fac640 1 -- 172.21.15.83:0/869345731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6040b3220 con 0x7fa60415ba30 2024-04-17T10:51:37.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa5ebfff640 1 -- 172.21.15.83:0/869345731 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f80129b0 con 0x7fa60415ba30 2024-04-17T10:51:37.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.639+0000 7fa609fac640 1 -- 172.21.15.83:0/869345731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6040b3ac0 con 0x7fa60415ba30 2024-04-17T10:51:37.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.643+0000 7fa609fac640 1 -- 172.21.15.83:0/869345731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6040b3610 con 0x7fa60415ba30 2024-04-17T10:51:37.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.643+0000 7fa5ebfff640 1 -- 172.21.15.83:0/869345731 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa5f8012b50 con 0x7fa60415ba30 2024-04-17T10:51:37.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.643+0000 7fa5ebfff640 1 --2- 172.21.15.83:0/869345731 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa5f0066f60 0x7fa5f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:37.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.643+0000 7fa5ebfff640 1 -- 172.21.15.83:0/869345731 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fa5f8085450 con 0x7fa60415ba30 2024-04-17T10:51:37.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.643+0000 7fa6037fe640 1 --2- 172.21.15.83:0/869345731 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa5f0066f60 0x7fa5f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:37.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.643+0000 7fa6037fe640 1 --2- 172.21.15.83:0/869345731 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa5f0066f60 0x7fa5f0069420 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fa5f40ae000 tx=0x7fa5f4004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:37.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.647+0000 7fa5ebfff640 1 -- 172.21.15.83:0/869345731 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa5f8056480 con 0x7fa60415ba30 2024-04-17T10:51:37.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.839+0000 7fa609fac640 1 -- 172.21.15.83:0/869345731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa604074f90 con 0x7fa60415ba30 2024-04-17T10:51:37.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.839+0000 7fa5ebfff640 1 -- 172.21.15.83:0/869345731 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fa5f8055d80 con 0x7fa60415ba30 2024-04-17T10:51:37.835 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:51:37.835 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:51:37.835 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:51:37.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.843+0000 7fa609fac640 1 -- 172.21.15.83:0/869345731 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa5f0066f60 msgr2=0x7fa5f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:37.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.843+0000 7fa609fac640 1 --2- 172.21.15.83:0/869345731 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa5f0066f60 0x7fa5f0069420 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fa5f40ae000 tx=0x7fa5f4004740 comp rx=0 tx=0).stop 2024-04-17T10:51:37.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.843+0000 7fa609fac640 1 -- 172.21.15.83:0/869345731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa60415ba30 msgr2=0x7fa6040b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:37.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.843+0000 7fa609fac640 1 --2- 172.21.15.83:0/869345731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa60415ba30 0x7fa6040b3fe0 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800d5f0 tx=0x7fa5f800dab0 comp rx=0 tx=0).stop 2024-04-17T10:51:37.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.843+0000 7fa609fac640 1 -- 172.21.15.83:0/869345731 shutdown_connections 2024-04-17T10:51:37.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.843+0000 7fa609fac640 1 --2- 172.21.15.83:0/869345731 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa5f0066f60 0x7fa5f0069420 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:37.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.843+0000 7fa609fac640 1 --2- 172.21.15.83:0/869345731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa60415c960 0x7fa60414fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:37.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.843+0000 7fa609fac640 1 --2- 172.21.15.83:0/869345731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa60415ba30 0x7fa6040b3fe0 unknown :-1 s=CLOSED pgs=2156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:37.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.843+0000 7fa609fac640 1 --2- 172.21.15.83:0/869345731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6040cfa10 0x7fa6040cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:37.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.843+0000 7fa609fac640 1 -- 172.21.15.83:0/869345731 >> 172.21.15.83:0/869345731 conn(0x7fa60406abe0 msgr2=0x7fa6040af2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:37.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.843+0000 7fa609fac640 1 -- 172.21.15.83:0/869345731 shutdown_connections 2024-04-17T10:51:37.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:37.843+0000 7fa609fac640 1 -- 172.21.15.83:0/869345731 wait complete. 2024-04-17T10:51:38.256 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:51:38.256 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:51:39.002 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:51:39.005 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:51:39.257 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:51:39.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.395+0000 7efd1f76b640 1 Processor -- start 2024-04-17T10:51:39.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.395+0000 7efd1f76b640 1 -- start start 2024-04-17T10:51:39.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.395+0000 7efd1f76b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7efd18072330 0x7efd180cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:39.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.395+0000 7efd1f76b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efd180cfa10 0x7efd1806c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:39.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.395+0000 7efd1f76b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd180cfea0 con 0x7efd1815ba30 2024-04-17T10:51:39.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.395+0000 7efd1f76b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd18071ea0 con 0x7efd180cfa10 2024-04-17T10:51:39.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.395+0000 7efd1f76b640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd1806e320 con 0x7efd18072330 2024-04-17T10:51:39.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.395+0000 7efd1dd08640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efd180cfa10 0x7efd1806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:39.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.395+0000 7efd1d507640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7efd1815ba30 0x7efd1815c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35132/0 (socket says 172.21.15.83:35132) 2024-04-17T10:51:39.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.395+0000 7efd1d507640 1 -- 172.21.15.83:0/58533468 learned_addr learned my addr 172.21.15.83:0/58533468 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:51:39.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.395+0000 7efd1cd06640 1 --2- 172.21.15.83:0/58533468 >> v2:172.21.15.167:3300/0 conn(0x7efd18072330 0x7efd180cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:39.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.395+0000 7efd1dd08640 1 -- 172.21.15.83:0/58533468 >> v2:172.21.15.167:3300/0 conn(0x7efd18072330 msgr2=0x7efd180cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:39.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.395+0000 7efd1dd08640 1 --2- 172.21.15.83:0/58533468 >> v2:172.21.15.167:3300/0 conn(0x7efd18072330 0x7efd180cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:39.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.395+0000 7efd1dd08640 1 -- 172.21.15.83:0/58533468 >> v1:172.21.15.83:6789/0 conn(0x7efd1815ba30 legacy=0x7efd1815c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:39.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.395+0000 7efd1dd08640 1 -- 172.21.15.83:0/58533468 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd1806e040 con 0x7efd180cfa10 2024-04-17T10:51:39.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.395+0000 7efd0ffff640 1 -- 172.21.15.83:0/58533468 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1615789581 0 0) 0x7efd180cfea0 con 0x7efd1815ba30 2024-04-17T10:51:39.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.395+0000 7efd1cd06640 1 --2- 172.21.15.83:0/58533468 >> v2:172.21.15.167:3300/0 conn(0x7efd18072330 0x7efd180cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:51:39.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.399+0000 7efd1dd08640 1 --2- 172.21.15.83:0/58533468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd180cfa10 0x7efd1806c490 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7efd180af2b0 tx=0x7efd1001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7799b6033a942a70 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:39.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.399+0000 7efd0ffff640 1 -- 172.21.15.83:0/58533468 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd100b2070 con 0x7efd180cfa10 2024-04-17T10:51:39.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.399+0000 7efd0ffff640 1 -- 172.21.15.83:0/58533468 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd100a5870 con 0x7efd180cfa10 2024-04-17T10:51:39.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.399+0000 7efd0ffff640 1 -- 172.21.15.83:0/58533468 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd100a5c40 con 0x7efd180cfa10 2024-04-17T10:51:39.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.399+0000 7efd1f76b640 1 -- 172.21.15.83:0/58533468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd180cfa10 msgr2=0x7efd1806c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:39.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.399+0000 7efd1f76b640 1 --2- 172.21.15.83:0/58533468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd180cfa10 0x7efd1806c490 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7efd180af2b0 tx=0x7efd1001c8a0 comp rx=0 tx=0).stop 2024-04-17T10:51:39.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.399+0000 7efd1f76b640 1 -- 172.21.15.83:0/58533468 shutdown_connections 2024-04-17T10:51:39.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.399+0000 7efd1f76b640 1 --2- 172.21.15.83:0/58533468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd180cfa10 0x7efd1806c490 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:39.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.399+0000 7efd1f76b640 1 --2- 172.21.15.83:0/58533468 >> v2:172.21.15.167:3300/0 conn(0x7efd18072330 0x7efd180cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:39.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.399+0000 7efd1f76b640 1 -- 172.21.15.83:0/58533468 >> 172.21.15.83:0/58533468 conn(0x7efd1806abe0 msgr2=0x7efd1806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:39.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.399+0000 7efd1f76b640 1 -- 172.21.15.83:0/58533468 shutdown_connections 2024-04-17T10:51:39.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.399+0000 7efd1f76b640 1 -- 172.21.15.83:0/58533468 wait complete. 2024-04-17T10:51:39.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.399+0000 7efd1f76b640 1 Processor -- start 2024-04-17T10:51:39.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.399+0000 7efd1f76b640 1 -- start start 2024-04-17T10:51:39.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.399+0000 7efd1f76b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd18072330 0x7efd1814efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:39.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.399+0000 7efd1f76b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd180cfa10 0x7efd181585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:39.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.399+0000 7efd1f76b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd1815ba30 0x7efd18138da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:39.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.399+0000 7efd1f76b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1806de20 con 0x7efd18072330 2024-04-17T10:51:39.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.399+0000 7efd1f76b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1806dca0 con 0x7efd1815ba30 2024-04-17T10:51:39.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.399+0000 7efd1f76b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd18168520 con 0x7efd180cfa10 2024-04-17T10:51:39.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.399+0000 7efd1d507640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd18072330 0x7efd1814efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:39.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.399+0000 7efd1d507640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd18072330 0x7efd1814efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46494/0 (socket says 172.21.15.83:46494) 2024-04-17T10:51:39.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.399+0000 7efd1d507640 1 -- 172.21.15.83:0/363581988 learned_addr learned my addr 172.21.15.83:0/363581988 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:39.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.403+0000 7efd1dd08640 1 --2- 172.21.15.83:0/363581988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd1815ba30 0x7efd18138da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:39.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.403+0000 7efd1d507640 1 -- 172.21.15.83:0/363581988 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd180cfa10 msgr2=0x7efd181585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:39.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.403+0000 7efd1cd06640 1 --2- 172.21.15.83:0/363581988 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd180cfa10 0x7efd181585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:39.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.403+0000 7efd1d507640 1 --2- 172.21.15.83:0/363581988 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd180cfa10 0x7efd181585b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:39.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.403+0000 7efd1d507640 1 -- 172.21.15.83:0/363581988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd1815ba30 msgr2=0x7efd18138da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:39.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.403+0000 7efd1d507640 1 --2- 172.21.15.83:0/363581988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd1815ba30 0x7efd18138da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:39.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.403+0000 7efd1d507640 1 -- 172.21.15.83:0/363581988 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd18159020 con 0x7efd18072330 2024-04-17T10:51:39.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.403+0000 7efd1d507640 1 --2- 172.21.15.83:0/363581988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd18072330 0x7efd1814efd0 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7efd140011f0 tx=0x7efd1400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:39.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.403+0000 7efd0dffb640 1 -- 172.21.15.83:0/363581988 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd14018070 con 0x7efd18072330 2024-04-17T10:51:39.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.403+0000 7efd0dffb640 1 -- 172.21.15.83:0/363581988 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd140074e0 con 0x7efd18072330 2024-04-17T10:51:39.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.403+0000 7efd1f76b640 1 -- 172.21.15.83:0/363581988 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd18159300 con 0x7efd18072330 2024-04-17T10:51:39.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.403+0000 7efd0dffb640 1 -- 172.21.15.83:0/363581988 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd14004a10 con 0x7efd18072330 2024-04-17T10:51:39.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.403+0000 7efd1f76b640 1 -- 172.21.15.83:0/363581988 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd18156c80 con 0x7efd18072330 2024-04-17T10:51:39.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.403+0000 7efd1f76b640 1 -- 172.21.15.83:0/363581988 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd18074f90 con 0x7efd18072330 2024-04-17T10:51:39.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.403+0000 7efd0dffb640 1 -- 172.21.15.83:0/363581988 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efd1401b070 con 0x7efd18072330 2024-04-17T10:51:39.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.403+0000 7efd0dffb640 1 --2- 172.21.15.83:0/363581988 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efcfc066f60 0x7efcfc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:39.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.403+0000 7efd1cd06640 1 --2- 172.21.15.83:0/363581988 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efcfc066f60 0x7efcfc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:39.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.403+0000 7efd0dffb640 1 -- 172.21.15.83:0/363581988 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7efd14084350 con 0x7efd18072330 2024-04-17T10:51:39.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.403+0000 7efd1cd06640 1 --2- 172.21.15.83:0/363581988 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efcfc066f60 0x7efcfc069420 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7efd08006fd0 tx=0x7efd08008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:39.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.407+0000 7efd0dffb640 1 -- 172.21.15.83:0/363581988 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efd14055380 con 0x7efd18072330 2024-04-17T10:51:39.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.603+0000 7efd1f76b640 1 -- 172.21.15.83:0/363581988 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd18156930 con 0x7efd18072330 2024-04-17T10:51:39.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.603+0000 7efd0dffb640 1 -- 172.21.15.83:0/363581988 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7efd1405b080 con 0x7efd18072330 2024-04-17T10:51:39.599 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:51:39.599 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:51:39.599 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:51:39.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.607+0000 7efd1f76b640 1 -- 172.21.15.83:0/363581988 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efcfc066f60 msgr2=0x7efcfc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:39.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.607+0000 7efd1f76b640 1 --2- 172.21.15.83:0/363581988 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efcfc066f60 0x7efcfc069420 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7efd08006fd0 tx=0x7efd08008040 comp rx=0 tx=0).stop 2024-04-17T10:51:39.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.607+0000 7efd1f76b640 1 -- 172.21.15.83:0/363581988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd18072330 msgr2=0x7efd1814efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:39.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.607+0000 7efd1f76b640 1 --2- 172.21.15.83:0/363581988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd18072330 0x7efd1814efd0 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7efd140011f0 tx=0x7efd1400c9b0 comp rx=0 tx=0).stop 2024-04-17T10:51:39.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.607+0000 7efd1f76b640 1 -- 172.21.15.83:0/363581988 shutdown_connections 2024-04-17T10:51:39.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.607+0000 7efd1f76b640 1 --2- 172.21.15.83:0/363581988 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efcfc066f60 0x7efcfc069420 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:39.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.607+0000 7efd1f76b640 1 --2- 172.21.15.83:0/363581988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd1815ba30 0x7efd18138da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:39.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.607+0000 7efd1f76b640 1 --2- 172.21.15.83:0/363581988 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd180cfa10 0x7efd181585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:39.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.607+0000 7efd1f76b640 1 --2- 172.21.15.83:0/363581988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd18072330 0x7efd1814efd0 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:39.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.607+0000 7efd1f76b640 1 -- 172.21.15.83:0/363581988 >> 172.21.15.83:0/363581988 conn(0x7efd1806abe0 msgr2=0x7efd1815ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:39.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.607+0000 7efd1f76b640 1 -- 172.21.15.83:0/363581988 shutdown_connections 2024-04-17T10:51:39.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:39.607+0000 7efd1f76b640 1 -- 172.21.15.83:0/363581988 wait complete. 2024-04-17T10:51:40.021 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:51:40.022 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:51:41.022 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:51:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.155+0000 7fc74a185640 1 Processor -- start 2024-04-17T10:51:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.159+0000 7fc74a185640 1 -- start start 2024-04-17T10:51:41.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.159+0000 7fc74a185640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc744072330 0x7fc7440cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:41.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.159+0000 7fc74a185640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc7440cfa10 0x7fc74406c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:41.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.159+0000 7fc74a185640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7440cfea0 con 0x7fc74415ba30 2024-04-17T10:51:41.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.159+0000 7fc74a185640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc744071ea0 con 0x7fc7440cfa10 2024-04-17T10:51:41.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.159+0000 7fc74a185640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc74406e320 con 0x7fc744072330 2024-04-17T10:51:41.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.159+0000 7fc742ffd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc744072330 0x7fc7440cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:41.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.159+0000 7fc743fff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc7440cfa10 0x7fc74406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:41.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.159+0000 7fc742ffd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc744072330 0x7fc7440cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:47208/0 (socket says 172.21.15.83:47208) 2024-04-17T10:51:41.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.159+0000 7fc742ffd640 1 -- 172.21.15.83:0/1152287109 learned_addr learned my addr 172.21.15.83:0/1152287109 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:41.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.159+0000 7fc742ffd640 1 -- 172.21.15.83:0/1152287109 >> v2:172.21.15.167:3300/0 conn(0x7fc7440cfa10 msgr2=0x7fc74406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:41.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.159+0000 7fc742ffd640 1 --2- 172.21.15.83:0/1152287109 >> v2:172.21.15.167:3300/0 conn(0x7fc7440cfa10 0x7fc74406c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:41.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.159+0000 7fc742ffd640 1 -- 172.21.15.83:0/1152287109 >> v1:172.21.15.167:6789/0 conn(0x7fc74415ba30 legacy=0x7fc74415c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:41.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.159+0000 7fc742ffd640 1 -- 172.21.15.83:0/1152287109 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74406e040 con 0x7fc744072330 2024-04-17T10:51:41.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.163+0000 7fc742ffd640 1 --2- 172.21.15.83:0/1152287109 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc744072330 0x7fc7440cf4d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc738009870 tx=0x7fc73801c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bac5d2af380aff0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:41.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.163+0000 7fc7427fc640 1 -- 172.21.15.83:0/1152287109 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7380b1070 con 0x7fc744072330 2024-04-17T10:51:41.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.163+0000 7fc7427fc640 1 -- 172.21.15.83:0/1152287109 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7380a8050 con 0x7fc744072330 2024-04-17T10:51:41.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.163+0000 7fc7427fc640 1 -- 172.21.15.83:0/1152287109 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7380ac860 con 0x7fc744072330 2024-04-17T10:51:41.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.163+0000 7fc74a185640 1 -- 172.21.15.83:0/1152287109 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc744072330 msgr2=0x7fc7440cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:41.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.163+0000 7fc74a185640 1 --2- 172.21.15.83:0/1152287109 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc744072330 0x7fc7440cf4d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc738009870 tx=0x7fc73801c4d0 comp rx=0 tx=0).stop 2024-04-17T10:51:41.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.163+0000 7fc74a185640 1 -- 172.21.15.83:0/1152287109 shutdown_connections 2024-04-17T10:51:41.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.163+0000 7fc74a185640 1 --2- 172.21.15.83:0/1152287109 >> v2:172.21.15.167:3300/0 conn(0x7fc7440cfa10 0x7fc74406c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.163+0000 7fc74a185640 1 --2- 172.21.15.83:0/1152287109 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc744072330 0x7fc7440cf4d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.163+0000 7fc74a185640 1 -- 172.21.15.83:0/1152287109 >> 172.21.15.83:0/1152287109 conn(0x7fc74406abe0 msgr2=0x7fc74406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.163+0000 7fc74a185640 1 -- 172.21.15.83:0/1152287109 shutdown_connections 2024-04-17T10:51:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.163+0000 7fc74a185640 1 -- 172.21.15.83:0/1152287109 wait complete. 2024-04-17T10:51:41.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.163+0000 7fc74a185640 1 Processor -- start 2024-04-17T10:51:41.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.163+0000 7fc74a185640 1 -- start start 2024-04-17T10:51:41.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.163+0000 7fc74a185640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc744072330 0x7fc74415cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:41.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.163+0000 7fc74a185640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7440cfa10 0x7fc744154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:41.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.163+0000 7fc74a185640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc74415ba30 0x7fc744154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:41.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.163+0000 7fc74a185640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74406de20 con 0x7fc7440cfa10 2024-04-17T10:51:41.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.163+0000 7fc74a185640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74406dca0 con 0x7fc744072330 2024-04-17T10:51:41.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.163+0000 7fc74a185640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc744168520 con 0x7fc74415ba30 2024-04-17T10:51:41.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.163+0000 7fc742ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7440cfa10 0x7fc744154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:41.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.163+0000 7fc742ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7440cfa10 0x7fc744154740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46496/0 (socket says 172.21.15.83:46496) 2024-04-17T10:51:41.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.163+0000 7fc742ffd640 1 -- 172.21.15.83:0/3604881729 learned_addr learned my addr 172.21.15.83:0/3604881729 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:41.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.167+0000 7fc7437fe640 1 --2- 172.21.15.83:0/3604881729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc744072330 0x7fc74415cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:41.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.167+0000 7fc743fff640 1 --2- 172.21.15.83:0/3604881729 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc74415ba30 0x7fc744154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:41.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.167+0000 7fc742ffd640 1 -- 172.21.15.83:0/3604881729 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc74415ba30 msgr2=0x7fc744154c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:41.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.167+0000 7fc742ffd640 1 --2- 172.21.15.83:0/3604881729 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc74415ba30 0x7fc744154c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:41.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.167+0000 7fc742ffd640 1 -- 172.21.15.83:0/3604881729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc744072330 msgr2=0x7fc74415cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:41.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.167+0000 7fc742ffd640 1 --2- 172.21.15.83:0/3604881729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc744072330 0x7fc74415cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:41.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.167+0000 7fc742ffd640 1 -- 172.21.15.83:0/3604881729 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7440ce5b0 con 0x7fc7440cfa10 2024-04-17T10:51:41.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.167+0000 7fc7437fe640 1 --2- 172.21.15.83:0/3604881729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc744072330 0x7fc74415cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:51:41.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.167+0000 7fc742ffd640 1 --2- 172.21.15.83:0/3604881729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7440cfa10 0x7fc744154740 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7fc73801ab80 tx=0x7fc73801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:41.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.167+0000 7fc72bfff640 1 -- 172.21.15.83:0/3604881729 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7380b1040 con 0x7fc7440cfa10 2024-04-17T10:51:41.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.167+0000 7fc72bfff640 1 -- 172.21.15.83:0/3604881729 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7380bc070 con 0x7fc7440cfa10 2024-04-17T10:51:41.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.167+0000 7fc74a185640 1 -- 172.21.15.83:0/3604881729 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7440ce890 con 0x7fc7440cfa10 2024-04-17T10:51:41.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.167+0000 7fc72bfff640 1 -- 172.21.15.83:0/3604881729 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7380b7690 con 0x7fc7440cfa10 2024-04-17T10:51:41.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.167+0000 7fc74a185640 1 -- 172.21.15.83:0/3604881729 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7440cee50 con 0x7fc7440cfa10 2024-04-17T10:51:41.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.167+0000 7fc74a185640 1 -- 172.21.15.83:0/3604881729 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc744074f90 con 0x7fc7440cfa10 2024-04-17T10:51:41.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.167+0000 7fc72bfff640 1 -- 172.21.15.83:0/3604881729 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc7380b7830 con 0x7fc7440cfa10 2024-04-17T10:51:41.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.167+0000 7fc72bfff640 1 --2- 172.21.15.83:0/3604881729 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc730066f60 0x7fc730069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:41.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.167+0000 7fc72bfff640 1 -- 172.21.15.83:0/3604881729 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fc73811f290 con 0x7fc7440cfa10 2024-04-17T10:51:41.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.167+0000 7fc7437fe640 1 --2- 172.21.15.83:0/3604881729 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc730066f60 0x7fc730069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:41.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.167+0000 7fc7437fe640 1 --2- 172.21.15.83:0/3604881729 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc730066f60 0x7fc730069420 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc7441f7a60 tx=0x7fc734009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:41.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.171+0000 7fc72bfff640 1 -- 172.21.15.83:0/3604881729 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc7380c6020 con 0x7fc7440cfa10 2024-04-17T10:51:41.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.367+0000 7fc74a185640 1 -- 172.21.15.83:0/3604881729 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc744159790 con 0x7fc7440cfa10 2024-04-17T10:51:41.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.367+0000 7fc72bfff640 1 -- 172.21.15.83:0/3604881729 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fc738016150 con 0x7fc7440cfa10 2024-04-17T10:51:41.361 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:51:41.361 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:51:41.361 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:51:41.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.367+0000 7fc74a185640 1 -- 172.21.15.83:0/3604881729 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc730066f60 msgr2=0x7fc730069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:41.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.367+0000 7fc74a185640 1 --2- 172.21.15.83:0/3604881729 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc730066f60 0x7fc730069420 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc7441f7a60 tx=0x7fc734009040 comp rx=0 tx=0).stop 2024-04-17T10:51:41.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.367+0000 7fc74a185640 1 -- 172.21.15.83:0/3604881729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7440cfa10 msgr2=0x7fc744154740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:41.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.367+0000 7fc74a185640 1 --2- 172.21.15.83:0/3604881729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7440cfa10 0x7fc744154740 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7fc73801ab80 tx=0x7fc73801ad10 comp rx=0 tx=0).stop 2024-04-17T10:51:41.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.371+0000 7fc74a185640 1 -- 172.21.15.83:0/3604881729 shutdown_connections 2024-04-17T10:51:41.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.371+0000 7fc74a185640 1 --2- 172.21.15.83:0/3604881729 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc730066f60 0x7fc730069420 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:41.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.371+0000 7fc74a185640 1 --2- 172.21.15.83:0/3604881729 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc74415ba30 0x7fc744154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:41.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.371+0000 7fc74a185640 1 --2- 172.21.15.83:0/3604881729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7440cfa10 0x7fc744154740 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:41.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.371+0000 7fc74a185640 1 --2- 172.21.15.83:0/3604881729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc744072330 0x7fc74415cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:41.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.371+0000 7fc74a185640 1 -- 172.21.15.83:0/3604881729 >> 172.21.15.83:0/3604881729 conn(0x7fc74406abe0 msgr2=0x7fc744159a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:41.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.371+0000 7fc74a185640 1 -- 172.21.15.83:0/3604881729 shutdown_connections 2024-04-17T10:51:41.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:41.371+0000 7fc74a185640 1 -- 172.21.15.83:0/3604881729 wait complete. 2024-04-17T10:51:41.783 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:51:41.783 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:51:42.784 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:51:42.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.923+0000 7fee3b670640 1 Processor -- start 2024-04-17T10:51:42.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.923+0000 7fee3b670640 1 -- start start 2024-04-17T10:51:42.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.923+0000 7fee3b670640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee341527c0 0x7fee3415ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:42.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.923+0000 7fee3b670640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fee340acec0 0x7fee340ad330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:42.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.923+0000 7fee3b670640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee3415c9b0 con 0x7fee34152ba0 2024-04-17T10:51:42.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.923+0000 7fee3b670640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee340ad870 con 0x7fee340acec0 2024-04-17T10:51:42.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.923+0000 7fee3b670640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee3415cc90 con 0x7fee341527c0 2024-04-17T10:51:42.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.923+0000 7fee39c0d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fee340acec0 0x7fee340ad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:42.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.923+0000 7fee3940c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee341527c0 0x7fee3415ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:42.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.923+0000 7fee38c0b640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fee34152ba0 0x7fee3415bf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:58504/0 (socket says 172.21.15.83:58504) 2024-04-17T10:51:42.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.923+0000 7fee38c0b640 1 -- 172.21.15.83:0/1254324683 learned_addr learned my addr 172.21.15.83:0/1254324683 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:51:42.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.923+0000 7fee39c0d640 1 -- 172.21.15.83:0/1254324683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee341527c0 msgr2=0x7fee3415ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:42.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.923+0000 7fee39c0d640 1 --2- 172.21.15.83:0/1254324683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee341527c0 0x7fee3415ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:42.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.923+0000 7fee39c0d640 1 -- 172.21.15.83:0/1254324683 >> v1:172.21.15.167:6789/0 conn(0x7fee34152ba0 legacy=0x7fee3415bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:42.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.923+0000 7fee39c0d640 1 -- 172.21.15.83:0/1254324683 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3415f710 con 0x7fee340acec0 2024-04-17T10:51:42.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.923+0000 7fee3940c640 1 --2- 172.21.15.83:0/1254324683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee341527c0 0x7fee3415ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:51:42.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.927+0000 7fee39c0d640 1 --2- 172.21.15.83:0/1254324683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee340acec0 0x7fee340ad330 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fee2c009d90 tx=0x7fee2c01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46dc551d3b3fc0aa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:42.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.927+0000 7fee2bfff640 1 -- 172.21.15.83:0/1254324683 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c0b2070 con 0x7fee340acec0 2024-04-17T10:51:42.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.927+0000 7fee2bfff640 1 -- 172.21.15.83:0/1254324683 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2c0a5940 con 0x7fee340acec0 2024-04-17T10:51:42.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.927+0000 7fee2bfff640 1 -- 172.21.15.83:0/1254324683 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c0a5d10 con 0x7fee340acec0 2024-04-17T10:51:42.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.927+0000 7fee3b670640 1 -- 172.21.15.83:0/1254324683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee340acec0 msgr2=0x7fee340ad330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:42.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.927+0000 7fee3b670640 1 --2- 172.21.15.83:0/1254324683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee340acec0 0x7fee340ad330 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fee2c009d90 tx=0x7fee2c01c970 comp rx=0 tx=0).stop 2024-04-17T10:51:42.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.927+0000 7fee3b670640 1 -- 172.21.15.83:0/1254324683 shutdown_connections 2024-04-17T10:51:42.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.927+0000 7fee3b670640 1 --2- 172.21.15.83:0/1254324683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee340acec0 0x7fee340ad330 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:42.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.927+0000 7fee3b670640 1 --2- 172.21.15.83:0/1254324683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee341527c0 0x7fee3415ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:42.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.927+0000 7fee3b670640 1 -- 172.21.15.83:0/1254324683 >> 172.21.15.83:0/1254324683 conn(0x7fee340af650 msgr2=0x7fee340cff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:42.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.927+0000 7fee3b670640 1 -- 172.21.15.83:0/1254324683 shutdown_connections 2024-04-17T10:51:42.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.927+0000 7fee3b670640 1 -- 172.21.15.83:0/1254324683 wait complete. 2024-04-17T10:51:42.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.927+0000 7fee3b670640 1 Processor -- start 2024-04-17T10:51:42.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.927+0000 7fee3b670640 1 -- start start 2024-04-17T10:51:42.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.931+0000 7fee3b670640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee340acec0 0x7fee3406e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:42.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.931+0000 7fee3b670640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee341527c0 0x7fee340cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:42.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.931+0000 7fee3b670640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee34152ba0 0x7fee340cde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:42.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.931+0000 7fee3b670640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3406b570 con 0x7fee341527c0 2024-04-17T10:51:42.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.931+0000 7fee3b670640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3406b3f0 con 0x7fee34152ba0 2024-04-17T10:51:42.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.931+0000 7fee3b670640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee3406b6f0 con 0x7fee340acec0 2024-04-17T10:51:42.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.931+0000 7fee38c0b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee341527c0 0x7fee340cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:42.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.931+0000 7fee38c0b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee341527c0 0x7fee340cc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46512/0 (socket says 172.21.15.83:46512) 2024-04-17T10:51:42.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.931+0000 7fee38c0b640 1 -- 172.21.15.83:0/982197413 learned_addr learned my addr 172.21.15.83:0/982197413 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:42.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.931+0000 7fee3940c640 1 --2- 172.21.15.83:0/982197413 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee340acec0 0x7fee3406e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:42.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.931+0000 7fee38c0b640 1 -- 172.21.15.83:0/982197413 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee340acec0 msgr2=0x7fee3406e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:42.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.931+0000 7fee38c0b640 1 --2- 172.21.15.83:0/982197413 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee340acec0 0x7fee3406e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:42.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.931+0000 7fee38c0b640 1 -- 172.21.15.83:0/982197413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee34152ba0 msgr2=0x7fee340cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:42.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.931+0000 7fee38c0b640 1 --2- 172.21.15.83:0/982197413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee34152ba0 0x7fee340cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:42.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.931+0000 7fee38c0b640 1 -- 172.21.15.83:0/982197413 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee340ccda0 con 0x7fee341527c0 2024-04-17T10:51:42.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.931+0000 7fee38c0b640 1 --2- 172.21.15.83:0/982197413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee341527c0 0x7fee340cc630 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7fee3000c3a0 tx=0x7fee3000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:42.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.931+0000 7fee29ffb640 1 -- 172.21.15.83:0/982197413 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee30017070 con 0x7fee341527c0 2024-04-17T10:51:42.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.931+0000 7fee29ffb640 1 -- 172.21.15.83:0/982197413 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee30005cf0 con 0x7fee341527c0 2024-04-17T10:51:42.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.931+0000 7fee3b670640 1 -- 172.21.15.83:0/982197413 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee340cd080 con 0x7fee341527c0 2024-04-17T10:51:42.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.931+0000 7fee29ffb640 1 -- 172.21.15.83:0/982197413 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee300072f0 con 0x7fee341527c0 2024-04-17T10:51:42.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.931+0000 7fee3b670640 1 -- 172.21.15.83:0/982197413 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee340cd640 con 0x7fee341527c0 2024-04-17T10:51:42.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.931+0000 7fee3b670640 1 -- 172.21.15.83:0/982197413 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee34074f90 con 0x7fee341527c0 2024-04-17T10:51:42.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.935+0000 7fee29ffb640 1 -- 172.21.15.83:0/982197413 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fee3001a070 con 0x7fee341527c0 2024-04-17T10:51:42.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.935+0000 7fee29ffb640 1 --2- 172.21.15.83:0/982197413 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fee18066f60 0x7fee18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:42.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.935+0000 7fee3940c640 1 --2- 172.21.15.83:0/982197413 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fee18066f60 0x7fee18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:42.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.935+0000 7fee29ffb640 1 -- 172.21.15.83:0/982197413 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fee30084490 con 0x7fee341527c0 2024-04-17T10:51:42.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.935+0000 7fee3940c640 1 --2- 172.21.15.83:0/982197413 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fee18066f60 0x7fee18069420 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fee24009580 tx=0x7fee24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:42.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:42.939+0000 7fee29ffb640 1 -- 172.21.15.83:0/982197413 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fee300553b0 con 0x7fee341527c0 2024-04-17T10:51:43.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:43.123+0000 7fee3b670640 1 -- 172.21.15.83:0/982197413 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee3406c160 con 0x7fee341527c0 2024-04-17T10:51:43.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:43.127+0000 7fee29ffb640 1 -- 172.21.15.83:0/982197413 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fee30054cb0 con 0x7fee341527c0 2024-04-17T10:51:43.120 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:51:43.120 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:51:43.120 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:51:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:43.127+0000 7fee3b670640 1 -- 172.21.15.83:0/982197413 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fee18066f60 msgr2=0x7fee18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:43.127+0000 7fee3b670640 1 --2- 172.21.15.83:0/982197413 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fee18066f60 0x7fee18069420 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fee24009580 tx=0x7fee24008040 comp rx=0 tx=0).stop 2024-04-17T10:51:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:43.127+0000 7fee3b670640 1 -- 172.21.15.83:0/982197413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee341527c0 msgr2=0x7fee340cc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:43.127+0000 7fee3b670640 1 --2- 172.21.15.83:0/982197413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee341527c0 0x7fee340cc630 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7fee3000c3a0 tx=0x7fee3000c860 comp rx=0 tx=0).stop 2024-04-17T10:51:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:43.127+0000 7fee3b670640 1 -- 172.21.15.83:0/982197413 shutdown_connections 2024-04-17T10:51:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:43.127+0000 7fee3b670640 1 --2- 172.21.15.83:0/982197413 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fee18066f60 0x7fee18069420 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:43.127+0000 7fee3b670640 1 --2- 172.21.15.83:0/982197413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee34152ba0 0x7fee340cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:43.127+0000 7fee3b670640 1 --2- 172.21.15.83:0/982197413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee341527c0 0x7fee340cc630 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:43.127+0000 7fee3b670640 1 --2- 172.21.15.83:0/982197413 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee340acec0 0x7fee3406e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:43.127+0000 7fee3b670640 1 -- 172.21.15.83:0/982197413 >> 172.21.15.83:0/982197413 conn(0x7fee340af650 msgr2=0x7fee340b4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:43.127+0000 7fee3b670640 1 -- 172.21.15.83:0/982197413 shutdown_connections 2024-04-17T10:51:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:43.127+0000 7fee3b670640 1 -- 172.21.15.83:0/982197413 wait complete. 2024-04-17T10:51:43.540 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:51:43.540 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:51:44.542 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:51:44.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.675+0000 7fd6d0cbe640 1 Processor -- start 2024-04-17T10:51:44.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.675+0000 7fd6d0cbe640 1 -- start start 2024-04-17T10:51:44.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.679+0000 7fd6d0cbe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd6cc15ba30 0x7fd6cc072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:44.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.679+0000 7fd6d0cbe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6cc15c960 0x7fd6cc0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:44.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.679+0000 7fd6d0cbe640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6cc0cfea0 con 0x7fd6cc0cfa10 2024-04-17T10:51:44.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.679+0000 7fd6d0cbe640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6cc071ea0 con 0x7fd6cc15ba30 2024-04-17T10:51:44.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.679+0000 7fd6d0cbe640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc06e320 con 0x7fd6cc15c960 2024-04-17T10:51:44.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.679+0000 7fd6ca59c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd6cc15ba30 0x7fd6cc072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:44.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.679+0000 7fd6cad9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd6cc0cfa10 0x7fd6cc06c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35138/0 (socket says 172.21.15.83:35138) 2024-04-17T10:51:44.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.679+0000 7fd6ca59c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd6cc15ba30 0x7fd6cc072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46524/0 (socket says 172.21.15.83:46524) 2024-04-17T10:51:44.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.679+0000 7fd6cad9d640 1 -- 172.21.15.83:0/1438455002 learned_addr learned my addr 172.21.15.83:0/1438455002 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:51:44.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.679+0000 7fd6c1d9b640 1 --2- 172.21.15.83:0/1438455002 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6cc15c960 0x7fd6cc0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:44.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.679+0000 7fd6ca59c640 1 -- 172.21.15.83:0/1438455002 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6cc15c960 msgr2=0x7fd6cc0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:44.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.679+0000 7fd6ca59c640 1 --2- 172.21.15.83:0/1438455002 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6cc15c960 0x7fd6cc0cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:44.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.679+0000 7fd6ca59c640 1 -- 172.21.15.83:0/1438455002 >> v1:172.21.15.83:6789/0 conn(0x7fd6cc0cfa10 legacy=0x7fd6cc06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:44.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.679+0000 7fd6c9d9b640 1 -- 172.21.15.83:0/1438455002 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1941550898 0 0) 0x7fd6cc0cfea0 con 0x7fd6cc0cfa10 2024-04-17T10:51:44.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.679+0000 7fd6ca59c640 1 -- 172.21.15.83:0/1438455002 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6cc06e040 con 0x7fd6cc15ba30 2024-04-17T10:51:44.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.679+0000 7fd6c1d9b640 1 --2- 172.21.15.83:0/1438455002 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6cc15c960 0x7fd6cc0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:51:44.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.679+0000 7fd6ca59c640 1 --2- 172.21.15.83:0/1438455002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6cc15ba30 0x7fd6cc072330 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc0097c0 tx=0x7fd6bc01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76d4947b21a7447 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:44.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.679+0000 7fd6c9d9b640 1 -- 172.21.15.83:0/1438455002 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc0b0070 con 0x7fd6cc15ba30 2024-04-17T10:51:44.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.679+0000 7fd6c9d9b640 1 -- 172.21.15.83:0/1438455002 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6bc0ab600 con 0x7fd6cc15ba30 2024-04-17T10:51:44.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.679+0000 7fd6c9d9b640 1 -- 172.21.15.83:0/1438455002 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc0ab9d0 con 0x7fd6cc15ba30 2024-04-17T10:51:44.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6d0cbe640 1 -- 172.21.15.83:0/1438455002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6cc15ba30 msgr2=0x7fd6cc072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:44.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6d0cbe640 1 --2- 172.21.15.83:0/1438455002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6cc15ba30 0x7fd6cc072330 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc0097c0 tx=0x7fd6bc01c410 comp rx=0 tx=0).stop 2024-04-17T10:51:44.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6d0cbe640 1 -- 172.21.15.83:0/1438455002 shutdown_connections 2024-04-17T10:51:44.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6d0cbe640 1 --2- 172.21.15.83:0/1438455002 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6cc15c960 0x7fd6cc0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:44.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6d0cbe640 1 --2- 172.21.15.83:0/1438455002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6cc15ba30 0x7fd6cc072330 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:44.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6d0cbe640 1 -- 172.21.15.83:0/1438455002 >> 172.21.15.83:0/1438455002 conn(0x7fd6cc06abe0 msgr2=0x7fd6cc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:44.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6d0cbe640 1 -- 172.21.15.83:0/1438455002 shutdown_connections 2024-04-17T10:51:44.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6d0cbe640 1 -- 172.21.15.83:0/1438455002 wait complete. 2024-04-17T10:51:44.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6d0cbe640 1 Processor -- start 2024-04-17T10:51:44.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6d0cbe640 1 -- start start 2024-04-17T10:51:44.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6d0cbe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6cc0cfa10 0x7fd6cc153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:44.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6d0cbe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6cc15ba30 0x7fd6cc151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:44.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6d0cbe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6cc15c960 0x7fd6cc0b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:44.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6d0cbe640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc06de20 con 0x7fd6cc0cfa10 2024-04-17T10:51:44.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6d0cbe640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc06dca0 con 0x7fd6cc15ba30 2024-04-17T10:51:44.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6d0cbe640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc15cd40 con 0x7fd6cc15c960 2024-04-17T10:51:44.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6ca59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6cc0cfa10 0x7fd6cc153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:44.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6ca59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6cc0cfa10 0x7fd6cc153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46540/0 (socket says 172.21.15.83:46540) 2024-04-17T10:51:44.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6ca59c640 1 -- 172.21.15.83:0/2013499195 learned_addr learned my addr 172.21.15.83:0/2013499195 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:44.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6c1d9b640 1 --2- 172.21.15.83:0/2013499195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6cc15ba30 0x7fd6cc151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:44.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6ca59c640 1 -- 172.21.15.83:0/2013499195 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6cc15c960 msgr2=0x7fd6cc0b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:44.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6cad9d640 1 --2- 172.21.15.83:0/2013499195 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6cc15c960 0x7fd6cc0b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:44.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6ca59c640 1 --2- 172.21.15.83:0/2013499195 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6cc15c960 0x7fd6cc0b41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:44.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6ca59c640 1 -- 172.21.15.83:0/2013499195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6cc15ba30 msgr2=0x7fd6cc151d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:44.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6ca59c640 1 --2- 172.21.15.83:0/2013499195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6cc15ba30 0x7fd6cc151d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:44.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6ca59c640 1 -- 172.21.15.83:0/2013499195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6cc152250 con 0x7fd6cc0cfa10 2024-04-17T10:51:44.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6c1d9b640 1 --2- 172.21.15.83:0/2013499195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6cc15ba30 0x7fd6cc151d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:51:44.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6ca59c640 1 --2- 172.21.15.83:0/2013499195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6cc0cfa10 0x7fd6cc153dd0 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc0ae040 tx=0x7fd6bc0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:44.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6c37fe640 1 -- 172.21.15.83:0/2013499195 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc0b0050 con 0x7fd6cc0cfa10 2024-04-17T10:51:44.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6c37fe640 1 -- 172.21.15.83:0/2013499195 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6bc0bc070 con 0x7fd6cc0cfa10 2024-04-17T10:51:44.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6d0cbe640 1 -- 172.21.15.83:0/2013499195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6cc1543b0 con 0x7fd6cc0cfa10 2024-04-17T10:51:44.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6c37fe640 1 -- 172.21.15.83:0/2013499195 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc0b5a50 con 0x7fd6cc0cfa10 2024-04-17T10:51:44.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.683+0000 7fd6d0cbe640 1 -- 172.21.15.83:0/2013499195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6cc154970 con 0x7fd6cc0cfa10 2024-04-17T10:51:44.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.687+0000 7fd6d0cbe640 1 -- 172.21.15.83:0/2013499195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6cc074f90 con 0x7fd6cc0cfa10 2024-04-17T10:51:44.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.687+0000 7fd6c37fe640 1 -- 172.21.15.83:0/2013499195 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd6bc0ab4a0 con 0x7fd6cc0cfa10 2024-04-17T10:51:44.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.687+0000 7fd6c37fe640 1 --2- 172.21.15.83:0/2013499195 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd6ac066f60 0x7fd6ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:44.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.687+0000 7fd6c1d9b640 1 --2- 172.21.15.83:0/2013499195 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd6ac066f60 0x7fd6ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:44.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.687+0000 7fd6c37fe640 1 -- 172.21.15.83:0/2013499195 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fd6bc11f980 con 0x7fd6cc0cfa10 2024-04-17T10:51:44.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.687+0000 7fd6c1d9b640 1 --2- 172.21.15.83:0/2013499195 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd6ac066f60 0x7fd6ac069420 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc15d870 tx=0x7fd6b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:44.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.691+0000 7fd6c37fe640 1 -- 172.21.15.83:0/2013499195 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd6bc016090 con 0x7fd6cc0cfa10 2024-04-17T10:51:44.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.871+0000 7fd6d0cbe640 1 -- 172.21.15.83:0/2013499195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6cc0b0070 con 0x7fd6cc0cfa10 2024-04-17T10:51:44.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.871+0000 7fd6c37fe640 1 -- 172.21.15.83:0/2013499195 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fd6bc0f08a0 con 0x7fd6cc0cfa10 2024-04-17T10:51:44.865 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:51:44.865 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:51:44.865 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:51:44.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.871+0000 7fd6d0cbe640 1 -- 172.21.15.83:0/2013499195 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd6ac066f60 msgr2=0x7fd6ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:44.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.871+0000 7fd6d0cbe640 1 --2- 172.21.15.83:0/2013499195 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd6ac066f60 0x7fd6ac069420 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc15d870 tx=0x7fd6b4008040 comp rx=0 tx=0).stop 2024-04-17T10:51:44.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.871+0000 7fd6d0cbe640 1 -- 172.21.15.83:0/2013499195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6cc0cfa10 msgr2=0x7fd6cc153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:44.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.871+0000 7fd6d0cbe640 1 --2- 172.21.15.83:0/2013499195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6cc0cfa10 0x7fd6cc153dd0 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc0ae040 tx=0x7fd6bc0191b0 comp rx=0 tx=0).stop 2024-04-17T10:51:44.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.871+0000 7fd6d0cbe640 1 -- 172.21.15.83:0/2013499195 shutdown_connections 2024-04-17T10:51:44.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.871+0000 7fd6d0cbe640 1 --2- 172.21.15.83:0/2013499195 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd6ac066f60 0x7fd6ac069420 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:44.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.871+0000 7fd6d0cbe640 1 --2- 172.21.15.83:0/2013499195 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6cc15c960 0x7fd6cc0b41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:44.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.875+0000 7fd6d0cbe640 1 --2- 172.21.15.83:0/2013499195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6cc15ba30 0x7fd6cc151d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:44.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.875+0000 7fd6d0cbe640 1 --2- 172.21.15.83:0/2013499195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6cc0cfa10 0x7fd6cc153dd0 unknown :-1 s=CLOSED pgs=2164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:44.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.875+0000 7fd6d0cbe640 1 -- 172.21.15.83:0/2013499195 >> 172.21.15.83:0/2013499195 conn(0x7fd6cc06abe0 msgr2=0x7fd6cc15d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:44.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.875+0000 7fd6d0cbe640 1 -- 172.21.15.83:0/2013499195 shutdown_connections 2024-04-17T10:51:44.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:44.875+0000 7fd6d0cbe640 1 -- 172.21.15.83:0/2013499195 wait complete. 2024-04-17T10:51:45.284 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:51:45.285 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:51:46.285 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:51:46.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c2203d640 1 Processor -- start 2024-04-17T10:51:46.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c2203d640 1 -- start start 2024-04-17T10:51:46.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c2203d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3c1c15ba30 0x7f3c1c072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:46.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c2203d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c1c0cfbe0 0x7f3c1c06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:46.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c2203d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c1c06e320 con 0x7f3c1c15c960 2024-04-17T10:51:46.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c2203d640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c1c071ea0 con 0x7f3c1c15ba30 2024-04-17T10:51:46.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c2203d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1c06e040 con 0x7f3c1c0cfbe0 2024-04-17T10:51:46.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c13fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3c1c15c960 0x7f3c1c0cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35152/0 (socket says 172.21.15.83:35152) 2024-04-17T10:51:46.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c13fff640 1 -- 172.21.15.83:0/913330031 learned_addr learned my addr 172.21.15.83:0/913330031 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:51:46.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c1affd640 1 -- 172.21.15.83:0/913330031 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3599847869 0 0) 0x7f3c1c06e320 con 0x7f3c1c15c960 2024-04-17T10:51:46.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c1affd640 1 -- 172.21.15.83:0/913330031 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c08003530 con 0x7f3c1c15c960 2024-04-17T10:51:46.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c1bfff640 1 --2- 172.21.15.83:0/913330031 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c1c0cfbe0 0x7f3c1c06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:46.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c1b7fe640 1 --2- 172.21.15.83:0/913330031 >> v2:172.21.15.167:3300/0 conn(0x7f3c1c15ba30 0x7f3c1c072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:46.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c1affd640 1 -- 172.21.15.83:0/913330031 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1889328049 0 0) 0x7f3c08003530 con 0x7f3c1c15c960 2024-04-17T10:51:46.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c1affd640 1 -- 172.21.15.83:0/913330031 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c1c0cfbe0 msgr2=0x7f3c1c06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:46.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c1affd640 1 --2- 172.21.15.83:0/913330031 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c1c0cfbe0 0x7f3c1c06c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:46.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c1affd640 1 -- 172.21.15.83:0/913330031 >> v2:172.21.15.167:3300/0 conn(0x7f3c1c15ba30 msgr2=0x7f3c1c072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:46.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c1affd640 1 --2- 172.21.15.83:0/913330031 >> v2:172.21.15.167:3300/0 conn(0x7f3c1c15ba30 0x7f3c1c072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:46.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c1affd640 1 -- 172.21.15.83:0/913330031 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c1c071850 con 0x7f3c1c15c960 2024-04-17T10:51:46.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c1affd640 1 -- 172.21.15.83:0/913330031 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f3c04001680 con 0x7f3c1c15c960 2024-04-17T10:51:46.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c1bfff640 1 --2- 172.21.15.83:0/913330031 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c1c0cfbe0 0x7f3c1c06c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:51:46.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c1b7fe640 1 --2- 172.21.15.83:0/913330031 >> v2:172.21.15.167:3300/0 conn(0x7f3c1c15ba30 0x7f3c1c072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:51:46.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c1affd640 1 -- 172.21.15.83:0/913330031 >> v1:172.21.15.83:6789/0 conn(0x7f3c1c15c960 legacy=0x7f3c1c0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:46.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c1affd640 1 --2- 172.21.15.83:0/913330031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c080039c0 0x7f3c08003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:46.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c1affd640 1 --2- 172.21.15.83:0/913330031 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c1c0cfbe0 0x7f3c08004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:46.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c1affd640 1 --2- 172.21.15.83:0/913330031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c08004d50 0x7f3c080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:46.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c1affd640 1 -- 172.21.15.83:0/913330031 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1c071ea0 con 0x7f3c080039c0 2024-04-17T10:51:46.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c1affd640 1 -- 172.21.15.83:0/913330031 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1c06e040 con 0x7f3c08004d50 2024-04-17T10:51:46.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c1affd640 1 -- 172.21.15.83:0/913330031 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0800bad0 con 0x7f3c1c0cfbe0 2024-04-17T10:51:46.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c1b7fe640 1 --2- 172.21.15.83:0/913330031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c080039c0 0x7f3c08003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:46.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c1bfff640 1 --2- 172.21.15.83:0/913330031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c08004d50 0x7f3c080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:46.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c13fff640 1 --2- 172.21.15.83:0/913330031 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c1c0cfbe0 0x7f3c08004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:46.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c1b7fe640 1 -- 172.21.15.83:0/913330031 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c1c0cfbe0 msgr2=0x7f3c08004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:46.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c1b7fe640 1 --2- 172.21.15.83:0/913330031 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c1c0cfbe0 0x7f3c08004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:46.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c1b7fe640 1 -- 172.21.15.83:0/913330031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c08004d50 msgr2=0x7f3c080051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:46.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c1b7fe640 1 --2- 172.21.15.83:0/913330031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c08004d50 0x7f3c080051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:46.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.423+0000 7f3c1b7fe640 1 -- 172.21.15.83:0/913330031 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c08003530 con 0x7f3c080039c0 2024-04-17T10:51:46.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.427+0000 7f3c1b7fe640 1 --2- 172.21.15.83:0/913330031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c080039c0 0x7f3c08003de0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0098d0 tx=0x7f3c0c01c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1592ddf164071417 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:46.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.427+0000 7f3c1affd640 1 -- 172.21.15.83:0/913330031 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c00f040 con 0x7f3c080039c0 2024-04-17T10:51:46.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.427+0000 7f3c1affd640 1 -- 172.21.15.83:0/913330031 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c0c0af070 con 0x7f3c080039c0 2024-04-17T10:51:46.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.427+0000 7f3c1affd640 1 -- 172.21.15.83:0/913330031 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c0aa630 con 0x7f3c080039c0 2024-04-17T10:51:46.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.427+0000 7f3c2203d640 1 -- 172.21.15.83:0/913330031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c080039c0 msgr2=0x7f3c08003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:46.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.427+0000 7f3c2203d640 1 --2- 172.21.15.83:0/913330031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c080039c0 0x7f3c08003de0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0098d0 tx=0x7f3c0c01c650 comp rx=0 tx=0).stop 2024-04-17T10:51:46.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.427+0000 7f3c1bfff640 1 -- 172.21.15.83:0/913330031 reap_dead start 2024-04-17T10:51:46.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.427+0000 7f3c2203d640 1 -- 172.21.15.83:0/913330031 shutdown_connections 2024-04-17T10:51:46.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.427+0000 7f3c2203d640 1 -- 172.21.15.83:0/913330031 >> 172.21.15.83:0/913330031 conn(0x7f3c1c06abe0 msgr2=0x7f3c1c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:46.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.427+0000 7f3c2203d640 1 -- 172.21.15.83:0/913330031 shutdown_connections 2024-04-17T10:51:46.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.427+0000 7f3c2203d640 1 -- 172.21.15.83:0/913330031 wait complete. 2024-04-17T10:51:46.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.427+0000 7f3c2203d640 1 Processor -- start 2024-04-17T10:51:46.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.427+0000 7f3c2203d640 1 -- start start 2024-04-17T10:51:46.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.427+0000 7f3c2203d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c080039c0 0x7f3c1c0adc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:46.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.427+0000 7f3c2203d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c1c070470 0x7f3c1c0708f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:46.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.427+0000 7f3c2203d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c1c152c30 0x7f3c1c1530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:46.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.427+0000 7f3c2203d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1c15cee0 con 0x7f3c1c070470 2024-04-17T10:51:46.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.427+0000 7f3c2203d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1c15cd60 con 0x7f3c080039c0 2024-04-17T10:51:46.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.427+0000 7f3c2203d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1c06cca0 con 0x7f3c1c152c30 2024-04-17T10:51:46.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.427+0000 7f3c13fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c1c070470 0x7f3c1c0708f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:46.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.427+0000 7f3c13fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c1c070470 0x7f3c1c0708f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46556/0 (socket says 172.21.15.83:46556) 2024-04-17T10:51:46.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.427+0000 7f3c13fff640 1 -- 172.21.15.83:0/3266935738 learned_addr learned my addr 172.21.15.83:0/3266935738 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:46.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.431+0000 7f3c1b7fe640 1 --2- 172.21.15.83:0/3266935738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c080039c0 0x7f3c1c0adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:46.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.431+0000 7f3c1bfff640 1 --2- 172.21.15.83:0/3266935738 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c1c152c30 0x7f3c1c1530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:46.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.431+0000 7f3c13fff640 1 -- 172.21.15.83:0/3266935738 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c1c152c30 msgr2=0x7f3c1c1530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:46.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.431+0000 7f3c13fff640 1 --2- 172.21.15.83:0/3266935738 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c1c152c30 0x7f3c1c1530c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:46.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.431+0000 7f3c13fff640 1 -- 172.21.15.83:0/3266935738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c080039c0 msgr2=0x7f3c1c0adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:46.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.431+0000 7f3c13fff640 1 --2- 172.21.15.83:0/3266935738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c080039c0 0x7f3c1c0adc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:46.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.431+0000 7f3c13fff640 1 -- 172.21.15.83:0/3266935738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c1c0cf3f0 con 0x7f3c1c070470 2024-04-17T10:51:46.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.431+0000 7f3c1bfff640 1 --2- 172.21.15.83:0/3266935738 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c1c152c30 0x7f3c1c1530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:51:46.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.431+0000 7f3c1b7fe640 1 --2- 172.21.15.83:0/3266935738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c080039c0 0x7f3c1c0adc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:51:46.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.431+0000 7f3c13fff640 1 --2- 172.21.15.83:0/3266935738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c1c070470 0x7f3c1c0708f0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400e850 tx=0x7f3c0400ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:46.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.431+0000 7f3c18ff9640 1 -- 172.21.15.83:0/3266935738 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0400c870 con 0x7f3c1c070470 2024-04-17T10:51:46.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.431+0000 7f3c18ff9640 1 -- 172.21.15.83:0/3266935738 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c04018070 con 0x7f3c1c070470 2024-04-17T10:51:46.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.431+0000 7f3c2203d640 1 -- 172.21.15.83:0/3266935738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c1c0ce630 con 0x7f3c1c070470 2024-04-17T10:51:46.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.431+0000 7f3c18ff9640 1 -- 172.21.15.83:0/3266935738 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0400cc50 con 0x7f3c1c070470 2024-04-17T10:51:46.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.431+0000 7f3c2203d640 1 -- 172.21.15.83:0/3266935738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c1c0cebf0 con 0x7f3c1c070470 2024-04-17T10:51:46.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.431+0000 7f3c2203d640 1 -- 172.21.15.83:0/3266935738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c1c070e30 con 0x7f3c1c070470 2024-04-17T10:51:46.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.431+0000 7f3c18ff9640 1 -- 172.21.15.83:0/3266935738 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3c04013420 con 0x7f3c1c070470 2024-04-17T10:51:46.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.431+0000 7f3c18ff9640 1 --2- 172.21.15.83:0/3266935738 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3bfc066f90 0x7f3bfc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:46.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.431+0000 7f3c18ff9640 1 -- 172.21.15.83:0/3266935738 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f3c04084c90 con 0x7f3c1c070470 2024-04-17T10:51:46.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.431+0000 7f3c1b7fe640 1 --2- 172.21.15.83:0/3266935738 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3bfc066f90 0x7f3bfc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:46.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.431+0000 7f3c1b7fe640 1 --2- 172.21.15.83:0/3266935738 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3bfc066f90 0x7f3bfc069450 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0ae000 tx=0x7f3c0c0046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:46.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.435+0000 7f3c18ff9640 1 -- 172.21.15.83:0/3266935738 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3c04055cc0 con 0x7f3c1c070470 2024-04-17T10:51:46.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.627+0000 7f3c2203d640 1 -- 172.21.15.83:0/3266935738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c1c074f90 con 0x7f3c1c070470 2024-04-17T10:51:46.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.627+0000 7f3c18ff9640 1 -- 172.21.15.83:0/3266935738 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f3c040555c0 con 0x7f3c1c070470 2024-04-17T10:51:46.621 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:51:46.621 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:51:46.621 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:51:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.627+0000 7f3c2203d640 1 -- 172.21.15.83:0/3266935738 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3bfc066f90 msgr2=0x7f3bfc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.627+0000 7f3c2203d640 1 --2- 172.21.15.83:0/3266935738 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3bfc066f90 0x7f3bfc069450 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0ae000 tx=0x7f3c0c0046d0 comp rx=0 tx=0).stop 2024-04-17T10:51:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.627+0000 7f3c2203d640 1 -- 172.21.15.83:0/3266935738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c1c070470 msgr2=0x7f3c1c0708f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.627+0000 7f3c2203d640 1 --2- 172.21.15.83:0/3266935738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c1c070470 0x7f3c1c0708f0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400e850 tx=0x7f3c0400ed10 comp rx=0 tx=0).stop 2024-04-17T10:51:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.627+0000 7f3c2203d640 1 -- 172.21.15.83:0/3266935738 shutdown_connections 2024-04-17T10:51:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.627+0000 7f3c2203d640 1 --2- 172.21.15.83:0/3266935738 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3bfc066f90 0x7f3bfc069450 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.627+0000 7f3c2203d640 1 --2- 172.21.15.83:0/3266935738 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c1c152c30 0x7f3c1c1530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.627+0000 7f3c2203d640 1 --2- 172.21.15.83:0/3266935738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c1c070470 0x7f3c1c0708f0 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.627+0000 7f3c2203d640 1 --2- 172.21.15.83:0/3266935738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c080039c0 0x7f3c1c0adc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.627+0000 7f3c2203d640 1 -- 172.21.15.83:0/3266935738 >> 172.21.15.83:0/3266935738 conn(0x7f3c1c06abe0 msgr2=0x7f3c1c06d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.627+0000 7f3c2203d640 1 -- 172.21.15.83:0/3266935738 shutdown_connections 2024-04-17T10:51:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:46.627+0000 7f3c2203d640 1 -- 172.21.15.83:0/3266935738 wait complete. 2024-04-17T10:51:47.040 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:51:47.040 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:51:48.041 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:51:48.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.179+0000 7f30bcb22640 1 Processor -- start 2024-04-17T10:51:48.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.179+0000 7f30bcb22640 1 -- start start 2024-04-17T10:51:48.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.179+0000 7f30bcb22640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f30b815db50 0x7f30b8070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:48.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.179+0000 7f30bcb22640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30b80aedd0 0x7f30b806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:48.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.179+0000 7f30bcb22640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f30b806dae0 0x7f30b80cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:48.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.179+0000 7f30bcb22640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30b80709b0 con 0x7f30b815db50 2024-04-17T10:51:48.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.179+0000 7f30bcb22640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30b815e440 con 0x7f30b806dae0 2024-04-17T10:51:48.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.179+0000 7f30bcb22640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b815e720 con 0x7f30b80aedd0 2024-04-17T10:51:48.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.179+0000 7f30b659c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f30b815db50 0x7f30b8070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:48.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.179+0000 7f30b659c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f30b815db50 0x7f30b8070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59334/0 (socket says 172.21.15.83:59334) 2024-04-17T10:51:48.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.179+0000 7f30b659c640 1 -- 172.21.15.83:0/133464929 learned_addr learned my addr 172.21.15.83:0/133464929 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:48.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.179+0000 7f30b5d9b640 1 --2- 172.21.15.83:0/133464929 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30b80aedd0 0x7f30b806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:48.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.179+0000 7f30b6d9d640 1 --2- 172.21.15.83:0/133464929 >> v2:172.21.15.167:3300/0 conn(0x7f30b806dae0 0x7f30b80cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:48.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.179+0000 7f30b659c640 1 -- 172.21.15.83:0/133464929 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30b80aedd0 msgr2=0x7f30b806d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:48.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.179+0000 7f30b659c640 1 --2- 172.21.15.83:0/133464929 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30b80aedd0 0x7f30b806d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:48.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.179+0000 7f30b659c640 1 -- 172.21.15.83:0/133464929 >> v2:172.21.15.167:3300/0 conn(0x7f30b806dae0 msgr2=0x7f30b80cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:48.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.179+0000 7f30b659c640 1 --2- 172.21.15.83:0/133464929 >> v2:172.21.15.167:3300/0 conn(0x7f30b806dae0 0x7f30b80cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:48.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.179+0000 7f30b659c640 1 -- 172.21.15.83:0/133464929 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b815ea00 con 0x7f30b815db50 2024-04-17T10:51:48.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.183+0000 7f30b659c640 1 --2- 172.21.15.83:0/133464929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30b815db50 0x7f30b8070470 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f30b80b2ac0 tx=0x7f30a801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991b6042c42e84f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:48.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.183+0000 7f30b559a640 1 -- 172.21.15.83:0/133464929 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a80b0070 con 0x7f30b815db50 2024-04-17T10:51:48.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.183+0000 7f30b559a640 1 -- 172.21.15.83:0/133464929 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a80ab600 con 0x7f30b815db50 2024-04-17T10:51:48.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.183+0000 7f30b559a640 1 -- 172.21.15.83:0/133464929 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a80ab9d0 con 0x7f30b815db50 2024-04-17T10:51:48.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.183+0000 7f30bcb22640 1 -- 172.21.15.83:0/133464929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30b815db50 msgr2=0x7f30b8070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:48.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.183+0000 7f30bcb22640 1 --2- 172.21.15.83:0/133464929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30b815db50 0x7f30b8070470 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f30b80b2ac0 tx=0x7f30a801c260 comp rx=0 tx=0).stop 2024-04-17T10:51:48.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.183+0000 7f30bcb22640 1 -- 172.21.15.83:0/133464929 shutdown_connections 2024-04-17T10:51:48.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.183+0000 7f30bcb22640 1 --2- 172.21.15.83:0/133464929 >> v2:172.21.15.167:3300/0 conn(0x7f30b806dae0 0x7f30b80cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:48.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.183+0000 7f30bcb22640 1 --2- 172.21.15.83:0/133464929 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30b80aedd0 0x7f30b806d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:48.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.183+0000 7f30bcb22640 1 --2- 172.21.15.83:0/133464929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30b815db50 0x7f30b8070470 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:48.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.183+0000 7f30bcb22640 1 -- 172.21.15.83:0/133464929 >> 172.21.15.83:0/133464929 conn(0x7f30b806abe0 msgr2=0x7f30b806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:48.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.183+0000 7f30bcb22640 1 -- 172.21.15.83:0/133464929 shutdown_connections 2024-04-17T10:51:48.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.183+0000 7f30bcb22640 1 -- 172.21.15.83:0/133464929 wait complete. 2024-04-17T10:51:48.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.183+0000 7f30bcb22640 1 Processor -- start 2024-04-17T10:51:48.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.183+0000 7f30bcb22640 1 -- start start 2024-04-17T10:51:48.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.187+0000 7f30bcb22640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30b806dae0 0x7f30b8150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:48.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.187+0000 7f30bcb22640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30b80aedd0 0x7f30b8155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:48.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.187+0000 7f30bcb22640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30b815db50 0x7f30b8155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:48.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.187+0000 7f30bcb22640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b80b2f90 con 0x7f30b806dae0 2024-04-17T10:51:48.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.187+0000 7f30bcb22640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b80b2e10 con 0x7f30b80aedd0 2024-04-17T10:51:48.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.187+0000 7f30bcb22640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b80b3110 con 0x7f30b815db50 2024-04-17T10:51:48.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.187+0000 7f30b659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30b806dae0 0x7f30b8150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:48.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.187+0000 7f30b659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30b806dae0 0x7f30b8150330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59346/0 (socket says 172.21.15.83:59346) 2024-04-17T10:51:48.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.187+0000 7f30b659c640 1 -- 172.21.15.83:0/1934357981 learned_addr learned my addr 172.21.15.83:0/1934357981 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:48.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.187+0000 7f30b5d9b640 1 --2- 172.21.15.83:0/1934357981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30b80aedd0 0x7f30b8155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:48.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.187+0000 7f30b6d9d640 1 --2- 172.21.15.83:0/1934357981 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30b815db50 0x7f30b8155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:48.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.187+0000 7f30b659c640 1 -- 172.21.15.83:0/1934357981 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30b815db50 msgr2=0x7f30b8155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:48.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.187+0000 7f30b659c640 1 --2- 172.21.15.83:0/1934357981 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30b815db50 0x7f30b8155a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:48.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.187+0000 7f30b659c640 1 -- 172.21.15.83:0/1934357981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30b80aedd0 msgr2=0x7f30b8155520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:48.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.187+0000 7f30b659c640 1 --2- 172.21.15.83:0/1934357981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30b80aedd0 0x7f30b8155520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:48.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.187+0000 7f30b659c640 1 -- 172.21.15.83:0/1934357981 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b815ac10 con 0x7f30b806dae0 2024-04-17T10:51:48.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.187+0000 7f30b6d9d640 1 --2- 172.21.15.83:0/1934357981 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30b815db50 0x7f30b8155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:51:48.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.187+0000 7f30b659c640 1 --2- 172.21.15.83:0/1934357981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30b806dae0 0x7f30b8150330 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f30a80ae040 tx=0x7f30a801aaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:48.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.187+0000 7f30a6ffd640 1 -- 172.21.15.83:0/1934357981 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a80b0040 con 0x7f30b806dae0 2024-04-17T10:51:48.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.187+0000 7f30a6ffd640 1 -- 172.21.15.83:0/1934357981 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a80bc070 con 0x7f30b806dae0 2024-04-17T10:51:48.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.187+0000 7f30a6ffd640 1 -- 172.21.15.83:0/1934357981 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a80b6880 con 0x7f30b806dae0 2024-04-17T10:51:48.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.187+0000 7f30bcb22640 1 -- 172.21.15.83:0/1934357981 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30b815aef0 con 0x7f30b806dae0 2024-04-17T10:51:48.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.187+0000 7f30bcb22640 1 -- 172.21.15.83:0/1934357981 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30b815b4b0 con 0x7f30b806dae0 2024-04-17T10:51:48.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.187+0000 7f30bcb22640 1 -- 172.21.15.83:0/1934357981 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30b8074f90 con 0x7f30b806dae0 2024-04-17T10:51:48.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.187+0000 7f30a6ffd640 1 -- 172.21.15.83:0/1934357981 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f30a80ab480 con 0x7f30b806dae0 2024-04-17T10:51:48.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.191+0000 7f30a6ffd640 1 --2- 172.21.15.83:0/1934357981 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3098066f60 0x7f3098069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:48.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.191+0000 7f30a6ffd640 1 -- 172.21.15.83:0/1934357981 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f30a811f4a0 con 0x7f30b806dae0 2024-04-17T10:51:48.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.191+0000 7f30b5d9b640 1 --2- 172.21.15.83:0/1934357981 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3098066f60 0x7f3098069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:48.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.191+0000 7f30b5d9b640 1 --2- 172.21.15.83:0/1934357981 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3098066f60 0x7f3098069420 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f30b81540b0 tx=0x7f30ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:48.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.195+0000 7f30a6ffd640 1 -- 172.21.15.83:0/1934357981 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f30a8016110 con 0x7f30b806dae0 2024-04-17T10:51:48.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.387+0000 7f30bcb22640 1 -- 172.21.15.83:0/1934357981 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30b8153e00 con 0x7f30b806dae0 2024-04-17T10:51:48.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.387+0000 7f30a6ffd640 1 -- 172.21.15.83:0/1934357981 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f30a80f04d0 con 0x7f30b806dae0 2024-04-17T10:51:48.380 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:51:48.380 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:51:48.380 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:51:48.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.387+0000 7f30bcb22640 1 -- 172.21.15.83:0/1934357981 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3098066f60 msgr2=0x7f3098069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:48.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.387+0000 7f30bcb22640 1 --2- 172.21.15.83:0/1934357981 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3098066f60 0x7f3098069420 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f30b81540b0 tx=0x7f30ac008040 comp rx=0 tx=0).stop 2024-04-17T10:51:48.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.387+0000 7f30bcb22640 1 -- 172.21.15.83:0/1934357981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30b806dae0 msgr2=0x7f30b8150330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:48.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.387+0000 7f30bcb22640 1 --2- 172.21.15.83:0/1934357981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30b806dae0 0x7f30b8150330 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f30a80ae040 tx=0x7f30a801aaa0 comp rx=0 tx=0).stop 2024-04-17T10:51:48.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.387+0000 7f30bcb22640 1 -- 172.21.15.83:0/1934357981 shutdown_connections 2024-04-17T10:51:48.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.387+0000 7f30bcb22640 1 --2- 172.21.15.83:0/1934357981 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3098066f60 0x7f3098069420 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:48.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.387+0000 7f30bcb22640 1 --2- 172.21.15.83:0/1934357981 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30b815db50 0x7f30b8155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:48.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.387+0000 7f30bcb22640 1 --2- 172.21.15.83:0/1934357981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30b80aedd0 0x7f30b8155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:48.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.387+0000 7f30bcb22640 1 --2- 172.21.15.83:0/1934357981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30b806dae0 0x7f30b8150330 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:48.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.387+0000 7f30bcb22640 1 -- 172.21.15.83:0/1934357981 >> 172.21.15.83:0/1934357981 conn(0x7f30b806abe0 msgr2=0x7f30b80b0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:48.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.387+0000 7f30bcb22640 1 -- 172.21.15.83:0/1934357981 shutdown_connections 2024-04-17T10:51:48.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:48.387+0000 7f30bcb22640 1 -- 172.21.15.83:0/1934357981 wait complete. 2024-04-17T10:51:48.799 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:51:48.800 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:51:49.800 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:51:49.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.939+0000 7fa23e0d4640 1 Processor -- start 2024-04-17T10:51:49.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.939+0000 7fa23e0d4640 1 -- start start 2024-04-17T10:51:49.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.939+0000 7fa23e0d4640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa238146a00 0x7fa238146e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:49.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.939+0000 7fa23e0d4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa23806d690 0x7fa2381620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:49.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.939+0000 7fa23e0d4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa2380b2670 0x7fa2380b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:49.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.939+0000 7fa23e0d4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa238162610 con 0x7fa2380b2670 2024-04-17T10:51:49.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.939+0000 7fa23e0d4640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa23806c960 con 0x7fa238146a00 2024-04-17T10:51:49.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.939+0000 7fa23e0d4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23806d200 con 0x7fa23806d690 2024-04-17T10:51:49.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.939+0000 7fa237fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa2380b2670 0x7fa2380b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:49.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.939+0000 7fa237fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa2380b2670 0x7fa2380b2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59358/0 (socket says 172.21.15.83:59358) 2024-04-17T10:51:49.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.939+0000 7fa237fff640 1 -- 172.21.15.83:0/855820760 learned_addr learned my addr 172.21.15.83:0/855820760 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:49.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.939+0000 7fa2377fe640 1 --2- 172.21.15.83:0/855820760 >> v2:172.21.15.167:3300/0 conn(0x7fa238146a00 0x7fa238146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:49.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.939+0000 7fa236ffd640 1 --2- 172.21.15.83:0/855820760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa23806d690 0x7fa2381620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:49.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.939+0000 7fa237fff640 1 -- 172.21.15.83:0/855820760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa23806d690 msgr2=0x7fa2381620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:49.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.939+0000 7fa237fff640 1 --2- 172.21.15.83:0/855820760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa23806d690 0x7fa2381620d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:49.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.939+0000 7fa237fff640 1 -- 172.21.15.83:0/855820760 >> v2:172.21.15.167:3300/0 conn(0x7fa238146a00 msgr2=0x7fa238146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:49.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.939+0000 7fa237fff640 1 --2- 172.21.15.83:0/855820760 >> v2:172.21.15.167:3300/0 conn(0x7fa238146a00 0x7fa238146e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:49.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.939+0000 7fa237fff640 1 -- 172.21.15.83:0/855820760 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa23806daf0 con 0x7fa2380b2670 2024-04-17T10:51:49.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.943+0000 7fa237fff640 1 --2- 172.21.15.83:0/855820760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2380b2670 0x7fa2380b2af0 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7fa23806ffe0 tx=0x7fa22801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b9ec6349b7a777 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:49.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.943+0000 7fa2367fc640 1 -- 172.21.15.83:0/855820760 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2280a5780 con 0x7fa2380b2670 2024-04-17T10:51:49.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.943+0000 7fa2367fc640 1 -- 172.21.15.83:0/855820760 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2280a5920 con 0x7fa2380b2670 2024-04-17T10:51:49.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.943+0000 7fa2367fc640 1 -- 172.21.15.83:0/855820760 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2280a5cf0 con 0x7fa2380b2670 2024-04-17T10:51:49.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.943+0000 7fa23e0d4640 1 -- 172.21.15.83:0/855820760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2380b2670 msgr2=0x7fa2380b2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:49.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.943+0000 7fa23e0d4640 1 --2- 172.21.15.83:0/855820760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2380b2670 0x7fa2380b2af0 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7fa23806ffe0 tx=0x7fa22801c9f0 comp rx=0 tx=0).stop 2024-04-17T10:51:49.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.943+0000 7fa23e0d4640 1 -- 172.21.15.83:0/855820760 shutdown_connections 2024-04-17T10:51:49.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.943+0000 7fa23e0d4640 1 --2- 172.21.15.83:0/855820760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2380b2670 0x7fa2380b2af0 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:49.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.943+0000 7fa23e0d4640 1 --2- 172.21.15.83:0/855820760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa23806d690 0x7fa2381620d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:49.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.943+0000 7fa23e0d4640 1 --2- 172.21.15.83:0/855820760 >> v2:172.21.15.167:3300/0 conn(0x7fa238146a00 0x7fa238146e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:49.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.943+0000 7fa23e0d4640 1 -- 172.21.15.83:0/855820760 >> 172.21.15.83:0/855820760 conn(0x7fa23806abe0 msgr2=0x7fa23806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:49.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.943+0000 7fa23e0d4640 1 -- 172.21.15.83:0/855820760 shutdown_connections 2024-04-17T10:51:49.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.943+0000 7fa23e0d4640 1 -- 172.21.15.83:0/855820760 wait complete. 2024-04-17T10:51:49.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.943+0000 7fa23e0d4640 1 Processor -- start 2024-04-17T10:51:49.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.943+0000 7fa23e0d4640 1 -- start start 2024-04-17T10:51:49.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.943+0000 7fa23e0d4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa23806d690 0x7fa238160570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:49.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.943+0000 7fa23e0d4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2380b2670 0x7fa23813cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:49.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.943+0000 7fa23e0d4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa238146a00 0x7fa23815be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:49.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.943+0000 7fa23e0d4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa238070e00 con 0x7fa23806d690 2024-04-17T10:51:49.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.943+0000 7fa23e0d4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa238070c80 con 0x7fa238146a00 2024-04-17T10:51:49.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.943+0000 7fa23e0d4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa238070f80 con 0x7fa2380b2670 2024-04-17T10:51:49.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.943+0000 7fa2377fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa23806d690 0x7fa238160570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:49.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.943+0000 7fa2377fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa23806d690 0x7fa238160570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59372/0 (socket says 172.21.15.83:59372) 2024-04-17T10:51:49.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.943+0000 7fa2377fe640 1 -- 172.21.15.83:0/2889209228 learned_addr learned my addr 172.21.15.83:0/2889209228 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:49.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.947+0000 7fa236ffd640 1 --2- 172.21.15.83:0/2889209228 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2380b2670 0x7fa23813cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:49.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.947+0000 7fa237fff640 1 --2- 172.21.15.83:0/2889209228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa238146a00 0x7fa23815be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:49.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.947+0000 7fa2377fe640 1 -- 172.21.15.83:0/2889209228 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2380b2670 msgr2=0x7fa23813cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:49.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.947+0000 7fa2377fe640 1 --2- 172.21.15.83:0/2889209228 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2380b2670 0x7fa23813cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:49.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.947+0000 7fa2377fe640 1 -- 172.21.15.83:0/2889209228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa238146a00 msgr2=0x7fa23815be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:49.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.947+0000 7fa2377fe640 1 --2- 172.21.15.83:0/2889209228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa238146a00 0x7fa23815be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:49.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.947+0000 7fa2377fe640 1 -- 172.21.15.83:0/2889209228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa23813d6c0 con 0x7fa23806d690 2024-04-17T10:51:49.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.947+0000 7fa2377fe640 1 --2- 172.21.15.83:0/2889209228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa23806d690 0x7fa238160570 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7fa23000d800 tx=0x7fa23000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:49.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.947+0000 7fa21ffff640 1 -- 172.21.15.83:0/2889209228 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23000b6c0 con 0x7fa23806d690 2024-04-17T10:51:49.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.947+0000 7fa21ffff640 1 -- 172.21.15.83:0/2889209228 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa230019070 con 0x7fa23806d690 2024-04-17T10:51:49.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.947+0000 7fa23e0d4640 1 -- 172.21.15.83:0/2889209228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa23813d9a0 con 0x7fa23806d690 2024-04-17T10:51:49.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.947+0000 7fa21ffff640 1 -- 172.21.15.83:0/2889209228 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23000ba60 con 0x7fa23806d690 2024-04-17T10:51:49.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.947+0000 7fa23e0d4640 1 -- 172.21.15.83:0/2889209228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa23813df60 con 0x7fa23806d690 2024-04-17T10:51:49.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.947+0000 7fa23e0d4640 1 -- 172.21.15.83:0/2889209228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa238074f90 con 0x7fa23806d690 2024-04-17T10:51:49.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.947+0000 7fa21ffff640 1 -- 172.21.15.83:0/2889209228 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa23000bc00 con 0x7fa23806d690 2024-04-17T10:51:49.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.947+0000 7fa21ffff640 1 --2- 172.21.15.83:0/2889209228 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa224066f90 0x7fa224069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:49.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.947+0000 7fa21ffff640 1 -- 172.21.15.83:0/2889209228 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fa2300855b0 con 0x7fa23806d690 2024-04-17T10:51:49.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.947+0000 7fa236ffd640 1 --2- 172.21.15.83:0/2889209228 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa224066f90 0x7fa224069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:49.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.947+0000 7fa236ffd640 1 --2- 172.21.15.83:0/2889209228 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa224066f90 0x7fa224069450 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa238140520 tx=0x7fa22c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:49.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:49.951+0000 7fa21ffff640 1 -- 172.21.15.83:0/2889209228 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa2300564d0 con 0x7fa23806d690 2024-04-17T10:51:50.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:50.151+0000 7fa23e0d4640 1 -- 172.21.15.83:0/2889209228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa23806de90 con 0x7fa23806d690 2024-04-17T10:51:50.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:50.151+0000 7fa21ffff640 1 -- 172.21.15.83:0/2889209228 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fa230055dd0 con 0x7fa23806d690 2024-04-17T10:51:50.145 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:51:50.145 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:51:50.145 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:51:50.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:50.151+0000 7fa23e0d4640 1 -- 172.21.15.83:0/2889209228 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa224066f90 msgr2=0x7fa224069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:50.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:50.151+0000 7fa23e0d4640 1 --2- 172.21.15.83:0/2889209228 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa224066f90 0x7fa224069450 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa238140520 tx=0x7fa22c008040 comp rx=0 tx=0).stop 2024-04-17T10:51:50.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:50.151+0000 7fa23e0d4640 1 -- 172.21.15.83:0/2889209228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa23806d690 msgr2=0x7fa238160570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:50.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:50.151+0000 7fa23e0d4640 1 --2- 172.21.15.83:0/2889209228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa23806d690 0x7fa238160570 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7fa23000d800 tx=0x7fa23000dcc0 comp rx=0 tx=0).stop 2024-04-17T10:51:50.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:50.151+0000 7fa23e0d4640 1 -- 172.21.15.83:0/2889209228 shutdown_connections 2024-04-17T10:51:50.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:50.151+0000 7fa23e0d4640 1 --2- 172.21.15.83:0/2889209228 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa224066f90 0x7fa224069450 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:50.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:50.151+0000 7fa23e0d4640 1 --2- 172.21.15.83:0/2889209228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa238146a00 0x7fa23815be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:50.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:50.151+0000 7fa23e0d4640 1 --2- 172.21.15.83:0/2889209228 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2380b2670 0x7fa23813cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:50.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:50.151+0000 7fa23e0d4640 1 --2- 172.21.15.83:0/2889209228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa23806d690 0x7fa238160570 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:50.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:50.151+0000 7fa23e0d4640 1 -- 172.21.15.83:0/2889209228 >> 172.21.15.83:0/2889209228 conn(0x7fa23806abe0 msgr2=0x7fa2380b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:50.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:50.151+0000 7fa23e0d4640 1 -- 172.21.15.83:0/2889209228 shutdown_connections 2024-04-17T10:51:50.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:50.151+0000 7fa23e0d4640 1 -- 172.21.15.83:0/2889209228 wait complete. 2024-04-17T10:51:50.565 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:51:50.566 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:51:51.566 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:51:51.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.703+0000 7f525df4d640 1 Processor -- start 2024-04-17T10:51:51.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.703+0000 7f525df4d640 1 -- start start 2024-04-17T10:51:51.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.703+0000 7f525df4d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5258072330 0x7f52580cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:51.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.703+0000 7f525df4d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52580cfa10 0x7f525806c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:51.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.703+0000 7f525df4d640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52580cfea0 con 0x7f525815ba30 2024-04-17T10:51:51.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.703+0000 7f525df4d640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5258071ea0 con 0x7f5258072330 2024-04-17T10:51:51.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.703+0000 7f525df4d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f525806e320 con 0x7f52580cfa10 2024-04-17T10:51:51.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.703+0000 7f5257fff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52580cfa10 0x7f525806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:51.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.703+0000 7f5256ffd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5258072330 0x7f52580cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:51.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.703+0000 7f52577fe640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f525815ba30 0x7f525815c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:60494/0 (socket says 172.21.15.83:60494) 2024-04-17T10:51:51.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.703+0000 7f52577fe640 1 -- 172.21.15.83:0/863062605 learned_addr learned my addr 172.21.15.83:0/863062605 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:51:51.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.703+0000 7f5257fff640 1 -- 172.21.15.83:0/863062605 >> v2:172.21.15.167:3300/0 conn(0x7f5258072330 msgr2=0x7f52580cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:51.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.703+0000 7f5257fff640 1 --2- 172.21.15.83:0/863062605 >> v2:172.21.15.167:3300/0 conn(0x7f5258072330 0x7f52580cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:51.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.703+0000 7f5257fff640 1 -- 172.21.15.83:0/863062605 >> v1:172.21.15.167:6789/0 conn(0x7f525815ba30 legacy=0x7f525815c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:51.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.703+0000 7f5257fff640 1 -- 172.21.15.83:0/863062605 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525806e040 con 0x7f52580cfa10 2024-04-17T10:51:51.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.707+0000 7f5257fff640 1 --2- 172.21.15.83:0/863062605 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52580cfa10 0x7f525806c490 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5258072020 tx=0x7f524801c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ed89d1be304780a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:51.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.707+0000 7f52567fc640 1 -- 172.21.15.83:0/863062605 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52480b2070 con 0x7f52580cfa10 2024-04-17T10:51:51.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.707+0000 7f52567fc640 1 -- 172.21.15.83:0/863062605 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52480a5890 con 0x7f52580cfa10 2024-04-17T10:51:51.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.707+0000 7f52567fc640 1 -- 172.21.15.83:0/863062605 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52480a5c60 con 0x7f52580cfa10 2024-04-17T10:51:51.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.707+0000 7f525df4d640 1 -- 172.21.15.83:0/863062605 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52580cfa10 msgr2=0x7f525806c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:51.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.707+0000 7f525df4d640 1 --2- 172.21.15.83:0/863062605 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52580cfa10 0x7f525806c490 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5258072020 tx=0x7f524801c8c0 comp rx=0 tx=0).stop 2024-04-17T10:51:51.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.707+0000 7f525df4d640 1 -- 172.21.15.83:0/863062605 shutdown_connections 2024-04-17T10:51:51.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.707+0000 7f525df4d640 1 --2- 172.21.15.83:0/863062605 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52580cfa10 0x7f525806c490 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:51.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.707+0000 7f525df4d640 1 --2- 172.21.15.83:0/863062605 >> v2:172.21.15.167:3300/0 conn(0x7f5258072330 0x7f52580cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:51.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.707+0000 7f525df4d640 1 -- 172.21.15.83:0/863062605 >> 172.21.15.83:0/863062605 conn(0x7f525806abe0 msgr2=0x7f525806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:51.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.707+0000 7f525df4d640 1 -- 172.21.15.83:0/863062605 shutdown_connections 2024-04-17T10:51:51.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.707+0000 7f525df4d640 1 -- 172.21.15.83:0/863062605 wait complete. 2024-04-17T10:51:51.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.707+0000 7f525df4d640 1 Processor -- start 2024-04-17T10:51:51.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.707+0000 7f525df4d640 1 -- start start 2024-04-17T10:51:51.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.711+0000 7f525df4d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5258072330 0x7f525813e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:51.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.711+0000 7f525df4d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52580cfa10 0x7f5258154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:51.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.711+0000 7f525df4d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f525815ba30 0x7f5258155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:51.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.711+0000 7f525df4d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525806de20 con 0x7f52580cfa10 2024-04-17T10:51:51.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.711+0000 7f525df4d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525806dca0 con 0x7f5258072330 2024-04-17T10:51:51.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.711+0000 7f525df4d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5258168520 con 0x7f525815ba30 2024-04-17T10:51:51.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.711+0000 7f5256ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52580cfa10 0x7f5258154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:51.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.711+0000 7f5256ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52580cfa10 0x7f5258154e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59374/0 (socket says 172.21.15.83:59374) 2024-04-17T10:51:51.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.711+0000 7f5256ffd640 1 -- 172.21.15.83:0/1986555092 learned_addr learned my addr 172.21.15.83:0/1986555092 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:51.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.711+0000 7f5257fff640 1 --2- 172.21.15.83:0/1986555092 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f525815ba30 0x7f5258155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:51.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.711+0000 7f52577fe640 1 --2- 172.21.15.83:0/1986555092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5258072330 0x7f525813e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:51.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.711+0000 7f5256ffd640 1 -- 172.21.15.83:0/1986555092 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f525815ba30 msgr2=0x7f5258155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:51.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.711+0000 7f5256ffd640 1 --2- 172.21.15.83:0/1986555092 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f525815ba30 0x7f5258155390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:51.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.711+0000 7f5256ffd640 1 -- 172.21.15.83:0/1986555092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5258072330 msgr2=0x7f525813e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:51.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.711+0000 7f5256ffd640 1 --2- 172.21.15.83:0/1986555092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5258072330 0x7f525813e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:51.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.711+0000 7f5256ffd640 1 -- 172.21.15.83:0/1986555092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5258153500 con 0x7f52580cfa10 2024-04-17T10:51:51.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.711+0000 7f5256ffd640 1 --2- 172.21.15.83:0/1986555092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52580cfa10 0x7f5258154e50 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f524c00e870 tx=0x7f524c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:51.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.711+0000 7f523ffff640 1 -- 172.21.15.83:0/1986555092 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c017070 con 0x7f52580cfa10 2024-04-17T10:51:51.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.711+0000 7f523ffff640 1 -- 172.21.15.83:0/1986555092 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524c00c940 con 0x7f52580cfa10 2024-04-17T10:51:51.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.711+0000 7f525df4d640 1 -- 172.21.15.83:0/1986555092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52581537e0 con 0x7f52580cfa10 2024-04-17T10:51:51.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.711+0000 7f523ffff640 1 -- 172.21.15.83:0/1986555092 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c00ccf0 con 0x7f52580cfa10 2024-04-17T10:51:51.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.711+0000 7f525df4d640 1 -- 172.21.15.83:0/1986555092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5258153da0 con 0x7f52580cfa10 2024-04-17T10:51:51.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.711+0000 7f525df4d640 1 -- 172.21.15.83:0/1986555092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5258074f90 con 0x7f52580cfa10 2024-04-17T10:51:51.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.711+0000 7f523ffff640 1 -- 172.21.15.83:0/1986555092 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f524c01a070 con 0x7f52580cfa10 2024-04-17T10:51:51.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.711+0000 7f523ffff640 1 --2- 172.21.15.83:0/1986555092 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5244066f60 0x7f5244069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:51.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.711+0000 7f523ffff640 1 -- 172.21.15.83:0/1986555092 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f524c085760 con 0x7f52580cfa10 2024-04-17T10:51:51.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.711+0000 7f52577fe640 1 --2- 172.21.15.83:0/1986555092 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5244066f60 0x7f5244069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:51.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.715+0000 7f52577fe640 1 --2- 172.21.15.83:0/1986555092 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5244066f60 0x7f5244069420 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f5250000f70 tx=0x7f5250009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:51.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.719+0000 7f523ffff640 1 -- 172.21.15.83:0/1986555092 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f524c056700 con 0x7f52580cfa10 2024-04-17T10:51:51.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.903+0000 7f525df4d640 1 -- 172.21.15.83:0/1986555092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52580cf2b0 con 0x7f52580cfa10 2024-04-17T10:51:51.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.903+0000 7f523ffff640 1 -- 172.21.15.83:0/1986555092 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f524c056000 con 0x7f52580cfa10 2024-04-17T10:51:51.898 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:51:51.898 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:51:51.899 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:51:51.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.907+0000 7f525df4d640 1 -- 172.21.15.83:0/1986555092 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5244066f60 msgr2=0x7f5244069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:51.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.907+0000 7f525df4d640 1 --2- 172.21.15.83:0/1986555092 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5244066f60 0x7f5244069420 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f5250000f70 tx=0x7f5250009040 comp rx=0 tx=0).stop 2024-04-17T10:51:51.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.907+0000 7f525df4d640 1 -- 172.21.15.83:0/1986555092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52580cfa10 msgr2=0x7f5258154e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:51.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.907+0000 7f525df4d640 1 --2- 172.21.15.83:0/1986555092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52580cfa10 0x7f5258154e50 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f524c00e870 tx=0x7f524c00ec70 comp rx=0 tx=0).stop 2024-04-17T10:51:51.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.907+0000 7f525df4d640 1 -- 172.21.15.83:0/1986555092 shutdown_connections 2024-04-17T10:51:51.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.907+0000 7f525df4d640 1 --2- 172.21.15.83:0/1986555092 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5244066f60 0x7f5244069420 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:51.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.907+0000 7f525df4d640 1 --2- 172.21.15.83:0/1986555092 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f525815ba30 0x7f5258155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:51.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.907+0000 7f525df4d640 1 --2- 172.21.15.83:0/1986555092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52580cfa10 0x7f5258154e50 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:51.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.907+0000 7f525df4d640 1 --2- 172.21.15.83:0/1986555092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5258072330 0x7f525813e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:51.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.907+0000 7f525df4d640 1 -- 172.21.15.83:0/1986555092 >> 172.21.15.83:0/1986555092 conn(0x7f525806abe0 msgr2=0x7f52580ce130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:51.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.907+0000 7f525df4d640 1 -- 172.21.15.83:0/1986555092 shutdown_connections 2024-04-17T10:51:51.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:51.907+0000 7f525df4d640 1 -- 172.21.15.83:0/1986555092 wait complete. 2024-04-17T10:51:52.318 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:51:52.318 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:51:53.319 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:51:53.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.459+0000 7f0bfe7ba640 1 Processor -- start 2024-04-17T10:51:53.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.459+0000 7f0bfe7ba640 1 -- start start 2024-04-17T10:51:53.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.459+0000 7f0bfe7ba640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0bf8072330 0x7f0bf80cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:53.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.459+0000 7f0bfe7ba640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bf80cfa10 0x7f0bf806c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:53.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.459+0000 7f0bfe7ba640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bf80cfea0 con 0x7f0bf815ba30 2024-04-17T10:51:53.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.459+0000 7f0bfe7ba640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bf8071ea0 con 0x7f0bf8072330 2024-04-17T10:51:53.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.459+0000 7f0bfe7ba640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf806e320 con 0x7f0bf80cfa10 2024-04-17T10:51:53.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.459+0000 7f0bf7fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0bf815ba30 0x7f0bf815c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35796/0 (socket says 172.21.15.83:35796) 2024-04-17T10:51:53.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.459+0000 7f0bf7fff640 1 -- 172.21.15.83:0/239014049 learned_addr learned my addr 172.21.15.83:0/239014049 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:51:53.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.459+0000 7f0bf77fe640 1 --2- 172.21.15.83:0/239014049 >> v2:172.21.15.167:3300/0 conn(0x7f0bf8072330 0x7f0bf80cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:53.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.459+0000 7f0bfcd57640 1 --2- 172.21.15.83:0/239014049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bf80cfa10 0x7f0bf806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:53.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.459+0000 7f0bf6ffd640 1 -- 172.21.15.83:0/239014049 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3148297324 0 0) 0x7f0bf80cfea0 con 0x7f0bf815ba30 2024-04-17T10:51:53.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.459+0000 7f0bf6ffd640 1 -- 172.21.15.83:0/239014049 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0be4003530 con 0x7f0bf815ba30 2024-04-17T10:51:53.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bf6ffd640 1 -- 172.21.15.83:0/239014049 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3824731251 0 0) 0x7f0be4003530 con 0x7f0bf815ba30 2024-04-17T10:51:53.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bf6ffd640 1 -- 172.21.15.83:0/239014049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bf80cfa10 msgr2=0x7f0bf806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:53.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bf6ffd640 1 --2- 172.21.15.83:0/239014049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bf80cfa10 0x7f0bf806c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:53.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bf6ffd640 1 -- 172.21.15.83:0/239014049 >> v2:172.21.15.167:3300/0 conn(0x7f0bf8072330 msgr2=0x7f0bf80cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:53.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bf6ffd640 1 --2- 172.21.15.83:0/239014049 >> v2:172.21.15.167:3300/0 conn(0x7f0bf8072330 0x7f0bf80cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:53.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bf6ffd640 1 -- 172.21.15.83:0/239014049 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf806e040 con 0x7f0bf815ba30 2024-04-17T10:51:53.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bf6ffd640 1 -- 172.21.15.83:0/239014049 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f0be8001680 con 0x7f0bf815ba30 2024-04-17T10:51:53.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bf6ffd640 1 -- 172.21.15.83:0/239014049 >> v1:172.21.15.83:6789/0 conn(0x7f0bf815ba30 legacy=0x7f0bf815c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:53.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bf6ffd640 1 --2- 172.21.15.83:0/239014049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0be40039c0 0x7f0be4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:53.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bf6ffd640 1 --2- 172.21.15.83:0/239014049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0be4004810 0x7f0be4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:53.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bf6ffd640 1 --2- 172.21.15.83:0/239014049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bf80cfa10 0x7f0be40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:53.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bf6ffd640 1 -- 172.21.15.83:0/239014049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf8071ea0 con 0x7f0be4004810 2024-04-17T10:51:53.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bf6ffd640 1 -- 172.21.15.83:0/239014049 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf806e320 con 0x7f0be40039c0 2024-04-17T10:51:53.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bf6ffd640 1 -- 172.21.15.83:0/239014049 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be400ba20 con 0x7f0bf80cfa10 2024-04-17T10:51:53.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bf77fe640 1 --2- 172.21.15.83:0/239014049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0be4004810 0x7f0be4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:53.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bf7fff640 1 --2- 172.21.15.83:0/239014049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0be40039c0 0x7f0be4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:53.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bf77fe640 1 -- 172.21.15.83:0/239014049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bf80cfa10 msgr2=0x7f0be40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:53.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bfcd57640 1 --2- 172.21.15.83:0/239014049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bf80cfa10 0x7f0be40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:53.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bf77fe640 1 --2- 172.21.15.83:0/239014049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bf80cfa10 0x7f0be40051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:53.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bf77fe640 1 -- 172.21.15.83:0/239014049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0be40039c0 msgr2=0x7f0be4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:53.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bf77fe640 1 --2- 172.21.15.83:0/239014049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0be40039c0 0x7f0be4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:53.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bf77fe640 1 -- 172.21.15.83:0/239014049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be4003530 con 0x7f0be4004810 2024-04-17T10:51:53.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bf7fff640 1 --2- 172.21.15.83:0/239014049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0be40039c0 0x7f0be4003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:51:53.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bf77fe640 1 --2- 172.21.15.83:0/239014049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0be4004810 0x7f0be4004c90 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f0be000e170 tx=0x7f0be001cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f71adc93aca74e63 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:53.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bf6ffd640 1 -- 172.21.15.83:0/239014049 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be00b2070 con 0x7f0be4004810 2024-04-17T10:51:53.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bf6ffd640 1 -- 172.21.15.83:0/239014049 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be00a5ac0 con 0x7f0be4004810 2024-04-17T10:51:53.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bf6ffd640 1 -- 172.21.15.83:0/239014049 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be0009c10 con 0x7f0be4004810 2024-04-17T10:51:53.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bfe7ba640 1 -- 172.21.15.83:0/239014049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0be4004810 msgr2=0x7f0be4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:53.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bfe7ba640 1 --2- 172.21.15.83:0/239014049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0be4004810 0x7f0be4004c90 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f0be000e170 tx=0x7f0be001cca0 comp rx=0 tx=0).stop 2024-04-17T10:51:53.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bfcd57640 1 -- 172.21.15.83:0/239014049 reap_dead start 2024-04-17T10:51:53.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bfe7ba640 1 -- 172.21.15.83:0/239014049 shutdown_connections 2024-04-17T10:51:53.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bfe7ba640 1 -- 172.21.15.83:0/239014049 >> 172.21.15.83:0/239014049 conn(0x7f0bf806abe0 msgr2=0x7f0bf806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:53.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bfe7ba640 1 -- 172.21.15.83:0/239014049 shutdown_connections 2024-04-17T10:51:53.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.463+0000 7f0bfe7ba640 1 -- 172.21.15.83:0/239014049 wait complete. 2024-04-17T10:51:53.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.467+0000 7f0bfe7ba640 1 Processor -- start 2024-04-17T10:51:53.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.467+0000 7f0bfe7ba640 1 -- start start 2024-04-17T10:51:53.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.467+0000 7f0bfe7ba640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0be4004810 0x7f0bf80af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:53.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.467+0000 7f0bfe7ba640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bf81fa720 0x7f0bf8148180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:53.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.467+0000 7f0bfe7ba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bf80ad840 0x7f0bf80adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:53.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.467+0000 7f0bfe7ba640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf806de20 con 0x7f0bf80ad840 2024-04-17T10:51:53.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.467+0000 7f0bfe7ba640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf806dca0 con 0x7f0be4004810 2024-04-17T10:51:53.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.467+0000 7f0bfe7ba640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf806cca0 con 0x7f0bf81fa720 2024-04-17T10:51:53.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.467+0000 7f0bfcd57640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bf80ad840 0x7f0bf80adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:53.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.467+0000 7f0bfcd57640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bf80ad840 0x7f0bf80adcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59398/0 (socket says 172.21.15.83:59398) 2024-04-17T10:51:53.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.467+0000 7f0bfcd57640 1 -- 172.21.15.83:0/1462254419 learned_addr learned my addr 172.21.15.83:0/1462254419 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:53.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.467+0000 7f0bf77fe640 1 --2- 172.21.15.83:0/1462254419 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bf81fa720 0x7f0bf8148180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:53.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.467+0000 7f0bf7fff640 1 --2- 172.21.15.83:0/1462254419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0be4004810 0x7f0bf80af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:53.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.467+0000 7f0bfcd57640 1 -- 172.21.15.83:0/1462254419 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bf81fa720 msgr2=0x7f0bf8148180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:53.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.467+0000 7f0bfcd57640 1 --2- 172.21.15.83:0/1462254419 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bf81fa720 0x7f0bf8148180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:53.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.467+0000 7f0bfcd57640 1 -- 172.21.15.83:0/1462254419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0be4004810 msgr2=0x7f0bf80af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:53.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.467+0000 7f0bfcd57640 1 --2- 172.21.15.83:0/1462254419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0be4004810 0x7f0bf80af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:53.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.467+0000 7f0bfcd57640 1 -- 172.21.15.83:0/1462254419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf81fab80 con 0x7f0bf80ad840 2024-04-17T10:51:53.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.467+0000 7f0bf7fff640 1 --2- 172.21.15.83:0/1462254419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0be4004810 0x7f0bf80af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:51:53.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.467+0000 7f0bfcd57640 1 --2- 172.21.15.83:0/1462254419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bf80ad840 0x7f0bf80adcc0 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8072c70 tx=0x7f0bf0010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:53.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.467+0000 7f0bf4ff9640 1 -- 172.21.15.83:0/1462254419 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf000f040 con 0x7f0bf80ad840 2024-04-17T10:51:53.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.467+0000 7f0bf4ff9640 1 -- 172.21.15.83:0/1462254419 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf001c070 con 0x7f0bf80ad840 2024-04-17T10:51:53.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.467+0000 7f0bfe7ba640 1 -- 172.21.15.83:0/1462254419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf80afa70 con 0x7f0bf80ad840 2024-04-17T10:51:53.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.467+0000 7f0bf4ff9640 1 -- 172.21.15.83:0/1462254419 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf000eb00 con 0x7f0bf80ad840 2024-04-17T10:51:53.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.467+0000 7f0bfe7ba640 1 -- 172.21.15.83:0/1462254419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf80b0030 con 0x7f0bf80ad840 2024-04-17T10:51:53.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.467+0000 7f0bfe7ba640 1 -- 172.21.15.83:0/1462254419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf80ae200 con 0x7f0bf80ad840 2024-04-17T10:51:53.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.467+0000 7f0bf4ff9640 1 -- 172.21.15.83:0/1462254419 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0bf000eca0 con 0x7f0bf80ad840 2024-04-17T10:51:53.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.471+0000 7f0bf4ff9640 1 --2- 172.21.15.83:0/1462254419 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0bd8066f90 0x7f0bd8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:53.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.471+0000 7f0bf4ff9640 1 -- 172.21.15.83:0/1462254419 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f0bf0084790 con 0x7f0bf80ad840 2024-04-17T10:51:53.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.471+0000 7f0bf7fff640 1 --2- 172.21.15.83:0/1462254419 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0bd8066f90 0x7f0bd8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:53.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.471+0000 7f0bf7fff640 1 --2- 172.21.15.83:0/1462254419 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0bd8066f90 0x7f0bd8069450 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f0be8001930 tx=0x7f0be8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:53.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.475+0000 7f0bf4ff9640 1 -- 172.21.15.83:0/1462254419 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0bf0025020 con 0x7f0bf80ad840 2024-04-17T10:51:53.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.667+0000 7f0bfe7ba640 1 -- 172.21.15.83:0/1462254419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0bf815ace0 con 0x7f0bf80ad840 2024-04-17T10:51:53.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.671+0000 7f0bf4ff9640 1 -- 172.21.15.83:0/1462254419 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f0bf00550e0 con 0x7f0bf80ad840 2024-04-17T10:51:53.665 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:51:53.665 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:51:53.665 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:51:53.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.671+0000 7f0bfe7ba640 1 -- 172.21.15.83:0/1462254419 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0bd8066f90 msgr2=0x7f0bd8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:53.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.671+0000 7f0bfe7ba640 1 --2- 172.21.15.83:0/1462254419 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0bd8066f90 0x7f0bd8069450 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f0be8001930 tx=0x7f0be8009040 comp rx=0 tx=0).stop 2024-04-17T10:51:53.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.671+0000 7f0bfe7ba640 1 -- 172.21.15.83:0/1462254419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bf80ad840 msgr2=0x7f0bf80adcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:53.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.671+0000 7f0bfe7ba640 1 --2- 172.21.15.83:0/1462254419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bf80ad840 0x7f0bf80adcc0 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8072c70 tx=0x7f0bf0010c20 comp rx=0 tx=0).stop 2024-04-17T10:51:53.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.675+0000 7f0bfe7ba640 1 -- 172.21.15.83:0/1462254419 shutdown_connections 2024-04-17T10:51:53.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.675+0000 7f0bfe7ba640 1 --2- 172.21.15.83:0/1462254419 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0bd8066f90 0x7f0bd8069450 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:53.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.675+0000 7f0bfe7ba640 1 --2- 172.21.15.83:0/1462254419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bf80ad840 0x7f0bf80adcc0 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:53.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.675+0000 7f0bfe7ba640 1 --2- 172.21.15.83:0/1462254419 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bf81fa720 0x7f0bf8148180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:53.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.675+0000 7f0bfe7ba640 1 --2- 172.21.15.83:0/1462254419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0be4004810 0x7f0bf80af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:53.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.675+0000 7f0bfe7ba640 1 -- 172.21.15.83:0/1462254419 >> 172.21.15.83:0/1462254419 conn(0x7f0bf806abe0 msgr2=0x7f0bf8153e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:53.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.675+0000 7f0bfe7ba640 1 -- 172.21.15.83:0/1462254419 shutdown_connections 2024-04-17T10:51:53.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:53.675+0000 7f0bfe7ba640 1 -- 172.21.15.83:0/1462254419 wait complete. 2024-04-17T10:51:54.087 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:51:54.087 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:51:55.088 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:51:55.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.227+0000 7f1d658a8640 1 Processor -- start 2024-04-17T10:51:55.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.227+0000 7f1d658a8640 1 -- start start 2024-04-17T10:51:55.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.227+0000 7f1d658a8640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1d6015db50 0x7f1d60070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:55.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.227+0000 7f1d658a8640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1d6006dcb0 0x7f1d600b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:55.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.227+0000 7f1d658a8640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d600709b0 con 0x7f1d600aedd0 2024-04-17T10:51:55.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.227+0000 7f1d658a8640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d6015e440 con 0x7f1d6006dcb0 2024-04-17T10:51:55.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.227+0000 7f1d658a8640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d6015e720 con 0x7f1d6015db50 2024-04-17T10:51:55.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.227+0000 7f1d5e7fc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1d600aedd0 0x7f1d6006d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35810/0 (socket says 172.21.15.83:35810) 2024-04-17T10:51:55.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.227+0000 7f1d5e7fc640 1 -- 172.21.15.83:0/15523285 learned_addr learned my addr 172.21.15.83:0/15523285 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:51:55.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.227+0000 7f1d5f7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1d6006dcb0 0x7f1d600b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:55.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.227+0000 7f1d5dffb640 1 -- 172.21.15.83:0/15523285 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2690599542 0 0) 0x7f1d54001120 con 0x7f1d600aedd0 2024-04-17T10:51:55.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.231+0000 7f1d5effd640 1 --2- 172.21.15.83:0/15523285 >> v2:172.21.15.167:3300/0 conn(0x7f1d6015db50 0x7f1d60070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:55.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.231+0000 7f1d5dffb640 1 -- 172.21.15.83:0/15523285 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d48003530 con 0x7f1d600aedd0 2024-04-17T10:51:55.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.231+0000 7f1d5f7fe640 1 -- 172.21.15.83:0/15523285 >> v2:172.21.15.167:3300/0 conn(0x7f1d6015db50 msgr2=0x7f1d60070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:55.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.231+0000 7f1d5f7fe640 1 --2- 172.21.15.83:0/15523285 >> v2:172.21.15.167:3300/0 conn(0x7f1d6015db50 0x7f1d60070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:55.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.231+0000 7f1d5f7fe640 1 -- 172.21.15.83:0/15523285 >> v1:172.21.15.83:6789/0 conn(0x7f1d600aedd0 legacy=0x7f1d6006d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:55.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.231+0000 7f1d5f7fe640 1 -- 172.21.15.83:0/15523285 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d6015ea00 con 0x7f1d6006dcb0 2024-04-17T10:51:55.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.231+0000 7f1d5f7fe640 1 --2- 172.21.15.83:0/15523285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d6006dcb0 0x7f1d600b2e10 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f1d58009d90 tx=0x7f1d5801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dbc7c13abfa8a59 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:55.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.231+0000 7f1d5dffb640 1 -- 172.21.15.83:0/15523285 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d580b2070 con 0x7f1d6006dcb0 2024-04-17T10:51:55.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.231+0000 7f1d5dffb640 1 -- 172.21.15.83:0/15523285 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d580a59c0 con 0x7f1d6006dcb0 2024-04-17T10:51:55.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.231+0000 7f1d5dffb640 1 -- 172.21.15.83:0/15523285 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d580a5d70 con 0x7f1d6006dcb0 2024-04-17T10:51:55.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.231+0000 7f1d658a8640 1 -- 172.21.15.83:0/15523285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d6006dcb0 msgr2=0x7f1d600b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:55.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.231+0000 7f1d658a8640 1 --2- 172.21.15.83:0/15523285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d6006dcb0 0x7f1d600b2e10 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f1d58009d90 tx=0x7f1d5801c900 comp rx=0 tx=0).stop 2024-04-17T10:51:55.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.231+0000 7f1d658a8640 1 -- 172.21.15.83:0/15523285 shutdown_connections 2024-04-17T10:51:55.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.231+0000 7f1d658a8640 1 --2- 172.21.15.83:0/15523285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d6006dcb0 0x7f1d600b2e10 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:55.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.231+0000 7f1d658a8640 1 --2- 172.21.15.83:0/15523285 >> v2:172.21.15.167:3300/0 conn(0x7f1d6015db50 0x7f1d60070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:55.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.231+0000 7f1d658a8640 1 -- 172.21.15.83:0/15523285 >> 172.21.15.83:0/15523285 conn(0x7f1d6006abe0 msgr2=0x7f1d6006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:55.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.231+0000 7f1d658a8640 1 -- 172.21.15.83:0/15523285 shutdown_connections 2024-04-17T10:51:55.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.231+0000 7f1d658a8640 1 -- 172.21.15.83:0/15523285 wait complete. 2024-04-17T10:51:55.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.235+0000 7f1d658a8640 1 Processor -- start 2024-04-17T10:51:55.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.235+0000 7f1d658a8640 1 -- start start 2024-04-17T10:51:55.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.235+0000 7f1d658a8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d6006dcb0 0x7f1d600ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:55.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.235+0000 7f1d658a8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d600aedd0 0x7f1d6014e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:55.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.235+0000 7f1d658a8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d6015db50 0x7f1d6014dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:55.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.235+0000 7f1d658a8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d600b3ce0 con 0x7f1d600aedd0 2024-04-17T10:51:55.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.235+0000 7f1d658a8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d600b3b60 con 0x7f1d6015db50 2024-04-17T10:51:55.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.235+0000 7f1d658a8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d600b3e60 con 0x7f1d6006dcb0 2024-04-17T10:51:55.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.235+0000 7f1d5e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d600aedd0 0x7f1d6014e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:55.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.235+0000 7f1d5e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d600aedd0 0x7f1d6014e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59416/0 (socket says 172.21.15.83:59416) 2024-04-17T10:51:55.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.235+0000 7f1d5e7fc640 1 -- 172.21.15.83:0/2690237984 learned_addr learned my addr 172.21.15.83:0/2690237984 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:55.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.235+0000 7f1d5f7fe640 1 --2- 172.21.15.83:0/2690237984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d6015db50 0x7f1d6014dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:55.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.235+0000 7f1d5effd640 1 --2- 172.21.15.83:0/2690237984 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d6006dcb0 0x7f1d600ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:55.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.235+0000 7f1d5e7fc640 1 -- 172.21.15.83:0/2690237984 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d6006dcb0 msgr2=0x7f1d600ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:55.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.235+0000 7f1d5e7fc640 1 --2- 172.21.15.83:0/2690237984 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d6006dcb0 0x7f1d600ae3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:55.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.235+0000 7f1d5e7fc640 1 -- 172.21.15.83:0/2690237984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d6015db50 msgr2=0x7f1d6014dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:55.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.235+0000 7f1d5e7fc640 1 --2- 172.21.15.83:0/2690237984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d6015db50 0x7f1d6014dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:55.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.235+0000 7f1d5e7fc640 1 -- 172.21.15.83:0/2690237984 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d60167d40 con 0x7f1d600aedd0 2024-04-17T10:51:55.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.235+0000 7f1d5e7fc640 1 --2- 172.21.15.83:0/2690237984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d600aedd0 0x7f1d6014e2d0 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f1d5400c730 tx=0x7f1d5400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:55.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.235+0000 7f1d47fff640 1 -- 172.21.15.83:0/2690237984 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d54017070 con 0x7f1d600aedd0 2024-04-17T10:51:55.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.235+0000 7f1d47fff640 1 -- 172.21.15.83:0/2690237984 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d54005ae0 con 0x7f1d600aedd0 2024-04-17T10:51:55.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.235+0000 7f1d658a8640 1 -- 172.21.15.83:0/2690237984 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d60168020 con 0x7f1d600aedd0 2024-04-17T10:51:55.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.235+0000 7f1d47fff640 1 -- 172.21.15.83:0/2690237984 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d54007e30 con 0x7f1d600aedd0 2024-04-17T10:51:55.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.235+0000 7f1d658a8640 1 -- 172.21.15.83:0/2690237984 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d60167820 con 0x7f1d600aedd0 2024-04-17T10:51:55.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.235+0000 7f1d658a8640 1 -- 172.21.15.83:0/2690237984 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d60074f90 con 0x7f1d600aedd0 2024-04-17T10:51:55.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.239+0000 7f1d47fff640 1 -- 172.21.15.83:0/2690237984 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1d5401a070 con 0x7f1d600aedd0 2024-04-17T10:51:55.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.239+0000 7f1d47fff640 1 --2- 172.21.15.83:0/2690237984 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1d4c066f60 0x7f1d4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:55.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.239+0000 7f1d5effd640 1 --2- 172.21.15.83:0/2690237984 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1d4c066f60 0x7f1d4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:55.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.239+0000 7f1d47fff640 1 -- 172.21.15.83:0/2690237984 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f1d540843d0 con 0x7f1d600aedd0 2024-04-17T10:51:55.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.239+0000 7f1d5effd640 1 --2- 172.21.15.83:0/2690237984 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1d4c066f60 0x7f1d4c069420 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f1d600ae380 tx=0x7f1d50008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:55.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.243+0000 7f1d47fff640 1 -- 172.21.15.83:0/2690237984 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1d600709b0 con 0x7f1d600aedd0 2024-04-17T10:51:55.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.435+0000 7f1d658a8640 1 -- 172.21.15.83:0/2690237984 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d600aea60 con 0x7f1d600aedd0 2024-04-17T10:51:55.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.435+0000 7f1d47fff640 1 -- 172.21.15.83:0/2690237984 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f1d54055400 con 0x7f1d600aedd0 2024-04-17T10:51:55.429 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:51:55.429 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:51:55.430 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:51:55.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.435+0000 7f1d658a8640 1 -- 172.21.15.83:0/2690237984 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1d4c066f60 msgr2=0x7f1d4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:55.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.435+0000 7f1d658a8640 1 --2- 172.21.15.83:0/2690237984 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1d4c066f60 0x7f1d4c069420 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f1d600ae380 tx=0x7f1d50008040 comp rx=0 tx=0).stop 2024-04-17T10:51:55.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.435+0000 7f1d658a8640 1 -- 172.21.15.83:0/2690237984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d600aedd0 msgr2=0x7f1d6014e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:55.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.435+0000 7f1d658a8640 1 --2- 172.21.15.83:0/2690237984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d600aedd0 0x7f1d6014e2d0 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f1d5400c730 tx=0x7f1d5400cbf0 comp rx=0 tx=0).stop 2024-04-17T10:51:55.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.439+0000 7f1d658a8640 1 -- 172.21.15.83:0/2690237984 shutdown_connections 2024-04-17T10:51:55.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.439+0000 7f1d658a8640 1 --2- 172.21.15.83:0/2690237984 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1d4c066f60 0x7f1d4c069420 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:55.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.439+0000 7f1d658a8640 1 --2- 172.21.15.83:0/2690237984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d6015db50 0x7f1d6014dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:55.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.439+0000 7f1d658a8640 1 --2- 172.21.15.83:0/2690237984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d600aedd0 0x7f1d6014e2d0 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:55.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.439+0000 7f1d658a8640 1 --2- 172.21.15.83:0/2690237984 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d6006dcb0 0x7f1d600ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:55.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.439+0000 7f1d658a8640 1 -- 172.21.15.83:0/2690237984 >> 172.21.15.83:0/2690237984 conn(0x7f1d6006abe0 msgr2=0x7f1d600af1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:55.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.439+0000 7f1d658a8640 1 -- 172.21.15.83:0/2690237984 shutdown_connections 2024-04-17T10:51:55.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:55.439+0000 7f1d658a8640 1 -- 172.21.15.83:0/2690237984 wait complete. 2024-04-17T10:51:55.851 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:51:55.851 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:51:56.852 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:51:56.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.991+0000 7fc57c3f4640 1 Processor -- start 2024-04-17T10:51:56.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.991+0000 7fc57c3f4640 1 -- start start 2024-04-17T10:51:56.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.991+0000 7fc57c3f4640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc57415eeb0 0x7fc57406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:56.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.991+0000 7fc57c3f4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5740cf780 0x7fc57406bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:56.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.991+0000 7fc57c3f4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc57415f290 0x7fc57406c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:56.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.991+0000 7fc57c3f4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5740cdb90 con 0x7fc57415f290 2024-04-17T10:51:56.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.991+0000 7fc57c3f4640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc57406e980 con 0x7fc57415eeb0 2024-04-17T10:51:56.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.991+0000 7fc57c3f4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc57406e0e0 con 0x7fc5740cf780 2024-04-17T10:51:56.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.991+0000 7fc57a991640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc57415f290 0x7fc57406c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:56.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.991+0000 7fc57a991640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc57415f290 0x7fc57406c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43842/0 (socket says 172.21.15.83:43842) 2024-04-17T10:51:56.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.991+0000 7fc57a991640 1 -- 172.21.15.83:0/259294237 learned_addr learned my addr 172.21.15.83:0/259294237 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:56.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.991+0000 7fc57998f640 1 --2- 172.21.15.83:0/259294237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5740cf780 0x7fc57406bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:56.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.991+0000 7fc57a190640 1 --2- 172.21.15.83:0/259294237 >> v2:172.21.15.167:3300/0 conn(0x7fc57415eeb0 0x7fc57406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:56.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.991+0000 7fc57a991640 1 -- 172.21.15.83:0/259294237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5740cf780 msgr2=0x7fc57406bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:56.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.991+0000 7fc57a991640 1 --2- 172.21.15.83:0/259294237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5740cf780 0x7fc57406bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:56.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.991+0000 7fc57a991640 1 -- 172.21.15.83:0/259294237 >> v2:172.21.15.167:3300/0 conn(0x7fc57415eeb0 msgr2=0x7fc57406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:56.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.991+0000 7fc57a991640 1 --2- 172.21.15.83:0/259294237 >> v2:172.21.15.167:3300/0 conn(0x7fc57415eeb0 0x7fc57406d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:56.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.991+0000 7fc57a991640 1 -- 172.21.15.83:0/259294237 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc57406e3c0 con 0x7fc57415f290 2024-04-17T10:51:56.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.991+0000 7fc57a190640 1 --2- 172.21.15.83:0/259294237 >> v2:172.21.15.167:3300/0 conn(0x7fc57415eeb0 0x7fc57406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:51:56.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.995+0000 7fc57a991640 1 --2- 172.21.15.83:0/259294237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc57415f290 0x7fc57406c130 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fc574068520 tx=0x7fc56c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e245dced2857200 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:56.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.995+0000 7fc57918e640 1 -- 172.21.15.83:0/259294237 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56c0a5860 con 0x7fc57415f290 2024-04-17T10:51:56.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.995+0000 7fc57918e640 1 -- 172.21.15.83:0/259294237 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc56c0a5a00 con 0x7fc57415f290 2024-04-17T10:51:56.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.995+0000 7fc57918e640 1 -- 172.21.15.83:0/259294237 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56c0a5dd0 con 0x7fc57415f290 2024-04-17T10:51:56.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.995+0000 7fc57c3f4640 1 -- 172.21.15.83:0/259294237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc57415f290 msgr2=0x7fc57406c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:56.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.995+0000 7fc57c3f4640 1 --2- 172.21.15.83:0/259294237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc57415f290 0x7fc57406c130 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fc574068520 tx=0x7fc56c01c9c0 comp rx=0 tx=0).stop 2024-04-17T10:51:56.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.995+0000 7fc57c3f4640 1 -- 172.21.15.83:0/259294237 shutdown_connections 2024-04-17T10:51:56.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.995+0000 7fc57c3f4640 1 --2- 172.21.15.83:0/259294237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc57415f290 0x7fc57406c130 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:56.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.995+0000 7fc57c3f4640 1 --2- 172.21.15.83:0/259294237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5740cf780 0x7fc57406bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:56.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.995+0000 7fc57c3f4640 1 --2- 172.21.15.83:0/259294237 >> v2:172.21.15.167:3300/0 conn(0x7fc57415eeb0 0x7fc57406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:56.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.995+0000 7fc57c3f4640 1 -- 172.21.15.83:0/259294237 >> 172.21.15.83:0/259294237 conn(0x7fc5740cc910 msgr2=0x7fc5740ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:56.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.995+0000 7fc57c3f4640 1 -- 172.21.15.83:0/259294237 shutdown_connections 2024-04-17T10:51:56.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.995+0000 7fc57c3f4640 1 -- 172.21.15.83:0/259294237 wait complete. 2024-04-17T10:51:56.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.995+0000 7fc57c3f4640 1 Processor -- start 2024-04-17T10:51:56.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.995+0000 7fc57c3f4640 1 -- start start 2024-04-17T10:51:56.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.999+0000 7fc57c3f4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5740cf780 0x7fc5740c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:56.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.999+0000 7fc57c3f4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc57415eeb0 0x7fc5740c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:56.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.999+0000 7fc57c3f4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc57415f290 0x7fc5740c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:56.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.999+0000 7fc57c3f4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5741602e0 con 0x7fc57415eeb0 2024-04-17T10:51:56.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.999+0000 7fc57c3f4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc574160160 con 0x7fc57415f290 2024-04-17T10:51:56.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.999+0000 7fc57c3f4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc574160460 con 0x7fc5740cf780 2024-04-17T10:51:56.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.999+0000 7fc57998f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc57415eeb0 0x7fc5740c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:56.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.999+0000 7fc57998f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc57415eeb0 0x7fc5740c6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43848/0 (socket says 172.21.15.83:43848) 2024-04-17T10:51:56.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.999+0000 7fc57998f640 1 -- 172.21.15.83:0/1484453252 learned_addr learned my addr 172.21.15.83:0/1484453252 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:56.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.999+0000 7fc57a190640 1 --2- 172.21.15.83:0/1484453252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5740cf780 0x7fc5740c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:56.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.999+0000 7fc57a991640 1 --2- 172.21.15.83:0/1484453252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc57415f290 0x7fc5740c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:56.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.999+0000 7fc57998f640 1 -- 172.21.15.83:0/1484453252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5740cf780 msgr2=0x7fc5740c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:56.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.999+0000 7fc57998f640 1 --2- 172.21.15.83:0/1484453252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5740cf780 0x7fc5740c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:56.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.999+0000 7fc57998f640 1 -- 172.21.15.83:0/1484453252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc57415f290 msgr2=0x7fc5740c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:56.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.999+0000 7fc57998f640 1 --2- 172.21.15.83:0/1484453252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc57415f290 0x7fc5740c74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:56.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.999+0000 7fc57998f640 1 -- 172.21.15.83:0/1484453252 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5740c9a80 con 0x7fc57415eeb0 2024-04-17T10:51:56.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.999+0000 7fc57a190640 1 --2- 172.21.15.83:0/1484453252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5740cf780 0x7fc5740c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:51:56.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.999+0000 7fc57998f640 1 --2- 172.21.15.83:0/1484453252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc57415eeb0 0x7fc5740c6f60 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7fc56800d800 tx=0x7fc56800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:56.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.999+0000 7fc572ffd640 1 -- 172.21.15.83:0/1484453252 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56800b6c0 con 0x7fc57415eeb0 2024-04-17T10:51:56.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.999+0000 7fc572ffd640 1 -- 172.21.15.83:0/1484453252 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc568019070 con 0x7fc57415eeb0 2024-04-17T10:51:56.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.999+0000 7fc57c3f4640 1 -- 172.21.15.83:0/1484453252 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5740c9d60 con 0x7fc57415eeb0 2024-04-17T10:51:56.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.999+0000 7fc572ffd640 1 -- 172.21.15.83:0/1484453252 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56800ba80 con 0x7fc57415eeb0 2024-04-17T10:51:56.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.999+0000 7fc57c3f4640 1 -- 172.21.15.83:0/1484453252 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5740c96a0 con 0x7fc57415eeb0 2024-04-17T10:51:56.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.999+0000 7fc57c3f4640 1 -- 172.21.15.83:0/1484453252 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc574074f90 con 0x7fc57415eeb0 2024-04-17T10:51:56.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.999+0000 7fc572ffd640 1 -- 172.21.15.83:0/1484453252 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc56800bc20 con 0x7fc57415eeb0 2024-04-17T10:51:56.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.999+0000 7fc572ffd640 1 --2- 172.21.15.83:0/1484453252 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc560066f90 0x7fc560069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:56.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.999+0000 7fc572ffd640 1 -- 172.21.15.83:0/1484453252 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fc568085380 con 0x7fc57415eeb0 2024-04-17T10:51:56.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.999+0000 7fc57a190640 1 --2- 172.21.15.83:0/1484453252 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc560066f90 0x7fc560069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:56.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:56.999+0000 7fc57a190640 1 --2- 172.21.15.83:0/1484453252 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc560066f90 0x7fc560069450 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fc56400b840 tx=0x7fc564008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:56.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:57.003+0000 7fc572ffd640 1 -- 172.21.15.83:0/1484453252 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc5680563b0 con 0x7fc57415eeb0 2024-04-17T10:51:57.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:57.191+0000 7fc57c3f4640 1 -- 172.21.15.83:0/1484453252 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc574066b50 con 0x7fc57415eeb0 2024-04-17T10:51:57.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:57.195+0000 7fc572ffd640 1 -- 172.21.15.83:0/1484453252 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fc568055cb0 con 0x7fc57415eeb0 2024-04-17T10:51:57.188 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:51:57.188 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:51:57.188 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:51:57.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:57.195+0000 7fc57c3f4640 1 -- 172.21.15.83:0/1484453252 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc560066f90 msgr2=0x7fc560069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:57.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:57.195+0000 7fc57c3f4640 1 --2- 172.21.15.83:0/1484453252 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc560066f90 0x7fc560069450 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fc56400b840 tx=0x7fc564008040 comp rx=0 tx=0).stop 2024-04-17T10:51:57.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:57.195+0000 7fc57c3f4640 1 -- 172.21.15.83:0/1484453252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc57415eeb0 msgr2=0x7fc5740c6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:57.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:57.195+0000 7fc57c3f4640 1 --2- 172.21.15.83:0/1484453252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc57415eeb0 0x7fc5740c6f60 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7fc56800d800 tx=0x7fc56800dcc0 comp rx=0 tx=0).stop 2024-04-17T10:51:57.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:57.195+0000 7fc57c3f4640 1 -- 172.21.15.83:0/1484453252 shutdown_connections 2024-04-17T10:51:57.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:57.195+0000 7fc57c3f4640 1 --2- 172.21.15.83:0/1484453252 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc560066f90 0x7fc560069450 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:57.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:57.195+0000 7fc57c3f4640 1 --2- 172.21.15.83:0/1484453252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc57415f290 0x7fc5740c74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:57.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:57.195+0000 7fc57c3f4640 1 --2- 172.21.15.83:0/1484453252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc57415eeb0 0x7fc5740c6f60 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:57.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:57.195+0000 7fc57c3f4640 1 --2- 172.21.15.83:0/1484453252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5740cf780 0x7fc5740c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:57.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:57.195+0000 7fc57c3f4640 1 -- 172.21.15.83:0/1484453252 >> 172.21.15.83:0/1484453252 conn(0x7fc5740cc910 msgr2=0x7fc57406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:57.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:57.195+0000 7fc57c3f4640 1 -- 172.21.15.83:0/1484453252 shutdown_connections 2024-04-17T10:51:57.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:57.195+0000 7fc57c3f4640 1 -- 172.21.15.83:0/1484453252 wait complete. 2024-04-17T10:51:57.607 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:51:57.607 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:51:58.608 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:51:58.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.747+0000 7f80997f6640 1 Processor -- start 2024-04-17T10:51:58.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.747+0000 7f80997f6640 1 -- start start 2024-04-17T10:51:58.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.747+0000 7f80997f6640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f809415ba30 0x7f8094072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:58.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.747+0000 7f80997f6640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f809406e320 con 0x7f80940cfbe0 2024-04-17T10:51:58.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.747+0000 7f80997f6640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f809406e040 con 0x7f809415c960 2024-04-17T10:51:58.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.747+0000 7f80997f6640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8094071ea0 con 0x7f809415ba30 2024-04-17T10:51:58.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.747+0000 7f8092ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f809415ba30 0x7f8094072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:58.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.747+0000 7f80937fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f80940cfbe0 0x7f809406c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58682/0 (socket says 172.21.15.83:58682) 2024-04-17T10:51:58.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.747+0000 7f80937fe640 1 -- 172.21.15.83:0/394599130 learned_addr learned my addr 172.21.15.83:0/394599130 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:51:58.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.747+0000 7f8092ffd640 1 -- 172.21.15.83:0/394599130 >> v1:172.21.15.167:6789/0 conn(0x7f809415c960 legacy=0x7f80940cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:51:58.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.747+0000 7f8092ffd640 1 -- 172.21.15.83:0/394599130 >> v1:172.21.15.83:6789/0 conn(0x7f80940cfbe0 legacy=0x7f809406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:58.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.747+0000 7f8092ffd640 1 -- 172.21.15.83:0/394599130 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8094071850 con 0x7f809415ba30 2024-04-17T10:51:58.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.747+0000 7f8091ffb640 1 -- 172.21.15.83:0/394599130 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3481804530 0 0) 0x7f809406e320 con 0x7f80940cfbe0 2024-04-17T10:51:58.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.747+0000 7f8092ffd640 1 --2- 172.21.15.83:0/394599130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f809415ba30 0x7f8094072330 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f808c009d90 tx=0x7f808c01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f824331e40bbd14 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:58.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.751+0000 7f8091ffb640 1 -- 172.21.15.83:0/394599130 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c0b2070 con 0x7f809415ba30 2024-04-17T10:51:58.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.751+0000 7f8091ffb640 1 -- 172.21.15.83:0/394599130 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f808c0a57e0 con 0x7f809415ba30 2024-04-17T10:51:58.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.751+0000 7f8091ffb640 1 -- 172.21.15.83:0/394599130 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c0a5bb0 con 0x7f809415ba30 2024-04-17T10:51:58.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.751+0000 7f80997f6640 1 -- 172.21.15.83:0/394599130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f809415ba30 msgr2=0x7f8094072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:58.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.751+0000 7f80997f6640 1 --2- 172.21.15.83:0/394599130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f809415ba30 0x7f8094072330 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f808c009d90 tx=0x7f808c01c8a0 comp rx=0 tx=0).stop 2024-04-17T10:51:58.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.751+0000 7f80997f6640 1 -- 172.21.15.83:0/394599130 shutdown_connections 2024-04-17T10:51:58.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.751+0000 7f80997f6640 1 --2- 172.21.15.83:0/394599130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f809415ba30 0x7f8094072330 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:58.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.751+0000 7f80997f6640 1 -- 172.21.15.83:0/394599130 >> 172.21.15.83:0/394599130 conn(0x7f809406abe0 msgr2=0x7f809406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:58.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.751+0000 7f80997f6640 1 -- 172.21.15.83:0/394599130 shutdown_connections 2024-04-17T10:51:58.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.751+0000 7f80997f6640 1 -- 172.21.15.83:0/394599130 wait complete. 2024-04-17T10:51:58.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.751+0000 7f80997f6640 1 Processor -- start 2024-04-17T10:51:58.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.751+0000 7f80997f6640 1 -- start start 2024-04-17T10:51:58.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.751+0000 7f80997f6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80940cfbe0 0x7f809413e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:58.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.751+0000 7f80997f6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f809415ba30 0x7f809415e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:58.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.751+0000 7f80997f6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f809415c960 0x7f809415e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:58.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.751+0000 7f80997f6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809406de20 con 0x7f809415c960 2024-04-17T10:51:58.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.751+0000 7f80997f6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809406dca0 con 0x7f809415ba30 2024-04-17T10:51:58.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.751+0000 7f80997f6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8094169530 con 0x7f80940cfbe0 2024-04-17T10:51:58.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.751+0000 7f80937fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f809415c960 0x7f809415e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.751+0000 7f80937fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f809415c960 0x7f809415e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43866/0 (socket says 172.21.15.83:43866) 2024-04-17T10:51:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.751+0000 7f80937fe640 1 -- 172.21.15.83:0/2636890608 learned_addr learned my addr 172.21.15.83:0/2636890608 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:51:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.751+0000 7f80927fc640 1 --2- 172.21.15.83:0/2636890608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f809415ba30 0x7f809415e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.751+0000 7f8092ffd640 1 --2- 172.21.15.83:0/2636890608 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80940cfbe0 0x7f809413e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.751+0000 7f80937fe640 1 -- 172.21.15.83:0/2636890608 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80940cfbe0 msgr2=0x7f809413e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.751+0000 7f80937fe640 1 --2- 172.21.15.83:0/2636890608 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80940cfbe0 0x7f809413e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.751+0000 7f80937fe640 1 -- 172.21.15.83:0/2636890608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f809415ba30 msgr2=0x7f809415e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.751+0000 7f80937fe640 1 --2- 172.21.15.83:0/2636890608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f809415ba30 0x7f809415e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.751+0000 7f80937fe640 1 -- 172.21.15.83:0/2636890608 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8094152680 con 0x7f809415c960 2024-04-17T10:51:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.751+0000 7f80937fe640 1 --2- 172.21.15.83:0/2636890608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f809415c960 0x7f809415e610 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f808400c450 tx=0x7f808400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.751+0000 7f807b7fe640 1 -- 172.21.15.83:0/2636890608 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8084017070 con 0x7f809415c960 2024-04-17T10:51:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.751+0000 7f807b7fe640 1 -- 172.21.15.83:0/2636890608 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8084005cf0 con 0x7f809415c960 2024-04-17T10:51:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.751+0000 7f807b7fe640 1 -- 172.21.15.83:0/2636890608 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8084007310 con 0x7f809415c960 2024-04-17T10:51:58.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.755+0000 7f80997f6640 1 -- 172.21.15.83:0/2636890608 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8094152960 con 0x7f809415c960 2024-04-17T10:51:58.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.755+0000 7f80997f6640 1 -- 172.21.15.83:0/2636890608 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8094152f20 con 0x7f809415c960 2024-04-17T10:51:58.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.755+0000 7f80997f6640 1 -- 172.21.15.83:0/2636890608 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8094074f90 con 0x7f809415c960 2024-04-17T10:51:58.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.755+0000 7f807b7fe640 1 -- 172.21.15.83:0/2636890608 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f808401a070 con 0x7f809415c960 2024-04-17T10:51:58.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.755+0000 7f807b7fe640 1 --2- 172.21.15.83:0/2636890608 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8080066f90 0x7f8080069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:51:58.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.755+0000 7f807b7fe640 1 -- 172.21.15.83:0/2636890608 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f8084084470 con 0x7f809415c960 2024-04-17T10:51:58.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.755+0000 7f8092ffd640 1 --2- 172.21.15.83:0/2636890608 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8080066f90 0x7f8080069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:51:58.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.755+0000 7f8092ffd640 1 --2- 172.21.15.83:0/2636890608 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8080066f90 0x7f8080069450 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f808c0af000 tx=0x7f808c004ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:51:58.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.759+0000 7f807b7fe640 1 -- 172.21.15.83:0/2636890608 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8084055410 con 0x7f809415c960 2024-04-17T10:51:58.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.951+0000 7f80997f6640 1 -- 172.21.15.83:0/2636890608 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8094151550 con 0x7f809415c960 2024-04-17T10:51:58.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.955+0000 7f807b7fe640 1 -- 172.21.15.83:0/2636890608 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f8084054d10 con 0x7f809415c960 2024-04-17T10:51:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:51:58.948 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:51:58.948 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:51:58.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.955+0000 7f80997f6640 1 -- 172.21.15.83:0/2636890608 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8080066f90 msgr2=0x7f8080069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:58.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.955+0000 7f80997f6640 1 --2- 172.21.15.83:0/2636890608 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8080066f90 0x7f8080069450 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f808c0af000 tx=0x7f808c004ac0 comp rx=0 tx=0).stop 2024-04-17T10:51:58.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.955+0000 7f80997f6640 1 -- 172.21.15.83:0/2636890608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f809415c960 msgr2=0x7f809415e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:51:58.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.955+0000 7f80997f6640 1 --2- 172.21.15.83:0/2636890608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f809415c960 0x7f809415e610 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f808400c450 tx=0x7f808400c910 comp rx=0 tx=0).stop 2024-04-17T10:51:58.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.955+0000 7f80997f6640 1 -- 172.21.15.83:0/2636890608 shutdown_connections 2024-04-17T10:51:58.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.955+0000 7f80997f6640 1 --2- 172.21.15.83:0/2636890608 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8080066f90 0x7f8080069450 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:58.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.955+0000 7f80997f6640 1 --2- 172.21.15.83:0/2636890608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f809415c960 0x7f809415e610 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:58.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.955+0000 7f80997f6640 1 --2- 172.21.15.83:0/2636890608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f809415ba30 0x7f809415e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:58.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.955+0000 7f80997f6640 1 --2- 172.21.15.83:0/2636890608 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80940cfbe0 0x7f809413e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:51:58.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.955+0000 7f80997f6640 1 -- 172.21.15.83:0/2636890608 >> 172.21.15.83:0/2636890608 conn(0x7f809406abe0 msgr2=0x7f809415a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:51:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.955+0000 7f80997f6640 1 -- 172.21.15.83:0/2636890608 shutdown_connections 2024-04-17T10:51:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:51:58.955+0000 7f80997f6640 1 -- 172.21.15.83:0/2636890608 wait complete. 2024-04-17T10:51:59.370 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:51:59.370 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:00.370 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:00.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.511+0000 7f6fbc7bd640 1 Processor -- start 2024-04-17T10:52:00.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.511+0000 7f6fbc7bd640 1 -- start start 2024-04-17T10:52:00.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.511+0000 7f6fbc7bd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6fb415dfa0 0x7f6fb40b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:00.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.511+0000 7f6fbc7bd640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fb4162a00 con 0x7f6fb40b5b40 2024-04-17T10:52:00.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.511+0000 7f6fbc7bd640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fb4161e80 con 0x7f6fb40ce980 2024-04-17T10:52:00.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.511+0000 7f6fbc7bd640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb4162160 con 0x7f6fb415dfa0 2024-04-17T10:52:00.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.511+0000 7f6fbad5a640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6fb40b5b40 0x7f6fb40d1d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58694/0 (socket says 172.21.15.83:58694) 2024-04-17T10:52:00.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.511+0000 7f6fbad5a640 1 -- 172.21.15.83:0/421905739 learned_addr learned my addr 172.21.15.83:0/421905739 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:52:00.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.511+0000 7f6fba559640 1 --2- 172.21.15.83:0/421905739 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6fb415dfa0 0x7f6fb40b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:00.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.511+0000 7f6fb9557640 1 -- 172.21.15.83:0/421905739 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1184981118 0 0) 0x7f6fb4162a00 con 0x7f6fb40b5b40 2024-04-17T10:52:00.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.511+0000 7f6fb9557640 1 -- 172.21.15.83:0/421905739 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fa0003530 con 0x7f6fb40b5b40 2024-04-17T10:52:00.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.511+0000 7f6fb9557640 1 -- 172.21.15.83:0/421905739 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1242924035 0 0) 0x7f6fa0003530 con 0x7f6fb40b5b40 2024-04-17T10:52:00.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.511+0000 7f6fb9557640 1 -- 172.21.15.83:0/421905739 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6fb415dfa0 msgr2=0x7f6fb40b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:00.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.511+0000 7f6fb9557640 1 --2- 172.21.15.83:0/421905739 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6fb415dfa0 0x7f6fb40b6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:00.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.511+0000 7f6fb9557640 1 -- 172.21.15.83:0/421905739 >> v1:172.21.15.167:6789/0 conn(0x7f6fb40ce980 legacy=0x7f6fb40b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:00.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.511+0000 7f6fb9557640 1 -- 172.21.15.83:0/421905739 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb4162440 con 0x7f6fb40b5b40 2024-04-17T10:52:00.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.511+0000 7f6fb9557640 1 -- 172.21.15.83:0/421905739 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f6fac001a00 con 0x7f6fb40b5b40 2024-04-17T10:52:00.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.511+0000 7f6fb9557640 1 -- 172.21.15.83:0/421905739 >> v1:172.21.15.83:6789/0 conn(0x7f6fb40b5b40 legacy=0x7f6fb40d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:00.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.511+0000 7f6fb9557640 1 --2- 172.21.15.83:0/421905739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fa00039f0 0x7f6fa0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:00.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.511+0000 7f6fb9557640 1 --2- 172.21.15.83:0/421905739 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6fb415dfa0 0x7f6fa0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:00.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.511+0000 7f6fb9557640 1 --2- 172.21.15.83:0/421905739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fa0004d50 0x7f6fa00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:00.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.511+0000 7f6fb9557640 1 -- 172.21.15.83:0/421905739 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb4162160 con 0x7f6fa0004d50 2024-04-17T10:52:00.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.511+0000 7f6fb9557640 1 -- 172.21.15.83:0/421905739 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa000bb00 con 0x7f6fa00039f0 2024-04-17T10:52:00.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.511+0000 7f6fb9557640 1 -- 172.21.15.83:0/421905739 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa000bd00 con 0x7f6fb415dfa0 2024-04-17T10:52:00.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.511+0000 7f6fbad5a640 1 --2- 172.21.15.83:0/421905739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fa0004d50 0x7f6fa00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:00.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.511+0000 7f6fbad5a640 1 -- 172.21.15.83:0/421905739 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6fb415dfa0 msgr2=0x7f6fa0004810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T10:52:00.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.511+0000 7f6fbad5a640 1 --2- 172.21.15.83:0/421905739 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6fb415dfa0 0x7f6fa0004810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:00.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.511+0000 7f6fbad5a640 1 -- 172.21.15.83:0/421905739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fa00039f0 msgr2=0x7f6fa0003e10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T10:52:00.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.511+0000 7f6fbad5a640 1 --2- 172.21.15.83:0/421905739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fa00039f0 0x7f6fa0003e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:00.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.511+0000 7f6fbad5a640 1 -- 172.21.15.83:0/421905739 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa0003530 con 0x7f6fa0004d50 2024-04-17T10:52:00.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.515+0000 7f6fbad5a640 1 --2- 172.21.15.83:0/421905739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fa0004d50 0x7f6fa00051d0 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00a850 tx=0x7f6fac01fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d5e34e712aa63 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:00.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.515+0000 7f6fb9557640 1 -- 172.21.15.83:0/421905739 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac00a4a0 con 0x7f6fa0004d50 2024-04-17T10:52:00.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.515+0000 7f6fb9557640 1 -- 172.21.15.83:0/421905739 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fac0ad070 con 0x7f6fa0004d50 2024-04-17T10:52:00.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.515+0000 7f6fb9557640 1 -- 172.21.15.83:0/421905739 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac0a9980 con 0x7f6fa0004d50 2024-04-17T10:52:00.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.515+0000 7f6fbc7bd640 1 -- 172.21.15.83:0/421905739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fa0004d50 msgr2=0x7f6fa00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:00.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.515+0000 7f6fbc7bd640 1 --2- 172.21.15.83:0/421905739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fa0004d50 0x7f6fa00051d0 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00a850 tx=0x7f6fac01fb00 comp rx=0 tx=0).stop 2024-04-17T10:52:00.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.515+0000 7f6fbad5a640 1 -- 172.21.15.83:0/421905739 reap_dead start 2024-04-17T10:52:00.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.515+0000 7f6fbc7bd640 1 -- 172.21.15.83:0/421905739 shutdown_connections 2024-04-17T10:52:00.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.515+0000 7f6fbc7bd640 1 -- 172.21.15.83:0/421905739 >> 172.21.15.83:0/421905739 conn(0x7f6fb40d0600 msgr2=0x7f6fb40d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:00.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.515+0000 7f6fbc7bd640 1 -- 172.21.15.83:0/421905739 shutdown_connections 2024-04-17T10:52:00.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.515+0000 7f6fbc7bd640 1 -- 172.21.15.83:0/421905739 wait complete. 2024-04-17T10:52:00.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.515+0000 7f6fbc7bd640 1 Processor -- start 2024-04-17T10:52:00.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.515+0000 7f6fbc7bd640 1 -- start start 2024-04-17T10:52:00.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.519+0000 7f6fbc7bd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fa0004d50 0x7f6fb40c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.519+0000 7f6fbc7bd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fb41608e0 0x7f6fb41f7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.519+0000 7f6fbc7bd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6fb41f7b90 0x7f6fb414b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.519+0000 7f6fbc7bd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb4161b10 con 0x7f6fb41608e0 2024-04-17T10:52:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.519+0000 7f6fbc7bd640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb415e380 con 0x7f6fa0004d50 2024-04-17T10:52:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.519+0000 7f6fbc7bd640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb40cee00 con 0x7f6fb41f7b90 2024-04-17T10:52:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.519+0000 7f6fb9d58640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fb41608e0 0x7f6fb41f7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.519+0000 7f6fb9d58640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fb41608e0 0x7f6fb41f7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43878/0 (socket says 172.21.15.83:43878) 2024-04-17T10:52:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.519+0000 7f6fb9d58640 1 -- 172.21.15.83:0/1476818772 learned_addr learned my addr 172.21.15.83:0/1476818772 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.519+0000 7f6fba559640 1 --2- 172.21.15.83:0/1476818772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fa0004d50 0x7f6fb40c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.519+0000 7f6fb9d58640 1 -- 172.21.15.83:0/1476818772 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6fb41f7b90 msgr2=0x7f6fb414b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.519+0000 7f6fb9d58640 1 --2- 172.21.15.83:0/1476818772 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6fb41f7b90 0x7f6fb414b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.519+0000 7f6fb9d58640 1 -- 172.21.15.83:0/1476818772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fa0004d50 msgr2=0x7f6fb40c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.519+0000 7f6fb9d58640 1 --2- 172.21.15.83:0/1476818772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fa0004d50 0x7f6fb40c7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.519+0000 7f6fb9d58640 1 -- 172.21.15.83:0/1476818772 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb40c8840 con 0x7f6fb41608e0 2024-04-17T10:52:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.519+0000 7f6fb9d58640 1 --2- 172.21.15.83:0/1476818772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fb41608e0 0x7f6fb41f7650 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400c500 tx=0x7f6fa400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.519+0000 7f6faaffd640 1 -- 172.21.15.83:0/1476818772 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4017070 con 0x7f6fb41608e0 2024-04-17T10:52:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.519+0000 7f6faaffd640 1 -- 172.21.15.83:0/1476818772 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa40052d0 con 0x7f6fb41608e0 2024-04-17T10:52:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.519+0000 7f6fbc7bd640 1 -- 172.21.15.83:0/1476818772 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb40c8b20 con 0x7f6fb41608e0 2024-04-17T10:52:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.519+0000 7f6faaffd640 1 -- 172.21.15.83:0/1476818772 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4012870 con 0x7f6fb41608e0 2024-04-17T10:52:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.519+0000 7f6fbc7bd640 1 -- 172.21.15.83:0/1476818772 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb40c93c0 con 0x7f6fb41608e0 2024-04-17T10:52:00.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.519+0000 7f6fbc7bd640 1 -- 172.21.15.83:0/1476818772 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb40681a0 con 0x7f6fb41608e0 2024-04-17T10:52:00.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.519+0000 7f6faaffd640 1 -- 172.21.15.83:0/1476818772 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6fa401a070 con 0x7f6fb41608e0 2024-04-17T10:52:00.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.519+0000 7f6faaffd640 1 --2- 172.21.15.83:0/1476818772 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6f98066f60 0x7f6f98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:00.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.519+0000 7f6faaffd640 1 -- 172.21.15.83:0/1476818772 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f6fa4084410 con 0x7f6fb41608e0 2024-04-17T10:52:00.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.519+0000 7f6fba559640 1 --2- 172.21.15.83:0/1476818772 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6f98066f60 0x7f6f98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:00.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.519+0000 7f6fba559640 1 --2- 172.21.15.83:0/1476818772 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6f98066f60 0x7f6f98069420 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f6fb00099b0 tx=0x7f6fb0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:00.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.523+0000 7f6faaffd640 1 -- 172.21.15.83:0/1476818772 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6fa4005480 con 0x7f6fb41608e0 2024-04-17T10:52:00.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.715+0000 7f6fbc7bd640 1 -- 172.21.15.83:0/1476818772 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6fb40c8cf0 con 0x7f6fb41608e0 2024-04-17T10:52:00.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.715+0000 7f6faaffd640 1 -- 172.21.15.83:0/1476818772 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f6fa40553c0 con 0x7f6fb41608e0 2024-04-17T10:52:00.709 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:00.709 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:00.709 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:00.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.719+0000 7f6fbc7bd640 1 -- 172.21.15.83:0/1476818772 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6f98066f60 msgr2=0x7f6f98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:00.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.719+0000 7f6fbc7bd640 1 --2- 172.21.15.83:0/1476818772 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6f98066f60 0x7f6f98069420 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f6fb00099b0 tx=0x7f6fb0008040 comp rx=0 tx=0).stop 2024-04-17T10:52:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.719+0000 7f6fbc7bd640 1 -- 172.21.15.83:0/1476818772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fb41608e0 msgr2=0x7f6fb41f7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.719+0000 7f6fbc7bd640 1 --2- 172.21.15.83:0/1476818772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fb41608e0 0x7f6fb41f7650 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400c500 tx=0x7f6fa400c9c0 comp rx=0 tx=0).stop 2024-04-17T10:52:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.719+0000 7f6fbc7bd640 1 -- 172.21.15.83:0/1476818772 shutdown_connections 2024-04-17T10:52:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.719+0000 7f6fbc7bd640 1 --2- 172.21.15.83:0/1476818772 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6f98066f60 0x7f6f98069420 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.719+0000 7f6fbc7bd640 1 --2- 172.21.15.83:0/1476818772 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6fb41f7b90 0x7f6fb414b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.719+0000 7f6fbc7bd640 1 --2- 172.21.15.83:0/1476818772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fb41608e0 0x7f6fb41f7650 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.719+0000 7f6fbc7bd640 1 --2- 172.21.15.83:0/1476818772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fa0004d50 0x7f6fb40c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.719+0000 7f6fbc7bd640 1 -- 172.21.15.83:0/1476818772 >> 172.21.15.83:0/1476818772 conn(0x7f6fb40d0600 msgr2=0x7f6fb4152eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.719+0000 7f6fbc7bd640 1 -- 172.21.15.83:0/1476818772 shutdown_connections 2024-04-17T10:52:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:00.719+0000 7f6fbc7bd640 1 -- 172.21.15.83:0/1476818772 wait complete. 2024-04-17T10:52:01.131 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:01.131 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:02.132 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:02.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.271+0000 7f494e17f640 1 Processor -- start 2024-04-17T10:52:02.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.271+0000 7f494e17f640 1 -- start start 2024-04-17T10:52:02.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.271+0000 7f494e17f640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f494815ba30 0x7f4948072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:02.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.271+0000 7f494e17f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49480cfbe0 0x7f494806c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:02.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.271+0000 7f494e17f640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f494806e320 con 0x7f494815c960 2024-04-17T10:52:02.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.271+0000 7f494e17f640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4948071ea0 con 0x7f494815ba30 2024-04-17T10:52:02.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.271+0000 7f494e17f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f494806e040 con 0x7f49480cfbe0 2024-04-17T10:52:02.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.271+0000 7f4947fff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49480cfbe0 0x7f494806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:02.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.271+0000 7f4947fff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49480cfbe0 0x7f494806c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:60288/0 (socket says 172.21.15.83:60288) 2024-04-17T10:52:02.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.271+0000 7f4947fff640 1 -- 172.21.15.83:0/3515033667 learned_addr learned my addr 172.21.15.83:0/3515033667 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:02.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.271+0000 7f49477fe640 1 --2- 172.21.15.83:0/3515033667 >> v2:172.21.15.167:3300/0 conn(0x7f494815ba30 0x7f4948072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:02.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.271+0000 7f49477fe640 1 -- 172.21.15.83:0/3515033667 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49480cfbe0 msgr2=0x7f494806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:02.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.271+0000 7f49477fe640 1 --2- 172.21.15.83:0/3515033667 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49480cfbe0 0x7f494806c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:02.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.271+0000 7f49477fe640 1 -- 172.21.15.83:0/3515033667 >> v1:172.21.15.167:6789/0 conn(0x7f494815c960 legacy=0x7f49480cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:02.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.271+0000 7f49477fe640 1 -- 172.21.15.83:0/3515033667 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4948071850 con 0x7f494815ba30 2024-04-17T10:52:02.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.271+0000 7f4947fff640 1 --2- 172.21.15.83:0/3515033667 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49480cfbe0 0x7f494806c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:52:02.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.275+0000 7f49477fe640 1 --2- 172.21.15.83:0/3515033667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f494815ba30 0x7f4948072330 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f49380097c0 tx=0x7f493801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2e5d2dd515a17d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:02.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.275+0000 7f4946ffd640 1 -- 172.21.15.83:0/3515033667 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49380b0070 con 0x7f494815ba30 2024-04-17T10:52:02.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.275+0000 7f4946ffd640 1 -- 172.21.15.83:0/3515033667 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49380ab600 con 0x7f494815ba30 2024-04-17T10:52:02.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.275+0000 7f4946ffd640 1 -- 172.21.15.83:0/3515033667 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49380ab9d0 con 0x7f494815ba30 2024-04-17T10:52:02.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.275+0000 7f494e17f640 1 -- 172.21.15.83:0/3515033667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f494815ba30 msgr2=0x7f4948072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:02.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.275+0000 7f494e17f640 1 --2- 172.21.15.83:0/3515033667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f494815ba30 0x7f4948072330 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f49380097c0 tx=0x7f493801c410 comp rx=0 tx=0).stop 2024-04-17T10:52:02.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.275+0000 7f494e17f640 1 -- 172.21.15.83:0/3515033667 shutdown_connections 2024-04-17T10:52:02.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.275+0000 7f494e17f640 1 --2- 172.21.15.83:0/3515033667 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49480cfbe0 0x7f494806c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:02.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.275+0000 7f494e17f640 1 --2- 172.21.15.83:0/3515033667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f494815ba30 0x7f4948072330 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:02.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.275+0000 7f494e17f640 1 -- 172.21.15.83:0/3515033667 >> 172.21.15.83:0/3515033667 conn(0x7f494806abe0 msgr2=0x7f494806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:02.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.275+0000 7f494e17f640 1 -- 172.21.15.83:0/3515033667 shutdown_connections 2024-04-17T10:52:02.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.275+0000 7f494e17f640 1 -- 172.21.15.83:0/3515033667 wait complete. 2024-04-17T10:52:02.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.275+0000 7f494e17f640 1 Processor -- start 2024-04-17T10:52:02.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.275+0000 7f494e17f640 1 -- start start 2024-04-17T10:52:02.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.279+0000 7f494e17f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49480cfbe0 0x7f4948147a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:02.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.279+0000 7f494e17f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f494815ba30 0x7f49480b4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:02.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.279+0000 7f494e17f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f494815c960 0x7f49480b2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:02.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.279+0000 7f494e17f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494815cee0 con 0x7f49480cfbe0 2024-04-17T10:52:02.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.279+0000 7f494e17f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494815cd60 con 0x7f494815ba30 2024-04-17T10:52:02.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.279+0000 7f494e17f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4948168520 con 0x7f494815c960 2024-04-17T10:52:02.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.279+0000 7f49477fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49480cfbe0 0x7f4948147a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:02.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.279+0000 7f49477fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49480cfbe0 0x7f4948147a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43892/0 (socket says 172.21.15.83:43892) 2024-04-17T10:52:02.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.279+0000 7f49477fe640 1 -- 172.21.15.83:0/2985515483 learned_addr learned my addr 172.21.15.83:0/2985515483 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:02.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.279+0000 7f493effd640 1 --2- 172.21.15.83:0/2985515483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f494815ba30 0x7f49480b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:02.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.279+0000 7f4947fff640 1 --2- 172.21.15.83:0/2985515483 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f494815c960 0x7f49480b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:02.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.279+0000 7f49477fe640 1 -- 172.21.15.83:0/2985515483 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f494815c960 msgr2=0x7f49480b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:02.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.279+0000 7f49477fe640 1 --2- 172.21.15.83:0/2985515483 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f494815c960 0x7f49480b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:02.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.279+0000 7f49477fe640 1 -- 172.21.15.83:0/2985515483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f494815ba30 msgr2=0x7f49480b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:02.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.279+0000 7f49477fe640 1 --2- 172.21.15.83:0/2985515483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f494815ba30 0x7f49480b4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:02.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.279+0000 7f49477fe640 1 -- 172.21.15.83:0/2985515483 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49480b30c0 con 0x7f49480cfbe0 2024-04-17T10:52:02.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.279+0000 7f4947fff640 1 --2- 172.21.15.83:0/2985515483 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f494815c960 0x7f49480b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:52:02.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.279+0000 7f493effd640 1 --2- 172.21.15.83:0/2985515483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f494815ba30 0x7f49480b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:52:02.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.279+0000 7f49477fe640 1 --2- 172.21.15.83:0/2985515483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49480cfbe0 0x7f4948147a80 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f493801a040 tx=0x7f493801ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:02.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.279+0000 7f4944ff9640 1 -- 172.21.15.83:0/2985515483 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49380b0050 con 0x7f49480cfbe0 2024-04-17T10:52:02.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.279+0000 7f4944ff9640 1 -- 172.21.15.83:0/2985515483 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49380bc070 con 0x7f49480cfbe0 2024-04-17T10:52:02.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.279+0000 7f494e17f640 1 -- 172.21.15.83:0/2985515483 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49480b33a0 con 0x7f49480cfbe0 2024-04-17T10:52:02.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.279+0000 7f4944ff9640 1 -- 172.21.15.83:0/2985515483 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49380b5830 con 0x7f49480cfbe0 2024-04-17T10:52:02.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.279+0000 7f494e17f640 1 -- 172.21.15.83:0/2985515483 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49480b3960 con 0x7f49480cfbe0 2024-04-17T10:52:02.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.279+0000 7f494e17f640 1 -- 172.21.15.83:0/2985515483 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4948074f90 con 0x7f49480cfbe0 2024-04-17T10:52:02.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.279+0000 7f4944ff9640 1 -- 172.21.15.83:0/2985515483 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f49380b59d0 con 0x7f49480cfbe0 2024-04-17T10:52:02.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.279+0000 7f4944ff9640 1 --2- 172.21.15.83:0/2985515483 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4928066f60 0x7f4928069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:02.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.279+0000 7f4944ff9640 1 -- 172.21.15.83:0/2985515483 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f493811f8d0 con 0x7f49480cfbe0 2024-04-17T10:52:02.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.279+0000 7f493effd640 1 --2- 172.21.15.83:0/2985515483 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4928066f60 0x7f4928069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:02.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.279+0000 7f493effd640 1 --2- 172.21.15.83:0/2985515483 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4928066f60 0x7f4928069420 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f49300016f0 tx=0x7f4930009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:02.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.283+0000 7f4944ff9640 1 -- 172.21.15.83:0/2985515483 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4938016090 con 0x7f49480cfbe0 2024-04-17T10:52:02.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.483+0000 7f494e17f640 1 -- 172.21.15.83:0/2985515483 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4948074480 con 0x7f49480cfbe0 2024-04-17T10:52:02.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.487+0000 7f4944ff9640 1 -- 172.21.15.83:0/2985515483 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f493811f030 con 0x7f49480cfbe0 2024-04-17T10:52:02.480 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:02.480 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:02.480 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:02.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.487+0000 7f494e17f640 1 -- 172.21.15.83:0/2985515483 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4928066f60 msgr2=0x7f4928069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:02.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.487+0000 7f494e17f640 1 --2- 172.21.15.83:0/2985515483 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4928066f60 0x7f4928069420 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f49300016f0 tx=0x7f4930009040 comp rx=0 tx=0).stop 2024-04-17T10:52:02.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.487+0000 7f494e17f640 1 -- 172.21.15.83:0/2985515483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49480cfbe0 msgr2=0x7f4948147a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:02.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.487+0000 7f494e17f640 1 --2- 172.21.15.83:0/2985515483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49480cfbe0 0x7f4948147a80 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f493801a040 tx=0x7f493801ac50 comp rx=0 tx=0).stop 2024-04-17T10:52:02.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.487+0000 7f494e17f640 1 -- 172.21.15.83:0/2985515483 shutdown_connections 2024-04-17T10:52:02.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.487+0000 7f494e17f640 1 --2- 172.21.15.83:0/2985515483 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4928066f60 0x7f4928069420 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:02.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.487+0000 7f494e17f640 1 --2- 172.21.15.83:0/2985515483 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f494815c960 0x7f49480b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:02.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.487+0000 7f494e17f640 1 --2- 172.21.15.83:0/2985515483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f494815ba30 0x7f49480b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:02.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.487+0000 7f494e17f640 1 --2- 172.21.15.83:0/2985515483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49480cfbe0 0x7f4948147a80 unknown :-1 s=CLOSED pgs=2187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:02.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.487+0000 7f494e17f640 1 -- 172.21.15.83:0/2985515483 >> 172.21.15.83:0/2985515483 conn(0x7f494806abe0 msgr2=0x7f494815dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:02.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.487+0000 7f494e17f640 1 -- 172.21.15.83:0/2985515483 shutdown_connections 2024-04-17T10:52:02.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:02.487+0000 7f494e17f640 1 -- 172.21.15.83:0/2985515483 wait complete. 2024-04-17T10:52:02.901 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:02.901 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:03.902 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:04.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.043+0000 7effd2909640 1 Processor -- start 2024-04-17T10:52:04.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.043+0000 7effd2909640 1 -- start start 2024-04-17T10:52:04.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.043+0000 7effd2909640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7effcc0b5b40 0x7effcc0d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:04.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.043+0000 7effd2909640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effcc162a00 con 0x7effcc0ce980 2024-04-17T10:52:04.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.043+0000 7effd2909640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effcc161e80 con 0x7effcc15dfa0 2024-04-17T10:52:04.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.043+0000 7effd2909640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effcc162160 con 0x7effcc0b5b40 2024-04-17T10:52:04.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.043+0000 7effcb7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7effcc0ce980 0x7effcc0b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58700/0 (socket says 172.21.15.83:58700) 2024-04-17T10:52:04.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.043+0000 7effcb7fe640 1 -- 172.21.15.83:0/686649227 learned_addr learned my addr 172.21.15.83:0/686649227 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:52:04.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.043+0000 7effd0ea6640 1 --2- 172.21.15.83:0/686649227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7effcc0b5b40 0x7effcc0d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:04.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.043+0000 7effcaffd640 1 -- 172.21.15.83:0/686649227 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2653377110 0 0) 0x7effcc162a00 con 0x7effcc0ce980 2024-04-17T10:52:04.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.043+0000 7effcaffd640 1 -- 172.21.15.83:0/686649227 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effb8003530 con 0x7effcc0ce980 2024-04-17T10:52:04.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.043+0000 7effcaffd640 1 -- 172.21.15.83:0/686649227 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2087636719 0 0) 0x7effb8003530 con 0x7effcc0ce980 2024-04-17T10:52:04.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.043+0000 7effcaffd640 1 -- 172.21.15.83:0/686649227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7effcc0b5b40 msgr2=0x7effcc0d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:04.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.043+0000 7effcaffd640 1 --2- 172.21.15.83:0/686649227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7effcc0b5b40 0x7effcc0d1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:04.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.043+0000 7effcaffd640 1 -- 172.21.15.83:0/686649227 >> v1:172.21.15.167:6789/0 conn(0x7effcc15dfa0 legacy=0x7effcc0b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:04.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.043+0000 7effcaffd640 1 -- 172.21.15.83:0/686649227 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effcc162440 con 0x7effcc0ce980 2024-04-17T10:52:04.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.043+0000 7effcaffd640 1 -- 172.21.15.83:0/686649227 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7effb4001680 con 0x7effcc0ce980 2024-04-17T10:52:04.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.043+0000 7effcaffd640 1 -- 172.21.15.83:0/686649227 >> v1:172.21.15.83:6789/0 conn(0x7effcc0ce980 legacy=0x7effcc0b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:04.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.043+0000 7effcaffd640 1 --2- 172.21.15.83:0/686649227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7effcc0b5b40 0x7effb80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:04.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.043+0000 7effcaffd640 1 --2- 172.21.15.83:0/686649227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effb8003f70 0x7effb8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:04.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.043+0000 7effcaffd640 1 --2- 172.21.15.83:0/686649227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effb8004d50 0x7effb80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:04.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.043+0000 7effcaffd640 1 -- 172.21.15.83:0/686649227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effcc162160 con 0x7effb8004d50 2024-04-17T10:52:04.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.043+0000 7effcaffd640 1 -- 172.21.15.83:0/686649227 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb800bb60 con 0x7effb8003f70 2024-04-17T10:52:04.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.043+0000 7effcaffd640 1 -- 172.21.15.83:0/686649227 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb800bd60 con 0x7effcc0b5b40 2024-04-17T10:52:04.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.043+0000 7effd0ea6640 1 --2- 172.21.15.83:0/686649227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effb8004d50 0x7effb80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:04.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.043+0000 7effcb7fe640 1 --2- 172.21.15.83:0/686649227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effb8003f70 0x7effb8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:04.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.043+0000 7effcbfff640 1 --2- 172.21.15.83:0/686649227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7effcc0b5b40 0x7effb80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:04.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.043+0000 7effd0ea6640 1 -- 172.21.15.83:0/686649227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7effcc0b5b40 msgr2=0x7effb80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:04.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.043+0000 7effd0ea6640 1 --2- 172.21.15.83:0/686649227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7effcc0b5b40 0x7effb80039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:04.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.043+0000 7effd0ea6640 1 -- 172.21.15.83:0/686649227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effb8003f70 msgr2=0x7effb8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:04.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.043+0000 7effd0ea6640 1 --2- 172.21.15.83:0/686649227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effb8003f70 0x7effb8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:04.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.043+0000 7effd0ea6640 1 -- 172.21.15.83:0/686649227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb8003530 con 0x7effb8004d50 2024-04-17T10:52:04.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.047+0000 7effd0ea6640 1 --2- 172.21.15.83:0/686649227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effb8004d50 0x7effb80051d0 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7effc4009ab0 tx=0x7effc401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b668fe747851ce server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:04.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.047+0000 7effcaffd640 1 -- 172.21.15.83:0/686649227 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc40b2070 con 0x7effb8004d50 2024-04-17T10:52:04.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.047+0000 7effcaffd640 1 -- 172.21.15.83:0/686649227 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc40a5530 con 0x7effb8004d50 2024-04-17T10:52:04.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.047+0000 7effcaffd640 1 -- 172.21.15.83:0/686649227 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc40096b0 con 0x7effb8004d50 2024-04-17T10:52:04.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.047+0000 7effd2909640 1 -- 172.21.15.83:0/686649227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effb8004d50 msgr2=0x7effb80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:04.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.047+0000 7effd2909640 1 --2- 172.21.15.83:0/686649227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effb8004d50 0x7effb80051d0 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7effc4009ab0 tx=0x7effc401ca50 comp rx=0 tx=0).stop 2024-04-17T10:52:04.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.047+0000 7effd0ea6640 1 -- 172.21.15.83:0/686649227 reap_dead start 2024-04-17T10:52:04.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.047+0000 7effd2909640 1 -- 172.21.15.83:0/686649227 shutdown_connections 2024-04-17T10:52:04.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.047+0000 7effd2909640 1 -- 172.21.15.83:0/686649227 >> 172.21.15.83:0/686649227 conn(0x7effcc0d0600 msgr2=0x7effcc0d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:04.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.047+0000 7effd2909640 1 -- 172.21.15.83:0/686649227 shutdown_connections 2024-04-17T10:52:04.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.047+0000 7effd2909640 1 -- 172.21.15.83:0/686649227 wait complete. 2024-04-17T10:52:04.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.047+0000 7effd2909640 1 Processor -- start 2024-04-17T10:52:04.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.047+0000 7effd2909640 1 -- start start 2024-04-17T10:52:04.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.051+0000 7effd2909640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7effb8004d50 0x7effcc0c6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:04.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.051+0000 7effd2909640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effcc14f160 0x7effcc15f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:04.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.051+0000 7effd2909640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effcc14cce0 0x7effcc14d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:04.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.051+0000 7effd2909640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effcc161b10 con 0x7effcc14f160 2024-04-17T10:52:04.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.051+0000 7effd2909640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effcc15e380 con 0x7effcc14cce0 2024-04-17T10:52:04.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.051+0000 7effd2909640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effcc0cee00 con 0x7effb8004d50 2024-04-17T10:52:04.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.051+0000 7effcb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effcc14f160 0x7effcc15f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:04.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.051+0000 7effcb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effcc14f160 0x7effcc15f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43914/0 (socket says 172.21.15.83:43914) 2024-04-17T10:52:04.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.051+0000 7effcb7fe640 1 -- 172.21.15.83:0/2342146531 learned_addr learned my addr 172.21.15.83:0/2342146531 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:04.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.051+0000 7effd0ea6640 1 --2- 172.21.15.83:0/2342146531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effcc14cce0 0x7effcc14d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:04.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.051+0000 7effcbfff640 1 --2- 172.21.15.83:0/2342146531 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7effb8004d50 0x7effcc0c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:04.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.051+0000 7effcb7fe640 1 -- 172.21.15.83:0/2342146531 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7effb8004d50 msgr2=0x7effcc0c6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:04.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.051+0000 7effcb7fe640 1 --2- 172.21.15.83:0/2342146531 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7effb8004d50 0x7effcc0c6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:04.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.051+0000 7effcb7fe640 1 -- 172.21.15.83:0/2342146531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effcc14cce0 msgr2=0x7effcc14d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:04.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.051+0000 7effcb7fe640 1 --2- 172.21.15.83:0/2342146531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effcc14cce0 0x7effcc14d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:04.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.051+0000 7effcb7fe640 1 -- 172.21.15.83:0/2342146531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effcc160150 con 0x7effcc14f160 2024-04-17T10:52:04.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.051+0000 7effcb7fe640 1 --2- 172.21.15.83:0/2342146531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effcc14f160 0x7effcc15f7e0 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7effb4011980 tx=0x7effb4011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:04.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.051+0000 7effc8ff9640 1 -- 172.21.15.83:0/2342146531 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb4010040 con 0x7effcc14f160 2024-04-17T10:52:04.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.051+0000 7effc8ff9640 1 -- 172.21.15.83:0/2342146531 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb401a070 con 0x7effcc14f160 2024-04-17T10:52:04.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.051+0000 7effd2909640 1 -- 172.21.15.83:0/2342146531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effcc160430 con 0x7effcc14f160 2024-04-17T10:52:04.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.051+0000 7effc8ff9640 1 -- 172.21.15.83:0/2342146531 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb400fe20 con 0x7effcc14f160 2024-04-17T10:52:04.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.051+0000 7effd2909640 1 -- 172.21.15.83:0/2342146531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effcc1609f0 con 0x7effcc14f160 2024-04-17T10:52:04.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.051+0000 7effd2909640 1 -- 172.21.15.83:0/2342146531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effcc0681a0 con 0x7effcc14f160 2024-04-17T10:52:04.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.051+0000 7effc8ff9640 1 -- 172.21.15.83:0/2342146531 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7effb401f460 con 0x7effcc14f160 2024-04-17T10:52:04.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.051+0000 7effc8ff9640 1 --2- 172.21.15.83:0/2342146531 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7effac066f60 0x7effac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:04.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.051+0000 7effc8ff9640 1 -- 172.21.15.83:0/2342146531 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7effb4088380 con 0x7effcc14f160 2024-04-17T10:52:04.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.051+0000 7effcbfff640 1 --2- 172.21.15.83:0/2342146531 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7effac066f60 0x7effac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:04.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.051+0000 7effcbfff640 1 --2- 172.21.15.83:0/2342146531 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7effac066f60 0x7effac069420 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7effcc1501f0 tx=0x7effbc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:04.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.055+0000 7effc8ff9640 1 -- 172.21.15.83:0/2342146531 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7effb40593b0 con 0x7effcc14f160 2024-04-17T10:52:04.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.247+0000 7effd2909640 1 -- 172.21.15.83:0/2342146531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7effcc074f90 con 0x7effcc14f160 2024-04-17T10:52:04.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.247+0000 7effc8ff9640 1 -- 172.21.15.83:0/2342146531 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7effb4058cb0 con 0x7effcc14f160 2024-04-17T10:52:04.241 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:04.241 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:04.241 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:04.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.251+0000 7effd2909640 1 -- 172.21.15.83:0/2342146531 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7effac066f60 msgr2=0x7effac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:04.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.251+0000 7effd2909640 1 --2- 172.21.15.83:0/2342146531 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7effac066f60 0x7effac069420 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7effcc1501f0 tx=0x7effbc009040 comp rx=0 tx=0).stop 2024-04-17T10:52:04.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.251+0000 7effd2909640 1 -- 172.21.15.83:0/2342146531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effcc14f160 msgr2=0x7effcc15f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:04.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.251+0000 7effd2909640 1 --2- 172.21.15.83:0/2342146531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effcc14f160 0x7effcc15f7e0 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7effb4011980 tx=0x7effb4011e40 comp rx=0 tx=0).stop 2024-04-17T10:52:04.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.251+0000 7effd2909640 1 -- 172.21.15.83:0/2342146531 shutdown_connections 2024-04-17T10:52:04.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.251+0000 7effd2909640 1 --2- 172.21.15.83:0/2342146531 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7effac066f60 0x7effac069420 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:04.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.251+0000 7effd2909640 1 --2- 172.21.15.83:0/2342146531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effcc14cce0 0x7effcc14d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:04.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.251+0000 7effd2909640 1 --2- 172.21.15.83:0/2342146531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effcc14f160 0x7effcc15f7e0 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:04.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.251+0000 7effd2909640 1 --2- 172.21.15.83:0/2342146531 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7effb8004d50 0x7effcc0c6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:04.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.251+0000 7effd2909640 1 -- 172.21.15.83:0/2342146531 >> 172.21.15.83:0/2342146531 conn(0x7effcc0d0600 msgr2=0x7effcc0d1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:04.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.251+0000 7effd2909640 1 -- 172.21.15.83:0/2342146531 shutdown_connections 2024-04-17T10:52:04.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:04.251+0000 7effd2909640 1 -- 172.21.15.83:0/2342146531 wait complete. 2024-04-17T10:52:04.661 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:04.661 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:05.662 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:05.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.799+0000 7f9ef78b7640 1 Processor -- start 2024-04-17T10:52:05.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.799+0000 7f9ef78b7640 1 -- start start 2024-04-17T10:52:05.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.803+0000 7f9ef78b7640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9ef015f1d0 0x7f9ef015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:05.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.803+0000 7f9ef78b7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ef00d22d0 0x7f9ef00ae520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:05.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.803+0000 7f9ef78b7640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ef015f5b0 con 0x7f9ef00b2de0 2024-04-17T10:52:05.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.803+0000 7f9ef78b7640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ef00b2620 con 0x7f9ef015f1d0 2024-04-17T10:52:05.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.803+0000 7f9ef78b7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef00b2340 con 0x7f9ef00d22d0 2024-04-17T10:52:05.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.803+0000 7f9ef5e54640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ef00d22d0 0x7f9ef00ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:05.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.803+0000 7f9ef5e54640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ef00d22d0 0x7f9ef00ae520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:60342/0 (socket says 172.21.15.83:60342) 2024-04-17T10:52:05.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.803+0000 7f9ef5e54640 1 -- 172.21.15.83:0/3229408018 learned_addr learned my addr 172.21.15.83:0/3229408018 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:05.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.803+0000 7f9ef5653640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9ef015f1d0 0x7f9ef015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:05.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.803+0000 7f9ef5653640 1 -- 172.21.15.83:0/3229408018 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ef00d22d0 msgr2=0x7f9ef00ae520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:05.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.803+0000 7f9ef5653640 1 --2- 172.21.15.83:0/3229408018 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ef00d22d0 0x7f9ef00ae520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:05.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.803+0000 7f9ef5653640 1 -- 172.21.15.83:0/3229408018 >> v1:172.21.15.167:6789/0 conn(0x7f9ef00b2de0 legacy=0x7f9ef015fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:05.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.803+0000 7f9ef5653640 1 -- 172.21.15.83:0/3229408018 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef00b1760 con 0x7f9ef015f1d0 2024-04-17T10:52:05.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.803+0000 7f9ef5e54640 1 --2- 172.21.15.83:0/3229408018 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ef00d22d0 0x7f9ef00ae520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:52:05.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.807+0000 7f9ef5653640 1 --2- 172.21.15.83:0/3229408018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ef015f1d0 0x7f9ef015f830 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0097c0 tx=0x7f9eec01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c703ee5a8a3045 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:05.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.807+0000 7f9ee7fff640 1 -- 172.21.15.83:0/3229408018 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eec0a7070 con 0x7f9ef015f1d0 2024-04-17T10:52:05.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.807+0000 7f9ee7fff640 1 -- 172.21.15.83:0/3229408018 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eec0b0070 con 0x7f9ef015f1d0 2024-04-17T10:52:05.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.807+0000 7f9ee7fff640 1 -- 172.21.15.83:0/3229408018 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eec0ab890 con 0x7f9ef015f1d0 2024-04-17T10:52:05.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.807+0000 7f9ef78b7640 1 -- 172.21.15.83:0/3229408018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ef015f1d0 msgr2=0x7f9ef015f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:05.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.807+0000 7f9ef78b7640 1 --2- 172.21.15.83:0/3229408018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ef015f1d0 0x7f9ef015f830 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0097c0 tx=0x7f9eec01c470 comp rx=0 tx=0).stop 2024-04-17T10:52:05.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.807+0000 7f9ef78b7640 1 -- 172.21.15.83:0/3229408018 shutdown_connections 2024-04-17T10:52:05.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.807+0000 7f9ef78b7640 1 --2- 172.21.15.83:0/3229408018 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ef00d22d0 0x7f9ef00ae520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:05.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.807+0000 7f9ef78b7640 1 --2- 172.21.15.83:0/3229408018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ef015f1d0 0x7f9ef015f830 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:05.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.807+0000 7f9ef78b7640 1 -- 172.21.15.83:0/3229408018 >> 172.21.15.83:0/3229408018 conn(0x7f9ef00d17b0 msgr2=0x7f9ef00d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:05.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.807+0000 7f9ef78b7640 1 -- 172.21.15.83:0/3229408018 shutdown_connections 2024-04-17T10:52:05.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.807+0000 7f9ef78b7640 1 -- 172.21.15.83:0/3229408018 wait complete. 2024-04-17T10:52:05.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.807+0000 7f9ef78b7640 1 Processor -- start 2024-04-17T10:52:05.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.807+0000 7f9ef78b7640 1 -- start start 2024-04-17T10:52:05.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.807+0000 7f9ef78b7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ef00d22d0 0x7f9ef014a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:05.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.807+0000 7f9ef78b7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef015f1d0 0x7f9ef00c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:05.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.807+0000 7f9ef78b7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ef00d02f0 0x7f9ef00b4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:05.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.807+0000 7f9ef78b7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef00af270 con 0x7f9ef015f1d0 2024-04-17T10:52:05.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.807+0000 7f9ef78b7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef0160690 con 0x7f9ef00d02f0 2024-04-17T10:52:05.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.807+0000 7f9ef78b7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef00af3f0 con 0x7f9ef00d22d0 2024-04-17T10:52:05.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.807+0000 7f9ef4e52640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef015f1d0 0x7f9ef00c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:05.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.807+0000 7f9ef4e52640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef015f1d0 0x7f9ef00c6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43918/0 (socket says 172.21.15.83:43918) 2024-04-17T10:52:05.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.807+0000 7f9ef4e52640 1 -- 172.21.15.83:0/3020396523 learned_addr learned my addr 172.21.15.83:0/3020396523 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:05.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.807+0000 7f9ef5653640 1 --2- 172.21.15.83:0/3020396523 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ef00d22d0 0x7f9ef014a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:05.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.807+0000 7f9ef5e54640 1 --2- 172.21.15.83:0/3020396523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ef00d02f0 0x7f9ef00b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:05.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.807+0000 7f9ef4e52640 1 -- 172.21.15.83:0/3020396523 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ef00d22d0 msgr2=0x7f9ef014a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:05.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.807+0000 7f9ef4e52640 1 --2- 172.21.15.83:0/3020396523 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ef00d22d0 0x7f9ef014a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:05.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.807+0000 7f9ef4e52640 1 -- 172.21.15.83:0/3020396523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ef00d02f0 msgr2=0x7f9ef00b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:05.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.807+0000 7f9ef4e52640 1 --2- 172.21.15.83:0/3020396523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ef00d02f0 0x7f9ef00b4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:05.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.811+0000 7f9ef4e52640 1 -- 172.21.15.83:0/3020396523 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef0153c90 con 0x7f9ef015f1d0 2024-04-17T10:52:05.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.811+0000 7f9ef4e52640 1 --2- 172.21.15.83:0/3020396523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef015f1d0 0x7f9ef00c6d60 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f9ee000c420 tx=0x7f9ee000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:05.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.811+0000 7f9ee5ffb640 1 -- 172.21.15.83:0/3020396523 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee00052d0 con 0x7f9ef015f1d0 2024-04-17T10:52:05.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.811+0000 7f9ee5ffb640 1 -- 172.21.15.83:0/3020396523 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee0005470 con 0x7f9ef015f1d0 2024-04-17T10:52:05.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.811+0000 7f9ef78b7640 1 -- 172.21.15.83:0/3020396523 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ef01533f0 con 0x7f9ef015f1d0 2024-04-17T10:52:05.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.811+0000 7f9ee5ffb640 1 -- 172.21.15.83:0/3020396523 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0007260 con 0x7f9ef015f1d0 2024-04-17T10:52:05.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.811+0000 7f9ef78b7640 1 -- 172.21.15.83:0/3020396523 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ef01539b0 con 0x7f9ef015f1d0 2024-04-17T10:52:05.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.811+0000 7f9ef78b7640 1 -- 172.21.15.83:0/3020396523 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ef0074f90 con 0x7f9ef015f1d0 2024-04-17T10:52:05.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.811+0000 7f9ee5ffb640 1 -- 172.21.15.83:0/3020396523 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9ee0007400 con 0x7f9ef015f1d0 2024-04-17T10:52:05.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.811+0000 7f9ee5ffb640 1 --2- 172.21.15.83:0/3020396523 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9ed4066f90 0x7f9ed4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:05.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.811+0000 7f9ee5ffb640 1 -- 172.21.15.83:0/3020396523 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f9ee00843b0 con 0x7f9ef015f1d0 2024-04-17T10:52:05.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.811+0000 7f9ef5653640 1 --2- 172.21.15.83:0/3020396523 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9ed4066f90 0x7f9ed4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:05.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.811+0000 7f9ef5653640 1 --2- 172.21.15.83:0/3020396523 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9ed4066f90 0x7f9ed4069450 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0af000 tx=0x7f9eec0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:05.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:05.815+0000 7f9ee5ffb640 1 -- 172.21.15.83:0/3020396523 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9ee0054d90 con 0x7f9ef015f1d0 2024-04-17T10:52:06.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:06.011+0000 7f9ef78b7640 1 -- 172.21.15.83:0/3020396523 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ef0072690 con 0x7f9ef015f1d0 2024-04-17T10:52:06.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:06.011+0000 7f9ee5ffb640 1 -- 172.21.15.83:0/3020396523 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f9ee0058e30 con 0x7f9ef015f1d0 2024-04-17T10:52:06.004 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:06.004 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:06.004 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:06.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:06.011+0000 7f9ef78b7640 1 -- 172.21.15.83:0/3020396523 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9ed4066f90 msgr2=0x7f9ed4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:06.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:06.011+0000 7f9ef78b7640 1 --2- 172.21.15.83:0/3020396523 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9ed4066f90 0x7f9ed4069450 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0af000 tx=0x7f9eec0a7000 comp rx=0 tx=0).stop 2024-04-17T10:52:06.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:06.011+0000 7f9ef78b7640 1 -- 172.21.15.83:0/3020396523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef015f1d0 msgr2=0x7f9ef00c6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:06.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:06.011+0000 7f9ef78b7640 1 --2- 172.21.15.83:0/3020396523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef015f1d0 0x7f9ef00c6d60 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f9ee000c420 tx=0x7f9ee000c8e0 comp rx=0 tx=0).stop 2024-04-17T10:52:06.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:06.011+0000 7f9ef78b7640 1 -- 172.21.15.83:0/3020396523 shutdown_connections 2024-04-17T10:52:06.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:06.011+0000 7f9ef78b7640 1 --2- 172.21.15.83:0/3020396523 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9ed4066f90 0x7f9ed4069450 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:06.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:06.011+0000 7f9ef78b7640 1 --2- 172.21.15.83:0/3020396523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ef00d02f0 0x7f9ef00b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:06.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:06.011+0000 7f9ef78b7640 1 --2- 172.21.15.83:0/3020396523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef015f1d0 0x7f9ef00c6d60 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:06.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:06.011+0000 7f9ef78b7640 1 --2- 172.21.15.83:0/3020396523 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ef00d22d0 0x7f9ef014a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:06.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:06.011+0000 7f9ef78b7640 1 -- 172.21.15.83:0/3020396523 >> 172.21.15.83:0/3020396523 conn(0x7f9ef00d17b0 msgr2=0x7f9ef00cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:06.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:06.011+0000 7f9ef78b7640 1 -- 172.21.15.83:0/3020396523 shutdown_connections 2024-04-17T10:52:06.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:06.011+0000 7f9ef78b7640 1 -- 172.21.15.83:0/3020396523 wait complete. 2024-04-17T10:52:06.426 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:06.426 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:07.428 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:07.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.567+0000 7f1c7d846640 1 Processor -- start 2024-04-17T10:52:07.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.567+0000 7f1c7d846640 1 -- start start 2024-04-17T10:52:07.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.567+0000 7f1c7d846640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1c7815db50 0x7f1c78070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:07.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.567+0000 7f1c7d846640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1c780aedd0 0x7f1c7806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:07.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.567+0000 7f1c7d846640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c7806dae0 0x7f1c780cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:07.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.567+0000 7f1c7d846640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c780709b0 con 0x7f1c780aedd0 2024-04-17T10:52:07.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.567+0000 7f1c7d846640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c7815e440 con 0x7f1c7815db50 2024-04-17T10:52:07.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.567+0000 7f1c7d846640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7815e720 con 0x7f1c7806dae0 2024-04-17T10:52:07.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.567+0000 7f1c767fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1c780aedd0 0x7f1c7806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:07.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.567+0000 7f1c767fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1c780aedd0 0x7f1c7806d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47302/0 (socket says 172.21.15.83:47302) 2024-04-17T10:52:07.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.567+0000 7f1c767fc640 1 -- 172.21.15.83:0/4078926231 learned_addr learned my addr 172.21.15.83:0/4078926231 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:07.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.567+0000 7f1c777fe640 1 --2- 172.21.15.83:0/4078926231 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c7806dae0 0x7f1c780cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:07.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.567+0000 7f1c767fc640 1 -- 172.21.15.83:0/4078926231 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c7806dae0 msgr2=0x7f1c780cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:07.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.567+0000 7f1c76ffd640 1 --2- 172.21.15.83:0/4078926231 >> v2:172.21.15.167:3300/0 conn(0x7f1c7815db50 0x7f1c78070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:07.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.567+0000 7f1c767fc640 1 --2- 172.21.15.83:0/4078926231 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c7806dae0 0x7f1c780cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:07.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.567+0000 7f1c767fc640 1 -- 172.21.15.83:0/4078926231 >> v2:172.21.15.167:3300/0 conn(0x7f1c7815db50 msgr2=0x7f1c78070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:07.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.567+0000 7f1c767fc640 1 --2- 172.21.15.83:0/4078926231 >> v2:172.21.15.167:3300/0 conn(0x7f1c7815db50 0x7f1c78070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:07.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.567+0000 7f1c767fc640 1 -- 172.21.15.83:0/4078926231 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7815ea00 con 0x7f1c780aedd0 2024-04-17T10:52:07.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.567+0000 7f1c777fe640 1 --2- 172.21.15.83:0/4078926231 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c7806dae0 0x7f1c780cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:52:07.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.571+0000 7f1c767fc640 1 --2- 172.21.15.83:0/4078926231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c780aedd0 0x7f1c7806d5a0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f1c7806e960 tx=0x7f1c6c01e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab334d7ebd82b7a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:07.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.571+0000 7f1c75ffb640 1 -- 172.21.15.83:0/4078926231 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c0b4070 con 0x7f1c780aedd0 2024-04-17T10:52:07.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.571+0000 7f1c75ffb640 1 -- 172.21.15.83:0/4078926231 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6c0a78b0 con 0x7f1c780aedd0 2024-04-17T10:52:07.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.571+0000 7f1c75ffb640 1 -- 172.21.15.83:0/4078926231 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c0a7c80 con 0x7f1c780aedd0 2024-04-17T10:52:07.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.571+0000 7f1c7d846640 1 -- 172.21.15.83:0/4078926231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c780aedd0 msgr2=0x7f1c7806d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:07.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.571+0000 7f1c7d846640 1 --2- 172.21.15.83:0/4078926231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c780aedd0 0x7f1c7806d5a0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f1c7806e960 tx=0x7f1c6c01e930 comp rx=0 tx=0).stop 2024-04-17T10:52:07.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.571+0000 7f1c7d846640 1 -- 172.21.15.83:0/4078926231 shutdown_connections 2024-04-17T10:52:07.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.571+0000 7f1c7d846640 1 --2- 172.21.15.83:0/4078926231 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c7806dae0 0x7f1c780cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:07.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.571+0000 7f1c7d846640 1 --2- 172.21.15.83:0/4078926231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c780aedd0 0x7f1c7806d5a0 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:07.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.571+0000 7f1c7d846640 1 --2- 172.21.15.83:0/4078926231 >> v2:172.21.15.167:3300/0 conn(0x7f1c7815db50 0x7f1c78070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:07.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.571+0000 7f1c7d846640 1 -- 172.21.15.83:0/4078926231 >> 172.21.15.83:0/4078926231 conn(0x7f1c7806abe0 msgr2=0x7f1c7806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:07.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.571+0000 7f1c7d846640 1 -- 172.21.15.83:0/4078926231 shutdown_connections 2024-04-17T10:52:07.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.571+0000 7f1c7d846640 1 -- 172.21.15.83:0/4078926231 wait complete. 2024-04-17T10:52:07.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.571+0000 7f1c7d846640 1 Processor -- start 2024-04-17T10:52:07.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.571+0000 7f1c7d846640 1 -- start start 2024-04-17T10:52:07.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.571+0000 7f1c7d846640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c7806dae0 0x7f1c78152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:07.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.571+0000 7f1c7d846640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c780aedd0 0x7f1c78150b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:07.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.571+0000 7f1c7d846640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c7815db50 0x7f1c780cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:07.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.571+0000 7f1c7d846640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7806c610 con 0x7f1c780aedd0 2024-04-17T10:52:07.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.571+0000 7f1c7d846640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7806c490 con 0x7f1c7815db50 2024-04-17T10:52:07.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.571+0000 7f1c7d846640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7806c790 con 0x7f1c7806dae0 2024-04-17T10:52:07.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.571+0000 7f1c767fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c780aedd0 0x7f1c78150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:07.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.571+0000 7f1c767fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c780aedd0 0x7f1c78150b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47314/0 (socket says 172.21.15.83:47314) 2024-04-17T10:52:07.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.575+0000 7f1c767fc640 1 -- 172.21.15.83:0/2017958080 learned_addr learned my addr 172.21.15.83:0/2017958080 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:07.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.575+0000 7f1c76ffd640 1 --2- 172.21.15.83:0/2017958080 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c7806dae0 0x7f1c78152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:07.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.575+0000 7f1c777fe640 1 --2- 172.21.15.83:0/2017958080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c7815db50 0x7f1c780cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:07.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.575+0000 7f1c767fc640 1 -- 172.21.15.83:0/2017958080 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c7806dae0 msgr2=0x7f1c78152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:07.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.575+0000 7f1c767fc640 1 --2- 172.21.15.83:0/2017958080 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c7806dae0 0x7f1c78152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:07.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.575+0000 7f1c767fc640 1 -- 172.21.15.83:0/2017958080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c7815db50 msgr2=0x7f1c780cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.575+0000 7f1c767fc640 1 --2- 172.21.15.83:0/2017958080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c7815db50 0x7f1c780cc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.575+0000 7f1c767fc640 1 -- 172.21.15.83:0/2017958080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c78158f40 con 0x7f1c780aedd0 2024-04-17T10:52:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.575+0000 7f1c777fe640 1 --2- 172.21.15.83:0/2017958080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c7815db50 0x7f1c780cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:52:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.575+0000 7f1c767fc640 1 --2- 172.21.15.83:0/2017958080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c780aedd0 0x7f1c78150b10 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00b2c0 tx=0x7f1c6c01b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.575+0000 7f1c5ffff640 1 -- 172.21.15.83:0/2017958080 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c0bf070 con 0x7f1c780aedd0 2024-04-17T10:52:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.575+0000 7f1c5ffff640 1 -- 172.21.15.83:0/2017958080 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6c0b6050 con 0x7f1c780aedd0 2024-04-17T10:52:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.575+0000 7f1c7d846640 1 -- 172.21.15.83:0/2017958080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c781591e0 con 0x7f1c780aedd0 2024-04-17T10:52:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.575+0000 7f1c7d846640 1 -- 172.21.15.83:0/2017958080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c78153330 con 0x7f1c780aedd0 2024-04-17T10:52:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.575+0000 7f1c5ffff640 1 -- 172.21.15.83:0/2017958080 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c0b4030 con 0x7f1c780aedd0 2024-04-17T10:52:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.575+0000 7f1c7d846640 1 -- 172.21.15.83:0/2017958080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c580052a0 con 0x7f1c780aedd0 2024-04-17T10:52:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.575+0000 7f1c5ffff640 1 -- 172.21.15.83:0/2017958080 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1c6c0c2070 con 0x7f1c780aedd0 2024-04-17T10:52:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.575+0000 7f1c5ffff640 1 --2- 172.21.15.83:0/2017958080 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1c64066f60 0x7f1c64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.575+0000 7f1c5ffff640 1 -- 172.21.15.83:0/2017958080 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f1c6c11ed90 con 0x7f1c780aedd0 2024-04-17T10:52:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.575+0000 7f1c76ffd640 1 --2- 172.21.15.83:0/2017958080 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1c64066f60 0x7f1c64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:07.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.575+0000 7f1c76ffd640 1 --2- 172.21.15.83:0/2017958080 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1c64066f60 0x7f1c64069420 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1c6800a8d0 tx=0x7f1c68008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:07.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.579+0000 7f1c5ffff640 1 -- 172.21.15.83:0/2017958080 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1c6c0187e0 con 0x7f1c780aedd0 2024-04-17T10:52:07.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.771+0000 7f1c7d846640 1 -- 172.21.15.83:0/2017958080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c58005860 con 0x7f1c780aedd0 2024-04-17T10:52:07.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.771+0000 7f1c5ffff640 1 -- 172.21.15.83:0/2017958080 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f1c6c0a7880 con 0x7f1c780aedd0 2024-04-17T10:52:07.766 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:07.766 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:07.767 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:07.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.775+0000 7f1c7d846640 1 -- 172.21.15.83:0/2017958080 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1c64066f60 msgr2=0x7f1c64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:07.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.775+0000 7f1c7d846640 1 --2- 172.21.15.83:0/2017958080 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1c64066f60 0x7f1c64069420 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1c6800a8d0 tx=0x7f1c68008040 comp rx=0 tx=0).stop 2024-04-17T10:52:07.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.775+0000 7f1c7d846640 1 -- 172.21.15.83:0/2017958080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c780aedd0 msgr2=0x7f1c78150b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:07.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.775+0000 7f1c7d846640 1 --2- 172.21.15.83:0/2017958080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c780aedd0 0x7f1c78150b10 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00b2c0 tx=0x7f1c6c01b6d0 comp rx=0 tx=0).stop 2024-04-17T10:52:07.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.775+0000 7f1c7d846640 1 -- 172.21.15.83:0/2017958080 shutdown_connections 2024-04-17T10:52:07.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.775+0000 7f1c7d846640 1 --2- 172.21.15.83:0/2017958080 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1c64066f60 0x7f1c64069420 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:07.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.775+0000 7f1c7d846640 1 --2- 172.21.15.83:0/2017958080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c7815db50 0x7f1c780cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:07.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.775+0000 7f1c7d846640 1 --2- 172.21.15.83:0/2017958080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c780aedd0 0x7f1c78150b10 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:07.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.775+0000 7f1c7d846640 1 --2- 172.21.15.83:0/2017958080 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c7806dae0 0x7f1c78152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:07.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.775+0000 7f1c7d846640 1 -- 172.21.15.83:0/2017958080 >> 172.21.15.83:0/2017958080 conn(0x7f1c7806abe0 msgr2=0x7f1c780b1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:07.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.775+0000 7f1c7d846640 1 -- 172.21.15.83:0/2017958080 shutdown_connections 2024-04-17T10:52:07.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:07.775+0000 7f1c7d846640 1 -- 172.21.15.83:0/2017958080 wait complete. 2024-04-17T10:52:08.189 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:08.189 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:09.119 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:52:09.122 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:52:09.190 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:09.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.319+0000 7ff379101640 1 Processor -- start 2024-04-17T10:52:09.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.319+0000 7ff379101640 1 -- start start 2024-04-17T10:52:09.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.323+0000 7ff379101640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff37415dac0 0x7ff37406e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:09.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.323+0000 7ff379101640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff37406eb20 0x7ff374161710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:09.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.323+0000 7ff379101640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff37406bd90 con 0x7ff374161c50 2024-04-17T10:52:09.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.323+0000 7ff379101640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff37406c9a0 con 0x7ff37415dac0 2024-04-17T10:52:09.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.323+0000 7ff379101640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff37406c070 con 0x7ff37406eb20 2024-04-17T10:52:09.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.323+0000 7ff372d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff37415dac0 0x7ff37406e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:09.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.323+0000 7ff37359e640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff374161c50 0x7ff37406ce30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58946/0 (socket says 172.21.15.83:58946) 2024-04-17T10:52:09.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.323+0000 7ff37359e640 1 -- 172.21.15.83:0/843568880 learned_addr learned my addr 172.21.15.83:0/843568880 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:52:09.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.323+0000 7ff372d9d640 1 -- 172.21.15.83:0/843568880 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff37406eb20 msgr2=0x7ff374161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:09.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.323+0000 7ff372d9d640 1 --2- 172.21.15.83:0/843568880 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff37406eb20 0x7ff374161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:09.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.323+0000 7ff372d9d640 1 -- 172.21.15.83:0/843568880 >> v1:172.21.15.83:6789/0 conn(0x7ff374161c50 legacy=0x7ff37406ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:09.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.323+0000 7ff372d9d640 1 -- 172.21.15.83:0/843568880 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff37406c350 con 0x7ff37415dac0 2024-04-17T10:52:09.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.323+0000 7ff371d9b640 1 -- 172.21.15.83:0/843568880 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1047834512 0 0) 0x7ff37406bd90 con 0x7ff374161c50 2024-04-17T10:52:09.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.323+0000 7ff372d9d640 1 --2- 172.21.15.83:0/843568880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff37415dac0 0x7ff37406e5e0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7ff36c0097c0 tx=0x7ff36c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=128662899f452ce6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:09.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.323+0000 7ff371d9b640 1 -- 172.21.15.83:0/843568880 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c0b1070 con 0x7ff37415dac0 2024-04-17T10:52:09.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.323+0000 7ff371d9b640 1 -- 172.21.15.83:0/843568880 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff37406dd50 con 0x7ff37415dac0 2024-04-17T10:52:09.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.323+0000 7ff371d9b640 1 -- 172.21.15.83:0/843568880 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c0ac630 con 0x7ff37415dac0 2024-04-17T10:52:09.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff379101640 1 -- 172.21.15.83:0/843568880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff37415dac0 msgr2=0x7ff37406e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:09.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff379101640 1 --2- 172.21.15.83:0/843568880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff37415dac0 0x7ff37406e5e0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7ff36c0097c0 tx=0x7ff36c01c4a0 comp rx=0 tx=0).stop 2024-04-17T10:52:09.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff379101640 1 -- 172.21.15.83:0/843568880 shutdown_connections 2024-04-17T10:52:09.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff379101640 1 --2- 172.21.15.83:0/843568880 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff37406eb20 0x7ff374161710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:09.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff379101640 1 --2- 172.21.15.83:0/843568880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff37415dac0 0x7ff37406e5e0 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:09.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff379101640 1 -- 172.21.15.83:0/843568880 >> 172.21.15.83:0/843568880 conn(0x7ff3740b3220 msgr2=0x7ff3740b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:09.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff379101640 1 -- 172.21.15.83:0/843568880 shutdown_connections 2024-04-17T10:52:09.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff379101640 1 -- 172.21.15.83:0/843568880 wait complete. 2024-04-17T10:52:09.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff379101640 1 Processor -- start 2024-04-17T10:52:09.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff379101640 1 -- start start 2024-04-17T10:52:09.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff379101640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff37406eb20 0x7ff3740b0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:09.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff379101640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff37415dac0 0x7ff3740af840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:09.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff379101640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff374161c50 0x7ff37414e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:09.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff379101640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff374162510 con 0x7ff374161c50 2024-04-17T10:52:09.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff379101640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff374162390 con 0x7ff37406eb20 2024-04-17T10:52:09.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff379101640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff37416aab0 con 0x7ff37415dac0 2024-04-17T10:52:09.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff37359e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff374161c50 0x7ff37414e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:09.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff37359e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff374161c50 0x7ff37414e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47336/0 (socket says 172.21.15.83:47336) 2024-04-17T10:52:09.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff37359e640 1 -- 172.21.15.83:0/1739569348 learned_addr learned my addr 172.21.15.83:0/1739569348 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:09.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff37259c640 1 --2- 172.21.15.83:0/1739569348 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff37415dac0 0x7ff3740af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:09.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff372d9d640 1 --2- 172.21.15.83:0/1739569348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff37406eb20 0x7ff3740b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:09.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff37359e640 1 -- 172.21.15.83:0/1739569348 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff37415dac0 msgr2=0x7ff3740af840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:09.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff37359e640 1 --2- 172.21.15.83:0/1739569348 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff37415dac0 0x7ff3740af840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:09.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff37359e640 1 -- 172.21.15.83:0/1739569348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff37406eb20 msgr2=0x7ff3740b0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:09.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff37359e640 1 --2- 172.21.15.83:0/1739569348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff37406eb20 0x7ff3740b0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:09.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff37359e640 1 -- 172.21.15.83:0/1739569348 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff37414f1b0 con 0x7ff374161c50 2024-04-17T10:52:09.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff37259c640 1 --2- 172.21.15.83:0/1739569348 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff37415dac0 0x7ff3740af840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:52:09.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff37359e640 1 --2- 172.21.15.83:0/1739569348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff374161c50 0x7ff37414e250 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7ff36400c780 tx=0x7ff36400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:09.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff36b7fe640 1 -- 172.21.15.83:0/1739569348 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff364018070 con 0x7ff374161c50 2024-04-17T10:52:09.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff36b7fe640 1 -- 172.21.15.83:0/1739569348 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff36400a640 con 0x7ff374161c50 2024-04-17T10:52:09.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff379101640 1 -- 172.21.15.83:0/1739569348 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff37414fb10 con 0x7ff374161c50 2024-04-17T10:52:09.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff36b7fe640 1 -- 172.21.15.83:0/1739569348 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36400aa30 con 0x7ff374161c50 2024-04-17T10:52:09.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.327+0000 7ff379101640 1 -- 172.21.15.83:0/1739569348 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3741503b0 con 0x7ff374161c50 2024-04-17T10:52:09.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.331+0000 7ff379101640 1 -- 172.21.15.83:0/1739569348 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff374074f90 con 0x7ff374161c50 2024-04-17T10:52:09.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.331+0000 7ff36b7fe640 1 -- 172.21.15.83:0/1739569348 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff36400abd0 con 0x7ff374161c50 2024-04-17T10:52:09.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.331+0000 7ff36b7fe640 1 --2- 172.21.15.83:0/1739569348 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff354066f60 0x7ff354069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:09.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.331+0000 7ff36b7fe640 1 -- 172.21.15.83:0/1739569348 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7ff3640846e0 con 0x7ff374161c50 2024-04-17T10:52:09.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.331+0000 7ff372d9d640 1 --2- 172.21.15.83:0/1739569348 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff354066f60 0x7ff354069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:09.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.331+0000 7ff372d9d640 1 --2- 172.21.15.83:0/1739569348 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff354066f60 0x7ff354069420 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ff36c0ae000 tx=0x7ff36c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:09.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.335+0000 7ff36b7fe640 1 -- 172.21.15.83:0/1739569348 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff364055570 con 0x7ff374161c50 2024-04-17T10:52:09.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.515+0000 7ff379101640 1 -- 172.21.15.83:0/1739569348 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3740730a0 con 0x7ff374161c50 2024-04-17T10:52:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.515+0000 7ff36b7fe640 1 -- 172.21.15.83:0/1739569348 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7ff364054e70 con 0x7ff374161c50 2024-04-17T10:52:09.510 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:09.511 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:09.511 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:09.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.519+0000 7ff379101640 1 -- 172.21.15.83:0/1739569348 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff354066f60 msgr2=0x7ff354069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:09.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.519+0000 7ff379101640 1 --2- 172.21.15.83:0/1739569348 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff354066f60 0x7ff354069420 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ff36c0ae000 tx=0x7ff36c008040 comp rx=0 tx=0).stop 2024-04-17T10:52:09.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.519+0000 7ff379101640 1 -- 172.21.15.83:0/1739569348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff374161c50 msgr2=0x7ff37414e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:09.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.519+0000 7ff379101640 1 --2- 172.21.15.83:0/1739569348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff374161c50 0x7ff37414e250 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7ff36400c780 tx=0x7ff36400cc40 comp rx=0 tx=0).stop 2024-04-17T10:52:09.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.519+0000 7ff379101640 1 -- 172.21.15.83:0/1739569348 shutdown_connections 2024-04-17T10:52:09.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.519+0000 7ff379101640 1 --2- 172.21.15.83:0/1739569348 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff354066f60 0x7ff354069420 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:09.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.519+0000 7ff379101640 1 --2- 172.21.15.83:0/1739569348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff374161c50 0x7ff37414e250 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:09.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.519+0000 7ff379101640 1 --2- 172.21.15.83:0/1739569348 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff37415dac0 0x7ff3740af840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:09.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.519+0000 7ff379101640 1 --2- 172.21.15.83:0/1739569348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff37406eb20 0x7ff3740b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:09.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.519+0000 7ff379101640 1 -- 172.21.15.83:0/1739569348 >> 172.21.15.83:0/1739569348 conn(0x7ff3740b3220 msgr2=0x7ff37406ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:09.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.519+0000 7ff379101640 1 -- 172.21.15.83:0/1739569348 shutdown_connections 2024-04-17T10:52:09.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:09.519+0000 7ff379101640 1 -- 172.21.15.83:0/1739569348 wait complete. 2024-04-17T10:52:09.931 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:09.931 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:10.932 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:11.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.067+0000 7f6303506640 1 Processor -- start 2024-04-17T10:52:11.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.067+0000 7f6303506640 1 -- start start 2024-04-17T10:52:11.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f6303506640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f62fc15ba30 0x7f62fc072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:11.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f6303506640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62fc15c960 0x7f62fc0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:11.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f6303506640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62fc0cfea0 con 0x7f62fc0cfa10 2024-04-17T10:52:11.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f6303506640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62fc071ea0 con 0x7f62fc15ba30 2024-04-17T10:52:11.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f6303506640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62fc06e320 con 0x7f62fc15c960 2024-04-17T10:52:11.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f6301aa3640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f62fc0cfa10 0x7f62fc06c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58958/0 (socket says 172.21.15.83:58958) 2024-04-17T10:52:11.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f6301aa3640 1 -- 172.21.15.83:0/830654148 learned_addr learned my addr 172.21.15.83:0/830654148 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:52:11.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f63012a2640 1 --2- 172.21.15.83:0/830654148 >> v2:172.21.15.167:3300/0 conn(0x7f62fc15ba30 0x7f62fc072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:11.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f6300aa1640 1 --2- 172.21.15.83:0/830654148 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62fc15c960 0x7f62fc0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:11.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f62f3fff640 1 -- 172.21.15.83:0/830654148 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4078205977 0 0) 0x7f62fc0cfea0 con 0x7f62fc0cfa10 2024-04-17T10:52:11.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f62f3fff640 1 -- 172.21.15.83:0/830654148 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62e8003530 con 0x7f62fc0cfa10 2024-04-17T10:52:11.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f62f3fff640 1 -- 172.21.15.83:0/830654148 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3952559451 0 0) 0x7f62e8003530 con 0x7f62fc0cfa10 2024-04-17T10:52:11.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f62f3fff640 1 -- 172.21.15.83:0/830654148 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62fc15c960 msgr2=0x7f62fc0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:11.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f62f3fff640 1 --2- 172.21.15.83:0/830654148 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62fc15c960 0x7f62fc0cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:11.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f62f3fff640 1 -- 172.21.15.83:0/830654148 >> v2:172.21.15.167:3300/0 conn(0x7f62fc15ba30 msgr2=0x7f62fc072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:11.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f62f3fff640 1 --2- 172.21.15.83:0/830654148 >> v2:172.21.15.167:3300/0 conn(0x7f62fc15ba30 0x7f62fc072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:11.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f62f3fff640 1 -- 172.21.15.83:0/830654148 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62fc06e040 con 0x7f62fc0cfa10 2024-04-17T10:52:11.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f62f3fff640 1 -- 172.21.15.83:0/830654148 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f62f4001680 con 0x7f62fc0cfa10 2024-04-17T10:52:11.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f62f3fff640 1 -- 172.21.15.83:0/830654148 >> v1:172.21.15.83:6789/0 conn(0x7f62fc0cfa10 legacy=0x7f62fc06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:11.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f62f3fff640 1 --2- 172.21.15.83:0/830654148 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62fc15c960 0x7f62e80039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:11.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f62f3fff640 1 --2- 172.21.15.83:0/830654148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62e8003f40 0x7f62e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:11.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f62f3fff640 1 --2- 172.21.15.83:0/830654148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62e8004d50 0x7f62e80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:11.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f62f3fff640 1 -- 172.21.15.83:0/830654148 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62fc071ea0 con 0x7f62e8003f40 2024-04-17T10:52:11.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f62f3fff640 1 -- 172.21.15.83:0/830654148 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62fc06e320 con 0x7f62e8004d50 2024-04-17T10:52:11.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f62f3fff640 1 -- 172.21.15.83:0/830654148 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e800fe30 con 0x7f62fc15c960 2024-04-17T10:52:11.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f6300aa1640 1 --2- 172.21.15.83:0/830654148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62e8003f40 0x7f62e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:11.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f63012a2640 1 --2- 172.21.15.83:0/830654148 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62fc15c960 0x7f62e80039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:11.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f6301aa3640 1 --2- 172.21.15.83:0/830654148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62e8004d50 0x7f62e80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:11.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f6300aa1640 1 -- 172.21.15.83:0/830654148 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62fc15c960 msgr2=0x7f62e80039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:11.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f6300aa1640 1 --2- 172.21.15.83:0/830654148 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62fc15c960 0x7f62e80039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:11.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f6300aa1640 1 -- 172.21.15.83:0/830654148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62e8004d50 msgr2=0x7f62e80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:11.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f6300aa1640 1 --2- 172.21.15.83:0/830654148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62e8004d50 0x7f62e80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:11.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.071+0000 7f6300aa1640 1 -- 172.21.15.83:0/830654148 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e8003530 con 0x7f62e8003f40 2024-04-17T10:52:11.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.075+0000 7f6300aa1640 1 --2- 172.21.15.83:0/830654148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62e8003f40 0x7f62e8004810 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f62f800e170 tx=0x7f62fc06d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=644a2425edef5274 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:11.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.075+0000 7f62f3fff640 1 -- 172.21.15.83:0/830654148 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f8009c10 con 0x7f62e8003f40 2024-04-17T10:52:11.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.075+0000 7f62f3fff640 1 -- 172.21.15.83:0/830654148 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f80a5a50 con 0x7f62e8003f40 2024-04-17T10:52:11.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.075+0000 7f62f3fff640 1 -- 172.21.15.83:0/830654148 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f80a5e40 con 0x7f62e8003f40 2024-04-17T10:52:11.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.075+0000 7f6303506640 1 -- 172.21.15.83:0/830654148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62e8003f40 msgr2=0x7f62e8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:11.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.075+0000 7f6303506640 1 --2- 172.21.15.83:0/830654148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62e8003f40 0x7f62e8004810 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f62f800e170 tx=0x7f62fc06d3b0 comp rx=0 tx=0).stop 2024-04-17T10:52:11.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.075+0000 7f6301aa3640 1 -- 172.21.15.83:0/830654148 reap_dead start 2024-04-17T10:52:11.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.075+0000 7f6303506640 1 -- 172.21.15.83:0/830654148 shutdown_connections 2024-04-17T10:52:11.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.075+0000 7f6303506640 1 -- 172.21.15.83:0/830654148 >> 172.21.15.83:0/830654148 conn(0x7f62fc06abe0 msgr2=0x7f62fc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:11.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.075+0000 7f6303506640 1 -- 172.21.15.83:0/830654148 shutdown_connections 2024-04-17T10:52:11.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.075+0000 7f6303506640 1 -- 172.21.15.83:0/830654148 wait complete. 2024-04-17T10:52:11.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.075+0000 7f6303506640 1 Processor -- start 2024-04-17T10:52:11.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.075+0000 7f6303506640 1 -- start start 2024-04-17T10:52:11.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.075+0000 7f6303506640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62e8003f40 0x7f62fc159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:11.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.075+0000 7f6303506640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62fc0ce750 0x7f62fc15a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:11.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.075+0000 7f6303506640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62fc0cebb0 0x7f62fc15ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:11.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.075+0000 7f6303506640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62fc06de20 con 0x7f62e8003f40 2024-04-17T10:52:11.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.075+0000 7f6303506640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62fc06dca0 con 0x7f62fc0ce750 2024-04-17T10:52:11.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.075+0000 7f6303506640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62fc15cd40 con 0x7f62fc0cebb0 2024-04-17T10:52:11.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.075+0000 7f63012a2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62e8003f40 0x7f62fc159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:11.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.075+0000 7f63012a2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62e8003f40 0x7f62fc159a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47350/0 (socket says 172.21.15.83:47350) 2024-04-17T10:52:11.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.075+0000 7f63012a2640 1 -- 172.21.15.83:0/3336558887 learned_addr learned my addr 172.21.15.83:0/3336558887 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:11.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.079+0000 7f63012a2640 1 -- 172.21.15.83:0/3336558887 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62fc0cebb0 msgr2=0x7f62fc15ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:11.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.079+0000 7f6301aa3640 1 --2- 172.21.15.83:0/3336558887 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62fc0cebb0 0x7f62fc15ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:11.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.079+0000 7f6300aa1640 1 --2- 172.21.15.83:0/3336558887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62fc0ce750 0x7f62fc15a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:11.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.079+0000 7f63012a2640 1 --2- 172.21.15.83:0/3336558887 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62fc0cebb0 0x7f62fc15ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:11.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.079+0000 7f63012a2640 1 -- 172.21.15.83:0/3336558887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62fc0ce750 msgr2=0x7f62fc15a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:11.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.079+0000 7f63012a2640 1 --2- 172.21.15.83:0/3336558887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62fc0ce750 0x7f62fc15a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:11.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.079+0000 7f63012a2640 1 -- 172.21.15.83:0/3336558887 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62fc1589a0 con 0x7f62e8003f40 2024-04-17T10:52:11.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.079+0000 7f6300aa1640 1 --2- 172.21.15.83:0/3336558887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62fc0ce750 0x7f62fc15a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:52:11.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.079+0000 7f63012a2640 1 --2- 172.21.15.83:0/3336558887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62e8003f40 0x7f62fc159a40 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f62ec011660 tx=0x7f62ec011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:11.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.079+0000 7f62f1ffb640 1 -- 172.21.15.83:0/3336558887 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec00f9c0 con 0x7f62e8003f40 2024-04-17T10:52:11.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.079+0000 7f62f1ffb640 1 -- 172.21.15.83:0/3336558887 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ec00fb60 con 0x7f62e8003f40 2024-04-17T10:52:11.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.079+0000 7f6303506640 1 -- 172.21.15.83:0/3336558887 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62fc158c80 con 0x7f62e8003f40 2024-04-17T10:52:11.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.079+0000 7f62f1ffb640 1 -- 172.21.15.83:0/3336558887 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec015460 con 0x7f62e8003f40 2024-04-17T10:52:11.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.079+0000 7f6303506640 1 -- 172.21.15.83:0/3336558887 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62fc159240 con 0x7f62e8003f40 2024-04-17T10:52:11.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.079+0000 7f6303506640 1 -- 172.21.15.83:0/3336558887 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62fc074f90 con 0x7f62e8003f40 2024-04-17T10:52:11.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.079+0000 7f62f1ffb640 1 -- 172.21.15.83:0/3336558887 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f62ec015600 con 0x7f62e8003f40 2024-04-17T10:52:11.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.079+0000 7f62f1ffb640 1 --2- 172.21.15.83:0/3336558887 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f62e0066f60 0x7f62e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:11.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.079+0000 7f6300aa1640 1 --2- 172.21.15.83:0/3336558887 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f62e0066f60 0x7f62e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:11.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.079+0000 7f62f1ffb640 1 -- 172.21.15.83:0/3336558887 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f62ec085b50 con 0x7f62e8003f40 2024-04-17T10:52:11.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.079+0000 7f6300aa1640 1 --2- 172.21.15.83:0/3336558887 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f62e0066f60 0x7f62e0069420 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f62f80ae000 tx=0x7f62f801cc60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:11.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.083+0000 7f62f1ffb640 1 -- 172.21.15.83:0/3336558887 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f62ec019030 con 0x7f62e8003f40 2024-04-17T10:52:11.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.259+0000 7f6303506640 1 -- 172.21.15.83:0/3336558887 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62fc06b1d0 con 0x7f62e8003f40 2024-04-17T10:52:11.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.259+0000 7f62f1ffb640 1 -- 172.21.15.83:0/3336558887 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f62ec056c80 con 0x7f62e8003f40 2024-04-17T10:52:11.255 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:11.255 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:11.255 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:11.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.263+0000 7f6303506640 1 -- 172.21.15.83:0/3336558887 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f62e0066f60 msgr2=0x7f62e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:11.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.263+0000 7f6303506640 1 --2- 172.21.15.83:0/3336558887 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f62e0066f60 0x7f62e0069420 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f62f80ae000 tx=0x7f62f801cc60 comp rx=0 tx=0).stop 2024-04-17T10:52:11.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.263+0000 7f6303506640 1 -- 172.21.15.83:0/3336558887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62e8003f40 msgr2=0x7f62fc159a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:11.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.263+0000 7f6303506640 1 --2- 172.21.15.83:0/3336558887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62e8003f40 0x7f62fc159a40 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f62ec011660 tx=0x7f62ec011b20 comp rx=0 tx=0).stop 2024-04-17T10:52:11.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.263+0000 7f6303506640 1 -- 172.21.15.83:0/3336558887 shutdown_connections 2024-04-17T10:52:11.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.263+0000 7f6303506640 1 --2- 172.21.15.83:0/3336558887 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f62e0066f60 0x7f62e0069420 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:11.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.263+0000 7f6303506640 1 --2- 172.21.15.83:0/3336558887 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62fc0cebb0 0x7f62fc15ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:11.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.263+0000 7f6303506640 1 --2- 172.21.15.83:0/3336558887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62fc0ce750 0x7f62fc15a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:11.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.263+0000 7f6303506640 1 --2- 172.21.15.83:0/3336558887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62e8003f40 0x7f62fc159a40 unknown :-1 s=CLOSED pgs=2199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:11.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.263+0000 7f6303506640 1 -- 172.21.15.83:0/3336558887 >> 172.21.15.83:0/3336558887 conn(0x7f62fc06abe0 msgr2=0x7f62fc0afcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:11.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.263+0000 7f6303506640 1 -- 172.21.15.83:0/3336558887 shutdown_connections 2024-04-17T10:52:11.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:11.263+0000 7f6303506640 1 -- 172.21.15.83:0/3336558887 wait complete. 2024-04-17T10:52:11.675 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:11.676 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:12.677 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:12.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.815+0000 7f08b8c01640 1 Processor -- start 2024-04-17T10:52:12.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.815+0000 7f08b8c01640 1 -- start start 2024-04-17T10:52:12.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.819+0000 7f08b8c01640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f08b415db30 0x7f08b40d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:12.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.819+0000 7f08b8c01640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f08b415df10 0x7f08b40cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:12.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.819+0000 7f08b8c01640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08b40d0be0 con 0x7f08b40cf720 2024-04-17T10:52:12.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.819+0000 7f08b8c01640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08b40ce4d0 con 0x7f08b415db30 2024-04-17T10:52:12.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.819+0000 7f08b8c01640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08b40d0ec0 con 0x7f08b415df10 2024-04-17T10:52:12.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.819+0000 7f08b259c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f08b415db30 0x7f08b40d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:12.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.819+0000 7f08b2d9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f08b40cf720 0x7f08b406b0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58970/0 (socket says 172.21.15.83:58970) 2024-04-17T10:52:12.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.819+0000 7f08b2d9d640 1 -- 172.21.15.83:0/3847867867 learned_addr learned my addr 172.21.15.83:0/3847867867 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:52:12.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.819+0000 7f08b259c640 1 --2- 172.21.15.83:0/3847867867 >> v2:172.21.15.83:3300/0 conn(0x7f08b415db30 0x7f08b40d0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47352/0 (socket says 172.21.15.83:47352) 2024-04-17T10:52:12.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.819+0000 7f08abfff640 1 --2- 172.21.15.83:0/3847867867 >> v2:172.21.15.167:3300/0 conn(0x7f08b415df10 0x7f08b40cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:12.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.819+0000 7f08b259c640 1 -- 172.21.15.83:0/3847867867 >> v2:172.21.15.167:3300/0 conn(0x7f08b415df10 msgr2=0x7f08b40cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:12.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.819+0000 7f08b259c640 1 --2- 172.21.15.83:0/3847867867 >> v2:172.21.15.167:3300/0 conn(0x7f08b415df10 0x7f08b40cf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:12.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.819+0000 7f08b259c640 1 -- 172.21.15.83:0/3847867867 >> v1:172.21.15.83:6789/0 conn(0x7f08b40cf720 legacy=0x7f08b406b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:12.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.819+0000 7f08b259c640 1 -- 172.21.15.83:0/3847867867 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b40ce700 con 0x7f08b415db30 2024-04-17T10:52:12.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.819+0000 7f08b259c640 1 --2- 172.21.15.83:0/3847867867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08b415db30 0x7f08b40d0320 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f08b408ce80 tx=0x7f08a401c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a3aa6f2187c196 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:12.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.819+0000 7f08b1d9b640 1 -- 172.21.15.83:0/3847867867 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a40b0070 con 0x7f08b415db30 2024-04-17T10:52:12.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.819+0000 7f08b1d9b640 1 -- 172.21.15.83:0/3847867867 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a40ab600 con 0x7f08b415db30 2024-04-17T10:52:12.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.819+0000 7f08b1d9b640 1 -- 172.21.15.83:0/3847867867 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a40ab9d0 con 0x7f08b415db30 2024-04-17T10:52:12.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.823+0000 7f08b8c01640 1 -- 172.21.15.83:0/3847867867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08b415db30 msgr2=0x7f08b40d0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:12.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.823+0000 7f08b8c01640 1 --2- 172.21.15.83:0/3847867867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08b415db30 0x7f08b40d0320 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f08b408ce80 tx=0x7f08a401c310 comp rx=0 tx=0).stop 2024-04-17T10:52:12.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.823+0000 7f08b8c01640 1 -- 172.21.15.83:0/3847867867 shutdown_connections 2024-04-17T10:52:12.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.823+0000 7f08b8c01640 1 --2- 172.21.15.83:0/3847867867 >> v2:172.21.15.167:3300/0 conn(0x7f08b415df10 0x7f08b40cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:12.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.823+0000 7f08b8c01640 1 --2- 172.21.15.83:0/3847867867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08b415db30 0x7f08b40d0320 unknown :-1 s=CLOSED pgs=2201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:12.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.823+0000 7f08b8c01640 1 -- 172.21.15.83:0/3847867867 >> 172.21.15.83:0/3847867867 conn(0x7f08b409cd50 msgr2=0x7f08b40d39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:12.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.823+0000 7f08b8c01640 1 -- 172.21.15.83:0/3847867867 shutdown_connections 2024-04-17T10:52:12.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.823+0000 7f08b8c01640 1 -- 172.21.15.83:0/3847867867 wait complete. 2024-04-17T10:52:12.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.823+0000 7f08b8c01640 1 Processor -- start 2024-04-17T10:52:12.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.823+0000 7f08b8c01640 1 -- start start 2024-04-17T10:52:12.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.823+0000 7f08b8c01640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08b415db30 0x7f08b414d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:12.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.823+0000 7f08b8c01640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08b415df10 0x7f08b40ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:12.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.823+0000 7f08b8c01640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08b40ca5f0 0x7f08b40c7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:12.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.823+0000 7f08b8c01640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b40d0860 con 0x7f08b415df10 2024-04-17T10:52:12.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.823+0000 7f08b8c01640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b40cecc0 con 0x7f08b40ca5f0 2024-04-17T10:52:12.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.823+0000 7f08b8c01640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b406b800 con 0x7f08b415db30 2024-04-17T10:52:12.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.823+0000 7f08abfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08b415df10 0x7f08b40ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:12.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.823+0000 7f08abfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08b415df10 0x7f08b40ca0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47366/0 (socket says 172.21.15.83:47366) 2024-04-17T10:52:12.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.823+0000 7f08abfff640 1 -- 172.21.15.83:0/2307192929 learned_addr learned my addr 172.21.15.83:0/2307192929 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:12.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.823+0000 7f08b2d9d640 1 --2- 172.21.15.83:0/2307192929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08b40ca5f0 0x7f08b40c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:12.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.823+0000 7f08abfff640 1 -- 172.21.15.83:0/2307192929 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08b415db30 msgr2=0x7f08b414d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:12.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.823+0000 7f08abfff640 1 --2- 172.21.15.83:0/2307192929 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08b415db30 0x7f08b414d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:12.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.823+0000 7f08abfff640 1 -- 172.21.15.83:0/2307192929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08b40ca5f0 msgr2=0x7f08b40c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:12.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.823+0000 7f08abfff640 1 --2- 172.21.15.83:0/2307192929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08b40ca5f0 0x7f08b40c7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:12.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.823+0000 7f08abfff640 1 -- 172.21.15.83:0/2307192929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b40c9010 con 0x7f08b415df10 2024-04-17T10:52:12.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.823+0000 7f08abfff640 1 --2- 172.21.15.83:0/2307192929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08b415df10 0x7f08b40ca0b0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f089c00d620 tx=0x7f089c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:12.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.823+0000 7f08aaffd640 1 -- 172.21.15.83:0/2307192929 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c00b620 con 0x7f08b415df10 2024-04-17T10:52:12.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.823+0000 7f08aaffd640 1 -- 172.21.15.83:0/2307192929 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f089c019070 con 0x7f08b415df10 2024-04-17T10:52:12.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.823+0000 7f08aaffd640 1 -- 172.21.15.83:0/2307192929 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c00b9e0 con 0x7f08b415df10 2024-04-17T10:52:12.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.823+0000 7f08b8c01640 1 -- 172.21.15.83:0/2307192929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08b40c92f0 con 0x7f08b415df10 2024-04-17T10:52:12.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.823+0000 7f08b8c01640 1 -- 172.21.15.83:0/2307192929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08b40c98b0 con 0x7f08b415df10 2024-04-17T10:52:12.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.827+0000 7f08b8c01640 1 -- 172.21.15.83:0/2307192929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08b40681a0 con 0x7f08b415df10 2024-04-17T10:52:12.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.827+0000 7f08aaffd640 1 -- 172.21.15.83:0/2307192929 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f089c00bb80 con 0x7f08b415df10 2024-04-17T10:52:12.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.827+0000 7f08aaffd640 1 --2- 172.21.15.83:0/2307192929 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0894066f60 0x7f0894069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:12.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.827+0000 7f08b259c640 1 --2- 172.21.15.83:0/2307192929 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0894066f60 0x7f0894069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:12.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.827+0000 7f08aaffd640 1 -- 172.21.15.83:0/2307192929 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f089c0854e0 con 0x7f08b415df10 2024-04-17T10:52:12.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.827+0000 7f08b259c640 1 --2- 172.21.15.83:0/2307192929 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0894066f60 0x7f0894069420 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f08a40ad000 tx=0x7f08a4004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:12.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:12.831+0000 7f08aaffd640 1 -- 172.21.15.83:0/2307192929 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f089c056480 con 0x7f08b415df10 2024-04-17T10:52:13.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:13.007+0000 7f08b8c01640 1 -- 172.21.15.83:0/2307192929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08b4074f90 con 0x7f08b415df10 2024-04-17T10:52:13.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:13.011+0000 7f08aaffd640 1 -- 172.21.15.83:0/2307192929 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f089c055d80 con 0x7f08b415df10 2024-04-17T10:52:13.003 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:13.004 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:13.004 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:13.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:13.011+0000 7f08b8c01640 1 -- 172.21.15.83:0/2307192929 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0894066f60 msgr2=0x7f0894069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:13.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:13.011+0000 7f08b8c01640 1 --2- 172.21.15.83:0/2307192929 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0894066f60 0x7f0894069420 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f08a40ad000 tx=0x7f08a4004740 comp rx=0 tx=0).stop 2024-04-17T10:52:13.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:13.011+0000 7f08b8c01640 1 -- 172.21.15.83:0/2307192929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08b415df10 msgr2=0x7f08b40ca0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:13.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:13.011+0000 7f08b8c01640 1 --2- 172.21.15.83:0/2307192929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08b415df10 0x7f08b40ca0b0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f089c00d620 tx=0x7f089c00dae0 comp rx=0 tx=0).stop 2024-04-17T10:52:13.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:13.011+0000 7f08b8c01640 1 -- 172.21.15.83:0/2307192929 shutdown_connections 2024-04-17T10:52:13.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:13.011+0000 7f08b8c01640 1 --2- 172.21.15.83:0/2307192929 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0894066f60 0x7f0894069420 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:13.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:13.011+0000 7f08b8c01640 1 --2- 172.21.15.83:0/2307192929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08b40ca5f0 0x7f08b40c7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:13.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:13.011+0000 7f08b8c01640 1 --2- 172.21.15.83:0/2307192929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08b415df10 0x7f08b40ca0b0 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:13.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:13.011+0000 7f08b8c01640 1 --2- 172.21.15.83:0/2307192929 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08b415db30 0x7f08b414d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:13.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:13.011+0000 7f08b8c01640 1 -- 172.21.15.83:0/2307192929 >> 172.21.15.83:0/2307192929 conn(0x7f08b409cd50 msgr2=0x7f08b40b5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:13.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:13.011+0000 7f08b8c01640 1 -- 172.21.15.83:0/2307192929 shutdown_connections 2024-04-17T10:52:13.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:13.011+0000 7f08b8c01640 1 -- 172.21.15.83:0/2307192929 wait complete. 2024-04-17T10:52:13.425 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:13.425 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:14.426 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:14.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe43ee94640 1 Processor -- start 2024-04-17T10:52:14.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe43ee94640 1 -- start start 2024-04-17T10:52:14.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe43ee94640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe43815ba30 0x7fe438072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:14.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe43ee94640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4380cfbe0 0x7fe43806c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:14.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe43ee94640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe43806e320 con 0x7fe43815c960 2024-04-17T10:52:14.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe43ee94640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe438071ea0 con 0x7fe43815ba30 2024-04-17T10:52:14.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe43ee94640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe43806e040 con 0x7fe4380cfbe0 2024-04-17T10:52:14.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe437fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe43815c960 0x7fe4380cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58984/0 (socket says 172.21.15.83:58984) 2024-04-17T10:52:14.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe437fff640 1 -- 172.21.15.83:0/4062295081 learned_addr learned my addr 172.21.15.83:0/4062295081 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:52:14.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe43d431640 1 --2- 172.21.15.83:0/4062295081 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4380cfbe0 0x7fe43806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:14.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe43cc30640 1 --2- 172.21.15.83:0/4062295081 >> v2:172.21.15.167:3300/0 conn(0x7fe43815ba30 0x7fe438072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:14.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe4377fe640 1 -- 172.21.15.83:0/4062295081 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2345135396 0 0) 0x7fe43806e320 con 0x7fe43815c960 2024-04-17T10:52:14.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe4377fe640 1 -- 172.21.15.83:0/4062295081 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe420003530 con 0x7fe43815c960 2024-04-17T10:52:14.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe4377fe640 1 -- 172.21.15.83:0/4062295081 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1753833605 0 0) 0x7fe420003530 con 0x7fe43815c960 2024-04-17T10:52:14.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe4377fe640 1 -- 172.21.15.83:0/4062295081 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4380cfbe0 msgr2=0x7fe43806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:14.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe4377fe640 1 --2- 172.21.15.83:0/4062295081 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4380cfbe0 0x7fe43806c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:14.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe4377fe640 1 -- 172.21.15.83:0/4062295081 >> v2:172.21.15.167:3300/0 conn(0x7fe43815ba30 msgr2=0x7fe438072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:14.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe4377fe640 1 --2- 172.21.15.83:0/4062295081 >> v2:172.21.15.167:3300/0 conn(0x7fe43815ba30 0x7fe438072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:14.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe4377fe640 1 -- 172.21.15.83:0/4062295081 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe438071850 con 0x7fe43815c960 2024-04-17T10:52:14.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe4377fe640 1 -- 172.21.15.83:0/4062295081 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fe428001a00 con 0x7fe43815c960 2024-04-17T10:52:14.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe4377fe640 1 -- 172.21.15.83:0/4062295081 >> v1:172.21.15.83:6789/0 conn(0x7fe43815c960 legacy=0x7fe4380cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:14.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe4377fe640 1 --2- 172.21.15.83:0/4062295081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4200039c0 0x7fe420003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:14.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe4377fe640 1 --2- 172.21.15.83:0/4062295081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe420004810 0x7fe420004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:14.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe4377fe640 1 --2- 172.21.15.83:0/4062295081 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4380cfbe0 0x7fe4200051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:14.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe4377fe640 1 -- 172.21.15.83:0/4062295081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe438071ea0 con 0x7fe420004810 2024-04-17T10:52:14.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe4377fe640 1 -- 172.21.15.83:0/4062295081 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43806e040 con 0x7fe4200039c0 2024-04-17T10:52:14.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe4377fe640 1 -- 172.21.15.83:0/4062295081 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42000b9d0 con 0x7fe4380cfbe0 2024-04-17T10:52:14.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe437fff640 1 --2- 172.21.15.83:0/4062295081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe420004810 0x7fe420004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:14.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe43d431640 1 --2- 172.21.15.83:0/4062295081 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4380cfbe0 0x7fe4200051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:14.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe43cc30640 1 --2- 172.21.15.83:0/4062295081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4200039c0 0x7fe420003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:14.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe437fff640 1 -- 172.21.15.83:0/4062295081 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4380cfbe0 msgr2=0x7fe4200051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:14.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe437fff640 1 --2- 172.21.15.83:0/4062295081 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4380cfbe0 0x7fe4200051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:14.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe437fff640 1 -- 172.21.15.83:0/4062295081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4200039c0 msgr2=0x7fe420003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:14.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe437fff640 1 --2- 172.21.15.83:0/4062295081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4200039c0 0x7fe420003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:14.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe437fff640 1 -- 172.21.15.83:0/4062295081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe420003530 con 0x7fe420004810 2024-04-17T10:52:14.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.563+0000 7fe43cc30640 1 --2- 172.21.15.83:0/4062295081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4200039c0 0x7fe420003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:52:14.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.567+0000 7fe437fff640 1 --2- 172.21.15.83:0/4062295081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe420004810 0x7fe420004c90 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7fe42800a8f0 tx=0x7fe42801d3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57194d81204a4a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:14.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.567+0000 7fe4377fe640 1 -- 172.21.15.83:0/4062295081 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42800a4e0 con 0x7fe420004810 2024-04-17T10:52:14.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.567+0000 7fe4377fe640 1 -- 172.21.15.83:0/4062295081 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4280ad650 con 0x7fe420004810 2024-04-17T10:52:14.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.567+0000 7fe4377fe640 1 -- 172.21.15.83:0/4062295081 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4280ada40 con 0x7fe420004810 2024-04-17T10:52:14.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.567+0000 7fe43ee94640 1 -- 172.21.15.83:0/4062295081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe420004810 msgr2=0x7fe420004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:14.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.567+0000 7fe43ee94640 1 --2- 172.21.15.83:0/4062295081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe420004810 0x7fe420004c90 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7fe42800a8f0 tx=0x7fe42801d3e0 comp rx=0 tx=0).stop 2024-04-17T10:52:14.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.567+0000 7fe43d431640 1 -- 172.21.15.83:0/4062295081 reap_dead start 2024-04-17T10:52:14.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.567+0000 7fe43ee94640 1 -- 172.21.15.83:0/4062295081 shutdown_connections 2024-04-17T10:52:14.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.567+0000 7fe43ee94640 1 -- 172.21.15.83:0/4062295081 >> 172.21.15.83:0/4062295081 conn(0x7fe43806abe0 msgr2=0x7fe43806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:14.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.567+0000 7fe43ee94640 1 -- 172.21.15.83:0/4062295081 shutdown_connections 2024-04-17T10:52:14.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.567+0000 7fe43ee94640 1 -- 172.21.15.83:0/4062295081 wait complete. 2024-04-17T10:52:14.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.567+0000 7fe43ee94640 1 Processor -- start 2024-04-17T10:52:14.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.567+0000 7fe43ee94640 1 -- start start 2024-04-17T10:52:14.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.567+0000 7fe43ee94640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe43814a6f0 0x7fe438150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:14.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.567+0000 7fe43ee94640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe438150550 0x7fe4380ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:14.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.567+0000 7fe43ee94640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe43814ae90 0x7fe4380add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:14.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.567+0000 7fe43ee94640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43806de20 con 0x7fe43814ae90 2024-04-17T10:52:14.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.567+0000 7fe43ee94640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43806dca0 con 0x7fe438150550 2024-04-17T10:52:14.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.567+0000 7fe43ee94640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe43806cca0 con 0x7fe43814a6f0 2024-04-17T10:52:14.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.567+0000 7fe43d431640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe43814ae90 0x7fe4380add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:14.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.567+0000 7fe43d431640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe43814ae90 0x7fe4380add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47386/0 (socket says 172.21.15.83:47386) 2024-04-17T10:52:14.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.567+0000 7fe43d431640 1 -- 172.21.15.83:0/3238152623 learned_addr learned my addr 172.21.15.83:0/3238152623 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:14.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.571+0000 7fe43cc30640 1 --2- 172.21.15.83:0/3238152623 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe43814a6f0 0x7fe438150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:14.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.571+0000 7fe437fff640 1 --2- 172.21.15.83:0/3238152623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe438150550 0x7fe4380ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:14.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.571+0000 7fe43d431640 1 -- 172.21.15.83:0/3238152623 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe43814a6f0 msgr2=0x7fe438150010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:14.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.571+0000 7fe43d431640 1 --2- 172.21.15.83:0/3238152623 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe43814a6f0 0x7fe438150010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:14.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.571+0000 7fe43d431640 1 -- 172.21.15.83:0/3238152623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe438150550 msgr2=0x7fe4380ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:14.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.571+0000 7fe43d431640 1 --2- 172.21.15.83:0/3238152623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe438150550 0x7fe4380ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:14.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.571+0000 7fe43d431640 1 -- 172.21.15.83:0/3238152623 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4380b4740 con 0x7fe43814ae90 2024-04-17T10:52:14.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.571+0000 7fe437fff640 1 --2- 172.21.15.83:0/3238152623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe438150550 0x7fe4380ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:52:14.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.571+0000 7fe43d431640 1 --2- 172.21.15.83:0/3238152623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe43814ae90 0x7fe4380add80 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7fe438070100 tx=0x7fe42c011860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:14.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.571+0000 7fe4357fa640 1 -- 172.21.15.83:0/3238152623 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42c004440 con 0x7fe43814ae90 2024-04-17T10:52:14.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.571+0000 7fe4357fa640 1 -- 172.21.15.83:0/3238152623 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe42c01b070 con 0x7fe43814ae90 2024-04-17T10:52:14.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.571+0000 7fe43ee94640 1 -- 172.21.15.83:0/3238152623 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe43814f250 con 0x7fe43814ae90 2024-04-17T10:52:14.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.571+0000 7fe4357fa640 1 -- 172.21.15.83:0/3238152623 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42c003860 con 0x7fe43814ae90 2024-04-17T10:52:14.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.571+0000 7fe43ee94640 1 -- 172.21.15.83:0/3238152623 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe43814f810 con 0x7fe43814ae90 2024-04-17T10:52:14.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.571+0000 7fe43ee94640 1 -- 172.21.15.83:0/3238152623 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe438074f90 con 0x7fe43814ae90 2024-04-17T10:52:14.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.571+0000 7fe4357fa640 1 -- 172.21.15.83:0/3238152623 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe42c003a00 con 0x7fe43814ae90 2024-04-17T10:52:14.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.571+0000 7fe4357fa640 1 --2- 172.21.15.83:0/3238152623 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe424066f60 0x7fe424069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:14.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.571+0000 7fe4357fa640 1 -- 172.21.15.83:0/3238152623 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fe42c084f30 con 0x7fe43814ae90 2024-04-17T10:52:14.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.571+0000 7fe43cc30640 1 --2- 172.21.15.83:0/3238152623 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe424066f60 0x7fe424069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:14.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.571+0000 7fe43cc30640 1 --2- 172.21.15.83:0/3238152623 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe424066f60 0x7fe424069420 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe4380b3dd0 tx=0x7fe430008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:14.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.575+0000 7fe4357fa640 1 -- 172.21.15.83:0/3238152623 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe42c025020 con 0x7fe43814ae90 2024-04-17T10:52:14.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.767+0000 7fe43ee94640 1 -- 172.21.15.83:0/3238152623 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe438074480 con 0x7fe43814ae90 2024-04-17T10:52:14.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.771+0000 7fe4357fa640 1 -- 172.21.15.83:0/3238152623 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fe42c055ed0 con 0x7fe43814ae90 2024-04-17T10:52:14.763 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:14.764 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:14.764 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:14.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.771+0000 7fe43ee94640 1 -- 172.21.15.83:0/3238152623 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe424066f60 msgr2=0x7fe424069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:14.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.771+0000 7fe43ee94640 1 --2- 172.21.15.83:0/3238152623 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe424066f60 0x7fe424069420 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe4380b3dd0 tx=0x7fe430008040 comp rx=0 tx=0).stop 2024-04-17T10:52:14.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.771+0000 7fe43ee94640 1 -- 172.21.15.83:0/3238152623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe43814ae90 msgr2=0x7fe4380add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:14.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.771+0000 7fe43ee94640 1 --2- 172.21.15.83:0/3238152623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe43814ae90 0x7fe4380add80 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7fe438070100 tx=0x7fe42c011860 comp rx=0 tx=0).stop 2024-04-17T10:52:14.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.771+0000 7fe43ee94640 1 -- 172.21.15.83:0/3238152623 shutdown_connections 2024-04-17T10:52:14.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.771+0000 7fe43ee94640 1 --2- 172.21.15.83:0/3238152623 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe424066f60 0x7fe424069420 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:14.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.771+0000 7fe43ee94640 1 --2- 172.21.15.83:0/3238152623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe43814ae90 0x7fe4380add80 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:14.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.771+0000 7fe43ee94640 1 --2- 172.21.15.83:0/3238152623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe438150550 0x7fe4380ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:14.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.771+0000 7fe43ee94640 1 --2- 172.21.15.83:0/3238152623 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe43814a6f0 0x7fe438150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:14.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.771+0000 7fe43ee94640 1 -- 172.21.15.83:0/3238152623 >> 172.21.15.83:0/3238152623 conn(0x7fe43806abe0 msgr2=0x7fe438159710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:14.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.771+0000 7fe43ee94640 1 -- 172.21.15.83:0/3238152623 shutdown_connections 2024-04-17T10:52:14.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:14.771+0000 7fe43ee94640 1 -- 172.21.15.83:0/3238152623 wait complete. 2024-04-17T10:52:15.184 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:15.185 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:16.186 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:16.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.323+0000 7fdf0c5ec640 1 Processor -- start 2024-04-17T10:52:16.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.323+0000 7fdf0c5ec640 1 -- start start 2024-04-17T10:52:16.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.323+0000 7fdf0c5ec640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf0415ba30 0x7fdf04072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:16.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.323+0000 7fdf0c5ec640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdf0415c960 0x7fdf040cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:16.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.323+0000 7fdf0c5ec640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdf040cfa10 0x7fdf0406c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:16.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.323+0000 7fdf0c5ec640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf04071ea0 con 0x7fdf0415c960 2024-04-17T10:52:16.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.323+0000 7fdf0c5ec640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf040cfea0 con 0x7fdf040cfa10 2024-04-17T10:52:16.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.323+0000 7fdf0c5ec640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0406e320 con 0x7fdf0415ba30 2024-04-17T10:52:16.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.323+0000 7fdf09b87640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdf0415c960 0x7fdf040cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:16.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.323+0000 7fdf09b87640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdf0415c960 0x7fdf040cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47390/0 (socket says 172.21.15.83:47390) 2024-04-17T10:52:16.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.323+0000 7fdf09b87640 1 -- 172.21.15.83:0/2917169129 learned_addr learned my addr 172.21.15.83:0/2917169129 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:16.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.323+0000 7fdf0ab89640 1 --2- 172.21.15.83:0/2917169129 >> v2:172.21.15.167:3300/0 conn(0x7fdf040cfa10 0x7fdf0406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:16.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.323+0000 7fdf0a388640 1 --2- 172.21.15.83:0/2917169129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf0415ba30 0x7fdf04072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:16.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.323+0000 7fdf09b87640 1 -- 172.21.15.83:0/2917169129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf0415ba30 msgr2=0x7fdf04072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:16.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.323+0000 7fdf09b87640 1 --2- 172.21.15.83:0/2917169129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf0415ba30 0x7fdf04072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:16.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.323+0000 7fdf09b87640 1 -- 172.21.15.83:0/2917169129 >> v2:172.21.15.167:3300/0 conn(0x7fdf040cfa10 msgr2=0x7fdf0406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:16.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.323+0000 7fdf09b87640 1 --2- 172.21.15.83:0/2917169129 >> v2:172.21.15.167:3300/0 conn(0x7fdf040cfa10 0x7fdf0406c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:16.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.323+0000 7fdf09b87640 1 -- 172.21.15.83:0/2917169129 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0406e040 con 0x7fdf0415c960 2024-04-17T10:52:16.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.323+0000 7fdf0ab89640 1 --2- 172.21.15.83:0/2917169129 >> v2:172.21.15.167:3300/0 conn(0x7fdf040cfa10 0x7fdf0406c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:52:16.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.327+0000 7fdf09b87640 1 --2- 172.21.15.83:0/2917169129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf0415c960 0x7fdf040cf4d0 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7fdf0406d810 tx=0x7fdef401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d95fe21ae6b043 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:16.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.327+0000 7fdf09386640 1 -- 172.21.15.83:0/2917169129 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef40b0070 con 0x7fdf0415c960 2024-04-17T10:52:16.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.327+0000 7fdf09386640 1 -- 172.21.15.83:0/2917169129 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef40ab600 con 0x7fdf0415c960 2024-04-17T10:52:16.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.327+0000 7fdf09386640 1 -- 172.21.15.83:0/2917169129 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef40ab9d0 con 0x7fdf0415c960 2024-04-17T10:52:16.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.327+0000 7fdf0c5ec640 1 -- 172.21.15.83:0/2917169129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf0415c960 msgr2=0x7fdf040cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:16.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.327+0000 7fdf0c5ec640 1 --2- 172.21.15.83:0/2917169129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf0415c960 0x7fdf040cf4d0 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7fdf0406d810 tx=0x7fdef401c490 comp rx=0 tx=0).stop 2024-04-17T10:52:16.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.327+0000 7fdf0c5ec640 1 -- 172.21.15.83:0/2917169129 shutdown_connections 2024-04-17T10:52:16.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.327+0000 7fdf0c5ec640 1 --2- 172.21.15.83:0/2917169129 >> v2:172.21.15.167:3300/0 conn(0x7fdf040cfa10 0x7fdf0406c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:16.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.327+0000 7fdf0c5ec640 1 --2- 172.21.15.83:0/2917169129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf0415c960 0x7fdf040cf4d0 unknown :-1 s=CLOSED pgs=2206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:16.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.327+0000 7fdf0c5ec640 1 --2- 172.21.15.83:0/2917169129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf0415ba30 0x7fdf04072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:16.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.327+0000 7fdf0c5ec640 1 -- 172.21.15.83:0/2917169129 >> 172.21.15.83:0/2917169129 conn(0x7fdf0406abe0 msgr2=0x7fdf0406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:16.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.327+0000 7fdf0c5ec640 1 -- 172.21.15.83:0/2917169129 shutdown_connections 2024-04-17T10:52:16.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.327+0000 7fdf0c5ec640 1 -- 172.21.15.83:0/2917169129 wait complete. 2024-04-17T10:52:16.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.331+0000 7fdf0c5ec640 1 Processor -- start 2024-04-17T10:52:16.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.331+0000 7fdf0c5ec640 1 -- start start 2024-04-17T10:52:16.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.331+0000 7fdf0c5ec640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf040cfa10 0x7fdf041506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:16.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.331+0000 7fdf0c5ec640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf0415ba30 0x7fdf040af500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:16.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.331+0000 7fdf0c5ec640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf0415c960 0x7fdf040afa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:16.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.331+0000 7fdf0c5ec640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0406dca0 con 0x7fdf040cfa10 2024-04-17T10:52:16.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.331+0000 7fdf0c5ec640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf040aeae0 con 0x7fdf0415ba30 2024-04-17T10:52:16.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.331+0000 7fdf0c5ec640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0406de20 con 0x7fdf0415c960 2024-04-17T10:52:16.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.331+0000 7fdf0a388640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf040cfa10 0x7fdf041506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:16.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.331+0000 7fdf0a388640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf040cfa10 0x7fdf041506d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47402/0 (socket says 172.21.15.83:47402) 2024-04-17T10:52:16.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.331+0000 7fdf0a388640 1 -- 172.21.15.83:0/3000288289 learned_addr learned my addr 172.21.15.83:0/3000288289 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:16.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.331+0000 7fdf09b87640 1 --2- 172.21.15.83:0/3000288289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf0415ba30 0x7fdf040af500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:16.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.331+0000 7fdf0ab89640 1 --2- 172.21.15.83:0/3000288289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf0415c960 0x7fdf040afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:16.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.331+0000 7fdf0a388640 1 -- 172.21.15.83:0/3000288289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf0415c960 msgr2=0x7fdf040afa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:16.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.331+0000 7fdf0a388640 1 --2- 172.21.15.83:0/3000288289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf0415c960 0x7fdf040afa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:16.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.331+0000 7fdf0a388640 1 -- 172.21.15.83:0/3000288289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf0415ba30 msgr2=0x7fdf040af500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:16.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.331+0000 7fdf0a388640 1 --2- 172.21.15.83:0/3000288289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf0415ba30 0x7fdf040af500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:16.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.331+0000 7fdf0a388640 1 -- 172.21.15.83:0/3000288289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf04156ac0 con 0x7fdf040cfa10 2024-04-17T10:52:16.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.331+0000 7fdf0a388640 1 --2- 172.21.15.83:0/3000288289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf040cfa10 0x7fdf041506d0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000d5f0 tx=0x7fdf0000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:16.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.331+0000 7fdefaffd640 1 -- 172.21.15.83:0/3000288289 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf000125f0 con 0x7fdf040cfa10 2024-04-17T10:52:16.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.331+0000 7fdefaffd640 1 -- 172.21.15.83:0/3000288289 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf00019070 con 0x7fdf040cfa10 2024-04-17T10:52:16.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.331+0000 7fdefaffd640 1 -- 172.21.15.83:0/3000288289 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf000129b0 con 0x7fdf040cfa10 2024-04-17T10:52:16.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.331+0000 7fdf0c5ec640 1 -- 172.21.15.83:0/3000288289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf04156da0 con 0x7fdf040cfa10 2024-04-17T10:52:16.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.331+0000 7fdf0c5ec640 1 -- 172.21.15.83:0/3000288289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf04156750 con 0x7fdf040cfa10 2024-04-17T10:52:16.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.331+0000 7fdf0c5ec640 1 -- 172.21.15.83:0/3000288289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf04074f90 con 0x7fdf040cfa10 2024-04-17T10:52:16.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.331+0000 7fdefaffd640 1 -- 172.21.15.83:0/3000288289 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdf00012b50 con 0x7fdf040cfa10 2024-04-17T10:52:16.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.331+0000 7fdefaffd640 1 --2- 172.21.15.83:0/3000288289 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdee8066f90 0x7fdee8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:16.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.335+0000 7fdefaffd640 1 -- 172.21.15.83:0/3000288289 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fdf000853e0 con 0x7fdf040cfa10 2024-04-17T10:52:16.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.335+0000 7fdf09b87640 1 --2- 172.21.15.83:0/3000288289 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdee8066f90 0x7fdee8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:16.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.335+0000 7fdf09b87640 1 --2- 172.21.15.83:0/3000288289 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdee8066f90 0x7fdee8069450 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fdf04070060 tx=0x7fdef4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:16.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.339+0000 7fdefaffd640 1 -- 172.21.15.83:0/3000288289 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdf00056410 con 0x7fdf040cfa10 2024-04-17T10:52:16.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.531+0000 7fdf0c5ec640 1 -- 172.21.15.83:0/3000288289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf040b0c00 con 0x7fdf040cfa10 2024-04-17T10:52:16.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.531+0000 7fdefaffd640 1 -- 172.21.15.83:0/3000288289 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fdf00055d10 con 0x7fdf040cfa10 2024-04-17T10:52:16.526 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:16.526 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:16.526 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.535+0000 7fdf0c5ec640 1 -- 172.21.15.83:0/3000288289 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdee8066f90 msgr2=0x7fdee8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.535+0000 7fdf0c5ec640 1 --2- 172.21.15.83:0/3000288289 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdee8066f90 0x7fdee8069450 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fdf04070060 tx=0x7fdef4008040 comp rx=0 tx=0).stop 2024-04-17T10:52:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.535+0000 7fdf0c5ec640 1 -- 172.21.15.83:0/3000288289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf040cfa10 msgr2=0x7fdf041506d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.535+0000 7fdf0c5ec640 1 --2- 172.21.15.83:0/3000288289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf040cfa10 0x7fdf041506d0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000d5f0 tx=0x7fdf0000dab0 comp rx=0 tx=0).stop 2024-04-17T10:52:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.535+0000 7fdf0c5ec640 1 -- 172.21.15.83:0/3000288289 shutdown_connections 2024-04-17T10:52:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.535+0000 7fdf0c5ec640 1 --2- 172.21.15.83:0/3000288289 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdee8066f90 0x7fdee8069450 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.535+0000 7fdf0c5ec640 1 --2- 172.21.15.83:0/3000288289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf0415c960 0x7fdf040afa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.535+0000 7fdf0c5ec640 1 --2- 172.21.15.83:0/3000288289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf0415ba30 0x7fdf040af500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.535+0000 7fdf0c5ec640 1 --2- 172.21.15.83:0/3000288289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf040cfa10 0x7fdf041506d0 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.535+0000 7fdf0c5ec640 1 -- 172.21.15.83:0/3000288289 >> 172.21.15.83:0/3000288289 conn(0x7fdf0406abe0 msgr2=0x7fdf0415e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.535+0000 7fdf0c5ec640 1 -- 172.21.15.83:0/3000288289 shutdown_connections 2024-04-17T10:52:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:16.535+0000 7fdf0c5ec640 1 -- 172.21.15.83:0/3000288289 wait complete. 2024-04-17T10:52:16.947 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:16.948 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:17.948 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:18.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.086+0000 7fca6f650640 1 Processor -- start 2024-04-17T10:52:18.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.086+0000 7fca6f650640 1 -- start start 2024-04-17T10:52:18.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.086+0000 7fca6f650640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fca6806dcb0 0x7fca68165650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:18.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.086+0000 7fca6f650640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca6815e440 con 0x7fca6815db50 2024-04-17T10:52:18.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.086+0000 7fca6f650640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca6815e720 con 0x7fca680aedd0 2024-04-17T10:52:18.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.086+0000 7fca6f650640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca6815ea00 con 0x7fca6806dcb0 2024-04-17T10:52:18.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.086+0000 7fca6dbed640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fca6806dcb0 0x7fca68165650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:18.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.086+0000 7fca6dbed640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fca6806dcb0 0x7fca68165650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52322/0 (socket says 172.21.15.83:52322) 2024-04-17T10:52:18.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.086+0000 7fca6d3ec640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fca6815db50 0x7fca68070470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39834/0 (socket says 172.21.15.83:39834) 2024-04-17T10:52:18.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.086+0000 7fca6dbed640 1 -- 172.21.15.83:0/973830123 learned_addr learned my addr 172.21.15.83:0/973830123 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:18.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.086+0000 7fca6dbed640 1 -- 172.21.15.83:0/973830123 >> v1:172.21.15.167:6789/0 conn(0x7fca680aedd0 legacy=0x7fca6806d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:18.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.086+0000 7fca5ffff640 1 -- 172.21.15.83:0/973830123 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 968945065 0 0) 0x7fca6815e440 con 0x7fca6815db50 2024-04-17T10:52:18.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.086+0000 7fca6dbed640 1 -- 172.21.15.83:0/973830123 >> v1:172.21.15.83:6789/0 conn(0x7fca6815db50 legacy=0x7fca68070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:18.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.086+0000 7fca6dbed640 1 -- 172.21.15.83:0/973830123 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca6815ece0 con 0x7fca6806dcb0 2024-04-17T10:52:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.090+0000 7fca6dbed640 1 --2- 172.21.15.83:0/973830123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca6806dcb0 0x7fca68165650 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7fca60009e80 tx=0x7fca6001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75b8c41f93fae0f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.090+0000 7fca5ffff640 1 -- 172.21.15.83:0/973830123 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca600a5770 con 0x7fca6806dcb0 2024-04-17T10:52:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.090+0000 7fca5ffff640 1 -- 172.21.15.83:0/973830123 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca600a5910 con 0x7fca6806dcb0 2024-04-17T10:52:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.090+0000 7fca5ffff640 1 -- 172.21.15.83:0/973830123 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca600a5ce0 con 0x7fca6806dcb0 2024-04-17T10:52:18.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.090+0000 7fca6f650640 1 -- 172.21.15.83:0/973830123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca6806dcb0 msgr2=0x7fca68165650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:18.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.090+0000 7fca6f650640 1 --2- 172.21.15.83:0/973830123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca6806dcb0 0x7fca68165650 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7fca60009e80 tx=0x7fca6001c9a0 comp rx=0 tx=0).stop 2024-04-17T10:52:18.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.090+0000 7fca6f650640 1 -- 172.21.15.83:0/973830123 shutdown_connections 2024-04-17T10:52:18.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.090+0000 7fca6f650640 1 --2- 172.21.15.83:0/973830123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca6806dcb0 0x7fca68165650 unknown :-1 s=CLOSED pgs=2209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:18.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.090+0000 7fca6f650640 1 -- 172.21.15.83:0/973830123 >> 172.21.15.83:0/973830123 conn(0x7fca6806abe0 msgr2=0x7fca6806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:18.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.090+0000 7fca6f650640 1 -- 172.21.15.83:0/973830123 shutdown_connections 2024-04-17T10:52:18.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.090+0000 7fca6f650640 1 -- 172.21.15.83:0/973830123 wait complete. 2024-04-17T10:52:18.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.090+0000 7fca6f650640 1 Processor -- start 2024-04-17T10:52:18.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.090+0000 7fca6f650640 1 -- start start 2024-04-17T10:52:18.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.090+0000 7fca6f650640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca6806dcb0 0x7fca680ae600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:18.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.090+0000 7fca6f650640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca680aedd0 0x7fca680afde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:18.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.090+0000 7fca6f650640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca6815db50 0x7fca6814eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:18.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.090+0000 7fca6f650640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca68168b50 con 0x7fca6815db50 2024-04-17T10:52:18.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.090+0000 7fca6f650640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca681689d0 con 0x7fca680aedd0 2024-04-17T10:52:18.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.090+0000 7fca6f650640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca68168cd0 con 0x7fca6806dcb0 2024-04-17T10:52:18.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.090+0000 7fca6dbed640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca6815db50 0x7fca6814eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:18.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.090+0000 7fca6dbed640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca6815db50 0x7fca6814eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52334/0 (socket says 172.21.15.83:52334) 2024-04-17T10:52:18.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.090+0000 7fca6dbed640 1 -- 172.21.15.83:0/1984692918 learned_addr learned my addr 172.21.15.83:0/1984692918 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:18.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.090+0000 7fca6cbeb640 1 --2- 172.21.15.83:0/1984692918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca680aedd0 0x7fca680afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:18.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.090+0000 7fca6dbed640 1 -- 172.21.15.83:0/1984692918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca6806dcb0 msgr2=0x7fca680ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:18.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.090+0000 7fca6d3ec640 1 --2- 172.21.15.83:0/1984692918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca6806dcb0 0x7fca680ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:18.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.090+0000 7fca6dbed640 1 --2- 172.21.15.83:0/1984692918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca6806dcb0 0x7fca680ae600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:18.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.090+0000 7fca6dbed640 1 -- 172.21.15.83:0/1984692918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca680aedd0 msgr2=0x7fca680afde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:18.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.090+0000 7fca6dbed640 1 --2- 172.21.15.83:0/1984692918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca680aedd0 0x7fca680afde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:18.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.090+0000 7fca6dbed640 1 -- 172.21.15.83:0/1984692918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca68154910 con 0x7fca6815db50 2024-04-17T10:52:18.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.090+0000 7fca6cbeb640 1 --2- 172.21.15.83:0/1984692918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca680aedd0 0x7fca680afde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:52:18.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.090+0000 7fca6dbed640 1 --2- 172.21.15.83:0/1984692918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca6815db50 0x7fca6814eae0 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7fca600af040 tx=0x7fca60019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:18.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.094+0000 7fca5dffb640 1 -- 172.21.15.83:0/1984692918 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca600bc020 con 0x7fca6815db50 2024-04-17T10:52:18.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.094+0000 7fca5dffb640 1 -- 172.21.15.83:0/1984692918 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca600b8670 con 0x7fca6815db50 2024-04-17T10:52:18.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.094+0000 7fca5dffb640 1 -- 172.21.15.83:0/1984692918 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca600b8a40 con 0x7fca6815db50 2024-04-17T10:52:18.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.094+0000 7fca6f650640 1 -- 172.21.15.83:0/1984692918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca68157d00 con 0x7fca6815db50 2024-04-17T10:52:18.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.094+0000 7fca6f650640 1 -- 172.21.15.83:0/1984692918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca680aeb40 con 0x7fca6815db50 2024-04-17T10:52:18.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.094+0000 7fca6f650640 1 -- 172.21.15.83:0/1984692918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca68074f90 con 0x7fca6815db50 2024-04-17T10:52:18.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.094+0000 7fca5dffb640 1 -- 172.21.15.83:0/1984692918 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fca600bf020 con 0x7fca6815db50 2024-04-17T10:52:18.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.094+0000 7fca5dffb640 1 --2- 172.21.15.83:0/1984692918 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fca4c066f60 0x7fca4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:18.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.094+0000 7fca5dffb640 1 -- 172.21.15.83:0/1984692918 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fca60120200 con 0x7fca6815db50 2024-04-17T10:52:18.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.094+0000 7fca6d3ec640 1 --2- 172.21.15.83:0/1984692918 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fca4c066f60 0x7fca4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:18.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.094+0000 7fca6d3ec640 1 --2- 172.21.15.83:0/1984692918 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fca4c066f60 0x7fca4c069420 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fca64000f70 tx=0x7fca64009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:18.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.098+0000 7fca5dffb640 1 -- 172.21.15.83:0/1984692918 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fca600c6030 con 0x7fca6815db50 2024-04-17T10:52:18.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.294+0000 7fca6f650640 1 -- 172.21.15.83:0/1984692918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca680b2af0 con 0x7fca6815db50 2024-04-17T10:52:18.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.294+0000 7fca5dffb640 1 -- 172.21.15.83:0/1984692918 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fca600165c0 con 0x7fca6815db50 2024-04-17T10:52:18.288 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:18.288 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:18.289 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:18.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.294+0000 7fca6f650640 1 -- 172.21.15.83:0/1984692918 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fca4c066f60 msgr2=0x7fca4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:18.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.294+0000 7fca6f650640 1 --2- 172.21.15.83:0/1984692918 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fca4c066f60 0x7fca4c069420 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fca64000f70 tx=0x7fca64009040 comp rx=0 tx=0).stop 2024-04-17T10:52:18.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.294+0000 7fca6f650640 1 -- 172.21.15.83:0/1984692918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca6815db50 msgr2=0x7fca6814eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:18.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.294+0000 7fca6f650640 1 --2- 172.21.15.83:0/1984692918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca6815db50 0x7fca6814eae0 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7fca600af040 tx=0x7fca60019740 comp rx=0 tx=0).stop 2024-04-17T10:52:18.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.294+0000 7fca6f650640 1 -- 172.21.15.83:0/1984692918 shutdown_connections 2024-04-17T10:52:18.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.294+0000 7fca6f650640 1 --2- 172.21.15.83:0/1984692918 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fca4c066f60 0x7fca4c069420 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:18.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.294+0000 7fca6f650640 1 --2- 172.21.15.83:0/1984692918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca6815db50 0x7fca6814eae0 unknown :-1 s=CLOSED pgs=2210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:18.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.294+0000 7fca6f650640 1 --2- 172.21.15.83:0/1984692918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca680aedd0 0x7fca680afde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:18.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.294+0000 7fca6f650640 1 --2- 172.21.15.83:0/1984692918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca6806dcb0 0x7fca680ae600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:18.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.294+0000 7fca6f650640 1 -- 172.21.15.83:0/1984692918 >> 172.21.15.83:0/1984692918 conn(0x7fca6806abe0 msgr2=0x7fca680b0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:18.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.294+0000 7fca6f650640 1 -- 172.21.15.83:0/1984692918 shutdown_connections 2024-04-17T10:52:18.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:18.294+0000 7fca6f650640 1 -- 172.21.15.83:0/1984692918 wait complete. 2024-04-17T10:52:18.710 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:18.710 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:19.711 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:19.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.846+0000 7f442b006640 1 Processor -- start 2024-04-17T10:52:19.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.846+0000 7f442b006640 1 -- start start 2024-04-17T10:52:19.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.850+0000 7f442b006640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4424146a00 0x7f4424146e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:19.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.850+0000 7f442b006640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f442406d690 0x7f44241620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:19.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.850+0000 7f442b006640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f44240b2670 0x7f44240b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:19.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.850+0000 7f442b006640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4424162610 con 0x7f442406d690 2024-04-17T10:52:19.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.850+0000 7f442b006640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f442406c960 con 0x7f44240b2670 2024-04-17T10:52:19.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.850+0000 7f442b006640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f442406d200 con 0x7f4424146a00 2024-04-17T10:52:19.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.850+0000 7f4423fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f442406d690 0x7f44241620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:19.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.850+0000 7f4423fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f442406d690 0x7f44241620d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52346/0 (socket says 172.21.15.83:52346) 2024-04-17T10:52:19.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.850+0000 7f4423fff640 1 -- 172.21.15.83:0/687453232 learned_addr learned my addr 172.21.15.83:0/687453232 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:19.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.850+0000 7f4423fff640 1 -- 172.21.15.83:0/687453232 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4424146a00 msgr2=0x7f4424146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:19.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.850+0000 7f44295a3640 1 --2- 172.21.15.83:0/687453232 >> v2:172.21.15.167:3300/0 conn(0x7f44240b2670 0x7f44240b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:19.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.850+0000 7f4428da2640 1 --2- 172.21.15.83:0/687453232 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4424146a00 0x7f4424146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:19.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.850+0000 7f4423fff640 1 --2- 172.21.15.83:0/687453232 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4424146a00 0x7f4424146e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:19.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.850+0000 7f4423fff640 1 -- 172.21.15.83:0/687453232 >> v2:172.21.15.167:3300/0 conn(0x7f44240b2670 msgr2=0x7f44240b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:19.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.850+0000 7f4423fff640 1 --2- 172.21.15.83:0/687453232 >> v2:172.21.15.167:3300/0 conn(0x7f44240b2670 0x7f44240b2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:19.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.850+0000 7f4423fff640 1 -- 172.21.15.83:0/687453232 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f442406daf0 con 0x7f442406d690 2024-04-17T10:52:19.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.850+0000 7f4423fff640 1 --2- 172.21.15.83:0/687453232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f442406d690 0x7f44241620d0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f44240b43a0 tx=0x7f440c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5421c0fc97845b62 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:19.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.850+0000 7f44237fe640 1 -- 172.21.15.83:0/687453232 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c0ab5f0 con 0x7f442406d690 2024-04-17T10:52:19.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.850+0000 7f44237fe640 1 -- 172.21.15.83:0/687453232 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f440c0b0070 con 0x7f442406d690 2024-04-17T10:52:19.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.850+0000 7f44237fe640 1 -- 172.21.15.83:0/687453232 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c0ab9b0 con 0x7f442406d690 2024-04-17T10:52:19.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.854+0000 7f442b006640 1 -- 172.21.15.83:0/687453232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f442406d690 msgr2=0x7f44241620d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:19.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.854+0000 7f442b006640 1 --2- 172.21.15.83:0/687453232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f442406d690 0x7f44241620d0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f44240b43a0 tx=0x7f440c01c520 comp rx=0 tx=0).stop 2024-04-17T10:52:19.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.854+0000 7f442b006640 1 -- 172.21.15.83:0/687453232 shutdown_connections 2024-04-17T10:52:19.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.854+0000 7f442b006640 1 --2- 172.21.15.83:0/687453232 >> v2:172.21.15.167:3300/0 conn(0x7f44240b2670 0x7f44240b2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:19.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.854+0000 7f442b006640 1 --2- 172.21.15.83:0/687453232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f442406d690 0x7f44241620d0 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:19.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.854+0000 7f442b006640 1 --2- 172.21.15.83:0/687453232 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4424146a00 0x7f4424146e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:19.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.854+0000 7f442b006640 1 -- 172.21.15.83:0/687453232 >> 172.21.15.83:0/687453232 conn(0x7f442406abe0 msgr2=0x7f442406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:19.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.854+0000 7f442b006640 1 -- 172.21.15.83:0/687453232 shutdown_connections 2024-04-17T10:52:19.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.854+0000 7f442b006640 1 -- 172.21.15.83:0/687453232 wait complete. 2024-04-17T10:52:19.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.854+0000 7f442b006640 1 Processor -- start 2024-04-17T10:52:19.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.854+0000 7f442b006640 1 -- start start 2024-04-17T10:52:19.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.854+0000 7f442b006640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f442406d690 0x7f442415d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:19.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.854+0000 7f442b006640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44240b2670 0x7f442413a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:19.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.854+0000 7f442b006640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4424146a00 0x7f442413c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:19.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.854+0000 7f442b006640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4424070e00 con 0x7f4424146a00 2024-04-17T10:52:19.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.854+0000 7f442b006640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4424070c80 con 0x7f44240b2670 2024-04-17T10:52:19.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.854+0000 7f442b006640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4424070f80 con 0x7f442406d690 2024-04-17T10:52:19.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.854+0000 7f44295a3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4424146a00 0x7f442413c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:19.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.854+0000 7f44295a3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4424146a00 0x7f442413c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52362/0 (socket says 172.21.15.83:52362) 2024-04-17T10:52:19.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.854+0000 7f44295a3640 1 -- 172.21.15.83:0/651382704 learned_addr learned my addr 172.21.15.83:0/651382704 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:19.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.854+0000 7f44295a3640 1 -- 172.21.15.83:0/651382704 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f442406d690 msgr2=0x7f442415d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:19.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.854+0000 7f44295a3640 1 --2- 172.21.15.83:0/651382704 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f442406d690 0x7f442415d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:19.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.854+0000 7f44295a3640 1 -- 172.21.15.83:0/651382704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44240b2670 msgr2=0x7f442413a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:19.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.854+0000 7f4423fff640 1 --2- 172.21.15.83:0/651382704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44240b2670 0x7f442413a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:19.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.854+0000 7f44295a3640 1 --2- 172.21.15.83:0/651382704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44240b2670 0x7f442413a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:19.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.854+0000 7f44295a3640 1 -- 172.21.15.83:0/651382704 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f442413aa30 con 0x7f4424146a00 2024-04-17T10:52:19.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.854+0000 7f44295a3640 1 --2- 172.21.15.83:0/651382704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4424146a00 0x7f442413c200 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7f441c00cbb0 tx=0x7f441c00a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:19.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.854+0000 7f44217fa640 1 -- 172.21.15.83:0/651382704 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441c015070 con 0x7f4424146a00 2024-04-17T10:52:19.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.854+0000 7f44217fa640 1 -- 172.21.15.83:0/651382704 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f441c00abd0 con 0x7f4424146a00 2024-04-17T10:52:19.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.854+0000 7f442b006640 1 -- 172.21.15.83:0/651382704 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f442413e320 con 0x7f4424146a00 2024-04-17T10:52:19.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.854+0000 7f44217fa640 1 -- 172.21.15.83:0/651382704 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441c00d040 con 0x7f4424146a00 2024-04-17T10:52:19.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.854+0000 7f442b006640 1 -- 172.21.15.83:0/651382704 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f442413e8e0 con 0x7f4424146a00 2024-04-17T10:52:19.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.858+0000 7f442b006640 1 -- 172.21.15.83:0/651382704 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4424074f90 con 0x7f4424146a00 2024-04-17T10:52:19.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.858+0000 7f44217fa640 1 -- 172.21.15.83:0/651382704 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f441c011410 con 0x7f4424146a00 2024-04-17T10:52:19.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.858+0000 7f44217fa640 1 --2- 172.21.15.83:0/651382704 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4404066f60 0x7f4404069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:19.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.858+0000 7f44217fa640 1 -- 172.21.15.83:0/651382704 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f441c084bd0 con 0x7f4424146a00 2024-04-17T10:52:19.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.858+0000 7f4428da2640 1 --2- 172.21.15.83:0/651382704 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4404066f60 0x7f4404069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:19.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.858+0000 7f4428da2640 1 --2- 172.21.15.83:0/651382704 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4404066f60 0x7f4404069420 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f44241461f0 tx=0x7f4414008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:19.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:19.862+0000 7f44217fa640 1 -- 172.21.15.83:0/651382704 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f441c055b70 con 0x7f4424146a00 2024-04-17T10:52:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:20.046+0000 7f442b006640 1 -- 172.21.15.83:0/651382704 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f442406b160 con 0x7f4424146a00 2024-04-17T10:52:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:20.046+0000 7f44217fa640 1 -- 172.21.15.83:0/651382704 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f441c055470 con 0x7f4424146a00 2024-04-17T10:52:20.041 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:20.041 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:20.041 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:20.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:20.046+0000 7f442b006640 1 -- 172.21.15.83:0/651382704 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4404066f60 msgr2=0x7f4404069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:20.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:20.046+0000 7f442b006640 1 --2- 172.21.15.83:0/651382704 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4404066f60 0x7f4404069420 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f44241461f0 tx=0x7f4414008040 comp rx=0 tx=0).stop 2024-04-17T10:52:20.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:20.046+0000 7f442b006640 1 -- 172.21.15.83:0/651382704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4424146a00 msgr2=0x7f442413c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:20.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:20.046+0000 7f442b006640 1 --2- 172.21.15.83:0/651382704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4424146a00 0x7f442413c200 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7f441c00cbb0 tx=0x7f441c00a4c0 comp rx=0 tx=0).stop 2024-04-17T10:52:20.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:20.050+0000 7f442b006640 1 -- 172.21.15.83:0/651382704 shutdown_connections 2024-04-17T10:52:20.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:20.050+0000 7f442b006640 1 --2- 172.21.15.83:0/651382704 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4404066f60 0x7f4404069420 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:20.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:20.050+0000 7f442b006640 1 --2- 172.21.15.83:0/651382704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4424146a00 0x7f442413c200 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:20.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:20.050+0000 7f442b006640 1 --2- 172.21.15.83:0/651382704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44240b2670 0x7f442413a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:20.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:20.050+0000 7f442b006640 1 --2- 172.21.15.83:0/651382704 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f442406d690 0x7f442415d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:20.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:20.050+0000 7f442b006640 1 -- 172.21.15.83:0/651382704 >> 172.21.15.83:0/651382704 conn(0x7f442406abe0 msgr2=0x7f44240ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:20.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:20.050+0000 7f442b006640 1 -- 172.21.15.83:0/651382704 shutdown_connections 2024-04-17T10:52:20.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:20.050+0000 7f442b006640 1 -- 172.21.15.83:0/651382704 wait complete. 2024-04-17T10:52:20.461 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:20.461 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:21.463 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:21.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.602+0000 7f96afd54640 1 Processor -- start 2024-04-17T10:52:21.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.602+0000 7f96afd54640 1 -- start start 2024-04-17T10:52:21.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.602+0000 7f96afd54640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96a80cfca0 0x7f96a8166550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:21.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.602+0000 7f96afd54640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96a8072760 con 0x7f96a80b3f80 2024-04-17T10:52:21.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.602+0000 7f96afd54640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96a80cf200 con 0x7f96a80aecd0 2024-04-17T10:52:21.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.602+0000 7f96afd54640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a80cf810 con 0x7f96a80cfca0 2024-04-17T10:52:21.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.602+0000 7f96ae2f1640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f96a80b3f80 0x7f96a80b2670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39848/0 (socket says 172.21.15.83:39848) 2024-04-17T10:52:21.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.602+0000 7f96ae2f1640 1 -- 172.21.15.83:0/1275154311 learned_addr learned my addr 172.21.15.83:0/1275154311 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:52:21.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.602+0000 7f96ad2ef640 1 --2- 172.21.15.83:0/1275154311 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96a80cfca0 0x7f96a8166550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:21.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.602+0000 7f96acaee640 1 -- 172.21.15.83:0/1275154311 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3049272236 0 0) 0x7f96a8072760 con 0x7f96a80b3f80 2024-04-17T10:52:21.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.602+0000 7f96acaee640 1 -- 172.21.15.83:0/1275154311 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9694003530 con 0x7f96a80b3f80 2024-04-17T10:52:21.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.602+0000 7f96acaee640 1 -- 172.21.15.83:0/1275154311 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 315229885 0 0) 0x7f9694003530 con 0x7f96a80b3f80 2024-04-17T10:52:21.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.602+0000 7f96acaee640 1 -- 172.21.15.83:0/1275154311 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96a80cfca0 msgr2=0x7f96a8166550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:21.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.602+0000 7f96acaee640 1 --2- 172.21.15.83:0/1275154311 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96a80cfca0 0x7f96a8166550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:21.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.602+0000 7f96acaee640 1 -- 172.21.15.83:0/1275154311 >> v1:172.21.15.167:6789/0 conn(0x7f96a80aecd0 legacy=0x7f96a8071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:21.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.602+0000 7f96acaee640 1 -- 172.21.15.83:0/1275154311 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a80af0b0 con 0x7f96a80b3f80 2024-04-17T10:52:21.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.602+0000 7f96acaee640 1 -- 172.21.15.83:0/1275154311 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f969c001680 con 0x7f96a80b3f80 2024-04-17T10:52:21.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.602+0000 7f96acaee640 1 -- 172.21.15.83:0/1275154311 >> v1:172.21.15.83:6789/0 conn(0x7f96a80b3f80 legacy=0x7f96a80b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:21.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.606+0000 7f96acaee640 1 --2- 172.21.15.83:0/1275154311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96940039f0 0x7f9694003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:21.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.606+0000 7f96acaee640 1 --2- 172.21.15.83:0/1275154311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9694004810 0x7f9694004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:21.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.606+0000 7f96acaee640 1 --2- 172.21.15.83:0/1275154311 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96a80cfca0 0x7f9694005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:21.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.606+0000 7f96acaee640 1 -- 172.21.15.83:0/1275154311 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969400bad0 con 0x7f96940039f0 2024-04-17T10:52:21.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.606+0000 7f96acaee640 1 -- 172.21.15.83:0/1275154311 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969400bc80 con 0x7f9694004810 2024-04-17T10:52:21.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.606+0000 7f96acaee640 1 -- 172.21.15.83:0/1275154311 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969400be80 con 0x7f96a80cfca0 2024-04-17T10:52:21.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.606+0000 7f96adaf0640 1 --2- 172.21.15.83:0/1275154311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96940039f0 0x7f9694003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:21.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.606+0000 7f96ae2f1640 1 --2- 172.21.15.83:0/1275154311 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96a80cfca0 0x7f9694005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:21.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.606+0000 7f96ad2ef640 1 --2- 172.21.15.83:0/1275154311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9694004810 0x7f9694004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.606+0000 7f96adaf0640 1 -- 172.21.15.83:0/1275154311 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96a80cfca0 msgr2=0x7f9694005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.606+0000 7f96adaf0640 1 --2- 172.21.15.83:0/1275154311 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96a80cfca0 0x7f9694005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.606+0000 7f96adaf0640 1 -- 172.21.15.83:0/1275154311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9694004810 msgr2=0x7f9694004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.606+0000 7f96adaf0640 1 --2- 172.21.15.83:0/1275154311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9694004810 0x7f9694004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.606+0000 7f96adaf0640 1 -- 172.21.15.83:0/1275154311 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9694003530 con 0x7f96940039f0 2024-04-17T10:52:21.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.606+0000 7f96ae2f1640 1 --2- 172.21.15.83:0/1275154311 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96a80cfca0 0x7f9694005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:52:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.606+0000 7f96adaf0640 1 --2- 172.21.15.83:0/1275154311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96940039f0 0x7f9694003e10 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f96a0001e70 tx=0x7f96a001c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc34ec0f9c1b8cef server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.606+0000 7f96acaee640 1 -- 172.21.15.83:0/1275154311 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a00af070 con 0x7f96940039f0 2024-04-17T10:52:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.606+0000 7f96acaee640 1 -- 172.21.15.83:0/1275154311 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a0003b80 con 0x7f96940039f0 2024-04-17T10:52:21.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.606+0000 7f96acaee640 1 -- 172.21.15.83:0/1275154311 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a0001a90 con 0x7f96940039f0 2024-04-17T10:52:21.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.606+0000 7f96afd54640 1 -- 172.21.15.83:0/1275154311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96940039f0 msgr2=0x7f9694003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:21.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.606+0000 7f96afd54640 1 --2- 172.21.15.83:0/1275154311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96940039f0 0x7f9694003e10 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f96a0001e70 tx=0x7f96a001c780 comp rx=0 tx=0).stop 2024-04-17T10:52:21.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.606+0000 7f96ae2f1640 1 -- 172.21.15.83:0/1275154311 reap_dead start 2024-04-17T10:52:21.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.606+0000 7f96afd54640 1 -- 172.21.15.83:0/1275154311 shutdown_connections 2024-04-17T10:52:21.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.606+0000 7f96afd54640 1 -- 172.21.15.83:0/1275154311 >> 172.21.15.83:0/1275154311 conn(0x7f96a806abe0 msgr2=0x7f96a806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:21.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.606+0000 7f96afd54640 1 -- 172.21.15.83:0/1275154311 shutdown_connections 2024-04-17T10:52:21.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.606+0000 7f96afd54640 1 -- 172.21.15.83:0/1275154311 wait complete. 2024-04-17T10:52:21.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.610+0000 7f96afd54640 1 Processor -- start 2024-04-17T10:52:21.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.610+0000 7f96afd54640 1 -- start start 2024-04-17T10:52:21.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.610+0000 7f96afd54640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96940039f0 0x7f96a8143eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:21.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.610+0000 7f96afd54640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96a80af2e0 0x7f96a80af740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:21.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.610+0000 7f96afd54640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96a80b1150 0x7f96a80b1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:21.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.610+0000 7f96afd54640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a816df20 con 0x7f96a80b1150 2024-04-17T10:52:21.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.610+0000 7f96afd54640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a816dda0 con 0x7f96a80af2e0 2024-04-17T10:52:21.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.610+0000 7f96afd54640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a816e0a0 con 0x7f96940039f0 2024-04-17T10:52:21.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.610+0000 7f96ae2f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96a80b1150 0x7f96a80b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:21.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.610+0000 7f96ae2f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96a80b1150 0x7f96a80b1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52376/0 (socket says 172.21.15.83:52376) 2024-04-17T10:52:21.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.610+0000 7f96ae2f1640 1 -- 172.21.15.83:0/4181980364 learned_addr learned my addr 172.21.15.83:0/4181980364 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:21.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.610+0000 7f96adaf0640 1 --2- 172.21.15.83:0/4181980364 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96940039f0 0x7f96a8143eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:21.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.610+0000 7f96ad2ef640 1 --2- 172.21.15.83:0/4181980364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96a80af2e0 0x7f96a80af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:21.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.610+0000 7f96ae2f1640 1 -- 172.21.15.83:0/4181980364 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96940039f0 msgr2=0x7f96a8143eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:21.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.610+0000 7f96ae2f1640 1 --2- 172.21.15.83:0/4181980364 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96940039f0 0x7f96a8143eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:21.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.610+0000 7f96ae2f1640 1 -- 172.21.15.83:0/4181980364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96a80af2e0 msgr2=0x7f96a80af740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:21.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.610+0000 7f96ae2f1640 1 --2- 172.21.15.83:0/4181980364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96a80af2e0 0x7f96a80af740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:21.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.610+0000 7f96ae2f1640 1 -- 172.21.15.83:0/4181980364 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a80b2770 con 0x7f96a80b1150 2024-04-17T10:52:21.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.610+0000 7f96ad2ef640 1 --2- 172.21.15.83:0/4181980364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96a80af2e0 0x7f96a80af740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:52:21.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.610+0000 7f96ae2f1640 1 --2- 172.21.15.83:0/4181980364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96a80b1150 0x7f96a80b1600 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f96a8070400 tx=0x7f969c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:21.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.610+0000 7f969a7fc640 1 -- 172.21.15.83:0/4181980364 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c007230 con 0x7f96a80b1150 2024-04-17T10:52:21.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.610+0000 7f969a7fc640 1 -- 172.21.15.83:0/4181980364 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969c0073d0 con 0x7f96a80b1150 2024-04-17T10:52:21.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.610+0000 7f96afd54640 1 -- 172.21.15.83:0/4181980364 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96a80b1ed0 con 0x7f96a80b1150 2024-04-17T10:52:21.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.610+0000 7f969a7fc640 1 -- 172.21.15.83:0/4181980364 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c0047f0 con 0x7f96a80b1150 2024-04-17T10:52:21.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.610+0000 7f96afd54640 1 -- 172.21.15.83:0/4181980364 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a80b2490 con 0x7f96a80b1150 2024-04-17T10:52:21.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.610+0000 7f96afd54640 1 -- 172.21.15.83:0/4181980364 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96a8074f90 con 0x7f96a80b1150 2024-04-17T10:52:21.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.614+0000 7f969a7fc640 1 -- 172.21.15.83:0/4181980364 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f969c01b020 con 0x7f96a80b1150 2024-04-17T10:52:21.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.614+0000 7f969a7fc640 1 --2- 172.21.15.83:0/4181980364 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f968c066f60 0x7f968c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:21.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.614+0000 7f969a7fc640 1 -- 172.21.15.83:0/4181980364 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f969c084c30 con 0x7f96a80b1150 2024-04-17T10:52:21.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.614+0000 7f96adaf0640 1 --2- 172.21.15.83:0/4181980364 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f968c066f60 0x7f968c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:21.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.614+0000 7f96adaf0640 1 --2- 172.21.15.83:0/4181980364 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f968c066f60 0x7f968c069420 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f96a00ac000 tx=0x7f96a0001520 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:21.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.618+0000 7f969a7fc640 1 -- 172.21.15.83:0/4181980364 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f969c01b310 con 0x7f96a80b1150 2024-04-17T10:52:21.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.794+0000 7f96afd54640 1 -- 172.21.15.83:0/4181980364 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96a806b340 con 0x7f96a80b1150 2024-04-17T10:52:21.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.798+0000 7f969a7fc640 1 -- 172.21.15.83:0/4181980364 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f969c055bd0 con 0x7f96a80b1150 2024-04-17T10:52:21.791 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:21.791 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:21.792 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:21.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.798+0000 7f96afd54640 1 -- 172.21.15.83:0/4181980364 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f968c066f60 msgr2=0x7f968c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.798+0000 7f96afd54640 1 --2- 172.21.15.83:0/4181980364 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f968c066f60 0x7f968c069420 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f96a00ac000 tx=0x7f96a0001520 comp rx=0 tx=0).stop 2024-04-17T10:52:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.798+0000 7f96afd54640 1 -- 172.21.15.83:0/4181980364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96a80b1150 msgr2=0x7f96a80b1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.798+0000 7f96afd54640 1 --2- 172.21.15.83:0/4181980364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96a80b1150 0x7f96a80b1600 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f96a8070400 tx=0x7f969c00dc00 comp rx=0 tx=0).stop 2024-04-17T10:52:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.798+0000 7f96afd54640 1 -- 172.21.15.83:0/4181980364 shutdown_connections 2024-04-17T10:52:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.798+0000 7f96afd54640 1 --2- 172.21.15.83:0/4181980364 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f968c066f60 0x7f968c069420 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.798+0000 7f96afd54640 1 --2- 172.21.15.83:0/4181980364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96a80b1150 0x7f96a80b1600 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.798+0000 7f96afd54640 1 --2- 172.21.15.83:0/4181980364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96a80af2e0 0x7f96a80af740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.798+0000 7f96afd54640 1 --2- 172.21.15.83:0/4181980364 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96940039f0 0x7f96a8143eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.798+0000 7f96afd54640 1 -- 172.21.15.83:0/4181980364 >> 172.21.15.83:0/4181980364 conn(0x7f96a806abe0 msgr2=0x7f96a815c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.798+0000 7f96afd54640 1 -- 172.21.15.83:0/4181980364 shutdown_connections 2024-04-17T10:52:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:21.798+0000 7f96afd54640 1 -- 172.21.15.83:0/4181980364 wait complete. 2024-04-17T10:52:22.212 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:22.212 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:23.214 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:23.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.354+0000 7f242fe53640 1 Processor -- start 2024-04-17T10:52:23.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.354+0000 7f242fe53640 1 -- start start 2024-04-17T10:52:23.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.354+0000 7f242fe53640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242815ba30 0x7f2428072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:23.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.354+0000 7f242fe53640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f242806e320 con 0x7f24280cfbe0 2024-04-17T10:52:23.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.354+0000 7f242fe53640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f242806e040 con 0x7f242815c960 2024-04-17T10:52:23.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.354+0000 7f242fe53640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2428071ea0 con 0x7f242815ba30 2024-04-17T10:52:23.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.354+0000 7f242e3f0640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f24280cfbe0 0x7f242806c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39852/0 (socket says 172.21.15.83:39852) 2024-04-17T10:52:23.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.354+0000 7f242e3f0640 1 -- 172.21.15.83:0/2723539066 learned_addr learned my addr 172.21.15.83:0/2723539066 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:52:23.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.354+0000 7f242dbef640 1 --2- 172.21.15.83:0/2723539066 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242815ba30 0x7f2428072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:23.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.354+0000 7f242cbed640 1 -- 172.21.15.83:0/2723539066 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3543620915 0 0) 0x7f242806e320 con 0x7f24280cfbe0 2024-04-17T10:52:23.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.354+0000 7f242cbed640 1 -- 172.21.15.83:0/2723539066 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2414003530 con 0x7f24280cfbe0 2024-04-17T10:52:23.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.354+0000 7f242cbed640 1 -- 172.21.15.83:0/2723539066 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 469532830 0 0) 0x7f2414003530 con 0x7f24280cfbe0 2024-04-17T10:52:23.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.354+0000 7f242cbed640 1 -- 172.21.15.83:0/2723539066 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242815ba30 msgr2=0x7f2428072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:23.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.354+0000 7f242cbed640 1 --2- 172.21.15.83:0/2723539066 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242815ba30 0x7f2428072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:23.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.354+0000 7f242cbed640 1 -- 172.21.15.83:0/2723539066 >> v1:172.21.15.167:6789/0 conn(0x7f242815c960 legacy=0x7f24280cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:23.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.354+0000 7f242cbed640 1 -- 172.21.15.83:0/2723539066 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2428071850 con 0x7f24280cfbe0 2024-04-17T10:52:23.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.354+0000 7f242cbed640 1 -- 172.21.15.83:0/2723539066 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f2420001a20 con 0x7f24280cfbe0 2024-04-17T10:52:23.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.354+0000 7f242cbed640 1 -- 172.21.15.83:0/2723539066 >> v1:172.21.15.83:6789/0 conn(0x7f24280cfbe0 legacy=0x7f242806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:23.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.354+0000 7f242cbed640 1 --2- 172.21.15.83:0/2723539066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24140039f0 0x7f2414003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:23.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.354+0000 7f242cbed640 1 --2- 172.21.15.83:0/2723539066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2414004810 0x7f2414004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:23.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.354+0000 7f242cbed640 1 --2- 172.21.15.83:0/2723539066 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242815ba30 0x7f241400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:23.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.354+0000 7f242cbed640 1 -- 172.21.15.83:0/2723539066 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2428071ea0 con 0x7f2414004810 2024-04-17T10:52:23.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.354+0000 7f242cbed640 1 -- 172.21.15.83:0/2723539066 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2414005530 con 0x7f24140039f0 2024-04-17T10:52:23.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.354+0000 7f242cbed640 1 -- 172.21.15.83:0/2723539066 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f241400bf10 con 0x7f242815ba30 2024-04-17T10:52:23.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.354+0000 7f242d3ee640 1 --2- 172.21.15.83:0/2723539066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2414004810 0x7f2414004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:23.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.354+0000 7f242dbef640 1 --2- 172.21.15.83:0/2723539066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24140039f0 0x7f2414003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:23.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.358+0000 7f242d3ee640 1 -- 172.21.15.83:0/2723539066 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242815ba30 msgr2=0x7f241400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:23.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.358+0000 7f242d3ee640 1 --2- 172.21.15.83:0/2723539066 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242815ba30 0x7f241400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:23.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.358+0000 7f242d3ee640 1 -- 172.21.15.83:0/2723539066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24140039f0 msgr2=0x7f2414003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:23.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.358+0000 7f242d3ee640 1 --2- 172.21.15.83:0/2723539066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24140039f0 0x7f2414003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:23.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.358+0000 7f242d3ee640 1 -- 172.21.15.83:0/2723539066 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2414003530 con 0x7f2414004810 2024-04-17T10:52:23.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.358+0000 7f242dbef640 1 --2- 172.21.15.83:0/2723539066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24140039f0 0x7f2414003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:52:23.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.358+0000 7f242d3ee640 1 --2- 172.21.15.83:0/2723539066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2414004810 0x7f2414004c90 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7f2418001780 tx=0x7f242806d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0c3e38b564a911 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:23.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.358+0000 7f242cbed640 1 -- 172.21.15.83:0/2723539066 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24180013a0 con 0x7f2414004810 2024-04-17T10:52:23.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.358+0000 7f242cbed640 1 -- 172.21.15.83:0/2723539066 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24180052f0 con 0x7f2414004810 2024-04-17T10:52:23.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.358+0000 7f242cbed640 1 -- 172.21.15.83:0/2723539066 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418003840 con 0x7f2414004810 2024-04-17T10:52:23.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.358+0000 7f242fe53640 1 -- 172.21.15.83:0/2723539066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2414004810 msgr2=0x7f2414004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:23.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.358+0000 7f242fe53640 1 --2- 172.21.15.83:0/2723539066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2414004810 0x7f2414004c90 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7f2418001780 tx=0x7f242806d3b0 comp rx=0 tx=0).stop 2024-04-17T10:52:23.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.358+0000 7f242e3f0640 1 -- 172.21.15.83:0/2723539066 reap_dead start 2024-04-17T10:52:23.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.358+0000 7f242fe53640 1 -- 172.21.15.83:0/2723539066 shutdown_connections 2024-04-17T10:52:23.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.358+0000 7f242fe53640 1 -- 172.21.15.83:0/2723539066 >> 172.21.15.83:0/2723539066 conn(0x7f242806abe0 msgr2=0x7f242806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:23.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.358+0000 7f242fe53640 1 -- 172.21.15.83:0/2723539066 shutdown_connections 2024-04-17T10:52:23.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.358+0000 7f242fe53640 1 -- 172.21.15.83:0/2723539066 wait complete. 2024-04-17T10:52:23.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.362+0000 7f242fe53640 1 Processor -- start 2024-04-17T10:52:23.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.362+0000 7f242fe53640 1 -- start start 2024-04-17T10:52:23.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.362+0000 7f242fe53640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2414004810 0x7f24280cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:23.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.362+0000 7f242fe53640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24281530e0 0x7f24281528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:23.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.362+0000 7f242fe53640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24280cf6e0 0x7f2428159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:23.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.362+0000 7f242fe53640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242815cee0 con 0x7f24280cf6e0 2024-04-17T10:52:23.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.362+0000 7f242fe53640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242815cd60 con 0x7f24281530e0 2024-04-17T10:52:23.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.362+0000 7f242fe53640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24280b2770 con 0x7f2414004810 2024-04-17T10:52:23.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.362+0000 7f242e3f0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24280cf6e0 0x7f2428159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:23.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.362+0000 7f242e3f0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24280cf6e0 0x7f2428159460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52388/0 (socket says 172.21.15.83:52388) 2024-04-17T10:52:23.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.362+0000 7f242e3f0640 1 -- 172.21.15.83:0/448553296 learned_addr learned my addr 172.21.15.83:0/448553296 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:23.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.362+0000 7f242d3ee640 1 --2- 172.21.15.83:0/448553296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24281530e0 0x7f24281528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:23.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.362+0000 7f242dbef640 1 --2- 172.21.15.83:0/448553296 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2414004810 0x7f24280cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:23.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.362+0000 7f242e3f0640 1 -- 172.21.15.83:0/448553296 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2414004810 msgr2=0x7f24280cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:23.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.362+0000 7f242e3f0640 1 --2- 172.21.15.83:0/448553296 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2414004810 0x7f24280cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:23.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.362+0000 7f242e3f0640 1 -- 172.21.15.83:0/448553296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24281530e0 msgr2=0x7f24281528a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:23.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.362+0000 7f242e3f0640 1 --2- 172.21.15.83:0/448553296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24281530e0 0x7f24281528a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:23.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.362+0000 7f242e3f0640 1 -- 172.21.15.83:0/448553296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24280ce640 con 0x7f24280cf6e0 2024-04-17T10:52:23.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.362+0000 7f242e3f0640 1 --2- 172.21.15.83:0/448553296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24280cf6e0 0x7f2428159460 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f24280676a0 tx=0x7f242000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:23.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.362+0000 7f241e7fc640 1 -- 172.21.15.83:0/448553296 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242000a610 con 0x7f24280cf6e0 2024-04-17T10:52:23.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.362+0000 7f241e7fc640 1 -- 172.21.15.83:0/448553296 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f242000a7d0 con 0x7f24280cf6e0 2024-04-17T10:52:23.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.362+0000 7f242fe53640 1 -- 172.21.15.83:0/448553296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24280ce920 con 0x7f24280cf6e0 2024-04-17T10:52:23.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.362+0000 7f241e7fc640 1 -- 172.21.15.83:0/448553296 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242000abc0 con 0x7f24280cf6e0 2024-04-17T10:52:23.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.362+0000 7f242fe53640 1 -- 172.21.15.83:0/448553296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24280ceee0 con 0x7f24280cf6e0 2024-04-17T10:52:23.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.362+0000 7f242fe53640 1 -- 172.21.15.83:0/448553296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2428074f90 con 0x7f24280cf6e0 2024-04-17T10:52:23.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.362+0000 7f241e7fc640 1 -- 172.21.15.83:0/448553296 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2420011070 con 0x7f24280cf6e0 2024-04-17T10:52:23.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.366+0000 7f241e7fc640 1 --2- 172.21.15.83:0/448553296 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f240c066f60 0x7f240c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:23.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.366+0000 7f241e7fc640 1 -- 172.21.15.83:0/448553296 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f2420084b50 con 0x7f24280cf6e0 2024-04-17T10:52:23.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.366+0000 7f242dbef640 1 --2- 172.21.15.83:0/448553296 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f240c066f60 0x7f240c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:23.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.366+0000 7f242dbef640 1 --2- 172.21.15.83:0/448553296 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f240c066f60 0x7f240c069420 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f242400ecc0 tx=0x7f2424008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:23.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.370+0000 7f241e7fc640 1 -- 172.21.15.83:0/448553296 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f242001a060 con 0x7f24280cf6e0 2024-04-17T10:52:23.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.554+0000 7f242fe53640 1 -- 172.21.15.83:0/448553296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24280b34b0 con 0x7f24280cf6e0 2024-04-17T10:52:23.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.554+0000 7f241e7fc640 1 -- 172.21.15.83:0/448553296 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f242000f260 con 0x7f24280cf6e0 2024-04-17T10:52:23.548 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:23.548 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:23.548 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:23.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.554+0000 7f242fe53640 1 -- 172.21.15.83:0/448553296 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f240c066f60 msgr2=0x7f240c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:23.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.554+0000 7f242fe53640 1 --2- 172.21.15.83:0/448553296 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f240c066f60 0x7f240c069420 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f242400ecc0 tx=0x7f2424008040 comp rx=0 tx=0).stop 2024-04-17T10:52:23.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.554+0000 7f242fe53640 1 -- 172.21.15.83:0/448553296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24280cf6e0 msgr2=0x7f2428159460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:23.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.554+0000 7f242fe53640 1 --2- 172.21.15.83:0/448553296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24280cf6e0 0x7f2428159460 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f24280676a0 tx=0x7f242000ccd0 comp rx=0 tx=0).stop 2024-04-17T10:52:23.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.554+0000 7f242fe53640 1 -- 172.21.15.83:0/448553296 shutdown_connections 2024-04-17T10:52:23.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.554+0000 7f242fe53640 1 --2- 172.21.15.83:0/448553296 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f240c066f60 0x7f240c069420 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:23.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.554+0000 7f242fe53640 1 --2- 172.21.15.83:0/448553296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24280cf6e0 0x7f2428159460 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:23.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.554+0000 7f242fe53640 1 --2- 172.21.15.83:0/448553296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24281530e0 0x7f24281528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:23.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.554+0000 7f242fe53640 1 --2- 172.21.15.83:0/448553296 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2414004810 0x7f24280cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:23.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.554+0000 7f242fe53640 1 -- 172.21.15.83:0/448553296 >> 172.21.15.83:0/448553296 conn(0x7f242806abe0 msgr2=0x7f242806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:23.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.554+0000 7f242fe53640 1 -- 172.21.15.83:0/448553296 shutdown_connections 2024-04-17T10:52:23.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:23.554+0000 7f242fe53640 1 -- 172.21.15.83:0/448553296 wait complete. 2024-04-17T10:52:23.969 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:23.969 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:24.970 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:25.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.110+0000 7f9843101640 1 Processor -- start 2024-04-17T10:52:25.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.110+0000 7f9843101640 1 -- start start 2024-04-17T10:52:25.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.110+0000 7f9843101640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f983c06c300 0x7f983c0cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:25.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.110+0000 7f9843101640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f983c15e050 con 0x7f983c072c60 2024-04-17T10:52:25.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.110+0000 7f9843101640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f983c15e330 con 0x7f983c15dbc0 2024-04-17T10:52:25.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.110+0000 7f9843101640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f983c15e610 con 0x7f983c06c300 2024-04-17T10:52:25.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.110+0000 7f9840e9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f983c072c60 0x7f983c06bbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39862/0 (socket says 172.21.15.83:39862) 2024-04-17T10:52:25.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.110+0000 7f9840e9d640 1 -- 172.21.15.83:0/2140182122 learned_addr learned my addr 172.21.15.83:0/2140182122 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:52:25.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.110+0000 7f984169e640 1 --2- 172.21.15.83:0/2140182122 >> v2:172.21.15.83:3300/0 conn(0x7f983c06c300 0x7f983c0cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:25.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.110+0000 7f984169e640 1 -- 172.21.15.83:0/2140182122 >> v1:172.21.15.167:6789/0 conn(0x7f983c15dbc0 legacy=0x7f983c15ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:25.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.110+0000 7f984169e640 1 -- 172.21.15.83:0/2140182122 >> v1:172.21.15.83:6789/0 conn(0x7f983c072c60 legacy=0x7f983c06bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:25.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.110+0000 7f984169e640 1 -- 172.21.15.83:0/2140182122 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983c15e8f0 con 0x7f983c06c300 2024-04-17T10:52:25.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.110+0000 7f983b7fe640 1 -- 172.21.15.83:0/2140182122 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251227833 0 0) 0x7f983c15e050 con 0x7f983c072c60 2024-04-17T10:52:25.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.114+0000 7f984169e640 1 --2- 172.21.15.83:0/2140182122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f983c06c300 0x7f983c0cf850 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f982c0097c0 tx=0x7f982c01c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b2183b93e53fce server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:25.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.114+0000 7f983b7fe640 1 -- 172.21.15.83:0/2140182122 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c0b0070 con 0x7f983c06c300 2024-04-17T10:52:25.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.114+0000 7f983b7fe640 1 -- 172.21.15.83:0/2140182122 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982c004230 con 0x7f983c06c300 2024-04-17T10:52:25.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.114+0000 7f983b7fe640 1 -- 172.21.15.83:0/2140182122 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c003620 con 0x7f983c06c300 2024-04-17T10:52:25.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.114+0000 7f9843101640 1 -- 172.21.15.83:0/2140182122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f983c06c300 msgr2=0x7f983c0cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:25.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.114+0000 7f9843101640 1 --2- 172.21.15.83:0/2140182122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f983c06c300 0x7f983c0cf850 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f982c0097c0 tx=0x7f982c01c370 comp rx=0 tx=0).stop 2024-04-17T10:52:25.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.114+0000 7f9843101640 1 -- 172.21.15.83:0/2140182122 shutdown_connections 2024-04-17T10:52:25.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.114+0000 7f9843101640 1 --2- 172.21.15.83:0/2140182122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f983c06c300 0x7f983c0cf850 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:25.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.114+0000 7f9843101640 1 -- 172.21.15.83:0/2140182122 >> 172.21.15.83:0/2140182122 conn(0x7f983c0cc910 msgr2=0x7f983c0ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:25.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.114+0000 7f9843101640 1 -- 172.21.15.83:0/2140182122 shutdown_connections 2024-04-17T10:52:25.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.114+0000 7f9843101640 1 -- 172.21.15.83:0/2140182122 wait complete. 2024-04-17T10:52:25.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.114+0000 7f9843101640 1 Processor -- start 2024-04-17T10:52:25.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.114+0000 7f9843101640 1 -- start start 2024-04-17T10:52:25.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.114+0000 7f9843101640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f983c06c300 0x7f983c0c4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:25.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.114+0000 7f9843101640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f983c072c60 0x7f983c1597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:25.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.114+0000 7f9843101640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f983c06ed00 0x7f983c06f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:25.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.114+0000 7f9843101640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983c15b7f0 con 0x7f983c06c300 2024-04-17T10:52:25.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.114+0000 7f9843101640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983c15b670 con 0x7f983c072c60 2024-04-17T10:52:25.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.114+0000 7f9843101640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f983c169030 con 0x7f983c06ed00 2024-04-17T10:52:25.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.114+0000 7f9840e9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f983c06c300 0x7f983c0c4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:25.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.114+0000 7f9840e9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f983c06c300 0x7f983c0c4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52416/0 (socket says 172.21.15.83:52416) 2024-04-17T10:52:25.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.114+0000 7f9840e9d640 1 -- 172.21.15.83:0/2874727422 learned_addr learned my addr 172.21.15.83:0/2874727422 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:25.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.114+0000 7f983bfff640 1 --2- 172.21.15.83:0/2874727422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f983c072c60 0x7f983c1597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:25.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.114+0000 7f984169e640 1 --2- 172.21.15.83:0/2874727422 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f983c06ed00 0x7f983c06f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:25.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.114+0000 7f9840e9d640 1 -- 172.21.15.83:0/2874727422 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f983c06ed00 msgr2=0x7f983c06f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:25.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.114+0000 7f9840e9d640 1 --2- 172.21.15.83:0/2874727422 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f983c06ed00 0x7f983c06f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:25.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.114+0000 7f9840e9d640 1 -- 172.21.15.83:0/2874727422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f983c072c60 msgr2=0x7f983c1597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:25.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.114+0000 7f9840e9d640 1 --2- 172.21.15.83:0/2874727422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f983c072c60 0x7f983c1597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:25.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.114+0000 7f9840e9d640 1 -- 172.21.15.83:0/2874727422 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983c15d710 con 0x7f983c06c300 2024-04-17T10:52:25.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.114+0000 7f9840e9d640 1 --2- 172.21.15.83:0/2874727422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f983c06c300 0x7f983c0c4090 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f983400c890 tx=0x7f983400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:25.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.118+0000 7f98397fa640 1 -- 172.21.15.83:0/2874727422 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9834018070 con 0x7f983c06c300 2024-04-17T10:52:25.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.118+0000 7f98397fa640 1 -- 172.21.15.83:0/2874727422 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f983400a640 con 0x7f983c06c300 2024-04-17T10:52:25.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.118+0000 7f98397fa640 1 -- 172.21.15.83:0/2874727422 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983400aa30 con 0x7f983c06c300 2024-04-17T10:52:25.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.118+0000 7f9843101640 1 -- 172.21.15.83:0/2874727422 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f983c09ef10 con 0x7f983c06c300 2024-04-17T10:52:25.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.118+0000 7f9843101640 1 -- 172.21.15.83:0/2874727422 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f983c0b2af0 con 0x7f983c06c300 2024-04-17T10:52:25.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.118+0000 7f9843101640 1 -- 172.21.15.83:0/2874727422 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f983c074f90 con 0x7f983c06c300 2024-04-17T10:52:25.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.118+0000 7f98397fa640 1 -- 172.21.15.83:0/2874727422 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f983401b070 con 0x7f983c06c300 2024-04-17T10:52:25.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.118+0000 7f98397fa640 1 --2- 172.21.15.83:0/2874727422 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9828066f60 0x7f9828069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:25.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.118+0000 7f98397fa640 1 -- 172.21.15.83:0/2874727422 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f9834084710 con 0x7f983c06c300 2024-04-17T10:52:25.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.118+0000 7f983bfff640 1 --2- 172.21.15.83:0/2874727422 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9828066f60 0x7f9828069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:25.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.118+0000 7f983bfff640 1 --2- 172.21.15.83:0/2874727422 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9828066f60 0x7f9828069420 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f983c09ed00 tx=0x7f9830008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:25.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.122+0000 7f98397fa640 1 -- 172.21.15.83:0/2874727422 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f983c09ef10 con 0x7f983c06c300 2024-04-17T10:52:25.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.318+0000 7f9843101640 1 -- 172.21.15.83:0/2874727422 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f983c068150 con 0x7f983c06c300 2024-04-17T10:52:25.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.318+0000 7f98397fa640 1 -- 172.21.15.83:0/2874727422 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f98340556b0 con 0x7f983c06c300 2024-04-17T10:52:25.313 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:25.313 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:25.313 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:25.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.318+0000 7f9843101640 1 -- 172.21.15.83:0/2874727422 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9828066f60 msgr2=0x7f9828069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:25.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.318+0000 7f9843101640 1 --2- 172.21.15.83:0/2874727422 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9828066f60 0x7f9828069420 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f983c09ed00 tx=0x7f9830008040 comp rx=0 tx=0).stop 2024-04-17T10:52:25.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.318+0000 7f9843101640 1 -- 172.21.15.83:0/2874727422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f983c06c300 msgr2=0x7f983c0c4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:25.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.318+0000 7f9843101640 1 --2- 172.21.15.83:0/2874727422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f983c06c300 0x7f983c0c4090 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f983400c890 tx=0x7f983400cd50 comp rx=0 tx=0).stop 2024-04-17T10:52:25.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.318+0000 7f9843101640 1 -- 172.21.15.83:0/2874727422 shutdown_connections 2024-04-17T10:52:25.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.318+0000 7f9843101640 1 --2- 172.21.15.83:0/2874727422 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9828066f60 0x7f9828069420 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:25.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.318+0000 7f9843101640 1 --2- 172.21.15.83:0/2874727422 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f983c06ed00 0x7f983c06f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:25.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.318+0000 7f9843101640 1 --2- 172.21.15.83:0/2874727422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f983c072c60 0x7f983c1597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:25.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.318+0000 7f9843101640 1 --2- 172.21.15.83:0/2874727422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f983c06c300 0x7f983c0c4090 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:25.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.318+0000 7f9843101640 1 -- 172.21.15.83:0/2874727422 >> 172.21.15.83:0/2874727422 conn(0x7f983c0cc910 msgr2=0x7f983c1571c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:25.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.318+0000 7f9843101640 1 -- 172.21.15.83:0/2874727422 shutdown_connections 2024-04-17T10:52:25.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:25.318+0000 7f9843101640 1 -- 172.21.15.83:0/2874727422 wait complete. 2024-04-17T10:52:25.733 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:25.733 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:26.734 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:26.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.870+0000 7f844aecc640 1 Processor -- start 2024-04-17T10:52:26.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.870+0000 7f844aecc640 1 -- start start 2024-04-17T10:52:26.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.870+0000 7f844aecc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f844415ba30 0x7f8444072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:26.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.870+0000 7f844aecc640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f844415c960 0x7f84440cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:26.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.870+0000 7f844aecc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f84440cfa10 0x7f844406c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:26.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.870+0000 7f844aecc640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8444071ea0 con 0x7f84440cfa10 2024-04-17T10:52:26.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.870+0000 7f844aecc640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84440cfea0 con 0x7f844415c960 2024-04-17T10:52:26.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.870+0000 7f844aecc640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f844406e320 con 0x7f844415ba30 2024-04-17T10:52:26.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.870+0000 7f8449469640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f84440cfa10 0x7f844406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:26.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.870+0000 7f8449469640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f84440cfa10 0x7f844406c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44530/0 (socket says 172.21.15.83:44530) 2024-04-17T10:52:26.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.870+0000 7f8449469640 1 -- 172.21.15.83:0/2166704420 learned_addr learned my addr 172.21.15.83:0/2166704420 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:26.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.870+0000 7f8443fff640 1 --2- 172.21.15.83:0/2166704420 >> v2:172.21.15.167:3300/0 conn(0x7f844415c960 0x7f84440cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:26.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.870+0000 7f8448c68640 1 --2- 172.21.15.83:0/2166704420 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f844415ba30 0x7f8444072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:26.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.870+0000 7f8449469640 1 -- 172.21.15.83:0/2166704420 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f844415ba30 msgr2=0x7f8444072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:26.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.870+0000 7f8449469640 1 --2- 172.21.15.83:0/2166704420 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f844415ba30 0x7f8444072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:26.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.870+0000 7f8449469640 1 -- 172.21.15.83:0/2166704420 >> v2:172.21.15.167:3300/0 conn(0x7f844415c960 msgr2=0x7f84440cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:26.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.870+0000 7f8449469640 1 --2- 172.21.15.83:0/2166704420 >> v2:172.21.15.167:3300/0 conn(0x7f844415c960 0x7f84440cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:26.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.870+0000 7f8449469640 1 -- 172.21.15.83:0/2166704420 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f844406e040 con 0x7f84440cfa10 2024-04-17T10:52:26.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.874+0000 7f8449469640 1 --2- 172.21.15.83:0/2166704420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84440cfa10 0x7f844406c490 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f8444070060 tx=0x7f843c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb97b8b4bd98dc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:26.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.874+0000 7f84437fe640 1 -- 172.21.15.83:0/2166704420 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843c0b0070 con 0x7f84440cfa10 2024-04-17T10:52:26.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.874+0000 7f84437fe640 1 -- 172.21.15.83:0/2166704420 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f843c0ab600 con 0x7f84440cfa10 2024-04-17T10:52:26.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.874+0000 7f84437fe640 1 -- 172.21.15.83:0/2166704420 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843c0ab9d0 con 0x7f84440cfa10 2024-04-17T10:52:26.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.874+0000 7f844aecc640 1 -- 172.21.15.83:0/2166704420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84440cfa10 msgr2=0x7f844406c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:26.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.874+0000 7f844aecc640 1 --2- 172.21.15.83:0/2166704420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84440cfa10 0x7f844406c490 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f8444070060 tx=0x7f843c01c490 comp rx=0 tx=0).stop 2024-04-17T10:52:26.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.874+0000 7f844aecc640 1 -- 172.21.15.83:0/2166704420 shutdown_connections 2024-04-17T10:52:26.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.874+0000 7f844aecc640 1 --2- 172.21.15.83:0/2166704420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84440cfa10 0x7f844406c490 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:26.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.874+0000 7f844aecc640 1 --2- 172.21.15.83:0/2166704420 >> v2:172.21.15.167:3300/0 conn(0x7f844415c960 0x7f84440cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:26.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.874+0000 7f844aecc640 1 --2- 172.21.15.83:0/2166704420 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f844415ba30 0x7f8444072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:26.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.874+0000 7f844aecc640 1 -- 172.21.15.83:0/2166704420 >> 172.21.15.83:0/2166704420 conn(0x7f844406abe0 msgr2=0x7f844406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:26.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.874+0000 7f844aecc640 1 -- 172.21.15.83:0/2166704420 shutdown_connections 2024-04-17T10:52:26.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.874+0000 7f844aecc640 1 -- 172.21.15.83:0/2166704420 wait complete. 2024-04-17T10:52:26.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.874+0000 7f844aecc640 1 Processor -- start 2024-04-17T10:52:26.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.874+0000 7f844aecc640 1 -- start start 2024-04-17T10:52:26.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.878+0000 7f844aecc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84440cfa10 0x7f8444143fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:26.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.878+0000 7f844aecc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f844415ba30 0x7f84440ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:26.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.878+0000 7f844aecc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f844415c960 0x7f844414fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:26.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.878+0000 7f844aecc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844406de20 con 0x7f844415c960 2024-04-17T10:52:26.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.878+0000 7f844aecc640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844406dca0 con 0x7f84440cfa10 2024-04-17T10:52:26.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.878+0000 7f844aecc640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f844415cd40 con 0x7f844415ba30 2024-04-17T10:52:26.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.878+0000 7f8449469640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f844415c960 0x7f844414fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:26.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.878+0000 7f8449469640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f844415c960 0x7f844414fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44542/0 (socket says 172.21.15.83:44542) 2024-04-17T10:52:26.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.878+0000 7f8449469640 1 -- 172.21.15.83:0/2284587714 learned_addr learned my addr 172.21.15.83:0/2284587714 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:26.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.878+0000 7f8443fff640 1 --2- 172.21.15.83:0/2284587714 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f844415ba30 0x7f84440ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:26.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.878+0000 7f8448c68640 1 --2- 172.21.15.83:0/2284587714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84440cfa10 0x7f8444143fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:26.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.878+0000 7f8449469640 1 -- 172.21.15.83:0/2284587714 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f844415ba30 msgr2=0x7f84440ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:26.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.878+0000 7f8449469640 1 --2- 172.21.15.83:0/2284587714 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f844415ba30 0x7f84440ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:26.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.878+0000 7f8449469640 1 -- 172.21.15.83:0/2284587714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84440cfa10 msgr2=0x7f8444143fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:26.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.878+0000 7f8449469640 1 --2- 172.21.15.83:0/2284587714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84440cfa10 0x7f8444143fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:26.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.878+0000 7f8449469640 1 -- 172.21.15.83:0/2284587714 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8444144510 con 0x7f844415c960 2024-04-17T10:52:26.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.878+0000 7f8449469640 1 --2- 172.21.15.83:0/2284587714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f844415c960 0x7f844414fe30 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f843c01ab40 tx=0x7f843c019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:26.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.878+0000 7f84417fa640 1 -- 172.21.15.83:0/2284587714 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843c0bb070 con 0x7f844415c960 2024-04-17T10:52:26.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.878+0000 7f84417fa640 1 -- 172.21.15.83:0/2284587714 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f843c0b6440 con 0x7f844415c960 2024-04-17T10:52:26.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.878+0000 7f844aecc640 1 -- 172.21.15.83:0/2284587714 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84440b40e0 con 0x7f844415c960 2024-04-17T10:52:26.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.878+0000 7f84417fa640 1 -- 172.21.15.83:0/2284587714 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843c0b0040 con 0x7f844415c960 2024-04-17T10:52:26.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.878+0000 7f844aecc640 1 -- 172.21.15.83:0/2284587714 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84440b46a0 con 0x7f844415c960 2024-04-17T10:52:26.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.878+0000 7f844aecc640 1 -- 172.21.15.83:0/2284587714 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8444074f90 con 0x7f844415c960 2024-04-17T10:52:26.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.882+0000 7f84417fa640 1 -- 172.21.15.83:0/2284587714 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f843c0be070 con 0x7f844415c960 2024-04-17T10:52:26.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.882+0000 7f84417fa640 1 --2- 172.21.15.83:0/2284587714 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8424066f90 0x7f8424069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:26.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.882+0000 7f84417fa640 1 -- 172.21.15.83:0/2284587714 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f843c11e960 con 0x7f844415c960 2024-04-17T10:52:26.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.882+0000 7f8448c68640 1 --2- 172.21.15.83:0/2284587714 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8424066f90 0x7f8424069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:26.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.882+0000 7f8448c68640 1 --2- 172.21.15.83:0/2284587714 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8424066f90 0x7f8424069450 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f8434009f80 tx=0x7f843400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:26.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:26.886+0000 7f84417fa640 1 -- 172.21.15.83:0/2284587714 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f843c016110 con 0x7f844415c960 2024-04-17T10:52:27.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:27.082+0000 7f844aecc640 1 -- 172.21.15.83:0/2284587714 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84440710d0 con 0x7f844415c960 2024-04-17T10:52:27.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:27.086+0000 7f84417fa640 1 -- 172.21.15.83:0/2284587714 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f843c0ef900 con 0x7f844415c960 2024-04-17T10:52:27.079 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:27.080 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:27.080 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:27.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:27.086+0000 7f844aecc640 1 -- 172.21.15.83:0/2284587714 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8424066f90 msgr2=0x7f8424069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:27.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:27.086+0000 7f844aecc640 1 --2- 172.21.15.83:0/2284587714 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8424066f90 0x7f8424069450 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f8434009f80 tx=0x7f843400b040 comp rx=0 tx=0).stop 2024-04-17T10:52:27.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:27.086+0000 7f844aecc640 1 -- 172.21.15.83:0/2284587714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f844415c960 msgr2=0x7f844414fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:27.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:27.086+0000 7f844aecc640 1 --2- 172.21.15.83:0/2284587714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f844415c960 0x7f844414fe30 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f843c01ab40 tx=0x7f843c019230 comp rx=0 tx=0).stop 2024-04-17T10:52:27.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:27.086+0000 7f844aecc640 1 -- 172.21.15.83:0/2284587714 shutdown_connections 2024-04-17T10:52:27.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:27.086+0000 7f844aecc640 1 --2- 172.21.15.83:0/2284587714 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8424066f90 0x7f8424069450 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:27.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:27.086+0000 7f844aecc640 1 --2- 172.21.15.83:0/2284587714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f844415c960 0x7f844414fe30 unknown :-1 s=CLOSED pgs=2223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:27.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:27.086+0000 7f844aecc640 1 --2- 172.21.15.83:0/2284587714 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f844415ba30 0x7f84440ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:27.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:27.086+0000 7f844aecc640 1 --2- 172.21.15.83:0/2284587714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84440cfa10 0x7f8444143fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:27.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:27.086+0000 7f844aecc640 1 -- 172.21.15.83:0/2284587714 >> 172.21.15.83:0/2284587714 conn(0x7f844406abe0 msgr2=0x7f8444152280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:27.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:27.086+0000 7f844aecc640 1 -- 172.21.15.83:0/2284587714 shutdown_connections 2024-04-17T10:52:27.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:27.086+0000 7f844aecc640 1 -- 172.21.15.83:0/2284587714 wait complete. 2024-04-17T10:52:27.502 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:27.503 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:28.504 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:28.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.642+0000 7fb7dc97b640 1 Processor -- start 2024-04-17T10:52:28.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.642+0000 7fb7dc97b640 1 -- start start 2024-04-17T10:52:28.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.642+0000 7fb7dc97b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb7d80aecd0 0x7fb7d80cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:28.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.642+0000 7fb7dc97b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb7d8166550 0x7fb7d8071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:28.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.642+0000 7fb7dc97b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb7d80b3f80 0x7fb7d80b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:28.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.642+0000 7fb7dc97b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7d80cf810 con 0x7fb7d8166550 2024-04-17T10:52:28.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.642+0000 7fb7dc97b640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7d80b4950 con 0x7fb7d80aecd0 2024-04-17T10:52:28.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.642+0000 7fb7dc97b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d81669b0 con 0x7fb7d80b3f80 2024-04-17T10:52:28.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.642+0000 7fb7d5d9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb7d8166550 0x7fb7d8071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:28.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.642+0000 7fb7d5d9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb7d8166550 0x7fb7d8071be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44550/0 (socket says 172.21.15.83:44550) 2024-04-17T10:52:28.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.642+0000 7fb7d5d9b640 1 -- 172.21.15.83:0/4115556057 learned_addr learned my addr 172.21.15.83:0/4115556057 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:28.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.642+0000 7fb7d6d9d640 1 --2- 172.21.15.83:0/4115556057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb7d80b3f80 0x7fb7d80b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:28.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.642+0000 7fb7d659c640 1 --2- 172.21.15.83:0/4115556057 >> v2:172.21.15.167:3300/0 conn(0x7fb7d80aecd0 0x7fb7d80cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:28.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.642+0000 7fb7d5d9b640 1 -- 172.21.15.83:0/4115556057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb7d80b3f80 msgr2=0x7fb7d80b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:28.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.642+0000 7fb7d5d9b640 1 --2- 172.21.15.83:0/4115556057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb7d80b3f80 0x7fb7d80b4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:28.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.642+0000 7fb7d5d9b640 1 -- 172.21.15.83:0/4115556057 >> v2:172.21.15.167:3300/0 conn(0x7fb7d80aecd0 msgr2=0x7fb7d80cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:28.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.642+0000 7fb7d5d9b640 1 --2- 172.21.15.83:0/4115556057 >> v2:172.21.15.167:3300/0 conn(0x7fb7d80aecd0 0x7fb7d80cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:28.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.642+0000 7fb7d5d9b640 1 -- 172.21.15.83:0/4115556057 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d8072760 con 0x7fb7d8166550 2024-04-17T10:52:28.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.642+0000 7fb7d659c640 1 --2- 172.21.15.83:0/4115556057 >> v2:172.21.15.167:3300/0 conn(0x7fb7d80aecd0 0x7fb7d80cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:52:28.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.646+0000 7fb7d5d9b640 1 --2- 172.21.15.83:0/4115556057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d8166550 0x7fb7d8071be0 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7fb7d806ffe0 tx=0x7fb7c801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e5dd7128c695ab server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:28.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.646+0000 7fb7d559a640 1 -- 172.21.15.83:0/4115556057 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c80b0070 con 0x7fb7d8166550 2024-04-17T10:52:28.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.646+0000 7fb7d559a640 1 -- 172.21.15.83:0/4115556057 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c80ab600 con 0x7fb7d8166550 2024-04-17T10:52:28.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.646+0000 7fb7d559a640 1 -- 172.21.15.83:0/4115556057 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c80ab9d0 con 0x7fb7d8166550 2024-04-17T10:52:28.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.646+0000 7fb7dc97b640 1 -- 172.21.15.83:0/4115556057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d8166550 msgr2=0x7fb7d8071be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:28.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.646+0000 7fb7dc97b640 1 --2- 172.21.15.83:0/4115556057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d8166550 0x7fb7d8071be0 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7fb7d806ffe0 tx=0x7fb7c801c490 comp rx=0 tx=0).stop 2024-04-17T10:52:28.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.646+0000 7fb7dc97b640 1 -- 172.21.15.83:0/4115556057 shutdown_connections 2024-04-17T10:52:28.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.646+0000 7fb7dc97b640 1 --2- 172.21.15.83:0/4115556057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb7d80b3f80 0x7fb7d80b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:28.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.646+0000 7fb7dc97b640 1 --2- 172.21.15.83:0/4115556057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d8166550 0x7fb7d8071be0 unknown :-1 s=CLOSED pgs=2224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:28.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.646+0000 7fb7dc97b640 1 --2- 172.21.15.83:0/4115556057 >> v2:172.21.15.167:3300/0 conn(0x7fb7d80aecd0 0x7fb7d80cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:28.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.646+0000 7fb7dc97b640 1 -- 172.21.15.83:0/4115556057 >> 172.21.15.83:0/4115556057 conn(0x7fb7d806abe0 msgr2=0x7fb7d806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:28.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.646+0000 7fb7dc97b640 1 -- 172.21.15.83:0/4115556057 shutdown_connections 2024-04-17T10:52:28.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.646+0000 7fb7dc97b640 1 -- 172.21.15.83:0/4115556057 wait complete. 2024-04-17T10:52:28.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.646+0000 7fb7dc97b640 1 Processor -- start 2024-04-17T10:52:28.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.650+0000 7fb7dc97b640 1 -- start start 2024-04-17T10:52:28.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.650+0000 7fb7dc97b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb7d80aecd0 0x7fb7d80ae510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:28.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.650+0000 7fb7dc97b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d80b3f80 0x7fb7d80ab710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:28.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.650+0000 7fb7dc97b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7d8166550 0x7fb7d8148b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:28.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.650+0000 7fb7dc97b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d80b2e80 con 0x7fb7d80b3f80 2024-04-17T10:52:28.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.650+0000 7fb7dc97b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d8072120 con 0x7fb7d8166550 2024-04-17T10:52:28.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.650+0000 7fb7dc97b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d80b3000 con 0x7fb7d80aecd0 2024-04-17T10:52:28.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.650+0000 7fb7d5d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d80b3f80 0x7fb7d80ab710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:28.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.650+0000 7fb7d5d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d80b3f80 0x7fb7d80ab710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44552/0 (socket says 172.21.15.83:44552) 2024-04-17T10:52:28.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.650+0000 7fb7d5d9b640 1 -- 172.21.15.83:0/1375832962 learned_addr learned my addr 172.21.15.83:0/1375832962 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:28.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.650+0000 7fb7d659c640 1 --2- 172.21.15.83:0/1375832962 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb7d80aecd0 0x7fb7d80ae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:28.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.650+0000 7fb7d6d9d640 1 --2- 172.21.15.83:0/1375832962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7d8166550 0x7fb7d8148b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:28.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.650+0000 7fb7d5d9b640 1 -- 172.21.15.83:0/1375832962 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb7d80aecd0 msgr2=0x7fb7d80ae510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:28.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.650+0000 7fb7d5d9b640 1 --2- 172.21.15.83:0/1375832962 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb7d80aecd0 0x7fb7d80ae510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:28.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.650+0000 7fb7d5d9b640 1 -- 172.21.15.83:0/1375832962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7d8166550 msgr2=0x7fb7d8148b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:28.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.650+0000 7fb7d5d9b640 1 --2- 172.21.15.83:0/1375832962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7d8166550 0x7fb7d8148b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:28.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.650+0000 7fb7d5d9b640 1 -- 172.21.15.83:0/1375832962 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d8158340 con 0x7fb7d80b3f80 2024-04-17T10:52:28.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.650+0000 7fb7d659c640 1 --2- 172.21.15.83:0/1375832962 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb7d80aecd0 0x7fb7d80ae510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:52:28.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.650+0000 7fb7d6d9d640 1 --2- 172.21.15.83:0/1375832962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7d8166550 0x7fb7d8148b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:52:28.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.650+0000 7fb7d5d9b640 1 --2- 172.21.15.83:0/1375832962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d80b3f80 0x7fb7d80ab710 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7fb7c801cd30 tx=0x7fb7c8019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:28.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.650+0000 7fb7c6ffd640 1 -- 172.21.15.83:0/1375832962 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c80b8030 con 0x7fb7d80b3f80 2024-04-17T10:52:28.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.650+0000 7fb7c6ffd640 1 -- 172.21.15.83:0/1375832962 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c80b4410 con 0x7fb7d80b3f80 2024-04-17T10:52:28.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.650+0000 7fb7dc97b640 1 -- 172.21.15.83:0/1375832962 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d80abd80 con 0x7fb7d80b3f80 2024-04-17T10:52:28.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.650+0000 7fb7c6ffd640 1 -- 172.21.15.83:0/1375832962 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c80b0040 con 0x7fb7d80b3f80 2024-04-17T10:52:28.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.650+0000 7fb7dc97b640 1 -- 172.21.15.83:0/1375832962 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d80ac340 con 0x7fb7d80b3f80 2024-04-17T10:52:28.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.650+0000 7fb7dc97b640 1 -- 172.21.15.83:0/1375832962 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7d8074f90 con 0x7fb7d80b3f80 2024-04-17T10:52:28.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.650+0000 7fb7c6ffd640 1 -- 172.21.15.83:0/1375832962 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb7c80bb070 con 0x7fb7d80b3f80 2024-04-17T10:52:28.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.650+0000 7fb7c6ffd640 1 --2- 172.21.15.83:0/1375832962 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb7b8066f60 0x7fb7b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:28.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.650+0000 7fb7c6ffd640 1 -- 172.21.15.83:0/1375832962 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fb7c811ebc0 con 0x7fb7d80b3f80 2024-04-17T10:52:28.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.650+0000 7fb7d659c640 1 --2- 172.21.15.83:0/1375832962 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb7b8066f60 0x7fb7b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:28.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.654+0000 7fb7d659c640 1 --2- 172.21.15.83:0/1375832962 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb7b8066f60 0x7fb7b8069420 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc008930 tx=0x7fb7cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:28.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.658+0000 7fb7c6ffd640 1 -- 172.21.15.83:0/1375832962 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb7c8016110 con 0x7fb7d80b3f80 2024-04-17T10:52:28.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.846+0000 7fb7dc97b640 1 -- 172.21.15.83:0/1375832962 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7d80b0550 con 0x7fb7d80b3f80 2024-04-17T10:52:28.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.846+0000 7fb7c6ffd640 1 -- 172.21.15.83:0/1375832962 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fb7c80efbf0 con 0x7fb7d80b3f80 2024-04-17T10:52:28.842 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:28.842 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:28.842 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:28.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.850+0000 7fb7dc97b640 1 -- 172.21.15.83:0/1375832962 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb7b8066f60 msgr2=0x7fb7b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:28.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.850+0000 7fb7dc97b640 1 --2- 172.21.15.83:0/1375832962 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb7b8066f60 0x7fb7b8069420 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc008930 tx=0x7fb7cc008040 comp rx=0 tx=0).stop 2024-04-17T10:52:28.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.850+0000 7fb7dc97b640 1 -- 172.21.15.83:0/1375832962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d80b3f80 msgr2=0x7fb7d80ab710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:28.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.850+0000 7fb7dc97b640 1 --2- 172.21.15.83:0/1375832962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d80b3f80 0x7fb7d80ab710 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7fb7c801cd30 tx=0x7fb7c8019230 comp rx=0 tx=0).stop 2024-04-17T10:52:28.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.850+0000 7fb7dc97b640 1 -- 172.21.15.83:0/1375832962 shutdown_connections 2024-04-17T10:52:28.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.850+0000 7fb7dc97b640 1 --2- 172.21.15.83:0/1375832962 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb7b8066f60 0x7fb7b8069420 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:28.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.850+0000 7fb7dc97b640 1 --2- 172.21.15.83:0/1375832962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7d8166550 0x7fb7d8148b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:28.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.850+0000 7fb7dc97b640 1 --2- 172.21.15.83:0/1375832962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d80b3f80 0x7fb7d80ab710 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:28.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.850+0000 7fb7dc97b640 1 --2- 172.21.15.83:0/1375832962 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb7d80aecd0 0x7fb7d80ae510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:28.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.850+0000 7fb7dc97b640 1 -- 172.21.15.83:0/1375832962 >> 172.21.15.83:0/1375832962 conn(0x7fb7d806abe0 msgr2=0x7fb7d8070960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:28.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.850+0000 7fb7dc97b640 1 -- 172.21.15.83:0/1375832962 shutdown_connections 2024-04-17T10:52:28.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:28.850+0000 7fb7dc97b640 1 -- 172.21.15.83:0/1375832962 wait complete. 2024-04-17T10:52:29.263 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:29.263 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:30.265 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:30.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.402+0000 7f5418ca7640 1 Processor -- start 2024-04-17T10:52:30.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.402+0000 7f5418ca7640 1 -- start start 2024-04-17T10:52:30.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f5418ca7640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f541415ba30 0x7f5414072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:30.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f5418ca7640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f541406e320 con 0x7f54140cfbe0 2024-04-17T10:52:30.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f5418ca7640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f541406e040 con 0x7f541415c960 2024-04-17T10:52:30.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f5418ca7640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5414071ea0 con 0x7f541415ba30 2024-04-17T10:52:30.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f5412d9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f54140cfbe0 0x7f541406c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38798/0 (socket says 172.21.15.83:38798) 2024-04-17T10:52:30.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f5412d9d640 1 -- 172.21.15.83:0/3577739822 learned_addr learned my addr 172.21.15.83:0/3577739822 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:52:30.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f541259c640 1 --2- 172.21.15.83:0/3577739822 >> v2:172.21.15.167:3300/0 conn(0x7f541415ba30 0x7f5414072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:30.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f541159a640 1 -- 172.21.15.83:0/3577739822 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 545664159 0 0) 0x7f541406e320 con 0x7f54140cfbe0 2024-04-17T10:52:30.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f541159a640 1 -- 172.21.15.83:0/3577739822 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53fc003530 con 0x7f54140cfbe0 2024-04-17T10:52:30.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f541159a640 1 -- 172.21.15.83:0/3577739822 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1780221069 0 0) 0x7f53fc003530 con 0x7f54140cfbe0 2024-04-17T10:52:30.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f541159a640 1 -- 172.21.15.83:0/3577739822 >> v2:172.21.15.167:3300/0 conn(0x7f541415ba30 msgr2=0x7f5414072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:30.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f541159a640 1 --2- 172.21.15.83:0/3577739822 >> v2:172.21.15.167:3300/0 conn(0x7f541415ba30 0x7f5414072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:30.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f541159a640 1 -- 172.21.15.83:0/3577739822 >> v1:172.21.15.167:6789/0 conn(0x7f541415c960 legacy=0x7f54140cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:30.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f541159a640 1 -- 172.21.15.83:0/3577739822 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5414071850 con 0x7f54140cfbe0 2024-04-17T10:52:30.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f541159a640 1 -- 172.21.15.83:0/3577739822 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f540c001a20 con 0x7f54140cfbe0 2024-04-17T10:52:30.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f541159a640 1 -- 172.21.15.83:0/3577739822 >> v1:172.21.15.83:6789/0 conn(0x7f54140cfbe0 legacy=0x7f541406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:30.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f541159a640 1 --2- 172.21.15.83:0/3577739822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53fc0039c0 0x7f53fc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:30.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f541159a640 1 --2- 172.21.15.83:0/3577739822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53fc004810 0x7f53fc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:30.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f541159a640 1 --2- 172.21.15.83:0/3577739822 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53fc0051d0 0x7f53fc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:30.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f541159a640 1 -- 172.21.15.83:0/3577739822 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5414071ea0 con 0x7f53fc0039c0 2024-04-17T10:52:30.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f541159a640 1 -- 172.21.15.83:0/3577739822 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53fc00c170 con 0x7f53fc004810 2024-04-17T10:52:30.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f541159a640 1 -- 172.21.15.83:0/3577739822 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53fc00c370 con 0x7f53fc0051d0 2024-04-17T10:52:30.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f541259c640 1 --2- 172.21.15.83:0/3577739822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53fc0039c0 0x7f53fc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:30.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f5411d9b640 1 --2- 172.21.15.83:0/3577739822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53fc004810 0x7f53fc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:30.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f5412d9d640 1 --2- 172.21.15.83:0/3577739822 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53fc0051d0 0x7f53fc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:30.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f541259c640 1 -- 172.21.15.83:0/3577739822 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53fc0051d0 msgr2=0x7f53fc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:30.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f541259c640 1 --2- 172.21.15.83:0/3577739822 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53fc0051d0 0x7f53fc00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:30.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f541259c640 1 -- 172.21.15.83:0/3577739822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53fc004810 msgr2=0x7f53fc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:30.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f541259c640 1 --2- 172.21.15.83:0/3577739822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53fc004810 0x7f53fc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:30.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f541259c640 1 -- 172.21.15.83:0/3577739822 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53fc003530 con 0x7f53fc0039c0 2024-04-17T10:52:30.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f5412d9d640 1 -- 172.21.15.83:0/3577739822 reap_dead start 2024-04-17T10:52:30.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f541259c640 1 --2- 172.21.15.83:0/3577739822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53fc0039c0 0x7f53fc003de0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f5404009ae0 tx=0x7f540401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7107f3833dea6ab3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:30.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f541159a640 1 -- 172.21.15.83:0/3577739822 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54040096b0 con 0x7f53fc0039c0 2024-04-17T10:52:30.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f541159a640 1 -- 172.21.15.83:0/3577739822 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54040a6450 con 0x7f53fc0039c0 2024-04-17T10:52:30.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.406+0000 7f541159a640 1 -- 172.21.15.83:0/3577739822 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54040a6840 con 0x7f53fc0039c0 2024-04-17T10:52:30.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f5418ca7640 1 -- 172.21.15.83:0/3577739822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53fc0039c0 msgr2=0x7f53fc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:30.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f5418ca7640 1 --2- 172.21.15.83:0/3577739822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53fc0039c0 0x7f53fc003de0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f5404009ae0 tx=0x7f540401ca70 comp rx=0 tx=0).stop 2024-04-17T10:52:30.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f5418ca7640 1 -- 172.21.15.83:0/3577739822 shutdown_connections 2024-04-17T10:52:30.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f5418ca7640 1 --2- 172.21.15.83:0/3577739822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53fc0039c0 0x7f53fc003de0 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:30.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f5418ca7640 1 -- 172.21.15.83:0/3577739822 >> 172.21.15.83:0/3577739822 conn(0x7f541406abe0 msgr2=0x7f541406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:30.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f5418ca7640 1 -- 172.21.15.83:0/3577739822 shutdown_connections 2024-04-17T10:52:30.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f5418ca7640 1 -- 172.21.15.83:0/3577739822 wait complete. 2024-04-17T10:52:30.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f5418ca7640 1 Processor -- start 2024-04-17T10:52:30.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f5418ca7640 1 -- start start 2024-04-17T10:52:30.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f5418ca7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53fc0039c0 0x7f54141481f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:30.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f5418ca7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5414154ab0 0x7f5414144e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:30.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f5418ca7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54140ad840 0x7f54140adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:30.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f5418ca7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54140cfa60 con 0x7f5414154ab0 2024-04-17T10:52:30.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f5418ca7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54140cf8e0 con 0x7f54140ad840 2024-04-17T10:52:30.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f5418ca7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f541415cd40 con 0x7f53fc0039c0 2024-04-17T10:52:30.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f5411d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5414154ab0 0x7f5414144e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:30.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f5411d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5414154ab0 0x7f5414144e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44576/0 (socket says 172.21.15.83:44576) 2024-04-17T10:52:30.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f5411d9b640 1 -- 172.21.15.83:0/1905823557 learned_addr learned my addr 172.21.15.83:0/1905823557 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:30.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f541259c640 1 --2- 172.21.15.83:0/1905823557 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53fc0039c0 0x7f54141481f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:30.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f5411d9b640 1 -- 172.21.15.83:0/1905823557 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53fc0039c0 msgr2=0x7f54141481f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:30.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f5411d9b640 1 --2- 172.21.15.83:0/1905823557 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53fc0039c0 0x7f54141481f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:30.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f5411d9b640 1 -- 172.21.15.83:0/1905823557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54140ad840 msgr2=0x7f54140adcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:30.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f5412d9d640 1 --2- 172.21.15.83:0/1905823557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54140ad840 0x7f54140adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:30.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f5411d9b640 1 --2- 172.21.15.83:0/1905823557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54140ad840 0x7f54140adcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:30.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f5411d9b640 1 -- 172.21.15.83:0/1905823557 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5414148730 con 0x7f5414154ab0 2024-04-17T10:52:30.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f541259c640 1 --2- 172.21.15.83:0/1905823557 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53fc0039c0 0x7f54141481f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:52:30.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f5412d9d640 1 --2- 172.21.15.83:0/1905823557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54140ad840 0x7f54140adcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:52:30.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f5411d9b640 1 --2- 172.21.15.83:0/1905823557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5414154ab0 0x7f5414144e30 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7f540800e4c0 tx=0x7f540800e980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:30.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f5402ffd640 1 -- 172.21.15.83:0/1905823557 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54080052e0 con 0x7f5414154ab0 2024-04-17T10:52:30.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f5402ffd640 1 -- 172.21.15.83:0/1905823557 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5408018070 con 0x7f5414154ab0 2024-04-17T10:52:30.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f5418ca7640 1 -- 172.21.15.83:0/1905823557 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5414154f50 con 0x7f5414154ab0 2024-04-17T10:52:30.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f5402ffd640 1 -- 172.21.15.83:0/1905823557 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5408003840 con 0x7f5414154ab0 2024-04-17T10:52:30.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.410+0000 7f5418ca7640 1 -- 172.21.15.83:0/1905823557 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5414155510 con 0x7f5414154ab0 2024-04-17T10:52:30.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.414+0000 7f5418ca7640 1 -- 172.21.15.83:0/1905823557 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5414074f90 con 0x7f5414154ab0 2024-04-17T10:52:30.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.414+0000 7f5402ffd640 1 -- 172.21.15.83:0/1905823557 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f54080039e0 con 0x7f5414154ab0 2024-04-17T10:52:30.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.414+0000 7f5402ffd640 1 --2- 172.21.15.83:0/1905823557 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f53f4066f60 0x7f53f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:30.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.414+0000 7f5402ffd640 1 -- 172.21.15.83:0/1905823557 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f5408084d10 con 0x7f5414154ab0 2024-04-17T10:52:30.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.414+0000 7f541259c640 1 --2- 172.21.15.83:0/1905823557 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f53f4066f60 0x7f53f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:30.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.414+0000 7f541259c640 1 --2- 172.21.15.83:0/1905823557 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f53f4066f60 0x7f53f4069420 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f54040ad000 tx=0x7f54040a9040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:30.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.418+0000 7f5402ffd640 1 -- 172.21.15.83:0/1905823557 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5408055d70 con 0x7f5414154ab0 2024-04-17T10:52:30.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.610+0000 7f5418ca7640 1 -- 172.21.15.83:0/1905823557 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54140cd650 con 0x7f5414154ab0 2024-04-17T10:52:30.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.610+0000 7f5402ffd640 1 -- 172.21.15.83:0/1905823557 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f5408005480 con 0x7f5414154ab0 2024-04-17T10:52:30.604 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:30.604 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:30.604 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:30.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.610+0000 7f5418ca7640 1 -- 172.21.15.83:0/1905823557 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f53f4066f60 msgr2=0x7f53f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:30.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.610+0000 7f5418ca7640 1 --2- 172.21.15.83:0/1905823557 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f53f4066f60 0x7f53f4069420 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f54040ad000 tx=0x7f54040a9040 comp rx=0 tx=0).stop 2024-04-17T10:52:30.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.610+0000 7f5418ca7640 1 -- 172.21.15.83:0/1905823557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5414154ab0 msgr2=0x7f5414144e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:30.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.610+0000 7f5418ca7640 1 --2- 172.21.15.83:0/1905823557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5414154ab0 0x7f5414144e30 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7f540800e4c0 tx=0x7f540800e980 comp rx=0 tx=0).stop 2024-04-17T10:52:30.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.610+0000 7f5418ca7640 1 -- 172.21.15.83:0/1905823557 shutdown_connections 2024-04-17T10:52:30.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.610+0000 7f5418ca7640 1 --2- 172.21.15.83:0/1905823557 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f53f4066f60 0x7f53f4069420 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:30.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.610+0000 7f5418ca7640 1 --2- 172.21.15.83:0/1905823557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54140ad840 0x7f54140adcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:30.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.610+0000 7f5418ca7640 1 --2- 172.21.15.83:0/1905823557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5414154ab0 0x7f5414144e30 unknown :-1 s=CLOSED pgs=2228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:30.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.610+0000 7f5418ca7640 1 --2- 172.21.15.83:0/1905823557 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53fc0039c0 0x7f54141481f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:30.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.610+0000 7f5418ca7640 1 -- 172.21.15.83:0/1905823557 >> 172.21.15.83:0/1905823557 conn(0x7f541406abe0 msgr2=0x7f5414070470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:30.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.610+0000 7f5418ca7640 1 -- 172.21.15.83:0/1905823557 shutdown_connections 2024-04-17T10:52:30.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:30.610+0000 7f5418ca7640 1 -- 172.21.15.83:0/1905823557 wait complete. 2024-04-17T10:52:31.023 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:31.024 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:32.024 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:32.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.158+0000 7f204d5ab640 1 Processor -- start 2024-04-17T10:52:32.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.158+0000 7f204d5ab640 1 -- start start 2024-04-17T10:52:32.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.158+0000 7f204d5ab640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f204815dfa0 0x7f20480b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:32.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.158+0000 7f204d5ab640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f20480ce980 0x7f20480b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:32.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.158+0000 7f204d5ab640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2048162a00 con 0x7f20480cee00 2024-04-17T10:52:32.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.158+0000 7f204d5ab640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2048161e80 con 0x7f20480ce980 2024-04-17T10:52:32.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.158+0000 7f204d5ab640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2048162160 con 0x7f204815dfa0 2024-04-17T10:52:32.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.158+0000 7f2046ffd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f204815dfa0 0x7f20480b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:32.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.158+0000 7f2046ffd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f204815dfa0 0x7f20480b6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:34574/0 (socket says 172.21.15.83:34574) 2024-04-17T10:52:32.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.158+0000 7f2046ffd640 1 -- 172.21.15.83:0/264184237 learned_addr learned my addr 172.21.15.83:0/264184237 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:32.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.158+0000 7f20467fc640 1 --2- 172.21.15.83:0/264184237 >> v2:172.21.15.167:3300/0 conn(0x7f20480ce980 0x7f20480b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:32.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.158+0000 7f2046ffd640 1 -- 172.21.15.83:0/264184237 >> v2:172.21.15.167:3300/0 conn(0x7f20480ce980 msgr2=0x7f20480b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:32.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.158+0000 7f2046ffd640 1 --2- 172.21.15.83:0/264184237 >> v2:172.21.15.167:3300/0 conn(0x7f20480ce980 0x7f20480b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:32.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.158+0000 7f2046ffd640 1 -- 172.21.15.83:0/264184237 >> v1:172.21.15.167:6789/0 conn(0x7f20480cee00 legacy=0x7f20480d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:32.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.158+0000 7f2046ffd640 1 -- 172.21.15.83:0/264184237 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2048162440 con 0x7f204815dfa0 2024-04-17T10:52:32.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.162+0000 7f2046ffd640 1 --2- 172.21.15.83:0/264184237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f204815dfa0 0x7f20480b6c20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2038009870 tx=0x7f203801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee7bbe70c23c64c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:32.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.162+0000 7f2045ffb640 1 -- 172.21.15.83:0/264184237 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20380b1070 con 0x7f204815dfa0 2024-04-17T10:52:32.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.162+0000 7f2045ffb640 1 -- 172.21.15.83:0/264184237 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20380ac410 con 0x7f204815dfa0 2024-04-17T10:52:32.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.162+0000 7f2045ffb640 1 -- 172.21.15.83:0/264184237 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20380ac7e0 con 0x7f204815dfa0 2024-04-17T10:52:32.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.162+0000 7f204d5ab640 1 -- 172.21.15.83:0/264184237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f204815dfa0 msgr2=0x7f20480b6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:32.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.162+0000 7f204d5ab640 1 --2- 172.21.15.83:0/264184237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f204815dfa0 0x7f20480b6c20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2038009870 tx=0x7f203801c4a0 comp rx=0 tx=0).stop 2024-04-17T10:52:32.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.162+0000 7f204d5ab640 1 -- 172.21.15.83:0/264184237 shutdown_connections 2024-04-17T10:52:32.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.162+0000 7f204d5ab640 1 --2- 172.21.15.83:0/264184237 >> v2:172.21.15.167:3300/0 conn(0x7f20480ce980 0x7f20480b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:32.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.162+0000 7f204d5ab640 1 --2- 172.21.15.83:0/264184237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f204815dfa0 0x7f20480b6c20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:32.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.162+0000 7f204d5ab640 1 -- 172.21.15.83:0/264184237 >> 172.21.15.83:0/264184237 conn(0x7f20480d0600 msgr2=0x7f20480d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:32.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.162+0000 7f204d5ab640 1 -- 172.21.15.83:0/264184237 shutdown_connections 2024-04-17T10:52:32.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.162+0000 7f204d5ab640 1 -- 172.21.15.83:0/264184237 wait complete. 2024-04-17T10:52:32.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.162+0000 7f204d5ab640 1 Processor -- start 2024-04-17T10:52:32.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.162+0000 7f204d5ab640 1 -- start start 2024-04-17T10:52:32.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.162+0000 7f204d5ab640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20480ce980 0x7f2048153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:32.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.162+0000 7f204d5ab640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20480cee00 0x7f20480b7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:32.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.162+0000 7f204d5ab640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f204815dfa0 0x7f204814b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:32.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.162+0000 7f204d5ab640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2048161b10 con 0x7f204815dfa0 2024-04-17T10:52:32.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.162+0000 7f204d5ab640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20480d2850 con 0x7f20480cee00 2024-04-17T10:52:32.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.162+0000 7f204d5ab640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204816b660 con 0x7f20480ce980 2024-04-17T10:52:32.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.162+0000 7f20477fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f204815dfa0 0x7f204814b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:32.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.162+0000 7f20477fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f204815dfa0 0x7f204814b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44582/0 (socket says 172.21.15.83:44582) 2024-04-17T10:52:32.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.162+0000 7f20477fe640 1 -- 172.21.15.83:0/2160418755 learned_addr learned my addr 172.21.15.83:0/2160418755 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:32.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.166+0000 7f20467fc640 1 --2- 172.21.15.83:0/2160418755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20480cee00 0x7f20480b7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:32.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.166+0000 7f2046ffd640 1 --2- 172.21.15.83:0/2160418755 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20480ce980 0x7f2048153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:32.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.166+0000 7f20477fe640 1 -- 172.21.15.83:0/2160418755 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20480ce980 msgr2=0x7f2048153100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:32.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.166+0000 7f20477fe640 1 --2- 172.21.15.83:0/2160418755 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20480ce980 0x7f2048153100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:32.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.166+0000 7f20477fe640 1 -- 172.21.15.83:0/2160418755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20480cee00 msgr2=0x7f20480b7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:32.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.166+0000 7f20477fe640 1 --2- 172.21.15.83:0/2160418755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20480cee00 0x7f20480b7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:32.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.166+0000 7f20477fe640 1 -- 172.21.15.83:0/2160418755 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20480b8ce0 con 0x7f204815dfa0 2024-04-17T10:52:32.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.166+0000 7f2046ffd640 1 --2- 172.21.15.83:0/2160418755 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20480ce980 0x7f2048153100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:52:32.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.166+0000 7f20467fc640 1 --2- 172.21.15.83:0/2160418755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20480cee00 0x7f20480b7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:52:32.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.166+0000 7f20477fe640 1 --2- 172.21.15.83:0/2160418755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f204815dfa0 0x7f204814b640 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f204000c730 tx=0x7f204000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:32.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.166+0000 7f203f7fe640 1 -- 172.21.15.83:0/2160418755 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2040003b80 con 0x7f204815dfa0 2024-04-17T10:52:32.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.166+0000 7f203f7fe640 1 -- 172.21.15.83:0/2160418755 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2040003d20 con 0x7f204815dfa0 2024-04-17T10:52:32.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.166+0000 7f204d5ab640 1 -- 172.21.15.83:0/2160418755 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20480b8fc0 con 0x7f204815dfa0 2024-04-17T10:52:32.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.166+0000 7f203f7fe640 1 -- 172.21.15.83:0/2160418755 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204000aa30 con 0x7f204815dfa0 2024-04-17T10:52:32.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.166+0000 7f204d5ab640 1 -- 172.21.15.83:0/2160418755 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f204815fcd0 con 0x7f204815dfa0 2024-04-17T10:52:32.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.166+0000 7f204d5ab640 1 -- 172.21.15.83:0/2160418755 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2048074f90 con 0x7f204815dfa0 2024-04-17T10:52:32.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.166+0000 7f203f7fe640 1 -- 172.21.15.83:0/2160418755 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f204000abd0 con 0x7f204815dfa0 2024-04-17T10:52:32.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.166+0000 7f203f7fe640 1 --2- 172.21.15.83:0/2160418755 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2028066f90 0x7f2028069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:32.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.166+0000 7f203f7fe640 1 -- 172.21.15.83:0/2160418755 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f20400858c0 con 0x7f204815dfa0 2024-04-17T10:52:32.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.166+0000 7f2046ffd640 1 --2- 172.21.15.83:0/2160418755 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2028066f90 0x7f2028069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:32.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.166+0000 7f2046ffd640 1 --2- 172.21.15.83:0/2160418755 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2028066f90 0x7f2028069450 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f20380ae000 tx=0x7f20380a6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:32.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.170+0000 7f203f7fe640 1 -- 172.21.15.83:0/2160418755 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f20400569f0 con 0x7f204815dfa0 2024-04-17T10:52:32.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.354+0000 7f204d5ab640 1 -- 172.21.15.83:0/2160418755 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20480d17a0 con 0x7f204815dfa0 2024-04-17T10:52:32.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.354+0000 7f203f7fe640 1 -- 172.21.15.83:0/2160418755 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f20400562f0 con 0x7f204815dfa0 2024-04-17T10:52:32.349 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:32.349 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:32.349 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.358+0000 7f204d5ab640 1 -- 172.21.15.83:0/2160418755 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2028066f90 msgr2=0x7f2028069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.358+0000 7f204d5ab640 1 --2- 172.21.15.83:0/2160418755 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2028066f90 0x7f2028069450 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f20380ae000 tx=0x7f20380a6040 comp rx=0 tx=0).stop 2024-04-17T10:52:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.358+0000 7f204d5ab640 1 -- 172.21.15.83:0/2160418755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f204815dfa0 msgr2=0x7f204814b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.358+0000 7f204d5ab640 1 --2- 172.21.15.83:0/2160418755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f204815dfa0 0x7f204814b640 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f204000c730 tx=0x7f204000cbf0 comp rx=0 tx=0).stop 2024-04-17T10:52:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.358+0000 7f204d5ab640 1 -- 172.21.15.83:0/2160418755 shutdown_connections 2024-04-17T10:52:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.358+0000 7f204d5ab640 1 --2- 172.21.15.83:0/2160418755 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2028066f90 0x7f2028069450 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.358+0000 7f204d5ab640 1 --2- 172.21.15.83:0/2160418755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f204815dfa0 0x7f204814b640 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.358+0000 7f204d5ab640 1 --2- 172.21.15.83:0/2160418755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20480cee00 0x7f20480b7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.358+0000 7f204d5ab640 1 --2- 172.21.15.83:0/2160418755 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20480ce980 0x7f2048153100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.358+0000 7f204d5ab640 1 -- 172.21.15.83:0/2160418755 >> 172.21.15.83:0/2160418755 conn(0x7f20480d0600 msgr2=0x7f20480cab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.358+0000 7f204d5ab640 1 -- 172.21.15.83:0/2160418755 shutdown_connections 2024-04-17T10:52:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:32.358+0000 7f204d5ab640 1 -- 172.21.15.83:0/2160418755 wait complete. 2024-04-17T10:52:32.768 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:32.769 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:33.769 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:33.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.906+0000 7f514a94a640 1 Processor -- start 2024-04-17T10:52:33.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.906+0000 7f514a94a640 1 -- start start 2024-04-17T10:52:33.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.906+0000 7f514a94a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f514415ba30 0x7f5144072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:33.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.906+0000 7f514a94a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f514406e320 con 0x7f51440cfbe0 2024-04-17T10:52:33.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.906+0000 7f514a94a640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f514406e040 con 0x7f514415c960 2024-04-17T10:52:33.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.906+0000 7f514a94a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5144071ea0 con 0x7f514415ba30 2024-04-17T10:52:33.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.906+0000 7f5143fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f514415ba30 0x7f5144072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:33.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.906+0000 7f5143fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f514415ba30 0x7f5144072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44596/0 (socket says 172.21.15.83:44596) 2024-04-17T10:52:33.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.906+0000 7f5143fff640 1 -- 172.21.15.83:0/967916396 learned_addr learned my addr 172.21.15.83:0/967916396 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:33.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.906+0000 7f5148ee7640 1 --1- 172.21.15.83:0/967916396 >> v1:172.21.15.83:6789/0 conn(0x7f51440cfbe0 0x7f514406c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38810/0 (socket says 172.21.15.83:38810) 2024-04-17T10:52:33.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.906+0000 7f5143fff640 1 -- 172.21.15.83:0/967916396 >> v1:172.21.15.167:6789/0 conn(0x7f514415c960 legacy=0x7f51440cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:33.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.906+0000 7f5143fff640 1 -- 172.21.15.83:0/967916396 >> v1:172.21.15.83:6789/0 conn(0x7f51440cfbe0 legacy=0x7f514406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:33.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.906+0000 7f5143fff640 1 -- 172.21.15.83:0/967916396 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5144071850 con 0x7f514415ba30 2024-04-17T10:52:33.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.906+0000 7f5143fff640 1 --2- 172.21.15.83:0/967916396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f514415ba30 0x7f5144072330 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f5134009870 tx=0x7f513401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f07395a0a73489 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:33.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.906+0000 7f51437fe640 1 -- 172.21.15.83:0/967916396 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51340b1070 con 0x7f514415ba30 2024-04-17T10:52:33.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f51437fe640 1 -- 172.21.15.83:0/967916396 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51340ac640 con 0x7f514415ba30 2024-04-17T10:52:33.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f51437fe640 1 -- 172.21.15.83:0/967916396 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51340aca10 con 0x7f514415ba30 2024-04-17T10:52:33.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f514a94a640 1 -- 172.21.15.83:0/967916396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f514415ba30 msgr2=0x7f5144072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:33.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f514a94a640 1 --2- 172.21.15.83:0/967916396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f514415ba30 0x7f5144072330 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f5134009870 tx=0x7f513401c490 comp rx=0 tx=0).stop 2024-04-17T10:52:33.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f514a94a640 1 -- 172.21.15.83:0/967916396 shutdown_connections 2024-04-17T10:52:33.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f514a94a640 1 --2- 172.21.15.83:0/967916396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f514415ba30 0x7f5144072330 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:33.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f514a94a640 1 -- 172.21.15.83:0/967916396 >> 172.21.15.83:0/967916396 conn(0x7f514406abe0 msgr2=0x7f514406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:33.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f514a94a640 1 -- 172.21.15.83:0/967916396 shutdown_connections 2024-04-17T10:52:33.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f514a94a640 1 -- 172.21.15.83:0/967916396 wait complete. 2024-04-17T10:52:33.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f514a94a640 1 Processor -- start 2024-04-17T10:52:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f514a94a640 1 -- start start 2024-04-17T10:52:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f514a94a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51440cfbe0 0x7f51441473a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f514a94a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f514415ba30 0x7f514414ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f514a94a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f514415c960 0x7f51440cee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f514a94a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514406de20 con 0x7f514415ba30 2024-04-17T10:52:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f514a94a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514406dca0 con 0x7f514415c960 2024-04-17T10:52:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f514a94a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5144169530 con 0x7f51440cfbe0 2024-04-17T10:52:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f513bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f514415ba30 0x7f514414ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f513bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f514415ba30 0x7f514414ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44598/0 (socket says 172.21.15.83:44598) 2024-04-17T10:52:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f513bfff640 1 -- 172.21.15.83:0/3653690355 learned_addr learned my addr 172.21.15.83:0/3653690355 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f5148ee7640 1 --2- 172.21.15.83:0/3653690355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f514415c960 0x7f51440cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f5143fff640 1 --2- 172.21.15.83:0/3653690355 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51440cfbe0 0x7f51441473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f513bfff640 1 -- 172.21.15.83:0/3653690355 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51440cfbe0 msgr2=0x7f51441473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f513bfff640 1 --2- 172.21.15.83:0/3653690355 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51440cfbe0 0x7f51441473a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f513bfff640 1 -- 172.21.15.83:0/3653690355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f514415c960 msgr2=0x7f51440cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f513bfff640 1 --2- 172.21.15.83:0/3653690355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f514415c960 0x7f51440cee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f513bfff640 1 -- 172.21.15.83:0/3653690355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514414f920 con 0x7f514415ba30 2024-04-17T10:52:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f513bfff640 1 --2- 172.21.15.83:0/3653690355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f514415ba30 0x7f514414ef40 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f512c00c450 tx=0x7f512c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f51417fa640 1 -- 172.21.15.83:0/3653690355 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c012830 con 0x7f514415ba30 2024-04-17T10:52:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f51417fa640 1 -- 172.21.15.83:0/3653690355 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512c017070 con 0x7f514415ba30 2024-04-17T10:52:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f514a94a640 1 -- 172.21.15.83:0/3653690355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f514414fc00 con 0x7f514415ba30 2024-04-17T10:52:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f51417fa640 1 -- 172.21.15.83:0/3653690355 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c012c30 con 0x7f514415ba30 2024-04-17T10:52:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.910+0000 7f514a94a640 1 -- 172.21.15.83:0/3653690355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51441501c0 con 0x7f514415ba30 2024-04-17T10:52:33.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.914+0000 7f514a94a640 1 -- 172.21.15.83:0/3653690355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5144074f90 con 0x7f514415ba30 2024-04-17T10:52:33.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.914+0000 7f51417fa640 1 -- 172.21.15.83:0/3653690355 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f512c01c460 con 0x7f514415ba30 2024-04-17T10:52:33.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.914+0000 7f51417fa640 1 --2- 172.21.15.83:0/3653690355 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5124066f60 0x7f5124069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:33.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.914+0000 7f5143fff640 1 --2- 172.21.15.83:0/3653690355 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5124066f60 0x7f5124069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:33.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.914+0000 7f51417fa640 1 -- 172.21.15.83:0/3653690355 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f512c084e80 con 0x7f514415ba30 2024-04-17T10:52:33.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.914+0000 7f5143fff640 1 --2- 172.21.15.83:0/3653690355 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5124066f60 0x7f5124069420 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f51340ae000 tx=0x7f5134004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:33.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:33.918+0000 7f51417fa640 1 -- 172.21.15.83:0/3653690355 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f512c055e20 con 0x7f514415ba30 2024-04-17T10:52:34.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:34.106+0000 7f514a94a640 1 -- 172.21.15.83:0/3653690355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f514415afc0 con 0x7f514415ba30 2024-04-17T10:52:34.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:34.106+0000 7f51417fa640 1 -- 172.21.15.83:0/3653690355 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f512c055720 con 0x7f514415ba30 2024-04-17T10:52:34.101 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:34.101 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:34.101 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:34.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:34.106+0000 7f514a94a640 1 -- 172.21.15.83:0/3653690355 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5124066f60 msgr2=0x7f5124069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:34.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:34.106+0000 7f514a94a640 1 --2- 172.21.15.83:0/3653690355 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5124066f60 0x7f5124069420 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f51340ae000 tx=0x7f5134004740 comp rx=0 tx=0).stop 2024-04-17T10:52:34.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:34.106+0000 7f514a94a640 1 -- 172.21.15.83:0/3653690355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f514415ba30 msgr2=0x7f514414ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:34.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:34.106+0000 7f514a94a640 1 --2- 172.21.15.83:0/3653690355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f514415ba30 0x7f514414ef40 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f512c00c450 tx=0x7f512c00c910 comp rx=0 tx=0).stop 2024-04-17T10:52:34.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:34.110+0000 7f514a94a640 1 -- 172.21.15.83:0/3653690355 shutdown_connections 2024-04-17T10:52:34.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:34.110+0000 7f514a94a640 1 --2- 172.21.15.83:0/3653690355 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5124066f60 0x7f5124069420 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:34.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:34.110+0000 7f514a94a640 1 --2- 172.21.15.83:0/3653690355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f514415c960 0x7f51440cee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:34.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:34.110+0000 7f514a94a640 1 --2- 172.21.15.83:0/3653690355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f514415ba30 0x7f514414ef40 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:34.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:34.110+0000 7f514a94a640 1 --2- 172.21.15.83:0/3653690355 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51440cfbe0 0x7f51441473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:34.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:34.110+0000 7f514a94a640 1 -- 172.21.15.83:0/3653690355 >> 172.21.15.83:0/3653690355 conn(0x7f514406abe0 msgr2=0x7f514406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:34.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:34.110+0000 7f514a94a640 1 -- 172.21.15.83:0/3653690355 shutdown_connections 2024-04-17T10:52:34.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:34.110+0000 7f514a94a640 1 -- 172.21.15.83:0/3653690355 wait complete. 2024-04-17T10:52:34.521 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:34.522 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:35.523 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:35.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc3031db640 1 Processor -- start 2024-04-17T10:52:35.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc3031db640 1 -- start start 2024-04-17T10:52:35.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc3031db640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc2fc0cf060 0x7fc2fc0b4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:35.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc3031db640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2fc0b2bd0 con 0x7fc2fc0b3990 2024-04-17T10:52:35.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc3031db640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2fc0b2eb0 con 0x7fc2fc070b80 2024-04-17T10:52:35.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc3031db640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2fc0b2980 con 0x7fc2fc0cf060 2024-04-17T10:52:35.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc2fbfff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc2fc0b3990 0x7fc2fc070470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38816/0 (socket says 172.21.15.83:38816) 2024-04-17T10:52:35.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc2fbfff640 1 -- 172.21.15.83:0/3256791779 learned_addr learned my addr 172.21.15.83:0/3256791779 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:52:35.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc300f77640 1 --2- 172.21.15.83:0/3256791779 >> v2:172.21.15.167:3300/0 conn(0x7fc2fc0cf060 0x7fc2fc0b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:35.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc2fb7fe640 1 -- 172.21.15.83:0/3256791779 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2531579532 0 0) 0x7fc2fc0b2bd0 con 0x7fc2fc0b3990 2024-04-17T10:52:35.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc2fb7fe640 1 -- 172.21.15.83:0/3256791779 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2e4003450 con 0x7fc2fc0b3990 2024-04-17T10:52:35.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc2fb7fe640 1 -- 172.21.15.83:0/3256791779 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3263004809 0 0) 0x7fc2e4003450 con 0x7fc2fc0b3990 2024-04-17T10:52:35.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc2fb7fe640 1 -- 172.21.15.83:0/3256791779 >> v2:172.21.15.167:3300/0 conn(0x7fc2fc0cf060 msgr2=0x7fc2fc0b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:35.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc2fb7fe640 1 --2- 172.21.15.83:0/3256791779 >> v2:172.21.15.167:3300/0 conn(0x7fc2fc0cf060 0x7fc2fc0b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:35.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc2fb7fe640 1 -- 172.21.15.83:0/3256791779 >> v1:172.21.15.167:6789/0 conn(0x7fc2fc070b80 legacy=0x7fc2fc06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:35.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc2fb7fe640 1 -- 172.21.15.83:0/3256791779 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2fc0b3190 con 0x7fc2fc0b3990 2024-04-17T10:52:35.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc2fb7fe640 1 -- 172.21.15.83:0/3256791779 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fc2f0001680 con 0x7fc2fc0b3990 2024-04-17T10:52:35.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc2fb7fe640 1 -- 172.21.15.83:0/3256791779 >> v1:172.21.15.83:6789/0 conn(0x7fc2fc0b3990 legacy=0x7fc2fc070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:35.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc2fb7fe640 1 --2- 172.21.15.83:0/3256791779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2e4003ab0 0x7fc2e4003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:35.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc2fb7fe640 1 --2- 172.21.15.83:0/3256791779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2e4008860 0x7fc2e400acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:35.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc2fb7fe640 1 --2- 172.21.15.83:0/3256791779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2e400b4d0 0x7fc2e400d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:35.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc2fb7fe640 1 -- 172.21.15.83:0/3256791779 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc0b2980 con 0x7fc2e4003ab0 2024-04-17T10:52:35.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc2fb7fe640 1 -- 172.21.15.83:0/3256791779 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e400e270 con 0x7fc2e4008860 2024-04-17T10:52:35.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc2fb7fe640 1 -- 172.21.15.83:0/3256791779 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e400e470 con 0x7fc2e400b4d0 2024-04-17T10:52:35.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc300f77640 1 --2- 172.21.15.83:0/3256791779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2e4003ab0 0x7fc2e4003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:35.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc2fbfff640 1 --2- 172.21.15.83:0/3256791779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2e4008860 0x7fc2e400acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:35.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc300f77640 1 -- 172.21.15.83:0/3256791779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2e400b4d0 msgr2=0x7fc2e400d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:35.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc300f77640 1 --2- 172.21.15.83:0/3256791779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2e400b4d0 0x7fc2e400d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:35.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc300f77640 1 -- 172.21.15.83:0/3256791779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2e4008860 msgr2=0x7fc2e400acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:35.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc300f77640 1 --2- 172.21.15.83:0/3256791779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2e4008860 0x7fc2e400acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:35.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc300f77640 1 -- 172.21.15.83:0/3256791779 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e4003450 con 0x7fc2e4003ab0 2024-04-17T10:52:35.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc301778640 1 -- 172.21.15.83:0/3256791779 reap_dead start 2024-04-17T10:52:35.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.658+0000 7fc2fbfff640 1 --2- 172.21.15.83:0/3256791779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2e4008860 0x7fc2e400acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:52:35.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.662+0000 7fc300f77640 1 --2- 172.21.15.83:0/3256791779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2e4003ab0 0x7fc2e4003ed0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009fc0 tx=0x7fc2ec01cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31bfd8928d48482d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:35.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.662+0000 7fc2fb7fe640 1 -- 172.21.15.83:0/3256791779 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec0b2070 con 0x7fc2e4003ab0 2024-04-17T10:52:35.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.662+0000 7fc2fb7fe640 1 -- 172.21.15.83:0/3256791779 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ec0a5b60 con 0x7fc2e4003ab0 2024-04-17T10:52:35.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.662+0000 7fc2fb7fe640 1 -- 172.21.15.83:0/3256791779 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec009c10 con 0x7fc2e4003ab0 2024-04-17T10:52:35.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.662+0000 7fc3031db640 1 -- 172.21.15.83:0/3256791779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2e4003ab0 msgr2=0x7fc2e4003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:35.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.662+0000 7fc3031db640 1 --2- 172.21.15.83:0/3256791779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2e4003ab0 0x7fc2e4003ed0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009fc0 tx=0x7fc2ec01cc90 comp rx=0 tx=0).stop 2024-04-17T10:52:35.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.662+0000 7fc3031db640 1 -- 172.21.15.83:0/3256791779 shutdown_connections 2024-04-17T10:52:35.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.662+0000 7fc3031db640 1 --2- 172.21.15.83:0/3256791779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2e4003ab0 0x7fc2e4003ed0 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:35.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.662+0000 7fc3031db640 1 -- 172.21.15.83:0/3256791779 >> 172.21.15.83:0/3256791779 conn(0x7fc2fc06abe0 msgr2=0x7fc2fc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:35.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.662+0000 7fc3031db640 1 -- 172.21.15.83:0/3256791779 shutdown_connections 2024-04-17T10:52:35.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.662+0000 7fc3031db640 1 -- 172.21.15.83:0/3256791779 wait complete. 2024-04-17T10:52:35.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.662+0000 7fc3031db640 1 Processor -- start 2024-04-17T10:52:35.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.662+0000 7fc3031db640 1 -- start start 2024-04-17T10:52:35.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.666+0000 7fc3031db640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2e4003ab0 0x7fc2fc0afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:35.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.666+0000 7fc3031db640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2fc15d750 0x7fc2fc15dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:35.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.666+0000 7fc3031db640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2fc154da0 0x7fc2fc155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:35.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.666+0000 7fc3031db640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc0cf5c0 con 0x7fc2fc154da0 2024-04-17T10:52:35.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.666+0000 7fc3031db640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc0cf440 con 0x7fc2e4003ab0 2024-04-17T10:52:35.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.666+0000 7fc3031db640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc0cf740 con 0x7fc2fc15d750 2024-04-17T10:52:35.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.666+0000 7fc301778640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2fc154da0 0x7fc2fc155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:35.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.666+0000 7fc301778640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2fc154da0 0x7fc2fc155220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44610/0 (socket says 172.21.15.83:44610) 2024-04-17T10:52:35.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.666+0000 7fc301778640 1 -- 172.21.15.83:0/4160485917 learned_addr learned my addr 172.21.15.83:0/4160485917 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:35.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.666+0000 7fc300f77640 1 --2- 172.21.15.83:0/4160485917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2e4003ab0 0x7fc2fc0afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:35.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.666+0000 7fc2fbfff640 1 --2- 172.21.15.83:0/4160485917 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2fc15d750 0x7fc2fc15dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:35.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.666+0000 7fc301778640 1 -- 172.21.15.83:0/4160485917 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2fc15d750 msgr2=0x7fc2fc15dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:35.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.666+0000 7fc301778640 1 --2- 172.21.15.83:0/4160485917 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2fc15d750 0x7fc2fc15dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:35.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.666+0000 7fc301778640 1 -- 172.21.15.83:0/4160485917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2e4003ab0 msgr2=0x7fc2fc0afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:35.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.666+0000 7fc301778640 1 --2- 172.21.15.83:0/4160485917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2e4003ab0 0x7fc2fc0afe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:35.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.666+0000 7fc301778640 1 -- 172.21.15.83:0/4160485917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2fc1511b0 con 0x7fc2fc154da0 2024-04-17T10:52:35.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.666+0000 7fc2fbfff640 1 --2- 172.21.15.83:0/4160485917 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2fc15d750 0x7fc2fc15dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:52:35.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.666+0000 7fc301778640 1 --2- 172.21.15.83:0/4160485917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2fc154da0 0x7fc2fc155220 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc0b4b00 tx=0x7fc2f400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:35.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.666+0000 7fc2f97fa640 1 -- 172.21.15.83:0/4160485917 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4007480 con 0x7fc2fc154da0 2024-04-17T10:52:35.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.666+0000 7fc2f97fa640 1 -- 172.21.15.83:0/4160485917 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f4007640 con 0x7fc2fc154da0 2024-04-17T10:52:35.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.666+0000 7fc2f97fa640 1 -- 172.21.15.83:0/4160485917 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4004260 con 0x7fc2fc154da0 2024-04-17T10:52:35.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.666+0000 7fc3031db640 1 -- 172.21.15.83:0/4160485917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2fc150bf0 con 0x7fc2fc154da0 2024-04-17T10:52:35.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.666+0000 7fc3031db640 1 -- 172.21.15.83:0/4160485917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2fc145b50 con 0x7fc2fc154da0 2024-04-17T10:52:35.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.666+0000 7fc3031db640 1 -- 172.21.15.83:0/4160485917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2fc155760 con 0x7fc2fc154da0 2024-04-17T10:52:35.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.666+0000 7fc2f97fa640 1 -- 172.21.15.83:0/4160485917 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc2f4011070 con 0x7fc2fc154da0 2024-04-17T10:52:35.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.666+0000 7fc2f97fa640 1 --2- 172.21.15.83:0/4160485917 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc2e8066f60 0x7fc2e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:35.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.670+0000 7fc2f97fa640 1 -- 172.21.15.83:0/4160485917 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fc2f4083ab0 con 0x7fc2fc154da0 2024-04-17T10:52:35.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.670+0000 7fc300f77640 1 --2- 172.21.15.83:0/4160485917 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc2e8066f60 0x7fc2e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:35.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.670+0000 7fc300f77640 1 --2- 172.21.15.83:0/4160485917 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc2e8066f60 0x7fc2e8069420 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0af000 tx=0x7fc2ec0a7040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:35.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.674+0000 7fc2f97fa640 1 -- 172.21.15.83:0/4160485917 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc2f40549c0 con 0x7fc2fc154da0 2024-04-17T10:52:35.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.858+0000 7fc3031db640 1 -- 172.21.15.83:0/4160485917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2fc074f90 con 0x7fc2fc154da0 2024-04-17T10:52:35.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.858+0000 7fc2f97fa640 1 -- 172.21.15.83:0/4160485917 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fc2f40542c0 con 0x7fc2fc154da0 2024-04-17T10:52:35.855 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:35.855 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:35.855 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:35.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.862+0000 7fc3031db640 1 -- 172.21.15.83:0/4160485917 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc2e8066f60 msgr2=0x7fc2e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:35.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.862+0000 7fc3031db640 1 --2- 172.21.15.83:0/4160485917 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc2e8066f60 0x7fc2e8069420 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0af000 tx=0x7fc2ec0a7040 comp rx=0 tx=0).stop 2024-04-17T10:52:35.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.862+0000 7fc3031db640 1 -- 172.21.15.83:0/4160485917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2fc154da0 msgr2=0x7fc2fc155220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:35.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.862+0000 7fc3031db640 1 --2- 172.21.15.83:0/4160485917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2fc154da0 0x7fc2fc155220 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc0b4b00 tx=0x7fc2f400c820 comp rx=0 tx=0).stop 2024-04-17T10:52:35.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.862+0000 7fc3031db640 1 -- 172.21.15.83:0/4160485917 shutdown_connections 2024-04-17T10:52:35.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.862+0000 7fc3031db640 1 --2- 172.21.15.83:0/4160485917 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc2e8066f60 0x7fc2e8069420 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:35.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.862+0000 7fc3031db640 1 --2- 172.21.15.83:0/4160485917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2fc154da0 0x7fc2fc155220 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:35.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.862+0000 7fc3031db640 1 --2- 172.21.15.83:0/4160485917 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2fc15d750 0x7fc2fc15dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:35.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.862+0000 7fc3031db640 1 --2- 172.21.15.83:0/4160485917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2e4003ab0 0x7fc2fc0afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:35.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.862+0000 7fc3031db640 1 -- 172.21.15.83:0/4160485917 >> 172.21.15.83:0/4160485917 conn(0x7fc2fc06abe0 msgr2=0x7fc2fc06d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:35.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.862+0000 7fc3031db640 1 -- 172.21.15.83:0/4160485917 shutdown_connections 2024-04-17T10:52:35.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:35.862+0000 7fc3031db640 1 -- 172.21.15.83:0/4160485917 wait complete. 2024-04-17T10:52:36.275 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:36.276 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:37.277 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:37.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.414+0000 7fd7b676a640 1 Processor -- start 2024-04-17T10:52:37.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.414+0000 7fd7b676a640 1 -- start start 2024-04-17T10:52:37.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.414+0000 7fd7b676a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd7b015dfa0 0x7fd7b00b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:37.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.414+0000 7fd7b676a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7b0162a00 con 0x7fd7b00b5b40 2024-04-17T10:52:37.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.414+0000 7fd7b676a640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7b0161e80 con 0x7fd7b00ce980 2024-04-17T10:52:37.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.414+0000 7fd7b676a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7b0162160 con 0x7fd7b015dfa0 2024-04-17T10:52:37.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.414+0000 7fd7affff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd7b015dfa0 0x7fd7b00b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:37.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.414+0000 7fd7affff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd7b015dfa0 0x7fd7b00b6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36842/0 (socket says 172.21.15.83:36842) 2024-04-17T10:52:37.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.414+0000 7fd7affff640 1 -- 172.21.15.83:0/1466850581 learned_addr learned my addr 172.21.15.83:0/1466850581 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:37.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.414+0000 7fd7affff640 1 -- 172.21.15.83:0/1466850581 >> v1:172.21.15.167:6789/0 conn(0x7fd7b00ce980 legacy=0x7fd7b00b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:37.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.414+0000 7fd7affff640 1 -- 172.21.15.83:0/1466850581 >> v1:172.21.15.83:6789/0 conn(0x7fd7b00b5b40 legacy=0x7fd7b00d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:37.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.414+0000 7fd7aeffd640 1 -- 172.21.15.83:0/1466850581 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1572843227 0 0) 0x7fd7b0162a00 con 0x7fd7b00b5b40 2024-04-17T10:52:37.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.414+0000 7fd7affff640 1 -- 172.21.15.83:0/1466850581 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b0162440 con 0x7fd7b015dfa0 2024-04-17T10:52:37.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.418+0000 7fd7affff640 1 --2- 172.21.15.83:0/1466850581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7b015dfa0 0x7fd7b00b6c20 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0009870 tx=0x7fd7a001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=953d40497b9c965f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:37.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.418+0000 7fd7aeffd640 1 -- 172.21.15.83:0/1466850581 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a00ab600 con 0x7fd7b015dfa0 2024-04-17T10:52:37.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.418+0000 7fd7aeffd640 1 -- 172.21.15.83:0/1466850581 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a00b0070 con 0x7fd7b015dfa0 2024-04-17T10:52:37.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.418+0000 7fd7aeffd640 1 -- 172.21.15.83:0/1466850581 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a00ab9c0 con 0x7fd7b015dfa0 2024-04-17T10:52:37.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.418+0000 7fd7b676a640 1 -- 172.21.15.83:0/1466850581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7b015dfa0 msgr2=0x7fd7b00b6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:37.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.418+0000 7fd7b676a640 1 --2- 172.21.15.83:0/1466850581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7b015dfa0 0x7fd7b00b6c20 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0009870 tx=0x7fd7a001c460 comp rx=0 tx=0).stop 2024-04-17T10:52:37.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.418+0000 7fd7b676a640 1 -- 172.21.15.83:0/1466850581 shutdown_connections 2024-04-17T10:52:37.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.418+0000 7fd7b676a640 1 --2- 172.21.15.83:0/1466850581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7b015dfa0 0x7fd7b00b6c20 unknown :-1 s=CLOSED pgs=2237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:37.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.418+0000 7fd7b676a640 1 -- 172.21.15.83:0/1466850581 >> 172.21.15.83:0/1466850581 conn(0x7fd7b00d0600 msgr2=0x7fd7b00d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:37.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.418+0000 7fd7b676a640 1 -- 172.21.15.83:0/1466850581 shutdown_connections 2024-04-17T10:52:37.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.418+0000 7fd7b676a640 1 -- 172.21.15.83:0/1466850581 wait complete. 2024-04-17T10:52:37.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.418+0000 7fd7b676a640 1 Processor -- start 2024-04-17T10:52:37.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.418+0000 7fd7b676a640 1 -- start start 2024-04-17T10:52:37.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.418+0000 7fd7b676a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b00b5b40 0x7fd7b00c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:37.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.418+0000 7fd7b676a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7b00ce980 0x7fd7b014c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:37.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.418+0000 7fd7b676a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b015dfa0 0x7fd7b0160b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:37.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.418+0000 7fd7b676a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b0161b10 con 0x7fd7b00ce980 2024-04-17T10:52:37.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.418+0000 7fd7b676a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b00d2850 con 0x7fd7b015dfa0 2024-04-17T10:52:37.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.418+0000 7fd7b676a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b016c670 con 0x7fd7b00b5b40 2024-04-17T10:52:37.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.418+0000 7fd7af7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7b00ce980 0x7fd7b014c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:37.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.418+0000 7fd7af7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7b00ce980 0x7fd7b014c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36856/0 (socket says 172.21.15.83:36856) 2024-04-17T10:52:37.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.418+0000 7fd7af7fe640 1 -- 172.21.15.83:0/2389873063 learned_addr learned my addr 172.21.15.83:0/2389873063 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:37.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.418+0000 7fd7affff640 1 --2- 172.21.15.83:0/2389873063 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b00b5b40 0x7fd7b00c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:37.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.418+0000 7fd7af7fe640 1 -- 172.21.15.83:0/2389873063 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b00b5b40 msgr2=0x7fd7b00c5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:37.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.418+0000 7fd7af7fe640 1 --2- 172.21.15.83:0/2389873063 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b00b5b40 0x7fd7b00c5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:37.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.418+0000 7fd7af7fe640 1 -- 172.21.15.83:0/2389873063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b015dfa0 msgr2=0x7fd7b0160b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:37.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.418+0000 7fd7b4d07640 1 --2- 172.21.15.83:0/2389873063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b015dfa0 0x7fd7b0160b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:37.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.418+0000 7fd7af7fe640 1 --2- 172.21.15.83:0/2389873063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b015dfa0 0x7fd7b0160b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:37.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.418+0000 7fd7af7fe640 1 -- 172.21.15.83:0/2389873063 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b014ccc0 con 0x7fd7b00ce980 2024-04-17T10:52:37.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.422+0000 7fd7af7fe640 1 --2- 172.21.15.83:0/2389873063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7b00ce980 0x7fd7b014c220 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7fd7a400c450 tx=0x7fd7a400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:37.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.422+0000 7fd7acff9640 1 -- 172.21.15.83:0/2389873063 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4012830 con 0x7fd7b00ce980 2024-04-17T10:52:37.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.422+0000 7fd7acff9640 1 -- 172.21.15.83:0/2389873063 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a4017070 con 0x7fd7b00ce980 2024-04-17T10:52:37.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.422+0000 7fd7b676a640 1 -- 172.21.15.83:0/2389873063 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b01524b0 con 0x7fd7b00ce980 2024-04-17T10:52:37.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.422+0000 7fd7acff9640 1 -- 172.21.15.83:0/2389873063 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4012c30 con 0x7fd7b00ce980 2024-04-17T10:52:37.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.422+0000 7fd7b676a640 1 -- 172.21.15.83:0/2389873063 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b0152a70 con 0x7fd7b00ce980 2024-04-17T10:52:37.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.422+0000 7fd7b676a640 1 -- 172.21.15.83:0/2389873063 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7b00681a0 con 0x7fd7b00ce980 2024-04-17T10:52:37.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.422+0000 7fd7acff9640 1 -- 172.21.15.83:0/2389873063 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd7a401c460 con 0x7fd7b00ce980 2024-04-17T10:52:37.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.422+0000 7fd7acff9640 1 --2- 172.21.15.83:0/2389873063 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd79c066f60 0x7fd79c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:37.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.422+0000 7fd7acff9640 1 -- 172.21.15.83:0/2389873063 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fd7a4084e90 con 0x7fd7b00ce980 2024-04-17T10:52:37.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.422+0000 7fd7affff640 1 --2- 172.21.15.83:0/2389873063 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd79c066f60 0x7fd79c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:37.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.422+0000 7fd7affff640 1 --2- 172.21.15.83:0/2389873063 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd79c066f60 0x7fd79c069420 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fd7a00af000 tx=0x7fd7a00a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:37.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.426+0000 7fd7acff9640 1 -- 172.21.15.83:0/2389873063 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd7a4055e30 con 0x7fd7b00ce980 2024-04-17T10:52:37.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.622+0000 7fd7b676a640 1 -- 172.21.15.83:0/2389873063 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7b0074f90 con 0x7fd7b00ce980 2024-04-17T10:52:37.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.622+0000 7fd7acff9640 1 -- 172.21.15.83:0/2389873063 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fd7a4055730 con 0x7fd7b00ce980 2024-04-17T10:52:37.617 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:37.617 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:37.618 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:37.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.626+0000 7fd7b676a640 1 -- 172.21.15.83:0/2389873063 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd79c066f60 msgr2=0x7fd79c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:37.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.626+0000 7fd7b676a640 1 --2- 172.21.15.83:0/2389873063 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd79c066f60 0x7fd79c069420 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fd7a00af000 tx=0x7fd7a00a7000 comp rx=0 tx=0).stop 2024-04-17T10:52:37.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.626+0000 7fd7b676a640 1 -- 172.21.15.83:0/2389873063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7b00ce980 msgr2=0x7fd7b014c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:37.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.626+0000 7fd7b676a640 1 --2- 172.21.15.83:0/2389873063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7b00ce980 0x7fd7b014c220 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7fd7a400c450 tx=0x7fd7a400c910 comp rx=0 tx=0).stop 2024-04-17T10:52:37.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.626+0000 7fd7b676a640 1 -- 172.21.15.83:0/2389873063 shutdown_connections 2024-04-17T10:52:37.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.626+0000 7fd7b676a640 1 --2- 172.21.15.83:0/2389873063 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd79c066f60 0x7fd79c069420 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:37.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.626+0000 7fd7b676a640 1 --2- 172.21.15.83:0/2389873063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b015dfa0 0x7fd7b0160b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:37.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.626+0000 7fd7b676a640 1 --2- 172.21.15.83:0/2389873063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7b00ce980 0x7fd7b014c220 unknown :-1 s=CLOSED pgs=2238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:37.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.626+0000 7fd7b676a640 1 --2- 172.21.15.83:0/2389873063 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b00b5b40 0x7fd7b00c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:37.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.626+0000 7fd7b676a640 1 -- 172.21.15.83:0/2389873063 >> 172.21.15.83:0/2389873063 conn(0x7fd7b00d0600 msgr2=0x7fd7b00cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:37.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.626+0000 7fd7b676a640 1 -- 172.21.15.83:0/2389873063 shutdown_connections 2024-04-17T10:52:37.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:37.626+0000 7fd7b676a640 1 -- 172.21.15.83:0/2389873063 wait complete. 2024-04-17T10:52:38.039 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:38.039 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:39.040 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:39.169 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:52:39.171 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:52:39.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.178+0000 7fb186144640 1 Processor -- start 2024-04-17T10:52:39.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.178+0000 7fb186144640 1 -- start start 2024-04-17T10:52:39.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.182+0000 7fb186144640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb18015a890 0x7fb18015ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:39.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.182+0000 7fb186144640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb18006ebf0 0x7fb180071580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:39.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.182+0000 7fb186144640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1800cfee0 con 0x7fb180071ac0 2024-04-17T10:52:39.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.182+0000 7fb186144640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb18006e760 con 0x7fb18006ebf0 2024-04-17T10:52:39.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.182+0000 7fb186144640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18015f360 con 0x7fb18015a890 2024-04-17T10:52:39.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.182+0000 7fb176ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb18006ebf0 0x7fb180071580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:39.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.182+0000 7fb17ffff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb180071ac0 0x7fb18015c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58444/0 (socket says 172.21.15.83:58444) 2024-04-17T10:52:39.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.182+0000 7fb17ffff640 1 -- 172.21.15.83:0/2341874934 learned_addr learned my addr 172.21.15.83:0/2341874934 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:52:39.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.182+0000 7fb17f7fe640 1 --2- 172.21.15.83:0/2341874934 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb18015a890 0x7fb18015ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:39.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.182+0000 7fb17effd640 1 -- 172.21.15.83:0/2341874934 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1722875866 0 0) 0x7fb1800cfee0 con 0x7fb180071ac0 2024-04-17T10:52:39.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.182+0000 7fb17effd640 1 -- 172.21.15.83:0/2341874934 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb16c003530 con 0x7fb180071ac0 2024-04-17T10:52:39.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.182+0000 7fb176ffd640 1 -- 172.21.15.83:0/2341874934 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb18015a890 msgr2=0x7fb18015ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:39.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.182+0000 7fb176ffd640 1 --2- 172.21.15.83:0/2341874934 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb18015a890 0x7fb18015ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:39.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.182+0000 7fb176ffd640 1 -- 172.21.15.83:0/2341874934 >> v1:172.21.15.83:6789/0 conn(0x7fb180071ac0 legacy=0x7fb18015c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:39.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.182+0000 7fb176ffd640 1 -- 172.21.15.83:0/2341874934 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1800d0180 con 0x7fb18006ebf0 2024-04-17T10:52:39.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.182+0000 7fb17f7fe640 1 --2- 172.21.15.83:0/2341874934 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb18015a890 0x7fb18015ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:52:39.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb176ffd640 1 --2- 172.21.15.83:0/2341874934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb18006ebf0 0x7fb180071580 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7fb1680097c0 tx=0x7fb16801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65adbb05caed90a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:39.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb17effd640 1 -- 172.21.15.83:0/2341874934 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1680b0070 con 0x7fb18006ebf0 2024-04-17T10:52:39.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb17effd640 1 -- 172.21.15.83:0/2341874934 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1680ab600 con 0x7fb18006ebf0 2024-04-17T10:52:39.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb17effd640 1 -- 172.21.15.83:0/2341874934 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1680ab9d0 con 0x7fb18006ebf0 2024-04-17T10:52:39.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb186144640 1 -- 172.21.15.83:0/2341874934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb18006ebf0 msgr2=0x7fb180071580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:39.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb186144640 1 --2- 172.21.15.83:0/2341874934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb18006ebf0 0x7fb180071580 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7fb1680097c0 tx=0x7fb16801c4a0 comp rx=0 tx=0).stop 2024-04-17T10:52:39.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb186144640 1 -- 172.21.15.83:0/2341874934 shutdown_connections 2024-04-17T10:52:39.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb186144640 1 --2- 172.21.15.83:0/2341874934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb18006ebf0 0x7fb180071580 unknown :-1 s=CLOSED pgs=2240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:39.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb186144640 1 --2- 172.21.15.83:0/2341874934 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb18015a890 0x7fb18015ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:39.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb186144640 1 -- 172.21.15.83:0/2341874934 >> 172.21.15.83:0/2341874934 conn(0x7fb1800b2410 msgr2=0x7fb1800b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:39.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb186144640 1 -- 172.21.15.83:0/2341874934 shutdown_connections 2024-04-17T10:52:39.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb186144640 1 -- 172.21.15.83:0/2341874934 wait complete. 2024-04-17T10:52:39.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb186144640 1 Processor -- start 2024-04-17T10:52:39.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb186144640 1 -- start start 2024-04-17T10:52:39.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb186144640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb18006ebf0 0x7fb1801413a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:39.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb186144640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb180071ac0 0x7fb180152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:39.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb186144640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb18015a890 0x7fb1801534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:39.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb186144640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18006dce0 con 0x7fb18006ebf0 2024-04-17T10:52:39.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb186144640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18015bc80 con 0x7fb18015a890 2024-04-17T10:52:39.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb186144640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18006de60 con 0x7fb180071ac0 2024-04-17T10:52:39.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb17f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb18006ebf0 0x7fb1801413a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:39.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb17f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb18006ebf0 0x7fb1801413a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36872/0 (socket says 172.21.15.83:36872) 2024-04-17T10:52:39.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb17f7fe640 1 -- 172.21.15.83:0/2100836674 learned_addr learned my addr 172.21.15.83:0/2100836674 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:39.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb17ffff640 1 --2- 172.21.15.83:0/2100836674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb18015a890 0x7fb1801534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:39.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb176ffd640 1 --2- 172.21.15.83:0/2100836674 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb180071ac0 0x7fb180152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:39.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb17f7fe640 1 -- 172.21.15.83:0/2100836674 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb180071ac0 msgr2=0x7fb180152f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:39.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb17f7fe640 1 --2- 172.21.15.83:0/2100836674 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb180071ac0 0x7fb180152f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:39.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb17f7fe640 1 -- 172.21.15.83:0/2100836674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb18015a890 msgr2=0x7fb1801534d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:39.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb17f7fe640 1 --2- 172.21.15.83:0/2100836674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb18015a890 0x7fb1801534d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:39.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb17f7fe640 1 -- 172.21.15.83:0/2100836674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb180153de0 con 0x7fb18006ebf0 2024-04-17T10:52:39.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb176ffd640 1 --2- 172.21.15.83:0/2100836674 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb180071ac0 0x7fb180152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:52:39.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb17f7fe640 1 --2- 172.21.15.83:0/2100836674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb18006ebf0 0x7fb1801413a0 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7fb17000d800 tx=0x7fb17000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:39.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb17cff9640 1 -- 172.21.15.83:0/2100836674 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb170018020 con 0x7fb18006ebf0 2024-04-17T10:52:39.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb17cff9640 1 -- 172.21.15.83:0/2100836674 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1700096c0 con 0x7fb18006ebf0 2024-04-17T10:52:39.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.186+0000 7fb186144640 1 -- 172.21.15.83:0/2100836674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb180153b00 con 0x7fb18006ebf0 2024-04-17T10:52:39.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.190+0000 7fb17cff9640 1 -- 172.21.15.83:0/2100836674 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb170009ab0 con 0x7fb18006ebf0 2024-04-17T10:52:39.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.190+0000 7fb186144640 1 -- 172.21.15.83:0/2100836674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb180151400 con 0x7fb18006ebf0 2024-04-17T10:52:39.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.190+0000 7fb186144640 1 -- 172.21.15.83:0/2100836674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb180074f90 con 0x7fb18006ebf0 2024-04-17T10:52:39.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.190+0000 7fb17cff9640 1 -- 172.21.15.83:0/2100836674 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb17001b020 con 0x7fb18006ebf0 2024-04-17T10:52:39.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.190+0000 7fb17cff9640 1 --2- 172.21.15.83:0/2100836674 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb160066f60 0x7fb160069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:39.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.190+0000 7fb17cff9640 1 -- 172.21.15.83:0/2100836674 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fb170085510 con 0x7fb18006ebf0 2024-04-17T10:52:39.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.190+0000 7fb176ffd640 1 --2- 172.21.15.83:0/2100836674 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb160066f60 0x7fb160069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:39.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.190+0000 7fb176ffd640 1 --2- 172.21.15.83:0/2100836674 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb160066f60 0x7fb160069420 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb1680ad000 tx=0x7fb168008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:39.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.194+0000 7fb17cff9640 1 -- 172.21.15.83:0/2100836674 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb170022020 con 0x7fb18006ebf0 2024-04-17T10:52:39.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.386+0000 7fb186144640 1 -- 172.21.15.83:0/2100836674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb18015e180 con 0x7fb18006ebf0 2024-04-17T10:52:39.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.390+0000 7fb17cff9640 1 -- 172.21.15.83:0/2100836674 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fb170056540 con 0x7fb18006ebf0 2024-04-17T10:52:39.384 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:39.384 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:39.384 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:39.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.390+0000 7fb186144640 1 -- 172.21.15.83:0/2100836674 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb160066f60 msgr2=0x7fb160069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:39.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.390+0000 7fb186144640 1 --2- 172.21.15.83:0/2100836674 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb160066f60 0x7fb160069420 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb1680ad000 tx=0x7fb168008040 comp rx=0 tx=0).stop 2024-04-17T10:52:39.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.390+0000 7fb186144640 1 -- 172.21.15.83:0/2100836674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb18006ebf0 msgr2=0x7fb1801413a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:39.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.390+0000 7fb186144640 1 --2- 172.21.15.83:0/2100836674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb18006ebf0 0x7fb1801413a0 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7fb17000d800 tx=0x7fb17000dcc0 comp rx=0 tx=0).stop 2024-04-17T10:52:39.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.390+0000 7fb186144640 1 -- 172.21.15.83:0/2100836674 shutdown_connections 2024-04-17T10:52:39.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.390+0000 7fb186144640 1 --2- 172.21.15.83:0/2100836674 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb160066f60 0x7fb160069420 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:39.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.390+0000 7fb186144640 1 --2- 172.21.15.83:0/2100836674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb18015a890 0x7fb1801534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:39.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.390+0000 7fb186144640 1 --2- 172.21.15.83:0/2100836674 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb180071ac0 0x7fb180152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:39.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.390+0000 7fb186144640 1 --2- 172.21.15.83:0/2100836674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb18006ebf0 0x7fb1801413a0 unknown :-1 s=CLOSED pgs=2241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:39.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.390+0000 7fb186144640 1 -- 172.21.15.83:0/2100836674 >> 172.21.15.83:0/2100836674 conn(0x7fb1800b2410 msgr2=0x7fb1800cd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:39.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.390+0000 7fb186144640 1 -- 172.21.15.83:0/2100836674 shutdown_connections 2024-04-17T10:52:39.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:39.390+0000 7fb186144640 1 -- 172.21.15.83:0/2100836674 wait complete. 2024-04-17T10:52:39.806 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:39.806 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:40.807 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:40.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.946+0000 7fbf52c42640 1 Processor -- start 2024-04-17T10:52:40.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.946+0000 7fbf52c42640 1 -- start start 2024-04-17T10:52:40.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.946+0000 7fbf52c42640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbf4c147680 0x7fbf4c071370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:40.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.946+0000 7fbf52c42640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf4c0c9430 0x7fbf4c167850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:40.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.946+0000 7fbf52c42640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf4c071b00 con 0x7fbf4c0cb460 2024-04-17T10:52:40.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.946+0000 7fbf52c42640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf4c0718b0 con 0x7fbf4c147680 2024-04-17T10:52:40.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.946+0000 7fbf52c42640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c071de0 con 0x7fbf4c0c9430 2024-04-17T10:52:40.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.946+0000 7fbf509de640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbf4c147680 0x7fbf4c071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:40.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.946+0000 7fbf509de640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbf4c147680 0x7fbf4c071370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36874/0 (socket says 172.21.15.83:36874) 2024-04-17T10:52:40.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.946+0000 7fbf509de640 1 -- 172.21.15.83:0/2001287811 learned_addr learned my addr 172.21.15.83:0/2001287811 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:40.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.946+0000 7fbf509de640 1 -- 172.21.15.83:0/2001287811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf4c0c9430 msgr2=0x7fbf4c167850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:40.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.946+0000 7fbf509de640 1 --2- 172.21.15.83:0/2001287811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf4c0c9430 0x7fbf4c167850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:40.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.946+0000 7fbf509de640 1 -- 172.21.15.83:0/2001287811 >> v1:172.21.15.167:6789/0 conn(0x7fbf4c0cb460 legacy=0x7fbf4c0c8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:40.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.946+0000 7fbf509de640 1 -- 172.21.15.83:0/2001287811 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf4c0720c0 con 0x7fbf4c147680 2024-04-17T10:52:40.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.950+0000 7fbf509de640 1 --2- 172.21.15.83:0/2001287811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf4c147680 0x7fbf4c071370 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009870 tx=0x7fbf3801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c9bf74ad4a2a4be server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:40.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.950+0000 7fbf4b7fe640 1 -- 172.21.15.83:0/2001287811 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf380ab5f0 con 0x7fbf4c147680 2024-04-17T10:52:40.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.950+0000 7fbf4b7fe640 1 -- 172.21.15.83:0/2001287811 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf380b0070 con 0x7fbf4c147680 2024-04-17T10:52:40.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.950+0000 7fbf4b7fe640 1 -- 172.21.15.83:0/2001287811 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf380ab9b0 con 0x7fbf4c147680 2024-04-17T10:52:40.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.950+0000 7fbf52c42640 1 -- 172.21.15.83:0/2001287811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf4c147680 msgr2=0x7fbf4c071370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:40.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.950+0000 7fbf52c42640 1 --2- 172.21.15.83:0/2001287811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf4c147680 0x7fbf4c071370 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009870 tx=0x7fbf3801c540 comp rx=0 tx=0).stop 2024-04-17T10:52:40.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.950+0000 7fbf52c42640 1 -- 172.21.15.83:0/2001287811 shutdown_connections 2024-04-17T10:52:40.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.950+0000 7fbf52c42640 1 --2- 172.21.15.83:0/2001287811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf4c0c9430 0x7fbf4c167850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:40.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.950+0000 7fbf52c42640 1 --2- 172.21.15.83:0/2001287811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf4c147680 0x7fbf4c071370 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:40.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.950+0000 7fbf52c42640 1 -- 172.21.15.83:0/2001287811 >> 172.21.15.83:0/2001287811 conn(0x7fbf4c0cc450 msgr2=0x7fbf4c0cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:40.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.950+0000 7fbf52c42640 1 -- 172.21.15.83:0/2001287811 shutdown_connections 2024-04-17T10:52:40.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.950+0000 7fbf52c42640 1 -- 172.21.15.83:0/2001287811 wait complete. 2024-04-17T10:52:40.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.950+0000 7fbf52c42640 1 Processor -- start 2024-04-17T10:52:40.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.950+0000 7fbf52c42640 1 -- start start 2024-04-17T10:52:40.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.950+0000 7fbf52c42640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf4c0c9430 0x7fbf4c1431e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:40.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.950+0000 7fbf52c42640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf4c0cb460 0x7fbf4c15f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:40.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.950+0000 7fbf52c42640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf4c147680 0x7fbf4c13e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:40.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.950+0000 7fbf52c42640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c16a730 con 0x7fbf4c147680 2024-04-17T10:52:40.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.950+0000 7fbf52c42640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c16a5b0 con 0x7fbf4c0c9430 2024-04-17T10:52:40.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.950+0000 7fbf52c42640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c16a8b0 con 0x7fbf4c0cb460 2024-04-17T10:52:40.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.950+0000 7fbf511df640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf4c147680 0x7fbf4c13e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:40.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.950+0000 7fbf511df640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf4c147680 0x7fbf4c13e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36888/0 (socket says 172.21.15.83:36888) 2024-04-17T10:52:40.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.950+0000 7fbf511df640 1 -- 172.21.15.83:0/3998808255 learned_addr learned my addr 172.21.15.83:0/3998808255 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:40.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.950+0000 7fbf509de640 1 --2- 172.21.15.83:0/3998808255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf4c0c9430 0x7fbf4c1431e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:40.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.950+0000 7fbf4bfff640 1 --2- 172.21.15.83:0/3998808255 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf4c0cb460 0x7fbf4c15f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:40.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.954+0000 7fbf511df640 1 -- 172.21.15.83:0/3998808255 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf4c0cb460 msgr2=0x7fbf4c15f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:40.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.954+0000 7fbf511df640 1 --2- 172.21.15.83:0/3998808255 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf4c0cb460 0x7fbf4c15f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:40.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.954+0000 7fbf511df640 1 -- 172.21.15.83:0/3998808255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf4c0c9430 msgr2=0x7fbf4c1431e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:40.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.954+0000 7fbf511df640 1 --2- 172.21.15.83:0/3998808255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf4c0c9430 0x7fbf4c1431e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:40.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.954+0000 7fbf511df640 1 -- 172.21.15.83:0/3998808255 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf4c143720 con 0x7fbf4c147680 2024-04-17T10:52:40.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.954+0000 7fbf511df640 1 --2- 172.21.15.83:0/3998808255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf4c147680 0x7fbf4c13e330 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7fbf4400b9e0 tx=0x7fbf4400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:40.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.954+0000 7fbf497fa640 1 -- 172.21.15.83:0/3998808255 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44019070 con 0x7fbf4c147680 2024-04-17T10:52:40.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.954+0000 7fbf497fa640 1 -- 172.21.15.83:0/3998808255 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4400cae0 con 0x7fbf4c147680 2024-04-17T10:52:40.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.954+0000 7fbf52c42640 1 -- 172.21.15.83:0/3998808255 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf4c15ffa0 con 0x7fbf4c147680 2024-04-17T10:52:40.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.954+0000 7fbf497fa640 1 -- 172.21.15.83:0/3998808255 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44014440 con 0x7fbf4c147680 2024-04-17T10:52:40.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.954+0000 7fbf52c42640 1 -- 172.21.15.83:0/3998808255 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf4c1fa960 con 0x7fbf4c147680 2024-04-17T10:52:40.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.954+0000 7fbf52c42640 1 -- 172.21.15.83:0/3998808255 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf4c074f90 con 0x7fbf4c147680 2024-04-17T10:52:40.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.954+0000 7fbf497fa640 1 -- 172.21.15.83:0/3998808255 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbf4401c070 con 0x7fbf4c147680 2024-04-17T10:52:40.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.954+0000 7fbf497fa640 1 --2- 172.21.15.83:0/3998808255 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbf30066f90 0x7fbf30069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:40.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.954+0000 7fbf509de640 1 --2- 172.21.15.83:0/3998808255 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbf30066f90 0x7fbf30069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:40.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.954+0000 7fbf497fa640 1 -- 172.21.15.83:0/3998808255 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fbf44084c00 con 0x7fbf4c147680 2024-04-17T10:52:40.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.954+0000 7fbf509de640 1 --2- 172.21.15.83:0/3998808255 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbf30066f90 0x7fbf30069450 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fbf380af000 tx=0x7fbf380a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:40.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:40.958+0000 7fbf497fa640 1 -- 172.21.15.83:0/3998808255 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbf4408b080 con 0x7fbf4c147680 2024-04-17T10:52:41.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:41.150+0000 7fbf52c42640 1 -- 172.21.15.83:0/3998808255 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf4c0b0330 con 0x7fbf4c147680 2024-04-17T10:52:41.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:41.150+0000 7fbf497fa640 1 -- 172.21.15.83:0/3998808255 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fbf44055c30 con 0x7fbf4c147680 2024-04-17T10:52:41.147 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:41.147 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:41.147 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:41.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:41.154+0000 7fbf52c42640 1 -- 172.21.15.83:0/3998808255 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbf30066f90 msgr2=0x7fbf30069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:41.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:41.154+0000 7fbf52c42640 1 --2- 172.21.15.83:0/3998808255 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbf30066f90 0x7fbf30069450 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fbf380af000 tx=0x7fbf380a7000 comp rx=0 tx=0).stop 2024-04-17T10:52:41.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:41.154+0000 7fbf52c42640 1 -- 172.21.15.83:0/3998808255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf4c147680 msgr2=0x7fbf4c13e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:41.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:41.154+0000 7fbf52c42640 1 --2- 172.21.15.83:0/3998808255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf4c147680 0x7fbf4c13e330 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7fbf4400b9e0 tx=0x7fbf4400bea0 comp rx=0 tx=0).stop 2024-04-17T10:52:41.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:41.154+0000 7fbf52c42640 1 -- 172.21.15.83:0/3998808255 shutdown_connections 2024-04-17T10:52:41.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:41.154+0000 7fbf52c42640 1 --2- 172.21.15.83:0/3998808255 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbf30066f90 0x7fbf30069450 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:41.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:41.154+0000 7fbf52c42640 1 --2- 172.21.15.83:0/3998808255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf4c147680 0x7fbf4c13e330 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:41.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:41.154+0000 7fbf52c42640 1 --2- 172.21.15.83:0/3998808255 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf4c0cb460 0x7fbf4c15f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:41.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:41.154+0000 7fbf52c42640 1 --2- 172.21.15.83:0/3998808255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf4c0c9430 0x7fbf4c1431e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:41.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:41.154+0000 7fbf52c42640 1 -- 172.21.15.83:0/3998808255 >> 172.21.15.83:0/3998808255 conn(0x7fbf4c0cc450 msgr2=0x7fbf4c143ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:41.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:41.154+0000 7fbf52c42640 1 -- 172.21.15.83:0/3998808255 shutdown_connections 2024-04-17T10:52:41.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:41.154+0000 7fbf52c42640 1 -- 172.21.15.83:0/3998808255 wait complete. 2024-04-17T10:52:41.569 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:41.569 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:42.570 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:42.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.706+0000 7efee0431640 1 Processor -- start 2024-04-17T10:52:42.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.706+0000 7efee0431640 1 -- start start 2024-04-17T10:52:42.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.706+0000 7efee0431640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efed80aecd0 0x7efed80cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:42.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.706+0000 7efee0431640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7efed8166550 0x7efed8071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:42.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.706+0000 7efee0431640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efed8072760 con 0x7efed80b3f80 2024-04-17T10:52:42.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.706+0000 7efee0431640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efed80cf810 con 0x7efed8166550 2024-04-17T10:52:42.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.706+0000 7efee0431640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed81669b0 con 0x7efed80aecd0 2024-04-17T10:52:42.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.706+0000 7efedd9cc640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7efed8166550 0x7efed8071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:42.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.706+0000 7efede1cd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efed80aecd0 0x7efed80cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:42.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.706+0000 7efede9ce640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7efed80b3f80 0x7efed80b2670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:33730/0 (socket says 172.21.15.83:33730) 2024-04-17T10:52:42.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.706+0000 7efede1cd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efed80aecd0 0x7efed80cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:33740/0 (socket says 172.21.15.83:33740) 2024-04-17T10:52:42.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.706+0000 7efede9ce640 1 -- 172.21.15.83:0/2310988916 learned_addr learned my addr 172.21.15.83:0/2310988916 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:52:42.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.706+0000 7efede1cd640 1 -- 172.21.15.83:0/2310988916 >> v2:172.21.15.167:3300/0 conn(0x7efed8166550 msgr2=0x7efed8071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:42.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.706+0000 7efede1cd640 1 --2- 172.21.15.83:0/2310988916 >> v2:172.21.15.167:3300/0 conn(0x7efed8166550 0x7efed8071be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:42.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.706+0000 7efede1cd640 1 -- 172.21.15.83:0/2310988916 >> v1:172.21.15.167:6789/0 conn(0x7efed80b3f80 legacy=0x7efed80b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:42.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.706+0000 7efede1cd640 1 -- 172.21.15.83:0/2310988916 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed80cf200 con 0x7efed80aecd0 2024-04-17T10:52:42.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.710+0000 7efede1cd640 1 --2- 172.21.15.83:0/2310988916 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efed80aecd0 0x7efed80cfca0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efed40097c0 tx=0x7efed401c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61d04fcedf0f69c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:42.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.710+0000 7efedd1cb640 1 -- 172.21.15.83:0/2310988916 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed40ab5f0 con 0x7efed80aecd0 2024-04-17T10:52:42.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.710+0000 7efedd1cb640 1 -- 172.21.15.83:0/2310988916 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed40b0070 con 0x7efed80aecd0 2024-04-17T10:52:42.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.710+0000 7efedd1cb640 1 -- 172.21.15.83:0/2310988916 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed40ab9b0 con 0x7efed80aecd0 2024-04-17T10:52:42.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.710+0000 7efee0431640 1 -- 172.21.15.83:0/2310988916 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efed80aecd0 msgr2=0x7efed80cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:42.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.710+0000 7efee0431640 1 --2- 172.21.15.83:0/2310988916 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efed80aecd0 0x7efed80cfca0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efed40097c0 tx=0x7efed401c3f0 comp rx=0 tx=0).stop 2024-04-17T10:52:42.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.710+0000 7efee0431640 1 -- 172.21.15.83:0/2310988916 shutdown_connections 2024-04-17T10:52:42.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.710+0000 7efee0431640 1 --2- 172.21.15.83:0/2310988916 >> v2:172.21.15.167:3300/0 conn(0x7efed8166550 0x7efed8071be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:42.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.710+0000 7efee0431640 1 --2- 172.21.15.83:0/2310988916 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efed80aecd0 0x7efed80cfca0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:42.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.710+0000 7efee0431640 1 -- 172.21.15.83:0/2310988916 >> 172.21.15.83:0/2310988916 conn(0x7efed806abe0 msgr2=0x7efed806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:42.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.710+0000 7efee0431640 1 -- 172.21.15.83:0/2310988916 shutdown_connections 2024-04-17T10:52:42.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.710+0000 7efee0431640 1 -- 172.21.15.83:0/2310988916 wait complete. 2024-04-17T10:52:42.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.710+0000 7efee0431640 1 Processor -- start 2024-04-17T10:52:42.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.714+0000 7efee0431640 1 -- start start 2024-04-17T10:52:42.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.714+0000 7efee0431640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efed80aecd0 0x7efed80ac360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:42.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.714+0000 7efee0431640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efed80b3f80 0x7efed80a9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:42.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.714+0000 7efee0431640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efed8166550 0x7efed80aa560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:42.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.714+0000 7efee0431640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed80b47a0 con 0x7efed8166550 2024-04-17T10:52:42.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.714+0000 7efee0431640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed8072120 con 0x7efed80aecd0 2024-04-17T10:52:42.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.714+0000 7efee0431640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed80b4920 con 0x7efed80b3f80 2024-04-17T10:52:42.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.714+0000 7efede9ce640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efed8166550 0x7efed80aa560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:42.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.714+0000 7efede9ce640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efed8166550 0x7efed80aa560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36902/0 (socket says 172.21.15.83:36902) 2024-04-17T10:52:42.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.714+0000 7efede9ce640 1 -- 172.21.15.83:0/2044661997 learned_addr learned my addr 172.21.15.83:0/2044661997 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:42.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.714+0000 7efedd9cc640 1 --2- 172.21.15.83:0/2044661997 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efed80b3f80 0x7efed80a9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:42.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.714+0000 7efede1cd640 1 --2- 172.21.15.83:0/2044661997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efed80aecd0 0x7efed80ac360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:42.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.714+0000 7efede9ce640 1 -- 172.21.15.83:0/2044661997 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efed80b3f80 msgr2=0x7efed80a9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:42.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.714+0000 7efede9ce640 1 --2- 172.21.15.83:0/2044661997 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efed80b3f80 0x7efed80a9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:42.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.714+0000 7efede9ce640 1 -- 172.21.15.83:0/2044661997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efed80aecd0 msgr2=0x7efed80ac360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:42.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.714+0000 7efede9ce640 1 --2- 172.21.15.83:0/2044661997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efed80aecd0 0x7efed80ac360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:42.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.714+0000 7efede9ce640 1 -- 172.21.15.83:0/2044661997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed80ade50 con 0x7efed8166550 2024-04-17T10:52:42.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.714+0000 7efede9ce640 1 --2- 172.21.15.83:0/2044661997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efed8166550 0x7efed80aa560 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7efed000caa0 tx=0x7efed000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:42.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.714+0000 7efeceffd640 1 -- 172.21.15.83:0/2044661997 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed0017070 con 0x7efed8166550 2024-04-17T10:52:42.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.714+0000 7efeceffd640 1 -- 172.21.15.83:0/2044661997 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed000ac00 con 0x7efed8166550 2024-04-17T10:52:42.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.714+0000 7efee0431640 1 -- 172.21.15.83:0/2044661997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed80ae130 con 0x7efed8166550 2024-04-17T10:52:42.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.714+0000 7efeceffd640 1 -- 172.21.15.83:0/2044661997 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed0012620 con 0x7efed8166550 2024-04-17T10:52:42.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.714+0000 7efee0431640 1 -- 172.21.15.83:0/2044661997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed80aaaa0 con 0x7efed8166550 2024-04-17T10:52:42.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.714+0000 7efee0431640 1 -- 172.21.15.83:0/2044661997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efed8074f90 con 0x7efed8166550 2024-04-17T10:52:42.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.714+0000 7efeceffd640 1 -- 172.21.15.83:0/2044661997 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efed001a070 con 0x7efed8166550 2024-04-17T10:52:42.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.714+0000 7efeceffd640 1 --2- 172.21.15.83:0/2044661997 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efebc066f60 0x7efebc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:42.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.714+0000 7efeceffd640 1 -- 172.21.15.83:0/2044661997 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7efed0084d10 con 0x7efed8166550 2024-04-17T10:52:42.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.714+0000 7efede1cd640 1 --2- 172.21.15.83:0/2044661997 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efebc066f60 0x7efebc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:42.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.718+0000 7efede1cd640 1 --2- 172.21.15.83:0/2044661997 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efebc066f60 0x7efebc069420 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7efed40af000 tx=0x7efed40a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:42.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.722+0000 7efeceffd640 1 -- 172.21.15.83:0/2044661997 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efed0055d40 con 0x7efed8166550 2024-04-17T10:52:42.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.906+0000 7efee0431640 1 -- 172.21.15.83:0/2044661997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efed80b3ad0 con 0x7efed8166550 2024-04-17T10:52:42.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.910+0000 7efeceffd640 1 -- 172.21.15.83:0/2044661997 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7efed0055640 con 0x7efed8166550 2024-04-17T10:52:42.904 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:42.904 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:42.904 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:42.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.910+0000 7efee0431640 1 -- 172.21.15.83:0/2044661997 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efebc066f60 msgr2=0x7efebc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:42.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.910+0000 7efee0431640 1 --2- 172.21.15.83:0/2044661997 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efebc066f60 0x7efebc069420 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7efed40af000 tx=0x7efed40a7000 comp rx=0 tx=0).stop 2024-04-17T10:52:42.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.910+0000 7efee0431640 1 -- 172.21.15.83:0/2044661997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efed8166550 msgr2=0x7efed80aa560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:42.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.910+0000 7efee0431640 1 --2- 172.21.15.83:0/2044661997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efed8166550 0x7efed80aa560 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7efed000caa0 tx=0x7efed000cf60 comp rx=0 tx=0).stop 2024-04-17T10:52:42.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.910+0000 7efee0431640 1 -- 172.21.15.83:0/2044661997 shutdown_connections 2024-04-17T10:52:42.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.910+0000 7efee0431640 1 --2- 172.21.15.83:0/2044661997 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7efebc066f60 0x7efebc069420 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:42.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.910+0000 7efee0431640 1 --2- 172.21.15.83:0/2044661997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efed8166550 0x7efed80aa560 unknown :-1 s=CLOSED pgs=2244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:42.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.910+0000 7efee0431640 1 --2- 172.21.15.83:0/2044661997 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efed80b3f80 0x7efed80a9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:42.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.910+0000 7efee0431640 1 --2- 172.21.15.83:0/2044661997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efed80aecd0 0x7efed80ac360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:42.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.910+0000 7efee0431640 1 -- 172.21.15.83:0/2044661997 >> 172.21.15.83:0/2044661997 conn(0x7efed806abe0 msgr2=0x7efed80b0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:42.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.910+0000 7efee0431640 1 -- 172.21.15.83:0/2044661997 shutdown_connections 2024-04-17T10:52:42.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:42.910+0000 7efee0431640 1 -- 172.21.15.83:0/2044661997 wait complete. 2024-04-17T10:52:43.324 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:43.324 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:44.325 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:44.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.458+0000 7feb08873640 1 Processor -- start 2024-04-17T10:52:44.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb08873640 1 -- start start 2024-04-17T10:52:44.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb08873640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7feb0415db50 0x7feb04070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:44.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb08873640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb0406dcb0 0x7feb040b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:44.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb08873640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb040709b0 con 0x7feb040aedd0 2024-04-17T10:52:44.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb08873640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb0415e440 con 0x7feb0415db50 2024-04-17T10:52:44.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb08873640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0415e720 con 0x7feb0406dcb0 2024-04-17T10:52:44.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb01d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7feb040aedd0 0x7feb0406d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58458/0 (socket says 172.21.15.83:58458) 2024-04-17T10:52:44.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb01d9b640 1 -- 172.21.15.83:0/2195297742 learned_addr learned my addr 172.21.15.83:0/2195297742 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:52:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb0259c640 1 --2- 172.21.15.83:0/2195297742 >> v2:172.21.15.167:3300/0 conn(0x7feb0415db50 0x7feb04070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb02d9d640 1 --2- 172.21.15.83:0/2195297742 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb0406dcb0 0x7feb040b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb0159a640 1 -- 172.21.15.83:0/2195297742 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 879155428 0 0) 0x7feaf8001120 con 0x7feb040aedd0 2024-04-17T10:52:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb0159a640 1 -- 172.21.15.83:0/2195297742 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaec003530 con 0x7feb040aedd0 2024-04-17T10:52:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb0159a640 1 -- 172.21.15.83:0/2195297742 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3667527736 0 0) 0x7feaec003530 con 0x7feb040aedd0 2024-04-17T10:52:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb0159a640 1 -- 172.21.15.83:0/2195297742 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb0406dcb0 msgr2=0x7feb040b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb0159a640 1 --2- 172.21.15.83:0/2195297742 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb0406dcb0 0x7feb040b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb0159a640 1 -- 172.21.15.83:0/2195297742 >> v2:172.21.15.167:3300/0 conn(0x7feb0415db50 msgr2=0x7feb04070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb0159a640 1 --2- 172.21.15.83:0/2195297742 >> v2:172.21.15.167:3300/0 conn(0x7feb0415db50 0x7feb04070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb0159a640 1 -- 172.21.15.83:0/2195297742 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0415ea00 con 0x7feb040aedd0 2024-04-17T10:52:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb0159a640 1 -- 172.21.15.83:0/2195297742 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7feaf8001850 con 0x7feb040aedd0 2024-04-17T10:52:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb0159a640 1 -- 172.21.15.83:0/2195297742 >> v1:172.21.15.83:6789/0 conn(0x7feb040aedd0 legacy=0x7feb0406d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb0159a640 1 --2- 172.21.15.83:0/2195297742 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb0406dcb0 0x7feaec0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb0159a640 1 --2- 172.21.15.83:0/2195297742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaec003f90 0x7feaec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb0159a640 1 --2- 172.21.15.83:0/2195297742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feaec004d50 0x7feaec0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb0159a640 1 -- 172.21.15.83:0/2195297742 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0415e440 con 0x7feaec003f90 2024-04-17T10:52:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb0159a640 1 -- 172.21.15.83:0/2195297742 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0415e720 con 0x7feaec004d50 2024-04-17T10:52:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb0159a640 1 -- 172.21.15.83:0/2195297742 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaec00b9d0 con 0x7feb0406dcb0 2024-04-17T10:52:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb01d9b640 1 --2- 172.21.15.83:0/2195297742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaec003f90 0x7feaec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb0259c640 1 --2- 172.21.15.83:0/2195297742 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb0406dcb0 0x7feaec0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb01d9b640 1 -- 172.21.15.83:0/2195297742 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb0406dcb0 msgr2=0x7feaec0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb01d9b640 1 --2- 172.21.15.83:0/2195297742 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb0406dcb0 0x7feaec0039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb01d9b640 1 -- 172.21.15.83:0/2195297742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feaec004d50 msgr2=0x7feaec0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb01d9b640 1 --2- 172.21.15.83:0/2195297742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feaec004d50 0x7feaec0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.462+0000 7feb01d9b640 1 -- 172.21.15.83:0/2195297742 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaec003530 con 0x7feaec003f90 2024-04-17T10:52:44.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.466+0000 7feb01d9b640 1 --2- 172.21.15.83:0/2195297742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaec003f90 0x7feaec004810 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7feaf800c4c0 tx=0x7feaf801f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd1611b3939f36 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:44.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.466+0000 7feb0159a640 1 -- 172.21.15.83:0/2195297742 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf800c0e0 con 0x7feaec003f90 2024-04-17T10:52:44.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.466+0000 7feb0159a640 1 -- 172.21.15.83:0/2195297742 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf8005ae0 con 0x7feaec003f90 2024-04-17T10:52:44.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.466+0000 7feb0159a640 1 -- 172.21.15.83:0/2195297742 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf80070f0 con 0x7feaec003f90 2024-04-17T10:52:44.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.466+0000 7feb08873640 1 -- 172.21.15.83:0/2195297742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaec003f90 msgr2=0x7feaec004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:44.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.466+0000 7feb08873640 1 --2- 172.21.15.83:0/2195297742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaec003f90 0x7feaec004810 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7feaf800c4c0 tx=0x7feaf801f330 comp rx=0 tx=0).stop 2024-04-17T10:52:44.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.466+0000 7feb02d9d640 1 -- 172.21.15.83:0/2195297742 reap_dead start 2024-04-17T10:52:44.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.466+0000 7feb08873640 1 -- 172.21.15.83:0/2195297742 shutdown_connections 2024-04-17T10:52:44.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.466+0000 7feb08873640 1 -- 172.21.15.83:0/2195297742 >> 172.21.15.83:0/2195297742 conn(0x7feb0406abe0 msgr2=0x7feb0406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:44.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.466+0000 7feb08873640 1 -- 172.21.15.83:0/2195297742 shutdown_connections 2024-04-17T10:52:44.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.466+0000 7feb08873640 1 -- 172.21.15.83:0/2195297742 wait complete. 2024-04-17T10:52:44.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.466+0000 7feb08873640 1 Processor -- start 2024-04-17T10:52:44.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.466+0000 7feb08873640 1 -- start start 2024-04-17T10:52:44.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.470+0000 7feb08873640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaec003f90 0x7feb040b2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:44.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.470+0000 7feb08873640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb04155670 0x7feb04155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:44.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.470+0000 7feb08873640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb04150700 0x7feb04150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:44.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.470+0000 7feb08873640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0415df30 con 0x7feaec003f90 2024-04-17T10:52:44.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.470+0000 7feb08873640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0415efc0 con 0x7feb04155670 2024-04-17T10:52:44.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.470+0000 7feb08873640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0415e0b0 con 0x7feb04150700 2024-04-17T10:52:44.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.470+0000 7feb0259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaec003f90 0x7feb040b2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:44.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.470+0000 7feb0259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaec003f90 0x7feb040b2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36920/0 (socket says 172.21.15.83:36920) 2024-04-17T10:52:44.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.470+0000 7feb0259c640 1 -- 172.21.15.83:0/2842965764 learned_addr learned my addr 172.21.15.83:0/2842965764 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:44.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.470+0000 7feb02d9d640 1 --2- 172.21.15.83:0/2842965764 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb04150700 0x7feb04150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:44.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.470+0000 7feb0259c640 1 -- 172.21.15.83:0/2842965764 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb04150700 msgr2=0x7feb04150bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:44.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.470+0000 7feb0259c640 1 --2- 172.21.15.83:0/2842965764 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb04150700 0x7feb04150bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:44.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.470+0000 7feb0259c640 1 -- 172.21.15.83:0/2842965764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb04155670 msgr2=0x7feb04155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:44.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.470+0000 7feb01d9b640 1 --2- 172.21.15.83:0/2842965764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb04155670 0x7feb04155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:44.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.470+0000 7feb0259c640 1 --2- 172.21.15.83:0/2842965764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb04155670 0x7feb04155ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:44.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.470+0000 7feb0259c640 1 -- 172.21.15.83:0/2842965764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb041513e0 con 0x7feaec003f90 2024-04-17T10:52:44.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.470+0000 7feb0259c640 1 --2- 172.21.15.83:0/2842965764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaec003f90 0x7feb040b2f70 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7feaf4010890 tx=0x7feaf4010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:44.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.470+0000 7feaf2ffd640 1 -- 172.21.15.83:0/2842965764 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf400f040 con 0x7feaec003f90 2024-04-17T10:52:44.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.470+0000 7feaf2ffd640 1 -- 172.21.15.83:0/2842965764 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf400eb60 con 0x7feaec003f90 2024-04-17T10:52:44.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.470+0000 7feb08873640 1 -- 172.21.15.83:0/2842965764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb041516c0 con 0x7feaec003f90 2024-04-17T10:52:44.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.470+0000 7feaf2ffd640 1 -- 172.21.15.83:0/2842965764 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf401c400 con 0x7feaec003f90 2024-04-17T10:52:44.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.470+0000 7feb08873640 1 -- 172.21.15.83:0/2842965764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb04151c80 con 0x7feaec003f90 2024-04-17T10:52:44.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.470+0000 7feb08873640 1 -- 172.21.15.83:0/2842965764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb04074f90 con 0x7feaec003f90 2024-04-17T10:52:44.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.470+0000 7feaf2ffd640 1 -- 172.21.15.83:0/2842965764 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feaf401c5a0 con 0x7feaec003f90 2024-04-17T10:52:44.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.470+0000 7feaf2ffd640 1 --2- 172.21.15.83:0/2842965764 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feae4066f90 0x7feae4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:44.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.470+0000 7feb01d9b640 1 --2- 172.21.15.83:0/2842965764 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feae4066f90 0x7feae4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:44.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.470+0000 7feaf2ffd640 1 -- 172.21.15.83:0/2842965764 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7feaf4085750 con 0x7feaec003f90 2024-04-17T10:52:44.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.474+0000 7feb01d9b640 1 --2- 172.21.15.83:0/2842965764 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feae4066f90 0x7feae4069450 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7feaf80ae000 tx=0x7feaf8000ff0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:44.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.474+0000 7feaf2ffd640 1 -- 172.21.15.83:0/2842965764 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feaf4018030 con 0x7feaec003f90 2024-04-17T10:52:44.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.670+0000 7feb08873640 1 -- 172.21.15.83:0/2842965764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb04156010 con 0x7feaec003f90 2024-04-17T10:52:44.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.670+0000 7feaf2ffd640 1 -- 172.21.15.83:0/2842965764 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7feaf40566f0 con 0x7feaec003f90 2024-04-17T10:52:44.665 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:44.665 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:44.665 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:44.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.670+0000 7feb08873640 1 -- 172.21.15.83:0/2842965764 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feae4066f90 msgr2=0x7feae4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:44.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.670+0000 7feb08873640 1 --2- 172.21.15.83:0/2842965764 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feae4066f90 0x7feae4069450 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7feaf80ae000 tx=0x7feaf8000ff0 comp rx=0 tx=0).stop 2024-04-17T10:52:44.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.674+0000 7feb08873640 1 -- 172.21.15.83:0/2842965764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaec003f90 msgr2=0x7feb040b2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:44.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.674+0000 7feb08873640 1 --2- 172.21.15.83:0/2842965764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaec003f90 0x7feb040b2f70 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7feaf4010890 tx=0x7feaf4010d50 comp rx=0 tx=0).stop 2024-04-17T10:52:44.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.674+0000 7feb08873640 1 -- 172.21.15.83:0/2842965764 shutdown_connections 2024-04-17T10:52:44.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.674+0000 7feb08873640 1 --2- 172.21.15.83:0/2842965764 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feae4066f90 0x7feae4069450 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.674+0000 7feb08873640 1 --2- 172.21.15.83:0/2842965764 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb04150700 0x7feb04150bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.674+0000 7feb08873640 1 --2- 172.21.15.83:0/2842965764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb04155670 0x7feb04155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.674+0000 7feb08873640 1 --2- 172.21.15.83:0/2842965764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feaec003f90 0x7feb040b2f70 unknown :-1 s=CLOSED pgs=2247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.674+0000 7feb08873640 1 -- 172.21.15.83:0/2842965764 >> 172.21.15.83:0/2842965764 conn(0x7feb0406abe0 msgr2=0x7feb0415cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.674+0000 7feb08873640 1 -- 172.21.15.83:0/2842965764 shutdown_connections 2024-04-17T10:52:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:44.674+0000 7feb08873640 1 -- 172.21.15.83:0/2842965764 wait complete. 2024-04-17T10:52:45.086 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:45.087 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:46.088 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:46.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.222+0000 7f65c3070640 1 Processor -- start 2024-04-17T10:52:46.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.222+0000 7f65c3070640 1 -- start start 2024-04-17T10:52:46.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.226+0000 7f65c3070640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f65bc15ba30 0x7f65bc072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:46.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.226+0000 7f65c3070640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65bc15c960 0x7f65bc0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:46.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.226+0000 7f65c3070640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65bc0cfea0 con 0x7f65bc0cfa10 2024-04-17T10:52:46.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.226+0000 7f65c3070640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65bc071ea0 con 0x7f65bc15ba30 2024-04-17T10:52:46.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.226+0000 7f65c3070640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc06e320 con 0x7f65bc15c960 2024-04-17T10:52:46.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.226+0000 7f65c0e0c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f65bc15ba30 0x7f65bc072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:46.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.226+0000 7f65bbfff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65bc15c960 0x7f65bc0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:46.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.226+0000 7f65c160d640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f65bc0cfa10 0x7f65bc06c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:33736/0 (socket says 172.21.15.83:33736) 2024-04-17T10:52:46.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.226+0000 7f65c160d640 1 -- 172.21.15.83:0/2648849521 learned_addr learned my addr 172.21.15.83:0/2648849521 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:52:46.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.226+0000 7f65c0e0c640 1 -- 172.21.15.83:0/2648849521 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65bc15c960 msgr2=0x7f65bc0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:46.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.226+0000 7f65c0e0c640 1 --2- 172.21.15.83:0/2648849521 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65bc15c960 0x7f65bc0cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:46.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.226+0000 7f65c0e0c640 1 -- 172.21.15.83:0/2648849521 >> v1:172.21.15.167:6789/0 conn(0x7f65bc0cfa10 legacy=0x7f65bc06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:46.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.226+0000 7f65c0e0c640 1 -- 172.21.15.83:0/2648849521 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65bc06e040 con 0x7f65bc15ba30 2024-04-17T10:52:46.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.230+0000 7f65c0e0c640 1 --2- 172.21.15.83:0/2648849521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65bc15ba30 0x7f65bc072330 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f65ac0097c0 tx=0x7f65ac01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d83b3724ce6ccb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:46.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.230+0000 7f65bb7fe640 1 -- 172.21.15.83:0/2648849521 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ac0b0070 con 0x7f65bc15ba30 2024-04-17T10:52:46.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.230+0000 7f65bb7fe640 1 -- 172.21.15.83:0/2648849521 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65ac0ab600 con 0x7f65bc15ba30 2024-04-17T10:52:46.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.230+0000 7f65bb7fe640 1 -- 172.21.15.83:0/2648849521 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ac0ab9d0 con 0x7f65bc15ba30 2024-04-17T10:52:46.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.230+0000 7f65c3070640 1 -- 172.21.15.83:0/2648849521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65bc15ba30 msgr2=0x7f65bc072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:46.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.230+0000 7f65c3070640 1 --2- 172.21.15.83:0/2648849521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65bc15ba30 0x7f65bc072330 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f65ac0097c0 tx=0x7f65ac01c410 comp rx=0 tx=0).stop 2024-04-17T10:52:46.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.230+0000 7f65c3070640 1 -- 172.21.15.83:0/2648849521 shutdown_connections 2024-04-17T10:52:46.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.230+0000 7f65c3070640 1 --2- 172.21.15.83:0/2648849521 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65bc15c960 0x7f65bc0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:46.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.230+0000 7f65c3070640 1 --2- 172.21.15.83:0/2648849521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65bc15ba30 0x7f65bc072330 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:46.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.230+0000 7f65c3070640 1 -- 172.21.15.83:0/2648849521 >> 172.21.15.83:0/2648849521 conn(0x7f65bc06abe0 msgr2=0x7f65bc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:46.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.230+0000 7f65c3070640 1 -- 172.21.15.83:0/2648849521 shutdown_connections 2024-04-17T10:52:46.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.230+0000 7f65c3070640 1 -- 172.21.15.83:0/2648849521 wait complete. 2024-04-17T10:52:46.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.230+0000 7f65c3070640 1 Processor -- start 2024-04-17T10:52:46.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.230+0000 7f65c3070640 1 -- start start 2024-04-17T10:52:46.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.234+0000 7f65c3070640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65bc0cfa10 0x7f65bc14a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:46.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.234+0000 7f65c3070640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65bc15ba30 0x7f65bc14a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:46.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.234+0000 7f65c3070640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65bc15c960 0x7f65bc14eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:46.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.234+0000 7f65c3070640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc06de20 con 0x7f65bc0cfa10 2024-04-17T10:52:46.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.234+0000 7f65c3070640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc06dca0 con 0x7f65bc15ba30 2024-04-17T10:52:46.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.234+0000 7f65c3070640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc15cd40 con 0x7f65bc15c960 2024-04-17T10:52:46.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.234+0000 7f65c0e0c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65bc0cfa10 0x7f65bc14a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:46.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.234+0000 7f65c0e0c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65bc0cfa10 0x7f65bc14a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36928/0 (socket says 172.21.15.83:36928) 2024-04-17T10:52:46.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.234+0000 7f65c0e0c640 1 -- 172.21.15.83:0/2636535468 learned_addr learned my addr 172.21.15.83:0/2636535468 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:46.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.234+0000 7f65c160d640 1 --2- 172.21.15.83:0/2636535468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65bc15c960 0x7f65bc14eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:46.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.234+0000 7f65bbfff640 1 --2- 172.21.15.83:0/2636535468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65bc15ba30 0x7f65bc14a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:46.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.234+0000 7f65c0e0c640 1 -- 172.21.15.83:0/2636535468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65bc15c960 msgr2=0x7f65bc14eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:46.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.234+0000 7f65c0e0c640 1 --2- 172.21.15.83:0/2636535468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65bc15c960 0x7f65bc14eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:46.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.234+0000 7f65c0e0c640 1 -- 172.21.15.83:0/2636535468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65bc15ba30 msgr2=0x7f65bc14a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:46.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.234+0000 7f65c0e0c640 1 --2- 172.21.15.83:0/2636535468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65bc15ba30 0x7f65bc14a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:46.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.234+0000 7f65c0e0c640 1 -- 172.21.15.83:0/2636535468 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65bc14dde0 con 0x7f65bc0cfa10 2024-04-17T10:52:46.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.234+0000 7f65bbfff640 1 --2- 172.21.15.83:0/2636535468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65bc15ba30 0x7f65bc14a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:52:46.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.234+0000 7f65c0e0c640 1 --2- 172.21.15.83:0/2636535468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65bc0cfa10 0x7f65bc14a070 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f65ac0ae040 tx=0x7f65ac0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:46.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.234+0000 7f65b97fa640 1 -- 172.21.15.83:0/2636535468 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ac0b0050 con 0x7f65bc0cfa10 2024-04-17T10:52:46.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.234+0000 7f65b97fa640 1 -- 172.21.15.83:0/2636535468 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65ac0bc070 con 0x7f65bc0cfa10 2024-04-17T10:52:46.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.234+0000 7f65b97fa640 1 -- 172.21.15.83:0/2636535468 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ac0b5870 con 0x7f65bc0cfa10 2024-04-17T10:52:46.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.234+0000 7f65c3070640 1 -- 172.21.15.83:0/2636535468 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65bc155090 con 0x7f65bc0cfa10 2024-04-17T10:52:46.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.234+0000 7f65c3070640 1 -- 172.21.15.83:0/2636535468 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65bc14d790 con 0x7f65bc0cfa10 2024-04-17T10:52:46.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.234+0000 7f65c3070640 1 -- 172.21.15.83:0/2636535468 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65bc074f90 con 0x7f65bc0cfa10 2024-04-17T10:52:46.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.234+0000 7f65b97fa640 1 -- 172.21.15.83:0/2636535468 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f65ac0b5a10 con 0x7f65bc0cfa10 2024-04-17T10:52:46.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.234+0000 7f65b97fa640 1 --2- 172.21.15.83:0/2636535468 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f65a8066f60 0x7f65a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:46.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.234+0000 7f65bbfff640 1 --2- 172.21.15.83:0/2636535468 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f65a8066f60 0x7f65a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:46.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.234+0000 7f65b97fa640 1 -- 172.21.15.83:0/2636535468 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f65ac11f710 con 0x7f65bc0cfa10 2024-04-17T10:52:46.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.234+0000 7f65bbfff640 1 --2- 172.21.15.83:0/2636535468 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f65a8066f60 0x7f65a8069420 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0cbca0 tx=0x7f65b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:46.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.238+0000 7f65b97fa640 1 -- 172.21.15.83:0/2636535468 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f65ac016090 con 0x7f65bc0cfa10 2024-04-17T10:52:46.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.426+0000 7f65c3070640 1 -- 172.21.15.83:0/2636535468 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65bc074480 con 0x7f65bc0cfa10 2024-04-17T10:52:46.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.426+0000 7f65b97fa640 1 -- 172.21.15.83:0/2636535468 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f65ac0f06b0 con 0x7f65bc0cfa10 2024-04-17T10:52:46.422 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:46.422 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:46.422 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:46.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.430+0000 7f65c3070640 1 -- 172.21.15.83:0/2636535468 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f65a8066f60 msgr2=0x7f65a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:46.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.430+0000 7f65c3070640 1 --2- 172.21.15.83:0/2636535468 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f65a8066f60 0x7f65a8069420 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0cbca0 tx=0x7f65b0008040 comp rx=0 tx=0).stop 2024-04-17T10:52:46.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.430+0000 7f65c3070640 1 -- 172.21.15.83:0/2636535468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65bc0cfa10 msgr2=0x7f65bc14a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:46.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.430+0000 7f65c3070640 1 --2- 172.21.15.83:0/2636535468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65bc0cfa10 0x7f65bc14a070 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f65ac0ae040 tx=0x7f65ac0191b0 comp rx=0 tx=0).stop 2024-04-17T10:52:46.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.430+0000 7f65c3070640 1 -- 172.21.15.83:0/2636535468 shutdown_connections 2024-04-17T10:52:46.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.430+0000 7f65c3070640 1 --2- 172.21.15.83:0/2636535468 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f65a8066f60 0x7f65a8069420 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:46.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.430+0000 7f65c3070640 1 --2- 172.21.15.83:0/2636535468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65bc15c960 0x7f65bc14eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:46.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.430+0000 7f65c3070640 1 --2- 172.21.15.83:0/2636535468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65bc15ba30 0x7f65bc14a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:46.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.430+0000 7f65c3070640 1 --2- 172.21.15.83:0/2636535468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65bc0cfa10 0x7f65bc14a070 unknown :-1 s=CLOSED pgs=2248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:46.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.430+0000 7f65c3070640 1 -- 172.21.15.83:0/2636535468 >> 172.21.15.83:0/2636535468 conn(0x7f65bc06abe0 msgr2=0x7f65bc0af760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:46.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.430+0000 7f65c3070640 1 -- 172.21.15.83:0/2636535468 shutdown_connections 2024-04-17T10:52:46.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:46.430+0000 7f65c3070640 1 -- 172.21.15.83:0/2636535468 wait complete. 2024-04-17T10:52:46.842 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:46.842 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:47.843 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:47.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.978+0000 7f470f148640 1 Processor -- start 2024-04-17T10:52:47.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.978+0000 7f470f148640 1 -- start start 2024-04-17T10:52:47.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.982+0000 7f470f148640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f470815ba30 0x7f4708072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:47.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.982+0000 7f470f148640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f47080cfbe0 0x7f470806c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:47.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.982+0000 7f470f148640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f470806e320 con 0x7f470815c960 2024-04-17T10:52:47.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.982+0000 7f470f148640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4708071ea0 con 0x7f470815ba30 2024-04-17T10:52:47.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.982+0000 7f470f148640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f470806e040 con 0x7f47080cfbe0 2024-04-17T10:52:47.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.982+0000 7f470cee4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f470815ba30 0x7f4708072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:47.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.982+0000 7f46fffff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f470815c960 0x7f47080cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53352/0 (socket says 172.21.15.83:53352) 2024-04-17T10:52:47.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.982+0000 7f46fffff640 1 -- 172.21.15.83:0/2175683303 learned_addr learned my addr 172.21.15.83:0/2175683303 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:52:47.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.982+0000 7f470cee4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f470815ba30 0x7f4708072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46698/0 (socket says 172.21.15.83:46698) 2024-04-17T10:52:47.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.982+0000 7f470d6e5640 1 --2- 172.21.15.83:0/2175683303 >> v2:172.21.15.167:3300/0 conn(0x7f47080cfbe0 0x7f470806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:47.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.982+0000 7f470cee4640 1 -- 172.21.15.83:0/2175683303 >> v2:172.21.15.167:3300/0 conn(0x7f47080cfbe0 msgr2=0x7f470806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:47.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.982+0000 7f470cee4640 1 --2- 172.21.15.83:0/2175683303 >> v2:172.21.15.167:3300/0 conn(0x7f47080cfbe0 0x7f470806c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:47.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.982+0000 7f470cee4640 1 -- 172.21.15.83:0/2175683303 >> v1:172.21.15.83:6789/0 conn(0x7f470815c960 legacy=0x7f47080cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:47.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.982+0000 7f470cee4640 1 -- 172.21.15.83:0/2175683303 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4708071850 con 0x7f470815ba30 2024-04-17T10:52:47.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.982+0000 7f470cee4640 1 --2- 172.21.15.83:0/2175683303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470815ba30 0x7f4708072330 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f47080676a0 tx=0x7f470401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96ab08a3130b2eb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:47.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.982+0000 7f46ff7fe640 1 -- 172.21.15.83:0/2175683303 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47040b0070 con 0x7f470815ba30 2024-04-17T10:52:47.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.982+0000 7f46ff7fe640 1 -- 172.21.15.83:0/2175683303 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47040ab600 con 0x7f470815ba30 2024-04-17T10:52:47.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.982+0000 7f46ff7fe640 1 -- 172.21.15.83:0/2175683303 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47040ab9d0 con 0x7f470815ba30 2024-04-17T10:52:47.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.986+0000 7f470f148640 1 -- 172.21.15.83:0/2175683303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470815ba30 msgr2=0x7f4708072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:47.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.986+0000 7f470f148640 1 --2- 172.21.15.83:0/2175683303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470815ba30 0x7f4708072330 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f47080676a0 tx=0x7f470401c3f0 comp rx=0 tx=0).stop 2024-04-17T10:52:47.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.986+0000 7f470f148640 1 -- 172.21.15.83:0/2175683303 shutdown_connections 2024-04-17T10:52:47.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.986+0000 7f470f148640 1 --2- 172.21.15.83:0/2175683303 >> v2:172.21.15.167:3300/0 conn(0x7f47080cfbe0 0x7f470806c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:47.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.986+0000 7f470f148640 1 --2- 172.21.15.83:0/2175683303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470815ba30 0x7f4708072330 unknown :-1 s=CLOSED pgs=2250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:47.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.986+0000 7f470f148640 1 -- 172.21.15.83:0/2175683303 >> 172.21.15.83:0/2175683303 conn(0x7f470806abe0 msgr2=0x7f470806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:47.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.986+0000 7f470f148640 1 -- 172.21.15.83:0/2175683303 shutdown_connections 2024-04-17T10:52:47.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.986+0000 7f470f148640 1 -- 172.21.15.83:0/2175683303 wait complete. 2024-04-17T10:52:47.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.986+0000 7f470f148640 1 Processor -- start 2024-04-17T10:52:47.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.986+0000 7f470f148640 1 -- start start 2024-04-17T10:52:47.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.986+0000 7f470f148640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47080cfbe0 0x7f470814afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:47.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.986+0000 7f470f148640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470815ba30 0x7f47081506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:47.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.986+0000 7f470f148640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f470815c960 0x7f4708149250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:47.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.986+0000 7f470f148640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470815cee0 con 0x7f470815ba30 2024-04-17T10:52:47.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.986+0000 7f470f148640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470815cd60 con 0x7f47080cfbe0 2024-04-17T10:52:47.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.986+0000 7f470f148640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4708168520 con 0x7f470815c960 2024-04-17T10:52:47.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.986+0000 7f46fffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470815ba30 0x7f47081506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:47.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.986+0000 7f46fffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470815ba30 0x7f47081506c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46708/0 (socket says 172.21.15.83:46708) 2024-04-17T10:52:47.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.986+0000 7f46fffff640 1 -- 172.21.15.83:0/1340109625 learned_addr learned my addr 172.21.15.83:0/1340109625 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:47.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.986+0000 7f46fffff640 1 -- 172.21.15.83:0/1340109625 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f470815c960 msgr2=0x7f4708149250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:47.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.986+0000 7f46fffff640 1 --2- 172.21.15.83:0/1340109625 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f470815c960 0x7f4708149250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:47.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.986+0000 7f470cee4640 1 --2- 172.21.15.83:0/1340109625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47080cfbe0 0x7f470814afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:47.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.986+0000 7f46fffff640 1 -- 172.21.15.83:0/1340109625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47080cfbe0 msgr2=0x7f470814afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:47.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.986+0000 7f46fffff640 1 --2- 172.21.15.83:0/1340109625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47080cfbe0 0x7f470814afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:47.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.986+0000 7f46fffff640 1 -- 172.21.15.83:0/1340109625 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4708149f40 con 0x7f470815ba30 2024-04-17T10:52:47.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.986+0000 7f46fffff640 1 --2- 172.21.15.83:0/1340109625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470815ba30 0x7f47081506c0 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f46f800c450 tx=0x7f46f800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:47.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.986+0000 7f46fd7fa640 1 -- 172.21.15.83:0/1340109625 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f8017070 con 0x7f470815ba30 2024-04-17T10:52:47.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.986+0000 7f46fd7fa640 1 -- 172.21.15.83:0/1340109625 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f8005cf0 con 0x7f470815ba30 2024-04-17T10:52:47.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.986+0000 7f470f148640 1 -- 172.21.15.83:0/1340109625 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f470814a220 con 0x7f470815ba30 2024-04-17T10:52:47.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.986+0000 7f46fd7fa640 1 -- 172.21.15.83:0/1340109625 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f8007290 con 0x7f470815ba30 2024-04-17T10:52:47.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.986+0000 7f470f148640 1 -- 172.21.15.83:0/1340109625 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f470814a7e0 con 0x7f470815ba30 2024-04-17T10:52:47.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.990+0000 7f470f148640 1 -- 172.21.15.83:0/1340109625 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4708074f90 con 0x7f470815ba30 2024-04-17T10:52:47.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.990+0000 7f46fd7fa640 1 -- 172.21.15.83:0/1340109625 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f46f801a070 con 0x7f470815ba30 2024-04-17T10:52:47.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.990+0000 7f46fd7fa640 1 --2- 172.21.15.83:0/1340109625 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f46f4066f60 0x7f46f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:47.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.990+0000 7f46fd7fa640 1 -- 172.21.15.83:0/1340109625 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f46f80842d0 con 0x7f470815ba30 2024-04-17T10:52:47.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.990+0000 7f470cee4640 1 --2- 172.21.15.83:0/1340109625 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f46f4066f60 0x7f46f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:47.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.990+0000 7f470cee4640 1 --2- 172.21.15.83:0/1340109625 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f46f4066f60 0x7f46f4069420 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f47040ad000 tx=0x7f4704008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:47.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:47.994+0000 7f46fd7fa640 1 -- 172.21.15.83:0/1340109625 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f46f8055300 con 0x7f470815ba30 2024-04-17T10:52:48.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:48.190+0000 7f470f148640 1 -- 172.21.15.83:0/1340109625 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4708152ce0 con 0x7f470815ba30 2024-04-17T10:52:48.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:48.194+0000 7f46fd7fa640 1 -- 172.21.15.83:0/1340109625 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f46f8054c00 con 0x7f470815ba30 2024-04-17T10:52:48.187 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:48.188 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:48.188 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:48.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:48.194+0000 7f470f148640 1 -- 172.21.15.83:0/1340109625 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f46f4066f60 msgr2=0x7f46f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:48.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:48.194+0000 7f470f148640 1 --2- 172.21.15.83:0/1340109625 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f46f4066f60 0x7f46f4069420 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f47040ad000 tx=0x7f4704008040 comp rx=0 tx=0).stop 2024-04-17T10:52:48.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:48.194+0000 7f470f148640 1 -- 172.21.15.83:0/1340109625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470815ba30 msgr2=0x7f47081506c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:48.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:48.194+0000 7f470f148640 1 --2- 172.21.15.83:0/1340109625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470815ba30 0x7f47081506c0 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f46f800c450 tx=0x7f46f800c910 comp rx=0 tx=0).stop 2024-04-17T10:52:48.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:48.194+0000 7f470f148640 1 -- 172.21.15.83:0/1340109625 shutdown_connections 2024-04-17T10:52:48.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:48.194+0000 7f470f148640 1 --2- 172.21.15.83:0/1340109625 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f46f4066f60 0x7f46f4069420 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:48.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:48.194+0000 7f470f148640 1 --2- 172.21.15.83:0/1340109625 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f470815c960 0x7f4708149250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:48.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:48.194+0000 7f470f148640 1 --2- 172.21.15.83:0/1340109625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f470815ba30 0x7f47081506c0 unknown :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:48.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:48.194+0000 7f470f148640 1 --2- 172.21.15.83:0/1340109625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47080cfbe0 0x7f470814afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:48.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:48.194+0000 7f470f148640 1 -- 172.21.15.83:0/1340109625 >> 172.21.15.83:0/1340109625 conn(0x7f470806abe0 msgr2=0x7f4708148a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:48.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:48.194+0000 7f470f148640 1 -- 172.21.15.83:0/1340109625 shutdown_connections 2024-04-17T10:52:48.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:48.194+0000 7f470f148640 1 -- 172.21.15.83:0/1340109625 wait complete. 2024-04-17T10:52:48.608 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:48.609 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:49.610 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:49.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.746+0000 7f2016155640 1 Processor -- start 2024-04-17T10:52:49.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.746+0000 7f2016155640 1 -- start start 2024-04-17T10:52:49.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.750+0000 7f2016155640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2010146a00 0x7f2010146e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:49.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.750+0000 7f2016155640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f20100b2d80 0x7f20101620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:49.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.750+0000 7f2016155640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f201006daf0 con 0x7f201006d690 2024-04-17T10:52:49.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.750+0000 7f2016155640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2010162610 con 0x7f20100b2d80 2024-04-17T10:52:49.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.750+0000 7f2016155640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f201006c960 con 0x7f2010146a00 2024-04-17T10:52:49.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.750+0000 7f200ffff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f20100b2d80 0x7f20101620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:49.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.750+0000 7f200f7fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2010146a00 0x7f2010146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:49.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.750+0000 7f200effd640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f201006d690 0x7f20100b2670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:36802/0 (socket says 172.21.15.83:36802) 2024-04-17T10:52:49.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.750+0000 7f200ffff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f20100b2d80 0x7f20101620d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:47130/0 (socket says 172.21.15.83:47130) 2024-04-17T10:52:49.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.750+0000 7f200effd640 1 -- 172.21.15.83:0/3294341458 learned_addr learned my addr 172.21.15.83:0/3294341458 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:52:49.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.750+0000 7f200ffff640 1 -- 172.21.15.83:0/3294341458 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2010146a00 msgr2=0x7f2010146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:49.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.750+0000 7f200ffff640 1 --2- 172.21.15.83:0/3294341458 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2010146a00 0x7f2010146e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:49.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.750+0000 7f200ffff640 1 -- 172.21.15.83:0/3294341458 >> v1:172.21.15.167:6789/0 conn(0x7f201006d690 legacy=0x7f20100b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:49.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.750+0000 7f200ffff640 1 -- 172.21.15.83:0/3294341458 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2010147570 con 0x7f20100b2d80 2024-04-17T10:52:49.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.750+0000 7f200f7fe640 1 --2- 172.21.15.83:0/3294341458 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2010146a00 0x7f2010146e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:52:49.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.754+0000 7f200ffff640 1 --2- 172.21.15.83:0/3294341458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20100b2d80 0x7f20101620d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2008009d90 tx=0x7f200801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d4898f3dd42db49 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:49.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.754+0000 7f200e7fc640 1 -- 172.21.15.83:0/3294341458 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20080b2070 con 0x7f20100b2d80 2024-04-17T10:52:49.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.754+0000 7f200e7fc640 1 -- 172.21.15.83:0/3294341458 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20080a5940 con 0x7f20100b2d80 2024-04-17T10:52:49.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.754+0000 7f200e7fc640 1 -- 172.21.15.83:0/3294341458 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20080a5d10 con 0x7f20100b2d80 2024-04-17T10:52:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.754+0000 7f2016155640 1 -- 172.21.15.83:0/3294341458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20100b2d80 msgr2=0x7f20101620d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.754+0000 7f2016155640 1 --2- 172.21.15.83:0/3294341458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20100b2d80 0x7f20101620d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2008009d90 tx=0x7f200801c970 comp rx=0 tx=0).stop 2024-04-17T10:52:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.754+0000 7f2016155640 1 -- 172.21.15.83:0/3294341458 shutdown_connections 2024-04-17T10:52:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.754+0000 7f2016155640 1 --2- 172.21.15.83:0/3294341458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20100b2d80 0x7f20101620d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.754+0000 7f2016155640 1 --2- 172.21.15.83:0/3294341458 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2010146a00 0x7f2010146e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.754+0000 7f2016155640 1 -- 172.21.15.83:0/3294341458 >> 172.21.15.83:0/3294341458 conn(0x7f201006abe0 msgr2=0x7f201006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.754+0000 7f2016155640 1 -- 172.21.15.83:0/3294341458 shutdown_connections 2024-04-17T10:52:49.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.754+0000 7f2016155640 1 -- 172.21.15.83:0/3294341458 wait complete. 2024-04-17T10:52:49.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.758+0000 7f2016155640 1 Processor -- start 2024-04-17T10:52:49.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.758+0000 7f2016155640 1 -- start start 2024-04-17T10:52:49.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.758+0000 7f2016155640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f201006d690 0x7f201015f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:49.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.758+0000 7f2016155640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20100b2d80 0x7f201015d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:49.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.758+0000 7f2016155640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2010146a00 0x7f2010145570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:49.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.758+0000 7f2016155640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2010071010 con 0x7f2010146a00 2024-04-17T10:52:49.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.758+0000 7f2016155640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2010070e90 con 0x7f201006d690 2024-04-17T10:52:49.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.758+0000 7f2016155640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2010140890 con 0x7f20100b2d80 2024-04-17T10:52:49.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.758+0000 7f200ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2010146a00 0x7f2010145570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:49.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.758+0000 7f200ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2010146a00 0x7f2010145570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46718/0 (socket says 172.21.15.83:46718) 2024-04-17T10:52:49.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.758+0000 7f200ffff640 1 -- 172.21.15.83:0/3705014158 learned_addr learned my addr 172.21.15.83:0/3705014158 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:49.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.758+0000 7f200f7fe640 1 --2- 172.21.15.83:0/3705014158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f201006d690 0x7f201015f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:49.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.758+0000 7f200effd640 1 --2- 172.21.15.83:0/3705014158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20100b2d80 0x7f201015d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:49.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.758+0000 7f200ffff640 1 -- 172.21.15.83:0/3705014158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20100b2d80 msgr2=0x7f201015d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:49.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.758+0000 7f200ffff640 1 --2- 172.21.15.83:0/3705014158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20100b2d80 0x7f201015d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:49.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.758+0000 7f200ffff640 1 -- 172.21.15.83:0/3705014158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f201006d690 msgr2=0x7f201015f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:49.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.758+0000 7f200ffff640 1 --2- 172.21.15.83:0/3705014158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f201006d690 0x7f201015f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:49.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.758+0000 7f200ffff640 1 -- 172.21.15.83:0/3705014158 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20100afba0 con 0x7f2010146a00 2024-04-17T10:52:49.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.758+0000 7f200f7fe640 1 --2- 172.21.15.83:0/3705014158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f201006d690 0x7f201015f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:52:49.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.758+0000 7f200effd640 1 --2- 172.21.15.83:0/3705014158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20100b2d80 0x7f201015d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:52:49.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.758+0000 7f200ffff640 1 --2- 172.21.15.83:0/3705014158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2010146a00 0x7f2010145570 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f200801b020 tx=0x7f2008019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:49.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.758+0000 7f2007fff640 1 -- 172.21.15.83:0/3705014158 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20080b2050 con 0x7f2010146a00 2024-04-17T10:52:49.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.758+0000 7f2007fff640 1 -- 172.21.15.83:0/3705014158 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20080be070 con 0x7f2010146a00 2024-04-17T10:52:49.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.758+0000 7f2016155640 1 -- 172.21.15.83:0/3705014158 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20100afe80 con 0x7f2010146a00 2024-04-17T10:52:49.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.758+0000 7f2007fff640 1 -- 172.21.15.83:0/3705014158 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20080ada50 con 0x7f2010146a00 2024-04-17T10:52:49.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.758+0000 7f2016155640 1 -- 172.21.15.83:0/3705014158 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20100b0440 con 0x7f2010146a00 2024-04-17T10:52:49.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.758+0000 7f2016155640 1 -- 172.21.15.83:0/3705014158 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2010074f90 con 0x7f2010146a00 2024-04-17T10:52:49.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.762+0000 7f2007fff640 1 -- 172.21.15.83:0/3705014158 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f20080adbf0 con 0x7f2010146a00 2024-04-17T10:52:49.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.762+0000 7f2007fff640 1 --2- 172.21.15.83:0/3705014158 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1ff0066f60 0x7f1ff0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:49.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.762+0000 7f200f7fe640 1 --2- 172.21.15.83:0/3705014158 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1ff0066f60 0x7f1ff0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:49.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.762+0000 7f2007fff640 1 -- 172.21.15.83:0/3705014158 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f20081201f0 con 0x7f2010146a00 2024-04-17T10:52:49.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.762+0000 7f200f7fe640 1 --2- 172.21.15.83:0/3705014158 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1ff0066f60 0x7f1ff0069420 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f20101f9cb0 tx=0x7f1ff8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.766+0000 7f2007fff640 1 -- 172.21.15.83:0/3705014158 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2008016560 con 0x7f2010146a00 2024-04-17T10:52:49.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.954+0000 7f2016155640 1 -- 172.21.15.83:0/3705014158 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2010160d30 con 0x7f2010146a00 2024-04-17T10:52:49.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.954+0000 7f2007fff640 1 -- 172.21.15.83:0/3705014158 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f20080f11a0 con 0x7f2010146a00 2024-04-17T10:52:49.949 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:49.949 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:49.950 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:49.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.958+0000 7f2016155640 1 -- 172.21.15.83:0/3705014158 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1ff0066f60 msgr2=0x7f1ff0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:49.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.958+0000 7f2016155640 1 --2- 172.21.15.83:0/3705014158 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1ff0066f60 0x7f1ff0069420 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f20101f9cb0 tx=0x7f1ff8008040 comp rx=0 tx=0).stop 2024-04-17T10:52:49.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.958+0000 7f2016155640 1 -- 172.21.15.83:0/3705014158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2010146a00 msgr2=0x7f2010145570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:49.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.958+0000 7f2016155640 1 --2- 172.21.15.83:0/3705014158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2010146a00 0x7f2010145570 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f200801b020 tx=0x7f2008019710 comp rx=0 tx=0).stop 2024-04-17T10:52:49.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.958+0000 7f2016155640 1 -- 172.21.15.83:0/3705014158 shutdown_connections 2024-04-17T10:52:49.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.958+0000 7f2016155640 1 --2- 172.21.15.83:0/3705014158 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1ff0066f60 0x7f1ff0069420 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:49.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.958+0000 7f2016155640 1 --2- 172.21.15.83:0/3705014158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2010146a00 0x7f2010145570 unknown :-1 s=CLOSED pgs=2252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:49.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.958+0000 7f2016155640 1 --2- 172.21.15.83:0/3705014158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20100b2d80 0x7f201015d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:49.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.958+0000 7f2016155640 1 --2- 172.21.15.83:0/3705014158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f201006d690 0x7f201015f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:49.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.958+0000 7f2016155640 1 -- 172.21.15.83:0/3705014158 >> 172.21.15.83:0/3705014158 conn(0x7f201006abe0 msgr2=0x7f201015fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:49.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.958+0000 7f2016155640 1 -- 172.21.15.83:0/3705014158 shutdown_connections 2024-04-17T10:52:49.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:49.958+0000 7f2016155640 1 -- 172.21.15.83:0/3705014158 wait complete. 2024-04-17T10:52:50.369 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:50.369 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:51.370 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:51.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.502+0000 7f270abf2640 1 Processor -- start 2024-04-17T10:52:51.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.502+0000 7f270abf2640 1 -- start start 2024-04-17T10:52:51.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.502+0000 7f270abf2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f270415ba30 0x7f2704072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:51.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.502+0000 7f270abf2640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f27040cfbe0 0x7f270406c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:51.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.502+0000 7f270abf2640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f270406e320 con 0x7f270415c960 2024-04-17T10:52:51.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.502+0000 7f270abf2640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2704071ea0 con 0x7f270415ba30 2024-04-17T10:52:51.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.502+0000 7f270abf2640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f270406e040 con 0x7f27040cfbe0 2024-04-17T10:52:51.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.502+0000 7f270898e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f270415ba30 0x7f2704072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:51.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.502+0000 7f2703fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f270415c960 0x7f27040cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53366/0 (socket says 172.21.15.83:53366) 2024-04-17T10:52:51.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.502+0000 7f2703fff640 1 -- 172.21.15.83:0/4290248324 learned_addr learned my addr 172.21.15.83:0/4290248324 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:52:51.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.502+0000 7f270898e640 1 -- 172.21.15.83:0/4290248324 >> v2:172.21.15.167:3300/0 conn(0x7f27040cfbe0 msgr2=0x7f270406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:51.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.502+0000 7f270898e640 1 --2- 172.21.15.83:0/4290248324 >> v2:172.21.15.167:3300/0 conn(0x7f27040cfbe0 0x7f270406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:51.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.502+0000 7f270898e640 1 -- 172.21.15.83:0/4290248324 >> v1:172.21.15.83:6789/0 conn(0x7f270415c960 legacy=0x7f27040cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:51.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.502+0000 7f270898e640 1 -- 172.21.15.83:0/4290248324 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2704071850 con 0x7f270415ba30 2024-04-17T10:52:51.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.502+0000 7f27037fe640 1 -- 172.21.15.83:0/4290248324 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1915299967 0 0) 0x7f270406e320 con 0x7f270415c960 2024-04-17T10:52:51.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f270898e640 1 --2- 172.21.15.83:0/4290248324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f270415ba30 0x7f2704072330 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f27040676a0 tx=0x7f26f801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6886f78dc7b2aadd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:51.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f27037fe640 1 -- 172.21.15.83:0/4290248324 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f80b0070 con 0x7f270415ba30 2024-04-17T10:52:51.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f27037fe640 1 -- 172.21.15.83:0/4290248324 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f80ab600 con 0x7f270415ba30 2024-04-17T10:52:51.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f27037fe640 1 -- 172.21.15.83:0/4290248324 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f80ab9d0 con 0x7f270415ba30 2024-04-17T10:52:51.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f270abf2640 1 -- 172.21.15.83:0/4290248324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f270415ba30 msgr2=0x7f2704072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:51.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f270abf2640 1 --2- 172.21.15.83:0/4290248324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f270415ba30 0x7f2704072330 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f27040676a0 tx=0x7f26f801c3f0 comp rx=0 tx=0).stop 2024-04-17T10:52:51.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f270abf2640 1 -- 172.21.15.83:0/4290248324 shutdown_connections 2024-04-17T10:52:51.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f270abf2640 1 --2- 172.21.15.83:0/4290248324 >> v2:172.21.15.167:3300/0 conn(0x7f27040cfbe0 0x7f270406c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:51.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f270abf2640 1 --2- 172.21.15.83:0/4290248324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f270415ba30 0x7f2704072330 unknown :-1 s=CLOSED pgs=2254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:51.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f270abf2640 1 -- 172.21.15.83:0/4290248324 >> 172.21.15.83:0/4290248324 conn(0x7f270406abe0 msgr2=0x7f270406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:51.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f270abf2640 1 -- 172.21.15.83:0/4290248324 shutdown_connections 2024-04-17T10:52:51.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f270abf2640 1 -- 172.21.15.83:0/4290248324 wait complete. 2024-04-17T10:52:51.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f270abf2640 1 Processor -- start 2024-04-17T10:52:51.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f270abf2640 1 -- start start 2024-04-17T10:52:51.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f270abf2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27040cfbe0 0x7f270414db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f270abf2640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f270415ba30 0x7f27040cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f270abf2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f270415c960 0x7f27040cee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f270abf2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270406de20 con 0x7f27040cfbe0 2024-04-17T10:52:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f270abf2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270406dca0 con 0x7f270415c960 2024-04-17T10:52:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f270abf2640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2704168520 con 0x7f270415ba30 2024-04-17T10:52:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f270898e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27040cfbe0 0x7f270414db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f270898e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27040cfbe0 0x7f270414db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46742/0 (socket says 172.21.15.83:46742) 2024-04-17T10:52:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f270898e640 1 -- 172.21.15.83:0/3905502827 learned_addr learned my addr 172.21.15.83:0/3905502827 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f270898e640 1 -- 172.21.15.83:0/3905502827 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f270415ba30 msgr2=0x7f27040cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f2703fff640 1 --2- 172.21.15.83:0/3905502827 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f270415ba30 0x7f27040cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f270918f640 1 --2- 172.21.15.83:0/3905502827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f270415c960 0x7f27040cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f270898e640 1 --2- 172.21.15.83:0/3905502827 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f270415ba30 0x7f27040cd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f270898e640 1 -- 172.21.15.83:0/3905502827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f270415c960 msgr2=0x7f27040cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:51.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f270898e640 1 --2- 172.21.15.83:0/3905502827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f270415c960 0x7f27040cee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:51.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f270898e640 1 -- 172.21.15.83:0/3905502827 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27040cd9e0 con 0x7f27040cfbe0 2024-04-17T10:52:51.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f270898e640 1 --2- 172.21.15.83:0/3905502827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27040cfbe0 0x7f270414db30 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f26f801a020 tx=0x7f26f801ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:51.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f27017fa640 1 -- 172.21.15.83:0/3905502827 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f80b0040 con 0x7f27040cfbe0 2024-04-17T10:52:51.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f27017fa640 1 -- 172.21.15.83:0/3905502827 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f8004230 con 0x7f27040cfbe0 2024-04-17T10:52:51.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.506+0000 7f27017fa640 1 -- 172.21.15.83:0/3905502827 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f8003620 con 0x7f27040cfbe0 2024-04-17T10:52:51.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.510+0000 7f270abf2640 1 -- 172.21.15.83:0/3905502827 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27041f7fc0 con 0x7f27040cfbe0 2024-04-17T10:52:51.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.510+0000 7f270abf2640 1 -- 172.21.15.83:0/3905502827 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27041f8580 con 0x7f27040cfbe0 2024-04-17T10:52:51.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.510+0000 7f270abf2640 1 -- 172.21.15.83:0/3905502827 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2704074f90 con 0x7f27040cfbe0 2024-04-17T10:52:51.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.510+0000 7f27017fa640 1 -- 172.21.15.83:0/3905502827 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f26f80ab4c0 con 0x7f27040cfbe0 2024-04-17T10:52:51.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.510+0000 7f27017fa640 1 --2- 172.21.15.83:0/3905502827 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f26e8066f60 0x7f26e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:51.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.510+0000 7f2703fff640 1 --2- 172.21.15.83:0/3905502827 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f26e8066f60 0x7f26e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:51.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.510+0000 7f27017fa640 1 -- 172.21.15.83:0/3905502827 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f26f811f840 con 0x7f27040cfbe0 2024-04-17T10:52:51.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.510+0000 7f2703fff640 1 --2- 172.21.15.83:0/3905502827 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f26e8066f60 0x7f26e8069420 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f27041f7a60 tx=0x7f26f0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:51.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.514+0000 7f27017fa640 1 -- 172.21.15.83:0/3905502827 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f26f8016070 con 0x7f27040cfbe0 2024-04-17T10:52:51.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.698+0000 7f270abf2640 1 -- 172.21.15.83:0/3905502827 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2704147a40 con 0x7f27040cfbe0 2024-04-17T10:52:51.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.698+0000 7f27017fa640 1 -- 172.21.15.83:0/3905502827 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f26f80f07e0 con 0x7f27040cfbe0 2024-04-17T10:52:51.693 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:51.693 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:51.693 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:51.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.698+0000 7f270abf2640 1 -- 172.21.15.83:0/3905502827 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f26e8066f60 msgr2=0x7f26e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:51.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.698+0000 7f270abf2640 1 --2- 172.21.15.83:0/3905502827 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f26e8066f60 0x7f26e8069420 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f27041f7a60 tx=0x7f26f0009040 comp rx=0 tx=0).stop 2024-04-17T10:52:51.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.698+0000 7f270abf2640 1 -- 172.21.15.83:0/3905502827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27040cfbe0 msgr2=0x7f270414db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:51.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.698+0000 7f270abf2640 1 --2- 172.21.15.83:0/3905502827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27040cfbe0 0x7f270414db30 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f26f801a020 tx=0x7f26f801ac30 comp rx=0 tx=0).stop 2024-04-17T10:52:51.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.698+0000 7f270abf2640 1 -- 172.21.15.83:0/3905502827 shutdown_connections 2024-04-17T10:52:51.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.698+0000 7f270abf2640 1 --2- 172.21.15.83:0/3905502827 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f26e8066f60 0x7f26e8069420 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:51.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.698+0000 7f270abf2640 1 --2- 172.21.15.83:0/3905502827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f270415c960 0x7f27040cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:51.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.698+0000 7f270abf2640 1 --2- 172.21.15.83:0/3905502827 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f270415ba30 0x7f27040cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:51.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.698+0000 7f270abf2640 1 --2- 172.21.15.83:0/3905502827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27040cfbe0 0x7f270414db30 unknown :-1 s=CLOSED pgs=2255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:51.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.698+0000 7f270abf2640 1 -- 172.21.15.83:0/3905502827 >> 172.21.15.83:0/3905502827 conn(0x7f270406abe0 msgr2=0x7f2704070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:51.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.698+0000 7f270abf2640 1 -- 172.21.15.83:0/3905502827 shutdown_connections 2024-04-17T10:52:51.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:51.698+0000 7f270abf2640 1 -- 172.21.15.83:0/3905502827 wait complete. 2024-04-17T10:52:52.112 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:52.112 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:53.113 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:53.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fcefb5a1640 1 Processor -- start 2024-04-17T10:52:53.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fcefb5a1640 1 -- start start 2024-04-17T10:52:53.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fcefb5a1640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcef415ba30 0x7fcef4072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:53.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fcefb5a1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcef40cfbe0 0x7fcef406c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:53.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fcefb5a1640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef406e320 con 0x7fcef415c960 2024-04-17T10:52:53.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fcefb5a1640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef4071ea0 con 0x7fcef415ba30 2024-04-17T10:52:53.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fcefb5a1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef406e040 con 0x7fcef40cfbe0 2024-04-17T10:52:53.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fcef8b3c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcef415c960 0x7fcef40cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53380/0 (socket says 172.21.15.83:53380) 2024-04-17T10:52:53.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fcef8b3c640 1 -- 172.21.15.83:0/153140909 learned_addr learned my addr 172.21.15.83:0/153140909 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:52:53.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fcef933d640 1 --2- 172.21.15.83:0/153140909 >> v2:172.21.15.167:3300/0 conn(0x7fcef415ba30 0x7fcef4072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:53.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fcef9b3e640 1 --2- 172.21.15.83:0/153140909 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcef40cfbe0 0x7fcef406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fceebfff640 1 -- 172.21.15.83:0/153140909 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3469874641 0 0) 0x7fcef406e320 con 0x7fcef415c960 2024-04-17T10:52:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fceebfff640 1 -- 172.21.15.83:0/153140909 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcee0003530 con 0x7fcef415c960 2024-04-17T10:52:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fceebfff640 1 -- 172.21.15.83:0/153140909 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1302651169 0 0) 0x7fcee0003530 con 0x7fcef415c960 2024-04-17T10:52:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fceebfff640 1 -- 172.21.15.83:0/153140909 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcef40cfbe0 msgr2=0x7fcef406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fceebfff640 1 --2- 172.21.15.83:0/153140909 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcef40cfbe0 0x7fcef406c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fceebfff640 1 -- 172.21.15.83:0/153140909 >> v2:172.21.15.167:3300/0 conn(0x7fcef415ba30 msgr2=0x7fcef4072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fceebfff640 1 --2- 172.21.15.83:0/153140909 >> v2:172.21.15.167:3300/0 conn(0x7fcef415ba30 0x7fcef4072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fceebfff640 1 -- 172.21.15.83:0/153140909 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef4071850 con 0x7fcef415c960 2024-04-17T10:52:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fceebfff640 1 -- 172.21.15.83:0/153140909 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fcee4001680 con 0x7fcef415c960 2024-04-17T10:52:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fceebfff640 1 -- 172.21.15.83:0/153140909 >> v1:172.21.15.83:6789/0 conn(0x7fcef415c960 legacy=0x7fcef40cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fceebfff640 1 --2- 172.21.15.83:0/153140909 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcef40cfbe0 0x7fcee00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fceebfff640 1 --2- 172.21.15.83:0/153140909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcee0003f40 0x7fcee0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:53.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fceebfff640 1 --2- 172.21.15.83:0/153140909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcee0004d50 0x7fcee00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:53.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fceebfff640 1 -- 172.21.15.83:0/153140909 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef4071ea0 con 0x7fcee0004d50 2024-04-17T10:52:53.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fceebfff640 1 -- 172.21.15.83:0/153140909 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef406e040 con 0x7fcee0003f40 2024-04-17T10:52:53.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fceebfff640 1 -- 172.21.15.83:0/153140909 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee000ba20 con 0x7fcef40cfbe0 2024-04-17T10:52:53.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fcef9b3e640 1 --2- 172.21.15.83:0/153140909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcee0004d50 0x7fcee00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:53.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fcef8b3c640 1 --2- 172.21.15.83:0/153140909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcee0003f40 0x7fcee0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:53.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fcef9b3e640 1 -- 172.21.15.83:0/153140909 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcef40cfbe0 msgr2=0x7fcee00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:53.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fcef9b3e640 1 --2- 172.21.15.83:0/153140909 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcef40cfbe0 0x7fcee00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:53.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fcef9b3e640 1 -- 172.21.15.83:0/153140909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcee0003f40 msgr2=0x7fcee0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:53.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fcef9b3e640 1 --2- 172.21.15.83:0/153140909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcee0003f40 0x7fcee0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:53.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.254+0000 7fcef9b3e640 1 -- 172.21.15.83:0/153140909 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee0003530 con 0x7fcee0004d50 2024-04-17T10:52:53.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.258+0000 7fcef9b3e640 1 --2- 172.21.15.83:0/153140909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcee0004d50 0x7fcee00051d0 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7fceec00e170 tx=0x7fceec01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf194c63cf963e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:53.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.258+0000 7fceebfff640 1 -- 172.21.15.83:0/153140909 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec0b2070 con 0x7fcee0004d50 2024-04-17T10:52:53.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.258+0000 7fceebfff640 1 -- 172.21.15.83:0/153140909 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceec0a5cb0 con 0x7fcee0004d50 2024-04-17T10:52:53.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.258+0000 7fceebfff640 1 -- 172.21.15.83:0/153140909 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec009c10 con 0x7fcee0004d50 2024-04-17T10:52:53.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.258+0000 7fcefb5a1640 1 -- 172.21.15.83:0/153140909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcee0004d50 msgr2=0x7fcee00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:53.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.258+0000 7fcefb5a1640 1 --2- 172.21.15.83:0/153140909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcee0004d50 0x7fcee00051d0 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7fceec00e170 tx=0x7fceec01cca0 comp rx=0 tx=0).stop 2024-04-17T10:52:53.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.258+0000 7fcef9b3e640 1 -- 172.21.15.83:0/153140909 reap_dead start 2024-04-17T10:52:53.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.258+0000 7fcefb5a1640 1 -- 172.21.15.83:0/153140909 shutdown_connections 2024-04-17T10:52:53.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.258+0000 7fcefb5a1640 1 -- 172.21.15.83:0/153140909 >> 172.21.15.83:0/153140909 conn(0x7fcef406abe0 msgr2=0x7fcef406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:53.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.258+0000 7fcefb5a1640 1 -- 172.21.15.83:0/153140909 shutdown_connections 2024-04-17T10:52:53.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.258+0000 7fcefb5a1640 1 -- 172.21.15.83:0/153140909 wait complete. 2024-04-17T10:52:53.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.258+0000 7fcefb5a1640 1 Processor -- start 2024-04-17T10:52:53.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.258+0000 7fcefb5a1640 1 -- start start 2024-04-17T10:52:53.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.258+0000 7fcefb5a1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcee0004d50 0x7fcef414b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:53.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.258+0000 7fcefb5a1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcef414c470 0x7fcef40b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:53.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.258+0000 7fcefb5a1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcef40b4440 0x7fcef415d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:53.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.258+0000 7fcefb5a1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef40aeae0 con 0x7fcef414c470 2024-04-17T10:52:53.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.258+0000 7fcefb5a1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef415cde0 con 0x7fcee0004d50 2024-04-17T10:52:53.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.258+0000 7fcefb5a1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef406dca0 con 0x7fcef40b4440 2024-04-17T10:52:53.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.258+0000 7fcef8b3c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcef414c470 0x7fcef40b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:53.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.258+0000 7fcef8b3c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcef414c470 0x7fcef40b3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46750/0 (socket says 172.21.15.83:46750) 2024-04-17T10:52:53.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.258+0000 7fcef8b3c640 1 -- 172.21.15.83:0/135970241 learned_addr learned my addr 172.21.15.83:0/135970241 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:53.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.262+0000 7fcef933d640 1 --2- 172.21.15.83:0/135970241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcee0004d50 0x7fcef414b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:53.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.262+0000 7fcef9b3e640 1 --2- 172.21.15.83:0/135970241 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcef40b4440 0x7fcef415d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:53.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.262+0000 7fcef8b3c640 1 -- 172.21.15.83:0/135970241 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcef40b4440 msgr2=0x7fcef415d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:53.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.262+0000 7fcef8b3c640 1 --2- 172.21.15.83:0/135970241 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcef40b4440 0x7fcef415d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:53.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.262+0000 7fcef8b3c640 1 -- 172.21.15.83:0/135970241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcee0004d50 msgr2=0x7fcef414b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:53.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.262+0000 7fcef8b3c640 1 --2- 172.21.15.83:0/135970241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcee0004d50 0x7fcef414b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:53.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.262+0000 7fcef8b3c640 1 -- 172.21.15.83:0/135970241 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef40b2b80 con 0x7fcef414c470 2024-04-17T10:52:53.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.262+0000 7fcef8b3c640 1 --2- 172.21.15.83:0/135970241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcef414c470 0x7fcef40b3f00 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7fcee4011950 tx=0x7fcee4011e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:53.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.262+0000 7fcee9ffb640 1 -- 172.21.15.83:0/135970241 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee4010040 con 0x7fcef414c470 2024-04-17T10:52:53.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.262+0000 7fcee9ffb640 1 -- 172.21.15.83:0/135970241 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee401a070 con 0x7fcef414c470 2024-04-17T10:52:53.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.262+0000 7fcefb5a1640 1 -- 172.21.15.83:0/135970241 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef40b2e60 con 0x7fcef414c470 2024-04-17T10:52:53.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.262+0000 7fcee9ffb640 1 -- 172.21.15.83:0/135970241 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee400fd40 con 0x7fcef414c470 2024-04-17T10:52:53.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.262+0000 7fcefb5a1640 1 -- 172.21.15.83:0/135970241 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef40b3420 con 0x7fcef414c470 2024-04-17T10:52:53.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.262+0000 7fcefb5a1640 1 -- 172.21.15.83:0/135970241 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef4074f90 con 0x7fcef414c470 2024-04-17T10:52:53.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.262+0000 7fcee9ffb640 1 -- 172.21.15.83:0/135970241 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcee401f460 con 0x7fcef414c470 2024-04-17T10:52:53.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.262+0000 7fcee9ffb640 1 --2- 172.21.15.83:0/135970241 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fced8066f60 0x7fced8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:53.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.262+0000 7fcee9ffb640 1 -- 172.21.15.83:0/135970241 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fcee40883f0 con 0x7fcef414c470 2024-04-17T10:52:53.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.262+0000 7fcef933d640 1 --2- 172.21.15.83:0/135970241 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fced8066f60 0x7fced8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:53.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.262+0000 7fcef933d640 1 --2- 172.21.15.83:0/135970241 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fced8066f60 0x7fced8069420 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fcef00047c0 tx=0x7fcef0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:53.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.266+0000 7fcee9ffb640 1 -- 172.21.15.83:0/135970241 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcee4059300 con 0x7fcef414c470 2024-04-17T10:52:53.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.462+0000 7fcefb5a1640 1 -- 172.21.15.83:0/135970241 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcef4158610 con 0x7fcef414c470 2024-04-17T10:52:53.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.462+0000 7fcee9ffb640 1 -- 172.21.15.83:0/135970241 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fcee4058c00 con 0x7fcef414c470 2024-04-17T10:52:53.456 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:53.456 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:53.456 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:53.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.462+0000 7fcefb5a1640 1 -- 172.21.15.83:0/135970241 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fced8066f60 msgr2=0x7fced8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:53.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.462+0000 7fcefb5a1640 1 --2- 172.21.15.83:0/135970241 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fced8066f60 0x7fced8069420 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fcef00047c0 tx=0x7fcef0008040 comp rx=0 tx=0).stop 2024-04-17T10:52:53.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.462+0000 7fcefb5a1640 1 -- 172.21.15.83:0/135970241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcef414c470 msgr2=0x7fcef40b3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:53.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.462+0000 7fcefb5a1640 1 --2- 172.21.15.83:0/135970241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcef414c470 0x7fcef40b3f00 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7fcee4011950 tx=0x7fcee4011e10 comp rx=0 tx=0).stop 2024-04-17T10:52:53.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.462+0000 7fcefb5a1640 1 -- 172.21.15.83:0/135970241 shutdown_connections 2024-04-17T10:52:53.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.462+0000 7fcefb5a1640 1 --2- 172.21.15.83:0/135970241 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fced8066f60 0x7fced8069420 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:53.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.462+0000 7fcefb5a1640 1 --2- 172.21.15.83:0/135970241 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcef40b4440 0x7fcef415d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:53.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.462+0000 7fcefb5a1640 1 --2- 172.21.15.83:0/135970241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcef414c470 0x7fcef40b3f00 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:53.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.462+0000 7fcefb5a1640 1 --2- 172.21.15.83:0/135970241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcee0004d50 0x7fcef414b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:53.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.462+0000 7fcefb5a1640 1 -- 172.21.15.83:0/135970241 >> 172.21.15.83:0/135970241 conn(0x7fcef406abe0 msgr2=0x7fcef415a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:53.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.462+0000 7fcefb5a1640 1 -- 172.21.15.83:0/135970241 shutdown_connections 2024-04-17T10:52:53.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:53.462+0000 7fcefb5a1640 1 -- 172.21.15.83:0/135970241 wait complete. 2024-04-17T10:52:53.878 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:53.879 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:54.880 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:55.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.018+0000 7fbee1324640 1 Processor -- start 2024-04-17T10:52:55.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.018+0000 7fbee1324640 1 -- start start 2024-04-17T10:52:55.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.018+0000 7fbee1324640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbedc0cfbe0 0x7fbedc06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:55.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.018+0000 7fbee1324640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbedc06e320 con 0x7fbedc15ba30 2024-04-17T10:52:55.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.018+0000 7fbee1324640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbedc06e040 con 0x7fbedc072330 2024-04-17T10:52:55.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.018+0000 7fbee1324640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc071ea0 con 0x7fbedc0cfbe0 2024-04-17T10:52:55.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.018+0000 7fbedaffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbedc15ba30 0x7fbedc15c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53384/0 (socket says 172.21.15.83:53384) 2024-04-17T10:52:55.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.018+0000 7fbedaffd640 1 -- 172.21.15.83:0/3433906831 learned_addr learned my addr 172.21.15.83:0/3433906831 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:52:55.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.018+0000 7fbed9ffb640 1 -- 172.21.15.83:0/3433906831 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 861722632 0 0) 0x7fbedc06e320 con 0x7fbedc15ba30 2024-04-17T10:52:55.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.018+0000 7fbed9ffb640 1 -- 172.21.15.83:0/3433906831 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbec4003530 con 0x7fbedc15ba30 2024-04-17T10:52:55.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.018+0000 7fbedb7fe640 1 --2- 172.21.15.83:0/3433906831 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbedc0cfbe0 0x7fbedc06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:55.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.018+0000 7fbed9ffb640 1 -- 172.21.15.83:0/3433906831 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 122949359 0 0) 0x7fbec4003530 con 0x7fbedc15ba30 2024-04-17T10:52:55.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.018+0000 7fbed9ffb640 1 -- 172.21.15.83:0/3433906831 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbedc0cfbe0 msgr2=0x7fbedc06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:55.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.018+0000 7fbed9ffb640 1 --2- 172.21.15.83:0/3433906831 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbedc0cfbe0 0x7fbedc06c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:55.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.018+0000 7fbed9ffb640 1 -- 172.21.15.83:0/3433906831 >> v1:172.21.15.167:6789/0 conn(0x7fbedc072330 legacy=0x7fbedc0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:55.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.018+0000 7fbed9ffb640 1 -- 172.21.15.83:0/3433906831 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbedc071850 con 0x7fbedc15ba30 2024-04-17T10:52:55.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.018+0000 7fbed9ffb640 1 -- 172.21.15.83:0/3433906831 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fbecc001680 con 0x7fbedc15ba30 2024-04-17T10:52:55.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.018+0000 7fbed9ffb640 1 -- 172.21.15.83:0/3433906831 >> v1:172.21.15.83:6789/0 conn(0x7fbedc15ba30 legacy=0x7fbedc15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:55.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.018+0000 7fbed9ffb640 1 --2- 172.21.15.83:0/3433906831 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbedc0cfbe0 0x7fbec40039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:55.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.018+0000 7fbed9ffb640 1 --2- 172.21.15.83:0/3433906831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbec4004020 0x7fbec4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:55.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.018+0000 7fbed9ffb640 1 --2- 172.21.15.83:0/3433906831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbec4004d50 0x7fbec40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:55.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.018+0000 7fbed9ffb640 1 -- 172.21.15.83:0/3433906831 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc071ea0 con 0x7fbec4004020 2024-04-17T10:52:55.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.018+0000 7fbed9ffb640 1 -- 172.21.15.83:0/3433906831 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec400bc10 con 0x7fbec4004d50 2024-04-17T10:52:55.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.018+0000 7fbed9ffb640 1 -- 172.21.15.83:0/3433906831 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec400be10 con 0x7fbedc0cfbe0 2024-04-17T10:52:55.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.018+0000 7fbeda7fc640 1 --2- 172.21.15.83:0/3433906831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbec4004020 0x7fbec4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:55.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.018+0000 7fbedaffd640 1 --2- 172.21.15.83:0/3433906831 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbedc0cfbe0 0x7fbec40039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:55.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.018+0000 7fbedb7fe640 1 --2- 172.21.15.83:0/3433906831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbec4004d50 0x7fbec40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:55.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.018+0000 7fbeda7fc640 1 -- 172.21.15.83:0/3433906831 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbedc0cfbe0 msgr2=0x7fbec40039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:55.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.018+0000 7fbeda7fc640 1 --2- 172.21.15.83:0/3433906831 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbedc0cfbe0 0x7fbec40039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:55.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.018+0000 7fbeda7fc640 1 -- 172.21.15.83:0/3433906831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbec4004d50 msgr2=0x7fbec40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:55.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.018+0000 7fbeda7fc640 1 --2- 172.21.15.83:0/3433906831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbec4004d50 0x7fbec40051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:55.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.018+0000 7fbeda7fc640 1 -- 172.21.15.83:0/3433906831 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec4003530 con 0x7fbec4004020 2024-04-17T10:52:55.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.022+0000 7fbeda7fc640 1 --2- 172.21.15.83:0/3433906831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbec4004020 0x7fbec4004810 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7fbed0001780 tx=0x7fbed001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc88fa9338118ed server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:55.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.022+0000 7fbed9ffb640 1 -- 172.21.15.83:0/3433906831 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed00013a0 con 0x7fbec4004020 2024-04-17T10:52:55.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.022+0000 7fbed9ffb640 1 -- 172.21.15.83:0/3433906831 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed00052f0 con 0x7fbec4004020 2024-04-17T10:52:55.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.022+0000 7fbed9ffb640 1 -- 172.21.15.83:0/3433906831 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed0003840 con 0x7fbec4004020 2024-04-17T10:52:55.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.022+0000 7fbee1324640 1 -- 172.21.15.83:0/3433906831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbec4004020 msgr2=0x7fbec4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:55.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.022+0000 7fbee1324640 1 --2- 172.21.15.83:0/3433906831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbec4004020 0x7fbec4004810 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7fbed0001780 tx=0x7fbed001c420 comp rx=0 tx=0).stop 2024-04-17T10:52:55.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.022+0000 7fbedb7fe640 1 -- 172.21.15.83:0/3433906831 reap_dead start 2024-04-17T10:52:55.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.022+0000 7fbee1324640 1 -- 172.21.15.83:0/3433906831 shutdown_connections 2024-04-17T10:52:55.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.022+0000 7fbee1324640 1 -- 172.21.15.83:0/3433906831 >> 172.21.15.83:0/3433906831 conn(0x7fbedc06abe0 msgr2=0x7fbedc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:55.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.022+0000 7fbee1324640 1 -- 172.21.15.83:0/3433906831 shutdown_connections 2024-04-17T10:52:55.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.022+0000 7fbee1324640 1 -- 172.21.15.83:0/3433906831 wait complete. 2024-04-17T10:52:55.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.022+0000 7fbee1324640 1 Processor -- start 2024-04-17T10:52:55.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.022+0000 7fbee1324640 1 -- start start 2024-04-17T10:52:55.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.026+0000 7fbee1324640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbec4004020 0x7fbedc1f7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:55.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.026+0000 7fbee1324640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbedc1492e0 0x7fbedc151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:55.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.026+0000 7fbee1324640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbedc0cf030 0x7fbedc13ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:55.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.026+0000 7fbee1324640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc06ce20 con 0x7fbedc0cf030 2024-04-17T10:52:55.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.026+0000 7fbee1324640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc06cca0 con 0x7fbec4004020 2024-04-17T10:52:55.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.026+0000 7fbee1324640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc06cfa0 con 0x7fbedc1492e0 2024-04-17T10:52:55.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.026+0000 7fbedb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbedc0cf030 0x7fbedc13ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:55.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.026+0000 7fbedb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbedc0cf030 0x7fbedc13ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46768/0 (socket says 172.21.15.83:46768) 2024-04-17T10:52:55.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.026+0000 7fbedb7fe640 1 -- 172.21.15.83:0/244430823 learned_addr learned my addr 172.21.15.83:0/244430823 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:55.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.026+0000 7fbeda7fc640 1 --2- 172.21.15.83:0/244430823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbedc1492e0 0x7fbedc151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:55.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.026+0000 7fbedaffd640 1 --2- 172.21.15.83:0/244430823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbec4004020 0x7fbedc1f7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:55.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.026+0000 7fbedb7fe640 1 -- 172.21.15.83:0/244430823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbedc1492e0 msgr2=0x7fbedc151ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:55.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.026+0000 7fbedb7fe640 1 --2- 172.21.15.83:0/244430823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbedc1492e0 0x7fbedc151ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:55.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.026+0000 7fbedb7fe640 1 -- 172.21.15.83:0/244430823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbec4004020 msgr2=0x7fbedc1f7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:55.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.026+0000 7fbedb7fe640 1 --2- 172.21.15.83:0/244430823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbec4004020 0x7fbedc1f7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:55.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.026+0000 7fbedb7fe640 1 -- 172.21.15.83:0/244430823 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbedc1f88a0 con 0x7fbedc0cf030 2024-04-17T10:52:55.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.026+0000 7fbedb7fe640 1 --2- 172.21.15.83:0/244430823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbedc0cf030 0x7fbedc13ade0 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7fbedc072c70 tx=0x7fbed40049a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:55.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.026+0000 7fbec37fe640 1 -- 172.21.15.83:0/244430823 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed4019070 con 0x7fbedc0cf030 2024-04-17T10:52:55.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.026+0000 7fbec37fe640 1 -- 172.21.15.83:0/244430823 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed4007d10 con 0x7fbedc0cf030 2024-04-17T10:52:55.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.026+0000 7fbee1324640 1 -- 172.21.15.83:0/244430823 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbedc1f82e0 con 0x7fbedc0cf030 2024-04-17T10:52:55.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.026+0000 7fbec37fe640 1 -- 172.21.15.83:0/244430823 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed4014640 con 0x7fbedc0cf030 2024-04-17T10:52:55.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.026+0000 7fbee1324640 1 -- 172.21.15.83:0/244430823 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbedc0ce830 con 0x7fbedc0cf030 2024-04-17T10:52:55.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.026+0000 7fbee1324640 1 -- 172.21.15.83:0/244430823 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbedc074f90 con 0x7fbedc0cf030 2024-04-17T10:52:55.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.026+0000 7fbec37fe640 1 -- 172.21.15.83:0/244430823 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbed401c070 con 0x7fbedc0cf030 2024-04-17T10:52:55.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.026+0000 7fbec37fe640 1 --2- 172.21.15.83:0/244430823 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbec8066f90 0x7fbec8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:55.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.026+0000 7fbec37fe640 1 -- 172.21.15.83:0/244430823 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fbed4084bb0 con 0x7fbedc0cf030 2024-04-17T10:52:55.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.026+0000 7fbedaffd640 1 --2- 172.21.15.83:0/244430823 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbec8066f90 0x7fbec8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:55.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.026+0000 7fbedaffd640 1 --2- 172.21.15.83:0/244430823 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbec8066f90 0x7fbec8069450 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fbecc001ec0 tx=0x7fbecc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:55.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.030+0000 7fbec37fe640 1 -- 172.21.15.83:0/244430823 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbed4055500 con 0x7fbedc0cf030 2024-04-17T10:52:55.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.222+0000 7fbee1324640 1 -- 172.21.15.83:0/244430823 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbedc153cf0 con 0x7fbedc0cf030 2024-04-17T10:52:55.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.222+0000 7fbec37fe640 1 -- 172.21.15.83:0/244430823 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fbed40595a0 con 0x7fbedc0cf030 2024-04-17T10:52:55.216 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:55.216 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:55.216 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:55.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.222+0000 7fbee1324640 1 -- 172.21.15.83:0/244430823 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbec8066f90 msgr2=0x7fbec8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:55.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.222+0000 7fbee1324640 1 --2- 172.21.15.83:0/244430823 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbec8066f90 0x7fbec8069450 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fbecc001ec0 tx=0x7fbecc009040 comp rx=0 tx=0).stop 2024-04-17T10:52:55.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.222+0000 7fbee1324640 1 -- 172.21.15.83:0/244430823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbedc0cf030 msgr2=0x7fbedc13ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:55.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.222+0000 7fbee1324640 1 --2- 172.21.15.83:0/244430823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbedc0cf030 0x7fbedc13ade0 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7fbedc072c70 tx=0x7fbed40049a0 comp rx=0 tx=0).stop 2024-04-17T10:52:55.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.222+0000 7fbee1324640 1 -- 172.21.15.83:0/244430823 shutdown_connections 2024-04-17T10:52:55.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.222+0000 7fbee1324640 1 --2- 172.21.15.83:0/244430823 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbec8066f90 0x7fbec8069450 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:55.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.222+0000 7fbee1324640 1 --2- 172.21.15.83:0/244430823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbedc0cf030 0x7fbedc13ade0 unknown :-1 s=CLOSED pgs=2261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:55.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.222+0000 7fbee1324640 1 --2- 172.21.15.83:0/244430823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbedc1492e0 0x7fbedc151ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:55.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.222+0000 7fbee1324640 1 --2- 172.21.15.83:0/244430823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbec4004020 0x7fbedc1f7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:55.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.222+0000 7fbee1324640 1 -- 172.21.15.83:0/244430823 >> 172.21.15.83:0/244430823 conn(0x7fbedc06abe0 msgr2=0x7fbedc1599b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:55.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.222+0000 7fbee1324640 1 -- 172.21.15.83:0/244430823 shutdown_connections 2024-04-17T10:52:55.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:55.222+0000 7fbee1324640 1 -- 172.21.15.83:0/244430823 wait complete. 2024-04-17T10:52:55.636 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:55.636 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:56.637 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:56.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce74125640 1 Processor -- start 2024-04-17T10:52:56.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce74125640 1 -- start start 2024-04-17T10:52:56.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce74125640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fce6c15de90 0x7fce6c15efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:56.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce74125640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce6c0ce7f0 0x7fce6c0cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:56.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce74125640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce6c06bd90 con 0x7fce6c0ceca0 2024-04-17T10:52:56.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce74125640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce6c06c030 con 0x7fce6c15de90 2024-04-17T10:52:56.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce74125640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c06c310 con 0x7fce6c0ce7f0 2024-04-17T10:52:56.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce716c0640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fce6c0ceca0 0x7fce6c0ce0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53988/0 (socket says 172.21.15.83:53988) 2024-04-17T10:52:56.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce716c0640 1 -- 172.21.15.83:0/1843471318 learned_addr learned my addr 172.21.15.83:0/1843471318 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:52:56.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce71ec1640 1 --2- 172.21.15.83:0/1843471318 >> v2:172.21.15.167:3300/0 conn(0x7fce6c15de90 0x7fce6c15efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:56.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce726c2640 1 --2- 172.21.15.83:0/1843471318 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce6c0ce7f0 0x7fce6c0cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:56.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce70ebf640 1 -- 172.21.15.83:0/1843471318 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3092844043 0 0) 0x7fce6c06bd90 con 0x7fce6c0ceca0 2024-04-17T10:52:56.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce70ebf640 1 -- 172.21.15.83:0/1843471318 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce58003530 con 0x7fce6c0ceca0 2024-04-17T10:52:56.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce70ebf640 1 -- 172.21.15.83:0/1843471318 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3233656031 0 0) 0x7fce58003530 con 0x7fce6c0ceca0 2024-04-17T10:52:56.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce70ebf640 1 -- 172.21.15.83:0/1843471318 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce6c0ce7f0 msgr2=0x7fce6c0cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:56.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce70ebf640 1 --2- 172.21.15.83:0/1843471318 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce6c0ce7f0 0x7fce6c0cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:56.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce70ebf640 1 -- 172.21.15.83:0/1843471318 >> v2:172.21.15.167:3300/0 conn(0x7fce6c15de90 msgr2=0x7fce6c15efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:56.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce70ebf640 1 --2- 172.21.15.83:0/1843471318 >> v2:172.21.15.167:3300/0 conn(0x7fce6c15de90 0x7fce6c15efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:56.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce70ebf640 1 -- 172.21.15.83:0/1843471318 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6c06c5f0 con 0x7fce6c0ceca0 2024-04-17T10:52:56.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce70ebf640 1 -- 172.21.15.83:0/1843471318 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fce5c0016a0 con 0x7fce6c0ceca0 2024-04-17T10:52:56.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce70ebf640 1 -- 172.21.15.83:0/1843471318 >> v1:172.21.15.83:6789/0 conn(0x7fce6c0ceca0 legacy=0x7fce6c0ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:56.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce70ebf640 1 --2- 172.21.15.83:0/1843471318 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce6c0ce7f0 0x7fce580039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:56.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce70ebf640 1 --2- 172.21.15.83:0/1843471318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce58003f40 0x7fce58004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:56.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce70ebf640 1 --2- 172.21.15.83:0/1843471318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce58004d50 0x7fce580051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:56.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce70ebf640 1 -- 172.21.15.83:0/1843471318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c06c030 con 0x7fce58003f40 2024-04-17T10:52:56.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce70ebf640 1 -- 172.21.15.83:0/1843471318 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c06c310 con 0x7fce58004d50 2024-04-17T10:52:56.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce70ebf640 1 -- 172.21.15.83:0/1843471318 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce5800b9d0 con 0x7fce6c0ce7f0 2024-04-17T10:52:56.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce716c0640 1 --2- 172.21.15.83:0/1843471318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce58003f40 0x7fce58004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:56.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce71ec1640 1 --2- 172.21.15.83:0/1843471318 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce6c0ce7f0 0x7fce580039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:56.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce726c2640 1 --2- 172.21.15.83:0/1843471318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce58004d50 0x7fce580051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:56.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce716c0640 1 -- 172.21.15.83:0/1843471318 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce6c0ce7f0 msgr2=0x7fce580039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:56.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce716c0640 1 --2- 172.21.15.83:0/1843471318 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce6c0ce7f0 0x7fce580039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:56.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce716c0640 1 -- 172.21.15.83:0/1843471318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce58004d50 msgr2=0x7fce580051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:56.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce716c0640 1 --2- 172.21.15.83:0/1843471318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce58004d50 0x7fce580051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:56.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce716c0640 1 -- 172.21.15.83:0/1843471318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce58003530 con 0x7fce58003f40 2024-04-17T10:52:56.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.774+0000 7fce71ec1640 1 --2- 172.21.15.83:0/1843471318 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce6c0ce7f0 0x7fce580039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:52:56.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.778+0000 7fce716c0640 1 --2- 172.21.15.83:0/1843471318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce58003f40 0x7fce58004810 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7fce6c06bf60 tx=0x7fce5c01f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1940c51c7196eb4b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:56.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.778+0000 7fce70ebf640 1 -- 172.21.15.83:0/1843471318 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c007d80 con 0x7fce58003f40 2024-04-17T10:52:56.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.778+0000 7fce70ebf640 1 -- 172.21.15.83:0/1843471318 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5c005b00 con 0x7fce58003f40 2024-04-17T10:52:56.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.778+0000 7fce70ebf640 1 -- 172.21.15.83:0/1843471318 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c007070 con 0x7fce58003f40 2024-04-17T10:52:56.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.778+0000 7fce74125640 1 -- 172.21.15.83:0/1843471318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce58003f40 msgr2=0x7fce58004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:56.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.778+0000 7fce74125640 1 --2- 172.21.15.83:0/1843471318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce58003f40 0x7fce58004810 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7fce6c06bf60 tx=0x7fce5c01f370 comp rx=0 tx=0).stop 2024-04-17T10:52:56.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.778+0000 7fce726c2640 1 -- 172.21.15.83:0/1843471318 reap_dead start 2024-04-17T10:52:56.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.778+0000 7fce74125640 1 -- 172.21.15.83:0/1843471318 shutdown_connections 2024-04-17T10:52:56.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.778+0000 7fce74125640 1 -- 172.21.15.83:0/1843471318 >> 172.21.15.83:0/1843471318 conn(0x7fce6c0d17b0 msgr2=0x7fce6c0d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:56.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.778+0000 7fce74125640 1 -- 172.21.15.83:0/1843471318 shutdown_connections 2024-04-17T10:52:56.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.778+0000 7fce74125640 1 -- 172.21.15.83:0/1843471318 wait complete. 2024-04-17T10:52:56.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.778+0000 7fce74125640 1 Processor -- start 2024-04-17T10:52:56.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.778+0000 7fce74125640 1 -- start start 2024-04-17T10:52:56.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.782+0000 7fce74125640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce58003f40 0x7fce6c14ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:56.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.782+0000 7fce74125640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce6c0b43d0 0x7fce6c0b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:56.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.782+0000 7fce74125640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce6c0b4850 0x7fce6c0b6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:56.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.782+0000 7fce74125640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c06cdf0 con 0x7fce6c0b43d0 2024-04-17T10:52:56.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.782+0000 7fce74125640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c15e270 con 0x7fce6c0b4850 2024-04-17T10:52:56.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.782+0000 7fce74125640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c0b69b0 con 0x7fce58003f40 2024-04-17T10:52:56.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.782+0000 7fce716c0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce6c0b43d0 0x7fce6c0b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:56.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.782+0000 7fce716c0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce6c0b43d0 0x7fce6c0b5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47650/0 (socket says 172.21.15.83:47650) 2024-04-17T10:52:56.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.782+0000 7fce716c0640 1 -- 172.21.15.83:0/1848151764 learned_addr learned my addr 172.21.15.83:0/1848151764 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:56.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.782+0000 7fce71ec1640 1 --2- 172.21.15.83:0/1848151764 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce58003f40 0x7fce6c14ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:56.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.782+0000 7fce726c2640 1 --2- 172.21.15.83:0/1848151764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce6c0b4850 0x7fce6c0b6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:56.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.782+0000 7fce716c0640 1 -- 172.21.15.83:0/1848151764 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce58003f40 msgr2=0x7fce6c14ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:56.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.782+0000 7fce716c0640 1 --2- 172.21.15.83:0/1848151764 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce58003f40 0x7fce6c14ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:56.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.782+0000 7fce716c0640 1 -- 172.21.15.83:0/1848151764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce6c0b4850 msgr2=0x7fce6c0b6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:56.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.782+0000 7fce716c0640 1 --2- 172.21.15.83:0/1848151764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce6c0b4850 0x7fce6c0b6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:56.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.782+0000 7fce716c0640 1 -- 172.21.15.83:0/1848151764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6c0ce430 con 0x7fce6c0b43d0 2024-04-17T10:52:56.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.782+0000 7fce71ec1640 1 --2- 172.21.15.83:0/1848151764 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce58003f40 0x7fce6c14ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:52:56.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.782+0000 7fce716c0640 1 --2- 172.21.15.83:0/1848151764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce6c0b43d0 0x7fce6c0b5f30 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7fce5c01fc10 tx=0x7fce58005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:56.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.782+0000 7fce627fc640 1 -- 172.21.15.83:0/1848151764 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c001cc0 con 0x7fce6c0b43d0 2024-04-17T10:52:56.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.782+0000 7fce627fc640 1 -- 172.21.15.83:0/1848151764 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5c005cb0 con 0x7fce6c0b43d0 2024-04-17T10:52:56.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.782+0000 7fce74125640 1 -- 172.21.15.83:0/1848151764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce6c14f1c0 con 0x7fce6c0b43d0 2024-04-17T10:52:56.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.782+0000 7fce627fc640 1 -- 172.21.15.83:0/1848151764 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c005e60 con 0x7fce6c0b43d0 2024-04-17T10:52:56.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.782+0000 7fce74125640 1 -- 172.21.15.83:0/1848151764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce6c0c6ed0 con 0x7fce6c0b43d0 2024-04-17T10:52:56.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.782+0000 7fce74125640 1 -- 172.21.15.83:0/1848151764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce6c074f90 con 0x7fce6c0b43d0 2024-04-17T10:52:56.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.782+0000 7fce627fc640 1 -- 172.21.15.83:0/1848151764 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fce5c0049f0 con 0x7fce6c0b43d0 2024-04-17T10:52:56.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.782+0000 7fce627fc640 1 --2- 172.21.15.83:0/1848151764 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fce50066f90 0x7fce50069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:56.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.782+0000 7fce71ec1640 1 --2- 172.21.15.83:0/1848151764 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fce50066f90 0x7fce50069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:56.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.782+0000 7fce627fc640 1 -- 172.21.15.83:0/1848151764 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fce5c007e00 con 0x7fce6c0b43d0 2024-04-17T10:52:56.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.782+0000 7fce71ec1640 1 --2- 172.21.15.83:0/1848151764 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fce50066f90 0x7fce50069450 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fce68009b40 tx=0x7fce68008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:56.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.786+0000 7fce627fc640 1 -- 172.21.15.83:0/1848151764 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fce5c0190e0 con 0x7fce6c0b43d0 2024-04-17T10:52:56.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.970+0000 7fce74125640 1 -- 172.21.15.83:0/1848151764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce6c072690 con 0x7fce6c0b43d0 2024-04-17T10:52:56.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.974+0000 7fce627fc640 1 -- 172.21.15.83:0/1848151764 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fce5c0f34c0 con 0x7fce6c0b43d0 2024-04-17T10:52:56.968 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:56.968 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:56.968 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:56.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.974+0000 7fce74125640 1 -- 172.21.15.83:0/1848151764 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fce50066f90 msgr2=0x7fce50069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:56.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.974+0000 7fce74125640 1 --2- 172.21.15.83:0/1848151764 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fce50066f90 0x7fce50069450 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fce68009b40 tx=0x7fce68008040 comp rx=0 tx=0).stop 2024-04-17T10:52:56.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.974+0000 7fce74125640 1 -- 172.21.15.83:0/1848151764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce6c0b43d0 msgr2=0x7fce6c0b5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:56.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.974+0000 7fce74125640 1 --2- 172.21.15.83:0/1848151764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce6c0b43d0 0x7fce6c0b5f30 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7fce5c01fc10 tx=0x7fce58005840 comp rx=0 tx=0).stop 2024-04-17T10:52:56.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.974+0000 7fce74125640 1 -- 172.21.15.83:0/1848151764 shutdown_connections 2024-04-17T10:52:56.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.974+0000 7fce74125640 1 --2- 172.21.15.83:0/1848151764 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fce50066f90 0x7fce50069450 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:56.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.974+0000 7fce74125640 1 --2- 172.21.15.83:0/1848151764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce6c0b4850 0x7fce6c0b6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:56.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.974+0000 7fce74125640 1 --2- 172.21.15.83:0/1848151764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce6c0b43d0 0x7fce6c0b5f30 unknown :-1 s=CLOSED pgs=2264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:56.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.974+0000 7fce74125640 1 --2- 172.21.15.83:0/1848151764 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce58003f40 0x7fce6c14ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:56.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.974+0000 7fce74125640 1 -- 172.21.15.83:0/1848151764 >> 172.21.15.83:0/1848151764 conn(0x7fce6c0d17b0 msgr2=0x7fce6c0d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:56.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.974+0000 7fce74125640 1 -- 172.21.15.83:0/1848151764 shutdown_connections 2024-04-17T10:52:56.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:56.974+0000 7fce74125640 1 -- 172.21.15.83:0/1848151764 wait complete. 2024-04-17T10:52:57.388 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:57.388 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:52:58.389 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:52:58.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.522+0000 7fdc99f40640 1 Processor -- start 2024-04-17T10:52:58.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.522+0000 7fdc99f40640 1 -- start start 2024-04-17T10:52:58.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.526+0000 7fdc99f40640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc94072330 0x7fdc940cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:58.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.526+0000 7fdc99f40640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdc940cfa10 0x7fdc9406c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:58.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.526+0000 7fdc99f40640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc940cfea0 con 0x7fdc9415ba30 2024-04-17T10:52:58.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.526+0000 7fdc99f40640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc94071ea0 con 0x7fdc940cfa10 2024-04-17T10:52:58.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.526+0000 7fdc99f40640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9406e320 con 0x7fdc94072330 2024-04-17T10:52:58.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.526+0000 7fdc937fe640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fdc9415ba30 0x7fdc9415c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:50194/0 (socket says 172.21.15.83:50194) 2024-04-17T10:52:58.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.526+0000 7fdc937fe640 1 -- 172.21.15.83:0/2406975821 learned_addr learned my addr 172.21.15.83:0/2406975821 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:52:58.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.526+0000 7fdc93fff640 1 --2- 172.21.15.83:0/2406975821 >> v2:172.21.15.167:3300/0 conn(0x7fdc940cfa10 0x7fdc9406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:58.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.526+0000 7fdc8affd640 1 --2- 172.21.15.83:0/2406975821 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc94072330 0x7fdc940cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:58.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.526+0000 7fdc8affd640 1 -- 172.21.15.83:0/2406975821 >> v2:172.21.15.167:3300/0 conn(0x7fdc940cfa10 msgr2=0x7fdc9406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:52:58.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.526+0000 7fdc8affd640 1 --2- 172.21.15.83:0/2406975821 >> v2:172.21.15.167:3300/0 conn(0x7fdc940cfa10 0x7fdc9406c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:58.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.526+0000 7fdc8affd640 1 -- 172.21.15.83:0/2406975821 >> v1:172.21.15.167:6789/0 conn(0x7fdc9415ba30 legacy=0x7fdc9415c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:58.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.526+0000 7fdc8affd640 1 -- 172.21.15.83:0/2406975821 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc9406e040 con 0x7fdc94072330 2024-04-17T10:52:58.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.526+0000 7fdc93fff640 1 --2- 172.21.15.83:0/2406975821 >> v2:172.21.15.167:3300/0 conn(0x7fdc940cfa10 0x7fdc9406c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:52:58.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.530+0000 7fdc8affd640 1 --2- 172.21.15.83:0/2406975821 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc94072330 0x7fdc940cf4d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c0097c0 tx=0x7fdc7c01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8668a0ec64d6645 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:58.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.530+0000 7fdc92ffd640 1 -- 172.21.15.83:0/2406975821 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c0b0070 con 0x7fdc94072330 2024-04-17T10:52:58.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.530+0000 7fdc92ffd640 1 -- 172.21.15.83:0/2406975821 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc7c0ab600 con 0x7fdc94072330 2024-04-17T10:52:58.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.530+0000 7fdc92ffd640 1 -- 172.21.15.83:0/2406975821 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c0ab9d0 con 0x7fdc94072330 2024-04-17T10:52:58.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.530+0000 7fdc99f40640 1 -- 172.21.15.83:0/2406975821 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc94072330 msgr2=0x7fdc940cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:58.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.530+0000 7fdc99f40640 1 --2- 172.21.15.83:0/2406975821 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc94072330 0x7fdc940cf4d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c0097c0 tx=0x7fdc7c01c410 comp rx=0 tx=0).stop 2024-04-17T10:52:58.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.530+0000 7fdc99f40640 1 -- 172.21.15.83:0/2406975821 shutdown_connections 2024-04-17T10:52:58.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.530+0000 7fdc99f40640 1 --2- 172.21.15.83:0/2406975821 >> v2:172.21.15.167:3300/0 conn(0x7fdc940cfa10 0x7fdc9406c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:58.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.530+0000 7fdc99f40640 1 --2- 172.21.15.83:0/2406975821 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc94072330 0x7fdc940cf4d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:58.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.530+0000 7fdc99f40640 1 -- 172.21.15.83:0/2406975821 >> 172.21.15.83:0/2406975821 conn(0x7fdc9406abe0 msgr2=0x7fdc9406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:58.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.530+0000 7fdc99f40640 1 -- 172.21.15.83:0/2406975821 shutdown_connections 2024-04-17T10:52:58.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.530+0000 7fdc99f40640 1 -- 172.21.15.83:0/2406975821 wait complete. 2024-04-17T10:52:58.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.530+0000 7fdc99f40640 1 Processor -- start 2024-04-17T10:52:58.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.530+0000 7fdc99f40640 1 -- start start 2024-04-17T10:52:58.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.530+0000 7fdc99f40640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc94072330 0x7fdc94147e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:58.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.530+0000 7fdc99f40640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc940cfa10 0x7fdc940b3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:58.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.530+0000 7fdc99f40640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc9415ba30 0x7fdc941473b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:58.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.530+0000 7fdc99f40640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9406de20 con 0x7fdc9415ba30 2024-04-17T10:52:58.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.530+0000 7fdc99f40640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9406dca0 con 0x7fdc940cfa10 2024-04-17T10:52:58.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.530+0000 7fdc99f40640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc94168520 con 0x7fdc94072330 2024-04-17T10:52:58.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.530+0000 7fdc93fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc9415ba30 0x7fdc941473b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:58.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.530+0000 7fdc93fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc9415ba30 0x7fdc941473b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47662/0 (socket says 172.21.15.83:47662) 2024-04-17T10:52:58.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.530+0000 7fdc93fff640 1 -- 172.21.15.83:0/3120556538 learned_addr learned my addr 172.21.15.83:0/3120556538 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:52:58.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.530+0000 7fdc937fe640 1 --2- 172.21.15.83:0/3120556538 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc94072330 0x7fdc94147e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:58.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.530+0000 7fdc8affd640 1 --2- 172.21.15.83:0/3120556538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc940cfa10 0x7fdc940b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:58.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.530+0000 7fdc93fff640 1 -- 172.21.15.83:0/3120556538 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc94072330 msgr2=0x7fdc94147e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:58.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.530+0000 7fdc93fff640 1 --2- 172.21.15.83:0/3120556538 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc94072330 0x7fdc94147e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:58.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.530+0000 7fdc93fff640 1 -- 172.21.15.83:0/3120556538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc940cfa10 msgr2=0x7fdc940b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:58.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.530+0000 7fdc93fff640 1 --2- 172.21.15.83:0/3120556538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc940cfa10 0x7fdc940b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:58.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.530+0000 7fdc93fff640 1 -- 172.21.15.83:0/3120556538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc94150010 con 0x7fdc9415ba30 2024-04-17T10:52:58.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.530+0000 7fdc937fe640 1 --2- 172.21.15.83:0/3120556538 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc94072330 0x7fdc94147e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:52:58.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.534+0000 7fdc93fff640 1 --2- 172.21.15.83:0/3120556538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc9415ba30 0x7fdc941473b0 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009e60 tx=0x7fdc8c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:58.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.534+0000 7fdc90ff9640 1 -- 172.21.15.83:0/3120556538 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c018030 con 0x7fdc9415ba30 2024-04-17T10:52:58.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.534+0000 7fdc90ff9640 1 -- 172.21.15.83:0/3120556538 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8c00cdf0 con 0x7fdc9415ba30 2024-04-17T10:52:58.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.534+0000 7fdc99f40640 1 -- 172.21.15.83:0/3120556538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc940b45e0 con 0x7fdc9415ba30 2024-04-17T10:52:58.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.534+0000 7fdc90ff9640 1 -- 172.21.15.83:0/3120556538 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c003bc0 con 0x7fdc9415ba30 2024-04-17T10:52:58.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.534+0000 7fdc99f40640 1 -- 172.21.15.83:0/3120556538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc94151ce0 con 0x7fdc9415ba30 2024-04-17T10:52:58.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.534+0000 7fdc99f40640 1 -- 172.21.15.83:0/3120556538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc9414c6f0 con 0x7fdc9415ba30 2024-04-17T10:52:58.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.534+0000 7fdc90ff9640 1 -- 172.21.15.83:0/3120556538 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdc8c01b070 con 0x7fdc9415ba30 2024-04-17T10:52:58.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.534+0000 7fdc90ff9640 1 --2- 172.21.15.83:0/3120556538 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdc74066f60 0x7fdc74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:52:58.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.534+0000 7fdc90ff9640 1 -- 172.21.15.83:0/3120556538 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fdc8c0859d0 con 0x7fdc9415ba30 2024-04-17T10:52:58.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.534+0000 7fdc937fe640 1 --2- 172.21.15.83:0/3120556538 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdc74066f60 0x7fdc74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:52:58.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.534+0000 7fdc937fe640 1 --2- 172.21.15.83:0/3120556538 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdc74066f60 0x7fdc74069420 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fdc84005890 tx=0x7fdc84009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:52:58.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.538+0000 7fdc90ff9640 1 -- 172.21.15.83:0/3120556538 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdc9414c6f0 con 0x7fdc9415ba30 2024-04-17T10:52:58.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.734+0000 7fdc99f40640 1 -- 172.21.15.83:0/3120556538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc94074f90 con 0x7fdc9415ba30 2024-04-17T10:52:58.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.738+0000 7fdc90ff9640 1 -- 172.21.15.83:0/3120556538 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fdc8c010040 con 0x7fdc9415ba30 2024-04-17T10:52:58.732 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:52:58.732 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:52:58.732 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:52:58.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.738+0000 7fdc99f40640 1 -- 172.21.15.83:0/3120556538 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdc74066f60 msgr2=0x7fdc74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:58.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.738+0000 7fdc99f40640 1 --2- 172.21.15.83:0/3120556538 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdc74066f60 0x7fdc74069420 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fdc84005890 tx=0x7fdc84009040 comp rx=0 tx=0).stop 2024-04-17T10:52:58.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.738+0000 7fdc99f40640 1 -- 172.21.15.83:0/3120556538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc9415ba30 msgr2=0x7fdc941473b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:52:58.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.738+0000 7fdc99f40640 1 --2- 172.21.15.83:0/3120556538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc9415ba30 0x7fdc941473b0 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009e60 tx=0x7fdc8c00c490 comp rx=0 tx=0).stop 2024-04-17T10:52:58.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.738+0000 7fdc99f40640 1 -- 172.21.15.83:0/3120556538 shutdown_connections 2024-04-17T10:52:58.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.738+0000 7fdc99f40640 1 --2- 172.21.15.83:0/3120556538 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdc74066f60 0x7fdc74069420 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:58.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.738+0000 7fdc99f40640 1 --2- 172.21.15.83:0/3120556538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc9415ba30 0x7fdc941473b0 unknown :-1 s=CLOSED pgs=2265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:58.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.738+0000 7fdc99f40640 1 --2- 172.21.15.83:0/3120556538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc940cfa10 0x7fdc940b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:58.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.738+0000 7fdc99f40640 1 --2- 172.21.15.83:0/3120556538 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc94072330 0x7fdc94147e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:52:58.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.738+0000 7fdc99f40640 1 -- 172.21.15.83:0/3120556538 >> 172.21.15.83:0/3120556538 conn(0x7fdc9406abe0 msgr2=0x7fdc9415a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:52:58.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.738+0000 7fdc99f40640 1 -- 172.21.15.83:0/3120556538 shutdown_connections 2024-04-17T10:52:58.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:52:58.738+0000 7fdc99f40640 1 -- 172.21.15.83:0/3120556538 wait complete. 2024-04-17T10:52:59.151 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:52:59.151 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:00.152 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:00.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f4631430640 1 Processor -- start 2024-04-17T10:53:00.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f4631430640 1 -- start start 2024-04-17T10:53:00.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f4631430640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f462c15ba30 0x7f462c072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:00.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f4631430640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f462c0cfbe0 0x7f462c06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:00.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f4631430640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f462c06e320 con 0x7f462c15c960 2024-04-17T10:53:00.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f4631430640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f462c071ea0 con 0x7f462c0cfbe0 2024-04-17T10:53:00.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f4631430640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462c06e040 con 0x7f462c15ba30 2024-04-17T10:53:00.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f462a7fc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f462c15c960 0x7f462c0cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53996/0 (socket says 172.21.15.83:53996) 2024-04-17T10:53:00.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f462a7fc640 1 -- 172.21.15.83:0/1877687895 learned_addr learned my addr 172.21.15.83:0/1877687895 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:53:00.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f462affd640 1 --2- 172.21.15.83:0/1877687895 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f462c15ba30 0x7f462c072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:00.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f462b7fe640 1 --2- 172.21.15.83:0/1877687895 >> v2:172.21.15.167:3300/0 conn(0x7f462c0cfbe0 0x7f462c06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:00.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f4629ffb640 1 -- 172.21.15.83:0/1877687895 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1196784435 0 0) 0x7f462c06e320 con 0x7f462c15c960 2024-04-17T10:53:00.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f4629ffb640 1 -- 172.21.15.83:0/1877687895 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4614003530 con 0x7f462c15c960 2024-04-17T10:53:00.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f4629ffb640 1 -- 172.21.15.83:0/1877687895 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1652330841 0 0) 0x7f4614003530 con 0x7f462c15c960 2024-04-17T10:53:00.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f4629ffb640 1 -- 172.21.15.83:0/1877687895 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f462c15ba30 msgr2=0x7f462c072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:00.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f4629ffb640 1 --2- 172.21.15.83:0/1877687895 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f462c15ba30 0x7f462c072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:00.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f4629ffb640 1 -- 172.21.15.83:0/1877687895 >> v2:172.21.15.167:3300/0 conn(0x7f462c0cfbe0 msgr2=0x7f462c06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:00.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f4629ffb640 1 --2- 172.21.15.83:0/1877687895 >> v2:172.21.15.167:3300/0 conn(0x7f462c0cfbe0 0x7f462c06c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:00.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f4629ffb640 1 -- 172.21.15.83:0/1877687895 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f462c071850 con 0x7f462c15c960 2024-04-17T10:53:00.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f4629ffb640 1 -- 172.21.15.83:0/1877687895 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f46200016a0 con 0x7f462c15c960 2024-04-17T10:53:00.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f4629ffb640 1 -- 172.21.15.83:0/1877687895 >> v1:172.21.15.83:6789/0 conn(0x7f462c15c960 legacy=0x7f462c0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:00.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f4629ffb640 1 --2- 172.21.15.83:0/1877687895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46140039c0 0x7f4614003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:00.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f4629ffb640 1 --2- 172.21.15.83:0/1877687895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4614004810 0x7f4614004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:00.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f4629ffb640 1 --2- 172.21.15.83:0/1877687895 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f462c15ba30 0x7f46140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:00.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f4629ffb640 1 -- 172.21.15.83:0/1877687895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462c071ea0 con 0x7f46140039c0 2024-04-17T10:53:00.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f4629ffb640 1 -- 172.21.15.83:0/1877687895 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462c06e040 con 0x7f4614004810 2024-04-17T10:53:00.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f4629ffb640 1 -- 172.21.15.83:0/1877687895 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f461400b9d0 con 0x7f462c15ba30 2024-04-17T10:53:00.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f462affd640 1 --2- 172.21.15.83:0/1877687895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46140039c0 0x7f4614003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:00.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f462a7fc640 1 --2- 172.21.15.83:0/1877687895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4614004810 0x7f4614004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:00.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f462affd640 1 -- 172.21.15.83:0/1877687895 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f462c15ba30 msgr2=0x7f46140051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:00.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f462affd640 1 --2- 172.21.15.83:0/1877687895 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f462c15ba30 0x7f46140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:00.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f462affd640 1 -- 172.21.15.83:0/1877687895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4614004810 msgr2=0x7f4614004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:00.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f462affd640 1 --2- 172.21.15.83:0/1877687895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4614004810 0x7f4614004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:00.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.286+0000 7f462affd640 1 -- 172.21.15.83:0/1877687895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4614003530 con 0x7f46140039c0 2024-04-17T10:53:00.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.290+0000 7f462affd640 1 --2- 172.21.15.83:0/1877687895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46140039c0 0x7f4614003de0 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f461c009b10 tx=0x7f461c01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d70897518f6d40 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:00.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.290+0000 7f4629ffb640 1 -- 172.21.15.83:0/1877687895 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c0096b0 con 0x7f46140039c0 2024-04-17T10:53:00.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.290+0000 7f4629ffb640 1 -- 172.21.15.83:0/1877687895 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f461c0a8410 con 0x7f46140039c0 2024-04-17T10:53:00.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.290+0000 7f4629ffb640 1 -- 172.21.15.83:0/1877687895 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c0a8800 con 0x7f46140039c0 2024-04-17T10:53:00.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.290+0000 7f4631430640 1 -- 172.21.15.83:0/1877687895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46140039c0 msgr2=0x7f4614003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:00.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.290+0000 7f4631430640 1 --2- 172.21.15.83:0/1877687895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46140039c0 0x7f4614003de0 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f461c009b10 tx=0x7f461c01cab0 comp rx=0 tx=0).stop 2024-04-17T10:53:00.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.290+0000 7f462b7fe640 1 -- 172.21.15.83:0/1877687895 reap_dead start 2024-04-17T10:53:00.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.290+0000 7f4631430640 1 -- 172.21.15.83:0/1877687895 shutdown_connections 2024-04-17T10:53:00.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.290+0000 7f4631430640 1 -- 172.21.15.83:0/1877687895 >> 172.21.15.83:0/1877687895 conn(0x7f462c06abe0 msgr2=0x7f462c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:00.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.290+0000 7f4631430640 1 -- 172.21.15.83:0/1877687895 shutdown_connections 2024-04-17T10:53:00.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.290+0000 7f4631430640 1 -- 172.21.15.83:0/1877687895 wait complete. 2024-04-17T10:53:00.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.290+0000 7f4631430640 1 Processor -- start 2024-04-17T10:53:00.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.290+0000 7f4631430640 1 -- start start 2024-04-17T10:53:00.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.294+0000 7f4631430640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46140039c0 0x7f462c0ac6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:00.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.294+0000 7f4631430640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f462c165050 0x7f462c14b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:00.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.294+0000 7f4631430640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f462c0add70 0x7f462c0aa670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:00.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.294+0000 7f4631430640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462c15cf00 con 0x7f462c165050 2024-04-17T10:53:00.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.294+0000 7f4631430640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462c15cd80 con 0x7f46140039c0 2024-04-17T10:53:00.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.294+0000 7f4631430640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462c166510 con 0x7f462c0add70 2024-04-17T10:53:00.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.294+0000 7f462a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f462c165050 0x7f462c14b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:00.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.294+0000 7f462a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f462c165050 0x7f462c14b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47680/0 (socket says 172.21.15.83:47680) 2024-04-17T10:53:00.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.294+0000 7f462a7fc640 1 -- 172.21.15.83:0/3328412211 learned_addr learned my addr 172.21.15.83:0/3328412211 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:00.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.294+0000 7f462affd640 1 --2- 172.21.15.83:0/3328412211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46140039c0 0x7f462c0ac6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:00.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.294+0000 7f462b7fe640 1 --2- 172.21.15.83:0/3328412211 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f462c0add70 0x7f462c0aa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:00.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.294+0000 7f462a7fc640 1 -- 172.21.15.83:0/3328412211 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f462c0add70 msgr2=0x7f462c0aa670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:00.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.294+0000 7f462a7fc640 1 --2- 172.21.15.83:0/3328412211 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f462c0add70 0x7f462c0aa670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:00.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.294+0000 7f462a7fc640 1 -- 172.21.15.83:0/3328412211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46140039c0 msgr2=0x7f462c0ac6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:00.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.294+0000 7f462a7fc640 1 --2- 172.21.15.83:0/3328412211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46140039c0 0x7f462c0ac6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:00.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.294+0000 7f462a7fc640 1 -- 172.21.15.83:0/3328412211 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f462c0accd0 con 0x7f462c165050 2024-04-17T10:53:00.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.294+0000 7f462affd640 1 --2- 172.21.15.83:0/3328412211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46140039c0 0x7f462c0ac6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:53:00.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.294+0000 7f462a7fc640 1 --2- 172.21.15.83:0/3328412211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f462c165050 0x7f462c14b330 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f462000d840 tx=0x7f462000dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:00.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.294+0000 7f46137fe640 1 -- 172.21.15.83:0/3328412211 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4620019070 con 0x7f462c165050 2024-04-17T10:53:00.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.294+0000 7f46137fe640 1 -- 172.21.15.83:0/3328412211 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46200074e0 con 0x7f462c165050 2024-04-17T10:53:00.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.294+0000 7f4631430640 1 -- 172.21.15.83:0/3328412211 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f462c0acfb0 con 0x7f462c165050 2024-04-17T10:53:00.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.294+0000 7f46137fe640 1 -- 172.21.15.83:0/3328412211 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4620004a30 con 0x7f462c165050 2024-04-17T10:53:00.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.294+0000 7f4631430640 1 -- 172.21.15.83:0/3328412211 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f462c0ad570 con 0x7f462c165050 2024-04-17T10:53:00.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.294+0000 7f4631430640 1 -- 172.21.15.83:0/3328412211 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f462c074f90 con 0x7f462c165050 2024-04-17T10:53:00.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.294+0000 7f46137fe640 1 -- 172.21.15.83:0/3328412211 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4620004bd0 con 0x7f462c165050 2024-04-17T10:53:00.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.294+0000 7f46137fe640 1 --2- 172.21.15.83:0/3328412211 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4618066f90 0x7f4618069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:00.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.294+0000 7f46137fe640 1 -- 172.21.15.83:0/3328412211 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f4620084ad0 con 0x7f462c165050 2024-04-17T10:53:00.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.294+0000 7f462affd640 1 --2- 172.21.15.83:0/3328412211 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4618066f90 0x7f4618069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:00.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.294+0000 7f462affd640 1 --2- 172.21.15.83:0/3328412211 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4618066f90 0x7f4618069450 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f461c0af000 tx=0x7f461c0b2040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:00.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.298+0000 7f46137fe640 1 -- 172.21.15.83:0/3328412211 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4620055a70 con 0x7f462c165050 2024-04-17T10:53:00.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.490+0000 7f4631430640 1 -- 172.21.15.83:0/3328412211 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f462c15efa0 con 0x7f462c165050 2024-04-17T10:53:00.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.490+0000 7f46137fe640 1 -- 172.21.15.83:0/3328412211 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f4620055370 con 0x7f462c165050 2024-04-17T10:53:00.484 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:00.484 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:00.484 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:00.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.494+0000 7f4631430640 1 -- 172.21.15.83:0/3328412211 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4618066f90 msgr2=0x7f4618069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:00.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.494+0000 7f4631430640 1 --2- 172.21.15.83:0/3328412211 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4618066f90 0x7f4618069450 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f461c0af000 tx=0x7f461c0b2040 comp rx=0 tx=0).stop 2024-04-17T10:53:00.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.494+0000 7f4631430640 1 -- 172.21.15.83:0/3328412211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f462c165050 msgr2=0x7f462c14b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:00.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.494+0000 7f4631430640 1 --2- 172.21.15.83:0/3328412211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f462c165050 0x7f462c14b330 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f462000d840 tx=0x7f462000dd00 comp rx=0 tx=0).stop 2024-04-17T10:53:00.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.494+0000 7f4631430640 1 -- 172.21.15.83:0/3328412211 shutdown_connections 2024-04-17T10:53:00.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.494+0000 7f4631430640 1 --2- 172.21.15.83:0/3328412211 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4618066f90 0x7f4618069450 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:00.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.494+0000 7f4631430640 1 --2- 172.21.15.83:0/3328412211 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f462c0add70 0x7f462c0aa670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:00.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.494+0000 7f4631430640 1 --2- 172.21.15.83:0/3328412211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f462c165050 0x7f462c14b330 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:00.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.494+0000 7f4631430640 1 --2- 172.21.15.83:0/3328412211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46140039c0 0x7f462c0ac6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:00.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.494+0000 7f4631430640 1 -- 172.21.15.83:0/3328412211 >> 172.21.15.83:0/3328412211 conn(0x7f462c06abe0 msgr2=0x7f462c0b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:00.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.494+0000 7f4631430640 1 -- 172.21.15.83:0/3328412211 shutdown_connections 2024-04-17T10:53:00.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:00.494+0000 7f4631430640 1 -- 172.21.15.83:0/3328412211 wait complete. 2024-04-17T10:53:00.906 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:00.906 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:01.907 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:02.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.042+0000 7fe8298d0640 1 Processor -- start 2024-04-17T10:53:02.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.042+0000 7fe8298d0640 1 -- start start 2024-04-17T10:53:02.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.046+0000 7fe8298d0640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe82415db50 0x7fe824070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:02.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.046+0000 7fe8298d0640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8240709b0 con 0x7fe82406dcb0 2024-04-17T10:53:02.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.046+0000 7fe8298d0640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe82415e440 con 0x7fe8240aedd0 2024-04-17T10:53:02.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.046+0000 7fe8298d0640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe82415e720 con 0x7fe82415db50 2024-04-17T10:53:02.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.046+0000 7fe8237fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe82406dcb0 0x7fe8240b2e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54012/0 (socket says 172.21.15.83:54012) 2024-04-17T10:53:02.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.046+0000 7fe8237fe640 1 -- 172.21.15.83:0/4081838265 learned_addr learned my addr 172.21.15.83:0/4081838265 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:53:02.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.046+0000 7fe822ffd640 1 --2- 172.21.15.83:0/4081838265 >> v2:172.21.15.167:3300/0 conn(0x7fe82415db50 0x7fe824070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:02.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.046+0000 7fe821ffb640 1 -- 172.21.15.83:0/4081838265 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1179129580 0 0) 0x7fe81c001120 con 0x7fe82406dcb0 2024-04-17T10:53:02.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.046+0000 7fe821ffb640 1 -- 172.21.15.83:0/4081838265 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe810003530 con 0x7fe82406dcb0 2024-04-17T10:53:02.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.046+0000 7fe821ffb640 1 -- 172.21.15.83:0/4081838265 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1861044182 0 0) 0x7fe810003530 con 0x7fe82406dcb0 2024-04-17T10:53:02.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.046+0000 7fe821ffb640 1 -- 172.21.15.83:0/4081838265 >> v2:172.21.15.167:3300/0 conn(0x7fe82415db50 msgr2=0x7fe824070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:02.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.046+0000 7fe821ffb640 1 --2- 172.21.15.83:0/4081838265 >> v2:172.21.15.167:3300/0 conn(0x7fe82415db50 0x7fe824070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:02.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.046+0000 7fe821ffb640 1 -- 172.21.15.83:0/4081838265 >> v1:172.21.15.167:6789/0 conn(0x7fe8240aedd0 legacy=0x7fe82406d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:02.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.046+0000 7fe821ffb640 1 -- 172.21.15.83:0/4081838265 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe82415ea00 con 0x7fe82406dcb0 2024-04-17T10:53:02.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.046+0000 7fe821ffb640 1 -- 172.21.15.83:0/4081838265 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fe81c001850 con 0x7fe82406dcb0 2024-04-17T10:53:02.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.046+0000 7fe821ffb640 1 -- 172.21.15.83:0/4081838265 >> v1:172.21.15.83:6789/0 conn(0x7fe82406dcb0 legacy=0x7fe8240b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:02.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.046+0000 7fe821ffb640 1 --2- 172.21.15.83:0/4081838265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8100039c0 0x7fe810003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:02.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.046+0000 7fe821ffb640 1 --2- 172.21.15.83:0/4081838265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe810004810 0x7fe810004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:02.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.046+0000 7fe821ffb640 1 --2- 172.21.15.83:0/4081838265 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8100051d0 0x7fe81000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:02.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.046+0000 7fe821ffb640 1 -- 172.21.15.83:0/4081838265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82415e720 con 0x7fe810004810 2024-04-17T10:53:02.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.046+0000 7fe821ffb640 1 -- 172.21.15.83:0/4081838265 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81000e280 con 0x7fe8100039c0 2024-04-17T10:53:02.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.046+0000 7fe821ffb640 1 -- 172.21.15.83:0/4081838265 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe81000e480 con 0x7fe8100051d0 2024-04-17T10:53:02.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.046+0000 7fe8227fc640 1 --2- 172.21.15.83:0/4081838265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe810004810 0x7fe810004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:02.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.046+0000 7fe822ffd640 1 --2- 172.21.15.83:0/4081838265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8100039c0 0x7fe810003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:02.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.046+0000 7fe8237fe640 1 --2- 172.21.15.83:0/4081838265 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8100051d0 0x7fe81000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:02.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.046+0000 7fe8227fc640 1 -- 172.21.15.83:0/4081838265 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8100051d0 msgr2=0x7fe81000dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:02.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.046+0000 7fe8227fc640 1 --2- 172.21.15.83:0/4081838265 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8100051d0 0x7fe81000dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:02.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.046+0000 7fe8227fc640 1 -- 172.21.15.83:0/4081838265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8100039c0 msgr2=0x7fe810003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:02.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.046+0000 7fe8227fc640 1 --2- 172.21.15.83:0/4081838265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8100039c0 0x7fe810003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:02.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.046+0000 7fe8227fc640 1 -- 172.21.15.83:0/4081838265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe810003530 con 0x7fe810004810 2024-04-17T10:53:02.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.046+0000 7fe8237fe640 1 --2- 172.21.15.83:0/4081838265 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8100051d0 0x7fe81000dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:53:02.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.046+0000 7fe8237fe640 1 -- 172.21.15.83:0/4081838265 reap_dead start 2024-04-17T10:53:02.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe8227fc640 1 --2- 172.21.15.83:0/4081838265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe810004810 0x7fe810004c90 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7fe80c001780 tx=0x7fe80c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5afcea4ab9017e9c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:02.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe821ffb640 1 -- 172.21.15.83:0/4081838265 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80c0013a0 con 0x7fe810004810 2024-04-17T10:53:02.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe821ffb640 1 -- 172.21.15.83:0/4081838265 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe80c0052f0 con 0x7fe810004810 2024-04-17T10:53:02.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe821ffb640 1 -- 172.21.15.83:0/4081838265 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80c003840 con 0x7fe810004810 2024-04-17T10:53:02.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe8298d0640 1 -- 172.21.15.83:0/4081838265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe810004810 msgr2=0x7fe810004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:02.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe8298d0640 1 --2- 172.21.15.83:0/4081838265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe810004810 0x7fe810004c90 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7fe80c001780 tx=0x7fe80c01c400 comp rx=0 tx=0).stop 2024-04-17T10:53:02.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe8298d0640 1 -- 172.21.15.83:0/4081838265 shutdown_connections 2024-04-17T10:53:02.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe8298d0640 1 --2- 172.21.15.83:0/4081838265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe810004810 0x7fe810004c90 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:02.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe8298d0640 1 -- 172.21.15.83:0/4081838265 >> 172.21.15.83:0/4081838265 conn(0x7fe82406abe0 msgr2=0x7fe82406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:02.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe8298d0640 1 -- 172.21.15.83:0/4081838265 shutdown_connections 2024-04-17T10:53:02.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe8298d0640 1 -- 172.21.15.83:0/4081838265 wait complete. 2024-04-17T10:53:02.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe8298d0640 1 Processor -- start 2024-04-17T10:53:02.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe8298d0640 1 -- start start 2024-04-17T10:53:02.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe8298d0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe810004810 0x7fe824155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:02.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe8298d0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8241573a0 0x7fe824147f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:02.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe8298d0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8240ad870 0x7fe8240adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:02.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe8298d0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82415df30 con 0x7fe810004810 2024-04-17T10:53:02.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe8298d0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82415efc0 con 0x7fe8241573a0 2024-04-17T10:53:02.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe8298d0640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe82415e0b0 con 0x7fe8240ad870 2024-04-17T10:53:02.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe822ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe810004810 0x7fe824155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:02.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe822ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe810004810 0x7fe824155b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47690/0 (socket says 172.21.15.83:47690) 2024-04-17T10:53:02.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe822ffd640 1 -- 172.21.15.83:0/132460833 learned_addr learned my addr 172.21.15.83:0/132460833 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:02.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe8227fc640 1 --2- 172.21.15.83:0/132460833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8241573a0 0x7fe824147f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:02.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe8237fe640 1 --2- 172.21.15.83:0/132460833 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8240ad870 0x7fe8240adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:02.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe822ffd640 1 -- 172.21.15.83:0/132460833 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8240ad870 msgr2=0x7fe8240adcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:02.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe822ffd640 1 --2- 172.21.15.83:0/132460833 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8240ad870 0x7fe8240adcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:02.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe822ffd640 1 -- 172.21.15.83:0/132460833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8241573a0 msgr2=0x7fe824147f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:02.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe822ffd640 1 --2- 172.21.15.83:0/132460833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8241573a0 0x7fe824147f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:02.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe822ffd640 1 -- 172.21.15.83:0/132460833 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe824148970 con 0x7fe810004810 2024-04-17T10:53:02.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe8237fe640 1 --2- 172.21.15.83:0/132460833 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8240ad870 0x7fe8240adcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:53:02.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe822ffd640 1 --2- 172.21.15.83:0/132460833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe810004810 0x7fe824155b20 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7fe814007f60 tx=0x7fe814007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:02.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe81bfff640 1 -- 172.21.15.83:0/132460833 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81401a020 con 0x7fe810004810 2024-04-17T10:53:02.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe81bfff640 1 -- 172.21.15.83:0/132460833 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe814016470 con 0x7fe810004810 2024-04-17T10:53:02.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe8298d0640 1 -- 172.21.15.83:0/132460833 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe824157b10 con 0x7fe810004810 2024-04-17T10:53:02.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe81bfff640 1 -- 172.21.15.83:0/132460833 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe814016860 con 0x7fe810004810 2024-04-17T10:53:02.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.050+0000 7fe8298d0640 1 -- 172.21.15.83:0/132460833 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8241580d0 con 0x7fe810004810 2024-04-17T10:53:02.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.054+0000 7fe8298d0640 1 -- 172.21.15.83:0/132460833 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe824074f90 con 0x7fe810004810 2024-04-17T10:53:02.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.054+0000 7fe81bfff640 1 -- 172.21.15.83:0/132460833 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe81401d020 con 0x7fe810004810 2024-04-17T10:53:02.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.054+0000 7fe81bfff640 1 --2- 172.21.15.83:0/132460833 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe804066f60 0x7fe804069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:02.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.054+0000 7fe81bfff640 1 -- 172.21.15.83:0/132460833 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fe8140864f0 con 0x7fe810004810 2024-04-17T10:53:02.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.054+0000 7fe8227fc640 1 --2- 172.21.15.83:0/132460833 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe804066f60 0x7fe804069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:02.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.054+0000 7fe8227fc640 1 --2- 172.21.15.83:0/132460833 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe804066f60 0x7fe804069420 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fe80c001750 tx=0x7fe80c0011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:02.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.058+0000 7fe81bfff640 1 -- 172.21.15.83:0/132460833 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe814024030 con 0x7fe810004810 2024-04-17T10:53:02.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.242+0000 7fe8298d0640 1 -- 172.21.15.83:0/132460833 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8240b2af0 con 0x7fe810004810 2024-04-17T10:53:02.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.242+0000 7fe81bfff640 1 -- 172.21.15.83:0/132460833 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fe814057590 con 0x7fe810004810 2024-04-17T10:53:02.237 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:02.237 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:02.237 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:02.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.246+0000 7fe8298d0640 1 -- 172.21.15.83:0/132460833 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe804066f60 msgr2=0x7fe804069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:02.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.246+0000 7fe8298d0640 1 --2- 172.21.15.83:0/132460833 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe804066f60 0x7fe804069420 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fe80c001750 tx=0x7fe80c0011a0 comp rx=0 tx=0).stop 2024-04-17T10:53:02.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.246+0000 7fe8298d0640 1 -- 172.21.15.83:0/132460833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe810004810 msgr2=0x7fe824155b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:02.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.246+0000 7fe8298d0640 1 --2- 172.21.15.83:0/132460833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe810004810 0x7fe824155b20 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7fe814007f60 tx=0x7fe814007f90 comp rx=0 tx=0).stop 2024-04-17T10:53:02.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.246+0000 7fe8298d0640 1 -- 172.21.15.83:0/132460833 shutdown_connections 2024-04-17T10:53:02.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.246+0000 7fe8298d0640 1 --2- 172.21.15.83:0/132460833 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe804066f60 0x7fe804069420 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:02.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.246+0000 7fe8298d0640 1 --2- 172.21.15.83:0/132460833 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8240ad870 0x7fe8240adcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:02.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.246+0000 7fe8298d0640 1 --2- 172.21.15.83:0/132460833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8241573a0 0x7fe824147f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:02.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.246+0000 7fe8298d0640 1 --2- 172.21.15.83:0/132460833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe810004810 0x7fe824155b20 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:02.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.246+0000 7fe8298d0640 1 -- 172.21.15.83:0/132460833 >> 172.21.15.83:0/132460833 conn(0x7fe82406abe0 msgr2=0x7fe8240af550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:02.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.246+0000 7fe8298d0640 1 -- 172.21.15.83:0/132460833 shutdown_connections 2024-04-17T10:53:02.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:02.246+0000 7fe8298d0640 1 -- 172.21.15.83:0/132460833 wait complete. 2024-04-17T10:53:02.656 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:02.657 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:03.658 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:03.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34ab00f640 1 Processor -- start 2024-04-17T10:53:03.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34ab00f640 1 -- start start 2024-04-17T10:53:03.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34ab00f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34a415ba30 0x7f34a4072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:03.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34ab00f640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f34a40cfbe0 0x7f34a406c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:03.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34ab00f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34a406e320 con 0x7f34a415c960 2024-04-17T10:53:03.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34ab00f640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34a4071ea0 con 0x7f34a40cfbe0 2024-04-17T10:53:03.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34ab00f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34a406e040 con 0x7f34a415ba30 2024-04-17T10:53:03.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34a3fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f34a415c960 0x7f34a40cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54014/0 (socket says 172.21.15.83:54014) 2024-04-17T10:53:03.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34a3fff640 1 -- 172.21.15.83:0/3067208722 learned_addr learned my addr 172.21.15.83:0/3067208722 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:53:03.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34a95ac640 1 --2- 172.21.15.83:0/3067208722 >> v2:172.21.15.167:3300/0 conn(0x7f34a40cfbe0 0x7f34a406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:03.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34a8dab640 1 --2- 172.21.15.83:0/3067208722 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34a415ba30 0x7f34a4072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:03.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34a37fe640 1 -- 172.21.15.83:0/3067208722 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1728076947 0 0) 0x7f34a406e320 con 0x7f34a415c960 2024-04-17T10:53:03.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34a37fe640 1 -- 172.21.15.83:0/3067208722 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f348c003530 con 0x7f34a415c960 2024-04-17T10:53:03.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34a37fe640 1 -- 172.21.15.83:0/3067208722 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1274292028 0 0) 0x7f348c003530 con 0x7f34a415c960 2024-04-17T10:53:03.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34a37fe640 1 -- 172.21.15.83:0/3067208722 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34a415ba30 msgr2=0x7f34a4072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:03.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34a37fe640 1 --2- 172.21.15.83:0/3067208722 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34a415ba30 0x7f34a4072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:03.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34a37fe640 1 -- 172.21.15.83:0/3067208722 >> v2:172.21.15.167:3300/0 conn(0x7f34a40cfbe0 msgr2=0x7f34a406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:03.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34a37fe640 1 --2- 172.21.15.83:0/3067208722 >> v2:172.21.15.167:3300/0 conn(0x7f34a40cfbe0 0x7f34a406c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:03.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34a37fe640 1 -- 172.21.15.83:0/3067208722 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a4071850 con 0x7f34a415c960 2024-04-17T10:53:03.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34a37fe640 1 -- 172.21.15.83:0/3067208722 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f34940016a0 con 0x7f34a415c960 2024-04-17T10:53:03.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34a37fe640 1 -- 172.21.15.83:0/3067208722 >> v1:172.21.15.83:6789/0 conn(0x7f34a415c960 legacy=0x7f34a40cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:03.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34a37fe640 1 --2- 172.21.15.83:0/3067208722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f348c0039c0 0x7f348c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:03.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34a37fe640 1 --2- 172.21.15.83:0/3067208722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f348c004810 0x7f348c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:03.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34a37fe640 1 --2- 172.21.15.83:0/3067208722 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34a415ba30 0x7f348c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:03.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34a37fe640 1 -- 172.21.15.83:0/3067208722 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a4071ea0 con 0x7f348c0039c0 2024-04-17T10:53:03.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34a37fe640 1 -- 172.21.15.83:0/3067208722 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a406e040 con 0x7f348c004810 2024-04-17T10:53:03.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34a37fe640 1 -- 172.21.15.83:0/3067208722 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348c00ba20 con 0x7f34a415ba30 2024-04-17T10:53:03.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34a8dab640 1 --2- 172.21.15.83:0/3067208722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f348c0039c0 0x7f348c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:03.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34a3fff640 1 --2- 172.21.15.83:0/3067208722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f348c004810 0x7f348c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:03.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34a95ac640 1 --2- 172.21.15.83:0/3067208722 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34a415ba30 0x7f348c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:03.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34a8dab640 1 -- 172.21.15.83:0/3067208722 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34a415ba30 msgr2=0x7f348c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:03.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34a8dab640 1 --2- 172.21.15.83:0/3067208722 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34a415ba30 0x7f348c0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:03.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34a8dab640 1 -- 172.21.15.83:0/3067208722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f348c004810 msgr2=0x7f348c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:03.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34a8dab640 1 --2- 172.21.15.83:0/3067208722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f348c004810 0x7f348c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:03.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.794+0000 7f34a8dab640 1 -- 172.21.15.83:0/3067208722 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348c003530 con 0x7f348c0039c0 2024-04-17T10:53:03.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.798+0000 7f34a8dab640 1 --2- 172.21.15.83:0/3067208722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f348c0039c0 0x7f348c003de0 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f3498009b10 tx=0x7f349801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=220e444e25144ec7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:03.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.798+0000 7f34a37fe640 1 -- 172.21.15.83:0/3067208722 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34980096b0 con 0x7f348c0039c0 2024-04-17T10:53:03.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.798+0000 7f34a37fe640 1 -- 172.21.15.83:0/3067208722 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34980a9470 con 0x7f348c0039c0 2024-04-17T10:53:03.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.798+0000 7f34a37fe640 1 -- 172.21.15.83:0/3067208722 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34980a9860 con 0x7f348c0039c0 2024-04-17T10:53:03.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.798+0000 7f34ab00f640 1 -- 172.21.15.83:0/3067208722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f348c0039c0 msgr2=0x7f348c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:03.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.798+0000 7f34ab00f640 1 --2- 172.21.15.83:0/3067208722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f348c0039c0 0x7f348c003de0 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f3498009b10 tx=0x7f349801ca70 comp rx=0 tx=0).stop 2024-04-17T10:53:03.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.798+0000 7f34a95ac640 1 -- 172.21.15.83:0/3067208722 reap_dead start 2024-04-17T10:53:03.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.798+0000 7f34ab00f640 1 -- 172.21.15.83:0/3067208722 shutdown_connections 2024-04-17T10:53:03.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.798+0000 7f34ab00f640 1 -- 172.21.15.83:0/3067208722 >> 172.21.15.83:0/3067208722 conn(0x7f34a406abe0 msgr2=0x7f34a406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:03.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.798+0000 7f34ab00f640 1 -- 172.21.15.83:0/3067208722 shutdown_connections 2024-04-17T10:53:03.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.798+0000 7f34ab00f640 1 -- 172.21.15.83:0/3067208722 wait complete. 2024-04-17T10:53:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.802+0000 7f34ab00f640 1 Processor -- start 2024-04-17T10:53:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.802+0000 7f34ab00f640 1 -- start start 2024-04-17T10:53:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.802+0000 7f34ab00f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f348c0039c0 0x7f34a4159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.802+0000 7f34ab00f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34a413ab60 0x7f34a415a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.802+0000 7f34ab00f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34a413ba40 0x7f34a413bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.802+0000 7f34ab00f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a415cee0 con 0x7f34a413ab60 2024-04-17T10:53:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.802+0000 7f34ab00f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a415cd60 con 0x7f348c0039c0 2024-04-17T10:53:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.802+0000 7f34ab00f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34a4166510 con 0x7f34a413ba40 2024-04-17T10:53:03.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.802+0000 7f34a3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34a413ab60 0x7f34a415a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:03.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.802+0000 7f34a3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34a413ab60 0x7f34a415a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47708/0 (socket says 172.21.15.83:47708) 2024-04-17T10:53:03.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.802+0000 7f34a3fff640 1 -- 172.21.15.83:0/2995614249 learned_addr learned my addr 172.21.15.83:0/2995614249 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:03.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.802+0000 7f34a8dab640 1 --2- 172.21.15.83:0/2995614249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f348c0039c0 0x7f34a4159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:03.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.802+0000 7f34a95ac640 1 --2- 172.21.15.83:0/2995614249 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34a413ba40 0x7f34a413bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:03.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.802+0000 7f34a3fff640 1 -- 172.21.15.83:0/2995614249 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34a413ba40 msgr2=0x7f34a413bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:03.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.802+0000 7f34a3fff640 1 --2- 172.21.15.83:0/2995614249 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34a413ba40 0x7f34a413bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:03.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.802+0000 7f34a3fff640 1 -- 172.21.15.83:0/2995614249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f348c0039c0 msgr2=0x7f34a4159ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:03.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.802+0000 7f34a3fff640 1 --2- 172.21.15.83:0/2995614249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f348c0039c0 0x7f34a4159ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:03.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.802+0000 7f34a3fff640 1 -- 172.21.15.83:0/2995614249 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a413c7e0 con 0x7f34a413ab60 2024-04-17T10:53:03.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.802+0000 7f34a3fff640 1 --2- 172.21.15.83:0/2995614249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34a413ab60 0x7f34a415a0e0 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f349400e780 tx=0x7f349400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:03.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.802+0000 7f34a17fa640 1 -- 172.21.15.83:0/2995614249 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349400c640 con 0x7f34a413ab60 2024-04-17T10:53:03.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.802+0000 7f34a17fa640 1 -- 172.21.15.83:0/2995614249 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3494018070 con 0x7f34a413ab60 2024-04-17T10:53:03.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.802+0000 7f34ab00f640 1 -- 172.21.15.83:0/2995614249 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34a413cac0 con 0x7f34a413ab60 2024-04-17T10:53:03.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.802+0000 7f34a17fa640 1 -- 172.21.15.83:0/2995614249 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349400ca40 con 0x7f34a413ab60 2024-04-17T10:53:03.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.802+0000 7f34ab00f640 1 -- 172.21.15.83:0/2995614249 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34a4152260 con 0x7f34a413ab60 2024-04-17T10:53:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.802+0000 7f34ab00f640 1 -- 172.21.15.83:0/2995614249 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34a4152050 con 0x7f34a413ab60 2024-04-17T10:53:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.802+0000 7f34a17fa640 1 -- 172.21.15.83:0/2995614249 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f349400cbe0 con 0x7f34a413ab60 2024-04-17T10:53:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.802+0000 7f34a17fa640 1 --2- 172.21.15.83:0/2995614249 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3490066f60 0x7f3490069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.802+0000 7f34a17fa640 1 -- 172.21.15.83:0/2995614249 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f3494084dd0 con 0x7f34a413ab60 2024-04-17T10:53:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.802+0000 7f34a8dab640 1 --2- 172.21.15.83:0/2995614249 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3490066f60 0x7f3490069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.806+0000 7f34a8dab640 1 --2- 172.21.15.83:0/2995614249 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3490066f60 0x7f3490069420 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f34980ae000 tx=0x7f34980046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:03.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.810+0000 7f34a17fa640 1 -- 172.21.15.83:0/2995614249 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3494055d70 con 0x7f34a413ab60 2024-04-17T10:53:03.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.982+0000 7f34ab00f640 1 -- 172.21.15.83:0/2995614249 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34a4074f90 con 0x7f34a413ab60 2024-04-17T10:53:03.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.986+0000 7f34a17fa640 1 -- 172.21.15.83:0/2995614249 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f3494086070 con 0x7f34a413ab60 2024-04-17T10:53:03.979 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:03.979 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:03.980 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:03.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.986+0000 7f34ab00f640 1 -- 172.21.15.83:0/2995614249 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3490066f60 msgr2=0x7f3490069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:03.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.986+0000 7f34ab00f640 1 --2- 172.21.15.83:0/2995614249 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3490066f60 0x7f3490069420 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f34980ae000 tx=0x7f34980046d0 comp rx=0 tx=0).stop 2024-04-17T10:53:03.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.986+0000 7f34ab00f640 1 -- 172.21.15.83:0/2995614249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34a413ab60 msgr2=0x7f34a415a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:03.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.986+0000 7f34ab00f640 1 --2- 172.21.15.83:0/2995614249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34a413ab60 0x7f34a415a0e0 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f349400e780 tx=0x7f349400ec40 comp rx=0 tx=0).stop 2024-04-17T10:53:03.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.986+0000 7f34ab00f640 1 -- 172.21.15.83:0/2995614249 shutdown_connections 2024-04-17T10:53:03.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.986+0000 7f34ab00f640 1 --2- 172.21.15.83:0/2995614249 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3490066f60 0x7f3490069420 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:03.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.986+0000 7f34ab00f640 1 --2- 172.21.15.83:0/2995614249 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34a413ba40 0x7f34a413bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:03.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.986+0000 7f34ab00f640 1 --2- 172.21.15.83:0/2995614249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34a413ab60 0x7f34a415a0e0 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:03.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.986+0000 7f34ab00f640 1 --2- 172.21.15.83:0/2995614249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f348c0039c0 0x7f34a4159ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:03.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.986+0000 7f34ab00f640 1 -- 172.21.15.83:0/2995614249 >> 172.21.15.83:0/2995614249 conn(0x7f34a406abe0 msgr2=0x7f34a4157a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:03.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.986+0000 7f34ab00f640 1 -- 172.21.15.83:0/2995614249 shutdown_connections 2024-04-17T10:53:03.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:03.986+0000 7f34ab00f640 1 -- 172.21.15.83:0/2995614249 wait complete. 2024-04-17T10:53:04.398 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:04.399 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:05.400 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:05.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.538+0000 7f483f8fc640 1 Processor -- start 2024-04-17T10:53:05.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.538+0000 7f483f8fc640 1 -- start start 2024-04-17T10:53:05.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.538+0000 7f483f8fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f483815dfa0 0x7f48380b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:05.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.538+0000 7f483f8fc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48380b5b40 0x7f48380d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:05.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.538+0000 7f483f8fc640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4838162a00 con 0x7f48380ce980 2024-04-17T10:53:05.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.538+0000 7f483f8fc640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4838161e80 con 0x7f483815dfa0 2024-04-17T10:53:05.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.538+0000 7f483f8fc640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4838162160 con 0x7f48380b5b40 2024-04-17T10:53:05.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.538+0000 7f483d698640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f483815dfa0 0x7f48380b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:05.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.538+0000 7f483ce97640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f48380ce980 0x7f48380b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54030/0 (socket says 172.21.15.83:54030) 2024-04-17T10:53:05.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.538+0000 7f483ce97640 1 -- 172.21.15.83:0/1869937382 learned_addr learned my addr 172.21.15.83:0/1869937382 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:53:05.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.538+0000 7f482ffff640 1 -- 172.21.15.83:0/1869937382 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4234297883 0 0) 0x7f4838162a00 con 0x7f48380ce980 2024-04-17T10:53:05.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.538+0000 7f482ffff640 1 -- 172.21.15.83:0/1869937382 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4824003530 con 0x7f48380ce980 2024-04-17T10:53:05.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.538+0000 7f483de99640 1 --2- 172.21.15.83:0/1869937382 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48380b5b40 0x7f48380d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:05.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.538+0000 7f483d698640 1 -- 172.21.15.83:0/1869937382 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48380b5b40 msgr2=0x7f48380d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:05.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.538+0000 7f483d698640 1 --2- 172.21.15.83:0/1869937382 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48380b5b40 0x7f48380d1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:05.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.538+0000 7f483d698640 1 -- 172.21.15.83:0/1869937382 >> v1:172.21.15.83:6789/0 conn(0x7f48380ce980 legacy=0x7f48380b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:05.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.538+0000 7f483d698640 1 -- 172.21.15.83:0/1869937382 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4838162440 con 0x7f483815dfa0 2024-04-17T10:53:05.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f483d698640 1 --2- 172.21.15.83:0/1869937382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f483815dfa0 0x7f48380b6c20 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f48340097c0 tx=0x7f483401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f2f78f34eb0ec9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:05.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f482ffff640 1 -- 172.21.15.83:0/1869937382 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48340b0070 con 0x7f483815dfa0 2024-04-17T10:53:05.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f482ffff640 1 -- 172.21.15.83:0/1869937382 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48340ab600 con 0x7f483815dfa0 2024-04-17T10:53:05.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f482ffff640 1 -- 172.21.15.83:0/1869937382 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48340ab9d0 con 0x7f483815dfa0 2024-04-17T10:53:05.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f483f8fc640 1 -- 172.21.15.83:0/1869937382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f483815dfa0 msgr2=0x7f48380b6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:05.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f483f8fc640 1 --2- 172.21.15.83:0/1869937382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f483815dfa0 0x7f48380b6c20 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f48340097c0 tx=0x7f483401c3e0 comp rx=0 tx=0).stop 2024-04-17T10:53:05.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f483f8fc640 1 -- 172.21.15.83:0/1869937382 shutdown_connections 2024-04-17T10:53:05.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f483f8fc640 1 --2- 172.21.15.83:0/1869937382 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48380b5b40 0x7f48380d1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:05.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f483f8fc640 1 --2- 172.21.15.83:0/1869937382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f483815dfa0 0x7f48380b6c20 unknown :-1 s=CLOSED pgs=2276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:05.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f483f8fc640 1 -- 172.21.15.83:0/1869937382 >> 172.21.15.83:0/1869937382 conn(0x7f48380d0600 msgr2=0x7f48380d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:05.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f483f8fc640 1 -- 172.21.15.83:0/1869937382 shutdown_connections 2024-04-17T10:53:05.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f483f8fc640 1 -- 172.21.15.83:0/1869937382 wait complete. 2024-04-17T10:53:05.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f483f8fc640 1 Processor -- start 2024-04-17T10:53:05.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f483f8fc640 1 -- start start 2024-04-17T10:53:05.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f483f8fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48380b5b40 0x7f48380c6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:05.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f483f8fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48380ce980 0x7f48381fa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:05.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f483f8fc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48381fab20 0x7f48381585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:05.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f483f8fc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483816ae50 con 0x7f48380b5b40 2024-04-17T10:53:05.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f483f8fc640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4838161b10 con 0x7f48380ce980 2024-04-17T10:53:05.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f483f8fc640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f483816afd0 con 0x7f48381fab20 2024-04-17T10:53:05.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f483d698640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48380b5b40 0x7f48380c6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:05.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f483d698640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48380b5b40 0x7f48380c6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47724/0 (socket says 172.21.15.83:47724) 2024-04-17T10:53:05.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f483d698640 1 -- 172.21.15.83:0/3744848471 learned_addr learned my addr 172.21.15.83:0/3744848471 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:05.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f483de99640 1 --2- 172.21.15.83:0/3744848471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48381fab20 0x7f48381585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:05.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f483ce97640 1 --2- 172.21.15.83:0/3744848471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48380ce980 0x7f48381fa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:05.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f483d698640 1 -- 172.21.15.83:0/3744848471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48381fab20 msgr2=0x7f48381585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:05.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f483d698640 1 --2- 172.21.15.83:0/3744848471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48381fab20 0x7f48381585b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:05.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f483d698640 1 -- 172.21.15.83:0/3744848471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48380ce980 msgr2=0x7f48381fa5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:05.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f483d698640 1 --2- 172.21.15.83:0/3744848471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48380ce980 0x7f48381fa5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:05.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f483d698640 1 -- 172.21.15.83:0/3744848471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48381fafa0 con 0x7f48380b5b40 2024-04-17T10:53:05.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f483ce97640 1 --2- 172.21.15.83:0/3744848471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48380ce980 0x7f48381fa5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:53:05.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f483de99640 1 --2- 172.21.15.83:0/3744848471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48381fab20 0x7f48381585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:53:05.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f483d698640 1 --2- 172.21.15.83:0/3744848471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48380b5b40 0x7f48380c6c40 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f48340ae040 tx=0x7f483401ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:05.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.542+0000 7f482dffb640 1 -- 172.21.15.83:0/3744848471 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48340b0050 con 0x7f48380b5b40 2024-04-17T10:53:05.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.546+0000 7f482dffb640 1 -- 172.21.15.83:0/3744848471 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48340bc070 con 0x7f48380b5b40 2024-04-17T10:53:05.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.546+0000 7f482dffb640 1 -- 172.21.15.83:0/3744848471 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4834006e20 con 0x7f48380b5b40 2024-04-17T10:53:05.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.546+0000 7f483f8fc640 1 -- 172.21.15.83:0/3744848471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48381fb280 con 0x7f48380b5b40 2024-04-17T10:53:05.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.546+0000 7f483f8fc640 1 -- 172.21.15.83:0/3744848471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48380c7180 con 0x7f48380b5b40 2024-04-17T10:53:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.546+0000 7f483f8fc640 1 -- 172.21.15.83:0/3744848471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4838074f90 con 0x7f48380b5b40 2024-04-17T10:53:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.546+0000 7f482dffb640 1 -- 172.21.15.83:0/3744848471 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f48340b5870 con 0x7f48380b5b40 2024-04-17T10:53:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.546+0000 7f482dffb640 1 --2- 172.21.15.83:0/3744848471 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f481c066f60 0x7f481c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.546+0000 7f483ce97640 1 --2- 172.21.15.83:0/3744848471 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f481c066f60 0x7f481c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.546+0000 7f482dffb640 1 -- 172.21.15.83:0/3744848471 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f483411f750 con 0x7f48380b5b40 2024-04-17T10:53:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.546+0000 7f483ce97640 1 --2- 172.21.15.83:0/3744848471 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f481c066f60 0x7f481c069420 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f48380cb720 tx=0x7f4828009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:05.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.550+0000 7f482dffb640 1 -- 172.21.15.83:0/3744848471 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4834016090 con 0x7f48380b5b40 2024-04-17T10:53:05.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.738+0000 7f483f8fc640 1 -- 172.21.15.83:0/3744848471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48380d4790 con 0x7f48380b5b40 2024-04-17T10:53:05.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.738+0000 7f482dffb640 1 -- 172.21.15.83:0/3744848471 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f48340f0780 con 0x7f48380b5b40 2024-04-17T10:53:05.734 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:05.734 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:05.734 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.742+0000 7f483f8fc640 1 -- 172.21.15.83:0/3744848471 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f481c066f60 msgr2=0x7f481c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.742+0000 7f483f8fc640 1 --2- 172.21.15.83:0/3744848471 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f481c066f60 0x7f481c069420 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f48380cb720 tx=0x7f4828009040 comp rx=0 tx=0).stop 2024-04-17T10:53:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.742+0000 7f483f8fc640 1 -- 172.21.15.83:0/3744848471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48380b5b40 msgr2=0x7f48380c6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.742+0000 7f483f8fc640 1 --2- 172.21.15.83:0/3744848471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48380b5b40 0x7f48380c6c40 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f48340ae040 tx=0x7f483401ac20 comp rx=0 tx=0).stop 2024-04-17T10:53:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.742+0000 7f483f8fc640 1 -- 172.21.15.83:0/3744848471 shutdown_connections 2024-04-17T10:53:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.742+0000 7f483f8fc640 1 --2- 172.21.15.83:0/3744848471 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f481c066f60 0x7f481c069420 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.742+0000 7f483f8fc640 1 --2- 172.21.15.83:0/3744848471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48381fab20 0x7f48381585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.742+0000 7f483f8fc640 1 --2- 172.21.15.83:0/3744848471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48380ce980 0x7f48381fa5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.742+0000 7f483f8fc640 1 --2- 172.21.15.83:0/3744848471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48380b5b40 0x7f48380c6c40 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.742+0000 7f483f8fc640 1 -- 172.21.15.83:0/3744848471 >> 172.21.15.83:0/3744848471 conn(0x7f48380d0600 msgr2=0x7f483815e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.742+0000 7f483f8fc640 1 -- 172.21.15.83:0/3744848471 shutdown_connections 2024-04-17T10:53:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:05.742+0000 7f483f8fc640 1 -- 172.21.15.83:0/3744848471 wait complete. 2024-04-17T10:53:06.153 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:06.154 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:07.155 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:07.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.290+0000 7f32354fa640 1 Processor -- start 2024-04-17T10:53:07.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.294+0000 7f32354fa640 1 -- start start 2024-04-17T10:53:07.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.294+0000 7f32354fa640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f323006dcb0 0x7f3230165650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:07.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.294+0000 7f32354fa640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f323015e440 con 0x7f323015db50 2024-04-17T10:53:07.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.294+0000 7f32354fa640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f323015e720 con 0x7f32300aedd0 2024-04-17T10:53:07.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.294+0000 7f32354fa640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f323015ea00 con 0x7f323006dcb0 2024-04-17T10:53:07.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.294+0000 7f322effd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f323015db50 0x7f3230070470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33962/0 (socket says 172.21.15.83:33962) 2024-04-17T10:53:07.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.294+0000 7f322effd640 1 -- 172.21.15.83:0/2617016356 learned_addr learned my addr 172.21.15.83:0/2617016356 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:53:07.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.294+0000 7f322f7fe640 1 --2- 172.21.15.83:0/2617016356 >> v2:172.21.15.167:3300/0 conn(0x7f323006dcb0 0x7f3230165650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:07.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.294+0000 7f322dffb640 1 -- 172.21.15.83:0/2617016356 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4000128632 0 0) 0x7f323015e440 con 0x7f323015db50 2024-04-17T10:53:07.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.294+0000 7f322dffb640 1 -- 172.21.15.83:0/2617016356 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3218003530 con 0x7f323015db50 2024-04-17T10:53:07.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.294+0000 7f322dffb640 1 -- 172.21.15.83:0/2617016356 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 67355204 0 0) 0x7f3218003530 con 0x7f323015db50 2024-04-17T10:53:07.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.294+0000 7f322dffb640 1 -- 172.21.15.83:0/2617016356 >> v2:172.21.15.167:3300/0 conn(0x7f323006dcb0 msgr2=0x7f3230165650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:07.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.294+0000 7f322dffb640 1 --2- 172.21.15.83:0/2617016356 >> v2:172.21.15.167:3300/0 conn(0x7f323006dcb0 0x7f3230165650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:07.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.294+0000 7f322dffb640 1 -- 172.21.15.83:0/2617016356 >> v1:172.21.15.167:6789/0 conn(0x7f32300aedd0 legacy=0x7f323006d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:07.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.294+0000 7f322dffb640 1 -- 172.21.15.83:0/2617016356 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323015ece0 con 0x7f323015db50 2024-04-17T10:53:07.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.294+0000 7f322dffb640 1 -- 172.21.15.83:0/2617016356 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f3220001680 con 0x7f323015db50 2024-04-17T10:53:07.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.294+0000 7f322dffb640 1 -- 172.21.15.83:0/2617016356 >> v1:172.21.15.83:6789/0 conn(0x7f323015db50 legacy=0x7f3230070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:07.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.294+0000 7f322dffb640 1 --2- 172.21.15.83:0/2617016356 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32180039f0 0x7f3218003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:07.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.294+0000 7f322dffb640 1 --2- 172.21.15.83:0/2617016356 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3218004810 0x7f3218004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:07.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.294+0000 7f322dffb640 1 --2- 172.21.15.83:0/2617016356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32180051d0 0x7f321800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:07.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.294+0000 7f322dffb640 1 -- 172.21.15.83:0/2617016356 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323015ea00 con 0x7f32180051d0 2024-04-17T10:53:07.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.294+0000 7f322dffb640 1 -- 172.21.15.83:0/2617016356 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321800c0a0 con 0x7f32180039f0 2024-04-17T10:53:07.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.294+0000 7f322dffb640 1 -- 172.21.15.83:0/2617016356 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f321800c2a0 con 0x7f3218004810 2024-04-17T10:53:07.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.294+0000 7f322f7fe640 1 --2- 172.21.15.83:0/2617016356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32180051d0 0x7f321800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:07.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.294+0000 7f322effd640 1 --2- 172.21.15.83:0/2617016356 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32180039f0 0x7f3218003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:07.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.294+0000 7f322e7fc640 1 --2- 172.21.15.83:0/2617016356 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3218004810 0x7f3218004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:07.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.294+0000 7f322f7fe640 1 -- 172.21.15.83:0/2617016356 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3218004810 msgr2=0x7f3218004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:07.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.294+0000 7f322f7fe640 1 --2- 172.21.15.83:0/2617016356 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3218004810 0x7f3218004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:07.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.294+0000 7f322f7fe640 1 -- 172.21.15.83:0/2617016356 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32180039f0 msgr2=0x7f3218003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:07.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.294+0000 7f322f7fe640 1 --2- 172.21.15.83:0/2617016356 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32180039f0 0x7f3218003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:07.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.294+0000 7f322f7fe640 1 -- 172.21.15.83:0/2617016356 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3218003530 con 0x7f32180051d0 2024-04-17T10:53:07.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.294+0000 7f322effd640 1 --2- 172.21.15.83:0/2617016356 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32180039f0 0x7f3218003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:53:07.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.298+0000 7f322f7fe640 1 -- 172.21.15.83:0/2617016356 reap_dead start 2024-04-17T10:53:07.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.298+0000 7f322f7fe640 1 --2- 172.21.15.83:0/2617016356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32180051d0 0x7f321800b9d0 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f3228009f90 tx=0x7f322801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70ac29e8ab6b9f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:07.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.298+0000 7f322dffb640 1 -- 172.21.15.83:0/2617016356 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3228009c10 con 0x7f32180051d0 2024-04-17T10:53:07.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.298+0000 7f322dffb640 1 -- 172.21.15.83:0/2617016356 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32280a5c90 con 0x7f32180051d0 2024-04-17T10:53:07.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.298+0000 7f322dffb640 1 -- 172.21.15.83:0/2617016356 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3228003be0 con 0x7f32180051d0 2024-04-17T10:53:07.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.298+0000 7f32354fa640 1 -- 172.21.15.83:0/2617016356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32180051d0 msgr2=0x7f321800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:07.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.298+0000 7f32354fa640 1 --2- 172.21.15.83:0/2617016356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32180051d0 0x7f321800b9d0 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f3228009f90 tx=0x7f322801cc80 comp rx=0 tx=0).stop 2024-04-17T10:53:07.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.298+0000 7f32354fa640 1 -- 172.21.15.83:0/2617016356 shutdown_connections 2024-04-17T10:53:07.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.298+0000 7f32354fa640 1 --2- 172.21.15.83:0/2617016356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32180051d0 0x7f321800b9d0 unknown :-1 s=CLOSED pgs=2279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:07.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.298+0000 7f32354fa640 1 -- 172.21.15.83:0/2617016356 >> 172.21.15.83:0/2617016356 conn(0x7f323006abe0 msgr2=0x7f323006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:07.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.298+0000 7f32354fa640 1 -- 172.21.15.83:0/2617016356 shutdown_connections 2024-04-17T10:53:07.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.298+0000 7f32354fa640 1 -- 172.21.15.83:0/2617016356 wait complete. 2024-04-17T10:53:07.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.298+0000 7f32354fa640 1 Processor -- start 2024-04-17T10:53:07.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.298+0000 7f32354fa640 1 -- start start 2024-04-17T10:53:07.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.302+0000 7f32354fa640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32180051d0 0x7f3230155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:07.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.302+0000 7f32354fa640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3230156900 0x7f32300ab340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:07.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.302+0000 7f32354fa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32300ab880 0x7f323015c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:07.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.302+0000 7f32354fa640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323015df30 con 0x7f32300ab880 2024-04-17T10:53:07.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.302+0000 7f32354fa640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323015e2c0 con 0x7f32180051d0 2024-04-17T10:53:07.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.302+0000 7f32354fa640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323015e0b0 con 0x7f3230156900 2024-04-17T10:53:07.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.302+0000 7f322f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32300ab880 0x7f323015c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:07.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.302+0000 7f322f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32300ab880 0x7f323015c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34868/0 (socket says 172.21.15.83:34868) 2024-04-17T10:53:07.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.302+0000 7f322f7fe640 1 -- 172.21.15.83:0/1140253577 learned_addr learned my addr 172.21.15.83:0/1140253577 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:07.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.302+0000 7f322e7fc640 1 --2- 172.21.15.83:0/1140253577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3230156900 0x7f32300ab340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:07.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.302+0000 7f322effd640 1 --2- 172.21.15.83:0/1140253577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32180051d0 0x7f3230155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:07.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.302+0000 7f322f7fe640 1 -- 172.21.15.83:0/1140253577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3230156900 msgr2=0x7f32300ab340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:07.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.302+0000 7f322f7fe640 1 --2- 172.21.15.83:0/1140253577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3230156900 0x7f32300ab340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:07.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.302+0000 7f322f7fe640 1 -- 172.21.15.83:0/1140253577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32180051d0 msgr2=0x7f3230155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:07.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.302+0000 7f322f7fe640 1 --2- 172.21.15.83:0/1140253577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32180051d0 0x7f3230155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:07.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.302+0000 7f322f7fe640 1 -- 172.21.15.83:0/1140253577 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323015b3a0 con 0x7f32300ab880 2024-04-17T10:53:07.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.302+0000 7f322e7fc640 1 --2- 172.21.15.83:0/1140253577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3230156900 0x7f32300ab340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:53:07.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.302+0000 7f322effd640 1 --2- 172.21.15.83:0/1140253577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32180051d0 0x7f3230155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:53:07.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.302+0000 7f322f7fe640 1 --2- 172.21.15.83:0/1140253577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32300ab880 0x7f323015c160 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f322801c6f0 tx=0x7f3228009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:07.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.302+0000 7f32177fe640 1 -- 172.21.15.83:0/1140253577 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3228009c10 con 0x7f32300ab880 2024-04-17T10:53:07.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.302+0000 7f32177fe640 1 -- 172.21.15.83:0/1140253577 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32280a6e50 con 0x7f32300ab880 2024-04-17T10:53:07.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.302+0000 7f32354fa640 1 -- 172.21.15.83:0/1140253577 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f323015b680 con 0x7f32300ab880 2024-04-17T10:53:07.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.302+0000 7f32177fe640 1 -- 172.21.15.83:0/1140253577 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3228003e40 con 0x7f32300ab880 2024-04-17T10:53:07.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.302+0000 7f32354fa640 1 -- 172.21.15.83:0/1140253577 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3230155a50 con 0x7f32300ab880 2024-04-17T10:53:07.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.302+0000 7f32354fa640 1 -- 172.21.15.83:0/1140253577 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3230074f90 con 0x7f32300ab880 2024-04-17T10:53:07.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.302+0000 7f32177fe640 1 -- 172.21.15.83:0/1140253577 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f32280be020 con 0x7f32300ab880 2024-04-17T10:53:07.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.302+0000 7f32177fe640 1 --2- 172.21.15.83:0/1140253577 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f321c066f90 0x7f321c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:07.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.302+0000 7f32177fe640 1 -- 172.21.15.83:0/1140253577 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f3228120560 con 0x7f32300ab880 2024-04-17T10:53:07.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.302+0000 7f322effd640 1 --2- 172.21.15.83:0/1140253577 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f321c066f90 0x7f321c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:07.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.306+0000 7f322effd640 1 --2- 172.21.15.83:0/1140253577 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f321c066f90 0x7f321c069450 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f3220000c10 tx=0x7f3220001160 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:07.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.306+0000 7f32177fe640 1 -- 172.21.15.83:0/1140253577 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3228016990 con 0x7f32300ab880 2024-04-17T10:53:07.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.502+0000 7f32354fa640 1 -- 172.21.15.83:0/1140253577 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32300b2af0 con 0x7f32300ab880 2024-04-17T10:53:07.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.502+0000 7f32177fe640 1 -- 172.21.15.83:0/1140253577 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f32280f1720 con 0x7f32300ab880 2024-04-17T10:53:07.498 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:07.498 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:07.498 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:07.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.506+0000 7f32354fa640 1 -- 172.21.15.83:0/1140253577 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f321c066f90 msgr2=0x7f321c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:07.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.506+0000 7f32354fa640 1 --2- 172.21.15.83:0/1140253577 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f321c066f90 0x7f321c069450 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f3220000c10 tx=0x7f3220001160 comp rx=0 tx=0).stop 2024-04-17T10:53:07.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.506+0000 7f32354fa640 1 -- 172.21.15.83:0/1140253577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32300ab880 msgr2=0x7f323015c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:07.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.506+0000 7f32354fa640 1 --2- 172.21.15.83:0/1140253577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32300ab880 0x7f323015c160 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f322801c6f0 tx=0x7f3228009750 comp rx=0 tx=0).stop 2024-04-17T10:53:07.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.506+0000 7f32354fa640 1 -- 172.21.15.83:0/1140253577 shutdown_connections 2024-04-17T10:53:07.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.506+0000 7f32354fa640 1 --2- 172.21.15.83:0/1140253577 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f321c066f90 0x7f321c069450 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:07.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.506+0000 7f32354fa640 1 --2- 172.21.15.83:0/1140253577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32300ab880 0x7f323015c160 unknown :-1 s=CLOSED pgs=2280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:07.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.506+0000 7f32354fa640 1 --2- 172.21.15.83:0/1140253577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3230156900 0x7f32300ab340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:07.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.506+0000 7f32354fa640 1 --2- 172.21.15.83:0/1140253577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32180051d0 0x7f3230155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:07.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.506+0000 7f32354fa640 1 -- 172.21.15.83:0/1140253577 >> 172.21.15.83:0/1140253577 conn(0x7f323006abe0 msgr2=0x7f323006c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:07.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.506+0000 7f32354fa640 1 -- 172.21.15.83:0/1140253577 shutdown_connections 2024-04-17T10:53:07.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:07.506+0000 7f32354fa640 1 -- 172.21.15.83:0/1140253577 wait complete. 2024-04-17T10:53:07.920 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:07.921 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:08.921 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:09.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca4e970640 1 Processor -- start 2024-04-17T10:53:09.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca4e970640 1 -- start start 2024-04-17T10:53:09.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca4e970640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fca480ce980 0x7fca480b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:09.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca4e970640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca48162a00 con 0x7fca4815dfa0 2024-04-17T10:53:09.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca4e970640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca48161e80 con 0x7fca480cee00 2024-04-17T10:53:09.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca4e970640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca48162160 con 0x7fca480ce980 2024-04-17T10:53:09.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca47fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fca4815dfa0 0x7fca480b6c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33968/0 (socket says 172.21.15.83:33968) 2024-04-17T10:53:09.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca47fff640 1 -- 172.21.15.83:0/3857450199 learned_addr learned my addr 172.21.15.83:0/3857450199 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:53:09.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca477fe640 1 --2- 172.21.15.83:0/3857450199 >> v2:172.21.15.167:3300/0 conn(0x7fca480ce980 0x7fca480b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:09.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca46ffd640 1 -- 172.21.15.83:0/3857450199 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1542985360 0 0) 0x7fca48162a00 con 0x7fca4815dfa0 2024-04-17T10:53:09.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca46ffd640 1 -- 172.21.15.83:0/3857450199 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca30003530 con 0x7fca4815dfa0 2024-04-17T10:53:09.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca46ffd640 1 -- 172.21.15.83:0/3857450199 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1651492929 0 0) 0x7fca30003530 con 0x7fca4815dfa0 2024-04-17T10:53:09.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca46ffd640 1 -- 172.21.15.83:0/3857450199 >> v2:172.21.15.167:3300/0 conn(0x7fca480ce980 msgr2=0x7fca480b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:09.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca46ffd640 1 --2- 172.21.15.83:0/3857450199 >> v2:172.21.15.167:3300/0 conn(0x7fca480ce980 0x7fca480b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:09.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca46ffd640 1 -- 172.21.15.83:0/3857450199 >> v1:172.21.15.167:6789/0 conn(0x7fca480cee00 legacy=0x7fca480d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:09.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca46ffd640 1 -- 172.21.15.83:0/3857450199 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca48162440 con 0x7fca4815dfa0 2024-04-17T10:53:09.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca46ffd640 1 -- 172.21.15.83:0/3857450199 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fca38001680 con 0x7fca4815dfa0 2024-04-17T10:53:09.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca46ffd640 1 -- 172.21.15.83:0/3857450199 >> v1:172.21.15.83:6789/0 conn(0x7fca4815dfa0 legacy=0x7fca480b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:09.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca46ffd640 1 --2- 172.21.15.83:0/3857450199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca300039c0 0x7fca30003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:09.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca46ffd640 1 --2- 172.21.15.83:0/3857450199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca30004810 0x7fca30004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:09.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca46ffd640 1 --2- 172.21.15.83:0/3857450199 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca300051d0 0x7fca3000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:09.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca46ffd640 1 -- 172.21.15.83:0/3857450199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca48162160 con 0x7fca300039c0 2024-04-17T10:53:09.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca46ffd640 1 -- 172.21.15.83:0/3857450199 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3000c170 con 0x7fca30004810 2024-04-17T10:53:09.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca46ffd640 1 -- 172.21.15.83:0/3857450199 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3000c370 con 0x7fca300051d0 2024-04-17T10:53:09.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca47fff640 1 --2- 172.21.15.83:0/3857450199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca300039c0 0x7fca30003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:09.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca477fe640 1 --2- 172.21.15.83:0/3857450199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca30004810 0x7fca30004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:09.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca4cf0d640 1 --2- 172.21.15.83:0/3857450199 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca300051d0 0x7fca3000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:09.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca47fff640 1 -- 172.21.15.83:0/3857450199 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca300051d0 msgr2=0x7fca3000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:09.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca47fff640 1 --2- 172.21.15.83:0/3857450199 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca300051d0 0x7fca3000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:09.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca47fff640 1 -- 172.21.15.83:0/3857450199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca30004810 msgr2=0x7fca30004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:09.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca47fff640 1 --2- 172.21.15.83:0/3857450199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca30004810 0x7fca30004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:09.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca4cf0d640 1 -- 172.21.15.83:0/3857450199 reap_dead start 2024-04-17T10:53:09.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.058+0000 7fca47fff640 1 -- 172.21.15.83:0/3857450199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca30003530 con 0x7fca300039c0 2024-04-17T10:53:09.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.062+0000 7fca47fff640 1 --2- 172.21.15.83:0/3857450199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca300039c0 0x7fca30003de0 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7fca3800c210 tx=0x7fca3801cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a73ef1778dd86ac server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:09.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.062+0000 7fca46ffd640 1 -- 172.21.15.83:0/3857450199 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca38007d40 con 0x7fca300039c0 2024-04-17T10:53:09.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.062+0000 7fca46ffd640 1 -- 172.21.15.83:0/3857450199 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca380a9070 con 0x7fca300039c0 2024-04-17T10:53:09.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.062+0000 7fca46ffd640 1 -- 172.21.15.83:0/3857450199 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca380a5df0 con 0x7fca300039c0 2024-04-17T10:53:09.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.062+0000 7fca4e970640 1 -- 172.21.15.83:0/3857450199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca300039c0 msgr2=0x7fca30003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:09.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.062+0000 7fca4e970640 1 --2- 172.21.15.83:0/3857450199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca300039c0 0x7fca30003de0 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7fca3800c210 tx=0x7fca3801cc50 comp rx=0 tx=0).stop 2024-04-17T10:53:09.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.062+0000 7fca4e970640 1 -- 172.21.15.83:0/3857450199 shutdown_connections 2024-04-17T10:53:09.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.062+0000 7fca4e970640 1 --2- 172.21.15.83:0/3857450199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca300039c0 0x7fca30003de0 unknown :-1 s=CLOSED pgs=2282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:09.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.062+0000 7fca4e970640 1 -- 172.21.15.83:0/3857450199 >> 172.21.15.83:0/3857450199 conn(0x7fca480d0600 msgr2=0x7fca480d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:09.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.062+0000 7fca4e970640 1 -- 172.21.15.83:0/3857450199 shutdown_connections 2024-04-17T10:53:09.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.062+0000 7fca4e970640 1 -- 172.21.15.83:0/3857450199 wait complete. 2024-04-17T10:53:09.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.062+0000 7fca4e970640 1 Processor -- start 2024-04-17T10:53:09.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.062+0000 7fca4e970640 1 -- start start 2024-04-17T10:53:09.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.066+0000 7fca4e970640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca300039c0 0x7fca4814cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:09.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.066+0000 7fca4e970640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca480b76f0 0x7fca481f7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:09.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.066+0000 7fca4e970640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca481f76a0 0x7fca48152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:09.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.066+0000 7fca4e970640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca480b5430 con 0x7fca480b76f0 2024-04-17T10:53:09.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.066+0000 7fca4e970640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca48161b10 con 0x7fca300039c0 2024-04-17T10:53:09.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.066+0000 7fca4e970640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca480b55b0 con 0x7fca481f76a0 2024-04-17T10:53:09.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.066+0000 7fca477fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca480b76f0 0x7fca481f7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:09.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.066+0000 7fca477fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca480b76f0 0x7fca481f7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34876/0 (socket says 172.21.15.83:34876) 2024-04-17T10:53:09.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.066+0000 7fca477fe640 1 -- 172.21.15.83:0/429226304 learned_addr learned my addr 172.21.15.83:0/429226304 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:09.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.066+0000 7fca47fff640 1 --2- 172.21.15.83:0/429226304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca300039c0 0x7fca4814cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:09.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.066+0000 7fca4cf0d640 1 --2- 172.21.15.83:0/429226304 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca481f76a0 0x7fca48152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:09.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.066+0000 7fca477fe640 1 -- 172.21.15.83:0/429226304 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca481f76a0 msgr2=0x7fca48152130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:09.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.066+0000 7fca477fe640 1 --2- 172.21.15.83:0/429226304 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca481f76a0 0x7fca48152130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:09.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.066+0000 7fca477fe640 1 -- 172.21.15.83:0/429226304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca300039c0 msgr2=0x7fca4814cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:09.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.066+0000 7fca477fe640 1 --2- 172.21.15.83:0/429226304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca300039c0 0x7fca4814cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:09.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.066+0000 7fca477fe640 1 -- 172.21.15.83:0/429226304 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca481f69b0 con 0x7fca480b76f0 2024-04-17T10:53:09.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.066+0000 7fca477fe640 1 --2- 172.21.15.83:0/429226304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca480b76f0 0x7fca481f7160 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7fca3c011660 tx=0x7fca3c011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:09.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.066+0000 7fca44ff9640 1 -- 172.21.15.83:0/429226304 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c00f9c0 con 0x7fca480b76f0 2024-04-17T10:53:09.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.066+0000 7fca44ff9640 1 -- 172.21.15.83:0/429226304 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3c00fb60 con 0x7fca480b76f0 2024-04-17T10:53:09.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.066+0000 7fca4e970640 1 -- 172.21.15.83:0/429226304 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca4814eb40 con 0x7fca480b76f0 2024-04-17T10:53:09.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.066+0000 7fca44ff9640 1 -- 172.21.15.83:0/429226304 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c015460 con 0x7fca480b76f0 2024-04-17T10:53:09.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.066+0000 7fca4e970640 1 -- 172.21.15.83:0/429226304 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca4814f100 con 0x7fca480b76f0 2024-04-17T10:53:09.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.066+0000 7fca4e970640 1 -- 172.21.15.83:0/429226304 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca480681a0 con 0x7fca480b76f0 2024-04-17T10:53:09.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.066+0000 7fca44ff9640 1 -- 172.21.15.83:0/429226304 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fca3c015600 con 0x7fca480b76f0 2024-04-17T10:53:09.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.066+0000 7fca44ff9640 1 --2- 172.21.15.83:0/429226304 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fca34066f60 0x7fca34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:09.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.066+0000 7fca44ff9640 1 -- 172.21.15.83:0/429226304 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fca3c085a20 con 0x7fca480b76f0 2024-04-17T10:53:09.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.066+0000 7fca47fff640 1 --2- 172.21.15.83:0/429226304 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fca34066f60 0x7fca34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:09.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.070+0000 7fca47fff640 1 --2- 172.21.15.83:0/429226304 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fca34066f60 0x7fca34069420 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fca380ab000 tx=0x7fca380b0040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:09.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.070+0000 7fca44ff9640 1 -- 172.21.15.83:0/429226304 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fca3c019030 con 0x7fca480b76f0 2024-04-17T10:53:09.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.258+0000 7fca4e970640 1 -- 172.21.15.83:0/429226304 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca48074f90 con 0x7fca480b76f0 2024-04-17T10:53:09.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.258+0000 7fca44ff9640 1 -- 172.21.15.83:0/429226304 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fca3c0569c0 con 0x7fca480b76f0 2024-04-17T10:53:09.254 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:09.254 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:09.254 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:09.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.262+0000 7fca4e970640 1 -- 172.21.15.83:0/429226304 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fca34066f60 msgr2=0x7fca34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:09.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.262+0000 7fca4e970640 1 --2- 172.21.15.83:0/429226304 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fca34066f60 0x7fca34069420 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fca380ab000 tx=0x7fca380b0040 comp rx=0 tx=0).stop 2024-04-17T10:53:09.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.262+0000 7fca4e970640 1 -- 172.21.15.83:0/429226304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca480b76f0 msgr2=0x7fca481f7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:09.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.262+0000 7fca4e970640 1 --2- 172.21.15.83:0/429226304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca480b76f0 0x7fca481f7160 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7fca3c011660 tx=0x7fca3c011b20 comp rx=0 tx=0).stop 2024-04-17T10:53:09.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.262+0000 7fca4e970640 1 -- 172.21.15.83:0/429226304 shutdown_connections 2024-04-17T10:53:09.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.262+0000 7fca4e970640 1 --2- 172.21.15.83:0/429226304 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fca34066f60 0x7fca34069420 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:09.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.262+0000 7fca4e970640 1 --2- 172.21.15.83:0/429226304 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca481f76a0 0x7fca48152130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:09.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.262+0000 7fca4e970640 1 --2- 172.21.15.83:0/429226304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca480b76f0 0x7fca481f7160 unknown :-1 s=CLOSED pgs=2283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:09.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.262+0000 7fca4e970640 1 --2- 172.21.15.83:0/429226304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca300039c0 0x7fca4814cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:09.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.262+0000 7fca4e970640 1 -- 172.21.15.83:0/429226304 >> 172.21.15.83:0/429226304 conn(0x7fca480d0600 msgr2=0x7fca480cd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:09.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.262+0000 7fca4e970640 1 -- 172.21.15.83:0/429226304 shutdown_connections 2024-04-17T10:53:09.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:09.262+0000 7fca4e970640 1 -- 172.21.15.83:0/429226304 wait complete. 2024-04-17T10:53:09.294 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:53:09.296 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:53:09.675 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:09.675 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:10.676 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:10.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.810+0000 7fa0123d9640 1 Processor -- start 2024-04-17T10:53:10.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.814+0000 7fa0123d9640 1 -- start start 2024-04-17T10:53:10.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.814+0000 7fa0123d9640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa00c15ba30 0x7fa00c072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:10.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.814+0000 7fa0123d9640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa00c15c960 0x7fa00c0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:10.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.814+0000 7fa0123d9640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa00c0cfa10 0x7fa00c06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:10.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.814+0000 7fa0123d9640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa00c071ea0 con 0x7fa00c15ba30 2024-04-17T10:53:10.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.814+0000 7fa0123d9640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa00c0cfea0 con 0x7fa00c15c960 2024-04-17T10:53:10.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.814+0000 7fa0123d9640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa00c06e320 con 0x7fa00c0cfa10 2024-04-17T10:53:10.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.814+0000 7fa00bfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa00c15ba30 0x7fa00c072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:10.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.814+0000 7fa00bfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa00c15ba30 0x7fa00c072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34884/0 (socket says 172.21.15.83:34884) 2024-04-17T10:53:10.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.814+0000 7fa00bfff640 1 -- 172.21.15.83:0/1225419227 learned_addr learned my addr 172.21.15.83:0/1225419227 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:10.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.814+0000 7fa010976640 1 --2- 172.21.15.83:0/1225419227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa00c0cfa10 0x7fa00c06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:10.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.814+0000 7fa00b7fe640 1 --2- 172.21.15.83:0/1225419227 >> v2:172.21.15.167:3300/0 conn(0x7fa00c15c960 0x7fa00c0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:10.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.814+0000 7fa00bfff640 1 -- 172.21.15.83:0/1225419227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa00c0cfa10 msgr2=0x7fa00c06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:10.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.814+0000 7fa00bfff640 1 --2- 172.21.15.83:0/1225419227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa00c0cfa10 0x7fa00c06c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:10.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.814+0000 7fa00bfff640 1 -- 172.21.15.83:0/1225419227 >> v2:172.21.15.167:3300/0 conn(0x7fa00c15c960 msgr2=0x7fa00c0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:10.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.814+0000 7fa00bfff640 1 --2- 172.21.15.83:0/1225419227 >> v2:172.21.15.167:3300/0 conn(0x7fa00c15c960 0x7fa00c0cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:10.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.814+0000 7fa00bfff640 1 -- 172.21.15.83:0/1225419227 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa00c06e040 con 0x7fa00c15ba30 2024-04-17T10:53:10.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.814+0000 7fa010976640 1 --2- 172.21.15.83:0/1225419227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa00c0cfa10 0x7fa00c06c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:53:10.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.814+0000 7fa00b7fe640 1 --2- 172.21.15.83:0/1225419227 >> v2:172.21.15.167:3300/0 conn(0x7fa00c15c960 0x7fa00c0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:53:10.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa00bfff640 1 --2- 172.21.15.83:0/1225419227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa00c15ba30 0x7fa00c072330 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7fa00c0af2b0 tx=0x7f9ffc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28934191951f157a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:10.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa00affd640 1 -- 172.21.15.83:0/1225419227 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc0b0070 con 0x7fa00c15ba30 2024-04-17T10:53:10.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa00affd640 1 -- 172.21.15.83:0/1225419227 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ffc0ab600 con 0x7fa00c15ba30 2024-04-17T10:53:10.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa00affd640 1 -- 172.21.15.83:0/1225419227 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc0ab9d0 con 0x7fa00c15ba30 2024-04-17T10:53:10.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa0123d9640 1 -- 172.21.15.83:0/1225419227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa00c15ba30 msgr2=0x7fa00c072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:10.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa0123d9640 1 --2- 172.21.15.83:0/1225419227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa00c15ba30 0x7fa00c072330 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7fa00c0af2b0 tx=0x7f9ffc01c490 comp rx=0 tx=0).stop 2024-04-17T10:53:10.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa0123d9640 1 -- 172.21.15.83:0/1225419227 shutdown_connections 2024-04-17T10:53:10.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa0123d9640 1 --2- 172.21.15.83:0/1225419227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa00c0cfa10 0x7fa00c06c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:10.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa0123d9640 1 --2- 172.21.15.83:0/1225419227 >> v2:172.21.15.167:3300/0 conn(0x7fa00c15c960 0x7fa00c0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:10.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa0123d9640 1 --2- 172.21.15.83:0/1225419227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa00c15ba30 0x7fa00c072330 unknown :-1 s=CLOSED pgs=2284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:10.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa0123d9640 1 -- 172.21.15.83:0/1225419227 >> 172.21.15.83:0/1225419227 conn(0x7fa00c06abe0 msgr2=0x7fa00c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:10.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa0123d9640 1 -- 172.21.15.83:0/1225419227 shutdown_connections 2024-04-17T10:53:10.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa0123d9640 1 -- 172.21.15.83:0/1225419227 wait complete. 2024-04-17T10:53:10.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa0123d9640 1 Processor -- start 2024-04-17T10:53:10.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa0123d9640 1 -- start start 2024-04-17T10:53:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa0123d9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa00c0cfa10 0x7fa00c145ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa0123d9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa00c15ba30 0x7fa00c0b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa0123d9640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa00c15c960 0x7fa00c0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa0123d9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa00c06de20 con 0x7fa00c15ba30 2024-04-17T10:53:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa0123d9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa00c06dca0 con 0x7fa00c0cfa10 2024-04-17T10:53:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa0123d9640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa00c15cd40 con 0x7fa00c15c960 2024-04-17T10:53:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa00b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa00c15ba30 0x7fa00c0b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa00b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa00c15ba30 0x7fa00c0b05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34886/0 (socket says 172.21.15.83:34886) 2024-04-17T10:53:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa00b7fe640 1 -- 172.21.15.83:0/2856395746 learned_addr learned my addr 172.21.15.83:0/2856395746 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa010976640 1 --2- 172.21.15.83:0/2856395746 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa00c15c960 0x7fa00c0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa00bfff640 1 --2- 172.21.15.83:0/2856395746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa00c0cfa10 0x7fa00c145ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa00b7fe640 1 -- 172.21.15.83:0/2856395746 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa00c15c960 msgr2=0x7fa00c0ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa00b7fe640 1 --2- 172.21.15.83:0/2856395746 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa00c15c960 0x7fa00c0ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa00b7fe640 1 -- 172.21.15.83:0/2856395746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa00c0cfa10 msgr2=0x7fa00c145ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:10.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa00b7fe640 1 --2- 172.21.15.83:0/2856395746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa00c0cfa10 0x7fa00c145ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:10.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa00b7fe640 1 -- 172.21.15.83:0/2856395746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa00c0b0b00 con 0x7fa00c15ba30 2024-04-17T10:53:10.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa00b7fe640 1 --2- 172.21.15.83:0/2856395746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa00c15ba30 0x7fa00c0b05c0 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7fa00000c820 tx=0x7fa00000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:10.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa008ff9640 1 -- 172.21.15.83:0/2856395746 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa000015070 con 0x7fa00c15ba30 2024-04-17T10:53:10.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa008ff9640 1 -- 172.21.15.83:0/2856395746 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa000007a30 con 0x7fa00c15ba30 2024-04-17T10:53:10.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa0123d9640 1 -- 172.21.15.83:0/2856395746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa00c153400 con 0x7fa00c15ba30 2024-04-17T10:53:10.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa008ff9640 1 -- 172.21.15.83:0/2856395746 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa000007e40 con 0x7fa00c15ba30 2024-04-17T10:53:10.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.818+0000 7fa0123d9640 1 -- 172.21.15.83:0/2856395746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa00c1539c0 con 0x7fa00c15ba30 2024-04-17T10:53:10.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.822+0000 7fa0123d9640 1 -- 172.21.15.83:0/2856395746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa00c074f90 con 0x7fa00c15ba30 2024-04-17T10:53:10.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.822+0000 7fa008ff9640 1 -- 172.21.15.83:0/2856395746 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa000018070 con 0x7fa00c15ba30 2024-04-17T10:53:10.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.822+0000 7fa008ff9640 1 --2- 172.21.15.83:0/2856395746 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9ff8066f60 0x7f9ff8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:10.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.822+0000 7fa00bfff640 1 --2- 172.21.15.83:0/2856395746 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9ff8066f60 0x7f9ff8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:10.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.822+0000 7fa008ff9640 1 -- 172.21.15.83:0/2856395746 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fa000085010 con 0x7fa00c15ba30 2024-04-17T10:53:10.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.822+0000 7fa00bfff640 1 --2- 172.21.15.83:0/2856395746 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9ff8066f60 0x7f9ff8069420 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fa00c0b2410 tx=0x7f9ffc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:10.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:10.826+0000 7fa008ff9640 1 -- 172.21.15.83:0/2856395746 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa000055f20 con 0x7fa00c15ba30 2024-04-17T10:53:11.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:11.018+0000 7fa0123d9640 1 -- 172.21.15.83:0/2856395746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa00c0b0fd0 con 0x7fa00c15ba30 2024-04-17T10:53:11.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:11.018+0000 7fa008ff9640 1 -- 172.21.15.83:0/2856395746 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fa000055820 con 0x7fa00c15ba30 2024-04-17T10:53:11.013 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:11.013 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:11.013 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:11.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:11.018+0000 7fa0123d9640 1 -- 172.21.15.83:0/2856395746 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9ff8066f60 msgr2=0x7f9ff8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:11.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:11.018+0000 7fa0123d9640 1 --2- 172.21.15.83:0/2856395746 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9ff8066f60 0x7f9ff8069420 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fa00c0b2410 tx=0x7f9ffc009040 comp rx=0 tx=0).stop 2024-04-17T10:53:11.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:11.018+0000 7fa0123d9640 1 -- 172.21.15.83:0/2856395746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa00c15ba30 msgr2=0x7fa00c0b05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:11.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:11.018+0000 7fa0123d9640 1 --2- 172.21.15.83:0/2856395746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa00c15ba30 0x7fa00c0b05c0 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7fa00000c820 tx=0x7fa00000cce0 comp rx=0 tx=0).stop 2024-04-17T10:53:11.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:11.022+0000 7fa0123d9640 1 -- 172.21.15.83:0/2856395746 shutdown_connections 2024-04-17T10:53:11.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:11.022+0000 7fa0123d9640 1 --2- 172.21.15.83:0/2856395746 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9ff8066f60 0x7f9ff8069420 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:11.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:11.022+0000 7fa0123d9640 1 --2- 172.21.15.83:0/2856395746 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa00c15c960 0x7fa00c0ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:11.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:11.022+0000 7fa0123d9640 1 --2- 172.21.15.83:0/2856395746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa00c15ba30 0x7fa00c0b05c0 unknown :-1 s=CLOSED pgs=2285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:11.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:11.022+0000 7fa0123d9640 1 --2- 172.21.15.83:0/2856395746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa00c0cfa10 0x7fa00c145ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:11.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:11.022+0000 7fa0123d9640 1 -- 172.21.15.83:0/2856395746 >> 172.21.15.83:0/2856395746 conn(0x7fa00c06abe0 msgr2=0x7fa00c0cd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:11.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:11.022+0000 7fa0123d9640 1 -- 172.21.15.83:0/2856395746 shutdown_connections 2024-04-17T10:53:11.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:11.022+0000 7fa0123d9640 1 -- 172.21.15.83:0/2856395746 wait complete. 2024-04-17T10:53:11.434 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:11.434 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:12.436 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:12.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.573+0000 7f6cce1c9640 1 Processor -- start 2024-04-17T10:53:12.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.573+0000 7f6cce1c9640 1 -- start start 2024-04-17T10:53:12.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.573+0000 7f6cce1c9640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6cc80ce750 0x7f6cc80d2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:12.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.573+0000 7f6cce1c9640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc80b0a70 0x7f6cc8161a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:12.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.573+0000 7f6cce1c9640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6cc8161f80 0x7f6cc80cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:12.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.573+0000 7f6cce1c9640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cc806f290 con 0x7f6cc80ce750 2024-04-17T10:53:12.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.573+0000 7f6cce1c9640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cc806e980 con 0x7f6cc8161f80 2024-04-17T10:53:12.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.573+0000 7f6cce1c9640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc8162810 con 0x7f6cc80b0a70 2024-04-17T10:53:12.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.573+0000 7f6cc77fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6cc80ce750 0x7f6cc80d2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:12.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.573+0000 7f6cc77fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6cc80ce750 0x7f6cc80d2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34902/0 (socket says 172.21.15.83:34902) 2024-04-17T10:53:12.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.573+0000 7f6cc77fe640 1 -- 172.21.15.83:0/2095446114 learned_addr learned my addr 172.21.15.83:0/2095446114 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:12.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.573+0000 7f6cbeffd640 1 --2- 172.21.15.83:0/2095446114 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc80b0a70 0x7f6cc8161a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:12.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.573+0000 7f6cc7fff640 1 --2- 172.21.15.83:0/2095446114 >> v2:172.21.15.167:3300/0 conn(0x7f6cc8161f80 0x7f6cc80cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:12.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.573+0000 7f6cc77fe640 1 -- 172.21.15.83:0/2095446114 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc80b0a70 msgr2=0x7f6cc8161a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:12.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.573+0000 7f6cc77fe640 1 --2- 172.21.15.83:0/2095446114 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc80b0a70 0x7f6cc8161a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:12.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.573+0000 7f6cc77fe640 1 -- 172.21.15.83:0/2095446114 >> v2:172.21.15.167:3300/0 conn(0x7f6cc8161f80 msgr2=0x7f6cc80cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:12.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.573+0000 7f6cc77fe640 1 --2- 172.21.15.83:0/2095446114 >> v2:172.21.15.167:3300/0 conn(0x7f6cc8161f80 0x7f6cc80cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:12.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.573+0000 7f6cc77fe640 1 -- 172.21.15.83:0/2095446114 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc806ec20 con 0x7f6cc80ce750 2024-04-17T10:53:12.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.573+0000 7f6cbeffd640 1 --2- 172.21.15.83:0/2095446114 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc80b0a70 0x7f6cc8161a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:53:12.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.573+0000 7f6cc7fff640 1 --2- 172.21.15.83:0/2095446114 >> v2:172.21.15.167:3300/0 conn(0x7f6cc8161f80 0x7f6cc80cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:53:12.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.577+0000 7f6cc77fe640 1 --2- 172.21.15.83:0/2095446114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc80ce750 0x7f6cc80d2290 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8071fc0 tx=0x7f6cb801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8ec8661bd5d4e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:12.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.577+0000 7f6cc6ffd640 1 -- 172.21.15.83:0/2095446114 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb80ab5f0 con 0x7f6cc80ce750 2024-04-17T10:53:12.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.577+0000 7f6cc6ffd640 1 -- 172.21.15.83:0/2095446114 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb80b0070 con 0x7f6cc80ce750 2024-04-17T10:53:12.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.577+0000 7f6cc6ffd640 1 -- 172.21.15.83:0/2095446114 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb80ab9b0 con 0x7f6cc80ce750 2024-04-17T10:53:12.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.577+0000 7f6cce1c9640 1 -- 172.21.15.83:0/2095446114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc80ce750 msgr2=0x7f6cc80d2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:12.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.577+0000 7f6cce1c9640 1 --2- 172.21.15.83:0/2095446114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc80ce750 0x7f6cc80d2290 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8071fc0 tx=0x7f6cb801c520 comp rx=0 tx=0).stop 2024-04-17T10:53:12.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.577+0000 7f6cce1c9640 1 -- 172.21.15.83:0/2095446114 shutdown_connections 2024-04-17T10:53:12.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.577+0000 7f6cce1c9640 1 --2- 172.21.15.83:0/2095446114 >> v2:172.21.15.167:3300/0 conn(0x7f6cc8161f80 0x7f6cc80cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:12.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.577+0000 7f6cce1c9640 1 --2- 172.21.15.83:0/2095446114 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc80b0a70 0x7f6cc8161a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:12.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.577+0000 7f6cce1c9640 1 --2- 172.21.15.83:0/2095446114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc80ce750 0x7f6cc80d2290 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:12.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.577+0000 7f6cce1c9640 1 -- 172.21.15.83:0/2095446114 >> 172.21.15.83:0/2095446114 conn(0x7f6cc80d1770 msgr2=0x7f6cc80d1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:12.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.577+0000 7f6cce1c9640 1 -- 172.21.15.83:0/2095446114 shutdown_connections 2024-04-17T10:53:12.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.577+0000 7f6cce1c9640 1 -- 172.21.15.83:0/2095446114 wait complete. 2024-04-17T10:53:12.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.577+0000 7f6cce1c9640 1 Processor -- start 2024-04-17T10:53:12.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.577+0000 7f6cce1c9640 1 -- start start 2024-04-17T10:53:12.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.577+0000 7f6cce1c9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cc80b0a70 0x7f6cc8152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:12.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.577+0000 7f6cce1c9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc80ce750 0x7f6cc815c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:12.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.577+0000 7f6cce1c9640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc8161f80 0x7f6cc815c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:12.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.577+0000 7f6cce1c9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc806c700 con 0x7f6cc80ce750 2024-04-17T10:53:12.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.577+0000 7f6cce1c9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc81625a0 con 0x7f6cc80b0a70 2024-04-17T10:53:12.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.577+0000 7f6cce1c9640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc806c880 con 0x7f6cc8161f80 2024-04-17T10:53:12.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.577+0000 7f6cbeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc80ce750 0x7f6cc815c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:12.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.577+0000 7f6cbeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc80ce750 0x7f6cc815c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34918/0 (socket says 172.21.15.83:34918) 2024-04-17T10:53:12.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.577+0000 7f6cbeffd640 1 -- 172.21.15.83:0/3961000011 learned_addr learned my addr 172.21.15.83:0/3961000011 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:12.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.581+0000 7f6cc77fe640 1 --2- 172.21.15.83:0/3961000011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cc80b0a70 0x7f6cc8152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:12.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.581+0000 7f6cc7fff640 1 --2- 172.21.15.83:0/3961000011 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc8161f80 0x7f6cc815c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:12.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.581+0000 7f6cbeffd640 1 -- 172.21.15.83:0/3961000011 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc8161f80 msgr2=0x7f6cc815c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:12.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.581+0000 7f6cbeffd640 1 --2- 172.21.15.83:0/3961000011 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc8161f80 0x7f6cc815c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:12.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.581+0000 7f6cbeffd640 1 -- 172.21.15.83:0/3961000011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cc80b0a70 msgr2=0x7f6cc8152780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:12.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.581+0000 7f6cbeffd640 1 --2- 172.21.15.83:0/3961000011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cc80b0a70 0x7f6cc8152780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:12.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.581+0000 7f6cbeffd640 1 -- 172.21.15.83:0/3961000011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc80c7350 con 0x7f6cc80ce750 2024-04-17T10:53:12.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.581+0000 7f6cbeffd640 1 --2- 172.21.15.83:0/3961000011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc80ce750 0x7f6cc815c100 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000c7f0 tx=0x7f6cb000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:12.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.581+0000 7f6cc4ff9640 1 -- 172.21.15.83:0/3961000011 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0015070 con 0x7f6cc80ce750 2024-04-17T10:53:12.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.581+0000 7f6cc4ff9640 1 -- 172.21.15.83:0/3961000011 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb0007a30 con 0x7f6cc80ce750 2024-04-17T10:53:12.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.581+0000 7f6cce1c9640 1 -- 172.21.15.83:0/3961000011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cc80c7630 con 0x7f6cc80ce750 2024-04-17T10:53:12.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.581+0000 7f6cc4ff9640 1 -- 172.21.15.83:0/3961000011 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0007e00 con 0x7f6cc80ce750 2024-04-17T10:53:12.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.581+0000 7f6cce1c9640 1 -- 172.21.15.83:0/3961000011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc8152cc0 con 0x7f6cc80ce750 2024-04-17T10:53:12.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.581+0000 7f6cce1c9640 1 -- 172.21.15.83:0/3961000011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cc8074f90 con 0x7f6cc80ce750 2024-04-17T10:53:12.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.581+0000 7f6cc4ff9640 1 -- 172.21.15.83:0/3961000011 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6cb0018070 con 0x7f6cc80ce750 2024-04-17T10:53:12.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.581+0000 7f6cc4ff9640 1 --2- 172.21.15.83:0/3961000011 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6ca8066f60 0x7f6ca8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:12.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.581+0000 7f6cc4ff9640 1 -- 172.21.15.83:0/3961000011 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f6cb0084f50 con 0x7f6cc80ce750 2024-04-17T10:53:12.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.581+0000 7f6cc77fe640 1 --2- 172.21.15.83:0/3961000011 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6ca8066f60 0x7f6ca8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:12.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.581+0000 7f6cc77fe640 1 --2- 172.21.15.83:0/3961000011 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6ca8066f60 0x7f6ca8069420 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80af000 tx=0x7f6cb80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:12.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.585+0000 7f6cc4ff9640 1 -- 172.21.15.83:0/3961000011 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6cb0055f80 con 0x7f6cc80ce750 2024-04-17T10:53:12.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.773+0000 7f6cce1c9640 1 -- 172.21.15.83:0/3961000011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6cc80b76f0 con 0x7f6cc80ce750 2024-04-17T10:53:12.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.773+0000 7f6cc4ff9640 1 -- 172.21.15.83:0/3961000011 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f6cb0055880 con 0x7f6cc80ce750 2024-04-17T10:53:12.768 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:12.768 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:12.768 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:12.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.773+0000 7f6cce1c9640 1 -- 172.21.15.83:0/3961000011 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6ca8066f60 msgr2=0x7f6ca8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:12.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.773+0000 7f6cce1c9640 1 --2- 172.21.15.83:0/3961000011 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6ca8066f60 0x7f6ca8069420 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80af000 tx=0x7f6cb80a7000 comp rx=0 tx=0).stop 2024-04-17T10:53:12.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.773+0000 7f6cce1c9640 1 -- 172.21.15.83:0/3961000011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc80ce750 msgr2=0x7f6cc815c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:12.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.773+0000 7f6cce1c9640 1 --2- 172.21.15.83:0/3961000011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc80ce750 0x7f6cc815c100 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000c7f0 tx=0x7f6cb000ccb0 comp rx=0 tx=0).stop 2024-04-17T10:53:12.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.773+0000 7f6cce1c9640 1 -- 172.21.15.83:0/3961000011 shutdown_connections 2024-04-17T10:53:12.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.773+0000 7f6cce1c9640 1 --2- 172.21.15.83:0/3961000011 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6ca8066f60 0x7f6ca8069420 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:12.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.773+0000 7f6cce1c9640 1 --2- 172.21.15.83:0/3961000011 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cc8161f80 0x7f6cc815c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:12.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.773+0000 7f6cce1c9640 1 --2- 172.21.15.83:0/3961000011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc80ce750 0x7f6cc815c100 unknown :-1 s=CLOSED pgs=2287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:12.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.773+0000 7f6cce1c9640 1 --2- 172.21.15.83:0/3961000011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cc80b0a70 0x7f6cc8152780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:12.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.773+0000 7f6cce1c9640 1 -- 172.21.15.83:0/3961000011 >> 172.21.15.83:0/3961000011 conn(0x7f6cc80d1770 msgr2=0x7f6cc80b2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:12.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.777+0000 7f6cce1c9640 1 -- 172.21.15.83:0/3961000011 shutdown_connections 2024-04-17T10:53:12.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:12.777+0000 7f6cce1c9640 1 -- 172.21.15.83:0/3961000011 wait complete. 2024-04-17T10:53:13.189 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:13.189 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:14.190 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:14.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.325+0000 7fb46352e640 1 Processor -- start 2024-04-17T10:53:14.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.325+0000 7fb46352e640 1 -- start start 2024-04-17T10:53:14.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.325+0000 7fb46352e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb45c15ba30 0x7fb45c072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:14.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.325+0000 7fb46352e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb45c0cfbe0 0x7fb45c06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:14.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.325+0000 7fb46352e640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb45c06e320 con 0x7fb45c15c960 2024-04-17T10:53:14.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.325+0000 7fb46352e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb45c071ea0 con 0x7fb45c15ba30 2024-04-17T10:53:14.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.325+0000 7fb46352e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c06e040 con 0x7fb45c0cfbe0 2024-04-17T10:53:14.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.325+0000 7fb4612ca640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb45c15ba30 0x7fb45c072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:14.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.325+0000 7fb4612ca640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb45c15ba30 0x7fb45c072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34934/0 (socket says 172.21.15.83:34934) 2024-04-17T10:53:14.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.325+0000 7fb4612ca640 1 -- 172.21.15.83:0/4024537970 learned_addr learned my addr 172.21.15.83:0/4024537970 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:14.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.325+0000 7fb461acb640 1 --2- 172.21.15.83:0/4024537970 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb45c0cfbe0 0x7fb45c06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:14.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.325+0000 7fb4612ca640 1 -- 172.21.15.83:0/4024537970 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb45c0cfbe0 msgr2=0x7fb45c06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:14.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.325+0000 7fb4612ca640 1 --2- 172.21.15.83:0/4024537970 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb45c0cfbe0 0x7fb45c06c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:14.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.325+0000 7fb4612ca640 1 -- 172.21.15.83:0/4024537970 >> v1:172.21.15.167:6789/0 conn(0x7fb45c15c960 legacy=0x7fb45c0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:14.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.325+0000 7fb4612ca640 1 -- 172.21.15.83:0/4024537970 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45c071850 con 0x7fb45c15ba30 2024-04-17T10:53:14.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb4612ca640 1 --2- 172.21.15.83:0/4024537970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb45c15ba30 0x7fb45c072330 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7fb458009870 tx=0x7fb45801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb3c90069d7f91 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:14.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb453fff640 1 -- 172.21.15.83:0/4024537970 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4580b0070 con 0x7fb45c15ba30 2024-04-17T10:53:14.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb453fff640 1 -- 172.21.15.83:0/4024537970 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4580ab600 con 0x7fb45c15ba30 2024-04-17T10:53:14.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb453fff640 1 -- 172.21.15.83:0/4024537970 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4580ab9d0 con 0x7fb45c15ba30 2024-04-17T10:53:14.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb46352e640 1 -- 172.21.15.83:0/4024537970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb45c15ba30 msgr2=0x7fb45c072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:14.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb46352e640 1 --2- 172.21.15.83:0/4024537970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb45c15ba30 0x7fb45c072330 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7fb458009870 tx=0x7fb45801c490 comp rx=0 tx=0).stop 2024-04-17T10:53:14.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb46352e640 1 -- 172.21.15.83:0/4024537970 shutdown_connections 2024-04-17T10:53:14.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb46352e640 1 --2- 172.21.15.83:0/4024537970 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb45c0cfbe0 0x7fb45c06c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:14.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb46352e640 1 --2- 172.21.15.83:0/4024537970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb45c15ba30 0x7fb45c072330 unknown :-1 s=CLOSED pgs=2288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:14.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb46352e640 1 -- 172.21.15.83:0/4024537970 >> 172.21.15.83:0/4024537970 conn(0x7fb45c06abe0 msgr2=0x7fb45c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:14.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb46352e640 1 -- 172.21.15.83:0/4024537970 shutdown_connections 2024-04-17T10:53:14.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb46352e640 1 -- 172.21.15.83:0/4024537970 wait complete. 2024-04-17T10:53:14.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb46352e640 1 Processor -- start 2024-04-17T10:53:14.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb46352e640 1 -- start start 2024-04-17T10:53:14.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb46352e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb45c0cfbe0 0x7fb45c15a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:14.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb46352e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb45c15ba30 0x7fb45c13f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:14.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb46352e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb45c15c960 0x7fb45c1553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:14.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb46352e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c06de20 con 0x7fb45c0cfbe0 2024-04-17T10:53:14.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb46352e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c06dca0 con 0x7fb45c15c960 2024-04-17T10:53:14.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb46352e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c168520 con 0x7fb45c15ba30 2024-04-17T10:53:14.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb4612ca640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb45c0cfbe0 0x7fb45c15a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:14.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb4612ca640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb45c0cfbe0 0x7fb45c15a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34944/0 (socket says 172.21.15.83:34944) 2024-04-17T10:53:14.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb4612ca640 1 -- 172.21.15.83:0/2617262272 learned_addr learned my addr 172.21.15.83:0/2617262272 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:14.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb460ac9640 1 --2- 172.21.15.83:0/2617262272 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb45c15ba30 0x7fb45c13f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:14.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb4612ca640 1 -- 172.21.15.83:0/2617262272 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb45c15ba30 msgr2=0x7fb45c13f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:14.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb4612ca640 1 --2- 172.21.15.83:0/2617262272 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb45c15ba30 0x7fb45c13f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:14.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb4612ca640 1 -- 172.21.15.83:0/2617262272 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb45c15c960 msgr2=0x7fb45c1553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:14.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb4612ca640 1 --2- 172.21.15.83:0/2617262272 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb45c15c960 0x7fb45c1553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:14.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb4612ca640 1 -- 172.21.15.83:0/2617262272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45c150140 con 0x7fb45c0cfbe0 2024-04-17T10:53:14.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb460ac9640 1 --2- 172.21.15.83:0/2617262272 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb45c15ba30 0x7fb45c13f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:53:14.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb4612ca640 1 --2- 172.21.15.83:0/2617262272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb45c0cfbe0 0x7fb45c15a320 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7fb4580ae040 tx=0x7fb45801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:14.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb451ffb640 1 -- 172.21.15.83:0/2617262272 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4580b0040 con 0x7fb45c0cfbe0 2024-04-17T10:53:14.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb451ffb640 1 -- 172.21.15.83:0/2617262272 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4580bc070 con 0x7fb45c0cfbe0 2024-04-17T10:53:14.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb46352e640 1 -- 172.21.15.83:0/2617262272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb45c150420 con 0x7fb45c0cfbe0 2024-04-17T10:53:14.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb451ffb640 1 -- 172.21.15.83:0/2617262272 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4580b6880 con 0x7fb45c0cfbe0 2024-04-17T10:53:14.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.329+0000 7fb46352e640 1 -- 172.21.15.83:0/2617262272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb45c1509e0 con 0x7fb45c0cfbe0 2024-04-17T10:53:14.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.333+0000 7fb46352e640 1 -- 172.21.15.83:0/2617262272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb45c074f90 con 0x7fb45c0cfbe0 2024-04-17T10:53:14.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.333+0000 7fb451ffb640 1 -- 172.21.15.83:0/2617262272 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb4580b6a20 con 0x7fb45c0cfbe0 2024-04-17T10:53:14.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.333+0000 7fb451ffb640 1 --2- 172.21.15.83:0/2617262272 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb440066f60 0x7fb440069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:14.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.333+0000 7fb451ffb640 1 -- 172.21.15.83:0/2617262272 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fb45811f590 con 0x7fb45c0cfbe0 2024-04-17T10:53:14.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.333+0000 7fb460ac9640 1 --2- 172.21.15.83:0/2617262272 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb440066f60 0x7fb440069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:14.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.333+0000 7fb460ac9640 1 --2- 172.21.15.83:0/2617262272 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb440066f60 0x7fb440069420 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fb45c14f610 tx=0x7fb44c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:14.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.337+0000 7fb451ffb640 1 -- 172.21.15.83:0/2617262272 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb458016110 con 0x7fb45c0cfbe0 2024-04-17T10:53:14.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.529+0000 7fb46352e640 1 -- 172.21.15.83:0/2617262272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb45c069060 con 0x7fb45c0cfbe0 2024-04-17T10:53:14.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.533+0000 7fb451ffb640 1 -- 172.21.15.83:0/2617262272 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fb4580f05f0 con 0x7fb45c0cfbe0 2024-04-17T10:53:14.527 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:14.527 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:14.528 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:14.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.533+0000 7fb46352e640 1 -- 172.21.15.83:0/2617262272 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb440066f60 msgr2=0x7fb440069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:14.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.533+0000 7fb46352e640 1 --2- 172.21.15.83:0/2617262272 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb440066f60 0x7fb440069420 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fb45c14f610 tx=0x7fb44c009040 comp rx=0 tx=0).stop 2024-04-17T10:53:14.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.533+0000 7fb46352e640 1 -- 172.21.15.83:0/2617262272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb45c0cfbe0 msgr2=0x7fb45c15a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:14.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.533+0000 7fb46352e640 1 --2- 172.21.15.83:0/2617262272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb45c0cfbe0 0x7fb45c15a320 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7fb4580ae040 tx=0x7fb45801acd0 comp rx=0 tx=0).stop 2024-04-17T10:53:14.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.533+0000 7fb46352e640 1 -- 172.21.15.83:0/2617262272 shutdown_connections 2024-04-17T10:53:14.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.533+0000 7fb46352e640 1 --2- 172.21.15.83:0/2617262272 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb440066f60 0x7fb440069420 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:14.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.533+0000 7fb46352e640 1 --2- 172.21.15.83:0/2617262272 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb45c15c960 0x7fb45c1553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:14.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.533+0000 7fb46352e640 1 --2- 172.21.15.83:0/2617262272 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb45c15ba30 0x7fb45c13f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:14.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.533+0000 7fb46352e640 1 --2- 172.21.15.83:0/2617262272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb45c0cfbe0 0x7fb45c15a320 unknown :-1 s=CLOSED pgs=2289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:14.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.533+0000 7fb46352e640 1 -- 172.21.15.83:0/2617262272 >> 172.21.15.83:0/2617262272 conn(0x7fb45c06abe0 msgr2=0x7fb45c15e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:14.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.533+0000 7fb46352e640 1 -- 172.21.15.83:0/2617262272 shutdown_connections 2024-04-17T10:53:14.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:14.533+0000 7fb46352e640 1 -- 172.21.15.83:0/2617262272 wait complete. 2024-04-17T10:53:14.948 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:14.948 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:15.950 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:16.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.081+0000 7f752d558640 1 Processor -- start 2024-04-17T10:53:16.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.081+0000 7f752d558640 1 -- start start 2024-04-17T10:53:16.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.085+0000 7f752d558640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f752815ba30 0x7f7528072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:16.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.085+0000 7f752d558640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f752806e320 con 0x7f75280cfbe0 2024-04-17T10:53:16.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.085+0000 7f752d558640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f752806e040 con 0x7f752815c960 2024-04-17T10:53:16.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.085+0000 7f752d558640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7528071ea0 con 0x7f752815ba30 2024-04-17T10:53:16.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.085+0000 7f7526ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f752815ba30 0x7f7528072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:16.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.085+0000 7f7526ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f752815ba30 0x7f7528072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34960/0 (socket says 172.21.15.83:34960) 2024-04-17T10:53:16.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.085+0000 7f7526ffd640 1 -- 172.21.15.83:0/3073905615 learned_addr learned my addr 172.21.15.83:0/3073905615 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:16.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.085+0000 7f7526ffd640 1 -- 172.21.15.83:0/3073905615 >> v1:172.21.15.167:6789/0 conn(0x7f752815c960 legacy=0x7f75280cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:16.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.085+0000 7f7526ffd640 1 -- 172.21.15.83:0/3073905615 >> v1:172.21.15.83:6789/0 conn(0x7f75280cfbe0 legacy=0x7f752806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:16.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.085+0000 7f7526ffd640 1 -- 172.21.15.83:0/3073905615 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7528071850 con 0x7f752815ba30 2024-04-17T10:53:16.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.085+0000 7f75267fc640 1 -- 172.21.15.83:0/3073905615 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1466463252 0 0) 0x7f752806e320 con 0x7f75280cfbe0 2024-04-17T10:53:16.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.085+0000 7f7526ffd640 1 --2- 172.21.15.83:0/3073905615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f752815ba30 0x7f7528072330 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f7518009870 tx=0x7f751801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c4ef2f65553bec server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:16.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.085+0000 7f75267fc640 1 -- 172.21.15.83:0/3073905615 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75180b1070 con 0x7f752815ba30 2024-04-17T10:53:16.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.085+0000 7f75267fc640 1 -- 172.21.15.83:0/3073905615 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75180ac640 con 0x7f752815ba30 2024-04-17T10:53:16.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.085+0000 7f75267fc640 1 -- 172.21.15.83:0/3073905615 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75180aca10 con 0x7f752815ba30 2024-04-17T10:53:16.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.089+0000 7f752d558640 1 -- 172.21.15.83:0/3073905615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f752815ba30 msgr2=0x7f7528072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:16.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.089+0000 7f752d558640 1 --2- 172.21.15.83:0/3073905615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f752815ba30 0x7f7528072330 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f7518009870 tx=0x7f751801c490 comp rx=0 tx=0).stop 2024-04-17T10:53:16.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.089+0000 7f752d558640 1 -- 172.21.15.83:0/3073905615 shutdown_connections 2024-04-17T10:53:16.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.089+0000 7f752d558640 1 --2- 172.21.15.83:0/3073905615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f752815ba30 0x7f7528072330 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:16.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.089+0000 7f752d558640 1 -- 172.21.15.83:0/3073905615 >> 172.21.15.83:0/3073905615 conn(0x7f752806abe0 msgr2=0x7f752806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:16.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.089+0000 7f752d558640 1 -- 172.21.15.83:0/3073905615 shutdown_connections 2024-04-17T10:53:16.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.089+0000 7f752d558640 1 -- 172.21.15.83:0/3073905615 wait complete. 2024-04-17T10:53:16.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.089+0000 7f752d558640 1 Processor -- start 2024-04-17T10:53:16.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.089+0000 7f752d558640 1 -- start start 2024-04-17T10:53:16.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.089+0000 7f752d558640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75280cfbe0 0x7f752813e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:16.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.089+0000 7f752d558640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f752815ba30 0x7f752815e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:16.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.089+0000 7f752d558640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f752815c960 0x7f752815e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:16.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.089+0000 7f752d558640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752806de20 con 0x7f75280cfbe0 2024-04-17T10:53:16.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.089+0000 7f752d558640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752806dca0 con 0x7f752815c960 2024-04-17T10:53:16.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.089+0000 7f752d558640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7528169530 con 0x7f752815ba30 2024-04-17T10:53:16.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.089+0000 7f7526ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75280cfbe0 0x7f752813e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:16.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.089+0000 7f7526ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75280cfbe0 0x7f752813e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34974/0 (socket says 172.21.15.83:34974) 2024-04-17T10:53:16.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.089+0000 7f7526ffd640 1 -- 172.21.15.83:0/612283956 learned_addr learned my addr 172.21.15.83:0/612283956 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:16.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.089+0000 7f7526ffd640 1 -- 172.21.15.83:0/612283956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f752815ba30 msgr2=0x7f752815e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:16.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.089+0000 7f7526ffd640 1 --2- 172.21.15.83:0/612283956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f752815ba30 0x7f752815e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:16.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.089+0000 7f7526ffd640 1 -- 172.21.15.83:0/612283956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f752815c960 msgr2=0x7f752815e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:16.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.089+0000 7f75277fe640 1 --2- 172.21.15.83:0/612283956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f752815c960 0x7f752815e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:16.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.089+0000 7f7526ffd640 1 --2- 172.21.15.83:0/612283956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f752815c960 0x7f752815e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:16.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.089+0000 7f7526ffd640 1 -- 172.21.15.83:0/612283956 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7528152680 con 0x7f75280cfbe0 2024-04-17T10:53:16.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.089+0000 7f7526ffd640 1 --2- 172.21.15.83:0/612283956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75280cfbe0 0x7f752813e0e0 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f75180af040 tx=0x7f7518019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:16.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.089+0000 7f751ffff640 1 -- 172.21.15.83:0/612283956 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75180bb020 con 0x7f75280cfbe0 2024-04-17T10:53:16.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.089+0000 7f751ffff640 1 -- 172.21.15.83:0/612283956 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75180b7640 con 0x7f75280cfbe0 2024-04-17T10:53:16.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.089+0000 7f752d558640 1 -- 172.21.15.83:0/612283956 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7528152960 con 0x7f75280cfbe0 2024-04-17T10:53:16.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.089+0000 7f751ffff640 1 -- 172.21.15.83:0/612283956 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75180b1040 con 0x7f75280cfbe0 2024-04-17T10:53:16.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.089+0000 7f752d558640 1 -- 172.21.15.83:0/612283956 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7528152f20 con 0x7f75280cfbe0 2024-04-17T10:53:16.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.093+0000 7f752d558640 1 -- 172.21.15.83:0/612283956 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7528074f90 con 0x7f75280cfbe0 2024-04-17T10:53:16.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.093+0000 7f751ffff640 1 -- 172.21.15.83:0/612283956 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f75180be020 con 0x7f75280cfbe0 2024-04-17T10:53:16.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.093+0000 7f751ffff640 1 --2- 172.21.15.83:0/612283956 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7508066f90 0x7f7508069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:16.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.093+0000 7f751ffff640 1 -- 172.21.15.83:0/612283956 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f751811f450 con 0x7f75280cfbe0 2024-04-17T10:53:16.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.093+0000 7f751e7fc640 1 --2- 172.21.15.83:0/612283956 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7508066f90 0x7f7508069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:16.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.093+0000 7f751e7fc640 1 --2- 172.21.15.83:0/612283956 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7508066f90 0x7f7508069450 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f75280cfbb0 tx=0x7f7510006bd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:16.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.097+0000 7f751ffff640 1 -- 172.21.15.83:0/612283956 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f75180c5030 con 0x7f75280cfbe0 2024-04-17T10:53:16.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.293+0000 7f752d558640 1 -- 172.21.15.83:0/612283956 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75280b10e0 con 0x7f75280cfbe0 2024-04-17T10:53:16.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.293+0000 7f751ffff640 1 -- 172.21.15.83:0/612283956 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f7518016110 con 0x7f75280cfbe0 2024-04-17T10:53:16.288 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:16.288 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:16.288 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.293+0000 7f752d558640 1 -- 172.21.15.83:0/612283956 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7508066f90 msgr2=0x7f7508069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.293+0000 7f752d558640 1 --2- 172.21.15.83:0/612283956 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7508066f90 0x7f7508069450 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f75280cfbb0 tx=0x7f7510006bd0 comp rx=0 tx=0).stop 2024-04-17T10:53:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.293+0000 7f752d558640 1 -- 172.21.15.83:0/612283956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75280cfbe0 msgr2=0x7f752813e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.293+0000 7f752d558640 1 --2- 172.21.15.83:0/612283956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75280cfbe0 0x7f752813e0e0 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f75180af040 tx=0x7f7518019230 comp rx=0 tx=0).stop 2024-04-17T10:53:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.293+0000 7f752d558640 1 -- 172.21.15.83:0/612283956 shutdown_connections 2024-04-17T10:53:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.293+0000 7f752d558640 1 --2- 172.21.15.83:0/612283956 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7508066f90 0x7f7508069450 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.293+0000 7f752d558640 1 --2- 172.21.15.83:0/612283956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f752815c960 0x7f752815e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.293+0000 7f752d558640 1 --2- 172.21.15.83:0/612283956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f752815ba30 0x7f752815e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.293+0000 7f752d558640 1 --2- 172.21.15.83:0/612283956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75280cfbe0 0x7f752813e0e0 unknown :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.293+0000 7f752d558640 1 -- 172.21.15.83:0/612283956 >> 172.21.15.83:0/612283956 conn(0x7f752806abe0 msgr2=0x7f752815a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.293+0000 7f752d558640 1 -- 172.21.15.83:0/612283956 shutdown_connections 2024-04-17T10:53:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:16.293+0000 7f752d558640 1 -- 172.21.15.83:0/612283956 wait complete. 2024-04-17T10:53:16.709 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:16.709 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:17.710 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:17.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.845+0000 7fe12aee7640 1 Processor -- start 2024-04-17T10:53:17.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.845+0000 7fe12aee7640 1 -- start start 2024-04-17T10:53:17.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.845+0000 7fe12aee7640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe12415dfa0 0x7fe1240b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:17.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.845+0000 7fe12aee7640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe1240b5b40 0x7fe1240d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:17.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.845+0000 7fe12aee7640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe124162a00 con 0x7fe1240ce980 2024-04-17T10:53:17.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.845+0000 7fe12aee7640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe124161e80 con 0x7fe12415dfa0 2024-04-17T10:53:17.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.845+0000 7fe12aee7640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe124162160 con 0x7fe1240b5b40 2024-04-17T10:53:17.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.845+0000 7fe128c83640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe12415dfa0 0x7fe1240b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:17.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.845+0000 7fe123fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe1240ce980 0x7fe1240b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:37094/0 (socket says 172.21.15.83:37094) 2024-04-17T10:53:17.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.845+0000 7fe123fff640 1 -- 172.21.15.83:0/303156874 learned_addr learned my addr 172.21.15.83:0/303156874 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:53:17.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.845+0000 7fe129484640 1 --2- 172.21.15.83:0/303156874 >> v2:172.21.15.167:3300/0 conn(0x7fe1240b5b40 0x7fe1240d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:17.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.845+0000 7fe128c83640 1 -- 172.21.15.83:0/303156874 >> v2:172.21.15.167:3300/0 conn(0x7fe1240b5b40 msgr2=0x7fe1240d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:17.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.845+0000 7fe128c83640 1 --2- 172.21.15.83:0/303156874 >> v2:172.21.15.167:3300/0 conn(0x7fe1240b5b40 0x7fe1240d1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:17.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.845+0000 7fe128c83640 1 -- 172.21.15.83:0/303156874 >> v1:172.21.15.83:6789/0 conn(0x7fe1240ce980 legacy=0x7fe1240b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:17.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.845+0000 7fe128c83640 1 -- 172.21.15.83:0/303156874 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe124162440 con 0x7fe12415dfa0 2024-04-17T10:53:17.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.845+0000 7fe1237fe640 1 -- 172.21.15.83:0/303156874 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 123085601 0 0) 0x7fe124162a00 con 0x7fe1240ce980 2024-04-17T10:53:17.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.849+0000 7fe128c83640 1 --2- 172.21.15.83:0/303156874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe12415dfa0 0x7fe1240b6c20 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7fe1240ca980 tx=0x7fe11401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01326dfb2705ce7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:17.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.849+0000 7fe1237fe640 1 -- 172.21.15.83:0/303156874 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1140b0070 con 0x7fe12415dfa0 2024-04-17T10:53:17.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.849+0000 7fe1237fe640 1 -- 172.21.15.83:0/303156874 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1140ab600 con 0x7fe12415dfa0 2024-04-17T10:53:17.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.849+0000 7fe1237fe640 1 -- 172.21.15.83:0/303156874 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1140ab9d0 con 0x7fe12415dfa0 2024-04-17T10:53:17.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.849+0000 7fe12aee7640 1 -- 172.21.15.83:0/303156874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe12415dfa0 msgr2=0x7fe1240b6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:17.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.849+0000 7fe12aee7640 1 --2- 172.21.15.83:0/303156874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe12415dfa0 0x7fe1240b6c20 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7fe1240ca980 tx=0x7fe11401c3c0 comp rx=0 tx=0).stop 2024-04-17T10:53:17.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.849+0000 7fe12aee7640 1 -- 172.21.15.83:0/303156874 shutdown_connections 2024-04-17T10:53:17.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.849+0000 7fe12aee7640 1 --2- 172.21.15.83:0/303156874 >> v2:172.21.15.167:3300/0 conn(0x7fe1240b5b40 0x7fe1240d1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:17.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.849+0000 7fe12aee7640 1 --2- 172.21.15.83:0/303156874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe12415dfa0 0x7fe1240b6c20 unknown :-1 s=CLOSED pgs=2294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:17.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.849+0000 7fe12aee7640 1 -- 172.21.15.83:0/303156874 >> 172.21.15.83:0/303156874 conn(0x7fe1240d0600 msgr2=0x7fe1240d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:17.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.849+0000 7fe12aee7640 1 -- 172.21.15.83:0/303156874 shutdown_connections 2024-04-17T10:53:17.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.849+0000 7fe12aee7640 1 -- 172.21.15.83:0/303156874 wait complete. 2024-04-17T10:53:17.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.849+0000 7fe12aee7640 1 Processor -- start 2024-04-17T10:53:17.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.849+0000 7fe12aee7640 1 -- start start 2024-04-17T10:53:17.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.849+0000 7fe12aee7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1240b5b40 0x7fe124156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:17.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.849+0000 7fe12aee7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1240ce980 0x7fe124153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:17.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.849+0000 7fe12aee7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe12415dfa0 0x7fe1240cb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:17.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.849+0000 7fe12aee7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12416ae50 con 0x7fe1240ce980 2024-04-17T10:53:17.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.849+0000 7fe12aee7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe124161b10 con 0x7fe12415dfa0 2024-04-17T10:53:17.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.849+0000 7fe12aee7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe12416afd0 con 0x7fe1240b5b40 2024-04-17T10:53:17.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.853+0000 7fe123fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1240ce980 0x7fe124153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:17.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.853+0000 7fe123fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1240ce980 0x7fe124153fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41324/0 (socket says 172.21.15.83:41324) 2024-04-17T10:53:17.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.853+0000 7fe123fff640 1 -- 172.21.15.83:0/4076688202 learned_addr learned my addr 172.21.15.83:0/4076688202 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:17.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.853+0000 7fe129484640 1 --2- 172.21.15.83:0/4076688202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe12415dfa0 0x7fe1240cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:17.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.853+0000 7fe123fff640 1 -- 172.21.15.83:0/4076688202 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1240b5b40 msgr2=0x7fe124156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:17.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.853+0000 7fe123fff640 1 --2- 172.21.15.83:0/4076688202 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1240b5b40 0x7fe124156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:17.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.853+0000 7fe123fff640 1 -- 172.21.15.83:0/4076688202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe12415dfa0 msgr2=0x7fe1240cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:17.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.853+0000 7fe123fff640 1 --2- 172.21.15.83:0/4076688202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe12415dfa0 0x7fe1240cb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:17.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.853+0000 7fe123fff640 1 -- 172.21.15.83:0/4076688202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12415b390 con 0x7fe1240ce980 2024-04-17T10:53:17.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.853+0000 7fe129484640 1 --2- 172.21.15.83:0/4076688202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe12415dfa0 0x7fe1240cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:53:17.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.853+0000 7fe123fff640 1 --2- 172.21.15.83:0/4076688202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1240ce980 0x7fe124153fc0 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7fe118007fd0 tx=0x7fe11800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:17.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.853+0000 7fe1217fa640 1 -- 172.21.15.83:0/4076688202 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe118017070 con 0x7fe1240ce980 2024-04-17T10:53:17.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.853+0000 7fe1217fa640 1 -- 172.21.15.83:0/4076688202 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe118005d10 con 0x7fe1240ce980 2024-04-17T10:53:17.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.853+0000 7fe1217fa640 1 -- 172.21.15.83:0/4076688202 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe118007310 con 0x7fe1240ce980 2024-04-17T10:53:17.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.853+0000 7fe12aee7640 1 -- 172.21.15.83:0/4076688202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe12415b670 con 0x7fe1240ce980 2024-04-17T10:53:17.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.853+0000 7fe12aee7640 1 -- 172.21.15.83:0/4076688202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe12415bc30 con 0x7fe1240ce980 2024-04-17T10:53:17.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.853+0000 7fe12aee7640 1 -- 172.21.15.83:0/4076688202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe124074f90 con 0x7fe1240ce980 2024-04-17T10:53:17.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.853+0000 7fe1217fa640 1 -- 172.21.15.83:0/4076688202 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe11801a070 con 0x7fe1240ce980 2024-04-17T10:53:17.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.853+0000 7fe1217fa640 1 --2- 172.21.15.83:0/4076688202 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe110066f60 0x7fe110069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:17.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.853+0000 7fe1217fa640 1 -- 172.21.15.83:0/4076688202 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fe1180844c0 con 0x7fe1240ce980 2024-04-17T10:53:17.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.853+0000 7fe128c83640 1 --2- 172.21.15.83:0/4076688202 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe110066f60 0x7fe110069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:17.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.853+0000 7fe128c83640 1 --2- 172.21.15.83:0/4076688202 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe110066f60 0x7fe110069420 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe1140ad000 tx=0x7fe114008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:17.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:17.857+0000 7fe1217fa640 1 -- 172.21.15.83:0/4076688202 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe118055460 con 0x7fe1240ce980 2024-04-17T10:53:18.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:18.049+0000 7fe12aee7640 1 -- 172.21.15.83:0/4076688202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1240ce530 con 0x7fe1240ce980 2024-04-17T10:53:18.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:18.049+0000 7fe1217fa640 1 -- 172.21.15.83:0/4076688202 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fe118054d60 con 0x7fe1240ce980 2024-04-17T10:53:18.045 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:18.045 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:18.045 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:18.049+0000 7fe12aee7640 1 -- 172.21.15.83:0/4076688202 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe110066f60 msgr2=0x7fe110069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:18.049+0000 7fe12aee7640 1 --2- 172.21.15.83:0/4076688202 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe110066f60 0x7fe110069420 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe1140ad000 tx=0x7fe114008040 comp rx=0 tx=0).stop 2024-04-17T10:53:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:18.053+0000 7fe12aee7640 1 -- 172.21.15.83:0/4076688202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1240ce980 msgr2=0x7fe124153fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:18.053+0000 7fe12aee7640 1 --2- 172.21.15.83:0/4076688202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1240ce980 0x7fe124153fc0 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7fe118007fd0 tx=0x7fe11800c8e0 comp rx=0 tx=0).stop 2024-04-17T10:53:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:18.053+0000 7fe12aee7640 1 -- 172.21.15.83:0/4076688202 shutdown_connections 2024-04-17T10:53:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:18.053+0000 7fe12aee7640 1 --2- 172.21.15.83:0/4076688202 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe110066f60 0x7fe110069420 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:18.053+0000 7fe12aee7640 1 --2- 172.21.15.83:0/4076688202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe12415dfa0 0x7fe1240cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:18.053+0000 7fe12aee7640 1 --2- 172.21.15.83:0/4076688202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1240ce980 0x7fe124153fc0 unknown :-1 s=CLOSED pgs=2295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:18.053+0000 7fe12aee7640 1 --2- 172.21.15.83:0/4076688202 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1240b5b40 0x7fe124156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:18.053+0000 7fe12aee7640 1 -- 172.21.15.83:0/4076688202 >> 172.21.15.83:0/4076688202 conn(0x7fe1240d0600 msgr2=0x7fe1240d39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:18.053+0000 7fe12aee7640 1 -- 172.21.15.83:0/4076688202 shutdown_connections 2024-04-17T10:53:18.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:18.053+0000 7fe12aee7640 1 -- 172.21.15.83:0/4076688202 wait complete. 2024-04-17T10:53:18.465 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:18.465 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:19.465 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:19.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.601+0000 7f3b51166640 1 Processor -- start 2024-04-17T10:53:19.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.601+0000 7f3b51166640 1 -- start start 2024-04-17T10:53:19.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.601+0000 7f3b51166640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b4c0b2de0 0x7f3b4c15ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:19.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.601+0000 7f3b51166640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3b4c0d22d0 0x7f3b4c0ae520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:19.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.601+0000 7f3b51166640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b4c15f5b0 con 0x7f3b4c15f1d0 2024-04-17T10:53:19.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.601+0000 7f3b51166640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b4c0b2620 con 0x7f3b4c0d22d0 2024-04-17T10:53:19.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.601+0000 7f3b51166640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c0b2340 con 0x7f3b4c0b2de0 2024-04-17T10:53:19.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.601+0000 7f3b4259c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b4c0b2de0 0x7f3b4c15ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:19.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.601+0000 7f3b4b59e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3b4c0d22d0 0x7f3b4c0ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:19.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.601+0000 7f3b4259c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b4c0b2de0 0x7f3b4c15ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:43410/0 (socket says 172.21.15.83:43410) 2024-04-17T10:53:19.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.601+0000 7f3b4259c640 1 -- 172.21.15.83:0/361139654 learned_addr learned my addr 172.21.15.83:0/361139654 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:19.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.601+0000 7f3b4259c640 1 -- 172.21.15.83:0/361139654 >> v2:172.21.15.167:3300/0 conn(0x7f3b4c0d22d0 msgr2=0x7f3b4c0ae520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:19.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.601+0000 7f3b4259c640 1 --2- 172.21.15.83:0/361139654 >> v2:172.21.15.167:3300/0 conn(0x7f3b4c0d22d0 0x7f3b4c0ae520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:19.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.601+0000 7f3b4259c640 1 -- 172.21.15.83:0/361139654 >> v1:172.21.15.167:6789/0 conn(0x7f3b4c15f1d0 legacy=0x7f3b4c15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:19.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.601+0000 7f3b4259c640 1 -- 172.21.15.83:0/361139654 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b4c0b1760 con 0x7f3b4c0b2de0 2024-04-17T10:53:19.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.605+0000 7f3b4259c640 1 --2- 172.21.15.83:0/361139654 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b4c0b2de0 0x7f3b4c15ff40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3b34009870 tx=0x7f3b3401c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95870825d23631e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:19.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.605+0000 7f3b4a59c640 1 -- 172.21.15.83:0/361139654 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b340a95f0 con 0x7f3b4c0b2de0 2024-04-17T10:53:19.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.605+0000 7f3b4a59c640 1 -- 172.21.15.83:0/361139654 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b340a9790 con 0x7f3b4c0b2de0 2024-04-17T10:53:19.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.605+0000 7f3b4a59c640 1 -- 172.21.15.83:0/361139654 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b340a9b60 con 0x7f3b4c0b2de0 2024-04-17T10:53:19.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.605+0000 7f3b51166640 1 -- 172.21.15.83:0/361139654 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b4c0b2de0 msgr2=0x7f3b4c15ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:19.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.605+0000 7f3b51166640 1 --2- 172.21.15.83:0/361139654 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b4c0b2de0 0x7f3b4c15ff40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3b34009870 tx=0x7f3b3401c530 comp rx=0 tx=0).stop 2024-04-17T10:53:19.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.605+0000 7f3b51166640 1 -- 172.21.15.83:0/361139654 shutdown_connections 2024-04-17T10:53:19.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.605+0000 7f3b51166640 1 --2- 172.21.15.83:0/361139654 >> v2:172.21.15.167:3300/0 conn(0x7f3b4c0d22d0 0x7f3b4c0ae520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:19.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.605+0000 7f3b51166640 1 --2- 172.21.15.83:0/361139654 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b4c0b2de0 0x7f3b4c15ff40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:19.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.605+0000 7f3b51166640 1 -- 172.21.15.83:0/361139654 >> 172.21.15.83:0/361139654 conn(0x7f3b4c0d17b0 msgr2=0x7f3b4c0d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:19.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.605+0000 7f3b51166640 1 -- 172.21.15.83:0/361139654 shutdown_connections 2024-04-17T10:53:19.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.605+0000 7f3b51166640 1 -- 172.21.15.83:0/361139654 wait complete. 2024-04-17T10:53:19.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.609+0000 7f3b51166640 1 Processor -- start 2024-04-17T10:53:19.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.609+0000 7f3b51166640 1 -- start start 2024-04-17T10:53:19.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.609+0000 7f3b51166640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b4c0d22d0 0x7f3b4c153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:19.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.609+0000 7f3b51166640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b4c15f1d0 0x7f3b4c14b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:19.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.609+0000 7f3b51166640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b4c06cf10 0x7f3b4c157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:19.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.609+0000 7f3b51166640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c0af270 con 0x7f3b4c0d22d0 2024-04-17T10:53:19.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.609+0000 7f3b51166640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c160690 con 0x7f3b4c06cf10 2024-04-17T10:53:19.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.609+0000 7f3b51166640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c0af3f0 con 0x7f3b4c15f1d0 2024-04-17T10:53:19.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.609+0000 7f3b4ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b4c0d22d0 0x7f3b4c153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:19.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.609+0000 7f3b4ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b4c0d22d0 0x7f3b4c153030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41340/0 (socket says 172.21.15.83:41340) 2024-04-17T10:53:19.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.609+0000 7f3b4ad9d640 1 -- 172.21.15.83:0/3855170449 learned_addr learned my addr 172.21.15.83:0/3855170449 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:19.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.609+0000 7f3b4259c640 1 --2- 172.21.15.83:0/3855170449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b4c15f1d0 0x7f3b4c14b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:19.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.609+0000 7f3b4b59e640 1 --2- 172.21.15.83:0/3855170449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b4c06cf10 0x7f3b4c157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:19.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.609+0000 7f3b4ad9d640 1 -- 172.21.15.83:0/3855170449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b4c15f1d0 msgr2=0x7f3b4c14b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:19.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.609+0000 7f3b4ad9d640 1 --2- 172.21.15.83:0/3855170449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b4c15f1d0 0x7f3b4c14b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:19.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.609+0000 7f3b4ad9d640 1 -- 172.21.15.83:0/3855170449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b4c06cf10 msgr2=0x7f3b4c157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:19.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.609+0000 7f3b4ad9d640 1 --2- 172.21.15.83:0/3855170449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b4c06cf10 0x7f3b4c157a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:19.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.609+0000 7f3b4ad9d640 1 -- 172.21.15.83:0/3855170449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b4c06dcb0 con 0x7f3b4c0d22d0 2024-04-17T10:53:19.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.609+0000 7f3b4259c640 1 --2- 172.21.15.83:0/3855170449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b4c15f1d0 0x7f3b4c14b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:53:19.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.609+0000 7f3b4ad9d640 1 --2- 172.21.15.83:0/3855170449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b4c0d22d0 0x7f3b4c153030 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c00c370 tx=0x7f3b3c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:19.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.609+0000 7f3b43fff640 1 -- 172.21.15.83:0/3855170449 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c0052d0 con 0x7f3b4c0d22d0 2024-04-17T10:53:19.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.609+0000 7f3b43fff640 1 -- 172.21.15.83:0/3855170449 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b3c005470 con 0x7f3b4c0d22d0 2024-04-17T10:53:19.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.609+0000 7f3b43fff640 1 -- 172.21.15.83:0/3855170449 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c007260 con 0x7f3b4c0d22d0 2024-04-17T10:53:19.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.609+0000 7f3b51166640 1 -- 172.21.15.83:0/3855170449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b4c06d410 con 0x7f3b4c0d22d0 2024-04-17T10:53:19.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.609+0000 7f3b51166640 1 -- 172.21.15.83:0/3855170449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b4c06d9d0 con 0x7f3b4c0d22d0 2024-04-17T10:53:19.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.609+0000 7f3b51166640 1 -- 172.21.15.83:0/3855170449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b4c074f90 con 0x7f3b4c0d22d0 2024-04-17T10:53:19.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.609+0000 7f3b43fff640 1 -- 172.21.15.83:0/3855170449 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3b3c007400 con 0x7f3b4c0d22d0 2024-04-17T10:53:19.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.613+0000 7f3b43fff640 1 --2- 172.21.15.83:0/3855170449 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3b2c066f60 0x7f3b2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:19.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.613+0000 7f3b43fff640 1 -- 172.21.15.83:0/3855170449 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f3b3c084500 con 0x7f3b4c0d22d0 2024-04-17T10:53:19.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.613+0000 7f3b4259c640 1 --2- 172.21.15.83:0/3855170449 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3b2c066f60 0x7f3b2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:19.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.613+0000 7f3b4259c640 1 --2- 172.21.15.83:0/3855170449 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3b2c066f60 0x7f3b2c069420 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f3b340ae000 tx=0x7f3b340b3040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:19.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.617+0000 7f3b43fff640 1 -- 172.21.15.83:0/3855170449 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3b3c0554a0 con 0x7f3b4c0d22d0 2024-04-17T10:53:19.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.801+0000 7f3b51166640 1 -- 172.21.15.83:0/3855170449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b4c0d1300 con 0x7f3b4c0d22d0 2024-04-17T10:53:19.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.801+0000 7f3b43fff640 1 -- 172.21.15.83:0/3855170449 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f3b3c054da0 con 0x7f3b4c0d22d0 2024-04-17T10:53:19.798 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:19.798 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:19.799 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:19.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.805+0000 7f3b51166640 1 -- 172.21.15.83:0/3855170449 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3b2c066f60 msgr2=0x7f3b2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:19.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.805+0000 7f3b51166640 1 --2- 172.21.15.83:0/3855170449 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3b2c066f60 0x7f3b2c069420 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f3b340ae000 tx=0x7f3b340b3040 comp rx=0 tx=0).stop 2024-04-17T10:53:19.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.805+0000 7f3b51166640 1 -- 172.21.15.83:0/3855170449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b4c0d22d0 msgr2=0x7f3b4c153030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:19.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.805+0000 7f3b51166640 1 --2- 172.21.15.83:0/3855170449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b4c0d22d0 0x7f3b4c153030 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c00c370 tx=0x7f3b3c00c830 comp rx=0 tx=0).stop 2024-04-17T10:53:19.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.805+0000 7f3b51166640 1 -- 172.21.15.83:0/3855170449 shutdown_connections 2024-04-17T10:53:19.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.805+0000 7f3b51166640 1 --2- 172.21.15.83:0/3855170449 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3b2c066f60 0x7f3b2c069420 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:19.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.805+0000 7f3b51166640 1 --2- 172.21.15.83:0/3855170449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b4c06cf10 0x7f3b4c157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:19.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.805+0000 7f3b51166640 1 --2- 172.21.15.83:0/3855170449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b4c15f1d0 0x7f3b4c14b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:19.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.805+0000 7f3b51166640 1 --2- 172.21.15.83:0/3855170449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b4c0d22d0 0x7f3b4c153030 unknown :-1 s=CLOSED pgs=2296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:19.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.805+0000 7f3b51166640 1 -- 172.21.15.83:0/3855170449 >> 172.21.15.83:0/3855170449 conn(0x7f3b4c0d17b0 msgr2=0x7f3b4c15f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:19.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.805+0000 7f3b51166640 1 -- 172.21.15.83:0/3855170449 shutdown_connections 2024-04-17T10:53:19.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:19.805+0000 7f3b51166640 1 -- 172.21.15.83:0/3855170449 wait complete. 2024-04-17T10:53:20.220 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:20.220 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:21.221 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:21.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.357+0000 7f6095a03640 1 Processor -- start 2024-04-17T10:53:21.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.357+0000 7f6095a03640 1 -- start start 2024-04-17T10:53:21.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.361+0000 7f6095a03640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f609015dfa0 0x7f60900b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:21.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.361+0000 7f6095a03640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f60900ce980 0x7f60900b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:21.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.361+0000 7f6095a03640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6090162a00 con 0x7f60900cee30 2024-04-17T10:53:21.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.361+0000 7f6095a03640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6090161e80 con 0x7f60900ce980 2024-04-17T10:53:21.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.361+0000 7f6095a03640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6090162160 con 0x7f609015dfa0 2024-04-17T10:53:21.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.361+0000 7f608e7fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f60900ce980 0x7f60900b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:21.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.361+0000 7f608e7fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f60900ce980 0x7f60900b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41346/0 (socket says 172.21.15.83:41346) 2024-04-17T10:53:21.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.361+0000 7f608e7fc640 1 -- 172.21.15.83:0/2260998121 learned_addr learned my addr 172.21.15.83:0/2260998121 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:21.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.361+0000 7f608effd640 1 --2- 172.21.15.83:0/2260998121 >> v2:172.21.15.167:3300/0 conn(0x7f609015dfa0 0x7f60900b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:21.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.361+0000 7f608e7fc640 1 -- 172.21.15.83:0/2260998121 >> v2:172.21.15.167:3300/0 conn(0x7f609015dfa0 msgr2=0x7f60900b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:21.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.361+0000 7f608e7fc640 1 --2- 172.21.15.83:0/2260998121 >> v2:172.21.15.167:3300/0 conn(0x7f609015dfa0 0x7f60900b6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:21.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.361+0000 7f608e7fc640 1 -- 172.21.15.83:0/2260998121 >> v1:172.21.15.167:6789/0 conn(0x7f60900cee30 legacy=0x7f60900d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:21.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.361+0000 7f608e7fc640 1 -- 172.21.15.83:0/2260998121 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6090162440 con 0x7f60900ce980 2024-04-17T10:53:21.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.361+0000 7f608effd640 1 --2- 172.21.15.83:0/2260998121 >> v2:172.21.15.167:3300/0 conn(0x7f609015dfa0 0x7f60900b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:53:21.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.361+0000 7f608e7fc640 1 --2- 172.21.15.83:0/2260998121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60900ce980 0x7f60900b5430 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f609008c3f0 tx=0x7f608001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16834a3bb1238a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:21.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.361+0000 7f608dffb640 1 -- 172.21.15.83:0/2260998121 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60800b0070 con 0x7f60900ce980 2024-04-17T10:53:21.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.361+0000 7f608dffb640 1 -- 172.21.15.83:0/2260998121 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60800ab600 con 0x7f60900ce980 2024-04-17T10:53:21.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.361+0000 7f608dffb640 1 -- 172.21.15.83:0/2260998121 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60800ab9d0 con 0x7f60900ce980 2024-04-17T10:53:21.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.361+0000 7f6095a03640 1 -- 172.21.15.83:0/2260998121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60900ce980 msgr2=0x7f60900b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:21.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.361+0000 7f6095a03640 1 --2- 172.21.15.83:0/2260998121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60900ce980 0x7f60900b5430 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f609008c3f0 tx=0x7f608001c420 comp rx=0 tx=0).stop 2024-04-17T10:53:21.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.361+0000 7f6095a03640 1 -- 172.21.15.83:0/2260998121 shutdown_connections 2024-04-17T10:53:21.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.361+0000 7f6095a03640 1 --2- 172.21.15.83:0/2260998121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60900ce980 0x7f60900b5430 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:21.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.361+0000 7f6095a03640 1 --2- 172.21.15.83:0/2260998121 >> v2:172.21.15.167:3300/0 conn(0x7f609015dfa0 0x7f60900b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:21.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.361+0000 7f6095a03640 1 -- 172.21.15.83:0/2260998121 >> 172.21.15.83:0/2260998121 conn(0x7f60900d0600 msgr2=0x7f60900d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:21.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.365+0000 7f6095a03640 1 -- 172.21.15.83:0/2260998121 shutdown_connections 2024-04-17T10:53:21.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.365+0000 7f6095a03640 1 -- 172.21.15.83:0/2260998121 wait complete. 2024-04-17T10:53:21.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.365+0000 7f6095a03640 1 Processor -- start 2024-04-17T10:53:21.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.365+0000 7f6095a03640 1 -- start start 2024-04-17T10:53:21.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.365+0000 7f6095a03640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60900ce980 0x7f60900ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:21.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.365+0000 7f6095a03640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60900cee30 0x7f6090152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:21.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.365+0000 7f6095a03640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f609015dfa0 0x7f609014d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:21.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.365+0000 7f6095a03640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6090161b10 con 0x7f609015dfa0 2024-04-17T10:53:21.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.365+0000 7f6095a03640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60900d2850 con 0x7f60900ce980 2024-04-17T10:53:21.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.365+0000 7f6095a03640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f609016b660 con 0x7f60900cee30 2024-04-17T10:53:21.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.365+0000 7f608f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f609015dfa0 0x7f609014d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:21.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.365+0000 7f608f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f609015dfa0 0x7f609014d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41358/0 (socket says 172.21.15.83:41358) 2024-04-17T10:53:21.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.365+0000 7f608f7fe640 1 -- 172.21.15.83:0/2407272910 learned_addr learned my addr 172.21.15.83:0/2407272910 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:21.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.365+0000 7f608effd640 1 --2- 172.21.15.83:0/2407272910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60900ce980 0x7f60900ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:21.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.365+0000 7f608e7fc640 1 --2- 172.21.15.83:0/2407272910 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60900cee30 0x7f6090152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:21.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.365+0000 7f608f7fe640 1 -- 172.21.15.83:0/2407272910 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60900cee30 msgr2=0x7f6090152d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:21.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.365+0000 7f608f7fe640 1 --2- 172.21.15.83:0/2407272910 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60900cee30 0x7f6090152d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:21.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.365+0000 7f608f7fe640 1 -- 172.21.15.83:0/2407272910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60900ce980 msgr2=0x7f60900ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:21.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.365+0000 7f608f7fe640 1 --2- 172.21.15.83:0/2407272910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60900ce980 0x7f60900ca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:21.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.365+0000 7f608f7fe640 1 -- 172.21.15.83:0/2407272910 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f609014c0b0 con 0x7f609015dfa0 2024-04-17T10:53:21.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.365+0000 7f608f7fe640 1 --2- 172.21.15.83:0/2407272910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f609015dfa0 0x7f609014d350 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f608800b9a0 tx=0x7f608800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:21.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.365+0000 7f6077fff640 1 -- 172.21.15.83:0/2407272910 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6088019070 con 0x7f609015dfa0 2024-04-17T10:53:21.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.365+0000 7f6077fff640 1 -- 172.21.15.83:0/2407272910 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f608800cba0 con 0x7f609015dfa0 2024-04-17T10:53:21.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.365+0000 7f6095a03640 1 -- 172.21.15.83:0/2407272910 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f609014bd20 con 0x7f609015dfa0 2024-04-17T10:53:21.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.365+0000 7f6077fff640 1 -- 172.21.15.83:0/2407272910 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6088014440 con 0x7f609015dfa0 2024-04-17T10:53:21.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.365+0000 7f6095a03640 1 -- 172.21.15.83:0/2407272910 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f609014cb50 con 0x7f609015dfa0 2024-04-17T10:53:21.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.369+0000 7f6095a03640 1 -- 172.21.15.83:0/2407272910 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6090074f90 con 0x7f609015dfa0 2024-04-17T10:53:21.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.369+0000 7f6077fff640 1 -- 172.21.15.83:0/2407272910 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f608801c070 con 0x7f609015dfa0 2024-04-17T10:53:21.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.369+0000 7f6077fff640 1 --2- 172.21.15.83:0/2407272910 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f607c066f90 0x7f607c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:21.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.369+0000 7f6077fff640 1 -- 172.21.15.83:0/2407272910 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f6088084c80 con 0x7f609015dfa0 2024-04-17T10:53:21.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.369+0000 7f608effd640 1 --2- 172.21.15.83:0/2407272910 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f607c066f90 0x7f607c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:21.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.369+0000 7f608effd640 1 --2- 172.21.15.83:0/2407272910 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f607c066f90 0x7f607c069450 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f60900c76e0 tx=0x7f6084008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:21.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.373+0000 7f6077fff640 1 -- 172.21.15.83:0/2407272910 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6088055c30 con 0x7f609015dfa0 2024-04-17T10:53:21.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.557+0000 7f6095a03640 1 -- 172.21.15.83:0/2407272910 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60900d17a0 con 0x7f609015dfa0 2024-04-17T10:53:21.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.561+0000 7f6077fff640 1 -- 172.21.15.83:0/2407272910 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f6088055530 con 0x7f609015dfa0 2024-04-17T10:53:21.555 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:21.555 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:21.556 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:21.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.561+0000 7f6095a03640 1 -- 172.21.15.83:0/2407272910 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f607c066f90 msgr2=0x7f607c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:21.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.561+0000 7f6095a03640 1 --2- 172.21.15.83:0/2407272910 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f607c066f90 0x7f607c069450 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f60900c76e0 tx=0x7f6084008040 comp rx=0 tx=0).stop 2024-04-17T10:53:21.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.561+0000 7f6095a03640 1 -- 172.21.15.83:0/2407272910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f609015dfa0 msgr2=0x7f609014d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:21.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.561+0000 7f6095a03640 1 --2- 172.21.15.83:0/2407272910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f609015dfa0 0x7f609014d350 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f608800b9a0 tx=0x7f608800be60 comp rx=0 tx=0).stop 2024-04-17T10:53:21.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.561+0000 7f6095a03640 1 -- 172.21.15.83:0/2407272910 shutdown_connections 2024-04-17T10:53:21.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.561+0000 7f6095a03640 1 --2- 172.21.15.83:0/2407272910 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f607c066f90 0x7f607c069450 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:21.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.561+0000 7f6095a03640 1 --2- 172.21.15.83:0/2407272910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f609015dfa0 0x7f609014d350 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:21.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.561+0000 7f6095a03640 1 --2- 172.21.15.83:0/2407272910 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60900cee30 0x7f6090152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:21.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.561+0000 7f6095a03640 1 --2- 172.21.15.83:0/2407272910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60900ce980 0x7f60900ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:21.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.561+0000 7f6095a03640 1 -- 172.21.15.83:0/2407272910 >> 172.21.15.83:0/2407272910 conn(0x7f60900d0600 msgr2=0x7f60900cd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:21.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.561+0000 7f6095a03640 1 -- 172.21.15.83:0/2407272910 shutdown_connections 2024-04-17T10:53:21.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:21.561+0000 7f6095a03640 1 -- 172.21.15.83:0/2407272910 wait complete. 2024-04-17T10:53:21.975 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:21.975 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:22.976 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:23.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.113+0000 7ff07dbd4640 1 Processor -- start 2024-04-17T10:53:23.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.113+0000 7ff07dbd4640 1 -- start start 2024-04-17T10:53:23.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.113+0000 7ff07dbd4640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff07815ba30 0x7ff078072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:23.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.113+0000 7ff07dbd4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff0780cfbe0 0x7ff07806c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:23.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.113+0000 7ff07dbd4640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff07806e320 con 0x7ff07815c960 2024-04-17T10:53:23.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.113+0000 7ff07dbd4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff078071ea0 con 0x7ff0780cfbe0 2024-04-17T10:53:23.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.113+0000 7ff07dbd4640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff07806e040 con 0x7ff07815ba30 2024-04-17T10:53:23.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.113+0000 7ff077fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff0780cfbe0 0x7ff07806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:23.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.113+0000 7ff076ffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff07815c960 0x7ff0780cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:37096/0 (socket says 172.21.15.83:37096) 2024-04-17T10:53:23.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.113+0000 7ff076ffd640 1 -- 172.21.15.83:0/637044208 learned_addr learned my addr 172.21.15.83:0/637044208 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:53:23.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.113+0000 7ff0777fe640 1 --2- 172.21.15.83:0/637044208 >> v2:172.21.15.167:3300/0 conn(0x7ff07815ba30 0x7ff078072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:23.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.113+0000 7ff0767fc640 1 -- 172.21.15.83:0/637044208 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2811675154 0 0) 0x7ff07806e320 con 0x7ff07815c960 2024-04-17T10:53:23.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.113+0000 7ff0767fc640 1 -- 172.21.15.83:0/637044208 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff064003530 con 0x7ff07815c960 2024-04-17T10:53:23.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.113+0000 7ff077fff640 1 -- 172.21.15.83:0/637044208 >> v2:172.21.15.167:3300/0 conn(0x7ff07815ba30 msgr2=0x7ff078072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:23.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.113+0000 7ff077fff640 1 --2- 172.21.15.83:0/637044208 >> v2:172.21.15.167:3300/0 conn(0x7ff07815ba30 0x7ff078072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:23.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.113+0000 7ff077fff640 1 -- 172.21.15.83:0/637044208 >> v1:172.21.15.83:6789/0 conn(0x7ff07815c960 legacy=0x7ff0780cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:23.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.113+0000 7ff077fff640 1 -- 172.21.15.83:0/637044208 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff078071850 con 0x7ff0780cfbe0 2024-04-17T10:53:23.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.117+0000 7ff077fff640 1 --2- 172.21.15.83:0/637044208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0780cfbe0 0x7ff07806c490 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7ff0780b2410 tx=0x7ff07001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef1fcfde669cee0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:23.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.117+0000 7ff0767fc640 1 -- 172.21.15.83:0/637044208 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0700b2070 con 0x7ff0780cfbe0 2024-04-17T10:53:23.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.117+0000 7ff0767fc640 1 -- 172.21.15.83:0/637044208 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0700a57b0 con 0x7ff0780cfbe0 2024-04-17T10:53:23.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.117+0000 7ff0767fc640 1 -- 172.21.15.83:0/637044208 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0700a5b60 con 0x7ff0780cfbe0 2024-04-17T10:53:23.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.117+0000 7ff07dbd4640 1 -- 172.21.15.83:0/637044208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0780cfbe0 msgr2=0x7ff07806c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:23.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.117+0000 7ff07dbd4640 1 --2- 172.21.15.83:0/637044208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0780cfbe0 0x7ff07806c490 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7ff0780b2410 tx=0x7ff07001c8a0 comp rx=0 tx=0).stop 2024-04-17T10:53:23.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.117+0000 7ff07dbd4640 1 -- 172.21.15.83:0/637044208 shutdown_connections 2024-04-17T10:53:23.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.117+0000 7ff07dbd4640 1 --2- 172.21.15.83:0/637044208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0780cfbe0 0x7ff07806c490 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:23.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.117+0000 7ff07dbd4640 1 --2- 172.21.15.83:0/637044208 >> v2:172.21.15.167:3300/0 conn(0x7ff07815ba30 0x7ff078072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:23.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.117+0000 7ff07dbd4640 1 -- 172.21.15.83:0/637044208 >> 172.21.15.83:0/637044208 conn(0x7ff07806abe0 msgr2=0x7ff07806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:23.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.117+0000 7ff07dbd4640 1 -- 172.21.15.83:0/637044208 shutdown_connections 2024-04-17T10:53:23.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.117+0000 7ff07dbd4640 1 -- 172.21.15.83:0/637044208 wait complete. 2024-04-17T10:53:23.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.117+0000 7ff07dbd4640 1 Processor -- start 2024-04-17T10:53:23.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.117+0000 7ff07dbd4640 1 -- start start 2024-04-17T10:53:23.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.117+0000 7ff07dbd4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0780cfbe0 0x7ff078145150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:23.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.117+0000 7ff07dbd4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff07815ba30 0x7ff0781484e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:23.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.117+0000 7ff07dbd4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff07815c960 0x7ff078148a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:23.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.117+0000 7ff07dbd4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07815cee0 con 0x7ff0780cfbe0 2024-04-17T10:53:23.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.117+0000 7ff07dbd4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07815cd60 con 0x7ff07815ba30 2024-04-17T10:53:23.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.117+0000 7ff07dbd4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff078168520 con 0x7ff07815c960 2024-04-17T10:53:23.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.117+0000 7ff0777fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0780cfbe0 0x7ff078145150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:23.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.117+0000 7ff0777fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0780cfbe0 0x7ff078145150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41378/0 (socket says 172.21.15.83:41378) 2024-04-17T10:53:23.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.117+0000 7ff0777fe640 1 -- 172.21.15.83:0/3659827385 learned_addr learned my addr 172.21.15.83:0/3659827385 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:23.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.117+0000 7ff0777fe640 1 -- 172.21.15.83:0/3659827385 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff07815c960 msgr2=0x7ff078148a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:23.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.117+0000 7ff0777fe640 1 --2- 172.21.15.83:0/3659827385 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff07815c960 0x7ff078148a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:23.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.117+0000 7ff0777fe640 1 -- 172.21.15.83:0/3659827385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff07815ba30 msgr2=0x7ff0781484e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:23.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.117+0000 7ff0777fe640 1 --2- 172.21.15.83:0/3659827385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff07815ba30 0x7ff0781484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:23.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.117+0000 7ff0777fe640 1 -- 172.21.15.83:0/3659827385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0780cf620 con 0x7ff0780cfbe0 2024-04-17T10:53:23.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.117+0000 7ff0777fe640 1 --2- 172.21.15.83:0/3659827385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0780cfbe0 0x7ff078145150 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7ff06000d800 tx=0x7ff06000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:23.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.121+0000 7ff06ffff640 1 -- 172.21.15.83:0/3659827385 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06000b6c0 con 0x7ff0780cfbe0 2024-04-17T10:53:23.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.121+0000 7ff06ffff640 1 -- 172.21.15.83:0/3659827385 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff060019070 con 0x7ff0780cfbe0 2024-04-17T10:53:23.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.121+0000 7ff07dbd4640 1 -- 172.21.15.83:0/3659827385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0780cf900 con 0x7ff0780cfbe0 2024-04-17T10:53:23.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.121+0000 7ff06ffff640 1 -- 172.21.15.83:0/3659827385 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06000ba60 con 0x7ff0780cfbe0 2024-04-17T10:53:23.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.121+0000 7ff07dbd4640 1 -- 172.21.15.83:0/3659827385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff078146710 con 0x7ff0780cfbe0 2024-04-17T10:53:23.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.121+0000 7ff07dbd4640 1 -- 172.21.15.83:0/3659827385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff078074f90 con 0x7ff0780cfbe0 2024-04-17T10:53:23.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.121+0000 7ff06ffff640 1 -- 172.21.15.83:0/3659827385 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff06000bc00 con 0x7ff0780cfbe0 2024-04-17T10:53:23.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.121+0000 7ff06ffff640 1 --2- 172.21.15.83:0/3659827385 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff058066f60 0x7ff058069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:23.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.121+0000 7ff06ffff640 1 -- 172.21.15.83:0/3659827385 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7ff060085500 con 0x7ff0780cfbe0 2024-04-17T10:53:23.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.121+0000 7ff076ffd640 1 --2- 172.21.15.83:0/3659827385 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff058066f60 0x7ff058069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:23.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.121+0000 7ff076ffd640 1 --2- 172.21.15.83:0/3659827385 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff058066f60 0x7ff058069420 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7ff068007fd0 tx=0x7ff068007900 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:23.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.125+0000 7ff06ffff640 1 -- 172.21.15.83:0/3659827385 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff0600564a0 con 0x7ff0780cfbe0 2024-04-17T10:53:23.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.313+0000 7ff07dbd4640 1 -- 172.21.15.83:0/3659827385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff078074480 con 0x7ff0780cfbe0 2024-04-17T10:53:23.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.313+0000 7ff06ffff640 1 -- 172.21.15.83:0/3659827385 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7ff060055da0 con 0x7ff0780cfbe0 2024-04-17T10:53:23.309 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:23.310 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:23.310 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:23.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.317+0000 7ff07dbd4640 1 -- 172.21.15.83:0/3659827385 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff058066f60 msgr2=0x7ff058069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:23.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.317+0000 7ff07dbd4640 1 --2- 172.21.15.83:0/3659827385 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff058066f60 0x7ff058069420 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7ff068007fd0 tx=0x7ff068007900 comp rx=0 tx=0).stop 2024-04-17T10:53:23.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.317+0000 7ff07dbd4640 1 -- 172.21.15.83:0/3659827385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0780cfbe0 msgr2=0x7ff078145150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:23.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.317+0000 7ff07dbd4640 1 --2- 172.21.15.83:0/3659827385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0780cfbe0 0x7ff078145150 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7ff06000d800 tx=0x7ff06000dcc0 comp rx=0 tx=0).stop 2024-04-17T10:53:23.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.317+0000 7ff07dbd4640 1 -- 172.21.15.83:0/3659827385 shutdown_connections 2024-04-17T10:53:23.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.317+0000 7ff07dbd4640 1 --2- 172.21.15.83:0/3659827385 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff058066f60 0x7ff058069420 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:23.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.317+0000 7ff07dbd4640 1 --2- 172.21.15.83:0/3659827385 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff07815c960 0x7ff078148a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:23.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.317+0000 7ff07dbd4640 1 --2- 172.21.15.83:0/3659827385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff07815ba30 0x7ff0781484e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:23.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.317+0000 7ff07dbd4640 1 --2- 172.21.15.83:0/3659827385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0780cfbe0 0x7ff078145150 unknown :-1 s=CLOSED pgs=2301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:23.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.317+0000 7ff07dbd4640 1 -- 172.21.15.83:0/3659827385 >> 172.21.15.83:0/3659827385 conn(0x7ff07806abe0 msgr2=0x7ff07815d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:23.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.317+0000 7ff07dbd4640 1 -- 172.21.15.83:0/3659827385 shutdown_connections 2024-04-17T10:53:23.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:23.317+0000 7ff07dbd4640 1 -- 172.21.15.83:0/3659827385 wait complete. 2024-04-17T10:53:23.729 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:23.729 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:24.730 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:24.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.869+0000 7fc8551b5640 1 Processor -- start 2024-04-17T10:53:24.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.869+0000 7fc8551b5640 1 -- start start 2024-04-17T10:53:24.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.869+0000 7fc8551b5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc85015ba30 0x7fc850072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:24.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.869+0000 7fc8551b5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc85015c960 0x7fc8500cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:24.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.869+0000 7fc8551b5640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8500cfea0 con 0x7fc8500cfa10 2024-04-17T10:53:24.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.869+0000 7fc8551b5640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc850071ea0 con 0x7fc85015c960 2024-04-17T10:53:24.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.869+0000 7fc8551b5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85006e320 con 0x7fc85015ba30 2024-04-17T10:53:24.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.869+0000 7fc84e59c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc85015c960 0x7fc8500cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:24.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.869+0000 7fc84ed9d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc85015ba30 0x7fc850072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:24.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.869+0000 7fc84f59e640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fc8500cfa10 0x7fc85006c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:35598/0 (socket says 172.21.15.83:35598) 2024-04-17T10:53:24.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.869+0000 7fc84f59e640 1 -- 172.21.15.83:0/4035807414 learned_addr learned my addr 172.21.15.83:0/4035807414 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:53:24.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.869+0000 7fc84ed9d640 1 -- 172.21.15.83:0/4035807414 >> v2:172.21.15.167:3300/0 conn(0x7fc85015c960 msgr2=0x7fc8500cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:24.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.869+0000 7fc84ed9d640 1 --2- 172.21.15.83:0/4035807414 >> v2:172.21.15.167:3300/0 conn(0x7fc85015c960 0x7fc8500cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:24.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.869+0000 7fc84ed9d640 1 -- 172.21.15.83:0/4035807414 >> v1:172.21.15.167:6789/0 conn(0x7fc8500cfa10 legacy=0x7fc85006c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:24.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.869+0000 7fc84ed9d640 1 -- 172.21.15.83:0/4035807414 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85006e040 con 0x7fc85015ba30 2024-04-17T10:53:24.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.873+0000 7fc84ed9d640 1 --2- 172.21.15.83:0/4035807414 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc85015ba30 0x7fc850072330 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc8400097c0 tx=0x7fc84001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43102bc62914449f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:24.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.873+0000 7fc84dd9b640 1 -- 172.21.15.83:0/4035807414 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8400b0070 con 0x7fc85015ba30 2024-04-17T10:53:24.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.873+0000 7fc84dd9b640 1 -- 172.21.15.83:0/4035807414 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8400ab600 con 0x7fc85015ba30 2024-04-17T10:53:24.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.873+0000 7fc84dd9b640 1 -- 172.21.15.83:0/4035807414 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8400ab9d0 con 0x7fc85015ba30 2024-04-17T10:53:24.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.873+0000 7fc8551b5640 1 -- 172.21.15.83:0/4035807414 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc85015ba30 msgr2=0x7fc850072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:24.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.873+0000 7fc8551b5640 1 --2- 172.21.15.83:0/4035807414 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc85015ba30 0x7fc850072330 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc8400097c0 tx=0x7fc84001c410 comp rx=0 tx=0).stop 2024-04-17T10:53:24.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.873+0000 7fc8551b5640 1 -- 172.21.15.83:0/4035807414 shutdown_connections 2024-04-17T10:53:24.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.873+0000 7fc8551b5640 1 --2- 172.21.15.83:0/4035807414 >> v2:172.21.15.167:3300/0 conn(0x7fc85015c960 0x7fc8500cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:24.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.873+0000 7fc8551b5640 1 --2- 172.21.15.83:0/4035807414 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc85015ba30 0x7fc850072330 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:24.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.873+0000 7fc8551b5640 1 -- 172.21.15.83:0/4035807414 >> 172.21.15.83:0/4035807414 conn(0x7fc85006abe0 msgr2=0x7fc85006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:24.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.873+0000 7fc8551b5640 1 -- 172.21.15.83:0/4035807414 shutdown_connections 2024-04-17T10:53:24.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.873+0000 7fc8551b5640 1 -- 172.21.15.83:0/4035807414 wait complete. 2024-04-17T10:53:24.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.877+0000 7fc8551b5640 1 Processor -- start 2024-04-17T10:53:24.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.877+0000 7fc8551b5640 1 -- start start 2024-04-17T10:53:24.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.877+0000 7fc8551b5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8500cfa10 0x7fc85015a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:24.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.877+0000 7fc8551b5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc85015ba30 0x7fc850157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:24.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.877+0000 7fc8551b5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc85015c960 0x7fc850157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:24.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.877+0000 7fc8551b5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85006de20 con 0x7fc85015ba30 2024-04-17T10:53:24.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.877+0000 7fc8551b5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85006dca0 con 0x7fc8500cfa10 2024-04-17T10:53:24.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.877+0000 7fc8551b5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85015cd40 con 0x7fc85015c960 2024-04-17T10:53:24.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.877+0000 7fc84e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc85015ba30 0x7fc850157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:24.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.877+0000 7fc84e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc85015ba30 0x7fc850157fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41394/0 (socket says 172.21.15.83:41394) 2024-04-17T10:53:24.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.877+0000 7fc84e59c640 1 -- 172.21.15.83:0/130814634 learned_addr learned my addr 172.21.15.83:0/130814634 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:24.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.877+0000 7fc84ed9d640 1 --2- 172.21.15.83:0/130814634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8500cfa10 0x7fc85015a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:24.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.877+0000 7fc84f59e640 1 --2- 172.21.15.83:0/130814634 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc85015c960 0x7fc850157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:24.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.877+0000 7fc84e59c640 1 -- 172.21.15.83:0/130814634 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc85015c960 msgr2=0x7fc850157480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:24.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.877+0000 7fc84e59c640 1 --2- 172.21.15.83:0/130814634 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc85015c960 0x7fc850157480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:24.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.877+0000 7fc84e59c640 1 -- 172.21.15.83:0/130814634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8500cfa10 msgr2=0x7fc85015a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:24.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.877+0000 7fc84e59c640 1 --2- 172.21.15.83:0/130814634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8500cfa10 0x7fc85015a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:24.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.877+0000 7fc84e59c640 1 -- 172.21.15.83:0/130814634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc850158800 con 0x7fc85015ba30 2024-04-17T10:53:24.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.877+0000 7fc84ed9d640 1 --2- 172.21.15.83:0/130814634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8500cfa10 0x7fc85015a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:53:24.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.877+0000 7fc84e59c640 1 --2- 172.21.15.83:0/130814634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc85015ba30 0x7fc850157fd0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7fc84400e930 tx=0x7fc84400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:24.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.877+0000 7fc8377fe640 1 -- 172.21.15.83:0/130814634 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc844017070 con 0x7fc85015ba30 2024-04-17T10:53:24.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.877+0000 7fc8377fe640 1 -- 172.21.15.83:0/130814634 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84400ca00 con 0x7fc85015ba30 2024-04-17T10:53:24.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.877+0000 7fc8551b5640 1 -- 172.21.15.83:0/130814634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8501569a0 con 0x7fc85015ba30 2024-04-17T10:53:24.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.877+0000 7fc8377fe640 1 -- 172.21.15.83:0/130814634 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84400cdb0 con 0x7fc85015ba30 2024-04-17T10:53:24.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.877+0000 7fc8551b5640 1 -- 172.21.15.83:0/130814634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc850156f60 con 0x7fc85015ba30 2024-04-17T10:53:24.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.877+0000 7fc8551b5640 1 -- 172.21.15.83:0/130814634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc850074f90 con 0x7fc85015ba30 2024-04-17T10:53:24.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.881+0000 7fc8377fe640 1 -- 172.21.15.83:0/130814634 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc84401a070 con 0x7fc85015ba30 2024-04-17T10:53:24.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.881+0000 7fc8377fe640 1 --2- 172.21.15.83:0/130814634 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc83c066f60 0x7fc83c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:24.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.881+0000 7fc84ed9d640 1 --2- 172.21.15.83:0/130814634 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc83c066f60 0x7fc83c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:24.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.881+0000 7fc8377fe640 1 -- 172.21.15.83:0/130814634 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fc844085410 con 0x7fc85015ba30 2024-04-17T10:53:24.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.881+0000 7fc84ed9d640 1 --2- 172.21.15.83:0/130814634 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc83c066f60 0x7fc83c069420 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fc8400ad000 tx=0x7fc840004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:24.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:24.885+0000 7fc8377fe640 1 -- 172.21.15.83:0/130814634 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc844056500 con 0x7fc85015ba30 2024-04-17T10:53:25.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:25.061+0000 7fc8551b5640 1 -- 172.21.15.83:0/130814634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc85015d020 con 0x7fc85015ba30 2024-04-17T10:53:25.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:25.065+0000 7fc8377fe640 1 -- 172.21.15.83:0/130814634 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fc844055e00 con 0x7fc85015ba30 2024-04-17T10:53:25.060 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:25.060 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:25.060 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:25.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:25.065+0000 7fc8551b5640 1 -- 172.21.15.83:0/130814634 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc83c066f60 msgr2=0x7fc83c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:25.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:25.065+0000 7fc8551b5640 1 --2- 172.21.15.83:0/130814634 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc83c066f60 0x7fc83c069420 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fc8400ad000 tx=0x7fc840004740 comp rx=0 tx=0).stop 2024-04-17T10:53:25.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:25.065+0000 7fc8551b5640 1 -- 172.21.15.83:0/130814634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc85015ba30 msgr2=0x7fc850157fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:25.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:25.065+0000 7fc8551b5640 1 --2- 172.21.15.83:0/130814634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc85015ba30 0x7fc850157fd0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7fc84400e930 tx=0x7fc84400edf0 comp rx=0 tx=0).stop 2024-04-17T10:53:25.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:25.065+0000 7fc8551b5640 1 -- 172.21.15.83:0/130814634 shutdown_connections 2024-04-17T10:53:25.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:25.065+0000 7fc8551b5640 1 --2- 172.21.15.83:0/130814634 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc83c066f60 0x7fc83c069420 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:25.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:25.065+0000 7fc8551b5640 1 --2- 172.21.15.83:0/130814634 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc85015c960 0x7fc850157480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:25.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:25.065+0000 7fc8551b5640 1 --2- 172.21.15.83:0/130814634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc85015ba30 0x7fc850157fd0 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:25.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:25.065+0000 7fc8551b5640 1 --2- 172.21.15.83:0/130814634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8500cfa10 0x7fc85015a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:25.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:25.065+0000 7fc8551b5640 1 -- 172.21.15.83:0/130814634 >> 172.21.15.83:0/130814634 conn(0x7fc85006abe0 msgr2=0x7fc850070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:25.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:25.065+0000 7fc8551b5640 1 -- 172.21.15.83:0/130814634 shutdown_connections 2024-04-17T10:53:25.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:25.065+0000 7fc8551b5640 1 -- 172.21.15.83:0/130814634 wait complete. 2024-04-17T10:53:25.479 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:25.479 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:26.480 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:26.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ee36db640 1 Processor -- start 2024-04-17T10:53:26.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ee36db640 1 -- start start 2024-04-17T10:53:26.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ee36db640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8edc15db50 0x7f8edc070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:26.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ee36db640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8edc0aedd0 0x7f8edc06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:26.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ee36db640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8edc0709b0 con 0x7f8edc06dae0 2024-04-17T10:53:26.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ee36db640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8edc15e440 con 0x7f8edc0aedd0 2024-04-17T10:53:26.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ee36db640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc15e720 con 0x7f8edc15db50 2024-04-17T10:53:26.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ee1c78640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8edc06dae0 0x7f8edc0cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:37104/0 (socket says 172.21.15.83:37104) 2024-04-17T10:53:26.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ee1c78640 1 -- 172.21.15.83:0/1487727829 learned_addr learned my addr 172.21.15.83:0/1487727829 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:53:26.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ee1477640 1 --2- 172.21.15.83:0/1487727829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8edc15db50 0x7f8edc070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:26.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ee0c76640 1 --2- 172.21.15.83:0/1487727829 >> v2:172.21.15.167:3300/0 conn(0x7f8edc0aedd0 0x7f8edc06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:26.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ed3fff640 1 -- 172.21.15.83:0/1487727829 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3016331960 0 0) 0x7f8ed40014a0 con 0x7f8edc06dae0 2024-04-17T10:53:26.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ed3fff640 1 -- 172.21.15.83:0/1487727829 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ec8003530 con 0x7f8edc06dae0 2024-04-17T10:53:26.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ed3fff640 1 -- 172.21.15.83:0/1487727829 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2908509253 0 0) 0x7f8ec8003530 con 0x7f8edc06dae0 2024-04-17T10:53:26.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ed3fff640 1 -- 172.21.15.83:0/1487727829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8edc15db50 msgr2=0x7f8edc070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:26.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ed3fff640 1 --2- 172.21.15.83:0/1487727829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8edc15db50 0x7f8edc070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:26.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ed3fff640 1 -- 172.21.15.83:0/1487727829 >> v2:172.21.15.167:3300/0 conn(0x7f8edc0aedd0 msgr2=0x7f8edc06d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:26.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ed3fff640 1 --2- 172.21.15.83:0/1487727829 >> v2:172.21.15.167:3300/0 conn(0x7f8edc0aedd0 0x7f8edc06d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:26.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ed3fff640 1 -- 172.21.15.83:0/1487727829 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8edc15ea00 con 0x7f8edc06dae0 2024-04-17T10:53:26.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ed3fff640 1 -- 172.21.15.83:0/1487727829 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f8ed4001bd0 con 0x7f8edc06dae0 2024-04-17T10:53:26.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ed3fff640 1 -- 172.21.15.83:0/1487727829 >> v1:172.21.15.83:6789/0 conn(0x7f8edc06dae0 legacy=0x7f8edc0cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:26.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ed3fff640 1 --2- 172.21.15.83:0/1487727829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8edc15db50 0x7f8ec80039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:26.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ed3fff640 1 --2- 172.21.15.83:0/1487727829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ec8003f40 0x7f8ec8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:26.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ed3fff640 1 --2- 172.21.15.83:0/1487727829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ec8004d50 0x7f8ec80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:26.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ed3fff640 1 -- 172.21.15.83:0/1487727829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc15e440 con 0x7f8ec8004d50 2024-04-17T10:53:26.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ed3fff640 1 -- 172.21.15.83:0/1487727829 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc15e720 con 0x7f8ec8003f40 2024-04-17T10:53:26.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ed3fff640 1 -- 172.21.15.83:0/1487727829 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec800b9d0 con 0x7f8edc15db50 2024-04-17T10:53:26.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ee1c78640 1 --2- 172.21.15.83:0/1487727829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ec8004d50 0x7f8ec80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:26.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ee1477640 1 --2- 172.21.15.83:0/1487727829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8edc15db50 0x7f8ec80039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:26.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ee0c76640 1 --2- 172.21.15.83:0/1487727829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ec8003f40 0x7f8ec8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:26.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ee1c78640 1 -- 172.21.15.83:0/1487727829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8edc15db50 msgr2=0x7f8ec80039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:26.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ee1c78640 1 --2- 172.21.15.83:0/1487727829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8edc15db50 0x7f8ec80039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:26.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ee1c78640 1 -- 172.21.15.83:0/1487727829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ec8003f40 msgr2=0x7f8ec8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:26.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ee1c78640 1 --2- 172.21.15.83:0/1487727829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ec8003f40 0x7f8ec8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:26.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.617+0000 7f8ee1c78640 1 -- 172.21.15.83:0/1487727829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec8003530 con 0x7f8ec8004d50 2024-04-17T10:53:26.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.621+0000 7f8ee1c78640 1 --2- 172.21.15.83:0/1487727829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ec8004d50 0x7f8ec80051d0 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400aa70 tx=0x7f8ed401d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0c1a428cf05f9d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:26.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.621+0000 7f8ed3fff640 1 -- 172.21.15.83:0/1487727829 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed400a690 con 0x7f8ec8004d50 2024-04-17T10:53:26.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.621+0000 7f8ed3fff640 1 -- 172.21.15.83:0/1487727829 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed40a9070 con 0x7f8ec8004d50 2024-04-17T10:53:26.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.621+0000 7f8ed3fff640 1 -- 172.21.15.83:0/1487727829 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed40ae910 con 0x7f8ec8004d50 2024-04-17T10:53:26.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.621+0000 7f8ee36db640 1 -- 172.21.15.83:0/1487727829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ec8004d50 msgr2=0x7f8ec80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:26.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.621+0000 7f8ee36db640 1 --2- 172.21.15.83:0/1487727829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ec8004d50 0x7f8ec80051d0 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400aa70 tx=0x7f8ed401d360 comp rx=0 tx=0).stop 2024-04-17T10:53:26.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.621+0000 7f8ee1c78640 1 -- 172.21.15.83:0/1487727829 reap_dead start 2024-04-17T10:53:26.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.621+0000 7f8ee36db640 1 -- 172.21.15.83:0/1487727829 shutdown_connections 2024-04-17T10:53:26.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.621+0000 7f8ee36db640 1 -- 172.21.15.83:0/1487727829 >> 172.21.15.83:0/1487727829 conn(0x7f8edc06abe0 msgr2=0x7f8edc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:26.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.621+0000 7f8ee36db640 1 -- 172.21.15.83:0/1487727829 shutdown_connections 2024-04-17T10:53:26.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.621+0000 7f8ee36db640 1 -- 172.21.15.83:0/1487727829 wait complete. 2024-04-17T10:53:26.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.621+0000 7f8ee36db640 1 Processor -- start 2024-04-17T10:53:26.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.621+0000 7f8ee36db640 1 -- start start 2024-04-17T10:53:26.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.621+0000 7f8ee36db640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ec8004d50 0x7f8edc151240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:26.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.621+0000 7f8ee36db640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8edc153440 0x7f8edc14d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:26.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.621+0000 7f8ee36db640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8edc0ad840 0x7f8edc0adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:26.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.621+0000 7f8ee36db640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc15df30 con 0x7f8ec8004d50 2024-04-17T10:53:26.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.621+0000 7f8ee36db640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc15efc0 con 0x7f8edc0ad840 2024-04-17T10:53:26.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.621+0000 7f8ee36db640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc15e0b0 con 0x7f8edc153440 2024-04-17T10:53:26.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.621+0000 7f8ee1477640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ec8004d50 0x7f8edc151240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:26.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.621+0000 7f8ee1477640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ec8004d50 0x7f8edc151240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41410/0 (socket says 172.21.15.83:41410) 2024-04-17T10:53:26.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.621+0000 7f8ee1477640 1 -- 172.21.15.83:0/2187840825 learned_addr learned my addr 172.21.15.83:0/2187840825 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:26.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.625+0000 7f8ee0c76640 1 --2- 172.21.15.83:0/2187840825 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8edc153440 0x7f8edc14d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:26.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.625+0000 7f8ee1c78640 1 --2- 172.21.15.83:0/2187840825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8edc0ad840 0x7f8edc0adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:26.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.625+0000 7f8ee1477640 1 -- 172.21.15.83:0/2187840825 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8edc153440 msgr2=0x7f8edc14d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:26.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.625+0000 7f8ee1477640 1 --2- 172.21.15.83:0/2187840825 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8edc153440 0x7f8edc14d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:26.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.625+0000 7f8ee1477640 1 -- 172.21.15.83:0/2187840825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8edc0ad840 msgr2=0x7f8edc0adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:26.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.625+0000 7f8ee1477640 1 --2- 172.21.15.83:0/2187840825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8edc0ad840 0x7f8edc0adcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:26.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.625+0000 7f8ee1477640 1 -- 172.21.15.83:0/2187840825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8edc1538c0 con 0x7f8ec8004d50 2024-04-17T10:53:26.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.625+0000 7f8ee1477640 1 --2- 172.21.15.83:0/2187840825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ec8004d50 0x7f8edc151240 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8011610 tx=0x7f8ed8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:26.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.625+0000 7f8ed1ffb640 1 -- 172.21.15.83:0/2187840825 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed800f8f0 con 0x7f8ec8004d50 2024-04-17T10:53:26.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.625+0000 7f8ed1ffb640 1 -- 172.21.15.83:0/2187840825 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed800fa90 con 0x7f8ec8004d50 2024-04-17T10:53:26.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.625+0000 7f8ee36db640 1 -- 172.21.15.83:0/2187840825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8edc1518d0 con 0x7f8ec8004d50 2024-04-17T10:53:26.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.625+0000 7f8ed1ffb640 1 -- 172.21.15.83:0/2187840825 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed800fe40 con 0x7f8ec8004d50 2024-04-17T10:53:26.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.625+0000 7f8ee36db640 1 -- 172.21.15.83:0/2187840825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8edc151e90 con 0x7f8ec8004d50 2024-04-17T10:53:26.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.625+0000 7f8ee36db640 1 -- 172.21.15.83:0/2187840825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8edc0ae200 con 0x7f8ec8004d50 2024-04-17T10:53:26.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.625+0000 7f8ed1ffb640 1 -- 172.21.15.83:0/2187840825 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8ed8015460 con 0x7f8ec8004d50 2024-04-17T10:53:26.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.625+0000 7f8ed1ffb640 1 --2- 172.21.15.83:0/2187840825 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8ec0066f60 0x7f8ec0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:26.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.625+0000 7f8ed1ffb640 1 -- 172.21.15.83:0/2187840825 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f8ed8085900 con 0x7f8ec8004d50 2024-04-17T10:53:26.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.625+0000 7f8ee0c76640 1 --2- 172.21.15.83:0/2187840825 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8ec0066f60 0x7f8ec0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:26.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.625+0000 7f8ee0c76640 1 --2- 172.21.15.83:0/2187840825 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8ec0066f60 0x7f8ec0069420 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f8edc1f7080 tx=0x7f8ecc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:26.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.629+0000 7f8ed1ffb640 1 -- 172.21.15.83:0/2187840825 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8ed8019030 con 0x7f8ec8004d50 2024-04-17T10:53:26.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.809+0000 7f8ee36db640 1 -- 172.21.15.83:0/2187840825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8edc074f90 con 0x7f8ec8004d50 2024-04-17T10:53:26.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.809+0000 7f8ed1ffb640 1 -- 172.21.15.83:0/2187840825 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f8ed8056250 con 0x7f8ec8004d50 2024-04-17T10:53:26.804 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:26.805 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:26.805 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:26.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.809+0000 7f8ee36db640 1 -- 172.21.15.83:0/2187840825 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8ec0066f60 msgr2=0x7f8ec0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:26.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.809+0000 7f8ee36db640 1 --2- 172.21.15.83:0/2187840825 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8ec0066f60 0x7f8ec0069420 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f8edc1f7080 tx=0x7f8ecc008040 comp rx=0 tx=0).stop 2024-04-17T10:53:26.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.809+0000 7f8ee36db640 1 -- 172.21.15.83:0/2187840825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ec8004d50 msgr2=0x7f8edc151240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:26.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.809+0000 7f8ee36db640 1 --2- 172.21.15.83:0/2187840825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ec8004d50 0x7f8edc151240 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8011610 tx=0x7f8ed8011ad0 comp rx=0 tx=0).stop 2024-04-17T10:53:26.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.809+0000 7f8ee36db640 1 -- 172.21.15.83:0/2187840825 shutdown_connections 2024-04-17T10:53:26.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.809+0000 7f8ee36db640 1 --2- 172.21.15.83:0/2187840825 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8ec0066f60 0x7f8ec0069420 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:26.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.809+0000 7f8ee36db640 1 --2- 172.21.15.83:0/2187840825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8edc0ad840 0x7f8edc0adcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:26.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.809+0000 7f8ee36db640 1 --2- 172.21.15.83:0/2187840825 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8edc153440 0x7f8edc14d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:26.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.809+0000 7f8ee36db640 1 --2- 172.21.15.83:0/2187840825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ec8004d50 0x7f8edc151240 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:26.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.809+0000 7f8ee36db640 1 -- 172.21.15.83:0/2187840825 >> 172.21.15.83:0/2187840825 conn(0x7f8edc06abe0 msgr2=0x7f8edc0af600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:26.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.809+0000 7f8ee36db640 1 -- 172.21.15.83:0/2187840825 shutdown_connections 2024-04-17T10:53:26.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:26.809+0000 7f8ee36db640 1 -- 172.21.15.83:0/2187840825 wait complete. 2024-04-17T10:53:27.225 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:27.225 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:28.226 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:28.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.369+0000 7f62c57db640 1 Processor -- start 2024-04-17T10:53:28.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.369+0000 7f62c57db640 1 -- start start 2024-04-17T10:53:28.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.369+0000 7f62c57db640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62c00cfbe0 0x7f62c006c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:28.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.369+0000 7f62c57db640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62c006e320 con 0x7f62c0072330 2024-04-17T10:53:28.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.369+0000 7f62c57db640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62c006e040 con 0x7f62c015ba30 2024-04-17T10:53:28.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.369+0000 7f62c57db640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62c0071ea0 con 0x7f62c00cfbe0 2024-04-17T10:53:28.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.369+0000 7f62be7fc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f62c0072330 0x7f62c00cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42614/0 (socket says 172.21.15.83:42614) 2024-04-17T10:53:28.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.369+0000 7f62be7fc640 1 -- 172.21.15.83:0/2741122673 learned_addr learned my addr 172.21.15.83:0/2741122673 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:53:28.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.369+0000 7f62bdffb640 1 -- 172.21.15.83:0/2741122673 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2948230756 0 0) 0x7f62c006e320 con 0x7f62c0072330 2024-04-17T10:53:28.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.369+0000 7f62bf7fe640 1 --2- 172.21.15.83:0/2741122673 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62c00cfbe0 0x7f62c006c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:28.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.369+0000 7f62bdffb640 1 -- 172.21.15.83:0/2741122673 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62a8003530 con 0x7f62c0072330 2024-04-17T10:53:28.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.369+0000 7f62bdffb640 1 -- 172.21.15.83:0/2741122673 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 233728125 0 0) 0x7f62a8003530 con 0x7f62c0072330 2024-04-17T10:53:28.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.369+0000 7f62bdffb640 1 -- 172.21.15.83:0/2741122673 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62c00cfbe0 msgr2=0x7f62c006c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:28.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.369+0000 7f62bdffb640 1 --2- 172.21.15.83:0/2741122673 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62c00cfbe0 0x7f62c006c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:28.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.369+0000 7f62bdffb640 1 -- 172.21.15.83:0/2741122673 >> v1:172.21.15.167:6789/0 conn(0x7f62c015ba30 legacy=0x7f62c015c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:28.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.369+0000 7f62bdffb640 1 -- 172.21.15.83:0/2741122673 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c0071850 con 0x7f62c0072330 2024-04-17T10:53:28.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.369+0000 7f62bdffb640 1 -- 172.21.15.83:0/2741122673 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f62b4001680 con 0x7f62c0072330 2024-04-17T10:53:28.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.369+0000 7f62bdffb640 1 -- 172.21.15.83:0/2741122673 >> v1:172.21.15.83:6789/0 conn(0x7f62c0072330 legacy=0x7f62c00cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:28.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.369+0000 7f62bdffb640 1 --2- 172.21.15.83:0/2741122673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62a80039f0 0x7f62a8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:28.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.369+0000 7f62bdffb640 1 --2- 172.21.15.83:0/2741122673 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62c00cfbe0 0x7f62a8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:28.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.369+0000 7f62bdffb640 1 --2- 172.21.15.83:0/2741122673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62a8004d50 0x7f62a80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:28.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.369+0000 7f62bdffb640 1 -- 172.21.15.83:0/2741122673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c0071ea0 con 0x7f62a8004d50 2024-04-17T10:53:28.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.369+0000 7f62bdffb640 1 -- 172.21.15.83:0/2741122673 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a800bb00 con 0x7f62a80039f0 2024-04-17T10:53:28.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.369+0000 7f62bdffb640 1 -- 172.21.15.83:0/2741122673 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a800bd00 con 0x7f62c00cfbe0 2024-04-17T10:53:28.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.369+0000 7f62bf7fe640 1 --2- 172.21.15.83:0/2741122673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62a8004d50 0x7f62a80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:28.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.369+0000 7f62be7fc640 1 --2- 172.21.15.83:0/2741122673 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62c00cfbe0 0x7f62a8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:28.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.369+0000 7f62beffd640 1 --2- 172.21.15.83:0/2741122673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62a80039f0 0x7f62a8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:28.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.369+0000 7f62bf7fe640 1 -- 172.21.15.83:0/2741122673 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62c00cfbe0 msgr2=0x7f62a8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:28.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.369+0000 7f62bf7fe640 1 --2- 172.21.15.83:0/2741122673 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62c00cfbe0 0x7f62a8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:28.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.369+0000 7f62bf7fe640 1 -- 172.21.15.83:0/2741122673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62a80039f0 msgr2=0x7f62a8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:28.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.369+0000 7f62bf7fe640 1 --2- 172.21.15.83:0/2741122673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62a80039f0 0x7f62a8003e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:28.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.369+0000 7f62bf7fe640 1 -- 172.21.15.83:0/2741122673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a8003530 con 0x7f62a8004d50 2024-04-17T10:53:28.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.373+0000 7f62bf7fe640 1 --2- 172.21.15.83:0/2741122673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62a8004d50 0x7f62a80051d0 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f62b8009fa0 tx=0x7f62b801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325e5268d3704451 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:28.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.373+0000 7f62bdffb640 1 -- 172.21.15.83:0/2741122673 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b80b2070 con 0x7f62a8004d50 2024-04-17T10:53:28.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.373+0000 7f62bdffb640 1 -- 172.21.15.83:0/2741122673 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62b80a5c00 con 0x7f62a8004d50 2024-04-17T10:53:28.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.373+0000 7f62bdffb640 1 -- 172.21.15.83:0/2741122673 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b8009bf0 con 0x7f62a8004d50 2024-04-17T10:53:28.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.373+0000 7f62c57db640 1 -- 172.21.15.83:0/2741122673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62a8004d50 msgr2=0x7f62a80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:28.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.373+0000 7f62c57db640 1 --2- 172.21.15.83:0/2741122673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62a8004d50 0x7f62a80051d0 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f62b8009fa0 tx=0x7f62b801cc40 comp rx=0 tx=0).stop 2024-04-17T10:53:28.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.373+0000 7f62bf7fe640 1 -- 172.21.15.83:0/2741122673 reap_dead start 2024-04-17T10:53:28.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.373+0000 7f62c57db640 1 -- 172.21.15.83:0/2741122673 shutdown_connections 2024-04-17T10:53:28.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.373+0000 7f62c57db640 1 -- 172.21.15.83:0/2741122673 >> 172.21.15.83:0/2741122673 conn(0x7f62c006abe0 msgr2=0x7f62c006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:28.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.373+0000 7f62c57db640 1 -- 172.21.15.83:0/2741122673 shutdown_connections 2024-04-17T10:53:28.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.373+0000 7f62c57db640 1 -- 172.21.15.83:0/2741122673 wait complete. 2024-04-17T10:53:28.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.373+0000 7f62c57db640 1 Processor -- start 2024-04-17T10:53:28.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.373+0000 7f62c57db640 1 -- start start 2024-04-17T10:53:28.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.373+0000 7f62c57db640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62a8004d50 0x7f62c014c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:28.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.373+0000 7f62c57db640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62c014ec40 0x7f62c0155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:28.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.373+0000 7f62c57db640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62c01555c0 0x7f62c0152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:28.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.373+0000 7f62c57db640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c006dca0 con 0x7f62a8004d50 2024-04-17T10:53:28.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.373+0000 7f62c57db640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c00aeae0 con 0x7f62c01555c0 2024-04-17T10:53:28.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.373+0000 7f62c57db640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62c006de20 con 0x7f62c014ec40 2024-04-17T10:53:28.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.373+0000 7f62beffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62a8004d50 0x7f62c014c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:28.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.373+0000 7f62beffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62a8004d50 0x7f62c014c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36614/0 (socket says 172.21.15.83:36614) 2024-04-17T10:53:28.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.373+0000 7f62beffd640 1 -- 172.21.15.83:0/1996732879 learned_addr learned my addr 172.21.15.83:0/1996732879 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:28.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.373+0000 7f62be7fc640 1 --2- 172.21.15.83:0/1996732879 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62c014ec40 0x7f62c0155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:28.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.373+0000 7f62bf7fe640 1 --2- 172.21.15.83:0/1996732879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62c01555c0 0x7f62c0152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:28.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.373+0000 7f62beffd640 1 -- 172.21.15.83:0/1996732879 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62c014ec40 msgr2=0x7f62c0155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:28.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.373+0000 7f62beffd640 1 --2- 172.21.15.83:0/1996732879 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62c014ec40 0x7f62c0155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:28.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.373+0000 7f62beffd640 1 -- 172.21.15.83:0/1996732879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62c01555c0 msgr2=0x7f62c0152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:28.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.373+0000 7f62beffd640 1 --2- 172.21.15.83:0/1996732879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62c01555c0 0x7f62c0152ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:28.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.373+0000 7f62beffd640 1 -- 172.21.15.83:0/1996732879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c014dbe0 con 0x7f62a8004d50 2024-04-17T10:53:28.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.377+0000 7f62bf7fe640 1 --2- 172.21.15.83:0/1996732879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62c01555c0 0x7f62c0152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:53:28.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.377+0000 7f62beffd640 1 --2- 172.21.15.83:0/1996732879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62a8004d50 0x7f62c014c2f0 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f62b000d7d0 tx=0x7f62b000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:28.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.377+0000 7f62a77fe640 1 -- 172.21.15.83:0/1996732879 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b0005270 con 0x7f62a8004d50 2024-04-17T10:53:28.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.377+0000 7f62a77fe640 1 -- 172.21.15.83:0/1996732879 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62b0017070 con 0x7f62a8004d50 2024-04-17T10:53:28.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.377+0000 7f62c57db640 1 -- 172.21.15.83:0/1996732879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62c014de80 con 0x7f62a8004d50 2024-04-17T10:53:28.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.377+0000 7f62a77fe640 1 -- 172.21.15.83:0/1996732879 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b000b6b0 con 0x7f62a8004d50 2024-04-17T10:53:28.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.377+0000 7f62c57db640 1 -- 172.21.15.83:0/1996732879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62c014e440 con 0x7f62a8004d50 2024-04-17T10:53:28.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.377+0000 7f62c57db640 1 -- 172.21.15.83:0/1996732879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62c0074f90 con 0x7f62a8004d50 2024-04-17T10:53:28.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.377+0000 7f62a77fe640 1 -- 172.21.15.83:0/1996732879 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f62b000b850 con 0x7f62a8004d50 2024-04-17T10:53:28.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.377+0000 7f62a77fe640 1 --2- 172.21.15.83:0/1996732879 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f62ac066f60 0x7f62ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:28.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.377+0000 7f62a77fe640 1 -- 172.21.15.83:0/1996732879 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f62b0084ec0 con 0x7f62a8004d50 2024-04-17T10:53:28.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.377+0000 7f62be7fc640 1 --2- 172.21.15.83:0/1996732879 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f62ac066f60 0x7f62ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:28.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.377+0000 7f62be7fc640 1 --2- 172.21.15.83:0/1996732879 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f62ac066f60 0x7f62ac069420 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f62c0071470 tx=0x7f62b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:28.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.381+0000 7f62a77fe640 1 -- 172.21.15.83:0/1996732879 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f62b0055e60 con 0x7f62a8004d50 2024-04-17T10:53:28.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.573+0000 7f62c57db640 1 -- 172.21.15.83:0/1996732879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62c00cbb10 con 0x7f62a8004d50 2024-04-17T10:53:28.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.577+0000 7f62a77fe640 1 -- 172.21.15.83:0/1996732879 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f62b001ae40 con 0x7f62a8004d50 2024-04-17T10:53:28.570 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:28.570 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:28.570 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:28.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.581+0000 7f62c57db640 1 -- 172.21.15.83:0/1996732879 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f62ac066f60 msgr2=0x7f62ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:28.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.581+0000 7f62c57db640 1 --2- 172.21.15.83:0/1996732879 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f62ac066f60 0x7f62ac069420 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f62c0071470 tx=0x7f62b4009040 comp rx=0 tx=0).stop 2024-04-17T10:53:28.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.581+0000 7f62c57db640 1 -- 172.21.15.83:0/1996732879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62a8004d50 msgr2=0x7f62c014c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:28.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.581+0000 7f62c57db640 1 --2- 172.21.15.83:0/1996732879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62a8004d50 0x7f62c014c2f0 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f62b000d7d0 tx=0x7f62b000dc90 comp rx=0 tx=0).stop 2024-04-17T10:53:28.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.581+0000 7f62c57db640 1 -- 172.21.15.83:0/1996732879 shutdown_connections 2024-04-17T10:53:28.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.581+0000 7f62c57db640 1 --2- 172.21.15.83:0/1996732879 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f62ac066f60 0x7f62ac069420 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:28.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.581+0000 7f62c57db640 1 --2- 172.21.15.83:0/1996732879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62c01555c0 0x7f62c0152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:28.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.581+0000 7f62c57db640 1 --2- 172.21.15.83:0/1996732879 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62c014ec40 0x7f62c0155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:28.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.581+0000 7f62c57db640 1 --2- 172.21.15.83:0/1996732879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62a8004d50 0x7f62c014c2f0 unknown :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:28.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.581+0000 7f62c57db640 1 -- 172.21.15.83:0/1996732879 >> 172.21.15.83:0/1996732879 conn(0x7f62c006abe0 msgr2=0x7f62c006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:28.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.581+0000 7f62c57db640 1 -- 172.21.15.83:0/1996732879 shutdown_connections 2024-04-17T10:53:28.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:28.581+0000 7f62c57db640 1 -- 172.21.15.83:0/1996732879 wait complete. 2024-04-17T10:53:28.992 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:28.992 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:29.993 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:30.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.133+0000 7f1919c3e640 1 Processor -- start 2024-04-17T10:53:30.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.133+0000 7f1919c3e640 1 -- start start 2024-04-17T10:53:30.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.133+0000 7f1919c3e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f191415db50 0x7f1914070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:30.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.133+0000 7f1919c3e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f19140aedd0 0x7f191406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:30.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.133+0000 7f1919c3e640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19140709b0 con 0x7f191406dae0 2024-04-17T10:53:30.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.133+0000 7f1919c3e640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f191415e440 con 0x7f19140aedd0 2024-04-17T10:53:30.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.133+0000 7f1919c3e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f191415e720 con 0x7f191415db50 2024-04-17T10:53:30.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.133+0000 7f1912ffd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f19140aedd0 0x7f191406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:30.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.133+0000 7f19137fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f191415db50 0x7f1914070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:30.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.133+0000 7f1913fff640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f191406dae0 0x7f19140cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:38138/0 (socket says 172.21.15.83:38138) 2024-04-17T10:53:30.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.133+0000 7f1913fff640 1 -- 172.21.15.83:0/736065663 learned_addr learned my addr 172.21.15.83:0/736065663 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:53:30.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.133+0000 7f19127fc640 1 -- 172.21.15.83:0/736065663 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3015729379 0 0) 0x7f190c0014a0 con 0x7f191406dae0 2024-04-17T10:53:30.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.133+0000 7f19127fc640 1 -- 172.21.15.83:0/736065663 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18fc005740 con 0x7f191406dae0 2024-04-17T10:53:30.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.133+0000 7f19137fe640 1 -- 172.21.15.83:0/736065663 >> v2:172.21.15.167:3300/0 conn(0x7f19140aedd0 msgr2=0x7f191406d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:30.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.133+0000 7f19137fe640 1 --2- 172.21.15.83:0/736065663 >> v2:172.21.15.167:3300/0 conn(0x7f19140aedd0 0x7f191406d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:30.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.133+0000 7f19137fe640 1 -- 172.21.15.83:0/736065663 >> v1:172.21.15.167:6789/0 conn(0x7f191406dae0 legacy=0x7f19140cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:30.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.133+0000 7f19137fe640 1 -- 172.21.15.83:0/736065663 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f191415ea00 con 0x7f191415db50 2024-04-17T10:53:30.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.133+0000 7f1912ffd640 1 --2- 172.21.15.83:0/736065663 >> v2:172.21.15.167:3300/0 conn(0x7f19140aedd0 0x7f191406d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:53:30.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.137+0000 7f19137fe640 1 --2- 172.21.15.83:0/736065663 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f191415db50 0x7f1914070470 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f19080097c0 tx=0x7f190801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57d8f06667bb51d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:30.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.137+0000 7f19127fc640 1 -- 172.21.15.83:0/736065663 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19080b2070 con 0x7f191415db50 2024-04-17T10:53:30.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.137+0000 7f19127fc640 1 -- 172.21.15.83:0/736065663 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19080ad640 con 0x7f191415db50 2024-04-17T10:53:30.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.137+0000 7f19127fc640 1 -- 172.21.15.83:0/736065663 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19080ada10 con 0x7f191415db50 2024-04-17T10:53:30.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.137+0000 7f1919c3e640 1 -- 172.21.15.83:0/736065663 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f191415db50 msgr2=0x7f1914070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:30.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.137+0000 7f1919c3e640 1 --2- 172.21.15.83:0/736065663 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f191415db50 0x7f1914070470 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f19080097c0 tx=0x7f190801c4a0 comp rx=0 tx=0).stop 2024-04-17T10:53:30.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.137+0000 7f1919c3e640 1 -- 172.21.15.83:0/736065663 shutdown_connections 2024-04-17T10:53:30.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.137+0000 7f1919c3e640 1 --2- 172.21.15.83:0/736065663 >> v2:172.21.15.167:3300/0 conn(0x7f19140aedd0 0x7f191406d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:30.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.137+0000 7f1919c3e640 1 --2- 172.21.15.83:0/736065663 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f191415db50 0x7f1914070470 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:30.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.137+0000 7f1919c3e640 1 -- 172.21.15.83:0/736065663 >> 172.21.15.83:0/736065663 conn(0x7f191406abe0 msgr2=0x7f191406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:30.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.137+0000 7f1919c3e640 1 -- 172.21.15.83:0/736065663 shutdown_connections 2024-04-17T10:53:30.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.137+0000 7f1919c3e640 1 -- 172.21.15.83:0/736065663 wait complete. 2024-04-17T10:53:30.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.137+0000 7f1919c3e640 1 Processor -- start 2024-04-17T10:53:30.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.137+0000 7f1919c3e640 1 -- start start 2024-04-17T10:53:30.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.141+0000 7f1919c3e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f191406dae0 0x7f19141faa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:30.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.141+0000 7f1919c3e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19140aedd0 0x7f19140adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:30.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.141+0000 7f1919c3e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f191415db50 0x7f191414d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:30.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.141+0000 7f1919c3e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191406c610 con 0x7f19140aedd0 2024-04-17T10:53:30.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.141+0000 7f1919c3e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191406c490 con 0x7f191406dae0 2024-04-17T10:53:30.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.141+0000 7f1919c3e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f191406c790 con 0x7f191415db50 2024-04-17T10:53:30.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.141+0000 7f1912ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19140aedd0 0x7f19140adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:30.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.141+0000 7f1912ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19140aedd0 0x7f19140adaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36630/0 (socket says 172.21.15.83:36630) 2024-04-17T10:53:30.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.141+0000 7f1912ffd640 1 -- 172.21.15.83:0/4110678455 learned_addr learned my addr 172.21.15.83:0/4110678455 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:30.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.141+0000 7f19137fe640 1 --2- 172.21.15.83:0/4110678455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f191406dae0 0x7f19141faa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:30.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.141+0000 7f1913fff640 1 --2- 172.21.15.83:0/4110678455 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f191415db50 0x7f191414d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:30.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.141+0000 7f1912ffd640 1 -- 172.21.15.83:0/4110678455 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f191415db50 msgr2=0x7f191414d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:30.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.141+0000 7f1912ffd640 1 --2- 172.21.15.83:0/4110678455 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f191415db50 0x7f191414d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:30.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.141+0000 7f1912ffd640 1 -- 172.21.15.83:0/4110678455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f191406dae0 msgr2=0x7f19141faa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:30.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.141+0000 7f1912ffd640 1 --2- 172.21.15.83:0/4110678455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f191406dae0 0x7f19141faa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:30.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.141+0000 7f1912ffd640 1 -- 172.21.15.83:0/4110678455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19141faf60 con 0x7f19140aedd0 2024-04-17T10:53:30.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.141+0000 7f19137fe640 1 --2- 172.21.15.83:0/4110678455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f191406dae0 0x7f19141faa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:53:30.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.141+0000 7f1912ffd640 1 --2- 172.21.15.83:0/4110678455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19140aedd0 0x7f19140adaa0 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f190400ec10 tx=0x7f190400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:30.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.141+0000 7f18fbfff640 1 -- 172.21.15.83:0/4110678455 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1904017070 con 0x7f19140aedd0 2024-04-17T10:53:30.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.141+0000 7f18fbfff640 1 -- 172.21.15.83:0/4110678455 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f190400cc30 con 0x7f19140aedd0 2024-04-17T10:53:30.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.141+0000 7f18fbfff640 1 -- 172.21.15.83:0/4110678455 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1904013470 con 0x7f19140aedd0 2024-04-17T10:53:30.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.141+0000 7f1919c3e640 1 -- 172.21.15.83:0/4110678455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1914155aa0 con 0x7f19140aedd0 2024-04-17T10:53:30.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.141+0000 7f1919c3e640 1 -- 172.21.15.83:0/4110678455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1914156060 con 0x7f19140aedd0 2024-04-17T10:53:30.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.141+0000 7f1919c3e640 1 -- 172.21.15.83:0/4110678455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1914074f90 con 0x7f19140aedd0 2024-04-17T10:53:30.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.141+0000 7f18fbfff640 1 -- 172.21.15.83:0/4110678455 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f190401a070 con 0x7f19140aedd0 2024-04-17T10:53:30.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.141+0000 7f18fbfff640 1 --2- 172.21.15.83:0/4110678455 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1900066f60 0x7f1900069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:30.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.141+0000 7f18fbfff640 1 -- 172.21.15.83:0/4110678455 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f1904084fc0 con 0x7f19140aedd0 2024-04-17T10:53:30.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.141+0000 7f19137fe640 1 --2- 172.21.15.83:0/4110678455 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1900066f60 0x7f1900069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:30.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.145+0000 7f19137fe640 1 --2- 172.21.15.83:0/4110678455 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1900066f60 0x7f1900069420 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f19080af000 tx=0x7f19080a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:30.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.145+0000 7f18fbfff640 1 -- 172.21.15.83:0/4110678455 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f19141faf60 con 0x7f19140aedd0 2024-04-17T10:53:30.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.341+0000 7f1919c3e640 1 -- 172.21.15.83:0/4110678455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19140b2af0 con 0x7f19140aedd0 2024-04-17T10:53:30.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.341+0000 7f18fbfff640 1 -- 172.21.15.83:0/4110678455 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f1904055f60 con 0x7f19140aedd0 2024-04-17T10:53:30.333 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:30.333 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:30.333 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:30.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.341+0000 7f1919c3e640 1 -- 172.21.15.83:0/4110678455 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1900066f60 msgr2=0x7f1900069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:30.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.341+0000 7f1919c3e640 1 --2- 172.21.15.83:0/4110678455 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1900066f60 0x7f1900069420 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f19080af000 tx=0x7f19080a7070 comp rx=0 tx=0).stop 2024-04-17T10:53:30.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.341+0000 7f1919c3e640 1 -- 172.21.15.83:0/4110678455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19140aedd0 msgr2=0x7f19140adaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:30.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.341+0000 7f1919c3e640 1 --2- 172.21.15.83:0/4110678455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19140aedd0 0x7f19140adaa0 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f190400ec10 tx=0x7f190400c520 comp rx=0 tx=0).stop 2024-04-17T10:53:30.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.345+0000 7f1919c3e640 1 -- 172.21.15.83:0/4110678455 shutdown_connections 2024-04-17T10:53:30.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.345+0000 7f1919c3e640 1 --2- 172.21.15.83:0/4110678455 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1900066f60 0x7f1900069420 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:30.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.345+0000 7f1919c3e640 1 --2- 172.21.15.83:0/4110678455 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f191415db50 0x7f191414d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:30.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.345+0000 7f1919c3e640 1 --2- 172.21.15.83:0/4110678455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19140aedd0 0x7f19140adaa0 unknown :-1 s=CLOSED pgs=2309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:30.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.345+0000 7f1919c3e640 1 --2- 172.21.15.83:0/4110678455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f191406dae0 0x7f19141faa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:30.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.345+0000 7f1919c3e640 1 -- 172.21.15.83:0/4110678455 >> 172.21.15.83:0/4110678455 conn(0x7f191406abe0 msgr2=0x7f19140b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:30.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.345+0000 7f1919c3e640 1 -- 172.21.15.83:0/4110678455 shutdown_connections 2024-04-17T10:53:30.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:30.345+0000 7f1919c3e640 1 -- 172.21.15.83:0/4110678455 wait complete. 2024-04-17T10:53:30.754 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:30.754 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:31.755 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:31.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.897+0000 7fa61ca15640 1 Processor -- start 2024-04-17T10:53:31.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.897+0000 7fa61ca15640 1 -- start start 2024-04-17T10:53:31.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.897+0000 7fa61ca15640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa61815b250 0x7fa6180ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:31.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.897+0000 7fa61ca15640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa61815ed20 con 0x7fa61815e480 2024-04-17T10:53:31.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.897+0000 7fa61ca15640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa61815f000 con 0x7fa6180713a0 2024-04-17T10:53:31.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.897+0000 7fa61ca15640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61815f2e0 con 0x7fa61815b250 2024-04-17T10:53:31.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.897+0000 7fa60dd9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa61815e480 0x7fa61815b630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42628/0 (socket says 172.21.15.83:42628) 2024-04-17T10:53:31.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.897+0000 7fa60dd9b640 1 -- 172.21.15.83:0/1009246320 learned_addr learned my addr 172.21.15.83:0/1009246320 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:53:31.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.897+0000 7fa61659c640 1 --2- 172.21.15.83:0/1009246320 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa61815b250 0x7fa6180ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:31.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.897+0000 7fa615d9b640 1 -- 172.21.15.83:0/1009246320 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 450052396 0 0) 0x7fa61815ed20 con 0x7fa61815e480 2024-04-17T10:53:31.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.897+0000 7fa615d9b640 1 -- 172.21.15.83:0/1009246320 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa604003530 con 0x7fa61815e480 2024-04-17T10:53:31.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.897+0000 7fa615d9b640 1 -- 172.21.15.83:0/1009246320 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4147573709 0 0) 0x7fa604003530 con 0x7fa61815e480 2024-04-17T10:53:31.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.897+0000 7fa615d9b640 1 -- 172.21.15.83:0/1009246320 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa61815b250 msgr2=0x7fa6180ce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:31.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.897+0000 7fa615d9b640 1 --2- 172.21.15.83:0/1009246320 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa61815b250 0x7fa6180ce1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:31.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.897+0000 7fa615d9b640 1 -- 172.21.15.83:0/1009246320 >> v1:172.21.15.167:6789/0 conn(0x7fa6180713a0 legacy=0x7fa6180cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:31.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.897+0000 7fa615d9b640 1 -- 172.21.15.83:0/1009246320 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6180cbaf0 con 0x7fa61815e480 2024-04-17T10:53:31.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.897+0000 7fa615d9b640 1 -- 172.21.15.83:0/1009246320 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fa600001680 con 0x7fa61815e480 2024-04-17T10:53:31.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.897+0000 7fa615d9b640 1 -- 172.21.15.83:0/1009246320 >> v1:172.21.15.83:6789/0 conn(0x7fa61815e480 legacy=0x7fa61815b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:31.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.897+0000 7fa615d9b640 1 --2- 172.21.15.83:0/1009246320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6040039f0 0x7fa604003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:31.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.897+0000 7fa615d9b640 1 --2- 172.21.15.83:0/1009246320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa604004810 0x7fa604004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:31.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.897+0000 7fa615d9b640 1 --2- 172.21.15.83:0/1009246320 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa61815b250 0x7fa6040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:31.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.897+0000 7fa615d9b640 1 -- 172.21.15.83:0/1009246320 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61815f2e0 con 0x7fa604004810 2024-04-17T10:53:31.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.897+0000 7fa615d9b640 1 -- 172.21.15.83:0/1009246320 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60400ba80 con 0x7fa6040039f0 2024-04-17T10:53:31.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.897+0000 7fa615d9b640 1 -- 172.21.15.83:0/1009246320 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa60400de90 con 0x7fa61815b250 2024-04-17T10:53:31.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.897+0000 7fa60dd9b640 1 --2- 172.21.15.83:0/1009246320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa604004810 0x7fa604004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:31.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.897+0000 7fa61659c640 1 --2- 172.21.15.83:0/1009246320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6040039f0 0x7fa604003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:31.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.897+0000 7fa616d9d640 1 --2- 172.21.15.83:0/1009246320 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa61815b250 0x7fa6040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:31.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.897+0000 7fa60dd9b640 1 -- 172.21.15.83:0/1009246320 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa61815b250 msgr2=0x7fa6040051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:31.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.897+0000 7fa60dd9b640 1 --2- 172.21.15.83:0/1009246320 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa61815b250 0x7fa6040051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:31.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.897+0000 7fa60dd9b640 1 -- 172.21.15.83:0/1009246320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6040039f0 msgr2=0x7fa604003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:31.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.897+0000 7fa60dd9b640 1 --2- 172.21.15.83:0/1009246320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6040039f0 0x7fa604003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:31.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.897+0000 7fa60dd9b640 1 -- 172.21.15.83:0/1009246320 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa604003530 con 0x7fa604004810 2024-04-17T10:53:31.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.901+0000 7fa60dd9b640 1 --2- 172.21.15.83:0/1009246320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa604004810 0x7fa604004c90 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7fa60000c210 tx=0x7fa60001f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59e9e4e2adfbc46 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:31.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.901+0000 7fa615d9b640 1 -- 172.21.15.83:0/1009246320 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa600007d20 con 0x7fa604004810 2024-04-17T10:53:31.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.901+0000 7fa615d9b640 1 -- 172.21.15.83:0/1009246320 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa600005ae0 con 0x7fa604004810 2024-04-17T10:53:31.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.901+0000 7fa615d9b640 1 -- 172.21.15.83:0/1009246320 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa600007070 con 0x7fa604004810 2024-04-17T10:53:31.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.901+0000 7fa61ca15640 1 -- 172.21.15.83:0/1009246320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa604004810 msgr2=0x7fa604004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:31.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.901+0000 7fa61ca15640 1 --2- 172.21.15.83:0/1009246320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa604004810 0x7fa604004c90 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7fa60000c210 tx=0x7fa60001f390 comp rx=0 tx=0).stop 2024-04-17T10:53:31.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.901+0000 7fa616d9d640 1 -- 172.21.15.83:0/1009246320 reap_dead start 2024-04-17T10:53:31.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.901+0000 7fa61ca15640 1 -- 172.21.15.83:0/1009246320 shutdown_connections 2024-04-17T10:53:31.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.901+0000 7fa61ca15640 1 -- 172.21.15.83:0/1009246320 >> 172.21.15.83:0/1009246320 conn(0x7fa61806bcf0 msgr2=0x7fa61806c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:31.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.901+0000 7fa61ca15640 1 -- 172.21.15.83:0/1009246320 shutdown_connections 2024-04-17T10:53:31.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.901+0000 7fa61ca15640 1 -- 172.21.15.83:0/1009246320 wait complete. 2024-04-17T10:53:31.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.901+0000 7fa61ca15640 1 Processor -- start 2024-04-17T10:53:31.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.901+0000 7fa61ca15640 1 -- start start 2024-04-17T10:53:31.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.905+0000 7fa61ca15640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa604004810 0x7fa6180b3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:31.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.905+0000 7fa61ca15640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa618147ae0 0x7fa618147f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:31.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.905+0000 7fa61ca15640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6180c88e0 0x7fa6180c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:31.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.905+0000 7fa61ca15640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61815e8e0 con 0x7fa604004810 2024-04-17T10:53:31.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.905+0000 7fa61ca15640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61815bd40 con 0x7fa6180c88e0 2024-04-17T10:53:31.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.905+0000 7fa61ca15640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61815ea60 con 0x7fa618147ae0 2024-04-17T10:53:31.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.905+0000 7fa61659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa604004810 0x7fa6180b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:31.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.905+0000 7fa61659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa604004810 0x7fa6180b3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36652/0 (socket says 172.21.15.83:36652) 2024-04-17T10:53:31.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.905+0000 7fa61659c640 1 -- 172.21.15.83:0/3642547215 learned_addr learned my addr 172.21.15.83:0/3642547215 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:31.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.905+0000 7fa60dd9b640 1 --2- 172.21.15.83:0/3642547215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa618147ae0 0x7fa618147f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:31.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.905+0000 7fa616d9d640 1 --2- 172.21.15.83:0/3642547215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6180c88e0 0x7fa6180c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:31.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.905+0000 7fa61659c640 1 -- 172.21.15.83:0/3642547215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa618147ae0 msgr2=0x7fa618147f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:31.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.905+0000 7fa61659c640 1 --2- 172.21.15.83:0/3642547215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa618147ae0 0x7fa618147f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:31.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.905+0000 7fa61659c640 1 -- 172.21.15.83:0/3642547215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6180c88e0 msgr2=0x7fa6180c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:31.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.905+0000 7fa61659c640 1 --2- 172.21.15.83:0/3642547215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6180c88e0 0x7fa6180c8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:31.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.905+0000 7fa61659c640 1 -- 172.21.15.83:0/3642547215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6180c96a0 con 0x7fa604004810 2024-04-17T10:53:31.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.905+0000 7fa61659c640 1 --2- 172.21.15.83:0/3642547215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa604004810 0x7fa6180b3630 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7fa6080115e0 tx=0x7fa608011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:31.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.905+0000 7fa60f7fe640 1 -- 172.21.15.83:0/3642547215 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60800f8f0 con 0x7fa604004810 2024-04-17T10:53:31.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.905+0000 7fa60f7fe640 1 -- 172.21.15.83:0/3642547215 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa60800fa90 con 0x7fa604004810 2024-04-17T10:53:31.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.905+0000 7fa60f7fe640 1 -- 172.21.15.83:0/3642547215 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60800fe40 con 0x7fa604004810 2024-04-17T10:53:31.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.905+0000 7fa61ca15640 1 -- 172.21.15.83:0/3642547215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6180c9980 con 0x7fa604004810 2024-04-17T10:53:31.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.905+0000 7fa61ca15640 1 -- 172.21.15.83:0/3642547215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6180c9f40 con 0x7fa604004810 2024-04-17T10:53:31.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.905+0000 7fa61ca15640 1 -- 172.21.15.83:0/3642547215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa618074f90 con 0x7fa604004810 2024-04-17T10:53:31.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.905+0000 7fa60f7fe640 1 -- 172.21.15.83:0/3642547215 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa608015460 con 0x7fa604004810 2024-04-17T10:53:31.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.905+0000 7fa60f7fe640 1 --2- 172.21.15.83:0/3642547215 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa5f8066f60 0x7fa5f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:31.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.905+0000 7fa60f7fe640 1 -- 172.21.15.83:0/3642547215 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fa608085a20 con 0x7fa604004810 2024-04-17T10:53:31.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.905+0000 7fa60dd9b640 1 --2- 172.21.15.83:0/3642547215 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa5f8066f60 0x7fa5f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:31.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.909+0000 7fa60dd9b640 1 --2- 172.21.15.83:0/3642547215 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa5f8066f60 0x7fa5f8069420 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fa6000af000 tx=0x7fa6000b2040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:31.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:31.913+0000 7fa60f7fe640 1 -- 172.21.15.83:0/3642547215 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa608019030 con 0x7fa604004810 2024-04-17T10:53:32.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:32.101+0000 7fa61ca15640 1 -- 172.21.15.83:0/3642547215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa618148730 con 0x7fa604004810 2024-04-17T10:53:32.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:32.101+0000 7fa60f7fe640 1 -- 172.21.15.83:0/3642547215 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fa6080569c0 con 0x7fa604004810 2024-04-17T10:53:32.093 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:32.093 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:32.093 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:32.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:32.101+0000 7fa61ca15640 1 -- 172.21.15.83:0/3642547215 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa5f8066f60 msgr2=0x7fa5f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:32.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:32.101+0000 7fa61ca15640 1 --2- 172.21.15.83:0/3642547215 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa5f8066f60 0x7fa5f8069420 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fa6000af000 tx=0x7fa6000b2040 comp rx=0 tx=0).stop 2024-04-17T10:53:32.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:32.101+0000 7fa61ca15640 1 -- 172.21.15.83:0/3642547215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa604004810 msgr2=0x7fa6180b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:32.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:32.101+0000 7fa61ca15640 1 --2- 172.21.15.83:0/3642547215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa604004810 0x7fa6180b3630 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7fa6080115e0 tx=0x7fa608011aa0 comp rx=0 tx=0).stop 2024-04-17T10:53:32.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:32.105+0000 7fa61ca15640 1 -- 172.21.15.83:0/3642547215 shutdown_connections 2024-04-17T10:53:32.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:32.105+0000 7fa61ca15640 1 --2- 172.21.15.83:0/3642547215 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa5f8066f60 0x7fa5f8069420 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:32.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:32.105+0000 7fa61ca15640 1 --2- 172.21.15.83:0/3642547215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6180c88e0 0x7fa6180c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:32.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:32.105+0000 7fa61ca15640 1 --2- 172.21.15.83:0/3642547215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa618147ae0 0x7fa618147f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:32.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:32.105+0000 7fa61ca15640 1 --2- 172.21.15.83:0/3642547215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa604004810 0x7fa6180b3630 unknown :-1 s=CLOSED pgs=2312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:32.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:32.105+0000 7fa61ca15640 1 -- 172.21.15.83:0/3642547215 >> 172.21.15.83:0/3642547215 conn(0x7fa61806bcf0 msgr2=0x7fa61815a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:32.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:32.105+0000 7fa61ca15640 1 -- 172.21.15.83:0/3642547215 shutdown_connections 2024-04-17T10:53:32.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:32.105+0000 7fa61ca15640 1 -- 172.21.15.83:0/3642547215 wait complete. 2024-04-17T10:53:32.512 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:32.512 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:33.513 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:33.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.653+0000 7f57ea214640 1 Processor -- start 2024-04-17T10:53:33.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.653+0000 7f57ea214640 1 -- start start 2024-04-17T10:53:33.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.657+0000 7f57ea214640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f57e406fb70 0x7f57e40713a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:33.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.657+0000 7f57ea214640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57e40ccbf0 0x7f57e406b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:33.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.657+0000 7f57ea214640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e4071bf0 con 0x7f57e415f550 2024-04-17T10:53:33.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.657+0000 7f57ea214640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57e4071ed0 con 0x7f57e406fb70 2024-04-17T10:53:33.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.657+0000 7f57ea214640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e40721b0 con 0x7f57e40ccbf0 2024-04-17T10:53:33.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.657+0000 7f57e37fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f57e406fb70 0x7f57e40713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:33.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.657+0000 7f57e37fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f57e406fb70 0x7f57e40713a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36656/0 (socket says 172.21.15.83:36656) 2024-04-17T10:53:33.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.657+0000 7f57e37fe640 1 -- 172.21.15.83:0/4021090740 learned_addr learned my addr 172.21.15.83:0/4021090740 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:33.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.657+0000 7f57e2ffd640 1 --2- 172.21.15.83:0/4021090740 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57e40ccbf0 0x7f57e406b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:33.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.657+0000 7f57e37fe640 1 -- 172.21.15.83:0/4021090740 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57e40ccbf0 msgr2=0x7f57e406b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:33.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.657+0000 7f57e37fe640 1 --2- 172.21.15.83:0/4021090740 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57e40ccbf0 0x7f57e406b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:33.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.657+0000 7f57e37fe640 1 -- 172.21.15.83:0/4021090740 >> v1:172.21.15.167:6789/0 conn(0x7f57e415f550 legacy=0x7f57e40c5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:33.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.657+0000 7f57e37fe640 1 -- 172.21.15.83:0/4021090740 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e406abe0 con 0x7f57e406fb70 2024-04-17T10:53:33.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.657+0000 7f57e37fe640 1 --2- 172.21.15.83:0/4021090740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57e406fb70 0x7f57e40713a0 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f57d4009870 tx=0x7f57d401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1512ff92d08023b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:33.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.657+0000 7f57e27fc640 1 -- 172.21.15.83:0/4021090740 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d40b2070 con 0x7f57e406fb70 2024-04-17T10:53:33.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.657+0000 7f57e27fc640 1 -- 172.21.15.83:0/4021090740 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d40a9070 con 0x7f57e406fb70 2024-04-17T10:53:33.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.657+0000 7f57e27fc640 1 -- 172.21.15.83:0/4021090740 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d40ad880 con 0x7f57e406fb70 2024-04-17T10:53:33.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.657+0000 7f57ea214640 1 -- 172.21.15.83:0/4021090740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57e406fb70 msgr2=0x7f57e40713a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:33.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.657+0000 7f57ea214640 1 --2- 172.21.15.83:0/4021090740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57e406fb70 0x7f57e40713a0 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f57d4009870 tx=0x7f57d401c560 comp rx=0 tx=0).stop 2024-04-17T10:53:33.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.657+0000 7f57ea214640 1 -- 172.21.15.83:0/4021090740 shutdown_connections 2024-04-17T10:53:33.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.657+0000 7f57ea214640 1 --2- 172.21.15.83:0/4021090740 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57e40ccbf0 0x7f57e406b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:33.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.657+0000 7f57ea214640 1 --2- 172.21.15.83:0/4021090740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57e406fb70 0x7f57e40713a0 unknown :-1 s=CLOSED pgs=2313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:33.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.657+0000 7f57ea214640 1 -- 172.21.15.83:0/4021090740 >> 172.21.15.83:0/4021090740 conn(0x7f57e406bcf0 msgr2=0x7f57e406c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:33.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.661+0000 7f57ea214640 1 -- 172.21.15.83:0/4021090740 shutdown_connections 2024-04-17T10:53:33.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.661+0000 7f57ea214640 1 -- 172.21.15.83:0/4021090740 wait complete. 2024-04-17T10:53:33.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.661+0000 7f57ea214640 1 Processor -- start 2024-04-17T10:53:33.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.661+0000 7f57ea214640 1 -- start start 2024-04-17T10:53:33.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.661+0000 7f57ea214640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57e406fb70 0x7f57e40c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:33.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.661+0000 7f57ea214640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57e40ccbf0 0x7f57e4151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:33.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.661+0000 7f57ea214640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57e415f550 0x7f57e4151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:33.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.661+0000 7f57ea214640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e406baa0 con 0x7f57e40ccbf0 2024-04-17T10:53:33.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.661+0000 7f57ea214640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e406b920 con 0x7f57e415f550 2024-04-17T10:53:33.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.661+0000 7f57ea214640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e4169100 con 0x7f57e406fb70 2024-04-17T10:53:33.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.661+0000 7f57e2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57e40ccbf0 0x7f57e4151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:33.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.661+0000 7f57e2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57e40ccbf0 0x7f57e4151ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36672/0 (socket says 172.21.15.83:36672) 2024-04-17T10:53:33.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.661+0000 7f57e2ffd640 1 -- 172.21.15.83:0/323718775 learned_addr learned my addr 172.21.15.83:0/323718775 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:33.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.661+0000 7f57e3fff640 1 --2- 172.21.15.83:0/323718775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57e415f550 0x7f57e4151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.661+0000 7f57e2ffd640 1 -- 172.21.15.83:0/323718775 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57e406fb70 msgr2=0x7f57e40c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.661+0000 7f57e2ffd640 1 --2- 172.21.15.83:0/323718775 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57e406fb70 0x7f57e40c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.661+0000 7f57e2ffd640 1 -- 172.21.15.83:0/323718775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57e415f550 msgr2=0x7f57e4151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.661+0000 7f57e2ffd640 1 --2- 172.21.15.83:0/323718775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57e415f550 0x7f57e4151ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.661+0000 7f57e2ffd640 1 -- 172.21.15.83:0/323718775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e4156a90 con 0x7f57e40ccbf0 2024-04-17T10:53:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.661+0000 7f57e2ffd640 1 --2- 172.21.15.83:0/323718775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57e40ccbf0 0x7f57e4151ab0 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f57d800d800 tx=0x7f57d800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.661+0000 7f57cbfff640 1 -- 172.21.15.83:0/323718775 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d800b6c0 con 0x7f57e40ccbf0 2024-04-17T10:53:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.661+0000 7f57cbfff640 1 -- 172.21.15.83:0/323718775 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d8019070 con 0x7f57e40ccbf0 2024-04-17T10:53:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.661+0000 7f57ea214640 1 -- 172.21.15.83:0/323718775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e4158ba0 con 0x7f57e40ccbf0 2024-04-17T10:53:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.661+0000 7f57cbfff640 1 -- 172.21.15.83:0/323718775 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d800bae0 con 0x7f57e40ccbf0 2024-04-17T10:53:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.661+0000 7f57ea214640 1 -- 172.21.15.83:0/323718775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e4159160 con 0x7f57e40ccbf0 2024-04-17T10:53:33.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.665+0000 7f57ea214640 1 -- 172.21.15.83:0/323718775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57e4074f90 con 0x7f57e40ccbf0 2024-04-17T10:53:33.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.665+0000 7f57cbfff640 1 -- 172.21.15.83:0/323718775 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f57d800bc80 con 0x7f57e40ccbf0 2024-04-17T10:53:33.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.665+0000 7f57cbfff640 1 --2- 172.21.15.83:0/323718775 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f57d0066f60 0x7f57d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:33.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.665+0000 7f57e37fe640 1 --2- 172.21.15.83:0/323718775 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f57d0066f60 0x7f57d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:33.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.665+0000 7f57cbfff640 1 -- 172.21.15.83:0/323718775 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f57d8085590 con 0x7f57e40ccbf0 2024-04-17T10:53:33.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.665+0000 7f57e37fe640 1 --2- 172.21.15.83:0/323718775 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f57d0066f60 0x7f57d0069420 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f57d40af000 tx=0x7f57d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:33.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.669+0000 7f57cbfff640 1 -- 172.21.15.83:0/323718775 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f57e4156a90 con 0x7f57e40ccbf0 2024-04-17T10:53:33.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.861+0000 7f57ea214640 1 -- 172.21.15.83:0/323718775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57e406c9c0 con 0x7f57e40ccbf0 2024-04-17T10:53:33.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.861+0000 7f57cbfff640 1 -- 172.21.15.83:0/323718775 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f57d8056530 con 0x7f57e40ccbf0 2024-04-17T10:53:33.855 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:33.855 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:33.855 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:33.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.865+0000 7f57ea214640 1 -- 172.21.15.83:0/323718775 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f57d0066f60 msgr2=0x7f57d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:33.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.865+0000 7f57ea214640 1 --2- 172.21.15.83:0/323718775 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f57d0066f60 0x7f57d0069420 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f57d40af000 tx=0x7f57d4008040 comp rx=0 tx=0).stop 2024-04-17T10:53:33.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.865+0000 7f57ea214640 1 -- 172.21.15.83:0/323718775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57e40ccbf0 msgr2=0x7f57e4151ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:33.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.865+0000 7f57ea214640 1 --2- 172.21.15.83:0/323718775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57e40ccbf0 0x7f57e4151ab0 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f57d800d800 tx=0x7f57d800dcc0 comp rx=0 tx=0).stop 2024-04-17T10:53:33.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.865+0000 7f57ea214640 1 -- 172.21.15.83:0/323718775 shutdown_connections 2024-04-17T10:53:33.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.865+0000 7f57ea214640 1 --2- 172.21.15.83:0/323718775 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f57d0066f60 0x7f57d0069420 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:33.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.865+0000 7f57ea214640 1 --2- 172.21.15.83:0/323718775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57e415f550 0x7f57e4151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:33.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.865+0000 7f57ea214640 1 --2- 172.21.15.83:0/323718775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57e40ccbf0 0x7f57e4151ab0 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:33.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.865+0000 7f57ea214640 1 --2- 172.21.15.83:0/323718775 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57e406fb70 0x7f57e40c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:33.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.865+0000 7f57ea214640 1 -- 172.21.15.83:0/323718775 >> 172.21.15.83:0/323718775 conn(0x7f57e406bcf0 msgr2=0x7f57e415c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:33.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.865+0000 7f57ea214640 1 -- 172.21.15.83:0/323718775 shutdown_connections 2024-04-17T10:53:33.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:33.865+0000 7f57ea214640 1 -- 172.21.15.83:0/323718775 wait complete. 2024-04-17T10:53:34.276 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:34.276 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:35.277 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:35.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.421+0000 7f61b47d9640 1 Processor -- start 2024-04-17T10:53:35.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.421+0000 7f61b47d9640 1 -- start start 2024-04-17T10:53:35.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.421+0000 7f61b47d9640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f61ac0cfbe0 0x7f61ac06c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:35.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.421+0000 7f61b47d9640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61ac06e320 con 0x7f61ac15ba30 2024-04-17T10:53:35.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.421+0000 7f61b47d9640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61ac06e040 con 0x7f61ac072330 2024-04-17T10:53:35.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.421+0000 7f61b47d9640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61ac071ea0 con 0x7f61ac0cfbe0 2024-04-17T10:53:35.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.421+0000 7f61b2575640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f61ac15ba30 0x7f61ac15c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42644/0 (socket says 172.21.15.83:42644) 2024-04-17T10:53:35.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.421+0000 7f61b2575640 1 -- 172.21.15.83:0/3351326255 learned_addr learned my addr 172.21.15.83:0/3351326255 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:53:35.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.421+0000 7f61b2d76640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f61ac0cfbe0 0x7f61ac06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:35.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.421+0000 7f61b2d76640 1 -- 172.21.15.83:0/3351326255 >> v1:172.21.15.167:6789/0 conn(0x7f61ac072330 legacy=0x7f61ac0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:35.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.421+0000 7f61b2d76640 1 -- 172.21.15.83:0/3351326255 >> v1:172.21.15.83:6789/0 conn(0x7f61ac15ba30 legacy=0x7f61ac15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:35.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.421+0000 7f61b1573640 1 -- 172.21.15.83:0/3351326255 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 751252965 0 0) 0x7f61ac06e320 con 0x7f61ac15ba30 2024-04-17T10:53:35.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.421+0000 7f61b2d76640 1 -- 172.21.15.83:0/3351326255 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ac071850 con 0x7f61ac0cfbe0 2024-04-17T10:53:35.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.421+0000 7f61b2d76640 1 --2- 172.21.15.83:0/3351326255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ac0cfbe0 0x7f61ac06c490 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f61a4009d90 tx=0x7f61a401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8a4862ad05b0b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:35.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b1573640 1 -- 172.21.15.83:0/3351326255 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a40b2070 con 0x7f61ac0cfbe0 2024-04-17T10:53:35.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b1573640 1 -- 172.21.15.83:0/3351326255 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a40a5870 con 0x7f61ac0cfbe0 2024-04-17T10:53:35.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b1573640 1 -- 172.21.15.83:0/3351326255 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a40a5c40 con 0x7f61ac0cfbe0 2024-04-17T10:53:35.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b47d9640 1 -- 172.21.15.83:0/3351326255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ac0cfbe0 msgr2=0x7f61ac06c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:35.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b47d9640 1 --2- 172.21.15.83:0/3351326255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ac0cfbe0 0x7f61ac06c490 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f61a4009d90 tx=0x7f61a401c8a0 comp rx=0 tx=0).stop 2024-04-17T10:53:35.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b47d9640 1 -- 172.21.15.83:0/3351326255 shutdown_connections 2024-04-17T10:53:35.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b47d9640 1 --2- 172.21.15.83:0/3351326255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ac0cfbe0 0x7f61ac06c490 unknown :-1 s=CLOSED pgs=2316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:35.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b47d9640 1 -- 172.21.15.83:0/3351326255 >> 172.21.15.83:0/3351326255 conn(0x7f61ac06abe0 msgr2=0x7f61ac06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:35.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b47d9640 1 -- 172.21.15.83:0/3351326255 shutdown_connections 2024-04-17T10:53:35.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b47d9640 1 -- 172.21.15.83:0/3351326255 wait complete. 2024-04-17T10:53:35.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b47d9640 1 Processor -- start 2024-04-17T10:53:35.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b47d9640 1 -- start start 2024-04-17T10:53:35.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b47d9640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61ac072330 0x7f61ac154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:35.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b47d9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ac0cfbe0 0x7f61ac14b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:35.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b47d9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61ac15ba30 0x7f61ac146b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:35.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b47d9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac1696b0 con 0x7f61ac0cfbe0 2024-04-17T10:53:35.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b47d9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac169530 con 0x7f61ac15ba30 2024-04-17T10:53:35.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b47d9640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac169830 con 0x7f61ac072330 2024-04-17T10:53:35.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b1d74640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ac0cfbe0 0x7f61ac14b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:35.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b1d74640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ac0cfbe0 0x7f61ac14b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36692/0 (socket says 172.21.15.83:36692) 2024-04-17T10:53:35.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b1d74640 1 -- 172.21.15.83:0/2531321541 learned_addr learned my addr 172.21.15.83:0/2531321541 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:35.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b2575640 1 --2- 172.21.15.83:0/2531321541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61ac072330 0x7f61ac154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:35.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b2d76640 1 --2- 172.21.15.83:0/2531321541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61ac15ba30 0x7f61ac146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:35.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b1d74640 1 -- 172.21.15.83:0/2531321541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61ac072330 msgr2=0x7f61ac154c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:35.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b1d74640 1 --2- 172.21.15.83:0/2531321541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61ac072330 0x7f61ac154c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:35.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b1d74640 1 -- 172.21.15.83:0/2531321541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61ac15ba30 msgr2=0x7f61ac146b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:35.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b1d74640 1 --2- 172.21.15.83:0/2531321541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61ac15ba30 0x7f61ac146b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:35.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b1d74640 1 -- 172.21.15.83:0/2531321541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ac151e10 con 0x7f61ac0cfbe0 2024-04-17T10:53:35.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b2575640 1 --2- 172.21.15.83:0/2531321541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61ac072330 0x7f61ac154c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:53:35.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b2d76640 1 --2- 172.21.15.83:0/2531321541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61ac15ba30 0x7f61ac146b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:53:35.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b1d74640 1 --2- 172.21.15.83:0/2531321541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ac0cfbe0 0x7f61ac14b6f0 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f619c00b410 tx=0x7f619c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:35.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61a2ffd640 1 -- 172.21.15.83:0/2531321541 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c015030 con 0x7f61ac0cfbe0 2024-04-17T10:53:35.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61a2ffd640 1 -- 172.21.15.83:0/2531321541 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619c004020 con 0x7f61ac0cfbe0 2024-04-17T10:53:35.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b47d9640 1 -- 172.21.15.83:0/2531321541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61ac1520f0 con 0x7f61ac0cfbe0 2024-04-17T10:53:35.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61a2ffd640 1 -- 172.21.15.83:0/2531321541 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c004450 con 0x7f61ac0cfbe0 2024-04-17T10:53:35.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.425+0000 7f61b47d9640 1 -- 172.21.15.83:0/2531321541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61ac1492e0 con 0x7f61ac0cfbe0 2024-04-17T10:53:35.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.429+0000 7f61b47d9640 1 -- 172.21.15.83:0/2531321541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61ac074f90 con 0x7f61ac0cfbe0 2024-04-17T10:53:35.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.429+0000 7f61a2ffd640 1 -- 172.21.15.83:0/2531321541 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f619c018070 con 0x7f61ac0cfbe0 2024-04-17T10:53:35.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.429+0000 7f61a2ffd640 1 --2- 172.21.15.83:0/2531321541 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6190066f60 0x7f6190069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:35.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.429+0000 7f61b2575640 1 --2- 172.21.15.83:0/2531321541 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6190066f60 0x7f6190069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:35.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.429+0000 7f61a2ffd640 1 -- 172.21.15.83:0/2531321541 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f619c084cd0 con 0x7f61ac0cfbe0 2024-04-17T10:53:35.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.429+0000 7f61b2575640 1 --2- 172.21.15.83:0/2531321541 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6190066f60 0x7f6190069420 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f61a8000f70 tx=0x7f61a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:35.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.433+0000 7f61a2ffd640 1 -- 172.21.15.83:0/2531321541 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f619c08b080 con 0x7f61ac0cfbe0 2024-04-17T10:53:35.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.625+0000 7f61b47d9640 1 -- 172.21.15.83:0/2531321541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61ac0af100 con 0x7f61ac0cfbe0 2024-04-17T10:53:35.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.629+0000 7f61a2ffd640 1 -- 172.21.15.83:0/2531321541 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f619c055bf0 con 0x7f61ac0cfbe0 2024-04-17T10:53:35.619 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:35.619 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:35.620 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:35.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.629+0000 7f61b47d9640 1 -- 172.21.15.83:0/2531321541 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6190066f60 msgr2=0x7f6190069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:35.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.629+0000 7f61b47d9640 1 --2- 172.21.15.83:0/2531321541 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6190066f60 0x7f6190069420 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f61a8000f70 tx=0x7f61a8009040 comp rx=0 tx=0).stop 2024-04-17T10:53:35.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.629+0000 7f61b47d9640 1 -- 172.21.15.83:0/2531321541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ac0cfbe0 msgr2=0x7f61ac14b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:35.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.629+0000 7f61b47d9640 1 --2- 172.21.15.83:0/2531321541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ac0cfbe0 0x7f61ac14b6f0 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f619c00b410 tx=0x7f619c00b8d0 comp rx=0 tx=0).stop 2024-04-17T10:53:35.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.629+0000 7f61b47d9640 1 -- 172.21.15.83:0/2531321541 shutdown_connections 2024-04-17T10:53:35.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.629+0000 7f61b47d9640 1 --2- 172.21.15.83:0/2531321541 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6190066f60 0x7f6190069420 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:35.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.629+0000 7f61b47d9640 1 --2- 172.21.15.83:0/2531321541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61ac15ba30 0x7f61ac146b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:35.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.629+0000 7f61b47d9640 1 --2- 172.21.15.83:0/2531321541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ac0cfbe0 0x7f61ac14b6f0 unknown :-1 s=CLOSED pgs=2317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:35.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.629+0000 7f61b47d9640 1 --2- 172.21.15.83:0/2531321541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61ac072330 0x7f61ac154c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:35.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.629+0000 7f61b47d9640 1 -- 172.21.15.83:0/2531321541 >> 172.21.15.83:0/2531321541 conn(0x7f61ac06abe0 msgr2=0x7f61ac0af600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:35.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.629+0000 7f61b47d9640 1 -- 172.21.15.83:0/2531321541 shutdown_connections 2024-04-17T10:53:35.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:35.629+0000 7f61b47d9640 1 -- 172.21.15.83:0/2531321541 wait complete. 2024-04-17T10:53:36.041 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:36.042 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:37.043 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:37.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.185+0000 7f9e02c74640 1 Processor -- start 2024-04-17T10:53:37.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.185+0000 7f9e02c74640 1 -- start start 2024-04-17T10:53:37.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.185+0000 7f9e02c74640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9dfc06b2f0 0x7f9dfc15f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:37.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.185+0000 7f9e02c74640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dfc15c0b0 con 0x7f9dfc159670 2024-04-17T10:53:37.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.185+0000 7f9e02c74640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dfc15c390 con 0x7f9dfc0b37f0 2024-04-17T10:53:37.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.185+0000 7f9e02c74640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dfc0cd0c0 con 0x7f9dfc06b2f0 2024-04-17T10:53:37.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.185+0000 7f9e01211640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9dfc06b2f0 0x7f9dfc15f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:37.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.185+0000 7f9e00a10640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9dfc159670 0x7f9dfc0af6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55386/0 (socket says 172.21.15.83:55386) 2024-04-17T10:53:37.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.185+0000 7f9e00a10640 1 -- 172.21.15.83:0/2613045349 learned_addr learned my addr 172.21.15.83:0/2613045349 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:53:37.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.185+0000 7f9e01211640 1 -- 172.21.15.83:0/2613045349 >> v1:172.21.15.167:6789/0 conn(0x7f9dfc0b37f0 legacy=0x7f9dfc06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:37.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.185+0000 7f9e01211640 1 -- 172.21.15.83:0/2613045349 >> v1:172.21.15.83:6789/0 conn(0x7f9dfc159670 legacy=0x7f9dfc0af6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:37.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.185+0000 7f9e01211640 1 -- 172.21.15.83:0/2613045349 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dfc15c670 con 0x7f9dfc06b2f0 2024-04-17T10:53:37.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.185+0000 7f9dfb7fe640 1 -- 172.21.15.83:0/2613045349 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1449475362 0 0) 0x7f9dfc15c0b0 con 0x7f9dfc159670 2024-04-17T10:53:37.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9e01211640 1 --2- 172.21.15.83:0/2613045349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dfc06b2f0 0x7f9dfc15f600 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f9df4009d90 tx=0x7f9df401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b61f32b89c1f800 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:37.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9dfb7fe640 1 -- 172.21.15.83:0/2613045349 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df40b2070 con 0x7f9dfc06b2f0 2024-04-17T10:53:37.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9dfb7fe640 1 -- 172.21.15.83:0/2613045349 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df40a57a0 con 0x7f9dfc06b2f0 2024-04-17T10:53:37.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9dfb7fe640 1 -- 172.21.15.83:0/2613045349 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df40a5b70 con 0x7f9dfc06b2f0 2024-04-17T10:53:37.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9e02c74640 1 -- 172.21.15.83:0/2613045349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dfc06b2f0 msgr2=0x7f9dfc15f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:37.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9e02c74640 1 --2- 172.21.15.83:0/2613045349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dfc06b2f0 0x7f9dfc15f600 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f9df4009d90 tx=0x7f9df401c8e0 comp rx=0 tx=0).stop 2024-04-17T10:53:37.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9e02c74640 1 -- 172.21.15.83:0/2613045349 shutdown_connections 2024-04-17T10:53:37.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9e02c74640 1 --2- 172.21.15.83:0/2613045349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dfc06b2f0 0x7f9dfc15f600 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:37.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9e02c74640 1 -- 172.21.15.83:0/2613045349 >> 172.21.15.83:0/2613045349 conn(0x7f9dfc06bcf0 msgr2=0x7f9dfc06c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:37.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9e02c74640 1 -- 172.21.15.83:0/2613045349 shutdown_connections 2024-04-17T10:53:37.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9e02c74640 1 -- 172.21.15.83:0/2613045349 wait complete. 2024-04-17T10:53:37.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9e02c74640 1 Processor -- start 2024-04-17T10:53:37.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9e02c74640 1 -- start start 2024-04-17T10:53:37.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9e02c74640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dfc06b2f0 0x7f9dfc1502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:37.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9e02c74640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dfc0b37f0 0x7f9dfc14a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:37.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9e02c74640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dfc159670 0x7f9dfc14aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:37.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9e02c74640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc169ac0 con 0x7f9dfc159670 2024-04-17T10:53:37.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9e02c74640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc0affc0 con 0x7f9dfc06b2f0 2024-04-17T10:53:37.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9e02c74640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc169c40 con 0x7f9dfc0b37f0 2024-04-17T10:53:37.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9e01211640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dfc159670 0x7f9dfc14aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:37.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9e01211640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dfc159670 0x7f9dfc14aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52146/0 (socket says 172.21.15.83:52146) 2024-04-17T10:53:37.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9e01211640 1 -- 172.21.15.83:0/3236898278 learned_addr learned my addr 172.21.15.83:0/3236898278 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:37.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9dfbfff640 1 --2- 172.21.15.83:0/3236898278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dfc0b37f0 0x7f9dfc14a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:37.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9e00a10640 1 --2- 172.21.15.83:0/3236898278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dfc06b2f0 0x7f9dfc1502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:37.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9e01211640 1 -- 172.21.15.83:0/3236898278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dfc0b37f0 msgr2=0x7f9dfc14a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:37.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9e01211640 1 --2- 172.21.15.83:0/3236898278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dfc0b37f0 0x7f9dfc14a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:37.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9e01211640 1 -- 172.21.15.83:0/3236898278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dfc06b2f0 msgr2=0x7f9dfc1502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:37.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9e01211640 1 --2- 172.21.15.83:0/3236898278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dfc06b2f0 0x7f9dfc1502c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:37.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9e01211640 1 -- 172.21.15.83:0/3236898278 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dfc151570 con 0x7f9dfc159670 2024-04-17T10:53:37.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9e01211640 1 --2- 172.21.15.83:0/3236898278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dfc159670 0x7f9dfc14aa90 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f9df4009d90 tx=0x7f9df4019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:37.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9df97fa640 1 -- 172.21.15.83:0/3236898278 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df40bd020 con 0x7f9dfc159670 2024-04-17T10:53:37.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9df97fa640 1 -- 172.21.15.83:0/3236898278 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df40b8670 con 0x7f9dfc159670 2024-04-17T10:53:37.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9df97fa640 1 -- 172.21.15.83:0/3236898278 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df40b2040 con 0x7f9dfc159670 2024-04-17T10:53:37.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9e02c74640 1 -- 172.21.15.83:0/3236898278 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dfc14f7e0 con 0x7f9dfc159670 2024-04-17T10:53:37.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.189+0000 7f9e02c74640 1 -- 172.21.15.83:0/3236898278 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dfc14d190 con 0x7f9dfc159670 2024-04-17T10:53:37.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.193+0000 7f9e02c74640 1 -- 172.21.15.83:0/3236898278 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dfc074f90 con 0x7f9dfc159670 2024-04-17T10:53:37.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.193+0000 7f9df97fa640 1 -- 172.21.15.83:0/3236898278 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9df40c0020 con 0x7f9dfc159670 2024-04-17T10:53:37.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.193+0000 7f9df97fa640 1 --2- 172.21.15.83:0/3236898278 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9de0066f60 0x7f9de0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:37.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.193+0000 7f9e00a10640 1 --2- 172.21.15.83:0/3236898278 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9de0066f60 0x7f9de0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:37.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.193+0000 7f9df97fa640 1 -- 172.21.15.83:0/3236898278 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f9df411ffb0 con 0x7f9dfc159670 2024-04-17T10:53:37.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.193+0000 7f9e00a10640 1 --2- 172.21.15.83:0/3236898278 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9de0066f60 0x7f9de0069420 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f9df0000f70 tx=0x7f9df0007bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:37.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.197+0000 7f9df97fa640 1 -- 172.21.15.83:0/3236898278 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9df40c7030 con 0x7f9dfc159670 2024-04-17T10:53:37.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.393+0000 7f9e02c74640 1 -- 172.21.15.83:0/3236898278 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9dfc068150 con 0x7f9dfc159670 2024-04-17T10:53:37.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.393+0000 7f9df97fa640 1 -- 172.21.15.83:0/3236898278 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f9df4016560 con 0x7f9dfc159670 2024-04-17T10:53:37.385 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:37.385 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:37.385 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:37.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.397+0000 7f9e02c74640 1 -- 172.21.15.83:0/3236898278 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9de0066f60 msgr2=0x7f9de0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:37.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.397+0000 7f9e02c74640 1 --2- 172.21.15.83:0/3236898278 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9de0066f60 0x7f9de0069420 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f9df0000f70 tx=0x7f9df0007bc0 comp rx=0 tx=0).stop 2024-04-17T10:53:37.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.397+0000 7f9e02c74640 1 -- 172.21.15.83:0/3236898278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dfc159670 msgr2=0x7f9dfc14aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:37.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.397+0000 7f9e02c74640 1 --2- 172.21.15.83:0/3236898278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dfc159670 0x7f9dfc14aa90 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f9df4009d90 tx=0x7f9df4019680 comp rx=0 tx=0).stop 2024-04-17T10:53:37.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.397+0000 7f9e02c74640 1 -- 172.21.15.83:0/3236898278 shutdown_connections 2024-04-17T10:53:37.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.397+0000 7f9e02c74640 1 --2- 172.21.15.83:0/3236898278 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9de0066f60 0x7f9de0069420 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:37.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.397+0000 7f9e02c74640 1 --2- 172.21.15.83:0/3236898278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dfc159670 0x7f9dfc14aa90 unknown :-1 s=CLOSED pgs=2320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:37.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.397+0000 7f9e02c74640 1 --2- 172.21.15.83:0/3236898278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dfc0b37f0 0x7f9dfc14a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:37.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.397+0000 7f9e02c74640 1 --2- 172.21.15.83:0/3236898278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dfc06b2f0 0x7f9dfc1502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:37.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.397+0000 7f9e02c74640 1 -- 172.21.15.83:0/3236898278 >> 172.21.15.83:0/3236898278 conn(0x7f9dfc06bcf0 msgr2=0x7f9dfc15db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:37.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.397+0000 7f9e02c74640 1 -- 172.21.15.83:0/3236898278 shutdown_connections 2024-04-17T10:53:37.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:37.397+0000 7f9e02c74640 1 -- 172.21.15.83:0/3236898278 wait complete. 2024-04-17T10:53:37.805 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:37.805 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:38.806 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:38.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.945+0000 7f13790c0640 1 Processor -- start 2024-04-17T10:53:38.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.945+0000 7f13790c0640 1 -- start start 2024-04-17T10:53:38.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.949+0000 7f13790c0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13740aedd0 0x7f137406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:38.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.949+0000 7f13790c0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f137406dae0 0x7f13740b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:38.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.949+0000 7f13790c0640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f137415e440 con 0x7f137415db50 2024-04-17T10:53:38.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.949+0000 7f13790c0640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f137415e720 con 0x7f137406dae0 2024-04-17T10:53:38.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.949+0000 7f13790c0640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137415ea00 con 0x7f13740aedd0 2024-04-17T10:53:38.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.949+0000 7f137359e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f137406dae0 0x7f13740b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:38.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.949+0000 7f137359e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f137406dae0 0x7f13740b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52158/0 (socket says 172.21.15.83:52158) 2024-04-17T10:53:38.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.949+0000 7f137359e640 1 -- 172.21.15.83:0/786514288 learned_addr learned my addr 172.21.15.83:0/786514288 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:38.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.949+0000 7f136a59c640 1 --2- 172.21.15.83:0/786514288 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13740aedd0 0x7f137406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:38.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.949+0000 7f137359e640 1 -- 172.21.15.83:0/786514288 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13740aedd0 msgr2=0x7f137406d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:38.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.949+0000 7f137359e640 1 --2- 172.21.15.83:0/786514288 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13740aedd0 0x7f137406d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:38.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.949+0000 7f137359e640 1 -- 172.21.15.83:0/786514288 >> v1:172.21.15.167:6789/0 conn(0x7f137415db50 legacy=0x7f1374070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:38.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.949+0000 7f137359e640 1 -- 172.21.15.83:0/786514288 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137415ece0 con 0x7f137406dae0 2024-04-17T10:53:38.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.949+0000 7f136a59c640 1 --2- 172.21.15.83:0/786514288 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13740aedd0 0x7f137406d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:53:38.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.949+0000 7f137359e640 1 --2- 172.21.15.83:0/786514288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f137406dae0 0x7f13740b2e10 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f136c009e50 tx=0x7f136c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd9b7211da57682 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:38.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.949+0000 7f137259c640 1 -- 172.21.15.83:0/786514288 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c0a5670 con 0x7f137406dae0 2024-04-17T10:53:38.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.949+0000 7f137259c640 1 -- 172.21.15.83:0/786514288 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f136c0a5810 con 0x7f137406dae0 2024-04-17T10:53:38.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.949+0000 7f137259c640 1 -- 172.21.15.83:0/786514288 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c0a5be0 con 0x7f137406dae0 2024-04-17T10:53:38.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.953+0000 7f13790c0640 1 -- 172.21.15.83:0/786514288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f137406dae0 msgr2=0x7f13740b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:38.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.953+0000 7f13790c0640 1 --2- 172.21.15.83:0/786514288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f137406dae0 0x7f13740b2e10 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f136c009e50 tx=0x7f136c01c9f0 comp rx=0 tx=0).stop 2024-04-17T10:53:38.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.953+0000 7f13790c0640 1 -- 172.21.15.83:0/786514288 shutdown_connections 2024-04-17T10:53:38.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.953+0000 7f13790c0640 1 --2- 172.21.15.83:0/786514288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f137406dae0 0x7f13740b2e10 unknown :-1 s=CLOSED pgs=2321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:38.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.953+0000 7f13790c0640 1 --2- 172.21.15.83:0/786514288 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13740aedd0 0x7f137406d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:38.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.953+0000 7f13790c0640 1 -- 172.21.15.83:0/786514288 >> 172.21.15.83:0/786514288 conn(0x7f137406abe0 msgr2=0x7f137406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:38.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.953+0000 7f13790c0640 1 -- 172.21.15.83:0/786514288 shutdown_connections 2024-04-17T10:53:38.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.953+0000 7f13790c0640 1 -- 172.21.15.83:0/786514288 wait complete. 2024-04-17T10:53:38.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.953+0000 7f13790c0640 1 Processor -- start 2024-04-17T10:53:38.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.953+0000 7f13790c0640 1 -- start start 2024-04-17T10:53:38.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.953+0000 7f13790c0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f137406dae0 0x7f13741f3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:38.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.953+0000 7f13790c0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13740aedd0 0x7f1374154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:38.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.953+0000 7f13790c0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f137415db50 0x7f13740a9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:38.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.953+0000 7f13790c0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13740b3ef0 con 0x7f137415db50 2024-04-17T10:53:38.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.953+0000 7f13790c0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13740b3d70 con 0x7f13740aedd0 2024-04-17T10:53:38.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.953+0000 7f13790c0640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13740b4070 con 0x7f137406dae0 2024-04-17T10:53:38.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.953+0000 7f137359e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f137415db50 0x7f13740a9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:38.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.953+0000 7f137359e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f137415db50 0x7f13740a9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52166/0 (socket says 172.21.15.83:52166) 2024-04-17T10:53:38.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.953+0000 7f137359e640 1 -- 172.21.15.83:0/418306752 learned_addr learned my addr 172.21.15.83:0/418306752 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:38.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.953+0000 7f1372d9d640 1 --2- 172.21.15.83:0/418306752 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f137406dae0 0x7f13741f3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:38.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.953+0000 7f136a59c640 1 --2- 172.21.15.83:0/418306752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13740aedd0 0x7f1374154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:38.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.953+0000 7f137359e640 1 -- 172.21.15.83:0/418306752 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f137406dae0 msgr2=0x7f13741f3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:38.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.953+0000 7f137359e640 1 --2- 172.21.15.83:0/418306752 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f137406dae0 0x7f13741f3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:38.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.953+0000 7f137359e640 1 -- 172.21.15.83:0/418306752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13740aedd0 msgr2=0x7f1374154690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:38.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.953+0000 7f137359e640 1 --2- 172.21.15.83:0/418306752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13740aedd0 0x7f1374154690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:38.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.953+0000 7f137359e640 1 -- 172.21.15.83:0/418306752 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1374157fa0 con 0x7f137415db50 2024-04-17T10:53:38.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.953+0000 7f1372d9d640 1 --2- 172.21.15.83:0/418306752 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f137406dae0 0x7f13741f3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:53:38.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.953+0000 7f137359e640 1 --2- 172.21.15.83:0/418306752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f137415db50 0x7f13740a9e40 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f136c0af040 tx=0x7f136c019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:38.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.957+0000 7f136bfff640 1 -- 172.21.15.83:0/418306752 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c0bc020 con 0x7f137415db50 2024-04-17T10:53:38.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.957+0000 7f136bfff640 1 -- 172.21.15.83:0/418306752 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f136c0b8670 con 0x7f137415db50 2024-04-17T10:53:38.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.957+0000 7f13790c0640 1 -- 172.21.15.83:0/418306752 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13740aaa90 con 0x7f137415db50 2024-04-17T10:53:38.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.957+0000 7f136bfff640 1 -- 172.21.15.83:0/418306752 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c0b8a40 con 0x7f137415db50 2024-04-17T10:53:38.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.957+0000 7f13790c0640 1 -- 172.21.15.83:0/418306752 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1374158520 con 0x7f137415db50 2024-04-17T10:53:38.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.957+0000 7f13790c0640 1 -- 172.21.15.83:0/418306752 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1374074f90 con 0x7f137415db50 2024-04-17T10:53:38.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.957+0000 7f136bfff640 1 -- 172.21.15.83:0/418306752 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f136c0bf020 con 0x7f137415db50 2024-04-17T10:53:38.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.957+0000 7f136bfff640 1 --2- 172.21.15.83:0/418306752 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1354066f60 0x7f1354069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:38.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.957+0000 7f1372d9d640 1 --2- 172.21.15.83:0/418306752 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1354066f60 0x7f1354069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:38.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.957+0000 7f136bfff640 1 -- 172.21.15.83:0/418306752 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f136c120130 con 0x7f137415db50 2024-04-17T10:53:38.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.957+0000 7f1372d9d640 1 --2- 172.21.15.83:0/418306752 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1354066f60 0x7f1354069420 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f13640098b0 tx=0x7f1364008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:38.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:38.961+0000 7f136bfff640 1 -- 172.21.15.83:0/418306752 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f136c0c6030 con 0x7f137415db50 2024-04-17T10:53:39.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:39.153+0000 7f13790c0640 1 -- 172.21.15.83:0/418306752 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13740b2af0 con 0x7f137415db50 2024-04-17T10:53:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:39.153+0000 7f136bfff640 1 -- 172.21.15.83:0/418306752 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f136c0165e0 con 0x7f137415db50 2024-04-17T10:53:39.146 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:39.146 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:39.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:39.157+0000 7f13790c0640 1 -- 172.21.15.83:0/418306752 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1354066f60 msgr2=0x7f1354069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:39.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:39.157+0000 7f13790c0640 1 --2- 172.21.15.83:0/418306752 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1354066f60 0x7f1354069420 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f13640098b0 tx=0x7f1364008040 comp rx=0 tx=0).stop 2024-04-17T10:53:39.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:39.157+0000 7f13790c0640 1 -- 172.21.15.83:0/418306752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f137415db50 msgr2=0x7f13740a9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:39.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:39.157+0000 7f13790c0640 1 --2- 172.21.15.83:0/418306752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f137415db50 0x7f13740a9e40 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f136c0af040 tx=0x7f136c019790 comp rx=0 tx=0).stop 2024-04-17T10:53:39.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:39.157+0000 7f13790c0640 1 -- 172.21.15.83:0/418306752 shutdown_connections 2024-04-17T10:53:39.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:39.157+0000 7f13790c0640 1 --2- 172.21.15.83:0/418306752 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1354066f60 0x7f1354069420 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:39.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:39.157+0000 7f13790c0640 1 --2- 172.21.15.83:0/418306752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f137415db50 0x7f13740a9e40 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:39.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:39.157+0000 7f13790c0640 1 --2- 172.21.15.83:0/418306752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13740aedd0 0x7f1374154690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:39.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:39.157+0000 7f13790c0640 1 --2- 172.21.15.83:0/418306752 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f137406dae0 0x7f13741f3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:39.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:39.157+0000 7f13790c0640 1 -- 172.21.15.83:0/418306752 >> 172.21.15.83:0/418306752 conn(0x7f137406abe0 msgr2=0x7f137406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:39.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:39.157+0000 7f13790c0640 1 -- 172.21.15.83:0/418306752 shutdown_connections 2024-04-17T10:53:39.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:39.157+0000 7f13790c0640 1 -- 172.21.15.83:0/418306752 wait complete. 2024-04-17T10:53:39.385 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:53:39.388 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:53:39.567 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:39.567 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:40.568 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:40.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.705+0000 7ff470cd7640 1 Processor -- start 2024-04-17T10:53:40.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.705+0000 7ff470cd7640 1 -- start start 2024-04-17T10:53:40.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.709+0000 7ff470cd7640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff46c0aecd0 0x7ff46c0cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:40.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.709+0000 7ff470cd7640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff46c072760 con 0x7ff46c166550 2024-04-17T10:53:40.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.709+0000 7ff470cd7640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff46c0cf200 con 0x7ff46c0b3f80 2024-04-17T10:53:40.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.709+0000 7ff470cd7640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff46c0cf810 con 0x7ff46c0aecd0 2024-04-17T10:53:40.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.709+0000 7ff46a59c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff46c0aecd0 0x7ff46c0cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:40.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.709+0000 7ff469d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff46c166550 0x7ff46c071be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55398/0 (socket says 172.21.15.83:55398) 2024-04-17T10:53:40.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.709+0000 7ff469d9b640 1 -- 172.21.15.83:0/368389370 learned_addr learned my addr 172.21.15.83:0/368389370 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:53:40.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.709+0000 7ff46a59c640 1 -- 172.21.15.83:0/368389370 >> v1:172.21.15.167:6789/0 conn(0x7ff46c0b3f80 legacy=0x7ff46c0b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:40.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.709+0000 7ff46a59c640 1 -- 172.21.15.83:0/368389370 >> v1:172.21.15.83:6789/0 conn(0x7ff46c166550 legacy=0x7ff46c071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:40.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.709+0000 7ff46a59c640 1 -- 172.21.15.83:0/368389370 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff46c0af0b0 con 0x7ff46c0aecd0 2024-04-17T10:53:40.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.709+0000 7ff46959a640 1 -- 172.21.15.83:0/368389370 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 476247275 0 0) 0x7ff46c072760 con 0x7ff46c166550 2024-04-17T10:53:40.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.709+0000 7ff46a59c640 1 --2- 172.21.15.83:0/368389370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff46c0aecd0 0x7ff46c0cfca0 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7ff45c0097c0 tx=0x7ff45c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a43c2f2e45a687d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:40.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.709+0000 7ff46959a640 1 -- 172.21.15.83:0/368389370 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c0b0070 con 0x7ff46c0aecd0 2024-04-17T10:53:40.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.709+0000 7ff46959a640 1 -- 172.21.15.83:0/368389370 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45c0ab600 con 0x7ff46c0aecd0 2024-04-17T10:53:40.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.709+0000 7ff46959a640 1 -- 172.21.15.83:0/368389370 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c0ab9d0 con 0x7ff46c0aecd0 2024-04-17T10:53:40.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff470cd7640 1 -- 172.21.15.83:0/368389370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff46c0aecd0 msgr2=0x7ff46c0cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:40.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff470cd7640 1 --2- 172.21.15.83:0/368389370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff46c0aecd0 0x7ff46c0cfca0 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7ff45c0097c0 tx=0x7ff45c01c3d0 comp rx=0 tx=0).stop 2024-04-17T10:53:40.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff470cd7640 1 -- 172.21.15.83:0/368389370 shutdown_connections 2024-04-17T10:53:40.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff470cd7640 1 --2- 172.21.15.83:0/368389370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff46c0aecd0 0x7ff46c0cfca0 unknown :-1 s=CLOSED pgs=2324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:40.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff470cd7640 1 -- 172.21.15.83:0/368389370 >> 172.21.15.83:0/368389370 conn(0x7ff46c06abe0 msgr2=0x7ff46c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:40.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff470cd7640 1 -- 172.21.15.83:0/368389370 shutdown_connections 2024-04-17T10:53:40.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff470cd7640 1 -- 172.21.15.83:0/368389370 wait complete. 2024-04-17T10:53:40.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff470cd7640 1 Processor -- start 2024-04-17T10:53:40.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff470cd7640 1 -- start start 2024-04-17T10:53:40.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff470cd7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff46c0aecd0 0x7ff46c157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:40.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff470cd7640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff46c0b3f80 0x7ff46c156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:40.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff470cd7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff46c166550 0x7ff46c14d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:40.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff470cd7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46c16ff30 con 0x7ff46c0aecd0 2024-04-17T10:53:40.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff470cd7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46c16fdb0 con 0x7ff46c166550 2024-04-17T10:53:40.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff470cd7640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff46c1700b0 con 0x7ff46c0b3f80 2024-04-17T10:53:40.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff46a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff46c0aecd0 0x7ff46c157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:40.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff46a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff46c0aecd0 0x7ff46c157ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52176/0 (socket says 172.21.15.83:52176) 2024-04-17T10:53:40.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff46a59c640 1 -- 172.21.15.83:0/1742956600 learned_addr learned my addr 172.21.15.83:0/1742956600 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:40.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff46ad9d640 1 --2- 172.21.15.83:0/1742956600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff46c166550 0x7ff46c14d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:40.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff469d9b640 1 --2- 172.21.15.83:0/1742956600 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff46c0b3f80 0x7ff46c156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:40.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff46a59c640 1 -- 172.21.15.83:0/1742956600 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff46c0b3f80 msgr2=0x7ff46c156520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:40.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff46a59c640 1 --2- 172.21.15.83:0/1742956600 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff46c0b3f80 0x7ff46c156520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:40.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff46a59c640 1 -- 172.21.15.83:0/1742956600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff46c166550 msgr2=0x7ff46c14d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:40.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff46a59c640 1 --2- 172.21.15.83:0/1742956600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff46c166550 0x7ff46c14d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:40.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff46a59c640 1 -- 172.21.15.83:0/1742956600 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff46c152ca0 con 0x7ff46c0aecd0 2024-04-17T10:53:40.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff46ad9d640 1 --2- 172.21.15.83:0/1742956600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff46c166550 0x7ff46c14d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:53:40.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff46a59c640 1 --2- 172.21.15.83:0/1742956600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff46c0aecd0 0x7ff46c157ea0 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7ff45c0ae040 tx=0x7ff45c01ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:40.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff45affd640 1 -- 172.21.15.83:0/1742956600 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c0b0050 con 0x7ff46c0aecd0 2024-04-17T10:53:40.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff45affd640 1 -- 172.21.15.83:0/1742956600 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45c0bc070 con 0x7ff46c0aecd0 2024-04-17T10:53:40.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff470cd7640 1 -- 172.21.15.83:0/1742956600 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff46c154ee0 con 0x7ff46c0aecd0 2024-04-17T10:53:40.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff45affd640 1 -- 172.21.15.83:0/1742956600 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c0b5870 con 0x7ff46c0aecd0 2024-04-17T10:53:40.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff470cd7640 1 -- 172.21.15.83:0/1742956600 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff46c153220 con 0x7ff46c0aecd0 2024-04-17T10:53:40.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.713+0000 7ff470cd7640 1 -- 172.21.15.83:0/1742956600 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff46c074f90 con 0x7ff46c0aecd0 2024-04-17T10:53:40.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.717+0000 7ff45affd640 1 -- 172.21.15.83:0/1742956600 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff45c0b5a10 con 0x7ff46c0aecd0 2024-04-17T10:53:40.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.717+0000 7ff45affd640 1 --2- 172.21.15.83:0/1742956600 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff44c066f60 0x7ff44c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:40.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.717+0000 7ff45affd640 1 -- 172.21.15.83:0/1742956600 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7ff45c11f870 con 0x7ff46c0aecd0 2024-04-17T10:53:40.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.717+0000 7ff469d9b640 1 --2- 172.21.15.83:0/1742956600 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff44c066f60 0x7ff44c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:40.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.717+0000 7ff469d9b640 1 --2- 172.21.15.83:0/1742956600 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff44c066f60 0x7ff44c069420 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7ff46c156030 tx=0x7ff460009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:40.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.721+0000 7ff45affd640 1 -- 172.21.15.83:0/1742956600 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff45c016050 con 0x7ff46c0aecd0 2024-04-17T10:53:40.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.909+0000 7ff470cd7640 1 -- 172.21.15.83:0/1742956600 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff46c074480 con 0x7ff46c0aecd0 2024-04-17T10:53:40.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.909+0000 7ff45affd640 1 -- 172.21.15.83:0/1742956600 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7ff45c0f08a0 con 0x7ff46c0aecd0 2024-04-17T10:53:40.901 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:40.901 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:40.902 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:40.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.913+0000 7ff470cd7640 1 -- 172.21.15.83:0/1742956600 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff44c066f60 msgr2=0x7ff44c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:40.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.913+0000 7ff470cd7640 1 --2- 172.21.15.83:0/1742956600 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff44c066f60 0x7ff44c069420 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7ff46c156030 tx=0x7ff460009040 comp rx=0 tx=0).stop 2024-04-17T10:53:40.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.913+0000 7ff470cd7640 1 -- 172.21.15.83:0/1742956600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff46c0aecd0 msgr2=0x7ff46c157ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:40.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.913+0000 7ff470cd7640 1 --2- 172.21.15.83:0/1742956600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff46c0aecd0 0x7ff46c157ea0 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7ff45c0ae040 tx=0x7ff45c01ac10 comp rx=0 tx=0).stop 2024-04-17T10:53:40.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.913+0000 7ff470cd7640 1 -- 172.21.15.83:0/1742956600 shutdown_connections 2024-04-17T10:53:40.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.913+0000 7ff470cd7640 1 --2- 172.21.15.83:0/1742956600 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff44c066f60 0x7ff44c069420 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:40.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.913+0000 7ff470cd7640 1 --2- 172.21.15.83:0/1742956600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff46c166550 0x7ff46c14d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:40.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.913+0000 7ff470cd7640 1 --2- 172.21.15.83:0/1742956600 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff46c0b3f80 0x7ff46c156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:40.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.913+0000 7ff470cd7640 1 --2- 172.21.15.83:0/1742956600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff46c0aecd0 0x7ff46c157ea0 unknown :-1 s=CLOSED pgs=2325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:40.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.913+0000 7ff470cd7640 1 -- 172.21.15.83:0/1742956600 >> 172.21.15.83:0/1742956600 conn(0x7ff46c06abe0 msgr2=0x7ff46c0b00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:40.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.913+0000 7ff470cd7640 1 -- 172.21.15.83:0/1742956600 shutdown_connections 2024-04-17T10:53:40.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:40.913+0000 7ff470cd7640 1 -- 172.21.15.83:0/1742956600 wait complete. 2024-04-17T10:53:41.321 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:41.322 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:42.323 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:42.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.457+0000 7f4c1ca1d640 1 Processor -- start 2024-04-17T10:53:42.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.457+0000 7f4c1ca1d640 1 -- start start 2024-04-17T10:53:42.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.461+0000 7f4c1ca1d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c180aedd0 0x7f4c1806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:42.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.461+0000 7f4c1ca1d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4c1806dae0 0x7f4c180b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:42.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.461+0000 7f4c1ca1d640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c1815e440 con 0x7f4c1815db50 2024-04-17T10:53:42.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.461+0000 7f4c1ca1d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c1815e720 con 0x7f4c1806dae0 2024-04-17T10:53:42.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.461+0000 7f4c1ca1d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1815ea00 con 0x7f4c180aedd0 2024-04-17T10:53:42.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.461+0000 7f4c16d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4c1806dae0 0x7f4c180b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:42.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.461+0000 7f4c16d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4c1806dae0 0x7f4c180b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52184/0 (socket says 172.21.15.83:52184) 2024-04-17T10:53:42.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.461+0000 7f4c16d9d640 1 -- 172.21.15.83:0/2662275677 learned_addr learned my addr 172.21.15.83:0/2662275677 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:42.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.461+0000 7f4c15d9b640 1 --2- 172.21.15.83:0/2662275677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c180aedd0 0x7f4c1806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:42.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.461+0000 7f4c16d9d640 1 -- 172.21.15.83:0/2662275677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c180aedd0 msgr2=0x7f4c1806d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:42.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.461+0000 7f4c16d9d640 1 --2- 172.21.15.83:0/2662275677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c180aedd0 0x7f4c1806d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:42.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.461+0000 7f4c16d9d640 1 -- 172.21.15.83:0/2662275677 >> v1:172.21.15.167:6789/0 conn(0x7f4c1815db50 legacy=0x7f4c18070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:42.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.461+0000 7f4c16d9d640 1 -- 172.21.15.83:0/2662275677 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c1815ece0 con 0x7f4c1806dae0 2024-04-17T10:53:42.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.461+0000 7f4c16d9d640 1 --2- 172.21.15.83:0/2662275677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c1806dae0 0x7f4c180b2e10 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f4c00009870 tx=0x7f4c0001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec1b34822bf71da server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:42.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.461+0000 7f4c1559a640 1 -- 172.21.15.83:0/2662275677 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c000b0070 con 0x7f4c1806dae0 2024-04-17T10:53:42.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.461+0000 7f4c1559a640 1 -- 172.21.15.83:0/2662275677 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c000ab600 con 0x7f4c1806dae0 2024-04-17T10:53:42.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.461+0000 7f4c1559a640 1 -- 172.21.15.83:0/2662275677 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c000ab9d0 con 0x7f4c1806dae0 2024-04-17T10:53:42.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.465+0000 7f4c1ca1d640 1 -- 172.21.15.83:0/2662275677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c1806dae0 msgr2=0x7f4c180b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:42.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.465+0000 7f4c1ca1d640 1 --2- 172.21.15.83:0/2662275677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c1806dae0 0x7f4c180b2e10 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f4c00009870 tx=0x7f4c0001c520 comp rx=0 tx=0).stop 2024-04-17T10:53:42.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.465+0000 7f4c1ca1d640 1 -- 172.21.15.83:0/2662275677 shutdown_connections 2024-04-17T10:53:42.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.465+0000 7f4c1ca1d640 1 --2- 172.21.15.83:0/2662275677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c1806dae0 0x7f4c180b2e10 unknown :-1 s=CLOSED pgs=2326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:42.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.465+0000 7f4c1ca1d640 1 --2- 172.21.15.83:0/2662275677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c180aedd0 0x7f4c1806d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:42.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.465+0000 7f4c1ca1d640 1 -- 172.21.15.83:0/2662275677 >> 172.21.15.83:0/2662275677 conn(0x7f4c1806abe0 msgr2=0x7f4c1806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:42.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.465+0000 7f4c1ca1d640 1 -- 172.21.15.83:0/2662275677 shutdown_connections 2024-04-17T10:53:42.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.465+0000 7f4c1ca1d640 1 -- 172.21.15.83:0/2662275677 wait complete. 2024-04-17T10:53:42.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.465+0000 7f4c1ca1d640 1 Processor -- start 2024-04-17T10:53:42.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.465+0000 7f4c1ca1d640 1 -- start start 2024-04-17T10:53:42.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.465+0000 7f4c1ca1d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c1806dae0 0x7f4c1814b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:42.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.465+0000 7f4c1ca1d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c180aedd0 0x7f4c180ad3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:42.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.465+0000 7f4c1ca1d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c1815db50 0x7f4c1806c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:42.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.465+0000 7f4c1ca1d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c180b3ef0 con 0x7f4c1806dae0 2024-04-17T10:53:42.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.465+0000 7f4c1ca1d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c180b3d70 con 0x7f4c1815db50 2024-04-17T10:53:42.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.465+0000 7f4c1ca1d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c180b4070 con 0x7f4c180aedd0 2024-04-17T10:53:42.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.465+0000 7f4c1659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c1806dae0 0x7f4c1814b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:42.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.465+0000 7f4c1659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c1806dae0 0x7f4c1814b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52186/0 (socket says 172.21.15.83:52186) 2024-04-17T10:53:42.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.465+0000 7f4c1659c640 1 -- 172.21.15.83:0/3488203531 learned_addr learned my addr 172.21.15.83:0/3488203531 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:42.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.465+0000 7f4c15d9b640 1 --2- 172.21.15.83:0/3488203531 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c180aedd0 0x7f4c180ad3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:42.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.465+0000 7f4c16d9d640 1 --2- 172.21.15.83:0/3488203531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c1815db50 0x7f4c1806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:42.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.465+0000 7f4c1659c640 1 -- 172.21.15.83:0/3488203531 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c180aedd0 msgr2=0x7f4c180ad3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:42.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.465+0000 7f4c1659c640 1 --2- 172.21.15.83:0/3488203531 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c180aedd0 0x7f4c180ad3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:42.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.465+0000 7f4c1659c640 1 -- 172.21.15.83:0/3488203531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c1815db50 msgr2=0x7f4c1806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:42.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.465+0000 7f4c1659c640 1 --2- 172.21.15.83:0/3488203531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c1815db50 0x7f4c1806c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:42.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.465+0000 7f4c1659c640 1 -- 172.21.15.83:0/3488203531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c1814b790 con 0x7f4c1806dae0 2024-04-17T10:53:42.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.465+0000 7f4c1659c640 1 --2- 172.21.15.83:0/3488203531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c1806dae0 0x7f4c1814b250 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f4c1000b410 tx=0x7f4c1000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:42.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.465+0000 7f4c0effd640 1 -- 172.21.15.83:0/3488203531 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c10015030 con 0x7f4c1806dae0 2024-04-17T10:53:42.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.465+0000 7f4c0effd640 1 -- 172.21.15.83:0/3488203531 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c10004020 con 0x7f4c1806dae0 2024-04-17T10:53:42.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.465+0000 7f4c1ca1d640 1 -- 172.21.15.83:0/3488203531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c180addc0 con 0x7f4c1806dae0 2024-04-17T10:53:42.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.465+0000 7f4c0effd640 1 -- 172.21.15.83:0/3488203531 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c10004450 con 0x7f4c1806dae0 2024-04-17T10:53:42.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.465+0000 7f4c1ca1d640 1 -- 172.21.15.83:0/3488203531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c180ae380 con 0x7f4c1806dae0 2024-04-17T10:53:42.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.469+0000 7f4c1ca1d640 1 -- 172.21.15.83:0/3488203531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c18074f90 con 0x7f4c1806dae0 2024-04-17T10:53:42.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.469+0000 7f4c0effd640 1 -- 172.21.15.83:0/3488203531 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4c10018070 con 0x7f4c1806dae0 2024-04-17T10:53:42.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.469+0000 7f4c0effd640 1 --2- 172.21.15.83:0/3488203531 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4bf8066f90 0x7f4bf8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:42.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.469+0000 7f4c15d9b640 1 --2- 172.21.15.83:0/3488203531 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4bf8066f90 0x7f4bf8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:42.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.469+0000 7f4c0effd640 1 -- 172.21.15.83:0/3488203531 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f4c10084c70 con 0x7f4c1806dae0 2024-04-17T10:53:42.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.469+0000 7f4c15d9b640 1 --2- 172.21.15.83:0/3488203531 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4bf8066f90 0x7f4bf8069450 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f4c08003fd0 tx=0x7f4c080039a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:42.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.473+0000 7f4c0effd640 1 -- 172.21.15.83:0/3488203531 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4c10055b80 con 0x7f4c1806dae0 2024-04-17T10:53:42.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.665+0000 7f4c1ca1d640 1 -- 172.21.15.83:0/3488203531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c180b2af0 con 0x7f4c1806dae0 2024-04-17T10:53:42.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.665+0000 7f4c0effd640 1 -- 172.21.15.83:0/3488203531 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f4c10055480 con 0x7f4c1806dae0 2024-04-17T10:53:42.658 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:42.659 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:42.659 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:42.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.669+0000 7f4c1ca1d640 1 -- 172.21.15.83:0/3488203531 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4bf8066f90 msgr2=0x7f4bf8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:42.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.669+0000 7f4c1ca1d640 1 --2- 172.21.15.83:0/3488203531 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4bf8066f90 0x7f4bf8069450 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f4c08003fd0 tx=0x7f4c080039a0 comp rx=0 tx=0).stop 2024-04-17T10:53:42.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.669+0000 7f4c1ca1d640 1 -- 172.21.15.83:0/3488203531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c1806dae0 msgr2=0x7f4c1814b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:42.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.669+0000 7f4c1ca1d640 1 --2- 172.21.15.83:0/3488203531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c1806dae0 0x7f4c1814b250 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f4c1000b410 tx=0x7f4c1000b8d0 comp rx=0 tx=0).stop 2024-04-17T10:53:42.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.669+0000 7f4c1ca1d640 1 -- 172.21.15.83:0/3488203531 shutdown_connections 2024-04-17T10:53:42.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.669+0000 7f4c1ca1d640 1 --2- 172.21.15.83:0/3488203531 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4bf8066f90 0x7f4bf8069450 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:42.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.669+0000 7f4c1ca1d640 1 --2- 172.21.15.83:0/3488203531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c1815db50 0x7f4c1806c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:42.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.669+0000 7f4c1ca1d640 1 --2- 172.21.15.83:0/3488203531 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c180aedd0 0x7f4c180ad3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:42.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.669+0000 7f4c1ca1d640 1 --2- 172.21.15.83:0/3488203531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c1806dae0 0x7f4c1814b250 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:42.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.669+0000 7f4c1ca1d640 1 -- 172.21.15.83:0/3488203531 >> 172.21.15.83:0/3488203531 conn(0x7f4c1806abe0 msgr2=0x7f4c18070470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:42.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.669+0000 7f4c1ca1d640 1 -- 172.21.15.83:0/3488203531 shutdown_connections 2024-04-17T10:53:42.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:42.669+0000 7f4c1ca1d640 1 -- 172.21.15.83:0/3488203531 wait complete. 2024-04-17T10:53:43.079 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:43.079 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:44.080 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:44.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.217+0000 7f69fd188640 1 Processor -- start 2024-04-17T10:53:44.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.217+0000 7f69fd188640 1 -- start start 2024-04-17T10:53:44.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.221+0000 7f69fd188640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69f80cfbe0 0x7f69f806c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:44.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.221+0000 7f69fd188640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f806e320 con 0x7f69f815ba30 2024-04-17T10:53:44.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.221+0000 7f69fd188640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f806e040 con 0x7f69f8072330 2024-04-17T10:53:44.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.221+0000 7f69fd188640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f8071ea0 con 0x7f69f80cfbe0 2024-04-17T10:53:44.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.221+0000 7f69f6d9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f69f815ba30 0x7f69f815c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55408/0 (socket says 172.21.15.83:55408) 2024-04-17T10:53:44.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.221+0000 7f69f6d9d640 1 -- 172.21.15.83:0/1201895639 learned_addr learned my addr 172.21.15.83:0/1201895639 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:53:44.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.221+0000 7f69f5d9b640 1 -- 172.21.15.83:0/1201895639 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3555156589 0 0) 0x7f69f806e320 con 0x7f69f815ba30 2024-04-17T10:53:44.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.221+0000 7f69f5d9b640 1 -- 172.21.15.83:0/1201895639 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69e0003530 con 0x7f69f815ba30 2024-04-17T10:53:44.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.221+0000 7f69f759e640 1 --2- 172.21.15.83:0/1201895639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69f80cfbe0 0x7f69f806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:44.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.221+0000 7f69f5d9b640 1 -- 172.21.15.83:0/1201895639 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1996616008 0 0) 0x7f69e0003530 con 0x7f69f815ba30 2024-04-17T10:53:44.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.221+0000 7f69f5d9b640 1 -- 172.21.15.83:0/1201895639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69f80cfbe0 msgr2=0x7f69f806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:44.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.221+0000 7f69f5d9b640 1 --2- 172.21.15.83:0/1201895639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69f80cfbe0 0x7f69f806c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:44.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.221+0000 7f69f5d9b640 1 -- 172.21.15.83:0/1201895639 >> v1:172.21.15.167:6789/0 conn(0x7f69f8072330 legacy=0x7f69f80cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:44.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.221+0000 7f69f5d9b640 1 -- 172.21.15.83:0/1201895639 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f8071850 con 0x7f69f815ba30 2024-04-17T10:53:44.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.221+0000 7f69f5d9b640 1 -- 172.21.15.83:0/1201895639 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f69e8001680 con 0x7f69f815ba30 2024-04-17T10:53:44.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.221+0000 7f69f5d9b640 1 -- 172.21.15.83:0/1201895639 >> v1:172.21.15.83:6789/0 conn(0x7f69f815ba30 legacy=0x7f69f815c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:44.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.221+0000 7f69f5d9b640 1 --2- 172.21.15.83:0/1201895639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69e00039f0 0x7f69e0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:44.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.221+0000 7f69f5d9b640 1 --2- 172.21.15.83:0/1201895639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69e0004810 0x7f69e0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:44.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.221+0000 7f69f5d9b640 1 --2- 172.21.15.83:0/1201895639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69f80cfbe0 0x7f69e000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:44.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.221+0000 7f69f5d9b640 1 -- 172.21.15.83:0/1201895639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f8071ea0 con 0x7f69e0004810 2024-04-17T10:53:44.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.221+0000 7f69f5d9b640 1 -- 172.21.15.83:0/1201895639 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e0005530 con 0x7f69e00039f0 2024-04-17T10:53:44.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.221+0000 7f69f5d9b640 1 -- 172.21.15.83:0/1201895639 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e000bf10 con 0x7f69f80cfbe0 2024-04-17T10:53:44.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.221+0000 7f69f659c640 1 --2- 172.21.15.83:0/1201895639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69e0004810 0x7f69e0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:44.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.221+0000 7f69f6d9d640 1 --2- 172.21.15.83:0/1201895639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69e00039f0 0x7f69e0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:44.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.221+0000 7f69f759e640 1 --2- 172.21.15.83:0/1201895639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69f80cfbe0 0x7f69e000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:44.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.221+0000 7f69f659c640 1 -- 172.21.15.83:0/1201895639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69f80cfbe0 msgr2=0x7f69e000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:44.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.221+0000 7f69f659c640 1 --2- 172.21.15.83:0/1201895639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69f80cfbe0 0x7f69e000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:44.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.221+0000 7f69f659c640 1 -- 172.21.15.83:0/1201895639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69e00039f0 msgr2=0x7f69e0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:44.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.221+0000 7f69f659c640 1 --2- 172.21.15.83:0/1201895639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69e00039f0 0x7f69e0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:44.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.221+0000 7f69f659c640 1 -- 172.21.15.83:0/1201895639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e0003530 con 0x7f69e0004810 2024-04-17T10:53:44.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.225+0000 7f69f659c640 1 --2- 172.21.15.83:0/1201895639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69e0004810 0x7f69e0004c90 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f69ec001780 tx=0x7f69ec01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f3337337a15001 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:44.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.225+0000 7f69f5d9b640 1 -- 172.21.15.83:0/1201895639 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ec0013a0 con 0x7f69e0004810 2024-04-17T10:53:44.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.225+0000 7f69f5d9b640 1 -- 172.21.15.83:0/1201895639 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69ec0052f0 con 0x7f69e0004810 2024-04-17T10:53:44.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.225+0000 7f69f5d9b640 1 -- 172.21.15.83:0/1201895639 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ec003840 con 0x7f69e0004810 2024-04-17T10:53:44.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.225+0000 7f69fd188640 1 -- 172.21.15.83:0/1201895639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69e0004810 msgr2=0x7f69e0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:44.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.225+0000 7f69fd188640 1 --2- 172.21.15.83:0/1201895639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69e0004810 0x7f69e0004c90 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f69ec001780 tx=0x7f69ec01c420 comp rx=0 tx=0).stop 2024-04-17T10:53:44.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.225+0000 7f69f759e640 1 -- 172.21.15.83:0/1201895639 reap_dead start 2024-04-17T10:53:44.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.225+0000 7f69fd188640 1 -- 172.21.15.83:0/1201895639 shutdown_connections 2024-04-17T10:53:44.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.225+0000 7f69fd188640 1 -- 172.21.15.83:0/1201895639 >> 172.21.15.83:0/1201895639 conn(0x7f69f806abe0 msgr2=0x7f69f806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:44.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.225+0000 7f69fd188640 1 -- 172.21.15.83:0/1201895639 shutdown_connections 2024-04-17T10:53:44.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.225+0000 7f69fd188640 1 -- 172.21.15.83:0/1201895639 wait complete. 2024-04-17T10:53:44.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.225+0000 7f69fd188640 1 Processor -- start 2024-04-17T10:53:44.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.225+0000 7f69fd188640 1 -- start start 2024-04-17T10:53:44.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.225+0000 7f69fd188640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69e0004810 0x7f69f80aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:44.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.225+0000 7f69fd188640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69f814f3e0 0x7f69f814f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:44.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.225+0000 7f69fd188640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69f81f7c90 0x7f69f81f8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:44.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.225+0000 7f69fd188640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f806ce20 con 0x7f69f81f7c90 2024-04-17T10:53:44.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.225+0000 7f69fd188640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f806cca0 con 0x7f69e0004810 2024-04-17T10:53:44.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.225+0000 7f69fd188640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f806cfa0 con 0x7f69f814f3e0 2024-04-17T10:53:44.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.225+0000 7f69f759e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69f81f7c90 0x7f69f81f8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:44.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.225+0000 7f69f759e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69f81f7c90 0x7f69f81f8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52210/0 (socket says 172.21.15.83:52210) 2024-04-17T10:53:44.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.225+0000 7f69f759e640 1 -- 172.21.15.83:0/38157376 learned_addr learned my addr 172.21.15.83:0/38157376 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:44.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.225+0000 7f69f6d9d640 1 --2- 172.21.15.83:0/38157376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69e0004810 0x7f69f80aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:44.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.225+0000 7f69f659c640 1 --2- 172.21.15.83:0/38157376 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69f814f3e0 0x7f69f814f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:44.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.225+0000 7f69f759e640 1 -- 172.21.15.83:0/38157376 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69f814f3e0 msgr2=0x7f69f814f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:44.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.225+0000 7f69f759e640 1 --2- 172.21.15.83:0/38157376 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69f814f3e0 0x7f69f814f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:44.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.225+0000 7f69f759e640 1 -- 172.21.15.83:0/38157376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69e0004810 msgr2=0x7f69f80aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:44.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.225+0000 7f69f759e640 1 --2- 172.21.15.83:0/38157376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69e0004810 0x7f69f80aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:44.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.225+0000 7f69f759e640 1 -- 172.21.15.83:0/38157376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f80af720 con 0x7f69f81f7c90 2024-04-17T10:53:44.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.225+0000 7f69f759e640 1 --2- 172.21.15.83:0/38157376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69f81f7c90 0x7f69f81f8110 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f69f80b1320 tx=0x7f69f0009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:44.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.229+0000 7f69df7fe640 1 -- 172.21.15.83:0/38157376 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f0019070 con 0x7f69f81f7c90 2024-04-17T10:53:44.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.229+0000 7f69df7fe640 1 -- 172.21.15.83:0/38157376 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f0007d70 con 0x7f69f81f7c90 2024-04-17T10:53:44.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.229+0000 7f69fd188640 1 -- 172.21.15.83:0/38157376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f80af990 con 0x7f69f81f7c90 2024-04-17T10:53:44.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.229+0000 7f69df7fe640 1 -- 172.21.15.83:0/38157376 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f0014640 con 0x7f69f81f7c90 2024-04-17T10:53:44.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.229+0000 7f69fd188640 1 -- 172.21.15.83:0/38157376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f8150920 con 0x7f69f81f7c90 2024-04-17T10:53:44.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.229+0000 7f69fd188640 1 -- 172.21.15.83:0/38157376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f8074f90 con 0x7f69f81f7c90 2024-04-17T10:53:44.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.229+0000 7f69df7fe640 1 -- 172.21.15.83:0/38157376 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f69f001c070 con 0x7f69f81f7c90 2024-04-17T10:53:44.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.229+0000 7f69df7fe640 1 --2- 172.21.15.83:0/38157376 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f69e4066f90 0x7f69e4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:44.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.229+0000 7f69df7fe640 1 -- 172.21.15.83:0/38157376 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f69f0084c80 con 0x7f69f81f7c90 2024-04-17T10:53:44.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.229+0000 7f69f6d9d640 1 --2- 172.21.15.83:0/38157376 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f69e4066f90 0x7f69e4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:44.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.229+0000 7f69f6d9d640 1 --2- 172.21.15.83:0/38157376 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f69e4066f90 0x7f69e4069450 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f69e8001ec0 tx=0x7f69e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:44.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.233+0000 7f69df7fe640 1 -- 172.21.15.83:0/38157376 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f69f0055b90 con 0x7f69f81f7c90 2024-04-17T10:53:44.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.429+0000 7f69fd188640 1 -- 172.21.15.83:0/38157376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69f806b160 con 0x7f69f81f7c90 2024-04-17T10:53:44.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.429+0000 7f69df7fe640 1 -- 172.21.15.83:0/38157376 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f69f0055490 con 0x7f69f81f7c90 2024-04-17T10:53:44.422 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:44.422 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:44.422 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:44.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.433+0000 7f69fd188640 1 -- 172.21.15.83:0/38157376 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f69e4066f90 msgr2=0x7f69e4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:44.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.433+0000 7f69fd188640 1 --2- 172.21.15.83:0/38157376 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f69e4066f90 0x7f69e4069450 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f69e8001ec0 tx=0x7f69e8009040 comp rx=0 tx=0).stop 2024-04-17T10:53:44.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.433+0000 7f69fd188640 1 -- 172.21.15.83:0/38157376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69f81f7c90 msgr2=0x7f69f81f8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:44.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.433+0000 7f69fd188640 1 --2- 172.21.15.83:0/38157376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69f81f7c90 0x7f69f81f8110 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f69f80b1320 tx=0x7f69f0009dc0 comp rx=0 tx=0).stop 2024-04-17T10:53:44.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.433+0000 7f69fd188640 1 -- 172.21.15.83:0/38157376 shutdown_connections 2024-04-17T10:53:44.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.433+0000 7f69fd188640 1 --2- 172.21.15.83:0/38157376 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f69e4066f90 0x7f69e4069450 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:44.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.433+0000 7f69fd188640 1 --2- 172.21.15.83:0/38157376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69f81f7c90 0x7f69f81f8110 unknown :-1 s=CLOSED pgs=2330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:44.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.433+0000 7f69fd188640 1 --2- 172.21.15.83:0/38157376 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69f814f3e0 0x7f69f814f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:44.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.433+0000 7f69fd188640 1 --2- 172.21.15.83:0/38157376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69e0004810 0x7f69f80aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:44.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.433+0000 7f69fd188640 1 -- 172.21.15.83:0/38157376 >> 172.21.15.83:0/38157376 conn(0x7f69f806abe0 msgr2=0x7f69f81531d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:44.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.433+0000 7f69fd188640 1 -- 172.21.15.83:0/38157376 shutdown_connections 2024-04-17T10:53:44.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:44.433+0000 7f69fd188640 1 -- 172.21.15.83:0/38157376 wait complete. 2024-04-17T10:53:44.843 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:44.843 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:45.844 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:45.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.985+0000 7f7338926640 1 Processor -- start 2024-04-17T10:53:45.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.985+0000 7f7338926640 1 -- start start 2024-04-17T10:53:45.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.989+0000 7f7338926640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f73340cfca0 0x7f7334166550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:45.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.989+0000 7f7338926640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73340b3f80 0x7f73340b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:45.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.989+0000 7f7338926640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7334072760 con 0x7f73340aecd0 2024-04-17T10:53:45.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.989+0000 7f7338926640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73340cf810 con 0x7f73340cfca0 2024-04-17T10:53:45.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.989+0000 7f7338926640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73340b4950 con 0x7f73340b3f80 2024-04-17T10:53:45.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.989+0000 7f733259c640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f73340aecd0 0x7f7334071be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:34180/0 (socket says 172.21.15.83:34180) 2024-04-17T10:53:45.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.989+0000 7f733259c640 1 -- 172.21.15.83:0/2622536688 learned_addr learned my addr 172.21.15.83:0/2622536688 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:53:45.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.989+0000 7f7332d9d640 1 --2- 172.21.15.83:0/2622536688 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73340b3f80 0x7f73340b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:45.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.989+0000 7f7331d9b640 1 --2- 172.21.15.83:0/2622536688 >> v2:172.21.15.167:3300/0 conn(0x7f73340cfca0 0x7f7334166550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:45.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.989+0000 7f7331d9b640 1 -- 172.21.15.83:0/2622536688 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73340b3f80 msgr2=0x7f73340b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:45.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.989+0000 7f7331d9b640 1 --2- 172.21.15.83:0/2622536688 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73340b3f80 0x7f73340b4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:45.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.989+0000 7f7331d9b640 1 -- 172.21.15.83:0/2622536688 >> v1:172.21.15.167:6789/0 conn(0x7f73340aecd0 legacy=0x7f7334071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:45.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.989+0000 7f7331d9b640 1 -- 172.21.15.83:0/2622536688 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73340cf200 con 0x7f73340cfca0 2024-04-17T10:53:45.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.993+0000 7f7331d9b640 1 --2- 172.21.15.83:0/2622536688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73340cfca0 0x7f7334166550 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f731c0097c0 tx=0x7f731c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db2ce697d4fde13f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:45.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.993+0000 7f733159a640 1 -- 172.21.15.83:0/2622536688 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c0b0070 con 0x7f73340cfca0 2024-04-17T10:53:45.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.993+0000 7f733159a640 1 -- 172.21.15.83:0/2622536688 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731c0ab600 con 0x7f73340cfca0 2024-04-17T10:53:45.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.993+0000 7f733159a640 1 -- 172.21.15.83:0/2622536688 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c0ab9d0 con 0x7f73340cfca0 2024-04-17T10:53:45.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.993+0000 7f7338926640 1 -- 172.21.15.83:0/2622536688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73340cfca0 msgr2=0x7f7334166550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:45.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.993+0000 7f7338926640 1 --2- 172.21.15.83:0/2622536688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73340cfca0 0x7f7334166550 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f731c0097c0 tx=0x7f731c01c410 comp rx=0 tx=0).stop 2024-04-17T10:53:45.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.993+0000 7f7338926640 1 -- 172.21.15.83:0/2622536688 shutdown_connections 2024-04-17T10:53:45.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.993+0000 7f7338926640 1 --2- 172.21.15.83:0/2622536688 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73340b3f80 0x7f73340b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:45.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.993+0000 7f7338926640 1 --2- 172.21.15.83:0/2622536688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73340cfca0 0x7f7334166550 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:45.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.993+0000 7f7338926640 1 -- 172.21.15.83:0/2622536688 >> 172.21.15.83:0/2622536688 conn(0x7f733406abe0 msgr2=0x7f733406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:45.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.993+0000 7f7338926640 1 -- 172.21.15.83:0/2622536688 shutdown_connections 2024-04-17T10:53:45.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.993+0000 7f7338926640 1 -- 172.21.15.83:0/2622536688 wait complete. 2024-04-17T10:53:45.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.993+0000 7f7338926640 1 Processor -- start 2024-04-17T10:53:45.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.993+0000 7f7338926640 1 -- start start 2024-04-17T10:53:45.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.993+0000 7f7338926640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73340aecd0 0x7f733415a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:45.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.993+0000 7f7338926640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73340b3f80 0x7f73340a9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:45.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.993+0000 7f7338926640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73340cfca0 0x7f73340a9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:45.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.993+0000 7f7338926640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73340b3000 con 0x7f73340cfca0 2024-04-17T10:53:45.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.993+0000 7f7338926640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73340b2e80 con 0x7f73340b3f80 2024-04-17T10:53:45.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.993+0000 7f7338926640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73340b3180 con 0x7f73340aecd0 2024-04-17T10:53:45.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.993+0000 7f7332d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73340cfca0 0x7f73340a9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:45.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.993+0000 7f7332d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73340cfca0 0x7f73340a9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52216/0 (socket says 172.21.15.83:52216) 2024-04-17T10:53:45.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.993+0000 7f7332d9d640 1 -- 172.21.15.83:0/3898322945 learned_addr learned my addr 172.21.15.83:0/3898322945 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:45.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.993+0000 7f7331d9b640 1 --2- 172.21.15.83:0/3898322945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73340b3f80 0x7f73340a9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:45.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.993+0000 7f733259c640 1 --2- 172.21.15.83:0/3898322945 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73340aecd0 0x7f733415a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:45.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.993+0000 7f7332d9d640 1 -- 172.21.15.83:0/3898322945 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73340aecd0 msgr2=0x7f733415a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:45.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.993+0000 7f7332d9d640 1 --2- 172.21.15.83:0/3898322945 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73340aecd0 0x7f733415a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:45.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.993+0000 7f7332d9d640 1 -- 172.21.15.83:0/3898322945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73340b3f80 msgr2=0x7f73340a9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:45.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.993+0000 7f7332d9d640 1 --2- 172.21.15.83:0/3898322945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73340b3f80 0x7f73340a9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:45.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.997+0000 7f7332d9d640 1 -- 172.21.15.83:0/3898322945 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73340ab650 con 0x7f73340cfca0 2024-04-17T10:53:45.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.997+0000 7f7331d9b640 1 --2- 172.21.15.83:0/3898322945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73340b3f80 0x7f73340a9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:53:45.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.997+0000 7f7332d9d640 1 --2- 172.21.15.83:0/3898322945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73340cfca0 0x7f73340a9970 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f732400e820 tx=0x7f732400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:45.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.997+0000 7f732affd640 1 -- 172.21.15.83:0/3898322945 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7324017070 con 0x7f73340cfca0 2024-04-17T10:53:45.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.997+0000 7f732affd640 1 -- 172.21.15.83:0/3898322945 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732400c940 con 0x7f73340cfca0 2024-04-17T10:53:45.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.997+0000 7f732affd640 1 -- 172.21.15.83:0/3898322945 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732400cd10 con 0x7f73340cfca0 2024-04-17T10:53:45.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.997+0000 7f7338926640 1 -- 172.21.15.83:0/3898322945 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73340ab930 con 0x7f73340cfca0 2024-04-17T10:53:45.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.997+0000 7f7338926640 1 -- 172.21.15.83:0/3898322945 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73340abef0 con 0x7f73340cfca0 2024-04-17T10:53:45.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.997+0000 7f7338926640 1 -- 172.21.15.83:0/3898322945 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7334074f90 con 0x7f73340cfca0 2024-04-17T10:53:45.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.997+0000 7f732affd640 1 -- 172.21.15.83:0/3898322945 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f732401a070 con 0x7f73340cfca0 2024-04-17T10:53:45.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.997+0000 7f732affd640 1 --2- 172.21.15.83:0/3898322945 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7314066f60 0x7f7314069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:45.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.997+0000 7f732affd640 1 -- 172.21.15.83:0/3898322945 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f73240852e0 con 0x7f73340cfca0 2024-04-17T10:53:45.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.997+0000 7f733259c640 1 --2- 172.21.15.83:0/3898322945 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7314066f60 0x7f7314069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:45.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:45.997+0000 7f733259c640 1 --2- 172.21.15.83:0/3898322945 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7314066f60 0x7f7314069420 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f732c0012f0 tx=0x7f732c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:45.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:46.001+0000 7f732affd640 1 -- 172.21.15.83:0/3898322945 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7324056280 con 0x7f73340cfca0 2024-04-17T10:53:46.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:46.193+0000 7f7338926640 1 -- 172.21.15.83:0/3898322945 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f733406ca50 con 0x7f73340cfca0 2024-04-17T10:53:46.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:46.193+0000 7f732affd640 1 -- 172.21.15.83:0/3898322945 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f7324055b80 con 0x7f73340cfca0 2024-04-17T10:53:46.187 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:46.187 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:46.187 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:46.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:46.197+0000 7f7338926640 1 -- 172.21.15.83:0/3898322945 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7314066f60 msgr2=0x7f7314069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:46.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:46.197+0000 7f7338926640 1 --2- 172.21.15.83:0/3898322945 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7314066f60 0x7f7314069420 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f732c0012f0 tx=0x7f732c009040 comp rx=0 tx=0).stop 2024-04-17T10:53:46.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:46.197+0000 7f7338926640 1 -- 172.21.15.83:0/3898322945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73340cfca0 msgr2=0x7f73340a9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:46.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:46.197+0000 7f7338926640 1 --2- 172.21.15.83:0/3898322945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73340cfca0 0x7f73340a9970 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f732400e820 tx=0x7f732400ece0 comp rx=0 tx=0).stop 2024-04-17T10:53:46.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:46.197+0000 7f7338926640 1 -- 172.21.15.83:0/3898322945 shutdown_connections 2024-04-17T10:53:46.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:46.197+0000 7f7338926640 1 --2- 172.21.15.83:0/3898322945 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7314066f60 0x7f7314069420 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:46.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:46.197+0000 7f7338926640 1 --2- 172.21.15.83:0/3898322945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73340cfca0 0x7f73340a9970 unknown :-1 s=CLOSED pgs=2331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:46.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:46.197+0000 7f7338926640 1 --2- 172.21.15.83:0/3898322945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73340b3f80 0x7f73340a9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:46.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:46.197+0000 7f7338926640 1 --2- 172.21.15.83:0/3898322945 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73340aecd0 0x7f733415a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:46.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:46.197+0000 7f7338926640 1 -- 172.21.15.83:0/3898322945 >> 172.21.15.83:0/3898322945 conn(0x7f733406abe0 msgr2=0x7f733406cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:46.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:46.197+0000 7f7338926640 1 -- 172.21.15.83:0/3898322945 shutdown_connections 2024-04-17T10:53:46.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:46.197+0000 7f7338926640 1 -- 172.21.15.83:0/3898322945 wait complete. 2024-04-17T10:53:46.609 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:46.609 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:47.611 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:47.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f7393c5b640 1 Processor -- start 2024-04-17T10:53:47.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f7393c5b640 1 -- start start 2024-04-17T10:53:47.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f7393c5b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f738c06dcb0 0x7f738c165650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:47.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f7393c5b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f738c15e440 con 0x7f738c15db50 2024-04-17T10:53:47.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f7393c5b640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f738c15e720 con 0x7f738c0aedd0 2024-04-17T10:53:47.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f7393c5b640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f738c15ea00 con 0x7f738c06dcb0 2024-04-17T10:53:47.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f73919f7640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f738c15db50 0x7f738c070470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35528/0 (socket says 172.21.15.83:35528) 2024-04-17T10:53:47.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f73919f7640 1 -- 172.21.15.83:0/2077098271 learned_addr learned my addr 172.21.15.83:0/2077098271 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:53:47.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f73921f8640 1 --2- 172.21.15.83:0/2077098271 >> v2:172.21.15.167:3300/0 conn(0x7f738c06dcb0 0x7f738c165650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:47.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f73909f5640 1 -- 172.21.15.83:0/2077098271 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812860207 0 0) 0x7f738c15e440 con 0x7f738c15db50 2024-04-17T10:53:47.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f73909f5640 1 -- 172.21.15.83:0/2077098271 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7378003530 con 0x7f738c15db50 2024-04-17T10:53:47.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f73909f5640 1 -- 172.21.15.83:0/2077098271 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 327399784 0 0) 0x7f7378003530 con 0x7f738c15db50 2024-04-17T10:53:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f73909f5640 1 -- 172.21.15.83:0/2077098271 >> v2:172.21.15.167:3300/0 conn(0x7f738c06dcb0 msgr2=0x7f738c165650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f73909f5640 1 --2- 172.21.15.83:0/2077098271 >> v2:172.21.15.167:3300/0 conn(0x7f738c06dcb0 0x7f738c165650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f73909f5640 1 -- 172.21.15.83:0/2077098271 >> v1:172.21.15.167:6789/0 conn(0x7f738c0aedd0 legacy=0x7f738c06d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f73909f5640 1 -- 172.21.15.83:0/2077098271 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f738c15ece0 con 0x7f738c15db50 2024-04-17T10:53:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f73909f5640 1 -- 172.21.15.83:0/2077098271 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f7388001680 con 0x7f738c15db50 2024-04-17T10:53:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f73909f5640 1 -- 172.21.15.83:0/2077098271 >> v1:172.21.15.83:6789/0 conn(0x7f738c15db50 legacy=0x7f738c070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f73909f5640 1 --2- 172.21.15.83:0/2077098271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73780039f0 0x7f7378003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f73909f5640 1 --2- 172.21.15.83:0/2077098271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7378004810 0x7f7378004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:47.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f73909f5640 1 --2- 172.21.15.83:0/2077098271 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7378005220 0x7f737800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:47.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f73909f5640 1 -- 172.21.15.83:0/2077098271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738c15ea00 con 0x7f73780039f0 2024-04-17T10:53:47.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f73909f5640 1 -- 172.21.15.83:0/2077098271 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737800c150 con 0x7f7378004810 2024-04-17T10:53:47.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f73909f5640 1 -- 172.21.15.83:0/2077098271 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f737800c350 con 0x7f7378005220 2024-04-17T10:53:47.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f73919f7640 1 --2- 172.21.15.83:0/2077098271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73780039f0 0x7f7378003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:47.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f73911f6640 1 --2- 172.21.15.83:0/2077098271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7378004810 0x7f7378004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:47.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f73921f8640 1 --2- 172.21.15.83:0/2077098271 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7378005220 0x7f737800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:47.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f73919f7640 1 -- 172.21.15.83:0/2077098271 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7378005220 msgr2=0x7f737800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:47.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f73919f7640 1 --2- 172.21.15.83:0/2077098271 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7378005220 0x7f737800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:47.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f73919f7640 1 -- 172.21.15.83:0/2077098271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7378004810 msgr2=0x7f7378004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:47.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f73919f7640 1 --2- 172.21.15.83:0/2077098271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7378004810 0x7f7378004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:47.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f73919f7640 1 -- 172.21.15.83:0/2077098271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7378003530 con 0x7f73780039f0 2024-04-17T10:53:47.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f73921f8640 1 --2- 172.21.15.83:0/2077098271 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7378005220 0x7f737800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:53:47.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.749+0000 7f73921f8640 1 -- 172.21.15.83:0/2077098271 reap_dead start 2024-04-17T10:53:47.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.753+0000 7f73919f7640 1 --2- 172.21.15.83:0/2077098271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73780039f0 0x7f7378003e10 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f738800c190 tx=0x7f738801cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34126041f49251ca server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:47.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.753+0000 7f73909f5640 1 -- 172.21.15.83:0/2077098271 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7388007d00 con 0x7f73780039f0 2024-04-17T10:53:47.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.753+0000 7f73909f5640 1 -- 172.21.15.83:0/2077098271 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73880a9020 con 0x7f73780039f0 2024-04-17T10:53:47.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.753+0000 7f73909f5640 1 -- 172.21.15.83:0/2077098271 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7388005ae0 con 0x7f73780039f0 2024-04-17T10:53:47.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.753+0000 7f7393c5b640 1 -- 172.21.15.83:0/2077098271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73780039f0 msgr2=0x7f7378003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:47.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.753+0000 7f7393c5b640 1 --2- 172.21.15.83:0/2077098271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73780039f0 0x7f7378003e10 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f738800c190 tx=0x7f738801cbb0 comp rx=0 tx=0).stop 2024-04-17T10:53:47.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.753+0000 7f7393c5b640 1 -- 172.21.15.83:0/2077098271 shutdown_connections 2024-04-17T10:53:47.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.753+0000 7f7393c5b640 1 --2- 172.21.15.83:0/2077098271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73780039f0 0x7f7378003e10 unknown :-1 s=CLOSED pgs=2333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:47.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.753+0000 7f7393c5b640 1 -- 172.21.15.83:0/2077098271 >> 172.21.15.83:0/2077098271 conn(0x7f738c06abe0 msgr2=0x7f738c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:47.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.753+0000 7f7393c5b640 1 -- 172.21.15.83:0/2077098271 shutdown_connections 2024-04-17T10:53:47.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.753+0000 7f7393c5b640 1 -- 172.21.15.83:0/2077098271 wait complete. 2024-04-17T10:53:47.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.753+0000 7f7393c5b640 1 Processor -- start 2024-04-17T10:53:47.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.753+0000 7f7393c5b640 1 -- start start 2024-04-17T10:53:47.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.757+0000 7f7393c5b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73780039f0 0x7f738c0ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:47.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.757+0000 7f7393c5b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c156970 0x7f738c1560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:47.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.757+0000 7f7393c5b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f738c1594a0 0x7f738c06c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:47.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.757+0000 7f7393c5b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738c15e0b0 con 0x7f738c156970 2024-04-17T10:53:47.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.757+0000 7f7393c5b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738c15df30 con 0x7f738c1594a0 2024-04-17T10:53:47.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.757+0000 7f7393c5b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f738c0cfd60 con 0x7f73780039f0 2024-04-17T10:53:47.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.757+0000 7f73911f6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c156970 0x7f738c1560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:47.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.757+0000 7f73911f6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c156970 0x7f738c1560b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35396/0 (socket says 172.21.15.83:35396) 2024-04-17T10:53:47.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.757+0000 7f73911f6640 1 -- 172.21.15.83:0/2100331138 learned_addr learned my addr 172.21.15.83:0/2100331138 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:47.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.757+0000 7f73919f7640 1 --2- 172.21.15.83:0/2100331138 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73780039f0 0x7f738c0ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:47.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.757+0000 7f73921f8640 1 --2- 172.21.15.83:0/2100331138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f738c1594a0 0x7f738c06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:47.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.757+0000 7f73911f6640 1 -- 172.21.15.83:0/2100331138 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73780039f0 msgr2=0x7f738c0ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:47.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.757+0000 7f73911f6640 1 --2- 172.21.15.83:0/2100331138 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73780039f0 0x7f738c0ade00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:47.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.757+0000 7f73911f6640 1 -- 172.21.15.83:0/2100331138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f738c1594a0 msgr2=0x7f738c06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:47.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.757+0000 7f73911f6640 1 --2- 172.21.15.83:0/2100331138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f738c1594a0 0x7f738c06c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:47.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.757+0000 7f73911f6640 1 -- 172.21.15.83:0/2100331138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f738c1e6340 con 0x7f738c156970 2024-04-17T10:53:47.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.757+0000 7f73921f8640 1 --2- 172.21.15.83:0/2100331138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f738c1594a0 0x7f738c06c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:53:47.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.757+0000 7f73911f6640 1 --2- 172.21.15.83:0/2100331138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c156970 0x7f738c1560b0 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7f737c00e4d0 tx=0x7f737c00e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:47.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.757+0000 7f73827fc640 1 -- 172.21.15.83:0/2100331138 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c0053a0 con 0x7f738c156970 2024-04-17T10:53:47.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.757+0000 7f73827fc640 1 -- 172.21.15.83:0/2100331138 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737c018070 con 0x7f738c156970 2024-04-17T10:53:47.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.757+0000 7f7393c5b640 1 -- 172.21.15.83:0/2100331138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f738c1e6620 con 0x7f738c156970 2024-04-17T10:53:47.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.757+0000 7f73827fc640 1 -- 172.21.15.83:0/2100331138 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c003820 con 0x7f738c156970 2024-04-17T10:53:47.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.757+0000 7f7393c5b640 1 -- 172.21.15.83:0/2100331138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f738c1e6be0 con 0x7f738c156970 2024-04-17T10:53:47.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.757+0000 7f7393c5b640 1 -- 172.21.15.83:0/2100331138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f738c074f90 con 0x7f738c156970 2024-04-17T10:53:47.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.757+0000 7f73827fc640 1 -- 172.21.15.83:0/2100331138 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f737c0039c0 con 0x7f738c156970 2024-04-17T10:53:47.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.757+0000 7f73827fc640 1 --2- 172.21.15.83:0/2100331138 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7370066f60 0x7f7370069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:47.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.757+0000 7f73827fc640 1 -- 172.21.15.83:0/2100331138 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f737c084d30 con 0x7f738c156970 2024-04-17T10:53:47.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.757+0000 7f73919f7640 1 --2- 172.21.15.83:0/2100331138 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7370066f60 0x7f7370069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:47.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.761+0000 7f73919f7640 1 --2- 172.21.15.83:0/2100331138 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7370066f60 0x7f7370069420 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f73880aa000 tx=0x7f738c15e8f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:47.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.765+0000 7f73827fc640 1 -- 172.21.15.83:0/2100331138 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f737c055cd0 con 0x7f738c156970 2024-04-17T10:53:47.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.945+0000 7f7393c5b640 1 -- 172.21.15.83:0/2100331138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f738c0b2550 con 0x7f738c156970 2024-04-17T10:53:47.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.945+0000 7f73827fc640 1 -- 172.21.15.83:0/2100331138 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f737c0555d0 con 0x7f738c156970 2024-04-17T10:53:47.938 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:47.938 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:47.938 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:47.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.949+0000 7f7393c5b640 1 -- 172.21.15.83:0/2100331138 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7370066f60 msgr2=0x7f7370069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:47.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.949+0000 7f7393c5b640 1 --2- 172.21.15.83:0/2100331138 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7370066f60 0x7f7370069420 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f73880aa000 tx=0x7f738c15e8f0 comp rx=0 tx=0).stop 2024-04-17T10:53:47.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.949+0000 7f7393c5b640 1 -- 172.21.15.83:0/2100331138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c156970 msgr2=0x7f738c1560b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:47.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.949+0000 7f7393c5b640 1 --2- 172.21.15.83:0/2100331138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c156970 0x7f738c1560b0 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7f737c00e4d0 tx=0x7f737c00e990 comp rx=0 tx=0).stop 2024-04-17T10:53:47.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.949+0000 7f7393c5b640 1 -- 172.21.15.83:0/2100331138 shutdown_connections 2024-04-17T10:53:47.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.949+0000 7f7393c5b640 1 --2- 172.21.15.83:0/2100331138 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7370066f60 0x7f7370069420 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:47.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.949+0000 7f7393c5b640 1 --2- 172.21.15.83:0/2100331138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f738c1594a0 0x7f738c06c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:47.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.949+0000 7f7393c5b640 1 --2- 172.21.15.83:0/2100331138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f738c156970 0x7f738c1560b0 unknown :-1 s=CLOSED pgs=2334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:47.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.949+0000 7f7393c5b640 1 --2- 172.21.15.83:0/2100331138 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73780039f0 0x7f738c0ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:47.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.949+0000 7f7393c5b640 1 -- 172.21.15.83:0/2100331138 >> 172.21.15.83:0/2100331138 conn(0x7f738c06abe0 msgr2=0x7f738c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:47.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.949+0000 7f7393c5b640 1 -- 172.21.15.83:0/2100331138 shutdown_connections 2024-04-17T10:53:47.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:47.949+0000 7f7393c5b640 1 -- 172.21.15.83:0/2100331138 wait complete. 2024-04-17T10:53:48.358 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:48.358 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:49.359 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:49.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.496+0000 7f9860f88640 1 Processor -- start 2024-04-17T10:53:49.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.496+0000 7f9860f88640 1 -- start start 2024-04-17T10:53:49.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.496+0000 7f9860f88640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f985c0ce980 0x7f985c0b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:49.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.496+0000 7f9860f88640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f985c0cee00 0x7f985c0b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:49.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.496+0000 7f9860f88640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f985c162a00 con 0x7f985c15dfa0 2024-04-17T10:53:49.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.496+0000 7f9860f88640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f985c161e80 con 0x7f985c0ce980 2024-04-17T10:53:49.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.496+0000 7f9860f88640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985c162160 con 0x7f985c0cee00 2024-04-17T10:53:49.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.496+0000 7f9859d9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f985c0ce980 0x7f985c0b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:49.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.496+0000 7f9859d9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f985c0ce980 0x7f985c0b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35406/0 (socket says 172.21.15.83:35406) 2024-04-17T10:53:49.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.496+0000 7f9859d9b640 1 -- 172.21.15.83:0/1000172064 learned_addr learned my addr 172.21.15.83:0/1000172064 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:49.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.496+0000 7f985ad9d640 1 --2- 172.21.15.83:0/1000172064 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f985c0cee00 0x7f985c0b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:49.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.496+0000 7f9859d9b640 1 -- 172.21.15.83:0/1000172064 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f985c0cee00 msgr2=0x7f985c0b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:49.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.496+0000 7f9859d9b640 1 --2- 172.21.15.83:0/1000172064 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f985c0cee00 0x7f985c0b5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:49.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.496+0000 7f9859d9b640 1 -- 172.21.15.83:0/1000172064 >> v1:172.21.15.167:6789/0 conn(0x7f985c15dfa0 legacy=0x7f985c0b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:49.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.496+0000 7f9859d9b640 1 -- 172.21.15.83:0/1000172064 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985c162440 con 0x7f985c0ce980 2024-04-17T10:53:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.500+0000 7f9859d9b640 1 --2- 172.21.15.83:0/1000172064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f985c0ce980 0x7f985c0b5430 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f984c009870 tx=0x7f984c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9594cc7fe1ab01fa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.500+0000 7f985959a640 1 -- 172.21.15.83:0/1000172064 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c0b1070 con 0x7f985c0ce980 2024-04-17T10:53:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.500+0000 7f985959a640 1 -- 172.21.15.83:0/1000172064 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984c0ac620 con 0x7f985c0ce980 2024-04-17T10:53:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.500+0000 7f985959a640 1 -- 172.21.15.83:0/1000172064 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c0ac9f0 con 0x7f985c0ce980 2024-04-17T10:53:49.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.500+0000 7f9860f88640 1 -- 172.21.15.83:0/1000172064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f985c0ce980 msgr2=0x7f985c0b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:49.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.500+0000 7f9860f88640 1 --2- 172.21.15.83:0/1000172064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f985c0ce980 0x7f985c0b5430 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f984c009870 tx=0x7f984c01c460 comp rx=0 tx=0).stop 2024-04-17T10:53:49.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.500+0000 7f9860f88640 1 -- 172.21.15.83:0/1000172064 shutdown_connections 2024-04-17T10:53:49.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.500+0000 7f9860f88640 1 --2- 172.21.15.83:0/1000172064 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f985c0cee00 0x7f985c0b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:49.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.500+0000 7f9860f88640 1 --2- 172.21.15.83:0/1000172064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f985c0ce980 0x7f985c0b5430 secure :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f984c009870 tx=0x7f984c01c460 comp rx=0 tx=0).stop 2024-04-17T10:53:49.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.500+0000 7f9860f88640 1 -- 172.21.15.83:0/1000172064 >> 172.21.15.83:0/1000172064 conn(0x7f985c0d0600 msgr2=0x7f985c0d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:49.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.500+0000 7f9860f88640 1 -- 172.21.15.83:0/1000172064 shutdown_connections 2024-04-17T10:53:49.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.500+0000 7f9860f88640 1 -- 172.21.15.83:0/1000172064 wait complete. 2024-04-17T10:53:49.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.500+0000 7f9860f88640 1 Processor -- start 2024-04-17T10:53:49.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.500+0000 7f9860f88640 1 -- start start 2024-04-17T10:53:49.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.500+0000 7f9860f88640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f985c0cee00 0x7f985c14d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:49.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.500+0000 7f9860f88640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f985c15dfa0 0x7f985c0cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:49.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.500+0000 7f9860f88640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f985c160da0 0x7f985c15a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:49.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.500+0000 7f9860f88640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985c0d26c0 con 0x7f985c0cee00 2024-04-17T10:53:49.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.500+0000 7f9860f88640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985c0d2540 con 0x7f985c160da0 2024-04-17T10:53:49.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.500+0000 7f9860f88640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985c0d2840 con 0x7f985c15dfa0 2024-04-17T10:53:49.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.500+0000 7f985a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f985c0cee00 0x7f985c14d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:49.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.500+0000 7f985a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f985c0cee00 0x7f985c14d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35420/0 (socket says 172.21.15.83:35420) 2024-04-17T10:53:49.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.500+0000 7f985a59c640 1 -- 172.21.15.83:0/2342296676 learned_addr learned my addr 172.21.15.83:0/2342296676 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:49.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.500+0000 7f9859d9b640 1 --2- 172.21.15.83:0/2342296676 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f985c15dfa0 0x7f985c0cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:49.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.500+0000 7f985ad9d640 1 --2- 172.21.15.83:0/2342296676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f985c160da0 0x7f985c15a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:49.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.504+0000 7f985a59c640 1 -- 172.21.15.83:0/2342296676 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f985c15dfa0 msgr2=0x7f985c0cd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:49.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.504+0000 7f985a59c640 1 --2- 172.21.15.83:0/2342296676 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f985c15dfa0 0x7f985c0cd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:49.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.504+0000 7f985a59c640 1 -- 172.21.15.83:0/2342296676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f985c160da0 msgr2=0x7f985c15a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:49.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.504+0000 7f985a59c640 1 --2- 172.21.15.83:0/2342296676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f985c160da0 0x7f985c15a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:49.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.504+0000 7f985a59c640 1 -- 172.21.15.83:0/2342296676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985c0cd8e0 con 0x7f985c0cee00 2024-04-17T10:53:49.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.504+0000 7f985a59c640 1 --2- 172.21.15.83:0/2342296676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f985c0cee00 0x7f985c14d0e0 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f985000c450 tx=0x7f985000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:49.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.504+0000 7f984affd640 1 -- 172.21.15.83:0/2342296676 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9850011790 con 0x7f985c0cee00 2024-04-17T10:53:49.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.504+0000 7f984affd640 1 -- 172.21.15.83:0/2342296676 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9850017070 con 0x7f985c0cee00 2024-04-17T10:53:49.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.504+0000 7f984affd640 1 -- 172.21.15.83:0/2342296676 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9850011b90 con 0x7f985c0cee00 2024-04-17T10:53:49.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.504+0000 7f9860f88640 1 -- 172.21.15.83:0/2342296676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f985c0cc300 con 0x7f985c0cee00 2024-04-17T10:53:49.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.504+0000 7f9860f88640 1 -- 172.21.15.83:0/2342296676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f985c0cc8c0 con 0x7f985c0cee00 2024-04-17T10:53:49.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.504+0000 7f9860f88640 1 -- 172.21.15.83:0/2342296676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f985c074f90 con 0x7f985c0cee00 2024-04-17T10:53:49.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.504+0000 7f984affd640 1 -- 172.21.15.83:0/2342296676 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9850011d30 con 0x7f985c0cee00 2024-04-17T10:53:49.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.504+0000 7f984affd640 1 --2- 172.21.15.83:0/2342296676 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f983c066f60 0x7f983c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:49.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.504+0000 7f984affd640 1 -- 172.21.15.83:0/2342296676 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f9850084e50 con 0x7f985c0cee00 2024-04-17T10:53:49.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.504+0000 7f9859d9b640 1 --2- 172.21.15.83:0/2342296676 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f983c066f60 0x7f983c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:49.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.504+0000 7f9859d9b640 1 --2- 172.21.15.83:0/2342296676 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f983c066f60 0x7f983c069420 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f984c0ae000 tx=0x7f984c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:49.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.508+0000 7f984affd640 1 -- 172.21.15.83:0/2342296676 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9850055e80 con 0x7f985c0cee00 2024-04-17T10:53:49.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.696+0000 7f9860f88640 1 -- 172.21.15.83:0/2342296676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f985c0d17a0 con 0x7f985c0cee00 2024-04-17T10:53:49.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.696+0000 7f984affd640 1 -- 172.21.15.83:0/2342296676 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f9850055780 con 0x7f985c0cee00 2024-04-17T10:53:49.689 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:49.689 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:49.689 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:49.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.696+0000 7f9860f88640 1 -- 172.21.15.83:0/2342296676 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f983c066f60 msgr2=0x7f983c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:49.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.696+0000 7f9860f88640 1 --2- 172.21.15.83:0/2342296676 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f983c066f60 0x7f983c069420 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f984c0ae000 tx=0x7f984c0a7000 comp rx=0 tx=0).stop 2024-04-17T10:53:49.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.696+0000 7f9860f88640 1 -- 172.21.15.83:0/2342296676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f985c0cee00 msgr2=0x7f985c14d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:49.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.696+0000 7f9860f88640 1 --2- 172.21.15.83:0/2342296676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f985c0cee00 0x7f985c14d0e0 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f985000c450 tx=0x7f985000c910 comp rx=0 tx=0).stop 2024-04-17T10:53:49.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.700+0000 7f9860f88640 1 -- 172.21.15.83:0/2342296676 shutdown_connections 2024-04-17T10:53:49.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.700+0000 7f9860f88640 1 --2- 172.21.15.83:0/2342296676 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f983c066f60 0x7f983c069420 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:49.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.700+0000 7f9860f88640 1 --2- 172.21.15.83:0/2342296676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f985c160da0 0x7f985c15a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:49.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.700+0000 7f9860f88640 1 --2- 172.21.15.83:0/2342296676 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f985c15dfa0 0x7f985c0cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:49.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.700+0000 7f9860f88640 1 --2- 172.21.15.83:0/2342296676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f985c0cee00 0x7f985c14d0e0 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:49.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.700+0000 7f9860f88640 1 -- 172.21.15.83:0/2342296676 >> 172.21.15.83:0/2342296676 conn(0x7f985c0d0600 msgr2=0x7f985c1599d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:49.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.700+0000 7f9860f88640 1 -- 172.21.15.83:0/2342296676 shutdown_connections 2024-04-17T10:53:49.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:49.700+0000 7f9860f88640 1 -- 172.21.15.83:0/2342296676 wait complete. 2024-04-17T10:53:50.109 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:50.109 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:51.110 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:51.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.248+0000 7fc8f0c6a640 1 Processor -- start 2024-04-17T10:53:51.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.248+0000 7fc8f0c6a640 1 -- start start 2024-04-17T10:53:51.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.248+0000 7fc8f0c6a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc8ec15db50 0x7fc8ec070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:51.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.248+0000 7fc8f0c6a640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc8ec0aedd0 0x7fc8ec06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:51.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.248+0000 7fc8f0c6a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8ec06dae0 0x7fc8ec0cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:51.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.248+0000 7fc8f0c6a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8ec0709b0 con 0x7fc8ec15db50 2024-04-17T10:53:51.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.248+0000 7fc8f0c6a640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8ec15e440 con 0x7fc8ec0aedd0 2024-04-17T10:53:51.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.248+0000 7fc8f0c6a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec15e720 con 0x7fc8ec06dae0 2024-04-17T10:53:51.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.248+0000 7fc8ea59c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc8ec15db50 0x7fc8ec070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:51.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.248+0000 7fc8ea59c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc8ec15db50 0x7fc8ec070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35436/0 (socket says 172.21.15.83:35436) 2024-04-17T10:53:51.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.248+0000 7fc8ea59c640 1 -- 172.21.15.83:0/3129347564 learned_addr learned my addr 172.21.15.83:0/3129347564 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:51.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.248+0000 7fc8e9d9b640 1 --2- 172.21.15.83:0/3129347564 >> v2:172.21.15.167:3300/0 conn(0x7fc8ec0aedd0 0x7fc8ec06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:51.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.248+0000 7fc8ead9d640 1 --2- 172.21.15.83:0/3129347564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8ec06dae0 0x7fc8ec0cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:51.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.248+0000 7fc8ea59c640 1 -- 172.21.15.83:0/3129347564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8ec06dae0 msgr2=0x7fc8ec0cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:51.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.248+0000 7fc8ea59c640 1 --2- 172.21.15.83:0/3129347564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8ec06dae0 0x7fc8ec0cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:51.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.248+0000 7fc8ea59c640 1 -- 172.21.15.83:0/3129347564 >> v2:172.21.15.167:3300/0 conn(0x7fc8ec0aedd0 msgr2=0x7fc8ec06d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:51.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.248+0000 7fc8ea59c640 1 --2- 172.21.15.83:0/3129347564 >> v2:172.21.15.167:3300/0 conn(0x7fc8ec0aedd0 0x7fc8ec06d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:51.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.248+0000 7fc8ea59c640 1 -- 172.21.15.83:0/3129347564 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8ec15ea00 con 0x7fc8ec15db50 2024-04-17T10:53:51.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.248+0000 7fc8ead9d640 1 --2- 172.21.15.83:0/3129347564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8ec06dae0 0x7fc8ec0cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:53:51.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.248+0000 7fc8e9d9b640 1 --2- 172.21.15.83:0/3129347564 >> v2:172.21.15.167:3300/0 conn(0x7fc8ec0aedd0 0x7fc8ec06d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:53:51.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.252+0000 7fc8ea59c640 1 --2- 172.21.15.83:0/3129347564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8ec15db50 0x7fc8ec070470 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec0b4b40 tx=0x7fc8dc01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f3cad8ad470e8b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:51.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.252+0000 7fc8e959a640 1 -- 172.21.15.83:0/3129347564 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc0b0070 con 0x7fc8ec15db50 2024-04-17T10:53:51.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.252+0000 7fc8e959a640 1 -- 172.21.15.83:0/3129347564 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8dc0ab600 con 0x7fc8ec15db50 2024-04-17T10:53:51.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.252+0000 7fc8e959a640 1 -- 172.21.15.83:0/3129347564 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc0ab9d0 con 0x7fc8ec15db50 2024-04-17T10:53:51.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.252+0000 7fc8f0c6a640 1 -- 172.21.15.83:0/3129347564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8ec15db50 msgr2=0x7fc8ec070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:51.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.252+0000 7fc8f0c6a640 1 --2- 172.21.15.83:0/3129347564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8ec15db50 0x7fc8ec070470 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec0b4b40 tx=0x7fc8dc01c260 comp rx=0 tx=0).stop 2024-04-17T10:53:51.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.252+0000 7fc8f0c6a640 1 -- 172.21.15.83:0/3129347564 shutdown_connections 2024-04-17T10:53:51.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.252+0000 7fc8f0c6a640 1 --2- 172.21.15.83:0/3129347564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8ec06dae0 0x7fc8ec0cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:51.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.252+0000 7fc8f0c6a640 1 --2- 172.21.15.83:0/3129347564 >> v2:172.21.15.167:3300/0 conn(0x7fc8ec0aedd0 0x7fc8ec06d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:51.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.252+0000 7fc8f0c6a640 1 --2- 172.21.15.83:0/3129347564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8ec15db50 0x7fc8ec070470 unknown :-1 s=CLOSED pgs=2337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:51.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.252+0000 7fc8f0c6a640 1 -- 172.21.15.83:0/3129347564 >> 172.21.15.83:0/3129347564 conn(0x7fc8ec06abe0 msgr2=0x7fc8ec06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:51.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.252+0000 7fc8f0c6a640 1 -- 172.21.15.83:0/3129347564 shutdown_connections 2024-04-17T10:53:51.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.252+0000 7fc8f0c6a640 1 -- 172.21.15.83:0/3129347564 wait complete. 2024-04-17T10:53:51.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.256+0000 7fc8f0c6a640 1 Processor -- start 2024-04-17T10:53:51.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.256+0000 7fc8f0c6a640 1 -- start start 2024-04-17T10:53:51.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.256+0000 7fc8f0c6a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8ec06dae0 0x7fc8ec157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:51.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.256+0000 7fc8f0c6a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8ec0aedd0 0x7fc8ec0b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:51.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.256+0000 7fc8f0c6a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8ec15db50 0x7fc8ec0b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:51.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.256+0000 7fc8f0c6a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec0b2f90 con 0x7fc8ec15db50 2024-04-17T10:53:51.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.256+0000 7fc8f0c6a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec0b2e10 con 0x7fc8ec06dae0 2024-04-17T10:53:51.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.256+0000 7fc8f0c6a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec0b3110 con 0x7fc8ec0aedd0 2024-04-17T10:53:51.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.256+0000 7fc8ead9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8ec15db50 0x7fc8ec0b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:51.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.256+0000 7fc8ead9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8ec15db50 0x7fc8ec0b1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35442/0 (socket says 172.21.15.83:35442) 2024-04-17T10:53:51.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.256+0000 7fc8ead9d640 1 -- 172.21.15.83:0/717347745 learned_addr learned my addr 172.21.15.83:0/717347745 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:51.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.256+0000 7fc8e9d9b640 1 --2- 172.21.15.83:0/717347745 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8ec0aedd0 0x7fc8ec0b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:51.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.256+0000 7fc8ea59c640 1 --2- 172.21.15.83:0/717347745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8ec06dae0 0x7fc8ec157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:51.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.256+0000 7fc8ead9d640 1 -- 172.21.15.83:0/717347745 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8ec0aedd0 msgr2=0x7fc8ec0b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:51.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.256+0000 7fc8ead9d640 1 --2- 172.21.15.83:0/717347745 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8ec0aedd0 0x7fc8ec0b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:51.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.256+0000 7fc8ead9d640 1 -- 172.21.15.83:0/717347745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8ec06dae0 msgr2=0x7fc8ec157e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:51.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.256+0000 7fc8ead9d640 1 --2- 172.21.15.83:0/717347745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8ec06dae0 0x7fc8ec157e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:51.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.256+0000 7fc8ead9d640 1 -- 172.21.15.83:0/717347745 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8ec158a20 con 0x7fc8ec15db50 2024-04-17T10:53:51.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.256+0000 7fc8ea59c640 1 --2- 172.21.15.83:0/717347745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8ec06dae0 0x7fc8ec157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:53:51.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.256+0000 7fc8e9d9b640 1 --2- 172.21.15.83:0/717347745 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8ec0aedd0 0x7fc8ec0b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:53:51.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.256+0000 7fc8ead9d640 1 --2- 172.21.15.83:0/717347745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8ec15db50 0x7fc8ec0b1890 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400cf20 tx=0x7fc8e400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:51.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.256+0000 7fc8daffd640 1 -- 172.21.15.83:0/717347745 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e401a070 con 0x7fc8ec15db50 2024-04-17T10:53:51.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.256+0000 7fc8daffd640 1 -- 172.21.15.83:0/717347745 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e400d840 con 0x7fc8ec15db50 2024-04-17T10:53:51.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.256+0000 7fc8f0c6a640 1 -- 172.21.15.83:0/717347745 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8ec0ae270 con 0x7fc8ec15db50 2024-04-17T10:53:51.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.256+0000 7fc8daffd640 1 -- 172.21.15.83:0/717347745 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e400dc30 con 0x7fc8ec15db50 2024-04-17T10:53:51.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.256+0000 7fc8f0c6a640 1 -- 172.21.15.83:0/717347745 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8ec0ad4b0 con 0x7fc8ec15db50 2024-04-17T10:53:51.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.256+0000 7fc8f0c6a640 1 -- 172.21.15.83:0/717347745 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8ec163e10 con 0x7fc8ec15db50 2024-04-17T10:53:51.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.260+0000 7fc8daffd640 1 -- 172.21.15.83:0/717347745 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc8e401d070 con 0x7fc8ec15db50 2024-04-17T10:53:51.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.260+0000 7fc8daffd640 1 --2- 172.21.15.83:0/717347745 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc8cc066f60 0x7fc8cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:51.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.260+0000 7fc8daffd640 1 -- 172.21.15.83:0/717347745 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fc8e4085bf0 con 0x7fc8ec15db50 2024-04-17T10:53:51.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.260+0000 7fc8ea59c640 1 --2- 172.21.15.83:0/717347745 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc8cc066f60 0x7fc8cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:51.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.260+0000 7fc8ea59c640 1 --2- 172.21.15.83:0/717347745 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc8cc066f60 0x7fc8cc069420 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0ad000 tx=0x7fc8dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:51.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.264+0000 7fc8daffd640 1 -- 172.21.15.83:0/717347745 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc8e4024020 con 0x7fc8ec15db50 2024-04-17T10:53:51.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.452+0000 7fc8f0c6a640 1 -- 172.21.15.83:0/717347745 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8ec0b2af0 con 0x7fc8ec15db50 2024-04-17T10:53:51.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.456+0000 7fc8daffd640 1 -- 172.21.15.83:0/717347745 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fc8ec158a20 con 0x7fc8ec15db50 2024-04-17T10:53:51.448 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:51.448 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:51.448 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:51.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.456+0000 7fc8f0c6a640 1 -- 172.21.15.83:0/717347745 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc8cc066f60 msgr2=0x7fc8cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:51.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.456+0000 7fc8f0c6a640 1 --2- 172.21.15.83:0/717347745 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc8cc066f60 0x7fc8cc069420 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0ad000 tx=0x7fc8dc008040 comp rx=0 tx=0).stop 2024-04-17T10:53:51.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.456+0000 7fc8f0c6a640 1 -- 172.21.15.83:0/717347745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8ec15db50 msgr2=0x7fc8ec0b1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:51.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.456+0000 7fc8f0c6a640 1 --2- 172.21.15.83:0/717347745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8ec15db50 0x7fc8ec0b1890 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400cf20 tx=0x7fc8e400a710 comp rx=0 tx=0).stop 2024-04-17T10:53:51.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.456+0000 7fc8f0c6a640 1 -- 172.21.15.83:0/717347745 shutdown_connections 2024-04-17T10:53:51.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.456+0000 7fc8f0c6a640 1 --2- 172.21.15.83:0/717347745 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc8cc066f60 0x7fc8cc069420 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:51.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.456+0000 7fc8f0c6a640 1 --2- 172.21.15.83:0/717347745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8ec15db50 0x7fc8ec0b1890 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:51.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.456+0000 7fc8f0c6a640 1 --2- 172.21.15.83:0/717347745 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8ec0aedd0 0x7fc8ec0b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:51.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.456+0000 7fc8f0c6a640 1 --2- 172.21.15.83:0/717347745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8ec06dae0 0x7fc8ec157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:51.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.456+0000 7fc8f0c6a640 1 -- 172.21.15.83:0/717347745 >> 172.21.15.83:0/717347745 conn(0x7fc8ec06abe0 msgr2=0x7fc8ec070d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:51.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.456+0000 7fc8f0c6a640 1 -- 172.21.15.83:0/717347745 shutdown_connections 2024-04-17T10:53:51.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:51.456+0000 7fc8f0c6a640 1 -- 172.21.15.83:0/717347745 wait complete. 2024-04-17T10:53:51.869 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:51.869 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:52.870 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:52.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.008+0000 7fa1cbdde640 1 Processor -- start 2024-04-17T10:53:52.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.008+0000 7fa1cbdde640 1 -- start start 2024-04-17T10:53:53.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.008+0000 7fa1cbdde640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1c4072330 0x7fa1c40cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:53.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.008+0000 7fa1cbdde640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa1c40cfa10 0x7fa1c406c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:53.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.008+0000 7fa1cbdde640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1c40cfea0 con 0x7fa1c415ba30 2024-04-17T10:53:53.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.008+0000 7fa1cbdde640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1c4071ea0 con 0x7fa1c40cfa10 2024-04-17T10:53:53.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.008+0000 7fa1cbdde640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c406e320 con 0x7fa1c4072330 2024-04-17T10:53:53.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.008+0000 7fa1ca37b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa1c40cfa10 0x7fa1c406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:53.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.008+0000 7fa1ca37b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa1c40cfa10 0x7fa1c406c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35458/0 (socket says 172.21.15.83:35458) 2024-04-17T10:53:53.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.008+0000 7fa1ca37b640 1 -- 172.21.15.83:0/4255504497 learned_addr learned my addr 172.21.15.83:0/4255504497 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:53.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.008+0000 7fa1c9379640 1 --2- 172.21.15.83:0/4255504497 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1c4072330 0x7fa1c40cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:53.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.008+0000 7fa1ca37b640 1 -- 172.21.15.83:0/4255504497 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1c4072330 msgr2=0x7fa1c40cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:53.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.008+0000 7fa1ca37b640 1 --2- 172.21.15.83:0/4255504497 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1c4072330 0x7fa1c40cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:53.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.008+0000 7fa1ca37b640 1 -- 172.21.15.83:0/4255504497 >> v1:172.21.15.167:6789/0 conn(0x7fa1c415ba30 legacy=0x7fa1c415c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:53.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.008+0000 7fa1ca37b640 1 -- 172.21.15.83:0/4255504497 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c406e040 con 0x7fa1c40cfa10 2024-04-17T10:53:53.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.008+0000 7fa1c9379640 1 --2- 172.21.15.83:0/4255504497 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1c4072330 0x7fa1c40cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:53:53.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.012+0000 7fa1ca37b640 1 --2- 172.21.15.83:0/4255504497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1c40cfa10 0x7fa1c406c490 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc009870 tx=0x7fa1bc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81faf75e7f232db0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:53.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.012+0000 7fa1c8b78640 1 -- 172.21.15.83:0/4255504497 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc0b2070 con 0x7fa1c40cfa10 2024-04-17T10:53:53.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.012+0000 7fa1c8b78640 1 -- 172.21.15.83:0/4255504497 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1bc0a9070 con 0x7fa1c40cfa10 2024-04-17T10:53:53.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.012+0000 7fa1c8b78640 1 -- 172.21.15.83:0/4255504497 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc0ad880 con 0x7fa1c40cfa10 2024-04-17T10:53:53.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.012+0000 7fa1cbdde640 1 -- 172.21.15.83:0/4255504497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1c40cfa10 msgr2=0x7fa1c406c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:53.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.012+0000 7fa1cbdde640 1 --2- 172.21.15.83:0/4255504497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1c40cfa10 0x7fa1c406c490 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc009870 tx=0x7fa1bc01c490 comp rx=0 tx=0).stop 2024-04-17T10:53:53.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.012+0000 7fa1cbdde640 1 -- 172.21.15.83:0/4255504497 shutdown_connections 2024-04-17T10:53:53.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.012+0000 7fa1cbdde640 1 --2- 172.21.15.83:0/4255504497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1c40cfa10 0x7fa1c406c490 unknown :-1 s=CLOSED pgs=2339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:53.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.012+0000 7fa1cbdde640 1 --2- 172.21.15.83:0/4255504497 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1c4072330 0x7fa1c40cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:53.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.012+0000 7fa1cbdde640 1 -- 172.21.15.83:0/4255504497 >> 172.21.15.83:0/4255504497 conn(0x7fa1c406abe0 msgr2=0x7fa1c406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:53.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.012+0000 7fa1cbdde640 1 -- 172.21.15.83:0/4255504497 shutdown_connections 2024-04-17T10:53:53.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.012+0000 7fa1cbdde640 1 -- 172.21.15.83:0/4255504497 wait complete. 2024-04-17T10:53:53.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.012+0000 7fa1cbdde640 1 Processor -- start 2024-04-17T10:53:53.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.012+0000 7fa1cbdde640 1 -- start start 2024-04-17T10:53:53.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.012+0000 7fa1cbdde640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1c4072330 0x7fa1c40adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:53.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.012+0000 7fa1cbdde640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1c40cfa10 0x7fa1c415e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:53.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.012+0000 7fa1cbdde640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1c415ba30 0x7fa1c415c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:53.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.012+0000 7fa1cbdde640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c406de20 con 0x7fa1c4072330 2024-04-17T10:53:53.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.012+0000 7fa1cbdde640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c406dca0 con 0x7fa1c415ba30 2024-04-17T10:53:53.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.012+0000 7fa1cbdde640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c4168520 con 0x7fa1c40cfa10 2024-04-17T10:53:53.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.012+0000 7fa1c9b7a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1c4072330 0x7fa1c40adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:53.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.012+0000 7fa1c9b7a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1c4072330 0x7fa1c40adaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35466/0 (socket says 172.21.15.83:35466) 2024-04-17T10:53:53.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.012+0000 7fa1c9b7a640 1 -- 172.21.15.83:0/3201541524 learned_addr learned my addr 172.21.15.83:0/3201541524 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:53.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.012+0000 7fa1c9379640 1 --2- 172.21.15.83:0/3201541524 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1c40cfa10 0x7fa1c415e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:53.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.012+0000 7fa1c9b7a640 1 -- 172.21.15.83:0/3201541524 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1c40cfa10 msgr2=0x7fa1c415e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:53.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.012+0000 7fa1c9b7a640 1 --2- 172.21.15.83:0/3201541524 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1c40cfa10 0x7fa1c415e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:53.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.012+0000 7fa1c9b7a640 1 -- 172.21.15.83:0/3201541524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1c415ba30 msgr2=0x7fa1c415c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:53.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.012+0000 7fa1ca37b640 1 --2- 172.21.15.83:0/3201541524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1c415ba30 0x7fa1c415c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:53.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.012+0000 7fa1c9b7a640 1 --2- 172.21.15.83:0/3201541524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1c415ba30 0x7fa1c415c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:53.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.012+0000 7fa1c9b7a640 1 -- 172.21.15.83:0/3201541524 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c414f8b0 con 0x7fa1c4072330 2024-04-17T10:53:53.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.012+0000 7fa1c9379640 1 --2- 172.21.15.83:0/3201541524 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1c40cfa10 0x7fa1c415e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:53:53.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.016+0000 7fa1c9b7a640 1 --2- 172.21.15.83:0/3201541524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1c4072330 0x7fa1c40adaa0 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7fa1b400c7f0 tx=0x7fa1b400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:53.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.016+0000 7fa1c27fc640 1 -- 172.21.15.83:0/3201541524 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b4016020 con 0x7fa1c4072330 2024-04-17T10:53:53.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.016+0000 7fa1c27fc640 1 -- 172.21.15.83:0/3201541524 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b400aa50 con 0x7fa1c4072330 2024-04-17T10:53:53.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.016+0000 7fa1cbdde640 1 -- 172.21.15.83:0/3201541524 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1c413e240 con 0x7fa1c4072330 2024-04-17T10:53:53.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.016+0000 7fa1c27fc640 1 -- 172.21.15.83:0/3201541524 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b400ae40 con 0x7fa1c4072330 2024-04-17T10:53:53.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.016+0000 7fa1cbdde640 1 -- 172.21.15.83:0/3201541524 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1c413e800 con 0x7fa1c4072330 2024-04-17T10:53:53.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.016+0000 7fa1cbdde640 1 -- 172.21.15.83:0/3201541524 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1c4074f90 con 0x7fa1c4072330 2024-04-17T10:53:53.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.016+0000 7fa1c27fc640 1 -- 172.21.15.83:0/3201541524 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa1b4019020 con 0x7fa1c4072330 2024-04-17T10:53:53.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.016+0000 7fa1c27fc640 1 --2- 172.21.15.83:0/3201541524 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa1b0066f60 0x7fa1b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:53.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.016+0000 7fa1c27fc640 1 -- 172.21.15.83:0/3201541524 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fa1b4085150 con 0x7fa1c4072330 2024-04-17T10:53:53.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.016+0000 7fa1c9379640 1 --2- 172.21.15.83:0/3201541524 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa1b0066f60 0x7fa1b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:53.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.016+0000 7fa1c9379640 1 --2- 172.21.15.83:0/3201541524 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa1b0066f60 0x7fa1b0069420 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa1c414fc20 tx=0x7fa1b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:53.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.020+0000 7fa1c27fc640 1 -- 172.21.15.83:0/3201541524 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa1b4020030 con 0x7fa1c4072330 2024-04-17T10:53:53.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.208+0000 7fa1cbdde640 1 -- 172.21.15.83:0/3201541524 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1c406b300 con 0x7fa1c4072330 2024-04-17T10:53:53.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.208+0000 7fa1c27fc640 1 -- 172.21.15.83:0/3201541524 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fa1b40560f0 con 0x7fa1c4072330 2024-04-17T10:53:53.203 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:53.203 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:53.203 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:53.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.212+0000 7fa1cbdde640 1 -- 172.21.15.83:0/3201541524 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa1b0066f60 msgr2=0x7fa1b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:53.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.212+0000 7fa1cbdde640 1 --2- 172.21.15.83:0/3201541524 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa1b0066f60 0x7fa1b0069420 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa1c414fc20 tx=0x7fa1b8008040 comp rx=0 tx=0).stop 2024-04-17T10:53:53.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.212+0000 7fa1cbdde640 1 -- 172.21.15.83:0/3201541524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1c4072330 msgr2=0x7fa1c40adaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:53.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.212+0000 7fa1cbdde640 1 --2- 172.21.15.83:0/3201541524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1c4072330 0x7fa1c40adaa0 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7fa1b400c7f0 tx=0x7fa1b400ccb0 comp rx=0 tx=0).stop 2024-04-17T10:53:53.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.212+0000 7fa1cbdde640 1 -- 172.21.15.83:0/3201541524 shutdown_connections 2024-04-17T10:53:53.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.212+0000 7fa1cbdde640 1 --2- 172.21.15.83:0/3201541524 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa1b0066f60 0x7fa1b0069420 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:53.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.212+0000 7fa1cbdde640 1 --2- 172.21.15.83:0/3201541524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1c415ba30 0x7fa1c415c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:53.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.212+0000 7fa1cbdde640 1 --2- 172.21.15.83:0/3201541524 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1c40cfa10 0x7fa1c415e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:53.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.212+0000 7fa1cbdde640 1 --2- 172.21.15.83:0/3201541524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1c4072330 0x7fa1c40adaa0 unknown :-1 s=CLOSED pgs=2340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:53.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.212+0000 7fa1cbdde640 1 -- 172.21.15.83:0/3201541524 >> 172.21.15.83:0/3201541524 conn(0x7fa1c406abe0 msgr2=0x7fa1c40b0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:53.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.212+0000 7fa1cbdde640 1 -- 172.21.15.83:0/3201541524 shutdown_connections 2024-04-17T10:53:53.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:53.212+0000 7fa1cbdde640 1 -- 172.21.15.83:0/3201541524 wait complete. 2024-04-17T10:53:53.623 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:53.623 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:54.624 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:54.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.764+0000 7f89e0581640 1 Processor -- start 2024-04-17T10:53:54.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.764+0000 7f89e0581640 1 -- start start 2024-04-17T10:53:54.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.764+0000 7f89e0581640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f89d80cfbe0 0x7f89d806c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:54.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.764+0000 7f89e0581640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89d806e320 con 0x7f89d815ba30 2024-04-17T10:53:54.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.764+0000 7f89e0581640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89d806e040 con 0x7f89d8072330 2024-04-17T10:53:54.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.764+0000 7f89e0581640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89d8071ea0 con 0x7f89d80cfbe0 2024-04-17T10:53:54.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.764+0000 7f89deb1e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f89d80cfbe0 0x7f89d806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:54.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.764+0000 7f89deb1e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f89d80cfbe0 0x7f89d806c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35470/0 (socket says 172.21.15.83:35470) 2024-04-17T10:53:54.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.764+0000 7f89de31d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f89d815ba30 0x7f89d815c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35540/0 (socket says 172.21.15.83:35540) 2024-04-17T10:53:54.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.764+0000 7f89deb1e640 1 -- 172.21.15.83:0/4226291373 learned_addr learned my addr 172.21.15.83:0/4226291373 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:54.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.764+0000 7f89deb1e640 1 -- 172.21.15.83:0/4226291373 >> v1:172.21.15.167:6789/0 conn(0x7f89d8072330 legacy=0x7f89d80cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:54.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.764+0000 7f89deb1e640 1 -- 172.21.15.83:0/4226291373 >> v1:172.21.15.83:6789/0 conn(0x7f89d815ba30 legacy=0x7f89d815c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:54.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.764+0000 7f89deb1e640 1 -- 172.21.15.83:0/4226291373 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d8071850 con 0x7f89d80cfbe0 2024-04-17T10:53:54.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.764+0000 7f89dd31b640 1 -- 172.21.15.83:0/4226291373 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1990451206 0 0) 0x7f89d806e320 con 0x7f89d815ba30 2024-04-17T10:53:54.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89deb1e640 1 --2- 172.21.15.83:0/4226291373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89d80cfbe0 0x7f89d806c490 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f89d0009e50 tx=0x7f89d001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85f899fef0e16af server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:54.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89dd31b640 1 -- 172.21.15.83:0/4226291373 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d00a5730 con 0x7f89d80cfbe0 2024-04-17T10:53:54.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89dd31b640 1 -- 172.21.15.83:0/4226291373 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d00a58d0 con 0x7f89d80cfbe0 2024-04-17T10:53:54.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89dd31b640 1 -- 172.21.15.83:0/4226291373 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d00a5ca0 con 0x7f89d80cfbe0 2024-04-17T10:53:54.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89e0581640 1 -- 172.21.15.83:0/4226291373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89d80cfbe0 msgr2=0x7f89d806c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:54.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89e0581640 1 --2- 172.21.15.83:0/4226291373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89d80cfbe0 0x7f89d806c490 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f89d0009e50 tx=0x7f89d001c960 comp rx=0 tx=0).stop 2024-04-17T10:53:54.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89e0581640 1 -- 172.21.15.83:0/4226291373 shutdown_connections 2024-04-17T10:53:54.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89e0581640 1 --2- 172.21.15.83:0/4226291373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89d80cfbe0 0x7f89d806c490 unknown :-1 s=CLOSED pgs=2342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:54.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89e0581640 1 -- 172.21.15.83:0/4226291373 >> 172.21.15.83:0/4226291373 conn(0x7f89d806abe0 msgr2=0x7f89d806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:54.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89e0581640 1 -- 172.21.15.83:0/4226291373 shutdown_connections 2024-04-17T10:53:54.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89e0581640 1 -- 172.21.15.83:0/4226291373 wait complete. 2024-04-17T10:53:54.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89e0581640 1 Processor -- start 2024-04-17T10:53:54.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89e0581640 1 -- start start 2024-04-17T10:53:54.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89e0581640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89d8072330 0x7f89d814e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:54.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89e0581640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89d80cfbe0 0x7f89d8144400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:54.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89e0581640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89d815ba30 0x7f89d815c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:54.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89e0581640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d81696b0 con 0x7f89d815ba30 2024-04-17T10:53:54.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89e0581640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d8169530 con 0x7f89d80cfbe0 2024-04-17T10:53:54.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89e0581640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89d8169830 con 0x7f89d8072330 2024-04-17T10:53:54.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89deb1e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89d815ba30 0x7f89d815c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:54.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89deb1e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89d815ba30 0x7f89d815c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35472/0 (socket says 172.21.15.83:35472) 2024-04-17T10:53:54.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89deb1e640 1 -- 172.21.15.83:0/2832398627 learned_addr learned my addr 172.21.15.83:0/2832398627 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:54.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89de31d640 1 --2- 172.21.15.83:0/2832398627 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89d8072330 0x7f89d814e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:54.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89ddb1c640 1 --2- 172.21.15.83:0/2832398627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89d80cfbe0 0x7f89d8144400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:54.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89deb1e640 1 -- 172.21.15.83:0/2832398627 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89d8072330 msgr2=0x7f89d814e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:54.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89deb1e640 1 --2- 172.21.15.83:0/2832398627 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89d8072330 0x7f89d814e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:54.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89deb1e640 1 -- 172.21.15.83:0/2832398627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89d80cfbe0 msgr2=0x7f89d8144400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:54.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89deb1e640 1 --2- 172.21.15.83:0/2832398627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89d80cfbe0 0x7f89d8144400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:54.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89deb1e640 1 -- 172.21.15.83:0/2832398627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d81521e0 con 0x7f89d815ba30 2024-04-17T10:53:54.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89ddb1c640 1 --2- 172.21.15.83:0/2832398627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89d80cfbe0 0x7f89d8144400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:53:54.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89deb1e640 1 --2- 172.21.15.83:0/2832398627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89d815ba30 0x7f89d815c9c0 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f89d0009e50 tx=0x7f89d0019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:54.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89ceffd640 1 -- 172.21.15.83:0/2832398627 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d00bc030 con 0x7f89d815ba30 2024-04-17T10:53:54.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89ceffd640 1 -- 172.21.15.83:0/2832398627 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d00b8470 con 0x7f89d815ba30 2024-04-17T10:53:54.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89e0581640 1 -- 172.21.15.83:0/2832398627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89d81524c0 con 0x7f89d815ba30 2024-04-17T10:53:54.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89ceffd640 1 -- 172.21.15.83:0/2832398627 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d00b8820 con 0x7f89d815ba30 2024-04-17T10:53:54.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.768+0000 7f89e0581640 1 -- 172.21.15.83:0/2832398627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89d8152a80 con 0x7f89d815ba30 2024-04-17T10:53:54.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.772+0000 7f89e0581640 1 -- 172.21.15.83:0/2832398627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89d8147ca0 con 0x7f89d815ba30 2024-04-17T10:53:54.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.772+0000 7f89ceffd640 1 -- 172.21.15.83:0/2832398627 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f89d00bf070 con 0x7f89d815ba30 2024-04-17T10:53:54.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.772+0000 7f89ceffd640 1 --2- 172.21.15.83:0/2832398627 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f89bc066f60 0x7f89bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:54.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.772+0000 7f89ceffd640 1 -- 172.21.15.83:0/2832398627 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f89d011fbe0 con 0x7f89d815ba30 2024-04-17T10:53:54.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.772+0000 7f89de31d640 1 --2- 172.21.15.83:0/2832398627 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f89bc066f60 0x7f89bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:54.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.772+0000 7f89de31d640 1 --2- 172.21.15.83:0/2832398627 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f89bc066f60 0x7f89bc069420 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f89d4000f70 tx=0x7f89d4007c30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:54.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.776+0000 7f89ceffd640 1 -- 172.21.15.83:0/2832398627 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f89d00165e0 con 0x7f89d815ba30 2024-04-17T10:53:54.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.972+0000 7f89e0581640 1 -- 172.21.15.83:0/2832398627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89d8152cc0 con 0x7f89d815ba30 2024-04-17T10:53:54.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.972+0000 7f89ceffd640 1 -- 172.21.15.83:0/2832398627 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f89d8152cc0 con 0x7f89d815ba30 2024-04-17T10:53:54.965 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:54.965 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:54.965 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:54.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.972+0000 7f89e0581640 1 -- 172.21.15.83:0/2832398627 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f89bc066f60 msgr2=0x7f89bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:54.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.972+0000 7f89e0581640 1 --2- 172.21.15.83:0/2832398627 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f89bc066f60 0x7f89bc069420 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f89d4000f70 tx=0x7f89d4007c30 comp rx=0 tx=0).stop 2024-04-17T10:53:54.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.972+0000 7f89e0581640 1 -- 172.21.15.83:0/2832398627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89d815ba30 msgr2=0x7f89d815c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:54.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.972+0000 7f89e0581640 1 --2- 172.21.15.83:0/2832398627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89d815ba30 0x7f89d815c9c0 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f89d0009e50 tx=0x7f89d0019700 comp rx=0 tx=0).stop 2024-04-17T10:53:54.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.976+0000 7f89e0581640 1 -- 172.21.15.83:0/2832398627 shutdown_connections 2024-04-17T10:53:54.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.976+0000 7f89e0581640 1 --2- 172.21.15.83:0/2832398627 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f89bc066f60 0x7f89bc069420 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:54.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.976+0000 7f89e0581640 1 --2- 172.21.15.83:0/2832398627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89d815ba30 0x7f89d815c9c0 unknown :-1 s=CLOSED pgs=2343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:54.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.976+0000 7f89e0581640 1 --2- 172.21.15.83:0/2832398627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89d80cfbe0 0x7f89d8144400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:54.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.976+0000 7f89e0581640 1 --2- 172.21.15.83:0/2832398627 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89d8072330 0x7f89d814e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:54.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.976+0000 7f89e0581640 1 -- 172.21.15.83:0/2832398627 >> 172.21.15.83:0/2832398627 conn(0x7f89d806abe0 msgr2=0x7f89d8154050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:54.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.976+0000 7f89e0581640 1 -- 172.21.15.83:0/2832398627 shutdown_connections 2024-04-17T10:53:54.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:54.976+0000 7f89e0581640 1 -- 172.21.15.83:0/2832398627 wait complete. 2024-04-17T10:53:55.386 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:55.386 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:56.387 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:56.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.524+0000 7f48d9ce8640 1 Processor -- start 2024-04-17T10:53:56.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.524+0000 7f48d9ce8640 1 -- start start 2024-04-17T10:53:56.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d9ce8640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f48d4072330 0x7f48d40cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:56.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d9ce8640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48d40cfea0 con 0x7f48d415ba30 2024-04-17T10:53:56.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d9ce8640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48d406e320 con 0x7f48d40cfa10 2024-04-17T10:53:56.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d9ce8640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d4071ea0 con 0x7f48d4072330 2024-04-17T10:53:56.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d37fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f48d415ba30 0x7f48d415c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35554/0 (socket says 172.21.15.83:35554) 2024-04-17T10:53:56.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d37fe640 1 -- 172.21.15.83:0/1919411741 learned_addr learned my addr 172.21.15.83:0/1919411741 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:53:56.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d27fc640 1 -- 172.21.15.83:0/1919411741 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2392193844 0 0) 0x7f48d40cfea0 con 0x7f48d415ba30 2024-04-17T10:53:56.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d2ffd640 1 --2- 172.21.15.83:0/1919411741 >> v2:172.21.15.167:3300/0 conn(0x7f48d4072330 0x7f48d40cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:56.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d27fc640 1 -- 172.21.15.83:0/1919411741 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48bc003530 con 0x7f48d415ba30 2024-04-17T10:53:56.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d27fc640 1 -- 172.21.15.83:0/1919411741 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 328203200 0 0) 0x7f48bc003530 con 0x7f48d415ba30 2024-04-17T10:53:56.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d27fc640 1 -- 172.21.15.83:0/1919411741 >> v2:172.21.15.167:3300/0 conn(0x7f48d4072330 msgr2=0x7f48d40cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:56.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d27fc640 1 --2- 172.21.15.83:0/1919411741 >> v2:172.21.15.167:3300/0 conn(0x7f48d4072330 0x7f48d40cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:56.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d27fc640 1 -- 172.21.15.83:0/1919411741 >> v1:172.21.15.167:6789/0 conn(0x7f48d40cfa10 legacy=0x7f48d406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:56.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d27fc640 1 -- 172.21.15.83:0/1919411741 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d406e040 con 0x7f48d415ba30 2024-04-17T10:53:56.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d27fc640 1 -- 172.21.15.83:0/1919411741 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f48c8001680 con 0x7f48d415ba30 2024-04-17T10:53:56.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d27fc640 1 -- 172.21.15.83:0/1919411741 >> v1:172.21.15.83:6789/0 conn(0x7f48d415ba30 legacy=0x7f48d415c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:56.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d27fc640 1 --2- 172.21.15.83:0/1919411741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48bc0039c0 0x7f48bc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:56.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d27fc640 1 --2- 172.21.15.83:0/1919411741 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48bc004810 0x7f48bc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:56.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d27fc640 1 --2- 172.21.15.83:0/1919411741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48bc0051d0 0x7f48bc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:56.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d27fc640 1 -- 172.21.15.83:0/1919411741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d4071ea0 con 0x7f48bc0039c0 2024-04-17T10:53:56.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d27fc640 1 -- 172.21.15.83:0/1919411741 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48bc00c170 con 0x7f48bc0051d0 2024-04-17T10:53:56.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d27fc640 1 -- 172.21.15.83:0/1919411741 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48bc00c370 con 0x7f48bc004810 2024-04-17T10:53:56.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d37fe640 1 --2- 172.21.15.83:0/1919411741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48bc0039c0 0x7f48bc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:56.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d2ffd640 1 --2- 172.21.15.83:0/1919411741 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48bc004810 0x7f48bc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:56.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d3fff640 1 --2- 172.21.15.83:0/1919411741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48bc0051d0 0x7f48bc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:56.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d37fe640 1 -- 172.21.15.83:0/1919411741 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48bc004810 msgr2=0x7f48bc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:56.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d37fe640 1 --2- 172.21.15.83:0/1919411741 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48bc004810 0x7f48bc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:56.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d37fe640 1 -- 172.21.15.83:0/1919411741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48bc0051d0 msgr2=0x7f48bc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:56.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d37fe640 1 --2- 172.21.15.83:0/1919411741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48bc0051d0 0x7f48bc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:56.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d37fe640 1 -- 172.21.15.83:0/1919411741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48bc003530 con 0x7f48bc0039c0 2024-04-17T10:53:56.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d3fff640 1 --2- 172.21.15.83:0/1919411741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48bc0051d0 0x7f48bc00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:53:56.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d2ffd640 1 --2- 172.21.15.83:0/1919411741 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48bc004810 0x7f48bc004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:53:56.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.528+0000 7f48d3fff640 1 -- 172.21.15.83:0/1919411741 reap_dead start 2024-04-17T10:53:56.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.532+0000 7f48d37fe640 1 --2- 172.21.15.83:0/1919411741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48bc0039c0 0x7f48bc003de0 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f48c800c240 tx=0x7f48c801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f1f168f279cfa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:56.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.532+0000 7f48d27fc640 1 -- 172.21.15.83:0/1919411741 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c8007d20 con 0x7f48bc0039c0 2024-04-17T10:53:56.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.532+0000 7f48d27fc640 1 -- 172.21.15.83:0/1919411741 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c80a9070 con 0x7f48bc0039c0 2024-04-17T10:53:56.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.532+0000 7f48d27fc640 1 -- 172.21.15.83:0/1919411741 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c8005da0 con 0x7f48bc0039c0 2024-04-17T10:53:56.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.536+0000 7f48d9ce8640 1 -- 172.21.15.83:0/1919411741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48bc0039c0 msgr2=0x7f48bc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:56.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.536+0000 7f48d9ce8640 1 --2- 172.21.15.83:0/1919411741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48bc0039c0 0x7f48bc003de0 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f48c800c240 tx=0x7f48c801cc80 comp rx=0 tx=0).stop 2024-04-17T10:53:56.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.536+0000 7f48d9ce8640 1 -- 172.21.15.83:0/1919411741 shutdown_connections 2024-04-17T10:53:56.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.536+0000 7f48d9ce8640 1 --2- 172.21.15.83:0/1919411741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48bc0039c0 0x7f48bc003de0 unknown :-1 s=CLOSED pgs=2345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:56.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.536+0000 7f48d9ce8640 1 -- 172.21.15.83:0/1919411741 >> 172.21.15.83:0/1919411741 conn(0x7f48d406abe0 msgr2=0x7f48d406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:56.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.536+0000 7f48d9ce8640 1 -- 172.21.15.83:0/1919411741 shutdown_connections 2024-04-17T10:53:56.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.536+0000 7f48d9ce8640 1 -- 172.21.15.83:0/1919411741 wait complete. 2024-04-17T10:53:56.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.536+0000 7f48d9ce8640 1 Processor -- start 2024-04-17T10:53:56.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.536+0000 7f48d9ce8640 1 -- start start 2024-04-17T10:53:56.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.536+0000 7f48d9ce8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48bc0039c0 0x7f48d4149f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:56.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.536+0000 7f48d9ce8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48d41f4b40 0x7f48d41f4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:56.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.536+0000 7f48d9ce8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48d4153920 0x7f48d4153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:56.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.536+0000 7f48d9ce8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d406de20 con 0x7f48bc0039c0 2024-04-17T10:53:56.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.536+0000 7f48d9ce8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d406dca0 con 0x7f48d41f4b40 2024-04-17T10:53:56.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.536+0000 7f48d9ce8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d415cd70 con 0x7f48d4153920 2024-04-17T10:53:56.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.536+0000 7f48d37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48bc0039c0 0x7f48d4149f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:56.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.536+0000 7f48d37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48bc0039c0 0x7f48d4149f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35490/0 (socket says 172.21.15.83:35490) 2024-04-17T10:53:56.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.536+0000 7f48d37fe640 1 -- 172.21.15.83:0/3637831754 learned_addr learned my addr 172.21.15.83:0/3637831754 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:56.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.536+0000 7f48d2ffd640 1 --2- 172.21.15.83:0/3637831754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48d41f4b40 0x7f48d41f4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:56.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.536+0000 7f48d3fff640 1 --2- 172.21.15.83:0/3637831754 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48d4153920 0x7f48d4153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:56.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.536+0000 7f48d37fe640 1 -- 172.21.15.83:0/3637831754 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48d4153920 msgr2=0x7f48d4153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:56.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.536+0000 7f48d37fe640 1 --2- 172.21.15.83:0/3637831754 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48d4153920 0x7f48d4153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:56.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.536+0000 7f48d37fe640 1 -- 172.21.15.83:0/3637831754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48d41f4b40 msgr2=0x7f48d41f4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:56.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.536+0000 7f48d37fe640 1 --2- 172.21.15.83:0/3637831754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48d41f4b40 0x7f48d41f4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:56.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.536+0000 7f48d37fe640 1 -- 172.21.15.83:0/3637831754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d414a4c0 con 0x7f48bc0039c0 2024-04-17T10:53:56.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.536+0000 7f48d37fe640 1 --2- 172.21.15.83:0/3637831754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48bc0039c0 0x7f48d4149f80 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f48c800c240 tx=0x7f48bc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:56.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.536+0000 7f48bbfff640 1 -- 172.21.15.83:0/3637831754 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c8007180 con 0x7f48bc0039c0 2024-04-17T10:53:56.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.536+0000 7f48bbfff640 1 -- 172.21.15.83:0/3637831754 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c8007320 con 0x7f48bc0039c0 2024-04-17T10:53:56.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.536+0000 7f48bbfff640 1 -- 172.21.15.83:0/3637831754 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c80076d0 con 0x7f48bc0039c0 2024-04-17T10:53:56.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.536+0000 7f48d9ce8640 1 -- 172.21.15.83:0/3637831754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d41f5600 con 0x7f48bc0039c0 2024-04-17T10:53:56.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.536+0000 7f48d9ce8640 1 -- 172.21.15.83:0/3637831754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d41598c0 con 0x7f48bc0039c0 2024-04-17T10:53:56.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.540+0000 7f48d9ce8640 1 -- 172.21.15.83:0/3637831754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d4074f90 con 0x7f48bc0039c0 2024-04-17T10:53:56.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.540+0000 7f48bbfff640 1 -- 172.21.15.83:0/3637831754 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f48c80a5840 con 0x7f48bc0039c0 2024-04-17T10:53:56.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.540+0000 7f48bbfff640 1 --2- 172.21.15.83:0/3637831754 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f48c0066f60 0x7f48c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:56.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.540+0000 7f48bbfff640 1 -- 172.21.15.83:0/3637831754 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f48c811e050 con 0x7f48bc0039c0 2024-04-17T10:53:56.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.540+0000 7f48d2ffd640 1 --2- 172.21.15.83:0/3637831754 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f48c0066f60 0x7f48c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:56.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.540+0000 7f48d2ffd640 1 --2- 172.21.15.83:0/3637831754 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f48c0066f60 0x7f48c0069420 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f48d41f91f0 tx=0x7f48c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:56.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.544+0000 7f48bbfff640 1 -- 172.21.15.83:0/3637831754 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f48c80bf070 con 0x7f48bc0039c0 2024-04-17T10:53:56.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.740+0000 7f48d9ce8640 1 -- 172.21.15.83:0/3637831754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48d40cde50 con 0x7f48bc0039c0 2024-04-17T10:53:56.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.740+0000 7f48bbfff640 1 -- 172.21.15.83:0/3637831754 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f48c8016990 con 0x7f48bc0039c0 2024-04-17T10:53:56.733 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:56.733 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:56.734 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:56.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.744+0000 7f48d9ce8640 1 -- 172.21.15.83:0/3637831754 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f48c0066f60 msgr2=0x7f48c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:56.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.744+0000 7f48d9ce8640 1 --2- 172.21.15.83:0/3637831754 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f48c0066f60 0x7f48c0069420 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f48d41f91f0 tx=0x7f48c4008040 comp rx=0 tx=0).stop 2024-04-17T10:53:56.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.744+0000 7f48d9ce8640 1 -- 172.21.15.83:0/3637831754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48bc0039c0 msgr2=0x7f48d4149f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:56.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.744+0000 7f48d9ce8640 1 --2- 172.21.15.83:0/3637831754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48bc0039c0 0x7f48d4149f80 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f48c800c240 tx=0x7f48bc005840 comp rx=0 tx=0).stop 2024-04-17T10:53:56.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.744+0000 7f48d9ce8640 1 -- 172.21.15.83:0/3637831754 shutdown_connections 2024-04-17T10:53:56.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.744+0000 7f48d9ce8640 1 --2- 172.21.15.83:0/3637831754 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f48c0066f60 0x7f48c0069420 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:56.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.744+0000 7f48d9ce8640 1 --2- 172.21.15.83:0/3637831754 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48d4153920 0x7f48d4153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:56.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.744+0000 7f48d9ce8640 1 --2- 172.21.15.83:0/3637831754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48d41f4b40 0x7f48d41f4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:56.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.744+0000 7f48d9ce8640 1 --2- 172.21.15.83:0/3637831754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48bc0039c0 0x7f48d4149f80 unknown :-1 s=CLOSED pgs=2346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:56.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.744+0000 7f48d9ce8640 1 -- 172.21.15.83:0/3637831754 >> 172.21.15.83:0/3637831754 conn(0x7f48d406abe0 msgr2=0x7f48d415dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:56.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.744+0000 7f48d9ce8640 1 -- 172.21.15.83:0/3637831754 shutdown_connections 2024-04-17T10:53:56.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:56.744+0000 7f48d9ce8640 1 -- 172.21.15.83:0/3637831754 wait complete. 2024-04-17T10:53:57.153 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:57.153 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:58.154 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:53:58.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.292+0000 7f990896d640 1 Processor -- start 2024-04-17T10:53:58.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.292+0000 7f990896d640 1 -- start start 2024-04-17T10:53:58.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.296+0000 7f990896d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99040cf060 0x7f99040b4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:58.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.296+0000 7f990896d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f99040b3990 0x7f9904070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:58.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.296+0000 7f990896d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99040b2bd0 con 0x7f99040b45a0 2024-04-17T10:53:58.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.296+0000 7f990896d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99040b2980 con 0x7f99040b3990 2024-04-17T10:53:58.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.296+0000 7f990896d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99040cf440 con 0x7f99040cf060 2024-04-17T10:53:58.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.296+0000 7f9901d9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f99040b3990 0x7f9904070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:58.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.296+0000 7f9902d9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f99040b45a0 0x7f990415d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:36530/0 (socket says 172.21.15.83:36530) 2024-04-17T10:53:58.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.296+0000 7f9901d9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f99040b3990 0x7f9904070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56486/0 (socket says 172.21.15.83:56486) 2024-04-17T10:53:58.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.296+0000 7f9902d9d640 1 -- 172.21.15.83:0/4252040004 learned_addr learned my addr 172.21.15.83:0/4252040004 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:53:58.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.296+0000 7f990259c640 1 --2- 172.21.15.83:0/4252040004 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99040cf060 0x7f99040b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:58.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.296+0000 7f9901d9b640 1 -- 172.21.15.83:0/4252040004 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99040cf060 msgr2=0x7f99040b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:53:58.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.296+0000 7f9901d9b640 1 --2- 172.21.15.83:0/4252040004 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99040cf060 0x7f99040b4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:58.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.296+0000 7f9901d9b640 1 -- 172.21.15.83:0/4252040004 >> v1:172.21.15.83:6789/0 conn(0x7f99040b45a0 legacy=0x7f990415d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:58.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.296+0000 7f990159a640 1 -- 172.21.15.83:0/4252040004 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2975292501 0 0) 0x7f99040b2bd0 con 0x7f99040b45a0 2024-04-17T10:53:58.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.296+0000 7f9901d9b640 1 -- 172.21.15.83:0/4252040004 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99040cf5c0 con 0x7f99040b3990 2024-04-17T10:53:58.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.296+0000 7f9901d9b640 1 --2- 172.21.15.83:0/4252040004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99040b3990 0x7f9904070470 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f98f80097c0 tx=0x7f98f801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0626342353e28c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:58.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.296+0000 7f990159a640 1 -- 172.21.15.83:0/4252040004 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f80b2070 con 0x7f99040b3990 2024-04-17T10:53:58.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.296+0000 7f990159a640 1 -- 172.21.15.83:0/4252040004 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f80ad680 con 0x7f99040b3990 2024-04-17T10:53:58.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.296+0000 7f990159a640 1 -- 172.21.15.83:0/4252040004 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f80ada50 con 0x7f99040b3990 2024-04-17T10:53:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.296+0000 7f990896d640 1 -- 172.21.15.83:0/4252040004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99040b3990 msgr2=0x7f9904070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.296+0000 7f990896d640 1 --2- 172.21.15.83:0/4252040004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99040b3990 0x7f9904070470 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f98f80097c0 tx=0x7f98f801c470 comp rx=0 tx=0).stop 2024-04-17T10:53:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.296+0000 7f990896d640 1 -- 172.21.15.83:0/4252040004 shutdown_connections 2024-04-17T10:53:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.296+0000 7f990896d640 1 --2- 172.21.15.83:0/4252040004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99040b3990 0x7f9904070470 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.296+0000 7f990896d640 1 --2- 172.21.15.83:0/4252040004 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99040cf060 0x7f99040b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.296+0000 7f990896d640 1 -- 172.21.15.83:0/4252040004 >> 172.21.15.83:0/4252040004 conn(0x7f990406abe0 msgr2=0x7f990406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:58.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.296+0000 7f990896d640 1 -- 172.21.15.83:0/4252040004 shutdown_connections 2024-04-17T10:53:58.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.296+0000 7f990896d640 1 -- 172.21.15.83:0/4252040004 wait complete. 2024-04-17T10:53:58.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.300+0000 7f990896d640 1 Processor -- start 2024-04-17T10:53:58.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.300+0000 7f990896d640 1 -- start start 2024-04-17T10:53:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.300+0000 7f990896d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99040b3990 0x7f9904142e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.300+0000 7f990896d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99040b45a0 0x7f9904152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.300+0000 7f990896d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99040cf060 0x7f990415c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.300+0000 7f990896d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99040cfc00 con 0x7f99040b3990 2024-04-17T10:53:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.300+0000 7f990896d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990415b310 con 0x7f99040b45a0 2024-04-17T10:53:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.300+0000 7f990896d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9904164d80 con 0x7f99040cf060 2024-04-17T10:53:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.300+0000 7f990259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99040b3990 0x7f9904142e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.300+0000 7f990259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99040b3990 0x7f9904142e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56488/0 (socket says 172.21.15.83:56488) 2024-04-17T10:53:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.300+0000 7f990259c640 1 -- 172.21.15.83:0/255347010 learned_addr learned my addr 172.21.15.83:0/255347010 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:53:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.300+0000 7f9901d9b640 1 --2- 172.21.15.83:0/255347010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99040b45a0 0x7f9904152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.300+0000 7f9902d9d640 1 --2- 172.21.15.83:0/255347010 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99040cf060 0x7f990415c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.300+0000 7f990259c640 1 -- 172.21.15.83:0/255347010 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99040cf060 msgr2=0x7f990415c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.300+0000 7f990259c640 1 --2- 172.21.15.83:0/255347010 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99040cf060 0x7f990415c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.300+0000 7f990259c640 1 -- 172.21.15.83:0/255347010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99040b45a0 msgr2=0x7f9904152d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.300+0000 7f990259c640 1 --2- 172.21.15.83:0/255347010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99040b45a0 0x7f9904152d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.300+0000 7f990259c640 1 -- 172.21.15.83:0/255347010 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990406d6a0 con 0x7f99040b3990 2024-04-17T10:53:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.300+0000 7f990259c640 1 --2- 172.21.15.83:0/255347010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99040b3990 0x7f9904142e10 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7f98f400cd20 tx=0x7f98f400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.300+0000 7f98f2ffd640 1 -- 172.21.15.83:0/255347010 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f40039a0 con 0x7f99040b3990 2024-04-17T10:53:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.300+0000 7f98f2ffd640 1 -- 172.21.15.83:0/255347010 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f4003b40 con 0x7f99040b3990 2024-04-17T10:53:58.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.300+0000 7f990896d640 1 -- 172.21.15.83:0/255347010 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9904154bf0 con 0x7f99040b3990 2024-04-17T10:53:58.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.300+0000 7f98f2ffd640 1 -- 172.21.15.83:0/255347010 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f4004020 con 0x7f99040b3990 2024-04-17T10:53:58.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.300+0000 7f990896d640 1 -- 172.21.15.83:0/255347010 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99041551b0 con 0x7f99040b3990 2024-04-17T10:53:58.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.304+0000 7f990896d640 1 -- 172.21.15.83:0/255347010 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9904074f90 con 0x7f99040b3990 2024-04-17T10:53:58.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.304+0000 7f98f2ffd640 1 -- 172.21.15.83:0/255347010 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f98f40041c0 con 0x7f99040b3990 2024-04-17T10:53:58.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.304+0000 7f98f2ffd640 1 --2- 172.21.15.83:0/255347010 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f98e4066f60 0x7f98e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:53:58.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.304+0000 7f98f2ffd640 1 -- 172.21.15.83:0/255347010 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f98f4085860 con 0x7f99040b3990 2024-04-17T10:53:58.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.304+0000 7f9901d9b640 1 --2- 172.21.15.83:0/255347010 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f98e4066f60 0x7f98e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:53:58.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.304+0000 7f9901d9b640 1 --2- 172.21.15.83:0/255347010 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f98e4066f60 0x7f98e4069420 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f98f80af000 tx=0x7f98f80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:53:58.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.308+0000 7f98f2ffd640 1 -- 172.21.15.83:0/255347010 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f98f4014030 con 0x7f99040b3990 2024-04-17T10:53:58.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.492+0000 7f990896d640 1 -- 172.21.15.83:0/255347010 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99040b1040 con 0x7f99040b3990 2024-04-17T10:53:58.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.492+0000 7f98f2ffd640 1 -- 172.21.15.83:0/255347010 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f98f4056990 con 0x7f99040b3990 2024-04-17T10:53:58.485 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:53:58.486 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:53:58.486 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:53:58.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.496+0000 7f990896d640 1 -- 172.21.15.83:0/255347010 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f98e4066f60 msgr2=0x7f98e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:58.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.496+0000 7f990896d640 1 --2- 172.21.15.83:0/255347010 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f98e4066f60 0x7f98e4069420 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f98f80af000 tx=0x7f98f80a7000 comp rx=0 tx=0).stop 2024-04-17T10:53:58.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.496+0000 7f990896d640 1 -- 172.21.15.83:0/255347010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99040b3990 msgr2=0x7f9904142e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:53:58.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.496+0000 7f990896d640 1 --2- 172.21.15.83:0/255347010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99040b3990 0x7f9904142e10 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7f98f400cd20 tx=0x7f98f400a5c0 comp rx=0 tx=0).stop 2024-04-17T10:53:58.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.496+0000 7f990896d640 1 -- 172.21.15.83:0/255347010 shutdown_connections 2024-04-17T10:53:58.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.496+0000 7f990896d640 1 --2- 172.21.15.83:0/255347010 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f98e4066f60 0x7f98e4069420 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:58.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.496+0000 7f990896d640 1 --2- 172.21.15.83:0/255347010 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99040cf060 0x7f990415c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:58.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.496+0000 7f990896d640 1 --2- 172.21.15.83:0/255347010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99040b45a0 0x7f9904152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:58.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.496+0000 7f990896d640 1 --2- 172.21.15.83:0/255347010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99040b3990 0x7f9904142e10 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:53:58.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.496+0000 7f990896d640 1 -- 172.21.15.83:0/255347010 >> 172.21.15.83:0/255347010 conn(0x7f990406abe0 msgr2=0x7f9904158c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:53:58.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.496+0000 7f990896d640 1 -- 172.21.15.83:0/255347010 shutdown_connections 2024-04-17T10:53:58.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:53:58.496+0000 7f990896d640 1 -- 172.21.15.83:0/255347010 wait complete. 2024-04-17T10:53:58.906 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:53:58.906 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:53:59.907 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:54:00.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.044+0000 7f8633411640 1 Processor -- start 2024-04-17T10:54:00.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.044+0000 7f8633411640 1 -- start start 2024-04-17T10:54:00.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.044+0000 7f8633411640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f862c0d09e0 0x7f862c168bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:00.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.044+0000 7f8633411640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f862c0cea30 con 0x7f862c15de30 2024-04-17T10:54:00.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.044+0000 7f8633411640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f862c0ced00 con 0x7f862c0b6cd0 2024-04-17T10:54:00.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.044+0000 7f8633411640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f862c15e210 con 0x7f862c0d09e0 2024-04-17T10:54:00.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.044+0000 7f86311ad640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f862c15de30 0x7f862c0ce320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:36538/0 (socket says 172.21.15.83:36538) 2024-04-17T10:54:00.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.044+0000 7f86311ad640 1 -- 172.21.15.83:0/3869573599 learned_addr learned my addr 172.21.15.83:0/3869573599 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:00.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.044+0000 7f86319ae640 1 --2- 172.21.15.83:0/3869573599 >> v2:172.21.15.83:3300/0 conn(0x7f862c0d09e0 0x7f862c168bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:00.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.044+0000 7f8623fff640 1 -- 172.21.15.83:0/3869573599 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3807546499 0 0) 0x7f862c0cea30 con 0x7f862c15de30 2024-04-17T10:54:00.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.044+0000 7f8623fff640 1 -- 172.21.15.83:0/3869573599 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8618003530 con 0x7f862c15de30 2024-04-17T10:54:00.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.044+0000 7f86319ae640 1 -- 172.21.15.83:0/3869573599 >> v1:172.21.15.167:6789/0 conn(0x7f862c0b6cd0 legacy=0x7f862c0d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:00.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.044+0000 7f86319ae640 1 -- 172.21.15.83:0/3869573599 >> v1:172.21.15.83:6789/0 conn(0x7f862c15de30 legacy=0x7f862c0ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:00.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.044+0000 7f86319ae640 1 -- 172.21.15.83:0/3869573599 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862c0cefe0 con 0x7f862c0d09e0 2024-04-17T10:54:00.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.044+0000 7f86319ae640 1 --2- 172.21.15.83:0/3869573599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f862c0d09e0 0x7f862c168bd0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7f8624009d90 tx=0x7f862401c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4beb694de8f0af05 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:00.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f8623fff640 1 -- 172.21.15.83:0/3869573599 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86240b2070 con 0x7f862c0d09e0 2024-04-17T10:54:00.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f8623fff640 1 -- 172.21.15.83:0/3869573599 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86240a5810 con 0x7f862c0d09e0 2024-04-17T10:54:00.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f8623fff640 1 -- 172.21.15.83:0/3869573599 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86240a5be0 con 0x7f862c0d09e0 2024-04-17T10:54:00.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f8633411640 1 -- 172.21.15.83:0/3869573599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f862c0d09e0 msgr2=0x7f862c168bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:00.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f8633411640 1 --2- 172.21.15.83:0/3869573599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f862c0d09e0 0x7f862c168bd0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7f8624009d90 tx=0x7f862401c860 comp rx=0 tx=0).stop 2024-04-17T10:54:00.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f8633411640 1 -- 172.21.15.83:0/3869573599 shutdown_connections 2024-04-17T10:54:00.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f8633411640 1 --2- 172.21.15.83:0/3869573599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f862c0d09e0 0x7f862c168bd0 unknown :-1 s=CLOSED pgs=2351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:00.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f8633411640 1 -- 172.21.15.83:0/3869573599 >> 172.21.15.83:0/3869573599 conn(0x7f862c06b0b0 msgr2=0x7f862c06b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:00.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f8633411640 1 -- 172.21.15.83:0/3869573599 shutdown_connections 2024-04-17T10:54:00.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f8633411640 1 -- 172.21.15.83:0/3869573599 wait complete. 2024-04-17T10:54:00.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f8633411640 1 Processor -- start 2024-04-17T10:54:00.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f8633411640 1 -- start start 2024-04-17T10:54:00.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f8633411640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f862c0b6cd0 0x7f862c15cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:00.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f8633411640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f862c0d09e0 0x7f862c149600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:00.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f8633411640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f862c15de30 0x7f862c159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:00.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f8633411640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862c161b70 con 0x7f862c15de30 2024-04-17T10:54:00.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f8633411640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862c1619f0 con 0x7f862c0b6cd0 2024-04-17T10:54:00.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f8633411640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f862c16b930 con 0x7f862c0d09e0 2024-04-17T10:54:00.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f86319ae640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f862c15de30 0x7f862c159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:00.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f86319ae640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f862c15de30 0x7f862c159e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56494/0 (socket says 172.21.15.83:56494) 2024-04-17T10:54:00.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f86319ae640 1 -- 172.21.15.83:0/1494024729 learned_addr learned my addr 172.21.15.83:0/1494024729 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:00.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f86311ad640 1 --2- 172.21.15.83:0/1494024729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f862c0b6cd0 0x7f862c15cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:00.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f86319ae640 1 -- 172.21.15.83:0/1494024729 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f862c0d09e0 msgr2=0x7f862c149600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:00.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f86309ac640 1 --2- 172.21.15.83:0/1494024729 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f862c0d09e0 0x7f862c149600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:00.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f86319ae640 1 --2- 172.21.15.83:0/1494024729 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f862c0d09e0 0x7f862c149600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:00.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f86319ae640 1 -- 172.21.15.83:0/1494024729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f862c0b6cd0 msgr2=0x7f862c15cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:00.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f86319ae640 1 --2- 172.21.15.83:0/1494024729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f862c0b6cd0 0x7f862c15cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:00.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f86319ae640 1 -- 172.21.15.83:0/1494024729 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862c15bdd0 con 0x7f862c15de30 2024-04-17T10:54:00.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f86319ae640 1 --2- 172.21.15.83:0/1494024729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f862c15de30 0x7f862c159e00 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f86240b0040 tx=0x7f8624019600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:00.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f8621ffb640 1 -- 172.21.15.83:0/1494024729 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86240b2050 con 0x7f862c15de30 2024-04-17T10:54:00.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f8621ffb640 1 -- 172.21.15.83:0/1494024729 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86240be070 con 0x7f862c15de30 2024-04-17T10:54:00.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f8633411640 1 -- 172.21.15.83:0/1494024729 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f862c0c7bf0 con 0x7f862c15de30 2024-04-17T10:54:00.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f8621ffb640 1 -- 172.21.15.83:0/1494024729 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86240ad850 con 0x7f862c15de30 2024-04-17T10:54:00.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.048+0000 7f8633411640 1 -- 172.21.15.83:0/1494024729 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f862c15c330 con 0x7f862c15de30 2024-04-17T10:54:00.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.052+0000 7f8633411640 1 -- 172.21.15.83:0/1494024729 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f862c074f90 con 0x7f862c15de30 2024-04-17T10:54:00.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.052+0000 7f8621ffb640 1 -- 172.21.15.83:0/1494024729 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f86240ad9f0 con 0x7f862c15de30 2024-04-17T10:54:00.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.052+0000 7f8621ffb640 1 --2- 172.21.15.83:0/1494024729 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8610066f60 0x7f8610069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:00.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.052+0000 7f8621ffb640 1 -- 172.21.15.83:0/1494024729 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f8624120b60 con 0x7f862c15de30 2024-04-17T10:54:00.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.052+0000 7f86311ad640 1 --2- 172.21.15.83:0/1494024729 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8610066f60 0x7f8610069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:00.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.052+0000 7f86311ad640 1 --2- 172.21.15.83:0/1494024729 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8610066f60 0x7f8610069420 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f86280011f0 tx=0x7f8628007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:00.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.064+0000 7f8621ffb640 1 -- 172.21.15.83:0/1494024729 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f862c15bdd0 con 0x7f862c15de30 2024-04-17T10:54:00.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.240+0000 7f8633411640 1 -- 172.21.15.83:0/1494024729 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f862c0d17e0 con 0x7f862c15de30 2024-04-17T10:54:00.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.244+0000 7f8621ffb640 1 -- 172.21.15.83:0/1494024729 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f86240164e0 con 0x7f862c15de30 2024-04-17T10:54:00.235 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:00.236 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:54:00.236 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:54:00.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.244+0000 7f8633411640 1 -- 172.21.15.83:0/1494024729 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8610066f60 msgr2=0x7f8610069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:00.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.244+0000 7f8633411640 1 --2- 172.21.15.83:0/1494024729 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8610066f60 0x7f8610069420 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f86280011f0 tx=0x7f8628007c00 comp rx=0 tx=0).stop 2024-04-17T10:54:00.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.244+0000 7f8633411640 1 -- 172.21.15.83:0/1494024729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f862c15de30 msgr2=0x7f862c159e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:00.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.244+0000 7f8633411640 1 --2- 172.21.15.83:0/1494024729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f862c15de30 0x7f862c159e00 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f86240b0040 tx=0x7f8624019600 comp rx=0 tx=0).stop 2024-04-17T10:54:00.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.244+0000 7f8633411640 1 -- 172.21.15.83:0/1494024729 shutdown_connections 2024-04-17T10:54:00.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.244+0000 7f8633411640 1 --2- 172.21.15.83:0/1494024729 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8610066f60 0x7f8610069420 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:00.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.244+0000 7f8633411640 1 --2- 172.21.15.83:0/1494024729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f862c15de30 0x7f862c159e00 unknown :-1 s=CLOSED pgs=2352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:00.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.244+0000 7f8633411640 1 --2- 172.21.15.83:0/1494024729 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f862c0d09e0 0x7f862c149600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:00.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.244+0000 7f8633411640 1 --2- 172.21.15.83:0/1494024729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f862c0b6cd0 0x7f862c15cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:00.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.244+0000 7f8633411640 1 -- 172.21.15.83:0/1494024729 >> 172.21.15.83:0/1494024729 conn(0x7f862c06b0b0 msgr2=0x7f862c160820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:00.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.244+0000 7f8633411640 1 -- 172.21.15.83:0/1494024729 shutdown_connections 2024-04-17T10:54:00.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:00.244+0000 7f8633411640 1 -- 172.21.15.83:0/1494024729 wait complete. 2024-04-17T10:54:00.655 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:54:00.656 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:54:01.656 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:54:01.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.796+0000 7f4366fc8640 1 Processor -- start 2024-04-17T10:54:01.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.796+0000 7f4366fc8640 1 -- start start 2024-04-17T10:54:01.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.796+0000 7f4366fc8640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f43600ce980 0x7f43600b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:01.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.796+0000 7f4366fc8640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4360162a00 con 0x7f43600cee00 2024-04-17T10:54:01.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.796+0000 7f4366fc8640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4360161e80 con 0x7f436015dfa0 2024-04-17T10:54:01.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.796+0000 7f4366fc8640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4360162160 con 0x7f43600ce980 2024-04-17T10:54:01.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.796+0000 7f435ffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f43600ce980 0x7f43600b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:01.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.796+0000 7f4365565640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f43600cee00 0x7f43600d1d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:36550/0 (socket says 172.21.15.83:36550) 2024-04-17T10:54:01.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.796+0000 7f4365565640 1 -- 172.21.15.83:0/2433722792 learned_addr learned my addr 172.21.15.83:0/2433722792 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:01.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.796+0000 7f435ffff640 1 -- 172.21.15.83:0/2433722792 >> v1:172.21.15.167:6789/0 conn(0x7f436015dfa0 legacy=0x7f43600b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:01.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.796+0000 7f435ffff640 1 -- 172.21.15.83:0/2433722792 >> v1:172.21.15.83:6789/0 conn(0x7f43600cee00 legacy=0x7f43600d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:01.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.796+0000 7f435ffff640 1 -- 172.21.15.83:0/2433722792 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4360162440 con 0x7f43600ce980 2024-04-17T10:54:01.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f435ffff640 1 --2- 172.21.15.83:0/2433722792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43600ce980 0x7f43600b5430 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f4354009d90 tx=0x7f435401c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e09fc4e6b26026 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:01.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f435f7fe640 1 -- 172.21.15.83:0/2433722792 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43540b2070 con 0x7f43600ce980 2024-04-17T10:54:01.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f435f7fe640 1 -- 172.21.15.83:0/2433722792 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43540a5840 con 0x7f43600ce980 2024-04-17T10:54:01.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f435f7fe640 1 -- 172.21.15.83:0/2433722792 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43540a5c10 con 0x7f43600ce980 2024-04-17T10:54:01.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f4366fc8640 1 -- 172.21.15.83:0/2433722792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43600ce980 msgr2=0x7f43600b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:01.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f4366fc8640 1 --2- 172.21.15.83:0/2433722792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43600ce980 0x7f43600b5430 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f4354009d90 tx=0x7f435401c870 comp rx=0 tx=0).stop 2024-04-17T10:54:01.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f4366fc8640 1 -- 172.21.15.83:0/2433722792 shutdown_connections 2024-04-17T10:54:01.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f4366fc8640 1 --2- 172.21.15.83:0/2433722792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43600ce980 0x7f43600b5430 unknown :-1 s=CLOSED pgs=2354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:01.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f4366fc8640 1 -- 172.21.15.83:0/2433722792 >> 172.21.15.83:0/2433722792 conn(0x7f43600d0600 msgr2=0x7f43600d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:01.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f4366fc8640 1 -- 172.21.15.83:0/2433722792 shutdown_connections 2024-04-17T10:54:01.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f4366fc8640 1 -- 172.21.15.83:0/2433722792 wait complete. 2024-04-17T10:54:01.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f4366fc8640 1 Processor -- start 2024-04-17T10:54:01.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f4366fc8640 1 -- start start 2024-04-17T10:54:01.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f4366fc8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43600ce980 0x7f43600c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:01.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f4366fc8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43600cee00 0x7f43600c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:01.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f4366fc8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f436015dfa0 0x7f4360160310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:01.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f4366fc8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436016c670 con 0x7f43600cee00 2024-04-17T10:54:01.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f4366fc8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4360161b10 con 0x7f43600ce980 2024-04-17T10:54:01.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f4366fc8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f436016c7f0 con 0x7f436015dfa0 2024-04-17T10:54:01.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f435ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43600cee00 0x7f43600c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:01.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f435ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43600cee00 0x7f43600c6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56500/0 (socket says 172.21.15.83:56500) 2024-04-17T10:54:01.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f435ffff640 1 -- 172.21.15.83:0/1871199557 learned_addr learned my addr 172.21.15.83:0/1871199557 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:01.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f4365565640 1 --2- 172.21.15.83:0/1871199557 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f436015dfa0 0x7f4360160310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:01.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f4364d64640 1 --2- 172.21.15.83:0/1871199557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43600ce980 0x7f43600c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:01.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f435ffff640 1 -- 172.21.15.83:0/1871199557 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f436015dfa0 msgr2=0x7f4360160310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:01.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f435ffff640 1 --2- 172.21.15.83:0/1871199557 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f436015dfa0 0x7f4360160310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f435ffff640 1 -- 172.21.15.83:0/1871199557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43600ce980 msgr2=0x7f43600c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f435ffff640 1 --2- 172.21.15.83:0/1871199557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43600ce980 0x7f43600c5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f435ffff640 1 -- 172.21.15.83:0/1871199557 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f436015eeb0 con 0x7f43600cee00 2024-04-17T10:54:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f4365565640 1 --2- 172.21.15.83:0/1871199557 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f436015dfa0 0x7f4360160310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f4364d64640 1 --2- 172.21.15.83:0/1871199557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43600ce980 0x7f43600c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f435ffff640 1 --2- 172.21.15.83:0/1871199557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43600cee00 0x7f43600c6290 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7f43540b0040 tx=0x7f4354019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f435d7fa640 1 -- 172.21.15.83:0/1871199557 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43540bd020 con 0x7f43600cee00 2024-04-17T10:54:01.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f435d7fa640 1 -- 172.21.15.83:0/1871199557 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43540b8670 con 0x7f43600cee00 2024-04-17T10:54:01.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f435d7fa640 1 -- 172.21.15.83:0/1871199557 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43540b2040 con 0x7f43600cee00 2024-04-17T10:54:01.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f4366fc8640 1 -- 172.21.15.83:0/1871199557 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f436015e610 con 0x7f43600cee00 2024-04-17T10:54:01.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.800+0000 7f4366fc8640 1 -- 172.21.15.83:0/1871199557 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f436015ebd0 con 0x7f43600cee00 2024-04-17T10:54:01.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.804+0000 7f4366fc8640 1 -- 172.21.15.83:0/1871199557 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43600681a0 con 0x7f43600cee00 2024-04-17T10:54:01.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.804+0000 7f435d7fa640 1 -- 172.21.15.83:0/1871199557 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f43540c0020 con 0x7f43600cee00 2024-04-17T10:54:01.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.804+0000 7f435d7fa640 1 --2- 172.21.15.83:0/1871199557 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f434c066f90 0x7f434c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:01.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.804+0000 7f435d7fa640 1 -- 172.21.15.83:0/1871199557 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f4354120000 con 0x7f43600cee00 2024-04-17T10:54:01.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.804+0000 7f4364d64640 1 --2- 172.21.15.83:0/1871199557 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f434c066f90 0x7f434c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:01.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.804+0000 7f4364d64640 1 --2- 172.21.15.83:0/1871199557 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f434c066f90 0x7f434c069450 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f435000a850 tx=0x7f4350008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:01.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:01.808+0000 7f435d7fa640 1 -- 172.21.15.83:0/1871199557 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f43540c7070 con 0x7f43600cee00 2024-04-17T10:54:02.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:02.008+0000 7f4366fc8640 1 -- 172.21.15.83:0/1871199557 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4360074f90 con 0x7f43600cee00 2024-04-17T10:54:02.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:02.008+0000 7f435d7fa640 1 -- 172.21.15.83:0/1871199557 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f436015eeb0 con 0x7f43600cee00 2024-04-17T10:54:02.002 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:02.002 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:54:02.002 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:54:02.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:02.012+0000 7f4366fc8640 1 -- 172.21.15.83:0/1871199557 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f434c066f90 msgr2=0x7f434c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:02.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:02.012+0000 7f4366fc8640 1 --2- 172.21.15.83:0/1871199557 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f434c066f90 0x7f434c069450 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f435000a850 tx=0x7f4350008040 comp rx=0 tx=0).stop 2024-04-17T10:54:02.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:02.012+0000 7f4366fc8640 1 -- 172.21.15.83:0/1871199557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43600cee00 msgr2=0x7f43600c6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:02.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:02.012+0000 7f4366fc8640 1 --2- 172.21.15.83:0/1871199557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43600cee00 0x7f43600c6290 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7f43540b0040 tx=0x7f4354019610 comp rx=0 tx=0).stop 2024-04-17T10:54:02.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:02.012+0000 7f4366fc8640 1 -- 172.21.15.83:0/1871199557 shutdown_connections 2024-04-17T10:54:02.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:02.012+0000 7f4366fc8640 1 --2- 172.21.15.83:0/1871199557 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f434c066f90 0x7f434c069450 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:02.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:02.012+0000 7f4366fc8640 1 --2- 172.21.15.83:0/1871199557 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f436015dfa0 0x7f4360160310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:02.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:02.012+0000 7f4366fc8640 1 --2- 172.21.15.83:0/1871199557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43600cee00 0x7f43600c6290 unknown :-1 s=CLOSED pgs=2355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:02.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:02.012+0000 7f4366fc8640 1 --2- 172.21.15.83:0/1871199557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43600ce980 0x7f43600c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:02.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:02.012+0000 7f4366fc8640 1 -- 172.21.15.83:0/1871199557 >> 172.21.15.83:0/1871199557 conn(0x7f43600d0600 msgr2=0x7f436015cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:02.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:02.012+0000 7f4366fc8640 1 -- 172.21.15.83:0/1871199557 shutdown_connections 2024-04-17T10:54:02.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:02.012+0000 7f4366fc8640 1 -- 172.21.15.83:0/1871199557 wait complete. 2024-04-17T10:54:02.423 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:54:02.424 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:54:03.425 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:54:03.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.564+0000 7fdb28a60640 1 Processor -- start 2024-04-17T10:54:03.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.564+0000 7fdb28a60640 1 -- start start 2024-04-17T10:54:03.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.568+0000 7fdb28a60640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb2415ba30 0x7fdb24072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:03.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.568+0000 7fdb28a60640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb2406e320 con 0x7fdb240cfbe0 2024-04-17T10:54:03.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.568+0000 7fdb28a60640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb2406e040 con 0x7fdb2415c960 2024-04-17T10:54:03.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.568+0000 7fdb28a60640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb24071ea0 con 0x7fdb2415ba30 2024-04-17T10:54:03.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.568+0000 7fdb22d9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fdb240cfbe0 0x7fdb2406c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:36558/0 (socket says 172.21.15.83:36558) 2024-04-17T10:54:03.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.568+0000 7fdb22d9d640 1 -- 172.21.15.83:0/925771170 learned_addr learned my addr 172.21.15.83:0/925771170 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:03.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.568+0000 7fdb2259c640 1 --2- 172.21.15.83:0/925771170 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb2415ba30 0x7fdb24072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:03.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.568+0000 7fdb2159a640 1 -- 172.21.15.83:0/925771170 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3286898910 0 0) 0x7fdb2406e320 con 0x7fdb240cfbe0 2024-04-17T10:54:03.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.568+0000 7fdb2159a640 1 -- 172.21.15.83:0/925771170 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb0c003530 con 0x7fdb240cfbe0 2024-04-17T10:54:03.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.568+0000 7fdb2159a640 1 -- 172.21.15.83:0/925771170 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3877206077 0 0) 0x7fdb0c003530 con 0x7fdb240cfbe0 2024-04-17T10:54:03.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.568+0000 7fdb2159a640 1 -- 172.21.15.83:0/925771170 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb2415ba30 msgr2=0x7fdb24072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.568+0000 7fdb2159a640 1 --2- 172.21.15.83:0/925771170 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb2415ba30 0x7fdb24072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.568+0000 7fdb2159a640 1 -- 172.21.15.83:0/925771170 >> v1:172.21.15.167:6789/0 conn(0x7fdb2415c960 legacy=0x7fdb240cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.568+0000 7fdb2159a640 1 -- 172.21.15.83:0/925771170 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb24071850 con 0x7fdb240cfbe0 2024-04-17T10:54:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.568+0000 7fdb2159a640 1 -- 172.21.15.83:0/925771170 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fdb1c001a20 con 0x7fdb240cfbe0 2024-04-17T10:54:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.568+0000 7fdb2159a640 1 -- 172.21.15.83:0/925771170 >> v1:172.21.15.83:6789/0 conn(0x7fdb240cfbe0 legacy=0x7fdb2406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.568+0000 7fdb2159a640 1 --2- 172.21.15.83:0/925771170 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb2415ba30 0x7fdb0c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.568+0000 7fdb2159a640 1 --2- 172.21.15.83:0/925771170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb0c003f70 0x7fdb0c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.568+0000 7fdb2159a640 1 --2- 172.21.15.83:0/925771170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb0c004d50 0x7fdb0c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.568+0000 7fdb2159a640 1 -- 172.21.15.83:0/925771170 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb24071ea0 con 0x7fdb0c004d50 2024-04-17T10:54:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.568+0000 7fdb2159a640 1 -- 172.21.15.83:0/925771170 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c00bb60 con 0x7fdb0c003f70 2024-04-17T10:54:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.568+0000 7fdb2159a640 1 -- 172.21.15.83:0/925771170 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c00bd60 con 0x7fdb2415ba30 2024-04-17T10:54:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.568+0000 7fdb22d9d640 1 --2- 172.21.15.83:0/925771170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb0c004d50 0x7fdb0c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.568+0000 7fdb2259c640 1 --2- 172.21.15.83:0/925771170 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb2415ba30 0x7fdb0c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.568+0000 7fdb21d9b640 1 --2- 172.21.15.83:0/925771170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb0c003f70 0x7fdb0c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.568+0000 7fdb22d9d640 1 -- 172.21.15.83:0/925771170 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb2415ba30 msgr2=0x7fdb0c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.568+0000 7fdb22d9d640 1 --2- 172.21.15.83:0/925771170 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb2415ba30 0x7fdb0c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.568+0000 7fdb22d9d640 1 -- 172.21.15.83:0/925771170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb0c003f70 msgr2=0x7fdb0c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.568+0000 7fdb22d9d640 1 --2- 172.21.15.83:0/925771170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb0c003f70 0x7fdb0c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.568+0000 7fdb22d9d640 1 -- 172.21.15.83:0/925771170 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0c003530 con 0x7fdb0c004d50 2024-04-17T10:54:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.568+0000 7fdb21d9b640 1 --2- 172.21.15.83:0/925771170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb0c003f70 0x7fdb0c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:03.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.572+0000 7fdb22d9d640 1 --2- 172.21.15.83:0/925771170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb0c004d50 0x7fdb0c0051d0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c00a8f0 tx=0x7fdb1c01fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8635024e6b832b66 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:03.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.572+0000 7fdb2159a640 1 -- 172.21.15.83:0/925771170 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c00a4c0 con 0x7fdb0c004d50 2024-04-17T10:54:03.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.572+0000 7fdb2159a640 1 -- 172.21.15.83:0/925771170 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb1c0aa720 con 0x7fdb0c004d50 2024-04-17T10:54:03.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.572+0000 7fdb2159a640 1 -- 172.21.15.83:0/925771170 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c0aab10 con 0x7fdb0c004d50 2024-04-17T10:54:03.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.572+0000 7fdb28a60640 1 -- 172.21.15.83:0/925771170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb0c004d50 msgr2=0x7fdb0c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:03.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.572+0000 7fdb28a60640 1 --2- 172.21.15.83:0/925771170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb0c004d50 0x7fdb0c0051d0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c00a8f0 tx=0x7fdb1c01fb60 comp rx=0 tx=0).stop 2024-04-17T10:54:03.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.572+0000 7fdb22d9d640 1 -- 172.21.15.83:0/925771170 reap_dead start 2024-04-17T10:54:03.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.572+0000 7fdb28a60640 1 -- 172.21.15.83:0/925771170 shutdown_connections 2024-04-17T10:54:03.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.572+0000 7fdb28a60640 1 -- 172.21.15.83:0/925771170 >> 172.21.15.83:0/925771170 conn(0x7fdb2406abe0 msgr2=0x7fdb2406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:03.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.572+0000 7fdb28a60640 1 -- 172.21.15.83:0/925771170 shutdown_connections 2024-04-17T10:54:03.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.572+0000 7fdb28a60640 1 -- 172.21.15.83:0/925771170 wait complete. 2024-04-17T10:54:03.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.572+0000 7fdb28a60640 1 Processor -- start 2024-04-17T10:54:03.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.572+0000 7fdb28a60640 1 -- start start 2024-04-17T10:54:03.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.572+0000 7fdb28a60640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb24152460 0x7fdb240aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:03.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.572+0000 7fdb28a60640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb240af1a0 0x7fdb24155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:03.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.572+0000 7fdb28a60640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb24152840 0x7fdb24155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:03.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.572+0000 7fdb28a60640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb240aeae0 con 0x7fdb240af1a0 2024-04-17T10:54:03.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.572+0000 7fdb28a60640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2415cde0 con 0x7fdb24152460 2024-04-17T10:54:03.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.572+0000 7fdb28a60640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2406dca0 con 0x7fdb24152840 2024-04-17T10:54:03.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.572+0000 7fdb21d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb240af1a0 0x7fdb24155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:03.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.572+0000 7fdb21d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb240af1a0 0x7fdb24155030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56516/0 (socket says 172.21.15.83:56516) 2024-04-17T10:54:03.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.572+0000 7fdb21d9b640 1 -- 172.21.15.83:0/3545918989 learned_addr learned my addr 172.21.15.83:0/3545918989 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:03.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.572+0000 7fdb2259c640 1 --2- 172.21.15.83:0/3545918989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb24152460 0x7fdb240aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:03.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.572+0000 7fdb22d9d640 1 --2- 172.21.15.83:0/3545918989 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb24152840 0x7fdb24155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:03.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.572+0000 7fdb21d9b640 1 -- 172.21.15.83:0/3545918989 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb24152840 msgr2=0x7fdb24155570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:03.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.572+0000 7fdb21d9b640 1 --2- 172.21.15.83:0/3545918989 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb24152840 0x7fdb24155570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:03.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.572+0000 7fdb21d9b640 1 -- 172.21.15.83:0/3545918989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb24152460 msgr2=0x7fdb240aec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:03.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.572+0000 7fdb21d9b640 1 --2- 172.21.15.83:0/3545918989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb24152460 0x7fdb240aec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:03.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.572+0000 7fdb21d9b640 1 -- 172.21.15.83:0/3545918989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb240af640 con 0x7fdb240af1a0 2024-04-17T10:54:03.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.576+0000 7fdb21d9b640 1 --2- 172.21.15.83:0/3545918989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb240af1a0 0x7fdb24155030 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7fdb1800d770 tx=0x7fdb1800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:03.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.576+0000 7fdb12ffd640 1 -- 172.21.15.83:0/3545918989 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb18005270 con 0x7fdb240af1a0 2024-04-17T10:54:03.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.576+0000 7fdb12ffd640 1 -- 172.21.15.83:0/3545918989 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb18017070 con 0x7fdb240af1a0 2024-04-17T10:54:03.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.576+0000 7fdb28a60640 1 -- 172.21.15.83:0/3545918989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb240af920 con 0x7fdb240af1a0 2024-04-17T10:54:03.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.576+0000 7fdb12ffd640 1 -- 172.21.15.83:0/3545918989 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1800b6b0 con 0x7fdb240af1a0 2024-04-17T10:54:03.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.576+0000 7fdb28a60640 1 -- 172.21.15.83:0/3545918989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb240afee0 con 0x7fdb240af1a0 2024-04-17T10:54:03.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.576+0000 7fdb28a60640 1 -- 172.21.15.83:0/3545918989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb24074f90 con 0x7fdb240af1a0 2024-04-17T10:54:03.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.576+0000 7fdb12ffd640 1 -- 172.21.15.83:0/3545918989 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdb1800b850 con 0x7fdb240af1a0 2024-04-17T10:54:03.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.576+0000 7fdb12ffd640 1 --2- 172.21.15.83:0/3545918989 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdb04066f60 0x7fdb04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:03.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.576+0000 7fdb12ffd640 1 -- 172.21.15.83:0/3545918989 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fdb18084fc0 con 0x7fdb240af1a0 2024-04-17T10:54:03.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.576+0000 7fdb2259c640 1 --2- 172.21.15.83:0/3545918989 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdb04066f60 0x7fdb04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:03.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.576+0000 7fdb2259c640 1 --2- 172.21.15.83:0/3545918989 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdb04066f60 0x7fdb04069420 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fdb240b0910 tx=0x7fdb14008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:03.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.580+0000 7fdb12ffd640 1 -- 172.21.15.83:0/3545918989 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdb18055f60 con 0x7fdb240af1a0 2024-04-17T10:54:03.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.768+0000 7fdb28a60640 1 -- 172.21.15.83:0/3545918989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb2415e380 con 0x7fdb240af1a0 2024-04-17T10:54:03.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.772+0000 7fdb12ffd640 1 -- 172.21.15.83:0/3545918989 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fdb18055860 con 0x7fdb240af1a0 2024-04-17T10:54:03.764 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:03.764 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:54:03.764 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:54:03.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.772+0000 7fdb28a60640 1 -- 172.21.15.83:0/3545918989 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdb04066f60 msgr2=0x7fdb04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:03.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.772+0000 7fdb28a60640 1 --2- 172.21.15.83:0/3545918989 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdb04066f60 0x7fdb04069420 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fdb240b0910 tx=0x7fdb14008040 comp rx=0 tx=0).stop 2024-04-17T10:54:03.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.772+0000 7fdb28a60640 1 -- 172.21.15.83:0/3545918989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb240af1a0 msgr2=0x7fdb24155030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:03.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.772+0000 7fdb28a60640 1 --2- 172.21.15.83:0/3545918989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb240af1a0 0x7fdb24155030 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7fdb1800d770 tx=0x7fdb1800dc30 comp rx=0 tx=0).stop 2024-04-17T10:54:03.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.772+0000 7fdb28a60640 1 -- 172.21.15.83:0/3545918989 shutdown_connections 2024-04-17T10:54:03.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.772+0000 7fdb28a60640 1 --2- 172.21.15.83:0/3545918989 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fdb04066f60 0x7fdb04069420 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:03.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.772+0000 7fdb28a60640 1 --2- 172.21.15.83:0/3545918989 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb24152840 0x7fdb24155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:03.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.772+0000 7fdb28a60640 1 --2- 172.21.15.83:0/3545918989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb240af1a0 0x7fdb24155030 unknown :-1 s=CLOSED pgs=2358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:03.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.772+0000 7fdb28a60640 1 --2- 172.21.15.83:0/3545918989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb24152460 0x7fdb240aec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:03.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.772+0000 7fdb28a60640 1 -- 172.21.15.83:0/3545918989 >> 172.21.15.83:0/3545918989 conn(0x7fdb2406abe0 msgr2=0x7fdb240cf4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:03.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.772+0000 7fdb28a60640 1 -- 172.21.15.83:0/3545918989 shutdown_connections 2024-04-17T10:54:03.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:03.772+0000 7fdb28a60640 1 -- 172.21.15.83:0/3545918989 wait complete. 2024-04-17T10:54:04.183 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:54:04.184 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:54:05.184 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:54:05.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.320+0000 7fbbb1183640 1 Processor -- start 2024-04-17T10:54:05.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.320+0000 7fbbb1183640 1 -- start start 2024-04-17T10:54:05.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.320+0000 7fbbb1183640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbbac15df40 0x7fbbac06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:05.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.320+0000 7fbbb1183640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbac06afa0 0x7fbbac06e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:05.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.320+0000 7fbbb1183640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbac0ad1a0 con 0x7fbbac0b5c80 2024-04-17T10:54:05.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.320+0000 7fbbb1183640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbac06ee40 con 0x7fbbac15df40 2024-04-17T10:54:05.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.320+0000 7fbbb1183640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac0ad480 con 0x7fbbac06afa0 2024-04-17T10:54:05.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.320+0000 7fbbaad9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbbac15df40 0x7fbbac06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:05.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.320+0000 7fbbaad9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbbac15df40 0x7fbbac06e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56528/0 (socket says 172.21.15.83:56528) 2024-04-17T10:54:05.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.320+0000 7fbbaad9d640 1 -- 172.21.15.83:0/617078159 learned_addr learned my addr 172.21.15.83:0/617078159 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:05.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.320+0000 7fbbab59e640 1 --1- 172.21.15.83:0/617078159 >> v1:172.21.15.83:6789/0 conn(0x7fbbac0b5c80 0x7fbbac06b680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:36560/0 (socket says 172.21.15.83:36560) 2024-04-17T10:54:05.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.320+0000 7fbba259c640 1 --2- 172.21.15.83:0/617078159 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbac06afa0 0x7fbbac06e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:05.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.320+0000 7fbbaad9d640 1 -- 172.21.15.83:0/617078159 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbac06afa0 msgr2=0x7fbbac06e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:05.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.320+0000 7fbbaad9d640 1 --2- 172.21.15.83:0/617078159 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbac06afa0 0x7fbbac06e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:05.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.320+0000 7fbbaad9d640 1 -- 172.21.15.83:0/617078159 >> v1:172.21.15.83:6789/0 conn(0x7fbbac0b5c80 legacy=0x7fbbac06b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:05.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.320+0000 7fbbaad9d640 1 -- 172.21.15.83:0/617078159 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbac0ad760 con 0x7fbbac15df40 2024-04-17T10:54:05.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.324+0000 7fbbaad9d640 1 --2- 172.21.15.83:0/617078159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbac15df40 0x7fbbac06e3c0 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009870 tx=0x7fbb9c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5f7cce5078c22 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:05.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.324+0000 7fbbaa59c640 1 -- 172.21.15.83:0/617078159 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c0b0070 con 0x7fbbac15df40 2024-04-17T10:54:05.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.324+0000 7fbbaa59c640 1 -- 172.21.15.83:0/617078159 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb9c0ab600 con 0x7fbbac15df40 2024-04-17T10:54:05.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.324+0000 7fbbaa59c640 1 -- 172.21.15.83:0/617078159 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c0ab9d0 con 0x7fbbac15df40 2024-04-17T10:54:05.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.324+0000 7fbbb1183640 1 -- 172.21.15.83:0/617078159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbac15df40 msgr2=0x7fbbac06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:05.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.324+0000 7fbbb1183640 1 --2- 172.21.15.83:0/617078159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbac15df40 0x7fbbac06e3c0 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009870 tx=0x7fbb9c01c440 comp rx=0 tx=0).stop 2024-04-17T10:54:05.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.324+0000 7fbbb1183640 1 -- 172.21.15.83:0/617078159 shutdown_connections 2024-04-17T10:54:05.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.324+0000 7fbbb1183640 1 --2- 172.21.15.83:0/617078159 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbac06afa0 0x7fbbac06e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:05.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.324+0000 7fbbb1183640 1 --2- 172.21.15.83:0/617078159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbac15df40 0x7fbbac06e3c0 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:05.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.324+0000 7fbbb1183640 1 -- 172.21.15.83:0/617078159 >> 172.21.15.83:0/617078159 conn(0x7fbbac0b4eb0 msgr2=0x7fbbac0b5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:05.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.324+0000 7fbbb1183640 1 -- 172.21.15.83:0/617078159 shutdown_connections 2024-04-17T10:54:05.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.324+0000 7fbbb1183640 1 -- 172.21.15.83:0/617078159 wait complete. 2024-04-17T10:54:05.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.324+0000 7fbbb1183640 1 Processor -- start 2024-04-17T10:54:05.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.324+0000 7fbbb1183640 1 -- start start 2024-04-17T10:54:05.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.328+0000 7fbbb1183640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbac06afa0 0x7fbbac153b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:05.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.328+0000 7fbbb1183640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbac0b5c80 0x7fbbac147c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:05.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.328+0000 7fbbb1183640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbac15df40 0x7fbbac1602d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:05.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.328+0000 7fbbb1183640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac16adf0 con 0x7fbbac06afa0 2024-04-17T10:54:05.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.328+0000 7fbbb1183640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac16ac70 con 0x7fbbac15df40 2024-04-17T10:54:05.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.328+0000 7fbbb1183640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac16af70 con 0x7fbbac0b5c80 2024-04-17T10:54:05.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.328+0000 7fbbaad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbac06afa0 0x7fbbac153b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:05.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.328+0000 7fbbaad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbac06afa0 0x7fbbac153b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56542/0 (socket says 172.21.15.83:56542) 2024-04-17T10:54:05.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.328+0000 7fbbaad9d640 1 -- 172.21.15.83:0/2995970813 learned_addr learned my addr 172.21.15.83:0/2995970813 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:05.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.328+0000 7fbba259c640 1 --2- 172.21.15.83:0/2995970813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbac0b5c80 0x7fbbac147c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:05.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.328+0000 7fbbab59e640 1 --2- 172.21.15.83:0/2995970813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbac15df40 0x7fbbac1602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:05.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.328+0000 7fbbaad9d640 1 -- 172.21.15.83:0/2995970813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbac0b5c80 msgr2=0x7fbbac147c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:05.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.328+0000 7fbbaad9d640 1 --2- 172.21.15.83:0/2995970813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbac0b5c80 0x7fbbac147c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:05.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.328+0000 7fbbaad9d640 1 -- 172.21.15.83:0/2995970813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbac15df40 msgr2=0x7fbbac1602d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:05.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.328+0000 7fbbaad9d640 1 --2- 172.21.15.83:0/2995970813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbac15df40 0x7fbbac1602d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:05.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.328+0000 7fbbaad9d640 1 -- 172.21.15.83:0/2995970813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbac148150 con 0x7fbbac06afa0 2024-04-17T10:54:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.328+0000 7fbba259c640 1 --2- 172.21.15.83:0/2995970813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbac0b5c80 0x7fbbac147c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.328+0000 7fbbab59e640 1 --2- 172.21.15.83:0/2995970813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbac15df40 0x7fbbac1602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.328+0000 7fbbaad9d640 1 --2- 172.21.15.83:0/2995970813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbac06afa0 0x7fbbac153b30 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0ae040 tx=0x7fbb9c01ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.328+0000 7fbba3fff640 1 -- 172.21.15.83:0/2995970813 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c0ba020 con 0x7fbbac06afa0 2024-04-17T10:54:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.328+0000 7fbba3fff640 1 -- 172.21.15.83:0/2995970813 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb9c0b6620 con 0x7fbbac06afa0 2024-04-17T10:54:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.328+0000 7fbbb1183640 1 -- 172.21.15.83:0/2995970813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbac160b20 con 0x7fbbac06afa0 2024-04-17T10:54:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.328+0000 7fbba3fff640 1 -- 172.21.15.83:0/2995970813 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c0b0040 con 0x7fbbac06afa0 2024-04-17T10:54:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.328+0000 7fbbb1183640 1 -- 172.21.15.83:0/2995970813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbac148420 con 0x7fbbac06afa0 2024-04-17T10:54:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.328+0000 7fbbb1183640 1 -- 172.21.15.83:0/2995970813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbac074f90 con 0x7fbbac06afa0 2024-04-17T10:54:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.328+0000 7fbba3fff640 1 -- 172.21.15.83:0/2995970813 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbb9c0bd020 con 0x7fbbac06afa0 2024-04-17T10:54:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.328+0000 7fbba3fff640 1 --2- 172.21.15.83:0/2995970813 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbb8c066f60 0x7fbb8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.328+0000 7fbba3fff640 1 -- 172.21.15.83:0/2995970813 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fbb9c11f2e0 con 0x7fbbac06afa0 2024-04-17T10:54:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.328+0000 7fbba259c640 1 --2- 172.21.15.83:0/2995970813 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbb8c066f60 0x7fbb8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.332+0000 7fbba259c640 1 --2- 172.21.15.83:0/2995970813 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbb8c066f60 0x7fbb8c069420 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fbbac1fa650 tx=0x7fbb94008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:05.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.336+0000 7fbba3fff640 1 -- 172.21.15.83:0/2995970813 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbb9c0c4030 con 0x7fbbac06afa0 2024-04-17T10:54:05.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.520+0000 7fbbb1183640 1 -- 172.21.15.83:0/2995970813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbbac068140 con 0x7fbbac06afa0 2024-04-17T10:54:05.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.520+0000 7fbba3fff640 1 -- 172.21.15.83:0/2995970813 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fbb9c016150 con 0x7fbbac06afa0 2024-04-17T10:54:05.514 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:05.514 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:54:05.514 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:54:05.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.524+0000 7fbbb1183640 1 -- 172.21.15.83:0/2995970813 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbb8c066f60 msgr2=0x7fbb8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:05.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.524+0000 7fbbb1183640 1 --2- 172.21.15.83:0/2995970813 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbb8c066f60 0x7fbb8c069420 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fbbac1fa650 tx=0x7fbb94008040 comp rx=0 tx=0).stop 2024-04-17T10:54:05.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.524+0000 7fbbb1183640 1 -- 172.21.15.83:0/2995970813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbac06afa0 msgr2=0x7fbbac153b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:05.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.524+0000 7fbbb1183640 1 --2- 172.21.15.83:0/2995970813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbac06afa0 0x7fbbac153b30 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0ae040 tx=0x7fbb9c01ac80 comp rx=0 tx=0).stop 2024-04-17T10:54:05.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.524+0000 7fbbb1183640 1 -- 172.21.15.83:0/2995970813 shutdown_connections 2024-04-17T10:54:05.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.524+0000 7fbbb1183640 1 --2- 172.21.15.83:0/2995970813 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbb8c066f60 0x7fbb8c069420 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:05.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.524+0000 7fbbb1183640 1 --2- 172.21.15.83:0/2995970813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbac15df40 0x7fbbac1602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:05.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.524+0000 7fbbb1183640 1 --2- 172.21.15.83:0/2995970813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbac0b5c80 0x7fbbac147c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:05.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.524+0000 7fbbb1183640 1 --2- 172.21.15.83:0/2995970813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbac06afa0 0x7fbbac153b30 unknown :-1 s=CLOSED pgs=2361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:05.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.524+0000 7fbbb1183640 1 -- 172.21.15.83:0/2995970813 >> 172.21.15.83:0/2995970813 conn(0x7fbbac0b4eb0 msgr2=0x7fbbac0afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:05.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.524+0000 7fbbb1183640 1 -- 172.21.15.83:0/2995970813 shutdown_connections 2024-04-17T10:54:05.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:05.524+0000 7fbbb1183640 1 -- 172.21.15.83:0/2995970813 wait complete. 2024-04-17T10:54:05.934 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:54:05.934 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:54:06.936 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:54:07.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.072+0000 7f6e4228b640 1 Processor -- start 2024-04-17T10:54:07.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.072+0000 7f6e4228b640 1 -- start start 2024-04-17T10:54:07.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.076+0000 7f6e4228b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e3c070b80 0x7f6e3c164d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:07.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.076+0000 7f6e4228b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e3c0b2bd0 con 0x7f6e3c0b3990 2024-04-17T10:54:07.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.076+0000 7f6e4228b640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e3c0b2eb0 con 0x7f6e3c0cf060 2024-04-17T10:54:07.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.076+0000 7f6e4228b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3c0b2980 con 0x7f6e3c070b80 2024-04-17T10:54:07.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.076+0000 7f6e3b7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6e3c0b3990 0x7f6e3c070470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:45384/0 (socket says 172.21.15.83:45384) 2024-04-17T10:54:07.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.076+0000 7f6e3b7fe640 1 -- 172.21.15.83:0/2134722274 learned_addr learned my addr 172.21.15.83:0/2134722274 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:07.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.076+0000 7f6e40828640 1 --2- 172.21.15.83:0/2134722274 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e3c070b80 0x7f6e3c164d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:07.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.076+0000 7f6e3affd640 1 -- 172.21.15.83:0/2134722274 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 120893890 0 0) 0x7f6e3c0b2bd0 con 0x7f6e3c0b3990 2024-04-17T10:54:07.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.076+0000 7f6e3affd640 1 -- 172.21.15.83:0/2134722274 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e24003450 con 0x7f6e3c0b3990 2024-04-17T10:54:07.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.076+0000 7f6e3affd640 1 -- 172.21.15.83:0/2134722274 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3743980111 0 0) 0x7f6e24003450 con 0x7f6e3c0b3990 2024-04-17T10:54:07.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.076+0000 7f6e3affd640 1 -- 172.21.15.83:0/2134722274 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e3c070b80 msgr2=0x7f6e3c164d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:07.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.076+0000 7f6e3affd640 1 --2- 172.21.15.83:0/2134722274 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e3c070b80 0x7f6e3c164d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:07.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.076+0000 7f6e3affd640 1 -- 172.21.15.83:0/2134722274 >> v1:172.21.15.167:6789/0 conn(0x7f6e3c0cf060 legacy=0x7f6e3c0b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:07.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.076+0000 7f6e3affd640 1 -- 172.21.15.83:0/2134722274 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3c0b3190 con 0x7f6e3c0b3990 2024-04-17T10:54:07.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.076+0000 7f6e3affd640 1 -- 172.21.15.83:0/2134722274 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f6e30001680 con 0x7f6e3c0b3990 2024-04-17T10:54:07.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.076+0000 7f6e3affd640 1 -- 172.21.15.83:0/2134722274 >> v1:172.21.15.83:6789/0 conn(0x7f6e3c0b3990 legacy=0x7f6e3c070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:07.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.076+0000 7f6e3affd640 1 --2- 172.21.15.83:0/2134722274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e24003ae0 0x7f6e24003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:07.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.076+0000 7f6e3affd640 1 --2- 172.21.15.83:0/2134722274 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e3c070b80 0x7f6e2400a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:07.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.076+0000 7f6e3affd640 1 --2- 172.21.15.83:0/2134722274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e2400af90 0x7f6e2400d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:07.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.076+0000 7f6e3affd640 1 -- 172.21.15.83:0/2134722274 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3c0b2980 con 0x7f6e2400af90 2024-04-17T10:54:07.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.076+0000 7f6e3affd640 1 -- 172.21.15.83:0/2134722274 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2400dc10 con 0x7f6e24003ae0 2024-04-17T10:54:07.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.076+0000 7f6e3affd640 1 -- 172.21.15.83:0/2134722274 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2400de10 con 0x7f6e3c070b80 2024-04-17T10:54:07.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.076+0000 7f6e40828640 1 --2- 172.21.15.83:0/2134722274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e2400af90 0x7f6e2400d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:07.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.076+0000 7f6e3b7fe640 1 --2- 172.21.15.83:0/2134722274 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e3c070b80 0x7f6e2400a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:07.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.076+0000 7f6e3bfff640 1 --2- 172.21.15.83:0/2134722274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e24003ae0 0x7f6e24003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:07.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.076+0000 7f6e40828640 1 -- 172.21.15.83:0/2134722274 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e3c070b80 msgr2=0x7f6e2400a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:07.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.076+0000 7f6e40828640 1 --2- 172.21.15.83:0/2134722274 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e3c070b80 0x7f6e2400a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:07.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.076+0000 7f6e40828640 1 -- 172.21.15.83:0/2134722274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e24003ae0 msgr2=0x7f6e24003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:07.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.076+0000 7f6e40828640 1 --2- 172.21.15.83:0/2134722274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e24003ae0 0x7f6e24003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:07.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.076+0000 7f6e40828640 1 -- 172.21.15.83:0/2134722274 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e24003450 con 0x7f6e2400af90 2024-04-17T10:54:07.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.080+0000 7f6e40828640 1 --2- 172.21.15.83:0/2134722274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e2400af90 0x7f6e2400d400 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c009ab0 tx=0x7f6e2c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc215b54139aa4f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:07.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.080+0000 7f6e3affd640 1 -- 172.21.15.83:0/2134722274 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c0b2070 con 0x7f6e2400af90 2024-04-17T10:54:07.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.080+0000 7f6e3affd640 1 -- 172.21.15.83:0/2134722274 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2c0a5530 con 0x7f6e2400af90 2024-04-17T10:54:07.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.080+0000 7f6e3affd640 1 -- 172.21.15.83:0/2134722274 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c0096b0 con 0x7f6e2400af90 2024-04-17T10:54:07.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.080+0000 7f6e4228b640 1 -- 172.21.15.83:0/2134722274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e2400af90 msgr2=0x7f6e2400d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:07.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.080+0000 7f6e4228b640 1 --2- 172.21.15.83:0/2134722274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e2400af90 0x7f6e2400d400 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c009ab0 tx=0x7f6e2c01ca50 comp rx=0 tx=0).stop 2024-04-17T10:54:07.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.080+0000 7f6e40828640 1 -- 172.21.15.83:0/2134722274 reap_dead start 2024-04-17T10:54:07.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.080+0000 7f6e4228b640 1 -- 172.21.15.83:0/2134722274 shutdown_connections 2024-04-17T10:54:07.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.080+0000 7f6e4228b640 1 -- 172.21.15.83:0/2134722274 >> 172.21.15.83:0/2134722274 conn(0x7f6e3c06abe0 msgr2=0x7f6e3c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:07.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.080+0000 7f6e4228b640 1 -- 172.21.15.83:0/2134722274 shutdown_connections 2024-04-17T10:54:07.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.080+0000 7f6e4228b640 1 -- 172.21.15.83:0/2134722274 wait complete. 2024-04-17T10:54:07.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.080+0000 7f6e4228b640 1 Processor -- start 2024-04-17T10:54:07.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.080+0000 7f6e4228b640 1 -- start start 2024-04-17T10:54:07.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.084+0000 7f6e4228b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e3c144fe0 0x7f6e3c150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:07.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.084+0000 7f6e4228b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e3c0aec20 0x7f6e3c0b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:07.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.084+0000 7f6e4228b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e3c0af0a0 0x7f6e3c0b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:07.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.084+0000 7f6e4228b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3c0b4770 con 0x7f6e3c0af0a0 2024-04-17T10:54:07.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.084+0000 7f6e4228b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3c15b310 con 0x7f6e3c0aec20 2024-04-17T10:54:07.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.084+0000 7f6e4228b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3c0cf440 con 0x7f6e3c144fe0 2024-04-17T10:54:07.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.084+0000 7f6e40828640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e3c0af0a0 0x7f6e3c0b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:07.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.084+0000 7f6e40828640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e3c0af0a0 0x7f6e3c0b1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59662/0 (socket says 172.21.15.83:59662) 2024-04-17T10:54:07.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.084+0000 7f6e40828640 1 -- 172.21.15.83:0/282336892 learned_addr learned my addr 172.21.15.83:0/282336892 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:07.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.084+0000 7f6e3b7fe640 1 --2- 172.21.15.83:0/282336892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e3c0aec20 0x7f6e3c0b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:07.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.084+0000 7f6e3bfff640 1 --2- 172.21.15.83:0/282336892 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e3c144fe0 0x7f6e3c150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:07.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.084+0000 7f6e40828640 1 -- 172.21.15.83:0/282336892 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e3c144fe0 msgr2=0x7f6e3c150610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:07.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.084+0000 7f6e40828640 1 --2- 172.21.15.83:0/282336892 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e3c144fe0 0x7f6e3c150610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:07.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.084+0000 7f6e40828640 1 -- 172.21.15.83:0/282336892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e3c0aec20 msgr2=0x7f6e3c0b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:07.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.084+0000 7f6e40828640 1 --2- 172.21.15.83:0/282336892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e3c0aec20 0x7f6e3c0b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:07.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.084+0000 7f6e40828640 1 -- 172.21.15.83:0/282336892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3c151910 con 0x7f6e3c0af0a0 2024-04-17T10:54:07.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.084+0000 7f6e40828640 1 --2- 172.21.15.83:0/282336892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e3c0af0a0 0x7f6e3c0b1890 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c009ab0 tx=0x7f6e2c0197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:07.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.084+0000 7f6e38ff9640 1 -- 172.21.15.83:0/282336892 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c0b2040 con 0x7f6e3c0af0a0 2024-04-17T10:54:07.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.084+0000 7f6e38ff9640 1 -- 172.21.15.83:0/282336892 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2c00f000 con 0x7f6e3c0af0a0 2024-04-17T10:54:07.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.084+0000 7f6e4228b640 1 -- 172.21.15.83:0/282336892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e3c151bf0 con 0x7f6e3c0af0a0 2024-04-17T10:54:07.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.084+0000 7f6e38ff9640 1 -- 172.21.15.83:0/282336892 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c0096b0 con 0x7f6e3c0af0a0 2024-04-17T10:54:07.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.084+0000 7f6e4228b640 1 -- 172.21.15.83:0/282336892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e3c1453c0 con 0x7f6e3c0af0a0 2024-04-17T10:54:07.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.084+0000 7f6e4228b640 1 -- 172.21.15.83:0/282336892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e3c074f90 con 0x7f6e3c0af0a0 2024-04-17T10:54:07.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.084+0000 7f6e38ff9640 1 -- 172.21.15.83:0/282336892 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6e2c0bd030 con 0x7f6e3c0af0a0 2024-04-17T10:54:07.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.084+0000 7f6e38ff9640 1 --2- 172.21.15.83:0/282336892 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6e28066f60 0x7f6e28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:07.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.084+0000 7f6e38ff9640 1 -- 172.21.15.83:0/282336892 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f6e2c11e570 con 0x7f6e3c0af0a0 2024-04-17T10:54:07.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.084+0000 7f6e3bfff640 1 --2- 172.21.15.83:0/282336892 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6e28066f60 0x7f6e28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:07.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.088+0000 7f6e3bfff640 1 --2- 172.21.15.83:0/282336892 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6e28066f60 0x7f6e28069420 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f6e34005260 tx=0x7f6e34009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:07.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.088+0000 7f6e38ff9640 1 -- 172.21.15.83:0/282336892 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6e2c0c4060 con 0x7f6e3c0af0a0 2024-04-17T10:54:07.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.280+0000 7f6e4228b640 1 -- 172.21.15.83:0/282336892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e3c070570 con 0x7f6e3c0af0a0 2024-04-17T10:54:07.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.284+0000 7f6e38ff9640 1 -- 172.21.15.83:0/282336892 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f6e2c016760 con 0x7f6e3c0af0a0 2024-04-17T10:54:07.275 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:07.276 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:54:07.276 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:54:07.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.284+0000 7f6e4228b640 1 -- 172.21.15.83:0/282336892 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6e28066f60 msgr2=0x7f6e28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:07.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.284+0000 7f6e4228b640 1 --2- 172.21.15.83:0/282336892 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6e28066f60 0x7f6e28069420 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f6e34005260 tx=0x7f6e34009040 comp rx=0 tx=0).stop 2024-04-17T10:54:07.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.284+0000 7f6e4228b640 1 -- 172.21.15.83:0/282336892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e3c0af0a0 msgr2=0x7f6e3c0b1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:07.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.284+0000 7f6e4228b640 1 --2- 172.21.15.83:0/282336892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e3c0af0a0 0x7f6e3c0b1890 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c009ab0 tx=0x7f6e2c0197f0 comp rx=0 tx=0).stop 2024-04-17T10:54:07.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.284+0000 7f6e4228b640 1 -- 172.21.15.83:0/282336892 shutdown_connections 2024-04-17T10:54:07.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.284+0000 7f6e4228b640 1 --2- 172.21.15.83:0/282336892 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6e28066f60 0x7f6e28069420 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:07.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.284+0000 7f6e4228b640 1 --2- 172.21.15.83:0/282336892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e3c0af0a0 0x7f6e3c0b1890 unknown :-1 s=CLOSED pgs=2364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:07.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.284+0000 7f6e4228b640 1 --2- 172.21.15.83:0/282336892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e3c0aec20 0x7f6e3c0b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:07.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.284+0000 7f6e4228b640 1 --2- 172.21.15.83:0/282336892 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e3c144fe0 0x7f6e3c150610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:07.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.284+0000 7f6e4228b640 1 -- 172.21.15.83:0/282336892 >> 172.21.15.83:0/282336892 conn(0x7f6e3c06abe0 msgr2=0x7f6e3c153260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:07.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.284+0000 7f6e4228b640 1 -- 172.21.15.83:0/282336892 shutdown_connections 2024-04-17T10:54:07.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:07.284+0000 7f6e4228b640 1 -- 172.21.15.83:0/282336892 wait complete. 2024-04-17T10:54:07.697 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:54:07.698 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:54:08.699 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:54:08.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.836+0000 7fed8c792640 1 Processor -- start 2024-04-17T10:54:08.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.836+0000 7fed8c792640 1 -- start start 2024-04-17T10:54:08.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.840+0000 7fed8c792640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fed8415de30 0x7fed840ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:08.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.840+0000 7fed8c792640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed840b6cd0 0x7fed840d2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:08.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.840+0000 7fed8c792640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed840ced00 con 0x7fed840ce860 2024-04-17T10:54:08.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.840+0000 7fed8c792640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed8415e210 con 0x7fed8415de30 2024-04-17T10:54:08.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.840+0000 7fed8c792640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed840cefe0 con 0x7fed840b6cd0 2024-04-17T10:54:08.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.840+0000 7fed89d2d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fed840ce860 0x7fed840d02d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:45396/0 (socket says 172.21.15.83:45396) 2024-04-17T10:54:08.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.840+0000 7fed89d2d640 1 -- 172.21.15.83:0/3052408712 learned_addr learned my addr 172.21.15.83:0/3052408712 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:08.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.840+0000 7fed8ad2f640 1 --2- 172.21.15.83:0/3052408712 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed840b6cd0 0x7fed840d2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:08.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.840+0000 7fed8a52e640 1 --2- 172.21.15.83:0/3052408712 >> v2:172.21.15.167:3300/0 conn(0x7fed8415de30 0x7fed840ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:08.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.840+0000 7fed8952c640 1 -- 172.21.15.83:0/3052408712 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3401026672 0 0) 0x7fed840ced00 con 0x7fed840ce860 2024-04-17T10:54:08.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.840+0000 7fed8952c640 1 -- 172.21.15.83:0/3052408712 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed6c003530 con 0x7fed840ce860 2024-04-17T10:54:08.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.840+0000 7fed8952c640 1 -- 172.21.15.83:0/3052408712 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3285025412 0 0) 0x7fed6c003530 con 0x7fed840ce860 2024-04-17T10:54:08.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.840+0000 7fed8952c640 1 -- 172.21.15.83:0/3052408712 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed840b6cd0 msgr2=0x7fed840d2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:08.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.840+0000 7fed8952c640 1 --2- 172.21.15.83:0/3052408712 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed840b6cd0 0x7fed840d2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:08.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.840+0000 7fed8952c640 1 -- 172.21.15.83:0/3052408712 >> v2:172.21.15.167:3300/0 conn(0x7fed8415de30 msgr2=0x7fed840ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:08.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.840+0000 7fed8952c640 1 --2- 172.21.15.83:0/3052408712 >> v2:172.21.15.167:3300/0 conn(0x7fed8415de30 0x7fed840ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:08.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.840+0000 7fed8952c640 1 -- 172.21.15.83:0/3052408712 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed840d2040 con 0x7fed840ce860 2024-04-17T10:54:08.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.840+0000 7fed8952c640 1 -- 172.21.15.83:0/3052408712 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fed78001680 con 0x7fed840ce860 2024-04-17T10:54:08.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.840+0000 7fed8952c640 1 -- 172.21.15.83:0/3052408712 >> v1:172.21.15.83:6789/0 conn(0x7fed840ce860 legacy=0x7fed840d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:08.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.840+0000 7fed8952c640 1 --2- 172.21.15.83:0/3052408712 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed840b6cd0 0x7fed6c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:08.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.840+0000 7fed8952c640 1 --2- 172.21.15.83:0/3052408712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed6c003f40 0x7fed6c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:08.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.840+0000 7fed8952c640 1 --2- 172.21.15.83:0/3052408712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed6c004d50 0x7fed6c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:08.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.840+0000 7fed8952c640 1 -- 172.21.15.83:0/3052408712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed840cefe0 con 0x7fed6c003f40 2024-04-17T10:54:08.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.840+0000 7fed8952c640 1 -- 172.21.15.83:0/3052408712 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c00b9d0 con 0x7fed6c004d50 2024-04-17T10:54:08.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.840+0000 7fed8952c640 1 -- 172.21.15.83:0/3052408712 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c00bba0 con 0x7fed840b6cd0 2024-04-17T10:54:08.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.840+0000 7fed89d2d640 1 --2- 172.21.15.83:0/3052408712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed6c003f40 0x7fed6c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:08.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.840+0000 7fed8a52e640 1 --2- 172.21.15.83:0/3052408712 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed840b6cd0 0x7fed6c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:08.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.840+0000 7fed89d2d640 1 -- 172.21.15.83:0/3052408712 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed840b6cd0 msgr2=0x7fed6c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:08.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.840+0000 7fed89d2d640 1 --2- 172.21.15.83:0/3052408712 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed840b6cd0 0x7fed6c0039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:08.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.840+0000 7fed89d2d640 1 -- 172.21.15.83:0/3052408712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed6c004d50 msgr2=0x7fed6c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:08.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.840+0000 7fed89d2d640 1 --2- 172.21.15.83:0/3052408712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed6c004d50 0x7fed6c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:08.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.840+0000 7fed89d2d640 1 -- 172.21.15.83:0/3052408712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6c003530 con 0x7fed6c003f40 2024-04-17T10:54:08.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed89d2d640 1 --2- 172.21.15.83:0/3052408712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed6c003f40 0x7fed6c004810 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7fed7800c240 tx=0x7fed7801cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76944c8c2a43af6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:08.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed8952c640 1 -- 172.21.15.83:0/3052408712 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed78007d60 con 0x7fed6c003f40 2024-04-17T10:54:08.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed8952c640 1 -- 172.21.15.83:0/3052408712 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed780a9020 con 0x7fed6c003f40 2024-04-17T10:54:08.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed8952c640 1 -- 172.21.15.83:0/3052408712 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed780a5b10 con 0x7fed6c003f40 2024-04-17T10:54:08.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed8c792640 1 -- 172.21.15.83:0/3052408712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed6c003f40 msgr2=0x7fed6c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:08.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed8c792640 1 --2- 172.21.15.83:0/3052408712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed6c003f40 0x7fed6c004810 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7fed7800c240 tx=0x7fed7801cc50 comp rx=0 tx=0).stop 2024-04-17T10:54:08.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed8ad2f640 1 -- 172.21.15.83:0/3052408712 reap_dead start 2024-04-17T10:54:08.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed8c792640 1 -- 172.21.15.83:0/3052408712 shutdown_connections 2024-04-17T10:54:08.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed8c792640 1 -- 172.21.15.83:0/3052408712 >> 172.21.15.83:0/3052408712 conn(0x7fed8406b0b0 msgr2=0x7fed8406b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:08.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed8c792640 1 -- 172.21.15.83:0/3052408712 shutdown_connections 2024-04-17T10:54:08.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed8c792640 1 -- 172.21.15.83:0/3052408712 wait complete. 2024-04-17T10:54:08.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed8c792640 1 Processor -- start 2024-04-17T10:54:08.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed8c792640 1 -- start start 2024-04-17T10:54:08.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed8c792640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed6c003f40 0x7fed84163cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:08.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed8c792640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed840aa320 0x7fed840b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:08.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed8c792640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed8415c670 0x7fed8415cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:08.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed8c792640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed840d3050 con 0x7fed6c003f40 2024-04-17T10:54:08.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed8c792640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed84161c00 con 0x7fed840aa320 2024-04-17T10:54:08.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed8c792640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed840d31d0 con 0x7fed8415c670 2024-04-17T10:54:08.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed8a52e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed6c003f40 0x7fed84163cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:08.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed8a52e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed6c003f40 0x7fed84163cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59678/0 (socket says 172.21.15.83:59678) 2024-04-17T10:54:08.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed8a52e640 1 -- 172.21.15.83:0/2747200888 learned_addr learned my addr 172.21.15.83:0/2747200888 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:08.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed89d2d640 1 --2- 172.21.15.83:0/2747200888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed840aa320 0x7fed840b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:08.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed8ad2f640 1 --2- 172.21.15.83:0/2747200888 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed8415c670 0x7fed8415cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:08.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed8a52e640 1 -- 172.21.15.83:0/2747200888 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed8415c670 msgr2=0x7fed8415cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:08.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed8a52e640 1 --2- 172.21.15.83:0/2747200888 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed8415c670 0x7fed8415cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:08.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed8a52e640 1 -- 172.21.15.83:0/2747200888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed840aa320 msgr2=0x7fed840b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:08.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed8a52e640 1 --2- 172.21.15.83:0/2747200888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed840aa320 0x7fed840b49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:08.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed8a52e640 1 -- 172.21.15.83:0/2747200888 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed84164290 con 0x7fed6c003f40 2024-04-17T10:54:08.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed8a52e640 1 --2- 172.21.15.83:0/2747200888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed6c003f40 0x7fed84163cb0 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7fed740108c0 tx=0x7fed74010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:08.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed82ffd640 1 -- 172.21.15.83:0/2747200888 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7400f040 con 0x7fed6c003f40 2024-04-17T10:54:08.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed82ffd640 1 -- 172.21.15.83:0/2747200888 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7400eb60 con 0x7fed6c003f40 2024-04-17T10:54:08.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed8c792640 1 -- 172.21.15.83:0/2747200888 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed84164570 con 0x7fed6c003f40 2024-04-17T10:54:08.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.844+0000 7fed82ffd640 1 -- 172.21.15.83:0/2747200888 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7401c400 con 0x7fed6c003f40 2024-04-17T10:54:08.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.848+0000 7fed8c792640 1 -- 172.21.15.83:0/2747200888 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed84164b30 con 0x7fed6c003f40 2024-04-17T10:54:08.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.848+0000 7fed8c792640 1 -- 172.21.15.83:0/2747200888 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed84074f90 con 0x7fed6c003f40 2024-04-17T10:54:08.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.848+0000 7fed82ffd640 1 -- 172.21.15.83:0/2747200888 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fed7401c5a0 con 0x7fed6c003f40 2024-04-17T10:54:08.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.848+0000 7fed82ffd640 1 --2- 172.21.15.83:0/2747200888 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fed70066f60 0x7fed70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:08.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.848+0000 7fed82ffd640 1 -- 172.21.15.83:0/2747200888 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fed74085a40 con 0x7fed6c003f40 2024-04-17T10:54:08.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.848+0000 7fed89d2d640 1 --2- 172.21.15.83:0/2747200888 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fed70066f60 0x7fed70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:08.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.848+0000 7fed89d2d640 1 --2- 172.21.15.83:0/2747200888 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fed70066f60 0x7fed70069420 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fed780aa000 tx=0x7fed78001ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:08.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:08.852+0000 7fed82ffd640 1 -- 172.21.15.83:0/2747200888 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fed74018030 con 0x7fed6c003f40 2024-04-17T10:54:09.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:09.036+0000 7fed8c792640 1 -- 172.21.15.83:0/2747200888 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed8415d060 con 0x7fed6c003f40 2024-04-17T10:54:09.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:09.036+0000 7fed82ffd640 1 -- 172.21.15.83:0/2747200888 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fed74056b70 con 0x7fed6c003f40 2024-04-17T10:54:09.031 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:09.031 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:54:09.031 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:54:09.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:09.040+0000 7fed8c792640 1 -- 172.21.15.83:0/2747200888 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fed70066f60 msgr2=0x7fed70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:09.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:09.040+0000 7fed8c792640 1 --2- 172.21.15.83:0/2747200888 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fed70066f60 0x7fed70069420 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fed780aa000 tx=0x7fed78001ac0 comp rx=0 tx=0).stop 2024-04-17T10:54:09.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:09.040+0000 7fed8c792640 1 -- 172.21.15.83:0/2747200888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed6c003f40 msgr2=0x7fed84163cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:09.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:09.040+0000 7fed8c792640 1 --2- 172.21.15.83:0/2747200888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed6c003f40 0x7fed84163cb0 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7fed740108c0 tx=0x7fed74010d80 comp rx=0 tx=0).stop 2024-04-17T10:54:09.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:09.040+0000 7fed8c792640 1 -- 172.21.15.83:0/2747200888 shutdown_connections 2024-04-17T10:54:09.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:09.040+0000 7fed8c792640 1 --2- 172.21.15.83:0/2747200888 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fed70066f60 0x7fed70069420 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:09.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:09.040+0000 7fed8c792640 1 --2- 172.21.15.83:0/2747200888 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed8415c670 0x7fed8415cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:09.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:09.040+0000 7fed8c792640 1 --2- 172.21.15.83:0/2747200888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed840aa320 0x7fed840b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:09.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:09.040+0000 7fed8c792640 1 --2- 172.21.15.83:0/2747200888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed6c003f40 0x7fed84163cb0 unknown :-1 s=CLOSED pgs=2367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:09.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:09.040+0000 7fed8c792640 1 -- 172.21.15.83:0/2747200888 >> 172.21.15.83:0/2747200888 conn(0x7fed8406b0b0 msgr2=0x7fed8415e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:09.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:09.040+0000 7fed8c792640 1 -- 172.21.15.83:0/2747200888 shutdown_connections 2024-04-17T10:54:09.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:09.040+0000 7fed8c792640 1 -- 172.21.15.83:0/2747200888 wait complete. 2024-04-17T10:54:09.426 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:54:09.429 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:54:09.450 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:54:09.450 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:54:10.451 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:54:10.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.584+0000 7fc1303fb640 1 Processor -- start 2024-04-17T10:54:10.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.584+0000 7fc1303fb640 1 -- start start 2024-04-17T10:54:10.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.588+0000 7fc1303fb640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1280b3990 0x7fc128070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:10.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.588+0000 7fc1303fb640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1280b2bd0 con 0x7fc1280cf060 2024-04-17T10:54:10.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.588+0000 7fc1303fb640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1280b2eb0 con 0x7fc1280709b0 2024-04-17T10:54:10.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.588+0000 7fc1303fb640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc128070e20 con 0x7fc1280b3990 2024-04-17T10:54:10.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.588+0000 7fc12e197640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc1280cf060 0x7fc1280b4060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:45404/0 (socket says 172.21.15.83:45404) 2024-04-17T10:54:10.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.588+0000 7fc12e197640 1 -- 172.21.15.83:0/3090591453 learned_addr learned my addr 172.21.15.83:0/3090591453 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:10.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.588+0000 7fc12d195640 1 -- 172.21.15.83:0/3090591453 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 584380802 0 0) 0x7fc1280b2bd0 con 0x7fc1280cf060 2024-04-17T10:54:10.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.588+0000 7fc12d195640 1 -- 172.21.15.83:0/3090591453 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc114003450 con 0x7fc1280cf060 2024-04-17T10:54:10.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.588+0000 7fc12d996640 1 --2- 172.21.15.83:0/3090591453 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1280b3990 0x7fc128070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:10.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.588+0000 7fc12d195640 1 -- 172.21.15.83:0/3090591453 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1281837486 0 0) 0x7fc114003450 con 0x7fc1280cf060 2024-04-17T10:54:10.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.588+0000 7fc12d195640 1 -- 172.21.15.83:0/3090591453 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1280b3990 msgr2=0x7fc128070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:10.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.588+0000 7fc12d195640 1 --2- 172.21.15.83:0/3090591453 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1280b3990 0x7fc128070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:10.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.588+0000 7fc12d195640 1 -- 172.21.15.83:0/3090591453 >> v1:172.21.15.167:6789/0 conn(0x7fc1280709b0 legacy=0x7fc12806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:10.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.588+0000 7fc12d195640 1 -- 172.21.15.83:0/3090591453 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1280b3190 con 0x7fc1280cf060 2024-04-17T10:54:10.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.588+0000 7fc12d195640 1 -- 172.21.15.83:0/3090591453 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fc124001680 con 0x7fc1280cf060 2024-04-17T10:54:10.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.588+0000 7fc12d195640 1 -- 172.21.15.83:0/3090591453 >> v1:172.21.15.83:6789/0 conn(0x7fc1280cf060 legacy=0x7fc1280b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:10.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.588+0000 7fc12d195640 1 --2- 172.21.15.83:0/3090591453 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1280b3990 0x7fc114003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:10.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.588+0000 7fc12d195640 1 --2- 172.21.15.83:0/3090591453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc114004060 0x7fc11400a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:10.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.588+0000 7fc12d195640 1 --2- 172.21.15.83:0/3090591453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc11400b000 0x7fc11400d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:10.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.588+0000 7fc12d195640 1 -- 172.21.15.83:0/3090591453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc128070e20 con 0x7fc114004060 2024-04-17T10:54:10.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.588+0000 7fc12d195640 1 -- 172.21.15.83:0/3090591453 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11400dcd0 con 0x7fc11400b000 2024-04-17T10:54:10.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.588+0000 7fc12d195640 1 -- 172.21.15.83:0/3090591453 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11400ded0 con 0x7fc1280b3990 2024-04-17T10:54:10.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.588+0000 7fc12d996640 1 --2- 172.21.15.83:0/3090591453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc114004060 0x7fc11400a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:10.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.588+0000 7fc12e197640 1 --2- 172.21.15.83:0/3090591453 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1280b3990 0x7fc114003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:10.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.588+0000 7fc12d996640 1 -- 172.21.15.83:0/3090591453 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1280b3990 msgr2=0x7fc114003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:10.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.588+0000 7fc12e998640 1 --2- 172.21.15.83:0/3090591453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc11400b000 0x7fc11400d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:10.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.588+0000 7fc12d996640 1 --2- 172.21.15.83:0/3090591453 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1280b3990 0x7fc114003ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:10.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.588+0000 7fc12d996640 1 -- 172.21.15.83:0/3090591453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc11400b000 msgr2=0x7fc11400d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:10.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.588+0000 7fc12d996640 1 --2- 172.21.15.83:0/3090591453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc11400b000 0x7fc11400d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:10.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.588+0000 7fc12d996640 1 -- 172.21.15.83:0/3090591453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc114003450 con 0x7fc114004060 2024-04-17T10:54:10.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc12d996640 1 --2- 172.21.15.83:0/3090591453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc114004060 0x7fc11400a870 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7fc118009a90 tx=0x7fc11801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f90a62124dd4ae server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:10.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc12d195640 1 -- 172.21.15.83:0/3090591453 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1180b2070 con 0x7fc114004060 2024-04-17T10:54:10.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc12d195640 1 -- 172.21.15.83:0/3090591453 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1180a5620 con 0x7fc114004060 2024-04-17T10:54:10.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc12d195640 1 -- 172.21.15.83:0/3090591453 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc118009690 con 0x7fc114004060 2024-04-17T10:54:10.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc1303fb640 1 -- 172.21.15.83:0/3090591453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc114004060 msgr2=0x7fc11400a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:10.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc1303fb640 1 --2- 172.21.15.83:0/3090591453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc114004060 0x7fc11400a870 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7fc118009a90 tx=0x7fc11801ca30 comp rx=0 tx=0).stop 2024-04-17T10:54:10.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc12e998640 1 -- 172.21.15.83:0/3090591453 reap_dead start 2024-04-17T10:54:10.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc1303fb640 1 -- 172.21.15.83:0/3090591453 shutdown_connections 2024-04-17T10:54:10.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc1303fb640 1 -- 172.21.15.83:0/3090591453 >> 172.21.15.83:0/3090591453 conn(0x7fc12806abe0 msgr2=0x7fc12806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:10.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc1303fb640 1 -- 172.21.15.83:0/3090591453 shutdown_connections 2024-04-17T10:54:10.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc1303fb640 1 -- 172.21.15.83:0/3090591453 wait complete. 2024-04-17T10:54:10.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc1303fb640 1 Processor -- start 2024-04-17T10:54:10.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc1303fb640 1 -- start start 2024-04-17T10:54:10.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc1303fb640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1280a9ce0 0x7fc12815a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:10.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc1303fb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc12815a980 0x7fc128143390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:10.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc1303fb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1281438d0 0x7fc1281ee8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:10.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc1303fb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12815b310 con 0x7fc12815a980 2024-04-17T10:54:10.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc1303fb640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1280b2980 con 0x7fc1281438d0 2024-04-17T10:54:10.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc1303fb640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1280b4770 con 0x7fc1280a9ce0 2024-04-17T10:54:10.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc12d996640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc12815a980 0x7fc128143390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:10.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc12d996640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc12815a980 0x7fc128143390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59704/0 (socket says 172.21.15.83:59704) 2024-04-17T10:54:10.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc12d996640 1 -- 172.21.15.83:0/164254142 learned_addr learned my addr 172.21.15.83:0/164254142 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:10.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc12e197640 1 --2- 172.21.15.83:0/164254142 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1280a9ce0 0x7fc12815a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:10.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc12e998640 1 --2- 172.21.15.83:0/164254142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1281438d0 0x7fc1281ee8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:10.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc12d996640 1 -- 172.21.15.83:0/164254142 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1280a9ce0 msgr2=0x7fc12815a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:10.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc12d996640 1 --2- 172.21.15.83:0/164254142 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1280a9ce0 0x7fc12815a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:10.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc12d996640 1 -- 172.21.15.83:0/164254142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1281438d0 msgr2=0x7fc1281ee8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:10.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc12d996640 1 --2- 172.21.15.83:0/164254142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1281438d0 0x7fc1281ee8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:10.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc12d996640 1 -- 172.21.15.83:0/164254142 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1281440f0 con 0x7fc12815a980 2024-04-17T10:54:10.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc12d996640 1 --2- 172.21.15.83:0/164254142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc12815a980 0x7fc128143390 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7fc118009a90 tx=0x7fc1180197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:10.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc11effd640 1 -- 172.21.15.83:0/164254142 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1180b2040 con 0x7fc12815a980 2024-04-17T10:54:10.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc11effd640 1 -- 172.21.15.83:0/164254142 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1180b8440 con 0x7fc12815a980 2024-04-17T10:54:10.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.592+0000 7fc1303fb640 1 -- 172.21.15.83:0/164254142 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc128141210 con 0x7fc12815a980 2024-04-17T10:54:10.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.596+0000 7fc11effd640 1 -- 172.21.15.83:0/164254142 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1180b8810 con 0x7fc12815a980 2024-04-17T10:54:10.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.596+0000 7fc1303fb640 1 -- 172.21.15.83:0/164254142 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1281417d0 con 0x7fc12815a980 2024-04-17T10:54:10.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.596+0000 7fc1303fb640 1 -- 172.21.15.83:0/164254142 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc128074f90 con 0x7fc12815a980 2024-04-17T10:54:10.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.596+0000 7fc11effd640 1 -- 172.21.15.83:0/164254142 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc1180bf060 con 0x7fc12815a980 2024-04-17T10:54:10.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.596+0000 7fc11effd640 1 --2- 172.21.15.83:0/164254142 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc10c066f90 0x7fc10c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:10.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.596+0000 7fc11effd640 1 -- 172.21.15.83:0/164254142 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fc11811e810 con 0x7fc12815a980 2024-04-17T10:54:10.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.596+0000 7fc12e197640 1 --2- 172.21.15.83:0/164254142 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc10c066f90 0x7fc10c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:10.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.596+0000 7fc12e197640 1 --2- 172.21.15.83:0/164254142 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc10c066f90 0x7fc10c069450 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fc124001060 tx=0x7fc124000ff0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:10.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.600+0000 7fc11effd640 1 -- 172.21.15.83:0/164254142 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc118016740 con 0x7fc12815a980 2024-04-17T10:54:10.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.776+0000 7fc1303fb640 1 -- 172.21.15.83:0/164254142 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc128159d00 con 0x7fc12815a980 2024-04-17T10:54:10.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.776+0000 7fc11effd640 1 -- 172.21.15.83:0/164254142 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fc1180ef7b0 con 0x7fc12815a980 2024-04-17T10:54:10.770 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:10.770 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:54:10.770 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:54:10.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.780+0000 7fc1303fb640 1 -- 172.21.15.83:0/164254142 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc10c066f90 msgr2=0x7fc10c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:10.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.780+0000 7fc1303fb640 1 --2- 172.21.15.83:0/164254142 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc10c066f90 0x7fc10c069450 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fc124001060 tx=0x7fc124000ff0 comp rx=0 tx=0).stop 2024-04-17T10:54:10.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.780+0000 7fc1303fb640 1 -- 172.21.15.83:0/164254142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc12815a980 msgr2=0x7fc128143390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:10.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.780+0000 7fc1303fb640 1 --2- 172.21.15.83:0/164254142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc12815a980 0x7fc128143390 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7fc118009a90 tx=0x7fc1180197d0 comp rx=0 tx=0).stop 2024-04-17T10:54:10.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.780+0000 7fc1303fb640 1 -- 172.21.15.83:0/164254142 shutdown_connections 2024-04-17T10:54:10.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.780+0000 7fc1303fb640 1 --2- 172.21.15.83:0/164254142 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fc10c066f90 0x7fc10c069450 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:10.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.780+0000 7fc1303fb640 1 --2- 172.21.15.83:0/164254142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1281438d0 0x7fc1281ee8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:10.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.780+0000 7fc1303fb640 1 --2- 172.21.15.83:0/164254142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc12815a980 0x7fc128143390 unknown :-1 s=CLOSED pgs=2370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:10.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.780+0000 7fc1303fb640 1 --2- 172.21.15.83:0/164254142 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1280a9ce0 0x7fc12815a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:10.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.780+0000 7fc1303fb640 1 -- 172.21.15.83:0/164254142 >> 172.21.15.83:0/164254142 conn(0x7fc12806abe0 msgr2=0x7fc128155120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:10.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.780+0000 7fc1303fb640 1 -- 172.21.15.83:0/164254142 shutdown_connections 2024-04-17T10:54:10.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:10.780+0000 7fc1303fb640 1 -- 172.21.15.83:0/164254142 wait complete. 2024-04-17T10:54:11.189 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:54:11.190 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:54:12.191 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:54:12.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.328+0000 7fbd1d8e2640 1 Processor -- start 2024-04-17T10:54:12.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.328+0000 7fbd1d8e2640 1 -- start start 2024-04-17T10:54:12.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd1d8e2640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd1815ba30 0x7fbd18072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:12.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd1d8e2640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbd180cfbe0 0x7fbd1806c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:12.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd1d8e2640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd1806e320 con 0x7fbd1815c960 2024-04-17T10:54:12.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd1d8e2640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd18071ea0 con 0x7fbd180cfbe0 2024-04-17T10:54:12.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd1d8e2640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1806e040 con 0x7fbd1815ba30 2024-04-17T10:54:12.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd0e7fc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbd1815c960 0x7fbd180cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:45408/0 (socket says 172.21.15.83:45408) 2024-04-17T10:54:12.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd0e7fc640 1 -- 172.21.15.83:0/2003659711 learned_addr learned my addr 172.21.15.83:0/2003659711 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:12.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd16ffd640 1 --2- 172.21.15.83:0/2003659711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd1815ba30 0x7fbd18072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:12.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd177fe640 1 --2- 172.21.15.83:0/2003659711 >> v2:172.21.15.167:3300/0 conn(0x7fbd180cfbe0 0x7fbd1806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:12.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd167fc640 1 -- 172.21.15.83:0/2003659711 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3618600601 0 0) 0x7fbd1806e320 con 0x7fbd1815c960 2024-04-17T10:54:12.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd167fc640 1 -- 172.21.15.83:0/2003659711 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd04003530 con 0x7fbd1815c960 2024-04-17T10:54:12.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd167fc640 1 -- 172.21.15.83:0/2003659711 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3941724149 0 0) 0x7fbd04003530 con 0x7fbd1815c960 2024-04-17T10:54:12.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd167fc640 1 -- 172.21.15.83:0/2003659711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd1815ba30 msgr2=0x7fbd18072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:12.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd167fc640 1 --2- 172.21.15.83:0/2003659711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd1815ba30 0x7fbd18072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:12.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd167fc640 1 -- 172.21.15.83:0/2003659711 >> v2:172.21.15.167:3300/0 conn(0x7fbd180cfbe0 msgr2=0x7fbd1806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:12.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd167fc640 1 --2- 172.21.15.83:0/2003659711 >> v2:172.21.15.167:3300/0 conn(0x7fbd180cfbe0 0x7fbd1806c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:12.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd167fc640 1 -- 172.21.15.83:0/2003659711 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd18071850 con 0x7fbd1815c960 2024-04-17T10:54:12.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd167fc640 1 -- 172.21.15.83:0/2003659711 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fbd000016a0 con 0x7fbd1815c960 2024-04-17T10:54:12.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd167fc640 1 -- 172.21.15.83:0/2003659711 >> v1:172.21.15.83:6789/0 conn(0x7fbd1815c960 legacy=0x7fbd180cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:12.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd167fc640 1 --2- 172.21.15.83:0/2003659711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd040039c0 0x7fbd04003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:12.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd167fc640 1 --2- 172.21.15.83:0/2003659711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd1815ba30 0x7fbd04004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:12.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd167fc640 1 --2- 172.21.15.83:0/2003659711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd04004d50 0x7fbd040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:12.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd167fc640 1 -- 172.21.15.83:0/2003659711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd18071ea0 con 0x7fbd040039c0 2024-04-17T10:54:12.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd167fc640 1 -- 172.21.15.83:0/2003659711 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1806e040 con 0x7fbd04004d50 2024-04-17T10:54:12.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd167fc640 1 -- 172.21.15.83:0/2003659711 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0400b9d0 con 0x7fbd1815ba30 2024-04-17T10:54:12.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd16ffd640 1 --2- 172.21.15.83:0/2003659711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd040039c0 0x7fbd04003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:12.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd177fe640 1 --2- 172.21.15.83:0/2003659711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd04004d50 0x7fbd040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:12.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd0e7fc640 1 --2- 172.21.15.83:0/2003659711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd1815ba30 0x7fbd04004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:12.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd16ffd640 1 -- 172.21.15.83:0/2003659711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd1815ba30 msgr2=0x7fbd04004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:12.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd16ffd640 1 --2- 172.21.15.83:0/2003659711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd1815ba30 0x7fbd04004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:12.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd16ffd640 1 -- 172.21.15.83:0/2003659711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd04004d50 msgr2=0x7fbd040051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:12.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd16ffd640 1 --2- 172.21.15.83:0/2003659711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd04004d50 0x7fbd040051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:12.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd16ffd640 1 -- 172.21.15.83:0/2003659711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd04003530 con 0x7fbd040039c0 2024-04-17T10:54:12.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd16ffd640 1 --2- 172.21.15.83:0/2003659711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd040039c0 0x7fbd04003de0 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7fbd08009ae0 tx=0x7fbd0801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba51a34da250d1b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:12.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd167fc640 1 -- 172.21.15.83:0/2003659711 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd080096b0 con 0x7fbd040039c0 2024-04-17T10:54:12.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd167fc640 1 -- 172.21.15.83:0/2003659711 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd080a6450 con 0x7fbd040039c0 2024-04-17T10:54:12.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.332+0000 7fbd167fc640 1 -- 172.21.15.83:0/2003659711 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd080a6840 con 0x7fbd040039c0 2024-04-17T10:54:12.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.336+0000 7fbd1d8e2640 1 -- 172.21.15.83:0/2003659711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd040039c0 msgr2=0x7fbd04003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:12.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.336+0000 7fbd1d8e2640 1 --2- 172.21.15.83:0/2003659711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd040039c0 0x7fbd04003de0 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7fbd08009ae0 tx=0x7fbd0801ca70 comp rx=0 tx=0).stop 2024-04-17T10:54:12.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.336+0000 7fbd177fe640 1 -- 172.21.15.83:0/2003659711 reap_dead start 2024-04-17T10:54:12.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.336+0000 7fbd1d8e2640 1 -- 172.21.15.83:0/2003659711 shutdown_connections 2024-04-17T10:54:12.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.336+0000 7fbd1d8e2640 1 -- 172.21.15.83:0/2003659711 >> 172.21.15.83:0/2003659711 conn(0x7fbd1806abe0 msgr2=0x7fbd1806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:12.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.336+0000 7fbd1d8e2640 1 -- 172.21.15.83:0/2003659711 shutdown_connections 2024-04-17T10:54:12.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.336+0000 7fbd1d8e2640 1 -- 172.21.15.83:0/2003659711 wait complete. 2024-04-17T10:54:12.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.336+0000 7fbd1d8e2640 1 Processor -- start 2024-04-17T10:54:12.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.336+0000 7fbd1d8e2640 1 -- start start 2024-04-17T10:54:12.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.336+0000 7fbd1d8e2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd040039c0 0x7fbd18154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:12.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.336+0000 7fbd1d8e2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd181641f0 0x7fbd18159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:12.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.336+0000 7fbd1d8e2640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd1815a340 0x7fbd180ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:12.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.336+0000 7fbd1d8e2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1815cf00 con 0x7fbd181641f0 2024-04-17T10:54:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.336+0000 7fbd1d8e2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1815cd80 con 0x7fbd040039c0 2024-04-17T10:54:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.336+0000 7fbd1d8e2640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd18166510 con 0x7fbd1815a340 2024-04-17T10:54:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.336+0000 7fbd0e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd181641f0 0x7fbd18159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.336+0000 7fbd0e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd181641f0 0x7fbd18159e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59714/0 (socket says 172.21.15.83:59714) 2024-04-17T10:54:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.336+0000 7fbd0e7fc640 1 -- 172.21.15.83:0/2025740537 learned_addr learned my addr 172.21.15.83:0/2025740537 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.336+0000 7fbd177fe640 1 --2- 172.21.15.83:0/2025740537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd1815a340 0x7fbd180ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.336+0000 7fbd16ffd640 1 --2- 172.21.15.83:0/2025740537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd040039c0 0x7fbd18154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.336+0000 7fbd0e7fc640 1 -- 172.21.15.83:0/2025740537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd1815a340 msgr2=0x7fbd180ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.336+0000 7fbd0e7fc640 1 --2- 172.21.15.83:0/2025740537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd1815a340 0x7fbd180ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.336+0000 7fbd0e7fc640 1 -- 172.21.15.83:0/2025740537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd040039c0 msgr2=0x7fbd18154700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.336+0000 7fbd0e7fc640 1 --2- 172.21.15.83:0/2025740537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd040039c0 0x7fbd18154700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.336+0000 7fbd0e7fc640 1 -- 172.21.15.83:0/2025740537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd18154c40 con 0x7fbd181641f0 2024-04-17T10:54:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.336+0000 7fbd177fe640 1 --2- 172.21.15.83:0/2025740537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd1815a340 0x7fbd180ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.336+0000 7fbd0e7fc640 1 --2- 172.21.15.83:0/2025740537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd181641f0 0x7fbd18159e00 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7fbd0000d7a0 tx=0x7fbd0000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.336+0000 7fbd14ff9640 1 -- 172.21.15.83:0/2025740537 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd00019070 con 0x7fbd181641f0 2024-04-17T10:54:12.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.336+0000 7fbd14ff9640 1 -- 172.21.15.83:0/2025740537 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd000074e0 con 0x7fbd181641f0 2024-04-17T10:54:12.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.336+0000 7fbd1d8e2640 1 -- 172.21.15.83:0/2025740537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd18163150 con 0x7fbd181641f0 2024-04-17T10:54:12.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.336+0000 7fbd14ff9640 1 -- 172.21.15.83:0/2025740537 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd00004a10 con 0x7fbd181641f0 2024-04-17T10:54:12.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.336+0000 7fbd1d8e2640 1 -- 172.21.15.83:0/2025740537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd18163710 con 0x7fbd181641f0 2024-04-17T10:54:12.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.340+0000 7fbd1d8e2640 1 -- 172.21.15.83:0/2025740537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd18074f90 con 0x7fbd181641f0 2024-04-17T10:54:12.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.340+0000 7fbd14ff9640 1 -- 172.21.15.83:0/2025740537 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbd0001c070 con 0x7fbd181641f0 2024-04-17T10:54:12.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.340+0000 7fbd14ff9640 1 --2- 172.21.15.83:0/2025740537 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbcf8066f90 0x7fbcf8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:12.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.340+0000 7fbd16ffd640 1 --2- 172.21.15.83:0/2025740537 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbcf8066f90 0x7fbcf8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:12.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.340+0000 7fbd14ff9640 1 -- 172.21.15.83:0/2025740537 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fbd00084a90 con 0x7fbd181641f0 2024-04-17T10:54:12.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.340+0000 7fbd16ffd640 1 --2- 172.21.15.83:0/2025740537 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbcf8066f90 0x7fbcf8069450 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fbd080ad000 tx=0x7fbd080046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:12.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.344+0000 7fbd14ff9640 1 -- 172.21.15.83:0/2025740537 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbd00055ac0 con 0x7fbd181641f0 2024-04-17T10:54:12.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.532+0000 7fbd1d8e2640 1 -- 172.21.15.83:0/2025740537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd1815ed40 con 0x7fbd181641f0 2024-04-17T10:54:12.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.532+0000 7fbd14ff9640 1 -- 172.21.15.83:0/2025740537 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fbd000553c0 con 0x7fbd181641f0 2024-04-17T10:54:12.525 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:12.525 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:54:12.525 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:54:12.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.532+0000 7fbd1d8e2640 1 -- 172.21.15.83:0/2025740537 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbcf8066f90 msgr2=0x7fbcf8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:12.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.532+0000 7fbd1d8e2640 1 --2- 172.21.15.83:0/2025740537 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbcf8066f90 0x7fbcf8069450 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fbd080ad000 tx=0x7fbd080046d0 comp rx=0 tx=0).stop 2024-04-17T10:54:12.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.532+0000 7fbd1d8e2640 1 -- 172.21.15.83:0/2025740537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd181641f0 msgr2=0x7fbd18159e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:12.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.532+0000 7fbd1d8e2640 1 --2- 172.21.15.83:0/2025740537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd181641f0 0x7fbd18159e00 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7fbd0000d7a0 tx=0x7fbd0000dc60 comp rx=0 tx=0).stop 2024-04-17T10:54:12.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.532+0000 7fbd1d8e2640 1 -- 172.21.15.83:0/2025740537 shutdown_connections 2024-04-17T10:54:12.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.532+0000 7fbd1d8e2640 1 --2- 172.21.15.83:0/2025740537 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbcf8066f90 0x7fbcf8069450 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:12.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.532+0000 7fbd1d8e2640 1 --2- 172.21.15.83:0/2025740537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd1815a340 0x7fbd180ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:12.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.532+0000 7fbd1d8e2640 1 --2- 172.21.15.83:0/2025740537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd181641f0 0x7fbd18159e00 unknown :-1 s=CLOSED pgs=2373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:12.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.532+0000 7fbd1d8e2640 1 --2- 172.21.15.83:0/2025740537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd040039c0 0x7fbd18154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:12.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.536+0000 7fbd1d8e2640 1 -- 172.21.15.83:0/2025740537 >> 172.21.15.83:0/2025740537 conn(0x7fbd1806abe0 msgr2=0x7fbd18070470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:12.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.536+0000 7fbd1d8e2640 1 -- 172.21.15.83:0/2025740537 shutdown_connections 2024-04-17T10:54:12.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:12.536+0000 7fbd1d8e2640 1 -- 172.21.15.83:0/2025740537 wait complete. 2024-04-17T10:54:12.945 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:54:12.945 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:54:13.946 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:54:14.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.088+0000 7fe7032ef640 1 Processor -- start 2024-04-17T10:54:14.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.088+0000 7fe7032ef640 1 -- start start 2024-04-17T10:54:14.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.088+0000 7fe7032ef640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe6fc0ce980 0x7fe6fc0b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:14.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.088+0000 7fe7032ef640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe6fc0cee00 0x7fe6fc0b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:14.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.088+0000 7fe7032ef640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6fc162a00 con 0x7fe6fc15dfa0 2024-04-17T10:54:14.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.088+0000 7fe7032ef640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6fc161e80 con 0x7fe6fc0cee00 2024-04-17T10:54:14.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.088+0000 7fe7032ef640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6fc162160 con 0x7fe6fc0ce980 2024-04-17T10:54:14.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.088+0000 7fe70188c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe6fc0cee00 0x7fe6fc0b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:14.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.088+0000 7fe70108b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe6fc15dfa0 0x7fe6fc0b6c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:45422/0 (socket says 172.21.15.83:45422) 2024-04-17T10:54:14.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.088+0000 7fe70108b640 1 -- 172.21.15.83:0/1427040829 learned_addr learned my addr 172.21.15.83:0/1427040829 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:14.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.088+0000 7fe70088a640 1 --2- 172.21.15.83:0/1427040829 >> v2:172.21.15.167:3300/0 conn(0x7fe6fc0ce980 0x7fe6fc0b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:14.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.088+0000 7fe70188c640 1 -- 172.21.15.83:0/1427040829 >> v2:172.21.15.167:3300/0 conn(0x7fe6fc0ce980 msgr2=0x7fe6fc0b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:14.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.088+0000 7fe70188c640 1 --2- 172.21.15.83:0/1427040829 >> v2:172.21.15.167:3300/0 conn(0x7fe6fc0ce980 0x7fe6fc0b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:14.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.088+0000 7fe70188c640 1 -- 172.21.15.83:0/1427040829 >> v1:172.21.15.83:6789/0 conn(0x7fe6fc15dfa0 legacy=0x7fe6fc0b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:14.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.088+0000 7fe70188c640 1 -- 172.21.15.83:0/1427040829 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6fc162440 con 0x7fe6fc0cee00 2024-04-17T10:54:14.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.092+0000 7fe70188c640 1 --2- 172.21.15.83:0/1427040829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6fc0cee00 0x7fe6fc0b5970 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc08c8b0 tx=0x7fe6f401c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de6174c2270ab00 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:14.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.092+0000 7fe6f3fff640 1 -- 172.21.15.83:0/1427040829 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f40b2070 con 0x7fe6fc0cee00 2024-04-17T10:54:14.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.092+0000 7fe6f3fff640 1 -- 172.21.15.83:0/1427040829 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f40a58d0 con 0x7fe6fc0cee00 2024-04-17T10:54:14.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.092+0000 7fe6f3fff640 1 -- 172.21.15.83:0/1427040829 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f40a5ca0 con 0x7fe6fc0cee00 2024-04-17T10:54:14.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.092+0000 7fe7032ef640 1 -- 172.21.15.83:0/1427040829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6fc0cee00 msgr2=0x7fe6fc0b5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:14.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.092+0000 7fe7032ef640 1 --2- 172.21.15.83:0/1427040829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6fc0cee00 0x7fe6fc0b5970 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc08c8b0 tx=0x7fe6f401c870 comp rx=0 tx=0).stop 2024-04-17T10:54:14.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.092+0000 7fe7032ef640 1 -- 172.21.15.83:0/1427040829 shutdown_connections 2024-04-17T10:54:14.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.092+0000 7fe7032ef640 1 --2- 172.21.15.83:0/1427040829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6fc0cee00 0x7fe6fc0b5970 unknown :-1 s=CLOSED pgs=2375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:14.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.092+0000 7fe7032ef640 1 --2- 172.21.15.83:0/1427040829 >> v2:172.21.15.167:3300/0 conn(0x7fe6fc0ce980 0x7fe6fc0b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:14.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.092+0000 7fe7032ef640 1 -- 172.21.15.83:0/1427040829 >> 172.21.15.83:0/1427040829 conn(0x7fe6fc0d0600 msgr2=0x7fe6fc0d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:14.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.092+0000 7fe7032ef640 1 -- 172.21.15.83:0/1427040829 shutdown_connections 2024-04-17T10:54:14.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.092+0000 7fe7032ef640 1 -- 172.21.15.83:0/1427040829 wait complete. 2024-04-17T10:54:14.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.092+0000 7fe7032ef640 1 Processor -- start 2024-04-17T10:54:14.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.092+0000 7fe7032ef640 1 -- start start 2024-04-17T10:54:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.092+0000 7fe7032ef640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6fc0ce980 0x7fe6fc154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.092+0000 7fe7032ef640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6fc0cee00 0x7fe6fc15a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.092+0000 7fe7032ef640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6fc15dfa0 0x7fe6fc15f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.092+0000 7fe7032ef640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc0d26c0 con 0x7fe6fc0ce980 2024-04-17T10:54:14.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.092+0000 7fe7032ef640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc0d2540 con 0x7fe6fc0cee00 2024-04-17T10:54:14.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.092+0000 7fe7032ef640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc0d2840 con 0x7fe6fc15dfa0 2024-04-17T10:54:14.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.092+0000 7fe70108b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6fc0ce980 0x7fe6fc154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:14.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.092+0000 7fe70108b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6fc0ce980 0x7fe6fc154aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59732/0 (socket says 172.21.15.83:59732) 2024-04-17T10:54:14.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.092+0000 7fe70108b640 1 -- 172.21.15.83:0/199232940 learned_addr learned my addr 172.21.15.83:0/199232940 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:14.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.092+0000 7fe70188c640 1 --2- 172.21.15.83:0/199232940 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6fc15dfa0 0x7fe6fc15f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:14.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.092+0000 7fe70108b640 1 -- 172.21.15.83:0/199232940 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6fc15dfa0 msgr2=0x7fe6fc15f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:14.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.092+0000 7fe70108b640 1 --2- 172.21.15.83:0/199232940 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6fc15dfa0 0x7fe6fc15f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:14.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.092+0000 7fe70108b640 1 -- 172.21.15.83:0/199232940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6fc0cee00 msgr2=0x7fe6fc15a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:14.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.092+0000 7fe70108b640 1 --2- 172.21.15.83:0/199232940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6fc0cee00 0x7fe6fc15a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:14.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.092+0000 7fe70108b640 1 -- 172.21.15.83:0/199232940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6fc157680 con 0x7fe6fc0ce980 2024-04-17T10:54:14.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.096+0000 7fe70108b640 1 --2- 172.21.15.83:0/199232940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6fc0ce980 0x7fe6fc154aa0 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec00c3a0 tx=0x7fe6ec00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:14.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.096+0000 7fe6f1ffb640 1 -- 172.21.15.83:0/199232940 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec017070 con 0x7fe6fc0ce980 2024-04-17T10:54:14.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.096+0000 7fe6f1ffb640 1 -- 172.21.15.83:0/199232940 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6ec005cf0 con 0x7fe6fc0ce980 2024-04-17T10:54:14.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.096+0000 7fe7032ef640 1 -- 172.21.15.83:0/199232940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6fc15fe40 con 0x7fe6fc0ce980 2024-04-17T10:54:14.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.096+0000 7fe6f1ffb640 1 -- 172.21.15.83:0/199232940 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec0072f0 con 0x7fe6fc0ce980 2024-04-17T10:54:14.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.096+0000 7fe7032ef640 1 -- 172.21.15.83:0/199232940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6fc0cc4d0 con 0x7fe6fc0ce980 2024-04-17T10:54:14.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.096+0000 7fe7032ef640 1 -- 172.21.15.83:0/199232940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6fc074f90 con 0x7fe6fc0ce980 2024-04-17T10:54:14.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.096+0000 7fe6f1ffb640 1 -- 172.21.15.83:0/199232940 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe6ec01a070 con 0x7fe6fc0ce980 2024-04-17T10:54:14.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.096+0000 7fe6f1ffb640 1 --2- 172.21.15.83:0/199232940 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe6e0066f90 0x7fe6e0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:14.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.096+0000 7fe6f1ffb640 1 -- 172.21.15.83:0/199232940 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fe6ec084440 con 0x7fe6fc0ce980 2024-04-17T10:54:14.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.096+0000 7fe70088a640 1 --2- 172.21.15.83:0/199232940 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe6e0066f90 0x7fe6e0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:14.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.096+0000 7fe70088a640 1 --2- 172.21.15.83:0/199232940 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe6e0066f90 0x7fe6e0069450 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc157dd0 tx=0x7fe6f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:14.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.100+0000 7fe6f1ffb640 1 -- 172.21.15.83:0/199232940 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe6fc157680 con 0x7fe6fc0ce980 2024-04-17T10:54:14.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.284+0000 7fe7032ef640 1 -- 172.21.15.83:0/199232940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6fc0b5c90 con 0x7fe6fc0ce980 2024-04-17T10:54:14.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.284+0000 7fe6f1ffb640 1 -- 172.21.15.83:0/199232940 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fe6ec0553e0 con 0x7fe6fc0ce980 2024-04-17T10:54:14.279 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:14.279 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:54:14.279 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:54:14.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.288+0000 7fe7032ef640 1 -- 172.21.15.83:0/199232940 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe6e0066f90 msgr2=0x7fe6e0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:14.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.288+0000 7fe7032ef640 1 --2- 172.21.15.83:0/199232940 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe6e0066f90 0x7fe6e0069450 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc157dd0 tx=0x7fe6f8008040 comp rx=0 tx=0).stop 2024-04-17T10:54:14.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.288+0000 7fe7032ef640 1 -- 172.21.15.83:0/199232940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6fc0ce980 msgr2=0x7fe6fc154aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:14.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.288+0000 7fe7032ef640 1 --2- 172.21.15.83:0/199232940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6fc0ce980 0x7fe6fc154aa0 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec00c3a0 tx=0x7fe6ec00c860 comp rx=0 tx=0).stop 2024-04-17T10:54:14.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.288+0000 7fe7032ef640 1 -- 172.21.15.83:0/199232940 shutdown_connections 2024-04-17T10:54:14.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.288+0000 7fe7032ef640 1 --2- 172.21.15.83:0/199232940 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe6e0066f90 0x7fe6e0069450 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:14.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.288+0000 7fe7032ef640 1 --2- 172.21.15.83:0/199232940 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6fc15dfa0 0x7fe6fc15f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:14.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.288+0000 7fe7032ef640 1 --2- 172.21.15.83:0/199232940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6fc0cee00 0x7fe6fc15a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:14.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.288+0000 7fe7032ef640 1 --2- 172.21.15.83:0/199232940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6fc0ce980 0x7fe6fc154aa0 unknown :-1 s=CLOSED pgs=2376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:14.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.288+0000 7fe7032ef640 1 -- 172.21.15.83:0/199232940 >> 172.21.15.83:0/199232940 conn(0x7fe6fc0d0600 msgr2=0x7fe6fc15c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:14.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.288+0000 7fe7032ef640 1 -- 172.21.15.83:0/199232940 shutdown_connections 2024-04-17T10:54:14.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:14.288+0000 7fe7032ef640 1 -- 172.21.15.83:0/199232940 wait complete. 2024-04-17T10:54:14.699 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:54:14.700 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:54:15.701 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:54:15.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.840+0000 7fe55a66e640 1 Processor -- start 2024-04-17T10:54:15.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.840+0000 7fe55a66e640 1 -- start start 2024-04-17T10:54:15.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe55a66e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe55415ba30 0x7fe554072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:15.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe55a66e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe55415c960 0x7fe5540cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:15.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe55a66e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5540cfea0 con 0x7fe5540cfa10 2024-04-17T10:54:15.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe55a66e640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe554071ea0 con 0x7fe55415ba30 2024-04-17T10:54:15.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe55a66e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe55406e320 con 0x7fe55415c960 2024-04-17T10:54:15.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe558c0b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe5540cfa10 0x7fe55406c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:45432/0 (socket says 172.21.15.83:45432) 2024-04-17T10:54:15.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe558c0b640 1 -- 172.21.15.83:0/382205148 learned_addr learned my addr 172.21.15.83:0/382205148 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:15.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe553fff640 1 --2- 172.21.15.83:0/382205148 >> v2:172.21.15.167:3300/0 conn(0x7fe55415ba30 0x7fe554072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:15.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe5537fe640 1 --2- 172.21.15.83:0/382205148 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe55415c960 0x7fe5540cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:15.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe552ffd640 1 -- 172.21.15.83:0/382205148 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3334377155 0 0) 0x7fe5540cfea0 con 0x7fe5540cfa10 2024-04-17T10:54:15.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe552ffd640 1 -- 172.21.15.83:0/382205148 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe53c003530 con 0x7fe5540cfa10 2024-04-17T10:54:15.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe552ffd640 1 -- 172.21.15.83:0/382205148 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3796341284 0 0) 0x7fe53c003530 con 0x7fe5540cfa10 2024-04-17T10:54:15.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe552ffd640 1 -- 172.21.15.83:0/382205148 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe55415c960 msgr2=0x7fe5540cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:15.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe552ffd640 1 --2- 172.21.15.83:0/382205148 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe55415c960 0x7fe5540cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:15.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe552ffd640 1 -- 172.21.15.83:0/382205148 >> v2:172.21.15.167:3300/0 conn(0x7fe55415ba30 msgr2=0x7fe554072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:15.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe552ffd640 1 --2- 172.21.15.83:0/382205148 >> v2:172.21.15.167:3300/0 conn(0x7fe55415ba30 0x7fe554072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:15.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe552ffd640 1 -- 172.21.15.83:0/382205148 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55406e040 con 0x7fe5540cfa10 2024-04-17T10:54:15.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe552ffd640 1 -- 172.21.15.83:0/382205148 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fe54c001a00 con 0x7fe5540cfa10 2024-04-17T10:54:15.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe552ffd640 1 -- 172.21.15.83:0/382205148 >> v1:172.21.15.83:6789/0 conn(0x7fe5540cfa10 legacy=0x7fe55406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:15.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe552ffd640 1 --2- 172.21.15.83:0/382205148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe53c0039c0 0x7fe53c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:15.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe552ffd640 1 --2- 172.21.15.83:0/382205148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe53c004810 0x7fe53c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:15.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe552ffd640 1 --2- 172.21.15.83:0/382205148 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe55415c960 0x7fe53c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:15.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe552ffd640 1 -- 172.21.15.83:0/382205148 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe554071ea0 con 0x7fe53c004810 2024-04-17T10:54:15.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe552ffd640 1 -- 172.21.15.83:0/382205148 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55406e320 con 0x7fe53c0039c0 2024-04-17T10:54:15.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe552ffd640 1 -- 172.21.15.83:0/382205148 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c00b9d0 con 0x7fe55415c960 2024-04-17T10:54:15.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe5537fe640 1 --2- 172.21.15.83:0/382205148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe53c004810 0x7fe53c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:15.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe553fff640 1 --2- 172.21.15.83:0/382205148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe53c0039c0 0x7fe53c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:15.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe558c0b640 1 --2- 172.21.15.83:0/382205148 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe55415c960 0x7fe53c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:15.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe5537fe640 1 -- 172.21.15.83:0/382205148 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe55415c960 msgr2=0x7fe53c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:15.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe5537fe640 1 --2- 172.21.15.83:0/382205148 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe55415c960 0x7fe53c0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:15.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe5537fe640 1 -- 172.21.15.83:0/382205148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe53c0039c0 msgr2=0x7fe53c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:15.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe5537fe640 1 --2- 172.21.15.83:0/382205148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe53c0039c0 0x7fe53c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:15.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe5537fe640 1 -- 172.21.15.83:0/382205148 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe53c003530 con 0x7fe53c004810 2024-04-17T10:54:15.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.844+0000 7fe553fff640 1 --2- 172.21.15.83:0/382205148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe53c0039c0 0x7fe53c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:15.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.848+0000 7fe5537fe640 1 --2- 172.21.15.83:0/382205148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe53c004810 0x7fe53c004c90 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7fe548009b10 tx=0x7fe55406d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3585061e8abd56c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:15.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.848+0000 7fe552ffd640 1 -- 172.21.15.83:0/382205148 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5480096b0 con 0x7fe53c004810 2024-04-17T10:54:15.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.848+0000 7fe552ffd640 1 -- 172.21.15.83:0/382205148 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5480a9470 con 0x7fe53c004810 2024-04-17T10:54:15.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.848+0000 7fe552ffd640 1 -- 172.21.15.83:0/382205148 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5480a9860 con 0x7fe53c004810 2024-04-17T10:54:15.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.848+0000 7fe55a66e640 1 -- 172.21.15.83:0/382205148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe53c004810 msgr2=0x7fe53c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:15.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.848+0000 7fe55a66e640 1 --2- 172.21.15.83:0/382205148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe53c004810 0x7fe53c004c90 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7fe548009b10 tx=0x7fe55406d3b0 comp rx=0 tx=0).stop 2024-04-17T10:54:15.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.848+0000 7fe558c0b640 1 -- 172.21.15.83:0/382205148 reap_dead start 2024-04-17T10:54:15.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.848+0000 7fe55a66e640 1 -- 172.21.15.83:0/382205148 shutdown_connections 2024-04-17T10:54:15.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.848+0000 7fe55a66e640 1 -- 172.21.15.83:0/382205148 >> 172.21.15.83:0/382205148 conn(0x7fe55406abe0 msgr2=0x7fe55406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:15.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.848+0000 7fe55a66e640 1 -- 172.21.15.83:0/382205148 shutdown_connections 2024-04-17T10:54:15.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.848+0000 7fe55a66e640 1 -- 172.21.15.83:0/382205148 wait complete. 2024-04-17T10:54:15.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.848+0000 7fe55a66e640 1 Processor -- start 2024-04-17T10:54:15.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.848+0000 7fe55a66e640 1 -- start start 2024-04-17T10:54:15.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.848+0000 7fe55a66e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe53c004810 0x7fe554154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:15.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.848+0000 7fe55a66e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe554158ae0 0x7fe5540ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:15.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.848+0000 7fe55a66e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe554158f60 0x7fe5540add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:15.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.848+0000 7fe55a66e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55406de20 con 0x7fe554158ae0 2024-04-17T10:54:15.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.848+0000 7fe55a66e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55406dca0 con 0x7fe53c004810 2024-04-17T10:54:15.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.848+0000 7fe55a66e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe55415cd40 con 0x7fe554158f60 2024-04-17T10:54:15.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.848+0000 7fe5537fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe554158ae0 0x7fe5540ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:15.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.848+0000 7fe5537fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe554158ae0 0x7fe5540ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59754/0 (socket says 172.21.15.83:59754) 2024-04-17T10:54:15.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.848+0000 7fe5537fe640 1 -- 172.21.15.83:0/474838161 learned_addr learned my addr 172.21.15.83:0/474838161 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:15.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.848+0000 7fe553fff640 1 --2- 172.21.15.83:0/474838161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe53c004810 0x7fe554154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:15.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.848+0000 7fe558c0b640 1 --2- 172.21.15.83:0/474838161 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe554158f60 0x7fe5540add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:15.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.848+0000 7fe5537fe640 1 -- 172.21.15.83:0/474838161 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe554158f60 msgr2=0x7fe5540add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:15.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.848+0000 7fe5537fe640 1 --2- 172.21.15.83:0/474838161 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe554158f60 0x7fe5540add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:15.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.848+0000 7fe5537fe640 1 -- 172.21.15.83:0/474838161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe53c004810 msgr2=0x7fe554154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:15.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.848+0000 7fe5537fe640 1 --2- 172.21.15.83:0/474838161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe53c004810 0x7fe554154850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:15.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.848+0000 7fe5537fe640 1 -- 172.21.15.83:0/474838161 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5541f6f00 con 0x7fe554158ae0 2024-04-17T10:54:15.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.848+0000 7fe558c0b640 1 --2- 172.21.15.83:0/474838161 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe554158f60 0x7fe5540add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:15.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.848+0000 7fe5537fe640 1 --2- 172.21.15.83:0/474838161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe554158ae0 0x7fe5540ad840 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7fe54801c8a0 tx=0x7fe53c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:15.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.852+0000 7fe550ff9640 1 -- 172.21.15.83:0/474838161 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54800f000 con 0x7fe554158ae0 2024-04-17T10:54:15.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.852+0000 7fe550ff9640 1 -- 172.21.15.83:0/474838161 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5480a9c80 con 0x7fe554158ae0 2024-04-17T10:54:15.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.852+0000 7fe55a66e640 1 -- 172.21.15.83:0/474838161 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5541f71e0 con 0x7fe554158ae0 2024-04-17T10:54:15.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.852+0000 7fe550ff9640 1 -- 172.21.15.83:0/474838161 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe548004260 con 0x7fe554158ae0 2024-04-17T10:54:15.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.852+0000 7fe55a66e640 1 -- 172.21.15.83:0/474838161 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5541f77a0 con 0x7fe554158ae0 2024-04-17T10:54:15.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.852+0000 7fe55a66e640 1 -- 172.21.15.83:0/474838161 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe554074f90 con 0x7fe554158ae0 2024-04-17T10:54:15.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.852+0000 7fe550ff9640 1 -- 172.21.15.83:0/474838161 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe5480c3060 con 0x7fe554158ae0 2024-04-17T10:54:15.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.852+0000 7fe550ff9640 1 --2- 172.21.15.83:0/474838161 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe540066f90 0x7fe540069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:15.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.852+0000 7fe550ff9640 1 -- 172.21.15.83:0/474838161 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fe54811f0a0 con 0x7fe554158ae0 2024-04-17T10:54:15.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.852+0000 7fe553fff640 1 --2- 172.21.15.83:0/474838161 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe540066f90 0x7fe540069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:15.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.852+0000 7fe553fff640 1 --2- 172.21.15.83:0/474838161 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe540066f90 0x7fe540069450 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fe5540b4950 tx=0x7fe544008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:15.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:15.856+0000 7fe550ff9640 1 -- 172.21.15.83:0/474838161 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe5480ad020 con 0x7fe554158ae0 2024-04-17T10:54:16.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:16.044+0000 7fe55a66e640 1 -- 172.21.15.83:0/474838161 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe554154d90 con 0x7fe554158ae0 2024-04-17T10:54:16.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:16.044+0000 7fe550ff9640 1 -- 172.21.15.83:0/474838161 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fe5480167c0 con 0x7fe554158ae0 2024-04-17T10:54:16.037 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:16.037 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:54:16.037 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:54:16.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:16.044+0000 7fe55a66e640 1 -- 172.21.15.83:0/474838161 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe540066f90 msgr2=0x7fe540069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:16.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:16.044+0000 7fe55a66e640 1 --2- 172.21.15.83:0/474838161 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe540066f90 0x7fe540069450 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fe5540b4950 tx=0x7fe544008040 comp rx=0 tx=0).stop 2024-04-17T10:54:16.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:16.044+0000 7fe55a66e640 1 -- 172.21.15.83:0/474838161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe554158ae0 msgr2=0x7fe5540ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:16.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:16.044+0000 7fe55a66e640 1 --2- 172.21.15.83:0/474838161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe554158ae0 0x7fe5540ad840 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7fe54801c8a0 tx=0x7fe53c005840 comp rx=0 tx=0).stop 2024-04-17T10:54:16.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:16.044+0000 7fe55a66e640 1 -- 172.21.15.83:0/474838161 shutdown_connections 2024-04-17T10:54:16.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:16.044+0000 7fe55a66e640 1 --2- 172.21.15.83:0/474838161 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe540066f90 0x7fe540069450 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:16.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:16.044+0000 7fe55a66e640 1 --2- 172.21.15.83:0/474838161 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe554158f60 0x7fe5540add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:16.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:16.044+0000 7fe55a66e640 1 --2- 172.21.15.83:0/474838161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe554158ae0 0x7fe5540ad840 unknown :-1 s=CLOSED pgs=2379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:16.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:16.044+0000 7fe55a66e640 1 --2- 172.21.15.83:0/474838161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe53c004810 0x7fe554154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:16.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:16.044+0000 7fe55a66e640 1 -- 172.21.15.83:0/474838161 >> 172.21.15.83:0/474838161 conn(0x7fe55406abe0 msgr2=0x7fe5540b0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:16.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:16.048+0000 7fe55a66e640 1 -- 172.21.15.83:0/474838161 shutdown_connections 2024-04-17T10:54:16.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:16.048+0000 7fe55a66e640 1 -- 172.21.15.83:0/474838161 wait complete. 2024-04-17T10:54:16.457 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:54:16.457 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:54:17.458 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:54:17.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.600+0000 7f9f283b3640 1 Processor -- start 2024-04-17T10:54:17.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.600+0000 7f9f283b3640 1 -- start start 2024-04-17T10:54:17.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.600+0000 7f9f283b3640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9f20072330 0x7f9f200cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:17.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.600+0000 7f9f283b3640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f200cfea0 con 0x7f9f200cfa10 2024-04-17T10:54:17.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.600+0000 7f9f283b3640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f2006e320 con 0x7f9f2015ba30 2024-04-17T10:54:17.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.600+0000 7f9f283b3640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f20071ea0 con 0x7f9f20072330 2024-04-17T10:54:17.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.600+0000 7f9f2594e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9f20072330 0x7f9f200cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:17.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.600+0000 7f9f26950640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9f200cfa10 0x7f9f2006c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38572/0 (socket says 172.21.15.83:38572) 2024-04-17T10:54:17.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.600+0000 7f9f2594e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9f20072330 0x7f9f200cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50932/0 (socket says 172.21.15.83:50932) 2024-04-17T10:54:17.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.600+0000 7f9f26950640 1 -- 172.21.15.83:0/857301753 learned_addr learned my addr 172.21.15.83:0/857301753 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:17.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.604+0000 7f9f2594e640 1 -- 172.21.15.83:0/857301753 >> v1:172.21.15.167:6789/0 conn(0x7f9f2015ba30 legacy=0x7f9f2015c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:17.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.604+0000 7f9f2594e640 1 -- 172.21.15.83:0/857301753 >> v1:172.21.15.83:6789/0 conn(0x7f9f200cfa10 legacy=0x7f9f2006c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:17.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.604+0000 7f9f2594e640 1 -- 172.21.15.83:0/857301753 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2006e040 con 0x7f9f20072330 2024-04-17T10:54:17.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.604+0000 7f9f2514d640 1 -- 172.21.15.83:0/857301753 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3978310626 0 0) 0x7f9f200cfea0 con 0x7f9f200cfa10 2024-04-17T10:54:17.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.604+0000 7f9f2594e640 1 --2- 172.21.15.83:0/857301753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f20072330 0x7f9f200cf4d0 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f9f100097c0 tx=0x7f9f1001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce08c65e8c0f16a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:17.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.604+0000 7f9f2514d640 1 -- 172.21.15.83:0/857301753 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f100b0070 con 0x7f9f20072330 2024-04-17T10:54:17.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.604+0000 7f9f2514d640 1 -- 172.21.15.83:0/857301753 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f100ab600 con 0x7f9f20072330 2024-04-17T10:54:17.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.604+0000 7f9f2514d640 1 -- 172.21.15.83:0/857301753 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f100ab9d0 con 0x7f9f20072330 2024-04-17T10:54:17.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.604+0000 7f9f283b3640 1 -- 172.21.15.83:0/857301753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f20072330 msgr2=0x7f9f200cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:17.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.604+0000 7f9f283b3640 1 --2- 172.21.15.83:0/857301753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f20072330 0x7f9f200cf4d0 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f9f100097c0 tx=0x7f9f1001c3d0 comp rx=0 tx=0).stop 2024-04-17T10:54:17.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.604+0000 7f9f283b3640 1 -- 172.21.15.83:0/857301753 shutdown_connections 2024-04-17T10:54:17.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.604+0000 7f9f283b3640 1 --2- 172.21.15.83:0/857301753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f20072330 0x7f9f200cf4d0 unknown :-1 s=CLOSED pgs=2381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:17.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.604+0000 7f9f283b3640 1 -- 172.21.15.83:0/857301753 >> 172.21.15.83:0/857301753 conn(0x7f9f2006abe0 msgr2=0x7f9f2006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:17.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.604+0000 7f9f283b3640 1 -- 172.21.15.83:0/857301753 shutdown_connections 2024-04-17T10:54:17.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.604+0000 7f9f283b3640 1 -- 172.21.15.83:0/857301753 wait complete. 2024-04-17T10:54:17.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.608+0000 7f9f283b3640 1 Processor -- start 2024-04-17T10:54:17.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.608+0000 7f9f283b3640 1 -- start start 2024-04-17T10:54:17.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.608+0000 7f9f283b3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f20072330 0x7f9f20146cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:17.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.608+0000 7f9f283b3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f200cfa10 0x7f9f2014dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:17.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.608+0000 7f9f283b3640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f2015ba30 0x7f9f2014e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:17.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.608+0000 7f9f283b3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2006de20 con 0x7f9f200cfa10 2024-04-17T10:54:17.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.608+0000 7f9f283b3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2006dca0 con 0x7f9f20072330 2024-04-17T10:54:17.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.608+0000 7f9f283b3640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f20169530 con 0x7f9f2015ba30 2024-04-17T10:54:17.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.608+0000 7f9f2594e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f200cfa10 0x7f9f2014dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:17.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.608+0000 7f9f2594e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f200cfa10 0x7f9f2014dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50948/0 (socket says 172.21.15.83:50948) 2024-04-17T10:54:17.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.608+0000 7f9f2594e640 1 -- 172.21.15.83:0/4243681406 learned_addr learned my addr 172.21.15.83:0/4243681406 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:17.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.608+0000 7f9f26950640 1 --2- 172.21.15.83:0/4243681406 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f2015ba30 0x7f9f2014e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:17.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.608+0000 7f9f2614f640 1 --2- 172.21.15.83:0/4243681406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f20072330 0x7f9f20146cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:17.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.608+0000 7f9f2594e640 1 -- 172.21.15.83:0/4243681406 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f2015ba30 msgr2=0x7f9f2014e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:17.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.608+0000 7f9f2594e640 1 --2- 172.21.15.83:0/4243681406 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f2015ba30 0x7f9f2014e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:17.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.608+0000 7f9f2594e640 1 -- 172.21.15.83:0/4243681406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f20072330 msgr2=0x7f9f20146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:17.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.608+0000 7f9f2594e640 1 --2- 172.21.15.83:0/4243681406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f20072330 0x7f9f20146cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:17.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.608+0000 7f9f2594e640 1 -- 172.21.15.83:0/4243681406 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2015d340 con 0x7f9f200cfa10 2024-04-17T10:54:17.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.608+0000 7f9f26950640 1 --2- 172.21.15.83:0/4243681406 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f2015ba30 0x7f9f2014e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:17.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.608+0000 7f9f2594e640 1 --2- 172.21.15.83:0/4243681406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f200cfa10 0x7f9f2014dbf0 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7f9f100097c0 tx=0x7f9f10019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:17.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.608+0000 7f9f16ffd640 1 -- 172.21.15.83:0/4243681406 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f100b0040 con 0x7f9f200cfa10 2024-04-17T10:54:17.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.608+0000 7f9f16ffd640 1 -- 172.21.15.83:0/4243681406 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f10004030 con 0x7f9f200cfa10 2024-04-17T10:54:17.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.608+0000 7f9f283b3640 1 -- 172.21.15.83:0/4243681406 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f2015d620 con 0x7f9f200cfa10 2024-04-17T10:54:17.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.608+0000 7f9f16ffd640 1 -- 172.21.15.83:0/4243681406 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f10004400 con 0x7f9f200cfa10 2024-04-17T10:54:17.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.608+0000 7f9f283b3640 1 -- 172.21.15.83:0/4243681406 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f2015dbe0 con 0x7f9f200cfa10 2024-04-17T10:54:17.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.608+0000 7f9f283b3640 1 -- 172.21.15.83:0/4243681406 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f20074f90 con 0x7f9f200cfa10 2024-04-17T10:54:17.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.608+0000 7f9f16ffd640 1 -- 172.21.15.83:0/4243681406 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9f10003620 con 0x7f9f200cfa10 2024-04-17T10:54:17.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.608+0000 7f9f16ffd640 1 --2- 172.21.15.83:0/4243681406 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9f04066f60 0x7f9f04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:17.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.608+0000 7f9f16ffd640 1 -- 172.21.15.83:0/4243681406 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f9f1011f680 con 0x7f9f200cfa10 2024-04-17T10:54:17.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.608+0000 7f9f2614f640 1 --2- 172.21.15.83:0/4243681406 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9f04066f60 0x7f9f04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:17.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.612+0000 7f9f2614f640 1 --2- 172.21.15.83:0/4243681406 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9f04066f60 0x7f9f04069420 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00a750 tx=0x7f9f1c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:17.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.616+0000 7f9f16ffd640 1 -- 172.21.15.83:0/4243681406 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9f10016050 con 0x7f9f200cfa10 2024-04-17T10:54:17.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.804+0000 7f9f283b3640 1 -- 172.21.15.83:0/4243681406 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f2006b060 con 0x7f9f200cfa10 2024-04-17T10:54:17.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.804+0000 7f9f16ffd640 1 -- 172.21.15.83:0/4243681406 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f9f100f0620 con 0x7f9f200cfa10 2024-04-17T10:54:17.798 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:17.798 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:54:17.798 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:54:17.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.808+0000 7f9f283b3640 1 -- 172.21.15.83:0/4243681406 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9f04066f60 msgr2=0x7f9f04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:17.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.808+0000 7f9f283b3640 1 --2- 172.21.15.83:0/4243681406 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9f04066f60 0x7f9f04069420 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00a750 tx=0x7f9f1c008040 comp rx=0 tx=0).stop 2024-04-17T10:54:17.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.808+0000 7f9f283b3640 1 -- 172.21.15.83:0/4243681406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f200cfa10 msgr2=0x7f9f2014dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:17.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.808+0000 7f9f283b3640 1 --2- 172.21.15.83:0/4243681406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f200cfa10 0x7f9f2014dbf0 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7f9f100097c0 tx=0x7f9f10019170 comp rx=0 tx=0).stop 2024-04-17T10:54:17.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.808+0000 7f9f283b3640 1 -- 172.21.15.83:0/4243681406 shutdown_connections 2024-04-17T10:54:17.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.808+0000 7f9f283b3640 1 --2- 172.21.15.83:0/4243681406 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9f04066f60 0x7f9f04069420 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:17.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.808+0000 7f9f283b3640 1 --2- 172.21.15.83:0/4243681406 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f2015ba30 0x7f9f2014e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:17.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.808+0000 7f9f283b3640 1 --2- 172.21.15.83:0/4243681406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f200cfa10 0x7f9f2014dbf0 unknown :-1 s=CLOSED pgs=2382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:17.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.808+0000 7f9f283b3640 1 --2- 172.21.15.83:0/4243681406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f20072330 0x7f9f20146cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:17.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.808+0000 7f9f283b3640 1 -- 172.21.15.83:0/4243681406 >> 172.21.15.83:0/4243681406 conn(0x7f9f2006abe0 msgr2=0x7f9f20159bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:17.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.808+0000 7f9f283b3640 1 -- 172.21.15.83:0/4243681406 shutdown_connections 2024-04-17T10:54:17.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:17.808+0000 7f9f283b3640 1 -- 172.21.15.83:0/4243681406 wait complete. 2024-04-17T10:54:18.217 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:54:18.218 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:54:19.218 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:54:19.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.356+0000 7f13072af640 1 Processor -- start 2024-04-17T10:54:19.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.356+0000 7f13072af640 1 -- start start 2024-04-17T10:54:19.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.356+0000 7f13072af640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f130015ba30 0x7f1300072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:19.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.356+0000 7f13072af640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13000cfbe0 0x7f130006c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:19.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.356+0000 7f13072af640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f130006e320 con 0x7f130015c960 2024-04-17T10:54:19.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.356+0000 7f13072af640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1300071ea0 con 0x7f130015ba30 2024-04-17T10:54:19.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.356+0000 7f13072af640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f130006e040 con 0x7f13000cfbe0 2024-04-17T10:54:19.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.356+0000 7f130504b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f130015ba30 0x7f1300072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:19.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.356+0000 7f130504b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f130015ba30 0x7f1300072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50952/0 (socket says 172.21.15.83:50952) 2024-04-17T10:54:19.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.356+0000 7f130504b640 1 -- 172.21.15.83:0/617959810 learned_addr learned my addr 172.21.15.83:0/617959810 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:19.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.356+0000 7f130584c640 1 --2- 172.21.15.83:0/617959810 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13000cfbe0 0x7f130006c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:19.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.356+0000 7f130504b640 1 -- 172.21.15.83:0/617959810 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13000cfbe0 msgr2=0x7f130006c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:19.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.356+0000 7f130504b640 1 --2- 172.21.15.83:0/617959810 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13000cfbe0 0x7f130006c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:19.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.356+0000 7f130504b640 1 -- 172.21.15.83:0/617959810 >> v1:172.21.15.167:6789/0 conn(0x7f130015c960 legacy=0x7f13000cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:19.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.356+0000 7f130504b640 1 -- 172.21.15.83:0/617959810 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1300071850 con 0x7f130015ba30 2024-04-17T10:54:19.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.360+0000 7f130504b640 1 --2- 172.21.15.83:0/617959810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f130015ba30 0x7f1300072330 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7f12f0009870 tx=0x7f12f001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b00f5dbd416bd310 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:19.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.360+0000 7f12f7fff640 1 -- 172.21.15.83:0/617959810 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f00b0070 con 0x7f130015ba30 2024-04-17T10:54:19.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.360+0000 7f12f7fff640 1 -- 172.21.15.83:0/617959810 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f00ab600 con 0x7f130015ba30 2024-04-17T10:54:19.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.360+0000 7f12f7fff640 1 -- 172.21.15.83:0/617959810 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f00ab9d0 con 0x7f130015ba30 2024-04-17T10:54:19.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.360+0000 7f13072af640 1 -- 172.21.15.83:0/617959810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f130015ba30 msgr2=0x7f1300072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:19.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.360+0000 7f13072af640 1 --2- 172.21.15.83:0/617959810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f130015ba30 0x7f1300072330 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7f12f0009870 tx=0x7f12f001c4d0 comp rx=0 tx=0).stop 2024-04-17T10:54:19.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.360+0000 7f13072af640 1 -- 172.21.15.83:0/617959810 shutdown_connections 2024-04-17T10:54:19.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.360+0000 7f13072af640 1 --2- 172.21.15.83:0/617959810 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13000cfbe0 0x7f130006c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:19.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.360+0000 7f13072af640 1 --2- 172.21.15.83:0/617959810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f130015ba30 0x7f1300072330 unknown :-1 s=CLOSED pgs=2383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:19.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.360+0000 7f13072af640 1 -- 172.21.15.83:0/617959810 >> 172.21.15.83:0/617959810 conn(0x7f130006abe0 msgr2=0x7f130006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:19.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.360+0000 7f13072af640 1 -- 172.21.15.83:0/617959810 shutdown_connections 2024-04-17T10:54:19.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.360+0000 7f13072af640 1 -- 172.21.15.83:0/617959810 wait complete. 2024-04-17T10:54:19.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.360+0000 7f13072af640 1 Processor -- start 2024-04-17T10:54:19.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.360+0000 7f13072af640 1 -- start start 2024-04-17T10:54:19.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.364+0000 7f13072af640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13000cfbe0 0x7f1300151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:19.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.364+0000 7f13072af640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f130015ba30 0x7f130013b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:19.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.364+0000 7f13072af640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f130015c960 0x7f130014fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:19.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.364+0000 7f13072af640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130015cee0 con 0x7f130015c960 2024-04-17T10:54:19.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.364+0000 7f13072af640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130015cd60 con 0x7f13000cfbe0 2024-04-17T10:54:19.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.364+0000 7f13072af640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1300168520 con 0x7f130015ba30 2024-04-17T10:54:19.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.364+0000 7f130584c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f130015c960 0x7f130014fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:19.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.364+0000 7f130584c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f130015c960 0x7f130014fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50956/0 (socket says 172.21.15.83:50956) 2024-04-17T10:54:19.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.364+0000 7f130584c640 1 -- 172.21.15.83:0/2205407678 learned_addr learned my addr 172.21.15.83:0/2205407678 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:19.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.364+0000 7f130484a640 1 --2- 172.21.15.83:0/2205407678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f130015ba30 0x7f130013b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:19.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.364+0000 7f130504b640 1 --2- 172.21.15.83:0/2205407678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13000cfbe0 0x7f1300151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:19.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.364+0000 7f130584c640 1 -- 172.21.15.83:0/2205407678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f130015ba30 msgr2=0x7f130013b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:19.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.364+0000 7f130584c640 1 --2- 172.21.15.83:0/2205407678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f130015ba30 0x7f130013b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:19.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.364+0000 7f130584c640 1 -- 172.21.15.83:0/2205407678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13000cfbe0 msgr2=0x7f1300151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:19.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.364+0000 7f130584c640 1 --2- 172.21.15.83:0/2205407678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13000cfbe0 0x7f1300151820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:19.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.364+0000 7f130584c640 1 -- 172.21.15.83:0/2205407678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f130013b720 con 0x7f130015c960 2024-04-17T10:54:19.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.364+0000 7f130584c640 1 --2- 172.21.15.83:0/2205407678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f130015c960 0x7f130014fcc0 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f12f8007e60 tx=0x7f12f800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:19.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.364+0000 7f12f5ffb640 1 -- 172.21.15.83:0/2205407678 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f8014420 con 0x7f130015c960 2024-04-17T10:54:19.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.364+0000 7f12f5ffb640 1 -- 172.21.15.83:0/2205407678 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f8019070 con 0x7f130015c960 2024-04-17T10:54:19.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.364+0000 7f13072af640 1 -- 172.21.15.83:0/2205407678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1300150780 con 0x7f130015c960 2024-04-17T10:54:19.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.364+0000 7f12f5ffb640 1 -- 172.21.15.83:0/2205407678 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f8014820 con 0x7f130015c960 2024-04-17T10:54:19.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.364+0000 7f13072af640 1 -- 172.21.15.83:0/2205407678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1300150d40 con 0x7f130015c960 2024-04-17T10:54:19.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.364+0000 7f13072af640 1 -- 172.21.15.83:0/2205407678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1300074f90 con 0x7f130015c960 2024-04-17T10:54:19.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.364+0000 7f12f5ffb640 1 -- 172.21.15.83:0/2205407678 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f12f80149c0 con 0x7f130015c960 2024-04-17T10:54:19.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.364+0000 7f12f5ffb640 1 --2- 172.21.15.83:0/2205407678 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f12e4066f60 0x7f12e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:19.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.364+0000 7f12f5ffb640 1 -- 172.21.15.83:0/2205407678 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f12f80857d0 con 0x7f130015c960 2024-04-17T10:54:19.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.364+0000 7f130504b640 1 --2- 172.21.15.83:0/2205407678 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f12e4066f60 0x7f12e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:19.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.368+0000 7f130504b640 1 --2- 172.21.15.83:0/2205407678 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f12e4066f60 0x7f12e4069420 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f12f00ad000 tx=0x7f12f00a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:19.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.372+0000 7f12f5ffb640 1 -- 172.21.15.83:0/2205407678 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f12f8056800 con 0x7f130015c960 2024-04-17T10:54:19.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.560+0000 7f13072af640 1 -- 172.21.15.83:0/2205407678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f130006d840 con 0x7f130015c960 2024-04-17T10:54:19.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.560+0000 7f12f5ffb640 1 -- 172.21.15.83:0/2205407678 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f12f8056100 con 0x7f130015c960 2024-04-17T10:54:19.553 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:19.554 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:54:19.554 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:54:19.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.564+0000 7f13072af640 1 -- 172.21.15.83:0/2205407678 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f12e4066f60 msgr2=0x7f12e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:19.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.564+0000 7f13072af640 1 --2- 172.21.15.83:0/2205407678 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f12e4066f60 0x7f12e4069420 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f12f00ad000 tx=0x7f12f00a6000 comp rx=0 tx=0).stop 2024-04-17T10:54:19.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.564+0000 7f13072af640 1 -- 172.21.15.83:0/2205407678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f130015c960 msgr2=0x7f130014fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:19.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.564+0000 7f13072af640 1 --2- 172.21.15.83:0/2205407678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f130015c960 0x7f130014fcc0 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f12f8007e60 tx=0x7f12f800e4b0 comp rx=0 tx=0).stop 2024-04-17T10:54:19.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.564+0000 7f13072af640 1 -- 172.21.15.83:0/2205407678 shutdown_connections 2024-04-17T10:54:19.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.564+0000 7f13072af640 1 --2- 172.21.15.83:0/2205407678 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f12e4066f60 0x7f12e4069420 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:19.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.564+0000 7f13072af640 1 --2- 172.21.15.83:0/2205407678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f130015c960 0x7f130014fcc0 unknown :-1 s=CLOSED pgs=2384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:19.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.564+0000 7f13072af640 1 --2- 172.21.15.83:0/2205407678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f130015ba30 0x7f130013b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:19.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.564+0000 7f13072af640 1 --2- 172.21.15.83:0/2205407678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13000cfbe0 0x7f1300151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:19.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.564+0000 7f13072af640 1 -- 172.21.15.83:0/2205407678 >> 172.21.15.83:0/2205407678 conn(0x7f130006abe0 msgr2=0x7f1300148a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:19.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.564+0000 7f13072af640 1 -- 172.21.15.83:0/2205407678 shutdown_connections 2024-04-17T10:54:19.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:19.564+0000 7f13072af640 1 -- 172.21.15.83:0/2205407678 wait complete. 2024-04-17T10:54:19.973 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:54:19.974 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:54:20.975 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:54:21.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.108+0000 7f536e783640 1 Processor -- start 2024-04-17T10:54:21.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.108+0000 7f536e783640 1 -- start start 2024-04-17T10:54:21.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.108+0000 7f536e783640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f536815ba30 0x7f5368072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:21.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.108+0000 7f536e783640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f536815c960 0x7f53680cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:21.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.108+0000 7f536e783640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53680cfea0 con 0x7f53680cfa10 2024-04-17T10:54:21.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.108+0000 7f536e783640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5368071ea0 con 0x7f536815c960 2024-04-17T10:54:21.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.108+0000 7f536e783640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536806e320 con 0x7f536815ba30 2024-04-17T10:54:21.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.108+0000 7f5367fff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f536815ba30 0x7f5368072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:21.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.108+0000 7f5367fff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f536815ba30 0x7f5368072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:43332/0 (socket says 172.21.15.83:43332) 2024-04-17T10:54:21.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.108+0000 7f5367fff640 1 -- 172.21.15.83:0/3487905807 learned_addr learned my addr 172.21.15.83:0/3487905807 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:21.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.108+0000 7f53677fe640 1 --2- 172.21.15.83:0/3487905807 >> v2:172.21.15.167:3300/0 conn(0x7f536815c960 0x7f53680cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:21.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.108+0000 7f53677fe640 1 -- 172.21.15.83:0/3487905807 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f536815ba30 msgr2=0x7f5368072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:21.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.108+0000 7f53677fe640 1 --2- 172.21.15.83:0/3487905807 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f536815ba30 0x7f5368072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:21.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.108+0000 7f53677fe640 1 -- 172.21.15.83:0/3487905807 >> v1:172.21.15.167:6789/0 conn(0x7f53680cfa10 legacy=0x7f536806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:21.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.108+0000 7f53677fe640 1 -- 172.21.15.83:0/3487905807 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536806e040 con 0x7f536815c960 2024-04-17T10:54:21.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.108+0000 7f5367fff640 1 --2- 172.21.15.83:0/3487905807 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f536815ba30 0x7f5368072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:54:21.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.112+0000 7f53677fe640 1 --2- 172.21.15.83:0/3487905807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f536815c960 0x7f53680cf4d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f53580097c0 tx=0x7f535801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e55ce062d3af0160 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:21.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.112+0000 7f5366ffd640 1 -- 172.21.15.83:0/3487905807 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53580b0070 con 0x7f536815c960 2024-04-17T10:54:21.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.112+0000 7f5366ffd640 1 -- 172.21.15.83:0/3487905807 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53580ab600 con 0x7f536815c960 2024-04-17T10:54:21.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.112+0000 7f5366ffd640 1 -- 172.21.15.83:0/3487905807 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53580ab9d0 con 0x7f536815c960 2024-04-17T10:54:21.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.112+0000 7f536e783640 1 -- 172.21.15.83:0/3487905807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f536815c960 msgr2=0x7f53680cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:21.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.112+0000 7f536e783640 1 --2- 172.21.15.83:0/3487905807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f536815c960 0x7f53680cf4d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f53580097c0 tx=0x7f535801c410 comp rx=0 tx=0).stop 2024-04-17T10:54:21.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.112+0000 7f536e783640 1 -- 172.21.15.83:0/3487905807 shutdown_connections 2024-04-17T10:54:21.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.112+0000 7f536e783640 1 --2- 172.21.15.83:0/3487905807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f536815c960 0x7f53680cf4d0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:21.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.112+0000 7f536e783640 1 --2- 172.21.15.83:0/3487905807 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f536815ba30 0x7f5368072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:21.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.112+0000 7f536e783640 1 -- 172.21.15.83:0/3487905807 >> 172.21.15.83:0/3487905807 conn(0x7f536806abe0 msgr2=0x7f536806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:21.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.112+0000 7f536e783640 1 -- 172.21.15.83:0/3487905807 shutdown_connections 2024-04-17T10:54:21.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.112+0000 7f536e783640 1 -- 172.21.15.83:0/3487905807 wait complete. 2024-04-17T10:54:21.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.112+0000 7f536e783640 1 Processor -- start 2024-04-17T10:54:21.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.112+0000 7f536e783640 1 -- start start 2024-04-17T10:54:21.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.116+0000 7f536e783640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53680cfa10 0x7f5368142130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:21.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.116+0000 7f536e783640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f536815ba30 0x7f53680b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:21.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.116+0000 7f536e783640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f536815c960 0x7f536815e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:21.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.116+0000 7f536e783640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536806dca0 con 0x7f536815c960 2024-04-17T10:54:21.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.116+0000 7f536e783640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53680aeae0 con 0x7f536815ba30 2024-04-17T10:54:21.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.116+0000 7f536e783640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536806de20 con 0x7f53680cfa10 2024-04-17T10:54:21.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.116+0000 7f536cd20640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f536815c960 0x7f536815e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:21.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.116+0000 7f536cd20640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f536815c960 0x7f536815e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50960/0 (socket says 172.21.15.83:50960) 2024-04-17T10:54:21.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.116+0000 7f536cd20640 1 -- 172.21.15.83:0/132587996 learned_addr learned my addr 172.21.15.83:0/132587996 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:21.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.116+0000 7f5367fff640 1 --2- 172.21.15.83:0/132587996 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53680cfa10 0x7f5368142130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:21.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.116+0000 7f53677fe640 1 --2- 172.21.15.83:0/132587996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f536815ba30 0x7f53680b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:21.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.116+0000 7f536cd20640 1 -- 172.21.15.83:0/132587996 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53680cfa10 msgr2=0x7f5368142130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:21.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.116+0000 7f536cd20640 1 --2- 172.21.15.83:0/132587996 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53680cfa10 0x7f5368142130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:21.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.116+0000 7f536cd20640 1 -- 172.21.15.83:0/132587996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f536815ba30 msgr2=0x7f53680b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.116+0000 7f536cd20640 1 --2- 172.21.15.83:0/132587996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f536815ba30 0x7f53680b3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.116+0000 7f536cd20640 1 -- 172.21.15.83:0/132587996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5368142ad0 con 0x7f536815c960 2024-04-17T10:54:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.116+0000 7f536cd20640 1 --2- 172.21.15.83:0/132587996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f536815c960 0x7f536815e2a0 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f536000c760 tx=0x7f536000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.116+0000 7f5364ff9640 1 -- 172.21.15.83:0/132587996 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5360003b80 con 0x7f536815c960 2024-04-17T10:54:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.116+0000 7f5364ff9640 1 -- 172.21.15.83:0/132587996 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5360003d20 con 0x7f536815c960 2024-04-17T10:54:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.116+0000 7f5364ff9640 1 -- 172.21.15.83:0/132587996 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536000aa30 con 0x7f536815c960 2024-04-17T10:54:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.116+0000 7f536e783640 1 -- 172.21.15.83:0/132587996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53680b2ae0 con 0x7f536815c960 2024-04-17T10:54:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.116+0000 7f536e783640 1 -- 172.21.15.83:0/132587996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53680b30a0 con 0x7f536815c960 2024-04-17T10:54:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.116+0000 7f536e783640 1 -- 172.21.15.83:0/132587996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5368074f90 con 0x7f536815c960 2024-04-17T10:54:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.116+0000 7f5364ff9640 1 -- 172.21.15.83:0/132587996 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f536000abd0 con 0x7f536815c960 2024-04-17T10:54:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.116+0000 7f5364ff9640 1 --2- 172.21.15.83:0/132587996 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5354066f90 0x7f5354069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.116+0000 7f5367fff640 1 --2- 172.21.15.83:0/132587996 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5354066f90 0x7f5354069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.116+0000 7f5364ff9640 1 -- 172.21.15.83:0/132587996 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f5360084f40 con 0x7f536815c960 2024-04-17T10:54:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.120+0000 7f5367fff640 1 --2- 172.21.15.83:0/132587996 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5354066f90 0x7f5354069450 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f535c009a40 tx=0x7f535c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:21.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.120+0000 7f5364ff9640 1 -- 172.21.15.83:0/132587996 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5360055ee0 con 0x7f536815c960 2024-04-17T10:54:21.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.308+0000 7f536e783640 1 -- 172.21.15.83:0/132587996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f536815d200 con 0x7f536815c960 2024-04-17T10:54:21.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.308+0000 7f5364ff9640 1 -- 172.21.15.83:0/132587996 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f53600557e0 con 0x7f536815c960 2024-04-17T10:54:21.303 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:21.303 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:54:21.304 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:54:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.312+0000 7f536e783640 1 -- 172.21.15.83:0/132587996 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5354066f90 msgr2=0x7f5354069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.312+0000 7f536e783640 1 --2- 172.21.15.83:0/132587996 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5354066f90 0x7f5354069450 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f535c009a40 tx=0x7f535c008040 comp rx=0 tx=0).stop 2024-04-17T10:54:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.312+0000 7f536e783640 1 -- 172.21.15.83:0/132587996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f536815c960 msgr2=0x7f536815e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.312+0000 7f536e783640 1 --2- 172.21.15.83:0/132587996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f536815c960 0x7f536815e2a0 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f536000c760 tx=0x7f536000cc20 comp rx=0 tx=0).stop 2024-04-17T10:54:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.312+0000 7f536e783640 1 -- 172.21.15.83:0/132587996 shutdown_connections 2024-04-17T10:54:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.312+0000 7f536e783640 1 --2- 172.21.15.83:0/132587996 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f5354066f90 0x7f5354069450 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:21.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.312+0000 7f536e783640 1 --2- 172.21.15.83:0/132587996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f536815c960 0x7f536815e2a0 unknown :-1 s=CLOSED pgs=2385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:21.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.312+0000 7f536e783640 1 --2- 172.21.15.83:0/132587996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f536815ba30 0x7f53680b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:21.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.312+0000 7f536e783640 1 --2- 172.21.15.83:0/132587996 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53680cfa10 0x7f5368142130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:21.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.312+0000 7f536e783640 1 -- 172.21.15.83:0/132587996 >> 172.21.15.83:0/132587996 conn(0x7f536806abe0 msgr2=0x7f53680af2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:21.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.312+0000 7f536e783640 1 -- 172.21.15.83:0/132587996 shutdown_connections 2024-04-17T10:54:21.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:21.312+0000 7f536e783640 1 -- 172.21.15.83:0/132587996 wait complete. 2024-04-17T10:54:21.723 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:54:21.723 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:54:22.724 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:54:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.860+0000 7f3532c01640 1 Processor -- start 2024-04-17T10:54:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.864+0000 7f3532c01640 1 -- start start 2024-04-17T10:54:22.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.864+0000 7f3532c01640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f352c0aedd0 0x7f352c06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:22.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.864+0000 7f3532c01640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f352c06dae0 0x7f352c0b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:22.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.864+0000 7f3532c01640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f352c15e440 con 0x7f352c15db50 2024-04-17T10:54:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.864+0000 7f3532c01640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f352c15e720 con 0x7f352c0aedd0 2024-04-17T10:54:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.864+0000 7f3532c01640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f352c15ea00 con 0x7f352c06dae0 2024-04-17T10:54:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.864+0000 7f352bfff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f352c0aedd0 0x7f352c06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.864+0000 7f353119e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f352c06dae0 0x7f352c0b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.864+0000 7f353099d640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f352c15db50 0x7f352c070470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:34404/0 (socket says 172.21.15.83:34404) 2024-04-17T10:54:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.864+0000 7f353099d640 1 -- 172.21.15.83:0/1089954630 learned_addr learned my addr 172.21.15.83:0/1089954630 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.864+0000 7f353119e640 1 -- 172.21.15.83:0/1089954630 >> v2:172.21.15.167:3300/0 conn(0x7f352c0aedd0 msgr2=0x7f352c06d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:22.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.864+0000 7f353119e640 1 --2- 172.21.15.83:0/1089954630 >> v2:172.21.15.167:3300/0 conn(0x7f352c0aedd0 0x7f352c06d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:22.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.864+0000 7f353119e640 1 -- 172.21.15.83:0/1089954630 >> v1:172.21.15.167:6789/0 conn(0x7f352c15db50 legacy=0x7f352c070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:22.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.864+0000 7f353119e640 1 -- 172.21.15.83:0/1089954630 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352c15ece0 con 0x7f352c06dae0 2024-04-17T10:54:22.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.864+0000 7f352bfff640 1 --2- 172.21.15.83:0/1089954630 >> v2:172.21.15.167:3300/0 conn(0x7f352c0aedd0 0x7f352c06d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:54:22.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.868+0000 7f353119e640 1 --2- 172.21.15.83:0/1089954630 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f352c06dae0 0x7f352c0b2e10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3524009d90 tx=0x7f352401c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b8f9815f257a378 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:22.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.868+0000 7f352b7fe640 1 -- 172.21.15.83:0/1089954630 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35240b2070 con 0x7f352c06dae0 2024-04-17T10:54:22.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.868+0000 7f352b7fe640 1 -- 172.21.15.83:0/1089954630 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35240a5830 con 0x7f352c06dae0 2024-04-17T10:54:22.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.868+0000 7f352b7fe640 1 -- 172.21.15.83:0/1089954630 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35240a5c00 con 0x7f352c06dae0 2024-04-17T10:54:22.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.868+0000 7f3532c01640 1 -- 172.21.15.83:0/1089954630 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f352c06dae0 msgr2=0x7f352c0b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:22.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.868+0000 7f3532c01640 1 --2- 172.21.15.83:0/1089954630 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f352c06dae0 0x7f352c0b2e10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3524009d90 tx=0x7f352401c970 comp rx=0 tx=0).stop 2024-04-17T10:54:22.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.868+0000 7f3532c01640 1 -- 172.21.15.83:0/1089954630 shutdown_connections 2024-04-17T10:54:22.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.868+0000 7f3532c01640 1 --2- 172.21.15.83:0/1089954630 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f352c06dae0 0x7f352c0b2e10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:22.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.868+0000 7f3532c01640 1 --2- 172.21.15.83:0/1089954630 >> v2:172.21.15.167:3300/0 conn(0x7f352c0aedd0 0x7f352c06d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:22.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.868+0000 7f3532c01640 1 -- 172.21.15.83:0/1089954630 >> 172.21.15.83:0/1089954630 conn(0x7f352c06abe0 msgr2=0x7f352c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:22.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.868+0000 7f3532c01640 1 -- 172.21.15.83:0/1089954630 shutdown_connections 2024-04-17T10:54:22.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.868+0000 7f3532c01640 1 -- 172.21.15.83:0/1089954630 wait complete. 2024-04-17T10:54:22.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.868+0000 7f3532c01640 1 Processor -- start 2024-04-17T10:54:22.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.868+0000 7f3532c01640 1 -- start start 2024-04-17T10:54:22.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.868+0000 7f3532c01640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f352c06dae0 0x7f352c143360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:22.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.868+0000 7f3532c01640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f352c0aedd0 0x7f352c0cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:22.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.868+0000 7f3532c01640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f352c15db50 0x7f352c156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:22.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.868+0000 7f3532c01640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c0b3ef0 con 0x7f352c0aedd0 2024-04-17T10:54:22.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.868+0000 7f3532c01640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c0b3d70 con 0x7f352c06dae0 2024-04-17T10:54:22.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.868+0000 7f3532c01640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f352c0b4070 con 0x7f352c15db50 2024-04-17T10:54:22.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.872+0000 7f352bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f352c0aedd0 0x7f352c0cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:22.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.872+0000 7f352bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f352c0aedd0 0x7f352c0cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50976/0 (socket says 172.21.15.83:50976) 2024-04-17T10:54:22.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.872+0000 7f352bfff640 1 -- 172.21.15.83:0/137801001 learned_addr learned my addr 172.21.15.83:0/137801001 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:22.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.872+0000 7f353099d640 1 --2- 172.21.15.83:0/137801001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f352c06dae0 0x7f352c143360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:22.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.872+0000 7f353119e640 1 --2- 172.21.15.83:0/137801001 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f352c15db50 0x7f352c156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:22.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.872+0000 7f352bfff640 1 -- 172.21.15.83:0/137801001 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f352c15db50 msgr2=0x7f352c156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:22.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.872+0000 7f352bfff640 1 --2- 172.21.15.83:0/137801001 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f352c15db50 0x7f352c156740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:22.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.872+0000 7f352bfff640 1 -- 172.21.15.83:0/137801001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f352c06dae0 msgr2=0x7f352c143360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:22.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.872+0000 7f352bfff640 1 --2- 172.21.15.83:0/137801001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f352c06dae0 0x7f352c143360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:22.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.872+0000 7f352bfff640 1 -- 172.21.15.83:0/137801001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352c0cdc20 con 0x7f352c0aedd0 2024-04-17T10:54:22.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.872+0000 7f353099d640 1 --2- 172.21.15.83:0/137801001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f352c06dae0 0x7f352c143360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:22.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.872+0000 7f352bfff640 1 --2- 172.21.15.83:0/137801001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f352c0aedd0 0x7f352c0cc410 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f352000ebe0 tx=0x7f352000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:22.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.872+0000 7f35297fa640 1 -- 172.21.15.83:0/137801001 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3520017070 con 0x7f352c0aedd0 2024-04-17T10:54:22.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.872+0000 7f35297fa640 1 -- 172.21.15.83:0/137801001 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f352000cc30 con 0x7f352c0aedd0 2024-04-17T10:54:22.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.872+0000 7f3532c01640 1 -- 172.21.15.83:0/137801001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f352c151bd0 con 0x7f352c0aedd0 2024-04-17T10:54:22.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.872+0000 7f35297fa640 1 -- 172.21.15.83:0/137801001 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3520013470 con 0x7f352c0aedd0 2024-04-17T10:54:22.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.872+0000 7f3532c01640 1 -- 172.21.15.83:0/137801001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f352c0cce60 con 0x7f352c0aedd0 2024-04-17T10:54:22.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.872+0000 7f3532c01640 1 -- 172.21.15.83:0/137801001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f352c074f90 con 0x7f352c0aedd0 2024-04-17T10:54:22.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.872+0000 7f35297fa640 1 -- 172.21.15.83:0/137801001 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f352001a070 con 0x7f352c0aedd0 2024-04-17T10:54:22.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.872+0000 7f35297fa640 1 --2- 172.21.15.83:0/137801001 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3518066f60 0x7f3518069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:22.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.872+0000 7f35297fa640 1 -- 172.21.15.83:0/137801001 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f3520084ff0 con 0x7f352c0aedd0 2024-04-17T10:54:22.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.872+0000 7f353099d640 1 --2- 172.21.15.83:0/137801001 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3518066f60 0x7f3518069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:22.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.872+0000 7f353099d640 1 --2- 172.21.15.83:0/137801001 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3518066f60 0x7f3518069420 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f351c005890 tx=0x7f351c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:22.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:22.876+0000 7f35297fa640 1 -- 172.21.15.83:0/137801001 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3520055fa0 con 0x7f352c0aedd0 2024-04-17T10:54:23.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:23.064+0000 7f3532c01640 1 -- 172.21.15.83:0/137801001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f352c0b2af0 con 0x7f352c0aedd0 2024-04-17T10:54:23.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:23.064+0000 7f35297fa640 1 -- 172.21.15.83:0/137801001 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f35200558a0 con 0x7f352c0aedd0 2024-04-17T10:54:23.057 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:23.057 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:54:23.057 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:54:23.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:23.064+0000 7f3532c01640 1 -- 172.21.15.83:0/137801001 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3518066f60 msgr2=0x7f3518069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:23.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:23.064+0000 7f3532c01640 1 --2- 172.21.15.83:0/137801001 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3518066f60 0x7f3518069420 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f351c005890 tx=0x7f351c009040 comp rx=0 tx=0).stop 2024-04-17T10:54:23.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:23.064+0000 7f3532c01640 1 -- 172.21.15.83:0/137801001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f352c0aedd0 msgr2=0x7f352c0cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:23.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:23.064+0000 7f3532c01640 1 --2- 172.21.15.83:0/137801001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f352c0aedd0 0x7f352c0cc410 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f352000ebe0 tx=0x7f352000c520 comp rx=0 tx=0).stop 2024-04-17T10:54:23.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:23.064+0000 7f3532c01640 1 -- 172.21.15.83:0/137801001 shutdown_connections 2024-04-17T10:54:23.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:23.064+0000 7f3532c01640 1 --2- 172.21.15.83:0/137801001 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3518066f60 0x7f3518069420 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:23.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:23.064+0000 7f3532c01640 1 --2- 172.21.15.83:0/137801001 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f352c15db50 0x7f352c156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:23.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:23.064+0000 7f3532c01640 1 --2- 172.21.15.83:0/137801001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f352c0aedd0 0x7f352c0cc410 unknown :-1 s=CLOSED pgs=2386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:23.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:23.064+0000 7f3532c01640 1 --2- 172.21.15.83:0/137801001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f352c06dae0 0x7f352c143360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:23.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:23.064+0000 7f3532c01640 1 -- 172.21.15.83:0/137801001 >> 172.21.15.83:0/137801001 conn(0x7f352c06abe0 msgr2=0x7f352c155c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:23.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:23.064+0000 7f3532c01640 1 -- 172.21.15.83:0/137801001 shutdown_connections 2024-04-17T10:54:23.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:23.068+0000 7f3532c01640 1 -- 172.21.15.83:0/137801001 wait complete. 2024-04-17T10:54:23.478 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:54:23.478 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:54:24.479 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:54:24.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a1141f640 1 Processor -- start 2024-04-17T10:54:24.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a1141f640 1 -- start start 2024-04-17T10:54:24.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a1141f640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2a0c0aedd0 0x7f2a0c06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:24.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a1141f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a0c15e440 con 0x7f2a0c15db50 2024-04-17T10:54:24.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a1141f640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a0c15e720 con 0x7f2a0c06dae0 2024-04-17T10:54:24.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a1141f640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a0c15ea00 con 0x7f2a0c0aedd0 2024-04-17T10:54:24.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a0affd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2a0c15db50 0x7f2a0c070470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38586/0 (socket says 172.21.15.83:38586) 2024-04-17T10:54:24.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a0affd640 1 -- 172.21.15.83:0/3379333565 learned_addr learned my addr 172.21.15.83:0/3379333565 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:24.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a0a7fc640 1 --2- 172.21.15.83:0/3379333565 >> v2:172.21.15.167:3300/0 conn(0x7f2a0c0aedd0 0x7f2a0c06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:24.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a09ffb640 1 -- 172.21.15.83:0/3379333565 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3927617747 0 0) 0x7f2a0c15e440 con 0x7f2a0c15db50 2024-04-17T10:54:24.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a09ffb640 1 -- 172.21.15.83:0/3379333565 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29f4003530 con 0x7f2a0c15db50 2024-04-17T10:54:24.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a09ffb640 1 -- 172.21.15.83:0/3379333565 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 721437946 0 0) 0x7f29f4003530 con 0x7f2a0c15db50 2024-04-17T10:54:24.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a09ffb640 1 -- 172.21.15.83:0/3379333565 >> v2:172.21.15.167:3300/0 conn(0x7f2a0c0aedd0 msgr2=0x7f2a0c06d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:24.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a09ffb640 1 --2- 172.21.15.83:0/3379333565 >> v2:172.21.15.167:3300/0 conn(0x7f2a0c0aedd0 0x7f2a0c06d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:24.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a09ffb640 1 -- 172.21.15.83:0/3379333565 >> v1:172.21.15.167:6789/0 conn(0x7f2a0c06dae0 legacy=0x7f2a0c0b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:24.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a09ffb640 1 -- 172.21.15.83:0/3379333565 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a0c15ece0 con 0x7f2a0c15db50 2024-04-17T10:54:24.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a09ffb640 1 -- 172.21.15.83:0/3379333565 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f29fc001680 con 0x7f2a0c15db50 2024-04-17T10:54:24.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a09ffb640 1 -- 172.21.15.83:0/3379333565 >> v1:172.21.15.83:6789/0 conn(0x7f2a0c15db50 legacy=0x7f2a0c070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:24.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a09ffb640 1 --2- 172.21.15.83:0/3379333565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29f40039c0 0x7f29f4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:24.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a09ffb640 1 --2- 172.21.15.83:0/3379333565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29f4004810 0x7f29f4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:24.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a09ffb640 1 --2- 172.21.15.83:0/3379333565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29f40051d0 0x7f29f400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:24.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a09ffb640 1 -- 172.21.15.83:0/3379333565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c15ea00 con 0x7f29f40039c0 2024-04-17T10:54:24.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a09ffb640 1 -- 172.21.15.83:0/3379333565 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f400c070 con 0x7f29f40051d0 2024-04-17T10:54:24.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a09ffb640 1 -- 172.21.15.83:0/3379333565 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29f400c270 con 0x7f29f4004810 2024-04-17T10:54:24.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a0affd640 1 --2- 172.21.15.83:0/3379333565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29f40039c0 0x7f29f4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:24.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a0a7fc640 1 --2- 172.21.15.83:0/3379333565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29f4004810 0x7f29f4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:24.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a0b7fe640 1 --2- 172.21.15.83:0/3379333565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29f40051d0 0x7f29f400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:24.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a0affd640 1 -- 172.21.15.83:0/3379333565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29f4004810 msgr2=0x7f29f4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:24.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a0affd640 1 --2- 172.21.15.83:0/3379333565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29f4004810 0x7f29f4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:24.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a0affd640 1 -- 172.21.15.83:0/3379333565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29f40051d0 msgr2=0x7f29f400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:24.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a0affd640 1 --2- 172.21.15.83:0/3379333565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29f40051d0 0x7f29f400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:24.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a0affd640 1 -- 172.21.15.83:0/3379333565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f4003530 con 0x7f29f40039c0 2024-04-17T10:54:24.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a0b7fe640 1 --2- 172.21.15.83:0/3379333565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29f40051d0 0x7f29f400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:24.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a0a7fc640 1 --2- 172.21.15.83:0/3379333565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29f4004810 0x7f29f4004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:24.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.620+0000 7f2a0b7fe640 1 -- 172.21.15.83:0/3379333565 reap_dead start 2024-04-17T10:54:24.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.624+0000 7f2a0affd640 1 --2- 172.21.15.83:0/3379333565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29f40039c0 0x7f29f4003de0 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f29fc00c1c0 tx=0x7f29fc01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22e140975951925 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:24.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.624+0000 7f2a09ffb640 1 -- 172.21.15.83:0/3379333565 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc007d00 con 0x7f29f40039c0 2024-04-17T10:54:24.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.624+0000 7f2a09ffb640 1 -- 172.21.15.83:0/3379333565 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29fc0a9020 con 0x7f29f40039c0 2024-04-17T10:54:24.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.624+0000 7f2a09ffb640 1 -- 172.21.15.83:0/3379333565 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc005b00 con 0x7f29f40039c0 2024-04-17T10:54:24.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.624+0000 7f2a1141f640 1 -- 172.21.15.83:0/3379333565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29f40039c0 msgr2=0x7f29f4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:24.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.624+0000 7f2a1141f640 1 --2- 172.21.15.83:0/3379333565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29f40039c0 0x7f29f4003de0 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f29fc00c1c0 tx=0x7f29fc01cc60 comp rx=0 tx=0).stop 2024-04-17T10:54:24.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.624+0000 7f2a1141f640 1 -- 172.21.15.83:0/3379333565 shutdown_connections 2024-04-17T10:54:24.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.624+0000 7f2a1141f640 1 --2- 172.21.15.83:0/3379333565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29f40039c0 0x7f29f4003de0 unknown :-1 s=CLOSED pgs=2388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:24.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.624+0000 7f2a1141f640 1 -- 172.21.15.83:0/3379333565 >> 172.21.15.83:0/3379333565 conn(0x7f2a0c06abe0 msgr2=0x7f2a0c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:24.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.624+0000 7f2a1141f640 1 -- 172.21.15.83:0/3379333565 shutdown_connections 2024-04-17T10:54:24.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.624+0000 7f2a1141f640 1 -- 172.21.15.83:0/3379333565 wait complete. 2024-04-17T10:54:24.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.624+0000 7f2a1141f640 1 Processor -- start 2024-04-17T10:54:24.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.624+0000 7f2a1141f640 1 -- start start 2024-04-17T10:54:24.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.628+0000 7f2a1141f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29f40039c0 0x7f2a0c0ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:24.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.628+0000 7f2a1141f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a0c151760 0x7f2a0c1470c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:24.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.628+0000 7f2a1141f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a0c1f2780 0x7f2a0c0abcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:24.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.628+0000 7f2a1141f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c0cfee0 con 0x7f2a0c1f2780 2024-04-17T10:54:24.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.628+0000 7f2a1141f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c0cfd60 con 0x7f29f40039c0 2024-04-17T10:54:24.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.628+0000 7f2a1141f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c0d0060 con 0x7f2a0c151760 2024-04-17T10:54:24.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.628+0000 7f2a0b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a0c1f2780 0x7f2a0c0abcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:24.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.628+0000 7f2a0b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a0c1f2780 0x7f2a0c0abcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50986/0 (socket says 172.21.15.83:50986) 2024-04-17T10:54:24.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.628+0000 7f2a0b7fe640 1 -- 172.21.15.83:0/3977645296 learned_addr learned my addr 172.21.15.83:0/3977645296 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:24.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.628+0000 7f2a0affd640 1 --2- 172.21.15.83:0/3977645296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29f40039c0 0x7f2a0c0ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:24.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.628+0000 7f2a0a7fc640 1 --2- 172.21.15.83:0/3977645296 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a0c151760 0x7f2a0c1470c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:24.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.628+0000 7f2a0b7fe640 1 -- 172.21.15.83:0/3977645296 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a0c151760 msgr2=0x7f2a0c1470c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:24.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.628+0000 7f2a0b7fe640 1 --2- 172.21.15.83:0/3977645296 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a0c151760 0x7f2a0c1470c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:24.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.628+0000 7f2a0b7fe640 1 -- 172.21.15.83:0/3977645296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29f40039c0 msgr2=0x7f2a0c0ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:24.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.628+0000 7f2a0b7fe640 1 --2- 172.21.15.83:0/3977645296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29f40039c0 0x7f2a0c0ade00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:24.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.628+0000 7f2a0b7fe640 1 -- 172.21.15.83:0/3977645296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a0c0a9430 con 0x7f2a0c1f2780 2024-04-17T10:54:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.628+0000 7f2a0a7fc640 1 --2- 172.21.15.83:0/3977645296 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a0c151760 0x7f2a0c1470c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.628+0000 7f2a0b7fe640 1 --2- 172.21.15.83:0/3977645296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a0c1f2780 0x7f2a0c0abcc0 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0722a0 tx=0x7f2a0400d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.628+0000 7f29f37fe640 1 -- 172.21.15.83:0/3977645296 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a04018070 con 0x7f2a0c1f2780 2024-04-17T10:54:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.628+0000 7f29f37fe640 1 -- 172.21.15.83:0/3977645296 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a04003d90 con 0x7f2a0c1f2780 2024-04-17T10:54:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.628+0000 7f2a1141f640 1 -- 172.21.15.83:0/3977645296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a0c0a96d0 con 0x7f2a0c1f2780 2024-04-17T10:54:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.628+0000 7f29f37fe640 1 -- 172.21.15.83:0/3977645296 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a040052b0 con 0x7f2a0c1f2780 2024-04-17T10:54:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.628+0000 7f2a1141f640 1 -- 172.21.15.83:0/3977645296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a0c0a9c90 con 0x7f2a0c1f2780 2024-04-17T10:54:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.628+0000 7f2a1141f640 1 -- 172.21.15.83:0/3977645296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a0c074f90 con 0x7f2a0c1f2780 2024-04-17T10:54:24.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.628+0000 7f29f37fe640 1 -- 172.21.15.83:0/3977645296 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2a0401b070 con 0x7f2a0c1f2780 2024-04-17T10:54:24.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.628+0000 7f29f37fe640 1 --2- 172.21.15.83:0/3977645296 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f29f8066f60 0x7f29f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:24.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.628+0000 7f29f37fe640 1 -- 172.21.15.83:0/3977645296 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f2a04084b90 con 0x7f2a0c1f2780 2024-04-17T10:54:24.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.628+0000 7f2a0affd640 1 --2- 172.21.15.83:0/3977645296 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f29f8066f60 0x7f29f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:24.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.632+0000 7f2a0affd640 1 --2- 172.21.15.83:0/3977645296 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f29f8066f60 0x7f29f8069420 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f29fc0aa000 tx=0x7f2a0c15e8f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:24.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.632+0000 7f29f37fe640 1 -- 172.21.15.83:0/3977645296 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2a04022020 con 0x7f2a0c1f2780 2024-04-17T10:54:24.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.820+0000 7f2a1141f640 1 -- 172.21.15.83:0/3977645296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a0c0b2af0 con 0x7f2a0c1f2780 2024-04-17T10:54:24.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.820+0000 7f29f37fe640 1 -- 172.21.15.83:0/3977645296 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f2a04055ba0 con 0x7f2a0c1f2780 2024-04-17T10:54:24.813 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:24.814 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:54:24.814 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:54:24.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.820+0000 7f2a1141f640 1 -- 172.21.15.83:0/3977645296 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f29f8066f60 msgr2=0x7f29f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:24.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.820+0000 7f2a1141f640 1 --2- 172.21.15.83:0/3977645296 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f29f8066f60 0x7f29f8069420 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f29fc0aa000 tx=0x7f2a0c15e8f0 comp rx=0 tx=0).stop 2024-04-17T10:54:24.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.820+0000 7f2a1141f640 1 -- 172.21.15.83:0/3977645296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a0c1f2780 msgr2=0x7f2a0c0abcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:24.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.820+0000 7f2a1141f640 1 --2- 172.21.15.83:0/3977645296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a0c1f2780 0x7f2a0c0abcc0 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0722a0 tx=0x7f2a0400d9d0 comp rx=0 tx=0).stop 2024-04-17T10:54:24.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.824+0000 7f2a1141f640 1 -- 172.21.15.83:0/3977645296 shutdown_connections 2024-04-17T10:54:24.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.824+0000 7f2a1141f640 1 --2- 172.21.15.83:0/3977645296 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f29f8066f60 0x7f29f8069420 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:24.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.824+0000 7f2a1141f640 1 --2- 172.21.15.83:0/3977645296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a0c1f2780 0x7f2a0c0abcc0 unknown :-1 s=CLOSED pgs=2389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:24.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.824+0000 7f2a1141f640 1 --2- 172.21.15.83:0/3977645296 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a0c151760 0x7f2a0c1470c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:24.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.824+0000 7f2a1141f640 1 --2- 172.21.15.83:0/3977645296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29f40039c0 0x7f2a0c0ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:24.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.824+0000 7f2a1141f640 1 -- 172.21.15.83:0/3977645296 >> 172.21.15.83:0/3977645296 conn(0x7f2a0c06abe0 msgr2=0x7f2a0c0b3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:24.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.824+0000 7f2a1141f640 1 -- 172.21.15.83:0/3977645296 shutdown_connections 2024-04-17T10:54:24.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:24.824+0000 7f2a1141f640 1 -- 172.21.15.83:0/3977645296 wait complete. 2024-04-17T10:54:25.234 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:54:25.234 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:54:26.234 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:54:26.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.372+0000 7fef0bdbd640 1 Processor -- start 2024-04-17T10:54:26.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.372+0000 7fef0bdbd640 1 -- start start 2024-04-17T10:54:26.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef0bdbd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef04146a00 0x7fef04146e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:26.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef0bdbd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fef040b2d80 0x7fef041620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:26.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef0bdbd640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef0406daf0 con 0x7fef0406d690 2024-04-17T10:54:26.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef0bdbd640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef04162610 con 0x7fef040b2d80 2024-04-17T10:54:26.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef0bdbd640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0406c960 con 0x7fef04146a00 2024-04-17T10:54:26.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef09358640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fef0406d690 0x7fef040b2670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:38594/0 (socket says 172.21.15.83:38594) 2024-04-17T10:54:26.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef09358640 1 -- 172.21.15.83:0/3458969164 learned_addr learned my addr 172.21.15.83:0/3458969164 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:26.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef0a35a640 1 --2- 172.21.15.83:0/3458969164 >> v2:172.21.15.167:3300/0 conn(0x7fef040b2d80 0x7fef041620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:26.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef08b57640 1 -- 172.21.15.83:0/3458969164 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2114835556 0 0) 0x7fef0406daf0 con 0x7fef0406d690 2024-04-17T10:54:26.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef09b59640 1 --2- 172.21.15.83:0/3458969164 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef04146a00 0x7fef04146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:26.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef08b57640 1 -- 172.21.15.83:0/3458969164 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feeec003530 con 0x7fef0406d690 2024-04-17T10:54:26.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef08b57640 1 -- 172.21.15.83:0/3458969164 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3445811545 0 0) 0x7feeec003530 con 0x7fef0406d690 2024-04-17T10:54:26.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef08b57640 1 -- 172.21.15.83:0/3458969164 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef04146a00 msgr2=0x7fef04146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:26.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef08b57640 1 --2- 172.21.15.83:0/3458969164 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef04146a00 0x7fef04146e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:26.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef08b57640 1 -- 172.21.15.83:0/3458969164 >> v2:172.21.15.167:3300/0 conn(0x7fef040b2d80 msgr2=0x7fef041620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:26.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef08b57640 1 --2- 172.21.15.83:0/3458969164 >> v2:172.21.15.167:3300/0 conn(0x7fef040b2d80 0x7fef041620d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:26.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef08b57640 1 -- 172.21.15.83:0/3458969164 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef04147570 con 0x7fef0406d690 2024-04-17T10:54:26.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef08b57640 1 -- 172.21.15.83:0/3458969164 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7feef8001680 con 0x7fef0406d690 2024-04-17T10:54:26.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef09b59640 1 --2- 172.21.15.83:0/3458969164 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef04146a00 0x7fef04146e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:54:26.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef08b57640 1 -- 172.21.15.83:0/3458969164 >> v1:172.21.15.83:6789/0 conn(0x7fef0406d690 legacy=0x7fef040b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:26.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef08b57640 1 --2- 172.21.15.83:0/3458969164 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef04146a00 0x7feeec0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:26.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef08b57640 1 --2- 172.21.15.83:0/3458969164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feeec003f00 0x7feeec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:26.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef08b57640 1 --2- 172.21.15.83:0/3458969164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feeec004d50 0x7feeec0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:26.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef08b57640 1 -- 172.21.15.83:0/3458969164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0406c960 con 0x7feeec003f00 2024-04-17T10:54:26.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef08b57640 1 -- 172.21.15.83:0/3458969164 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec00b9d0 con 0x7feeec004d50 2024-04-17T10:54:26.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef08b57640 1 -- 172.21.15.83:0/3458969164 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeec00bb80 con 0x7fef04146a00 2024-04-17T10:54:26.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef09358640 1 --2- 172.21.15.83:0/3458969164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feeec003f00 0x7feeec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:26.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef0a35a640 1 --2- 172.21.15.83:0/3458969164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feeec004d50 0x7feeec0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:26.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef09b59640 1 --2- 172.21.15.83:0/3458969164 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef04146a00 0x7feeec0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:26.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef09358640 1 -- 172.21.15.83:0/3458969164 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef04146a00 msgr2=0x7feeec0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:26.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef09358640 1 --2- 172.21.15.83:0/3458969164 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef04146a00 0x7feeec0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:26.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef09358640 1 -- 172.21.15.83:0/3458969164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feeec004d50 msgr2=0x7feeec0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:26.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef09358640 1 --2- 172.21.15.83:0/3458969164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feeec004d50 0x7feeec0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:26.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.376+0000 7fef09358640 1 -- 172.21.15.83:0/3458969164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec003530 con 0x7feeec003f00 2024-04-17T10:54:26.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef09358640 1 --2- 172.21.15.83:0/3458969164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feeec003f00 0x7feeec004810 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7feef800c210 tx=0x7feef801f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3647c5046acec91 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:26.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef08b57640 1 -- 172.21.15.83:0/3458969164 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef8007d40 con 0x7feeec003f00 2024-04-17T10:54:26.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef08b57640 1 -- 172.21.15.83:0/3458969164 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef8005b00 con 0x7feeec003f00 2024-04-17T10:54:26.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef08b57640 1 -- 172.21.15.83:0/3458969164 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef8007070 con 0x7feeec003f00 2024-04-17T10:54:26.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef0bdbd640 1 -- 172.21.15.83:0/3458969164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feeec003f00 msgr2=0x7feeec004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:26.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef0bdbd640 1 --2- 172.21.15.83:0/3458969164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feeec003f00 0x7feeec004810 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7feef800c210 tx=0x7feef801f3d0 comp rx=0 tx=0).stop 2024-04-17T10:54:26.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef0a35a640 1 -- 172.21.15.83:0/3458969164 reap_dead start 2024-04-17T10:54:26.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef0bdbd640 1 -- 172.21.15.83:0/3458969164 shutdown_connections 2024-04-17T10:54:26.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef0bdbd640 1 -- 172.21.15.83:0/3458969164 >> 172.21.15.83:0/3458969164 conn(0x7fef0406abe0 msgr2=0x7fef0406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:26.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef0bdbd640 1 -- 172.21.15.83:0/3458969164 shutdown_connections 2024-04-17T10:54:26.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef0bdbd640 1 -- 172.21.15.83:0/3458969164 wait complete. 2024-04-17T10:54:26.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef0bdbd640 1 Processor -- start 2024-04-17T10:54:26.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef0bdbd640 1 -- start start 2024-04-17T10:54:26.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef0bdbd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef0415d790 0x7fef04144f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:26.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef0bdbd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef04145480 0x7fef040c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:26.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef0bdbd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef040c7c30 0x7fef04133b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:26.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef0bdbd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0406ce90 con 0x7fef040c7c30 2024-04-17T10:54:26.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef0bdbd640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0406d200 con 0x7fef0415d790 2024-04-17T10:54:26.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef0bdbd640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef04146e70 con 0x7fef04145480 2024-04-17T10:54:26.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef0a35a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef040c7c30 0x7fef04133b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:26.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef0a35a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef040c7c30 0x7fef04133b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51012/0 (socket says 172.21.15.83:51012) 2024-04-17T10:54:26.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef0a35a640 1 -- 172.21.15.83:0/2876380772 learned_addr learned my addr 172.21.15.83:0/2876380772 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:26.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef09b59640 1 --2- 172.21.15.83:0/2876380772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef0415d790 0x7fef04144f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:26.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef09358640 1 --2- 172.21.15.83:0/2876380772 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef04145480 0x7fef040c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:26.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef0a35a640 1 -- 172.21.15.83:0/2876380772 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef04145480 msgr2=0x7fef040c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:26.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef0a35a640 1 --2- 172.21.15.83:0/2876380772 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef04145480 0x7fef040c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:26.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef0a35a640 1 -- 172.21.15.83:0/2876380772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef0415d790 msgr2=0x7fef04144f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:26.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef0a35a640 1 --2- 172.21.15.83:0/2876380772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef0415d790 0x7fef04144f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:26.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef0a35a640 1 -- 172.21.15.83:0/2876380772 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef040c4cd0 con 0x7fef040c7c30 2024-04-17T10:54:26.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef09358640 1 --2- 172.21.15.83:0/2876380772 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef04145480 0x7fef040c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:26.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef0a35a640 1 --2- 172.21.15.83:0/2876380772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef040c7c30 0x7fef04133b30 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7fef0406df20 tx=0x7feefc0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:26.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef027fc640 1 -- 172.21.15.83:0/2876380772 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc011e40 con 0x7fef040c7c30 2024-04-17T10:54:26.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef027fc640 1 -- 172.21.15.83:0/2876380772 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feefc01b070 con 0x7fef040c7c30 2024-04-17T10:54:26.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef0bdbd640 1 -- 172.21.15.83:0/2876380772 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef040c4fb0 con 0x7fef040c7c30 2024-04-17T10:54:26.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef027fc640 1 -- 172.21.15.83:0/2876380772 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc0039a0 con 0x7fef040c7c30 2024-04-17T10:54:26.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.380+0000 7fef0bdbd640 1 -- 172.21.15.83:0/2876380772 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef040c5570 con 0x7fef040c7c30 2024-04-17T10:54:26.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.384+0000 7fef0bdbd640 1 -- 172.21.15.83:0/2876380772 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef04074f90 con 0x7fef040c7c30 2024-04-17T10:54:26.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.384+0000 7fef027fc640 1 -- 172.21.15.83:0/2876380772 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feefc003b40 con 0x7fef040c7c30 2024-04-17T10:54:26.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.384+0000 7fef027fc640 1 --2- 172.21.15.83:0/2876380772 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feef0066f90 0x7feef0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:26.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.384+0000 7fef027fc640 1 -- 172.21.15.83:0/2876380772 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7feefc0858f0 con 0x7fef040c7c30 2024-04-17T10:54:26.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.384+0000 7fef09b59640 1 --2- 172.21.15.83:0/2876380772 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feef0066f90 0x7feef0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:26.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.384+0000 7fef09b59640 1 --2- 172.21.15.83:0/2876380772 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feef0066f90 0x7feef0069450 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7feef40107c0 tx=0x7feef4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:26.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.388+0000 7fef027fc640 1 -- 172.21.15.83:0/2876380772 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feefc056890 con 0x7fef040c7c30 2024-04-17T10:54:26.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.580+0000 7fef0bdbd640 1 -- 172.21.15.83:0/2876380772 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef040d02f0 con 0x7fef040c7c30 2024-04-17T10:54:26.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.584+0000 7fef027fc640 1 -- 172.21.15.83:0/2876380772 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7feefc056190 con 0x7fef040c7c30 2024-04-17T10:54:26.576 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:26.576 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:54:26.576 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:54:26.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.584+0000 7fef0bdbd640 1 -- 172.21.15.83:0/2876380772 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feef0066f90 msgr2=0x7feef0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:26.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.584+0000 7fef0bdbd640 1 --2- 172.21.15.83:0/2876380772 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feef0066f90 0x7feef0069450 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7feef40107c0 tx=0x7feef4008040 comp rx=0 tx=0).stop 2024-04-17T10:54:26.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.584+0000 7fef0bdbd640 1 -- 172.21.15.83:0/2876380772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef040c7c30 msgr2=0x7fef04133b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:26.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.584+0000 7fef0bdbd640 1 --2- 172.21.15.83:0/2876380772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef040c7c30 0x7fef04133b30 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7fef0406df20 tx=0x7feefc0115a0 comp rx=0 tx=0).stop 2024-04-17T10:54:26.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.584+0000 7fef0bdbd640 1 -- 172.21.15.83:0/2876380772 shutdown_connections 2024-04-17T10:54:26.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.584+0000 7fef0bdbd640 1 --2- 172.21.15.83:0/2876380772 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7feef0066f90 0x7feef0069450 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:26.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.584+0000 7fef0bdbd640 1 --2- 172.21.15.83:0/2876380772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef040c7c30 0x7fef04133b30 unknown :-1 s=CLOSED pgs=2392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:26.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.584+0000 7fef0bdbd640 1 --2- 172.21.15.83:0/2876380772 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef04145480 0x7fef040c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:26.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.584+0000 7fef0bdbd640 1 --2- 172.21.15.83:0/2876380772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef0415d790 0x7fef04144f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:26.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.584+0000 7fef0bdbd640 1 -- 172.21.15.83:0/2876380772 >> 172.21.15.83:0/2876380772 conn(0x7fef0406abe0 msgr2=0x7fef04160990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:26.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.584+0000 7fef0bdbd640 1 -- 172.21.15.83:0/2876380772 shutdown_connections 2024-04-17T10:54:26.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:26.584+0000 7fef0bdbd640 1 -- 172.21.15.83:0/2876380772 wait complete. 2024-04-17T10:54:26.998 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:54:26.998 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:54:27.999 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:54:28.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.140+0000 7f909a63d640 1 Processor -- start 2024-04-17T10:54:28.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.140+0000 7f909a63d640 1 -- start start 2024-04-17T10:54:28.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.140+0000 7f909a63d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f90940b3990 0x7f9094070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:28.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.140+0000 7f909a63d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f90940709b0 0x7f909406c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:28.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.140+0000 7f909a63d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9094070dc0 con 0x7f90940cf060 2024-04-17T10:54:28.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.140+0000 7f909a63d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90940b2980 con 0x7f90940709b0 2024-04-17T10:54:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.140+0000 7f909a63d640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90940cf440 con 0x7f90940b3990 2024-04-17T10:54:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.140+0000 7f9098bda640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f90940709b0 0x7f909406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.140+0000 7f9093fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f90940cf060 0x7f90940b4060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39306/0 (socket says 172.21.15.83:39306) 2024-04-17T10:54:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.140+0000 7f9093fff640 1 -- 172.21.15.83:0/3140647428 learned_addr learned my addr 172.21.15.83:0/3140647428 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.140+0000 7f9098bda640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f90940709b0 0x7f909406c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60080/0 (socket says 172.21.15.83:60080) 2024-04-17T10:54:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.140+0000 7f90937fe640 1 --2- 172.21.15.83:0/3140647428 >> v2:172.21.15.167:3300/0 conn(0x7f90940b3990 0x7f9094070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.140+0000 7f9098bda640 1 -- 172.21.15.83:0/3140647428 >> v2:172.21.15.167:3300/0 conn(0x7f90940b3990 msgr2=0x7f9094070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.140+0000 7f9098bda640 1 --2- 172.21.15.83:0/3140647428 >> v2:172.21.15.167:3300/0 conn(0x7f90940b3990 0x7f9094070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.140+0000 7f9098bda640 1 -- 172.21.15.83:0/3140647428 >> v1:172.21.15.83:6789/0 conn(0x7f90940cf060 legacy=0x7f90940b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.140+0000 7f9098bda640 1 -- 172.21.15.83:0/3140647428 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90940cf5c0 con 0x7f90940709b0 2024-04-17T10:54:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.140+0000 7f9092ffd640 1 -- 172.21.15.83:0/3140647428 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2405751390 0 0) 0x7f9088001120 con 0x7f90940cf060 2024-04-17T10:54:28.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.144+0000 7f9098bda640 1 --2- 172.21.15.83:0/3140647428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90940709b0 0x7f909406c490 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7f90940b2380 tx=0x7f908c01eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d277ba679b0240cb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:28.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.144+0000 7f9092ffd640 1 -- 172.21.15.83:0/3140647428 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c00b2d0 con 0x7f90940709b0 2024-04-17T10:54:28.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.144+0000 7f9092ffd640 1 -- 172.21.15.83:0/3140647428 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f908c0a7a00 con 0x7f90940709b0 2024-04-17T10:54:28.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.144+0000 7f9092ffd640 1 -- 172.21.15.83:0/3140647428 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c0a7dd0 con 0x7f90940709b0 2024-04-17T10:54:28.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.144+0000 7f909a63d640 1 -- 172.21.15.83:0/3140647428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90940709b0 msgr2=0x7f909406c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:28.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.144+0000 7f909a63d640 1 --2- 172.21.15.83:0/3140647428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90940709b0 0x7f909406c490 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7f90940b2380 tx=0x7f908c01eb30 comp rx=0 tx=0).stop 2024-04-17T10:54:28.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.144+0000 7f909a63d640 1 -- 172.21.15.83:0/3140647428 shutdown_connections 2024-04-17T10:54:28.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.144+0000 7f909a63d640 1 --2- 172.21.15.83:0/3140647428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90940709b0 0x7f909406c490 unknown :-1 s=CLOSED pgs=2394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:28.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.144+0000 7f909a63d640 1 --2- 172.21.15.83:0/3140647428 >> v2:172.21.15.167:3300/0 conn(0x7f90940b3990 0x7f9094070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:28.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.144+0000 7f909a63d640 1 -- 172.21.15.83:0/3140647428 >> 172.21.15.83:0/3140647428 conn(0x7f909406abe0 msgr2=0x7f909406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:28.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.144+0000 7f909a63d640 1 -- 172.21.15.83:0/3140647428 shutdown_connections 2024-04-17T10:54:28.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.144+0000 7f909a63d640 1 -- 172.21.15.83:0/3140647428 wait complete. 2024-04-17T10:54:28.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.144+0000 7f909a63d640 1 Processor -- start 2024-04-17T10:54:28.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.144+0000 7f909a63d640 1 -- start start 2024-04-17T10:54:28.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.148+0000 7f909a63d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90940709b0 0x7f9094146ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:28.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.148+0000 7f909a63d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90940b3990 0x7f9094163fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:28.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.148+0000 7f909a63d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90940cf060 0x7f90941644f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:28.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.148+0000 7f909a63d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90940b3470 con 0x7f90940b3990 2024-04-17T10:54:28.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.148+0000 7f909a63d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90940b3190 con 0x7f90940cf060 2024-04-17T10:54:28.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.148+0000 7f909a63d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f909415b310 con 0x7f90940709b0 2024-04-17T10:54:28.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.148+0000 7f90937fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90940b3990 0x7f9094163fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:28.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.148+0000 7f90937fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90940b3990 0x7f9094163fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60092/0 (socket says 172.21.15.83:60092) 2024-04-17T10:54:28.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.148+0000 7f90937fe640 1 -- 172.21.15.83:0/2483495937 learned_addr learned my addr 172.21.15.83:0/2483495937 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:28.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.148+0000 7f9093fff640 1 --2- 172.21.15.83:0/2483495937 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90940709b0 0x7f9094146ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:28.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.148+0000 7f9098bda640 1 --2- 172.21.15.83:0/2483495937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90940cf060 0x7f90941644f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:28.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.148+0000 7f90937fe640 1 -- 172.21.15.83:0/2483495937 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90940709b0 msgr2=0x7f9094146ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:28.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.148+0000 7f90937fe640 1 --2- 172.21.15.83:0/2483495937 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90940709b0 0x7f9094146ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:28.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.148+0000 7f90937fe640 1 -- 172.21.15.83:0/2483495937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90940cf060 msgr2=0x7f90941644f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:28.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.148+0000 7f90937fe640 1 --2- 172.21.15.83:0/2483495937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90940cf060 0x7f90941644f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:28.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.148+0000 7f90937fe640 1 -- 172.21.15.83:0/2483495937 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9094152ad0 con 0x7f90940b3990 2024-04-17T10:54:28.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.148+0000 7f9093fff640 1 --2- 172.21.15.83:0/2483495937 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90940709b0 0x7f9094146ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:28.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.148+0000 7f9098bda640 1 --2- 172.21.15.83:0/2483495937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90940cf060 0x7f90941644f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:28.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.148+0000 7f90937fe640 1 --2- 172.21.15.83:0/2483495937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90940b3990 0x7f9094163fb0 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7f908400d800 tx=0x7f908400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:28.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.148+0000 7f9090ff9640 1 -- 172.21.15.83:0/2483495937 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908400b6c0 con 0x7f90940b3990 2024-04-17T10:54:28.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.148+0000 7f9090ff9640 1 -- 172.21.15.83:0/2483495937 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9084019070 con 0x7f90940b3990 2024-04-17T10:54:28.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.148+0000 7f909a63d640 1 -- 172.21.15.83:0/2483495937 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9094152db0 con 0x7f90940b3990 2024-04-17T10:54:28.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.148+0000 7f9090ff9640 1 -- 172.21.15.83:0/2483495937 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908400bae0 con 0x7f90940b3990 2024-04-17T10:54:28.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.148+0000 7f909a63d640 1 -- 172.21.15.83:0/2483495937 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9094153370 con 0x7f90940b3990 2024-04-17T10:54:28.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.148+0000 7f909a63d640 1 -- 172.21.15.83:0/2483495937 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9094074f90 con 0x7f90940b3990 2024-04-17T10:54:28.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.148+0000 7f9090ff9640 1 -- 172.21.15.83:0/2483495937 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f908400bc80 con 0x7f90940b3990 2024-04-17T10:54:28.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.148+0000 7f9090ff9640 1 --2- 172.21.15.83:0/2483495937 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9080066f60 0x7f9080069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:28.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.148+0000 7f9090ff9640 1 -- 172.21.15.83:0/2483495937 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f9084085580 con 0x7f90940b3990 2024-04-17T10:54:28.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.148+0000 7f9093fff640 1 --2- 172.21.15.83:0/2483495937 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9080066f60 0x7f9080069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:28.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.148+0000 7f9093fff640 1 --2- 172.21.15.83:0/2483495937 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9080066f60 0x7f9080069420 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f9088007fd0 tx=0x7f9088009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:28.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.152+0000 7f9090ff9640 1 -- 172.21.15.83:0/2483495937 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9084056520 con 0x7f90940b3990 2024-04-17T10:54:28.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.348+0000 7f909a63d640 1 -- 172.21.15.83:0/2483495937 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9094151740 con 0x7f90940b3990 2024-04-17T10:54:28.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.348+0000 7f9090ff9640 1 -- 172.21.15.83:0/2483495937 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f9084055e20 con 0x7f90940b3990 2024-04-17T10:54:28.341 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:28.341 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:54:28.341 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:54:28.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.348+0000 7f909a63d640 1 -- 172.21.15.83:0/2483495937 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9080066f60 msgr2=0x7f9080069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:28.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.348+0000 7f909a63d640 1 --2- 172.21.15.83:0/2483495937 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9080066f60 0x7f9080069420 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f9088007fd0 tx=0x7f9088009040 comp rx=0 tx=0).stop 2024-04-17T10:54:28.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.348+0000 7f909a63d640 1 -- 172.21.15.83:0/2483495937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90940b3990 msgr2=0x7f9094163fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:28.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.348+0000 7f909a63d640 1 --2- 172.21.15.83:0/2483495937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90940b3990 0x7f9094163fb0 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7f908400d800 tx=0x7f908400dcc0 comp rx=0 tx=0).stop 2024-04-17T10:54:28.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.348+0000 7f909a63d640 1 -- 172.21.15.83:0/2483495937 shutdown_connections 2024-04-17T10:54:28.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.348+0000 7f909a63d640 1 --2- 172.21.15.83:0/2483495937 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9080066f60 0x7f9080069420 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:28.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.348+0000 7f909a63d640 1 --2- 172.21.15.83:0/2483495937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90940cf060 0x7f90941644f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:28.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.348+0000 7f909a63d640 1 --2- 172.21.15.83:0/2483495937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90940b3990 0x7f9094163fb0 unknown :-1 s=CLOSED pgs=2395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:28.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.348+0000 7f909a63d640 1 --2- 172.21.15.83:0/2483495937 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90940709b0 0x7f9094146ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:28.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.348+0000 7f909a63d640 1 -- 172.21.15.83:0/2483495937 >> 172.21.15.83:0/2483495937 conn(0x7f909406abe0 msgr2=0x7f90941593f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:28.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.348+0000 7f909a63d640 1 -- 172.21.15.83:0/2483495937 shutdown_connections 2024-04-17T10:54:28.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:28.348+0000 7f909a63d640 1 -- 172.21.15.83:0/2483495937 wait complete. 2024-04-17T10:54:28.761 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:54:28.761 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:54:29.762 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:54:29.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.895+0000 7f951af40640 1 Processor -- start 2024-04-17T10:54:29.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.899+0000 7f951af40640 1 -- start start 2024-04-17T10:54:29.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.899+0000 7f951af40640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f951415ba30 0x7f9514072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:29.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.899+0000 7f951af40640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f951406e320 con 0x7f95140cfbe0 2024-04-17T10:54:29.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.899+0000 7f951af40640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f951406e040 con 0x7f951415c960 2024-04-17T10:54:29.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.899+0000 7f951af40640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9514071ea0 con 0x7f951415ba30 2024-04-17T10:54:29.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.899+0000 7f95194dd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f95140cfbe0 0x7f951406c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39320/0 (socket says 172.21.15.83:39320) 2024-04-17T10:54:29.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.899+0000 7f95194dd640 1 -- 172.21.15.83:0/965315880 learned_addr learned my addr 172.21.15.83:0/965315880 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:29.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.899+0000 7f9518cdc640 1 --2- 172.21.15.83:0/965315880 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f951415ba30 0x7f9514072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:29.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.899+0000 7f95137fe640 1 -- 172.21.15.83:0/965315880 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2073596944 0 0) 0x7f951406e320 con 0x7f95140cfbe0 2024-04-17T10:54:29.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.899+0000 7f95137fe640 1 -- 172.21.15.83:0/965315880 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94fc003530 con 0x7f95140cfbe0 2024-04-17T10:54:29.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.899+0000 7f95137fe640 1 -- 172.21.15.83:0/965315880 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2703379978 0 0) 0x7f94fc003530 con 0x7f95140cfbe0 2024-04-17T10:54:29.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.899+0000 7f95137fe640 1 -- 172.21.15.83:0/965315880 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f951415ba30 msgr2=0x7f9514072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:29.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.899+0000 7f95137fe640 1 --2- 172.21.15.83:0/965315880 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f951415ba30 0x7f9514072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:29.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.899+0000 7f95137fe640 1 -- 172.21.15.83:0/965315880 >> v1:172.21.15.167:6789/0 conn(0x7f951415c960 legacy=0x7f95140cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:29.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.899+0000 7f95137fe640 1 -- 172.21.15.83:0/965315880 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9514071850 con 0x7f95140cfbe0 2024-04-17T10:54:29.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.899+0000 7f95137fe640 1 -- 172.21.15.83:0/965315880 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f950c001a20 con 0x7f95140cfbe0 2024-04-17T10:54:29.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.899+0000 7f9518cdc640 1 --2- 172.21.15.83:0/965315880 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f951415ba30 0x7f9514072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:54:29.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.899+0000 7f95137fe640 1 -- 172.21.15.83:0/965315880 >> v1:172.21.15.83:6789/0 conn(0x7f95140cfbe0 legacy=0x7f951406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:29.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.899+0000 7f95137fe640 1 --2- 172.21.15.83:0/965315880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94fc0039f0 0x7f94fc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:29.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.899+0000 7f95137fe640 1 --2- 172.21.15.83:0/965315880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94fc004810 0x7f94fc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:29.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.899+0000 7f95137fe640 1 --2- 172.21.15.83:0/965315880 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f951415ba30 0x7f94fc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:29.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.899+0000 7f95137fe640 1 -- 172.21.15.83:0/965315880 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9514071ea0 con 0x7f94fc0039f0 2024-04-17T10:54:29.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.899+0000 7f95137fe640 1 -- 172.21.15.83:0/965315880 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc0054f0 con 0x7f94fc004810 2024-04-17T10:54:29.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.899+0000 7f95137fe640 1 -- 172.21.15.83:0/965315880 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc00bf10 con 0x7f951415ba30 2024-04-17T10:54:29.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.899+0000 7f9518cdc640 1 --2- 172.21.15.83:0/965315880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94fc0039f0 0x7f94fc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:29.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.899+0000 7f95194dd640 1 --2- 172.21.15.83:0/965315880 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f951415ba30 0x7f94fc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:29.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.899+0000 7f9513fff640 1 --2- 172.21.15.83:0/965315880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94fc004810 0x7f94fc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:29.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.899+0000 7f9518cdc640 1 -- 172.21.15.83:0/965315880 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f951415ba30 msgr2=0x7f94fc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:29.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.899+0000 7f9518cdc640 1 --2- 172.21.15.83:0/965315880 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f951415ba30 0x7f94fc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:29.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.899+0000 7f9518cdc640 1 -- 172.21.15.83:0/965315880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94fc004810 msgr2=0x7f94fc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:29.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.899+0000 7f9518cdc640 1 --2- 172.21.15.83:0/965315880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94fc004810 0x7f94fc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:29.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.899+0000 7f9518cdc640 1 -- 172.21.15.83:0/965315880 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94fc003530 con 0x7f94fc0039f0 2024-04-17T10:54:29.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.903+0000 7f9518cdc640 1 --2- 172.21.15.83:0/965315880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94fc0039f0 0x7f94fc003e10 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f95080098d0 tx=0x7f951406d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4009de9b0fc4f217 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:29.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.903+0000 7f95137fe640 1 -- 172.21.15.83:0/965315880 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950800f040 con 0x7f94fc0039f0 2024-04-17T10:54:29.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.903+0000 7f95137fe640 1 -- 172.21.15.83:0/965315880 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95080b0070 con 0x7f94fc0039f0 2024-04-17T10:54:29.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.903+0000 7f95137fe640 1 -- 172.21.15.83:0/965315880 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95080ab800 con 0x7f94fc0039f0 2024-04-17T10:54:29.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.903+0000 7f951af40640 1 -- 172.21.15.83:0/965315880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94fc0039f0 msgr2=0x7f94fc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:29.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.903+0000 7f951af40640 1 --2- 172.21.15.83:0/965315880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94fc0039f0 0x7f94fc003e10 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f95080098d0 tx=0x7f951406d3b0 comp rx=0 tx=0).stop 2024-04-17T10:54:29.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.903+0000 7f95194dd640 1 -- 172.21.15.83:0/965315880 reap_dead start 2024-04-17T10:54:29.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.903+0000 7f951af40640 1 -- 172.21.15.83:0/965315880 shutdown_connections 2024-04-17T10:54:29.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.903+0000 7f951af40640 1 -- 172.21.15.83:0/965315880 >> 172.21.15.83:0/965315880 conn(0x7f951406abe0 msgr2=0x7f951406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:29.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.903+0000 7f951af40640 1 -- 172.21.15.83:0/965315880 shutdown_connections 2024-04-17T10:54:29.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.903+0000 7f951af40640 1 -- 172.21.15.83:0/965315880 wait complete. 2024-04-17T10:54:29.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.903+0000 7f951af40640 1 Processor -- start 2024-04-17T10:54:29.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.903+0000 7f951af40640 1 -- start start 2024-04-17T10:54:29.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.903+0000 7f951af40640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94fc0039f0 0x7f95141561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:29.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.903+0000 7f951af40640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f95140b2770 0x7f95141f6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:29.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.903+0000 7f951af40640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95140b0460 0x7f95140b0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:29.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.903+0000 7f951af40640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951415cec0 con 0x7f94fc0039f0 2024-04-17T10:54:29.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.903+0000 7f951af40640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951415cd40 con 0x7f95140b0460 2024-04-17T10:54:29.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.903+0000 7f951af40640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9514167520 con 0x7f95140b2770 2024-04-17T10:54:29.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.903+0000 7f9518cdc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94fc0039f0 0x7f95141561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:29.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.903+0000 7f9518cdc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94fc0039f0 0x7f95141561f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60118/0 (socket says 172.21.15.83:60118) 2024-04-17T10:54:29.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.903+0000 7f9518cdc640 1 -- 172.21.15.83:0/3708562813 learned_addr learned my addr 172.21.15.83:0/3708562813 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:29.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.903+0000 7f9513fff640 1 --2- 172.21.15.83:0/3708562813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f95140b2770 0x7f95141f6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:29.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.903+0000 7f9518cdc640 1 -- 172.21.15.83:0/3708562813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f95140b2770 msgr2=0x7f95141f6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:29.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.903+0000 7f9518cdc640 1 --2- 172.21.15.83:0/3708562813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f95140b2770 0x7f95141f6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:29.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.903+0000 7f9518cdc640 1 -- 172.21.15.83:0/3708562813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95140b0460 msgr2=0x7f95140b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:29.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.903+0000 7f95194dd640 1 --2- 172.21.15.83:0/3708562813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95140b0460 0x7f95140b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:29.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.907+0000 7f9518cdc640 1 --2- 172.21.15.83:0/3708562813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95140b0460 0x7f95140b0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:29.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.907+0000 7f9518cdc640 1 -- 172.21.15.83:0/3708562813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95140b2c00 con 0x7f94fc0039f0 2024-04-17T10:54:29.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.907+0000 7f9518cdc640 1 --2- 172.21.15.83:0/3708562813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94fc0039f0 0x7f95141561f0 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7f950801c400 tx=0x7f94fc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:29.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.907+0000 7f95117fa640 1 -- 172.21.15.83:0/3708562813 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95080b5780 con 0x7f94fc0039f0 2024-04-17T10:54:29.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.907+0000 7f95117fa640 1 -- 172.21.15.83:0/3708562813 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95080b5940 con 0x7f94fc0039f0 2024-04-17T10:54:29.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.907+0000 7f951af40640 1 -- 172.21.15.83:0/3708562813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95140b2ee0 con 0x7f94fc0039f0 2024-04-17T10:54:29.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.907+0000 7f95117fa640 1 -- 172.21.15.83:0/3708562813 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95080b5d30 con 0x7f94fc0039f0 2024-04-17T10:54:29.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.907+0000 7f951af40640 1 -- 172.21.15.83:0/3708562813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95140b34a0 con 0x7f94fc0039f0 2024-04-17T10:54:29.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.907+0000 7f951af40640 1 -- 172.21.15.83:0/3708562813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9514074f90 con 0x7f94fc0039f0 2024-04-17T10:54:29.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.907+0000 7f95117fa640 1 -- 172.21.15.83:0/3708562813 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f95080c6060 con 0x7f94fc0039f0 2024-04-17T10:54:29.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.907+0000 7f95117fa640 1 --2- 172.21.15.83:0/3708562813 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9500066f90 0x7f9500069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:29.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.907+0000 7f95117fa640 1 -- 172.21.15.83:0/3708562813 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f950811ecb0 con 0x7f94fc0039f0 2024-04-17T10:54:29.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.907+0000 7f9513fff640 1 --2- 172.21.15.83:0/3708562813 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9500066f90 0x7f9500069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:29.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.907+0000 7f9513fff640 1 --2- 172.21.15.83:0/3708562813 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9500066f90 0x7f9500069450 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f9504000c10 tx=0x7f9504009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:29.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:29.911+0000 7f95117fa640 1 -- 172.21.15.83:0/3708562813 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f95080ba070 con 0x7f94fc0039f0 2024-04-17T10:54:30.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:30.103+0000 7f951af40640 1 -- 172.21.15.83:0/3708562813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95141501c0 con 0x7f94fc0039f0 2024-04-17T10:54:30.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:30.107+0000 7f95117fa640 1 -- 172.21.15.83:0/3708562813 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f95080162e0 con 0x7f94fc0039f0 2024-04-17T10:54:30.099 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:30.100 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:54:30.100 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:54:30.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:30.107+0000 7f951af40640 1 -- 172.21.15.83:0/3708562813 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9500066f90 msgr2=0x7f9500069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:30.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:30.107+0000 7f951af40640 1 --2- 172.21.15.83:0/3708562813 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9500066f90 0x7f9500069450 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f9504000c10 tx=0x7f9504009040 comp rx=0 tx=0).stop 2024-04-17T10:54:30.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:30.107+0000 7f951af40640 1 -- 172.21.15.83:0/3708562813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94fc0039f0 msgr2=0x7f95141561f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:30.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:30.107+0000 7f951af40640 1 --2- 172.21.15.83:0/3708562813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94fc0039f0 0x7f95141561f0 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7f950801c400 tx=0x7f94fc005840 comp rx=0 tx=0).stop 2024-04-17T10:54:30.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:30.107+0000 7f951af40640 1 -- 172.21.15.83:0/3708562813 shutdown_connections 2024-04-17T10:54:30.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:30.107+0000 7f951af40640 1 --2- 172.21.15.83:0/3708562813 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9500066f90 0x7f9500069450 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:30.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:30.107+0000 7f951af40640 1 --2- 172.21.15.83:0/3708562813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95140b0460 0x7f95140b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:30.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:30.107+0000 7f951af40640 1 --2- 172.21.15.83:0/3708562813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f95140b2770 0x7f95141f6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:30.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:30.107+0000 7f951af40640 1 --2- 172.21.15.83:0/3708562813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94fc0039f0 0x7f95141561f0 unknown :-1 s=CLOSED pgs=2398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:30.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:30.107+0000 7f951af40640 1 -- 172.21.15.83:0/3708562813 >> 172.21.15.83:0/3708562813 conn(0x7f951406abe0 msgr2=0x7f951415fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:30.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:30.107+0000 7f951af40640 1 -- 172.21.15.83:0/3708562813 shutdown_connections 2024-04-17T10:54:30.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:30.107+0000 7f951af40640 1 -- 172.21.15.83:0/3708562813 wait complete. 2024-04-17T10:54:30.520 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:54:30.520 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:54:31.521 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:54:31.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.659+0000 7faad7a4f640 1 Processor -- start 2024-04-17T10:54:31.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.659+0000 7faad7a4f640 1 -- start start 2024-04-17T10:54:31.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.659+0000 7faad7a4f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faad00aedd0 0x7faad006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:31.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.659+0000 7faad7a4f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faad006dae0 0x7faad00b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:31.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.659+0000 7faad7a4f640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faad015e440 con 0x7faad015db50 2024-04-17T10:54:31.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.659+0000 7faad7a4f640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faad015e720 con 0x7faad006dae0 2024-04-17T10:54:31.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.659+0000 7faad7a4f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faad015ea00 con 0x7faad00aedd0 2024-04-17T10:54:31.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.659+0000 7faad5fec640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faad006dae0 0x7faad00b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:31.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.659+0000 7faad5fec640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faad006dae0 0x7faad00b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60122/0 (socket says 172.21.15.83:60122) 2024-04-17T10:54:31.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.659+0000 7faad5fec640 1 -- 172.21.15.83:0/907992507 learned_addr learned my addr 172.21.15.83:0/907992507 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:31.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.659+0000 7faad4fea640 1 --2- 172.21.15.83:0/907992507 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faad00aedd0 0x7faad006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:31.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.659+0000 7faad5fec640 1 -- 172.21.15.83:0/907992507 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faad00aedd0 msgr2=0x7faad006d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:31.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.659+0000 7faad5fec640 1 --2- 172.21.15.83:0/907992507 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faad00aedd0 0x7faad006d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:31.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.659+0000 7faad5fec640 1 -- 172.21.15.83:0/907992507 >> v1:172.21.15.167:6789/0 conn(0x7faad015db50 legacy=0x7faad0070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:31.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.659+0000 7faad5fec640 1 -- 172.21.15.83:0/907992507 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad015ece0 con 0x7faad006dae0 2024-04-17T10:54:31.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.663+0000 7faad5fec640 1 --2- 172.21.15.83:0/907992507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faad006dae0 0x7faad00b2e10 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7faac800d150 tx=0x7faac801ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67ef4c297f6064a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:31.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.663+0000 7faac7fff640 1 -- 172.21.15.83:0/907992507 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac800b2d0 con 0x7faad006dae0 2024-04-17T10:54:31.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faac7fff640 1 -- 172.21.15.83:0/907992507 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faac800b470 con 0x7faad006dae0 2024-04-17T10:54:31.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faac7fff640 1 -- 172.21.15.83:0/907992507 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac80a7b20 con 0x7faad006dae0 2024-04-17T10:54:31.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faad7a4f640 1 -- 172.21.15.83:0/907992507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faad006dae0 msgr2=0x7faad00b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:31.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faad7a4f640 1 --2- 172.21.15.83:0/907992507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faad006dae0 0x7faad00b2e10 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7faac800d150 tx=0x7faac801ec30 comp rx=0 tx=0).stop 2024-04-17T10:54:31.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faad7a4f640 1 -- 172.21.15.83:0/907992507 shutdown_connections 2024-04-17T10:54:31.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faad7a4f640 1 --2- 172.21.15.83:0/907992507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faad006dae0 0x7faad00b2e10 unknown :-1 s=CLOSED pgs=2399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:31.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faad7a4f640 1 --2- 172.21.15.83:0/907992507 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faad00aedd0 0x7faad006d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:31.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faad7a4f640 1 -- 172.21.15.83:0/907992507 >> 172.21.15.83:0/907992507 conn(0x7faad006abe0 msgr2=0x7faad006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:31.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faad7a4f640 1 -- 172.21.15.83:0/907992507 shutdown_connections 2024-04-17T10:54:31.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faad7a4f640 1 -- 172.21.15.83:0/907992507 wait complete. 2024-04-17T10:54:31.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faad7a4f640 1 Processor -- start 2024-04-17T10:54:31.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faad7a4f640 1 -- start start 2024-04-17T10:54:31.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faad7a4f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faad006dae0 0x7faad0150230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:31.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faad7a4f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faad00aedd0 0x7faad0153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:31.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faad7a4f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faad015db50 0x7faad014e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:31.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faad7a4f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad00b3ef0 con 0x7faad006dae0 2024-04-17T10:54:31.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faad7a4f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad00b3d70 con 0x7faad00aedd0 2024-04-17T10:54:31.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faad7a4f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faad00b4070 con 0x7faad015db50 2024-04-17T10:54:31.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faad57eb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faad006dae0 0x7faad0150230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:31.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faad57eb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faad006dae0 0x7faad0150230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60128/0 (socket says 172.21.15.83:60128) 2024-04-17T10:54:31.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faad57eb640 1 -- 172.21.15.83:0/2033748647 learned_addr learned my addr 172.21.15.83:0/2033748647 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:31.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faad4fea640 1 --2- 172.21.15.83:0/2033748647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faad00aedd0 0x7faad0153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:31.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faad5fec640 1 --2- 172.21.15.83:0/2033748647 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faad015db50 0x7faad014e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:31.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faad57eb640 1 -- 172.21.15.83:0/2033748647 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faad015db50 msgr2=0x7faad014e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:31.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faad57eb640 1 --2- 172.21.15.83:0/2033748647 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faad015db50 0x7faad014e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:31.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faad57eb640 1 -- 172.21.15.83:0/2033748647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faad00aedd0 msgr2=0x7faad0153da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:31.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faad57eb640 1 --2- 172.21.15.83:0/2033748647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faad00aedd0 0x7faad0153da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:31.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faad57eb640 1 -- 172.21.15.83:0/2033748647 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad0150b20 con 0x7faad006dae0 2024-04-17T10:54:31.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faad4fea640 1 --2- 172.21.15.83:0/2033748647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faad00aedd0 0x7faad0153da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:31.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faad5fec640 1 --2- 172.21.15.83:0/2033748647 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faad015db50 0x7faad014e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:31.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faad57eb640 1 --2- 172.21.15.83:0/2033748647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faad006dae0 0x7faad0150230 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7faacc00c370 tx=0x7faacc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:31.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faac5ffb640 1 -- 172.21.15.83:0/2033748647 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faacc0052d0 con 0x7faad006dae0 2024-04-17T10:54:31.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faac5ffb640 1 -- 172.21.15.83:0/2033748647 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faacc005470 con 0x7faad006dae0 2024-04-17T10:54:31.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faad7a4f640 1 -- 172.21.15.83:0/2033748647 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faad0150de0 con 0x7faad006dae0 2024-04-17T10:54:31.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faac5ffb640 1 -- 172.21.15.83:0/2033748647 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faacc007260 con 0x7faad006dae0 2024-04-17T10:54:31.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.667+0000 7faad7a4f640 1 -- 172.21.15.83:0/2033748647 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faad0157230 con 0x7faad006dae0 2024-04-17T10:54:31.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.671+0000 7faad7a4f640 1 -- 172.21.15.83:0/2033748647 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faad0074f90 con 0x7faad006dae0 2024-04-17T10:54:31.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.671+0000 7faac5ffb640 1 -- 172.21.15.83:0/2033748647 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faacc007400 con 0x7faad006dae0 2024-04-17T10:54:31.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.671+0000 7faac5ffb640 1 --2- 172.21.15.83:0/2033748647 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faab4066f60 0x7faab4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:31.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.671+0000 7faac5ffb640 1 -- 172.21.15.83:0/2033748647 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7faacc084500 con 0x7faad006dae0 2024-04-17T10:54:31.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.671+0000 7faad4fea640 1 --2- 172.21.15.83:0/2033748647 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faab4066f60 0x7faab4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:31.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.671+0000 7faad4fea640 1 --2- 172.21.15.83:0/2033748647 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faab4066f60 0x7faab4069420 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7faad00cde00 tx=0x7faac0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:31.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.675+0000 7faac5ffb640 1 -- 172.21.15.83:0/2033748647 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faacc0554a0 con 0x7faad006dae0 2024-04-17T10:54:31.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.863+0000 7faad7a4f640 1 -- 172.21.15.83:0/2033748647 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faad00b2af0 con 0x7faad006dae0 2024-04-17T10:54:31.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.863+0000 7faac5ffb640 1 -- 172.21.15.83:0/2033748647 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7faacc054da0 con 0x7faad006dae0 2024-04-17T10:54:31.859 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:31.859 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:54:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:54:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.867+0000 7faad7a4f640 1 -- 172.21.15.83:0/2033748647 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faab4066f60 msgr2=0x7faab4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.867+0000 7faad7a4f640 1 --2- 172.21.15.83:0/2033748647 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faab4066f60 0x7faab4069420 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7faad00cde00 tx=0x7faac0008040 comp rx=0 tx=0).stop 2024-04-17T10:54:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.867+0000 7faad7a4f640 1 -- 172.21.15.83:0/2033748647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faad006dae0 msgr2=0x7faad0150230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.867+0000 7faad7a4f640 1 --2- 172.21.15.83:0/2033748647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faad006dae0 0x7faad0150230 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7faacc00c370 tx=0x7faacc00c830 comp rx=0 tx=0).stop 2024-04-17T10:54:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.867+0000 7faad7a4f640 1 -- 172.21.15.83:0/2033748647 shutdown_connections 2024-04-17T10:54:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.867+0000 7faad7a4f640 1 --2- 172.21.15.83:0/2033748647 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faab4066f60 0x7faab4069420 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.867+0000 7faad7a4f640 1 --2- 172.21.15.83:0/2033748647 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faad015db50 0x7faad014e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.867+0000 7faad7a4f640 1 --2- 172.21.15.83:0/2033748647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faad00aedd0 0x7faad0153da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.867+0000 7faad7a4f640 1 --2- 172.21.15.83:0/2033748647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faad006dae0 0x7faad0150230 unknown :-1 s=CLOSED pgs=2400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.867+0000 7faad7a4f640 1 -- 172.21.15.83:0/2033748647 >> 172.21.15.83:0/2033748647 conn(0x7faad006abe0 msgr2=0x7faad00af650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.867+0000 7faad7a4f640 1 -- 172.21.15.83:0/2033748647 shutdown_connections 2024-04-17T10:54:31.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:31.867+0000 7faad7a4f640 1 -- 172.21.15.83:0/2033748647 wait complete. 2024-04-17T10:54:32.278 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:54:32.278 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:54:33.280 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:54:33.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.415+0000 7fe3a8c6a640 1 Processor -- start 2024-04-17T10:54:33.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.415+0000 7fe3a8c6a640 1 -- start start 2024-04-17T10:54:33.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.419+0000 7fe3a8c6a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe3a415ba30 0x7fe3a4072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:33.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.419+0000 7fe3a8c6a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3a40cfbe0 0x7fe3a406c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:33.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.419+0000 7fe3a8c6a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a406e320 con 0x7fe3a415c960 2024-04-17T10:54:33.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.419+0000 7fe3a8c6a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a4071ea0 con 0x7fe3a415ba30 2024-04-17T10:54:33.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.419+0000 7fe3a8c6a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a406e040 con 0x7fe3a40cfbe0 2024-04-17T10:54:33.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.419+0000 7fe3a259c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe3a415ba30 0x7fe3a4072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:33.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.419+0000 7fe3a259c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe3a415ba30 0x7fe3a4072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60130/0 (socket says 172.21.15.83:60130) 2024-04-17T10:54:33.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.419+0000 7fe3a259c640 1 -- 172.21.15.83:0/1717426176 learned_addr learned my addr 172.21.15.83:0/1717426176 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:33.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.419+0000 7fe3a1d9b640 1 --1- 172.21.15.83:0/1717426176 >> v1:172.21.15.83:6789/0 conn(0x7fe3a415c960 0x7fe3a40cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39332/0 (socket says 172.21.15.83:39332) 2024-04-17T10:54:33.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.419+0000 7fe3a259c640 1 -- 172.21.15.83:0/1717426176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3a40cfbe0 msgr2=0x7fe3a406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:33.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.419+0000 7fe3a2d9d640 1 --2- 172.21.15.83:0/1717426176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3a40cfbe0 0x7fe3a406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:33.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.419+0000 7fe3a159a640 1 -- 172.21.15.83:0/1717426176 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2235290450 0 0) 0x7fe3a406e320 con 0x7fe3a415c960 2024-04-17T10:54:33.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.419+0000 7fe3a259c640 1 --2- 172.21.15.83:0/1717426176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3a40cfbe0 0x7fe3a406c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:33.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.419+0000 7fe3a259c640 1 -- 172.21.15.83:0/1717426176 >> v1:172.21.15.83:6789/0 conn(0x7fe3a415c960 legacy=0x7fe3a40cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:33.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.419+0000 7fe3a259c640 1 -- 172.21.15.83:0/1717426176 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a4071850 con 0x7fe3a415ba30 2024-04-17T10:54:33.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.419+0000 7fe3a259c640 1 --2- 172.21.15.83:0/1717426176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3a415ba30 0x7fe3a4072330 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7fe394009e50 tx=0x7fe39401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaef011b36aae5ec server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:33.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.419+0000 7fe3a159a640 1 -- 172.21.15.83:0/1717426176 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3940a56b0 con 0x7fe3a415ba30 2024-04-17T10:54:33.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.419+0000 7fe3a159a640 1 -- 172.21.15.83:0/1717426176 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3940a5850 con 0x7fe3a415ba30 2024-04-17T10:54:33.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.419+0000 7fe3a159a640 1 -- 172.21.15.83:0/1717426176 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3940a5c20 con 0x7fe3a415ba30 2024-04-17T10:54:33.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.419+0000 7fe3a8c6a640 1 -- 172.21.15.83:0/1717426176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3a415ba30 msgr2=0x7fe3a4072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:33.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.419+0000 7fe3a8c6a640 1 --2- 172.21.15.83:0/1717426176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3a415ba30 0x7fe3a4072330 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7fe394009e50 tx=0x7fe39401c9a0 comp rx=0 tx=0).stop 2024-04-17T10:54:33.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.419+0000 7fe3a8c6a640 1 -- 172.21.15.83:0/1717426176 shutdown_connections 2024-04-17T10:54:33.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.419+0000 7fe3a8c6a640 1 --2- 172.21.15.83:0/1717426176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3a40cfbe0 0x7fe3a406c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:33.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.419+0000 7fe3a8c6a640 1 --2- 172.21.15.83:0/1717426176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3a415ba30 0x7fe3a4072330 unknown :-1 s=CLOSED pgs=2402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:33.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.419+0000 7fe3a8c6a640 1 -- 172.21.15.83:0/1717426176 >> 172.21.15.83:0/1717426176 conn(0x7fe3a406abe0 msgr2=0x7fe3a406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:33.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.419+0000 7fe3a8c6a640 1 -- 172.21.15.83:0/1717426176 shutdown_connections 2024-04-17T10:54:33.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.419+0000 7fe3a8c6a640 1 -- 172.21.15.83:0/1717426176 wait complete. 2024-04-17T10:54:33.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.423+0000 7fe3a8c6a640 1 Processor -- start 2024-04-17T10:54:33.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.423+0000 7fe3a8c6a640 1 -- start start 2024-04-17T10:54:33.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.423+0000 7fe3a8c6a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3a40cfbe0 0x7fe3a414d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:33.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.423+0000 7fe3a8c6a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3a415ba30 0x7fe3a41551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:33.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.423+0000 7fe3a8c6a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3a415c960 0x7fe3a4155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:33.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.423+0000 7fe3a8c6a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a406de20 con 0x7fe3a40cfbe0 2024-04-17T10:54:33.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.423+0000 7fe3a8c6a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a406dca0 con 0x7fe3a415c960 2024-04-17T10:54:33.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.423+0000 7fe3a8c6a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a4168520 con 0x7fe3a415ba30 2024-04-17T10:54:33.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.423+0000 7fe3a259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3a40cfbe0 0x7fe3a414d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:33.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.423+0000 7fe3a259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3a40cfbe0 0x7fe3a414d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60136/0 (socket says 172.21.15.83:60136) 2024-04-17T10:54:33.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.423+0000 7fe3a259c640 1 -- 172.21.15.83:0/3856553954 learned_addr learned my addr 172.21.15.83:0/3856553954 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:33.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.423+0000 7fe3a1d9b640 1 --2- 172.21.15.83:0/3856553954 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3a415ba30 0x7fe3a41551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:33.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.423+0000 7fe3a259c640 1 -- 172.21.15.83:0/3856553954 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3a415ba30 msgr2=0x7fe3a41551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:33.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.423+0000 7fe3a259c640 1 --2- 172.21.15.83:0/3856553954 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3a415ba30 0x7fe3a41551e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:33.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.423+0000 7fe3a259c640 1 -- 172.21.15.83:0/3856553954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3a415c960 msgr2=0x7fe3a4155720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:33.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.423+0000 7fe3a259c640 1 --2- 172.21.15.83:0/3856553954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3a415c960 0x7fe3a4155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:33.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.423+0000 7fe3a259c640 1 -- 172.21.15.83:0/3856553954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a414f430 con 0x7fe3a40cfbe0 2024-04-17T10:54:33.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.423+0000 7fe3a259c640 1 --2- 172.21.15.83:0/3856553954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3a40cfbe0 0x7fe3a414d520 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7fe394009e50 tx=0x7fe394019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:33.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.423+0000 7fe392ffd640 1 -- 172.21.15.83:0/3856553954 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3940bc020 con 0x7fe3a40cfbe0 2024-04-17T10:54:33.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.423+0000 7fe392ffd640 1 -- 172.21.15.83:0/3856553954 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3940b8670 con 0x7fe3a40cfbe0 2024-04-17T10:54:33.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.423+0000 7fe3a8c6a640 1 -- 172.21.15.83:0/3856553954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a4159390 con 0x7fe3a40cfbe0 2024-04-17T10:54:33.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.423+0000 7fe392ffd640 1 -- 172.21.15.83:0/3856553954 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3940b8a20 con 0x7fe3a40cfbe0 2024-04-17T10:54:33.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.423+0000 7fe3a8c6a640 1 -- 172.21.15.83:0/3856553954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a4159950 con 0x7fe3a40cfbe0 2024-04-17T10:54:33.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.423+0000 7fe3a8c6a640 1 -- 172.21.15.83:0/3856553954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a4074f90 con 0x7fe3a40cfbe0 2024-04-17T10:54:33.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.427+0000 7fe392ffd640 1 -- 172.21.15.83:0/3856553954 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe3940bf020 con 0x7fe3a40cfbe0 2024-04-17T10:54:33.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.427+0000 7fe392ffd640 1 --2- 172.21.15.83:0/3856553954 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe384066f60 0x7fe384069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:33.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.427+0000 7fe392ffd640 1 -- 172.21.15.83:0/3856553954 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fe394120100 con 0x7fe3a40cfbe0 2024-04-17T10:54:33.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.427+0000 7fe3a1d9b640 1 --2- 172.21.15.83:0/3856553954 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe384066f60 0x7fe384069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:33.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.427+0000 7fe3a1d9b640 1 --2- 172.21.15.83:0/3856553954 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe384066f60 0x7fe384069420 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fe3a40cd6d0 tx=0x7fe398009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.431+0000 7fe392ffd640 1 -- 172.21.15.83:0/3856553954 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe3940c6030 con 0x7fe3a40cfbe0 2024-04-17T10:54:33.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.611+0000 7fe3a8c6a640 1 -- 172.21.15.83:0/3856553954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3a415ddf0 con 0x7fe3a40cfbe0 2024-04-17T10:54:33.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.611+0000 7fe392ffd640 1 -- 172.21.15.83:0/3856553954 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fe394016620 con 0x7fe3a40cfbe0 2024-04-17T10:54:33.606 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:33.607 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:54:33.607 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:54:33.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.615+0000 7fe3a8c6a640 1 -- 172.21.15.83:0/3856553954 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe384066f60 msgr2=0x7fe384069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:33.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.615+0000 7fe3a8c6a640 1 --2- 172.21.15.83:0/3856553954 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe384066f60 0x7fe384069420 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fe3a40cd6d0 tx=0x7fe398009040 comp rx=0 tx=0).stop 2024-04-17T10:54:33.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.615+0000 7fe3a8c6a640 1 -- 172.21.15.83:0/3856553954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3a40cfbe0 msgr2=0x7fe3a414d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:33.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.615+0000 7fe3a8c6a640 1 --2- 172.21.15.83:0/3856553954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3a40cfbe0 0x7fe3a414d520 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7fe394009e50 tx=0x7fe394019740 comp rx=0 tx=0).stop 2024-04-17T10:54:33.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.615+0000 7fe3a8c6a640 1 -- 172.21.15.83:0/3856553954 shutdown_connections 2024-04-17T10:54:33.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.615+0000 7fe3a8c6a640 1 --2- 172.21.15.83:0/3856553954 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe384066f60 0x7fe384069420 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:33.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.615+0000 7fe3a8c6a640 1 --2- 172.21.15.83:0/3856553954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3a415c960 0x7fe3a4155720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:33.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.615+0000 7fe3a8c6a640 1 --2- 172.21.15.83:0/3856553954 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3a415ba30 0x7fe3a41551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:33.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.615+0000 7fe3a8c6a640 1 --2- 172.21.15.83:0/3856553954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3a40cfbe0 0x7fe3a414d520 unknown :-1 s=CLOSED pgs=2403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:33.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.615+0000 7fe3a8c6a640 1 -- 172.21.15.83:0/3856553954 >> 172.21.15.83:0/3856553954 conn(0x7fe3a406abe0 msgr2=0x7fe3a415e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:33.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.615+0000 7fe3a8c6a640 1 -- 172.21.15.83:0/3856553954 shutdown_connections 2024-04-17T10:54:33.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:33.615+0000 7fe3a8c6a640 1 -- 172.21.15.83:0/3856553954 wait complete. 2024-04-17T10:54:34.028 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:54:34.028 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:54:35.029 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:54:35.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.163+0000 7fafb9627640 1 Processor -- start 2024-04-17T10:54:35.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.167+0000 7fafb9627640 1 -- start start 2024-04-17T10:54:35.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.167+0000 7fafb9627640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fafb415dfa0 0x7fafb40b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:35.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.167+0000 7fafb9627640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafb4162a00 con 0x7fafb40ce980 2024-04-17T10:54:35.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.167+0000 7fafb9627640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafb4161e80 con 0x7fafb40b5b40 2024-04-17T10:54:35.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.167+0000 7fafb9627640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafb4162160 con 0x7fafb415dfa0 2024-04-17T10:54:35.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.167+0000 7fafb2ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fafb415dfa0 0x7fafb40b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:35.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.167+0000 7fafb27fc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fafb40ce980 0x7fafb40b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39342/0 (socket says 172.21.15.83:39342) 2024-04-17T10:54:35.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.167+0000 7fafb27fc640 1 -- 172.21.15.83:0/3458953405 learned_addr learned my addr 172.21.15.83:0/3458953405 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:35.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.167+0000 7fafb2ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fafb415dfa0 0x7fafb40b6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60150/0 (socket says 172.21.15.83:60150) 2024-04-17T10:54:35.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.167+0000 7fafb2ffd640 1 -- 172.21.15.83:0/3458953405 >> v1:172.21.15.167:6789/0 conn(0x7fafb40b5b40 legacy=0x7fafb40d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:35.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.167+0000 7fafb2ffd640 1 -- 172.21.15.83:0/3458953405 >> v1:172.21.15.83:6789/0 conn(0x7fafb40ce980 legacy=0x7fafb40b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:35.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.167+0000 7fafb1ffb640 1 -- 172.21.15.83:0/3458953405 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1367147998 0 0) 0x7fafb4162a00 con 0x7fafb40ce980 2024-04-17T10:54:35.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.167+0000 7fafb2ffd640 1 -- 172.21.15.83:0/3458953405 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb4162440 con 0x7fafb415dfa0 2024-04-17T10:54:35.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.167+0000 7fafb2ffd640 1 --2- 172.21.15.83:0/3458953405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafb415dfa0 0x7fafb40b6c20 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7fafa40097c0 tx=0x7fafa401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c095d8125901369 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:35.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.167+0000 7fafb1ffb640 1 -- 172.21.15.83:0/3458953405 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa40b0070 con 0x7fafb415dfa0 2024-04-17T10:54:35.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7fafb1ffb640 1 -- 172.21.15.83:0/3458953405 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa40ab600 con 0x7fafb415dfa0 2024-04-17T10:54:35.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7fafb1ffb640 1 -- 172.21.15.83:0/3458953405 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa40ab9d0 con 0x7fafb415dfa0 2024-04-17T10:54:35.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7fafb9627640 1 -- 172.21.15.83:0/3458953405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafb415dfa0 msgr2=0x7fafb40b6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:35.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7fafb9627640 1 --2- 172.21.15.83:0/3458953405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafb415dfa0 0x7fafb40b6c20 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7fafa40097c0 tx=0x7fafa401c3a0 comp rx=0 tx=0).stop 2024-04-17T10:54:35.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7fafb9627640 1 -- 172.21.15.83:0/3458953405 shutdown_connections 2024-04-17T10:54:35.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7fafb9627640 1 --2- 172.21.15.83:0/3458953405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafb415dfa0 0x7fafb40b6c20 unknown :-1 s=CLOSED pgs=2405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:35.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7fafb9627640 1 -- 172.21.15.83:0/3458953405 >> 172.21.15.83:0/3458953405 conn(0x7fafb40d0600 msgr2=0x7fafb40d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:35.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7fafb9627640 1 -- 172.21.15.83:0/3458953405 shutdown_connections 2024-04-17T10:54:35.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7fafb9627640 1 -- 172.21.15.83:0/3458953405 wait complete. 2024-04-17T10:54:35.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7fafb9627640 1 Processor -- start 2024-04-17T10:54:35.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7fafb9627640 1 -- start start 2024-04-17T10:54:35.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7fafb9627640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafb40b5b40 0x7fafb40cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:35.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7fafb9627640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafb40ce980 0x7fafb414ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:35.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7fafb9627640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafb415dfa0 0x7fafb414b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:35.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7fafb9627640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb4161b10 con 0x7fafb40b5b40 2024-04-17T10:54:35.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7fafb9627640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb40d2850 con 0x7fafb415dfa0 2024-04-17T10:54:35.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7fafb9627640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb416c670 con 0x7fafb40ce980 2024-04-17T10:54:35.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7fafb2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafb40b5b40 0x7fafb40cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:35.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7fafb2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafb40b5b40 0x7fafb40cd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60166/0 (socket says 172.21.15.83:60166) 2024-04-17T10:54:35.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7fafb2ffd640 1 -- 172.21.15.83:0/3669088402 learned_addr learned my addr 172.21.15.83:0/3669088402 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:35.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7fafb2ffd640 1 -- 172.21.15.83:0/3669088402 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafb40ce980 msgr2=0x7fafb414ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:35.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7fafb27fc640 1 --2- 172.21.15.83:0/3669088402 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafb40ce980 0x7fafb414ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:35.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7fafb37fe640 1 --2- 172.21.15.83:0/3669088402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafb415dfa0 0x7fafb414b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:35.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7fafb2ffd640 1 --2- 172.21.15.83:0/3669088402 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafb40ce980 0x7fafb414ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:35.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7fafb2ffd640 1 -- 172.21.15.83:0/3669088402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafb415dfa0 msgr2=0x7fafb414b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:35.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7fafb2ffd640 1 --2- 172.21.15.83:0/3669088402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafb415dfa0 0x7fafb414b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:35.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7fafb2ffd640 1 -- 172.21.15.83:0/3669088402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb40c9250 con 0x7fafb40b5b40 2024-04-17T10:54:35.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7fafb2ffd640 1 --2- 172.21.15.83:0/3669088402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafb40b5b40 0x7fafb40cd2d0 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7fafa40ae040 tx=0x7fafa401abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:35.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7faf9b7fe640 1 -- 172.21.15.83:0/3669088402 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa40b0050 con 0x7fafb40b5b40 2024-04-17T10:54:35.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7faf9b7fe640 1 -- 172.21.15.83:0/3669088402 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa40bc070 con 0x7fafb40b5b40 2024-04-17T10:54:35.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7fafb9627640 1 -- 172.21.15.83:0/3669088402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb40c9530 con 0x7fafb40b5b40 2024-04-17T10:54:35.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7faf9b7fe640 1 -- 172.21.15.83:0/3669088402 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa40b5a70 con 0x7fafb40b5b40 2024-04-17T10:54:35.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.171+0000 7fafb9627640 1 -- 172.21.15.83:0/3669088402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb40c9af0 con 0x7fafb40b5b40 2024-04-17T10:54:35.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.175+0000 7fafb9627640 1 -- 172.21.15.83:0/3669088402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafb40681a0 con 0x7fafb40b5b40 2024-04-17T10:54:35.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.175+0000 7faf9b7fe640 1 -- 172.21.15.83:0/3669088402 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fafa40ab4c0 con 0x7fafb40b5b40 2024-04-17T10:54:35.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.175+0000 7faf9b7fe640 1 --2- 172.21.15.83:0/3669088402 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fafa0066f60 0x7fafa0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:35.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.175+0000 7faf9b7fe640 1 -- 172.21.15.83:0/3669088402 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fafa411f940 con 0x7fafb40b5b40 2024-04-17T10:54:35.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.175+0000 7fafb27fc640 1 --2- 172.21.15.83:0/3669088402 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fafa0066f60 0x7fafa0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:35.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.175+0000 7fafb27fc640 1 --2- 172.21.15.83:0/3669088402 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fafa0066f60 0x7fafa0069420 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fafa8000f70 tx=0x7fafa8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:35.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.179+0000 7faf9b7fe640 1 -- 172.21.15.83:0/3669088402 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fafa4016050 con 0x7fafb40b5b40 2024-04-17T10:54:35.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.363+0000 7fafb9627640 1 -- 172.21.15.83:0/3669088402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fafb4161520 con 0x7fafb40b5b40 2024-04-17T10:54:35.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.363+0000 7faf9b7fe640 1 -- 172.21.15.83:0/3669088402 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fafa40f0a00 con 0x7fafb40b5b40 2024-04-17T10:54:35.358 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:35.359 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:54:35.359 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:54:35.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.367+0000 7fafb9627640 1 -- 172.21.15.83:0/3669088402 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fafa0066f60 msgr2=0x7fafa0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:35.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.367+0000 7fafb9627640 1 --2- 172.21.15.83:0/3669088402 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fafa0066f60 0x7fafa0069420 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fafa8000f70 tx=0x7fafa8009040 comp rx=0 tx=0).stop 2024-04-17T10:54:35.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.367+0000 7fafb9627640 1 -- 172.21.15.83:0/3669088402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafb40b5b40 msgr2=0x7fafb40cd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:35.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.367+0000 7fafb9627640 1 --2- 172.21.15.83:0/3669088402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafb40b5b40 0x7fafb40cd2d0 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7fafa40ae040 tx=0x7fafa401abe0 comp rx=0 tx=0).stop 2024-04-17T10:54:35.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.367+0000 7fafb9627640 1 -- 172.21.15.83:0/3669088402 shutdown_connections 2024-04-17T10:54:35.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.367+0000 7fafb9627640 1 --2- 172.21.15.83:0/3669088402 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fafa0066f60 0x7fafa0069420 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:35.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.367+0000 7fafb9627640 1 --2- 172.21.15.83:0/3669088402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafb415dfa0 0x7fafb414b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:35.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.367+0000 7fafb9627640 1 --2- 172.21.15.83:0/3669088402 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafb40ce980 0x7fafb414ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:35.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.367+0000 7fafb9627640 1 --2- 172.21.15.83:0/3669088402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafb40b5b40 0x7fafb40cd2d0 unknown :-1 s=CLOSED pgs=2406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:35.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.367+0000 7fafb9627640 1 -- 172.21.15.83:0/3669088402 >> 172.21.15.83:0/3669088402 conn(0x7fafb40d0600 msgr2=0x7fafb40cc410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:35.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.367+0000 7fafb9627640 1 -- 172.21.15.83:0/3669088402 shutdown_connections 2024-04-17T10:54:35.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:35.367+0000 7fafb9627640 1 -- 172.21.15.83:0/3669088402 wait complete. 2024-04-17T10:54:35.778 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:54:35.778 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:54:36.779 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:54:36.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.915+0000 7f42eda1d640 1 Processor -- start 2024-04-17T10:54:36.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.915+0000 7f42eda1d640 1 -- start start 2024-04-17T10:54:36.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.915+0000 7f42eda1d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f42e815ddd0 0x7f42e80d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:36.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.915+0000 7f42eda1d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42e80d18c0 con 0x7f42e80cf5f0 2024-04-17T10:54:36.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.915+0000 7f42eda1d640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42e80d1b60 con 0x7f42e80b2f70 2024-04-17T10:54:36.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.915+0000 7f42eda1d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42e80d1e40 con 0x7f42e815ddd0 2024-04-17T10:54:36.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.915+0000 7f42e6ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f42e815ddd0 0x7f42e80d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:36.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.915+0000 7f42e77fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f42e80cf5f0 0x7f42e806be50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57692/0 (socket says 172.21.15.83:57692) 2024-04-17T10:54:36.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.915+0000 7f42e6ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f42e815ddd0 0x7f42e80d2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57870/0 (socket says 172.21.15.83:57870) 2024-04-17T10:54:36.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.915+0000 7f42e77fe640 1 -- 172.21.15.83:0/3784404250 learned_addr learned my addr 172.21.15.83:0/3784404250 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:36.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.915+0000 7f42e6ffd640 1 -- 172.21.15.83:0/3784404250 >> v1:172.21.15.167:6789/0 conn(0x7f42e80b2f70 legacy=0x7f42e806b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:36.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.915+0000 7f42e67fc640 1 -- 172.21.15.83:0/3784404250 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2609503592 0 0) 0x7f42e80d18c0 con 0x7f42e80cf5f0 2024-04-17T10:54:36.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.915+0000 7f42e6ffd640 1 -- 172.21.15.83:0/3784404250 >> v1:172.21.15.83:6789/0 conn(0x7f42e80cf5f0 legacy=0x7f42e806be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:36.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.915+0000 7f42e6ffd640 1 -- 172.21.15.83:0/3784404250 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e806b220 con 0x7f42e815ddd0 2024-04-17T10:54:36.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42e6ffd640 1 --2- 172.21.15.83:0/3784404250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42e815ddd0 0x7f42e80d2020 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f42d80097c0 tx=0x7f42d801c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0014d2b6ae000be server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:36.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42e67fc640 1 -- 172.21.15.83:0/3784404250 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d80b0070 con 0x7f42e815ddd0 2024-04-17T10:54:36.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42e67fc640 1 -- 172.21.15.83:0/3784404250 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d80ab600 con 0x7f42e815ddd0 2024-04-17T10:54:36.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42e67fc640 1 -- 172.21.15.83:0/3784404250 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d80ab9d0 con 0x7f42e815ddd0 2024-04-17T10:54:36.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42eda1d640 1 -- 172.21.15.83:0/3784404250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42e815ddd0 msgr2=0x7f42e80d2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:36.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42eda1d640 1 --2- 172.21.15.83:0/3784404250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42e815ddd0 0x7f42e80d2020 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f42d80097c0 tx=0x7f42d801c340 comp rx=0 tx=0).stop 2024-04-17T10:54:36.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42eda1d640 1 -- 172.21.15.83:0/3784404250 shutdown_connections 2024-04-17T10:54:36.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42eda1d640 1 --2- 172.21.15.83:0/3784404250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42e815ddd0 0x7f42e80d2020 unknown :-1 s=CLOSED pgs=2408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:36.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42eda1d640 1 -- 172.21.15.83:0/3784404250 >> 172.21.15.83:0/3784404250 conn(0x7f42e80d3a40 msgr2=0x7f42e80d3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:36.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42eda1d640 1 -- 172.21.15.83:0/3784404250 shutdown_connections 2024-04-17T10:54:36.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42eda1d640 1 -- 172.21.15.83:0/3784404250 wait complete. 2024-04-17T10:54:36.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42eda1d640 1 Processor -- start 2024-04-17T10:54:36.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42eda1d640 1 -- start start 2024-04-17T10:54:36.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42eda1d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42e80b2f70 0x7f42e80cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:36.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42eda1d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42e80cf5f0 0x7f42e81583a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:36.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42eda1d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42e815ddd0 0x7f42e8156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:36.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42eda1d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e815e330 con 0x7f42e80b2f70 2024-04-17T10:54:36.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42eda1d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e815e1b0 con 0x7f42e80cf5f0 2024-04-17T10:54:36.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42eda1d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42e816c270 con 0x7f42e815ddd0 2024-04-17T10:54:36.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42e6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42e80b2f70 0x7f42e80cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:36.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42e6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42e80b2f70 0x7f42e80cd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57886/0 (socket says 172.21.15.83:57886) 2024-04-17T10:54:36.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42e6ffd640 1 -- 172.21.15.83:0/2895378811 learned_addr learned my addr 172.21.15.83:0/2895378811 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:36.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42e6ffd640 1 -- 172.21.15.83:0/2895378811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42e815ddd0 msgr2=0x7f42e8156820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:36.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42e6ffd640 1 --2- 172.21.15.83:0/2895378811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42e815ddd0 0x7f42e8156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:36.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42e6ffd640 1 -- 172.21.15.83:0/2895378811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42e80cf5f0 msgr2=0x7f42e81583a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:36.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42e6ffd640 1 --2- 172.21.15.83:0/2895378811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42e80cf5f0 0x7f42e81583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:36.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42e6ffd640 1 -- 172.21.15.83:0/2895378811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e80c8920 con 0x7f42e80b2f70 2024-04-17T10:54:36.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42e6ffd640 1 --2- 172.21.15.83:0/2895378811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42e80b2f70 0x7f42e80cd7e0 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7f42d80ae040 tx=0x7f42d801ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:36.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42e4ff9640 1 -- 172.21.15.83:0/2895378811 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d80b0050 con 0x7f42e80b2f70 2024-04-17T10:54:36.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42e4ff9640 1 -- 172.21.15.83:0/2895378811 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d80bc070 con 0x7f42e80b2f70 2024-04-17T10:54:36.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42eda1d640 1 -- 172.21.15.83:0/2895378811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42e80c8c00 con 0x7f42e80b2f70 2024-04-17T10:54:36.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42e4ff9640 1 -- 172.21.15.83:0/2895378811 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d80b5a70 con 0x7f42e80b2f70 2024-04-17T10:54:36.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.919+0000 7f42eda1d640 1 -- 172.21.15.83:0/2895378811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42e80c91c0 con 0x7f42e80b2f70 2024-04-17T10:54:36.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.923+0000 7f42eda1d640 1 -- 172.21.15.83:0/2895378811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42e8074f90 con 0x7f42e80b2f70 2024-04-17T10:54:36.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.923+0000 7f42e4ff9640 1 -- 172.21.15.83:0/2895378811 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f42d80ab4c0 con 0x7f42e80b2f70 2024-04-17T10:54:36.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.923+0000 7f42e4ff9640 1 --2- 172.21.15.83:0/2895378811 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f42c8066f90 0x7f42c8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:36.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.923+0000 7f42e4ff9640 1 -- 172.21.15.83:0/2895378811 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f42d811f8e0 con 0x7f42e80b2f70 2024-04-17T10:54:36.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.923+0000 7f42dffff640 1 --2- 172.21.15.83:0/2895378811 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f42c8066f90 0x7f42c8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:36.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.923+0000 7f42dffff640 1 --2- 172.21.15.83:0/2895378811 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f42c8066f90 0x7f42c8069450 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f42e80ccac0 tx=0x7f42d0006bd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:36.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:36.927+0000 7f42e4ff9640 1 -- 172.21.15.83:0/2895378811 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f42d8016050 con 0x7f42e80b2f70 2024-04-17T10:54:37.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:37.119+0000 7f42eda1d640 1 -- 172.21.15.83:0/2895378811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42e8068640 con 0x7f42e80b2f70 2024-04-17T10:54:37.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:37.123+0000 7f42e4ff9640 1 -- 172.21.15.83:0/2895378811 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f42d80f0910 con 0x7f42e80b2f70 2024-04-17T10:54:37.115 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:37.115 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:54:37.115 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:54:37.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:37.123+0000 7f42eda1d640 1 -- 172.21.15.83:0/2895378811 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f42c8066f90 msgr2=0x7f42c8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:37.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:37.123+0000 7f42eda1d640 1 --2- 172.21.15.83:0/2895378811 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f42c8066f90 0x7f42c8069450 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f42e80ccac0 tx=0x7f42d0006bd0 comp rx=0 tx=0).stop 2024-04-17T10:54:37.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:37.123+0000 7f42eda1d640 1 -- 172.21.15.83:0/2895378811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42e80b2f70 msgr2=0x7f42e80cd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:37.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:37.123+0000 7f42eda1d640 1 --2- 172.21.15.83:0/2895378811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42e80b2f70 0x7f42e80cd7e0 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7f42d80ae040 tx=0x7f42d801ab80 comp rx=0 tx=0).stop 2024-04-17T10:54:37.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:37.123+0000 7f42eda1d640 1 -- 172.21.15.83:0/2895378811 shutdown_connections 2024-04-17T10:54:37.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:37.123+0000 7f42eda1d640 1 --2- 172.21.15.83:0/2895378811 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f42c8066f90 0x7f42c8069450 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:37.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:37.123+0000 7f42eda1d640 1 --2- 172.21.15.83:0/2895378811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42e815ddd0 0x7f42e8156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:37.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:37.123+0000 7f42eda1d640 1 --2- 172.21.15.83:0/2895378811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42e80cf5f0 0x7f42e81583a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:37.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:37.123+0000 7f42eda1d640 1 --2- 172.21.15.83:0/2895378811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42e80b2f70 0x7f42e80cd7e0 unknown :-1 s=CLOSED pgs=2409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:37.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:37.123+0000 7f42eda1d640 1 -- 172.21.15.83:0/2895378811 >> 172.21.15.83:0/2895378811 conn(0x7f42e80d3a40 msgr2=0x7f42e806b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:37.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:37.123+0000 7f42eda1d640 1 -- 172.21.15.83:0/2895378811 shutdown_connections 2024-04-17T10:54:37.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:37.123+0000 7f42eda1d640 1 -- 172.21.15.83:0/2895378811 wait complete. 2024-04-17T10:54:37.536 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 51, 'flags_state': {'joinable': False, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T10:50:34.233452+0000', 'modified': '2024-04-17T10:51:05.174081+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 143, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-17T10:54:37.536 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-17T10:54:38.537 DEBUG:tasks.cephfs.filesystem:status = { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 10, "epoch": 90, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 10, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-17T10:50:34.233452+0000", "damaged": [], "data_pools": [ 21 ], "enabled": true, "epoch": 90, "ever_allowed_features": 32, "explicitly_allowed_features": 32, "failed": [ 0 ], "flags": 51, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": true, "balance_automate": false, "joinable": false, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": {}, "last_failure": 0, "last_failure_osd_epoch": 143, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 20, "modified": "2024-04-17T10:51:05.174081+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": {} } } ], "standbys": [ { "addr": "172.21.15.167:6835/4064366741", "addrs": { "addrvec": [ { "addr": "172.21.15.167:6834", "nonce": 4064366741, "type": "v2" }, { "addr": "172.21.15.167:6835", "nonce": 4064366741, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 83, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6143, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.167:6833/2584390968", "addrs": { "addrvec": [ { "addr": "172.21.15.167:6832", "nonce": 2584390968, "type": "v2" }, { "addr": "172.21.15.167:6833", "nonce": 2584390968, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 84, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7054, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.83:6835/1853711871", "addrs": { "addrvec": [ { "addr": "172.21.15.83:6834", "nonce": 1853711871, "type": "v2" }, { "addr": "172.21.15.83:6835", "nonce": 1853711871, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 83, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 9708, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.83:6837/3362045924", "addrs": { "addrvec": [ { "addr": "172.21.15.83:6836", "nonce": 3362045924, "type": "v2" }, { "addr": "172.21.15.83:6837", "nonce": 3362045924, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 90, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 9927, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-17T10:54:38.538 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T10:54:38.538 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:54:38.597 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:54:38.649 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T10:54:38.650 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:54:38.650 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:54:38.657 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:54:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f629b3c6640 1 Processor -- start 2024-04-17T10:54:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f629b3c6640 1 -- start start 2024-04-17T10:54:38.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f629b3c6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6294072330 0x7f62940cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:38.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f629b3c6640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62940cfea0 con 0x7f62940cfa10 2024-04-17T10:54:38.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f629b3c6640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f629406e320 con 0x7f629415ba30 2024-04-17T10:54:38.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f629b3c6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6294071ea0 con 0x7f6294072330 2024-04-17T10:54:38.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f6299963640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f62940cfa10 0x7f629406c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57704/0 (socket says 172.21.15.83:57704) 2024-04-17T10:54:38.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f6299963640 1 -- 172.21.15.83:0/545400694 learned_addr learned my addr 172.21.15.83:0/545400694 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:38.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f6298961640 1 --2- 172.21.15.83:0/545400694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6294072330 0x7f62940cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:38.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f628bfff640 1 -- 172.21.15.83:0/545400694 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1629663481 0 0) 0x7f62940cfea0 con 0x7f62940cfa10 2024-04-17T10:54:38.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f628bfff640 1 -- 172.21.15.83:0/545400694 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6280003530 con 0x7f62940cfa10 2024-04-17T10:54:38.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f628bfff640 1 -- 172.21.15.83:0/545400694 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3099077262 0 0) 0x7f6280003530 con 0x7f62940cfa10 2024-04-17T10:54:38.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f628bfff640 1 -- 172.21.15.83:0/545400694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6294072330 msgr2=0x7f62940cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:38.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f628bfff640 1 --2- 172.21.15.83:0/545400694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6294072330 0x7f62940cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:38.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f628bfff640 1 -- 172.21.15.83:0/545400694 >> v1:172.21.15.167:6789/0 conn(0x7f629415ba30 legacy=0x7f629415c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:38.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f628bfff640 1 -- 172.21.15.83:0/545400694 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629406e040 con 0x7f62940cfa10 2024-04-17T10:54:38.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f628bfff640 1 -- 172.21.15.83:0/545400694 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f6290001680 con 0x7f62940cfa10 2024-04-17T10:54:38.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f628bfff640 1 -- 172.21.15.83:0/545400694 >> v1:172.21.15.83:6789/0 conn(0x7f62940cfa10 legacy=0x7f629406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:38.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f628bfff640 1 --2- 172.21.15.83:0/545400694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62800039f0 0x7f6280003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:38.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f628bfff640 1 --2- 172.21.15.83:0/545400694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6294072330 0x7f6280004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:38.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f628bfff640 1 --2- 172.21.15.83:0/545400694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6280004d50 0x7f62800051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:38.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f628bfff640 1 -- 172.21.15.83:0/545400694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6294071ea0 con 0x7f62800039f0 2024-04-17T10:54:38.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f628bfff640 1 -- 172.21.15.83:0/545400694 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628000bb00 con 0x7f6280004d50 2024-04-17T10:54:38.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f628bfff640 1 -- 172.21.15.83:0/545400694 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f628000bd00 con 0x7f6294072330 2024-04-17T10:54:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f6299162640 1 --2- 172.21.15.83:0/545400694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62800039f0 0x7f6280003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f6299963640 1 --2- 172.21.15.83:0/545400694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6280004d50 0x7f62800051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f6299162640 1 -- 172.21.15.83:0/545400694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6294072330 msgr2=0x7f6280004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f6298961640 1 --2- 172.21.15.83:0/545400694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6294072330 0x7f6280004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f6299162640 1 --2- 172.21.15.83:0/545400694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6294072330 0x7f6280004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f6299162640 1 -- 172.21.15.83:0/545400694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6280004d50 msgr2=0x7f62800051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f6299162640 1 --2- 172.21.15.83:0/545400694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6280004d50 0x7f62800051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f6299162640 1 -- 172.21.15.83:0/545400694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6280003530 con 0x7f62800039f0 2024-04-17T10:54:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.831+0000 7f6299963640 1 --2- 172.21.15.83:0/545400694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6280004d50 0x7f62800051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:38.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.835+0000 7f6299162640 1 --2- 172.21.15.83:0/545400694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62800039f0 0x7f6280003e10 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7f628c001e70 tx=0x7f629406d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=364cf5da140e5a03 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:38.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f628bfff640 1 -- 172.21.15.83:0/545400694 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628c0af070 con 0x7f62800039f0 2024-04-17T10:54:38.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f628bfff640 1 -- 172.21.15.83:0/545400694 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628c003b80 con 0x7f62800039f0 2024-04-17T10:54:38.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f628bfff640 1 -- 172.21.15.83:0/545400694 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628c001a90 con 0x7f62800039f0 2024-04-17T10:54:38.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f629b3c6640 1 -- 172.21.15.83:0/545400694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62800039f0 msgr2=0x7f6280003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:38.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f629b3c6640 1 --2- 172.21.15.83:0/545400694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62800039f0 0x7f6280003e10 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7f628c001e70 tx=0x7f629406d3b0 comp rx=0 tx=0).stop 2024-04-17T10:54:38.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f6299963640 1 -- 172.21.15.83:0/545400694 reap_dead start 2024-04-17T10:54:38.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f629b3c6640 1 -- 172.21.15.83:0/545400694 shutdown_connections 2024-04-17T10:54:38.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f629b3c6640 1 -- 172.21.15.83:0/545400694 >> 172.21.15.83:0/545400694 conn(0x7f629406abe0 msgr2=0x7f629406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:38.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f629b3c6640 1 -- 172.21.15.83:0/545400694 shutdown_connections 2024-04-17T10:54:38.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f629b3c6640 1 -- 172.21.15.83:0/545400694 wait complete. 2024-04-17T10:54:38.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f629b3c6640 1 Processor -- start 2024-04-17T10:54:38.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f629b3c6640 1 -- start start 2024-04-17T10:54:38.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f629b3c6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62800039f0 0x7f629414dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:38.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f629b3c6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f629414e6f0 0x7f629415df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:38.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f629b3c6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f629415e480 0x7f62940ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:38.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f629b3c6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629406de20 con 0x7f629414e6f0 2024-04-17T10:54:38.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f629b3c6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629406dca0 con 0x7f629415e480 2024-04-17T10:54:38.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f629b3c6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6294167520 con 0x7f62800039f0 2024-04-17T10:54:38.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f6298961640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f629414e6f0 0x7f629415df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:38.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f6298961640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f629414e6f0 0x7f629415df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57910/0 (socket says 172.21.15.83:57910) 2024-04-17T10:54:38.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f6298961640 1 -- 172.21.15.83:0/3348381222 learned_addr learned my addr 172.21.15.83:0/3348381222 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:38.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f6299162640 1 --2- 172.21.15.83:0/3348381222 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62800039f0 0x7f629414dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:38.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f6299963640 1 --2- 172.21.15.83:0/3348381222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f629415e480 0x7f62940ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:38.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f6298961640 1 -- 172.21.15.83:0/3348381222 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62800039f0 msgr2=0x7f629414dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:38.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f6298961640 1 --2- 172.21.15.83:0/3348381222 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62800039f0 0x7f629414dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:38.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f6298961640 1 -- 172.21.15.83:0/3348381222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f629415e480 msgr2=0x7f62940ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:38.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f6298961640 1 --2- 172.21.15.83:0/3348381222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f629415e480 0x7f62940ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:38.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f6298961640 1 -- 172.21.15.83:0/3348381222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62940b47b0 con 0x7f629414e6f0 2024-04-17T10:54:38.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f6299963640 1 --2- 172.21.15.83:0/3348381222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f629415e480 0x7f62940ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:38.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f6298961640 1 --2- 172.21.15.83:0/3348381222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f629414e6f0 0x7f629415df40 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7f6284010890 tx=0x7f6284010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:38.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f6289ffb640 1 -- 172.21.15.83:0/3348381222 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628400f040 con 0x7f629414e6f0 2024-04-17T10:54:38.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f6289ffb640 1 -- 172.21.15.83:0/3348381222 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628400eb60 con 0x7f629414e6f0 2024-04-17T10:54:38.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f629b3c6640 1 -- 172.21.15.83:0/3348381222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62940b3f10 con 0x7f629414e6f0 2024-04-17T10:54:38.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f6289ffb640 1 -- 172.21.15.83:0/3348381222 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628401c400 con 0x7f629414e6f0 2024-04-17T10:54:38.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.839+0000 7f629b3c6640 1 -- 172.21.15.83:0/3348381222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62940b44d0 con 0x7f629414e6f0 2024-04-17T10:54:38.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.843+0000 7f629b3c6640 1 -- 172.21.15.83:0/3348381222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6294158ab0 con 0x7f629414e6f0 2024-04-17T10:54:38.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.843+0000 7f6289ffb640 1 -- 172.21.15.83:0/3348381222 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f628401c5a0 con 0x7f629414e6f0 2024-04-17T10:54:38.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.843+0000 7f6289ffb640 1 --2- 172.21.15.83:0/3348381222 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6278066f60 0x7f6278069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:38.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.843+0000 7f6289ffb640 1 -- 172.21.15.83:0/3348381222 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f6284085670 con 0x7f629414e6f0 2024-04-17T10:54:38.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.843+0000 7f6299162640 1 --2- 172.21.15.83:0/3348381222 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6278066f60 0x7f6278069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:38.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.843+0000 7f6299162640 1 --2- 172.21.15.83:0/3348381222 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6278066f60 0x7f6278069420 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f628c0ac000 tx=0x7f628c0014b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:38.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:38.847+0000 7f6289ffb640 1 -- 172.21.15.83:0/3348381222 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6284018030 con 0x7f629414e6f0 2024-04-17T10:54:38.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.007+0000 7f629b3c6640 1 -- 172.21.15.83:0/3348381222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6294074f90 con 0x7f629414e6f0 2024-04-17T10:54:39.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.007+0000 7f6289ffb640 1 -- 172.21.15.83:0/3348381222 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+16786 (secure 0 0 0) 0x7f6284056610 con 0x7f629414e6f0 2024-04-17T10:54:39.002 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:39.002 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":143,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:51:04.828803+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:50:32.488186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-17T10:50:33.207211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.32","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/3575114599":"2024-04-18T10:51:04.828753+0000","172.21.15.83:6836/3575114599":"2024-04-18T10:51:04.828753+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:54:39.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.011+0000 7f629b3c6640 1 -- 172.21.15.83:0/3348381222 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6278066f60 msgr2=0x7f6278069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:39.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.011+0000 7f629b3c6640 1 --2- 172.21.15.83:0/3348381222 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6278066f60 0x7f6278069420 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f628c0ac000 tx=0x7f628c0014b0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.011+0000 7f629b3c6640 1 -- 172.21.15.83:0/3348381222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f629414e6f0 msgr2=0x7f629415df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:39.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.011+0000 7f629b3c6640 1 --2- 172.21.15.83:0/3348381222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f629414e6f0 0x7f629415df40 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7f6284010890 tx=0x7f6284010d50 comp rx=0 tx=0).stop 2024-04-17T10:54:39.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.011+0000 7f629b3c6640 1 -- 172.21.15.83:0/3348381222 shutdown_connections 2024-04-17T10:54:39.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.011+0000 7f629b3c6640 1 --2- 172.21.15.83:0/3348381222 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6278066f60 0x7f6278069420 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.011+0000 7f629b3c6640 1 --2- 172.21.15.83:0/3348381222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f629415e480 0x7f62940ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.011+0000 7f629b3c6640 1 --2- 172.21.15.83:0/3348381222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f629414e6f0 0x7f629415df40 unknown :-1 s=CLOSED pgs=2412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.011+0000 7f629b3c6640 1 --2- 172.21.15.83:0/3348381222 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62800039f0 0x7f629414dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.011+0000 7f629b3c6640 1 -- 172.21.15.83:0/3348381222 >> 172.21.15.83:0/3348381222 conn(0x7f629406abe0 msgr2=0x7f62940b1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:39.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.011+0000 7f629b3c6640 1 -- 172.21.15.83:0/3348381222 shutdown_connections 2024-04-17T10:54:39.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.011+0000 7f629b3c6640 1 -- 172.21.15.83:0/3348381222 wait complete. 2024-04-17T10:54:39.020 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:50:32.488186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '139', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:50:33.207211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:54:39.021 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:54:39.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.143+0000 7f36b016d640 1 Processor -- start 2024-04-17T10:54:39.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.143+0000 7f36b016d640 1 -- start start 2024-04-17T10:54:39.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.143+0000 7f36b016d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f36a8147370 0x7f36a806ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:39.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.143+0000 7f36b016d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f36a80b38d0 0x7f36a806abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.143+0000 7f36b016d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36a806b120 0x7f36a8161660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.143+0000 7f36b016d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36a806d340 con 0x7f36a80b38d0 2024-04-17T10:54:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.143+0000 7f36b016d640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36a815e210 con 0x7f36a8147370 2024-04-17T10:54:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.143+0000 7f36b016d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a815e4b0 con 0x7f36a806b120 2024-04-17T10:54:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.143+0000 7f36ad708640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f36a80b38d0 0x7f36a806abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.143+0000 7f36ad708640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f36a80b38d0 0x7f36a806abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57924/0 (socket says 172.21.15.83:57924) 2024-04-17T10:54:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.143+0000 7f36ad708640 1 -- 172.21.15.83:0/3506037011 learned_addr learned my addr 172.21.15.83:0/3506037011 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.147+0000 7f36adf09640 1 --2- 172.21.15.83:0/3506037011 >> v2:172.21.15.167:3300/0 conn(0x7f36a8147370 0x7f36a806ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.147+0000 7f36ae70a640 1 --2- 172.21.15.83:0/3506037011 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36a806b120 0x7f36a8161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.147+0000 7f36ad708640 1 -- 172.21.15.83:0/3506037011 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36a806b120 msgr2=0x7f36a8161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.147+0000 7f36ad708640 1 --2- 172.21.15.83:0/3506037011 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36a806b120 0x7f36a8161660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.147+0000 7f36ad708640 1 -- 172.21.15.83:0/3506037011 >> v2:172.21.15.167:3300/0 conn(0x7f36a8147370 msgr2=0x7f36a806ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.147+0000 7f36ad708640 1 --2- 172.21.15.83:0/3506037011 >> v2:172.21.15.167:3300/0 conn(0x7f36a8147370 0x7f36a806ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.147+0000 7f36ad708640 1 -- 172.21.15.83:0/3506037011 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a815e790 con 0x7f36a80b38d0 2024-04-17T10:54:39.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.147+0000 7f36adf09640 1 --2- 172.21.15.83:0/3506037011 >> v2:172.21.15.167:3300/0 conn(0x7f36a8147370 0x7f36a806ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:39.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.147+0000 7f36ad708640 1 --2- 172.21.15.83:0/3506037011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36a80b38d0 0x7f36a806abe0 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f36a806cb30 tx=0x7f369801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1312f4f43a4e1a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:39.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.147+0000 7f36acf07640 1 -- 172.21.15.83:0/3506037011 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36980b0070 con 0x7f36a80b38d0 2024-04-17T10:54:39.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.147+0000 7f36acf07640 1 -- 172.21.15.83:0/3506037011 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36980ab600 con 0x7f36a80b38d0 2024-04-17T10:54:39.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.147+0000 7f36acf07640 1 -- 172.21.15.83:0/3506037011 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36980ab9d0 con 0x7f36a80b38d0 2024-04-17T10:54:39.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.147+0000 7f36b016d640 1 -- 172.21.15.83:0/3506037011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36a80b38d0 msgr2=0x7f36a806abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:39.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.147+0000 7f36b016d640 1 --2- 172.21.15.83:0/3506037011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36a80b38d0 0x7f36a806abe0 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f36a806cb30 tx=0x7f369801c260 comp rx=0 tx=0).stop 2024-04-17T10:54:39.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.147+0000 7f36b016d640 1 -- 172.21.15.83:0/3506037011 shutdown_connections 2024-04-17T10:54:39.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.147+0000 7f36b016d640 1 --2- 172.21.15.83:0/3506037011 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36a806b120 0x7f36a8161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.147+0000 7f36b016d640 1 --2- 172.21.15.83:0/3506037011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36a80b38d0 0x7f36a806abe0 unknown :-1 s=CLOSED pgs=2413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.147+0000 7f36b016d640 1 --2- 172.21.15.83:0/3506037011 >> v2:172.21.15.167:3300/0 conn(0x7f36a8147370 0x7f36a806ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.147+0000 7f36b016d640 1 -- 172.21.15.83:0/3506037011 >> 172.21.15.83:0/3506037011 conn(0x7f36a806bcf0 msgr2=0x7f36a806c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:39.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.147+0000 7f36b016d640 1 -- 172.21.15.83:0/3506037011 shutdown_connections 2024-04-17T10:54:39.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.147+0000 7f36b016d640 1 -- 172.21.15.83:0/3506037011 wait complete. 2024-04-17T10:54:39.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.151+0000 7f36b016d640 1 Processor -- start 2024-04-17T10:54:39.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.151+0000 7f36b016d640 1 -- start start 2024-04-17T10:54:39.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.151+0000 7f36b016d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36a806b120 0x7f36a8144410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:39.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.151+0000 7f36b016d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36a80b38d0 0x7f36a80cf980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:39.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.151+0000 7f36b016d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36a8147370 0x7f36a80cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:39.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.151+0000 7f36b016d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a80d0690 con 0x7f36a806b120 2024-04-17T10:54:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.151+0000 7f36b016d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a80d0510 con 0x7f36a80b38d0 2024-04-17T10:54:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.151+0000 7f36b016d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a80d0810 con 0x7f36a8147370 2024-04-17T10:54:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.151+0000 7f36adf09640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36a806b120 0x7f36a8144410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.151+0000 7f36adf09640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36a806b120 0x7f36a8144410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57930/0 (socket says 172.21.15.83:57930) 2024-04-17T10:54:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.151+0000 7f36adf09640 1 -- 172.21.15.83:0/2651457482 learned_addr learned my addr 172.21.15.83:0/2651457482 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.151+0000 7f36ad708640 1 --2- 172.21.15.83:0/2651457482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36a80b38d0 0x7f36a80cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.151+0000 7f36ae70a640 1 --2- 172.21.15.83:0/2651457482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36a8147370 0x7f36a80cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.151+0000 7f36adf09640 1 -- 172.21.15.83:0/2651457482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36a8147370 msgr2=0x7f36a80cd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.151+0000 7f36adf09640 1 --2- 172.21.15.83:0/2651457482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36a8147370 0x7f36a80cd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.151+0000 7f36adf09640 1 -- 172.21.15.83:0/2651457482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36a80b38d0 msgr2=0x7f36a80cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.151+0000 7f36adf09640 1 --2- 172.21.15.83:0/2651457482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36a80b38d0 0x7f36a80cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.151+0000 7f36adf09640 1 -- 172.21.15.83:0/2651457482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a815fc70 con 0x7f36a806b120 2024-04-17T10:54:39.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.151+0000 7f36ad708640 1 --2- 172.21.15.83:0/2651457482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36a80b38d0 0x7f36a80cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:39.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.151+0000 7f36ae70a640 1 --2- 172.21.15.83:0/2651457482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36a8147370 0x7f36a80cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:39.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.151+0000 7f36adf09640 1 --2- 172.21.15.83:0/2651457482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36a806b120 0x7f36a8144410 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f36a400c820 tx=0x7f36a400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:39.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.151+0000 7f369e7fc640 1 -- 172.21.15.83:0/2651457482 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a4015070 con 0x7f36a806b120 2024-04-17T10:54:39.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.151+0000 7f369e7fc640 1 -- 172.21.15.83:0/2651457482 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a4007a30 con 0x7f36a806b120 2024-04-17T10:54:39.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.151+0000 7f36b016d640 1 -- 172.21.15.83:0/2651457482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a815ff50 con 0x7f36a806b120 2024-04-17T10:54:39.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.151+0000 7f369e7fc640 1 -- 172.21.15.83:0/2651457482 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a4007e40 con 0x7f36a806b120 2024-04-17T10:54:39.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.151+0000 7f36b016d640 1 -- 172.21.15.83:0/2651457482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a8160510 con 0x7f36a806b120 2024-04-17T10:54:39.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.151+0000 7f36b016d640 1 -- 172.21.15.83:0/2651457482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36a8074f90 con 0x7f36a806b120 2024-04-17T10:54:39.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.151+0000 7f369e7fc640 1 -- 172.21.15.83:0/2651457482 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f36a4018070 con 0x7f36a806b120 2024-04-17T10:54:39.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.151+0000 7f369e7fc640 1 --2- 172.21.15.83:0/2651457482 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f368c066f60 0x7f368c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:39.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.151+0000 7f369e7fc640 1 -- 172.21.15.83:0/2651457482 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f36a4084f40 con 0x7f36a806b120 2024-04-17T10:54:39.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.151+0000 7f36ad708640 1 --2- 172.21.15.83:0/2651457482 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f368c066f60 0x7f368c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:39.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.155+0000 7f36ad708640 1 --2- 172.21.15.83:0/2651457482 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f368c066f60 0x7f368c069420 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f36980ad000 tx=0x7f3698008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:39.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.155+0000 7f369e7fc640 1 -- 172.21.15.83:0/2651457482 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f36a4055f70 con 0x7f36a806b120 2024-04-17T10:54:39.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.315+0000 7f36b016d640 1 -- 172.21.15.83:0/2651457482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f36a8071d90 con 0x7f36a806b120 2024-04-17T10:54:39.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.315+0000 7f369e7fc640 1 -- 172.21.15.83:0/2651457482 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v143) v1 ==== 93+0+10 (secure 0 0 0) 0x7f36a4055870 con 0x7f36a806b120 2024-04-17T10:54:39.308 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:54:39.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.315+0000 7f36b016d640 1 -- 172.21.15.83:0/2651457482 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f368c066f60 msgr2=0x7f368c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:39.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.315+0000 7f36b016d640 1 --2- 172.21.15.83:0/2651457482 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f368c066f60 0x7f368c069420 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f36980ad000 tx=0x7f3698008040 comp rx=0 tx=0).stop 2024-04-17T10:54:39.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.315+0000 7f36b016d640 1 -- 172.21.15.83:0/2651457482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36a806b120 msgr2=0x7f36a8144410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:39.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.315+0000 7f36b016d640 1 --2- 172.21.15.83:0/2651457482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36a806b120 0x7f36a8144410 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f36a400c820 tx=0x7f36a400cce0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.315+0000 7f36b016d640 1 -- 172.21.15.83:0/2651457482 shutdown_connections 2024-04-17T10:54:39.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.315+0000 7f36b016d640 1 --2- 172.21.15.83:0/2651457482 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f368c066f60 0x7f368c069420 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.315+0000 7f36b016d640 1 --2- 172.21.15.83:0/2651457482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36a8147370 0x7f36a80cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.315+0000 7f36b016d640 1 --2- 172.21.15.83:0/2651457482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36a80b38d0 0x7f36a80cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.315+0000 7f36b016d640 1 --2- 172.21.15.83:0/2651457482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36a806b120 0x7f36a8144410 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.315+0000 7f36b016d640 1 -- 172.21.15.83:0/2651457482 >> 172.21.15.83:0/2651457482 conn(0x7f36a806bcf0 msgr2=0x7f36a806c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:39.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.315+0000 7f36b016d640 1 -- 172.21.15.83:0/2651457482 shutdown_connections 2024-04-17T10:54:39.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.315+0000 7f36b016d640 1 -- 172.21.15.83:0/2651457482 wait complete. 2024-04-17T10:54:39.327 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:54:39.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.459+0000 7f7dd3826640 1 Processor -- start 2024-04-17T10:54:39.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.459+0000 7f7dd3826640 1 -- start start 2024-04-17T10:54:39.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.463+0000 7f7dd3826640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7dcc15b6e0 0x7f7dcc071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:39.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.463+0000 7f7dd3826640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7dcc15bac0 0x7f7dcc0cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:39.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.463+0000 7f7dd3826640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dcc06c1f0 con 0x7f7dcc070470 2024-04-17T10:54:39.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.463+0000 7f7dd3826640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dcc0cdd30 con 0x7f7dcc15b6e0 2024-04-17T10:54:39.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.463+0000 7f7dd3826640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dcc070a10 con 0x7f7dcc15bac0 2024-04-17T10:54:39.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.463+0000 7f7dd15c2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7dcc15b6e0 0x7f7dcc071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:39.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.463+0000 7f7dd15c2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7dcc15b6e0 0x7f7dcc071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57932/0 (socket says 172.21.15.83:57932) 2024-04-17T10:54:39.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.463+0000 7f7dd1dc3640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7dcc070470 0x7f7dcc06ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57720/0 (socket says 172.21.15.83:57720) 2024-04-17T10:54:39.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.463+0000 7f7dd15c2640 1 -- 172.21.15.83:0/397710981 learned_addr learned my addr 172.21.15.83:0/397710981 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:39.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.463+0000 7f7dd0dc1640 1 --2- 172.21.15.83:0/397710981 >> v2:172.21.15.167:3300/0 conn(0x7f7dcc15bac0 0x7f7dcc0cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:39.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.463+0000 7f7dd15c2640 1 -- 172.21.15.83:0/397710981 >> v2:172.21.15.167:3300/0 conn(0x7f7dcc15bac0 msgr2=0x7f7dcc0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:39.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.463+0000 7f7dd15c2640 1 --2- 172.21.15.83:0/397710981 >> v2:172.21.15.167:3300/0 conn(0x7f7dcc15bac0 0x7f7dcc0cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.463+0000 7f7dd15c2640 1 -- 172.21.15.83:0/397710981 >> v1:172.21.15.83:6789/0 conn(0x7f7dcc070470 legacy=0x7f7dcc06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:39.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.463+0000 7f7dd15c2640 1 -- 172.21.15.83:0/397710981 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dcc070cf0 con 0x7f7dcc15b6e0 2024-04-17T10:54:39.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.463+0000 7f7dc3fff640 1 -- 172.21.15.83:0/397710981 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 232126878 0 0) 0x7f7dcc06c1f0 con 0x7f7dcc070470 2024-04-17T10:54:39.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.463+0000 7f7dd0dc1640 1 --2- 172.21.15.83:0/397710981 >> v2:172.21.15.167:3300/0 conn(0x7f7dcc15bac0 0x7f7dcc0cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:39.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.463+0000 7f7dd15c2640 1 --2- 172.21.15.83:0/397710981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dcc15b6e0 0x7f7dcc071cc0 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc071c50 tx=0x7f7dc801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac7995f212ef3b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:39.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.463+0000 7f7dc3fff640 1 -- 172.21.15.83:0/397710981 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc80b0070 con 0x7f7dcc15b6e0 2024-04-17T10:54:39.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.463+0000 7f7dc3fff640 1 -- 172.21.15.83:0/397710981 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc80ab600 con 0x7f7dcc15b6e0 2024-04-17T10:54:39.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.463+0000 7f7dc3fff640 1 -- 172.21.15.83:0/397710981 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc80ab9d0 con 0x7f7dcc15b6e0 2024-04-17T10:54:39.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.463+0000 7f7dd3826640 1 -- 172.21.15.83:0/397710981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dcc15b6e0 msgr2=0x7f7dcc071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:39.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.463+0000 7f7dd3826640 1 --2- 172.21.15.83:0/397710981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dcc15b6e0 0x7f7dcc071cc0 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc071c50 tx=0x7f7dc801c520 comp rx=0 tx=0).stop 2024-04-17T10:54:39.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.463+0000 7f7dd3826640 1 -- 172.21.15.83:0/397710981 shutdown_connections 2024-04-17T10:54:39.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.463+0000 7f7dd3826640 1 --2- 172.21.15.83:0/397710981 >> v2:172.21.15.167:3300/0 conn(0x7f7dcc15bac0 0x7f7dcc0cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.463+0000 7f7dd3826640 1 --2- 172.21.15.83:0/397710981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dcc15b6e0 0x7f7dcc071cc0 unknown :-1 s=CLOSED pgs=2416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.463+0000 7f7dd3826640 1 -- 172.21.15.83:0/397710981 >> 172.21.15.83:0/397710981 conn(0x7f7dcc0b15b0 msgr2=0x7f7dcc0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:39.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.463+0000 7f7dd3826640 1 -- 172.21.15.83:0/397710981 shutdown_connections 2024-04-17T10:54:39.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.463+0000 7f7dd3826640 1 -- 172.21.15.83:0/397710981 wait complete. 2024-04-17T10:54:39.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.467+0000 7f7dd3826640 1 Processor -- start 2024-04-17T10:54:39.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.467+0000 7f7dd3826640 1 -- start start 2024-04-17T10:54:39.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.467+0000 7f7dd3826640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7dcc070470 0x7f7dcc165b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:39.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.467+0000 7f7dd3826640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dcc15b6e0 0x7f7dcc148be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:39.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.467+0000 7f7dd3826640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7dcc15bac0 0x7f7dcc149120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:39.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.467+0000 7f7dd3826640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc06cee0 con 0x7f7dcc15b6e0 2024-04-17T10:54:39.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.467+0000 7f7dd3826640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc06b730 con 0x7f7dcc15bac0 2024-04-17T10:54:39.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.467+0000 7f7dd3826640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc06d060 con 0x7f7dcc070470 2024-04-17T10:54:39.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.467+0000 7f7dd0dc1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dcc15b6e0 0x7f7dcc148be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:39.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.467+0000 7f7dd0dc1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dcc15b6e0 0x7f7dcc148be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57942/0 (socket says 172.21.15.83:57942) 2024-04-17T10:54:39.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.467+0000 7f7dd0dc1640 1 -- 172.21.15.83:0/3954345263 learned_addr learned my addr 172.21.15.83:0/3954345263 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:39.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.467+0000 7f7dd15c2640 1 --2- 172.21.15.83:0/3954345263 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7dcc070470 0x7f7dcc165b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:39.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.467+0000 7f7dd0dc1640 1 -- 172.21.15.83:0/3954345263 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7dcc070470 msgr2=0x7f7dcc165b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:39.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.467+0000 7f7dd0dc1640 1 --2- 172.21.15.83:0/3954345263 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7dcc070470 0x7f7dcc165b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.467+0000 7f7dd0dc1640 1 -- 172.21.15.83:0/3954345263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7dcc15bac0 msgr2=0x7f7dcc149120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:39.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.467+0000 7f7dd0dc1640 1 --2- 172.21.15.83:0/3954345263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7dcc15bac0 0x7f7dcc149120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.467+0000 7f7dd0dc1640 1 -- 172.21.15.83:0/3954345263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dcc14b7d0 con 0x7f7dcc15b6e0 2024-04-17T10:54:39.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.467+0000 7f7dd0dc1640 1 --2- 172.21.15.83:0/3954345263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dcc15b6e0 0x7f7dcc148be0 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc00c7f0 tx=0x7f7dbc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:39.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.467+0000 7f7dc1ffb640 1 -- 172.21.15.83:0/3954345263 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc015070 con 0x7f7dcc15b6e0 2024-04-17T10:54:39.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.467+0000 7f7dc1ffb640 1 -- 172.21.15.83:0/3954345263 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dbc007820 con 0x7f7dcc15b6e0 2024-04-17T10:54:39.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.467+0000 7f7dc1ffb640 1 -- 172.21.15.83:0/3954345263 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc007c10 con 0x7f7dcc15b6e0 2024-04-17T10:54:39.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.467+0000 7f7dd3826640 1 -- 172.21.15.83:0/3954345263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dcc14c550 con 0x7f7dcc15b6e0 2024-04-17T10:54:39.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.467+0000 7f7dd3826640 1 -- 172.21.15.83:0/3954345263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dcc14bd50 con 0x7f7dcc15b6e0 2024-04-17T10:54:39.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.467+0000 7f7dd3826640 1 -- 172.21.15.83:0/3954345263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dcc074f90 con 0x7f7dcc15b6e0 2024-04-17T10:54:39.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.467+0000 7f7dc1ffb640 1 -- 172.21.15.83:0/3954345263 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7dbc018070 con 0x7f7dcc15b6e0 2024-04-17T10:54:39.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.471+0000 7f7dc1ffb640 1 --2- 172.21.15.83:0/3954345263 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7db0066f60 0x7f7db0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:39.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.471+0000 7f7dc1ffb640 1 -- 172.21.15.83:0/3954345263 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f7dbc085160 con 0x7f7dcc15b6e0 2024-04-17T10:54:39.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.471+0000 7f7dd15c2640 1 --2- 172.21.15.83:0/3954345263 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7db0066f60 0x7f7db0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:39.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.471+0000 7f7dd15c2640 1 --2- 172.21.15.83:0/3954345263 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7db0066f60 0x7f7db0069420 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f7dc80ad000 tx=0x7f7dc8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:39.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.475+0000 7f7dc1ffb640 1 -- 172.21.15.83:0/3954345263 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7dbc056100 con 0x7f7dcc15b6e0 2024-04-17T10:54:39.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.619+0000 7f7dd3826640 1 -- 172.21.15.83:0/3954345263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7dcc0b2450 con 0x7f7dcc15b6e0 2024-04-17T10:54:39.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.623+0000 7f7dc1ffb640 1 -- 172.21.15.83:0/3954345263 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v143) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7dbc055a00 con 0x7f7dcc15b6e0 2024-04-17T10:54:39.615 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:54:39.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.623+0000 7f7dd3826640 1 -- 172.21.15.83:0/3954345263 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7db0066f60 msgr2=0x7f7db0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:39.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.623+0000 7f7dd3826640 1 --2- 172.21.15.83:0/3954345263 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7db0066f60 0x7f7db0069420 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f7dc80ad000 tx=0x7f7dc8008040 comp rx=0 tx=0).stop 2024-04-17T10:54:39.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.623+0000 7f7dd3826640 1 -- 172.21.15.83:0/3954345263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dcc15b6e0 msgr2=0x7f7dcc148be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:39.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.623+0000 7f7dd3826640 1 --2- 172.21.15.83:0/3954345263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dcc15b6e0 0x7f7dcc148be0 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc00c7f0 tx=0x7f7dbc00ccb0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.623+0000 7f7dd3826640 1 -- 172.21.15.83:0/3954345263 shutdown_connections 2024-04-17T10:54:39.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.623+0000 7f7dd3826640 1 --2- 172.21.15.83:0/3954345263 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7db0066f60 0x7f7db0069420 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.623+0000 7f7dd3826640 1 --2- 172.21.15.83:0/3954345263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7dcc15bac0 0x7f7dcc149120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.623+0000 7f7dd3826640 1 --2- 172.21.15.83:0/3954345263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dcc15b6e0 0x7f7dcc148be0 unknown :-1 s=CLOSED pgs=2417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.623+0000 7f7dd3826640 1 --2- 172.21.15.83:0/3954345263 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7dcc070470 0x7f7dcc165b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.623+0000 7f7dd3826640 1 -- 172.21.15.83:0/3954345263 >> 172.21.15.83:0/3954345263 conn(0x7f7dcc0b15b0 msgr2=0x7f7dcc0ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:39.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.623+0000 7f7dd3826640 1 -- 172.21.15.83:0/3954345263 shutdown_connections 2024-04-17T10:54:39.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.623+0000 7f7dd3826640 1 -- 172.21.15.83:0/3954345263 wait complete. 2024-04-17T10:54:39.632 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:54:39.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77d178d640 1 Processor -- start 2024-04-17T10:54:39.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77d178d640 1 -- start start 2024-04-17T10:54:39.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77d178d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f77cc0b5b60 0x7f77cc0b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:39.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77d178d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77cc15e4d0 con 0x7f77cc0ce400 2024-04-17T10:54:39.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77d178d640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77cc162580 con 0x7f77cc15e0f0 2024-04-17T10:54:39.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77d178d640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77cc162b40 con 0x7f77cc0b5b60 2024-04-17T10:54:39.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77c3fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f77cc0ce400 0x7f77cc0b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57736/0 (socket says 172.21.15.83:57736) 2024-04-17T10:54:39.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77c3fff640 1 -- 172.21.15.83:0/2644365316 learned_addr learned my addr 172.21.15.83:0/2644365316 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:39.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77cb7fe640 1 --2- 172.21.15.83:0/2644365316 >> v2:172.21.15.167:3300/0 conn(0x7f77cc0b5b60 0x7f77cc0b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:39.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77ca7fc640 1 -- 172.21.15.83:0/2644365316 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2113895267 0 0) 0x7f77cc15e4d0 con 0x7f77cc0ce400 2024-04-17T10:54:39.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77ca7fc640 1 -- 172.21.15.83:0/2644365316 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77b8003530 con 0x7f77cc0ce400 2024-04-17T10:54:39.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77ca7fc640 1 -- 172.21.15.83:0/2644365316 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2874153556 0 0) 0x7f77b8003530 con 0x7f77cc0ce400 2024-04-17T10:54:39.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77ca7fc640 1 -- 172.21.15.83:0/2644365316 >> v2:172.21.15.167:3300/0 conn(0x7f77cc0b5b60 msgr2=0x7f77cc0b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:39.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77ca7fc640 1 --2- 172.21.15.83:0/2644365316 >> v2:172.21.15.167:3300/0 conn(0x7f77cc0b5b60 0x7f77cc0b6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77ca7fc640 1 -- 172.21.15.83:0/2644365316 >> v1:172.21.15.167:6789/0 conn(0x7f77cc15e0f0 legacy=0x7f77cc161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:39.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77ca7fc640 1 -- 172.21.15.83:0/2644365316 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77cc162860 con 0x7f77cc0ce400 2024-04-17T10:54:39.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77ca7fc640 1 -- 172.21.15.83:0/2644365316 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f77b4001680 con 0x7f77cc0ce400 2024-04-17T10:54:39.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77cb7fe640 1 --2- 172.21.15.83:0/2644365316 >> v2:172.21.15.167:3300/0 conn(0x7f77cc0b5b60 0x7f77cc0b6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:54:39.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77ca7fc640 1 -- 172.21.15.83:0/2644365316 >> v1:172.21.15.83:6789/0 conn(0x7f77cc0ce400 legacy=0x7f77cc0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:39.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77ca7fc640 1 --2- 172.21.15.83:0/2644365316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77b8003a30 0x7f77b8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:39.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77ca7fc640 1 --2- 172.21.15.83:0/2644365316 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77b8004810 0x7f77b8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:39.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77ca7fc640 1 --2- 172.21.15.83:0/2644365316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77b80052f0 0x7f77b800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:39.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77ca7fc640 1 -- 172.21.15.83:0/2644365316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc162b40 con 0x7f77b8003a30 2024-04-17T10:54:39.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77ca7fc640 1 -- 172.21.15.83:0/2644365316 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b800c240 con 0x7f77b80052f0 2024-04-17T10:54:39.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77ca7fc640 1 -- 172.21.15.83:0/2644365316 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77b800c440 con 0x7f77b8004810 2024-04-17T10:54:39.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77caffd640 1 --2- 172.21.15.83:0/2644365316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77b8003a30 0x7f77b8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:39.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77cb7fe640 1 --2- 172.21.15.83:0/2644365316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77b80052f0 0x7f77b800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:39.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77caffd640 1 -- 172.21.15.83:0/2644365316 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77b8004810 msgr2=0x7f77b8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:39.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77caffd640 1 --2- 172.21.15.83:0/2644365316 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77b8004810 0x7f77b8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77caffd640 1 -- 172.21.15.83:0/2644365316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77b80052f0 msgr2=0x7f77b800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:39.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77caffd640 1 --2- 172.21.15.83:0/2644365316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77b80052f0 0x7f77b800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77caffd640 1 -- 172.21.15.83:0/2644365316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b8003530 con 0x7f77b8003a30 2024-04-17T10:54:39.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77cb7fe640 1 --2- 172.21.15.83:0/2644365316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77b80052f0 0x7f77b800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:39.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.767+0000 7f77cb7fe640 1 -- 172.21.15.83:0/2644365316 reap_dead start 2024-04-17T10:54:39.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.771+0000 7f77caffd640 1 --2- 172.21.15.83:0/2644365316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77b8003a30 0x7f77b8003e50 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7f77bc001750 tx=0x7f77bc01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff3d84bcc2b8ed95 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:39.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.771+0000 7f77ca7fc640 1 -- 172.21.15.83:0/2644365316 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc0013a0 con 0x7f77b8003a30 2024-04-17T10:54:39.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.771+0000 7f77ca7fc640 1 -- 172.21.15.83:0/2644365316 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77bc0052f0 con 0x7f77b8003a30 2024-04-17T10:54:39.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.771+0000 7f77ca7fc640 1 -- 172.21.15.83:0/2644365316 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc0a4e60 con 0x7f77b8003a30 2024-04-17T10:54:39.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.771+0000 7f77d178d640 1 -- 172.21.15.83:0/2644365316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77b8003a30 msgr2=0x7f77b8003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:39.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.771+0000 7f77d178d640 1 --2- 172.21.15.83:0/2644365316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77b8003a30 0x7f77b8003e50 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7f77bc001750 tx=0x7f77bc01c390 comp rx=0 tx=0).stop 2024-04-17T10:54:39.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.771+0000 7f77d178d640 1 -- 172.21.15.83:0/2644365316 shutdown_connections 2024-04-17T10:54:39.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.771+0000 7f77d178d640 1 --2- 172.21.15.83:0/2644365316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77b8003a30 0x7f77b8003e50 unknown :-1 s=CLOSED pgs=2419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.771+0000 7f77d178d640 1 -- 172.21.15.83:0/2644365316 >> 172.21.15.83:0/2644365316 conn(0x7f77cc0d0450 msgr2=0x7f77cc0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:39.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.771+0000 7f77d178d640 1 -- 172.21.15.83:0/2644365316 shutdown_connections 2024-04-17T10:54:39.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.771+0000 7f77d178d640 1 -- 172.21.15.83:0/2644365316 wait complete. 2024-04-17T10:54:39.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.771+0000 7f77d178d640 1 Processor -- start 2024-04-17T10:54:39.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.771+0000 7f77d178d640 1 -- start start 2024-04-17T10:54:39.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.771+0000 7f77d178d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77b8003a30 0x7f77cc0c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:39.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.771+0000 7f77d178d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77cc1510f0 0x7f77cc157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:39.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.771+0000 7f77d178d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77cc1f71e0 0x7f77cc15c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:39.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.771+0000 7f77d178d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc0b59e0 con 0x7f77b8003a30 2024-04-17T10:54:39.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.771+0000 7f77d178d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc0b5860 con 0x7f77cc1f71e0 2024-04-17T10:54:39.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.771+0000 7f77d178d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc0d2540 con 0x7f77cc1510f0 2024-04-17T10:54:39.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.771+0000 7f77caffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77b8003a30 0x7f77cc0c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:39.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.771+0000 7f77caffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77b8003a30 0x7f77cc0c9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57952/0 (socket says 172.21.15.83:57952) 2024-04-17T10:54:39.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.771+0000 7f77caffd640 1 -- 172.21.15.83:0/2188449795 learned_addr learned my addr 172.21.15.83:0/2188449795 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:39.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.771+0000 7f77cb7fe640 1 --2- 172.21.15.83:0/2188449795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77cc1f71e0 0x7f77cc15c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:39.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.775+0000 7f77c3fff640 1 --2- 172.21.15.83:0/2188449795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77cc1510f0 0x7f77cc157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:39.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.775+0000 7f77caffd640 1 -- 172.21.15.83:0/2188449795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77cc1510f0 msgr2=0x7f77cc157a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:39.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.775+0000 7f77caffd640 1 --2- 172.21.15.83:0/2188449795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77cc1510f0 0x7f77cc157a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.775+0000 7f77caffd640 1 -- 172.21.15.83:0/2188449795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77cc1f71e0 msgr2=0x7f77cc15c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:39.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.775+0000 7f77caffd640 1 --2- 172.21.15.83:0/2188449795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77cc1f71e0 0x7f77cc15c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.775+0000 7f77caffd640 1 -- 172.21.15.83:0/2188449795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77cc156600 con 0x7f77b8003a30 2024-04-17T10:54:39.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.775+0000 7f77cb7fe640 1 --2- 172.21.15.83:0/2188449795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77cc1f71e0 0x7f77cc15c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:54:39.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.775+0000 7f77caffd640 1 --2- 172.21.15.83:0/2188449795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77b8003a30 0x7f77cc0c9540 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f77bc01c1c0 tx=0x7f77b8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:39.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.775+0000 7f77c37fe640 1 -- 172.21.15.83:0/2188449795 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc007a40 con 0x7f77b8003a30 2024-04-17T10:54:39.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.775+0000 7f77c37fe640 1 -- 172.21.15.83:0/2188449795 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77bc0ba070 con 0x7f77b8003a30 2024-04-17T10:54:39.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.775+0000 7f77d178d640 1 -- 172.21.15.83:0/2188449795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77cc0c9a80 con 0x7f77b8003a30 2024-04-17T10:54:39.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.775+0000 7f77c37fe640 1 -- 172.21.15.83:0/2188449795 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc0054a0 con 0x7f77b8003a30 2024-04-17T10:54:39.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.775+0000 7f77d178d640 1 -- 172.21.15.83:0/2188449795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77cc15ff30 con 0x7f77b8003a30 2024-04-17T10:54:39.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.775+0000 7f77d178d640 1 -- 172.21.15.83:0/2188449795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77cc0681a0 con 0x7f77b8003a30 2024-04-17T10:54:39.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.775+0000 7f77c37fe640 1 -- 172.21.15.83:0/2188449795 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f77bc007030 con 0x7f77b8003a30 2024-04-17T10:54:39.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.775+0000 7f77c37fe640 1 --2- 172.21.15.83:0/2188449795 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f77ac066f90 0x7f77ac069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:39.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.775+0000 7f77c37fe640 1 -- 172.21.15.83:0/2188449795 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f77bc11ee60 con 0x7f77b8003a30 2024-04-17T10:54:39.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.775+0000 7f77c3fff640 1 --2- 172.21.15.83:0/2188449795 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f77ac066f90 0x7f77ac069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:39.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.775+0000 7f77c3fff640 1 --2- 172.21.15.83:0/2188449795 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f77ac066f90 0x7f77ac069450 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f77b4001ec0 tx=0x7f77b4001c90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:39.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.779+0000 7f77c37fe640 1 -- 172.21.15.83:0/2188449795 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f77cc156600 con 0x7f77b8003a30 2024-04-17T10:54:39.816 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:54:39.818 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:54:39.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.939+0000 7f77d178d640 1 -- 172.21.15.83:0/2188449795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f77cc074f90 con 0x7f77b8003a30 2024-04-17T10:54:39.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.939+0000 7f77c37fe640 1 -- 172.21.15.83:0/2188449795 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v143) v1 ==== 100+0+11 (secure 0 0 0) 0x7f77bc0160d0 con 0x7f77b8003a30 2024-04-17T10:54:39.932 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:54:39.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.939+0000 7f77d178d640 1 -- 172.21.15.83:0/2188449795 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f77ac066f90 msgr2=0x7f77ac069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:39.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.939+0000 7f77d178d640 1 --2- 172.21.15.83:0/2188449795 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f77ac066f90 0x7f77ac069450 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f77b4001ec0 tx=0x7f77b4001c90 comp rx=0 tx=0).stop 2024-04-17T10:54:39.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.939+0000 7f77d178d640 1 -- 172.21.15.83:0/2188449795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77b8003a30 msgr2=0x7f77cc0c9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:39.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.939+0000 7f77d178d640 1 --2- 172.21.15.83:0/2188449795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77b8003a30 0x7f77cc0c9540 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f77bc01c1c0 tx=0x7f77b8005840 comp rx=0 tx=0).stop 2024-04-17T10:54:39.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.939+0000 7f77d178d640 1 -- 172.21.15.83:0/2188449795 shutdown_connections 2024-04-17T10:54:39.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.939+0000 7f77d178d640 1 --2- 172.21.15.83:0/2188449795 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f77ac066f90 0x7f77ac069450 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.939+0000 7f77d178d640 1 --2- 172.21.15.83:0/2188449795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77cc1f71e0 0x7f77cc15c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.939+0000 7f77d178d640 1 --2- 172.21.15.83:0/2188449795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77cc1510f0 0x7f77cc157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.939+0000 7f77d178d640 1 --2- 172.21.15.83:0/2188449795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77b8003a30 0x7f77cc0c9540 unknown :-1 s=CLOSED pgs=2420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:39.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.939+0000 7f77d178d640 1 -- 172.21.15.83:0/2188449795 >> 172.21.15.83:0/2188449795 conn(0x7f77cc0d0450 msgr2=0x7f77cc0b87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:39.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.939+0000 7f77d178d640 1 -- 172.21.15.83:0/2188449795 shutdown_connections 2024-04-17T10:54:39.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:39.939+0000 7f77d178d640 1 -- 172.21.15.83:0/2188449795 wait complete. 2024-04-17T10:54:39.951 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:54:40.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.079+0000 7fe22fae5640 1 Processor -- start 2024-04-17T10:54:40.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.079+0000 7fe22fae5640 1 -- start start 2024-04-17T10:54:40.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.079+0000 7fe22fae5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe228154670 0x7fe2280aea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:40.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.079+0000 7fe22fae5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe22806cc80 0x7fe228070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:40.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.079+0000 7fe22fae5640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2280b2ff0 con 0x7fe2280b4090 2024-04-17T10:54:40.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.079+0000 7fe22fae5640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe228154a50 con 0x7fe22806cc80 2024-04-17T10:54:40.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.079+0000 7fe22fae5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2280b32d0 con 0x7fe228154670 2024-04-17T10:54:40.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.079+0000 7fe22e082640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe22806cc80 0x7fe228070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:40.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.079+0000 7fe22e082640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe22806cc80 0x7fe228070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57966/0 (socket says 172.21.15.83:57966) 2024-04-17T10:54:40.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.079+0000 7fe22e082640 1 -- 172.21.15.83:0/2851602963 learned_addr learned my addr 172.21.15.83:0/2851602963 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:40.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.079+0000 7fe22d080640 1 --1- 172.21.15.83:0/2851602963 >> v1:172.21.15.83:6789/0 conn(0x7fe2280b4090 0x7fe22806c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57748/0 (socket says 172.21.15.83:57748) 2024-04-17T10:54:40.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.079+0000 7fe22d881640 1 --2- 172.21.15.83:0/2851602963 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe228154670 0x7fe2280aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:40.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.079+0000 7fe22e082640 1 -- 172.21.15.83:0/2851602963 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe228154670 msgr2=0x7fe2280aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:40.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.079+0000 7fe22e082640 1 --2- 172.21.15.83:0/2851602963 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe228154670 0x7fe2280aea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:40.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.079+0000 7fe22e082640 1 -- 172.21.15.83:0/2851602963 >> v1:172.21.15.83:6789/0 conn(0x7fe2280b4090 legacy=0x7fe22806c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:40.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.079+0000 7fe22c87f640 1 -- 172.21.15.83:0/2851602963 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1486641573 0 0) 0x7fe2280b2ff0 con 0x7fe2280b4090 2024-04-17T10:54:40.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.079+0000 7fe22e082640 1 -- 172.21.15.83:0/2851602963 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2280b35b0 con 0x7fe22806cc80 2024-04-17T10:54:40.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.083+0000 7fe22e082640 1 --2- 172.21.15.83:0/2851602963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe22806cc80 0x7fe228070470 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7fe220009e50 tx=0x7fe22001da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7406c0327e4f255b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:40.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.083+0000 7fe22c87f640 1 -- 172.21.15.83:0/2851602963 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2200a6940 con 0x7fe22806cc80 2024-04-17T10:54:40.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.083+0000 7fe22c87f640 1 -- 172.21.15.83:0/2851602963 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2200a6ae0 con 0x7fe22806cc80 2024-04-17T10:54:40.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.083+0000 7fe22c87f640 1 -- 172.21.15.83:0/2851602963 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2200ae3f0 con 0x7fe22806cc80 2024-04-17T10:54:40.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.083+0000 7fe22fae5640 1 -- 172.21.15.83:0/2851602963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe22806cc80 msgr2=0x7fe228070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:40.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.083+0000 7fe22fae5640 1 --2- 172.21.15.83:0/2851602963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe22806cc80 0x7fe228070470 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7fe220009e50 tx=0x7fe22001da80 comp rx=0 tx=0).stop 2024-04-17T10:54:40.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.083+0000 7fe22fae5640 1 -- 172.21.15.83:0/2851602963 shutdown_connections 2024-04-17T10:54:40.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.083+0000 7fe22fae5640 1 --2- 172.21.15.83:0/2851602963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe22806cc80 0x7fe228070470 unknown :-1 s=CLOSED pgs=2422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:40.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.083+0000 7fe22fae5640 1 --2- 172.21.15.83:0/2851602963 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe228154670 0x7fe2280aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:40.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.083+0000 7fe22fae5640 1 -- 172.21.15.83:0/2851602963 >> 172.21.15.83:0/2851602963 conn(0x7fe2280b15b0 msgr2=0x7fe2280b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:40.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.083+0000 7fe22fae5640 1 -- 172.21.15.83:0/2851602963 shutdown_connections 2024-04-17T10:54:40.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.083+0000 7fe22fae5640 1 -- 172.21.15.83:0/2851602963 wait complete. 2024-04-17T10:54:40.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.083+0000 7fe22fae5640 1 Processor -- start 2024-04-17T10:54:40.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.083+0000 7fe22fae5640 1 -- start start 2024-04-17T10:54:40.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.083+0000 7fe22fae5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe22806cc80 0x7fe22815cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:40.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.083+0000 7fe22fae5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2280b4090 0x7fe228149880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:40.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.083+0000 7fe22fae5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe228154670 0x7fe2281473d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:40.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.083+0000 7fe22fae5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2280aef90 con 0x7fe228154670 2024-04-17T10:54:40.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.083+0000 7fe22fae5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22815b3c0 con 0x7fe22806cc80 2024-04-17T10:54:40.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.083+0000 7fe22fae5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2280af110 con 0x7fe2280b4090 2024-04-17T10:54:40.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.083+0000 7fe22e082640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe228154670 0x7fe2281473d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:40.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.083+0000 7fe22e082640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe228154670 0x7fe2281473d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57978/0 (socket says 172.21.15.83:57978) 2024-04-17T10:54:40.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.083+0000 7fe22e082640 1 -- 172.21.15.83:0/1423346263 learned_addr learned my addr 172.21.15.83:0/1423346263 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:40.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.087+0000 7fe22d080640 1 --2- 172.21.15.83:0/1423346263 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2280b4090 0x7fe228149880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:40.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.087+0000 7fe22e082640 1 -- 172.21.15.83:0/1423346263 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2280b4090 msgr2=0x7fe228149880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:40.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.087+0000 7fe22e082640 1 --2- 172.21.15.83:0/1423346263 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2280b4090 0x7fe228149880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:40.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.087+0000 7fe22d881640 1 --2- 172.21.15.83:0/1423346263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe22806cc80 0x7fe22815cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:40.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.087+0000 7fe22e082640 1 -- 172.21.15.83:0/1423346263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe22806cc80 msgr2=0x7fe22815cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:40.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.087+0000 7fe22e082640 1 --2- 172.21.15.83:0/1423346263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe22806cc80 0x7fe22815cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:40.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.087+0000 7fe22e082640 1 -- 172.21.15.83:0/1423346263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22814ffe0 con 0x7fe228154670 2024-04-17T10:54:40.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.087+0000 7fe22d881640 1 --2- 172.21.15.83:0/1423346263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe22806cc80 0x7fe22815cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:40.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.087+0000 7fe22e082640 1 --2- 172.21.15.83:0/1423346263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe228154670 0x7fe2281473d0 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7fe2200b0040 tx=0x7fe22001c2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:40.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.087+0000 7fe21e7fc640 1 -- 172.21.15.83:0/1423346263 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2200a6940 con 0x7fe228154670 2024-04-17T10:54:40.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.087+0000 7fe21e7fc640 1 -- 172.21.15.83:0/1423346263 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2200bd070 con 0x7fe228154670 2024-04-17T10:54:40.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.087+0000 7fe22fae5640 1 -- 172.21.15.83:0/1423346263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe22814fd00 con 0x7fe228154670 2024-04-17T10:54:40.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.087+0000 7fe21e7fc640 1 -- 172.21.15.83:0/1423346263 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe220009510 con 0x7fe228154670 2024-04-17T10:54:40.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.087+0000 7fe22fae5640 1 -- 172.21.15.83:0/1423346263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe22814e2b0 con 0x7fe228154670 2024-04-17T10:54:40.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.087+0000 7fe22fae5640 1 -- 172.21.15.83:0/1423346263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe228074f90 con 0x7fe228154670 2024-04-17T10:54:40.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.087+0000 7fe21e7fc640 1 -- 172.21.15.83:0/1423346263 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe2200b8660 con 0x7fe228154670 2024-04-17T10:54:40.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.087+0000 7fe21e7fc640 1 --2- 172.21.15.83:0/1423346263 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe20c066f60 0x7fe20c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:40.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.087+0000 7fe21e7fc640 1 -- 172.21.15.83:0/1423346263 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fe2201202c0 con 0x7fe228154670 2024-04-17T10:54:40.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.087+0000 7fe22d881640 1 --2- 172.21.15.83:0/1423346263 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe20c066f60 0x7fe20c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:40.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.087+0000 7fe22d881640 1 --2- 172.21.15.83:0/1423346263 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe20c066f60 0x7fe20c069420 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fe224006fd0 tx=0x7fe224008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:40.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.091+0000 7fe21e7fc640 1 -- 172.21.15.83:0/1423346263 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe220017730 con 0x7fe228154670 2024-04-17T10:54:40.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.255+0000 7fe22fae5640 1 -- 172.21.15.83:0/1423346263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe2280b2a00 con 0x7fe228154670 2024-04-17T10:54:40.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.255+0000 7fe21e7fc640 1 -- 172.21.15.83:0/1423346263 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v143) v1 ==== 80+0+122 (secure 0 0 0) 0x7fe2200f1260 con 0x7fe228154670 2024-04-17T10:54:40.250 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6837/3575114599 2024-04-18T10:51:04.828753+0000 2024-04-17T10:54:40.250 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6836/3575114599 2024-04-18T10:51:04.828753+0000 2024-04-17T10:54:40.250 INFO:teuthology.orchestra.run.smithi083.stderr:listed 2 entries 2024-04-17T10:54:40.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.259+0000 7fe22fae5640 1 -- 172.21.15.83:0/1423346263 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe20c066f60 msgr2=0x7fe20c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:40.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.259+0000 7fe22fae5640 1 --2- 172.21.15.83:0/1423346263 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe20c066f60 0x7fe20c069420 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fe224006fd0 tx=0x7fe224008040 comp rx=0 tx=0).stop 2024-04-17T10:54:40.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.259+0000 7fe22fae5640 1 -- 172.21.15.83:0/1423346263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe228154670 msgr2=0x7fe2281473d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:40.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.259+0000 7fe22fae5640 1 --2- 172.21.15.83:0/1423346263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe228154670 0x7fe2281473d0 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7fe2200b0040 tx=0x7fe22001c2c0 comp rx=0 tx=0).stop 2024-04-17T10:54:40.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.259+0000 7fe22fae5640 1 -- 172.21.15.83:0/1423346263 shutdown_connections 2024-04-17T10:54:40.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.259+0000 7fe22fae5640 1 --2- 172.21.15.83:0/1423346263 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe20c066f60 0x7fe20c069420 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:40.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.259+0000 7fe22fae5640 1 --2- 172.21.15.83:0/1423346263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe228154670 0x7fe2281473d0 unknown :-1 s=CLOSED pgs=2423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:40.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.259+0000 7fe22fae5640 1 --2- 172.21.15.83:0/1423346263 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2280b4090 0x7fe228149880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:40.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.259+0000 7fe22fae5640 1 --2- 172.21.15.83:0/1423346263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe22806cc80 0x7fe22815cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:40.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.259+0000 7fe22fae5640 1 -- 172.21.15.83:0/1423346263 >> 172.21.15.83:0/1423346263 conn(0x7fe2280b15b0 msgr2=0x7fe22815a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:40.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.259+0000 7fe22fae5640 1 -- 172.21.15.83:0/1423346263 shutdown_connections 2024-04-17T10:54:40.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.259+0000 7fe22fae5640 1 -- 172.21.15.83:0/1423346263 wait complete. 2024-04-17T10:54:40.267 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2024-04-17T10:54:40.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:54:40.267 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-17T10:54:40.277 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[83273]: fuse finished with error 0 and tester_r 0 2024-04-17T10:54:40.339 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-17T10:54:40.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:54:40.340 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T10:54:40.516 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:54:40.516 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:54:40.523 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:54:40.523 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-04-17T10:54:40.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:54:40.523 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-17T10:54:40.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:54:40.573 INFO:teuthology.orchestra.run.smithi167.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-17T10:54:40.573 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:54:40.573 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:54:40.578 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16063 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:54:40.579 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-17T10:54:40.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:54:40.579 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-17T10:54:40.628 INFO:teuthology.orchestra.run.smithi083.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-17T10:54:40.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:54:40.629 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:54:40.629 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:54:40.635 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:54:40.635 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-04-17T10:54:40.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:54:40.636 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-17T10:54:40.684 INFO:teuthology.orchestra.run.smithi167.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-17T10:54:40.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:54:40.685 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:54:40.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.819+0000 7f179bc0b640 1 Processor -- start 2024-04-17T10:54:40.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.819+0000 7f179bc0b640 1 -- start start 2024-04-17T10:54:40.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.823+0000 7f179bc0b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f179415db50 0x7f1794070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:40.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.823+0000 7f179bc0b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f17940aedd0 0x7f179406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:40.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.823+0000 7f179bc0b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17940709b0 con 0x7f179406dae0 2024-04-17T10:54:40.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.823+0000 7f179bc0b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f179415e440 con 0x7f179415db50 2024-04-17T10:54:40.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.823+0000 7f179bc0b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f179415e720 con 0x7f17940aedd0 2024-04-17T10:54:40.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.823+0000 7f17999a7640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f179415db50 0x7f1794070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:40.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.823+0000 7f179a1a8640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f179406dae0 0x7f17940cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57764/0 (socket says 172.21.15.83:57764) 2024-04-17T10:54:40.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.823+0000 7f179a1a8640 1 -- 172.21.15.83:0/1834592584 learned_addr learned my addr 172.21.15.83:0/1834592584 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:40.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.823+0000 7f17991a6640 1 --2- 172.21.15.83:0/1834592584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f17940aedd0 0x7f179406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:40.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.823+0000 7f17999a7640 1 -- 172.21.15.83:0/1834592584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f17940aedd0 msgr2=0x7f179406d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:40.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.823+0000 7f17999a7640 1 --2- 172.21.15.83:0/1834592584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f17940aedd0 0x7f179406d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:40.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.823+0000 7f17999a7640 1 -- 172.21.15.83:0/1834592584 >> v1:172.21.15.83:6789/0 conn(0x7f179406dae0 legacy=0x7f17940cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:40.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.823+0000 7f17989a5640 1 -- 172.21.15.83:0/1834592584 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060805823 0 0) 0x7f178c0014a0 con 0x7f179406dae0 2024-04-17T10:54:40.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.823+0000 7f17999a7640 1 -- 172.21.15.83:0/1834592584 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f179415ea00 con 0x7f179415db50 2024-04-17T10:54:40.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.823+0000 7f17991a6640 1 --2- 172.21.15.83:0/1834592584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f17940aedd0 0x7f179406d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:40.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.823+0000 7f17999a7640 1 --2- 172.21.15.83:0/1834592584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f179415db50 0x7f1794070470 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f17900097c0 tx=0x7f179001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4f3a26d5a8e777d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:40.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.823+0000 7f17989a5640 1 -- 172.21.15.83:0/1834592584 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17900b2070 con 0x7f179415db50 2024-04-17T10:54:40.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.823+0000 7f17989a5640 1 -- 172.21.15.83:0/1834592584 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17900ad640 con 0x7f179415db50 2024-04-17T10:54:40.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.823+0000 7f17989a5640 1 -- 172.21.15.83:0/1834592584 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17900ada10 con 0x7f179415db50 2024-04-17T10:54:40.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.827+0000 7f179bc0b640 1 -- 172.21.15.83:0/1834592584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f179415db50 msgr2=0x7f1794070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:40.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.827+0000 7f179bc0b640 1 --2- 172.21.15.83:0/1834592584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f179415db50 0x7f1794070470 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f17900097c0 tx=0x7f179001c4a0 comp rx=0 tx=0).stop 2024-04-17T10:54:40.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.827+0000 7f179bc0b640 1 -- 172.21.15.83:0/1834592584 shutdown_connections 2024-04-17T10:54:40.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.827+0000 7f179bc0b640 1 --2- 172.21.15.83:0/1834592584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f17940aedd0 0x7f179406d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:40.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.827+0000 7f179bc0b640 1 --2- 172.21.15.83:0/1834592584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f179415db50 0x7f1794070470 unknown :-1 s=CLOSED pgs=2425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:40.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.827+0000 7f179bc0b640 1 -- 172.21.15.83:0/1834592584 >> 172.21.15.83:0/1834592584 conn(0x7f179406abe0 msgr2=0x7f179406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:40.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.827+0000 7f179bc0b640 1 -- 172.21.15.83:0/1834592584 shutdown_connections 2024-04-17T10:54:40.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.827+0000 7f179bc0b640 1 -- 172.21.15.83:0/1834592584 wait complete. 2024-04-17T10:54:40.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.827+0000 7f179bc0b640 1 Processor -- start 2024-04-17T10:54:40.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.827+0000 7f179bc0b640 1 -- start start 2024-04-17T10:54:40.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.827+0000 7f179bc0b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f179406dae0 0x7f1794154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:40.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.827+0000 7f179bc0b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17940aedd0 0x7f17941442e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:40.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.827+0000 7f179bc0b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f179415db50 0x7f1794153c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:40.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.827+0000 7f179bc0b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17941657d0 con 0x7f17940aedd0 2024-04-17T10:54:40.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.827+0000 7f179bc0b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1794165650 con 0x7f179406dae0 2024-04-17T10:54:40.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.827+0000 7f179bc0b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1794165950 con 0x7f179415db50 2024-04-17T10:54:40.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.827+0000 7f17991a6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17940aedd0 0x7f17941442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:40.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.827+0000 7f17991a6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17940aedd0 0x7f17941442e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57994/0 (socket says 172.21.15.83:57994) 2024-04-17T10:54:40.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.827+0000 7f17991a6640 1 -- 172.21.15.83:0/635207801 learned_addr learned my addr 172.21.15.83:0/635207801 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:40.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.827+0000 7f17991a6640 1 -- 172.21.15.83:0/635207801 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f179415db50 msgr2=0x7f1794153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:40.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.827+0000 7f17991a6640 1 --2- 172.21.15.83:0/635207801 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f179415db50 0x7f1794153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:40.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.827+0000 7f17991a6640 1 -- 172.21.15.83:0/635207801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f179406dae0 msgr2=0x7f1794154860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:40.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.827+0000 7f17991a6640 1 --2- 172.21.15.83:0/635207801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f179406dae0 0x7f1794154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:40.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.827+0000 7f17991a6640 1 -- 172.21.15.83:0/635207801 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1794167040 con 0x7f17940aedd0 2024-04-17T10:54:40.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.827+0000 7f17991a6640 1 --2- 172.21.15.83:0/635207801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17940aedd0 0x7f17941442e0 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f178400c7f0 tx=0x7f178400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:40.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.827+0000 7f178a7fc640 1 -- 172.21.15.83:0/635207801 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17840079d0 con 0x7f17940aedd0 2024-04-17T10:54:40.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.827+0000 7f178a7fc640 1 -- 172.21.15.83:0/635207801 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1784007b70 con 0x7f17940aedd0 2024-04-17T10:54:40.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.827+0000 7f179bc0b640 1 -- 172.21.15.83:0/635207801 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1794167310 con 0x7f17940aedd0 2024-04-17T10:54:40.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.827+0000 7f178a7fc640 1 -- 172.21.15.83:0/635207801 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17840113f0 con 0x7f17940aedd0 2024-04-17T10:54:40.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.827+0000 7f179bc0b640 1 -- 172.21.15.83:0/635207801 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17941678d0 con 0x7f17940aedd0 2024-04-17T10:54:40.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.831+0000 7f179bc0b640 1 -- 172.21.15.83:0/635207801 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1794074f90 con 0x7f17940aedd0 2024-04-17T10:54:40.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.831+0000 7f178a7fc640 1 -- 172.21.15.83:0/635207801 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1784011590 con 0x7f17940aedd0 2024-04-17T10:54:40.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.831+0000 7f178a7fc640 1 --2- 172.21.15.83:0/635207801 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1778066f60 0x7f1778069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:40.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.831+0000 7f178a7fc640 1 -- 172.21.15.83:0/635207801 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f1784085230 con 0x7f17940aedd0 2024-04-17T10:54:40.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.831+0000 7f17999a7640 1 --2- 172.21.15.83:0/635207801 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1778066f60 0x7f1778069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:40.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.831+0000 7f17999a7640 1 --2- 172.21.15.83:0/635207801 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1778066f60 0x7f1778069420 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f17900af000 tx=0x7f1790004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:40.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:40.835+0000 7f178a7fc640 1 -- 172.21.15.83:0/635207801 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1784015020 con 0x7f17940aedd0 2024-04-17T10:54:41.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.011+0000 7f179bc0b640 1 -- 172.21.15.83:0/635207801 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17940b2af0 con 0x7f17940aedd0 2024-04-17T10:54:41.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.011+0000 7f178a7fc640 1 -- 172.21.15.83:0/635207801 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f17840561d0 con 0x7f17940aedd0 2024-04-17T10:54:41.007 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:41.007 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:54:41.007 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:54:41.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.015+0000 7f179bc0b640 1 -- 172.21.15.83:0/635207801 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1778066f60 msgr2=0x7f1778069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:41.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.015+0000 7f179bc0b640 1 --2- 172.21.15.83:0/635207801 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1778066f60 0x7f1778069420 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f17900af000 tx=0x7f1790004740 comp rx=0 tx=0).stop 2024-04-17T10:54:41.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.015+0000 7f179bc0b640 1 -- 172.21.15.83:0/635207801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17940aedd0 msgr2=0x7f17941442e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:41.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.015+0000 7f179bc0b640 1 --2- 172.21.15.83:0/635207801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17940aedd0 0x7f17941442e0 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f178400c7f0 tx=0x7f178400ccb0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.015+0000 7f179bc0b640 1 -- 172.21.15.83:0/635207801 shutdown_connections 2024-04-17T10:54:41.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.015+0000 7f179bc0b640 1 --2- 172.21.15.83:0/635207801 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1778066f60 0x7f1778069420 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.015+0000 7f179bc0b640 1 --2- 172.21.15.83:0/635207801 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f179415db50 0x7f1794153c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.015+0000 7f179bc0b640 1 --2- 172.21.15.83:0/635207801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17940aedd0 0x7f17941442e0 unknown :-1 s=CLOSED pgs=2426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.015+0000 7f179bc0b640 1 --2- 172.21.15.83:0/635207801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f179406dae0 0x7f1794154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.015+0000 7f179bc0b640 1 -- 172.21.15.83:0/635207801 >> 172.21.15.83:0/635207801 conn(0x7f179406abe0 msgr2=0x7f17940afbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:41.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.015+0000 7f179bc0b640 1 -- 172.21.15.83:0/635207801 shutdown_connections 2024-04-17T10:54:41.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.015+0000 7f179bc0b640 1 -- 172.21.15.83:0/635207801 wait complete. 2024-04-17T10:54:41.025 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:54:41.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.147+0000 7fab0fa1d640 1 Processor -- start 2024-04-17T10:54:41.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.147+0000 7fab0fa1d640 1 -- start start 2024-04-17T10:54:41.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.147+0000 7fab0fa1d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab080b5b40 0x7fab080d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:41.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.147+0000 7fab0fa1d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab08162a00 con 0x7fab0815dfa0 2024-04-17T10:54:41.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.147+0000 7fab0fa1d640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab08161e80 con 0x7fab080ce980 2024-04-17T10:54:41.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.147+0000 7fab0fa1d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab08162160 con 0x7fab080b5b40 2024-04-17T10:54:41.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.147+0000 7fab0d7b9640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fab0815dfa0 0x7fab080b6c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57768/0 (socket says 172.21.15.83:57768) 2024-04-17T10:54:41.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.147+0000 7fab0d7b9640 1 -- 172.21.15.83:0/3871640416 learned_addr learned my addr 172.21.15.83:0/3871640416 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:41.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.147+0000 7faafffff640 1 -- 172.21.15.83:0/3871640416 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 281628554 0 0) 0x7fab08162a00 con 0x7fab0815dfa0 2024-04-17T10:54:41.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.147+0000 7faafffff640 1 -- 172.21.15.83:0/3871640416 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faaf4003530 con 0x7fab0815dfa0 2024-04-17T10:54:41.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.147+0000 7fab0dfba640 1 --2- 172.21.15.83:0/3871640416 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab080b5b40 0x7fab080d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:41.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.147+0000 7faafffff640 1 -- 172.21.15.83:0/3871640416 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1923915294 0 0) 0x7faaf4003530 con 0x7fab0815dfa0 2024-04-17T10:54:41.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.147+0000 7faafffff640 1 -- 172.21.15.83:0/3871640416 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab080b5b40 msgr2=0x7fab080d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:41.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.147+0000 7faafffff640 1 --2- 172.21.15.83:0/3871640416 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab080b5b40 0x7fab080d1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.147+0000 7faafffff640 1 -- 172.21.15.83:0/3871640416 >> v1:172.21.15.167:6789/0 conn(0x7fab080ce980 legacy=0x7fab080b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:41.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.147+0000 7faafffff640 1 -- 172.21.15.83:0/3871640416 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab08162440 con 0x7fab0815dfa0 2024-04-17T10:54:41.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.147+0000 7faafffff640 1 -- 172.21.15.83:0/3871640416 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fab04001680 con 0x7fab0815dfa0 2024-04-17T10:54:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.147+0000 7faafffff640 1 -- 172.21.15.83:0/3871640416 >> v1:172.21.15.83:6789/0 conn(0x7fab0815dfa0 legacy=0x7fab080b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.147+0000 7faafffff640 1 --2- 172.21.15.83:0/3871640416 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab080b5b40 0x7faaf40039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.147+0000 7faafffff640 1 --2- 172.21.15.83:0/3871640416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faaf4004020 0x7faaf4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.147+0000 7faafffff640 1 --2- 172.21.15.83:0/3871640416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf4004d50 0x7faaf40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.147+0000 7faafffff640 1 -- 172.21.15.83:0/3871640416 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab08162160 con 0x7faaf4004d50 2024-04-17T10:54:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.147+0000 7faafffff640 1 -- 172.21.15.83:0/3871640416 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf400bc10 con 0x7faaf4004020 2024-04-17T10:54:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.147+0000 7faafffff640 1 -- 172.21.15.83:0/3871640416 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaf400be10 con 0x7fab080b5b40 2024-04-17T10:54:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.147+0000 7fab0dfba640 1 --2- 172.21.15.83:0/3871640416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf4004d50 0x7faaf40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.147+0000 7fab0cfb8640 1 --2- 172.21.15.83:0/3871640416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faaf4004020 0x7faaf4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.147+0000 7fab0d7b9640 1 --2- 172.21.15.83:0/3871640416 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab080b5b40 0x7faaf40039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.147+0000 7fab0dfba640 1 -- 172.21.15.83:0/3871640416 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab080b5b40 msgr2=0x7faaf40039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.147+0000 7fab0dfba640 1 --2- 172.21.15.83:0/3871640416 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab080b5b40 0x7faaf40039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.147+0000 7fab0dfba640 1 -- 172.21.15.83:0/3871640416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faaf4004020 msgr2=0x7faaf4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:41.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.147+0000 7fab0dfba640 1 --2- 172.21.15.83:0/3871640416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faaf4004020 0x7faaf4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.147+0000 7fab0dfba640 1 -- 172.21.15.83:0/3871640416 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf4003530 con 0x7faaf4004d50 2024-04-17T10:54:41.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.151+0000 7fab0dfba640 1 --2- 172.21.15.83:0/3871640416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf4004d50 0x7faaf40051d0 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7fab00009fc0 tx=0x7fab0001ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c029a668dc3c592 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:41.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.151+0000 7faafffff640 1 -- 172.21.15.83:0/3871640416 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab000b2070 con 0x7faaf4004d50 2024-04-17T10:54:41.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.151+0000 7faafffff640 1 -- 172.21.15.83:0/3871640416 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab000a5c10 con 0x7faaf4004d50 2024-04-17T10:54:41.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.151+0000 7faafffff640 1 -- 172.21.15.83:0/3871640416 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab00009bc0 con 0x7faaf4004d50 2024-04-17T10:54:41.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.151+0000 7fab0fa1d640 1 -- 172.21.15.83:0/3871640416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf4004d50 msgr2=0x7faaf40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:41.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.151+0000 7fab0fa1d640 1 --2- 172.21.15.83:0/3871640416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf4004d50 0x7faaf40051d0 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7fab00009fc0 tx=0x7fab0001ce00 comp rx=0 tx=0).stop 2024-04-17T10:54:41.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.151+0000 7fab0dfba640 1 -- 172.21.15.83:0/3871640416 reap_dead start 2024-04-17T10:54:41.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.151+0000 7fab0fa1d640 1 -- 172.21.15.83:0/3871640416 shutdown_connections 2024-04-17T10:54:41.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.151+0000 7fab0fa1d640 1 -- 172.21.15.83:0/3871640416 >> 172.21.15.83:0/3871640416 conn(0x7fab080d0600 msgr2=0x7fab080d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:41.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.151+0000 7fab0fa1d640 1 -- 172.21.15.83:0/3871640416 shutdown_connections 2024-04-17T10:54:41.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.151+0000 7fab0fa1d640 1 -- 172.21.15.83:0/3871640416 wait complete. 2024-04-17T10:54:41.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.151+0000 7fab0fa1d640 1 Processor -- start 2024-04-17T10:54:41.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.151+0000 7fab0fa1d640 1 -- start start 2024-04-17T10:54:41.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.151+0000 7fab0fa1d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab080c6560 0x7fab080c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:41.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.151+0000 7fab0fa1d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab080c6940 0x7fab080c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:41.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.151+0000 7fab0fa1d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab081f3840 0x7fab080cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:41.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.151+0000 7fab0fa1d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab08161b10 con 0x7fab080c6560 2024-04-17T10:54:41.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.151+0000 7fab0fa1d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0815e380 con 0x7fab081f3840 2024-04-17T10:54:41.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.151+0000 7fab0fa1d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab080cee00 con 0x7fab080c6940 2024-04-17T10:54:41.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.151+0000 7fab0d7b9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab080c6560 0x7fab080c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:41.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.151+0000 7fab0d7b9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab080c6560 0x7fab080c91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58006/0 (socket says 172.21.15.83:58006) 2024-04-17T10:54:41.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.151+0000 7fab0d7b9640 1 -- 172.21.15.83:0/1381549115 learned_addr learned my addr 172.21.15.83:0/1381549115 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:41.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.151+0000 7fab0cfb8640 1 --2- 172.21.15.83:0/1381549115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab080c6940 0x7fab080c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:41.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.151+0000 7fab0dfba640 1 --2- 172.21.15.83:0/1381549115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab081f3840 0x7fab080cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:41.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.151+0000 7fab0d7b9640 1 -- 172.21.15.83:0/1381549115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab080c6940 msgr2=0x7fab080c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.151+0000 7fab0d7b9640 1 --2- 172.21.15.83:0/1381549115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab080c6940 0x7fab080c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.151+0000 7fab0d7b9640 1 -- 172.21.15.83:0/1381549115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab081f3840 msgr2=0x7fab080cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.151+0000 7fab0d7b9640 1 --2- 172.21.15.83:0/1381549115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab081f3840 0x7fab080cbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.151+0000 7fab0d7b9640 1 -- 172.21.15.83:0/1381549115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab081f3db0 con 0x7fab080c6560 2024-04-17T10:54:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.151+0000 7fab0dfba640 1 --2- 172.21.15.83:0/1381549115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab081f3840 0x7fab080cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.155+0000 7fab0d7b9640 1 --2- 172.21.15.83:0/1381549115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab080c6560 0x7fab080c91c0 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7fab04011980 tx=0x7fab04011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.155+0000 7faafdffb640 1 -- 172.21.15.83:0/1381549115 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab04010040 con 0x7fab080c6560 2024-04-17T10:54:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.155+0000 7faafdffb640 1 -- 172.21.15.83:0/1381549115 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0400fc40 con 0x7fab080c6560 2024-04-17T10:54:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.155+0000 7fab0fa1d640 1 -- 172.21.15.83:0/1381549115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab081f4090 con 0x7fab080c6560 2024-04-17T10:54:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.155+0000 7faafdffb640 1 -- 172.21.15.83:0/1381549115 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab04015410 con 0x7fab080c6560 2024-04-17T10:54:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.155+0000 7fab0fa1d640 1 -- 172.21.15.83:0/1381549115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab081f4650 con 0x7fab080c6560 2024-04-17T10:54:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.155+0000 7fab0fa1d640 1 -- 172.21.15.83:0/1381549115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab08074f90 con 0x7fab080c6560 2024-04-17T10:54:41.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.155+0000 7faafdffb640 1 -- 172.21.15.83:0/1381549115 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fab040155b0 con 0x7fab080c6560 2024-04-17T10:54:41.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.155+0000 7faafdffb640 1 --2- 172.21.15.83:0/1381549115 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faaec066f60 0x7faaec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:41.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.155+0000 7faafdffb640 1 -- 172.21.15.83:0/1381549115 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fab04088630 con 0x7fab080c6560 2024-04-17T10:54:41.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.155+0000 7fab0cfb8640 1 --2- 172.21.15.83:0/1381549115 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faaec066f60 0x7faaec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:41.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.155+0000 7fab0cfb8640 1 --2- 172.21.15.83:0/1381549115 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faaec066f60 0x7faaec069420 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fab0815eb00 tx=0x7faaf8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:41.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.159+0000 7faafdffb640 1 -- 172.21.15.83:0/1381549115 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fab040595d0 con 0x7fab080c6560 2024-04-17T10:54:41.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.303+0000 7fab0fa1d640 1 -- 172.21.15.83:0/1381549115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab080d0ab0 con 0x7fab080c6560 2024-04-17T10:54:41.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.307+0000 7faafdffb640 1 -- 172.21.15.83:0/1381549115 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+16786 (secure 0 0 0) 0x7fab04058ed0 con 0x7fab080c6560 2024-04-17T10:54:41.300 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:41.301 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":143,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:51:04.828803+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:50:32.488186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-17T10:50:33.207211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.32","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/3575114599":"2024-04-18T10:51:04.828753+0000","172.21.15.83:6836/3575114599":"2024-04-18T10:51:04.828753+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:54:41.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.307+0000 7fab0fa1d640 1 -- 172.21.15.83:0/1381549115 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faaec066f60 msgr2=0x7faaec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:41.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.307+0000 7fab0fa1d640 1 --2- 172.21.15.83:0/1381549115 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faaec066f60 0x7faaec069420 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fab0815eb00 tx=0x7faaf8009040 comp rx=0 tx=0).stop 2024-04-17T10:54:41.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.307+0000 7fab0fa1d640 1 -- 172.21.15.83:0/1381549115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab080c6560 msgr2=0x7fab080c91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:41.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.307+0000 7fab0fa1d640 1 --2- 172.21.15.83:0/1381549115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab080c6560 0x7fab080c91c0 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7fab04011980 tx=0x7fab04011e40 comp rx=0 tx=0).stop 2024-04-17T10:54:41.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.307+0000 7fab0fa1d640 1 -- 172.21.15.83:0/1381549115 shutdown_connections 2024-04-17T10:54:41.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.307+0000 7fab0fa1d640 1 --2- 172.21.15.83:0/1381549115 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7faaec066f60 0x7faaec069420 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.307+0000 7fab0fa1d640 1 --2- 172.21.15.83:0/1381549115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab081f3840 0x7fab080cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.307+0000 7fab0fa1d640 1 --2- 172.21.15.83:0/1381549115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab080c6940 0x7fab080c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.307+0000 7fab0fa1d640 1 --2- 172.21.15.83:0/1381549115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab080c6560 0x7fab080c91c0 unknown :-1 s=CLOSED pgs=2429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.307+0000 7fab0fa1d640 1 -- 172.21.15.83:0/1381549115 >> 172.21.15.83:0/1381549115 conn(0x7fab080d0600 msgr2=0x7fab0815be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:41.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.307+0000 7fab0fa1d640 1 -- 172.21.15.83:0/1381549115 shutdown_connections 2024-04-17T10:54:41.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.307+0000 7fab0fa1d640 1 -- 172.21.15.83:0/1381549115 wait complete. 2024-04-17T10:54:41.318 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T10:38:00.250968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:50:32.488186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '139', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:50:33.207211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:54:41.318 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:54:41.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.447+0000 7f0bdb528640 1 Processor -- start 2024-04-17T10:54:41.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.447+0000 7f0bdb528640 1 -- start start 2024-04-17T10:54:41.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.451+0000 7f0bdb528640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0bd40b4210 0x7f0bd415e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:41.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.451+0000 7f0bdb528640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bd415bd70 con 0x7f0bd406b520 2024-04-17T10:54:41.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.451+0000 7f0bdb528640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bd415bf60 con 0x7f0bd4154de0 2024-04-17T10:54:41.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.451+0000 7f0bdb528640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bd40b4870 con 0x7f0bd40b4210 2024-04-17T10:54:41.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.451+0000 7f0bd8ac3640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0bd406b520 0x7f0bd40b3b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57774/0 (socket says 172.21.15.83:57774) 2024-04-17T10:54:41.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.451+0000 7f0bd8ac3640 1 -- 172.21.15.83:0/2972332769 learned_addr learned my addr 172.21.15.83:0/2972332769 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:41.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.451+0000 7f0bd9ac5640 1 --2- 172.21.15.83:0/2972332769 >> v2:172.21.15.167:3300/0 conn(0x7f0bd40b4210 0x7f0bd415e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:41.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.451+0000 7f0bcbfff640 1 -- 172.21.15.83:0/2972332769 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1386348342 0 0) 0x7f0bd415bd70 con 0x7f0bd406b520 2024-04-17T10:54:41.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.451+0000 7f0bcbfff640 1 -- 172.21.15.83:0/2972332769 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bc0005740 con 0x7f0bd406b520 2024-04-17T10:54:41.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.451+0000 7f0bcbfff640 1 -- 172.21.15.83:0/2972332769 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3331923226 0 0) 0x7f0bc0005740 con 0x7f0bd406b520 2024-04-17T10:54:41.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.451+0000 7f0bcbfff640 1 -- 172.21.15.83:0/2972332769 >> v2:172.21.15.167:3300/0 conn(0x7f0bd40b4210 msgr2=0x7f0bd415e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:41.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.451+0000 7f0bcbfff640 1 --2- 172.21.15.83:0/2972332769 >> v2:172.21.15.167:3300/0 conn(0x7f0bd40b4210 0x7f0bd415e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.451+0000 7f0bcbfff640 1 -- 172.21.15.83:0/2972332769 >> v1:172.21.15.167:6789/0 conn(0x7f0bd4154de0 legacy=0x7f0bd406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:41.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.451+0000 7f0bcbfff640 1 -- 172.21.15.83:0/2972332769 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd415afb0 con 0x7f0bd406b520 2024-04-17T10:54:41.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.451+0000 7f0bcbfff640 1 -- 172.21.15.83:0/2972332769 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f0bc4001680 con 0x7f0bd406b520 2024-04-17T10:54:41.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.451+0000 7f0bcbfff640 1 -- 172.21.15.83:0/2972332769 >> v1:172.21.15.83:6789/0 conn(0x7f0bd406b520 legacy=0x7f0bd40b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:41.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.451+0000 7f0bcbfff640 1 --2- 172.21.15.83:0/2972332769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc0005c40 0x7f0bc0006060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:41.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.451+0000 7f0bcbfff640 1 --2- 172.21.15.83:0/2972332769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bc0006a20 0x7f0bc0006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:41.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.451+0000 7f0bcbfff640 1 --2- 172.21.15.83:0/2972332769 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bc00073e0 0x7f0bc000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:41.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.451+0000 7f0bcbfff640 1 -- 172.21.15.83:0/2972332769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc000e2e0 con 0x7f0bc0006a20 2024-04-17T10:54:41.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.451+0000 7f0bcbfff640 1 -- 172.21.15.83:0/2972332769 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc000e490 con 0x7f0bc0005c40 2024-04-17T10:54:41.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.451+0000 7f0bcbfff640 1 -- 172.21.15.83:0/2972332769 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc000e690 con 0x7f0bc00073e0 2024-04-17T10:54:41.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.451+0000 7f0bd8ac3640 1 --2- 172.21.15.83:0/2972332769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bc0006a20 0x7f0bc0006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:41.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.451+0000 7f0bd92c4640 1 --2- 172.21.15.83:0/2972332769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc0005c40 0x7f0bc0006060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:41.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.451+0000 7f0bd9ac5640 1 --2- 172.21.15.83:0/2972332769 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bc00073e0 0x7f0bc000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:41.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.451+0000 7f0bd8ac3640 1 -- 172.21.15.83:0/2972332769 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bc00073e0 msgr2=0x7f0bc000dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:41.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.451+0000 7f0bd8ac3640 1 --2- 172.21.15.83:0/2972332769 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bc00073e0 0x7f0bc000dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.451+0000 7f0bd8ac3640 1 -- 172.21.15.83:0/2972332769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc0005c40 msgr2=0x7f0bc0006060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:41.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.451+0000 7f0bd8ac3640 1 --2- 172.21.15.83:0/2972332769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc0005c40 0x7f0bc0006060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.451+0000 7f0bd8ac3640 1 -- 172.21.15.83:0/2972332769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc0005740 con 0x7f0bc0006a20 2024-04-17T10:54:41.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.451+0000 7f0bd9ac5640 1 -- 172.21.15.83:0/2972332769 reap_dead start 2024-04-17T10:54:41.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bd8ac3640 1 --2- 172.21.15.83:0/2972332769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bc0006a20 0x7f0bc0006ea0 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400c240 tx=0x7f0bc401f2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc906a57a8509bb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:41.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bcbfff640 1 -- 172.21.15.83:0/2972332769 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4007d20 con 0x7f0bc0006a20 2024-04-17T10:54:41.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bcbfff640 1 -- 172.21.15.83:0/2972332769 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bc4005ae0 con 0x7f0bc0006a20 2024-04-17T10:54:41.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bcbfff640 1 -- 172.21.15.83:0/2972332769 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4007070 con 0x7f0bc0006a20 2024-04-17T10:54:41.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bdb528640 1 -- 172.21.15.83:0/2972332769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bc0006a20 msgr2=0x7f0bc0006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:41.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bdb528640 1 --2- 172.21.15.83:0/2972332769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bc0006a20 0x7f0bc0006ea0 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400c240 tx=0x7f0bc401f2f0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bdb528640 1 -- 172.21.15.83:0/2972332769 shutdown_connections 2024-04-17T10:54:41.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bdb528640 1 --2- 172.21.15.83:0/2972332769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bc0006a20 0x7f0bc0006ea0 unknown :-1 s=CLOSED pgs=2431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bdb528640 1 -- 172.21.15.83:0/2972332769 >> 172.21.15.83:0/2972332769 conn(0x7f0bd40b15b0 msgr2=0x7f0bd40b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:41.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bdb528640 1 -- 172.21.15.83:0/2972332769 shutdown_connections 2024-04-17T10:54:41.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bdb528640 1 -- 172.21.15.83:0/2972332769 wait complete. 2024-04-17T10:54:41.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bdb528640 1 Processor -- start 2024-04-17T10:54:41.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bdb528640 1 -- start start 2024-04-17T10:54:41.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bdb528640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bc0006a20 0x7f0bd41481d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:41.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bdb528640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bd4156f20 0x7f0bd41573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:41.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bdb528640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bd40ad840 0x7f0bd40adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:41.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bdb528640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd415b850 con 0x7f0bc0006a20 2024-04-17T10:54:41.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bdb528640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd415b570 con 0x7f0bd4156f20 2024-04-17T10:54:41.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bdb528640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd41551c0 con 0x7f0bd40ad840 2024-04-17T10:54:41.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bd92c4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bc0006a20 0x7f0bd41481d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:41.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bd92c4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bc0006a20 0x7f0bd41481d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58030/0 (socket says 172.21.15.83:58030) 2024-04-17T10:54:41.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bd92c4640 1 -- 172.21.15.83:0/75262973 learned_addr learned my addr 172.21.15.83:0/75262973 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:41.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bd8ac3640 1 --2- 172.21.15.83:0/75262973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bd4156f20 0x7f0bd41573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:41.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bd9ac5640 1 --2- 172.21.15.83:0/75262973 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bd40ad840 0x7f0bd40adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:41.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bd92c4640 1 -- 172.21.15.83:0/75262973 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bd40ad840 msgr2=0x7f0bd40adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:41.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bd92c4640 1 --2- 172.21.15.83:0/75262973 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bd40ad840 0x7f0bd40adcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bd92c4640 1 -- 172.21.15.83:0/75262973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bd4156f20 msgr2=0x7f0bd41573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:41.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bd92c4640 1 --2- 172.21.15.83:0/75262973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bd4156f20 0x7f0bd41573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bd92c4640 1 -- 172.21.15.83:0/75262973 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd41579e0 con 0x7f0bc0006a20 2024-04-17T10:54:41.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bd9ac5640 1 --2- 172.21.15.83:0/75262973 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bd40ad840 0x7f0bd40adcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:41.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bd92c4640 1 --2- 172.21.15.83:0/75262973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bc0006a20 0x7f0bd41481d0 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000e4b0 tx=0x7f0bd000e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:41.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bc9ffb640 1 -- 172.21.15.83:0/75262973 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd00052e0 con 0x7f0bc0006a20 2024-04-17T10:54:41.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bc9ffb640 1 -- 172.21.15.83:0/75262973 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd0018070 con 0x7f0bc0006a20 2024-04-17T10:54:41.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bc9ffb640 1 -- 172.21.15.83:0/75262973 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0005480 con 0x7f0bc0006a20 2024-04-17T10:54:41.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.455+0000 7f0bdb528640 1 -- 172.21.15.83:0/75262973 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bd4157cc0 con 0x7f0bc0006a20 2024-04-17T10:54:41.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.459+0000 7f0bdb528640 1 -- 172.21.15.83:0/75262973 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd4152030 con 0x7f0bc0006a20 2024-04-17T10:54:41.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.459+0000 7f0bdb528640 1 -- 172.21.15.83:0/75262973 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bd4074f90 con 0x7f0bc0006a20 2024-04-17T10:54:41.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.459+0000 7f0bc9ffb640 1 -- 172.21.15.83:0/75262973 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0bd0003840 con 0x7f0bc0006a20 2024-04-17T10:54:41.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.459+0000 7f0bc9ffb640 1 --2- 172.21.15.83:0/75262973 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0bb8066f90 0x7f0bb8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:41.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.459+0000 7f0bc9ffb640 1 -- 172.21.15.83:0/75262973 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f0bd0084cb0 con 0x7f0bc0006a20 2024-04-17T10:54:41.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.459+0000 7f0bd8ac3640 1 --2- 172.21.15.83:0/75262973 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0bb8066f90 0x7f0bb8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:41.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.459+0000 7f0bd8ac3640 1 --2- 172.21.15.83:0/75262973 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0bb8066f90 0x7f0bb8069450 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40af000 tx=0x7f0bc40aa000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:41.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.463+0000 7f0bc9ffb640 1 -- 172.21.15.83:0/75262973 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0bd0055c50 con 0x7f0bc0006a20 2024-04-17T10:54:41.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.623+0000 7f0bdb528640 1 -- 172.21.15.83:0/75262973 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0bd4071ef0 con 0x7f0bc0006a20 2024-04-17T10:54:41.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.623+0000 7f0bc9ffb640 1 -- 172.21.15.83:0/75262973 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v143) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0bd0055550 con 0x7f0bc0006a20 2024-04-17T10:54:41.617 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-17T10:54:41.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.623+0000 7f0bdb528640 1 -- 172.21.15.83:0/75262973 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0bb8066f90 msgr2=0x7f0bb8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:41.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.623+0000 7f0bdb528640 1 --2- 172.21.15.83:0/75262973 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0bb8066f90 0x7f0bb8069450 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40af000 tx=0x7f0bc40aa000 comp rx=0 tx=0).stop 2024-04-17T10:54:41.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.623+0000 7f0bdb528640 1 -- 172.21.15.83:0/75262973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bc0006a20 msgr2=0x7f0bd41481d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:41.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.623+0000 7f0bdb528640 1 --2- 172.21.15.83:0/75262973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bc0006a20 0x7f0bd41481d0 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000e4b0 tx=0x7f0bd000e970 comp rx=0 tx=0).stop 2024-04-17T10:54:41.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.623+0000 7f0bdb528640 1 -- 172.21.15.83:0/75262973 shutdown_connections 2024-04-17T10:54:41.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.623+0000 7f0bdb528640 1 --2- 172.21.15.83:0/75262973 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0bb8066f90 0x7f0bb8069450 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.623+0000 7f0bdb528640 1 --2- 172.21.15.83:0/75262973 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bd40ad840 0x7f0bd40adcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.623+0000 7f0bdb528640 1 --2- 172.21.15.83:0/75262973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bd4156f20 0x7f0bd41573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.623+0000 7f0bdb528640 1 --2- 172.21.15.83:0/75262973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bc0006a20 0x7f0bd41481d0 unknown :-1 s=CLOSED pgs=2432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.623+0000 7f0bdb528640 1 -- 172.21.15.83:0/75262973 >> 172.21.15.83:0/75262973 conn(0x7f0bd40b15b0 msgr2=0x7f0bd40cfd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:41.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.623+0000 7f0bdb528640 1 -- 172.21.15.83:0/75262973 shutdown_connections 2024-04-17T10:54:41.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.623+0000 7f0bdb528640 1 -- 172.21.15.83:0/75262973 wait complete. 2024-04-17T10:54:41.635 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:54:41.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.755+0000 7f75ab572640 1 Processor -- start 2024-04-17T10:54:41.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.755+0000 7f75ab572640 1 -- start start 2024-04-17T10:54:41.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.755+0000 7f75ab572640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75a40ce400 0x7f75a40b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:41.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.755+0000 7f75ab572640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75a415e4d0 con 0x7f75a415e0f0 2024-04-17T10:54:41.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.755+0000 7f75ab572640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75a4162580 con 0x7f75a40b5450 2024-04-17T10:54:41.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.755+0000 7f75ab572640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a4162b40 con 0x7f75a40ce400 2024-04-17T10:54:41.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.755+0000 7f75a930e640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f75a415e0f0 0x7f75a4161c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57786/0 (socket says 172.21.15.83:57786) 2024-04-17T10:54:41.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.755+0000 7f75a930e640 1 -- 172.21.15.83:0/513061207 learned_addr learned my addr 172.21.15.83:0/513061207 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:41.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.755+0000 7f75a8b0d640 1 --2- 172.21.15.83:0/513061207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75a40ce400 0x7f75a40b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:41.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.755+0000 7f759bfff640 1 -- 172.21.15.83:0/513061207 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1538543239 0 0) 0x7f75a415e4d0 con 0x7f75a415e0f0 2024-04-17T10:54:41.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.755+0000 7f759bfff640 1 -- 172.21.15.83:0/513061207 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7590003530 con 0x7f75a415e0f0 2024-04-17T10:54:41.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.755+0000 7f759bfff640 1 -- 172.21.15.83:0/513061207 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 58668725 0 0) 0x7f7590003530 con 0x7f75a415e0f0 2024-04-17T10:54:41.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.755+0000 7f759bfff640 1 -- 172.21.15.83:0/513061207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75a40ce400 msgr2=0x7f75a40b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:41.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.755+0000 7f759bfff640 1 --2- 172.21.15.83:0/513061207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75a40ce400 0x7f75a40b6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.755+0000 7f759bfff640 1 -- 172.21.15.83:0/513061207 >> v1:172.21.15.167:6789/0 conn(0x7f75a40b5450 legacy=0x7f75a40d2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:41.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.755+0000 7f759bfff640 1 -- 172.21.15.83:0/513061207 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a4162860 con 0x7f75a415e0f0 2024-04-17T10:54:41.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.755+0000 7f759bfff640 1 -- 172.21.15.83:0/513061207 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f75a0001680 con 0x7f75a415e0f0 2024-04-17T10:54:41.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.755+0000 7f759bfff640 1 -- 172.21.15.83:0/513061207 >> v1:172.21.15.83:6789/0 conn(0x7f75a415e0f0 legacy=0x7f75a4161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:41.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.755+0000 7f759bfff640 1 --2- 172.21.15.83:0/513061207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7590003a60 0x7f7590003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:41.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.755+0000 7f759bfff640 1 --2- 172.21.15.83:0/513061207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7590004810 0x7f7590004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:41.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.755+0000 7f759bfff640 1 --2- 172.21.15.83:0/513061207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75a40ce400 0x7f7590005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:41.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.755+0000 7f759bfff640 1 -- 172.21.15.83:0/513061207 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a4162b40 con 0x7f7590003a60 2024-04-17T10:54:41.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.755+0000 7f759bfff640 1 -- 172.21.15.83:0/513061207 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759000bb90 con 0x7f7590004810 2024-04-17T10:54:41.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.755+0000 7f759bfff640 1 -- 172.21.15.83:0/513061207 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759000bd90 con 0x7f75a40ce400 2024-04-17T10:54:41.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.755+0000 7f75a930e640 1 --2- 172.21.15.83:0/513061207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7590003a60 0x7f7590003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:41.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.755+0000 7f75a8b0d640 1 --2- 172.21.15.83:0/513061207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7590004810 0x7f7590004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:41.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.755+0000 7f75a930e640 1 -- 172.21.15.83:0/513061207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75a40ce400 msgr2=0x7f7590005280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:41.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.755+0000 7f75a930e640 1 --2- 172.21.15.83:0/513061207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75a40ce400 0x7f7590005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.755+0000 7f75a930e640 1 -- 172.21.15.83:0/513061207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7590004810 msgr2=0x7f7590004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:41.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.755+0000 7f75a930e640 1 --2- 172.21.15.83:0/513061207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7590004810 0x7f7590004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.755+0000 7f75a930e640 1 -- 172.21.15.83:0/513061207 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7590003530 con 0x7f7590003a60 2024-04-17T10:54:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.759+0000 7f75a930e640 1 --2- 172.21.15.83:0/513061207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7590003a60 0x7f7590003e80 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7f75a0001ec0 tx=0x7f75a001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57780bab36ad8265 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.759+0000 7f759bfff640 1 -- 172.21.15.83:0/513061207 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a0007d40 con 0x7f7590003a60 2024-04-17T10:54:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.759+0000 7f759bfff640 1 -- 172.21.15.83:0/513061207 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a00a9070 con 0x7f7590003a60 2024-04-17T10:54:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.759+0000 7f759bfff640 1 -- 172.21.15.83:0/513061207 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a0005d40 con 0x7f7590003a60 2024-04-17T10:54:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.759+0000 7f75ab572640 1 -- 172.21.15.83:0/513061207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7590003a60 msgr2=0x7f7590003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.759+0000 7f75ab572640 1 --2- 172.21.15.83:0/513061207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7590003a60 0x7f7590003e80 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7f75a0001ec0 tx=0x7f75a001cc60 comp rx=0 tx=0).stop 2024-04-17T10:54:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.759+0000 7f75a9b0f640 1 -- 172.21.15.83:0/513061207 reap_dead start 2024-04-17T10:54:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.759+0000 7f75ab572640 1 -- 172.21.15.83:0/513061207 shutdown_connections 2024-04-17T10:54:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.759+0000 7f75ab572640 1 -- 172.21.15.83:0/513061207 >> 172.21.15.83:0/513061207 conn(0x7f75a40d0450 msgr2=0x7f75a40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.759+0000 7f75ab572640 1 -- 172.21.15.83:0/513061207 shutdown_connections 2024-04-17T10:54:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.759+0000 7f75ab572640 1 -- 172.21.15.83:0/513061207 wait complete. 2024-04-17T10:54:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.759+0000 7f75ab572640 1 Processor -- start 2024-04-17T10:54:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.759+0000 7f75ab572640 1 -- start start 2024-04-17T10:54:41.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.759+0000 7f75ab572640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7590003a60 0x7f75a40cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:41.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.759+0000 7f75ab572640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75a4150c70 0x7f75a40cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:41.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.759+0000 7f75ab572640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75a406cce0 0x7f75a41500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:41.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.759+0000 7f75ab572640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a40b5dc0 con 0x7f7590003a60 2024-04-17T10:54:41.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.759+0000 7f75ab572640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a40b5c40 con 0x7f75a406cce0 2024-04-17T10:54:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.759+0000 7f75ab572640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a4169f50 con 0x7f75a4150c70 2024-04-17T10:54:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.759+0000 7f75a930e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7590003a60 0x7f75a40cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.759+0000 7f75a930e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7590003a60 0x7f75a40cbd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58046/0 (socket says 172.21.15.83:58046) 2024-04-17T10:54:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.759+0000 7f75a930e640 1 -- 172.21.15.83:0/91339207 learned_addr learned my addr 172.21.15.83:0/91339207 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.759+0000 7f75a9b0f640 1 --2- 172.21.15.83:0/91339207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75a406cce0 0x7f75a41500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.759+0000 7f75a8b0d640 1 --2- 172.21.15.83:0/91339207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75a4150c70 0x7f75a40cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.759+0000 7f75a930e640 1 -- 172.21.15.83:0/91339207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75a4150c70 msgr2=0x7f75a40cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.759+0000 7f75a930e640 1 --2- 172.21.15.83:0/91339207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75a4150c70 0x7f75a40cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.759+0000 7f75a930e640 1 -- 172.21.15.83:0/91339207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75a406cce0 msgr2=0x7f75a41500d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.759+0000 7f75a930e640 1 --2- 172.21.15.83:0/91339207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75a406cce0 0x7f75a41500d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.759+0000 7f75a930e640 1 -- 172.21.15.83:0/91339207 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a40cab40 con 0x7f7590003a60 2024-04-17T10:54:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.759+0000 7f75a930e640 1 --2- 172.21.15.83:0/91339207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7590003a60 0x7f75a40cbd10 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f75a001ca90 tx=0x7f7590005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.763+0000 7f7599ffb640 1 -- 172.21.15.83:0/91339207 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a0007d20 con 0x7f7590003a60 2024-04-17T10:54:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.763+0000 7f7599ffb640 1 -- 172.21.15.83:0/91339207 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a0005d20 con 0x7f7590003a60 2024-04-17T10:54:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.763+0000 7f75ab572640 1 -- 172.21.15.83:0/91339207 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a40cb550 con 0x7f7590003a60 2024-04-17T10:54:41.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.763+0000 7f7599ffb640 1 -- 172.21.15.83:0/91339207 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a00a5a30 con 0x7f7590003a60 2024-04-17T10:54:41.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.763+0000 7f75ab572640 1 -- 172.21.15.83:0/91339207 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a40cade0 con 0x7f7590003a60 2024-04-17T10:54:41.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.763+0000 7f75ab572640 1 -- 172.21.15.83:0/91339207 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75a40681a0 con 0x7f7590003a60 2024-04-17T10:54:41.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.763+0000 7f7599ffb640 1 -- 172.21.15.83:0/91339207 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f75a00a5bd0 con 0x7f7590003a60 2024-04-17T10:54:41.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.763+0000 7f7599ffb640 1 --2- 172.21.15.83:0/91339207 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7588066f60 0x7f7588069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.763+0000 7f75a8b0d640 1 --2- 172.21.15.83:0/91339207 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7588066f60 0x7f7588069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.763+0000 7f7599ffb640 1 -- 172.21.15.83:0/91339207 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f75a011e800 con 0x7f7590003a60 2024-04-17T10:54:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.763+0000 7f75a8b0d640 1 --2- 172.21.15.83:0/91339207 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7588066f60 0x7f7588069420 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f75a40d4d00 tx=0x7f7594008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:41.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.767+0000 7f7599ffb640 1 -- 172.21.15.83:0/91339207 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f75a0016970 con 0x7f7590003a60 2024-04-17T10:54:41.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.923+0000 7f75ab572640 1 -- 172.21.15.83:0/91339207 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f75a4074f90 con 0x7f7590003a60 2024-04-17T10:54:41.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.923+0000 7f7599ffb640 1 -- 172.21.15.83:0/91339207 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v143) v1 ==== 104+0+11 (secure 0 0 0) 0x7f75a00b8380 con 0x7f7590003a60 2024-04-17T10:54:41.916 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-17T10:54:41.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.923+0000 7f75ab572640 1 -- 172.21.15.83:0/91339207 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7588066f60 msgr2=0x7f7588069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:41.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.923+0000 7f75ab572640 1 --2- 172.21.15.83:0/91339207 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7588066f60 0x7f7588069420 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f75a40d4d00 tx=0x7f7594008040 comp rx=0 tx=0).stop 2024-04-17T10:54:41.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.923+0000 7f75ab572640 1 -- 172.21.15.83:0/91339207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7590003a60 msgr2=0x7f75a40cbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:41.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.923+0000 7f75ab572640 1 --2- 172.21.15.83:0/91339207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7590003a60 0x7f75a40cbd10 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f75a001ca90 tx=0x7f7590005840 comp rx=0 tx=0).stop 2024-04-17T10:54:41.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.923+0000 7f75ab572640 1 -- 172.21.15.83:0/91339207 shutdown_connections 2024-04-17T10:54:41.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.923+0000 7f75ab572640 1 --2- 172.21.15.83:0/91339207 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7588066f60 0x7f7588069420 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.923+0000 7f75ab572640 1 --2- 172.21.15.83:0/91339207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75a406cce0 0x7f75a41500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.923+0000 7f75ab572640 1 --2- 172.21.15.83:0/91339207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75a4150c70 0x7f75a40cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.923+0000 7f75ab572640 1 --2- 172.21.15.83:0/91339207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7590003a60 0x7f75a40cbd10 unknown :-1 s=CLOSED pgs=2435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:41.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.923+0000 7f75ab572640 1 -- 172.21.15.83:0/91339207 >> 172.21.15.83:0/91339207 conn(0x7f75a40d0450 msgr2=0x7f75a40b7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:41.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.923+0000 7f75ab572640 1 -- 172.21.15.83:0/91339207 shutdown_connections 2024-04-17T10:54:41.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:41.923+0000 7f75ab572640 1 -- 172.21.15.83:0/91339207 wait complete. 2024-04-17T10:54:41.933 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:54:42.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.051+0000 7f01f8a2d640 1 Processor -- start 2024-04-17T10:54:42.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.051+0000 7f01f8a2d640 1 -- start start 2024-04-17T10:54:42.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.051+0000 7f01f8a2d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01f415b6e0 0x7f01f4071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:42.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.051+0000 7f01f8a2d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f01f4070470 0x7f01f406ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:42.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.051+0000 7f01f8a2d640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f406c1f0 con 0x7f01f415bac0 2024-04-17T10:54:42.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.051+0000 7f01f8a2d640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f4070a10 con 0x7f01f4070470 2024-04-17T10:54:42.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.051+0000 7f01f8a2d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f4070cf0 con 0x7f01f415b6e0 2024-04-17T10:54:42.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.051+0000 7f01f2d9d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f01f4070470 0x7f01f406ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:42.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.051+0000 7f01f1d9b640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f01f415bac0 0x7f01f40cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:55194/0 (socket says 172.21.15.83:55194) 2024-04-17T10:54:42.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.051+0000 7f01f1d9b640 1 -- 172.21.15.83:0/4106376805 learned_addr learned my addr 172.21.15.83:0/4106376805 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:42.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.051+0000 7f01f259c640 1 --2- 172.21.15.83:0/4106376805 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01f415b6e0 0x7f01f4071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:42.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.051+0000 7f01f259c640 1 -- 172.21.15.83:0/4106376805 >> v2:172.21.15.167:3300/0 conn(0x7f01f4070470 msgr2=0x7f01f406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:42.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.051+0000 7f01f259c640 1 --2- 172.21.15.83:0/4106376805 >> v2:172.21.15.167:3300/0 conn(0x7f01f4070470 0x7f01f406ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.051+0000 7f01f259c640 1 -- 172.21.15.83:0/4106376805 >> v1:172.21.15.167:6789/0 conn(0x7f01f415bac0 legacy=0x7f01f40cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:42.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.051+0000 7f01f259c640 1 -- 172.21.15.83:0/4106376805 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f4070fd0 con 0x7f01f415b6e0 2024-04-17T10:54:42.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.055+0000 7f01f259c640 1 --2- 172.21.15.83:0/4106376805 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01f415b6e0 0x7f01f4071cc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f01e40097c0 tx=0x7f01e401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f62b31f7433443a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:42.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.055+0000 7f01f159a640 1 -- 172.21.15.83:0/4106376805 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e40b0070 con 0x7f01f415b6e0 2024-04-17T10:54:42.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.055+0000 7f01f159a640 1 -- 172.21.15.83:0/4106376805 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e40ab600 con 0x7f01f415b6e0 2024-04-17T10:54:42.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.055+0000 7f01f159a640 1 -- 172.21.15.83:0/4106376805 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e40ab9d0 con 0x7f01f415b6e0 2024-04-17T10:54:42.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.055+0000 7f01f8a2d640 1 -- 172.21.15.83:0/4106376805 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01f415b6e0 msgr2=0x7f01f4071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:42.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.055+0000 7f01f8a2d640 1 --2- 172.21.15.83:0/4106376805 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01f415b6e0 0x7f01f4071cc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f01e40097c0 tx=0x7f01e401c4a0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.055+0000 7f01f8a2d640 1 -- 172.21.15.83:0/4106376805 shutdown_connections 2024-04-17T10:54:42.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.055+0000 7f01f8a2d640 1 --2- 172.21.15.83:0/4106376805 >> v2:172.21.15.167:3300/0 conn(0x7f01f4070470 0x7f01f406ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.055+0000 7f01f8a2d640 1 --2- 172.21.15.83:0/4106376805 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01f415b6e0 0x7f01f4071cc0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.055+0000 7f01f8a2d640 1 -- 172.21.15.83:0/4106376805 >> 172.21.15.83:0/4106376805 conn(0x7f01f40b15b0 msgr2=0x7f01f40b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:42.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.055+0000 7f01f8a2d640 1 -- 172.21.15.83:0/4106376805 shutdown_connections 2024-04-17T10:54:42.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.055+0000 7f01f8a2d640 1 -- 172.21.15.83:0/4106376805 wait complete. 2024-04-17T10:54:42.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.055+0000 7f01f8a2d640 1 Processor -- start 2024-04-17T10:54:42.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.055+0000 7f01f8a2d640 1 -- start start 2024-04-17T10:54:42.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.059+0000 7f01f8a2d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01f4070470 0x7f01f4154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:42.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.059+0000 7f01f8a2d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01f415b6e0 0x7f01f40afd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:42.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.059+0000 7f01f8a2d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01f415bac0 0x7f01f40b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:42.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.059+0000 7f01f8a2d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f406d270 con 0x7f01f415bac0 2024-04-17T10:54:42.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.059+0000 7f01f8a2d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f406d0f0 con 0x7f01f4070470 2024-04-17T10:54:42.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.059+0000 7f01f8a2d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f406b760 con 0x7f01f415b6e0 2024-04-17T10:54:42.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.059+0000 7f01f2d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01f415bac0 0x7f01f40b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:42.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.059+0000 7f01f2d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01f415bac0 0x7f01f40b02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58050/0 (socket says 172.21.15.83:58050) 2024-04-17T10:54:42.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.059+0000 7f01f2d9d640 1 -- 172.21.15.83:0/3761089025 learned_addr learned my addr 172.21.15.83:0/3761089025 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:42.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.059+0000 7f01f259c640 1 --2- 172.21.15.83:0/3761089025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01f4070470 0x7f01f4154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:42.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.059+0000 7f01f2d9d640 1 -- 172.21.15.83:0/3761089025 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01f415b6e0 msgr2=0x7f01f40afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:42.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.059+0000 7f01f1d9b640 1 --2- 172.21.15.83:0/3761089025 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01f415b6e0 0x7f01f40afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:42.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.059+0000 7f01f2d9d640 1 --2- 172.21.15.83:0/3761089025 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01f415b6e0 0x7f01f40afd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.059+0000 7f01f2d9d640 1 -- 172.21.15.83:0/3761089025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01f4070470 msgr2=0x7f01f4154fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:42.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.059+0000 7f01f2d9d640 1 --2- 172.21.15.83:0/3761089025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01f4070470 0x7f01f4154fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.059+0000 7f01f2d9d640 1 -- 172.21.15.83:0/3761089025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f414ffe0 con 0x7f01f415bac0 2024-04-17T10:54:42.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.059+0000 7f01f2d9d640 1 --2- 172.21.15.83:0/3761089025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01f415bac0 0x7f01f40b02a0 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009dd0 tx=0x7f01ec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:42.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.059+0000 7f01e2ffd640 1 -- 172.21.15.83:0/3761089025 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec018020 con 0x7f01f415bac0 2024-04-17T10:54:42.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.059+0000 7f01e2ffd640 1 -- 172.21.15.83:0/3761089025 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ec014480 con 0x7f01f415bac0 2024-04-17T10:54:42.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.059+0000 7f01f8a2d640 1 -- 172.21.15.83:0/3761089025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f41502c0 con 0x7f01f415bac0 2024-04-17T10:54:42.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.059+0000 7f01e2ffd640 1 -- 172.21.15.83:0/3761089025 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec014870 con 0x7f01f415bac0 2024-04-17T10:54:42.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.059+0000 7f01f8a2d640 1 -- 172.21.15.83:0/3761089025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f414b690 con 0x7f01f415bac0 2024-04-17T10:54:42.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.059+0000 7f01f8a2d640 1 -- 172.21.15.83:0/3761089025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f4074f90 con 0x7f01f415bac0 2024-04-17T10:54:42.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.059+0000 7f01e2ffd640 1 -- 172.21.15.83:0/3761089025 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f01ec01b020 con 0x7f01f415bac0 2024-04-17T10:54:42.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.059+0000 7f01e2ffd640 1 --2- 172.21.15.83:0/3761089025 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f01d4066f60 0x7f01d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:42.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.059+0000 7f01f259c640 1 --2- 172.21.15.83:0/3761089025 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f01d4066f60 0x7f01d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:42.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.059+0000 7f01e2ffd640 1 -- 172.21.15.83:0/3761089025 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f01ec085b10 con 0x7f01f415bac0 2024-04-17T10:54:42.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.063+0000 7f01f259c640 1 --2- 172.21.15.83:0/3761089025 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f01d4066f60 0x7f01d4069420 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f01e40ad000 tx=0x7f01e40a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:42.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.063+0000 7f01e2ffd640 1 -- 172.21.15.83:0/3761089025 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f01ec022070 con 0x7f01f415bac0 2024-04-17T10:54:42.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.211+0000 7f01f8a2d640 1 -- 172.21.15.83:0/3761089025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f01f4153320 con 0x7f01f415bac0 2024-04-17T10:54:42.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.211+0000 7f01e2ffd640 1 -- 172.21.15.83:0/3761089025 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v143) v1 ==== 100+0+11 (secure 0 0 0) 0x7f01ec010040 con 0x7f01f415bac0 2024-04-17T10:54:42.205 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-17T10:54:42.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.215+0000 7f01f8a2d640 1 -- 172.21.15.83:0/3761089025 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f01d4066f60 msgr2=0x7f01d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:42.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.215+0000 7f01f8a2d640 1 --2- 172.21.15.83:0/3761089025 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f01d4066f60 0x7f01d4069420 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f01e40ad000 tx=0x7f01e40a6000 comp rx=0 tx=0).stop 2024-04-17T10:54:42.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.215+0000 7f01f8a2d640 1 -- 172.21.15.83:0/3761089025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01f415bac0 msgr2=0x7f01f40b02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:42.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.215+0000 7f01f8a2d640 1 --2- 172.21.15.83:0/3761089025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01f415bac0 0x7f01f40b02a0 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009dd0 tx=0x7f01ec00c490 comp rx=0 tx=0).stop 2024-04-17T10:54:42.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.215+0000 7f01f8a2d640 1 -- 172.21.15.83:0/3761089025 shutdown_connections 2024-04-17T10:54:42.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.215+0000 7f01f8a2d640 1 --2- 172.21.15.83:0/3761089025 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f01d4066f60 0x7f01d4069420 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.215+0000 7f01f8a2d640 1 --2- 172.21.15.83:0/3761089025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01f415bac0 0x7f01f40b02a0 unknown :-1 s=CLOSED pgs=2436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.215+0000 7f01f8a2d640 1 --2- 172.21.15.83:0/3761089025 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01f415b6e0 0x7f01f40afd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.215+0000 7f01f8a2d640 1 --2- 172.21.15.83:0/3761089025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01f4070470 0x7f01f4154fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.215+0000 7f01f8a2d640 1 -- 172.21.15.83:0/3761089025 >> 172.21.15.83:0/3761089025 conn(0x7f01f40b15b0 msgr2=0x7f01f415ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:42.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.215+0000 7f01f8a2d640 1 -- 172.21.15.83:0/3761089025 shutdown_connections 2024-04-17T10:54:42.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.215+0000 7f01f8a2d640 1 -- 172.21.15.83:0/3761089025 wait complete. 2024-04-17T10:54:42.223 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:54:42.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.335+0000 7fe2e7729640 1 Processor -- start 2024-04-17T10:54:42.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.335+0000 7fe2e7729640 1 -- start start 2024-04-17T10:54:42.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.335+0000 7fe2e7729640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe2e015dfa0 0x7fe2e00b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:42.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.335+0000 7fe2e7729640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2e00ce980 0x7fe2e00b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:42.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.335+0000 7fe2e7729640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e0162a00 con 0x7fe2e00cee00 2024-04-17T10:54:42.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.335+0000 7fe2e7729640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2e0161e80 con 0x7fe2e015dfa0 2024-04-17T10:54:42.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.335+0000 7fe2e7729640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e0162160 con 0x7fe2e00ce980 2024-04-17T10:54:42.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.335+0000 7fe2e54c5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe2e015dfa0 0x7fe2e00b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:42.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.335+0000 7fe2e54c5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe2e015dfa0 0x7fe2e00b6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58064/0 (socket says 172.21.15.83:58064) 2024-04-17T10:54:42.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.335+0000 7fe2e54c5640 1 -- 172.21.15.83:0/2212779380 learned_addr learned my addr 172.21.15.83:0/2212779380 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:42.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.339+0000 7fe2e4cc4640 1 --2- 172.21.15.83:0/2212779380 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2e00ce980 0x7fe2e00b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:42.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.339+0000 7fe2e54c5640 1 -- 172.21.15.83:0/2212779380 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2e00ce980 msgr2=0x7fe2e00b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:42.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.339+0000 7fe2e54c5640 1 --2- 172.21.15.83:0/2212779380 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2e00ce980 0x7fe2e00b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.339+0000 7fe2e54c5640 1 -- 172.21.15.83:0/2212779380 >> v1:172.21.15.167:6789/0 conn(0x7fe2e00cee00 legacy=0x7fe2e00d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:42.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.339+0000 7fe2e54c5640 1 -- 172.21.15.83:0/2212779380 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e0162440 con 0x7fe2e015dfa0 2024-04-17T10:54:42.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.339+0000 7fe2e54c5640 1 --2- 172.21.15.83:0/2212779380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2e015dfa0 0x7fe2e00b6c20 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc009870 tx=0x7fe2dc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fea7d8418042cba server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:42.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.339+0000 7fe2d7fff640 1 -- 172.21.15.83:0/2212779380 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc0b0070 con 0x7fe2e015dfa0 2024-04-17T10:54:42.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.339+0000 7fe2d7fff640 1 -- 172.21.15.83:0/2212779380 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2dc0ab600 con 0x7fe2e015dfa0 2024-04-17T10:54:42.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.339+0000 7fe2d7fff640 1 -- 172.21.15.83:0/2212779380 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc0ab9d0 con 0x7fe2e015dfa0 2024-04-17T10:54:42.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.339+0000 7fe2e7729640 1 -- 172.21.15.83:0/2212779380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2e015dfa0 msgr2=0x7fe2e00b6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:42.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.339+0000 7fe2e7729640 1 --2- 172.21.15.83:0/2212779380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2e015dfa0 0x7fe2e00b6c20 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc009870 tx=0x7fe2dc01c460 comp rx=0 tx=0).stop 2024-04-17T10:54:42.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.339+0000 7fe2e7729640 1 -- 172.21.15.83:0/2212779380 shutdown_connections 2024-04-17T10:54:42.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.339+0000 7fe2e7729640 1 --2- 172.21.15.83:0/2212779380 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2e00ce980 0x7fe2e00b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.339+0000 7fe2e7729640 1 --2- 172.21.15.83:0/2212779380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2e015dfa0 0x7fe2e00b6c20 unknown :-1 s=CLOSED pgs=2437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.339+0000 7fe2e7729640 1 -- 172.21.15.83:0/2212779380 >> 172.21.15.83:0/2212779380 conn(0x7fe2e00d0600 msgr2=0x7fe2e00d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:42.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.339+0000 7fe2e7729640 1 -- 172.21.15.83:0/2212779380 shutdown_connections 2024-04-17T10:54:42.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.339+0000 7fe2e7729640 1 -- 172.21.15.83:0/2212779380 wait complete. 2024-04-17T10:54:42.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.343+0000 7fe2e7729640 1 Processor -- start 2024-04-17T10:54:42.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.343+0000 7fe2e7729640 1 -- start start 2024-04-17T10:54:42.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.343+0000 7fe2e7729640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2e00ce980 0x7fe2e00c74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:42.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.343+0000 7fe2e7729640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2e00cee00 0x7fe2e014cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:42.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.343+0000 7fe2e7729640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2e015dfa0 0x7fe2e015faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:42.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.343+0000 7fe2e7729640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e0161b10 con 0x7fe2e00cee00 2024-04-17T10:54:42.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.343+0000 7fe2e7729640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e00d2850 con 0x7fe2e00ce980 2024-04-17T10:54:42.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.343+0000 7fe2e7729640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e016b660 con 0x7fe2e015dfa0 2024-04-17T10:54:42.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.343+0000 7fe2e4cc4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2e00cee00 0x7fe2e014cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:42.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.343+0000 7fe2e4cc4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2e00cee00 0x7fe2e014cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58080/0 (socket says 172.21.15.83:58080) 2024-04-17T10:54:42.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.343+0000 7fe2e4cc4640 1 -- 172.21.15.83:0/1847298126 learned_addr learned my addr 172.21.15.83:0/1847298126 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:42.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.343+0000 7fe2e4cc4640 1 -- 172.21.15.83:0/1847298126 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2e015dfa0 msgr2=0x7fe2e015faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:42.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.343+0000 7fe2e54c5640 1 --2- 172.21.15.83:0/1847298126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2e00ce980 0x7fe2e00c74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:42.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.343+0000 7fe2e5cc6640 1 --2- 172.21.15.83:0/1847298126 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2e015dfa0 0x7fe2e015faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:42.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.343+0000 7fe2e4cc4640 1 --2- 172.21.15.83:0/1847298126 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2e015dfa0 0x7fe2e015faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.343+0000 7fe2e4cc4640 1 -- 172.21.15.83:0/1847298126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2e00ce980 msgr2=0x7fe2e00c74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:42.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.343+0000 7fe2e4cc4640 1 --2- 172.21.15.83:0/1847298126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2e00ce980 0x7fe2e00c74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.343+0000 7fe2e4cc4640 1 -- 172.21.15.83:0/1847298126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e00c7aa0 con 0x7fe2e00cee00 2024-04-17T10:54:42.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.343+0000 7fe2e4cc4640 1 --2- 172.21.15.83:0/1847298126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2e00cee00 0x7fe2e014cf70 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7fe2d000c7f0 tx=0x7fe2d000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:42.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.343+0000 7fe2d5ffb640 1 -- 172.21.15.83:0/1847298126 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0015070 con 0x7fe2e00cee00 2024-04-17T10:54:42.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.343+0000 7fe2d5ffb640 1 -- 172.21.15.83:0/1847298126 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d0007820 con 0x7fe2e00cee00 2024-04-17T10:54:42.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.343+0000 7fe2e7729640 1 -- 172.21.15.83:0/1847298126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e00c8470 con 0x7fe2e00cee00 2024-04-17T10:54:42.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.343+0000 7fe2d5ffb640 1 -- 172.21.15.83:0/1847298126 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0007c10 con 0x7fe2e00cee00 2024-04-17T10:54:42.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.343+0000 7fe2e7729640 1 -- 172.21.15.83:0/1847298126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e0168a20 con 0x7fe2e00cee00 2024-04-17T10:54:42.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.343+0000 7fe2e7729640 1 -- 172.21.15.83:0/1847298126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e0074f90 con 0x7fe2e00cee00 2024-04-17T10:54:42.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.343+0000 7fe2d5ffb640 1 -- 172.21.15.83:0/1847298126 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe2d0018070 con 0x7fe2e00cee00 2024-04-17T10:54:42.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.343+0000 7fe2d5ffb640 1 --2- 172.21.15.83:0/1847298126 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe2c4066f60 0x7fe2c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.343+0000 7fe2d5ffb640 1 -- 172.21.15.83:0/1847298126 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fe2d0085130 con 0x7fe2e00cee00 2024-04-17T10:54:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.343+0000 7fe2e54c5640 1 --2- 172.21.15.83:0/1847298126 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe2c4066f60 0x7fe2c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.347+0000 7fe2e54c5640 1 --2- 172.21.15.83:0/1847298126 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe2c4066f60 0x7fe2c4069420 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc0ad000 tx=0x7fe2dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:42.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.351+0000 7fe2d5ffb640 1 -- 172.21.15.83:0/1847298126 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe2d00560d0 con 0x7fe2e00cee00 2024-04-17T10:54:42.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.527+0000 7fe2e7729640 1 -- 172.21.15.83:0/1847298126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2e00d17a0 con 0x7fe2e00cee00 2024-04-17T10:54:42.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.527+0000 7fe2d5ffb640 1 -- 172.21.15.83:0/1847298126 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fe2d00559d0 con 0x7fe2e00cee00 2024-04-17T10:54:42.522 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:42.523 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:54:42.523 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:54:42.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.531+0000 7fe2e7729640 1 -- 172.21.15.83:0/1847298126 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe2c4066f60 msgr2=0x7fe2c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:42.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.531+0000 7fe2e7729640 1 --2- 172.21.15.83:0/1847298126 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe2c4066f60 0x7fe2c4069420 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc0ad000 tx=0x7fe2dc008040 comp rx=0 tx=0).stop 2024-04-17T10:54:42.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.531+0000 7fe2e7729640 1 -- 172.21.15.83:0/1847298126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2e00cee00 msgr2=0x7fe2e014cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:42.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.531+0000 7fe2e7729640 1 --2- 172.21.15.83:0/1847298126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2e00cee00 0x7fe2e014cf70 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7fe2d000c7f0 tx=0x7fe2d000ccb0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.531+0000 7fe2e7729640 1 -- 172.21.15.83:0/1847298126 shutdown_connections 2024-04-17T10:54:42.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.531+0000 7fe2e7729640 1 --2- 172.21.15.83:0/1847298126 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fe2c4066f60 0x7fe2c4069420 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.531+0000 7fe2e7729640 1 --2- 172.21.15.83:0/1847298126 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2e015dfa0 0x7fe2e015faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.531+0000 7fe2e7729640 1 --2- 172.21.15.83:0/1847298126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2e00cee00 0x7fe2e014cf70 unknown :-1 s=CLOSED pgs=2438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.531+0000 7fe2e7729640 1 --2- 172.21.15.83:0/1847298126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2e00ce980 0x7fe2e00c74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.531+0000 7fe2e7729640 1 -- 172.21.15.83:0/1847298126 >> 172.21.15.83:0/1847298126 conn(0x7fe2e00d0600 msgr2=0x7fe2e00cc510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:42.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.531+0000 7fe2e7729640 1 -- 172.21.15.83:0/1847298126 shutdown_connections 2024-04-17T10:54:42.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.531+0000 7fe2e7729640 1 -- 172.21.15.83:0/1847298126 wait complete. 2024-04-17T10:54:42.540 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:54:42.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.667+0000 7f2922673640 1 Processor -- start 2024-04-17T10:54:42.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.667+0000 7f2922673640 1 -- start start 2024-04-17T10:54:42.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.671+0000 7f2922673640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f291c15ba30 0x7f291c072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:42.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.671+0000 7f2922673640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f291c15c960 0x7f291c0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:42.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.671+0000 7f2922673640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f291c0cfea0 con 0x7f291c0cfa10 2024-04-17T10:54:42.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.671+0000 7f2922673640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f291c071ea0 con 0x7f291c15c960 2024-04-17T10:54:42.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.671+0000 7f2922673640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f291c06e320 con 0x7f291c15ba30 2024-04-17T10:54:42.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.671+0000 7f291b7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f291c15c960 0x7f291c0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:42.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.671+0000 7f291b7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f291c15c960 0x7f291c0cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58082/0 (socket says 172.21.15.83:58082) 2024-04-17T10:54:42.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.671+0000 7f291b7fe640 1 -- 172.21.15.83:0/823901426 learned_addr learned my addr 172.21.15.83:0/823901426 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:42.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.671+0000 7f291bfff640 1 --2- 172.21.15.83:0/823901426 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f291c15ba30 0x7f291c072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:42.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.671+0000 7f291b7fe640 1 -- 172.21.15.83:0/823901426 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f291c15ba30 msgr2=0x7f291c072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:42.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.671+0000 7f291b7fe640 1 --2- 172.21.15.83:0/823901426 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f291c15ba30 0x7f291c072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.671+0000 7f291b7fe640 1 -- 172.21.15.83:0/823901426 >> v1:172.21.15.167:6789/0 conn(0x7f291c0cfa10 legacy=0x7f291c06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:42.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.671+0000 7f291b7fe640 1 -- 172.21.15.83:0/823901426 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f291c06e040 con 0x7f291c15c960 2024-04-17T10:54:42.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.671+0000 7f291b7fe640 1 --2- 172.21.15.83:0/823901426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f291c15c960 0x7f291c0cf4d0 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f2910009870 tx=0x7f291001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e189d6e93ad513d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:42.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.671+0000 7f291affd640 1 -- 172.21.15.83:0/823901426 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29100b0070 con 0x7f291c15c960 2024-04-17T10:54:42.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.671+0000 7f291affd640 1 -- 172.21.15.83:0/823901426 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29100ab600 con 0x7f291c15c960 2024-04-17T10:54:42.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.671+0000 7f291affd640 1 -- 172.21.15.83:0/823901426 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29100ab9d0 con 0x7f291c15c960 2024-04-17T10:54:42.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.671+0000 7f2922673640 1 -- 172.21.15.83:0/823901426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f291c15c960 msgr2=0x7f291c0cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:42.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.671+0000 7f2922673640 1 --2- 172.21.15.83:0/823901426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f291c15c960 0x7f291c0cf4d0 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f2910009870 tx=0x7f291001c490 comp rx=0 tx=0).stop 2024-04-17T10:54:42.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.671+0000 7f2922673640 1 -- 172.21.15.83:0/823901426 shutdown_connections 2024-04-17T10:54:42.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.671+0000 7f2922673640 1 --2- 172.21.15.83:0/823901426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f291c15c960 0x7f291c0cf4d0 unknown :-1 s=CLOSED pgs=2439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.671+0000 7f2922673640 1 --2- 172.21.15.83:0/823901426 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f291c15ba30 0x7f291c072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.671+0000 7f2922673640 1 -- 172.21.15.83:0/823901426 >> 172.21.15.83:0/823901426 conn(0x7f291c06abe0 msgr2=0x7f291c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:42.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.671+0000 7f2922673640 1 -- 172.21.15.83:0/823901426 shutdown_connections 2024-04-17T10:54:42.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.671+0000 7f2922673640 1 -- 172.21.15.83:0/823901426 wait complete. 2024-04-17T10:54:42.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.675+0000 7f2922673640 1 Processor -- start 2024-04-17T10:54:42.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.675+0000 7f2922673640 1 -- start start 2024-04-17T10:54:42.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.675+0000 7f2922673640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f291c0cfa10 0x7f291c0b2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:42.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.675+0000 7f2922673640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f291c15ba30 0x7f291c142120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:42.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.675+0000 7f2922673640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f291c15c960 0x7f291c0b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:42.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.675+0000 7f2922673640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291c06dca0 con 0x7f291c0cfa10 2024-04-17T10:54:42.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.675+0000 7f2922673640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291c0aeae0 con 0x7f291c15c960 2024-04-17T10:54:42.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.675+0000 7f2922673640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f291c06de20 con 0x7f291c15ba30 2024-04-17T10:54:42.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.675+0000 7f291bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f291c0cfa10 0x7f291c0b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:42.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.675+0000 7f291bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f291c0cfa10 0x7f291c0b2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58096/0 (socket says 172.21.15.83:58096) 2024-04-17T10:54:42.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.675+0000 7f291bfff640 1 -- 172.21.15.83:0/3659296436 learned_addr learned my addr 172.21.15.83:0/3659296436 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:42.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.675+0000 7f2920c10640 1 --2- 172.21.15.83:0/3659296436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f291c15c960 0x7f291c0b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:42.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.675+0000 7f291b7fe640 1 --2- 172.21.15.83:0/3659296436 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f291c15ba30 0x7f291c142120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:42.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.675+0000 7f291bfff640 1 -- 172.21.15.83:0/3659296436 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f291c15ba30 msgr2=0x7f291c142120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:42.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.675+0000 7f291bfff640 1 --2- 172.21.15.83:0/3659296436 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f291c15ba30 0x7f291c142120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.675+0000 7f291bfff640 1 -- 172.21.15.83:0/3659296436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f291c15c960 msgr2=0x7f291c0b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:42.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.675+0000 7f291bfff640 1 --2- 172.21.15.83:0/3659296436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f291c15c960 0x7f291c0b40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.675+0000 7f291bfff640 1 -- 172.21.15.83:0/3659296436 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f291c0b2d60 con 0x7f291c0cfa10 2024-04-17T10:54:42.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.675+0000 7f291b7fe640 1 --2- 172.21.15.83:0/3659296436 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f291c15ba30 0x7f291c142120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:42.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.675+0000 7f291bfff640 1 --2- 172.21.15.83:0/3659296436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f291c0cfa10 0x7f291c0b2820 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f290c00d800 tx=0x7f290c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:42.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.675+0000 7f2918ff9640 1 -- 172.21.15.83:0/3659296436 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c00b6c0 con 0x7f291c0cfa10 2024-04-17T10:54:42.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.675+0000 7f2918ff9640 1 -- 172.21.15.83:0/3659296436 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290c019070 con 0x7f291c0cfa10 2024-04-17T10:54:42.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.675+0000 7f2922673640 1 -- 172.21.15.83:0/3659296436 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f291c0b3000 con 0x7f291c0cfa10 2024-04-17T10:54:42.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.675+0000 7f2918ff9640 1 -- 172.21.15.83:0/3659296436 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c00ba60 con 0x7f291c0cfa10 2024-04-17T10:54:42.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.675+0000 7f2922673640 1 -- 172.21.15.83:0/3659296436 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f291c0b35c0 con 0x7f291c0cfa10 2024-04-17T10:54:42.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.675+0000 7f2922673640 1 -- 172.21.15.83:0/3659296436 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f291c074f90 con 0x7f291c0cfa10 2024-04-17T10:54:42.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.679+0000 7f2918ff9640 1 -- 172.21.15.83:0/3659296436 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f290c00bc00 con 0x7f291c0cfa10 2024-04-17T10:54:42.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.679+0000 7f2918ff9640 1 --2- 172.21.15.83:0/3659296436 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2908066f60 0x7f2908069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:42.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.679+0000 7f291b7fe640 1 --2- 172.21.15.83:0/3659296436 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2908066f60 0x7f2908069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:42.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.679+0000 7f2918ff9640 1 -- 172.21.15.83:0/3659296436 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f290c0854b0 con 0x7f291c0cfa10 2024-04-17T10:54:42.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.679+0000 7f291b7fe640 1 --2- 172.21.15.83:0/3659296436 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2908066f60 0x7f2908069420 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f291c06d810 tx=0x7f2910008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:42.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.683+0000 7f2918ff9640 1 -- 172.21.15.83:0/3659296436 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f290c0564e0 con 0x7f291c0cfa10 2024-04-17T10:54:42.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.835+0000 7f2922673640 1 -- 172.21.15.83:0/3659296436 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f291c152cc0 con 0x7f291c0cfa10 2024-04-17T10:54:42.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.839+0000 7f2918ff9640 1 -- 172.21.15.83:0/3659296436 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+16786 (secure 0 0 0) 0x7f290c055de0 con 0x7f291c0cfa10 2024-04-17T10:54:42.833 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:42.833 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":143,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:51:04.828803+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:50:32.488186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-17T10:50:33.207211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.32","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/3575114599":"2024-04-18T10:51:04.828753+0000","172.21.15.83:6836/3575114599":"2024-04-18T10:51:04.828753+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:54:42.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.843+0000 7f2922673640 1 -- 172.21.15.83:0/3659296436 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2908066f60 msgr2=0x7f2908069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:42.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.843+0000 7f2922673640 1 --2- 172.21.15.83:0/3659296436 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2908066f60 0x7f2908069420 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f291c06d810 tx=0x7f2910008040 comp rx=0 tx=0).stop 2024-04-17T10:54:42.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.843+0000 7f2922673640 1 -- 172.21.15.83:0/3659296436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f291c0cfa10 msgr2=0x7f291c0b2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:42.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.843+0000 7f2922673640 1 --2- 172.21.15.83:0/3659296436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f291c0cfa10 0x7f291c0b2820 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f290c00d800 tx=0x7f290c00dcc0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.843+0000 7f2922673640 1 -- 172.21.15.83:0/3659296436 shutdown_connections 2024-04-17T10:54:42.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.843+0000 7f2922673640 1 --2- 172.21.15.83:0/3659296436 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2908066f60 0x7f2908069420 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.843+0000 7f2922673640 1 --2- 172.21.15.83:0/3659296436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f291c15c960 0x7f291c0b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.843+0000 7f2922673640 1 --2- 172.21.15.83:0/3659296436 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f291c15ba30 0x7f291c142120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.843+0000 7f2922673640 1 --2- 172.21.15.83:0/3659296436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f291c0cfa10 0x7f291c0b2820 unknown :-1 s=CLOSED pgs=2440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.843+0000 7f2922673640 1 -- 172.21.15.83:0/3659296436 >> 172.21.15.83:0/3659296436 conn(0x7f291c06abe0 msgr2=0x7f291c15d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:42.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.843+0000 7f2922673640 1 -- 172.21.15.83:0/3659296436 shutdown_connections 2024-04-17T10:54:42.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.843+0000 7f2922673640 1 -- 172.21.15.83:0/3659296436 wait complete. 2024-04-17T10:54:42.851 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-17T10:54:42.852 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:54:42.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.963+0000 7ff31d4cd640 1 Processor -- start 2024-04-17T10:54:42.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.963+0000 7ff31d4cd640 1 -- start start 2024-04-17T10:54:42.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.967+0000 7ff31d4cd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff318146b40 0x7ff31806d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:42.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.967+0000 7ff31d4cd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff31806abe0 0x7ff31806b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:42.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.967+0000 7ff31d4cd640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff318161ae0 con 0x7ff318146f20 2024-04-17T10:54:42.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.967+0000 7ff31d4cd640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff31806b570 con 0x7ff318146b40 2024-04-17T10:54:42.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.967+0000 7ff31d4cd640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3180727e0 con 0x7ff31806abe0 2024-04-17T10:54:42.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.967+0000 7ff316ffd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff318146b40 0x7ff31806d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:42.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.967+0000 7ff30e7fc640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7ff318146f20 0x7ff3180720d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:55214/0 (socket says 172.21.15.83:55214) 2024-04-17T10:54:42.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.967+0000 7ff30e7fc640 1 -- 172.21.15.83:0/2384154692 learned_addr learned my addr 172.21.15.83:0/2384154692 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:42.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.967+0000 7ff3177fe640 1 --2- 172.21.15.83:0/2384154692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff31806abe0 0x7ff31806b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:42.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.967+0000 7ff3177fe640 1 -- 172.21.15.83:0/2384154692 >> v2:172.21.15.167:3300/0 conn(0x7ff318146b40 msgr2=0x7ff31806d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:42.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.967+0000 7ff3177fe640 1 --2- 172.21.15.83:0/2384154692 >> v2:172.21.15.167:3300/0 conn(0x7ff318146b40 0x7ff31806d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.967+0000 7ff3177fe640 1 -- 172.21.15.83:0/2384154692 >> v1:172.21.15.167:6789/0 conn(0x7ff318146f20 legacy=0x7ff3180720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:42.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.967+0000 7ff3167fc640 1 -- 172.21.15.83:0/2384154692 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 538633334 0 0) 0x7ff318161ae0 con 0x7ff318146f20 2024-04-17T10:54:42.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.967+0000 7ff3177fe640 1 -- 172.21.15.83:0/2384154692 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff318161dc0 con 0x7ff31806abe0 2024-04-17T10:54:42.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.967+0000 7ff316ffd640 1 --2- 172.21.15.83:0/2384154692 >> v2:172.21.15.167:3300/0 conn(0x7ff318146b40 0x7ff31806d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:54:42.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.971+0000 7ff3177fe640 1 --2- 172.21.15.83:0/2384154692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff31806abe0 0x7ff31806b030 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff310009dc0 tx=0x7ff31001c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0135ba3cd8131cd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:42.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.971+0000 7ff3167fc640 1 -- 172.21.15.83:0/2384154692 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3100b2070 con 0x7ff31806abe0 2024-04-17T10:54:42.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.971+0000 7ff3167fc640 1 -- 172.21.15.83:0/2384154692 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3100a5900 con 0x7ff31806abe0 2024-04-17T10:54:42.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.971+0000 7ff3167fc640 1 -- 172.21.15.83:0/2384154692 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3100a5cd0 con 0x7ff31806abe0 2024-04-17T10:54:42.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.971+0000 7ff31d4cd640 1 -- 172.21.15.83:0/2384154692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff31806abe0 msgr2=0x7ff31806b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:42.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.971+0000 7ff31d4cd640 1 --2- 172.21.15.83:0/2384154692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff31806abe0 0x7ff31806b030 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff310009dc0 tx=0x7ff31001c950 comp rx=0 tx=0).stop 2024-04-17T10:54:42.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.971+0000 7ff31d4cd640 1 -- 172.21.15.83:0/2384154692 shutdown_connections 2024-04-17T10:54:42.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.971+0000 7ff31d4cd640 1 --2- 172.21.15.83:0/2384154692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff31806abe0 0x7ff31806b030 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.971+0000 7ff31d4cd640 1 --2- 172.21.15.83:0/2384154692 >> v2:172.21.15.167:3300/0 conn(0x7ff318146b40 0x7ff31806d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.971+0000 7ff31d4cd640 1 -- 172.21.15.83:0/2384154692 >> 172.21.15.83:0/2384154692 conn(0x7ff31806bcf0 msgr2=0x7ff31806c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:42.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.971+0000 7ff31d4cd640 1 -- 172.21.15.83:0/2384154692 shutdown_connections 2024-04-17T10:54:42.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.971+0000 7ff31d4cd640 1 -- 172.21.15.83:0/2384154692 wait complete. 2024-04-17T10:54:42.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.971+0000 7ff31d4cd640 1 Processor -- start 2024-04-17T10:54:42.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.971+0000 7ff31d4cd640 1 -- start start 2024-04-17T10:54:42.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.971+0000 7ff31d4cd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff31806abe0 0x7ff31815b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:42.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.971+0000 7ff31d4cd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff318146b40 0x7ff3180b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:42.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.971+0000 7ff31d4cd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff318146f20 0x7ff3180b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:42.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.971+0000 7ff31d4cd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3180d1140 con 0x7ff318146b40 2024-04-17T10:54:42.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.971+0000 7ff31d4cd640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3180d0fc0 con 0x7ff31806abe0 2024-04-17T10:54:42.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.971+0000 7ff31d4cd640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3180d12c0 con 0x7ff318146f20 2024-04-17T10:54:42.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.971+0000 7ff30e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff318146b40 0x7ff3180b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:42.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.971+0000 7ff30e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff318146b40 0x7ff3180b0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58112/0 (socket says 172.21.15.83:58112) 2024-04-17T10:54:42.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.971+0000 7ff30e7fc640 1 -- 172.21.15.83:0/4127942360 learned_addr learned my addr 172.21.15.83:0/4127942360 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:42.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.971+0000 7ff316ffd640 1 --2- 172.21.15.83:0/4127942360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff31806abe0 0x7ff31815b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:42.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.971+0000 7ff3177fe640 1 --2- 172.21.15.83:0/4127942360 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff318146f20 0x7ff3180b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:42.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.971+0000 7ff30e7fc640 1 -- 172.21.15.83:0/4127942360 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff318146f20 msgr2=0x7ff3180b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:42.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.971+0000 7ff30e7fc640 1 --2- 172.21.15.83:0/4127942360 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff318146f20 0x7ff3180b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.971+0000 7ff30e7fc640 1 -- 172.21.15.83:0/4127942360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff31806abe0 msgr2=0x7ff31815b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:42.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.971+0000 7ff30e7fc640 1 --2- 172.21.15.83:0/4127942360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff31806abe0 0x7ff31815b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:42.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.971+0000 7ff30e7fc640 1 -- 172.21.15.83:0/4127942360 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31813e600 con 0x7ff318146b40 2024-04-17T10:54:42.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.975+0000 7ff30e7fc640 1 --2- 172.21.15.83:0/4127942360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff318146b40 0x7ff3180b0f20 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7ff30000c480 tx=0x7ff30000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:42.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.975+0000 7ff30ffff640 1 -- 172.21.15.83:0/4127942360 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff300018070 con 0x7ff318146b40 2024-04-17T10:54:42.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.975+0000 7ff30ffff640 1 -- 172.21.15.83:0/4127942360 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3000074e0 con 0x7ff318146b40 2024-04-17T10:54:42.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.975+0000 7ff31d4cd640 1 -- 172.21.15.83:0/4127942360 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3180b1800 con 0x7ff318146b40 2024-04-17T10:54:42.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.975+0000 7ff30ffff640 1 -- 172.21.15.83:0/4127942360 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3000049f0 con 0x7ff318146b40 2024-04-17T10:54:42.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.975+0000 7ff31d4cd640 1 -- 172.21.15.83:0/4127942360 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff31813e8a0 con 0x7ff318146b40 2024-04-17T10:54:42.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.975+0000 7ff31d4cd640 1 -- 172.21.15.83:0/4127942360 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff318074f90 con 0x7ff318146b40 2024-04-17T10:54:42.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.975+0000 7ff30ffff640 1 -- 172.21.15.83:0/4127942360 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff30001b070 con 0x7ff318146b40 2024-04-17T10:54:42.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.975+0000 7ff30ffff640 1 --2- 172.21.15.83:0/4127942360 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff2f8066f60 0x7ff2f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:42.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.975+0000 7ff30ffff640 1 -- 172.21.15.83:0/4127942360 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7ff300084440 con 0x7ff318146b40 2024-04-17T10:54:42.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.975+0000 7ff316ffd640 1 --2- 172.21.15.83:0/4127942360 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff2f8066f60 0x7ff2f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:42.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.975+0000 7ff316ffd640 1 --2- 172.21.15.83:0/4127942360 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff2f8066f60 0x7ff2f8069420 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7ff308009580 tx=0x7ff308008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:42.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:42.979+0000 7ff30ffff640 1 -- 172.21.15.83:0/4127942360 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff300055470 con 0x7ff318146b40 2024-04-17T10:54:43.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.155+0000 7ff31d4cd640 1 -- 172.21.15.83:0/4127942360 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff318071bf0 con 0x7ff318146b40 2024-04-17T10:54:43.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.159+0000 7ff30ffff640 1 -- 172.21.15.83:0/4127942360 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7ff300054d70 con 0x7ff318146b40 2024-04-17T10:54:43.152 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:43.152 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:54:43.153 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:54:43.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.159+0000 7ff31d4cd640 1 -- 172.21.15.83:0/4127942360 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff2f8066f60 msgr2=0x7ff2f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:43.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.159+0000 7ff31d4cd640 1 --2- 172.21.15.83:0/4127942360 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff2f8066f60 0x7ff2f8069420 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7ff308009580 tx=0x7ff308008040 comp rx=0 tx=0).stop 2024-04-17T10:54:43.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.159+0000 7ff31d4cd640 1 -- 172.21.15.83:0/4127942360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff318146b40 msgr2=0x7ff3180b0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:43.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.159+0000 7ff31d4cd640 1 --2- 172.21.15.83:0/4127942360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff318146b40 0x7ff3180b0f20 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7ff30000c480 tx=0x7ff30000c940 comp rx=0 tx=0).stop 2024-04-17T10:54:43.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.159+0000 7ff31d4cd640 1 -- 172.21.15.83:0/4127942360 shutdown_connections 2024-04-17T10:54:43.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.159+0000 7ff31d4cd640 1 --2- 172.21.15.83:0/4127942360 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7ff2f8066f60 0x7ff2f8069420 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:43.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.159+0000 7ff31d4cd640 1 --2- 172.21.15.83:0/4127942360 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff318146f20 0x7ff3180b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:43.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.159+0000 7ff31d4cd640 1 --2- 172.21.15.83:0/4127942360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff318146b40 0x7ff3180b0f20 unknown :-1 s=CLOSED pgs=2441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:43.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.159+0000 7ff31d4cd640 1 --2- 172.21.15.83:0/4127942360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff31806abe0 0x7ff31815b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:43.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.159+0000 7ff31d4cd640 1 -- 172.21.15.83:0/4127942360 >> 172.21.15.83:0/4127942360 conn(0x7ff31806bcf0 msgr2=0x7ff318143180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:43.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.159+0000 7ff31d4cd640 1 -- 172.21.15.83:0/4127942360 shutdown_connections 2024-04-17T10:54:43.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.159+0000 7ff31d4cd640 1 -- 172.21.15.83:0/4127942360 wait complete. 2024-04-17T10:54:43.171 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:54:43.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.291+0000 7f7a908a8640 1 Processor -- start 2024-04-17T10:54:43.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.291+0000 7f7a908a8640 1 -- start start 2024-04-17T10:54:43.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.291+0000 7f7a908a8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a8c0d2340 0x7f7a8c0ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:43.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.291+0000 7f7a908a8640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a8c0cc330 con 0x7f7a8c0cfa20 2024-04-17T10:54:43.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.291+0000 7f7a908a8640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a8c0cec30 con 0x7f7a8c0cf3f0 2024-04-17T10:54:43.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.291+0000 7f7a908a8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c0b1890 con 0x7f7a8c0d2340 2024-04-17T10:54:43.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.291+0000 7f7a8a59c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7a8c0cfa20 0x7f7a8c0cfe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57800/0 (socket says 172.21.15.83:57800) 2024-04-17T10:54:43.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.291+0000 7f7a8a59c640 1 -- 172.21.15.83:0/2561166671 learned_addr learned my addr 172.21.15.83:0/2561166671 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:43.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.291+0000 7f7a8959a640 1 -- 172.21.15.83:0/2561166671 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1468011013 0 0) 0x7f7a8c0cc330 con 0x7f7a8c0cfa20 2024-04-17T10:54:43.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.291+0000 7f7a8959a640 1 -- 172.21.15.83:0/2561166671 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a74003530 con 0x7f7a8c0cfa20 2024-04-17T10:54:43.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.291+0000 7f7a8ad9d640 1 --2- 172.21.15.83:0/2561166671 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a8c0d2340 0x7f7a8c0ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:43.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.291+0000 7f7a8959a640 1 -- 172.21.15.83:0/2561166671 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 927054680 0 0) 0x7f7a74003530 con 0x7f7a8c0cfa20 2024-04-17T10:54:43.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.291+0000 7f7a8959a640 1 -- 172.21.15.83:0/2561166671 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a8c0d2340 msgr2=0x7f7a8c0ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:43.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.291+0000 7f7a8959a640 1 --2- 172.21.15.83:0/2561166671 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a8c0d2340 0x7f7a8c0ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:43.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.291+0000 7f7a8959a640 1 -- 172.21.15.83:0/2561166671 >> v1:172.21.15.167:6789/0 conn(0x7f7a8c0cf3f0 legacy=0x7f7a8c0b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:43.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.291+0000 7f7a8959a640 1 -- 172.21.15.83:0/2561166671 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a8c0cef10 con 0x7f7a8c0cfa20 2024-04-17T10:54:43.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.291+0000 7f7a8959a640 1 -- 172.21.15.83:0/2561166671 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f7a7c001680 con 0x7f7a8c0cfa20 2024-04-17T10:54:43.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.291+0000 7f7a8959a640 1 -- 172.21.15.83:0/2561166671 >> v1:172.21.15.83:6789/0 conn(0x7f7a8c0cfa20 legacy=0x7f7a8c0cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:43.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.291+0000 7f7a8959a640 1 --2- 172.21.15.83:0/2561166671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a740039f0 0x7f7a74003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:43.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.291+0000 7f7a8959a640 1 --2- 172.21.15.83:0/2561166671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a74004810 0x7f7a74004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:43.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.291+0000 7f7a8959a640 1 --2- 172.21.15.83:0/2561166671 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a8c0d2340 0x7f7a740052a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:43.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.291+0000 7f7a8959a640 1 -- 172.21.15.83:0/2561166671 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c0b1890 con 0x7f7a74004810 2024-04-17T10:54:43.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.291+0000 7f7a8959a640 1 -- 172.21.15.83:0/2561166671 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7400bb90 con 0x7f7a740039f0 2024-04-17T10:54:43.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.291+0000 7f7a8959a640 1 -- 172.21.15.83:0/2561166671 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7400bd90 con 0x7f7a8c0d2340 2024-04-17T10:54:43.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.291+0000 7f7a89d9b640 1 --2- 172.21.15.83:0/2561166671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a74004810 0x7f7a74004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:43.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.291+0000 7f7a89d9b640 1 -- 172.21.15.83:0/2561166671 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a8c0d2340 msgr2=0x7f7a740052a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:43.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.291+0000 7f7a89d9b640 1 --2- 172.21.15.83:0/2561166671 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a8c0d2340 0x7f7a740052a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:43.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.291+0000 7f7a89d9b640 1 -- 172.21.15.83:0/2561166671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a740039f0 msgr2=0x7f7a74003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:43.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.291+0000 7f7a89d9b640 1 --2- 172.21.15.83:0/2561166671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a740039f0 0x7f7a74003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:43.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.291+0000 7f7a89d9b640 1 -- 172.21.15.83:0/2561166671 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a74003530 con 0x7f7a74004810 2024-04-17T10:54:43.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.295+0000 7f7a89d9b640 1 --2- 172.21.15.83:0/2561166671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a74004810 0x7f7a74004c90 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f7a80003db0 tx=0x7f7a8001c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ec61914044c479 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:43.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.295+0000 7f7a8959a640 1 -- 172.21.15.83:0/2561166671 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a800b0070 con 0x7f7a74004810 2024-04-17T10:54:43.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.295+0000 7f7a8959a640 1 -- 172.21.15.83:0/2561166671 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a800ab620 con 0x7f7a74004810 2024-04-17T10:54:43.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.295+0000 7f7a8959a640 1 -- 172.21.15.83:0/2561166671 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a800039b0 con 0x7f7a74004810 2024-04-17T10:54:43.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.295+0000 7f7a908a8640 1 -- 172.21.15.83:0/2561166671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a74004810 msgr2=0x7f7a74004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:43.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.295+0000 7f7a908a8640 1 --2- 172.21.15.83:0/2561166671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a74004810 0x7f7a74004c90 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f7a80003db0 tx=0x7f7a8001c320 comp rx=0 tx=0).stop 2024-04-17T10:54:43.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.295+0000 7f7a8ad9d640 1 -- 172.21.15.83:0/2561166671 reap_dead start 2024-04-17T10:54:43.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.295+0000 7f7a908a8640 1 -- 172.21.15.83:0/2561166671 shutdown_connections 2024-04-17T10:54:43.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.295+0000 7f7a908a8640 1 -- 172.21.15.83:0/2561166671 >> 172.21.15.83:0/2561166671 conn(0x7f7a8c0d1820 msgr2=0x7f7a8c0d1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:43.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.295+0000 7f7a908a8640 1 -- 172.21.15.83:0/2561166671 shutdown_connections 2024-04-17T10:54:43.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.295+0000 7f7a908a8640 1 -- 172.21.15.83:0/2561166671 wait complete. 2024-04-17T10:54:43.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.295+0000 7f7a908a8640 1 Processor -- start 2024-04-17T10:54:43.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.295+0000 7f7a908a8640 1 -- start start 2024-04-17T10:54:43.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.299+0000 7f7a908a8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a74004810 0x7f7a8c0c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:43.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.299+0000 7f7a908a8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a8c0c88f0 0x7f7a8c14bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:43.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.299+0000 7f7a908a8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a8c06f550 0x7f7a8c06fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:43.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.299+0000 7f7a908a8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c0cd760 con 0x7f7a8c0c88f0 2024-04-17T10:54:43.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.299+0000 7f7a908a8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c0cd5e0 con 0x7f7a8c06f550 2024-04-17T10:54:43.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.299+0000 7f7a908a8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c167a40 con 0x7f7a74004810 2024-04-17T10:54:43.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.299+0000 7f7a89d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a8c0c88f0 0x7f7a8c14bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:43.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.299+0000 7f7a89d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a8c0c88f0 0x7f7a8c14bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58122/0 (socket says 172.21.15.83:58122) 2024-04-17T10:54:43.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.299+0000 7f7a89d9b640 1 -- 172.21.15.83:0/3799361873 learned_addr learned my addr 172.21.15.83:0/3799361873 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:43.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.299+0000 7f7a89d9b640 1 -- 172.21.15.83:0/3799361873 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a74004810 msgr2=0x7f7a8c0c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:43.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.299+0000 7f7a89d9b640 1 --2- 172.21.15.83:0/3799361873 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a74004810 0x7f7a8c0c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:43.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.299+0000 7f7a89d9b640 1 -- 172.21.15.83:0/3799361873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a8c06f550 msgr2=0x7f7a8c06fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:43.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.299+0000 7f7a89d9b640 1 --2- 172.21.15.83:0/3799361873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a8c06f550 0x7f7a8c06fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:43.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.299+0000 7f7a89d9b640 1 -- 172.21.15.83:0/3799361873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a8c14c220 con 0x7f7a8c0c88f0 2024-04-17T10:54:43.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.299+0000 7f7a89d9b640 1 --2- 172.21.15.83:0/3799361873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a8c0c88f0 0x7f7a8c14bce0 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f7a800ae040 tx=0x7f7a74005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:43.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.299+0000 7f7a7affd640 1 -- 172.21.15.83:0/3799361873 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a800bb020 con 0x7f7a8c0c88f0 2024-04-17T10:54:43.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.299+0000 7f7a7affd640 1 -- 172.21.15.83:0/3799361873 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a800ab400 con 0x7f7a8c0c88f0 2024-04-17T10:54:43.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.299+0000 7f7a908a8640 1 -- 172.21.15.83:0/3799361873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a8c06d450 con 0x7f7a8c0c88f0 2024-04-17T10:54:43.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.299+0000 7f7a7affd640 1 -- 172.21.15.83:0/3799361873 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a800b0040 con 0x7f7a8c0c88f0 2024-04-17T10:54:43.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.299+0000 7f7a908a8640 1 -- 172.21.15.83:0/3799361873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a8c0b4690 con 0x7f7a8c0c88f0 2024-04-17T10:54:43.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.299+0000 7f7a908a8640 1 -- 172.21.15.83:0/3799361873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a8c074f90 con 0x7f7a8c0c88f0 2024-04-17T10:54:43.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.299+0000 7f7a7affd640 1 -- 172.21.15.83:0/3799361873 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7a800be020 con 0x7f7a8c0c88f0 2024-04-17T10:54:43.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.299+0000 7f7a7affd640 1 --2- 172.21.15.83:0/3799361873 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7a6c066f90 0x7f7a6c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:43.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.299+0000 7f7a7affd640 1 -- 172.21.15.83:0/3799361873 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f7a8011fb20 con 0x7f7a8c0c88f0 2024-04-17T10:54:43.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.303+0000 7f7a8a59c640 1 --2- 172.21.15.83:0/3799361873 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7a6c066f90 0x7f7a6c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:43.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.303+0000 7f7a8a59c640 1 --2- 172.21.15.83:0/3799361873 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7a6c066f90 0x7f7a6c069450 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c000fe0 tx=0x7f7a7c0014e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:43.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.303+0000 7f7a7affd640 1 -- 172.21.15.83:0/3799361873 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7a800c5030 con 0x7f7a8c0c88f0 2024-04-17T10:54:43.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.483+0000 7f7a908a8640 1 -- 172.21.15.83:0/3799361873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a8c0b0a00 con 0x7f7a8c0c88f0 2024-04-17T10:54:43.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.483+0000 7f7a7affd640 1 -- 172.21.15.83:0/3799361873 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f7a80016030 con 0x7f7a8c0c88f0 2024-04-17T10:54:43.478 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:43.478 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":51,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T10:50:34.233452+0000","modified":"2024-04-17T10:51:05.174081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-17T10:54:43.479 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 90 2024-04-17T10:54:43.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.487+0000 7f7a908a8640 1 -- 172.21.15.83:0/3799361873 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7a6c066f90 msgr2=0x7f7a6c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:43.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.487+0000 7f7a908a8640 1 --2- 172.21.15.83:0/3799361873 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7a6c066f90 0x7f7a6c069450 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c000fe0 tx=0x7f7a7c0014e0 comp rx=0 tx=0).stop 2024-04-17T10:54:43.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.487+0000 7f7a908a8640 1 -- 172.21.15.83:0/3799361873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a8c0c88f0 msgr2=0x7f7a8c14bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:43.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.487+0000 7f7a908a8640 1 --2- 172.21.15.83:0/3799361873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a8c0c88f0 0x7f7a8c14bce0 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f7a800ae040 tx=0x7f7a74005840 comp rx=0 tx=0).stop 2024-04-17T10:54:43.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.487+0000 7f7a908a8640 1 -- 172.21.15.83:0/3799361873 shutdown_connections 2024-04-17T10:54:43.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.487+0000 7f7a908a8640 1 --2- 172.21.15.83:0/3799361873 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7a6c066f90 0x7f7a6c069450 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:43.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.487+0000 7f7a908a8640 1 --2- 172.21.15.83:0/3799361873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a8c06f550 0x7f7a8c06fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:43.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.487+0000 7f7a908a8640 1 --2- 172.21.15.83:0/3799361873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a8c0c88f0 0x7f7a8c14bce0 unknown :-1 s=CLOSED pgs=2444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:43.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.487+0000 7f7a908a8640 1 --2- 172.21.15.83:0/3799361873 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a74004810 0x7f7a8c0c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:43.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.487+0000 7f7a908a8640 1 -- 172.21.15.83:0/3799361873 >> 172.21.15.83:0/3799361873 conn(0x7f7a8c0d1820 msgr2=0x7f7a8c0d1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:43.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.487+0000 7f7a908a8640 1 -- 172.21.15.83:0/3799361873 shutdown_connections 2024-04-17T10:54:43.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.487+0000 7f7a908a8640 1 -- 172.21.15.83:0/3799361873 wait complete. 2024-04-17T10:54:43.496 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:54:43.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.607+0000 7f4d16b79640 1 Processor -- start 2024-04-17T10:54:43.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.607+0000 7f4d16b79640 1 -- start start 2024-04-17T10:54:43.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.611+0000 7f4d16b79640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4d1015db50 0x7f4d10070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:43.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.611+0000 7f4d16b79640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4d100aedd0 0x7f4d1006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:43.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.611+0000 7f4d16b79640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d100709b0 con 0x7f4d1006dae0 2024-04-17T10:54:43.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.611+0000 7f4d16b79640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d1015e440 con 0x7f4d100aedd0 2024-04-17T10:54:43.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.611+0000 7f4d16b79640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d1015e720 con 0x7f4d1015db50 2024-04-17T10:54:43.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.611+0000 7f4d15116640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f4d1006dae0 0x7f4d100cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57808/0 (socket says 172.21.15.83:57808) 2024-04-17T10:54:43.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.611+0000 7f4d15116640 1 -- 172.21.15.83:0/2373234401 learned_addr learned my addr 172.21.15.83:0/2373234401 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:43.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.611+0000 7f4d07fff640 1 --2- 172.21.15.83:0/2373234401 >> v2:172.21.15.83:3300/0 conn(0x7f4d100aedd0 0x7f4d1006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:43.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.611+0000 7f4d077fe640 1 -- 172.21.15.83:0/2373234401 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2044914928 0 0) 0x7f4d080014a0 con 0x7f4d1006dae0 2024-04-17T10:54:43.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.611+0000 7f4d077fe640 1 -- 172.21.15.83:0/2373234401 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cf8003530 con 0x7f4d1006dae0 2024-04-17T10:54:43.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.611+0000 7f4d14915640 1 --2- 172.21.15.83:0/2373234401 >> v2:172.21.15.167:3300/0 conn(0x7f4d1015db50 0x7f4d10070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:43.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.611+0000 7f4d07fff640 1 -- 172.21.15.83:0/2373234401 >> v2:172.21.15.167:3300/0 conn(0x7f4d1015db50 msgr2=0x7f4d10070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:43.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.611+0000 7f4d07fff640 1 --2- 172.21.15.83:0/2373234401 >> v2:172.21.15.167:3300/0 conn(0x7f4d1015db50 0x7f4d10070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:43.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.611+0000 7f4d07fff640 1 -- 172.21.15.83:0/2373234401 >> v1:172.21.15.83:6789/0 conn(0x7f4d1006dae0 legacy=0x7f4d100cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:43.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.611+0000 7f4d07fff640 1 -- 172.21.15.83:0/2373234401 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1015ea00 con 0x7f4d100aedd0 2024-04-17T10:54:43.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.611+0000 7f4d14915640 1 --2- 172.21.15.83:0/2373234401 >> v2:172.21.15.167:3300/0 conn(0x7f4d1015db50 0x7f4d10070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:43.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.611+0000 7f4d07fff640 1 --2- 172.21.15.83:0/2373234401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d100aedd0 0x7f4d1006d5a0 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f4d000097c0 tx=0x7f4d0001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d4a87529ae668 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:43.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.611+0000 7f4d077fe640 1 -- 172.21.15.83:0/2373234401 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d000b0070 con 0x7f4d100aedd0 2024-04-17T10:54:43.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.611+0000 7f4d077fe640 1 -- 172.21.15.83:0/2373234401 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d00004230 con 0x7f4d100aedd0 2024-04-17T10:54:43.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.611+0000 7f4d077fe640 1 -- 172.21.15.83:0/2373234401 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d00003620 con 0x7f4d100aedd0 2024-04-17T10:54:43.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.615+0000 7f4d16b79640 1 -- 172.21.15.83:0/2373234401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d100aedd0 msgr2=0x7f4d1006d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:43.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.615+0000 7f4d16b79640 1 --2- 172.21.15.83:0/2373234401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d100aedd0 0x7f4d1006d5a0 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f4d000097c0 tx=0x7f4d0001c460 comp rx=0 tx=0).stop 2024-04-17T10:54:43.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.615+0000 7f4d16b79640 1 -- 172.21.15.83:0/2373234401 shutdown_connections 2024-04-17T10:54:43.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.615+0000 7f4d16b79640 1 --2- 172.21.15.83:0/2373234401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d100aedd0 0x7f4d1006d5a0 unknown :-1 s=CLOSED pgs=2446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:43.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.615+0000 7f4d16b79640 1 --2- 172.21.15.83:0/2373234401 >> v2:172.21.15.167:3300/0 conn(0x7f4d1015db50 0x7f4d10070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:43.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.615+0000 7f4d16b79640 1 -- 172.21.15.83:0/2373234401 >> 172.21.15.83:0/2373234401 conn(0x7f4d1006abe0 msgr2=0x7f4d1006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:43.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.615+0000 7f4d16b79640 1 -- 172.21.15.83:0/2373234401 shutdown_connections 2024-04-17T10:54:43.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.615+0000 7f4d16b79640 1 -- 172.21.15.83:0/2373234401 wait complete. 2024-04-17T10:54:43.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.615+0000 7f4d16b79640 1 Processor -- start 2024-04-17T10:54:43.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.615+0000 7f4d16b79640 1 -- start start 2024-04-17T10:54:43.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.615+0000 7f4d16b79640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d1006dae0 0x7f4d10152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:43.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.615+0000 7f4d16b79640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d100aedd0 0x7f4d10156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:43.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.615+0000 7f4d16b79640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d1015db50 0x7f4d10150340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:43.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.615+0000 7f4d16b79640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1006c610 con 0x7f4d100aedd0 2024-04-17T10:54:43.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.615+0000 7f4d16b79640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1006c490 con 0x7f4d1015db50 2024-04-17T10:54:43.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.615+0000 7f4d16b79640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1006c790 con 0x7f4d1006dae0 2024-04-17T10:54:43.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.615+0000 7f4d07fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d100aedd0 0x7f4d10156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:43.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.615+0000 7f4d07fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d100aedd0 0x7f4d10156e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58144/0 (socket says 172.21.15.83:58144) 2024-04-17T10:54:43.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.615+0000 7f4d07fff640 1 -- 172.21.15.83:0/3604142757 learned_addr learned my addr 172.21.15.83:0/3604142757 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:43.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.615+0000 7f4d07fff640 1 -- 172.21.15.83:0/3604142757 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d1006dae0 msgr2=0x7f4d10152c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:43.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.615+0000 7f4d07fff640 1 --2- 172.21.15.83:0/3604142757 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d1006dae0 0x7f4d10152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:43.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.615+0000 7f4d15116640 1 --2- 172.21.15.83:0/3604142757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d1015db50 0x7f4d10150340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:43.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.615+0000 7f4d07fff640 1 -- 172.21.15.83:0/3604142757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d1015db50 msgr2=0x7f4d10150340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:43.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.615+0000 7f4d07fff640 1 --2- 172.21.15.83:0/3604142757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d1015db50 0x7f4d10150340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:43.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.615+0000 7f4d07fff640 1 -- 172.21.15.83:0/3604142757 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d10153560 con 0x7f4d100aedd0 2024-04-17T10:54:43.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.615+0000 7f4d07fff640 1 --2- 172.21.15.83:0/3604142757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d100aedd0 0x7f4d10156e40 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7f4d000ae040 tx=0x7f4d00019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:43.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.615+0000 7f4d057fa640 1 -- 172.21.15.83:0/3604142757 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d000b9020 con 0x7f4d100aedd0 2024-04-17T10:54:43.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.615+0000 7f4d057fa640 1 -- 172.21.15.83:0/3604142757 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d00003e10 con 0x7f4d100aedd0 2024-04-17T10:54:43.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.615+0000 7f4d16b79640 1 -- 172.21.15.83:0/3604142757 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d10153840 con 0x7f4d100aedd0 2024-04-17T10:54:43.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.615+0000 7f4d057fa640 1 -- 172.21.15.83:0/3604142757 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d000b0040 con 0x7f4d100aedd0 2024-04-17T10:54:43.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.615+0000 7f4d16b79640 1 -- 172.21.15.83:0/3604142757 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d10153e00 con 0x7f4d100aedd0 2024-04-17T10:54:43.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.619+0000 7f4d16b79640 1 -- 172.21.15.83:0/3604142757 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d10074f90 con 0x7f4d100aedd0 2024-04-17T10:54:43.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.619+0000 7f4d057fa640 1 -- 172.21.15.83:0/3604142757 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4d000bc020 con 0x7f4d100aedd0 2024-04-17T10:54:43.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.619+0000 7f4d057fa640 1 --2- 172.21.15.83:0/3604142757 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4cfc066f60 0x7f4cfc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:43.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.619+0000 7f4d057fa640 1 -- 172.21.15.83:0/3604142757 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f4d0011ef70 con 0x7f4d100aedd0 2024-04-17T10:54:43.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.619+0000 7f4d14915640 1 --2- 172.21.15.83:0/3604142757 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4cfc066f60 0x7f4cfc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:43.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.619+0000 7f4d14915640 1 --2- 172.21.15.83:0/3604142757 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4cfc066f60 0x7f4cfc069420 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00a890 tx=0x7f4d0c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:43.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.623+0000 7f4d057fa640 1 -- 172.21.15.83:0/3604142757 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4d000c3030 con 0x7f4d100aedd0 2024-04-17T10:54:43.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.771+0000 7f4d16b79640 1 -- 172.21.15.83:0/3604142757 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d10074480 con 0x7f4d100aedd0 2024-04-17T10:54:43.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.771+0000 7f4d057fa640 1 -- 172.21.15.83:0/3604142757 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+16786 (secure 0 0 0) 0x7f4d00116e20 con 0x7f4d100aedd0 2024-04-17T10:54:43.765 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:43.765 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":143,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:51:04.828803+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:50:32.488186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-17T10:50:33.207211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.32","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/3575114599":"2024-04-18T10:51:04.828753+0000","172.21.15.83:6836/3575114599":"2024-04-18T10:51:04.828753+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:54:43.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.775+0000 7f4d16b79640 1 -- 172.21.15.83:0/3604142757 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4cfc066f60 msgr2=0x7f4cfc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:43.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.775+0000 7f4d16b79640 1 --2- 172.21.15.83:0/3604142757 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4cfc066f60 0x7f4cfc069420 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00a890 tx=0x7f4d0c008040 comp rx=0 tx=0).stop 2024-04-17T10:54:43.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.775+0000 7f4d16b79640 1 -- 172.21.15.83:0/3604142757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d100aedd0 msgr2=0x7f4d10156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:43.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.775+0000 7f4d16b79640 1 --2- 172.21.15.83:0/3604142757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d100aedd0 0x7f4d10156e40 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7f4d000ae040 tx=0x7f4d00019200 comp rx=0 tx=0).stop 2024-04-17T10:54:43.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.775+0000 7f4d16b79640 1 -- 172.21.15.83:0/3604142757 shutdown_connections 2024-04-17T10:54:43.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.775+0000 7f4d16b79640 1 --2- 172.21.15.83:0/3604142757 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4cfc066f60 0x7f4cfc069420 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:43.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.775+0000 7f4d16b79640 1 --2- 172.21.15.83:0/3604142757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d1015db50 0x7f4d10150340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:43.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.775+0000 7f4d16b79640 1 --2- 172.21.15.83:0/3604142757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d100aedd0 0x7f4d10156e40 unknown :-1 s=CLOSED pgs=2447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:43.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.775+0000 7f4d16b79640 1 --2- 172.21.15.83:0/3604142757 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d1006dae0 0x7f4d10152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:43.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.775+0000 7f4d16b79640 1 -- 172.21.15.83:0/3604142757 >> 172.21.15.83:0/3604142757 conn(0x7f4d1006abe0 msgr2=0x7f4d100cf420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:43.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.775+0000 7f4d16b79640 1 -- 172.21.15.83:0/3604142757 shutdown_connections 2024-04-17T10:54:43.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.775+0000 7f4d16b79640 1 -- 172.21.15.83:0/3604142757 wait complete. 2024-04-17T10:54:43.784 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-17T10:54:43.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.911+0000 7f6cd8cd2640 1 Processor -- start 2024-04-17T10:54:43.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.911+0000 7f6cd8cd2640 1 -- start start 2024-04-17T10:54:43.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.911+0000 7f6cd8cd2640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6cd415b6e0 0x7f6cd4071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:43.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.911+0000 7f6cd8cd2640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cd4070470 0x7f6cd406ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:43.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.911+0000 7f6cd8cd2640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cd406c1f0 con 0x7f6cd415bac0 2024-04-17T10:54:43.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.911+0000 7f6cd8cd2640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cd4070a10 con 0x7f6cd415b6e0 2024-04-17T10:54:43.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.911+0000 7f6cd8cd2640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd4070cf0 con 0x7f6cd4070470 2024-04-17T10:54:43.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.911+0000 7f6cd259c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6cd415b6e0 0x7f6cd4071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:43.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.911+0000 7f6cd1d9b640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f6cd415bac0 0x7f6cd40cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:55226/0 (socket says 172.21.15.83:55226) 2024-04-17T10:54:43.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.911+0000 7f6cd2d9d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cd4070470 0x7f6cd406ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:43.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.911+0000 7f6cd259c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6cd415b6e0 0x7f6cd4071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:46770/0 (socket says 172.21.15.83:46770) 2024-04-17T10:54:43.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.911+0000 7f6cd1d9b640 1 -- 172.21.15.83:0/2940677208 learned_addr learned my addr 172.21.15.83:0/2940677208 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:43.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.911+0000 7f6cd2d9d640 1 -- 172.21.15.83:0/2940677208 >> v2:172.21.15.167:3300/0 conn(0x7f6cd415b6e0 msgr2=0x7f6cd4071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:43.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.911+0000 7f6cd2d9d640 1 --2- 172.21.15.83:0/2940677208 >> v2:172.21.15.167:3300/0 conn(0x7f6cd415b6e0 0x7f6cd4071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:43.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.911+0000 7f6cd2d9d640 1 -- 172.21.15.83:0/2940677208 >> v1:172.21.15.167:6789/0 conn(0x7f6cd415bac0 legacy=0x7f6cd40cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:43.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.911+0000 7f6cd2d9d640 1 -- 172.21.15.83:0/2940677208 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd4070fd0 con 0x7f6cd4070470 2024-04-17T10:54:43.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.915+0000 7f6cd2d9d640 1 --2- 172.21.15.83:0/2940677208 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cd4070470 0x7f6cd406ae10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009d90 tx=0x7f6ccc01c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f2e99059bc711d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:43.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.915+0000 7f6cd159a640 1 -- 172.21.15.83:0/2940677208 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc0b2070 con 0x7f6cd4070470 2024-04-17T10:54:43.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.915+0000 7f6cd159a640 1 -- 172.21.15.83:0/2940677208 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ccc0a57a0 con 0x7f6cd4070470 2024-04-17T10:54:43.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.915+0000 7f6cd159a640 1 -- 172.21.15.83:0/2940677208 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc0a5b70 con 0x7f6cd4070470 2024-04-17T10:54:43.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.915+0000 7f6cd8cd2640 1 -- 172.21.15.83:0/2940677208 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cd4070470 msgr2=0x7f6cd406ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:43.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.915+0000 7f6cd8cd2640 1 --2- 172.21.15.83:0/2940677208 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cd4070470 0x7f6cd406ae10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009d90 tx=0x7f6ccc01c8e0 comp rx=0 tx=0).stop 2024-04-17T10:54:43.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.915+0000 7f6cd8cd2640 1 -- 172.21.15.83:0/2940677208 shutdown_connections 2024-04-17T10:54:43.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.915+0000 7f6cd8cd2640 1 --2- 172.21.15.83:0/2940677208 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cd4070470 0x7f6cd406ae10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:43.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.915+0000 7f6cd8cd2640 1 --2- 172.21.15.83:0/2940677208 >> v2:172.21.15.167:3300/0 conn(0x7f6cd415b6e0 0x7f6cd4071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:43.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.915+0000 7f6cd8cd2640 1 -- 172.21.15.83:0/2940677208 >> 172.21.15.83:0/2940677208 conn(0x7f6cd40b15b0 msgr2=0x7f6cd40b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:43.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.915+0000 7f6cd8cd2640 1 -- 172.21.15.83:0/2940677208 shutdown_connections 2024-04-17T10:54:43.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.915+0000 7f6cd8cd2640 1 -- 172.21.15.83:0/2940677208 wait complete. 2024-04-17T10:54:43.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.919+0000 7f6cd8cd2640 1 Processor -- start 2024-04-17T10:54:43.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.919+0000 7f6cd8cd2640 1 -- start start 2024-04-17T10:54:43.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.919+0000 7f6cd8cd2640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cd4070470 0x7f6cd415d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:43.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.919+0000 7f6cd8cd2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cd415b6e0 0x7f6cd4142f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:43.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.919+0000 7f6cd8cd2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cd415bac0 0x7f6cd415bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:43.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.919+0000 7f6cd8cd2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd406d2b0 con 0x7f6cd415b6e0 2024-04-17T10:54:43.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.919+0000 7f6cd8cd2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd406d130 con 0x7f6cd415bac0 2024-04-17T10:54:43.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.919+0000 7f6cd8cd2640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd406b760 con 0x7f6cd4070470 2024-04-17T10:54:43.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.919+0000 7f6cd1d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cd415b6e0 0x7f6cd4142f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:43.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.919+0000 7f6cd1d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cd415b6e0 0x7f6cd4142f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58160/0 (socket says 172.21.15.83:58160) 2024-04-17T10:54:43.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.919+0000 7f6cd1d9b640 1 -- 172.21.15.83:0/4055202701 learned_addr learned my addr 172.21.15.83:0/4055202701 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:43.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.919+0000 7f6cd2d9d640 1 --2- 172.21.15.83:0/4055202701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cd415bac0 0x7f6cd415bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:43.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.919+0000 7f6cd259c640 1 --2- 172.21.15.83:0/4055202701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cd4070470 0x7f6cd415d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:43.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.919+0000 7f6cd1d9b640 1 -- 172.21.15.83:0/4055202701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cd4070470 msgr2=0x7f6cd415d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:43.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.919+0000 7f6cd1d9b640 1 --2- 172.21.15.83:0/4055202701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cd4070470 0x7f6cd415d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:43.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.919+0000 7f6cd1d9b640 1 -- 172.21.15.83:0/4055202701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cd415bac0 msgr2=0x7f6cd415bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:43.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.919+0000 7f6cd1d9b640 1 --2- 172.21.15.83:0/4055202701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cd415bac0 0x7f6cd415bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:43.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.919+0000 7f6cd1d9b640 1 -- 172.21.15.83:0/4055202701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd415c720 con 0x7f6cd415b6e0 2024-04-17T10:54:43.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.919+0000 7f6cd259c640 1 --2- 172.21.15.83:0/4055202701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cd4070470 0x7f6cd415d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:43.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.919+0000 7f6cd1d9b640 1 --2- 172.21.15.83:0/4055202701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cd415b6e0 0x7f6cd4142f50 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f6cc800c450 tx=0x7f6cc800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:43.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.919+0000 7f6cc2ffd640 1 -- 172.21.15.83:0/4055202701 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc8017070 con 0x7f6cd415b6e0 2024-04-17T10:54:43.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.919+0000 7f6cc2ffd640 1 -- 172.21.15.83:0/4055202701 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc8005cf0 con 0x7f6cd415b6e0 2024-04-17T10:54:43.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.919+0000 7f6cd8cd2640 1 -- 172.21.15.83:0/4055202701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd415ca00 con 0x7f6cd415b6e0 2024-04-17T10:54:43.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.919+0000 7f6cc2ffd640 1 -- 172.21.15.83:0/4055202701 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc8007290 con 0x7f6cd415b6e0 2024-04-17T10:54:43.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.919+0000 7f6cd8cd2640 1 -- 172.21.15.83:0/4055202701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd415cfc0 con 0x7f6cd415b6e0 2024-04-17T10:54:43.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.919+0000 7f6cd8cd2640 1 -- 172.21.15.83:0/4055202701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cd4074f90 con 0x7f6cd415b6e0 2024-04-17T10:54:43.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.919+0000 7f6cc2ffd640 1 -- 172.21.15.83:0/4055202701 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6cc801a070 con 0x7f6cd415b6e0 2024-04-17T10:54:43.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.919+0000 7f6cc2ffd640 1 --2- 172.21.15.83:0/4055202701 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6cb4066f60 0x7f6cb4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:43.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.919+0000 7f6cc2ffd640 1 -- 172.21.15.83:0/4055202701 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f6cc80842d0 con 0x7f6cd415b6e0 2024-04-17T10:54:43.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.919+0000 7f6cd259c640 1 --2- 172.21.15.83:0/4055202701 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6cb4066f60 0x7f6cb4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:43.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.923+0000 7f6cd259c640 1 --2- 172.21.15.83:0/4055202701 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6cb4066f60 0x7f6cb4069420 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009ae0 tx=0x7f6cc4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:43.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:43.923+0000 7f6cc2ffd640 1 -- 172.21.15.83:0/4055202701 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6cc8055300 con 0x7f6cd415b6e0 2024-04-17T10:54:44.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.103+0000 7f6cd8cd2640 1 -- 172.21.15.83:0/4055202701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6cd40cda90 con 0x7f6cd415b6e0 2024-04-17T10:54:44.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.599+0000 7f6cc2ffd640 1 -- 172.21.15.83:0/4055202701 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v91) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6cc8054c00 con 0x7f6cd415b6e0 2024-04-17T10:54:44.592 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-17T10:54:44.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.599+0000 7f6cd8cd2640 1 -- 172.21.15.83:0/4055202701 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6cb4066f60 msgr2=0x7f6cb4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:44.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.599+0000 7f6cd8cd2640 1 --2- 172.21.15.83:0/4055202701 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6cb4066f60 0x7f6cb4069420 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009ae0 tx=0x7f6cc4008040 comp rx=0 tx=0).stop 2024-04-17T10:54:44.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.599+0000 7f6cd8cd2640 1 -- 172.21.15.83:0/4055202701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cd415b6e0 msgr2=0x7f6cd4142f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:44.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.599+0000 7f6cd8cd2640 1 --2- 172.21.15.83:0/4055202701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cd415b6e0 0x7f6cd4142f50 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f6cc800c450 tx=0x7f6cc800c910 comp rx=0 tx=0).stop 2024-04-17T10:54:44.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.599+0000 7f6cd8cd2640 1 -- 172.21.15.83:0/4055202701 shutdown_connections 2024-04-17T10:54:44.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.599+0000 7f6cd8cd2640 1 --2- 172.21.15.83:0/4055202701 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f6cb4066f60 0x7f6cb4069420 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:44.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.599+0000 7f6cd8cd2640 1 --2- 172.21.15.83:0/4055202701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cd415bac0 0x7f6cd415bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:44.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.599+0000 7f6cd8cd2640 1 --2- 172.21.15.83:0/4055202701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cd415b6e0 0x7f6cd4142f50 unknown :-1 s=CLOSED pgs=2448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:44.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.599+0000 7f6cd8cd2640 1 --2- 172.21.15.83:0/4055202701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cd4070470 0x7f6cd415d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:44.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.599+0000 7f6cd8cd2640 1 -- 172.21.15.83:0/4055202701 >> 172.21.15.83:0/4055202701 conn(0x7f6cd40b15b0 msgr2=0x7f6cd415f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:44.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.599+0000 7f6cd8cd2640 1 -- 172.21.15.83:0/4055202701 shutdown_connections 2024-04-17T10:54:44.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.599+0000 7f6cd8cd2640 1 -- 172.21.15.83:0/4055202701 wait complete. 2024-04-17T10:54:44.616 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-17T10:54:44.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f112eca8640 1 Processor -- start 2024-04-17T10:54:44.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f112eca8640 1 -- start start 2024-04-17T10:54:44.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f112eca8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f112815e0f0 0x7f11280b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:44.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f112eca8640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f11280ce400 0x7f1128161c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:44.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f112eca8640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f112815e4d0 con 0x7f11280b5450 2024-04-17T10:54:44.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f112eca8640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1128162b40 con 0x7f11280ce400 2024-04-17T10:54:44.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f112eca8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1128162580 con 0x7f112815e0f0 2024-04-17T10:54:44.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f112d245640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f11280b5450 0x7f11280d2540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57816/0 (socket says 172.21.15.83:57816) 2024-04-17T10:54:44.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f112d245640 1 -- 172.21.15.83:0/2851022022 learned_addr learned my addr 172.21.15.83:0/2851022022 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:44.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f1127fff640 1 --2- 172.21.15.83:0/2851022022 >> v2:172.21.15.167:3300/0 conn(0x7f11280ce400 0x7f1128161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:44.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f11277fe640 1 -- 172.21.15.83:0/2851022022 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2142826851 0 0) 0x7f112815e4d0 con 0x7f11280b5450 2024-04-17T10:54:44.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f11277fe640 1 -- 172.21.15.83:0/2851022022 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1118003530 con 0x7f11280b5450 2024-04-17T10:54:44.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f112ca44640 1 --2- 172.21.15.83:0/2851022022 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f112815e0f0 0x7f11280b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:44.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f11277fe640 1 -- 172.21.15.83:0/2851022022 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1307673717 0 0) 0x7f1118003530 con 0x7f11280b5450 2024-04-17T10:54:44.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f11277fe640 1 -- 172.21.15.83:0/2851022022 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f112815e0f0 msgr2=0x7f11280b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:44.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f11277fe640 1 --2- 172.21.15.83:0/2851022022 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f112815e0f0 0x7f11280b6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:44.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f11277fe640 1 -- 172.21.15.83:0/2851022022 >> v2:172.21.15.167:3300/0 conn(0x7f11280ce400 msgr2=0x7f1128161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:44.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f11277fe640 1 --2- 172.21.15.83:0/2851022022 >> v2:172.21.15.167:3300/0 conn(0x7f11280ce400 0x7f1128161c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:44.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f11277fe640 1 -- 172.21.15.83:0/2851022022 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1128162860 con 0x7f11280b5450 2024-04-17T10:54:44.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f11277fe640 1 -- 172.21.15.83:0/2851022022 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f1120001a00 con 0x7f11280b5450 2024-04-17T10:54:44.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f11277fe640 1 -- 172.21.15.83:0/2851022022 >> v1:172.21.15.83:6789/0 conn(0x7f11280b5450 legacy=0x7f11280d2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:44.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f11277fe640 1 --2- 172.21.15.83:0/2851022022 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f112815e0f0 0x7f1118003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:44.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f11277fe640 1 --2- 172.21.15.83:0/2851022022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1118003fb0 0x7f1118004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:44.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f11277fe640 1 --2- 172.21.15.83:0/2851022022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1118004d50 0x7f11180051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:44.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f11277fe640 1 -- 172.21.15.83:0/2851022022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1128162b40 con 0x7f1118003fb0 2024-04-17T10:54:44.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f11277fe640 1 -- 172.21.15.83:0/2851022022 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1128162580 con 0x7f1118004d50 2024-04-17T10:54:44.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f11277fe640 1 -- 172.21.15.83:0/2851022022 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f111800bb30 con 0x7f112815e0f0 2024-04-17T10:54:44.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f1127fff640 1 --2- 172.21.15.83:0/2851022022 >> v2:172.21.15.167:3300/0 conn(0x7f11280ce400 0x7f1128161c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:54:44.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f1127fff640 1 --2- 172.21.15.83:0/2851022022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1118003fb0 0x7f1118004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f112d245640 1 --2- 172.21.15.83:0/2851022022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1118004d50 0x7f11180051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f112ca44640 1 --2- 172.21.15.83:0/2851022022 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f112815e0f0 0x7f1118003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f1127fff640 1 -- 172.21.15.83:0/2851022022 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f112815e0f0 msgr2=0x7f1118003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f1127fff640 1 --2- 172.21.15.83:0/2851022022 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f112815e0f0 0x7f1118003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f1127fff640 1 -- 172.21.15.83:0/2851022022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1118004d50 msgr2=0x7f11180051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f1127fff640 1 --2- 172.21.15.83:0/2851022022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1118004d50 0x7f11180051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f1127fff640 1 -- 172.21.15.83:0/2851022022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1118003530 con 0x7f1118003fb0 2024-04-17T10:54:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.747+0000 7f112ca44640 1 --2- 172.21.15.83:0/2851022022 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f112815e0f0 0x7f1118003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.751+0000 7f1127fff640 1 --2- 172.21.15.83:0/2851022022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1118003fb0 0x7f1118004810 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f1114009880 tx=0x7f111401c5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c15dfbb6fd04f38 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.751+0000 7f11277fe640 1 -- 172.21.15.83:0/2851022022 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111400f040 con 0x7f1118003fb0 2024-04-17T10:54:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.751+0000 7f11277fe640 1 -- 172.21.15.83:0/2851022022 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1114004220 con 0x7f1118003fb0 2024-04-17T10:54:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.751+0000 7f11277fe640 1 -- 172.21.15.83:0/2851022022 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1114003620 con 0x7f1118003fb0 2024-04-17T10:54:44.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.751+0000 7f112eca8640 1 -- 172.21.15.83:0/2851022022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1118003fb0 msgr2=0x7f1118004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:44.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.751+0000 7f112eca8640 1 --2- 172.21.15.83:0/2851022022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1118003fb0 0x7f1118004810 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f1114009880 tx=0x7f111401c5c0 comp rx=0 tx=0).stop 2024-04-17T10:54:44.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.751+0000 7f112d245640 1 -- 172.21.15.83:0/2851022022 reap_dead start 2024-04-17T10:54:44.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.751+0000 7f112eca8640 1 -- 172.21.15.83:0/2851022022 shutdown_connections 2024-04-17T10:54:44.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.751+0000 7f112eca8640 1 -- 172.21.15.83:0/2851022022 >> 172.21.15.83:0/2851022022 conn(0x7f11280d0450 msgr2=0x7f11280d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:44.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.751+0000 7f112eca8640 1 -- 172.21.15.83:0/2851022022 shutdown_connections 2024-04-17T10:54:44.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.751+0000 7f112eca8640 1 -- 172.21.15.83:0/2851022022 wait complete. 2024-04-17T10:54:44.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.751+0000 7f112eca8640 1 Processor -- start 2024-04-17T10:54:44.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.751+0000 7f112eca8640 1 -- start start 2024-04-17T10:54:44.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.755+0000 7f112eca8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1118003fb0 0x7f112815fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:44.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.755+0000 7f112eca8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f112815e650 0x7f1128160fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:44.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.755+0000 7f112eca8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11280c9640 0x7f11280cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:44.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.755+0000 7f112eca8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11280b5dc0 con 0x7f11280c9640 2024-04-17T10:54:44.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.755+0000 7f112eca8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11280b5c40 con 0x7f1118003fb0 2024-04-17T10:54:44.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.755+0000 7f112eca8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1128162150 con 0x7f112815e650 2024-04-17T10:54:44.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.755+0000 7f112d245640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11280c9640 0x7f11280cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:44.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.755+0000 7f112d245640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11280c9640 0x7f11280cc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58174/0 (socket says 172.21.15.83:58174) 2024-04-17T10:54:44.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.755+0000 7f112d245640 1 -- 172.21.15.83:0/4166055141 learned_addr learned my addr 172.21.15.83:0/4166055141 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:44.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.755+0000 7f112ca44640 1 --2- 172.21.15.83:0/4166055141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1118003fb0 0x7f112815fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:44.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.755+0000 7f1127fff640 1 --2- 172.21.15.83:0/4166055141 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f112815e650 0x7f1128160fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:44.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.755+0000 7f112d245640 1 -- 172.21.15.83:0/4166055141 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f112815e650 msgr2=0x7f1128160fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:44.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.755+0000 7f112d245640 1 --2- 172.21.15.83:0/4166055141 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f112815e650 0x7f1128160fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:44.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.755+0000 7f112d245640 1 -- 172.21.15.83:0/4166055141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1118003fb0 msgr2=0x7f112815fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:44.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.755+0000 7f112d245640 1 --2- 172.21.15.83:0/4166055141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1118003fb0 0x7f112815fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:44.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.755+0000 7f112d245640 1 -- 172.21.15.83:0/4166055141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f112815ebf0 con 0x7f11280c9640 2024-04-17T10:54:44.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.755+0000 7f1127fff640 1 --2- 172.21.15.83:0/4166055141 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f112815e650 0x7f1128160fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:44.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.755+0000 7f112d245640 1 --2- 172.21.15.83:0/4166055141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11280c9640 0x7f11280cc9a0 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7f1120000f90 tx=0x7f1120007b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:44.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.755+0000 7f11257fa640 1 -- 172.21.15.83:0/4166055141 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112000e9c0 con 0x7f11280c9640 2024-04-17T10:54:44.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.755+0000 7f11257fa640 1 -- 172.21.15.83:0/4166055141 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1120018070 con 0x7f11280c9640 2024-04-17T10:54:44.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.755+0000 7f112eca8640 1 -- 172.21.15.83:0/4166055141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f112815eed0 con 0x7f11280c9640 2024-04-17T10:54:44.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.755+0000 7f11257fa640 1 -- 172.21.15.83:0/4166055141 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112000edc0 con 0x7f11280c9640 2024-04-17T10:54:44.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.755+0000 7f112eca8640 1 -- 172.21.15.83:0/4166055141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f112815f490 con 0x7f11280c9640 2024-04-17T10:54:44.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.755+0000 7f112eca8640 1 -- 172.21.15.83:0/4166055141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11280681a0 con 0x7f11280c9640 2024-04-17T10:54:44.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.755+0000 7f11257fa640 1 -- 172.21.15.83:0/4166055141 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f112001d460 con 0x7f11280c9640 2024-04-17T10:54:44.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.755+0000 7f11257fa640 1 --2- 172.21.15.83:0/4166055141 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f110c066f60 0x7f110c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:44.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.755+0000 7f112ca44640 1 --2- 172.21.15.83:0/4166055141 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f110c066f60 0x7f110c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:44.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.755+0000 7f11257fa640 1 -- 172.21.15.83:0/4166055141 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f11200852f0 con 0x7f11280c9640 2024-04-17T10:54:44.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.755+0000 7f112ca44640 1 --2- 172.21.15.83:0/4166055141 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f110c066f60 0x7f110c069420 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f111c009910 tx=0x7f111c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:44.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.759+0000 7f11257fa640 1 -- 172.21.15.83:0/4166055141 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1120056290 con 0x7f11280c9640 2024-04-17T10:54:44.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:44.947+0000 7f112eca8640 1 -- 172.21.15.83:0/4166055141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1128074f90 con 0x7f11280c9640 2024-04-17T10:54:45.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.599+0000 7f11257fa640 1 -- 172.21.15.83:0/4166055141 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v92) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1120055b90 con 0x7f11280c9640 2024-04-17T10:54:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.603+0000 7f112eca8640 1 -- 172.21.15.83:0/4166055141 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f110c066f60 msgr2=0x7f110c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.603+0000 7f112eca8640 1 --2- 172.21.15.83:0/4166055141 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f110c066f60 0x7f110c069420 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f111c009910 tx=0x7f111c008040 comp rx=0 tx=0).stop 2024-04-17T10:54:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.603+0000 7f112eca8640 1 -- 172.21.15.83:0/4166055141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11280c9640 msgr2=0x7f11280cc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.603+0000 7f112eca8640 1 --2- 172.21.15.83:0/4166055141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11280c9640 0x7f11280cc9a0 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7f1120000f90 tx=0x7f1120007b50 comp rx=0 tx=0).stop 2024-04-17T10:54:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.603+0000 7f112eca8640 1 -- 172.21.15.83:0/4166055141 shutdown_connections 2024-04-17T10:54:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.603+0000 7f112eca8640 1 --2- 172.21.15.83:0/4166055141 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f110c066f60 0x7f110c069420 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.603+0000 7f112eca8640 1 --2- 172.21.15.83:0/4166055141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11280c9640 0x7f11280cc9a0 unknown :-1 s=CLOSED pgs=2451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.603+0000 7f112eca8640 1 --2- 172.21.15.83:0/4166055141 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f112815e650 0x7f1128160fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:45.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.603+0000 7f112eca8640 1 --2- 172.21.15.83:0/4166055141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1118003fb0 0x7f112815fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:45.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.603+0000 7f112eca8640 1 -- 172.21.15.83:0/4166055141 >> 172.21.15.83:0/4166055141 conn(0x7f11280d0450 msgr2=0x7f11280d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:45.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.603+0000 7f112eca8640 1 -- 172.21.15.83:0/4166055141 shutdown_connections 2024-04-17T10:54:45.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.603+0000 7f112eca8640 1 -- 172.21.15.83:0/4166055141 wait complete. 2024-04-17T10:54:45.617 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-17T10:54:45.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.751+0000 7fd4d2d41640 1 Processor -- start 2024-04-17T10:54:45.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.751+0000 7fd4d2d41640 1 -- start start 2024-04-17T10:54:45.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.751+0000 7fd4d2d41640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd4cc15cb40 0x7fd4cc06abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:45.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.751+0000 7fd4d2d41640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4cc0b39d0 con 0x7fd4cc0aedf0 2024-04-17T10:54:45.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.751+0000 7fd4d2d41640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4cc070c40 con 0x7fd4cc06b120 2024-04-17T10:54:45.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.751+0000 7fd4d2d41640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4cc071290 con 0x7fd4cc15cb40 2024-04-17T10:54:45.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.751+0000 7fd4c3fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd4cc15cb40 0x7fd4cc06abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:45.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.751+0000 7fd4d0add640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd4cc0aedf0 0x7fd4cc0b30f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57820/0 (socket says 172.21.15.83:57820) 2024-04-17T10:54:45.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.751+0000 7fd4d0add640 1 -- 172.21.15.83:0/1155889967 learned_addr learned my addr 172.21.15.83:0/1155889967 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:45.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.751+0000 7fd4c3fff640 1 -- 172.21.15.83:0/1155889967 >> v1:172.21.15.167:6789/0 conn(0x7fd4cc06b120 legacy=0x7fd4cc071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:45.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.751+0000 7fd4c37fe640 1 -- 172.21.15.83:0/1155889967 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4238506087 0 0) 0x7fd4cc0b39d0 con 0x7fd4cc0aedf0 2024-04-17T10:54:45.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.751+0000 7fd4c3fff640 1 -- 172.21.15.83:0/1155889967 >> v1:172.21.15.83:6789/0 conn(0x7fd4cc0aedf0 legacy=0x7fd4cc0b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:45.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.751+0000 7fd4c3fff640 1 -- 172.21.15.83:0/1155889967 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4cc070f20 con 0x7fd4cc15cb40 2024-04-17T10:54:45.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.755+0000 7fd4c3fff640 1 --2- 172.21.15.83:0/1155889967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4cc15cb40 0x7fd4cc06abe0 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc0097c0 tx=0x7fd4bc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d397c32a52c09351 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:45.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.755+0000 7fd4c37fe640 1 -- 172.21.15.83:0/1155889967 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc0b0070 con 0x7fd4cc15cb40 2024-04-17T10:54:45.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.755+0000 7fd4c37fe640 1 -- 172.21.15.83:0/1155889967 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4bc0ab600 con 0x7fd4cc15cb40 2024-04-17T10:54:45.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.755+0000 7fd4c37fe640 1 -- 172.21.15.83:0/1155889967 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc0ab9d0 con 0x7fd4cc15cb40 2024-04-17T10:54:45.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.755+0000 7fd4d2d41640 1 -- 172.21.15.83:0/1155889967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4cc15cb40 msgr2=0x7fd4cc06abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:45.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.755+0000 7fd4d2d41640 1 --2- 172.21.15.83:0/1155889967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4cc15cb40 0x7fd4cc06abe0 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc0097c0 tx=0x7fd4bc01c460 comp rx=0 tx=0).stop 2024-04-17T10:54:45.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.755+0000 7fd4d2d41640 1 -- 172.21.15.83:0/1155889967 shutdown_connections 2024-04-17T10:54:45.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.755+0000 7fd4d2d41640 1 --2- 172.21.15.83:0/1155889967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4cc15cb40 0x7fd4cc06abe0 unknown :-1 s=CLOSED pgs=2453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:45.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.755+0000 7fd4d2d41640 1 -- 172.21.15.83:0/1155889967 >> 172.21.15.83:0/1155889967 conn(0x7fd4cc0cc050 msgr2=0x7fd4cc0cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:45.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.755+0000 7fd4d2d41640 1 -- 172.21.15.83:0/1155889967 shutdown_connections 2024-04-17T10:54:45.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.755+0000 7fd4d2d41640 1 -- 172.21.15.83:0/1155889967 wait complete. 2024-04-17T10:54:45.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.755+0000 7fd4d2d41640 1 Processor -- start 2024-04-17T10:54:45.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.755+0000 7fd4d2d41640 1 -- start start 2024-04-17T10:54:45.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.759+0000 7fd4d2d41640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4cc06b120 0x7fd4cc1571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:45.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.759+0000 7fd4d2d41640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4cc0aedf0 0x7fd4cc148c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:45.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.759+0000 7fd4d2d41640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4cc15cb40 0x7fd4cc1560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:45.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.759+0000 7fd4d2d41640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc169c30 con 0x7fd4cc15cb40 2024-04-17T10:54:45.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.759+0000 7fd4d2d41640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc169ab0 con 0x7fd4cc06b120 2024-04-17T10:54:45.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.759+0000 7fd4d2d41640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc169db0 con 0x7fd4cc0aedf0 2024-04-17T10:54:45.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.759+0000 7fd4d12de640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4cc15cb40 0x7fd4cc1560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:45.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.759+0000 7fd4d12de640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4cc15cb40 0x7fd4cc1560c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58188/0 (socket says 172.21.15.83:58188) 2024-04-17T10:54:45.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.759+0000 7fd4d12de640 1 -- 172.21.15.83:0/1342999296 learned_addr learned my addr 172.21.15.83:0/1342999296 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:45.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.759+0000 7fd4d12de640 1 -- 172.21.15.83:0/1342999296 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4cc0aedf0 msgr2=0x7fd4cc148c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:45.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.759+0000 7fd4d12de640 1 --2- 172.21.15.83:0/1342999296 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4cc0aedf0 0x7fd4cc148c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:45.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.759+0000 7fd4d12de640 1 -- 172.21.15.83:0/1342999296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4cc06b120 msgr2=0x7fd4cc1571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:45.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.759+0000 7fd4d0add640 1 --2- 172.21.15.83:0/1342999296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4cc06b120 0x7fd4cc1571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:45.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.759+0000 7fd4d12de640 1 --2- 172.21.15.83:0/1342999296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4cc06b120 0x7fd4cc1571f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:45.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.759+0000 7fd4d12de640 1 -- 172.21.15.83:0/1342999296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4cc155a60 con 0x7fd4cc15cb40 2024-04-17T10:54:45.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.759+0000 7fd4d12de640 1 --2- 172.21.15.83:0/1342999296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4cc15cb40 0x7fd4cc1560c0 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7fd4c400c7b0 tx=0x7fd4c400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:45.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.759+0000 7fd4c17fa640 1 -- 172.21.15.83:0/1342999296 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c400ce60 con 0x7fd4cc15cb40 2024-04-17T10:54:45.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.759+0000 7fd4c17fa640 1 -- 172.21.15.83:0/1342999296 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c4004b20 con 0x7fd4cc15cb40 2024-04-17T10:54:45.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.759+0000 7fd4d2d41640 1 -- 172.21.15.83:0/1342999296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4cc154ca0 con 0x7fd4cc15cb40 2024-04-17T10:54:45.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.759+0000 7fd4c17fa640 1 -- 172.21.15.83:0/1342999296 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c400a850 con 0x7fd4cc15cb40 2024-04-17T10:54:45.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.759+0000 7fd4d2d41640 1 -- 172.21.15.83:0/1342999296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4cc155260 con 0x7fd4cc15cb40 2024-04-17T10:54:45.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.759+0000 7fd4d2d41640 1 -- 172.21.15.83:0/1342999296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4cc075000 con 0x7fd4cc15cb40 2024-04-17T10:54:45.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.759+0000 7fd4c17fa640 1 -- 172.21.15.83:0/1342999296 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd4c400a9f0 con 0x7fd4cc15cb40 2024-04-17T10:54:45.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.759+0000 7fd4c17fa640 1 --2- 172.21.15.83:0/1342999296 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd4b8066f60 0x7fd4b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:45.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.759+0000 7fd4c17fa640 1 -- 172.21.15.83:0/1342999296 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fd4c4085520 con 0x7fd4cc15cb40 2024-04-17T10:54:45.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.759+0000 7fd4d0add640 1 --2- 172.21.15.83:0/1342999296 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd4b8066f60 0x7fd4b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:45.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.759+0000 7fd4d0add640 1 --2- 172.21.15.83:0/1342999296 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd4b8066f60 0x7fd4b8069420 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8000f70 tx=0x7fd4c8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:45.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.763+0000 7fd4c17fa640 1 -- 172.21.15.83:0/1342999296 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd4c40564c0 con 0x7fd4cc15cb40 2024-04-17T10:54:45.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:45.923+0000 7fd4d2d41640 1 -- 172.21.15.83:0/1342999296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd4cc167670 con 0x7fd4cc15cb40 2024-04-17T10:54:46.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.623+0000 7fd4c17fa640 1 -- 172.21.15.83:0/1342999296 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v144) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd4c4055dc0 con 0x7fd4cc15cb40 2024-04-17T10:54:46.616 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2024-04-17T10:54:46.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.623+0000 7fd4d2d41640 1 -- 172.21.15.83:0/1342999296 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd4b8066f60 msgr2=0x7fd4b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:46.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.623+0000 7fd4d2d41640 1 --2- 172.21.15.83:0/1342999296 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd4b8066f60 0x7fd4b8069420 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8000f70 tx=0x7fd4c8009040 comp rx=0 tx=0).stop 2024-04-17T10:54:46.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.623+0000 7fd4d2d41640 1 -- 172.21.15.83:0/1342999296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4cc15cb40 msgr2=0x7fd4cc1560c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:46.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.623+0000 7fd4d2d41640 1 --2- 172.21.15.83:0/1342999296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4cc15cb40 0x7fd4cc1560c0 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7fd4c400c7b0 tx=0x7fd4c400cc70 comp rx=0 tx=0).stop 2024-04-17T10:54:46.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.623+0000 7fd4d2d41640 1 -- 172.21.15.83:0/1342999296 shutdown_connections 2024-04-17T10:54:46.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.623+0000 7fd4d2d41640 1 --2- 172.21.15.83:0/1342999296 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fd4b8066f60 0x7fd4b8069420 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:46.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.623+0000 7fd4d2d41640 1 --2- 172.21.15.83:0/1342999296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4cc15cb40 0x7fd4cc1560c0 unknown :-1 s=CLOSED pgs=2454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:46.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.623+0000 7fd4d2d41640 1 --2- 172.21.15.83:0/1342999296 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4cc0aedf0 0x7fd4cc148c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:46.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.623+0000 7fd4d2d41640 1 --2- 172.21.15.83:0/1342999296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4cc06b120 0x7fd4cc1571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:46.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.623+0000 7fd4d2d41640 1 -- 172.21.15.83:0/1342999296 >> 172.21.15.83:0/1342999296 conn(0x7fd4cc0cc050 msgr2=0x7fd4cc15d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:46.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.623+0000 7fd4d2d41640 1 -- 172.21.15.83:0/1342999296 shutdown_connections 2024-04-17T10:54:46.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.623+0000 7fd4d2d41640 1 -- 172.21.15.83:0/1342999296 wait complete. 2024-04-17T10:54:46.641 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-17T10:54:46.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f4310b3e640 1 Processor -- start 2024-04-17T10:54:46.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f4310b3e640 1 -- start start 2024-04-17T10:54:46.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f4310b3e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f430c06b2f0 0x7f430c15b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:46.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f4310b3e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f430c0cf3e0 con 0x7f430c0ced50 2024-04-17T10:54:46.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f4310b3e640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f430c0cd890 con 0x7f430c0aebd0 2024-04-17T10:54:46.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f4310b3e640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f430c15bed0 con 0x7f430c06b2f0 2024-04-17T10:54:46.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f4309d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f430c0ced50 0x7f430c06abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42252/0 (socket says 172.21.15.83:42252) 2024-04-17T10:54:46.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f4309d9b640 1 -- 172.21.15.83:0/2228174821 learned_addr learned my addr 172.21.15.83:0/2228174821 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:46.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f430ad9d640 1 --2- 172.21.15.83:0/2228174821 >> v2:172.21.15.167:3300/0 conn(0x7f430c06b2f0 0x7f430c15b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:46.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f430959a640 1 -- 172.21.15.83:0/2228174821 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1387322005 0 0) 0x7f430c0cf3e0 con 0x7f430c0ced50 2024-04-17T10:54:46.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f430959a640 1 -- 172.21.15.83:0/2228174821 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42f4003530 con 0x7f430c0ced50 2024-04-17T10:54:46.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f430959a640 1 -- 172.21.15.83:0/2228174821 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 862807178 0 0) 0x7f42f4003530 con 0x7f430c0ced50 2024-04-17T10:54:46.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f430959a640 1 -- 172.21.15.83:0/2228174821 >> v2:172.21.15.167:3300/0 conn(0x7f430c06b2f0 msgr2=0x7f430c15b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:46.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f430959a640 1 --2- 172.21.15.83:0/2228174821 >> v2:172.21.15.167:3300/0 conn(0x7f430c06b2f0 0x7f430c15b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:46.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f430959a640 1 -- 172.21.15.83:0/2228174821 >> v1:172.21.15.167:6789/0 conn(0x7f430c0aebd0 legacy=0x7f430c0cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:46.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f430959a640 1 -- 172.21.15.83:0/2228174821 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f430c0b4230 con 0x7f430c0ced50 2024-04-17T10:54:46.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f430959a640 1 -- 172.21.15.83:0/2228174821 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f4300001680 con 0x7f430c0ced50 2024-04-17T10:54:46.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f430ad9d640 1 --2- 172.21.15.83:0/2228174821 >> v2:172.21.15.167:3300/0 conn(0x7f430c06b2f0 0x7f430c15b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:54:46.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f430959a640 1 -- 172.21.15.83:0/2228174821 >> v1:172.21.15.83:6789/0 conn(0x7f430c0ced50 legacy=0x7f430c06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:46.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f430959a640 1 --2- 172.21.15.83:0/2228174821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42f40039c0 0x7f42f4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:46.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f430959a640 1 --2- 172.21.15.83:0/2228174821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42f4004810 0x7f42f4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:46.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f430959a640 1 --2- 172.21.15.83:0/2228174821 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f40051d0 0x7f42f400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:46.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f430959a640 1 -- 172.21.15.83:0/2228174821 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430c15bed0 con 0x7f42f40039c0 2024-04-17T10:54:46.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f430959a640 1 -- 172.21.15.83:0/2228174821 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f400c120 con 0x7f42f4004810 2024-04-17T10:54:46.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f430959a640 1 -- 172.21.15.83:0/2228174821 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f400c320 con 0x7f42f40051d0 2024-04-17T10:54:46.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f430a59c640 1 --2- 172.21.15.83:0/2228174821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42f40039c0 0x7f42f4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:46.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f4309d9b640 1 --2- 172.21.15.83:0/2228174821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42f4004810 0x7f42f4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:46.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f430a59c640 1 -- 172.21.15.83:0/2228174821 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f40051d0 msgr2=0x7f42f400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:46.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f430a59c640 1 --2- 172.21.15.83:0/2228174821 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f40051d0 0x7f42f400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:46.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f430a59c640 1 -- 172.21.15.83:0/2228174821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42f4004810 msgr2=0x7f42f4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:46.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f430a59c640 1 --2- 172.21.15.83:0/2228174821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42f4004810 0x7f42f4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:46.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f430ad9d640 1 -- 172.21.15.83:0/2228174821 reap_dead start 2024-04-17T10:54:46.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.775+0000 7f430a59c640 1 -- 172.21.15.83:0/2228174821 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f4003530 con 0x7f42f40039c0 2024-04-17T10:54:46.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.779+0000 7f430a59c640 1 --2- 172.21.15.83:0/2228174821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42f40039c0 0x7f42f4003de0 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7f42fc001780 tx=0x7f42fc01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8152c9cac16669 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:46.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.779+0000 7f430959a640 1 -- 172.21.15.83:0/2228174821 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42fc0013a0 con 0x7f42f40039c0 2024-04-17T10:54:46.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.779+0000 7f430959a640 1 -- 172.21.15.83:0/2228174821 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42fc0052f0 con 0x7f42f40039c0 2024-04-17T10:54:46.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.779+0000 7f430959a640 1 -- 172.21.15.83:0/2228174821 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42fc003840 con 0x7f42f40039c0 2024-04-17T10:54:46.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.779+0000 7f4310b3e640 1 -- 172.21.15.83:0/2228174821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42f40039c0 msgr2=0x7f42f4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:46.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.779+0000 7f4310b3e640 1 --2- 172.21.15.83:0/2228174821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42f40039c0 0x7f42f4003de0 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7f42fc001780 tx=0x7f42fc01c400 comp rx=0 tx=0).stop 2024-04-17T10:54:46.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.779+0000 7f4310b3e640 1 -- 172.21.15.83:0/2228174821 shutdown_connections 2024-04-17T10:54:46.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.779+0000 7f4310b3e640 1 --2- 172.21.15.83:0/2228174821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42f40039c0 0x7f42f4003de0 unknown :-1 s=CLOSED pgs=2456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:46.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.779+0000 7f4310b3e640 1 -- 172.21.15.83:0/2228174821 >> 172.21.15.83:0/2228174821 conn(0x7f430c0cc050 msgr2=0x7f430c0cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:46.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.779+0000 7f4310b3e640 1 -- 172.21.15.83:0/2228174821 shutdown_connections 2024-04-17T10:54:46.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.779+0000 7f4310b3e640 1 -- 172.21.15.83:0/2228174821 wait complete. 2024-04-17T10:54:46.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.783+0000 7f4310b3e640 1 Processor -- start 2024-04-17T10:54:46.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.783+0000 7f4310b3e640 1 -- start start 2024-04-17T10:54:46.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.783+0000 7f4310b3e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f40039c0 0x7f430c145570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:46.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.783+0000 7f4310b3e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f430c1f83b0 0x7f430c1f8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:46.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.783+0000 7f4310b3e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f430c0a94a0 0x7f430c0a9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:46.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.783+0000 7f4310b3e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430c0cf160 con 0x7f430c1f83b0 2024-04-17T10:54:46.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.783+0000 7f4310b3e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430c15e5c0 con 0x7f430c0a94a0 2024-04-17T10:54:46.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.783+0000 7f4310b3e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f430c0cd590 con 0x7f42f40039c0 2024-04-17T10:54:46.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.783+0000 7f4309d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f430c1f83b0 0x7f430c1f8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:46.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.783+0000 7f4309d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f430c1f83b0 0x7f430c1f8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44232/0 (socket says 172.21.15.83:44232) 2024-04-17T10:54:46.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.783+0000 7f4309d9b640 1 -- 172.21.15.83:0/4235937851 learned_addr learned my addr 172.21.15.83:0/4235937851 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:46.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.783+0000 7f430a59c640 1 --2- 172.21.15.83:0/4235937851 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f40039c0 0x7f430c145570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:46.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.783+0000 7f4309d9b640 1 -- 172.21.15.83:0/4235937851 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f40039c0 msgr2=0x7f430c145570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:46.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.783+0000 7f4309d9b640 1 --2- 172.21.15.83:0/4235937851 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f40039c0 0x7f430c145570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:46.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.783+0000 7f4309d9b640 1 -- 172.21.15.83:0/4235937851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f430c0a94a0 msgr2=0x7f430c0a9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:46.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.783+0000 7f4309d9b640 1 --2- 172.21.15.83:0/4235937851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f430c0a94a0 0x7f430c0a9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:46.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.783+0000 7f4309d9b640 1 -- 172.21.15.83:0/4235937851 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f430c0aa1d0 con 0x7f430c1f83b0 2024-04-17T10:54:46.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.783+0000 7f4309d9b640 1 --2- 172.21.15.83:0/4235937851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f430c1f83b0 0x7f430c1f8830 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7f430000e800 tx=0x7f430000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:46.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.783+0000 7f42faffd640 1 -- 172.21.15.83:0/4235937851 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430000c6c0 con 0x7f430c1f83b0 2024-04-17T10:54:46.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.783+0000 7f42faffd640 1 -- 172.21.15.83:0/4235937851 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4300018070 con 0x7f430c1f83b0 2024-04-17T10:54:46.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.783+0000 7f42faffd640 1 -- 172.21.15.83:0/4235937851 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430000caa0 con 0x7f430c1f83b0 2024-04-17T10:54:46.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.783+0000 7f4310b3e640 1 -- 172.21.15.83:0/4235937851 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f430c0aa4b0 con 0x7f430c1f83b0 2024-04-17T10:54:46.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.783+0000 7f4310b3e640 1 -- 172.21.15.83:0/4235937851 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f430c1e7850 con 0x7f430c1f83b0 2024-04-17T10:54:46.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.783+0000 7f4310b3e640 1 -- 172.21.15.83:0/4235937851 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f430c075000 con 0x7f430c1f83b0 2024-04-17T10:54:46.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.783+0000 7f42faffd640 1 -- 172.21.15.83:0/4235937851 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f430000cc40 con 0x7f430c1f83b0 2024-04-17T10:54:46.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.783+0000 7f42faffd640 1 --2- 172.21.15.83:0/4235937851 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f42ec066f60 0x7f42ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:46.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.783+0000 7f42faffd640 1 -- 172.21.15.83:0/4235937851 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5386+0+0 (secure 0 0 0) 0x7f4300084c60 con 0x7f430c1f83b0 2024-04-17T10:54:46.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.783+0000 7f430a59c640 1 --2- 172.21.15.83:0/4235937851 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f42ec066f60 0x7f42ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:46.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.787+0000 7f430a59c640 1 --2- 172.21.15.83:0/4235937851 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f42ec066f60 0x7f42ec069420 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f42fc001750 tx=0x7f42fc0011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:46.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.787+0000 7f42faffd640 1 -- 172.21.15.83:0/4235937851 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4300055e40 con 0x7f430c1f83b0 2024-04-17T10:54:46.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:46.951+0000 7f4310b3e640 1 -- 172.21.15.83:0/4235937851 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f430c159830 con 0x7f430c1f83b0 2024-04-17T10:54:47.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.623+0000 7f42faffd640 1 -- 172.21.15.83:0/4235937851 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v145) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4300055740 con 0x7f430c1f83b0 2024-04-17T10:54:47.619 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2024-04-17T10:54:47.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.631+0000 7f4310b3e640 1 -- 172.21.15.83:0/4235937851 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f42ec066f60 msgr2=0x7f42ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:47.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.631+0000 7f4310b3e640 1 --2- 172.21.15.83:0/4235937851 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f42ec066f60 0x7f42ec069420 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f42fc001750 tx=0x7f42fc0011a0 comp rx=0 tx=0).stop 2024-04-17T10:54:47.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.631+0000 7f4310b3e640 1 -- 172.21.15.83:0/4235937851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f430c1f83b0 msgr2=0x7f430c1f8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:47.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.631+0000 7f4310b3e640 1 --2- 172.21.15.83:0/4235937851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f430c1f83b0 0x7f430c1f8830 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7f430000e800 tx=0x7f430000ecc0 comp rx=0 tx=0).stop 2024-04-17T10:54:47.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.631+0000 7f4310b3e640 1 -- 172.21.15.83:0/4235937851 shutdown_connections 2024-04-17T10:54:47.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.631+0000 7f4310b3e640 1 --2- 172.21.15.83:0/4235937851 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f42ec066f60 0x7f42ec069420 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:47.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.631+0000 7f4310b3e640 1 --2- 172.21.15.83:0/4235937851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f430c0a94a0 0x7f430c0a9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:47.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.631+0000 7f4310b3e640 1 --2- 172.21.15.83:0/4235937851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f430c1f83b0 0x7f430c1f8830 unknown :-1 s=CLOSED pgs=2457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:47.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.631+0000 7f4310b3e640 1 --2- 172.21.15.83:0/4235937851 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f40039c0 0x7f430c145570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:47.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.631+0000 7f4310b3e640 1 -- 172.21.15.83:0/4235937851 >> 172.21.15.83:0/4235937851 conn(0x7f430c0cc050 msgr2=0x7f430c15c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:47.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.631+0000 7f4310b3e640 1 -- 172.21.15.83:0/4235937851 shutdown_connections 2024-04-17T10:54:47.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.631+0000 7f4310b3e640 1 -- 172.21.15.83:0/4235937851 wait complete. 2024-04-17T10:54:47.655 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-17T10:54:47.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.779+0000 7f68dc189640 1 Processor -- start 2024-04-17T10:54:47.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.779+0000 7f68dc189640 1 -- start start 2024-04-17T10:54:47.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.783+0000 7f68dc189640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68d4154de0 0x7f68d406ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:47.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.783+0000 7f68dc189640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f68d406b350 0x7f68d40b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:47.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.783+0000 7f68dc189640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68d415bd70 con 0x7f68d40b4040 2024-04-17T10:54:47.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.783+0000 7f68dc189640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68d40b4870 con 0x7f68d406b350 2024-04-17T10:54:47.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.783+0000 7f68dc189640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d415bf60 con 0x7f68d4154de0 2024-04-17T10:54:47.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.783+0000 7f68d9724640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f68d406b350 0x7f68d40b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:47.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.783+0000 7f68da726640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f68d40b4040 0x7f68d4071580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42254/0 (socket says 172.21.15.83:42254) 2024-04-17T10:54:47.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.783+0000 7f68da726640 1 -- 172.21.15.83:0/991464781 learned_addr learned my addr 172.21.15.83:0/991464781 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:47.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.783+0000 7f68d9f25640 1 --2- 172.21.15.83:0/991464781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68d4154de0 0x7f68d406ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:47.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.783+0000 7f68d9724640 1 -- 172.21.15.83:0/991464781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68d4154de0 msgr2=0x7f68d406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:47.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.783+0000 7f68d9724640 1 --2- 172.21.15.83:0/991464781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68d4154de0 0x7f68d406ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:47.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.783+0000 7f68d9724640 1 -- 172.21.15.83:0/991464781 >> v1:172.21.15.83:6789/0 conn(0x7f68d40b4040 legacy=0x7f68d4071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:47.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.783+0000 7f68d9724640 1 -- 172.21.15.83:0/991464781 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d415afb0 con 0x7f68d406b350 2024-04-17T10:54:47.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.783+0000 7f68d8f23640 1 -- 172.21.15.83:0/991464781 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 157504508 0 0) 0x7f68d415bd70 con 0x7f68d40b4040 2024-04-17T10:54:47.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.783+0000 7f68d9f25640 1 --2- 172.21.15.83:0/991464781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68d4154de0 0x7f68d406ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:47.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.783+0000 7f68d9724640 1 --2- 172.21.15.83:0/991464781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68d406b350 0x7f68d40b3b00 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f68c40097c0 tx=0x7f68c401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d686b925f3b36386 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:47.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.783+0000 7f68d8f23640 1 -- 172.21.15.83:0/991464781 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c40b0070 con 0x7f68d406b350 2024-04-17T10:54:47.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.783+0000 7f68d8f23640 1 -- 172.21.15.83:0/991464781 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c40a4e50 con 0x7f68d406b350 2024-04-17T10:54:47.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.783+0000 7f68d8f23640 1 -- 172.21.15.83:0/991464781 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c40ab820 con 0x7f68d406b350 2024-04-17T10:54:47.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.787+0000 7f68dc189640 1 -- 172.21.15.83:0/991464781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68d406b350 msgr2=0x7f68d40b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:47.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.787+0000 7f68dc189640 1 --2- 172.21.15.83:0/991464781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68d406b350 0x7f68d40b3b00 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f68c40097c0 tx=0x7f68c401c4a0 comp rx=0 tx=0).stop 2024-04-17T10:54:47.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.787+0000 7f68dc189640 1 -- 172.21.15.83:0/991464781 shutdown_connections 2024-04-17T10:54:47.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.787+0000 7f68dc189640 1 --2- 172.21.15.83:0/991464781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68d406b350 0x7f68d40b3b00 unknown :-1 s=CLOSED pgs=2459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:47.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.787+0000 7f68dc189640 1 --2- 172.21.15.83:0/991464781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68d4154de0 0x7f68d406ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:47.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.787+0000 7f68dc189640 1 -- 172.21.15.83:0/991464781 >> 172.21.15.83:0/991464781 conn(0x7f68d40b15b0 msgr2=0x7f68d40b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:47.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.787+0000 7f68dc189640 1 -- 172.21.15.83:0/991464781 shutdown_connections 2024-04-17T10:54:47.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.787+0000 7f68dc189640 1 -- 172.21.15.83:0/991464781 wait complete. 2024-04-17T10:54:47.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.787+0000 7f68dc189640 1 Processor -- start 2024-04-17T10:54:47.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.787+0000 7f68dc189640 1 -- start start 2024-04-17T10:54:47.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.787+0000 7f68dc189640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68d406b350 0x7f68d41508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:47.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.787+0000 7f68dc189640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68d40b4040 0x7f68d414d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:47.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.787+0000 7f68dc189640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68d4154de0 0x7f68d4153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:47.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.787+0000 7f68dc189640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d40b4490 con 0x7f68d40b4040 2024-04-17T10:54:47.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.787+0000 7f68dc189640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d41551c0 con 0x7f68d406b350 2024-04-17T10:54:47.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.787+0000 7f68dc189640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d40b4610 con 0x7f68d4154de0 2024-04-17T10:54:47.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.787+0000 7f68d9724640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68d40b4040 0x7f68d414d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:47.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.787+0000 7f68d9724640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68d40b4040 0x7f68d414d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44258/0 (socket says 172.21.15.83:44258) 2024-04-17T10:54:47.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.787+0000 7f68d9724640 1 -- 172.21.15.83:0/79798334 learned_addr learned my addr 172.21.15.83:0/79798334 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:47.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.787+0000 7f68d9724640 1 -- 172.21.15.83:0/79798334 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68d4154de0 msgr2=0x7f68d4153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:47.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.787+0000 7f68d9724640 1 --2- 172.21.15.83:0/79798334 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68d4154de0 0x7f68d4153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:47.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.787+0000 7f68d9724640 1 -- 172.21.15.83:0/79798334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68d406b350 msgr2=0x7f68d41508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:47.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.787+0000 7f68d9724640 1 --2- 172.21.15.83:0/79798334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68d406b350 0x7f68d41508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:47.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.787+0000 7f68d9724640 1 -- 172.21.15.83:0/79798334 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d414dc30 con 0x7f68d40b4040 2024-04-17T10:54:47.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.787+0000 7f68d9724640 1 --2- 172.21.15.83:0/79798334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68d40b4040 0x7f68d414d090 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7f68c401a0d0 tx=0x7f68c4019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:47.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.787+0000 7f68ca7fc640 1 -- 172.21.15.83:0/79798334 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c40b8020 con 0x7f68d40b4040 2024-04-17T10:54:47.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.787+0000 7f68ca7fc640 1 -- 172.21.15.83:0/79798334 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c40b4410 con 0x7f68d40b4040 2024-04-17T10:54:47.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.787+0000 7f68dc189640 1 -- 172.21.15.83:0/79798334 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68d414df10 con 0x7f68d40b4040 2024-04-17T10:54:47.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.787+0000 7f68ca7fc640 1 -- 172.21.15.83:0/79798334 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c40b0040 con 0x7f68d40b4040 2024-04-17T10:54:47.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.787+0000 7f68dc189640 1 -- 172.21.15.83:0/79798334 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68d414e4d0 con 0x7f68d40b4040 2024-04-17T10:54:47.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.791+0000 7f68dc189640 1 -- 172.21.15.83:0/79798334 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68d4074f90 con 0x7f68d40b4040 2024-04-17T10:54:47.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.791+0000 7f68ca7fc640 1 -- 172.21.15.83:0/79798334 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f68c40bb020 con 0x7f68d40b4040 2024-04-17T10:54:47.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.791+0000 7f68ca7fc640 1 --2- 172.21.15.83:0/79798334 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f68b8066f60 0x7f68b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:47.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.791+0000 7f68d9f25640 1 --2- 172.21.15.83:0/79798334 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f68b8066f60 0x7f68b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:47.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.791+0000 7f68ca7fc640 1 -- 172.21.15.83:0/79798334 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f68c411eb70 con 0x7f68d40b4040 2024-04-17T10:54:47.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.791+0000 7f68d9f25640 1 --2- 172.21.15.83:0/79798334 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f68b8066f60 0x7f68b8069420 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f68d000a8d0 tx=0x7f68d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:47.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.795+0000 7f68ca7fc640 1 -- 172.21.15.83:0/79798334 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f68c40c2030 con 0x7f68d40b4040 2024-04-17T10:54:47.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.959+0000 7f68dc189640 1 -- 172.21.15.83:0/79798334 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f68d40b3030 con 0x7f68d40b4040 2024-04-17T10:54:47.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.963+0000 7f68ca7fc640 1 -- 172.21.15.83:0/79798334 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f68c4016090 con 0x7f68d40b4040 2024-04-17T10:54:47.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.963+0000 7f68dc189640 1 -- 172.21.15.83:0/79798334 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f68b8066f60 msgr2=0x7f68b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:47.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.963+0000 7f68dc189640 1 --2- 172.21.15.83:0/79798334 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f68b8066f60 0x7f68b8069420 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f68d000a8d0 tx=0x7f68d0008040 comp rx=0 tx=0).stop 2024-04-17T10:54:47.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.963+0000 7f68dc189640 1 -- 172.21.15.83:0/79798334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68d40b4040 msgr2=0x7f68d414d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:47.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.963+0000 7f68dc189640 1 --2- 172.21.15.83:0/79798334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68d40b4040 0x7f68d414d090 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7f68c401a0d0 tx=0x7f68c4019240 comp rx=0 tx=0).stop 2024-04-17T10:54:47.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.963+0000 7f68dc189640 1 -- 172.21.15.83:0/79798334 shutdown_connections 2024-04-17T10:54:47.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.963+0000 7f68dc189640 1 --2- 172.21.15.83:0/79798334 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f68b8066f60 0x7f68b8069420 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:47.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.963+0000 7f68dc189640 1 --2- 172.21.15.83:0/79798334 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68d4154de0 0x7f68d4153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:47.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.963+0000 7f68dc189640 1 --2- 172.21.15.83:0/79798334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68d40b4040 0x7f68d414d090 unknown :-1 s=CLOSED pgs=2460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:47.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.963+0000 7f68dc189640 1 --2- 172.21.15.83:0/79798334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68d406b350 0x7f68d41508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:47.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.963+0000 7f68dc189640 1 -- 172.21.15.83:0/79798334 >> 172.21.15.83:0/79798334 conn(0x7f68d40b15b0 msgr2=0x7f68d40cee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:47.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.967+0000 7f68dc189640 1 -- 172.21.15.83:0/79798334 shutdown_connections 2024-04-17T10:54:47.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:47.967+0000 7f68dc189640 1 -- 172.21.15.83:0/79798334 wait complete. 2024-04-17T10:54:47.977 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive' 2024-04-17T10:54:48.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.095+0000 7f1b0c3da640 1 Processor -- start 2024-04-17T10:54:48.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.095+0000 7f1b0c3da640 1 -- start start 2024-04-17T10:54:48.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.095+0000 7f1b0c3da640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1b04072330 0x7f1b040cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:48.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.095+0000 7f1b0c3da640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b040cfa10 0x7f1b0406c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:48.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.095+0000 7f1b0c3da640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b040cfea0 con 0x7f1b0415ba30 2024-04-17T10:54:48.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.095+0000 7f1b0c3da640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b04071ea0 con 0x7f1b04072330 2024-04-17T10:54:48.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.095+0000 7f1b0c3da640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0406e320 con 0x7f1b040cfa10 2024-04-17T10:54:48.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.095+0000 7f1b09975640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1b04072330 0x7f1b040cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:48.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.095+0000 7f1b0a176640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1b0415ba30 0x7f1b0415c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42258/0 (socket says 172.21.15.83:42258) 2024-04-17T10:54:48.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.095+0000 7f1b0a176640 1 -- 172.21.15.83:0/784369323 learned_addr learned my addr 172.21.15.83:0/784369323 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:48.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.095+0000 7f1b09975640 1 -- 172.21.15.83:0/784369323 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b040cfa10 msgr2=0x7f1b0406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:48.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.095+0000 7f1b09975640 1 --2- 172.21.15.83:0/784369323 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b040cfa10 0x7f1b0406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:48.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.095+0000 7f1b09975640 1 -- 172.21.15.83:0/784369323 >> v1:172.21.15.83:6789/0 conn(0x7f1b0415ba30 legacy=0x7f1b0415c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:48.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.095+0000 7f1b09975640 1 -- 172.21.15.83:0/784369323 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0406e040 con 0x7f1b04072330 2024-04-17T10:54:48.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.099+0000 7f1b09975640 1 --2- 172.21.15.83:0/784369323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b04072330 0x7f1b040cf4d0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7f1b04072020 tx=0x7f1af801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4955631be49990 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:48.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.099+0000 7f1b09174640 1 -- 172.21.15.83:0/784369323 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af80b0070 con 0x7f1b04072330 2024-04-17T10:54:48.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.099+0000 7f1b09174640 1 -- 172.21.15.83:0/784369323 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af80ab600 con 0x7f1b04072330 2024-04-17T10:54:48.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.099+0000 7f1b09174640 1 -- 172.21.15.83:0/784369323 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af80ab9d0 con 0x7f1b04072330 2024-04-17T10:54:48.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.099+0000 7f1b0c3da640 1 -- 172.21.15.83:0/784369323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b04072330 msgr2=0x7f1b040cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:48.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.099+0000 7f1b0c3da640 1 --2- 172.21.15.83:0/784369323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b04072330 0x7f1b040cf4d0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7f1b04072020 tx=0x7f1af801c410 comp rx=0 tx=0).stop 2024-04-17T10:54:48.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.099+0000 7f1b0c3da640 1 -- 172.21.15.83:0/784369323 shutdown_connections 2024-04-17T10:54:48.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.099+0000 7f1b0c3da640 1 --2- 172.21.15.83:0/784369323 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b040cfa10 0x7f1b0406c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:48.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.099+0000 7f1b0c3da640 1 --2- 172.21.15.83:0/784369323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b04072330 0x7f1b040cf4d0 unknown :-1 s=CLOSED pgs=2462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:48.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.099+0000 7f1b0c3da640 1 -- 172.21.15.83:0/784369323 >> 172.21.15.83:0/784369323 conn(0x7f1b0406abe0 msgr2=0x7f1b0406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:48.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.099+0000 7f1b0c3da640 1 -- 172.21.15.83:0/784369323 shutdown_connections 2024-04-17T10:54:48.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.099+0000 7f1b0c3da640 1 -- 172.21.15.83:0/784369323 wait complete. 2024-04-17T10:54:48.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.099+0000 7f1b0c3da640 1 Processor -- start 2024-04-17T10:54:48.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.099+0000 7f1b0c3da640 1 -- start start 2024-04-17T10:54:48.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.103+0000 7f1b0c3da640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b04072330 0x7f1b04141240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:48.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.103+0000 7f1b0c3da640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b040cfa10 0x7f1b04157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:48.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.103+0000 7f1b0c3da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b0415ba30 0x7f1b04158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:48.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.103+0000 7f1b0c3da640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0406de20 con 0x7f1b0415ba30 2024-04-17T10:54:48.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.103+0000 7f1b0c3da640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0406dca0 con 0x7f1b04072330 2024-04-17T10:54:48.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.103+0000 7f1b0c3da640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b04168520 con 0x7f1b040cfa10 2024-04-17T10:54:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.103+0000 7f1b0a977640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b0415ba30 0x7f1b04158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.103+0000 7f1b0a977640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b0415ba30 0x7f1b04158370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44280/0 (socket says 172.21.15.83:44280) 2024-04-17T10:54:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.103+0000 7f1b0a977640 1 -- 172.21.15.83:0/2678531607 learned_addr learned my addr 172.21.15.83:0/2678531607 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.103+0000 7f1b0a977640 1 -- 172.21.15.83:0/2678531607 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b040cfa10 msgr2=0x7f1b04157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.103+0000 7f1b0a977640 1 --2- 172.21.15.83:0/2678531607 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b040cfa10 0x7f1b04157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.103+0000 7f1b0a977640 1 -- 172.21.15.83:0/2678531607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b04072330 msgr2=0x7f1b04141240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.103+0000 7f1b0a977640 1 --2- 172.21.15.83:0/2678531607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b04072330 0x7f1b04141240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.103+0000 7f1b0a977640 1 -- 172.21.15.83:0/2678531607 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0415e350 con 0x7f1b0415ba30 2024-04-17T10:54:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.103+0000 7f1b0a977640 1 --2- 172.21.15.83:0/2678531607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b0415ba30 0x7f1b04158370 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00b970 tx=0x7f1afc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.103+0000 7f1af6ffd640 1 -- 172.21.15.83:0/2678531607 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc019070 con 0x7f1b0415ba30 2024-04-17T10:54:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.103+0000 7f1af6ffd640 1 -- 172.21.15.83:0/2678531607 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1afc00cb60 con 0x7f1b0415ba30 2024-04-17T10:54:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.103+0000 7f1b0c3da640 1 -- 172.21.15.83:0/2678531607 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b0415e630 con 0x7f1b0415ba30 2024-04-17T10:54:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.103+0000 7f1af6ffd640 1 -- 172.21.15.83:0/2678531607 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc014440 con 0x7f1b0415ba30 2024-04-17T10:54:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.103+0000 7f1b0c3da640 1 -- 172.21.15.83:0/2678531607 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b0415d5e0 con 0x7f1b0415ba30 2024-04-17T10:54:48.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.103+0000 7f1b0c3da640 1 -- 172.21.15.83:0/2678531607 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b04074f90 con 0x7f1b0415ba30 2024-04-17T10:54:48.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.103+0000 7f1af6ffd640 1 -- 172.21.15.83:0/2678531607 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1afc01c070 con 0x7f1b0415ba30 2024-04-17T10:54:48.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.103+0000 7f1af6ffd640 1 --2- 172.21.15.83:0/2678531607 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1ae8066f90 0x7f1ae8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:48.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.103+0000 7f1af6ffd640 1 -- 172.21.15.83:0/2678531607 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1afc0849e0 con 0x7f1b0415ba30 2024-04-17T10:54:48.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.103+0000 7f1b0a176640 1 --2- 172.21.15.83:0/2678531607 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1ae8066f90 0x7f1ae8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:48.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.103+0000 7f1b0a176640 1 --2- 172.21.15.83:0/2678531607 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1ae8066f90 0x7f1ae8069450 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f1b00000f70 tx=0x7f1b00007a90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:48.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.107+0000 7f1af6ffd640 1 -- 172.21.15.83:0/2678531607 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1afc055d80 con 0x7f1b0415ba30 2024-04-17T10:54:48.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.323+0000 7f1b0c3da640 1 -- 172.21.15.83:0/2678531607 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]} v 0) v1 -- 0x7f1b040708f0 con 0x7f1b0415ba30 2024-04-17T10:54:48.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.647+0000 7f1af6ffd640 1 -- 172.21.15.83:0/2678531607 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]}]=0 v776) v1 ==== 159+0+0 (secure 0 0 0) 0x7f1afc055680 con 0x7f1b0415ba30 2024-04-17T10:54:48.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.651+0000 7f1b0c3da640 1 -- 172.21.15.83:0/2678531607 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1ae8066f90 msgr2=0x7f1ae8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:48.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.651+0000 7f1b0c3da640 1 --2- 172.21.15.83:0/2678531607 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1ae8066f90 0x7f1ae8069450 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f1b00000f70 tx=0x7f1b00007a90 comp rx=0 tx=0).stop 2024-04-17T10:54:48.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.651+0000 7f1b0c3da640 1 -- 172.21.15.83:0/2678531607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b0415ba30 msgr2=0x7f1b04158370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:48.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.651+0000 7f1b0c3da640 1 --2- 172.21.15.83:0/2678531607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b0415ba30 0x7f1b04158370 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00b970 tx=0x7f1afc00be30 comp rx=0 tx=0).stop 2024-04-17T10:54:48.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.651+0000 7f1b0c3da640 1 -- 172.21.15.83:0/2678531607 shutdown_connections 2024-04-17T10:54:48.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.651+0000 7f1b0c3da640 1 --2- 172.21.15.83:0/2678531607 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1ae8066f90 0x7f1ae8069450 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:48.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.651+0000 7f1b0c3da640 1 --2- 172.21.15.83:0/2678531607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b0415ba30 0x7f1b04158370 unknown :-1 s=CLOSED pgs=2463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:48.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.651+0000 7f1b0c3da640 1 --2- 172.21.15.83:0/2678531607 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b040cfa10 0x7f1b04157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:48.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.651+0000 7f1b0c3da640 1 --2- 172.21.15.83:0/2678531607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b04072330 0x7f1b04141240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:48.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.651+0000 7f1b0c3da640 1 -- 172.21.15.83:0/2678531607 >> 172.21.15.83:0/2678531607 conn(0x7f1b0406abe0 msgr2=0x7f1b040ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:48.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.655+0000 7f1b0c3da640 1 -- 172.21.15.83:0/2678531607 shutdown_connections 2024-04-17T10:54:48.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:48.655+0000 7f1b0c3da640 1 -- 172.21.15.83:0/2678531607 wait complete. 2024-04-17T10:54:48.677 INFO:tasks.cephfs_test_runner:test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ERROR 2024-04-17T10:54:48.677 INFO:tasks.cephfs_test_runner: 2024-04-17T10:54:48.678 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-17T10:54:48.678 INFO:tasks.cephfs_test_runner:ERROR: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-17T10:54:48.678 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-17T10:54:48.678 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-17T10:54:48.678 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/qa/tasks/cephfs/test_scrub_checks.py", line 190, in test_scrub_when_mds_is_inactive 2024-04-17T10:54:48.678 INFO:tasks.cephfs_test_runner: status = self.fs.wait_for_daemons() 2024-04-17T10:54:48.678 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/qa/tasks/cephfs/filesystem.py", line 1240, in wait_for_daemons 2024-04-17T10:54:48.678 INFO:tasks.cephfs_test_runner: raise RuntimeError("Timed out waiting for MDS daemons to become healthy") 2024-04-17T10:54:48.678 INFO:tasks.cephfs_test_runner:RuntimeError: Timed out waiting for MDS daemons to become healthy 2024-04-17T10:54:48.678 INFO:tasks.cephfs_test_runner: 2024-04-17T10:54:48.678 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-17T10:54:48.678 INFO:tasks.cephfs_test_runner:Ran 9 tests in 964.186s 2024-04-17T10:54:48.678 INFO:tasks.cephfs_test_runner: 2024-04-17T10:54:48.678 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-04-17T10:54:48.678 INFO:tasks.cephfs_test_runner: 2024-04-17T10:54:48.679 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-17T10:54:48.679 INFO:tasks.cephfs_test_runner:ERROR: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-17T10:54:48.679 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-17T10:54:48.679 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-17T10:54:48.679 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/qa/tasks/cephfs/test_scrub_checks.py", line 190, in test_scrub_when_mds_is_inactive 2024-04-17T10:54:48.679 INFO:tasks.cephfs_test_runner: status = self.fs.wait_for_daemons() 2024-04-17T10:54:48.679 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/qa/tasks/cephfs/filesystem.py", line 1240, in wait_for_daemons 2024-04-17T10:54:48.679 INFO:tasks.cephfs_test_runner: raise RuntimeError("Timed out waiting for MDS daemons to become healthy") 2024-04-17T10:54:48.679 INFO:tasks.cephfs_test_runner:RuntimeError: Timed out waiting for MDS daemons to become healthy 2024-04-17T10:54:48.679 INFO:tasks.cephfs_test_runner: 2024-04-17T10:54:48.680 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-17T10:54:48.890 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=b50cc263a42145e6848c416d20b4d678 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-17T10:54:48.892 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-17T10:54:48.901 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-17T10:54:48.940 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-17T10:54:48.940 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:54:48.940 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:54:48.955 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16063 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:54:48.955 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-17T10:54:48.955 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:54:49.004 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336504k,nr_inodes=4084126,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16063 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:54:49.004 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:54:49.005 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:54:49.011 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:54:49.011 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-17T10:54:49.011 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:54:49.060 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336508k,nr_inodes=4084127,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21900 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:54:49.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:54:49.060 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-17T10:54:49.066 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-17T10:54:49.067 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T10:54:49.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:54:49.067 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:54:49.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:54:49.125 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:54:49.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:54:49.199 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2024-04-17T10:54:49.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:54:49.377 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-04-17T10:54:49.384 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-17T10:54:49.384 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-17T10:54:49.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:54:49.384 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-17T10:54:49.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:54:49.441 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-17T10:54:49.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:54:49.496 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link delete ceph-brx 2024-04-17T10:54:49.665 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-17T10:54:49.673 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-17T10:54:49.674 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:54:49.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.807+0000 7f1deb326640 1 Processor -- start 2024-04-17T10:54:49.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.807+0000 7f1deb326640 1 -- start start 2024-04-17T10:54:49.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1deb326640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1de415db50 0x7f1de4070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:49.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1deb326640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1de406dcb0 0x7f1de40b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:49.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1deb326640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1de40709b0 con 0x7f1de40aedd0 2024-04-17T10:54:49.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1deb326640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1de415e440 con 0x7f1de406dcb0 2024-04-17T10:54:49.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1deb326640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1de415e720 con 0x7f1de415db50 2024-04-17T10:54:49.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1de88c1640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1de40aedd0 0x7f1de406d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42260/0 (socket says 172.21.15.83:42260) 2024-04-17T10:54:49.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1de88c1640 1 -- 172.21.15.83:0/2357603643 learned_addr learned my addr 172.21.15.83:0/2357603643 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:49.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1ddbfff640 1 -- 172.21.15.83:0/2357603643 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1326698201 0 0) 0x7f1de0001120 con 0x7f1de40aedd0 2024-04-17T10:54:49.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1de90c2640 1 --2- 172.21.15.83:0/2357603643 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1de415db50 0x7f1de4070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:49.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1ddbfff640 1 -- 172.21.15.83:0/2357603643 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dd0003530 con 0x7f1de40aedd0 2024-04-17T10:54:49.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1de98c3640 1 --2- 172.21.15.83:0/2357603643 >> v2:172.21.15.167:3300/0 conn(0x7f1de406dcb0 0x7f1de40b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:49.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1ddbfff640 1 -- 172.21.15.83:0/2357603643 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 234649644 0 0) 0x7f1dd0003530 con 0x7f1de40aedd0 2024-04-17T10:54:49.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1ddbfff640 1 -- 172.21.15.83:0/2357603643 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1de415db50 msgr2=0x7f1de4070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:49.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1ddbfff640 1 --2- 172.21.15.83:0/2357603643 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1de415db50 0x7f1de4070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:49.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1ddbfff640 1 -- 172.21.15.83:0/2357603643 >> v2:172.21.15.167:3300/0 conn(0x7f1de406dcb0 msgr2=0x7f1de40b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:49.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1ddbfff640 1 --2- 172.21.15.83:0/2357603643 >> v2:172.21.15.167:3300/0 conn(0x7f1de406dcb0 0x7f1de40b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:49.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1ddbfff640 1 -- 172.21.15.83:0/2357603643 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de415ea00 con 0x7f1de40aedd0 2024-04-17T10:54:49.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1ddbfff640 1 -- 172.21.15.83:0/2357603643 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f1de0001850 con 0x7f1de40aedd0 2024-04-17T10:54:49.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1de90c2640 1 --2- 172.21.15.83:0/2357603643 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1de415db50 0x7f1de4070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:54:49.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1ddbfff640 1 -- 172.21.15.83:0/2357603643 >> v1:172.21.15.83:6789/0 conn(0x7f1de40aedd0 legacy=0x7f1de406d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:49.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1ddbfff640 1 --2- 172.21.15.83:0/2357603643 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1de415db50 0x7f1dd00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:49.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1ddbfff640 1 --2- 172.21.15.83:0/2357603643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dd0003f00 0x7f1dd0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:49.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1ddbfff640 1 --2- 172.21.15.83:0/2357603643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd0004d50 0x7f1dd00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:49.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1ddbfff640 1 -- 172.21.15.83:0/2357603643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de415e440 con 0x7f1dd0004d50 2024-04-17T10:54:49.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1ddbfff640 1 -- 172.21.15.83:0/2357603643 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de415e720 con 0x7f1dd0003f00 2024-04-17T10:54:49.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1ddbfff640 1 -- 172.21.15.83:0/2357603643 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd000ba00 con 0x7f1de415db50 2024-04-17T10:54:49.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1de98c3640 1 --2- 172.21.15.83:0/2357603643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd0004d50 0x7f1dd00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:49.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1de88c1640 1 --2- 172.21.15.83:0/2357603643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dd0003f00 0x7f1dd0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:49.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1de90c2640 1 --2- 172.21.15.83:0/2357603643 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1de415db50 0x7f1dd00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:49.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1de98c3640 1 -- 172.21.15.83:0/2357603643 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1de415db50 msgr2=0x7f1dd00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:49.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1de98c3640 1 --2- 172.21.15.83:0/2357603643 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1de415db50 0x7f1dd00039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:49.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1de98c3640 1 -- 172.21.15.83:0/2357603643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dd0003f00 msgr2=0x7f1dd0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:49.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1de98c3640 1 --2- 172.21.15.83:0/2357603643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dd0003f00 0x7f1dd0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:49.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.811+0000 7f1de98c3640 1 -- 172.21.15.83:0/2357603643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd0003530 con 0x7f1dd0004d50 2024-04-17T10:54:49.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.815+0000 7f1de98c3640 1 --2- 172.21.15.83:0/2357603643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd0004d50 0x7f1dd00051d0 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc009fc0 tx=0x7f1ddc01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50184a65151888f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:49.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.815+0000 7f1ddbfff640 1 -- 172.21.15.83:0/2357603643 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc0b2070 con 0x7f1dd0004d50 2024-04-17T10:54:49.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.815+0000 7f1ddbfff640 1 -- 172.21.15.83:0/2357603643 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ddc0a5cd0 con 0x7f1dd0004d50 2024-04-17T10:54:49.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.815+0000 7f1ddbfff640 1 -- 172.21.15.83:0/2357603643 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc009c10 con 0x7f1dd0004d50 2024-04-17T10:54:49.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.815+0000 7f1deb326640 1 -- 172.21.15.83:0/2357603643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd0004d50 msgr2=0x7f1dd00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:49.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.815+0000 7f1deb326640 1 --2- 172.21.15.83:0/2357603643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd0004d50 0x7f1dd00051d0 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc009fc0 tx=0x7f1ddc01ccc0 comp rx=0 tx=0).stop 2024-04-17T10:54:49.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.815+0000 7f1de98c3640 1 -- 172.21.15.83:0/2357603643 reap_dead start 2024-04-17T10:54:49.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.815+0000 7f1deb326640 1 -- 172.21.15.83:0/2357603643 shutdown_connections 2024-04-17T10:54:49.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.815+0000 7f1deb326640 1 -- 172.21.15.83:0/2357603643 >> 172.21.15.83:0/2357603643 conn(0x7f1de406abe0 msgr2=0x7f1de406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:49.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.815+0000 7f1deb326640 1 -- 172.21.15.83:0/2357603643 shutdown_connections 2024-04-17T10:54:49.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.815+0000 7f1deb326640 1 -- 172.21.15.83:0/2357603643 wait complete. 2024-04-17T10:54:49.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.815+0000 7f1deb326640 1 Processor -- start 2024-04-17T10:54:49.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.815+0000 7f1deb326640 1 -- start start 2024-04-17T10:54:49.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.819+0000 7f1deb326640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dd0004d50 0x7f1de4142910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:49.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.819+0000 7f1deb326640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1de4144ff0 0x7f1de4159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:49.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.819+0000 7f1deb326640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1de415c400 0x7f1de415c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:49.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.819+0000 7f1deb326640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de415df30 con 0x7f1de4144ff0 2024-04-17T10:54:49.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.819+0000 7f1deb326640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de415efc0 con 0x7f1dd0004d50 2024-04-17T10:54:49.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.819+0000 7f1deb326640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1de415e0b0 con 0x7f1de415c400 2024-04-17T10:54:49.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.819+0000 7f1de88c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1de4144ff0 0x7f1de4159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:49.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.819+0000 7f1de88c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1de4144ff0 0x7f1de4159a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44296/0 (socket says 172.21.15.83:44296) 2024-04-17T10:54:49.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.819+0000 7f1de88c1640 1 -- 172.21.15.83:0/3171232339 learned_addr learned my addr 172.21.15.83:0/3171232339 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:49.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.819+0000 7f1de90c2640 1 --2- 172.21.15.83:0/3171232339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dd0004d50 0x7f1de4142910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:49.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.819+0000 7f1de98c3640 1 --2- 172.21.15.83:0/3171232339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1de415c400 0x7f1de415c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:49.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.819+0000 7f1de88c1640 1 -- 172.21.15.83:0/3171232339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1de415c400 msgr2=0x7f1de415c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:49.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.819+0000 7f1de88c1640 1 --2- 172.21.15.83:0/3171232339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1de415c400 0x7f1de415c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:49.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.819+0000 7f1de88c1640 1 -- 172.21.15.83:0/3171232339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dd0004d50 msgr2=0x7f1de4142910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:49.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.819+0000 7f1de88c1640 1 --2- 172.21.15.83:0/3171232339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dd0004d50 0x7f1de4142910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:49.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.819+0000 7f1de88c1640 1 -- 172.21.15.83:0/3171232339 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de4159f50 con 0x7f1de4144ff0 2024-04-17T10:54:49.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.819+0000 7f1de88c1640 1 --2- 172.21.15.83:0/3171232339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1de4144ff0 0x7f1de4159a10 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f1de000e8c0 tx=0x7f1de000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:49.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.819+0000 7f1dd9ffb640 1 -- 172.21.15.83:0/3171232339 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de000c870 con 0x7f1de4144ff0 2024-04-17T10:54:49.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.819+0000 7f1dd9ffb640 1 -- 172.21.15.83:0/3171232339 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de0018070 con 0x7f1de4144ff0 2024-04-17T10:54:49.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.819+0000 7f1deb326640 1 -- 172.21.15.83:0/3171232339 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1de41454f0 con 0x7f1de4144ff0 2024-04-17T10:54:49.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.819+0000 7f1dd9ffb640 1 -- 172.21.15.83:0/3171232339 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de000cc90 con 0x7f1de4144ff0 2024-04-17T10:54:49.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.819+0000 7f1deb326640 1 -- 172.21.15.83:0/3171232339 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1de4145ab0 con 0x7f1de4144ff0 2024-04-17T10:54:49.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.819+0000 7f1deb326640 1 -- 172.21.15.83:0/3171232339 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1de4074f90 con 0x7f1de4144ff0 2024-04-17T10:54:49.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.819+0000 7f1dd9ffb640 1 -- 172.21.15.83:0/3171232339 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1de001d460 con 0x7f1de4144ff0 2024-04-17T10:54:49.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.819+0000 7f1dd9ffb640 1 --2- 172.21.15.83:0/3171232339 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1dc8066f60 0x7f1dc8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:49.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.819+0000 7f1de90c2640 1 --2- 172.21.15.83:0/3171232339 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1dc8066f60 0x7f1dc8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:49.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.819+0000 7f1dd9ffb640 1 -- 172.21.15.83:0/3171232339 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1de0084ca0 con 0x7f1de4144ff0 2024-04-17T10:54:49.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.819+0000 7f1de90c2640 1 --2- 172.21.15.83:0/3171232339 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1dc8066f60 0x7f1dc8069420 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4006aa0 tx=0x7f1dd4006a30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:49.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.823+0000 7f1dd9ffb640 1 -- 172.21.15.83:0/3171232339 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1de40709b0 con 0x7f1de4144ff0 2024-04-17T10:54:49.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.983+0000 7f1deb326640 1 -- 172.21.15.83:0/3171232339 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1de415a180 con 0x7f1dc8066f60 2024-04-17T10:54:49.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.983+0000 7f1dd9ffb640 1 -- 172.21.15.83:0/3171232339 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13494 (secure 0 0 0) 0x7f1de415a180 con 0x7f1dc8066f60 2024-04-17T10:54:49.977 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:49.977 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:54:49.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.983+0000 7f1deb326640 1 -- 172.21.15.83:0/3171232339 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1dc8066f60 msgr2=0x7f1dc8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:49.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.983+0000 7f1deb326640 1 --2- 172.21.15.83:0/3171232339 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1dc8066f60 0x7f1dc8069420 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4006aa0 tx=0x7f1dd4006a30 comp rx=0 tx=0).stop 2024-04-17T10:54:49.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.983+0000 7f1deb326640 1 -- 172.21.15.83:0/3171232339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1de4144ff0 msgr2=0x7f1de4159a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:49.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.983+0000 7f1deb326640 1 --2- 172.21.15.83:0/3171232339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1de4144ff0 0x7f1de4159a10 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f1de000e8c0 tx=0x7f1de000ed80 comp rx=0 tx=0).stop 2024-04-17T10:54:49.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.983+0000 7f1deb326640 1 -- 172.21.15.83:0/3171232339 shutdown_connections 2024-04-17T10:54:49.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.983+0000 7f1deb326640 1 --2- 172.21.15.83:0/3171232339 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1dc8066f60 0x7f1dc8069420 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:49.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.983+0000 7f1deb326640 1 --2- 172.21.15.83:0/3171232339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1de415c400 0x7f1de415c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:49.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.983+0000 7f1deb326640 1 --2- 172.21.15.83:0/3171232339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1de4144ff0 0x7f1de4159a10 unknown :-1 s=CLOSED pgs=2466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:49.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.983+0000 7f1deb326640 1 --2- 172.21.15.83:0/3171232339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dd0004d50 0x7f1de4142910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:49.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.983+0000 7f1deb326640 1 -- 172.21.15.83:0/3171232339 >> 172.21.15.83:0/3171232339 conn(0x7f1de406abe0 msgr2=0x7f1de41572d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:49.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.983+0000 7f1deb326640 1 -- 172.21.15.83:0/3171232339 shutdown_connections 2024-04-17T10:54:49.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:49.983+0000 7f1deb326640 1 -- 172.21.15.83:0/3171232339 wait complete. 2024-04-17T10:54:49.997 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":643,"stamp":"2024-04-17T10:54:48.485871+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2614208,"kb_used_data":2117664,"kb_used_omap":12,"kb_used_meta":496435,"kb_avail":752360512,"statfs":{"total":773094113280,"available":770417164288,"internally_reserved":0,"allocated":2168487936,"data_stored":2160394546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":508350000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1076459209,"num_objects":-1046,"num_object_clones":0,"num_object_copies":-2092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-1128,"num_write_kb":-1051360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-11,"num_bytes_recovered":-11534336,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002909"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":424,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-17T10:52:04.530478+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:52:04.530478+0000","last_peered":"2024-04-17T10:52:04.530478+0000","last_clean":"2024-04-17T10:52:04.530478+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:52:04.530478+0000","last_undegraded":"2024-04-17T10:52:04.530478+0000","last_fullsized":"2024-04-17T10:52:04.530478+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575053,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":357740,"kb_used_data":308512,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94014100,"statfs":{"total":96636764160,"available":96270438400,"internally_reserved":0,"allocated":315916288,"data_stored":314902168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575052,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":331244,"kb_used_data":259808,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94040596,"statfs":{"total":96636764160,"available":96297570304,"internally_reserved":0,"allocated":266043392,"data_stored":265028222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575053,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311204,"kb_used_data":253656,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94060636,"statfs":{"total":96636764160,"available":96318091264,"internally_reserved":0,"allocated":259743744,"data_stored":258736642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575052,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395744,"kb_used_data":324308,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":93976096,"statfs":{"total":96636764160,"available":96231522304,"internally_reserved":0,"allocated":332091392,"data_stored":331088364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575054,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306660,"kb_used_data":235224,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94065180,"statfs":{"total":96636764160,"available":96322744320,"internally_reserved":0,"allocated":240869376,"data_stored":239862342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575052,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299340,"kb_used_data":254272,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94072500,"statfs":{"total":96636764160,"available":96330240000,"internally_reserved":0,"allocated":260374528,"data_stored":259356125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575052,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":344036,"kb_used_data":262872,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94027804,"statfs":{"total":96636764160,"available":96284471296,"internally_reserved":0,"allocated":269180928,"data_stored":268174386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575052,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":268240,"kb_used_data":219012,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94103600,"statfs":{"total":96636764160,"available":96362086400,"internally_reserved":0,"allocated":224268288,"data_stored":223246297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:54:49.998 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:54:50.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.139+0000 7f336282f640 1 Processor -- start 2024-04-17T10:54:50.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.139+0000 7f336282f640 1 -- start start 2024-04-17T10:54:50.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.139+0000 7f336282f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f335c0ceca0 0x7f335c0ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:50.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.139+0000 7f336282f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f335c0ce620 0x7f335c0cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:50.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.139+0000 7f336282f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f335c15f6c0 con 0x7f335c15de90 2024-04-17T10:54:50.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.139+0000 7f336282f640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f335c06bd90 con 0x7f335c0ce620 2024-04-17T10:54:50.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.139+0000 7f336282f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f335c06c030 con 0x7f335c0ceca0 2024-04-17T10:54:50.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.139+0000 7f3360dcc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f335c0ce620 0x7f335c0cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:50.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.139+0000 7f3360dcc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f335c0ce620 0x7f335c0cf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44306/0 (socket says 172.21.15.83:44306) 2024-04-17T10:54:50.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.139+0000 7f335bfff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f335c15de90 0x7f335c15efb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42268/0 (socket says 172.21.15.83:42268) 2024-04-17T10:54:50.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.139+0000 7f3360dcc640 1 -- 172.21.15.83:0/3665655745 learned_addr learned my addr 172.21.15.83:0/3665655745 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:50.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.139+0000 7f335b7fe640 1 --2- 172.21.15.83:0/3665655745 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f335c0ceca0 0x7f335c0ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:50.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.139+0000 7f335affd640 1 -- 172.21.15.83:0/3665655745 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3970554381 0 0) 0x7f334c001120 con 0x7f335c15de90 2024-04-17T10:54:50.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.139+0000 7f335affd640 1 -- 172.21.15.83:0/3665655745 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3344003530 con 0x7f335c15de90 2024-04-17T10:54:50.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.139+0000 7f3360dcc640 1 -- 172.21.15.83:0/3665655745 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f335c0ceca0 msgr2=0x7f335c0ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:50.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.139+0000 7f3360dcc640 1 --2- 172.21.15.83:0/3665655745 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f335c0ceca0 0x7f335c0ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:50.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.139+0000 7f3360dcc640 1 -- 172.21.15.83:0/3665655745 >> v1:172.21.15.83:6789/0 conn(0x7f335c15de90 legacy=0x7f335c15efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:50.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.139+0000 7f3360dcc640 1 -- 172.21.15.83:0/3665655745 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f335c06c310 con 0x7f335c0ce620 2024-04-17T10:54:50.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.139+0000 7f335b7fe640 1 --2- 172.21.15.83:0/3665655745 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f335c0ceca0 0x7f335c0ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:54:50.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f3360dcc640 1 --2- 172.21.15.83:0/3665655745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f335c0ce620 0x7f335c0cf8d0 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f3354009e80 tx=0x7f335401ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3910c862158f61d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:50.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f335affd640 1 -- 172.21.15.83:0/3665655745 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33540a57d0 con 0x7f335c0ce620 2024-04-17T10:54:50.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f335affd640 1 -- 172.21.15.83:0/3665655745 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33540a5970 con 0x7f335c0ce620 2024-04-17T10:54:50.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f335affd640 1 -- 172.21.15.83:0/3665655745 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33540a5d20 con 0x7f335c0ce620 2024-04-17T10:54:50.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f336282f640 1 -- 172.21.15.83:0/3665655745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f335c0ce620 msgr2=0x7f335c0cf8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:50.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f336282f640 1 --2- 172.21.15.83:0/3665655745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f335c0ce620 0x7f335c0cf8d0 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f3354009e80 tx=0x7f335401ca10 comp rx=0 tx=0).stop 2024-04-17T10:54:50.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f336282f640 1 -- 172.21.15.83:0/3665655745 shutdown_connections 2024-04-17T10:54:50.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f336282f640 1 --2- 172.21.15.83:0/3665655745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f335c0ce620 0x7f335c0cf8d0 unknown :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:50.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f336282f640 1 --2- 172.21.15.83:0/3665655745 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f335c0ceca0 0x7f335c0ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:50.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f336282f640 1 -- 172.21.15.83:0/3665655745 >> 172.21.15.83:0/3665655745 conn(0x7f335c0d17b0 msgr2=0x7f335c0d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:50.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f336282f640 1 -- 172.21.15.83:0/3665655745 shutdown_connections 2024-04-17T10:54:50.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f336282f640 1 -- 172.21.15.83:0/3665655745 wait complete. 2024-04-17T10:54:50.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f336282f640 1 Processor -- start 2024-04-17T10:54:50.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f336282f640 1 -- start start 2024-04-17T10:54:50.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f336282f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f335c0ce620 0x7f335c152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:50.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f336282f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f335c0ceca0 0x7f335c162120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:50.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f336282f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f335c15de90 0x7f335c06e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:50.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f336282f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335c15e3f0 con 0x7f335c15de90 2024-04-17T10:54:50.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f336282f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335c15e270 con 0x7f335c0ceca0 2024-04-17T10:54:50.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f336282f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f335c0d0620 con 0x7f335c0ce620 2024-04-17T10:54:50.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f3360dcc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f335c15de90 0x7f335c06e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:50.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f3360dcc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f335c15de90 0x7f335c06e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44308/0 (socket says 172.21.15.83:44308) 2024-04-17T10:54:50.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f3360dcc640 1 -- 172.21.15.83:0/2650807154 learned_addr learned my addr 172.21.15.83:0/2650807154 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:50.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f335bfff640 1 --2- 172.21.15.83:0/2650807154 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f335c0ce620 0x7f335c152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:50.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f335b7fe640 1 --2- 172.21.15.83:0/2650807154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f335c0ceca0 0x7f335c162120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:50.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f3360dcc640 1 -- 172.21.15.83:0/2650807154 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f335c0ce620 msgr2=0x7f335c152bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:50.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f3360dcc640 1 --2- 172.21.15.83:0/2650807154 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f335c0ce620 0x7f335c152bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:50.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f3360dcc640 1 -- 172.21.15.83:0/2650807154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f335c0ceca0 msgr2=0x7f335c162120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:50.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f3360dcc640 1 --2- 172.21.15.83:0/2650807154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f335c0ceca0 0x7f335c162120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:50.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f3360dcc640 1 -- 172.21.15.83:0/2650807154 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f335c153100 con 0x7f335c15de90 2024-04-17T10:54:50.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f335bfff640 1 --2- 172.21.15.83:0/2650807154 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f335c0ce620 0x7f335c152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:50.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f335b7fe640 1 --2- 172.21.15.83:0/2650807154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f335c0ceca0 0x7f335c162120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:50.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.143+0000 7f3360dcc640 1 --2- 172.21.15.83:0/2650807154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f335c15de90 0x7f335c06e500 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f33540af040 tx=0x7f33540197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:50.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.147+0000 7f3358ff9640 1 -- 172.21.15.83:0/2650807154 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33540b8670 con 0x7f335c15de90 2024-04-17T10:54:50.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.147+0000 7f3358ff9640 1 -- 172.21.15.83:0/2650807154 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33540be070 con 0x7f335c15de90 2024-04-17T10:54:50.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.147+0000 7f336282f640 1 -- 172.21.15.83:0/2650807154 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f335c153540 con 0x7f335c15de90 2024-04-17T10:54:50.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.147+0000 7f3358ff9640 1 -- 172.21.15.83:0/2650807154 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33540b8a70 con 0x7f335c15de90 2024-04-17T10:54:50.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.147+0000 7f336282f640 1 -- 172.21.15.83:0/2650807154 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f335c153b00 con 0x7f335c15de90 2024-04-17T10:54:50.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.147+0000 7f336282f640 1 -- 172.21.15.83:0/2650807154 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f335c074f90 con 0x7f335c15de90 2024-04-17T10:54:50.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.147+0000 7f3358ff9640 1 -- 172.21.15.83:0/2650807154 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f33540b8c10 con 0x7f335c15de90 2024-04-17T10:54:50.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.147+0000 7f3358ff9640 1 --2- 172.21.15.83:0/2650807154 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3348066f60 0x7f3348069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:50.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.147+0000 7f3358ff9640 1 -- 172.21.15.83:0/2650807154 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f335411fda0 con 0x7f335c15de90 2024-04-17T10:54:50.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.147+0000 7f335bfff640 1 --2- 172.21.15.83:0/2650807154 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3348066f60 0x7f3348069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:50.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.147+0000 7f335bfff640 1 --2- 172.21.15.83:0/2650807154 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3348066f60 0x7f3348069420 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f334c000f70 tx=0x7f334c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:50.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.151+0000 7f3358ff9640 1 -- 172.21.15.83:0/2650807154 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3354016600 con 0x7f335c15de90 2024-04-17T10:54:50.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.307+0000 7f336282f640 1 -- 172.21.15.83:0/2650807154 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f335c1532f0 con 0x7f3348066f60 2024-04-17T10:54:50.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.307+0000 7f3358ff9640 1 -- 172.21.15.83:0/2650807154 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13494 (secure 0 0 0) 0x7f335c1532f0 con 0x7f3348066f60 2024-04-17T10:54:50.302 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:50.303 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:54:50.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.311+0000 7f336282f640 1 -- 172.21.15.83:0/2650807154 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3348066f60 msgr2=0x7f3348069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:50.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.311+0000 7f336282f640 1 --2- 172.21.15.83:0/2650807154 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3348066f60 0x7f3348069420 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f334c000f70 tx=0x7f334c009040 comp rx=0 tx=0).stop 2024-04-17T10:54:50.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.311+0000 7f336282f640 1 -- 172.21.15.83:0/2650807154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f335c15de90 msgr2=0x7f335c06e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:50.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.311+0000 7f336282f640 1 --2- 172.21.15.83:0/2650807154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f335c15de90 0x7f335c06e500 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f33540af040 tx=0x7f33540197b0 comp rx=0 tx=0).stop 2024-04-17T10:54:50.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.311+0000 7f336282f640 1 -- 172.21.15.83:0/2650807154 shutdown_connections 2024-04-17T10:54:50.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.311+0000 7f336282f640 1 --2- 172.21.15.83:0/2650807154 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3348066f60 0x7f3348069420 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:50.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.311+0000 7f336282f640 1 --2- 172.21.15.83:0/2650807154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f335c15de90 0x7f335c06e500 unknown :-1 s=CLOSED pgs=2469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:50.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.311+0000 7f336282f640 1 --2- 172.21.15.83:0/2650807154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f335c0ceca0 0x7f335c162120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:50.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.311+0000 7f336282f640 1 --2- 172.21.15.83:0/2650807154 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f335c0ce620 0x7f335c152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:50.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.311+0000 7f336282f640 1 -- 172.21.15.83:0/2650807154 >> 172.21.15.83:0/2650807154 conn(0x7f335c0d17b0 msgr2=0x7f335c0cd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:50.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.311+0000 7f336282f640 1 -- 172.21.15.83:0/2650807154 shutdown_connections 2024-04-17T10:54:50.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.311+0000 7f336282f640 1 -- 172.21.15.83:0/2650807154 wait complete. 2024-04-17T10:54:50.321 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":643,"stamp":"2024-04-17T10:54:48.485871+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2614208,"kb_used_data":2117664,"kb_used_omap":12,"kb_used_meta":496435,"kb_avail":752360512,"statfs":{"total":773094113280,"available":770417164288,"internally_reserved":0,"allocated":2168487936,"data_stored":2160394546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":508350000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1076459209,"num_objects":-1046,"num_object_clones":0,"num_object_copies":-2092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-1128,"num_write_kb":-1051360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-11,"num_bytes_recovered":-11534336,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002909"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":424,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-17T10:52:04.530478+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:52:04.530478+0000","last_peered":"2024-04-17T10:52:04.530478+0000","last_clean":"2024-04-17T10:52:04.530478+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:52:04.530478+0000","last_undegraded":"2024-04-17T10:52:04.530478+0000","last_fullsized":"2024-04-17T10:52:04.530478+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575053,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":357740,"kb_used_data":308512,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94014100,"statfs":{"total":96636764160,"available":96270438400,"internally_reserved":0,"allocated":315916288,"data_stored":314902168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575052,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":331244,"kb_used_data":259808,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94040596,"statfs":{"total":96636764160,"available":96297570304,"internally_reserved":0,"allocated":266043392,"data_stored":265028222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575053,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311204,"kb_used_data":253656,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94060636,"statfs":{"total":96636764160,"available":96318091264,"internally_reserved":0,"allocated":259743744,"data_stored":258736642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575052,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395744,"kb_used_data":324308,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":93976096,"statfs":{"total":96636764160,"available":96231522304,"internally_reserved":0,"allocated":332091392,"data_stored":331088364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575054,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306660,"kb_used_data":235224,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94065180,"statfs":{"total":96636764160,"available":96322744320,"internally_reserved":0,"allocated":240869376,"data_stored":239862342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575052,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299340,"kb_used_data":254272,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94072500,"statfs":{"total":96636764160,"available":96330240000,"internally_reserved":0,"allocated":260374528,"data_stored":259356125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575052,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":344036,"kb_used_data":262872,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94027804,"statfs":{"total":96636764160,"available":96284471296,"internally_reserved":0,"allocated":269180928,"data_stored":268174386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575052,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":268240,"kb_used_data":219012,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94103600,"statfs":{"total":96636764160,"available":96362086400,"internally_reserved":0,"allocated":224268288,"data_stored":223246297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:54:50.322 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-17T10:54:50.322 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:54:50.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.451+0000 7fb795964640 1 Processor -- start 2024-04-17T10:54:50.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.451+0000 7fb795964640 1 -- start start 2024-04-17T10:54:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.451+0000 7fb795964640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb79015ba30 0x7fb790072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:50.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.451+0000 7fb795964640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb79015c960 0x7fb7900cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:50.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.451+0000 7fb795964640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7900cfea0 con 0x7fb7900cfa10 2024-04-17T10:54:50.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.451+0000 7fb795964640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb790071ea0 con 0x7fb79015c960 2024-04-17T10:54:50.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.451+0000 7fb795964640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb79006e320 con 0x7fb79015ba30 2024-04-17T10:54:50.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.451+0000 7fb78f7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb7900cfa10 0x7fb79006c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42274/0 (socket says 172.21.15.83:42274) 2024-04-17T10:54:50.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.451+0000 7fb78f7fe640 1 -- 172.21.15.83:0/233525959 learned_addr learned my addr 172.21.15.83:0/233525959 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:50.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.451+0000 7fb78e7fc640 1 --2- 172.21.15.83:0/233525959 >> v2:172.21.15.83:3300/0 conn(0x7fb79015c960 0x7fb7900cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:50.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.451+0000 7fb78dffb640 1 -- 172.21.15.83:0/233525959 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1628357087 0 0) 0x7fb7900cfea0 con 0x7fb7900cfa10 2024-04-17T10:54:50.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.451+0000 7fb78effd640 1 --2- 172.21.15.83:0/233525959 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb79015ba30 0x7fb790072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:50.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.451+0000 7fb78dffb640 1 -- 172.21.15.83:0/233525959 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb778003530 con 0x7fb7900cfa10 2024-04-17T10:54:50.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.451+0000 7fb78e7fc640 1 -- 172.21.15.83:0/233525959 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb79015ba30 msgr2=0x7fb790072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:50.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.451+0000 7fb78e7fc640 1 --2- 172.21.15.83:0/233525959 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb79015ba30 0x7fb790072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:50.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.451+0000 7fb78e7fc640 1 -- 172.21.15.83:0/233525959 >> v1:172.21.15.83:6789/0 conn(0x7fb7900cfa10 legacy=0x7fb79006c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:50.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.451+0000 7fb78e7fc640 1 -- 172.21.15.83:0/233525959 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb79006e040 con 0x7fb79015c960 2024-04-17T10:54:50.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.455+0000 7fb78e7fc640 1 --2- 172.21.15.83:0/233525959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb79015c960 0x7fb7900cf4d0 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7fb7840097c0 tx=0x7fb78401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef111ef6ddf7e5e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:50.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.455+0000 7fb78dffb640 1 -- 172.21.15.83:0/233525959 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7840b0070 con 0x7fb79015c960 2024-04-17T10:54:50.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.455+0000 7fb78dffb640 1 -- 172.21.15.83:0/233525959 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7840ab600 con 0x7fb79015c960 2024-04-17T10:54:50.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.455+0000 7fb78dffb640 1 -- 172.21.15.83:0/233525959 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7840ab9d0 con 0x7fb79015c960 2024-04-17T10:54:50.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.455+0000 7fb795964640 1 -- 172.21.15.83:0/233525959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb79015c960 msgr2=0x7fb7900cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:50.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.455+0000 7fb795964640 1 --2- 172.21.15.83:0/233525959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb79015c960 0x7fb7900cf4d0 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7fb7840097c0 tx=0x7fb78401c410 comp rx=0 tx=0).stop 2024-04-17T10:54:50.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.455+0000 7fb795964640 1 -- 172.21.15.83:0/233525959 shutdown_connections 2024-04-17T10:54:50.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.455+0000 7fb795964640 1 --2- 172.21.15.83:0/233525959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb79015c960 0x7fb7900cf4d0 unknown :-1 s=CLOSED pgs=2471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:50.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.455+0000 7fb795964640 1 --2- 172.21.15.83:0/233525959 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb79015ba30 0x7fb790072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:50.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.455+0000 7fb795964640 1 -- 172.21.15.83:0/233525959 >> 172.21.15.83:0/233525959 conn(0x7fb79006abe0 msgr2=0x7fb79006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:50.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.455+0000 7fb795964640 1 -- 172.21.15.83:0/233525959 shutdown_connections 2024-04-17T10:54:50.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.455+0000 7fb795964640 1 -- 172.21.15.83:0/233525959 wait complete. 2024-04-17T10:54:50.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.455+0000 7fb795964640 1 Processor -- start 2024-04-17T10:54:50.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.459+0000 7fb795964640 1 -- start start 2024-04-17T10:54:50.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.459+0000 7fb795964640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7900cfa10 0x7fb790148420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:50.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.459+0000 7fb795964640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb79015ba30 0x7fb79014f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:50.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.459+0000 7fb795964640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb79015c960 0x7fb790151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:50.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.459+0000 7fb795964640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79006dca0 con 0x7fb7900cfa10 2024-04-17T10:54:50.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.459+0000 7fb795964640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7900aeae0 con 0x7fb79015ba30 2024-04-17T10:54:50.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.459+0000 7fb795964640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb79006de20 con 0x7fb79015c960 2024-04-17T10:54:50.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.459+0000 7fb78effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7900cfa10 0x7fb790148420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:50.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.459+0000 7fb78effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7900cfa10 0x7fb790148420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44336/0 (socket says 172.21.15.83:44336) 2024-04-17T10:54:50.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.459+0000 7fb78effd640 1 -- 172.21.15.83:0/2104294487 learned_addr learned my addr 172.21.15.83:0/2104294487 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:50.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.459+0000 7fb78effd640 1 -- 172.21.15.83:0/2104294487 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb79015c960 msgr2=0x7fb790151340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:50.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.459+0000 7fb78effd640 1 --2- 172.21.15.83:0/2104294487 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb79015c960 0x7fb790151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:50.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.459+0000 7fb78effd640 1 -- 172.21.15.83:0/2104294487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb79015ba30 msgr2=0x7fb79014f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:50.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.459+0000 7fb78effd640 1 --2- 172.21.15.83:0/2104294487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb79015ba30 0x7fb79014f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:50.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.459+0000 7fb78effd640 1 -- 172.21.15.83:0/2104294487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7901558a0 con 0x7fb7900cfa10 2024-04-17T10:54:50.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.459+0000 7fb78effd640 1 --2- 172.21.15.83:0/2104294487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7900cfa10 0x7fb790148420 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7fb78000d800 tx=0x7fb78000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:50.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.459+0000 7fb777fff640 1 -- 172.21.15.83:0/2104294487 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78000b6c0 con 0x7fb7900cfa10 2024-04-17T10:54:50.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.459+0000 7fb777fff640 1 -- 172.21.15.83:0/2104294487 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb780019070 con 0x7fb7900cfa10 2024-04-17T10:54:50.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.459+0000 7fb795964640 1 -- 172.21.15.83:0/2104294487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb790155b80 con 0x7fb7900cfa10 2024-04-17T10:54:50.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.459+0000 7fb777fff640 1 -- 172.21.15.83:0/2104294487 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78000ba60 con 0x7fb7900cfa10 2024-04-17T10:54:50.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.459+0000 7fb795964640 1 -- 172.21.15.83:0/2104294487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7901491a0 con 0x7fb7900cfa10 2024-04-17T10:54:50.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.459+0000 7fb795964640 1 -- 172.21.15.83:0/2104294487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb790074f90 con 0x7fb7900cfa10 2024-04-17T10:54:50.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.459+0000 7fb777fff640 1 -- 172.21.15.83:0/2104294487 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb78000bc00 con 0x7fb7900cfa10 2024-04-17T10:54:50.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.459+0000 7fb777fff640 1 --2- 172.21.15.83:0/2104294487 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb77c066f60 0x7fb77c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:50.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.459+0000 7fb777fff640 1 -- 172.21.15.83:0/2104294487 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb780085140 con 0x7fb7900cfa10 2024-04-17T10:54:50.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.459+0000 7fb78e7fc640 1 --2- 172.21.15.83:0/2104294487 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb77c066f60 0x7fb77c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:50.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.463+0000 7fb78e7fc640 1 --2- 172.21.15.83:0/2104294487 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb77c066f60 0x7fb77c069420 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fb7840ad000 tx=0x7fb7840046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:50.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.463+0000 7fb777fff640 1 -- 172.21.15.83:0/2104294487 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb7800564e0 con 0x7fb7900cfa10 2024-04-17T10:54:50.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.619+0000 7fb795964640 1 -- 172.21.15.83:0/2104294487 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb790074480 con 0x7fb77c066f60 2024-04-17T10:54:50.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.619+0000 7fb777fff640 1 -- 172.21.15.83:0/2104294487 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13564 (secure 0 0 0) 0x7fb790074480 con 0x7fb77c066f60 2024-04-17T10:54:50.615 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:50.615 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:54:50.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.623+0000 7fb795964640 1 -- 172.21.15.83:0/2104294487 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb77c066f60 msgr2=0x7fb77c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:50.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.623+0000 7fb795964640 1 --2- 172.21.15.83:0/2104294487 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb77c066f60 0x7fb77c069420 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fb7840ad000 tx=0x7fb7840046d0 comp rx=0 tx=0).stop 2024-04-17T10:54:50.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.623+0000 7fb795964640 1 -- 172.21.15.83:0/2104294487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7900cfa10 msgr2=0x7fb790148420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:50.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.623+0000 7fb795964640 1 --2- 172.21.15.83:0/2104294487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7900cfa10 0x7fb790148420 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7fb78000d800 tx=0x7fb78000dcc0 comp rx=0 tx=0).stop 2024-04-17T10:54:50.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.623+0000 7fb795964640 1 -- 172.21.15.83:0/2104294487 shutdown_connections 2024-04-17T10:54:50.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.623+0000 7fb795964640 1 --2- 172.21.15.83:0/2104294487 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb77c066f60 0x7fb77c069420 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:50.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.623+0000 7fb795964640 1 --2- 172.21.15.83:0/2104294487 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb79015c960 0x7fb790151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:50.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.623+0000 7fb795964640 1 --2- 172.21.15.83:0/2104294487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb79015ba30 0x7fb79014f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:50.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.623+0000 7fb795964640 1 --2- 172.21.15.83:0/2104294487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7900cfa10 0x7fb790148420 unknown :-1 s=CLOSED pgs=2472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:50.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.623+0000 7fb795964640 1 -- 172.21.15.83:0/2104294487 >> 172.21.15.83:0/2104294487 conn(0x7fb79006abe0 msgr2=0x7fb790155060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:50.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.623+0000 7fb795964640 1 -- 172.21.15.83:0/2104294487 shutdown_connections 2024-04-17T10:54:50.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.623+0000 7fb795964640 1 -- 172.21.15.83:0/2104294487 wait complete. 2024-04-17T10:54:50.635 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":644,"stamp":"2024-04-17T10:54:50.486278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":158,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":1575832,"kb_used_data":1079272,"kb_used_omap":12,"kb_used_meta":496435,"kb_avail":753398888,"statfs":{"total":773094113280,"available":771480461312,"internally_reserved":0,"allocated":1105174528,"data_stored":1097017176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":508350000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1076459209,"num_objects":-1046,"num_object_clones":0,"num_object_copies":-2092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-1128,"num_write_kb":-1051360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-11,"num_bytes_recovered":-11534336,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002483"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":428,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-17T10:54:47.637200+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:54:47.637200+0000","last_peered":"2024-04-17T10:54:47.637200+0000","last_clean":"2024-04-17T10:54:47.637200+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:54:47.637200+0000","last_undegraded":"2024-04-17T10:54:47.637200+0000","last_fullsized":"2024-04-17T10:54:47.637200+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575054,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51592,"kb_used_data":2360,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320248,"statfs":{"total":96636764160,"available":96583933952,"internally_reserved":0,"allocated":2416640,"data_stored":1388906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575053,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":331248,"kb_used_data":259812,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94040592,"statfs":{"total":96636764160,"available":96297566208,"internally_reserved":0,"allocated":266047488,"data_stored":265034108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575054,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59332,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312508,"statfs":{"total":96636764160,"available":96576008192,"internally_reserved":0,"allocated":1822720,"data_stored":798538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575053,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395760,"kb_used_data":324324,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":93976080,"statfs":{"total":96636764160,"available":96231505920,"internally_reserved":0,"allocated":332107776,"data_stored":331094396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575055,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306672,"kb_used_data":235236,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94065168,"statfs":{"total":96636764160,"available":96322732032,"internally_reserved":0,"allocated":240881664,"data_stored":239868284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575053,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":296688,"kb_used_data":251620,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94075152,"statfs":{"total":96636764160,"available":96332955648,"internally_reserved":0,"allocated":257658880,"data_stored":256645500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575053,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":82948,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288892,"statfs":{"total":96636764160,"available":96551825408,"internally_reserved":0,"allocated":1822720,"data_stored":798538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575053,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51592,"kb_used_data":2360,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320248,"statfs":{"total":96636764160,"available":96583933952,"internally_reserved":0,"allocated":2416640,"data_stored":1388906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:54:50.635 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:54:50.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.767+0000 7fa5445a6640 1 Processor -- start 2024-04-17T10:54:50.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.767+0000 7fa5445a6640 1 -- start start 2024-04-17T10:54:50.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.767+0000 7fa5445a6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa53c0cf060 0x7fa53c0b4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:50.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.767+0000 7fa5445a6640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa53c0b3990 0x7fa53c070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:50.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.767+0000 7fa5445a6640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa53c0b2bd0 con 0x7fa53c0b45a0 2024-04-17T10:54:50.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.767+0000 7fa5445a6640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa53c0b2980 con 0x7fa53c0b3990 2024-04-17T10:54:50.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.767+0000 7fa5445a6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c0cf440 con 0x7fa53c0cf060 2024-04-17T10:54:50.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.767+0000 7fa541b41640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa53c0b3990 0x7fa53c070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:50.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.767+0000 7fa542342640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa53c0cf060 0x7fa53c0b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:50.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.767+0000 7fa542b43640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fa53c0b45a0 0x7fa53c15d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:54202/0 (socket says 172.21.15.83:54202) 2024-04-17T10:54:50.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.767+0000 7fa542b43640 1 -- 172.21.15.83:0/2988951311 learned_addr learned my addr 172.21.15.83:0/2988951311 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:50.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.767+0000 7fa541b41640 1 -- 172.21.15.83:0/2988951311 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa53c0cf060 msgr2=0x7fa53c0b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:50.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.767+0000 7fa541b41640 1 --2- 172.21.15.83:0/2988951311 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa53c0cf060 0x7fa53c0b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:50.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.767+0000 7fa541b41640 1 -- 172.21.15.83:0/2988951311 >> v1:172.21.15.167:6789/0 conn(0x7fa53c0b45a0 legacy=0x7fa53c15d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:50.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.767+0000 7fa541b41640 1 -- 172.21.15.83:0/2988951311 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa53c0cf5c0 con 0x7fa53c0b3990 2024-04-17T10:54:50.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.771+0000 7fa541b41640 1 --2- 172.21.15.83:0/2988951311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa53c0b3990 0x7fa53c070470 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0097c0 tx=0x7fa52c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bb966b383274121 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:50.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.771+0000 7fa541340640 1 -- 172.21.15.83:0/2988951311 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c0b2070 con 0x7fa53c0b3990 2024-04-17T10:54:50.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.771+0000 7fa541340640 1 -- 172.21.15.83:0/2988951311 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52c0ad680 con 0x7fa53c0b3990 2024-04-17T10:54:50.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.771+0000 7fa541340640 1 -- 172.21.15.83:0/2988951311 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c0ada50 con 0x7fa53c0b3990 2024-04-17T10:54:50.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.771+0000 7fa5445a6640 1 -- 172.21.15.83:0/2988951311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa53c0b3990 msgr2=0x7fa53c070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:50.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.771+0000 7fa5445a6640 1 --2- 172.21.15.83:0/2988951311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa53c0b3990 0x7fa53c070470 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0097c0 tx=0x7fa52c01c470 comp rx=0 tx=0).stop 2024-04-17T10:54:50.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.771+0000 7fa5445a6640 1 -- 172.21.15.83:0/2988951311 shutdown_connections 2024-04-17T10:54:50.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.771+0000 7fa5445a6640 1 --2- 172.21.15.83:0/2988951311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa53c0b3990 0x7fa53c070470 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:50.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.771+0000 7fa5445a6640 1 --2- 172.21.15.83:0/2988951311 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa53c0cf060 0x7fa53c0b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:50.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.771+0000 7fa5445a6640 1 -- 172.21.15.83:0/2988951311 >> 172.21.15.83:0/2988951311 conn(0x7fa53c06abe0 msgr2=0x7fa53c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:50.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.771+0000 7fa5445a6640 1 -- 172.21.15.83:0/2988951311 shutdown_connections 2024-04-17T10:54:50.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.771+0000 7fa5445a6640 1 -- 172.21.15.83:0/2988951311 wait complete. 2024-04-17T10:54:50.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.771+0000 7fa5445a6640 1 Processor -- start 2024-04-17T10:54:50.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.771+0000 7fa5445a6640 1 -- start start 2024-04-17T10:54:50.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.775+0000 7fa5445a6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa53c0b3990 0x7fa53c1f5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:50.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.775+0000 7fa5445a6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa53c0b45a0 0x7fa53c156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:50.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.775+0000 7fa5445a6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa53c0cf060 0x7fa53c14e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:50.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.775+0000 7fa5445a6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c0cfc00 con 0x7fa53c0b45a0 2024-04-17T10:54:50.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.775+0000 7fa5445a6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c15b310 con 0x7fa53c0b3990 2024-04-17T10:54:50.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.775+0000 7fa5445a6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c164d80 con 0x7fa53c0cf060 2024-04-17T10:54:50.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.775+0000 7fa541b41640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa53c0b45a0 0x7fa53c156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:50.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.775+0000 7fa541b41640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa53c0b45a0 0x7fa53c156b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44352/0 (socket says 172.21.15.83:44352) 2024-04-17T10:54:50.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.775+0000 7fa541b41640 1 -- 172.21.15.83:0/1803435369 learned_addr learned my addr 172.21.15.83:0/1803435369 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:50.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.775+0000 7fa542342640 1 --2- 172.21.15.83:0/1803435369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa53c0b3990 0x7fa53c1f5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:50.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.775+0000 7fa542b43640 1 --2- 172.21.15.83:0/1803435369 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa53c0cf060 0x7fa53c14e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:50.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.775+0000 7fa541b41640 1 -- 172.21.15.83:0/1803435369 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa53c0cf060 msgr2=0x7fa53c14e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:50.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.775+0000 7fa541b41640 1 --2- 172.21.15.83:0/1803435369 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa53c0cf060 0x7fa53c14e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:50.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.775+0000 7fa541b41640 1 -- 172.21.15.83:0/1803435369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa53c0b3990 msgr2=0x7fa53c1f5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:50.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.775+0000 7fa541b41640 1 --2- 172.21.15.83:0/1803435369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa53c0b3990 0x7fa53c1f5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:50.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.775+0000 7fa541b41640 1 -- 172.21.15.83:0/1803435369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa53c155600 con 0x7fa53c0b45a0 2024-04-17T10:54:50.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.775+0000 7fa541b41640 1 --2- 172.21.15.83:0/1803435369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa53c0b45a0 0x7fa53c156b80 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7fa52c01a0a0 tx=0x7fa52c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:50.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.775+0000 7fa532ffd640 1 -- 172.21.15.83:0/1803435369 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c0b2040 con 0x7fa53c0b45a0 2024-04-17T10:54:50.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.775+0000 7fa532ffd640 1 -- 172.21.15.83:0/1803435369 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52c0bd070 con 0x7fa53c0b45a0 2024-04-17T10:54:50.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.775+0000 7fa5445a6640 1 -- 172.21.15.83:0/1803435369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa53c1558a0 con 0x7fa53c0b45a0 2024-04-17T10:54:50.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.775+0000 7fa532ffd640 1 -- 172.21.15.83:0/1803435369 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c0add70 con 0x7fa53c0b45a0 2024-04-17T10:54:50.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.775+0000 7fa5445a6640 1 -- 172.21.15.83:0/1803435369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa53c155e60 con 0x7fa53c0b45a0 2024-04-17T10:54:50.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.775+0000 7fa5445a6640 1 -- 172.21.15.83:0/1803435369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa53c074f90 con 0x7fa53c0b45a0 2024-04-17T10:54:50.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.775+0000 7fa532ffd640 1 -- 172.21.15.83:0/1803435369 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa52c0c2460 con 0x7fa53c0b45a0 2024-04-17T10:54:50.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.775+0000 7fa532ffd640 1 --2- 172.21.15.83:0/1803435369 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa520066f60 0x7fa520069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:50.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.775+0000 7fa532ffd640 1 -- 172.21.15.83:0/1803435369 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa52c11ef80 con 0x7fa53c0b45a0 2024-04-17T10:54:50.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.775+0000 7fa542342640 1 --2- 172.21.15.83:0/1803435369 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa520066f60 0x7fa520069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:50.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.779+0000 7fa542342640 1 --2- 172.21.15.83:0/1803435369 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa520066f60 0x7fa520069420 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fa53c153f90 tx=0x7fa538008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:50.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.779+0000 7fa532ffd640 1 -- 172.21.15.83:0/1803435369 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa52c016090 con 0x7fa53c0b45a0 2024-04-17T10:54:50.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.939+0000 7fa5445a6640 1 -- 172.21.15.83:0/1803435369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa53c06caf0 con 0x7fa53c0b45a0 2024-04-17T10:54:50.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.943+0000 7fa532ffd640 1 -- 172.21.15.83:0/1803435369 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+12958 (secure 0 0 0) 0x7fa52c0f0280 con 0x7fa53c0b45a0 2024-04-17T10:54:50.936 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:50.937 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":145,"fsid":"3de9b7be-af7f-488a-969e-2b6b8db63bcd","created":"2024-04-17T10:37:47.299708+0000","modified":"2024-04-17T10:54:47.625343+0000","last_up_change":"2024-04-17T10:37:59.327051+0000","last_in_change":"2024-04-17T10:37:50.436350+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T10:38:00.250968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5641e679-fdd4-491b-9c50-3c7909264821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6817","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6819","nonce":2375732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6821","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6823","nonce":2375732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6829","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6831","nonce":2375732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6825","nonce":2375732},{"type":"v1","addr":"172.21.15.83:6827","nonce":2375732}]},"public_addr":"172.21.15.83:6819/2375732","cluster_addr":"172.21.15.83:6823/2375732","heartbeat_back_addr":"172.21.15.83:6831/2375732","heartbeat_front_addr":"172.21.15.83:6827/2375732","state":["exists","up"]},{"osd":1,"uuid":"37721235-ee08-4822-9539-9f65a28dc6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6809","nonce":2278974484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6811","nonce":2278974484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6815","nonce":2278974484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2278974484},{"type":"v1","addr":"172.21.15.83:6813","nonce":2278974484}]},"public_addr":"172.21.15.83:6809/2278974484","cluster_addr":"172.21.15.83:6811/2278974484","heartbeat_back_addr":"172.21.15.83:6815/2278974484","heartbeat_front_addr":"172.21.15.83:6813/2278974484","state":["exists","up"]},{"osd":2,"uuid":"d4c58363-8146-4b9d-8331-feb0dd150dd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6818","nonce":3642321714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6822","nonce":3642321714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6830","nonce":3642321714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3642321714},{"type":"v1","addr":"172.21.15.83:6826","nonce":3642321714}]},"public_addr":"172.21.15.83:6818/3642321714","cluster_addr":"172.21.15.83:6822/3642321714","heartbeat_back_addr":"172.21.15.83:6830/3642321714","heartbeat_front_addr":"172.21.15.83:6826/3642321714","state":["exists","up"]},{"osd":3,"uuid":"74b46730-8ac3-4eb1-b573-45dd02be747d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6801","nonce":367778918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6803","nonce":367778918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6807","nonce":367778918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":367778918},{"type":"v1","addr":"172.21.15.83:6805","nonce":367778918}]},"public_addr":"172.21.15.83:6801/367778918","cluster_addr":"172.21.15.83:6803/367778918","heartbeat_back_addr":"172.21.15.83:6807/367778918","heartbeat_front_addr":"172.21.15.83:6805/367778918","state":["exists","up"]},{"osd":4,"uuid":"d28dd508-25ed-4dc0-bff9-bcd2480906ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6813","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6817","nonce":77470876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6825","nonce":77470876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6831","nonce":77470876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":77470876},{"type":"v1","addr":"172.21.15.167:6829","nonce":77470876}]},"public_addr":"172.21.15.167:6817/77470876","cluster_addr":"172.21.15.167:6825/77470876","heartbeat_back_addr":"172.21.15.167:6831/77470876","heartbeat_front_addr":"172.21.15.167:6829/77470876","state":["exists","up"]},{"osd":5,"uuid":"200a346c-44d2-4cf2-bc5a-66d9cfa79c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6801","nonce":1776227116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6803","nonce":1776227116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6807","nonce":1776227116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1776227116},{"type":"v1","addr":"172.21.15.167:6805","nonce":1776227116}]},"public_addr":"172.21.15.167:6801/1776227116","cluster_addr":"172.21.15.167:6803/1776227116","heartbeat_back_addr":"172.21.15.167:6807/1776227116","heartbeat_front_addr":"172.21.15.167:6805/1776227116","state":["exists","up"]},{"osd":6,"uuid":"30bd412b-9333-4b1a-bc0a-96bea209dae7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6815","nonce":1109787903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6819","nonce":1109787903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6828","nonce":1109787903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1109787903},{"type":"v1","addr":"172.21.15.167:6824","nonce":1109787903}]},"public_addr":"172.21.15.167:6815/1109787903","cluster_addr":"172.21.15.167:6819/1109787903","heartbeat_back_addr":"172.21.15.167:6828/1109787903","heartbeat_front_addr":"172.21.15.167:6824/1109787903","state":["exists","up"]},{"osd":7,"uuid":"23b53196-ca17-4727-b79c-1c43b245c21a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6809","nonce":285324536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6811","nonce":285324536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6823","nonce":285324536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":285324536},{"type":"v1","addr":"172.21.15.167:6816","nonce":285324536}]},"public_addr":"172.21.15.167:6809/285324536","cluster_addr":"172.21.15.167:6811/285324536","heartbeat_back_addr":"172.21.15.167:6823/285324536","heartbeat_front_addr":"172.21.15.167:6816/285324536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039226+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.979377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:56.039183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.935229+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.953957+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.743165+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.981123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T10:37:55.980675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/3575114599":"2024-04-18T10:51:04.828753+0000","172.21.15.83:6836/3575114599":"2024-04-18T10:51:04.828753+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:54:50.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.943+0000 7fa5445a6640 1 -- 172.21.15.83:0/1803435369 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa520066f60 msgr2=0x7fa520069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:50.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.943+0000 7fa5445a6640 1 --2- 172.21.15.83:0/1803435369 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa520066f60 0x7fa520069420 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fa53c153f90 tx=0x7fa538008040 comp rx=0 tx=0).stop 2024-04-17T10:54:50.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.943+0000 7fa5445a6640 1 -- 172.21.15.83:0/1803435369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa53c0b45a0 msgr2=0x7fa53c156b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:50.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.943+0000 7fa5445a6640 1 --2- 172.21.15.83:0/1803435369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa53c0b45a0 0x7fa53c156b80 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7fa52c01a0a0 tx=0x7fa52c01acb0 comp rx=0 tx=0).stop 2024-04-17T10:54:50.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.943+0000 7fa5445a6640 1 -- 172.21.15.83:0/1803435369 shutdown_connections 2024-04-17T10:54:50.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.943+0000 7fa5445a6640 1 --2- 172.21.15.83:0/1803435369 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fa520066f60 0x7fa520069420 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:50.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.943+0000 7fa5445a6640 1 --2- 172.21.15.83:0/1803435369 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa53c0cf060 0x7fa53c14e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:50.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.943+0000 7fa5445a6640 1 --2- 172.21.15.83:0/1803435369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa53c0b45a0 0x7fa53c156b80 unknown :-1 s=CLOSED pgs=2473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:50.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.943+0000 7fa5445a6640 1 --2- 172.21.15.83:0/1803435369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa53c0b3990 0x7fa53c1f5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:50.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.943+0000 7fa5445a6640 1 -- 172.21.15.83:0/1803435369 >> 172.21.15.83:0/1803435369 conn(0x7fa53c06abe0 msgr2=0x7fa53c158090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:50.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.943+0000 7fa5445a6640 1 -- 172.21.15.83:0/1803435369 shutdown_connections 2024-04-17T10:54:50.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:50.943+0000 7fa5445a6640 1 -- 172.21.15.83:0/1803435369 wait complete. 2024-04-17T10:54:51.956 INFO:tasks.ceph:Scrubbing osd.0 2024-04-17T10:54:51.956 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-17T10:54:52.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.095+0000 7f7945d62640 1 Processor -- start 2024-04-17T10:54:52.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.095+0000 7f7945d62640 1 -- start start 2024-04-17T10:54:52.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.095+0000 7f7945d62640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79401612c0 0x7f79400d1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:52.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.095+0000 7f7945d62640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f79400d0990 0x7f79400b4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:52.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.095+0000 7f7945d62640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79400b34b0 con 0x7f79400d2510 2024-04-17T10:54:52.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.095+0000 7f7945d62640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79400b0600 con 0x7f79400d0990 2024-04-17T10:54:52.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.095+0000 7f7945d62640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79400b3750 con 0x7f79401612c0 2024-04-17T10:54:52.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.095+0000 7f793effd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f79400d2510 0x7f79400d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42288/0 (socket says 172.21.15.83:42288) 2024-04-17T10:54:52.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.095+0000 7f793effd640 1 -- 172.21.15.83:0/4078050488 learned_addr learned my addr 172.21.15.83:0/4078050488 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:52.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.095+0000 7f793ffff640 1 --2- 172.21.15.83:0/4078050488 >> v2:172.21.15.83:3300/0 conn(0x7f79400d0990 0x7f79400b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:52.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.095+0000 7f793e7fc640 1 -- 172.21.15.83:0/4078050488 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2034363696 0 0) 0x7f79400b34b0 con 0x7f79400d2510 2024-04-17T10:54:52.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.095+0000 7f793e7fc640 1 -- 172.21.15.83:0/4078050488 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7928003530 con 0x7f79400d2510 2024-04-17T10:54:52.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.095+0000 7f793f7fe640 1 --2- 172.21.15.83:0/4078050488 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79401612c0 0x7f79400d1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:52.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.095+0000 7f793ffff640 1 -- 172.21.15.83:0/4078050488 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79401612c0 msgr2=0x7f79400d1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:52.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.095+0000 7f793ffff640 1 --2- 172.21.15.83:0/4078050488 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79401612c0 0x7f79400d1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.095+0000 7f793ffff640 1 -- 172.21.15.83:0/4078050488 >> v1:172.21.15.83:6789/0 conn(0x7f79400d2510 legacy=0x7f79400d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:52.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.095+0000 7f793ffff640 1 -- 172.21.15.83:0/4078050488 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79400b3a30 con 0x7f79400d0990 2024-04-17T10:54:52.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.099+0000 7f793ffff640 1 --2- 172.21.15.83:0/4078050488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79400d0990 0x7f79400b4230 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f7938009d90 tx=0x7f793801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79df169b630fe0b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:52.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.099+0000 7f793e7fc640 1 -- 172.21.15.83:0/4078050488 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79380b2070 con 0x7f79400d0990 2024-04-17T10:54:52.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.099+0000 7f793e7fc640 1 -- 172.21.15.83:0/4078050488 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79380a59a0 con 0x7f79400d0990 2024-04-17T10:54:52.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.099+0000 7f793e7fc640 1 -- 172.21.15.83:0/4078050488 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79380a5d70 con 0x7f79400d0990 2024-04-17T10:54:52.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.099+0000 7f7945d62640 1 -- 172.21.15.83:0/4078050488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79400d0990 msgr2=0x7f79400b4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:52.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.099+0000 7f7945d62640 1 --2- 172.21.15.83:0/4078050488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79400d0990 0x7f79400b4230 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f7938009d90 tx=0x7f793801c940 comp rx=0 tx=0).stop 2024-04-17T10:54:52.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.099+0000 7f7945d62640 1 -- 172.21.15.83:0/4078050488 shutdown_connections 2024-04-17T10:54:52.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.099+0000 7f7945d62640 1 --2- 172.21.15.83:0/4078050488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79400d0990 0x7f79400b4230 unknown :-1 s=CLOSED pgs=2475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.099+0000 7f7945d62640 1 --2- 172.21.15.83:0/4078050488 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79401612c0 0x7f79400d1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.099+0000 7f7945d62640 1 -- 172.21.15.83:0/4078050488 >> 172.21.15.83:0/4078050488 conn(0x7f794009cd10 msgr2=0x7f79400d3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:52.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.099+0000 7f7945d62640 1 -- 172.21.15.83:0/4078050488 shutdown_connections 2024-04-17T10:54:52.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.099+0000 7f7945d62640 1 -- 172.21.15.83:0/4078050488 wait complete. 2024-04-17T10:54:52.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.103+0000 7f7945d62640 1 Processor -- start 2024-04-17T10:54:52.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.103+0000 7f7945d62640 1 -- start start 2024-04-17T10:54:52.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.103+0000 7f7945d62640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79400d0990 0x7f79400c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:52.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.103+0000 7f7945d62640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79400d2510 0x7f7940155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:52.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.103+0000 7f7945d62640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79401612c0 0x7f7940156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:52.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.103+0000 7f7945d62640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79400b5100 con 0x7f79400d2510 2024-04-17T10:54:52.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.103+0000 7f7945d62640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79400b4f80 con 0x7f79401612c0 2024-04-17T10:54:52.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.103+0000 7f7945d62640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79400b5280 con 0x7f79400d0990 2024-04-17T10:54:52.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.103+0000 7f793effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79400d2510 0x7f7940155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:52.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.103+0000 7f793effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79400d2510 0x7f7940155dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44370/0 (socket says 172.21.15.83:44370) 2024-04-17T10:54:52.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.103+0000 7f793effd640 1 -- 172.21.15.83:0/3534836517 learned_addr learned my addr 172.21.15.83:0/3534836517 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:52.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.103+0000 7f793f7fe640 1 --2- 172.21.15.83:0/3534836517 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79400d0990 0x7f79400c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:52.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.103+0000 7f793ffff640 1 --2- 172.21.15.83:0/3534836517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79401612c0 0x7f7940156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:52.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.103+0000 7f793effd640 1 -- 172.21.15.83:0/3534836517 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79400d0990 msgr2=0x7f79400c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:52.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.103+0000 7f793effd640 1 --2- 172.21.15.83:0/3534836517 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79400d0990 0x7f79400c9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.103+0000 7f793effd640 1 -- 172.21.15.83:0/3534836517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79401612c0 msgr2=0x7f7940156590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:52.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.103+0000 7f793effd640 1 --2- 172.21.15.83:0/3534836517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79401612c0 0x7f7940156590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.103+0000 7f793effd640 1 -- 172.21.15.83:0/3534836517 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79400c86b0 con 0x7f79400d2510 2024-04-17T10:54:52.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.103+0000 7f793effd640 1 --2- 172.21.15.83:0/3534836517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79400d2510 0x7f7940155dc0 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f793400c420 tx=0x7f793400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:52.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.103+0000 7f7927fff640 1 -- 172.21.15.83:0/3534836517 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7934017070 con 0x7f79400d2510 2024-04-17T10:54:52.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.103+0000 7f7927fff640 1 -- 172.21.15.83:0/3534836517 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7934005d10 con 0x7f79400d2510 2024-04-17T10:54:52.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.103+0000 7f7945d62640 1 -- 172.21.15.83:0/3534836517 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79400ca380 con 0x7f79400d2510 2024-04-17T10:54:52.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.103+0000 7f7927fff640 1 -- 172.21.15.83:0/3534836517 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7934007310 con 0x7f79400d2510 2024-04-17T10:54:52.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.103+0000 7f7945d62640 1 -- 172.21.15.83:0/3534836517 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79400c8c30 con 0x7f79400d2510 2024-04-17T10:54:52.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.103+0000 7f7945d62640 1 -- 172.21.15.83:0/3534836517 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7940075000 con 0x7f79400d2510 2024-04-17T10:54:52.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.103+0000 7f7927fff640 1 -- 172.21.15.83:0/3534836517 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f793401a070 con 0x7f79400d2510 2024-04-17T10:54:52.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.107+0000 7f7927fff640 1 --2- 172.21.15.83:0/3534836517 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f792c066f60 0x7f792c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:52.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.107+0000 7f793f7fe640 1 --2- 172.21.15.83:0/3534836517 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f792c066f60 0x7f792c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:52.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.107+0000 7f7927fff640 1 -- 172.21.15.83:0/3534836517 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f79340848a0 con 0x7f79400d2510 2024-04-17T10:54:52.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.107+0000 7f7927fff640 1 --2- 172.21.15.83:0/3534836517 >> [v2:172.21.15.83:6817/2375732,v1:172.21.15.83:6819/2375732] conn(0x7f792c06ce20 0x7f792c06f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:52.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.107+0000 7f7927fff640 1 -- 172.21.15.83:0/3534836517 --> [v2:172.21.15.83:6817/2375732,v1:172.21.15.83:6819/2375732] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7934020ec0 con 0x7f792c06ce20 2024-04-17T10:54:52.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.107+0000 7f7927fff640 1 -- 172.21.15.83:0/3534836517 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=145) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7934084d30 con 0x7f79400d2510 2024-04-17T10:54:52.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.107+0000 7f793ffff640 1 --2- 172.21.15.83:0/3534836517 >> [v2:172.21.15.83:6817/2375732,v1:172.21.15.83:6819/2375732] conn(0x7f792c06ce20 0x7f792c06f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:52.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.107+0000 7f793f7fe640 1 --2- 172.21.15.83:0/3534836517 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f792c066f60 0x7f792c069420 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f79300096f0 tx=0x7f7930008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:52.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.107+0000 7f793ffff640 1 --2- 172.21.15.83:0/3534836517 >> [v2:172.21.15.83:6817/2375732,v1:172.21.15.83:6819/2375732] conn(0x7f792c06ce20 0x7f792c06f240 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:52.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.107+0000 7f7927fff640 1 -- 172.21.15.83:0/3534836517 <== osd.0 v2:172.21.15.83:6817/2375732 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7934020ec0 con 0x7f792c06ce20 2024-04-17T10:54:52.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.119+0000 7f7945d62640 1 -- 172.21.15.83:0/3534836517 --> [v2:172.21.15.83:6817/2375732,v1:172.21.15.83:6819/2375732] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f79400d1840 con 0x7f792c06ce20 2024-04-17T10:54:52.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.119+0000 7f7927fff640 1 -- 172.21.15.83:0/3534836517 <== osd.0 v2:172.21.15.83:6817/2375732 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f79400d1840 con 0x7f792c06ce20 2024-04-17T10:54:52.114 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:54:52.114 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-17T10:54:52.114 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:54:52.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.119+0000 7f7945d62640 1 -- 172.21.15.83:0/3534836517 >> [v2:172.21.15.83:6817/2375732,v1:172.21.15.83:6819/2375732] conn(0x7f792c06ce20 msgr2=0x7f792c06f240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:52.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.119+0000 7f7945d62640 1 --2- 172.21.15.83:0/3534836517 >> [v2:172.21.15.83:6817/2375732,v1:172.21.15.83:6819/2375732] conn(0x7f792c06ce20 0x7f792c06f240 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.119+0000 7f7945d62640 1 -- 172.21.15.83:0/3534836517 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f792c066f60 msgr2=0x7f792c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:52.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.119+0000 7f7945d62640 1 --2- 172.21.15.83:0/3534836517 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f792c066f60 0x7f792c069420 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f79300096f0 tx=0x7f7930008040 comp rx=0 tx=0).stop 2024-04-17T10:54:52.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.119+0000 7f7945d62640 1 -- 172.21.15.83:0/3534836517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79400d2510 msgr2=0x7f7940155dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:52.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.119+0000 7f7945d62640 1 --2- 172.21.15.83:0/3534836517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79400d2510 0x7f7940155dc0 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f793400c420 tx=0x7f793400c8e0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.119+0000 7f793ffff640 1 -- 172.21.15.83:0/3534836517 reap_dead start 2024-04-17T10:54:52.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.119+0000 7f7945d62640 1 -- 172.21.15.83:0/3534836517 shutdown_connections 2024-04-17T10:54:52.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.119+0000 7f7945d62640 1 -- 172.21.15.83:0/3534836517 >> 172.21.15.83:0/3534836517 conn(0x7f794009cd10 msgr2=0x7f79400d3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:52.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.119+0000 7f7945d62640 1 -- 172.21.15.83:0/3534836517 shutdown_connections 2024-04-17T10:54:52.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.119+0000 7f7945d62640 1 -- 172.21.15.83:0/3534836517 wait complete. 2024-04-17T10:54:52.130 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-17T10:54:52.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.251+0000 7f3cda2b6640 1 Processor -- start 2024-04-17T10:54:52.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.251+0000 7f3cda2b6640 1 -- start start 2024-04-17T10:54:52.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.251+0000 7f3cda2b6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3cd4154670 0x7f3cd40aea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:52.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.251+0000 7f3cda2b6640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cd40b2ff0 con 0x7f3cd40b4090 2024-04-17T10:54:52.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.251+0000 7f3cda2b6640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cd40b32d0 con 0x7f3cd406cc80 2024-04-17T10:54:52.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.251+0000 7f3cda2b6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd4154a50 con 0x7f3cd4154670 2024-04-17T10:54:52.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.251+0000 7f3cd37fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3cd40b4090 0x7f3cd406c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42304/0 (socket says 172.21.15.83:42304) 2024-04-17T10:54:52.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.251+0000 7f3cd37fe640 1 -- 172.21.15.83:0/3456015602 learned_addr learned my addr 172.21.15.83:0/3456015602 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:52.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.251+0000 7f3cd2ffd640 1 -- 172.21.15.83:0/3456015602 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1887386268 0 0) 0x7f3cd40b2ff0 con 0x7f3cd40b4090 2024-04-17T10:54:52.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.251+0000 7f3cd3fff640 1 --2- 172.21.15.83:0/3456015602 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3cd4154670 0x7f3cd40aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:52.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.251+0000 7f3cd2ffd640 1 -- 172.21.15.83:0/3456015602 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cc0004640 con 0x7f3cd40b4090 2024-04-17T10:54:52.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.251+0000 7f3cd2ffd640 1 -- 172.21.15.83:0/3456015602 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3995378825 0 0) 0x7f3cc0004640 con 0x7f3cd40b4090 2024-04-17T10:54:52.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.251+0000 7f3cd2ffd640 1 -- 172.21.15.83:0/3456015602 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3cd4154670 msgr2=0x7f3cd40aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:52.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.251+0000 7f3cd2ffd640 1 --2- 172.21.15.83:0/3456015602 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3cd4154670 0x7f3cd40aea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.251+0000 7f3cd2ffd640 1 -- 172.21.15.83:0/3456015602 >> v1:172.21.15.167:6789/0 conn(0x7f3cd406cc80 legacy=0x7f3cd4070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:52.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.251+0000 7f3cd2ffd640 1 -- 172.21.15.83:0/3456015602 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd40b35b0 con 0x7f3cd40b4090 2024-04-17T10:54:52.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.251+0000 7f3cd2ffd640 1 -- 172.21.15.83:0/3456015602 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f3cbc001680 con 0x7f3cd40b4090 2024-04-17T10:54:52.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.251+0000 7f3cd2ffd640 1 -- 172.21.15.83:0/3456015602 >> v1:172.21.15.83:6789/0 conn(0x7f3cd40b4090 legacy=0x7f3cd406c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:52.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.251+0000 7f3cd2ffd640 1 --2- 172.21.15.83:0/3456015602 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3cd4154670 0x7f3cc0004b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:52.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.251+0000 7f3cd2ffd640 1 --2- 172.21.15.83:0/3456015602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cc00050f0 0x7f3cc0005920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:52.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.251+0000 7f3cd2ffd640 1 --2- 172.21.15.83:0/3456015602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3cc0005e60 0x7f3cc00062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:52.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.251+0000 7f3cd2ffd640 1 -- 172.21.15.83:0/3456015602 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd4154a50 con 0x7f3cc00050f0 2024-04-17T10:54:52.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.251+0000 7f3cd2ffd640 1 -- 172.21.15.83:0/3456015602 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc000ccd0 con 0x7f3cc0005e60 2024-04-17T10:54:52.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.255+0000 7f3cd2ffd640 1 -- 172.21.15.83:0/3456015602 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc000ced0 con 0x7f3cd4154670 2024-04-17T10:54:52.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.255+0000 7f3cd37fe640 1 --2- 172.21.15.83:0/3456015602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cc00050f0 0x7f3cc0005920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:52.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.255+0000 7f3cd3fff640 1 --2- 172.21.15.83:0/3456015602 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3cd4154670 0x7f3cc0004b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:52.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.255+0000 7f3cd37fe640 1 -- 172.21.15.83:0/3456015602 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3cd4154670 msgr2=0x7f3cc0004b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:52.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.255+0000 7f3cd37fe640 1 --2- 172.21.15.83:0/3456015602 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3cd4154670 0x7f3cc0004b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.255+0000 7f3cd37fe640 1 -- 172.21.15.83:0/3456015602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3cc0005e60 msgr2=0x7f3cc00062e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:52.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.255+0000 7f3cd8853640 1 --2- 172.21.15.83:0/3456015602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3cc0005e60 0x7f3cc00062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:52.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.255+0000 7f3cd37fe640 1 --2- 172.21.15.83:0/3456015602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3cc0005e60 0x7f3cc00062e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.255+0000 7f3cd37fe640 1 -- 172.21.15.83:0/3456015602 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc0004640 con 0x7f3cc00050f0 2024-04-17T10:54:52.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.255+0000 7f3cd8853640 1 --2- 172.21.15.83:0/3456015602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3cc0005e60 0x7f3cc00062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:52.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.255+0000 7f3cd37fe640 1 --2- 172.21.15.83:0/3456015602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cc00050f0 0x7f3cc0005920 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc00c240 tx=0x7f3cbc01cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b10ea7d22e1fbb82 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:52.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.255+0000 7f3cd2ffd640 1 -- 172.21.15.83:0/3456015602 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc007d20 con 0x7f3cc00050f0 2024-04-17T10:54:52.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.255+0000 7f3cd2ffd640 1 -- 172.21.15.83:0/3456015602 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cbc0a9070 con 0x7f3cc00050f0 2024-04-17T10:54:52.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.255+0000 7f3cd2ffd640 1 -- 172.21.15.83:0/3456015602 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc0a5df0 con 0x7f3cc00050f0 2024-04-17T10:54:52.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.259+0000 7f3cda2b6640 1 -- 172.21.15.83:0/3456015602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cc00050f0 msgr2=0x7f3cc0005920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:52.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.259+0000 7f3cda2b6640 1 --2- 172.21.15.83:0/3456015602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cc00050f0 0x7f3cc0005920 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc00c240 tx=0x7f3cbc01cc50 comp rx=0 tx=0).stop 2024-04-17T10:54:52.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.259+0000 7f3cd8853640 1 -- 172.21.15.83:0/3456015602 reap_dead start 2024-04-17T10:54:52.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.259+0000 7f3cda2b6640 1 -- 172.21.15.83:0/3456015602 shutdown_connections 2024-04-17T10:54:52.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.259+0000 7f3cda2b6640 1 -- 172.21.15.83:0/3456015602 >> 172.21.15.83:0/3456015602 conn(0x7f3cd40b15b0 msgr2=0x7f3cd40b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:52.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.259+0000 7f3cda2b6640 1 -- 172.21.15.83:0/3456015602 shutdown_connections 2024-04-17T10:54:52.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.259+0000 7f3cda2b6640 1 -- 172.21.15.83:0/3456015602 wait complete. 2024-04-17T10:54:52.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.259+0000 7f3cda2b6640 1 Processor -- start 2024-04-17T10:54:52.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.259+0000 7f3cda2b6640 1 -- start start 2024-04-17T10:54:52.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.259+0000 7f3cda2b6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3cc00050f0 0x7f3cd4156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:52.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.259+0000 7f3cda2b6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3cd4153990 0x7f3cd414de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:52.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.259+0000 7f3cda2b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cd40ab460 0x7f3cd40ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:52.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.259+0000 7f3cda2b6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd40aebd0 con 0x7f3cd40ab460 2024-04-17T10:54:52.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.259+0000 7f3cda2b6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd40aea50 con 0x7f3cc00050f0 2024-04-17T10:54:52.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.259+0000 7f3cda2b6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd40aed50 con 0x7f3cd4153990 2024-04-17T10:54:52.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.259+0000 7f3cd8853640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cd40ab460 0x7f3cd40ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:52.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.259+0000 7f3cd8853640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cd40ab460 0x7f3cd40ab8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44396/0 (socket says 172.21.15.83:44396) 2024-04-17T10:54:52.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.259+0000 7f3cd8853640 1 -- 172.21.15.83:0/1893645089 learned_addr learned my addr 172.21.15.83:0/1893645089 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:52.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.259+0000 7f3cd3fff640 1 --2- 172.21.15.83:0/1893645089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3cc00050f0 0x7f3cd4156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:52.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.259+0000 7f3cd37fe640 1 --2- 172.21.15.83:0/1893645089 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3cd4153990 0x7f3cd414de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:52.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.259+0000 7f3cd8853640 1 -- 172.21.15.83:0/1893645089 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3cd4153990 msgr2=0x7f3cd414de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:52.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.259+0000 7f3cd8853640 1 --2- 172.21.15.83:0/1893645089 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3cd4153990 0x7f3cd414de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.259+0000 7f3cd8853640 1 -- 172.21.15.83:0/1893645089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3cc00050f0 msgr2=0x7f3cd4156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:52.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.259+0000 7f3cd8853640 1 --2- 172.21.15.83:0/1893645089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3cc00050f0 0x7f3cd4156de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.259+0000 7f3cd8853640 1 -- 172.21.15.83:0/1893645089 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd4157410 con 0x7f3cd40ab460 2024-04-17T10:54:52.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.259+0000 7f3cd8853640 1 --2- 172.21.15.83:0/1893645089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cd40ab460 0x7f3cd40ab8e0 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f3cd406d940 tx=0x7f3ccc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:52.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.259+0000 7f3cd0ff9640 1 -- 172.21.15.83:0/1893645089 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc006d60 con 0x7f3cd40ab460 2024-04-17T10:54:52.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.259+0000 7f3cd0ff9640 1 -- 172.21.15.83:0/1893645089 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ccc003980 con 0x7f3cd40ab460 2024-04-17T10:54:52.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.259+0000 7f3cda2b6640 1 -- 172.21.15.83:0/1893645089 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd41576f0 con 0x7f3cd40ab460 2024-04-17T10:54:52.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.259+0000 7f3cd0ff9640 1 -- 172.21.15.83:0/1893645089 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc003d70 con 0x7f3cd40ab460 2024-04-17T10:54:52.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.259+0000 7f3cda2b6640 1 -- 172.21.15.83:0/1893645089 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd4157cb0 con 0x7f3cd40ab460 2024-04-17T10:54:52.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.259+0000 7f3cda2b6640 1 -- 172.21.15.83:0/1893645089 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd4074f90 con 0x7f3cd40ab460 2024-04-17T10:54:52.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.263+0000 7f3cd0ff9640 1 -- 172.21.15.83:0/1893645089 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3ccc01b020 con 0x7f3cd40ab460 2024-04-17T10:54:52.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.263+0000 7f3cd0ff9640 1 --2- 172.21.15.83:0/1893645089 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3cb4066f60 0x7f3cb4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:52.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.263+0000 7f3cd0ff9640 1 -- 172.21.15.83:0/1893645089 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3ccc084a50 con 0x7f3cd40ab460 2024-04-17T10:54:52.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.263+0000 7f3cd3fff640 1 --2- 172.21.15.83:0/1893645089 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3cb4066f60 0x7f3cb4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:52.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.263+0000 7f3cd3fff640 1 --2- 172.21.15.83:0/1893645089 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3cb4066f60 0x7f3cb4069420 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4009a30 tx=0x7f3cc4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:52.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.267+0000 7f3cd0ff9640 1 -- 172.21.15.83:0/1893645089 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3ccc055d60 con 0x7f3cd40ab460 2024-04-17T10:54:52.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.423+0000 7f3cda2b6640 1 -- 172.21.15.83:0/1893645089 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f3cd40b2a00 con 0x7f3cb4066f60 2024-04-17T10:54:52.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.423+0000 7f3cd0ff9640 1 -- 172.21.15.83:0/1893645089 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3cd40b2a00 con 0x7f3cb4066f60 2024-04-17T10:54:52.417 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 0 to deep-scrub 2024-04-17T10:54:52.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.427+0000 7f3cda2b6640 1 -- 172.21.15.83:0/1893645089 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3cb4066f60 msgr2=0x7f3cb4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:52.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.427+0000 7f3cda2b6640 1 --2- 172.21.15.83:0/1893645089 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3cb4066f60 0x7f3cb4069420 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4009a30 tx=0x7f3cc4008040 comp rx=0 tx=0).stop 2024-04-17T10:54:52.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.427+0000 7f3cda2b6640 1 -- 172.21.15.83:0/1893645089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cd40ab460 msgr2=0x7f3cd40ab8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:52.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.427+0000 7f3cda2b6640 1 --2- 172.21.15.83:0/1893645089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cd40ab460 0x7f3cd40ab8e0 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f3cd406d940 tx=0x7f3ccc00dc00 comp rx=0 tx=0).stop 2024-04-17T10:54:52.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.427+0000 7f3cda2b6640 1 -- 172.21.15.83:0/1893645089 shutdown_connections 2024-04-17T10:54:52.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.427+0000 7f3cda2b6640 1 --2- 172.21.15.83:0/1893645089 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f3cb4066f60 0x7f3cb4069420 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.427+0000 7f3cda2b6640 1 --2- 172.21.15.83:0/1893645089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cd40ab460 0x7f3cd40ab8e0 unknown :-1 s=CLOSED pgs=2479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.427+0000 7f3cda2b6640 1 --2- 172.21.15.83:0/1893645089 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3cd4153990 0x7f3cd414de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.427+0000 7f3cda2b6640 1 --2- 172.21.15.83:0/1893645089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3cc00050f0 0x7f3cd4156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.427+0000 7f3cda2b6640 1 -- 172.21.15.83:0/1893645089 >> 172.21.15.83:0/1893645089 conn(0x7f3cd40b15b0 msgr2=0x7f3cd40af350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:52.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.427+0000 7f3cda2b6640 1 -- 172.21.15.83:0/1893645089 shutdown_connections 2024-04-17T10:54:52.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.427+0000 7f3cda2b6640 1 -- 172.21.15.83:0/1893645089 wait complete. 2024-04-17T10:54:52.436 INFO:tasks.ceph:Scrubbing osd.1 2024-04-17T10:54:52.436 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-17T10:54:52.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.567+0000 7f2c67083640 1 Processor -- start 2024-04-17T10:54:52.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.567+0000 7f2c67083640 1 -- start start 2024-04-17T10:54:52.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.567+0000 7f2c67083640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c600aebd0 0x7f2c600ced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:52.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.567+0000 7f2c67083640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2c6015b990 0x7f2c600cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:52.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.567+0000 7f2c67083640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2c600cd6c0 0x7f2c6006abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:52.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.567+0000 7f2c67083640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c6015e5c0 con 0x7f2c600cd6c0 2024-04-17T10:54:52.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.567+0000 7f2c67083640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c600cf3e0 con 0x7f2c6015b990 2024-04-17T10:54:52.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.567+0000 7f2c67083640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c600b4230 con 0x7f2c600aebd0 2024-04-17T10:54:52.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.567+0000 7f2c65620640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2c600cd6c0 0x7f2c6006abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:52.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.567+0000 7f2c65620640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2c600cd6c0 0x7f2c6006abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44400/0 (socket says 172.21.15.83:44400) 2024-04-17T10:54:52.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.567+0000 7f2c65620640 1 -- 172.21.15.83:0/2283456194 learned_addr learned my addr 172.21.15.83:0/2283456194 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:52.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.567+0000 7f2c64e1f640 1 --2- 172.21.15.83:0/2283456194 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c600aebd0 0x7f2c600ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:52.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.567+0000 7f2c5ffff640 1 --2- 172.21.15.83:0/2283456194 >> v2:172.21.15.167:3300/0 conn(0x7f2c6015b990 0x7f2c600cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:52.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.567+0000 7f2c65620640 1 -- 172.21.15.83:0/2283456194 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c600aebd0 msgr2=0x7f2c600ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:52.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.567+0000 7f2c65620640 1 --2- 172.21.15.83:0/2283456194 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c600aebd0 0x7f2c600ced50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.567+0000 7f2c65620640 1 -- 172.21.15.83:0/2283456194 >> v2:172.21.15.167:3300/0 conn(0x7f2c6015b990 msgr2=0x7f2c600cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:52.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.567+0000 7f2c65620640 1 --2- 172.21.15.83:0/2283456194 >> v2:172.21.15.167:3300/0 conn(0x7f2c6015b990 0x7f2c600cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.567+0000 7f2c65620640 1 -- 172.21.15.83:0/2283456194 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c600b4510 con 0x7f2c600cd6c0 2024-04-17T10:54:52.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.567+0000 7f2c64e1f640 1 --2- 172.21.15.83:0/2283456194 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c600aebd0 0x7f2c600ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:52.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.571+0000 7f2c65620640 1 --2- 172.21.15.83:0/2283456194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c600cd6c0 0x7f2c6006abe0 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800bb10 tx=0x7f2c5801ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca1f78b08800afa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:52.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.571+0000 7f2c5f7fe640 1 -- 172.21.15.83:0/2283456194 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c580b4070 con 0x7f2c600cd6c0 2024-04-17T10:54:52.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.571+0000 7f2c5f7fe640 1 -- 172.21.15.83:0/2283456194 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c580a7970 con 0x7f2c600cd6c0 2024-04-17T10:54:52.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.571+0000 7f2c5f7fe640 1 -- 172.21.15.83:0/2283456194 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c580a7d40 con 0x7f2c600cd6c0 2024-04-17T10:54:52.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.571+0000 7f2c67083640 1 -- 172.21.15.83:0/2283456194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c600cd6c0 msgr2=0x7f2c6006abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:52.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.571+0000 7f2c67083640 1 --2- 172.21.15.83:0/2283456194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c600cd6c0 0x7f2c6006abe0 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800bb10 tx=0x7f2c5801ebf0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.571+0000 7f2c67083640 1 -- 172.21.15.83:0/2283456194 shutdown_connections 2024-04-17T10:54:52.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.571+0000 7f2c67083640 1 --2- 172.21.15.83:0/2283456194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c600cd6c0 0x7f2c6006abe0 unknown :-1 s=CLOSED pgs=2480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.571+0000 7f2c67083640 1 --2- 172.21.15.83:0/2283456194 >> v2:172.21.15.167:3300/0 conn(0x7f2c6015b990 0x7f2c600cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.571+0000 7f2c67083640 1 --2- 172.21.15.83:0/2283456194 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c600aebd0 0x7f2c600ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.571+0000 7f2c67083640 1 -- 172.21.15.83:0/2283456194 >> 172.21.15.83:0/2283456194 conn(0x7f2c600cc050 msgr2=0x7f2c600cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:52.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.571+0000 7f2c67083640 1 -- 172.21.15.83:0/2283456194 shutdown_connections 2024-04-17T10:54:52.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.571+0000 7f2c67083640 1 -- 172.21.15.83:0/2283456194 wait complete. 2024-04-17T10:54:52.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.575+0000 7f2c67083640 1 Processor -- start 2024-04-17T10:54:52.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.575+0000 7f2c67083640 1 -- start start 2024-04-17T10:54:52.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.575+0000 7f2c67083640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c600aebd0 0x7f2c60150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:52.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.575+0000 7f2c67083640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c6015b990 0x7f2c60151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:52.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.575+0000 7f2c67083640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c60154730 0x7f2c6015cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:52.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.575+0000 7f2c67083640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c60165a40 con 0x7f2c600aebd0 2024-04-17T10:54:52.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.575+0000 7f2c67083640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c601658c0 con 0x7f2c60154730 2024-04-17T10:54:52.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.575+0000 7f2c67083640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c60165bc0 con 0x7f2c6015b990 2024-04-17T10:54:52.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.575+0000 7f2c64e1f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c600aebd0 0x7f2c60150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:52.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.575+0000 7f2c64e1f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c600aebd0 0x7f2c60150d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44414/0 (socket says 172.21.15.83:44414) 2024-04-17T10:54:52.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.575+0000 7f2c64e1f640 1 -- 172.21.15.83:0/1924955049 learned_addr learned my addr 172.21.15.83:0/1924955049 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:52.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.575+0000 7f2c64e1f640 1 -- 172.21.15.83:0/1924955049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c6015b990 msgr2=0x7f2c60151640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:52.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.575+0000 7f2c64e1f640 1 --2- 172.21.15.83:0/1924955049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c6015b990 0x7f2c60151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.575+0000 7f2c64e1f640 1 -- 172.21.15.83:0/1924955049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c60154730 msgr2=0x7f2c6015cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:52.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.575+0000 7f2c64e1f640 1 --2- 172.21.15.83:0/1924955049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c60154730 0x7f2c6015cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.575+0000 7f2c64e1f640 1 -- 172.21.15.83:0/1924955049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c60151f90 con 0x7f2c600aebd0 2024-04-17T10:54:52.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.575+0000 7f2c64e1f640 1 --2- 172.21.15.83:0/1924955049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c600aebd0 0x7f2c60150d90 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7f2c5000cf10 tx=0x7f2c5000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:52.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.575+0000 7f2c5d7fa640 1 -- 172.21.15.83:0/1924955049 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5001a070 con 0x7f2c600aebd0 2024-04-17T10:54:52.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.575+0000 7f2c5d7fa640 1 -- 172.21.15.83:0/1924955049 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5000d840 con 0x7f2c600aebd0 2024-04-17T10:54:52.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.575+0000 7f2c67083640 1 -- 172.21.15.83:0/1924955049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c60159f20 con 0x7f2c600aebd0 2024-04-17T10:54:52.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.575+0000 7f2c5d7fa640 1 -- 172.21.15.83:0/1924955049 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5000dc10 con 0x7f2c600aebd0 2024-04-17T10:54:52.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.575+0000 7f2c67083640 1 -- 172.21.15.83:0/1924955049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c6015a4e0 con 0x7f2c600aebd0 2024-04-17T10:54:52.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.575+0000 7f2c67083640 1 -- 172.21.15.83:0/1924955049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2c60075000 con 0x7f2c600aebd0 2024-04-17T10:54:52.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.575+0000 7f2c5d7fa640 1 -- 172.21.15.83:0/1924955049 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2c5001d070 con 0x7f2c600aebd0 2024-04-17T10:54:52.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.575+0000 7f2c5d7fa640 1 --2- 172.21.15.83:0/1924955049 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2c4c066f60 0x7f2c4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:52.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.575+0000 7f2c5ffff640 1 --2- 172.21.15.83:0/1924955049 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2c4c066f60 0x7f2c4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:52.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.575+0000 7f2c5d7fa640 1 -- 172.21.15.83:0/1924955049 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2c500854c0 con 0x7f2c600aebd0 2024-04-17T10:54:52.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.575+0000 7f2c5d7fa640 1 --2- 172.21.15.83:0/1924955049 >> [v2:172.21.15.83:6808/2278974484,v1:172.21.15.83:6809/2278974484] conn(0x7f2c4c06ce20 0x7f2c4c06f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:52.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.575+0000 7f2c5d7fa640 1 -- 172.21.15.83:0/1924955049 --> [v2:172.21.15.83:6808/2278974484,v1:172.21.15.83:6809/2278974484] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c50019ec0 con 0x7f2c4c06ce20 2024-04-17T10:54:52.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.575+0000 7f2c5d7fa640 1 -- 172.21.15.83:0/1924955049 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=145) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c5008c050 con 0x7f2c600aebd0 2024-04-17T10:54:52.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.579+0000 7f2c65620640 1 --2- 172.21.15.83:0/1924955049 >> [v2:172.21.15.83:6808/2278974484,v1:172.21.15.83:6809/2278974484] conn(0x7f2c4c06ce20 0x7f2c4c06f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:52.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.579+0000 7f2c5ffff640 1 --2- 172.21.15.83:0/1924955049 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2c4c066f60 0x7f2c4c069420 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f2c601fa730 tx=0x7f2c54008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:52.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.579+0000 7f2c65620640 1 --2- 172.21.15.83:0/1924955049 >> [v2:172.21.15.83:6808/2278974484,v1:172.21.15.83:6809/2278974484] conn(0x7f2c4c06ce20 0x7f2c4c06f240 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:52.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.579+0000 7f2c5d7fa640 1 -- 172.21.15.83:0/1924955049 <== osd.1 v2:172.21.15.83:6808/2278974484 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2c50019ec0 con 0x7f2c4c06ce20 2024-04-17T10:54:52.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.591+0000 7f2c67083640 1 -- 172.21.15.83:0/1924955049 --> [v2:172.21.15.83:6808/2278974484,v1:172.21.15.83:6809/2278974484] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2c600ccce0 con 0x7f2c4c06ce20 2024-04-17T10:54:52.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.591+0000 7f2c5d7fa640 1 -- 172.21.15.83:0/1924955049 <== osd.1 v2:172.21.15.83:6808/2278974484 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f2c600ccce0 con 0x7f2c4c06ce20 2024-04-17T10:54:52.585 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:54:52.585 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-17T10:54:52.585 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:54:52.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.591+0000 7f2c67083640 1 -- 172.21.15.83:0/1924955049 >> [v2:172.21.15.83:6808/2278974484,v1:172.21.15.83:6809/2278974484] conn(0x7f2c4c06ce20 msgr2=0x7f2c4c06f240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:52.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.591+0000 7f2c67083640 1 --2- 172.21.15.83:0/1924955049 >> [v2:172.21.15.83:6808/2278974484,v1:172.21.15.83:6809/2278974484] conn(0x7f2c4c06ce20 0x7f2c4c06f240 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.591+0000 7f2c67083640 1 -- 172.21.15.83:0/1924955049 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2c4c066f60 msgr2=0x7f2c4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:52.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.591+0000 7f2c67083640 1 --2- 172.21.15.83:0/1924955049 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f2c4c066f60 0x7f2c4c069420 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f2c601fa730 tx=0x7f2c54008040 comp rx=0 tx=0).stop 2024-04-17T10:54:52.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.591+0000 7f2c67083640 1 -- 172.21.15.83:0/1924955049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c600aebd0 msgr2=0x7f2c60150d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:52.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.591+0000 7f2c67083640 1 --2- 172.21.15.83:0/1924955049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c600aebd0 0x7f2c60150d90 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7f2c5000cf10 tx=0x7f2c5000a710 comp rx=0 tx=0).stop 2024-04-17T10:54:52.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.591+0000 7f2c65620640 1 -- 172.21.15.83:0/1924955049 reap_dead start 2024-04-17T10:54:52.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.591+0000 7f2c67083640 1 -- 172.21.15.83:0/1924955049 shutdown_connections 2024-04-17T10:54:52.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.591+0000 7f2c67083640 1 -- 172.21.15.83:0/1924955049 >> 172.21.15.83:0/1924955049 conn(0x7f2c600cc050 msgr2=0x7f2c6015ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:52.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.591+0000 7f2c67083640 1 -- 172.21.15.83:0/1924955049 shutdown_connections 2024-04-17T10:54:52.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.591+0000 7f2c67083640 1 -- 172.21.15.83:0/1924955049 wait complete. 2024-04-17T10:54:52.600 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-17T10:54:52.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.723+0000 7f4a48821640 1 Processor -- start 2024-04-17T10:54:52.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.723+0000 7f4a48821640 1 -- start start 2024-04-17T10:54:52.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.727+0000 7f4a48821640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a44154670 0x7f4a440aea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:52.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.727+0000 7f4a48821640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4a440b4090 0x7f4a4406c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:52.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.727+0000 7f4a48821640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a440b2ff0 con 0x7f4a440b44f0 2024-04-17T10:54:52.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.727+0000 7f4a48821640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a44154a50 con 0x7f4a440b4090 2024-04-17T10:54:52.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.727+0000 7f4a48821640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a440b32d0 con 0x7f4a44154670 2024-04-17T10:54:52.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.727+0000 7f4a4259c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a44154670 0x7f4a440aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:52.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.727+0000 7f4a41d9b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4a440b4090 0x7f4a4406c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:52.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.727+0000 7f4a4259c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a44154670 0x7f4a440aea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.83:57796/0 (socket says 172.21.15.83:57796) 2024-04-17T10:54:52.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.727+0000 7f4a42d9d640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f4a440b44f0 0x7f4a44070470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.83:54210/0 (socket says 172.21.15.83:54210) 2024-04-17T10:54:52.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.727+0000 7f4a4259c640 1 -- 172.21.15.83:0/1444383133 learned_addr learned my addr 172.21.15.83:0/1444383133 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:52.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.727+0000 7f4a41d9b640 1 -- 172.21.15.83:0/1444383133 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a44154670 msgr2=0x7f4a440aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:52.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.727+0000 7f4a41d9b640 1 --2- 172.21.15.83:0/1444383133 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a44154670 0x7f4a440aea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.727+0000 7f4a41d9b640 1 -- 172.21.15.83:0/1444383133 >> v1:172.21.15.167:6789/0 conn(0x7f4a440b44f0 legacy=0x7f4a44070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:52.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.727+0000 7f4a41d9b640 1 -- 172.21.15.83:0/1444383133 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a440b35b0 con 0x7f4a440b4090 2024-04-17T10:54:52.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.727+0000 7f4a41d9b640 1 --2- 172.21.15.83:0/1444383133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a440b4090 0x7f4a4406c570 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4a380097c0 tx=0x7f4a3801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66415940ff91c0ec server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:52.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.727+0000 7f4a4159a640 1 -- 172.21.15.83:0/1444383133 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a380a7070 con 0x7f4a440b4090 2024-04-17T10:54:52.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.727+0000 7f4a4159a640 1 -- 172.21.15.83:0/1444383133 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a380b0070 con 0x7f4a440b4090 2024-04-17T10:54:52.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.727+0000 7f4a4159a640 1 -- 172.21.15.83:0/1444383133 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a380ab890 con 0x7f4a440b4090 2024-04-17T10:54:52.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.731+0000 7f4a48821640 1 -- 172.21.15.83:0/1444383133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a440b4090 msgr2=0x7f4a4406c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:52.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.731+0000 7f4a48821640 1 --2- 172.21.15.83:0/1444383133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a440b4090 0x7f4a4406c570 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4a380097c0 tx=0x7f4a3801c470 comp rx=0 tx=0).stop 2024-04-17T10:54:52.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.731+0000 7f4a48821640 1 -- 172.21.15.83:0/1444383133 shutdown_connections 2024-04-17T10:54:52.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.731+0000 7f4a48821640 1 --2- 172.21.15.83:0/1444383133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a440b4090 0x7f4a4406c570 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.731+0000 7f4a48821640 1 --2- 172.21.15.83:0/1444383133 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a44154670 0x7f4a440aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.731+0000 7f4a48821640 1 -- 172.21.15.83:0/1444383133 >> 172.21.15.83:0/1444383133 conn(0x7f4a440b15b0 msgr2=0x7f4a440b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:52.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.731+0000 7f4a48821640 1 -- 172.21.15.83:0/1444383133 shutdown_connections 2024-04-17T10:54:52.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.731+0000 7f4a48821640 1 -- 172.21.15.83:0/1444383133 wait complete. 2024-04-17T10:54:52.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.731+0000 7f4a48821640 1 Processor -- start 2024-04-17T10:54:52.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.731+0000 7f4a48821640 1 -- start start 2024-04-17T10:54:52.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.731+0000 7f4a48821640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a440b4090 0x7f4a4414f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:52.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.731+0000 7f4a48821640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a440b44f0 0x7f4a441568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:52.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.731+0000 7f4a48821640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a44154670 0x7f4a4415a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:52.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.731+0000 7f4a48821640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a440aef90 con 0x7f4a440b44f0 2024-04-17T10:54:52.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.731+0000 7f4a48821640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4406cec0 con 0x7f4a440b4090 2024-04-17T10:54:52.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.731+0000 7f4a48821640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a440af110 con 0x7f4a44154670 2024-04-17T10:54:52.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.731+0000 7f4a41d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a440b44f0 0x7f4a441568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:52.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.731+0000 7f4a41d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a440b44f0 0x7f4a441568d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44418/0 (socket says 172.21.15.83:44418) 2024-04-17T10:54:52.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.731+0000 7f4a41d9b640 1 -- 172.21.15.83:0/2731337182 learned_addr learned my addr 172.21.15.83:0/2731337182 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:52.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.731+0000 7f4a4259c640 1 --2- 172.21.15.83:0/2731337182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a440b4090 0x7f4a4414f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:52.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.731+0000 7f4a42d9d640 1 --2- 172.21.15.83:0/2731337182 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a44154670 0x7f4a4415a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:52.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.731+0000 7f4a41d9b640 1 -- 172.21.15.83:0/2731337182 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a44154670 msgr2=0x7f4a4415a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:52.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.731+0000 7f4a41d9b640 1 --2- 172.21.15.83:0/2731337182 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a44154670 0x7f4a4415a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.731+0000 7f4a41d9b640 1 -- 172.21.15.83:0/2731337182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a440b4090 msgr2=0x7f4a4414f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:52.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.731+0000 7f4a41d9b640 1 --2- 172.21.15.83:0/2731337182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a440b4090 0x7f4a4414f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.731+0000 7f4a41d9b640 1 -- 172.21.15.83:0/2731337182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a44155830 con 0x7f4a440b44f0 2024-04-17T10:54:52.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.731+0000 7f4a41d9b640 1 --2- 172.21.15.83:0/2731337182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a440b44f0 0x7f4a441568d0 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7f4a3801a0a0 tx=0x7f4a3801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:52.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.731+0000 7f4a32ffd640 1 -- 172.21.15.83:0/2731337182 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a380bf610 con 0x7f4a440b44f0 2024-04-17T10:54:52.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.731+0000 7f4a32ffd640 1 -- 172.21.15.83:0/2731337182 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a380bf7b0 con 0x7f4a440b44f0 2024-04-17T10:54:52.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.731+0000 7f4a32ffd640 1 -- 172.21.15.83:0/2731337182 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a380bfba0 con 0x7f4a440b44f0 2024-04-17T10:54:52.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.731+0000 7f4a48821640 1 -- 172.21.15.83:0/2731337182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a44155b10 con 0x7f4a440b44f0 2024-04-17T10:54:52.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.731+0000 7f4a48821640 1 -- 172.21.15.83:0/2731337182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a441560d0 con 0x7f4a440b44f0 2024-04-17T10:54:52.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.735+0000 7f4a48821640 1 -- 172.21.15.83:0/2731337182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a44074f90 con 0x7f4a440b44f0 2024-04-17T10:54:52.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.735+0000 7f4a32ffd640 1 -- 172.21.15.83:0/2731337182 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4a380bd020 con 0x7f4a440b44f0 2024-04-17T10:54:52.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.735+0000 7f4a32ffd640 1 --2- 172.21.15.83:0/2731337182 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4a24066f60 0x7f4a24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:52.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.735+0000 7f4a32ffd640 1 -- 172.21.15.83:0/2731337182 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4a3811ef90 con 0x7f4a440b44f0 2024-04-17T10:54:52.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.735+0000 7f4a4259c640 1 --2- 172.21.15.83:0/2731337182 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4a24066f60 0x7f4a24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:52.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.735+0000 7f4a4259c640 1 --2- 172.21.15.83:0/2731337182 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4a24066f60 0x7f4a24069420 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f4a34006cd0 tx=0x7f4a3400a350 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:52.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.739+0000 7f4a32ffd640 1 -- 172.21.15.83:0/2731337182 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4a380c6030 con 0x7f4a440b44f0 2024-04-17T10:54:52.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.887+0000 7f4a48821640 1 -- 172.21.15.83:0/2731337182 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a440b2a00 con 0x7f4a24066f60 2024-04-17T10:54:52.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.887+0000 7f4a32ffd640 1 -- 172.21.15.83:0/2731337182 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4a440b2a00 con 0x7f4a24066f60 2024-04-17T10:54:52.880 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 1 to deep-scrub 2024-04-17T10:54:52.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.887+0000 7f4a48821640 1 -- 172.21.15.83:0/2731337182 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4a24066f60 msgr2=0x7f4a24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:52.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.887+0000 7f4a48821640 1 --2- 172.21.15.83:0/2731337182 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4a24066f60 0x7f4a24069420 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f4a34006cd0 tx=0x7f4a3400a350 comp rx=0 tx=0).stop 2024-04-17T10:54:52.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.887+0000 7f4a48821640 1 -- 172.21.15.83:0/2731337182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a440b44f0 msgr2=0x7f4a441568d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:52.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.887+0000 7f4a48821640 1 --2- 172.21.15.83:0/2731337182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a440b44f0 0x7f4a441568d0 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7f4a3801a0a0 tx=0x7f4a3801acb0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.887+0000 7f4a48821640 1 -- 172.21.15.83:0/2731337182 shutdown_connections 2024-04-17T10:54:52.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.887+0000 7f4a48821640 1 --2- 172.21.15.83:0/2731337182 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f4a24066f60 0x7f4a24069420 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.887+0000 7f4a48821640 1 --2- 172.21.15.83:0/2731337182 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a44154670 0x7f4a4415a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.887+0000 7f4a48821640 1 --2- 172.21.15.83:0/2731337182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a440b44f0 0x7f4a441568d0 unknown :-1 s=CLOSED pgs=2482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.887+0000 7f4a48821640 1 --2- 172.21.15.83:0/2731337182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a440b4090 0x7f4a4414f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:52.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.887+0000 7f4a48821640 1 -- 172.21.15.83:0/2731337182 >> 172.21.15.83:0/2731337182 conn(0x7f4a440b15b0 msgr2=0x7f4a4415cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:52.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.887+0000 7f4a48821640 1 -- 172.21.15.83:0/2731337182 shutdown_connections 2024-04-17T10:54:52.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:52.887+0000 7f4a48821640 1 -- 172.21.15.83:0/2731337182 wait complete. 2024-04-17T10:54:52.897 INFO:tasks.ceph:Scrubbing osd.2 2024-04-17T10:54:52.898 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-17T10:54:53.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032fdcd640 1 Processor -- start 2024-04-17T10:54:53.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032fdcd640 1 -- start start 2024-04-17T10:54:53.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032fdcd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03280acde0 0x7f032815f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032fdcd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f032815f9b0 0x7f03281520a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032fdcd640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03280ad3c0 con 0x7f03280aec30 2024-04-17T10:54:53.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032fdcd640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03280adfd0 con 0x7f032815f9b0 2024-04-17T10:54:53.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032fdcd640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03280ad6a0 con 0x7f03280acde0 2024-04-17T10:54:53.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032db69640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f03280aec30 0x7f03280af010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42308/0 (socket says 172.21.15.83:42308) 2024-04-17T10:54:53.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032db69640 1 -- 172.21.15.83:0/4145874865 learned_addr learned my addr 172.21.15.83:0/4145874865 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:53.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032d368640 1 --2- 172.21.15.83:0/4145874865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03280acde0 0x7f032815f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032e36a640 1 --2- 172.21.15.83:0/4145874865 >> v2:172.21.15.167:3300/0 conn(0x7f032815f9b0 0x7f03281520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032cb67640 1 -- 172.21.15.83:0/4145874865 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4089064894 0 0) 0x7f03280ad3c0 con 0x7f03280aec30 2024-04-17T10:54:53.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032cb67640 1 -- 172.21.15.83:0/4145874865 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0314003530 con 0x7f03280aec30 2024-04-17T10:54:53.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032cb67640 1 -- 172.21.15.83:0/4145874865 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2061539019 0 0) 0x7f0314003530 con 0x7f03280aec30 2024-04-17T10:54:53.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032cb67640 1 -- 172.21.15.83:0/4145874865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03280acde0 msgr2=0x7f032815f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:53.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032cb67640 1 --2- 172.21.15.83:0/4145874865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03280acde0 0x7f032815f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032cb67640 1 -- 172.21.15.83:0/4145874865 >> v2:172.21.15.167:3300/0 conn(0x7f032815f9b0 msgr2=0x7f03281520a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:53.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032cb67640 1 --2- 172.21.15.83:0/4145874865 >> v2:172.21.15.167:3300/0 conn(0x7f032815f9b0 0x7f03281520a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032cb67640 1 -- 172.21.15.83:0/4145874865 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03280ad980 con 0x7f03280aec30 2024-04-17T10:54:53.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032cb67640 1 -- 172.21.15.83:0/4145874865 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f0320001a00 con 0x7f03280aec30 2024-04-17T10:54:53.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032cb67640 1 -- 172.21.15.83:0/4145874865 >> v1:172.21.15.83:6789/0 conn(0x7f03280aec30 legacy=0x7f03280af010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:53.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032cb67640 1 --2- 172.21.15.83:0/4145874865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03140039c0 0x7f0314003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032cb67640 1 --2- 172.21.15.83:0/4145874865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03280acde0 0x7f0314004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032cb67640 1 --2- 172.21.15.83:0/4145874865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0314004d50 0x7f03140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032cb67640 1 -- 172.21.15.83:0/4145874865 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03280adfd0 con 0x7f0314004d50 2024-04-17T10:54:53.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032cb67640 1 -- 172.21.15.83:0/4145874865 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03280ad6a0 con 0x7f03140039c0 2024-04-17T10:54:53.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032cb67640 1 -- 172.21.15.83:0/4145874865 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f031400b9d0 con 0x7f03280acde0 2024-04-17T10:54:53.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032e36a640 1 --2- 172.21.15.83:0/4145874865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0314004d50 0x7f03140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032db69640 1 --2- 172.21.15.83:0/4145874865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03140039c0 0x7f0314003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032d368640 1 --2- 172.21.15.83:0/4145874865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03280acde0 0x7f0314004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032e36a640 1 -- 172.21.15.83:0/4145874865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03280acde0 msgr2=0x7f0314004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:53.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032e36a640 1 --2- 172.21.15.83:0/4145874865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03280acde0 0x7f0314004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032e36a640 1 -- 172.21.15.83:0/4145874865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03140039c0 msgr2=0x7f0314003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:53.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032e36a640 1 --2- 172.21.15.83:0/4145874865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03140039c0 0x7f0314003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.019+0000 7f032e36a640 1 -- 172.21.15.83:0/4145874865 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0314003530 con 0x7f0314004d50 2024-04-17T10:54:53.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.023+0000 7f032e36a640 1 --2- 172.21.15.83:0/4145874865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0314004d50 0x7f03140051d0 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7f0324009ae0 tx=0x7f032401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b4bfb60c7fee server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:53.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.023+0000 7f032cb67640 1 -- 172.21.15.83:0/4145874865 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03240b2070 con 0x7f0314004d50 2024-04-17T10:54:53.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.023+0000 7f032cb67640 1 -- 172.21.15.83:0/4145874865 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03240a5590 con 0x7f0314004d50 2024-04-17T10:54:53.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.023+0000 7f032cb67640 1 -- 172.21.15.83:0/4145874865 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03240096b0 con 0x7f0314004d50 2024-04-17T10:54:53.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.023+0000 7f032fdcd640 1 -- 172.21.15.83:0/4145874865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0314004d50 msgr2=0x7f03140051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:53.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.023+0000 7f032fdcd640 1 --2- 172.21.15.83:0/4145874865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0314004d50 0x7f03140051d0 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7f0324009ae0 tx=0x7f032401cab0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.023+0000 7f032e36a640 1 -- 172.21.15.83:0/4145874865 reap_dead start 2024-04-17T10:54:53.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.023+0000 7f032fdcd640 1 -- 172.21.15.83:0/4145874865 shutdown_connections 2024-04-17T10:54:53.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.023+0000 7f032fdcd640 1 -- 172.21.15.83:0/4145874865 >> 172.21.15.83:0/4145874865 conn(0x7f03280d0450 msgr2=0x7f03280d0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:53.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.023+0000 7f032fdcd640 1 -- 172.21.15.83:0/4145874865 shutdown_connections 2024-04-17T10:54:53.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.023+0000 7f032fdcd640 1 -- 172.21.15.83:0/4145874865 wait complete. 2024-04-17T10:54:53.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.023+0000 7f032fdcd640 1 Processor -- start 2024-04-17T10:54:53.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.023+0000 7f032fdcd640 1 -- start start 2024-04-17T10:54:53.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f032fdcd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0314004d50 0x7f032814a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f032fdcd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03280cc750 0x7f03280ce390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f032fdcd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03280ce8d0 0x7f0328070c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f032fdcd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03280af8a0 con 0x7f03280ce8d0 2024-04-17T10:54:53.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f032fdcd640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03280af720 con 0x7f0314004d50 2024-04-17T10:54:53.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f032fdcd640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f032815f470 con 0x7f03280cc750 2024-04-17T10:54:53.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f032e36a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03280ce8d0 0x7f0328070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f032e36a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03280ce8d0 0x7f0328070c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44446/0 (socket says 172.21.15.83:44446) 2024-04-17T10:54:53.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f032e36a640 1 -- 172.21.15.83:0/1893648770 learned_addr learned my addr 172.21.15.83:0/1893648770 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:53.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f032db69640 1 --2- 172.21.15.83:0/1893648770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0314004d50 0x7f032814a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f032e36a640 1 -- 172.21.15.83:0/1893648770 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03280cc750 msgr2=0x7f03280ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:53.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f032d368640 1 --2- 172.21.15.83:0/1893648770 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03280cc750 0x7f03280ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f032e36a640 1 --2- 172.21.15.83:0/1893648770 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03280cc750 0x7f03280ce390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f032e36a640 1 -- 172.21.15.83:0/1893648770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0314004d50 msgr2=0x7f032814a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:53.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f032e36a640 1 --2- 172.21.15.83:0/1893648770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0314004d50 0x7f032814a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f032e36a640 1 -- 172.21.15.83:0/1893648770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0328143360 con 0x7f03280ce8d0 2024-04-17T10:54:53.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f032db69640 1 --2- 172.21.15.83:0/1893648770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0314004d50 0x7f032814a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:53.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f032e36a640 1 --2- 172.21.15.83:0/1893648770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03280ce8d0 0x7f0328070c10 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f03240b0040 tx=0x7f0314005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:53.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f031e7fc640 1 -- 172.21.15.83:0/1893648770 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03240b2040 con 0x7f03280ce8d0 2024-04-17T10:54:53.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f031e7fc640 1 -- 172.21.15.83:0/1893648770 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03240a5e40 con 0x7f03280ce8d0 2024-04-17T10:54:53.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f032fdcd640 1 -- 172.21.15.83:0/1893648770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f032814aac0 con 0x7f03280ce8d0 2024-04-17T10:54:53.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f031e7fc640 1 -- 172.21.15.83:0/1893648770 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03240096b0 con 0x7f03280ce8d0 2024-04-17T10:54:53.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f032fdcd640 1 -- 172.21.15.83:0/1893648770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f032814b080 con 0x7f03280ce8d0 2024-04-17T10:54:53.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f032fdcd640 1 -- 172.21.15.83:0/1893648770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f03280681b0 con 0x7f03280ce8d0 2024-04-17T10:54:53.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f031e7fc640 1 -- 172.21.15.83:0/1893648770 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0324004220 con 0x7f03280ce8d0 2024-04-17T10:54:53.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f031e7fc640 1 --2- 172.21.15.83:0/1893648770 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f030c066f60 0x7f030c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f031e7fc640 1 -- 172.21.15.83:0/1893648770 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f032411eaa0 con 0x7f03280ce8d0 2024-04-17T10:54:53.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f032db69640 1 --2- 172.21.15.83:0/1893648770 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f030c066f60 0x7f030c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f031e7fc640 1 --2- 172.21.15.83:0/1893648770 >> [v2:172.21.15.83:6816/3642321714,v1:172.21.15.83:6818/3642321714] conn(0x7f030c06ce20 0x7f030c06f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f031e7fc640 1 -- 172.21.15.83:0/1893648770 --> [v2:172.21.15.83:6816/3642321714,v1:172.21.15.83:6818/3642321714] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f03240a5590 con 0x7f030c06ce20 2024-04-17T10:54:53.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f031e7fc640 1 -- 172.21.15.83:0/1893648770 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=145) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03241232c0 con 0x7f03280ce8d0 2024-04-17T10:54:53.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f032d368640 1 --2- 172.21.15.83:0/1893648770 >> [v2:172.21.15.83:6816/3642321714,v1:172.21.15.83:6818/3642321714] conn(0x7f030c06ce20 0x7f030c06f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f032db69640 1 --2- 172.21.15.83:0/1893648770 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f030c066f60 0x7f030c069420 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f0320001b70 tx=0x7f0320000f50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:53.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.027+0000 7f032d368640 1 --2- 172.21.15.83:0/1893648770 >> [v2:172.21.15.83:6816/3642321714,v1:172.21.15.83:6818/3642321714] conn(0x7f030c06ce20 0x7f030c06f240 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:53.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.031+0000 7f031e7fc640 1 -- 172.21.15.83:0/1893648770 <== osd.2 v2:172.21.15.83:6816/3642321714 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f03240a5590 con 0x7f030c06ce20 2024-04-17T10:54:53.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.043+0000 7f032fdcd640 1 -- 172.21.15.83:0/1893648770 --> [v2:172.21.15.83:6816/3642321714,v1:172.21.15.83:6818/3642321714] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0328075000 con 0x7f030c06ce20 2024-04-17T10:54:53.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.043+0000 7f031e7fc640 1 -- 172.21.15.83:0/1893648770 <== osd.2 v2:172.21.15.83:6816/3642321714 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f0328075000 con 0x7f030c06ce20 2024-04-17T10:54:53.037 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:54:53.037 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-17T10:54:53.037 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:54:53.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.043+0000 7f032fdcd640 1 -- 172.21.15.83:0/1893648770 >> [v2:172.21.15.83:6816/3642321714,v1:172.21.15.83:6818/3642321714] conn(0x7f030c06ce20 msgr2=0x7f030c06f240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:53.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.043+0000 7f032fdcd640 1 --2- 172.21.15.83:0/1893648770 >> [v2:172.21.15.83:6816/3642321714,v1:172.21.15.83:6818/3642321714] conn(0x7f030c06ce20 0x7f030c06f240 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.043+0000 7f032fdcd640 1 -- 172.21.15.83:0/1893648770 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f030c066f60 msgr2=0x7f030c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:53.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.043+0000 7f032fdcd640 1 --2- 172.21.15.83:0/1893648770 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f030c066f60 0x7f030c069420 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f0320001b70 tx=0x7f0320000f50 comp rx=0 tx=0).stop 2024-04-17T10:54:53.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.043+0000 7f032fdcd640 1 -- 172.21.15.83:0/1893648770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03280ce8d0 msgr2=0x7f0328070c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:53.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.043+0000 7f032fdcd640 1 --2- 172.21.15.83:0/1893648770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03280ce8d0 0x7f0328070c10 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f03240b0040 tx=0x7f0314005840 comp rx=0 tx=0).stop 2024-04-17T10:54:53.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.043+0000 7f032e36a640 1 -- 172.21.15.83:0/1893648770 reap_dead start 2024-04-17T10:54:53.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.043+0000 7f032fdcd640 1 -- 172.21.15.83:0/1893648770 shutdown_connections 2024-04-17T10:54:53.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.043+0000 7f032fdcd640 1 -- 172.21.15.83:0/1893648770 >> 172.21.15.83:0/1893648770 conn(0x7f03280d0450 msgr2=0x7f03280b49c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:53.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.043+0000 7f032fdcd640 1 -- 172.21.15.83:0/1893648770 shutdown_connections 2024-04-17T10:54:53.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.043+0000 7f032fdcd640 1 -- 172.21.15.83:0/1893648770 wait complete. 2024-04-17T10:54:53.051 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-17T10:54:53.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.167+0000 7f8f9aeaf640 1 Processor -- start 2024-04-17T10:54:53.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.167+0000 7f8f9aeaf640 1 -- start start 2024-04-17T10:54:53.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.171+0000 7f8f9aeaf640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8f9415e0f0 0x7f8f940b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.171+0000 7f8f9aeaf640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8f940ce400 0x7f8f94161c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.171+0000 7f8f9aeaf640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f9415e4d0 con 0x7f8f940b5450 2024-04-17T10:54:53.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.171+0000 7f8f9aeaf640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f94162b40 con 0x7f8f9415e0f0 2024-04-17T10:54:53.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.171+0000 7f8f9aeaf640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f94162580 con 0x7f8f940ce400 2024-04-17T10:54:53.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.171+0000 7f8f98c4b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8f9415e0f0 0x7f8f940b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.171+0000 7f8f98c4b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8f9415e0f0 0x7f8f940b6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44458/0 (socket says 172.21.15.83:44458) 2024-04-17T10:54:53.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.171+0000 7f8f98c4b640 1 -- 172.21.15.83:0/4159619096 learned_addr learned my addr 172.21.15.83:0/4159619096 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:53.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.171+0000 7f8f93fff640 1 --2- 172.21.15.83:0/4159619096 >> v2:172.21.15.167:3300/0 conn(0x7f8f940ce400 0x7f8f94161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.171+0000 7f8f98c4b640 1 -- 172.21.15.83:0/4159619096 >> v2:172.21.15.167:3300/0 conn(0x7f8f940ce400 msgr2=0x7f8f94161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:53.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.171+0000 7f8f98c4b640 1 --2- 172.21.15.83:0/4159619096 >> v2:172.21.15.167:3300/0 conn(0x7f8f940ce400 0x7f8f94161c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.171+0000 7f8f98c4b640 1 -- 172.21.15.83:0/4159619096 >> v1:172.21.15.167:6789/0 conn(0x7f8f940b5450 legacy=0x7f8f940d2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:53.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.171+0000 7f8f98c4b640 1 -- 172.21.15.83:0/4159619096 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f94162860 con 0x7f8f9415e0f0 2024-04-17T10:54:53.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.171+0000 7f8f98c4b640 1 --2- 172.21.15.83:0/4159619096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f9415e0f0 0x7f8f940b6d00 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7f8f94068170 tx=0x7f8f8401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf09b8b0927d3b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:53.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.171+0000 7f8f937fe640 1 -- 172.21.15.83:0/4159619096 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f840b1070 con 0x7f8f9415e0f0 2024-04-17T10:54:53.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.171+0000 7f8f937fe640 1 -- 172.21.15.83:0/4159619096 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f840ac640 con 0x7f8f9415e0f0 2024-04-17T10:54:53.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.171+0000 7f8f937fe640 1 -- 172.21.15.83:0/4159619096 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f840aca10 con 0x7f8f9415e0f0 2024-04-17T10:54:53.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.175+0000 7f8f9aeaf640 1 -- 172.21.15.83:0/4159619096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f9415e0f0 msgr2=0x7f8f940b6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:53.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.175+0000 7f8f9aeaf640 1 --2- 172.21.15.83:0/4159619096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f9415e0f0 0x7f8f940b6d00 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7f8f94068170 tx=0x7f8f8401c450 comp rx=0 tx=0).stop 2024-04-17T10:54:53.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.175+0000 7f8f9aeaf640 1 -- 172.21.15.83:0/4159619096 shutdown_connections 2024-04-17T10:54:53.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.175+0000 7f8f9aeaf640 1 --2- 172.21.15.83:0/4159619096 >> v2:172.21.15.167:3300/0 conn(0x7f8f940ce400 0x7f8f94161c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.175+0000 7f8f9aeaf640 1 --2- 172.21.15.83:0/4159619096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f9415e0f0 0x7f8f940b6d00 unknown :-1 s=CLOSED pgs=2486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.175+0000 7f8f9aeaf640 1 -- 172.21.15.83:0/4159619096 >> 172.21.15.83:0/4159619096 conn(0x7f8f940d0450 msgr2=0x7f8f940d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:53.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.175+0000 7f8f9aeaf640 1 -- 172.21.15.83:0/4159619096 shutdown_connections 2024-04-17T10:54:53.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.175+0000 7f8f9aeaf640 1 -- 172.21.15.83:0/4159619096 wait complete. 2024-04-17T10:54:53.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.175+0000 7f8f9aeaf640 1 Processor -- start 2024-04-17T10:54:53.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.175+0000 7f8f9aeaf640 1 -- start start 2024-04-17T10:54:53.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.175+0000 7f8f9aeaf640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f940b5450 0x7f8f940c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.175+0000 7f8f9aeaf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f940ce400 0x7f8f940cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.175+0000 7f8f9aeaf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f9415e0f0 0x7f8f940cc350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.175+0000 7f8f9aeaf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f941622d0 con 0x7f8f9415e0f0 2024-04-17T10:54:53.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.175+0000 7f8f9aeaf640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f94162150 con 0x7f8f940ce400 2024-04-17T10:54:53.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.175+0000 7f8f9aeaf640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9416af50 con 0x7f8f940b5450 2024-04-17T10:54:53.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.175+0000 7f8f9944c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f9415e0f0 0x7f8f940cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.175+0000 7f8f9944c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f9415e0f0 0x7f8f940cc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44464/0 (socket says 172.21.15.83:44464) 2024-04-17T10:54:53.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.175+0000 7f8f9944c640 1 -- 172.21.15.83:0/4096567766 learned_addr learned my addr 172.21.15.83:0/4096567766 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:53.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.175+0000 7f8f93fff640 1 --2- 172.21.15.83:0/4096567766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f940ce400 0x7f8f940cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.175+0000 7f8f98c4b640 1 --2- 172.21.15.83:0/4096567766 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f940b5450 0x7f8f940c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.175+0000 7f8f9944c640 1 -- 172.21.15.83:0/4096567766 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f940b5450 msgr2=0x7f8f940c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:53.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.175+0000 7f8f9944c640 1 --2- 172.21.15.83:0/4096567766 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f940b5450 0x7f8f940c9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.175+0000 7f8f9944c640 1 -- 172.21.15.83:0/4096567766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f940ce400 msgr2=0x7f8f940cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:53.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.175+0000 7f8f9944c640 1 --2- 172.21.15.83:0/4096567766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f940ce400 0x7f8f940cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.175+0000 7f8f9944c640 1 -- 172.21.15.83:0/4096567766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f940cab40 con 0x7f8f9415e0f0 2024-04-17T10:54:53.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.175+0000 7f8f9944c640 1 --2- 172.21.15.83:0/4096567766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f9415e0f0 0x7f8f940cc350 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c00b9c0 tx=0x7f8f8c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:53.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.175+0000 7f8f917fa640 1 -- 172.21.15.83:0/4096567766 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c019070 con 0x7f8f9415e0f0 2024-04-17T10:54:53.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.175+0000 7f8f917fa640 1 -- 172.21.15.83:0/4096567766 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8c00cba0 con 0x7f8f9415e0f0 2024-04-17T10:54:53.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.175+0000 7f8f9aeaf640 1 -- 172.21.15.83:0/4096567766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f940cae20 con 0x7f8f9415e0f0 2024-04-17T10:54:53.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.175+0000 7f8f917fa640 1 -- 172.21.15.83:0/4096567766 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c014440 con 0x7f8f9415e0f0 2024-04-17T10:54:53.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.175+0000 7f8f9aeaf640 1 -- 172.21.15.83:0/4096567766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f940cb3e0 con 0x7f8f9415e0f0 2024-04-17T10:54:53.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.179+0000 7f8f9aeaf640 1 -- 172.21.15.83:0/4096567766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f940681a0 con 0x7f8f9415e0f0 2024-04-17T10:54:53.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.179+0000 7f8f917fa640 1 -- 172.21.15.83:0/4096567766 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8f8c01c070 con 0x7f8f9415e0f0 2024-04-17T10:54:53.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.179+0000 7f8f917fa640 1 --2- 172.21.15.83:0/4096567766 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8f80066f60 0x7f8f80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.179+0000 7f8f98c4b640 1 --2- 172.21.15.83:0/4096567766 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8f80066f60 0x7f8f80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.179+0000 7f8f917fa640 1 -- 172.21.15.83:0/4096567766 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8f8c084a90 con 0x7f8f9415e0f0 2024-04-17T10:54:53.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.179+0000 7f8f98c4b640 1 --2- 172.21.15.83:0/4096567766 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8f80066f60 0x7f8f80069420 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f8f840ae000 tx=0x7f8f840a6070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:53.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.183+0000 7f8f917fa640 1 -- 172.21.15.83:0/4096567766 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8f8c055da0 con 0x7f8f9415e0f0 2024-04-17T10:54:53.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.335+0000 7f8f9aeaf640 1 -- 172.21.15.83:0/4096567766 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f94074f90 con 0x7f8f80066f60 2024-04-17T10:54:53.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.335+0000 7f8f917fa640 1 -- 172.21.15.83:0/4096567766 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8f94074f90 con 0x7f8f80066f60 2024-04-17T10:54:53.331 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 2 to deep-scrub 2024-04-17T10:54:53.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.339+0000 7f8f9aeaf640 1 -- 172.21.15.83:0/4096567766 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8f80066f60 msgr2=0x7f8f80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:53.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.339+0000 7f8f9aeaf640 1 --2- 172.21.15.83:0/4096567766 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8f80066f60 0x7f8f80069420 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f8f840ae000 tx=0x7f8f840a6070 comp rx=0 tx=0).stop 2024-04-17T10:54:53.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.339+0000 7f8f9aeaf640 1 -- 172.21.15.83:0/4096567766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f9415e0f0 msgr2=0x7f8f940cc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:53.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.339+0000 7f8f9aeaf640 1 --2- 172.21.15.83:0/4096567766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f9415e0f0 0x7f8f940cc350 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c00b9c0 tx=0x7f8f8c00be80 comp rx=0 tx=0).stop 2024-04-17T10:54:53.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.339+0000 7f8f9aeaf640 1 -- 172.21.15.83:0/4096567766 shutdown_connections 2024-04-17T10:54:53.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.339+0000 7f8f9aeaf640 1 --2- 172.21.15.83:0/4096567766 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8f80066f60 0x7f8f80069420 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.339+0000 7f8f9aeaf640 1 --2- 172.21.15.83:0/4096567766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f9415e0f0 0x7f8f940cc350 unknown :-1 s=CLOSED pgs=2487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.339+0000 7f8f9aeaf640 1 --2- 172.21.15.83:0/4096567766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f940ce400 0x7f8f940cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.339+0000 7f8f9aeaf640 1 --2- 172.21.15.83:0/4096567766 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f940b5450 0x7f8f940c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.339+0000 7f8f9aeaf640 1 -- 172.21.15.83:0/4096567766 >> 172.21.15.83:0/4096567766 conn(0x7f8f940d0450 msgr2=0x7f8f9415ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:53.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.339+0000 7f8f9aeaf640 1 -- 172.21.15.83:0/4096567766 shutdown_connections 2024-04-17T10:54:53.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.339+0000 7f8f9aeaf640 1 -- 172.21.15.83:0/4096567766 wait complete. 2024-04-17T10:54:53.348 INFO:tasks.ceph:Scrubbing osd.3 2024-04-17T10:54:53.348 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-17T10:54:53.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.467+0000 7f62bf3c5640 1 Processor -- start 2024-04-17T10:54:53.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.467+0000 7f62bf3c5640 1 -- start start 2024-04-17T10:54:53.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.467+0000 7f62bf3c5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f62b815b760 0x7f62b815eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.467+0000 7f62bf3c5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f62b8161740 0x7f62b806abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.467+0000 7f62bf3c5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62b806b120 0x7f62b80ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.467+0000 7f62bf3c5640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62b806b590 con 0x7f62b8161740 2024-04-17T10:54:53.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.467+0000 7f62bf3c5640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62b80ad840 con 0x7f62b815b760 2024-04-17T10:54:53.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.467+0000 7f62bf3c5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62b80b2600 con 0x7f62b806b120 2024-04-17T10:54:53.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.467+0000 7f62bc960640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f62b8161740 0x7f62b806abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.467+0000 7f62bc960640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f62b8161740 0x7f62b806abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44478/0 (socket says 172.21.15.83:44478) 2024-04-17T10:54:53.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.467+0000 7f62bc960640 1 -- 172.21.15.83:0/718218853 learned_addr learned my addr 172.21.15.83:0/718218853 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:53.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.467+0000 7f62bc960640 1 -- 172.21.15.83:0/718218853 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62b806b120 msgr2=0x7f62b80ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:53.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.467+0000 7f62bc960640 1 --2- 172.21.15.83:0/718218853 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62b806b120 0x7f62b80ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.467+0000 7f62bc960640 1 -- 172.21.15.83:0/718218853 >> v2:172.21.15.167:3300/0 conn(0x7f62b815b760 msgr2=0x7f62b815eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:53.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.467+0000 7f62bd161640 1 --2- 172.21.15.83:0/718218853 >> v2:172.21.15.167:3300/0 conn(0x7f62b815b760 0x7f62b815eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.467+0000 7f62bc960640 1 --2- 172.21.15.83:0/718218853 >> v2:172.21.15.167:3300/0 conn(0x7f62b815b760 0x7f62b815eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.467+0000 7f62bc960640 1 -- 172.21.15.83:0/718218853 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b815f0d0 con 0x7f62b8161740 2024-04-17T10:54:53.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.471+0000 7f62bc960640 1 --2- 172.21.15.83:0/718218853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b8161740 0x7f62b806abe0 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7f62b8072c80 tx=0x7f62a801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fdaccebcc1cb49b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:53.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.471+0000 7f62affff640 1 -- 172.21.15.83:0/718218853 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a80b1070 con 0x7f62b8161740 2024-04-17T10:54:53.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.471+0000 7f62affff640 1 -- 172.21.15.83:0/718218853 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a80ac630 con 0x7f62b8161740 2024-04-17T10:54:53.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.471+0000 7f62affff640 1 -- 172.21.15.83:0/718218853 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a80a8050 con 0x7f62b8161740 2024-04-17T10:54:53.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.471+0000 7f62bf3c5640 1 -- 172.21.15.83:0/718218853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b8161740 msgr2=0x7f62b806abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:53.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.471+0000 7f62bf3c5640 1 --2- 172.21.15.83:0/718218853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b8161740 0x7f62b806abe0 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7f62b8072c80 tx=0x7f62a801c520 comp rx=0 tx=0).stop 2024-04-17T10:54:53.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.471+0000 7f62bf3c5640 1 -- 172.21.15.83:0/718218853 shutdown_connections 2024-04-17T10:54:53.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.471+0000 7f62bf3c5640 1 --2- 172.21.15.83:0/718218853 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62b806b120 0x7f62b80ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.471+0000 7f62bf3c5640 1 --2- 172.21.15.83:0/718218853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b8161740 0x7f62b806abe0 unknown :-1 s=CLOSED pgs=2488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.471+0000 7f62bf3c5640 1 --2- 172.21.15.83:0/718218853 >> v2:172.21.15.167:3300/0 conn(0x7f62b815b760 0x7f62b815eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.471+0000 7f62bf3c5640 1 -- 172.21.15.83:0/718218853 >> 172.21.15.83:0/718218853 conn(0x7f62b80cc050 msgr2=0x7f62b80cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:53.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.471+0000 7f62bf3c5640 1 -- 172.21.15.83:0/718218853 shutdown_connections 2024-04-17T10:54:53.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.471+0000 7f62bf3c5640 1 -- 172.21.15.83:0/718218853 wait complete. 2024-04-17T10:54:53.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62bf3c5640 1 Processor -- start 2024-04-17T10:54:53.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62bf3c5640 1 -- start start 2024-04-17T10:54:53.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62bf3c5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62b806b120 0x7f62b8144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62bf3c5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b815b760 0x7f62b806c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62bf3c5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62b8161740 0x7f62b806d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62bf3c5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b8162510 con 0x7f62b815b760 2024-04-17T10:54:53.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62bf3c5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b8162230 con 0x7f62b806b120 2024-04-17T10:54:53.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62bf3c5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62b80b2a90 con 0x7f62b8161740 2024-04-17T10:54:53.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62bc960640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b815b760 0x7f62b806c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62bc960640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b815b760 0x7f62b806c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44494/0 (socket says 172.21.15.83:44494) 2024-04-17T10:54:53.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62bc960640 1 -- 172.21.15.83:0/631317840 learned_addr learned my addr 172.21.15.83:0/631317840 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:53.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62bd161640 1 --2- 172.21.15.83:0/631317840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62b806b120 0x7f62b8144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62bc960640 1 -- 172.21.15.83:0/631317840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62b8161740 msgr2=0x7f62b806d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:53.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62bd962640 1 --2- 172.21.15.83:0/631317840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62b8161740 0x7f62b806d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62bc960640 1 --2- 172.21.15.83:0/631317840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62b8161740 0x7f62b806d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62bc960640 1 -- 172.21.15.83:0/631317840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62b806b120 msgr2=0x7f62b8144bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:53.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62bc960640 1 --2- 172.21.15.83:0/631317840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62b806b120 0x7f62b8144bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62bc960640 1 -- 172.21.15.83:0/631317840 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b80704f0 con 0x7f62b815b760 2024-04-17T10:54:53.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62bd161640 1 --2- 172.21.15.83:0/631317840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62b806b120 0x7f62b8144bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:53.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62bc960640 1 --2- 172.21.15.83:0/631317840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b815b760 0x7f62b806c670 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7f62a801abd0 tx=0x7f62a80192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:53.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62adffb640 1 -- 172.21.15.83:0/631317840 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a80bc070 con 0x7f62b815b760 2024-04-17T10:54:53.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62adffb640 1 -- 172.21.15.83:0/631317840 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a80ab800 con 0x7f62b815b760 2024-04-17T10:54:53.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62bf3c5640 1 -- 172.21.15.83:0/631317840 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62b81506f0 con 0x7f62b815b760 2024-04-17T10:54:53.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62adffb640 1 -- 172.21.15.83:0/631317840 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a80b1070 con 0x7f62b815b760 2024-04-17T10:54:53.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62bf3c5640 1 -- 172.21.15.83:0/631317840 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b8150cb0 con 0x7f62b815b760 2024-04-17T10:54:53.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62bf3c5640 1 -- 172.21.15.83:0/631317840 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f62b8075000 con 0x7f62b815b760 2024-04-17T10:54:53.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62adffb640 1 -- 172.21.15.83:0/631317840 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f62a80bf070 con 0x7f62b815b760 2024-04-17T10:54:53.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62adffb640 1 --2- 172.21.15.83:0/631317840 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f629c066f60 0x7f629c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62adffb640 1 -- 172.21.15.83:0/631317840 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f62a811edc0 con 0x7f62b815b760 2024-04-17T10:54:53.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62bd161640 1 --2- 172.21.15.83:0/631317840 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f629c066f60 0x7f629c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62adffb640 1 --2- 172.21.15.83:0/631317840 >> [v2:172.21.15.83:6800/367778918,v1:172.21.15.83:6801/367778918] conn(0x7f629c06ce20 0x7f629c06f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62adffb640 1 -- 172.21.15.83:0/631317840 --> [v2:172.21.15.83:6800/367778918,v1:172.21.15.83:6801/367778918] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f62a80aee90 con 0x7f629c06ce20 2024-04-17T10:54:53.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62adffb640 1 -- 172.21.15.83:0/631317840 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=145) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62a811f250 con 0x7f62b815b760 2024-04-17T10:54:53.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62bd962640 1 --2- 172.21.15.83:0/631317840 >> [v2:172.21.15.83:6800/367778918,v1:172.21.15.83:6801/367778918] conn(0x7f629c06ce20 0x7f629c06f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.475+0000 7f62bd161640 1 --2- 172.21.15.83:0/631317840 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f629c066f60 0x7f629c069420 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f62b4006fd0 tx=0x7f62b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:53.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.479+0000 7f62bd962640 1 --2- 172.21.15.83:0/631317840 >> [v2:172.21.15.83:6800/367778918,v1:172.21.15.83:6801/367778918] conn(0x7f629c06ce20 0x7f629c06f240 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:53.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.479+0000 7f62adffb640 1 -- 172.21.15.83:0/631317840 <== osd.3 v2:172.21.15.83:6800/367778918 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f62a80aee90 con 0x7f629c06ce20 2024-04-17T10:54:53.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.491+0000 7f62bf3c5640 1 -- 172.21.15.83:0/631317840 --> [v2:172.21.15.83:6800/367778918,v1:172.21.15.83:6801/367778918] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f62b806b9c0 con 0x7f629c06ce20 2024-04-17T10:54:53.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.491+0000 7f62adffb640 1 -- 172.21.15.83:0/631317840 <== osd.3 v2:172.21.15.83:6800/367778918 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f62b806b9c0 con 0x7f629c06ce20 2024-04-17T10:54:53.485 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:54:53.485 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-17T10:54:53.485 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:54:53.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.491+0000 7f62bf3c5640 1 -- 172.21.15.83:0/631317840 >> [v2:172.21.15.83:6800/367778918,v1:172.21.15.83:6801/367778918] conn(0x7f629c06ce20 msgr2=0x7f629c06f240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:53.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.491+0000 7f62bf3c5640 1 --2- 172.21.15.83:0/631317840 >> [v2:172.21.15.83:6800/367778918,v1:172.21.15.83:6801/367778918] conn(0x7f629c06ce20 0x7f629c06f240 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.491+0000 7f62bf3c5640 1 -- 172.21.15.83:0/631317840 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f629c066f60 msgr2=0x7f629c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:53.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.491+0000 7f62bf3c5640 1 --2- 172.21.15.83:0/631317840 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f629c066f60 0x7f629c069420 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f62b4006fd0 tx=0x7f62b4008040 comp rx=0 tx=0).stop 2024-04-17T10:54:53.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.491+0000 7f62bf3c5640 1 -- 172.21.15.83:0/631317840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b815b760 msgr2=0x7f62b806c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:53.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.491+0000 7f62bf3c5640 1 --2- 172.21.15.83:0/631317840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b815b760 0x7f62b806c670 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7f62a801abd0 tx=0x7f62a80192c0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.491+0000 7f62bd962640 1 -- 172.21.15.83:0/631317840 reap_dead start 2024-04-17T10:54:53.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.491+0000 7f62bf3c5640 1 -- 172.21.15.83:0/631317840 shutdown_connections 2024-04-17T10:54:53.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.491+0000 7f62bf3c5640 1 -- 172.21.15.83:0/631317840 >> 172.21.15.83:0/631317840 conn(0x7f62b80cc050 msgr2=0x7f62b80ae0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:53.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.491+0000 7f62bf3c5640 1 -- 172.21.15.83:0/631317840 shutdown_connections 2024-04-17T10:54:53.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.491+0000 7f62bf3c5640 1 -- 172.21.15.83:0/631317840 wait complete. 2024-04-17T10:54:53.500 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-17T10:54:53.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.619+0000 7f37eb617640 1 Processor -- start 2024-04-17T10:54:53.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.619+0000 7f37eb617640 1 -- start start 2024-04-17T10:54:53.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.619+0000 7f37eb617640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37e415b6e0 0x7f37e4071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.619+0000 7f37eb617640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f37e415bac0 0x7f37e40cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.619+0000 7f37eb617640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f37e4070470 0x7f37e406ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.619+0000 7f37eb617640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37e406c1f0 con 0x7f37e4070470 2024-04-17T10:54:53.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.619+0000 7f37eb617640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37e40cdd30 con 0x7f37e415bac0 2024-04-17T10:54:53.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.619+0000 7f37eb617640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e4070a10 con 0x7f37e415b6e0 2024-04-17T10:54:53.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.619+0000 7f37e9bb4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f37e4070470 0x7f37e406ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.619+0000 7f37e9bb4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f37e4070470 0x7f37e406ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44506/0 (socket says 172.21.15.83:44506) 2024-04-17T10:54:53.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.619+0000 7f37e9bb4640 1 -- 172.21.15.83:0/14362530 learned_addr learned my addr 172.21.15.83:0/14362530 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:53.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.619+0000 7f37e93b3640 1 --2- 172.21.15.83:0/14362530 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37e415b6e0 0x7f37e4071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.619+0000 7f37e8bb2640 1 --2- 172.21.15.83:0/14362530 >> v2:172.21.15.167:3300/0 conn(0x7f37e415bac0 0x7f37e40cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.619+0000 7f37e9bb4640 1 -- 172.21.15.83:0/14362530 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37e415b6e0 msgr2=0x7f37e4071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:53.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.619+0000 7f37e9bb4640 1 --2- 172.21.15.83:0/14362530 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37e415b6e0 0x7f37e4071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.619+0000 7f37e9bb4640 1 -- 172.21.15.83:0/14362530 >> v2:172.21.15.167:3300/0 conn(0x7f37e415bac0 msgr2=0x7f37e40cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:53.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.619+0000 7f37e9bb4640 1 --2- 172.21.15.83:0/14362530 >> v2:172.21.15.167:3300/0 conn(0x7f37e415bac0 0x7f37e40cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.619+0000 7f37e9bb4640 1 -- 172.21.15.83:0/14362530 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e4070cf0 con 0x7f37e4070470 2024-04-17T10:54:53.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.623+0000 7f37e9bb4640 1 --2- 172.21.15.83:0/14362530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37e4070470 0x7f37e406ae10 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7f37e40b49f0 tx=0x7f37dc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c89bd3bbba8d375 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:53.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.623+0000 7f37dbfff640 1 -- 172.21.15.83:0/14362530 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc0a5670 con 0x7f37e4070470 2024-04-17T10:54:53.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.623+0000 7f37dbfff640 1 -- 172.21.15.83:0/14362530 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37dc0a5810 con 0x7f37e4070470 2024-04-17T10:54:53.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.623+0000 7f37dbfff640 1 -- 172.21.15.83:0/14362530 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc0a5be0 con 0x7f37e4070470 2024-04-17T10:54:53.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.623+0000 7f37eb617640 1 -- 172.21.15.83:0/14362530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37e4070470 msgr2=0x7f37e406ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:53.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.623+0000 7f37eb617640 1 --2- 172.21.15.83:0/14362530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37e4070470 0x7f37e406ae10 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7f37e40b49f0 tx=0x7f37dc01c9f0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.623+0000 7f37eb617640 1 -- 172.21.15.83:0/14362530 shutdown_connections 2024-04-17T10:54:53.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.623+0000 7f37eb617640 1 --2- 172.21.15.83:0/14362530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37e4070470 0x7f37e406ae10 unknown :-1 s=CLOSED pgs=2490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.623+0000 7f37eb617640 1 --2- 172.21.15.83:0/14362530 >> v2:172.21.15.167:3300/0 conn(0x7f37e415bac0 0x7f37e40cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.623+0000 7f37eb617640 1 --2- 172.21.15.83:0/14362530 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37e415b6e0 0x7f37e4071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.623+0000 7f37eb617640 1 -- 172.21.15.83:0/14362530 >> 172.21.15.83:0/14362530 conn(0x7f37e40b15b0 msgr2=0x7f37e40b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:53.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.623+0000 7f37eb617640 1 -- 172.21.15.83:0/14362530 shutdown_connections 2024-04-17T10:54:53.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.623+0000 7f37eb617640 1 -- 172.21.15.83:0/14362530 wait complete. 2024-04-17T10:54:53.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.623+0000 7f37eb617640 1 Processor -- start 2024-04-17T10:54:53.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.623+0000 7f37eb617640 1 -- start start 2024-04-17T10:54:53.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.623+0000 7f37eb617640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37e4070470 0x7f37e4149fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.623+0000 7f37eb617640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37e415b6e0 0x7f37e40b2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.623+0000 7f37eb617640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37e415bac0 0x7f37e40ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.623+0000 7f37eb617640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e406d270 con 0x7f37e415bac0 2024-04-17T10:54:53.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.623+0000 7f37eb617640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e406d0f0 con 0x7f37e415b6e0 2024-04-17T10:54:53.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.623+0000 7f37eb617640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e406b760 con 0x7f37e4070470 2024-04-17T10:54:53.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.623+0000 7f37e9bb4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37e415bac0 0x7f37e40ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.623+0000 7f37e9bb4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37e415bac0 0x7f37e40ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44516/0 (socket says 172.21.15.83:44516) 2024-04-17T10:54:53.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.623+0000 7f37e9bb4640 1 -- 172.21.15.83:0/2808345828 learned_addr learned my addr 172.21.15.83:0/2808345828 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:53.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.623+0000 7f37e8bb2640 1 --2- 172.21.15.83:0/2808345828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37e415b6e0 0x7f37e40b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.623+0000 7f37e93b3640 1 --2- 172.21.15.83:0/2808345828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37e4070470 0x7f37e4149fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.623+0000 7f37e9bb4640 1 -- 172.21.15.83:0/2808345828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37e4070470 msgr2=0x7f37e4149fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:53.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.623+0000 7f37e9bb4640 1 --2- 172.21.15.83:0/2808345828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37e4070470 0x7f37e4149fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.623+0000 7f37e9bb4640 1 -- 172.21.15.83:0/2808345828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37e415b6e0 msgr2=0x7f37e40b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.623+0000 7f37e9bb4640 1 --2- 172.21.15.83:0/2808345828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37e415b6e0 0x7f37e40b2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.623+0000 7f37e9bb4640 1 -- 172.21.15.83:0/2808345828 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e41fa090 con 0x7f37e415bac0 2024-04-17T10:54:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.623+0000 7f37e8bb2640 1 --2- 172.21.15.83:0/2808345828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37e415b6e0 0x7f37e40b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:54:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.627+0000 7f37e9bb4640 1 --2- 172.21.15.83:0/2808345828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37e415bac0 0x7f37e40ad840 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0af040 tx=0x7f37dc019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.627+0000 7f37d9ffb640 1 -- 172.21.15.83:0/2808345828 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc0bc020 con 0x7f37e415bac0 2024-04-17T10:54:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.627+0000 7f37d9ffb640 1 -- 172.21.15.83:0/2808345828 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37dc0b8690 con 0x7f37e415bac0 2024-04-17T10:54:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.627+0000 7f37d9ffb640 1 -- 172.21.15.83:0/2808345828 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc009510 con 0x7f37e415bac0 2024-04-17T10:54:53.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.627+0000 7f37eb617640 1 -- 172.21.15.83:0/2808345828 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37e41fa370 con 0x7f37e415bac0 2024-04-17T10:54:53.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.627+0000 7f37eb617640 1 -- 172.21.15.83:0/2808345828 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37e41fa930 con 0x7f37e415bac0 2024-04-17T10:54:53.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.627+0000 7f37eb617640 1 -- 172.21.15.83:0/2808345828 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37e4074f90 con 0x7f37e415bac0 2024-04-17T10:54:53.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.627+0000 7f37d9ffb640 1 -- 172.21.15.83:0/2808345828 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f37dc0bf020 con 0x7f37e415bac0 2024-04-17T10:54:53.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.627+0000 7f37d9ffb640 1 --2- 172.21.15.83:0/2808345828 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f37c8066f60 0x7f37c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.627+0000 7f37e93b3640 1 --2- 172.21.15.83:0/2808345828 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f37c8066f60 0x7f37c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.627+0000 7f37d9ffb640 1 -- 172.21.15.83:0/2808345828 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f37dc11fc20 con 0x7f37e415bac0 2024-04-17T10:54:53.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.627+0000 7f37e93b3640 1 --2- 172.21.15.83:0/2808345828 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f37c8066f60 0x7f37c8069420 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f37e414a880 tx=0x7f37e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:53.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.631+0000 7f37d9ffb640 1 -- 172.21.15.83:0/2808345828 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f37dc0c6030 con 0x7f37e415bac0 2024-04-17T10:54:53.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.779+0000 7f37eb617640 1 -- 172.21.15.83:0/2808345828 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f37e40b1b10 con 0x7f37c8066f60 2024-04-17T10:54:53.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.783+0000 7f37d9ffb640 1 -- 172.21.15.83:0/2808345828 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f37e40b1b10 con 0x7f37c8066f60 2024-04-17T10:54:53.775 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 3 to deep-scrub 2024-04-17T10:54:53.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.783+0000 7f37eb617640 1 -- 172.21.15.83:0/2808345828 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f37c8066f60 msgr2=0x7f37c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:53.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.783+0000 7f37eb617640 1 --2- 172.21.15.83:0/2808345828 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f37c8066f60 0x7f37c8069420 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f37e414a880 tx=0x7f37e0008040 comp rx=0 tx=0).stop 2024-04-17T10:54:53.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.783+0000 7f37eb617640 1 -- 172.21.15.83:0/2808345828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37e415bac0 msgr2=0x7f37e40ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:53.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.783+0000 7f37eb617640 1 --2- 172.21.15.83:0/2808345828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37e415bac0 0x7f37e40ad840 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0af040 tx=0x7f37dc019790 comp rx=0 tx=0).stop 2024-04-17T10:54:53.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.783+0000 7f37eb617640 1 -- 172.21.15.83:0/2808345828 shutdown_connections 2024-04-17T10:54:53.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.783+0000 7f37eb617640 1 --2- 172.21.15.83:0/2808345828 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f37c8066f60 0x7f37c8069420 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.783+0000 7f37eb617640 1 --2- 172.21.15.83:0/2808345828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37e415bac0 0x7f37e40ad840 unknown :-1 s=CLOSED pgs=2491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.783+0000 7f37eb617640 1 --2- 172.21.15.83:0/2808345828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37e415b6e0 0x7f37e40b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.783+0000 7f37eb617640 1 --2- 172.21.15.83:0/2808345828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37e4070470 0x7f37e4149fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.783+0000 7f37eb617640 1 -- 172.21.15.83:0/2808345828 >> 172.21.15.83:0/2808345828 conn(0x7f37e40b15b0 msgr2=0x7f37e40b0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:53.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.783+0000 7f37eb617640 1 -- 172.21.15.83:0/2808345828 shutdown_connections 2024-04-17T10:54:53.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.783+0000 7f37eb617640 1 -- 172.21.15.83:0/2808345828 wait complete. 2024-04-17T10:54:53.793 INFO:tasks.ceph:Scrubbing osd.4 2024-04-17T10:54:53.794 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-17T10:54:53.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.919+0000 7f0b7d0d3640 1 Processor -- start 2024-04-17T10:54:53.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.919+0000 7f0b7d0d3640 1 -- start start 2024-04-17T10:54:53.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.919+0000 7f0b7d0d3640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b78161740 0x7f0b7806abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.919+0000 7f0b7d0d3640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b7815bb40 con 0x7f0b7806b120 2024-04-17T10:54:53.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.919+0000 7f0b7d0d3640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b78161c70 con 0x7f0b7815b760 2024-04-17T10:54:53.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.919+0000 7f0b7d0d3640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7806b590 con 0x7f0b78161740 2024-04-17T10:54:53.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.919+0000 7f0b7759e640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0b7806b120 0x7f0b780b2a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42322/0 (socket says 172.21.15.83:42322) 2024-04-17T10:54:53.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.919+0000 7f0b7759e640 1 -- 172.21.15.83:0/2314388147 learned_addr learned my addr 172.21.15.83:0/2314388147 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:53.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.919+0000 7f0b75d9b640 1 -- 172.21.15.83:0/2314388147 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1325453000 0 0) 0x7f0b7815bb40 con 0x7f0b7806b120 2024-04-17T10:54:53.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.919+0000 7f0b75d9b640 1 -- 172.21.15.83:0/2314388147 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b60003530 con 0x7f0b7806b120 2024-04-17T10:54:53.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.919+0000 7f0b7659c640 1 --2- 172.21.15.83:0/2314388147 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b78161740 0x7f0b7806abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.919+0000 7f0b75d9b640 1 -- 172.21.15.83:0/2314388147 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2710715775 0 0) 0x7f0b60003530 con 0x7f0b7806b120 2024-04-17T10:54:53.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.919+0000 7f0b75d9b640 1 -- 172.21.15.83:0/2314388147 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b78161740 msgr2=0x7f0b7806abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:53.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.919+0000 7f0b75d9b640 1 --2- 172.21.15.83:0/2314388147 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b78161740 0x7f0b7806abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.919+0000 7f0b75d9b640 1 -- 172.21.15.83:0/2314388147 >> v1:172.21.15.167:6789/0 conn(0x7f0b7815b760 legacy=0x7f0b7815eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:53.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.919+0000 7f0b75d9b640 1 -- 172.21.15.83:0/2314388147 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b78161f50 con 0x7f0b7806b120 2024-04-17T10:54:53.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.919+0000 7f0b75d9b640 1 -- 172.21.15.83:0/2314388147 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f0b70001a00 con 0x7f0b7806b120 2024-04-17T10:54:53.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.919+0000 7f0b75d9b640 1 -- 172.21.15.83:0/2314388147 >> v1:172.21.15.83:6789/0 conn(0x7f0b7806b120 legacy=0x7f0b780b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:53.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.919+0000 7f0b75d9b640 1 --2- 172.21.15.83:0/2314388147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b600039f0 0x7f0b60003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.919+0000 7f0b75d9b640 1 --2- 172.21.15.83:0/2314388147 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b78161740 0x7f0b60004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.919+0000 7f0b75d9b640 1 --2- 172.21.15.83:0/2314388147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b60004d50 0x7f0b600051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.919+0000 7f0b75d9b640 1 -- 172.21.15.83:0/2314388147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7806b590 con 0x7f0b60004d50 2024-04-17T10:54:53.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.919+0000 7f0b75d9b640 1 -- 172.21.15.83:0/2314388147 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6000bb60 con 0x7f0b600039f0 2024-04-17T10:54:53.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.919+0000 7f0b75d9b640 1 -- 172.21.15.83:0/2314388147 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6000bd60 con 0x7f0b78161740 2024-04-17T10:54:53.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.919+0000 7f0b7759e640 1 --2- 172.21.15.83:0/2314388147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b60004d50 0x7f0b600051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.919+0000 7f0b76d9d640 1 --2- 172.21.15.83:0/2314388147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b600039f0 0x7f0b60003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.919+0000 7f0b7659c640 1 --2- 172.21.15.83:0/2314388147 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b78161740 0x7f0b60004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.923+0000 7f0b7759e640 1 -- 172.21.15.83:0/2314388147 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b78161740 msgr2=0x7f0b60004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:53.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.923+0000 7f0b7759e640 1 --2- 172.21.15.83:0/2314388147 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b78161740 0x7f0b60004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.923+0000 7f0b7759e640 1 -- 172.21.15.83:0/2314388147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b600039f0 msgr2=0x7f0b60003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:53.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.923+0000 7f0b7759e640 1 --2- 172.21.15.83:0/2314388147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b600039f0 0x7f0b60003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.923+0000 7f0b7759e640 1 -- 172.21.15.83:0/2314388147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b60003530 con 0x7f0b60004d50 2024-04-17T10:54:53.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.923+0000 7f0b7759e640 1 --2- 172.21.15.83:0/2314388147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b60004d50 0x7f0b600051d0 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7f0b7000a850 tx=0x7f0b7001d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b003ac3ec52106 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:53.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.923+0000 7f0b75d9b640 1 -- 172.21.15.83:0/2314388147 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7000a4a0 con 0x7f0b60004d50 2024-04-17T10:54:53.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.923+0000 7f0b75d9b640 1 -- 172.21.15.83:0/2314388147 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b700ad650 con 0x7f0b60004d50 2024-04-17T10:54:53.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.923+0000 7f0b75d9b640 1 -- 172.21.15.83:0/2314388147 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b700ada40 con 0x7f0b60004d50 2024-04-17T10:54:53.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.923+0000 7f0b7d0d3640 1 -- 172.21.15.83:0/2314388147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b60004d50 msgr2=0x7f0b600051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:53.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.923+0000 7f0b7d0d3640 1 --2- 172.21.15.83:0/2314388147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b60004d50 0x7f0b600051d0 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7f0b7000a850 tx=0x7f0b7001d3a0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.923+0000 7f0b7759e640 1 -- 172.21.15.83:0/2314388147 reap_dead start 2024-04-17T10:54:53.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.923+0000 7f0b7d0d3640 1 -- 172.21.15.83:0/2314388147 shutdown_connections 2024-04-17T10:54:53.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.923+0000 7f0b7d0d3640 1 -- 172.21.15.83:0/2314388147 >> 172.21.15.83:0/2314388147 conn(0x7f0b780cc050 msgr2=0x7f0b780cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:53.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.923+0000 7f0b7d0d3640 1 -- 172.21.15.83:0/2314388147 shutdown_connections 2024-04-17T10:54:53.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.923+0000 7f0b7d0d3640 1 -- 172.21.15.83:0/2314388147 wait complete. 2024-04-17T10:54:53.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b7d0d3640 1 Processor -- start 2024-04-17T10:54:53.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b7d0d3640 1 -- start start 2024-04-17T10:54:53.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b7d0d3640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b7814a170 0x7f0b78070ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b7d0d3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b781e75a0 0x7f0b7815cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b7d0d3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b7815d300 0x7f0b781e8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b7d0d3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b780b2600 con 0x7f0b7815d300 2024-04-17T10:54:53.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b7d0d3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b78162510 con 0x7f0b781e75a0 2024-04-17T10:54:53.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b7d0d3640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7806abe0 con 0x7f0b7814a170 2024-04-17T10:54:53.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b7759e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b7815d300 0x7f0b781e8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b7759e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b7815d300 0x7f0b781e8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44536/0 (socket says 172.21.15.83:44536) 2024-04-17T10:54:53.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b7759e640 1 -- 172.21.15.83:0/2548295119 learned_addr learned my addr 172.21.15.83:0/2548295119 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:53.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b76d9d640 1 --2- 172.21.15.83:0/2548295119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b7814a170 0x7f0b78070ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b7659c640 1 --2- 172.21.15.83:0/2548295119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b781e75a0 0x7f0b7815cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b7759e640 1 -- 172.21.15.83:0/2548295119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b7814a170 msgr2=0x7f0b78070ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:53.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b7759e640 1 --2- 172.21.15.83:0/2548295119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b7814a170 0x7f0b78070ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b7759e640 1 -- 172.21.15.83:0/2548295119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b781e75a0 msgr2=0x7f0b7815cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:53.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b7759e640 1 --2- 172.21.15.83:0/2548295119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b781e75a0 0x7f0b7815cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b7759e640 1 -- 172.21.15.83:0/2548295119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b781e85b0 con 0x7f0b7815d300 2024-04-17T10:54:53.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b7659c640 1 --2- 172.21.15.83:0/2548295119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b781e75a0 0x7f0b7815cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:53.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b7759e640 1 --2- 172.21.15.83:0/2548295119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b7815d300 0x7f0b781e8d30 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7f0b700a6010 tx=0x7f0b60005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:53.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b5f7fe640 1 -- 172.21.15.83:0/2548295119 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7000a480 con 0x7f0b7815d300 2024-04-17T10:54:53.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b5f7fe640 1 -- 172.21.15.83:0/2548295119 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b700bb070 con 0x7f0b7815d300 2024-04-17T10:54:53.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b7d0d3640 1 -- 172.21.15.83:0/2548295119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b781e7d10 con 0x7f0b7815d300 2024-04-17T10:54:53.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b5f7fe640 1 -- 172.21.15.83:0/2548295119 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b700011d0 con 0x7f0b7815d300 2024-04-17T10:54:53.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b7d0d3640 1 -- 172.21.15.83:0/2548295119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b781e82d0 con 0x7f0b7815d300 2024-04-17T10:54:53.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b7d0d3640 1 -- 172.21.15.83:0/2548295119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0b78075000 con 0x7f0b7815d300 2024-04-17T10:54:53.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b5f7fe640 1 -- 172.21.15.83:0/2548295119 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0b700b5070 con 0x7f0b7815d300 2024-04-17T10:54:53.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b5f7fe640 1 --2- 172.21.15.83:0/2548295119 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0b64066f60 0x7f0b64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b5f7fe640 1 -- 172.21.15.83:0/2548295119 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0b7011f9d0 con 0x7f0b7815d300 2024-04-17T10:54:53.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b76d9d640 1 --2- 172.21.15.83:0/2548295119 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0b64066f60 0x7f0b64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b5f7fe640 1 --2- 172.21.15.83:0/2548295119 >> [v2:172.21.15.167:6813/77470876,v1:172.21.15.167:6817/77470876] conn(0x7f0b6406ce20 0x7f0b6406f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:53.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b5f7fe640 1 -- 172.21.15.83:0/2548295119 --> [v2:172.21.15.167:6813/77470876,v1:172.21.15.167:6817/77470876] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0b700c6e50 con 0x7f0b6406ce20 2024-04-17T10:54:53.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b5f7fe640 1 -- 172.21.15.83:0/2548295119 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=145) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b7011fe60 con 0x7f0b7815d300 2024-04-17T10:54:53.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.927+0000 7f0b76d9d640 1 --2- 172.21.15.83:0/2548295119 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0b64066f60 0x7f0b64069420 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f0b68000c10 tx=0x7f0b68009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:53.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.931+0000 7f0b7659c640 1 --2- 172.21.15.83:0/2548295119 >> [v2:172.21.15.167:6813/77470876,v1:172.21.15.167:6817/77470876] conn(0x7f0b6406ce20 0x7f0b6406f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:53.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.931+0000 7f0b7659c640 1 --2- 172.21.15.83:0/2548295119 >> [v2:172.21.15.167:6813/77470876,v1:172.21.15.167:6817/77470876] conn(0x7f0b6406ce20 0x7f0b6406f240 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:53.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.931+0000 7f0b5f7fe640 1 -- 172.21.15.83:0/2548295119 <== osd.4 v2:172.21.15.167:6813/77470876 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0b700c6e50 con 0x7f0b6406ce20 2024-04-17T10:54:53.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.943+0000 7f0b7d0d3640 1 -- 172.21.15.83:0/2548295119 --> [v2:172.21.15.167:6813/77470876,v1:172.21.15.167:6817/77470876] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0b781581a0 con 0x7f0b6406ce20 2024-04-17T10:54:53.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.947+0000 7f0b5f7fe640 1 -- 172.21.15.83:0/2548295119 <== osd.4 v2:172.21.15.167:6813/77470876 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f0b781581a0 con 0x7f0b6406ce20 2024-04-17T10:54:53.939 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:54:53.939 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-17T10:54:53.939 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:54:53.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.947+0000 7f0b7d0d3640 1 -- 172.21.15.83:0/2548295119 >> [v2:172.21.15.167:6813/77470876,v1:172.21.15.167:6817/77470876] conn(0x7f0b6406ce20 msgr2=0x7f0b6406f240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:53.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.947+0000 7f0b7d0d3640 1 --2- 172.21.15.83:0/2548295119 >> [v2:172.21.15.167:6813/77470876,v1:172.21.15.167:6817/77470876] conn(0x7f0b6406ce20 0x7f0b6406f240 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:53.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.947+0000 7f0b7d0d3640 1 -- 172.21.15.83:0/2548295119 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0b64066f60 msgr2=0x7f0b64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:53.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.947+0000 7f0b7d0d3640 1 --2- 172.21.15.83:0/2548295119 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0b64066f60 0x7f0b64069420 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f0b68000c10 tx=0x7f0b68009040 comp rx=0 tx=0).stop 2024-04-17T10:54:53.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.947+0000 7f0b7d0d3640 1 -- 172.21.15.83:0/2548295119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b7815d300 msgr2=0x7f0b781e8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:53.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.947+0000 7f0b7d0d3640 1 --2- 172.21.15.83:0/2548295119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b7815d300 0x7f0b781e8d30 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7f0b700a6010 tx=0x7f0b60005840 comp rx=0 tx=0).stop 2024-04-17T10:54:53.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.947+0000 7f0b7759e640 1 -- 172.21.15.83:0/2548295119 reap_dead start 2024-04-17T10:54:53.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.947+0000 7f0b7d0d3640 1 -- 172.21.15.83:0/2548295119 shutdown_connections 2024-04-17T10:54:53.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.947+0000 7f0b7d0d3640 1 -- 172.21.15.83:0/2548295119 >> 172.21.15.83:0/2548295119 conn(0x7f0b780cc050 msgr2=0x7f0b780b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:53.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.947+0000 7f0b7d0d3640 1 -- 172.21.15.83:0/2548295119 shutdown_connections 2024-04-17T10:54:53.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:53.947+0000 7f0b7d0d3640 1 -- 172.21.15.83:0/2548295119 wait complete. 2024-04-17T10:54:53.954 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-17T10:54:54.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.075+0000 7fbde526b640 1 Processor -- start 2024-04-17T10:54:54.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.075+0000 7fbde526b640 1 -- start start 2024-04-17T10:54:54.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.075+0000 7fbde526b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbde00afee0 0x7fbde015ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.075+0000 7fbde526b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbde00b4270 0x7fbde00b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.075+0000 7fbde526b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbde00b4bc0 con 0x7fbde0071510 2024-04-17T10:54:54.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.075+0000 7fbde526b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbde00b02c0 con 0x7fbde00b4270 2024-04-17T10:54:54.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.075+0000 7fbde526b640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbde015b6e0 con 0x7fbde00afee0 2024-04-17T10:54:54.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.075+0000 7fbddf7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbde00b4270 0x7fbde00b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:54.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.075+0000 7fbddf7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbde00b4270 0x7fbde00b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44548/0 (socket says 172.21.15.83:44548) 2024-04-17T10:54:54.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.075+0000 7fbddf7fe640 1 -- 172.21.15.83:0/3578531895 learned_addr learned my addr 172.21.15.83:0/3578531895 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:54.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.075+0000 7fbdde7fc640 1 --1- 172.21.15.83:0/3578531895 >> v1:172.21.15.83:6789/0 conn(0x7fbde0071510 0x7fbde015ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42330/0 (socket says 172.21.15.83:42330) 2024-04-17T10:54:54.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.075+0000 7fbddeffd640 1 --2- 172.21.15.83:0/3578531895 >> v2:172.21.15.167:3300/0 conn(0x7fbde00afee0 0x7fbde015ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:54.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.075+0000 7fbddf7fe640 1 -- 172.21.15.83:0/3578531895 >> v2:172.21.15.167:3300/0 conn(0x7fbde00afee0 msgr2=0x7fbde015ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:54.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.075+0000 7fbddf7fe640 1 --2- 172.21.15.83:0/3578531895 >> v2:172.21.15.167:3300/0 conn(0x7fbde00afee0 0x7fbde015ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.075+0000 7fbddf7fe640 1 -- 172.21.15.83:0/3578531895 >> v1:172.21.15.83:6789/0 conn(0x7fbde0071510 legacy=0x7fbde015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:54.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.075+0000 7fbddf7fe640 1 -- 172.21.15.83:0/3578531895 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde015b9c0 con 0x7fbde00b4270 2024-04-17T10:54:54.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.079+0000 7fbddf7fe640 1 --2- 172.21.15.83:0/3578531895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde00b4270 0x7fbde00b4680 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7fbde006cc90 tx=0x7fbdd001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e41ece44347827c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:54.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.079+0000 7fbdddffb640 1 -- 172.21.15.83:0/3578531895 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd00b0070 con 0x7fbde00b4270 2024-04-17T10:54:54.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.079+0000 7fbdddffb640 1 -- 172.21.15.83:0/3578531895 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd00ab600 con 0x7fbde00b4270 2024-04-17T10:54:54.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.079+0000 7fbdddffb640 1 -- 172.21.15.83:0/3578531895 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd00ab9d0 con 0x7fbde00b4270 2024-04-17T10:54:54.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.079+0000 7fbde526b640 1 -- 172.21.15.83:0/3578531895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde00b4270 msgr2=0x7fbde00b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:54.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.079+0000 7fbde526b640 1 --2- 172.21.15.83:0/3578531895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde00b4270 0x7fbde00b4680 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7fbde006cc90 tx=0x7fbdd001c520 comp rx=0 tx=0).stop 2024-04-17T10:54:54.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.079+0000 7fbde526b640 1 -- 172.21.15.83:0/3578531895 shutdown_connections 2024-04-17T10:54:54.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.079+0000 7fbde526b640 1 --2- 172.21.15.83:0/3578531895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde00b4270 0x7fbde00b4680 unknown :-1 s=CLOSED pgs=2496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.079+0000 7fbde526b640 1 --2- 172.21.15.83:0/3578531895 >> v2:172.21.15.167:3300/0 conn(0x7fbde00afee0 0x7fbde015ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.079+0000 7fbde526b640 1 -- 172.21.15.83:0/3578531895 >> 172.21.15.83:0/3578531895 conn(0x7fbde006bcf0 msgr2=0x7fbde006c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:54.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.079+0000 7fbde526b640 1 -- 172.21.15.83:0/3578531895 shutdown_connections 2024-04-17T10:54:54.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.079+0000 7fbde526b640 1 -- 172.21.15.83:0/3578531895 wait complete. 2024-04-17T10:54:54.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.079+0000 7fbde526b640 1 Processor -- start 2024-04-17T10:54:54.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.079+0000 7fbde526b640 1 -- start start 2024-04-17T10:54:54.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.079+0000 7fbde526b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbde0071510 0x7fbde01510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.079+0000 7fbde526b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbde00afee0 0x7fbde0150320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.079+0000 7fbde526b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde00b4270 0x7fbde0150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.079+0000 7fbde526b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde015c630 con 0x7fbde00b4270 2024-04-17T10:54:54.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.079+0000 7fbde526b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde015c4b0 con 0x7fbde0071510 2024-04-17T10:54:54.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.079+0000 7fbde526b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde015c7b0 con 0x7fbde00afee0 2024-04-17T10:54:54.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.079+0000 7fbddf7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde00b4270 0x7fbde0150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:54.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.079+0000 7fbddf7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde00b4270 0x7fbde0150860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44552/0 (socket says 172.21.15.83:44552) 2024-04-17T10:54:54.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.079+0000 7fbddf7fe640 1 -- 172.21.15.83:0/2518134997 learned_addr learned my addr 172.21.15.83:0/2518134997 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:54.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.079+0000 7fbdde7fc640 1 --2- 172.21.15.83:0/2518134997 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbde00afee0 0x7fbde0150320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:54.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.079+0000 7fbddf7fe640 1 -- 172.21.15.83:0/2518134997 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbde00afee0 msgr2=0x7fbde0150320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:54.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.079+0000 7fbddf7fe640 1 --2- 172.21.15.83:0/2518134997 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbde00afee0 0x7fbde0150320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.079+0000 7fbddeffd640 1 --2- 172.21.15.83:0/2518134997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbde0071510 0x7fbde01510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:54.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.079+0000 7fbddf7fe640 1 -- 172.21.15.83:0/2518134997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbde0071510 msgr2=0x7fbde01510f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:54.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.079+0000 7fbddf7fe640 1 --2- 172.21.15.83:0/2518134997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbde0071510 0x7fbde01510f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.079+0000 7fbddf7fe640 1 -- 172.21.15.83:0/2518134997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde00cf060 con 0x7fbde00b4270 2024-04-17T10:54:54.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.083+0000 7fbddf7fe640 1 --2- 172.21.15.83:0/2518134997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde00b4270 0x7fbde0150860 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7fbdd00ae040 tx=0x7fbdd00192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:54.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.083+0000 7fbdc77fe640 1 -- 172.21.15.83:0/2518134997 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd00b0040 con 0x7fbde00b4270 2024-04-17T10:54:54.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.083+0000 7fbdc77fe640 1 -- 172.21.15.83:0/2518134997 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd00b9070 con 0x7fbde00b4270 2024-04-17T10:54:54.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.083+0000 7fbde526b640 1 -- 172.21.15.83:0/2518134997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde00cf340 con 0x7fbde00b4270 2024-04-17T10:54:54.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.083+0000 7fbdc77fe640 1 -- 172.21.15.83:0/2518134997 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd00b4670 con 0x7fbde00b4270 2024-04-17T10:54:54.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.083+0000 7fbde526b640 1 -- 172.21.15.83:0/2518134997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde01530d0 con 0x7fbde00b4270 2024-04-17T10:54:54.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.083+0000 7fbde526b640 1 -- 172.21.15.83:0/2518134997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde0074f90 con 0x7fbde00b4270 2024-04-17T10:54:54.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.083+0000 7fbdc77fe640 1 -- 172.21.15.83:0/2518134997 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbdd00b4810 con 0x7fbde00b4270 2024-04-17T10:54:54.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.083+0000 7fbdc77fe640 1 --2- 172.21.15.83:0/2518134997 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbdcc066f60 0x7fbdcc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.083+0000 7fbdc77fe640 1 -- 172.21.15.83:0/2518134997 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbdd011e270 con 0x7fbde00b4270 2024-04-17T10:54:54.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.083+0000 7fbddeffd640 1 --2- 172.21.15.83:0/2518134997 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbdcc066f60 0x7fbdcc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:54.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.083+0000 7fbddeffd640 1 --2- 172.21.15.83:0/2518134997 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbdcc066f60 0x7fbdcc069420 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8009f30 tx=0x7fbdd800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:54.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.087+0000 7fbdc77fe640 1 -- 172.21.15.83:0/2518134997 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbdd00c2020 con 0x7fbde00b4270 2024-04-17T10:54:54.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.231+0000 7fbde526b640 1 -- 172.21.15.83:0/2518134997 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fbde00b09b0 con 0x7fbdcc066f60 2024-04-17T10:54:54.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.235+0000 7fbdc77fe640 1 -- 172.21.15.83:0/2518134997 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbde00b09b0 con 0x7fbdcc066f60 2024-04-17T10:54:54.227 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 4 to deep-scrub 2024-04-17T10:54:54.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.235+0000 7fbde526b640 1 -- 172.21.15.83:0/2518134997 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbdcc066f60 msgr2=0x7fbdcc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:54.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.235+0000 7fbde526b640 1 --2- 172.21.15.83:0/2518134997 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbdcc066f60 0x7fbdcc069420 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8009f30 tx=0x7fbdd800b040 comp rx=0 tx=0).stop 2024-04-17T10:54:54.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.235+0000 7fbde526b640 1 -- 172.21.15.83:0/2518134997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde00b4270 msgr2=0x7fbde0150860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:54.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.235+0000 7fbde526b640 1 --2- 172.21.15.83:0/2518134997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde00b4270 0x7fbde0150860 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7fbdd00ae040 tx=0x7fbdd00192c0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.235+0000 7fbde526b640 1 -- 172.21.15.83:0/2518134997 shutdown_connections 2024-04-17T10:54:54.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.235+0000 7fbde526b640 1 --2- 172.21.15.83:0/2518134997 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fbdcc066f60 0x7fbdcc069420 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.235+0000 7fbde526b640 1 --2- 172.21.15.83:0/2518134997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde00b4270 0x7fbde0150860 unknown :-1 s=CLOSED pgs=2497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.235+0000 7fbde526b640 1 --2- 172.21.15.83:0/2518134997 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbde00afee0 0x7fbde0150320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.235+0000 7fbde526b640 1 --2- 172.21.15.83:0/2518134997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbde0071510 0x7fbde01510f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.235+0000 7fbde526b640 1 -- 172.21.15.83:0/2518134997 >> 172.21.15.83:0/2518134997 conn(0x7fbde006bcf0 msgr2=0x7fbde00ccad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:54.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.235+0000 7fbde526b640 1 -- 172.21.15.83:0/2518134997 shutdown_connections 2024-04-17T10:54:54.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.235+0000 7fbde526b640 1 -- 172.21.15.83:0/2518134997 wait complete. 2024-04-17T10:54:54.244 INFO:tasks.ceph:Scrubbing osd.5 2024-04-17T10:54:54.244 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-17T10:54:54.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.363+0000 7f68ce2c2640 1 Processor -- start 2024-04-17T10:54:54.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.363+0000 7f68ce2c2640 1 -- start start 2024-04-17T10:54:54.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.363+0000 7f68ce2c2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f68c815bad0 0x7f68c80cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.363+0000 7f68ce2c2640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68c806d580 con 0x7f68c806abe0 2024-04-17T10:54:54.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.363+0000 7f68ce2c2640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68c806d820 con 0x7f68c80cf7d0 2024-04-17T10:54:54.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.363+0000 7f68ce2c2640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68c806db00 con 0x7f68c815bad0 2024-04-17T10:54:54.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.363+0000 7f68c7fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f68c806abe0 0x7f68c806afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42346/0 (socket says 172.21.15.83:42346) 2024-04-17T10:54:54.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.363+0000 7f68c7fff640 1 -- 172.21.15.83:0/268384064 learned_addr learned my addr 172.21.15.83:0/268384064 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:54.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.363+0000 7f68c77fe640 1 --2- 172.21.15.83:0/268384064 >> v2:172.21.15.83:3300/0 conn(0x7f68c815bad0 0x7f68c80cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:54.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.363+0000 7f68c6ffd640 1 -- 172.21.15.83:0/268384064 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1495255901 0 0) 0x7f68c806d580 con 0x7f68c806abe0 2024-04-17T10:54:54.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.363+0000 7f68c6ffd640 1 -- 172.21.15.83:0/268384064 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68b0003530 con 0x7f68c806abe0 2024-04-17T10:54:54.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.363+0000 7f68c77fe640 1 -- 172.21.15.83:0/268384064 >> v1:172.21.15.167:6789/0 conn(0x7f68c80cf7d0 legacy=0x7f68c806e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:54.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.363+0000 7f68c77fe640 1 -- 172.21.15.83:0/268384064 >> v1:172.21.15.83:6789/0 conn(0x7f68c806abe0 legacy=0x7f68c806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:54.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.363+0000 7f68c77fe640 1 -- 172.21.15.83:0/268384064 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c806dde0 con 0x7f68c815bad0 2024-04-17T10:54:54.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.367+0000 7f68c77fe640 1 --2- 172.21.15.83:0/268384064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c815bad0 0x7f68c80cf290 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7f68bc0097c0 tx=0x7f68bc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ea9a3f82c1cc3c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:54.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.367+0000 7f68c6ffd640 1 -- 172.21.15.83:0/268384064 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc0b0070 con 0x7f68c815bad0 2024-04-17T10:54:54.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.367+0000 7f68c6ffd640 1 -- 172.21.15.83:0/268384064 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68bc0ab600 con 0x7f68c815bad0 2024-04-17T10:54:54.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.367+0000 7f68c6ffd640 1 -- 172.21.15.83:0/268384064 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc0ab9d0 con 0x7f68c815bad0 2024-04-17T10:54:54.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.367+0000 7f68ce2c2640 1 -- 172.21.15.83:0/268384064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c815bad0 msgr2=0x7f68c80cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:54.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.367+0000 7f68ce2c2640 1 --2- 172.21.15.83:0/268384064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c815bad0 0x7f68c80cf290 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7f68bc0097c0 tx=0x7f68bc01c460 comp rx=0 tx=0).stop 2024-04-17T10:54:54.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.367+0000 7f68ce2c2640 1 -- 172.21.15.83:0/268384064 shutdown_connections 2024-04-17T10:54:54.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.367+0000 7f68ce2c2640 1 --2- 172.21.15.83:0/268384064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c815bad0 0x7f68c80cf290 unknown :-1 s=CLOSED pgs=2499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.367+0000 7f68ce2c2640 1 -- 172.21.15.83:0/268384064 >> 172.21.15.83:0/268384064 conn(0x7f68c80b2410 msgr2=0x7f68c80b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:54.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.367+0000 7f68ce2c2640 1 -- 172.21.15.83:0/268384064 shutdown_connections 2024-04-17T10:54:54.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.367+0000 7f68ce2c2640 1 -- 172.21.15.83:0/268384064 wait complete. 2024-04-17T10:54:54.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.367+0000 7f68ce2c2640 1 Processor -- start 2024-04-17T10:54:54.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.367+0000 7f68ce2c2640 1 -- start start 2024-04-17T10:54:54.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.367+0000 7f68ce2c2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68c806abe0 0x7f68c8151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.367+0000 7f68ce2c2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c80cf7d0 0x7f68c81f9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.367+0000 7f68ce2c2640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68c815bad0 0x7f68c81fa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.367+0000 7f68ce2c2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c81694c0 con 0x7f68c80cf7d0 2024-04-17T10:54:54.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.367+0000 7f68ce2c2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c806f7d0 con 0x7f68c806abe0 2024-04-17T10:54:54.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.367+0000 7f68ce2c2640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c8169640 con 0x7f68c815bad0 2024-04-17T10:54:54.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.367+0000 7f68c77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c80cf7d0 0x7f68c81f9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:54.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.367+0000 7f68c77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c80cf7d0 0x7f68c81f9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44570/0 (socket says 172.21.15.83:44570) 2024-04-17T10:54:54.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.367+0000 7f68c77fe640 1 -- 172.21.15.83:0/1373362557 learned_addr learned my addr 172.21.15.83:0/1373362557 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:54.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.367+0000 7f68c77fe640 1 -- 172.21.15.83:0/1373362557 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68c815bad0 msgr2=0x7f68c81fa1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:54.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.367+0000 7f68c77fe640 1 --2- 172.21.15.83:0/1373362557 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68c815bad0 0x7f68c81fa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.367+0000 7f68c77fe640 1 -- 172.21.15.83:0/1373362557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68c806abe0 msgr2=0x7f68c8151fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:54.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.367+0000 7f68c77fe640 1 --2- 172.21.15.83:0/1373362557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68c806abe0 0x7f68c8151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.367+0000 7f68c77fe640 1 -- 172.21.15.83:0/1373362557 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c80b49c0 con 0x7f68c80cf7d0 2024-04-17T10:54:54.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.371+0000 7f68c77fe640 1 --2- 172.21.15.83:0/1373362557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c80cf7d0 0x7f68c81f9ca0 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7f68bc019070 tx=0x7f68bc019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:54.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.371+0000 7f68c4ff9640 1 -- 172.21.15.83:0/1373362557 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc0b0040 con 0x7f68c80cf7d0 2024-04-17T10:54:54.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.371+0000 7f68c4ff9640 1 -- 172.21.15.83:0/1373362557 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68bc0ba070 con 0x7f68c80cf7d0 2024-04-17T10:54:54.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.371+0000 7f68c4ff9640 1 -- 172.21.15.83:0/1373362557 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc0b4850 con 0x7f68c80cf7d0 2024-04-17T10:54:54.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.371+0000 7f68ce2c2640 1 -- 172.21.15.83:0/1373362557 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c80b4ca0 con 0x7f68c80cf7d0 2024-04-17T10:54:54.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.371+0000 7f68ce2c2640 1 -- 172.21.15.83:0/1373362557 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c81524f0 con 0x7f68c80cf7d0 2024-04-17T10:54:54.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.371+0000 7f68ce2c2640 1 -- 172.21.15.83:0/1373362557 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f68c81526c0 con 0x7f68c80cf7d0 2024-04-17T10:54:54.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.371+0000 7f68c4ff9640 1 -- 172.21.15.83:0/1373362557 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f68bc0b49f0 con 0x7f68c80cf7d0 2024-04-17T10:54:54.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.371+0000 7f68c4ff9640 1 --2- 172.21.15.83:0/1373362557 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f68b4066f60 0x7f68b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.371+0000 7f68c7fff640 1 --2- 172.21.15.83:0/1373362557 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f68b4066f60 0x7f68b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:54.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.371+0000 7f68c4ff9640 1 -- 172.21.15.83:0/1373362557 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f68bc11f5f0 con 0x7f68c80cf7d0 2024-04-17T10:54:54.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.371+0000 7f68c4ff9640 1 --2- 172.21.15.83:0/1373362557 >> [v2:172.21.15.167:6800/1776227116,v1:172.21.15.167:6801/1776227116] conn(0x7f68b406ce20 0x7f68b406f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.371+0000 7f68c4ff9640 1 -- 172.21.15.83:0/1373362557 --> [v2:172.21.15.167:6800/1776227116,v1:172.21.15.167:6801/1776227116] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f68b406f8f0 con 0x7f68b406ce20 2024-04-17T10:54:54.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.371+0000 7f68c4ff9640 1 -- 172.21.15.83:0/1373362557 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=145) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68bc11fa80 con 0x7f68c80cf7d0 2024-04-17T10:54:54.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.371+0000 7f68c7fff640 1 --2- 172.21.15.83:0/1373362557 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f68b4066f60 0x7f68b4069420 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f68c0007fd0 tx=0x7f68c0007900 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:54.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.371+0000 7f68cc85f640 1 --2- 172.21.15.83:0/1373362557 >> [v2:172.21.15.167:6800/1776227116,v1:172.21.15.167:6801/1776227116] conn(0x7f68b406ce20 0x7f68b406f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:54.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.371+0000 7f68cc85f640 1 --2- 172.21.15.83:0/1373362557 >> [v2:172.21.15.167:6800/1776227116,v1:172.21.15.167:6801/1776227116] conn(0x7f68b406ce20 0x7f68b406f240 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:54.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.375+0000 7f68c4ff9640 1 -- 172.21.15.83:0/1373362557 <== osd.5 v2:172.21.15.167:6800/1776227116 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f68b406f8f0 con 0x7f68b406ce20 2024-04-17T10:54:54.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.387+0000 7f68ce2c2640 1 -- 172.21.15.83:0/1373362557 --> [v2:172.21.15.167:6800/1776227116,v1:172.21.15.167:6801/1776227116] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f68c80b4e70 con 0x7f68b406ce20 2024-04-17T10:54:54.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.387+0000 7f68c4ff9640 1 -- 172.21.15.83:0/1373362557 <== osd.5 v2:172.21.15.167:6800/1776227116 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f68c80b4e70 con 0x7f68b406ce20 2024-04-17T10:54:54.382 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:54:54.383 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-17T10:54:54.383 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:54:54.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.387+0000 7f68ce2c2640 1 -- 172.21.15.83:0/1373362557 >> [v2:172.21.15.167:6800/1776227116,v1:172.21.15.167:6801/1776227116] conn(0x7f68b406ce20 msgr2=0x7f68b406f240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:54.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.387+0000 7f68ce2c2640 1 --2- 172.21.15.83:0/1373362557 >> [v2:172.21.15.167:6800/1776227116,v1:172.21.15.167:6801/1776227116] conn(0x7f68b406ce20 0x7f68b406f240 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.387+0000 7f68ce2c2640 1 -- 172.21.15.83:0/1373362557 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f68b4066f60 msgr2=0x7f68b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:54.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.387+0000 7f68ce2c2640 1 --2- 172.21.15.83:0/1373362557 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f68b4066f60 0x7f68b4069420 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f68c0007fd0 tx=0x7f68c0007900 comp rx=0 tx=0).stop 2024-04-17T10:54:54.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.387+0000 7f68ce2c2640 1 -- 172.21.15.83:0/1373362557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c80cf7d0 msgr2=0x7f68c81f9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:54.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.387+0000 7f68ce2c2640 1 --2- 172.21.15.83:0/1373362557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c80cf7d0 0x7f68c81f9ca0 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7f68bc019070 tx=0x7f68bc019200 comp rx=0 tx=0).stop 2024-04-17T10:54:54.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.387+0000 7f68cc85f640 1 -- 172.21.15.83:0/1373362557 reap_dead start 2024-04-17T10:54:54.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.387+0000 7f68ce2c2640 1 -- 172.21.15.83:0/1373362557 shutdown_connections 2024-04-17T10:54:54.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.387+0000 7f68ce2c2640 1 -- 172.21.15.83:0/1373362557 >> 172.21.15.83:0/1373362557 conn(0x7f68c80b2410 msgr2=0x7f68c806e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:54.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.387+0000 7f68ce2c2640 1 -- 172.21.15.83:0/1373362557 shutdown_connections 2024-04-17T10:54:54.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.387+0000 7f68ce2c2640 1 -- 172.21.15.83:0/1373362557 wait complete. 2024-04-17T10:54:54.397 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-17T10:54:54.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9ba7731640 1 Processor -- start 2024-04-17T10:54:54.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9ba7731640 1 -- start start 2024-04-17T10:54:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9ba7731640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ba0154de0 0x7f9ba006ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9ba7731640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ba015bd70 con 0x7f9ba006b350 2024-04-17T10:54:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9ba7731640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ba015bf60 con 0x7f9ba00b4210 2024-04-17T10:54:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9ba7731640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba00b4870 con 0x7f9ba0154de0 2024-04-17T10:54:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9ba4ccc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9ba006b350 0x7f9ba00b3b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42360/0 (socket says 172.21.15.83:42360) 2024-04-17T10:54:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9ba4ccc640 1 -- 172.21.15.83:0/2504740280 learned_addr learned my addr 172.21.15.83:0/2504740280 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9b97fff640 1 -- 172.21.15.83:0/2504740280 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3163129658 0 0) 0x7f9ba015bd70 con 0x7f9ba006b350 2024-04-17T10:54:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9ba54cd640 1 --2- 172.21.15.83:0/2504740280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ba0154de0 0x7f9ba006ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9b97fff640 1 -- 172.21.15.83:0/2504740280 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b8c003530 con 0x7f9ba006b350 2024-04-17T10:54:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9b97fff640 1 -- 172.21.15.83:0/2504740280 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1473844429 0 0) 0x7f9b8c003530 con 0x7f9ba006b350 2024-04-17T10:54:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9b97fff640 1 -- 172.21.15.83:0/2504740280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ba0154de0 msgr2=0x7f9ba006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9b97fff640 1 --2- 172.21.15.83:0/2504740280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ba0154de0 0x7f9ba006ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9b97fff640 1 -- 172.21.15.83:0/2504740280 >> v1:172.21.15.167:6789/0 conn(0x7f9ba00b4210 legacy=0x7f9ba006cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9b97fff640 1 -- 172.21.15.83:0/2504740280 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba015afb0 con 0x7f9ba006b350 2024-04-17T10:54:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9b97fff640 1 -- 172.21.15.83:0/2504740280 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f9b90001680 con 0x7f9ba006b350 2024-04-17T10:54:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9b97fff640 1 -- 172.21.15.83:0/2504740280 >> v1:172.21.15.83:6789/0 conn(0x7f9ba006b350 legacy=0x7f9ba00b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9b97fff640 1 --2- 172.21.15.83:0/2504740280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b8c003a60 0x7f9b8c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9b97fff640 1 --2- 172.21.15.83:0/2504740280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b8c004810 0x7f9b8c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9b97fff640 1 --2- 172.21.15.83:0/2504740280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ba0154de0 0x7f9b8c005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9b97fff640 1 -- 172.21.15.83:0/2504740280 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c00bb60 con 0x7f9b8c003a60 2024-04-17T10:54:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9b97fff640 1 -- 172.21.15.83:0/2504740280 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c00bd10 con 0x7f9b8c004810 2024-04-17T10:54:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9b97fff640 1 -- 172.21.15.83:0/2504740280 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c00bf10 con 0x7f9ba0154de0 2024-04-17T10:54:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9ba54cd640 1 --2- 172.21.15.83:0/2504740280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b8c003a60 0x7f9b8c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9ba4ccc640 1 --2- 172.21.15.83:0/2504740280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b8c004810 0x7f9b8c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9ba5cce640 1 --2- 172.21.15.83:0/2504740280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ba0154de0 0x7f9b8c005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9ba54cd640 1 -- 172.21.15.83:0/2504740280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ba0154de0 msgr2=0x7f9b8c005280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9ba54cd640 1 --2- 172.21.15.83:0/2504740280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ba0154de0 0x7f9b8c005280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9ba54cd640 1 -- 172.21.15.83:0/2504740280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b8c004810 msgr2=0x7f9b8c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9ba54cd640 1 --2- 172.21.15.83:0/2504740280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b8c004810 0x7f9b8c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9ba54cd640 1 -- 172.21.15.83:0/2504740280 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8c003530 con 0x7f9b8c003a60 2024-04-17T10:54:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9ba4ccc640 1 --2- 172.21.15.83:0/2504740280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b8c004810 0x7f9b8c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.519+0000 7f9ba5cce640 1 --2- 172.21.15.83:0/2504740280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ba0154de0 0x7f9b8c005280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.523+0000 7f9ba54cd640 1 --2- 172.21.15.83:0/2504740280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b8c003a60 0x7f9b8c003e80 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c00ba30 tx=0x7f9b9c01c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c22ba65a773c10 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.523+0000 7f9b97fff640 1 -- 172.21.15.83:0/2504740280 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c0b3070 con 0x7f9b8c003a60 2024-04-17T10:54:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.523+0000 7f9b97fff640 1 -- 172.21.15.83:0/2504740280 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9c0ae630 con 0x7f9b8c003a60 2024-04-17T10:54:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.523+0000 7f9b97fff640 1 -- 172.21.15.83:0/2504740280 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c0aa050 con 0x7f9b8c003a60 2024-04-17T10:54:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.523+0000 7f9ba7731640 1 -- 172.21.15.83:0/2504740280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b8c003a60 msgr2=0x7f9b8c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.523+0000 7f9ba7731640 1 --2- 172.21.15.83:0/2504740280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b8c003a60 0x7f9b8c003e80 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c00ba30 tx=0x7f9b9c01c6a0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.523+0000 7f9ba5cce640 1 -- 172.21.15.83:0/2504740280 reap_dead start 2024-04-17T10:54:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.523+0000 7f9ba7731640 1 -- 172.21.15.83:0/2504740280 shutdown_connections 2024-04-17T10:54:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.523+0000 7f9ba7731640 1 -- 172.21.15.83:0/2504740280 >> 172.21.15.83:0/2504740280 conn(0x7f9ba00b15b0 msgr2=0x7f9ba00b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.523+0000 7f9ba7731640 1 -- 172.21.15.83:0/2504740280 shutdown_connections 2024-04-17T10:54:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.523+0000 7f9ba7731640 1 -- 172.21.15.83:0/2504740280 wait complete. 2024-04-17T10:54:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.523+0000 7f9ba7731640 1 Processor -- start 2024-04-17T10:54:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.523+0000 7f9ba7731640 1 -- start start 2024-04-17T10:54:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.527+0000 7f9ba7731640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b8c003a60 0x7f9ba0142270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.527+0000 7f9ba7731640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ba0156000 0x7f9ba0158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.527+0000 7f9ba7731640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ba0142b60 0x7f9ba0159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.527+0000 7f9ba7731640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba01551c0 con 0x7f9ba0156000 2024-04-17T10:54:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.527+0000 7f9ba7731640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba006b770 con 0x7f9ba0142b60 2024-04-17T10:54:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.527+0000 7f9ba7731640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba0166730 con 0x7f9b8c003a60 2024-04-17T10:54:54.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.527+0000 7f9ba4ccc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ba0156000 0x7f9ba0158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:54.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.527+0000 7f9ba4ccc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ba0156000 0x7f9ba0158dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44580/0 (socket says 172.21.15.83:44580) 2024-04-17T10:54:54.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.527+0000 7f9ba4ccc640 1 -- 172.21.15.83:0/3807632608 learned_addr learned my addr 172.21.15.83:0/3807632608 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:54.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.527+0000 7f9ba54cd640 1 --2- 172.21.15.83:0/3807632608 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b8c003a60 0x7f9ba0142270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:54.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.527+0000 7f9ba5cce640 1 --2- 172.21.15.83:0/3807632608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ba0142b60 0x7f9ba0159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:54.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.527+0000 7f9ba4ccc640 1 -- 172.21.15.83:0/3807632608 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b8c003a60 msgr2=0x7f9ba0142270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:54.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.527+0000 7f9ba4ccc640 1 --2- 172.21.15.83:0/3807632608 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b8c003a60 0x7f9ba0142270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.527+0000 7f9ba4ccc640 1 -- 172.21.15.83:0/3807632608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ba0142b60 msgr2=0x7f9ba0159f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.527+0000 7f9ba4ccc640 1 --2- 172.21.15.83:0/3807632608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ba0142b60 0x7f9ba0159f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.527+0000 7f9ba4ccc640 1 -- 172.21.15.83:0/3807632608 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba0159760 con 0x7f9ba0156000 2024-04-17T10:54:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.527+0000 7f9ba4ccc640 1 --2- 172.21.15.83:0/3807632608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ba0156000 0x7f9ba0158dd0 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7f9b9000d870 tx=0x7f9b9000dd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.527+0000 7f9b95ffb640 1 -- 172.21.15.83:0/3807632608 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b90019070 con 0x7f9ba0156000 2024-04-17T10:54:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.527+0000 7f9b95ffb640 1 -- 172.21.15.83:0/3807632608 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b900074e0 con 0x7f9ba0156000 2024-04-17T10:54:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.527+0000 7f9b95ffb640 1 -- 172.21.15.83:0/3807632608 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b90004a30 con 0x7f9ba0156000 2024-04-17T10:54:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.527+0000 7f9ba7731640 1 -- 172.21.15.83:0/3807632608 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ba0156500 con 0x7f9ba0156000 2024-04-17T10:54:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.527+0000 7f9ba7731640 1 -- 172.21.15.83:0/3807632608 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ba0156ac0 con 0x7f9ba0156000 2024-04-17T10:54:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.527+0000 7f9ba7731640 1 -- 172.21.15.83:0/3807632608 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ba0074f90 con 0x7f9ba0156000 2024-04-17T10:54:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.527+0000 7f9b95ffb640 1 -- 172.21.15.83:0/3807632608 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9b9001c070 con 0x7f9ba0156000 2024-04-17T10:54:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.527+0000 7f9b95ffb640 1 --2- 172.21.15.83:0/3807632608 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9b84066f90 0x7f9b84069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.527+0000 7f9ba54cd640 1 --2- 172.21.15.83:0/3807632608 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9b84066f90 0x7f9b84069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.527+0000 7f9b95ffb640 1 -- 172.21.15.83:0/3807632608 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9b900847c0 con 0x7f9ba0156000 2024-04-17T10:54:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.531+0000 7f9ba54cd640 1 --2- 172.21.15.83:0/3807632608 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9b84066f90 0x7f9b84069450 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0b0000 tx=0x7f9b9c0046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.535+0000 7f9b95ffb640 1 -- 172.21.15.83:0/3807632608 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9b900554b0 con 0x7f9ba0156000 2024-04-17T10:54:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.687+0000 7f9ba7731640 1 -- 172.21.15.83:0/3807632608 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ba00b3280 con 0x7f9b84066f90 2024-04-17T10:54:54.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.687+0000 7f9b95ffb640 1 -- 172.21.15.83:0/3807632608 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9ba00b3280 con 0x7f9b84066f90 2024-04-17T10:54:54.682 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 5 to deep-scrub 2024-04-17T10:54:54.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.691+0000 7f9ba7731640 1 -- 172.21.15.83:0/3807632608 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9b84066f90 msgr2=0x7f9b84069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:54.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.691+0000 7f9ba7731640 1 --2- 172.21.15.83:0/3807632608 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9b84066f90 0x7f9b84069450 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0b0000 tx=0x7f9b9c0046d0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.691+0000 7f9ba7731640 1 -- 172.21.15.83:0/3807632608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ba0156000 msgr2=0x7f9ba0158dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:54.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.691+0000 7f9ba7731640 1 --2- 172.21.15.83:0/3807632608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ba0156000 0x7f9ba0158dd0 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7f9b9000d870 tx=0x7f9b9000dd30 comp rx=0 tx=0).stop 2024-04-17T10:54:54.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.691+0000 7f9ba7731640 1 -- 172.21.15.83:0/3807632608 shutdown_connections 2024-04-17T10:54:54.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.691+0000 7f9ba7731640 1 --2- 172.21.15.83:0/3807632608 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9b84066f90 0x7f9b84069450 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.691+0000 7f9ba7731640 1 --2- 172.21.15.83:0/3807632608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ba0142b60 0x7f9ba0159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.691+0000 7f9ba7731640 1 --2- 172.21.15.83:0/3807632608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ba0156000 0x7f9ba0158dd0 unknown :-1 s=CLOSED pgs=2503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.691+0000 7f9ba7731640 1 --2- 172.21.15.83:0/3807632608 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b8c003a60 0x7f9ba0142270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.691+0000 7f9ba7731640 1 -- 172.21.15.83:0/3807632608 >> 172.21.15.83:0/3807632608 conn(0x7f9ba00b15b0 msgr2=0x7f9ba00b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:54.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.691+0000 7f9ba7731640 1 -- 172.21.15.83:0/3807632608 shutdown_connections 2024-04-17T10:54:54.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.691+0000 7f9ba7731640 1 -- 172.21.15.83:0/3807632608 wait complete. 2024-04-17T10:54:54.700 INFO:tasks.ceph:Scrubbing osd.6 2024-04-17T10:54:54.701 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-17T10:54:54.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.815+0000 7fb96a851640 1 Processor -- start 2024-04-17T10:54:54.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.819+0000 7fb96a851640 1 -- start start 2024-04-17T10:54:54.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.819+0000 7fb96a851640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb9640aea50 0x7fb96415daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.819+0000 7fb96a851640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb96415dfe0 0x7fb96406c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.819+0000 7fb96a851640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb96406bc40 con 0x7fb964157a40 2024-04-17T10:54:54.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.819+0000 7fb96a851640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb96406e200 con 0x7fb9640aea50 2024-04-17T10:54:54.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.819+0000 7fb96a851640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb96415e430 con 0x7fb96415dfe0 2024-04-17T10:54:54.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.819+0000 7fb95b7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb9640aea50 0x7fb96415daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:54.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.819+0000 7fb95b7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb9640aea50 0x7fb96415daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44590/0 (socket says 172.21.15.83:44590) 2024-04-17T10:54:54.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.819+0000 7fb95b7fe640 1 -- 172.21.15.83:0/4171989468 learned_addr learned my addr 172.21.15.83:0/4171989468 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:54.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.819+0000 7fb95b7fe640 1 -- 172.21.15.83:0/4171989468 >> v2:172.21.15.167:3300/0 conn(0x7fb96415dfe0 msgr2=0x7fb96406c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:54.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.819+0000 7fb968dee640 1 --2- 172.21.15.83:0/4171989468 >> v2:172.21.15.167:3300/0 conn(0x7fb96415dfe0 0x7fb96406c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:54.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.819+0000 7fb95b7fe640 1 --2- 172.21.15.83:0/4171989468 >> v2:172.21.15.167:3300/0 conn(0x7fb96415dfe0 0x7fb96406c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.819+0000 7fb95b7fe640 1 -- 172.21.15.83:0/4171989468 >> v1:172.21.15.167:6789/0 conn(0x7fb964157a40 legacy=0x7fb9640b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:54.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.819+0000 7fb95b7fe640 1 -- 172.21.15.83:0/4171989468 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9640aeee0 con 0x7fb9640aea50 2024-04-17T10:54:54.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.823+0000 7fb95b7fe640 1 --2- 172.21.15.83:0/4171989468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9640aea50 0x7fb96415daa0 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7fb96415ebc0 tx=0x7fb94c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fdbb3e0028ef743 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:54.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.823+0000 7fb9637fe640 1 -- 172.21.15.83:0/4171989468 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c0b1070 con 0x7fb9640aea50 2024-04-17T10:54:54.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.823+0000 7fb9637fe640 1 -- 172.21.15.83:0/4171989468 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94c0ac630 con 0x7fb9640aea50 2024-04-17T10:54:54.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.823+0000 7fb9637fe640 1 -- 172.21.15.83:0/4171989468 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c0a8050 con 0x7fb9640aea50 2024-04-17T10:54:54.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.823+0000 7fb96a851640 1 -- 172.21.15.83:0/4171989468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9640aea50 msgr2=0x7fb96415daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:54.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.823+0000 7fb96a851640 1 --2- 172.21.15.83:0/4171989468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9640aea50 0x7fb96415daa0 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7fb96415ebc0 tx=0x7fb94c01c4e0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.823+0000 7fb96a851640 1 -- 172.21.15.83:0/4171989468 shutdown_connections 2024-04-17T10:54:54.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.823+0000 7fb96a851640 1 --2- 172.21.15.83:0/4171989468 >> v2:172.21.15.167:3300/0 conn(0x7fb96415dfe0 0x7fb96406c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.823+0000 7fb96a851640 1 --2- 172.21.15.83:0/4171989468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9640aea50 0x7fb96415daa0 unknown :-1 s=CLOSED pgs=2504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.823+0000 7fb96a851640 1 -- 172.21.15.83:0/4171989468 >> 172.21.15.83:0/4171989468 conn(0x7fb9640cc050 msgr2=0x7fb9640cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:54.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.823+0000 7fb96a851640 1 -- 172.21.15.83:0/4171989468 shutdown_connections 2024-04-17T10:54:54.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.823+0000 7fb96a851640 1 -- 172.21.15.83:0/4171989468 wait complete. 2024-04-17T10:54:54.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.823+0000 7fb96a851640 1 Processor -- start 2024-04-17T10:54:54.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.823+0000 7fb96a851640 1 -- start start 2024-04-17T10:54:54.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.823+0000 7fb96a851640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb964157a40 0x7fb964071590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.823+0000 7fb96a851640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb96415dfe0 0x7fb964146980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.823+0000 7fb96a851640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb964146ec0 0x7fb964145d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.823+0000 7fb96a851640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96406cb70 con 0x7fb964157a40 2024-04-17T10:54:54.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.823+0000 7fb96a851640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96406c9f0 con 0x7fb96415dfe0 2024-04-17T10:54:54.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.823+0000 7fb96a851640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb96406ccf0 con 0x7fb964146ec0 2024-04-17T10:54:54.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.823+0000 7fb963fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb964157a40 0x7fb964071590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:54.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.823+0000 7fb963fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb964157a40 0x7fb964071590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44596/0 (socket says 172.21.15.83:44596) 2024-04-17T10:54:54.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.823+0000 7fb963fff640 1 -- 172.21.15.83:0/1875156984 learned_addr learned my addr 172.21.15.83:0/1875156984 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:54.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.823+0000 7fb95b7fe640 1 --2- 172.21.15.83:0/1875156984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb96415dfe0 0x7fb964146980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:54.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.823+0000 7fb968dee640 1 --2- 172.21.15.83:0/1875156984 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb964146ec0 0x7fb964145d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:54.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.823+0000 7fb963fff640 1 -- 172.21.15.83:0/1875156984 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb964146ec0 msgr2=0x7fb964145d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:54.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.823+0000 7fb963fff640 1 --2- 172.21.15.83:0/1875156984 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb964146ec0 0x7fb964145d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.823+0000 7fb963fff640 1 -- 172.21.15.83:0/1875156984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb96415dfe0 msgr2=0x7fb964146980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:54.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.827+0000 7fb963fff640 1 --2- 172.21.15.83:0/1875156984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb96415dfe0 0x7fb964146980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.827+0000 7fb963fff640 1 -- 172.21.15.83:0/1875156984 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9641539c0 con 0x7fb964157a40 2024-04-17T10:54:54.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.827+0000 7fb95b7fe640 1 --2- 172.21.15.83:0/1875156984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb96415dfe0 0x7fb964146980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:54.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.827+0000 7fb963fff640 1 --2- 172.21.15.83:0/1875156984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb964157a40 0x7fb964071590 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7fb95400c450 tx=0x7fb95400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:54.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.827+0000 7fb9617fa640 1 -- 172.21.15.83:0/1875156984 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb954012830 con 0x7fb964157a40 2024-04-17T10:54:54.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.827+0000 7fb9617fa640 1 -- 172.21.15.83:0/1875156984 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb954017070 con 0x7fb964157a40 2024-04-17T10:54:54.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.827+0000 7fb96a851640 1 -- 172.21.15.83:0/1875156984 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9640b0500 con 0x7fb964157a40 2024-04-17T10:54:54.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.827+0000 7fb9617fa640 1 -- 172.21.15.83:0/1875156984 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb954012c30 con 0x7fb964157a40 2024-04-17T10:54:54.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.827+0000 7fb96a851640 1 -- 172.21.15.83:0/1875156984 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9640b0220 con 0x7fb964157a40 2024-04-17T10:54:54.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.827+0000 7fb96a851640 1 -- 172.21.15.83:0/1875156984 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb964075000 con 0x7fb964157a40 2024-04-17T10:54:54.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.827+0000 7fb9617fa640 1 -- 172.21.15.83:0/1875156984 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb95401c460 con 0x7fb964157a40 2024-04-17T10:54:54.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.827+0000 7fb9617fa640 1 --2- 172.21.15.83:0/1875156984 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb944066f60 0x7fb944069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.827+0000 7fb9617fa640 1 -- 172.21.15.83:0/1875156984 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb9540854d0 con 0x7fb964157a40 2024-04-17T10:54:54.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.827+0000 7fb95b7fe640 1 --2- 172.21.15.83:0/1875156984 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb944066f60 0x7fb944069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:54.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.827+0000 7fb9617fa640 1 --2- 172.21.15.83:0/1875156984 >> [v2:172.21.15.167:6812/1109787903,v1:172.21.15.167:6815/1109787903] conn(0x7fb94406ce20 0x7fb94406f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.827+0000 7fb9617fa640 1 -- 172.21.15.83:0/1875156984 --> [v2:172.21.15.167:6812/1109787903,v1:172.21.15.167:6815/1109787903] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb9540054b0 con 0x7fb94406ce20 2024-04-17T10:54:54.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.827+0000 7fb9617fa640 1 -- 172.21.15.83:0/1875156984 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=145) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb954085960 con 0x7fb964157a40 2024-04-17T10:54:54.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.827+0000 7fb95b7fe640 1 --2- 172.21.15.83:0/1875156984 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb944066f60 0x7fb944069420 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fb94c0ae000 tx=0x7fb94c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:54.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.827+0000 7fb968dee640 1 --2- 172.21.15.83:0/1875156984 >> [v2:172.21.15.167:6812/1109787903,v1:172.21.15.167:6815/1109787903] conn(0x7fb94406ce20 0x7fb94406f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:54.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.827+0000 7fb968dee640 1 --2- 172.21.15.83:0/1875156984 >> [v2:172.21.15.167:6812/1109787903,v1:172.21.15.167:6815/1109787903] conn(0x7fb94406ce20 0x7fb94406f240 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:54.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.831+0000 7fb9617fa640 1 -- 172.21.15.83:0/1875156984 <== osd.6 v2:172.21.15.167:6812/1109787903 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb9540054b0 con 0x7fb94406ce20 2024-04-17T10:54:54.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.843+0000 7fb96a851640 1 -- 172.21.15.83:0/1875156984 --> [v2:172.21.15.167:6812/1109787903,v1:172.21.15.167:6815/1109787903] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb9640ccef0 con 0x7fb94406ce20 2024-04-17T10:54:54.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.843+0000 7fb9617fa640 1 -- 172.21.15.83:0/1875156984 <== osd.6 v2:172.21.15.167:6812/1109787903 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fb9640ccef0 con 0x7fb94406ce20 2024-04-17T10:54:54.838 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:54:54.838 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-17T10:54:54.838 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:54:54.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.843+0000 7fb96a851640 1 -- 172.21.15.83:0/1875156984 >> [v2:172.21.15.167:6812/1109787903,v1:172.21.15.167:6815/1109787903] conn(0x7fb94406ce20 msgr2=0x7fb94406f240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:54.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.843+0000 7fb96a851640 1 --2- 172.21.15.83:0/1875156984 >> [v2:172.21.15.167:6812/1109787903,v1:172.21.15.167:6815/1109787903] conn(0x7fb94406ce20 0x7fb94406f240 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.843+0000 7fb96a851640 1 -- 172.21.15.83:0/1875156984 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb944066f60 msgr2=0x7fb944069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:54.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.843+0000 7fb96a851640 1 --2- 172.21.15.83:0/1875156984 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb944066f60 0x7fb944069420 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fb94c0ae000 tx=0x7fb94c0a7000 comp rx=0 tx=0).stop 2024-04-17T10:54:54.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.843+0000 7fb96a851640 1 -- 172.21.15.83:0/1875156984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb964157a40 msgr2=0x7fb964071590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:54.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.843+0000 7fb96a851640 1 --2- 172.21.15.83:0/1875156984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb964157a40 0x7fb964071590 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7fb95400c450 tx=0x7fb95400c910 comp rx=0 tx=0).stop 2024-04-17T10:54:54.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.843+0000 7fb968dee640 1 -- 172.21.15.83:0/1875156984 reap_dead start 2024-04-17T10:54:54.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.843+0000 7fb96a851640 1 -- 172.21.15.83:0/1875156984 shutdown_connections 2024-04-17T10:54:54.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.843+0000 7fb96a851640 1 -- 172.21.15.83:0/1875156984 >> 172.21.15.83:0/1875156984 conn(0x7fb9640cc050 msgr2=0x7fb9640b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:54.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.843+0000 7fb96a851640 1 -- 172.21.15.83:0/1875156984 shutdown_connections 2024-04-17T10:54:54.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.843+0000 7fb96a851640 1 -- 172.21.15.83:0/1875156984 wait complete. 2024-04-17T10:54:54.852 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-17T10:54:54.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.963+0000 7f1f341c4640 1 Processor -- start 2024-04-17T10:54:54.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.963+0000 7f1f341c4640 1 -- start start 2024-04-17T10:54:54.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.963+0000 7f1f341c4640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1f2c0b2e30 0x7f1f2c0d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.963+0000 7f1f341c4640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f2c06c150 con 0x7f1f2c0b0900 2024-04-17T10:54:54.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.963+0000 7f1f341c4640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f2c0b1c40 con 0x7f1f2c0ce910 2024-04-17T10:54:54.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.963+0000 7f1f341c4640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f2c0d1020 con 0x7f1f2c0b2e30 2024-04-17T10:54:54.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.963+0000 7f1f3175f640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1f2c0b0900 0x7f1f2c0b2720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42374/0 (socket says 172.21.15.83:42374) 2024-04-17T10:54:54.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.963+0000 7f1f3175f640 1 -- 172.21.15.83:0/500228473 learned_addr learned my addr 172.21.15.83:0/500228473 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:54.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.963+0000 7f1f30f5e640 1 -- 172.21.15.83:0/500228473 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 839513386 0 0) 0x7f1f2c06c150 con 0x7f1f2c0b0900 2024-04-17T10:54:54.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.963+0000 7f1f32761640 1 --2- 172.21.15.83:0/500228473 >> v2:172.21.15.167:3300/0 conn(0x7f1f2c0b2e30 0x7f1f2c0d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:54.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.963+0000 7f1f30f5e640 1 -- 172.21.15.83:0/500228473 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f18003530 con 0x7f1f2c0b0900 2024-04-17T10:54:54.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.963+0000 7f1f30f5e640 1 -- 172.21.15.83:0/500228473 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1783064100 0 0) 0x7f1f18003530 con 0x7f1f2c0b0900 2024-04-17T10:54:54.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.963+0000 7f1f30f5e640 1 -- 172.21.15.83:0/500228473 >> v2:172.21.15.167:3300/0 conn(0x7f1f2c0b2e30 msgr2=0x7f1f2c0d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:54.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.963+0000 7f1f30f5e640 1 --2- 172.21.15.83:0/500228473 >> v2:172.21.15.167:3300/0 conn(0x7f1f2c0b2e30 0x7f1f2c0d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.963+0000 7f1f30f5e640 1 -- 172.21.15.83:0/500228473 >> v1:172.21.15.167:6789/0 conn(0x7f1f2c0ce910 legacy=0x7f1f2c0ae490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:54.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.963+0000 7f1f30f5e640 1 -- 172.21.15.83:0/500228473 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2c0b1f20 con 0x7f1f2c0b0900 2024-04-17T10:54:54.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.963+0000 7f1f32761640 1 --2- 172.21.15.83:0/500228473 >> v2:172.21.15.167:3300/0 conn(0x7f1f2c0b2e30 0x7f1f2c0d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:54:54.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.963+0000 7f1f30f5e640 1 -- 172.21.15.83:0/500228473 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f1f1c001680 con 0x7f1f2c0b0900 2024-04-17T10:54:54.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.967+0000 7f1f30f5e640 1 -- 172.21.15.83:0/500228473 >> v1:172.21.15.83:6789/0 conn(0x7f1f2c0b0900 legacy=0x7f1f2c0b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:54.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.967+0000 7f1f30f5e640 1 --2- 172.21.15.83:0/500228473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f18003a30 0x7f1f18003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.967+0000 7f1f30f5e640 1 --2- 172.21.15.83:0/500228473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f18004810 0x7f1f18004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.967+0000 7f1f30f5e640 1 --2- 172.21.15.83:0/500228473 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f180052f0 0x7f1f1800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.967+0000 7f1f30f5e640 1 -- 172.21.15.83:0/500228473 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c0d1020 con 0x7f1f18003a30 2024-04-17T10:54:54.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.967+0000 7f1f30f5e640 1 -- 172.21.15.83:0/500228473 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1800c240 con 0x7f1f18004810 2024-04-17T10:54:54.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.967+0000 7f1f30f5e640 1 -- 172.21.15.83:0/500228473 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1800c440 con 0x7f1f180052f0 2024-04-17T10:54:54.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.967+0000 7f1f31f60640 1 --2- 172.21.15.83:0/500228473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f18003a30 0x7f1f18003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:54.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.967+0000 7f1f31f60640 1 -- 172.21.15.83:0/500228473 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f180052f0 msgr2=0x7f1f1800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:54.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.967+0000 7f1f31f60640 1 --2- 172.21.15.83:0/500228473 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f180052f0 0x7f1f1800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.967+0000 7f1f31f60640 1 -- 172.21.15.83:0/500228473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f18004810 msgr2=0x7f1f18004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:54.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.967+0000 7f1f31f60640 1 --2- 172.21.15.83:0/500228473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f18004810 0x7f1f18004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.967+0000 7f1f31f60640 1 -- 172.21.15.83:0/500228473 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f18003530 con 0x7f1f18003a30 2024-04-17T10:54:54.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.967+0000 7f1f32761640 1 -- 172.21.15.83:0/500228473 reap_dead start 2024-04-17T10:54:54.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.967+0000 7f1f31f60640 1 --2- 172.21.15.83:0/500228473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f18003a30 0x7f1f18003e50 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7f1f28001750 tx=0x7f1f2801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186441fe5ac0a9bd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:54.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.967+0000 7f1f30f5e640 1 -- 172.21.15.83:0/500228473 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f280013a0 con 0x7f1f18003a30 2024-04-17T10:54:54.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.967+0000 7f1f30f5e640 1 -- 172.21.15.83:0/500228473 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f280052f0 con 0x7f1f18003a30 2024-04-17T10:54:54.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.967+0000 7f1f30f5e640 1 -- 172.21.15.83:0/500228473 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f28003840 con 0x7f1f18003a30 2024-04-17T10:54:54.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.967+0000 7f1f341c4640 1 -- 172.21.15.83:0/500228473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f18003a30 msgr2=0x7f1f18003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:54.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.967+0000 7f1f341c4640 1 --2- 172.21.15.83:0/500228473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f18003a30 0x7f1f18003e50 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7f1f28001750 tx=0x7f1f2801c3c0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.967+0000 7f1f341c4640 1 -- 172.21.15.83:0/500228473 shutdown_connections 2024-04-17T10:54:54.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.967+0000 7f1f341c4640 1 --2- 172.21.15.83:0/500228473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f18003a30 0x7f1f18003e50 unknown :-1 s=CLOSED pgs=2507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.967+0000 7f1f341c4640 1 -- 172.21.15.83:0/500228473 >> 172.21.15.83:0/500228473 conn(0x7f1f2c0d17a0 msgr2=0x7f1f2c0d1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:54.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.967+0000 7f1f341c4640 1 -- 172.21.15.83:0/500228473 shutdown_connections 2024-04-17T10:54:54.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.967+0000 7f1f341c4640 1 -- 172.21.15.83:0/500228473 wait complete. 2024-04-17T10:54:54.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.971+0000 7f1f341c4640 1 Processor -- start 2024-04-17T10:54:54.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.971+0000 7f1f341c4640 1 -- start start 2024-04-17T10:54:54.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.971+0000 7f1f341c4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f18003a30 0x7f1f2c152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.971+0000 7f1f341c4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f2c0b6bb0 0x7f1f2c0b5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.971+0000 7f1f341c4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f2c0b5ab0 0x7f1f2c0c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.971+0000 7f1f341c4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c06c990 con 0x7f1f2c0b5ab0 2024-04-17T10:54:54.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.971+0000 7f1f341c4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c06c810 con 0x7f1f2c0b6bb0 2024-04-17T10:54:54.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.971+0000 7f1f341c4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c06cb10 con 0x7f1f18003a30 2024-04-17T10:54:54.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.971+0000 7f1f32761640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f2c0b5ab0 0x7f1f2c0c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:54.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.971+0000 7f1f32761640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f2c0b5ab0 0x7f1f2c0c6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44602/0 (socket says 172.21.15.83:44602) 2024-04-17T10:54:54.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.971+0000 7f1f32761640 1 -- 172.21.15.83:0/2005363114 learned_addr learned my addr 172.21.15.83:0/2005363114 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:54.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.971+0000 7f1f31f60640 1 --2- 172.21.15.83:0/2005363114 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f18003a30 0x7f1f2c152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:54.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.971+0000 7f1f3175f640 1 --2- 172.21.15.83:0/2005363114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f2c0b6bb0 0x7f1f2c0b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:54.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.971+0000 7f1f32761640 1 -- 172.21.15.83:0/2005363114 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f18003a30 msgr2=0x7f1f2c152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:54.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.971+0000 7f1f32761640 1 --2- 172.21.15.83:0/2005363114 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f18003a30 0x7f1f2c152dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.971+0000 7f1f32761640 1 -- 172.21.15.83:0/2005363114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f2c0b6bb0 msgr2=0x7f1f2c0b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:54.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.971+0000 7f1f32761640 1 --2- 172.21.15.83:0/2005363114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f2c0b6bb0 0x7f1f2c0b5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:54.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.971+0000 7f1f32761640 1 -- 172.21.15.83:0/2005363114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2c0b44d0 con 0x7f1f2c0b5ab0 2024-04-17T10:54:54.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.971+0000 7f1f32761640 1 --2- 172.21.15.83:0/2005363114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f2c0b5ab0 0x7f1f2c0c6fb0 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c072410 tx=0x7f1f2400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:54.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.971+0000 7f1f227fc640 1 -- 172.21.15.83:0/2005363114 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2401a070 con 0x7f1f2c0b5ab0 2024-04-17T10:54:54.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.971+0000 7f1f227fc640 1 -- 172.21.15.83:0/2005363114 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2400de30 con 0x7f1f2c0b5ab0 2024-04-17T10:54:54.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.971+0000 7f1f341c4640 1 -- 172.21.15.83:0/2005363114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f2c0b47b0 con 0x7f1f2c0b5ab0 2024-04-17T10:54:54.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.971+0000 7f1f227fc640 1 -- 172.21.15.83:0/2005363114 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f24015640 con 0x7f1f2c0b5ab0 2024-04-17T10:54:54.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.971+0000 7f1f341c4640 1 -- 172.21.15.83:0/2005363114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f2c0b4d70 con 0x7f1f2c0b5ab0 2024-04-17T10:54:54.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.971+0000 7f1f341c4640 1 -- 172.21.15.83:0/2005363114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f2c074f90 con 0x7f1f2c0b5ab0 2024-04-17T10:54:54.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.971+0000 7f1f227fc640 1 -- 172.21.15.83:0/2005363114 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1f2401d070 con 0x7f1f2c0b5ab0 2024-04-17T10:54:54.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.971+0000 7f1f227fc640 1 --2- 172.21.15.83:0/2005363114 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1f10066f60 0x7f1f10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:54.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.975+0000 7f1f31f60640 1 --2- 172.21.15.83:0/2005363114 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1f10066f60 0x7f1f10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:54.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.975+0000 7f1f227fc640 1 -- 172.21.15.83:0/2005363114 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1f24084b60 con 0x7f1f2c0b5ab0 2024-04-17T10:54:54.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.975+0000 7f1f31f60640 1 --2- 172.21.15.83:0/2005363114 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1f10066f60 0x7f1f10069420 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f1f180057f0 tx=0x7f1f280011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:54.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:54.979+0000 7f1f227fc640 1 -- 172.21.15.83:0/2005363114 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1f24010070 con 0x7f1f2c0b5ab0 2024-04-17T10:54:55.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.123+0000 7f1f341c4640 1 -- 172.21.15.83:0/2005363114 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f2c0d1f00 con 0x7f1f10066f60 2024-04-17T10:54:55.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.123+0000 7f1f227fc640 1 -- 172.21.15.83:0/2005363114 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1f2c0d1f00 con 0x7f1f10066f60 2024-04-17T10:54:55.117 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 6 to deep-scrub 2024-04-17T10:54:55.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.127+0000 7f1f341c4640 1 -- 172.21.15.83:0/2005363114 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1f10066f60 msgr2=0x7f1f10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:55.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.127+0000 7f1f341c4640 1 --2- 172.21.15.83:0/2005363114 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1f10066f60 0x7f1f10069420 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f1f180057f0 tx=0x7f1f280011a0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.127+0000 7f1f341c4640 1 -- 172.21.15.83:0/2005363114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f2c0b5ab0 msgr2=0x7f1f2c0c6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:55.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.127+0000 7f1f341c4640 1 --2- 172.21.15.83:0/2005363114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f2c0b5ab0 0x7f1f2c0c6fb0 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c072410 tx=0x7f1f2400d4a0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.127+0000 7f1f341c4640 1 -- 172.21.15.83:0/2005363114 shutdown_connections 2024-04-17T10:54:55.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.127+0000 7f1f341c4640 1 --2- 172.21.15.83:0/2005363114 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1f10066f60 0x7f1f10069420 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.127+0000 7f1f341c4640 1 --2- 172.21.15.83:0/2005363114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f2c0b5ab0 0x7f1f2c0c6fb0 unknown :-1 s=CLOSED pgs=2508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.127+0000 7f1f341c4640 1 --2- 172.21.15.83:0/2005363114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f2c0b6bb0 0x7f1f2c0b5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.127+0000 7f1f341c4640 1 --2- 172.21.15.83:0/2005363114 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f18003a30 0x7f1f2c152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.127+0000 7f1f341c4640 1 -- 172.21.15.83:0/2005363114 >> 172.21.15.83:0/2005363114 conn(0x7f1f2c0d17a0 msgr2=0x7f1f2c0d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:55.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.127+0000 7f1f341c4640 1 -- 172.21.15.83:0/2005363114 shutdown_connections 2024-04-17T10:54:55.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.127+0000 7f1f341c4640 1 -- 172.21.15.83:0/2005363114 wait complete. 2024-04-17T10:54:55.135 INFO:tasks.ceph:Scrubbing osd.7 2024-04-17T10:54:55.135 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-17T10:54:55.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.267+0000 7f1bd21b8640 1 Processor -- start 2024-04-17T10:54:55.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.267+0000 7f1bd21b8640 1 -- start start 2024-04-17T10:54:55.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.267+0000 7f1bd21b8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1bcc0ced50 0x7f1bcc15b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:55.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.267+0000 7f1bd21b8640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bcc0cf3e0 con 0x7f1bcc06abe0 2024-04-17T10:54:55.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.267+0000 7f1bd21b8640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bcc0cd890 con 0x7f1bcc0aebd0 2024-04-17T10:54:55.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.267+0000 7f1bd21b8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc0cf1b0 con 0x7f1bcc0ced50 2024-04-17T10:54:55.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.267+0000 7f1bcbfff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1bcc06abe0 0x7f1bcc15ea50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42382/0 (socket says 172.21.15.83:42382) 2024-04-17T10:54:55.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.267+0000 7f1bcbfff640 1 -- 172.21.15.83:0/3398945800 learned_addr learned my addr 172.21.15.83:0/3398945800 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:55.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.267+0000 7f1bcaffd640 1 -- 172.21.15.83:0/3398945800 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 375468445 0 0) 0x7f1bcc0cf3e0 con 0x7f1bcc06abe0 2024-04-17T10:54:55.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.267+0000 7f1bc3fff640 1 --2- 172.21.15.83:0/3398945800 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1bcc0ced50 0x7f1bcc15b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:55.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.267+0000 7f1bcaffd640 1 -- 172.21.15.83:0/3398945800 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bb8003530 con 0x7f1bcc06abe0 2024-04-17T10:54:55.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.267+0000 7f1bcaffd640 1 -- 172.21.15.83:0/3398945800 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2263539011 0 0) 0x7f1bb8003530 con 0x7f1bcc06abe0 2024-04-17T10:54:55.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.267+0000 7f1bcaffd640 1 -- 172.21.15.83:0/3398945800 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1bcc0ced50 msgr2=0x7f1bcc15b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:55.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.267+0000 7f1bcaffd640 1 --2- 172.21.15.83:0/3398945800 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1bcc0ced50 0x7f1bcc15b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.267+0000 7f1bcaffd640 1 -- 172.21.15.83:0/3398945800 >> v1:172.21.15.167:6789/0 conn(0x7f1bcc0aebd0 legacy=0x7f1bcc0cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:55.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.267+0000 7f1bcaffd640 1 -- 172.21.15.83:0/3398945800 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bcc0b4230 con 0x7f1bcc06abe0 2024-04-17T10:54:55.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.267+0000 7f1bcaffd640 1 -- 172.21.15.83:0/3398945800 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7f1bc4001a00 con 0x7f1bcc06abe0 2024-04-17T10:54:55.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.267+0000 7f1bcaffd640 1 -- 172.21.15.83:0/3398945800 >> v1:172.21.15.83:6789/0 conn(0x7f1bcc06abe0 legacy=0x7f1bcc15ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:55.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.267+0000 7f1bcaffd640 1 --2- 172.21.15.83:0/3398945800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bb80039f0 0x7f1bb8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:55.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.267+0000 7f1bcaffd640 1 --2- 172.21.15.83:0/3398945800 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1bcc0ced50 0x7f1bb8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:55.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.267+0000 7f1bcaffd640 1 --2- 172.21.15.83:0/3398945800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bb8004d50 0x7f1bb80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:55.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.267+0000 7f1bcaffd640 1 -- 172.21.15.83:0/3398945800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb800bb60 con 0x7f1bb8004d50 2024-04-17T10:54:55.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.267+0000 7f1bcaffd640 1 -- 172.21.15.83:0/3398945800 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb800bce0 con 0x7f1bb80039f0 2024-04-17T10:54:55.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.267+0000 7f1bcaffd640 1 -- 172.21.15.83:0/3398945800 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb800e0f0 con 0x7f1bcc0ced50 2024-04-17T10:54:55.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.267+0000 7f1bcbfff640 1 --2- 172.21.15.83:0/3398945800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bb8004d50 0x7f1bb80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:55.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.271+0000 7f1bcb7fe640 1 --2- 172.21.15.83:0/3398945800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bb80039f0 0x7f1bb8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:55.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.271+0000 7f1bc3fff640 1 --2- 172.21.15.83:0/3398945800 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1bcc0ced50 0x7f1bb8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:55.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.271+0000 7f1bcbfff640 1 -- 172.21.15.83:0/3398945800 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1bcc0ced50 msgr2=0x7f1bb8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:55.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.271+0000 7f1bcbfff640 1 --2- 172.21.15.83:0/3398945800 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1bcc0ced50 0x7f1bb8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.271+0000 7f1bcbfff640 1 -- 172.21.15.83:0/3398945800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bb80039f0 msgr2=0x7f1bb8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:55.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.271+0000 7f1bcbfff640 1 --2- 172.21.15.83:0/3398945800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bb80039f0 0x7f1bb8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.271+0000 7f1bcbfff640 1 -- 172.21.15.83:0/3398945800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb8003530 con 0x7f1bb8004d50 2024-04-17T10:54:55.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.271+0000 7f1bcbfff640 1 --2- 172.21.15.83:0/3398945800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bb8004d50 0x7f1bb80051d0 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7f1bc400a860 tx=0x7f1bc401d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a50e103a68611542 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:55.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.271+0000 7f1bcaffd640 1 -- 172.21.15.83:0/3398945800 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc400a480 con 0x7f1bb8004d50 2024-04-17T10:54:55.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.271+0000 7f1bcaffd640 1 -- 172.21.15.83:0/3398945800 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc40ad630 con 0x7f1bb8004d50 2024-04-17T10:54:55.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.271+0000 7f1bcaffd640 1 -- 172.21.15.83:0/3398945800 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc40ada60 con 0x7f1bb8004d50 2024-04-17T10:54:55.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.271+0000 7f1bd21b8640 1 -- 172.21.15.83:0/3398945800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bb8004d50 msgr2=0x7f1bb80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:55.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.271+0000 7f1bd21b8640 1 --2- 172.21.15.83:0/3398945800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bb8004d50 0x7f1bb80051d0 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7f1bc400a860 tx=0x7f1bc401d380 comp rx=0 tx=0).stop 2024-04-17T10:54:55.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.271+0000 7f1bcbfff640 1 -- 172.21.15.83:0/3398945800 reap_dead start 2024-04-17T10:54:55.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.271+0000 7f1bd21b8640 1 -- 172.21.15.83:0/3398945800 shutdown_connections 2024-04-17T10:54:55.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.271+0000 7f1bd21b8640 1 -- 172.21.15.83:0/3398945800 >> 172.21.15.83:0/3398945800 conn(0x7f1bcc0cc050 msgr2=0x7f1bcc0cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:55.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.271+0000 7f1bd21b8640 1 -- 172.21.15.83:0/3398945800 shutdown_connections 2024-04-17T10:54:55.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.271+0000 7f1bd21b8640 1 -- 172.21.15.83:0/3398945800 wait complete. 2024-04-17T10:54:55.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.275+0000 7f1bd21b8640 1 Processor -- start 2024-04-17T10:54:55.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.275+0000 7f1bd21b8640 1 -- start start 2024-04-17T10:54:55.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.275+0000 7f1bd21b8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bb8004d50 0x7f1bcc149060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:55.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.275+0000 7f1bd21b8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1bcc1f6f60 0x7f1bcc15d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:55.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.275+0000 7f1bd21b8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bcc15dd30 0x7f1bcc155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:55.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.275+0000 7f1bd21b8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc0b47f0 con 0x7f1bb8004d50 2024-04-17T10:54:55.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.275+0000 7f1bd21b8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc15e5c0 con 0x7f1bcc15dd30 2024-04-17T10:54:55.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.275+0000 7f1bd21b8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc0aefb0 con 0x7f1bcc1f6f60 2024-04-17T10:54:55.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.275+0000 7f1bcb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bb8004d50 0x7f1bcc149060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:55.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.275+0000 7f1bcb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bb8004d50 0x7f1bcc149060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44620/0 (socket says 172.21.15.83:44620) 2024-04-17T10:54:55.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.275+0000 7f1bcb7fe640 1 -- 172.21.15.83:0/1400208875 learned_addr learned my addr 172.21.15.83:0/1400208875 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:55.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.275+0000 7f1bc3fff640 1 --2- 172.21.15.83:0/1400208875 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1bcc1f6f60 0x7f1bcc15d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:55.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.275+0000 7f1bcbfff640 1 --2- 172.21.15.83:0/1400208875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bcc15dd30 0x7f1bcc155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:55.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.275+0000 7f1bcb7fe640 1 -- 172.21.15.83:0/1400208875 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1bcc1f6f60 msgr2=0x7f1bcc15d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:55.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.275+0000 7f1bcb7fe640 1 --2- 172.21.15.83:0/1400208875 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1bcc1f6f60 0x7f1bcc15d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.275+0000 7f1bcb7fe640 1 -- 172.21.15.83:0/1400208875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bcc15dd30 msgr2=0x7f1bcc155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:55.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.275+0000 7f1bcb7fe640 1 --2- 172.21.15.83:0/1400208875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bcc15dd30 0x7f1bcc155190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.275+0000 7f1bcb7fe640 1 -- 172.21.15.83:0/1400208875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bcc06b240 con 0x7f1bb8004d50 2024-04-17T10:54:55.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.275+0000 7f1bc3fff640 1 --2- 172.21.15.83:0/1400208875 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1bcc1f6f60 0x7f1bcc15d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:55.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.275+0000 7f1bcbfff640 1 --2- 172.21.15.83:0/1400208875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bcc15dd30 0x7f1bcc155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:55.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.275+0000 7f1bcb7fe640 1 --2- 172.21.15.83:0/1400208875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bb8004d50 0x7f1bcc149060 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00e4e0 tx=0x7f1bbc00e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:55.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.275+0000 7f1bc8ff9640 1 -- 172.21.15.83:0/1400208875 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc0052e0 con 0x7f1bb8004d50 2024-04-17T10:54:55.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.275+0000 7f1bc8ff9640 1 -- 172.21.15.83:0/1400208875 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bbc018070 con 0x7f1bb8004d50 2024-04-17T10:54:55.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.275+0000 7f1bd21b8640 1 -- 172.21.15.83:0/1400208875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bcc1495a0 con 0x7f1bb8004d50 2024-04-17T10:54:55.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.275+0000 7f1bc8ff9640 1 -- 172.21.15.83:0/1400208875 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc003820 con 0x7f1bb8004d50 2024-04-17T10:54:55.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.275+0000 7f1bd21b8640 1 -- 172.21.15.83:0/1400208875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bcc1f7700 con 0x7f1bb8004d50 2024-04-17T10:54:55.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.275+0000 7f1bd21b8640 1 -- 172.21.15.83:0/1400208875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1bcc075000 con 0x7f1bb8004d50 2024-04-17T10:54:55.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.275+0000 7f1bc8ff9640 1 -- 172.21.15.83:0/1400208875 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1bbc0039c0 con 0x7f1bb8004d50 2024-04-17T10:54:55.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.275+0000 7f1bc8ff9640 1 --2- 172.21.15.83:0/1400208875 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1bac066f60 0x7f1bac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:55.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.279+0000 7f1bc8ff9640 1 -- 172.21.15.83:0/1400208875 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1bbc0849b0 con 0x7f1bb8004d50 2024-04-17T10:54:55.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.279+0000 7f1bc3fff640 1 --2- 172.21.15.83:0/1400208875 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1bac066f60 0x7f1bac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:55.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.279+0000 7f1bc8ff9640 1 --2- 172.21.15.83:0/1400208875 >> [v2:172.21.15.167:6808/285324536,v1:172.21.15.167:6809/285324536] conn(0x7f1bac06ce20 0x7f1bac06f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:55.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.279+0000 7f1bc8ff9640 1 -- 172.21.15.83:0/1400208875 --> [v2:172.21.15.167:6808/285324536,v1:172.21.15.167:6809/285324536] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1bbc003e50 con 0x7f1bac06ce20 2024-04-17T10:54:55.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.279+0000 7f1bc8ff9640 1 -- 172.21.15.83:0/1400208875 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=145) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bbc0892c0 con 0x7f1bb8004d50 2024-04-17T10:54:55.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.279+0000 7f1bc3fff640 1 --2- 172.21.15.83:0/1400208875 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1bac066f60 0x7f1bac069420 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc15a460 tx=0x7f1bb4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:55.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.279+0000 7f1bcbfff640 1 --2- 172.21.15.83:0/1400208875 >> [v2:172.21.15.167:6808/285324536,v1:172.21.15.167:6809/285324536] conn(0x7f1bac06ce20 0x7f1bac06f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:55.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.279+0000 7f1bcbfff640 1 --2- 172.21.15.83:0/1400208875 >> [v2:172.21.15.167:6808/285324536,v1:172.21.15.167:6809/285324536] conn(0x7f1bac06ce20 0x7f1bac06f240 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:55.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.279+0000 7f1bc8ff9640 1 -- 172.21.15.83:0/1400208875 <== osd.7 v2:172.21.15.167:6808/285324536 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f1bbc003e50 con 0x7f1bac06ce20 2024-04-17T10:54:55.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.291+0000 7f1bd21b8640 1 -- 172.21.15.83:0/1400208875 --> [v2:172.21.15.167:6808/285324536,v1:172.21.15.167:6809/285324536] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1bcc0cc550 con 0x7f1bac06ce20 2024-04-17T10:54:55.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.291+0000 7f1bc8ff9640 1 -- 172.21.15.83:0/1400208875 <== osd.7 v2:172.21.15.167:6808/285324536 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f1bcc0cc550 con 0x7f1bac06ce20 2024-04-17T10:54:55.287 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-17T10:54:55.287 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-17T10:54:55.287 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-17T10:54:55.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.295+0000 7f1bd21b8640 1 -- 172.21.15.83:0/1400208875 >> [v2:172.21.15.167:6808/285324536,v1:172.21.15.167:6809/285324536] conn(0x7f1bac06ce20 msgr2=0x7f1bac06f240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:55.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.295+0000 7f1bd21b8640 1 --2- 172.21.15.83:0/1400208875 >> [v2:172.21.15.167:6808/285324536,v1:172.21.15.167:6809/285324536] conn(0x7f1bac06ce20 0x7f1bac06f240 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.295+0000 7f1bd21b8640 1 -- 172.21.15.83:0/1400208875 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1bac066f60 msgr2=0x7f1bac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:55.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.295+0000 7f1bd21b8640 1 --2- 172.21.15.83:0/1400208875 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f1bac066f60 0x7f1bac069420 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc15a460 tx=0x7f1bb4008040 comp rx=0 tx=0).stop 2024-04-17T10:54:55.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.295+0000 7f1bd21b8640 1 -- 172.21.15.83:0/1400208875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bb8004d50 msgr2=0x7f1bcc149060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:55.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.295+0000 7f1bd21b8640 1 --2- 172.21.15.83:0/1400208875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bb8004d50 0x7f1bcc149060 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00e4e0 tx=0x7f1bbc00e9a0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.295+0000 7f1bcbfff640 1 -- 172.21.15.83:0/1400208875 reap_dead start 2024-04-17T10:54:55.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.295+0000 7f1bd21b8640 1 -- 172.21.15.83:0/1400208875 shutdown_connections 2024-04-17T10:54:55.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.295+0000 7f1bd21b8640 1 -- 172.21.15.83:0/1400208875 >> 172.21.15.83:0/1400208875 conn(0x7f1bcc0cc050 msgr2=0x7f1bcc158040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:55.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.295+0000 7f1bd21b8640 1 -- 172.21.15.83:0/1400208875 shutdown_connections 2024-04-17T10:54:55.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.295+0000 7f1bd21b8640 1 -- 172.21.15.83:0/1400208875 wait complete. 2024-04-17T10:54:55.302 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-17T10:54:55.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.411+0000 7f7b358b4640 1 Processor -- start 2024-04-17T10:54:55.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.411+0000 7f7b358b4640 1 -- start start 2024-04-17T10:54:55.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.415+0000 7f7b358b4640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7b3015bc40 0x7f7b300cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:55.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.415+0000 7f7b358b4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7b300d2330 0x7f7b300ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:55.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.415+0000 7f7b358b4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7b300ce6f0 0x7f7b300cc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:55.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.415+0000 7f7b358b4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b300cd7f0 con 0x7f7b300ce6f0 2024-04-17T10:54:55.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.415+0000 7f7b358b4640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b300d02c0 con 0x7f7b3015bc40 2024-04-17T10:54:55.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.415+0000 7f7b358b4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b300b1840 con 0x7f7b300d2330 2024-04-17T10:54:55.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.415+0000 7f7b2f7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7b300ce6f0 0x7f7b300cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:55.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.415+0000 7f7b2f7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7b300ce6f0 0x7f7b300cc490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44634/0 (socket says 172.21.15.83:44634) 2024-04-17T10:54:55.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.415+0000 7f7b2f7fe640 1 -- 172.21.15.83:0/1886833804 learned_addr learned my addr 172.21.15.83:0/1886833804 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:55.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.415+0000 7f7b2effd640 1 --2- 172.21.15.83:0/1886833804 >> v2:172.21.15.167:3300/0 conn(0x7f7b3015bc40 0x7f7b300cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:55.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.415+0000 7f7b2f7fe640 1 -- 172.21.15.83:0/1886833804 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7b300d2330 msgr2=0x7f7b300ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:55.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.415+0000 7f7b2f7fe640 1 --2- 172.21.15.83:0/1886833804 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7b300d2330 0x7f7b300ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.415+0000 7f7b2f7fe640 1 -- 172.21.15.83:0/1886833804 >> v2:172.21.15.167:3300/0 conn(0x7f7b3015bc40 msgr2=0x7f7b300cfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:55.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.415+0000 7f7b2f7fe640 1 --2- 172.21.15.83:0/1886833804 >> v2:172.21.15.167:3300/0 conn(0x7f7b3015bc40 0x7f7b300cfd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.415+0000 7f7b2f7fe640 1 -- 172.21.15.83:0/1886833804 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b300b1230 con 0x7f7b300ce6f0 2024-04-17T10:54:55.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.415+0000 7f7b2f7fe640 1 --2- 172.21.15.83:0/1886833804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b300ce6f0 0x7f7b300cc490 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7f7b3006b600 tx=0x7f7b2801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6c8ea5ae61a4d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:55.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b2dffb640 1 -- 172.21.15.83:0/1886833804 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b280a55e0 con 0x7f7b300ce6f0 2024-04-17T10:54:55.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b2dffb640 1 -- 172.21.15.83:0/1886833804 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b280a5780 con 0x7f7b300ce6f0 2024-04-17T10:54:55.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b2dffb640 1 -- 172.21.15.83:0/1886833804 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b280a5b50 con 0x7f7b300ce6f0 2024-04-17T10:54:55.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b358b4640 1 -- 172.21.15.83:0/1886833804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b300ce6f0 msgr2=0x7f7b300cc490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:55.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b358b4640 1 --2- 172.21.15.83:0/1886833804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b300ce6f0 0x7f7b300cc490 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7f7b3006b600 tx=0x7f7b2801c960 comp rx=0 tx=0).stop 2024-04-17T10:54:55.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b358b4640 1 -- 172.21.15.83:0/1886833804 shutdown_connections 2024-04-17T10:54:55.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b358b4640 1 --2- 172.21.15.83:0/1886833804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b300ce6f0 0x7f7b300cc490 unknown :-1 s=CLOSED pgs=2512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b358b4640 1 --2- 172.21.15.83:0/1886833804 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7b300d2330 0x7f7b300ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b358b4640 1 --2- 172.21.15.83:0/1886833804 >> v2:172.21.15.167:3300/0 conn(0x7f7b3015bc40 0x7f7b300cfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b358b4640 1 -- 172.21.15.83:0/1886833804 >> 172.21.15.83:0/1886833804 conn(0x7f7b300d17a0 msgr2=0x7f7b300d1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:55.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b358b4640 1 -- 172.21.15.83:0/1886833804 shutdown_connections 2024-04-17T10:54:55.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b358b4640 1 -- 172.21.15.83:0/1886833804 wait complete. 2024-04-17T10:54:55.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b358b4640 1 Processor -- start 2024-04-17T10:54:55.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b358b4640 1 -- start start 2024-04-17T10:54:55.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b358b4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b300ce6f0 0x7f7b30149760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:55.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b358b4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7b300d2330 0x7f7b300c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:55.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b358b4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b3015bc40 0x7f7b301f6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:55.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b358b4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b300b1e30 con 0x7f7b3015bc40 2024-04-17T10:54:55.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b358b4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b300b1cb0 con 0x7f7b300ce6f0 2024-04-17T10:54:55.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b358b4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b300b1fb0 con 0x7f7b300d2330 2024-04-17T10:54:55.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b2f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b3015bc40 0x7f7b301f6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b2f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b3015bc40 0x7f7b301f6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44640/0 (socket says 172.21.15.83:44640) 2024-04-17T10:54:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b2f7fe640 1 -- 172.21.15.83:0/4007780539 learned_addr learned my addr 172.21.15.83:0/4007780539 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b2effd640 1 --2- 172.21.15.83:0/4007780539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b300ce6f0 0x7f7b30149760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b2e7fc640 1 --2- 172.21.15.83:0/4007780539 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7b300d2330 0x7f7b300c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b2f7fe640 1 -- 172.21.15.83:0/4007780539 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7b300d2330 msgr2=0x7f7b300c5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b2f7fe640 1 --2- 172.21.15.83:0/4007780539 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7b300d2330 0x7f7b300c5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b2f7fe640 1 -- 172.21.15.83:0/4007780539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b300ce6f0 msgr2=0x7f7b30149760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b2f7fe640 1 --2- 172.21.15.83:0/4007780539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b300ce6f0 0x7f7b30149760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b2f7fe640 1 -- 172.21.15.83:0/4007780539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b300c5990 con 0x7f7b3015bc40 2024-04-17T10:54:55.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b2f7fe640 1 --2- 172.21.15.83:0/4007780539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b3015bc40 0x7f7b301f6010 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7f7b2801b010 tx=0x7f7b28019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:55.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b17fff640 1 -- 172.21.15.83:0/4007780539 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b280bc020 con 0x7f7b3015bc40 2024-04-17T10:54:55.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b17fff640 1 -- 172.21.15.83:0/4007780539 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b280b8690 con 0x7f7b3015bc40 2024-04-17T10:54:55.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b358b4640 1 -- 172.21.15.83:0/4007780539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b300c5c70 con 0x7f7b3015bc40 2024-04-17T10:54:55.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b17fff640 1 -- 172.21.15.83:0/4007780539 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b28009510 con 0x7f7b3015bc40 2024-04-17T10:54:55.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.419+0000 7f7b358b4640 1 -- 172.21.15.83:0/4007780539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b3014a1d0 con 0x7f7b3015bc40 2024-04-17T10:54:55.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.423+0000 7f7b358b4640 1 -- 172.21.15.83:0/4007780539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b300c5e40 con 0x7f7b3015bc40 2024-04-17T10:54:55.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.423+0000 7f7b17fff640 1 -- 172.21.15.83:0/4007780539 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7b280bf020 con 0x7f7b3015bc40 2024-04-17T10:54:55.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.423+0000 7f7b17fff640 1 --2- 172.21.15.83:0/4007780539 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7b1c066f60 0x7f7b1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:55.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.423+0000 7f7b17fff640 1 -- 172.21.15.83:0/4007780539 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7b2811fe00 con 0x7f7b3015bc40 2024-04-17T10:54:55.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.423+0000 7f7b2effd640 1 --2- 172.21.15.83:0/4007780539 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7b1c066f60 0x7f7b1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:55.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.423+0000 7f7b2effd640 1 --2- 172.21.15.83:0/4007780539 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7b1c066f60 0x7f7b1c069420 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000a8d0 tx=0x7f7b20008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:55.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.427+0000 7f7b17fff640 1 -- 172.21.15.83:0/4007780539 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7b280c6030 con 0x7f7b3015bc40 2024-04-17T10:54:55.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.587+0000 7f7b358b4640 1 -- 172.21.15.83:0/4007780539 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b30149d40 con 0x7f7b1c066f60 2024-04-17T10:54:55.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.587+0000 7f7b17fff640 1 -- 172.21.15.83:0/4007780539 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7b30149d40 con 0x7f7b1c066f60 2024-04-17T10:54:55.581 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 7 to deep-scrub 2024-04-17T10:54:55.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.587+0000 7f7b358b4640 1 -- 172.21.15.83:0/4007780539 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7b1c066f60 msgr2=0x7f7b1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:55.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.587+0000 7f7b358b4640 1 --2- 172.21.15.83:0/4007780539 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7b1c066f60 0x7f7b1c069420 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000a8d0 tx=0x7f7b20008040 comp rx=0 tx=0).stop 2024-04-17T10:54:55.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.587+0000 7f7b358b4640 1 -- 172.21.15.83:0/4007780539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b3015bc40 msgr2=0x7f7b301f6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:55.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.587+0000 7f7b358b4640 1 --2- 172.21.15.83:0/4007780539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b3015bc40 0x7f7b301f6010 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7f7b2801b010 tx=0x7f7b28019700 comp rx=0 tx=0).stop 2024-04-17T10:54:55.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.587+0000 7f7b358b4640 1 -- 172.21.15.83:0/4007780539 shutdown_connections 2024-04-17T10:54:55.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.587+0000 7f7b358b4640 1 --2- 172.21.15.83:0/4007780539 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f7b1c066f60 0x7f7b1c069420 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.587+0000 7f7b358b4640 1 --2- 172.21.15.83:0/4007780539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b3015bc40 0x7f7b301f6010 unknown :-1 s=CLOSED pgs=2513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.587+0000 7f7b358b4640 1 --2- 172.21.15.83:0/4007780539 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7b300d2330 0x7f7b300c5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.587+0000 7f7b358b4640 1 --2- 172.21.15.83:0/4007780539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b300ce6f0 0x7f7b30149760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.591+0000 7f7b358b4640 1 -- 172.21.15.83:0/4007780539 >> 172.21.15.83:0/4007780539 conn(0x7f7b300d17a0 msgr2=0x7f7b30154790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:55.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.591+0000 7f7b358b4640 1 -- 172.21.15.83:0/4007780539 shutdown_connections 2024-04-17T10:54:55.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.591+0000 7f7b358b4640 1 -- 172.21.15.83:0/4007780539 wait complete. 2024-04-17T10:54:55.599 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:54:55.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.711+0000 7f0a92954640 1 Processor -- start 2024-04-17T10:54:55.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.711+0000 7f0a92954640 1 -- start start 2024-04-17T10:54:55.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.711+0000 7f0a92954640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a8c0ce980 0x7f0a8c0b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:55.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.711+0000 7f0a92954640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0a8c0cee00 0x7f0a8c0b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:55.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.715+0000 7f0a92954640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a8c162a00 con 0x7f0a8c15dfa0 2024-04-17T10:54:55.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.715+0000 7f0a92954640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a8c161e80 con 0x7f0a8c0cee00 2024-04-17T10:54:55.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.715+0000 7f0a92954640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c162160 con 0x7f0a8c0ce980 2024-04-17T10:54:55.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.715+0000 7f0a8bfff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0a8c15dfa0 0x7f0a8c0b6c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:42388/0 (socket says 172.21.15.83:42388) 2024-04-17T10:54:55.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.715+0000 7f0a8bfff640 1 -- 172.21.15.83:0/1962441979 learned_addr learned my addr 172.21.15.83:0/1962441979 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:54:55.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.715+0000 7f0a90ef1640 1 --2- 172.21.15.83:0/1962441979 >> v2:172.21.15.83:3300/0 conn(0x7f0a8c0cee00 0x7f0a8c0b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:55.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.715+0000 7f0a8affd640 1 -- 172.21.15.83:0/1962441979 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 315269765 0 0) 0x7f0a8c162a00 con 0x7f0a8c15dfa0 2024-04-17T10:54:55.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.715+0000 7f0a8b7fe640 1 --2- 172.21.15.83:0/1962441979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a8c0ce980 0x7f0a8c0b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:55.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.715+0000 7f0a8affd640 1 -- 172.21.15.83:0/1962441979 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a74003530 con 0x7f0a8c15dfa0 2024-04-17T10:54:55.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.715+0000 7f0a90ef1640 1 -- 172.21.15.83:0/1962441979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a8c0ce980 msgr2=0x7f0a8c0b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:54:55.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.715+0000 7f0a90ef1640 1 --2- 172.21.15.83:0/1962441979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a8c0ce980 0x7f0a8c0b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.715+0000 7f0a90ef1640 1 -- 172.21.15.83:0/1962441979 >> v1:172.21.15.83:6789/0 conn(0x7f0a8c15dfa0 legacy=0x7f0a8c0b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:55.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.715+0000 7f0a90ef1640 1 -- 172.21.15.83:0/1962441979 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a8c162440 con 0x7f0a8c0cee00 2024-04-17T10:54:55.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.715+0000 7f0a8b7fe640 1 --2- 172.21.15.83:0/1962441979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a8c0ce980 0x7f0a8c0b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:54:55.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.715+0000 7f0a90ef1640 1 --2- 172.21.15.83:0/1962441979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a8c0cee00 0x7f0a8c0b5970 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009d90 tx=0x7f0a8401c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263af713397acf01 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:55.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.715+0000 7f0a8affd640 1 -- 172.21.15.83:0/1962441979 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a840b2070 con 0x7f0a8c0cee00 2024-04-17T10:54:55.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.715+0000 7f0a8affd640 1 -- 172.21.15.83:0/1962441979 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a840a5910 con 0x7f0a8c0cee00 2024-04-17T10:54:55.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.715+0000 7f0a8affd640 1 -- 172.21.15.83:0/1962441979 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a840a5ce0 con 0x7f0a8c0cee00 2024-04-17T10:54:55.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.715+0000 7f0a92954640 1 -- 172.21.15.83:0/1962441979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a8c0cee00 msgr2=0x7f0a8c0b5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:55.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.715+0000 7f0a92954640 1 --2- 172.21.15.83:0/1962441979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a8c0cee00 0x7f0a8c0b5970 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009d90 tx=0x7f0a8401c8b0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.715+0000 7f0a92954640 1 -- 172.21.15.83:0/1962441979 shutdown_connections 2024-04-17T10:54:55.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.715+0000 7f0a92954640 1 --2- 172.21.15.83:0/1962441979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a8c0cee00 0x7f0a8c0b5970 unknown :-1 s=CLOSED pgs=2515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.715+0000 7f0a92954640 1 --2- 172.21.15.83:0/1962441979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a8c0ce980 0x7f0a8c0b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.715+0000 7f0a92954640 1 -- 172.21.15.83:0/1962441979 >> 172.21.15.83:0/1962441979 conn(0x7f0a8c0d0600 msgr2=0x7f0a8c0d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:55.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.715+0000 7f0a92954640 1 -- 172.21.15.83:0/1962441979 shutdown_connections 2024-04-17T10:54:55.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.715+0000 7f0a92954640 1 -- 172.21.15.83:0/1962441979 wait complete. 2024-04-17T10:54:55.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.719+0000 7f0a92954640 1 Processor -- start 2024-04-17T10:54:55.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.719+0000 7f0a92954640 1 -- start start 2024-04-17T10:54:55.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.719+0000 7f0a92954640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a8c0ce980 0x7f0a8c150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:55.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.719+0000 7f0a92954640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a8c0cee00 0x7f0a8c0c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:55.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.719+0000 7f0a92954640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a8c15dfa0 0x7f0a8c0cac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:55.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.719+0000 7f0a92954640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c0d26c0 con 0x7f0a8c0ce980 2024-04-17T10:54:55.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.719+0000 7f0a92954640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c0d2540 con 0x7f0a8c15dfa0 2024-04-17T10:54:55.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.719+0000 7f0a92954640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c0d2840 con 0x7f0a8c0cee00 2024-04-17T10:54:55.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.719+0000 7f0a8bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a8c0ce980 0x7f0a8c150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:55.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.719+0000 7f0a8bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a8c0ce980 0x7f0a8c150950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44660/0 (socket says 172.21.15.83:44660) 2024-04-17T10:54:55.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.719+0000 7f0a8bfff640 1 -- 172.21.15.83:0/591203126 learned_addr learned my addr 172.21.15.83:0/591203126 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:54:55.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.719+0000 7f0a8b7fe640 1 --2- 172.21.15.83:0/591203126 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a8c0cee00 0x7f0a8c0c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:55.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.719+0000 7f0a8bfff640 1 -- 172.21.15.83:0/591203126 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a8c0cee00 msgr2=0x7f0a8c0c51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:55.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.719+0000 7f0a8bfff640 1 --2- 172.21.15.83:0/591203126 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a8c0cee00 0x7f0a8c0c51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.719+0000 7f0a8bfff640 1 -- 172.21.15.83:0/591203126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a8c15dfa0 msgr2=0x7f0a8c0cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:55.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.719+0000 7f0a8bfff640 1 --2- 172.21.15.83:0/591203126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a8c15dfa0 0x7f0a8c0cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.719+0000 7f0a8bfff640 1 -- 172.21.15.83:0/591203126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a8c14f5a0 con 0x7f0a8c0ce980 2024-04-17T10:54:55.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.719+0000 7f0a8bfff640 1 --2- 172.21.15.83:0/591203126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a8c0ce980 0x7f0a8c150950 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00c450 tx=0x7f0a7c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:55.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.719+0000 7f0a88ff9640 1 -- 172.21.15.83:0/591203126 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c017070 con 0x7f0a8c0ce980 2024-04-17T10:54:55.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.719+0000 7f0a88ff9640 1 -- 172.21.15.83:0/591203126 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a7c005cf0 con 0x7f0a8c0ce980 2024-04-17T10:54:55.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.719+0000 7f0a92954640 1 -- 172.21.15.83:0/591203126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a8c0cd570 con 0x7f0a8c0ce980 2024-04-17T10:54:55.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.719+0000 7f0a88ff9640 1 -- 172.21.15.83:0/591203126 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c007290 con 0x7f0a8c0ce980 2024-04-17T10:54:55.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.719+0000 7f0a92954640 1 -- 172.21.15.83:0/591203126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a8c0cd850 con 0x7f0a8c0ce980 2024-04-17T10:54:55.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.719+0000 7f0a92954640 1 -- 172.21.15.83:0/591203126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a8c074f90 con 0x7f0a8c0ce980 2024-04-17T10:54:55.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.719+0000 7f0a88ff9640 1 -- 172.21.15.83:0/591203126 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0a7c01a070 con 0x7f0a8c0ce980 2024-04-17T10:54:55.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.723+0000 7f0a88ff9640 1 --2- 172.21.15.83:0/591203126 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0a78066f60 0x7f0a78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:54:55.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.723+0000 7f0a8b7fe640 1 --2- 172.21.15.83:0/591203126 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0a78066f60 0x7f0a78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:54:55.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.723+0000 7f0a88ff9640 1 -- 172.21.15.83:0/591203126 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0a7c0841d0 con 0x7f0a8c0ce980 2024-04-17T10:54:55.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.723+0000 7f0a8b7fe640 1 --2- 172.21.15.83:0/591203126 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0a78066f60 0x7f0a78069420 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f0a80006fd0 tx=0x7f0a80008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:54:55.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.727+0000 7f0a88ff9640 1 -- 172.21.15.83:0/591203126 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0a7c055450 con 0x7f0a8c0ce980 2024-04-17T10:54:55.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.867+0000 7f0a92954640 1 -- 172.21.15.83:0/591203126 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0a8c0b5a20 con 0x7f0a78066f60 2024-04-17T10:54:55.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.867+0000 7f0a88ff9640 1 -- 172.21.15.83:0/591203126 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13480 (secure 0 0 0) 0x7f0a8c0b5a20 con 0x7f0a78066f60 2024-04-17T10:54:55.863 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:54:55.863 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:54:55.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.871+0000 7f0a92954640 1 -- 172.21.15.83:0/591203126 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0a78066f60 msgr2=0x7f0a78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:55.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.871+0000 7f0a92954640 1 --2- 172.21.15.83:0/591203126 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0a78066f60 0x7f0a78069420 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f0a80006fd0 tx=0x7f0a80008040 comp rx=0 tx=0).stop 2024-04-17T10:54:55.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.871+0000 7f0a92954640 1 -- 172.21.15.83:0/591203126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a8c0ce980 msgr2=0x7f0a8c150950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:54:55.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.871+0000 7f0a92954640 1 --2- 172.21.15.83:0/591203126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a8c0ce980 0x7f0a8c150950 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00c450 tx=0x7f0a7c00c910 comp rx=0 tx=0).stop 2024-04-17T10:54:55.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.871+0000 7f0a92954640 1 -- 172.21.15.83:0/591203126 shutdown_connections 2024-04-17T10:54:55.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.871+0000 7f0a92954640 1 --2- 172.21.15.83:0/591203126 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f0a78066f60 0x7f0a78069420 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.871+0000 7f0a92954640 1 --2- 172.21.15.83:0/591203126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a8c15dfa0 0x7f0a8c0cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.871+0000 7f0a92954640 1 --2- 172.21.15.83:0/591203126 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a8c0cee00 0x7f0a8c0c51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.871+0000 7f0a92954640 1 --2- 172.21.15.83:0/591203126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a8c0ce980 0x7f0a8c150950 unknown :-1 s=CLOSED pgs=2516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:54:55.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.871+0000 7f0a92954640 1 -- 172.21.15.83:0/591203126 >> 172.21.15.83:0/591203126 conn(0x7f0a8c0d0600 msgr2=0x7f0a8c151470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:54:55.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.871+0000 7f0a92954640 1 -- 172.21.15.83:0/591203126 shutdown_connections 2024-04-17T10:54:55.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:54:55.871+0000 7f0a92954640 1 -- 172.21.15.83:0/591203126 wait complete. 2024-04-17T10:54:55.880 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":646,"stamp":"2024-04-17T10:54:54.486996+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":769992,"kb_used_data":273432,"kb_used_omap":12,"kb_used_meta":496435,"kb_avail":754204728,"statfs":{"total":773094113280,"available":772305641472,"internally_reserved":0,"allocated":279994368,"data_stored":271804610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":508350000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1076459209,"num_objects":-1046,"num_object_clones":0,"num_object_copies":-2092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-1128,"num_write_kb":-1051360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-11,"num_bytes_recovered":-11534336,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001895"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":428,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-17T10:54:47.637200+0000","last_change":"2024-04-17T10:38:02.077409+0000","last_active":"2024-04-17T10:54:47.637200+0000","last_peered":"2024-04-17T10:54:47.637200+0000","last_clean":"2024-04-17T10:54:47.637200+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:54:47.637200+0000","last_undegraded":"2024-04-17T10:54:47.637200+0000","last_fullsized":"2024-04-17T10:54:47.637200+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:38:00.559745+0000","last_clean_scrub_stamp":"2024-04-17T10:38:00.559745+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:23.052288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575055,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51592,"kb_used_data":2360,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320248,"statfs":{"total":96636764160,"available":96583933952,"internally_reserved":0,"allocated":2416640,"data_stored":1388906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575053,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":331248,"kb_used_data":259812,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94040592,"statfs":{"total":96636764160,"available":96297566208,"internally_reserved":0,"allocated":266047488,"data_stored":265034108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575055,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59332,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312508,"statfs":{"total":96636764160,"available":96576008192,"internally_reserved":0,"allocated":1822720,"data_stored":798538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575054,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":73216,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298624,"statfs":{"total":96636764160,"available":96561790976,"internally_reserved":0,"allocated":1822720,"data_stored":798538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575056,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":73216,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298624,"statfs":{"total":96636764160,"available":96561790976,"internally_reserved":0,"allocated":1822720,"data_stored":798538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575054,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46848,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324992,"statfs":{"total":96636764160,"available":96588791808,"internally_reserved":0,"allocated":1822720,"data_stored":798538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575054,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":82948,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288892,"statfs":{"total":96636764160,"available":96551825408,"internally_reserved":0,"allocated":1822720,"data_stored":798538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575054,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51592,"kb_used_data":2360,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320248,"statfs":{"total":96636764160,"available":96583933952,"internally_reserved":0,"allocated":2416640,"data_stored":1388906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:54:55.883 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-17T10:38:00.559745+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=10, tm_min=38, tm_sec=0, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=10, tm_min=54, tm_sec=50, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T10:54:55.884 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-17T10:55:09.863 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:55:09.866 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:55:15.885 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:55:16.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.022+0000 7f8de2622640 1 Processor -- start 2024-04-17T10:55:16.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.022+0000 7f8de2622640 1 -- start start 2024-04-17T10:55:16.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.026+0000 7f8de2622640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8ddc15ddd0 0x7f8ddc0d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:55:16.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.026+0000 7f8de2622640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ddc0b2f70 0x7f8ddc0d2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:55:16.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.026+0000 7f8de2622640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ddc0d18c0 con 0x7f8ddc06b740 2024-04-17T10:55:16.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.026+0000 7f8de2622640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ddc0d1e40 con 0x7f8ddc15ddd0 2024-04-17T10:55:16.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.026+0000 7f8de2622640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc0d1b60 con 0x7f8ddc0b2f70 2024-04-17T10:55:16.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.026+0000 7f8ddbfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8ddc15ddd0 0x7f8ddc0d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:55:16.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.026+0000 7f8de0bbf640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8ddc06b740 0x7f8ddc06bbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34758/0 (socket says 172.21.15.83:34758) 2024-04-17T10:55:16.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.026+0000 7f8de0bbf640 1 -- 172.21.15.83:0/1795565366 learned_addr learned my addr 172.21.15.83:0/1795565366 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:55:16.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.026+0000 7f8ddb7fe640 1 --2- 172.21.15.83:0/1795565366 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ddc0b2f70 0x7f8ddc0d2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:55:16.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.026+0000 7f8ddbfff640 1 -- 172.21.15.83:0/1795565366 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ddc0b2f70 msgr2=0x7f8ddc0d2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:55:16.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.026+0000 7f8ddbfff640 1 --2- 172.21.15.83:0/1795565366 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ddc0b2f70 0x7f8ddc0d2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:55:16.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.026+0000 7f8ddbfff640 1 -- 172.21.15.83:0/1795565366 >> v1:172.21.15.83:6789/0 conn(0x7f8ddc06b740 legacy=0x7f8ddc06bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:55:16.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.026+0000 7f8ddbfff640 1 -- 172.21.15.83:0/1795565366 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ddc06b220 con 0x7f8ddc15ddd0 2024-04-17T10:55:16.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.026+0000 7f8ddaffd640 1 -- 172.21.15.83:0/1795565366 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3717545784 0 0) 0x7f8ddc0d18c0 con 0x7f8ddc06b740 2024-04-17T10:55:16.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.026+0000 7f8ddbfff640 1 --2- 172.21.15.83:0/1795565366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ddc15ddd0 0x7f8ddc0d2020 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc0097c0 tx=0x7f8dcc01c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f4a10a471fa72e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:55:16.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.026+0000 7f8ddaffd640 1 -- 172.21.15.83:0/1795565366 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc0b0070 con 0x7f8ddc15ddd0 2024-04-17T10:55:16.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.026+0000 7f8ddaffd640 1 -- 172.21.15.83:0/1795565366 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dcc0ab600 con 0x7f8ddc15ddd0 2024-04-17T10:55:16.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.026+0000 7f8ddaffd640 1 -- 172.21.15.83:0/1795565366 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc0ab9d0 con 0x7f8ddc15ddd0 2024-04-17T10:55:16.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.030+0000 7f8de2622640 1 -- 172.21.15.83:0/1795565366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ddc15ddd0 msgr2=0x7f8ddc0d2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:55:16.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.030+0000 7f8de2622640 1 --2- 172.21.15.83:0/1795565366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ddc15ddd0 0x7f8ddc0d2020 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc0097c0 tx=0x7f8dcc01c320 comp rx=0 tx=0).stop 2024-04-17T10:55:16.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.030+0000 7f8de2622640 1 -- 172.21.15.83:0/1795565366 shutdown_connections 2024-04-17T10:55:16.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.030+0000 7f8de2622640 1 --2- 172.21.15.83:0/1795565366 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ddc0b2f70 0x7f8ddc0d2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:55:16.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.030+0000 7f8de2622640 1 --2- 172.21.15.83:0/1795565366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ddc15ddd0 0x7f8ddc0d2020 unknown :-1 s=CLOSED pgs=2518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:55:16.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.030+0000 7f8de2622640 1 -- 172.21.15.83:0/1795565366 >> 172.21.15.83:0/1795565366 conn(0x7f8ddc0d3a40 msgr2=0x7f8ddc0d3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:55:16.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.030+0000 7f8de2622640 1 -- 172.21.15.83:0/1795565366 shutdown_connections 2024-04-17T10:55:16.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.030+0000 7f8de2622640 1 -- 172.21.15.83:0/1795565366 wait complete. 2024-04-17T10:55:16.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.030+0000 7f8de2622640 1 Processor -- start 2024-04-17T10:55:16.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.030+0000 7f8de2622640 1 -- start start 2024-04-17T10:55:16.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.030+0000 7f8de2622640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ddc06b740 0x7f8ddc0c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:55:16.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.030+0000 7f8de2622640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ddc0b2f70 0x7f8ddc155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:55:16.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.030+0000 7f8de2622640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ddc15ddd0 0x7f8ddc15bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:55:16.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.030+0000 7f8de2622640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc16aa50 con 0x7f8ddc06b740 2024-04-17T10:55:16.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.030+0000 7f8de2622640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc0cfa00 con 0x7f8ddc15ddd0 2024-04-17T10:55:16.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.030+0000 7f8de2622640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc16abd0 con 0x7f8ddc0b2f70 2024-04-17T10:55:16.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.030+0000 7f8ddbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ddc06b740 0x7f8ddc0c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:55:16.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.030+0000 7f8ddbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ddc06b740 0x7f8ddc0c7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51432/0 (socket says 172.21.15.83:51432) 2024-04-17T10:55:16.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.030+0000 7f8ddbfff640 1 -- 172.21.15.83:0/3456236431 learned_addr learned my addr 172.21.15.83:0/3456236431 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:55:16.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.030+0000 7f8ddb7fe640 1 --2- 172.21.15.83:0/3456236431 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ddc0b2f70 0x7f8ddc155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:55:16.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.030+0000 7f8ddbfff640 1 -- 172.21.15.83:0/3456236431 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ddc0b2f70 msgr2=0x7f8ddc155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:55:16.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.030+0000 7f8ddbfff640 1 --2- 172.21.15.83:0/3456236431 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ddc0b2f70 0x7f8ddc155070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:55:16.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.030+0000 7f8de0bbf640 1 --2- 172.21.15.83:0/3456236431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ddc15ddd0 0x7f8ddc15bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:55:16.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.030+0000 7f8ddbfff640 1 -- 172.21.15.83:0/3456236431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ddc15ddd0 msgr2=0x7f8ddc15bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:55:16.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.030+0000 7f8ddbfff640 1 --2- 172.21.15.83:0/3456236431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ddc15ddd0 0x7f8ddc15bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:55:16.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.030+0000 7f8ddbfff640 1 -- 172.21.15.83:0/3456236431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ddc0c84c0 con 0x7f8ddc06b740 2024-04-17T10:55:16.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.030+0000 7f8ddbfff640 1 --2- 172.21.15.83:0/3456236431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ddc06b740 0x7f8ddc0c7f80 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc0ae040 tx=0x7f8dcc01abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:55:16.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.030+0000 7f8dd8ff9640 1 -- 172.21.15.83:0/3456236431 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc0b0050 con 0x7f8ddc06b740 2024-04-17T10:55:16.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.030+0000 7f8dd8ff9640 1 -- 172.21.15.83:0/3456236431 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dcc0bc070 con 0x7f8ddc06b740 2024-04-17T10:55:16.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.030+0000 7f8de2622640 1 -- 172.21.15.83:0/3456236431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ddc155c40 con 0x7f8ddc06b740 2024-04-17T10:55:16.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.030+0000 7f8dd8ff9640 1 -- 172.21.15.83:0/3456236431 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc0b5a70 con 0x7f8ddc06b740 2024-04-17T10:55:16.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.030+0000 7f8de2622640 1 -- 172.21.15.83:0/3456236431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ddc156200 con 0x7f8ddc06b740 2024-04-17T10:55:16.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.034+0000 7f8de2622640 1 -- 172.21.15.83:0/3456236431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ddc074f90 con 0x7f8ddc06b740 2024-04-17T10:55:16.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.034+0000 7f8dd8ff9640 1 -- 172.21.15.83:0/3456236431 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8dcc0ab4c0 con 0x7f8ddc06b740 2024-04-17T10:55:16.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.034+0000 7f8dd8ff9640 1 --2- 172.21.15.83:0/3456236431 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8dc8066f60 0x7f8dc8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:55:16.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.034+0000 7f8dd8ff9640 1 -- 172.21.15.83:0/3456236431 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8dcc11f4b0 con 0x7f8ddc06b740 2024-04-17T10:55:16.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.034+0000 7f8ddb7fe640 1 --2- 172.21.15.83:0/3456236431 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8dc8066f60 0x7f8dc8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:55:16.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.034+0000 7f8ddb7fe640 1 --2- 172.21.15.83:0/3456236431 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8dc8066f60 0x7f8dc8069420 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc15fb50 tx=0x7f8dd0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:55:16.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.038+0000 7f8dd8ff9640 1 -- 172.21.15.83:0/3456236431 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8ddc0c84c0 con 0x7f8ddc06b740 2024-04-17T10:55:16.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.194+0000 7f8de2622640 1 -- 172.21.15.83:0/3456236431 --> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ddc0ca5a0 con 0x7f8dc8066f60 2024-04-17T10:55:16.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.198+0000 7f8dd8ff9640 1 -- 172.21.15.83:0/3456236431 <== mgr.4101 v2:172.21.15.83:6832/2577814962 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13380 (secure 0 0 0) 0x7f8ddc0ca5a0 con 0x7f8dc8066f60 2024-04-17T10:55:16.192 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:55:16.192 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-17T10:55:16.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.198+0000 7f8de2622640 1 -- 172.21.15.83:0/3456236431 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8dc8066f60 msgr2=0x7f8dc8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:55:16.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.198+0000 7f8de2622640 1 --2- 172.21.15.83:0/3456236431 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8dc8066f60 0x7f8dc8069420 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc15fb50 tx=0x7f8dd0008040 comp rx=0 tx=0).stop 2024-04-17T10:55:16.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.198+0000 7f8de2622640 1 -- 172.21.15.83:0/3456236431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ddc06b740 msgr2=0x7f8ddc0c7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:55:16.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.198+0000 7f8de2622640 1 --2- 172.21.15.83:0/3456236431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ddc06b740 0x7f8ddc0c7f80 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc0ae040 tx=0x7f8dcc01abc0 comp rx=0 tx=0).stop 2024-04-17T10:55:16.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.198+0000 7f8de2622640 1 -- 172.21.15.83:0/3456236431 shutdown_connections 2024-04-17T10:55:16.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.198+0000 7f8de2622640 1 --2- 172.21.15.83:0/3456236431 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f8dc8066f60 0x7f8dc8069420 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:55:16.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.198+0000 7f8de2622640 1 --2- 172.21.15.83:0/3456236431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ddc15ddd0 0x7f8ddc15bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:55:16.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.198+0000 7f8de2622640 1 --2- 172.21.15.83:0/3456236431 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ddc0b2f70 0x7f8ddc155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:55:16.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.198+0000 7f8de2622640 1 --2- 172.21.15.83:0/3456236431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ddc06b740 0x7f8ddc0c7f80 unknown :-1 s=CLOSED pgs=2519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:55:16.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.198+0000 7f8de2622640 1 -- 172.21.15.83:0/3456236431 >> 172.21.15.83:0/3456236431 conn(0x7f8ddc0d3a40 msgr2=0x7f8ddc0b3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:55:16.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.198+0000 7f8de2622640 1 -- 172.21.15.83:0/3456236431 shutdown_connections 2024-04-17T10:55:16.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.198+0000 7f8de2622640 1 -- 172.21.15.83:0/3456236431 wait complete. 2024-04-17T10:55:16.211 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":656,"stamp":"2024-04-17T10:55:14.492036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":511944,"kb_used_data":15400,"kb_used_omap":12,"kb_used_meta":496435,"kb_avail":754462776,"statfs":{"total":773094113280,"available":772569882624,"internally_reserved":0,"allocated":15769600,"data_stored":7569040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":508350000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003114"},"pg_stats":[{"pgid":"1.0","version":"145'119","reported_seq":439,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-17T10:54:55.998836+0000","last_change":"2024-04-17T10:54:55.998919+0000","last_active":"2024-04-17T10:54:55.998836+0000","last_peered":"2024-04-17T10:54:55.998836+0000","last_clean":"2024-04-17T10:54:55.998836+0000","last_became_active":"2024-04-17T10:38:02.075010+0000","last_became_peered":"2024-04-17T10:38:02.075010+0000","last_unstale":"2024-04-17T10:54:55.998836+0000","last_undegraded":"2024-04-17T10:54:55.998836+0000","last_fullsized":"2024-04-17T10:54:55.998836+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"145'119","last_scrub_stamp":"2024-04-17T10:54:55.998636+0000","last_deep_scrub":"145'119","last_deep_scrub_stamp":"2024-04-17T10:54:55.998636+0000","last_clean_scrub_stamp":"2024-04-17T10:54:55.998636+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:43:10.306383+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575059,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51584,"kb_used_data":2360,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320256,"statfs":{"total":96636764160,"available":96583942144,"internally_reserved":0,"allocated":2416640,"data_stored":1388906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575057,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":73216,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298624,"statfs":{"total":96636764160,"available":96561790976,"internally_reserved":0,"allocated":1822720,"data_stored":798538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575059,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59332,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312508,"statfs":{"total":96636764160,"available":96576008192,"internally_reserved":0,"allocated":1822720,"data_stored":798538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575058,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":73216,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298624,"statfs":{"total":96636764160,"available":96561790976,"internally_reserved":0,"allocated":1822720,"data_stored":798538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575060,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":73216,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298624,"statfs":{"total":96636764160,"available":96561790976,"internally_reserved":0,"allocated":1822720,"data_stored":798538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575058,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46848,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324992,"statfs":{"total":96636764160,"available":96588791808,"internally_reserved":0,"allocated":1822720,"data_stored":798538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575058,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":82948,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288892,"statfs":{"total":96636764160,"available":96551825408,"internally_reserved":0,"allocated":1822720,"data_stored":798538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575058,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51584,"kb_used_data":2360,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320256,"statfs":{"total":96636764160,"available":96583942144,"internally_reserved":0,"allocated":2416640,"data_stored":1388906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:55:16.212 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-17T10:55:16.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.330+0000 7f9dfd07b640 1 Processor -- start 2024-04-17T10:55:16.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.330+0000 7f9dfd07b640 1 -- start start 2024-04-17T10:55:16.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.334+0000 7f9dfd07b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9df8154de0 0x7f9df806ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:55:16.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.334+0000 7f9dfd07b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9df80b4210 0x7f9df806cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:55:16.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.334+0000 7f9dfd07b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df815bd70 con 0x7f9df806b350 2024-04-17T10:55:16.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.334+0000 7f9dfd07b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9df80b4870 con 0x7f9df80b4210 2024-04-17T10:55:16.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.334+0000 7f9dfd07b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df815bf60 con 0x7f9df8154de0 2024-04-17T10:55:16.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.334+0000 7f9df759e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9df80b4210 0x7f9df806cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:55:16.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.334+0000 7f9deffff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9df806b350 0x7f9df80b3b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34762/0 (socket says 172.21.15.83:34762) 2024-04-17T10:55:16.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.334+0000 7f9deffff640 1 -- 172.21.15.83:0/2240753472 learned_addr learned my addr 172.21.15.83:0/2240753472 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:55:16.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.334+0000 7f9df6d9d640 1 --2- 172.21.15.83:0/2240753472 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9df8154de0 0x7f9df806ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:55:16.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.334+0000 7f9df759e640 1 -- 172.21.15.83:0/2240753472 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9df8154de0 msgr2=0x7f9df806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:55:16.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.334+0000 7f9df759e640 1 --2- 172.21.15.83:0/2240753472 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9df8154de0 0x7f9df806ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:55:16.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.334+0000 7f9df759e640 1 -- 172.21.15.83:0/2240753472 >> v1:172.21.15.83:6789/0 conn(0x7f9df806b350 legacy=0x7f9df80b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:55:16.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.334+0000 7f9df759e640 1 -- 172.21.15.83:0/2240753472 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df815afb0 con 0x7f9df80b4210 2024-04-17T10:55:16.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.334+0000 7f9df759e640 1 --2- 172.21.15.83:0/2240753472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9df80b4210 0x7f9df806cbe0 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7f9df0009d90 tx=0x7f9df001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102dad0f8cc3473a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:55:16.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.334+0000 7f9df659c640 1 -- 172.21.15.83:0/2240753472 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df00b2070 con 0x7f9df80b4210 2024-04-17T10:55:16.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.334+0000 7f9df659c640 1 -- 172.21.15.83:0/2240753472 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df00a5830 con 0x7f9df80b4210 2024-04-17T10:55:16.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.334+0000 7f9df659c640 1 -- 172.21.15.83:0/2240753472 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df00a5c00 con 0x7f9df80b4210 2024-04-17T10:55:16.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.338+0000 7f9dfd07b640 1 -- 172.21.15.83:0/2240753472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9df80b4210 msgr2=0x7f9df806cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:55:16.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.338+0000 7f9dfd07b640 1 --2- 172.21.15.83:0/2240753472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9df80b4210 0x7f9df806cbe0 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7f9df0009d90 tx=0x7f9df001c970 comp rx=0 tx=0).stop 2024-04-17T10:55:16.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.338+0000 7f9dfd07b640 1 -- 172.21.15.83:0/2240753472 shutdown_connections 2024-04-17T10:55:16.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.338+0000 7f9dfd07b640 1 --2- 172.21.15.83:0/2240753472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9df80b4210 0x7f9df806cbe0 unknown :-1 s=CLOSED pgs=2521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:55:16.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.338+0000 7f9dfd07b640 1 --2- 172.21.15.83:0/2240753472 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9df8154de0 0x7f9df806ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:55:16.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.338+0000 7f9dfd07b640 1 -- 172.21.15.83:0/2240753472 >> 172.21.15.83:0/2240753472 conn(0x7f9df80b15b0 msgr2=0x7f9df80b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:55:16.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.338+0000 7f9dfd07b640 1 -- 172.21.15.83:0/2240753472 shutdown_connections 2024-04-17T10:55:16.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.338+0000 7f9dfd07b640 1 -- 172.21.15.83:0/2240753472 wait complete. 2024-04-17T10:55:16.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.338+0000 7f9dfd07b640 1 Processor -- start 2024-04-17T10:55:16.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.338+0000 7f9dfd07b640 1 -- start start 2024-04-17T10:55:16.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.338+0000 7f9dfd07b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9df806b350 0x7f9df8151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:55:16.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.338+0000 7f9dfd07b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9df80b4210 0x7f9df80ad990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:55:16.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.338+0000 7f9dfd07b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9df8154de0 0x7f9df81530e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:55:16.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.338+0000 7f9dfd07b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df81551c0 con 0x7f9df806b350 2024-04-17T10:55:16.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.338+0000 7f9dfd07b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df806b750 con 0x7f9df8154de0 2024-04-17T10:55:16.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.338+0000 7f9dfd07b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df815e390 con 0x7f9df80b4210 2024-04-17T10:55:16.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.338+0000 7f9df6d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9df806b350 0x7f9df8151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:55:16.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.338+0000 7f9df6d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9df806b350 0x7f9df8151c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51448/0 (socket says 172.21.15.83:51448) 2024-04-17T10:55:16.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.338+0000 7f9df6d9d640 1 -- 172.21.15.83:0/1843702380 learned_addr learned my addr 172.21.15.83:0/1843702380 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:55:16.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.338+0000 7f9df6d9d640 1 -- 172.21.15.83:0/1843702380 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9df80b4210 msgr2=0x7f9df80ad990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:55:16.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.338+0000 7f9df6d9d640 1 --2- 172.21.15.83:0/1843702380 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9df80b4210 0x7f9df80ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:55:16.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.338+0000 7f9df6d9d640 1 -- 172.21.15.83:0/1843702380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9df8154de0 msgr2=0x7f9df81530e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:55:16.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.338+0000 7f9df6d9d640 1 --2- 172.21.15.83:0/1843702380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9df8154de0 0x7f9df81530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:55:16.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.338+0000 7f9df6d9d640 1 -- 172.21.15.83:0/1843702380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df80ae0e0 con 0x7f9df806b350 2024-04-17T10:55:16.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.338+0000 7f9df6d9d640 1 --2- 172.21.15.83:0/1843702380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9df806b350 0x7f9df8151c90 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 crypto rx=0x7f9de800d5f0 tx=0x7f9de800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:55:16.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.338+0000 7f9def7fe640 1 -- 172.21.15.83:0/1843702380 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de80125f0 con 0x7f9df806b350 2024-04-17T10:55:16.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.338+0000 7f9def7fe640 1 -- 172.21.15.83:0/1843702380 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de8019070 con 0x7f9df806b350 2024-04-17T10:55:16.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.338+0000 7f9dfd07b640 1 -- 172.21.15.83:0/1843702380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df80ae3c0 con 0x7f9df806b350 2024-04-17T10:55:16.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.338+0000 7f9def7fe640 1 -- 172.21.15.83:0/1843702380 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de80129b0 con 0x7f9df806b350 2024-04-17T10:55:16.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.338+0000 7f9dfd07b640 1 -- 172.21.15.83:0/1843702380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df80a9960 con 0x7f9df806b350 2024-04-17T10:55:16.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.342+0000 7f9dfd07b640 1 -- 172.21.15.83:0/1843702380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df8074f90 con 0x7f9df806b350 2024-04-17T10:55:16.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.342+0000 7f9def7fe640 1 -- 172.21.15.83:0/1843702380 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9de8012b50 con 0x7f9df806b350 2024-04-17T10:55:16.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.342+0000 7f9def7fe640 1 --2- 172.21.15.83:0/1843702380 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9dd8066f60 0x7f9dd8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:55:16.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.342+0000 7f9deffff640 1 --2- 172.21.15.83:0/1843702380 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9dd8066f60 0x7f9dd8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:55:16.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.342+0000 7f9def7fe640 1 -- 172.21.15.83:0/1843702380 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9de8085180 con 0x7f9df806b350 2024-04-17T10:55:16.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.342+0000 7f9deffff640 1 --2- 172.21.15.83:0/1843702380 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9dd8066f60 0x7f9dd8069420 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f9de0000f70 tx=0x7f9de0007a90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:55:16.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.346+0000 7f9def7fe640 1 -- 172.21.15.83:0/1843702380 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9de8056490 con 0x7f9df806b350 2024-04-17T10:55:16.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.494+0000 7f9dfd07b640 1 -- 172.21.15.83:0/1843702380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f9df80b10c0 con 0x7f9df806b350 2024-04-17T10:55:16.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.498+0000 7f9def7fe640 1 -- 172.21.15.83:0/1843702380 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v28)=0 v28) v1 ==== 123+0+0 (secure 0 0 0) 0x7f9de8055d90 con 0x7f9df806b350 2024-04-17T10:55:16.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.502+0000 7f9def7fe640 1 -- 172.21.15.83:0/1843702380 <== mon.0 v2:172.21.15.83:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f9de8059ee0 con 0x7f9df806b350 2024-04-17T10:55:16.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.502+0000 7f9dfd07b640 1 -- 172.21.15.83:0/1843702380 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9dd8066f60 msgr2=0x7f9dd8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:55:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.502+0000 7f9dfd07b640 1 --2- 172.21.15.83:0/1843702380 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9dd8066f60 0x7f9dd8069420 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f9de0000f70 tx=0x7f9de0007a90 comp rx=0 tx=0).stop 2024-04-17T10:55:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.502+0000 7f9dfd07b640 1 -- 172.21.15.83:0/1843702380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9df806b350 msgr2=0x7f9df8151c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:55:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.502+0000 7f9dfd07b640 1 --2- 172.21.15.83:0/1843702380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9df806b350 0x7f9df8151c90 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 crypto rx=0x7f9de800d5f0 tx=0x7f9de800dab0 comp rx=0 tx=0).stop 2024-04-17T10:55:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.502+0000 7f9dfd07b640 1 -- 172.21.15.83:0/1843702380 shutdown_connections 2024-04-17T10:55:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.502+0000 7f9dfd07b640 1 --2- 172.21.15.83:0/1843702380 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7f9dd8066f60 0x7f9dd8069420 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:55:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.502+0000 7f9dfd07b640 1 --2- 172.21.15.83:0/1843702380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9df8154de0 0x7f9df81530e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:55:16.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.502+0000 7f9dfd07b640 1 --2- 172.21.15.83:0/1843702380 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9df80b4210 0x7f9df80ad990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:55:16.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.502+0000 7f9dfd07b640 1 --2- 172.21.15.83:0/1843702380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9df806b350 0x7f9df8151c90 unknown :-1 s=CLOSED pgs=2522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:55:16.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.502+0000 7f9dfd07b640 1 -- 172.21.15.83:0/1843702380 >> 172.21.15.83:0/1843702380 conn(0x7f9df80b15b0 msgr2=0x7f9df80ce650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:55:16.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.502+0000 7f9dfd07b640 1 -- 172.21.15.83:0/1843702380 shutdown_connections 2024-04-17T10:55:16.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.502+0000 7f9dfd07b640 1 -- 172.21.15.83:0/1843702380 wait complete. 2024-04-17T10:55:16.514 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-17T10:55:16.514 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:55:16.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.630+0000 7fb5a5d5d640 1 Processor -- start 2024-04-17T10:55:16.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.630+0000 7fb5a5d5d640 1 -- start start 2024-04-17T10:55:16.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.630+0000 7fb5a5d5d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb5a015db50 0x7fb5a0070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:55:16.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.630+0000 7fb5a5d5d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5a00aedd0 0x7fb5a006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:55:16.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.630+0000 7fb5a5d5d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5a00709b0 con 0x7fb5a006dae0 2024-04-17T10:55:16.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.630+0000 7fb5a5d5d640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5a015e440 con 0x7fb5a015db50 2024-04-17T10:55:16.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.630+0000 7fb5a5d5d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a015e720 con 0x7fb5a00aedd0 2024-04-17T10:55:16.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.630+0000 7fb59ffff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb5a006dae0 0x7fb5a00cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34768/0 (socket says 172.21.15.83:34768) 2024-04-17T10:55:16.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.630+0000 7fb59ffff640 1 -- 172.21.15.83:0/2907530966 learned_addr learned my addr 172.21.15.83:0/2907530966 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-17T10:55:16.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb59e7fc640 1 -- 172.21.15.83:0/2907530966 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1543742268 0 0) 0x7fb5980014a0 con 0x7fb5a006dae0 2024-04-17T10:55:16.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb59e7fc640 1 -- 172.21.15.83:0/2907530966 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb588005740 con 0x7fb5a006dae0 2024-04-17T10:55:16.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb59f7fe640 1 --2- 172.21.15.83:0/2907530966 >> v2:172.21.15.167:3300/0 conn(0x7fb5a015db50 0x7fb5a0070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:55:16.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb59effd640 1 --2- 172.21.15.83:0/2907530966 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5a00aedd0 0x7fb5a006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:55:16.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb59e7fc640 1 -- 172.21.15.83:0/2907530966 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 682998066 0 0) 0x7fb588005740 con 0x7fb5a006dae0 2024-04-17T10:55:16.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb59e7fc640 1 -- 172.21.15.83:0/2907530966 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5a00aedd0 msgr2=0x7fb5a006d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:55:16.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb59e7fc640 1 --2- 172.21.15.83:0/2907530966 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5a00aedd0 0x7fb5a006d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:55:16.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb59e7fc640 1 -- 172.21.15.83:0/2907530966 >> v2:172.21.15.167:3300/0 conn(0x7fb5a015db50 msgr2=0x7fb5a0070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:55:16.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb59e7fc640 1 --2- 172.21.15.83:0/2907530966 >> v2:172.21.15.167:3300/0 conn(0x7fb5a015db50 0x7fb5a0070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:55:16.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb59e7fc640 1 -- 172.21.15.83:0/2907530966 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a015ea00 con 0x7fb5a006dae0 2024-04-17T10:55:16.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb59e7fc640 1 -- 172.21.15.83:0/2907530966 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2430120986 0 0) 0x7fb598001bd0 con 0x7fb5a006dae0 2024-04-17T10:55:16.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb59e7fc640 1 -- 172.21.15.83:0/2907530966 >> v1:172.21.15.83:6789/0 conn(0x7fb5a006dae0 legacy=0x7fb5a00cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:55:16.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb59e7fc640 1 --2- 172.21.15.83:0/2907530966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb588005bd0 0x7fb588005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:55:16.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb59e7fc640 1 --2- 172.21.15.83:0/2907530966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb588006a20 0x7fb588006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:55:16.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb59e7fc640 1 --2- 172.21.15.83:0/2907530966 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5a00aedd0 0x7fb5880073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:55:16.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb59e7fc640 1 -- 172.21.15.83:0/2907530966 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a015e440 con 0x7fb588006a20 2024-04-17T10:55:16.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb59e7fc640 1 -- 172.21.15.83:0/2907530966 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a015e720 con 0x7fb588005bd0 2024-04-17T10:55:16.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb59e7fc640 1 -- 172.21.15.83:0/2907530966 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb58800dce0 con 0x7fb5a00aedd0 2024-04-17T10:55:16.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb59effd640 1 --2- 172.21.15.83:0/2907530966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb588006a20 0x7fb588006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:55:16.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb59f7fe640 1 --2- 172.21.15.83:0/2907530966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb588005bd0 0x7fb588005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:55:16.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb59effd640 1 -- 172.21.15.83:0/2907530966 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5a00aedd0 msgr2=0x7fb5880073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:55:16.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb59effd640 1 --2- 172.21.15.83:0/2907530966 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5a00aedd0 0x7fb5880073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:55:16.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb59effd640 1 -- 172.21.15.83:0/2907530966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb588005bd0 msgr2=0x7fb588005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:55:16.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb59effd640 1 --2- 172.21.15.83:0/2907530966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb588005bd0 0x7fb588005ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:55:16.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb59effd640 1 -- 172.21.15.83:0/2907530966 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb588005740 con 0x7fb588006a20 2024-04-17T10:55:16.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb59effd640 1 --2- 172.21.15.83:0/2907530966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb588006a20 0x7fb588006ea0 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7fb59400ba00 tx=0x7fb59401c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4351cda351e125c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:55:16.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb59e7fc640 1 -- 172.21.15.83:0/2907530966 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5940ae070 con 0x7fb588006a20 2024-04-17T10:55:16.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb59e7fc640 1 -- 172.21.15.83:0/2907530966 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fb5940a9660 con 0x7fb588006a20 2024-04-17T10:55:16.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb59e7fc640 1 -- 172.21.15.83:0/2907530966 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59400b600 con 0x7fb588006a20 2024-04-17T10:55:16.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb5a5d5d640 1 -- 172.21.15.83:0/2907530966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb588006a20 msgr2=0x7fb588006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:55:16.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb5a5d5d640 1 --2- 172.21.15.83:0/2907530966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb588006a20 0x7fb588006ea0 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7fb59400ba00 tx=0x7fb59401c680 comp rx=0 tx=0).stop 2024-04-17T10:55:16.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb59ffff640 1 -- 172.21.15.83:0/2907530966 reap_dead start 2024-04-17T10:55:16.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb5a5d5d640 1 -- 172.21.15.83:0/2907530966 shutdown_connections 2024-04-17T10:55:16.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.634+0000 7fb5a5d5d640 1 -- 172.21.15.83:0/2907530966 >> 172.21.15.83:0/2907530966 conn(0x7fb5a006abe0 msgr2=0x7fb5a006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:55:16.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.638+0000 7fb5a5d5d640 1 -- 172.21.15.83:0/2907530966 shutdown_connections 2024-04-17T10:55:16.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.638+0000 7fb5a5d5d640 1 -- 172.21.15.83:0/2907530966 wait complete. 2024-04-17T10:55:16.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.638+0000 7fb5a5d5d640 1 Processor -- start 2024-04-17T10:55:16.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.638+0000 7fb5a5d5d640 1 -- start start 2024-04-17T10:55:16.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.638+0000 7fb5a5d5d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb588006a20 0x7fb5a0141bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:55:16.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.638+0000 7fb5a5d5d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5a0144350 0x7fb5a01447d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:55:16.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.638+0000 7fb5a5d5d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5a00ad840 0x7fb5a00adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:55:16.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.638+0000 7fb5a5d5d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a015df30 con 0x7fb5a0144350 2024-04-17T10:55:16.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.638+0000 7fb5a5d5d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a015efc0 con 0x7fb588006a20 2024-04-17T10:55:16.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.638+0000 7fb5a5d5d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a015e0b0 con 0x7fb5a00ad840 2024-04-17T10:55:16.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.638+0000 7fb59effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5a0144350 0x7fb5a01447d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:55:16.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.638+0000 7fb59effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5a0144350 0x7fb5a01447d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51454/0 (socket says 172.21.15.83:51454) 2024-04-17T10:55:16.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.638+0000 7fb59effd640 1 -- 172.21.15.83:0/2816690289 learned_addr learned my addr 172.21.15.83:0/2816690289 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-17T10:55:16.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.638+0000 7fb59f7fe640 1 --2- 172.21.15.83:0/2816690289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb588006a20 0x7fb5a0141bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:55:16.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.638+0000 7fb59ffff640 1 --2- 172.21.15.83:0/2816690289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5a00ad840 0x7fb5a00adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:55:16.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.638+0000 7fb59effd640 1 -- 172.21.15.83:0/2816690289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5a00ad840 msgr2=0x7fb5a00adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:55:16.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.638+0000 7fb59effd640 1 --2- 172.21.15.83:0/2816690289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5a00ad840 0x7fb5a00adcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:55:16.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.638+0000 7fb59effd640 1 -- 172.21.15.83:0/2816690289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb588006a20 msgr2=0x7fb5a0141bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:55:16.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.638+0000 7fb59effd640 1 --2- 172.21.15.83:0/2816690289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb588006a20 0x7fb5a0141bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:55:16.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.638+0000 7fb59effd640 1 -- 172.21.15.83:0/2816690289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a0144f10 con 0x7fb5a0144350 2024-04-17T10:55:16.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.638+0000 7fb59f7fe640 1 --2- 172.21.15.83:0/2816690289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb588006a20 0x7fb5a0141bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:55:16.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.638+0000 7fb59effd640 1 --2- 172.21.15.83:0/2816690289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5a0144350 0x7fb5a01447d0 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7fb594098090 tx=0x7fb588007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:55:16.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.638+0000 7fb587fff640 1 -- 172.21.15.83:0/2816690289 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5940ae040 con 0x7fb5a0144350 2024-04-17T10:55:16.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.638+0000 7fb587fff640 1 -- 172.21.15.83:0/2816690289 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fb5940b7030 con 0x7fb5a0144350 2024-04-17T10:55:16.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.638+0000 7fb5a5d5d640 1 -- 172.21.15.83:0/2816690289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5a01451f0 con 0x7fb5a0144350 2024-04-17T10:55:16.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.638+0000 7fb587fff640 1 -- 172.21.15.83:0/2816690289 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5940b3660 con 0x7fb5a0144350 2024-04-17T10:55:16.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.638+0000 7fb5a5d5d640 1 -- 172.21.15.83:0/2816690289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a01425f0 con 0x7fb5a0144350 2024-04-17T10:55:16.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.638+0000 7fb5a5d5d640 1 -- 172.21.15.83:0/2816690289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5a0074f90 con 0x7fb5a0144350 2024-04-17T10:55:16.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.642+0000 7fb587fff640 1 -- 172.21.15.83:0/2816690289 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb5940b1070 con 0x7fb5a0144350 2024-04-17T10:55:16.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.642+0000 7fb587fff640 1 --2- 172.21.15.83:0/2816690289 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb58c066f60 0x7fb58c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:55:16.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.642+0000 7fb587fff640 1 -- 172.21.15.83:0/2816690289 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb59411f3f0 con 0x7fb5a0144350 2024-04-17T10:55:16.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.642+0000 7fb59f7fe640 1 --2- 172.21.15.83:0/2816690289 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb58c066f60 0x7fb58c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:55:16.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.642+0000 7fb59f7fe640 1 --2- 172.21.15.83:0/2816690289 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb58c066f60 0x7fb58c069420 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fb590009940 tx=0x7fb590008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:55:16.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.646+0000 7fb587fff640 1 -- 172.21.15.83:0/2816690289 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb594016390 con 0x7fb5a0144350 2024-04-17T10:55:16.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.834+0000 7fb5a5d5d640 1 -- 172.21.15.83:0/2816690289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5a00cc060 con 0x7fb5a0144350 2024-04-17T10:55:16.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.834+0000 7fb587fff640 1 -- 172.21.15.83:0/2816690289 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+3705 (secure 0 0 0) 0x7fb5940f0700 con 0x7fb5a0144350 2024-04-17T10:55:16.831 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-17T10:55:16.831 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":92,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/4064366741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4064366741},{"type":"v1","addr":"172.21.15.167:6835","nonce":4064366741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":7054,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2584390968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2584390968},{"type":"v1","addr":"172.21.15.167:6833","nonce":2584390968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/1853711871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1853711871},{"type":"v1","addr":"172.21.15.83:6835","nonce":1853711871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3362045924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3362045924},{"type":"v1","addr":"172.21.15.83:6837","nonce":3362045924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[]} 2024-04-17T10:55:16.831 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 92 2024-04-17T10:55:16.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.838+0000 7fb5a5d5d640 1 -- 172.21.15.83:0/2816690289 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb58c066f60 msgr2=0x7fb58c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:55:16.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.838+0000 7fb5a5d5d640 1 --2- 172.21.15.83:0/2816690289 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb58c066f60 0x7fb58c069420 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fb590009940 tx=0x7fb590008040 comp rx=0 tx=0).stop 2024-04-17T10:55:16.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.838+0000 7fb5a5d5d640 1 -- 172.21.15.83:0/2816690289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5a0144350 msgr2=0x7fb5a01447d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:55:16.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.838+0000 7fb5a5d5d640 1 --2- 172.21.15.83:0/2816690289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5a0144350 0x7fb5a01447d0 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7fb594098090 tx=0x7fb588007a50 comp rx=0 tx=0).stop 2024-04-17T10:55:16.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.838+0000 7fb5a5d5d640 1 -- 172.21.15.83:0/2816690289 shutdown_connections 2024-04-17T10:55:16.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.838+0000 7fb5a5d5d640 1 --2- 172.21.15.83:0/2816690289 >> [v2:172.21.15.83:6832/2577814962,v1:172.21.15.83:6833/2577814962] conn(0x7fb58c066f60 0x7fb58c069420 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:55:16.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.838+0000 7fb5a5d5d640 1 --2- 172.21.15.83:0/2816690289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5a00ad840 0x7fb5a00adcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:55:16.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.838+0000 7fb5a5d5d640 1 --2- 172.21.15.83:0/2816690289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5a0144350 0x7fb5a01447d0 unknown :-1 s=CLOSED pgs=2525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:55:16.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.838+0000 7fb5a5d5d640 1 --2- 172.21.15.83:0/2816690289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb588006a20 0x7fb5a0141bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:55:16.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.838+0000 7fb5a5d5d640 1 -- 172.21.15.83:0/2816690289 >> 172.21.15.83:0/2816690289 conn(0x7fb5a006abe0 msgr2=0x7fb5a00cf420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:55:16.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.838+0000 7fb5a5d5d640 1 -- 172.21.15.83:0/2816690289 shutdown_connections 2024-04-17T10:55:16.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-17T10:55:16.838+0000 7fb5a5d5d640 1 -- 172.21.15.83:0/2816690289 wait complete. 2024-04-17T10:55:16.850 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-17T10:55:16.851 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-17T10:55:16.851 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-17T10:55:16.851 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:55:16.862 INFO:tasks.ceph.mds.a:Stopped 2024-04-17T10:55:16.862 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-17T10:55:16.862 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:55:16.914 INFO:tasks.ceph.mds.c:Stopped 2024-04-17T10:55:16.914 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-17T10:55:16.914 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:55:16.933 INFO:tasks.ceph.mds.b:Stopped 2024-04-17T10:55:16.933 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-17T10:55:16.934 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:55:16.997 INFO:tasks.ceph.mds.d:Stopped 2024-04-17T10:55:16.998 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-17T10:55:16.998 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-17T10:55:16.998 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:55:17.078 INFO:tasks.ceph.osd.0:Stopped 2024-04-17T10:55:17.079 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-17T10:55:17.079 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:55:17.154 INFO:tasks.ceph.osd.1:Stopped 2024-04-17T10:55:17.154 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-17T10:55:17.154 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:55:21.402 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-04-17T10:55:23.256 INFO:tasks.ceph.osd.2:Stopped 2024-04-17T10:55:23.256 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-17T10:55:23.256 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:55:23.331 INFO:tasks.ceph.osd.3:Stopped 2024-04-17T10:55:23.332 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-17T10:55:23.332 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:55:23.417 INFO:tasks.ceph.osd.4:Stopped 2024-04-17T10:55:23.417 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-17T10:55:23.417 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:55:23.510 INFO:tasks.ceph.osd.5:Stopped 2024-04-17T10:55:23.511 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-17T10:55:23.511 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:55:23.610 INFO:tasks.ceph.osd.6:Stopped 2024-04-17T10:55:23.610 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-17T10:55:23.611 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:55:23.696 INFO:tasks.ceph.osd.7:Stopped 2024-04-17T10:55:23.696 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-17T10:55:23.697 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-17T10:55:23.697 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:55:23.726 INFO:tasks.ceph.mgr.y:Stopped 2024-04-17T10:55:23.726 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-17T10:55:23.726 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:55:23.776 INFO:tasks.ceph.mgr.x:Stopped 2024-04-17T10:55:23.776 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-17T10:55:23.776 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-17T10:55:23.776 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:55:23.796 INFO:tasks.ceph.mon.a:Stopped 2024-04-17T10:55:23.796 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-17T10:55:23.796 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:55:23.831 INFO:tasks.ceph.mon.b:Stopped 2024-04-17T10:55:23.832 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-17T10:55:23.832 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:55:23.894 INFO:tasks.ceph.mon.c:Stopped 2024-04-17T10:55:23.895 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-17T10:55:23.895 DEBUG:teuthology.orchestra.run.smithi083:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'Replacing daemon mds' | egrep -v 'Scrub error on inode' | egrep -v 'Behind on trimming' | egrep -v 'Metadata damage detected' | egrep -v 'bad backtrace on inode' | egrep -v 'overall HEALTH_' | egrep -v '\(MDS_TRIM\)' | head -n 1 2024-04-17T10:55:23.913 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi083.front.sepia.ceph.com 2024-04-17T10:55:23.913 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-17T10:55:24.329 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi083.front.sepia.ceph.com 2024-04-17T10:55:24.330 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-17T10:55:24.585 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi083.front.sepia.ceph.com 2024-04-17T10:55:24.585 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-17T10:55:24.791 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi083.front.sepia.ceph.com 2024-04-17T10:55:24.791 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-17T10:55:24.975 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi167.front.sepia.ceph.com 2024-04-17T10:55:24.975 DEBUG:teuthology.orchestra.run.smithi167:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-17T10:55:25.267 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi167.front.sepia.ceph.com 2024-04-17T10:55:25.397 DEBUG:teuthology.orchestra.run.smithi167:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-17T10:55:25.506 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi167.front.sepia.ceph.com 2024-04-17T10:55:25.506 DEBUG:teuthology.orchestra.run.smithi167:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-17T10:55:25.626 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi167.front.sepia.ceph.com 2024-04-17T10:55:25.626 DEBUG:teuthology.orchestra.run.smithi167:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-17T10:55:25.751 INFO:tasks.ceph:Archiving mon data... 2024-04-17T10:55:25.752 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2024-04-17_09:52:10-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7660174/data/mon.a.tgz 2024-04-17T10:55:25.753 DEBUG:teuthology.orchestra.run.smithi083:> mktemp 2024-04-17T10:55:25.759 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/tmp.gWX8nUkTLX 2024-04-17T10:55:25.759 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.gWX8nUkTLX 2024-04-17T10:55:25.902 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0666 /tmp/tmp.gWX8nUkTLX 2024-04-17T10:55:25.966 DEBUG:teuthology.orchestra.remote:smithi083:/tmp/tmp.gWX8nUkTLX is 424KB 2024-04-17T10:55:26.029 DEBUG:teuthology.orchestra.run.smithi083:> rm -fr /tmp/tmp.gWX8nUkTLX 2024-04-17T10:55:26.035 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2024-04-17_09:52:10-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7660174/data/mon.b.tgz 2024-04-17T10:55:26.035 DEBUG:teuthology.orchestra.run.smithi167:> mktemp 2024-04-17T10:55:26.041 INFO:teuthology.orchestra.run.smithi167.stdout:/tmp/tmp.mNAxL8jFPt 2024-04-17T10:55:26.041 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.mNAxL8jFPt 2024-04-17T10:55:26.180 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0666 /tmp/tmp.mNAxL8jFPt 2024-04-17T10:55:26.247 DEBUG:teuthology.orchestra.remote:smithi167:/tmp/tmp.mNAxL8jFPt is 426KB 2024-04-17T10:55:26.310 DEBUG:teuthology.orchestra.run.smithi167:> rm -fr /tmp/tmp.mNAxL8jFPt 2024-04-17T10:55:26.315 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2024-04-17_09:52:10-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7660174/data/mon.c.tgz 2024-04-17T10:55:26.315 DEBUG:teuthology.orchestra.run.smithi167:> mktemp 2024-04-17T10:55:26.363 INFO:teuthology.orchestra.run.smithi167.stdout:/tmp/tmp.oB90y172V5 2024-04-17T10:55:26.364 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.oB90y172V5 2024-04-17T10:55:26.402 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-04-17T10:55:26.507 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0666 /tmp/tmp.oB90y172V5 2024-04-17T10:55:26.572 DEBUG:teuthology.orchestra.remote:smithi167:/tmp/tmp.oB90y172V5 is 426KB 2024-04-17T10:55:26.589 DEBUG:teuthology.orchestra.run.smithi167:> rm -fr /tmp/tmp.oB90y172V5 2024-04-17T10:55:26.595 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-17T10:55:26.595 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-17T10:55:26.597 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-17T10:55:26.655 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-17T10:55:26.695 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-17T10:55:26.696 INFO:tasks.ceph:Archiving crash dumps... 2024-04-17T10:55:26.697 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-17_09:52:10-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7660174/remote/smithi083/crash 2024-04-17T10:55:26.698 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-17T10:55:26.712 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-17_09:52:10-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7660174/remote/smithi167/crash 2024-04-17T10:55:26.713 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-17T10:55:26.749 INFO:tasks.ceph:Shutting down logrotate 2024-04-17T10:55:26.749 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:55:26.756 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:55:26.827 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-17T10:55:26.840 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-17T10:55:26.881 INFO:tasks.ceph:Compressing logs... 2024-04-17T10:55:26.881 DEBUG:teuthology.orchestra.run.smithi083:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-17T10:55:26.885 DEBUG:teuthology.orchestra.run.smithi167:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-17T10:55:26.904 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54249.log 2024-04-17T10:55:26.905 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54946.log 2024-04-17T10:55:26.905 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56684.log 2024-04-17T10:55:26.905 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54249.log: /var/log/ceph/ceph-client.admin.54946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86433.log 2024-04-17T10:55:26.913 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56684.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54946.log.gz 2024-04-17T10:55:26.914 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54249.log.gz 2024-04-17T10:55:26.914 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69970.log 2024-04-17T10:55:26.914 INFO:teuthology.orchestra.run.smithi083.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56684.log.gz 2024-04-17T10:55:26.914 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76386.log 2024-04-17T10:55:26.914 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86433.log: /var/log/ceph/ceph-client.admin.69970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76162.log 2024-04-17T10:55:26.915 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69970.log.gz 2024-04-17T10:55:26.915 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34090.log 2024-04-17T10:55:26.915 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.76386.log: /var/log/ceph/ceph-client.admin.76162.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.86433.log.gz 2024-04-17T10:55:26.915 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86719.log 2024-04-17T10:55:26.915 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.76386.log.gz 2024-04-17T10:55:26.915 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34090.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76162.log.gz 2024-04-17T10:55:26.915 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55643.log 2024-04-17T10:55:26.916 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34090.log.gz 2024-04-17T10:55:26.916 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78735.log 2024-04-17T10:55:26.916 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73186.log 2024-04-17T10:55:26.916 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.86719.log.gz 2024-04-17T10:55:26.916 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.78735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87797.log 2024-04-17T10:55:26.916 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55643.log.gz 2024-04-17T10:55:26.916 INFO:teuthology.orchestra.run.smithi083.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.78735.log.gz 2024-04-17T10:55:26.916 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.73186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85463.log 2024-04-17T10:55:26.917 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73186.log.gz 2024-04-17T10:55:26.917 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86327.log 2024-04-17T10:55:26.917 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.87797.log.gz 2024-04-17T10:55:26.917 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36576.log 2024-04-17T10:55:26.917 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87911.log 2024-04-17T10:55:26.917 INFO:teuthology.orchestra.run.smithi083.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.85463.log.gz 2024-04-17T10:55:26.917 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.86327.log.gz 2024-04-17T10:55:26.918 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.36576.log: /var/log/ceph/ceph-client.admin.35202.log 2024-04-17T10:55:26.918 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87911.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36576.log.gz 2024-04-17T10:55:26.918 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87608.log 2024-04-17T10:55:26.918 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.87911.log.gz 2024-04-17T10:55:26.918 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71763.log 2024-04-17T10:55:26.918 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56528.log 2024-04-17T10:55:26.918 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35202.log.gz 2024-04-17T10:55:26.919 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.71763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53037.log 2024-04-17T10:55:26.919 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87608.log.gz 2024-04-17T10:55:26.919 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84886.log 2024-04-17T10:55:26.919 INFO:teuthology.orchestra.run.smithi083.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.71763.log.gz 2024-04-17T10:55:26.919 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56528.log.gz 2024-04-17T10:55:26.919 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75926.log 2024-04-17T10:55:26.919 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.84886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81667.log 2024-04-17T10:55:26.972 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27907.log 2024-04-17T10:55:26.973 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-17T10:55:26.973 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.27907.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-17T10:55:26.973 INFO:teuthology.orchestra.run.smithi167.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27907.log.gz 2024-04-17T10:55:26.973 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27739.log 2024-04-17T10:55:26.973 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27835.log 2024-04-17T10:55:26.973 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.27739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27859.log 2024-04-17T10:55:26.974 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.27835.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.27739.log.gz 2024-04-17T10:55:26.974 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-17T10:55:26.974 INFO:teuthology.orchestra.run.smithi167.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.27835.log.gz 2024-04-17T10:55:26.974 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.27859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27787.log 2024-04-17T10:55:26.974 INFO:teuthology.orchestra.run.smithi167.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27859.log.gz 2024-04-17T10:55:26.974 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-17T10:55:26.974 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph-client.admin.27787.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-17T10:55:26.975 INFO:teuthology.orchestra.run.smithi167.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.27787.log.gz 2024-04-17T10:55:26.975 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33699.log 2024-04-17T10:55:26.975 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-17T10:55:26.975 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.3.33699.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-17T10:55:26.975 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33582.log 2024-04-17T10:55:26.975 INFO:teuthology.orchestra.run.smithi167.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.3.33699.log.gz 2024-04-17T10:55:26.975 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-17T10:55:26.976 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.1.33582.log: 90.3% -- replaced with /var/log/ceph/ceph-client.1.33582.log.gz 2024-04-17T10:55:26.976 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27811.log 2024-04-17T10:55:26.976 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-mds.d.log: 92.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-17T10:55:26.976 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-17T10:55:26.976 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.27811.log: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-17T10:55:26.977 INFO:teuthology.orchestra.run.smithi167.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.27811.log.gz 2024-04-17T10:55:26.977 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27763.log 2024-04-17T10:55:26.977 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-17T10:55:26.980 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.27763.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27763.log.gz 2024-04-17T10:55:26.980 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-17T10:55:26.981 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-mon.b.log: 94.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-17T10:55:26.981 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27883.log 2024-04-17T10:55:26.996 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.admin.27883.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.27883.log.gz 2024-04-17T10:55:26.996 INFO:teuthology.orchestra.run.smithi167.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-17T10:55:27.016 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75926.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.53037.log.gz 2024-04-17T10:55:27.017 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85961.log 2024-04-17T10:55:27.017 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.84886.log.gz 2024-04-17T10:55:27.017 INFO:teuthology.orchestra.run.smithi083.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.75926.log.gz 2024-04-17T10:55:27.017 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.81667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82765.log 2024-04-17T10:55:27.017 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.81667.log.gz 2024-04-17T10:55:27.017 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70480.log 2024-04-17T10:55:27.018 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.85961.log.gz 2024-04-17T10:55:27.018 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.82765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34099.log 2024-04-17T10:55:27.018 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56262.log 2024-04-17T10:55:27.018 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.82765.log.gz 2024-04-17T10:55:27.018 INFO:teuthology.orchestra.run.smithi083.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.70480.log.gz 2024-04-17T10:55:27.018 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53722.log 2024-04-17T10:55:27.018 INFO:teuthology.orchestra.run.smithi083.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34099.log.gz 2024-04-17T10:55:27.019 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86641.log 2024-04-17T10:55:27.019 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53722.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.56262.log.gz -5 2024-04-17T10:55:27.019 INFO:teuthology.orchestra.run.smithi083.stderr: --verbose -- /var/log/ceph/ceph-client.admin.86745.log 2024-04-17T10:55:27.019 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53722.log.gz 2024-04-17T10:55:27.019 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37195.log 2024-04-17T10:55:27.019 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86745.log: 89.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.86641.log.gz 2024-04-17T10:55:27.020 INFO:teuthology.orchestra.run.smithi083.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.34435.log 2024-04-17T10:55:27.020 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.86745.log.gz 2024-04-17T10:55:27.020 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55799.log 2024-04-17T10:55:27.020 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37195.log.gz 2024-04-17T10:55:27.020 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81901.log 2024-04-17T10:55:27.020 INFO:teuthology.orchestra.run.smithi083.stderr: 89.2%/var/log/ceph/ceph-client.admin.55799.log: -- replaced with /var/log/ceph/ceph-client.admin.34435.log.gz 2024-04-17T10:55:27.020 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86511.log 2024-04-17T10:55:27.021 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.81901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37705.log 2024-04-17T10:55:27.021 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55799.log.gz 2024-04-17T10:55:27.021 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.81901.log.gz 2024-04-17T10:55:27.021 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86511.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-17T10:55:27.021 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.86511.log.gz 2024-04-17T10:55:27.021 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79224.log 2024-04-17T10:55:27.021 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73316.log 2024-04-17T10:55:27.022 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-mgr.y.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37705.log.gz 2024-04-17T10:55:27.022 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.79224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50193.log 2024-04-17T10:55:27.022 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.79224.log.gz 2024-04-17T10:55:27.022 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.73316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55974.log 2024-04-17T10:55:27.022 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73316.log.gz 2024-04-17T10:55:27.022 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79460.log 2024-04-17T10:55:27.022 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50193.log: /var/log/ceph/ceph-client.admin.55974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83168.log 2024-04-17T10:55:27.023 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55974.log.gz 2024-04-17T10:55:27.023 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.79460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86589.log 2024-04-17T10:55:27.023 INFO:teuthology.orchestra.run.smithi083.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.50193.log.gz 2024-04-17T10:55:27.023 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.79460.log.gz 2024-04-17T10:55:27.023 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.83168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73160.log 2024-04-17T10:55:27.023 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87112.log 2024-04-17T10:55:27.023 INFO:teuthology.orchestra.run.smithi083.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.83168.log.gz 2024-04-17T10:55:27.023 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.73160.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.86589.log.gz 2024-04-17T10:55:27.024 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55125.log 2024-04-17T10:55:27.024 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73160.log.gz 2024-04-17T10:55:27.024 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85803.log 2024-04-17T10:55:27.024 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.87112.log.gz 2024-04-17T10:55:27.025 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.55125.log: -- /var/log/ceph/ceph-client.admin.75699.log 2024-04-17T10:55:27.025 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85803.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55125.log.gz 2024-04-17T10:55:27.025 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35943.log 2024-04-17T10:55:27.025 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75699.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.85803.log.gz 2024-04-17T10:55:27.026 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33577.log 2024-04-17T10:55:27.026 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75699.log.gz 2024-04-17T10:55:27.026 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36628.log 2024-04-17T10:55:27.026 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36732.log 2024-04-17T10:55:27.026 INFO:teuthology.orchestra.run.smithi083.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35943.log.gz 2024-04-17T10:55:27.026 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36628.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33577.log.gz 2024-04-17T10:55:27.026 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86485.log 2024-04-17T10:55:27.027 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36628.log.gz 2024-04-17T10:55:27.027 INFO:teuthology.orchestra.run.smithi083.stderr:gzip/var/log/ceph/ceph-client.admin.36732.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.72423.log 2024-04-17T10:55:27.027 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88762.log 88.7% 2024-04-17T10:55:27.027 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36732.log.gz 2024-04-17T10:55:27.027 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72423.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.86485.log.gz 2024-04-17T10:55:27.027 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88710.log 2024-04-17T10:55:27.027 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72423.log.gz 2024-04-17T10:55:27.027 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78649.log 2024-04-17T10:55:27.028 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82143.log 2024-04-17T10:55:27.028 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.88762.log.gz 2024-04-17T10:55:27.028 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.78649.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.88710.log.gz 2024-04-17T10:55:27.028 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72319.log 2024-04-17T10:55:27.028 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78649.log.gz 2024-04-17T10:55:27.028 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.82143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51254.log 2024-04-17T10:55:27.028 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72319.log: gzip -5 --verbose 88.9% -- -- replaced with /var/log/ceph/ceph-client.admin.82143.log.gz /var/log/ceph/ceph-client.admin.37300.log 2024-04-17T10:55:27.028 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-04-17T10:55:27.029 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.72319.log.gz 2024-04-17T10:55:27.029 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53201.log 2024-04-17T10:55:27.029 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51254.log.gz 2024-04-17T10:55:27.029 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.37300.log: /var/log/ceph/ceph-client.admin.53956.log 2024-04-17T10:55:27.029 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55284.log 2024-04-17T10:55:27.029 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37300.log.gz 2024-04-17T10:55:27.029 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53201.log.gz 2024-04-17T10:55:27.029 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34409.log 2024-04-17T10:55:27.030 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53956.log.gz 2024-04-17T10:55:27.030 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.55284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78917.log 2024-04-17T10:55:27.030 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74704.log 2024-04-17T10:55:27.030 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.78917.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34409.log.gz 2024-04-17T10:55:27.030 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54858.log 2024-04-17T10:55:27.030 INFO:teuthology.orchestra.run.smithi083.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.78917.log.gz 2024-04-17T10:55:27.030 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88171.log 2024-04-17T10:55:27.031 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.74704.log: /var/log/ceph/ceph-client.admin.54858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55179.log 2024-04-17T10:55:27.031 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88171.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54858.log.gz 2024-04-17T10:55:27.031 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78002.log 2024-04-17T10:55:27.031 INFO:teuthology.orchestra.run.smithi083.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.88171.log.gz 2024-04-17T10:55:27.031 INFO:teuthology.orchestra.run.smithi083.stderr: 91.0%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.74704.log.gz --verbose 2024-04-17T10:55:27.031 INFO:teuthology.orchestra.run.smithi083.stderr: -- /var/log/ceph/ceph-client.admin.72215.log 2024-04-17T10:55:27.031 INFO:teuthology.orchestra.run.smithi083.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.55284.log.gz 2024-04-17T10:55:27.031 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55179.log: /var/log/ceph/ceph-client.admin.78002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81641.log 2024-04-17T10:55:27.032 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81532.log 2024-04-17T10:55:27.032 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.72215.log.gz 2024-04-17T10:55:27.032 INFO:teuthology.orchestra.run.smithi083.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.78002.log.gz 2024-04-17T10:55:27.032 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.81641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33313.log 2024-04-17T10:55:27.032 INFO:teuthology.orchestra.run.smithi083.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.55179.log.gz 2024-04-17T10:55:27.032 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.81641.log.gz 2024-04-17T10:55:27.032 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.81532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72885.log 2024-04-17T10:55:27.033 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33313.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.81532.log.gz 2024-04-17T10:55:27.033 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37757.log 2024-04-17T10:55:27.033 INFO:teuthology.orchestra.run.smithi083.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33313.log.gz 2024-04-17T10:55:27.033 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52911.log 2024-04-17T10:55:27.033 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53417.log 2024-04-17T10:55:27.033 INFO:teuthology.orchestra.run.smithi083.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.72885.log.gz 2024-04-17T10:55:27.033 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52911.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37757.log.gz 2024-04-17T10:55:27.033 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53547.log 2024-04-17T10:55:27.034 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75299.log 2024-04-17T10:55:27.034 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53417.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.52911.log.gz 2024-04-17T10:55:27.034 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53547.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53417.log.gz 2024-04-17T10:55:27.034 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76670.log 2024-04-17T10:55:27.034 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53547.log.gz 2024-04-17T10:55:27.034 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75299.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-17T10:55:27.034 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.75299.log.gz 2024-04-17T10:55:27.034 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.76670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37942.log 2024-04-17T10:55:27.035 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69905.log 2024-04-17T10:55:27.035 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76670.log.gz 2024-04-17T10:55:27.035 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69345.log 2024-04-17T10:55:27.035 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37942.log.gz 2024-04-17T10:55:27.035 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50353.log 2024-04-17T10:55:27.035 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69233.log 2024-04-17T10:55:27.035 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69345.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69905.log.gz 2024-04-17T10:55:27.035 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78597.log 2024-04-17T10:55:27.036 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.69345.log.gz 2024-04-17T10:55:27.036 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50353.log.gz 2024-04-17T10:55:27.036 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54275.log 2024-04-17T10:55:27.036 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.69233.log.gz 2024-04-17T10:55:27.036 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.78597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36125.log 2024-04-17T10:55:27.036 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78597.log.gz 2024-04-17T10:55:27.036 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69605.log 2024-04-17T10:55:27.037 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54275.log: /var/log/ceph/ceph-client.admin.36125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56606.log 2024-04-17T10:55:27.037 INFO:teuthology.orchestra.run.smithi083.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54275.log.gz 2024-04-17T10:55:27.037 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69605.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36125.log.gz 2024-04-17T10:55:27.037 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36151.log 2024-04-17T10:55:27.037 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.69605.log.gz 2024-04-17T10:55:27.037 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50900.log 2024-04-17T10:55:27.037 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36151.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56606.log.gz 2024-04-17T10:55:27.037 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88119.log 2024-04-17T10:55:27.038 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36151.log.gz 2024-04-17T10:55:27.038 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75273.log 2024-04-17T10:55:27.038 INFO:teuthology.orchestra.run.smithi083.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50900.log.gz 2024-04-17T10:55:27.038 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88372.log 2024-04-17T10:55:27.038 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.88119.log.gz 2024-04-17T10:55:27.038 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74804.log 2024-04-17T10:55:27.038 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75273.log.gz 2024-04-17T10:55:27.038 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88632.log 2024-04-17T10:55:27.039 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.88372.log.gz 2024-04-17T10:55:27.039 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.74804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82558.log 2024-04-17T10:55:27.039 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35860.log 2024-04-17T10:55:27.039 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.88632.log.gz 2024-04-17T10:55:27.039 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.82558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88015.log 2024-04-17T10:55:27.039 INFO:teuthology.orchestra.run.smithi083.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.74804.log.gz 2024-04-17T10:55:27.039 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35860.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.82558.log.gz 2024-04-17T10:55:27.039 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35729.log 2024-04-17T10:55:27.040 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35860.log.gz/var/log/ceph/ceph-client.admin.88015.log: 2024-04-17T10:55:27.040 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79381.log 2024-04-17T10:55:27.040 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.88015.log.gz 2024-04-17T10:55:27.040 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72241.log 2024-04-17T10:55:27.040 INFO:teuthology.orchestra.run.smithi083.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35729.log.gz 2024-04-17T10:55:27.040 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72963.log 2024-04-17T10:55:27.040 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.79381.log: /var/log/ceph/ceph-client.admin.72241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55479.log 2024-04-17T10:55:27.040 INFO:teuthology.orchestra.run.smithi083.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.79381.log.gz 2024-04-17T10:55:27.041 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72241.log.gz 2024-04-17T10:55:27.041 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54037.log 2024-04-17T10:55:27.041 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78078.log 2024-04-17T10:55:27.041 INFO:teuthology.orchestra.run.smithi083.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55479.log.gz 89.0% 2024-04-17T10:55:27.041 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.72963.log.gz 2024-04-17T10:55:27.041 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54947.log 2024-04-17T10:55:27.041 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54037.log: /var/log/ceph/ceph-client.admin.78078.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54037.log.gzgzip 2024-04-17T10:55:27.041 INFO:teuthology.orchestra.run.smithi083.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.34461.log 2024-04-17T10:55:27.042 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.78078.log.gz 2024-04-17T10:55:27.042 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55721.log 2024-04-17T10:55:27.042 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36497.log 2024-04-17T10:55:27.042 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34461.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54947.log.gz 2024-04-17T10:55:27.042 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70166.log 2024-04-17T10:55:27.042 INFO:teuthology.orchestra.run.smithi083.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34461.log.gz 2024-04-17T10:55:27.042 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36497.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55721.log.gz 2024-04-17T10:55:27.043 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36602.log 2024-04-17T10:55:27.043 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36497.log.gz 2024-04-17T10:55:27.043 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34565.log 2024-04-17T10:55:27.043 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84860.log 2024-04-17T10:55:27.043 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36602.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70166.log.gz 2024-04-17T10:55:27.043 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52572.log 2024-04-17T10:55:27.043 INFO:teuthology.orchestra.run.smithi083.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36602.log.gz 2024-04-17T10:55:27.043 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34565.log.gz 2024-04-17T10:55:27.044 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.84860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78865.log 2024-04-17T10:55:27.044 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85491.log 2024-04-17T10:55:27.044 INFO:teuthology.orchestra.run.smithi083.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.84860.log.gz 2024-04-17T10:55:27.044 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.78865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87164.log 2024-04-17T10:55:27.044 INFO:teuthology.orchestra.run.smithi083.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.52572.log.gz 2024-04-17T10:55:27.044 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85491.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.78865.log.gz 2024-04-17T10:55:27.044 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87963.log 2024-04-17T10:55:27.044 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.85491.log.gz 2024-04-17T10:55:27.045 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87400.log 2024-04-17T10:55:27.045 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87963.log: gzip -5 --verbose -- 88.0% /var/log/ceph/ceph-client.admin.88197.log 2024-04-17T10:55:27.045 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.87164.log.gz 2024-04-17T10:55:27.045 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.87963.log.gz 2024-04-17T10:55:27.045 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85933.log 2024-04-17T10:55:27.045 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88197.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.87400.log.gz 2024-04-17T10:55:27.045 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76594.log 2024-04-17T10:55:27.045 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.88197.log.gz 2024-04-17T10:55:27.045 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82318.log 2024-04-17T10:55:27.046 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86981.log 2024-04-17T10:55:27.046 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85933.log.gz 2024-04-17T10:55:27.046 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.76594.log: /var/log/ceph/ceph-client.admin.82318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73472.log 2024-04-17T10:55:27.046 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76594.log.gz 2024-04-17T10:55:27.046 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.82318.log.gz 2024-04-17T10:55:27.046 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55591.log 2024-04-17T10:55:27.046 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.86981.log.gz 2024-04-17T10:55:27.047 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.73472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56736.log 2024-04-17T10:55:27.047 INFO:teuthology.orchestra.run.smithi083.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.73472.log.gz 2024-04-17T10:55:27.047 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79892.log 2024-04-17T10:55:27.047 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55591.log.gz 2024-04-17T10:55:27.047 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33447.log 2024-04-17T10:55:27.047 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76725.log 2024-04-17T10:55:27.047 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.79892.log: /var/log/ceph/ceph-client.admin.33447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79577.log 2024-04-17T10:55:27.047 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.79892.log.gz 2024-04-17T10:55:27.048 INFO:teuthology.orchestra.run.smithi083.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.56736.log.gz 2024-04-17T10:55:27.048 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33447.log.gz 2024-04-17T10:55:27.048 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55229.log 2024-04-17T10:55:27.048 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.76725.log: /var/log/ceph/ceph-client.admin.79577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86693.log 2024-04-17T10:55:27.048 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79577.log.gz 2024-04-17T10:55:27.048 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75214.log 2024-04-17T10:55:27.048 INFO:teuthology.orchestra.run.smithi083.stderr: 87.6%/var/log/ceph/ceph-client.admin.86693.log: gzip -- replaced with /var/log/ceph/ceph-client.admin.55229.log.gz -5 2024-04-17T10:55:27.048 INFO:teuthology.orchestra.run.smithi083.stderr: --verbose -- /var/log/ceph/ceph-client.admin.79761.log 2024-04-17T10:55:27.049 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75214.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.86693.log.gz 2024-04-17T10:55:27.049 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87686.log 2024-04-17T10:55:27.049 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.79761.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75214.log.gz 2024-04-17T10:55:27.049 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88896.log 2024-04-17T10:55:27.049 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87686.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79761.log.gz 2024-04-17T10:55:27.049 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82247.log 2024-04-17T10:55:27.049 INFO:teuthology.orchestra.run.smithi083.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.87686.log.gz 2024-04-17T10:55:27.049 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76360.log 2024-04-17T10:55:27.050 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.88896.log.gz 2024-04-17T10:55:27.050 INFO:teuthology.orchestra.run.smithi083.stderr: -5/var/log/ceph/ceph-client.admin.82247.log: --verbose -- /var/log/ceph/ceph-client.admin.27742.log 2024-04-17T10:55:27.050 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.76360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76438.log 2024-04-17T10:55:27.050 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.82247.log.gz 2024-04-17T10:55:27.050 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.27742.log: 89.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.76360.log.gz -5 2024-04-17T10:55:27.050 INFO:teuthology.orchestra.run.smithi083.stderr: --verbose -- /var/log/ceph/ceph-client.admin.54431.log 2024-04-17T10:55:27.050 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.27742.log.gz 2024-04-17T10:55:27.051 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.76438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51911.log 2024-04-17T10:55:27.051 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.76438.log.gz 2024-04-17T10:55:27.051 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54623.log 2024-04-17T10:55:27.051 INFO:teuthology.orchestra.run.smithi083.stderr:gzip/var/log/ceph/ceph-client.admin.51911.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.33884.log 2024-04-17T10:55:27.051 INFO:teuthology.orchestra.run.smithi083.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54431.log.gz 2024-04-17T10:55:27.051 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69048.log 2024-04-17T10:55:27.051 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51911.log.gz 2024-04-17T10:55:27.051 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33884.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54623.log.gz 2024-04-17T10:55:27.052 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84756.log 2024-04-17T10:55:27.052 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69048.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33884.log.gz 2024-04-17T10:55:27.052 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33134.log 2024-04-17T10:55:27.052 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.84756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72793.log 2024-04-17T10:55:27.052 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% 90.8% -- replaced with /var/log/ceph/ceph-client.admin.69048.log.gz 2024-04-17T10:55:27.052 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33134.log: -- replaced with /var/log/ceph/ceph-client.admin.84756.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54649.log 2024-04-17T10:55:27.052 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-04-17T10:55:27.052 INFO:teuthology.orchestra.run.smithi083.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33134.log.gz/var/log/ceph/ceph-client.admin.72793.log: 2024-04-17T10:55:27.053 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33837.log 2024-04-17T10:55:27.053 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72793.log.gz 2024-04-17T10:55:27.053 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85699.log 2024-04-17T10:55:27.053 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54301.log 2024-04-17T10:55:27.053 INFO:teuthology.orchestra.run.smithi083.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54649.log.gz 2024-04-17T10:55:27.053 INFO:teuthology.orchestra.run.smithi083.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33837.log.gz 2024-04-17T10:55:27.053 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82852.log 2024-04-17T10:55:27.053 INFO:teuthology.orchestra.run.smithi083.stderr:gzip/var/log/ceph/ceph-client.admin.54301.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.75060.log 2024-04-17T10:55:27.054 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.85699.log.gz 2024-04-17T10:55:27.054 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.82852.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54301.log.gz -5 2024-04-17T10:55:27.054 INFO:teuthology.orchestra.run.smithi083.stderr: --verbose -- /var/log/ceph/ceph-client.admin.79603.log 2024-04-17T10:55:27.054 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75060.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.82852.log.gz 2024-04-17T10:55:27.054 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82448.log 2024-04-17T10:55:27.054 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.79603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35436.log 2024-04-17T10:55:27.054 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.82448.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.79603.log.gz 2024-04-17T10:55:27.054 INFO:teuthology.orchestra.run.smithi083.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.75060.log.gz 2024-04-17T10:55:27.055 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33287.log 2024-04-17T10:55:27.055 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37731.log 2024-04-17T10:55:27.055 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.82448.log.gz 2024-04-17T10:55:27.055 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.80023.log 2024-04-17T10:55:27.055 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35436.log.gz 2024-04-17T10:55:27.055 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37731.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33287.log.gz 2024-04-17T10:55:27.055 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87745.log 2024-04-17T10:55:27.055 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.80023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50822.log 2024-04-17T10:55:27.056 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37731.log.gz 2024-04-17T10:55:27.056 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85777.log 2024-04-17T10:55:27.056 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72129.log 2024-04-17T10:55:27.056 INFO:teuthology.orchestra.run.smithi083.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.87745.log.gz 2024-04-17T10:55:27.056 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88870.log 2024-04-17T10:55:27.056 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50822.log.gz 2024-04-17T10:55:27.056 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35488.log 2024-04-17T10:55:27.056 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72129.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.85777.log.gz 2024-04-17T10:55:27.057 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88870.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72129.log.gz 2024-04-17T10:55:27.057 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69631.log 2024-04-17T10:55:27.057 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35488.log: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78282.log 2024-04-17T10:55:27.057 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.88870.log.gz 2024-04-17T10:55:27.057 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69631.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35488.log.gz 2024-04-17T10:55:27.057 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87719.log 2024-04-17T10:55:27.057 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74984.log 2024-04-17T10:55:27.057 INFO:teuthology.orchestra.run.smithi083.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69631.log.gz 2024-04-17T10:55:27.057 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.78282.log: /var/log/ceph/ceph-client.admin.87719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51412.log 2024-04-17T10:55:27.058 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78282.log.gz 2024-04-17T10:55:27.058 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1%gzip/var/log/ceph/ceph-client.admin.74984.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.85411.log 2024-04-17T10:55:27.058 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.87719.log.gz 2024-04-17T10:55:27.058 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.74984.log.gz 2024-04-17T10:55:27.058 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76334.log 2024-04-17T10:55:27.058 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51412.log: /var/log/ceph/ceph-client.admin.85411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52105.log 2024-04-17T10:55:27.058 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51412.log.gz/var/log/ceph/ceph-client.admin.76334.log: 2024-04-17T10:55:27.058 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36099.log 2024-04-17T10:55:27.059 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.85411.log.gz 2024-04-17T10:55:27.059 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76334.log.gz 2024-04-17T10:55:27.059 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52105.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52105.log.gz 2024-04-17T10:55:27.059 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69311.log 2024-04-17T10:55:27.059 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35306.log 2024-04-17T10:55:27.059 INFO:teuthology.orchestra.run.smithi083.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.36099.log.gz 2024-04-17T10:55:27.059 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70300.log 2024-04-17T10:55:27.059 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52622.log 2024-04-17T10:55:27.059 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35306.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.69311.log.gz 2024-04-17T10:55:27.060 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69657.log 2024-04-17T10:55:27.060 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35306.log.gz 2024-04-17T10:55:27.060 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70300.log.gz 2024-04-17T10:55:27.060 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52622.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-17T10:55:27.060 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69788.log 2024-04-17T10:55:27.060 INFO:teuthology.orchestra.run.smithi083.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.52622.log.gz 2024-04-17T10:55:27.060 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.69657.log.gz 2024-04-17T10:55:27.060 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54832.log 2024-04-17T10:55:27.061 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53261.log 2024-04-17T10:55:27.061 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69788.log: /var/log/ceph/ceph-client.admin.54832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53443.log 2024-04-17T10:55:27.061 INFO:teuthology.orchestra.run.smithi083.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.69788.log.gz 2024-04-17T10:55:27.061 INFO:teuthology.orchestra.run.smithi083.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54832.log.gz 2024-04-17T10:55:27.061 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53261.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53261.log.gz 2024-04-17T10:55:27.061 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55539.log 2024-04-17T10:55:27.061 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53521.log 2024-04-17T10:55:27.061 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55539.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53443.log.gz 2024-04-17T10:55:27.061 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79434.log 2024-04-17T10:55:27.062 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37601.log 2024-04-17T10:55:27.062 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1%/var/log/ceph/ceph-client.admin.53521.log: -- replaced with /var/log/ceph/ceph-client.admin.55539.log.gz 2024-04-17T10:55:27.062 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.79434.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53521.log.gz 2024-04-17T10:55:27.062 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.79434.log.gz 2024-04-17T10:55:27.062 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82826.log 2024-04-17T10:55:27.062 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55896.log 2024-04-17T10:55:27.062 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37601.log: /var/log/ceph/ceph-client.admin.82826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36966.log 2024-04-17T10:55:27.063 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37601.log.gz 2024-04-17T10:55:27.063 INFO:teuthology.orchestra.run.smithi083.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.82826.log.gz 2024-04-17T10:55:27.063 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55896.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55896.log.gz 2024-04-17T10:55:27.063 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33681.log 2024-04-17T10:55:27.063 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34643.log 2024-04-17T10:55:27.063 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1%/var/log/ceph/ceph-client.admin.33681.log: -- replaced with /var/log/ceph/ceph-client.admin.36966.log.gz 2024-04-17T10:55:27.063 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52130.log 2024-04-17T10:55:27.063 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33681.log.gz 2024-04-17T10:55:27.063 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34643.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34643.log.gz 2024-04-17T10:55:27.064 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53365.log 2024-04-17T10:55:27.064 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35886.log 2024-04-17T10:55:27.064 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51751.log 2024-04-17T10:55:27.064 INFO:teuthology.orchestra.run.smithi083.stderr: 91.0% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52130.log.gz 2024-04-17T10:55:27.064 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53365.log.gz 2024-04-17T10:55:27.064 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35886.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35886.log.gz 2024-04-17T10:55:27.064 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75647.log 2024-04-17T10:55:27.064 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78969.log 2024-04-17T10:55:27.064 INFO:teuthology.orchestra.run.smithi083.stderr: 89.4%/var/log/ceph/ceph-client.admin.75647.log: -- replaced with /var/log/ceph/ceph-client.admin.51751.log.gzgzip 2024-04-17T10:55:27.065 INFO:teuthology.orchestra.run.smithi083.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.33551.log 2024-04-17T10:55:27.065 INFO:teuthology.orchestra.run.smithi083.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.75647.log.gz 2024-04-17T10:55:27.065 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.78969.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.78969.log.gz 2024-04-17T10:55:27.065 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88476.log 2024-04-17T10:55:27.065 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33169.log 2024-04-17T10:55:27.065 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88476.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33551.log.gz 2024-04-17T10:55:27.065 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81506.log 2024-04-17T10:55:27.065 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83090.log 2024-04-17T10:55:27.066 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88476.log.gz 2024-04-17T10:55:27.066 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33169.log: /var/log/ceph/ceph-client.admin.81506.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33169.log.gz 2024-04-17T10:55:27.066 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.81506.log.gz 2024-04-17T10:55:27.066 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69814.log 2024-04-17T10:55:27.066 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.83090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54780.log 2024-04-17T10:55:27.066 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1%/var/log/ceph/ceph-client.admin.69814.log: -- replaced with /var/log/ceph/ceph-client.admin.83090.log.gz 2024-04-17T10:55:27.066 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35384.log 2024-04-17T10:55:27.066 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.69814.log.gz 2024-04-17T10:55:27.067 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54780.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54780.log.gz 2024-04-17T10:55:27.067 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70378.log 2024-04-17T10:55:27.067 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.73527.log 2024-04-17T10:55:27.067 INFO:teuthology.orchestra.run.smithi083.stderr: 87.7%/var/log/ceph/ceph-client.admin.70378.log: -- replaced with /var/log/ceph/ceph-client.admin.35384.log.gz 2024-04-17T10:55:27.067 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84834.log 2024-04-17T10:55:27.067 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56209.log 2024-04-17T10:55:27.067 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.70378.log.gz 2024-04-17T10:55:27.067 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.73527.log: /var/log/ceph/ceph-client.admin.84834.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.84834.log.gz 2024-04-17T10:55:27.068 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86929.log 2024-04-17T10:55:27.068 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50039.log 2024-04-17T10:55:27.068 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56209.log: /var/log/ceph/ceph-client.admin.86929.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.56209.log.gz 2024-04-17T10:55:27.068 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.86929.log.gz 2024-04-17T10:55:27.068 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79094.log 2024-04-17T10:55:27.068 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81480.log 2024-04-17T10:55:27.068 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50039.log: /var/log/ceph/ceph-client.admin.79094.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79094.log.gz 2024-04-17T10:55:27.068 INFO:teuthology.orchestra.run.smithi083.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.50039.log.gz 2024-04-17T10:55:27.069 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79968.log 2024-04-17T10:55:27.069 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.81480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87007.log 2024-04-17T10:55:27.069 INFO:teuthology.orchestra.run.smithi083.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.81480.log.gz/var/log/ceph/ceph-client.admin.79968.log: 2024-04-17T10:55:27.069 INFO:teuthology.orchestra.run.smithi083.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.79968.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70218.log 2024-04-17T10:55:27.069 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-04-17T10:55:27.069 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50621.log 2024-04-17T10:55:27.069 INFO:teuthology.orchestra.run.smithi083.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.87007.log.gz 2024-04-17T10:55:27.069 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70218.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-17T10:55:27.070 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50621.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.70218.log.gz 2024-04-17T10:55:27.070 INFO:teuthology.orchestra.run.smithi083.stderr: 89.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81584.log 2024-04-17T10:55:27.070 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50621.log.gz 2024-04-17T10:55:27.070 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73342.log 2024-04-17T10:55:27.070 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.81584.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.81584.log.gz 2024-04-17T10:55:27.070 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84501.log 2024-04-17T10:55:27.070 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.73342.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73342.log.gz 2024-04-17T10:55:27.070 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53618.log 2024-04-17T10:55:27.071 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.84501.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.84501.log.gz 2024-04-17T10:55:27.071 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56396.log 2024-04-17T10:55:27.071 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53618.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53618.log.gz 2024-04-17T10:55:27.071 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73019.log 2024-04-17T10:55:27.071 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56396.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56396.log.gz 2024-04-17T10:55:27.071 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36940.log 2024-04-17T10:55:27.071 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.73019.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73019.log.gz 2024-04-17T10:55:27.072 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88041.log 2024-04-17T10:55:27.072 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36940.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36940.log.gz 2024-04-17T10:55:27.072 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34357.log 2024-04-17T10:55:27.072 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88041.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88041.log.gz 2024-04-17T10:55:27.072 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33629.log 2024-04-17T10:55:27.072 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34357.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34357.log.gz 2024-04-17T10:55:27.085 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53904.log 2024-04-17T10:55:27.085 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33629.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33629.log.gz 2024-04-17T10:55:27.085 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52180.log 2024-04-17T10:55:27.085 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53904.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53904.log.gz 2024-04-17T10:55:27.085 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55401.log 2024-04-17T10:55:27.086 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52180.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.52180.log.gz 2024-04-17T10:55:27.086 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35644.log 2024-04-17T10:55:27.086 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55401.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55401.log.gz 2024-04-17T10:55:27.086 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53839.log 2024-04-17T10:55:27.086 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35644.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35644.log.gz 2024-04-17T10:55:27.086 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53175.log 2024-04-17T10:55:27.086 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53839.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53839.log.gz 2024-04-17T10:55:27.086 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76253.log 2024-04-17T10:55:27.087 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53175.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53175.log.gz 2024-04-17T10:55:27.087 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86195.log 2024-04-17T10:55:27.087 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.76253.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.76253.log.gz 2024-04-17T10:55:27.087 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35592.log 2024-04-17T10:55:27.087 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86195.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.86195.log.gz 2024-04-17T10:55:27.087 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84600.log 2024-04-17T10:55:27.087 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35592.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35592.log.gz 2024-04-17T10:55:27.088 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88346.log 2024-04-17T10:55:27.088 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.84600.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84600.log.gz 2024-04-17T10:55:27.088 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37274.log 2024-04-17T10:55:27.088 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88346.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.88346.log.gz 2024-04-17T10:55:27.088 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69710.log 2024-04-17T10:55:27.088 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37274.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37274.log.gz 2024-04-17T10:55:27.088 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53592.log 2024-04-17T10:55:27.089 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69710.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69710.log.gz 2024-04-17T10:55:27.089 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35150.log 2024-04-17T10:55:27.089 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53592.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53592.log.gz 2024-04-17T10:55:27.089 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88658.log 2024-04-17T10:55:27.089 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35150.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35150.log.gz 2024-04-17T10:55:27.089 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55343.log 2024-04-17T10:55:27.089 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88658.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.88658.log.gz 2024-04-17T10:55:27.089 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69475.log 2024-04-17T10:55:27.090 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55343.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.55343.log.gz 2024-04-17T10:55:27.090 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54197.log 2024-04-17T10:55:27.090 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69475.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.69475.log.gz 2024-04-17T10:55:27.090 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85174.log 2024-04-17T10:55:27.090 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54197.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54197.log.gz 2024-04-17T10:55:27.090 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72728.log 2024-04-17T10:55:27.090 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85174.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.85174.log.gz 2024-04-17T10:55:27.090 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84964.log 2024-04-17T10:55:27.091 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72728.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72728.log.gz 2024-04-17T10:55:27.091 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56841.log 2024-04-17T10:55:27.091 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.84964.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.84964.log.gz 2024-04-17T10:55:27.091 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50796.log 2024-04-17T10:55:27.091 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.56841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79042.log 2024-04-17T10:55:27.091 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50796.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75413.log 2024-04-17T10:55:27.091 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50796.log.gz 2024-04-17T10:55:27.092 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.79042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83064.log 2024-04-17T10:55:27.092 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75413.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.79042.log.gz 2024-04-17T10:55:27.092 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75413.log.gz 2024-04-17T10:55:27.103 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34098.log 2024-04-17T10:55:27.104 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.83064.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.83064.log.gz 2024-04-17T10:55:27.111 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50413.log 2024-04-17T10:55:27.112 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34098.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34098.log.gz 2024-04-17T10:55:27.119 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84808.log 2024-04-17T10:55:27.120 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50413.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50413.log.gz 2024-04-17T10:55:27.127 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70048.log 2024-04-17T10:55:27.128 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.84808.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.84808.log.gz 2024-04-17T10:55:27.139 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74958.log 2024-04-17T10:55:27.140 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70048.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70048.log.gz 2024-04-17T10:55:27.141 INFO:teuthology.orchestra.run.smithi083.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-17T10:55:27.141 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36758.log 2024-04-17T10:55:27.142 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.74958.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.74958.log.gz 2024-04-17T10:55:27.142 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55773.log 2024-04-17T10:55:27.143 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36758.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36758.log.gz 2024-04-17T10:55:27.143 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50327.log 2024-04-17T10:55:27.144 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55773.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55773.log.gz 2024-04-17T10:55:27.144 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56786.log 2024-04-17T10:55:27.145 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50327.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50327.log.gz 2024-04-17T10:55:27.145 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34094.log 2024-04-17T10:55:27.146 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56786.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56786.log.gz 2024-04-17T10:55:27.147 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52332.log 2024-04-17T10:55:27.147 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34094.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34094.log.gz 2024-04-17T10:55:27.148 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87348.log 2024-04-17T10:55:27.149 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52332.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52332.log.gz 2024-04-17T10:55:27.149 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80103.log 2024-04-17T10:55:27.149 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87348.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.87348.log.gz 2024-04-17T10:55:27.150 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54457.log 2024-04-17T10:55:27.151 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.80103.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.80103.log.gz 2024-04-17T10:55:27.151 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33880.log 2024-04-17T10:55:27.152 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54457.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.54457.log.gz 2024-04-17T10:55:27.153 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83142.log 2024-04-17T10:55:27.153 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33880.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33880.log.gz 2024-04-17T10:55:27.154 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85200.log 2024-04-17T10:55:27.155 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.83142.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.83142.log.gz 2024-04-17T10:55:27.155 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82709.log 2024-04-17T10:55:27.156 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85200.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.85200.log.gz 2024-04-17T10:55:27.156 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51671.log 2024-04-17T10:55:27.157 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.82709.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.82709.log.gz 2024-04-17T10:55:27.157 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51464.log 2024-04-17T10:55:27.158 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51671.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51671.log.gz 2024-04-17T10:55:27.159 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51334.log 2024-04-17T10:55:27.159 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51464.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51464.log.gz 2024-04-17T10:55:27.160 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37404.log 2024-04-17T10:55:27.160 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51334.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51334.log.gz 2024-04-17T10:55:27.161 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87216.log 2024-04-17T10:55:27.162 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37404.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37404.log.gz 2024-04-17T10:55:27.162 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33419.log 2024-04-17T10:55:27.163 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87216.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.87216.log.gz 2024-04-17T10:55:27.164 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72624.log 2024-04-17T10:55:27.164 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33419.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33419.log.gz 2024-04-17T10:55:27.165 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79735.log 2024-04-17T10:55:27.165 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72624.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72624.log.gz 2024-04-17T10:55:27.166 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78409.log 2024-04-17T10:55:27.166 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.79735.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.79735.log.gz 2024-04-17T10:55:27.167 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81849.log 2024-04-17T10:55:27.168 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.78409.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.78409.log.gz 2024-04-17T10:55:27.168 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35755.log 2024-04-17T10:55:27.169 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.81849.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.81849.log.gz 2024-04-17T10:55:27.169 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82800.log 2024-04-17T10:55:27.170 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35755.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35755.log.gz 2024-04-17T10:55:27.170 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54353.log 2024-04-17T10:55:27.171 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.82800.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.82800.log.gz 2024-04-17T10:55:27.172 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36257.log 2024-04-17T10:55:27.172 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54353.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54353.log.gz 2024-04-17T10:55:27.173 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84990.log 2024-04-17T10:55:27.173 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36257.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36257.log.gz 2024-04-17T10:55:27.174 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54562.log 2024-04-17T10:55:27.174 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.84990.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84990.log.gz 2024-04-17T10:55:27.175 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34305.log 2024-04-17T10:55:27.178 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.54562.log: 92.2% -- replaced with /var/log/ceph/ceph-client.0.54562.log.gz 2024-04-17T10:55:27.178 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82507.log 2024-04-17T10:55:27.179 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34305.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34305.log.gz 2024-04-17T10:55:27.179 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55948.log 2024-04-17T10:55:27.180 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.82507.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.82507.log.gz 2024-04-17T10:55:27.180 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70192.log 2024-04-17T10:55:27.181 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55948.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55948.log.gz 2024-04-17T10:55:27.182 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78539.log 2024-04-17T10:55:27.183 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70192.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70192.log.gz 2024-04-17T10:55:27.184 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75439.log 2024-04-17T10:55:27.184 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.78539.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.78539.log.gz 2024-04-17T10:55:27.185 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36654.log 2024-04-17T10:55:27.185 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75439.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.75439.log.gz 2024-04-17T10:55:27.186 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54950.log 2024-04-17T10:55:27.186 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36654.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36654.log.gz 2024-04-17T10:55:27.186 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85357.log 2024-04-17T10:55:27.187 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54950.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54950.log.gz 2024-04-17T10:55:27.188 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75770.log 2024-04-17T10:55:27.188 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85357.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.85357.log.gz 2024-04-17T10:55:27.189 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82984.log 2024-04-17T10:55:27.190 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75770.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75770.log.gz 2024-04-17T10:55:27.190 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69579.log 2024-04-17T10:55:27.191 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.82984.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.82984.log.gz 2024-04-17T10:55:27.191 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85016.log 2024-04-17T10:55:27.192 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69579.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69579.log.gz 2024-04-17T10:55:27.193 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37169.log 2024-04-17T10:55:27.193 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85016.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.85016.log.gz 2024-04-17T10:55:27.194 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51540.log 2024-04-17T10:55:27.194 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37169.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37169.log.gz 2024-04-17T10:55:27.195 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88398.log 2024-04-17T10:55:27.195 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51540.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51540.log.gz 2024-04-17T10:55:27.196 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76620.log 2024-04-17T10:55:27.197 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88398.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.88398.log.gz 2024-04-17T10:55:27.197 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35280.log 2024-04-17T10:55:27.198 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.76620.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.76620.log.gz 2024-04-17T10:55:27.199 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82370.log 2024-04-17T10:55:27.199 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35280.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35280.log.gz 2024-04-17T10:55:27.200 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34331.log 2024-04-17T10:55:27.200 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.82370.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.82370.log.gz 2024-04-17T10:55:27.201 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52408.log 2024-04-17T10:55:27.202 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34331.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34331.log.gz 2024-04-17T10:55:27.202 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79172.log 2024-04-17T10:55:27.203 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52408.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52408.log.gz 2024-04-17T10:55:27.203 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69285.log 2024-04-17T10:55:27.204 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.79172.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79172.log.gz 2024-04-17T10:55:27.204 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56316.log 2024-04-17T10:55:27.205 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69285.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69285.log.gz 2024-04-17T10:55:27.206 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81428.log 2024-04-17T10:55:27.207 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56316.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56316.log.gz 2024-04-17T10:55:27.207 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52786.log 2024-04-17T10:55:27.208 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.81428.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.81428.log.gz 2024-04-17T10:55:27.208 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51831.log 2024-04-17T10:55:27.209 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52786.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.52786.log.gz 2024-04-17T10:55:27.209 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82474.log 2024-04-17T10:55:27.210 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51831.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51831.log.gz 2024-04-17T10:55:27.211 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82880.log 2024-04-17T10:55:27.211 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.82474.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.82474.log.gz 2024-04-17T10:55:27.212 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37458.log 2024-04-17T10:55:27.213 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.82880.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.82880.log.gz 2024-04-17T10:55:27.213 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34487.log 2024-04-17T10:55:27.214 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37458.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37458.log.gz 2024-04-17T10:55:27.214 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33811.log 2024-04-17T10:55:27.215 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34487.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34487.log.gz 2024-04-17T10:55:27.215 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79787.log 2024-04-17T10:55:27.216 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33811.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33811.log.gz 2024-04-17T10:55:27.216 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54065.log 2024-04-17T10:55:27.217 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.79787.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.79787.log.gz 2024-04-17T10:55:27.218 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78156.log 2024-04-17T10:55:27.219 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54065.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54065.log.gz 2024-04-17T10:55:27.219 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78839.log 2024-04-17T10:55:27.220 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.78156.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.78156.log.gz 2024-04-17T10:55:27.220 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50965.log 2024-04-17T10:55:27.221 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.78839.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.78839.log.gz 2024-04-17T10:55:27.221 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56086.log 2024-04-17T10:55:27.222 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50965.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50965.log.gz 2024-04-17T10:55:27.223 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72155.log 2024-04-17T10:55:27.224 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56086.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56086.log.gz 2024-04-17T10:55:27.224 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78571.log 2024-04-17T10:55:27.225 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72155.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72155.log.gz 2024-04-17T10:55:27.225 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36283.log 2024-04-17T10:55:27.226 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.78571.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.78571.log.gz 2024-04-17T10:55:27.226 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88580.log 2024-04-17T10:55:27.227 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36283.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36283.log.gz 2024-04-17T10:55:27.227 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56895.log 2024-04-17T10:55:27.228 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88580.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.88580.log.gz 2024-04-17T10:55:27.228 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75162.log 2024-04-17T10:55:27.229 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56895.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56895.log.gz 2024-04-17T10:55:27.229 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88554.log 2024-04-17T10:55:27.230 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75162.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75162.log.gz 2024-04-17T10:55:27.230 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78435.log 2024-04-17T10:55:27.231 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88554.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.88554.log.gz 2024-04-17T10:55:27.232 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76542.log 2024-04-17T10:55:27.232 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.78435.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.78435.log.gz 2024-04-17T10:55:27.233 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70326.log 2024-04-17T10:55:27.234 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.76542.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.76542.log.gz 2024-04-17T10:55:27.234 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54754.log 2024-04-17T10:55:27.235 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70326.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.70326.log.gz 2024-04-17T10:55:27.235 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86301.log 2024-04-17T10:55:27.236 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54754.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54754.log.gz 2024-04-17T10:55:27.236 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70248.log 2024-04-17T10:55:27.237 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86301.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.86301.log.gz 2024-04-17T10:55:27.237 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76190.log 2024-04-17T10:55:27.238 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70248.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.70248.log.gz 2024-04-17T10:55:27.238 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37018.log 2024-04-17T10:55:27.239 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.76190.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76190.log.gz 2024-04-17T10:55:27.239 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76490.log 2024-04-17T10:55:27.240 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37018.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37018.log.gz 2024-04-17T10:55:27.240 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-17T10:55:27.241 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.76490.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76490.log.gz 2024-04-17T10:55:27.242 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72911.log 2024-04-17T10:55:27.255 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87059.log 2024-04-17T10:55:27.256 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72911.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72911.log.gz 2024-04-17T10:55:27.279 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72293.log 2024-04-17T10:55:27.280 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87059.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.87059.log.gz 2024-04-17T10:55:27.280 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79551.log 2024-04-17T10:55:27.281 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72293.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72293.log.gz 2024-04-17T10:55:27.295 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69449.log 2024-04-17T10:55:27.295 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.79551.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.79551.log.gz 2024-04-17T10:55:27.303 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51438.log 2024-04-17T10:55:27.304 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69449.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69449.log.gz 2024-04-17T10:55:27.311 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36362.log 2024-04-17T10:55:27.312 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51438.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51438.log.gz 2024-04-17T10:55:27.319 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36309.log 2024-04-17T10:55:27.320 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36362.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36362.log.gz 2024-04-17T10:55:27.327 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86143.log 2024-04-17T10:55:27.328 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36309.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36309.log.gz 2024-04-17T10:55:27.335 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87085.log 2024-04-17T10:55:27.336 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86143.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.86143.log.gz 2024-04-17T10:55:27.343 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33339.log 2024-04-17T10:55:27.344 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87085.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.87085.log.gz 2024-04-17T10:55:27.351 INFO:teuthology.orchestra.run.smithi167.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-17T10:55:27.352 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37679.log 2024-04-17T10:55:27.352 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33339.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33339.log.gz 2024-04-17T10:55:27.363 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35462.log 2024-04-17T10:55:27.364 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37679.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37679.log.gz 2024-04-17T10:55:27.372 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37833.log 2024-04-17T10:55:27.372 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35462.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35462.log.gz 2024-04-17T10:55:27.379 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54091.log 2024-04-17T10:55:27.380 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37833.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37833.log.gz 2024-04-17T10:55:27.387 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85252.log 2024-04-17T10:55:27.388 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54091.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54091.log.gz 2024-04-17T10:55:27.395 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72397.log 2024-04-17T10:55:27.396 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85252.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.85252.log.gz 2024-04-17T10:55:27.407 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55453.log 2024-04-17T10:55:27.408 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72397.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72397.log.gz 2024-04-17T10:55:27.415 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84572.log 2024-04-17T10:55:27.416 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55453.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55453.log.gz 2024-04-17T10:55:27.423 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82958.log 2024-04-17T10:55:27.424 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.84572.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.84572.log.gz 2024-04-17T10:55:27.431 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84678.log 2024-04-17T10:55:27.432 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.82958.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.82958.log.gz 2024-04-17T10:55:27.439 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71993.log 2024-04-17T10:55:27.440 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.84678.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.84678.log.gz 2024-04-17T10:55:27.447 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69151.log 2024-04-17T10:55:27.448 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.71993.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.71993.log.gz 2024-04-17T10:55:27.455 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80077.log 2024-04-17T10:55:27.456 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69151.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.69151.log.gz 2024-04-17T10:55:27.463 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83036.log 2024-04-17T10:55:27.464 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.80077.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80077.log.gz 2024-04-17T10:55:27.471 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88788.log 2024-04-17T10:55:27.472 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.83036.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.83036.log.gz 2024-04-17T10:55:27.479 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81402.log 2024-04-17T10:55:27.480 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88788.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.88788.log.gz 2024-04-17T10:55:27.487 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74880.log 2024-04-17T10:55:27.488 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.81402.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.81402.log.gz 2024-04-17T10:55:27.495 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50848.log 2024-04-17T10:55:27.496 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.74880.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74880.log.gz 2024-04-17T10:55:27.503 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56580.log 2024-04-17T10:55:27.504 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50848.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50848.log.gz 2024-04-17T10:55:27.523 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34513.log 2024-04-17T10:55:27.524 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56580.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56580.log.gz 2024-04-17T10:55:27.531 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51193.log 2024-04-17T10:55:27.532 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34513.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34513.log.gz 2024-04-17T10:55:27.539 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51701.log 2024-04-17T10:55:27.540 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51193.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51193.log.gz 2024-04-17T10:55:27.547 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75960.log 2024-04-17T10:55:27.548 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51701.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.51701.log.gz 2024-04-17T10:55:27.551 INFO:teuthology.orchestra.run.smithi167.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-17T10:55:27.555 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69527.log 2024-04-17T10:55:27.556 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75960.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75960.log.gz 2024-04-17T10:55:27.563 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85751.log 2024-04-17T10:55:27.564 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69527.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69527.log.gz 2024-04-17T10:55:27.572 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79068.log 2024-04-17T10:55:27.572 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85751.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.85751.log.gz 2024-04-17T10:55:27.579 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86851.log 2024-04-17T10:55:27.580 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.79068.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.79068.log.gz 2024-04-17T10:55:27.587 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85331.log 2024-04-17T10:55:27.588 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86851.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.86851.log.gz 2024-04-17T10:55:27.595 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51490.log 2024-04-17T10:55:27.596 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85331.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.85331.log.gz 2024-04-17T10:55:27.603 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51595.log 2024-04-17T10:55:27.604 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51490.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.51490.log.gz 2024-04-17T10:55:27.611 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52961.log 2024-04-17T10:55:27.619 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.51595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82605.log 2024-04-17T10:55:27.620 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52961.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.52961.log.gz 2024-04-17T10:55:27.627 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36862.log 2024-04-17T10:55:27.628 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.82605.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.82605.log.gz 2024-04-17T10:55:27.635 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54171.log 2024-04-17T10:55:27.636 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36862.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36862.log.gz 2024-04-17T10:55:27.643 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35540.log 2024-04-17T10:55:27.644 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54171.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54171.log.gz 2024-04-17T10:55:27.651 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78309.log 2024-04-17T10:55:27.652 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35540.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35540.log.gz 2024-04-17T10:55:27.659 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87426.log 2024-04-17T10:55:27.660 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.78309.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.78309.log.gz 2024-04-17T10:55:27.679 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72546.log 2024-04-17T10:55:27.680 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87426.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.87426.log.gz 2024-04-17T10:55:27.691 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-17T10:55:27.692 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72546.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.72546.log.gz 2024-04-17T10:55:27.699 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34097.log 2024-04-17T10:55:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33473.log 2024-04-17T10:55:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34097.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34097.log.gz 2024-04-17T10:55:27.719 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70430.log 2024-04-17T10:55:27.720 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33473.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33473.log.gz 2024-04-17T10:55:27.727 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36231.log 2024-04-17T10:55:27.728 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70430.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.70430.log.gz 2024-04-17T10:55:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82292.log 2024-04-17T10:55:27.736 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36231.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36231.log.gz 2024-04-17T10:55:27.743 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37653.log 2024-04-17T10:55:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.82292.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.82292.log.gz 2024-04-17T10:55:27.752 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79146.log 2024-04-17T10:55:27.752 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37653.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37653.log.gz 2024-04-17T10:55:27.759 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36177.log 2024-04-17T10:55:27.759 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.79146.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79146.log.gz 2024-04-17T10:55:27.767 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85569.log 2024-04-17T10:55:27.768 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36177.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36177.log.gz 2024-04-17T10:55:27.775 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33888.log 2024-04-17T10:55:27.776 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85569.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.85569.log.gz 2024-04-17T10:55:27.783 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75673.log 2024-04-17T10:55:27.784 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33888.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33888.log.gz 2024-04-17T10:55:27.791 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56344.log 2024-04-17T10:55:27.791 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75673.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.75673.log.gz 2024-04-17T10:55:27.799 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73264.log 2024-04-17T10:55:27.800 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56344.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56344.log.gz 2024-04-17T10:55:27.815 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79198.log 2024-04-17T10:55:27.816 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.73264.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73264.log.gz 2024-04-17T10:55:27.839 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36784.log 2024-04-17T10:55:27.840 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.79198.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.79198.log.gz 2024-04-17T10:55:27.855 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84475.log 2024-04-17T10:55:27.856 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36784.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36784.log.gz 2024-04-17T10:55:27.867 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85543.log 2024-04-17T10:55:27.868 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.84475.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.84475.log.gz 2024-04-17T10:55:27.879 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75848.log 2024-04-17T10:55:27.879 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85543.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.85543.log.gz 2024-04-17T10:55:27.887 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75621.log 2024-04-17T10:55:27.888 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75848.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75848.log.gz 2024-04-17T10:55:27.895 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33733.log 2024-04-17T10:55:27.896 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75621.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75621.log.gz 2024-04-17T10:55:27.903 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55617.log 2024-04-17T10:55:27.903 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33733.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33733.log.gz 2024-04-17T10:55:27.919 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75874.log 2024-04-17T10:55:27.920 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55617.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55617.log.gz 2024-04-17T10:55:27.935 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37547.log 2024-04-17T10:55:27.936 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75874.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.75874.log.gz 2024-04-17T10:55:27.951 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35566.log 2024-04-17T10:55:27.952 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37547.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37547.log.gz 2024-04-17T10:55:27.967 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87138.log 2024-04-17T10:55:27.968 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35566.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35566.log.gz 2024-04-17T10:55:27.975 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54117.log 2024-04-17T10:55:27.976 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87138.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.87138.log.gz 2024-04-17T10:55:27.992 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71813.log 2024-04-17T10:55:27.992 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54117.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54117.log.gz 2024-04-17T10:55:27.999 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73368.log 2024-04-17T10:55:28.000 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.71813.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.71813.log.gz 2024-04-17T10:55:28.007 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54889.log 2024-04-17T10:55:28.008 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.73368.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73368.log.gz 2024-04-17T10:55:28.016 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71863.log 2024-04-17T10:55:28.016 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54889.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54889.log.gz 2024-04-17T10:55:28.031 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86459.log 2024-04-17T10:55:28.032 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.71863.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.71863.log.gz 2024-04-17T10:55:28.039 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79407.log 2024-04-17T10:55:28.040 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86459.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.86459.log.gz 2024-04-17T10:55:28.040 INFO:teuthology.orchestra.run.smithi083.stderr: 93.7% -- replaced with /var/log/ceph/ceph-client.0.80023.log.gz 2024-04-17T10:55:28.041 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86903.log 2024-04-17T10:55:28.042 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.79407.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.79407.log.gz 2024-04-17T10:55:28.042 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33603.log 2024-04-17T10:55:28.043 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86903.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.86903.log.gz 2024-04-17T10:55:28.044 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35912.log 2024-04-17T10:55:28.044 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33603.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33603.log.gz 2024-04-17T10:55:28.045 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50089.log 2024-04-17T10:55:28.046 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35912.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35912.log.gz 2024-04-17T10:55:28.046 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52861.log 2024-04-17T10:55:28.047 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50089.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.50089.log.gz 2024-04-17T10:55:28.047 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53774.log 2024-04-17T10:55:28.049 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52861.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.52861.log.gz 2024-04-17T10:55:28.049 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55669.log 2024-04-17T10:55:28.050 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53774.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53774.log.gz 2024-04-17T10:55:28.050 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55695.log 2024-04-17T10:55:28.051 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55669.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55669.log.gz 2024-04-17T10:55:28.052 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82065.log 2024-04-17T10:55:28.053 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55695.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55695.log.gz 2024-04-17T10:55:28.053 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55827.log 2024-04-17T10:55:28.053 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.82065.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.82065.log.gz 2024-04-17T10:55:28.054 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34539.log 2024-04-17T10:55:28.055 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55827.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55827.log.gz 2024-04-17T10:55:28.055 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87885.log 2024-04-17T10:55:28.056 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34539.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34539.log.gz 2024-04-17T10:55:28.057 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87771.log 2024-04-17T10:55:28.057 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87885.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.87885.log.gz 2024-04-17T10:55:28.058 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81927.log 2024-04-17T10:55:28.059 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87771.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.87771.log.gz 2024-04-17T10:55:28.059 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36471.log 2024-04-17T10:55:28.060 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.81927.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.81927.log.gz 2024-04-17T10:55:28.060 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88684.log 2024-04-17T10:55:28.061 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36471.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36471.log.gz 2024-04-17T10:55:28.062 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33525.log 2024-04-17T10:55:28.062 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88684.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.88684.log.gz 2024-04-17T10:55:28.063 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81454.log 2024-04-17T10:55:28.063 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33525.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33525.log.gz 2024-04-17T10:55:28.064 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54009.log 2024-04-17T10:55:28.065 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.81454.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.81454.log.gz 2024-04-17T10:55:28.065 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53087.log 2024-04-17T10:55:28.066 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54009.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54009.log.gz 2024-04-17T10:55:28.066 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86169.log 2024-04-17T10:55:28.067 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53087.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53087.log.gz 2024-04-17T10:55:28.068 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37430.log 2024-04-17T10:55:28.068 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86169.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.86169.log.gz 2024-04-17T10:55:28.069 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79657.log 2024-04-17T10:55:28.070 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37430.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37430.log.gz 2024-04-17T10:55:28.070 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68948.log 2024-04-17T10:55:28.071 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.79657.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79657.log.gz 2024-04-17T10:55:28.071 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37352.log 2024-04-17T10:55:28.073 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.68948.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.68948.log.gz 2024-04-17T10:55:28.073 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68998.log 2024-04-17T10:55:28.074 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37352.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37352.log.gz 2024-04-17T10:55:28.074 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36388.log 2024-04-17T10:55:28.076 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.68998.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.68998.log.gz 2024-04-17T10:55:28.076 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83218.log 2024-04-17T10:55:28.077 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36388.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36388.log.gz 2024-04-17T10:55:28.077 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35176.log 2024-04-17T10:55:28.078 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.83218.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.83218.log.gz 2024-04-17T10:55:28.078 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84449.log 2024-04-17T10:55:28.079 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35176.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35176.log.gz 2024-04-17T10:55:28.079 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33885.log 2024-04-17T10:55:28.080 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.84449.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.84449.log.gz 2024-04-17T10:55:28.081 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87937.log 2024-04-17T10:55:28.081 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33885.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33885.log.gz 2024-04-17T10:55:28.082 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72937.log 2024-04-17T10:55:28.083 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87937.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87937.log.gz 2024-04-17T10:55:28.083 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84704.log 2024-04-17T10:55:28.084 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72937.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72937.log.gz 2024-04-17T10:55:28.084 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54913.log 2024-04-17T10:55:28.085 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.84704.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.84704.log.gz 2024-04-17T10:55:28.086 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37070.log 2024-04-17T10:55:28.086 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54913.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.54913.log.gz 2024-04-17T10:55:28.087 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84938.log 2024-04-17T10:55:28.088 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37070.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37070.log.gz 2024-04-17T10:55:28.088 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53807.log 2024-04-17T10:55:28.089 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.84938.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.84938.log.gz 2024-04-17T10:55:28.089 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51308.log 2024-04-17T10:55:28.090 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53807.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53807.log.gz 2024-04-17T10:55:28.090 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69683.log 2024-04-17T10:55:28.091 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51308.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51308.log.gz 2024-04-17T10:55:28.092 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79709.log 2024-04-17T10:55:28.092 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69683.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69683.log.gz 2024-04-17T10:55:28.093 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79683.log 2024-04-17T10:55:28.094 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.79709.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.79709.log.gz 2024-04-17T10:55:28.094 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54806.log 2024-04-17T10:55:28.095 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.79683.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.79683.log.gz 2024-04-17T10:55:28.095 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82737.log 2024-04-17T10:55:28.096 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54806.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54806.log.gz 2024-04-17T10:55:28.097 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34078.log 2024-04-17T10:55:28.098 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.82737.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.82737.log.gz 2024-04-17T10:55:28.098 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75517.log 2024-04-17T10:55:28.099 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34078.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34078.log.gz 2024-04-17T10:55:28.099 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36073.log 2024-04-17T10:55:28.100 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75517.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.75517.log.gz 2024-04-17T10:55:28.100 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35514.log 2024-04-17T10:55:28.101 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36073.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36073.log.gz 2024-04-17T10:55:28.102 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35332.log 2024-04-17T10:55:28.102 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35514.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35514.log.gz 2024-04-17T10:55:28.103 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86537.log 2024-04-17T10:55:28.104 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35332.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35332.log.gz 2024-04-17T10:55:28.104 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.34847.log 2024-04-17T10:55:28.105 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86537.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.86537.log.gz 2024-04-17T10:55:28.105 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73422.log 2024-04-17T10:55:28.107 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.34847.log: 90.9% -- replaced with /var/log/ceph/ceph-client.0.34847.log.gz 2024-04-17T10:55:28.107 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-17T10:55:28.108 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.73422.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.73422.log.gz 2024-04-17T10:55:28.109 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76083.log 2024-04-17T10:55:28.112 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph.log: 87.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-17T10:55:28.113 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72598.log 2024-04-17T10:55:28.113 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.76083.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76083.log.gz 2024-04-17T10:55:28.114 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75822.log 2024-04-17T10:55:28.115 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72598.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72598.log.gz 2024-04-17T10:55:28.115 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83116.log 2024-04-17T10:55:28.116 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75822.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75822.log.gz 2024-04-17T10:55:28.116 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69207.log 2024-04-17T10:55:28.117 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.83116.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.83116.log.gz 2024-04-17T10:55:28.117 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87582.log 2024-04-17T10:55:28.118 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69207.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.69207.log.gz 2024-04-17T10:55:28.119 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72045.log 2024-04-17T10:55:28.120 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87582.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87582.log.gz 2024-04-17T10:55:28.120 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73080.log 2024-04-17T10:55:28.121 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72045.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72045.log.gz 2024-04-17T10:55:28.122 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76568.log 2024-04-17T10:55:28.122 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.73080.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.73080.log.gz 2024-04-17T10:55:28.123 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53982.log 2024-04-17T10:55:28.123 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.76568.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.76568.log.gz 2024-04-17T10:55:28.124 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86091.log 2024-04-17T10:55:28.125 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53982.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53982.log.gz 2024-04-17T10:55:28.125 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54507.log 2024-04-17T10:55:28.126 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86091.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.86091.log.gz 2024-04-17T10:55:28.126 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85278.log 2024-04-17T10:55:28.127 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54507.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54507.log.gz 2024-04-17T10:55:28.128 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34383.log 2024-04-17T10:55:28.128 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85278.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.85278.log.gz 2024-04-17T10:55:28.129 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87322.log 2024-04-17T10:55:28.130 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34383.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34383.log.gz 2024-04-17T10:55:28.130 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33785.log 2024-04-17T10:55:28.131 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87322.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.87322.log.gz 2024-04-17T10:55:28.132 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33260.log 2024-04-17T10:55:28.132 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33785.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33785.log.gz 2024-04-17T10:55:28.133 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73290.log 2024-04-17T10:55:28.133 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33260.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33260.log.gz 2024-04-17T10:55:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86353.log 2024-04-17T10:55:28.135 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.73290.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.73290.log.gz 2024-04-17T10:55:28.135 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50465.log 2024-04-17T10:55:28.136 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86353.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.86353.log.gz 2024-04-17T10:55:28.137 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53644.log 2024-04-17T10:55:28.137 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50465.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50465.log.gz 2024-04-17T10:55:28.138 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75744.log 2024-04-17T10:55:28.138 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53644.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53644.log.gz 2024-04-17T10:55:28.139 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72019.log 2024-04-17T10:55:28.140 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75744.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75744.log.gz 2024-04-17T10:55:28.140 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75569.log 2024-04-17T10:55:28.141 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72019.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.72019.log.gz 2024-04-17T10:55:28.142 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54675.log 2024-04-17T10:55:28.142 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75569.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75569.log.gz 2024-04-17T10:55:28.143 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35038.log 2024-04-17T10:55:28.143 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54675.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54675.log.gz 2024-04-17T10:55:28.144 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71941.log 2024-04-17T10:55:28.145 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35038.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35038.log.gz 2024-04-17T10:55:28.145 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75379.log 2024-04-17T10:55:28.146 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.71941.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71941.log.gz 2024-04-17T10:55:28.146 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53339.log 2024-04-17T10:55:28.147 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75379.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.75379.log.gz 2024-04-17T10:55:28.148 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86222.log 2024-04-17T10:55:28.148 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53339.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53339.log.gz 2024-04-17T10:55:28.149 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85148.log 2024-04-17T10:55:28.150 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86222.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.86222.log.gz 2024-04-17T10:55:28.150 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82195.log 2024-04-17T10:55:28.151 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85148.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.85148.log.gz 2024-04-17T10:55:28.152 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77952.log 2024-04-17T10:55:28.152 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.82195.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.82195.log.gz 2024-04-17T10:55:28.153 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81987.log 2024-04-17T10:55:28.154 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.77952.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.77952.log.gz 2024-04-17T10:55:28.154 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55505.log 2024-04-17T10:55:28.155 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.81987.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.81987.log.gz 2024-04-17T10:55:28.156 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53930.log 2024-04-17T10:55:28.156 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55505.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.55505.log.gz 2024-04-17T10:55:28.157 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81790.log 2024-04-17T10:55:28.157 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53930.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53930.log.gz 2024-04-17T10:55:28.158 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85226.log 2024-04-17T10:55:28.159 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.81790.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.81790.log.gz 2024-04-17T10:55:28.159 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35410.log 2024-04-17T10:55:28.160 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85226.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85226.log.gz 2024-04-17T10:55:28.160 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79866.log 2024-04-17T10:55:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35410.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35410.log.gz 2024-04-17T10:55:28.162 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75353.log 2024-04-17T10:55:28.162 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.79866.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.79866.log.gz 2024-04-17T10:55:28.163 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53696.log 2024-04-17T10:55:28.163 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75353.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75353.log.gz 2024-04-17T10:55:28.164 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76057.log 2024-04-17T10:55:28.165 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53696.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53696.log.gz 2024-04-17T10:55:28.165 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78891.log 2024-04-17T10:55:28.166 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.76057.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76057.log.gz 2024-04-17T10:55:28.166 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54405.log 2024-04-17T10:55:28.167 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.78891.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78891.log.gz 2024-04-17T10:55:28.168 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79631.log 2024-04-17T10:55:28.168 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54405.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54405.log.gz 2024-04-17T10:55:28.169 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37044.log 2024-04-17T10:55:28.170 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.79631.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.79631.log.gz 2024-04-17T10:55:28.170 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35066.log 2024-04-17T10:55:28.171 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37044.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37044.log.gz 2024-04-17T10:55:28.171 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74932.log 2024-04-17T10:55:28.172 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35066.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35066.log.gz 2024-04-17T10:55:28.173 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36021.log 2024-04-17T10:55:28.174 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.74932.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.74932.log.gz 2024-04-17T10:55:28.174 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70131.log 2024-04-17T10:55:28.175 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36021.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36021.log.gz 2024-04-17T10:55:28.175 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50744.log 2024-04-17T10:55:28.176 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70131.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.70131.log.gz 2024-04-17T10:55:28.176 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51056.log 2024-04-17T10:55:28.177 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50744.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50744.log.gz 2024-04-17T10:55:28.178 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72371.log 2024-04-17T10:55:28.178 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51056.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51056.log.gz 2024-04-17T10:55:28.179 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53670.log 2024-04-17T10:55:28.179 INFO:teuthology.orchestra.run.smithi083.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.0.73527.log.gz 2024-04-17T10:55:28.180 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72371.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72371.log.gz 2024-04-17T10:55:28.180 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82683.log 2024-04-17T10:55:28.181 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53670.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53670.log.gz 2024-04-17T10:55:28.181 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50770.log 2024-04-17T10:55:28.182 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.82683.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.82683.log.gz 2024-04-17T10:55:28.182 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81252.log 2024-04-17T10:55:28.183 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50770.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50770.log.gz 2024-04-17T10:55:28.184 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76218.log 2024-04-17T10:55:28.185 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.81252.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.81252.log.gz 2024-04-17T10:55:28.185 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85647.log 2024-04-17T10:55:28.186 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.76218.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.76218.log.gz 2024-04-17T10:55:28.186 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86039.log 2024-04-17T10:55:28.187 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85647.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.85647.log.gz 2024-04-17T10:55:28.188 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74754.log 2024-04-17T10:55:28.189 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86039.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.86039.log.gz 2024-04-17T10:55:28.189 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33391.log 2024-04-17T10:55:28.190 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.74754.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.74754.log.gz 2024-04-17T10:55:28.191 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84652.log 2024-04-17T10:55:28.191 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33391.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33391.log.gz 2024-04-17T10:55:28.192 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87033.log 2024-04-17T10:55:28.193 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.84652.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.84652.log.gz 2024-04-17T10:55:28.193 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81961.log 2024-04-17T10:55:28.194 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87033.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.87033.log.gz 2024-04-17T10:55:28.194 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27626.log 2024-04-17T10:55:28.195 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.81961.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.81961.log.gz 2024-04-17T10:55:28.196 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50439.log 2024-04-17T10:55:28.197 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.27626.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.27626.log.gz 2024-04-17T10:55:28.197 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70274.log 2024-04-17T10:55:28.198 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50439.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50439.log.gz 2024-04-17T10:55:28.198 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56235.log 2024-04-17T10:55:28.199 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70274.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70274.log.gz 2024-04-17T10:55:28.200 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81352.log 2024-04-17T10:55:28.201 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56235.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56235.log.gz 2024-04-17T10:55:28.201 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75188.log 2024-04-17T10:55:28.202 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.81352.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.81352.log.gz 2024-04-17T10:55:28.202 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35358.log 2024-04-17T10:55:28.203 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75188.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75188.log.gz 2024-04-17T10:55:28.204 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85881.log 2024-04-17T10:55:28.205 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35358.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35358.log.gz 2024-04-17T10:55:28.205 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51228.log 2024-04-17T10:55:28.206 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85881.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.85881.log.gz 2024-04-17T10:55:28.206 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87634.log 2024-04-17T10:55:28.207 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51228.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51228.log.gz 2024-04-17T10:55:28.208 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52282.log 2024-04-17T10:55:28.208 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87634.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.87634.log.gz 2024-04-17T10:55:28.209 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50647.log 2024-04-17T10:55:28.210 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52282.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.52282.log.gz 2024-04-17T10:55:28.210 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36440.log 2024-04-17T10:55:28.211 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50647.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50647.log.gz 2024-04-17T10:55:28.212 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86823.log 2024-04-17T10:55:28.213 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36440.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36440.log.gz 2024-04-17T10:55:28.213 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84782.log 2024-04-17T10:55:28.213 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86823.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.86823.log.gz 2024-04-17T10:55:28.214 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37486.log 2024-04-17T10:55:28.215 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.84782.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.84782.log.gz 2024-04-17T10:55:28.215 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56052.log 2024-04-17T10:55:28.216 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37486.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37486.log.gz 2024-04-17T10:55:28.216 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82039.log 2024-04-17T10:55:28.217 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56052.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56052.log.gz 2024-04-17T10:55:28.218 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78943.log 2024-04-17T10:55:28.218 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.82039.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.82039.log.gz 2024-04-17T10:55:28.219 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79355.log 2024-04-17T10:55:28.220 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.78943.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.78943.log.gz 2024-04-17T10:55:28.220 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85304.log 2024-04-17T10:55:28.221 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.79355.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.79355.log.gz 2024-04-17T10:55:28.221 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85907.log 2024-04-17T10:55:28.222 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85304.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.85304.log.gz 2024-04-17T10:55:28.223 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88606.log 2024-04-17T10:55:28.223 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85907.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.85907.log.gz 2024-04-17T10:55:28.224 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72077.log 2024-04-17T10:55:28.225 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88606.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.88606.log.gz 2024-04-17T10:55:28.225 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55427.log 2024-04-17T10:55:28.226 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72077.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.72077.log.gz 2024-04-17T10:55:28.227 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86379.log 2024-04-17T10:55:28.227 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55427.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55427.log.gz 2024-04-17T10:55:28.233 INFO:teuthology.orchestra.run.smithi083.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.0.76725.log.gz 2024-04-17T10:55:28.233 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86407.log 2024-04-17T10:55:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86379.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.86379.log.gz 2024-04-17T10:55:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52674.log 2024-04-17T10:55:28.235 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86407.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.86407.log.gz 2024-04-17T10:55:28.236 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79488.log 2024-04-17T10:55:28.236 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52674.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52674.log.gz 2024-04-17T10:55:28.237 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77902.log 2024-04-17T10:55:28.237 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.79488.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.79488.log.gz 2024-04-17T10:55:28.238 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51861.log 2024-04-17T10:55:28.238 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.77902.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.77902.log.gz 2024-04-17T10:55:28.239 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56118.log 2024-04-17T10:55:28.240 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51861.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.51861.log.gz 2024-04-17T10:55:28.241 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87660.log 2024-04-17T10:55:28.241 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56118.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56118.log.gz 2024-04-17T10:55:28.241 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34897.log 2024-04-17T10:55:28.242 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87660.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.87660.log.gz 2024-04-17T10:55:28.242 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56632.log 2024-04-17T10:55:28.243 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34897.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34897.log.gz 2024-04-17T10:55:28.243 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52707.log 2024-04-17T10:55:28.244 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56632.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56632.log.gz 2024-04-17T10:55:28.244 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74906.log 2024-04-17T10:55:28.245 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52707.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52707.log.gz 2024-04-17T10:55:28.245 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85383.log 2024-04-17T10:55:28.246 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.74906.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.74906.log.gz 2024-04-17T10:55:28.246 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83010.log 2024-04-17T10:55:28.247 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85383.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.85383.log.gz 2024-04-17T10:55:28.248 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86955.log 2024-04-17T10:55:28.248 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.83010.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.83010.log.gz 2024-04-17T10:55:28.249 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33365.log 2024-04-17T10:55:28.250 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86955.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.86955.log.gz 2024-04-17T10:55:28.250 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87530.log 2024-04-17T10:55:28.251 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33365.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33365.log.gz 2024-04-17T10:55:28.251 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69762.log 2024-04-17T10:55:28.252 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87530.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.87530.log.gz 2024-04-17T10:55:28.252 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76109.log 2024-04-17T10:55:28.253 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69762.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69762.log.gz 2024-04-17T10:55:28.253 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86771.log 2024-04-17T10:55:28.254 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.76109.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76109.log.gz 2024-04-17T10:55:28.254 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75010.log 2024-04-17T10:55:28.255 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86771.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.86771.log.gz 2024-04-17T10:55:28.255 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78052.log 2024-04-17T10:55:28.256 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75010.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.75010.log.gz 2024-04-17T10:55:28.257 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82657.log 2024-04-17T10:55:28.258 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.78052.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.78052.log.gz 2024-04-17T10:55:28.258 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36992.log 2024-04-17T10:55:28.259 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.82657.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.82657.log.gz 2024-04-17T10:55:28.259 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35834.log 2024-04-17T10:55:28.260 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36992.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36992.log.gz 2024-04-17T10:55:28.260 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88223.log 2024-04-17T10:55:28.261 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35834.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35834.log.gz 2024-04-17T10:55:28.262 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82422.log 2024-04-17T10:55:28.263 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88223.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.88223.log.gz 2024-04-17T10:55:28.263 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85122.log 2024-04-17T10:55:28.263 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.82422.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.82422.log.gz 2024-04-17T10:55:28.264 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27862.log 2024-04-17T10:55:28.264 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85122.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.85122.log.gz 2024-04-17T10:55:28.265 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33707.log 2024-04-17T10:55:28.265 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.27862.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.27862.log.gz 2024-04-17T10:55:28.266 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51135.log 2024-04-17T10:55:28.267 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33707.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33707.log.gz 2024-04-17T10:55:28.267 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72103.log 2024-04-17T10:55:28.268 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51135.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51135.log.gz 2024-04-17T10:55:28.268 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51993.log 2024-04-17T10:55:28.269 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72103.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72103.log.gz 2024-04-17T10:55:28.269 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52232.log 2024-04-17T10:55:28.270 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51993.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51993.log.gz 2024-04-17T10:55:28.270 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37521.log 2024-04-17T10:55:28.271 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52232.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.52232.log.gz 2024-04-17T10:55:28.272 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69736.log 2024-04-17T10:55:28.273 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37521.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37521.log.gz 2024-04-17T10:55:28.273 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85673.log 2024-04-17T10:55:28.274 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69736.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69736.log.gz 2024-04-17T10:55:28.274 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86615.log 2024-04-17T10:55:28.275 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85673.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.85673.log.gz 2024-04-17T10:55:28.275 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52472.log 2024-04-17T10:55:28.276 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86615.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.86615.log.gz 2024-04-17T10:55:28.276 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56288.log 2024-04-17T10:55:28.277 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52472.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.52472.log.gz 2024-04-17T10:55:28.278 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33655.log 2024-04-17T10:55:28.278 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56288.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56288.log.gz 2024-04-17T10:55:28.279 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87242.log 2024-04-17T10:55:28.279 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33655.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33655.log.gz 2024-04-17T10:55:28.280 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72267.log 2024-04-17T10:55:28.280 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87242.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.87242.log.gz 2024-04-17T10:55:28.281 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85096.log 2024-04-17T10:55:28.281 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72267.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72267.log.gz 2024-04-17T10:55:28.282 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88736.log 2024-04-17T10:55:28.283 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85096.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.85096.log.gz 2024-04-17T10:55:28.283 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78623.log 2024-04-17T10:55:28.284 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88736.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.88736.log.gz 2024-04-17T10:55:28.284 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82117.log 2024-04-17T10:55:28.285 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.78623.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.78623.log.gz 2024-04-17T10:55:28.285 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81875.log 2024-04-17T10:55:28.285 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.82117.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.82117.log.gz 2024-04-17T10:55:28.286 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52442.log 2024-04-17T10:55:28.286 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.81875.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.81875.log.gz 2024-04-17T10:55:28.287 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52358.log 2024-04-17T10:55:28.288 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52442.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52442.log.gz 2024-04-17T10:55:28.288 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88294.log 2024-04-17T10:55:28.289 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52358.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.52358.log.gz 2024-04-17T10:55:28.290 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81708.log 2024-04-17T10:55:28.290 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88294.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.88294.log.gz 2024-04-17T10:55:28.291 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82932.log 2024-04-17T10:55:28.291 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.81708.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.81708.log.gz 2024-04-17T10:55:28.292 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49989.log 2024-04-17T10:55:28.292 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.82932.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.82932.log.gz 2024-04-17T10:55:28.293 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75110.log 2024-04-17T10:55:28.294 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49989.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.49989.log.gz 2024-04-17T10:55:28.295 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74854.log 2024-04-17T10:55:28.295 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75110.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75110.log.gz 2024-04-17T10:55:28.296 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87859.log 2024-04-17T10:55:28.297 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.74854.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.74854.log.gz 2024-04-17T10:55:28.297 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-17T10:55:28.298 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87859.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.87859.log.gz 2024-04-17T10:55:28.298 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79120.log 2024-04-17T10:55:28.302 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph.audit.log: 89.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-17T10:55:28.303 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33889.log 2024-04-17T10:55:28.304 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.79120.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.79120.log.gz 2024-04-17T10:55:28.304 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35677.log 2024-04-17T10:55:28.305 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33889.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33889.log.gz 2024-04-17T10:55:28.305 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-17T10:55:28.306 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35677.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35677.log.gz 2024-04-17T10:55:28.306 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69423.log 2024-04-17T10:55:28.319 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50595.log 2024-04-17T10:55:28.320 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69423.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69423.log.gz 2024-04-17T10:55:28.335 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52811.log 2024-04-17T10:55:28.336 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50595.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50595.log.gz 2024-04-17T10:55:28.359 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54143.log 2024-04-17T10:55:28.360 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52811.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.52811.log.gz 2024-04-17T10:55:28.371 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36205.log 2024-04-17T10:55:28.372 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54143.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54143.log.gz 2024-04-17T10:55:28.379 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84546.log 2024-04-17T10:55:28.380 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36205.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36205.log.gz 2024-04-17T10:55:28.387 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34093.log 2024-04-17T10:55:28.388 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.84546.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.84546.log.gz 2024-04-17T10:55:28.395 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37221.log 2024-04-17T10:55:28.396 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34093.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34093.log.gz 2024-04-17T10:55:28.403 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78104.log 2024-04-17T10:55:28.404 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37221.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37221.log.gz 2024-04-17T10:55:28.411 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69553.log 2024-04-17T10:55:28.412 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.78104.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.78104.log.gz 2024-04-17T10:55:28.419 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-17T10:55:28.420 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69553.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.69553.log.gz 2024-04-17T10:55:28.427 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56710.log 2024-04-17T10:55:28.435 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88450.log 2024-04-17T10:55:28.436 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56710.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56710.log.gz 2024-04-17T10:55:28.443 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73054.log 2024-04-17T10:55:28.444 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88450.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.88450.log.gz 2024-04-17T10:55:28.451 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69259.log 2024-04-17T10:55:28.452 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.73054.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73054.log.gz 2024-04-17T10:55:28.459 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53123.log 2024-04-17T10:55:28.460 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69259.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.69259.log.gz 2024-04-17T10:55:28.467 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54727.log 2024-04-17T10:55:28.468 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53123.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53123.log.gz 2024-04-17T10:55:28.475 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50517.log 2024-04-17T10:55:28.476 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54727.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54727.log.gz 2024-04-17T10:55:28.483 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22773.log 2024-04-17T10:55:28.484 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50517.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50517.log.gz 2024-04-17T10:55:28.491 INFO:teuthology.orchestra.run.smithi083.stderr: 90.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85829.log 2024-04-17T10:55:28.491 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph.tmp-client.admin.22773.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22773.log.gz 2024-04-17T10:55:28.491 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-17T10:55:28.499 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70022.log 2024-04-17T10:55:28.500 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85829.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.85829.log.gz 2024-04-17T10:55:28.501 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53469.log 2024-04-17T10:55:28.501 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70022.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70022.log.gz 2024-04-17T10:55:28.502 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76516.log 2024-04-17T10:55:28.503 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53469.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53469.log.gz 2024-04-17T10:55:28.503 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78206.log 2024-04-17T10:55:28.504 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.76516.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76516.log.gz 2024-04-17T10:55:28.504 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53287.log 2024-04-17T10:55:28.505 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.78206.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.78206.log.gz 2024-04-17T10:55:28.506 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50569.log 2024-04-17T10:55:28.507 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53287.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53287.log.gz 2024-04-17T10:55:28.507 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79840.log 2024-04-17T10:55:28.508 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50569.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50569.log.gz 2024-04-17T10:55:28.508 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88067.log 2024-04-17T10:55:28.509 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.79840.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.79840.log.gz 2024-04-17T10:55:28.510 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56658.log 2024-04-17T10:55:28.511 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88067.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.88067.log.gz 2024-04-17T10:55:28.511 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33890.log 2024-04-17T10:55:28.512 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56658.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56658.log.gz 2024-04-17T10:55:28.512 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69397.log 2024-04-17T10:55:28.513 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33890.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33890.log.gz 2024-04-17T10:55:28.514 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35995.log 2024-04-17T10:55:28.514 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69397.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.69397.log.gz 2024-04-17T10:55:28.515 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76412.log 2024-04-17T10:55:28.516 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35995.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35995.log.gz 2024-04-17T10:55:28.516 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72761.log 2024-04-17T10:55:28.517 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.76412.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76412.log.gz 2024-04-17T10:55:28.517 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33887.log 2024-04-17T10:55:28.518 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72761.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.72761.log.gz 2024-04-17T10:55:28.519 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82013.log 2024-04-17T10:55:28.520 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33887.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33887.log.gz 2024-04-17T10:55:28.520 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72345.log 2024-04-17T10:55:28.521 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.82013.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.82013.log.gz 2024-04-17T10:55:28.521 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78513.log 2024-04-17T10:55:28.522 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72345.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.72345.log.gz 2024-04-17T10:55:28.523 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51386.log 2024-04-17T10:55:28.523 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.78513.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78513.log.gz 2024-04-17T10:55:28.524 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85070.log 2024-04-17T10:55:28.525 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51386.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51386.log.gz 2024-04-17T10:55:28.525 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51030.log 2024-04-17T10:55:28.526 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85070.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.85070.log.gz 2024-04-17T10:55:28.526 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50143.log 2024-04-17T10:55:28.527 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51030.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51030.log.gz 2024-04-17T10:55:28.528 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56448.log 2024-04-17T10:55:28.529 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50143.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.50143.log.gz 2024-04-17T10:55:28.530 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34724.log 2024-04-17T10:55:28.531 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56448.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56448.log.gz 2024-04-17T10:55:28.531 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82906.log 2024-04-17T10:55:28.532 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34724.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34724.log.gz 2024-04-17T10:55:28.532 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75465.log 2024-04-17T10:55:28.533 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.82906.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.82906.log.gz 2024-04-17T10:55:28.533 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51109.log 2024-04-17T10:55:28.534 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75465.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75465.log.gz 2024-04-17T10:55:28.534 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35808.log 2024-04-17T10:55:28.535 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51109.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51109.log.gz 2024-04-17T10:55:28.536 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86117.log 2024-04-17T10:55:28.537 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35808.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35808.log.gz 2024-04-17T10:55:28.537 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52026.log 2024-04-17T10:55:28.538 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86117.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.86117.log.gz 2024-04-17T10:55:28.538 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73134.log 2024-04-17T10:55:28.539 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52026.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52026.log.gz 2024-04-17T10:55:28.540 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88502.log 2024-04-17T10:55:28.540 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.73134.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73134.log.gz 2024-04-17T10:55:28.541 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81738.log 2024-04-17T10:55:28.542 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88502.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.88502.log.gz 2024-04-17T10:55:28.542 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53313.log 2024-04-17T10:55:28.543 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.81738.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.81738.log.gz 2024-04-17T10:55:28.543 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69501.log 2024-04-17T10:55:28.544 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53313.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53313.log.gz 2024-04-17T10:55:28.545 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34669.log 2024-04-17T10:55:28.546 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69501.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69501.log.gz 2024-04-17T10:55:28.546 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84626.log 2024-04-17T10:55:28.547 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34669.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.34669.log.gz 2024-04-17T10:55:28.548 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69840.log 2024-04-17T10:55:28.549 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.84626.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.84626.log.gz 2024-04-17T10:55:28.549 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33499.log 2024-04-17T10:55:28.550 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69840.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.69840.log.gz 2024-04-17T10:55:28.551 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85517.log 2024-04-17T10:55:28.551 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33499.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33499.log.gz 2024-04-17T10:55:28.552 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35124.log 2024-04-17T10:55:28.553 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85517.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.85517.log.gz 2024-04-17T10:55:28.553 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53748.log 2024-04-17T10:55:28.554 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35124.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35124.log.gz 2024-04-17T10:55:28.554 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50249.log 2024-04-17T10:55:28.555 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53748.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53748.log.gz 2024-04-17T10:55:28.556 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36550.log 2024-04-17T10:55:28.556 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50249.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50249.log.gz 2024-04-17T10:55:28.557 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51163.log 2024-04-17T10:55:28.558 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36550.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36550.log.gz 2024-04-17T10:55:28.558 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85595.log 2024-04-17T10:55:28.559 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51163.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51163.log.gz 2024-04-17T10:55:28.560 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36888.log 2024-04-17T10:55:28.560 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85595.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.85595.log.gz 2024-04-17T10:55:28.561 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37573.log 2024-04-17T10:55:28.562 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36888.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36888.log.gz 2024-04-17T10:55:28.562 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85621.log 2024-04-17T10:55:28.563 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37573.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37573.log.gz 2024-04-17T10:55:28.563 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69873.log 2024-04-17T10:55:28.564 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85621.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.85621.log.gz 2024-04-17T10:55:28.565 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78130.log 2024-04-17T10:55:28.565 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69873.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.69873.log.gz 2024-04-17T10:55:28.566 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56476.log 2024-04-17T10:55:28.567 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.78130.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.78130.log.gz 2024-04-17T10:55:28.567 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84912.log 2024-04-17T10:55:28.568 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56476.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56476.log.gz 2024-04-17T10:55:28.568 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87268.log 2024-04-17T10:55:28.569 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.84912.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.84912.log.gz 2024-04-17T10:55:28.570 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88093.log 2024-04-17T10:55:28.570 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87268.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.87268.log.gz 2024-04-17T10:55:28.571 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86065.log 2024-04-17T10:55:28.572 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88093.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.88093.log.gz 2024-04-17T10:55:28.572 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50491.log 2024-04-17T10:55:28.573 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86065.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.86065.log.gz 2024-04-17T10:55:28.574 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35969.log 2024-04-17T10:55:28.574 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50491.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50491.log.gz 2024-04-17T10:55:28.575 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84730.log 2024-04-17T10:55:28.576 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35969.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35969.log.gz 2024-04-17T10:55:28.576 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37783.log 2024-04-17T10:55:28.577 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.84730.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.84730.log.gz 2024-04-17T10:55:28.577 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71713.log 2024-04-17T10:55:28.579 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37783.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.37783.log.gz 2024-04-17T10:55:28.579 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82221.log 2024-04-17T10:55:28.580 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.71713.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.71713.log.gz 2024-04-17T10:55:28.581 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79516.log 2024-04-17T10:55:28.582 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.82221.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.82221.log.gz 2024-04-17T10:55:28.582 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72572.log 2024-04-17T10:55:28.583 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.79516.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.79516.log.gz 2024-04-17T10:55:28.583 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73212.log 2024-04-17T10:55:28.584 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72572.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72572.log.gz 2024-04-17T10:55:28.585 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36680.log 2024-04-17T10:55:28.585 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.73212.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73212.log.gz 2024-04-17T10:55:28.586 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72991.log 2024-04-17T10:55:28.587 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36680.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36680.log.gz 2024-04-17T10:55:28.587 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55747.log 2024-04-17T10:55:28.588 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72991.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72991.log.gz 2024-04-17T10:55:28.589 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71967.log 2024-04-17T10:55:28.589 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55747.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55747.log.gz 2024-04-17T10:55:28.590 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72181.log 2024-04-17T10:55:28.591 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.71967.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71967.log.gz 2024-04-17T10:55:28.591 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75136.log 2024-04-17T10:55:28.592 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72181.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.72181.log.gz 2024-04-17T10:55:28.592 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33067.log 2024-04-17T10:55:28.593 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75136.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.75136.log.gz 2024-04-17T10:55:28.594 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78787.log 2024-04-17T10:55:28.594 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33067.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33067.log.gz 2024-04-17T10:55:28.595 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51941.log 2024-04-17T10:55:28.596 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.78787.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.78787.log.gz 2024-04-17T10:55:28.596 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52739.log 2024-04-17T10:55:28.598 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51941.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.51941.log.gz 2024-04-17T10:55:28.598 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55922.log 2024-04-17T10:55:28.599 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52739.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.52739.log.gz 2024-04-17T10:55:28.600 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78256.log 2024-04-17T10:55:28.600 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55922.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55922.log.gz 2024-04-17T10:55:28.601 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75491.log 2024-04-17T10:55:28.602 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.78256.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.78256.log.gz 2024-04-17T10:55:28.602 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56026.log 2024-04-17T10:55:28.603 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75491.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75491.log.gz 2024-04-17T10:55:28.604 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75900.log 2024-04-17T10:55:28.604 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56026.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56026.log.gz 2024-04-17T10:55:28.605 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75240.log 2024-04-17T10:55:28.606 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75900.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75900.log.gz 2024-04-17T10:55:28.606 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73396.log 2024-04-17T10:55:28.607 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75240.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75240.log.gz 2024-04-17T10:55:28.607 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85725.log 2024-04-17T10:55:28.608 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.73396.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.73396.log.gz 2024-04-17T10:55:28.609 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81764.log 2024-04-17T10:55:28.609 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85725.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.85725.log.gz 2024-04-17T10:55:28.610 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88145.log 2024-04-17T10:55:28.611 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.81764.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.81764.log.gz 2024-04-17T10:55:28.611 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36523.log 2024-04-17T10:55:28.612 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88145.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.88145.log.gz 2024-04-17T10:55:28.613 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35254.log 2024-04-17T10:55:28.613 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36523.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36523.log.gz 2024-04-17T10:55:28.614 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88528.log 2024-04-17T10:55:28.615 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35254.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35254.log.gz 2024-04-17T10:55:28.615 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35781.log 2024-04-17T10:55:28.616 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88528.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.88528.log.gz 2024-04-17T10:55:28.616 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35228.log 2024-04-17T10:55:28.617 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35781.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35781.log.gz 2024-04-17T10:55:28.618 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87296.log 2024-04-17T10:55:28.619 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35228.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35228.log.gz 2024-04-17T10:55:28.619 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70075.log 2024-04-17T10:55:28.620 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87296.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87296.log.gz 2024-04-17T10:55:28.620 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86563.log 2024-04-17T10:55:28.621 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70075.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.70075.log.gz 2024-04-17T10:55:28.622 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87478.log 2024-04-17T10:55:28.622 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86563.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.86563.log.gz 2024-04-17T10:55:28.623 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78995.log 2024-04-17T10:55:28.624 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87478.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.87478.log.gz 2024-04-17T10:55:28.624 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70352.log 2024-04-17T10:55:28.625 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.78995.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.78995.log.gz 2024-04-17T10:55:28.626 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36336.log 2024-04-17T10:55:28.626 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70352.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70352.log.gz 2024-04-17T10:55:28.627 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82396.log 2024-04-17T10:55:28.628 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36336.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36336.log.gz 2024-04-17T10:55:28.628 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70103.log 2024-04-17T10:55:28.629 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.82396.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.82396.log.gz 2024-04-17T10:55:28.629 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56422.log 2024-04-17T10:55:28.630 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70103.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.70103.log.gz 2024-04-17T10:55:28.631 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86013.log 2024-04-17T10:55:28.631 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56422.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56422.log.gz 2024-04-17T10:55:28.632 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75327.log 2024-04-17T10:55:28.633 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86013.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.86013.log.gz 2024-04-17T10:55:28.633 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53391.log 2024-04-17T10:55:28.634 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75327.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75327.log.gz 2024-04-17T10:55:28.634 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86274.log 2024-04-17T10:55:28.635 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53391.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53391.log.gz 2024-04-17T10:55:28.636 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78487.log 2024-04-17T10:55:28.637 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86274.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.86274.log.gz 2024-04-17T10:55:28.637 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50933.log 2024-04-17T10:55:28.638 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.78487.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.78487.log.gz 2024-04-17T10:55:28.638 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88424.log 2024-04-17T10:55:28.639 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50933.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50933.log.gz 2024-04-17T10:55:28.640 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75595.log 2024-04-17T10:55:28.641 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88424.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.88424.log.gz 2024-04-17T10:55:28.641 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82091.log 2024-04-17T10:55:28.642 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75595.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.75595.log.gz 2024-04-17T10:55:28.642 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79813.log 2024-04-17T10:55:28.643 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.82091.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.82091.log.gz 2024-04-17T10:55:28.643 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69101.log 2024-04-17T10:55:28.644 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.79813.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79813.log.gz 2024-04-17T10:55:28.645 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87190.log 2024-04-17T10:55:28.646 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69101.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.69101.log.gz 2024-04-17T10:55:28.647 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78675.log 2024-04-17T10:55:28.647 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87190.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.87190.log.gz 2024-04-17T10:55:28.648 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33234.log 2024-04-17T10:55:28.649 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.78675.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.78675.log.gz 2024-04-17T10:55:28.649 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82169.log 2024-04-17T10:55:28.650 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33234.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33234.log.gz 2024-04-17T10:55:28.650 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36836.log 2024-04-17T10:55:28.651 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.82169.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.82169.log.gz 2024-04-17T10:55:28.652 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55153.log 2024-04-17T10:55:28.653 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36836.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36836.log.gz 2024-04-17T10:55:28.653 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55870.log 2024-04-17T10:55:28.654 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55153.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55153.log.gz 2024-04-17T10:55:28.654 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87504.log 2024-04-17T10:55:28.655 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55870.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55870.log.gz 2024-04-17T10:55:28.656 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33891.log 2024-04-17T10:55:28.657 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87504.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.87504.log.gz 2024-04-17T10:55:28.657 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88814.log 2024-04-17T10:55:28.658 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33891.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33891.log.gz 2024-04-17T10:55:28.658 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79290.log 2024-04-17T10:55:28.659 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88814.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.88814.log.gz 2024-04-17T10:55:28.660 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51360.log 2024-04-17T10:55:28.660 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.79290.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.79290.log.gz 2024-04-17T10:55:28.661 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36706.log 2024-04-17T10:55:28.662 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51360.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51360.log.gz 2024-04-17T10:55:28.662 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56554.log 2024-04-17T10:55:28.663 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36706.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36706.log.gz 2024-04-17T10:55:28.664 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51781.log 2024-04-17T10:55:28.664 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56554.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56554.log.gz 2024-04-17T10:55:28.665 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50301.log 2024-04-17T10:55:28.666 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51781.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.51781.log.gz 2024-04-17T10:55:28.667 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35703.log 2024-04-17T10:55:28.668 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50301.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50301.log.gz 2024-04-17T10:55:28.668 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86667.log 2024-04-17T10:55:28.669 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35703.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35703.log.gz 2024-04-17T10:55:28.669 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54223.log 2024-04-17T10:55:28.670 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86667.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.86667.log.gz 2024-04-17T10:55:28.670 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37627.log 2024-04-17T10:55:28.671 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54223.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54223.log.gz 2024-04-17T10:55:28.672 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36047.log 2024-04-17T10:55:28.673 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37627.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37627.log.gz 2024-04-17T10:55:28.673 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54949.log 2024-04-17T10:55:28.674 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36047.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36047.log.gz 2024-04-17T10:55:28.674 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85044.log 2024-04-17T10:55:28.675 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54949.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54949.log.gz 2024-04-17T10:55:28.676 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36914.log 2024-04-17T10:55:28.676 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85044.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85044.log.gz 2024-04-17T10:55:28.677 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71913.log 2024-04-17T10:55:28.678 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36914.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36914.log.gz 2024-04-17T10:55:28.678 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56000.log 2024-04-17T10:55:28.679 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.71913.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71913.log.gz 2024-04-17T10:55:28.680 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76136.log 2024-04-17T10:55:28.680 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56000.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56000.log.gz 2024-04-17T10:55:28.681 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37378.log 2024-04-17T10:55:28.682 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.76136.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.76136.log.gz 2024-04-17T10:55:28.682 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73106.log 2024-04-17T10:55:28.683 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37378.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37378.log.gz 2024-04-17T10:55:28.683 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50543.log 2024-04-17T10:55:28.684 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.73106.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.73106.log.gz 2024-04-17T10:55:28.685 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72676.log 2024-04-17T10:55:28.685 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50543.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50543.log.gz 2024-04-17T10:55:28.686 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51082.log 2024-04-17T10:55:28.687 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72676.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72676.log.gz 2024-04-17T10:55:28.687 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87825.log 2024-04-17T10:55:28.688 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51082.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51082.log.gz 2024-04-17T10:55:28.689 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34617.log 2024-04-17T10:55:28.689 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87825.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.87825.log.gz 2024-04-17T10:55:28.690 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34279.log 2024-04-17T10:55:28.691 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34617.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34617.log.gz 2024-04-17T10:55:28.691 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72501.log 2024-04-17T10:55:28.692 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34279.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34279.log.gz 2024-04-17T10:55:28.692 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50874.log 2024-04-17T10:55:28.693 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72501.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72501.log.gz 2024-04-17T10:55:28.693 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52058.log 2024-04-17T10:55:28.694 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50874.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50874.log.gz 2024-04-17T10:55:28.695 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87556.log 2024-04-17T10:55:28.696 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52058.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.52058.log.gz 2024-04-17T10:55:28.697 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72859.log 2024-04-17T10:55:28.697 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87556.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.87556.log.gz 2024-04-17T10:55:28.698 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86877.log 2024-04-17T10:55:28.699 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72859.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72859.log.gz 2024-04-17T10:55:28.699 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55099.log 2024-04-17T10:55:28.700 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86877.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.86877.log.gz 2024-04-17T10:55:28.700 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54327.log 2024-04-17T10:55:28.701 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55099.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55099.log.gz 2024-04-17T10:55:28.702 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87374.log 2024-04-17T10:55:28.703 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54327.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.54327.log.gz 2024-04-17T10:55:28.703 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81302.log 2024-04-17T10:55:28.704 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87374.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.87374.log.gz 2024-04-17T10:55:28.704 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78709.log 2024-04-17T10:55:28.705 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.81302.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.81302.log.gz 2024-04-17T10:55:28.706 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78761.log 2024-04-17T10:55:28.707 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.78709.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.78709.log.gz 2024-04-17T10:55:28.707 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37888.log 2024-04-17T10:55:28.708 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.78761.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.78761.log.gz 2024-04-17T10:55:28.709 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81558.log 2024-04-17T10:55:28.719 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.37888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86797.log 2024-04-17T10:55:28.720 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.81558.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.81558.log.gz 2024-04-17T10:55:28.739 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33759.log 2024-04-17T10:55:28.740 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86797.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.86797.log.gz 2024-04-17T10:55:28.747 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55565.log 2024-04-17T10:55:28.748 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33759.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33759.log.gz 2024-04-17T10:55:28.755 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82344.log 2024-04-17T10:55:28.756 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55565.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.55565.log.gz 2024-04-17T10:55:28.763 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52522.log 2024-04-17T10:55:28.764 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.82344.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.82344.log.gz 2024-04-17T10:55:28.771 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36810.log 2024-04-17T10:55:28.772 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52522.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.52522.log.gz 2024-04-17T10:55:28.787 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53495.log 2024-04-17T10:55:28.788 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36810.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36810.log.gz 2024-04-17T10:55:28.798 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69371.log 2024-04-17T10:55:28.798 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53495.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.53495.log.gz 2024-04-17T10:55:28.803 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81823.log 2024-04-17T10:55:28.804 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69371.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69371.log.gz 2024-04-17T10:55:28.811 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72702.log 2024-04-17T10:55:28.812 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.81823.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.81823.log.gz 2024-04-17T10:55:28.831 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79918.log 2024-04-17T10:55:28.832 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72702.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72702.log.gz 2024-04-17T10:55:28.851 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76305.log 2024-04-17T10:55:28.852 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.79918.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.79918.log.gz 2024-04-17T10:55:28.859 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75992.log 2024-04-17T10:55:28.860 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.76305.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.76305.log.gz 2024-04-17T10:55:28.867 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88320.log 2024-04-17T10:55:28.868 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75992.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.75992.log.gz 2024-04-17T10:55:28.875 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78359.log 2024-04-17T10:55:28.875 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88320.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.88320.log.gz 2024-04-17T10:55:28.883 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.83273.log 2024-04-17T10:55:28.884 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.78359.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.78359.log.gz 2024-04-17T10:55:28.891 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36414.log 2024-04-17T10:55:28.899 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.83273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56183.log 2024-04-17T10:55:28.900 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36414.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36414.log.gz 2024-04-17T10:55:28.911 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37104.log 2024-04-17T10:55:28.911 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56183.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56183.log.gz 2024-04-17T10:55:28.919 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82631.log 2024-04-17T10:55:28.920 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37104.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37104.log.gz 2024-04-17T10:55:28.931 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76464.log 2024-04-17T10:55:28.932 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.82631.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.82631.log.gz 2024-04-17T10:55:28.939 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.34988.log 2024-04-17T10:55:28.940 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.76464.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76464.log.gz 2024-04-17T10:55:28.955 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78813.log 2024-04-17T10:55:28.956 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.2.34988.log: 91.0% -- replaced with /var/log/ceph/ceph-client.2.34988.log.gz 2024-04-17T10:55:28.971 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72650.log 2024-04-17T10:55:28.972 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.78813.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.78813.log.gz 2024-04-17T10:55:28.987 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87452.log 2024-04-17T10:55:28.988 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72650.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72650.log.gz 2024-04-17T10:55:29.003 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75543.log 2024-04-17T10:55:29.004 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87452.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.87452.log.gz 2024-04-17T10:55:29.019 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78461.log 2024-04-17T10:55:29.020 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75543.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75543.log.gz 2024-04-17T10:55:29.035 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51280.log 2024-04-17T10:55:29.036 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.78461.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78461.log.gz 2024-04-17T10:55:29.040 INFO:teuthology.orchestra.run.smithi167.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-17T10:55:29.043 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69996.log 2024-04-17T10:55:29.044 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51280.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51280.log.gz 2024-04-17T10:55:29.059 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50718.log 2024-04-17T10:55:29.060 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69996.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69996.log.gz 2024-04-17T10:55:29.075 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50673.log 2024-04-17T10:55:29.076 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50718.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50718.log.gz 2024-04-17T10:55:29.083 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35618.log 2024-04-17T10:55:29.084 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50673.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50673.log.gz 2024-04-17T10:55:29.091 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86248.log 2024-04-17T10:55:29.092 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35618.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35618.log.gz 2024-04-17T10:55:29.099 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85855.log 2024-04-17T10:55:29.100 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.86248.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.86248.log.gz 2024-04-17T10:55:29.107 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56370.log 2024-04-17T10:55:29.108 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85855.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.85855.log.gz 2024-04-17T10:55:29.115 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34088.log 2024-04-17T10:55:29.116 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56370.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56370.log.gz 2024-04-17T10:55:29.123 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37248.log 2024-04-17T10:55:29.124 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34088.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34088.log.gz 2024-04-17T10:55:29.131 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53011.log 2024-04-17T10:55:29.132 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37248.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37248.log.gz 2024-04-17T10:55:29.151 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88846.log 2024-04-17T10:55:29.152 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53011.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53011.log.gz 2024-04-17T10:55:29.159 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54701.log 2024-04-17T10:55:29.160 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88846.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.88846.log.gz 2024-04-17T10:55:29.171 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72475.log 2024-04-17T10:55:29.172 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54701.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54701.log.gz 2024-04-17T10:55:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50275.log 2024-04-17T10:55:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72475.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72475.log.gz 2024-04-17T10:55:29.187 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56502.log 2024-04-17T10:55:29.188 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50275.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50275.log.gz 2024-04-17T10:55:29.195 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79258.log 2024-04-17T10:55:29.196 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56502.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56502.log.gz 2024-04-17T10:55:29.196 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53227.log 2024-04-17T10:55:29.197 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.79258.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.79258.log.gz 2024-04-17T10:55:29.203 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50387.log 2024-04-17T10:55:29.215 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88249.log 2024-04-17T10:55:29.216 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50387.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50387.log.gz 2024-04-17T10:55:29.219 INFO:teuthology.orchestra.run.smithi083.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53227.log.gz 2024-04-17T10:55:29.224 INFO:teuthology.orchestra.run.smithi167.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-17T10:55:29.231 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76279.log 2024-04-17T10:55:29.232 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.88249.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.88249.log.gz 2024-04-17T10:55:29.243 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37326.log 2024-04-17T10:55:29.244 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.76279.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76279.log.gz 2024-04-17T10:55:29.255 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81614.log 2024-04-17T10:55:29.256 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37326.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37326.log.gz 2024-04-17T10:55:29.271 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75796.log 2024-04-17T10:55:29.272 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.81614.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.81614.log.gz 2024-04-17T10:55:29.287 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70535.log 2024-04-17T10:55:29.288 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.75796.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.75796.log.gz 2024-04-17T10:55:29.303 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85987.log 2024-04-17T10:55:29.319 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.70535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73238.log 2024-04-17T10:55:29.320 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85987.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.85987.log.gz 2024-04-17T10:55:29.327 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54379.log 2024-04-17T10:55:29.328 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.73238.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73238.log.gz 2024-04-17T10:55:29.343 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70404.log 2024-04-17T10:55:29.344 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54379.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54379.log.gz 2024-04-17T10:55:29.359 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53149.log 2024-04-17T10:55:29.360 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70404.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70404.log.gz 2024-04-17T10:55:29.362 INFO:teuthology.orchestra.run.smithi167.stderr: 94.2% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-17T10:55:29.375 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85437.log 2024-04-17T10:55:29.376 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53149.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53149.log.gz 2024-04-17T10:55:29.387 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34591.log 2024-04-17T10:55:29.388 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.85437.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.85437.log.gz 2024-04-17T10:55:29.403 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72449.log 2024-04-17T10:55:29.404 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34591.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34591.log.gz 2024-04-17T10:55:29.419 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35098.log 2024-04-17T10:55:29.420 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.72449.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72449.log.gz 2024-04-17T10:55:29.435 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87989.log 2024-04-17T10:55:29.436 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35098.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35098.log.gz 2024-04-17T10:55:29.452 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.87989.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.87989.log.gz 2024-04-17T10:55:29.715 INFO:teuthology.orchestra.run.smithi167.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-17T10:55:29.766 INFO:teuthology.orchestra.run.smithi083.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-17T10:55:29.900 INFO:teuthology.orchestra.run.smithi083.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-17T10:55:29.938 INFO:teuthology.orchestra.run.smithi083.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.0.83273.log.gz 2024-04-17T10:55:30.512 INFO:teuthology.orchestra.run.smithi083.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.0.70535.log.gz 2024-04-17T10:55:30.547 INFO:teuthology.orchestra.run.smithi083.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.0.51595.log.gz 2024-04-17T10:55:30.689 INFO:teuthology.orchestra.run.smithi083.stderr: 94.2% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-17T10:55:31.027 INFO:teuthology.orchestra.run.smithi083.stderr: 94.2% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-17T10:55:33.620 INFO:teuthology.orchestra.run.smithi083.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-17T10:55:33.934 INFO:teuthology.orchestra.run.smithi083.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.0.56841.log.gz 2024-04-17T10:55:34.451 INFO:teuthology.orchestra.run.smithi083.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.0.37888.log.gz 2024-04-17T10:55:46.586 INFO:teuthology.orchestra.run.smithi167.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-17T10:55:46.588 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-04-17T10:55:46.588 INFO:teuthology.orchestra.run.smithi167.stderr:real 0m19.658s 2024-04-17T10:55:46.588 INFO:teuthology.orchestra.run.smithi167.stderr:user 0m29.151s 2024-04-17T10:55:46.588 INFO:teuthology.orchestra.run.smithi167.stderr:sys 0m1.003s 2024-04-17T10:55:51.664 INFO:teuthology.orchestra.run.smithi083.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-17T10:55:51.666 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-04-17T10:55:51.666 INFO:teuthology.orchestra.run.smithi083.stderr:real 0m24.780s 2024-04-17T10:55:51.666 INFO:teuthology.orchestra.run.smithi083.stderr:user 0m58.725s 2024-04-17T10:55:51.666 INFO:teuthology.orchestra.run.smithi083.stderr:sys 0m2.099s 2024-04-17T10:55:51.667 INFO:tasks.ceph:Archiving logs... 2024-04-17T10:55:51.668 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-17_09:52:10-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7660174/remote/smithi083/log 2024-04-17T10:55:51.669 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-17T10:55:58.776 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-17_09:52:10-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7660174/remote/smithi167/log 2024-04-17T10:55:58.777 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-17T10:56:02.456 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-17T10:56:02.466 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-17T10:56:02.466 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-17T10:56:02.469 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-17T10:56:02.546 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-17T10:56:02.556 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-17T10:56:02.556 DEBUG:teuthology.orchestra.run.smithi083:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T10:56:02.558 DEBUG:teuthology.orchestra.run.smithi167:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T10:56:02.571 INFO:teuthology.orchestra.run.smithi083.stdout: remote refid st t when poll reach delay offset jitter 2024-04-17T10:56:02.571 INFO:teuthology.orchestra.run.smithi083.stdout:============================================================================== 2024-04-17T10:56:02.571 INFO:teuthology.orchestra.run.smithi083.stdout:+hv01.front.sepi 138.236.128.36 3 u 56 64 377 0.092 -5.910 2.604 2024-04-17T10:56:02.571 INFO:teuthology.orchestra.run.smithi083.stdout:+hv02.front.sepi 63.231.80.2 3 u 50 64 377 0.099 -5.748 1.262 2024-04-17T10:56:02.571 INFO:teuthology.orchestra.run.smithi083.stdout:*hv03.front.sepi 74.6.168.72 3 u 42 64 377 0.080 -5.565 1.886 2024-04-17T10:56:02.571 INFO:teuthology.orchestra.run.smithi083.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-04-17T10:56:02.573 INFO:teuthology.orchestra.run.smithi167.stdout: remote refid st t when poll reach delay offset jitter 2024-04-17T10:56:02.573 INFO:teuthology.orchestra.run.smithi167.stdout:============================================================================== 2024-04-17T10:56:02.573 INFO:teuthology.orchestra.run.smithi167.stdout:+hv01.front.sepi 138.236.128.36 3 u 46 64 377 0.059 -5.842 0.810 2024-04-17T10:56:02.574 INFO:teuthology.orchestra.run.smithi167.stdout:+hv02.front.sepi 63.231.80.2 3 u 54 64 377 0.100 -2.910 0.570 2024-04-17T10:56:02.574 INFO:teuthology.orchestra.run.smithi167.stdout:*hv03.front.sepi 74.6.168.72 3 u 47 64 377 0.104 -4.877 1.055 2024-04-17T10:56:02.574 INFO:teuthology.orchestra.run.smithi167.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-04-17T10:56:02.575 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-17T10:56:02.591 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-17T10:56:02.592 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-17T10:56:02.623 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-17T10:56:02.656 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-17T10:56:02.689 INFO:teuthology.task.internal:Duration was 1731.310727 seconds 2024-04-17T10:56:02.690 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-17T10:56:02.731 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-17T10:56:02.731 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-17T10:56:02.734 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-17T10:56:02.771 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-17T10:56:02.771 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi083.front.sepia.ceph.com 2024-04-17T10:56:02.772 DEBUG:teuthology.orchestra.run.smithi083:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-17T10:56:02.828 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi167.front.sepia.ceph.com 2024-04-17T10:56:02.828 DEBUG:teuthology.orchestra.run.smithi167:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-17T10:56:02.838 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-17T10:56:02.838 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-17T10:56:02.876 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-17T10:56:02.907 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-17T10:56:02.907 DEBUG:teuthology.orchestra.run.smithi083:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-17T10:56:02.953 DEBUG:teuthology.orchestra.run.smithi167:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-17T10:56:03.034 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-17T10:56:03.044 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-17T10:56:03.044 DEBUG:teuthology.orchestra.run.smithi083:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-17T10:56:03.055 DEBUG:teuthology.orchestra.run.smithi167:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-17T10:56:03.065 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-17T10:56:03.076 DEBUG:teuthology.orchestra.run.smithi083:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-17T10:56:03.101 DEBUG:teuthology.orchestra.run.smithi167:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-17T10:56:03.108 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = core 2024-04-17T10:56:03.116 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = core 2024-04-17T10:56:03.129 DEBUG:teuthology.orchestra.run.smithi083:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-17T10:56:03.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:56:03.167 DEBUG:teuthology.orchestra.run.smithi167:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-17T10:56:03.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:56:03.175 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-17T10:56:03.185 INFO:teuthology.task.internal:Transferring archived files... 2024-04-17T10:56:03.186 DEBUG:teuthology.misc:Transferring archived files from smithi083:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-17_09:52:10-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7660174/remote/smithi083 2024-04-17T10:56:03.186 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-17T10:56:03.234 DEBUG:teuthology.misc:Transferring archived files from smithi167:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-17_09:52:10-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7660174/remote/smithi167 2024-04-17T10:56:03.234 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-17T10:56:03.255 INFO:teuthology.task.internal:Removing archive directory... 2024-04-17T10:56:03.255 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-17T10:56:03.272 DEBUG:teuthology.orchestra.run.smithi167:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-17T10:56:03.299 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-17T10:56:03.311 INFO:teuthology.task.internal:Not uploading archives. 2024-04-17T10:56:03.311 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-17T10:56:03.341 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-17T10:56:03.341 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-17T10:56:03.344 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-17T10:56:03.348 INFO:teuthology.orchestra.run.smithi083.stdout: 658622 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 17 10:56 /home/ubuntu/cephtest 2024-04-17T10:56:03.349 INFO:teuthology.orchestra.run.smithi167.stdout: 658623 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 17 10:56 /home/ubuntu/cephtest 2024-04-17T10:56:03.350 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-17T10:56:03.375 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-17T10:56:03.441 INFO:teuthology.nuke:Checking targets against current locks 2024-04-17T10:56:03.467 DEBUG:teuthology.nuke:shortname: smithi083 2024-04-17T10:56:03.467 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-17T10:56:03.493 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-17_09:52:10-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7660174', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-17 10:17:26.101311', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHr9+q9Te3W+pbDiE+Htwdkau6E+Sf3z9uafMwd2gkpuElBki7FIY7+wmf/mUPwhJNOmtGTHAuXdIn30VmzrvHo='} 2024-04-17T10:56:03.496 DEBUG:teuthology.nuke:shortname: smithi167 2024-04-17T10:56:03.497 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-17T10:56:03.522 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-17_09:52:10-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7660174', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-17 10:17:26.100029', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBECZPeOI70yBWy+c5CdXEWaPmEAYKBn0CSit7jaHx6+l7Dc3NPw0uERZI1qUx7JfM4v9YWgerWoL4WcglL5TIyM='} 2024-04-17T10:56:03.554 INFO:teuthology.orchestra.console.smithi083:Power off 2024-04-17T10:56:03.554 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-17T10:56:03.572 INFO:teuthology.orchestra.console.smithi167:Power off 2024-04-17T10:56:03.573 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-17T10:56:03.582 DEBUG:teuthology.orchestra.console.smithi083:power off output: Chassis Power Control: Down/Off 2024-04-17T10:56:03.582 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T10:56:03.595 DEBUG:teuthology.orchestra.console.smithi167:power off output: Chassis Power Control: Down/Off 2024-04-17T10:56:03.596 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T10:56:03.611 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2024-04-17T10:56:03.622 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on 2024-04-17T10:56:07.613 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T10:56:07.623 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T10:56:07.724 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2024-04-17T10:56:07.731 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on 2024-04-17T10:56:11.726 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T10:56:11.734 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T10:56:11.837 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is off 2024-04-17T10:56:11.846 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is off 2024-04-17T10:56:11.939 INFO:teuthology.orchestra.console.smithi083:Power off completed 2024-04-17T10:56:11.948 INFO:teuthology.orchestra.console.smithi167:Power off completed 2024-04-17T10:56:12.053 INFO:teuthology.run:Summary data: description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/scrub} duration: 1731.3107266426086 failure_reason: 'Test failure: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls)' flavor: default owner: scheduled_neesingh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=b50cc263a42145e6848c416d20b4d678 status: fail success: false 2024-04-17T10:56:12.053 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-17T10:56:12.133 INFO:teuthology.run:FAIL